2024-03-06T07:07:32.769 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-06T07:07:32.770 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} --owner scheduled_leonidus@teuthology --name leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi --verbose --archive /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051 -- /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/orig.config.yaml 2024-03-06T07:07:32.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-06T07:07:32.902 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051 branch: wip-lusov-qdb description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} email: null first_in_suite: false job_id: '7583051' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - \(MDS_INSUFFICIENT_STANDBY\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(MDS_DAMAGE\) - \(FS_DEGRADED\) sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: disabled: true install: ceph: flavor: default sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 5072 sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd sleep_before_teardown: 0 subset: 246/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKlJGDxhBdHVX8Hp00522psyuWhMLXB/aG4jVBFKhKAsR3LKm9OW0V1VJaVoRTXpWDmAAhqsQM3BEqqQ+WD0sMA= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsQr8Ua6rXi+ytsg6qwdYXTQqxCgD5xaBlRGtibzO85/nLUHFG+vqU6l3Ffz0C3NZQMYg1E+6E0GitKVr3+MEE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - exec: mon.a: - ceph config set mgr mgr/crash/warn_recent_interval 0 - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_failover teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-06_07:00:27 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-06T07:07:32.902 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa; will attempt to use it 2024-03-06T07:07:32.903 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks 2024-03-06T07:07:32.903 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-06T07:07:32.904 INFO:teuthology.task.internal:Checking packages... 2024-03-06T07:07:32.930 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd' 2024-03-06T07:07:32.930 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-06T07:07:32.930 INFO:teuthology.packaging:ref: None 2024-03-06T07:07:32.930 INFO:teuthology.packaging:tag: None 2024-03-06T07:07:32.930 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-06T07:07:32.930 INFO:teuthology.packaging:sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:07:32.930 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-06T07:07:33.079 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1739.g1bc250c7 2024-03-06T07:07:33.080 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-06T07:07:33.088 INFO:teuthology.task.internal:no buildpackages task found 2024-03-06T07:07:33.088 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-06T07:07:33.130 INFO:teuthology.task.internal:Saving configuration 2024-03-06T07:07:33.146 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-06T07:07:33.163 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-06T07:07:33.187 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:56.388916', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKlJGDxhBdHVX8Hp00522psyuWhMLXB/aG4jVBFKhKAsR3LKm9OW0V1VJaVoRTXpWDmAAhqsQM3BEqqQ+WD0sMA='} 2024-03-06T07:07:33.212 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:56.390240', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsQr8Ua6rXi+ytsg6qwdYXTQqxCgD5xaBlRGtibzO85/nLUHFG+vqU6l3Ffz0C3NZQMYg1E+6E0GitKVr3+MEE='} 2024-03-06T07:07:33.212 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-06T07:07:33.218 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-06T07:07:33.218 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-03-06T07:07:33.218 INFO:teuthology.run_tasks:Running task console_log... 2024-03-06T07:07:33.305 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc1108430d0>, signals=[15]) 2024-03-06T07:07:33.306 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-06T07:07:33.312 INFO:teuthology.task.internal:Opening connections... 2024-03-06T07:07:33.313 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:07:33.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:07:33.431 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:07:33.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:07:33.527 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-06T07:07:33.535 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-03-06T07:07:33.553 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-03-06T07:07:33.554 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-03-06T07:07:33.611 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-03-06T07:07:33.611 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-06T07:07:33.612 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-06T07:07:33.613 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-03-06T07:07:33.634 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-03-06T07:07:33.653 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-03-06T07:07:33.653 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-03-06T07:07:33.709 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-03-06T07:07:33.709 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-03-06T07:07:33.710 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-06T07:07:33.711 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-03-06T07:07:33.711 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-06T07:07:33.711 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-06T07:07:33.711 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-06T07:07:33.711 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-03-06T07:07:33.734 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-06T07:07:33.744 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-06T07:07:33.789 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-06T07:07:33.789 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-03-06T07:07:33.792 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-03-06T07:07:33.808 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-06T07:07:33.824 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-06T07:07:33.824 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-03-06T07:07:33.849 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-03-06T07:07:33.941 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-06T07:07:33.995 INFO:teuthology.run_tasks:Running task kernel... 2024-03-06T07:07:34.015 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-06T07:07:34.015 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-03-06T07:07:34.015 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-06T07:07:34.016 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-06T07:07:34.016 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-06T07:07:34.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:07:34.033 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-06T07:07:34.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:07:34.034 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-03-06T07:07:34.090 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-361.el9.x86_64 2024-03-06T07:07:34.090 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.14.0-361.el9.x86_64 2024-03-06T07:07:34.090 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2024-03-06T07:07:34.092 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-361.el9.x86_64 2024-03-06T07:07:34.092 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.14.0-361.el9.x86_64 2024-03-06T07:07:34.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-03-06T07:07:36.212 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-06T07:07:36.504 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - AppStream 9.2 MB/s | 6.7 MB 00:00 2024-03-06T07:07:37.637 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-03-06T07:07:37.930 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-06T07:07:38.886 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-03-06T07:07:43.246 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 19 MB 00:03 2024-03-06T07:07:47.236 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 918 kB/s | 8.0 MB 00:08 2024-03-06T07:07:50.521 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 19 MB 00:02 2024-03-06T07:07:54.844 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 772 kB/s | 6.1 MB 00:08 2024-03-06T07:07:55.229 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.1 MB 00:01 2024-03-06T07:07:56.286 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 15 kB 00:00 2024-03-06T07:07:56.880 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2024-03-06T07:08:00.039 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 21 MB 00:03 2024-03-06T07:08:05.114 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-06T07:08:07.429 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-06T07:08:07.429 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-06T07:08:07.521 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-06T07:08:07.531 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-06T07:08:07.531 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-03-06T07:08:07.531 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-06T07:08:07.531 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-06T07:08:07.531 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-06T07:08:07.531 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout:Install 4 Packages 2024-03-06T07:08:07.532 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:08:07.533 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 100 M 2024-03-06T07:08:07.533 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 123 M 2024-03-06T07:08:07.533 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-06T07:08:15.518 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 20 MB 00:19 2024-03-06T07:08:15.981 INFO:teuthology.orchestra.run.smithi026.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 730 kB/s | 6.0 MB 00:08 2024-03-06T07:08:20.434 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-06T07:08:22.657 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-06T07:08:22.657 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-06T07:08:22.755 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-06T07:08:22.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:08:22.765 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-06T07:08:22.765 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-06T07:08:22.765 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-03-06T07:08:22.765 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:08:22.765 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 100 M 2024-03-06T07:08:22.765 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 123 M 2024-03-06T07:08:22.765 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-06T07:08:24.301 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 4.0 MB/s | 6.0 MB 00:01 2024-03-06T07:08:31.758 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 2.3 MB/s | 21 MB 00:08 2024-03-06T07:08:32.192 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 4.2 MB/s | 33 MB 00:07 2024-03-06T07:08:35.233 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 3.2 MB/s | 39 MB 00:12 2024-03-06T07:08:35.233 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:08:35.234 INFO:teuthology.orchestra.run.smithi117.stdout:Total 8.0 MB/s | 100 MB 00:12 2024-03-06T07:08:36.116 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-06T07:08:36.396 INFO:teuthology.orchestra.run.smithi026.stdout:(2/4): kernel-modules-core-5.14.0-427.el9.x86_6 1.6 MB/s | 33 MB 00:20 2024-03-06T07:08:36.489 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-06T07:08:36.489 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-06T07:08:36.962 INFO:teuthology.orchestra.run.smithi026.stdout:(3/4): kernel-core-5.14.0-427.el9.x86_64.rpm 726 kB/s | 21 MB 00:29 2024-03-06T07:08:38.144 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-06T07:08:38.144 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-06T07:08:41.108 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-06T07:08:41.782 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:08:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:08:44.678 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:08:44.852 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:08:49.581 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:08:49.788 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:08:54.523 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:02.293 INFO:teuthology.orchestra.run.smithi026.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 736 kB/s | 39 MB 00:54 2024-03-06T07:09:02.293 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:09:02.293 INFO:teuthology.orchestra.run.smithi026.stdout:Total 1.8 MB/s | 100 MB 00:54 2024-03-06T07:09:03.212 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-06T07:09:03.642 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-06T07:09:03.642 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-06T07:09:05.039 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-06T07:09:05.039 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-06T07:09:07.891 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-06T07:09:08.574 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:09:08.701 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:09:11.646 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:09:11.827 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:09:16.923 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:09:17.039 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:22.160 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:23.737 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:23.737 INFO:teuthology.orchestra.run.smithi117.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-06T07:09:23.738 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:09:23.820 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:09:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:09:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:09:24.635 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:09:24.636 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-06T07:09:25.130 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:25.273 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:25.273 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:25.273 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-06T07:09:25.273 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-06T07:09:25.273 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-06T07:09:25.273 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-06T07:09:25.273 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-03-06T07:09:25.274 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:25.393 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:25.393 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:25.393 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 68.el9 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8264254 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-03-06T07:09:25.427 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-06T07:09:25.428 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-03-06T07:09:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:09:25.429 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-03-06T07:09:25.429 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-06T07:09:25.429 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 5.14.0-427.el9.x86_64 2024-03-06T07:09:25.430 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-06T07:09:26.055 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-03-06T07:09:27.429 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-06T07:09:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-03-06T07:09:27.472 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-06T07:09:27.490 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.LHkXfROAoC 2024-03-06T07:09:27.490 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LHkXfROAoC 2024-03-06T07:09:27.558 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.LHkXfROAoC 2024-03-06T07:09:27.693 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.LHkXfROAoC is 6KB 2024-03-06T07:09:27.742 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.LHkXfROAoC 2024-03-06T07:09:27.756 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-03-06T07:09:27.826 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-06T07:09:27.826 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-06T07:09:27.826 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-06T07:09:27.827 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-06T07:09:27.828 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-06T07:09:28.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-03-06T07:09:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:52.798 INFO:teuthology.orchestra.run.smithi026.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-06T07:09:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:09:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:09:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:09:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:09:53.757 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-06T07:09:54.288 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:54.442 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:54.442 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:54.443 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-06T07:09:54.443 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-06T07:09:54.443 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-06T07:09:54.443 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-06T07:09:54.443 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=None, version=distro) 2024-03-06T07:09:54.443 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:54.569 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:54.569 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:54.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 68.el9 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8264254 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-06T07:09:54.613 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:09:54.614 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-03-06T07:09:54.615 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-06T07:09:54.615 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 5.14.0-427.el9.x86_64 2024-03-06T07:09:54.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-06T07:09:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-03-06T07:09:56.730 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-06T07:09:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-03-06T07:09:56.769 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-03-06T07:09:56.784 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.pBqmb3Rg3q 2024-03-06T07:09:56.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pBqmb3Rg3q 2024-03-06T07:09:56.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.pBqmb3Rg3q 2024-03-06T07:09:57.014 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.pBqmb3Rg3q is 6KB 2024-03-06T07:09:57.062 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.pBqmb3Rg3q 2024-03-06T07:09:57.077 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-03-06T07:09:57.145 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-06T07:09:57.145 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-06T07:09:57.145 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-06T07:09:57.145 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-06T07:09:57.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-06T07:09:57.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-03-06T07:09:58.009 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-06T07:09:58.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-06T07:09:58.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:16.380 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-06T07:10:25.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-06T07:10:25.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:27.296 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-06T07:10:27.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-06T07:10:27.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:28.448 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-06T07:10:40.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-06T07:10:40.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:45.824 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-03-06T07:10:54.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-06T07:10:54.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:58.880 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-06T07:11:13.216 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-03-06T07:11:13.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-06T07:11:13.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:11:14.173 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-06T07:11:14.614 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-06T07:11:14.614 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-427.el9.x86_64"... 2024-03-06T07:11:14.615 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-06T07:11:14.629 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-427.el9.x86_64 2024-03-06T07:11:14.629 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-06T07:11:14.629 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-06T07:11:14.629 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-06T07:11:15.630 DEBUG:teuthology.parallel:result is None 2024-03-06T07:11:25.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-06T07:11:25.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:11:25.633 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-03-06T07:11:26.189 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-06T07:11:26.189 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-06T07:11:26.189 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-03-06T07:11:26.203 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-427.el9.x86_64 2024-03-06T07:11:26.203 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-06T07:11:26.203 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-06T07:11:26.203 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-06T07:11:27.204 DEBUG:teuthology.parallel:result is None 2024-03-06T07:11:27.205 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-06T07:11:27.214 INFO:teuthology.task.internal:Creating test directory... 2024-03-06T07:11:27.214 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-06T07:11:27.217 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-06T07:11:27.237 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-06T07:11:27.243 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-06T07:11:27.277 INFO:teuthology.task.internal:Creating archive directory... 2024-03-06T07:11:27.277 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-06T07:11:27.279 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-06T07:11:27.307 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-06T07:11:27.318 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-06T07:11:27.318 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-06T07:11:27.335 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-06T07:11:27.399 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:27.423 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:27.442 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:27.464 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:27.466 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-06T07:11:27.477 INFO:teuthology.task.internal:Configuring sudo... 2024-03-06T07:11:27.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-06T07:11:27.479 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-06T07:11:27.511 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-06T07:11:27.521 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-06T07:11:27.521 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-06T07:11:27.547 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-06T07:11:27.565 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:27.646 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:27.687 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:27.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:27.810 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:11:27.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-06T07:11:27.879 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:27.922 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:27.976 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:28.061 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:28.107 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:11:28.107 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-06T07:11:28.179 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-03-06T07:11:28.181 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-03-06T07:11:28.238 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:11:28.274 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:11:28.667 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-06T07:11:28.677 INFO:teuthology.task.internal:Starting timer... 2024-03-06T07:11:28.677 INFO:teuthology.run_tasks:Running task pcp... 2024-03-06T07:11:28.705 INFO:teuthology.run_tasks:Running task selinux... 2024-03-06T07:11:28.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-03-06T07:11:28.807 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-03-06T07:11:28.809 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-03-06T07:11:28.861 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-03-06T07:11:28.863 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-06T07:11:28.863 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-03-06T07:11:28.886 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-03-06T07:11:28.886 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-03-06T07:11:28.907 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-03-06T07:11:28.908 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-03-06T07:11:28.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:11:28.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:11:28.954 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:11:28.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:11:28.983 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-06T07:11:28.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-03-06T07:11:29.029 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-03-06T07:11:29.066 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-06T07:11:29.076 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-06T07:11:29.093 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-06T07:11:29.118 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-06T07:15:46.871 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-03-06T07:15:46.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-06T07:15:46.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:15:46.977 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-03-06T07:15:47.055 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-06T07:15:47.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-06T07:15:47.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:15:47.152 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-06T07:15:47.231 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-06T07:15:47.231 INFO:teuthology.run_tasks:Running task clock... 2024-03-06T07:15:47.242 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-06T07:15:47.242 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-06T07:15:47.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:15:47.246 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-06T07:15:47.246 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:15:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-06T07:15:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-06T07:15:47.303 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-06T07:15:47.320 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-06T07:15:47.323 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-03-06T07:15:47.337 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-03-06T07:15:47.349 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-03-06T07:15:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-06T07:15:47.362 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-03-06T07:15:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-06T07:15:47.379 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-06T07:15:47.395 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-06T07:15:47.421 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:15:47.425 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:15:47.425 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-03-06T07:15:47.425 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.426 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.426 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.426 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.450 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:15:47.453 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:15:47.453 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-03-06T07:15:47.453 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.453 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.453 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.453 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:15:47.454 INFO:teuthology.run_tasks:Running task install... 2024-03-06T07:15:47.465 DEBUG:teuthology.task.install:project ceph 2024-03-06T07:15:47.465 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}} 2024-03-06T07:15:47.465 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'} 2024-03-06T07:15:47.465 INFO:teuthology.task.install:Using flavor: default 2024-03-06T07:15:47.472 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-06T07:15:47.472 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-06T07:15:47.473 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-06T07:15:47.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:15:47.475 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-06T07:15:47.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:15:47.627 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-06T07:15:47.627 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-06T07:15:47.697 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-06T07:15:47.698 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-06T07:15:47.922 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-06T07:15:47.923 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:15:47.923 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-06T07:15:47.953 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-06T07:15:47.953 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-06T07:15:48.025 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-06T07:15:48.044 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-06T07:15:48.045 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:15:48.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-06T07:15:48.078 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-06T07:15:48.078 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-06T07:15:48.107 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-06T07:15:48.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-06T07:15:48.175 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-03-06T07:15:48.176 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-03-06T07:15:48.234 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-06T07:15:48.303 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-03-06T07:15:48.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-03-06T07:15:48.421 INFO:teuthology.orchestra.run.smithi117.stdout:74 files removed 2024-03-06T07:15:48.448 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-06T07:15:48.552 INFO:teuthology.orchestra.run.smithi026.stdout:74 files removed 2024-03-06T07:15:48.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-06T07:15:49.708 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 88 kB/s | 83 kB 00:00 2024-03-06T07:15:49.911 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 82 kB/s | 83 kB 00:01 2024-03-06T07:15:50.450 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 14 kB/s | 10 kB 00:00 2024-03-06T07:15:50.778 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 12 kB/s | 10 kB 00:00 2024-03-06T07:15:51.268 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-03-06T07:15:51.679 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-03-06T07:15:51.861 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-06T07:15:52.114 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-06T07:15:53.403 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-06T07:15:53.623 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-06T07:15:55.110 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-03-06T07:16:01.485 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 19 MB 00:05 2024-03-06T07:16:03.629 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 851 kB/s | 8.0 MB 00:09 2024-03-06T07:16:06.294 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.1 MB 00:01 2024-03-06T07:16:07.421 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 19 MB 00:02 2024-03-06T07:16:07.853 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-03-06T07:16:12.598 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 21 MB 00:04 2024-03-06T07:16:17.603 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-06T07:16:20.086 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:20.087 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:20.101 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-06T07:16:20.102 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-06T07:16:20.103 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-06T07:16:20.104 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:20.106 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:20.109 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-06T07:16:20.110 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-06T07:16:20.191 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-06T07:16:20.210 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-06T07:16:20.211 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-06T07:16:20.212 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-06T07:16:20.213 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-06T07:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-06T07:16:20.215 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-06T07:16:20.216 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-06T07:16:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-06T07:16:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-06T07:16:20.219 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-06T07:16:20.220 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-06T07:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-06T07:16:20.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-06T07:16:20.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-06T07:16:20.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-06T07:16:20.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-06T07:16:20.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-06T07:16:20.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-06T07:16:20.224 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-06T07:16:20.224 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout:Install 150 Packages 2024-03-06T07:16:20.225 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 26 Packages 2024-03-06T07:16:20.226 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:16:20.226 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 224 M 2024-03-06T07:16:20.226 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-06T07:16:20.937 INFO:teuthology.orchestra.run.smithi117.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-03-06T07:16:21.262 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 613 kB/s | 6.1 MB 00:10 2024-03-06T07:16:21.512 INFO:teuthology.orchestra.run.smithi117.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 1.5 MB/s | 888 kB 00:00 2024-03-06T07:16:21.654 INFO:teuthology.orchestra.run.smithi117.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 1.0 MB/s | 145 kB 00:00 2024-03-06T07:16:22.971 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-03-06T07:16:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:(4/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 1.5 MB/s | 2.3 MB 00:01 2024-03-06T07:16:23.529 INFO:teuthology.orchestra.run.smithi117.stdout:(5/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 1.9 MB/s | 5.3 MB 00:02 2024-03-06T07:16:24.763 INFO:teuthology.orchestra.run.smithi117.stdout:(6/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 971 kB/s | 1.5 MB 00:01 2024-03-06T07:16:24.848 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-06T07:16:25.755 INFO:teuthology.orchestra.run.smithi117.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 2.1 MB/s | 4.7 MB 00:02 2024-03-06T07:16:29.981 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-03-06T07:16:31.462 INFO:teuthology.orchestra.run.smithi117.stdout:(8/176): ceph-common-19.0.0-1739.g1bc250c7.el9. 1.9 MB/s | 20 MB 00:10 2024-03-06T07:16:31.554 INFO:teuthology.orchestra.run.smithi117.stdout:(9/176): ceph-selinux-19.0.0-1739.g1bc250c7.el9 275 kB/s | 25 kB 00:00 2024-03-06T07:16:32.562 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:32.562 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:32.575 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-06T07:16:32.576 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-06T07:16:32.577 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-06T07:16:32.578 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:32.579 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:32.582 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-06T07:16:32.583 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-06T07:16:32.668 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-06T07:16:32.688 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================ 2024-03-06T07:16:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-06T07:16:32.688 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================ 2024-03-06T07:16:32.688 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-06T07:16:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-06T07:16:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-06T07:16:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-06T07:16:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-06T07:16:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-06T07:16:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-06T07:16:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-06T07:16:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-06T07:16:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-06T07:16:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-06T07:16:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-06T07:16:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-06T07:16:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-06T07:16:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-06T07:16:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-06T07:16:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================ 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout:Install 150 Packages 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 26 Packages 2024-03-06T07:16:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:16:32.705 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 224 M 2024-03-06T07:16:32.705 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-06T07:16:33.674 INFO:teuthology.orchestra.run.smithi026.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-03-06T07:16:36.012 INFO:teuthology.orchestra.run.smithi117.stdout:(10/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el 976 kB/s | 9.8 MB 00:10 2024-03-06T07:16:36.487 INFO:teuthology.orchestra.run.smithi117.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 464 kB/s | 220 kB 00:00 2024-03-06T07:16:36.595 INFO:teuthology.orchestra.run.smithi117.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 249 kB/s | 27 kB 00:00 2024-03-06T07:16:36.598 INFO:teuthology.orchestra.run.smithi026.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 304 kB/s | 888 kB 00:02 2024-03-06T07:16:36.696 INFO:teuthology.orchestra.run.smithi117.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 313 kB/s | 31 kB 00:00 2024-03-06T07:16:36.916 INFO:teuthology.orchestra.run.smithi026.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 458 kB/s | 145 kB 00:00 2024-03-06T07:16:38.062 INFO:teuthology.orchestra.run.smithi117.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 547 kB/s | 747 kB 00:01 2024-03-06T07:16:38.454 INFO:teuthology.orchestra.run.smithi117.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 417 kB/s | 163 kB 00:00 2024-03-06T07:16:38.763 INFO:teuthology.orchestra.run.smithi117.stdout:(16/176): librados-devel-19.0.0-1739.g1bc250c7. 411 kB/s | 127 kB 00:00 2024-03-06T07:16:39.888 INFO:teuthology.orchestra.run.smithi117.stdout:(17/176): libradosstriper1-19.0.0-1739.g1bc250c 446 kB/s | 501 kB 00:01 2024-03-06T07:16:41.124 INFO:teuthology.orchestra.run.smithi026.stdout:(4/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 562 kB/s | 2.3 MB 00:04 2024-03-06T07:16:44.474 INFO:teuthology.orchestra.run.smithi026.stdout:(5/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 457 kB/s | 1.5 MB 00:03 2024-03-06T07:16:44.966 INFO:teuthology.orchestra.run.smithi026.stdout:(6/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 460 kB/s | 5.3 MB 00:11 2024-03-06T07:16:48.645 INFO:teuthology.orchestra.run.smithi117.stdout:(18/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 721 kB/s | 17 MB 00:23 2024-03-06T07:16:48.745 INFO:teuthology.orchestra.run.smithi117.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 452 kB/s | 45 kB 00:00 2024-03-06T07:16:49.296 INFO:teuthology.orchestra.run.smithi117.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 239 kB/s | 131 kB 00:00 2024-03-06T07:16:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 524 kB/s | 162 kB 00:00 2024-03-06T07:16:50.155 INFO:teuthology.orchestra.run.smithi117.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 583 kB/s | 321 kB 00:00 2024-03-06T07:16:50.638 INFO:teuthology.orchestra.run.smithi117.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 616 kB/s | 298 kB 00:00 2024-03-06T07:16:50.988 INFO:teuthology.orchestra.run.smithi117.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 283 kB/s | 99 kB 00:00 2024-03-06T07:16:51.247 INFO:teuthology.orchestra.run.smithi117.stdout:(25/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 330 kB/s | 85 kB 00:00 2024-03-06T07:16:51.639 INFO:teuthology.orchestra.run.smithi117.stdout:(26/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 446 kB/s | 5.1 MB 00:11 2024-03-06T07:16:52.022 INFO:teuthology.orchestra.run.smithi117.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 445 kB/s | 171 kB 00:00 2024-03-06T07:16:52.148 INFO:teuthology.orchestra.run.smithi117.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 252 kB/s | 31 kB 00:00 2024-03-06T07:16:52.456 INFO:teuthology.orchestra.run.smithi117.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 448 kB/s | 138 kB 00:00 2024-03-06T07:16:53.457 INFO:teuthology.orchestra.run.smithi026.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 534 kB/s | 4.7 MB 00:08 2024-03-06T07:16:55.564 INFO:teuthology.orchestra.run.smithi117.stdout:(30/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 733 kB/s | 3.1 MB 00:04 2024-03-06T07:16:56.265 INFO:teuthology.orchestra.run.smithi117.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 479 kB/s | 1.8 MB 00:03 2024-03-06T07:16:57.631 INFO:teuthology.orchestra.run.smithi117.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 182 kB/s | 248 kB 00:01 2024-03-06T07:16:57.973 INFO:teuthology.orchestra.run.smithi117.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 144 kB/s | 49 kB 00:00 2024-03-06T07:16:58.131 INFO:teuthology.orchestra.run.smithi117.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 97 kB/s | 15 kB 00:00 2024-03-06T07:16:59.357 INFO:teuthology.orchestra.run.smithi117.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 215 kB/s | 264 kB 00:01 2024-03-06T07:17:02.098 INFO:teuthology.orchestra.run.smithi117.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 273 kB/s | 748 kB 00:02 2024-03-06T07:17:02.215 INFO:teuthology.orchestra.run.smithi117.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 157 kB/s | 18 kB 00:00 2024-03-06T07:17:02.365 INFO:teuthology.orchestra.run.smithi117.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 662 kB 00:00 2024-03-06T07:17:02.415 INFO:teuthology.orchestra.run.smithi117.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-03-06T07:17:02.440 INFO:teuthology.orchestra.run.smithi117.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 4.2 MB/s | 106 kB 00:00 2024-03-06T07:17:02.466 INFO:teuthology.orchestra.run.smithi117.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 816 kB/s | 20 kB 00:00 2024-03-06T07:17:02.508 INFO:teuthology.orchestra.run.smithi117.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-06T07:17:02.533 INFO:teuthology.orchestra.run.smithi117.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-03-06T07:17:02.625 INFO:teuthology.orchestra.run.smithi117.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-03-06T07:17:02.659 INFO:teuthology.orchestra.run.smithi117.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 518 kB/s | 17 kB 00:00 2024-03-06T07:17:02.692 INFO:teuthology.orchestra.run.smithi117.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-06T07:17:02.726 INFO:teuthology.orchestra.run.smithi117.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-03-06T07:17:02.760 INFO:teuthology.orchestra.run.smithi117.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-06T07:17:02.810 INFO:teuthology.orchestra.run.smithi117.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-06T07:17:02.852 INFO:teuthology.orchestra.run.smithi117.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-03-06T07:17:02.885 INFO:teuthology.orchestra.run.smithi117.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 736 kB/s | 24 kB 00:00 2024-03-06T07:17:02.919 INFO:teuthology.orchestra.run.smithi117.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-06T07:17:02.961 INFO:teuthology.orchestra.run.smithi117.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-06T07:17:02.995 INFO:teuthology.orchestra.run.smithi117.stdout:(54/176): lua-5.4.4-4.el9.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2024-03-06T07:17:03.028 INFO:teuthology.orchestra.run.smithi117.stdout:(55/176): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-06T07:17:03.145 INFO:teuthology.orchestra.run.smithi117.stdout:(56/176): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-06T07:17:03.245 INFO:teuthology.orchestra.run.smithi117.stdout:(57/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 30 MB/s | 3.0 MB 00:00 2024-03-06T07:17:03.304 INFO:teuthology.orchestra.run.smithi117.stdout:(58/176): postgresql-13.11-1.el9.x86_64.rpm 26 MB/s | 1.5 MB 00:00 2024-03-06T07:17:03.346 INFO:teuthology.orchestra.run.smithi117.stdout:(59/176): postgresql-contrib-13.11-1.el9.x86_64 19 MB/s | 811 kB 00:00 2024-03-06T07:17:03.379 INFO:teuthology.orchestra.run.smithi117.stdout:(60/176): postgresql-private-libs-13.11-1.el9.x 3.9 MB/s | 134 kB 00:00 2024-03-06T07:17:03.522 INFO:teuthology.orchestra.run.smithi117.stdout:(61/176): postgresql-server-13.11-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-03-06T07:17:03.580 INFO:teuthology.orchestra.run.smithi117.stdout:(62/176): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-03-06T07:17:03.613 INFO:teuthology.orchestra.run.smithi117.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-03-06T07:17:03.772 INFO:teuthology.orchestra.run.smithi117.stdout:(64/176): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-06T07:17:03.814 INFO:teuthology.orchestra.run.smithi117.stdout:(65/176): python3-jinja2-2.11.3-5.el9.noarch.rp 5.3 MB/s | 227 kB 00:00 2024-03-06T07:17:03.848 INFO:teuthology.orchestra.run.smithi117.stdout:(66/176): python3-jsonpatch-1.21-16.el9.noarch. 727 kB/s | 24 kB 00:00 2024-03-06T07:17:03.881 INFO:teuthology.orchestra.run.smithi117.stdout:(67/176): python3-jsonpointer-2.0-4.el9.noarch. 544 kB/s | 18 kB 00:00 2024-03-06T07:17:03.915 INFO:teuthology.orchestra.run.smithi117.stdout:(68/176): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2024-03-06T07:17:03.949 INFO:teuthology.orchestra.run.smithi117.stdout:(69/176): python3-markupsafe-1.1.1-12.el9.x86_6 941 kB/s | 32 kB 00:00 2024-03-06T07:17:04.074 INFO:teuthology.orchestra.run.smithi117.stdout:(70/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-06T07:17:04.124 INFO:teuthology.orchestra.run.smithi117.stdout:(71/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 217 kB 00:00 2024-03-06T07:17:04.158 INFO:teuthology.orchestra.run.smithi117.stdout:(72/176): python3-oauthlib-3.1.1-5.el9.noarch.r 5.1 MB/s | 174 kB 00:00 2024-03-06T07:17:04.192 INFO:teuthology.orchestra.run.smithi117.stdout:(73/176): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-03-06T07:17:04.217 INFO:teuthology.orchestra.run.smithi117.stdout:(74/176): python3-prettytable-0.7.2-27.el9.noar 1.6 MB/s | 41 kB 00:00 2024-03-06T07:17:04.259 INFO:teuthology.orchestra.run.smithi117.stdout:(75/176): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-06T07:17:04.293 INFO:teuthology.orchestra.run.smithi117.stdout:(76/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.9 MB/s | 132 kB 00:00 2024-03-06T07:17:04.326 INFO:teuthology.orchestra.run.smithi117.stdout:(77/176): python3-pyasn1-modules-0.4.8-6.el9.no 6.2 MB/s | 211 kB 00:00 2024-03-06T07:17:04.360 INFO:teuthology.orchestra.run.smithi117.stdout:(78/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-06T07:17:04.393 INFO:teuthology.orchestra.run.smithi117.stdout:(79/176): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-06T07:17:04.710 INFO:teuthology.orchestra.run.smithi117.stdout:(80/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 50 MB/s | 16 MB 00:00 2024-03-06T07:17:04.744 INFO:teuthology.orchestra.run.smithi117.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-06T07:17:04.777 INFO:teuthology.orchestra.run.smithi117.stdout:(82/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.2 MB/s | 41 kB 00:00 2024-03-06T07:17:04.819 INFO:teuthology.orchestra.run.smithi117.stdout:(83/176): qatlib-23.11.0-1.el9.x86_64.rpm 5.1 MB/s | 215 kB 00:00 2024-03-06T07:17:04.869 INFO:teuthology.orchestra.run.smithi117.stdout:(84/176): qatlib-service-23.11.0-1.el9.x86_64.r 675 kB/s | 34 kB 00:00 2024-03-06T07:17:04.903 INFO:teuthology.orchestra.run.smithi117.stdout:(85/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-03-06T07:17:04.945 INFO:teuthology.orchestra.run.smithi117.stdout:(86/176): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-06T07:17:04.987 INFO:teuthology.orchestra.run.smithi117.stdout:(87/176): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-06T07:17:05.021 INFO:teuthology.orchestra.run.smithi117.stdout:(88/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2024-03-06T07:17:05.056 INFO:teuthology.orchestra.run.smithi117.stdout:(89/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-06T07:17:05.088 INFO:teuthology.orchestra.run.smithi117.stdout:(90/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-06T07:17:05.146 INFO:teuthology.orchestra.run.smithi117.stdout:(91/176): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-06T07:17:05.188 INFO:teuthology.orchestra.run.smithi117.stdout:(92/176): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-03-06T07:17:05.230 INFO:teuthology.orchestra.run.smithi117.stdout:(93/176): libgfortran-11.4.1-3.el9.x86_64.rpm 19 MB/s | 802 kB 00:00 2024-03-06T07:17:05.273 INFO:teuthology.orchestra.run.smithi117.stdout:(94/176): libquadmath-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 194 kB 00:00 2024-03-06T07:17:05.314 INFO:teuthology.orchestra.run.smithi117.stdout:(95/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-03-06T07:17:05.356 INFO:teuthology.orchestra.run.smithi117.stdout:(96/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-06T07:17:05.415 INFO:teuthology.orchestra.run.smithi117.stdout:(97/176): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-06T07:17:05.499 INFO:teuthology.orchestra.run.smithi117.stdout:(98/176): ceph-mgr-diskprediction-local-19.0.0- 764 kB/s | 7.4 MB 00:09 2024-03-06T07:17:05.524 INFO:teuthology.orchestra.run.smithi117.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 947 kB/s | 103 kB 00:00 2024-03-06T07:17:05.599 INFO:teuthology.orchestra.run.smithi117.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 1.2 MB/s | 124 kB 00:00 2024-03-06T07:17:05.733 INFO:teuthology.orchestra.run.smithi117.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 542 kB/s | 113 kB 00:00 2024-03-06T07:17:05.758 INFO:teuthology.orchestra.run.smithi117.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 1.2 MB/s | 187 kB 00:00 2024-03-06T07:17:05.875 INFO:teuthology.orchestra.run.smithi117.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-03-06T07:17:06.109 INFO:teuthology.orchestra.run.smithi117.stdout:(104/176): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-03-06T07:17:06.342 INFO:teuthology.orchestra.run.smithi117.stdout:(105/176): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-03-06T07:17:06.392 INFO:teuthology.orchestra.run.smithi117.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 6.1 MB/s | 309 kB 00:00 2024-03-06T07:17:06.426 INFO:teuthology.orchestra.run.smithi117.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 585 kB/s | 19 kB 00:00 2024-03-06T07:17:06.626 INFO:teuthology.orchestra.run.smithi117.stdout:(108/176): libarrow-9.0.0-10.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-03-06T07:17:06.677 INFO:teuthology.orchestra.run.smithi117.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-03-06T07:17:06.727 INFO:teuthology.orchestra.run.smithi117.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 971 kB/s | 49 kB 00:00 2024-03-06T07:17:06.769 INFO:teuthology.orchestra.run.smithi117.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-03-06T07:17:06.819 INFO:teuthology.orchestra.run.smithi117.stdout:(112/176): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-03-06T07:17:06.878 INFO:teuthology.orchestra.run.smithi117.stdout:(113/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-03-06T07:17:06.928 INFO:teuthology.orchestra.run.smithi117.stdout:(114/176): python3-devel-3.9.17-2.el9.x86_64.rp 205 kB/s | 245 kB 00:01 2024-03-06T07:17:06.953 INFO:teuthology.orchestra.run.smithi117.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-03-06T07:17:06.979 INFO:teuthology.orchestra.run.smithi117.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-03-06T07:17:07.003 INFO:teuthology.orchestra.run.smithi117.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-03-06T07:17:07.029 INFO:teuthology.orchestra.run.smithi117.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-03-06T07:17:07.054 INFO:teuthology.orchestra.run.smithi117.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-03-06T07:17:07.079 INFO:teuthology.orchestra.run.smithi117.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 7.1 MB/s | 358 kB 00:00 2024-03-06T07:17:07.105 INFO:teuthology.orchestra.run.smithi117.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-03-06T07:17:07.130 INFO:teuthology.orchestra.run.smithi117.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 710 kB/s | 35 kB 00:00 2024-03-06T07:17:07.155 INFO:teuthology.orchestra.run.smithi117.stdout:(123/176): python3-google-auth-2.28.0-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-03-06T07:17:07.222 INFO:teuthology.orchestra.run.smithi117.stdout:(124/176): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-03-06T07:17:07.248 INFO:teuthology.orchestra.run.smithi117.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-03-06T07:17:07.273 INFO:teuthology.orchestra.run.smithi117.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-03-06T07:17:07.289 INFO:teuthology.orchestra.run.smithi026.stdout:(8/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el9 723 kB/s | 9.8 MB 00:13 2024-03-06T07:17:07.298 INFO:teuthology.orchestra.run.smithi117.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-03-06T07:17:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-03-06T07:17:07.349 INFO:teuthology.orchestra.run.smithi117.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-03-06T07:17:07.374 INFO:teuthology.orchestra.run.smithi117.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-03-06T07:17:07.380 INFO:teuthology.orchestra.run.smithi026.stdout:(9/176): ceph-selinux-19.0.0-1739.g1bc250c7.el9 275 kB/s | 25 kB 00:00 2024-03-06T07:17:07.408 INFO:teuthology.orchestra.run.smithi117.stdout:(131/176): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-03-06T07:17:07.425 INFO:teuthology.orchestra.run.smithi117.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 948 kB/s | 46 kB 00:00 2024-03-06T07:17:07.450 INFO:teuthology.orchestra.run.smithi117.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-03-06T07:17:07.467 INFO:teuthology.orchestra.run.smithi117.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-03-06T07:17:07.501 INFO:teuthology.orchestra.run.smithi117.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-06T07:17:07.518 INFO:teuthology.orchestra.run.smithi117.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-03-06T07:17:07.551 INFO:teuthology.orchestra.run.smithi117.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-06T07:17:07.568 INFO:teuthology.orchestra.run.smithi117.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-03-06T07:17:07.593 INFO:teuthology.orchestra.run.smithi117.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 739 kB/s | 31 kB 00:00 2024-03-06T07:17:07.619 INFO:teuthology.orchestra.run.smithi117.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-06T07:17:07.636 INFO:teuthology.orchestra.run.smithi117.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-06T07:17:07.661 INFO:teuthology.orchestra.run.smithi117.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 860 kB/s | 36 kB 00:00 2024-03-06T07:17:07.678 INFO:teuthology.orchestra.run.smithi117.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-03-06T07:17:07.711 INFO:teuthology.orchestra.run.smithi117.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-06T07:17:07.737 INFO:teuthology.orchestra.run.smithi117.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-06T07:17:07.762 INFO:teuthology.orchestra.run.smithi117.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 8.4 MB/s | 427 kB 00:00 2024-03-06T07:17:07.779 INFO:teuthology.orchestra.run.smithi117.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 488 kB/s | 20 kB 00:00 2024-03-06T07:17:07.813 INFO:teuthology.orchestra.run.smithi117.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-03-06T07:17:07.854 INFO:teuthology.orchestra.run.smithi117.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-03-06T07:17:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:(10/176): ceph-common-19.0.0-1739.g1bc250c7.el9 559 kB/s | 20 MB 00:37 2024-03-06T07:17:11.189 INFO:teuthology.orchestra.run.smithi026.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 323 kB/s | 220 kB 00:00 2024-03-06T07:17:11.314 INFO:teuthology.orchestra.run.smithi026.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 216 kB/s | 27 kB 00:00 2024-03-06T07:17:11.523 INFO:teuthology.orchestra.run.smithi026.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 150 kB/s | 31 kB 00:00 2024-03-06T07:17:12.871 INFO:teuthology.orchestra.run.smithi117.stdout:(150/176): librados2-19.0.0-1739.g1bc250c7.el9. 687 kB/s | 3.4 MB 00:05 2024-03-06T07:17:12.920 INFO:teuthology.orchestra.run.smithi117.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2024-03-06T07:17:13.513 INFO:teuthology.orchestra.run.smithi117.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 3.8 MB/s | 2.3 MB 00:00 2024-03-06T07:17:13.755 INFO:teuthology.orchestra.run.smithi117.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 7.5 MB/s | 1.8 MB 00:00 2024-03-06T07:17:13.781 INFO:teuthology.orchestra.run.smithi026.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 331 kB/s | 747 kB 00:02 2024-03-06T07:17:13.804 INFO:teuthology.orchestra.run.smithi117.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 830 kB/s | 41 kB 00:00 2024-03-06T07:17:13.855 INFO:teuthology.orchestra.run.smithi117.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-03-06T07:17:13.897 INFO:teuthology.orchestra.run.smithi117.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2024-03-06T07:17:13.930 INFO:teuthology.orchestra.run.smithi117.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 35 kB 00:00 2024-03-06T07:17:13.972 INFO:teuthology.orchestra.run.smithi117.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-03-06T07:17:14.023 INFO:teuthology.orchestra.run.smithi117.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-03-06T07:17:14.106 INFO:teuthology.orchestra.run.smithi117.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 9.4 MB/s | 802 kB 00:00 2024-03-06T07:17:14.157 INFO:teuthology.orchestra.run.smithi117.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 8.5 MB/s | 433 kB 00:00 2024-03-06T07:17:14.215 INFO:teuthology.orchestra.run.smithi117.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 377 kB/s | 22 kB 00:00 2024-03-06T07:17:14.292 INFO:teuthology.orchestra.run.smithi117.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 107 kB 00:00 2024-03-06T07:17:14.349 INFO:teuthology.orchestra.run.smithi117.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 154 kB 00:00 2024-03-06T07:17:14.414 INFO:teuthology.orchestra.run.smithi026.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 258 kB/s | 163 kB 00:00 2024-03-06T07:17:14.416 INFO:teuthology.orchestra.run.smithi117.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 135 kB 00:00 2024-03-06T07:17:14.450 INFO:teuthology.orchestra.run.smithi117.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.8 MB/s | 63 kB 00:00 2024-03-06T07:17:14.475 INFO:teuthology.orchestra.run.smithi117.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 28 kB 00:00 2024-03-06T07:17:14.567 INFO:teuthology.orchestra.run.smithi117.stdout:(168/176): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-06T07:17:14.726 INFO:teuthology.orchestra.run.smithi117.stdout:(169/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-06T07:17:14.851 INFO:teuthology.orchestra.run.smithi117.stdout:(170/176): util-linux-2.37.4-18.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-06T07:17:14.901 INFO:teuthology.orchestra.run.smithi117.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 8.6 MB/s | 438 kB 00:00 2024-03-06T07:17:14.908 INFO:teuthology.orchestra.run.smithi026.stdout:(16/176): librados-devel-19.0.0-1739.g1bc250c7. 258 kB/s | 127 kB 00:00 2024-03-06T07:17:14.935 INFO:teuthology.orchestra.run.smithi117.stdout:(172/176): util-linux-user-2.37.4-18.el9.x86_64 901 kB/s | 30 kB 00:00 2024-03-06T07:17:14.977 INFO:teuthology.orchestra.run.smithi117.stdout:(173/176): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-06T07:17:15.044 INFO:teuthology.orchestra.run.smithi117.stdout:(174/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-06T07:17:16.356 INFO:teuthology.orchestra.run.smithi026.stdout:(17/176): libradosstriper1-19.0.0-1739.g1bc250c 346 kB/s | 501 kB 00:01 2024-03-06T07:17:17.210 INFO:teuthology.orchestra.run.smithi117.stdout:(175/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 343 kB/s | 3.1 MB 00:09 2024-03-06T07:17:26.814 INFO:teuthology.orchestra.run.smithi026.stdout:(18/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 411 kB/s | 17 MB 00:41 2024-03-06T07:17:27.105 INFO:teuthology.orchestra.run.smithi026.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 155 kB/s | 45 kB 00:00 2024-03-06T07:17:27.697 INFO:teuthology.orchestra.run.smithi026.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 222 kB/s | 131 kB 00:00 2024-03-06T07:17:28.405 INFO:teuthology.orchestra.run.smithi026.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 228 kB/s | 162 kB 00:00 2024-03-06T07:17:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 238 kB/s | 321 kB 00:01 2024-03-06T07:17:30.723 INFO:teuthology.orchestra.run.smithi026.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 308 kB/s | 298 kB 00:00 2024-03-06T07:17:31.023 INFO:teuthology.orchestra.run.smithi026.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 331 kB/s | 99 kB 00:00 2024-03-06T07:17:31.373 INFO:teuthology.orchestra.run.smithi026.stdout:(25/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 349 kB/s | 5.1 MB 00:15 2024-03-06T07:17:31.398 INFO:teuthology.orchestra.run.smithi026.stdout:(26/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 227 kB/s | 85 kB 00:00 2024-03-06T07:17:31.890 INFO:teuthology.orchestra.run.smithi026.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 347 kB/s | 171 kB 00:00 2024-03-06T07:17:32.015 INFO:teuthology.orchestra.run.smithi026.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 251 kB/s | 31 kB 00:00 2024-03-06T07:17:32.407 INFO:teuthology.orchestra.run.smithi026.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 352 kB/s | 138 kB 00:00 2024-03-06T07:17:35.457 INFO:teuthology.orchestra.run.smithi117.stdout:(176/176): ceph-test-19.0.0-1739.g1bc250c7.el9. 759 kB/s | 47 MB 01:03 2024-03-06T07:17:35.461 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:17:35.462 INFO:teuthology.orchestra.run.smithi117.stdout:Total 3.0 MB/s | 224 MB 01:15 2024-03-06T07:17:35.500 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-06T07:17:35.621 INFO:teuthology.orchestra.run.smithi026.stdout:(30/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 568 kB/s | 1.8 MB 00:03 2024-03-06T07:17:35.808 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-06T07:17:35.808 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-06T07:17:37.766 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-06T07:17:37.766 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-06T07:17:38.076 INFO:teuthology.orchestra.run.smithi026.stdout:(31/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 472 kB/s | 3.1 MB 00:06 2024-03-06T07:17:38.634 INFO:teuthology.orchestra.run.smithi026.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 445 kB/s | 248 kB 00:00 2024-03-06T07:17:38.843 INFO:teuthology.orchestra.run.smithi026.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 237 kB/s | 49 kB 00:00 2024-03-06T07:17:38.968 INFO:teuthology.orchestra.run.smithi026.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 123 kB/s | 15 kB 00:00 2024-03-06T07:17:39.527 INFO:teuthology.orchestra.run.smithi026.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 472 kB/s | 264 kB 00:00 2024-03-06T07:17:40.952 INFO:teuthology.orchestra.run.smithi026.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 525 kB/s | 748 kB 00:01 2024-03-06T07:17:41.069 INFO:teuthology.orchestra.run.smithi026.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 156 kB/s | 18 kB 00:00 2024-03-06T07:17:41.335 INFO:teuthology.orchestra.run.smithi026.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-06T07:17:41.395 INFO:teuthology.orchestra.run.smithi026.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-06T07:17:41.419 INFO:teuthology.orchestra.run.smithi026.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 4.2 MB/s | 106 kB 00:00 2024-03-06T07:17:41.453 INFO:teuthology.orchestra.run.smithi026.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 611 kB/s | 20 kB 00:00 2024-03-06T07:17:41.495 INFO:teuthology.orchestra.run.smithi026.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-06T07:17:41.520 INFO:teuthology.orchestra.run.smithi026.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-03-06T07:17:41.621 INFO:teuthology.orchestra.run.smithi026.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-06T07:17:41.646 INFO:teuthology.orchestra.run.smithi026.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 687 kB/s | 17 kB 00:00 2024-03-06T07:17:41.671 INFO:teuthology.orchestra.run.smithi026.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 403 kB/s | 10 kB 00:00 2024-03-06T07:17:41.705 INFO:teuthology.orchestra.run.smithi026.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-03-06T07:17:41.730 INFO:teuthology.orchestra.run.smithi026.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-03-06T07:17:41.772 INFO:teuthology.orchestra.run.smithi026.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-06T07:17:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-03-06T07:17:41.848 INFO:teuthology.orchestra.run.smithi026.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 733 kB/s | 24 kB 00:00 2024-03-06T07:17:41.873 INFO:teuthology.orchestra.run.smithi026.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.8 MB/s | 46 kB 00:00 2024-03-06T07:17:41.923 INFO:teuthology.orchestra.run.smithi026.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-06T07:17:41.973 INFO:teuthology.orchestra.run.smithi026.stdout:(54/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-06T07:17:42.032 INFO:teuthology.orchestra.run.smithi026.stdout:(55/176): openblas-0.3.21-2.el9.x86_64.rpm 587 kB/s | 34 kB 00:00 2024-03-06T07:17:42.157 INFO:teuthology.orchestra.run.smithi026.stdout:(56/176): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-06T07:17:42.358 INFO:teuthology.orchestra.run.smithi026.stdout:(57/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-03-06T07:17:42.416 INFO:teuthology.orchestra.run.smithi026.stdout:(58/176): postgresql-13.11-1.el9.x86_64.rpm 26 MB/s | 1.5 MB 00:00 2024-03-06T07:17:42.467 INFO:teuthology.orchestra.run.smithi026.stdout:(59/176): postgresql-contrib-13.11-1.el9.x86_64 16 MB/s | 811 kB 00:00 2024-03-06T07:17:42.500 INFO:teuthology.orchestra.run.smithi026.stdout:(60/176): postgresql-private-libs-13.11-1.el9.x 3.9 MB/s | 134 kB 00:00 2024-03-06T07:17:42.634 INFO:teuthology.orchestra.run.smithi026.stdout:(61/176): postgresql-server-13.11-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-03-06T07:17:42.693 INFO:teuthology.orchestra.run.smithi026.stdout:(62/176): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-03-06T07:17:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-03-06T07:17:42.885 INFO:teuthology.orchestra.run.smithi026.stdout:(64/176): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 5.8 MB 00:00 2024-03-06T07:17:42.927 INFO:teuthology.orchestra.run.smithi026.stdout:(65/176): python3-jinja2-2.11.3-5.el9.noarch.rp 5.3 MB/s | 227 kB 00:00 2024-03-06T07:17:42.985 INFO:teuthology.orchestra.run.smithi026.stdout:(66/176): python3-jsonpatch-1.21-16.el9.noarch. 417 kB/s | 24 kB 00:00 2024-03-06T07:17:43.019 INFO:teuthology.orchestra.run.smithi026.stdout:(67/176): python3-jsonpointer-2.0-4.el9.noarch. 540 kB/s | 18 kB 00:00 2024-03-06T07:17:43.052 INFO:teuthology.orchestra.run.smithi026.stdout:(68/176): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2024-03-06T07:17:43.086 INFO:teuthology.orchestra.run.smithi026.stdout:(69/176): python3-markupsafe-1.1.1-12.el9.x86_6 942 kB/s | 32 kB 00:00 2024-03-06T07:17:43.229 INFO:teuthology.orchestra.run.smithi026.stdout:(70/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-06T07:17:43.262 INFO:teuthology.orchestra.run.smithi026.stdout:(71/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.4 MB/s | 217 kB 00:00 2024-03-06T07:17:43.320 INFO:teuthology.orchestra.run.smithi026.stdout:(72/176): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-03-06T07:17:43.354 INFO:teuthology.orchestra.run.smithi026.stdout:(73/176): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-03-06T07:17:43.388 INFO:teuthology.orchestra.run.smithi026.stdout:(74/176): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-03-06T07:17:43.430 INFO:teuthology.orchestra.run.smithi026.stdout:(75/176): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-06T07:17:43.471 INFO:teuthology.orchestra.run.smithi026.stdout:(76/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-03-06T07:17:43.513 INFO:teuthology.orchestra.run.smithi026.stdout:(77/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-03-06T07:17:43.547 INFO:teuthology.orchestra.run.smithi026.stdout:(78/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-06T07:17:43.581 INFO:teuthology.orchestra.run.smithi026.stdout:(79/176): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-06T07:17:44.056 INFO:teuthology.orchestra.run.smithi026.stdout:(80/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-03-06T07:17:44.089 INFO:teuthology.orchestra.run.smithi026.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-06T07:17:44.115 INFO:teuthology.orchestra.run.smithi026.stdout:(82/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.6 MB/s | 41 kB 00:00 2024-03-06T07:17:44.156 INFO:teuthology.orchestra.run.smithi026.stdout:(83/176): qatlib-23.11.0-1.el9.x86_64.rpm 5.1 MB/s | 215 kB 00:00 2024-03-06T07:17:44.182 INFO:teuthology.orchestra.run.smithi026.stdout:(84/176): qatlib-service-23.11.0-1.el9.x86_64.r 1.3 MB/s | 34 kB 00:00 2024-03-06T07:17:44.207 INFO:teuthology.orchestra.run.smithi026.stdout:(85/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-03-06T07:17:44.249 INFO:teuthology.orchestra.run.smithi026.stdout:(86/176): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-06T07:17:44.274 INFO:teuthology.orchestra.run.smithi026.stdout:(87/176): uuid-1.6.2-55.el9.x86_64.rpm 2.2 MB/s | 56 kB 00:00 2024-03-06T07:17:44.317 INFO:teuthology.orchestra.run.smithi026.stdout:(88/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-03-06T07:17:44.342 INFO:teuthology.orchestra.run.smithi026.stdout:(89/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 63 kB 00:00 2024-03-06T07:17:44.367 INFO:teuthology.orchestra.run.smithi026.stdout:(90/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-03-06T07:17:44.400 INFO:teuthology.orchestra.run.smithi026.stdout:(91/176): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-03-06T07:17:44.451 INFO:teuthology.orchestra.run.smithi026.stdout:(92/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-03-06T07:17:44.501 INFO:teuthology.orchestra.run.smithi026.stdout:(93/176): libgfortran-11.4.1-3.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-03-06T07:17:44.535 INFO:teuthology.orchestra.run.smithi026.stdout:(94/176): libquadmath-11.4.1-3.el9.x86_64.rpm 5.7 MB/s | 194 kB 00:00 2024-03-06T07:17:44.577 INFO:teuthology.orchestra.run.smithi026.stdout:(95/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-03-06T07:17:44.619 INFO:teuthology.orchestra.run.smithi026.stdout:(96/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-06T07:17:44.669 INFO:teuthology.orchestra.run.smithi026.stdout:(97/176): python3-cryptography-36.0.1-4.el9.x86 23 MB/s | 1.1 MB 00:00 2024-03-06T07:17:44.702 INFO:teuthology.orchestra.run.smithi026.stdout:(98/176): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-06T07:17:44.705 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-06T07:17:44.744 INFO:teuthology.orchestra.run.smithi026.stdout:(99/176): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-03-06T07:17:44.778 INFO:teuthology.orchestra.run.smithi026.stdout:(100/176): python3-requests-2.25.1-8.el9.noarch 3.3 MB/s | 113 kB 00:00 2024-03-06T07:17:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-06T07:17:44.813 INFO:teuthology.orchestra.run.smithi026.stdout:(101/176): python3-urllib3-1.26.5-5.el9.noarch. 5.5 MB/s | 187 kB 00:00 2024-03-06T07:17:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-06T07:17:44.858 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:17:44.884 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:17:44.945 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-06T07:17:44.954 INFO:teuthology.orchestra.run.smithi026.stdout:(102/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.7 MB/s | 245 kB 00:00 2024-03-06T07:17:44.983 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-06T07:17:45.037 INFO:teuthology.orchestra.run.smithi026.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-03-06T07:17:45.045 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-06T07:17:45.087 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-06T07:17:45.113 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-06T07:17:45.166 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-06T07:17:45.180 INFO:teuthology.orchestra.run.smithi026.stdout:(104/176): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-03-06T07:17:45.480 INFO:teuthology.orchestra.run.smithi026.stdout:(105/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-03-06T07:17:45.486 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-06T07:17:45.541 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:17:45.564 INFO:teuthology.orchestra.run.smithi026.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-03-06T07:17:45.622 INFO:teuthology.orchestra.run.smithi026.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-03-06T07:17:45.663 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:17:45.692 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:17:45.783 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:17:45.821 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:17:45.872 INFO:teuthology.orchestra.run.smithi026.stdout:(108/176): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-03-06T07:17:45.931 INFO:teuthology.orchestra.run.smithi026.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-03-06T07:17:45.983 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-06T07:17:45.989 INFO:teuthology.orchestra.run.smithi026.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-03-06T07:17:46.044 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-06T07:17:46.048 INFO:teuthology.orchestra.run.smithi026.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-06T07:17:46.072 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:17:46.107 INFO:teuthology.orchestra.run.smithi026.stdout:(112/176): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-03-06T07:17:46.176 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:17:46.182 INFO:teuthology.orchestra.run.smithi026.stdout:(113/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-03-06T07:17:46.198 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:17:46.249 INFO:teuthology.orchestra.run.smithi026.stdout:(114/176): python3-asyncssh-2.13.2-4.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-03-06T07:17:46.263 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:17:46.301 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-06T07:17:46.307 INFO:teuthology.orchestra.run.smithi026.stdout:(115/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 745 kB/s | 43 kB 00:00 2024-03-06T07:17:46.324 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:17:46.339 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:17:46.366 INFO:teuthology.orchestra.run.smithi026.stdout:(116/176): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-03-06T07:17:46.425 INFO:teuthology.orchestra.run.smithi026.stdout:(117/176): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-03-06T07:17:46.483 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:17:46.497 INFO:teuthology.orchestra.run.smithi026.stdout:(118/176): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-06T07:17:46.514 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:17:46.545 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-06T07:17:46.550 INFO:teuthology.orchestra.run.smithi026.stdout:(119/176): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-03-06T07:17:46.609 INFO:teuthology.orchestra.run.smithi026.stdout:(120/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-03-06T07:17:46.668 INFO:teuthology.orchestra.run.smithi026.stdout:(121/176): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-03-06T07:17:46.720 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-06T07:17:46.727 INFO:teuthology.orchestra.run.smithi026.stdout:(122/176): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-06T07:17:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:(123/176): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-03-06T07:17:46.841 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:17:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-06T07:17:46.908 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:17:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:(125/176): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-03-06T07:17:47.011 INFO:teuthology.orchestra.run.smithi026.stdout:(126/176): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-03-06T07:17:47.019 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:17:47.069 INFO:teuthology.orchestra.run.smithi026.stdout:(127/176): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-03-06T07:17:47.078 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:17:47.128 INFO:teuthology.orchestra.run.smithi026.stdout:(128/176): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-03-06T07:17:47.187 INFO:teuthology.orchestra.run.smithi026.stdout:(129/176): python3-jaraco-text-3.2.0-6.el9.noar 338 kB/s | 20 kB 00:00 2024-03-06T07:17:47.262 INFO:teuthology.orchestra.run.smithi026.stdout:(130/176): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-03-06T07:17:47.300 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:17:47.321 INFO:teuthology.orchestra.run.smithi026.stdout:(131/176): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-03-06T07:17:47.379 INFO:teuthology.orchestra.run.smithi026.stdout:(132/176): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-06T07:17:47.438 INFO:teuthology.orchestra.run.smithi026.stdout:(133/176): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-03-06T07:17:47.442 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-06T07:17:47.497 INFO:teuthology.orchestra.run.smithi026.stdout:(134/176): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-03-06T07:17:47.539 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-06T07:17:47.575 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-06T07:17:47.588 INFO:teuthology.orchestra.run.smithi026.stdout:(135/176): ceph-mgr-diskprediction-local-19.0.0 634 kB/s | 7.4 MB 00:11 2024-03-06T07:17:47.605 INFO:teuthology.orchestra.run.smithi026.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-06T07:17:47.645 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-06T07:17:47.656 INFO:teuthology.orchestra.run.smithi026.stdout:(137/176): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-03-06T07:17:47.698 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-06T07:17:47.706 INFO:teuthology.orchestra.run.smithi026.stdout:(138/176): python3-repoze-lru-0.7-16.el9.noarch 617 kB/s | 31 kB 00:00 2024-03-06T07:17:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-06T07:17:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-06T07:17:47.764 INFO:teuthology.orchestra.run.smithi026.stdout:(139/176): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-03-06T07:17:47.790 INFO:teuthology.orchestra.run.smithi026.stdout:(140/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 447 kB/s | 90 kB 00:00 2024-03-06T07:17:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-06T07:17:47.815 INFO:teuthology.orchestra.run.smithi026.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-06T07:17:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-06T07:17:47.874 INFO:teuthology.orchestra.run.smithi026.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 429 kB/s | 36 kB 00:00 2024-03-06T07:17:47.899 INFO:teuthology.orchestra.run.smithi026.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-03-06T07:17:47.902 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-06T07:17:47.946 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-06T07:17:47.957 INFO:teuthology.orchestra.run.smithi026.stdout:(144/176): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-06T07:17:47.979 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-06T07:17:47.999 INFO:teuthology.orchestra.run.smithi026.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-06T07:17:48.006 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-06T07:17:48.058 INFO:teuthology.orchestra.run.smithi026.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-06T07:17:48.077 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-06T07:17:48.092 INFO:teuthology.orchestra.run.smithi026.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-03-06T07:17:48.112 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-06T07:17:48.146 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-06T07:17:48.159 INFO:teuthology.orchestra.run.smithi026.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-06T07:17:48.166 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-06T07:17:48.237 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-06T07:17:48.289 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-06T07:17:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-06T07:17:48.384 INFO:teuthology.orchestra.run.smithi026.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-03-06T07:17:48.499 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-06T07:17:48.550 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-06T07:17:49.066 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-06T07:17:49.145 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-06T07:17:49.242 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-06T07:17:49.299 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:17:49.307 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:17:49.337 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-06T07:17:49.375 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-06T07:17:49.413 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-06T07:17:49.512 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-06T07:17:49.583 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-06T07:17:49.887 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-06T07:17:50.078 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-06T07:17:50.143 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-06T07:17:50.187 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-06T07:17:50.324 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-06T07:17:50.382 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-06T07:17:50.437 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-06T07:17:50.509 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-06T07:17:50.541 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-06T07:17:50.577 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-06T07:17:50.624 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-06T07:17:50.660 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-06T07:17:50.703 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-06T07:17:50.745 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-06T07:17:50.877 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-06T07:17:50.926 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-06T07:17:50.956 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-06T07:17:50.997 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-06T07:17:51.091 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-06T07:17:51.122 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-06T07:17:51.163 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-06T07:17:51.189 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-06T07:17:51.205 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-06T07:17:51.282 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-06T07:17:51.404 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-06T07:17:51.517 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-06T07:17:51.554 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-06T07:17:51.602 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-06T07:17:51.661 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-06T07:17:51.840 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-06T07:17:51.883 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-06T07:17:52.010 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-06T07:17:52.048 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-06T07:17:52.081 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-06T07:17:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-06T07:17:52.158 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-06T07:17:52.187 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-06T07:17:52.216 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-06T07:17:52.250 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-06T07:17:52.289 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-06T07:17:52.320 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-06T07:17:52.351 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-06T07:17:52.426 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-06T07:17:52.461 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-06T07:17:52.485 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-06T07:17:52.527 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-06T07:17:52.562 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-06T07:17:52.651 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-06T07:17:52.734 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-06T07:17:52.776 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-06T07:17:52.808 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-06T07:17:52.855 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-06T07:17:52.952 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-06T07:17:53.000 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-06T07:17:53.725 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-06T07:17:53.782 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-06T07:17:53.816 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-06T07:17:53.864 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-06T07:17:53.935 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-06T07:17:53.975 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-06T07:17:54.151 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-06T07:17:54.181 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-06T07:17:54.268 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-06T07:17:54.909 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-06T07:17:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-06T07:17:55.284 INFO:teuthology.orchestra.run.smithi026.stdout:(150/176): librados2-19.0.0-1739.g1bc250c7.el9. 488 kB/s | 3.4 MB 00:07 2024-03-06T07:17:55.325 INFO:teuthology.orchestra.run.smithi026.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-03-06T07:17:55.442 INFO:teuthology.orchestra.run.smithi026.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 19 MB/s | 2.3 MB 00:00 2024-03-06T07:17:55.543 INFO:teuthology.orchestra.run.smithi026.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-03-06T07:17:55.593 INFO:teuthology.orchestra.run.smithi026.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 827 kB/s | 41 kB 00:00 2024-03-06T07:17:55.651 INFO:teuthology.orchestra.run.smithi026.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 247 kB 00:00 2024-03-06T07:17:55.702 INFO:teuthology.orchestra.run.smithi026.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 69 kB 00:00 2024-03-06T07:17:55.769 INFO:teuthology.orchestra.run.smithi026.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 530 kB/s | 35 kB 00:00 2024-03-06T07:17:55.811 INFO:teuthology.orchestra.run.smithi026.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-03-06T07:17:55.861 INFO:teuthology.orchestra.run.smithi026.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-03-06T07:17:55.919 INFO:teuthology.orchestra.run.smithi026.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-03-06T07:17:55.970 INFO:teuthology.orchestra.run.smithi026.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 8.5 MB/s | 433 kB 00:00 2024-03-06T07:17:56.012 INFO:teuthology.orchestra.run.smithi026.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 528 kB/s | 22 kB 00:00 2024-03-06T07:17:56.054 INFO:teuthology.orchestra.run.smithi026.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 107 kB 00:00 2024-03-06T07:17:56.095 INFO:teuthology.orchestra.run.smithi026.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 154 kB 00:00 2024-03-06T07:17:56.138 INFO:teuthology.orchestra.run.smithi026.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 135 kB 00:00 2024-03-06T07:17:56.179 INFO:teuthology.orchestra.run.smithi026.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 63 kB 00:00 2024-03-06T07:17:56.221 INFO:teuthology.orchestra.run.smithi026.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 663 kB/s | 28 kB 00:00 2024-03-06T07:17:56.227 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-06T07:17:56.293 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-06T07:17:56.296 INFO:teuthology.orchestra.run.smithi026.stdout:(168/176): openssl-3.0.7-27.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-03-06T07:17:56.319 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-06T07:17:56.413 INFO:teuthology.orchestra.run.smithi026.stdout:(169/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-06T07:17:56.622 INFO:teuthology.orchestra.run.smithi026.stdout:(170/176): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-06T07:17:56.631 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-06T07:17:56.658 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:17:56.722 INFO:teuthology.orchestra.run.smithi026.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 4.3 MB/s | 438 kB 00:00 2024-03-06T07:17:56.747 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:17:56.779 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-06T07:17:56.789 INFO:teuthology.orchestra.run.smithi026.stdout:(172/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 382 kB/s | 3.1 MB 00:08 2024-03-06T07:17:56.807 INFO:teuthology.orchestra.run.smithi026.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 359 kB/s | 30 kB 00:00 2024-03-06T07:17:56.816 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-06T07:17:56.890 INFO:teuthology.orchestra.run.smithi026.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 3.1 MB/s | 321 kB 00:00 2024-03-06T07:17:57.008 INFO:teuthology.orchestra.run.smithi026.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2024-03-06T07:17:57.262 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-06T07:17:57.294 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:17:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:17:57.371 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-06T07:17:59.542 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:17:59.576 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:00.113 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:00.154 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:00.242 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:00.243 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-06T07:18:00.243 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-06T07:18:00.243 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:00.280 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:07.831 INFO:teuthology.orchestra.run.smithi026.stdout:(176/176): ceph-test-19.0.0-1739.g1bc250c7.el9. 803 kB/s | 47 MB 01:00 2024-03-06T07:18:07.835 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:18:07.836 INFO:teuthology.orchestra.run.smithi026.stdout:Total 2.3 MB/s | 224 MB 01:35 2024-03-06T07:18:07.881 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-06T07:18:08.216 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-06T07:18:08.216 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-06T07:18:10.301 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-06T07:18:10.302 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-06T07:18:11.746 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-03-06T07:18:11.747 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:11.968 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:11.983 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:11.983 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:11.983 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-06T07:18:11.983 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:13.463 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:13.488 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:13.488 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:13.488 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-06T07:18:13.488 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:13.488 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:13.488 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:13.713 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:13.736 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:13.736 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:13.736 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-06T07:18:13.736 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:13.736 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:13.736 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:14.172 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:14.194 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:14.194 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:14.194 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-06T07:18:14.194 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:14.194 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:14.194 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:14.291 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-06T07:18:14.659 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:14.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:14.721 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-06T07:18:14.781 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-06T07:18:14.804 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:15.033 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:15.063 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:16.029 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:16.063 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:16.164 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:16.326 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-06T07:18:16.357 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:16.380 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:16.380 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:16.380 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-06T07:18:16.380 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:16.380 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:16.380 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:16.424 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:16.435 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:16.484 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-06T07:18:17.347 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-06T07:18:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:17.398 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:17.398 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-06T07:18:17.399 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:17.399 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:17.399 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:17.444 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:17.465 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:17.465 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:17.465 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-06T07:18:17.465 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:17.511 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:17.518 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-06T07:18:17.532 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:17.532 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:17.532 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-06T07:18:17.532 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:17.532 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:17.532 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:17.618 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-06T07:18:17.653 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-06T07:18:17.729 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:18:17.759 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:18:17.812 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-06T07:18:17.853 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:17.863 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-06T07:18:17.875 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:17.875 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:17.875 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-06T07:18:17.875 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:17.876 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:17.876 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:17.922 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-06T07:18:17.974 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-06T07:18:17.999 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-06T07:18:18.043 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-06T07:18:18.391 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-06T07:18:18.517 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:18:18.625 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:18:18.651 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:18:18.727 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:18:18.765 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:18:18.921 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-06T07:18:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-06T07:18:19.005 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:18:19.128 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:18:19.156 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:18:19.221 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:18:19.260 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-06T07:18:19.283 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:18:19.297 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:18:19.450 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:18:19.481 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:18:19.513 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-06T07:18:19.805 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-06T07:18:19.874 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:18:19.934 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:18:20.052 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:20.331 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:20.444 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-06T07:18:20.541 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-06T07:18:20.573 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-06T07:18:20.644 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-06T07:18:20.697 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-06T07:18:20.727 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-06T07:18:20.762 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-06T07:18:20.803 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-06T07:18:20.902 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-06T07:18:20.935 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-06T07:18:20.978 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-06T07:18:21.011 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-06T07:18:21.039 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-06T07:18:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-06T07:18:21.144 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-06T07:18:21.196 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-06T07:18:21.224 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-06T07:18:21.295 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-06T07:18:21.339 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-06T07:18:21.414 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-06T07:18:21.551 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-06T07:18:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-06T07:18:22.163 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-06T07:18:22.238 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-06T07:18:22.318 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-06T07:18:22.341 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-06T07:18:22.356 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:18:22.363 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:18:22.391 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-06T07:18:22.404 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-06T07:18:22.421 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-06T07:18:22.441 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-06T07:18:22.448 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-06T07:18:22.481 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-06T07:18:22.575 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-06T07:18:22.637 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-06T07:18:22.658 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-06T07:18:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-06T07:18:22.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-06T07:18:22.843 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-06T07:18:22.896 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-06T07:18:22.913 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-06T07:18:22.991 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-06T07:18:23.031 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-06T07:18:23.077 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-06T07:18:23.083 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-06T07:18:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-06T07:18:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-06T07:18:23.167 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-06T07:18:23.212 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-06T07:18:23.227 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-06T07:18:23.293 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-06T07:18:23.342 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-06T07:18:23.353 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-06T07:18:23.398 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-06T07:18:23.454 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-06T07:18:23.529 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-06T07:18:23.565 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-06T07:18:23.604 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-06T07:18:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-06T07:18:23.687 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-06T07:18:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-06T07:18:23.772 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-06T07:18:23.913 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-06T07:18:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-06T07:18:23.992 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:24.011 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:24.024 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-06T07:18:24.066 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-06T07:18:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-06T07:18:24.199 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-06T07:18:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-06T07:18:24.265 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-06T07:18:24.290 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-06T07:18:24.369 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-06T07:18:24.495 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-06T07:18:24.609 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-06T07:18:24.648 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-06T07:18:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-06T07:18:24.778 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-06T07:18:24.820 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-06T07:18:24.895 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-06T07:18:24.945 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-06T07:18:24.977 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-06T07:18:25.028 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-06T07:18:25.034 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-06T07:18:25.074 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-06T07:18:25.099 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-06T07:18:25.124 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-06T07:18:25.151 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-06T07:18:25.151 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:18:25.171 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-06T07:18:25.217 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-06T07:18:25.244 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:18:25.249 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-06T07:18:25.271 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-06T07:18:25.289 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-06T07:18:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-06T07:18:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:18:25.325 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-06T07:18:25.355 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-06T07:18:25.376 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-06T07:18:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:18:25.409 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-06T07:18:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-06T07:18:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-06T07:18:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-06T07:18:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-06T07:18:25.480 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-06T07:18:25.510 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-06T07:18:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-06T07:18:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-06T07:18:25.588 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-06T07:18:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-06T07:18:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-06T07:18:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-06T07:18:25.654 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-06T07:18:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-06T07:18:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-06T07:18:25.695 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-06T07:18:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-06T07:18:25.729 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-06T07:18:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:18:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-06T07:18:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-06T07:18:25.912 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-06T07:18:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-06T07:18:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:18:25.992 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-06T07:18:26.040 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-06T07:18:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:18:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:18:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-06T07:18:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-06T07:18:26.147 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-06T07:18:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-06T07:18:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-06T07:18:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:18:26.972 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-06T07:18:27.024 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-06T07:18:27.058 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-06T07:18:27.106 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-06T07:18:27.196 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-06T07:18:27.233 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-06T07:18:27.408 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-06T07:18:27.439 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-06T07:18:27.519 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-06T07:18:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-06T07:18:28.316 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-06T07:18:29.593 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-06T07:18:29.668 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-06T07:18:29.694 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-06T07:18:30.022 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-06T07:18:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:18:30.141 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:18:30.171 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-06T07:18:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-06T07:18:30.707 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-06T07:18:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:18:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:18:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-06T07:18:31.192 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:18:31.192 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-06T07:18:31.192 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-06T07:18:31.192 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:18:31.193 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-06T07:18:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-06T07:18:31.211 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-06T07:18:31.211 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-06T07:18:31.211 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-06T07:18:31.212 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-06T07:18:31.213 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-06T07:18:31.214 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-06T07:18:31.215 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-06T07:18:31.216 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-06T07:18:31.217 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-06T07:18:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-06T07:18:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-06T07:18:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-06T07:18:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-06T07:18:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-06T07:18:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-06T07:18:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-06T07:18:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:18:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-06T07:18:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-06T07:18:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-06T07:18:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: acl-2.3.1-4.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-06T07:18:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: jq-1.6-16.el9.x86_64 2024-03-06T07:18:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-06T07:18:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.11-1.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-06T07:18:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-06T07:18:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-06T07:18:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-06T07:18:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-06T07:18:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-06T07:18:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:18:33.875 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-06T07:18:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:33.889 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-06T07:18:33.889 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-06T07:18:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:34.414 DEBUG:teuthology.parallel:result is None 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-03-06T07:18:46.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:46.309 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:46.334 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:46.334 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:46.334 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-06T07:18:46.334 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:47.862 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:47.886 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:47.886 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:47.887 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-06T07:18:47.887 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:47.887 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:48.112 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:48.134 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:48.135 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-06T07:18:48.135 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:48.135 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:48.597 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:48.619 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:48.619 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:48.619 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-06T07:18:48.619 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:48.619 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:48.619 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:48.715 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-06T07:18:49.193 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:49.222 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:49.254 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-06T07:18:49.308 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-06T07:18:49.336 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:49.576 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:50.621 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:50.654 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:50.759 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-06T07:18:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:50.987 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:50.987 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-06T07:18:50.987 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:50.987 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-06T07:18:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-06T07:18:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:52.064 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:52.064 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-06T07:18:52.064 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:52.064 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:52.130 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:52.130 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-06T07:18:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:52.215 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:52.215 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-06T07:18:52.215 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:52.215 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:52.535 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:52.557 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:52.557 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:52.557 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-06T07:18:52.557 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:52.557 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:52.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:18:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-06T07:18:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-06T07:18:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-06T07:18:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-06T07:18:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-06T07:18:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-06T07:18:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-06T07:18:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-06T07:18:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-06T07:18:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-06T07:18:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-06T07:18:57.974 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-06T07:18:58.004 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-06T07:18:58.034 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-06T07:18:58.100 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-06T07:18:58.153 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-06T07:18:58.190 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-06T07:18:58.847 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:58.888 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:58.907 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:59.773 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-06T07:18:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-06T07:18:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-06T07:18:59.974 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-06T07:19:00.012 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-06T07:19:00.037 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-06T07:19:00.065 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-06T07:19:00.097 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-06T07:19:00.097 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:19:00.181 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:19:00.209 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-06T07:19:00.240 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-06T07:19:00.240 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:19:00.325 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:19:00.351 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-06T07:19:00.384 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-06T07:19:00.409 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-06T07:19:00.456 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-06T07:19:00.485 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-06T07:19:00.524 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-06T07:19:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-06T07:19:00.598 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-06T07:19:00.624 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-06T07:19:00.649 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-06T07:19:00.674 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:19:00.699 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-06T07:19:00.705 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:19:00.705 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:19:00.886 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:19:00.886 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:19:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:19:00.992 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:19:01.019 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-06T07:19:01.044 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-06T07:19:01.070 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-06T07:19:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:19:05.763 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:19:05.763 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-06T07:19:05.763 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-06T07:19:05.763 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-06T07:19:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-06T07:19:05.765 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-06T07:19:05.782 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-06T07:19:05.782 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-06T07:19:05.782 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-06T07:19:05.782 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-06T07:19:05.782 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-06T07:19:05.782 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-06T07:19:05.783 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-06T07:19:05.784 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-06T07:19:05.785 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-06T07:19:05.786 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-06T07:19:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-06T07:19:05.788 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-06T07:19:05.791 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-06T07:19:05.792 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-06T07:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-06T07:19:05.794 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-06T07:19:05.795 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-06T07:19:05.796 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-06T07:19:05.797 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-06T07:19:08.455 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: acl-2.3.1-4.el9.x86_64 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-06T07:19:08.456 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:08.457 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.458 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.460 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-06T07:19:08.461 INFO:teuthology.orchestra.run.smithi026.stdout: jq-1.6-16.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-06T07:19:08.462 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-06T07:19:08.463 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-06T07:19:08.464 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.11-1.el9.x86_64 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-06T07:19:08.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-06T07:19:08.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-06T07:19:08.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-06T07:19:08.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-06T07:19:08.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-06T07:19:08.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.471 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-03-06T07:19:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-06T07:19:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-06T07:19:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-06T07:19:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-06T07:19:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-06T07:19:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-06T07:19:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:08.473 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-06T07:19:09.088 DEBUG:teuthology.parallel:result is None 2024-03-06T07:19:09.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:19:09.312 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-06T07:19:09.334 INFO:teuthology.orchestra.run.smithi026.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:09.334 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:09.335 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-06T07:19:09.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:19:09.480 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-06T07:19:09.505 INFO:teuthology.orchestra.run.smithi117.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:09.505 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:09.505 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-06T07:19:09.507 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-06T07:19:09.508 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:09.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-06T07:19:09.539 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:09.539 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-06T07:19:09.579 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-06T07:19:09.580 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:09.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-03-06T07:19:09.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-06T07:19:09.676 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:09.676 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-03-06T07:19:09.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-06T07:19:09.774 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-06T07:19:09.774 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:09.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-06T07:19:09.801 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-06T07:19:09.868 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:09.868 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-06T07:19:09.898 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-06T07:19:09.966 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-06T07:19:09.967 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:09.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-03-06T07:19:09.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-06T07:19:10.061 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:10.061 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-03-06T07:19:10.088 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-06T07:19:10.155 INFO:teuthology.run_tasks:Running task ceph... 2024-03-06T07:19:10.241 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-06T07:19:10.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2024-03-06T07:19:10.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2024-03-06T07:19:10.272 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-06T07:19:10.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-06T07:19:10.311 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-06T07:19:10.338 INFO:tasks.ceph:Creating extra log directories... 2024-03-06T07:19:10.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-06T07:19:10.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-06T07:19:10.411 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-06T07:19:10.411 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-06T07:19:10.411 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-06T07:19:10.411 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:10.412 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-06T07:19:10.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:10.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:10.598 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:10.598 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-06T07:19:10.612 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:10.731 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:10.763 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-06T07:19:10.763 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-06T07:19:10.763 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051', 'branch': 'wip-lusov-qdb', 'description': 'fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}', 'email': None, 'first_in_suite': False, 'job_id': '7583051', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'ceph-fuse': {'disabled': True}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 5072, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'sleep_before_teardown': 0, 'subset': '246/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKlJGDxhBdHVX8Hp00522psyuWhMLXB/aG4jVBFKhKAsR3LKm9OW0V1VJaVoRTXpWDmAAhqsQM3BEqqQ+WD0sMA=', 'smithi117.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsQr8Ua6rXi+ytsg6qwdYXTQqxCgD5xaBlRGtibzO85/nLUHFG+vqU6l3Ffz0C3NZQMYg1E+6E0GitKVr3+MEE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'cluster': 'ceph'}}, {'kclient': None}, {'exec': {'mon.a': ['ceph config set mgr mgr/crash/warn_recent_interval 0']}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_failover']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-06_07:00:27', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-06T07:19:10.764 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-06T07:19:10.766 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-06T07:19:10.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2024-03-06T07:19:10.821 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2024-03-06T07:19:10.879 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:10.879 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-03-06T07:19:10.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-06T07:19:10.900 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-06 07:19:01.461622479 +0000 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-06 07:15:23.020452625 +0000 2024-03-06T07:19:10.955 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-06 07:15:23.020452625 +0000 2024-03-06T07:19:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-06 07:15:23.020452625 +0000 2024-03-06T07:19:10.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-06T07:19:11.020 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-06T07:19:11.020 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-06T07:19:11.020 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 9.7899e-05 s, 5.2 MB/s 2024-03-06T07:19:11.021 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-06T07:19:11.077 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-06 07:19:01.461622479 +0000 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-06 07:15:23.386447754 +0000 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-06 07:15:23.386447754 +0000 2024-03-06T07:19:11.132 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-06 07:15:23.386447754 +0000 2024-03-06T07:19:11.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-06T07:19:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-06T07:19:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-06T07:19:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 7.9285e-05 s, 6.5 MB/s 2024-03-06T07:19:11.199 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-06T07:19:11.254 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-03-06T07:19:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-06 07:19:01.461622479 +0000 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-06 07:15:23.719443321 +0000 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-06 07:15:23.719443321 +0000 2024-03-06T07:19:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-06 07:15:23.719443321 +0000 2024-03-06T07:19:11.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-06T07:19:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-06T07:19:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-06T07:19:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 8.9049e-05 s, 5.7 MB/s 2024-03-06T07:19:11.376 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-06T07:19:11.431 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-06 07:19:01.462622466 +0000 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-06 07:15:24.055438849 +0000 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-06 07:15:24.055438849 +0000 2024-03-06T07:19:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-06 07:15:24.055438849 +0000 2024-03-06T07:19:11.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-06T07:19:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-06T07:19:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-06T07:19:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 8.017e-05 s, 6.4 MB/s 2024-03-06T07:19:11.552 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-06T07:19:11.607 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:11.608 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:11.608 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-03-06T07:19:11.624 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-06T07:19:11.624 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-06 07:18:26.551260123 +0000 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-06 07:15:20.455391726 +0000 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-06 07:15:20.455391726 +0000 2024-03-06T07:19:11.679 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-06 07:15:20.455391726 +0000 2024-03-06T07:19:11.680 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-06T07:19:11.744 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-06T07:19:11.745 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-06T07:19:11.745 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000209673 s, 2.4 MB/s 2024-03-06T07:19:11.746 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-06T07:19:11.801 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-03-06T07:19:11.856 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-06 07:18:26.551260123 +0000 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-06 07:15:20.797384952 +0000 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-06 07:15:20.797384952 +0000 2024-03-06T07:19:11.857 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-06 07:15:20.797384952 +0000 2024-03-06T07:19:11.857 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-06T07:19:11.922 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-06T07:19:11.922 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-06T07:19:11.922 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000193204 s, 2.7 MB/s 2024-03-06T07:19:11.923 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-06T07:19:11.979 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-03-06T07:19:12.034 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-06T07:19:12.034 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:12.034 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-06T07:19:12.034 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:12.034 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:12.034 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-06 07:18:26.551260123 +0000 2024-03-06T07:19:12.034 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-06 07:15:21.125378456 +0000 2024-03-06T07:19:12.035 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-06 07:15:21.125378456 +0000 2024-03-06T07:19:12.035 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-06 07:15:21.125378456 +0000 2024-03-06T07:19:12.035 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-06T07:19:12.100 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-06T07:19:12.100 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-06T07:19:12.100 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000207931 s, 2.5 MB/s 2024-03-06T07:19:12.101 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-06T07:19:12.157 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-06 07:18:26.551260123 +0000 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-06 07:15:21.471371604 +0000 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-06 07:15:21.471371604 +0000 2024-03-06T07:19:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-06 07:15:21.471371604 +0000 2024-03-06T07:19:12.214 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-06T07:19:12.279 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-06T07:19:12.280 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-06T07:19:12.280 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000163652 s, 3.1 MB/s 2024-03-06T07:19:12.281 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-06T07:19:12.336 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:12.336 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-06T07:19:12.336 INFO:tasks.ceph:Generating config... 2024-03-06T07:19:12.337 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-06T07:19:12.337 INFO:tasks.ceph:[client] debug client = 20 2024-03-06T07:19:12.337 INFO:tasks.ceph:[client] debug ms = 1 2024-03-06T07:19:12.338 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-06T07:19:12.338 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-06T07:19:12.338 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-06T07:19:12.338 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-06T07:19:12.339 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-06T07:19:12.339 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-06T07:19:12.339 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-06T07:19:12.339 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-06T07:19:12.339 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-06T07:19:12.339 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-06T07:19:12.339 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-06T07:19:12.339 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-06T07:19:12.339 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-03-06T07:19:12.339 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-06T07:19:12.339 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-06T07:19:12.339 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-06T07:19:12.339 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-06T07:19:12.339 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-06T07:19:12.340 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-06T07:19:12.340 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-06T07:19:12.340 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-06T07:19:12.340 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-06T07:19:12.340 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-06T07:19:12.340 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-06T07:19:12.340 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-06T07:19:12.340 INFO:tasks.ceph:Setting up mon.a... 2024-03-06T07:19:12.340 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:12.398 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.keyring 2024-03-06T07:19:12.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-06T07:19:12.446 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-06T07:19:12.473 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.26'), ('mon.b', '172.21.15.117'), ('mon.c', '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]')] 2024-03-06T07:19:12.473 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.26,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-06T07:19:12.474 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:12.474 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-06T07:19:12.529 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.26 --add b 172.21.15.117 --addv c '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:12.614 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:12.614 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: generated fsid 4c08c062-261c-4d4a-a3b3-e195823d80fd 2024-03-06T07:19:12.614 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = quincy 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 0 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 4c08c062-261c-4d4a-a3b3-e195823d80fd 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:last_changed 2024-03-06T07:19:12.613655+0000 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-03-06T07:19:12.613655+0000 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:min_mon_release 17 (quincy) 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:election_strategy: 1 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:2: [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] mon.c 2024-03-06T07:19:12.615 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-06T07:19:12.617 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-06T07:19:12.631 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4c08c062-261c-4d4a-a3b3-e195823d80fd... 2024-03-06T07:19:12.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-06T07:19:12.674 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2024-03-06T07:19:12.722 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2024-03-06T07:19:12.723 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.724 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "172.21.15.26,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on skipped message = False 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 4c08c062-261c-4d4a-a3b3-e195823d80fd 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 600 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2024-03-06T07:19:12.725 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug trim objects = true 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.726 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore compression mode = aggressive 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 20 2024-03-06T07:19:12.727 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 20 2024-03-06T07:19:12.728 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-03-06T07:19:12.728 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2024-03-06T07:19:12.728 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 10 2024-03-06T07:19:12.728 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2024-03-06T07:19:12.728 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2024-03-06T07:19:12.728 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2024-03-06T07:19:12.729 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2024-03-06T07:19:12.729 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2024-03-06T07:19:12.729 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-03-06T07:19:12.729 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.729 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2024-03-06T07:19:12.729 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-03-06T07:19:12.729 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2024-03-06T07:19:12.730 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-03-06T07:19:12.730 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-03-06T07:19:12.730 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:12.730 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:12.730 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2024-03-06T07:19:12.730 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.731 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2024-03-06T07:19:12.731 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-03-06T07:19:12.731 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-03-06T07:19:12.731 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2024-03-06T07:19:12.731 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-03-06T07:19:12.731 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2024-03-06T07:19:12.732 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2024-03-06T07:19:12.732 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:12.732 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2024-03-06T07:19:12.732 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2024-03-06T07:19:12.732 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:12.732 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.733 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-06T07:19:12.733 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2024-03-06T07:19:12.733 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2024-03-06T07:19:12.733 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.733 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2024-03-06T07:19:12.733 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-06T07:19:12.733 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-06T07:19:12.734 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.734 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2024-03-06T07:19:12.734 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2024-03-06T07:19:12.734 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.734 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2024-03-06T07:19:12.734 INFO:teuthology.orchestra.run.smithi026.stdout: mon op complaint time = 120 2024-03-06T07:19:12.735 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:12.735 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2024-03-06T07:19:12.735 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2024-03-06T07:19:12.735 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2024-03-06T07:19:12.735 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2024-03-06T07:19:12.735 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-06T07:19:12.736 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-06T07:19:12.736 INFO:teuthology.orchestra.run.smithi026.stdout: client mount timeout = 600 2024-03-06T07:19:12.736 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-03-06T07:19:12.736 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-03-06T07:19:12.736 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-03-06T07:19:12.736 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-03-06T07:19:12.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mds] 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds = 20 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds balancer = 20 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug frag = True 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug scatterstat = True 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: mds op complaint time = 180 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: mds verify scatter = True 2024-03-06T07:19:12.738 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-03-06T07:19:12.739 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-03-06T07:19:12.739 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-03-06T07:19:12.741 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-03-06T07:19:12.742 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.743 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.26,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on skipped message = False 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = 4c08c062-261c-4d4a-a3b3-e195823d80fd 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-03-06T07:19:12.744 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-03-06T07:19:12.745 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.746 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore compression mode = aggressive 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 20 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 20 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 10 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-03-06T07:19:12.747 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-03-06T07:19:12.748 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-06T07:19:12.749 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:12.750 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-06T07:19:12.751 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-03-06T07:19:12.752 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-03-06T07:19:12.753 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-03-06T07:19:12.754 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-03-06T07:19:12.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-03-06T07:19:12.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-03-06T07:19:12.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-03-06T07:19:12.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-03-06T07:19:12.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-03-06T07:19:12.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-03-06T07:19:12.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = True 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = True 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-03-06T07:19:12.756 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-03-06T07:19:12.757 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-03-06T07:19:12.757 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-06T07:19:12.757 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-06T07:19:12.837 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-06T07:19:12.837 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:12.837 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-06T07:19:12.853 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:12.853 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-06T07:19:12.908 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:19:12.908 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:12.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-06T07:19:12.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-06T07:19:12.987 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:12.987 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:13.042 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:19:13.042 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:13.042 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-06T07:19:13.042 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-06T07:19:13.081 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:13.081 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:13.136 INFO:tasks.ceph:Setting up mon nodes... 2024-03-06T07:19:13.136 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-06T07:19:13.137 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-06T07:19:13.190 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-06T07:19:13.193 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-06T07:19:13.255 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-06T07:19:13.258 INFO:tasks.ceph:Setting up mds nodes... 2024-03-06T07:19:13.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-06T07:19:13.313 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-06T07:19:13.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-06T07:19:13.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-06T07:19:13.436 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-06T07:19:13.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-06T07:19:13.465 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-06T07:19:13.519 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-06T07:19:13.523 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-06T07:19:13.549 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-06T07:19:13.644 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-06T07:19:13.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-06T07:19:13.673 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-06T07:19:13.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-06T07:19:13.715 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-06T07:19:13.731 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-06T07:19:13.773 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-06T07:19:13.791 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-06T07:19:13.791 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-06T07:19:13.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:13.819 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:13.819 INFO:tasks.ceph:role: osd.0 2024-03-06T07:19:13.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:19:13.819 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-06T07:19:13.885 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:13.885 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:13.886 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:13.917 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-03-06T07:19:13.919 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-03-06T07:19:13.919 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:13.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:14.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:14.128 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:14.128 INFO:tasks.ceph:role: osd.1 2024-03-06T07:19:14.128 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:19:14.129 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-06T07:19:14.194 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:14.194 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:14.194 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:14.194 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:14.194 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:14.195 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:14.195 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:14.195 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:14.195 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:14.195 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:14.228 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-03-06T07:19:14.230 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-03-06T07:19:14.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:14.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:14.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:14.438 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:14.438 INFO:tasks.ceph:role: osd.2 2024-03-06T07:19:14.438 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:19:14.438 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-06T07:19:14.504 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:14.504 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:14.505 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:14.536 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-03-06T07:19:14.538 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-03-06T07:19:14.538 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:14.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:14.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:14.745 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:14.745 INFO:tasks.ceph:role: osd.3 2024-03-06T07:19:14.745 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:19:14.745 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-06T07:19:14.811 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:14.811 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:14.811 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:14.812 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:14.812 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:14.812 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:14.812 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:14.812 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:14.812 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:14.812 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:14.844 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-03-06T07:19:14.846 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-03-06T07:19:14.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:14.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:14.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:15.079+0000 7fa140011640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-06T07:19:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:15.080+0000 7fa140011640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-06T07:19:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:15.080+0000 7fa140011640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-06T07:19:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:15.080+0000 7fa140011640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-06T07:19:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:15.080+0000 7fa140011640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-06T07:19:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:15.080+0000 7fa140011640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-06T07:19:17.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:17.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:18.067+0000 7fe33b7ac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-06T07:19:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:18.067+0000 7fe33b7ac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-06T07:19:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:18.067+0000 7fe33b7ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-06T07:19:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:18.067+0000 7fe33b7ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-06T07:19:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:18.067+0000 7fe33b7ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-06T07:19:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:18.067+0000 7fe33b7ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-06T07:19:20.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:20.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:21.049+0000 7fa24220e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-06T07:19:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:21.049+0000 7fa24220e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-06T07:19:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:21.049+0000 7fa24220e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-06T07:19:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:21.049+0000 7fa24220e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-06T07:19:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:21.049+0000 7fa24220e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-06T07:19:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:21.049+0000 7fa24220e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-06T07:19:23.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:23.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:24.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:24.016+0000 7f03fa80d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-06T07:19:24.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:24.016+0000 7f03fa80d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-06T07:19:24.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:24.016+0000 7f03fa80d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-06T07:19:24.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:24.016+0000 7f03fa80d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-06T07:19:24.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:24.016+0000 7f03fa80d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-06T07:19:24.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:24.017+0000 7f03fa80d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-06T07:19:26.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:26.891 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:26.923 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:26.923 INFO:tasks.ceph:role: osd.4 2024-03-06T07:19:26.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:19:26.923 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:27.002 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:27.505 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-06T07:19:27.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-06T07:19:27.507 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:27.629 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:27.667 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:27.737 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:27.737 INFO:tasks.ceph:role: osd.5 2024-03-06T07:19:27.737 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:19:27.737 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-06T07:19:27.813 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:27.813 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:27.813 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:27.814 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:27.814 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:27.814 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:27.814 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:27.814 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:27.814 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:27.814 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:28.275 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-06T07:19:28.278 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-06T07:19:28.278 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:28.371 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:28.404 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:28.473 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:28.473 INFO:tasks.ceph:role: osd.6 2024-03-06T07:19:28.473 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:19:28.473 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-06T07:19:28.547 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:28.547 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:28.547 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:28.548 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:28.548 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:28.548 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:28.548 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:28.548 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:28.548 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:28.548 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:29.051 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-06T07:19:29.053 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-06T07:19:29.054 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:29.197 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:29.231 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:29.300 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:29.301 INFO:tasks.ceph:role: osd.7 2024-03-06T07:19:29.301 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:19:29.301 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-06T07:19:29.376 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:29.376 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:29.376 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:29.376 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:29.377 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:29.377 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:29.377 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:29.377 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:29.377 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:29.377 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:29.885 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-06T07:19:29.887 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-06T07:19:29.887 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:30.027 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:30.063 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:30.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:30.162+0000 7fbb10412640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-06T07:19:30.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:30.162+0000 7fbb10412640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-06T07:19:30.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:30.162+0000 7fbb10412640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-06T07:19:30.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:30.162+0000 7fbb10412640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-06T07:19:30.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:30.162+0000 7fbb10412640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-06T07:19:30.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:30.162+0000 7fbb10412640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-06T07:19:33.020 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:33.051 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:33.140+0000 7f12a7349640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-06T07:19:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:33.140+0000 7f12a7349640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-06T07:19:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:33.140+0000 7f12a7349640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-06T07:19:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:33.140+0000 7f12a7349640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-06T07:19:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:33.140+0000 7f12a7349640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-06T07:19:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:33.141+0000 7f12a7349640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-06T07:19:36.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:36.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:36.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:36.126+0000 7f9596f49640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-06T07:19:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:36.126+0000 7f9596f49640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-06T07:19:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:36.126+0000 7f9596f49640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-06T07:19:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:36.126+0000 7f9596f49640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-06T07:19:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:36.126+0000 7f9596f49640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-06T07:19:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:36.127+0000 7f9596f49640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-06T07:19:38.973 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:39.005 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:39.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:39.093+0000 7fa175008640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-06T07:19:39.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:39.093+0000 7fa175008640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-06T07:19:39.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:39.093+0000 7fa175008640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-06T07:19:39.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:39.093+0000 7fa175008640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-06T07:19:39.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:39.093+0000 7fa175008640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-06T07:19:39.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:39.094+0000 7fa175008640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-06T07:19:40.765 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:40.768 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:41.963 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:41.995 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-06T07:19:41.995 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:41.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-06T07:19:42.027 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:42.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-06T07:19:42.098 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:42.098 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-06T07:19:42.168 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:42.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-06T07:19:42.237 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:42.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-06T07:19:42.306 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:42.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-06T07:19:42.375 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:42.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-06T07:19:42.444 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.444 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-06T07:19:42.476 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.476 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-06T07:19:42.546 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.548 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-06T07:19:42.617 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.618 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-06T07:19:42.688 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.688 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-06T07:19:42.759 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.759 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-06T07:19:42.827 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-06T07:19:42.896 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:42.896 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-06T07:19:42.917 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:42.917 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-06T07:19:42.952 INFO:tasks.ceph:Adding keys to all mons... 2024-03-06T07:19:42.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-06T07:19:42.960 DEBUG:teuthology.orchestra.run.smithi117:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.y] 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA== 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw== 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ== 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2024-03-06T07:19:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ== 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ== 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw== 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA== 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg== 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q== 2024-03-06T07:19:42.996 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg== 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q== 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA== 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.6] 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g== 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.7] 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w== 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout:[client.1] 2024-03-06T07:19:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg== 2024-03-06T07:19:43.018 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.y] 2024-03-06T07:19:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA== 2024-03-06T07:19:43.019 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-03-06T07:19:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw== 2024-03-06T07:19:43.019 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-03-06T07:19:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ== 2024-03-06T07:19:43.019 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2024-03-06T07:19:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ== 2024-03-06T07:19:43.020 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2024-03-06T07:19:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ== 2024-03-06T07:19:43.020 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2024-03-06T07:19:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw== 2024-03-06T07:19:43.021 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.3] 2024-03-06T07:19:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA== 2024-03-06T07:19:43.021 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.x] 2024-03-06T07:19:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg== 2024-03-06T07:19:43.021 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-03-06T07:19:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q== 2024-03-06T07:19:43.022 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-03-06T07:19:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg== 2024-03-06T07:19:43.022 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.4] 2024-03-06T07:19:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q== 2024-03-06T07:19:43.022 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.5] 2024-03-06T07:19:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA== 2024-03-06T07:19:43.023 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.6] 2024-03-06T07:19:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g== 2024-03-06T07:19:43.023 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.7] 2024-03-06T07:19:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w== 2024-03-06T07:19:43.023 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-03-06T07:19:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:19:43.024 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2024-03-06T07:19:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg== 2024-03-06T07:19:43.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:43.041 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:43.121 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.180 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.182 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.240 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.293 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.347 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.401 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:43.455 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:43.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.508 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.565 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:43.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.623 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.676 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.678 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.733 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:43.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:43.842 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:43.897 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:43.899 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:43.954 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-06T07:19:43.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-06T07:19:43.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:44.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-06T07:19:44.103 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-06T07:19:44.129 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:44.226 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-06T07:19:44.253 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-06T07:19:44.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:44.415 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-06T07:19:44.442 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:44.444 DEBUG:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:44.496 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-06T07:19:44.496 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-06T07:19:44.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-06T07:19:44.499 INFO:tasks.ceph.mon.a:Started 2024-03-06T07:19:44.499 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-06T07:19:44.499 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-06T07:19:44.539 INFO:tasks.ceph.mon.b:Started 2024-03-06T07:19:44.539 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-06T07:19:44.539 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-06T07:19:44.541 INFO:tasks.ceph.mon.c:Started 2024-03-06T07:19:44.541 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-06T07:19:44.541 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-06T07:19:44.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-06T07:19:44.544 INFO:tasks.ceph.mgr.y:Started 2024-03-06T07:19:44.544 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-06T07:19:44.544 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-06T07:19:44.546 INFO:tasks.ceph.mgr.x:Started 2024-03-06T07:19:44.546 INFO:tasks.ceph:Setting crush tunables to default 2024-03-06T07:19:44.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd crush tunables default 2024-03-06T07:19:44.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.646+0000 7f4500d6b640 1 Processor -- start 2024-03-06T07:19:44.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.646+0000 7f4500d6b640 1 -- start start 2024-03-06T07:19:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f4500d6b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 0x7f44fc0c4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f4500d6b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 0x7f44fc1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f4500d6b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 0x7f44fc158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f4500d6b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc0c2cf0 con 0x7f44fc0c47d0 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f4500d6b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc0c4050 con 0x7f44fc06f120 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f4500d6b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0c1910 con 0x7f44fc152ab0 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f44fad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 0x7f44fc158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f44fad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 0x7f44fc158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37956/0 (socket says 172.21.15.26:37956) 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f44fad9d640 1 -- 172.21.15.26:0/407478830 learned_addr learned my addr 172.21.15.26:0/407478830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.647+0000 7f44fa59c640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 0x7f44fc0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fad9d640 1 -- 172.21.15.26:0/407478830 >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 msgr2=0x7f44fc158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fad9d640 1 -- 172.21.15.26:0/407478830 >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 msgr2=0x7f44fc158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fad9d640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 0x7f44fc158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44f9d9b640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 0x7f44fc1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fad9d640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 0x7f44fc158e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fa59c640 1 -- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 msgr2=0x7f44fc0c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fa59c640 1 -- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 msgr2=0x7f44fc0c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fa59c640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 0x7f44fc0c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44fa59c640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 0x7f44fc0c4290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44f9d9b640 1 -- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 msgr2=0x7f44fc1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44f9d9b640 1 -- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 msgr2=0x7f44fc1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44f9d9b640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 0x7f44fc1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:44.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.648+0000 7f44f9d9b640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 0x7f44fc1588c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44fad9d640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.26:3300/0 conn(0x7f44fc0c47d0 0x7f44fc158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44f9d9b640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 0x7f44fc1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44fa59c640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 0x7f44fc0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44fad9d640 1 -- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 msgr2=0x7f44fc0c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44fad9d640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 0x7f44fc0c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44fad9d640 1 -- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 msgr2=0x7f44fc1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44fad9d640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 0x7f44fc1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.849+0000 7f44fad9d640 1 -- 172.21.15.26:0/407478830 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc0c1bf0 con 0x7f44fc0c47d0 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.850+0000 7f44fad9d640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc158e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009b60 tx=0x7f44f402bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7237fda1ab24c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.850+0000 7f44f959a640 1 -- 172.21.15.26:0/407478830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4032070 con 0x7f44fc0c47d0 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.850+0000 7f44f959a640 1 -- 172.21.15.26:0/407478830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f402cd70 con 0x7f44fc0c47d0 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.850+0000 7f44f959a640 1 -- 172.21.15.26:0/407478830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4036640 con 0x7f44fc0c47d0 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 -- 172.21.15.26:0/407478830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 msgr2=0x7f44fc158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc158e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009b60 tx=0x7f44f402bfc0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 -- 172.21.15.26:0/407478830 shutdown_connections 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc158e00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 --2- 172.21.15.26:0/407478830 >> v2:172.21.15.117:3300/0 conn(0x7f44fc06f120 0x7f44fc1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 --2- 172.21.15.26:0/407478830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc152ab0 0x7f44fc0c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 -- 172.21.15.26:0/407478830 >> 172.21.15.26:0/407478830 conn(0x7f44fc0c7800 msgr2=0x7f44fc0c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:44.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 -- 172.21.15.26:0/407478830 shutdown_connections 2024-03-06T07:19:44.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.851+0000 7f4500d6b640 1 -- 172.21.15.26:0/407478830 wait complete. 2024-03-06T07:19:44.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.852+0000 7f4500d6b640 1 Processor -- start 2024-03-06T07:19:44.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f4500d6b640 1 -- start start 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f4500d6b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc06f120 0x7f44fc06ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f4500d6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc14e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f4500d6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc152ab0 0x7f44fc14eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f4500d6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0c1380 con 0x7f44fc0c47d0 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f4500d6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0c6f60 con 0x7f44fc152ab0 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f4500d6b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0c1500 con 0x7f44fc06f120 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f44f9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc14e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f44f9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc14e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37992/0 (socket says 172.21.15.26:37992) 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f44f9d9b640 1 -- 172.21.15.26:0/2878847685 learned_addr learned my addr 172.21.15.26:0/2878847685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f44fa59c640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc06f120 0x7f44fc06ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.853+0000 7f44fad9d640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc152ab0 0x7f44fc14eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44f9d9b640 1 -- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc06f120 msgr2=0x7f44fc06ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44f9d9b640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc06f120 0x7f44fc06ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44f9d9b640 1 -- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc152ab0 msgr2=0x7f44fc14eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44f9d9b640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc152ab0 0x7f44fc14eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44f9d9b640 1 -- 172.21.15.26:0/2878847685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc1549c0 con 0x7f44fc0c47d0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44f9d9b640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc14e5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00b6b0 tx=0x7f44ec00bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44eaffd640 1 -- 172.21.15.26:0/2878847685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec018070 con 0x7f44fc0c47d0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44eaffd640 1 -- 172.21.15.26:0/2878847685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec010830 con 0x7f44fc0c47d0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc1ee500 con 0x7f44fc0c47d0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44eaffd640 1 -- 172.21.15.26:0/2878847685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec010c20 con 0x7f44fc0c47d0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc1eeac0 con 0x7f44fc0c47d0 2024-03-06T07:19:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44eaffd640 1 -- 172.21.15.26:0/2878847685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f44ec01b070 con 0x7f44fc0c47d0 2024-03-06T07:19:44.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.854+0000 7f44eaffd640 1 -- 172.21.15.26:0/2878847685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f44ec014770 con 0x7f44fc0c47d0 2024-03-06T07:19:44.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.855+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc069460 con 0x7f44fc0c47d0 2024-03-06T07:19:44.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.857+0000 7f44eaffd640 1 -- 172.21.15.26:0/2878847685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f44fc069460 con 0x7f44fc0c47d0 2024-03-06T07:19:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.924+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f44fc04f330 con 0x7f44fc0c47d0 2024-03-06T07:19:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.925+0000 7f44eaffd640 1 -- 172.21.15.26:0/2878847685 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f44ec0149c0 con 0x7f44fc0c47d0 2024-03-06T07:19:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-03-06T07:19:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 msgr2=0x7f44fc14e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc14e5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00b6b0 tx=0x7f44ec00bb70 comp rx=0 tx=0).stop 2024-03-06T07:19:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 shutdown_connections 2024-03-06T07:19:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc152ab0 0x7f44fc14eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc0c47d0 0x7f44fc14e5e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 --2- 172.21.15.26:0/2878847685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc06f120 0x7f44fc06ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 >> 172.21.15.26:0/2878847685 conn(0x7f44fc0c7800 msgr2=0x7f44fc0a7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 shutdown_connections 2024-03-06T07:19:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:44.926+0000 7f4500d6b640 1 -- 172.21.15.26:0/2878847685 wait complete. 2024-03-06T07:19:44.938 INFO:tasks.ceph:check_enable_crimson: False 2024-03-06T07:19:44.939 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-06T07:19:44.939 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:44.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-06T07:19:44.968 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:44.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-06T07:19:45.035 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:45.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-06T07:19:45.102 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:19:45.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-06T07:19:45.169 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:45.169 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-06T07:19:45.196 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:45.197 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-06T07:19:45.263 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:45.264 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-06T07:19:45.294 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:45.292+0000 7fe604a2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-06T07:19:45.320 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:19:45.320 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-06T07:19:45.337 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:45.336+0000 7fe38f83e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-06T07:19:45.351 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:45.349+0000 7fe604a2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-06T07:19:45.377 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new f435f5f5-06f4-4a6c-aade-2e030c8e66cb 0 2024-03-06T07:19:45.393 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:45.392+0000 7fe38f83e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-06T07:19:45.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.515+0000 7f0caed8c640 1 Processor -- start 2024-03-06T07:19:45.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.515+0000 7f0caed8c640 1 -- start start 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0caed8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ca814cde0 0x7f0ca8152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0caed8c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0ca80c2890 0x7f0ca80c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0caed8c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca8072ec0 con 0x7f0ca8072620 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0caed8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca80c2120 con 0x7f0ca814cde0 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0caed8c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca80731a0 con 0x7f0ca80c2890 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0ca7fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0ca8072620 0x7f0ca806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:35312/0 (socket says 172.21.15.117:35312) 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0cad329640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0ca80c2890 0x7f0ca80c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0ca7fff640 1 -- 172.21.15.117:0/2607776746 learned_addr learned my addr 172.21.15.117:0/2607776746 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0cacb28640 1 --2- 172.21.15.117:0/2607776746 >> v2:172.21.15.26:3300/0 conn(0x7f0ca814cde0 0x7f0ca8152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0cad329640 1 -- 172.21.15.117:0/2607776746 >> v2:172.21.15.26:3300/0 conn(0x7f0ca814cde0 msgr2=0x7f0ca8152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0cad329640 1 --2- 172.21.15.117:0/2607776746 >> v2:172.21.15.26:3300/0 conn(0x7f0ca814cde0 0x7f0ca8152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0cad329640 1 -- 172.21.15.117:0/2607776746 >> v1:172.21.15.117:6789/0 conn(0x7f0ca8072620 legacy=0x7f0ca806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0cad329640 1 -- 172.21.15.117:0/2607776746 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca8073480 con 0x7f0ca80c2890 2024-03-06T07:19:45.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0ca77fe640 1 -- 172.21.15.117:0/2607776746 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089091689 0 0) 0x7f0ca8072ec0 con 0x7f0ca8072620 2024-03-06T07:19:45.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.516+0000 7f0cacb28640 1 --2- 172.21.15.117:0/2607776746 >> v2:172.21.15.26:3300/0 conn(0x7f0ca814cde0 0x7f0ca8152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:45.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.517+0000 7f0cad329640 1 --2- 172.21.15.117:0/2607776746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca80c2890 0x7f0ca80c2ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8150a40 tx=0x7f0ca002be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d0cffa76cc1078 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:45.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0ca77fe640 1 -- 172.21.15.117:0/2607776746 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0039070 con 0x7f0ca80c2890 2024-03-06T07:19:45.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0ca77fe640 1 -- 172.21.15.117:0/2607776746 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca002cbf0 con 0x7f0ca80c2890 2024-03-06T07:19:45.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0ca77fe640 1 -- 172.21.15.117:0/2607776746 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0034440 con 0x7f0ca80c2890 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 -- 172.21.15.117:0/2607776746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca80c2890 msgr2=0x7f0ca80c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 --2- 172.21.15.117:0/2607776746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca80c2890 0x7f0ca80c2ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8150a40 tx=0x7f0ca002be40 comp rx=0 tx=0).stop 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 -- 172.21.15.117:0/2607776746 shutdown_connections 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 --2- 172.21.15.117:0/2607776746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca80c2890 0x7f0ca80c2ca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 --2- 172.21.15.117:0/2607776746 >> v2:172.21.15.26:3300/0 conn(0x7f0ca814cde0 0x7f0ca8152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 -- 172.21.15.117:0/2607776746 >> 172.21.15.117:0/2607776746 conn(0x7f0ca80a9b00 msgr2=0x7f0ca80c4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 -- 172.21.15.117:0/2607776746 shutdown_connections 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.518+0000 7f0caed8c640 1 -- 172.21.15.117:0/2607776746 wait complete. 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.519+0000 7f0caed8c640 1 Processor -- start 2024-03-06T07:19:45.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.519+0000 7f0caed8c640 1 -- start start 2024-03-06T07:19:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0caed8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca8072620 0x7f0ca80ab820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0caed8c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ca80c2890 0x7f0ca81e8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0caed8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca814cde0 0x7f0ca8148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0caed8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca806b850 con 0x7f0ca814cde0 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0caed8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca806b6d0 con 0x7f0ca8072620 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0caed8c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca814d1c0 con 0x7f0ca80c2890 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0cacb28640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca8072620 0x7f0ca80ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ca80c2890 0x7f0ca81e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0cacb28640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca8072620 0x7f0ca80ab820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37598/0 (socket says 172.21.15.117:37598) 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0cacb28640 1 -- 172.21.15.117:0/4000407020 learned_addr learned my addr 172.21.15.117:0/4000407020 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.520+0000 7f0cad329640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca814cde0 0x7f0ca8148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0cacb28640 1 -- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ca80c2890 msgr2=0x7f0ca81e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0cacb28640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ca80c2890 0x7f0ca81e8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0cacb28640 1 -- 172.21.15.117:0/4000407020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca814cde0 msgr2=0x7f0ca8148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0cacb28640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca814cde0 0x7f0ca8148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0cacb28640 1 -- 172.21.15.117:0/4000407020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80c3db0 con 0x7f0ca8072620 2024-03-06T07:19:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0ca7fff640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ca80c2890 0x7f0ca81e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0cacb28640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca8072620 0x7f0ca80ab820 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00d7d0 tx=0x7f0c9c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0ca57fa640 1 -- 172.21.15.117:0/4000407020 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c004030 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0ca57fa640 1 -- 172.21.15.117:0/4000407020 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c0041d0 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0ca57fa640 1 -- 172.21.15.117:0/4000407020 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c00b6c0 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca81546b0 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca8154c70 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0ca57fa640 1 -- 172.21.15.117:0/4000407020 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0c9c00b860 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.521+0000 7f0ca57fa640 1 -- 172.21.15.117:0/4000407020 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0c9c014770 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.522+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca804f330 con 0x7f0ca8072620 2024-03-06T07:19:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.524+0000 7f0ca57fa640 1 -- 172.21.15.117:0/4000407020 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0c9c00bce0 con 0x7f0ca8072620 2024-03-06T07:19:45.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.596+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f435f5f5-06f4-4a6c-aade-2e030c8e66cb", "id": 0} v 0) v1 -- 0x7f0ca814d4c0 con 0x7f0ca8072620 2024-03-06T07:19:45.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.599+0000 7f0ca57fa640 1 -- 172.21.15.117:0/4000407020 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f435f5f5-06f4-4a6c-aade-2e030c8e66cb", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c9c003610 con 0x7f0ca8072620 2024-03-06T07:19:45.601 INFO:teuthology.orchestra.run.smithi117.stdout:0 2024-03-06T07:19:45.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca8072620 msgr2=0x7f0ca80ab820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca8072620 0x7f0ca80ab820 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00d7d0 tx=0x7f0c9c00dc90 comp rx=0 tx=0).stop 2024-03-06T07:19:45.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 shutdown_connections 2024-03-06T07:19:45.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca814cde0 0x7f0ca8148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ca80c2890 0x7f0ca81e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 --2- 172.21.15.117:0/4000407020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca8072620 0x7f0ca80ab820 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 >> 172.21.15.117:0/4000407020 conn(0x7f0ca80a9b00 msgr2=0x7f0ca81530a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:45.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 shutdown_connections 2024-03-06T07:19:45.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.600+0000 7f0caed8c640 1 -- 172.21.15.117:0/4000407020 wait complete. 2024-03-06T07:19:45.613 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new f298b038-2b12-4ed8-8fc3-3bce46a1e6c8 1 2024-03-06T07:19:45.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.710+0000 7f2f2c1eb640 1 Processor -- start 2024-03-06T07:19:45.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f2c1eb640 1 -- start start 2024-03-06T07:19:45.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f2c1eb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f24157220 0x7f2f240c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f2c1eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f240c8460 0x7f2f2406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f2c1eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f240c7f50 con 0x7f2f240c58c0 2024-03-06T07:19:45.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f2c1eb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2406ecc0 con 0x7f2f24157220 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f2c1eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f240c3e80 con 0x7f2f240c8460 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f29786640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f240c8460 0x7f2f2406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.711+0000 7f2f29786640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f240c8460 0x7f2f2406f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37604/0 (socket says 172.21.15.117:37604) 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.712+0000 7f2f29786640 1 -- 172.21.15.117:0/2584417329 learned_addr learned my addr 172.21.15.117:0/2584417329 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.712+0000 7f2f29f87640 1 --2- 172.21.15.117:0/2584417329 >> v2:172.21.15.26:3300/0 conn(0x7f2f24157220 0x7f2f240c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.712+0000 7f2f29786640 1 -- 172.21.15.117:0/2584417329 >> v2:172.21.15.26:3300/0 conn(0x7f2f24157220 msgr2=0x7f2f240c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.712+0000 7f2f29786640 1 --2- 172.21.15.117:0/2584417329 >> v2:172.21.15.26:3300/0 conn(0x7f2f24157220 0x7f2f240c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.712+0000 7f2f29786640 1 -- 172.21.15.117:0/2584417329 >> v1:172.21.15.26:6789/0 conn(0x7f2f240c58c0 legacy=0x7f2f240725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.712+0000 7f2f29786640 1 -- 172.21.15.117:0/2584417329 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f240c4160 con 0x7f2f240c8460 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.712+0000 7f2f29786640 1 --2- 172.21.15.117:0/2584417329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f2406f1a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f24073760 tx=0x7f2f1402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f6e4fea6903a79 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f28f85640 1 -- 172.21.15.117:0/2584417329 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1402e070 con 0x7f2f240c8460 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f28f85640 1 -- 172.21.15.117:0/2584417329 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14037070 con 0x7f2f240c8460 2024-03-06T07:19:45.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f28f85640 1 -- 172.21.15.117:0/2584417329 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14031a00 con 0x7f2f240c8460 2024-03-06T07:19:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/2584417329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 msgr2=0x7f2f2406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 --2- 172.21.15.117:0/2584417329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f2406f1a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f24073760 tx=0x7f2f1402b880 comp rx=0 tx=0).stop 2024-03-06T07:19:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/2584417329 shutdown_connections 2024-03-06T07:19:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 --2- 172.21.15.117:0/2584417329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f2406f1a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 --2- 172.21.15.117:0/2584417329 >> v2:172.21.15.26:3300/0 conn(0x7f2f24157220 0x7f2f240c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/2584417329 >> 172.21.15.117:0/2584417329 conn(0x7f2f240c88f0 msgr2=0x7f2f240c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/2584417329 shutdown_connections 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.713+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/2584417329 wait complete. 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.714+0000 7f2f2c1eb640 1 Processor -- start 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.714+0000 7f2f2c1eb640 1 -- start start 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2c1eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f240c58c0 0x7f2f2414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2c1eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f24143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2c1eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f24157220 0x7f2f24144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2c1eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240c45c0 con 0x7f2f240c58c0 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2c1eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240c4440 con 0x7f2f240c8460 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2c1eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240c3500 con 0x7f2f24157220 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f29786640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f24143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2a788640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f24157220 0x7f2f24144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f29786640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f24143cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37608/0 (socket says 172.21.15.117:37608) 2024-03-06T07:19:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f29786640 1 -- 172.21.15.117:0/921977765 learned_addr learned my addr 172.21.15.117:0/921977765 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f29f87640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f240c58c0 0x7f2f2414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2a788640 1 -- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 msgr2=0x7f2f24143cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2a788640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f24143cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2a788640 1 -- 172.21.15.117:0/921977765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f240c58c0 msgr2=0x7f2f2414d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2a788640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f240c58c0 0x7f2f2414d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f2a788640 1 -- 172.21.15.117:0/921977765 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2414fa00 con 0x7f2f24157220 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.715+0000 7f2f29786640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f24143cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f2a788640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f24157220 0x7f2f24144230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00be00 tx=0x7f2f1c00bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f1a7fc640 1 -- 172.21.15.117:0/921977765 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c003970 con 0x7f2f24157220 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f1a7fc640 1 -- 172.21.15.117:0/921977765 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c003b10 con 0x7f2f24157220 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f1a7fc640 1 -- 172.21.15.117:0/921977765 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c004020 con 0x7f2f24157220 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2406cf10 con 0x7f2f24157220 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241eb8f0 con 0x7f2f24157220 2024-03-06T07:19:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f1a7fc640 1 -- 172.21.15.117:0/921977765 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2f1c0041c0 con 0x7f2f24157220 2024-03-06T07:19:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.716+0000 7f2f1a7fc640 1 -- 172.21.15.117:0/921977765 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2f1c01d3f0 con 0x7f2f24157220 2024-03-06T07:19:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.717+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2404f330 con 0x7f2f24157220 2024-03-06T07:19:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.719+0000 7f2f1a7fc640 1 -- 172.21.15.117:0/921977765 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2f1c004690 con 0x7f2f24157220 2024-03-06T07:19:45.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.791+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f298b038-2b12-4ed8-8fc3-3bce46a1e6c8", "id": 1} v 0) v1 -- 0x7f2f2406e440 con 0x7f2f24157220 2024-03-06T07:19:45.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.797+0000 7f2f1a7fc640 1 -- 172.21.15.117:0/921977765 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f298b038-2b12-4ed8-8fc3-3bce46a1e6c8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2f1c014440 con 0x7f2f24157220 2024-03-06T07:19:45.799 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-03-06T07:19:45.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f24157220 msgr2=0x7f2f24144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f24157220 0x7f2f24144230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00be00 tx=0x7f2f1c00bef0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 shutdown_connections 2024-03-06T07:19:45.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f24157220 0x7f2f24144230 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240c8460 0x7f2f24143cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 --2- 172.21.15.117:0/921977765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f240c58c0 0x7f2f2414d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 >> 172.21.15.117:0/921977765 conn(0x7f2f240c88f0 msgr2=0x7f2f2406f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:45.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 shutdown_connections 2024-03-06T07:19:45.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.798+0000 7f2f2c1eb640 1 -- 172.21.15.117:0/921977765 wait complete. 2024-03-06T07:19:45.811 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new d26cf89c-159b-4198-a291-5695dc6809a9 2 2024-03-06T07:19:45.904 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:45.902+0000 7fe604a2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-06T07:19:45.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.911+0000 7f280c748640 1 Processor -- start 2024-03-06T07:19:45.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.911+0000 7f280c748640 1 -- start start 2024-03-06T07:19:45.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280c748640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28041533d0 0x7f280406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280c748640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28040c3440 0x7f28040c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280c748640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28041537b0 0x7f28040c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280c748640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280406d180 con 0x7f28041533d0 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280c748640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280406c7b0 con 0x7f28041537b0 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280c748640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280406c1d0 con 0x7f28040c3440 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280ace5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28041537b0 0x7f28040c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f2809ce3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28040c3440 0x7f28040c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f2809ce3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28040c3440 0x7f28040c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:34392/0 (socket says 172.21.15.117:34392) 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280ace5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28041537b0 0x7f28040c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37612/0 (socket says 172.21.15.117:37612) 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f2809ce3640 1 -- 172.21.15.117:0/4107540182 learned_addr learned my addr 172.21.15.117:0/4107540182 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280a4e4640 1 --2- 172.21.15.117:0/4107540182 >> v2:172.21.15.26:3300/0 conn(0x7f28041533d0 0x7f280406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280ace5640 1 -- 172.21.15.117:0/4107540182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28040c3440 msgr2=0x7f28040c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280ace5640 1 --2- 172.21.15.117:0/4107540182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28040c3440 0x7f28040c2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280ace5640 1 -- 172.21.15.117:0/4107540182 >> v2:172.21.15.26:3300/0 conn(0x7f28041533d0 msgr2=0x7f280406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:45.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280ace5640 1 --2- 172.21.15.117:0/4107540182 >> v2:172.21.15.26:3300/0 conn(0x7f28041533d0 0x7f280406cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280ace5640 1 -- 172.21.15.117:0/4107540182 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040c3a00 con 0x7f28041537b0 2024-03-06T07:19:45.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.912+0000 7f280a4e4640 1 --2- 172.21.15.117:0/4107540182 >> v2:172.21.15.26:3300/0 conn(0x7f28041533d0 0x7f280406cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.913+0000 7f280ace5640 1 --2- 172.21.15.117:0/4107540182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28041537b0 0x7f28040c6b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28040adaa0 tx=0x7f27fc02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec78c3b4273cc95 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.913+0000 7f28094e2640 1 -- 172.21.15.117:0/4107540182 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc039070 con 0x7f28041537b0 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.913+0000 7f28094e2640 1 -- 172.21.15.117:0/4107540182 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc02cc60 con 0x7f28041537b0 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.913+0000 7f28094e2640 1 -- 172.21.15.117:0/4107540182 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc034660 con 0x7f28041537b0 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 -- 172.21.15.117:0/4107540182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28041537b0 msgr2=0x7f28040c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 --2- 172.21.15.117:0/4107540182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28041537b0 0x7f28040c6b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28040adaa0 tx=0x7f27fc02beb0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 -- 172.21.15.117:0/4107540182 shutdown_connections 2024-03-06T07:19:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 --2- 172.21.15.117:0/4107540182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28040c3440 0x7f28040c2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 --2- 172.21.15.117:0/4107540182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28041537b0 0x7f28040c6b60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 --2- 172.21.15.117:0/4107540182 >> v2:172.21.15.26:3300/0 conn(0x7f28041533d0 0x7f280406cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 -- 172.21.15.117:0/4107540182 >> 172.21.15.117:0/4107540182 conn(0x7f28040a9540 msgr2=0x7f28040a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 -- 172.21.15.117:0/4107540182 shutdown_connections 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.914+0000 7f280c748640 1 -- 172.21.15.117:0/4107540182 wait complete. 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.915+0000 7f280c748640 1 Processor -- start 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.915+0000 7f280c748640 1 -- start start 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280c748640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28040c3440 0x7f28040bc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280c748640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28041533d0 0x7f28041e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280c748640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28041537b0 0x7f28040ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280c748640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c3ce0 con 0x7f28041537b0 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280c748640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c26d0 con 0x7f28040c3440 2024-03-06T07:19:45.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280c748640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c3e60 con 0x7f28041533d0 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28040c3440 0x7f28040bc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f2809ce3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28041533d0 0x7f28041e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28040c3440 0x7f28040bc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37614/0 (socket says 172.21.15.117:37614) 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 -- 172.21.15.117:0/2824103327 learned_addr learned my addr 172.21.15.117:0/2824103327 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280ace5640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28041537b0 0x7f28040ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 -- 172.21.15.117:0/2824103327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28041533d0 msgr2=0x7f28041e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28041533d0 0x7f28041e9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 -- 172.21.15.117:0/2824103327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28041537b0 msgr2=0x7f28040ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28041537b0 0x7f28040ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 -- 172.21.15.117:0/2824103327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040ad590 con 0x7f28040c3440 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f280a4e4640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28040c3440 0x7f28040bc7b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e810 tx=0x7f27f800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f27f6ffd640 1 -- 172.21.15.117:0/2824103327 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800f040 con 0x7f28040c3440 2024-03-06T07:19:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.916+0000 7f27f6ffd640 1 -- 172.21.15.117:0/2824103327 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8004030 con 0x7f28040c3440 2024-03-06T07:19:45.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.917+0000 7f27f6ffd640 1 -- 172.21.15.117:0/2824103327 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8004400 con 0x7f28040c3440 2024-03-06T07:19:45.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.917+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28041ec230 con 0x7f28040c3440 2024-03-06T07:19:45.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.917+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280414cf70 con 0x7f28040c3440 2024-03-06T07:19:45.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.917+0000 7f27f6ffd640 1 -- 172.21.15.117:0/2824103327 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f27f800ca60 con 0x7f28040c3440 2024-03-06T07:19:45.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.917+0000 7f27f6ffd640 1 -- 172.21.15.117:0/2824103327 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f27f8013920 con 0x7f28040c3440 2024-03-06T07:19:45.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.917+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280404f330 con 0x7f28040c3440 2024-03-06T07:19:45.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.920+0000 7f27f6ffd640 1 -- 172.21.15.117:0/2824103327 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f27f8017020 con 0x7f28040c3440 2024-03-06T07:19:45.961 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:45.959+0000 7fe604a2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-06T07:19:45.961 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:45.960+0000 7fe38f83e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-06T07:19:45.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.991+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d26cf89c-159b-4198-a291-5695dc6809a9", "id": 2} v 0) v1 -- 0x7f28040c6630 con 0x7f28040c3440 2024-03-06T07:19:45.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.994+0000 7f27f6ffd640 1 -- 172.21.15.117:0/2824103327 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d26cf89c-159b-4198-a291-5695dc6809a9", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f27f8013500 con 0x7f28040c3440 2024-03-06T07:19:45.997 INFO:teuthology.orchestra.run.smithi117.stdout:2 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28040c3440 msgr2=0x7f28040bc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28040c3440 0x7f28040bc7b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e810 tx=0x7f27f800ecd0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 shutdown_connections 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28041537b0 0x7f28040ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28041533d0 0x7f28041e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 --2- 172.21.15.117:0/2824103327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28040c3440 0x7f28040bc7b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 >> 172.21.15.117:0/2824103327 conn(0x7f28040a9540 msgr2=0x7f280414fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:45.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 shutdown_connections 2024-03-06T07:19:45.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:45.995+0000 7f280c748640 1 -- 172.21.15.117:0/2824103327 wait complete. 2024-03-06T07:19:46.007 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 364b50e5-d473-48ed-ab52-4dcce8fe3471 3 2024-03-06T07:19:46.007 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.005+0000 7fe604a2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-06T07:19:46.018 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.017+0000 7fe38f83e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-06T07:19:46.064 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.062+0000 7fe604a2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-06T07:19:46.065 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.063+0000 7fe38f83e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-06T07:19:46.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.106+0000 7f2bdfe33640 1 Processor -- start 2024-03-06T07:19:46.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.106+0000 7f2bdfe33640 1 -- start start 2024-03-06T07:19:46.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bdfe33640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2bd806c340 0x7f2bd8152860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.109 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.107+0000 7fe604a2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-06T07:19:46.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bdfe33640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80c7060 con 0x7f2bd80a6cc0 2024-03-06T07:19:46.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bdfe33640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80c7340 con 0x7f2bd8155370 2024-03-06T07:19:46.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bdfe33640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd80c2b10 con 0x7f2bd806c340 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bddbcf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2bd806c340 0x7f2bd8152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bddbcf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2bd806c340 0x7f2bd8152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37626/0 (socket says 172.21.15.117:37626) 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bddbcf640 1 -- 172.21.15.117:0/1561847269 learned_addr learned my addr 172.21.15.117:0/1561847269 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bddbcf640 1 -- 172.21.15.117:0/1561847269 >> v1:172.21.15.117:6789/0 conn(0x7f2bd8155370 legacy=0x7f2bd8155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bddbcf640 1 -- 172.21.15.117:0/1561847269 >> v1:172.21.15.26:6789/0 conn(0x7f2bd80a6cc0 legacy=0x7f2bd814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.107+0000 7f2bddbcf640 1 -- 172.21.15.117:0/1561847269 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80c27a0 con 0x7f2bd806c340 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bddbcf640 1 --2- 172.21.15.117:0/1561847269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd8152860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80c2c90 tx=0x7f2bd402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf031a9ced876d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bdcbcd640 1 -- 172.21.15.117:0/1561847269 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4035020 con 0x7f2bd806c340 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bdcbcd640 1 -- 172.21.15.117:0/1561847269 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4031640 con 0x7f2bd806c340 2024-03-06T07:19:46.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bdcbcd640 1 -- 172.21.15.117:0/1561847269 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd402d040 con 0x7f2bd806c340 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bdfe33640 1 -- 172.21.15.117:0/1561847269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 msgr2=0x7f2bd8152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bdfe33640 1 --2- 172.21.15.117:0/1561847269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd8152860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80c2c90 tx=0x7f2bd402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bdfe33640 1 -- 172.21.15.117:0/1561847269 shutdown_connections 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.108+0000 7f2bdfe33640 1 --2- 172.21.15.117:0/1561847269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd8152860 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.109+0000 7f2bdfe33640 1 -- 172.21.15.117:0/1561847269 >> 172.21.15.117:0/1561847269 conn(0x7f2bd80a9540 msgr2=0x7f2bd80a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.109+0000 7f2bdfe33640 1 -- 172.21.15.117:0/1561847269 shutdown_connections 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.109+0000 7f2bdfe33640 1 -- 172.21.15.117:0/1561847269 wait complete. 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 Processor -- start 2024-03-06T07:19:46.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 -- start start 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd81e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd80a6cc0 0x7f2bd80a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd8155370 0x7f2bd80a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd8155e90 con 0x7f2bd8155370 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80c2440 con 0x7f2bd806c340 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdfe33640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd8156010 con 0x7f2bd80a6cc0 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bddbcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd81e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bddbcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd81e6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37634/0 (socket says 172.21.15.117:37634) 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bddbcf640 1 -- 172.21.15.117:0/3008553924 learned_addr learned my addr 172.21.15.117:0/3008553924 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bdd3ce640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd80a6cc0 0x7f2bd80a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.110+0000 7f2bde3d0640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd8155370 0x7f2bd80a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bddbcf640 1 -- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd80a6cc0 msgr2=0x7f2bd80a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bddbcf640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd80a6cc0 0x7f2bd80a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bddbcf640 1 -- 172.21.15.117:0/3008553924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd8155370 msgr2=0x7f2bd80a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bddbcf640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd8155370 0x7f2bd80a77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bddbcf640 1 -- 172.21.15.117:0/3008553924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80715c0 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bddbcf640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd81e6d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2bd402b930 tx=0x7f2bd40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bce7fc640 1 -- 172.21.15.117:0/3008553924 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4040070 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bce7fc640 1 -- 172.21.15.117:0/3008553924 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4039c10 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd8149010 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bce7fc640 1 -- 172.21.15.117:0/3008553924 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4035040 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd81e76c0 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bce7fc640 1 -- 172.21.15.117:0/3008553924 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2bd4038020 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.111+0000 7f2bce7fc640 1 -- 172.21.15.117:0/3008553924 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f2bd403ad80 con 0x7f2bd806c340 2024-03-06T07:19:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.112+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd806e140 con 0x7f2bd806c340 2024-03-06T07:19:46.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.114+0000 7f2bce7fc640 1 -- 172.21.15.117:0/3008553924 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2bd806e140 con 0x7f2bd806c340 2024-03-06T07:19:46.120 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.119+0000 7fe38f83e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-06T07:19:46.164 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.163+0000 7fe38f83e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-06T07:19:46.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.185+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "364b50e5-d473-48ed-ab52-4dcce8fe3471", "id": 3} v 0) v1 -- 0x7f2bd806daa0 con 0x7f2bd806c340 2024-03-06T07:19:46.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.188+0000 7f2bce7fc640 1 -- 172.21.15.117:0/3008553924 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "364b50e5-d473-48ed-ab52-4dcce8fe3471", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2bd80715c0 con 0x7f2bd806c340 2024-03-06T07:19:46.190 INFO:teuthology.orchestra.run.smithi117.stdout:3 2024-03-06T07:19:46.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 msgr2=0x7f2bd81e6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd81e6d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2bd402b930 tx=0x7f2bd40046c0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 shutdown_connections 2024-03-06T07:19:46.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd8155370 0x7f2bd80a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd80a6cc0 0x7f2bd80a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 --2- 172.21.15.117:0/3008553924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd806c340 0x7f2bd81e6d50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 >> 172.21.15.117:0/3008553924 conn(0x7f2bd80a9540 msgr2=0x7f2bd8147a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 shutdown_connections 2024-03-06T07:19:46.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.189+0000 7f2bdfe33640 1 -- 172.21.15.117:0/3008553924 wait complete. 2024-03-06T07:19:46.202 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 72a8daac-45a0-4413-a095-8fed3c23b00d 4 2024-03-06T07:19:46.230 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.229+0000 7fe604a2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-06T07:19:46.285 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.283+0000 7fe38f83e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-06T07:19:46.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.299+0000 7f7e84042640 1 Processor -- start 2024-03-06T07:19:46.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.299+0000 7f7e84042640 1 -- start start 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e84042640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c06c340 0x7f7e7c152860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e84042640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e7c0a6cc0 0x7f7e7c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e84042640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e7c14f540 0x7f7e7c155370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e84042640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c0c2b10 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e84042640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c0c2440 con 0x7f7e7c14f540 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e84042640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0c7060 con 0x7f7e7c06c340 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e81dde640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c06c340 0x7f7e7c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e825df640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e7c14f540 0x7f7e7c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e81dde640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c06c340 0x7f7e7c152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:34430/0 (socket says 172.21.15.117:34430) 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e81dde640 1 -- 172.21.15.117:0/14331028 learned_addr learned my addr 172.21.15.117:0/14331028 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e825df640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e7c14f540 0x7f7e7c155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37636/0 (socket says 172.21.15.117:37636) 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.300+0000 7f7e815dd640 1 --2- 172.21.15.117:0/14331028 >> v2:172.21.15.26:3300/0 conn(0x7f7e7c0a6cc0 0x7f7e7c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.301+0000 7f7e825df640 1 -- 172.21.15.117:0/14331028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c06c340 msgr2=0x7f7e7c152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.301+0000 7f7e825df640 1 --2- 172.21.15.117:0/14331028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c06c340 0x7f7e7c152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.301+0000 7f7e825df640 1 -- 172.21.15.117:0/14331028 >> v2:172.21.15.26:3300/0 conn(0x7f7e7c0a6cc0 msgr2=0x7f7e7c14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.301+0000 7f7e825df640 1 --2- 172.21.15.117:0/14331028 >> v2:172.21.15.26:3300/0 conn(0x7f7e7c0a6cc0 0x7f7e7c14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.301+0000 7f7e825df640 1 -- 172.21.15.117:0/14331028 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c0c7340 con 0x7f7e7c14f540 2024-03-06T07:19:46.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.301+0000 7f7e815dd640 1 --2- 172.21.15.117:0/14331028 >> v2:172.21.15.26:3300/0 conn(0x7f7e7c0a6cc0 0x7f7e7c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:46.305 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.302+0000 7fe604a2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.302+0000 7f7e825df640 1 --2- 172.21.15.117:0/14331028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c14f540 0x7f7e7c155370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c071380 tx=0x7f7e7402bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75dafc6f207b8638 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.302+0000 7f7e80ddc640 1 -- 172.21.15.117:0/14331028 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74039070 con 0x7f7e7c14f540 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.302+0000 7f7e80ddc640 1 -- 172.21.15.117:0/14331028 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7402cab0 con 0x7f7e7c14f540 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.302+0000 7f7e80ddc640 1 -- 172.21.15.117:0/14331028 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74034440 con 0x7f7e7c14f540 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 -- 172.21.15.117:0/14331028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c14f540 msgr2=0x7f7e7c155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 --2- 172.21.15.117:0/14331028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c14f540 0x7f7e7c155370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c071380 tx=0x7f7e7402bd00 comp rx=0 tx=0).stop 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 -- 172.21.15.117:0/14331028 shutdown_connections 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 --2- 172.21.15.117:0/14331028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c14f540 0x7f7e7c155370 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 --2- 172.21.15.117:0/14331028 >> v2:172.21.15.26:3300/0 conn(0x7f7e7c0a6cc0 0x7f7e7c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 --2- 172.21.15.117:0/14331028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c06c340 0x7f7e7c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 -- 172.21.15.117:0/14331028 >> 172.21.15.117:0/14331028 conn(0x7f7e7c0a9540 msgr2=0x7f7e7c0a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 -- 172.21.15.117:0/14331028 shutdown_connections 2024-03-06T07:19:46.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.303+0000 7f7e84042640 1 -- 172.21.15.117:0/14331028 wait complete. 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.304+0000 7f7e84042640 1 Processor -- start 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.304+0000 7f7e84042640 1 -- start start 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.304+0000 7f7e84042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e7c06c340 0x7f7e7c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.304+0000 7f7e84042640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c0a6cc0 0x7f7e7c153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e84042640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c14f540 0x7f7e7c1edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e84042640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c06c8a0 con 0x7f7e7c06c340 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e84042640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c06c720 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e84042640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1558b0 con 0x7f7e7c14f540 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c0a6cc0 0x7f7e7c153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c0a6cc0 0x7f7e7c153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37650/0 (socket says 172.21.15.117:37650) 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 -- 172.21.15.117:0/443362719 learned_addr learned my addr 172.21.15.117:0/443362719 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e825df640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c14f540 0x7f7e7c1edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e81dde640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e7c06c340 0x7f7e7c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 -- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c14f540 msgr2=0x7f7e7c1edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c14f540 0x7f7e7c1edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 -- 172.21.15.117:0/443362719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e7c06c340 msgr2=0x7f7e7c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e7c06c340 0x7f7e7c1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 -- 172.21.15.117:0/443362719 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c1552f0 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e815dd640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c0a6cc0 0x7f7e7c153d10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ebe0 tx=0x7f7e7800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e727fc640 1 -- 172.21.15.117:0/443362719 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78017070 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e727fc640 1 -- 172.21.15.117:0/443362719 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7800cc30 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c154560 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e727fc640 1 -- 172.21.15.117:0/443362719 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78013470 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c0726b0 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.305+0000 7f7e727fc640 1 -- 172.21.15.117:0/443362719 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7e7801a070 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.306+0000 7f7e727fc640 1 -- 172.21.15.117:0/443362719 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7e7801c9b0 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.306+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c04f330 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.308+0000 7f7e727fc640 1 -- 172.21.15.117:0/443362719 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7e7801cc50 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.351 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.349+0000 7fe604a2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-06T07:19:46.357 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.356+0000 7fe38f83e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:46.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.379+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72a8daac-45a0-4413-a095-8fed3c23b00d", "id": 4} v 0) v1 -- 0x7f7e7c072180 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.382+0000 7f7e727fc640 1 -- 172.21.15.117:0/443362719 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72a8daac-45a0-4413-a095-8fed3c23b00d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7e7801c4a0 con 0x7f7e7c0a6cc0 2024-03-06T07:19:46.384 INFO:teuthology.orchestra.run.smithi117.stdout:4 2024-03-06T07:19:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c0a6cc0 msgr2=0x7f7e7c153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c0a6cc0 0x7f7e7c153d10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ebe0 tx=0x7f7e7800c520 comp rx=0 tx=0).stop 2024-03-06T07:19:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 shutdown_connections 2024-03-06T07:19:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7c14f540 0x7f7e7c1edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e7c0a6cc0 0x7f7e7c153d10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 --2- 172.21.15.117:0/443362719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e7c06c340 0x7f7e7c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 >> 172.21.15.117:0/443362719 conn(0x7f7e7c0a9540 msgr2=0x7f7e7c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 shutdown_connections 2024-03-06T07:19:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.383+0000 7f7e84042640 1 -- 172.21.15.117:0/443362719 wait complete. 2024-03-06T07:19:46.395 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 63672daf-3c97-4b49-851c-428e61b9fdc2 5 2024-03-06T07:19:46.408 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.405+0000 7fe38f83e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-06T07:19:46.408 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.404+0000 7fe604a2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-06T07:19:46.456 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.454+0000 7fe604a2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-06T07:19:46.460 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.459+0000 7fe38f83e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-06T07:19:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.506+0000 7f11cec65640 1 Processor -- start 2024-03-06T07:19:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.506+0000 7f11cec65640 1 -- start start 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cec65640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f11c806c240 0x7f11c8159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cec65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cec65640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c8155e50 con 0x7f11c806dc80 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cec65640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c8159650 con 0x7f11c806c240 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cec65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8155c00 con 0x7f11c806e140 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11c7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f11c806dc80 0x7f11c80c1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.117:53908/0 (socket says 172.21.15.117:53908) 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11c7fff640 1 -- 172.21.15.117:0/1656608134 learned_addr learned my addr 172.21.15.117:0/1656608134 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cca01640 1 --2- 172.21.15.117:0/1656608134 >> v2:172.21.15.117:3300/0 conn(0x7f11c806c240 0x7f11c8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cd202640 1 --2- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11c77fe640 1 -- 172.21.15.117:0/1656608134 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563337927 0 0) 0x7f11c8155e50 con 0x7f11c806dc80 2024-03-06T07:19:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11c77fe640 1 -- 172.21.15.117:0/1656608134 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b8003520 con 0x7f11c806dc80 2024-03-06T07:19:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cca01640 1 -- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 msgr2=0x7f11c8152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.507+0000 7f11cca01640 1 --2- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.508+0000 7f11cca01640 1 -- 172.21.15.117:0/1656608134 >> v1:172.21.15.26:6789/0 conn(0x7f11c806dc80 legacy=0x7f11c80c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.508+0000 7f11cca01640 1 -- 172.21.15.117:0/1656608134 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c8158910 con 0x7f11c806c240 2024-03-06T07:19:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.508+0000 7f11cd202640 1 --2- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:46.511 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.508+0000 7fe38f83e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.509+0000 7f11cca01640 1 --2- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806c240 0x7f11c8159110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0097d0 tx=0x7f11bc02b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5a92ad693d9077 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.509+0000 7f11c77fe640 1 -- 172.21.15.117:0/1656608134 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc031770 con 0x7f11c806c240 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.509+0000 7f11c77fe640 1 -- 172.21.15.117:0/1656608134 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc037070 con 0x7f11c806c240 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.509+0000 7f11c77fe640 1 -- 172.21.15.117:0/1656608134 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc031b30 con 0x7f11c806c240 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.509+0000 7f11cec65640 1 -- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806c240 msgr2=0x7f11c8159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.509+0000 7f11cec65640 1 --2- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806c240 0x7f11c8159110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0097d0 tx=0x7f11bc02b810 comp rx=0 tx=0).stop 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.510+0000 7f11cec65640 1 -- 172.21.15.117:0/1656608134 shutdown_connections 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.510+0000 7f11cec65640 1 --2- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.510+0000 7f11cec65640 1 --2- 172.21.15.117:0/1656608134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806c240 0x7f11c8159110 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.510+0000 7f11cec65640 1 -- 172.21.15.117:0/1656608134 >> 172.21.15.117:0/1656608134 conn(0x7f11c80c3b40 msgr2=0x7f11c80c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.510+0000 7f11cec65640 1 -- 172.21.15.117:0/1656608134 shutdown_connections 2024-03-06T07:19:46.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.510+0000 7f11cec65640 1 -- 172.21.15.117:0/1656608134 wait complete. 2024-03-06T07:19:46.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.511+0000 7f11cec65640 1 Processor -- start 2024-03-06T07:19:46.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.511+0000 7f11cec65640 1 -- start start 2024-03-06T07:19:46.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c806c240 0x7f11c8152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806dc80 0x7f11c81eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8158c80 con 0x7f11c806c240 2024-03-06T07:19:46.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c806f2f0 con 0x7f11c806dc80 2024-03-06T07:19:46.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c806c620 con 0x7f11c806e140 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806dc80 0x7f11c81eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cd202640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806dc80 0x7f11c81eeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37660/0 (socket says 172.21.15.117:37660) 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cd202640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:34460/0 (socket says 172.21.15.117:34460) 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 -- 172.21.15.117:0/710327759 learned_addr learned my addr 172.21.15.117:0/710327759 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cca01640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c806c240 0x7f11c8152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 -- 172.21.15.117:0/710327759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 msgr2=0x7f11c8157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 -- 172.21.15.117:0/710327759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c806c240 msgr2=0x7f11c8152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c806c240 0x7f11c8152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 -- 172.21.15.117:0/710327759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c81534f0 con 0x7f11c806dc80 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c7fff640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806dc80 0x7f11c81eeb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c400 tx=0x7f11b400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c57fa640 1 -- 172.21.15.117:0/710327759 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4017070 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c57fa640 1 -- 172.21.15.117:0/710327759 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b4005d00 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c80c6290 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11c57fa640 1 -- 172.21.15.117:0/710327759 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4007310 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.512+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c81ee370 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.513+0000 7f11c57fa640 1 -- 172.21.15.117:0/710327759 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f11b401a070 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.513+0000 7f11c57fa640 1 -- 172.21.15.117:0/710327759 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f11b4012980 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.513+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c804f330 con 0x7f11c806dc80 2024-03-06T07:19:46.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.515+0000 7f11c57fa640 1 -- 172.21.15.117:0/710327759 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f11b4021020 con 0x7f11c806dc80 2024-03-06T07:19:46.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.588+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "63672daf-3c97-4b49-851c-428e61b9fdc2", "id": 5} v 0) v1 -- 0x7f11c814b6f0 con 0x7f11c806dc80 2024-03-06T07:19:46.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.591+0000 7f11c57fa640 1 -- 172.21.15.117:0/710327759 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63672daf-3c97-4b49-851c-428e61b9fdc2", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f11b40075e0 con 0x7f11c806dc80 2024-03-06T07:19:46.593 INFO:teuthology.orchestra.run.smithi117.stdout:5 2024-03-06T07:19:46.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806dc80 msgr2=0x7f11c81eeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806dc80 0x7f11c81eeb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c400 tx=0x7f11b400c800 comp rx=0 tx=0).stop 2024-03-06T07:19:46.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 shutdown_connections 2024-03-06T07:19:46.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c806e140 0x7f11c8157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c806dc80 0x7f11c81eeb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 --2- 172.21.15.117:0/710327759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c806c240 0x7f11c8152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 >> 172.21.15.117:0/710327759 conn(0x7f11c80c3b40 msgr2=0x7f11c814c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 shutdown_connections 2024-03-06T07:19:46.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.592+0000 7f11cec65640 1 -- 172.21.15.117:0/710327759 wait complete. 2024-03-06T07:19:46.604 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new ba4402fa-8b2c-4969-84ca-50316c20363d 6 2024-03-06T07:19:46.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.703+0000 7f31ee3d0640 1 Processor -- start 2024-03-06T07:19:46.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.703+0000 7f31ee3d0640 1 -- start start 2024-03-06T07:19:46.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31ee3d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31e80c2f20 0x7f31e8152e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31ee3d0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80c61f0 con 0x7f31e80c6670 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31ee3d0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e81534b0 con 0x7f31e806c6f0 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31ee3d0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e8153790 con 0x7f31e80c2f20 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31ec96d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f31e806c6f0 0x7f31e806caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:35318/0 (socket says 172.21.15.117:35318) 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31ec96d640 1 -- 172.21.15.117:0/3177850745 learned_addr learned my addr 172.21.15.117:0/3177850745 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31e7fff640 1 --2- 172.21.15.117:0/3177850745 >> v2:172.21.15.26:3300/0 conn(0x7f31e80c2f20 0x7f31e8152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31e6ffd640 1 -- 172.21.15.117:0/3177850745 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856162071 0 0) 0x7f31e81534b0 con 0x7f31e806c6f0 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31e6ffd640 1 -- 172.21.15.117:0/3177850745 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d0003520 con 0x7f31e806c6f0 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31e7fff640 1 -- 172.21.15.117:0/3177850745 >> v1:172.21.15.117:6789/0 conn(0x7f31e806c6f0 legacy=0x7f31e806caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31e7fff640 1 -- 172.21.15.117:0/3177850745 >> v1:172.21.15.26:6789/0 conn(0x7f31e80c6670 legacy=0x7f31e80c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31e6ffd640 1 -- 172.21.15.117:0/3177850745 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968408982 0 0) 0x7f31e80c61f0 con 0x7f31e80c6670 2024-03-06T07:19:46.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.704+0000 7f31e7fff640 1 -- 172.21.15.117:0/3177850745 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8153a70 con 0x7f31e80c2f20 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.705+0000 7f31e7fff640 1 --2- 172.21.15.117:0/3177850745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c2f20 0x7f31e8152e80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0097a0 tx=0x7f31dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32e8445592535b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.705+0000 7f31e6ffd640 1 -- 172.21.15.117:0/3177850745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0315a0 con 0x7f31e80c2f20 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.705+0000 7f31e6ffd640 1 -- 172.21.15.117:0/3177850745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc037070 con 0x7f31e80c2f20 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.705+0000 7f31e6ffd640 1 -- 172.21.15.117:0/3177850745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc031960 con 0x7f31e80c2f20 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.706+0000 7f31ee3d0640 1 -- 172.21.15.117:0/3177850745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c2f20 msgr2=0x7f31e8152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.706+0000 7f31ee3d0640 1 --2- 172.21.15.117:0/3177850745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c2f20 0x7f31e8152e80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0097a0 tx=0x7f31dc02b840 comp rx=0 tx=0).stop 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.706+0000 7f31ee3d0640 1 -- 172.21.15.117:0/3177850745 shutdown_connections 2024-03-06T07:19:46.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.706+0000 7f31ee3d0640 1 --2- 172.21.15.117:0/3177850745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c2f20 0x7f31e8152e80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.706+0000 7f31ee3d0640 1 -- 172.21.15.117:0/3177850745 >> 172.21.15.117:0/3177850745 conn(0x7f31e80a9540 msgr2=0x7f31e80a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.706+0000 7f31ee3d0640 1 -- 172.21.15.117:0/3177850745 shutdown_connections 2024-03-06T07:19:46.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.706+0000 7f31ee3d0640 1 -- 172.21.15.117:0/3177850745 wait complete. 2024-03-06T07:19:46.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.707+0000 7f31ee3d0640 1 Processor -- start 2024-03-06T07:19:46.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.707+0000 7f31ee3d0640 1 -- start start 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31ee3d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e806c6f0 0x7f31e80c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31ee3d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80c2f20 0x7f31e8072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31ee3d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c6670 0x7f31e8072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31ee3d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80c3b50 con 0x7f31e80c6670 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31ee3d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80c2190 con 0x7f31e806c6f0 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31ee3d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80c3cd0 con 0x7f31e80c2f20 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e806c6f0 0x7f31e80c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e806c6f0 0x7f31e80c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37662/0 (socket says 172.21.15.117:37662) 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80c2f20 0x7f31e8072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e7fff640 1 -- 172.21.15.117:0/2139296394 learned_addr learned my addr 172.21.15.117:0/2139296394 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31ec96d640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c6670 0x7f31e8072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e77fe640 1 -- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e806c6f0 msgr2=0x7f31e80c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e77fe640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e806c6f0 0x7f31e80c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e77fe640 1 -- 172.21.15.117:0/2139296394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c6670 msgr2=0x7f31e8072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e77fe640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c6670 0x7f31e8072570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.708+0000 7f31e77fe640 1 -- 172.21.15.117:0/2139296394 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8072ab0 con 0x7f31e80c2f20 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31ec96d640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c6670 0x7f31e8072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31e7fff640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e806c6f0 0x7f31e80c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31e77fe640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80c2f20 0x7f31e8072030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c450 tx=0x7f31d800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31e4ff9640 1 -- 172.21.15.117:0/2139296394 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8017070 con 0x7f31e80c2f20 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31e4ff9640 1 -- 172.21.15.117:0/2139296394 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d80035f0 con 0x7f31e80c2f20 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31e4ff9640 1 -- 172.21.15.117:0/2139296394 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d80039e0 con 0x7f31e80c2f20 2024-03-06T07:19:46.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e81dd3a0 con 0x7f31e80c2f20 2024-03-06T07:19:46.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e81eb4f0 con 0x7f31e80c2f20 2024-03-06T07:19:46.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31e4ff9640 1 -- 172.21.15.117:0/2139296394 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f31d801a070 con 0x7f31e80c2f20 2024-03-06T07:19:46.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.709+0000 7f31e4ff9640 1 -- 172.21.15.117:0/2139296394 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f31d80129f0 con 0x7f31e80c2f20 2024-03-06T07:19:46.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.710+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e8075300 con 0x7f31e80c2f20 2024-03-06T07:19:46.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.712+0000 7f31e4ff9640 1 -- 172.21.15.117:0/2139296394 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f31e8075300 con 0x7f31e80c2f20 2024-03-06T07:19:46.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.784+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ba4402fa-8b2c-4969-84ca-50316c20363d", "id": 6} v 0) v1 -- 0x7f31e804f330 con 0x7f31e80c2f20 2024-03-06T07:19:46.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.788+0000 7f31e4ff9640 1 -- 172.21.15.117:0/2139296394 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba4402fa-8b2c-4969-84ca-50316c20363d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f31d8021020 con 0x7f31e80c2f20 2024-03-06T07:19:46.790 INFO:teuthology.orchestra.run.smithi117.stdout:6 2024-03-06T07:19:46.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80c2f20 msgr2=0x7f31e8072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80c2f20 0x7f31e8072030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c450 tx=0x7f31d800c910 comp rx=0 tx=0).stop 2024-03-06T07:19:46.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 shutdown_connections 2024-03-06T07:19:46.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e80c6670 0x7f31e8072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80c2f20 0x7f31e8072030 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 --2- 172.21.15.117:0/2139296394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e806c6f0 0x7f31e80c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 >> 172.21.15.117:0/2139296394 conn(0x7f31e80a9540 msgr2=0x7f31e814f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 shutdown_connections 2024-03-06T07:19:46.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.789+0000 7f31ee3d0640 1 -- 172.21.15.117:0/2139296394 wait complete. 2024-03-06T07:19:46.801 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 2f61ed3c-76d7-4537-9573-401d99fec1af 7 2024-03-06T07:19:46.856 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.855+0000 7fe604a2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-06T07:19:46.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.899+0000 7f76a81f7640 1 Processor -- start 2024-03-06T07:19:46.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.900+0000 7f76a81f7640 1 -- start start 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.900+0000 7f76a81f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76a00c3440 0x7f76a00c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.900+0000 7f76a81f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a00c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.900+0000 7f76a81f7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a00c3190 con 0x7f76a00c2db0 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.900+0000 7f76a81f7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00c6200 con 0x7f76a00c3440 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.900+0000 7f76a81f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a006c740 con 0x7f76a0153880 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a6794640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a00c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a6794640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a00c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:34490/0 (socket says 172.21.15.117:34490) 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a6794640 1 -- 172.21.15.117:0/3561967131 learned_addr learned my addr 172.21.15.117:0/3561967131 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a5792640 1 --2- 172.21.15.117:0/3561967131 >> v2:172.21.15.26:3300/0 conn(0x7f76a00c3440 0x7f76a00c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a6794640 1 -- 172.21.15.117:0/3561967131 >> v2:172.21.15.26:3300/0 conn(0x7f76a00c3440 msgr2=0x7f76a00c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:46.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a6794640 1 --2- 172.21.15.117:0/3561967131 >> v2:172.21.15.26:3300/0 conn(0x7f76a00c3440 0x7f76a00c2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a6794640 1 -- 172.21.15.117:0/3561967131 >> v1:172.21.15.26:6789/0 conn(0x7f76a00c2db0 legacy=0x7f76a006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.901+0000 7f76a6794640 1 -- 172.21.15.117:0/3561967131 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00c3990 con 0x7f76a0153880 2024-03-06T07:19:46.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a6794640 1 --2- 172.21.15.117:0/3561967131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a00c6b60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7698009e50 tx=0x7f769802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d0a8bd6cab5b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a4f91640 1 -- 172.21.15.117:0/3561967131 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769802ca70 con 0x7f76a0153880 2024-03-06T07:19:46.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a4f91640 1 -- 172.21.15.117:0/3561967131 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769802cc10 con 0x7f76a0153880 2024-03-06T07:19:46.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a4f91640 1 -- 172.21.15.117:0/3561967131 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76980343f0 con 0x7f76a0153880 2024-03-06T07:19:46.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 -- 172.21.15.117:0/3561967131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 msgr2=0x7f76a00c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 --2- 172.21.15.117:0/3561967131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a00c6b60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7698009e50 tx=0x7f769802bed0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 -- 172.21.15.117:0/3561967131 shutdown_connections 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 --2- 172.21.15.117:0/3561967131 >> v2:172.21.15.26:3300/0 conn(0x7f76a00c3440 0x7f76a00c2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 --2- 172.21.15.117:0/3561967131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a00c6b60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 -- 172.21.15.117:0/3561967131 >> 172.21.15.117:0/3561967131 conn(0x7f76a00a9540 msgr2=0x7f76a00a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 -- 172.21.15.117:0/3561967131 shutdown_connections 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.902+0000 7f76a81f7640 1 -- 172.21.15.117:0/3561967131 wait complete. 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.903+0000 7f76a81f7640 1 Processor -- start 2024-03-06T07:19:46.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a81f7640 1 -- start start 2024-03-06T07:19:46.907 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:46.905+0000 7fe38f83e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a81f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a00c2db0 0x7f76a014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a81f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00c3440 0x7f76a01dc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a81f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a01498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a81f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00c26d0 con 0x7f76a00c2db0 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a81f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a006c3d0 con 0x7f76a00c3440 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a81f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a006c0c0 con 0x7f76a0153880 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a6794640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a01498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a5792640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00c3440 0x7f76a01dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a6794640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a01498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:34506/0 (socket says 172.21.15.117:34506) 2024-03-06T07:19:46.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a6794640 1 -- 172.21.15.117:0/2959524261 learned_addr learned my addr 172.21.15.117:0/2959524261 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.904+0000 7f76a5f93640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a00c2db0 0x7f76a014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a6794640 1 -- 172.21.15.117:0/2959524261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00c3440 msgr2=0x7f76a01dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a6794640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00c3440 0x7f76a01dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a6794640 1 -- 172.21.15.117:0/2959524261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a00c2db0 msgr2=0x7f76a014c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a6794640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a00c2db0 0x7f76a014c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a6794640 1 -- 172.21.15.117:0/2959524261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a014ff90 con 0x7f76a0153880 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a6794640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a01498f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7698009e50 tx=0x7f7698004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76967fc640 1 -- 172.21.15.117:0/2959524261 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698044020 con 0x7f76a0153880 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76967fc640 1 -- 172.21.15.117:0/2959524261 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76980406b0 con 0x7f76a0153880 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a01ee5c0 con 0x7f76a0153880 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76967fc640 1 -- 172.21.15.117:0/2959524261 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698009510 con 0x7f76a0153880 2024-03-06T07:19:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a01eeb80 con 0x7f76a0153880 2024-03-06T07:19:46.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76967fc640 1 -- 172.21.15.117:0/2959524261 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7698047020 con 0x7f76a0153880 2024-03-06T07:19:46.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.905+0000 7f76967fc640 1 -- 172.21.15.117:0/2959524261 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f7698049a60 con 0x7f76a0153880 2024-03-06T07:19:46.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.906+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a004f330 con 0x7f76a0153880 2024-03-06T07:19:46.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.908+0000 7f76967fc640 1 -- 172.21.15.117:0/2959524261 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f769804e030 con 0x7f76a0153880 2024-03-06T07:19:46.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.980+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2f61ed3c-76d7-4537-9573-401d99fec1af", "id": 7} v 0) v1 -- 0x7f76a00c63f0 con 0x7f76a0153880 2024-03-06T07:19:46.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.988+0000 7f76967fc640 1 -- 172.21.15.117:0/2959524261 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f61ed3c-76d7-4537-9573-401d99fec1af", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f769802ca80 con 0x7f76a0153880 2024-03-06T07:19:46.990 INFO:teuthology.orchestra.run.smithi117.stdout:7 2024-03-06T07:19:46.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 msgr2=0x7f76a01498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:46.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a01498f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7698009e50 tx=0x7f7698004a40 comp rx=0 tx=0).stop 2024-03-06T07:19:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 shutdown_connections 2024-03-06T07:19:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a0153880 0x7f76a01498f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00c3440 0x7f76a01dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 --2- 172.21.15.117:0/2959524261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a00c2db0 0x7f76a014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 >> 172.21.15.117:0/2959524261 conn(0x7f76a00a9540 msgr2=0x7f76a01527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 shutdown_connections 2024-03-06T07:19:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-06T07:19:46.989+0000 7f76a81f7640 1 -- 172.21.15.117:0/2959524261 wait complete. 2024-03-06T07:19:46.997 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:46.995+0000 7fe604a2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-06T07:19:47.001 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-06T07:19:47.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-06T07:19:47.004 INFO:tasks.ceph.osd.0:Started 2024-03-06T07:19:47.004 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-06T07:19:47.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-06T07:19:47.006 INFO:tasks.ceph.osd.1:Started 2024-03-06T07:19:47.006 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-06T07:19:47.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-06T07:19:47.008 INFO:tasks.ceph.osd.2:Started 2024-03-06T07:19:47.009 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-06T07:19:47.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-06T07:19:47.011 INFO:tasks.ceph.osd.3:Started 2024-03-06T07:19:47.011 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-06T07:19:47.011 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-06T07:19:47.013 INFO:tasks.ceph.osd.4:Started 2024-03-06T07:19:47.013 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-06T07:19:47.013 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-06T07:19:47.015 INFO:tasks.ceph.osd.5:Started 2024-03-06T07:19:47.015 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-06T07:19:47.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-06T07:19:47.017 INFO:tasks.ceph.osd.6:Started 2024-03-06T07:19:47.017 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-06T07:19:47.018 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-06T07:19:47.020 INFO:tasks.ceph.osd.7:Started 2024-03-06T07:19:47.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:47.059 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.057+0000 7fe38f83e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-06T07:19:47.142 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:47.140+0000 7fe604a2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-06T07:19:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.151+0000 7f1b62e9d640 1 Processor -- start 2024-03-06T07:19:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.151+0000 7f1b62e9d640 1 -- start start 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.151+0000 7f1b62e9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b5c1506b0 0x7f1b5c153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.151+0000 7f1b62e9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0acc00 0x7f1b5c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.151+0000 7f1b62e9d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c070aa0 con 0x7f1b5c156ea0 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.151+0000 7f1b62e9d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c156450 con 0x7f1b5c1506b0 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.151+0000 7f1b62e9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c156a10 con 0x7f1b5c0acc00 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b6143a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0acc00 0x7f1b5c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b60c39640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b5c1506b0 0x7f1b5c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b60c39640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b5c1506b0 0x7f1b5c153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35986/0 (socket says 172.21.15.26:35986) 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b60c39640 1 -- 172.21.15.26:0/636106116 learned_addr learned my addr 172.21.15.26:0/636106116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:47.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b5bfff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1b5c156ea0 0x7f1b5c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52122/0 (socket says 172.21.15.26:52122) 2024-03-06T07:19:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b6143a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0acc00 0x7f1b5c0c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:34276/0 (socket says 172.21.15.26:34276) 2024-03-06T07:19:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b60c39640 1 -- 172.21.15.26:0/636106116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0acc00 msgr2=0x7f1b5c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b60c39640 1 --2- 172.21.15.26:0/636106116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0acc00 0x7f1b5c0c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b60c39640 1 -- 172.21.15.26:0/636106116 >> v1:172.21.15.117:6789/0 conn(0x7f1b5c156ea0 legacy=0x7f1b5c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.152+0000 7f1b60c39640 1 -- 172.21.15.26:0/636106116 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c070d80 con 0x7f1b5c1506b0 2024-03-06T07:19:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b60c39640 1 --2- 172.21.15.26:0/636106116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 0x7f1b5c153990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000dc90 tx=0x7f1b5002fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9e3b8a1c388b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b5b7fe640 1 -- 172.21.15.26:0/636106116 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50030930 con 0x7f1b5c1506b0 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b5b7fe640 1 -- 172.21.15.26:0/636106116 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50030ad0 con 0x7f1b5c1506b0 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b5b7fe640 1 -- 172.21.15.26:0/636106116 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000b450 con 0x7f1b5c1506b0 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 -- 172.21.15.26:0/636106116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 msgr2=0x7f1b5c153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 --2- 172.21.15.26:0/636106116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 0x7f1b5c153990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000dc90 tx=0x7f1b5002fd90 comp rx=0 tx=0).stop 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 -- 172.21.15.26:0/636106116 shutdown_connections 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 --2- 172.21.15.26:0/636106116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0acc00 0x7f1b5c0c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 --2- 172.21.15.26:0/636106116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 0x7f1b5c153990 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 -- 172.21.15.26:0/636106116 >> 172.21.15.26:0/636106116 conn(0x7f1b5c06b1a0 msgr2=0x7f1b5c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 -- 172.21.15.26:0/636106116 shutdown_connections 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.153+0000 7f1b62e9d640 1 -- 172.21.15.26:0/636106116 wait complete. 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.155+0000 7f1b62e9d640 1 Processor -- start 2024-03-06T07:19:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.155+0000 7f1b62e9d640 1 -- start start 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0acc00 0x7f1b5c1ea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 0x7f1b5c0a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c156ea0 0x7f1b5c1e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0acc00 0x7f1b5c1ea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ad1d0 con 0x7f1b5c0acc00 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ad050 con 0x7f1b5c1506b0 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0acc00 0x7f1b5c1ea9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53900/0 (socket says 172.21.15.26:53900) 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 -- 172.21.15.26:0/326281524 learned_addr learned my addr 172.21.15.26:0/326281524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ad350 con 0x7f1b5c156ea0 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b5bfff640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 0x7f1b5c0a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b6143a640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c156ea0 0x7f1b5c1e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 -- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c156ea0 msgr2=0x7f1b5c1e73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c156ea0 0x7f1b5c1e73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 -- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 msgr2=0x7f1b5c0a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 0x7f1b5c0a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 -- 172.21.15.26:0/326281524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c155cc0 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b60c39640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0acc00 0x7f1b5c1ea9b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1b5003a040 tx=0x7f1b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b597fa640 1 -- 172.21.15.26:0/326281524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50047020 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b597fa640 1 -- 172.21.15.26:0/326281524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5000be30 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c155fa0 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b597fa640 1 -- 172.21.15.26:0/326281524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50042670 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.156+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c1e6eb0 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.157+0000 7f1b597fa640 1 -- 172.21.15.26:0/326281524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1b5004a020 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.157+0000 7f1b597fa640 1 -- 172.21.15.26:0/326281524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1b5004cb10 con 0x7f1b5c0acc00 2024-03-06T07:19:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.158+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c0ab640 con 0x7f1b5c0acc00 2024-03-06T07:19:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.160+0000 7f1b597fa640 1 -- 172.21.15.26:0/326281524 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1b5c0ab640 con 0x7f1b5c0acc00 2024-03-06T07:19:47.188 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:47.186+0000 7fe604a2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-06T07:19:47.203 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.201+0000 7fe38f83e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-06T07:19:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.221+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b5c04f330 con 0x7f1b5c0acc00 2024-03-06T07:19:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.222+0000 7f1b597fa640 1 -- 172.21.15.26:0/326281524 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f1b50051030 con 0x7f1b5c0acc00 2024-03-06T07:19:47.224 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:47.224 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:19:46.982231+0000","last_up_change":"0.000000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.222+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0acc00 msgr2=0x7f1b5c1ea9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.222+0000 7f1b62e9d640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0acc00 0x7f1b5c1ea9b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1b5003a040 tx=0x7f1b500046c0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.223+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 shutdown_connections 2024-03-06T07:19:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.223+0000 7f1b62e9d640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c156ea0 0x7f1b5c1e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.223+0000 7f1b62e9d640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c1506b0 0x7f1b5c0a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.223+0000 7f1b62e9d640 1 --2- 172.21.15.26:0/326281524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0acc00 0x7f1b5c1ea9b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.223+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 >> 172.21.15.26:0/326281524 conn(0x7f1b5c06b1a0 msgr2=0x7f1b5c14e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.223+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 shutdown_connections 2024-03-06T07:19:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:47.223+0000 7f1b62e9d640 1 -- 172.21.15.26:0/326281524 wait complete. 2024-03-06T07:19:47.234 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-06T07:19:47.235 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-06T07:19:47.252 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.250+0000 7fe38f83e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-06T07:19:47.300 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:47.298+0000 7fe604a2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:47.370 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.367+0000 7fe38f83e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:47.395 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:47.393+0000 7fe604a2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-06T07:19:47.468 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.466+0000 7fe38f83e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-06T07:19:47.549 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:47.547+0000 7fe604a2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:47.590 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:47.589+0000 7fe604a2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:47.613 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-06T07:19:47.612+0000 7fcdf5624640 -1 Falling back to public interface 2024-03-06T07:19:47.614 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-06T07:19:47.612+0000 7fcdf45d3640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-06T07:19:47.614 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-06T07:19:47.612+0000 7fa5350a0640 -1 Falling back to public interface 2024-03-06T07:19:47.617 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-03-06T07:19:47.616+0000 7fa5698cc640 -1 Falling back to public interface 2024-03-06T07:19:47.618 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-03-06T07:19:47.617+0000 7f97ef424640 -1 Falling back to public interface 2024-03-06T07:19:47.631 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.630+0000 7fe38f83e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:47.633 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-06T07:19:47.631+0000 7fd4b9e11640 -1 Falling back to public interface 2024-03-06T07:19:47.634 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-06T07:19:47.632+0000 7f8f7e1bb640 -1 Falling back to public interface 2024-03-06T07:19:47.634 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-03-06T07:19:47.633+0000 7fb2c8648640 -1 Falling back to public interface 2024-03-06T07:19:47.635 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-06T07:19:47.634+0000 7fd4b8dc0640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6823/0: (98) Address already in use 2024-03-06T07:19:47.638 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-03-06T07:19:47.637+0000 7f17e2e13640 -1 Falling back to public interface 2024-03-06T07:19:47.680 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.678+0000 7fe38f83e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:47.748 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:47.746+0000 7fe604a2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-06T07:19:47.841 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:47.839+0000 7fe38f83e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-06T07:19:48.040 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:48.137 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.128+0000 7f3d24df7640 1 Processor -- start 2024-03-06T07:19:48.137 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.128+0000 7f3d24df7640 1 -- start start 2024-03-06T07:19:48.137 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d24df7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.137 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d24df7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d200c7690 con 0x7f3d20157380 2024-03-06T07:19:48.137 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d24df7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d200c7970 con 0x7f3d200c3cb0 2024-03-06T07:19:48.137 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d24df7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200c7c50 con 0x7f3d200c6c20 2024-03-06T07:19:48.137 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d17fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3d20157380 0x7f3d2006efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53120/0 (socket says 172.21.15.26:53120) 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d17fff640 1 -- 172.21.15.26:0/924003963 learned_addr learned my addr 172.21.15.26:0/924003963 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1ed9d640 1 --2- 172.21.15.26:0/924003963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1dd9b640 1 -- 172.21.15.26:0/924003963 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720683669 0 0) 0x7f3d200c7690 con 0x7f3d20157380 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1dd9b640 1 -- 172.21.15.26:0/924003963 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0c003520 con 0x7f3d20157380 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1dd9b640 1 -- 172.21.15.26:0/924003963 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068149133 0 0) 0x7f3d200c7970 con 0x7f3d200c3cb0 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1dd9b640 1 -- 172.21.15.26:0/924003963 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d200c7690 con 0x7f3d200c3cb0 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1ed9d640 1 -- 172.21.15.26:0/924003963 >> v1:172.21.15.117:6789/0 conn(0x7f3d200c3cb0 legacy=0x7f3d20156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1ed9d640 1 -- 172.21.15.26:0/924003963 >> v1:172.21.15.26:6789/0 conn(0x7f3d20157380 legacy=0x7f3d2006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.129+0000 7f3d1ed9d640 1 -- 172.21.15.26:0/924003963 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d200c7f30 con 0x7f3d200c6c20 2024-03-06T07:19:48.138 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.130+0000 7f3d1ed9d640 1 --2- 172.21.15.26:0/924003963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c8310 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009dc0 tx=0x7f3d1802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911cd52bb36d4177 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.130+0000 7f3d1dd9b640 1 -- 172.21.15.26:0/924003963 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18039070 con 0x7f3d200c6c20 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.130+0000 7f3d1dd9b640 1 -- 172.21.15.26:0/924003963 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1802cbb0 con 0x7f3d200c6c20 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.130+0000 7f3d1dd9b640 1 -- 172.21.15.26:0/924003963 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18034440 con 0x7f3d200c6c20 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.131+0000 7f3d24df7640 1 -- 172.21.15.26:0/924003963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 msgr2=0x7f3d200c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.131+0000 7f3d24df7640 1 --2- 172.21.15.26:0/924003963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c8310 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009dc0 tx=0x7f3d1802be00 comp rx=0 tx=0).stop 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.131+0000 7f3d24df7640 1 -- 172.21.15.26:0/924003963 shutdown_connections 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.131+0000 7f3d24df7640 1 --2- 172.21.15.26:0/924003963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c8310 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.131+0000 7f3d24df7640 1 -- 172.21.15.26:0/924003963 >> 172.21.15.26:0/924003963 conn(0x7f3d200672b0 msgr2=0x7f3d200c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.131+0000 7f3d24df7640 1 -- 172.21.15.26:0/924003963 shutdown_connections 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.131+0000 7f3d24df7640 1 -- 172.21.15.26:0/924003963 wait complete. 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.132+0000 7f3d24df7640 1 Processor -- start 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.132+0000 7f3d24df7640 1 -- start start 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d24df7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d200c3cb0 0x7f3d200c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.139 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d24df7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d24df7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d20157380 0x7f3d200c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d24df7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200c4210 con 0x7f3d200c3cb0 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d24df7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200c4090 con 0x7f3d20157380 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d24df7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2006f6d0 con 0x7f3d200c6c20 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d200c3cb0 0x7f3d200c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d200c3cb0 0x7f3d200c27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53914/0 (socket says 172.21.15.26:53914) 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 -- 172.21.15.26:0/3470439531 learned_addr learned my addr 172.21.15.26:0/3470439531 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1ed9d640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d20157380 0x7f3d200c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 -- 172.21.15.26:0/3470439531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 msgr2=0x7f3d200c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 -- 172.21.15.26:0/3470439531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d20157380 msgr2=0x7f3d200c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d20157380 0x7f3d200c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.140 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 -- 172.21.15.26:0/3470439531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d200c23e0 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.133+0000 7f3d1e59c640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d200c3cb0 0x7f3d200c27d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c500 tx=0x7f3d1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.134+0000 7f3d16ffd640 1 -- 172.21.15.26:0/3470439531 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d100052d0 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.134+0000 7f3d16ffd640 1 -- 172.21.15.26:0/3470439531 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d10005490 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.134+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d200c8c90 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.134+0000 7f3d16ffd640 1 -- 172.21.15.26:0/3470439531 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10012870 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.134+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d20152840 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.134+0000 7f3d16ffd640 1 -- 172.21.15.26:0/3470439531 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3d10012a10 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.134+0000 7f3d16ffd640 1 -- 172.21.15.26:0/3470439531 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3d1001bc90 con 0x7f3d200c3cb0 2024-03-06T07:19:48.141 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.135+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d20152ed0 con 0x7f3d200c3cb0 2024-03-06T07:19:48.142 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.137+0000 7f3d16ffd640 1 -- 172.21.15.26:0/3470439531 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3d20152ed0 con 0x7f3d200c3cb0 2024-03-06T07:19:48.201 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.200+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d2004f330 con 0x7f3d200c3cb0 2024-03-06T07:19:48.202 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.200+0000 7f3d16ffd640 1 -- 172.21.15.26:0/3470439531 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f3d200c23e0 con 0x7f3d200c3cb0 2024-03-06T07:19:48.202 INFO:teuthology.misc.health.smithi026.stdout: 2024-03-06T07:19:48.202 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":10,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:19:46.982231+0000","last_up_change":"0.000000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:48.204 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d200c3cb0 msgr2=0x7f3d200c27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.204 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d200c3cb0 0x7f3d200c27d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c500 tx=0x7f3d1000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.204 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 shutdown_connections 2024-03-06T07:19:48.204 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d20157380 0x7f3d200c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.204 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d200c6c20 0x7f3d200c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.205 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 --2- 172.21.15.26:0/3470439531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d200c3cb0 0x7f3d200c27d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.205 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 >> 172.21.15.26:0/3470439531 conn(0x7f3d200672b0 msgr2=0x7f3d200ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.205 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 shutdown_connections 2024-03-06T07:19:48.205 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:48.201+0000 7f3d24df7640 1 -- 172.21.15.26:0/3470439531 wait complete. 2024-03-06T07:19:48.212 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-06T07:19:48.564 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:48.562+0000 7fe604a2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-06T07:19:48.684 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:48.682+0000 7fe38f83e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-06T07:19:48.840 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:48.838+0000 7fe604a2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-06T07:19:48.891 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:48.889+0000 7fe604a2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-06T07:19:48.972 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:48.969+0000 7fe38f83e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-06T07:19:49.007 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:49.005+0000 7fe604a2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-06T07:19:49.026 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:49.023+0000 7fe38f83e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-06T07:19:49.149 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-03-06T07:19:49.147+0000 7fe38f83e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-06T07:19:50.039 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:50.037+0000 7fe5a4f1a640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-06T07:19:50.039 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-06T07:19:50.037+0000 7fe5a4f1a640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-06T07:19:51.076 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-06T07:19:51.074+0000 7fcdf5624640 -1 osd.4 0 log_to_monitors true 2024-03-06T07:19:51.093 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-03-06T07:19:51.091+0000 7fa5698cc640 -1 osd.0 0 log_to_monitors true 2024-03-06T07:19:51.108 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-03-06T07:19:51.106+0000 7f97ef424640 -1 osd.1 0 log_to_monitors true 2024-03-06T07:19:51.132 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-06T07:19:51.130+0000 7fa5350a0640 -1 osd.5 0 log_to_monitors true 2024-03-06T07:19:51.135 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-06T07:19:51.133+0000 7fd4b9e11640 -1 osd.6 0 log_to_monitors true 2024-03-06T07:19:51.135 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-06T07:19:51.133+0000 7f8f7e1bb640 -1 osd.7 0 log_to_monitors true 2024-03-06T07:19:51.155 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-03-06T07:19:51.153+0000 7f17e2e13640 -1 osd.2 0 log_to_monitors true 2024-03-06T07:19:51.160 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-03-06T07:19:51.157+0000 7fb2c8648640 -1 osd.3 0 log_to_monitors true 2024-03-06T07:19:53.987 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-06T07:19:53.985+0000 7f8f7a151640 -1 osd.7 0 waiting for initial osdmap 2024-03-06T07:19:53.993 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-06T07:19:53.991+0000 7fd4b5da7640 -1 osd.6 0 waiting for initial osdmap 2024-03-06T07:19:53.996 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-06T07:19:53.995+0000 7f8f71f71640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:54.005 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-06T07:19:54.004+0000 7fd4adbc7640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:54.010 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-06T07:19:54.008+0000 7fcdf15ba640 -1 osd.4 0 waiting for initial osdmap 2024-03-06T07:19:54.022 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-06T07:19:54.021+0000 7fcde93da640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:54.043 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-06T07:19:54.041+0000 7fa531036640 -1 osd.5 0 waiting for initial osdmap 2024-03-06T07:19:54.044 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-03-06T07:19:54.041+0000 7f17de7c0640 -1 osd.2 0 waiting for initial osdmap 2024-03-06T07:19:54.047 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-03-06T07:19:54.045+0000 7f97eb3ba640 -1 osd.1 0 waiting for initial osdmap 2024-03-06T07:19:54.048 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-03-06T07:19:54.046+0000 7fb2c45de640 -1 osd.3 0 waiting for initial osdmap 2024-03-06T07:19:54.055 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-03-06T07:19:54.053+0000 7fa565862640 -1 osd.0 0 waiting for initial osdmap 2024-03-06T07:19:54.056 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-06T07:19:54.055+0000 7fa528e56640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:54.058 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-03-06T07:19:54.055+0000 7fa55d682640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:54.058 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-03-06T07:19:54.055+0000 7f17d65e0640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:54.061 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-03-06T07:19:54.059+0000 7f97e31da640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:54.062 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-03-06T07:19:54.060+0000 7fb2bc3fe640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:55.019 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:55.121 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.118+0000 7f335dbe9640 1 Processor -- start 2024-03-06T07:19:55.121 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.118+0000 7f335dbe9640 1 -- start start 2024-03-06T07:19:55.123 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f335dbe9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33580afe60 0x7f33580c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.123 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f335dbe9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f335806c8c0 0x7f33581551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.123 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f335dbe9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335806cd40 con 0x7f33580c8bc0 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f335dbe9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33580c55b0 con 0x7f33580afe60 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f335dbe9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3358159290 con 0x7f335806c8c0 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f33577fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33580afe60 0x7f33580c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f33577fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33580afe60 0x7f33580c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53970/0 (socket says 172.21.15.26:53970) 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f33577fe640 1 -- 172.21.15.26:0/1696283230 learned_addr learned my addr 172.21.15.26:0/1696283230 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f3356ffd640 1 --2- 172.21.15.26:0/1696283230 >> v2:172.21.15.117:3300/0 conn(0x7f335806c8c0 0x7f33581551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f33577fe640 1 -- 172.21.15.26:0/1696283230 >> v2:172.21.15.117:3300/0 conn(0x7f335806c8c0 msgr2=0x7f33581551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f33577fe640 1 --2- 172.21.15.26:0/1696283230 >> v2:172.21.15.117:3300/0 conn(0x7f335806c8c0 0x7f33581551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f33577fe640 1 -- 172.21.15.26:0/1696283230 >> v1:172.21.15.117:6789/0 conn(0x7f33580c8bc0 legacy=0x7f3358155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.119+0000 7f33577fe640 1 -- 172.21.15.26:0/1696283230 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3358159570 con 0x7f33580afe60 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.120+0000 7f33577fe640 1 --2- 172.21.15.26:0/1696283230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f33580c75e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33580c1c90 tx=0x7f334802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e1932b238756b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.120+0000 7f33567fc640 1 -- 172.21.15.26:0/1696283230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334802e070 con 0x7f33580afe60 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.120+0000 7f33567fc640 1 -- 172.21.15.26:0/1696283230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3348037070 con 0x7f33580afe60 2024-03-06T07:19:55.124 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.120+0000 7f33567fc640 1 -- 172.21.15.26:0/1696283230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348031a00 con 0x7f33580afe60 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.120+0000 7f335dbe9640 1 -- 172.21.15.26:0/1696283230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 msgr2=0x7f33580c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.120+0000 7f335dbe9640 1 --2- 172.21.15.26:0/1696283230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f33580c75e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33580c1c90 tx=0x7f334802b880 comp rx=0 tx=0).stop 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.121+0000 7f335dbe9640 1 -- 172.21.15.26:0/1696283230 shutdown_connections 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.121+0000 7f335dbe9640 1 --2- 172.21.15.26:0/1696283230 >> v2:172.21.15.117:3300/0 conn(0x7f335806c8c0 0x7f33581551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.121+0000 7f335dbe9640 1 --2- 172.21.15.26:0/1696283230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f33580c75e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.121+0000 7f335dbe9640 1 -- 172.21.15.26:0/1696283230 >> 172.21.15.26:0/1696283230 conn(0x7f33580ac740 msgr2=0x7f33580acb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.121+0000 7f335dbe9640 1 -- 172.21.15.26:0/1696283230 shutdown_connections 2024-03-06T07:19:55.125 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.121+0000 7f335dbe9640 1 -- 172.21.15.26:0/1696283230 wait complete. 2024-03-06T07:19:55.128 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.122+0000 7f335dbe9640 1 Processor -- start 2024-03-06T07:19:55.128 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.122+0000 7f335dbe9640 1 -- start start 2024-03-06T07:19:55.128 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f335806c8c0 0x7f335814f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.128 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f335814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.128 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33580c8bc0 0x7f33581ee270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.128 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580b0240 con 0x7f33580afe60 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580c5d20 con 0x7f33580c8bc0 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33580b03c0 con 0x7f335806c8c0 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f335814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f335814f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53974/0 (socket says 172.21.15.26:53974) 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 -- 172.21.15.26:0/2157254098 learned_addr learned my addr 172.21.15.26:0/2157254098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3357fff640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33580c8bc0 0x7f33581ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f33577fe640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f335806c8c0 0x7f335814f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 -- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f335806c8c0 msgr2=0x7f335814f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f335806c8c0 0x7f335814f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 -- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33580c8bc0 msgr2=0x7f33581ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33580c8bc0 0x7f33581ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 -- 172.21.15.26:0/2157254098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33581ec0e0 con 0x7f33580afe60 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f33577fe640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f335806c8c0 0x7f335814f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:55.129 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f3356ffd640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f335814f6d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e8a0 tx=0x7f334c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f333ffff640 1 -- 172.21.15.26:0/2157254098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c017070 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f333ffff640 1 -- 172.21.15.26:0/2157254098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c00ee90 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33580aead0 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f333ffff640 1 -- 172.21.15.26:0/2157254098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c00cb00 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.123+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33580c4750 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.125+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335804f330 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.125+0000 7f333ffff640 1 -- 172.21.15.26:0/2157254098 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f334c01a070 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.125+0000 7f333ffff640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b440 0x7f334406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.126+0000 7f333ffff640 1 -- 172.21.15.26:0/2157254098 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f334c0899c0 con 0x7f33580afe60 2024-03-06T07:19:55.130 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.126+0000 7f33577fe640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b440 0x7f334406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.131 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.126+0000 7f33577fe640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b440 0x7f334406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3348036000 tx=0x7f334803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.132 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.130+0000 7f333ffff640 1 -- 172.21.15.26:0/2157254098 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f334c0587a0 con 0x7f33580afe60 2024-03-06T07:19:55.315 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.312+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33580af930 con 0x7f33580afe60 2024-03-06T07:19:55.315 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.313+0000 7f333ffff640 1 -- 172.21.15.26:0/2157254098 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10921 (secure 0 0 0) 0x7f334c05c840 con 0x7f33580afe60 2024-03-06T07:19:55.316 INFO:teuthology.misc.health.smithi026.stdout: 2024-03-06T07:19:55.316 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":13,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:19:55.000525+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b440 msgr2=0x7f334406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b440 0x7f334406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3348036000 tx=0x7f334803d040 comp rx=0 tx=0).stop 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 msgr2=0x7f335814f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f335814f6d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e8a0 tx=0x7f334c00ed60 comp rx=0 tx=0).stop 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 shutdown_connections 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b440 0x7f334406d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33580c8bc0 0x7f33581ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33580afe60 0x7f335814f6d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 --2- 172.21.15.26:0/2157254098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f335806c8c0 0x7f335814f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.318 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 >> 172.21.15.26:0/2157254098 conn(0x7f33580ac740 msgr2=0x7f33580abc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:55.319 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 shutdown_connections 2024-03-06T07:19:55.319 INFO:teuthology.misc.health.smithi026.stderr:2024-03-06T07:19:55.315+0000 7f335dbe9640 1 -- 172.21.15.26:0/2157254098 wait complete. 2024-03-06T07:19:55.327 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-06T07:19:55.327 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-06T07:19:55.328 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:19:55.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:19:55.330 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:19:55.331 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:19:55.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:19:55.333 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:19:55.333 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:19:55.333 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:19:55.337 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:19:55.337 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:19:55.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:19:55.339 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:19:55.339 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-06T07:19:55.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:55.408 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:19:55.408 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:19:55.423 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:19:55.423 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:19:55.424 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:19:55.422+0000 7fea3dc42640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-06T07:19:55.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.444+0000 7f198cb9a640 1 Processor -- start 2024-03-06T07:19:55.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.444+0000 7f198cb9a640 1 -- start start 2024-03-06T07:19:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f198cb9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1988152920 0x7f1988155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f198cb9a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f19880c2190 0x7f19880c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f198cb9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19880c1280 0x7f19880c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f198cb9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1988156160 con 0x7f19880c1280 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f198cb9a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198806afc0 con 0x7f19880c2190 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f198cb9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198806b2a0 con 0x7f1988152920 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19880c1280 0x7f19880c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19880c1280 0x7f19880c1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54028/0 (socket says 172.21.15.26:54028) 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 -- 172.21.15.26:0/3170030246 learned_addr learned my addr 172.21.15.26:0/3170030246 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f198659c640 1 --2- 172.21.15.26:0/3170030246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1988152920 0x7f1988155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1985d9b640 1 --2- 172.21.15.26:0/3170030246 >> v2:172.21.15.117:3300/0 conn(0x7f19880c2190 0x7f19880c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 -- 172.21.15.26:0/3170030246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1988152920 msgr2=0x7f1988155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 --2- 172.21.15.26:0/3170030246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1988152920 0x7f1988155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 -- 172.21.15.26:0/3170030246 >> v2:172.21.15.117:3300/0 conn(0x7f19880c2190 msgr2=0x7f19880c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 --2- 172.21.15.26:0/3170030246 >> v2:172.21.15.117:3300/0 conn(0x7f19880c2190 0x7f19880c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.445+0000 7f1986d9d640 1 -- 172.21.15.26:0/3170030246 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198806b580 con 0x7f19880c1280 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.446+0000 7f1986d9d640 1 --2- 172.21.15.26:0/3170030246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19880c1280 0x7f19880c1710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1988070ac0 tx=0x7f198002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e741e1718f8d550 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.446+0000 7f198559a640 1 -- 172.21.15.26:0/3170030246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198002ca70 con 0x7f19880c1280 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.446+0000 7f198559a640 1 -- 172.21.15.26:0/3170030246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198002cc10 con 0x7f19880c1280 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.446+0000 7f198559a640 1 -- 172.21.15.26:0/3170030246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800343f0 con 0x7f19880c1280 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 -- 172.21.15.26:0/3170030246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19880c1280 msgr2=0x7f19880c1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 --2- 172.21.15.26:0/3170030246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19880c1280 0x7f19880c1710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1988070ac0 tx=0x7f198002bed0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 -- 172.21.15.26:0/3170030246 shutdown_connections 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 --2- 172.21.15.26:0/3170030246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19880c1280 0x7f19880c1710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 --2- 172.21.15.26:0/3170030246 >> v2:172.21.15.117:3300/0 conn(0x7f19880c2190 0x7f19880c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 --2- 172.21.15.26:0/3170030246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1988152920 0x7f1988155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 -- 172.21.15.26:0/3170030246 >> 172.21.15.26:0/3170030246 conn(0x7f19880c4290 msgr2=0x7f19880c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:55.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 -- 172.21.15.26:0/3170030246 shutdown_connections 2024-03-06T07:19:55.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.447+0000 7f198cb9a640 1 -- 172.21.15.26:0/3170030246 wait complete. 2024-03-06T07:19:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.448+0000 7f198cb9a640 1 Processor -- start 2024-03-06T07:19:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.448+0000 7f198cb9a640 1 -- start start 2024-03-06T07:19:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198cb9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880c1280 0x7f19880aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198cb9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880c2190 0x7f19881ec250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198cb9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1988152920 0x7f1988158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198cb9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880c1c50 con 0x7f1988152920 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198cb9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880c0d60 con 0x7f19880c2190 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198cb9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880c1dd0 con 0x7f19880c1280 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1988152920 0x7f1988158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1988152920 0x7f1988158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54030/0 (socket says 172.21.15.26:54030) 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 -- 172.21.15.26:0/279203972 learned_addr learned my addr 172.21.15.26:0/279203972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198659c640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880c1280 0x7f19880aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1985d9b640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880c2190 0x7f19881ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 -- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880c1280 msgr2=0x7f19880aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880c1280 0x7f19880aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 -- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880c2190 msgr2=0x7f19881ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880c2190 0x7f19881ec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 -- 172.21.15.26:0/279203972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988156da0 con 0x7f1988152920 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1986d9d640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1988152920 0x7f1988158570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1980036040 tx=0x7f1980004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1976ffd640 1 -- 172.21.15.26:0/279203972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980043020 con 0x7f1988152920 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1976ffd640 1 -- 172.21.15.26:0/279203972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198003e450 con 0x7f1988152920 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19881567e0 con 0x7f1988152920 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.449+0000 7f1976ffd640 1 -- 172.21.15.26:0/279203972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980009510 con 0x7f1988152920 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.450+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19880ac230 con 0x7f1988152920 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.450+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19880c3d60 con 0x7f1988152920 2024-03-06T07:19:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.451+0000 7f1976ffd640 1 -- 172.21.15.26:0/279203972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f1980046020 con 0x7f1988152920 2024-03-06T07:19:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.451+0000 7f1976ffd640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f196806b440 0x7f196806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.451+0000 7f1976ffd640 1 -- 172.21.15.26:0/279203972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f19800ac1e0 con 0x7f1988152920 2024-03-06T07:19:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.452+0000 7f198659c640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f196806b440 0x7f196806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.452+0000 7f198659c640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f196806b440 0x7f196806d900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ac0 tx=0x7f1978009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.456+0000 7f1976ffd640 1 -- 172.21.15.26:0/279203972 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f19880c3d60 con 0x7f1988152920 2024-03-06T07:19:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.637+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f198804f330 con 0x7f1988152920 2024-03-06T07:19:55.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.637+0000 7f1976ffd640 1 -- 172.21.15.26:0/279203972 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10921 (secure 0 0 0) 0x7f198004d030 con 0x7f1988152920 2024-03-06T07:19:55.640 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:55.640 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:19:55.000525+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:55.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f196806b440 msgr2=0x7f196806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f196806b440 0x7f196806d900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ac0 tx=0x7f1978009190 comp rx=0 tx=0).stop 2024-03-06T07:19:55.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1988152920 msgr2=0x7f1988158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1988152920 0x7f1988158570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1980036040 tx=0x7f1980004a40 comp rx=0 tx=0).stop 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 shutdown_connections 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f196806b440 0x7f196806d900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1988152920 0x7f1988158570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880c2190 0x7f19881ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 --2- 172.21.15.26:0/279203972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880c1280 0x7f19880aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 >> 172.21.15.26:0/279203972 conn(0x7f19880c4290 msgr2=0x7f1988151150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 shutdown_connections 2024-03-06T07:19:55.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.639+0000 7f198cb9a640 1 -- 172.21.15.26:0/279203972 wait complete. 2024-03-06T07:19:55.652 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-03-06T07:19:55.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:19:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.752+0000 7f56c309a640 1 Processor -- start 2024-03-06T07:19:55.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.752+0000 7f56c309a640 1 -- start start 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.752+0000 7f56c309a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56bc0c5330 0x7f56bc0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.752+0000 7f56c309a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56bc0c88b0 0x7f56bc1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c309a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc155110 con 0x7f56bc0c7dd0 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c309a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc1559e0 con 0x7f56bc0c5330 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c309a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc0c8f40 con 0x7f56bc0c88b0 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c0e36640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56bc0c5330 0x7f56bc0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c0e36640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56bc0c5330 0x7f56bc0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54036/0 (socket says 172.21.15.26:54036) 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c0e36640 1 -- 172.21.15.26:0/694744263 learned_addr learned my addr 172.21.15.26:0/694744263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56b3fff640 1 --2- 172.21.15.26:0/694744263 >> v2:172.21.15.117:3300/0 conn(0x7f56bc0c88b0 0x7f56bc1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c0e36640 1 -- 172.21.15.26:0/694744263 >> v2:172.21.15.117:3300/0 conn(0x7f56bc0c88b0 msgr2=0x7f56bc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c0e36640 1 --2- 172.21.15.26:0/694744263 >> v2:172.21.15.117:3300/0 conn(0x7f56bc0c88b0 0x7f56bc1586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c0e36640 1 -- 172.21.15.26:0/694744263 >> v1:172.21.15.117:6789/0 conn(0x7f56bc0c7dd0 legacy=0x7f56bc06b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.753+0000 7f56c0e36640 1 -- 172.21.15.26:0/694744263 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc155390 con 0x7f56bc0c5330 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c0e36640 1 --2- 172.21.15.26:0/694744263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0c5710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56bc156240 tx=0x7f56b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9012af493545322 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56b37fe640 1 -- 172.21.15.26:0/694744263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8035020 con 0x7f56bc0c5330 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56b37fe640 1 -- 172.21.15.26:0/694744263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b8031870 con 0x7f56bc0c5330 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56b37fe640 1 -- 172.21.15.26:0/694744263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8031c40 con 0x7f56bc0c5330 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 -- 172.21.15.26:0/694744263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 msgr2=0x7f56bc0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 --2- 172.21.15.26:0/694744263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0c5710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56bc156240 tx=0x7f56b802b910 comp rx=0 tx=0).stop 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 -- 172.21.15.26:0/694744263 shutdown_connections 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 --2- 172.21.15.26:0/694744263 >> v2:172.21.15.117:3300/0 conn(0x7f56bc0c88b0 0x7f56bc1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 --2- 172.21.15.26:0/694744263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0c5710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 -- 172.21.15.26:0/694744263 >> 172.21.15.26:0/694744263 conn(0x7f56bc0ac560 msgr2=0x7f56bc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 -- 172.21.15.26:0/694744263 shutdown_connections 2024-03-06T07:19:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.754+0000 7f56c309a640 1 -- 172.21.15.26:0/694744263 wait complete. 2024-03-06T07:19:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.755+0000 7f56c309a640 1 Processor -- start 2024-03-06T07:19:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c309a640 1 -- start start 2024-03-06T07:19:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c309a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0be760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c309a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0c7dd0 0x7f56bc0ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c309a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0c88b0 0x7f56bc1eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c309a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc1529d0 con 0x7f56bc0c5330 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c309a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc06d090 con 0x7f56bc0c88b0 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c309a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc152b50 con 0x7f56bc0c7dd0 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c0e36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c0e36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0be760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54042/0 (socket says 172.21.15.26:54042) 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.756+0000 7f56c0e36640 1 -- 172.21.15.26:0/2477410084 learned_addr learned my addr 172.21.15.26:0/2477410084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56b3fff640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0c7dd0 0x7f56bc0ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c1637640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0c88b0 0x7f56bc1eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c0e36640 1 -- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0c7dd0 msgr2=0x7f56bc0ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c0e36640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0c7dd0 0x7f56bc0ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c0e36640 1 -- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0c88b0 msgr2=0x7f56bc1eaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c0e36640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0c88b0 0x7f56bc1eaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c0e36640 1 -- 172.21.15.26:0/2477410084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc1ea550 con 0x7f56bc0c5330 2024-03-06T07:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56b3fff640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0c7dd0 0x7f56bc0ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c1637640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0c88b0 0x7f56bc1eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c0e36640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0be760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f56b802b950 tx=0x7f56b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56b17fa640 1 -- 172.21.15.26:0/2477410084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8040070 con 0x7f56bc0c5330 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56b17fa640 1 -- 172.21.15.26:0/2477410084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b8030d00 con 0x7f56bc0c5330 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc154160 con 0x7f56bc0c5330 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56b17fa640 1 -- 172.21.15.26:0/2477410084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8035040 con 0x7f56bc0c5330 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.757+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc0c48c0 con 0x7f56bc0c5330 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.758+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc04f330 con 0x7f56bc0c5330 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.758+0000 7f56b17fa640 1 -- 172.21.15.26:0/2477410084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f56b804b060 con 0x7f56bc0c5330 2024-03-06T07:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.759+0000 7f56b17fa640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56a806b440 0x7f56a806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.759+0000 7f56b17fa640 1 -- 172.21.15.26:0/2477410084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f56b80ab990 con 0x7f56bc0c5330 2024-03-06T07:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.759+0000 7f56b3fff640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56a806b440 0x7f56a806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.760+0000 7f56b3fff640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56a806b440 0x7f56a806d900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56ac006fd0 tx=0x7f56ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.763+0000 7f56b17fa640 1 -- 172.21.15.26:0/2477410084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f56b807ae50 con 0x7f56bc0c5330 2024-03-06T07:19:55.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.983+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56bc0c4390 con 0x7f56bc0c5330 2024-03-06T07:19:55.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.983+0000 7f56b17fa640 1 -- 172.21.15.26:0/2477410084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7f56b807a750 con 0x7f56bc0c5330 2024-03-06T07:19:55.986 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:55.986 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-06T07:19:55.986 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-03-06T07:19:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56a806b440 msgr2=0x7f56a806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56a806b440 0x7f56a806d900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56ac006fd0 tx=0x7f56ac008040 comp rx=0 tx=0).stop 2024-03-06T07:19:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 msgr2=0x7f56bc0be760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0be760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f56b802b950 tx=0x7f56b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 shutdown_connections 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56a806b440 0x7f56a806d900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0c88b0 0x7f56bc1eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0c7dd0 0x7f56bc0ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 --2- 172.21.15.26:0/2477410084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56bc0c5330 0x7f56bc0be760 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 >> 172.21.15.26:0/2477410084 conn(0x7f56bc0ac560 msgr2=0x7f56bc0c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 shutdown_connections 2024-03-06T07:19:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:55.985+0000 7f56c309a640 1 -- 172.21.15.26:0/2477410084 wait complete. 2024-03-06T07:19:58.000 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:19:58.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.102+0000 7f0b6ee81640 1 Processor -- start 2024-03-06T07:19:58.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.102+0000 7f0b6ee81640 1 -- start start 2024-03-06T07:19:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6ee81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b6806afc0 0x7f0b6806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6ee81640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b681536c0 0x7f0b6815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6ee81640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b680a70f0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6ee81640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6806b830 con 0x7f0b6806afc0 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6ee81640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b68150a20 con 0x7f0b681536c0 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6cc1d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b6806afc0 0x7f0b6806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6cc1d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b6806afc0 0x7f0b6806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42366/0 (socket says 172.21.15.26:42366) 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6cc1d640 1 -- 172.21.15.26:0/3212901664 learned_addr learned my addr 172.21.15.26:0/3212901664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.103+0000 7f0b6d41e640 1 --2- 172.21.15.26:0/3212901664 >> v2:172.21.15.117:3300/0 conn(0x7f0b681536c0 0x7f0b6815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.104+0000 7f0b6d41e640 1 -- 172.21.15.26:0/3212901664 >> v2:172.21.15.26:3300/0 conn(0x7f0b6806afc0 msgr2=0x7f0b6806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.104+0000 7f0b6d41e640 1 --2- 172.21.15.26:0/3212901664 >> v2:172.21.15.26:3300/0 conn(0x7f0b6806afc0 0x7f0b6806f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.104+0000 7f0b6d41e640 1 -- 172.21.15.26:0/3212901664 >> v1:172.21.15.117:6789/0 conn(0x7f0b6806b3a0 legacy=0x7f0b68152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.104+0000 7f0b677fe640 1 -- 172.21.15.26:0/3212901664 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49767984 0 0) 0x7f0b680a70f0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.104+0000 7f0b6d41e640 1 -- 172.21.15.26:0/3212901664 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b681567f0 con 0x7f0b681536c0 2024-03-06T07:19:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.104+0000 7f0b6cc1d640 1 --2- 172.21.15.26:0/3212901664 >> v2:172.21.15.26:3300/0 conn(0x7f0b6806afc0 0x7f0b6806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.105+0000 7f0b6d41e640 1 --2- 172.21.15.26:0/3212901664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b6815d930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009d90 tx=0x7f0b6002bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b6a558c969c39b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.105+0000 7f0b677fe640 1 -- 172.21.15.26:0/3212901664 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60039070 con 0x7f0b681536c0 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.105+0000 7f0b677fe640 1 -- 172.21.15.26:0/3212901664 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6002cb40 con 0x7f0b681536c0 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.105+0000 7f0b677fe640 1 -- 172.21.15.26:0/3212901664 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60034440 con 0x7f0b681536c0 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3212901664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 msgr2=0x7f0b6815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3212901664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b6815d930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009d90 tx=0x7f0b6002bd90 comp rx=0 tx=0).stop 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3212901664 shutdown_connections 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3212901664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b6815d930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3212901664 >> v2:172.21.15.26:3300/0 conn(0x7f0b6806afc0 0x7f0b6806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3212901664 >> 172.21.15.26:0/3212901664 conn(0x7f0b680c4290 msgr2=0x7f0b680c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3212901664 shutdown_connections 2024-03-06T07:19:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.106+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3212901664 wait complete. 2024-03-06T07:19:58.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.107+0000 7f0b6ee81640 1 Processor -- start 2024-03-06T07:19:58.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.107+0000 7f0b6ee81640 1 -- start start 2024-03-06T07:19:58.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6ee81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806afc0 0x7f0b680bc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6ee81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806b3a0 0x7f0b680abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6ee81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b681e9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6ee81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68156db0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6ee81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681565a0 con 0x7f0b681536c0 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6ee81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680c7360 con 0x7f0b6806afc0 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806b3a0 0x7f0b680abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806b3a0 0x7f0b680abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42376/0 (socket says 172.21.15.26:42376) 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 -- 172.21.15.26:0/3738473592 learned_addr learned my addr 172.21.15.26:0/3738473592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6d41e640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b681e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6cc1d640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806afc0 0x7f0b680bc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 -- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806afc0 msgr2=0x7f0b680bc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806afc0 0x7f0b680bc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 -- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 msgr2=0x7f0b681e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b681e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 -- 172.21.15.26:0/3738473592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b681de020 con 0x7f0b6806b3a0 2024-03-06T07:19:58.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b6d41e640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b681e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:58.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.108+0000 7f0b67fff640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806b3a0 0x7f0b680abcf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c4e0 tx=0x7f0b5400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.181+0000 7f0b657fa640 1 -- 172.21.15.26:0/3738473592 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54017070 con 0x7f0b6806b3a0 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.181+0000 7f0b657fa640 1 -- 172.21.15.26:0/3738473592 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b540035f0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.181+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b681ecef0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.181+0000 7f0b657fa640 1 -- 172.21.15.26:0/3738473592 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540039e0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.181+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b681ddd40 con 0x7f0b6806b3a0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.182+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6804f330 con 0x7f0b6806b3a0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.182+0000 7f0b657fa640 1 -- 172.21.15.26:0/3738473592 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f0b5401a070 con 0x7f0b6806b3a0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.183+0000 7f0b657fa640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4c06b440 0x7f0b4c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.183+0000 7f0b657fa640 1 -- 172.21.15.26:0/3738473592 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0b54088fc0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.183+0000 7f0b6cc1d640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4c06b440 0x7f0b4c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.183+0000 7f0b6cc1d640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4c06b440 0x7f0b4c06d900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006ca0 tx=0x7f0b5c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.187+0000 7f0b657fa640 1 -- 172.21.15.26:0/3738473592 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0b54057cb0 con 0x7f0b6806b3a0 2024-03-06T07:19:58.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.406+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b68068460 con 0x7f0b6806b3a0 2024-03-06T07:19:58.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.407+0000 7f0b657fa640 1 -- 172.21.15.26:0/3738473592 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f0b5405bd50 con 0x7f0b6806b3a0 2024-03-06T07:19:58.410 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-06T07:19:58.410 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-03-06T07:19:58.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4c06b440 msgr2=0x7f0b4c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4c06b440 0x7f0b4c06d900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006ca0 tx=0x7f0b5c009240 comp rx=0 tx=0).stop 2024-03-06T07:19:58.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806b3a0 msgr2=0x7f0b680abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806b3a0 0x7f0b680abcf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c4e0 tx=0x7f0b5400c9a0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 shutdown_connections 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4c06b440 0x7f0b4c06d900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b681536c0 0x7f0b681e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806b3a0 0x7f0b680abcf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 --2- 172.21.15.26:0/3738473592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806afc0 0x7f0b680bc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 >> 172.21.15.26:0/3738473592 conn(0x7f0b680c4290 msgr2=0x7f0b68152df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 shutdown_connections 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.409+0000 7f0b6ee81640 1 -- 172.21.15.26:0/3738473592 wait complete. 2024-03-06T07:19:58.422 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.521+0000 7fc3f2e33640 1 Processor -- start 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.521+0000 7fc3f2e33640 1 -- start start 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f2e33640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc3ec152920 0x7fc3ec155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f2e33640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec0c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f2e33640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3ec0c1280 0x7fc3ec0c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f2e33640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec156160 con 0x7fc3ec0c1280 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f2e33640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec06afc0 con 0x7fc3ec152920 2024-03-06T07:19:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f2e33640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec06b2a0 con 0x7fc3ec0c2190 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f13d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3ec0c1280 0x7fc3ec0c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f13d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3ec0c1280 0x7fc3ec0c1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42378/0 (socket says 172.21.15.26:42378) 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f13d0640 1 -- 172.21.15.26:0/461421233 learned_addr learned my addr 172.21.15.26:0/461421233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3ebfff640 1 --2- 172.21.15.26:0/461421233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec0c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.522+0000 7fc3f0bcf640 1 --2- 172.21.15.26:0/461421233 >> v2:172.21.15.117:3300/0 conn(0x7fc3ec152920 0x7fc3ec155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3f13d0640 1 -- 172.21.15.26:0/461421233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 msgr2=0x7fc3ec0c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3f13d0640 1 --2- 172.21.15.26:0/461421233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec0c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3f13d0640 1 -- 172.21.15.26:0/461421233 >> v2:172.21.15.117:3300/0 conn(0x7fc3ec152920 msgr2=0x7fc3ec155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3f13d0640 1 --2- 172.21.15.26:0/461421233 >> v2:172.21.15.117:3300/0 conn(0x7fc3ec152920 0x7fc3ec155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3f13d0640 1 -- 172.21.15.26:0/461421233 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec06b580 con 0x7fc3ec0c1280 2024-03-06T07:19:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3f0bcf640 1 --2- 172.21.15.26:0/461421233 >> v2:172.21.15.117:3300/0 conn(0x7fc3ec152920 0x7fc3ec155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3ebfff640 1 --2- 172.21.15.26:0/461421233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec0c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3f13d0640 1 --2- 172.21.15.26:0/461421233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec0c1280 0x7fc3ec0c1710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec070a90 tx=0x7fc3e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8b76cc3e1c728d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3eb7fe640 1 -- 172.21.15.26:0/461421233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e402ca70 con 0x7fc3ec0c1280 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3eb7fe640 1 -- 172.21.15.26:0/461421233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e402cc10 con 0x7fc3ec0c1280 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.523+0000 7fc3eb7fe640 1 -- 172.21.15.26:0/461421233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40343f0 con 0x7fc3ec0c1280 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 -- 172.21.15.26:0/461421233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec0c1280 msgr2=0x7fc3ec0c1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/461421233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec0c1280 0x7fc3ec0c1710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec070a90 tx=0x7fc3e402bed0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 -- 172.21.15.26:0/461421233 shutdown_connections 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/461421233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec0c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/461421233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec0c1280 0x7fc3ec0c1710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/461421233 >> v2:172.21.15.117:3300/0 conn(0x7fc3ec152920 0x7fc3ec155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 -- 172.21.15.26:0/461421233 >> 172.21.15.26:0/461421233 conn(0x7fc3ec0c4290 msgr2=0x7fc3ec0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 -- 172.21.15.26:0/461421233 shutdown_connections 2024-03-06T07:19:58.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.524+0000 7fc3f2e33640 1 -- 172.21.15.26:0/461421233 wait complete. 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.525+0000 7fc3f2e33640 1 Processor -- start 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.525+0000 7fc3f2e33640 1 -- start start 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f2e33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3ec0c1280 0x7fc3ec1eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f2e33640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec06e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f2e33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec152920 0x7fc3ec1ed740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f2e33640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0c1c50 con 0x7fc3ec152920 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f2e33640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0c0d60 con 0x7fc3ec0c1280 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f2e33640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0c1dd0 con 0x7fc3ec0c2190 2024-03-06T07:19:58.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec152920 0x7fc3ec1ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec152920 0x7fc3ec1ed740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42386/0 (socket says 172.21.15.26:42386) 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 -- 172.21.15.26:0/1397083909 learned_addr learned my addr 172.21.15.26:0/1397083909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f0bcf640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3ec0c1280 0x7fc3ec1eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3ebfff640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 -- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 msgr2=0x7fc3ec06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec06e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 -- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3ec0c1280 msgr2=0x7fc3ec1eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3ec0c1280 0x7fc3ec1eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 -- 172.21.15.26:0/1397083909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec153360 con 0x7fc3ec152920 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.526+0000 7fc3f13d0640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec152920 0x7fc3ec1ed740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009f40 tx=0x7fc3e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.527+0000 7fc3e97fa640 1 -- 172.21.15.26:0/1397083909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4043020 con 0x7fc3ec152920 2024-03-06T07:19:58.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.527+0000 7fc3e97fa640 1 -- 172.21.15.26:0/1397083909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e403e450 con 0x7fc3ec152920 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.527+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec14fdf0 con 0x7fc3ec152920 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.527+0000 7fc3e97fa640 1 -- 172.21.15.26:0/1397083909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4009510 con 0x7fc3ec152920 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.527+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec14f550 con 0x7fc3ec152920 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.528+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec04f330 con 0x7fc3ec152920 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.528+0000 7fc3e97fa640 1 -- 172.21.15.26:0/1397083909 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fc3e4046020 con 0x7fc3ec152920 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.528+0000 7fc3e97fa640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc3d006b440 0x7fc3d006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.529+0000 7fc3e97fa640 1 -- 172.21.15.26:0/1397083909 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3e40ac700 con 0x7fc3ec152920 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.529+0000 7fc3f0bcf640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc3d006b440 0x7fc3d006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.529+0000 7fc3f0bcf640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc3d006b440 0x7fc3d006d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec14cc70 tx=0x7fc3e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.533+0000 7fc3e97fa640 1 -- 172.21.15.26:0/1397083909 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc3e404d020 con 0x7fc3ec152920 2024-03-06T07:19:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.715+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3ec06bca0 con 0x7fc3ec152920 2024-03-06T07:19:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.716+0000 7fc3e97fa640 1 -- 172.21.15.26:0/1397083909 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12896 (secure 0 0 0) 0x7fc3e407b3f0 con 0x7fc3ec152920 2024-03-06T07:19:58.718 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:58.718 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:19:58.005389+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.717+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc3d006b440 msgr2=0x7fc3d006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.717+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc3d006b440 0x7fc3d006d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec14cc70 tx=0x7fc3e0008040 comp rx=0 tx=0).stop 2024-03-06T07:19:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.717+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec152920 msgr2=0x7fc3ec1ed740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.717+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec152920 0x7fc3ec1ed740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009f40 tx=0x7fc3e4004a40 comp rx=0 tx=0).stop 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 shutdown_connections 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc3d006b440 0x7fc3d006d900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3ec152920 0x7fc3ec1ed740 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3ec0c2190 0x7fc3ec06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 --2- 172.21.15.26:0/1397083909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3ec0c1280 0x7fc3ec1eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 >> 172.21.15.26:0/1397083909 conn(0x7fc3ec0c4290 msgr2=0x7fc3ec150140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 shutdown_connections 2024-03-06T07:19:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.718+0000 7fc3f2e33640 1 -- 172.21.15.26:0/1397083909 wait complete. 2024-03-06T07:19:58.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:19:58.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:19:58.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.833+0000 7feea8f79640 1 Processor -- start 2024-03-06T07:19:58.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.833+0000 7feea8f79640 1 -- start start 2024-03-06T07:19:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea8f79640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 0x7feea4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea8f79640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feea4158a10 0x7feea4158e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea8f79640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea40c75e0 con 0x7feea40be150 2024-03-06T07:19:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea8f79640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea40c7f10 con 0x7feea4158a10 2024-03-06T07:19:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea8f79640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea40c78c0 con 0x7feea414bcc0 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea1d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feea40be150 0x7feea40c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51658/0 (socket says 172.21.15.26:51658) 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea1d9b640 1 -- 172.21.15.26:0/2930744257 learned_addr learned my addr 172.21.15.26:0/2930744257 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea2d9d640 1 --2- 172.21.15.26:0/2930744257 >> v2:172.21.15.117:3300/0 conn(0x7feea4158a10 0x7feea4158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea259c640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 0x7feea4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265591344 0 0) 0x7feea40c75e0 con 0x7feea40be150 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee90003520 con 0x7feea40be150 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 632017893 0 0) 0x7fee90003520 con 0x7feea40be150 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 msgr2=0x7feea4155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 0x7feea4155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 >> v2:172.21.15.117:3300/0 conn(0x7feea4158a10 msgr2=0x7feea4158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 --2- 172.21.15.26:0/2930744257 >> v2:172.21.15.117:3300/0 conn(0x7feea4158a10 0x7feea4158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.834+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40c7ba0 con 0x7feea40be150 2024-03-06T07:19:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fee8c001670 con 0x7feea40be150 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 >> v1:172.21.15.26:6789/0 conn(0x7feea40be150 legacy=0x7feea40c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 0x7fee900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90003f30 0x7fee90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee90004d40 0x7fee900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea40c78c0 con 0x7fee90004d40 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9000b9c0 con 0x7fee90003f30 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9000bb70 con 0x7feea414bcc0 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea2d9d640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee90004d40 0x7fee900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea259c640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 0x7fee900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea1d9b640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90003f30 0x7fee90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea2d9d640 1 -- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 msgr2=0x7fee900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea2d9d640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414bcc0 0x7fee900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea2d9d640 1 -- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90003f30 msgr2=0x7fee90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea2d9d640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90003f30 0x7fee90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.835+0000 7feea2d9d640 1 -- 172.21.15.26:0/2930744257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee90003520 con 0x7fee90004d40 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea2d9d640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee90004d40 0x7fee900051c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee94009fc0 tx=0x7fee90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc9418924b7c368 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94039070 con 0x7fee90004d40 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9402cc60 con 0x7fee90004d40 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea159a640 1 -- 172.21.15.26:0/2930744257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94009c10 con 0x7fee90004d40 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea8f79640 1 -- 172.21.15.26:0/2930744257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee90004d40 msgr2=0x7fee900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea8f79640 1 --2- 172.21.15.26:0/2930744257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee90004d40 0x7fee900051c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee94009fc0 tx=0x7fee90005830 comp rx=0 tx=0).stop 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea2d9d640 1 -- 172.21.15.26:0/2930744257 reap_dead start 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea8f79640 1 -- 172.21.15.26:0/2930744257 shutdown_connections 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea8f79640 1 -- 172.21.15.26:0/2930744257 >> 172.21.15.26:0/2930744257 conn(0x7feea40ac600 msgr2=0x7feea40aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea8f79640 1 -- 172.21.15.26:0/2930744257 shutdown_connections 2024-03-06T07:19:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.836+0000 7feea8f79640 1 -- 172.21.15.26:0/2930744257 wait complete. 2024-03-06T07:19:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 Processor -- start 2024-03-06T07:19:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 -- start start 2024-03-06T07:19:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90004d40 0x7feea41edef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414f770 0x7feea41ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea414ab60 0x7feea414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea40c5c60 con 0x7feea414ab60 2024-03-06T07:19:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea40c5ae0 con 0x7fee90004d40 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea8f79640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea4159360 con 0x7feea414f770 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea414ab60 0x7feea414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea414ab60 0x7feea414b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42402/0 (socket says 172.21.15.26:42402) 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.838+0000 7feea2d9d640 1 -- 172.21.15.26:0/4090374742 learned_addr learned my addr 172.21.15.26:0/4090374742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea1d9b640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414f770 0x7feea41ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea259c640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90004d40 0x7feea41edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea2d9d640 1 -- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414f770 msgr2=0x7feea41ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea2d9d640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414f770 0x7feea41ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea2d9d640 1 -- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90004d40 msgr2=0x7feea41edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea2d9d640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90004d40 0x7feea41edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea2d9d640 1 -- 172.21.15.26:0/4090374742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea414cad0 con 0x7feea414ab60 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea1d9b640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414f770 0x7feea41ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea259c640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90004d40 0x7feea41edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:19:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea2d9d640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea414ab60 0x7feea414b010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee94037040 tx=0x7fee94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7fee9affd640 1 -- 172.21.15.26:0/4090374742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94039030 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7fee9affd640 1 -- 172.21.15.26:0/4090374742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9403a040 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea414cdb0 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7fee9affd640 1 -- 172.21.15.26:0/4090374742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94003990 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.839+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea40c13e0 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.840+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea4075300 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.840+0000 7fee9affd640 1 -- 172.21.15.26:0/4090374742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fee94045030 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.841+0000 7fee9affd640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8406b440 0x7fee8406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.841+0000 7fee9affd640 1 -- 172.21.15.26:0/4090374742 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fee940abd70 con 0x7feea414ab60 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.841+0000 7feea259c640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8406b440 0x7fee8406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.841+0000 7feea259c640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8406b440 0x7fee8406d900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009ab0 tx=0x7fee9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:58.845+0000 7fee9affd640 1 -- 172.21.15.26:0/4090374742 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feea4075300 con 0x7feea414ab60 2024-03-06T07:19:59.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.023+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feea40c83e0 con 0x7feea414ab60 2024-03-06T07:19:59.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.024+0000 7fee9affd640 1 -- 172.21.15.26:0/4090374742 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee9404c020 con 0x7feea414ab60 2024-03-06T07:19:59.027 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8406b440 msgr2=0x7fee8406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8406b440 0x7fee8406d900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009ab0 tx=0x7fee9c008040 comp rx=0 tx=0).stop 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea414ab60 msgr2=0x7feea414b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea414ab60 0x7feea414b010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee94037040 tx=0x7fee94009750 comp rx=0 tx=0).stop 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 shutdown_connections 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8406b440 0x7fee8406d900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea414ab60 0x7feea414b010 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feea414f770 0x7feea41ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 --2- 172.21.15.26:0/4090374742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee90004d40 0x7feea41edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 >> 172.21.15.26:0/4090374742 conn(0x7feea40ac600 msgr2=0x7feea40ab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 shutdown_connections 2024-03-06T07:19:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.026+0000 7feea8f79640 1 -- 172.21.15.26:0/4090374742 wait complete. 2024-03-06T07:19:59.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:19:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.137+0000 7f10687e2640 1 Processor -- start 2024-03-06T07:19:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.137+0000 7f10687e2640 1 -- start start 2024-03-06T07:19:59.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f10687e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106006f580 0x7f106015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f10687e2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600c6a90 con 0x7f10600c3630 2024-03-06T07:19:59.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f10687e2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600c6d30 con 0x7f10600c3a10 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f10687e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10600c7660 con 0x7f106006f580 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f106657e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f10600c3630 0x7f10600c4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51674/0 (socket says 172.21.15.26:51674) 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f106657e640 1 -- 172.21.15.26:0/2260115416 learned_addr learned my addr 172.21.15.26:0/2260115416 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f1066d7f640 1 --2- 172.21.15.26:0/2260115416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106006f580 0x7f106015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f106557c640 1 -- 172.21.15.26:0/2260115416 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177587759 0 0) 0x7f10600c6a90 con 0x7f10600c3630 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.138+0000 7f106557c640 1 -- 172.21.15.26:0/2260115416 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1048003520 con 0x7f10600c3630 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.139+0000 7f1066d7f640 1 -- 172.21.15.26:0/2260115416 >> v1:172.21.15.117:6789/0 conn(0x7f10600c3a10 legacy=0x7f106006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.139+0000 7f106557c640 1 -- 172.21.15.26:0/2260115416 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405596395 0 0) 0x7f10600c6d30 con 0x7f10600c3a10 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.139+0000 7f1066d7f640 1 -- 172.21.15.26:0/2260115416 >> v1:172.21.15.26:6789/0 conn(0x7f10600c3630 legacy=0x7f10600c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.139+0000 7f1066d7f640 1 -- 172.21.15.26:0/2260115416 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600c7010 con 0x7f106006f580 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.139+0000 7f1066d7f640 1 --2- 172.21.15.26:0/2260115416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106006f580 0x7f106015dc80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f10600c77e0 tx=0x7f105002be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3dda9928bdc553f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f106557c640 1 -- 172.21.15.26:0/2260115416 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050039070 con 0x7f106006f580 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f106557c640 1 -- 172.21.15.26:0/2260115416 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105002cc30 con 0x7f106006f580 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f106557c640 1 -- 172.21.15.26:0/2260115416 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050034440 con 0x7f106006f580 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f10687e2640 1 -- 172.21.15.26:0/2260115416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106006f580 msgr2=0x7f106015dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f10687e2640 1 --2- 172.21.15.26:0/2260115416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106006f580 0x7f106015dc80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f10600c77e0 tx=0x7f105002be80 comp rx=0 tx=0).stop 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f10687e2640 1 -- 172.21.15.26:0/2260115416 shutdown_connections 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f10687e2640 1 --2- 172.21.15.26:0/2260115416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106006f580 0x7f106015dc80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f10687e2640 1 -- 172.21.15.26:0/2260115416 >> 172.21.15.26:0/2260115416 conn(0x7f106006b5f0 msgr2=0x7f10600aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f10687e2640 1 -- 172.21.15.26:0/2260115416 shutdown_connections 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.140+0000 7f10687e2640 1 -- 172.21.15.26:0/2260115416 wait complete. 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.141+0000 7f10687e2640 1 Processor -- start 2024-03-06T07:19:59.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.141+0000 7f10687e2640 1 -- start start 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f10687e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106006f580 0x7f10601ec440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f10687e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10600c3630 0x7f10600c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f10687e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10600c3a10 0x7f10601da750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f10687e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106006fb50 con 0x7f106006f580 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f10687e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106006f9d0 con 0x7f10600c3a10 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f10687e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10600c3ea0 con 0x7f10600c3630 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106006f580 0x7f10601ec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106006f580 0x7f10601ec440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42414/0 (socket says 172.21.15.26:42414) 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 -- 172.21.15.26:0/1226233832 learned_addr learned my addr 172.21.15.26:0/1226233832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f1065d7d640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10600c3630 0x7f10600c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f1066d7f640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10600c3a10 0x7f10601da750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 -- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10600c3630 msgr2=0x7f10600c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10600c3630 0x7f10600c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 -- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10600c3a10 msgr2=0x7f10601da750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10600c3a10 0x7f10601da750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 -- 172.21.15.26:0/1226233832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600bf990 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.142+0000 7f106657e640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106006f580 0x7f10601ec440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f105800c430 tx=0x7f105800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.143+0000 7f105effd640 1 -- 172.21.15.26:0/1226233832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058017070 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.143+0000 7f105effd640 1 -- 172.21.15.26:0/1226233832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058005ce0 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.143+0000 7f105effd640 1 -- 172.21.15.26:0/1226233832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10580072f0 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.143+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10601ec980 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.143+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10600a75c0 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.144+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106004f330 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.144+0000 7f105effd640 1 -- 172.21.15.26:0/1226233832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f105801a070 con 0x7f106006f580 2024-03-06T07:19:59.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.144+0000 7f105effd640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f104c06b470 0x7f104c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.145+0000 7f105effd640 1 -- 172.21.15.26:0/1226233832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f10580890d0 con 0x7f106006f580 2024-03-06T07:19:59.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.145+0000 7f1065d7d640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f104c06b470 0x7f104c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.145+0000 7f1065d7d640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f104c06b470 0x7f104c06d930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10600c1380 tx=0x7f1054009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.149+0000 7f105effd640 1 -- 172.21.15.26:0/1226233832 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1058057dc0 con 0x7f106006f580 2024-03-06T07:19:59.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.331+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f10601dde50 con 0x7f106006f580 2024-03-06T07:19:59.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.332+0000 7f105effd640 1 -- 172.21.15.26:0/1226233832 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f105805be60 con 0x7f106006f580 2024-03-06T07:19:59.334 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:19:59.334 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:19:59.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:19:59.335 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:19:59.335 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:19:59.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:19:59.335 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:19:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.333+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f104c06b470 msgr2=0x7f104c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.333+0000 7f10687e2640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f104c06b470 0x7f104c06d930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10600c1380 tx=0x7f1054009040 comp rx=0 tx=0).stop 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.333+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106006f580 msgr2=0x7f10601ec440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.333+0000 7f10687e2640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106006f580 0x7f10601ec440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f105800c430 tx=0x7f105800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 shutdown_connections 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f104c06b470 0x7f104c06d930 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10600c3a10 0x7f10601da750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10600c3630 0x7f10600c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 --2- 172.21.15.26:0/1226233832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106006f580 0x7f10601ec440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 >> 172.21.15.26:0/1226233832 conn(0x7f106006b5f0 msgr2=0x7f10600a8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 shutdown_connections 2024-03-06T07:19:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.334+0000 7f10687e2640 1 -- 172.21.15.26:0/1226233832 wait complete. 2024-03-06T07:19:59.347 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:19:59.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:19:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.449+0000 7fe034c5c640 1 Processor -- start 2024-03-06T07:19:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.449+0000 7fe034c5c640 1 -- start start 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe034c5c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0300c33f0 0x7fe0300c5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe034c5c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe0300c5b60 0x7fe0300c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe034c5c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03006e790 con 0x7fe0300c3d10 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe034c5c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03006ed10 con 0x7fe0300c33f0 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe034c5c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe030152960 con 0x7fe0300c5b60 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe02e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0300c33f0 0x7fe0300c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe02ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe0300c3d10 0x7fe030155450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51684/0 (socket says 172.21.15.26:51684) 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.450+0000 7fe02ed9d640 1 -- 172.21.15.26:0/1285626329 learned_addr learned my addr 172.21.15.26:0/1285626329 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02e59c640 1 -- 172.21.15.26:0/1285626329 >> v2:172.21.15.117:3300/0 conn(0x7fe0300c5b60 msgr2=0x7fe0300c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02dd9b640 1 --2- 172.21.15.26:0/1285626329 >> v2:172.21.15.117:3300/0 conn(0x7fe0300c5b60 0x7fe0300c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02e59c640 1 --2- 172.21.15.26:0/1285626329 >> v2:172.21.15.117:3300/0 conn(0x7fe0300c5b60 0x7fe0300c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02e59c640 1 -- 172.21.15.26:0/1285626329 >> v1:172.21.15.26:6789/0 conn(0x7fe0300c3d10 legacy=0x7fe030155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02d59a640 1 -- 172.21.15.26:0/1285626329 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15029709 0 0) 0x7fe03006e790 con 0x7fe0300c3d10 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02e59c640 1 -- 172.21.15.26:0/1285626329 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03006ea30 con 0x7fe0300c33f0 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02e59c640 1 --2- 172.21.15.26:0/1285626329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe0300c5620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0300c4180 tx=0x7fe02802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569c21f66e845431 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02d59a640 1 -- 172.21.15.26:0/1285626329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028038070 con 0x7fe0300c33f0 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02d59a640 1 -- 172.21.15.26:0/1285626329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe028031740 con 0x7fe0300c33f0 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.451+0000 7fe02d59a640 1 -- 172.21.15.26:0/1285626329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02802f050 con 0x7fe0300c33f0 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 -- 172.21.15.26:0/1285626329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 msgr2=0x7fe0300c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 --2- 172.21.15.26:0/1285626329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe0300c5620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0300c4180 tx=0x7fe02802b7e0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 -- 172.21.15.26:0/1285626329 shutdown_connections 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 --2- 172.21.15.26:0/1285626329 >> v2:172.21.15.117:3300/0 conn(0x7fe0300c5b60 0x7fe0300c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 --2- 172.21.15.26:0/1285626329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe0300c5620 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 -- 172.21.15.26:0/1285626329 >> 172.21.15.26:0/1285626329 conn(0x7fe0300a9b00 msgr2=0x7fe0300c4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:59.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 -- 172.21.15.26:0/1285626329 shutdown_connections 2024-03-06T07:19:59.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.452+0000 7fe034c5c640 1 -- 172.21.15.26:0/1285626329 wait complete. 2024-03-06T07:19:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.453+0000 7fe034c5c640 1 Processor -- start 2024-03-06T07:19:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.453+0000 7fe034c5c640 1 -- start start 2024-03-06T07:19:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe034c5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe03014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe034c5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0300c3d10 0x7fe0300c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe034c5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300c5b60 0x7fe0301eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe034c5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03014c630 con 0x7fe0300c33f0 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe034c5c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03014c4b0 con 0x7fe0300c3d10 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe034c5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c2410 con 0x7fe0300c5b60 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe03014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe03014f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42428/0 (socket says 172.21.15.26:42428) 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 -- 172.21.15.26:0/2078739011 learned_addr learned my addr 172.21.15.26:0/2078739011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02dd9b640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0300c3d10 0x7fe0300c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02ed9d640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300c5b60 0x7fe0301eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 -- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300c5b60 msgr2=0x7fe0301eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300c5b60 0x7fe0301eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 -- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0300c3d10 msgr2=0x7fe0300c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0300c3d10 0x7fe0300c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02e59c640 1 -- 172.21.15.26:0/2078739011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0301ee3f0 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.454+0000 7fe02ed9d640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300c5b60 0x7fe0301eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.455+0000 7fe02e59c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe03014f2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe028000c00 tx=0x7fe0280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.455+0000 7fe026ffd640 1 -- 172.21.15.26:0/2078739011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028038040 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.455+0000 7fe026ffd640 1 -- 172.21.15.26:0/2078739011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe028004230 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.455+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0301ee6d0 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.455+0000 7fe026ffd640 1 -- 172.21.15.26:0/2078739011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028033860 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.455+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030160740 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.456+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03004f330 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.456+0000 7fe026ffd640 1 -- 172.21.15.26:0/2078739011 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fe028033a00 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.457+0000 7fe026ffd640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe01006b440 0x7fe01006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.457+0000 7fe026ffd640 1 -- 172.21.15.26:0/2078739011 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe0280abc40 con 0x7fe0300c33f0 2024-03-06T07:19:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.457+0000 7fe02dd9b640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe01006b440 0x7fe01006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.457+0000 7fe02dd9b640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe01006b440 0x7fe01006d900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0301e8b20 tx=0x7fe018008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.461+0000 7fe026ffd640 1 -- 172.21.15.26:0/2078739011 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe0280b2080 con 0x7fe0300c33f0 2024-03-06T07:19:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.640+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe0301501e0 con 0x7fe0300c33f0 2024-03-06T07:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.790+0000 7fe026ffd640 1 -- 172.21.15.26:0/2078739011 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe02807b0a0 con 0x7fe0300c33f0 2024-03-06T07:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe01006b440 msgr2=0x7fe01006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe01006b440 0x7fe01006d900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0301e8b20 tx=0x7fe018008040 comp rx=0 tx=0).stop 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 msgr2=0x7fe03014f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe03014f2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe028000c00 tx=0x7fe0280046c0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 shutdown_connections 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe01006b440 0x7fe01006d900 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300c5b60 0x7fe0301eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0300c3d10 0x7fe0300c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 --2- 172.21.15.26:0/2078739011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0300c33f0 0x7fe03014f2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 >> 172.21.15.26:0/2078739011 conn(0x7fe0300a9b00 msgr2=0x7fe0300c4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 shutdown_connections 2024-03-06T07:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.792+0000 7fe034c5c640 1 -- 172.21.15.26:0/2078739011 wait complete. 2024-03-06T07:19:59.810 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-06T07:19:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.917+0000 7f2bd9b25640 1 Processor -- start 2024-03-06T07:19:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.917+0000 7f2bd9b25640 1 -- start start 2024-03-06T07:19:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd9b25640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2bd406ca70 0x7f2bd414e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd9b25640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40b0450 con 0x7f2bd40b0070 2024-03-06T07:19:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd9b25640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd41560e0 con 0x7f2bd4155c30 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd9b25640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd406c020 con 0x7f2bd406ca70 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2bd40b0070 0x7f2bd40c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51698/0 (socket says 172.21.15.26:51698) 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd37fe640 1 -- 172.21.15.26:0/2803176362 learned_addr learned my addr 172.21.15.26:0/2803176362 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd3fff640 1 --2- 172.21.15.26:0/2803176362 >> v2:172.21.15.117:3300/0 conn(0x7f2bd406ca70 0x7f2bd414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd27fc640 1 -- 172.21.15.26:0/2803176362 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249515273 0 0) 0x7f2bc4001110 con 0x7f2bd40b0070 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.918+0000 7f2bd27fc640 1 -- 172.21.15.26:0/2803176362 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bbc003520 con 0x7f2bd40b0070 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.919+0000 7f2bd3fff640 1 -- 172.21.15.26:0/2803176362 >> v1:172.21.15.117:6789/0 conn(0x7f2bd4155c30 legacy=0x7f2bd40c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.919+0000 7f2bd3fff640 1 -- 172.21.15.26:0/2803176362 >> v1:172.21.15.26:6789/0 conn(0x7f2bd40b0070 legacy=0x7f2bd40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.919+0000 7f2bd27fc640 1 -- 172.21.15.26:0/2803176362 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028273113 0 0) 0x7f2bd41560e0 con 0x7f2bd4155c30 2024-03-06T07:19:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.919+0000 7f2bd3fff640 1 -- 172.21.15.26:0/2803176362 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40c77f0 con 0x7f2bd406ca70 2024-03-06T07:19:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.919+0000 7f2bd3fff640 1 --2- 172.21.15.26:0/2803176362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd406ca70 0x7f2bd414e800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009d90 tx=0x7f2bcc02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33638c8acbe5c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd27fc640 1 -- 172.21.15.26:0/2803176362 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc039070 con 0x7f2bd406ca70 2024-03-06T07:19:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd27fc640 1 -- 172.21.15.26:0/2803176362 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc02cc60 con 0x7f2bd406ca70 2024-03-06T07:19:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd27fc640 1 -- 172.21.15.26:0/2803176362 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc034640 con 0x7f2bd406ca70 2024-03-06T07:19:59.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd9b25640 1 -- 172.21.15.26:0/2803176362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd406ca70 msgr2=0x7f2bd414e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/2803176362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd406ca70 0x7f2bd414e800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009d90 tx=0x7f2bcc02beb0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd9b25640 1 -- 172.21.15.26:0/2803176362 shutdown_connections 2024-03-06T07:19:59.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/2803176362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd406ca70 0x7f2bd414e800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd9b25640 1 -- 172.21.15.26:0/2803176362 >> 172.21.15.26:0/2803176362 conn(0x7f2bd40ac590 msgr2=0x7f2bd40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:59.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd9b25640 1 -- 172.21.15.26:0/2803176362 shutdown_connections 2024-03-06T07:19:59.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.920+0000 7f2bd9b25640 1 -- 172.21.15.26:0/2803176362 wait complete. 2024-03-06T07:19:59.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.921+0000 7f2bd9b25640 1 Processor -- start 2024-03-06T07:19:59.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.921+0000 7f2bd9b25640 1 -- start start 2024-03-06T07:19:59.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd9b25640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd406ca70 0x7f2bd41eba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd9b25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd40b0070 0x7f2bd4154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd9b25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd4155c30 0x7f2bd40c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd9b25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd406c550 con 0x7f2bd40b0070 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd9b25640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd406c270 con 0x7f2bd4155c30 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd9b25640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd406da80 con 0x7f2bd406ca70 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd40b0070 0x7f2bd4154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd40b0070 0x7f2bd4154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42444/0 (socket says 172.21.15.26:42444) 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd2ffd640 1 -- 172.21.15.26:0/3580500467 learned_addr learned my addr 172.21.15.26:0/3580500467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:19:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd3fff640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd4155c30 0x7f2bd40c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.922+0000 7f2bd37fe640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd406ca70 0x7f2bd41eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd2ffd640 1 -- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd406ca70 msgr2=0x7f2bd41eba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd2ffd640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd406ca70 0x7f2bd41eba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd2ffd640 1 -- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd4155c30 msgr2=0x7f2bd40c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd2ffd640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd4155c30 0x7f2bd40c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd2ffd640 1 -- 172.21.15.26:0/3580500467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4159070 con 0x7f2bd40b0070 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd37fe640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd406ca70 0x7f2bd41eba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd2ffd640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd40b0070 0x7f2bd4154560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c4b0 tx=0x7f2bc800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bbbfff640 1 -- 172.21.15.26:0/3580500467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8017070 con 0x7f2bd40b0070 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bbbfff640 1 -- 172.21.15.26:0/3580500467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8005480 con 0x7f2bd40b0070 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd41592e0 con 0x7f2bd40b0070 2024-03-06T07:19:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bbbfff640 1 -- 172.21.15.26:0/3580500467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8003830 con 0x7f2bd40b0070 2024-03-06T07:19:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.923+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd4154aa0 con 0x7f2bd40b0070 2024-03-06T07:19:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.924+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd4155a50 con 0x7f2bd40b0070 2024-03-06T07:19:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.924+0000 7f2bbbfff640 1 -- 172.21.15.26:0/3580500467 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bc801a070 con 0x7f2bd40b0070 2024-03-06T07:19:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.925+0000 7f2bbbfff640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2bc006b4d0 0x7f2bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.925+0000 7f2bbbfff640 1 -- 172.21.15.26:0/3580500467 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2bc8089290 con 0x7f2bd40b0070 2024-03-06T07:19:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.925+0000 7f2bd37fe640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2bc006b4d0 0x7f2bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.925+0000 7f2bd37fe640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2bc006b4d0 0x7f2bc006d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007fd0 tx=0x7f2bc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:19:59.929+0000 7f2bbbfff640 1 -- 172.21.15.26:0/3580500467 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2bd4155a50 con 0x7f2bd40b0070 2024-03-06T07:20:00.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.113+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f2bd40ac340 con 0x7f2bd40b0070 2024-03-06T07:20:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.795+0000 7f2bbbfff640 1 -- 172.21.15.26:0/3580500467 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f2bd40ac340 con 0x7f2bd40b0070 2024-03-06T07:20:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:20:00.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.799+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2bc006b4d0 msgr2=0x7f2bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.799+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2bc006b4d0 0x7f2bc006d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007fd0 tx=0x7f2bc4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:00.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd40b0070 msgr2=0x7f2bd4154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd40b0070 0x7f2bd4154560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c4b0 tx=0x7f2bc800c970 comp rx=0 tx=0).stop 2024-03-06T07:20:00.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 shutdown_connections 2024-03-06T07:20:00.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2bc006b4d0 0x7f2bc006d990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bd4155c30 0x7f2bd40c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bd40b0070 0x7f2bd4154560 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 --2- 172.21.15.26:0/3580500467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bd406ca70 0x7f2bd41eba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.800+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 >> 172.21.15.26:0/3580500467 conn(0x7f2bd40ac590 msgr2=0x7f2bd40aa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.807+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 shutdown_connections 2024-03-06T07:20:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.808+0000 7f2bd9b25640 1 -- 172.21.15.26:0/3580500467 wait complete. 2024-03-06T07:20:00.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:20:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.938+0000 7f45ae156640 1 Processor -- start 2024-03-06T07:20:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.938+0000 7f45ae156640 1 -- start start 2024-03-06T07:20:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45ae156640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45a80c5600 0x7f45a8156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45ae156640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45a80c8220 0x7f45a80c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45ae156640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a806eef0 con 0x7f45a80c59e0 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45ae156640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a806ecc0 con 0x7f45a80c8220 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45ae156640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a806fac0 con 0x7f45a80c5600 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45a7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45a80c8220 0x7f45a80c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45a7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45a80c8220 0x7f45a80c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42448/0 (socket says 172.21.15.26:42448) 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45a7fff640 1 -- 172.21.15.26:0/1427015444 learned_addr learned my addr 172.21.15.26:0/1427015444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.939+0000 7f45a77fe640 1 --2- 172.21.15.26:0/1427015444 >> v2:172.21.15.117:3300/0 conn(0x7f45a80c5600 0x7f45a8156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.940+0000 7f45a7fff640 1 -- 172.21.15.26:0/1427015444 >> v2:172.21.15.117:3300/0 conn(0x7f45a80c5600 msgr2=0x7f45a8156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.940+0000 7f45a7fff640 1 --2- 172.21.15.26:0/1427015444 >> v2:172.21.15.117:3300/0 conn(0x7f45a80c5600 0x7f45a8156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.940+0000 7f45a7fff640 1 -- 172.21.15.26:0/1427015444 >> v1:172.21.15.117:6789/0 conn(0x7f45a80c59e0 legacy=0x7f45a81571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.940+0000 7f45a7fff640 1 -- 172.21.15.26:0/1427015444 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a806f190 con 0x7f45a80c8220 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.940+0000 7f45a7fff640 1 --2- 172.21.15.26:0/1427015444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c8220 0x7f45a80c6a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45a8072040 tx=0x7f45a002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba9566b662cbe12 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.940+0000 7f45a67fc640 1 -- 172.21.15.26:0/1427015444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a002ca80 con 0x7f45a80c8220 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45a67fc640 1 -- 172.21.15.26:0/1427015444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a002cc20 con 0x7f45a80c8220 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45a67fc640 1 -- 172.21.15.26:0/1427015444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a00343f0 con 0x7f45a80c8220 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 -- 172.21.15.26:0/1427015444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c8220 msgr2=0x7f45a80c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 --2- 172.21.15.26:0/1427015444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c8220 0x7f45a80c6a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45a8072040 tx=0x7f45a002bee0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 -- 172.21.15.26:0/1427015444 shutdown_connections 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 --2- 172.21.15.26:0/1427015444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c8220 0x7f45a80c6a20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 --2- 172.21.15.26:0/1427015444 >> v2:172.21.15.117:3300/0 conn(0x7f45a80c5600 0x7f45a8156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 -- 172.21.15.26:0/1427015444 >> 172.21.15.26:0/1427015444 conn(0x7f45a806b5f0 msgr2=0x7f45a80c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 -- 172.21.15.26:0/1427015444 shutdown_connections 2024-03-06T07:20:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.941+0000 7f45ae156640 1 -- 172.21.15.26:0/1427015444 wait complete. 2024-03-06T07:20:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.942+0000 7f45ae156640 1 Processor -- start 2024-03-06T07:20:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.942+0000 7f45ae156640 1 -- start start 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45ae156640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45a80c5600 0x7f45a814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45ae156640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c59e0 0x7f45a8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45ae156640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a80c8220 0x7f45a81dc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45ae156640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80a8540 con 0x7f45a80c59e0 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45ae156640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a806f750 con 0x7f45a80c8220 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45ae156640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80c40b0 con 0x7f45a80c5600 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c59e0 0x7f45a8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c59e0 0x7f45a8153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42456/0 (socket says 172.21.15.26:42456) 2024-03-06T07:20:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a6ffd640 1 -- 172.21.15.26:0/2258446582 learned_addr learned my addr 172.21.15.26:0/2258446582 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a6ffd640 1 -- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45a80c5600 msgr2=0x7f45a814c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a7fff640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a80c8220 0x7f45a81dc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a77fe640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45a80c5600 0x7f45a814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a6ffd640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45a80c5600 0x7f45a814c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a6ffd640 1 -- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a80c8220 msgr2=0x7f45a81dc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.943+0000 7f45a6ffd640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a80c8220 0x7f45a81dc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.944+0000 7f45a6ffd640 1 -- 172.21.15.26:0/2258446582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a81e90b0 con 0x7f45a80c59e0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.944+0000 7f45a6ffd640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c59e0 0x7f45a8153b00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c440 tx=0x7f459c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.944+0000 7f458ffff640 1 -- 172.21.15.26:0/2258446582 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0052d0 con 0x7f45a80c59e0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.944+0000 7f458ffff640 1 -- 172.21.15.26:0/2258446582 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c005470 con 0x7f45a80c59e0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.944+0000 7f458ffff640 1 -- 172.21.15.26:0/2258446582 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c007260 con 0x7f45a80c59e0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.944+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a81dbd00 con 0x7f45a80c59e0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.944+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a81dd750 con 0x7f45a80c59e0 2024-03-06T07:20:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.945+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a804f330 con 0x7f45a80c59e0 2024-03-06T07:20:00.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.946+0000 7f458ffff640 1 -- 172.21.15.26:0/2258446582 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f459c007400 con 0x7f45a80c59e0 2024-03-06T07:20:00.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.946+0000 7f458ffff640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f459406b4d0 0x7f459406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.946+0000 7f458ffff640 1 -- 172.21.15.26:0/2258446582 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f459c089620 con 0x7f45a80c59e0 2024-03-06T07:20:00.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.946+0000 7f45a77fe640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f459406b4d0 0x7f459406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.947+0000 7f45a77fe640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f459406b4d0 0x7f459406d990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4598009a10 tx=0x7f4598008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:00.950+0000 7f458ffff640 1 -- 172.21.15.26:0/2258446582 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f459c058590 con 0x7f45a80c59e0 2024-03-06T07:20:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.168+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f45a80ad360 con 0x7f45a80c59e0 2024-03-06T07:20:01.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.797+0000 7f458ffff640 1 -- 172.21.15.26:0/2258446582 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f459c057e90 con 0x7f45a80c59e0 2024-03-06T07:20:01.800 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f459406b4d0 msgr2=0x7f459406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f459406b4d0 0x7f459406d990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4598009a10 tx=0x7f4598008040 comp rx=0 tx=0).stop 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c59e0 msgr2=0x7f45a8153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c59e0 0x7f45a8153b00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c440 tx=0x7f459c001e70 comp rx=0 tx=0).stop 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 shutdown_connections 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f459406b4d0 0x7f459406d990 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a80c8220 0x7f45a81dc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a80c59e0 0x7f45a8153b00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 --2- 172.21.15.26:0/2258446582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45a80c5600 0x7f45a814c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 >> 172.21.15.26:0/2258446582 conn(0x7f45a806b5f0 msgr2=0x7f45a80ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 shutdown_connections 2024-03-06T07:20:01.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.800+0000 7f45ae156640 1 -- 172.21.15.26:0/2258446582 wait complete. 2024-03-06T07:20:01.836 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-06T07:20:01.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-06T07:20:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.944+0000 7fc19ea55640 1 Processor -- start 2024-03-06T07:20:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.944+0000 7fc19ea55640 1 -- start start 2024-03-06T07:20:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.944+0000 7fc19ea55640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1980a9980 0x7fc1981521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19ea55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc19806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19ea55640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc198155780 con 0x7fc19814ebb0 2024-03-06T07:20:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19ea55640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc198155530 con 0x7fc1980a9980 2024-03-06T07:20:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19ea55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc198155a60 con 0x7fc1980c28d0 2024-03-06T07:20:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19cff2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc19806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc197fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1980a9980 0x7fc1981521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19cff2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc19806d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46824/0 (socket says 172.21.15.26:46824) 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc197fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1980a9980 0x7fc1981521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56262/0 (socket says 172.21.15.26:56262) 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19cff2640 1 -- 172.21.15.26:0/566656653 learned_addr learned my addr 172.21.15.26:0/566656653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc1977fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc19814ebb0 0x7fc1980a9fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:53794/0 (socket says 172.21.15.26:53794) 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc196ffd640 1 -- 172.21.15.26:0/566656653 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042683049 0 0) 0x7fc198155780 con 0x7fc19814ebb0 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc196ffd640 1 -- 172.21.15.26:0/566656653 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc180003520 con 0x7fc19814ebb0 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc197fff640 1 -- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 msgr2=0x7fc19806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc197fff640 1 --2- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc19806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc197fff640 1 -- 172.21.15.26:0/566656653 >> v1:172.21.15.117:6789/0 conn(0x7fc19814ebb0 legacy=0x7fc1980a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc197fff640 1 -- 172.21.15.26:0/566656653 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1981516d0 con 0x7fc1980a9980 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.945+0000 7fc19cff2640 1 --2- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc19806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.946+0000 7fc197fff640 1 --2- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 0x7fc1981521b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1880097a0 tx=0x7fc18802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504657b6b3391731 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.946+0000 7fc196ffd640 1 -- 172.21.15.26:0/566656653 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880315a0 con 0x7fc1980a9980 2024-03-06T07:20:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.946+0000 7fc196ffd640 1 -- 172.21.15.26:0/566656653 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188037070 con 0x7fc1980a9980 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.946+0000 7fc196ffd640 1 -- 172.21.15.26:0/566656653 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188031940 con 0x7fc1980a9980 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 -- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 msgr2=0x7fc1981521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 --2- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 0x7fc1981521b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1880097a0 tx=0x7fc18802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 -- 172.21.15.26:0/566656653 shutdown_connections 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 --2- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc19806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 --2- 172.21.15.26:0/566656653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 0x7fc1981521b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 -- 172.21.15.26:0/566656653 >> 172.21.15.26:0/566656653 conn(0x7fc1980c65b0 msgr2=0x7fc1980c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 -- 172.21.15.26:0/566656653 shutdown_connections 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.947+0000 7fc19ea55640 1 -- 172.21.15.26:0/566656653 wait complete. 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.948+0000 7fc19ea55640 1 Processor -- start 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.948+0000 7fc19ea55640 1 -- start start 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 0x7fc1981ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc198071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19814ebb0 0x7fc1981e8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980aa6c0 con 0x7fc19814ebb0 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198151c90 con 0x7fc1980a9980 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c2d60 con 0x7fc1980c28d0 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19814ebb0 0x7fc1981e8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19814ebb0 0x7fc1981e8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42460/0 (socket says 172.21.15.26:42460) 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 -- 172.21.15.26:0/661362516 learned_addr learned my addr 172.21.15.26:0/661362516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc197fff640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 0x7fc1981ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc1977fe640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc198071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 -- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 msgr2=0x7fc198071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc198071f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 -- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 msgr2=0x7fc1981ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 0x7fc1981ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 -- 172.21.15.26:0/661362516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980724d0 con 0x7fc19814ebb0 2024-03-06T07:20:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19cff2640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19814ebb0 0x7fc1981e8bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc190009e70 tx=0x7fc19000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc194ff9640 1 -- 172.21.15.26:0/661362516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000cbe0 con 0x7fc19814ebb0 2024-03-06T07:20:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc194ff9640 1 -- 172.21.15.26:0/661362516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19000cd80 con 0x7fc19814ebb0 2024-03-06T07:20:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc198072750 con 0x7fc19814ebb0 2024-03-06T07:20:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc194ff9640 1 -- 172.21.15.26:0/661362516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190003bb0 con 0x7fc19814ebb0 2024-03-06T07:20:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.949+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1981ec100 con 0x7fc19814ebb0 2024-03-06T07:20:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.950+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19804f330 con 0x7fc19814ebb0 2024-03-06T07:20:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.951+0000 7fc194ff9640 1 -- 172.21.15.26:0/661362516 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc19001b030 con 0x7fc19814ebb0 2024-03-06T07:20:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.951+0000 7fc194ff9640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc18406b4d0 0x7fc18406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.951+0000 7fc194ff9640 1 -- 172.21.15.26:0/661362516 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc19008a5b0 con 0x7fc19814ebb0 2024-03-06T07:20:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.951+0000 7fc197fff640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc18406b4d0 0x7fc18406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.952+0000 7fc197fff640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc18406b4d0 0x7fc18406d990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc188036000 tx=0x7fc18803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:01.956+0000 7fc194ff9640 1 -- 172.21.15.26:0/661362516 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc190022020 con 0x7fc19814ebb0 2024-03-06T07:20:02.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.148+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fc1980abcf0 con 0x7fc19814ebb0 2024-03-06T07:20:02.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.811+0000 7fc194ff9640 1 -- 172.21.15.26:0/661362516 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc1980724d0 con 0x7fc19814ebb0 2024-03-06T07:20:02.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.814+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc18406b4d0 msgr2=0x7fc18406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.814+0000 7fc19ea55640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc18406b4d0 0x7fc18406d990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc188036000 tx=0x7fc18803d040 comp rx=0 tx=0).stop 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.814+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19814ebb0 msgr2=0x7fc1981e8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.814+0000 7fc19ea55640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19814ebb0 0x7fc1981e8bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc190009e70 tx=0x7fc19000c490 comp rx=0 tx=0).stop 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.814+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 shutdown_connections 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.814+0000 7fc19ea55640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc18406b4d0 0x7fc18406d990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.815+0000 7fc19ea55640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19814ebb0 0x7fc1981e8bb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.815+0000 7fc19ea55640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1980c28d0 0x7fc198071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.815+0000 7fc19ea55640 1 --2- 172.21.15.26:0/661362516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980a9980 0x7fc1981ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.815+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 >> 172.21.15.26:0/661362516 conn(0x7fc1980c65b0 msgr2=0x7fc1981de940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.815+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 shutdown_connections 2024-03-06T07:20:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.815+0000 7fc19ea55640 1 -- 172.21.15.26:0/661362516 wait complete. 2024-03-06T07:20:02.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:20:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.943+0000 7efcadd02640 1 Processor -- start 2024-03-06T07:20:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.943+0000 7efcadd02640 1 -- start start 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efcadd02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca814f250 0x7efca8155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efcadd02640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca814f630 con 0x7efca80c8960 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efcadd02640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca80c76c0 con 0x7efca8158aa0 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efcadd02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca80c7470 con 0x7efca814f250 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efca80c8960 0x7efca80c53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51706/0 (socket says 172.21.15.26:51706) 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca6ffd640 1 -- 172.21.15.26:0/375080784 learned_addr learned my addr 172.21.15.26:0/375080784 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca77fe640 1 --2- 172.21.15.26:0/375080784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca814f250 0x7efca8155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca67fc640 1 -- 172.21.15.26:0/375080784 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018892551 0 0) 0x7efca814f630 con 0x7efca80c8960 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca67fc640 1 -- 172.21.15.26:0/375080784 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc90003520 con 0x7efca80c8960 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca67fc640 1 -- 172.21.15.26:0/375080784 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939160412 0 0) 0x7efca80c76c0 con 0x7efca8158aa0 2024-03-06T07:20:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca67fc640 1 -- 172.21.15.26:0/375080784 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca814f630 con 0x7efca8158aa0 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca77fe640 1 -- 172.21.15.26:0/375080784 >> v1:172.21.15.117:6789/0 conn(0x7efca8158aa0 legacy=0x7efca8158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca77fe640 1 -- 172.21.15.26:0/375080784 >> v1:172.21.15.26:6789/0 conn(0x7efca80c8960 legacy=0x7efca80c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.944+0000 7efca77fe640 1 -- 172.21.15.26:0/375080784 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80c79a0 con 0x7efca814f250 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.945+0000 7efca77fe640 1 --2- 172.21.15.26:0/375080784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca814f250 0x7efca8155bc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efca80c75f0 tx=0x7efc9802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd511db13ba83fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.945+0000 7efca67fc640 1 -- 172.21.15.26:0/375080784 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98031760 con 0x7efca814f250 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.945+0000 7efca67fc640 1 -- 172.21.15.26:0/375080784 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc98037070 con 0x7efca814f250 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.945+0000 7efca67fc640 1 -- 172.21.15.26:0/375080784 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98031b00 con 0x7efca814f250 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.946+0000 7efcadd02640 1 -- 172.21.15.26:0/375080784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca814f250 msgr2=0x7efca8155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.946+0000 7efcadd02640 1 --2- 172.21.15.26:0/375080784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca814f250 0x7efca8155bc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efca80c75f0 tx=0x7efc9802b800 comp rx=0 tx=0).stop 2024-03-06T07:20:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.946+0000 7efcadd02640 1 -- 172.21.15.26:0/375080784 shutdown_connections 2024-03-06T07:20:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.946+0000 7efcadd02640 1 --2- 172.21.15.26:0/375080784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca814f250 0x7efca8155bc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.946+0000 7efcadd02640 1 -- 172.21.15.26:0/375080784 >> 172.21.15.26:0/375080784 conn(0x7efca80ac610 msgr2=0x7efca80aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.946+0000 7efcadd02640 1 -- 172.21.15.26:0/375080784 shutdown_connections 2024-03-06T07:20:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.946+0000 7efcadd02640 1 -- 172.21.15.26:0/375080784 wait complete. 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.947+0000 7efcadd02640 1 Processor -- start 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.947+0000 7efcadd02640 1 -- start start 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efcadd02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca80c8960 0x7efca81543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efcadd02640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca814f250 0x7efca81ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efcadd02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca8158aa0 0x7efca81ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efcadd02640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca8159640 con 0x7efca8158aa0 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efcadd02640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca80c7f60 con 0x7efca814f250 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efcadd02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca80be310 con 0x7efca80c8960 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca80c8960 0x7efca81543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca80c8960 0x7efca81543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46842/0 (socket says 172.21.15.26:46842) 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca8158aa0 0x7efca81ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca77fe640 1 -- 172.21.15.26:0/2287094042 learned_addr learned my addr 172.21.15.26:0/2287094042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca6ffd640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca814f250 0x7efca81ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca7fff640 1 -- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca80c8960 msgr2=0x7efca81543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca7fff640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca80c8960 0x7efca81543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca7fff640 1 -- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca814f250 msgr2=0x7efca81ebf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.948+0000 7efca7fff640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca814f250 0x7efca81ebf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efca7fff640 1 -- 172.21.15.26:0/2287094042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca81eb910 con 0x7efca8158aa0 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efca6ffd640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca814f250 0x7efca81ebf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efca77fe640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca80c8960 0x7efca81543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efca7fff640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca8158aa0 0x7efca81ec4c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efca000c6c0 tx=0x7efca000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efc8ffff640 1 -- 172.21.15.26:0/2287094042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0003b70 con 0x7efca8158aa0 2024-03-06T07:20:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efc8ffff640 1 -- 172.21.15.26:0/2287094042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca0003d10 con 0x7efca8158aa0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca81ee080 con 0x7efca8158aa0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efc8ffff640 1 -- 172.21.15.26:0/2287094042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0005310 con 0x7efca8158aa0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.949+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca81ee920 con 0x7efca8158aa0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.950+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca81ee470 con 0x7efca8158aa0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.950+0000 7efc8ffff640 1 -- 172.21.15.26:0/2287094042 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efca00054b0 con 0x7efca8158aa0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.951+0000 7efc8ffff640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc9406b4d0 0x7efc9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.951+0000 7efc8ffff640 1 -- 172.21.15.26:0/2287094042 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7efca008a0a0 con 0x7efca8158aa0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.951+0000 7efca77fe640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc9406b4d0 0x7efc9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.951+0000 7efca77fe640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc9406b4d0 0x7efc9406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc98009720 tx=0x7efc9803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:02.955+0000 7efc8ffff640 1 -- 172.21.15.26:0/2287094042 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efca0058790 con 0x7efca8158aa0 2024-03-06T07:20:03.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:03.137+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efca8158510 con 0x7efca8158aa0 2024-03-06T07:20:04.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.829+0000 7efc8ffff640 1 -- 172.21.15.26:0/2287094042 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7efca005c830 con 0x7efca8158aa0 2024-03-06T07:20:04.832 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data_ec' created 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc9406b4d0 msgr2=0x7efc9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc9406b4d0 0x7efc9406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc98009720 tx=0x7efc9803d040 comp rx=0 tx=0).stop 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca8158aa0 msgr2=0x7efca81ec4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca8158aa0 0x7efca81ec4c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efca000c6c0 tx=0x7efca000cb80 comp rx=0 tx=0).stop 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 shutdown_connections 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc9406b4d0 0x7efc9406d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca8158aa0 0x7efca81ec4c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca814f250 0x7efca81ebf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 --2- 172.21.15.26:0/2287094042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca80c8960 0x7efca81543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.833+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 >> 172.21.15.26:0/2287094042 conn(0x7efca80ac610 msgr2=0x7efca806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.834+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 shutdown_connections 2024-03-06T07:20:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.834+0000 7efcadd02640 1 -- 172.21.15.26:0/2287094042 wait complete. 2024-03-06T07:20:04.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-06T07:20:04.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.962+0000 7fbd06565640 1 Processor -- start 2024-03-06T07:20:04.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbd06565640 1 -- start start 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbd06565640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd0014f180 0x7fbd000c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbd06565640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd001551d0 0x7fbd001566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbd06565640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd000c8f60 con 0x7fbd000ca000 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbd06565640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd00156c00 con 0x7fbd0014f180 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbd06565640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000c9240 con 0x7fbd001551d0 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbcfffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd0014f180 0x7fbd000c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd001551d0 0x7fbd001566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbcfffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd0014f180 0x7fbd000c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56286/0 (socket says 172.21.15.26:56286) 2024-03-06T07:20:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbd04b02640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fbd000ca000 0x7fbd0006c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:53818/0 (socket says 172.21.15.26:53818) 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd001551d0 0x7fbd001566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46856/0 (socket says 172.21.15.26:46856) 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.963+0000 7fbcfffff640 1 -- 172.21.15.26:0/1485259606 learned_addr learned my addr 172.21.15.26:0/1485259606 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.964+0000 7fbcff7fe640 1 -- 172.21.15.26:0/1485259606 >> v2:172.21.15.117:3300/0 conn(0x7fbd0014f180 msgr2=0x7fbd000c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.964+0000 7fbcff7fe640 1 --2- 172.21.15.26:0/1485259606 >> v2:172.21.15.117:3300/0 conn(0x7fbd0014f180 0x7fbd000c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.964+0000 7fbcff7fe640 1 -- 172.21.15.26:0/1485259606 >> v1:172.21.15.117:6789/0 conn(0x7fbd000ca000 legacy=0x7fbd0006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.964+0000 7fbcff7fe640 1 -- 172.21.15.26:0/1485259606 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000c9520 con 0x7fbd001551d0 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.964+0000 7fbcfeffd640 1 -- 172.21.15.26:0/1485259606 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805323022 0 0) 0x7fbd000c8f60 con 0x7fbd000ca000 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.964+0000 7fbcfffff640 1 --2- 172.21.15.26:0/1485259606 >> v2:172.21.15.117:3300/0 conn(0x7fbd0014f180 0x7fbd000c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbcff7fe640 1 --2- 172.21.15.26:0/1485259606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd001551d0 0x7fbd001566c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00097a0 tx=0x7fbcf002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8662e74374f0b967 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbcfeffd640 1 -- 172.21.15.26:0/1485259606 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0035020 con 0x7fbd001551d0 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbcfeffd640 1 -- 172.21.15.26:0/1485259606 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf0031830 con 0x7fbd001551d0 2024-03-06T07:20:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbcfeffd640 1 -- 172.21.15.26:0/1485259606 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0031c00 con 0x7fbd001551d0 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 -- 172.21.15.26:0/1485259606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd001551d0 msgr2=0x7fbd001566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 --2- 172.21.15.26:0/1485259606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd001551d0 0x7fbd001566c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00097a0 tx=0x7fbcf002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 -- 172.21.15.26:0/1485259606 shutdown_connections 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 --2- 172.21.15.26:0/1485259606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd001551d0 0x7fbd001566c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 --2- 172.21.15.26:0/1485259606 >> v2:172.21.15.117:3300/0 conn(0x7fbd0014f180 0x7fbd000c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 -- 172.21.15.26:0/1485259606 >> 172.21.15.26:0/1485259606 conn(0x7fbd00072d20 msgr2=0x7fbd000ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 -- 172.21.15.26:0/1485259606 shutdown_connections 2024-03-06T07:20:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.965+0000 7fbd06565640 1 -- 172.21.15.26:0/1485259606 wait complete. 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.966+0000 7fbd06565640 1 Processor -- start 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd06565640 1 -- start start 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd06565640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd000ca000 0x7fbd000af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd06565640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd0014f180 0x7fbd001eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd06565640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd001551d0 0x7fbd001eb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd06565640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0014f560 con 0x7fbd000ca000 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd06565640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000c9ae0 con 0x7fbd001551d0 2024-03-06T07:20:04.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd06565640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0014f6e0 con 0x7fbd0014f180 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbcfffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd000ca000 0x7fbd000af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbcfffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd000ca000 0x7fbd000af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42484/0 (socket says 172.21.15.26:42484) 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbcfffff640 1 -- 172.21.15.26:0/3915377151 learned_addr learned my addr 172.21.15.26:0/3915377151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbd04b02640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd001551d0 0x7fbd001eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.967+0000 7fbcff7fe640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd0014f180 0x7fbd001eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfffff640 1 -- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd0014f180 msgr2=0x7fbd001eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfffff640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd0014f180 0x7fbd001eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfffff640 1 -- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd001551d0 msgr2=0x7fbd001eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfffff640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd001551d0 0x7fbd001eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfffff640 1 -- 172.21.15.26:0/3915377151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000afb70 con 0x7fbd000ca000 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfffff640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd000ca000 0x7fbd000af120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbce800e9d0 tx=0x7fbce800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfcff9640 1 -- 172.21.15.26:0/3915377151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8017070 con 0x7fbd000ca000 2024-03-06T07:20:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfcff9640 1 -- 172.21.15.26:0/3915377151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce800ca60 con 0x7fbd000ca000 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbcfcff9640 1 -- 172.21.15.26:0/3915377151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800ce10 con 0x7fbd000ca000 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd001edf00 con 0x7fbd000ca000 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.968+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd001ee4c0 con 0x7fbd000ca000 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.969+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0004f330 con 0x7fbd000ca000 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.970+0000 7fbcfcff9640 1 -- 172.21.15.26:0/3915377151 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbce801a070 con 0x7fbd000ca000 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.970+0000 7fbcfcff9640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbce006b4d0 0x7fbce006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.970+0000 7fbcfcff9640 1 -- 172.21.15.26:0/3915377151 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbce808a700 con 0x7fbd000ca000 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.970+0000 7fbcff7fe640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbce006b4d0 0x7fbce006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.971+0000 7fbcff7fe640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbce006b4d0 0x7fbce006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0036000 tx=0x7fbcf003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:04.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:04.975+0000 7fbcfcff9640 1 -- 172.21.15.26:0/3915377151 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbce8058ca0 con 0x7fbd000ca000 2024-03-06T07:20:05.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.157+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fbd000afe70 con 0x7fbd000ca000 2024-03-06T07:20:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.833+0000 7fbcfcff9640 1 -- 172.21.15.26:0/3915377151 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbce805cd40 con 0x7fbd000ca000 2024-03-06T07:20:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:set pool 4 allow_ec_overwrites to true 2024-03-06T07:20:05.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbce006b4d0 msgr2=0x7fbce006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbce006b4d0 0x7fbce006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0036000 tx=0x7fbcf003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd000ca000 msgr2=0x7fbd000af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd000ca000 0x7fbd000af120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbce800e9d0 tx=0x7fbce800ee90 comp rx=0 tx=0).stop 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 shutdown_connections 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbce006b4d0 0x7fbce006d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd001551d0 0x7fbd001eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd0014f180 0x7fbd001eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 --2- 172.21.15.26:0/3915377151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd000ca000 0x7fbd000af120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 >> 172.21.15.26:0/3915377151 conn(0x7fbd00072d20 msgr2=0x7fbd00158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:05.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.839+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 shutdown_connections 2024-03-06T07:20:05.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.840+0000 7fbd06565640 1 -- 172.21.15.26:0/3915377151 wait complete. 2024-03-06T07:20:05.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-06T07:20:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.973+0000 7f30f745c640 1 Processor -- start 2024-03-06T07:20:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.973+0000 7f30f745c640 1 -- start start 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f745c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30f00c73d0 0x7f30f00c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f745c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f006c0b0 0x7f30f006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f745c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f0152ba0 con 0x7f30f01527c0 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f745c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f01552c0 con 0x7f30f00c73d0 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f745c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0155560 con 0x7f30f006c0b0 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f49f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30f00c73d0 0x7f30f00c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f51f8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f30f01527c0 0x7f30f00c88c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51716/0 (socket says 172.21.15.26:51716) 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f51f8640 1 -- 172.21.15.26:0/345990445 learned_addr learned my addr 172.21.15.26:0/345990445 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f59f9640 1 --2- 172.21.15.26:0/345990445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f006c0b0 0x7f30f006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f49f7640 1 -- 172.21.15.26:0/345990445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f006c0b0 msgr2=0x7f30f006c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f49f7640 1 --2- 172.21.15.26:0/345990445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f006c0b0 0x7f30f006c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f49f7640 1 -- 172.21.15.26:0/345990445 >> v1:172.21.15.26:6789/0 conn(0x7f30f01527c0 legacy=0x7f30f00c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.974+0000 7f30f49f7640 1 -- 172.21.15.26:0/345990445 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f0155840 con 0x7f30f00c73d0 2024-03-06T07:20:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.975+0000 7f30f49f7640 1 --2- 172.21.15.26:0/345990445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f00c73d0 0x7f30f00c7860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30e40097a0 tx=0x7f30e402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447766cb641ee481 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.975+0000 7f30effff640 1 -- 172.21.15.26:0/345990445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e40315a0 con 0x7f30f00c73d0 2024-03-06T07:20:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.975+0000 7f30effff640 1 -- 172.21.15.26:0/345990445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e4037070 con 0x7f30f00c73d0 2024-03-06T07:20:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.975+0000 7f30effff640 1 -- 172.21.15.26:0/345990445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4031960 con 0x7f30f00c73d0 2024-03-06T07:20:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 -- 172.21.15.26:0/345990445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f00c73d0 msgr2=0x7f30f00c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 --2- 172.21.15.26:0/345990445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f00c73d0 0x7f30f00c7860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30e40097a0 tx=0x7f30e402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 -- 172.21.15.26:0/345990445 shutdown_connections 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 --2- 172.21.15.26:0/345990445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f00c73d0 0x7f30f00c7860 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 --2- 172.21.15.26:0/345990445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f006c0b0 0x7f30f006c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 -- 172.21.15.26:0/345990445 >> 172.21.15.26:0/345990445 conn(0x7f30f00ac550 msgr2=0x7f30f00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 -- 172.21.15.26:0/345990445 shutdown_connections 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.976+0000 7f30f745c640 1 -- 172.21.15.26:0/345990445 wait complete. 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.977+0000 7f30f745c640 1 Processor -- start 2024-03-06T07:20:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.977+0000 7f30f745c640 1 -- start start 2024-03-06T07:20:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f745c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f006c0b0 0x7f30f01ef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f745c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f00c73d0 0x7f30f014a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f745c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30f01527c0 0x7f30f01eb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f745c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f006daf0 con 0x7f30f006c0b0 2024-03-06T07:20:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f745c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0155e00 con 0x7f30f01527c0 2024-03-06T07:20:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f745c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f00c8fd0 con 0x7f30f00c73d0 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f006c0b0 0x7f30f01ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f006c0b0 0x7f30f01ef180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42510/0 (socket says 172.21.15.26:42510) 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 -- 172.21.15.26:0/3195718961 learned_addr learned my addr 172.21.15.26:0/3195718961 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f59f9640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30f01527c0 0x7f30f01eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f49f7640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f00c73d0 0x7f30f014a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 -- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f00c73d0 msgr2=0x7f30f014a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f00c73d0 0x7f30f014a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 -- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30f01527c0 msgr2=0x7f30f01eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30f01527c0 0x7f30f01eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 -- 172.21.15.26:0/3195718961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f0148340 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f51f8640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f006c0b0 0x7f30f01ef180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30e000caa0 tx=0x7f30e000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30edffb640 1 -- 172.21.15.26:0/3195718961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0017070 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30edffb640 1 -- 172.21.15.26:0/3195718961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e000ac00 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30edffb640 1 -- 172.21.15.26:0/3195718961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0012400 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.978+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f01ee6a0 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.979+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f014a5e0 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.980+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f004f330 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.980+0000 7f30edffb640 1 -- 172.21.15.26:0/3195718961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30e001a070 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.980+0000 7f30edffb640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30dc06b4d0 0x7f30dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.980+0000 7f30edffb640 1 -- 172.21.15.26:0/3195718961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f30e008a120 con 0x7f30f006c0b0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.981+0000 7f30f49f7640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30dc06b4d0 0x7f30dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.981+0000 7f30f49f7640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30dc06b4d0 0x7f30dc06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30e4036000 tx=0x7f30e403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:05.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:05.985+0000 7f30edffb640 1 -- 172.21.15.26:0/3195718961 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f30f0152ba0 con 0x7f30f006c0b0 2024-03-06T07:20:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.208+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f30f006d600 con 0x7f30f006c0b0 2024-03-06T07:20:06.434 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:20:06.432+0000 7fea37c36640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:20:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.835+0000 7f30edffb640 1 -- 172.21.15.26:0/3195718961 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f30e00586c0 con 0x7f30f006c0b0 2024-03-06T07:20:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:added data pool 4 to fsmap 2024-03-06T07:20:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30dc06b4d0 msgr2=0x7f30dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30dc06b4d0 0x7f30dc06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30e4036000 tx=0x7f30e403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f006c0b0 msgr2=0x7f30f01ef180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f006c0b0 0x7f30f01ef180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30e000caa0 tx=0x7f30e000cf60 comp rx=0 tx=0).stop 2024-03-06T07:20:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 shutdown_connections 2024-03-06T07:20:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30dc06b4d0 0x7f30dc06d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30f01527c0 0x7f30f01eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30f00c73d0 0x7f30f014a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 --2- 172.21.15.26:0/3195718961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30f006c0b0 0x7f30f01ef180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 >> 172.21.15.26:0/3195718961 conn(0x7f30f00ac550 msgr2=0x7f30f006c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 shutdown_connections 2024-03-06T07:20:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.838+0000 7f30f745c640 1 -- 172.21.15.26:0/3195718961 wait complete. 2024-03-06T07:20:06.854 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.959+0000 7fcdc2864640 1 Processor -- start 2024-03-06T07:20:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.959+0000 7fcdc2864640 1 -- start start 2024-03-06T07:20:06.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdc2864640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:06.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdc2864640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdbc152c20 0x7fcdbc0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdc2864640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdbc072440 con 0x7fcdbc14f650 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdc2864640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc0c2b10 con 0x7fcdbc152c20 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdc2864640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc152140 con 0x7fcdbc0c3150 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdbb7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdbbfff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fcdbc14f650 0x7fcdbc071880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:53820/0 (socket says 172.21.15.26:53820) 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdbb7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc0c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46884/0 (socket says 172.21.15.26:46884) 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdbbfff640 1 -- 172.21.15.26:0/1861293096 learned_addr learned my addr 172.21.15.26:0/1861293096 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.960+0000 7fcdc0e01640 1 --2- 172.21.15.26:0/1861293096 >> v2:172.21.15.26:3300/0 conn(0x7fcdbc152c20 0x7fcdbc0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.961+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/1861293096 >> v2:172.21.15.26:3300/0 conn(0x7fcdbc152c20 msgr2=0x7fcdbc0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.961+0000 7fcdbb7fe640 1 --2- 172.21.15.26:0/1861293096 >> v2:172.21.15.26:3300/0 conn(0x7fcdbc152c20 0x7fcdbc0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.961+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/1861293096 >> v1:172.21.15.117:6789/0 conn(0x7fcdbc14f650 legacy=0x7fcdbc071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.961+0000 7fcdbaffd640 1 -- 172.21.15.26:0/1861293096 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498394559 0 0) 0x7fcda4001110 con 0x7fcdbc14f650 2024-03-06T07:20:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.961+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/1861293096 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc152420 con 0x7fcdbc0c3150 2024-03-06T07:20:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.961+0000 7fcdbb7fe640 1 --2- 172.21.15.26:0/1861293096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc0c5690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0097a0 tx=0x7fcdac02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ea3839127a2300 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdbaffd640 1 -- 172.21.15.26:0/1861293096 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac035020 con 0x7fcdbc0c3150 2024-03-06T07:20:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdbaffd640 1 -- 172.21.15.26:0/1861293096 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac031830 con 0x7fcdbc0c3150 2024-03-06T07:20:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdbaffd640 1 -- 172.21.15.26:0/1861293096 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac031c00 con 0x7fcdbc0c3150 2024-03-06T07:20:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 -- 172.21.15.26:0/1861293096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 msgr2=0x7fcdbc0c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 --2- 172.21.15.26:0/1861293096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc0c5690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0097a0 tx=0x7fcdac02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 -- 172.21.15.26:0/1861293096 shutdown_connections 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 --2- 172.21.15.26:0/1861293096 >> v2:172.21.15.26:3300/0 conn(0x7fcdbc152c20 0x7fcdbc0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 --2- 172.21.15.26:0/1861293096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc0c5690 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 -- 172.21.15.26:0/1861293096 >> 172.21.15.26:0/1861293096 conn(0x7fcdbc0c4290 msgr2=0x7fcdbc0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 -- 172.21.15.26:0/1861293096 shutdown_connections 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.962+0000 7fcdc2864640 1 -- 172.21.15.26:0/1861293096 wait complete. 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.963+0000 7fcdc2864640 1 Processor -- start 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc2864640 1 -- start start 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc2864640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc2864640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdbc14f650 0x7fcdbc0ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc2864640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc152c20 0x7fcdbc153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc2864640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc072110 con 0x7fcdbc152c20 2024-03-06T07:20:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc2864640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc071f90 con 0x7fcdbc14f650 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc2864640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc0c3f40 con 0x7fcdbc0c3150 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc0e01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc152c20 0x7fcdbc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc0e01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc152c20 0x7fcdbc153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42534/0 (socket says 172.21.15.26:42534) 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdc0e01640 1 -- 172.21.15.26:0/950457165 learned_addr learned my addr 172.21.15.26:0/950457165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdbbfff640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.964+0000 7fcdbb7fe640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdbc14f650 0x7fcdbc0ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc0e01640 1 -- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 msgr2=0x7fcdbc155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc0e01640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc0e01640 1 -- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdbc14f650 msgr2=0x7fcdbc0ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc0e01640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdbc14f650 0x7fcdbc0ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc0e01640 1 -- 172.21.15.26:0/950457165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc155bd0 con 0x7fcdbc152c20 2024-03-06T07:20:06.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc0e01640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc152c20 0x7fcdbc153740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009cb0 tx=0x7fcdb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:06.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdb8ff9640 1 -- 172.21.15.26:0/950457165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4018020 con 0x7fcdbc152c20 2024-03-06T07:20:06.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdb8ff9640 1 -- 172.21.15.26:0/950457165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb4014480 con 0x7fcdbc152c20 2024-03-06T07:20:06.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdb8ff9640 1 -- 172.21.15.26:0/950457165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4014870 con 0x7fcdbc152c20 2024-03-06T07:20:06.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc155eb0 con 0x7fcdbc152c20 2024-03-06T07:20:06.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.965+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc154d60 con 0x7fcdbc152c20 2024-03-06T07:20:06.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.966+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc14d950 con 0x7fcdbc152c20 2024-03-06T07:20:06.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.966+0000 7fcdb8ff9640 1 -- 172.21.15.26:0/950457165 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdb4005d10 con 0x7fcdbc152c20 2024-03-06T07:20:06.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.967+0000 7fcdb8ff9640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd9c06b500 0x7fcd9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:06.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.967+0000 7fcdb8ff9640 1 -- 172.21.15.26:0/950457165 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcdb408b090 con 0x7fcdbc152c20 2024-03-06T07:20:06.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.967+0000 7fcdbbfff640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd9c06b500 0x7fcd9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:06.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.967+0000 7fcdbbfff640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd9c06b500 0x7fcd9c06d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcda4000f60 tx=0x7fcda4007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:06.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:06.971+0000 7fcdb8ff9640 1 -- 172.21.15.26:0/950457165 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcdbc14d950 con 0x7fcdbc152c20 2024-03-06T07:20:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.189+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdbc04f330 con 0x7fcdbc152c20 2024-03-06T07:20:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.190+0000 7fcdb8ff9640 1 -- 172.21.15.26:0/950457165 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4973 (secure 0 0 0) 0x7fcdb4022030 con 0x7fcdbc152c20 2024-03-06T07:20:07.193 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:07.193 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:06.210418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-03-06T07:20:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd9c06b500 msgr2=0x7fcd9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd9c06b500 0x7fcd9c06d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcda4000f60 tx=0x7fcda4007b50 comp rx=0 tx=0).stop 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc152c20 msgr2=0x7fcdbc153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc152c20 0x7fcdbc153740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009cb0 tx=0x7fcdb400c490 comp rx=0 tx=0).stop 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 shutdown_connections 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd9c06b500 0x7fcd9c06d9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdbc152c20 0x7fcdbc153740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdbc14f650 0x7fcdbc0ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 --2- 172.21.15.26:0/950457165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdbc0c3150 0x7fcdbc155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 >> 172.21.15.26:0/950457165 conn(0x7fcdbc0c4290 msgr2=0x7fcdbc0c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 shutdown_connections 2024-03-06T07:20:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.192+0000 7fcdc2864640 1 -- 172.21.15.26:0/950457165 wait complete. 2024-03-06T07:20:07.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:07.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.305+0000 7f1e78881640 1 Processor -- start 2024-03-06T07:20:07.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.305+0000 7f1e78881640 1 -- start start 2024-03-06T07:20:07.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e78881640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e740c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e78881640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1e74158db0 0x7f1e7415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e78881640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e74155110 con 0x7f1e740c88b0 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e78881640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e741559e0 con 0x7f1e74158db0 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e78881640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c8f40 con 0x7f1e740c5330 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e71d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1e740c88b0 0x7f1e741586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36428/0 (socket says 172.21.15.26:36428) 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e71d9b640 1 -- 172.21.15.26:0/1035674708 learned_addr learned my addr 172.21.15.26:0/1035674708 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e72d9d640 1 --2- 172.21.15.26:0/1035674708 >> v2:172.21.15.117:3300/0 conn(0x7f1e74158db0 0x7f1e7415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.306+0000 7f1e7259c640 1 --2- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e740c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e7159a640 1 -- 172.21.15.26:0/1035674708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566797982 0 0) 0x7f1e68001110 con 0x7f1e740c88b0 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e7159a640 1 -- 172.21.15.26:0/1035674708 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e5c003520 con 0x7f1e740c88b0 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e72d9d640 1 -- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 msgr2=0x7f1e740c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e72d9d640 1 --2- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e740c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e72d9d640 1 -- 172.21.15.26:0/1035674708 >> v1:172.21.15.26:6789/0 conn(0x7f1e740c88b0 legacy=0x7f1e741586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e72d9d640 1 -- 172.21.15.26:0/1035674708 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74155390 con 0x7f1e74158db0 2024-03-06T07:20:07.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e7259c640 1 --2- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e740c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.307+0000 7f1e72d9d640 1 --2- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e74158db0 0x7f1e7415fa30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009d90 tx=0x7f1e6402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220a75413106ce01 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e7159a640 1 -- 172.21.15.26:0/1035674708 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64039070 con 0x7f1e74158db0 2024-03-06T07:20:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e7159a640 1 -- 172.21.15.26:0/1035674708 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6402cbc0 con 0x7f1e74158db0 2024-03-06T07:20:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e7159a640 1 -- 172.21.15.26:0/1035674708 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64034440 con 0x7f1e74158db0 2024-03-06T07:20:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 -- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e74158db0 msgr2=0x7f1e7415fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 --2- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e74158db0 0x7f1e7415fa30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009d90 tx=0x7f1e6402be10 comp rx=0 tx=0).stop 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 -- 172.21.15.26:0/1035674708 shutdown_connections 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 --2- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e74158db0 0x7f1e7415fa30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 --2- 172.21.15.26:0/1035674708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e740c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 -- 172.21.15.26:0/1035674708 >> 172.21.15.26:0/1035674708 conn(0x7f1e740ac560 msgr2=0x7f1e740ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 -- 172.21.15.26:0/1035674708 shutdown_connections 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.308+0000 7f1e78881640 1 -- 172.21.15.26:0/1035674708 wait complete. 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.309+0000 7f1e78881640 1 Processor -- start 2024-03-06T07:20:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e78881640 1 -- start start 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e78881640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e741584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e78881640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e740c88b0 0x7f1e74151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e78881640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7414bce0 0x7f1e7414b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e78881640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e741529d0 con 0x7f1e7414bce0 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e78881640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7406d090 con 0x7f1e740c88b0 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e78881640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74152b50 con 0x7f1e740c5330 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e72d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7414bce0 0x7f1e7414b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e72d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7414bce0 0x7f1e7414b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59234/0 (socket says 172.21.15.26:59234) 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e72d9d640 1 -- 172.21.15.26:0/4115386807 learned_addr learned my addr 172.21.15.26:0/4115386807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e7259c640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e741584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.310+0000 7f1e71d9b640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e740c88b0 0x7f1e74151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e72d9d640 1 -- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 msgr2=0x7f1e741584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e72d9d640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e741584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e72d9d640 1 -- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e740c88b0 msgr2=0x7f1e74151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e72d9d640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e740c88b0 0x7f1e74151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e72d9d640 1 -- 172.21.15.26:0/4115386807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740bdd60 con 0x7f1e7414bce0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e72d9d640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7414bce0 0x7f1e7414b080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009d90 tx=0x7f1e64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e62ffd640 1 -- 172.21.15.26:0/4115386807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64039050 con 0x7f1e7414bce0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e62ffd640 1 -- 172.21.15.26:0/4115386807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e64045070 con 0x7f1e7414bce0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e74156540 con 0x7f1e7414bce0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e62ffd640 1 -- 172.21.15.26:0/4115386807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64009450 con 0x7f1e7414bce0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.311+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e74156b00 con 0x7f1e7414bce0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.312+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7404f330 con 0x7f1e7414bce0 2024-03-06T07:20:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.312+0000 7f1e62ffd640 1 -- 172.21.15.26:0/4115386807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e6403e850 con 0x7f1e7414bce0 2024-03-06T07:20:07.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.313+0000 7f1e62ffd640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e5406b4d0 0x7f1e5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.313+0000 7f1e62ffd640 1 -- 172.21.15.26:0/4115386807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1e640ad180 con 0x7f1e7414bce0 2024-03-06T07:20:07.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.313+0000 7f1e7259c640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e5406b4d0 0x7f1e5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.313+0000 7f1e7259c640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e5406b4d0 0x7f1e5406d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009560 tx=0x7f1e6c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.317+0000 7f1e62ffd640 1 -- 172.21.15.26:0/4115386807 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e6407b820 con 0x7f1e7414bce0 2024-03-06T07:20:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.501+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e7406cdc0 con 0x7f1e7414bce0 2024-03-06T07:20:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.502+0000 7f1e62ffd640 1 -- 172.21.15.26:0/4115386807 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f1e6407f8c0 con 0x7f1e7414bce0 2024-03-06T07:20:07.505 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:07.505 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:07.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e5406b4d0 msgr2=0x7f1e5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e5406b4d0 0x7f1e5406d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009560 tx=0x7f1e6c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7414bce0 msgr2=0x7f1e7414b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7414bce0 0x7f1e7414b080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009d90 tx=0x7f1e64004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 shutdown_connections 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e5406b4d0 0x7f1e5406d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7414bce0 0x7f1e7414b080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e740c88b0 0x7f1e74151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 --2- 172.21.15.26:0/4115386807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e740c5330 0x7f1e741584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 >> 172.21.15.26:0/4115386807 conn(0x7f1e740ac560 msgr2=0x7f1e740ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 shutdown_connections 2024-03-06T07:20:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.504+0000 7f1e78881640 1 -- 172.21.15.26:0/4115386807 wait complete. 2024-03-06T07:20:07.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:07.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.615+0000 7fa56d22a640 1 Processor -- start 2024-03-06T07:20:07.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa56d22a640 1 -- start start 2024-03-06T07:20:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa56d22a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5681536c0 0x7fa56806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa56d22a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa56806b3a0 0x7fa56815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa56d22a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56806b7d0 con 0x7fa56806afc0 2024-03-06T07:20:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa56d22a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa568150a20 con 0x7fa5681536c0 2024-03-06T07:20:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa56d22a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5681565a0 con 0x7fa56806b3a0 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa55ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5681536c0 0x7fa56806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa55ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5681536c0 0x7fa56806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59246/0 (socket says 172.21.15.26:59246) 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa566d9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa56806afc0 0x7fa568152fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:58624/0 (socket says 172.21.15.26:58624) 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa55ffff640 1 -- 172.21.15.26:0/175962694 learned_addr learned my addr 172.21.15.26:0/175962694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.616+0000 7fa56759e640 1 --2- 172.21.15.26:0/175962694 >> v2:172.21.15.117:3300/0 conn(0x7fa56806b3a0 0x7fa56815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.617+0000 7fa56659c640 1 -- 172.21.15.26:0/175962694 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004025988 0 0) 0x7fa56806b7d0 con 0x7fa56806afc0 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.617+0000 7fa55ffff640 1 -- 172.21.15.26:0/175962694 >> v2:172.21.15.117:3300/0 conn(0x7fa56806b3a0 msgr2=0x7fa56815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.617+0000 7fa55ffff640 1 --2- 172.21.15.26:0/175962694 >> v2:172.21.15.117:3300/0 conn(0x7fa56806b3a0 0x7fa56815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.617+0000 7fa55ffff640 1 -- 172.21.15.26:0/175962694 >> v1:172.21.15.117:6789/0 conn(0x7fa56806afc0 legacy=0x7fa568152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.617+0000 7fa55ffff640 1 -- 172.21.15.26:0/175962694 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680a70f0 con 0x7fa5681536c0 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.617+0000 7fa55ffff640 1 --2- 172.21.15.26:0/175962694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681536c0 0x7fa56806f100 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa568150ba0 tx=0x7fa55002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d30b7921e05c9ea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56659c640 1 -- 172.21.15.26:0/175962694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55002e070 con 0x7fa5681536c0 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56659c640 1 -- 172.21.15.26:0/175962694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550037070 con 0x7fa5681536c0 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56659c640 1 -- 172.21.15.26:0/175962694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550031a00 con 0x7fa5681536c0 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 -- 172.21.15.26:0/175962694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681536c0 msgr2=0x7fa56806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 --2- 172.21.15.26:0/175962694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681536c0 0x7fa56806f100 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa568150ba0 tx=0x7fa55002b880 comp rx=0 tx=0).stop 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 -- 172.21.15.26:0/175962694 shutdown_connections 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 --2- 172.21.15.26:0/175962694 >> v2:172.21.15.117:3300/0 conn(0x7fa56806b3a0 0x7fa56815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 --2- 172.21.15.26:0/175962694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681536c0 0x7fa56806f100 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 -- 172.21.15.26:0/175962694 >> 172.21.15.26:0/175962694 conn(0x7fa5680c4290 msgr2=0x7fa5680c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 -- 172.21.15.26:0/175962694 shutdown_connections 2024-03-06T07:20:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.618+0000 7fa56d22a640 1 -- 172.21.15.26:0/175962694 wait complete. 2024-03-06T07:20:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.619+0000 7fa56d22a640 1 Processor -- start 2024-03-06T07:20:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.619+0000 7fa56d22a640 1 -- start start 2024-03-06T07:20:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa56d22a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa56806afc0 0x7fa5681dff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa56d22a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa56806b3a0 0x7fa5680abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa56d22a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5681536c0 0x7fa5680ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa56d22a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa568156db0 con 0x7fa56806afc0 2024-03-06T07:20:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa56d22a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa568156ad0 con 0x7fa56806b3a0 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa56d22a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5680c7360 con 0x7fa5681536c0 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa56806afc0 0x7fa5681dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa56806afc0 0x7fa5681dff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59260/0 (socket says 172.21.15.26:59260) 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa56806b3a0 0x7fa5680abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 -- 172.21.15.26:0/2730110678 learned_addr learned my addr 172.21.15.26:0/2730110678 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa56759e640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5681536c0 0x7fa5680ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 -- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5681536c0 msgr2=0x7fa5680ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5681536c0 0x7fa5680ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 -- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa56806b3a0 msgr2=0x7fa5680abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa56806b3a0 0x7fa5680abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.620+0000 7fa566d9d640 1 -- 172.21.15.26:0/2730110678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5681ecf30 con 0x7fa56806afc0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.621+0000 7fa566d9d640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa56806afc0 0x7fa5681dff30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c460 tx=0x7fa55800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.621+0000 7fa55f7fe640 1 -- 172.21.15.26:0/2730110678 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558018070 con 0x7fa56806afc0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.621+0000 7fa55f7fe640 1 -- 172.21.15.26:0/2730110678 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa558007500 con 0x7fa56806afc0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.621+0000 7fa55f7fe640 1 -- 172.21.15.26:0/2730110678 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558004a20 con 0x7fa56806afc0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.621+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5680aa880 con 0x7fa56806afc0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.621+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5681e0470 con 0x7fa56806afc0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.622+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56804f330 con 0x7fa56806afc0 2024-03-06T07:20:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.622+0000 7fa55f7fe640 1 -- 172.21.15.26:0/2730110678 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa55801b070 con 0x7fa56806afc0 2024-03-06T07:20:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.623+0000 7fa55f7fe640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa54806b500 0x7fa54806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.623+0000 7fa55f7fe640 1 -- 172.21.15.26:0/2730110678 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa558089680 con 0x7fa56806afc0 2024-03-06T07:20:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.623+0000 7fa55ffff640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa54806b500 0x7fa54806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.623+0000 7fa55ffff640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa54806b500 0x7fa54806d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa550036000 tx=0x7fa550008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.627+0000 7fa55f7fe640 1 -- 172.21.15.26:0/2730110678 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa558057b70 con 0x7fa56806afc0 2024-03-06T07:20:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.812+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa568068460 con 0x7fa56806afc0 2024-03-06T07:20:07.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.814+0000 7fa55f7fe640 1 -- 172.21.15.26:0/2730110678 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fa55805bc10 con 0x7fa56806afc0 2024-03-06T07:20:07.816 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:07.817 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa54806b500 msgr2=0x7fa54806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa54806b500 0x7fa54806d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa550036000 tx=0x7fa550008030 comp rx=0 tx=0).stop 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa56806afc0 msgr2=0x7fa5681dff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa56806afc0 0x7fa5681dff30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c460 tx=0x7fa55800c920 comp rx=0 tx=0).stop 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 shutdown_connections 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa54806b500 0x7fa54806d9c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5681536c0 0x7fa5680ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa56806b3a0 0x7fa5680abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 --2- 172.21.15.26:0/2730110678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa56806afc0 0x7fa5681dff30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 >> 172.21.15.26:0/2730110678 conn(0x7fa5680c4290 msgr2=0x7fa568152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 shutdown_connections 2024-03-06T07:20:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.816+0000 7fa56d22a640 1 -- 172.21.15.26:0/2730110678 wait complete. 2024-03-06T07:20:07.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.944+0000 7efee7e76640 1 Processor -- start 2024-03-06T07:20:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.944+0000 7efee7e76640 1 -- start start 2024-03-06T07:20:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee7e76640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efee014f360 0x7efee0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee7e76640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efee0158b60 0x7efee0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee7e76640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee00c73d0 con 0x7efee00c88c0 2024-03-06T07:20:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee7e76640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00c8dd0 con 0x7efee0158b60 2024-03-06T07:20:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee7e76640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00be2e0 con 0x7efee014f360 2024-03-06T07:20:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee6413640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efee0158b60 0x7efee0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee5411640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efee00c88c0 0x7efee00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36430/0 (socket says 172.21.15.26:36430) 2024-03-06T07:20:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee5411640 1 -- 172.21.15.26:0/1218770865 learned_addr learned my addr 172.21.15.26:0/1218770865 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee5c12640 1 --2- 172.21.15.26:0/1218770865 >> v2:172.21.15.117:3300/0 conn(0x7efee014f360 0x7efee0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee4c10640 1 -- 172.21.15.26:0/1218770865 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606797624 0 0) 0x7efee00c73d0 con 0x7efee00c88c0 2024-03-06T07:20:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee4c10640 1 -- 172.21.15.26:0/1218770865 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efecc003520 con 0x7efee00c88c0 2024-03-06T07:20:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee6413640 1 -- 172.21.15.26:0/1218770865 >> v2:172.21.15.117:3300/0 conn(0x7efee014f360 msgr2=0x7efee0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee6413640 1 --2- 172.21.15.26:0/1218770865 >> v2:172.21.15.117:3300/0 conn(0x7efee014f360 0x7efee0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee6413640 1 -- 172.21.15.26:0/1218770865 >> v1:172.21.15.26:6789/0 conn(0x7efee00c88c0 legacy=0x7efee00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.945+0000 7efee6413640 1 -- 172.21.15.26:0/1218770865 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00c7670 con 0x7efee0158b60 2024-03-06T07:20:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.946+0000 7efee6413640 1 --2- 172.21.15.26:0/1218770865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee0158b60 0x7efee0158ff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efee008c840 tx=0x7efed802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773a1a87b26105db server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.946+0000 7efee4c10640 1 -- 172.21.15.26:0/1218770865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8039070 con 0x7efee0158b60 2024-03-06T07:20:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.946+0000 7efee4c10640 1 -- 172.21.15.26:0/1218770865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed802cbf0 con 0x7efee0158b60 2024-03-06T07:20:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.946+0000 7efee4c10640 1 -- 172.21.15.26:0/1218770865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8034440 con 0x7efee0158b60 2024-03-06T07:20:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 -- 172.21.15.26:0/1218770865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee0158b60 msgr2=0x7efee0158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 --2- 172.21.15.26:0/1218770865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee0158b60 0x7efee0158ff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efee008c840 tx=0x7efed802be40 comp rx=0 tx=0).stop 2024-03-06T07:20:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 -- 172.21.15.26:0/1218770865 shutdown_connections 2024-03-06T07:20:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 --2- 172.21.15.26:0/1218770865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee0158b60 0x7efee0158ff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 --2- 172.21.15.26:0/1218770865 >> v2:172.21.15.117:3300/0 conn(0x7efee014f360 0x7efee0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 -- 172.21.15.26:0/1218770865 >> 172.21.15.26:0/1218770865 conn(0x7efee00ac570 msgr2=0x7efee00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 -- 172.21.15.26:0/1218770865 shutdown_connections 2024-03-06T07:20:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.947+0000 7efee7e76640 1 -- 172.21.15.26:0/1218770865 wait complete. 2024-03-06T07:20:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.948+0000 7efee7e76640 1 Processor -- start 2024-03-06T07:20:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.948+0000 7efee7e76640 1 -- start start 2024-03-06T07:20:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee00c88c0 0x7efee01ee770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee014f360 0x7efee014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efee0158b60 0x7efee014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00c5a50 con 0x7efee014f360 2024-03-06T07:20:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00c7c30 con 0x7efee00c88c0 2024-03-06T07:20:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00c5bd0 con 0x7efee0158b60 2024-03-06T07:20:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee014f360 0x7efee014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee014f360 0x7efee014bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59276/0 (socket says 172.21.15.26:59276) 2024-03-06T07:20:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 -- 172.21.15.26:0/1233066057 learned_addr learned my addr 172.21.15.26:0/1233066057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5c12640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee00c88c0 0x7efee01ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee6413640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efee0158b60 0x7efee014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 -- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efee0158b60 msgr2=0x7efee014c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efee0158b60 0x7efee014c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 -- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee00c88c0 msgr2=0x7efee01ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee00c88c0 0x7efee01ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 -- 172.21.15.26:0/1233066057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee01ef440 con 0x7efee014f360 2024-03-06T07:20:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee5411640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee014f360 0x7efee014bbe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efed0001eb0 tx=0x7efed000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efed67fc640 1 -- 172.21.15.26:0/1233066057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0017070 con 0x7efee014f360 2024-03-06T07:20:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efed67fc640 1 -- 172.21.15.26:0/1233066057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0005d00 con 0x7efee014f360 2024-03-06T07:20:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efed67fc640 1 -- 172.21.15.26:0/1233066057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0007310 con 0x7efee014f360 2024-03-06T07:20:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee01ef160 con 0x7efee014f360 2024-03-06T07:20:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.949+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01ec7f0 con 0x7efee014f360 2024-03-06T07:20:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.950+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee004f330 con 0x7efee014f360 2024-03-06T07:20:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.951+0000 7efed67fc640 1 -- 172.21.15.26:0/1233066057 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efed001a070 con 0x7efee014f360 2024-03-06T07:20:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.951+0000 7efed67fc640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efec406b500 0x7efec406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.951+0000 7efed67fc640 1 -- 172.21.15.26:0/1233066057 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efed0089780 con 0x7efee014f360 2024-03-06T07:20:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.951+0000 7efee5c12640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efec406b500 0x7efec406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.952+0000 7efee5c12640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efec406b500 0x7efec406d9c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efedc009b40 tx=0x7efedc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:07.955+0000 7efed67fc640 1 -- 172.21.15.26:0/1233066057 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efed0057d00 con 0x7efee014f360 2024-03-06T07:20:08.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.173+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efee00c1640 con 0x7efee014f360 2024-03-06T07:20:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.173+0000 7efed67fc640 1 -- 172.21.15.26:0/1233066057 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4973 (secure 0 0 0) 0x7efed005bda0 con 0x7efee014f360 2024-03-06T07:20:08.176 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:08.176 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:06.210418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:08.177 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efec406b500 msgr2=0x7efec406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efec406b500 0x7efec406d9c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efedc009b40 tx=0x7efedc009190 comp rx=0 tx=0).stop 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee014f360 msgr2=0x7efee014bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee014f360 0x7efee014bbe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efed0001eb0 tx=0x7efed000c820 comp rx=0 tx=0).stop 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 shutdown_connections 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efec406b500 0x7efec406d9c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efee0158b60 0x7efee014c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efee014f360 0x7efee014bbe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 --2- 172.21.15.26:0/1233066057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee00c88c0 0x7efee01ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 >> 172.21.15.26:0/1233066057 conn(0x7efee00ac570 msgr2=0x7efee00c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 shutdown_connections 2024-03-06T07:20:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.175+0000 7efee7e76640 1 -- 172.21.15.26:0/1233066057 wait complete. 2024-03-06T07:20:08.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:08.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.287+0000 7fe92759e640 1 Processor -- start 2024-03-06T07:20:08.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.287+0000 7fe92759e640 1 -- start start 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe92759e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe928152530 0x7fe92806d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe92759e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe92759e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe928152910 con 0x7fe9280c0370 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe92759e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9280c08c0 con 0x7fe928152530 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe92759e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280c0ba0 con 0x7fe928155890 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe925d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe9280c0370 0x7fe92806b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36436/0 (socket says 172.21.15.26:36436) 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe925d9b640 1 -- 172.21.15.26:0/2697393067 learned_addr learned my addr 172.21.15.26:0/2697393067 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe92659c640 1 --2- 172.21.15.26:0/2697393067 >> v2:172.21.15.117:3300/0 conn(0x7fe928152530 0x7fe92806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.288+0000 7fe926d9d640 1 --2- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.289+0000 7fe92559a640 1 -- 172.21.15.26:0/2697393067 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184286763 0 0) 0x7fe928152910 con 0x7fe9280c0370 2024-03-06T07:20:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.289+0000 7fe92559a640 1 -- 172.21.15.26:0/2697393067 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe914003520 con 0x7fe9280c0370 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.289+0000 7fe92659c640 1 -- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 msgr2=0x7fe92806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.289+0000 7fe92659c640 1 --2- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92806ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.289+0000 7fe92659c640 1 -- 172.21.15.26:0/2697393067 >> v1:172.21.15.26:6789/0 conn(0x7fe9280c0370 legacy=0x7fe92806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.289+0000 7fe92659c640 1 -- 172.21.15.26:0/2697393067 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe928155090 con 0x7fe928152530 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.289+0000 7fe92659c640 1 --2- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92806d920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9200097a0 tx=0x7fe92002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955b1412c79beb3d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92559a640 1 -- 172.21.15.26:0/2697393067 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200315a0 con 0x7fe928152530 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92559a640 1 -- 172.21.15.26:0/2697393067 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe920037070 con 0x7fe928152530 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92559a640 1 -- 172.21.15.26:0/2697393067 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920031960 con 0x7fe928152530 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 -- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 msgr2=0x7fe92806d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 --2- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92806d920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9200097a0 tx=0x7fe92002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 -- 172.21.15.26:0/2697393067 shutdown_connections 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 --2- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92806ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 --2- 172.21.15.26:0/2697393067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92806d920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 -- 172.21.15.26:0/2697393067 >> 172.21.15.26:0/2697393067 conn(0x7fe9280c4db0 msgr2=0x7fe9280c5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 -- 172.21.15.26:0/2697393067 shutdown_connections 2024-03-06T07:20:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.290+0000 7fe92759e640 1 -- 172.21.15.26:0/2697393067 wait complete. 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.291+0000 7fe92759e640 1 Processor -- start 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.291+0000 7fe92759e640 1 -- start start 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9280c0370 0x7fe9281549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92759e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92759e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe928155e80 con 0x7fe9280c0370 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92759e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe928155d00 con 0x7fe928152530 2024-03-06T07:20:08.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92759e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92806bf80 con 0x7fe928155890 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9280c0370 0x7fe9281549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9280c0370 0x7fe9281549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59290/0 (socket says 172.21.15.26:59290) 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe92659c640 1 -- 172.21.15.26:0/2074411675 learned_addr learned my addr 172.21.15.26:0/2074411675 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe925d9b640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.292+0000 7fe926d9d640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92659c640 1 -- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 msgr2=0x7fe92814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92659c640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92659c640 1 -- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 msgr2=0x7fe92814d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92659c640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92814d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92659c640 1 -- 172.21.15.26:0/2074411675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe928153e80 con 0x7fe9280c0370 2024-03-06T07:20:08.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe925d9b640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92659c640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9280c0370 0x7fe9281549f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe9200098c0 tx=0x7fe9200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe91effd640 1 -- 172.21.15.26:0/2074411675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920032630 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe91effd640 1 -- 172.21.15.26:0/2074411675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9200327f0 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe91effd640 1 -- 172.21.15.26:0/2074411675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920032be0 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9281eb3d0 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.293+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9281eb950 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.294+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92804f330 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.294+0000 7fe91effd640 1 -- 172.21.15.26:0/2074411675 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe92004e430 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.295+0000 7fe91effd640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe90806b4d0 0x7fe90806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.295+0000 7fe91effd640 1 -- 172.21.15.26:0/2074411675 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe92002f030 con 0x7fe9280c0370 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.295+0000 7fe925d9b640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe90806b4d0 0x7fe90806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.295+0000 7fe925d9b640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe90806b4d0 0x7fe90806d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe9281540f0 tx=0x7fe910009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:08.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.299+0000 7fe91effd640 1 -- 172.21.15.26:0/2074411675 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe92804f330 con 0x7fe9280c0370 2024-03-06T07:20:08.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.482+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe928074890 con 0x7fe9280c0370 2024-03-06T07:20:08.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.483+0000 7fe91effd640 1 -- 172.21.15.26:0/2074411675 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fe928074890 con 0x7fe9280c0370 2024-03-06T07:20:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:08.487 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:08.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe90806b4d0 msgr2=0x7fe90806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe90806b4d0 0x7fe90806d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe9281540f0 tx=0x7fe910009040 comp rx=0 tx=0).stop 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9280c0370 msgr2=0x7fe9281549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9280c0370 0x7fe9281549f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe9200098c0 tx=0x7fe9200046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 shutdown_connections 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe90806b4d0 0x7fe90806d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe928155890 0x7fe92814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe928152530 0x7fe92814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 --2- 172.21.15.26:0/2074411675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9280c0370 0x7fe9281549f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 >> 172.21.15.26:0/2074411675 conn(0x7fe9280c4db0 msgr2=0x7fe9281e16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 shutdown_connections 2024-03-06T07:20:08.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:08.486+0000 7fe92759e640 1 -- 172.21.15.26:0/2074411675 wait complete. 2024-03-06T07:20:08.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:20:01.171153+0000', 'modified': '2024-03-06T07:20:06.210418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/3090618513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3090618513}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3090618513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4263, 'qdb_cluster': [4263]} 2024-03-06T07:20:08.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:20:08.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:20:09.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.005+0000 7fc71b393640 1 Processor -- start 2024-03-06T07:20:09.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.006+0000 7fc71b393640 1 -- start start 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.006+0000 7fc71b393640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc714072f40 0x7fc7140c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.006+0000 7fc71b393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc714152060 0x7fc71414e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.006+0000 7fc71b393640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7140c1140 con 0x7fc71406d680 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.006+0000 7fc71b393640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140c2e30 con 0x7fc714072f40 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.006+0000 7fc71b393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140c1d50 con 0x7fc714152060 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc71912f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc71406d680 0x7fc7140c21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36442/0 (socket says 172.21.15.26:36442) 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc71912f640 1 -- 172.21.15.26:0/695933962 learned_addr learned my addr 172.21.15.26:0/695933962 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc719930640 1 --2- 172.21.15.26:0/695933962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc714152060 0x7fc71414e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc71892e640 1 --2- 172.21.15.26:0/695933962 >> v2:172.21.15.117:3300/0 conn(0x7fc714072f40 0x7fc7140c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc70bfff640 1 -- 172.21.15.26:0/695933962 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608436192 0 0) 0x7fc7140c1140 con 0x7fc71406d680 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc70bfff640 1 -- 172.21.15.26:0/695933962 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc700003520 con 0x7fc71406d680 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc719930640 1 -- 172.21.15.26:0/695933962 >> v2:172.21.15.117:3300/0 conn(0x7fc714072f40 msgr2=0x7fc7140c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc719930640 1 --2- 172.21.15.26:0/695933962 >> v2:172.21.15.117:3300/0 conn(0x7fc714072f40 0x7fc7140c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc719930640 1 -- 172.21.15.26:0/695933962 >> v1:172.21.15.26:6789/0 conn(0x7fc71406d680 legacy=0x7fc7140c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.007+0000 7fc719930640 1 -- 172.21.15.26:0/695933962 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140c1420 con 0x7fc714152060 2024-03-06T07:20:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc719930640 1 --2- 172.21.15.26:0/695933962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc714152060 0x7fc71414e440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009dc0 tx=0x7fc70c02be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e735f26370c50a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc70bfff640 1 -- 172.21.15.26:0/695933962 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c039070 con 0x7fc714152060 2024-03-06T07:20:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc70bfff640 1 -- 172.21.15.26:0/695933962 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c02cbb0 con 0x7fc714152060 2024-03-06T07:20:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc70bfff640 1 -- 172.21.15.26:0/695933962 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c034440 con 0x7fc714152060 2024-03-06T07:20:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc71b393640 1 -- 172.21.15.26:0/695933962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc714152060 msgr2=0x7fc71414e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc71b393640 1 --2- 172.21.15.26:0/695933962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc714152060 0x7fc71414e440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009dc0 tx=0x7fc70c02be00 comp rx=0 tx=0).stop 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc71b393640 1 -- 172.21.15.26:0/695933962 shutdown_connections 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc71b393640 1 --2- 172.21.15.26:0/695933962 >> v2:172.21.15.117:3300/0 conn(0x7fc714072f40 0x7fc7140c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc71b393640 1 --2- 172.21.15.26:0/695933962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc714152060 0x7fc71414e440 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc71b393640 1 -- 172.21.15.26:0/695933962 >> 172.21.15.26:0/695933962 conn(0x7fc71406b1a0 msgr2=0x7fc71406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.008+0000 7fc71b393640 1 -- 172.21.15.26:0/695933962 shutdown_connections 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.009+0000 7fc71b393640 1 -- 172.21.15.26:0/695933962 wait complete. 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 Processor -- start 2024-03-06T07:20:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 -- start start 2024-03-06T07:20:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc71406d680 0x7fc7141451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714072f40 0x7fc7140a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714152060 0x7fc7141ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141524f0 con 0x7fc714072f40 2024-03-06T07:20:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140c19e0 con 0x7fc714152060 2024-03-06T07:20:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71b393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71406da60 con 0x7fc71406d680 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71892e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714072f40 0x7fc7140a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71892e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714072f40 0x7fc7140a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59306/0 (socket says 172.21.15.26:59306) 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71892e640 1 -- 172.21.15.26:0/2158224301 learned_addr learned my addr 172.21.15.26:0/2158224301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc719930640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714152060 0x7fc7141ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.010+0000 7fc71912f640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc71406d680 0x7fc7141451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71892e640 1 -- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc71406d680 msgr2=0x7fc7141451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71892e640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc71406d680 0x7fc7141451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71892e640 1 -- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714152060 msgr2=0x7fc7141ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71892e640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714152060 0x7fc7141ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71892e640 1 -- 172.21.15.26:0/2158224301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714142840 con 0x7fc714072f40 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71892e640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714072f40 0x7fc7140a98c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc70400e980 tx=0x7fc70400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc709ffb640 1 -- 172.21.15.26:0/2158224301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704017070 con 0x7fc714072f40 2024-03-06T07:20:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc709ffb640 1 -- 172.21.15.26:0/2158224301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400ca00 con 0x7fc714072f40 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc714142b20 con 0x7fc714072f40 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc709ffb640 1 -- 172.21.15.26:0/2158224301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400cdd0 con 0x7fc714072f40 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.011+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7141430e0 con 0x7fc714072f40 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.012+0000 7fc709ffb640 1 -- 172.21.15.26:0/2158224301 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc70401a070 con 0x7fc714072f40 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.013+0000 7fc709ffb640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc6f806b4d0 0x7fc6f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.013+0000 7fc709ffb640 1 -- 172.21.15.26:0/2158224301 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc70408a400 con 0x7fc714072f40 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.013+0000 7fc71912f640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc6f806b4d0 0x7fc6f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.013+0000 7fc71912f640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc6f806b4d0 0x7fc6f806d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc710005880 tx=0x7fc710009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.014+0000 7fc71df7e740 1 Processor -- start 2024-03-06T07:20:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.014+0000 7fc71df7e740 1 -- start start 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.014+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94ace010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.014+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94ace550 0x560a94b17be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.014+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94b08420 0x560a94ac8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.014+0000 7fc71df7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a94a326b0 con 0x560a94b17800 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.014+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a949dba60 con 0x560a94b08420 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a94a2e750 con 0x560a94ace550 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94ace010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94ace010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59322/0 (socket says 172.21.15.26:59322) 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 -- 172.21.15.26:0/2081751258 learned_addr learned my addr 172.21.15.26:0/2081751258 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc719930640 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94b08420 0x560a94ac8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71892e640 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94ace550 0x560a94b17be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 -- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94ace550 msgr2=0x560a94b17be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94ace550 0x560a94b17be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 -- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94b08420 msgr2=0x560a94ac8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94b08420 0x560a94ac8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 -- 172.21.15.26:0/2081751258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a949f9f30 con 0x560a94b17800 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc71912f640 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94ace010 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc71000fab0 tx=0x7fc710000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32dbdfaa37039e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc6feffd640 1 -- 172.21.15.26:0/2081751258 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710025070 con 0x560a94b17800 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc6feffd640 1 -- 172.21.15.26:0/2081751258 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc710015bf0 con 0x560a94b17800 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.015+0000 7fc6feffd640 1 -- 172.21.15.26:0/2081751258 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710020400 con 0x560a94b17800 2024-03-06T07:20:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 -- 172.21.15.26:0/2081751258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 msgr2=0x560a94ace010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94ace010 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc71000fab0 tx=0x7fc710000ed0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 -- 172.21.15.26:0/2081751258 shutdown_connections 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94b08420 0x560a94ac8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94ace550 0x560a94b17be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2081751258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94ace010 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 -- 172.21.15.26:0/2081751258 >> 172.21.15.26:0/2081751258 conn(0x560a94aca7d0 msgr2=0x560a94aef3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 -- 172.21.15.26:0/2081751258 shutdown_connections 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 -- 172.21.15.26:0/2081751258 wait complete. 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 -- start start 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.016+0000 7fc71df7e740 1 Processor -- start 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94acd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94b1f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b164f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71df7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a94b0c590 con 0x560a94b17800 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a949eeea0 con 0x560a94af3380 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a94aefdb0 con 0x560a94b08420 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71df7e740 10 client.0 _ll_register_callbacks cb 0x560a949ee870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71892e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94b1f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71892e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94b1f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59336/0 (socket says 172.21.15.26:59336) 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71892e640 1 -- 172.21.15.26:0/2674103805 learned_addr learned my addr 172.21.15.26:0/2674103805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc71912f640 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94acd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.017+0000 7fc719930640 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71892e640 1 -- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 msgr2=0x560a94acd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71892e640 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94acd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71892e640 1 -- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 msgr2=0x560a94b164f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71892e640 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b164f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71892e640 1 -- 172.21.15.26:0/2674103805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a949ef270 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71892e640 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94b1f390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc70400c520 tx=0x7fc704058980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731dd75e0bd7197f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc6f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc6f3fff640 1 -- 172.21.15.26:0/2674103805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704070020 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc6f3fff640 1 -- 172.21.15.26:0/2674103805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc7040701c0 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc710015bf0 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc6f3fff640 1 -- 172.21.15.26:0/2674103805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc704021070 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 10 client.4395 fetch_fsmap learned FSMap version 7 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560a947472d0 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc6f3fff640 1 -- 172.21.15.26:0/2674103805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc710015bf0 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc6f3fff640 1 -- 172.21.15.26:0/2674103805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 7) v1 ==== 3322+0+0 (secure 0 0 0) 0x7fc704024060 con 0x560a94b17800 2024-03-06T07:20:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 10 client.4395 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 20 client.4395 populate_metadata read hostname 'smithi026' 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b16bb0 0x560a94aec160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 4 client.4395 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.018+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 --> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a94a3fbf0 con 0x560a94b16bb0 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.019+0000 7fc71912f640 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b16bb0 0x560a94aec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.019+0000 7fc71912f640 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b16bb0 0x560a94aec160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f52f46e611049d server_cookie=6d74c54764b655b0 in_seq=0 out_seq=0 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.019+0000 7fc6f3fff640 10 client.4395 ms_handle_connect on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc6f3fff640 1 -- 172.21.15.26:0/2674103805 <== mds.0 v2:172.21.15.117:6835/3090618513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560a94a3fbf0 con 0x560a94b16bb0 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc6f3fff640 10 client.4395 handle_command_reply: tid=0 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 client.4395 shutdown 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 msgr2=0x560a94b1f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94b1f390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc70400c520 tx=0x7fc704058980 comp rx=0 tx=0).stop 2024-03-06T07:20:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 shutdown_connections 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b16bb0 0x560a94aec160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94b1f390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc6f3fff640 0 client.4395 ms_handle_reset on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 --2- 172.21.15.26:0/2674103805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94acd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 >> 172.21.15.26:0/2674103805 conn(0x560a94ace550 msgr2=0x560a94aef5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 shutdown_connections 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 1 -- 172.21.15.26:0/2674103805 wait complete. 2024-03-06T07:20:09.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.020+0000 7fc71df7e740 20 client.4395 trim_cache size 0 max 16384 2024-03-06T07:20:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.037+0000 7fc71df7e740 1 Processor -- start 2024-03-06T07:20:09.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.037+0000 7fc71df7e740 1 -- start start 2024-03-06T07:20:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94aef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94aefb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71df7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141f3780 con 0x560a94b17800 2024-03-06T07:20:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714148e20 con 0x560a94af3380 2024-03-06T07:20:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc714072bc0 con 0x560a94b08420 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94aefb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94aefb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59346/0 (socket says 172.21.15.26:59346) 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 -- 172.21.15.26:0/1821776438 learned_addr learned my addr 172.21.15.26:0/1821776438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71912f640 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc71892e640 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94aef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 -- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 msgr2=0x560a94aef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94aef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 -- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 msgr2=0x560a94b226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 -- 172.21.15.26:0/1821776438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a948116b0 con 0x560a94b17800 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc719930640 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94aefb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009eb0 tx=0x7fc70c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2fc657cc5223e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc6fe7fc640 1 -- 172.21.15.26:0/1821776438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c04f070 con 0x560a94b17800 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc6fe7fc640 1 -- 172.21.15.26:0/1821776438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c03a040 con 0x560a94b17800 2024-03-06T07:20:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.038+0000 7fc6fe7fc640 1 -- 172.21.15.26:0/1821776438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c039040 con 0x560a94b17800 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 -- 172.21.15.26:0/1821776438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 msgr2=0x560a94aefb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94aefb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009eb0 tx=0x7fc70c02e000 comp rx=0 tx=0).stop 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 -- 172.21.15.26:0/1821776438 shutdown_connections 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b17800 0x560a94aefb40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b08420 0x560a94aef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1821776438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 -- 172.21.15.26:0/1821776438 >> 172.21.15.26:0/1821776438 conn(0x560a94ace550 msgr2=0x560a94b14b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 -- 172.21.15.26:0/1821776438 shutdown_connections 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 -- 172.21.15.26:0/1821776438 wait complete. 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.039+0000 7fc71df7e740 1 -- start start 2024-03-06T07:20:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 1 Processor -- start 2024-03-06T07:20:09.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b1c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b08420 0x560a94b31e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b17800 0x560a94b00090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a94b16a30 con 0x560a94b08420 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a94b234a0 con 0x560a94af3380 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a94b16fa0 con 0x560a94b17800 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71df7e740 10 client.0 _ll_register_callbacks cb 0x7fc7040584c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71912f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b1c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71892e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b08420 0x560a94b31e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71892e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b08420 0x560a94b31e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59352/0 (socket says 172.21.15.26:59352) 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71912f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b1c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59610/0 (socket says 172.21.15.26:59610) 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.040+0000 7fc71892e640 1 -- 172.21.15.26:0/1913621249 learned_addr learned my addr 172.21.15.26:0/1913621249 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc719930640 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b17800 0x560a94b00090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71892e640 1 -- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b17800 msgr2=0x560a94b00090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71892e640 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b17800 0x560a94b00090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71892e640 1 -- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 msgr2=0x560a94b1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71892e640 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b1c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71892e640 1 -- 172.21.15.26:0/1913621249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a94aee2b0 con 0x560a94b08420 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71912f640 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71892e640 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b08420 0x560a94b31e20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7040886d0 tx=0x7fc704088550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52657989bebb64c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc6f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc6f3fff640 1 -- 172.21.15.26:0/1913621249 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704070040 con 0x560a94b08420 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc6f3fff640 1 -- 172.21.15.26:0/1913621249 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc704057dd0 con 0x560a94b08420 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc70c03a040 con 0x560a94b08420 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc6f3fff640 1 -- 172.21.15.26:0/1913621249 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7040882b0 con 0x560a94b08420 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 10 client.4401 fetch_fsmap learned FSMap version 7 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560a94b0b030 con 0x560a94b08420 2024-03-06T07:20:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc6f3fff640 1 -- 172.21.15.26:0/1913621249 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c03a040 con 0x560a94b08420 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc6f3fff640 1 -- 172.21.15.26:0/1913621249 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 7) v1 ==== 3322+0+0 (secure 0 0 0) 0x7fc7040546d0 con 0x560a94b08420 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 10 client.4401 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 20 client.4401 populate_metadata read hostname 'smithi026' 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b00990 0x560a94b82b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 4 client.4401 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.041+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 --> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a94aced40 con 0x560a94b00990 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71912f640 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b00990 0x560a94b82b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc6f3fff640 10 client.4401 ms_handle_connect on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71912f640 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b00990 0x560a94b82b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea6f648ec4d4c71 server_cookie=9723986de74cc05f in_seq=0 out_seq=0 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc6f3fff640 1 -- 172.21.15.26:0/1913621249 <== mds.0 v2:172.21.15.117:6835/3090618513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x560a94aced40 con 0x560a94b00990 2024-03-06T07:20:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc6f3fff640 10 client.4401 handle_command_reply: tid=0 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 client.4401 shutdown 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b08420 msgr2=0x560a94b31e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b08420 0x560a94b31e20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7040886d0 tx=0x7fc704088550 comp rx=0 tx=0).stop 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 shutdown_connections 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x560a94b00990 0x560a94b82b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560a94b17800 0x560a94b00090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a94b08420 0x560a94b31e20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc6f3fff640 0 client.4401 ms_handle_reset on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 --2- 172.21.15.26:0/1913621249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a94af3380 0x560a94b1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 >> 172.21.15.26:0/1913621249 conn(0x560a94ace550 msgr2=0x560a94accee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 shutdown_connections 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.042+0000 7fc71df7e740 1 -- 172.21.15.26:0/1913621249 wait complete. 2024-03-06T07:20:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71df7e740 20 client.4401 trim_cache size 0 max 16384 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4263, 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 7.2444779930000003, 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 7, 2024-03-06T07:20:09.053 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 24, 2024-03-06T07:20:09.054 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 23, 2024-03-06T07:20:09.054 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 13.620273859999999 2024-03-06T07:20:09.054 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc6f806b4d0 msgr2=0x7fc6f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc6f806b4d0 0x7fc6f806d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc710005880 tx=0x7fc710009040 comp rx=0 tx=0).stop 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714072f40 msgr2=0x7fc7140a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714072f40 0x7fc7140a98c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc70400e980 tx=0x7fc70400ee40 comp rx=0 tx=0).stop 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 shutdown_connections 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc6f806b4d0 0x7fc6f806d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714152060 0x7fc7141ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714072f40 0x7fc7140a98c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 --2- 172.21.15.26:0/2158224301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc71406d680 0x7fc7141451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 >> 172.21.15.26:0/2158224301 conn(0x7fc71406b1a0 msgr2=0x7fc714150b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 shutdown_connections 2024-03-06T07:20:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.043+0000 7fc71b393640 1 -- 172.21.15.26:0/2158224301 wait complete. 2024-03-06T07:20:09.056 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-06T07:20:09.057 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:09.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-06T07:20:09.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.156+0000 7f3230e22640 1 Processor -- start 2024-03-06T07:20:09.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.156+0000 7f3230e22640 1 -- start start 2024-03-06T07:20:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3230e22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c14f250 0x7f322c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3230e22640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c06c150 con 0x7f322c0c7350 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3230e22640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c14f630 con 0x7f322c155c20 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3230e22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c06c480 con 0x7f322c14f250 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f322ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f322c0c7350 0x7f322c15fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36454/0 (socket says 172.21.15.26:36454) 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f322ad9d640 1 -- 172.21.15.26:0/4201049616 learned_addr learned my addr 172.21.15.26:0/4201049616 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f322a59c640 1 --2- 172.21.15.26:0/4201049616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c14f250 0x7f322c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3229d9b640 1 -- 172.21.15.26:0/4201049616 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154559988 0 0) 0x7f322c06c150 con 0x7f322c0c7350 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3229d9b640 1 -- 172.21.15.26:0/4201049616 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3218003520 con 0x7f322c0c7350 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3229d9b640 1 -- 172.21.15.26:0/4201049616 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679096713 0 0) 0x7f322c14f630 con 0x7f322c155c20 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f3229d9b640 1 -- 172.21.15.26:0/4201049616 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f322c06c150 con 0x7f322c155c20 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f322a59c640 1 -- 172.21.15.26:0/4201049616 >> v1:172.21.15.117:6789/0 conn(0x7f322c155c20 legacy=0x7f322c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f322a59c640 1 -- 172.21.15.26:0/4201049616 >> v1:172.21.15.26:6789/0 conn(0x7f322c0c7350 legacy=0x7f322c15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.157+0000 7f322a59c640 1 -- 172.21.15.26:0/4201049616 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0c5660 con 0x7f322c14f250 2024-03-06T07:20:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f322a59c640 1 --2- 172.21.15.26:0/4201049616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c14f250 0x7f322c0c8840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f321c0097d0 tx=0x7f321c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=502b6b256462822c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3229d9b640 1 -- 172.21.15.26:0/4201049616 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c031770 con 0x7f322c14f250 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3229d9b640 1 -- 172.21.15.26:0/4201049616 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c037070 con 0x7f322c14f250 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3229d9b640 1 -- 172.21.15.26:0/4201049616 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c031b10 con 0x7f322c14f250 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3230e22640 1 -- 172.21.15.26:0/4201049616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c14f250 msgr2=0x7f322c0c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3230e22640 1 --2- 172.21.15.26:0/4201049616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c14f250 0x7f322c0c8840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f321c0097d0 tx=0x7f321c02b810 comp rx=0 tx=0).stop 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3230e22640 1 -- 172.21.15.26:0/4201049616 shutdown_connections 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3230e22640 1 --2- 172.21.15.26:0/4201049616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c14f250 0x7f322c0c8840 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.158+0000 7f3230e22640 1 -- 172.21.15.26:0/4201049616 >> 172.21.15.26:0/4201049616 conn(0x7f322c0ac4d0 msgr2=0x7f322c0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.159+0000 7f3230e22640 1 -- 172.21.15.26:0/4201049616 shutdown_connections 2024-03-06T07:20:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.159+0000 7f3230e22640 1 -- 172.21.15.26:0/4201049616 wait complete. 2024-03-06T07:20:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.160+0000 7f3230e22640 1 Processor -- start 2024-03-06T07:20:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.160+0000 7f3230e22640 1 -- start start 2024-03-06T07:20:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.160+0000 7f3230e22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f322c0c7350 0x7f322c0bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3230e22640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c14f250 0x7f322c157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3230e22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c155c20 0x7f322c1ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3230e22640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0c5c20 con 0x7f322c14f250 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3230e22640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c06daf0 con 0x7f322c0c7350 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3230e22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1560a0 con 0x7f322c155c20 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c14f250 0x7f322c157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c14f250 0x7f322c157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59360/0 (socket says 172.21.15.26:59360) 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 -- 172.21.15.26:0/2836655787 learned_addr learned my addr 172.21.15.26:0/2836655787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f322a59c640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f322c0c7350 0x7f322c0bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f322ad9d640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c155c20 0x7f322c1ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 -- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c155c20 msgr2=0x7f322c1ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c155c20 0x7f322c1ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 -- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f322c0c7350 msgr2=0x7f322c0bf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f322c0c7350 0x7f322c0bf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 -- 172.21.15.26:0/2836655787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0be450 con 0x7f322c14f250 2024-03-06T07:20:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3221d9b640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c14f250 0x7f322c157220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f321400c430 tx=0x7f321400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f32237fe640 1 -- 172.21.15.26:0/2836655787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214017070 con 0x7f322c14f250 2024-03-06T07:20:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f32237fe640 1 -- 172.21.15.26:0/2836655787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32140052f0 con 0x7f322c14f250 2024-03-06T07:20:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c0be730 con 0x7f322c14f250 2024-03-06T07:20:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f32237fe640 1 -- 172.21.15.26:0/2836655787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214003830 con 0x7f322c14f250 2024-03-06T07:20:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.161+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c157760 con 0x7f322c14f250 2024-03-06T07:20:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.162+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c04f330 con 0x7f322c14f250 2024-03-06T07:20:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.163+0000 7f32237fe640 1 -- 172.21.15.26:0/2836655787 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f321401a070 con 0x7f322c14f250 2024-03-06T07:20:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.163+0000 7f32237fe640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f320c06b4d0 0x7f320c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.163+0000 7f32237fe640 1 -- 172.21.15.26:0/2836655787 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f32140897d0 con 0x7f322c14f250 2024-03-06T07:20:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.163+0000 7f322a59c640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f320c06b4d0 0x7f320c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.164+0000 7f322a59c640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f320c06b4d0 0x7f320c06d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f321c036000 tx=0x7f321c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.167+0000 7f32237fe640 1 -- 172.21.15.26:0/2836655787 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3214057d50 con 0x7f322c14f250 2024-03-06T07:20:09.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.388+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f322c06d860 con 0x7f322c14f250 2024-03-06T07:20:09.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.389+0000 7f32237fe640 1 -- 172.21.15.26:0/2836655787 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f321405bdf0 con 0x7f322c14f250 2024-03-06T07:20:09.392 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-06T07:20:09.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f320c06b4d0 msgr2=0x7f320c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f320c06b4d0 0x7f320c06d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f321c036000 tx=0x7f321c03d040 comp rx=0 tx=0).stop 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c14f250 msgr2=0x7f322c157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c14f250 0x7f322c157220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f321400c430 tx=0x7f321400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 shutdown_connections 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f320c06b4d0 0x7f320c06d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322c155c20 0x7f322c1ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c14f250 0x7f322c157220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 --2- 172.21.15.26:0/2836655787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f322c0c7350 0x7f322c0bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 >> 172.21.15.26:0/2836655787 conn(0x7f322c0ac4d0 msgr2=0x7f322c14ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 shutdown_connections 2024-03-06T07:20:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:09.391+0000 7f3230e22640 1 -- 172.21.15.26:0/2836655787 wait complete. 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-06T07:20:09.404 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-06T07:20:09.405 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-06T07:20:09.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:09.405 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-03-06T07:20:09.419 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-06T07:20:09.419 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:09.420 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-03-06T07:20:09.421 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-06T07:20:09.421 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42674sec preferred_lft 42674sec 2024-03-06T07:20:09.421 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-03-06T07:20:09.421 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:09.421 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:09.421 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:09.421 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-03-06T07:20:09.421 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-06T07:20:09.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:09.422 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:09.422 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:09.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-03-06T07:20:09.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-03-06T07:20:09.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-03-06T07:20:09.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-06T07:20:09.423 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:09.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:09.589 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-06T07:20:09.616 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-03-06T07:20:09.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:09.617 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-03-06T07:20:09.691 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-03-06T07:20:09.691 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-06T07:20:09.691 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-06T07:20:09.692 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-06T07:20:09.692 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-06T07:20:09.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:09.692 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:09.692 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:09.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-06T07:20:09.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-06T07:20:09.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-06T07:20:09.693 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:09.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:09.939 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-03-06T07:20:09.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:09.954 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-03-06T07:20:10.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.008 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:10.009 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:10.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:10.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-06T07:20:10.009 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:10.092 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-06T07:20:10.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-06T07:20:10.092 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:10.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.226 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:10.226 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:10.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-03-06T07:20:10.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-03-06T07:20:10.226 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:10.267 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-06T07:20:10.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:10.267 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:10.326 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-06T07:20:10.326 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:10.326 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:10.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-03-06T07:20:10.449 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-03-06T07:20:10.504 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-03-06T07:20:10.505 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-03-06T07:20:10.506 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-03-06T07:20:10.507 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-03-06T07:20:10.507 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-03-06T07:20:10.507 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:10.507 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:10.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.507 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-03-06T07:20:10.558 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-06T07:20:10.559 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-06T07:20:10.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-03-06T07:20:10.628 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-03-06T07:20:10.684 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-06T07:20:10.685 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:10.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.686 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-03-06T07:20:10.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:2024-03-06T07:20:10.685+0000 7f83fe438480 -1 init, newargv = 0x561fe3cd5d80 newargc=15 2024-03-06T07:20:10.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:ceph-fuse[43124]: starting ceph client 2024-03-06T07:20:10.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:ceph-fuse[43124]: starting fuse 2024-03-06T07:20:10.703 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-03-06T07:20:10.703 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-03-06T07:20:10.703 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-03-06T07:20:10.704 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-03-06T07:20:10.705 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-03-06T07:20:10.705 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-06T07:20:10.705 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-06T07:20:10.705 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.admin")) 2024-03-06T07:20:10.705 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-03-06T07:20:10.824 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.admin.43124.asok 2024-03-06T07:20:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-06T07:20:10.835 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43124.asok 2024-03-06T07:20:10.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43124.asok status 2024-03-06T07:20:10.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:10.878 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:10.973 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:20:10.973 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-03-06T07:20:10.973 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd", 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "admin", 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "43124", 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4413, 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-03-06T07:20:10.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4413 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 714802256 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 714802256 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:10.975 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.4413 192.168.0.1:0/714802256", 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/714802256", 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 7, 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 24, 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-03-06T07:20:10.976 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:20:10.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.983 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-03-06T07:20:10.983 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:10.998 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-03-06T07:20:10.999 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:10.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:10.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:11.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:11.067 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-06T07:20:11.142 INFO:teuthology.orchestra.run.smithi026.stdout:# file: . 2024-03-06T07:20:11.142 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-06T07:20:11.142 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:11.142 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:20:11.142 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:11.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.337+0000 7f0b59dbc640 1 Processor -- start 2024-03-06T07:20:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.337+0000 7f0b59dbc640 1 -- start start 2024-03-06T07:20:11.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b59dbc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b5414f250 0x7f0b540c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b59dbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c7350 0x7f0b540c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b59dbc640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5406c150 con 0x7f0b54155c20 2024-03-06T07:20:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b59dbc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5406c480 con 0x7f0b5414f250 2024-03-06T07:20:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b59dbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5414f630 con 0x7f0b540c7350 2024-03-06T07:20:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b537fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b5414f250 0x7f0b540c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b537fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b5414f250 0x7f0b540c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59374/0 (socket says 172.21.15.26:59374) 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b537fe640 1 -- 172.21.15.26:0/3263827970 learned_addr learned my addr 172.21.15.26:0/3263827970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b53fff640 1 --2- 172.21.15.26:0/3263827970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c7350 0x7f0b540c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b537fe640 1 -- 172.21.15.26:0/3263827970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c7350 msgr2=0x7f0b540c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b537fe640 1 --2- 172.21.15.26:0/3263827970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c7350 0x7f0b540c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b537fe640 1 -- 172.21.15.26:0/3263827970 >> v1:172.21.15.117:6789/0 conn(0x7f0b54155c20 legacy=0x7f0b5406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.338+0000 7f0b537fe640 1 -- 172.21.15.26:0/3263827970 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540c5660 con 0x7f0b5414f250 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.339+0000 7f0b537fe640 1 --2- 172.21.15.26:0/3263827970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5414f250 0x7f0b540c8840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009870 tx=0x7f0b4402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38928dfe59070465 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.339+0000 7f0b527fc640 1 -- 172.21.15.26:0/3263827970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440317c0 con 0x7f0b5414f250 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.339+0000 7f0b527fc640 1 -- 172.21.15.26:0/3263827970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44037070 con 0x7f0b5414f250 2024-03-06T07:20:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.339+0000 7f0b527fc640 1 -- 172.21.15.26:0/3263827970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031b80 con 0x7f0b5414f250 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 -- 172.21.15.26:0/3263827970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5414f250 msgr2=0x7f0b540c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/3263827970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5414f250 0x7f0b540c8840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009870 tx=0x7f0b4402b860 comp rx=0 tx=0).stop 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 -- 172.21.15.26:0/3263827970 shutdown_connections 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/3263827970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c7350 0x7f0b540c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/3263827970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b5414f250 0x7f0b540c8840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 -- 172.21.15.26:0/3263827970 >> 172.21.15.26:0/3263827970 conn(0x7f0b540ac4d0 msgr2=0x7f0b540ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 -- 172.21.15.26:0/3263827970 shutdown_connections 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.340+0000 7f0b59dbc640 1 -- 172.21.15.26:0/3263827970 wait complete. 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.341+0000 7f0b59dbc640 1 Processor -- start 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.341+0000 7f0b59dbc640 1 -- start start 2024-03-06T07:20:11.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b59dbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b540c7350 0x7f0b541592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b59dbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b5414f250 0x7f0b541eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b59dbc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54155c20 0x7f0b541521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b59dbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540c5c20 con 0x7f0b540c7350 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b59dbc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5406daf0 con 0x7f0b54155c20 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b59dbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b541560a0 con 0x7f0b5414f250 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b540c7350 0x7f0b541592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54155c20 0x7f0b541521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b540c7350 0x7f0b541592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59386/0 (socket says 172.21.15.26:59386) 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54155c20 0x7f0b541521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59640/0 (socket says 172.21.15.26:59640) 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b52ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b5414f250 0x7f0b541eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 -- 172.21.15.26:0/1227875288 learned_addr learned my addr 172.21.15.26:0/1227875288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 -- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b5414f250 msgr2=0x7f0b541eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b5414f250 0x7f0b541eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 -- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54155c20 msgr2=0x7f0b541521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54155c20 0x7f0b541521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 -- 172.21.15.26:0/1227875288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b541503f0 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.342+0000 7f0b537fe640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b540c7350 0x7f0b541592d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b44000c00 tx=0x7f0b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.343+0000 7f0b3bfff640 1 -- 172.21.15.26:0/1227875288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44032650 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.343+0000 7f0b3bfff640 1 -- 172.21.15.26:0/1227875288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b440327f0 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.343+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54150660 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.343+0000 7f0b3bfff640 1 -- 172.21.15.26:0/1227875288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44032be0 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.343+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54150c20 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.344+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5404f330 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.344+0000 7f0b3bfff640 1 -- 172.21.15.26:0/1227875288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b4404c060 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.345+0000 7f0b3bfff640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4006b4d0 0x7f0b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.345+0000 7f0b3bfff640 1 -- 172.21.15.26:0/1227875288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b440abf90 con 0x7f0b540c7350 2024-03-06T07:20:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.345+0000 7f0b52ffd640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4006b4d0 0x7f0b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.345+0000 7f0b52ffd640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4006b4d0 0x7f0b4006d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b540c9a60 tx=0x7f0b48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.349+0000 7f0b3bfff640 1 -- 172.21.15.26:0/1227875288 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0b44040070 con 0x7f0b540c7350 2024-03-06T07:20:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.531+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b540bdba0 con 0x7f0b540c7350 2024-03-06T07:20:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.532+0000 7f0b3bfff640 1 -- 172.21.15.26:0/1227875288 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f0b4407a480 con 0x7f0b540c7350 2024-03-06T07:20:11.535 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:11.535 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4006b4d0 msgr2=0x7f0b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4006b4d0 0x7f0b4006d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b540c9a60 tx=0x7f0b48009040 comp rx=0 tx=0).stop 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b540c7350 msgr2=0x7f0b541592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b540c7350 0x7f0b541592d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b44000c00 tx=0x7f0b440046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 shutdown_connections 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0b4006b4d0 0x7f0b4006d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54155c20 0x7f0b541521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b5414f250 0x7f0b541eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 --2- 172.21.15.26:0/1227875288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b540c7350 0x7f0b541592d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 >> 172.21.15.26:0/1227875288 conn(0x7f0b540ac4d0 msgr2=0x7f0b540ab690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 shutdown_connections 2024-03-06T07:20:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.535+0000 7f0b59dbc640 1 -- 172.21.15.26:0/1227875288 wait complete. 2024-03-06T07:20:11.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:11.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.651+0000 7fc759d8a640 1 Processor -- start 2024-03-06T07:20:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.651+0000 7fc759d8a640 1 -- start start 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc759d8a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7540c3240 0x7fc7540c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc759d8a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7540c2c90 0x7fc7540c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc759d8a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7541567e0 con 0x7fc754155fb0 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc759d8a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc754156a80 con 0x7fc7540c2c90 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc759d8a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc754156d60 con 0x7fc7540c3240 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc753fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7540c2c90 0x7fc7540c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc753fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7540c2c90 0x7fc7540c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59392/0 (socket says 172.21.15.26:59392) 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc753fff640 1 -- 172.21.15.26:0/174212895 learned_addr learned my addr 172.21.15.26:0/174212895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.652+0000 7fc752ffd640 1 --2- 172.21.15.26:0/174212895 >> v2:172.21.15.117:3300/0 conn(0x7fc7540c3240 0x7fc7540c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc753fff640 1 -- 172.21.15.26:0/174212895 >> v2:172.21.15.117:3300/0 conn(0x7fc7540c3240 msgr2=0x7fc7540c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc753fff640 1 --2- 172.21.15.26:0/174212895 >> v2:172.21.15.117:3300/0 conn(0x7fc7540c3240 0x7fc7540c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc753fff640 1 -- 172.21.15.26:0/174212895 >> v1:172.21.15.117:6789/0 conn(0x7fc754155fb0 legacy=0x7fc7540c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc753fff640 1 -- 172.21.15.26:0/174212895 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc754157040 con 0x7fc7540c2c90 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc752ffd640 1 --2- 172.21.15.26:0/174212895 >> v2:172.21.15.117:3300/0 conn(0x7fc7540c3240 0x7fc7540c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc753fff640 1 --2- 172.21.15.26:0/174212895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c2c90 0x7fc7540c56d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc754068730 tx=0x7fc74c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626ec51ea2e54387 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc7527fc640 1 -- 172.21.15.26:0/174212895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c02ca80 con 0x7fc7540c2c90 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.653+0000 7fc7527fc640 1 -- 172.21.15.26:0/174212895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c02cc20 con 0x7fc7540c2c90 2024-03-06T07:20:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc7527fc640 1 -- 172.21.15.26:0/174212895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0343f0 con 0x7fc7540c2c90 2024-03-06T07:20:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 -- 172.21.15.26:0/174212895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c2c90 msgr2=0x7fc7540c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 --2- 172.21.15.26:0/174212895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c2c90 0x7fc7540c56d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc754068730 tx=0x7fc74c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 -- 172.21.15.26:0/174212895 shutdown_connections 2024-03-06T07:20:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 --2- 172.21.15.26:0/174212895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c2c90 0x7fc7540c56d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 --2- 172.21.15.26:0/174212895 >> v2:172.21.15.117:3300/0 conn(0x7fc7540c3240 0x7fc7540c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 -- 172.21.15.26:0/174212895 >> 172.21.15.26:0/174212895 conn(0x7fc7540a94d0 msgr2=0x7fc7540a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 -- 172.21.15.26:0/174212895 shutdown_connections 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.654+0000 7fc759d8a640 1 -- 172.21.15.26:0/174212895 wait complete. 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.655+0000 7fc759d8a640 1 Processor -- start 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.655+0000 7fc759d8a640 1 -- start start 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc759d8a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7540c2c90 0x7fc7541e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc759d8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c3240 0x7fc7541e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc759d8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc754155fb0 0x7fc7541520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc759d8a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540c5d90 con 0x7fc7540c3240 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc759d8a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540c5c10 con 0x7fc754155fb0 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc759d8a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540c5f10 con 0x7fc7540c2c90 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c3240 0x7fc7541e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c3240 0x7fc7541e97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59396/0 (socket says 172.21.15.26:59396) 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 -- 172.21.15.26:0/1839441283 learned_addr learned my addr 172.21.15.26:0/1839441283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc753fff640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc754155fb0 0x7fc7541520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc7537fe640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7540c2c90 0x7fc7541e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 -- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7540c2c90 msgr2=0x7fc7541e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7540c2c90 0x7fc7541e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 -- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc754155fb0 msgr2=0x7fc7541520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc754155fb0 0x7fc7541520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.656+0000 7fc752ffd640 1 -- 172.21.15.26:0/1839441283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7541dcc20 con 0x7fc7540c3240 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.657+0000 7fc752ffd640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c3240 0x7fc7541e97d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc74800ebe0 tx=0x7fc74800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.657+0000 7fc73bfff640 1 -- 172.21.15.26:0/1839441283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748017070 con 0x7fc7540c3240 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.657+0000 7fc73bfff640 1 -- 172.21.15.26:0/1839441283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800cc30 con 0x7fc7540c3240 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.657+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7541515f0 con 0x7fc7540c3240 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.657+0000 7fc73bfff640 1 -- 172.21.15.26:0/1839441283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748013470 con 0x7fc7540c3240 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.657+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75414f1a0 con 0x7fc7540c3240 2024-03-06T07:20:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.658+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7541557a0 con 0x7fc7540c3240 2024-03-06T07:20:11.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.658+0000 7fc73bfff640 1 -- 172.21.15.26:0/1839441283 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc74801a070 con 0x7fc7540c3240 2024-03-06T07:20:11.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.658+0000 7fc73bfff640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc74006b500 0x7fc74006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.659+0000 7fc73bfff640 1 -- 172.21.15.26:0/1839441283 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc74808a690 con 0x7fc7540c3240 2024-03-06T07:20:11.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.659+0000 7fc7537fe640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc74006b500 0x7fc74006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.659+0000 7fc7537fe640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc74006b500 0x7fc74006d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7540726a0 tx=0x7fc744009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.663+0000 7fc73bfff640 1 -- 172.21.15.26:0/1839441283 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc7541557a0 con 0x7fc7540c3240 2024-03-06T07:20:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.845+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc754150070 con 0x7fc7540c3240 2024-03-06T07:20:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.846+0000 7fc73bfff640 1 -- 172.21.15.26:0/1839441283 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc754150070 con 0x7fc7540c3240 2024-03-06T07:20:11.848 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:11.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc74006b500 msgr2=0x7fc74006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc74006b500 0x7fc74006d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7540726a0 tx=0x7fc744009040 comp rx=0 tx=0).stop 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c3240 msgr2=0x7fc7541e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c3240 0x7fc7541e97d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc74800ebe0 tx=0x7fc74800c520 comp rx=0 tx=0).stop 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 shutdown_connections 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc74006b500 0x7fc74006d9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc754155fb0 0x7fc7541520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540c3240 0x7fc7541e97d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 --2- 172.21.15.26:0/1839441283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7540c2c90 0x7fc7541e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 >> 172.21.15.26:0/1839441283 conn(0x7fc7540a94d0 msgr2=0x7fc754152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 shutdown_connections 2024-03-06T07:20:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.848+0000 7fc759d8a640 1 -- 172.21.15.26:0/1839441283 wait complete. 2024-03-06T07:20:11.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.962+0000 7f3ce5888640 1 Processor -- start 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3ce5888640 1 -- start start 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3ce5888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce01529f0 0x7f3ce00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3ce5888640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce0152dd0 con 0x7f3ce00c57d0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3ce5888640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce0158b00 con 0x7f3ce00c5370 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3ce5888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0159710 con 0x7f3ce01529f0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3cdf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3ce00c57d0 0x7f3ce015fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36456/0 (socket says 172.21.15.26:36456) 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3cdf7fe640 1 -- 172.21.15.26:0/1550960681 learned_addr learned my addr 172.21.15.26:0/1550960681 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.963+0000 7f3cdeffd640 1 --2- 172.21.15.26:0/1550960681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce01529f0 0x7f3ce00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.964+0000 7f3cddffb640 1 -- 172.21.15.26:0/1550960681 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751242106 0 0) 0x7f3ce0152dd0 con 0x7f3ce00c57d0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.964+0000 7f3cddffb640 1 -- 172.21.15.26:0/1550960681 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc8003520 con 0x7f3ce00c57d0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.964+0000 7f3cddffb640 1 -- 172.21.15.26:0/1550960681 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722239560 0 0) 0x7f3ce0158b00 con 0x7f3ce00c5370 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.964+0000 7f3cdeffd640 1 -- 172.21.15.26:0/1550960681 >> v1:172.21.15.117:6789/0 conn(0x7f3ce00c5370 legacy=0x7f3ce0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.964+0000 7f3cdeffd640 1 -- 172.21.15.26:0/1550960681 >> v1:172.21.15.26:6789/0 conn(0x7f3ce00c57d0 legacy=0x7f3ce015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.964+0000 7f3cdeffd640 1 -- 172.21.15.26:0/1550960681 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0158de0 con 0x7f3ce01529f0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3cdeffd640 1 --2- 172.21.15.26:0/1550960681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce01529f0 0x7f3ce00c88f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00097d0 tx=0x7f3cd002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39376d6de73248a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3cddffb640 1 -- 172.21.15.26:0/1550960681 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0035020 con 0x7f3ce01529f0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3cddffb640 1 -- 172.21.15.26:0/1550960681 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd0031800 con 0x7f3ce01529f0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3cddffb640 1 -- 172.21.15.26:0/1550960681 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0031bd0 con 0x7f3ce01529f0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3ce5888640 1 -- 172.21.15.26:0/1550960681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce01529f0 msgr2=0x7f3ce00c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1550960681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce01529f0 0x7f3ce00c88f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00097d0 tx=0x7f3cd002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3ce5888640 1 -- 172.21.15.26:0/1550960681 shutdown_connections 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1550960681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce01529f0 0x7f3ce00c88f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3ce5888640 1 -- 172.21.15.26:0/1550960681 >> 172.21.15.26:0/1550960681 conn(0x7f3ce00ac580 msgr2=0x7f3ce00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3ce5888640 1 -- 172.21.15.26:0/1550960681 shutdown_connections 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.965+0000 7f3ce5888640 1 -- 172.21.15.26:0/1550960681 wait complete. 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.966+0000 7f3ce5888640 1 Processor -- start 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3ce5888640 1 -- start start 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3ce5888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce00c5370 0x7f3ce0153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3ce5888640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce00c57d0 0x7f3ce00c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3ce5888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce01529f0 0x7f3ce0157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3ce5888640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce01593a0 con 0x7f3ce00c57d0 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3ce5888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00c68b0 con 0x7f3ce01529f0 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3ce5888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0158740 con 0x7f3ce00c5370 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3cde7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce00c57d0 0x7f3ce00c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3cde7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce00c57d0 0x7f3ce00c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59408/0 (socket says 172.21.15.26:59408) 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3cde7fc640 1 -- 172.21.15.26:0/1218960061 learned_addr learned my addr 172.21.15.26:0/1218960061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3cdeffd640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce00c5370 0x7f3ce0153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.967+0000 7f3cdf7fe640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce01529f0 0x7f3ce0157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cdeffd640 1 -- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce01529f0 msgr2=0x7f3ce0157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cdeffd640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce01529f0 0x7f3ce0157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cdeffd640 1 -- 172.21.15.26:0/1218960061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce00c57d0 msgr2=0x7f3ce00c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cdeffd640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce00c57d0 0x7f3ce00c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cdeffd640 1 -- 172.21.15.26:0/1218960061 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce01ebeb0 con 0x7f3ce00c5370 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cde7fc640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce00c57d0 0x7f3ce00c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cdeffd640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce00c5370 0x7f3ce0153a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00cbaf0 tx=0x7f3cd00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cc7fff640 1 -- 172.21.15.26:0/1218960061 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0040070 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cc7fff640 1 -- 172.21.15.26:0/1218960061 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd0039cf0 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce01eb130 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3cc7fff640 1 -- 172.21.15.26:0/1218960061 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0035040 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.968+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce0155110 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.969+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce0069460 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.969+0000 7f3cc7fff640 1 -- 172.21.15.26:0/1218960061 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cd0038020 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.970+0000 7f3cc7fff640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.970+0000 7f3cc7fff640 1 -- 172.21.15.26:0/1218960061 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3cd00ab650 con 0x7f3ce00c5370 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.970+0000 7f3cde7fc640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.970+0000 7f3cde7fc640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00ac350 tx=0x7f3cd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:11.975+0000 7f3cc7fff640 1 -- 172.21.15.26:0/1218960061 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ce0069460 con 0x7f3ce00c5370 2024-03-06T07:20:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.156+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ce004f330 con 0x7f3ce00c5370 2024-03-06T07:20:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.156+0000 7f3cc7fff640 1 -- 172.21.15.26:0/1218960061 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3cd0049030 con 0x7f3ce00c5370 2024-03-06T07:20:12.159 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:12.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.158+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ccc06b4d0 msgr2=0x7f3ccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.158+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00ac350 tx=0x7f3cd4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:12.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.158+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce00c5370 msgr2=0x7f3ce0153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.158+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce00c5370 0x7f3ce0153a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00cbaf0 tx=0x7f3cd00046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 shutdown_connections 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce01529f0 0x7f3ce0157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce00c57d0 0x7f3ce00c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 --2- 172.21.15.26:0/1218960061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ce00c5370 0x7f3ce0153a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 >> 172.21.15.26:0/1218960061 conn(0x7f3ce00ac580 msgr2=0x7f3ce0150b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 shutdown_connections 2024-03-06T07:20:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.159+0000 7f3ce5888640 1 -- 172.21.15.26:0/1218960061 wait complete. 2024-03-06T07:20:12.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:12.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.273+0000 7facf473e640 1 Processor -- start 2024-03-06T07:20:12.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.273+0000 7facf473e640 1 -- start start 2024-03-06T07:20:12.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf473e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facec0c5990 0x7facec0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf473e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec0ab170 0x7facec06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf473e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec0c3ef0 con 0x7facec06c730 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf473e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec06cbe0 con 0x7facec0c5990 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf473e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec1556d0 con 0x7facec0ab170 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf24da640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facec0c5990 0x7facec0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf24da640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facec0c5990 0x7facec0c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59410/0 (socket says 172.21.15.26:59410) 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf24da640 1 -- 172.21.15.26:0/2619574517 learned_addr learned my addr 172.21.15.26:0/2619574517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf2cdb640 1 --1- 172.21.15.26:0/2619574517 >> v1:172.21.15.117:6789/0 conn(0x7facec06c730 0x7facec1523f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:58660/0 (socket says 172.21.15.26:58660) 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf1cd9640 1 --2- 172.21.15.26:0/2619574517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec0ab170 0x7facec06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf24da640 1 -- 172.21.15.26:0/2619574517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec0ab170 msgr2=0x7facec06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf24da640 1 --2- 172.21.15.26:0/2619574517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec0ab170 0x7facec06c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf14d8640 1 -- 172.21.15.26:0/2619574517 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171290011 0 0) 0x7facec0c3ef0 con 0x7facec06c730 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf24da640 1 -- 172.21.15.26:0/2619574517 >> v1:172.21.15.117:6789/0 conn(0x7facec06c730 legacy=0x7facec1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.274+0000 7facf24da640 1 -- 172.21.15.26:0/2619574517 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec1559b0 con 0x7facec0c5990 2024-03-06T07:20:12.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.275+0000 7facf24da640 1 --2- 172.21.15.26:0/2619574517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec0c0ab0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7face8009870 tx=0x7face802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b3003a1edc7925 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.275+0000 7facf14d8640 1 -- 172.21.15.26:0/2619574517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8035020 con 0x7facec0c5990 2024-03-06T07:20:12.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.275+0000 7facf14d8640 1 -- 172.21.15.26:0/2619574517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8031640 con 0x7facec0c5990 2024-03-06T07:20:12.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.275+0000 7facf14d8640 1 -- 172.21.15.26:0/2619574517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face802d040 con 0x7facec0c5990 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.275+0000 7facf473e640 1 -- 172.21.15.26:0/2619574517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 msgr2=0x7facec0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.275+0000 7facf473e640 1 --2- 172.21.15.26:0/2619574517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec0c0ab0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7face8009870 tx=0x7face802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.276+0000 7facf473e640 1 -- 172.21.15.26:0/2619574517 shutdown_connections 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.276+0000 7facf473e640 1 --2- 172.21.15.26:0/2619574517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec0ab170 0x7facec06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.276+0000 7facf473e640 1 --2- 172.21.15.26:0/2619574517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec0c0ab0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.276+0000 7facf473e640 1 -- 172.21.15.26:0/2619574517 >> 172.21.15.26:0/2619574517 conn(0x7facec0c6710 msgr2=0x7facec0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.276+0000 7facf473e640 1 -- 172.21.15.26:0/2619574517 shutdown_connections 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.276+0000 7facf473e640 1 -- 172.21.15.26:0/2619574517 wait complete. 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 Processor -- start 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 -- start start 2024-03-06T07:20:12.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec06c730 0x7facec0a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facec0ab170 0x7facec1e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec156490 con 0x7facec0c5990 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec155f70 con 0x7facec0ab170 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf473e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec156610 con 0x7facec06c730 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.277+0000 7facf2cdb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec1518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59412/0 (socket says 172.21.15.26:59412) 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 -- 172.21.15.26:0/3430827860 learned_addr learned my addr 172.21.15.26:0/3430827860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf24da640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec06c730 0x7facec0a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf1cd9640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facec0ab170 0x7facec1e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 -- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec06c730 msgr2=0x7facec0a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec06c730 0x7facec0a7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 -- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facec0ab170 msgr2=0x7facec1e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facec0ab170 0x7facec1e7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 -- 172.21.15.26:0/3430827860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0705a0 con 0x7facec0c5990 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf2cdb640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec1518d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7face400ead0 tx=0x7face400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7face2ffd640 1 -- 172.21.15.26:0/3430827860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4004d70 con 0x7facec0c5990 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7face2ffd640 1 -- 172.21.15.26:0/3430827860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4016020 con 0x7facec0c5990 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec070880 con 0x7facec0c5990 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7face2ffd640 1 -- 172.21.15.26:0/3430827860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face400cc10 con 0x7facec0c5990 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.278+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec1e7690 con 0x7facec0c5990 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.279+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec04f330 con 0x7facec0c5990 2024-03-06T07:20:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.280+0000 7face2ffd640 1 -- 172.21.15.26:0/3430827860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7face40070d0 con 0x7facec0c5990 2024-03-06T07:20:12.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.280+0000 7face2ffd640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7facd006b4d0 0x7facd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.280+0000 7face2ffd640 1 -- 172.21.15.26:0/3430827860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7face408c950 con 0x7facec0c5990 2024-03-06T07:20:12.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.280+0000 7facf24da640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7facd006b4d0 0x7facd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.280+0000 7facf24da640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7facd006b4d0 0x7facd006d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7face8036000 tx=0x7face803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.284+0000 7face2ffd640 1 -- 172.21.15.26:0/3430827860 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7face4020030 con 0x7facec0c5990 2024-03-06T07:20:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.463+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facec0c1740 con 0x7facec0c5990 2024-03-06T07:20:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.463+0000 7face2ffd640 1 -- 172.21.15.26:0/3430827860 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7face405aed0 con 0x7facec0c5990 2024-03-06T07:20:12.466 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7facd006b4d0 msgr2=0x7facd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7facd006b4d0 0x7facd006d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7face8036000 tx=0x7face803c040 comp rx=0 tx=0).stop 2024-03-06T07:20:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 msgr2=0x7facec1518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec1518d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7face400ead0 tx=0x7face400ef90 comp rx=0 tx=0).stop 2024-03-06T07:20:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 shutdown_connections 2024-03-06T07:20:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7facd006b4d0 0x7facd006d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facec0c5990 0x7facec1518d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facec0ab170 0x7facec1e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 --2- 172.21.15.26:0/3430827860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facec06c730 0x7facec0a7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 >> 172.21.15.26:0/3430827860 conn(0x7facec0c6710 msgr2=0x7facec154300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 shutdown_connections 2024-03-06T07:20:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.465+0000 7facf473e640 1 -- 172.21.15.26:0/3430827860 wait complete. 2024-03-06T07:20:12.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.580+0000 7f4eb48f4640 1 Processor -- start 2024-03-06T07:20:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.580+0000 7f4eb48f4640 1 -- start start 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eb48f4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4eb0155a00 0x7f4eb0159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eb48f4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb0155e70 con 0x7f4eb00be150 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eb48f4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb00c75e0 con 0x7f4eb014bcc0 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eb48f4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb00c7f10 con 0x7f4eb0155a00 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4ea7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4eb00be150 0x7f4eb0158a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36466/0 (socket says 172.21.15.26:36466) 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4ea7fff640 1 -- 172.21.15.26:0/798265455 learned_addr learned my addr 172.21.15.26:0/798265455 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eaed9d640 1 --2- 172.21.15.26:0/798265455 >> v2:172.21.15.117:3300/0 conn(0x7f4eb0155a00 0x7f4eb0159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eadd9b640 1 -- 172.21.15.26:0/798265455 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185309474 0 0) 0x7f4eb0155e70 con 0x7f4eb00be150 2024-03-06T07:20:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eadd9b640 1 -- 172.21.15.26:0/798265455 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e9c003520 con 0x7f4eb00be150 2024-03-06T07:20:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eadd9b640 1 -- 172.21.15.26:0/798265455 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477845826 0 0) 0x7f4eb00c75e0 con 0x7f4eb014bcc0 2024-03-06T07:20:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eaed9d640 1 -- 172.21.15.26:0/798265455 >> v1:172.21.15.117:6789/0 conn(0x7f4eb014bcc0 legacy=0x7f4eb00c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eaed9d640 1 -- 172.21.15.26:0/798265455 >> v1:172.21.15.26:6789/0 conn(0x7f4eb00be150 legacy=0x7f4eb0158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.581+0000 7f4eaed9d640 1 -- 172.21.15.26:0/798265455 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00c78c0 con 0x7f4eb0155a00 2024-03-06T07:20:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.582+0000 7f4eaed9d640 1 --2- 172.21.15.26:0/798265455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb0155a00 0x7f4eb0159120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009d90 tx=0x7f4ea802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1396d8c25a868578 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.582+0000 7f4eadd9b640 1 -- 172.21.15.26:0/798265455 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8039070 con 0x7f4eb0155a00 2024-03-06T07:20:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.582+0000 7f4eadd9b640 1 -- 172.21.15.26:0/798265455 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea802cd80 con 0x7f4eb0155a00 2024-03-06T07:20:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.582+0000 7f4eadd9b640 1 -- 172.21.15.26:0/798265455 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8034660 con 0x7f4eb0155a00 2024-03-06T07:20:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.583+0000 7f4eb48f4640 1 -- 172.21.15.26:0/798265455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb0155a00 msgr2=0x7f4eb0159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.583+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/798265455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb0155a00 0x7f4eb0159120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009d90 tx=0x7f4ea802bfd0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.583+0000 7f4eb48f4640 1 -- 172.21.15.26:0/798265455 shutdown_connections 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.583+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/798265455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb0155a00 0x7f4eb0159120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.583+0000 7f4eb48f4640 1 -- 172.21.15.26:0/798265455 >> 172.21.15.26:0/798265455 conn(0x7f4eb00ac600 msgr2=0x7f4eb00aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.583+0000 7f4eb48f4640 1 -- 172.21.15.26:0/798265455 shutdown_connections 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.583+0000 7f4eb48f4640 1 -- 172.21.15.26:0/798265455 wait complete. 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.584+0000 7f4eb48f4640 1 Processor -- start 2024-03-06T07:20:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.584+0000 7f4eb48f4640 1 -- start start 2024-03-06T07:20:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.584+0000 7f4eb48f4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eb00be150 0x7f4eb00c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.584+0000 7f4eb48f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb014bcc0 0x7f4eb014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.584+0000 7f4eb48f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eb0155a00 0x7f4eb0148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.584+0000 7f4eb48f4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb006d1d0 con 0x7f4eb0155a00 2024-03-06T07:20:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eb48f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb00c5ae0 con 0x7f4eb014bcc0 2024-03-06T07:20:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eb48f4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb00c5c60 con 0x7f4eb00be150 2024-03-06T07:20:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eb0155a00 0x7f4eb0148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eb0155a00 0x7f4eb0148280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59426/0 (socket says 172.21.15.26:59426) 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 -- 172.21.15.26:0/2218785574 learned_addr learned my addr 172.21.15.26:0/2218785574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4ea7fff640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb014bcc0 0x7f4eb014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eae59c640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eb00be150 0x7f4eb00c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 -- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eb00be150 msgr2=0x7f4eb00c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eb00be150 0x7f4eb00c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 -- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb014bcc0 msgr2=0x7f4eb014dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb014bcc0 0x7f4eb014dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 -- 172.21.15.26:0/2218785574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb014e9c0 con 0x7f4eb0155a00 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eaed9d640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eb0155a00 0x7f4eb0148280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8037040 tx=0x7f4ea8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4ea6ffd640 1 -- 172.21.15.26:0/2218785574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8039050 con 0x7f4eb0155a00 2024-03-06T07:20:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4ea6ffd640 1 -- 172.21.15.26:0/2218785574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea8045070 con 0x7f4eb0155a00 2024-03-06T07:20:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb00c4c30 con 0x7f4eb0155a00 2024-03-06T07:20:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4ea6ffd640 1 -- 172.21.15.26:0/2218785574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8009450 con 0x7f4eb0155a00 2024-03-06T07:20:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.585+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb01ee4f0 con 0x7f4eb0155a00 2024-03-06T07:20:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.586+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb004f330 con 0x7f4eb0155a00 2024-03-06T07:20:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.587+0000 7f4ea6ffd640 1 -- 172.21.15.26:0/2218785574 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ea803e830 con 0x7f4eb0155a00 2024-03-06T07:20:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.587+0000 7f4ea6ffd640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e9006b4d0 0x7f4e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.587+0000 7f4eae59c640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e9006b4d0 0x7f4e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.587+0000 7f4ea6ffd640 1 -- 172.21.15.26:0/2218785574 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4ea80ad370 con 0x7f4eb0155a00 2024-03-06T07:20:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.588+0000 7f4eae59c640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e9006b4d0 0x7f4e9006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000f60 tx=0x7f4ea0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.592+0000 7f4ea6ffd640 1 -- 172.21.15.26:0/2218785574 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4ea807b980 con 0x7f4eb0155a00 2024-03-06T07:20:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.771+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4eb00c83e0 con 0x7f4eb0155a00 2024-03-06T07:20:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.771+0000 7f4ea6ffd640 1 -- 172.21.15.26:0/2218785574 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4ea807fa20 con 0x7f4eb0155a00 2024-03-06T07:20:12.774 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e9006b4d0 msgr2=0x7f4e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e9006b4d0 0x7f4e9006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000f60 tx=0x7f4ea0009040 comp rx=0 tx=0).stop 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eb0155a00 msgr2=0x7f4eb0148280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eb0155a00 0x7f4eb0148280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8037040 tx=0x7f4ea8004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 shutdown_connections 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e9006b4d0 0x7f4e9006d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eb0155a00 0x7f4eb0148280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eb014bcc0 0x7f4eb014dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 --2- 172.21.15.26:0/2218785574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eb00be150 0x7f4eb00c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 >> 172.21.15.26:0/2218785574 conn(0x7f4eb00ac600 msgr2=0x7f4eb00bda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 shutdown_connections 2024-03-06T07:20:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.773+0000 7f4eb48f4640 1 -- 172.21.15.26:0/2218785574 wait complete. 2024-03-06T07:20:12.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:12.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.886+0000 7fe36cc8f640 1 Processor -- start 2024-03-06T07:20:12.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.886+0000 7fe36cc8f640 1 -- start start 2024-03-06T07:20:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.886+0000 7fe36cc8f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3680c6d70 0x7fe368160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.886+0000 7fe36cc8f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3680c71e0 con 0x7fe368155120 2024-03-06T07:20:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.886+0000 7fe36cc8f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe368159590 con 0x7fe3680bfe90 2024-03-06T07:20:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.886+0000 7fe36cc8f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe368158710 con 0x7fe3680c6d70 2024-03-06T07:20:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe35ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe368155120 0x7fe3681522d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36476/0 (socket says 172.21.15.26:36476) 2024-03-06T07:20:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe35ffff640 1 -- 172.21.15.26:0/3507787822 learned_addr learned my addr 172.21.15.26:0/3507787822 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe36659c640 1 --1- 172.21.15.26:0/3507787822 >> v1:172.21.15.117:6789/0 conn(0x7fe3680bfe90 0x7fe368158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:58670/0 (socket says 172.21.15.26:58670) 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe366d9d640 1 --2- 172.21.15.26:0/3507787822 >> v2:172.21.15.117:3300/0 conn(0x7fe3680c6d70 0x7fe368160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe365d9b640 1 -- 172.21.15.26:0/3507787822 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670455117 0 0) 0x7fe3680c71e0 con 0x7fe368155120 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe365d9b640 1 -- 172.21.15.26:0/3507787822 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe354003520 con 0x7fe368155120 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe365d9b640 1 -- 172.21.15.26:0/3507787822 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152061333 0 0) 0x7fe368159590 con 0x7fe3680bfe90 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe365d9b640 1 -- 172.21.15.26:0/3507787822 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3680c71e0 con 0x7fe3680bfe90 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe366d9d640 1 -- 172.21.15.26:0/3507787822 >> v1:172.21.15.117:6789/0 conn(0x7fe3680bfe90 legacy=0x7fe368158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe366d9d640 1 -- 172.21.15.26:0/3507787822 >> v1:172.21.15.26:6789/0 conn(0x7fe368155120 legacy=0x7fe3681522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.887+0000 7fe366d9d640 1 -- 172.21.15.26:0/3507787822 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680c59d0 con 0x7fe3680c6d70 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe366d9d640 1 --2- 172.21.15.26:0/3507787822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680c6d70 0x7fe368160ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe368158890 tx=0x7fe36002be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf8c8502e6b7824 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe365d9b640 1 -- 172.21.15.26:0/3507787822 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360039070 con 0x7fe3680c6d70 2024-03-06T07:20:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe365d9b640 1 -- 172.21.15.26:0/3507787822 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36002cc40 con 0x7fe3680c6d70 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe365d9b640 1 -- 172.21.15.26:0/3507787822 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360034440 con 0x7fe3680c6d70 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe36cc8f640 1 -- 172.21.15.26:0/3507787822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680c6d70 msgr2=0x7fe368160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/3507787822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680c6d70 0x7fe368160ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe368158890 tx=0x7fe36002be90 comp rx=0 tx=0).stop 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe36cc8f640 1 -- 172.21.15.26:0/3507787822 shutdown_connections 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/3507787822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680c6d70 0x7fe368160ab0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe36cc8f640 1 -- 172.21.15.26:0/3507787822 >> 172.21.15.26:0/3507787822 conn(0x7fe3680acaa0 msgr2=0x7fe3680aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe36cc8f640 1 -- 172.21.15.26:0/3507787822 shutdown_connections 2024-03-06T07:20:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.888+0000 7fe36cc8f640 1 -- 172.21.15.26:0/3507787822 wait complete. 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 Processor -- start 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 -- start start 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680bfe90 0x7fe3680aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3680c6d70 0x7fe3680abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe368155120 0x7fe36814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368158960 con 0x7fe368155120 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680c5f90 con 0x7fe3680bfe90 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36cc8f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680ca700 con 0x7fe3680c6d70 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe366d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe368155120 0x7fe36814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe366d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe368155120 0x7fe36814d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59442/0 (socket says 172.21.15.26:59442) 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe366d9d640 1 -- 172.21.15.26:0/2362326574 learned_addr learned my addr 172.21.15.26:0/2362326574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe35ffff640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3680c6d70 0x7fe3680abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.890+0000 7fe36659c640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680bfe90 0x7fe3680aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe366d9d640 1 -- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3680c6d70 msgr2=0x7fe3680abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe366d9d640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3680c6d70 0x7fe3680abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe366d9d640 1 -- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680bfe90 msgr2=0x7fe3680aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe366d9d640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680bfe90 0x7fe3680aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe366d9d640 1 -- 172.21.15.26:0/2362326574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368154510 con 0x7fe368155120 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe36659c640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680bfe90 0x7fe3680aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe366d9d640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe368155120 0x7fe36814d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe360009eb0 tx=0x7fe360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe35effd640 1 -- 172.21.15.26:0/2362326574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360039050 con 0x7fe368155120 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe35effd640 1 -- 172.21.15.26:0/2362326574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360045070 con 0x7fe368155120 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe35effd640 1 -- 172.21.15.26:0/2362326574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360009450 con 0x7fe368155120 2024-03-06T07:20:12.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3680ab4f0 con 0x7fe368155120 2024-03-06T07:20:12.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.891+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3680ae3f0 con 0x7fe368155120 2024-03-06T07:20:12.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.892+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368075300 con 0x7fe368155120 2024-03-06T07:20:12.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.893+0000 7fe35effd640 1 -- 172.21.15.26:0/2362326574 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe360040670 con 0x7fe368155120 2024-03-06T07:20:12.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.893+0000 7fe35effd640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe34806b4d0 0x7fe34806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.893+0000 7fe35effd640 1 -- 172.21.15.26:0/2362326574 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3600ace20 con 0x7fe368155120 2024-03-06T07:20:12.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.893+0000 7fe36659c640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe34806b4d0 0x7fe34806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.894+0000 7fe36659c640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe34806b4d0 0x7fe34806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe358000f60 tx=0x7fe358007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:12.897+0000 7fe35effd640 1 -- 172.21.15.26:0/2362326574 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe368075300 con 0x7fe368155120 2024-03-06T07:20:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.094+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe36804f330 con 0x7fe368155120 2024-03-06T07:20:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.094+0000 7fe35effd640 1 -- 172.21.15.26:0/2362326574 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe36007b430 con 0x7fe368155120 2024-03-06T07:20:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-03-06T07:20:13.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe34806b4d0 msgr2=0x7fe34806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe34806b4d0 0x7fe34806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe358000f60 tx=0x7fe358007cf0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe368155120 msgr2=0x7fe36814d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe368155120 0x7fe36814d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe360009eb0 tx=0x7fe360004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 shutdown_connections 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe34806b4d0 0x7fe34806d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe368155120 0x7fe36814d0f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3680c6d70 0x7fe3680abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 --2- 172.21.15.26:0/2362326574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3680bfe90 0x7fe3680aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 >> 172.21.15.26:0/2362326574 conn(0x7fe3680acaa0 msgr2=0x7fe3680ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 shutdown_connections 2024-03-06T07:20:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.096+0000 7fe36cc8f640 1 -- 172.21.15.26:0/2362326574 wait complete. 2024-03-06T07:20:13.109 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:20:13.109 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:13.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:13.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.265+0000 7f35df00e640 1 Processor -- start 2024-03-06T07:20:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.265+0000 7f35df00e640 1 -- start start 2024-03-06T07:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.265+0000 7f35df00e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35d80c88c0 0x7f35d8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.265+0000 7f35df00e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d8158b60 0x7f35d8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.265+0000 7f35df00e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d80c73d0 con 0x7f35d814f360 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.265+0000 7f35df00e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d80c8dd0 con 0x7f35d80c88c0 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.265+0000 7f35df00e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80be2e0 con 0x7f35d8158b60 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35d7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35d80c88c0 0x7f35d8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35dcdaa640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f35d814f360 0x7f35d80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36486/0 (socket says 172.21.15.26:36486) 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35dcdaa640 1 -- 172.21.15.26:0/1072512195 learned_addr learned my addr 172.21.15.26:0/1072512195 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35dd5ab640 1 --2- 172.21.15.26:0/1072512195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d8158b60 0x7f35d8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35d7fff640 1 -- 172.21.15.26:0/1072512195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d8158b60 msgr2=0x7f35d8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35d7fff640 1 --2- 172.21.15.26:0/1072512195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d8158b60 0x7f35d8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35d7fff640 1 -- 172.21.15.26:0/1072512195 >> v1:172.21.15.26:6789/0 conn(0x7f35d814f360 legacy=0x7f35d80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35d7fff640 1 -- 172.21.15.26:0/1072512195 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80c7670 con 0x7f35d80c88c0 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.266+0000 7f35d77fe640 1 -- 172.21.15.26:0/1072512195 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971741013 0 0) 0x7f35d80c73d0 con 0x7f35d814f360 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35d7fff640 1 --2- 172.21.15.26:0/1072512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d80c88c0 0x7f35d8155c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35c40097a0 tx=0x7f35c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb377d1ae312be37 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35d77fe640 1 -- 172.21.15.26:0/1072512195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c40315a0 con 0x7f35d80c88c0 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35d77fe640 1 -- 172.21.15.26:0/1072512195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c4037070 con 0x7f35d80c88c0 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35d77fe640 1 -- 172.21.15.26:0/1072512195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4031960 con 0x7f35d80c88c0 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 -- 172.21.15.26:0/1072512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d80c88c0 msgr2=0x7f35d8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 --2- 172.21.15.26:0/1072512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d80c88c0 0x7f35d8155c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35c40097a0 tx=0x7f35c402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 -- 172.21.15.26:0/1072512195 shutdown_connections 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 --2- 172.21.15.26:0/1072512195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d80c88c0 0x7f35d8155c10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 --2- 172.21.15.26:0/1072512195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d8158b60 0x7f35d8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 -- 172.21.15.26:0/1072512195 >> 172.21.15.26:0/1072512195 conn(0x7f35d80ac570 msgr2=0x7f35d80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 -- 172.21.15.26:0/1072512195 shutdown_connections 2024-03-06T07:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.267+0000 7f35df00e640 1 -- 172.21.15.26:0/1072512195 wait complete. 2024-03-06T07:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.268+0000 7f35df00e640 1 Processor -- start 2024-03-06T07:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35df00e640 1 -- start start 2024-03-06T07:20:13.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35df00e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d80c88c0 0x7f35d814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35df00e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d814f360 0x7f35d814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35df00e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d8158b60 0x7f35d80bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35df00e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c5a50 con 0x7f35d8158b60 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35df00e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c7c30 con 0x7f35d80c88c0 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35df00e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c5bd0 con 0x7f35d814f360 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35dd5ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d8158b60 0x7f35d80bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.269+0000 7f35dd5ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d8158b60 0x7f35d80bf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59462/0 (socket says 172.21.15.26:59462) 2024-03-06T07:20:13.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dd5ab640 1 -- 172.21.15.26:0/321836204 learned_addr learned my addr 172.21.15.26:0/321836204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dcdaa640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d80c88c0 0x7f35d814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35d7fff640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d814f360 0x7f35d814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dd5ab640 1 -- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d814f360 msgr2=0x7f35d814bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dd5ab640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d814f360 0x7f35d814bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dd5ab640 1 -- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d80c88c0 msgr2=0x7f35d814b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dd5ab640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d80c88c0 0x7f35d814b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dd5ab640 1 -- 172.21.15.26:0/321836204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8152730 con 0x7f35d8158b60 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35dd5ab640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d8158b60 0x7f35d80bf2e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35d000efc0 tx=0x7f35d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35d57fa640 1 -- 172.21.15.26:0/321836204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0018020 con 0x7f35d8158b60 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35d57fa640 1 -- 172.21.15.26:0/321836204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d0014480 con 0x7f35d8158b60 2024-03-06T07:20:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d81eeee0 con 0x7f35d8158b60 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35d57fa640 1 -- 172.21.15.26:0/321836204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0014870 con 0x7f35d8158b60 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.270+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d80c0aa0 con 0x7f35d8158b60 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.271+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d804f330 con 0x7f35d8158b60 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.272+0000 7f35d57fa640 1 -- 172.21.15.26:0/321836204 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35d0005d10 con 0x7f35d8158b60 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.272+0000 7f35d57fa640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35bc06b4d0 0x7f35bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.272+0000 7f35d57fa640 1 -- 172.21.15.26:0/321836204 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f35d008ae90 con 0x7f35d8158b60 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.272+0000 7f35dcdaa640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35bc06b4d0 0x7f35bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.273+0000 7f35dcdaa640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35bc06b4d0 0x7f35bc06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35cc007fd0 tx=0x7f35cc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.276+0000 7f35d57fa640 1 -- 172.21.15.26:0/321836204 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35d0022070 con 0x7f35d8158b60 2024-03-06T07:20:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.461+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35d80acb60 con 0x7f35d8158b60 2024-03-06T07:20:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.462+0000 7f35d57fa640 1 -- 172.21.15.26:0/321836204 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f35d0010040 con 0x7f35d8158b60 2024-03-06T07:20:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:13.465 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35bc06b4d0 msgr2=0x7f35bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35bc06b4d0 0x7f35bc06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35cc007fd0 tx=0x7f35cc007de0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d8158b60 msgr2=0x7f35d80bf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d8158b60 0x7f35d80bf2e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35d000efc0 tx=0x7f35d000c490 comp rx=0 tx=0).stop 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 shutdown_connections 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35bc06b4d0 0x7f35bc06d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d8158b60 0x7f35d80bf2e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d814f360 0x7f35d814bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 --2- 172.21.15.26:0/321836204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d80c88c0 0x7f35d814b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 >> 172.21.15.26:0/321836204 conn(0x7f35d80ac570 msgr2=0x7f35d80c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 shutdown_connections 2024-03-06T07:20:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.464+0000 7f35df00e640 1 -- 172.21.15.26:0/321836204 wait complete. 2024-03-06T07:20:13.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:13.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.579+0000 7f7e394c2640 1 Processor -- start 2024-03-06T07:20:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.579+0000 7f7e394c2640 1 -- start start 2024-03-06T07:20:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e394c2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e3414bcc0 0x7f7e34155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e394c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e34158a10 0x7f7e34158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e394c2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e340c75e0 con 0x7f7e340be150 2024-03-06T07:20:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e394c2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e340c7f10 con 0x7f7e3414bcc0 2024-03-06T07:20:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e394c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340c78c0 con 0x7f7e34158a10 2024-03-06T07:20:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e327fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7e340be150 0x7f7e340c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36492/0 (socket says 172.21.15.26:36492) 2024-03-06T07:20:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e327fc640 1 -- 172.21.15.26:0/3905662089 learned_addr learned my addr 172.21.15.26:0/3905662089 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e337fe640 1 --2- 172.21.15.26:0/3905662089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e34158a10 0x7f7e34158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e32ffd640 1 --2- 172.21.15.26:0/3905662089 >> v2:172.21.15.117:3300/0 conn(0x7f7e3414bcc0 0x7f7e34155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e31ffb640 1 -- 172.21.15.26:0/3905662089 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885680452 0 0) 0x7f7e340c75e0 con 0x7f7e340be150 2024-03-06T07:20:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e337fe640 1 -- 172.21.15.26:0/3905662089 >> v2:172.21.15.117:3300/0 conn(0x7f7e3414bcc0 msgr2=0x7f7e34155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e337fe640 1 --2- 172.21.15.26:0/3905662089 >> v2:172.21.15.117:3300/0 conn(0x7f7e3414bcc0 0x7f7e34155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e337fe640 1 -- 172.21.15.26:0/3905662089 >> v1:172.21.15.26:6789/0 conn(0x7f7e340be150 legacy=0x7f7e340c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e337fe640 1 -- 172.21.15.26:0/3905662089 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e340c7ba0 con 0x7f7e34158a10 2024-03-06T07:20:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.580+0000 7f7e32ffd640 1 --2- 172.21.15.26:0/3905662089 >> v2:172.21.15.117:3300/0 conn(0x7f7e3414bcc0 0x7f7e34155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.581+0000 7f7e337fe640 1 --2- 172.21.15.26:0/3905662089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e34158a10 0x7f7e34158e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009d90 tx=0x7f7e2402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bbc313571954a20 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.581+0000 7f7e31ffb640 1 -- 172.21.15.26:0/3905662089 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24039070 con 0x7f7e34158a10 2024-03-06T07:20:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.581+0000 7f7e31ffb640 1 -- 172.21.15.26:0/3905662089 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2402cce0 con 0x7f7e34158a10 2024-03-06T07:20:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.581+0000 7f7e31ffb640 1 -- 172.21.15.26:0/3905662089 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24034660 con 0x7f7e34158a10 2024-03-06T07:20:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 -- 172.21.15.26:0/3905662089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e34158a10 msgr2=0x7f7e34158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 --2- 172.21.15.26:0/3905662089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e34158a10 0x7f7e34158e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009d90 tx=0x7f7e2402bf30 comp rx=0 tx=0).stop 2024-03-06T07:20:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 -- 172.21.15.26:0/3905662089 shutdown_connections 2024-03-06T07:20:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 --2- 172.21.15.26:0/3905662089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e34158a10 0x7f7e34158e80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 --2- 172.21.15.26:0/3905662089 >> v2:172.21.15.117:3300/0 conn(0x7f7e3414bcc0 0x7f7e34155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 -- 172.21.15.26:0/3905662089 >> 172.21.15.26:0/3905662089 conn(0x7f7e340ac600 msgr2=0x7f7e340aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 -- 172.21.15.26:0/3905662089 shutdown_connections 2024-03-06T07:20:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.582+0000 7f7e394c2640 1 -- 172.21.15.26:0/3905662089 wait complete. 2024-03-06T07:20:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.583+0000 7f7e394c2640 1 Processor -- start 2024-03-06T07:20:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.583+0000 7f7e394c2640 1 -- start start 2024-03-06T07:20:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e340be150 0x7f7e340c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e3414bcc0 0x7f7e34151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e34158a10 0x7f7e341eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340c5c60 con 0x7f7e3414bcc0 2024-03-06T07:20:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340c5ae0 con 0x7f7e34158a10 2024-03-06T07:20:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e341593c0 con 0x7f7e340be150 2024-03-06T07:20:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e3414bcc0 0x7f7e34151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e3414bcc0 0x7f7e34151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59470/0 (socket says 172.21.15.26:59470) 2024-03-06T07:20:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 -- 172.21.15.26:0/2359931990 learned_addr learned my addr 172.21.15.26:0/2359931990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e32ffd640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e340be150 0x7f7e340c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e337fe640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e34158a10 0x7f7e341eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 -- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e340be150 msgr2=0x7f7e340c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e340be150 0x7f7e340c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 -- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e34158a10 msgr2=0x7f7e341eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e34158a10 0x7f7e341eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 -- 172.21.15.26:0/2359931990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34151400 con 0x7f7e3414bcc0 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e327fc640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e3414bcc0 0x7f7e34151f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c410 tx=0x7f7e1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e2b7fe640 1 -- 172.21.15.26:0/2359931990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0074e0 con 0x7f7e3414bcc0 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e2b7fe640 1 -- 172.21.15.26:0/2359931990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c0047e0 con 0x7f7e3414bcc0 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e341ee370 con 0x7f7e3414bcc0 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e2b7fe640 1 -- 172.21.15.26:0/2359931990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c004bd0 con 0x7f7e3414bcc0 2024-03-06T07:20:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.584+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e340c1810 con 0x7f7e3414bcc0 2024-03-06T07:20:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.586+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3404f330 con 0x7f7e3414bcc0 2024-03-06T07:20:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.586+0000 7f7e2b7fe640 1 -- 172.21.15.26:0/2359931990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e1c011070 con 0x7f7e3414bcc0 2024-03-06T07:20:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.586+0000 7f7e2b7fe640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e1406b4d0 0x7f7e1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.586+0000 7f7e2b7fe640 1 -- 172.21.15.26:0/2359931990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7e1c089f80 con 0x7f7e3414bcc0 2024-03-06T07:20:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.586+0000 7f7e32ffd640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e1406b4d0 0x7f7e1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.587+0000 7f7e32ffd640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e1406b4d0 0x7f7e1406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e340c9b40 tx=0x7f7e2c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.591+0000 7f7e2b7fe640 1 -- 172.21.15.26:0/2359931990 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7e1c058500 con 0x7f7e3414bcc0 2024-03-06T07:20:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.772+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e340c83e0 con 0x7f7e3414bcc0 2024-03-06T07:20:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.772+0000 7f7e2b7fe640 1 -- 172.21.15.26:0/2359931990 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e1c05c5a0 con 0x7f7e3414bcc0 2024-03-06T07:20:13.775 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e1406b4d0 msgr2=0x7f7e1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e1406b4d0 0x7f7e1406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e340c9b40 tx=0x7f7e2c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e3414bcc0 msgr2=0x7f7e34151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e3414bcc0 0x7f7e34151f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c410 tx=0x7f7e1c00c8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 shutdown_connections 2024-03-06T07:20:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e1406b4d0 0x7f7e1406d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e34158a10 0x7f7e341eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e3414bcc0 0x7f7e34151f70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 --2- 172.21.15.26:0/2359931990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e340be150 0x7f7e340c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 >> 172.21.15.26:0/2359931990 conn(0x7f7e340ac600 msgr2=0x7f7e340abe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 shutdown_connections 2024-03-06T07:20:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.774+0000 7f7e394c2640 1 -- 172.21.15.26:0/2359931990 wait complete. 2024-03-06T07:20:13.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.890+0000 7f5e2af8c640 1 Processor -- start 2024-03-06T07:20:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.890+0000 7f5e2af8c640 1 -- start start 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e2af8c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5e240b0070 0x7f5e24155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e2af8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e2406ca70 0x7f5e240c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e2af8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e240b0450 con 0x7f5e240c5380 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e2af8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2406c020 con 0x7f5e2406ca70 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e2af8c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2406cf20 con 0x7f5e240b0070 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e23fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e2406ca70 0x7f5e240c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e29529640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5e240c5380 0x7f5e2414e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36494/0 (socket says 172.21.15.26:36494) 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e29529640 1 -- 172.21.15.26:0/3009756165 learned_addr learned my addr 172.21.15.26:0/3009756165 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e28d28640 1 --2- 172.21.15.26:0/3009756165 >> v2:172.21.15.117:3300/0 conn(0x7f5e240b0070 0x7f5e24155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e23fff640 1 -- 172.21.15.26:0/3009756165 >> v2:172.21.15.117:3300/0 conn(0x7f5e240b0070 msgr2=0x7f5e24155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e23fff640 1 --2- 172.21.15.26:0/3009756165 >> v2:172.21.15.117:3300/0 conn(0x7f5e240b0070 0x7f5e24155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e23fff640 1 -- 172.21.15.26:0/3009756165 >> v1:172.21.15.26:6789/0 conn(0x7f5e240c5380 legacy=0x7f5e2414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e23fff640 1 -- 172.21.15.26:0/3009756165 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240c77f0 con 0x7f5e2406ca70 2024-03-06T07:20:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e237fe640 1 -- 172.21.15.26:0/3009756165 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339458813 0 0) 0x7f5e0c001110 con 0x7f5e240c5380 2024-03-06T07:20:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.891+0000 7f5e28d28640 1 --2- 172.21.15.26:0/3009756165 >> v2:172.21.15.117:3300/0 conn(0x7f5e240b0070 0x7f5e24155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.892+0000 7f5e23fff640 1 --2- 172.21.15.26:0/3009756165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e2406ca70 0x7f5e240c8900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e240aa3d0 tx=0x7f5e1402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af76a182ecf37a0a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.892+0000 7f5e237fe640 1 -- 172.21.15.26:0/3009756165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14039070 con 0x7f5e2406ca70 2024-03-06T07:20:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.892+0000 7f5e237fe640 1 -- 172.21.15.26:0/3009756165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1402cbf0 con 0x7f5e2406ca70 2024-03-06T07:20:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.892+0000 7f5e237fe640 1 -- 172.21.15.26:0/3009756165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14034440 con 0x7f5e2406ca70 2024-03-06T07:20:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 -- 172.21.15.26:0/3009756165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e2406ca70 msgr2=0x7f5e240c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/3009756165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e2406ca70 0x7f5e240c8900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e240aa3d0 tx=0x7f5e1402be40 comp rx=0 tx=0).stop 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 -- 172.21.15.26:0/3009756165 shutdown_connections 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/3009756165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e2406ca70 0x7f5e240c8900 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/3009756165 >> v2:172.21.15.117:3300/0 conn(0x7f5e240b0070 0x7f5e24155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 -- 172.21.15.26:0/3009756165 >> 172.21.15.26:0/3009756165 conn(0x7f5e240ac590 msgr2=0x7f5e240ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 -- 172.21.15.26:0/3009756165 shutdown_connections 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.893+0000 7f5e2af8c640 1 -- 172.21.15.26:0/3009756165 wait complete. 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.894+0000 7f5e2af8c640 1 Processor -- start 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.894+0000 7f5e2af8c640 1 -- start start 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e2af8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2406ca70 0x7f5e240c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e2af8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e240b0070 0x7f5e241eb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e2af8c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e240c5380 0x7f5e241565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e2af8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2406c550 con 0x7f5e240b0070 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e2af8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2406c270 con 0x7f5e2406ca70 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e2af8c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2406da80 con 0x7f5e240c5380 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e240b0070 0x7f5e241eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e240b0070 0x7f5e241eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59480/0 (socket says 172.21.15.26:59480) 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 -- 172.21.15.26:0/4259797076 learned_addr learned my addr 172.21.15.26:0/4259797076 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e29529640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e240c5380 0x7f5e241565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e28d28640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2406ca70 0x7f5e240c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 -- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e240c5380 msgr2=0x7f5e241565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e240c5380 0x7f5e241565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 -- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2406ca70 msgr2=0x7f5e240c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2406ca70 0x7f5e240c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 -- 172.21.15.26:0/4259797076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e241eac90 con 0x7f5e240b0070 2024-03-06T07:20:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.895+0000 7f5e23fff640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e240b0070 0x7f5e241eb700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5e14037040 tx=0x7f5e14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.896+0000 7f5e217fa640 1 -- 172.21.15.26:0/4259797076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14039050 con 0x7f5e240b0070 2024-03-06T07:20:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.896+0000 7f5e217fa640 1 -- 172.21.15.26:0/4259797076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e14045070 con 0x7f5e240b0070 2024-03-06T07:20:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.896+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e24155610 con 0x7f5e240b0070 2024-03-06T07:20:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.896+0000 7f5e217fa640 1 -- 172.21.15.26:0/4259797076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1403e850 con 0x7f5e240b0070 2024-03-06T07:20:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.896+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e24156ae0 con 0x7f5e240b0070 2024-03-06T07:20:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.897+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e24075300 con 0x7f5e240b0070 2024-03-06T07:20:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.897+0000 7f5e217fa640 1 -- 172.21.15.26:0/4259797076 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e1403e9f0 con 0x7f5e240b0070 2024-03-06T07:20:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.897+0000 7f5e217fa640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.898+0000 7f5e217fa640 1 -- 172.21.15.26:0/4259797076 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5e140ad0c0 con 0x7f5e240b0070 2024-03-06T07:20:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.898+0000 7f5e28d28640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.898+0000 7f5e28d28640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b4d0 0x7f5e0406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009560 tx=0x7f5e1c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:13.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:13.902+0000 7f5e217fa640 1 -- 172.21.15.26:0/4259797076 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5e24075300 con 0x7f5e240b0070 2024-03-06T07:20:14.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.081+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e2406d860 con 0x7f5e240b0070 2024-03-06T07:20:14.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.082+0000 7f5e217fa640 1 -- 172.21.15.26:0/4259797076 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e1407b760 con 0x7f5e240b0070 2024-03-06T07:20:14.084 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b4d0 msgr2=0x7f5e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b4d0 0x7f5e0406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009560 tx=0x7f5e1c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e240b0070 msgr2=0x7f5e241eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e240b0070 0x7f5e241eb700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5e14037040 tx=0x7f5e14004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 shutdown_connections 2024-03-06T07:20:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e240c5380 0x7f5e241565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e240b0070 0x7f5e241eb700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 --2- 172.21.15.26:0/4259797076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2406ca70 0x7f5e240c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.083+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 >> 172.21.15.26:0/4259797076 conn(0x7f5e240ac590 msgr2=0x7f5e240ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.084+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 shutdown_connections 2024-03-06T07:20:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.084+0000 7f5e2af8c640 1 -- 172.21.15.26:0/4259797076 wait complete. 2024-03-06T07:20:14.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:14.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.200+0000 7f9c86748640 1 Processor -- start 2024-03-06T07:20:14.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.200+0000 7f9c86748640 1 -- start start 2024-03-06T07:20:14.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.200+0000 7f9c86748640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c801538c0 0x7f9c801494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.200+0000 7f9c86748640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800a97c0 0x7f9c801499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.200+0000 7f9c86748640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c80154330 con 0x7f9c800ab310 2024-03-06T07:20:14.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.200+0000 7f9c86748640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c800c46a0 con 0x7f9c801538c0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.200+0000 7f9c86748640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c80153dc0 con 0x7f9c800a97c0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c801538c0 0x7f9c801494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c801538c0 0x7f9c801494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59490/0 (socket says 172.21.15.26:59490) 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9c800ab310 0x7f9c800ab6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36496/0 (socket says 172.21.15.26:36496) 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7f7fe640 1 -- 172.21.15.26:0/2819320062 learned_addr learned my addr 172.21.15.26:0/2819320062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c84ce5640 1 --2- 172.21.15.26:0/2819320062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800a97c0 0x7f9c801499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7f7fe640 1 -- 172.21.15.26:0/2819320062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800a97c0 msgr2=0x7f9c801499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7f7fe640 1 --2- 172.21.15.26:0/2819320062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800a97c0 0x7f9c801499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7f7fe640 1 -- 172.21.15.26:0/2819320062 >> v1:172.21.15.26:6789/0 conn(0x7f9c800ab310 legacy=0x7f9c800ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7effd640 1 -- 172.21.15.26:0/2819320062 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968157785 0 0) 0x7f9c80154330 con 0x7f9c800ab310 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.201+0000 7f9c7f7fe640 1 -- 172.21.15.26:0/2819320062 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800c5d30 con 0x7f9c801538c0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c7f7fe640 1 --2- 172.21.15.26:0/2819320062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801494a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009870 tx=0x7f9c7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f844d6dffdde8d8c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c7effd640 1 -- 172.21.15.26:0/2819320062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74035020 con 0x7f9c801538c0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c7effd640 1 -- 172.21.15.26:0/2819320062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c74031640 con 0x7f9c801538c0 2024-03-06T07:20:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c7effd640 1 -- 172.21.15.26:0/2819320062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7402d040 con 0x7f9c801538c0 2024-03-06T07:20:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 -- 172.21.15.26:0/2819320062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 msgr2=0x7f9c801494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 --2- 172.21.15.26:0/2819320062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801494a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009870 tx=0x7f9c7402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 -- 172.21.15.26:0/2819320062 shutdown_connections 2024-03-06T07:20:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 --2- 172.21.15.26:0/2819320062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801494a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 --2- 172.21.15.26:0/2819320062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800a97c0 0x7f9c801499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 -- 172.21.15.26:0/2819320062 >> 172.21.15.26:0/2819320062 conn(0x7f9c800c6710 msgr2=0x7f9c800c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 -- 172.21.15.26:0/2819320062 shutdown_connections 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.202+0000 7f9c86748640 1 -- 172.21.15.26:0/2819320062 wait complete. 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.203+0000 7f9c86748640 1 Processor -- start 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c86748640 1 -- start start 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c86748640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c800a97c0 0x7f9c8014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c86748640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800ab310 0x7f9c801ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c86748640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801ec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c86748640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800c4b30 con 0x7f9c801538c0 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c86748640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800c62f0 con 0x7f9c800a97c0 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c86748640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800c4cb0 con 0x7f9c800ab310 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c84ce5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801ec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c84ce5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801ec9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59494/0 (socket says 172.21.15.26:59494) 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c84ce5640 1 -- 172.21.15.26:0/2907057898 learned_addr learned my addr 172.21.15.26:0/2907057898 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c7f7fe640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800ab310 0x7f9c801ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.204+0000 7f9c7ffff640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c800a97c0 0x7f9c8014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c84ce5640 1 -- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800ab310 msgr2=0x7f9c801ec4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c84ce5640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800ab310 0x7f9c801ec4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c84ce5640 1 -- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c800a97c0 msgr2=0x7f9c8014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c84ce5640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c800a97c0 0x7f9c8014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c84ce5640 1 -- 172.21.15.26:0/2907057898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c80150940 con 0x7f9c801538c0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c84ce5640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801ec9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800efc0 tx=0x7f9c7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c7cff9640 1 -- 172.21.15.26:0/2907057898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78018020 con 0x7f9c801538c0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c7cff9640 1 -- 172.21.15.26:0/2907057898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78014480 con 0x7f9c801538c0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c800731e0 con 0x7f9c801538c0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c7cff9640 1 -- 172.21.15.26:0/2907057898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78014870 con 0x7f9c801538c0 2024-03-06T07:20:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.205+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c80070880 con 0x7f9c801538c0 2024-03-06T07:20:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.206+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8004f330 con 0x7f9c801538c0 2024-03-06T07:20:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.206+0000 7f9c7cff9640 1 -- 172.21.15.26:0/2907057898 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c78005d10 con 0x7f9c801538c0 2024-03-06T07:20:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.207+0000 7f9c7cff9640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.207+0000 7f9c7cff9640 1 -- 172.21.15.26:0/2907057898 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9c7808b240 con 0x7f9c801538c0 2024-03-06T07:20:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.207+0000 7f9c7ffff640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.207+0000 7f9c7ffff640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c70000f60 tx=0x7f9c70007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.212+0000 7f9c7cff9640 1 -- 172.21.15.26:0/2907057898 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c78022030 con 0x7f9c801538c0 2024-03-06T07:20:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.390+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c800a7bc0 con 0x7f9c801538c0 2024-03-06T07:20:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.391+0000 7f9c7cff9640 1 -- 172.21.15.26:0/2907057898 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c78010040 con 0x7f9c801538c0 2024-03-06T07:20:14.393 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c6c06b4d0 msgr2=0x7f9c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c70000f60 tx=0x7f9c70007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 msgr2=0x7f9c801ec9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801ec9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800efc0 tx=0x7f9c7800c490 comp rx=0 tx=0).stop 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 shutdown_connections 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c801538c0 0x7f9c801ec9f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c800ab310 0x7f9c801ec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 --2- 172.21.15.26:0/2907057898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c800a97c0 0x7f9c8014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 >> 172.21.15.26:0/2907057898 conn(0x7f9c800c6710 msgr2=0x7f9c800a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.392+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 shutdown_connections 2024-03-06T07:20:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.393+0000 7f9c86748640 1 -- 172.21.15.26:0/2907057898 wait complete. 2024-03-06T07:20:14.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.508+0000 7f5265925640 1 Processor -- start 2024-03-06T07:20:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.508+0000 7f5265925640 1 -- start start 2024-03-06T07:20:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f5265925640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f52600b0070 0x7f5260155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f5265925640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52600b0450 con 0x7f526006ca70 2024-03-06T07:20:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f5265925640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526006cf20 con 0x7f52600c5380 2024-03-06T07:20:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f5265925640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526006c020 con 0x7f52600b0070 2024-03-06T07:20:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f52567fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f526006ca70 0x7f52600c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36498/0 (socket says 172.21.15.26:36498) 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f52567fc640 1 -- 172.21.15.26:0/2693140881 learned_addr learned my addr 172.21.15.26:0/2693140881 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f525effd640 1 --2- 172.21.15.26:0/2693140881 >> v2:172.21.15.117:3300/0 conn(0x7f52600b0070 0x7f5260155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f525e7fc640 1 -- 172.21.15.26:0/2693140881 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656958306 0 0) 0x7f5248001110 con 0x7f526006ca70 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f525e7fc640 1 -- 172.21.15.26:0/2693140881 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f524c003520 con 0x7f526006ca70 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f525effd640 1 -- 172.21.15.26:0/2693140881 >> v1:172.21.15.117:6789/0 conn(0x7f52600c5380 legacy=0x7f526014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f525e7fc640 1 -- 172.21.15.26:0/2693140881 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695447684 0 0) 0x7f526006cf20 con 0x7f52600c5380 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f525effd640 1 -- 172.21.15.26:0/2693140881 >> v1:172.21.15.26:6789/0 conn(0x7f526006ca70 legacy=0x7f52600c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.509+0000 7f525effd640 1 -- 172.21.15.26:0/2693140881 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600c77f0 con 0x7f52600b0070 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.510+0000 7f525effd640 1 --2- 172.21.15.26:0/2693140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 0x7f5260155c30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52500097a0 tx=0x7f525002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=864557b2cbee065e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.510+0000 7f525e7fc640 1 -- 172.21.15.26:0/2693140881 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250035020 con 0x7f52600b0070 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.510+0000 7f525e7fc640 1 -- 172.21.15.26:0/2693140881 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250031830 con 0x7f52600b0070 2024-03-06T07:20:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.510+0000 7f525e7fc640 1 -- 172.21.15.26:0/2693140881 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250031c00 con 0x7f52600b0070 2024-03-06T07:20:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.511+0000 7f5265925640 1 -- 172.21.15.26:0/2693140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 msgr2=0x7f5260155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.511+0000 7f5265925640 1 --2- 172.21.15.26:0/2693140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 0x7f5260155c30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52500097a0 tx=0x7f525002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.511+0000 7f5265925640 1 -- 172.21.15.26:0/2693140881 shutdown_connections 2024-03-06T07:20:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.511+0000 7f5265925640 1 --2- 172.21.15.26:0/2693140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 0x7f5260155c30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.511+0000 7f5265925640 1 -- 172.21.15.26:0/2693140881 >> 172.21.15.26:0/2693140881 conn(0x7f52600ac590 msgr2=0x7f52600ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.511+0000 7f5265925640 1 -- 172.21.15.26:0/2693140881 shutdown_connections 2024-03-06T07:20:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.511+0000 7f5265925640 1 -- 172.21.15.26:0/2693140881 wait complete. 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.512+0000 7f5265925640 1 Processor -- start 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.512+0000 7f5265925640 1 -- start start 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f526006ca70 0x7f526014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 0x7f52600c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c5380 0x7f52601547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526006c550 con 0x7f526006ca70 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526006c270 con 0x7f52600b0070 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526006da80 con 0x7f52600c5380 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f526006ca70 0x7f526014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f526006ca70 0x7f526014ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59502/0 (socket says 172.21.15.26:59502) 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 -- 172.21.15.26:0/1305390405 learned_addr learned my addr 172.21.15.26:0/1305390405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f52567fc640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 0x7f52600c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525f7fe640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c5380 0x7f52601547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 -- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c5380 msgr2=0x7f52601547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c5380 0x7f52601547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 -- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 msgr2=0x7f52600c3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 0x7f52600c3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 -- 172.21.15.26:0/1305390405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600c43c0 con 0x7f526006ca70 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525effd640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f526006ca70 0x7f526014ce20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52500098c0 tx=0x7f52500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525cff9640 1 -- 172.21.15.26:0/1305390405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250035040 con 0x7f526006ca70 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525cff9640 1 -- 172.21.15.26:0/1305390405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525003a840 con 0x7f526006ca70 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f525cff9640 1 -- 172.21.15.26:0/1305390405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525003ac10 con 0x7f526006ca70 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526014e890 con 0x7f526006ca70 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.513+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5260154d00 con 0x7f526006ca70 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.514+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52600c3770 con 0x7f526006ca70 2024-03-06T07:20:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.515+0000 7f525cff9640 1 -- 172.21.15.26:0/1305390405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5250038020 con 0x7f526006ca70 2024-03-06T07:20:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.515+0000 7f525cff9640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.515+0000 7f525cff9640 1 -- 172.21.15.26:0/1305390405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f52500ac5b0 con 0x7f526006ca70 2024-03-06T07:20:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.515+0000 7f52567fc640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.516+0000 7f52567fc640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52600ac540 tx=0x7f5248009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.519+0000 7f525cff9640 1 -- 172.21.15.26:0/1305390405 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f525007aa90 con 0x7f526006ca70 2024-03-06T07:20:14.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.699+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f526004f330 con 0x7f526006ca70 2024-03-06T07:20:14.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.699+0000 7f525cff9640 1 -- 172.21.15.26:0/1305390405 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f525007eb30 con 0x7f526006ca70 2024-03-06T07:20:14.702 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 msgr2=0x7f524006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52600ac540 tx=0x7f5248009040 comp rx=0 tx=0).stop 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f526006ca70 msgr2=0x7f526014ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f526006ca70 0x7f526014ce20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52500098c0 tx=0x7f52500046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 shutdown_connections 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c5380 0x7f52601547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b0070 0x7f52600c3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 --2- 172.21.15.26:0/1305390405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f526006ca70 0x7f526014ce20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 >> 172.21.15.26:0/1305390405 conn(0x7f52600ac590 msgr2=0x7f52601519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 shutdown_connections 2024-03-06T07:20:14.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.701+0000 7f5265925640 1 -- 172.21.15.26:0/1305390405 wait complete. 2024-03-06T07:20:14.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:14.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.815+0000 7f944761c640 1 Processor -- start 2024-03-06T07:20:14.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.815+0000 7f944761c640 1 -- start start 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.815+0000 7f944761c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94400c69f0 0x7f94400c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.815+0000 7f944761c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9440155130 con 0x7f94401558d0 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.815+0000 7f944761c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400ad480 con 0x7f94400c8950 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.815+0000 7f944761c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9440155440 con 0x7f94400c69f0 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f9445bb9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f94401558d0 0x7f944015fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36506/0 (socket says 172.21.15.26:36506) 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f9445bb9640 1 -- 172.21.15.26:0/2593769783 learned_addr learned my addr 172.21.15.26:0/2593769783 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f94453b8640 1 --2- 172.21.15.26:0/2593769783 >> v2:172.21.15.117:3300/0 conn(0x7f94400c69f0 0x7f94400c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f9437fff640 1 -- 172.21.15.26:0/2593769783 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735914336 0 0) 0x7f9438001490 con 0x7f94401558d0 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f9437fff640 1 -- 172.21.15.26:0/2593769783 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f942c003520 con 0x7f94401558d0 2024-03-06T07:20:14.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f94453b8640 1 -- 172.21.15.26:0/2593769783 >> v1:172.21.15.117:6789/0 conn(0x7f94400c8950 legacy=0x7f94400acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f94453b8640 1 -- 172.21.15.26:0/2593769783 >> v1:172.21.15.26:6789/0 conn(0x7f94401558d0 legacy=0x7f944015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f9437fff640 1 -- 172.21.15.26:0/2593769783 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385577074 0 0) 0x7f94400ad480 con 0x7f94400c8950 2024-03-06T07:20:14.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.816+0000 7f94453b8640 1 -- 172.21.15.26:0/2593769783 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400c5d50 con 0x7f94400c69f0 2024-03-06T07:20:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f94453b8640 1 --2- 172.21.15.26:0/2593769783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94400c69f0 0x7f94400c6dd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f943c0097a0 tx=0x7f943c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a719bbbff8863e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f9437fff640 1 -- 172.21.15.26:0/2593769783 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c035020 con 0x7f94400c69f0 2024-03-06T07:20:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f9437fff640 1 -- 172.21.15.26:0/2593769783 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c031830 con 0x7f94400c69f0 2024-03-06T07:20:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f9437fff640 1 -- 172.21.15.26:0/2593769783 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c031c00 con 0x7f94400c69f0 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f944761c640 1 -- 172.21.15.26:0/2593769783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94400c69f0 msgr2=0x7f94400c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f944761c640 1 --2- 172.21.15.26:0/2593769783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94400c69f0 0x7f94400c6dd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f943c0097a0 tx=0x7f943c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f944761c640 1 -- 172.21.15.26:0/2593769783 shutdown_connections 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f944761c640 1 --2- 172.21.15.26:0/2593769783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94400c69f0 0x7f94400c6dd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.817+0000 7f944761c640 1 -- 172.21.15.26:0/2593769783 >> 172.21.15.26:0/2593769783 conn(0x7f94400cae70 msgr2=0x7f94400cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.818+0000 7f944761c640 1 -- 172.21.15.26:0/2593769783 shutdown_connections 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.818+0000 7f944761c640 1 -- 172.21.15.26:0/2593769783 wait complete. 2024-03-06T07:20:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 Processor -- start 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 -- start start 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400c69f0 0x7f944014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94400c8950 0x7f9440158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94401558d0 0x7f944014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9440152a20 con 0x7f94400c69f0 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400c6310 con 0x7f94401558d0 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f944761c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9440152ba0 con 0x7f94400c8950 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f94453b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400c69f0 0x7f944014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f94453b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400c69f0 0x7f944014eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59512/0 (socket says 172.21.15.26:59512) 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f94453b8640 1 -- 172.21.15.26:0/2302630273 learned_addr learned my addr 172.21.15.26:0/2302630273 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f9445bb9640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94401558d0 0x7f944014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.819+0000 7f9444bb7640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94400c8950 0x7f9440158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f94453b8640 1 -- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94400c8950 msgr2=0x7f9440158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f94453b8640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94400c8950 0x7f9440158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f94453b8640 1 -- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94401558d0 msgr2=0x7f944014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f94453b8640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94401558d0 0x7f944014c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f94453b8640 1 -- 172.21.15.26:0/2302630273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94401ea560 con 0x7f94400c69f0 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f94453b8640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400c69f0 0x7f944014eb40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f943c034040 tx=0x7f943c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f9435ffb640 1 -- 172.21.15.26:0/2302630273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c040070 con 0x7f94400c69f0 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f9435ffb640 1 -- 172.21.15.26:0/2302630273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c03a830 con 0x7f94400c69f0 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94401ea820 con 0x7f94400c69f0 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f9435ffb640 1 -- 172.21.15.26:0/2302630273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c035040 con 0x7f94400c69f0 2024-03-06T07:20:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.820+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94401eade0 con 0x7f94400c69f0 2024-03-06T07:20:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.821+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944004f330 con 0x7f94400c69f0 2024-03-06T07:20:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.822+0000 7f9435ffb640 1 -- 172.21.15.26:0/2302630273 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f943c04b060 con 0x7f94400c69f0 2024-03-06T07:20:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.822+0000 7f9435ffb640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942406b4d0 0x7f942406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.822+0000 7f9435ffb640 1 -- 172.21.15.26:0/2302630273 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f943c0abbb0 con 0x7f94400c69f0 2024-03-06T07:20:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.822+0000 7f9444bb7640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942406b4d0 0x7f942406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.823+0000 7f9444bb7640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942406b4d0 0x7f942406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94400bf980 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:14.826+0000 7f9435ffb640 1 -- 172.21.15.26:0/2302630273 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f943c07a130 con 0x7f94400c69f0 2024-03-06T07:20:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.021+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f94400ca650 con 0x7f94400c69f0 2024-03-06T07:20:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.021+0000 7f9435ffb640 1 -- 172.21.15.26:0/2302630273 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f943c07e1d0 con 0x7f94400c69f0 2024-03-06T07:20:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942406b4d0 msgr2=0x7f942406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942406b4d0 0x7f942406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94400bf980 tx=0x7f9430009040 comp rx=0 tx=0).stop 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400c69f0 msgr2=0x7f944014eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400c69f0 0x7f944014eb40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f943c034040 tx=0x7f943c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 shutdown_connections 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942406b4d0 0x7f942406d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94401558d0 0x7f944014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94400c8950 0x7f9440158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 --2- 172.21.15.26:0/2302630273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400c69f0 0x7f944014eb40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 >> 172.21.15.26:0/2302630273 conn(0x7f94400cae70 msgr2=0x7f94400c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 shutdown_connections 2024-03-06T07:20:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.023+0000 7f944761c640 1 -- 172.21.15.26:0/2302630273 wait complete. 2024-03-06T07:20:15.036 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-03-06T07:20:15.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:15.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:15.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:ceph-fuse[43124]: fuse finished with error 0 and tester_r 0 2024-03-06T07:20:15.063 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:15.080 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:20:15.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:15.081 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-06T07:20:15.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.283+0000 7fdb5ed5e640 1 Processor -- start 2024-03-06T07:20:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5ed5e640 1 -- start start 2024-03-06T07:20:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5ed5e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb580c88b0 0x7fdb580c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5ed5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb58155110 con 0x7fdb580c5330 2024-03-06T07:20:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5ed5e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb580c8f40 con 0x7fdb58158db0 2024-03-06T07:20:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5ed5e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb581559e0 con 0x7fdb580c88b0 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5cafa640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdb580c5330 0x7fdb581586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36514/0 (socket says 172.21.15.26:36514) 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5cafa640 1 -- 172.21.15.26:0/3164413294 learned_addr learned my addr 172.21.15.26:0/3164413294 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.284+0000 7fdb5d2fb640 1 --2- 172.21.15.26:0/3164413294 >> v2:172.21.15.117:3300/0 conn(0x7fdb580c88b0 0x7fdb580c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.285+0000 7fdb577fe640 1 -- 172.21.15.26:0/3164413294 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990196113 0 0) 0x7fdb4c001110 con 0x7fdb580c5330 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.285+0000 7fdb577fe640 1 -- 172.21.15.26:0/3164413294 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb48003520 con 0x7fdb580c5330 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.285+0000 7fdb577fe640 1 -- 172.21.15.26:0/3164413294 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487938470 0 0) 0x7fdb580c8f40 con 0x7fdb58158db0 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.285+0000 7fdb577fe640 1 -- 172.21.15.26:0/3164413294 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb4c001110 con 0x7fdb58158db0 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.285+0000 7fdb5d2fb640 1 -- 172.21.15.26:0/3164413294 >> v1:172.21.15.117:6789/0 conn(0x7fdb58158db0 legacy=0x7fdb5815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.285+0000 7fdb5d2fb640 1 -- 172.21.15.26:0/3164413294 >> v1:172.21.15.26:6789/0 conn(0x7fdb580c5330 legacy=0x7fdb581586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.285+0000 7fdb5d2fb640 1 -- 172.21.15.26:0/3164413294 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58155390 con 0x7fdb580c88b0 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5d2fb640 1 --2- 172.21.15.26:0/3164413294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 0x7fdb580c5710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009d90 tx=0x7fdb5002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6d8b079bd1c3b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb577fe640 1 -- 172.21.15.26:0/3164413294 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50039070 con 0x7fdb580c88b0 2024-03-06T07:20:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb577fe640 1 -- 172.21.15.26:0/3164413294 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5002cc60 con 0x7fdb580c88b0 2024-03-06T07:20:15.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb577fe640 1 -- 172.21.15.26:0/3164413294 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50034660 con 0x7fdb580c88b0 2024-03-06T07:20:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/3164413294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 msgr2=0x7fdb580c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/3164413294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 0x7fdb580c5710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009d90 tx=0x7fdb5002beb0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/3164413294 shutdown_connections 2024-03-06T07:20:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/3164413294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 0x7fdb580c5710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/3164413294 >> 172.21.15.26:0/3164413294 conn(0x7fdb580ac560 msgr2=0x7fdb580ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/3164413294 shutdown_connections 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.286+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/3164413294 wait complete. 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.287+0000 7fdb5ed5e640 1 Processor -- start 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.287+0000 7fdb5ed5e640 1 -- start start 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5ed5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb580c5330 0x7fdb581512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5ed5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 0x7fdb580ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5ed5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb581eb990 0x7fdb58148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5ed5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581529d0 con 0x7fdb580c5330 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5ed5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5806d090 con 0x7fdb580c88b0 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5ed5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58152b50 con 0x7fdb581eb990 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5cafa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb580c5330 0x7fdb581512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5cafa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb580c5330 0x7fdb581512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59528/0 (socket says 172.21.15.26:59528) 2024-03-06T07:20:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5cafa640 1 -- 172.21.15.26:0/1130164440 learned_addr learned my addr 172.21.15.26:0/1130164440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb57fff640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 0x7fdb580ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.288+0000 7fdb5d2fb640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb581eb990 0x7fdb58148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5cafa640 1 -- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb581eb990 msgr2=0x7fdb58148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5cafa640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb581eb990 0x7fdb58148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5cafa640 1 -- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 msgr2=0x7fdb580ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5cafa640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 0x7fdb580ab3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5cafa640 1 -- 172.21.15.26:0/1130164440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58151ab0 con 0x7fdb580c5330 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5cafa640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb580c5330 0x7fdb581512b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00c680 tx=0x7fdb4c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb557fa640 1 -- 172.21.15.26:0/1130164440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c017070 con 0x7fdb580c5330 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb557fa640 1 -- 172.21.15.26:0/1130164440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c005ab0 con 0x7fdb580c5330 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb557fa640 1 -- 172.21.15.26:0/1130164440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c007070 con 0x7fdb580c5330 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb581517f0 con 0x7fdb580c5330 2024-03-06T07:20:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.289+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb581eaeb0 con 0x7fdb580c5330 2024-03-06T07:20:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.290+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5804f330 con 0x7fdb580c5330 2024-03-06T07:20:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.291+0000 7fdb557fa640 1 -- 172.21.15.26:0/1130164440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb4c01a070 con 0x7fdb580c5330 2024-03-06T07:20:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.291+0000 7fdb557fa640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3c06b500 0x7fdb3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.291+0000 7fdb557fa640 1 -- 172.21.15.26:0/1130164440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdb4c089910 con 0x7fdb580c5330 2024-03-06T07:20:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.291+0000 7fdb57fff640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3c06b500 0x7fdb3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.291+0000 7fdb57fff640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3c06b500 0x7fdb3c06d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb44007fd0 tx=0x7fdb44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.295+0000 7fdb557fa640 1 -- 172.21.15.26:0/1130164440 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb58155110 con 0x7fdb580c5330 2024-03-06T07:20:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.478+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb5806cdc0 con 0x7fdb580c5330 2024-03-06T07:20:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.480+0000 7fdb557fa640 1 -- 172.21.15.26:0/1130164440 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fdb4c057e90 con 0x7fdb580c5330 2024-03-06T07:20:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:15.483 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3c06b500 msgr2=0x7fdb3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3c06b500 0x7fdb3c06d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb44007fd0 tx=0x7fdb44009040 comp rx=0 tx=0).stop 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb580c5330 msgr2=0x7fdb581512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb580c5330 0x7fdb581512b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00c680 tx=0x7fdb4c00cb40 comp rx=0 tx=0).stop 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 shutdown_connections 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3c06b500 0x7fdb3c06d9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb581eb990 0x7fdb58148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb580c88b0 0x7fdb580ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 --2- 172.21.15.26:0/1130164440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb580c5330 0x7fdb581512b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 >> 172.21.15.26:0/1130164440 conn(0x7fdb580ac560 msgr2=0x7fdb5814f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 shutdown_connections 2024-03-06T07:20:15.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.482+0000 7fdb5ed5e640 1 -- 172.21.15.26:0/1130164440 wait complete. 2024-03-06T07:20:15.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.595+0000 7fcc28ea4640 1 Processor -- start 2024-03-06T07:20:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.595+0000 7fcc28ea4640 1 -- start start 2024-03-06T07:20:15.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc28ea4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc240c88c0 0x7fcc24155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc28ea4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc240c73d0 con 0x7fcc2414f360 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc28ea4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc240be2e0 con 0x7fcc24158b60 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc28ea4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc24159700 con 0x7fcc240c88c0 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc21d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc240c88c0 0x7fcc24155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc2259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcc2414f360 0x7fcc240c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36518/0 (socket says 172.21.15.26:36518) 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc2259c640 1 -- 172.21.15.26:0/4134133182 learned_addr learned my addr 172.21.15.26:0/4134133182 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc21d9b640 1 -- 172.21.15.26:0/4134133182 >> v1:172.21.15.117:6789/0 conn(0x7fcc24158b60 legacy=0x7fcc24158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc21d9b640 1 -- 172.21.15.26:0/4134133182 >> v1:172.21.15.26:6789/0 conn(0x7fcc2414f360 legacy=0x7fcc240c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc2159a640 1 -- 172.21.15.26:0/4134133182 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221159647 0 0) 0x7fcc240be2e0 con 0x7fcc24158b60 2024-03-06T07:20:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.597+0000 7fcc21d9b640 1 -- 172.21.15.26:0/4134133182 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240c7670 con 0x7fcc240c88c0 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.598+0000 7fcc21d9b640 1 --2- 172.21.15.26:0/4134133182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc240c88c0 0x7fcc24155c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0097a0 tx=0x7fcc0c02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e91e04bfc0ab94 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.598+0000 7fcc2159a640 1 -- 172.21.15.26:0/4134133182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c037070 con 0x7fcc240c88c0 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.598+0000 7fcc2159a640 1 -- 172.21.15.26:0/4134133182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c030710 con 0x7fcc240c88c0 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.598+0000 7fcc2159a640 1 -- 172.21.15.26:0/4134133182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c030ae0 con 0x7fcc240c88c0 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.598+0000 7fcc28ea4640 1 -- 172.21.15.26:0/4134133182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc240c88c0 msgr2=0x7fcc24155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.598+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/4134133182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc240c88c0 0x7fcc24155c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0097a0 tx=0x7fcc0c02b710 comp rx=0 tx=0).stop 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.599+0000 7fcc28ea4640 1 -- 172.21.15.26:0/4134133182 shutdown_connections 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.599+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/4134133182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc240c88c0 0x7fcc24155c10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.599+0000 7fcc28ea4640 1 -- 172.21.15.26:0/4134133182 >> 172.21.15.26:0/4134133182 conn(0x7fcc240ac570 msgr2=0x7fcc240ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.599+0000 7fcc28ea4640 1 -- 172.21.15.26:0/4134133182 shutdown_connections 2024-03-06T07:20:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.599+0000 7fcc28ea4640 1 -- 172.21.15.26:0/4134133182 wait complete. 2024-03-06T07:20:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 Processor -- start 2024-03-06T07:20:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 -- start start 2024-03-06T07:20:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc240c88c0 0x7fcc2414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2414f360 0x7fcc24157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc24158b60 0x7fcc241ead30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240c7c30 con 0x7fcc2414f360 2024-03-06T07:20:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240c5a50 con 0x7fcc240c88c0 2024-03-06T07:20:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.600+0000 7fcc28ea4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240c5bd0 con 0x7fcc24158b60 2024-03-06T07:20:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2414f360 0x7fcc24157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2414f360 0x7fcc24157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59548/0 (socket says 172.21.15.26:59548) 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 -- 172.21.15.26:0/1499459868 learned_addr learned my addr 172.21.15.26:0/1499459868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc2259c640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc240c88c0 0x7fcc2414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc22d9d640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc24158b60 0x7fcc241ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 -- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc24158b60 msgr2=0x7fcc241ead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc24158b60 0x7fcc241ead30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 -- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc240c88c0 msgr2=0x7fcc2414b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc240c88c0 0x7fcc2414b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 -- 172.21.15.26:0/1499459868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2414b620 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc21d9b640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2414f360 0x7fcc24157ae0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098c0 tx=0x7fcc0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc1affd640 1 -- 172.21.15.26:0/1499459868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c042070 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc1affd640 1 -- 172.21.15.26:0/1499459868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c032a40 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc24152650 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc1affd640 1 -- 172.21.15.26:0/1499459868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c037040 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.601+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc240bff90 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.602+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2404f330 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.603+0000 7fcc1affd640 1 -- 172.21.15.26:0/1499459868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc0c045070 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.603+0000 7fcc1affd640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc0406b4d0 0x7fcc0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.603+0000 7fcc1affd640 1 -- 172.21.15.26:0/1499459868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcc0c0ac250 con 0x7fcc2414f360 2024-03-06T07:20:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.603+0000 7fcc2259c640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc0406b4d0 0x7fcc0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.604+0000 7fcc2259c640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc0406b4d0 0x7fcc0406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c007e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.607+0000 7fcc1affd640 1 -- 172.21.15.26:0/1499459868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcc2414b620 con 0x7fcc2414f360 2024-03-06T07:20:15.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.788+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc240c8d50 con 0x7fcc2414f360 2024-03-06T07:20:15.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.789+0000 7fcc1affd640 1 -- 172.21.15.26:0/1499459868 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fcc0c07a8f0 con 0x7fcc2414f360 2024-03-06T07:20:15.791 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:15.791 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:15.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc0406b4d0 msgr2=0x7fcc0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc0406b4d0 0x7fcc0406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c007e20 comp rx=0 tx=0).stop 2024-03-06T07:20:15.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2414f360 msgr2=0x7fcc24157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2414f360 0x7fcc24157ae0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098c0 tx=0x7fcc0c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 shutdown_connections 2024-03-06T07:20:15.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc0406b4d0 0x7fcc0406d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc24158b60 0x7fcc241ead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2414f360 0x7fcc24157ae0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 --2- 172.21.15.26:0/1499459868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc240c88c0 0x7fcc2414b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 >> 172.21.15.26:0/1499459868 conn(0x7fcc240ac570 msgr2=0x7fcc240ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 shutdown_connections 2024-03-06T07:20:15.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.791+0000 7fcc28ea4640 1 -- 172.21.15.26:0/1499459868 wait complete. 2024-03-06T07:20:15.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:20:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 Processor -- start 2024-03-06T07:20:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 -- start start 2024-03-06T07:20:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd814bcc0 0x7fdbd8155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbd80be150 0x7fdbd80c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdbd80c5910 0x7fdbd8158a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd80c7f10 con 0x7fdbd80be150 2024-03-06T07:20:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd80c75e0 con 0x7fdbd80c5910 2024-03-06T07:20:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbddf5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd80c78c0 con 0x7fdbd814bcc0 2024-03-06T07:20:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.907+0000 7fdbd6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbd80be150 0x7fdbd80c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbd80be150 0x7fdbd80c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59554/0 (socket says 172.21.15.26:59554) 2024-03-06T07:20:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd6ffd640 1 -- 172.21.15.26:0/814827213 learned_addr learned my addr 172.21.15.26:0/814827213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:15.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd77fe640 1 --2- 172.21.15.26:0/814827213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd814bcc0 0x7fdbd8155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd7fff640 1 --2- 172.21.15.26:0/814827213 >> v2:172.21.15.117:3300/0 conn(0x7fdbd80c5910 0x7fdbd8158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd6ffd640 1 -- 172.21.15.26:0/814827213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd814bcc0 msgr2=0x7fdbd8155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:15.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd6ffd640 1 --2- 172.21.15.26:0/814827213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd814bcc0 0x7fdbd8155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd6ffd640 1 -- 172.21.15.26:0/814827213 >> v2:172.21.15.117:3300/0 conn(0x7fdbd80c5910 msgr2=0x7fdbd8158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:15.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd6ffd640 1 --2- 172.21.15.26:0/814827213 >> v2:172.21.15.117:3300/0 conn(0x7fdbd80c5910 0x7fdbd8158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd6ffd640 1 -- 172.21.15.26:0/814827213 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80c7ba0 con 0x7fdbd80be150 2024-03-06T07:20:15.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.908+0000 7fdbd77fe640 1 --2- 172.21.15.26:0/814827213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd814bcc0 0x7fdbd8155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:15.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbd6ffd640 1 --2- 172.21.15.26:0/814827213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd80be150 0x7fdbd80c53d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80c7390 tx=0x7fdbc802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf00860d1afd437 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbd67fc640 1 -- 172.21.15.26:0/814827213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8034070 con 0x7fdbd80be150 2024-03-06T07:20:15.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbd67fc640 1 -- 172.21.15.26:0/814827213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc8030860 con 0x7fdbd80be150 2024-03-06T07:20:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbd67fc640 1 -- 172.21.15.26:0/814827213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8030c30 con 0x7fdbd80be150 2024-03-06T07:20:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 -- 172.21.15.26:0/814827213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd80be150 msgr2=0x7fdbd80c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/814827213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd80be150 0x7fdbd80c53d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80c7390 tx=0x7fdbc802b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 -- 172.21.15.26:0/814827213 shutdown_connections 2024-03-06T07:20:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/814827213 >> v2:172.21.15.117:3300/0 conn(0x7fdbd80c5910 0x7fdbd8158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/814827213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd80be150 0x7fdbd80c53d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/814827213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd814bcc0 0x7fdbd8155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 -- 172.21.15.26:0/814827213 >> 172.21.15.26:0/814827213 conn(0x7fdbd80ac600 msgr2=0x7fdbd80aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 -- 172.21.15.26:0/814827213 shutdown_connections 2024-03-06T07:20:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.909+0000 7fdbddf5c640 1 -- 172.21.15.26:0/814827213 wait complete. 2024-03-06T07:20:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.910+0000 7fdbddf5c640 1 Processor -- start 2024-03-06T07:20:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbddf5c640 1 -- start start 2024-03-06T07:20:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbddf5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd80be150 0x7fdbd81e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbddf5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd80c5910 0x7fdbd814a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbddf5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd814bcc0 0x7fdbd814adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbddf5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81590d0 con 0x7fdbd814bcc0 2024-03-06T07:20:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbddf5c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8158f50 con 0x7fdbd80c5910 2024-03-06T07:20:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbddf5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8159250 con 0x7fdbd80be150 2024-03-06T07:20:15.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd814bcc0 0x7fdbd814adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd80be150 0x7fdbd81e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd814bcc0 0x7fdbd814adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59560/0 (socket says 172.21.15.26:59560) 2024-03-06T07:20:15.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd80be150 0x7fdbd81e1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:38274/0 (socket says 172.21.15.26:38274) 2024-03-06T07:20:15.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbd7fff640 1 -- 172.21.15.26:0/646253690 learned_addr learned my addr 172.21.15.26:0/646253690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:15.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.911+0000 7fdbd6ffd640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd80c5910 0x7fdbd814a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbd7fff640 1 -- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd80be150 msgr2=0x7fdbd81e1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbd7fff640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd80be150 0x7fdbd81e1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbd7fff640 1 -- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd80c5910 msgr2=0x7fdbd814a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbd7fff640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd80c5910 0x7fdbd814a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbd7fff640 1 -- 172.21.15.26:0/646253690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd81eec90 con 0x7fdbd814bcc0 2024-03-06T07:20:15.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbd77fe640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd80be150 0x7fdbd81e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:15.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbd7fff640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd814bcc0 0x7fdbd814adf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000ef00 tx=0x7fdbd000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbbffff640 1 -- 172.21.15.26:0/646253690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0018020 con 0x7fdbd814bcc0 2024-03-06T07:20:15.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbbffff640 1 -- 172.21.15.26:0/646253690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd0014400 con 0x7fdbd814bcc0 2024-03-06T07:20:15.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd81eef30 con 0x7fdbd814bcc0 2024-03-06T07:20:15.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbbffff640 1 -- 172.21.15.26:0/646253690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00147f0 con 0x7fdbd814bcc0 2024-03-06T07:20:15.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.912+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd80bff00 con 0x7fdbd814bcc0 2024-03-06T07:20:15.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.913+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd804f330 con 0x7fdbd814bcc0 2024-03-06T07:20:15.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.913+0000 7fdbbffff640 1 -- 172.21.15.26:0/646253690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbd0005d10 con 0x7fdbd814bcc0 2024-03-06T07:20:15.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.914+0000 7fdbbffff640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdbc406b4d0 0x7fdbc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.914+0000 7fdbbffff640 1 -- 172.21.15.26:0/646253690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdbd008aa90 con 0x7fdbd814bcc0 2024-03-06T07:20:15.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.914+0000 7fdbd77fe640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdbc406b4d0 0x7fdbc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.914+0000 7fdbd77fe640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdbc406b4d0 0x7fdbc406d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009560 tx=0x7fdbcc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:15.918+0000 7fdbbffff640 1 -- 172.21.15.26:0/646253690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdbd0022070 con 0x7fdbd814bcc0 2024-03-06T07:20:16.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:16.138+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdbd80c83e0 con 0x7fdbd814bcc0 2024-03-06T07:20:17.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.100+0000 7fdbbffff640 1 -- 172.21.15.26:0/646253690 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdbd0058f80 con 0x7fdbd814bcc0 2024-03-06T07:20:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdbc406b4d0 msgr2=0x7fdbc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdbc406b4d0 0x7fdbc406d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009560 tx=0x7fdbcc009190 comp rx=0 tx=0).stop 2024-03-06T07:20:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd814bcc0 msgr2=0x7fdbd814adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd814bcc0 0x7fdbd814adf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000ef00 tx=0x7fdbd000b480 comp rx=0 tx=0).stop 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 shutdown_connections 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdbc406b4d0 0x7fdbc406d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd814bcc0 0x7fdbd814adf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd80c5910 0x7fdbd814a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 --2- 172.21.15.26:0/646253690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbd80be150 0x7fdbd81e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 >> 172.21.15.26:0/646253690 conn(0x7fdbd80ac600 msgr2=0x7fdbd8156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.103+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 shutdown_connections 2024-03-06T07:20:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.104+0000 7fdbddf5c640 1 -- 172.21.15.26:0/646253690 wait complete. 2024-03-06T07:20:17.118 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-06T07:20:17.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:20:17.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.227+0000 7f8f79b13640 1 Processor -- start 2024-03-06T07:20:17.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.227+0000 7f8f79b13640 1 -- start start 2024-03-06T07:20:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f79b13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f74155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f79b13640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f740c5380 0x7f8f740c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f79b13640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f740b0450 con 0x7f8f7406ca70 2024-03-06T07:20:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f79b13640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7406c020 con 0x7f8f740c5380 2024-03-06T07:20:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f79b13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740c77f0 con 0x7f8f740b0070 2024-03-06T07:20:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f73fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f740c5380 0x7f8f740c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f72ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f7406ca70 0x7f8f740c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44110/0 (socket says 172.21.15.26:44110) 2024-03-06T07:20:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f72ffd640 1 -- 172.21.15.26:0/1440783557 learned_addr learned my addr 172.21.15.26:0/1440783557 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f737fe640 1 --2- 172.21.15.26:0/1440783557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f74155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f73fff640 1 -- 172.21.15.26:0/1440783557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 msgr2=0x7f8f74155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f73fff640 1 --2- 172.21.15.26:0/1440783557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f74155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f73fff640 1 -- 172.21.15.26:0/1440783557 >> v1:172.21.15.26:6789/0 conn(0x7f8f7406ca70 legacy=0x7f8f740c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.228+0000 7f8f73fff640 1 -- 172.21.15.26:0/1440783557 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740c7ad0 con 0x7f8f740c5380 2024-03-06T07:20:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.229+0000 7f8f73fff640 1 --2- 172.21.15.26:0/1440783557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f740c57d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009d90 tx=0x7f8f6c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfc28dc5fcd6611 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.229+0000 7f8f727fc640 1 -- 172.21.15.26:0/1440783557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c039070 con 0x7f8f740c5380 2024-03-06T07:20:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.229+0000 7f8f727fc640 1 -- 172.21.15.26:0/1440783557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c02cbc0 con 0x7f8f740c5380 2024-03-06T07:20:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.229+0000 7f8f727fc640 1 -- 172.21.15.26:0/1440783557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c034440 con 0x7f8f740c5380 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 -- 172.21.15.26:0/1440783557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 msgr2=0x7f8f740c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 --2- 172.21.15.26:0/1440783557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f740c57d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009d90 tx=0x7f8f6c02be10 comp rx=0 tx=0).stop 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 -- 172.21.15.26:0/1440783557 shutdown_connections 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 --2- 172.21.15.26:0/1440783557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f740c57d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 --2- 172.21.15.26:0/1440783557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f74155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 -- 172.21.15.26:0/1440783557 >> 172.21.15.26:0/1440783557 conn(0x7f8f740ac590 msgr2=0x7f8f740ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 -- 172.21.15.26:0/1440783557 shutdown_connections 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.230+0000 7f8f79b13640 1 -- 172.21.15.26:0/1440783557 wait complete. 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.231+0000 7f8f79b13640 1 Processor -- start 2024-03-06T07:20:17.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.231+0000 7f8f79b13640 1 -- start start 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f79b13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7406ca70 0x7f8f74150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f79b13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f741ec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f79b13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f741ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f79b13640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7406cef0 con 0x7f8f740c5380 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f79b13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7406c550 con 0x7f8f7406ca70 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f79b13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7406da80 con 0x7f8f740b0070 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f741ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f741ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46668/0 (socket says 172.21.15.26:46668) 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 -- 172.21.15.26:0/3808491239 learned_addr learned my addr 172.21.15.26:0/3808491239 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f737fe640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7406ca70 0x7f8f74150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f72ffd640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f741ec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 -- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 msgr2=0x7f8f741ec0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f741ec0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 -- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7406ca70 msgr2=0x7f8f74150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7406ca70 0x7f8f74150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 -- 172.21.15.26:0/3808491239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74151bf0 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.232+0000 7f8f73fff640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f741ee3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009eb0 tx=0x7f8f6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.233+0000 7f8f5bfff640 1 -- 172.21.15.26:0/3808491239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c039050 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.233+0000 7f8f5bfff640 1 -- 172.21.15.26:0/3808491239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c045070 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.233+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f74151ed0 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.233+0000 7f8f5bfff640 1 -- 172.21.15.26:0/3808491239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c009450 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.233+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7414ff90 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.234+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7404f330 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.234+0000 7f8f5bfff640 1 -- 172.21.15.26:0/3808491239 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f6c03e850 con 0x7f8f740c5380 2024-03-06T07:20:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.234+0000 7f8f5bfff640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f6006b4d0 0x7f8f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.235+0000 7f8f5bfff640 1 -- 172.21.15.26:0/3808491239 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8f6c0ad2a0 con 0x7f8f740c5380 2024-03-06T07:20:17.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.235+0000 7f8f737fe640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f6006b4d0 0x7f8f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.235+0000 7f8f737fe640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f6006b4d0 0x7f8f6006d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009b40 tx=0x7f8f64009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.239+0000 7f8f5bfff640 1 -- 172.21.15.26:0/3808491239 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f6c07b8a0 con 0x7f8f740c5380 2024-03-06T07:20:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:17.458+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8f7406d860 con 0x7f8f740c5380 2024-03-06T07:20:18.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.101+0000 7f8f5bfff640 1 -- 172.21.15.26:0/3808491239 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8f6c07f940 con 0x7f8f740c5380 2024-03-06T07:20:18.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f6006b4d0 msgr2=0x7f8f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f6006b4d0 0x7f8f6006d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009b40 tx=0x7f8f64009190 comp rx=0 tx=0).stop 2024-03-06T07:20:18.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 msgr2=0x7f8f741ee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f741ee3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009eb0 tx=0x7f8f6c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:18.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 shutdown_connections 2024-03-06T07:20:18.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f6006b4d0 0x7f8f6006d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740c5380 0x7f8f741ee3e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f740b0070 0x7f8f741ec0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 --2- 172.21.15.26:0/3808491239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7406ca70 0x7f8f74150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.104+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 >> 172.21.15.26:0/3808491239 conn(0x7f8f740ac590 msgr2=0x7f8f740ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.105+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 shutdown_connections 2024-03-06T07:20:18.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.105+0000 7f8f79b13640 1 -- 172.21.15.26:0/3808491239 wait complete. 2024-03-06T07:20:18.120 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:20:18.120 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.226+0000 7fd90398c640 1 Processor -- start 2024-03-06T07:20:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.226+0000 7fd90398c640 1 -- start start 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.227+0000 7fd90398c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.227+0000 7fd90398c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0c73d0 con 0x7fd8fc14f360 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.227+0000 7fd90398c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0be2e0 con 0x7fd8fc158b60 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.227+0000 7fd90398c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc159700 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.227+0000 7fd901728640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd8fc14f360 0x7fd8fc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44114/0 (socket says 172.21.15.26:44114) 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.227+0000 7fd901728640 1 -- 172.21.15.26:0/790942643 learned_addr learned my addr 172.21.15.26:0/790942643 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.227+0000 7fd900f27640 1 --2- 172.21.15.26:0/790942643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd8fbfff640 1 -- 172.21.15.26:0/790942643 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223727669 0 0) 0x7fd8fc0c73d0 con 0x7fd8fc14f360 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd8fbfff640 1 -- 172.21.15.26:0/790942643 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e4003520 con 0x7fd8fc14f360 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd8fbfff640 1 -- 172.21.15.26:0/790942643 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976024716 0 0) 0x7fd8fc0be2e0 con 0x7fd8fc158b60 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd8fbfff640 1 -- 172.21.15.26:0/790942643 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8fc0c73d0 con 0x7fd8fc158b60 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd900f27640 1 -- 172.21.15.26:0/790942643 >> v1:172.21.15.117:6789/0 conn(0x7fd8fc158b60 legacy=0x7fd8fc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd900f27640 1 -- 172.21.15.26:0/790942643 >> v1:172.21.15.26:6789/0 conn(0x7fd8fc14f360 legacy=0x7fd8fc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd900f27640 1 -- 172.21.15.26:0/790942643 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0c7670 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.228+0000 7fd900f27640 1 --2- 172.21.15.26:0/790942643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc155c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00097d0 tx=0x7fd8f002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3313177388da7c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd8fbfff640 1 -- 172.21.15.26:0/790942643 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0035020 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd8fbfff640 1 -- 172.21.15.26:0/790942643 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f0031800 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd8fbfff640 1 -- 172.21.15.26:0/790942643 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0031bd0 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd90398c640 1 -- 172.21.15.26:0/790942643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 msgr2=0x7fd8fc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd90398c640 1 --2- 172.21.15.26:0/790942643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc155c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00097d0 tx=0x7fd8f002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd90398c640 1 -- 172.21.15.26:0/790942643 shutdown_connections 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd90398c640 1 --2- 172.21.15.26:0/790942643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc155c10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd90398c640 1 -- 172.21.15.26:0/790942643 >> 172.21.15.26:0/790942643 conn(0x7fd8fc0ac570 msgr2=0x7fd8fc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd90398c640 1 -- 172.21.15.26:0/790942643 shutdown_connections 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.229+0000 7fd90398c640 1 -- 172.21.15.26:0/790942643 wait complete. 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.230+0000 7fd90398c640 1 Processor -- start 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.230+0000 7fd90398c640 1 -- start start 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd90398c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd90398c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8fc14f360 0x7fd8fc0ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd90398c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8fc158b60 0x7fd8fc144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd90398c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c7c30 con 0x7fd8fc158b60 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd90398c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c5a50 con 0x7fd8fc14f360 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd90398c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c5bd0 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901f29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8fc158b60 0x7fd8fc144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc1573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46994/0 (socket says 172.21.15.26:46994) 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 -- 172.21.15.26:0/1817202426 learned_addr learned my addr 172.21.15.26:0/1817202426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd900f27640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8fc14f360 0x7fd8fc0ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901f29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8fc158b60 0x7fd8fc144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46672/0 (socket says 172.21.15.26:46672) 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 -- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8fc14f360 msgr2=0x7fd8fc0ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8fc14f360 0x7fd8fc0ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 -- 172.21.15.26:0/1817202426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8fc158b60 msgr2=0x7fd8fc144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8fc158b60 0x7fd8fc144d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.231+0000 7fd901728640 1 -- 172.21.15.26:0/1817202426 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0b0470 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.232+0000 7fd901728640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc1573e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c530 tx=0x7fd8f400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.232+0000 7fd8f9ffb640 1 -- 172.21.15.26:0/1817202426 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4017070 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.232+0000 7fd8f9ffb640 1 -- 172.21.15.26:0/1817202426 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4005d00 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.232+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc0ae590 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.232+0000 7fd8f9ffb640 1 -- 172.21.15.26:0/1817202426 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40072b0 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.232+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc157920 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.233+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc069460 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.233+0000 7fd8f9ffb640 1 -- 172.21.15.26:0/1817202426 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8f401a070 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.233+0000 7fd8f9ffb640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8e806b4d0 0x7fd8e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.234+0000 7fd8f9ffb640 1 -- 172.21.15.26:0/1817202426 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd8f4089820 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.234+0000 7fd900f27640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8e806b4d0 0x7fd8e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.234+0000 7fd900f27640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8e806b4d0 0x7fd8e806d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0036000 tx=0x7fd8f003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.238+0000 7fd8f9ffb640 1 -- 172.21.15.26:0/1817202426 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd8fc069460 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.459+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8fc04f330 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.459+0000 7fd8f9ffb640 1 -- 172.21.15.26:0/1817202426 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7fd8f405ce20 con 0x7fd8fc0c88c0 2024-03-06T07:20:18.461 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:18.461 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:18.462 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:18.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8e806b4d0 msgr2=0x7fd8e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8e806b4d0 0x7fd8e806d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0036000 tx=0x7fd8f003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:18.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 msgr2=0x7fd8fc1573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc1573e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c530 tx=0x7fd8f400c9f0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 shutdown_connections 2024-03-06T07:20:18.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8e806b4d0 0x7fd8e806d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8fc158b60 0x7fd8fc144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8fc14f360 0x7fd8fc0ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 --2- 172.21.15.26:0/1817202426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8fc0c88c0 0x7fd8fc1573e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 >> 172.21.15.26:0/1817202426 conn(0x7fd8fc0ac570 msgr2=0x7fd8fc0ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 shutdown_connections 2024-03-06T07:20:18.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.461+0000 7fd90398c640 1 -- 172.21.15.26:0/1817202426 wait complete. 2024-03-06T07:20:18.474 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.573+0000 7f2aa8de9640 1 Processor -- start 2024-03-06T07:20:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.573+0000 7f2aa8de9640 1 -- start start 2024-03-06T07:20:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa8de9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2aa4158db0 0x7f2aa40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa8de9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c88b0 0x7f2aa415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa8de9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa4155110 con 0x7f2aa40c5330 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa8de9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa41559e0 con 0x7f2aa4158db0 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa8de9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa40c8f40 con 0x7f2aa40c88b0 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c88b0 0x7f2aa415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa1d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2aa4158db0 0x7f2aa40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa259c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2aa40c5330 0x7f2aa41586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35900/0 (socket says 172.21.15.26:35900) 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c88b0 0x7f2aa415fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47008/0 (socket says 172.21.15.26:47008) 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa1d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2aa4158db0 0x7f2aa40c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36848/0 (socket says 172.21.15.26:36848) 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.574+0000 7f2aa259c640 1 -- 172.21.15.26:0/57947768 learned_addr learned my addr 172.21.15.26:0/57947768 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.575+0000 7f2aa1d9b640 1 -- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c88b0 msgr2=0x7f2aa415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:18.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.575+0000 7f2aa1d9b640 1 --2- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c88b0 0x7f2aa415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.575+0000 7f2aa1d9b640 1 -- 172.21.15.26:0/57947768 >> v1:172.21.15.117:6789/0 conn(0x7f2aa40c5330 legacy=0x7f2aa41586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.575+0000 7f2aa1d9b640 1 -- 172.21.15.26:0/57947768 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4155390 con 0x7f2aa4158db0 2024-03-06T07:20:18.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.575+0000 7f2aa159a640 1 -- 172.21.15.26:0/57947768 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088127965 0 0) 0x7f2a9c001110 con 0x7f2aa40c5330 2024-03-06T07:20:18.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.575+0000 7f2aa2d9d640 1 --2- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c88b0 0x7f2aa415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:18.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa1d9b640 1 --2- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa4158db0 0x7f2aa40c5710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0097a0 tx=0x7f2a8c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6696e6aadfd743b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa159a640 1 -- 172.21.15.26:0/57947768 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0315a0 con 0x7f2aa4158db0 2024-03-06T07:20:18.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa159a640 1 -- 172.21.15.26:0/57947768 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c037070 con 0x7f2aa4158db0 2024-03-06T07:20:18.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa159a640 1 -- 172.21.15.26:0/57947768 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c031960 con 0x7f2aa4158db0 2024-03-06T07:20:18.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 -- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa4158db0 msgr2=0x7f2aa40c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa4158db0 0x7f2aa40c5710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0097a0 tx=0x7f2a8c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 -- 172.21.15.26:0/57947768 shutdown_connections 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c88b0 0x7f2aa415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/57947768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa4158db0 0x7f2aa40c5710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 -- 172.21.15.26:0/57947768 >> 172.21.15.26:0/57947768 conn(0x7f2aa40ac560 msgr2=0x7f2aa40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 -- 172.21.15.26:0/57947768 shutdown_connections 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.576+0000 7f2aa8de9640 1 -- 172.21.15.26:0/57947768 wait complete. 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 Processor -- start 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 -- start start 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c5330 0x7f2aa40c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa40c88b0 0x7f2aa40c05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa41ee830 0x7f2aa41eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41529d0 con 0x7f2aa40c88b0 2024-03-06T07:20:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa406d090 con 0x7f2aa41ee830 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa8de9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa4152b50 con 0x7f2aa40c5330 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa40c88b0 0x7f2aa40c05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa41ee830 0x7f2aa41eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c5330 0x7f2aa40c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa40c88b0 0x7f2aa40c05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46686/0 (socket says 172.21.15.26:46686) 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.578+0000 7f2aa1d9b640 1 -- 172.21.15.26:0/3734031827 learned_addr learned my addr 172.21.15.26:0/3734031827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa259c640 1 -- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa41ee830 msgr2=0x7f2aa41eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa259c640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa41ee830 0x7f2aa41eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa259c640 1 -- 172.21.15.26:0/3734031827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa40c88b0 msgr2=0x7f2aa40c05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa259c640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa40c88b0 0x7f2aa40c05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa259c640 1 -- 172.21.15.26:0/3734031827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4157100 con 0x7f2aa40c5330 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa1d9b640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa40c88b0 0x7f2aa40c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa2d9d640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa41ee830 0x7f2aa41eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa259c640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c5330 0x7f2aa40c0090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c7e0 tx=0x7f2a9c00cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2a9affd640 1 -- 172.21.15.26:0/3734031827 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c018070 con 0x7f2aa40c5330 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2a9affd640 1 -- 172.21.15.26:0/3734031827 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0070d0 con 0x7f2aa40c5330 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2a9affd640 1 -- 172.21.15.26:0/3734031827 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0074c0 con 0x7f2aa40c5330 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa41573e0 con 0x7f2aa40c5330 2024-03-06T07:20:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.579+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa41579a0 con 0x7f2aa40c5330 2024-03-06T07:20:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.580+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa4068f40 con 0x7f2aa40c5330 2024-03-06T07:20:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.581+0000 7f2a9affd640 1 -- 172.21.15.26:0/3734031827 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a9c0047e0 con 0x7f2aa40c5330 2024-03-06T07:20:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.581+0000 7f2a9affd640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a8406b500 0x7f2a8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.581+0000 7f2a9affd640 1 -- 172.21.15.26:0/3734031827 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2a9c0894e0 con 0x7f2aa40c5330 2024-03-06T07:20:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.581+0000 7f2aa1d9b640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a8406b500 0x7f2a8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.582+0000 7f2aa1d9b640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a8406b500 0x7f2a8406d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0097a0 tx=0x7f2a8c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.585+0000 7f2a9affd640 1 -- 172.21.15.26:0/3734031827 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2aa4068f40 con 0x7f2aa40c5330 2024-03-06T07:20:18.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.767+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2aa404f330 con 0x7f2aa40c5330 2024-03-06T07:20:18.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.768+0000 7f2a9affd640 1 -- 172.21.15.26:0/3734031827 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f2aa4155110 con 0x7f2aa40c5330 2024-03-06T07:20:18.770 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:18.770 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a8406b500 msgr2=0x7f2a8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a8406b500 0x7f2a8406d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0097a0 tx=0x7f2a8c03d040 comp rx=0 tx=0).stop 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c5330 msgr2=0x7f2aa40c0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c5330 0x7f2aa40c0090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c7e0 tx=0x7f2a9c00cbe0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 shutdown_connections 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a8406b500 0x7f2a8406d9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aa41ee830 0x7f2aa41eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa40c88b0 0x7f2aa40c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 --2- 172.21.15.26:0/3734031827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aa40c5330 0x7f2aa40c0090 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 >> 172.21.15.26:0/3734031827 conn(0x7f2aa40ac560 msgr2=0x7f2aa414f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 shutdown_connections 2024-03-06T07:20:18.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.770+0000 7f2aa8de9640 1 -- 172.21.15.26:0/3734031827 wait complete. 2024-03-06T07:20:18.782 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:20:18.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.882+0000 7f89ba78a640 1 Processor -- start 2024-03-06T07:20:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.882+0000 7f89ba78a640 1 -- start start 2024-03-06T07:20:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89ba78a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89b40c5480 0x7f89b4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89ba78a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89ba78a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89b41586f0 0x7f89b4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89ba78a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b406df40 con 0x7f89b40c5480 2024-03-06T07:20:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89ba78a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40c5cc0 con 0x7f89b41586f0 2024-03-06T07:20:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89ba78a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41591c0 con 0x7f89b40c5860 2024-03-06T07:20:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89b3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89b40c5480 0x7f89b4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89b3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89b40c5480 0x7f89b4155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46700/0 (socket says 172.21.15.26:46700) 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89b3fff640 1 -- 172.21.15.26:0/421402315 learned_addr learned my addr 172.21.15.26:0/421402315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89b8d27640 1 --2- 172.21.15.26:0/421402315 >> v2:172.21.15.117:3300/0 conn(0x7f89b41586f0 0x7f89b4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.883+0000 7f89b37fe640 1 --2- 172.21.15.26:0/421402315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b3fff640 1 -- 172.21.15.26:0/421402315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 msgr2=0x7f89b406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b3fff640 1 --2- 172.21.15.26:0/421402315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b3fff640 1 -- 172.21.15.26:0/421402315 >> v2:172.21.15.117:3300/0 conn(0x7f89b41586f0 msgr2=0x7f89b4158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b3fff640 1 --2- 172.21.15.26:0/421402315 >> v2:172.21.15.117:3300/0 conn(0x7f89b41586f0 0x7f89b4158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b3fff640 1 -- 172.21.15.26:0/421402315 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b41594a0 con 0x7f89b40c5480 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b8d27640 1 --2- 172.21.15.26:0/421402315 >> v2:172.21.15.117:3300/0 conn(0x7f89b41586f0 0x7f89b4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b3fff640 1 --2- 172.21.15.26:0/421402315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b40c5480 0x7f89b4155ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89b4071060 tx=0x7f89a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee00d43238d6b2b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b2ffd640 1 -- 172.21.15.26:0/421402315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a402e070 con 0x7f89b40c5480 2024-03-06T07:20:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b2ffd640 1 -- 172.21.15.26:0/421402315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4037070 con 0x7f89b40c5480 2024-03-06T07:20:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.884+0000 7f89b2ffd640 1 -- 172.21.15.26:0/421402315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4031a00 con 0x7f89b40c5480 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 -- 172.21.15.26:0/421402315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b40c5480 msgr2=0x7f89b4155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 --2- 172.21.15.26:0/421402315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b40c5480 0x7f89b4155ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89b4071060 tx=0x7f89a402b880 comp rx=0 tx=0).stop 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 -- 172.21.15.26:0/421402315 shutdown_connections 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 --2- 172.21.15.26:0/421402315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 --2- 172.21.15.26:0/421402315 >> v2:172.21.15.117:3300/0 conn(0x7f89b41586f0 0x7f89b4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 --2- 172.21.15.26:0/421402315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b40c5480 0x7f89b4155ba0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 -- 172.21.15.26:0/421402315 >> 172.21.15.26:0/421402315 conn(0x7f89b406b1a0 msgr2=0x7f89b406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 -- 172.21.15.26:0/421402315 shutdown_connections 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.885+0000 7f89ba78a640 1 -- 172.21.15.26:0/421402315 wait complete. 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.886+0000 7f89ba78a640 1 Processor -- start 2024-03-06T07:20:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.886+0000 7f89ba78a640 1 -- start start 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89ba78a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89b40c5480 0x7f89b414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89ba78a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89ba78a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b41586f0 0x7f89b4151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89ba78a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4152a20 con 0x7f89b41586f0 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89ba78a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4155680 con 0x7f89b40c5480 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89ba78a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4155050 con 0x7f89b40c5860 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b41586f0 0x7f89b4151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b41586f0 0x7f89b4151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46706/0 (socket says 172.21.15.26:46706) 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 -- 172.21.15.26:0/292290233 learned_addr learned my addr 172.21.15.26:0/292290233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b37fe640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b3fff640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89b40c5480 0x7f89b414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 -- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 msgr2=0x7f89b414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b414ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 -- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89b40c5480 msgr2=0x7f89b414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89b40c5480 0x7f89b414b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.887+0000 7f89b8d27640 1 -- 172.21.15.26:0/292290233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b41ef0d0 con 0x7f89b41586f0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.888+0000 7f89b8d27640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b41586f0 0x7f89b4151100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00efc0 tx=0x7f89ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.888+0000 7f89b0ff9640 1 -- 172.21.15.26:0/292290233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac018030 con 0x7f89b41586f0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.888+0000 7f89b0ff9640 1 -- 172.21.15.26:0/292290233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac00cd30 con 0x7f89b41586f0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.888+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b40ad760 con 0x7f89b41586f0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.888+0000 7f89b0ff9640 1 -- 172.21.15.26:0/292290233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac003bb0 con 0x7f89b41586f0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.888+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b41eed60 con 0x7f89b41586f0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.889+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b404f330 con 0x7f89b41586f0 2024-03-06T07:20:18.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.889+0000 7f89b0ff9640 1 -- 172.21.15.26:0/292290233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89ac005d10 con 0x7f89b41586f0 2024-03-06T07:20:18.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.890+0000 7f89b0ff9640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89a006b4d0 0x7f89a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.890+0000 7f89b0ff9640 1 -- 172.21.15.26:0/292290233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f89ac08ac50 con 0x7f89b41586f0 2024-03-06T07:20:18.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.890+0000 7f89b3fff640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89a006b4d0 0x7f89a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.890+0000 7f89b3fff640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89a006b4d0 0x7f89a006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f89a4036000 tx=0x7f89a403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:18.894+0000 7f89b0ff9640 1 -- 172.21.15.26:0/292290233 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f89ac090020 con 0x7f89b41586f0 2024-03-06T07:20:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.069+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89b406e0c0 con 0x7f89a006b4d0 2024-03-06T07:20:19.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.078+0000 7f89b0ff9640 1 -- 172.21.15.26:0/292290233 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332635 (secure 0 0 0) 0x7f89b406e0c0 con 0x7f89a006b4d0 2024-03-06T07:20:19.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:19.086 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.081+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89a006b4d0 msgr2=0x7f89a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.081+0000 7f89ba78a640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89a006b4d0 0x7f89a006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f89a4036000 tx=0x7f89a403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.081+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b41586f0 msgr2=0x7f89b4151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.081+0000 7f89ba78a640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b41586f0 0x7f89b4151100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00efc0 tx=0x7f89ac00c490 comp rx=0 tx=0).stop 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 shutdown_connections 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89a006b4d0 0x7f89a006d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89b41586f0 0x7f89b4151100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89b40c5860 0x7f89b414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 --2- 172.21.15.26:0/292290233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89b40c5480 0x7f89b414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 >> 172.21.15.26:0/292290233 conn(0x7f89b406b1a0 msgr2=0x7f89b4152ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 shutdown_connections 2024-03-06T07:20:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.082+0000 7f89ba78a640 1 -- 172.21.15.26:0/292290233 wait complete. 2024-03-06T07:20:19.094 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-03-06T07:20:19.033981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230820,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743900,"statfs":{"total":773094113280,"available":772857753600,"internally_reserved":0,"allocated":2228224,"data_stored":1991782,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002684"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241454+0000","last_change":"2024-03-06T07:20:00.821957+0000","last_active":"2024-03-06T07:20:07.241454+0000","last_peered":"2024-03-06T07:20:07.241454+0000","last_clean":"2024-03-06T07:20:07.241454+0000","last_became_active":"2024-03-06T07:20:00.821672+0000","last_became_peered":"2024-03-06T07:20:00.821672+0000","last_unstale":"2024-03-06T07:20:07.241454+0000","last_undegraded":"2024-03-06T07:20:07.241454+0000","last_fullsized":"2024-03-06T07:20:07.241454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242911+0000","last_change":"2024-03-06T07:20:01.819400+0000","last_active":"2024-03-06T07:20:07.242911+0000","last_peered":"2024-03-06T07:20:07.242911+0000","last_clean":"2024-03-06T07:20:07.242911+0000","last_became_active":"2024-03-06T07:20:01.819073+0000","last_became_peered":"2024-03-06T07:20:01.819073+0000","last_unstale":"2024-03-06T07:20:07.242911+0000","last_undegraded":"2024-03-06T07:20:07.242911+0000","last_fullsized":"2024-03-06T07:20:07.242911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242373+0000","last_change":"2024-03-06T07:20:06.127375+0000","last_active":"2024-03-06T07:20:07.242373+0000","last_peered":"2024-03-06T07:20:07.242373+0000","last_clean":"2024-03-06T07:20:07.242373+0000","last_became_active":"2024-03-06T07:20:06.126687+0000","last_became_peered":"2024-03-06T07:20:06.126687+0000","last_unstale":"2024-03-06T07:20:07.242373+0000","last_undegraded":"2024-03-06T07:20:07.242373+0000","last_fullsized":"2024-03-06T07:20:07.242373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:11.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243026+0000","last_change":"2024-03-06T07:20:00.816186+0000","last_active":"2024-03-06T07:20:07.243026+0000","last_peered":"2024-03-06T07:20:07.243026+0000","last_clean":"2024-03-06T07:20:07.243026+0000","last_became_active":"2024-03-06T07:20:00.815110+0000","last_became_peered":"2024-03-06T07:20:00.815110+0000","last_unstale":"2024-03-06T07:20:07.243026+0000","last_undegraded":"2024-03-06T07:20:07.243026+0000","last_fullsized":"2024-03-06T07:20:07.243026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241124+0000","last_change":"2024-03-06T07:20:01.835360+0000","last_active":"2024-03-06T07:20:07.241124+0000","last_peered":"2024-03-06T07:20:07.241124+0000","last_clean":"2024-03-06T07:20:07.241124+0000","last_became_active":"2024-03-06T07:20:01.835066+0000","last_became_peered":"2024-03-06T07:20:01.835066+0000","last_unstale":"2024-03-06T07:20:07.241124+0000","last_undegraded":"2024-03-06T07:20:07.241124+0000","last_fullsized":"2024-03-06T07:20:07.241124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240859+0000","last_change":"2024-03-06T07:20:05.864904+0000","last_active":"2024-03-06T07:20:07.240859+0000","last_peered":"2024-03-06T07:20:07.240859+0000","last_clean":"2024-03-06T07:20:07.240859+0000","last_became_active":"2024-03-06T07:20:05.864535+0000","last_became_peered":"2024-03-06T07:20:05.864535+0000","last_unstale":"2024-03-06T07:20:07.240859+0000","last_undegraded":"2024-03-06T07:20:07.240859+0000","last_fullsized":"2024-03-06T07:20:07.240859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.016032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242211+0000","last_change":"2024-03-06T07:20:00.823477+0000","last_active":"2024-03-06T07:20:07.242211+0000","last_peered":"2024-03-06T07:20:07.242211+0000","last_clean":"2024-03-06T07:20:07.242211+0000","last_became_active":"2024-03-06T07:20:00.822898+0000","last_became_peered":"2024-03-06T07:20:00.822898+0000","last_unstale":"2024-03-06T07:20:07.242211+0000","last_undegraded":"2024-03-06T07:20:07.242211+0000","last_fullsized":"2024-03-06T07:20:07.242211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:23.183380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.832744+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.832237+0000","last_became_peered":"2024-03-06T07:20:01.832237+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241395+0000","last_change":"2024-03-06T07:20:05.862022+0000","last_active":"2024-03-06T07:20:07.241395+0000","last_peered":"2024-03-06T07:20:07.241395+0000","last_clean":"2024-03-06T07:20:07.241395+0000","last_became_active":"2024-03-06T07:20:05.861661+0000","last_became_peered":"2024-03-06T07:20:05.861661+0000","last_unstale":"2024-03-06T07:20:07.241395+0000","last_undegraded":"2024-03-06T07:20:07.241395+0000","last_fullsized":"2024-03-06T07:20:07.241395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:30.602011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241314+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241314+0000","last_peered":"2024-03-06T07:20:07.241314+0000","last_clean":"2024-03-06T07:20:07.241314+0000","last_became_active":"2024-03-06T07:20:00.822334+0000","last_became_peered":"2024-03-06T07:20:00.822334+0000","last_unstale":"2024-03-06T07:20:07.241314+0000","last_undegraded":"2024-03-06T07:20:07.241314+0000","last_fullsized":"2024-03-06T07:20:07.241314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241506+0000","last_change":"2024-03-06T07:20:01.832671+0000","last_active":"2024-03-06T07:20:07.241506+0000","last_peered":"2024-03-06T07:20:07.241506+0000","last_clean":"2024-03-06T07:20:07.241506+0000","last_became_active":"2024-03-06T07:20:01.832397+0000","last_became_peered":"2024-03-06T07:20:01.832397+0000","last_unstale":"2024-03-06T07:20:07.241506+0000","last_undegraded":"2024-03-06T07:20:07.241506+0000","last_fullsized":"2024-03-06T07:20:07.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241835+0000","last_change":"2024-03-06T07:20:05.853058+0000","last_active":"2024-03-06T07:20:07.241835+0000","last_peered":"2024-03-06T07:20:07.241835+0000","last_clean":"2024-03-06T07:20:07.241835+0000","last_became_active":"2024-03-06T07:20:05.852648+0000","last_became_peered":"2024-03-06T07:20:05.852648+0000","last_unstale":"2024-03-06T07:20:07.241835+0000","last_undegraded":"2024-03-06T07:20:07.241835+0000","last_fullsized":"2024-03-06T07:20:07.241835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:33.136033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838904+0000","last_change":"2024-03-06T07:20:01.817989+0000","last_active":"2024-03-06T07:20:06.838904+0000","last_peered":"2024-03-06T07:20:06.838904+0000","last_clean":"2024-03-06T07:20:06.838904+0000","last_became_active":"2024-03-06T07:20:01.817711+0000","last_became_peered":"2024-03-06T07:20:01.817711+0000","last_unstale":"2024-03-06T07:20:06.838904+0000","last_undegraded":"2024-03-06T07:20:06.838904+0000","last_fullsized":"2024-03-06T07:20:06.838904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241768+0000","last_change":"2024-03-06T07:20:00.822403+0000","last_active":"2024-03-06T07:20:07.241768+0000","last_peered":"2024-03-06T07:20:07.241768+0000","last_clean":"2024-03-06T07:20:07.241768+0000","last_became_active":"2024-03-06T07:20:00.822144+0000","last_became_peered":"2024-03-06T07:20:00.822144+0000","last_unstale":"2024-03-06T07:20:07.241768+0000","last_undegraded":"2024-03-06T07:20:07.241768+0000","last_fullsized":"2024-03-06T07:20:07.241768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241387+0000","last_change":"2024-03-06T07:20:05.864961+0000","last_active":"2024-03-06T07:20:07.241387+0000","last_peered":"2024-03-06T07:20:07.241387+0000","last_clean":"2024-03-06T07:20:07.241387+0000","last_became_active":"2024-03-06T07:20:05.864626+0000","last_became_peered":"2024-03-06T07:20:05.864626+0000","last_unstale":"2024-03-06T07:20:07.241387+0000","last_undegraded":"2024-03-06T07:20:07.241387+0000","last_fullsized":"2024-03-06T07:20:07.241387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:12.900271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243119+0000","last_change":"2024-03-06T07:20:01.830511+0000","last_active":"2024-03-06T07:20:07.243119+0000","last_peered":"2024-03-06T07:20:07.243119+0000","last_clean":"2024-03-06T07:20:07.243119+0000","last_became_active":"2024-03-06T07:20:01.830230+0000","last_became_peered":"2024-03-06T07:20:01.830230+0000","last_unstale":"2024-03-06T07:20:07.243119+0000","last_undegraded":"2024-03-06T07:20:07.243119+0000","last_fullsized":"2024-03-06T07:20:07.243119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242095+0000","last_change":"2024-03-06T07:20:00.816279+0000","last_active":"2024-03-06T07:20:07.242095+0000","last_peered":"2024-03-06T07:20:07.242095+0000","last_clean":"2024-03-06T07:20:07.242095+0000","last_became_active":"2024-03-06T07:20:00.815979+0000","last_became_peered":"2024-03-06T07:20:00.815979+0000","last_unstale":"2024-03-06T07:20:07.242095+0000","last_undegraded":"2024-03-06T07:20:07.242095+0000","last_fullsized":"2024-03-06T07:20:07.242095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240897+0000","last_change":"2024-03-06T07:20:05.864338+0000","last_active":"2024-03-06T07:20:07.240897+0000","last_peered":"2024-03-06T07:20:07.240897+0000","last_clean":"2024-03-06T07:20:07.240897+0000","last_became_active":"2024-03-06T07:20:05.863893+0000","last_became_peered":"2024-03-06T07:20:05.863893+0000","last_unstale":"2024-03-06T07:20:07.240897+0000","last_undegraded":"2024-03-06T07:20:07.240897+0000","last_fullsized":"2024-03-06T07:20:07.240897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:55.097092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242166+0000","last_change":"2024-03-06T07:20:01.813572+0000","last_active":"2024-03-06T07:20:07.242166+0000","last_peered":"2024-03-06T07:20:07.242166+0000","last_clean":"2024-03-06T07:20:07.242166+0000","last_became_active":"2024-03-06T07:20:01.813285+0000","last_became_peered":"2024-03-06T07:20:01.813285+0000","last_unstale":"2024-03-06T07:20:07.242166+0000","last_undegraded":"2024-03-06T07:20:07.242166+0000","last_fullsized":"2024-03-06T07:20:07.242166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:00.818384+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:00.818086+0000","last_became_peered":"2024-03-06T07:20:00.818086+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242538+0000","last_change":"2024-03-06T07:20:06.124240+0000","last_active":"2024-03-06T07:20:07.242538+0000","last_peered":"2024-03-06T07:20:07.242538+0000","last_clean":"2024-03-06T07:20:07.242538+0000","last_became_active":"2024-03-06T07:20:06.123859+0000","last_became_peered":"2024-03-06T07:20:06.123859+0000","last_unstale":"2024-03-06T07:20:07.242538+0000","last_undegraded":"2024-03-06T07:20:07.242538+0000","last_fullsized":"2024-03-06T07:20:07.242538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241784+0000","last_change":"2024-03-06T07:20:01.812138+0000","last_active":"2024-03-06T07:20:07.241784+0000","last_peered":"2024-03-06T07:20:07.241784+0000","last_clean":"2024-03-06T07:20:07.241784+0000","last_became_active":"2024-03-06T07:20:01.811851+0000","last_became_peered":"2024-03-06T07:20:01.811851+0000","last_unstale":"2024-03-06T07:20:07.241784+0000","last_undegraded":"2024-03-06T07:20:07.241784+0000","last_fullsized":"2024-03-06T07:20:07.241784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:22.590118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241483+0000","last_change":"2024-03-06T07:20:00.821941+0000","last_active":"2024-03-06T07:20:07.241483+0000","last_peered":"2024-03-06T07:20:07.241483+0000","last_clean":"2024-03-06T07:20:07.241483+0000","last_became_active":"2024-03-06T07:20:00.821533+0000","last_became_peered":"2024-03-06T07:20:00.821533+0000","last_unstale":"2024-03-06T07:20:07.241483+0000","last_undegraded":"2024-03-06T07:20:07.241483+0000","last_fullsized":"2024-03-06T07:20:07.241483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241052+0000","last_change":"2024-03-06T07:20:05.864042+0000","last_active":"2024-03-06T07:20:07.241052+0000","last_peered":"2024-03-06T07:20:07.241052+0000","last_clean":"2024-03-06T07:20:07.241052+0000","last_became_active":"2024-03-06T07:20:05.863616+0000","last_became_peered":"2024-03-06T07:20:05.863616+0000","last_unstale":"2024-03-06T07:20:07.241052+0000","last_undegraded":"2024-03-06T07:20:07.241052+0000","last_fullsized":"2024-03-06T07:20:07.241052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:12.894304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241818+0000","last_change":"2024-03-06T07:20:01.821307+0000","last_active":"2024-03-06T07:20:07.241818+0000","last_peered":"2024-03-06T07:20:07.241818+0000","last_clean":"2024-03-06T07:20:07.241818+0000","last_became_active":"2024-03-06T07:20:01.821010+0000","last_became_peered":"2024-03-06T07:20:01.821010+0000","last_unstale":"2024-03-06T07:20:07.241818+0000","last_undegraded":"2024-03-06T07:20:07.241818+0000","last_fullsized":"2024-03-06T07:20:07.241818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839526+0000","last_change":"2024-03-06T07:20:00.824564+0000","last_active":"2024-03-06T07:20:06.839526+0000","last_peered":"2024-03-06T07:20:06.839526+0000","last_clean":"2024-03-06T07:20:06.839526+0000","last_became_active":"2024-03-06T07:20:00.824292+0000","last_became_peered":"2024-03-06T07:20:00.824292+0000","last_unstale":"2024-03-06T07:20:06.839526+0000","last_undegraded":"2024-03-06T07:20:06.839526+0000","last_fullsized":"2024-03-06T07:20:06.839526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242082+0000","last_change":"2024-03-06T07:20:06.124710+0000","last_active":"2024-03-06T07:20:07.242082+0000","last_peered":"2024-03-06T07:20:07.242082+0000","last_clean":"2024-03-06T07:20:07.242082+0000","last_became_active":"2024-03-06T07:20:06.124119+0000","last_became_peered":"2024-03-06T07:20:06.124119+0000","last_unstale":"2024-03-06T07:20:07.242082+0000","last_undegraded":"2024-03-06T07:20:07.242082+0000","last_fullsized":"2024-03-06T07:20:07.242082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243393+0000","last_change":"2024-03-06T07:20:01.816965+0000","last_active":"2024-03-06T07:20:07.243393+0000","last_peered":"2024-03-06T07:20:07.243393+0000","last_clean":"2024-03-06T07:20:07.243393+0000","last_became_active":"2024-03-06T07:20:01.816482+0000","last_became_peered":"2024-03-06T07:20:01.816482+0000","last_unstale":"2024-03-06T07:20:07.243393+0000","last_undegraded":"2024-03-06T07:20:07.243393+0000","last_fullsized":"2024-03-06T07:20:07.243393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242030+0000","last_change":"2024-03-06T07:20:00.814404+0000","last_active":"2024-03-06T07:20:07.242030+0000","last_peered":"2024-03-06T07:20:07.242030+0000","last_clean":"2024-03-06T07:20:07.242030+0000","last_became_active":"2024-03-06T07:20:00.813542+0000","last_became_peered":"2024-03-06T07:20:00.813542+0000","last_unstale":"2024-03-06T07:20:07.242030+0000","last_undegraded":"2024-03-06T07:20:07.242030+0000","last_fullsized":"2024-03-06T07:20:07.242030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241898+0000","last_change":"2024-03-06T07:20:06.129464+0000","last_active":"2024-03-06T07:20:07.241898+0000","last_peered":"2024-03-06T07:20:07.241898+0000","last_clean":"2024-03-06T07:20:07.241898+0000","last_became_active":"2024-03-06T07:20:06.128688+0000","last_became_peered":"2024-03-06T07:20:06.128688+0000","last_unstale":"2024-03-06T07:20:07.241898+0000","last_undegraded":"2024-03-06T07:20:07.241898+0000","last_fullsized":"2024-03-06T07:20:07.241898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242308+0000","last_change":"2024-03-06T07:20:01.814773+0000","last_active":"2024-03-06T07:20:07.242308+0000","last_peered":"2024-03-06T07:20:07.242308+0000","last_clean":"2024-03-06T07:20:07.242308+0000","last_became_active":"2024-03-06T07:20:01.814497+0000","last_became_peered":"2024-03-06T07:20:01.814497+0000","last_unstale":"2024-03-06T07:20:07.242308+0000","last_undegraded":"2024-03-06T07:20:07.242308+0000","last_fullsized":"2024-03-06T07:20:07.242308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242368+0000","last_change":"2024-03-06T07:20:00.821645+0000","last_active":"2024-03-06T07:20:07.242368+0000","last_peered":"2024-03-06T07:20:07.242368+0000","last_clean":"2024-03-06T07:20:07.242368+0000","last_became_active":"2024-03-06T07:20:00.821353+0000","last_became_peered":"2024-03-06T07:20:00.821353+0000","last_unstale":"2024-03-06T07:20:07.242368+0000","last_undegraded":"2024-03-06T07:20:07.242368+0000","last_fullsized":"2024-03-06T07:20:07.242368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:02:35.568408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241505+0000","last_change":"2024-03-06T07:20:06.125251+0000","last_active":"2024-03-06T07:20:07.241505+0000","last_peered":"2024-03-06T07:20:07.241505+0000","last_clean":"2024-03-06T07:20:07.241505+0000","last_became_active":"2024-03-06T07:20:06.124788+0000","last_became_peered":"2024-03-06T07:20:06.124788+0000","last_unstale":"2024-03-06T07:20:07.241505+0000","last_undegraded":"2024-03-06T07:20:07.241505+0000","last_fullsized":"2024-03-06T07:20:07.241505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241587+0000","last_change":"2024-03-06T07:20:01.831321+0000","last_active":"2024-03-06T07:20:07.241587+0000","last_peered":"2024-03-06T07:20:07.241587+0000","last_clean":"2024-03-06T07:20:07.241587+0000","last_became_active":"2024-03-06T07:20:01.831041+0000","last_became_peered":"2024-03-06T07:20:01.831041+0000","last_unstale":"2024-03-06T07:20:07.241587+0000","last_undegraded":"2024-03-06T07:20:07.241587+0000","last_fullsized":"2024-03-06T07:20:07.241587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242408+0000","last_change":"2024-03-06T07:20:00.812059+0000","last_active":"2024-03-06T07:20:07.242408+0000","last_peered":"2024-03-06T07:20:07.242408+0000","last_clean":"2024-03-06T07:20:07.242408+0000","last_became_active":"2024-03-06T07:20:00.811207+0000","last_became_peered":"2024-03-06T07:20:00.811207+0000","last_unstale":"2024-03-06T07:20:07.242408+0000","last_undegraded":"2024-03-06T07:20:07.242408+0000","last_fullsized":"2024-03-06T07:20:07.242408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241904+0000","last_change":"2024-03-06T07:20:05.859872+0000","last_active":"2024-03-06T07:20:07.241904+0000","last_peered":"2024-03-06T07:20:07.241904+0000","last_clean":"2024-03-06T07:20:07.241904+0000","last_became_active":"2024-03-06T07:20:05.857727+0000","last_became_peered":"2024-03-06T07:20:05.857727+0000","last_unstale":"2024-03-06T07:20:07.241904+0000","last_undegraded":"2024-03-06T07:20:07.241904+0000","last_fullsized":"2024-03-06T07:20:07.241904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241527+0000","last_change":"2024-03-06T07:20:01.829871+0000","last_active":"2024-03-06T07:20:07.241527+0000","last_peered":"2024-03-06T07:20:07.241527+0000","last_clean":"2024-03-06T07:20:07.241527+0000","last_became_active":"2024-03-06T07:20:01.829526+0000","last_became_peered":"2024-03-06T07:20:01.829526+0000","last_unstale":"2024-03-06T07:20:07.241527+0000","last_undegraded":"2024-03-06T07:20:07.241527+0000","last_fullsized":"2024-03-06T07:20:07.241527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:23.704222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242548+0000","last_change":"2024-03-06T07:20:00.818847+0000","last_active":"2024-03-06T07:20:07.242548+0000","last_peered":"2024-03-06T07:20:07.242548+0000","last_clean":"2024-03-06T07:20:07.242548+0000","last_became_active":"2024-03-06T07:20:00.818547+0000","last_became_peered":"2024-03-06T07:20:00.818547+0000","last_unstale":"2024-03-06T07:20:07.242548+0000","last_undegraded":"2024-03-06T07:20:07.242548+0000","last_fullsized":"2024-03-06T07:20:07.242548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242474+0000","last_change":"2024-03-06T07:20:05.861172+0000","last_active":"2024-03-06T07:20:07.242474+0000","last_peered":"2024-03-06T07:20:07.242474+0000","last_clean":"2024-03-06T07:20:07.242474+0000","last_became_active":"2024-03-06T07:20:05.859989+0000","last_became_peered":"2024-03-06T07:20:05.859989+0000","last_unstale":"2024-03-06T07:20:07.242474+0000","last_undegraded":"2024-03-06T07:20:07.242474+0000","last_fullsized":"2024-03-06T07:20:07.242474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241692+0000","last_change":"2024-03-06T07:20:01.813503+0000","last_active":"2024-03-06T07:20:07.241692+0000","last_peered":"2024-03-06T07:20:07.241692+0000","last_clean":"2024-03-06T07:20:07.241692+0000","last_became_active":"2024-03-06T07:20:01.813227+0000","last_became_peered":"2024-03-06T07:20:01.813227+0000","last_unstale":"2024-03-06T07:20:07.241692+0000","last_undegraded":"2024-03-06T07:20:07.241692+0000","last_fullsized":"2024-03-06T07:20:07.241692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839223+0000","last_change":"2024-03-06T07:20:00.812787+0000","last_active":"2024-03-06T07:20:06.839223+0000","last_peered":"2024-03-06T07:20:06.839223+0000","last_clean":"2024-03-06T07:20:06.839223+0000","last_became_active":"2024-03-06T07:20:00.812505+0000","last_became_peered":"2024-03-06T07:20:00.812505+0000","last_unstale":"2024-03-06T07:20:06.839223+0000","last_undegraded":"2024-03-06T07:20:06.839223+0000","last_fullsized":"2024-03-06T07:20:06.839223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242555+0000","last_change":"2024-03-06T07:20:05.859379+0000","last_active":"2024-03-06T07:20:07.242555+0000","last_peered":"2024-03-06T07:20:07.242555+0000","last_clean":"2024-03-06T07:20:07.242555+0000","last_became_active":"2024-03-06T07:20:05.858723+0000","last_became_peered":"2024-03-06T07:20:05.858723+0000","last_unstale":"2024-03-06T07:20:07.242555+0000","last_undegraded":"2024-03-06T07:20:07.242555+0000","last_fullsized":"2024-03-06T07:20:07.242555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240883+0000","last_change":"2024-03-06T07:20:01.826415+0000","last_active":"2024-03-06T07:20:07.240883+0000","last_peered":"2024-03-06T07:20:07.240883+0000","last_clean":"2024-03-06T07:20:07.240883+0000","last_became_active":"2024-03-06T07:20:01.825703+0000","last_became_peered":"2024-03-06T07:20:01.825703+0000","last_unstale":"2024-03-06T07:20:07.240883+0000","last_undegraded":"2024-03-06T07:20:07.240883+0000","last_fullsized":"2024-03-06T07:20:07.240883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:47.116335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241563+0000","last_change":"2024-03-06T07:20:00.822080+0000","last_active":"2024-03-06T07:20:07.241563+0000","last_peered":"2024-03-06T07:20:07.241563+0000","last_clean":"2024-03-06T07:20:07.241563+0000","last_became_active":"2024-03-06T07:20:00.821813+0000","last_became_peered":"2024-03-06T07:20:00.821813+0000","last_unstale":"2024-03-06T07:20:07.241563+0000","last_undegraded":"2024-03-06T07:20:07.241563+0000","last_fullsized":"2024-03-06T07:20:07.241563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241612+0000","last_change":"2024-03-06T07:20:06.124233+0000","last_active":"2024-03-06T07:20:07.241612+0000","last_peered":"2024-03-06T07:20:07.241612+0000","last_clean":"2024-03-06T07:20:07.241612+0000","last_became_active":"2024-03-06T07:20:06.123834+0000","last_became_peered":"2024-03-06T07:20:06.123834+0000","last_unstale":"2024-03-06T07:20:07.241612+0000","last_undegraded":"2024-03-06T07:20:07.241612+0000","last_fullsized":"2024-03-06T07:20:07.241612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242846+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.242846+0000","last_peered":"2024-03-06T07:20:07.242846+0000","last_clean":"2024-03-06T07:20:07.242846+0000","last_became_active":"2024-03-06T07:20:01.830667+0000","last_became_peered":"2024-03-06T07:20:01.830667+0000","last_unstale":"2024-03-06T07:20:07.242846+0000","last_undegraded":"2024-03-06T07:20:07.242846+0000","last_fullsized":"2024-03-06T07:20:07.242846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241469+0000","last_change":"2024-03-06T07:20:00.814783+0000","last_active":"2024-03-06T07:20:07.241469+0000","last_peered":"2024-03-06T07:20:07.241469+0000","last_clean":"2024-03-06T07:20:07.241469+0000","last_became_active":"2024-03-06T07:20:00.814384+0000","last_became_peered":"2024-03-06T07:20:00.814384+0000","last_unstale":"2024-03-06T07:20:07.241469+0000","last_undegraded":"2024-03-06T07:20:07.241469+0000","last_fullsized":"2024-03-06T07:20:07.241469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241641+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:07.241641+0000","last_peered":"2024-03-06T07:20:07.241641+0000","last_clean":"2024-03-06T07:20:07.241641+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:07.241641+0000","last_undegraded":"2024-03-06T07:20:07.241641+0000","last_fullsized":"2024-03-06T07:20:07.241641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839262+0000","last_change":"2024-03-06T07:20:06.126879+0000","last_active":"2024-03-06T07:20:06.839262+0000","last_peered":"2024-03-06T07:20:06.839262+0000","last_clean":"2024-03-06T07:20:06.839262+0000","last_became_active":"2024-03-06T07:20:06.126475+0000","last_became_peered":"2024-03-06T07:20:06.126475+0000","last_unstale":"2024-03-06T07:20:06.839262+0000","last_undegraded":"2024-03-06T07:20:06.839262+0000","last_fullsized":"2024-03-06T07:20:06.839262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242057+0000","last_change":"2024-03-06T07:20:01.831806+0000","last_active":"2024-03-06T07:20:07.242057+0000","last_peered":"2024-03-06T07:20:07.242057+0000","last_clean":"2024-03-06T07:20:07.242057+0000","last_became_active":"2024-03-06T07:20:01.831545+0000","last_became_peered":"2024-03-06T07:20:01.831545+0000","last_unstale":"2024-03-06T07:20:07.242057+0000","last_undegraded":"2024-03-06T07:20:07.242057+0000","last_fullsized":"2024-03-06T07:20:07.242057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839559+0000","last_change":"2024-03-06T07:20:00.817587+0000","last_active":"2024-03-06T07:20:06.839559+0000","last_peered":"2024-03-06T07:20:06.839559+0000","last_clean":"2024-03-06T07:20:06.839559+0000","last_became_active":"2024-03-06T07:20:00.817212+0000","last_became_peered":"2024-03-06T07:20:00.817212+0000","last_unstale":"2024-03-06T07:20:06.839559+0000","last_undegraded":"2024-03-06T07:20:06.839559+0000","last_fullsized":"2024-03-06T07:20:06.839559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838986+0000","last_change":"2024-03-06T07:20:06.126921+0000","last_active":"2024-03-06T07:20:06.838986+0000","last_peered":"2024-03-06T07:20:06.838986+0000","last_clean":"2024-03-06T07:20:06.838986+0000","last_became_active":"2024-03-06T07:20:06.126385+0000","last_became_peered":"2024-03-06T07:20:06.126385+0000","last_unstale":"2024-03-06T07:20:06.838986+0000","last_undegraded":"2024-03-06T07:20:06.838986+0000","last_fullsized":"2024-03-06T07:20:06.838986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242044+0000","last_change":"2024-03-06T07:20:01.831450+0000","last_active":"2024-03-06T07:20:07.242044+0000","last_peered":"2024-03-06T07:20:07.242044+0000","last_clean":"2024-03-06T07:20:07.242044+0000","last_became_active":"2024-03-06T07:20:01.830525+0000","last_became_peered":"2024-03-06T07:20:01.830525+0000","last_unstale":"2024-03-06T07:20:07.242044+0000","last_undegraded":"2024-03-06T07:20:07.242044+0000","last_fullsized":"2024-03-06T07:20:07.242044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242394+0000","last_change":"2024-03-06T07:20:00.807621+0000","last_active":"2024-03-06T07:20:07.242394+0000","last_peered":"2024-03-06T07:20:07.242394+0000","last_clean":"2024-03-06T07:20:07.242394+0000","last_became_active":"2024-03-06T07:20:00.807340+0000","last_became_peered":"2024-03-06T07:20:00.807340+0000","last_unstale":"2024-03-06T07:20:07.242394+0000","last_undegraded":"2024-03-06T07:20:07.242394+0000","last_fullsized":"2024-03-06T07:20:07.242394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241535+0000","last_change":"2024-03-06T07:20:05.857778+0000","last_active":"2024-03-06T07:20:07.241535+0000","last_peered":"2024-03-06T07:20:07.241535+0000","last_clean":"2024-03-06T07:20:07.241535+0000","last_became_active":"2024-03-06T07:20:05.857279+0000","last_became_peered":"2024-03-06T07:20:05.857279+0000","last_unstale":"2024-03-06T07:20:07.241535+0000","last_undegraded":"2024-03-06T07:20:07.241535+0000","last_fullsized":"2024-03-06T07:20:07.241535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242638+0000","last_change":"2024-03-06T07:20:01.819993+0000","last_active":"2024-03-06T07:20:07.242638+0000","last_peered":"2024-03-06T07:20:07.242638+0000","last_clean":"2024-03-06T07:20:07.242638+0000","last_became_active":"2024-03-06T07:20:01.819691+0000","last_became_peered":"2024-03-06T07:20:01.819691+0000","last_unstale":"2024-03-06T07:20:07.242638+0000","last_undegraded":"2024-03-06T07:20:07.242638+0000","last_fullsized":"2024-03-06T07:20:07.242638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:19.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839555+0000","last_change":"2024-03-06T07:20:00.819667+0000","last_active":"2024-03-06T07:20:06.839555+0000","last_peered":"2024-03-06T07:20:06.839555+0000","last_clean":"2024-03-06T07:20:06.839555+0000","last_became_active":"2024-03-06T07:20:00.819371+0000","last_became_peered":"2024-03-06T07:20:00.819371+0000","last_unstale":"2024-03-06T07:20:06.839555+0000","last_undegraded":"2024-03-06T07:20:06.839555+0000","last_fullsized":"2024-03-06T07:20:06.839555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241355+0000","last_change":"2024-03-06T07:20:06.124150+0000","last_active":"2024-03-06T07:20:07.241355+0000","last_peered":"2024-03-06T07:20:07.241355+0000","last_clean":"2024-03-06T07:20:07.241355+0000","last_became_active":"2024-03-06T07:20:06.123794+0000","last_became_peered":"2024-03-06T07:20:06.123794+0000","last_unstale":"2024-03-06T07:20:07.241355+0000","last_undegraded":"2024-03-06T07:20:07.241355+0000","last_fullsized":"2024-03-06T07:20:07.241355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242273+0000","last_change":"2024-03-06T07:20:01.814512+0000","last_active":"2024-03-06T07:20:07.242273+0000","last_peered":"2024-03-06T07:20:07.242273+0000","last_clean":"2024-03-06T07:20:07.242273+0000","last_became_active":"2024-03-06T07:20:01.813177+0000","last_became_peered":"2024-03-06T07:20:01.813177+0000","last_unstale":"2024-03-06T07:20:07.242273+0000","last_undegraded":"2024-03-06T07:20:07.242273+0000","last_fullsized":"2024-03-06T07:20:07.242273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:05.673072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241319+0000","last_change":"2024-03-06T07:20:00.812789+0000","last_active":"2024-03-06T07:20:07.241319+0000","last_peered":"2024-03-06T07:20:07.241319+0000","last_clean":"2024-03-06T07:20:07.241319+0000","last_became_active":"2024-03-06T07:20:00.812391+0000","last_became_peered":"2024-03-06T07:20:00.812391+0000","last_unstale":"2024-03-06T07:20:07.241319+0000","last_undegraded":"2024-03-06T07:20:07.241319+0000","last_fullsized":"2024-03-06T07:20:07.241319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:54:26.011476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240875+0000","last_change":"2024-03-06T07:20:05.863928+0000","last_active":"2024-03-06T07:20:07.240875+0000","last_peered":"2024-03-06T07:20:07.240875+0000","last_clean":"2024-03-06T07:20:07.240875+0000","last_became_active":"2024-03-06T07:20:05.863555+0000","last_became_peered":"2024-03-06T07:20:05.863555+0000","last_unstale":"2024-03-06T07:20:07.240875+0000","last_undegraded":"2024-03-06T07:20:07.240875+0000","last_fullsized":"2024-03-06T07:20:07.240875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:53.757250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242121+0000","last_change":"2024-03-06T07:20:01.811116+0000","last_active":"2024-03-06T07:20:07.242121+0000","last_peered":"2024-03-06T07:20:07.242121+0000","last_clean":"2024-03-06T07:20:07.242121+0000","last_became_active":"2024-03-06T07:20:01.810828+0000","last_became_peered":"2024-03-06T07:20:01.810828+0000","last_unstale":"2024-03-06T07:20:07.242121+0000","last_undegraded":"2024-03-06T07:20:07.242121+0000","last_fullsized":"2024-03-06T07:20:07.242121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838933+0000","last_change":"2024-03-06T07:20:00.807531+0000","last_active":"2024-03-06T07:20:06.838933+0000","last_peered":"2024-03-06T07:20:06.838933+0000","last_clean":"2024-03-06T07:20:06.838933+0000","last_became_active":"2024-03-06T07:20:00.807208+0000","last_became_peered":"2024-03-06T07:20:00.807208+0000","last_unstale":"2024-03-06T07:20:06.838933+0000","last_undegraded":"2024-03-06T07:20:06.838933+0000","last_fullsized":"2024-03-06T07:20:06.838933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:14.638569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:05.853364+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:05.852877+0000","last_became_peered":"2024-03-06T07:20:05.852877+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838808+0000","last_change":"2024-03-06T07:20:01.824711+0000","last_active":"2024-03-06T07:20:06.838808+0000","last_peered":"2024-03-06T07:20:06.838808+0000","last_clean":"2024-03-06T07:20:06.838808+0000","last_became_active":"2024-03-06T07:20:01.824420+0000","last_became_peered":"2024-03-06T07:20:01.824420+0000","last_unstale":"2024-03-06T07:20:06.838808+0000","last_undegraded":"2024-03-06T07:20:06.838808+0000","last_fullsized":"2024-03-06T07:20:06.838808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838852+0000","last_change":"2024-03-06T07:20:00.816624+0000","last_active":"2024-03-06T07:20:06.838852+0000","last_peered":"2024-03-06T07:20:06.838852+0000","last_clean":"2024-03-06T07:20:06.838852+0000","last_became_active":"2024-03-06T07:20:00.816335+0000","last_became_peered":"2024-03-06T07:20:00.816335+0000","last_unstale":"2024-03-06T07:20:06.838852+0000","last_undegraded":"2024-03-06T07:20:06.838852+0000","last_fullsized":"2024-03-06T07:20:06.838852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243303+0000","last_change":"2024-03-06T07:20:06.124178+0000","last_active":"2024-03-06T07:20:07.243303+0000","last_peered":"2024-03-06T07:20:07.243303+0000","last_clean":"2024-03-06T07:20:07.243303+0000","last_became_active":"2024-03-06T07:20:06.123788+0000","last_became_peered":"2024-03-06T07:20:06.123788+0000","last_unstale":"2024-03-06T07:20:07.243303+0000","last_undegraded":"2024-03-06T07:20:07.243303+0000","last_fullsized":"2024-03-06T07:20:07.243303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:12.104491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241688+0000","last_change":"2024-03-06T07:20:01.817183+0000","last_active":"2024-03-06T07:20:07.241688+0000","last_peered":"2024-03-06T07:20:07.241688+0000","last_clean":"2024-03-06T07:20:07.241688+0000","last_became_active":"2024-03-06T07:20:01.816230+0000","last_became_peered":"2024-03-06T07:20:01.816230+0000","last_unstale":"2024-03-06T07:20:07.241688+0000","last_undegraded":"2024-03-06T07:20:07.241688+0000","last_fullsized":"2024-03-06T07:20:07.241688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242486+0000","last_change":"2024-03-06T07:20:00.817367+0000","last_active":"2024-03-06T07:20:07.242486+0000","last_peered":"2024-03-06T07:20:07.242486+0000","last_clean":"2024-03-06T07:20:07.242486+0000","last_became_active":"2024-03-06T07:20:00.816410+0000","last_became_peered":"2024-03-06T07:20:00.816410+0000","last_unstale":"2024-03-06T07:20:07.242486+0000","last_undegraded":"2024-03-06T07:20:07.242486+0000","last_fullsized":"2024-03-06T07:20:07.242486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243226+0000","last_change":"2024-03-06T07:20:06.126465+0000","last_active":"2024-03-06T07:20:07.243226+0000","last_peered":"2024-03-06T07:20:07.243226+0000","last_clean":"2024-03-06T07:20:07.243226+0000","last_became_active":"2024-03-06T07:20:06.126107+0000","last_became_peered":"2024-03-06T07:20:06.126107+0000","last_unstale":"2024-03-06T07:20:07.243226+0000","last_undegraded":"2024-03-06T07:20:07.243226+0000","last_fullsized":"2024-03-06T07:20:07.243226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:37.677442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241274+0000","last_change":"2024-03-06T07:20:01.822827+0000","last_active":"2024-03-06T07:20:07.241274+0000","last_peered":"2024-03-06T07:20:07.241274+0000","last_clean":"2024-03-06T07:20:07.241274+0000","last_became_active":"2024-03-06T07:20:01.822490+0000","last_became_peered":"2024-03-06T07:20:01.822490+0000","last_unstale":"2024-03-06T07:20:07.241274+0000","last_undegraded":"2024-03-06T07:20:07.241274+0000","last_fullsized":"2024-03-06T07:20:07.241274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241706+0000","last_change":"2024-03-06T07:20:00.811943+0000","last_active":"2024-03-06T07:20:07.241706+0000","last_peered":"2024-03-06T07:20:07.241706+0000","last_clean":"2024-03-06T07:20:07.241706+0000","last_became_active":"2024-03-06T07:20:00.811499+0000","last_became_peered":"2024-03-06T07:20:00.811499+0000","last_unstale":"2024-03-06T07:20:07.241706+0000","last_undegraded":"2024-03-06T07:20:07.241706+0000","last_fullsized":"2024-03-06T07:20:07.241706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241416+0000","last_change":"2024-03-06T07:20:06.127004+0000","last_active":"2024-03-06T07:20:07.241416+0000","last_peered":"2024-03-06T07:20:07.241416+0000","last_clean":"2024-03-06T07:20:07.241416+0000","last_became_active":"2024-03-06T07:20:06.126643+0000","last_became_peered":"2024-03-06T07:20:06.126643+0000","last_unstale":"2024-03-06T07:20:07.241416+0000","last_undegraded":"2024-03-06T07:20:07.241416+0000","last_fullsized":"2024-03-06T07:20:07.241416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.815676+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.815238+0000","last_became_peered":"2024-03-06T07:20:01.815238+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.693301+0000","last_change":"2024-03-06T07:20:00.818890+0000","last_active":"2024-03-06T07:20:10.693301+0000","last_peered":"2024-03-06T07:20:10.693301+0000","last_clean":"2024-03-06T07:20:10.693301+0000","last_became_active":"2024-03-06T07:20:00.818613+0000","last_became_peered":"2024-03-06T07:20:00.818613+0000","last_unstale":"2024-03-06T07:20:10.693301+0000","last_undegraded":"2024-03-06T07:20:10.693301+0000","last_fullsized":"2024-03-06T07:20:10.693301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242003+0000","last_change":"2024-03-06T07:20:05.862991+0000","last_active":"2024-03-06T07:20:07.242003+0000","last_peered":"2024-03-06T07:20:07.242003+0000","last_clean":"2024-03-06T07:20:07.242003+0000","last_became_active":"2024-03-06T07:20:05.862334+0000","last_became_peered":"2024-03-06T07:20:05.862334+0000","last_unstale":"2024-03-06T07:20:07.242003+0000","last_undegraded":"2024-03-06T07:20:07.242003+0000","last_fullsized":"2024-03-06T07:20:07.242003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241685+0000","last_change":"2024-03-06T07:20:01.815200+0000","last_active":"2024-03-06T07:20:07.241685+0000","last_peered":"2024-03-06T07:20:07.241685+0000","last_clean":"2024-03-06T07:20:07.241685+0000","last_became_active":"2024-03-06T07:20:01.814903+0000","last_became_peered":"2024-03-06T07:20:01.814903+0000","last_unstale":"2024-03-06T07:20:07.241685+0000","last_undegraded":"2024-03-06T07:20:07.241685+0000","last_fullsized":"2024-03-06T07:20:07.241685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242321+0000","last_change":"2024-03-06T07:20:00.822249+0000","last_active":"2024-03-06T07:20:07.242321+0000","last_peered":"2024-03-06T07:20:07.242321+0000","last_clean":"2024-03-06T07:20:07.242321+0000","last_became_active":"2024-03-06T07:20:00.821965+0000","last_became_peered":"2024-03-06T07:20:00.821965+0000","last_unstale":"2024-03-06T07:20:07.242321+0000","last_undegraded":"2024-03-06T07:20:07.242321+0000","last_fullsized":"2024-03-06T07:20:07.242321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241085+0000","last_change":"2024-03-06T07:20:05.862268+0000","last_active":"2024-03-06T07:20:07.241085+0000","last_peered":"2024-03-06T07:20:07.241085+0000","last_clean":"2024-03-06T07:20:07.241085+0000","last_became_active":"2024-03-06T07:20:05.861794+0000","last_became_peered":"2024-03-06T07:20:05.861794+0000","last_unstale":"2024-03-06T07:20:07.241085+0000","last_undegraded":"2024-03-06T07:20:07.241085+0000","last_fullsized":"2024-03-06T07:20:07.241085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:15.471029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241294+0000","last_change":"2024-03-06T07:20:01.835260+0000","last_active":"2024-03-06T07:20:07.241294+0000","last_peered":"2024-03-06T07:20:07.241294+0000","last_clean":"2024-03-06T07:20:07.241294+0000","last_became_active":"2024-03-06T07:20:01.835069+0000","last_became_peered":"2024-03-06T07:20:01.835069+0000","last_unstale":"2024-03-06T07:20:07.241294+0000","last_undegraded":"2024-03-06T07:20:07.241294+0000","last_fullsized":"2024-03-06T07:20:07.241294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241087+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241087+0000","last_peered":"2024-03-06T07:20:07.241087+0000","last_clean":"2024-03-06T07:20:07.241087+0000","last_became_active":"2024-03-06T07:20:00.822307+0000","last_became_peered":"2024-03-06T07:20:00.822307+0000","last_unstale":"2024-03-06T07:20:07.241087+0000","last_undegraded":"2024-03-06T07:20:07.241087+0000","last_fullsized":"2024-03-06T07:20:07.241087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242067+0000","last_change":"2024-03-06T07:20:05.857403+0000","last_active":"2024-03-06T07:20:07.242067+0000","last_peered":"2024-03-06T07:20:07.242067+0000","last_clean":"2024-03-06T07:20:07.242067+0000","last_became_active":"2024-03-06T07:20:05.857036+0000","last_became_peered":"2024-03-06T07:20:05.857036+0000","last_unstale":"2024-03-06T07:20:07.242067+0000","last_undegraded":"2024-03-06T07:20:07.242067+0000","last_fullsized":"2024-03-06T07:20:07.242067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:23.280198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241050+0000","last_change":"2024-03-06T07:20:01.833389+0000","last_active":"2024-03-06T07:20:07.241050+0000","last_peered":"2024-03-06T07:20:07.241050+0000","last_clean":"2024-03-06T07:20:07.241050+0000","last_became_active":"2024-03-06T07:20:01.833094+0000","last_became_peered":"2024-03-06T07:20:01.833094+0000","last_unstale":"2024-03-06T07:20:07.241050+0000","last_undegraded":"2024-03-06T07:20:07.241050+0000","last_fullsized":"2024-03-06T07:20:07.241050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839444+0000","last_change":"2024-03-06T07:20:00.817766+0000","last_active":"2024-03-06T07:20:06.839444+0000","last_peered":"2024-03-06T07:20:06.839444+0000","last_clean":"2024-03-06T07:20:06.839444+0000","last_became_active":"2024-03-06T07:20:00.817466+0000","last_became_peered":"2024-03-06T07:20:00.817466+0000","last_unstale":"2024-03-06T07:20:06.839444+0000","last_undegraded":"2024-03-06T07:20:06.839444+0000","last_fullsized":"2024-03-06T07:20:06.839444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242588+0000","last_change":"2024-03-06T07:20:06.127038+0000","last_active":"2024-03-06T07:20:07.242588+0000","last_peered":"2024-03-06T07:20:07.242588+0000","last_clean":"2024-03-06T07:20:07.242588+0000","last_became_active":"2024-03-06T07:20:06.126185+0000","last_became_peered":"2024-03-06T07:20:06.126185+0000","last_unstale":"2024-03-06T07:20:07.242588+0000","last_undegraded":"2024-03-06T07:20:07.242588+0000","last_fullsized":"2024-03-06T07:20:07.242588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241025+0000","last_change":"2024-03-06T07:20:06.126567+0000","last_active":"2024-03-06T07:20:07.241025+0000","last_peered":"2024-03-06T07:20:07.241025+0000","last_clean":"2024-03-06T07:20:07.241025+0000","last_became_active":"2024-03-06T07:20:06.126198+0000","last_became_peered":"2024-03-06T07:20:06.126198+0000","last_unstale":"2024-03-06T07:20:07.241025+0000","last_undegraded":"2024-03-06T07:20:07.241025+0000","last_fullsized":"2024-03-06T07:20:07.241025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240998+0000","last_change":"2024-03-06T07:20:01.817263+0000","last_active":"2024-03-06T07:20:07.240998+0000","last_peered":"2024-03-06T07:20:07.240998+0000","last_clean":"2024-03-06T07:20:07.240998+0000","last_became_active":"2024-03-06T07:20:01.816994+0000","last_became_peered":"2024-03-06T07:20:01.816994+0000","last_unstale":"2024-03-06T07:20:07.240998+0000","last_undegraded":"2024-03-06T07:20:07.240998+0000","last_fullsized":"2024-03-06T07:20:07.240998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241203+0000","last_change":"2024-03-06T07:20:00.814224+0000","last_active":"2024-03-06T07:20:07.241203+0000","last_peered":"2024-03-06T07:20:07.241203+0000","last_clean":"2024-03-06T07:20:07.241203+0000","last_became_active":"2024-03-06T07:20:00.813908+0000","last_became_peered":"2024-03-06T07:20:00.813908+0000","last_unstale":"2024-03-06T07:20:07.241203+0000","last_undegraded":"2024-03-06T07:20:07.241203+0000","last_fullsized":"2024-03-06T07:20:07.241203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839419+0000","last_change":"2024-03-06T07:20:05.863149+0000","last_active":"2024-03-06T07:20:06.839419+0000","last_peered":"2024-03-06T07:20:06.839419+0000","last_clean":"2024-03-06T07:20:06.839419+0000","last_became_active":"2024-03-06T07:20:05.862680+0000","last_became_peered":"2024-03-06T07:20:05.862680+0000","last_unstale":"2024-03-06T07:20:06.839419+0000","last_undegraded":"2024-03-06T07:20:06.839419+0000","last_fullsized":"2024-03-06T07:20:06.839419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242085+0000","last_change":"2024-03-06T07:20:01.822303+0000","last_active":"2024-03-06T07:20:07.242085+0000","last_peered":"2024-03-06T07:20:07.242085+0000","last_clean":"2024-03-06T07:20:07.242085+0000","last_became_active":"2024-03-06T07:20:01.822044+0000","last_became_peered":"2024-03-06T07:20:01.822044+0000","last_unstale":"2024-03-06T07:20:07.242085+0000","last_undegraded":"2024-03-06T07:20:07.242085+0000","last_fullsized":"2024-03-06T07:20:07.242085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242899+0000","last_change":"2024-03-06T07:20:00.819504+0000","last_active":"2024-03-06T07:20:07.242899+0000","last_peered":"2024-03-06T07:20:07.242899+0000","last_clean":"2024-03-06T07:20:07.242899+0000","last_became_active":"2024-03-06T07:20:00.819223+0000","last_became_peered":"2024-03-06T07:20:00.819223+0000","last_unstale":"2024-03-06T07:20:07.242899+0000","last_undegraded":"2024-03-06T07:20:07.242899+0000","last_fullsized":"2024-03-06T07:20:07.242899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839079+0000","last_change":"2024-03-06T07:20:00.816629+0000","last_active":"2024-03-06T07:20:06.839079+0000","last_peered":"2024-03-06T07:20:06.839079+0000","last_clean":"2024-03-06T07:20:06.839079+0000","last_became_active":"2024-03-06T07:20:00.816373+0000","last_became_peered":"2024-03-06T07:20:00.816373+0000","last_unstale":"2024-03-06T07:20:06.839079+0000","last_undegraded":"2024-03-06T07:20:06.839079+0000","last_fullsized":"2024-03-06T07:20:06.839079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241631+0000","last_change":"2024-03-06T07:20:01.829181+0000","last_active":"2024-03-06T07:20:07.241631+0000","last_peered":"2024-03-06T07:20:07.241631+0000","last_clean":"2024-03-06T07:20:07.241631+0000","last_became_active":"2024-03-06T07:20:01.827761+0000","last_became_peered":"2024-03-06T07:20:01.827761+0000","last_unstale":"2024-03-06T07:20:07.241631+0000","last_undegraded":"2024-03-06T07:20:07.241631+0000","last_fullsized":"2024-03-06T07:20:07.241631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839350+0000","last_change":"2024-03-06T07:20:05.857388+0000","last_active":"2024-03-06T07:20:06.839350+0000","last_peered":"2024-03-06T07:20:06.839350+0000","last_clean":"2024-03-06T07:20:06.839350+0000","last_became_active":"2024-03-06T07:20:05.856981+0000","last_became_peered":"2024-03-06T07:20:05.856981+0000","last_unstale":"2024-03-06T07:20:06.839350+0000","last_undegraded":"2024-03-06T07:20:06.839350+0000","last_fullsized":"2024-03-06T07:20:06.839350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:51.153823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243225+0000","last_change":"2024-03-06T07:20:00.822065+0000","last_active":"2024-03-06T07:20:07.243225+0000","last_peered":"2024-03-06T07:20:07.243225+0000","last_clean":"2024-03-06T07:20:07.243225+0000","last_became_active":"2024-03-06T07:20:00.821736+0000","last_became_peered":"2024-03-06T07:20:00.821736+0000","last_unstale":"2024-03-06T07:20:07.243225+0000","last_undegraded":"2024-03-06T07:20:07.243225+0000","last_fullsized":"2024-03-06T07:20:07.243225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241924+0000","last_change":"2024-03-06T07:20:01.819959+0000","last_active":"2024-03-06T07:20:07.241924+0000","last_peered":"2024-03-06T07:20:07.241924+0000","last_clean":"2024-03-06T07:20:07.241924+0000","last_became_active":"2024-03-06T07:20:01.819548+0000","last_became_peered":"2024-03-06T07:20:01.819548+0000","last_unstale":"2024-03-06T07:20:07.241924+0000","last_undegraded":"2024-03-06T07:20:07.241924+0000","last_fullsized":"2024-03-06T07:20:07.241924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:40.273910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243065+0000","last_change":"2024-03-06T07:20:05.864320+0000","last_active":"2024-03-06T07:20:07.243065+0000","last_peered":"2024-03-06T07:20:07.243065+0000","last_clean":"2024-03-06T07:20:07.243065+0000","last_became_active":"2024-03-06T07:20:05.863902+0000","last_became_peered":"2024-03-06T07:20:05.863902+0000","last_unstale":"2024-03-06T07:20:07.243065+0000","last_undegraded":"2024-03-06T07:20:07.243065+0000","last_fullsized":"2024-03-06T07:20:07.243065+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838709+0000","last_change":"2024-03-06T07:20:01.824719+0000","last_active":"2024-03-06T07:20:06.838709+0000","last_peered":"2024-03-06T07:20:06.838709+0000","last_clean":"2024-03-06T07:20:06.838709+0000","last_became_active":"2024-03-06T07:20:01.824424+0000","last_became_peered":"2024-03-06T07:20:01.824424+0000","last_unstale":"2024-03-06T07:20:06.838709+0000","last_undegraded":"2024-03-06T07:20:06.838709+0000","last_fullsized":"2024-03-06T07:20:06.838709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839111+0000","last_change":"2024-03-06T07:20:01.827824+0000","last_active":"2024-03-06T07:20:06.839111+0000","last_peered":"2024-03-06T07:20:06.839111+0000","last_clean":"2024-03-06T07:20:06.839111+0000","last_became_active":"2024-03-06T07:20:01.827552+0000","last_became_peered":"2024-03-06T07:20:01.827552+0000","last_unstale":"2024-03-06T07:20:06.839111+0000","last_undegraded":"2024-03-06T07:20:06.839111+0000","last_fullsized":"2024-03-06T07:20:06.839111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:24.188510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242476+0000","last_change":"2024-03-06T07:20:01.827949+0000","last_active":"2024-03-06T07:20:07.242476+0000","last_peered":"2024-03-06T07:20:07.242476+0000","last_clean":"2024-03-06T07:20:07.242476+0000","last_became_active":"2024-03-06T07:20:01.827535+0000","last_became_peered":"2024-03-06T07:20:01.827535+0000","last_unstale":"2024-03-06T07:20:07.242476+0000","last_undegraded":"2024-03-06T07:20:07.242476+0000","last_fullsized":"2024-03-06T07:20:07.242476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241603+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.241603+0000","last_peered":"2024-03-06T07:20:07.241603+0000","last_clean":"2024-03-06T07:20:07.241603+0000","last_became_active":"2024-03-06T07:20:01.828105+0000","last_became_peered":"2024-03-06T07:20:01.828105+0000","last_unstale":"2024-03-06T07:20:07.241603+0000","last_undegraded":"2024-03-06T07:20:07.241603+0000","last_fullsized":"2024-03-06T07:20:07.241603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243331+0000","last_change":"2024-03-06T07:20:01.810561+0000","last_active":"2024-03-06T07:20:07.243331+0000","last_peered":"2024-03-06T07:20:07.243331+0000","last_clean":"2024-03-06T07:20:07.243331+0000","last_became_active":"2024-03-06T07:20:01.810258+0000","last_became_peered":"2024-03-06T07:20:01.810258+0000","last_unstale":"2024-03-06T07:20:07.243331+0000","last_undegraded":"2024-03-06T07:20:07.243331+0000","last_fullsized":"2024-03-06T07:20:07.243331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242209+0000","last_change":"2024-03-06T07:20:01.812180+0000","last_active":"2024-03-06T07:20:07.242209+0000","last_peered":"2024-03-06T07:20:07.242209+0000","last_clean":"2024-03-06T07:20:07.242209+0000","last_became_active":"2024-03-06T07:20:01.811847+0000","last_became_peered":"2024-03-06T07:20:01.811847+0000","last_unstale":"2024-03-06T07:20:07.242209+0000","last_undegraded":"2024-03-06T07:20:07.242209+0000","last_fullsized":"2024-03-06T07:20:07.242209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839401+0000","last_change":"2024-03-06T07:20:01.821989+0000","last_active":"2024-03-06T07:20:06.839401+0000","last_peered":"2024-03-06T07:20:06.839401+0000","last_clean":"2024-03-06T07:20:06.839401+0000","last_became_active":"2024-03-06T07:20:01.821682+0000","last_became_peered":"2024-03-06T07:20:01.821682+0000","last_unstale":"2024-03-06T07:20:06.839401+0000","last_undegraded":"2024-03-06T07:20:06.839401+0000","last_fullsized":"2024-03-06T07:20:06.839401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243323+0000","last_change":"2024-03-06T07:20:01.815852+0000","last_active":"2024-03-06T07:20:07.243323+0000","last_peered":"2024-03-06T07:20:07.243323+0000","last_clean":"2024-03-06T07:20:07.243323+0000","last_became_active":"2024-03-06T07:20:01.815523+0000","last_became_peered":"2024-03-06T07:20:01.815523+0000","last_unstale":"2024-03-06T07:20:07.243323+0000","last_undegraded":"2024-03-06T07:20:07.243323+0000","last_fullsized":"2024-03-06T07:20:07.243323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241111+0000","last_change":"2024-03-06T07:20:01.828459+0000","last_active":"2024-03-06T07:20:07.241111+0000","last_peered":"2024-03-06T07:20:07.241111+0000","last_clean":"2024-03-06T07:20:07.241111+0000","last_became_active":"2024-03-06T07:20:01.828110+0000","last_became_peered":"2024-03-06T07:20:01.828110+0000","last_unstale":"2024-03-06T07:20:07.241111+0000","last_undegraded":"2024-03-06T07:20:07.241111+0000","last_fullsized":"2024-03-06T07:20:07.241111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242736+0000","last_change":"2024-03-06T07:20:01.810465+0000","last_active":"2024-03-06T07:20:07.242736+0000","last_peered":"2024-03-06T07:20:07.242736+0000","last_clean":"2024-03-06T07:20:07.242736+0000","last_became_active":"2024-03-06T07:20:01.810206+0000","last_became_peered":"2024-03-06T07:20:01.810206+0000","last_unstale":"2024-03-06T07:20:07.242736+0000","last_undegraded":"2024-03-06T07:20:07.242736+0000","last_fullsized":"2024-03-06T07:20:07.242736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242393+0000","last_change":"2024-03-06T07:20:01.834744+0000","last_active":"2024-03-06T07:20:07.242393+0000","last_peered":"2024-03-06T07:20:07.242393+0000","last_clean":"2024-03-06T07:20:07.242393+0000","last_became_active":"2024-03-06T07:20:01.834049+0000","last_became_peered":"2024-03-06T07:20:01.834049+0000","last_unstale":"2024-03-06T07:20:07.242393+0000","last_undegraded":"2024-03-06T07:20:07.242393+0000","last_fullsized":"2024-03-06T07:20:07.242393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241262+0000","last_change":"2024-03-06T07:20:01.830008+0000","last_active":"2024-03-06T07:20:07.241262+0000","last_peered":"2024-03-06T07:20:07.241262+0000","last_clean":"2024-03-06T07:20:07.241262+0000","last_became_active":"2024-03-06T07:20:01.828839+0000","last_became_peered":"2024-03-06T07:20:01.828839+0000","last_unstale":"2024-03-06T07:20:07.241262+0000","last_undegraded":"2024-03-06T07:20:07.241262+0000","last_fullsized":"2024-03-06T07:20:07.241262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839342+0000","last_change":"2024-03-06T07:20:01.815286+0000","last_active":"2024-03-06T07:20:06.839342+0000","last_peered":"2024-03-06T07:20:06.839342+0000","last_clean":"2024-03-06T07:20:06.839342+0000","last_became_active":"2024-03-06T07:20:01.814983+0000","last_became_peered":"2024-03-06T07:20:01.814983+0000","last_unstale":"2024-03-06T07:20:06.839342+0000","last_undegraded":"2024-03-06T07:20:06.839342+0000","last_fullsized":"2024-03-06T07:20:06.839342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242294+0000","last_change":"2024-03-06T07:20:01.816404+0000","last_active":"2024-03-06T07:20:07.242294+0000","last_peered":"2024-03-06T07:20:07.242294+0000","last_clean":"2024-03-06T07:20:07.242294+0000","last_became_active":"2024-03-06T07:20:01.815092+0000","last_became_peered":"2024-03-06T07:20:01.815092+0000","last_unstale":"2024-03-06T07:20:07.242294+0000","last_undegraded":"2024-03-06T07:20:07.242294+0000","last_fullsized":"2024-03-06T07:20:07.242294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839438+0000","last_change":"2024-03-06T07:20:01.819861+0000","last_active":"2024-03-06T07:20:06.839438+0000","last_peered":"2024-03-06T07:20:06.839438+0000","last_clean":"2024-03-06T07:20:06.839438+0000","last_became_active":"2024-03-06T07:20:01.819090+0000","last_became_peered":"2024-03-06T07:20:01.819090+0000","last_unstale":"2024-03-06T07:20:06.839438+0000","last_undegraded":"2024-03-06T07:20:06.839438+0000","last_fullsized":"2024-03-06T07:20:06.839438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.834134+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.833295+0000","last_became_peered":"2024-03-06T07:20:01.833295+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241004+0000","last_change":"2024-03-06T07:20:01.834489+0000","last_active":"2024-03-06T07:20:07.241004+0000","last_peered":"2024-03-06T07:20:07.241004+0000","last_clean":"2024-03-06T07:20:07.241004+0000","last_became_active":"2024-03-06T07:20:01.833824+0000","last_became_peered":"2024-03-06T07:20:01.833824+0000","last_unstale":"2024-03-06T07:20:07.241004+0000","last_undegraded":"2024-03-06T07:20:07.241004+0000","last_fullsized":"2024-03-06T07:20:07.241004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241109+0000","last_change":"2024-03-06T07:20:01.818055+0000","last_active":"2024-03-06T07:20:07.241109+0000","last_peered":"2024-03-06T07:20:07.241109+0000","last_clean":"2024-03-06T07:20:07.241109+0000","last_became_active":"2024-03-06T07:20:01.817719+0000","last_became_peered":"2024-03-06T07:20:01.817719+0000","last_unstale":"2024-03-06T07:20:07.241109+0000","last_undegraded":"2024-03-06T07:20:07.241109+0000","last_fullsized":"2024-03-06T07:20:07.241109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241162+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241162+0000","last_peered":"2024-03-06T07:20:07.241162+0000","last_clean":"2024-03-06T07:20:07.241162+0000","last_became_active":"2024-03-06T07:20:01.825795+0000","last_became_peered":"2024-03-06T07:20:01.825795+0000","last_unstale":"2024-03-06T07:20:07.241162+0000","last_undegraded":"2024-03-06T07:20:07.241162+0000","last_fullsized":"2024-03-06T07:20:07.241162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243013+0000","last_change":"2024-03-06T07:20:01.818654+0000","last_active":"2024-03-06T07:20:07.243013+0000","last_peered":"2024-03-06T07:20:07.243013+0000","last_clean":"2024-03-06T07:20:07.243013+0000","last_became_active":"2024-03-06T07:20:01.818368+0000","last_became_peered":"2024-03-06T07:20:01.818368+0000","last_unstale":"2024-03-06T07:20:07.243013+0000","last_undegraded":"2024-03-06T07:20:07.243013+0000","last_fullsized":"2024-03-06T07:20:07.243013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.817094+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.816841+0000","last_became_peered":"2024-03-06T07:20:01.816841+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:26.548988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839291+0000","last_change":"2024-03-06T07:20:01.831471+0000","last_active":"2024-03-06T07:20:06.839291+0000","last_peered":"2024-03-06T07:20:06.839291+0000","last_clean":"2024-03-06T07:20:06.839291+0000","last_became_active":"2024-03-06T07:20:01.831177+0000","last_became_peered":"2024-03-06T07:20:01.831177+0000","last_unstale":"2024-03-06T07:20:06.839291+0000","last_undegraded":"2024-03-06T07:20:06.839291+0000","last_fullsized":"2024-03-06T07:20:06.839291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839377+0000","last_change":"2024-03-06T07:20:01.831736+0000","last_active":"2024-03-06T07:20:06.839377+0000","last_peered":"2024-03-06T07:20:06.839377+0000","last_clean":"2024-03-06T07:20:06.839377+0000","last_became_active":"2024-03-06T07:20:01.831450+0000","last_became_peered":"2024-03-06T07:20:01.831450+0000","last_unstale":"2024-03-06T07:20:06.839377+0000","last_undegraded":"2024-03-06T07:20:06.839377+0000","last_fullsized":"2024-03-06T07:20:06.839377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242607+0000","last_change":"2024-03-06T07:20:01.832323+0000","last_active":"2024-03-06T07:20:07.242607+0000","last_peered":"2024-03-06T07:20:07.242607+0000","last_clean":"2024-03-06T07:20:07.242607+0000","last_became_active":"2024-03-06T07:20:01.832078+0000","last_became_peered":"2024-03-06T07:20:01.832078+0000","last_unstale":"2024-03-06T07:20:07.242607+0000","last_undegraded":"2024-03-06T07:20:07.242607+0000","last_fullsized":"2024-03-06T07:20:07.242607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241991+0000","last_change":"2024-03-06T07:20:01.831803+0000","last_active":"2024-03-06T07:20:07.241991+0000","last_peered":"2024-03-06T07:20:07.241991+0000","last_clean":"2024-03-06T07:20:07.241991+0000","last_became_active":"2024-03-06T07:20:01.831509+0000","last_became_peered":"2024-03-06T07:20:01.831509+0000","last_unstale":"2024-03-06T07:20:07.241991+0000","last_undegraded":"2024-03-06T07:20:07.241991+0000","last_fullsized":"2024-03-06T07:20:07.241991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242029+0000","last_change":"2024-03-06T07:20:01.828627+0000","last_active":"2024-03-06T07:20:07.242029+0000","last_peered":"2024-03-06T07:20:07.242029+0000","last_clean":"2024-03-06T07:20:07.242029+0000","last_became_active":"2024-03-06T07:20:01.828182+0000","last_became_peered":"2024-03-06T07:20:01.828182+0000","last_unstale":"2024-03-06T07:20:07.242029+0000","last_undegraded":"2024-03-06T07:20:07.242029+0000","last_fullsized":"2024-03-06T07:20:07.242029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242311+0000","last_change":"2024-03-06T07:20:01.834757+0000","last_active":"2024-03-06T07:20:07.242311+0000","last_peered":"2024-03-06T07:20:07.242311+0000","last_clean":"2024-03-06T07:20:07.242311+0000","last_became_active":"2024-03-06T07:20:01.833698+0000","last_became_peered":"2024-03-06T07:20:01.833698+0000","last_unstale":"2024-03-06T07:20:07.242311+0000","last_undegraded":"2024-03-06T07:20:07.242311+0000","last_fullsized":"2024-03-06T07:20:07.242311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241536+0000","last_change":"2024-03-06T07:20:01.816955+0000","last_active":"2024-03-06T07:20:07.241536+0000","last_peered":"2024-03-06T07:20:07.241536+0000","last_clean":"2024-03-06T07:20:07.241536+0000","last_became_active":"2024-03-06T07:20:01.816124+0000","last_became_peered":"2024-03-06T07:20:01.816124+0000","last_unstale":"2024-03-06T07:20:07.241536+0000","last_undegraded":"2024-03-06T07:20:07.241536+0000","last_fullsized":"2024-03-06T07:20:07.241536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241876+0000","last_change":"2024-03-06T07:20:01.831322+0000","last_active":"2024-03-06T07:20:07.241876+0000","last_peered":"2024-03-06T07:20:07.241876+0000","last_clean":"2024-03-06T07:20:07.241876+0000","last_became_active":"2024-03-06T07:20:01.830919+0000","last_became_peered":"2024-03-06T07:20:01.830919+0000","last_unstale":"2024-03-06T07:20:07.241876+0000","last_undegraded":"2024-03-06T07:20:07.241876+0000","last_fullsized":"2024-03-06T07:20:07.241876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241171+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241171+0000","last_peered":"2024-03-06T07:20:07.241171+0000","last_clean":"2024-03-06T07:20:07.241171+0000","last_became_active":"2024-03-06T07:20:01.826111+0000","last_became_peered":"2024-03-06T07:20:01.826111+0000","last_unstale":"2024-03-06T07:20:07.241171+0000","last_undegraded":"2024-03-06T07:20:07.241171+0000","last_fullsized":"2024-03-06T07:20:07.241171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241941+0000","last_change":"2024-03-06T07:20:01.822677+0000","last_active":"2024-03-06T07:20:07.241941+0000","last_peered":"2024-03-06T07:20:07.241941+0000","last_clean":"2024-03-06T07:20:07.241941+0000","last_became_active":"2024-03-06T07:20:01.822263+0000","last_became_peered":"2024-03-06T07:20:01.822263+0000","last_unstale":"2024-03-06T07:20:07.241941+0000","last_undegraded":"2024-03-06T07:20:07.241941+0000","last_fullsized":"2024-03-06T07:20:07.241941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242529+0000","last_change":"2024-03-06T07:20:01.818125+0000","last_active":"2024-03-06T07:20:07.242529+0000","last_peered":"2024-03-06T07:20:07.242529+0000","last_clean":"2024-03-06T07:20:07.242529+0000","last_became_active":"2024-03-06T07:20:01.817547+0000","last_became_peered":"2024-03-06T07:20:01.817547+0000","last_unstale":"2024-03-06T07:20:07.242529+0000","last_undegraded":"2024-03-06T07:20:07.242529+0000","last_fullsized":"2024-03-06T07:20:07.242529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":691206,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":691262,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:19.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:19.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.234+0000 7fb1e3cfb640 1 Processor -- start 2024-03-06T07:20:19.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.234+0000 7fb1e3cfb640 1 -- start start 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e3cfb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1dc0c5330 0x7fb1dc0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e3cfb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1dc0c88b0 0x7fb1dc1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e3cfb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1dc155110 con 0x7fb1dc0c7dd0 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e3cfb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc1559e0 con 0x7fb1dc0c88b0 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e3cfb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc0c8f40 con 0x7fb1dc0c5330 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1296640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1dc0c88b0 0x7fb1dc1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1296640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1dc0c88b0 0x7fb1dc1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46714/0 (socket says 172.21.15.26:46714) 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1296640 1 -- 172.21.15.26:0/2802255957 learned_addr learned my addr 172.21.15.26:0/2802255957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1a97640 1 --2- 172.21.15.26:0/2802255957 >> v2:172.21.15.117:3300/0 conn(0x7fb1dc0c5330 0x7fb1dc0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1296640 1 -- 172.21.15.26:0/2802255957 >> v2:172.21.15.117:3300/0 conn(0x7fb1dc0c5330 msgr2=0x7fb1dc0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1296640 1 --2- 172.21.15.26:0/2802255957 >> v2:172.21.15.117:3300/0 conn(0x7fb1dc0c5330 0x7fb1dc0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1296640 1 -- 172.21.15.26:0/2802255957 >> v1:172.21.15.117:6789/0 conn(0x7fb1dc0c7dd0 legacy=0x7fb1dc06b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.235+0000 7fb1e1296640 1 -- 172.21.15.26:0/2802255957 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc155390 con 0x7fb1dc0c88b0 2024-03-06T07:20:19.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.236+0000 7fb1e1296640 1 --2- 172.21.15.26:0/2802255957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c88b0 0x7fb1dc1586a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc08c6e0 tx=0x7fb1cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bd03374f4c1d9e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.236+0000 7fb1e0a95640 1 -- 172.21.15.26:0/2802255957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc035020 con 0x7fb1dc0c88b0 2024-03-06T07:20:19.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.236+0000 7fb1e0a95640 1 -- 172.21.15.26:0/2802255957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc031870 con 0x7fb1dc0c88b0 2024-03-06T07:20:19.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.236+0000 7fb1e0a95640 1 -- 172.21.15.26:0/2802255957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc031c40 con 0x7fb1dc0c88b0 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/2802255957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c88b0 msgr2=0x7fb1dc1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/2802255957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c88b0 0x7fb1dc1586a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc08c6e0 tx=0x7fb1cc02b910 comp rx=0 tx=0).stop 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/2802255957 shutdown_connections 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/2802255957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c88b0 0x7fb1dc1586a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/2802255957 >> v2:172.21.15.117:3300/0 conn(0x7fb1dc0c5330 0x7fb1dc0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/2802255957 >> 172.21.15.26:0/2802255957 conn(0x7fb1dc0ac560 msgr2=0x7fb1dc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/2802255957 shutdown_connections 2024-03-06T07:20:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.237+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/2802255957 wait complete. 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.238+0000 7fb1e3cfb640 1 Processor -- start 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.238+0000 7fb1e3cfb640 1 -- start start 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c5330 0x7fb1dc147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1dc0c7dd0 0x7fb1dc140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1dc0c88b0 0x7fb1dc141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc1529d0 con 0x7fb1dc0c5330 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc06d090 con 0x7fb1dc0c88b0 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc152b50 con 0x7fb1dc0c7dd0 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c5330 0x7fb1dc147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c5330 0x7fb1dc147390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46718/0 (socket says 172.21.15.26:46718) 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 -- 172.21.15.26:0/4236338690 learned_addr learned my addr 172.21.15.26:0/4236338690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1296640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1dc0c7dd0 0x7fb1dc140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e2298640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1dc0c88b0 0x7fb1dc141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 -- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1dc0c7dd0 msgr2=0x7fb1dc140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1dc0c7dd0 0x7fb1dc140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 -- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1dc0c88b0 msgr2=0x7fb1dc141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1dc0c88b0 0x7fb1dc141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 -- 172.21.15.26:0/4236338690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc141960 con 0x7fb1dc0c5330 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e1a97640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c5330 0x7fb1dc147390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9b0 tx=0x7fb1d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1d27fc640 1 -- 172.21.15.26:0/4236338690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4017070 con 0x7fb1dc0c5330 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1d27fc640 1 -- 172.21.15.26:0/4236338690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400c9a0 con 0x7fb1dc0c5330 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc141df0 con 0x7fb1dc0c5330 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1d27fc640 1 -- 172.21.15.26:0/4236338690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400cd90 con 0x7fb1dc0c5330 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.239+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc1eed40 con 0x7fb1dc0c5330 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.240+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc1eab40 con 0x7fb1dc0c5330 2024-03-06T07:20:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.241+0000 7fb1d27fc640 1 -- 172.21.15.26:0/4236338690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1d401a070 con 0x7fb1dc0c5330 2024-03-06T07:20:19.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.241+0000 7fb1d27fc640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1c006b4d0 0x7fb1c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.241+0000 7fb1d27fc640 1 -- 172.21.15.26:0/4236338690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb1d408a480 con 0x7fb1dc0c5330 2024-03-06T07:20:19.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.241+0000 7fb1e1296640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1c006b4d0 0x7fb1c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.242+0000 7fb1e1296640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1c006b4d0 0x7fb1c006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc036000 tx=0x7fb1cc03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.245+0000 7fb1d27fc640 1 -- 172.21.15.26:0/4236338690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb1d4058970 con 0x7fb1dc0c5330 2024-03-06T07:20:19.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.424+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1dc141ba0 con 0x7fb1c006b4d0 2024-03-06T07:20:19.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.432+0000 7fb1d27fc640 1 -- 172.21.15.26:0/4236338690 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332635 (secure 0 0 0) 0x7fb1cc099370 con 0x7fb1c006b4d0 2024-03-06T07:20:19.438 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:20:19.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1c006b4d0 msgr2=0x7fb1c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1c006b4d0 0x7fb1c006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc036000 tx=0x7fb1cc03c040 comp rx=0 tx=0).stop 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c5330 msgr2=0x7fb1dc147390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c5330 0x7fb1dc147390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9b0 tx=0x7fb1d400ee70 comp rx=0 tx=0).stop 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 shutdown_connections 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1c006b4d0 0x7fb1c006d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1dc0c88b0 0x7fb1dc141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1dc0c7dd0 0x7fb1dc140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 --2- 172.21.15.26:0/4236338690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1dc0c5330 0x7fb1dc147390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 >> 172.21.15.26:0/4236338690 conn(0x7fb1dc0ac560 msgr2=0x7fb1dc153d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 shutdown_connections 2024-03-06T07:20:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.436+0000 7fb1e3cfb640 1 -- 172.21.15.26:0/4236338690 wait complete. 2024-03-06T07:20:19.448 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-03-06T07:20:19.033981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230820,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743900,"statfs":{"total":773094113280,"available":772857753600,"internally_reserved":0,"allocated":2228224,"data_stored":1991782,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002684"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241454+0000","last_change":"2024-03-06T07:20:00.821957+0000","last_active":"2024-03-06T07:20:07.241454+0000","last_peered":"2024-03-06T07:20:07.241454+0000","last_clean":"2024-03-06T07:20:07.241454+0000","last_became_active":"2024-03-06T07:20:00.821672+0000","last_became_peered":"2024-03-06T07:20:00.821672+0000","last_unstale":"2024-03-06T07:20:07.241454+0000","last_undegraded":"2024-03-06T07:20:07.241454+0000","last_fullsized":"2024-03-06T07:20:07.241454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242911+0000","last_change":"2024-03-06T07:20:01.819400+0000","last_active":"2024-03-06T07:20:07.242911+0000","last_peered":"2024-03-06T07:20:07.242911+0000","last_clean":"2024-03-06T07:20:07.242911+0000","last_became_active":"2024-03-06T07:20:01.819073+0000","last_became_peered":"2024-03-06T07:20:01.819073+0000","last_unstale":"2024-03-06T07:20:07.242911+0000","last_undegraded":"2024-03-06T07:20:07.242911+0000","last_fullsized":"2024-03-06T07:20:07.242911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242373+0000","last_change":"2024-03-06T07:20:06.127375+0000","last_active":"2024-03-06T07:20:07.242373+0000","last_peered":"2024-03-06T07:20:07.242373+0000","last_clean":"2024-03-06T07:20:07.242373+0000","last_became_active":"2024-03-06T07:20:06.126687+0000","last_became_peered":"2024-03-06T07:20:06.126687+0000","last_unstale":"2024-03-06T07:20:07.242373+0000","last_undegraded":"2024-03-06T07:20:07.242373+0000","last_fullsized":"2024-03-06T07:20:07.242373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:11.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243026+0000","last_change":"2024-03-06T07:20:00.816186+0000","last_active":"2024-03-06T07:20:07.243026+0000","last_peered":"2024-03-06T07:20:07.243026+0000","last_clean":"2024-03-06T07:20:07.243026+0000","last_became_active":"2024-03-06T07:20:00.815110+0000","last_became_peered":"2024-03-06T07:20:00.815110+0000","last_unstale":"2024-03-06T07:20:07.243026+0000","last_undegraded":"2024-03-06T07:20:07.243026+0000","last_fullsized":"2024-03-06T07:20:07.243026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241124+0000","last_change":"2024-03-06T07:20:01.835360+0000","last_active":"2024-03-06T07:20:07.241124+0000","last_peered":"2024-03-06T07:20:07.241124+0000","last_clean":"2024-03-06T07:20:07.241124+0000","last_became_active":"2024-03-06T07:20:01.835066+0000","last_became_peered":"2024-03-06T07:20:01.835066+0000","last_unstale":"2024-03-06T07:20:07.241124+0000","last_undegraded":"2024-03-06T07:20:07.241124+0000","last_fullsized":"2024-03-06T07:20:07.241124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240859+0000","last_change":"2024-03-06T07:20:05.864904+0000","last_active":"2024-03-06T07:20:07.240859+0000","last_peered":"2024-03-06T07:20:07.240859+0000","last_clean":"2024-03-06T07:20:07.240859+0000","last_became_active":"2024-03-06T07:20:05.864535+0000","last_became_peered":"2024-03-06T07:20:05.864535+0000","last_unstale":"2024-03-06T07:20:07.240859+0000","last_undegraded":"2024-03-06T07:20:07.240859+0000","last_fullsized":"2024-03-06T07:20:07.240859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.016032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242211+0000","last_change":"2024-03-06T07:20:00.823477+0000","last_active":"2024-03-06T07:20:07.242211+0000","last_peered":"2024-03-06T07:20:07.242211+0000","last_clean":"2024-03-06T07:20:07.242211+0000","last_became_active":"2024-03-06T07:20:00.822898+0000","last_became_peered":"2024-03-06T07:20:00.822898+0000","last_unstale":"2024-03-06T07:20:07.242211+0000","last_undegraded":"2024-03-06T07:20:07.242211+0000","last_fullsized":"2024-03-06T07:20:07.242211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:23.183380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.832744+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.832237+0000","last_became_peered":"2024-03-06T07:20:01.832237+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241395+0000","last_change":"2024-03-06T07:20:05.862022+0000","last_active":"2024-03-06T07:20:07.241395+0000","last_peered":"2024-03-06T07:20:07.241395+0000","last_clean":"2024-03-06T07:20:07.241395+0000","last_became_active":"2024-03-06T07:20:05.861661+0000","last_became_peered":"2024-03-06T07:20:05.861661+0000","last_unstale":"2024-03-06T07:20:07.241395+0000","last_undegraded":"2024-03-06T07:20:07.241395+0000","last_fullsized":"2024-03-06T07:20:07.241395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:30.602011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241314+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241314+0000","last_peered":"2024-03-06T07:20:07.241314+0000","last_clean":"2024-03-06T07:20:07.241314+0000","last_became_active":"2024-03-06T07:20:00.822334+0000","last_became_peered":"2024-03-06T07:20:00.822334+0000","last_unstale":"2024-03-06T07:20:07.241314+0000","last_undegraded":"2024-03-06T07:20:07.241314+0000","last_fullsized":"2024-03-06T07:20:07.241314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241506+0000","last_change":"2024-03-06T07:20:01.832671+0000","last_active":"2024-03-06T07:20:07.241506+0000","last_peered":"2024-03-06T07:20:07.241506+0000","last_clean":"2024-03-06T07:20:07.241506+0000","last_became_active":"2024-03-06T07:20:01.832397+0000","last_became_peered":"2024-03-06T07:20:01.832397+0000","last_unstale":"2024-03-06T07:20:07.241506+0000","last_undegraded":"2024-03-06T07:20:07.241506+0000","last_fullsized":"2024-03-06T07:20:07.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241835+0000","last_change":"2024-03-06T07:20:05.853058+0000","last_active":"2024-03-06T07:20:07.241835+0000","last_peered":"2024-03-06T07:20:07.241835+0000","last_clean":"2024-03-06T07:20:07.241835+0000","last_became_active":"2024-03-06T07:20:05.852648+0000","last_became_peered":"2024-03-06T07:20:05.852648+0000","last_unstale":"2024-03-06T07:20:07.241835+0000","last_undegraded":"2024-03-06T07:20:07.241835+0000","last_fullsized":"2024-03-06T07:20:07.241835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:33.136033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838904+0000","last_change":"2024-03-06T07:20:01.817989+0000","last_active":"2024-03-06T07:20:06.838904+0000","last_peered":"2024-03-06T07:20:06.838904+0000","last_clean":"2024-03-06T07:20:06.838904+0000","last_became_active":"2024-03-06T07:20:01.817711+0000","last_became_peered":"2024-03-06T07:20:01.817711+0000","last_unstale":"2024-03-06T07:20:06.838904+0000","last_undegraded":"2024-03-06T07:20:06.838904+0000","last_fullsized":"2024-03-06T07:20:06.838904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241768+0000","last_change":"2024-03-06T07:20:00.822403+0000","last_active":"2024-03-06T07:20:07.241768+0000","last_peered":"2024-03-06T07:20:07.241768+0000","last_clean":"2024-03-06T07:20:07.241768+0000","last_became_active":"2024-03-06T07:20:00.822144+0000","last_became_peered":"2024-03-06T07:20:00.822144+0000","last_unstale":"2024-03-06T07:20:07.241768+0000","last_undegraded":"2024-03-06T07:20:07.241768+0000","last_fullsized":"2024-03-06T07:20:07.241768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241387+0000","last_change":"2024-03-06T07:20:05.864961+0000","last_active":"2024-03-06T07:20:07.241387+0000","last_peered":"2024-03-06T07:20:07.241387+0000","last_clean":"2024-03-06T07:20:07.241387+0000","last_became_active":"2024-03-06T07:20:05.864626+0000","last_became_peered":"2024-03-06T07:20:05.864626+0000","last_unstale":"2024-03-06T07:20:07.241387+0000","last_undegraded":"2024-03-06T07:20:07.241387+0000","last_fullsized":"2024-03-06T07:20:07.241387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:12.900271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243119+0000","last_change":"2024-03-06T07:20:01.830511+0000","last_active":"2024-03-06T07:20:07.243119+0000","last_peered":"2024-03-06T07:20:07.243119+0000","last_clean":"2024-03-06T07:20:07.243119+0000","last_became_active":"2024-03-06T07:20:01.830230+0000","last_became_peered":"2024-03-06T07:20:01.830230+0000","last_unstale":"2024-03-06T07:20:07.243119+0000","last_undegraded":"2024-03-06T07:20:07.243119+0000","last_fullsized":"2024-03-06T07:20:07.243119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242095+0000","last_change":"2024-03-06T07:20:00.816279+0000","last_active":"2024-03-06T07:20:07.242095+0000","last_peered":"2024-03-06T07:20:07.242095+0000","last_clean":"2024-03-06T07:20:07.242095+0000","last_became_active":"2024-03-06T07:20:00.815979+0000","last_became_peered":"2024-03-06T07:20:00.815979+0000","last_unstale":"2024-03-06T07:20:07.242095+0000","last_undegraded":"2024-03-06T07:20:07.242095+0000","last_fullsized":"2024-03-06T07:20:07.242095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240897+0000","last_change":"2024-03-06T07:20:05.864338+0000","last_active":"2024-03-06T07:20:07.240897+0000","last_peered":"2024-03-06T07:20:07.240897+0000","last_clean":"2024-03-06T07:20:07.240897+0000","last_became_active":"2024-03-06T07:20:05.863893+0000","last_became_peered":"2024-03-06T07:20:05.863893+0000","last_unstale":"2024-03-06T07:20:07.240897+0000","last_undegraded":"2024-03-06T07:20:07.240897+0000","last_fullsized":"2024-03-06T07:20:07.240897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:55.097092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242166+0000","last_change":"2024-03-06T07:20:01.813572+0000","last_active":"2024-03-06T07:20:07.242166+0000","last_peered":"2024-03-06T07:20:07.242166+0000","last_clean":"2024-03-06T07:20:07.242166+0000","last_became_active":"2024-03-06T07:20:01.813285+0000","last_became_peered":"2024-03-06T07:20:01.813285+0000","last_unstale":"2024-03-06T07:20:07.242166+0000","last_undegraded":"2024-03-06T07:20:07.242166+0000","last_fullsized":"2024-03-06T07:20:07.242166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:00.818384+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:00.818086+0000","last_became_peered":"2024-03-06T07:20:00.818086+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242538+0000","last_change":"2024-03-06T07:20:06.124240+0000","last_active":"2024-03-06T07:20:07.242538+0000","last_peered":"2024-03-06T07:20:07.242538+0000","last_clean":"2024-03-06T07:20:07.242538+0000","last_became_active":"2024-03-06T07:20:06.123859+0000","last_became_peered":"2024-03-06T07:20:06.123859+0000","last_unstale":"2024-03-06T07:20:07.242538+0000","last_undegraded":"2024-03-06T07:20:07.242538+0000","last_fullsized":"2024-03-06T07:20:07.242538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241784+0000","last_change":"2024-03-06T07:20:01.812138+0000","last_active":"2024-03-06T07:20:07.241784+0000","last_peered":"2024-03-06T07:20:07.241784+0000","last_clean":"2024-03-06T07:20:07.241784+0000","last_became_active":"2024-03-06T07:20:01.811851+0000","last_became_peered":"2024-03-06T07:20:01.811851+0000","last_unstale":"2024-03-06T07:20:07.241784+0000","last_undegraded":"2024-03-06T07:20:07.241784+0000","last_fullsized":"2024-03-06T07:20:07.241784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:22.590118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241483+0000","last_change":"2024-03-06T07:20:00.821941+0000","last_active":"2024-03-06T07:20:07.241483+0000","last_peered":"2024-03-06T07:20:07.241483+0000","last_clean":"2024-03-06T07:20:07.241483+0000","last_became_active":"2024-03-06T07:20:00.821533+0000","last_became_peered":"2024-03-06T07:20:00.821533+0000","last_unstale":"2024-03-06T07:20:07.241483+0000","last_undegraded":"2024-03-06T07:20:07.241483+0000","last_fullsized":"2024-03-06T07:20:07.241483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241052+0000","last_change":"2024-03-06T07:20:05.864042+0000","last_active":"2024-03-06T07:20:07.241052+0000","last_peered":"2024-03-06T07:20:07.241052+0000","last_clean":"2024-03-06T07:20:07.241052+0000","last_became_active":"2024-03-06T07:20:05.863616+0000","last_became_peered":"2024-03-06T07:20:05.863616+0000","last_unstale":"2024-03-06T07:20:07.241052+0000","last_undegraded":"2024-03-06T07:20:07.241052+0000","last_fullsized":"2024-03-06T07:20:07.241052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:12.894304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241818+0000","last_change":"2024-03-06T07:20:01.821307+0000","last_active":"2024-03-06T07:20:07.241818+0000","last_peered":"2024-03-06T07:20:07.241818+0000","last_clean":"2024-03-06T07:20:07.241818+0000","last_became_active":"2024-03-06T07:20:01.821010+0000","last_became_peered":"2024-03-06T07:20:01.821010+0000","last_unstale":"2024-03-06T07:20:07.241818+0000","last_undegraded":"2024-03-06T07:20:07.241818+0000","last_fullsized":"2024-03-06T07:20:07.241818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839526+0000","last_change":"2024-03-06T07:20:00.824564+0000","last_active":"2024-03-06T07:20:06.839526+0000","last_peered":"2024-03-06T07:20:06.839526+0000","last_clean":"2024-03-06T07:20:06.839526+0000","last_became_active":"2024-03-06T07:20:00.824292+0000","last_became_peered":"2024-03-06T07:20:00.824292+0000","last_unstale":"2024-03-06T07:20:06.839526+0000","last_undegraded":"2024-03-06T07:20:06.839526+0000","last_fullsized":"2024-03-06T07:20:06.839526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242082+0000","last_change":"2024-03-06T07:20:06.124710+0000","last_active":"2024-03-06T07:20:07.242082+0000","last_peered":"2024-03-06T07:20:07.242082+0000","last_clean":"2024-03-06T07:20:07.242082+0000","last_became_active":"2024-03-06T07:20:06.124119+0000","last_became_peered":"2024-03-06T07:20:06.124119+0000","last_unstale":"2024-03-06T07:20:07.242082+0000","last_undegraded":"2024-03-06T07:20:07.242082+0000","last_fullsized":"2024-03-06T07:20:07.242082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243393+0000","last_change":"2024-03-06T07:20:01.816965+0000","last_active":"2024-03-06T07:20:07.243393+0000","last_peered":"2024-03-06T07:20:07.243393+0000","last_clean":"2024-03-06T07:20:07.243393+0000","last_became_active":"2024-03-06T07:20:01.816482+0000","last_became_peered":"2024-03-06T07:20:01.816482+0000","last_unstale":"2024-03-06T07:20:07.243393+0000","last_undegraded":"2024-03-06T07:20:07.243393+0000","last_fullsized":"2024-03-06T07:20:07.243393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242030+0000","last_change":"2024-03-06T07:20:00.814404+0000","last_active":"2024-03-06T07:20:07.242030+0000","last_peered":"2024-03-06T07:20:07.242030+0000","last_clean":"2024-03-06T07:20:07.242030+0000","last_became_active":"2024-03-06T07:20:00.813542+0000","last_became_peered":"2024-03-06T07:20:00.813542+0000","last_unstale":"2024-03-06T07:20:07.242030+0000","last_undegraded":"2024-03-06T07:20:07.242030+0000","last_fullsized":"2024-03-06T07:20:07.242030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241898+0000","last_change":"2024-03-06T07:20:06.129464+0000","last_active":"2024-03-06T07:20:07.241898+0000","last_peered":"2024-03-06T07:20:07.241898+0000","last_clean":"2024-03-06T07:20:07.241898+0000","last_became_active":"2024-03-06T07:20:06.128688+0000","last_became_peered":"2024-03-06T07:20:06.128688+0000","last_unstale":"2024-03-06T07:20:07.241898+0000","last_undegraded":"2024-03-06T07:20:07.241898+0000","last_fullsized":"2024-03-06T07:20:07.241898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242308+0000","last_change":"2024-03-06T07:20:01.814773+0000","last_active":"2024-03-06T07:20:07.242308+0000","last_peered":"2024-03-06T07:20:07.242308+0000","last_clean":"2024-03-06T07:20:07.242308+0000","last_became_active":"2024-03-06T07:20:01.814497+0000","last_became_peered":"2024-03-06T07:20:01.814497+0000","last_unstale":"2024-03-06T07:20:07.242308+0000","last_undegraded":"2024-03-06T07:20:07.242308+0000","last_fullsized":"2024-03-06T07:20:07.242308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242368+0000","last_change":"2024-03-06T07:20:00.821645+0000","last_active":"2024-03-06T07:20:07.242368+0000","last_peered":"2024-03-06T07:20:07.242368+0000","last_clean":"2024-03-06T07:20:07.242368+0000","last_became_active":"2024-03-06T07:20:00.821353+0000","last_became_peered":"2024-03-06T07:20:00.821353+0000","last_unstale":"2024-03-06T07:20:07.242368+0000","last_undegraded":"2024-03-06T07:20:07.242368+0000","last_fullsized":"2024-03-06T07:20:07.242368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:02:35.568408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241505+0000","last_change":"2024-03-06T07:20:06.125251+0000","last_active":"2024-03-06T07:20:07.241505+0000","last_peered":"2024-03-06T07:20:07.241505+0000","last_clean":"2024-03-06T07:20:07.241505+0000","last_became_active":"2024-03-06T07:20:06.124788+0000","last_became_peered":"2024-03-06T07:20:06.124788+0000","last_unstale":"2024-03-06T07:20:07.241505+0000","last_undegraded":"2024-03-06T07:20:07.241505+0000","last_fullsized":"2024-03-06T07:20:07.241505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241587+0000","last_change":"2024-03-06T07:20:01.831321+0000","last_active":"2024-03-06T07:20:07.241587+0000","last_peered":"2024-03-06T07:20:07.241587+0000","last_clean":"2024-03-06T07:20:07.241587+0000","last_became_active":"2024-03-06T07:20:01.831041+0000","last_became_peered":"2024-03-06T07:20:01.831041+0000","last_unstale":"2024-03-06T07:20:07.241587+0000","last_undegraded":"2024-03-06T07:20:07.241587+0000","last_fullsized":"2024-03-06T07:20:07.241587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242408+0000","last_change":"2024-03-06T07:20:00.812059+0000","last_active":"2024-03-06T07:20:07.242408+0000","last_peered":"2024-03-06T07:20:07.242408+0000","last_clean":"2024-03-06T07:20:07.242408+0000","last_became_active":"2024-03-06T07:20:00.811207+0000","last_became_peered":"2024-03-06T07:20:00.811207+0000","last_unstale":"2024-03-06T07:20:07.242408+0000","last_undegraded":"2024-03-06T07:20:07.242408+0000","last_fullsized":"2024-03-06T07:20:07.242408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241904+0000","last_change":"2024-03-06T07:20:05.859872+0000","last_active":"2024-03-06T07:20:07.241904+0000","last_peered":"2024-03-06T07:20:07.241904+0000","last_clean":"2024-03-06T07:20:07.241904+0000","last_became_active":"2024-03-06T07:20:05.857727+0000","last_became_peered":"2024-03-06T07:20:05.857727+0000","last_unstale":"2024-03-06T07:20:07.241904+0000","last_undegraded":"2024-03-06T07:20:07.241904+0000","last_fullsized":"2024-03-06T07:20:07.241904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241527+0000","last_change":"2024-03-06T07:20:01.829871+0000","last_active":"2024-03-06T07:20:07.241527+0000","last_peered":"2024-03-06T07:20:07.241527+0000","last_clean":"2024-03-06T07:20:07.241527+0000","last_became_active":"2024-03-06T07:20:01.829526+0000","last_became_peered":"2024-03-06T07:20:01.829526+0000","last_unstale":"2024-03-06T07:20:07.241527+0000","last_undegraded":"2024-03-06T07:20:07.241527+0000","last_fullsized":"2024-03-06T07:20:07.241527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:23.704222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242548+0000","last_change":"2024-03-06T07:20:00.818847+0000","last_active":"2024-03-06T07:20:07.242548+0000","last_peered":"2024-03-06T07:20:07.242548+0000","last_clean":"2024-03-06T07:20:07.242548+0000","last_became_active":"2024-03-06T07:20:00.818547+0000","last_became_peered":"2024-03-06T07:20:00.818547+0000","last_unstale":"2024-03-06T07:20:07.242548+0000","last_undegraded":"2024-03-06T07:20:07.242548+0000","last_fullsized":"2024-03-06T07:20:07.242548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242474+0000","last_change":"2024-03-06T07:20:05.861172+0000","last_active":"2024-03-06T07:20:07.242474+0000","last_peered":"2024-03-06T07:20:07.242474+0000","last_clean":"2024-03-06T07:20:07.242474+0000","last_became_active":"2024-03-06T07:20:05.859989+0000","last_became_peered":"2024-03-06T07:20:05.859989+0000","last_unstale":"2024-03-06T07:20:07.242474+0000","last_undegraded":"2024-03-06T07:20:07.242474+0000","last_fullsized":"2024-03-06T07:20:07.242474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241692+0000","last_change":"2024-03-06T07:20:01.813503+0000","last_active":"2024-03-06T07:20:07.241692+0000","last_peered":"2024-03-06T07:20:07.241692+0000","last_clean":"2024-03-06T07:20:07.241692+0000","last_became_active":"2024-03-06T07:20:01.813227+0000","last_became_peered":"2024-03-06T07:20:01.813227+0000","last_unstale":"2024-03-06T07:20:07.241692+0000","last_undegraded":"2024-03-06T07:20:07.241692+0000","last_fullsized":"2024-03-06T07:20:07.241692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839223+0000","last_change":"2024-03-06T07:20:00.812787+0000","last_active":"2024-03-06T07:20:06.839223+0000","last_peered":"2024-03-06T07:20:06.839223+0000","last_clean":"2024-03-06T07:20:06.839223+0000","last_became_active":"2024-03-06T07:20:00.812505+0000","last_became_peered":"2024-03-06T07:20:00.812505+0000","last_unstale":"2024-03-06T07:20:06.839223+0000","last_undegraded":"2024-03-06T07:20:06.839223+0000","last_fullsized":"2024-03-06T07:20:06.839223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242555+0000","last_change":"2024-03-06T07:20:05.859379+0000","last_active":"2024-03-06T07:20:07.242555+0000","last_peered":"2024-03-06T07:20:07.242555+0000","last_clean":"2024-03-06T07:20:07.242555+0000","last_became_active":"2024-03-06T07:20:05.858723+0000","last_became_peered":"2024-03-06T07:20:05.858723+0000","last_unstale":"2024-03-06T07:20:07.242555+0000","last_undegraded":"2024-03-06T07:20:07.242555+0000","last_fullsized":"2024-03-06T07:20:07.242555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240883+0000","last_change":"2024-03-06T07:20:01.826415+0000","last_active":"2024-03-06T07:20:07.240883+0000","last_peered":"2024-03-06T07:20:07.240883+0000","last_clean":"2024-03-06T07:20:07.240883+0000","last_became_active":"2024-03-06T07:20:01.825703+0000","last_became_peered":"2024-03-06T07:20:01.825703+0000","last_unstale":"2024-03-06T07:20:07.240883+0000","last_undegraded":"2024-03-06T07:20:07.240883+0000","last_fullsized":"2024-03-06T07:20:07.240883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:47.116335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241563+0000","last_change":"2024-03-06T07:20:00.822080+0000","last_active":"2024-03-06T07:20:07.241563+0000","last_peered":"2024-03-06T07:20:07.241563+0000","last_clean":"2024-03-06T07:20:07.241563+0000","last_became_active":"2024-03-06T07:20:00.821813+0000","last_became_peered":"2024-03-06T07:20:00.821813+0000","last_unstale":"2024-03-06T07:20:07.241563+0000","last_undegraded":"2024-03-06T07:20:07.241563+0000","last_fullsized":"2024-03-06T07:20:07.241563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241612+0000","last_change":"2024-03-06T07:20:06.124233+0000","last_active":"2024-03-06T07:20:07.241612+0000","last_peered":"2024-03-06T07:20:07.241612+0000","last_clean":"2024-03-06T07:20:07.241612+0000","last_became_active":"2024-03-06T07:20:06.123834+0000","last_became_peered":"2024-03-06T07:20:06.123834+0000","last_unstale":"2024-03-06T07:20:07.241612+0000","last_undegraded":"2024-03-06T07:20:07.241612+0000","last_fullsized":"2024-03-06T07:20:07.241612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242846+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.242846+0000","last_peered":"2024-03-06T07:20:07.242846+0000","last_clean":"2024-03-06T07:20:07.242846+0000","last_became_active":"2024-03-06T07:20:01.830667+0000","last_became_peered":"2024-03-06T07:20:01.830667+0000","last_unstale":"2024-03-06T07:20:07.242846+0000","last_undegraded":"2024-03-06T07:20:07.242846+0000","last_fullsized":"2024-03-06T07:20:07.242846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241469+0000","last_change":"2024-03-06T07:20:00.814783+0000","last_active":"2024-03-06T07:20:07.241469+0000","last_peered":"2024-03-06T07:20:07.241469+0000","last_clean":"2024-03-06T07:20:07.241469+0000","last_became_active":"2024-03-06T07:20:00.814384+0000","last_became_peered":"2024-03-06T07:20:00.814384+0000","last_unstale":"2024-03-06T07:20:07.241469+0000","last_undegraded":"2024-03-06T07:20:07.241469+0000","last_fullsized":"2024-03-06T07:20:07.241469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241641+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:07.241641+0000","last_peered":"2024-03-06T07:20:07.241641+0000","last_clean":"2024-03-06T07:20:07.241641+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:07.241641+0000","last_undegraded":"2024-03-06T07:20:07.241641+0000","last_fullsized":"2024-03-06T07:20:07.241641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839262+0000","last_change":"2024-03-06T07:20:06.126879+0000","last_active":"2024-03-06T07:20:06.839262+0000","last_peered":"2024-03-06T07:20:06.839262+0000","last_clean":"2024-03-06T07:20:06.839262+0000","last_became_active":"2024-03-06T07:20:06.126475+0000","last_became_peered":"2024-03-06T07:20:06.126475+0000","last_unstale":"2024-03-06T07:20:06.839262+0000","last_undegraded":"2024-03-06T07:20:06.839262+0000","last_fullsized":"2024-03-06T07:20:06.839262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242057+0000","last_change":"2024-03-06T07:20:01.831806+0000","last_active":"2024-03-06T07:20:07.242057+0000","last_peered":"2024-03-06T07:20:07.242057+0000","last_clean":"2024-03-06T07:20:07.242057+0000","last_became_active":"2024-03-06T07:20:01.831545+0000","last_became_peered":"2024-03-06T07:20:01.831545+0000","last_unstale":"2024-03-06T07:20:07.242057+0000","last_undegraded":"2024-03-06T07:20:07.242057+0000","last_fullsized":"2024-03-06T07:20:07.242057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839559+0000","last_change":"2024-03-06T07:20:00.817587+0000","last_active":"2024-03-06T07:20:06.839559+0000","last_peered":"2024-03-06T07:20:06.839559+0000","last_clean":"2024-03-06T07:20:06.839559+0000","last_became_active":"2024-03-06T07:20:00.817212+0000","last_became_peered":"2024-03-06T07:20:00.817212+0000","last_unstale":"2024-03-06T07:20:06.839559+0000","last_undegraded":"2024-03-06T07:20:06.839559+0000","last_fullsized":"2024-03-06T07:20:06.839559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838986+0000","last_change":"2024-03-06T07:20:06.126921+0000","last_active":"2024-03-06T07:20:06.838986+0000","last_peered":"2024-03-06T07:20:06.838986+0000","last_clean":"2024-03-06T07:20:06.838986+0000","last_became_active":"2024-03-06T07:20:06.126385+0000","last_became_peered":"2024-03-06T07:20:06.126385+0000","last_unstale":"2024-03-06T07:20:06.838986+0000","last_undegraded":"2024-03-06T07:20:06.838986+0000","last_fullsized":"2024-03-06T07:20:06.838986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242044+0000","last_change":"2024-03-06T07:20:01.831450+0000","last_active":"2024-03-06T07:20:07.242044+0000","last_peered":"2024-03-06T07:20:07.242044+0000","last_clean":"2024-03-06T07:20:07.242044+0000","last_became_active":"2024-03-06T07:20:01.830525+0000","last_became_peered":"2024-03-06T07:20:01.830525+0000","last_unstale":"2024-03-06T07:20:07.242044+0000","last_undegraded":"2024-03-06T07:20:07.242044+0000","last_fullsized":"2024-03-06T07:20:07.242044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242394+0000","last_change":"2024-03-06T07:20:00.807621+0000","last_active":"2024-03-06T07:20:07.242394+0000","last_peered":"2024-03-06T07:20:07.242394+0000","last_clean":"2024-03-06T07:20:07.242394+0000","last_became_active":"2024-03-06T07:20:00.807340+0000","last_became_peered":"2024-03-06T07:20:00.807340+0000","last_unstale":"2024-03-06T07:20:07.242394+0000","last_undegraded":"2024-03-06T07:20:07.242394+0000","last_fullsized":"2024-03-06T07:20:07.242394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241535+0000","last_change":"2024-03-06T07:20:05.857778+0000","last_active":"2024-03-06T07:20:07.241535+0000","last_peered":"2024-03-06T07:20:07.241535+0000","last_clean":"2024-03-06T07:20:07.241535+0000","last_became_active":"2024-03-06T07:20:05.857279+0000","last_became_peered":"2024-03-06T07:20:05.857279+0000","last_unstale":"2024-03-06T07:20:07.241535+0000","last_undegraded":"2024-03-06T07:20:07.241535+0000","last_fullsized":"2024-03-06T07:20:07.241535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242638+0000","last_change":"2024-03-06T07:20:01.819993+0000","last_active":"2024-03-06T07:20:07.242638+0000","last_peered":"2024-03-06T07:20:07.242638+0000","last_clean":"2024-03-06T07:20:07.242638+0000","last_became_active":"2024-03-06T07:20:01.819691+0000","last_became_peered":"2024-03-06T07:20:01.819691+0000","last_unstale":"2024-03-06T07:20:07.242638+0000","last_undegraded":"2024-03-06T07:20:07.242638+0000","last_fullsized":"2024-03-06T07:20:07.242638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:19.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839555+0000","last_change":"2024-03-06T07:20:00.819667+0000","last_active":"2024-03-06T07:20:06.839555+0000","last_peered":"2024-03-06T07:20:06.839555+0000","last_clean":"2024-03-06T07:20:06.839555+0000","last_became_active":"2024-03-06T07:20:00.819371+0000","last_became_peered":"2024-03-06T07:20:00.819371+0000","last_unstale":"2024-03-06T07:20:06.839555+0000","last_undegraded":"2024-03-06T07:20:06.839555+0000","last_fullsized":"2024-03-06T07:20:06.839555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241355+0000","last_change":"2024-03-06T07:20:06.124150+0000","last_active":"2024-03-06T07:20:07.241355+0000","last_peered":"2024-03-06T07:20:07.241355+0000","last_clean":"2024-03-06T07:20:07.241355+0000","last_became_active":"2024-03-06T07:20:06.123794+0000","last_became_peered":"2024-03-06T07:20:06.123794+0000","last_unstale":"2024-03-06T07:20:07.241355+0000","last_undegraded":"2024-03-06T07:20:07.241355+0000","last_fullsized":"2024-03-06T07:20:07.241355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242273+0000","last_change":"2024-03-06T07:20:01.814512+0000","last_active":"2024-03-06T07:20:07.242273+0000","last_peered":"2024-03-06T07:20:07.242273+0000","last_clean":"2024-03-06T07:20:07.242273+0000","last_became_active":"2024-03-06T07:20:01.813177+0000","last_became_peered":"2024-03-06T07:20:01.813177+0000","last_unstale":"2024-03-06T07:20:07.242273+0000","last_undegraded":"2024-03-06T07:20:07.242273+0000","last_fullsized":"2024-03-06T07:20:07.242273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:05.673072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241319+0000","last_change":"2024-03-06T07:20:00.812789+0000","last_active":"2024-03-06T07:20:07.241319+0000","last_peered":"2024-03-06T07:20:07.241319+0000","last_clean":"2024-03-06T07:20:07.241319+0000","last_became_active":"2024-03-06T07:20:00.812391+0000","last_became_peered":"2024-03-06T07:20:00.812391+0000","last_unstale":"2024-03-06T07:20:07.241319+0000","last_undegraded":"2024-03-06T07:20:07.241319+0000","last_fullsized":"2024-03-06T07:20:07.241319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:54:26.011476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240875+0000","last_change":"2024-03-06T07:20:05.863928+0000","last_active":"2024-03-06T07:20:07.240875+0000","last_peered":"2024-03-06T07:20:07.240875+0000","last_clean":"2024-03-06T07:20:07.240875+0000","last_became_active":"2024-03-06T07:20:05.863555+0000","last_became_peered":"2024-03-06T07:20:05.863555+0000","last_unstale":"2024-03-06T07:20:07.240875+0000","last_undegraded":"2024-03-06T07:20:07.240875+0000","last_fullsized":"2024-03-06T07:20:07.240875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:53.757250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242121+0000","last_change":"2024-03-06T07:20:01.811116+0000","last_active":"2024-03-06T07:20:07.242121+0000","last_peered":"2024-03-06T07:20:07.242121+0000","last_clean":"2024-03-06T07:20:07.242121+0000","last_became_active":"2024-03-06T07:20:01.810828+0000","last_became_peered":"2024-03-06T07:20:01.810828+0000","last_unstale":"2024-03-06T07:20:07.242121+0000","last_undegraded":"2024-03-06T07:20:07.242121+0000","last_fullsized":"2024-03-06T07:20:07.242121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838933+0000","last_change":"2024-03-06T07:20:00.807531+0000","last_active":"2024-03-06T07:20:06.838933+0000","last_peered":"2024-03-06T07:20:06.838933+0000","last_clean":"2024-03-06T07:20:06.838933+0000","last_became_active":"2024-03-06T07:20:00.807208+0000","last_became_peered":"2024-03-06T07:20:00.807208+0000","last_unstale":"2024-03-06T07:20:06.838933+0000","last_undegraded":"2024-03-06T07:20:06.838933+0000","last_fullsized":"2024-03-06T07:20:06.838933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:14.638569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:05.853364+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:05.852877+0000","last_became_peered":"2024-03-06T07:20:05.852877+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838808+0000","last_change":"2024-03-06T07:20:01.824711+0000","last_active":"2024-03-06T07:20:06.838808+0000","last_peered":"2024-03-06T07:20:06.838808+0000","last_clean":"2024-03-06T07:20:06.838808+0000","last_became_active":"2024-03-06T07:20:01.824420+0000","last_became_peered":"2024-03-06T07:20:01.824420+0000","last_unstale":"2024-03-06T07:20:06.838808+0000","last_undegraded":"2024-03-06T07:20:06.838808+0000","last_fullsized":"2024-03-06T07:20:06.838808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838852+0000","last_change":"2024-03-06T07:20:00.816624+0000","last_active":"2024-03-06T07:20:06.838852+0000","last_peered":"2024-03-06T07:20:06.838852+0000","last_clean":"2024-03-06T07:20:06.838852+0000","last_became_active":"2024-03-06T07:20:00.816335+0000","last_became_peered":"2024-03-06T07:20:00.816335+0000","last_unstale":"2024-03-06T07:20:06.838852+0000","last_undegraded":"2024-03-06T07:20:06.838852+0000","last_fullsized":"2024-03-06T07:20:06.838852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243303+0000","last_change":"2024-03-06T07:20:06.124178+0000","last_active":"2024-03-06T07:20:07.243303+0000","last_peered":"2024-03-06T07:20:07.243303+0000","last_clean":"2024-03-06T07:20:07.243303+0000","last_became_active":"2024-03-06T07:20:06.123788+0000","last_became_peered":"2024-03-06T07:20:06.123788+0000","last_unstale":"2024-03-06T07:20:07.243303+0000","last_undegraded":"2024-03-06T07:20:07.243303+0000","last_fullsized":"2024-03-06T07:20:07.243303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:12.104491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241688+0000","last_change":"2024-03-06T07:20:01.817183+0000","last_active":"2024-03-06T07:20:07.241688+0000","last_peered":"2024-03-06T07:20:07.241688+0000","last_clean":"2024-03-06T07:20:07.241688+0000","last_became_active":"2024-03-06T07:20:01.816230+0000","last_became_peered":"2024-03-06T07:20:01.816230+0000","last_unstale":"2024-03-06T07:20:07.241688+0000","last_undegraded":"2024-03-06T07:20:07.241688+0000","last_fullsized":"2024-03-06T07:20:07.241688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242486+0000","last_change":"2024-03-06T07:20:00.817367+0000","last_active":"2024-03-06T07:20:07.242486+0000","last_peered":"2024-03-06T07:20:07.242486+0000","last_clean":"2024-03-06T07:20:07.242486+0000","last_became_active":"2024-03-06T07:20:00.816410+0000","last_became_peered":"2024-03-06T07:20:00.816410+0000","last_unstale":"2024-03-06T07:20:07.242486+0000","last_undegraded":"2024-03-06T07:20:07.242486+0000","last_fullsized":"2024-03-06T07:20:07.242486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243226+0000","last_change":"2024-03-06T07:20:06.126465+0000","last_active":"2024-03-06T07:20:07.243226+0000","last_peered":"2024-03-06T07:20:07.243226+0000","last_clean":"2024-03-06T07:20:07.243226+0000","last_became_active":"2024-03-06T07:20:06.126107+0000","last_became_peered":"2024-03-06T07:20:06.126107+0000","last_unstale":"2024-03-06T07:20:07.243226+0000","last_undegraded":"2024-03-06T07:20:07.243226+0000","last_fullsized":"2024-03-06T07:20:07.243226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:37.677442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241274+0000","last_change":"2024-03-06T07:20:01.822827+0000","last_active":"2024-03-06T07:20:07.241274+0000","last_peered":"2024-03-06T07:20:07.241274+0000","last_clean":"2024-03-06T07:20:07.241274+0000","last_became_active":"2024-03-06T07:20:01.822490+0000","last_became_peered":"2024-03-06T07:20:01.822490+0000","last_unstale":"2024-03-06T07:20:07.241274+0000","last_undegraded":"2024-03-06T07:20:07.241274+0000","last_fullsized":"2024-03-06T07:20:07.241274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241706+0000","last_change":"2024-03-06T07:20:00.811943+0000","last_active":"2024-03-06T07:20:07.241706+0000","last_peered":"2024-03-06T07:20:07.241706+0000","last_clean":"2024-03-06T07:20:07.241706+0000","last_became_active":"2024-03-06T07:20:00.811499+0000","last_became_peered":"2024-03-06T07:20:00.811499+0000","last_unstale":"2024-03-06T07:20:07.241706+0000","last_undegraded":"2024-03-06T07:20:07.241706+0000","last_fullsized":"2024-03-06T07:20:07.241706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241416+0000","last_change":"2024-03-06T07:20:06.127004+0000","last_active":"2024-03-06T07:20:07.241416+0000","last_peered":"2024-03-06T07:20:07.241416+0000","last_clean":"2024-03-06T07:20:07.241416+0000","last_became_active":"2024-03-06T07:20:06.126643+0000","last_became_peered":"2024-03-06T07:20:06.126643+0000","last_unstale":"2024-03-06T07:20:07.241416+0000","last_undegraded":"2024-03-06T07:20:07.241416+0000","last_fullsized":"2024-03-06T07:20:07.241416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.815676+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.815238+0000","last_became_peered":"2024-03-06T07:20:01.815238+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.693301+0000","last_change":"2024-03-06T07:20:00.818890+0000","last_active":"2024-03-06T07:20:10.693301+0000","last_peered":"2024-03-06T07:20:10.693301+0000","last_clean":"2024-03-06T07:20:10.693301+0000","last_became_active":"2024-03-06T07:20:00.818613+0000","last_became_peered":"2024-03-06T07:20:00.818613+0000","last_unstale":"2024-03-06T07:20:10.693301+0000","last_undegraded":"2024-03-06T07:20:10.693301+0000","last_fullsized":"2024-03-06T07:20:10.693301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242003+0000","last_change":"2024-03-06T07:20:05.862991+0000","last_active":"2024-03-06T07:20:07.242003+0000","last_peered":"2024-03-06T07:20:07.242003+0000","last_clean":"2024-03-06T07:20:07.242003+0000","last_became_active":"2024-03-06T07:20:05.862334+0000","last_became_peered":"2024-03-06T07:20:05.862334+0000","last_unstale":"2024-03-06T07:20:07.242003+0000","last_undegraded":"2024-03-06T07:20:07.242003+0000","last_fullsized":"2024-03-06T07:20:07.242003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241685+0000","last_change":"2024-03-06T07:20:01.815200+0000","last_active":"2024-03-06T07:20:07.241685+0000","last_peered":"2024-03-06T07:20:07.241685+0000","last_clean":"2024-03-06T07:20:07.241685+0000","last_became_active":"2024-03-06T07:20:01.814903+0000","last_became_peered":"2024-03-06T07:20:01.814903+0000","last_unstale":"2024-03-06T07:20:07.241685+0000","last_undegraded":"2024-03-06T07:20:07.241685+0000","last_fullsized":"2024-03-06T07:20:07.241685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242321+0000","last_change":"2024-03-06T07:20:00.822249+0000","last_active":"2024-03-06T07:20:07.242321+0000","last_peered":"2024-03-06T07:20:07.242321+0000","last_clean":"2024-03-06T07:20:07.242321+0000","last_became_active":"2024-03-06T07:20:00.821965+0000","last_became_peered":"2024-03-06T07:20:00.821965+0000","last_unstale":"2024-03-06T07:20:07.242321+0000","last_undegraded":"2024-03-06T07:20:07.242321+0000","last_fullsized":"2024-03-06T07:20:07.242321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241085+0000","last_change":"2024-03-06T07:20:05.862268+0000","last_active":"2024-03-06T07:20:07.241085+0000","last_peered":"2024-03-06T07:20:07.241085+0000","last_clean":"2024-03-06T07:20:07.241085+0000","last_became_active":"2024-03-06T07:20:05.861794+0000","last_became_peered":"2024-03-06T07:20:05.861794+0000","last_unstale":"2024-03-06T07:20:07.241085+0000","last_undegraded":"2024-03-06T07:20:07.241085+0000","last_fullsized":"2024-03-06T07:20:07.241085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:15.471029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241294+0000","last_change":"2024-03-06T07:20:01.835260+0000","last_active":"2024-03-06T07:20:07.241294+0000","last_peered":"2024-03-06T07:20:07.241294+0000","last_clean":"2024-03-06T07:20:07.241294+0000","last_became_active":"2024-03-06T07:20:01.835069+0000","last_became_peered":"2024-03-06T07:20:01.835069+0000","last_unstale":"2024-03-06T07:20:07.241294+0000","last_undegraded":"2024-03-06T07:20:07.241294+0000","last_fullsized":"2024-03-06T07:20:07.241294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241087+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241087+0000","last_peered":"2024-03-06T07:20:07.241087+0000","last_clean":"2024-03-06T07:20:07.241087+0000","last_became_active":"2024-03-06T07:20:00.822307+0000","last_became_peered":"2024-03-06T07:20:00.822307+0000","last_unstale":"2024-03-06T07:20:07.241087+0000","last_undegraded":"2024-03-06T07:20:07.241087+0000","last_fullsized":"2024-03-06T07:20:07.241087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242067+0000","last_change":"2024-03-06T07:20:05.857403+0000","last_active":"2024-03-06T07:20:07.242067+0000","last_peered":"2024-03-06T07:20:07.242067+0000","last_clean":"2024-03-06T07:20:07.242067+0000","last_became_active":"2024-03-06T07:20:05.857036+0000","last_became_peered":"2024-03-06T07:20:05.857036+0000","last_unstale":"2024-03-06T07:20:07.242067+0000","last_undegraded":"2024-03-06T07:20:07.242067+0000","last_fullsized":"2024-03-06T07:20:07.242067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:23.280198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241050+0000","last_change":"2024-03-06T07:20:01.833389+0000","last_active":"2024-03-06T07:20:07.241050+0000","last_peered":"2024-03-06T07:20:07.241050+0000","last_clean":"2024-03-06T07:20:07.241050+0000","last_became_active":"2024-03-06T07:20:01.833094+0000","last_became_peered":"2024-03-06T07:20:01.833094+0000","last_unstale":"2024-03-06T07:20:07.241050+0000","last_undegraded":"2024-03-06T07:20:07.241050+0000","last_fullsized":"2024-03-06T07:20:07.241050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839444+0000","last_change":"2024-03-06T07:20:00.817766+0000","last_active":"2024-03-06T07:20:06.839444+0000","last_peered":"2024-03-06T07:20:06.839444+0000","last_clean":"2024-03-06T07:20:06.839444+0000","last_became_active":"2024-03-06T07:20:00.817466+0000","last_became_peered":"2024-03-06T07:20:00.817466+0000","last_unstale":"2024-03-06T07:20:06.839444+0000","last_undegraded":"2024-03-06T07:20:06.839444+0000","last_fullsized":"2024-03-06T07:20:06.839444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242588+0000","last_change":"2024-03-06T07:20:06.127038+0000","last_active":"2024-03-06T07:20:07.242588+0000","last_peered":"2024-03-06T07:20:07.242588+0000","last_clean":"2024-03-06T07:20:07.242588+0000","last_became_active":"2024-03-06T07:20:06.126185+0000","last_became_peered":"2024-03-06T07:20:06.126185+0000","last_unstale":"2024-03-06T07:20:07.242588+0000","last_undegraded":"2024-03-06T07:20:07.242588+0000","last_fullsized":"2024-03-06T07:20:07.242588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241025+0000","last_change":"2024-03-06T07:20:06.126567+0000","last_active":"2024-03-06T07:20:07.241025+0000","last_peered":"2024-03-06T07:20:07.241025+0000","last_clean":"2024-03-06T07:20:07.241025+0000","last_became_active":"2024-03-06T07:20:06.126198+0000","last_became_peered":"2024-03-06T07:20:06.126198+0000","last_unstale":"2024-03-06T07:20:07.241025+0000","last_undegraded":"2024-03-06T07:20:07.241025+0000","last_fullsized":"2024-03-06T07:20:07.241025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240998+0000","last_change":"2024-03-06T07:20:01.817263+0000","last_active":"2024-03-06T07:20:07.240998+0000","last_peered":"2024-03-06T07:20:07.240998+0000","last_clean":"2024-03-06T07:20:07.240998+0000","last_became_active":"2024-03-06T07:20:01.816994+0000","last_became_peered":"2024-03-06T07:20:01.816994+0000","last_unstale":"2024-03-06T07:20:07.240998+0000","last_undegraded":"2024-03-06T07:20:07.240998+0000","last_fullsized":"2024-03-06T07:20:07.240998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241203+0000","last_change":"2024-03-06T07:20:00.814224+0000","last_active":"2024-03-06T07:20:07.241203+0000","last_peered":"2024-03-06T07:20:07.241203+0000","last_clean":"2024-03-06T07:20:07.241203+0000","last_became_active":"2024-03-06T07:20:00.813908+0000","last_became_peered":"2024-03-06T07:20:00.813908+0000","last_unstale":"2024-03-06T07:20:07.241203+0000","last_undegraded":"2024-03-06T07:20:07.241203+0000","last_fullsized":"2024-03-06T07:20:07.241203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839419+0000","last_change":"2024-03-06T07:20:05.863149+0000","last_active":"2024-03-06T07:20:06.839419+0000","last_peered":"2024-03-06T07:20:06.839419+0000","last_clean":"2024-03-06T07:20:06.839419+0000","last_became_active":"2024-03-06T07:20:05.862680+0000","last_became_peered":"2024-03-06T07:20:05.862680+0000","last_unstale":"2024-03-06T07:20:06.839419+0000","last_undegraded":"2024-03-06T07:20:06.839419+0000","last_fullsized":"2024-03-06T07:20:06.839419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242085+0000","last_change":"2024-03-06T07:20:01.822303+0000","last_active":"2024-03-06T07:20:07.242085+0000","last_peered":"2024-03-06T07:20:07.242085+0000","last_clean":"2024-03-06T07:20:07.242085+0000","last_became_active":"2024-03-06T07:20:01.822044+0000","last_became_peered":"2024-03-06T07:20:01.822044+0000","last_unstale":"2024-03-06T07:20:07.242085+0000","last_undegraded":"2024-03-06T07:20:07.242085+0000","last_fullsized":"2024-03-06T07:20:07.242085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242899+0000","last_change":"2024-03-06T07:20:00.819504+0000","last_active":"2024-03-06T07:20:07.242899+0000","last_peered":"2024-03-06T07:20:07.242899+0000","last_clean":"2024-03-06T07:20:07.242899+0000","last_became_active":"2024-03-06T07:20:00.819223+0000","last_became_peered":"2024-03-06T07:20:00.819223+0000","last_unstale":"2024-03-06T07:20:07.242899+0000","last_undegraded":"2024-03-06T07:20:07.242899+0000","last_fullsized":"2024-03-06T07:20:07.242899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839079+0000","last_change":"2024-03-06T07:20:00.816629+0000","last_active":"2024-03-06T07:20:06.839079+0000","last_peered":"2024-03-06T07:20:06.839079+0000","last_clean":"2024-03-06T07:20:06.839079+0000","last_became_active":"2024-03-06T07:20:00.816373+0000","last_became_peered":"2024-03-06T07:20:00.816373+0000","last_unstale":"2024-03-06T07:20:06.839079+0000","last_undegraded":"2024-03-06T07:20:06.839079+0000","last_fullsized":"2024-03-06T07:20:06.839079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241631+0000","last_change":"2024-03-06T07:20:01.829181+0000","last_active":"2024-03-06T07:20:07.241631+0000","last_peered":"2024-03-06T07:20:07.241631+0000","last_clean":"2024-03-06T07:20:07.241631+0000","last_became_active":"2024-03-06T07:20:01.827761+0000","last_became_peered":"2024-03-06T07:20:01.827761+0000","last_unstale":"2024-03-06T07:20:07.241631+0000","last_undegraded":"2024-03-06T07:20:07.241631+0000","last_fullsized":"2024-03-06T07:20:07.241631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839350+0000","last_change":"2024-03-06T07:20:05.857388+0000","last_active":"2024-03-06T07:20:06.839350+0000","last_peered":"2024-03-06T07:20:06.839350+0000","last_clean":"2024-03-06T07:20:06.839350+0000","last_became_active":"2024-03-06T07:20:05.856981+0000","last_became_peered":"2024-03-06T07:20:05.856981+0000","last_unstale":"2024-03-06T07:20:06.839350+0000","last_undegraded":"2024-03-06T07:20:06.839350+0000","last_fullsized":"2024-03-06T07:20:06.839350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:51.153823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243225+0000","last_change":"2024-03-06T07:20:00.822065+0000","last_active":"2024-03-06T07:20:07.243225+0000","last_peered":"2024-03-06T07:20:07.243225+0000","last_clean":"2024-03-06T07:20:07.243225+0000","last_became_active":"2024-03-06T07:20:00.821736+0000","last_became_peered":"2024-03-06T07:20:00.821736+0000","last_unstale":"2024-03-06T07:20:07.243225+0000","last_undegraded":"2024-03-06T07:20:07.243225+0000","last_fullsized":"2024-03-06T07:20:07.243225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241924+0000","last_change":"2024-03-06T07:20:01.819959+0000","last_active":"2024-03-06T07:20:07.241924+0000","last_peered":"2024-03-06T07:20:07.241924+0000","last_clean":"2024-03-06T07:20:07.241924+0000","last_became_active":"2024-03-06T07:20:01.819548+0000","last_became_peered":"2024-03-06T07:20:01.819548+0000","last_unstale":"2024-03-06T07:20:07.241924+0000","last_undegraded":"2024-03-06T07:20:07.241924+0000","last_fullsized":"2024-03-06T07:20:07.241924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:40.273910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243065+0000","last_change":"2024-03-06T07:20:05.864320+0000","last_active":"2024-03-06T07:20:07.243065+0000","last_peered":"2024-03-06T07:20:07.243065+0000","last_clean":"2024-03-06T07:20:07.243065+0000","last_became_active":"2024-03-06T07:20:05.863902+0000","last_became_peered":"2024-03-06T07:20:05.863902+0000","last_unstale":"2024-03-06T07:20:07.243065+0000","last_undegraded":"2024-03-06T07:20:07.243065+0000","last_fullsized":"2024-03-06T07:20:07.243065+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838709+0000","last_change":"2024-03-06T07:20:01.824719+0000","last_active":"2024-03-06T07:20:06.838709+0000","last_peered":"2024-03-06T07:20:06.838709+0000","last_clean":"2024-03-06T07:20:06.838709+0000","last_became_active":"2024-03-06T07:20:01.824424+0000","last_became_peered":"2024-03-06T07:20:01.824424+0000","last_unstale":"2024-03-06T07:20:06.838709+0000","last_undegraded":"2024-03-06T07:20:06.838709+0000","last_fullsized":"2024-03-06T07:20:06.838709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839111+0000","last_change":"2024-03-06T07:20:01.827824+0000","last_active":"2024-03-06T07:20:06.839111+0000","last_peered":"2024-03-06T07:20:06.839111+0000","last_clean":"2024-03-06T07:20:06.839111+0000","last_became_active":"2024-03-06T07:20:01.827552+0000","last_became_peered":"2024-03-06T07:20:01.827552+0000","last_unstale":"2024-03-06T07:20:06.839111+0000","last_undegraded":"2024-03-06T07:20:06.839111+0000","last_fullsized":"2024-03-06T07:20:06.839111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:24.188510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242476+0000","last_change":"2024-03-06T07:20:01.827949+0000","last_active":"2024-03-06T07:20:07.242476+0000","last_peered":"2024-03-06T07:20:07.242476+0000","last_clean":"2024-03-06T07:20:07.242476+0000","last_became_active":"2024-03-06T07:20:01.827535+0000","last_became_peered":"2024-03-06T07:20:01.827535+0000","last_unstale":"2024-03-06T07:20:07.242476+0000","last_undegraded":"2024-03-06T07:20:07.242476+0000","last_fullsized":"2024-03-06T07:20:07.242476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241603+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.241603+0000","last_peered":"2024-03-06T07:20:07.241603+0000","last_clean":"2024-03-06T07:20:07.241603+0000","last_became_active":"2024-03-06T07:20:01.828105+0000","last_became_peered":"2024-03-06T07:20:01.828105+0000","last_unstale":"2024-03-06T07:20:07.241603+0000","last_undegraded":"2024-03-06T07:20:07.241603+0000","last_fullsized":"2024-03-06T07:20:07.241603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243331+0000","last_change":"2024-03-06T07:20:01.810561+0000","last_active":"2024-03-06T07:20:07.243331+0000","last_peered":"2024-03-06T07:20:07.243331+0000","last_clean":"2024-03-06T07:20:07.243331+0000","last_became_active":"2024-03-06T07:20:01.810258+0000","last_became_peered":"2024-03-06T07:20:01.810258+0000","last_unstale":"2024-03-06T07:20:07.243331+0000","last_undegraded":"2024-03-06T07:20:07.243331+0000","last_fullsized":"2024-03-06T07:20:07.243331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242209+0000","last_change":"2024-03-06T07:20:01.812180+0000","last_active":"2024-03-06T07:20:07.242209+0000","last_peered":"2024-03-06T07:20:07.242209+0000","last_clean":"2024-03-06T07:20:07.242209+0000","last_became_active":"2024-03-06T07:20:01.811847+0000","last_became_peered":"2024-03-06T07:20:01.811847+0000","last_unstale":"2024-03-06T07:20:07.242209+0000","last_undegraded":"2024-03-06T07:20:07.242209+0000","last_fullsized":"2024-03-06T07:20:07.242209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839401+0000","last_change":"2024-03-06T07:20:01.821989+0000","last_active":"2024-03-06T07:20:06.839401+0000","last_peered":"2024-03-06T07:20:06.839401+0000","last_clean":"2024-03-06T07:20:06.839401+0000","last_became_active":"2024-03-06T07:20:01.821682+0000","last_became_peered":"2024-03-06T07:20:01.821682+0000","last_unstale":"2024-03-06T07:20:06.839401+0000","last_undegraded":"2024-03-06T07:20:06.839401+0000","last_fullsized":"2024-03-06T07:20:06.839401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243323+0000","last_change":"2024-03-06T07:20:01.815852+0000","last_active":"2024-03-06T07:20:07.243323+0000","last_peered":"2024-03-06T07:20:07.243323+0000","last_clean":"2024-03-06T07:20:07.243323+0000","last_became_active":"2024-03-06T07:20:01.815523+0000","last_became_peered":"2024-03-06T07:20:01.815523+0000","last_unstale":"2024-03-06T07:20:07.243323+0000","last_undegraded":"2024-03-06T07:20:07.243323+0000","last_fullsized":"2024-03-06T07:20:07.243323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241111+0000","last_change":"2024-03-06T07:20:01.828459+0000","last_active":"2024-03-06T07:20:07.241111+0000","last_peered":"2024-03-06T07:20:07.241111+0000","last_clean":"2024-03-06T07:20:07.241111+0000","last_became_active":"2024-03-06T07:20:01.828110+0000","last_became_peered":"2024-03-06T07:20:01.828110+0000","last_unstale":"2024-03-06T07:20:07.241111+0000","last_undegraded":"2024-03-06T07:20:07.241111+0000","last_fullsized":"2024-03-06T07:20:07.241111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242736+0000","last_change":"2024-03-06T07:20:01.810465+0000","last_active":"2024-03-06T07:20:07.242736+0000","last_peered":"2024-03-06T07:20:07.242736+0000","last_clean":"2024-03-06T07:20:07.242736+0000","last_became_active":"2024-03-06T07:20:01.810206+0000","last_became_peered":"2024-03-06T07:20:01.810206+0000","last_unstale":"2024-03-06T07:20:07.242736+0000","last_undegraded":"2024-03-06T07:20:07.242736+0000","last_fullsized":"2024-03-06T07:20:07.242736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242393+0000","last_change":"2024-03-06T07:20:01.834744+0000","last_active":"2024-03-06T07:20:07.242393+0000","last_peered":"2024-03-06T07:20:07.242393+0000","last_clean":"2024-03-06T07:20:07.242393+0000","last_became_active":"2024-03-06T07:20:01.834049+0000","last_became_peered":"2024-03-06T07:20:01.834049+0000","last_unstale":"2024-03-06T07:20:07.242393+0000","last_undegraded":"2024-03-06T07:20:07.242393+0000","last_fullsized":"2024-03-06T07:20:07.242393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241262+0000","last_change":"2024-03-06T07:20:01.830008+0000","last_active":"2024-03-06T07:20:07.241262+0000","last_peered":"2024-03-06T07:20:07.241262+0000","last_clean":"2024-03-06T07:20:07.241262+0000","last_became_active":"2024-03-06T07:20:01.828839+0000","last_became_peered":"2024-03-06T07:20:01.828839+0000","last_unstale":"2024-03-06T07:20:07.241262+0000","last_undegraded":"2024-03-06T07:20:07.241262+0000","last_fullsized":"2024-03-06T07:20:07.241262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839342+0000","last_change":"2024-03-06T07:20:01.815286+0000","last_active":"2024-03-06T07:20:06.839342+0000","last_peered":"2024-03-06T07:20:06.839342+0000","last_clean":"2024-03-06T07:20:06.839342+0000","last_became_active":"2024-03-06T07:20:01.814983+0000","last_became_peered":"2024-03-06T07:20:01.814983+0000","last_unstale":"2024-03-06T07:20:06.839342+0000","last_undegraded":"2024-03-06T07:20:06.839342+0000","last_fullsized":"2024-03-06T07:20:06.839342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242294+0000","last_change":"2024-03-06T07:20:01.816404+0000","last_active":"2024-03-06T07:20:07.242294+0000","last_peered":"2024-03-06T07:20:07.242294+0000","last_clean":"2024-03-06T07:20:07.242294+0000","last_became_active":"2024-03-06T07:20:01.815092+0000","last_became_peered":"2024-03-06T07:20:01.815092+0000","last_unstale":"2024-03-06T07:20:07.242294+0000","last_undegraded":"2024-03-06T07:20:07.242294+0000","last_fullsized":"2024-03-06T07:20:07.242294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839438+0000","last_change":"2024-03-06T07:20:01.819861+0000","last_active":"2024-03-06T07:20:06.839438+0000","last_peered":"2024-03-06T07:20:06.839438+0000","last_clean":"2024-03-06T07:20:06.839438+0000","last_became_active":"2024-03-06T07:20:01.819090+0000","last_became_peered":"2024-03-06T07:20:01.819090+0000","last_unstale":"2024-03-06T07:20:06.839438+0000","last_undegraded":"2024-03-06T07:20:06.839438+0000","last_fullsized":"2024-03-06T07:20:06.839438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.834134+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.833295+0000","last_became_peered":"2024-03-06T07:20:01.833295+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241004+0000","last_change":"2024-03-06T07:20:01.834489+0000","last_active":"2024-03-06T07:20:07.241004+0000","last_peered":"2024-03-06T07:20:07.241004+0000","last_clean":"2024-03-06T07:20:07.241004+0000","last_became_active":"2024-03-06T07:20:01.833824+0000","last_became_peered":"2024-03-06T07:20:01.833824+0000","last_unstale":"2024-03-06T07:20:07.241004+0000","last_undegraded":"2024-03-06T07:20:07.241004+0000","last_fullsized":"2024-03-06T07:20:07.241004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241109+0000","last_change":"2024-03-06T07:20:01.818055+0000","last_active":"2024-03-06T07:20:07.241109+0000","last_peered":"2024-03-06T07:20:07.241109+0000","last_clean":"2024-03-06T07:20:07.241109+0000","last_became_active":"2024-03-06T07:20:01.817719+0000","last_became_peered":"2024-03-06T07:20:01.817719+0000","last_unstale":"2024-03-06T07:20:07.241109+0000","last_undegraded":"2024-03-06T07:20:07.241109+0000","last_fullsized":"2024-03-06T07:20:07.241109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241162+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241162+0000","last_peered":"2024-03-06T07:20:07.241162+0000","last_clean":"2024-03-06T07:20:07.241162+0000","last_became_active":"2024-03-06T07:20:01.825795+0000","last_became_peered":"2024-03-06T07:20:01.825795+0000","last_unstale":"2024-03-06T07:20:07.241162+0000","last_undegraded":"2024-03-06T07:20:07.241162+0000","last_fullsized":"2024-03-06T07:20:07.241162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243013+0000","last_change":"2024-03-06T07:20:01.818654+0000","last_active":"2024-03-06T07:20:07.243013+0000","last_peered":"2024-03-06T07:20:07.243013+0000","last_clean":"2024-03-06T07:20:07.243013+0000","last_became_active":"2024-03-06T07:20:01.818368+0000","last_became_peered":"2024-03-06T07:20:01.818368+0000","last_unstale":"2024-03-06T07:20:07.243013+0000","last_undegraded":"2024-03-06T07:20:07.243013+0000","last_fullsized":"2024-03-06T07:20:07.243013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.817094+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.816841+0000","last_became_peered":"2024-03-06T07:20:01.816841+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:26.548988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839291+0000","last_change":"2024-03-06T07:20:01.831471+0000","last_active":"2024-03-06T07:20:06.839291+0000","last_peered":"2024-03-06T07:20:06.839291+0000","last_clean":"2024-03-06T07:20:06.839291+0000","last_became_active":"2024-03-06T07:20:01.831177+0000","last_became_peered":"2024-03-06T07:20:01.831177+0000","last_unstale":"2024-03-06T07:20:06.839291+0000","last_undegraded":"2024-03-06T07:20:06.839291+0000","last_fullsized":"2024-03-06T07:20:06.839291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839377+0000","last_change":"2024-03-06T07:20:01.831736+0000","last_active":"2024-03-06T07:20:06.839377+0000","last_peered":"2024-03-06T07:20:06.839377+0000","last_clean":"2024-03-06T07:20:06.839377+0000","last_became_active":"2024-03-06T07:20:01.831450+0000","last_became_peered":"2024-03-06T07:20:01.831450+0000","last_unstale":"2024-03-06T07:20:06.839377+0000","last_undegraded":"2024-03-06T07:20:06.839377+0000","last_fullsized":"2024-03-06T07:20:06.839377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242607+0000","last_change":"2024-03-06T07:20:01.832323+0000","last_active":"2024-03-06T07:20:07.242607+0000","last_peered":"2024-03-06T07:20:07.242607+0000","last_clean":"2024-03-06T07:20:07.242607+0000","last_became_active":"2024-03-06T07:20:01.832078+0000","last_became_peered":"2024-03-06T07:20:01.832078+0000","last_unstale":"2024-03-06T07:20:07.242607+0000","last_undegraded":"2024-03-06T07:20:07.242607+0000","last_fullsized":"2024-03-06T07:20:07.242607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241991+0000","last_change":"2024-03-06T07:20:01.831803+0000","last_active":"2024-03-06T07:20:07.241991+0000","last_peered":"2024-03-06T07:20:07.241991+0000","last_clean":"2024-03-06T07:20:07.241991+0000","last_became_active":"2024-03-06T07:20:01.831509+0000","last_became_peered":"2024-03-06T07:20:01.831509+0000","last_unstale":"2024-03-06T07:20:07.241991+0000","last_undegraded":"2024-03-06T07:20:07.241991+0000","last_fullsized":"2024-03-06T07:20:07.241991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242029+0000","last_change":"2024-03-06T07:20:01.828627+0000","last_active":"2024-03-06T07:20:07.242029+0000","last_peered":"2024-03-06T07:20:07.242029+0000","last_clean":"2024-03-06T07:20:07.242029+0000","last_became_active":"2024-03-06T07:20:01.828182+0000","last_became_peered":"2024-03-06T07:20:01.828182+0000","last_unstale":"2024-03-06T07:20:07.242029+0000","last_undegraded":"2024-03-06T07:20:07.242029+0000","last_fullsized":"2024-03-06T07:20:07.242029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242311+0000","last_change":"2024-03-06T07:20:01.834757+0000","last_active":"2024-03-06T07:20:07.242311+0000","last_peered":"2024-03-06T07:20:07.242311+0000","last_clean":"2024-03-06T07:20:07.242311+0000","last_became_active":"2024-03-06T07:20:01.833698+0000","last_became_peered":"2024-03-06T07:20:01.833698+0000","last_unstale":"2024-03-06T07:20:07.242311+0000","last_undegraded":"2024-03-06T07:20:07.242311+0000","last_fullsized":"2024-03-06T07:20:07.242311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241536+0000","last_change":"2024-03-06T07:20:01.816955+0000","last_active":"2024-03-06T07:20:07.241536+0000","last_peered":"2024-03-06T07:20:07.241536+0000","last_clean":"2024-03-06T07:20:07.241536+0000","last_became_active":"2024-03-06T07:20:01.816124+0000","last_became_peered":"2024-03-06T07:20:01.816124+0000","last_unstale":"2024-03-06T07:20:07.241536+0000","last_undegraded":"2024-03-06T07:20:07.241536+0000","last_fullsized":"2024-03-06T07:20:07.241536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241876+0000","last_change":"2024-03-06T07:20:01.831322+0000","last_active":"2024-03-06T07:20:07.241876+0000","last_peered":"2024-03-06T07:20:07.241876+0000","last_clean":"2024-03-06T07:20:07.241876+0000","last_became_active":"2024-03-06T07:20:01.830919+0000","last_became_peered":"2024-03-06T07:20:01.830919+0000","last_unstale":"2024-03-06T07:20:07.241876+0000","last_undegraded":"2024-03-06T07:20:07.241876+0000","last_fullsized":"2024-03-06T07:20:07.241876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241171+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241171+0000","last_peered":"2024-03-06T07:20:07.241171+0000","last_clean":"2024-03-06T07:20:07.241171+0000","last_became_active":"2024-03-06T07:20:01.826111+0000","last_became_peered":"2024-03-06T07:20:01.826111+0000","last_unstale":"2024-03-06T07:20:07.241171+0000","last_undegraded":"2024-03-06T07:20:07.241171+0000","last_fullsized":"2024-03-06T07:20:07.241171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241941+0000","last_change":"2024-03-06T07:20:01.822677+0000","last_active":"2024-03-06T07:20:07.241941+0000","last_peered":"2024-03-06T07:20:07.241941+0000","last_clean":"2024-03-06T07:20:07.241941+0000","last_became_active":"2024-03-06T07:20:01.822263+0000","last_became_peered":"2024-03-06T07:20:01.822263+0000","last_unstale":"2024-03-06T07:20:07.241941+0000","last_undegraded":"2024-03-06T07:20:07.241941+0000","last_fullsized":"2024-03-06T07:20:07.241941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242529+0000","last_change":"2024-03-06T07:20:01.818125+0000","last_active":"2024-03-06T07:20:07.242529+0000","last_peered":"2024-03-06T07:20:07.242529+0000","last_clean":"2024-03-06T07:20:07.242529+0000","last_became_active":"2024-03-06T07:20:01.817547+0000","last_became_peered":"2024-03-06T07:20:01.817547+0000","last_unstale":"2024-03-06T07:20:07.242529+0000","last_undegraded":"2024-03-06T07:20:07.242529+0000","last_fullsized":"2024-03-06T07:20:07.242529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":691206,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":691262,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:19.453 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:20:19.454 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-06T07:20:19.454 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-06T07:20:19.454 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-06T07:20:19.454 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-06T07:20:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.588+0000 7f75e928f640 1 Processor -- start 2024-03-06T07:20:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.588+0000 7f75e928f640 1 -- start start 2024-03-06T07:20:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e928f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75e40c5480 0x7f75e4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e928f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e928f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e40c5cc0 con 0x7f75e40c5860 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e928f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e406df40 con 0x7f75e40c5480 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e928f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e406fdb0 con 0x7f75e41586f0 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e2ffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75e40c5480 0x7f75e4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e2ffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75e40c5480 0x7f75e4155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36906/0 (socket says 172.21.15.26:36906) 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.589+0000 7f75e2ffd640 1 -- 172.21.15.26:0/841172202 learned_addr learned my addr 172.21.15.26:0/841172202 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.590+0000 7f75e2ffd640 1 -- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 msgr2=0x7f75e4158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.590+0000 7f75e2ffd640 1 --2- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e4158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.590+0000 7f75e2ffd640 1 -- 172.21.15.26:0/841172202 >> v1:172.21.15.117:6789/0 conn(0x7f75e40c5860 legacy=0x7f75e406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.590+0000 7f75e2ffd640 1 -- 172.21.15.26:0/841172202 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e41591c0 con 0x7f75e40c5480 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.590+0000 7f75e1ffb640 1 -- 172.21.15.26:0/841172202 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278070768 0 0) 0x7f75e40c5cc0 con 0x7f75e40c5860 2024-03-06T07:20:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.590+0000 7f75e37fe640 1 --2- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.590+0000 7f75e2ffd640 1 --2- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 0x7f75e4155ba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009e50 tx=0x7f75dc02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76191b9e7c65d8b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e1ffb640 1 -- 172.21.15.26:0/841172202 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc02ca70 con 0x7f75e40c5480 2024-03-06T07:20:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e1ffb640 1 -- 172.21.15.26:0/841172202 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc02cc10 con 0x7f75e40c5480 2024-03-06T07:20:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e1ffb640 1 -- 172.21.15.26:0/841172202 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0343f0 con 0x7f75e40c5480 2024-03-06T07:20:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 -- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 msgr2=0x7f75e4155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 --2- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 0x7f75e4155ba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009e50 tx=0x7f75dc02bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 -- 172.21.15.26:0/841172202 shutdown_connections 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 --2- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 --2- 172.21.15.26:0/841172202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 0x7f75e4155ba0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 -- 172.21.15.26:0/841172202 >> 172.21.15.26:0/841172202 conn(0x7f75e406b1a0 msgr2=0x7f75e406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 -- 172.21.15.26:0/841172202 shutdown_connections 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.591+0000 7f75e928f640 1 -- 172.21.15.26:0/841172202 wait complete. 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.592+0000 7f75e928f640 1 Processor -- start 2024-03-06T07:20:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.592+0000 7f75e928f640 1 -- start start 2024-03-06T07:20:19.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e928f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 0x7f75e41eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e928f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e40c5860 0x7f75e4156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e928f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e41e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e928f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e4155680 con 0x7f75e40c5860 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e928f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e41553a0 con 0x7f75e40c5480 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e928f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e4152a20 con 0x7f75e41586f0 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e40c5860 0x7f75e4156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 0x7f75e41eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e40c5860 0x7f75e4156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46732/0 (socket says 172.21.15.26:46732) 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e27fc640 1 -- 172.21.15.26:0/2486060971 learned_addr learned my addr 172.21.15.26:0/2486060971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.593+0000 7f75e37fe640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e41e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e27fc640 1 -- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 msgr2=0x7f75e41e0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e27fc640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e41e0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e27fc640 1 -- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 msgr2=0x7f75e41eb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e27fc640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 0x7f75e41eb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e27fc640 1 -- 172.21.15.26:0/2486060971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e41ebf10 con 0x7f75e40c5860 2024-03-06T07:20:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e27fc640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e40c5860 0x7f75e4156440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c4b0 tx=0x7f75d800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75cb7fe640 1 -- 172.21.15.26:0/2486060971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8017070 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75cb7fe640 1 -- 172.21.15.26:0/2486060971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d8005480 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e414db60 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75cb7fe640 1 -- 172.21.15.26:0/2486060971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8003830 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.594+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e4157d70 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.595+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e40c2390 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.595+0000 7f75cb7fe640 1 -- 172.21.15.26:0/2486060971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75d801a070 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.596+0000 7f75cb7fe640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75d006b4d0 0x7f75d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.596+0000 7f75cb7fe640 1 -- 172.21.15.26:0/2486060971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f75d80897b0 con 0x7f75e40c5860 2024-03-06T07:20:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.596+0000 7f75e2ffd640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75d006b4d0 0x7f75d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.596+0000 7f75e2ffd640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75d006b4d0 0x7f75d006d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75dc035000 tx=0x7f75dc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.600+0000 7f75cb7fe640 1 -- 172.21.15.26:0/2486060971 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f75e40c2390 con 0x7f75e40c5860 2024-03-06T07:20:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.833+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f75e4157f70 con 0x7f75e40c5860 2024-03-06T07:20:19.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.835+0000 7f75cb7fe640 1 -- 172.21.15.26:0/2486060971 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171791 (secure 0 0 0) 0x7f75e4157f70 con 0x7f75e40c5860 2024-03-06T07:20:19.839 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75d006b4d0 msgr2=0x7f75d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75d006b4d0 0x7f75d006d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75dc035000 tx=0x7f75dc030000 comp rx=0 tx=0).stop 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e40c5860 msgr2=0x7f75e4156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e40c5860 0x7f75e4156440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c4b0 tx=0x7f75d800c970 comp rx=0 tx=0).stop 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 shutdown_connections 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75d006b4d0 0x7f75d006d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75e41586f0 0x7f75e41e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e40c5860 0x7f75e4156440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 --2- 172.21.15.26:0/2486060971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75e40c5480 0x7f75e41eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 >> 172.21.15.26:0/2486060971 conn(0x7f75e406b1a0 msgr2=0x7f75e41513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 shutdown_connections 2024-03-06T07:20:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.838+0000 7f75e928f640 1 -- 172.21.15.26:0/2486060971 wait complete. 2024-03-06T07:20:19.850 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3679442139},{"type":"v1","addr":"172.21.15.117:6833","nonce":3679442139}]},"active_addr":"172.21.15.117:6833/3679442139","active_change":"2024-03-06T07:19:49.020787+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":717246748}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1518563965}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":4215909963}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":3698865435}]}]} 2024-03-06T07:20:19.854 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-06T07:20:19.854 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-06T07:20:19.854 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.954+0000 7f45b2471640 1 Processor -- start 2024-03-06T07:20:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.954+0000 7f45b2471640 1 -- start start 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b2471640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45ac152920 0x7f45ac155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b2471640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b2471640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac156180 con 0x7f45ac0c8a00 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b2471640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45ac1587b0 con 0x7f45ac152920 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b2471640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac159630 con 0x7f45ac0c5330 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45abfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45ac152920 0x7f45ac155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b0a0e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45abfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45ac152920 0x7f45ac155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36930/0 (socket says 172.21.15.26:36930) 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45ab7fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f45ac0c8a00 0x7f45ac158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35928/0 (socket says 172.21.15.26:35928) 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45abfff640 1 -- 172.21.15.26:0/2302536916 learned_addr learned my addr 172.21.15.26:0/2302536916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b0a0e640 1 -- 172.21.15.26:0/2302536916 >> v2:172.21.15.117:3300/0 conn(0x7f45ac152920 msgr2=0x7f45ac155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b0a0e640 1 --2- 172.21.15.26:0/2302536916 >> v2:172.21.15.117:3300/0 conn(0x7f45ac152920 0x7f45ac155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b0a0e640 1 -- 172.21.15.26:0/2302536916 >> v1:172.21.15.117:6789/0 conn(0x7f45ac0c8a00 legacy=0x7f45ac158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45b0a0e640 1 -- 172.21.15.26:0/2302536916 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac0c7760 con 0x7f45ac0c5330 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45aaffd640 1 -- 172.21.15.26:0/2302536916 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401538981 0 0) 0x7f459c001110 con 0x7f45ac0c8a00 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.955+0000 7f45abfff640 1 --2- 172.21.15.26:0/2302536916 >> v2:172.21.15.117:3300/0 conn(0x7f45ac152920 0x7f45ac155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.956+0000 7f45b0a0e640 1 --2- 172.21.15.26:0/2302536916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac0c57c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009d90 tx=0x7f45a402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8a3e73c7edd6ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.956+0000 7f45aaffd640 1 -- 172.21.15.26:0/2302536916 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4039070 con 0x7f45ac0c5330 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.956+0000 7f45aaffd640 1 -- 172.21.15.26:0/2302536916 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a402cbc0 con 0x7f45ac0c5330 2024-03-06T07:20:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.956+0000 7f45aaffd640 1 -- 172.21.15.26:0/2302536916 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4034440 con 0x7f45ac0c5330 2024-03-06T07:20:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 -- 172.21.15.26:0/2302536916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 msgr2=0x7f45ac0c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 --2- 172.21.15.26:0/2302536916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac0c57c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009d90 tx=0x7f45a402be10 comp rx=0 tx=0).stop 2024-03-06T07:20:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 -- 172.21.15.26:0/2302536916 shutdown_connections 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 --2- 172.21.15.26:0/2302536916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac0c57c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 --2- 172.21.15.26:0/2302536916 >> v2:172.21.15.117:3300/0 conn(0x7f45ac152920 0x7f45ac155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 -- 172.21.15.26:0/2302536916 >> 172.21.15.26:0/2302536916 conn(0x7f45ac0ac560 msgr2=0x7f45ac0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 -- 172.21.15.26:0/2302536916 shutdown_connections 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.957+0000 7f45b2471640 1 -- 172.21.15.26:0/2302536916 wait complete. 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.958+0000 7f45b2471640 1 Processor -- start 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.958+0000 7f45b2471640 1 -- start start 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45b2471640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac1eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45b2471640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45ac0c8a00 0x7f45ac0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45b2471640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac152920 0x7f45ac14dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45b2471640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac158a00 con 0x7f45ac0c8a00 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45b2471640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0c7d20 con 0x7f45ac152920 2024-03-06T07:20:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45b2471640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac06d030 con 0x7f45ac0c5330 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45ac0c8a00 0x7f45ac0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45abfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac1eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45ac0c8a00 0x7f45ac0c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46746/0 (socket says 172.21.15.26:46746) 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45ab7fe640 1 -- 172.21.15.26:0/4227277783 learned_addr learned my addr 172.21.15.26:0/4227277783 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45b0a0e640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac152920 0x7f45ac14dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45abfff640 1 -- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac152920 msgr2=0x7f45ac14dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45abfff640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac152920 0x7f45ac14dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45abfff640 1 -- 172.21.15.26:0/4227277783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45ac0c8a00 msgr2=0x7f45ac0c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45abfff640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45ac0c8a00 0x7f45ac0c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45abfff640 1 -- 172.21.15.26:0/4227277783 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac150e20 con 0x7f45ac0c5330 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45ab7fe640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45ac0c8a00 0x7f45ac0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.959+0000 7f45abfff640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac1eb560 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f459400e930 tx=0x7f459400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.960+0000 7f45a8ff9640 1 -- 172.21.15.26:0/4227277783 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400ca00 con 0x7f45ac0c5330 2024-03-06T07:20:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.960+0000 7f45a8ff9640 1 -- 172.21.15.26:0/4227277783 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400cba0 con 0x7f45ac0c5330 2024-03-06T07:20:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.960+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac14f570 con 0x7f45ac0c5330 2024-03-06T07:20:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.960+0000 7f45a8ff9640 1 -- 172.21.15.26:0/4227277783 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594004030 con 0x7f45ac0c5330 2024-03-06T07:20:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.960+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac14fb30 con 0x7f45ac0c5330 2024-03-06T07:20:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.961+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac04f330 con 0x7f45ac0c5330 2024-03-06T07:20:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.961+0000 7f45a8ff9640 1 -- 172.21.15.26:0/4227277783 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45940041d0 con 0x7f45ac0c5330 2024-03-06T07:20:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.961+0000 7f45a8ff9640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f458c06b4d0 0x7f458c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.961+0000 7f45a8ff9640 1 -- 172.21.15.26:0/4227277783 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f459408a3e0 con 0x7f45ac0c5330 2024-03-06T07:20:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.962+0000 7f45ab7fe640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f458c06b4d0 0x7f458c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.962+0000 7f45ab7fe640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f458c06b4d0 0x7f458c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0c3440 tx=0x7f459c007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:19.966+0000 7f45a8ff9640 1 -- 172.21.15.26:0/4227277783 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4594017020 con 0x7f45ac0c5330 2024-03-06T07:20:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.150+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45ac0ab3f0 con 0x7f45ac0c5330 2024-03-06T07:20:20.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.152+0000 7f45a8ff9640 1 -- 172.21.15.26:0/4227277783 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f4594058940 con 0x7f45ac0c5330 2024-03-06T07:20:20.154 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:20.154 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f458c06b4d0 msgr2=0x7f458c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f458c06b4d0 0x7f458c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0c3440 tx=0x7f459c007ea0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 msgr2=0x7f45ac1eb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac1eb560 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f459400e930 tx=0x7f459400edf0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 shutdown_connections 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f458c06b4d0 0x7f458c06d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac152920 0x7f45ac14dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45ac0c8a00 0x7f45ac0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 --2- 172.21.15.26:0/4227277783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45ac0c5330 0x7f45ac1eb560 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 >> 172.21.15.26:0/4227277783 conn(0x7f45ac0ac560 msgr2=0x7f45ac0caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 shutdown_connections 2024-03-06T07:20:20.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.155+0000 7f45b2471640 1 -- 172.21.15.26:0/4227277783 wait complete. 2024-03-06T07:20:20.167 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-06T07:20:20.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:20.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.267+0000 7fb88f4fb640 1 Processor -- start 2024-03-06T07:20:20.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.267+0000 7fb88f4fb640 1 -- start start 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.267+0000 7fb88f4fb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb888152920 0x7fb888155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.267+0000 7fb88f4fb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb888156180 con 0x7fb8880c8a00 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.267+0000 7fb88f4fb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb888159630 con 0x7fb8880c5330 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.267+0000 7fb88f4fb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8881587b0 con 0x7fb888152920 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb88ca96640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8880c8a00 0x7fb888158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44116/0 (socket says 172.21.15.26:44116) 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb88ca96640 1 -- 172.21.15.26:0/1449814908 learned_addr learned my addr 172.21.15.26:0/1449814908 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb88d297640 1 --2- 172.21.15.26:0/1449814908 >> v2:172.21.15.117:3300/0 conn(0x7fb888152920 0x7fb888155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb87ffff640 1 -- 172.21.15.26:0/1449814908 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381511832 0 0) 0x7fb878001110 con 0x7fb8880c8a00 2024-03-06T07:20:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb87ffff640 1 -- 172.21.15.26:0/1449814908 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb874003520 con 0x7fb8880c8a00 2024-03-06T07:20:20.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb88d297640 1 -- 172.21.15.26:0/1449814908 >> v1:172.21.15.117:6789/0 conn(0x7fb8880c5330 legacy=0x7fb88815fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb87ffff640 1 -- 172.21.15.26:0/1449814908 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400399685 0 0) 0x7fb888159630 con 0x7fb8880c5330 2024-03-06T07:20:20.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb88d297640 1 -- 172.21.15.26:0/1449814908 >> v1:172.21.15.26:6789/0 conn(0x7fb8880c8a00 legacy=0x7fb888158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.268+0000 7fb88d297640 1 -- 172.21.15.26:0/1449814908 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880c7760 con 0x7fb888152920 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb88d297640 1 --2- 172.21.15.26:0/1449814908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb888152920 0x7fb888155c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8840097a0 tx=0x7fb88402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1050428e49427b14 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb87ffff640 1 -- 172.21.15.26:0/1449814908 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884035020 con 0x7fb888152920 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb87ffff640 1 -- 172.21.15.26:0/1449814908 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884031830 con 0x7fb888152920 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb87ffff640 1 -- 172.21.15.26:0/1449814908 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884031be0 con 0x7fb888152920 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb88f4fb640 1 -- 172.21.15.26:0/1449814908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb888152920 msgr2=0x7fb888155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/1449814908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb888152920 0x7fb888155c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8840097a0 tx=0x7fb88402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb88f4fb640 1 -- 172.21.15.26:0/1449814908 shutdown_connections 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.269+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/1449814908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb888152920 0x7fb888155c40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.270+0000 7fb88f4fb640 1 -- 172.21.15.26:0/1449814908 >> 172.21.15.26:0/1449814908 conn(0x7fb8880ac560 msgr2=0x7fb8880ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.270+0000 7fb88f4fb640 1 -- 172.21.15.26:0/1449814908 shutdown_connections 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.270+0000 7fb88f4fb640 1 -- 172.21.15.26:0/1449814908 wait complete. 2024-03-06T07:20:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 Processor -- start 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 -- start start 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8880c5330 0x7fb8881edd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8880c8a00 0x7fb8881ee240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb888152920 0x7fb8880c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb888158a00 con 0x7fb888152920 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c7d20 con 0x7fb8880c8a00 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88f4fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88806d030 con 0x7fb8880c5330 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88da98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb888152920 0x7fb8880c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.271+0000 7fb88da98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb888152920 0x7fb8880c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46750/0 (socket says 172.21.15.26:46750) 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88da98640 1 -- 172.21.15.26:0/3993532038 learned_addr learned my addr 172.21.15.26:0/3993532038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88ca96640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8880c8a00 0x7fb8881ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88d297640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8880c5330 0x7fb8881edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88da98640 1 -- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8880c5330 msgr2=0x7fb8881edd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88da98640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8880c5330 0x7fb8881edd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88da98640 1 -- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8880c8a00 msgr2=0x7fb8881ee240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88da98640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8880c8a00 0x7fb8881ee240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88da98640 1 -- 172.21.15.26:0/3993532038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888155380 con 0x7fb888152920 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88da98640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb888152920 0x7fb8880c2270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c7b0 tx=0x7fb88000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb87dffb640 1 -- 172.21.15.26:0/3993532038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880003ba0 con 0x7fb888152920 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb87dffb640 1 -- 172.21.15.26:0/3993532038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880003d40 con 0x7fb888152920 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8881555f0 con 0x7fb888152920 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb87dffb640 1 -- 172.21.15.26:0/3993532038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880005270 con 0x7fb888152920 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.272+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8881eeba0 con 0x7fb888152920 2024-03-06T07:20:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.273+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88804f330 con 0x7fb888152920 2024-03-06T07:20:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.274+0000 7fb87dffb640 1 -- 172.21.15.26:0/3993532038 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb880005410 con 0x7fb888152920 2024-03-06T07:20:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.274+0000 7fb87dffb640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.274+0000 7fb87dffb640 1 -- 172.21.15.26:0/3993532038 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb88008a1e0 con 0x7fb888152920 2024-03-06T07:20:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.274+0000 7fb88d297640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.275+0000 7fb88d297640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86c06b4d0 0x7fb86c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb884036000 tx=0x7fb884008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.279+0000 7fb87dffb640 1 -- 172.21.15.26:0/3993532038 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb880058760 con 0x7fb888152920 2024-03-06T07:20:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.460+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8880bdfe0 con 0x7fb888152920 2024-03-06T07:20:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.461+0000 7fb87dffb640 1 -- 172.21.15.26:0/3993532038 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fb88005c800 con 0x7fb888152920 2024-03-06T07:20:20.463 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:20.463 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86c06b4d0 msgr2=0x7fb86c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86c06b4d0 0x7fb86c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb884036000 tx=0x7fb884008030 comp rx=0 tx=0).stop 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb888152920 msgr2=0x7fb8880c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb888152920 0x7fb8880c2270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c7b0 tx=0x7fb88000cbb0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 shutdown_connections 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb888152920 0x7fb8880c2270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8880c8a00 0x7fb8881ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 --2- 172.21.15.26:0/3993532038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8880c5330 0x7fb8881edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 >> 172.21.15.26:0/3993532038 conn(0x7fb8880ac560 msgr2=0x7fb8881565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 shutdown_connections 2024-03-06T07:20:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.463+0000 7fb88f4fb640 1 -- 172.21.15.26:0/3993532038 wait complete. 2024-03-06T07:20:20.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-06T07:20:20.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-06T07:20:20.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-06T07:20:20.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-06T07:20:20.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-06T07:20:20.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-06T07:20:20.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-06T07:20:20.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-06T07:20:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.643+0000 7f5a7d0af640 1 Processor -- start 2024-03-06T07:20:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.643+0000 7f5a7d0af640 1 -- start start 2024-03-06T07:20:20.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.644+0000 7f5a7d0af640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5a780c1390 0x7f5a780c2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.644+0000 7f5a7d0af640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5a780a9650 0x7f5a780c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.644+0000 7f5a7d0af640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780c2580 con 0x7f5a7808b170 2024-03-06T07:20:20.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.644+0000 7f5a7d0af640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780c2900 con 0x7f5a780c1390 2024-03-06T07:20:20.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.644+0000 7f5a7d0af640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780c2be0 con 0x7f5a780a9650 2024-03-06T07:20:20.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.644+0000 7f5a7659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5a7808b170 0x7f5a78164c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44132/0 (socket says 172.21.15.26:44132) 2024-03-06T07:20:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.644+0000 7f5a7659c640 1 -- 172.21.15.26:0/363800401 learned_addr learned my addr 172.21.15.26:0/363800401 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.645+0000 7f5a76d9d640 1 --2- 172.21.15.26:0/363800401 >> v2:172.21.15.26:3300/0 conn(0x7f5a780c1390 0x7f5a780c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.645+0000 7f5a76d9d640 1 -- 172.21.15.26:0/363800401 >> v2:172.21.15.117:3300/0 conn(0x7f5a780a9650 msgr2=0x7f5a780c6a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.645+0000 7f5a76d9d640 1 --2- 172.21.15.26:0/363800401 >> v2:172.21.15.117:3300/0 conn(0x7f5a780a9650 0x7f5a780c6a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.645+0000 7f5a76d9d640 1 -- 172.21.15.26:0/363800401 >> v1:172.21.15.26:6789/0 conn(0x7f5a7808b170 legacy=0x7f5a78164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.645+0000 7f5a76d9d640 1 -- 172.21.15.26:0/363800401 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a781662a0 con 0x7f5a780c1390 2024-03-06T07:20:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.647+0000 7f5a76d9d640 1 --2- 172.21.15.26:0/363800401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 0x7f5a780c2040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a7806e500 tx=0x7f5a6803c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6346f43c817f3d11 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.647+0000 7f5a75d9b640 1 -- 172.21.15.26:0/363800401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68048070 con 0x7f5a780c1390 2024-03-06T07:20:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.647+0000 7f5a75d9b640 1 -- 172.21.15.26:0/363800401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a68041710 con 0x7f5a780c1390 2024-03-06T07:20:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a75d9b640 1 -- 172.21.15.26:0/363800401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68041ae0 con 0x7f5a780c1390 2024-03-06T07:20:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 -- 172.21.15.26:0/363800401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 msgr2=0x7f5a780c2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 --2- 172.21.15.26:0/363800401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 0x7f5a780c2040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a7806e500 tx=0x7f5a6803c630 comp rx=0 tx=0).stop 2024-03-06T07:20:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 -- 172.21.15.26:0/363800401 shutdown_connections 2024-03-06T07:20:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 --2- 172.21.15.26:0/363800401 >> v2:172.21.15.117:3300/0 conn(0x7f5a780a9650 0x7f5a780c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f516aa17640 1 Processor -- start 2024-03-06T07:20:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 --2- 172.21.15.26:0/363800401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 0x7f5a780c2040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 -- 172.21.15.26:0/363800401 >> 172.21.15.26:0/363800401 conn(0x7f5a7806b410 msgr2=0x7f5a7806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 -- 172.21.15.26:0/363800401 shutdown_connections 2024-03-06T07:20:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f516aa17640 1 -- start start 2024-03-06T07:20:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.648+0000 7f5a7d0af640 1 -- 172.21.15.26:0/363800401 wait complete. 2024-03-06T07:20:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.649+0000 7f516aa17640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51640c3820 0x7f51640c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.649+0000 7f3151f15640 1 Processor -- start 2024-03-06T07:20:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f79c9fa0640 1 Processor -- start 2024-03-06T07:20:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f51637fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f516406d900 0x7f51640c5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35948/0 (socket says 172.21.15.26:35948) 2024-03-06T07:20:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f51637fe640 1 -- 172.21.15.26:0/3870604057 learned_addr learned my addr 172.21.15.26:0/3870604057 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f5168fb4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51640c2d40 0x7f516414f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44142/0 (socket says 172.21.15.26:44142) 2024-03-06T07:20:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5164152e20 con 0x7f51640c2d40 2024-03-06T07:20:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f3151f15640 1 -- start start 2024-03-06T07:20:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516406dff0 con 0x7f516406d900 2024-03-06T07:20:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51640c5e10 con 0x7f51640c3820 2024-03-06T07:20:20.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f3151f15640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31440dbce0 0x7f31440474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318474846 0 0) 0x7f5164152e20 con 0x7f51640c2d40 2024-03-06T07:20:20.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5148003520 con 0x7f51640c2d40 2024-03-06T07:20:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f3151f15640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314400bb50 con 0x7f31440df180 2024-03-06T07:20:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f3151f15640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3144048060 con 0x7f314400a8b0 2024-03-06T07:20:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f3151f15640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314400a420 con 0x7f31440dbce0 2024-03-06T07:20:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5a7d0af640 1 Processor -- start 2024-03-06T07:20:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f3151714640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f314400a8b0 0x7f314400ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35958/0 (socket says 172.21.15.26:35958) 2024-03-06T07:20:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f3151714640 1 -- 172.21.15.26:0/1058296591 learned_addr learned my addr 172.21.15.26:0/1058296591 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041444372 0 0) 0x7f3144048060 con 0x7f314400a8b0 2024-03-06T07:20:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f313c003520 con 0x7f314400a8b0 2024-03-06T07:20:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f4c13fff640 1 Processor -- start 2024-03-06T07:20:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f3150f13640 1 --2- 172.21.15.26:0/1058296591 >> v2:172.21.15.117:3300/0 conn(0x7f31440dbce0 0x7f31440474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172720452 0 0) 0x7f313c003520 con 0x7f314400a8b0 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5163fff640 1 --2- 172.21.15.26:0/3870604057 >> v2:172.21.15.26:3300/0 conn(0x7f51640c3820 0x7f51640c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866406376 0 0) 0x7f516406dff0 con 0x7f516406d900 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5148003880 con 0x7f516406d900 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2995777914 0 0) 0x7f5148003520 con 0x7f51640c2d40 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 >> v2:172.21.15.26:3300/0 conn(0x7f51640c3820 msgr2=0x7f51640c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 --2- 172.21.15.26:0/3870604057 >> v2:172.21.15.26:3300/0 conn(0x7f51640c3820 0x7f51640c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 >> v1:172.21.15.117:6789/0 conn(0x7f516406d900 legacy=0x7f51640c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164152ac0 con 0x7f51640c2d40 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f51540015d0 con 0x7f51640c2d40 2024-03-06T07:20:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 >> v1:172.21.15.26:6789/0 conn(0x7f51640c2d40 legacy=0x7f516414f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5148003d30 0x7f5148004130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148004850 0x7f5148004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5148009c30 0x7f5148005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51640c5e10 con 0x7f5148003d30 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514800c3d0 con 0x7f5148009c30 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514800c5d0 con 0x7f5148004850 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 >> v2:172.21.15.117:3300/0 conn(0x7f31440dbce0 msgr2=0x7f31440474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 --2- 172.21.15.26:0/1058296591 >> v2:172.21.15.117:3300/0 conn(0x7f31440dbce0 0x7f31440474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 >> v1:172.21.15.26:6789/0 conn(0x7f31440df180 legacy=0x7f3144006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314400a0f0 con 0x7f314400a8b0 2024-03-06T07:20:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f3138001670 con 0x7f314400a8b0 2024-03-06T07:20:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 >> v1:172.21.15.117:6789/0 conn(0x7f314400a8b0 legacy=0x7f314400ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f313c0039b0 0x7f313c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c004800 0x7f313c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313c005210 0x7f313c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.650+0000 7f79c9fa0640 1 -- start start 2024-03-06T07:20:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f79c9fa0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f79bc0d7ae0 0x7f79bc0071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f79c9fa0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc00f9d0 con 0x7f79bc04dae0 2024-03-06T07:20:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f79c9fa0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc0da4b0 con 0x7f79bc04dff0 2024-03-06T07:20:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f79c9fa0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc0da790 con 0x7f79bc0d7ae0 2024-03-06T07:20:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314400a420 con 0x7f313c005210 2024-03-06T07:20:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c00c140 con 0x7f313c004800 2024-03-06T07:20:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313c00c340 con 0x7f313c0039b0 2024-03-06T07:20:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7fee3dac9640 1 Processor -- start 2024-03-06T07:20:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7fee3dac9640 1 -- start start 2024-03-06T07:20:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f314bfff640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c004800 0x7f313c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f5163fff640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5148003d30 0x7f5148004130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f5a7d0af640 1 -- start start 2024-03-06T07:20:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3150f13640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f313c0039b0 0x7f313c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3151714640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313c005210 0x7f313c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3150f13640 1 -- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c004800 msgr2=0x7f313c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3150f13640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c004800 0x7f313c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3150f13640 1 -- 172.21.15.26:0/1058296591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313c005210 msgr2=0x7f313c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3150f13640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313c005210 0x7f313c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3150f13640 1 -- 172.21.15.26:0/1058296591 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314400a0f0 con 0x7f313c0039b0 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f79c3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f79bc04dae0 0x7f79bc0e4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44154/0 (socket says 172.21.15.26:44154) 2024-03-06T07:20:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f79c3fff640 1 -- 172.21.15.26:0/975725745 learned_addr learned my addr 172.21.15.26:0/975725745 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3151714640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313c005210 0x7f313c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.652+0000 7f3151714640 1 -- 172.21.15.26:0/1058296591 reap_dead start 2024-03-06T07:20:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.651+0000 7f4c13fff640 1 -- start start 2024-03-06T07:20:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c0062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234530040 0 0) 0x7f79bc00f9d0 con 0x7f79bc04dae0 2024-03-06T07:20:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b4003520 con 0x7f79bc04dae0 2024-03-06T07:20:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f79c37fe640 1 --2- 172.21.15.26:0/975725745 >> v2:172.21.15.26:3300/0 conn(0x7f79bc0d7ae0 0x7f79bc0071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f4c13fff640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c0d7d00 con 0x7f4c0c0d7920 2024-03-06T07:20:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f4c13fff640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c0db250 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f4c13fff640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c04ba00 con 0x7f4c0c050720 2024-03-06T07:20:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7d0af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a7808b170 0x7f5a781eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7fee3dac9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee380728a0 0x7fee380712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7fee3dac9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee38152940 0x7fee38071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7d0af640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a780a9650 0x7f5a78071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7d0af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 0x7f5a781edc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a76d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a7808b170 0x7f5a781eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7d0af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780a9bf0 con 0x7f5a780c1390 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7d0af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780a9a70 con 0x7f5a7808b170 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7d0af640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780c6f60 con 0x7f5a780a9650 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f4c137fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4c0c00fcf0 0x7f4c0c006800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35962/0 (socket says 172.21.15.26:35962) 2024-03-06T07:20:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f4c137fe640 1 -- 172.21.15.26:0/3102111360 learned_addr learned my addr 172.21.15.26:0/3102111360 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7fee3dac9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3806c730 con 0x7fee38070b30 2024-03-06T07:20:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7fee3dac9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee38152e20 con 0x7fee380728a0 2024-03-06T07:20:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7fee3dac9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3806c170 con 0x7fee38152940 2024-03-06T07:20:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f4c127fc640 1 --2- 172.21.15.26:0/3102111360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c0062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 0x7f5a781edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1141224006 0 0) 0x7f79b4003520 con 0x7f79bc04dae0 2024-03-06T07:20:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 0x7f5a781edc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46802/0 (socket says 172.21.15.26:46802) 2024-03-06T07:20:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a780a9650 0x7f5a78071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5a7759e640 1 -- 172.21.15.26:0/1169680420 learned_addr learned my addr 172.21.15.26:0/1169680420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 >> v2:172.21.15.26:3300/0 conn(0x7f79bc0d7ae0 msgr2=0x7f79bc0071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 --2- 172.21.15.26:0/975725745 >> v2:172.21.15.26:3300/0 conn(0x7f79bc0d7ae0 0x7f79bc0071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee37fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fee38070b30 0x7fee380abb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35974/0 (socket says 172.21.15.26:35974) 2024-03-06T07:20:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 >> v1:172.21.15.117:6789/0 conn(0x7f79bc04dff0 legacy=0x7f79bc04e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f3150f13640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f313c0039b0 0x7f313c003d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f314c069670 tx=0x7f313c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57806ab701bbde40 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5163fff640 1 -- 172.21.15.26:0/3870604057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148004850 msgr2=0x7f5148004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5163fff640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148004850 0x7f5148004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5163fff640 1 -- 172.21.15.26:0/3870604057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5148009c30 msgr2=0x7f5148005210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5163fff640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5148009c30 0x7f5148005210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5163fff640 1 -- 172.21.15.26:0/3870604057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164152ac0 con 0x7f5148003d30 2024-03-06T07:20:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.653+0000 7f5168fb4640 1 -- 172.21.15.26:0/3870604057 reap_dead start 2024-03-06T07:20:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee37fff640 1 -- 172.21.15.26:0/3866040790 learned_addr learned my addr 172.21.15.26:0/3866040790 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee377fe640 1 --2- 172.21.15.26:0/3866040790 >> v2:172.21.15.26:3300/0 conn(0x7fee380728a0 0x7fee380712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee36ffd640 1 --2- 172.21.15.26:0/3866040790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee38152940 0x7fee38071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0ad070 con 0x7f313c0039b0 2024-03-06T07:20:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c0a7c80 con 0x7f313c0039b0 2024-03-06T07:20:20.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0daa70 con 0x7f79bc04dae0 2024-03-06T07:20:20.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f79b0001670 con 0x7f79bc04dae0 2024-03-06T07:20:20.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f314b7fe640 1 -- 172.21.15.26:0/1058296591 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c074f80 con 0x7f313c0039b0 2024-03-06T07:20:20.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 >> v1:172.21.15.26:6789/0 conn(0x7f79bc04dae0 legacy=0x7f79bc0e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f4c127fc640 1 -- 172.21.15.26:0/3102111360 >> v1:172.21.15.117:6789/0 conn(0x7f4c0c00fcf0 legacy=0x7f4c0c006800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee367fc640 1 -- 172.21.15.26:0/3866040790 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256027888 0 0) 0x7fee3806c730 con 0x7fee38070b30 2024-03-06T07:20:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c27fc640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee367fc640 1 -- 172.21.15.26:0/3866040790 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20007930 con 0x7fee38070b30 2024-03-06T07:20:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee377fe640 1 -- 172.21.15.26:0/3866040790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee38152940 msgr2=0x7fee38071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f79c37fe640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c27fc640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b40051c0 0x7f79b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee377fe640 1 --2- 172.21.15.26:0/3866040790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee38152940 0x7fee38071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0da790 con 0x7f79b4004800 2024-03-06T07:20:20.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee377fe640 1 -- 172.21.15.26:0/3866040790 >> v1:172.21.15.117:6789/0 conn(0x7fee38070b30 legacy=0x7fee380abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b400c160 con 0x7f79b40039b0 2024-03-06T07:20:20.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b400c360 con 0x7f79b40051c0 2024-03-06T07:20:20.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c3fff640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b40051c0 0x7f79b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7fee377fe640 1 -- 172.21.15.26:0/3866040790 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3806c450 con 0x7fee380728a0 2024-03-06T07:20:20.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7fee36ffd640 1 --2- 172.21.15.26:0/3866040790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee38152940 0x7fee38071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:20.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f3151f15640 1 -- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f313c0039b0 msgr2=0x7f313c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f3151f15640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f313c0039b0 0x7f313c003d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f314c069670 tx=0x7f313c005830 comp rx=0 tx=0).stop 2024-03-06T07:20:20.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c2ffd640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f4c11ffb640 1 -- 172.21.15.26:0/3102111360 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926630186 0 0) 0x7f4c0c0db250 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f4c127fc640 1 -- 172.21.15.26:0/3102111360 >> v1:172.21.15.26:6789/0 conn(0x7f4c0c0d7920 legacy=0x7f4c0c00edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f4c127fc640 1 -- 172.21.15.26:0/3102111360 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c04bce0 con 0x7f4c0c050720 2024-03-06T07:20:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f5163fff640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5148003d30 0x7f5148004130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f515c00bba0 tx=0x7f5148005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8c82445149120a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c032070 con 0x7f5148003d30 2024-03-06T07:20:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c02cba0 con 0x7f5148003d30 2024-03-06T07:20:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f5162ffd640 1 -- 172.21.15.26:0/3870604057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00b820 con 0x7f5148003d30 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c37fe640 1 -- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b40051c0 msgr2=0x7f79b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c37fe640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b40051c0 0x7f79b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c37fe640 1 -- 172.21.15.26:0/975725745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b4004800 msgr2=0x7f79b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c37fe640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c37fe640 1 -- 172.21.15.26:0/975725745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b4003520 con 0x7f79b40039b0 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c2ffd640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f79c3fff640 1 -- 172.21.15.26:0/975725745 reap_dead start 2024-03-06T07:20:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f4817f93640 1 Processor -- start 2024-03-06T07:20:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f4817f93640 1 -- start start 2024-03-06T07:20:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f4c127fc640 1 --2- 172.21.15.26:0/3102111360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c0062c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c080097d0 tx=0x7f4c0802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49258e87dccc1cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f10716fb640 1 Processor -- start 2024-03-06T07:20:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f10716fb640 1 -- start start 2024-03-06T07:20:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f3151f15640 1 -- 172.21.15.26:0/1058296591 shutdown_connections 2024-03-06T07:20:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f3151f15640 1 --2- 172.21.15.26:0/1058296591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f313c0039b0 0x7f313c003d90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f3151f15640 1 -- 172.21.15.26:0/1058296591 >> 172.21.15.26:0/1058296591 conn(0x7f3144048250 msgr2=0x7f3144048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f4c11ffb640 1 -- 172.21.15.26:0/3102111360 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08031780 con 0x7f4c0c050720 2024-03-06T07:20:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f3151f15640 1 -- 172.21.15.26:0/1058296591 shutdown_connections 2024-03-06T07:20:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f4c11ffb640 1 -- 172.21.15.26:0/3102111360 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c08037070 con 0x7f4c0c050720 2024-03-06T07:20:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f4c11ffb640 1 -- 172.21.15.26:0/3102111360 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08031b40 con 0x7f4c0c050720 2024-03-06T07:20:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7fee377fe640 1 --2- 172.21.15.26:0/3866040790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee380728a0 0x7fee380712c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d7b0 tx=0x7fee2802bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1442933bef8740 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f3151f15640 1 -- 172.21.15.26:0/1058296591 wait complete. 2024-03-06T07:20:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5148003d30 msgr2=0x7f5148004130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f516aa17640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5148003d30 0x7f5148004130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f515c00bba0 tx=0x7f5148005830 comp rx=0 tx=0).stop 2024-03-06T07:20:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7fee367fc640 1 -- 172.21.15.26:0/3866040790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2802d910 con 0x7fee380728a0 2024-03-06T07:20:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f10716fb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f106c0c6d30 0x7f106c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f10716fb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f106c0c0320 0x7f106c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f10716fb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c1598c0 con 0x7f106c0acdd0 2024-03-06T07:20:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f10716fb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c159ba0 con 0x7f106c0c0320 2024-03-06T07:20:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f10716fb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c159e80 con 0x7f106c0c6d30 2024-03-06T07:20:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7fee367fc640 1 -- 172.21.15.26:0/3866040790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2802dab0 con 0x7fee380728a0 2024-03-06T07:20:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f106affd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f106c0acdd0 0x7f106c156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35984/0 (socket says 172.21.15.26:35984) 2024-03-06T07:20:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f106affd640 1 -- 172.21.15.26:0/3919193550 learned_addr learned my addr 172.21.15.26:0/3919193550 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4817f93640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4810156ad0 0x7f481006e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4817f93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f481015cbe0 0x7f48100c2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4817f93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f48100c29b0 0x7f4810159a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4817f93640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48100a9650 con 0x7f4810156ad0 2024-03-06T07:20:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4817f93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481015d070 con 0x7f48100c29b0 2024-03-06T07:20:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4817f93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481015d350 con 0x7f481015cbe0 2024-03-06T07:20:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f106b7fe640 1 --2- 172.21.15.26:0/3919193550 >> v2:172.21.15.26:3300/0 conn(0x7f106c0c0320 0x7f106c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4c13fff640 1 -- 172.21.15.26:0/3102111360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 msgr2=0x7f4c0c0062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f5a7659c640 1 -- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a7808b170 msgr2=0x7f5a781eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f5a7659c640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a7808b170 0x7f5a781eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f5a7659c640 1 -- 172.21.15.26:0/1169680420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 msgr2=0x7f5a781edc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f5a7659c640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a780c1390 0x7f5a781edc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.654+0000 7f5a7659c640 1 -- 172.21.15.26:0/1169680420 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780bca60 con 0x7f5a780a9650 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.655+0000 7f5a7659c640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a780a9650 0x7f5a78071c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5a70007fd0 tx=0x7f5a7000ddb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70013070 con 0x7f5a780a9650 2024-03-06T07:20:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f5a7d0af640 1 -- 172.21.15.26:0/1169680420 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a780bb760 con 0x7f5a780a9650 2024-03-06T07:20:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.656+0000 7f5a7d0af640 1 -- 172.21.15.26:0/1169680420 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781eb980 con 0x7f5a780a9650 2024-03-06T07:20:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7000a640 con 0x7f5a780a9650 2024-03-06T07:20:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000aa10 con 0x7f5a780a9650 2024-03-06T07:20:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f106a7fc640 1 --2- 172.21.15.26:0/3919193550 >> v2:172.21.15.117:3300/0 conn(0x7f106c0c6d30 0x7f106c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f106b7fe640 1 -- 172.21.15.26:0/3919193550 >> v2:172.21.15.117:3300/0 conn(0x7f106c0c6d30 msgr2=0x7f106c15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f106b7fe640 1 --2- 172.21.15.26:0/3919193550 >> v2:172.21.15.117:3300/0 conn(0x7f106c0c6d30 0x7f106c15ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f106b7fe640 1 -- 172.21.15.26:0/3919193550 >> v1:172.21.15.117:6789/0 conn(0x7f106c0acdd0 legacy=0x7f106c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f106b7fe640 1 -- 172.21.15.26:0/3919193550 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c15a160 con 0x7f106c0c0320 2024-03-06T07:20:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4c13fff640 1 --2- 172.21.15.26:0/3102111360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c0062c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c080097d0 tx=0x7f4c0802b820 comp rx=0 tx=0).stop 2024-03-06T07:20:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f5a7d0af640 1 -- 172.21.15.26:0/1169680420 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a58001030 con 0x7f5a780a9650 2024-03-06T07:20:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4c13fff640 1 -- 172.21.15.26:0/3102111360 shutdown_connections 2024-03-06T07:20:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4c13fff640 1 --2- 172.21.15.26:0/3102111360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c0062c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4c13fff640 1 -- 172.21.15.26:0/3102111360 >> 172.21.15.26:0/3102111360 conn(0x7f4c0c04fc00 msgr2=0x7f4c0c050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4815d2f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4810156ad0 0x7f481006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4816530640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f48100c29b0 0x7f4810159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f481552e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f481015cbe0 0x7f48100c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4816530640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f48100c29b0 0x7f4810159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37046/0 (socket says 172.21.15.26:37046) 2024-03-06T07:20:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f4816530640 1 -- 172.21.15.26:0/1585816971 learned_addr learned my addr 172.21.15.26:0/1585816971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4c13fff640 1 -- 172.21.15.26:0/3102111360 shutdown_connections 2024-03-06T07:20:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee367fc640 1 -- 172.21.15.26:0/3866040790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2802d910 con 0x7fee380728a0 2024-03-06T07:20:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4816530640 1 -- 172.21.15.26:0/1585816971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f481015cbe0 msgr2=0x7f48100c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4816530640 1 --2- 172.21.15.26:0/1585816971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f481015cbe0 0x7f48100c2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4816530640 1 -- 172.21.15.26:0/1585816971 >> v2:172.21.15.26:3300/0 conn(0x7f4810156ad0 msgr2=0x7f481006e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4816530640 1 --2- 172.21.15.26:0/1585816971 >> v2:172.21.15.26:3300/0 conn(0x7f4810156ad0 0x7f481006e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4816530640 1 -- 172.21.15.26:0/1585816971 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481015d630 con 0x7f48100c29b0 2024-03-06T07:20:20.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 -- 172.21.15.26:0/3866040790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee380728a0 msgr2=0x7fee380712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 --2- 172.21.15.26:0/3866040790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee380728a0 0x7fee380712c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d7b0 tx=0x7fee2802bb50 comp rx=0 tx=0).stop 2024-03-06T07:20:20.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 -- 172.21.15.26:0/3866040790 shutdown_connections 2024-03-06T07:20:20.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 --2- 172.21.15.26:0/3866040790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee38152940 0x7fee38071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 --2- 172.21.15.26:0/3866040790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee380728a0 0x7fee380712c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 -- 172.21.15.26:0/3866040790 >> 172.21.15.26:0/3866040790 conn(0x7fee380c3ab0 msgr2=0x7fee380c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.657+0000 7f3151f15640 1 Processor -- start 2024-03-06T07:20:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f3151f15640 1 -- start start 2024-03-06T07:20:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 -- 172.21.15.26:0/3866040790 shutdown_connections 2024-03-06T07:20:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7fee3dac9640 1 -- 172.21.15.26:0/3866040790 wait complete. 2024-03-06T07:20:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a70016070 con 0x7f5a780a9650 2024-03-06T07:20:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f5a5f7fe640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5a6406b500 0x7f5a6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.658+0000 7f4c13fff640 1 -- 172.21.15.26:0/3102111360 wait complete. 2024-03-06T07:20:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5a7008a5b0 con 0x7f5a780a9650 2024-03-06T07:20:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f4815d2f640 1 --2- 172.21.15.26:0/1585816971 >> v2:172.21.15.26:3300/0 conn(0x7f4810156ad0 0x7f481006e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f5a76d9d640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5a6406b500 0x7f5a6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f5a5f7fe640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f5a64074c70 0x7f5a640770b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f106b7fe640 1 --2- 172.21.15.26:0/3919193550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c0c0320 0x7f106c15d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f106c0c9f00 tx=0x7f106002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274b9d9a0d82081f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151f15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c0039b0 0x7f3144177630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151f15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3144177b70 0x7f3144175680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f5a7759e640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f5a64074c70 0x7f5a640770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3150f13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c0039b0 0x7f3144177630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3150f13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c0039b0 0x7f3144177630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37048/0 (socket says 172.21.15.26:37048) 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151f15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3144175bc0 0x7f31440438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f314bfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3144177b70 0x7f3144175680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f314bfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3144177b70 0x7f3144175680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47192/0 (socket says 172.21.15.26:47192) 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151714640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3144175bc0 0x7f31440438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151714640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3144175bc0 0x7f31440438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46848/0 (socket says 172.21.15.26:46848) 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3150f13640 1 -- 172.21.15.26:0/234611862 learned_addr learned my addr 172.21.15.26:0/234611862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440dc0c0 con 0x7f3144175bc0 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3144047d80 con 0x7f313c0039b0 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440df5f0 con 0x7f3144177b70 2024-03-06T07:20:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f4816530640 1 --2- 172.21.15.26:0/1585816971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48100c29b0 0x7f4810159a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f481006d360 tx=0x7f480c02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8b66814bc25107 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 --> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a64077780 con 0x7f5a64074c70 2024-03-06T07:20:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f79c37fe640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009aa0 tx=0x7f79b4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bebe6c74148dc806 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f1069ffb640 1 -- 172.21.15.26:0/3919193550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600315a0 con 0x7f106c0c0320 2024-03-06T07:20:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f1069ffb640 1 -- 172.21.15.26:0/3919193550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060037070 con 0x7f106c0c0320 2024-03-06T07:20:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a70091050 con 0x7f5a780a9650 2024-03-06T07:20:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.659+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 shutdown_connections 2024-03-06T07:20:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f516aa17640 1 --2- 172.21.15.26:0/3870604057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5148003d30 0x7f5148004130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 >> 172.21.15.26:0/3870604057 conn(0x7f51640a95b0 msgr2=0x7f51640a9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 shutdown_connections 2024-03-06T07:20:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f4814d2d640 1 -- 172.21.15.26:0/1585816971 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c02c9f0 con 0x7f48100c29b0 2024-03-06T07:20:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f516aa17640 1 -- 172.21.15.26:0/3870604057 wait complete. 2024-03-06T07:20:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f4814d2d640 1 -- 172.21.15.26:0/1585816971 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c02cb90 con 0x7f48100c29b0 2024-03-06T07:20:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f4814d2d640 1 -- 172.21.15.26:0/1585816971 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c0343f0 con 0x7f48100c29b0 2024-03-06T07:20:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f5a7759e640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f5a64074c70 0x7f5a640770b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f4c13fff640 1 Processor -- start 2024-03-06T07:20:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f4c13fff640 1 -- start start 2024-03-06T07:20:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f5a76d9d640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5a6406b500 0x7f5a6406d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a68045000 tx=0x7f5a68008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8039070 con 0x7f79b40039b0 2024-03-06T07:20:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b802c6b0 con 0x7f79b40039b0 2024-03-06T07:20:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f79c27fc640 1 -- 172.21.15.26:0/975725745 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8009670 con 0x7f79b40039b0 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f3150f13640 1 -- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3144177b70 msgr2=0x7f3144175680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f3150f13640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3144177b70 0x7f3144175680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f3150f13640 1 -- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3144175bc0 msgr2=0x7f31440438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f3150f13640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3144175bc0 0x7f31440438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f3150f13640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31441780b0 con 0x7f313c0039b0 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f3150f13640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c0039b0 0x7f3144177630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f314c06a830 tx=0x7f314c069930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f3151714640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3144175bc0 0x7f31440438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0bf020 con 0x7f313c0039b0 2024-03-06T07:20:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314403f450 con 0x7f313c0039b0 2024-03-06T07:20:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4817f93640 1 -- 172.21.15.26:0/1585816971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48100c29b0 msgr2=0x7f4810159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4817f93640 1 --2- 172.21.15.26:0/1585816971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48100c29b0 0x7f4810159a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f481006d360 tx=0x7f480c02be50 comp rx=0 tx=0).stop 2024-03-06T07:20:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3144007570 con 0x7f313c0039b0 2024-03-06T07:20:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4817f93640 1 -- 172.21.15.26:0/1585816971 shutdown_connections 2024-03-06T07:20:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4817f93640 1 --2- 172.21.15.26:0/1585816971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f481015cbe0 0x7f48100c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4817f93640 1 --2- 172.21.15.26:0/1585816971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48100c29b0 0x7f4810159a40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4817f93640 1 --2- 172.21.15.26:0/1585816971 >> v2:172.21.15.26:3300/0 conn(0x7f4810156ad0 0x7f481006e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4817f93640 1 -- 172.21.15.26:0/1585816971 >> 172.21.15.26:0/1585816971 conn(0x7f481006b1a0 msgr2=0x7f481006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c00fcf0 0x7f4c0c0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c173e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d7920 0x7f4c0c0d9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c13fff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c006f10 con 0x7f4c0c0d7920 2024-03-06T07:20:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c13fff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c04c2a0 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c13fff640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c00f4c0 con 0x7f4c0c050720 2024-03-06T07:20:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f1069ffb640 1 -- 172.21.15.26:0/3919193550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060031960 con 0x7f106c0c0320 2024-03-06T07:20:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f10716fb640 1 -- 172.21.15.26:0/3919193550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c0c0320 msgr2=0x7f106c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.660+0000 7f10716fb640 1 --2- 172.21.15.26:0/3919193550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c0c0320 0x7f106c15d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f106c0c9f00 tx=0x7f106002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:20.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c0bb400 con 0x7f313c0039b0 2024-03-06T07:20:20.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0ad040 con 0x7f313c0039b0 2024-03-06T07:20:20.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f79c9fa0640 1 -- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b40039b0 msgr2=0x7f79b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f79c9fa0640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009aa0 tx=0x7f79b4005830 comp rx=0 tx=0).stop 2024-03-06T07:20:20.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f79c9fa0640 1 -- 172.21.15.26:0/975725745 shutdown_connections 2024-03-06T07:20:20.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f79c9fa0640 1 --2- 172.21.15.26:0/975725745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c00fcf0 0x7f4c0c0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c00fcf0 0x7f4c0c0c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37062/0 (socket says 172.21.15.26:37062) 2024-03-06T07:20:20.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c12ffd640 1 -- 172.21.15.26:0/2462008551 learned_addr learned my addr 172.21.15.26:0/2462008551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f4c127fc640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== osd.0 v2:172.21.15.26:6800/1236614035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5a64077780 con 0x7f5a64074c70 2024-03-06T07:20:20.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7f79c9fa0640 1 -- 172.21.15.26:0/975725745 >> 172.21.15.26:0/975725745 conn(0x7f79bc04c660 msgr2=0x7f79bc04ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f10716fb640 1 -- 172.21.15.26:0/3919193550 shutdown_connections 2024-03-06T07:20:20.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f10716fb640 1 --2- 172.21.15.26:0/3919193550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c0c0320 0x7f106c15d360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f10716fb640 1 --2- 172.21.15.26:0/3919193550 >> v2:172.21.15.117:3300/0 conn(0x7f106c0c6d30 0x7f106c15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f10716fb640 1 -- 172.21.15.26:0/3919193550 >> 172.21.15.26:0/3919193550 conn(0x7f106c0cae70 msgr2=0x7f106c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c137fe640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d7920 0x7f4c0c0d9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c12ffd640 1 -- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 msgr2=0x7f4c0c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c12ffd640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c0c050720 0x7f4c0c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f516aa17640 1 Processor -- start 2024-03-06T07:20:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4817f93640 1 -- 172.21.15.26:0/1585816971 shutdown_connections 2024-03-06T07:20:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f10716fb640 1 -- 172.21.15.26:0/3919193550 shutdown_connections 2024-03-06T07:20:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f10716fb640 1 -- 172.21.15.26:0/3919193550 wait complete. 2024-03-06T07:20:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c12ffd640 1 -- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d7920 msgr2=0x7f4c0c0d9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c12ffd640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d7920 0x7f4c0c0d9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c12ffd640 1 -- 172.21.15.26:0/2462008551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c0348d0 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3144005390 con 0x7f313c0039b0 2024-03-06T07:20:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c12ffd640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c00fcf0 0x7f4c0c0c79d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c140500e0 tx=0x7f4c140736a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4817f93640 1 -- 172.21.15.26:0/1585816971 wait complete. 2024-03-06T07:20:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7fee3dac9640 1 Processor -- start 2024-03-06T07:20:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.661+0000 7fee3dac9640 1 -- start start 2024-03-06T07:20:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f79c9fa0640 1 -- 172.21.15.26:0/975725745 shutdown_connections 2024-03-06T07:20:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f79c9fa0640 1 -- 172.21.15.26:0/975725745 wait complete. 2024-03-06T07:20:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c140688e0 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1408f030 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c13fff640 1 -- 172.21.15.26:0/2462008551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c034bb0 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f516aa17640 1 -- start start 2024-03-06T07:20:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7fee3dac9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee38070b30 0x7fee38155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7fee3dac9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee380728a0 0x7fee381541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7fee3dac9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee38152940 0x7fee381546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.662+0000 7f4c13fff640 1 -- 172.21.15.26:0/2462008551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c0d4130 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee3dac9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380acf90 con 0x7fee38152940 2024-03-06T07:20:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee3dac9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38067210 con 0x7fee38070b30 2024-03-06T07:20:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee3dac9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee38153d10 con 0x7fee380728a0 2024-03-06T07:20:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14068b60 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee38152940 0x7fee381546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee38152940 0x7fee381546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46856/0 (socket says 172.21.15.26:46856) 2024-03-06T07:20:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 -- 172.21.15.26:0/1508066559 learned_addr learned my addr 172.21.15.26:0/1508066559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 -- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee380728a0 msgr2=0x7fee381541a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-06T07:20:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee380728a0 0x7fee381541a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f516aa17640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148003d30 0x7f51641539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 -- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee38070b30 msgr2=0x7fee38155970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-06T07:20:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f516aa17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5164156070 0x7f51641543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee38070b30 0x7fee38155970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.663+0000 7fee37fff640 1 -- 172.21.15.26:0/1508066559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee381533e0 con 0x7fee38152940 2024-03-06T07:20:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7fee37fff640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee38152940 0x7fee381546e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c3f0 tx=0x7fee2c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f79c9fa0640 1 Processor -- start 2024-03-06T07:20:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f5163fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148003d30 0x7f51641539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f5163fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148003d30 0x7f51641539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47208/0 (socket says 172.21.15.26:47208) 2024-03-06T07:20:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f4817f93640 1 Processor -- start 2024-03-06T07:20:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f10716fb640 1 Processor -- start 2024-03-06T07:20:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f10716fb640 1 -- start start 2024-03-06T07:20:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c017070 con 0x7fee38152940 2024-03-06T07:20:20.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f4817f93640 1 -- start start 2024-03-06T07:20:20.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f51637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5164156070 0x7f51641543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f51637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5164156070 0x7f51641543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46870/0 (socket says 172.21.15.26:46870) 2024-03-06T07:20:20.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f4c13fff640 1 -- 172.21.15.26:0/2462008551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bf4001030 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f5163fff640 1 -- 172.21.15.26:0/367257325 learned_addr learned my addr 172.21.15.26:0/367257325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f516aa17640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51641559b0 0x7f51640ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c005ce0 con 0x7fee38152940 2024-03-06T07:20:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0072d0 con 0x7fee38152940 2024-03-06T07:20:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee381536c0 con 0x7fee38152940 2024-03-06T07:20:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee38154d50 con 0x7fee38152940 2024-03-06T07:20:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51640c2ba0 con 0x7f5164156070 2024-03-06T07:20:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f5168fb4640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51641559b0 0x7f51640ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51640c2a20 con 0x7f51641559b0 2024-03-06T07:20:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51640c3c00 con 0x7f5148003d30 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f10716fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0acdd0 0x7f106c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f10716fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c0c0320 0x7f106c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f5163fff640 1 -- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51641559b0 msgr2=0x7f51640ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f5163fff640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51641559b0 0x7f51640ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f5163fff640 1 -- 172.21.15.26:0/367257325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5164156070 msgr2=0x7f51641543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f5163fff640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5164156070 0x7f51641543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f5163fff640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164153720 con 0x7f5148003d30 2024-03-06T07:20:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f51637fe640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5164156070 0x7f51641543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f10716fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0c6d30 0x7f106c1ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f10716fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c15d8a0 con 0x7f106c0c0320 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f10716fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c159570 con 0x7f106c0c6d30 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f10716fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c15da20 con 0x7f106c0acdd0 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0acdd0 0x7f106c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0acdd0 0x7f106c06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47218/0 (socket says 172.21.15.26:47218) 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 -- 172.21.15.26:0/125307778 learned_addr learned my addr 172.21.15.26:0/125307778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106b7fe640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0c6d30 0x7f106c1ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c14094660 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.664+0000 7f79c9fa0640 1 -- start start 2024-03-06T07:20:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f79c9fa0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b40039b0 0x7f79bc032bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f79c9fa0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79bc0d9fc0 0x7f79bc033650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f79c9fa0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79bc0d8950 0x7f79bc0d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f79c9fa0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc04e4b0 con 0x7f79b40039b0 2024-03-06T07:20:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f79c9fa0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0db270 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f79c9fa0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc00fdc0 con 0x7f79bc0d8950 2024-03-06T07:20:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f5163fff640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148003d30 0x7f51641539c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f515c00bcc0 tx=0x7f515c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f5168fb4640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51641559b0 0x7f51640ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 -- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0c6d30 msgr2=0x7f106c1ec1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0c6d30 0x7f106c1ec1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 -- 172.21.15.26:0/125307778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c0c0320 msgr2=0x7f106c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c0c0320 0x7f106c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.665+0000 7f106affd640 1 -- 172.21.15.26:0/125307778 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c06cb30 con 0x7f106c0acdd0 2024-03-06T07:20:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f314c0c2020 con 0x7f313c0039b0 2024-03-06T07:20:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f106affd640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0acdd0 0x7f106c06c5f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c3f0 tx=0x7f105c00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f79c2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79bc0d9fc0 0x7f79bc033650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f79c2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79bc0d9fc0 0x7f79bc033650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37084/0 (socket says 172.21.15.26:37084) 2024-03-06T07:20:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f79c2ffd640 1 -- 172.21.15.26:0/887991342 learned_addr learned my addr 172.21.15.26:0/887991342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c044020 con 0x7f5148003d30 2024-03-06T07:20:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c036e20 con 0x7f5148003d30 2024-03-06T07:20:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00b820 con 0x7f5148003d30 2024-03-06T07:20:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51640a7b90 con 0x7f5148003d30 2024-03-06T07:20:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f4bfb7fe640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0006b4d0 0x7f4c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f4c127fc640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0006b4d0 0x7f4c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4c14105580 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f31497fa640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f313006b500 0x7f313006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f79c37fe640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b40039b0 0x7f79bc032bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51641e7800 con 0x7f5148003d30 2024-03-06T07:20:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f314bfff640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f313006b500 0x7f313006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.666+0000 7f4c127fc640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0006b4d0 0x7f4c0006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c08036000 tx=0x7f4c08008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4bfb7fe640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4c00074c40 0x7f4c00077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 --> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c14094e50 con 0x7f4c00074c40 2024-03-06T07:20:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c1410c050 con 0x7f4c0c00fcf0 2024-03-06T07:20:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee3804f330 con 0x7fee38152940 2024-03-06T07:20:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f79c2ffd640 1 -- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79bc0d8950 msgr2=0x7f79bc0d8e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-06T07:20:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f79c2ffd640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79bc0d8950 0x7f79bc0d8e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f79c2ffd640 1 -- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b40039b0 msgr2=0x7f79bc032bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f79c2ffd640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b40039b0 0x7f79bc032bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f79c2ffd640 1 -- 172.21.15.26:0/887991342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0328e0 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4817f93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48100c29b0 0x7f48101efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f314c1274a0 con 0x7f313c0039b0 2024-03-06T07:20:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4c137fe640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4c00074c40 0x7f4c00077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f314bfff640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f313006b500 0x7f313006d9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f314402c3f0 tx=0x7f3140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4817f93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810156ad0 0x7f4810150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4817f93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48100c4610 0x7f48100c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f79c2ffd640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79bc0d9fc0 0x7f79bc033650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79c4069e90 tx=0x7f79c40736a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4817f93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c0b50 con 0x7f48100c29b0 2024-03-06T07:20:20.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4817f93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c09d0 con 0x7f4810156ad0 2024-03-06T07:20:20.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4817f93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4810156eb0 con 0x7f48100c4610 2024-03-06T07:20:20.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4815d2f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48100c29b0 0x7f48101efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f4815d2f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48100c29b0 0x7f48101efbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46894/0 (socket says 172.21.15.26:46894) 2024-03-06T07:20:20.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f31497fa640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f3130074c80 0x7f31300770c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f4815d2f640 1 -- 172.21.15.26:0/3469194554 learned_addr learned my addr 172.21.15.26:0/3469194554 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f481552e640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810156ad0 0x7f4810150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f4c137fe640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4c00074c40 0x7f4c00077080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f4816530640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48100c4610 0x7f48100c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c017070 con 0x7f106c0acdd0 2024-03-06T07:20:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c005ce0 con 0x7f106c0acdd0 2024-03-06T07:20:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0072f0 con 0x7f106c0acdd0 2024-03-06T07:20:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.667+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee2c01a070 con 0x7fee38152940 2024-03-06T07:20:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7fee1ffff640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee2406b5d0 0x7fee2406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fee2c089750 con 0x7fee38152940 2024-03-06T07:20:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7fee1ffff640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee24074cc0 0x7fee24077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 --> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee2c015e70 con 0x7fee24074cc0 2024-03-06T07:20:20.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee2c090050 con 0x7fee38152940 2024-03-06T07:20:20.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f481552e640 1 -- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48100c4610 msgr2=0x7f48100c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c40688f0 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f481552e640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48100c4610 0x7f48100c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f10716fb640 1 -- 172.21.15.26:0/125307778 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c14cd10 con 0x7f106c0acdd0 2024-03-06T07:20:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f10716fb640 1 -- 172.21.15.26:0/125307778 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c14d2d0 con 0x7f106c0acdd0 2024-03-06T07:20:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f481552e640 1 -- 172.21.15.26:0/3469194554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48100c29b0 msgr2=0x7f48101efbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc180ec0 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc1811a0 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c4068a90 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4094660 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f481552e640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48100c29b0 0x7f48101efbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f481552e640 1 -- 172.21.15.26:0/3469194554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100c4b10 con 0x7f4810156ad0 2024-03-06T07:20:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f481552e640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810156ad0 0x7f4810150fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f480000e880 tx=0x7f480000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7fee36ffd640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee24074cc0 0x7fee24077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f515c047020 con 0x7f5148003d30 2024-03-06T07:20:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5140001030 con 0x7f5148003d30 2024-03-06T07:20:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800017070 con 0x7f4810156ad0 2024-03-06T07:20:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7fee377fe640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee2406b5d0 0x7fee2406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== osd.3 v2:172.21.15.26:6816/1166415256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4c14094e50 con 0x7f4c00074c40 2024-03-06T07:20:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000c940 con 0x7f4810156ad0 2024-03-06T07:20:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800017070 con 0x7f4810156ad0 2024-03-06T07:20:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4810152e60 con 0x7f4810156ad0 2024-03-06T07:20:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f5160ff9640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f514c06b5d0 0x7f514c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4810151520 con 0x7f4810156ad0 2024-03-06T07:20:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f515c0ac9a0 con 0x7f5148003d30 2024-03-06T07:20:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f5160ff9640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f514c074d40 0x7f514c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.668+0000 7f3151714640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f3130074c80 0x7f31300770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3130077790 con 0x7f3130074c80 2024-03-06T07:20:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.669+0000 7fee36ffd640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee24074cc0 0x7fee24077100 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79a4001030 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f51637fe640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f514c06b5d0 0x7f514c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f515c03ce50 con 0x7f514c074d40 2024-03-06T07:20:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f515c0b3050 con 0x7f5148003d30 2024-03-06T07:20:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f5168fb4640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f514c074d40 0x7f514c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7fee377fe640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee2406b5d0 0x7fee2406da90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee28036000 tx=0x7fee28008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f314c12e050 con 0x7f313c0039b0 2024-03-06T07:20:20.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.670+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f105c01a070 con 0x7f106c0acdd0 2024-03-06T07:20:20.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f481004f330 con 0x7f4810156ad0 2024-03-06T07:20:20.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f10716fb640 1 -- 172.21.15.26:0/125307778 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f106c04f330 con 0x7f106c0acdd0 2024-03-06T07:20:20.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79c4094800 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f10537fe640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105806b5a0 0x7f105806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== osd.5 v2:172.21.15.117:6800/3827357056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fee2c015e70 con 0x7fee24074cc0 2024-03-06T07:20:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f5168fb4640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f514c074d40 0x7f514c077180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f106a7fc640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105806b5a0 0x7f105806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f51637fe640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f514c06b5d0 0x7f514c06da90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5150007fd0 tx=0x7f5150007d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f105c0898a0 con 0x7f106c0acdd0 2024-03-06T07:20:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f3151714640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f3130074c80 0x7f31300770c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== osd.4 v2:172.21.15.117:6801/1465648529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f515c03ce50 con 0x7f514c074d40 2024-03-06T07:20:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f480001a070 con 0x7f4810156ad0 2024-03-06T07:20:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f106a7fc640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105806b5a0 0x7f105806da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10640070b0 tx=0x7f1064007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f79affff640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79a806b5d0 0x7f79a806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f79c37fe640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79a806b5d0 0x7f79a806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.671+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f79c4105830 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f79affff640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f79a8074d40 0x7f79a8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f79c3fff640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f79a8074d40 0x7f79a8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 --> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79a8077850 con 0x7f79a8074d40 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79c410c050 con 0x7f79bc0d9fc0 2024-03-06T07:20:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f79c37fe640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79a806b5d0 0x7f79a806da90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f79b8036000 tx=0x7f79b8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f10537fe640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f1058074d10 0x7f1058077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== osd.1 v2:172.21.15.26:6805/2298328200 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3130077790 con 0x7f3130074c80 2024-03-06T07:20:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 --> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f105c010e50 con 0x7f1058074d10 2024-03-06T07:20:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105c090050 con 0x7f106c0acdd0 2024-03-06T07:20:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.672+0000 7f106b7fe640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f1058074d10 0x7f1058077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f48067fc640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f47f406b8d0 0x7f47f406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f480000f040 con 0x7f4810156ad0 2024-03-06T07:20:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f4815d2f640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f47f406b8d0 0x7f47f406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f48067fc640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7f47f4074f50 0x7f47f4077390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f79c3fff640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f79a8074d40 0x7f79a8077180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f106b7fe640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f1058074d10 0x7f1058077150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 --> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f4077a60 con 0x7f47f4074f50 2024-03-06T07:20:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4800091050 con 0x7f4810156ad0 2024-03-06T07:20:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.673+0000 7f4816530640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7f47f4074f50 0x7f47f4077390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.674+0000 7f4816530640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7f47f4074f50 0x7f47f4077390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.674+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== osd.6 v2:172.21.15.117:6816/1228719702 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f105c010e50 con 0x7f1058074d10 2024-03-06T07:20:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.674+0000 7f4815d2f640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f47f406b8d0 0x7f47f406dd90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4808009b20 tx=0x7f4808008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.674+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== osd.7 v2:172.21.15.117:6818/4144741400 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f47f4077a60 con 0x7f47f4074f50 2024-03-06T07:20:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.674+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== osd.2 v2:172.21.15.26:6824/3726964600 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f79a8077850 con 0x7f79a8074d40 2024-03-06T07:20:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.695+0000 7f5a7d0af640 1 -- 172.21.15.26:0/1169680420 --> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5a58002dd0 con 0x7f5a64074c70 2024-03-06T07:20:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.696+0000 7f5a5f7fe640 1 -- 172.21.15.26:0/1169680420 <== osd.0 v2:172.21.15.26:6800/1236614035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5a58002dd0 con 0x7f5a64074c70 2024-03-06T07:20:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 -- 172.21.15.26:0/1169680420 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f5a64074c70 msgr2=0x7f5a640770b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f5a64074c70 0x7f5a640770b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 -- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5a6406b500 msgr2=0x7f5a6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5a6406b500 0x7f5a6406d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a68045000 tx=0x7f5a68008030 comp rx=0 tx=0).stop 2024-03-06T07:20:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 -- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a780a9650 msgr2=0x7f5a78071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 --2- 172.21.15.26:0/1169680420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a780a9650 0x7f5a78071c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5a70007fd0 tx=0x7f5a7000ddb0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a7759e640 1 -- 172.21.15.26:0/1169680420 reap_dead start 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 -- 172.21.15.26:0/1169680420 shutdown_connections 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 -- 172.21.15.26:0/1169680420 >> 172.21.15.26:0/1169680420 conn(0x7f5a7806b410 msgr2=0x7f5a78163c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 -- 172.21.15.26:0/1169680420 shutdown_connections 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.697+0000 7f5a5d7fa640 1 -- 172.21.15.26:0/1169680420 wait complete. 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.702+0000 7f4c13fff640 1 -- 172.21.15.26:0/2462008551 --> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4bf4002dd0 con 0x7f4c00074c40 2024-03-06T07:20:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.702+0000 7f4bfb7fe640 1 -- 172.21.15.26:0/2462008551 <== osd.3 v2:172.21.15.26:6816/1166415256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4bf4002dd0 con 0x7f4c00074c40 2024-03-06T07:20:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.703+0000 7f4bf97fa640 1 -- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4c00074c40 msgr2=0x7f4c00077080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.703+0000 7f4bf97fa640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4c00074c40 0x7f4c00077080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.703+0000 7f4bf97fa640 1 -- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0006b4d0 msgr2=0x7f4c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4bf97fa640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0006b4d0 0x7f4c0006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c08036000 tx=0x7f4c08008030 comp rx=0 tx=0).stop 2024-03-06T07:20:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4bf97fa640 1 -- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c00fcf0 msgr2=0x7f4c0c0c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4bf97fa640 1 --2- 172.21.15.26:0/2462008551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c00fcf0 0x7f4c0c0c79d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c140500e0 tx=0x7f4c140736a0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4c137fe640 1 -- 172.21.15.26:0/2462008551 reap_dead start 2024-03-06T07:20:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4bf97fa640 1 -- 172.21.15.26:0/2462008551 shutdown_connections 2024-03-06T07:20:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4bf97fa640 1 -- 172.21.15.26:0/2462008551 >> 172.21.15.26:0/2462008551 conn(0x7f4c0c04fc00 msgr2=0x7f4c0c00ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4bf97fa640 1 -- 172.21.15.26:0/2462008551 shutdown_connections 2024-03-06T07:20:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.704+0000 7f4bf97fa640 1 -- 172.21.15.26:0/2462008551 wait complete. 2024-03-06T07:20:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.705+0000 7f10716fb640 1 -- 172.21.15.26:0/125307778 --> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f106c15ca30 con 0x7f1058074d10 2024-03-06T07:20:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.706+0000 7f10537fe640 1 -- 172.21.15.26:0/125307778 <== osd.6 v2:172.21.15.117:6816/1228719702 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f106c15ca30 con 0x7f1058074d10 2024-03-06T07:20:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.706+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 --> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee38140d70 con 0x7fee24074cc0 2024-03-06T07:20:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.706+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 --> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31441752c0 con 0x7f3130074c80 2024-03-06T07:20:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.706+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 --> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5140002dd0 con 0x7f514c074d40 2024-03-06T07:20:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.706+0000 7fee1ffff640 1 -- 172.21.15.26:0/1508066559 <== osd.5 v2:172.21.15.117:6800/3827357056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee38140d70 con 0x7fee24074cc0 2024-03-06T07:20:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.706+0000 7f10517fa640 1 -- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f1058074d10 msgr2=0x7f1058077150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.706+0000 7f10517fa640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f1058074d10 0x7f1058077150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f31497fa640 1 -- 172.21.15.26:0/234611862 <== osd.1 v2:172.21.15.26:6805/2298328200 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f31441752c0 con 0x7f3130074c80 2024-03-06T07:20:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 -- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105806b5a0 msgr2=0x7f105806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105806b5a0 0x7f105806da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10640070b0 tx=0x7f1064007040 comp rx=0 tx=0).stop 2024-03-06T07:20:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 -- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0acdd0 msgr2=0x7f106c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 --2- 172.21.15.26:0/125307778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0acdd0 0x7f106c06c5f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c3f0 tx=0x7f105c00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f106b7fe640 1 -- 172.21.15.26:0/125307778 reap_dead start 2024-03-06T07:20:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 -- 172.21.15.26:0/125307778 shutdown_connections 2024-03-06T07:20:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 -- 172.21.15.26:0/125307778 >> 172.21.15.26:0/125307778 conn(0x7f106c0cae70 msgr2=0x7f106c0c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f5160ff9640 1 -- 172.21.15.26:0/367257325 <== osd.4 v2:172.21.15.117:6801/1465648529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5140002dd0 con 0x7f514c074d40 2024-03-06T07:20:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee24074cc0 msgr2=0x7fee24077100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7fee3dac9640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee24074cc0 0x7fee24077100 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee2406b5d0 msgr2=0x7fee2406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7fee3dac9640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee2406b5d0 0x7fee2406da90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee28036000 tx=0x7fee28008030 comp rx=0 tx=0).stop 2024-03-06T07:20:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee38152940 msgr2=0x7fee381546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7fee3dac9640 1 --2- 172.21.15.26:0/1508066559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee38152940 0x7fee381546e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c3f0 tx=0x7fee2c001e70 comp rx=0 tx=0).stop 2024-03-06T07:20:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 -- 172.21.15.26:0/125307778 shutdown_connections 2024-03-06T07:20:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f10517fa640 1 -- 172.21.15.26:0/125307778 wait complete. 2024-03-06T07:20:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 --> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f481006e1a0 con 0x7f47f4074f50 2024-03-06T07:20:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f3130074c80 msgr2=0x7f31300770c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f3151f15640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f3130074c80 0x7f31300770c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7fee37fff640 1 -- 172.21.15.26:0/1508066559 reap_dead start 2024-03-06T07:20:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f514c074d40 msgr2=0x7f514c077180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f516aa17640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f514c074d40 0x7f514c077180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.707+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 --> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79a4002dd0 con 0x7f79a8074d40 2024-03-06T07:20:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 shutdown_connections 2024-03-06T07:20:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 >> 172.21.15.26:0/1508066559 conn(0x7fee380c3ab0 msgr2=0x7fee38073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f48067fc640 1 -- 172.21.15.26:0/3469194554 <== osd.7 v2:172.21.15.117:6818/4144741400 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f481006e1a0 con 0x7f47f4074f50 2024-03-06T07:20:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f313006b500 msgr2=0x7f313006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f313006b500 0x7f313006d9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f314402c3f0 tx=0x7f3140009040 comp rx=0 tx=0).stop 2024-03-06T07:20:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c0039b0 msgr2=0x7f3144177630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 --2- 172.21.15.26:0/234611862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c0039b0 0x7f3144177630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f314c06a830 tx=0x7f314c069930 comp rx=0 tx=0).stop 2024-03-06T07:20:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151714640 1 -- 172.21.15.26:0/234611862 reap_dead start 2024-03-06T07:20:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f514c06b5d0 msgr2=0x7f514c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f516aa17640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f514c06b5d0 0x7f514c06da90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5150007fd0 tx=0x7f5150007d80 comp rx=0 tx=0).stop 2024-03-06T07:20:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148003d30 msgr2=0x7f51641539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f516aa17640 1 --2- 172.21.15.26:0/367257325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5148003d30 0x7f51641539c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f515c00bcc0 tx=0x7f515c004ab0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f5168fb4640 1 -- 172.21.15.26:0/367257325 reap_dead start 2024-03-06T07:20:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 shutdown_connections 2024-03-06T07:20:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7fee3dac9640 1 -- 172.21.15.26:0/1508066559 wait complete. 2024-03-06T07:20:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f79affff640 1 -- 172.21.15.26:0/887991342 <== osd.2 v2:172.21.15.26:6824/3726964600 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f79a4002dd0 con 0x7f79a8074d40 2024-03-06T07:20:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 shutdown_connections 2024-03-06T07:20:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 >> 172.21.15.26:0/234611862 conn(0x7f3144048250 msgr2=0x7f31440dcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 shutdown_connections 2024-03-06T07:20:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 >> 172.21.15.26:0/367257325 conn(0x7f51640a95b0 msgr2=0x7f5164072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 shutdown_connections 2024-03-06T07:20:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f3151f15640 1 -- 172.21.15.26:0/234611862 wait complete. 2024-03-06T07:20:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 shutdown_connections 2024-03-06T07:20:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.708+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7f47f4074f50 msgr2=0x7f47f4077390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f516aa17640 1 -- 172.21.15.26:0/367257325 wait complete. 2024-03-06T07:20:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7f47f4074f50 0x7f47f4077390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f79a8074d40 msgr2=0x7f79a8077180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f79a8074d40 0x7f79a8077180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79a806b5d0 msgr2=0x7f79a806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79a806b5d0 0x7f79a806da90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f79b8036000 tx=0x7f79b8030000 comp rx=0 tx=0).stop 2024-03-06T07:20:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f47f406b8d0 msgr2=0x7f47f406dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f47f406b8d0 0x7f47f406dd90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4808009b20 tx=0x7f4808008040 comp rx=0 tx=0).stop 2024-03-06T07:20:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79bc0d9fc0 msgr2=0x7f79bc033650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 --2- 172.21.15.26:0/887991342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79bc0d9fc0 0x7f79bc033650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79c4069e90 tx=0x7f79c40736a0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810156ad0 msgr2=0x7f4810150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 --2- 172.21.15.26:0/3469194554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810156ad0 0x7f4810150fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f480000e880 tx=0x7f480000ed40 comp rx=0 tx=0).stop 2024-03-06T07:20:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4816530640 1 -- 172.21.15.26:0/3469194554 reap_dead start 2024-03-06T07:20:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c3fff640 1 -- 172.21.15.26:0/887991342 reap_dead start 2024-03-06T07:20:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 shutdown_connections 2024-03-06T07:20:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 >> 172.21.15.26:0/3469194554 conn(0x7f481006b1a0 msgr2=0x7f48100c3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 shutdown_connections 2024-03-06T07:20:20.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 >> 172.21.15.26:0/887991342 conn(0x7f79bc04c660 msgr2=0x7f79bc0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 shutdown_connections 2024-03-06T07:20:20.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.709+0000 7f4817f93640 1 -- 172.21.15.26:0/3469194554 wait complete. 2024-03-06T07:20:20.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.710+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 shutdown_connections 2024-03-06T07:20:20.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.710+0000 7f79c9fa0640 1 -- 172.21.15.26:0/887991342 wait complete. 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-03-06T07:20:20.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-06T07:20:20.782 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:20.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-06T07:20:20.783 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:20.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-06T07:20:20.784 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:20.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-06T07:20:20.785 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:20.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-06T07:20:20.786 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:20.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-06T07:20:20.787 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-03-06T07:20:20.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-06T07:20:20.788 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:20.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-06T07:20:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.992+0000 7f756bc84640 1 Processor -- start 2024-03-06T07:20:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.992+0000 7f756bc84640 1 -- start start 2024-03-06T07:20:20.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f1303fff640 1 Processor -- start 2024-03-06T07:20:20.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f1303fff640 1 -- start start 2024-03-06T07:20:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756bc84640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7564165f60 0x7f756408b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756bc84640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75640c3790 0x7f75640a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756bc84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75640a9b20 0x7f75640704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756bc84640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75640c29d0 con 0x7f75640c3790 2024-03-06T07:20:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756bc84640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75640c2cb0 con 0x7f7564165f60 2024-03-06T07:20:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756bc84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640c2f90 con 0x7f75640a9b20 2024-03-06T07:20:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756921f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75640c3790 0x7f75640a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756921f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75640c3790 0x7f75640a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46904/0 (socket says 172.21.15.26:46904) 2024-03-06T07:20:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.993+0000 7f756921f640 1 -- 172.21.15.26:0/773813504 learned_addr learned my addr 172.21.15.26:0/773813504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f7569a20640 1 --2- 172.21.15.26:0/773813504 >> v2:172.21.15.117:3300/0 conn(0x7f7564165f60 0x7f756408b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f756a221640 1 --2- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75640a9b20 0x7f75640704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f1303fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f13040c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f1303fff640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13040c79a0 con 0x7f13041569d0 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f1303fff640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1304159570 con 0x7f13040c5200 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f1303fff640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130415a330 con 0x7f13040c5640 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f13037fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f13040c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f13027fc640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f13040c5200 0x7f13040c9470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35986/0 (socket says 172.21.15.26:35986) 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f13037fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f13040c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47274/0 (socket says 172.21.15.26:47274) 2024-03-06T07:20:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f13027fc640 1 -- 172.21.15.26:0/3775192398 learned_addr learned my addr 172.21.15.26:0/3775192398 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f7569a20640 1 -- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75640a9b20 msgr2=0x7f75640704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f7569a20640 1 --2- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75640a9b20 0x7f75640704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f7569a20640 1 -- 172.21.15.26:0/773813504 >> v2:172.21.15.26:3300/0 conn(0x7f75640c3790 msgr2=0x7f75640a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f7569a20640 1 --2- 172.21.15.26:0/773813504 >> v2:172.21.15.26:3300/0 conn(0x7f75640c3790 0x7f75640a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.994+0000 7f7569a20640 1 -- 172.21.15.26:0/773813504 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640c3270 con 0x7f7564165f60 2024-03-06T07:20:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f51bfaa4640 1 Processor -- start 2024-03-06T07:20:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510558940 0 0) 0x7f1304159570 con 0x7f13040c5200 2024-03-06T07:20:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12ec003520 con 0x7f13040c5200 2024-03-06T07:20:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1921682825 0 0) 0x7f12ec003520 con 0x7f13040c5200 2024-03-06T07:20:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 msgr2=0x7f13040c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f13040c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 >> v1:172.21.15.26:6789/0 conn(0x7f13041569d0 legacy=0x7f13040c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304159850 con 0x7f13040c5200 2024-03-06T07:20:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f12fc0019f0 con 0x7f13040c5200 2024-03-06T07:20:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 >> v1:172.21.15.117:6789/0 conn(0x7f13040c5200 legacy=0x7f13040c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f51bfaa4640 1 -- start start 2024-03-06T07:20:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f12ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12ec003f60 0x7f12ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12ec004d40 0x7f12ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f05eaf87640 1 Processor -- start 2024-03-06T07:20:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7fe153d6c640 1 Processor -- start 2024-03-06T07:20:21.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130415a330 con 0x7f12ec003f60 2024-03-06T07:20:21.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00baa0 con 0x7f12ec004d40 2024-03-06T07:20:21.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f0c1a5e2640 1 Processor -- start 2024-03-06T07:20:21.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f0c1a5e2640 1 -- start start 2024-03-06T07:20:21.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7fe153d6c640 1 -- start start 2024-03-06T07:20:21.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fce78a99640 1 Processor -- start 2024-03-06T07:20:21.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fce78a99640 1 -- start start 2024-03-06T07:20:21.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f51bfaa4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51b80c1c10 0x7f51b806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fe153d6c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f51bd03f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51b80c1c10 0x7f51b806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fe153d6c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe14c0c8950 0x7fe14c0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.995+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00bca0 con 0x7f13040c5640 2024-03-06T07:20:21.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f51bd03f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51b80c1c10 0x7f51b806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46920/0 (socket says 172.21.15.26:46920) 2024-03-06T07:20:21.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f05eaf87640 1 -- start start 2024-03-06T07:20:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f51bd03f640 1 -- 172.21.15.26:0/1587914083 learned_addr learned my addr 172.21.15.26:0/1587914083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f05eaf87640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05e406b230 0x7f05e41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f05eaf87640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bb4b0 0x7f05e415a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fe151307640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe14c0c8950 0x7fe14c0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fe151307640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe14c0c8950 0x7fe14c0acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37124/0 (socket says 172.21.15.26:37124) 2024-03-06T07:20:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fe151307640 1 -- 172.21.15.26:0/1825954137 learned_addr learned my addr 172.21.15.26:0/1825954137 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c155130 con 0x7fe14c0ad2b0 2024-03-06T07:20:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c155440 con 0x7fe14c0c8950 2024-03-06T07:20:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c5d50 con 0x7fe14c0c69f0 2024-03-06T07:20:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f51bfaa4640 1 --2- 172.21.15.26:0/1587914083 >> v2:172.21.15.117:3300/0 conn(0x7f51b806dc30 0x7f51b815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.997+0000 7f05e9f85640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05e406b230 0x7f05e41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f05e9f85640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05e406b230 0x7f05e41594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46936/0 (socket says 172.21.15.26:46936) 2024-03-06T07:20:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f05e9f85640 1 -- 172.21.15.26:0/336019037 learned_addr learned my addr 172.21.15.26:0/336019037 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f05e9784640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bb4b0 0x7f05e415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f7569a20640 1 --2- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7564165f60 0x7f756408b070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f756406bfa0 tx=0x7f755c00a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fac86d3911c7a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e40bbc90 con 0x7f05e40bcd30 2024-03-06T07:20:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e40a6240 con 0x7f05e406b230 2024-03-06T07:20:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40bbf70 con 0x7f05e40bb4b0 2024-03-06T07:20:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c1a5e2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c1406eb50 0x7f0c1415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f1302ffd640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f12ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f13027fc640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12ec003f60 0x7f12ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f1302ffd640 1 -- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12ec004d40 msgr2=0x7f12ec0051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f1302ffd640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12ec004d40 0x7f12ec0051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f1302ffd640 1 -- 172.21.15.26:0/3775192398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12ec003f60 msgr2=0x7f12ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f1302ffd640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12ec003f60 0x7f12ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.996+0000 7f1302ffd640 1 -- 172.21.15.26:0/3775192398 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec003520 con 0x7f13040c5640 2024-03-06T07:20:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f1302ffd640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f12ec0039b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12f4003db0 tx=0x7f12ec005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3990267d675edc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f7568a1e640 1 -- 172.21.15.26:0/773813504 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c032070 con 0x7f7564165f60 2024-03-06T07:20:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f7568a1e640 1 -- 172.21.15.26:0/773813504 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c02cbc0 con 0x7f7564165f60 2024-03-06T07:20:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fce78a99640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce740c3bd0 0x7fce740731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fce78a99640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740c2270 con 0x7fce740c75b0 2024-03-06T07:20:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fce78a99640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740c4b60 con 0x7fce7406e5e0 2024-03-06T07:20:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fce78a99640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740c2020 con 0x7fce740c3bd0 2024-03-06T07:20:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b81522d0 con 0x7f51b8151ef0 2024-03-06T07:20:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b8155860 con 0x7f51b80c1c10 2024-03-06T07:20:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b80c2070 con 0x7f51b806dc30 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f51be041640 1 --2- 172.21.15.26:0/1587914083 >> v2:172.21.15.117:3300/0 conn(0x7f51b806dc30 0x7f51b815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c137fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c1406eb50 0x7f0c1415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c137fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c1406eb50 0x7f0c1415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46952/0 (socket says 172.21.15.26:46952) 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c137fe640 1 -- 172.21.15.26:0/2734844837 learned_addr learned my addr 172.21.15.26:0/2734844837 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/2734844837 >> v2:172.21.15.117:3300/0 conn(0x7f0c14070ca0 0x7f0c14153230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c140c2920 con 0x7f0c140acbb0 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c14066d00 con 0x7f0c1406eb50 2024-03-06T07:20:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c14066f50 con 0x7f0c14070ca0 2024-03-06T07:20:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe151b08640 1 --2- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4037070 con 0x7f13040c5640 2024-03-06T07:20:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f4032600 con 0x7f13040c5640 2024-03-06T07:20:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f1301ffb640 1 -- 172.21.15.26:0/3775192398 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4003980 con 0x7f13040c5640 2024-03-06T07:20:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe151307640 1 -- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 msgr2=0x7fe14c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe151307640 1 --2- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe151307640 1 -- 172.21.15.26:0/1825954137 >> v1:172.21.15.26:6789/0 conn(0x7fe14c0ad2b0 legacy=0x7fe14c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe151307640 1 -- 172.21.15.26:0/1825954137 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c0c6030 con 0x7fe14c0c8950 2024-03-06T07:20:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fe151b08640 1 --2- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7f7568a1e640 1 -- 172.21.15.26:0/773813504 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c036400 con 0x7f7564165f60 2024-03-06T07:20:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fce737fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce740c3bd0 0x7fce740731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fce737fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce740c3bd0 0x7fce740731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37134/0 (socket says 172.21.15.26:37134) 2024-03-06T07:20:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.998+0000 7fce737fe640 1 -- 172.21.15.26:0/20027717 learned_addr learned my addr 172.21.15.26:0/20027717 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f05e9784640 1 -- 172.21.15.26:0/336019037 >> v2:172.21.15.26:3300/0 conn(0x7f05e406b230 msgr2=0x7f05e41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f05e9784640 1 --2- 172.21.15.26:0/336019037 >> v2:172.21.15.26:3300/0 conn(0x7f05e406b230 0x7f05e41594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f05e9784640 1 -- 172.21.15.26:0/336019037 >> v1:172.21.15.26:6789/0 conn(0x7f05e40bcd30 legacy=0x7f05e4156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f05e9784640 1 -- 172.21.15.26:0/336019037 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40bc250 con 0x7f05e40bb4b0 2024-03-06T07:20:21.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f0c18b7f640 1 --2- 172.21.15.26:0/2734844837 >> v2:172.21.15.117:3300/0 conn(0x7f0c14070ca0 0x7f0c14153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813481081 0 0) 0x7fce740c4b60 con 0x7fce7406e5e0 2024-03-06T07:20:21.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5c003520 con 0x7fce7406e5e0 2024-03-06T07:20:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f756bc84640 1 -- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7564165f60 msgr2=0x7f756408b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f756bc84640 1 --2- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7564165f60 0x7f756408b070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f756406bfa0 tx=0x7f755c00a400 comp rx=0 tx=0).stop 2024-03-06T07:20:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7ff3d3fff640 1 Processor -- start 2024-03-06T07:20:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7ff3d3fff640 1 -- start start 2024-03-06T07:20:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f51be041640 1 -- 172.21.15.26:0/1587914083 >> v2:172.21.15.26:3300/0 conn(0x7f51b80c1c10 msgr2=0x7f51b806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f51be041640 1 --2- 172.21.15.26:0/1587914083 >> v2:172.21.15.26:3300/0 conn(0x7f51b80c1c10 0x7f51b806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f51be041640 1 -- 172.21.15.26:0/1587914083 >> v1:172.21.15.26:6789/0 conn(0x7f51b8151ef0 legacy=0x7f51b80aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f51be041640 1 -- 172.21.15.26:0/1587914083 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b8154c50 con 0x7f51b806dc30 2024-03-06T07:20:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f756bc84640 1 -- 172.21.15.26:0/773813504 shutdown_connections 2024-03-06T07:20:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f756bc84640 1 --2- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75640a9b20 0x7f75640704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f756bc84640 1 --2- 172.21.15.26:0/773813504 >> v2:172.21.15.26:3300/0 conn(0x7f75640c3790 0x7f75640a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f756bc84640 1 --2- 172.21.15.26:0/773813504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7564165f60 0x7f756408b070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f756bc84640 1 -- 172.21.15.26:0/773813504 >> 172.21.15.26:0/773813504 conn(0x7f7564067120 msgr2=0x7f756406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f1303fff640 1 -- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 msgr2=0x7f12ec0039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f1303fff640 1 --2- 172.21.15.26:0/3775192398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13040c5640 0x7f12ec0039b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12f4003db0 tx=0x7f12ec005830 comp rx=0 tx=0).stop 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f13037fe640 1 -- 172.21.15.26:0/3775192398 reap_dead start 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f1303fff640 1 -- 172.21.15.26:0/3775192398 shutdown_connections 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f1303fff640 1 -- 172.21.15.26:0/3775192398 >> 172.21.15.26:0/3775192398 conn(0x7f13040ac410 msgr2=0x7f13040ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f1303fff640 1 -- 172.21.15.26:0/3775192398 shutdown_connections 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f1303fff640 1 -- 172.21.15.26:0/3775192398 wait complete. 2024-03-06T07:20:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f756bc84640 1 -- 172.21.15.26:0/773813504 shutdown_connections 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2366453458 0 0) 0x7fce5c003520 con 0x7fce7406e5e0 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 >> v2:172.21.15.117:3300/0 conn(0x7fce740c3bd0 msgr2=0x7fce740731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 --2- 172.21.15.26:0/20027717 >> v2:172.21.15.117:3300/0 conn(0x7fce740c3bd0 0x7fce740731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 >> v1:172.21.15.26:6789/0 conn(0x7fce740c75b0 legacy=0x7fce7406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740c4e40 con 0x7fce7406e5e0 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fce64001670 con 0x7fce7406e5e0 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 >> v1:172.21.15.117:6789/0 conn(0x7fce7406e5e0 legacy=0x7fce74155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce5c0039b0 0x7fce5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c004800 0x7fce5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0051c0 0x7fce5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce740c2020 con 0x7fce5c0051c0 2024-03-06T07:20:21.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3cc0decd0 0x7ff3cc0e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d3fff640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3cc007040 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d3fff640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3cc006df0 con 0x7ff3cc0decd0 2024-03-06T07:20:21.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d3fff640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc007320 con 0x7ff3cc0d3aa0 2024-03-06T07:20:21.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f05e9784640 1 --2- 172.21.15.26:0/336019037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bb4b0 0x7f05e415a990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05d40097a0 tx=0x7f05d402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cb9a0b5cae6fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:20.999+0000 7f0c18b7f640 1 -- 172.21.15.26:0/2734844837 >> v2:172.21.15.26:3300/0 conn(0x7f0c1406eb50 msgr2=0x7f0c1415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f0c18b7f640 1 --2- 172.21.15.26:0/2734844837 >> v2:172.21.15.26:3300/0 conn(0x7f0c1406eb50 0x7f0c1415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f0c18b7f640 1 -- 172.21.15.26:0/2734844837 >> v1:172.21.15.26:6789/0 conn(0x7f0c140acbb0 legacy=0x7f0c140c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f0c18b7f640 1 -- 172.21.15.26:0/2734844837 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14159780 con 0x7f0c14070ca0 2024-03-06T07:20:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f756bc84640 1 -- 172.21.15.26:0/773813504 wait complete. 2024-03-06T07:20:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c00c010 con 0x7fce5c0039b0 2024-03-06T07:20:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce737fe640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce5c0039b0 0x7fce5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f05e8f83640 1 -- 172.21.15.26:0/336019037 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4039070 con 0x7f05e40bb4b0 2024-03-06T07:20:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f05e8f83640 1 -- 172.21.15.26:0/336019037 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40316f0 con 0x7f05e40bb4b0 2024-03-06T07:20:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7f05e8f83640 1 -- 172.21.15.26:0/336019037 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4031ac0 con 0x7f05e40bb4b0 2024-03-06T07:20:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47332/0 (socket says 172.21.15.26:47332) 2024-03-06T07:20:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d2ffd640 1 -- 172.21.15.26:0/2620062177 learned_addr learned my addr 172.21.15.26:0/2620062177 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7ff3d37fe640 1 --2- 172.21.15.26:0/2620062177 >> v2:172.21.15.26:3300/0 conn(0x7ff3cc0decd0 0x7ff3cc0e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fce72ffd640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c004800 0x7fce5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fce73fff640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0051c0 0x7fce5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c00c210 con 0x7fce5c004800 2024-03-06T07:20:21.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f51be041640 1 --2- 172.21.15.26:0/1587914083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b815bc30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51b806c470 tx=0x7f51b402fda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f825d05d17ff6e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.000+0000 7fe151307640 1 --2- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0c8950 0x7fe14c0acd70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009870 tx=0x7fe13c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ac08082fd69f506 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fe150b06640 1 -- 172.21.15.26:0/1825954137 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0317c0 con 0x7fe14c0c8950 2024-03-06T07:20:21.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fe150b06640 1 -- 172.21.15.26:0/1825954137 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c037070 con 0x7fe14c0c8950 2024-03-06T07:20:21.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fe150b06640 1 -- 172.21.15.26:0/1825954137 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c031b80 con 0x7fe14c0c8950 2024-03-06T07:20:21.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7ff3d2ffd640 1 -- 172.21.15.26:0/2620062177 >> v2:172.21.15.26:3300/0 conn(0x7ff3cc0decd0 msgr2=0x7ff3cc0e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7ff3d2ffd640 1 --2- 172.21.15.26:0/2620062177 >> v2:172.21.15.26:3300/0 conn(0x7ff3cc0decd0 0x7ff3cc0e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f51bc83e640 1 -- 172.21.15.26:0/1587914083 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b403d070 con 0x7f51b806dc30 2024-03-06T07:20:21.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7ff3d2ffd640 1 -- 172.21.15.26:0/2620062177 >> v1:172.21.15.26:6789/0 conn(0x7ff3cc0d6d00 legacy=0x7ff3cc046590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7ff3d2ffd640 1 -- 172.21.15.26:0/2620062177 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc0d6500 con 0x7ff3cc0d3aa0 2024-03-06T07:20:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f51bc83e640 1 -- 172.21.15.26:0/1587914083 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b4030b50 con 0x7f51b806dc30 2024-03-06T07:20:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f51bc83e640 1 -- 172.21.15.26:0/1587914083 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4038440 con 0x7f51b806dc30 2024-03-06T07:20:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f0c18b7f640 1 --2- 172.21.15.26:0/2734844837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c14070ca0 0x7f0c14153230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0c14075060 tx=0x7f0c0c02bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81efd8524b08ce87 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bb4b0 msgr2=0x7f05e415a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f05eaf87640 1 --2- 172.21.15.26:0/336019037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bb4b0 0x7f05e415a990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05d40097a0 tx=0x7f05d402b790 comp rx=0 tx=0).stop 2024-03-06T07:20:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f1303fff640 1 Processor -- start 2024-03-06T07:20:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7f756bc84640 1 Processor -- start 2024-03-06T07:20:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f756bc84640 1 -- start start 2024-03-06T07:20:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0c8950 msgr2=0x7fe14c0acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7fe153d6c640 1 --2- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0c8950 0x7fe14c0acd70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009870 tx=0x7fe13c02b860 comp rx=0 tx=0).stop 2024-03-06T07:20:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f1303fff640 1 -- start start 2024-03-06T07:20:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f0c12ffd640 1 -- 172.21.15.26:0/2734844837 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c039070 con 0x7f0c14070ca0 2024-03-06T07:20:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f0c12ffd640 1 -- 172.21.15.26:0/2734844837 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c02cb50 con 0x7f0c14070ca0 2024-03-06T07:20:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f0c12ffd640 1 -- 172.21.15.26:0/2734844837 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c034440 con 0x7f0c14070ca0 2024-03-06T07:20:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fce737fe640 1 -- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c004800 msgr2=0x7fce5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fce737fe640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c004800 0x7fce5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fce737fe640 1 -- 172.21.15.26:0/20027717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0051c0 msgr2=0x7fce5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fce737fe640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0051c0 0x7fce5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.001+0000 7fce737fe640 1 -- 172.21.15.26:0/20027717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c003520 con 0x7fce5c0039b0 2024-03-06T07:20:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7fce73fff640 1 -- 172.21.15.26:0/20027717 reap_dead start 2024-03-06T07:20:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 shutdown_connections 2024-03-06T07:20:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f05eaf87640 1 --2- 172.21.15.26:0/336019037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bb4b0 0x7f05e415a990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f05eaf87640 1 --2- 172.21.15.26:0/336019037 >> v2:172.21.15.26:3300/0 conn(0x7f05e406b230 0x7f05e41594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 >> 172.21.15.26:0/336019037 conn(0x7f05e40c3170 msgr2=0x7f05e40c3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 shutdown_connections 2024-03-06T07:20:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 shutdown_connections 2024-03-06T07:20:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f05eaf87640 1 -- 172.21.15.26:0/336019037 wait complete. 2024-03-06T07:20:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7ff3d2ffd640 1 --2- 172.21.15.26:0/2620062177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d7850 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400bb00 tx=0x7ff3c402dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28cd6d24ea3d2f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7fe153d6c640 1 --2- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0c8950 0x7fe14c0acd70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7fe153d6c640 1 --2- 172.21.15.26:0/1825954137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7ff3d1ffb640 1 -- 172.21.15.26:0/2620062177 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4034070 con 0x7ff3cc0d3aa0 2024-03-06T07:20:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7ff3d1ffb640 1 -- 172.21.15.26:0/2620062177 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c402ec50 con 0x7ff3cc0d3aa0 2024-03-06T07:20:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7ff3d1ffb640 1 -- 172.21.15.26:0/2620062177 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4038400 con 0x7ff3cc0d3aa0 2024-03-06T07:20:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 >> 172.21.15.26:0/1825954137 conn(0x7fe14c0cae70 msgr2=0x7fe14c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 msgr2=0x7f51b815bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.002+0000 7f51bfaa4640 1 --2- 172.21.15.26:0/1587914083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b815bc30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51b806c470 tx=0x7f51b402fda0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 shutdown_connections 2024-03-06T07:20:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f51bfaa4640 1 --2- 172.21.15.26:0/1587914083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b815bc30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f51bfaa4640 1 --2- 172.21.15.26:0/1587914083 >> v2:172.21.15.26:3300/0 conn(0x7f51b80c1c10 0x7f51b806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 >> 172.21.15.26:0/1587914083 conn(0x7f51b806b410 msgr2=0x7f51b806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1303fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13040c5640 0x7f13040c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756bc84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75640a9b20 0x7f75641fa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7fce737fe640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce5c0039b0 0x7fce5c003d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e1c0 tx=0x7fce5c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=645411cf856c03a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c14070ca0 msgr2=0x7f0c14153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/2734844837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c14070ca0 0x7f0c14153230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0c14075060 tx=0x7f0c0c02bda0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 shutdown_connections 2024-03-06T07:20:21.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1303fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13040c46d0 0x7f130415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756bc84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75640c3790 0x7f7564072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 shutdown_connections 2024-03-06T07:20:21.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/2734844837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c14070ca0 0x7f0c14153230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/2734844837 >> v2:172.21.15.26:3300/0 conn(0x7f0c1406eb50 0x7f0c1415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 >> 172.21.15.26:0/2734844837 conn(0x7f0c140c3bb0 msgr2=0x7f0c140c3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 shutdown_connections 2024-03-06T07:20:21.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756bc84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7564165f60 0x7f75641eb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756bc84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564070b80 con 0x7f75640c3790 2024-03-06T07:20:21.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f51bfaa4640 1 -- 172.21.15.26:0/1587914083 wait complete. 2024-03-06T07:20:21.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756bc84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564070a00 con 0x7f75640a9b20 2024-03-06T07:20:21.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756bc84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7564070d00 con 0x7f7564165f60 2024-03-06T07:20:21.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1302ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13040c5640 0x7f13040c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1302ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13040c5640 0x7f13040c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37154/0 (socket says 172.21.15.26:37154) 2024-03-06T07:20:21.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 -- 172.21.15.26:0/2620062177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 msgr2=0x7ff3cc0d7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 --2- 172.21.15.26:0/2620062177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d7850 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400bb00 tx=0x7ff3c402dea0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1303fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f130415cb10 0x7f13040bf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1303fff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1304159ea0 con 0x7f13040c46d0 2024-03-06T07:20:21.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1303fff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13040c7c50 con 0x7f13040c5640 2024-03-06T07:20:21.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1303fff640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1304156db0 con 0x7f130415cb10 2024-03-06T07:20:21.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f1302ffd640 1 -- 172.21.15.26:0/320715857 learned_addr learned my addr 172.21.15.26:0/320715857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f13027fc640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13040c46d0 0x7f130415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 -- 172.21.15.26:0/2620062177 shutdown_connections 2024-03-06T07:20:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 --2- 172.21.15.26:0/2620062177 >> v2:172.21.15.26:3300/0 conn(0x7ff3cc0decd0 0x7ff3cc0e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 --2- 172.21.15.26:0/2620062177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d7850 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 -- 172.21.15.26:0/2620062177 >> 172.21.15.26:0/2620062177 conn(0x7ff3cc0062d0 msgr2=0x7ff3cc0066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 -- 172.21.15.26:0/2620062177 shutdown_connections 2024-03-06T07:20:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f13037fe640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f130415cb10 0x7f13040bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7ff3d3fff640 1 -- 172.21.15.26:0/2620062177 wait complete. 2024-03-06T07:20:21.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756a221640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7564165f60 0x7f75641eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.003+0000 7f756921f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75640c3790 0x7f7564072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1302ffd640 1 -- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f130415cb10 msgr2=0x7f13040bf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1302ffd640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f130415cb10 0x7f13040bf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1302ffd640 1 -- 172.21.15.26:0/320715857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13040c46d0 msgr2=0x7f130415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1302ffd640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13040c46d0 0x7f130415c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f756a221640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7564165f60 0x7f75641eb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47348/0 (socket says 172.21.15.26:47348) 2024-03-06T07:20:21.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1302ffd640 1 -- 172.21.15.26:0/320715857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13041ef860 con 0x7f13040c5640 2024-03-06T07:20:21.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 shutdown_connections 2024-03-06T07:20:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/2734844837 wait complete. 2024-03-06T07:20:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1302ffd640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13040c5640 0x7f13040c4190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f12f4003db0 tx=0x7f12f40036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f05eaf87640 1 Processor -- start 2024-03-06T07:20:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f05eaf87640 1 -- start start 2024-03-06T07:20:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f12eb7fe640 1 -- 172.21.15.26:0/320715857 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4042020 con 0x7f13040c5640 2024-03-06T07:20:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130415a810 con 0x7f13040c5640 2024-03-06T07:20:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f12eb7fe640 1 -- 172.21.15.26:0/320715857 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f4031750 con 0x7f13040c5640 2024-03-06T07:20:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f12eb7fe640 1 -- 172.21.15.26:0/320715857 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4037040 con 0x7f13040c5640 2024-03-06T07:20:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130415cfc0 con 0x7f13040c5640 2024-03-06T07:20:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7fe153d6c640 1 -- 172.21.15.26:0/1825954137 wait complete. 2024-03-06T07:20:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f51bfaa4640 1 Processor -- start 2024-03-06T07:20:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f05eaf87640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e406b230 0x7f05e40c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7ff3d3fff640 1 Processor -- start 2024-03-06T07:20:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009cd0 con 0x7fce5c0039b0 2024-03-06T07:20:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c02caa0 con 0x7fce5c0039b0 2024-03-06T07:20:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7fce727fc640 1 -- 172.21.15.26:0/20027717 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0343f0 con 0x7fce5c0039b0 2024-03-06T07:20:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7fce78a99640 1 -- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce5c0039b0 msgr2=0x7fce5c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7fce78a99640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce5c0039b0 0x7fce5c003d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e1c0 tx=0x7fce5c005830 comp rx=0 tx=0).stop 2024-03-06T07:20:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f51bfaa4640 1 -- start start 2024-03-06T07:20:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f756a221640 1 -- 172.21.15.26:0/3216022104 learned_addr learned my addr 172.21.15.26:0/3216022104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f05eaf87640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e40bb4b0 0x7f05e40c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f05eaf87640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bcd30 0x7f05e41e4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05eaf87640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40bb910 con 0x7f05e40bb4b0 2024-03-06T07:20:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05eaf87640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e415aed0 con 0x7f05e406b230 2024-03-06T07:20:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05eaf87640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40bd1c0 con 0x7f05e40bcd30 2024-03-06T07:20:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.004+0000 7f7569a20640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75640a9b20 0x7f75641fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e40bb4b0 0x7f05e40c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05ea786640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bcd30 0x7f05e41e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e40bb4b0 0x7f05e40c23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47008/0 (socket says 172.21.15.26:47008) 2024-03-06T07:20:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 -- 172.21.15.26:0/295440671 learned_addr learned my addr 172.21.15.26:0/295440671 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05ea786640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bcd30 0x7f05e41e4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47368/0 (socket says 172.21.15.26:47368) 2024-03-06T07:20:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f7569a20640 1 -- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7564165f60 msgr2=0x7f75641eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f7569a20640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7564165f60 0x7f75641eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f0c1a5e2640 1 Processor -- start 2024-03-06T07:20:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f7569a20640 1 -- 172.21.15.26:0/3216022104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75640c3790 msgr2=0x7f7564072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f7569a20640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75640c3790 0x7f7564072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f7569a20640 1 -- 172.21.15.26:0/3216022104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75641faed0 con 0x7f75640a9b20 2024-03-06T07:20:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7f7569a20640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75640a9b20 0x7f75641fa990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f755c034000 tx=0x7f755c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f0c1a5e2640 1 -- start start 2024-03-06T07:20:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f755a7fc640 1 -- 172.21.15.26:0/3216022104 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c045070 con 0x7f75640a9b20 2024-03-06T07:20:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f756bc84640 1 -- 172.21.15.26:0/3216022104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75640c6e80 con 0x7f75640a9b20 2024-03-06T07:20:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f755a7fc640 1 -- 172.21.15.26:0/3216022104 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c041650 con 0x7f75640a9b20 2024-03-06T07:20:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 -- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bcd30 msgr2=0x7f05e41e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bcd30 0x7f05e41e4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 -- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e406b230 msgr2=0x7f05e40c1e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-06T07:20:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e406b230 0x7f05e40c1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f05e9784640 1 -- 172.21.15.26:0/295440671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e415b180 con 0x7f05e40bb4b0 2024-03-06T07:20:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f756bc84640 1 -- 172.21.15.26:0/3216022104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75641ec130 con 0x7f75640a9b20 2024-03-06T07:20:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7f755a7fc640 1 -- 172.21.15.26:0/3216022104 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c041a40 con 0x7f75640a9b20 2024-03-06T07:20:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.005+0000 7ff3d3fff640 1 -- start start 2024-03-06T07:20:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f05e9784640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e40bb4b0 0x7f05e40c23d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f05d40098c0 tx=0x7f05d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f05da7fc640 1 -- 172.21.15.26:0/295440671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4043020 con 0x7f05e40bb4b0 2024-03-06T07:20:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f12eb7fe640 1 -- 172.21.15.26:0/320715857 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12f4045020 con 0x7f13040c5640 2024-03-06T07:20:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f756bc84640 1 -- 172.21.15.26:0/3216022104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c005290 con 0x7f75640a9b20 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f51bfaa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b81ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f51bfaa4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b80c1c10 0x7f51b8072550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f51bfaa4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b8151ef0 0x7f51b8071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f51bd03f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b80c1c10 0x7f51b8072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f51bd03f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b80c1c10 0x7f51b8072550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47010/0 (socket says 172.21.15.26:47010) 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f51bd03f640 1 -- 172.21.15.26:0/2177566636 learned_addr learned my addr 172.21.15.26:0/2177566636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f51be041640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b8151ef0 0x7f51b8071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f51bfaa4640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81554f0 con 0x7f51b80c1c10 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f51bfaa4640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8155210 con 0x7f51b806dc30 2024-03-06T07:20:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f51bfaa4640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8154900 con 0x7f51b8151ef0 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f05eaf87640 1 -- 172.21.15.26:0/295440671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e41efd00 con 0x7f05e40bb4b0 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f05da7fc640 1 -- 172.21.15.26:0/295440671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4034690 con 0x7f05e40bb4b0 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f05da7fc640 1 -- 172.21.15.26:0/295440671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4039040 con 0x7f05e40bb4b0 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7f05eaf87640 1 -- 172.21.15.26:0/295440671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e41effe0 con 0x7f05e40bb4b0 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f05ea786640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bcd30 0x7f05e41e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f0c1a5e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c1406eb50 0x7f0c1415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c1406eb50 0x7f0c1415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c1406eb50 0x7f0c1415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47406/0 (socket says 172.21.15.26:47406) 2024-03-06T07:20:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f0c1a5e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14070ca0 0x7f0c14154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f0c13fff640 1 -- 172.21.15.26:0/3562149047 learned_addr learned my addr 172.21.15.26:0/3562149047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7fe153d6c640 1 Processor -- start 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.006+0000 7fe153d6c640 1 -- start start 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe153d6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0ad2b0 0x7fe14c0ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe153d6c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe153d6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14c0c8950 0x7fe14c149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe153d6c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c152a20 con 0x7fe14c0c8950 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe153d6c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c155fe0 con 0x7fe14c0ad2b0 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe153d6c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c152ba0 con 0x7fe14c0c69f0 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe151b08640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0ad2b0 0x7fe14c0ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7fe152309640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14c0c8950 0x7fe14c149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7fe151307640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7fe151307640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c148cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47374/0 (socket says 172.21.15.26:47374) 2024-03-06T07:20:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7fe151307640 1 -- 172.21.15.26:0/2530529572 learned_addr learned my addr 172.21.15.26:0/2530529572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f755a7fc640 1 -- 172.21.15.26:0/3216022104 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f755c048070 con 0x7f75640a9b20 2024-03-06T07:20:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f0c137fe640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14070ca0 0x7f0c14154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c140acbb0 0x7f0c1414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3cc0d6d00 0x7ff3cc0c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc0decd0 0x7ff3cc0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7ff3d3fff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc046e20 con 0x7ff3cc0decd0 2024-03-06T07:20:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7ff3d3fff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc046ca0 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fe151307640 1 -- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0ad2b0 msgr2=0x7fe14c0ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fe151307640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0ad2b0 0x7fe14c0ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fe151307640 1 -- 172.21.15.26:0/2530529572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14c0c8950 msgr2=0x7fe14c149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fe151307640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14c0c8950 0x7fe14c149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fe151307640 1 -- 172.21.15.26:0/2530529572 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c0c4dd0 con 0x7fe14c0c69f0 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fe151b08640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0ad2b0 0x7fe14c0ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fe152309640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14c0c8950 0x7fe14c149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.007+0000 7ff3d3fff640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc009a50 con 0x7ff3cc0d3aa0 2024-03-06T07:20:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13040c8450 con 0x7f13040c5640 2024-03-06T07:20:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f12eb7fe640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12f006b500 0x7f12f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f755a7fc640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b5a0 0x7f754806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f12eb7fe640 1 -- 172.21.15.26:0/320715857 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f12f40abf00 con 0x7f13040c5640 2024-03-06T07:20:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7ff3d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3cc0d6d00 0x7ff3cc0c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7ff3d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3cc0d6d00 0x7ff3cc0c5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37166/0 (socket says 172.21.15.26:37166) 2024-03-06T07:20:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f0c18b7f640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c140acbb0 0x7f0c1414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f13027fc640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12f006b500 0x7f12f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14067230 con 0x7f0c14070ca0 2024-03-06T07:20:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14159d40 con 0x7f0c140acbb0 2024-03-06T07:20:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f755a7fc640 1 -- 172.21.15.26:0/3216022104 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f755c0acfb0 con 0x7f75640a9b20 2024-03-06T07:20:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14073200 con 0x7f0c1406eb50 2024-03-06T07:20:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.008+0000 7f51bd840640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b81ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7ff3d27fc640 1 -- 172.21.15.26:0/2338664118 learned_addr learned my addr 172.21.15.26:0/2338664118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7ff3d37fe640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc0decd0 0x7ff3cc0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f756921f640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b5a0 0x7f754806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7ff3d2ffd640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f05da7fc640 1 -- 172.21.15.26:0/295440671 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05d4046020 con 0x7f05e40bb4b0 2024-03-06T07:20:21.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f51bd840640 1 -- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b8151ef0 msgr2=0x7f51b8071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f51bd840640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b8151ef0 0x7f51b8071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fce78a99640 1 -- 172.21.15.26:0/20027717 shutdown_connections 2024-03-06T07:20:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fce78a99640 1 --2- 172.21.15.26:0/20027717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce5c0039b0 0x7fce5c003d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fce78a99640 1 -- 172.21.15.26:0/20027717 >> 172.21.15.26:0/20027717 conn(0x7fce7406da00 msgr2=0x7fce7406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fce78a99640 1 -- 172.21.15.26:0/20027717 shutdown_connections 2024-03-06T07:20:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7fce78a99640 1 -- 172.21.15.26:0/20027717 wait complete. 2024-03-06T07:20:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f51bd840640 1 -- 172.21.15.26:0/2177566636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b80c1c10 msgr2=0x7f51b8072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f51bd840640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b80c1c10 0x7f51b8072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7ff3d27fc640 1 -- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 msgr2=0x7ff3cc0d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f05eaf87640 1 -- 172.21.15.26:0/295440671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc005290 con 0x7f05e40bb4b0 2024-03-06T07:20:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7ff3d27fc640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7ff3d27fc640 1 -- 172.21.15.26:0/2338664118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc0decd0 msgr2=0x7ff3cc0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7ff3d27fc640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc0decd0 0x7ff3cc0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f05da7fc640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05c806b4d0 0x7f05c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7fe151307640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c148cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009870 tx=0x7fe13c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7ff3d27fc640 1 -- 172.21.15.26:0/2338664118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc02c7c0 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f756921f640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b5a0 0x7f754806da60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7554009fd0 tx=0x7f7554008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7ff3d2ffd640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.009+0000 7f51bd840640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b80708b0 con 0x7f51b806dc30 2024-03-06T07:20:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f51bd03f640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b80c1c10 0x7f51b8072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f51be041640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b8151ef0 0x7f51b8071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f05da7fc640 1 -- 172.21.15.26:0/295440671 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f05d40abaa0 con 0x7f05e40bb4b0 2024-03-06T07:20:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7fe1427fc640 1 -- 172.21.15.26:0/2530529572 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c032860 con 0x7fe14c0c69f0 2024-03-06T07:20:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7ff3d27fc640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3cc0d6d00 0x7ff3cc0c5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c480 tx=0x7ff3c800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f05e9f85640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05c806b4d0 0x7f05c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f13027fc640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12f006b500 0x7f12f006d9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1304159390 tx=0x7f12fc006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7fe1427fc640 1 -- 172.21.15.26:0/2530529572 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c032a20 con 0x7fe14c0c69f0 2024-03-06T07:20:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c1591f0 con 0x7fe14c0c69f0 2024-03-06T07:20:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f0c13fff640 1 -- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c140acbb0 msgr2=0x7f0c1414f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f0c13fff640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c140acbb0 0x7f0c1414f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f0c13fff640 1 -- 172.21.15.26:0/3562149047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14070ca0 msgr2=0x7f0c14154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f0c13fff640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14070ca0 0x7f0c14154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c1f2da0 con 0x7fe14c0c69f0 2024-03-06T07:20:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f0c13fff640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14156650 con 0x7f0c1406eb50 2024-03-06T07:20:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.010+0000 7f0c13fff640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c1406eb50 0x7f0c1415cdc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c430 tx=0x7f0c0400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7f05e9f85640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05c806b4d0 0x7f05c806d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099e0 tx=0x7f05e0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7fe1427fc640 1 -- 172.21.15.26:0/2530529572 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c032e10 con 0x7fe14c0c69f0 2024-03-06T07:20:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7f0c10ff9640 1 -- 172.21.15.26:0/3562149047 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c040052d0 con 0x7f0c1406eb50 2024-03-06T07:20:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7f0c10ff9640 1 -- 172.21.15.26:0/3562149047 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04005470 con 0x7f0c1406eb50 2024-03-06T07:20:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7f0c10ff9640 1 -- 172.21.15.26:0/3562149047 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04007260 con 0x7f0c1406eb50 2024-03-06T07:20:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7f51bd840640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b81ea8e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c370 tx=0x7f51a800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c14156930 con 0x7f0c1406eb50 2024-03-06T07:20:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c14156ef0 con 0x7f0c1406eb50 2024-03-06T07:20:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7fce78a99640 1 Processor -- start 2024-03-06T07:20:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.011+0000 7fce78a99640 1 -- start start 2024-03-06T07:20:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe134005290 con 0x7fe14c0c69f0 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce78a99640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c0039b0 0x7fce741547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce78a99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74155ca0 0x7fce7413d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce78a99640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce74154cf0 0x7fce7413de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce78a99640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74071240 con 0x7fce74155ca0 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce78a99640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74072950 con 0x7fce74154cf0 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce78a99640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce740713c0 con 0x7fce5c0039b0 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce737fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c0039b0 0x7fce741547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce737fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c0039b0 0x7fce741547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47416/0 (socket says 172.21.15.26:47416) 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce737fe640 1 -- 172.21.15.26:0/2148692087 learned_addr learned my addr 172.21.15.26:0/2148692087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce72ffd640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74155ca0 0x7fce7413d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce73fff640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce74154cf0 0x7fce7413de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1404f330 con 0x7f0c1406eb50 2024-03-06T07:20:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce73fff640 1 -- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c0039b0 msgr2=0x7fce741547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce73fff640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c0039b0 0x7fce741547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce73fff640 1 -- 172.21.15.26:0/2148692087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74155ca0 msgr2=0x7fce7413d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce73fff640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74155ca0 0x7fce7413d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce73fff640 1 -- 172.21.15.26:0/2148692087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7413f5f0 con 0x7fce74154cf0 2024-03-06T07:20:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.012+0000 7fce737fe640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c0039b0 0x7fce741547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.013+0000 7fce73fff640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce74154cf0 0x7fce7413de40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce74074790 tx=0x7fce6800e820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.013+0000 7fe1427fc640 1 -- 172.21.15.26:0/2530529572 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe13c03b3f0 con 0x7fe14c0c69f0 2024-03-06T07:20:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.013+0000 7f0c10ff9640 1 -- 172.21.15.26:0/3562149047 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c04007400 con 0x7f0c1406eb50 2024-03-06T07:20:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.013+0000 7fe1427fc640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe13006b5a0 0x7fe13006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.013+0000 7fe151b08640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe13006b5a0 0x7fe13006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.013+0000 7f0c10ff9640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c0006b500 0x7f0c0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.013+0000 7fe1427fc640 1 -- 172.21.15.26:0/2530529572 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe13c0ac700 con 0x7fe14c0c69f0 2024-03-06T07:20:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.014+0000 7f0c10ff9640 1 -- 172.21.15.26:0/3562149047 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0c04089970 con 0x7f0c1406eb50 2024-03-06T07:20:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.014+0000 7f0c137fe640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c0006b500 0x7f0c0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.014+0000 7fe151b08640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe13006b5a0 0x7fe13006da60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe14c06d2b0 tx=0x7fe144007630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.014+0000 7f0c137fe640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c0006b500 0x7f0c0006d9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c14072d60 tx=0x7f0c08009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.016+0000 7f05da7fc640 1 -- 172.21.15.26:0/295440671 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f05d404d020 con 0x7f05e40bb4b0 2024-03-06T07:20:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.018+0000 7f755a7fc640 1 -- 172.21.15.26:0/3216022104 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f755c07bb60 con 0x7f75640a9b20 2024-03-06T07:20:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.018+0000 7fe1427fc640 1 -- 172.21.15.26:0/2530529572 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe13c040070 con 0x7fe14c0c69f0 2024-03-06T07:20:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.022+0000 7f0c10ff9640 1 -- 172.21.15.26:0/3562149047 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0c04057e60 con 0x7f0c1406eb50 2024-03-06T07:20:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7fce5bfff640 1 -- 172.21.15.26:0/2148692087 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68019070 con 0x7fce74154cf0 2024-03-06T07:20:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7f51ae7fc640 1 -- 172.21.15.26:0/2177566636 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80052d0 con 0x7f51b806dc30 2024-03-06T07:20:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7f51ae7fc640 1 -- 172.21.15.26:0/2177566636 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a8005490 con 0x7f51b806dc30 2024-03-06T07:20:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7fce78a99640 1 -- 172.21.15.26:0/2148692087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7413cee0 con 0x7fce74154cf0 2024-03-06T07:20:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7fce78a99640 1 -- 172.21.15.26:0/2148692087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce74073310 con 0x7fce74154cf0 2024-03-06T07:20:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7ff3bb7fe640 1 -- 172.21.15.26:0/2338664118 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8017070 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7ff3bb7fe640 1 -- 172.21.15.26:0/2338664118 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c80052d0 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7ff3bb7fe640 1 -- 172.21.15.26:0/2338664118 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8003830 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7ff3d3fff640 1 -- 172.21.15.26:0/2338664118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc045c10 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.027+0000 7f12eb7fe640 1 -- 172.21.15.26:0/320715857 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f13040c8450 con 0x7f13040c5640 2024-03-06T07:20:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.026+0000 7fce5bfff640 1 -- 172.21.15.26:0/2148692087 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68005320 con 0x7fce74154cf0 2024-03-06T07:20:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.027+0000 7fce5bfff640 1 -- 172.21.15.26:0/2148692087 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68007270 con 0x7fce74154cf0 2024-03-06T07:20:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.027+0000 7f51ae7fc640 1 -- 172.21.15.26:0/2177566636 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8007260 con 0x7f51b806dc30 2024-03-06T07:20:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.027+0000 7ff3d3fff640 1 -- 172.21.15.26:0/2338664118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc0d44b0 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.028+0000 7f51bfaa4640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b8072a90 con 0x7f51b806dc30 2024-03-06T07:20:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.028+0000 7f51bfaa4640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b8148510 con 0x7f51b806dc30 2024-03-06T07:20:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.029+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc0d4cb0 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.028+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce740693c0 con 0x7fce74154cf0 2024-03-06T07:20:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.029+0000 7fce5bfff640 1 -- 172.21.15.26:0/2148692087 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce6801c070 con 0x7fce74154cf0 2024-03-06T07:20:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.030+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b804f330 con 0x7f51b806dc30 2024-03-06T07:20:21.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.030+0000 7fce5bfff640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce6006b500 0x7fce6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.030+0000 7fce5bfff640 1 -- 172.21.15.26:0/2148692087 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fce68089510 con 0x7fce74154cf0 2024-03-06T07:20:21.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.030+0000 7fce737fe640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce6006b500 0x7fce6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.031+0000 7fce737fe640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce6006b500 0x7fce6006d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e1c0 tx=0x7fce6c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.031+0000 7ff3bb7fe640 1 -- 172.21.15.26:0/2338664118 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3c801a070 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.032+0000 7ff3bb7fe640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3c006b520 0x7ff3c006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.032+0000 7ff3bb7fe640 1 -- 172.21.15.26:0/2338664118 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff3c8089850 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.032+0000 7ff3d2ffd640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3c006b520 0x7ff3c006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.032+0000 7ff3d2ffd640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3c006b520 0x7ff3c006d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4039000 tx=0x7ff3c4034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.038+0000 7fce5bfff640 1 -- 172.21.15.26:0/2148692087 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fce680579f0 con 0x7fce74154cf0 2024-03-06T07:20:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.041+0000 7f51ae7fc640 1 -- 172.21.15.26:0/2177566636 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51a8007400 con 0x7f51b806dc30 2024-03-06T07:20:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.041+0000 7f51ae7fc640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f519c06b520 0x7f519c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.041+0000 7f51bd03f640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f519c06b520 0x7f519c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.042+0000 7f51bd03f640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f519c06b520 0x7f519c06d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51b00093e0 tx=0x7f51b0009150 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.047+0000 7f51ae7fc640 1 -- 172.21.15.26:0/2177566636 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f51a80563d0 con 0x7f51b806dc30 2024-03-06T07:20:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.047+0000 7ff3bb7fe640 1 -- 172.21.15.26:0/2338664118 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff3c8057db0 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.055+0000 7f51ae7fc640 1 -- 172.21.15.26:0/2177566636 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f51a8058a30 con 0x7f51b806dc30 2024-03-06T07:20:21.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.401+0000 7f756bc84640 1 -- 172.21.15.26:0/3216022104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f754c005580 con 0x7f75640a9b20 2024-03-06T07:20:21.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.402+0000 7f755a7fc640 1 -- 172.21.15.26:0/3216022104 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f755c07b460 con 0x7f75640a9b20 2024-03-06T07:20:21.403 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:21.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.405+0000 7f7547fff640 1 -- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b5a0 msgr2=0x7f754806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.405+0000 7f7547fff640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b5a0 0x7f754806da60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7554009fd0 tx=0x7f7554008040 comp rx=0 tx=0).stop 2024-03-06T07:20:21.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.405+0000 7f7547fff640 1 -- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75640a9b20 msgr2=0x7f75641fa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.405+0000 7f7547fff640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75640a9b20 0x7f75641fa990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f755c034000 tx=0x7f755c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.405+0000 7f7547fff640 1 -- 172.21.15.26:0/3216022104 shutdown_connections 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.406+0000 7f7547fff640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b5a0 0x7f754806da60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.406+0000 7f7547fff640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7564165f60 0x7f75641eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.406+0000 7f7547fff640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75640c3790 0x7f7564072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.406+0000 7f7547fff640 1 --2- 172.21.15.26:0/3216022104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75640a9b20 0x7f75641fa990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.406+0000 7f7547fff640 1 -- 172.21.15.26:0/3216022104 >> 172.21.15.26:0/3216022104 conn(0x7f7564067120 msgr2=0x7f7564162cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.406+0000 7f7547fff640 1 -- 172.21.15.26:0/3216022104 shutdown_connections 2024-03-06T07:20:21.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.406+0000 7f7547fff640 1 -- 172.21.15.26:0/3216022104 wait complete. 2024-03-06T07:20:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.414+0000 7f05eaf87640 1 -- 172.21.15.26:0/295440671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f05cc005580 con 0x7f05e40bb4b0 2024-03-06T07:20:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.415+0000 7f05da7fc640 1 -- 172.21.15.26:0/295440671 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05d407a5d0 con 0x7f05e40bb4b0 2024-03-06T07:20:21.416 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.419+0000 7f05c7fff640 1 -- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05c806b4d0 msgr2=0x7f05c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.419+0000 7f05c7fff640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05c806b4d0 0x7f05c806d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099e0 tx=0x7f05e0009240 comp rx=0 tx=0).stop 2024-03-06T07:20:21.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.419+0000 7f05c7fff640 1 -- 172.21.15.26:0/295440671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e40bb4b0 msgr2=0x7f05e40c23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.419+0000 7f05c7fff640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e40bb4b0 0x7f05e40c23d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f05d40098c0 tx=0x7f05d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.419+0000 7f05c7fff640 1 -- 172.21.15.26:0/295440671 shutdown_connections 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.420+0000 7f05c7fff640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05c806b4d0 0x7f05c806d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.420+0000 7f05c7fff640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05e40bcd30 0x7f05e41e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.420+0000 7f05c7fff640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e40bb4b0 0x7f05e40c23d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.420+0000 7f05c7fff640 1 --2- 172.21.15.26:0/295440671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e406b230 0x7f05e40c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.420+0000 7f05c7fff640 1 -- 172.21.15.26:0/295440671 >> 172.21.15.26:0/295440671 conn(0x7f05e40c3170 msgr2=0x7f05e4153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.420+0000 7f05c7fff640 1 -- 172.21.15.26:0/295440671 shutdown_connections 2024-03-06T07:20:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.420+0000 7f05c7fff640 1 -- 172.21.15.26:0/295440671 wait complete. 2024-03-06T07:20:21.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-03-06T07:20:21.424 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-03-06T07:20:21.439 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.444+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe134005580 con 0x7fe14c0c69f0 2024-03-06T07:20:21.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.444+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f51b81500c0 con 0x7f51b806dc30 2024-03-06T07:20:21.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.444+0000 7fe1427fc640 1 -- 172.21.15.26:0/2530529572 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe14c155130 con 0x7fe14c0c69f0 2024-03-06T07:20:21.446 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.445+0000 7f51ae7fc640 1 -- 172.21.15.26:0/2177566636 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51a8058330 con 0x7f51b806dc30 2024-03-06T07:20:21.447 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f519c06b520 msgr2=0x7f519c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f519c06b520 0x7f519c06d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51b00093e0 tx=0x7f51b0009150 comp rx=0 tx=0).stop 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 msgr2=0x7f51b81ea8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b81ea8e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c370 tx=0x7f51a800c830 comp rx=0 tx=0).stop 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 shutdown_connections 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f519c06b520 0x7f519c06d9e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b8151ef0 0x7f51b8071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b80c1c10 0x7f51b8072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 --2- 172.21.15.26:0/2177566636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b806dc30 0x7f51b81ea8e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 >> 172.21.15.26:0/2177566636 conn(0x7f51b806b410 msgr2=0x7f51b806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 shutdown_connections 2024-03-06T07:20:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7f519bfff640 1 -- 172.21.15.26:0/2177566636 wait complete. 2024-03-06T07:20:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.447+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe13006b5a0 msgr2=0x7fe13006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe13006b5a0 0x7fe13006da60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe14c06d2b0 tx=0x7fe144007630 comp rx=0 tx=0).stop 2024-03-06T07:20:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 msgr2=0x7fe14c148cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c148cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009870 tx=0x7fe13c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 shutdown_connections 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe13006b5a0 0x7fe13006da60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14c0c8950 0x7fe14c149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe14c0c69f0 0x7fe14c148cf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 --2- 172.21.15.26:0/2530529572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c0ad2b0 0x7fe14c0ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 >> 172.21.15.26:0/2530529572 conn(0x7fe14c0cae70 msgr2=0x7fe14c0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 shutdown_connections 2024-03-06T07:20:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.448+0000 7fe153d6c640 1 -- 172.21.15.26:0/2530529572 wait complete. 2024-03-06T07:20:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.449+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0c140ad440 con 0x7f0c1406eb50 2024-03-06T07:20:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.450+0000 7f0c10ff9640 1 -- 172.21.15.26:0/3562149047 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c0405ce20 con 0x7f0c1406eb50 2024-03-06T07:20:21.453 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-03-06T07:20:21.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.451+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f13040c4ab0 con 0x7f13040c5640 2024-03-06T07:20:21.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.452+0000 7f12eb7fe640 1 -- 172.21.15.26:0/320715857 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f13040c4ab0 con 0x7f13040c5640 2024-03-06T07:20:21.454 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-03-06T07:20:21.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.452+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff3cc0057d0 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.452+0000 7ff3bb7fe640 1 -- 172.21.15.26:0/2338664118 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3c805be50 con 0x7ff3cc0d6d00 2024-03-06T07:20:21.455 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:21.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c0006b500 msgr2=0x7f0c0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c0006b500 0x7f0c0006d9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c14072d60 tx=0x7f0c08009240 comp rx=0 tx=0).stop 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c1406eb50 msgr2=0x7f0c1415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c1406eb50 0x7f0c1415cdc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c430 tx=0x7f0c0400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 shutdown_connections 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c0006b500 0x7f0c0006d9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c140acbb0 0x7f0c1414f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14070ca0 0x7f0c14154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 --2- 172.21.15.26:0/3562149047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c1406eb50 0x7f0c1415cdc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 >> 172.21.15.26:0/3562149047 conn(0x7f0c140c3bb0 msgr2=0x7f0c1406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 shutdown_connections 2024-03-06T07:20:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.453+0000 7f0c1a5e2640 1 -- 172.21.15.26:0/3562149047 wait complete. 2024-03-06T07:20:21.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3c006b520 msgr2=0x7ff3c006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3c006b520 0x7ff3c006d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4039000 tx=0x7ff3c4034000 comp rx=0 tx=0).stop 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3cc0d6d00 msgr2=0x7ff3cc0c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3cc0d6d00 0x7ff3cc0c5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c480 tx=0x7ff3c800c880 comp rx=0 tx=0).stop 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 shutdown_connections 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3c006b520 0x7ff3c006d9e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc0decd0 0x7ff3cc0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3cc0d6d00 0x7ff3cc0c5200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 --2- 172.21.15.26:0/2338664118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3cc0d3aa0 0x7ff3cc0d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 >> 172.21.15.26:0/2338664118 conn(0x7ff3cc0062d0 msgr2=0x7ff3cc163dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 shutdown_connections 2024-03-06T07:20:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7ff3b97fa640 1 -- 172.21.15.26:0/2338664118 wait complete. 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12f006b500 msgr2=0x7f12f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7f1303fff640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12f006b500 0x7f12f006d9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1304159390 tx=0x7f12fc006040 comp rx=0 tx=0).stop 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13040c5640 msgr2=0x7f13040c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.455+0000 7f1303fff640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13040c5640 0x7f13040c4190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f12f4003db0 tx=0x7f12f40036f0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 shutdown_connections 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12f006b500 0x7f12f006d9c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f130415cb10 0x7f13040bf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13040c46d0 0x7f130415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 --2- 172.21.15.26:0/320715857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13040c5640 0x7f13040c4190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 >> 172.21.15.26:0/320715857 conn(0x7f13040ac410 msgr2=0x7f1304155f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 shutdown_connections 2024-03-06T07:20:21.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.456+0000 7f1303fff640 1 -- 172.21.15.26:0/320715857 wait complete. 2024-03-06T07:20:21.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.462+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fce740c6f20 con 0x7fce74154cf0 2024-03-06T07:20:21.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.462+0000 7fce5bfff640 1 -- 172.21.15.26:0/2148692087 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce6805ba90 con 0x7fce74154cf0 2024-03-06T07:20:21.464 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-03-06T07:20:21.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-03-06T07:20:21.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce6006b500 msgr2=0x7fce6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce6006b500 0x7fce6006d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e1c0 tx=0x7fce6c02e000 comp rx=0 tx=0).stop 2024-03-06T07:20:21.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce74154cf0 msgr2=0x7fce7413de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce74154cf0 0x7fce7413de40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce74074790 tx=0x7fce6800e820 comp rx=0 tx=0).stop 2024-03-06T07:20:21.468 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-03-06T07:20:21.469 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 shutdown_connections 2024-03-06T07:20:21.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce6006b500 0x7fce6006d9c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce74154cf0 0x7fce7413de40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74155ca0 0x7fce7413d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 --2- 172.21.15.26:0/2148692087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5c0039b0 0x7fce741547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 >> 172.21.15.26:0/2148692087 conn(0x7fce7406da00 msgr2=0x7fce7406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 shutdown_connections 2024-03-06T07:20:21.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.465+0000 7fce59ffb640 1 -- 172.21.15.26:0/2148692087 wait complete. 2024-03-06T07:20:21.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-03-06T07:20:21.471 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-03-06T07:20:21.474 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-03-06T07:20:21.477 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-03-06T07:20:21.481 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:21.481 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-06T07:20:21.482 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.621+0000 7f9d1ff1e640 1 Processor -- start 2024-03-06T07:20:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.621+0000 7f9d1ff1e640 1 -- start start 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1ff1e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9d18150580 0x7f9d1806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1ff1e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d180c1320 0x7f9d1806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1ff1e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d18153a60 con 0x7f9d180c6000 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1ff1e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d18153d40 con 0x7f9d180c1320 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1ff1e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1806c2c0 con 0x7f9d18150580 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1d4b9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d180c1320 0x7f9d1806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1e4bb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d180c6000 0x7f9d181566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44228/0 (socket says 172.21.15.26:44228) 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1e4bb640 1 -- 172.21.15.26:0/979305074 learned_addr learned my addr 172.21.15.26:0/979305074 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1dcba640 1 --2- 172.21.15.26:0/979305074 >> v2:172.21.15.117:3300/0 conn(0x7f9d18150580 0x7f9d1806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1ccb8640 1 -- 172.21.15.26:0/979305074 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703439332 0 0) 0x7f9d18153a60 con 0x7f9d180c6000 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1ccb8640 1 -- 172.21.15.26:0/979305074 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d00003520 con 0x7f9d180c6000 2024-03-06T07:20:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1d4b9640 1 -- 172.21.15.26:0/979305074 >> v2:172.21.15.117:3300/0 conn(0x7f9d18150580 msgr2=0x7f9d1806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1d4b9640 1 --2- 172.21.15.26:0/979305074 >> v2:172.21.15.117:3300/0 conn(0x7f9d18150580 0x7f9d1806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1d4b9640 1 -- 172.21.15.26:0/979305074 >> v1:172.21.15.26:6789/0 conn(0x7f9d180c6000 legacy=0x7f9d181566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.622+0000 7f9d1d4b9640 1 -- 172.21.15.26:0/979305074 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1806c5a0 con 0x7f9d180c1320 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.623+0000 7f9d1d4b9640 1 --2- 172.21.15.26:0/979305074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c1320 0x7f9d1806e250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d180a9550 tx=0x7f9d0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8723bfd05d4e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.623+0000 7f9d1ccb8640 1 -- 172.21.15.26:0/979305074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0315a0 con 0x7f9d180c1320 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.623+0000 7f9d1ccb8640 1 -- 172.21.15.26:0/979305074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c037070 con 0x7f9d180c1320 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.623+0000 7f9d1ccb8640 1 -- 172.21.15.26:0/979305074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c031960 con 0x7f9d180c1320 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/979305074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c1320 msgr2=0x7f9d1806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/979305074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c1320 0x7f9d1806e250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d180a9550 tx=0x7f9d0c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/979305074 shutdown_connections 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/979305074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c1320 0x7f9d1806e250 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/979305074 >> v2:172.21.15.117:3300/0 conn(0x7f9d18150580 0x7f9d1806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/979305074 >> 172.21.15.26:0/979305074 conn(0x7f9d180c6710 msgr2=0x7f9d180c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/979305074 shutdown_connections 2024-03-06T07:20:21.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.624+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/979305074 wait complete. 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.625+0000 7f9d1ff1e640 1 Processor -- start 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.625+0000 7f9d1ff1e640 1 -- start start 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1ff1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d180c1320 0x7f9d180c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1ff1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c6000 0x7f9d180a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1ff1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d18150580 0x7f9d18149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1ff1e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c0dc0 con 0x7f9d180c6000 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1ff1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c0c40 con 0x7f9d18150580 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1ff1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d18156dc0 con 0x7f9d180c1320 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c6000 0x7f9d180a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c6000 0x7f9d180a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47062/0 (socket says 172.21.15.26:47062) 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 -- 172.21.15.26:0/2457004063 learned_addr learned my addr 172.21.15.26:0/2457004063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1e4bb640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d18150580 0x7f9d18149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1dcba640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d180c1320 0x7f9d180c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 -- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d180c1320 msgr2=0x7f9d180c04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d180c1320 0x7f9d180c04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 -- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d18150580 msgr2=0x7f9d18149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d18150580 0x7f9d18149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 -- 172.21.15.26:0/2457004063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d181ecb70 con 0x7f9d180c6000 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d1d4b9640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c6000 0x7f9d180a7720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009890 tx=0x7f9d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d167fc640 1 -- 172.21.15.26:0/2457004063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c032630 con 0x7f9d180c6000 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d167fc640 1 -- 172.21.15.26:0/2457004063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c0327f0 con 0x7f9d180c6000 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.626+0000 7f9d167fc640 1 -- 172.21.15.26:0/2457004063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c032be0 con 0x7f9d180c6000 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.627+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d18072c40 con 0x7f9d180c6000 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.627+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d180a7c60 con 0x7f9d180c6000 2024-03-06T07:20:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.627+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1804f330 con 0x7f9d180c6000 2024-03-06T07:20:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.628+0000 7f9d167fc640 1 -- 172.21.15.26:0/2457004063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d0c04c020 con 0x7f9d180c6000 2024-03-06T07:20:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.628+0000 7f9d167fc640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d0406b4d0 0x7f9d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.628+0000 7f9d167fc640 1 -- 172.21.15.26:0/2457004063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9d0c02f020 con 0x7f9d180c6000 2024-03-06T07:20:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.628+0000 7f9d1dcba640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d0406b4d0 0x7f9d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.629+0000 7f9d1dcba640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d0406b4d0 0x7f9d0406d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009aa0 tx=0x7f9d10009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.632+0000 7f9d167fc640 1 -- 172.21.15.26:0/2457004063 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d0c040070 con 0x7f9d180c6000 2024-03-06T07:20:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.809+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d1814b930 con 0x7f9d0406b4d0 2024-03-06T07:20:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.825+0000 7f9d167fc640 1 -- 172.21.15.26:0/2457004063 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332611 (secure 0 0 0) 0x7f9d1814b930 con 0x7f9d0406b4d0 2024-03-06T07:20:21.832 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:20:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d0406b4d0 msgr2=0x7f9d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d0406b4d0 0x7f9d0406d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009aa0 tx=0x7f9d10009190 comp rx=0 tx=0).stop 2024-03-06T07:20:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c6000 msgr2=0x7f9d180a7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c6000 0x7f9d180a7720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009890 tx=0x7f9d0c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 shutdown_connections 2024-03-06T07:20:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d0406b4d0 0x7f9d0406d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d18150580 0x7f9d18149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180c6000 0x7f9d180a7720 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 --2- 172.21.15.26:0/2457004063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d180c1320 0x7f9d180c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 >> 172.21.15.26:0/2457004063 conn(0x7f9d180c6710 msgr2=0x7f9d180c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 shutdown_connections 2024-03-06T07:20:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.828+0000 7f9d1ff1e640 1 -- 172.21.15.26:0/2457004063 wait complete. 2024-03-06T07:20:21.841 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-03-06T07:20:21.034266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230820,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743900,"statfs":{"total":773094113280,"available":772857753600,"internally_reserved":0,"allocated":2228224,"data_stored":1992064,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002612"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241454+0000","last_change":"2024-03-06T07:20:00.821957+0000","last_active":"2024-03-06T07:20:07.241454+0000","last_peered":"2024-03-06T07:20:07.241454+0000","last_clean":"2024-03-06T07:20:07.241454+0000","last_became_active":"2024-03-06T07:20:00.821672+0000","last_became_peered":"2024-03-06T07:20:00.821672+0000","last_unstale":"2024-03-06T07:20:07.241454+0000","last_undegraded":"2024-03-06T07:20:07.241454+0000","last_fullsized":"2024-03-06T07:20:07.241454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242911+0000","last_change":"2024-03-06T07:20:01.819400+0000","last_active":"2024-03-06T07:20:07.242911+0000","last_peered":"2024-03-06T07:20:07.242911+0000","last_clean":"2024-03-06T07:20:07.242911+0000","last_became_active":"2024-03-06T07:20:01.819073+0000","last_became_peered":"2024-03-06T07:20:01.819073+0000","last_unstale":"2024-03-06T07:20:07.242911+0000","last_undegraded":"2024-03-06T07:20:07.242911+0000","last_fullsized":"2024-03-06T07:20:07.242911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242373+0000","last_change":"2024-03-06T07:20:06.127375+0000","last_active":"2024-03-06T07:20:07.242373+0000","last_peered":"2024-03-06T07:20:07.242373+0000","last_clean":"2024-03-06T07:20:07.242373+0000","last_became_active":"2024-03-06T07:20:06.126687+0000","last_became_peered":"2024-03-06T07:20:06.126687+0000","last_unstale":"2024-03-06T07:20:07.242373+0000","last_undegraded":"2024-03-06T07:20:07.242373+0000","last_fullsized":"2024-03-06T07:20:07.242373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:11.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243026+0000","last_change":"2024-03-06T07:20:00.816186+0000","last_active":"2024-03-06T07:20:07.243026+0000","last_peered":"2024-03-06T07:20:07.243026+0000","last_clean":"2024-03-06T07:20:07.243026+0000","last_became_active":"2024-03-06T07:20:00.815110+0000","last_became_peered":"2024-03-06T07:20:00.815110+0000","last_unstale":"2024-03-06T07:20:07.243026+0000","last_undegraded":"2024-03-06T07:20:07.243026+0000","last_fullsized":"2024-03-06T07:20:07.243026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241124+0000","last_change":"2024-03-06T07:20:01.835360+0000","last_active":"2024-03-06T07:20:07.241124+0000","last_peered":"2024-03-06T07:20:07.241124+0000","last_clean":"2024-03-06T07:20:07.241124+0000","last_became_active":"2024-03-06T07:20:01.835066+0000","last_became_peered":"2024-03-06T07:20:01.835066+0000","last_unstale":"2024-03-06T07:20:07.241124+0000","last_undegraded":"2024-03-06T07:20:07.241124+0000","last_fullsized":"2024-03-06T07:20:07.241124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240859+0000","last_change":"2024-03-06T07:20:05.864904+0000","last_active":"2024-03-06T07:20:07.240859+0000","last_peered":"2024-03-06T07:20:07.240859+0000","last_clean":"2024-03-06T07:20:07.240859+0000","last_became_active":"2024-03-06T07:20:05.864535+0000","last_became_peered":"2024-03-06T07:20:05.864535+0000","last_unstale":"2024-03-06T07:20:07.240859+0000","last_undegraded":"2024-03-06T07:20:07.240859+0000","last_fullsized":"2024-03-06T07:20:07.240859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.016032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242211+0000","last_change":"2024-03-06T07:20:00.823477+0000","last_active":"2024-03-06T07:20:07.242211+0000","last_peered":"2024-03-06T07:20:07.242211+0000","last_clean":"2024-03-06T07:20:07.242211+0000","last_became_active":"2024-03-06T07:20:00.822898+0000","last_became_peered":"2024-03-06T07:20:00.822898+0000","last_unstale":"2024-03-06T07:20:07.242211+0000","last_undegraded":"2024-03-06T07:20:07.242211+0000","last_fullsized":"2024-03-06T07:20:07.242211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:23.183380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.832744+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.832237+0000","last_became_peered":"2024-03-06T07:20:01.832237+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241395+0000","last_change":"2024-03-06T07:20:05.862022+0000","last_active":"2024-03-06T07:20:07.241395+0000","last_peered":"2024-03-06T07:20:07.241395+0000","last_clean":"2024-03-06T07:20:07.241395+0000","last_became_active":"2024-03-06T07:20:05.861661+0000","last_became_peered":"2024-03-06T07:20:05.861661+0000","last_unstale":"2024-03-06T07:20:07.241395+0000","last_undegraded":"2024-03-06T07:20:07.241395+0000","last_fullsized":"2024-03-06T07:20:07.241395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:30.602011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241314+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241314+0000","last_peered":"2024-03-06T07:20:07.241314+0000","last_clean":"2024-03-06T07:20:07.241314+0000","last_became_active":"2024-03-06T07:20:00.822334+0000","last_became_peered":"2024-03-06T07:20:00.822334+0000","last_unstale":"2024-03-06T07:20:07.241314+0000","last_undegraded":"2024-03-06T07:20:07.241314+0000","last_fullsized":"2024-03-06T07:20:07.241314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241506+0000","last_change":"2024-03-06T07:20:01.832671+0000","last_active":"2024-03-06T07:20:07.241506+0000","last_peered":"2024-03-06T07:20:07.241506+0000","last_clean":"2024-03-06T07:20:07.241506+0000","last_became_active":"2024-03-06T07:20:01.832397+0000","last_became_peered":"2024-03-06T07:20:01.832397+0000","last_unstale":"2024-03-06T07:20:07.241506+0000","last_undegraded":"2024-03-06T07:20:07.241506+0000","last_fullsized":"2024-03-06T07:20:07.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241835+0000","last_change":"2024-03-06T07:20:05.853058+0000","last_active":"2024-03-06T07:20:07.241835+0000","last_peered":"2024-03-06T07:20:07.241835+0000","last_clean":"2024-03-06T07:20:07.241835+0000","last_became_active":"2024-03-06T07:20:05.852648+0000","last_became_peered":"2024-03-06T07:20:05.852648+0000","last_unstale":"2024-03-06T07:20:07.241835+0000","last_undegraded":"2024-03-06T07:20:07.241835+0000","last_fullsized":"2024-03-06T07:20:07.241835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:33.136033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838904+0000","last_change":"2024-03-06T07:20:01.817989+0000","last_active":"2024-03-06T07:20:06.838904+0000","last_peered":"2024-03-06T07:20:06.838904+0000","last_clean":"2024-03-06T07:20:06.838904+0000","last_became_active":"2024-03-06T07:20:01.817711+0000","last_became_peered":"2024-03-06T07:20:01.817711+0000","last_unstale":"2024-03-06T07:20:06.838904+0000","last_undegraded":"2024-03-06T07:20:06.838904+0000","last_fullsized":"2024-03-06T07:20:06.838904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241768+0000","last_change":"2024-03-06T07:20:00.822403+0000","last_active":"2024-03-06T07:20:07.241768+0000","last_peered":"2024-03-06T07:20:07.241768+0000","last_clean":"2024-03-06T07:20:07.241768+0000","last_became_active":"2024-03-06T07:20:00.822144+0000","last_became_peered":"2024-03-06T07:20:00.822144+0000","last_unstale":"2024-03-06T07:20:07.241768+0000","last_undegraded":"2024-03-06T07:20:07.241768+0000","last_fullsized":"2024-03-06T07:20:07.241768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241387+0000","last_change":"2024-03-06T07:20:05.864961+0000","last_active":"2024-03-06T07:20:07.241387+0000","last_peered":"2024-03-06T07:20:07.241387+0000","last_clean":"2024-03-06T07:20:07.241387+0000","last_became_active":"2024-03-06T07:20:05.864626+0000","last_became_peered":"2024-03-06T07:20:05.864626+0000","last_unstale":"2024-03-06T07:20:07.241387+0000","last_undegraded":"2024-03-06T07:20:07.241387+0000","last_fullsized":"2024-03-06T07:20:07.241387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:12.900271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243119+0000","last_change":"2024-03-06T07:20:01.830511+0000","last_active":"2024-03-06T07:20:07.243119+0000","last_peered":"2024-03-06T07:20:07.243119+0000","last_clean":"2024-03-06T07:20:07.243119+0000","last_became_active":"2024-03-06T07:20:01.830230+0000","last_became_peered":"2024-03-06T07:20:01.830230+0000","last_unstale":"2024-03-06T07:20:07.243119+0000","last_undegraded":"2024-03-06T07:20:07.243119+0000","last_fullsized":"2024-03-06T07:20:07.243119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242095+0000","last_change":"2024-03-06T07:20:00.816279+0000","last_active":"2024-03-06T07:20:07.242095+0000","last_peered":"2024-03-06T07:20:07.242095+0000","last_clean":"2024-03-06T07:20:07.242095+0000","last_became_active":"2024-03-06T07:20:00.815979+0000","last_became_peered":"2024-03-06T07:20:00.815979+0000","last_unstale":"2024-03-06T07:20:07.242095+0000","last_undegraded":"2024-03-06T07:20:07.242095+0000","last_fullsized":"2024-03-06T07:20:07.242095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240897+0000","last_change":"2024-03-06T07:20:05.864338+0000","last_active":"2024-03-06T07:20:07.240897+0000","last_peered":"2024-03-06T07:20:07.240897+0000","last_clean":"2024-03-06T07:20:07.240897+0000","last_became_active":"2024-03-06T07:20:05.863893+0000","last_became_peered":"2024-03-06T07:20:05.863893+0000","last_unstale":"2024-03-06T07:20:07.240897+0000","last_undegraded":"2024-03-06T07:20:07.240897+0000","last_fullsized":"2024-03-06T07:20:07.240897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:55.097092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242166+0000","last_change":"2024-03-06T07:20:01.813572+0000","last_active":"2024-03-06T07:20:07.242166+0000","last_peered":"2024-03-06T07:20:07.242166+0000","last_clean":"2024-03-06T07:20:07.242166+0000","last_became_active":"2024-03-06T07:20:01.813285+0000","last_became_peered":"2024-03-06T07:20:01.813285+0000","last_unstale":"2024-03-06T07:20:07.242166+0000","last_undegraded":"2024-03-06T07:20:07.242166+0000","last_fullsized":"2024-03-06T07:20:07.242166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:00.818384+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:00.818086+0000","last_became_peered":"2024-03-06T07:20:00.818086+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242538+0000","last_change":"2024-03-06T07:20:06.124240+0000","last_active":"2024-03-06T07:20:07.242538+0000","last_peered":"2024-03-06T07:20:07.242538+0000","last_clean":"2024-03-06T07:20:07.242538+0000","last_became_active":"2024-03-06T07:20:06.123859+0000","last_became_peered":"2024-03-06T07:20:06.123859+0000","last_unstale":"2024-03-06T07:20:07.242538+0000","last_undegraded":"2024-03-06T07:20:07.242538+0000","last_fullsized":"2024-03-06T07:20:07.242538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241784+0000","last_change":"2024-03-06T07:20:01.812138+0000","last_active":"2024-03-06T07:20:07.241784+0000","last_peered":"2024-03-06T07:20:07.241784+0000","last_clean":"2024-03-06T07:20:07.241784+0000","last_became_active":"2024-03-06T07:20:01.811851+0000","last_became_peered":"2024-03-06T07:20:01.811851+0000","last_unstale":"2024-03-06T07:20:07.241784+0000","last_undegraded":"2024-03-06T07:20:07.241784+0000","last_fullsized":"2024-03-06T07:20:07.241784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:22.590118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241483+0000","last_change":"2024-03-06T07:20:00.821941+0000","last_active":"2024-03-06T07:20:07.241483+0000","last_peered":"2024-03-06T07:20:07.241483+0000","last_clean":"2024-03-06T07:20:07.241483+0000","last_became_active":"2024-03-06T07:20:00.821533+0000","last_became_peered":"2024-03-06T07:20:00.821533+0000","last_unstale":"2024-03-06T07:20:07.241483+0000","last_undegraded":"2024-03-06T07:20:07.241483+0000","last_fullsized":"2024-03-06T07:20:07.241483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241052+0000","last_change":"2024-03-06T07:20:05.864042+0000","last_active":"2024-03-06T07:20:07.241052+0000","last_peered":"2024-03-06T07:20:07.241052+0000","last_clean":"2024-03-06T07:20:07.241052+0000","last_became_active":"2024-03-06T07:20:05.863616+0000","last_became_peered":"2024-03-06T07:20:05.863616+0000","last_unstale":"2024-03-06T07:20:07.241052+0000","last_undegraded":"2024-03-06T07:20:07.241052+0000","last_fullsized":"2024-03-06T07:20:07.241052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:12.894304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241818+0000","last_change":"2024-03-06T07:20:01.821307+0000","last_active":"2024-03-06T07:20:07.241818+0000","last_peered":"2024-03-06T07:20:07.241818+0000","last_clean":"2024-03-06T07:20:07.241818+0000","last_became_active":"2024-03-06T07:20:01.821010+0000","last_became_peered":"2024-03-06T07:20:01.821010+0000","last_unstale":"2024-03-06T07:20:07.241818+0000","last_undegraded":"2024-03-06T07:20:07.241818+0000","last_fullsized":"2024-03-06T07:20:07.241818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839526+0000","last_change":"2024-03-06T07:20:00.824564+0000","last_active":"2024-03-06T07:20:06.839526+0000","last_peered":"2024-03-06T07:20:06.839526+0000","last_clean":"2024-03-06T07:20:06.839526+0000","last_became_active":"2024-03-06T07:20:00.824292+0000","last_became_peered":"2024-03-06T07:20:00.824292+0000","last_unstale":"2024-03-06T07:20:06.839526+0000","last_undegraded":"2024-03-06T07:20:06.839526+0000","last_fullsized":"2024-03-06T07:20:06.839526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242082+0000","last_change":"2024-03-06T07:20:06.124710+0000","last_active":"2024-03-06T07:20:07.242082+0000","last_peered":"2024-03-06T07:20:07.242082+0000","last_clean":"2024-03-06T07:20:07.242082+0000","last_became_active":"2024-03-06T07:20:06.124119+0000","last_became_peered":"2024-03-06T07:20:06.124119+0000","last_unstale":"2024-03-06T07:20:07.242082+0000","last_undegraded":"2024-03-06T07:20:07.242082+0000","last_fullsized":"2024-03-06T07:20:07.242082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243393+0000","last_change":"2024-03-06T07:20:01.816965+0000","last_active":"2024-03-06T07:20:07.243393+0000","last_peered":"2024-03-06T07:20:07.243393+0000","last_clean":"2024-03-06T07:20:07.243393+0000","last_became_active":"2024-03-06T07:20:01.816482+0000","last_became_peered":"2024-03-06T07:20:01.816482+0000","last_unstale":"2024-03-06T07:20:07.243393+0000","last_undegraded":"2024-03-06T07:20:07.243393+0000","last_fullsized":"2024-03-06T07:20:07.243393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242030+0000","last_change":"2024-03-06T07:20:00.814404+0000","last_active":"2024-03-06T07:20:07.242030+0000","last_peered":"2024-03-06T07:20:07.242030+0000","last_clean":"2024-03-06T07:20:07.242030+0000","last_became_active":"2024-03-06T07:20:00.813542+0000","last_became_peered":"2024-03-06T07:20:00.813542+0000","last_unstale":"2024-03-06T07:20:07.242030+0000","last_undegraded":"2024-03-06T07:20:07.242030+0000","last_fullsized":"2024-03-06T07:20:07.242030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241898+0000","last_change":"2024-03-06T07:20:06.129464+0000","last_active":"2024-03-06T07:20:07.241898+0000","last_peered":"2024-03-06T07:20:07.241898+0000","last_clean":"2024-03-06T07:20:07.241898+0000","last_became_active":"2024-03-06T07:20:06.128688+0000","last_became_peered":"2024-03-06T07:20:06.128688+0000","last_unstale":"2024-03-06T07:20:07.241898+0000","last_undegraded":"2024-03-06T07:20:07.241898+0000","last_fullsized":"2024-03-06T07:20:07.241898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242308+0000","last_change":"2024-03-06T07:20:01.814773+0000","last_active":"2024-03-06T07:20:07.242308+0000","last_peered":"2024-03-06T07:20:07.242308+0000","last_clean":"2024-03-06T07:20:07.242308+0000","last_became_active":"2024-03-06T07:20:01.814497+0000","last_became_peered":"2024-03-06T07:20:01.814497+0000","last_unstale":"2024-03-06T07:20:07.242308+0000","last_undegraded":"2024-03-06T07:20:07.242308+0000","last_fullsized":"2024-03-06T07:20:07.242308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242368+0000","last_change":"2024-03-06T07:20:00.821645+0000","last_active":"2024-03-06T07:20:07.242368+0000","last_peered":"2024-03-06T07:20:07.242368+0000","last_clean":"2024-03-06T07:20:07.242368+0000","last_became_active":"2024-03-06T07:20:00.821353+0000","last_became_peered":"2024-03-06T07:20:00.821353+0000","last_unstale":"2024-03-06T07:20:07.242368+0000","last_undegraded":"2024-03-06T07:20:07.242368+0000","last_fullsized":"2024-03-06T07:20:07.242368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:02:35.568408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241505+0000","last_change":"2024-03-06T07:20:06.125251+0000","last_active":"2024-03-06T07:20:07.241505+0000","last_peered":"2024-03-06T07:20:07.241505+0000","last_clean":"2024-03-06T07:20:07.241505+0000","last_became_active":"2024-03-06T07:20:06.124788+0000","last_became_peered":"2024-03-06T07:20:06.124788+0000","last_unstale":"2024-03-06T07:20:07.241505+0000","last_undegraded":"2024-03-06T07:20:07.241505+0000","last_fullsized":"2024-03-06T07:20:07.241505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241587+0000","last_change":"2024-03-06T07:20:01.831321+0000","last_active":"2024-03-06T07:20:07.241587+0000","last_peered":"2024-03-06T07:20:07.241587+0000","last_clean":"2024-03-06T07:20:07.241587+0000","last_became_active":"2024-03-06T07:20:01.831041+0000","last_became_peered":"2024-03-06T07:20:01.831041+0000","last_unstale":"2024-03-06T07:20:07.241587+0000","last_undegraded":"2024-03-06T07:20:07.241587+0000","last_fullsized":"2024-03-06T07:20:07.241587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242408+0000","last_change":"2024-03-06T07:20:00.812059+0000","last_active":"2024-03-06T07:20:07.242408+0000","last_peered":"2024-03-06T07:20:07.242408+0000","last_clean":"2024-03-06T07:20:07.242408+0000","last_became_active":"2024-03-06T07:20:00.811207+0000","last_became_peered":"2024-03-06T07:20:00.811207+0000","last_unstale":"2024-03-06T07:20:07.242408+0000","last_undegraded":"2024-03-06T07:20:07.242408+0000","last_fullsized":"2024-03-06T07:20:07.242408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241904+0000","last_change":"2024-03-06T07:20:05.859872+0000","last_active":"2024-03-06T07:20:07.241904+0000","last_peered":"2024-03-06T07:20:07.241904+0000","last_clean":"2024-03-06T07:20:07.241904+0000","last_became_active":"2024-03-06T07:20:05.857727+0000","last_became_peered":"2024-03-06T07:20:05.857727+0000","last_unstale":"2024-03-06T07:20:07.241904+0000","last_undegraded":"2024-03-06T07:20:07.241904+0000","last_fullsized":"2024-03-06T07:20:07.241904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241527+0000","last_change":"2024-03-06T07:20:01.829871+0000","last_active":"2024-03-06T07:20:07.241527+0000","last_peered":"2024-03-06T07:20:07.241527+0000","last_clean":"2024-03-06T07:20:07.241527+0000","last_became_active":"2024-03-06T07:20:01.829526+0000","last_became_peered":"2024-03-06T07:20:01.829526+0000","last_unstale":"2024-03-06T07:20:07.241527+0000","last_undegraded":"2024-03-06T07:20:07.241527+0000","last_fullsized":"2024-03-06T07:20:07.241527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:23.704222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242548+0000","last_change":"2024-03-06T07:20:00.818847+0000","last_active":"2024-03-06T07:20:07.242548+0000","last_peered":"2024-03-06T07:20:07.242548+0000","last_clean":"2024-03-06T07:20:07.242548+0000","last_became_active":"2024-03-06T07:20:00.818547+0000","last_became_peered":"2024-03-06T07:20:00.818547+0000","last_unstale":"2024-03-06T07:20:07.242548+0000","last_undegraded":"2024-03-06T07:20:07.242548+0000","last_fullsized":"2024-03-06T07:20:07.242548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242474+0000","last_change":"2024-03-06T07:20:05.861172+0000","last_active":"2024-03-06T07:20:07.242474+0000","last_peered":"2024-03-06T07:20:07.242474+0000","last_clean":"2024-03-06T07:20:07.242474+0000","last_became_active":"2024-03-06T07:20:05.859989+0000","last_became_peered":"2024-03-06T07:20:05.859989+0000","last_unstale":"2024-03-06T07:20:07.242474+0000","last_undegraded":"2024-03-06T07:20:07.242474+0000","last_fullsized":"2024-03-06T07:20:07.242474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241692+0000","last_change":"2024-03-06T07:20:01.813503+0000","last_active":"2024-03-06T07:20:07.241692+0000","last_peered":"2024-03-06T07:20:07.241692+0000","last_clean":"2024-03-06T07:20:07.241692+0000","last_became_active":"2024-03-06T07:20:01.813227+0000","last_became_peered":"2024-03-06T07:20:01.813227+0000","last_unstale":"2024-03-06T07:20:07.241692+0000","last_undegraded":"2024-03-06T07:20:07.241692+0000","last_fullsized":"2024-03-06T07:20:07.241692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839223+0000","last_change":"2024-03-06T07:20:00.812787+0000","last_active":"2024-03-06T07:20:06.839223+0000","last_peered":"2024-03-06T07:20:06.839223+0000","last_clean":"2024-03-06T07:20:06.839223+0000","last_became_active":"2024-03-06T07:20:00.812505+0000","last_became_peered":"2024-03-06T07:20:00.812505+0000","last_unstale":"2024-03-06T07:20:06.839223+0000","last_undegraded":"2024-03-06T07:20:06.839223+0000","last_fullsized":"2024-03-06T07:20:06.839223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242555+0000","last_change":"2024-03-06T07:20:05.859379+0000","last_active":"2024-03-06T07:20:07.242555+0000","last_peered":"2024-03-06T07:20:07.242555+0000","last_clean":"2024-03-06T07:20:07.242555+0000","last_became_active":"2024-03-06T07:20:05.858723+0000","last_became_peered":"2024-03-06T07:20:05.858723+0000","last_unstale":"2024-03-06T07:20:07.242555+0000","last_undegraded":"2024-03-06T07:20:07.242555+0000","last_fullsized":"2024-03-06T07:20:07.242555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240883+0000","last_change":"2024-03-06T07:20:01.826415+0000","last_active":"2024-03-06T07:20:07.240883+0000","last_peered":"2024-03-06T07:20:07.240883+0000","last_clean":"2024-03-06T07:20:07.240883+0000","last_became_active":"2024-03-06T07:20:01.825703+0000","last_became_peered":"2024-03-06T07:20:01.825703+0000","last_unstale":"2024-03-06T07:20:07.240883+0000","last_undegraded":"2024-03-06T07:20:07.240883+0000","last_fullsized":"2024-03-06T07:20:07.240883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:47.116335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241563+0000","last_change":"2024-03-06T07:20:00.822080+0000","last_active":"2024-03-06T07:20:07.241563+0000","last_peered":"2024-03-06T07:20:07.241563+0000","last_clean":"2024-03-06T07:20:07.241563+0000","last_became_active":"2024-03-06T07:20:00.821813+0000","last_became_peered":"2024-03-06T07:20:00.821813+0000","last_unstale":"2024-03-06T07:20:07.241563+0000","last_undegraded":"2024-03-06T07:20:07.241563+0000","last_fullsized":"2024-03-06T07:20:07.241563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241612+0000","last_change":"2024-03-06T07:20:06.124233+0000","last_active":"2024-03-06T07:20:07.241612+0000","last_peered":"2024-03-06T07:20:07.241612+0000","last_clean":"2024-03-06T07:20:07.241612+0000","last_became_active":"2024-03-06T07:20:06.123834+0000","last_became_peered":"2024-03-06T07:20:06.123834+0000","last_unstale":"2024-03-06T07:20:07.241612+0000","last_undegraded":"2024-03-06T07:20:07.241612+0000","last_fullsized":"2024-03-06T07:20:07.241612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242846+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.242846+0000","last_peered":"2024-03-06T07:20:07.242846+0000","last_clean":"2024-03-06T07:20:07.242846+0000","last_became_active":"2024-03-06T07:20:01.830667+0000","last_became_peered":"2024-03-06T07:20:01.830667+0000","last_unstale":"2024-03-06T07:20:07.242846+0000","last_undegraded":"2024-03-06T07:20:07.242846+0000","last_fullsized":"2024-03-06T07:20:07.242846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241469+0000","last_change":"2024-03-06T07:20:00.814783+0000","last_active":"2024-03-06T07:20:07.241469+0000","last_peered":"2024-03-06T07:20:07.241469+0000","last_clean":"2024-03-06T07:20:07.241469+0000","last_became_active":"2024-03-06T07:20:00.814384+0000","last_became_peered":"2024-03-06T07:20:00.814384+0000","last_unstale":"2024-03-06T07:20:07.241469+0000","last_undegraded":"2024-03-06T07:20:07.241469+0000","last_fullsized":"2024-03-06T07:20:07.241469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241641+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:07.241641+0000","last_peered":"2024-03-06T07:20:07.241641+0000","last_clean":"2024-03-06T07:20:07.241641+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:07.241641+0000","last_undegraded":"2024-03-06T07:20:07.241641+0000","last_fullsized":"2024-03-06T07:20:07.241641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839262+0000","last_change":"2024-03-06T07:20:06.126879+0000","last_active":"2024-03-06T07:20:06.839262+0000","last_peered":"2024-03-06T07:20:06.839262+0000","last_clean":"2024-03-06T07:20:06.839262+0000","last_became_active":"2024-03-06T07:20:06.126475+0000","last_became_peered":"2024-03-06T07:20:06.126475+0000","last_unstale":"2024-03-06T07:20:06.839262+0000","last_undegraded":"2024-03-06T07:20:06.839262+0000","last_fullsized":"2024-03-06T07:20:06.839262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242057+0000","last_change":"2024-03-06T07:20:01.831806+0000","last_active":"2024-03-06T07:20:07.242057+0000","last_peered":"2024-03-06T07:20:07.242057+0000","last_clean":"2024-03-06T07:20:07.242057+0000","last_became_active":"2024-03-06T07:20:01.831545+0000","last_became_peered":"2024-03-06T07:20:01.831545+0000","last_unstale":"2024-03-06T07:20:07.242057+0000","last_undegraded":"2024-03-06T07:20:07.242057+0000","last_fullsized":"2024-03-06T07:20:07.242057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839559+0000","last_change":"2024-03-06T07:20:00.817587+0000","last_active":"2024-03-06T07:20:06.839559+0000","last_peered":"2024-03-06T07:20:06.839559+0000","last_clean":"2024-03-06T07:20:06.839559+0000","last_became_active":"2024-03-06T07:20:00.817212+0000","last_became_peered":"2024-03-06T07:20:00.817212+0000","last_unstale":"2024-03-06T07:20:06.839559+0000","last_undegraded":"2024-03-06T07:20:06.839559+0000","last_fullsized":"2024-03-06T07:20:06.839559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838986+0000","last_change":"2024-03-06T07:20:06.126921+0000","last_active":"2024-03-06T07:20:06.838986+0000","last_peered":"2024-03-06T07:20:06.838986+0000","last_clean":"2024-03-06T07:20:06.838986+0000","last_became_active":"2024-03-06T07:20:06.126385+0000","last_became_peered":"2024-03-06T07:20:06.126385+0000","last_unstale":"2024-03-06T07:20:06.838986+0000","last_undegraded":"2024-03-06T07:20:06.838986+0000","last_fullsized":"2024-03-06T07:20:06.838986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242044+0000","last_change":"2024-03-06T07:20:01.831450+0000","last_active":"2024-03-06T07:20:07.242044+0000","last_peered":"2024-03-06T07:20:07.242044+0000","last_clean":"2024-03-06T07:20:07.242044+0000","last_became_active":"2024-03-06T07:20:01.830525+0000","last_became_peered":"2024-03-06T07:20:01.830525+0000","last_unstale":"2024-03-06T07:20:07.242044+0000","last_undegraded":"2024-03-06T07:20:07.242044+0000","last_fullsized":"2024-03-06T07:20:07.242044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242394+0000","last_change":"2024-03-06T07:20:00.807621+0000","last_active":"2024-03-06T07:20:07.242394+0000","last_peered":"2024-03-06T07:20:07.242394+0000","last_clean":"2024-03-06T07:20:07.242394+0000","last_became_active":"2024-03-06T07:20:00.807340+0000","last_became_peered":"2024-03-06T07:20:00.807340+0000","last_unstale":"2024-03-06T07:20:07.242394+0000","last_undegraded":"2024-03-06T07:20:07.242394+0000","last_fullsized":"2024-03-06T07:20:07.242394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241535+0000","last_change":"2024-03-06T07:20:05.857778+0000","last_active":"2024-03-06T07:20:07.241535+0000","last_peered":"2024-03-06T07:20:07.241535+0000","last_clean":"2024-03-06T07:20:07.241535+0000","last_became_active":"2024-03-06T07:20:05.857279+0000","last_became_peered":"2024-03-06T07:20:05.857279+0000","last_unstale":"2024-03-06T07:20:07.241535+0000","last_undegraded":"2024-03-06T07:20:07.241535+0000","last_fullsized":"2024-03-06T07:20:07.241535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242638+0000","last_change":"2024-03-06T07:20:01.819993+0000","last_active":"2024-03-06T07:20:07.242638+0000","last_peered":"2024-03-06T07:20:07.242638+0000","last_clean":"2024-03-06T07:20:07.242638+0000","last_became_active":"2024-03-06T07:20:01.819691+0000","last_became_peered":"2024-03-06T07:20:01.819691+0000","last_unstale":"2024-03-06T07:20:07.242638+0000","last_undegraded":"2024-03-06T07:20:07.242638+0000","last_fullsized":"2024-03-06T07:20:07.242638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:19.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839555+0000","last_change":"2024-03-06T07:20:00.819667+0000","last_active":"2024-03-06T07:20:06.839555+0000","last_peered":"2024-03-06T07:20:06.839555+0000","last_clean":"2024-03-06T07:20:06.839555+0000","last_became_active":"2024-03-06T07:20:00.819371+0000","last_became_peered":"2024-03-06T07:20:00.819371+0000","last_unstale":"2024-03-06T07:20:06.839555+0000","last_undegraded":"2024-03-06T07:20:06.839555+0000","last_fullsized":"2024-03-06T07:20:06.839555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241355+0000","last_change":"2024-03-06T07:20:06.124150+0000","last_active":"2024-03-06T07:20:07.241355+0000","last_peered":"2024-03-06T07:20:07.241355+0000","last_clean":"2024-03-06T07:20:07.241355+0000","last_became_active":"2024-03-06T07:20:06.123794+0000","last_became_peered":"2024-03-06T07:20:06.123794+0000","last_unstale":"2024-03-06T07:20:07.241355+0000","last_undegraded":"2024-03-06T07:20:07.241355+0000","last_fullsized":"2024-03-06T07:20:07.241355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242273+0000","last_change":"2024-03-06T07:20:01.814512+0000","last_active":"2024-03-06T07:20:07.242273+0000","last_peered":"2024-03-06T07:20:07.242273+0000","last_clean":"2024-03-06T07:20:07.242273+0000","last_became_active":"2024-03-06T07:20:01.813177+0000","last_became_peered":"2024-03-06T07:20:01.813177+0000","last_unstale":"2024-03-06T07:20:07.242273+0000","last_undegraded":"2024-03-06T07:20:07.242273+0000","last_fullsized":"2024-03-06T07:20:07.242273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:05.673072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241319+0000","last_change":"2024-03-06T07:20:00.812789+0000","last_active":"2024-03-06T07:20:07.241319+0000","last_peered":"2024-03-06T07:20:07.241319+0000","last_clean":"2024-03-06T07:20:07.241319+0000","last_became_active":"2024-03-06T07:20:00.812391+0000","last_became_peered":"2024-03-06T07:20:00.812391+0000","last_unstale":"2024-03-06T07:20:07.241319+0000","last_undegraded":"2024-03-06T07:20:07.241319+0000","last_fullsized":"2024-03-06T07:20:07.241319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:54:26.011476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240875+0000","last_change":"2024-03-06T07:20:05.863928+0000","last_active":"2024-03-06T07:20:07.240875+0000","last_peered":"2024-03-06T07:20:07.240875+0000","last_clean":"2024-03-06T07:20:07.240875+0000","last_became_active":"2024-03-06T07:20:05.863555+0000","last_became_peered":"2024-03-06T07:20:05.863555+0000","last_unstale":"2024-03-06T07:20:07.240875+0000","last_undegraded":"2024-03-06T07:20:07.240875+0000","last_fullsized":"2024-03-06T07:20:07.240875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:53.757250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242121+0000","last_change":"2024-03-06T07:20:01.811116+0000","last_active":"2024-03-06T07:20:07.242121+0000","last_peered":"2024-03-06T07:20:07.242121+0000","last_clean":"2024-03-06T07:20:07.242121+0000","last_became_active":"2024-03-06T07:20:01.810828+0000","last_became_peered":"2024-03-06T07:20:01.810828+0000","last_unstale":"2024-03-06T07:20:07.242121+0000","last_undegraded":"2024-03-06T07:20:07.242121+0000","last_fullsized":"2024-03-06T07:20:07.242121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838933+0000","last_change":"2024-03-06T07:20:00.807531+0000","last_active":"2024-03-06T07:20:06.838933+0000","last_peered":"2024-03-06T07:20:06.838933+0000","last_clean":"2024-03-06T07:20:06.838933+0000","last_became_active":"2024-03-06T07:20:00.807208+0000","last_became_peered":"2024-03-06T07:20:00.807208+0000","last_unstale":"2024-03-06T07:20:06.838933+0000","last_undegraded":"2024-03-06T07:20:06.838933+0000","last_fullsized":"2024-03-06T07:20:06.838933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:14.638569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:05.853364+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:05.852877+0000","last_became_peered":"2024-03-06T07:20:05.852877+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838808+0000","last_change":"2024-03-06T07:20:01.824711+0000","last_active":"2024-03-06T07:20:06.838808+0000","last_peered":"2024-03-06T07:20:06.838808+0000","last_clean":"2024-03-06T07:20:06.838808+0000","last_became_active":"2024-03-06T07:20:01.824420+0000","last_became_peered":"2024-03-06T07:20:01.824420+0000","last_unstale":"2024-03-06T07:20:06.838808+0000","last_undegraded":"2024-03-06T07:20:06.838808+0000","last_fullsized":"2024-03-06T07:20:06.838808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838852+0000","last_change":"2024-03-06T07:20:00.816624+0000","last_active":"2024-03-06T07:20:06.838852+0000","last_peered":"2024-03-06T07:20:06.838852+0000","last_clean":"2024-03-06T07:20:06.838852+0000","last_became_active":"2024-03-06T07:20:00.816335+0000","last_became_peered":"2024-03-06T07:20:00.816335+0000","last_unstale":"2024-03-06T07:20:06.838852+0000","last_undegraded":"2024-03-06T07:20:06.838852+0000","last_fullsized":"2024-03-06T07:20:06.838852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243303+0000","last_change":"2024-03-06T07:20:06.124178+0000","last_active":"2024-03-06T07:20:07.243303+0000","last_peered":"2024-03-06T07:20:07.243303+0000","last_clean":"2024-03-06T07:20:07.243303+0000","last_became_active":"2024-03-06T07:20:06.123788+0000","last_became_peered":"2024-03-06T07:20:06.123788+0000","last_unstale":"2024-03-06T07:20:07.243303+0000","last_undegraded":"2024-03-06T07:20:07.243303+0000","last_fullsized":"2024-03-06T07:20:07.243303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:12.104491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241688+0000","last_change":"2024-03-06T07:20:01.817183+0000","last_active":"2024-03-06T07:20:07.241688+0000","last_peered":"2024-03-06T07:20:07.241688+0000","last_clean":"2024-03-06T07:20:07.241688+0000","last_became_active":"2024-03-06T07:20:01.816230+0000","last_became_peered":"2024-03-06T07:20:01.816230+0000","last_unstale":"2024-03-06T07:20:07.241688+0000","last_undegraded":"2024-03-06T07:20:07.241688+0000","last_fullsized":"2024-03-06T07:20:07.241688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242486+0000","last_change":"2024-03-06T07:20:00.817367+0000","last_active":"2024-03-06T07:20:07.242486+0000","last_peered":"2024-03-06T07:20:07.242486+0000","last_clean":"2024-03-06T07:20:07.242486+0000","last_became_active":"2024-03-06T07:20:00.816410+0000","last_became_peered":"2024-03-06T07:20:00.816410+0000","last_unstale":"2024-03-06T07:20:07.242486+0000","last_undegraded":"2024-03-06T07:20:07.242486+0000","last_fullsized":"2024-03-06T07:20:07.242486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243226+0000","last_change":"2024-03-06T07:20:06.126465+0000","last_active":"2024-03-06T07:20:07.243226+0000","last_peered":"2024-03-06T07:20:07.243226+0000","last_clean":"2024-03-06T07:20:07.243226+0000","last_became_active":"2024-03-06T07:20:06.126107+0000","last_became_peered":"2024-03-06T07:20:06.126107+0000","last_unstale":"2024-03-06T07:20:07.243226+0000","last_undegraded":"2024-03-06T07:20:07.243226+0000","last_fullsized":"2024-03-06T07:20:07.243226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:37.677442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241274+0000","last_change":"2024-03-06T07:20:01.822827+0000","last_active":"2024-03-06T07:20:07.241274+0000","last_peered":"2024-03-06T07:20:07.241274+0000","last_clean":"2024-03-06T07:20:07.241274+0000","last_became_active":"2024-03-06T07:20:01.822490+0000","last_became_peered":"2024-03-06T07:20:01.822490+0000","last_unstale":"2024-03-06T07:20:07.241274+0000","last_undegraded":"2024-03-06T07:20:07.241274+0000","last_fullsized":"2024-03-06T07:20:07.241274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241706+0000","last_change":"2024-03-06T07:20:00.811943+0000","last_active":"2024-03-06T07:20:07.241706+0000","last_peered":"2024-03-06T07:20:07.241706+0000","last_clean":"2024-03-06T07:20:07.241706+0000","last_became_active":"2024-03-06T07:20:00.811499+0000","last_became_peered":"2024-03-06T07:20:00.811499+0000","last_unstale":"2024-03-06T07:20:07.241706+0000","last_undegraded":"2024-03-06T07:20:07.241706+0000","last_fullsized":"2024-03-06T07:20:07.241706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241416+0000","last_change":"2024-03-06T07:20:06.127004+0000","last_active":"2024-03-06T07:20:07.241416+0000","last_peered":"2024-03-06T07:20:07.241416+0000","last_clean":"2024-03-06T07:20:07.241416+0000","last_became_active":"2024-03-06T07:20:06.126643+0000","last_became_peered":"2024-03-06T07:20:06.126643+0000","last_unstale":"2024-03-06T07:20:07.241416+0000","last_undegraded":"2024-03-06T07:20:07.241416+0000","last_fullsized":"2024-03-06T07:20:07.241416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.815676+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.815238+0000","last_became_peered":"2024-03-06T07:20:01.815238+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.138838+0000","last_change":"2024-03-06T07:20:00.818890+0000","last_active":"2024-03-06T07:20:11.138838+0000","last_peered":"2024-03-06T07:20:11.138838+0000","last_clean":"2024-03-06T07:20:11.138838+0000","last_became_active":"2024-03-06T07:20:00.818613+0000","last_became_peered":"2024-03-06T07:20:00.818613+0000","last_unstale":"2024-03-06T07:20:11.138838+0000","last_undegraded":"2024-03-06T07:20:11.138838+0000","last_fullsized":"2024-03-06T07:20:11.138838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242003+0000","last_change":"2024-03-06T07:20:05.862991+0000","last_active":"2024-03-06T07:20:07.242003+0000","last_peered":"2024-03-06T07:20:07.242003+0000","last_clean":"2024-03-06T07:20:07.242003+0000","last_became_active":"2024-03-06T07:20:05.862334+0000","last_became_peered":"2024-03-06T07:20:05.862334+0000","last_unstale":"2024-03-06T07:20:07.242003+0000","last_undegraded":"2024-03-06T07:20:07.242003+0000","last_fullsized":"2024-03-06T07:20:07.242003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241685+0000","last_change":"2024-03-06T07:20:01.815200+0000","last_active":"2024-03-06T07:20:07.241685+0000","last_peered":"2024-03-06T07:20:07.241685+0000","last_clean":"2024-03-06T07:20:07.241685+0000","last_became_active":"2024-03-06T07:20:01.814903+0000","last_became_peered":"2024-03-06T07:20:01.814903+0000","last_unstale":"2024-03-06T07:20:07.241685+0000","last_undegraded":"2024-03-06T07:20:07.241685+0000","last_fullsized":"2024-03-06T07:20:07.241685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242321+0000","last_change":"2024-03-06T07:20:00.822249+0000","last_active":"2024-03-06T07:20:07.242321+0000","last_peered":"2024-03-06T07:20:07.242321+0000","last_clean":"2024-03-06T07:20:07.242321+0000","last_became_active":"2024-03-06T07:20:00.821965+0000","last_became_peered":"2024-03-06T07:20:00.821965+0000","last_unstale":"2024-03-06T07:20:07.242321+0000","last_undegraded":"2024-03-06T07:20:07.242321+0000","last_fullsized":"2024-03-06T07:20:07.242321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241085+0000","last_change":"2024-03-06T07:20:05.862268+0000","last_active":"2024-03-06T07:20:07.241085+0000","last_peered":"2024-03-06T07:20:07.241085+0000","last_clean":"2024-03-06T07:20:07.241085+0000","last_became_active":"2024-03-06T07:20:05.861794+0000","last_became_peered":"2024-03-06T07:20:05.861794+0000","last_unstale":"2024-03-06T07:20:07.241085+0000","last_undegraded":"2024-03-06T07:20:07.241085+0000","last_fullsized":"2024-03-06T07:20:07.241085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:15.471029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241294+0000","last_change":"2024-03-06T07:20:01.835260+0000","last_active":"2024-03-06T07:20:07.241294+0000","last_peered":"2024-03-06T07:20:07.241294+0000","last_clean":"2024-03-06T07:20:07.241294+0000","last_became_active":"2024-03-06T07:20:01.835069+0000","last_became_peered":"2024-03-06T07:20:01.835069+0000","last_unstale":"2024-03-06T07:20:07.241294+0000","last_undegraded":"2024-03-06T07:20:07.241294+0000","last_fullsized":"2024-03-06T07:20:07.241294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241087+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241087+0000","last_peered":"2024-03-06T07:20:07.241087+0000","last_clean":"2024-03-06T07:20:07.241087+0000","last_became_active":"2024-03-06T07:20:00.822307+0000","last_became_peered":"2024-03-06T07:20:00.822307+0000","last_unstale":"2024-03-06T07:20:07.241087+0000","last_undegraded":"2024-03-06T07:20:07.241087+0000","last_fullsized":"2024-03-06T07:20:07.241087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242067+0000","last_change":"2024-03-06T07:20:05.857403+0000","last_active":"2024-03-06T07:20:07.242067+0000","last_peered":"2024-03-06T07:20:07.242067+0000","last_clean":"2024-03-06T07:20:07.242067+0000","last_became_active":"2024-03-06T07:20:05.857036+0000","last_became_peered":"2024-03-06T07:20:05.857036+0000","last_unstale":"2024-03-06T07:20:07.242067+0000","last_undegraded":"2024-03-06T07:20:07.242067+0000","last_fullsized":"2024-03-06T07:20:07.242067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:23.280198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241050+0000","last_change":"2024-03-06T07:20:01.833389+0000","last_active":"2024-03-06T07:20:07.241050+0000","last_peered":"2024-03-06T07:20:07.241050+0000","last_clean":"2024-03-06T07:20:07.241050+0000","last_became_active":"2024-03-06T07:20:01.833094+0000","last_became_peered":"2024-03-06T07:20:01.833094+0000","last_unstale":"2024-03-06T07:20:07.241050+0000","last_undegraded":"2024-03-06T07:20:07.241050+0000","last_fullsized":"2024-03-06T07:20:07.241050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839444+0000","last_change":"2024-03-06T07:20:00.817766+0000","last_active":"2024-03-06T07:20:06.839444+0000","last_peered":"2024-03-06T07:20:06.839444+0000","last_clean":"2024-03-06T07:20:06.839444+0000","last_became_active":"2024-03-06T07:20:00.817466+0000","last_became_peered":"2024-03-06T07:20:00.817466+0000","last_unstale":"2024-03-06T07:20:06.839444+0000","last_undegraded":"2024-03-06T07:20:06.839444+0000","last_fullsized":"2024-03-06T07:20:06.839444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242588+0000","last_change":"2024-03-06T07:20:06.127038+0000","last_active":"2024-03-06T07:20:07.242588+0000","last_peered":"2024-03-06T07:20:07.242588+0000","last_clean":"2024-03-06T07:20:07.242588+0000","last_became_active":"2024-03-06T07:20:06.126185+0000","last_became_peered":"2024-03-06T07:20:06.126185+0000","last_unstale":"2024-03-06T07:20:07.242588+0000","last_undegraded":"2024-03-06T07:20:07.242588+0000","last_fullsized":"2024-03-06T07:20:07.242588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241025+0000","last_change":"2024-03-06T07:20:06.126567+0000","last_active":"2024-03-06T07:20:07.241025+0000","last_peered":"2024-03-06T07:20:07.241025+0000","last_clean":"2024-03-06T07:20:07.241025+0000","last_became_active":"2024-03-06T07:20:06.126198+0000","last_became_peered":"2024-03-06T07:20:06.126198+0000","last_unstale":"2024-03-06T07:20:07.241025+0000","last_undegraded":"2024-03-06T07:20:07.241025+0000","last_fullsized":"2024-03-06T07:20:07.241025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240998+0000","last_change":"2024-03-06T07:20:01.817263+0000","last_active":"2024-03-06T07:20:07.240998+0000","last_peered":"2024-03-06T07:20:07.240998+0000","last_clean":"2024-03-06T07:20:07.240998+0000","last_became_active":"2024-03-06T07:20:01.816994+0000","last_became_peered":"2024-03-06T07:20:01.816994+0000","last_unstale":"2024-03-06T07:20:07.240998+0000","last_undegraded":"2024-03-06T07:20:07.240998+0000","last_fullsized":"2024-03-06T07:20:07.240998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241203+0000","last_change":"2024-03-06T07:20:00.814224+0000","last_active":"2024-03-06T07:20:07.241203+0000","last_peered":"2024-03-06T07:20:07.241203+0000","last_clean":"2024-03-06T07:20:07.241203+0000","last_became_active":"2024-03-06T07:20:00.813908+0000","last_became_peered":"2024-03-06T07:20:00.813908+0000","last_unstale":"2024-03-06T07:20:07.241203+0000","last_undegraded":"2024-03-06T07:20:07.241203+0000","last_fullsized":"2024-03-06T07:20:07.241203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839419+0000","last_change":"2024-03-06T07:20:05.863149+0000","last_active":"2024-03-06T07:20:06.839419+0000","last_peered":"2024-03-06T07:20:06.839419+0000","last_clean":"2024-03-06T07:20:06.839419+0000","last_became_active":"2024-03-06T07:20:05.862680+0000","last_became_peered":"2024-03-06T07:20:05.862680+0000","last_unstale":"2024-03-06T07:20:06.839419+0000","last_undegraded":"2024-03-06T07:20:06.839419+0000","last_fullsized":"2024-03-06T07:20:06.839419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242085+0000","last_change":"2024-03-06T07:20:01.822303+0000","last_active":"2024-03-06T07:20:07.242085+0000","last_peered":"2024-03-06T07:20:07.242085+0000","last_clean":"2024-03-06T07:20:07.242085+0000","last_became_active":"2024-03-06T07:20:01.822044+0000","last_became_peered":"2024-03-06T07:20:01.822044+0000","last_unstale":"2024-03-06T07:20:07.242085+0000","last_undegraded":"2024-03-06T07:20:07.242085+0000","last_fullsized":"2024-03-06T07:20:07.242085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242899+0000","last_change":"2024-03-06T07:20:00.819504+0000","last_active":"2024-03-06T07:20:07.242899+0000","last_peered":"2024-03-06T07:20:07.242899+0000","last_clean":"2024-03-06T07:20:07.242899+0000","last_became_active":"2024-03-06T07:20:00.819223+0000","last_became_peered":"2024-03-06T07:20:00.819223+0000","last_unstale":"2024-03-06T07:20:07.242899+0000","last_undegraded":"2024-03-06T07:20:07.242899+0000","last_fullsized":"2024-03-06T07:20:07.242899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839079+0000","last_change":"2024-03-06T07:20:00.816629+0000","last_active":"2024-03-06T07:20:06.839079+0000","last_peered":"2024-03-06T07:20:06.839079+0000","last_clean":"2024-03-06T07:20:06.839079+0000","last_became_active":"2024-03-06T07:20:00.816373+0000","last_became_peered":"2024-03-06T07:20:00.816373+0000","last_unstale":"2024-03-06T07:20:06.839079+0000","last_undegraded":"2024-03-06T07:20:06.839079+0000","last_fullsized":"2024-03-06T07:20:06.839079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241631+0000","last_change":"2024-03-06T07:20:01.829181+0000","last_active":"2024-03-06T07:20:07.241631+0000","last_peered":"2024-03-06T07:20:07.241631+0000","last_clean":"2024-03-06T07:20:07.241631+0000","last_became_active":"2024-03-06T07:20:01.827761+0000","last_became_peered":"2024-03-06T07:20:01.827761+0000","last_unstale":"2024-03-06T07:20:07.241631+0000","last_undegraded":"2024-03-06T07:20:07.241631+0000","last_fullsized":"2024-03-06T07:20:07.241631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839350+0000","last_change":"2024-03-06T07:20:05.857388+0000","last_active":"2024-03-06T07:20:06.839350+0000","last_peered":"2024-03-06T07:20:06.839350+0000","last_clean":"2024-03-06T07:20:06.839350+0000","last_became_active":"2024-03-06T07:20:05.856981+0000","last_became_peered":"2024-03-06T07:20:05.856981+0000","last_unstale":"2024-03-06T07:20:06.839350+0000","last_undegraded":"2024-03-06T07:20:06.839350+0000","last_fullsized":"2024-03-06T07:20:06.839350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:51.153823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243225+0000","last_change":"2024-03-06T07:20:00.822065+0000","last_active":"2024-03-06T07:20:07.243225+0000","last_peered":"2024-03-06T07:20:07.243225+0000","last_clean":"2024-03-06T07:20:07.243225+0000","last_became_active":"2024-03-06T07:20:00.821736+0000","last_became_peered":"2024-03-06T07:20:00.821736+0000","last_unstale":"2024-03-06T07:20:07.243225+0000","last_undegraded":"2024-03-06T07:20:07.243225+0000","last_fullsized":"2024-03-06T07:20:07.243225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241924+0000","last_change":"2024-03-06T07:20:01.819959+0000","last_active":"2024-03-06T07:20:07.241924+0000","last_peered":"2024-03-06T07:20:07.241924+0000","last_clean":"2024-03-06T07:20:07.241924+0000","last_became_active":"2024-03-06T07:20:01.819548+0000","last_became_peered":"2024-03-06T07:20:01.819548+0000","last_unstale":"2024-03-06T07:20:07.241924+0000","last_undegraded":"2024-03-06T07:20:07.241924+0000","last_fullsized":"2024-03-06T07:20:07.241924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:40.273910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243065+0000","last_change":"2024-03-06T07:20:05.864320+0000","last_active":"2024-03-06T07:20:07.243065+0000","last_peered":"2024-03-06T07:20:07.243065+0000","last_clean":"2024-03-06T07:20:07.243065+0000","last_became_active":"2024-03-06T07:20:05.863902+0000","last_became_peered":"2024-03-06T07:20:05.863902+0000","last_unstale":"2024-03-06T07:20:07.243065+0000","last_undegraded":"2024-03-06T07:20:07.243065+0000","last_fullsized":"2024-03-06T07:20:07.243065+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838709+0000","last_change":"2024-03-06T07:20:01.824719+0000","last_active":"2024-03-06T07:20:06.838709+0000","last_peered":"2024-03-06T07:20:06.838709+0000","last_clean":"2024-03-06T07:20:06.838709+0000","last_became_active":"2024-03-06T07:20:01.824424+0000","last_became_peered":"2024-03-06T07:20:01.824424+0000","last_unstale":"2024-03-06T07:20:06.838709+0000","last_undegraded":"2024-03-06T07:20:06.838709+0000","last_fullsized":"2024-03-06T07:20:06.838709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839111+0000","last_change":"2024-03-06T07:20:01.827824+0000","last_active":"2024-03-06T07:20:06.839111+0000","last_peered":"2024-03-06T07:20:06.839111+0000","last_clean":"2024-03-06T07:20:06.839111+0000","last_became_active":"2024-03-06T07:20:01.827552+0000","last_became_peered":"2024-03-06T07:20:01.827552+0000","last_unstale":"2024-03-06T07:20:06.839111+0000","last_undegraded":"2024-03-06T07:20:06.839111+0000","last_fullsized":"2024-03-06T07:20:06.839111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:24.188510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242476+0000","last_change":"2024-03-06T07:20:01.827949+0000","last_active":"2024-03-06T07:20:07.242476+0000","last_peered":"2024-03-06T07:20:07.242476+0000","last_clean":"2024-03-06T07:20:07.242476+0000","last_became_active":"2024-03-06T07:20:01.827535+0000","last_became_peered":"2024-03-06T07:20:01.827535+0000","last_unstale":"2024-03-06T07:20:07.242476+0000","last_undegraded":"2024-03-06T07:20:07.242476+0000","last_fullsized":"2024-03-06T07:20:07.242476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241603+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.241603+0000","last_peered":"2024-03-06T07:20:07.241603+0000","last_clean":"2024-03-06T07:20:07.241603+0000","last_became_active":"2024-03-06T07:20:01.828105+0000","last_became_peered":"2024-03-06T07:20:01.828105+0000","last_unstale":"2024-03-06T07:20:07.241603+0000","last_undegraded":"2024-03-06T07:20:07.241603+0000","last_fullsized":"2024-03-06T07:20:07.241603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243331+0000","last_change":"2024-03-06T07:20:01.810561+0000","last_active":"2024-03-06T07:20:07.243331+0000","last_peered":"2024-03-06T07:20:07.243331+0000","last_clean":"2024-03-06T07:20:07.243331+0000","last_became_active":"2024-03-06T07:20:01.810258+0000","last_became_peered":"2024-03-06T07:20:01.810258+0000","last_unstale":"2024-03-06T07:20:07.243331+0000","last_undegraded":"2024-03-06T07:20:07.243331+0000","last_fullsized":"2024-03-06T07:20:07.243331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242209+0000","last_change":"2024-03-06T07:20:01.812180+0000","last_active":"2024-03-06T07:20:07.242209+0000","last_peered":"2024-03-06T07:20:07.242209+0000","last_clean":"2024-03-06T07:20:07.242209+0000","last_became_active":"2024-03-06T07:20:01.811847+0000","last_became_peered":"2024-03-06T07:20:01.811847+0000","last_unstale":"2024-03-06T07:20:07.242209+0000","last_undegraded":"2024-03-06T07:20:07.242209+0000","last_fullsized":"2024-03-06T07:20:07.242209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839401+0000","last_change":"2024-03-06T07:20:01.821989+0000","last_active":"2024-03-06T07:20:06.839401+0000","last_peered":"2024-03-06T07:20:06.839401+0000","last_clean":"2024-03-06T07:20:06.839401+0000","last_became_active":"2024-03-06T07:20:01.821682+0000","last_became_peered":"2024-03-06T07:20:01.821682+0000","last_unstale":"2024-03-06T07:20:06.839401+0000","last_undegraded":"2024-03-06T07:20:06.839401+0000","last_fullsized":"2024-03-06T07:20:06.839401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243323+0000","last_change":"2024-03-06T07:20:01.815852+0000","last_active":"2024-03-06T07:20:07.243323+0000","last_peered":"2024-03-06T07:20:07.243323+0000","last_clean":"2024-03-06T07:20:07.243323+0000","last_became_active":"2024-03-06T07:20:01.815523+0000","last_became_peered":"2024-03-06T07:20:01.815523+0000","last_unstale":"2024-03-06T07:20:07.243323+0000","last_undegraded":"2024-03-06T07:20:07.243323+0000","last_fullsized":"2024-03-06T07:20:07.243323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241111+0000","last_change":"2024-03-06T07:20:01.828459+0000","last_active":"2024-03-06T07:20:07.241111+0000","last_peered":"2024-03-06T07:20:07.241111+0000","last_clean":"2024-03-06T07:20:07.241111+0000","last_became_active":"2024-03-06T07:20:01.828110+0000","last_became_peered":"2024-03-06T07:20:01.828110+0000","last_unstale":"2024-03-06T07:20:07.241111+0000","last_undegraded":"2024-03-06T07:20:07.241111+0000","last_fullsized":"2024-03-06T07:20:07.241111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242736+0000","last_change":"2024-03-06T07:20:01.810465+0000","last_active":"2024-03-06T07:20:07.242736+0000","last_peered":"2024-03-06T07:20:07.242736+0000","last_clean":"2024-03-06T07:20:07.242736+0000","last_became_active":"2024-03-06T07:20:01.810206+0000","last_became_peered":"2024-03-06T07:20:01.810206+0000","last_unstale":"2024-03-06T07:20:07.242736+0000","last_undegraded":"2024-03-06T07:20:07.242736+0000","last_fullsized":"2024-03-06T07:20:07.242736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242393+0000","last_change":"2024-03-06T07:20:01.834744+0000","last_active":"2024-03-06T07:20:07.242393+0000","last_peered":"2024-03-06T07:20:07.242393+0000","last_clean":"2024-03-06T07:20:07.242393+0000","last_became_active":"2024-03-06T07:20:01.834049+0000","last_became_peered":"2024-03-06T07:20:01.834049+0000","last_unstale":"2024-03-06T07:20:07.242393+0000","last_undegraded":"2024-03-06T07:20:07.242393+0000","last_fullsized":"2024-03-06T07:20:07.242393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241262+0000","last_change":"2024-03-06T07:20:01.830008+0000","last_active":"2024-03-06T07:20:07.241262+0000","last_peered":"2024-03-06T07:20:07.241262+0000","last_clean":"2024-03-06T07:20:07.241262+0000","last_became_active":"2024-03-06T07:20:01.828839+0000","last_became_peered":"2024-03-06T07:20:01.828839+0000","last_unstale":"2024-03-06T07:20:07.241262+0000","last_undegraded":"2024-03-06T07:20:07.241262+0000","last_fullsized":"2024-03-06T07:20:07.241262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839342+0000","last_change":"2024-03-06T07:20:01.815286+0000","last_active":"2024-03-06T07:20:06.839342+0000","last_peered":"2024-03-06T07:20:06.839342+0000","last_clean":"2024-03-06T07:20:06.839342+0000","last_became_active":"2024-03-06T07:20:01.814983+0000","last_became_peered":"2024-03-06T07:20:01.814983+0000","last_unstale":"2024-03-06T07:20:06.839342+0000","last_undegraded":"2024-03-06T07:20:06.839342+0000","last_fullsized":"2024-03-06T07:20:06.839342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242294+0000","last_change":"2024-03-06T07:20:01.816404+0000","last_active":"2024-03-06T07:20:07.242294+0000","last_peered":"2024-03-06T07:20:07.242294+0000","last_clean":"2024-03-06T07:20:07.242294+0000","last_became_active":"2024-03-06T07:20:01.815092+0000","last_became_peered":"2024-03-06T07:20:01.815092+0000","last_unstale":"2024-03-06T07:20:07.242294+0000","last_undegraded":"2024-03-06T07:20:07.242294+0000","last_fullsized":"2024-03-06T07:20:07.242294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839438+0000","last_change":"2024-03-06T07:20:01.819861+0000","last_active":"2024-03-06T07:20:06.839438+0000","last_peered":"2024-03-06T07:20:06.839438+0000","last_clean":"2024-03-06T07:20:06.839438+0000","last_became_active":"2024-03-06T07:20:01.819090+0000","last_became_peered":"2024-03-06T07:20:01.819090+0000","last_unstale":"2024-03-06T07:20:06.839438+0000","last_undegraded":"2024-03-06T07:20:06.839438+0000","last_fullsized":"2024-03-06T07:20:06.839438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.834134+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.833295+0000","last_became_peered":"2024-03-06T07:20:01.833295+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241004+0000","last_change":"2024-03-06T07:20:01.834489+0000","last_active":"2024-03-06T07:20:07.241004+0000","last_peered":"2024-03-06T07:20:07.241004+0000","last_clean":"2024-03-06T07:20:07.241004+0000","last_became_active":"2024-03-06T07:20:01.833824+0000","last_became_peered":"2024-03-06T07:20:01.833824+0000","last_unstale":"2024-03-06T07:20:07.241004+0000","last_undegraded":"2024-03-06T07:20:07.241004+0000","last_fullsized":"2024-03-06T07:20:07.241004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241109+0000","last_change":"2024-03-06T07:20:01.818055+0000","last_active":"2024-03-06T07:20:07.241109+0000","last_peered":"2024-03-06T07:20:07.241109+0000","last_clean":"2024-03-06T07:20:07.241109+0000","last_became_active":"2024-03-06T07:20:01.817719+0000","last_became_peered":"2024-03-06T07:20:01.817719+0000","last_unstale":"2024-03-06T07:20:07.241109+0000","last_undegraded":"2024-03-06T07:20:07.241109+0000","last_fullsized":"2024-03-06T07:20:07.241109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241162+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241162+0000","last_peered":"2024-03-06T07:20:07.241162+0000","last_clean":"2024-03-06T07:20:07.241162+0000","last_became_active":"2024-03-06T07:20:01.825795+0000","last_became_peered":"2024-03-06T07:20:01.825795+0000","last_unstale":"2024-03-06T07:20:07.241162+0000","last_undegraded":"2024-03-06T07:20:07.241162+0000","last_fullsized":"2024-03-06T07:20:07.241162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243013+0000","last_change":"2024-03-06T07:20:01.818654+0000","last_active":"2024-03-06T07:20:07.243013+0000","last_peered":"2024-03-06T07:20:07.243013+0000","last_clean":"2024-03-06T07:20:07.243013+0000","last_became_active":"2024-03-06T07:20:01.818368+0000","last_became_peered":"2024-03-06T07:20:01.818368+0000","last_unstale":"2024-03-06T07:20:07.243013+0000","last_undegraded":"2024-03-06T07:20:07.243013+0000","last_fullsized":"2024-03-06T07:20:07.243013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.817094+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.816841+0000","last_became_peered":"2024-03-06T07:20:01.816841+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:26.548988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839291+0000","last_change":"2024-03-06T07:20:01.831471+0000","last_active":"2024-03-06T07:20:06.839291+0000","last_peered":"2024-03-06T07:20:06.839291+0000","last_clean":"2024-03-06T07:20:06.839291+0000","last_became_active":"2024-03-06T07:20:01.831177+0000","last_became_peered":"2024-03-06T07:20:01.831177+0000","last_unstale":"2024-03-06T07:20:06.839291+0000","last_undegraded":"2024-03-06T07:20:06.839291+0000","last_fullsized":"2024-03-06T07:20:06.839291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839377+0000","last_change":"2024-03-06T07:20:01.831736+0000","last_active":"2024-03-06T07:20:06.839377+0000","last_peered":"2024-03-06T07:20:06.839377+0000","last_clean":"2024-03-06T07:20:06.839377+0000","last_became_active":"2024-03-06T07:20:01.831450+0000","last_became_peered":"2024-03-06T07:20:01.831450+0000","last_unstale":"2024-03-06T07:20:06.839377+0000","last_undegraded":"2024-03-06T07:20:06.839377+0000","last_fullsized":"2024-03-06T07:20:06.839377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242607+0000","last_change":"2024-03-06T07:20:01.832323+0000","last_active":"2024-03-06T07:20:07.242607+0000","last_peered":"2024-03-06T07:20:07.242607+0000","last_clean":"2024-03-06T07:20:07.242607+0000","last_became_active":"2024-03-06T07:20:01.832078+0000","last_became_peered":"2024-03-06T07:20:01.832078+0000","last_unstale":"2024-03-06T07:20:07.242607+0000","last_undegraded":"2024-03-06T07:20:07.242607+0000","last_fullsized":"2024-03-06T07:20:07.242607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241991+0000","last_change":"2024-03-06T07:20:01.831803+0000","last_active":"2024-03-06T07:20:07.241991+0000","last_peered":"2024-03-06T07:20:07.241991+0000","last_clean":"2024-03-06T07:20:07.241991+0000","last_became_active":"2024-03-06T07:20:01.831509+0000","last_became_peered":"2024-03-06T07:20:01.831509+0000","last_unstale":"2024-03-06T07:20:07.241991+0000","last_undegraded":"2024-03-06T07:20:07.241991+0000","last_fullsized":"2024-03-06T07:20:07.241991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242029+0000","last_change":"2024-03-06T07:20:01.828627+0000","last_active":"2024-03-06T07:20:07.242029+0000","last_peered":"2024-03-06T07:20:07.242029+0000","last_clean":"2024-03-06T07:20:07.242029+0000","last_became_active":"2024-03-06T07:20:01.828182+0000","last_became_peered":"2024-03-06T07:20:01.828182+0000","last_unstale":"2024-03-06T07:20:07.242029+0000","last_undegraded":"2024-03-06T07:20:07.242029+0000","last_fullsized":"2024-03-06T07:20:07.242029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242311+0000","last_change":"2024-03-06T07:20:01.834757+0000","last_active":"2024-03-06T07:20:07.242311+0000","last_peered":"2024-03-06T07:20:07.242311+0000","last_clean":"2024-03-06T07:20:07.242311+0000","last_became_active":"2024-03-06T07:20:01.833698+0000","last_became_peered":"2024-03-06T07:20:01.833698+0000","last_unstale":"2024-03-06T07:20:07.242311+0000","last_undegraded":"2024-03-06T07:20:07.242311+0000","last_fullsized":"2024-03-06T07:20:07.242311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241536+0000","last_change":"2024-03-06T07:20:01.816955+0000","last_active":"2024-03-06T07:20:07.241536+0000","last_peered":"2024-03-06T07:20:07.241536+0000","last_clean":"2024-03-06T07:20:07.241536+0000","last_became_active":"2024-03-06T07:20:01.816124+0000","last_became_peered":"2024-03-06T07:20:01.816124+0000","last_unstale":"2024-03-06T07:20:07.241536+0000","last_undegraded":"2024-03-06T07:20:07.241536+0000","last_fullsized":"2024-03-06T07:20:07.241536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241876+0000","last_change":"2024-03-06T07:20:01.831322+0000","last_active":"2024-03-06T07:20:07.241876+0000","last_peered":"2024-03-06T07:20:07.241876+0000","last_clean":"2024-03-06T07:20:07.241876+0000","last_became_active":"2024-03-06T07:20:01.830919+0000","last_became_peered":"2024-03-06T07:20:01.830919+0000","last_unstale":"2024-03-06T07:20:07.241876+0000","last_undegraded":"2024-03-06T07:20:07.241876+0000","last_fullsized":"2024-03-06T07:20:07.241876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241171+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241171+0000","last_peered":"2024-03-06T07:20:07.241171+0000","last_clean":"2024-03-06T07:20:07.241171+0000","last_became_active":"2024-03-06T07:20:01.826111+0000","last_became_peered":"2024-03-06T07:20:01.826111+0000","last_unstale":"2024-03-06T07:20:07.241171+0000","last_undegraded":"2024-03-06T07:20:07.241171+0000","last_fullsized":"2024-03-06T07:20:07.241171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241941+0000","last_change":"2024-03-06T07:20:01.822677+0000","last_active":"2024-03-06T07:20:07.241941+0000","last_peered":"2024-03-06T07:20:07.241941+0000","last_clean":"2024-03-06T07:20:07.241941+0000","last_became_active":"2024-03-06T07:20:01.822263+0000","last_became_peered":"2024-03-06T07:20:01.822263+0000","last_unstale":"2024-03-06T07:20:07.241941+0000","last_undegraded":"2024-03-06T07:20:07.241941+0000","last_fullsized":"2024-03-06T07:20:07.241941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242529+0000","last_change":"2024-03-06T07:20:01.818125+0000","last_active":"2024-03-06T07:20:07.242529+0000","last_peered":"2024-03-06T07:20:07.242529+0000","last_clean":"2024-03-06T07:20:07.242529+0000","last_became_active":"2024-03-06T07:20:01.817547+0000","last_became_peered":"2024-03-06T07:20:01.817547+0000","last_unstale":"2024-03-06T07:20:07.242529+0000","last_undegraded":"2024-03-06T07:20:07.242529+0000","last_fullsized":"2024-03-06T07:20:07.242529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":691206,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":691262,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:21.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:21.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.981+0000 7f9e5d82a640 1 Processor -- start 2024-03-06T07:20:21.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.981+0000 7f9e5d82a640 1 -- start start 2024-03-06T07:20:21.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e5d82a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e580c5330 0x7f9e580c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e5d82a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58158db0 0x7f9e5815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e5d82a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e58155110 con 0x7f9e580c88b0 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e5d82a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e581559e0 con 0x7f9e580c5330 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e5d82a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580c8f40 con 0x7f9e58158db0 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e56ffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e580c5330 0x7f9e580c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58158db0 0x7f9e5815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e567fc640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9e580c88b0 0x7f9e581586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35992/0 (socket says 172.21.15.26:35992) 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e56ffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e580c5330 0x7f9e580c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37218/0 (socket says 172.21.15.26:37218) 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e567fc640 1 -- 172.21.15.26:0/1973221458 learned_addr learned my addr 172.21.15.26:0/1973221458 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e56ffd640 1 -- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58158db0 msgr2=0x7f9e5815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e56ffd640 1 --2- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58158db0 0x7f9e5815fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e56ffd640 1 -- 172.21.15.26:0/1973221458 >> v1:172.21.15.117:6789/0 conn(0x7f9e580c88b0 legacy=0x7f9e581586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e56ffd640 1 -- 172.21.15.26:0/1973221458 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58155390 con 0x7f9e580c5330 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.982+0000 7f9e577fe640 1 --2- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58158db0 0x7f9e5815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.983+0000 7f9e56ffd640 1 --2- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c5330 0x7f9e580c5710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9e500097a0 tx=0x7f9e5002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9403802a93d2934 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.983+0000 7f9e55ffb640 1 -- 172.21.15.26:0/1973221458 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500315a0 con 0x7f9e580c5330 2024-03-06T07:20:21.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.983+0000 7f9e55ffb640 1 -- 172.21.15.26:0/1973221458 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50037070 con 0x7f9e580c5330 2024-03-06T07:20:21.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.983+0000 7f9e55ffb640 1 -- 172.21.15.26:0/1973221458 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50031960 con 0x7f9e580c5330 2024-03-06T07:20:21.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 -- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c5330 msgr2=0x7f9e580c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c5330 0x7f9e580c5710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9e500097a0 tx=0x7f9e5002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 -- 172.21.15.26:0/1973221458 shutdown_connections 2024-03-06T07:20:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58158db0 0x7f9e5815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/1973221458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c5330 0x7f9e580c5710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 -- 172.21.15.26:0/1973221458 >> 172.21.15.26:0/1973221458 conn(0x7f9e580ac560 msgr2=0x7f9e580ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 -- 172.21.15.26:0/1973221458 shutdown_connections 2024-03-06T07:20:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.984+0000 7f9e5d82a640 1 -- 172.21.15.26:0/1973221458 wait complete. 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.985+0000 7f9e5d82a640 1 Processor -- start 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.985+0000 7f9e5d82a640 1 -- start start 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e5d82a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e580c5330 0x7f9e58158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e5d82a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c88b0 0x7f9e581e0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e5d82a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e5814e160 0x7f9e5814b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e5d82a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e581529d0 con 0x7f9e580c5330 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e5d82a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5806d090 con 0x7f9e580c88b0 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e5d82a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58152b50 con 0x7f9e5814e160 2024-03-06T07:20:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e56ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e580c5330 0x7f9e58158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c88b0 0x7f9e581e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e5814e160 0x7f9e5814b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c88b0 0x7f9e581e0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37226/0 (socket says 172.21.15.26:37226) 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 -- 172.21.15.26:0/2673629975 learned_addr learned my addr 172.21.15.26:0/2673629975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e577fe640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e5814e160 0x7f9e5814b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47454/0 (socket says 172.21.15.26:47454) 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 -- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e5814e160 msgr2=0x7f9e5814b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e5814e160 0x7f9e5814b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 -- 172.21.15.26:0/2673629975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e580c5330 msgr2=0x7f9e58158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e580c5330 0x7f9e58158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 -- 172.21.15.26:0/2673629975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58158b30 con 0x7f9e580c88b0 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.986+0000 7f9e567fc640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c88b0 0x7f9e581e0230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c440 tx=0x7f9e4c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.987+0000 7f9e3ffff640 1 -- 172.21.15.26:0/2673629975 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c017070 con 0x7f9e580c88b0 2024-03-06T07:20:21.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.987+0000 7f9e3ffff640 1 -- 172.21.15.26:0/2673629975 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c005ce0 con 0x7f9e580c88b0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.987+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e581df0c0 con 0x7f9e580c88b0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.987+0000 7f9e3ffff640 1 -- 172.21.15.26:0/2673629975 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0072f0 con 0x7f9e580c88b0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.987+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5814ea70 con 0x7f9e580c88b0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.988+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5804f330 con 0x7f9e580c88b0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.988+0000 7f9e3ffff640 1 -- 172.21.15.26:0/2673629975 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e4c01a070 con 0x7f9e580c88b0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.988+0000 7f9e3ffff640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e4406b4d0 0x7f9e4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.989+0000 7f9e3ffff640 1 -- 172.21.15.26:0/2673629975 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9e4c0897d0 con 0x7f9e580c88b0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.989+0000 7f9e56ffd640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e4406b4d0 0x7f9e4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.989+0000 7f9e56ffd640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e4406b4d0 0x7f9e4406d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e50036000 tx=0x7f9e5003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:21.993+0000 7f9e3ffff640 1 -- 172.21.15.26:0/2673629975 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e58155110 con 0x7f9e580c88b0 2024-03-06T07:20:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.174+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e580be0c0 con 0x7f9e4406b4d0 2024-03-06T07:20:22.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.182+0000 7f9e3ffff640 1 -- 172.21.15.26:0/2673629975 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332611 (secure 0 0 0) 0x7f9e580be0c0 con 0x7f9e4406b4d0 2024-03-06T07:20:22.191 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:22.194 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e4406b4d0 msgr2=0x7f9e4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e4406b4d0 0x7f9e4406d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e50036000 tx=0x7f9e5003d040 comp rx=0 tx=0).stop 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c88b0 msgr2=0x7f9e581e0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c88b0 0x7f9e581e0230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c440 tx=0x7f9e4c001e70 comp rx=0 tx=0).stop 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 shutdown_connections 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e4406b4d0 0x7f9e4406d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e5814e160 0x7f9e5814b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580c88b0 0x7f9e581e0230 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.185+0000 7f9e5d82a640 1 --2- 172.21.15.26:0/2673629975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e580c5330 0x7f9e58158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.186+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 >> 172.21.15.26:0/2673629975 conn(0x7f9e580ac560 msgr2=0x7f9e58153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.186+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 shutdown_connections 2024-03-06T07:20:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.186+0000 7f9e5d82a640 1 -- 172.21.15.26:0/2673629975 wait complete. 2024-03-06T07:20:22.198 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-03-06T07:20:21.034266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230820,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743900,"statfs":{"total":773094113280,"available":772857753600,"internally_reserved":0,"allocated":2228224,"data_stored":1992064,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002612"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241454+0000","last_change":"2024-03-06T07:20:00.821957+0000","last_active":"2024-03-06T07:20:07.241454+0000","last_peered":"2024-03-06T07:20:07.241454+0000","last_clean":"2024-03-06T07:20:07.241454+0000","last_became_active":"2024-03-06T07:20:00.821672+0000","last_became_peered":"2024-03-06T07:20:00.821672+0000","last_unstale":"2024-03-06T07:20:07.241454+0000","last_undegraded":"2024-03-06T07:20:07.241454+0000","last_fullsized":"2024-03-06T07:20:07.241454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242911+0000","last_change":"2024-03-06T07:20:01.819400+0000","last_active":"2024-03-06T07:20:07.242911+0000","last_peered":"2024-03-06T07:20:07.242911+0000","last_clean":"2024-03-06T07:20:07.242911+0000","last_became_active":"2024-03-06T07:20:01.819073+0000","last_became_peered":"2024-03-06T07:20:01.819073+0000","last_unstale":"2024-03-06T07:20:07.242911+0000","last_undegraded":"2024-03-06T07:20:07.242911+0000","last_fullsized":"2024-03-06T07:20:07.242911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242373+0000","last_change":"2024-03-06T07:20:06.127375+0000","last_active":"2024-03-06T07:20:07.242373+0000","last_peered":"2024-03-06T07:20:07.242373+0000","last_clean":"2024-03-06T07:20:07.242373+0000","last_became_active":"2024-03-06T07:20:06.126687+0000","last_became_peered":"2024-03-06T07:20:06.126687+0000","last_unstale":"2024-03-06T07:20:07.242373+0000","last_undegraded":"2024-03-06T07:20:07.242373+0000","last_fullsized":"2024-03-06T07:20:07.242373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:11.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243026+0000","last_change":"2024-03-06T07:20:00.816186+0000","last_active":"2024-03-06T07:20:07.243026+0000","last_peered":"2024-03-06T07:20:07.243026+0000","last_clean":"2024-03-06T07:20:07.243026+0000","last_became_active":"2024-03-06T07:20:00.815110+0000","last_became_peered":"2024-03-06T07:20:00.815110+0000","last_unstale":"2024-03-06T07:20:07.243026+0000","last_undegraded":"2024-03-06T07:20:07.243026+0000","last_fullsized":"2024-03-06T07:20:07.243026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241124+0000","last_change":"2024-03-06T07:20:01.835360+0000","last_active":"2024-03-06T07:20:07.241124+0000","last_peered":"2024-03-06T07:20:07.241124+0000","last_clean":"2024-03-06T07:20:07.241124+0000","last_became_active":"2024-03-06T07:20:01.835066+0000","last_became_peered":"2024-03-06T07:20:01.835066+0000","last_unstale":"2024-03-06T07:20:07.241124+0000","last_undegraded":"2024-03-06T07:20:07.241124+0000","last_fullsized":"2024-03-06T07:20:07.241124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240859+0000","last_change":"2024-03-06T07:20:05.864904+0000","last_active":"2024-03-06T07:20:07.240859+0000","last_peered":"2024-03-06T07:20:07.240859+0000","last_clean":"2024-03-06T07:20:07.240859+0000","last_became_active":"2024-03-06T07:20:05.864535+0000","last_became_peered":"2024-03-06T07:20:05.864535+0000","last_unstale":"2024-03-06T07:20:07.240859+0000","last_undegraded":"2024-03-06T07:20:07.240859+0000","last_fullsized":"2024-03-06T07:20:07.240859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.016032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242211+0000","last_change":"2024-03-06T07:20:00.823477+0000","last_active":"2024-03-06T07:20:07.242211+0000","last_peered":"2024-03-06T07:20:07.242211+0000","last_clean":"2024-03-06T07:20:07.242211+0000","last_became_active":"2024-03-06T07:20:00.822898+0000","last_became_peered":"2024-03-06T07:20:00.822898+0000","last_unstale":"2024-03-06T07:20:07.242211+0000","last_undegraded":"2024-03-06T07:20:07.242211+0000","last_fullsized":"2024-03-06T07:20:07.242211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:23.183380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.832744+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.832237+0000","last_became_peered":"2024-03-06T07:20:01.832237+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241395+0000","last_change":"2024-03-06T07:20:05.862022+0000","last_active":"2024-03-06T07:20:07.241395+0000","last_peered":"2024-03-06T07:20:07.241395+0000","last_clean":"2024-03-06T07:20:07.241395+0000","last_became_active":"2024-03-06T07:20:05.861661+0000","last_became_peered":"2024-03-06T07:20:05.861661+0000","last_unstale":"2024-03-06T07:20:07.241395+0000","last_undegraded":"2024-03-06T07:20:07.241395+0000","last_fullsized":"2024-03-06T07:20:07.241395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:30.602011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241314+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241314+0000","last_peered":"2024-03-06T07:20:07.241314+0000","last_clean":"2024-03-06T07:20:07.241314+0000","last_became_active":"2024-03-06T07:20:00.822334+0000","last_became_peered":"2024-03-06T07:20:00.822334+0000","last_unstale":"2024-03-06T07:20:07.241314+0000","last_undegraded":"2024-03-06T07:20:07.241314+0000","last_fullsized":"2024-03-06T07:20:07.241314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241506+0000","last_change":"2024-03-06T07:20:01.832671+0000","last_active":"2024-03-06T07:20:07.241506+0000","last_peered":"2024-03-06T07:20:07.241506+0000","last_clean":"2024-03-06T07:20:07.241506+0000","last_became_active":"2024-03-06T07:20:01.832397+0000","last_became_peered":"2024-03-06T07:20:01.832397+0000","last_unstale":"2024-03-06T07:20:07.241506+0000","last_undegraded":"2024-03-06T07:20:07.241506+0000","last_fullsized":"2024-03-06T07:20:07.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241835+0000","last_change":"2024-03-06T07:20:05.853058+0000","last_active":"2024-03-06T07:20:07.241835+0000","last_peered":"2024-03-06T07:20:07.241835+0000","last_clean":"2024-03-06T07:20:07.241835+0000","last_became_active":"2024-03-06T07:20:05.852648+0000","last_became_peered":"2024-03-06T07:20:05.852648+0000","last_unstale":"2024-03-06T07:20:07.241835+0000","last_undegraded":"2024-03-06T07:20:07.241835+0000","last_fullsized":"2024-03-06T07:20:07.241835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:33.136033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838904+0000","last_change":"2024-03-06T07:20:01.817989+0000","last_active":"2024-03-06T07:20:06.838904+0000","last_peered":"2024-03-06T07:20:06.838904+0000","last_clean":"2024-03-06T07:20:06.838904+0000","last_became_active":"2024-03-06T07:20:01.817711+0000","last_became_peered":"2024-03-06T07:20:01.817711+0000","last_unstale":"2024-03-06T07:20:06.838904+0000","last_undegraded":"2024-03-06T07:20:06.838904+0000","last_fullsized":"2024-03-06T07:20:06.838904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241768+0000","last_change":"2024-03-06T07:20:00.822403+0000","last_active":"2024-03-06T07:20:07.241768+0000","last_peered":"2024-03-06T07:20:07.241768+0000","last_clean":"2024-03-06T07:20:07.241768+0000","last_became_active":"2024-03-06T07:20:00.822144+0000","last_became_peered":"2024-03-06T07:20:00.822144+0000","last_unstale":"2024-03-06T07:20:07.241768+0000","last_undegraded":"2024-03-06T07:20:07.241768+0000","last_fullsized":"2024-03-06T07:20:07.241768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241387+0000","last_change":"2024-03-06T07:20:05.864961+0000","last_active":"2024-03-06T07:20:07.241387+0000","last_peered":"2024-03-06T07:20:07.241387+0000","last_clean":"2024-03-06T07:20:07.241387+0000","last_became_active":"2024-03-06T07:20:05.864626+0000","last_became_peered":"2024-03-06T07:20:05.864626+0000","last_unstale":"2024-03-06T07:20:07.241387+0000","last_undegraded":"2024-03-06T07:20:07.241387+0000","last_fullsized":"2024-03-06T07:20:07.241387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:12.900271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243119+0000","last_change":"2024-03-06T07:20:01.830511+0000","last_active":"2024-03-06T07:20:07.243119+0000","last_peered":"2024-03-06T07:20:07.243119+0000","last_clean":"2024-03-06T07:20:07.243119+0000","last_became_active":"2024-03-06T07:20:01.830230+0000","last_became_peered":"2024-03-06T07:20:01.830230+0000","last_unstale":"2024-03-06T07:20:07.243119+0000","last_undegraded":"2024-03-06T07:20:07.243119+0000","last_fullsized":"2024-03-06T07:20:07.243119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242095+0000","last_change":"2024-03-06T07:20:00.816279+0000","last_active":"2024-03-06T07:20:07.242095+0000","last_peered":"2024-03-06T07:20:07.242095+0000","last_clean":"2024-03-06T07:20:07.242095+0000","last_became_active":"2024-03-06T07:20:00.815979+0000","last_became_peered":"2024-03-06T07:20:00.815979+0000","last_unstale":"2024-03-06T07:20:07.242095+0000","last_undegraded":"2024-03-06T07:20:07.242095+0000","last_fullsized":"2024-03-06T07:20:07.242095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240897+0000","last_change":"2024-03-06T07:20:05.864338+0000","last_active":"2024-03-06T07:20:07.240897+0000","last_peered":"2024-03-06T07:20:07.240897+0000","last_clean":"2024-03-06T07:20:07.240897+0000","last_became_active":"2024-03-06T07:20:05.863893+0000","last_became_peered":"2024-03-06T07:20:05.863893+0000","last_unstale":"2024-03-06T07:20:07.240897+0000","last_undegraded":"2024-03-06T07:20:07.240897+0000","last_fullsized":"2024-03-06T07:20:07.240897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:55.097092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242166+0000","last_change":"2024-03-06T07:20:01.813572+0000","last_active":"2024-03-06T07:20:07.242166+0000","last_peered":"2024-03-06T07:20:07.242166+0000","last_clean":"2024-03-06T07:20:07.242166+0000","last_became_active":"2024-03-06T07:20:01.813285+0000","last_became_peered":"2024-03-06T07:20:01.813285+0000","last_unstale":"2024-03-06T07:20:07.242166+0000","last_undegraded":"2024-03-06T07:20:07.242166+0000","last_fullsized":"2024-03-06T07:20:07.242166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:00.818384+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:00.818086+0000","last_became_peered":"2024-03-06T07:20:00.818086+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:56.374136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242538+0000","last_change":"2024-03-06T07:20:06.124240+0000","last_active":"2024-03-06T07:20:07.242538+0000","last_peered":"2024-03-06T07:20:07.242538+0000","last_clean":"2024-03-06T07:20:07.242538+0000","last_became_active":"2024-03-06T07:20:06.123859+0000","last_became_peered":"2024-03-06T07:20:06.123859+0000","last_unstale":"2024-03-06T07:20:07.242538+0000","last_undegraded":"2024-03-06T07:20:07.242538+0000","last_fullsized":"2024-03-06T07:20:07.242538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241784+0000","last_change":"2024-03-06T07:20:01.812138+0000","last_active":"2024-03-06T07:20:07.241784+0000","last_peered":"2024-03-06T07:20:07.241784+0000","last_clean":"2024-03-06T07:20:07.241784+0000","last_became_active":"2024-03-06T07:20:01.811851+0000","last_became_peered":"2024-03-06T07:20:01.811851+0000","last_unstale":"2024-03-06T07:20:07.241784+0000","last_undegraded":"2024-03-06T07:20:07.241784+0000","last_fullsized":"2024-03-06T07:20:07.241784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:22.590118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241483+0000","last_change":"2024-03-06T07:20:00.821941+0000","last_active":"2024-03-06T07:20:07.241483+0000","last_peered":"2024-03-06T07:20:07.241483+0000","last_clean":"2024-03-06T07:20:07.241483+0000","last_became_active":"2024-03-06T07:20:00.821533+0000","last_became_peered":"2024-03-06T07:20:00.821533+0000","last_unstale":"2024-03-06T07:20:07.241483+0000","last_undegraded":"2024-03-06T07:20:07.241483+0000","last_fullsized":"2024-03-06T07:20:07.241483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241052+0000","last_change":"2024-03-06T07:20:05.864042+0000","last_active":"2024-03-06T07:20:07.241052+0000","last_peered":"2024-03-06T07:20:07.241052+0000","last_clean":"2024-03-06T07:20:07.241052+0000","last_became_active":"2024-03-06T07:20:05.863616+0000","last_became_peered":"2024-03-06T07:20:05.863616+0000","last_unstale":"2024-03-06T07:20:07.241052+0000","last_undegraded":"2024-03-06T07:20:07.241052+0000","last_fullsized":"2024-03-06T07:20:07.241052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:12.894304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241818+0000","last_change":"2024-03-06T07:20:01.821307+0000","last_active":"2024-03-06T07:20:07.241818+0000","last_peered":"2024-03-06T07:20:07.241818+0000","last_clean":"2024-03-06T07:20:07.241818+0000","last_became_active":"2024-03-06T07:20:01.821010+0000","last_became_peered":"2024-03-06T07:20:01.821010+0000","last_unstale":"2024-03-06T07:20:07.241818+0000","last_undegraded":"2024-03-06T07:20:07.241818+0000","last_fullsized":"2024-03-06T07:20:07.241818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839526+0000","last_change":"2024-03-06T07:20:00.824564+0000","last_active":"2024-03-06T07:20:06.839526+0000","last_peered":"2024-03-06T07:20:06.839526+0000","last_clean":"2024-03-06T07:20:06.839526+0000","last_became_active":"2024-03-06T07:20:00.824292+0000","last_became_peered":"2024-03-06T07:20:00.824292+0000","last_unstale":"2024-03-06T07:20:06.839526+0000","last_undegraded":"2024-03-06T07:20:06.839526+0000","last_fullsized":"2024-03-06T07:20:06.839526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242082+0000","last_change":"2024-03-06T07:20:06.124710+0000","last_active":"2024-03-06T07:20:07.242082+0000","last_peered":"2024-03-06T07:20:07.242082+0000","last_clean":"2024-03-06T07:20:07.242082+0000","last_became_active":"2024-03-06T07:20:06.124119+0000","last_became_peered":"2024-03-06T07:20:06.124119+0000","last_unstale":"2024-03-06T07:20:07.242082+0000","last_undegraded":"2024-03-06T07:20:07.242082+0000","last_fullsized":"2024-03-06T07:20:07.242082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243393+0000","last_change":"2024-03-06T07:20:01.816965+0000","last_active":"2024-03-06T07:20:07.243393+0000","last_peered":"2024-03-06T07:20:07.243393+0000","last_clean":"2024-03-06T07:20:07.243393+0000","last_became_active":"2024-03-06T07:20:01.816482+0000","last_became_peered":"2024-03-06T07:20:01.816482+0000","last_unstale":"2024-03-06T07:20:07.243393+0000","last_undegraded":"2024-03-06T07:20:07.243393+0000","last_fullsized":"2024-03-06T07:20:07.243393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242030+0000","last_change":"2024-03-06T07:20:00.814404+0000","last_active":"2024-03-06T07:20:07.242030+0000","last_peered":"2024-03-06T07:20:07.242030+0000","last_clean":"2024-03-06T07:20:07.242030+0000","last_became_active":"2024-03-06T07:20:00.813542+0000","last_became_peered":"2024-03-06T07:20:00.813542+0000","last_unstale":"2024-03-06T07:20:07.242030+0000","last_undegraded":"2024-03-06T07:20:07.242030+0000","last_fullsized":"2024-03-06T07:20:07.242030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241898+0000","last_change":"2024-03-06T07:20:06.129464+0000","last_active":"2024-03-06T07:20:07.241898+0000","last_peered":"2024-03-06T07:20:07.241898+0000","last_clean":"2024-03-06T07:20:07.241898+0000","last_became_active":"2024-03-06T07:20:06.128688+0000","last_became_peered":"2024-03-06T07:20:06.128688+0000","last_unstale":"2024-03-06T07:20:07.241898+0000","last_undegraded":"2024-03-06T07:20:07.241898+0000","last_fullsized":"2024-03-06T07:20:07.241898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242308+0000","last_change":"2024-03-06T07:20:01.814773+0000","last_active":"2024-03-06T07:20:07.242308+0000","last_peered":"2024-03-06T07:20:07.242308+0000","last_clean":"2024-03-06T07:20:07.242308+0000","last_became_active":"2024-03-06T07:20:01.814497+0000","last_became_peered":"2024-03-06T07:20:01.814497+0000","last_unstale":"2024-03-06T07:20:07.242308+0000","last_undegraded":"2024-03-06T07:20:07.242308+0000","last_fullsized":"2024-03-06T07:20:07.242308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242368+0000","last_change":"2024-03-06T07:20:00.821645+0000","last_active":"2024-03-06T07:20:07.242368+0000","last_peered":"2024-03-06T07:20:07.242368+0000","last_clean":"2024-03-06T07:20:07.242368+0000","last_became_active":"2024-03-06T07:20:00.821353+0000","last_became_peered":"2024-03-06T07:20:00.821353+0000","last_unstale":"2024-03-06T07:20:07.242368+0000","last_undegraded":"2024-03-06T07:20:07.242368+0000","last_fullsized":"2024-03-06T07:20:07.242368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:02:35.568408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241505+0000","last_change":"2024-03-06T07:20:06.125251+0000","last_active":"2024-03-06T07:20:07.241505+0000","last_peered":"2024-03-06T07:20:07.241505+0000","last_clean":"2024-03-06T07:20:07.241505+0000","last_became_active":"2024-03-06T07:20:06.124788+0000","last_became_peered":"2024-03-06T07:20:06.124788+0000","last_unstale":"2024-03-06T07:20:07.241505+0000","last_undegraded":"2024-03-06T07:20:07.241505+0000","last_fullsized":"2024-03-06T07:20:07.241505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241587+0000","last_change":"2024-03-06T07:20:01.831321+0000","last_active":"2024-03-06T07:20:07.241587+0000","last_peered":"2024-03-06T07:20:07.241587+0000","last_clean":"2024-03-06T07:20:07.241587+0000","last_became_active":"2024-03-06T07:20:01.831041+0000","last_became_peered":"2024-03-06T07:20:01.831041+0000","last_unstale":"2024-03-06T07:20:07.241587+0000","last_undegraded":"2024-03-06T07:20:07.241587+0000","last_fullsized":"2024-03-06T07:20:07.241587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242408+0000","last_change":"2024-03-06T07:20:00.812059+0000","last_active":"2024-03-06T07:20:07.242408+0000","last_peered":"2024-03-06T07:20:07.242408+0000","last_clean":"2024-03-06T07:20:07.242408+0000","last_became_active":"2024-03-06T07:20:00.811207+0000","last_became_peered":"2024-03-06T07:20:00.811207+0000","last_unstale":"2024-03-06T07:20:07.242408+0000","last_undegraded":"2024-03-06T07:20:07.242408+0000","last_fullsized":"2024-03-06T07:20:07.242408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241904+0000","last_change":"2024-03-06T07:20:05.859872+0000","last_active":"2024-03-06T07:20:07.241904+0000","last_peered":"2024-03-06T07:20:07.241904+0000","last_clean":"2024-03-06T07:20:07.241904+0000","last_became_active":"2024-03-06T07:20:05.857727+0000","last_became_peered":"2024-03-06T07:20:05.857727+0000","last_unstale":"2024-03-06T07:20:07.241904+0000","last_undegraded":"2024-03-06T07:20:07.241904+0000","last_fullsized":"2024-03-06T07:20:07.241904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241527+0000","last_change":"2024-03-06T07:20:01.829871+0000","last_active":"2024-03-06T07:20:07.241527+0000","last_peered":"2024-03-06T07:20:07.241527+0000","last_clean":"2024-03-06T07:20:07.241527+0000","last_became_active":"2024-03-06T07:20:01.829526+0000","last_became_peered":"2024-03-06T07:20:01.829526+0000","last_unstale":"2024-03-06T07:20:07.241527+0000","last_undegraded":"2024-03-06T07:20:07.241527+0000","last_fullsized":"2024-03-06T07:20:07.241527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:23.704222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242548+0000","last_change":"2024-03-06T07:20:00.818847+0000","last_active":"2024-03-06T07:20:07.242548+0000","last_peered":"2024-03-06T07:20:07.242548+0000","last_clean":"2024-03-06T07:20:07.242548+0000","last_became_active":"2024-03-06T07:20:00.818547+0000","last_became_peered":"2024-03-06T07:20:00.818547+0000","last_unstale":"2024-03-06T07:20:07.242548+0000","last_undegraded":"2024-03-06T07:20:07.242548+0000","last_fullsized":"2024-03-06T07:20:07.242548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242474+0000","last_change":"2024-03-06T07:20:05.861172+0000","last_active":"2024-03-06T07:20:07.242474+0000","last_peered":"2024-03-06T07:20:07.242474+0000","last_clean":"2024-03-06T07:20:07.242474+0000","last_became_active":"2024-03-06T07:20:05.859989+0000","last_became_peered":"2024-03-06T07:20:05.859989+0000","last_unstale":"2024-03-06T07:20:07.242474+0000","last_undegraded":"2024-03-06T07:20:07.242474+0000","last_fullsized":"2024-03-06T07:20:07.242474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241692+0000","last_change":"2024-03-06T07:20:01.813503+0000","last_active":"2024-03-06T07:20:07.241692+0000","last_peered":"2024-03-06T07:20:07.241692+0000","last_clean":"2024-03-06T07:20:07.241692+0000","last_became_active":"2024-03-06T07:20:01.813227+0000","last_became_peered":"2024-03-06T07:20:01.813227+0000","last_unstale":"2024-03-06T07:20:07.241692+0000","last_undegraded":"2024-03-06T07:20:07.241692+0000","last_fullsized":"2024-03-06T07:20:07.241692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839223+0000","last_change":"2024-03-06T07:20:00.812787+0000","last_active":"2024-03-06T07:20:06.839223+0000","last_peered":"2024-03-06T07:20:06.839223+0000","last_clean":"2024-03-06T07:20:06.839223+0000","last_became_active":"2024-03-06T07:20:00.812505+0000","last_became_peered":"2024-03-06T07:20:00.812505+0000","last_unstale":"2024-03-06T07:20:06.839223+0000","last_undegraded":"2024-03-06T07:20:06.839223+0000","last_fullsized":"2024-03-06T07:20:06.839223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242555+0000","last_change":"2024-03-06T07:20:05.859379+0000","last_active":"2024-03-06T07:20:07.242555+0000","last_peered":"2024-03-06T07:20:07.242555+0000","last_clean":"2024-03-06T07:20:07.242555+0000","last_became_active":"2024-03-06T07:20:05.858723+0000","last_became_peered":"2024-03-06T07:20:05.858723+0000","last_unstale":"2024-03-06T07:20:07.242555+0000","last_undegraded":"2024-03-06T07:20:07.242555+0000","last_fullsized":"2024-03-06T07:20:07.242555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:09.710561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240883+0000","last_change":"2024-03-06T07:20:01.826415+0000","last_active":"2024-03-06T07:20:07.240883+0000","last_peered":"2024-03-06T07:20:07.240883+0000","last_clean":"2024-03-06T07:20:07.240883+0000","last_became_active":"2024-03-06T07:20:01.825703+0000","last_became_peered":"2024-03-06T07:20:01.825703+0000","last_unstale":"2024-03-06T07:20:07.240883+0000","last_undegraded":"2024-03-06T07:20:07.240883+0000","last_fullsized":"2024-03-06T07:20:07.240883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:47.116335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241563+0000","last_change":"2024-03-06T07:20:00.822080+0000","last_active":"2024-03-06T07:20:07.241563+0000","last_peered":"2024-03-06T07:20:07.241563+0000","last_clean":"2024-03-06T07:20:07.241563+0000","last_became_active":"2024-03-06T07:20:00.821813+0000","last_became_peered":"2024-03-06T07:20:00.821813+0000","last_unstale":"2024-03-06T07:20:07.241563+0000","last_undegraded":"2024-03-06T07:20:07.241563+0000","last_fullsized":"2024-03-06T07:20:07.241563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241612+0000","last_change":"2024-03-06T07:20:06.124233+0000","last_active":"2024-03-06T07:20:07.241612+0000","last_peered":"2024-03-06T07:20:07.241612+0000","last_clean":"2024-03-06T07:20:07.241612+0000","last_became_active":"2024-03-06T07:20:06.123834+0000","last_became_peered":"2024-03-06T07:20:06.123834+0000","last_unstale":"2024-03-06T07:20:07.241612+0000","last_undegraded":"2024-03-06T07:20:07.241612+0000","last_fullsized":"2024-03-06T07:20:07.241612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:32.827933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242846+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.242846+0000","last_peered":"2024-03-06T07:20:07.242846+0000","last_clean":"2024-03-06T07:20:07.242846+0000","last_became_active":"2024-03-06T07:20:01.830667+0000","last_became_peered":"2024-03-06T07:20:01.830667+0000","last_unstale":"2024-03-06T07:20:07.242846+0000","last_undegraded":"2024-03-06T07:20:07.242846+0000","last_fullsized":"2024-03-06T07:20:07.242846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241469+0000","last_change":"2024-03-06T07:20:00.814783+0000","last_active":"2024-03-06T07:20:07.241469+0000","last_peered":"2024-03-06T07:20:07.241469+0000","last_clean":"2024-03-06T07:20:07.241469+0000","last_became_active":"2024-03-06T07:20:00.814384+0000","last_became_peered":"2024-03-06T07:20:00.814384+0000","last_unstale":"2024-03-06T07:20:07.241469+0000","last_undegraded":"2024-03-06T07:20:07.241469+0000","last_fullsized":"2024-03-06T07:20:07.241469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241641+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:07.241641+0000","last_peered":"2024-03-06T07:20:07.241641+0000","last_clean":"2024-03-06T07:20:07.241641+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:07.241641+0000","last_undegraded":"2024-03-06T07:20:07.241641+0000","last_fullsized":"2024-03-06T07:20:07.241641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839262+0000","last_change":"2024-03-06T07:20:06.126879+0000","last_active":"2024-03-06T07:20:06.839262+0000","last_peered":"2024-03-06T07:20:06.839262+0000","last_clean":"2024-03-06T07:20:06.839262+0000","last_became_active":"2024-03-06T07:20:06.126475+0000","last_became_peered":"2024-03-06T07:20:06.126475+0000","last_unstale":"2024-03-06T07:20:06.839262+0000","last_undegraded":"2024-03-06T07:20:06.839262+0000","last_fullsized":"2024-03-06T07:20:06.839262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242057+0000","last_change":"2024-03-06T07:20:01.831806+0000","last_active":"2024-03-06T07:20:07.242057+0000","last_peered":"2024-03-06T07:20:07.242057+0000","last_clean":"2024-03-06T07:20:07.242057+0000","last_became_active":"2024-03-06T07:20:01.831545+0000","last_became_peered":"2024-03-06T07:20:01.831545+0000","last_unstale":"2024-03-06T07:20:07.242057+0000","last_undegraded":"2024-03-06T07:20:07.242057+0000","last_fullsized":"2024-03-06T07:20:07.242057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:29.098544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839559+0000","last_change":"2024-03-06T07:20:00.817587+0000","last_active":"2024-03-06T07:20:06.839559+0000","last_peered":"2024-03-06T07:20:06.839559+0000","last_clean":"2024-03-06T07:20:06.839559+0000","last_became_active":"2024-03-06T07:20:00.817212+0000","last_became_peered":"2024-03-06T07:20:00.817212+0000","last_unstale":"2024-03-06T07:20:06.839559+0000","last_undegraded":"2024-03-06T07:20:06.839559+0000","last_fullsized":"2024-03-06T07:20:06.839559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838986+0000","last_change":"2024-03-06T07:20:06.126921+0000","last_active":"2024-03-06T07:20:06.838986+0000","last_peered":"2024-03-06T07:20:06.838986+0000","last_clean":"2024-03-06T07:20:06.838986+0000","last_became_active":"2024-03-06T07:20:06.126385+0000","last_became_peered":"2024-03-06T07:20:06.126385+0000","last_unstale":"2024-03-06T07:20:06.838986+0000","last_undegraded":"2024-03-06T07:20:06.838986+0000","last_fullsized":"2024-03-06T07:20:06.838986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242044+0000","last_change":"2024-03-06T07:20:01.831450+0000","last_active":"2024-03-06T07:20:07.242044+0000","last_peered":"2024-03-06T07:20:07.242044+0000","last_clean":"2024-03-06T07:20:07.242044+0000","last_became_active":"2024-03-06T07:20:01.830525+0000","last_became_peered":"2024-03-06T07:20:01.830525+0000","last_unstale":"2024-03-06T07:20:07.242044+0000","last_undegraded":"2024-03-06T07:20:07.242044+0000","last_fullsized":"2024-03-06T07:20:07.242044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242394+0000","last_change":"2024-03-06T07:20:00.807621+0000","last_active":"2024-03-06T07:20:07.242394+0000","last_peered":"2024-03-06T07:20:07.242394+0000","last_clean":"2024-03-06T07:20:07.242394+0000","last_became_active":"2024-03-06T07:20:00.807340+0000","last_became_peered":"2024-03-06T07:20:00.807340+0000","last_unstale":"2024-03-06T07:20:07.242394+0000","last_undegraded":"2024-03-06T07:20:07.242394+0000","last_fullsized":"2024-03-06T07:20:07.242394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241535+0000","last_change":"2024-03-06T07:20:05.857778+0000","last_active":"2024-03-06T07:20:07.241535+0000","last_peered":"2024-03-06T07:20:07.241535+0000","last_clean":"2024-03-06T07:20:07.241535+0000","last_became_active":"2024-03-06T07:20:05.857279+0000","last_became_peered":"2024-03-06T07:20:05.857279+0000","last_unstale":"2024-03-06T07:20:07.241535+0000","last_undegraded":"2024-03-06T07:20:07.241535+0000","last_fullsized":"2024-03-06T07:20:07.241535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:08.428609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242638+0000","last_change":"2024-03-06T07:20:01.819993+0000","last_active":"2024-03-06T07:20:07.242638+0000","last_peered":"2024-03-06T07:20:07.242638+0000","last_clean":"2024-03-06T07:20:07.242638+0000","last_became_active":"2024-03-06T07:20:01.819691+0000","last_became_peered":"2024-03-06T07:20:01.819691+0000","last_unstale":"2024-03-06T07:20:07.242638+0000","last_undegraded":"2024-03-06T07:20:07.242638+0000","last_fullsized":"2024-03-06T07:20:07.242638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:19.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839555+0000","last_change":"2024-03-06T07:20:00.819667+0000","last_active":"2024-03-06T07:20:06.839555+0000","last_peered":"2024-03-06T07:20:06.839555+0000","last_clean":"2024-03-06T07:20:06.839555+0000","last_became_active":"2024-03-06T07:20:00.819371+0000","last_became_peered":"2024-03-06T07:20:00.819371+0000","last_unstale":"2024-03-06T07:20:06.839555+0000","last_undegraded":"2024-03-06T07:20:06.839555+0000","last_fullsized":"2024-03-06T07:20:06.839555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241355+0000","last_change":"2024-03-06T07:20:06.124150+0000","last_active":"2024-03-06T07:20:07.241355+0000","last_peered":"2024-03-06T07:20:07.241355+0000","last_clean":"2024-03-06T07:20:07.241355+0000","last_became_active":"2024-03-06T07:20:06.123794+0000","last_became_peered":"2024-03-06T07:20:06.123794+0000","last_unstale":"2024-03-06T07:20:07.241355+0000","last_undegraded":"2024-03-06T07:20:07.241355+0000","last_fullsized":"2024-03-06T07:20:07.241355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242273+0000","last_change":"2024-03-06T07:20:01.814512+0000","last_active":"2024-03-06T07:20:07.242273+0000","last_peered":"2024-03-06T07:20:07.242273+0000","last_clean":"2024-03-06T07:20:07.242273+0000","last_became_active":"2024-03-06T07:20:01.813177+0000","last_became_peered":"2024-03-06T07:20:01.813177+0000","last_unstale":"2024-03-06T07:20:07.242273+0000","last_undegraded":"2024-03-06T07:20:07.242273+0000","last_fullsized":"2024-03-06T07:20:07.242273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:05.673072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241319+0000","last_change":"2024-03-06T07:20:00.812789+0000","last_active":"2024-03-06T07:20:07.241319+0000","last_peered":"2024-03-06T07:20:07.241319+0000","last_clean":"2024-03-06T07:20:07.241319+0000","last_became_active":"2024-03-06T07:20:00.812391+0000","last_became_peered":"2024-03-06T07:20:00.812391+0000","last_unstale":"2024-03-06T07:20:07.241319+0000","last_undegraded":"2024-03-06T07:20:07.241319+0000","last_fullsized":"2024-03-06T07:20:07.241319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:54:26.011476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240875+0000","last_change":"2024-03-06T07:20:05.863928+0000","last_active":"2024-03-06T07:20:07.240875+0000","last_peered":"2024-03-06T07:20:07.240875+0000","last_clean":"2024-03-06T07:20:07.240875+0000","last_became_active":"2024-03-06T07:20:05.863555+0000","last_became_peered":"2024-03-06T07:20:05.863555+0000","last_unstale":"2024-03-06T07:20:07.240875+0000","last_undegraded":"2024-03-06T07:20:07.240875+0000","last_fullsized":"2024-03-06T07:20:07.240875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:53.757250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242121+0000","last_change":"2024-03-06T07:20:01.811116+0000","last_active":"2024-03-06T07:20:07.242121+0000","last_peered":"2024-03-06T07:20:07.242121+0000","last_clean":"2024-03-06T07:20:07.242121+0000","last_became_active":"2024-03-06T07:20:01.810828+0000","last_became_peered":"2024-03-06T07:20:01.810828+0000","last_unstale":"2024-03-06T07:20:07.242121+0000","last_undegraded":"2024-03-06T07:20:07.242121+0000","last_fullsized":"2024-03-06T07:20:07.242121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838933+0000","last_change":"2024-03-06T07:20:00.807531+0000","last_active":"2024-03-06T07:20:06.838933+0000","last_peered":"2024-03-06T07:20:06.838933+0000","last_clean":"2024-03-06T07:20:06.838933+0000","last_became_active":"2024-03-06T07:20:00.807208+0000","last_became_peered":"2024-03-06T07:20:00.807208+0000","last_unstale":"2024-03-06T07:20:06.838933+0000","last_undegraded":"2024-03-06T07:20:06.838933+0000","last_fullsized":"2024-03-06T07:20:06.838933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:14.638569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241197+0000","last_change":"2024-03-06T07:20:05.853364+0000","last_active":"2024-03-06T07:20:07.241197+0000","last_peered":"2024-03-06T07:20:07.241197+0000","last_clean":"2024-03-06T07:20:07.241197+0000","last_became_active":"2024-03-06T07:20:05.852877+0000","last_became_peered":"2024-03-06T07:20:05.852877+0000","last_unstale":"2024-03-06T07:20:07.241197+0000","last_undegraded":"2024-03-06T07:20:07.241197+0000","last_fullsized":"2024-03-06T07:20:07.241197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:44.293127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838808+0000","last_change":"2024-03-06T07:20:01.824711+0000","last_active":"2024-03-06T07:20:06.838808+0000","last_peered":"2024-03-06T07:20:06.838808+0000","last_clean":"2024-03-06T07:20:06.838808+0000","last_became_active":"2024-03-06T07:20:01.824420+0000","last_became_peered":"2024-03-06T07:20:01.824420+0000","last_unstale":"2024-03-06T07:20:06.838808+0000","last_undegraded":"2024-03-06T07:20:06.838808+0000","last_fullsized":"2024-03-06T07:20:06.838808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838852+0000","last_change":"2024-03-06T07:20:00.816624+0000","last_active":"2024-03-06T07:20:06.838852+0000","last_peered":"2024-03-06T07:20:06.838852+0000","last_clean":"2024-03-06T07:20:06.838852+0000","last_became_active":"2024-03-06T07:20:00.816335+0000","last_became_peered":"2024-03-06T07:20:00.816335+0000","last_unstale":"2024-03-06T07:20:06.838852+0000","last_undegraded":"2024-03-06T07:20:06.838852+0000","last_fullsized":"2024-03-06T07:20:06.838852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:38.879593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243303+0000","last_change":"2024-03-06T07:20:06.124178+0000","last_active":"2024-03-06T07:20:07.243303+0000","last_peered":"2024-03-06T07:20:07.243303+0000","last_clean":"2024-03-06T07:20:07.243303+0000","last_became_active":"2024-03-06T07:20:06.123788+0000","last_became_peered":"2024-03-06T07:20:06.123788+0000","last_unstale":"2024-03-06T07:20:07.243303+0000","last_undegraded":"2024-03-06T07:20:07.243303+0000","last_fullsized":"2024-03-06T07:20:07.243303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:12.104491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241688+0000","last_change":"2024-03-06T07:20:01.817183+0000","last_active":"2024-03-06T07:20:07.241688+0000","last_peered":"2024-03-06T07:20:07.241688+0000","last_clean":"2024-03-06T07:20:07.241688+0000","last_became_active":"2024-03-06T07:20:01.816230+0000","last_became_peered":"2024-03-06T07:20:01.816230+0000","last_unstale":"2024-03-06T07:20:07.241688+0000","last_undegraded":"2024-03-06T07:20:07.241688+0000","last_fullsized":"2024-03-06T07:20:07.241688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:40.255639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242486+0000","last_change":"2024-03-06T07:20:00.817367+0000","last_active":"2024-03-06T07:20:07.242486+0000","last_peered":"2024-03-06T07:20:07.242486+0000","last_clean":"2024-03-06T07:20:07.242486+0000","last_became_active":"2024-03-06T07:20:00.816410+0000","last_became_peered":"2024-03-06T07:20:00.816410+0000","last_unstale":"2024-03-06T07:20:07.242486+0000","last_undegraded":"2024-03-06T07:20:07.242486+0000","last_fullsized":"2024-03-06T07:20:07.242486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243226+0000","last_change":"2024-03-06T07:20:06.126465+0000","last_active":"2024-03-06T07:20:07.243226+0000","last_peered":"2024-03-06T07:20:07.243226+0000","last_clean":"2024-03-06T07:20:07.243226+0000","last_became_active":"2024-03-06T07:20:06.126107+0000","last_became_peered":"2024-03-06T07:20:06.126107+0000","last_unstale":"2024-03-06T07:20:07.243226+0000","last_undegraded":"2024-03-06T07:20:07.243226+0000","last_fullsized":"2024-03-06T07:20:07.243226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:37.677442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241274+0000","last_change":"2024-03-06T07:20:01.822827+0000","last_active":"2024-03-06T07:20:07.241274+0000","last_peered":"2024-03-06T07:20:07.241274+0000","last_clean":"2024-03-06T07:20:07.241274+0000","last_became_active":"2024-03-06T07:20:01.822490+0000","last_became_peered":"2024-03-06T07:20:01.822490+0000","last_unstale":"2024-03-06T07:20:07.241274+0000","last_undegraded":"2024-03-06T07:20:07.241274+0000","last_fullsized":"2024-03-06T07:20:07.241274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241706+0000","last_change":"2024-03-06T07:20:00.811943+0000","last_active":"2024-03-06T07:20:07.241706+0000","last_peered":"2024-03-06T07:20:07.241706+0000","last_clean":"2024-03-06T07:20:07.241706+0000","last_became_active":"2024-03-06T07:20:00.811499+0000","last_became_peered":"2024-03-06T07:20:00.811499+0000","last_unstale":"2024-03-06T07:20:07.241706+0000","last_undegraded":"2024-03-06T07:20:07.241706+0000","last_fullsized":"2024-03-06T07:20:07.241706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:20:48.321786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241416+0000","last_change":"2024-03-06T07:20:06.127004+0000","last_active":"2024-03-06T07:20:07.241416+0000","last_peered":"2024-03-06T07:20:07.241416+0000","last_clean":"2024-03-06T07:20:07.241416+0000","last_became_active":"2024-03-06T07:20:06.126643+0000","last_became_peered":"2024-03-06T07:20:06.126643+0000","last_unstale":"2024-03-06T07:20:07.241416+0000","last_undegraded":"2024-03-06T07:20:07.241416+0000","last_fullsized":"2024-03-06T07:20:07.241416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241955+0000","last_change":"2024-03-06T07:20:01.815676+0000","last_active":"2024-03-06T07:20:07.241955+0000","last_peered":"2024-03-06T07:20:07.241955+0000","last_clean":"2024-03-06T07:20:07.241955+0000","last_became_active":"2024-03-06T07:20:01.815238+0000","last_became_peered":"2024-03-06T07:20:01.815238+0000","last_unstale":"2024-03-06T07:20:07.241955+0000","last_undegraded":"2024-03-06T07:20:07.241955+0000","last_fullsized":"2024-03-06T07:20:07.241955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:08.862783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.138838+0000","last_change":"2024-03-06T07:20:00.818890+0000","last_active":"2024-03-06T07:20:11.138838+0000","last_peered":"2024-03-06T07:20:11.138838+0000","last_clean":"2024-03-06T07:20:11.138838+0000","last_became_active":"2024-03-06T07:20:00.818613+0000","last_became_peered":"2024-03-06T07:20:00.818613+0000","last_unstale":"2024-03-06T07:20:11.138838+0000","last_undegraded":"2024-03-06T07:20:11.138838+0000","last_fullsized":"2024-03-06T07:20:11.138838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:21.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242003+0000","last_change":"2024-03-06T07:20:05.862991+0000","last_active":"2024-03-06T07:20:07.242003+0000","last_peered":"2024-03-06T07:20:07.242003+0000","last_clean":"2024-03-06T07:20:07.242003+0000","last_became_active":"2024-03-06T07:20:05.862334+0000","last_became_peered":"2024-03-06T07:20:05.862334+0000","last_unstale":"2024-03-06T07:20:07.242003+0000","last_undegraded":"2024-03-06T07:20:07.242003+0000","last_fullsized":"2024-03-06T07:20:07.242003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241685+0000","last_change":"2024-03-06T07:20:01.815200+0000","last_active":"2024-03-06T07:20:07.241685+0000","last_peered":"2024-03-06T07:20:07.241685+0000","last_clean":"2024-03-06T07:20:07.241685+0000","last_became_active":"2024-03-06T07:20:01.814903+0000","last_became_peered":"2024-03-06T07:20:01.814903+0000","last_unstale":"2024-03-06T07:20:07.241685+0000","last_undegraded":"2024-03-06T07:20:07.241685+0000","last_fullsized":"2024-03-06T07:20:07.241685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242321+0000","last_change":"2024-03-06T07:20:00.822249+0000","last_active":"2024-03-06T07:20:07.242321+0000","last_peered":"2024-03-06T07:20:07.242321+0000","last_clean":"2024-03-06T07:20:07.242321+0000","last_became_active":"2024-03-06T07:20:00.821965+0000","last_became_peered":"2024-03-06T07:20:00.821965+0000","last_unstale":"2024-03-06T07:20:07.242321+0000","last_undegraded":"2024-03-06T07:20:07.242321+0000","last_fullsized":"2024-03-06T07:20:07.242321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241085+0000","last_change":"2024-03-06T07:20:05.862268+0000","last_active":"2024-03-06T07:20:07.241085+0000","last_peered":"2024-03-06T07:20:07.241085+0000","last_clean":"2024-03-06T07:20:07.241085+0000","last_became_active":"2024-03-06T07:20:05.861794+0000","last_became_peered":"2024-03-06T07:20:05.861794+0000","last_unstale":"2024-03-06T07:20:07.241085+0000","last_undegraded":"2024-03-06T07:20:07.241085+0000","last_fullsized":"2024-03-06T07:20:07.241085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:15.471029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241294+0000","last_change":"2024-03-06T07:20:01.835260+0000","last_active":"2024-03-06T07:20:07.241294+0000","last_peered":"2024-03-06T07:20:07.241294+0000","last_clean":"2024-03-06T07:20:07.241294+0000","last_became_active":"2024-03-06T07:20:01.835069+0000","last_became_peered":"2024-03-06T07:20:01.835069+0000","last_unstale":"2024-03-06T07:20:07.241294+0000","last_undegraded":"2024-03-06T07:20:07.241294+0000","last_fullsized":"2024-03-06T07:20:07.241294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241087+0000","last_change":"2024-03-06T07:20:00.822629+0000","last_active":"2024-03-06T07:20:07.241087+0000","last_peered":"2024-03-06T07:20:07.241087+0000","last_clean":"2024-03-06T07:20:07.241087+0000","last_became_active":"2024-03-06T07:20:00.822307+0000","last_became_peered":"2024-03-06T07:20:00.822307+0000","last_unstale":"2024-03-06T07:20:07.241087+0000","last_undegraded":"2024-03-06T07:20:07.241087+0000","last_fullsized":"2024-03-06T07:20:07.241087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:22.699091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242067+0000","last_change":"2024-03-06T07:20:05.857403+0000","last_active":"2024-03-06T07:20:07.242067+0000","last_peered":"2024-03-06T07:20:07.242067+0000","last_clean":"2024-03-06T07:20:07.242067+0000","last_became_active":"2024-03-06T07:20:05.857036+0000","last_became_peered":"2024-03-06T07:20:05.857036+0000","last_unstale":"2024-03-06T07:20:07.242067+0000","last_undegraded":"2024-03-06T07:20:07.242067+0000","last_fullsized":"2024-03-06T07:20:07.242067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:16:23.280198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241050+0000","last_change":"2024-03-06T07:20:01.833389+0000","last_active":"2024-03-06T07:20:07.241050+0000","last_peered":"2024-03-06T07:20:07.241050+0000","last_clean":"2024-03-06T07:20:07.241050+0000","last_became_active":"2024-03-06T07:20:01.833094+0000","last_became_peered":"2024-03-06T07:20:01.833094+0000","last_unstale":"2024-03-06T07:20:07.241050+0000","last_undegraded":"2024-03-06T07:20:07.241050+0000","last_fullsized":"2024-03-06T07:20:07.241050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839444+0000","last_change":"2024-03-06T07:20:00.817766+0000","last_active":"2024-03-06T07:20:06.839444+0000","last_peered":"2024-03-06T07:20:06.839444+0000","last_clean":"2024-03-06T07:20:06.839444+0000","last_became_active":"2024-03-06T07:20:00.817466+0000","last_became_peered":"2024-03-06T07:20:00.817466+0000","last_unstale":"2024-03-06T07:20:06.839444+0000","last_undegraded":"2024-03-06T07:20:06.839444+0000","last_fullsized":"2024-03-06T07:20:06.839444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:25.543858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242588+0000","last_change":"2024-03-06T07:20:06.127038+0000","last_active":"2024-03-06T07:20:07.242588+0000","last_peered":"2024-03-06T07:20:07.242588+0000","last_clean":"2024-03-06T07:20:07.242588+0000","last_became_active":"2024-03-06T07:20:06.126185+0000","last_became_peered":"2024-03-06T07:20:06.126185+0000","last_unstale":"2024-03-06T07:20:07.242588+0000","last_undegraded":"2024-03-06T07:20:07.242588+0000","last_fullsized":"2024-03-06T07:20:07.242588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:49.192380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241025+0000","last_change":"2024-03-06T07:20:06.126567+0000","last_active":"2024-03-06T07:20:07.241025+0000","last_peered":"2024-03-06T07:20:07.241025+0000","last_clean":"2024-03-06T07:20:07.241025+0000","last_became_active":"2024-03-06T07:20:06.126198+0000","last_became_peered":"2024-03-06T07:20:06.126198+0000","last_unstale":"2024-03-06T07:20:07.241025+0000","last_undegraded":"2024-03-06T07:20:07.241025+0000","last_fullsized":"2024-03-06T07:20:07.241025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:45.092467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.240998+0000","last_change":"2024-03-06T07:20:01.817263+0000","last_active":"2024-03-06T07:20:07.240998+0000","last_peered":"2024-03-06T07:20:07.240998+0000","last_clean":"2024-03-06T07:20:07.240998+0000","last_became_active":"2024-03-06T07:20:01.816994+0000","last_became_peered":"2024-03-06T07:20:01.816994+0000","last_unstale":"2024-03-06T07:20:07.240998+0000","last_undegraded":"2024-03-06T07:20:07.240998+0000","last_fullsized":"2024-03-06T07:20:07.240998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241203+0000","last_change":"2024-03-06T07:20:00.814224+0000","last_active":"2024-03-06T07:20:07.241203+0000","last_peered":"2024-03-06T07:20:07.241203+0000","last_clean":"2024-03-06T07:20:07.241203+0000","last_became_active":"2024-03-06T07:20:00.813908+0000","last_became_peered":"2024-03-06T07:20:00.813908+0000","last_unstale":"2024-03-06T07:20:07.241203+0000","last_undegraded":"2024-03-06T07:20:07.241203+0000","last_fullsized":"2024-03-06T07:20:07.241203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839419+0000","last_change":"2024-03-06T07:20:05.863149+0000","last_active":"2024-03-06T07:20:06.839419+0000","last_peered":"2024-03-06T07:20:06.839419+0000","last_clean":"2024-03-06T07:20:06.839419+0000","last_became_active":"2024-03-06T07:20:05.862680+0000","last_became_peered":"2024-03-06T07:20:05.862680+0000","last_unstale":"2024-03-06T07:20:06.839419+0000","last_undegraded":"2024-03-06T07:20:06.839419+0000","last_fullsized":"2024-03-06T07:20:06.839419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:59.360529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242085+0000","last_change":"2024-03-06T07:20:01.822303+0000","last_active":"2024-03-06T07:20:07.242085+0000","last_peered":"2024-03-06T07:20:07.242085+0000","last_clean":"2024-03-06T07:20:07.242085+0000","last_became_active":"2024-03-06T07:20:01.822044+0000","last_became_peered":"2024-03-06T07:20:01.822044+0000","last_unstale":"2024-03-06T07:20:07.242085+0000","last_undegraded":"2024-03-06T07:20:07.242085+0000","last_fullsized":"2024-03-06T07:20:07.242085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242899+0000","last_change":"2024-03-06T07:20:00.819504+0000","last_active":"2024-03-06T07:20:07.242899+0000","last_peered":"2024-03-06T07:20:07.242899+0000","last_clean":"2024-03-06T07:20:07.242899+0000","last_became_active":"2024-03-06T07:20:00.819223+0000","last_became_peered":"2024-03-06T07:20:00.819223+0000","last_unstale":"2024-03-06T07:20:07.242899+0000","last_undegraded":"2024-03-06T07:20:07.242899+0000","last_fullsized":"2024-03-06T07:20:07.242899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:23.123123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839079+0000","last_change":"2024-03-06T07:20:00.816629+0000","last_active":"2024-03-06T07:20:06.839079+0000","last_peered":"2024-03-06T07:20:06.839079+0000","last_clean":"2024-03-06T07:20:06.839079+0000","last_became_active":"2024-03-06T07:20:00.816373+0000","last_became_peered":"2024-03-06T07:20:00.816373+0000","last_unstale":"2024-03-06T07:20:06.839079+0000","last_undegraded":"2024-03-06T07:20:06.839079+0000","last_fullsized":"2024-03-06T07:20:06.839079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:39.268780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241631+0000","last_change":"2024-03-06T07:20:01.829181+0000","last_active":"2024-03-06T07:20:07.241631+0000","last_peered":"2024-03-06T07:20:07.241631+0000","last_clean":"2024-03-06T07:20:07.241631+0000","last_became_active":"2024-03-06T07:20:01.827761+0000","last_became_peered":"2024-03-06T07:20:01.827761+0000","last_unstale":"2024-03-06T07:20:07.241631+0000","last_undegraded":"2024-03-06T07:20:07.241631+0000","last_fullsized":"2024-03-06T07:20:07.241631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839350+0000","last_change":"2024-03-06T07:20:05.857388+0000","last_active":"2024-03-06T07:20:06.839350+0000","last_peered":"2024-03-06T07:20:06.839350+0000","last_clean":"2024-03-06T07:20:06.839350+0000","last_became_active":"2024-03-06T07:20:05.856981+0000","last_became_peered":"2024-03-06T07:20:05.856981+0000","last_unstale":"2024-03-06T07:20:06.839350+0000","last_undegraded":"2024-03-06T07:20:06.839350+0000","last_fullsized":"2024-03-06T07:20:06.839350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:51.153823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243225+0000","last_change":"2024-03-06T07:20:00.822065+0000","last_active":"2024-03-06T07:20:07.243225+0000","last_peered":"2024-03-06T07:20:07.243225+0000","last_clean":"2024-03-06T07:20:07.243225+0000","last_became_active":"2024-03-06T07:20:00.821736+0000","last_became_peered":"2024-03-06T07:20:00.821736+0000","last_unstale":"2024-03-06T07:20:07.243225+0000","last_undegraded":"2024-03-06T07:20:07.243225+0000","last_fullsized":"2024-03-06T07:20:07.243225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:59.786981+0000","last_clean_scrub_stamp":"2024-03-06T07:19:59.786981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:10.137033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241924+0000","last_change":"2024-03-06T07:20:01.819959+0000","last_active":"2024-03-06T07:20:07.241924+0000","last_peered":"2024-03-06T07:20:07.241924+0000","last_clean":"2024-03-06T07:20:07.241924+0000","last_became_active":"2024-03-06T07:20:01.819548+0000","last_became_peered":"2024-03-06T07:20:01.819548+0000","last_unstale":"2024-03-06T07:20:07.241924+0000","last_undegraded":"2024-03-06T07:20:07.241924+0000","last_fullsized":"2024-03-06T07:20:07.241924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:05:40.273910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243065+0000","last_change":"2024-03-06T07:20:05.864320+0000","last_active":"2024-03-06T07:20:07.243065+0000","last_peered":"2024-03-06T07:20:07.243065+0000","last_clean":"2024-03-06T07:20:07.243065+0000","last_became_active":"2024-03-06T07:20:05.863902+0000","last_became_peered":"2024-03-06T07:20:05.863902+0000","last_unstale":"2024-03-06T07:20:07.243065+0000","last_undegraded":"2024-03-06T07:20:07.243065+0000","last_fullsized":"2024-03-06T07:20:07.243065+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:04.829600+0000","last_clean_scrub_stamp":"2024-03-06T07:20:04.829600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:45.078225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.838709+0000","last_change":"2024-03-06T07:20:01.824719+0000","last_active":"2024-03-06T07:20:06.838709+0000","last_peered":"2024-03-06T07:20:06.838709+0000","last_clean":"2024-03-06T07:20:06.838709+0000","last_became_active":"2024-03-06T07:20:01.824424+0000","last_became_peered":"2024-03-06T07:20:01.824424+0000","last_unstale":"2024-03-06T07:20:06.838709+0000","last_undegraded":"2024-03-06T07:20:06.838709+0000","last_fullsized":"2024-03-06T07:20:06.838709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839111+0000","last_change":"2024-03-06T07:20:01.827824+0000","last_active":"2024-03-06T07:20:06.839111+0000","last_peered":"2024-03-06T07:20:06.839111+0000","last_clean":"2024-03-06T07:20:06.839111+0000","last_became_active":"2024-03-06T07:20:01.827552+0000","last_became_peered":"2024-03-06T07:20:01.827552+0000","last_unstale":"2024-03-06T07:20:06.839111+0000","last_undegraded":"2024-03-06T07:20:06.839111+0000","last_fullsized":"2024-03-06T07:20:06.839111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:40:24.188510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242476+0000","last_change":"2024-03-06T07:20:01.827949+0000","last_active":"2024-03-06T07:20:07.242476+0000","last_peered":"2024-03-06T07:20:07.242476+0000","last_clean":"2024-03-06T07:20:07.242476+0000","last_became_active":"2024-03-06T07:20:01.827535+0000","last_became_peered":"2024-03-06T07:20:01.827535+0000","last_unstale":"2024-03-06T07:20:07.242476+0000","last_undegraded":"2024-03-06T07:20:07.242476+0000","last_fullsized":"2024-03-06T07:20:07.242476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241603+0000","last_change":"2024-03-06T07:20:01.830976+0000","last_active":"2024-03-06T07:20:07.241603+0000","last_peered":"2024-03-06T07:20:07.241603+0000","last_clean":"2024-03-06T07:20:07.241603+0000","last_became_active":"2024-03-06T07:20:01.828105+0000","last_became_peered":"2024-03-06T07:20:01.828105+0000","last_unstale":"2024-03-06T07:20:07.241603+0000","last_undegraded":"2024-03-06T07:20:07.241603+0000","last_fullsized":"2024-03-06T07:20:07.241603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:46.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243331+0000","last_change":"2024-03-06T07:20:01.810561+0000","last_active":"2024-03-06T07:20:07.243331+0000","last_peered":"2024-03-06T07:20:07.243331+0000","last_clean":"2024-03-06T07:20:07.243331+0000","last_became_active":"2024-03-06T07:20:01.810258+0000","last_became_peered":"2024-03-06T07:20:01.810258+0000","last_unstale":"2024-03-06T07:20:07.243331+0000","last_undegraded":"2024-03-06T07:20:07.243331+0000","last_fullsized":"2024-03-06T07:20:07.243331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242209+0000","last_change":"2024-03-06T07:20:01.812180+0000","last_active":"2024-03-06T07:20:07.242209+0000","last_peered":"2024-03-06T07:20:07.242209+0000","last_clean":"2024-03-06T07:20:07.242209+0000","last_became_active":"2024-03-06T07:20:01.811847+0000","last_became_peered":"2024-03-06T07:20:01.811847+0000","last_unstale":"2024-03-06T07:20:07.242209+0000","last_undegraded":"2024-03-06T07:20:07.242209+0000","last_fullsized":"2024-03-06T07:20:07.242209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839401+0000","last_change":"2024-03-06T07:20:01.821989+0000","last_active":"2024-03-06T07:20:06.839401+0000","last_peered":"2024-03-06T07:20:06.839401+0000","last_clean":"2024-03-06T07:20:06.839401+0000","last_became_active":"2024-03-06T07:20:01.821682+0000","last_became_peered":"2024-03-06T07:20:01.821682+0000","last_unstale":"2024-03-06T07:20:06.839401+0000","last_undegraded":"2024-03-06T07:20:06.839401+0000","last_fullsized":"2024-03-06T07:20:06.839401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243323+0000","last_change":"2024-03-06T07:20:01.815852+0000","last_active":"2024-03-06T07:20:07.243323+0000","last_peered":"2024-03-06T07:20:07.243323+0000","last_clean":"2024-03-06T07:20:07.243323+0000","last_became_active":"2024-03-06T07:20:01.815523+0000","last_became_peered":"2024-03-06T07:20:01.815523+0000","last_unstale":"2024-03-06T07:20:07.243323+0000","last_undegraded":"2024-03-06T07:20:07.243323+0000","last_fullsized":"2024-03-06T07:20:07.243323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241111+0000","last_change":"2024-03-06T07:20:01.828459+0000","last_active":"2024-03-06T07:20:07.241111+0000","last_peered":"2024-03-06T07:20:07.241111+0000","last_clean":"2024-03-06T07:20:07.241111+0000","last_became_active":"2024-03-06T07:20:01.828110+0000","last_became_peered":"2024-03-06T07:20:01.828110+0000","last_unstale":"2024-03-06T07:20:07.241111+0000","last_undegraded":"2024-03-06T07:20:07.241111+0000","last_fullsized":"2024-03-06T07:20:07.241111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:11.433540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242736+0000","last_change":"2024-03-06T07:20:01.810465+0000","last_active":"2024-03-06T07:20:07.242736+0000","last_peered":"2024-03-06T07:20:07.242736+0000","last_clean":"2024-03-06T07:20:07.242736+0000","last_became_active":"2024-03-06T07:20:01.810206+0000","last_became_peered":"2024-03-06T07:20:01.810206+0000","last_unstale":"2024-03-06T07:20:07.242736+0000","last_undegraded":"2024-03-06T07:20:07.242736+0000","last_fullsized":"2024-03-06T07:20:07.242736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242393+0000","last_change":"2024-03-06T07:20:01.834744+0000","last_active":"2024-03-06T07:20:07.242393+0000","last_peered":"2024-03-06T07:20:07.242393+0000","last_clean":"2024-03-06T07:20:07.242393+0000","last_became_active":"2024-03-06T07:20:01.834049+0000","last_became_peered":"2024-03-06T07:20:01.834049+0000","last_unstale":"2024-03-06T07:20:07.242393+0000","last_undegraded":"2024-03-06T07:20:07.242393+0000","last_fullsized":"2024-03-06T07:20:07.242393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241262+0000","last_change":"2024-03-06T07:20:01.830008+0000","last_active":"2024-03-06T07:20:07.241262+0000","last_peered":"2024-03-06T07:20:07.241262+0000","last_clean":"2024-03-06T07:20:07.241262+0000","last_became_active":"2024-03-06T07:20:01.828839+0000","last_became_peered":"2024-03-06T07:20:01.828839+0000","last_unstale":"2024-03-06T07:20:07.241262+0000","last_undegraded":"2024-03-06T07:20:07.241262+0000","last_fullsized":"2024-03-06T07:20:07.241262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:23.597892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839342+0000","last_change":"2024-03-06T07:20:01.815286+0000","last_active":"2024-03-06T07:20:06.839342+0000","last_peered":"2024-03-06T07:20:06.839342+0000","last_clean":"2024-03-06T07:20:06.839342+0000","last_became_active":"2024-03-06T07:20:01.814983+0000","last_became_peered":"2024-03-06T07:20:01.814983+0000","last_unstale":"2024-03-06T07:20:06.839342+0000","last_undegraded":"2024-03-06T07:20:06.839342+0000","last_fullsized":"2024-03-06T07:20:06.839342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:18.978544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242294+0000","last_change":"2024-03-06T07:20:01.816404+0000","last_active":"2024-03-06T07:20:07.242294+0000","last_peered":"2024-03-06T07:20:07.242294+0000","last_clean":"2024-03-06T07:20:07.242294+0000","last_became_active":"2024-03-06T07:20:01.815092+0000","last_became_peered":"2024-03-06T07:20:01.815092+0000","last_unstale":"2024-03-06T07:20:07.242294+0000","last_undegraded":"2024-03-06T07:20:07.242294+0000","last_fullsized":"2024-03-06T07:20:07.242294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:55.323040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839438+0000","last_change":"2024-03-06T07:20:01.819861+0000","last_active":"2024-03-06T07:20:06.839438+0000","last_peered":"2024-03-06T07:20:06.839438+0000","last_clean":"2024-03-06T07:20:06.839438+0000","last_became_active":"2024-03-06T07:20:01.819090+0000","last_became_peered":"2024-03-06T07:20:01.819090+0000","last_unstale":"2024-03-06T07:20:06.839438+0000","last_undegraded":"2024-03-06T07:20:06.839438+0000","last_fullsized":"2024-03-06T07:20:06.839438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:57.379267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.834134+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.833295+0000","last_became_peered":"2024-03-06T07:20:01.833295+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241004+0000","last_change":"2024-03-06T07:20:01.834489+0000","last_active":"2024-03-06T07:20:07.241004+0000","last_peered":"2024-03-06T07:20:07.241004+0000","last_clean":"2024-03-06T07:20:07.241004+0000","last_became_active":"2024-03-06T07:20:01.833824+0000","last_became_peered":"2024-03-06T07:20:01.833824+0000","last_unstale":"2024-03-06T07:20:07.241004+0000","last_undegraded":"2024-03-06T07:20:07.241004+0000","last_fullsized":"2024-03-06T07:20:07.241004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241109+0000","last_change":"2024-03-06T07:20:01.818055+0000","last_active":"2024-03-06T07:20:07.241109+0000","last_peered":"2024-03-06T07:20:07.241109+0000","last_clean":"2024-03-06T07:20:07.241109+0000","last_became_active":"2024-03-06T07:20:01.817719+0000","last_became_peered":"2024-03-06T07:20:01.817719+0000","last_unstale":"2024-03-06T07:20:07.241109+0000","last_undegraded":"2024-03-06T07:20:07.241109+0000","last_fullsized":"2024-03-06T07:20:07.241109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:13.725634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241162+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241162+0000","last_peered":"2024-03-06T07:20:07.241162+0000","last_clean":"2024-03-06T07:20:07.241162+0000","last_became_active":"2024-03-06T07:20:01.825795+0000","last_became_peered":"2024-03-06T07:20:01.825795+0000","last_unstale":"2024-03-06T07:20:07.241162+0000","last_undegraded":"2024-03-06T07:20:07.241162+0000","last_fullsized":"2024-03-06T07:20:07.241162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.243013+0000","last_change":"2024-03-06T07:20:01.818654+0000","last_active":"2024-03-06T07:20:07.243013+0000","last_peered":"2024-03-06T07:20:07.243013+0000","last_clean":"2024-03-06T07:20:07.243013+0000","last_became_active":"2024-03-06T07:20:01.818368+0000","last_became_peered":"2024-03-06T07:20:01.818368+0000","last_unstale":"2024-03-06T07:20:07.243013+0000","last_undegraded":"2024-03-06T07:20:07.243013+0000","last_fullsized":"2024-03-06T07:20:07.243013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242008+0000","last_change":"2024-03-06T07:20:01.817094+0000","last_active":"2024-03-06T07:20:07.242008+0000","last_peered":"2024-03-06T07:20:07.242008+0000","last_clean":"2024-03-06T07:20:07.242008+0000","last_became_active":"2024-03-06T07:20:01.816841+0000","last_became_peered":"2024-03-06T07:20:01.816841+0000","last_unstale":"2024-03-06T07:20:07.242008+0000","last_undegraded":"2024-03-06T07:20:07.242008+0000","last_fullsized":"2024-03-06T07:20:07.242008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:26.548988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839291+0000","last_change":"2024-03-06T07:20:01.831471+0000","last_active":"2024-03-06T07:20:06.839291+0000","last_peered":"2024-03-06T07:20:06.839291+0000","last_clean":"2024-03-06T07:20:06.839291+0000","last_became_active":"2024-03-06T07:20:01.831177+0000","last_became_peered":"2024-03-06T07:20:01.831177+0000","last_unstale":"2024-03-06T07:20:06.839291+0000","last_undegraded":"2024-03-06T07:20:06.839291+0000","last_fullsized":"2024-03-06T07:20:06.839291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:06.839377+0000","last_change":"2024-03-06T07:20:01.831736+0000","last_active":"2024-03-06T07:20:06.839377+0000","last_peered":"2024-03-06T07:20:06.839377+0000","last_clean":"2024-03-06T07:20:06.839377+0000","last_became_active":"2024-03-06T07:20:01.831450+0000","last_became_peered":"2024-03-06T07:20:01.831450+0000","last_unstale":"2024-03-06T07:20:06.839377+0000","last_undegraded":"2024-03-06T07:20:06.839377+0000","last_fullsized":"2024-03-06T07:20:06.839377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242607+0000","last_change":"2024-03-06T07:20:01.832323+0000","last_active":"2024-03-06T07:20:07.242607+0000","last_peered":"2024-03-06T07:20:07.242607+0000","last_clean":"2024-03-06T07:20:07.242607+0000","last_became_active":"2024-03-06T07:20:01.832078+0000","last_became_peered":"2024-03-06T07:20:01.832078+0000","last_unstale":"2024-03-06T07:20:07.242607+0000","last_undegraded":"2024-03-06T07:20:07.242607+0000","last_fullsized":"2024-03-06T07:20:07.242607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:28.790444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241991+0000","last_change":"2024-03-06T07:20:01.831803+0000","last_active":"2024-03-06T07:20:07.241991+0000","last_peered":"2024-03-06T07:20:07.241991+0000","last_clean":"2024-03-06T07:20:07.241991+0000","last_became_active":"2024-03-06T07:20:01.831509+0000","last_became_peered":"2024-03-06T07:20:01.831509+0000","last_unstale":"2024-03-06T07:20:07.241991+0000","last_undegraded":"2024-03-06T07:20:07.241991+0000","last_fullsized":"2024-03-06T07:20:07.241991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242029+0000","last_change":"2024-03-06T07:20:01.828627+0000","last_active":"2024-03-06T07:20:07.242029+0000","last_peered":"2024-03-06T07:20:07.242029+0000","last_clean":"2024-03-06T07:20:07.242029+0000","last_became_active":"2024-03-06T07:20:01.828182+0000","last_became_peered":"2024-03-06T07:20:01.828182+0000","last_unstale":"2024-03-06T07:20:07.242029+0000","last_undegraded":"2024-03-06T07:20:07.242029+0000","last_fullsized":"2024-03-06T07:20:07.242029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:03:26.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242311+0000","last_change":"2024-03-06T07:20:01.834757+0000","last_active":"2024-03-06T07:20:07.242311+0000","last_peered":"2024-03-06T07:20:07.242311+0000","last_clean":"2024-03-06T07:20:07.242311+0000","last_became_active":"2024-03-06T07:20:01.833698+0000","last_became_peered":"2024-03-06T07:20:01.833698+0000","last_unstale":"2024-03-06T07:20:07.242311+0000","last_undegraded":"2024-03-06T07:20:07.242311+0000","last_fullsized":"2024-03-06T07:20:07.242311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:42:08.856815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241536+0000","last_change":"2024-03-06T07:20:01.816955+0000","last_active":"2024-03-06T07:20:07.241536+0000","last_peered":"2024-03-06T07:20:07.241536+0000","last_clean":"2024-03-06T07:20:07.241536+0000","last_became_active":"2024-03-06T07:20:01.816124+0000","last_became_peered":"2024-03-06T07:20:01.816124+0000","last_unstale":"2024-03-06T07:20:07.241536+0000","last_undegraded":"2024-03-06T07:20:07.241536+0000","last_fullsized":"2024-03-06T07:20:07.241536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:41.054978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241876+0000","last_change":"2024-03-06T07:20:01.831322+0000","last_active":"2024-03-06T07:20:07.241876+0000","last_peered":"2024-03-06T07:20:07.241876+0000","last_clean":"2024-03-06T07:20:07.241876+0000","last_became_active":"2024-03-06T07:20:01.830919+0000","last_became_peered":"2024-03-06T07:20:01.830919+0000","last_unstale":"2024-03-06T07:20:07.241876+0000","last_undegraded":"2024-03-06T07:20:07.241876+0000","last_fullsized":"2024-03-06T07:20:07.241876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:51.059603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241171+0000","last_change":"2024-03-06T07:20:01.827463+0000","last_active":"2024-03-06T07:20:07.241171+0000","last_peered":"2024-03-06T07:20:07.241171+0000","last_clean":"2024-03-06T07:20:07.241171+0000","last_became_active":"2024-03-06T07:20:01.826111+0000","last_became_peered":"2024-03-06T07:20:01.826111+0000","last_unstale":"2024-03-06T07:20:07.241171+0000","last_undegraded":"2024-03-06T07:20:07.241171+0000","last_fullsized":"2024-03-06T07:20:07.241171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.462080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.241941+0000","last_change":"2024-03-06T07:20:01.822677+0000","last_active":"2024-03-06T07:20:07.241941+0000","last_peered":"2024-03-06T07:20:07.241941+0000","last_clean":"2024-03-06T07:20:07.241941+0000","last_became_active":"2024-03-06T07:20:01.822263+0000","last_became_peered":"2024-03-06T07:20:01.822263+0000","last_unstale":"2024-03-06T07:20:07.241941+0000","last_undegraded":"2024-03-06T07:20:07.241941+0000","last_fullsized":"2024-03-06T07:20:07.241941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:49.719761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-06T07:20:07.242529+0000","last_change":"2024-03-06T07:20:01.818125+0000","last_active":"2024-03-06T07:20:07.242529+0000","last_peered":"2024-03-06T07:20:07.242529+0000","last_clean":"2024-03-06T07:20:07.242529+0000","last_became_active":"2024-03-06T07:20:01.817547+0000","last_became_peered":"2024-03-06T07:20:01.817547+0000","last_unstale":"2024-03-06T07:20:07.242529+0000","last_undegraded":"2024-03-06T07:20:07.242529+0000","last_fullsized":"2024-03-06T07:20:07.242529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:00.792111+0000","last_clean_scrub_stamp":"2024-03-06T07:20:00.792111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:04.391120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":691206,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":691262,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:22.203 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-06T07:20:22.203 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-06T07:20:22.203 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-06T07:20:22.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:20:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.337+0000 7f670941c640 1 Processor -- start 2024-03-06T07:20:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.337+0000 7f670941c640 1 -- start start 2024-03-06T07:20:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f670941c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f670406f120 0x7f67041588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f670941c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f670406ebd0 0x7f6704158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f670941c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67040c1910 con 0x7f6704152ab0 2024-03-06T07:20:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f670941c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67040c2cf0 con 0x7f670406ebd0 2024-03-06T07:20:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f670941c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67040c4050 con 0x7f670406f120 2024-03-06T07:20:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f670406ebd0 0x7f6704158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f670406ebd0 0x7f6704158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47092/0 (socket says 172.21.15.26:47092) 2024-03-06T07:20:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67037fe640 1 -- 172.21.15.26:0/2156097234 learned_addr learned my addr 172.21.15.26:0/2156097234 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67027fc640 1 --2- 172.21.15.26:0/2156097234 >> v2:172.21.15.117:3300/0 conn(0x7f670406f120 0x7f67041588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67037fe640 1 -- 172.21.15.26:0/2156097234 >> v2:172.21.15.117:3300/0 conn(0x7f670406f120 msgr2=0x7f67041588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67037fe640 1 --2- 172.21.15.26:0/2156097234 >> v2:172.21.15.117:3300/0 conn(0x7f670406f120 0x7f67041588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67037fe640 1 -- 172.21.15.26:0/2156097234 >> v1:172.21.15.117:6789/0 conn(0x7f6704152ab0 legacy=0x7f67040c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.338+0000 7f67037fe640 1 -- 172.21.15.26:0/2156097234 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040c1bf0 con 0x7f670406ebd0 2024-03-06T07:20:22.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.339+0000 7f67037fe640 1 --2- 172.21.15.26:0/2156097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406ebd0 0x7f6704158e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67040c53c0 tx=0x7f66f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606b2f4bb830f978 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.339+0000 7f6701ffb640 1 -- 172.21.15.26:0/2156097234 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f402ca80 con 0x7f670406ebd0 2024-03-06T07:20:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.339+0000 7f6701ffb640 1 -- 172.21.15.26:0/2156097234 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f402cc20 con 0x7f670406ebd0 2024-03-06T07:20:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.339+0000 7f6701ffb640 1 -- 172.21.15.26:0/2156097234 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40343f0 con 0x7f670406ebd0 2024-03-06T07:20:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 -- 172.21.15.26:0/2156097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406ebd0 msgr2=0x7f6704158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 --2- 172.21.15.26:0/2156097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406ebd0 0x7f6704158e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67040c53c0 tx=0x7f66f402bee0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 -- 172.21.15.26:0/2156097234 shutdown_connections 2024-03-06T07:20:22.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 --2- 172.21.15.26:0/2156097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406ebd0 0x7f6704158e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 --2- 172.21.15.26:0/2156097234 >> v2:172.21.15.117:3300/0 conn(0x7f670406f120 0x7f67041588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 -- 172.21.15.26:0/2156097234 >> 172.21.15.26:0/2156097234 conn(0x7f67040c7800 msgr2=0x7f67040c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 -- 172.21.15.26:0/2156097234 shutdown_connections 2024-03-06T07:20:22.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.340+0000 7f670941c640 1 -- 172.21.15.26:0/2156097234 wait complete. 2024-03-06T07:20:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.341+0000 7f670941c640 1 Processor -- start 2024-03-06T07:20:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.341+0000 7f670941c640 1 -- start start 2024-03-06T07:20:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f670941c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f670406ebd0 0x7f670414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f670941c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406f120 0x7f67040a7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f670941c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6704152ab0 0x7f67040ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f670941c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67040c1380 con 0x7f670406f120 2024-03-06T07:20:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f670941c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67040c6f60 con 0x7f670406ebd0 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f670941c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67040c1500 con 0x7f6704152ab0 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406f120 0x7f67040a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406f120 0x7f67040a7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47106/0 (socket says 172.21.15.26:47106) 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f6702ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f670406ebd0 0x7f670414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 -- 172.21.15.26:0/3202881789 learned_addr learned my addr 172.21.15.26:0/3202881789 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67037fe640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6704152ab0 0x7f67040ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 -- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6704152ab0 msgr2=0x7f67040ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6704152ab0 0x7f67040ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 -- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f670406ebd0 msgr2=0x7f670414fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f670406ebd0 0x7f670414fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 -- 172.21.15.26:0/3202881789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040a8e00 con 0x7f670406f120 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f6702ffd640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f670406ebd0 0x7f670414fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.342+0000 7f67027fc640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406f120 0x7f67040a7020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e9b0 tx=0x7f66ec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.343+0000 7f66fb7fe640 1 -- 172.21.15.26:0/3202881789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec017070 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.343+0000 7f66fb7fe640 1 -- 172.21.15.26:0/3202881789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec00c9a0 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.343+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67040a9590 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.343+0000 7f66fb7fe640 1 -- 172.21.15.26:0/3202881789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00cd90 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.343+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67040a9be0 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.344+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6704067ad0 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.344+0000 7f66fb7fe640 1 -- 172.21.15.26:0/3202881789 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66ec01a070 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.344+0000 7f66fb7fe640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66e406b4d0 0x7f66e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.344+0000 7f66fb7fe640 1 -- 172.21.15.26:0/3202881789 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f66ec08a490 con 0x7f670406f120 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.345+0000 7f6702ffd640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66e406b4d0 0x7f66e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.345+0000 7f6702ffd640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66e406b4d0 0x7f66e406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f66fc001ad0 tx=0x7f66fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.349+0000 7f66fb7fe640 1 -- 172.21.15.26:0/3202881789 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6704067ad0 con 0x7f670406f120 2024-03-06T07:20:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.566+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f670404f330 con 0x7f670406f120 2024-03-06T07:20:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.566+0000 7f66fb7fe640 1 -- 172.21.15.26:0/3202881789 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f66ec058a10 con 0x7f670406f120 2024-03-06T07:20:22.569 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:20:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66e406b4d0 msgr2=0x7f66e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66e406b4d0 0x7f66e406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f66fc001ad0 tx=0x7f66fc009040 comp rx=0 tx=0).stop 2024-03-06T07:20:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406f120 msgr2=0x7f67040a7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406f120 0x7f67040a7020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e9b0 tx=0x7f66ec00ee70 comp rx=0 tx=0).stop 2024-03-06T07:20:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 shutdown_connections 2024-03-06T07:20:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66e406b4d0 0x7f66e406d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6704152ab0 0x7f67040ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670406f120 0x7f67040a7020 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 --2- 172.21.15.26:0/3202881789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f670406ebd0 0x7f670414fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 >> 172.21.15.26:0/3202881789 conn(0x7f67040c7800 msgr2=0x7f6704071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 shutdown_connections 2024-03-06T07:20:22.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.568+0000 7f670941c640 1 -- 172.21.15.26:0/3202881789 wait complete. 2024-03-06T07:20:22.581 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-06T07:20:22.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:22.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.681+0000 7f30491d7640 1 Processor -- start 2024-03-06T07:20:22.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.681+0000 7f30491d7640 1 -- start start 2024-03-06T07:20:22.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f30491d7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3044158db0 0x7f30440c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f30491d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3044155110 con 0x7f30440c5330 2024-03-06T07:20:22.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f30491d7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30440c8f40 con 0x7f30440c88b0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f30491d7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30441559e0 con 0x7f3044158db0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f3042d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f30440c5330 0x7f30441586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44236/0 (socket says 172.21.15.26:44236) 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f3042d9d640 1 -- 172.21.15.26:0/3522575691 learned_addr learned my addr 172.21.15.26:0/3522575691 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f304259c640 1 --2- 172.21.15.26:0/3522575691 >> v2:172.21.15.117:3300/0 conn(0x7f3044158db0 0x7f30440c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f3041d9b640 1 -- 172.21.15.26:0/3522575691 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309153989 0 0) 0x7f303c001110 con 0x7f30440c5330 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f3041d9b640 1 -- 172.21.15.26:0/3522575691 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3030003520 con 0x7f30440c5330 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f304259c640 1 -- 172.21.15.26:0/3522575691 >> v1:172.21.15.117:6789/0 conn(0x7f30440c88b0 legacy=0x7f304415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f3041d9b640 1 -- 172.21.15.26:0/3522575691 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592274871 0 0) 0x7f30440c8f40 con 0x7f30440c88b0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f304259c640 1 -- 172.21.15.26:0/3522575691 >> v1:172.21.15.26:6789/0 conn(0x7f30440c5330 legacy=0x7f30441586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.682+0000 7f304259c640 1 -- 172.21.15.26:0/3522575691 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044155390 con 0x7f3044158db0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.683+0000 7f304259c640 1 --2- 172.21.15.26:0/3522575691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3044158db0 0x7f30440c5710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f302c0097a0 tx=0x7f302c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc123e365359fa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.683+0000 7f3041d9b640 1 -- 172.21.15.26:0/3522575691 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c035020 con 0x7f3044158db0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.683+0000 7f3041d9b640 1 -- 172.21.15.26:0/3522575691 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c031830 con 0x7f3044158db0 2024-03-06T07:20:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.683+0000 7f3041d9b640 1 -- 172.21.15.26:0/3522575691 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c031be0 con 0x7f3044158db0 2024-03-06T07:20:22.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.684+0000 7f30491d7640 1 -- 172.21.15.26:0/3522575691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3044158db0 msgr2=0x7f30440c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.684+0000 7f30491d7640 1 --2- 172.21.15.26:0/3522575691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3044158db0 0x7f30440c5710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f302c0097a0 tx=0x7f302c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.684+0000 7f30491d7640 1 -- 172.21.15.26:0/3522575691 shutdown_connections 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.684+0000 7f30491d7640 1 --2- 172.21.15.26:0/3522575691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3044158db0 0x7f30440c5710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.684+0000 7f30491d7640 1 -- 172.21.15.26:0/3522575691 >> 172.21.15.26:0/3522575691 conn(0x7f30440ac560 msgr2=0x7f30440ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.684+0000 7f30491d7640 1 -- 172.21.15.26:0/3522575691 shutdown_connections 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.684+0000 7f30491d7640 1 -- 172.21.15.26:0/3522575691 wait complete. 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.685+0000 7f30491d7640 1 Processor -- start 2024-03-06T07:20:22.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.685+0000 7f30491d7640 1 -- start start 2024-03-06T07:20:22.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f30491d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30440c5330 0x7f304414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f30491d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30440c88b0 0x7f30441517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f30491d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30440c4620 0x7f30441eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f30491d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30441529d0 con 0x7f30440c4620 2024-03-06T07:20:22.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f30491d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304406d090 con 0x7f30440c5330 2024-03-06T07:20:22.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f30491d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3044152b50 con 0x7f30440c88b0 2024-03-06T07:20:22.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30440c4620 0x7f30441eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f3042d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30440c5330 0x7f304414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30440c4620 0x7f30441eb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47122/0 (socket says 172.21.15.26:47122) 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 -- 172.21.15.26:0/1857061512 learned_addr learned my addr 172.21.15.26:0/1857061512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304259c640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30440c88b0 0x7f30441517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 -- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30440c88b0 msgr2=0x7f30441517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30440c88b0 0x7f30441517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 -- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30440c5330 msgr2=0x7f304414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30440c5330 0x7f304414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 -- 172.21.15.26:0/1857061512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304414e980 con 0x7f30440c4620 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304259c640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30440c88b0 0x7f30441517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f3042d9d640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30440c5330 0x7f304414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f304359e640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30440c4620 0x7f30441eb0b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f303400c730 tx=0x7f303400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.686+0000 7f303b7fe640 1 -- 172.21.15.26:0/1857061512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034003da0 con 0x7f30440c4620 2024-03-06T07:20:22.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.687+0000 7f303b7fe640 1 -- 172.21.15.26:0/1857061512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303400ce40 con 0x7f30440c4620 2024-03-06T07:20:22.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.687+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304414ec60 con 0x7f30440c4620 2024-03-06T07:20:22.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.687+0000 7f303b7fe640 1 -- 172.21.15.26:0/1857061512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30340052b0 con 0x7f30440c4620 2024-03-06T07:20:22.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.687+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30441edff0 con 0x7f30440c4620 2024-03-06T07:20:22.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.688+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304404f330 con 0x7f30440c4620 2024-03-06T07:20:22.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.688+0000 7f303b7fe640 1 -- 172.21.15.26:0/1857061512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3034005450 con 0x7f30440c4620 2024-03-06T07:20:22.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.688+0000 7f303b7fe640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f302406b4d0 0x7f302406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.689+0000 7f303b7fe640 1 -- 172.21.15.26:0/1857061512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f303408a2d0 con 0x7f30440c4620 2024-03-06T07:20:22.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.689+0000 7f3042d9d640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f302406b4d0 0x7f302406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.689+0000 7f3042d9d640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f302406b4d0 0x7f302406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f303c005880 tx=0x7f303c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.693+0000 7f303b7fe640 1 -- 172.21.15.26:0/1857061512 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3034058850 con 0x7f30440c4620 2024-03-06T07:20:22.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.874+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30440c1650 con 0x7f30440c4620 2024-03-06T07:20:22.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.875+0000 7f303b7fe640 1 -- 172.21.15.26:0/1857061512 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f303405c8f0 con 0x7f30440c4620 2024-03-06T07:20:22.877 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:22.878 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:22.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f302406b4d0 msgr2=0x7f302406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f302406b4d0 0x7f302406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f303c005880 tx=0x7f303c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30440c4620 msgr2=0x7f30441eb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30440c4620 0x7f30441eb0b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f303400c730 tx=0x7f303400cbf0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 shutdown_connections 2024-03-06T07:20:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f302406b4d0 0x7f302406d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30440c4620 0x7f30441eb0b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30440c88b0 0x7f30441517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 --2- 172.21.15.26:0/1857061512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30440c5330 0x7f304414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 >> 172.21.15.26:0/1857061512 conn(0x7f30440ac560 msgr2=0x7f304414f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 shutdown_connections 2024-03-06T07:20:22.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.877+0000 7f30491d7640 1 -- 172.21.15.26:0/1857061512 wait complete. 2024-03-06T07:20:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:22.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:22.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.994+0000 7f35ccf81640 1 Processor -- start 2024-03-06T07:20:22.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.994+0000 7f35ccf81640 1 -- start start 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35ccf81640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35c806d750 0x7f35c80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35ccf81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35c80c3a70 0x7f35c8072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35ccf81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806db30 0x7f35c80c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35ccf81640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c8155b30 con 0x7f35c80c3a70 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35ccf81640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c8156060 con 0x7f35c806d750 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35ccf81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80a8430 con 0x7f35c806db30 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35c5d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35c80c3a70 0x7f35c8072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35c659c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35c806d750 0x7f35c80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35c5d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35c80c3a70 0x7f35c8072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47126/0 (socket says 172.21.15.26:47126) 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35c5d9b640 1 -- 172.21.15.26:0/2979439766 learned_addr learned my addr 172.21.15.26:0/2979439766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.995+0000 7f35c6d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806db30 0x7f35c80c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c5d9b640 1 -- 172.21.15.26:0/2979439766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806db30 msgr2=0x7f35c80c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c5d9b640 1 --2- 172.21.15.26:0/2979439766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806db30 0x7f35c80c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c5d9b640 1 -- 172.21.15.26:0/2979439766 >> v2:172.21.15.117:3300/0 conn(0x7f35c806d750 msgr2=0x7f35c80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c5d9b640 1 --2- 172.21.15.26:0/2979439766 >> v2:172.21.15.117:3300/0 conn(0x7f35c806d750 0x7f35c80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c5d9b640 1 -- 172.21.15.26:0/2979439766 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8155d80 con 0x7f35c80c3a70 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c659c640 1 --2- 172.21.15.26:0/2979439766 >> v2:172.21.15.117:3300/0 conn(0x7f35c806d750 0x7f35c80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c5d9b640 1 --2- 172.21.15.26:0/2979439766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c80c3a70 0x7f35c8072680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35c80c5440 tx=0x7f35b002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faebb71126eac0e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c559a640 1 -- 172.21.15.26:0/2979439766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0038070 con 0x7f35c80c3a70 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.996+0000 7f35c559a640 1 -- 172.21.15.26:0/2979439766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b002f050 con 0x7f35c80c3a70 2024-03-06T07:20:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35c559a640 1 -- 172.21.15.26:0/2979439766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0031970 con 0x7f35c80c3a70 2024-03-06T07:20:23.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 -- 172.21.15.26:0/2979439766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c80c3a70 msgr2=0x7f35c8072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 --2- 172.21.15.26:0/2979439766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c80c3a70 0x7f35c8072680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35c80c5440 tx=0x7f35b002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 -- 172.21.15.26:0/2979439766 shutdown_connections 2024-03-06T07:20:23.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 --2- 172.21.15.26:0/2979439766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806db30 0x7f35c80c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 --2- 172.21.15.26:0/2979439766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c80c3a70 0x7f35c8072680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 --2- 172.21.15.26:0/2979439766 >> v2:172.21.15.117:3300/0 conn(0x7f35c806d750 0x7f35c80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 -- 172.21.15.26:0/2979439766 >> 172.21.15.26:0/2979439766 conn(0x7f35c80c5560 msgr2=0x7f35c80c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 -- 172.21.15.26:0/2979439766 shutdown_connections 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.997+0000 7f35ccf81640 1 -- 172.21.15.26:0/2979439766 wait complete. 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.998+0000 7f35ccf81640 1 Processor -- start 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.998+0000 7f35ccf81640 1 -- start start 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35ccf81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806d750 0x7f35c81df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35ccf81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c806db30 0x7f35c814efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35ccf81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c80c3a70 0x7f35c81ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35ccf81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80a7cd0 con 0x7f35c806db30 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35ccf81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8072e10 con 0x7f35c80c3a70 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35ccf81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80a7e50 con 0x7f35c806d750 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35c5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c806db30 0x7f35c814efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35c5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c806db30 0x7f35c814efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47132/0 (socket says 172.21.15.26:47132) 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35c5d9b640 1 -- 172.21.15.26:0/219570401 learned_addr learned my addr 172.21.15.26:0/219570401 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35c6d9d640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c80c3a70 0x7f35c81ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:22.999+0000 7f35c659c640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806d750 0x7f35c81df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35c5d9b640 1 -- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806d750 msgr2=0x7f35c81df9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35c5d9b640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806d750 0x7f35c81df9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35c5d9b640 1 -- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c80c3a70 msgr2=0x7f35c81ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35c5d9b640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c80c3a70 0x7f35c81ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35c5d9b640 1 -- 172.21.15.26:0/219570401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c81e09f0 con 0x7f35c806db30 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35c5d9b640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c806db30 0x7f35c814efb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009960 tx=0x7f35b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35beffd640 1 -- 172.21.15.26:0/219570401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0042020 con 0x7f35c806db30 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35beffd640 1 -- 172.21.15.26:0/219570401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b003d680 con 0x7f35c806db30 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c81e0cd0 con 0x7f35c806db30 2024-03-06T07:20:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35beffd640 1 -- 172.21.15.26:0/219570401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0038040 con 0x7f35c806db30 2024-03-06T07:20:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.000+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c81df1d0 con 0x7f35c806db30 2024-03-06T07:20:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.001+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c804f330 con 0x7f35c806db30 2024-03-06T07:20:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.001+0000 7f35beffd640 1 -- 172.21.15.26:0/219570401 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35b0045020 con 0x7f35c806db30 2024-03-06T07:20:23.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.002+0000 7f35beffd640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35a806b4d0 0x7f35a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.002+0000 7f35beffd640 1 -- 172.21.15.26:0/219570401 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f35b00abcb0 con 0x7f35c806db30 2024-03-06T07:20:23.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.002+0000 7f35c659c640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35a806b4d0 0x7f35a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.002+0000 7f35c659c640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35a806b4d0 0x7f35a806d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009bd0 tx=0x7f35b800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.006+0000 7f35beffd640 1 -- 172.21.15.26:0/219570401 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35b004c030 con 0x7f35c806db30 2024-03-06T07:20:23.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.188+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35c80c2fc0 con 0x7f35c806db30 2024-03-06T07:20:23.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.188+0000 7f35beffd640 1 -- 172.21.15.26:0/219570401 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35b007a230 con 0x7f35c806db30 2024-03-06T07:20:23.190 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:23.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35a806b4d0 msgr2=0x7f35a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35a806b4d0 0x7f35a806d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009bd0 tx=0x7f35b800d040 comp rx=0 tx=0).stop 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c806db30 msgr2=0x7f35c814efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c806db30 0x7f35c814efb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009960 tx=0x7f35b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 shutdown_connections 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35a806b4d0 0x7f35a806d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c80c3a70 0x7f35c81ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c806db30 0x7f35c814efb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 --2- 172.21.15.26:0/219570401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35c806d750 0x7f35c81df9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 >> 172.21.15.26:0/219570401 conn(0x7f35c80c5560 msgr2=0x7f35c80c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 shutdown_connections 2024-03-06T07:20:23.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.190+0000 7f35ccf81640 1 -- 172.21.15.26:0/219570401 wait complete. 2024-03-06T07:20:23.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.305+0000 7f520ebe9640 1 Processor -- start 2024-03-06T07:20:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.305+0000 7f520ebe9640 1 -- start start 2024-03-06T07:20:23.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520ebe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c4740 0x7f52081539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520ebe9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52080c69a0 0x7f5208072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520ebe9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52080c62a0 con 0x7f52080a8290 2024-03-06T07:20:23.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520ebe9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5208153f00 con 0x7f52080c69a0 2024-03-06T07:20:23.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520ebe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52080c4c00 con 0x7f52080c4740 2024-03-06T07:20:23.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520d186640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52080c69a0 0x7f5208072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520c985640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52080a8290 0x7f52080701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44242/0 (socket says 172.21.15.26:44242) 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f520c985640 1 -- 172.21.15.26:0/4280181310 learned_addr learned my addr 172.21.15.26:0/4280181310 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.306+0000 7f5207fff640 1 --2- 172.21.15.26:0/4280181310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c4740 0x7f52081539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f520d186640 1 -- 172.21.15.26:0/4280181310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c4740 msgr2=0x7f52081539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f520d186640 1 --2- 172.21.15.26:0/4280181310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c4740 0x7f52081539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f520d186640 1 -- 172.21.15.26:0/4280181310 >> v1:172.21.15.26:6789/0 conn(0x7f52080a8290 legacy=0x7f52080701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f520d186640 1 -- 172.21.15.26:0/4280181310 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080c4ee0 con 0x7f52080c69a0 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f52077fe640 1 -- 172.21.15.26:0/4280181310 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171723941 0 0) 0x7f52080c62a0 con 0x7f52080a8290 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f520d186640 1 --2- 172.21.15.26:0/4280181310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080c69a0 0x7f5208072680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f80097a0 tx=0x7f51f802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70205bc5f852947e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f52077fe640 1 -- 172.21.15.26:0/4280181310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8038070 con 0x7f52080c69a0 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f52077fe640 1 -- 172.21.15.26:0/4280181310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f8031700 con 0x7f52080c69a0 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.307+0000 7f52077fe640 1 -- 172.21.15.26:0/4280181310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8031ad0 con 0x7f52080c69a0 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 -- 172.21.15.26:0/4280181310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080c69a0 msgr2=0x7f5208072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 --2- 172.21.15.26:0/4280181310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080c69a0 0x7f5208072680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f80097a0 tx=0x7f51f802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 -- 172.21.15.26:0/4280181310 shutdown_connections 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 --2- 172.21.15.26:0/4280181310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080c69a0 0x7f5208072680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 --2- 172.21.15.26:0/4280181310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c4740 0x7f52081539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 -- 172.21.15.26:0/4280181310 >> 172.21.15.26:0/4280181310 conn(0x7f52080c5560 msgr2=0x7f52080c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 -- 172.21.15.26:0/4280181310 shutdown_connections 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.308+0000 7f520ebe9640 1 -- 172.21.15.26:0/4280181310 wait complete. 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.309+0000 7f520ebe9640 1 Processor -- start 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.309+0000 7f520ebe9640 1 -- start start 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520ebe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080a8290 0x7f52081ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520ebe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52080c4740 0x7f520814bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520ebe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c69a0 0x7f52081566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520ebe9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080c6e50 con 0x7f52080a8290 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520ebe9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520806fc80 con 0x7f52080c4740 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520ebe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5208072bc0 con 0x7f52080c69a0 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080a8290 0x7f52081ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080a8290 0x7f52081ddbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47146/0 (socket says 172.21.15.26:47146) 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 -- 172.21.15.26:0/1564324869 learned_addr learned my addr 172.21.15.26:0/1564324869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f5207fff640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52080c4740 0x7f520814bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520d186640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c69a0 0x7f52081566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 -- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c69a0 msgr2=0x7f52081566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c69a0 0x7f52081566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 -- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52080c4740 msgr2=0x7f520814bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52080c4740 0x7f520814bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.310+0000 7f520c985640 1 -- 172.21.15.26:0/1564324869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520814bc30 con 0x7f52080a8290 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.311+0000 7f520c985640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080a8290 0x7f52081ddbf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f520000c890 tx=0x7f520000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.311+0000 7f52057fa640 1 -- 172.21.15.26:0/1564324869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200018070 con 0x7f52080a8290 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.311+0000 7f52057fa640 1 -- 172.21.15.26:0/1564324869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520000a640 con 0x7f52080a8290 2024-03-06T07:20:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.311+0000 7f52057fa640 1 -- 172.21.15.26:0/1564324869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520000aa30 con 0x7f52080a8290 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.311+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52081e8820 con 0x7f52080a8290 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.311+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52081e8dc0 con 0x7f52080a8290 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.312+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520804f330 con 0x7f52080a8290 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.312+0000 7f52057fa640 1 -- 172.21.15.26:0/1564324869 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f520001b070 con 0x7f52080a8290 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.313+0000 7f52057fa640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.313+0000 7f52057fa640 1 -- 172.21.15.26:0/1564324869 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5200089a40 con 0x7f52080a8290 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.313+0000 7f5207fff640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.313+0000 7f5207fff640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5208150100 tx=0x7f51fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.317+0000 7f52057fa640 1 -- 172.21.15.26:0/1564324869 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5200057fc0 con 0x7f52080a8290 2024-03-06T07:20:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.497+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52080735e0 con 0x7f52080a8290 2024-03-06T07:20:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.497+0000 7f52057fa640 1 -- 172.21.15.26:0/1564324869 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f520005c060 con 0x7f52080a8290 2024-03-06T07:20:23.499 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 msgr2=0x7f51f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5208150100 tx=0x7f51fc009190 comp rx=0 tx=0).stop 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080a8290 msgr2=0x7f52081ddbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080a8290 0x7f52081ddbf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f520000c890 tx=0x7f520000cd50 comp rx=0 tx=0).stop 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 shutdown_connections 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52080c69a0 0x7f52081566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52080c4740 0x7f520814bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 --2- 172.21.15.26:0/1564324869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52080a8290 0x7f52081ddbf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 >> 172.21.15.26:0/1564324869 conn(0x7f52080c5560 msgr2=0x7f520814d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 shutdown_connections 2024-03-06T07:20:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.499+0000 7f520ebe9640 1 -- 172.21.15.26:0/1564324869 wait complete. 2024-03-06T07:20:23.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.615+0000 7fd3ffe64640 1 Processor -- start 2024-03-06T07:20:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.615+0000 7fd3ffe64640 1 -- start start 2024-03-06T07:20:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3ffe64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3f81528d0 0x7fd3f80724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3ffe64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f8072a10 0x7fd3f8155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3ffe64640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f8152cb0 con 0x7fd3f80c67f0 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3ffe64640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f80c38b0 con 0x7fd3f81528d0 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3ffe64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f81594f0 con 0x7fd3f8072a10 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fdc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3f81528d0 0x7fd3f80724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fdc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3f81528d0 0x7fd3f80724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47158/0 (socket says 172.21.15.26:47158) 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1964238172 learned_addr learned my addr 172.21.15.26:0/1964238172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fd3ff640 1 --2- 172.21.15.26:0/1964238172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f8072a10 0x7fd3f8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1964238172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f8072a10 msgr2=0x7fd3f8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fdc00640 1 --2- 172.21.15.26:0/1964238172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f8072a10 0x7fd3f8155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1964238172 >> v1:172.21.15.26:6789/0 conn(0x7fd3f80c67f0 legacy=0x7fd3f8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.616+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1964238172 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80a7f70 con 0x7fd3f81528d0 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.617+0000 7fd3fdc00640 1 --2- 172.21.15.26:0/1964238172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f81528d0 0x7fd3f80724d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009870 tx=0x7fd3e802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f8d9cb9b65d661 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.617+0000 7fd3fcbfe640 1 -- 172.21.15.26:0/1964238172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e802f840 con 0x7fd3f81528d0 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.617+0000 7fd3fcbfe640 1 -- 172.21.15.26:0/1964238172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e802f9e0 con 0x7fd3f81528d0 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.617+0000 7fd3fcbfe640 1 -- 172.21.15.26:0/1964238172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e802fdb0 con 0x7fd3f81528d0 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1964238172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f81528d0 msgr2=0x7fd3f80724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1964238172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f81528d0 0x7fd3f80724d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009870 tx=0x7fd3e802ba20 comp rx=0 tx=0).stop 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1964238172 shutdown_connections 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1964238172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f8072a10 0x7fd3f8155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1964238172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f81528d0 0x7fd3f80724d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1964238172 >> 172.21.15.26:0/1964238172 conn(0x7fd3f80aa6a0 msgr2=0x7fd3f80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1964238172 shutdown_connections 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.618+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1964238172 wait complete. 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 Processor -- start 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 -- start start 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f8072a10 0x7fd3f814b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f80c67f0 0x7fd3f8153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f81528d0 0x7fd3f81538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f81591a0 con 0x7fd3f8072a10 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80a8250 con 0x7fd3f80c67f0 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.619+0000 7fd3ffe64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80c4ef0 con 0x7fd3f81528d0 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f8072a10 0x7fd3f814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f8072a10 0x7fd3f814b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47172/0 (socket says 172.21.15.26:47172) 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1143762333 learned_addr learned my addr 172.21.15.26:0/1143762333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fd3ff640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f80c67f0 0x7fd3f8153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fe401640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f81528d0 0x7fd3f81538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f81528d0 msgr2=0x7fd3f81538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f81528d0 0x7fd3f81538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f80c67f0 msgr2=0x7fd3f8153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f80c67f0 0x7fd3f8153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 -- 172.21.15.26:0/1143762333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f81503f0 con 0x7fd3f8072a10 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3fdc00640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f8072a10 0x7fd3f814b640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009870 tx=0x7fd3e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3ee7fc640 1 -- 172.21.15.26:0/1143762333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8037bc0 con 0x7fd3f8072a10 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3ee7fc640 1 -- 172.21.15.26:0/1143762333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e8046070 con 0x7fd3f8072a10 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f8150110 con 0x7fd3f8072a10 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3ee7fc640 1 -- 172.21.15.26:0/1143762333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8041400 con 0x7fd3f8072a10 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.620+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f814bb80 con 0x7fd3f8072a10 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.621+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f804f330 con 0x7fd3f8072a10 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.622+0000 7fd3ee7fc640 1 -- 172.21.15.26:0/1143762333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3e8040070 con 0x7fd3f8072a10 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.622+0000 7fd3ee7fc640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.622+0000 7fd3ee7fc640 1 -- 172.21.15.26:0/1143762333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd3e80ac980 con 0x7fd3f8072a10 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.622+0000 7fd3fd3ff640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.623+0000 7fd3fd3ff640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8154910 tx=0x7fd3f4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.626+0000 7fd3ee7fc640 1 -- 172.21.15.26:0/1143762333 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd3e8033030 con 0x7fd3f8072a10 2024-03-06T07:20:23.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.806+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3f806e020 con 0x7fd3f8072a10 2024-03-06T07:20:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.806+0000 7fd3ee7fc640 1 -- 172.21.15.26:0/1143762333 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3e807b020 con 0x7fd3f8072a10 2024-03-06T07:20:23.808 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3dc06b4d0 msgr2=0x7fd3dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8154910 tx=0x7fd3f4009190 comp rx=0 tx=0).stop 2024-03-06T07:20:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f8072a10 msgr2=0x7fd3f814b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f8072a10 0x7fd3f814b640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009870 tx=0x7fd3e8004730 comp rx=0 tx=0).stop 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 shutdown_connections 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f81528d0 0x7fd3f81538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f80c67f0 0x7fd3f8153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 --2- 172.21.15.26:0/1143762333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f8072a10 0x7fd3f814b640 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 >> 172.21.15.26:0/1143762333 conn(0x7fd3f80aa6a0 msgr2=0x7fd3f80c5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 shutdown_connections 2024-03-06T07:20:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.808+0000 7fd3ffe64640 1 -- 172.21.15.26:0/1143762333 wait complete. 2024-03-06T07:20:23.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:23.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.924+0000 7f7876116640 1 Processor -- start 2024-03-06T07:20:23.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.924+0000 7f7876116640 1 -- start start 2024-03-06T07:20:23.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f7876116640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78700c5370 0x7f78700c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f7876116640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7870155530 0x7f78701559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f7876116640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7870152dd0 con 0x7f78701529f0 2024-03-06T07:20:23.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f7876116640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7870159710 con 0x7f7870155530 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f7876116640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78700c68b0 con 0x7f78700c5370 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7870155530 0x7f78701559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786effd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78700c5370 0x7f78700c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786f7fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f78701529f0 0x7f78700c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:36010/0 (socket says 172.21.15.26:36010) 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7870155530 0x7f78701559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37318/0 (socket says 172.21.15.26:37318) 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786f7fe640 1 -- 172.21.15.26:0/2445560397 learned_addr learned my addr 172.21.15.26:0/2445560397 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786ffff640 1 -- 172.21.15.26:0/2445560397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78700c5370 msgr2=0x7f78700c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786ffff640 1 --2- 172.21.15.26:0/2445560397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78700c5370 0x7f78700c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786ffff640 1 -- 172.21.15.26:0/2445560397 >> v1:172.21.15.117:6789/0 conn(0x7f78701529f0 legacy=0x7f78700c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786ffff640 1 -- 172.21.15.26:0/2445560397 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7870158b00 con 0x7f7870155530 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.925+0000 7f786e7fc640 1 -- 172.21.15.26:0/2445560397 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677810972 0 0) 0x7f7870152dd0 con 0x7f78701529f0 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f786ffff640 1 --2- 172.21.15.26:0/2445560397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7870155530 0x7f78701559c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7868009d90 tx=0x7f786802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4678f6c5453f72d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f786e7fc640 1 -- 172.21.15.26:0/2445560397 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868039070 con 0x7f7870155530 2024-03-06T07:20:23.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f786e7fc640 1 -- 172.21.15.26:0/2445560397 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786802cbc0 con 0x7f7870155530 2024-03-06T07:20:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f786e7fc640 1 -- 172.21.15.26:0/2445560397 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868034440 con 0x7f7870155530 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f7876116640 1 -- 172.21.15.26:0/2445560397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7870155530 msgr2=0x7f78701559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f7876116640 1 --2- 172.21.15.26:0/2445560397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7870155530 0x7f78701559c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7868009d90 tx=0x7f786802be10 comp rx=0 tx=0).stop 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f7876116640 1 -- 172.21.15.26:0/2445560397 shutdown_connections 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f7876116640 1 --2- 172.21.15.26:0/2445560397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7870155530 0x7f78701559c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f7876116640 1 --2- 172.21.15.26:0/2445560397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78700c5370 0x7f78700c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.926+0000 7f7876116640 1 -- 172.21.15.26:0/2445560397 >> 172.21.15.26:0/2445560397 conn(0x7f78700ac580 msgr2=0x7f78700ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.927+0000 7f7876116640 1 -- 172.21.15.26:0/2445560397 shutdown_connections 2024-03-06T07:20:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.927+0000 7f7876116640 1 -- 172.21.15.26:0/2445560397 wait complete. 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 Processor -- start 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 -- start start 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78700c5370 0x7f7870157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78701529f0 0x7f78701ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7870155530 0x7f78701eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78701590c0 con 0x7f78700c5370 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78700c9000 con 0x7f78701529f0 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.928+0000 7f7876116640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78701593a0 con 0x7f7870155530 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78700c5370 0x7f7870157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78701529f0 0x7f78701ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78701529f0 0x7f78701ee8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37326/0 (socket says 172.21.15.26:37326) 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78700c5370 0x7f7870157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47184/0 (socket says 172.21.15.26:47184) 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 -- 172.21.15.26:0/1661847836 learned_addr learned my addr 172.21.15.26:0/1661847836 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786ffff640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7870155530 0x7f78701eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 -- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7870155530 msgr2=0x7f78701eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7870155530 0x7f78701eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 -- 172.21.15.26:0/1661847836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78700c5370 msgr2=0x7f7870157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78700c5370 0x7f7870157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 -- 172.21.15.26:0/1661847836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700bee80 con 0x7f78701529f0 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786f7fe640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78700c5370 0x7f7870157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f786effd640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78701529f0 0x7f78701ee8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f786000e9b0 tx=0x7f786000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f7857fff640 1 -- 172.21.15.26:0/1661847836 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860017070 con 0x7f78701529f0 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f7857fff640 1 -- 172.21.15.26:0/1661847836 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786000c9a0 con 0x7f78701529f0 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78700c48c0 con 0x7f78701529f0 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f7857fff640 1 -- 172.21.15.26:0/1661847836 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786000cd90 con 0x7f78701529f0 2024-03-06T07:20:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.929+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78700bf120 con 0x7f78701529f0 2024-03-06T07:20:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.930+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787004f330 con 0x7f78701529f0 2024-03-06T07:20:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.931+0000 7f7857fff640 1 -- 172.21.15.26:0/1661847836 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f786001a070 con 0x7f78701529f0 2024-03-06T07:20:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.931+0000 7f7857fff640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f785c06b500 0x7f785c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.931+0000 7f7857fff640 1 -- 172.21.15.26:0/1661847836 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f786008a460 con 0x7f78701529f0 2024-03-06T07:20:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.931+0000 7f786f7fe640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f785c06b500 0x7f785c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.932+0000 7f786f7fe640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f785c06b500 0x7f785c06d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7864000f60 tx=0x7f7864007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:23.936+0000 7f7857fff640 1 -- 172.21.15.26:0/1661847836 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7860058950 con 0x7f78701529f0 2024-03-06T07:20:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.117+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f78700c4af0 con 0x7f78701529f0 2024-03-06T07:20:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.117+0000 7f7857fff640 1 -- 172.21.15.26:0/1661847836 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f786005c9f0 con 0x7f78701529f0 2024-03-06T07:20:24.119 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f785c06b500 msgr2=0x7f785c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f785c06b500 0x7f785c06d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7864000f60 tx=0x7f7864007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78701529f0 msgr2=0x7f78701ee8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78701529f0 0x7f78701ee8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f786000e9b0 tx=0x7f786000ee70 comp rx=0 tx=0).stop 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 shutdown_connections 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f785c06b500 0x7f785c06d9c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7870155530 0x7f78701eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78701529f0 0x7f78701ee8b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 --2- 172.21.15.26:0/1661847836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78700c5370 0x7f7870157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 >> 172.21.15.26:0/1661847836 conn(0x7f78700ac580 msgr2=0x7f787006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 shutdown_connections 2024-03-06T07:20:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.120+0000 7f7876116640 1 -- 172.21.15.26:0/1661847836 wait complete. 2024-03-06T07:20:24.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.232+0000 7f4bf044e640 1 Processor -- start 2024-03-06T07:20:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.232+0000 7f4bf044e640 1 -- start start 2024-03-06T07:20:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bf044e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4be8152920 0x7f4be8155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bf044e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c2620 0x7f4be80c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bf044e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be806afc0 con 0x7f4be80c2190 2024-03-06T07:20:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bf044e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8156160 con 0x7f4be8152920 2024-03-06T07:20:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bf044e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be806b2a0 con 0x7f4be80c2620 2024-03-06T07:20:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bee1ea640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4be8152920 0x7f4be8155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bed9e9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4be80c2190 0x7f4be80c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44262/0 (socket says 172.21.15.26:44262) 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bee9eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c2620 0x7f4be80c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.233+0000 7f4bed9e9640 1 -- 172.21.15.26:0/3664037855 learned_addr learned my addr 172.21.15.26:0/3664037855 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.234+0000 7f4bee9eb640 1 -- 172.21.15.26:0/3664037855 >> v2:172.21.15.26:3300/0 conn(0x7f4be8152920 msgr2=0x7f4be8155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.234+0000 7f4bee9eb640 1 --2- 172.21.15.26:0/3664037855 >> v2:172.21.15.26:3300/0 conn(0x7f4be8152920 0x7f4be8155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.234+0000 7f4bee9eb640 1 -- 172.21.15.26:0/3664037855 >> v1:172.21.15.26:6789/0 conn(0x7f4be80c2190 legacy=0x7f4be80c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.234+0000 7f4bee9eb640 1 -- 172.21.15.26:0/3664037855 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be806b580 con 0x7f4be80c2620 2024-03-06T07:20:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.234+0000 7f4bee1ea640 1 --2- 172.21.15.26:0/3664037855 >> v2:172.21.15.26:3300/0 conn(0x7f4be8152920 0x7f4be8155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.234+0000 7f4bee9eb640 1 --2- 172.21.15.26:0/3664037855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c2620 0x7f4be80c1990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009d90 tx=0x7f4be402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142c2a866e8774b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bed1e8640 1 -- 172.21.15.26:0/3664037855 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4039070 con 0x7f4be80c2620 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bed1e8640 1 -- 172.21.15.26:0/3664037855 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be402cc50 con 0x7f4be80c2620 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bed1e8640 1 -- 172.21.15.26:0/3664037855 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4034440 con 0x7f4be80c2620 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 -- 172.21.15.26:0/3664037855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c2620 msgr2=0x7f4be80c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 --2- 172.21.15.26:0/3664037855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c2620 0x7f4be80c1990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009d90 tx=0x7f4be402bea0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 -- 172.21.15.26:0/3664037855 shutdown_connections 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 --2- 172.21.15.26:0/3664037855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c2620 0x7f4be80c1990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 --2- 172.21.15.26:0/3664037855 >> v2:172.21.15.26:3300/0 conn(0x7f4be8152920 0x7f4be8155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 -- 172.21.15.26:0/3664037855 >> 172.21.15.26:0/3664037855 conn(0x7f4be80c4290 msgr2=0x7f4be80c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 -- 172.21.15.26:0/3664037855 shutdown_connections 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.235+0000 7f4bf044e640 1 -- 172.21.15.26:0/3664037855 wait complete. 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.236+0000 7f4bf044e640 1 Processor -- start 2024-03-06T07:20:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.236+0000 7f4bf044e640 1 -- start start 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bf044e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c2190 0x7f4be814f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bf044e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be80c2620 0x7f4be81eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bf044e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8152920 0x7f4be81ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bf044e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80a7100 con 0x7f4be80c2190 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bf044e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80c0d60 con 0x7f4be80c2620 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bf044e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80a7280 con 0x7f4be8152920 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c2190 0x7f4be814f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c2190 0x7f4be814f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47200/0 (socket says 172.21.15.26:47200) 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 -- 172.21.15.26:0/1493054060 learned_addr learned my addr 172.21.15.26:0/1493054060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee9eb640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8152920 0x7f4be81ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bed9e9640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be80c2620 0x7f4be81eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 -- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8152920 msgr2=0x7f4be81ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8152920 0x7f4be81ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 -- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be80c2620 msgr2=0x7f4be81eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be80c2620 0x7f4be81eb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee1ea640 1 -- 172.21.15.26:0/1493054060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8073180 con 0x7f4be80c2190 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.237+0000 7f4bee9eb640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8152920 0x7f4be81ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.238+0000 7f4bee1ea640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c2190 0x7f4be814f550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4be000e960 tx=0x7f4be000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.238+0000 7f4bdeffd640 1 -- 172.21.15.26:0/1493054060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0017070 con 0x7f4be80c2190 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.238+0000 7f4bdeffd640 1 -- 172.21.15.26:0/1493054060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be000ca00 con 0x7f4be80c2190 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.238+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be8073420 con 0x7f4be80c2190 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.238+0000 7f4bdeffd640 1 -- 172.21.15.26:0/1493054060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000cdd0 con 0x7f4be80c2190 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.238+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be814fa90 con 0x7f4be80c2190 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.239+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be804f330 con 0x7f4be80c2190 2024-03-06T07:20:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.239+0000 7f4bdeffd640 1 -- 172.21.15.26:0/1493054060 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4be001a070 con 0x7f4be80c2190 2024-03-06T07:20:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.240+0000 7f4bdeffd640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.240+0000 7f4bdeffd640 1 -- 172.21.15.26:0/1493054060 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4be008a2c0 con 0x7f4be80c2190 2024-03-06T07:20:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.240+0000 7f4bed9e9640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.240+0000 7f4bed9e9640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4be8155f40 tx=0x7f4bd8007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.244+0000 7f4bdeffd640 1 -- 172.21.15.26:0/1493054060 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4be0058840 con 0x7f4be80c2190 2024-03-06T07:20:24.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.462+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be8068460 con 0x7f4be80c2190 2024-03-06T07:20:24.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.463+0000 7f4bdeffd640 1 -- 172.21.15.26:0/1493054060 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7f4be008a0e0 con 0x7f4be80c2190 2024-03-06T07:20:24.465 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:24.465 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bcc06b500 msgr2=0x7f4bcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4be8155f40 tx=0x7f4bd8007e80 comp rx=0 tx=0).stop 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c2190 msgr2=0x7f4be814f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c2190 0x7f4be814f550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4be000e960 tx=0x7f4be000ee20 comp rx=0 tx=0).stop 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 shutdown_connections 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8152920 0x7f4be81ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be80c2620 0x7f4be81eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 --2- 172.21.15.26:0/1493054060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c2190 0x7f4be814f550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 >> 172.21.15.26:0/1493054060 conn(0x7f4be80c4290 msgr2=0x7f4be814d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 shutdown_connections 2024-03-06T07:20:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.465+0000 7f4bf044e640 1 -- 172.21.15.26:0/1493054060 wait complete. 2024-03-06T07:20:24.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.577+0000 7fea8cd3e640 1 Processor -- start 2024-03-06T07:20:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.577+0000 7fea8cd3e640 1 -- start start 2024-03-06T07:20:24.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea8cd3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c8a00 0x7fea88158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea8cd3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea880c7760 con 0x7fea88159460 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea8cd3e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea880c7a40 con 0x7fea88152920 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea8cd3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea881587b0 con 0x7fea880c8a00 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea86d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fea88159460 0x7fea880c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44266/0 (socket says 172.21.15.26:44266) 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea86d9d640 1 -- 172.21.15.26:0/1133338716 learned_addr learned my addr 172.21.15.26:0/1133338716 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea7ffff640 1 --2- 172.21.15.26:0/1133338716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c8a00 0x7fea88158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea85d9b640 1 -- 172.21.15.26:0/1133338716 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882754552 0 0) 0x7fea880c7760 con 0x7fea88159460 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea85d9b640 1 -- 172.21.15.26:0/1133338716 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea74003520 con 0x7fea88159460 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea85d9b640 1 -- 172.21.15.26:0/1133338716 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23778369 0 0) 0x7fea880c7a40 con 0x7fea88152920 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea7ffff640 1 -- 172.21.15.26:0/1133338716 >> v1:172.21.15.117:6789/0 conn(0x7fea88152920 legacy=0x7fea88155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea7ffff640 1 -- 172.21.15.26:0/1133338716 >> v1:172.21.15.26:6789/0 conn(0x7fea88159460 legacy=0x7fea880c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.578+0000 7fea7ffff640 1 -- 172.21.15.26:0/1133338716 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea880c7d20 con 0x7fea880c8a00 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.579+0000 7fea7ffff640 1 --2- 172.21.15.26:0/1133338716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c8a00 0x7fea88158f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea700097d0 tx=0x7fea7002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74142e21e1a689d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.579+0000 7fea85d9b640 1 -- 172.21.15.26:0/1133338716 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70035020 con 0x7fea880c8a00 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.579+0000 7fea85d9b640 1 -- 172.21.15.26:0/1133338716 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea70031800 con 0x7fea880c8a00 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.579+0000 7fea85d9b640 1 -- 172.21.15.26:0/1133338716 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70031bd0 con 0x7fea880c8a00 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.580+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1133338716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c8a00 msgr2=0x7fea88158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.580+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1133338716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c8a00 0x7fea88158f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea700097d0 tx=0x7fea7002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.580+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1133338716 shutdown_connections 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.580+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1133338716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c8a00 0x7fea88158f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.580+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1133338716 >> 172.21.15.26:0/1133338716 conn(0x7fea880ac560 msgr2=0x7fea880ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.580+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1133338716 shutdown_connections 2024-03-06T07:20:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.580+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1133338716 wait complete. 2024-03-06T07:20:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.581+0000 7fea8cd3e640 1 Processor -- start 2024-03-06T07:20:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.581+0000 7fea8cd3e640 1 -- start start 2024-03-06T07:20:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8cd3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea880c8a00 0x7fea881574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8cd3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea88152920 0x7fea881e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8cd3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c1530 0x7fea881e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8cd3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c5a40 con 0x7fea88152920 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8cd3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8806d030 con 0x7fea880c8a00 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8cd3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c5bc0 con 0x7fea880c1530 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea88152920 0x7fea881e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea88152920 0x7fea881e7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47202/0 (socket says 172.21.15.26:47202) 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 -- 172.21.15.26:0/1844339191 learned_addr learned my addr 172.21.15.26:0/1844339191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea86d9d640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c1530 0x7fea881e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8659c640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea880c8a00 0x7fea881574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 -- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c1530 msgr2=0x7fea881e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c1530 0x7fea881e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 -- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea880c8a00 msgr2=0x7fea881574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea880c8a00 0x7fea881574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 -- 172.21.15.26:0/1844339191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88155a70 con 0x7fea88152920 2024-03-06T07:20:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea86d9d640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c1530 0x7fea881e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7ffff640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea88152920 0x7fea881e7c90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea700098f0 tx=0x7fea700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7effd640 1 -- 172.21.15.26:0/1844339191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70041070 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7effd640 1 -- 172.21.15.26:0/1844339191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7003ac80 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea881eeab0 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.582+0000 7fea7effd640 1 -- 172.21.15.26:0/1844339191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70035040 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.583+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea881eed90 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.584+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8804f330 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.584+0000 7fea7effd640 1 -- 172.21.15.26:0/1844339191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea70038020 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.584+0000 7fea7effd640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea6806b4d0 0x7fea6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.584+0000 7fea7effd640 1 -- 172.21.15.26:0/1844339191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea700abfa0 con 0x7fea88152920 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.584+0000 7fea8659c640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea6806b4d0 0x7fea6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.585+0000 7fea8659c640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea6806b4d0 0x7fea6806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea78000f60 tx=0x7fea78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.589+0000 7fea7effd640 1 -- 172.21.15.26:0/1844339191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea7007a520 con 0x7fea88152920 2024-03-06T07:20:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.768+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea8806cda0 con 0x7fea88152920 2024-03-06T07:20:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.769+0000 7fea7effd640 1 -- 172.21.15.26:0/1844339191 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fea7007e5c0 con 0x7fea88152920 2024-03-06T07:20:24.771 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:24.772 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea6806b4d0 msgr2=0x7fea6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea6806b4d0 0x7fea6806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea78000f60 tx=0x7fea78009040 comp rx=0 tx=0).stop 2024-03-06T07:20:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea88152920 msgr2=0x7fea881e7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea88152920 0x7fea881e7c90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea700098f0 tx=0x7fea700046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 shutdown_connections 2024-03-06T07:20:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea6806b4d0 0x7fea6806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea880c1530 0x7fea881e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea88152920 0x7fea881e7c90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 --2- 172.21.15.26:0/1844339191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea880c8a00 0x7fea881574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 >> 172.21.15.26:0/1844339191 conn(0x7fea880ac560 msgr2=0x7fea88151140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 shutdown_connections 2024-03-06T07:20:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.771+0000 7fea8cd3e640 1 -- 172.21.15.26:0/1844339191 wait complete. 2024-03-06T07:20:24.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:24.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.889+0000 7f997f181640 1 Processor -- start 2024-03-06T07:20:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.889+0000 7f997f181640 1 -- start start 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997f181640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f99781539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997f181640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99780c69a0 0x7f9978072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997f181640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99780c62a0 con 0x7f99780a8290 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997f181640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9978153f00 con 0x7f99780c69a0 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997f181640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c4c00 con 0x7f99780c4740 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997d71e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99780c69a0 0x7f9978072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997cf1d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f99780a8290 0x7f99780701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44270/0 (socket says 172.21.15.26:44270) 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f997cf1d640 1 -- 172.21.15.26:0/287121406 learned_addr learned my addr 172.21.15.26:0/287121406 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.890+0000 7f9977fff640 1 --2- 172.21.15.26:0/287121406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f99781539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f997d71e640 1 -- 172.21.15.26:0/287121406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 msgr2=0x7f99781539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f997d71e640 1 --2- 172.21.15.26:0/287121406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f99781539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f997d71e640 1 -- 172.21.15.26:0/287121406 >> v1:172.21.15.26:6789/0 conn(0x7f99780a8290 legacy=0x7f99780701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f99777fe640 1 -- 172.21.15.26:0/287121406 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701950062 0 0) 0x7f99780c62a0 con 0x7f99780a8290 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f997d71e640 1 -- 172.21.15.26:0/287121406 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780c4ee0 con 0x7f99780c69a0 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f9977fff640 1 --2- 172.21.15.26:0/287121406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f99781539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f997d71e640 1 --2- 172.21.15.26:0/287121406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f9978072680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99680097a0 tx=0x7f996802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978b97febd23d3b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.891+0000 7f99777fe640 1 -- 172.21.15.26:0/287121406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968038070 con 0x7f99780c69a0 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f99777fe640 1 -- 172.21.15.26:0/287121406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9968031700 con 0x7f99780c69a0 2024-03-06T07:20:24.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f99777fe640 1 -- 172.21.15.26:0/287121406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968031ad0 con 0x7f99780c69a0 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 -- 172.21.15.26:0/287121406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 msgr2=0x7f9978072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 --2- 172.21.15.26:0/287121406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f9978072680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99680097a0 tx=0x7f996802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 -- 172.21.15.26:0/287121406 shutdown_connections 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 --2- 172.21.15.26:0/287121406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f9978072680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 --2- 172.21.15.26:0/287121406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f99781539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 -- 172.21.15.26:0/287121406 >> 172.21.15.26:0/287121406 conn(0x7f99780c5560 msgr2=0x7f99780c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 -- 172.21.15.26:0/287121406 shutdown_connections 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.892+0000 7f997f181640 1 -- 172.21.15.26:0/287121406 wait complete. 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.893+0000 7f997f181640 1 Processor -- start 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.893+0000 7f997f181640 1 -- start start 2024-03-06T07:20:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997f181640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99780a8290 0x7f997814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997f181640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f997814ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997f181640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f997814b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997f181640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c6e50 con 0x7f99780c69a0 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997f181640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997806fc80 con 0x7f99780a8290 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997f181640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9978072bc0 con 0x7f99780c4740 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997d71e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f997814b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997d71e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f997814b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47226/0 (socket says 172.21.15.26:47226) 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997d71e640 1 -- 172.21.15.26:0/2053185523 learned_addr learned my addr 172.21.15.26:0/2053185523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997cf1d640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99780a8290 0x7f997814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f9977fff640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f997814ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f9977fff640 1 -- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99780a8290 msgr2=0x7f997814d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f9977fff640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99780a8290 0x7f997814d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f9977fff640 1 -- 172.21.15.26:0/2053185523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 msgr2=0x7f997814b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f9977fff640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f997814b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f9977fff640 1 -- 172.21.15.26:0/2053185523 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997814db80 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.894+0000 7f997d71e640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f997814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.895+0000 7f9977fff640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f997814ed30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e930 tx=0x7f996c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.895+0000 7f99757fa640 1 -- 172.21.15.26:0/2053185523 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c017070 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.895+0000 7f99757fa640 1 -- 172.21.15.26:0/2053185523 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00ca00 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.895+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997814de60 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.895+0000 7f99757fa640 1 -- 172.21.15.26:0/2053185523 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00cdd0 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.895+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9978154a70 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.896+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997804f330 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.896+0000 7f99757fa640 1 -- 172.21.15.26:0/2053185523 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f996c01a070 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.896+0000 7f99757fa640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f996406b4d0 0x7f996406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.897+0000 7f99757fa640 1 -- 172.21.15.26:0/2053185523 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f996c08a620 con 0x7f99780c4740 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.897+0000 7f997cf1d640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f996406b4d0 0x7f996406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.897+0000 7f997cf1d640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f996406b4d0 0x7f996406d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99700012e0 tx=0x7f9970001700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:24.901+0000 7f99757fa640 1 -- 172.21.15.26:0/2053185523 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f996c058ba0 con 0x7f99780c4740 2024-03-06T07:20:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.082+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99780c8480 con 0x7f99780c4740 2024-03-06T07:20:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.083+0000 7f99757fa640 1 -- 172.21.15.26:0/2053185523 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f996c05cc40 con 0x7f99780c4740 2024-03-06T07:20:25.084 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f996406b4d0 msgr2=0x7f996406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f996406b4d0 0x7f996406d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99700012e0 tx=0x7f9970001700 comp rx=0 tx=0).stop 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 msgr2=0x7f997814ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f997814ed30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e930 tx=0x7f996c00edf0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 shutdown_connections 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f996406b4d0 0x7f996406d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99780c69a0 0x7f997814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99780c4740 0x7f997814ed30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 --2- 172.21.15.26:0/2053185523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99780a8290 0x7f997814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 >> 172.21.15.26:0/2053185523 conn(0x7f99780c5560 msgr2=0x7f99780a9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 shutdown_connections 2024-03-06T07:20:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.085+0000 7f997f181640 1 -- 172.21.15.26:0/2053185523 wait complete. 2024-03-06T07:20:25.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.202+0000 7fc52e763640 1 Processor -- start 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.202+0000 7fc52e763640 1 -- start start 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52e763640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806c1f0 0x7fc5280c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52e763640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc52806cad0 0x7fc528154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52e763640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280c2790 con 0x7fc52814ef70 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52e763640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280c3090 con 0x7fc52806cad0 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52e763640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52814f350 con 0x7fc52806c1f0 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc527fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc52814ef70 0x7fc5280c2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44278/0 (socket says 172.21.15.26:44278) 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc527fff640 1 -- 172.21.15.26:0/613256680 learned_addr learned my addr 172.21.15.26:0/613256680 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52cd00640 1 --2- 172.21.15.26:0/613256680 >> v2:172.21.15.117:3300/0 conn(0x7fc52806cad0 0x7fc528154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc5277fe640 1 --2- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806c1f0 0x7fc5280c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc526ffd640 1 -- 172.21.15.26:0/613256680 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408377283 0 0) 0x7fc5280c2790 con 0x7fc52814ef70 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc526ffd640 1 -- 172.21.15.26:0/613256680 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc510003520 con 0x7fc52814ef70 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52cd00640 1 -- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806c1f0 msgr2=0x7fc5280c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52cd00640 1 --2- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806c1f0 0x7fc5280c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52cd00640 1 -- 172.21.15.26:0/613256680 >> v1:172.21.15.26:6789/0 conn(0x7fc52814ef70 legacy=0x7fc5280c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc52cd00640 1 -- 172.21.15.26:0/613256680 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280c1540 con 0x7fc52806cad0 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.203+0000 7fc5277fe640 1 --2- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806c1f0 0x7fc5280c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.204+0000 7fc52cd00640 1 --2- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52806cad0 0x7fc528154c30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc520009d90 tx=0x7fc52002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67f9a5e8ce11a19a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.204+0000 7fc526ffd640 1 -- 172.21.15.26:0/613256680 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520039070 con 0x7fc52806cad0 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.204+0000 7fc526ffd640 1 -- 172.21.15.26:0/613256680 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52002cbc0 con 0x7fc52806cad0 2024-03-06T07:20:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.204+0000 7fc526ffd640 1 -- 172.21.15.26:0/613256680 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520034440 con 0x7fc52806cad0 2024-03-06T07:20:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 -- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52806cad0 msgr2=0x7fc528154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 --2- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52806cad0 0x7fc528154c30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc520009d90 tx=0x7fc52002be10 comp rx=0 tx=0).stop 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 -- 172.21.15.26:0/613256680 shutdown_connections 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 --2- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52806cad0 0x7fc528154c30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 --2- 172.21.15.26:0/613256680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806c1f0 0x7fc5280c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 -- 172.21.15.26:0/613256680 >> 172.21.15.26:0/613256680 conn(0x7fc5280c6710 msgr2=0x7fc5280c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 -- 172.21.15.26:0/613256680 shutdown_connections 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.205+0000 7fc52e763640 1 -- 172.21.15.26:0/613256680 wait complete. 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.206+0000 7fc52e763640 1 Processor -- start 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.206+0000 7fc52e763640 1 -- start start 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52e763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc52806c1f0 0x7fc528148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52e763640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806cad0 0x7fc5281468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52e763640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52814ef70 0x7fc5280a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52e763640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528155750 con 0x7fc52806c1f0 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52e763640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52806cf60 con 0x7fc52814ef70 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52e763640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc528155170 con 0x7fc52806cad0 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc52806c1f0 0x7fc528148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc52806c1f0 0x7fc528148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47240/0 (socket says 172.21.15.26:47240) 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 -- 172.21.15.26:0/1650224685 learned_addr learned my addr 172.21.15.26:0/1650224685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52cd00640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52814ef70 0x7fc5280a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc5277fe640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806cad0 0x7fc5281468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 -- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806cad0 msgr2=0x7fc5281468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806cad0 0x7fc5281468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 -- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52814ef70 msgr2=0x7fc5280a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52814ef70 0x7fc5280a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 -- 172.21.15.26:0/1650224685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280a7c30 con 0x7fc52806c1f0 2024-03-06T07:20:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc52cd00640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52814ef70 0x7fc5280a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.207+0000 7fc527fff640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc52806c1f0 0x7fc528148480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c3c0 tx=0x7fc51800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.208+0000 7fc524ff9640 1 -- 172.21.15.26:0/1650224685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518017070 con 0x7fc52806c1f0 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.208+0000 7fc524ff9640 1 -- 172.21.15.26:0/1650224685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518005ce0 con 0x7fc52806c1f0 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.208+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5280a7f10 con 0x7fc52806c1f0 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.208+0000 7fc524ff9640 1 -- 172.21.15.26:0/1650224685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180072f0 con 0x7fc52806c1f0 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.208+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc528145bc0 con 0x7fc52806c1f0 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.209+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52804f330 con 0x7fc52806c1f0 2024-03-06T07:20:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.209+0000 7fc524ff9640 1 -- 172.21.15.26:0/1650224685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc51801a070 con 0x7fc52806c1f0 2024-03-06T07:20:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.210+0000 7fc524ff9640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc51406b4d0 0x7fc51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.210+0000 7fc524ff9640 1 -- 172.21.15.26:0/1650224685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc518089860 con 0x7fc52806c1f0 2024-03-06T07:20:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.210+0000 7fc5277fe640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc51406b4d0 0x7fc51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.210+0000 7fc5277fe640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc51406b4d0 0x7fc51406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009560 tx=0x7fc51c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.214+0000 7fc524ff9640 1 -- 172.21.15.26:0/1650224685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc518057d50 con 0x7fc52806c1f0 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.395+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5280c1110 con 0x7fc52806c1f0 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.396+0000 7fc524ff9640 1 -- 172.21.15.26:0/1650224685 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc51805bdf0 con 0x7fc52806c1f0 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc51406b4d0 msgr2=0x7fc51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc51406b4d0 0x7fc51406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009560 tx=0x7fc51c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc52806c1f0 msgr2=0x7fc528148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc52806c1f0 0x7fc528148480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c3c0 tx=0x7fc51800c880 comp rx=0 tx=0).stop 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 shutdown_connections 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc51406b4d0 0x7fc51406d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52814ef70 0x7fc5280a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc52806cad0 0x7fc5281468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 --2- 172.21.15.26:0/1650224685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc52806c1f0 0x7fc528148480 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 >> 172.21.15.26:0/1650224685 conn(0x7fc5280c6710 msgr2=0x7fc5280c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 shutdown_connections 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.398+0000 7fc52e763640 1 -- 172.21.15.26:0/1650224685 wait complete. 2024-03-06T07:20:25.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.516+0000 7fb831f3e640 1 Processor -- start 2024-03-06T07:20:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.516+0000 7fb831f3e640 1 -- start start 2024-03-06T07:20:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb831f3e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb82c155c30 0x7fb82c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb831f3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c0c5380 0x7fb82c0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb831f3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82c0b0450 con 0x7fb82c0b0070 2024-03-06T07:20:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb831f3e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82c06cfb0 con 0x7fb82c155c30 2024-03-06T07:20:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb831f3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c06c020 con 0x7fb82c0c5380 2024-03-06T07:20:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb82c155c30 0x7fb82c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb82c0b0070 0x7fb82c0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44282/0 (socket says 172.21.15.26:44282) 2024-03-06T07:20:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82b7fe640 1 -- 172.21.15.26:0/561951432 learned_addr learned my addr 172.21.15.26:0/561951432 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82bfff640 1 --2- 172.21.15.26:0/561951432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c0c5380 0x7fb82c0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82affd640 1 -- 172.21.15.26:0/561951432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c0c5380 msgr2=0x7fb82c0c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82affd640 1 --2- 172.21.15.26:0/561951432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c0c5380 0x7fb82c0c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82affd640 1 -- 172.21.15.26:0/561951432 >> v1:172.21.15.26:6789/0 conn(0x7fb82c0b0070 legacy=0x7fb82c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82affd640 1 -- 172.21.15.26:0/561951432 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c1560e0 con 0x7fb82c155c30 2024-03-06T07:20:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.517+0000 7fb82a7fc640 1 -- 172.21.15.26:0/561951432 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255833658 0 0) 0x7fb81c001490 con 0x7fb82c0b0070 2024-03-06T07:20:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.518+0000 7fb82affd640 1 --2- 172.21.15.26:0/561951432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c155c30 0x7fb82c06ca70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb8200097a0 tx=0x7fb82002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28435d5a1c6841b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.518+0000 7fb82a7fc640 1 -- 172.21.15.26:0/561951432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8200315a0 con 0x7fb82c155c30 2024-03-06T07:20:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.518+0000 7fb82a7fc640 1 -- 172.21.15.26:0/561951432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820037070 con 0x7fb82c155c30 2024-03-06T07:20:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.518+0000 7fb82a7fc640 1 -- 172.21.15.26:0/561951432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820031960 con 0x7fb82c155c30 2024-03-06T07:20:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 -- 172.21.15.26:0/561951432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c155c30 msgr2=0x7fb82c06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 --2- 172.21.15.26:0/561951432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c155c30 0x7fb82c06ca70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb8200097a0 tx=0x7fb82002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 -- 172.21.15.26:0/561951432 shutdown_connections 2024-03-06T07:20:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 --2- 172.21.15.26:0/561951432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c155c30 0x7fb82c06ca70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 --2- 172.21.15.26:0/561951432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c0c5380 0x7fb82c0c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 -- 172.21.15.26:0/561951432 >> 172.21.15.26:0/561951432 conn(0x7fb82c0ac590 msgr2=0x7fb82c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 -- 172.21.15.26:0/561951432 shutdown_connections 2024-03-06T07:20:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.519+0000 7fb831f3e640 1 -- 172.21.15.26:0/561951432 wait complete. 2024-03-06T07:20:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.520+0000 7fb831f3e640 1 Processor -- start 2024-03-06T07:20:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.521+0000 7fb831f3e640 1 -- start start 2024-03-06T07:20:25.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb831f3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c0b0070 0x7fb82c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb831f3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb82c0c5380 0x7fb82c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb831f3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c155c30 0x7fb82c14ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb831f3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c06c270 con 0x7fb82c0b0070 2024-03-06T07:20:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb831f3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0c7ad0 con 0x7fb82c0c5380 2024-03-06T07:20:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb831f3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c06c550 con 0x7fb82c155c30 2024-03-06T07:20:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c0b0070 0x7fb82c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c0b0070 0x7fb82c158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47248/0 (socket says 172.21.15.26:47248) 2024-03-06T07:20:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 -- 172.21.15.26:0/3846135206 learned_addr learned my addr 172.21.15.26:0/3846135206 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82affd640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb82c0c5380 0x7fb82c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82bfff640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c155c30 0x7fb82c14ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 -- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c155c30 msgr2=0x7fb82c14ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c155c30 0x7fb82c14ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 -- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb82c0c5380 msgr2=0x7fb82c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb82c0c5380 0x7fb82c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 -- 172.21.15.26:0/3846135206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c156f90 con 0x7fb82c0b0070 2024-03-06T07:20:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.522+0000 7fb82b7fe640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c0b0070 0x7fb82c158d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00c7c0 tx=0x7fb81c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.523+0000 7fb813fff640 1 -- 172.21.15.26:0/3846135206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c018070 con 0x7fb82c0b0070 2024-03-06T07:20:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.523+0000 7fb813fff640 1 -- 172.21.15.26:0/3846135206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00a640 con 0x7fb82c0b0070 2024-03-06T07:20:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.523+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c157270 con 0x7fb82c0b0070 2024-03-06T07:20:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.523+0000 7fb813fff640 1 -- 172.21.15.26:0/3846135206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00aa30 con 0x7fb82c0b0070 2024-03-06T07:20:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.523+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c157830 con 0x7fb82c0b0070 2024-03-06T07:20:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.524+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c04f330 con 0x7fb82c0b0070 2024-03-06T07:20:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.524+0000 7fb813fff640 1 -- 172.21.15.26:0/3846135206 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb81c01b070 con 0x7fb82c0b0070 2024-03-06T07:20:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.525+0000 7fb813fff640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb81806b4d0 0x7fb81806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.525+0000 7fb813fff640 1 -- 172.21.15.26:0/3846135206 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb81c089a70 con 0x7fb82c0b0070 2024-03-06T07:20:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.525+0000 7fb82affd640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb81806b4d0 0x7fb81806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.525+0000 7fb82affd640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb81806b4d0 0x7fb81806d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb820036000 tx=0x7fb82003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.529+0000 7fb813fff640 1 -- 172.21.15.26:0/3846135206 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb82c0b0450 con 0x7fb82c0b0070 2024-03-06T07:20:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.713+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb82c06d860 con 0x7fb82c0b0070 2024-03-06T07:20:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.713+0000 7fb813fff640 1 -- 172.21.15.26:0/3846135206 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb81c089e20 con 0x7fb82c0b0070 2024-03-06T07:20:25.715 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb81806b4d0 msgr2=0x7fb81806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb81806b4d0 0x7fb81806d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb820036000 tx=0x7fb82003d040 comp rx=0 tx=0).stop 2024-03-06T07:20:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c0b0070 msgr2=0x7fb82c158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c0b0070 0x7fb82c158d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00c7c0 tx=0x7fb81c00cc80 comp rx=0 tx=0).stop 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 shutdown_connections 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb81806b4d0 0x7fb81806d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb82c155c30 0x7fb82c14ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.715+0000 7fb831f3e640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb82c0c5380 0x7fb82c14e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.716+0000 7fb831f3e640 1 --2- 172.21.15.26:0/3846135206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb82c0b0070 0x7fb82c158d30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.716+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 >> 172.21.15.26:0/3846135206 conn(0x7fb82c0ac590 msgr2=0x7fb82c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.716+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 shutdown_connections 2024-03-06T07:20:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.716+0000 7fb831f3e640 1 -- 172.21.15.26:0/3846135206 wait complete. 2024-03-06T07:20:25.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:25.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.836+0000 7fe8b096b640 1 Processor -- start 2024-03-06T07:20:25.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.836+0000 7fe8b096b640 1 -- start start 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8b096b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8ac0a8290 0x7fe8ac1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8b096b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0c62a0 con 0x7fe8ac0c4740 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8b096b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0c4c00 con 0x7fe8ac0c69a0 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8b096b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac153f00 con 0x7fe8ac0a8290 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8aa59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8ac0a8290 0x7fe8ac1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8a3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe8ac0c4740 0x7fe8ac0701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44284/0 (socket says 172.21.15.26:44284) 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8a3fff640 1 -- 172.21.15.26:0/3289225136 learned_addr learned my addr 172.21.15.26:0/3289225136 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8aa59c640 1 -- 172.21.15.26:0/3289225136 >> v1:172.21.15.117:6789/0 conn(0x7fe8ac0c69a0 legacy=0x7fe8ac072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8aa59c640 1 -- 172.21.15.26:0/3289225136 >> v1:172.21.15.26:6789/0 conn(0x7fe8ac0c4740 legacy=0x7fe8ac0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8a9d9b640 1 -- 172.21.15.26:0/3289225136 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400211401 0 0) 0x7fe8ac0c4c00 con 0x7fe8ac0c69a0 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.837+0000 7fe8aa59c640 1 -- 172.21.15.26:0/3289225136 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0c4ee0 con 0x7fe8ac0a8290 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.838+0000 7fe8aa59c640 1 --2- 172.21.15.26:0/3289225136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0a8290 0x7fe8ac1539c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0097a0 tx=0x7fe89c02b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452791f0bd1f8e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.838+0000 7fe8a9d9b640 1 -- 172.21.15.26:0/3289225136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c037070 con 0x7fe8ac0a8290 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.838+0000 7fe8a9d9b640 1 -- 172.21.15.26:0/3289225136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c030710 con 0x7fe8ac0a8290 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.838+0000 7fe8a9d9b640 1 -- 172.21.15.26:0/3289225136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c030ae0 con 0x7fe8ac0a8290 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.839+0000 7fe8b096b640 1 -- 172.21.15.26:0/3289225136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0a8290 msgr2=0x7fe8ac1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.839+0000 7fe8b096b640 1 --2- 172.21.15.26:0/3289225136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0a8290 0x7fe8ac1539c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0097a0 tx=0x7fe89c02b680 comp rx=0 tx=0).stop 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.839+0000 7fe8b096b640 1 -- 172.21.15.26:0/3289225136 shutdown_connections 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.839+0000 7fe8b096b640 1 --2- 172.21.15.26:0/3289225136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0a8290 0x7fe8ac1539c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.839+0000 7fe8b096b640 1 -- 172.21.15.26:0/3289225136 >> 172.21.15.26:0/3289225136 conn(0x7fe8ac0c5560 msgr2=0x7fe8ac0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.839+0000 7fe8b096b640 1 -- 172.21.15.26:0/3289225136 shutdown_connections 2024-03-06T07:20:25.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.839+0000 7fe8b096b640 1 -- 172.21.15.26:0/3289225136 wait complete. 2024-03-06T07:20:25.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.841+0000 7fe8b096b640 1 Processor -- start 2024-03-06T07:20:25.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.841+0000 7fe8b096b640 1 -- start start 2024-03-06T07:20:25.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8b096b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8ac0a8290 0x7fe8ac0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8b096b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8b096b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8ac0c69a0 0x7fe8ac1502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8b096b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac072d90 con 0x7fe8ac0c4740 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8b096b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac06fc80 con 0x7fe8ac0a8290 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8b096b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac072f10 con 0x7fe8ac0c69a0 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8aa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8ac0a8290 0x7fe8ac0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac14fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47262/0 (socket says 172.21.15.26:47262) 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8aad9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8ac0c69a0 0x7fe8ac1502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 -- 172.21.15.26:0/2465464437 learned_addr learned my addr 172.21.15.26:0/2465464437 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 -- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8ac0c69a0 msgr2=0x7fe8ac1502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8ac0c69a0 0x7fe8ac1502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 -- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8ac0a8290 msgr2=0x7fe8ac0c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8ac0a8290 0x7fe8ac0c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 -- 172.21.15.26:0/2465464437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac155240 con 0x7fe8ac0c4740 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.842+0000 7fe8a3fff640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac14fd80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c7b0 tx=0x7fe89400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.843+0000 7fe8a2ffd640 1 -- 172.21.15.26:0/2465464437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894018070 con 0x7fe8ac0c4740 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.843+0000 7fe8a2ffd640 1 -- 172.21.15.26:0/2465464437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8940072d0 con 0x7fe8ac0c4740 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.843+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac155520 con 0x7fe8ac0c4740 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.843+0000 7fe8a2ffd640 1 -- 172.21.15.26:0/2465464437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940047e0 con 0x7fe8ac0c4740 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.843+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac155ae0 con 0x7fe8ac0c4740 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.844+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac04f330 con 0x7fe8ac0c4740 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.845+0000 7fe8a2ffd640 1 -- 172.21.15.26:0/2465464437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe89401b070 con 0x7fe8ac0c4740 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.845+0000 7fe8a2ffd640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe88c06b4d0 0x7fe88c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.845+0000 7fe8a2ffd640 1 -- 172.21.15.26:0/2465464437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe894089970 con 0x7fe8ac0c4740 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.845+0000 7fe8aa59c640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe88c06b4d0 0x7fe88c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.846+0000 7fe8aa59c640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe88c06b4d0 0x7fe88c06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe89c034000 tx=0x7fe89c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:25.850+0000 7fe8a2ffd640 1 -- 172.21.15.26:0/2465464437 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe894057dd0 con 0x7fe8ac0c4740 2024-03-06T07:20:26.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.029+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe8ac151fa0 con 0x7fe8ac0c4740 2024-03-06T07:20:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.030+0000 7fe8a2ffd640 1 -- 172.21.15.26:0/2465464437 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe89405be70 con 0x7fe8ac0c4740 2024-03-06T07:20:26.032 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:26.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe88c06b4d0 msgr2=0x7fe88c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe88c06b4d0 0x7fe88c06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe89c034000 tx=0x7fe89c008030 comp rx=0 tx=0).stop 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0c4740 msgr2=0x7fe8ac14fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac14fd80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c7b0 tx=0x7fe89400cc70 comp rx=0 tx=0).stop 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 shutdown_connections 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe88c06b4d0 0x7fe88c06d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8ac0c69a0 0x7fe8ac1502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac14fd80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 --2- 172.21.15.26:0/2465464437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8ac0a8290 0x7fe8ac0c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 >> 172.21.15.26:0/2465464437 conn(0x7fe8ac0c5560 msgr2=0x7fe8ac0a9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 shutdown_connections 2024-03-06T07:20:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.032+0000 7fe8b096b640 1 -- 172.21.15.26:0/2465464437 wait complete. 2024-03-06T07:20:26.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:26.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.144+0000 7f6af1193640 1 Processor -- start 2024-03-06T07:20:26.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.144+0000 7f6af1193640 1 -- start start 2024-03-06T07:20:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.145+0000 7f6af1193640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6aec159270 0x7f6aec155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.145+0000 7f6af1193640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0c73d0 con 0x7f6aec0c88c0 2024-03-06T07:20:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.145+0000 7f6af1193640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0be2e0 con 0x7f6aec14f360 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.145+0000 7f6af1193640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec159700 con 0x7f6aec159270 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.145+0000 7f6aeb59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6aec159270 0x7f6aec155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.145+0000 7f6aea59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6aec0c88c0 0x7f6aec158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44296/0 (socket says 172.21.15.26:44296) 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.145+0000 7f6aea59c640 1 -- 172.21.15.26:0/2018678243 learned_addr learned my addr 172.21.15.26:0/2018678243 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.146+0000 7f6aeb59e640 1 -- 172.21.15.26:0/2018678243 >> v1:172.21.15.117:6789/0 conn(0x7f6aec14f360 legacy=0x7f6aec0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.146+0000 7f6aeb59e640 1 -- 172.21.15.26:0/2018678243 >> v1:172.21.15.26:6789/0 conn(0x7f6aec0c88c0 legacy=0x7f6aec158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.146+0000 7f6ae9d9b640 1 -- 172.21.15.26:0/2018678243 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050094629 0 0) 0x7f6aec0be2e0 con 0x7f6aec14f360 2024-03-06T07:20:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.146+0000 7f6aeb59e640 1 -- 172.21.15.26:0/2018678243 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec0c7670 con 0x7f6aec159270 2024-03-06T07:20:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.146+0000 7f6aeb59e640 1 --2- 172.21.15.26:0/2018678243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec155c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009d90 tx=0x7f6ae402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681cecda6591f9b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.146+0000 7f6ae9d9b640 1 -- 172.21.15.26:0/2018678243 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4039070 con 0x7f6aec159270 2024-03-06T07:20:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6ae9d9b640 1 -- 172.21.15.26:0/2018678243 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae402cc60 con 0x7f6aec159270 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6ae9d9b640 1 -- 172.21.15.26:0/2018678243 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4034660 con 0x7f6aec159270 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6af1193640 1 -- 172.21.15.26:0/2018678243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 msgr2=0x7f6aec155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6af1193640 1 --2- 172.21.15.26:0/2018678243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec155c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009d90 tx=0x7f6ae402beb0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6af1193640 1 -- 172.21.15.26:0/2018678243 shutdown_connections 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6af1193640 1 --2- 172.21.15.26:0/2018678243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec155c10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6af1193640 1 -- 172.21.15.26:0/2018678243 >> 172.21.15.26:0/2018678243 conn(0x7f6aec0ac570 msgr2=0x7f6aec0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6af1193640 1 -- 172.21.15.26:0/2018678243 shutdown_connections 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.147+0000 7f6af1193640 1 -- 172.21.15.26:0/2018678243 wait complete. 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 Processor -- start 2024-03-06T07:20:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 -- start start 2024-03-06T07:20:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6aec0c88c0 0x7f6aec1eb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6aec14f360 0x7f6aec157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec0c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c7c30 con 0x7f6aec159270 2024-03-06T07:20:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c5a50 con 0x7f6aec14f360 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.149+0000 7f6af1193640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c5bd0 con 0x7f6aec0c88c0 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec0c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec0c01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47280/0 (socket says 172.21.15.26:47280) 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 -- 172.21.15.26:0/1452744015 learned_addr learned my addr 172.21.15.26:0/1452744015 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aea59c640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6aec14f360 0x7f6aec157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aead9d640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6aec0c88c0 0x7f6aec1eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 -- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6aec0c88c0 msgr2=0x7f6aec1eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6aec0c88c0 0x7f6aec1eb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 -- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6aec14f360 msgr2=0x7f6aec157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6aec14f360 0x7f6aec157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 -- 172.21.15.26:0/1452744015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec1eeda0 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6aeb59e640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec0c01f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4037040 tx=0x7f6ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6ad37fe640 1 -- 172.21.15.26:0/1452744015 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4039050 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6ad37fe640 1 -- 172.21.15.26:0/1452744015 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae403a040 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6ad37fe640 1 -- 172.21.15.26:0/1452744015 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae403e610 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec1ead80 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.150+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec1eb040 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.151+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec04f330 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.152+0000 7f6ad37fe640 1 -- 172.21.15.26:0/1452744015 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ae4047020 con 0x7f6aec159270 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.152+0000 7f6ad37fe640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ad806b4d0 0x7f6ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.152+0000 7f6ad37fe640 1 -- 172.21.15.26:0/1452744015 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ae40ac760 con 0x7f6aec159270 2024-03-06T07:20:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.152+0000 7f6aead9d640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ad806b4d0 0x7f6ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.153+0000 7f6aead9d640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ad806b4d0 0x7f6ad806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000f60 tx=0x7f6adc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.160+0000 7f6ad37fe640 1 -- 172.21.15.26:0/1452744015 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ae404e020 con 0x7f6aec159270 2024-03-06T07:20:26.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.379+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6aec0c8d50 con 0x7f6aec159270 2024-03-06T07:20:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.379+0000 7f6ad37fe640 1 -- 172.21.15.26:0/1452744015 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7f6ae407ada0 con 0x7f6aec159270 2024-03-06T07:20:26.381 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:26.381 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ad806b4d0 msgr2=0x7f6ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ad806b4d0 0x7f6ad806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000f60 tx=0x7f6adc009040 comp rx=0 tx=0).stop 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 msgr2=0x7f6aec0c01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec0c01f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4037040 tx=0x7f6ae4004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 shutdown_connections 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ad806b4d0 0x7f6ad806d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec159270 0x7f6aec0c01f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6aec14f360 0x7f6aec157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 --2- 172.21.15.26:0/1452744015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6aec0c88c0 0x7f6aec1eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 >> 172.21.15.26:0/1452744015 conn(0x7f6aec0ac570 msgr2=0x7f6aec0aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 shutdown_connections 2024-03-06T07:20:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.381+0000 7f6af1193640 1 -- 172.21.15.26:0/1452744015 wait complete. 2024-03-06T07:20:26.394 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:26.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.493+0000 7f4bed1f7640 1 Processor -- start 2024-03-06T07:20:26.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.493+0000 7f4bed1f7640 1 -- start start 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4bed1f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4be81527b0 0x7f4be8159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4bed1f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c7180 0x7f4be80c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4bed1f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4be8155b20 0x7f4be80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4bed1f7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8159640 con 0x7f4be8155b20 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4bed1f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be80c8df0 con 0x7f4be81527b0 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4bed1f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80c7c60 con 0x7f4be80c7180 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4be8155b20 0x7f4be80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be6d9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4be81527b0 0x7f4be8159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4be8155b20 0x7f4be80c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47292/0 (socket says 172.21.15.26:47292) 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 -- 172.21.15.26:0/3412803979 learned_addr learned my addr 172.21.15.26:0/3412803979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c7180 0x7f4be80c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 -- 172.21.15.26:0/3412803979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c7180 msgr2=0x7f4be80c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 --2- 172.21.15.26:0/3412803979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c7180 0x7f4be80c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 -- 172.21.15.26:0/3412803979 >> v2:172.21.15.117:3300/0 conn(0x7f4be81527b0 msgr2=0x7f4be8159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 --2- 172.21.15.26:0/3412803979 >> v2:172.21.15.117:3300/0 conn(0x7f4be81527b0 0x7f4be8159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be759e640 1 -- 172.21.15.26:0/3412803979 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80c8060 con 0x7f4be8155b20 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.494+0000 7f4be6d9d640 1 --2- 172.21.15.26:0/3412803979 >> v2:172.21.15.117:3300/0 conn(0x7f4be81527b0 0x7f4be8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.495+0000 7f4be759e640 1 --2- 172.21.15.26:0/3412803979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be8155b20 0x7f4be80c5330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4be80cba80 tx=0x7f4bd802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2606b0beea617e84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.495+0000 7f4be5d9b640 1 -- 172.21.15.26:0/3412803979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd802c9e0 con 0x7f4be8155b20 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.495+0000 7f4be5d9b640 1 -- 172.21.15.26:0/3412803979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd802cb80 con 0x7f4be8155b20 2024-03-06T07:20:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.495+0000 7f4be5d9b640 1 -- 172.21.15.26:0/3412803979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80343f0 con 0x7f4be8155b20 2024-03-06T07:20:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 -- 172.21.15.26:0/3412803979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be8155b20 msgr2=0x7f4be80c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/3412803979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be8155b20 0x7f4be80c5330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4be80cba80 tx=0x7f4bd802be40 comp rx=0 tx=0).stop 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 -- 172.21.15.26:0/3412803979 shutdown_connections 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/3412803979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be80c7180 0x7f4be80c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/3412803979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be8155b20 0x7f4be80c5330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/3412803979 >> v2:172.21.15.117:3300/0 conn(0x7f4be81527b0 0x7f4be8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 -- 172.21.15.26:0/3412803979 >> 172.21.15.26:0/3412803979 conn(0x7f4be80ac560 msgr2=0x7f4be80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 -- 172.21.15.26:0/3412803979 shutdown_connections 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.496+0000 7f4bed1f7640 1 -- 172.21.15.26:0/3412803979 wait complete. 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.497+0000 7f4bed1f7640 1 Processor -- start 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.497+0000 7f4bed1f7640 1 -- start start 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4bed1f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c7180 0x7f4be80be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4bed1f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be81527b0 0x7f4be80c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4bed1f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8155b20 0x7f4be8149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4bed1f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8155320 con 0x7f4be80c7180 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4bed1f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80c7980 con 0x7f4be81527b0 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4bed1f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8155600 con 0x7f4be8155b20 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c7180 0x7f4be80be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c7180 0x7f4be80be4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47308/0 (socket says 172.21.15.26:47308) 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 -- 172.21.15.26:0/1828687890 learned_addr learned my addr 172.21.15.26:0/1828687890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be759e640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8155b20 0x7f4be8149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be659c640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be81527b0 0x7f4be80c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 -- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8155b20 msgr2=0x7f4be8149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8155b20 0x7f4be8149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 -- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be81527b0 msgr2=0x7f4be80c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be81527b0 0x7f4be80c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.498+0000 7f4be6d9d640 1 -- 172.21.15.26:0/1828687890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80bea20 con 0x7f4be80c7180 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.499+0000 7f4be6d9d640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c7180 0x7f4be80be4e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4be000ec10 tx=0x7f4be000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.499+0000 7f4bcf7fe640 1 -- 172.21.15.26:0/1828687890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0017070 con 0x7f4be80c7180 2024-03-06T07:20:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.499+0000 7f4bcf7fe640 1 -- 172.21.15.26:0/1828687890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be000cc30 con 0x7f4be80c7180 2024-03-06T07:20:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.499+0000 7f4bcf7fe640 1 -- 172.21.15.26:0/1828687890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0013470 con 0x7f4be80c7180 2024-03-06T07:20:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.499+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be814e750 con 0x7f4be80c7180 2024-03-06T07:20:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.499+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be814ed10 con 0x7f4be80c7180 2024-03-06T07:20:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.500+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be804f330 con 0x7f4be80c7180 2024-03-06T07:20:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.500+0000 7f4bcf7fe640 1 -- 172.21.15.26:0/1828687890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4be001a070 con 0x7f4be80c7180 2024-03-06T07:20:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.501+0000 7f4bcf7fe640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bd406b4d0 0x7f4bd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.501+0000 7f4bcf7fe640 1 -- 172.21.15.26:0/1828687890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4be008a300 con 0x7f4be80c7180 2024-03-06T07:20:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.501+0000 7f4be659c640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bd406b4d0 0x7f4bd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.501+0000 7f4be659c640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bd406b4d0 0x7f4bd406d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4be80aaf70 tx=0x7f4bdc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.505+0000 7f4bcf7fe640 1 -- 172.21.15.26:0/1828687890 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4be0058880 con 0x7f4be80c7180 2024-03-06T07:20:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.685+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4be806ce00 con 0x7f4be80c7180 2024-03-06T07:20:26.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.686+0000 7f4bcf7fe640 1 -- 172.21.15.26:0/1828687890 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f4be005c920 con 0x7f4be80c7180 2024-03-06T07:20:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:26.688 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bd406b4d0 msgr2=0x7f4bd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bd406b4d0 0x7f4bd406d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4be80aaf70 tx=0x7f4bdc009190 comp rx=0 tx=0).stop 2024-03-06T07:20:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c7180 msgr2=0x7f4be80be4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c7180 0x7f4be80be4e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4be000ec10 tx=0x7f4be000c520 comp rx=0 tx=0).stop 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 shutdown_connections 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4bd406b4d0 0x7f4bd406d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4be8155b20 0x7f4be8149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4be81527b0 0x7f4be80c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 --2- 172.21.15.26:0/1828687890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4be80c7180 0x7f4be80be4e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.688+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 >> 172.21.15.26:0/1828687890 conn(0x7f4be80ac560 msgr2=0x7f4be814c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.689+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 shutdown_connections 2024-03-06T07:20:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.689+0000 7f4bed1f7640 1 -- 172.21.15.26:0/1828687890 wait complete. 2024-03-06T07:20:26.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.801+0000 7f54e44fb640 1 Processor -- start 2024-03-06T07:20:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.801+0000 7f54e44fb640 1 -- start start 2024-03-06T07:20:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e44fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc150580 0x7f54dc06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e44fb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54dc0c6000 0x7f54dc06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e44fb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc153a60 con 0x7f54dc0c1320 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e44fb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc153d40 con 0x7f54dc0c6000 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e44fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc06c2c0 con 0x7f54dc150580 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e1a96640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f54dc0c1320 0x7f54dc1566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44312/0 (socket says 172.21.15.26:44312) 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e1a96640 1 -- 172.21.15.26:0/1015299208 learned_addr learned my addr 172.21.15.26:0/1015299208 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e2297640 1 --2- 172.21.15.26:0/1015299208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc150580 0x7f54dc06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e2a98640 1 --2- 172.21.15.26:0/1015299208 >> v2:172.21.15.117:3300/0 conn(0x7f54dc0c6000 0x7f54dc06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e2297640 1 -- 172.21.15.26:0/1015299208 >> v2:172.21.15.117:3300/0 conn(0x7f54dc0c6000 msgr2=0x7f54dc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e2297640 1 --2- 172.21.15.26:0/1015299208 >> v2:172.21.15.117:3300/0 conn(0x7f54dc0c6000 0x7f54dc06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e2297640 1 -- 172.21.15.26:0/1015299208 >> v1:172.21.15.26:6789/0 conn(0x7f54dc0c1320 legacy=0x7f54dc1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e1295640 1 -- 172.21.15.26:0/1015299208 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904793054 0 0) 0x7f54dc153a60 con 0x7f54dc0c1320 2024-03-06T07:20:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e2297640 1 -- 172.21.15.26:0/1015299208 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc06c5a0 con 0x7f54dc150580 2024-03-06T07:20:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.802+0000 7f54e2a98640 1 --2- 172.21.15.26:0/1015299208 >> v2:172.21.15.117:3300/0 conn(0x7f54dc0c6000 0x7f54dc06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.803+0000 7f54e2297640 1 --2- 172.21.15.26:0/1015299208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc150580 0x7f54dc06dd10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54d80097a0 tx=0x7f54d802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572ce361ab564059 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.803+0000 7f54e1295640 1 -- 172.21.15.26:0/1015299208 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d80315a0 con 0x7f54dc150580 2024-03-06T07:20:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.803+0000 7f54e1295640 1 -- 172.21.15.26:0/1015299208 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d8037070 con 0x7f54dc150580 2024-03-06T07:20:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.803+0000 7f54e1295640 1 -- 172.21.15.26:0/1015299208 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8031960 con 0x7f54dc150580 2024-03-06T07:20:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.803+0000 7f54e44fb640 1 -- 172.21.15.26:0/1015299208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc150580 msgr2=0x7f54dc06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.803+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1015299208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc150580 0x7f54dc06dd10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54d80097a0 tx=0x7f54d802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.804+0000 7f54e44fb640 1 -- 172.21.15.26:0/1015299208 shutdown_connections 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.804+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1015299208 >> v2:172.21.15.117:3300/0 conn(0x7f54dc0c6000 0x7f54dc06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.804+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1015299208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc150580 0x7f54dc06dd10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.804+0000 7f54e44fb640 1 -- 172.21.15.26:0/1015299208 >> 172.21.15.26:0/1015299208 conn(0x7f54dc0c6710 msgr2=0x7f54dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.804+0000 7f54e44fb640 1 -- 172.21.15.26:0/1015299208 shutdown_connections 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.804+0000 7f54e44fb640 1 -- 172.21.15.26:0/1015299208 wait complete. 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 Processor -- start 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 -- start start 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc0c1320 0x7f54dc1e9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54dc0c6000 0x7f54dc148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54dc150580 0x7f54dc0a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0c0dc0 con 0x7f54dc0c6000 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0c0c40 con 0x7f54dc150580 2024-03-06T07:20:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e44fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc156dc0 con 0x7f54dc0c1320 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.805+0000 7f54e1a96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54dc0c6000 0x7f54dc148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e2a98640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54dc150580 0x7f54dc0a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54dc0c6000 0x7f54dc148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47322/0 (socket says 172.21.15.26:47322) 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 -- 172.21.15.26:0/1447385543 learned_addr learned my addr 172.21.15.26:0/1447385543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e2297640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc0c1320 0x7f54dc1e9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 -- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc0c1320 msgr2=0x7f54dc1e9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc0c1320 0x7f54dc1e9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 -- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54dc150580 msgr2=0x7f54dc0a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54dc150580 0x7f54dc0a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 -- 172.21.15.26:0/1447385543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc073270 con 0x7f54dc0c6000 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e2297640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc0c1320 0x7f54dc1e9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e2a98640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54dc150580 0x7f54dc0a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e1a96640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54dc0c6000 0x7f54dc148ee0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c500 tx=0x7f54cc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54d2ffd640 1 -- 172.21.15.26:0/1447385543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc018070 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54d2ffd640 1 -- 172.21.15.26:0/1447385543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc007500 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54dc0a64b0 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54d2ffd640 1 -- 172.21.15.26:0/1447385543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc004a20 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.806+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54dc149420 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.807+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54dc04f330 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.808+0000 7f54d2ffd640 1 -- 172.21.15.26:0/1447385543 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54cc01b070 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.808+0000 7f54d2ffd640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f54c006b4d0 0x7f54c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.808+0000 7f54d2ffd640 1 -- 172.21.15.26:0/1447385543 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f54cc0897b0 con 0x7f54dc0c6000 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.808+0000 7f54e2297640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f54c006b4d0 0x7f54c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.809+0000 7f54e2297640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f54c006b4d0 0x7f54c006d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54d8036000 tx=0x7f54d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:26.812+0000 7f54d2ffd640 1 -- 172.21.15.26:0/1447385543 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f54cc057c10 con 0x7f54dc0c6000 2024-03-06T07:20:27.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.032+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54dc0c5de0 con 0x7f54dc0c6000 2024-03-06T07:20:27.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.033+0000 7f54d2ffd640 1 -- 172.21.15.26:0/1447385543 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7f54cc05bcb0 con 0x7f54dc0c6000 2024-03-06T07:20:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:27.035 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:27.035 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.034+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f54c006b4d0 msgr2=0x7f54c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.034+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f54c006b4d0 0x7f54c006d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54d8036000 tx=0x7f54d8008030 comp rx=0 tx=0).stop 2024-03-06T07:20:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.034+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54dc0c6000 msgr2=0x7f54dc148ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.034+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54dc0c6000 0x7f54dc148ee0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c500 tx=0x7f54cc00c900 comp rx=0 tx=0).stop 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 shutdown_connections 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f54c006b4d0 0x7f54c006d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54dc150580 0x7f54dc0a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54dc0c6000 0x7f54dc148ee0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 --2- 172.21.15.26:0/1447385543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54dc0c1320 0x7f54dc1e9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 >> 172.21.15.26:0/1447385543 conn(0x7f54dc0c6710 msgr2=0x7f54dc06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 shutdown_connections 2024-03-06T07:20:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.035+0000 7f54e44fb640 1 -- 172.21.15.26:0/1447385543 wait complete. 2024-03-06T07:20:27.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:20:01.171153+0000', 'modified': '2024-03-06T07:20:17.459497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/3090618513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3090618513}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3090618513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4263, 'qdb_cluster': [4263]} 2024-03-06T07:20:27.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:20:27.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:20:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.555+0000 7fe3bff79640 1 Processor -- start 2024-03-06T07:20:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.555+0000 7fe3bff79640 1 -- start start 2024-03-06T07:20:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3bff79640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b80c8950 0x7fe3b80c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3bff79640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3b81558d0 0x7fe3b8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3bff79640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b8155130 con 0x7fe3b80c69f0 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3bff79640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b8155440 con 0x7fe3b81558d0 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3bff79640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80ad480 con 0x7fe3b80c8950 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3be516640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3b81558d0 0x7fe3b8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3be516640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3b81558d0 0x7fe3b8155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53814/0 (socket says 172.21.15.26:53814) 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3bdd15640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe3b80c69f0 0x7fe3b80acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:41422/0 (socket says 172.21.15.26:41422) 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3be516640 1 -- 172.21.15.26:0/705608027 learned_addr learned my addr 172.21.15.26:0/705608027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.556+0000 7fe3bd514640 1 --2- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b80c8950 0x7fe3b80c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.557+0000 7fe3be516640 1 -- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b80c8950 msgr2=0x7fe3b80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.557+0000 7fe3be516640 1 --2- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b80c8950 0x7fe3b80c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.557+0000 7fe3be516640 1 -- 172.21.15.26:0/705608027 >> v1:172.21.15.117:6789/0 conn(0x7fe3b80c69f0 legacy=0x7fe3b80acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.557+0000 7fe3bcd13640 1 -- 172.21.15.26:0/705608027 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586296606 0 0) 0x7fe3a8001490 con 0x7fe3b80c69f0 2024-03-06T07:20:27.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.557+0000 7fe3be516640 1 -- 172.21.15.26:0/705608027 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80c5d50 con 0x7fe3b81558d0 2024-03-06T07:20:27.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.557+0000 7fe3bd514640 1 --2- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b80c8950 0x7fe3b80c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:27.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.557+0000 7fe3be516640 1 --2- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b81558d0 0x7fe3b8155d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009870 tx=0x7fe3b002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f5416ebb616ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bcd13640 1 -- 172.21.15.26:0/705608027 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0035020 con 0x7fe3b81558d0 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bcd13640 1 -- 172.21.15.26:0/705608027 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b0031640 con 0x7fe3b81558d0 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bcd13640 1 -- 172.21.15.26:0/705608027 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b002d040 con 0x7fe3b81558d0 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 -- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b81558d0 msgr2=0x7fe3b8155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 --2- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b81558d0 0x7fe3b8155d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009870 tx=0x7fe3b002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 -- 172.21.15.26:0/705608027 shutdown_connections 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 --2- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b81558d0 0x7fe3b8155d60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 --2- 172.21.15.26:0/705608027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b80c8950 0x7fe3b80c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 -- 172.21.15.26:0/705608027 >> 172.21.15.26:0/705608027 conn(0x7fe3b80cae70 msgr2=0x7fe3b80cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 -- 172.21.15.26:0/705608027 shutdown_connections 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.558+0000 7fe3bff79640 1 -- 172.21.15.26:0/705608027 wait complete. 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.559+0000 7fe3bff79640 1 Processor -- start 2024-03-06T07:20:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.559+0000 7fe3bff79640 1 -- start start 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bff79640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b80c69f0 0x7fe3b80c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bff79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b80c8950 0x7fe3b806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bff79640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b81558d0 0x7fe3b81ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bff79640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8152a20 con 0x7fe3b80c8950 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bff79640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80c6310 con 0x7fe3b80c69f0 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bff79640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8152ba0 con 0x7fe3b81558d0 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b80c69f0 0x7fe3b80c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bd514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b80c8950 0x7fe3b806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b80c69f0 0x7fe3b80c1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53830/0 (socket says 172.21.15.26:53830) 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 -- 172.21.15.26:0/61597315 learned_addr learned my addr 172.21.15.26:0/61597315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3be516640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b81558d0 0x7fe3b81ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 -- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b81558d0 msgr2=0x7fe3b81ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b81558d0 0x7fe3b81ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 -- 172.21.15.26:0/61597315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b80c8950 msgr2=0x7fe3b806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b80c8950 0x7fe3b806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.560+0000 7fe3bdd15640 1 -- 172.21.15.26:0/61597315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b8157a10 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.561+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b80c69f0 0x7fe3b80c1a90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800c7c0 tx=0x7fe3a800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.561+0000 7fe3b67fc640 1 -- 172.21.15.26:0/61597315 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8018070 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.561+0000 7fe3b67fc640 1 -- 172.21.15.26:0/61597315 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a800a640 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.561+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b8157cf0 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.561+0000 7fe3b67fc640 1 -- 172.21.15.26:0/61597315 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a800aa30 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.561+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b81edfd0 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.562+0000 7fe3b67fc640 1 -- 172.21.15.26:0/61597315 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3a801b070 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.563+0000 7fe3b67fc640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3a406b500 0x7fe3a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.563+0000 7fe3b67fc640 1 -- 172.21.15.26:0/61597315 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3a8089ad0 con 0x7fe3b80c69f0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.563+0000 7fe3bd514640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3a406b500 0x7fe3a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.563+0000 7fe3bd514640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3a406b500 0x7fe3a406d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009560 tx=0x7fe3ac009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.564+0000 7fe3c2c35740 1 Processor -- start 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.564+0000 7fe3c2c35740 1 -- start start 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e260cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260cf4b0 0x555e260cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e260356b0 con 0x555e260d0f50 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e25fdea60 con 0x555e260d1330 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e26031750 con 0x555e260cf4b0 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e260cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e260cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49198/0 (socket says 172.21.15.26:49198) 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 -- 172.21.15.26:0/1123162224 learned_addr learned my addr 172.21.15.26:0/1123162224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3be516640 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260cf4b0 0x555e260cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bd514640 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 -- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260cf4b0 msgr2=0x555e260cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260cf4b0 0x555e260cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 -- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 msgr2=0x555e260cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 -- 172.21.15.26:0/1123162224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e25ffcf30 con 0x555e260d0f50 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e260cea30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8054410 tx=0x7fe3a8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2636f3012a3f7fa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe39f7fe640 1 -- 172.21.15.26:0/1123162224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8060610 con 0x555e260d0f50 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe39f7fe640 1 -- 172.21.15.26:0/1123162224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a80562b0 con 0x555e260d0f50 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.565+0000 7fe39f7fe640 1 -- 172.21.15.26:0/1123162224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8060180 con 0x555e260d0f50 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 -- 172.21.15.26:0/1123162224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 msgr2=0x555e260cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e260cea30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8054410 tx=0x7fe3a8001250 comp rx=0 tx=0).stop 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 -- 172.21.15.26:0/1123162224 shutdown_connections 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260cf4b0 0x555e260cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/1123162224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e260cea30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 -- 172.21.15.26:0/1123162224 >> 172.21.15.26:0/1123162224 conn(0x555e260cd7d0 msgr2=0x555e260f23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 -- 172.21.15.26:0/1123162224 shutdown_connections 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.566+0000 7fe3c2c35740 1 -- 172.21.15.26:0/1123162224 wait complete. 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 -- start start 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 Processor -- start 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260d0f50 0x555e2610d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e2611da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e26109aa0 0x555e26109f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e260f3230 con 0x555e26109aa0 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e25ff1ea0 con 0x555e260d1330 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.567+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e260f2c70 con 0x555e260d0f50 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3c2c35740 10 client.0 _ll_register_callbacks cb 0x555e25ff1870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3be516640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e26109aa0 0x555e26109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e2611da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260d0f50 0x555e2610d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3be516640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e26109aa0 0x555e26109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49210/0 (socket says 172.21.15.26:49210) 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e2611da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53850/0 (socket says 172.21.15.26:53850) 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3be516640 1 -- 172.21.15.26:0/2834551484 learned_addr learned my addr 172.21.15.26:0/2834551484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 -- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260d0f50 msgr2=0x555e2610d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260d0f50 0x555e2610d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 -- 172.21.15.26:0/2834551484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e26109aa0 msgr2=0x555e26109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e26109aa0 0x555e26109f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 -- 172.21.15.26:0/2834551484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e25ff2270 con 0x555e260d1330 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3be516640 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e26109aa0 0x555e26109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3bd514640 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e2611da00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3b806d2b0 tx=0x7fe3ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9545e61beaf73cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe39cff9640 1 -- 172.21.15.26:0/2834551484 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac027030 con 0x555e260d1330 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3a80562b0 con 0x555e260d1330 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe39cff9640 1 -- 172.21.15.26:0/2834551484 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3ac00f040 con 0x555e260d1330 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe39cff9640 1 -- 172.21.15.26:0/2834551484 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac033050 con 0x555e260d1330 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3c2c35740 10 client.4792 fetch_fsmap learned FSMap version 9 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.568+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x555e25d4a2d0 con 0x555e260d1330 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe39cff9640 1 -- 172.21.15.26:0/2834551484 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a80562b0 con 0x555e260d1330 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe39cff9640 1 -- 172.21.15.26:0/2834551484 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 9) v1 ==== 3322+0+0 (secure 0 0 0) 0x7fe3ac036020 con 0x555e260d1330 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3c2c35740 10 client.4792 fetch_fsmap finished waiting for FSMap version 9 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3c2c35740 10 client.4792 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3c2c35740 20 client.4792 populate_metadata read hostname 'smithi026' 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e260f4080 0x555e2612c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3c2c35740 4 client.4792 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:20:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 --> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e26042bf0 con 0x555e260f4080 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e260f4080 0x555e2612c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e260f4080 0x555e2612c690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cedfe1b45e18d9 server_cookie=7c916289e774e1e9 in_seq=0 out_seq=0 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.569+0000 7fe39cff9640 10 client.4792 ms_handle_connect on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe39cff9640 1 -- 172.21.15.26:0/2834551484 <== mds.0 v2:172.21.15.117:6835/3090618513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e26042bf0 con 0x555e260f4080 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe39cff9640 10 client.4792 handle_command_reply: tid=0 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 client.4792 shutdown 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 msgr2=0x555e2611da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e2611da00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3b806d2b0 tx=0x7fe3ac0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 shutdown_connections 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e260f4080 0x555e2612c690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e26109aa0 0x555e26109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e2611da00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/2834551484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e260d0f50 0x555e2610d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe39cff9640 0 client.4792 ms_handle_reset on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 >> 172.21.15.26:0/2834551484 conn(0x555e260cf4b0 msgr2=0x555e260ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 shutdown_connections 2024-03-06T07:20:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 1 -- 172.21.15.26:0/2834551484 wait complete. 2024-03-06T07:20:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.570+0000 7fe3c2c35740 20 client.4792 trim_cache size 0 max 16384 2024-03-06T07:20:27.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.587+0000 7fe3c2c35740 1 Processor -- start 2024-03-06T07:20:27.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.587+0000 7fe3c2c35740 1 -- start start 2024-03-06T07:20:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e2611d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81f7d90 con 0x555e260d0f50 2024-03-06T07:20:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81571e0 con 0x555e260d1330 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81f7790 con 0x555e26109aa0 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e2611d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e2611d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49222/0 (socket says 172.21.15.26:49222) 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 -- 172.21.15.26:0/3940673149 learned_addr learned my addr 172.21.15.26:0/3940673149 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bd514640 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3be516640 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 -- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 msgr2=0x555e260f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 -- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 msgr2=0x555e260f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 -- 172.21.15.26:0/3940673149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e25e146b0 con 0x555e260d0f50 2024-03-06T07:20:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bd514640 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e2611d980 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8054410 tx=0x7fe3a8089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4652f042c520d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe39effd640 1 -- 172.21.15.26:0/3940673149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8054620 con 0x555e260d0f50 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe39effd640 1 -- 172.21.15.26:0/3940673149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a8086070 con 0x555e260d0f50 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.588+0000 7fe39effd640 1 -- 172.21.15.26:0/3940673149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80547c0 con 0x555e260d0f50 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 -- 172.21.15.26:0/3940673149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 msgr2=0x555e2611d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e2611d980 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8054410 tx=0x7fe3a8089040 comp rx=0 tx=0).stop 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 -- 172.21.15.26:0/3940673149 shutdown_connections 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d1330 0x555e260f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/3940673149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d0f50 0x555e2611d980 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 -- 172.21.15.26:0/3940673149 >> 172.21.15.26:0/3940673149 conn(0x555e260cf4b0 msgr2=0x555e260f2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 -- 172.21.15.26:0/3940673149 shutdown_connections 2024-03-06T07:20:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 -- 172.21.15.26:0/3940673149 wait complete. 2024-03-06T07:20:27.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 -- start start 2024-03-06T07:20:27.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.589+0000 7fe3c2c35740 1 Processor -- start 2024-03-06T07:20:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d0f50 0x555e260f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d1330 0x555e2611a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3c2c35740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2610de10 con 0x555e260d1330 2024-03-06T07:20:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e26121930 con 0x555e260d0f50 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3c2c35740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e26121ea0 con 0x555e26109aa0 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3c2c35740 10 client.0 _ll_register_callbacks cb 0x7fe3ac011f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3bd514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d1330 0x555e2611a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3bd514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d1330 0x555e2611a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49234/0 (socket says 172.21.15.26:49234) 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3bdd15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d0f50 0x555e260f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3bd514640 1 -- 172.21.15.26:0/594002315 learned_addr learned my addr 172.21.15.26:0/594002315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.590+0000 7fe3be516640 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3bd514640 1 -- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 msgr2=0x555e260cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3bd514640 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3bd514640 1 -- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d0f50 msgr2=0x555e260f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3bd514640 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d0f50 0x555e260f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3bd514640 1 -- 172.21.15.26:0/594002315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e260c7360 con 0x555e260d1330 2024-03-06T07:20:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3bd514640 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d1330 0x555e2611a2d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3b806d2b0 tx=0x7fe3ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d2f7fbebe30594 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe39cff9640 1 -- 172.21.15.26:0/594002315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac027070 con 0x555e260d1330 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe39cff9640 1 -- 172.21.15.26:0/594002315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3ac02c610 con 0x555e260d1330 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3a8086070 con 0x555e260d1330 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe39cff9640 1 -- 172.21.15.26:0/594002315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac044050 con 0x555e260d1330 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 10 client.4791 fetch_fsmap learned FSMap version 9 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x555e260f70a0 con 0x555e260d1330 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe39cff9640 1 -- 172.21.15.26:0/594002315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a8086070 con 0x555e260d1330 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe39cff9640 1 -- 172.21.15.26:0/594002315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 9) v1 ==== 3322+0+0 (secure 0 0 0) 0x7fe3ac047020 con 0x555e260d1330 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 10 client.4791 fetch_fsmap finished waiting for FSMap version 9 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 10 client.4791 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 20 client.4791 populate_metadata read hostname 'smithi026' 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e2617ce40 0x555e2617d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 4 client.4791 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.591+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 --> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e2603cf40 con 0x555e2617ce40 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e2617ce40 0x555e2617d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe39cff9640 10 client.4791 ms_handle_connect on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3bdd15640 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e2617ce40 0x555e2617d220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd1eb1789fb9d2e server_cookie=c602390ddfd9c1ff in_seq=0 out_seq=0 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe39cff9640 1 -- 172.21.15.26:0/594002315 <== mds.0 v2:172.21.15.117:6835/3090618513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x555e2603cf40 con 0x555e2617ce40 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe39cff9640 10 client.4791 handle_command_reply: tid=0 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 client.4791 shutdown 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d1330 msgr2=0x555e2611a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d1330 0x555e2611a2d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3b806d2b0 tx=0x7fe3ac019000 comp rx=0 tx=0).stop 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 shutdown_connections 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:6835/3090618513,v1:172.21.15.117:6837/3090618513] conn(0x555e2617ce40 0x555e2617d220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555e26109aa0 0x555e260cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e260d1330 0x555e2611a2d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe39cff9640 0 client.4791 ms_handle_reset on v2:172.21.15.117:6835/3090618513 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 --2- 172.21.15.26:0/594002315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e260d0f50 0x555e260f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 >> 172.21.15.26:0/594002315 conn(0x555e260cf4b0 msgr2=0x555e2611d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 shutdown_connections 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 1 -- 172.21.15.26:0/594002315 wait complete. 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.592+0000 7fe3c2c35740 20 client.4791 trim_cache size 0 max 16384 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:20:27.601 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4263, 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 25.793356255999999, 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 9, 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 24, 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 24, 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 32.169154141999996 2024-03-06T07:20:27.602 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:20:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3a406b500 msgr2=0x7fe3a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3a406b500 0x7fe3a406d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009560 tx=0x7fe3ac009190 comp rx=0 tx=0).stop 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b80c69f0 msgr2=0x7fe3b80c1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b80c69f0 0x7fe3b80c1a90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800c7c0 tx=0x7fe3a800cc80 comp rx=0 tx=0).stop 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 shutdown_connections 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3a406b500 0x7fe3a406d9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3b81558d0 0x7fe3b81ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3b80c8950 0x7fe3b806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 --2- 172.21.15.26:0/61597315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3b80c69f0 0x7fe3b80c1a90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 >> 172.21.15.26:0/61597315 conn(0x7fe3b80cae70 msgr2=0x7fe3b8151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 shutdown_connections 2024-03-06T07:20:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.593+0000 7fe3bff79640 1 -- 172.21.15.26:0/61597315 wait complete. 2024-03-06T07:20:27.605 INFO:teuthology.run_tasks:Running task kclient... 2024-03-06T07:20:27.619 INFO:tasks.kclient:Mounting kernel clients... 2024-03-06T07:20:27.619 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-06T07:20:27.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:27.619 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-03-06T07:20:27.634 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-06T07:20:27.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:27.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:27.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:27.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-03-06T07:20:27.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:27.808 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-06T07:20:27.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:27.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-06T07:20:27.893 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-03-06T07:20:27.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:20:27.895 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-06T07:20:27.896 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-06T07:20:27.896 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:27.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.997+0000 7ff74011e640 1 Processor -- start 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.997+0000 7ff74011e640 1 -- start start 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff74011e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff74011e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff738155a40 0x7ff738155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff74011e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff738156410 con 0x7ff7380c1aa0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff74011e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7380c2680 con 0x7ff738155a40 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff74011e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380c28d0 con 0x7ff73814f790 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73d6b9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff7380c1aa0 0x7ff7380c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46080/0 (socket says 172.21.15.26:46080) 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73d6b9640 1 -- 172.21.15.26:0/3316947875 learned_addr learned my addr 172.21.15.26:0/3316947875 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73deba640 1 --2- 172.21.15.26:0/3316947875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73e6bb640 1 --2- 172.21.15.26:0/3316947875 >> v2:172.21.15.117:3300/0 conn(0x7ff738155a40 0x7ff738155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73ceb8640 1 -- 172.21.15.26:0/3316947875 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507981141 0 0) 0x7ff738156410 con 0x7ff7380c1aa0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73ceb8640 1 -- 172.21.15.26:0/3316947875 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff724003520 con 0x7ff7380c1aa0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73deba640 1 -- 172.21.15.26:0/3316947875 >> v2:172.21.15.117:3300/0 conn(0x7ff738155a40 msgr2=0x7ff738155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73deba640 1 --2- 172.21.15.26:0/3316947875 >> v2:172.21.15.117:3300/0 conn(0x7ff738155a40 0x7ff738155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73deba640 1 -- 172.21.15.26:0/3316947875 >> v1:172.21.15.26:6789/0 conn(0x7ff7380c1aa0 legacy=0x7ff7380c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73deba640 1 -- 172.21.15.26:0/3316947875 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380c2bb0 con 0x7ff73814f790 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.998+0000 7ff73e6bb640 1 --2- 172.21.15.26:0/3316947875 >> v2:172.21.15.117:3300/0 conn(0x7ff738155a40 0x7ff738155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.999+0000 7ff73deba640 1 --2- 172.21.15.26:0/3316947875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381529d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7340097a0 tx=0x7ff73402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a4d8b0aa89473e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.999+0000 7ff73ceb8640 1 -- 172.21.15.26:0/3316947875 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734035020 con 0x7ff73814f790 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.999+0000 7ff73ceb8640 1 -- 172.21.15.26:0/3316947875 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff734031830 con 0x7ff73814f790 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:27.999+0000 7ff73ceb8640 1 -- 172.21.15.26:0/3316947875 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734031be0 con 0x7ff73814f790 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 -- 172.21.15.26:0/3316947875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 msgr2=0x7ff7381529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 --2- 172.21.15.26:0/3316947875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381529d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7340097a0 tx=0x7ff73402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 -- 172.21.15.26:0/3316947875 shutdown_connections 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 --2- 172.21.15.26:0/3316947875 >> v2:172.21.15.117:3300/0 conn(0x7ff738155a40 0x7ff738155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 --2- 172.21.15.26:0/3316947875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381529d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 -- 172.21.15.26:0/3316947875 >> 172.21.15.26:0/3316947875 conn(0x7ff7380c6710 msgr2=0x7ff7380c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 -- 172.21.15.26:0/3316947875 shutdown_connections 2024-03-06T07:20:28.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.000+0000 7ff74011e640 1 -- 172.21.15.26:0/3316947875 wait complete. 2024-03-06T07:20:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.001+0000 7ff74011e640 1 Processor -- start 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.001+0000 7ff74011e640 1 -- start start 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7380c1aa0 0x7ff738154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff738155a40 0x7ff7381ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380c0d60 con 0x7ff738155a40 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff738155520 con 0x7ff7380c1aa0 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380c5e30 con 0x7ff73814f790 2024-03-06T07:20:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff738155a40 0x7ff7381ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff738155a40 0x7ff7381ea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49248/0 (socket says 172.21.15.26:49248) 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 -- 172.21.15.26:0/3134093402 learned_addr learned my addr 172.21.15.26:0/3134093402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73deba640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7380c1aa0 0x7ff738154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73d6b9640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 -- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 msgr2=0x7ff7381511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 -- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7380c1aa0 msgr2=0x7ff738154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7380c1aa0 0x7ff738154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 -- 172.21.15.26:0/3134093402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7381dc600 con 0x7ff738155a40 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73deba640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7380c1aa0 0x7ff738154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff73e6bb640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff738155a40 0x7ff7381ea2c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff73000efc0 tx=0x7ff73000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff72e7fc640 1 -- 172.21.15.26:0/3134093402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730018030 con 0x7ff738155a40 2024-03-06T07:20:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff72e7fc640 1 -- 172.21.15.26:0/3134093402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73000cdf0 con 0x7ff738155a40 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff72e7fc640 1 -- 172.21.15.26:0/3134093402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730003bb0 con 0x7ff738155a40 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7381dde40 con 0x7ff738155a40 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.002+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7381e9da0 con 0x7ff738155a40 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.003+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7381e9fa0 con 0x7ff738155a40 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.004+0000 7ff72e7fc640 1 -- 172.21.15.26:0/3134093402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff730005d10 con 0x7ff738155a40 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.004+0000 7ff72e7fc640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71c06b4d0 0x7ff71c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.004+0000 7ff72e7fc640 1 -- 172.21.15.26:0/3134093402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff73008acd0 con 0x7ff738155a40 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.004+0000 7ff73deba640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71c06b4d0 0x7ff71c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.005+0000 7ff73deba640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71c06b4d0 0x7ff71c06d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff734036000 tx=0x7ff734008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.008+0000 7ff72e7fc640 1 -- 172.21.15.26:0/3134093402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7381e9fa0 con 0x7ff738155a40 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.226+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff73804f330 con 0x7ff738155a40 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.227+0000 7ff72e7fc640 1 -- 172.21.15.26:0/3134093402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff730022070 con 0x7ff738155a40 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-06T07:20:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71c06b4d0 msgr2=0x7ff71c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71c06b4d0 0x7ff71c06d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff734036000 tx=0x7ff734008030 comp rx=0 tx=0).stop 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff738155a40 msgr2=0x7ff7381ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff738155a40 0x7ff7381ea2c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff73000efc0 tx=0x7ff73000c490 comp rx=0 tx=0).stop 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 shutdown_connections 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71c06b4d0 0x7ff71c06d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff738155a40 0x7ff7381ea2c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73814f790 0x7ff7381511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 --2- 172.21.15.26:0/3134093402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7380c1aa0 0x7ff738154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 >> 172.21.15.26:0/3134093402 conn(0x7ff7380c6710 msgr2=0x7ff7380a7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 shutdown_connections 2024-03-06T07:20:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:28.229+0000 7ff74011e640 1 -- 172.21.15.26:0/3134093402 wait complete. 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-06T07:20:28.241 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-06T07:20:28.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.242 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-03-06T07:20:28.256 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-06T07:20:28.256 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-06T07:20:28.256 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-03-06T07:20:28.256 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-06T07:20:28.257 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42655sec preferred_lft 42655sec 2024-03-06T07:20:28.258 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-03-06T07:20:28.258 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:28.258 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:28.258 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:28.258 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-03-06T07:20:28.258 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-06T07:20:28.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.258 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:28.258 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:28.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-03-06T07:20:28.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-03-06T07:20:28.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-03-06T07:20:28.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-06T07:20:28.259 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:28.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.364 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-06T07:20:28.390 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-03-06T07:20:28.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.392 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-03-06T07:20:28.447 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-03-06T07:20:28.447 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-06T07:20:28.447 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-06T07:20:28.447 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-06T07:20:28.447 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-06T07:20:28.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.448 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:28.448 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:28.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-06T07:20:28.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-06T07:20:28.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-06T07:20:28.448 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:28.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.543 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-03-06T07:20:28.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.557 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-03-06T07:20:28.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.612 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:28.612 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:28.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-06T07:20:28.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-06T07:20:28.612 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:28.694 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-06T07:20:28.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.694 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:28.695 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:28.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-06T07:20:28.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-06T07:20:28.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-06T07:20:28.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-06T07:20:28.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-06T07:20:28.695 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:28.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.821 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-03-06T07:20:28.821 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-06T07:20:28.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-03-06T07:20:28.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-03-06T07:20:28.821 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-06T07:20:28.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:28.862 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:28.917 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-06T07:20:28.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:28.917 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:28.972 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-06T07:20:28.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:28.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:29.056 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:29.057 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:29.173 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:29.174 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:29.174 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-06T07:20:29.174 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@4c08c062-261c-4d4a-a3b3-e195823d80fd.cephfs=/ 2024-03-06T07:20:29.174 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-06T07:20:29.174 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-06T07:20:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-03-06T07:20:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-06T07:20:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-03-06T07:20:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-06T07:20:29.176 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-06T07:20:29.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:29.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:29.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:29.205 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:29.261 INFO:teuthology.orchestra.run.smithi026.stdout:client4807 2024-03-06T07:20:29.261 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:20:29.262 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:29.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:29.317 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:29.373 INFO:teuthology.orchestra.run.smithi026.stdout:client4807 2024-03-06T07:20:29.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:29.373 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:29.429 INFO:teuthology.orchestra.run.smithi026.stdout:client4807 2024-03-06T07:20:29.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:29.429 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c08c062-261c-4d4a-a3b3-e195823d80fd.client4807/status') 2024-03-06T07:20:29.495 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4807 (3)172.21.15.26:0/2551194580 2024-03-06T07:20:29.496 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-03-06T07:20:29.496 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-03-06T07:20:29.496 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-03-06T07:20:29.496 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 2.0113e-05 s, 3.5 MB/s 2024-03-06T07:20:29.497 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-06T07:20:29.497 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-06T07:20:29.498 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:29.498 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-06T07:20:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.637+0000 7f876c6f4640 1 Processor -- start 2024-03-06T07:20:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.637+0000 7f876c6f4640 1 -- start start 2024-03-06T07:20:29.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876c6f4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87640c6780 0x7f8764158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876c6f4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87641592d0 0x7f87640c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876c6f4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8764072630 con 0x7f876406da40 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876c6f4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876406de20 con 0x7f87641592d0 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876c6f4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8764072910 con 0x7f87640c6780 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876ac91640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87641592d0 0x7f87640c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876a490640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f876406da40 0x7f8764155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46086/0 (socket says 172.21.15.26:46086) 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876a490640 1 -- 172.21.15.26:0/3587971089 learned_addr learned my addr 172.21.15.26:0/3587971089 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f8769c8f640 1 --2- 172.21.15.26:0/3587971089 >> v2:172.21.15.117:3300/0 conn(0x7f87640c6780 0x7f8764158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876ac91640 1 -- 172.21.15.26:0/3587971089 >> v2:172.21.15.117:3300/0 conn(0x7f87640c6780 msgr2=0x7f8764158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876ac91640 1 --2- 172.21.15.26:0/3587971089 >> v2:172.21.15.117:3300/0 conn(0x7f87640c6780 0x7f8764158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876ac91640 1 -- 172.21.15.26:0/3587971089 >> v1:172.21.15.26:6789/0 conn(0x7f876406da40 legacy=0x7f8764155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876ac91640 1 -- 172.21.15.26:0/3587971089 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764072bf0 con 0x7f87641592d0 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.638+0000 7f876948e640 1 -- 172.21.15.26:0/3587971089 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785158694 0 0) 0x7f8764072630 con 0x7f876406da40 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.639+0000 7f876ac91640 1 --2- 172.21.15.26:0/3587971089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f87640c3280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87640c7120 tx=0x7f875c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec9d66e0f139d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.639+0000 7f876948e640 1 -- 172.21.15.26:0/3587971089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0315a0 con 0x7f87641592d0 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.639+0000 7f876948e640 1 -- 172.21.15.26:0/3587971089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c037070 con 0x7f87641592d0 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.639+0000 7f876948e640 1 -- 172.21.15.26:0/3587971089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c031960 con 0x7f87641592d0 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 -- 172.21.15.26:0/3587971089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 msgr2=0x7f87640c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3587971089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f87640c3280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87640c7120 tx=0x7f875c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 -- 172.21.15.26:0/3587971089 shutdown_connections 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3587971089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f87640c3280 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3587971089 >> v2:172.21.15.117:3300/0 conn(0x7f87640c6780 0x7f8764158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 -- 172.21.15.26:0/3587971089 >> 172.21.15.26:0/3587971089 conn(0x7f87640aa630 msgr2=0x7f87640aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 -- 172.21.15.26:0/3587971089 shutdown_connections 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.640+0000 7f876c6f4640 1 -- 172.21.15.26:0/3587971089 wait complete. 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.641+0000 7f876c6f4640 1 Processor -- start 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.641+0000 7f876c6f4640 1 -- start start 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f876406da40 0x7f87641eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87640c6780 0x7f8764157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f8764156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c3940 con 0x7f87641592d0 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c37c0 con 0x7f876406da40 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c3ac0 con 0x7f87640c6780 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f8764156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f8764156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49270/0 (socket says 172.21.15.26:49270) 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 -- 172.21.15.26:0/3046529173 learned_addr learned my addr 172.21.15.26:0/3046529173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876a490640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f876406da40 0x7f87641eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f8769c8f640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87640c6780 0x7f8764157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 -- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87640c6780 msgr2=0x7f8764157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87640c6780 0x7f8764157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 -- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f876406da40 msgr2=0x7f87641eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f876406da40 0x7f87641eb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 -- 172.21.15.26:0/3046529173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87641eb920 con 0x7f87641592d0 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876ac91640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f8764156c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f875c009890 tx=0x7f875c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f875affd640 1 -- 172.21.15.26:0/3046529173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c032630 con 0x7f87641592d0 2024-03-06T07:20:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f875affd640 1 -- 172.21.15.26:0/3046529173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c0327d0 con 0x7f87641592d0 2024-03-06T07:20:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876406f6d0 con 0x7f87641592d0 2024-03-06T07:20:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f875affd640 1 -- 172.21.15.26:0/3046529173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c032ba0 con 0x7f87641592d0 2024-03-06T07:20:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.642+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876406fc90 con 0x7f87641592d0 2024-03-06T07:20:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.643+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876406d3a0 con 0x7f87641592d0 2024-03-06T07:20:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.644+0000 7f875affd640 1 -- 172.21.15.26:0/3046529173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f875c04e430 con 0x7f87641592d0 2024-03-06T07:20:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.644+0000 7f875affd640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f874806b4d0 0x7f874806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.644+0000 7f875affd640 1 -- 172.21.15.26:0/3046529173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f875c02f030 con 0x7f87641592d0 2024-03-06T07:20:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.644+0000 7f876a490640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f874806b4d0 0x7f874806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.645+0000 7f876a490640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f874806b4d0 0x7f874806d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f87600012e0 tx=0x7f8760001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.649+0000 7f875affd640 1 -- 172.21.15.26:0/3046529173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f876406d3a0 con 0x7f87641592d0 2024-03-06T07:20:29.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.871+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f876404f330 con 0x7f87641592d0 2024-03-06T07:20:29.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.871+0000 7f875affd640 1 -- 172.21.15.26:0/3046529173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f875c040070 con 0x7f87641592d0 2024-03-06T07:20:29.873 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-06T07:20:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f874806b4d0 msgr2=0x7f874806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f874806b4d0 0x7f874806d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f87600012e0 tx=0x7f8760001630 comp rx=0 tx=0).stop 2024-03-06T07:20:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 msgr2=0x7f8764156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f8764156c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f875c009890 tx=0x7f875c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 shutdown_connections 2024-03-06T07:20:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f874806b4d0 0x7f874806d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87641592d0 0x7f8764156c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87640c6780 0x7f8764157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 --2- 172.21.15.26:0/3046529173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f876406da40 0x7f87641eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 >> 172.21.15.26:0/3046529173 conn(0x7f87640aa630 msgr2=0x7f87640c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 shutdown_connections 2024-03-06T07:20:29.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:29.873+0000 7f876c6f4640 1 -- 172.21.15.26:0/3046529173 wait complete. 2024-03-06T07:20:29.885 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-06T07:20:29.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-06T07:20:29.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-03-06T07:20:29.885 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-06T07:20:29.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:29.885 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-06T07:20:29.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-06T07:20:29.886 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-06T07:20:29.886 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-06T07:20:29.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:29.886 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-03-06T07:20:29.902 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-06T07:20:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-06T07:20:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-03-06T07:20:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-03-06T07:20:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:29.903 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-03-06T07:20:29.904 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-03-06T07:20:29.904 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:29.904 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-06T07:20:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-06T07:20:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42643sec preferred_lft 42643sec 2024-03-06T07:20:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-03-06T07:20:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:29.906 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-06T07:20:29.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:29.906 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-06T07:20:29.906 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-06T07:20:29.906 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-03-06T07:20:29.906 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-03-06T07:20:29.907 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-03-06T07:20:29.907 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-06T07:20:29.907 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-06T07:20:30.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.061 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-06T07:20:30.088 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-03-06T07:20:30.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.090 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-03-06T07:20:30.163 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-03-06T07:20:30.163 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-06T07:20:30.163 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-06T07:20:30.163 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-06T07:20:30.163 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-06T07:20:30.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.164 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-06T07:20:30.164 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-06T07:20:30.164 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-06T07:20:30.164 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-06T07:20:30.164 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-06T07:20:30.164 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-06T07:20:30.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.402 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-06T07:20:30.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.418 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-03-06T07:20:30.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.475 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-06T07:20:30.475 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-06T07:20:30.475 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-06T07:20:30.476 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-06T07:20:30.476 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-06T07:20:30.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-06T07:20:30.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.561 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-06T07:20:30.562 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-06T07:20:30.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-06T07:20:30.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-06T07:20:30.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-06T07:20:30.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-06T07:20:30.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-06T07:20:30.562 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-06T07:20:30.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.666 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-06T07:20:30.667 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-06T07:20:30.667 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-03-06T07:20:30.667 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-03-06T07:20:30.667 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-06T07:20:30.709 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:30.709 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:30.764 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-06T07:20:30.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:30.765 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:30.820 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-06T07:20:30.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:30.821 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:30.906 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:30.907 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:31.035 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:31.035 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:31.035 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-06T07:20:31.035 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with new device syntax: 1@4c08c062-261c-4d4a-a3b3-e195823d80fd.cephfs=/ 2024-03-06T07:20:31.035 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.26:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-06T07:20:31.036 INFO:teuthology.orchestra.run.smithi117.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-06T07:20:31.036 INFO:teuthology.orchestra.run.smithi117.stdout: You just mounted a file system that supports labels which does not 2024-03-06T07:20:31.036 INFO:teuthology.orchestra.run.smithi117.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-06T07:20:31.036 INFO:teuthology.orchestra.run.smithi117.stdout: applications will generate AVC messages and not be allowed access to 2024-03-06T07:20:31.036 INFO:teuthology.orchestra.run.smithi117.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-06T07:20:31.038 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-06T07:20:31.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:31.038 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:31.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:31.065 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:31.121 INFO:teuthology.orchestra.run.smithi117.stdout:client4634 2024-03-06T07:20:31.122 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:20:31.122 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:31.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:31.177 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:31.232 INFO:teuthology.orchestra.run.smithi117.stdout:client4634 2024-03-06T07:20:31.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:31.232 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:31.288 INFO:teuthology.orchestra.run.smithi117.stdout:client4634 2024-03-06T07:20:31.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:31.289 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c08c062-261c-4d4a-a3b3-e195823d80fd.client4634/status') 2024-03-06T07:20:31.355 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.4634 (3)192.168.0.1:0/4197220190 2024-03-06T07:20:31.355 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-03-06T07:20:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-03-06T07:20:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-03-06T07:20:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:69 bytes copied, 2.0524e-05 s, 3.4 MB/s 2024-03-06T07:20:31.357 INFO:teuthology.run_tasks:Running task exec... 2024-03-06T07:20:31.367 INFO:teuthology.task.exec:Executing custom commands... 2024-03-06T07:20:31.367 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:20:31.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/crash/warn_recent_interval 0' 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.480+0000 7f224c04e640 1 Processor -- start 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.480+0000 7f224c04e640 1 -- start start 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f224c04e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22441526f0 0x7f22440c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f224c04e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f224406d840 0x7f2244155a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f224c04e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f224406c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f224c04e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22440c7390 con 0x7f22441526f0 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f224c04e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22440c7060 con 0x7f224406d840 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f224c04e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2244155fc0 con 0x7f22440c7820 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f2249dea640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22441526f0 0x7f22440c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f2249dea640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22441526f0 0x7f22440c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49284/0 (socket says 172.21.15.26:49284) 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f2249dea640 1 -- 172.21.15.26:0/4182605390 learned_addr learned my addr 172.21.15.26:0/4182605390 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f224a5eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f224406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.481+0000 7f22495e9640 1 --2- 172.21.15.26:0/4182605390 >> v2:172.21.15.117:3300/0 conn(0x7f224406d840 0x7f2244155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.482+0000 7f2249dea640 1 -- 172.21.15.26:0/4182605390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 msgr2=0x7f224406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.482+0000 7f2249dea640 1 --2- 172.21.15.26:0/4182605390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f224406c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.482+0000 7f2249dea640 1 -- 172.21.15.26:0/4182605390 >> v2:172.21.15.117:3300/0 conn(0x7f224406d840 msgr2=0x7f2244155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.482+0000 7f2249dea640 1 --2- 172.21.15.26:0/4182605390 >> v2:172.21.15.117:3300/0 conn(0x7f224406d840 0x7f2244155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.482+0000 7f2249dea640 1 -- 172.21.15.26:0/4182605390 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22440c5380 con 0x7f22441526f0 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.482+0000 7f224a5eb640 1 --2- 172.21.15.26:0/4182605390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f224406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.482+0000 7f2249dea640 1 --2- 172.21.15.26:0/4182605390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22441526f0 0x7f22440c91e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f224406b0a0 tx=0x7f223c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fe3b0b11b67d20 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f2248de8640 1 -- 172.21.15.26:0/4182605390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c02e070 con 0x7f22441526f0 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f2248de8640 1 -- 172.21.15.26:0/4182605390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c037070 con 0x7f22441526f0 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f2248de8640 1 -- 172.21.15.26:0/4182605390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c031a00 con 0x7f22441526f0 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 -- 172.21.15.26:0/4182605390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22441526f0 msgr2=0x7f22440c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 --2- 172.21.15.26:0/4182605390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22441526f0 0x7f22440c91e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f224406b0a0 tx=0x7f223c02b880 comp rx=0 tx=0).stop 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 -- 172.21.15.26:0/4182605390 shutdown_connections 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 --2- 172.21.15.26:0/4182605390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f224406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 --2- 172.21.15.26:0/4182605390 >> v2:172.21.15.117:3300/0 conn(0x7f224406d840 0x7f2244155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 --2- 172.21.15.26:0/4182605390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22441526f0 0x7f22440c91e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 -- 172.21.15.26:0/4182605390 >> 172.21.15.26:0/4182605390 conn(0x7f22440ac630 msgr2=0x7f22440aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 -- 172.21.15.26:0/4182605390 shutdown_connections 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.483+0000 7f224c04e640 1 -- 172.21.15.26:0/4182605390 wait complete. 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.484+0000 7f224c04e640 1 Processor -- start 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.484+0000 7f224c04e640 1 -- start start 2024-03-06T07:20:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f224c04e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224406d840 0x7f22440ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f224c04e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f2244144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f224c04e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22441526f0 0x7f22441445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f224c04e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c5ac0 con 0x7f224406d840 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f224c04e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c5940 con 0x7f22441526f0 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f224c04e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2244152ad0 con 0x7f22440c7820 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f2249dea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224406d840 0x7f22440ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f2249dea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224406d840 0x7f22440ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49296/0 (socket says 172.21.15.26:49296) 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f2249dea640 1 -- 172.21.15.26:0/466534213 learned_addr learned my addr 172.21.15.26:0/466534213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f224a5eb640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22441526f0 0x7f22441445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.485+0000 7f22495e9640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f2244144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f2249dea640 1 -- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 msgr2=0x7f2244144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f2249dea640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f2244144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f2249dea640 1 -- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22441526f0 msgr2=0x7f22441445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f2249dea640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22441526f0 0x7f22441445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f2249dea640 1 -- 172.21.15.26:0/466534213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2244156d90 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f2249dea640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224406d840 0x7f22440ab760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f223c000c00 tx=0x7f223c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f223a7fc640 1 -- 172.21.15.26:0/466534213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c032670 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f223a7fc640 1 -- 172.21.15.26:0/466534213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c032810 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2244156ab0 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f223a7fc640 1 -- 172.21.15.26:0/466534213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c032c00 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.486+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22440abca0 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.487+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224404f330 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.488+0000 7f223a7fc640 1 -- 172.21.15.26:0/466534213 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f223c04c060 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.488+0000 7f223a7fc640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f222806b4d0 0x7f222806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.488+0000 7f223a7fc640 1 -- 172.21.15.26:0/466534213 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f223c0abc80 con 0x7f224406d840 2024-03-06T07:20:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.488+0000 7f22495e9640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f222806b4d0 0x7f222806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.489+0000 7f22495e9640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f222806b4d0 0x7f222806d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f22441e8b60 tx=0x7f2234009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.492+0000 7f223a7fc640 1 -- 172.21.15.26:0/466534213 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f223c040070 con 0x7f224406d840 2024-03-06T07:20:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.673+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/crash/warn_recent_interval}] v 0) v1 -- 0x7f2244151ea0 con 0x7f224406d840 2024-03-06T07:20:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.676+0000 7f223a7fc640 1 -- 172.21.15.26:0/466534213 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/crash/warn_recent_interval}]=0 v10)=0 v10) v1 ==== 128+0+0 (secure 0 0 0) 0x7f223c07a7c0 con 0x7f224406d840 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f222806b4d0 msgr2=0x7f222806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f222806b4d0 0x7f222806d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f22441e8b60 tx=0x7f2234009190 comp rx=0 tx=0).stop 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224406d840 msgr2=0x7f22440ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224406d840 0x7f22440ab760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f223c000c00 tx=0x7f223c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 shutdown_connections 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f222806b4d0 0x7f222806d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22441526f0 0x7f22441445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22440c7820 0x7f2244144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 --2- 172.21.15.26:0/466534213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224406d840 0x7f22440ab760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 >> 172.21.15.26:0/466534213 conn(0x7f22440ac630 msgr2=0x7f22440c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 shutdown_connections 2024-03-06T07:20:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.678+0000 7f224c04e640 1 -- 172.21.15.26:0/466534213 wait complete. 2024-03-06T07:20:31.690 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-06T07:20:31.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.805+0000 7f552c79a640 1 Processor -- start 2024-03-06T07:20:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.805+0000 7f552c79a640 1 -- start start 2024-03-06T07:20:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.805+0000 7f552c79a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.805+0000 7f552c79a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5524155a40 0x7f5524155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.805+0000 7f552c79a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5524156410 con 0x7f55240c1aa0 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.805+0000 7f552c79a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55240c2680 con 0x7f5524155a40 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.805+0000 7f552c79a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c28d0 con 0x7f552414f790 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f5529d35640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f55240c1aa0 0x7f55240c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46092/0 (socket says 172.21.15.26:46092) 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f5529d35640 1 -- 172.21.15.26:0/834415608 learned_addr learned my addr 172.21.15.26:0/834415608 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f552ad37640 1 --2- 172.21.15.26:0/834415608 >> v2:172.21.15.117:3300/0 conn(0x7f5524155a40 0x7f5524155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f552a536640 1 --2- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f5529534640 1 -- 172.21.15.26:0/834415608 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133652422 0 0) 0x7f5524156410 con 0x7f55240c1aa0 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f5529534640 1 -- 172.21.15.26:0/834415608 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5510003520 con 0x7f55240c1aa0 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f552ad37640 1 -- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 msgr2=0x7f55241529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f552ad37640 1 --2- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f552ad37640 1 -- 172.21.15.26:0/834415608 >> v1:172.21.15.26:6789/0 conn(0x7f55240c1aa0 legacy=0x7f55240c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f552ad37640 1 -- 172.21.15.26:0/834415608 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240c2bb0 con 0x7f5524155a40 2024-03-06T07:20:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.806+0000 7f552a536640 1 --2- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.807+0000 7f552ad37640 1 --2- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5524155a40 0x7f5524155ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f551c009d90 tx=0x7f551c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=995ca916bf54e0b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.807+0000 7f5529534640 1 -- 172.21.15.26:0/834415608 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c039070 con 0x7f5524155a40 2024-03-06T07:20:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.807+0000 7f5529534640 1 -- 172.21.15.26:0/834415608 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c02cc50 con 0x7f5524155a40 2024-03-06T07:20:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.807+0000 7f5529534640 1 -- 172.21.15.26:0/834415608 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c034440 con 0x7f5524155a40 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 -- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5524155a40 msgr2=0x7f5524155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 --2- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5524155a40 0x7f5524155ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f551c009d90 tx=0x7f551c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 -- 172.21.15.26:0/834415608 shutdown_connections 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 --2- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5524155a40 0x7f5524155ed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 --2- 172.21.15.26:0/834415608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 -- 172.21.15.26:0/834415608 >> 172.21.15.26:0/834415608 conn(0x7f55240c6710 msgr2=0x7f55240c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 -- 172.21.15.26:0/834415608 shutdown_connections 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.808+0000 7f552c79a640 1 -- 172.21.15.26:0/834415608 wait complete. 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 Processor -- start 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 -- start start 2024-03-06T07:20:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55240c1aa0 0x7f552414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5524155a40 0x7f55240c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c0d60 con 0x7f5524155a40 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524155520 con 0x7f55240c1aa0 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.809+0000 7f552c79a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c5e30 con 0x7f552414f790 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552ad37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5524155a40 0x7f55240c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552a536640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55240c1aa0 0x7f552414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552a536640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55240c1aa0 0x7f552414bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53910/0 (socket says 172.21.15.26:53910) 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552a536640 1 -- 172.21.15.26:0/33814280 learned_addr learned my addr 172.21.15.26:0/33814280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f5529d35640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241ecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552ad37640 1 -- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 msgr2=0x7f55241ecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552ad37640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241ecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552ad37640 1 -- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55240c1aa0 msgr2=0x7f552414bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552ad37640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55240c1aa0 0x7f552414bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552ad37640 1 -- 172.21.15.26:0/33814280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552414c440 con 0x7f5524155a40 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552a536640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55240c1aa0 0x7f552414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552ad37640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5524155a40 0x7f55240c3340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f551c033000 tx=0x7f551c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f551affd640 1 -- 172.21.15.26:0/33814280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c039050 con 0x7f5524155a40 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f551affd640 1 -- 172.21.15.26:0/33814280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c03a040 con 0x7f5524155a40 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552414c720 con 0x7f5524155a40 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f551affd640 1 -- 172.21.15.26:0/33814280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c009450 con 0x7f5524155a40 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.810+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55241ec5e0 con 0x7f5524155a40 2024-03-06T07:20:31.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.811+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55241ec0f0 con 0x7f5524155a40 2024-03-06T07:20:31.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.812+0000 7f551affd640 1 -- 172.21.15.26:0/33814280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f551c047020 con 0x7f5524155a40 2024-03-06T07:20:31.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.812+0000 7f551affd640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f550806b4d0 0x7f550806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.812+0000 7f551affd640 1 -- 172.21.15.26:0/33814280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f551c0ac300 con 0x7f5524155a40 2024-03-06T07:20:31.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.812+0000 7f552a536640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f550806b4d0 0x7f550806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.813+0000 7f552a536640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f550806b4d0 0x7f550806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5520009560 tx=0x7f5520009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.817+0000 7f551affd640 1 -- 172.21.15.26:0/33814280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f551c04e020 con 0x7f5524155a40 2024-03-06T07:20:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.998+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f552404f330 con 0x7f5524155a40 2024-03-06T07:20:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:31.999+0000 7f551affd640 1 -- 172.21.15.26:0/33814280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f551c07a880 con 0x7f5524155a40 2024-03-06T07:20:32.001 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:32.001 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:32.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f550806b4d0 msgr2=0x7f550806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f550806b4d0 0x7f550806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5520009560 tx=0x7f5520009190 comp rx=0 tx=0).stop 2024-03-06T07:20:32.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5524155a40 msgr2=0x7f55240c3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5524155a40 0x7f55240c3340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f551c033000 tx=0x7f551c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:32.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 shutdown_connections 2024-03-06T07:20:32.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f550806b4d0 0x7f550806d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5524155a40 0x7f55240c3340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552414f790 0x7f55241ecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 --2- 172.21.15.26:0/33814280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55240c1aa0 0x7f552414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 >> 172.21.15.26:0/33814280 conn(0x7f55240c6710 msgr2=0x7f552414ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 shutdown_connections 2024-03-06T07:20:32.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.001+0000 7f552c79a640 1 -- 172.21.15.26:0/33814280 wait complete. 2024-03-06T07:20:32.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:32.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.117+0000 7f431cc1c640 1 Processor -- start 2024-03-06T07:20:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.117+0000 7f431cc1c640 1 -- start start 2024-03-06T07:20:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.117+0000 7f431cc1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f431814f250 0x7f4318155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.117+0000 7f431cc1c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4318158aa0 0x7f4318158f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.117+0000 7f431cc1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431814f630 con 0x7f43180c8960 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.117+0000 7f431cc1c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43180c7470 con 0x7f4318158aa0 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.117+0000 7f431cc1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43180c7f60 con 0x7f431814f250 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f4315d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f43180c8960 0x7f43180c53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46104/0 (socket says 172.21.15.26:46104) 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f4315d9b640 1 -- 172.21.15.26:0/3947903085 learned_addr learned my addr 172.21.15.26:0/3947903085 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f431659c640 1 --2- 172.21.15.26:0/3947903085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f431814f250 0x7f4318155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f4316d9d640 1 --2- 172.21.15.26:0/3947903085 >> v2:172.21.15.117:3300/0 conn(0x7f4318158aa0 0x7f4318158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f431559a640 1 -- 172.21.15.26:0/3947903085 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190302898 0 0) 0x7f431814f630 con 0x7f43180c8960 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f431559a640 1 -- 172.21.15.26:0/3947903085 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4304003520 con 0x7f43180c8960 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f431659c640 1 -- 172.21.15.26:0/3947903085 >> v2:172.21.15.117:3300/0 conn(0x7f4318158aa0 msgr2=0x7f4318158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f431659c640 1 --2- 172.21.15.26:0/3947903085 >> v2:172.21.15.117:3300/0 conn(0x7f4318158aa0 0x7f4318158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f431659c640 1 -- 172.21.15.26:0/3947903085 >> v1:172.21.15.26:6789/0 conn(0x7f43180c8960 legacy=0x7f43180c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f431659c640 1 -- 172.21.15.26:0/3947903085 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180c76c0 con 0x7f431814f250 2024-03-06T07:20:32.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.118+0000 7f4316d9d640 1 --2- 172.21.15.26:0/3947903085 >> v2:172.21.15.117:3300/0 conn(0x7f4318158aa0 0x7f4318158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:32.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.119+0000 7f431659c640 1 --2- 172.21.15.26:0/3947903085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f431814f250 0x7f4318155bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d90 tx=0x7f430802bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d09f53bfd6103f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.119+0000 7f431559a640 1 -- 172.21.15.26:0/3947903085 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308039070 con 0x7f431814f250 2024-03-06T07:20:32.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.119+0000 7f431559a640 1 -- 172.21.15.26:0/3947903085 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430802cb30 con 0x7f431814f250 2024-03-06T07:20:32.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.119+0000 7f431559a640 1 -- 172.21.15.26:0/3947903085 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308034440 con 0x7f431814f250 2024-03-06T07:20:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.119+0000 7f431cc1c640 1 -- 172.21.15.26:0/3947903085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f431814f250 msgr2=0x7f4318155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.119+0000 7f431cc1c640 1 --2- 172.21.15.26:0/3947903085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f431814f250 0x7f4318155bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d90 tx=0x7f430802bd80 comp rx=0 tx=0).stop 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.120+0000 7f431cc1c640 1 -- 172.21.15.26:0/3947903085 shutdown_connections 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.120+0000 7f431cc1c640 1 --2- 172.21.15.26:0/3947903085 >> v2:172.21.15.117:3300/0 conn(0x7f4318158aa0 0x7f4318158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.120+0000 7f431cc1c640 1 --2- 172.21.15.26:0/3947903085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f431814f250 0x7f4318155bc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.120+0000 7f431cc1c640 1 -- 172.21.15.26:0/3947903085 >> 172.21.15.26:0/3947903085 conn(0x7f43180ac610 msgr2=0x7f43180aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.120+0000 7f431cc1c640 1 -- 172.21.15.26:0/3947903085 shutdown_connections 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.120+0000 7f431cc1c640 1 -- 172.21.15.26:0/3947903085 wait complete. 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 Processor -- start 2024-03-06T07:20:32.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 -- start start 2024-03-06T07:20:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43180c8960 0x7f43180af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f431814f250 0x7f43181ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4318158aa0 0x7f43181ee590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43180be310 con 0x7f43180c8960 2024-03-06T07:20:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43180c7c80 con 0x7f431814f250 2024-03-06T07:20:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.121+0000 7f431cc1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43180c5af0 con 0x7f4318158aa0 2024-03-06T07:20:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43180c8960 0x7f43180af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43180c8960 0x7f43180af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49312/0 (socket says 172.21.15.26:49312) 2024-03-06T07:20:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 -- 172.21.15.26:0/1007012609 learned_addr learned my addr 172.21.15.26:0/1007012609 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f4315d9b640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f431814f250 0x7f43181ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f4316d9d640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4318158aa0 0x7f43181ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 -- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4318158aa0 msgr2=0x7f43181ee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4318158aa0 0x7f43181ee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 -- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f431814f250 msgr2=0x7f43181ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f431814f250 0x7f43181ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 -- 172.21.15.26:0/1007012609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180c4910 con 0x7f43180c8960 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f4315d9b640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f431814f250 0x7f43181ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431659c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43180c8960 0x7f43180af580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4308009eb0 tx=0x7f4308004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f430effd640 1 -- 172.21.15.26:0/1007012609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308039050 con 0x7f43180c8960 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f430effd640 1 -- 172.21.15.26:0/1007012609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4308045070 con 0x7f43180c8960 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43180ad650 con 0x7f43180c8960 2024-03-06T07:20:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f430effd640 1 -- 172.21.15.26:0/1007012609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308009450 con 0x7f43180c8960 2024-03-06T07:20:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.122+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43180c37a0 con 0x7f43180c8960 2024-03-06T07:20:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.123+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431804f330 con 0x7f43180c8960 2024-03-06T07:20:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.124+0000 7f430effd640 1 -- 172.21.15.26:0/1007012609 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f430803e850 con 0x7f43180c8960 2024-03-06T07:20:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.124+0000 7f430effd640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f42f806b4d0 0x7f42f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.124+0000 7f430effd640 1 -- 172.21.15.26:0/1007012609 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f43080ad3e0 con 0x7f43180c8960 2024-03-06T07:20:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.124+0000 7f4315d9b640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f42f806b4d0 0x7f42f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.125+0000 7f4315d9b640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f42f806b4d0 0x7f42f806d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43180ca060 tx=0x7f4300009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.129+0000 7f430effd640 1 -- 172.21.15.26:0/1007012609 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f430807c0d0 con 0x7f43180c8960 2024-03-06T07:20:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.310+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f431806cdc0 con 0x7f43180c8960 2024-03-06T07:20:32.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.310+0000 7f430effd640 1 -- 172.21.15.26:0/1007012609 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f430807b9d0 con 0x7f43180c8960 2024-03-06T07:20:32.312 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f42f806b4d0 msgr2=0x7f42f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f42f806b4d0 0x7f42f806d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43180ca060 tx=0x7f4300009040 comp rx=0 tx=0).stop 2024-03-06T07:20:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43180c8960 msgr2=0x7f43180af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43180c8960 0x7f43180af580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4308009eb0 tx=0x7f4308004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 shutdown_connections 2024-03-06T07:20:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f42f806b4d0 0x7f42f806d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4318158aa0 0x7f43181ee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f431814f250 0x7f43181ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 --2- 172.21.15.26:0/1007012609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43180c8960 0x7f43180af580 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 >> 172.21.15.26:0/1007012609 conn(0x7f43180ac610 msgr2=0x7f43180abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 shutdown_connections 2024-03-06T07:20:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.312+0000 7f431cc1c640 1 -- 172.21.15.26:0/1007012609 wait complete. 2024-03-06T07:20:32.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:32.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.428+0000 7f0d1d51e640 1 Processor -- start 2024-03-06T07:20:32.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.428+0000 7f0d1d51e640 1 -- start start 2024-03-06T07:20:32.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d1d51e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c67f0 0x7f0d18155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d1d51e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d18152cb0 con 0x7f0d180c4ef0 2024-03-06T07:20:32.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d1d51e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180c38b0 con 0x7f0d181528d0 2024-03-06T07:20:32.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d1d51e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181594f0 con 0x7f0d180c67f0 2024-03-06T07:20:32.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d177fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0d180c4ef0 0x7f0d18155760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46118/0 (socket says 172.21.15.26:46118) 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d177fe640 1 -- 172.21.15.26:0/858263452 learned_addr learned my addr 172.21.15.26:0/858263452 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d167fc640 1 --2- 172.21.15.26:0/858263452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c67f0 0x7f0d18155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d15ffb640 1 -- 172.21.15.26:0/858263452 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283764081 0 0) 0x7f0d18152cb0 con 0x7f0d180c4ef0 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d15ffb640 1 -- 172.21.15.26:0/858263452 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d04003520 con 0x7f0d180c4ef0 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d15ffb640 1 -- 172.21.15.26:0/858263452 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174233095 0 0) 0x7f0d180c38b0 con 0x7f0d181528d0 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d15ffb640 1 -- 172.21.15.26:0/858263452 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d18152cb0 con 0x7f0d181528d0 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d167fc640 1 -- 172.21.15.26:0/858263452 >> v1:172.21.15.117:6789/0 conn(0x7f0d181528d0 legacy=0x7f0d180724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d167fc640 1 -- 172.21.15.26:0/858263452 >> v1:172.21.15.26:6789/0 conn(0x7f0d180c4ef0 legacy=0x7f0d18155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.429+0000 7f0d167fc640 1 -- 172.21.15.26:0/858263452 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180a7f70 con 0x7f0d180c67f0 2024-03-06T07:20:32.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.430+0000 7f0d167fc640 1 --2- 172.21.15.26:0/858263452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c67f0 0x7f0d18155220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d000097d0 tx=0x7f0d0002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3c22deaf3e0654 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.430+0000 7f0d15ffb640 1 -- 172.21.15.26:0/858263452 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00034070 con 0x7f0d180c67f0 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.430+0000 7f0d15ffb640 1 -- 172.21.15.26:0/858263452 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00030880 con 0x7f0d180c67f0 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.430+0000 7f0d15ffb640 1 -- 172.21.15.26:0/858263452 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00030c30 con 0x7f0d180c67f0 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.431+0000 7f0d1d51e640 1 -- 172.21.15.26:0/858263452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c67f0 msgr2=0x7f0d18155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.431+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/858263452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c67f0 0x7f0d18155220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d000097d0 tx=0x7f0d0002b9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.431+0000 7f0d1d51e640 1 -- 172.21.15.26:0/858263452 shutdown_connections 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.431+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/858263452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c67f0 0x7f0d18155220 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.431+0000 7f0d1d51e640 1 -- 172.21.15.26:0/858263452 >> 172.21.15.26:0/858263452 conn(0x7f0d180aa6a0 msgr2=0x7f0d180aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.431+0000 7f0d1d51e640 1 -- 172.21.15.26:0/858263452 shutdown_connections 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.431+0000 7f0d1d51e640 1 -- 172.21.15.26:0/858263452 wait complete. 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.432+0000 7f0d1d51e640 1 Processor -- start 2024-03-06T07:20:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.432+0000 7f0d1d51e640 1 -- start start 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d1d51e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c4ef0 0x7f0d180701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d1d51e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d180c67f0 0x7f0d18145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d1d51e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d181528d0 0x7f0d18145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d1d51e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181591a0 con 0x7f0d181528d0 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d1d51e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180a8250 con 0x7f0d180c67f0 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d1d51e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18072be0 con 0x7f0d180c4ef0 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d181528d0 0x7f0d18145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d181528d0 0x7f0d18145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49320/0 (socket says 172.21.15.26:49320) 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 -- 172.21.15.26:0/4005106171 learned_addr learned my addr 172.21.15.26:0/4005106171 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d16ffd640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c4ef0 0x7f0d180701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d167fc640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d180c67f0 0x7f0d18145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 -- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c4ef0 msgr2=0x7f0d180701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c4ef0 0x7f0d180701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 -- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d180c67f0 msgr2=0x7f0d18145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d180c67f0 0x7f0d18145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 -- 172.21.15.26:0/4005106171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d18151f00 con 0x7f0d181528d0 2024-03-06T07:20:32.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.433+0000 7f0d177fe640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d181528d0 0x7f0d18145580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800cb40 tx=0x7f0d0800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.434+0000 7f0d0f7fe640 1 -- 172.21.15.26:0/4005106171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08017070 con 0x7f0d181528d0 2024-03-06T07:20:32.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.434+0000 7f0d0f7fe640 1 -- 172.21.15.26:0/4005106171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0800acc0 con 0x7f0d181528d0 2024-03-06T07:20:32.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.434+0000 7f0d0f7fe640 1 -- 172.21.15.26:0/4005106171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08012640 con 0x7f0d181528d0 2024-03-06T07:20:32.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.434+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d18154820 con 0x7f0d181528d0 2024-03-06T07:20:32.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.434+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d18150be0 con 0x7f0d181528d0 2024-03-06T07:20:32.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.435+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1804f330 con 0x7f0d181528d0 2024-03-06T07:20:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.435+0000 7f0d0f7fe640 1 -- 172.21.15.26:0/4005106171 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d0801a070 con 0x7f0d181528d0 2024-03-06T07:20:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.435+0000 7f0d0f7fe640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0cf806b500 0x7f0cf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.435+0000 7f0d0f7fe640 1 -- 172.21.15.26:0/4005106171 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0d0808a270 con 0x7f0d181528d0 2024-03-06T07:20:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.436+0000 7f0d16ffd640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0cf806b500 0x7f0cf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.436+0000 7f0d16ffd640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0cf806b500 0x7f0cf806d9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d10007fd0 tx=0x7f0d10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.440+0000 7f0d0f7fe640 1 -- 172.21.15.26:0/4005106171 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0d08058750 con 0x7f0d181528d0 2024-03-06T07:20:32.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.621+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d18073120 con 0x7f0d181528d0 2024-03-06T07:20:32.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.621+0000 7f0d0f7fe640 1 -- 172.21.15.26:0/4005106171 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d0805c7f0 con 0x7f0d181528d0 2024-03-06T07:20:32.623 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:32.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0cf806b500 msgr2=0x7f0cf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0cf806b500 0x7f0cf806d9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d10007fd0 tx=0x7f0d10009040 comp rx=0 tx=0).stop 2024-03-06T07:20:32.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d181528d0 msgr2=0x7f0d18145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d181528d0 0x7f0d18145580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800cb40 tx=0x7f0d0800a060 comp rx=0 tx=0).stop 2024-03-06T07:20:32.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 shutdown_connections 2024-03-06T07:20:32.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0cf806b500 0x7f0cf806d9c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d181528d0 0x7f0d18145580 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d180c67f0 0x7f0d18145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 --2- 172.21.15.26:0/4005106171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d180c4ef0 0x7f0d180701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 >> 172.21.15.26:0/4005106171 conn(0x7f0d180aa6a0 msgr2=0x7f0d180724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 shutdown_connections 2024-03-06T07:20:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.623+0000 7f0d1d51e640 1 -- 172.21.15.26:0/4005106171 wait complete. 2024-03-06T07:20:32.636 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.741+0000 7f478b97a640 1 Processor -- start 2024-03-06T07:20:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.741+0000 7f478b97a640 1 -- start start 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f478b97a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47840b0070 0x7f4784155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f478b97a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47840c5380 0x7f47840c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f478b97a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47840b0450 con 0x7f478406ca70 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f478b97a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478406c020 con 0x7f47840b0070 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f478b97a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478406cf20 con 0x7f47840c5380 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f4789716640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47840b0070 0x7f4784155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f4788f15640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f478406ca70 0x7f47840c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46126/0 (socket says 172.21.15.26:46126) 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f4789716640 1 --2- 172.21.15.26:0/507131383 >> v2:172.21.15.26:3300/0 conn(0x7f47840b0070 0x7f4784155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49326/0 (socket says 172.21.15.26:49326) 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f4788f15640 1 -- 172.21.15.26:0/507131383 learned_addr learned my addr 172.21.15.26:0/507131383 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.742+0000 7f4789f17640 1 --2- 172.21.15.26:0/507131383 >> v2:172.21.15.117:3300/0 conn(0x7f47840c5380 0x7f47840c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4789716640 1 -- 172.21.15.26:0/507131383 >> v2:172.21.15.117:3300/0 conn(0x7f47840c5380 msgr2=0x7f47840c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4789716640 1 --2- 172.21.15.26:0/507131383 >> v2:172.21.15.117:3300/0 conn(0x7f47840c5380 0x7f47840c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4789716640 1 -- 172.21.15.26:0/507131383 >> v1:172.21.15.26:6789/0 conn(0x7f478406ca70 legacy=0x7f47840c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4789716640 1 -- 172.21.15.26:0/507131383 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47840c77f0 con 0x7f47840b0070 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4789f17640 1 --2- 172.21.15.26:0/507131383 >> v2:172.21.15.117:3300/0 conn(0x7f47840c5380 0x7f47840c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4789716640 1 --2- 172.21.15.26:0/507131383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f4784155c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f47840be030 tx=0x7f477c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a9e8977f231b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4783fff640 1 -- 172.21.15.26:0/507131383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c0315a0 con 0x7f47840b0070 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4783fff640 1 -- 172.21.15.26:0/507131383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c037070 con 0x7f47840b0070 2024-03-06T07:20:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.743+0000 7f4783fff640 1 -- 172.21.15.26:0/507131383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c031960 con 0x7f47840b0070 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 -- 172.21.15.26:0/507131383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 msgr2=0x7f4784155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 --2- 172.21.15.26:0/507131383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f4784155c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f47840be030 tx=0x7f477c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 -- 172.21.15.26:0/507131383 shutdown_connections 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 --2- 172.21.15.26:0/507131383 >> v2:172.21.15.117:3300/0 conn(0x7f47840c5380 0x7f47840c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 --2- 172.21.15.26:0/507131383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f4784155c30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 -- 172.21.15.26:0/507131383 >> 172.21.15.26:0/507131383 conn(0x7f47840ac590 msgr2=0x7f47840ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 -- 172.21.15.26:0/507131383 shutdown_connections 2024-03-06T07:20:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.744+0000 7f478b97a640 1 -- 172.21.15.26:0/507131383 wait complete. 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.745+0000 7f478b97a640 1 Processor -- start 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.745+0000 7f478b97a640 1 -- start start 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f478b97a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f478406ca70 0x7f47840bfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f478b97a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f47841549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f478b97a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47840c5380 0x7f47841ead50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f478b97a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478406c550 con 0x7f47840b0070 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f478b97a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478406c270 con 0x7f47840c5380 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f478b97a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478406da80 con 0x7f478406ca70 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f47841549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f47841549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49338/0 (socket says 172.21.15.26:49338) 2024-03-06T07:20:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 -- 172.21.15.26:0/2628645854 learned_addr learned my addr 172.21.15.26:0/2628645854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4789716640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f478406ca70 0x7f47840bfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4789f17640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47840c5380 0x7f47841ead50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 -- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f478406ca70 msgr2=0x7f47840bfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f478406ca70 0x7f47840bfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 -- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47840c5380 msgr2=0x7f47841ead50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47840c5380 0x7f47841ead50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 -- 172.21.15.26:0/2628645854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47840c3a70 con 0x7f47840b0070 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.746+0000 7f4788f15640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f47841549c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f477800c3f0 tx=0x7f477800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.747+0000 7f4781ffb640 1 -- 172.21.15.26:0/2628645854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778017070 con 0x7f47840b0070 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.747+0000 7f4781ffb640 1 -- 172.21.15.26:0/2628645854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4778005ce0 con 0x7f47840b0070 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.747+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47841eeec0 con 0x7f47840b0070 2024-03-06T07:20:32.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.747+0000 7f4781ffb640 1 -- 172.21.15.26:0/2628645854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780072f0 con 0x7f47840b0070 2024-03-06T07:20:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.747+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47841ef1a0 con 0x7f47840b0070 2024-03-06T07:20:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.748+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478404f330 con 0x7f47840b0070 2024-03-06T07:20:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.748+0000 7f4781ffb640 1 -- 172.21.15.26:0/2628645854 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f477801a070 con 0x7f47840b0070 2024-03-06T07:20:32.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.748+0000 7f4781ffb640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f477006b500 0x7f477006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.748+0000 7f4781ffb640 1 -- 172.21.15.26:0/2628645854 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4778089760 con 0x7f47840b0070 2024-03-06T07:20:32.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.749+0000 7f4789716640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f477006b500 0x7f477006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.749+0000 7f4789716640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f477006b500 0x7f477006d9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.753+0000 7f4781ffb640 1 -- 172.21.15.26:0/2628645854 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f47840b0450 con 0x7f47840b0070 2024-03-06T07:20:32.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.933+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f478406d860 con 0x7f47840b0070 2024-03-06T07:20:32.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.933+0000 7f4781ffb640 1 -- 172.21.15.26:0/2628645854 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4778057c40 con 0x7f47840b0070 2024-03-06T07:20:32.935 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f477006b500 msgr2=0x7f477006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f477006b500 0x7f477006d9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c03d040 comp rx=0 tx=0).stop 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 msgr2=0x7f47841549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f47841549c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f477800c3f0 tx=0x7f477800c7f0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 shutdown_connections 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f477006b500 0x7f477006d9c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47840c5380 0x7f47841ead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47840b0070 0x7f47841549c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 --2- 172.21.15.26:0/2628645854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f478406ca70 0x7f47840bfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 >> 172.21.15.26:0/2628645854 conn(0x7f47840ac590 msgr2=0x7f47840c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 shutdown_connections 2024-03-06T07:20:32.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:32.935+0000 7f478b97a640 1 -- 172.21.15.26:0/2628645854 wait complete. 2024-03-06T07:20:32.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:33.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.050+0000 7f490c91e640 1 Processor -- start 2024-03-06T07:20:33.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.050+0000 7f490c91e640 1 -- start start 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f490c91e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490814bcc0 0x7f4908155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f490c91e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4908158a10 0x7f4908158e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f490c91e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49080c75e0 con 0x7f49080be150 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f490c91e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49080c7f10 con 0x7f4908158a10 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f490c91e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49080c78c0 con 0x7f490814bcc0 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f4906d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4908158a10 0x7f4908158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f4906d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4908158a10 0x7f4908158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49352/0 (socket says 172.21.15.26:49352) 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f4906d9d640 1 -- 172.21.15.26:0/2894845664 learned_addr learned my addr 172.21.15.26:0/2894845664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f490659c640 1 --2- 172.21.15.26:0/2894845664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490814bcc0 0x7f4908155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f4906d9d640 1 -- 172.21.15.26:0/2894845664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490814bcc0 msgr2=0x7f4908155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f4906d9d640 1 --2- 172.21.15.26:0/2894845664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490814bcc0 0x7f4908155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f4906d9d640 1 -- 172.21.15.26:0/2894845664 >> v1:172.21.15.117:6789/0 conn(0x7f49080be150 legacy=0x7f49080c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.051+0000 7f4906d9d640 1 -- 172.21.15.26:0/2894845664 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49080c7ba0 con 0x7f4908158a10 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f4906d9d640 1 --2- 172.21.15.26:0/2894845664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f4908158e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4900009e50 tx=0x7f490002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e55085387f0428 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f4905d9b640 1 -- 172.21.15.26:0/2894845664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490002cb90 con 0x7f4908158a10 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f4905d9b640 1 -- 172.21.15.26:0/2894845664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490002cd30 con 0x7f4908158a10 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f4905d9b640 1 -- 172.21.15.26:0/2894845664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900034610 con 0x7f4908158a10 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f490c91e640 1 -- 172.21.15.26:0/2894845664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 msgr2=0x7f4908158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f490c91e640 1 --2- 172.21.15.26:0/2894845664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f4908158e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4900009e50 tx=0x7f490002bff0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f490c91e640 1 -- 172.21.15.26:0/2894845664 shutdown_connections 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f490c91e640 1 --2- 172.21.15.26:0/2894845664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f4908158e20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f490c91e640 1 --2- 172.21.15.26:0/2894845664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490814bcc0 0x7f4908155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f490c91e640 1 -- 172.21.15.26:0/2894845664 >> 172.21.15.26:0/2894845664 conn(0x7f49080ac600 msgr2=0x7f49080aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.052+0000 7f490c91e640 1 -- 172.21.15.26:0/2894845664 shutdown_connections 2024-03-06T07:20:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.053+0000 7f490c91e640 1 -- 172.21.15.26:0/2894845664 wait complete. 2024-03-06T07:20:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 Processor -- start 2024-03-06T07:20:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 -- start start 2024-03-06T07:20:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49080be150 0x7f490814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490814bcc0 0x7f490814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f49081ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49080c5c60 con 0x7f4908158a10 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49080c5ae0 con 0x7f490814bcc0 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f490c91e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4908159360 con 0x7f49080be150 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.054+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f49081ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f49081ebfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49366/0 (socket says 172.21.15.26:49366) 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f490659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49080be150 0x7f490814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 -- 172.21.15.26:0/2925683725 learned_addr learned my addr 172.21.15.26:0/2925683725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f48fffff640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490814bcc0 0x7f490814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 -- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49080be150 msgr2=0x7f490814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49080be150 0x7f490814dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 -- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490814bcc0 msgr2=0x7f490814e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490814bcc0 0x7f490814e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 -- 172.21.15.26:0/2925683725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4908153a60 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f490659c640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49080be150 0x7f490814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f4906d9d640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f49081ebfc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4900036040 tx=0x7f4900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f48feffd640 1 -- 172.21.15.26:0/2925683725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900043020 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f48feffd640 1 -- 172.21.15.26:0/2925683725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490003e450 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4908153d40 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f48feffd640 1 -- 172.21.15.26:0/2925683725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900009510 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.055+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4908154a60 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.056+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4908154800 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.057+0000 7f48feffd640 1 -- 172.21.15.26:0/2925683725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4900046020 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.057+0000 7f48feffd640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e806b4d0 0x7f48e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.057+0000 7f48feffd640 1 -- 172.21.15.26:0/2925683725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49000ac990 con 0x7f4908158a10 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.057+0000 7f490659c640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e806b4d0 0x7f48e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.057+0000 7f490659c640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e806b4d0 0x7f48e806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009f30 tx=0x7f48f8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.061+0000 7f48feffd640 1 -- 172.21.15.26:0/2925683725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f490004d030 con 0x7f4908158a10 2024-03-06T07:20:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.240+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f490804f330 con 0x7f4908158a10 2024-03-06T07:20:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.241+0000 7f48feffd640 1 -- 172.21.15.26:0/2925683725 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f490007af10 con 0x7f4908158a10 2024-03-06T07:20:33.242 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:33.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e806b4d0 msgr2=0x7f48e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e806b4d0 0x7f48e806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009f30 tx=0x7f48f8009190 comp rx=0 tx=0).stop 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 msgr2=0x7f49081ebfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f49081ebfc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4900036040 tx=0x7f4900004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 shutdown_connections 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e806b4d0 0x7f48e806d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4908158a10 0x7f49081ebfc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490814bcc0 0x7f490814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 --2- 172.21.15.26:0/2925683725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49080be150 0x7f490814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 >> 172.21.15.26:0/2925683725 conn(0x7f49080ac600 msgr2=0x7f49080c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 shutdown_connections 2024-03-06T07:20:33.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.243+0000 7f490c91e640 1 -- 172.21.15.26:0/2925683725 wait complete. 2024-03-06T07:20:33.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:33.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.356+0000 7f8ffa665640 1 Processor -- start 2024-03-06T07:20:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.356+0000 7f8ffa665640 1 -- start start 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ffa665640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ff4150580 0x7f8ff406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ffa665640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c1320 0x7f8ff406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ffa665640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff4153a60 con 0x7f8ff40c6000 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ffa665640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff4153d40 con 0x7f8ff4150580 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ffa665640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff406c2c0 con 0x7f8ff40c1320 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c1320 0x7f8ff406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff3fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ff4150580 0x7f8ff406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff3fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ff4150580 0x7f8ff406dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53970/0 (socket says 172.21.15.26:53970) 2024-03-06T07:20:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff3fff640 1 -- 172.21.15.26:0/1328634735 learned_addr learned my addr 172.21.15.26:0/1328634735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff37fe640 1 -- 172.21.15.26:0/1328634735 >> v2:172.21.15.117:3300/0 conn(0x7f8ff4150580 msgr2=0x7f8ff406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/1328634735 >> v2:172.21.15.117:3300/0 conn(0x7f8ff4150580 0x7f8ff406dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff37fe640 1 -- 172.21.15.26:0/1328634735 >> v1:172.21.15.26:6789/0 conn(0x7f8ff40c6000 legacy=0x7f8ff41566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.357+0000 7f8ff37fe640 1 -- 172.21.15.26:0/1328634735 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff406c5a0 con 0x7f8ff40c1320 2024-03-06T07:20:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.358+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/1328634735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c1320 0x7f8ff406e250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40097a0 tx=0x7f8fe402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14868cf34905af98 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.358+0000 7f8ff2ffd640 1 -- 172.21.15.26:0/1328634735 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40315a0 con 0x7f8ff40c1320 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.358+0000 7f8ff2ffd640 1 -- 172.21.15.26:0/1328634735 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4037070 con 0x7f8ff40c1320 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.358+0000 7f8ff2ffd640 1 -- 172.21.15.26:0/1328634735 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4031960 con 0x7f8ff40c1320 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 -- 172.21.15.26:0/1328634735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c1320 msgr2=0x7f8ff406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 --2- 172.21.15.26:0/1328634735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c1320 0x7f8ff406e250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40097a0 tx=0x7f8fe402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 -- 172.21.15.26:0/1328634735 shutdown_connections 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 --2- 172.21.15.26:0/1328634735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c1320 0x7f8ff406e250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 --2- 172.21.15.26:0/1328634735 >> v2:172.21.15.117:3300/0 conn(0x7f8ff4150580 0x7f8ff406dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 -- 172.21.15.26:0/1328634735 >> 172.21.15.26:0/1328634735 conn(0x7f8ff40c6710 msgr2=0x7f8ff40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 -- 172.21.15.26:0/1328634735 shutdown_connections 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.359+0000 7f8ffa665640 1 -- 172.21.15.26:0/1328634735 wait complete. 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.360+0000 7f8ffa665640 1 Processor -- start 2024-03-06T07:20:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.360+0000 7f8ffa665640 1 -- start start 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ffa665640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff40c1320 0x7f8ff4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ffa665640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c6000 0x7f8ff414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ffa665640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff4150580 0x7f8ff4155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ffa665640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c0dc0 con 0x7f8ff40c1320 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ffa665640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c0c40 con 0x7f8ff4150580 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ffa665640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4156dc0 con 0x7f8ff40c6000 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff40c1320 0x7f8ff4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff40c1320 0x7f8ff4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49380/0 (socket says 172.21.15.26:49380) 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 -- 172.21.15.26:0/3142057144 learned_addr learned my addr 172.21.15.26:0/3142057144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c6000 0x7f8ff414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff8c02640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff4150580 0x7f8ff4155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 -- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c6000 msgr2=0x7f8ff414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c6000 0x7f8ff414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 -- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff4150580 msgr2=0x7f8ff4155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff4150580 0x7f8ff4155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 -- 172.21.15.26:0/3142057144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff41562a0 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.361+0000 7f8ff3fff640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff40c1320 0x7f8ff4073050 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e840 tx=0x7f8fe800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.362+0000 7f8ff0ff9640 1 -- 172.21.15.26:0/3142057144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8017070 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.362+0000 7f8ff0ff9640 1 -- 172.21.15.26:0/3142057144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe800c8e0 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.362+0000 7f8ff0ff9640 1 -- 172.21.15.26:0/3142057144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800ccd0 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.362+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff41e99f0 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.362+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff414ac80 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.363+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff404f330 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.363+0000 7f8ff0ff9640 1 -- 172.21.15.26:0/3142057144 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fe801a070 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.363+0000 7f8ff0ff9640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fe006b4d0 0x7f8fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.364+0000 7f8ff0ff9640 1 -- 172.21.15.26:0/3142057144 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8fe808a2f0 con 0x7f8ff40c1320 2024-03-06T07:20:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.364+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fe006b4d0 0x7f8fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.364+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fe006b4d0 0x7f8fe006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4036000 tx=0x7f8fe403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.368+0000 7f8ff0ff9640 1 -- 172.21.15.26:0/3142057144 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8fe8058870 con 0x7f8ff40c1320 2024-03-06T07:20:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.548+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ff4153840 con 0x7f8ff40c1320 2024-03-06T07:20:33.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.550+0000 7f8ff0ff9640 1 -- 172.21.15.26:0/3142057144 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f8fe805c910 con 0x7f8ff40c1320 2024-03-06T07:20:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:33.551 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:33.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fe006b4d0 msgr2=0x7f8fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fe006b4d0 0x7f8fe006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4036000 tx=0x7f8fe403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:33.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff40c1320 msgr2=0x7f8ff4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff40c1320 0x7f8ff4073050 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e840 tx=0x7f8fe800ed00 comp rx=0 tx=0).stop 2024-03-06T07:20:33.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 shutdown_connections 2024-03-06T07:20:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fe006b4d0 0x7f8fe006d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff4150580 0x7f8ff4155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ff40c6000 0x7f8ff414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 --2- 172.21.15.26:0/3142057144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff40c1320 0x7f8ff4073050 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.551+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 >> 172.21.15.26:0/3142057144 conn(0x7f8ff40c6710 msgr2=0x7f8ff40c4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.552+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 shutdown_connections 2024-03-06T07:20:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.552+0000 7f8ffa665640 1 -- 172.21.15.26:0/3142057144 wait complete. 2024-03-06T07:20:33.564 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:33.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.668+0000 7ff3f28fa640 1 Processor -- start 2024-03-06T07:20:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.668+0000 7ff3f28fa640 1 -- start start 2024-03-06T07:20:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.668+0000 7ff3f28fa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff3ec0c4c60 0x7ff3ec0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.668+0000 7ff3f28fa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec070a50 0x7ff3ec156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.668+0000 7ff3f28fa640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec070eb0 con 0x7ff3ec152e40 2024-03-06T07:20:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.668+0000 7ff3f28fa640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec156940 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.668+0000 7ff3f28fa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec157310 con 0x7ff3ec070a50 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3f0e97640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff3ec152e40 0x7ff3ec0724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46138/0 (socket says 172.21.15.26:46138) 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3f0e97640 1 -- 172.21.15.26:0/1585916911 learned_addr learned my addr 172.21.15.26:0/1585916911 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3eb7fe640 1 --2- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec070a50 0x7ff3ec156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3ebfff640 1 --2- 172.21.15.26:0/1585916911 >> v2:172.21.15.117:3300/0 conn(0x7ff3ec0c4c60 0x7ff3ec0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3eaffd640 1 -- 172.21.15.26:0/1585916911 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766506516 0 0) 0x7ff3ec070eb0 con 0x7ff3ec152e40 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3eaffd640 1 -- 172.21.15.26:0/1585916911 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d4003520 con 0x7ff3ec152e40 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3ebfff640 1 -- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec070a50 msgr2=0x7ff3ec156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3ebfff640 1 --2- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec070a50 0x7ff3ec156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3ebfff640 1 -- 172.21.15.26:0/1585916911 >> v1:172.21.15.26:6789/0 conn(0x7ff3ec152e40 legacy=0x7ff3ec0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3ebfff640 1 -- 172.21.15.26:0/1585916911 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0c80b0 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.669+0000 7ff3eb7fe640 1 --2- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec070a50 0x7ff3ec156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3ebfff640 1 --2- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec0c7a20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0097a0 tx=0x7ff3dc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6766acad6e48ea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3eaffd640 1 -- 172.21.15.26:0/1585916911 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0315a0 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3eaffd640 1 -- 172.21.15.26:0/1585916911 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc037070 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3eaffd640 1 -- 172.21.15.26:0/1585916911 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc031940 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3f28fa640 1 -- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec0c4c60 msgr2=0x7ff3ec0c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec0c7a20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0097a0 tx=0x7ff3dc02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3f28fa640 1 -- 172.21.15.26:0/1585916911 shutdown_connections 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec070a50 0x7ff3ec156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/1585916911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec0c7a20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.670+0000 7ff3f28fa640 1 -- 172.21.15.26:0/1585916911 >> 172.21.15.26:0/1585916911 conn(0x7ff3ec0aa6a0 msgr2=0x7ff3ec0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.671+0000 7ff3f28fa640 1 -- 172.21.15.26:0/1585916911 shutdown_connections 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.671+0000 7ff3f28fa640 1 -- 172.21.15.26:0/1585916911 wait complete. 2024-03-06T07:20:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 Processor -- start 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 -- start start 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec070a50 0x7ff3ec1df840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec152e40 0x7ff3ec149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1532d0 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1565d0 con 0x7ff3ec070a50 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.672+0000 7ff3f28fa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec072be0 con 0x7ff3ec152e40 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49388/0 (socket says 172.21.15.26:49388) 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2050103038 learned_addr learned my addr 172.21.15.26:0/2050103038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3ebfff640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec070a50 0x7ff3ec1df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3f0e97640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec152e40 0x7ff3ec149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec152e40 msgr2=0x7ff3ec149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec152e40 0x7ff3ec149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec070a50 msgr2=0x7ff3ec1df840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec070a50 0x7ff3ec1df840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2050103038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec156250 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3eb7fe640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec148c40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000ebe0 tx=0x7ff3e000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/2050103038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0017070 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/2050103038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e000cc30 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec150ee0 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/2050103038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0013470 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.673+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec1514a0 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.674+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec070e30 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.675+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/2050103038 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3e001a070 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.675+0000 7ff3e8ff9640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3d806b4d0 0x7ff3d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.675+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/2050103038 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff3e008a340 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.675+0000 7ff3ebfff640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3d806b4d0 0x7ff3d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.675+0000 7ff3ebfff640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3d806b4d0 0x7ff3d806d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc036000 tx=0x7ff3dc03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.679+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/2050103038 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff3ec070e30 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.862+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3ec04f330 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.863+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/2050103038 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3e00588c0 con 0x7ff3ec0c4c60 2024-03-06T07:20:33.865 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:33.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3d806b4d0 msgr2=0x7ff3d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3d806b4d0 0x7ff3d806d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc036000 tx=0x7ff3dc03d040 comp rx=0 tx=0).stop 2024-03-06T07:20:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ec0c4c60 msgr2=0x7ff3ec148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec148c40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000ebe0 tx=0x7ff3e000c520 comp rx=0 tx=0).stop 2024-03-06T07:20:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 shutdown_connections 2024-03-06T07:20:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3d806b4d0 0x7ff3d806d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3ec152e40 0x7ff3ec149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ec0c4c60 0x7ff3ec148c40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 --2- 172.21.15.26:0/2050103038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ec070a50 0x7ff3ec1df840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 >> 172.21.15.26:0/2050103038 conn(0x7ff3ec0aa6a0 msgr2=0x7ff3ec070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 shutdown_connections 2024-03-06T07:20:33.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.865+0000 7ff3f28fa640 1 -- 172.21.15.26:0/2050103038 wait complete. 2024-03-06T07:20:33.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26cd5ac640 1 Processor -- start 2024-03-06T07:20:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26cd5ac640 1 -- start start 2024-03-06T07:20:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26cd5ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26c8073190 0x7f26c815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26cd5ac640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c8072600 con 0x7f26c80c5880 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26cd5ac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c8153af0 con 0x7f26c80c5c60 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26cd5ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c80723b0 con 0x7f26c8073190 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26c6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f26c80c5880 0x7f26c80c6ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46144/0 (socket says 172.21.15.26:46144) 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.980+0000 7f26c6ffd640 1 -- 172.21.15.26:0/630922649 learned_addr learned my addr 172.21.15.26:0/630922649 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.981+0000 7f26c77fe640 1 --2- 172.21.15.26:0/630922649 >> v2:172.21.15.117:3300/0 conn(0x7f26c8073190 0x7f26c815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.981+0000 7f26c5ffb640 1 -- 172.21.15.26:0/630922649 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929213386 0 0) 0x7f26c8072600 con 0x7f26c80c5880 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.981+0000 7f26c5ffb640 1 -- 172.21.15.26:0/630922649 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b0003520 con 0x7f26c80c5880 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.981+0000 7f26c77fe640 1 -- 172.21.15.26:0/630922649 >> v1:172.21.15.117:6789/0 conn(0x7f26c80c5c60 legacy=0x7f26c80c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.981+0000 7f26c77fe640 1 -- 172.21.15.26:0/630922649 >> v1:172.21.15.26:6789/0 conn(0x7f26c80c5880 legacy=0x7f26c80c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.981+0000 7f26c77fe640 1 -- 172.21.15.26:0/630922649 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8153dd0 con 0x7f26c8073190 2024-03-06T07:20:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26c77fe640 1 --2- 172.21.15.26:0/630922649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c815d8c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009d90 tx=0x7f26b802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2334f268c55a331e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26c5ffb640 1 -- 172.21.15.26:0/630922649 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8039070 con 0x7f26c8073190 2024-03-06T07:20:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26c5ffb640 1 -- 172.21.15.26:0/630922649 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b802cbd0 con 0x7f26c8073190 2024-03-06T07:20:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26c5ffb640 1 -- 172.21.15.26:0/630922649 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8034440 con 0x7f26c8073190 2024-03-06T07:20:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26cd5ac640 1 -- 172.21.15.26:0/630922649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 msgr2=0x7f26c815d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/630922649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c815d8c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009d90 tx=0x7f26b802be20 comp rx=0 tx=0).stop 2024-03-06T07:20:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26cd5ac640 1 -- 172.21.15.26:0/630922649 shutdown_connections 2024-03-06T07:20:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/630922649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c815d8c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26cd5ac640 1 -- 172.21.15.26:0/630922649 >> 172.21.15.26:0/630922649 conn(0x7f26c80aa630 msgr2=0x7f26c80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26cd5ac640 1 -- 172.21.15.26:0/630922649 shutdown_connections 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.982+0000 7f26cd5ac640 1 -- 172.21.15.26:0/630922649 wait complete. 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 Processor -- start 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 -- start start 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c80a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26c80c5880 0x7f26c814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26c80c5c60 0x7f26c814bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c815de00 con 0x7f26c80c5880 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c80728e0 con 0x7f26c8073190 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26cd5ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c815df80 con 0x7f26c80c5c60 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26c80c5880 0x7f26c814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26c80c5880 0x7f26c814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49402/0 (socket says 172.21.15.26:49402) 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.984+0000 7f26c67fc640 1 -- 172.21.15.26:0/1454578647 learned_addr learned my addr 172.21.15.26:0/1454578647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c6ffd640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c80a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c77fe640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26c80c5c60 0x7f26c814bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c67fc640 1 -- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26c80c5c60 msgr2=0x7f26c814bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c67fc640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26c80c5c60 0x7f26c814bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c67fc640 1 -- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 msgr2=0x7f26c80a85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c67fc640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c80a85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c67fc640 1 -- 172.21.15.26:0/1454578647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c81e84a0 con 0x7f26c80c5880 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c6ffd640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c80a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26c67fc640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26c80c5880 0x7f26c814b7b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c3f0 tx=0x7f26bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26af7fe640 1 -- 172.21.15.26:0/1454578647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc0052d0 con 0x7f26c80c5880 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26af7fe640 1 -- 172.21.15.26:0/1454578647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc005490 con 0x7f26c80c5880 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c81e8760 con 0x7f26c80c5880 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26af7fe640 1 -- 172.21.15.26:0/1454578647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc007260 con 0x7f26c80c5880 2024-03-06T07:20:33.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.985+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c81e8d20 con 0x7f26c80c5880 2024-03-06T07:20:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.986+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c804f330 con 0x7f26c80c5880 2024-03-06T07:20:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.987+0000 7f26af7fe640 1 -- 172.21.15.26:0/1454578647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26bc007400 con 0x7f26c80c5880 2024-03-06T07:20:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.987+0000 7f26af7fe640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26b406b500 0x7f26b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.987+0000 7f26af7fe640 1 -- 172.21.15.26:0/1454578647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f26bc0898e0 con 0x7f26c80c5880 2024-03-06T07:20:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.987+0000 7f26c6ffd640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26b406b500 0x7f26b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.988+0000 7f26c6ffd640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26b406b500 0x7f26b406d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26c00011e0 tx=0x7f26c0007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:33.991+0000 7f26af7fe640 1 -- 172.21.15.26:0/1454578647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f26bc057e60 con 0x7f26c80c5880 2024-03-06T07:20:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.170+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26c80aa2d0 con 0x7f26c80c5880 2024-03-06T07:20:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.170+0000 7f26af7fe640 1 -- 172.21.15.26:0/1454578647 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26bc05ce20 con 0x7f26c80c5880 2024-03-06T07:20:34.172 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26b406b500 msgr2=0x7f26b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26b406b500 0x7f26b406d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26c00011e0 tx=0x7f26c0007d30 comp rx=0 tx=0).stop 2024-03-06T07:20:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26c80c5880 msgr2=0x7f26c814b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26c80c5880 0x7f26c814b7b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c3f0 tx=0x7f26bc00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 shutdown_connections 2024-03-06T07:20:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26b406b500 0x7f26b406d9c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26c80c5c60 0x7f26c814bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26c80c5880 0x7f26c814b7b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 --2- 172.21.15.26:0/1454578647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26c8073190 0x7f26c80a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 >> 172.21.15.26:0/1454578647 conn(0x7f26c80aa630 msgr2=0x7f26c8148750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 shutdown_connections 2024-03-06T07:20:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.172+0000 7f26cd5ac640 1 -- 172.21.15.26:0/1454578647 wait complete. 2024-03-06T07:20:34.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:34.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.287+0000 7f87e373e640 1 Processor -- start 2024-03-06T07:20:34.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.287+0000 7f87e373e640 1 -- start start 2024-03-06T07:20:34.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e373e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc14bcc0 0x7f87dc155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e373e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87dc158a10 0x7f87dc158e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e373e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc0c75e0 con 0x7f87dc0be150 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e373e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0c7f10 con 0x7f87dc158a10 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e373e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c78c0 con 0x7f87dc14bcc0 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e0cd9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87dc0be150 0x7f87dc0c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46154/0 (socket says 172.21.15.26:46154) 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e0cd9640 1 -- 172.21.15.26:0/445728597 learned_addr learned my addr 172.21.15.26:0/445728597 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e1cdb640 1 --2- 172.21.15.26:0/445728597 >> v2:172.21.15.117:3300/0 conn(0x7f87dc158a10 0x7f87dc158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87e14da640 1 --2- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc14bcc0 0x7f87dc155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87dbfff640 1 -- 172.21.15.26:0/445728597 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811033928 0 0) 0x7f87dc0c75e0 con 0x7f87dc0be150 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.288+0000 7f87dbfff640 1 -- 172.21.15.26:0/445728597 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c4003520 con 0x7f87dc0be150 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.289+0000 7f87e1cdb640 1 -- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc14bcc0 msgr2=0x7f87dc155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.289+0000 7f87e1cdb640 1 --2- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc14bcc0 0x7f87dc155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.289+0000 7f87e1cdb640 1 -- 172.21.15.26:0/445728597 >> v1:172.21.15.26:6789/0 conn(0x7f87dc0be150 legacy=0x7f87dc0c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.289+0000 7f87e1cdb640 1 -- 172.21.15.26:0/445728597 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0c7ba0 con 0x7f87dc158a10 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.289+0000 7f87e14da640 1 --2- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc14bcc0 0x7f87dc155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.289+0000 7f87e1cdb640 1 --2- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 0x7f87dc158e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009d90 tx=0x7f87cc02bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1367e67dc6fb6ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.289+0000 7f87dbfff640 1 -- 172.21.15.26:0/445728597 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc039070 con 0x7f87dc158a10 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87dbfff640 1 -- 172.21.15.26:0/445728597 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc02cce0 con 0x7f87dc158a10 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87dbfff640 1 -- 172.21.15.26:0/445728597 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc034660 con 0x7f87dc158a10 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 -- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 msgr2=0x7f87dc158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 --2- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 0x7f87dc158e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009d90 tx=0x7f87cc02bf30 comp rx=0 tx=0).stop 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 -- 172.21.15.26:0/445728597 shutdown_connections 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 --2- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 0x7f87dc158e20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 --2- 172.21.15.26:0/445728597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc14bcc0 0x7f87dc155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 -- 172.21.15.26:0/445728597 >> 172.21.15.26:0/445728597 conn(0x7f87dc0ac600 msgr2=0x7f87dc0aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 -- 172.21.15.26:0/445728597 shutdown_connections 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.290+0000 7f87e373e640 1 -- 172.21.15.26:0/445728597 wait complete. 2024-03-06T07:20:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.291+0000 7f87e373e640 1 Processor -- start 2024-03-06T07:20:34.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.291+0000 7f87e373e640 1 -- start start 2024-03-06T07:20:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e373e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc0be150 0x7f87dc14e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e373e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc14bcc0 0x7f87dc0c0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e373e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 0x7f87dc14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e373e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c5c60 con 0x7f87dc14bcc0 2024-03-06T07:20:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e373e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c5ae0 con 0x7f87dc158a10 2024-03-06T07:20:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e373e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc159360 con 0x7f87dc0be150 2024-03-06T07:20:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc14bcc0 0x7f87dc0c0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc14bcc0 0x7f87dc0c0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49410/0 (socket says 172.21.15.26:49410) 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 -- 172.21.15.26:0/1795439316 learned_addr learned my addr 172.21.15.26:0/1795439316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e1cdb640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 0x7f87dc14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e14da640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc0be150 0x7f87dc14e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 -- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc0be150 msgr2=0x7f87dc14e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc0be150 0x7f87dc14e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 -- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 msgr2=0x7f87dc14ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 0x7f87dc14ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.292+0000 7f87e0cd9640 1 -- 172.21.15.26:0/1795439316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc14c620 con 0x7f87dc14bcc0 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.293+0000 7f87e0cd9640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc14bcc0 0x7f87dc0c0970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c460 tx=0x7f87d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.293+0000 7f87d9ffb640 1 -- 172.21.15.26:0/1795439316 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0017070 con 0x7f87dc14bcc0 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.293+0000 7f87d9ffb640 1 -- 172.21.15.26:0/1795439316 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0005ce0 con 0x7f87dc14bcc0 2024-03-06T07:20:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.293+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc14c900 con 0x7f87dc14bcc0 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.293+0000 7f87d9ffb640 1 -- 172.21.15.26:0/1795439316 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0007290 con 0x7f87dc14bcc0 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.293+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc14cec0 con 0x7f87dc14bcc0 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.294+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc04f330 con 0x7f87dc14bcc0 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.294+0000 7f87d9ffb640 1 -- 172.21.15.26:0/1795439316 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87d001a070 con 0x7f87dc14bcc0 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.295+0000 7f87d9ffb640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.295+0000 7f87d9ffb640 1 -- 172.21.15.26:0/1795439316 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f87d0089780 con 0x7f87dc14bcc0 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.295+0000 7f87e14da640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.295+0000 7f87e14da640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009560 tx=0x7f87d4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.299+0000 7f87d9ffb640 1 -- 172.21.15.26:0/1795439316 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f87d0057d00 con 0x7f87dc14bcc0 2024-03-06T07:20:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.476+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87dc0c83e0 con 0x7f87dc14bcc0 2024-03-06T07:20:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.477+0000 7f87d9ffb640 1 -- 172.21.15.26:0/1795439316 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87d005bda0 con 0x7f87dc14bcc0 2024-03-06T07:20:34.478 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.478+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 msgr2=0x7f87c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.478+0000 7f87e373e640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009560 tx=0x7f87d4009190 comp rx=0 tx=0).stop 2024-03-06T07:20:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.478+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc14bcc0 msgr2=0x7f87dc0c0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.478+0000 7f87e373e640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc14bcc0 0x7f87dc0c0970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c460 tx=0x7f87d000c920 comp rx=0 tx=0).stop 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 shutdown_connections 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87dc158a10 0x7f87dc14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc14bcc0 0x7f87dc0c0970 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 --2- 172.21.15.26:0/1795439316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87dc0be150 0x7f87dc14e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 >> 172.21.15.26:0/1795439316 conn(0x7f87dc0ac600 msgr2=0x7f87dc0ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 shutdown_connections 2024-03-06T07:20:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.479+0000 7f87e373e640 1 -- 172.21.15.26:0/1795439316 wait complete. 2024-03-06T07:20:34.491 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:34.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.594+0000 7ff9193bf640 1 Processor -- start 2024-03-06T07:20:34.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.594+0000 7ff9193bf640 1 -- start start 2024-03-06T07:20:34.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff9193bf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff9141529f0 0x7ff9140c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff9193bf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff914152dd0 con 0x7ff9140c57d0 2024-03-06T07:20:34.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff9193bf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff914158b00 con 0x7ff9140c5370 2024-03-06T07:20:34.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff9193bf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff914159710 con 0x7ff9141529f0 2024-03-06T07:20:34.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff9137fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff9140c57d0 0x7ff91415fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46166/0 (socket says 172.21.15.26:46166) 2024-03-06T07:20:34.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff9137fe640 1 -- 172.21.15.26:0/3356498555 learned_addr learned my addr 172.21.15.26:0/3356498555 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:34.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff912ffd640 1 --2- 172.21.15.26:0/3356498555 >> v2:172.21.15.117:3300/0 conn(0x7ff9141529f0 0x7ff9140c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff911ffb640 1 -- 172.21.15.26:0/3356498555 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953898694 0 0) 0x7ff914152dd0 con 0x7ff9140c57d0 2024-03-06T07:20:34.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff911ffb640 1 -- 172.21.15.26:0/3356498555 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff900003520 con 0x7ff9140c57d0 2024-03-06T07:20:34.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff912ffd640 1 -- 172.21.15.26:0/3356498555 >> v1:172.21.15.117:6789/0 conn(0x7ff9140c5370 legacy=0x7ff914155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff912ffd640 1 -- 172.21.15.26:0/3356498555 >> v1:172.21.15.26:6789/0 conn(0x7ff9140c57d0 legacy=0x7ff91415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.595+0000 7ff912ffd640 1 -- 172.21.15.26:0/3356498555 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914158de0 con 0x7ff9141529f0 2024-03-06T07:20:34.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.596+0000 7ff912ffd640 1 --2- 172.21.15.26:0/3356498555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 0x7ff9140c88f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0097a0 tx=0x7ff90c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e844e02ee2a49af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.596+0000 7ff911ffb640 1 -- 172.21.15.26:0/3356498555 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c035020 con 0x7ff9141529f0 2024-03-06T07:20:34.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.596+0000 7ff911ffb640 1 -- 172.21.15.26:0/3356498555 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c031830 con 0x7ff9141529f0 2024-03-06T07:20:34.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.596+0000 7ff911ffb640 1 -- 172.21.15.26:0/3356498555 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c031be0 con 0x7ff9141529f0 2024-03-06T07:20:34.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.597+0000 7ff9193bf640 1 -- 172.21.15.26:0/3356498555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 msgr2=0x7ff9140c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.597+0000 7ff9193bf640 1 --2- 172.21.15.26:0/3356498555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 0x7ff9140c88f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0097a0 tx=0x7ff90c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.597+0000 7ff9193bf640 1 -- 172.21.15.26:0/3356498555 shutdown_connections 2024-03-06T07:20:34.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.597+0000 7ff9193bf640 1 --2- 172.21.15.26:0/3356498555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 0x7ff9140c88f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.597+0000 7ff9193bf640 1 -- 172.21.15.26:0/3356498555 >> 172.21.15.26:0/3356498555 conn(0x7ff9140ac580 msgr2=0x7ff9140ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:34.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.597+0000 7ff9193bf640 1 -- 172.21.15.26:0/3356498555 shutdown_connections 2024-03-06T07:20:34.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.597+0000 7ff9193bf640 1 -- 172.21.15.26:0/3356498555 wait complete. 2024-03-06T07:20:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.598+0000 7ff9193bf640 1 Processor -- start 2024-03-06T07:20:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.598+0000 7ff9193bf640 1 -- start start 2024-03-06T07:20:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9193bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9140c5370 0x7ff9141537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9193bf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9140c57d0 0x7ff914157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9193bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 0x7ff9141ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9193bf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141593a0 con 0x7ff9140c5370 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9193bf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140c68b0 con 0x7ff9141529f0 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9193bf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff914158740 con 0x7ff9140c57d0 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9140c5370 0x7ff9141537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9140c5370 0x7ff9141537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49420/0 (socket says 172.21.15.26:49420) 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 -- 172.21.15.26:0/1211839037 learned_addr learned my addr 172.21.15.26:0/1211839037 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9127fc640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9140c57d0 0x7ff914157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff9137fe640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 0x7ff9141ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 -- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9140c57d0 msgr2=0x7ff914157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9140c57d0 0x7ff914157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 -- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 msgr2=0x7ff9141ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 0x7ff9141ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 -- 172.21.15.26:0/1211839037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140c1390 con 0x7ff9140c5370 2024-03-06T07:20:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.599+0000 7ff912ffd640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9140c5370 0x7ff9141537b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff90c034040 tx=0x7ff90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.600+0000 7ff90b7fe640 1 -- 172.21.15.26:0/1211839037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c035040 con 0x7ff9140c5370 2024-03-06T07:20:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.600+0000 7ff90b7fe640 1 -- 172.21.15.26:0/1211839037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c004250 con 0x7ff9140c5370 2024-03-06T07:20:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.600+0000 7ff90b7fe640 1 -- 172.21.15.26:0/1211839037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c03a640 con 0x7ff9140c5370 2024-03-06T07:20:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.600+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9140be100 con 0x7ff9140c5370 2024-03-06T07:20:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.600+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141ee0f0 con 0x7ff9140c5370 2024-03-06T07:20:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.601+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91404f330 con 0x7ff9140c5370 2024-03-06T07:20:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.601+0000 7ff90b7fe640 1 -- 172.21.15.26:0/1211839037 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff90c038020 con 0x7ff9140c5370 2024-03-06T07:20:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.602+0000 7ff90b7fe640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8f406b500 0x7ff8f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.602+0000 7ff90b7fe640 1 -- 172.21.15.26:0/1211839037 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff90c0ac040 con 0x7ff9140c5370 2024-03-06T07:20:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.602+0000 7ff9127fc640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8f406b500 0x7ff8f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.602+0000 7ff9127fc640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8f406b500 0x7ff8f406d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ca4f0 tx=0x7ff8fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.606+0000 7ff90b7fe640 1 -- 172.21.15.26:0/1211839037 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff90c07a5c0 con 0x7ff9140c5370 2024-03-06T07:20:34.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.786+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff91406cec0 con 0x7ff9140c5370 2024-03-06T07:20:34.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.787+0000 7ff90b7fe640 1 -- 172.21.15.26:0/1211839037 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff90c07e660 con 0x7ff9140c5370 2024-03-06T07:20:34.789 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:34.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8f406b500 msgr2=0x7ff8f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8f406b500 0x7ff8f406d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ca4f0 tx=0x7ff8fc009040 comp rx=0 tx=0).stop 2024-03-06T07:20:34.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9140c5370 msgr2=0x7ff9141537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9140c5370 0x7ff9141537b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff90c034040 tx=0x7ff90c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 shutdown_connections 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8f406b500 0x7ff8f406d9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141529f0 0x7ff9141ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9140c57d0 0x7ff914157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 --2- 172.21.15.26:0/1211839037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9140c5370 0x7ff9141537b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 >> 172.21.15.26:0/1211839037 conn(0x7ff9140ac580 msgr2=0x7ff914150d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 shutdown_connections 2024-03-06T07:20:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.789+0000 7ff9193bf640 1 -- 172.21.15.26:0/1211839037 wait complete. 2024-03-06T07:20:34.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.902+0000 7fd6d3c1c640 1 Processor -- start 2024-03-06T07:20:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.902+0000 7fd6d3c1c640 1 -- start start 2024-03-06T07:20:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d3c1c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6cc158db0 0x7fd6cc0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d3c1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d3c1c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc155110 con 0x7fd6cc0c5330 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d3c1c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc1559e0 con 0x7fd6cc158db0 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d3c1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0c8f40 con 0x7fd6cc0c88b0 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d11b7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6cc158db0 0x7fd6cc0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d19b8640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd6cc0c5330 0x7fd6cc1586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:41450/0 (socket says 172.21.15.26:41450) 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d19b8640 1 -- 172.21.15.26:0/1193490837 learned_addr learned my addr 172.21.15.26:0/1193490837 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d21b9640 1 --2- 172.21.15.26:0/1193490837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d09b6640 1 -- 172.21.15.26:0/1193490837 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980804661 0 0) 0x7fd6c8001110 con 0x7fd6cc0c5330 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d09b6640 1 -- 172.21.15.26:0/1193490837 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b8003520 con 0x7fd6cc0c5330 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d21b9640 1 -- 172.21.15.26:0/1193490837 >> v2:172.21.15.117:3300/0 conn(0x7fd6cc158db0 msgr2=0x7fd6cc0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d21b9640 1 --2- 172.21.15.26:0/1193490837 >> v2:172.21.15.117:3300/0 conn(0x7fd6cc158db0 0x7fd6cc0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d21b9640 1 -- 172.21.15.26:0/1193490837 >> v1:172.21.15.117:6789/0 conn(0x7fd6cc0c5330 legacy=0x7fd6cc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d21b9640 1 -- 172.21.15.26:0/1193490837 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc155390 con 0x7fd6cc0c88b0 2024-03-06T07:20:34.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.903+0000 7fd6d11b7640 1 --2- 172.21.15.26:0/1193490837 >> v2:172.21.15.117:3300/0 conn(0x7fd6cc158db0 0x7fd6cc0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:34.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.904+0000 7fd6d21b9640 1 --2- 172.21.15.26:0/1193490837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc15fa30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009d90 tx=0x7fd6c402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d35020e31617a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.904+0000 7fd6d09b6640 1 -- 172.21.15.26:0/1193490837 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4039070 con 0x7fd6cc0c88b0 2024-03-06T07:20:34.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.904+0000 7fd6d09b6640 1 -- 172.21.15.26:0/1193490837 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c402cbc0 con 0x7fd6cc0c88b0 2024-03-06T07:20:34.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.904+0000 7fd6d09b6640 1 -- 172.21.15.26:0/1193490837 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4034440 con 0x7fd6cc0c88b0 2024-03-06T07:20:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1193490837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 msgr2=0x7fd6cc15fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1193490837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc15fa30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009d90 tx=0x7fd6c402be10 comp rx=0 tx=0).stop 2024-03-06T07:20:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1193490837 shutdown_connections 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1193490837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc15fa30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1193490837 >> v2:172.21.15.117:3300/0 conn(0x7fd6cc158db0 0x7fd6cc0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1193490837 >> 172.21.15.26:0/1193490837 conn(0x7fd6cc0ac560 msgr2=0x7fd6cc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1193490837 shutdown_connections 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.905+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1193490837 wait complete. 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.906+0000 7fd6d3c1c640 1 Processor -- start 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.906+0000 7fd6d3c1c640 1 -- start start 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.906+0000 7fd6d3c1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6cc0c5330 0x7fd6cc14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d3c1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d3c1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc149040 0x7fd6cc14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d3c1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc1529d0 con 0x7fd6cc149040 2024-03-06T07:20:34.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d3c1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc06d090 con 0x7fd6cc0c5330 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d3c1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc152b50 con 0x7fd6cc0c88b0 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc149040 0x7fd6cc14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d19b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6cc0c5330 0x7fd6cc14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc149040 0x7fd6cc14ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49422/0 (socket says 172.21.15.26:49422) 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 -- 172.21.15.26:0/1824782086 learned_addr learned my addr 172.21.15.26:0/1824782086 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d11b7640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 -- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 msgr2=0x7fd6cc150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 -- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6cc0c5330 msgr2=0x7fd6cc14b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6cc0c5330 0x7fd6cc14b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 -- 172.21.15.26:0/1824782086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc148df0 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d19b8640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6cc0c5330 0x7fd6cc14b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d21b9640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc149040 0x7fd6cc14ddc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009d90 tx=0x7fd6c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6c27fc640 1 -- 172.21.15.26:0/1824782086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4039050 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6c27fc640 1 -- 172.21.15.26:0/1824782086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4045070 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc148b10 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6c27fc640 1 -- 172.21.15.26:0/1824782086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4009450 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.907+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc14b7a0 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.908+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc04f330 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.909+0000 7fd6c27fc640 1 -- 172.21.15.26:0/1824782086 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6c403e830 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.909+0000 7fd6c27fc640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.909+0000 7fd6c27fc640 1 -- 172.21.15.26:0/1824782086 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd6c40ad3c0 con 0x7fd6cc149040 2024-03-06T07:20:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.909+0000 7fd6d19b8640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.910+0000 7fd6d19b8640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005880 tx=0x7fd6c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:34.914+0000 7fd6c27fc640 1 -- 172.21.15.26:0/1824782086 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd6c407baa0 con 0x7fd6cc149040 2024-03-06T07:20:35.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.096+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6cc06d900 con 0x7fd6cc149040 2024-03-06T07:20:35.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.097+0000 7fd6c27fc640 1 -- 172.21.15.26:0/1824782086 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fd6c407fb40 con 0x7fd6cc149040 2024-03-06T07:20:35.100 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:35.100 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:35.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.099+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 msgr2=0x7fd6b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.099+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005880 tx=0x7fd6c8009040 comp rx=0 tx=0).stop 2024-03-06T07:20:35.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.099+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc149040 msgr2=0x7fd6cc14ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.099+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc149040 0x7fd6cc14ddc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009d90 tx=0x7fd6c4004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:35.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 shutdown_connections 2024-03-06T07:20:35.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc149040 0x7fd6cc14ddc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6cc0c88b0 0x7fd6cc150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 --2- 172.21.15.26:0/1824782086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6cc0c5330 0x7fd6cc14b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 >> 172.21.15.26:0/1824782086 conn(0x7fd6cc0ac560 msgr2=0x7fd6cc0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 shutdown_connections 2024-03-06T07:20:35.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.100+0000 7fd6d3c1c640 1 -- 172.21.15.26:0/1824782086 wait complete. 2024-03-06T07:20:35.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:35.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.216+0000 7f9909f93640 1 Processor -- start 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.216+0000 7f9909f93640 1 -- start start 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9909f93640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f990406d750 0x7f99040732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9909f93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f9904072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9909f93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f990406db30 0x7f99040c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9909f93640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9904155b30 con 0x7f990406d750 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9909f93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9904156060 con 0x7f990406db30 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9909f93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040a8430 con 0x7f99040c3a70 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f99037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f990406d750 0x7f99040732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f99037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f990406d750 0x7f99040732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49428/0 (socket says 172.21.15.26:49428) 2024-03-06T07:20:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f99037fe640 1 -- 172.21.15.26:0/2812255926 learned_addr learned my addr 172.21.15.26:0/2812255926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9902ffd640 1 --2- 172.21.15.26:0/2812255926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f9904072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.217+0000 7f9903fff640 1 --2- 172.21.15.26:0/2812255926 >> v2:172.21.15.117:3300/0 conn(0x7f990406db30 0x7f99040c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99037fe640 1 -- 172.21.15.26:0/2812255926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 msgr2=0x7f9904072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99037fe640 1 --2- 172.21.15.26:0/2812255926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f9904072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99037fe640 1 -- 172.21.15.26:0/2812255926 >> v2:172.21.15.117:3300/0 conn(0x7f990406db30 msgr2=0x7f99040c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99037fe640 1 --2- 172.21.15.26:0/2812255926 >> v2:172.21.15.117:3300/0 conn(0x7f990406db30 0x7f99040c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99037fe640 1 -- 172.21.15.26:0/2812255926 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904155d80 con 0x7f990406d750 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99037fe640 1 --2- 172.21.15.26:0/2812255926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f99040732a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9904159760 tx=0x7f98f402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad96471593d34bf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99027fc640 1 -- 172.21.15.26:0/2812255926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4038070 con 0x7f990406d750 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.218+0000 7f99027fc640 1 -- 172.21.15.26:0/2812255926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f402f050 con 0x7f990406d750 2024-03-06T07:20:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f99027fc640 1 -- 172.21.15.26:0/2812255926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4031970 con 0x7f990406d750 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 -- 172.21.15.26:0/2812255926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 msgr2=0x7f99040732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 --2- 172.21.15.26:0/2812255926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f99040732a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9904159760 tx=0x7f98f402b7f0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 -- 172.21.15.26:0/2812255926 shutdown_connections 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 --2- 172.21.15.26:0/2812255926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f9904072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 --2- 172.21.15.26:0/2812255926 >> v2:172.21.15.117:3300/0 conn(0x7f990406db30 0x7f99040c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 --2- 172.21.15.26:0/2812255926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f99040732a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 -- 172.21.15.26:0/2812255926 >> 172.21.15.26:0/2812255926 conn(0x7f99040c5560 msgr2=0x7f99040c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 -- 172.21.15.26:0/2812255926 shutdown_connections 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.219+0000 7f9909f93640 1 -- 172.21.15.26:0/2812255926 wait complete. 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.220+0000 7f9909f93640 1 Processor -- start 2024-03-06T07:20:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.220+0000 7f9909f93640 1 -- start start 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9909f93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f990414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9909f93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990406db30 0x7f9904156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9909f93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f990406fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9909f93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040a7cd0 con 0x7f990406d750 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9909f93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9904072e10 con 0x7f990406db30 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9909f93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040a7e50 con 0x7f99040c3a70 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f990414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f990414dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49436/0 (socket says 172.21.15.26:49436) 2024-03-06T07:20:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 -- 172.21.15.26:0/3181812740 learned_addr learned my addr 172.21.15.26:0/3181812740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9902ffd640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990406db30 0x7f9904156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f9903fff640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f990406fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 -- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 msgr2=0x7f990406fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f990406fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 -- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990406db30 msgr2=0x7f9904156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990406db30 0x7f9904156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 -- 172.21.15.26:0/3181812740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904159490 con 0x7f990406d750 2024-03-06T07:20:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.221+0000 7f99037fe640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f990414dc80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98f402b830 tx=0x7f98f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.222+0000 7f98ebfff640 1 -- 172.21.15.26:0/3181812740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4043070 con 0x7f990406d750 2024-03-06T07:20:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.222+0000 7f98ebfff640 1 -- 172.21.15.26:0/3181812740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f403f680 con 0x7f990406d750 2024-03-06T07:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.222+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9904158ed0 con 0x7f990406d750 2024-03-06T07:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.222+0000 7f98ebfff640 1 -- 172.21.15.26:0/3181812740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4038040 con 0x7f990406d750 2024-03-06T07:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.222+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9904152fc0 con 0x7f990406d750 2024-03-06T07:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.223+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990404f330 con 0x7f990406d750 2024-03-06T07:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.223+0000 7f98ebfff640 1 -- 172.21.15.26:0/3181812740 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98f4046070 con 0x7f990406d750 2024-03-06T07:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.223+0000 7f98ebfff640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98f006b4d0 0x7f98f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.224+0000 7f98ebfff640 1 -- 172.21.15.26:0/3181812740 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f98f40ac050 con 0x7f990406d750 2024-03-06T07:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.224+0000 7f9902ffd640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98f006b4d0 0x7f98f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.224+0000 7f9902ffd640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98f006b4d0 0x7f98f006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99040c7a10 tx=0x7f98f8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.228+0000 7f98ebfff640 1 -- 172.21.15.26:0/3181812740 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f98f407a5d0 con 0x7f990406d750 2024-03-06T07:20:35.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.408+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99040aa430 con 0x7f990406d750 2024-03-06T07:20:35.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.409+0000 7f98ebfff640 1 -- 172.21.15.26:0/3181812740 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98f407e670 con 0x7f990406d750 2024-03-06T07:20:35.411 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:35.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.410+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98f006b4d0 msgr2=0x7f98f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.410+0000 7f9909f93640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98f006b4d0 0x7f98f006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99040c7a10 tx=0x7f98f8009190 comp rx=0 tx=0).stop 2024-03-06T07:20:35.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.410+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 msgr2=0x7f990414dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.410+0000 7f9909f93640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f990414dc80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98f402b830 tx=0x7f98f40046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 shutdown_connections 2024-03-06T07:20:35.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98f006b4d0 0x7f98f006d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99040c3a70 0x7f990406fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990406db30 0x7f9904156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 --2- 172.21.15.26:0/3181812740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990406d750 0x7f990414dc80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 >> 172.21.15.26:0/3181812740 conn(0x7f99040c5560 msgr2=0x7f99040a8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 shutdown_connections 2024-03-06T07:20:35.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.411+0000 7f9909f93640 1 -- 172.21.15.26:0/3181812740 wait complete. 2024-03-06T07:20:35.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.526+0000 7fd31fc40640 1 Processor -- start 2024-03-06T07:20:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.526+0000 7fd31fc40640 1 -- start start 2024-03-06T07:20:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31fc40640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3181529f0 0x7fd3180c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31fc40640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3180c57d0 0x7fd318155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31fc40640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd318152dd0 con 0x7fd3180c5370 2024-03-06T07:20:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31fc40640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd318159710 con 0x7fd3181529f0 2024-03-06T07:20:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31fc40640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd318156180 con 0x7fd3180c57d0 2024-03-06T07:20:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31d9dc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3181529f0 0x7fd3180c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31d1db640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd3180c5370 0x7fd318155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46174/0 (socket says 172.21.15.26:46174) 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31d1db640 1 -- 172.21.15.26:0/1081116852 learned_addr learned my addr 172.21.15.26:0/1081116852 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31e1dd640 1 --2- 172.21.15.26:0/1081116852 >> v2:172.21.15.117:3300/0 conn(0x7fd3180c57d0 0x7fd318155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31d9dc640 1 -- 172.21.15.26:0/1081116852 >> v2:172.21.15.117:3300/0 conn(0x7fd3180c57d0 msgr2=0x7fd318155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31d9dc640 1 --2- 172.21.15.26:0/1081116852 >> v2:172.21.15.117:3300/0 conn(0x7fd3180c57d0 0x7fd318155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31d9dc640 1 -- 172.21.15.26:0/1081116852 >> v1:172.21.15.26:6789/0 conn(0x7fd3180c5370 legacy=0x7fd318155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31d9dc640 1 -- 172.21.15.26:0/1081116852 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318158b00 con 0x7fd3181529f0 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31c9da640 1 -- 172.21.15.26:0/1081116852 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028529694 0 0) 0x7fd318152dd0 con 0x7fd3180c5370 2024-03-06T07:20:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.527+0000 7fd31e1dd640 1 --2- 172.21.15.26:0/1081116852 >> v2:172.21.15.117:3300/0 conn(0x7fd3180c57d0 0x7fd318155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.528+0000 7fd31d9dc640 1 --2- 172.21.15.26:0/1081116852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3181529f0 0x7fd3180c88f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd3180735c0 tx=0x7fd31002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15bf09ab0b65575 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.528+0000 7fd31c9da640 1 -- 172.21.15.26:0/1081116852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3100315a0 con 0x7fd3181529f0 2024-03-06T07:20:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.528+0000 7fd31c9da640 1 -- 172.21.15.26:0/1081116852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd310037070 con 0x7fd3181529f0 2024-03-06T07:20:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.528+0000 7fd31c9da640 1 -- 172.21.15.26:0/1081116852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310031960 con 0x7fd3181529f0 2024-03-06T07:20:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 -- 172.21.15.26:0/1081116852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3181529f0 msgr2=0x7fd3180c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1081116852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3181529f0 0x7fd3180c88f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd3180735c0 tx=0x7fd31002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 -- 172.21.15.26:0/1081116852 shutdown_connections 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1081116852 >> v2:172.21.15.117:3300/0 conn(0x7fd3180c57d0 0x7fd318155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1081116852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3181529f0 0x7fd3180c88f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 -- 172.21.15.26:0/1081116852 >> 172.21.15.26:0/1081116852 conn(0x7fd3180ac580 msgr2=0x7fd3180ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 -- 172.21.15.26:0/1081116852 shutdown_connections 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.529+0000 7fd31fc40640 1 -- 172.21.15.26:0/1081116852 wait complete. 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.530+0000 7fd31fc40640 1 Processor -- start 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.530+0000 7fd31fc40640 1 -- start start 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31fc40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3180c5370 0x7fd3181577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31fc40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180c57d0 0x7fd3180c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31fc40640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3181529f0 0x7fd3180c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31fc40640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181590c0 con 0x7fd3180c5370 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31fc40640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3180c68b0 con 0x7fd3180c57d0 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31fc40640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181593a0 con 0x7fd3181529f0 2024-03-06T07:20:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3180c5370 0x7fd3181577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3180c5370 0x7fd3181577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49454/0 (socket says 172.21.15.26:49454) 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 -- 172.21.15.26:0/1519270695 learned_addr learned my addr 172.21.15.26:0/1519270695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31e1dd640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3181529f0 0x7fd3180c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d1db640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180c57d0 0x7fd3180c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 -- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3181529f0 msgr2=0x7fd3180c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3181529f0 0x7fd3180c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 -- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180c57d0 msgr2=0x7fd3180c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180c57d0 0x7fd3180c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 -- 172.21.15.26:0/1519270695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318151850 con 0x7fd3180c5370 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd31d9dc640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3180c5370 0x7fd3181577e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd310009890 tx=0x7fd3100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.531+0000 7fd3167fc640 1 -- 172.21.15.26:0/1519270695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310032630 con 0x7fd3180c5370 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.532+0000 7fd3167fc640 1 -- 172.21.15.26:0/1519270695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3100327d0 con 0x7fd3180c5370 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.532+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3180c2f40 con 0x7fd3180c5370 2024-03-06T07:20:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.532+0000 7fd3167fc640 1 -- 172.21.15.26:0/1519270695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310032ba0 con 0x7fd3180c5370 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.532+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd318157d20 con 0x7fd3180c5370 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.533+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31804f330 con 0x7fd3180c5370 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.533+0000 7fd3167fc640 1 -- 172.21.15.26:0/1519270695 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd31004c020 con 0x7fd3180c5370 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.533+0000 7fd3167fc640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd30406b4d0 0x7fd30406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.533+0000 7fd3167fc640 1 -- 172.21.15.26:0/1519270695 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd31002f020 con 0x7fd3180c5370 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.533+0000 7fd31d1db640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd30406b4d0 0x7fd30406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.534+0000 7fd31d1db640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd30406b4d0 0x7fd30406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd3180ca5f0 tx=0x7fd30c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.538+0000 7fd3167fc640 1 -- 172.21.15.26:0/1519270695 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd310040070 con 0x7fd3180c5370 2024-03-06T07:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.717+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd31806cf00 con 0x7fd3180c5370 2024-03-06T07:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.718+0000 7fd3167fc640 1 -- 172.21.15.26:0/1519270695 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd31007ace0 con 0x7fd3180c5370 2024-03-06T07:20:35.719 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd30406b4d0 msgr2=0x7fd30406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd30406b4d0 0x7fd30406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd3180ca5f0 tx=0x7fd30c007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3180c5370 msgr2=0x7fd3181577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3180c5370 0x7fd3181577e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd310009890 tx=0x7fd3100046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 shutdown_connections 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd30406b4d0 0x7fd30406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3181529f0 0x7fd3180c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180c57d0 0x7fd3180c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 --2- 172.21.15.26:0/1519270695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3180c5370 0x7fd3181577e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 >> 172.21.15.26:0/1519270695 conn(0x7fd3180ac580 msgr2=0x7fd31814f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 shutdown_connections 2024-03-06T07:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.720+0000 7fd31fc40640 1 -- 172.21.15.26:0/1519270695 wait complete. 2024-03-06T07:20:35.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.836+0000 7f51301e2640 1 Processor -- start 2024-03-06T07:20:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.836+0000 7f51301e2640 1 -- start start 2024-03-06T07:20:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.837+0000 7f51301e2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51280a8290 0x7f51281539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.837+0000 7f51301e2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280c62a0 con 0x7f51280c4740 2024-03-06T07:20:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.837+0000 7f51301e2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280c4c00 con 0x7f51280c69a0 2024-03-06T07:20:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.837+0000 7f51301e2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5128153f00 con 0x7f51280a8290 2024-03-06T07:20:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.837+0000 7f512df7e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51280a8290 0x7f51281539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.837+0000 7f512d77d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51280c4740 0x7f51280701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46182/0 (socket says 172.21.15.26:46182) 2024-03-06T07:20:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.837+0000 7f512d77d640 1 -- 172.21.15.26:0/3882147365 learned_addr learned my addr 172.21.15.26:0/3882147365 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.838+0000 7f512df7e640 1 -- 172.21.15.26:0/3882147365 >> v1:172.21.15.117:6789/0 conn(0x7f51280c69a0 legacy=0x7f5128072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.838+0000 7f512df7e640 1 -- 172.21.15.26:0/3882147365 >> v1:172.21.15.26:6789/0 conn(0x7f51280c4740 legacy=0x7f51280701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.838+0000 7f512df7e640 1 -- 172.21.15.26:0/3882147365 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280c4ee0 con 0x7f51280a8290 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.838+0000 7f512df7e640 1 --2- 172.21.15.26:0/3882147365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f51281539c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f512402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c324057bdaf6e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.838+0000 7f512cf7c640 1 -- 172.21.15.26:0/3882147365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124039070 con 0x7f51280a8290 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f512cf7c640 1 -- 172.21.15.26:0/3882147365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512402cbd0 con 0x7f51280a8290 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f512cf7c640 1 -- 172.21.15.26:0/3882147365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124034440 con 0x7f51280a8290 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f51301e2640 1 -- 172.21.15.26:0/3882147365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 msgr2=0x7f51281539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f51301e2640 1 --2- 172.21.15.26:0/3882147365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f51281539c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f512402be20 comp rx=0 tx=0).stop 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f51301e2640 1 -- 172.21.15.26:0/3882147365 shutdown_connections 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f51301e2640 1 --2- 172.21.15.26:0/3882147365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f51281539c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f51301e2640 1 -- 172.21.15.26:0/3882147365 >> 172.21.15.26:0/3882147365 conn(0x7f51280c5560 msgr2=0x7f51280c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f51301e2640 1 -- 172.21.15.26:0/3882147365 shutdown_connections 2024-03-06T07:20:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.839+0000 7f51301e2640 1 -- 172.21.15.26:0/3882147365 wait complete. 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.840+0000 7f51301e2640 1 Processor -- start 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.840+0000 7f51301e2640 1 -- start start 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f51301e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f512814a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f51301e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51280c4740 0x7f5128155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f51301e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51280c69a0 0x7f5128155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f51301e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5128072d90 con 0x7f51280a8290 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f51301e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512806fc80 con 0x7f51280c69a0 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f51301e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5128072f10 con 0x7f51280c4740 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f512814a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f512814a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49474/0 (socket says 172.21.15.26:49474) 2024-03-06T07:20:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 -- 172.21.15.26:0/3113291022 learned_addr learned my addr 172.21.15.26:0/3113291022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512d77d640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51280c4740 0x7f5128155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512e77f640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51280c69a0 0x7f5128155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 -- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51280c4740 msgr2=0x7f5128155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51280c4740 0x7f5128155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 -- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51280c69a0 msgr2=0x7f5128155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51280c69a0 0x7f5128155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.841+0000 7f512df7e640 1 -- 172.21.15.26:0/3113291022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128156b60 con 0x7f51280a8290 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.842+0000 7f512df7e640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f512814a0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f5124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.842+0000 7f511e7fc640 1 -- 172.21.15.26:0/3113291022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124039050 con 0x7f51280a8290 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.842+0000 7f511e7fc640 1 -- 172.21.15.26:0/3113291022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124045070 con 0x7f51280a8290 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.842+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51281e9a60 con 0x7f51280a8290 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.842+0000 7f511e7fc640 1 -- 172.21.15.26:0/3113291022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512403ea70 con 0x7f51280a8290 2024-03-06T07:20:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.842+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5128150490 con 0x7f51280a8290 2024-03-06T07:20:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.843+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512804f330 con 0x7f51280a8290 2024-03-06T07:20:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.843+0000 7f511e7fc640 1 -- 172.21.15.26:0/3113291022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f512403ec10 con 0x7f51280a8290 2024-03-06T07:20:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.843+0000 7f511e7fc640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f510c06b500 0x7f510c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.844+0000 7f511e7fc640 1 -- 172.21.15.26:0/3113291022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f51240ad360 con 0x7f51280a8290 2024-03-06T07:20:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.844+0000 7f512d77d640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f510c06b500 0x7f510c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.844+0000 7f512d77d640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f510c06b500 0x7f510c06d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f51280ac5f0 tx=0x7f5118009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:35.848+0000 7f511e7fc640 1 -- 172.21.15.26:0/3113291022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f512407ba00 con 0x7f51280a8290 2024-03-06T07:20:36.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.028+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5128070e30 con 0x7f51280a8290 2024-03-06T07:20:36.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.028+0000 7f511e7fc640 1 -- 172.21.15.26:0/3113291022 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f512407faa0 con 0x7f51280a8290 2024-03-06T07:20:36.030 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f510c06b500 msgr2=0x7f510c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f510c06b500 0x7f510c06d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f51280ac5f0 tx=0x7f5118009040 comp rx=0 tx=0).stop 2024-03-06T07:20:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 msgr2=0x7f512814a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f512814a0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f5124004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 shutdown_connections 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f510c06b500 0x7f510c06d9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51280c69a0 0x7f5128155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51280c4740 0x7f5128155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 --2- 172.21.15.26:0/3113291022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51280a8290 0x7f512814a0d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 >> 172.21.15.26:0/3113291022 conn(0x7f51280c5560 msgr2=0x7f5128072580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 shutdown_connections 2024-03-06T07:20:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.030+0000 7f51301e2640 1 -- 172.21.15.26:0/3113291022 wait complete. 2024-03-06T07:20:36.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.145+0000 7efd0f26f640 1 Processor -- start 2024-03-06T07:20:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.145+0000 7efd0f26f640 1 -- start start 2024-03-06T07:20:36.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0f26f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd080b00d0 0x7efd080c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0f26f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efd080c7210 0x7efd080c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0f26f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd080b04b0 con 0x7efd0806b8e0 2024-03-06T07:20:36.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0f26f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd08156160 con 0x7efd080b00d0 2024-03-06T07:20:36.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0f26f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd080c5710 con 0x7efd080c7210 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0d00b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd080b00d0 0x7efd080c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0d00b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd080b00d0 0x7efd080c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49476/0 (socket says 172.21.15.26:49476) 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0d00b640 1 -- 172.21.15.26:0/2352482413 learned_addr learned my addr 172.21.15.26:0/2352482413 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0d80c640 1 --2- 172.21.15.26:0/2352482413 >> v2:172.21.15.117:3300/0 conn(0x7efd080c7210 0x7efd080c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0d00b640 1 -- 172.21.15.26:0/2352482413 >> v2:172.21.15.117:3300/0 conn(0x7efd080c7210 msgr2=0x7efd080c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0d00b640 1 --2- 172.21.15.26:0/2352482413 >> v2:172.21.15.117:3300/0 conn(0x7efd080c7210 0x7efd080c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.146+0000 7efd0d00b640 1 -- 172.21.15.26:0/2352482413 >> v1:172.21.15.117:6789/0 conn(0x7efd0806b8e0 legacy=0x7efd0806bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.147+0000 7efd0d00b640 1 -- 172.21.15.26:0/2352482413 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0806d210 con 0x7efd080b00d0 2024-03-06T07:20:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.147+0000 7efd0d00b640 1 --2- 172.21.15.26:0/2352482413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd080b00d0 0x7efd080c8ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd080be2e0 tx=0x7efd0002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e807151ab1f2fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.147+0000 7efd07fff640 1 -- 172.21.15.26:0/2352482413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00034070 con 0x7efd080b00d0 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.147+0000 7efd07fff640 1 -- 172.21.15.26:0/2352482413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd00030860 con 0x7efd080b00d0 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.147+0000 7efd07fff640 1 -- 172.21.15.26:0/2352482413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00030c30 con 0x7efd080b00d0 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 -- 172.21.15.26:0/2352482413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd080b00d0 msgr2=0x7efd080c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2352482413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd080b00d0 0x7efd080c8ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd080be2e0 tx=0x7efd0002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 -- 172.21.15.26:0/2352482413 shutdown_connections 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2352482413 >> v2:172.21.15.117:3300/0 conn(0x7efd080c7210 0x7efd080c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2352482413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd080b00d0 0x7efd080c8ff0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 -- 172.21.15.26:0/2352482413 >> 172.21.15.26:0/2352482413 conn(0x7efd080ac810 msgr2=0x7efd080acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 -- 172.21.15.26:0/2352482413 shutdown_connections 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.148+0000 7efd0f26f640 1 -- 172.21.15.26:0/2352482413 wait complete. 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.149+0000 7efd0f26f640 1 Processor -- start 2024-03-06T07:20:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.149+0000 7efd0f26f640 1 -- start start 2024-03-06T07:20:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0f26f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd0806b8e0 0x7efd081ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0f26f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd080b00d0 0x7efd080c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0f26f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd080c7210 0x7efd080c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0f26f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd080c5ba0 con 0x7efd0806b8e0 2024-03-06T07:20:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0f26f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd08152c50 con 0x7efd080c7210 2024-03-06T07:20:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0f26f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd080c5d20 con 0x7efd080b00d0 2024-03-06T07:20:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd0806b8e0 0x7efd081ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd0806b8e0 0x7efd081ee5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49482/0 (socket says 172.21.15.26:49482) 2024-03-06T07:20:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 -- 172.21.15.26:0/2360905496 learned_addr learned my addr 172.21.15.26:0/2360905496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0c80a640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd080b00d0 0x7efd080c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d80c640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd080c7210 0x7efd080c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 -- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd080b00d0 msgr2=0x7efd080c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd080b00d0 0x7efd080c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 -- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd080c7210 msgr2=0x7efd080c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd080c7210 0x7efd080c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 -- 172.21.15.26:0/2360905496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd080c2fe0 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.150+0000 7efd0d00b640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd0806b8e0 0x7efd081ee5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd0002b9e0 tx=0x7efd00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.151+0000 7efd05ffb640 1 -- 172.21.15.26:0/2360905496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00048070 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.151+0000 7efd05ffb640 1 -- 172.21.15.26:0/2360905496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd00030660 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.151+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd081ef020 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.151+0000 7efd05ffb640 1 -- 172.21.15.26:0/2360905496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00034040 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.151+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd080cb280 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.152+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0804f330 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.152+0000 7efd05ffb640 1 -- 172.21.15.26:0/2360905496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd0004b070 con 0x7efd0806b8e0 2024-03-06T07:20:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.152+0000 7efd05ffb640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf406b4d0 0x7efcf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.153+0000 7efd05ffb640 1 -- 172.21.15.26:0/2360905496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd000ac360 con 0x7efd0806b8e0 2024-03-06T07:20:36.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.153+0000 7efd0c80a640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf406b4d0 0x7efcf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.153+0000 7efd0c80a640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf406b4d0 0x7efcf406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd080af8f0 tx=0x7efcfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.157+0000 7efd05ffb640 1 -- 172.21.15.26:0/2360905496 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd0007a8e0 con 0x7efd0806b8e0 2024-03-06T07:20:36.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.336+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efd080c95e0 con 0x7efd0806b8e0 2024-03-06T07:20:36.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.337+0000 7efd05ffb640 1 -- 172.21.15.26:0/2360905496 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7efd0007e980 con 0x7efd0806b8e0 2024-03-06T07:20:36.338 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:36.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.338+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf406b4d0 msgr2=0x7efcf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.338+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf406b4d0 0x7efcf406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd080af8f0 tx=0x7efcfc009040 comp rx=0 tx=0).stop 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd0806b8e0 msgr2=0x7efd081ee5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd0806b8e0 0x7efd081ee5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd0002b9e0 tx=0x7efd00004730 comp rx=0 tx=0).stop 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 shutdown_connections 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf406b4d0 0x7efcf406d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd080c7210 0x7efd080c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd080b00d0 0x7efd080c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 --2- 172.21.15.26:0/2360905496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd0806b8e0 0x7efd081ee5a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 >> 172.21.15.26:0/2360905496 conn(0x7efd080ac810 msgr2=0x7efd081539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 shutdown_connections 2024-03-06T07:20:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.339+0000 7efd0f26f640 1 -- 172.21.15.26:0/2360905496 wait complete. 2024-03-06T07:20:36.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:36.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.455+0000 7f6c12adf640 1 Processor -- start 2024-03-06T07:20:36.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.455+0000 7f6c12adf640 1 -- start start 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.455+0000 7f6c12adf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c0c06f7b0 0x7f6c0c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.455+0000 7f6c12adf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c0c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.455+0000 7f6c12adf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c157340 con 0x7f6c0c072720 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.455+0000 7f6c12adf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c156b40 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.455+0000 7f6c12adf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c3740 con 0x7f6c0c1506f0 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c1087b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c0c072720 0x7f6c0c156430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46196/0 (socket says 172.21.15.26:46196) 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c0bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c0c06f7b0 0x7f6c0c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c1087b640 1 -- 172.21.15.26:0/3960599258 learned_addr learned my addr 172.21.15.26:0/3960599258 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c1107c640 1 --2- 172.21.15.26:0/3960599258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c0c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c0b7fe640 1 -- 172.21.15.26:0/3960599258 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230520824 0 0) 0x7f6c0c157340 con 0x7f6c0c072720 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c0b7fe640 1 -- 172.21.15.26:0/3960599258 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf4003520 con 0x7f6c0c072720 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c1107c640 1 -- 172.21.15.26:0/3960599258 >> v2:172.21.15.26:3300/0 conn(0x7f6c0c06f7b0 msgr2=0x7f6c0c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c1107c640 1 --2- 172.21.15.26:0/3960599258 >> v2:172.21.15.26:3300/0 conn(0x7f6c0c06f7b0 0x7f6c0c1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c1107c640 1 -- 172.21.15.26:0/3960599258 >> v1:172.21.15.26:6789/0 conn(0x7f6c0c072720 legacy=0x7f6c0c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c1107c640 1 -- 172.21.15.26:0/3960599258 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c156d80 con 0x7f6c0c1506f0 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.456+0000 7f6c0bfff640 1 --2- 172.21.15.26:0/3960599258 >> v2:172.21.15.26:3300/0 conn(0x7f6c0c06f7b0 0x7f6c0c1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c1107c640 1 --2- 172.21.15.26:0/3960599258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c0c3bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0097a0 tx=0x7f6bfc02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b28002651a62953 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c0b7fe640 1 -- 172.21.15.26:0/3960599258 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc031780 con 0x7f6c0c1506f0 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c0b7fe640 1 -- 172.21.15.26:0/3960599258 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc037070 con 0x7f6c0c1506f0 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c0b7fe640 1 -- 172.21.15.26:0/3960599258 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc031b20 con 0x7f6c0c1506f0 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 -- 172.21.15.26:0/3960599258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 msgr2=0x7f6c0c0c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 --2- 172.21.15.26:0/3960599258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c0c3bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0097a0 tx=0x7f6bfc02b820 comp rx=0 tx=0).stop 2024-03-06T07:20:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 -- 172.21.15.26:0/3960599258 shutdown_connections 2024-03-06T07:20:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 --2- 172.21.15.26:0/3960599258 >> v2:172.21.15.26:3300/0 conn(0x7f6c0c06f7b0 0x7f6c0c1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 --2- 172.21.15.26:0/3960599258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c0c3bd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 -- 172.21.15.26:0/3960599258 >> 172.21.15.26:0/3960599258 conn(0x7f6c0c0aa630 msgr2=0x7f6c0c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 -- 172.21.15.26:0/3960599258 shutdown_connections 2024-03-06T07:20:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.457+0000 7f6c12adf640 1 -- 172.21.15.26:0/3960599258 wait complete. 2024-03-06T07:20:36.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 Processor -- start 2024-03-06T07:20:36.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 -- start start 2024-03-06T07:20:36.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c0c06f7b0 0x7f6c0c14e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c0c072720 0x7f6c0c1488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c072b00 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c33d0 con 0x7f6c0c072720 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c12adf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c072c80 con 0x7f6c0c1506f0 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.459+0000 7f6c1087b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c0c06f7b0 0x7f6c0c14e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c0c06f7b0 0x7f6c0c14e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49504/0 (socket says 172.21.15.26:49504) 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 -- 172.21.15.26:0/970612344 learned_addr learned my addr 172.21.15.26:0/970612344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1107c640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c0bfff640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c0c072720 0x7f6c0c1488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 -- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 msgr2=0x7f6c0c148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c148de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 -- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c0c072720 msgr2=0x7f6c0c1488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c0c072720 0x7f6c0c1488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 -- 172.21.15.26:0/970612344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c1e9080 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c1087b640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c0c06f7b0 0x7f6c0c14e3b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400cb70 tx=0x7f6c0400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c097fa640 1 -- 172.21.15.26:0/970612344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04017070 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c097fa640 1 -- 172.21.15.26:0/970612344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0400ad40 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c097fa640 1 -- 172.21.15.26:0/970612344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04012620 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c0ad7e0 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.460+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c1eb710 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.461+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c1eb910 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.462+0000 7f6c097fa640 1 -- 172.21.15.26:0/970612344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c0401a070 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.462+0000 7f6c097fa640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6bf806b4d0 0x7f6bf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.462+0000 7f6c097fa640 1 -- 172.21.15.26:0/970612344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6c0408a300 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.462+0000 7f6c0bfff640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6bf806b4d0 0x7f6bf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.463+0000 7f6c0bfff640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6bf806b4d0 0x7f6bf806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009560 tx=0x7f6c00009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.466+0000 7f6c097fa640 1 -- 172.21.15.26:0/970612344 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c0c1e9080 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.649+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c0c073060 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.651+0000 7f6c097fa640 1 -- 172.21.15.26:0/970612344 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f6c04058800 con 0x7f6c0c06f7b0 2024-03-06T07:20:36.652 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:36.653 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6bf806b4d0 msgr2=0x7f6bf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6bf806b4d0 0x7f6bf806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009560 tx=0x7f6c00009190 comp rx=0 tx=0).stop 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c0c06f7b0 msgr2=0x7f6c0c14e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c0c06f7b0 0x7f6c0c14e3b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400cb70 tx=0x7f6c0400a430 comp rx=0 tx=0).stop 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 shutdown_connections 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6bf806b4d0 0x7f6bf806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c0c1506f0 0x7f6c0c148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c0c072720 0x7f6c0c1488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 --2- 172.21.15.26:0/970612344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c0c06f7b0 0x7f6c0c14e3b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 >> 172.21.15.26:0/970612344 conn(0x7f6c0c0aa630 msgr2=0x7f6c0c0a8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 shutdown_connections 2024-03-06T07:20:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.653+0000 7f6c12adf640 1 -- 172.21.15.26:0/970612344 wait complete. 2024-03-06T07:20:36.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:36.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:36.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.770+0000 7f15d43dd640 1 Processor -- start 2024-03-06T07:20:36.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.770+0000 7f15d43dd640 1 -- start start 2024-03-06T07:20:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d43dd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc1529d0 0x7f15cc0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d43dd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc152db0 con 0x7f15cc0c57e0 2024-03-06T07:20:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d43dd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc1590f0 con 0x7f15cc072cd0 2024-03-06T07:20:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d43dd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0c6dd0 con 0x7f15cc1529d0 2024-03-06T07:20:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d1978640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15cc0c57e0 0x7f15cc155120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46200/0 (socket says 172.21.15.26:46200) 2024-03-06T07:20:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d1978640 1 -- 172.21.15.26:0/3530078493 learned_addr learned my addr 172.21.15.26:0/3530078493 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d2179640 1 --2- 172.21.15.26:0/3530078493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc1529d0 0x7f15cc0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d1177640 1 -- 172.21.15.26:0/3530078493 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553576818 0 0) 0x7f15bc001110 con 0x7f15cc0c57e0 2024-03-06T07:20:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d1177640 1 -- 172.21.15.26:0/3530078493 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b8003520 con 0x7f15cc0c57e0 2024-03-06T07:20:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d1177640 1 -- 172.21.15.26:0/3530078493 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549677491 0 0) 0x7f15c4001490 con 0x7f15cc072cd0 2024-03-06T07:20:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d1177640 1 -- 172.21.15.26:0/3530078493 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15bc001110 con 0x7f15cc072cd0 2024-03-06T07:20:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d2179640 1 -- 172.21.15.26:0/3530078493 >> v1:172.21.15.117:6789/0 conn(0x7f15cc072cd0 legacy=0x7f15cc15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d2179640 1 -- 172.21.15.26:0/3530078493 >> v1:172.21.15.26:6789/0 conn(0x7f15cc0c57e0 legacy=0x7f15cc155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.771+0000 7f15d2179640 1 -- 172.21.15.26:0/3530078493 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc159650 con 0x7f15cc1529d0 2024-03-06T07:20:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.772+0000 7f15d2179640 1 --2- 172.21.15.26:0/3530078493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc1529d0 0x7f15cc0c52a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0c6f50 tx=0x7f15c802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf52d93db320575d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.772+0000 7f15d1177640 1 -- 172.21.15.26:0/3530078493 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8031760 con 0x7f15cc1529d0 2024-03-06T07:20:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.772+0000 7f15d1177640 1 -- 172.21.15.26:0/3530078493 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8037070 con 0x7f15cc1529d0 2024-03-06T07:20:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.772+0000 7f15d1177640 1 -- 172.21.15.26:0/3530078493 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8031b00 con 0x7f15cc1529d0 2024-03-06T07:20:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.773+0000 7f15d43dd640 1 -- 172.21.15.26:0/3530078493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc1529d0 msgr2=0x7f15cc0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.773+0000 7f15d43dd640 1 --2- 172.21.15.26:0/3530078493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc1529d0 0x7f15cc0c52a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0c6f50 tx=0x7f15c802b800 comp rx=0 tx=0).stop 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.773+0000 7f15d43dd640 1 -- 172.21.15.26:0/3530078493 shutdown_connections 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.773+0000 7f15d43dd640 1 --2- 172.21.15.26:0/3530078493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc1529d0 0x7f15cc0c52a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.773+0000 7f15d43dd640 1 -- 172.21.15.26:0/3530078493 >> 172.21.15.26:0/3530078493 conn(0x7f15cc0aa630 msgr2=0x7f15cc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.773+0000 7f15d43dd640 1 -- 172.21.15.26:0/3530078493 shutdown_connections 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.773+0000 7f15d43dd640 1 -- 172.21.15.26:0/3530078493 wait complete. 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 Processor -- start 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 -- start start 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc072cd0 0x7f15cc158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15cc0c57e0 0x7f15cc0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc1529d0 0x7f15cc0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc159370 con 0x7f15cc0c57e0 2024-03-06T07:20:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0c6940 con 0x7f15cc1529d0 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.774+0000 7f15d43dd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0723b0 con 0x7f15cc072cd0 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15cc0c57e0 0x7f15cc0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15cc0c57e0 0x7f15cc0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49514/0 (socket says 172.21.15.26:49514) 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 -- 172.21.15.26:0/1643606260 learned_addr learned my addr 172.21.15.26:0/1643606260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d2179640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc072cd0 0x7f15cc158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d297a640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc1529d0 0x7f15cc0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 -- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc072cd0 msgr2=0x7f15cc158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc072cd0 0x7f15cc158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 -- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc1529d0 msgr2=0x7f15cc0ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc1529d0 0x7f15cc0ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 -- 172.21.15.26:0/1643606260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc152060 con 0x7f15cc0c57e0 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d297a640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc1529d0 0x7f15cc0ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d1978640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15cc0c57e0 0x7f15cc0adda0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00c760 tx=0x7f15bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15c2ffd640 1 -- 172.21.15.26:0/1643606260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc017070 con 0x7f15cc0c57e0 2024-03-06T07:20:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15c2ffd640 1 -- 172.21.15.26:0/1643606260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc005ab0 con 0x7f15cc0c57e0 2024-03-06T07:20:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc1564b0 con 0x7f15cc0c57e0 2024-03-06T07:20:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15c2ffd640 1 -- 172.21.15.26:0/1643606260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc005e60 con 0x7f15cc0c57e0 2024-03-06T07:20:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.775+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc06c0f0 con 0x7f15cc0c57e0 2024-03-06T07:20:36.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.776+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc06f2b0 con 0x7f15cc0c57e0 2024-03-06T07:20:36.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.777+0000 7f15c2ffd640 1 -- 172.21.15.26:0/1643606260 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15bc01a070 con 0x7f15cc0c57e0 2024-03-06T07:20:36.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.777+0000 7f15c2ffd640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15b006b4d0 0x7f15b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:36.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.777+0000 7f15c2ffd640 1 -- 172.21.15.26:0/1643606260 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15bc089960 con 0x7f15cc0c57e0 2024-03-06T07:20:36.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.777+0000 7f15d2179640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15b006b4d0 0x7f15b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:36.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.778+0000 7f15d2179640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15b006b4d0 0x7f15b006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f15c8036000 tx=0x7f15c803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:36.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.781+0000 7f15c2ffd640 1 -- 172.21.15.26:0/1643606260 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15cc06f2b0 con 0x7f15cc0c57e0 2024-03-06T07:20:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.961+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15cc04f330 con 0x7f15cc0c57e0 2024-03-06T07:20:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.962+0000 7f15c2ffd640 1 -- 172.21.15.26:0/1643606260 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15cc152060 con 0x7f15cc0c57e0 2024-03-06T07:20:36.963 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15b006b4d0 msgr2=0x7f15b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15b006b4d0 0x7f15b006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f15c8036000 tx=0x7f15c803d040 comp rx=0 tx=0).stop 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15cc0c57e0 msgr2=0x7f15cc0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15cc0c57e0 0x7f15cc0adda0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00c760 tx=0x7f15bc00cc20 comp rx=0 tx=0).stop 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 shutdown_connections 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15b006b4d0 0x7f15b006d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc1529d0 0x7f15cc0ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15cc0c57e0 0x7f15cc0adda0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 --2- 172.21.15.26:0/1643606260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15cc072cd0 0x7f15cc158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 >> 172.21.15.26:0/1643606260 conn(0x7f15cc0aa630 msgr2=0x7f15cc14afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 shutdown_connections 2024-03-06T07:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:36.964+0000 7f15d43dd640 1 -- 172.21.15.26:0/1643606260 wait complete. 2024-03-06T07:20:36.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.079+0000 7fe3463d7640 1 Processor -- start 2024-03-06T07:20:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.079+0000 7fe3463d7640 1 -- start start 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe3463d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400b0070 0x7fe340155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe3463d7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe34006ca70 0x7fe3400c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe3463d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400b0450 con 0x7fe3400c5380 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe3463d7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34006c020 con 0x7fe34006ca70 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe3463d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400c77f0 con 0x7fe3400b0070 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe344974640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe3400c5380 0x7fe34014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44328/0 (socket says 172.21.15.26:44328) 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe344974640 1 -- 172.21.15.26:0/1036952519 learned_addr learned my addr 172.21.15.26:0/1036952519 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe33f7fe640 1 --2- 172.21.15.26:0/1036952519 >> v2:172.21.15.117:3300/0 conn(0x7fe34006ca70 0x7fe3400c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe33ffff640 1 --2- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400b0070 0x7fe340155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.080+0000 7fe33effd640 1 -- 172.21.15.26:0/1036952519 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802277551 0 0) 0x7fe338001490 con 0x7fe3400c5380 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33effd640 1 -- 172.21.15.26:0/1036952519 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c003520 con 0x7fe3400c5380 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33f7fe640 1 -- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400b0070 msgr2=0x7fe340155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33f7fe640 1 --2- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400b0070 0x7fe340155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33f7fe640 1 -- 172.21.15.26:0/1036952519 >> v1:172.21.15.26:6789/0 conn(0x7fe3400c5380 legacy=0x7fe34014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33f7fe640 1 -- 172.21.15.26:0/1036952519 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400c7ad0 con 0x7fe34006ca70 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33ffff640 1 --2- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400b0070 0x7fe340155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33f7fe640 1 --2- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 0x7fe3400c8900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc8290594dd7014 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.081+0000 7fe33effd640 1 -- 172.21.15.26:0/1036952519 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300315a0 con 0x7fe34006ca70 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe33effd640 1 -- 172.21.15.26:0/1036952519 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe330037070 con 0x7fe34006ca70 2024-03-06T07:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe33effd640 1 -- 172.21.15.26:0/1036952519 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330031960 con 0x7fe34006ca70 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 -- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 msgr2=0x7fe3400c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 --2- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 0x7fe3400c8900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 -- 172.21.15.26:0/1036952519 shutdown_connections 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 --2- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 0x7fe3400c8900 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 --2- 172.21.15.26:0/1036952519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400b0070 0x7fe340155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 -- 172.21.15.26:0/1036952519 >> 172.21.15.26:0/1036952519 conn(0x7fe3400ac590 msgr2=0x7fe3400ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 -- 172.21.15.26:0/1036952519 shutdown_connections 2024-03-06T07:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.082+0000 7fe3463d7640 1 -- 172.21.15.26:0/1036952519 wait complete. 2024-03-06T07:20:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.083+0000 7fe3463d7640 1 Processor -- start 2024-03-06T07:20:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.083+0000 7fe3463d7640 1 -- start start 2024-03-06T07:20:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe3463d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 0x7fe34014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe3463d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3400b0070 0x7fe340154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe3463d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400c5380 0x7fe3401ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe3463d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34006cef0 con 0x7fe3400b0070 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe3463d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34006c550 con 0x7fe34006ca70 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe3463d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34006da80 con 0x7fe3400c5380 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3400b0070 0x7fe340154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3400b0070 0x7fe340154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45690/0 (socket says 172.21.15.26:45690) 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 -- 172.21.15.26:0/3669481920 learned_addr learned my addr 172.21.15.26:0/3669481920 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33ffff640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 0x7fe34014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe344974640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400c5380 0x7fe3401ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 -- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400c5380 msgr2=0x7fe3401ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400c5380 0x7fe3401ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 -- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 msgr2=0x7fe34014b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 0x7fe34014b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.084+0000 7fe33f7fe640 1 -- 172.21.15.26:0/3669481920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340156b10 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.085+0000 7fe33f7fe640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3400b0070 0x7fe340154650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe3300098c0 tx=0x7fe3300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.085+0000 7fe33cff9640 1 -- 172.21.15.26:0/3669481920 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330032630 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.085+0000 7fe33cff9640 1 -- 172.21.15.26:0/3669481920 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3300327f0 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.085+0000 7fe33cff9640 1 -- 172.21.15.26:0/3669481920 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330032be0 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.085+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe340156df0 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.085+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3401573b0 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.086+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34004f330 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.086+0000 7fe33cff9640 1 -- 172.21.15.26:0/3669481920 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe33004e430 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.087+0000 7fe33cff9640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe32006b4d0 0x7fe32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.087+0000 7fe33cff9640 1 -- 172.21.15.26:0/3669481920 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe33002f030 con 0x7fe3400b0070 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.087+0000 7fe33ffff640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe32006b4d0 0x7fe32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.087+0000 7fe33ffff640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe32006b4d0 0x7fe32006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3400c3ed0 tx=0x7fe328009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.091+0000 7fe33cff9640 1 -- 172.21.15.26:0/3669481920 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe330040070 con 0x7fe3400b0070 2024-03-06T07:20:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.275+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3400af2f0 con 0x7fe3400b0070 2024-03-06T07:20:37.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.275+0000 7fe33cff9640 1 -- 172.21.15.26:0/3669481920 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe33007a6d0 con 0x7fe3400b0070 2024-03-06T07:20:37.277 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe32006b4d0 msgr2=0x7fe32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe32006b4d0 0x7fe32006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3400c3ed0 tx=0x7fe328009190 comp rx=0 tx=0).stop 2024-03-06T07:20:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3400b0070 msgr2=0x7fe340154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3400b0070 0x7fe340154650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe3300098c0 tx=0x7fe3300046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 shutdown_connections 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe32006b4d0 0x7fe32006d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3400c5380 0x7fe3401ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3400b0070 0x7fe340154650 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 --2- 172.21.15.26:0/3669481920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34006ca70 0x7fe34014b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.277+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 >> 172.21.15.26:0/3669481920 conn(0x7fe3400ac590 msgr2=0x7fe3400ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.278+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 shutdown_connections 2024-03-06T07:20:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.278+0000 7fe3463d7640 1 -- 172.21.15.26:0/3669481920 wait complete. 2024-03-06T07:20:37.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.393+0000 7f5f5a2c3640 1 Processor -- start 2024-03-06T07:20:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.393+0000 7f5f5a2c3640 1 -- start start 2024-03-06T07:20:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f5a2c3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5f540ab170 0x7f5f540c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f5a2c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f540c3ef0 con 0x7f5f540c5990 2024-03-06T07:20:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f5a2c3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f541556d0 con 0x7f5f5406c900 2024-03-06T07:20:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f5a2c3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f541559b0 con 0x7f5f540ab170 2024-03-06T07:20:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f53fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5f540c5990 0x7f5f5406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44340/0 (socket says 172.21.15.26:44340) 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f53fff640 1 -- 172.21.15.26:0/1955451740 learned_addr learned my addr 172.21.15.26:0/1955451740 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f58860640 1 --2- 172.21.15.26:0/1955451740 >> v2:172.21.15.117:3300/0 conn(0x7f5f540ab170 0x7f5f540c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f52ffd640 1 -- 172.21.15.26:0/1955451740 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256828151 0 0) 0x7f5f540c3ef0 con 0x7f5f540c5990 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f52ffd640 1 -- 172.21.15.26:0/1955451740 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f3c003520 con 0x7f5f540c5990 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f52ffd640 1 -- 172.21.15.26:0/1955451740 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073403012 0 0) 0x7f5f541556d0 con 0x7f5f5406c900 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f58860640 1 -- 172.21.15.26:0/1955451740 >> v1:172.21.15.117:6789/0 conn(0x7f5f5406c900 legacy=0x7f5f540a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f58860640 1 -- 172.21.15.26:0/1955451740 >> v1:172.21.15.26:6789/0 conn(0x7f5f540c5990 legacy=0x7f5f5406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.394+0000 7f5f58860640 1 -- 172.21.15.26:0/1955451740 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f54155c90 con 0x7f5f540ab170 2024-03-06T07:20:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.395+0000 7f5f58860640 1 --2- 172.21.15.26:0/1955451740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 0x7f5f540c0ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009dc0 tx=0x7f5f4c02bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22c9be4fcab45876 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.395+0000 7f5f52ffd640 1 -- 172.21.15.26:0/1955451740 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c039070 con 0x7f5f540ab170 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.395+0000 7f5f52ffd640 1 -- 172.21.15.26:0/1955451740 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c02ccd0 con 0x7f5f540ab170 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.395+0000 7f5f52ffd640 1 -- 172.21.15.26:0/1955451740 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c034660 con 0x7f5f540ab170 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.395+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/1955451740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 msgr2=0x7f5f540c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.395+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/1955451740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 0x7f5f540c0ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009dc0 tx=0x7f5f4c02bf20 comp rx=0 tx=0).stop 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.396+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/1955451740 shutdown_connections 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.396+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/1955451740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 0x7f5f540c0ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.396+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/1955451740 >> 172.21.15.26:0/1955451740 conn(0x7f5f540c6710 msgr2=0x7f5f540c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.396+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/1955451740 shutdown_connections 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.396+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/1955451740 wait complete. 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 Processor -- start 2024-03-06T07:20:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 -- start start 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f5406c900 0x7f5f540c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 0x7f5f541536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f540c5990 0x7f5f54153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f54156610 con 0x7f5f540c5990 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f54156490 con 0x7f5f540ab170 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f5a2c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540c3af0 con 0x7f5f5406c900 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.397+0000 7f5f58860640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f540c5990 0x7f5f54153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f540c5990 0x7f5f54153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45704/0 (socket says 172.21.15.26:45704) 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 -- 172.21.15.26:0/3758742975 learned_addr learned my addr 172.21.15.26:0/3758742975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f53fff640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f5406c900 0x7f5f540c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f537fe640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 0x7f5f541536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 -- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f5406c900 msgr2=0x7f5f540c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f5406c900 0x7f5f540c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 -- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 msgr2=0x7f5f541536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 0x7f5f541536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 -- 172.21.15.26:0/3758742975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5414f850 con 0x7f5f540c5990 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f58860640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f540c5990 0x7f5f54153c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c02c530 tx=0x7f5f4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f50ff9640 1 -- 172.21.15.26:0/3758742975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c039050 con 0x7f5f540c5990 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f50ff9640 1 -- 172.21.15.26:0/3758742975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c045070 con 0x7f5f540c5990 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f50ff9640 1 -- 172.21.15.26:0/3758742975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c009450 con 0x7f5f540c5990 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5414f570 con 0x7f5f540c5990 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.398+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5414cf40 con 0x7f5f540c5990 2024-03-06T07:20:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.399+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5404f330 con 0x7f5f540c5990 2024-03-06T07:20:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.400+0000 7f5f50ff9640 1 -- 172.21.15.26:0/3758742975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f4c03e850 con 0x7f5f540c5990 2024-03-06T07:20:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.400+0000 7f5f50ff9640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f4006b4d0 0x7f5f4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.400+0000 7f5f50ff9640 1 -- 172.21.15.26:0/3758742975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5f4c0ad2e0 con 0x7f5f540c5990 2024-03-06T07:20:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.400+0000 7f5f53fff640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f4006b4d0 0x7f5f4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.400+0000 7f5f53fff640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f4006b4d0 0x7f5f4006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005880 tx=0x7f5f48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.404+0000 7f5f50ff9640 1 -- 172.21.15.26:0/3758742975 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5f4c07b980 con 0x7f5f540c5990 2024-03-06T07:20:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.588+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f540c1740 con 0x7f5f540c5990 2024-03-06T07:20:37.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.588+0000 7f5f50ff9640 1 -- 172.21.15.26:0/3758742975 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f4c07fa20 con 0x7f5f540c5990 2024-03-06T07:20:37.590 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f4006b4d0 msgr2=0x7f5f4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f4006b4d0 0x7f5f4006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005880 tx=0x7f5f48009040 comp rx=0 tx=0).stop 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f540c5990 msgr2=0x7f5f54153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f540c5990 0x7f5f54153c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c02c530 tx=0x7f5f4c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 shutdown_connections 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f4006b4d0 0x7f5f4006d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f540c5990 0x7f5f54153c30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f540ab170 0x7f5f541536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 --2- 172.21.15.26:0/3758742975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f5406c900 0x7f5f540c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 >> 172.21.15.26:0/3758742975 conn(0x7f5f540c6710 msgr2=0x7f5f5414c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 shutdown_connections 2024-03-06T07:20:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.590+0000 7f5f5a2c3640 1 -- 172.21.15.26:0/3758742975 wait complete. 2024-03-06T07:20:37.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.706+0000 7fceb5ad6640 1 Processor -- start 2024-03-06T07:20:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.706+0000 7fceb5ad6640 1 -- start start 2024-03-06T07:20:37.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceb5ad6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb00724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceb5ad6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fceb00c67f0 0x7fceb0155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceb5ad6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb0152cb0 con 0x7fceb0072a10 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceb5ad6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb00c38b0 con 0x7fceb00c67f0 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceb5ad6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb01594f0 con 0x7fceb01528d0 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceaffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fceb00c67f0 0x7fceb0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fcea6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fceb0072a10 0x7fceb0155220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44348/0 (socket says 172.21.15.26:44348) 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fcea6ffd640 1 -- 172.21.15.26:0/521432561 learned_addr learned my addr 172.21.15.26:0/521432561 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceaf7fe640 1 --2- 172.21.15.26:0/521432561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb00724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceaffff640 1 -- 172.21.15.26:0/521432561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 msgr2=0x7fceb00724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceaffff640 1 --2- 172.21.15.26:0/521432561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb00724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceaffff640 1 -- 172.21.15.26:0/521432561 >> v1:172.21.15.26:6789/0 conn(0x7fceb0072a10 legacy=0x7fceb0155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.707+0000 7fceaffff640 1 -- 172.21.15.26:0/521432561 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00a7f70 con 0x7fceb00c67f0 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.708+0000 7fceaffff640 1 --2- 172.21.15.26:0/521432561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb00c67f0 0x7fceb0155930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009d90 tx=0x7fcea802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1519468808a2960 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.708+0000 7fceaeffd640 1 -- 172.21.15.26:0/521432561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8039070 con 0x7fceb00c67f0 2024-03-06T07:20:37.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.708+0000 7fceaeffd640 1 -- 172.21.15.26:0/521432561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea802cce0 con 0x7fceb00c67f0 2024-03-06T07:20:37.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.708+0000 7fceaeffd640 1 -- 172.21.15.26:0/521432561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8034660 con 0x7fceb00c67f0 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 -- 172.21.15.26:0/521432561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb00c67f0 msgr2=0x7fceb0155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/521432561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb00c67f0 0x7fceb0155930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009d90 tx=0x7fcea802bf30 comp rx=0 tx=0).stop 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 -- 172.21.15.26:0/521432561 shutdown_connections 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/521432561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb00c67f0 0x7fceb0155930 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/521432561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb00724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 -- 172.21.15.26:0/521432561 >> 172.21.15.26:0/521432561 conn(0x7fceb00aa6a0 msgr2=0x7fceb00aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 -- 172.21.15.26:0/521432561 shutdown_connections 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.709+0000 7fceb5ad6640 1 -- 172.21.15.26:0/521432561 wait complete. 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.710+0000 7fceb5ad6640 1 Processor -- start 2024-03-06T07:20:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.710+0000 7fceb5ad6640 1 -- start start 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb0072a10 0x7fceb00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceb00c67f0 0x7fceb00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb0147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb01591a0 con 0x7fceb0072a10 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb00a8250 con 0x7fceb00c67f0 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb00c4ef0 con 0x7fceb01528d0 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb0072a10 0x7fceb00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb0072a10 0x7fceb00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45718/0 (socket says 172.21.15.26:45718) 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 -- 172.21.15.26:0/933118434 learned_addr learned my addr 172.21.15.26:0/933118434 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fcea6ffd640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceb00c67f0 0x7fceb00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaffff640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb0147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 -- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 msgr2=0x7fceb0147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb0147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 -- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceb00c67f0 msgr2=0x7fceb00707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceb00c67f0 0x7fceb00707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 -- 172.21.15.26:0/933118434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb0146660 con 0x7fceb0072a10 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fcea6ffd640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceb00c67f0 0x7fceb00707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaffff640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb0147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceaf7fe640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb0072a10 0x7fceb00702b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcea000e8a0 tx=0x7fcea000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceacff9640 1 -- 172.21.15.26:0/933118434 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0017070 con 0x7fceb0072a10 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceacff9640 1 -- 172.21.15.26:0/933118434 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea000c940 con 0x7fceb0072a10 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb0158470 con 0x7fceb0072a10 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceacff9640 1 -- 172.21.15.26:0/933118434 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea000cd30 con 0x7fceb0072a10 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.711+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb0158750 con 0x7fceb0072a10 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.712+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb0075300 con 0x7fceb0072a10 2024-03-06T07:20:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.713+0000 7fceacff9640 1 -- 172.21.15.26:0/933118434 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcea001a070 con 0x7fceb0072a10 2024-03-06T07:20:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.713+0000 7fceacff9640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce9006b500 0x7fce9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.713+0000 7fceacff9640 1 -- 172.21.15.26:0/933118434 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcea008a3b0 con 0x7fceb0072a10 2024-03-06T07:20:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.713+0000 7fcea6ffd640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce9006b500 0x7fce9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.714+0000 7fcea6ffd640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce9006b500 0x7fce9006d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fceb006cdd0 tx=0x7fce98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.718+0000 7fceacff9640 1 -- 172.21.15.26:0/933118434 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fceb0075300 con 0x7fceb0072a10 2024-03-06T07:20:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.899+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fceb004f330 con 0x7fceb0072a10 2024-03-06T07:20:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.899+0000 7fceacff9640 1 -- 172.21.15.26:0/933118434 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcea0058ef0 con 0x7fceb0072a10 2024-03-06T07:20:37.901 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce9006b500 msgr2=0x7fce9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce9006b500 0x7fce9006d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fceb006cdd0 tx=0x7fce98009040 comp rx=0 tx=0).stop 2024-03-06T07:20:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb0072a10 msgr2=0x7fceb00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb0072a10 0x7fceb00702b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcea000e8a0 tx=0x7fcea000ed60 comp rx=0 tx=0).stop 2024-03-06T07:20:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 shutdown_connections 2024-03-06T07:20:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce9006b500 0x7fce9006d9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fceb01528d0 0x7fceb0147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceb00c67f0 0x7fceb00707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 --2- 172.21.15.26:0/933118434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb0072a10 0x7fceb00702b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 >> 172.21.15.26:0/933118434 conn(0x7fceb00aa6a0 msgr2=0x7fceb006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 shutdown_connections 2024-03-06T07:20:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:37.901+0000 7fceb5ad6640 1 -- 172.21.15.26:0/933118434 wait complete. 2024-03-06T07:20:37.926 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. 2024-03-06T07:20:37.926 INFO:tasks.cephfs_test_runner:test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:20:37.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:38.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.026+0000 7f3563f3e640 1 Processor -- start 2024-03-06T07:20:38.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.026+0000 7f3563f3e640 1 -- start start 2024-03-06T07:20:38.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3563f3e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f355c0c6780 0x7f355c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3563f3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c072630 con 0x7f355c06da40 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3563f3e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c072910 con 0x7f355c1592d0 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3563f3e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c06de20 con 0x7f355c0c6780 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3561cda640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f355c06da40 0x7f355c155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44354/0 (socket says 172.21.15.26:44354) 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3561cda640 1 -- 172.21.15.26:0/3302896592 learned_addr learned my addr 172.21.15.26:0/3302896592 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f35614d9640 1 --2- 172.21.15.26:0/3302896592 >> v2:172.21.15.117:3300/0 conn(0x7f355c0c6780 0x7f355c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3560cd8640 1 -- 172.21.15.26:0/3302896592 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423875751 0 0) 0x7f355c072630 con 0x7f355c06da40 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3560cd8640 1 -- 172.21.15.26:0/3302896592 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3548003520 con 0x7f355c06da40 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3560cd8640 1 -- 172.21.15.26:0/3302896592 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660229445 0 0) 0x7f355c072910 con 0x7f355c1592d0 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f3560cd8640 1 -- 172.21.15.26:0/3302896592 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f355c072630 con 0x7f355c1592d0 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f35614d9640 1 -- 172.21.15.26:0/3302896592 >> v1:172.21.15.117:6789/0 conn(0x7f355c1592d0 legacy=0x7f355c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f35614d9640 1 -- 172.21.15.26:0/3302896592 >> v1:172.21.15.26:6789/0 conn(0x7f355c06da40 legacy=0x7f355c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.027+0000 7f35614d9640 1 -- 172.21.15.26:0/3302896592 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c072bf0 con 0x7f355c0c6780 2024-03-06T07:20:38.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.028+0000 7f35614d9640 1 --2- 172.21.15.26:0/3302896592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c0c6780 0x7f355c158d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f354c0097a0 tx=0x7f354c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6124ca75e83e525a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.028+0000 7f3560cd8640 1 -- 172.21.15.26:0/3302896592 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c035020 con 0x7f355c0c6780 2024-03-06T07:20:38.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.028+0000 7f3560cd8640 1 -- 172.21.15.26:0/3302896592 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c031830 con 0x7f355c0c6780 2024-03-06T07:20:38.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.028+0000 7f3560cd8640 1 -- 172.21.15.26:0/3302896592 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c031c00 con 0x7f355c0c6780 2024-03-06T07:20:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.029+0000 7f3563f3e640 1 -- 172.21.15.26:0/3302896592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c0c6780 msgr2=0x7f355c158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.029+0000 7f3563f3e640 1 --2- 172.21.15.26:0/3302896592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c0c6780 0x7f355c158d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f354c0097a0 tx=0x7f354c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.029+0000 7f3563f3e640 1 -- 172.21.15.26:0/3302896592 shutdown_connections 2024-03-06T07:20:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.029+0000 7f3563f3e640 1 --2- 172.21.15.26:0/3302896592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c0c6780 0x7f355c158d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.029+0000 7f3563f3e640 1 -- 172.21.15.26:0/3302896592 >> 172.21.15.26:0/3302896592 conn(0x7f355c0aa630 msgr2=0x7f355c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.029+0000 7f3563f3e640 1 -- 172.21.15.26:0/3302896592 shutdown_connections 2024-03-06T07:20:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.029+0000 7f3563f3e640 1 -- 172.21.15.26:0/3302896592 wait complete. 2024-03-06T07:20:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 Processor -- start 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 -- start start 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c06da40 0x7f355c1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f355c0c6780 0x7f355c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f355c1592d0 0x7f355c14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0c3b10 con 0x7f355c1592d0 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0c3990 con 0x7f355c06da40 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.030+0000 7f3563f3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0c7380 con 0x7f355c0c6780 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f355c1592d0 0x7f355c14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f355c1592d0 0x7f355c14e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45726/0 (socket says 172.21.15.26:45726) 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 -- 172.21.15.26:0/517452031 learned_addr learned my addr 172.21.15.26:0/517452031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f3561cda640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c06da40 0x7f355c1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35614d9640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f355c0c6780 0x7f355c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 -- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f355c0c6780 msgr2=0x7f355c151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f355c0c6780 0x7f355c151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 -- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c06da40 msgr2=0x7f355c1523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c06da40 0x7f355c1523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 -- 172.21.15.26:0/517452031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c152900 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35624db640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f355c1592d0 0x7f355c14e7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f355400c3c0 tx=0x7f3554001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35527fc640 1 -- 172.21.15.26:0/517452031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554017070 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35527fc640 1 -- 172.21.15.26:0/517452031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35540052d0 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c1ee8d0 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f35527fc640 1 -- 172.21.15.26:0/517452031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554003810 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.031+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c06fc10 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.032+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c04f330 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.033+0000 7f35527fc640 1 -- 172.21.15.26:0/517452031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f355401a070 con 0x7f355c1592d0 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.033+0000 7f35527fc640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f354006b500 0x7f354006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.033+0000 7f35527fc640 1 -- 172.21.15.26:0/517452031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3554089710 con 0x7f355c1592d0 2024-03-06T07:20:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.033+0000 7f3561cda640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f354006b500 0x7f354006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.033+0000 7f3561cda640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f354006b500 0x7f354006d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3558001560 tx=0x7f3558001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.037+0000 7f35527fc640 1 -- 172.21.15.26:0/517452031 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3554057c10 con 0x7f355c1592d0 2024-03-06T07:20:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.224+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f355c073240 con 0x7f355c1592d0 2024-03-06T07:20:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.225+0000 7f35527fc640 1 -- 172.21.15.26:0/517452031 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f355405bcb0 con 0x7f355c1592d0 2024-03-06T07:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:38.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f354006b500 msgr2=0x7f354006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f354006b500 0x7f354006d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3558001560 tx=0x7f3558001a90 comp rx=0 tx=0).stop 2024-03-06T07:20:38.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f355c1592d0 msgr2=0x7f355c14e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f355c1592d0 0x7f355c14e7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f355400c3c0 tx=0x7f3554001e70 comp rx=0 tx=0).stop 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 shutdown_connections 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f354006b500 0x7f354006d9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f355c1592d0 0x7f355c14e7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f355c0c6780 0x7f355c151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 --2- 172.21.15.26:0/517452031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f355c06da40 0x7f355c1523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 >> 172.21.15.26:0/517452031 conn(0x7f355c0aa630 msgr2=0x7f355c154500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.227+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 shutdown_connections 2024-03-06T07:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.228+0000 7f3563f3e640 1 -- 172.21.15.26:0/517452031 wait complete. 2024-03-06T07:20:38.240 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:38.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.344+0000 7f2d9fbe9640 1 Processor -- start 2024-03-06T07:20:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.344+0000 7f2d9fbe9640 1 -- start start 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9fbe9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2d980be150 0x7f2d98155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9fbe9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d980c75e0 con 0x7f2d9814bcc0 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9fbe9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d980c78c0 con 0x7f2d98158a10 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9fbe9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d980c7f10 con 0x7f2d980be150 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9d985640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2d9814bcc0 0x7f2d980c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44364/0 (socket says 172.21.15.26:44364) 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9d985640 1 -- 172.21.15.26:0/1750571765 learned_addr learned my addr 172.21.15.26:0/1750571765 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9d184640 1 --2- 172.21.15.26:0/1750571765 >> v2:172.21.15.117:3300/0 conn(0x7f2d980be150 0x7f2d98155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9c983640 1 -- 172.21.15.26:0/1750571765 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904840510 0 0) 0x7f2d980c75e0 con 0x7f2d9814bcc0 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9c983640 1 -- 172.21.15.26:0/1750571765 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d84003520 con 0x7f2d9814bcc0 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9d184640 1 -- 172.21.15.26:0/1750571765 >> v1:172.21.15.117:6789/0 conn(0x7f2d98158a10 legacy=0x7f2d98158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9d184640 1 -- 172.21.15.26:0/1750571765 >> v1:172.21.15.26:6789/0 conn(0x7f2d9814bcc0 legacy=0x7f2d980c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.345+0000 7f2d9d184640 1 -- 172.21.15.26:0/1750571765 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d980c7ba0 con 0x7f2d980be150 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.346+0000 7f2d9d184640 1 --2- 172.21.15.26:0/1750571765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d980be150 0x7f2d98155a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d880097a0 tx=0x7f2d8802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b635250ddd6f4d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.346+0000 7f2d9c983640 1 -- 172.21.15.26:0/1750571765 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8802f810 con 0x7f2d980be150 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.346+0000 7f2d9c983640 1 -- 172.21.15.26:0/1750571765 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8802f9b0 con 0x7f2d980be150 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.346+0000 7f2d9c983640 1 -- 172.21.15.26:0/1750571765 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8802fd60 con 0x7f2d980be150 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.347+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/1750571765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d980be150 msgr2=0x7f2d98155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.347+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/1750571765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d980be150 0x7f2d98155a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d880097a0 tx=0x7f2d8802b9f0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.347+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/1750571765 shutdown_connections 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.347+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/1750571765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d980be150 0x7f2d98155a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.347+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/1750571765 >> 172.21.15.26:0/1750571765 conn(0x7f2d980ac600 msgr2=0x7f2d980aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.347+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/1750571765 shutdown_connections 2024-03-06T07:20:38.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.347+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/1750571765 wait complete. 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.348+0000 7f2d9fbe9640 1 Processor -- start 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.348+0000 7f2d9fbe9640 1 -- start start 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.348+0000 7f2d9fbe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d980be150 0x7f2d9814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.348+0000 7f2d9fbe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d9814bcc0 0x7f2d9814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.348+0000 7f2d9fbe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d98158a10 0x7f2d9814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9fbe9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d980c5ae0 con 0x7f2d980be150 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9fbe9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9806d1d0 con 0x7f2d9814bcc0 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9fbe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d980c5c60 con 0x7f2d98158a10 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d980be150 0x7f2d9814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d980be150 0x7f2d9814d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45730/0 (socket says 172.21.15.26:45730) 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 -- 172.21.15.26:0/540171656 learned_addr learned my addr 172.21.15.26:0/540171656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:38.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d184640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d9814bcc0 0x7f2d9814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9e186640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d98158a10 0x7f2d9814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 -- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d98158a10 msgr2=0x7f2d9814e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d98158a10 0x7f2d9814e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 -- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d9814bcc0 msgr2=0x7f2d9814d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d9814bcc0 0x7f2d9814d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 -- 172.21.15.26:0/540171656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d981ec250 con 0x7f2d980be150 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9d985640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d980be150 0x7f2d9814d2a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400c530 tx=0x7f2d9400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d8e7fc640 1 -- 172.21.15.26:0/540171656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94017070 con 0x7f2d980be150 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d8e7fc640 1 -- 172.21.15.26:0/540171656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94005ce0 con 0x7f2d980be150 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d981de8d0 con 0x7f2d980be150 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d8e7fc640 1 -- 172.21.15.26:0/540171656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940072b0 con 0x7f2d980be150 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.349+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9814c1d0 con 0x7f2d980be150 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.350+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9806cd20 con 0x7f2d980be150 2024-03-06T07:20:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.351+0000 7f2d8e7fc640 1 -- 172.21.15.26:0/540171656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d9401a070 con 0x7f2d980be150 2024-03-06T07:20:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.351+0000 7f2d8e7fc640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.351+0000 7f2d8e7fc640 1 -- 172.21.15.26:0/540171656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2d940897c0 con 0x7f2d980be150 2024-03-06T07:20:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.351+0000 7f2d9d184640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.352+0000 7f2d9d184640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d88036000 tx=0x7f2d88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.355+0000 7f2d8e7fc640 1 -- 172.21.15.26:0/540171656 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d9806cd20 con 0x7f2d980be150 2024-03-06T07:20:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.539+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d980c8620 con 0x7f2d980be150 2024-03-06T07:20:38.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.539+0000 7f2d8e7fc640 1 -- 172.21.15.26:0/540171656 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d980c8620 con 0x7f2d980be150 2024-03-06T07:20:38.541 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d7c06b500 msgr2=0x7f2d7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d88036000 tx=0x7f2d88008030 comp rx=0 tx=0).stop 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d980be150 msgr2=0x7f2d9814d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d980be150 0x7f2d9814d2a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400c530 tx=0x7f2d9400c9f0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 shutdown_connections 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d98158a10 0x7f2d9814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d9814bcc0 0x7f2d9814d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 --2- 172.21.15.26:0/540171656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d980be150 0x7f2d9814d2a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 >> 172.21.15.26:0/540171656 conn(0x7f2d980ac600 msgr2=0x7f2d98158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 shutdown_connections 2024-03-06T07:20:38.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.541+0000 7f2d9fbe9640 1 -- 172.21.15.26:0/540171656 wait complete. 2024-03-06T07:20:38.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.656+0000 7fb8cc465640 1 Processor -- start 2024-03-06T07:20:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.657+0000 7fb8cc465640 1 -- start start 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.657+0000 7fb8cc465640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8c406d750 0x7fb8c40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.657+0000 7fb8cc465640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c40c3a70 0x7fb8c4072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.657+0000 7fb8cc465640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c4156060 con 0x7fb8c406db30 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.657+0000 7fb8cc465640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c4155b30 con 0x7fb8c406d750 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.657+0000 7fb8cc465640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40a8430 con 0x7fb8c40c3a70 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c40c3a70 0x7fb8c4072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c40c3a70 0x7fb8c4072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:49396/0 (socket says 172.21.15.26:49396) 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c9a00640 1 -- 172.21.15.26:0/893119824 learned_addr learned my addr 172.21.15.26:0/893119824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8ca201640 1 --2- 172.21.15.26:0/893119824 >> v2:172.21.15.117:3300/0 conn(0x7fb8c406d750 0x7fb8c40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c9a00640 1 -- 172.21.15.26:0/893119824 >> v2:172.21.15.117:3300/0 conn(0x7fb8c406d750 msgr2=0x7fb8c40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c9a00640 1 --2- 172.21.15.26:0/893119824 >> v2:172.21.15.117:3300/0 conn(0x7fb8c406d750 0x7fb8c40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c9a00640 1 -- 172.21.15.26:0/893119824 >> v1:172.21.15.117:6789/0 conn(0x7fb8c406db30 legacy=0x7fb8c40c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c91ff640 1 -- 172.21.15.26:0/893119824 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092491484 0 0) 0x7fb8c4156060 con 0x7fb8c406db30 2024-03-06T07:20:38.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8c9a00640 1 -- 172.21.15.26:0/893119824 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4155d80 con 0x7fb8c40c3a70 2024-03-06T07:20:38.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.658+0000 7fb8ca201640 1 --2- 172.21.15.26:0/893119824 >> v2:172.21.15.117:3300/0 conn(0x7fb8c406d750 0x7fb8c40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.659+0000 7fb8c9a00640 1 --2- 172.21.15.26:0/893119824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c40c3a70 0x7fb8c4072680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009870 tx=0x7fb8b802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d608bfa25ad8a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.659+0000 7fb8c91ff640 1 -- 172.21.15.26:0/893119824 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80317c0 con 0x7fb8c40c3a70 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.659+0000 7fb8c91ff640 1 -- 172.21.15.26:0/893119824 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b8037070 con 0x7fb8c40c3a70 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.659+0000 7fb8c91ff640 1 -- 172.21.15.26:0/893119824 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8031b80 con 0x7fb8c40c3a70 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 -- 172.21.15.26:0/893119824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c40c3a70 msgr2=0x7fb8c4072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 --2- 172.21.15.26:0/893119824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c40c3a70 0x7fb8c4072680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009870 tx=0x7fb8b802b860 comp rx=0 tx=0).stop 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 -- 172.21.15.26:0/893119824 shutdown_connections 2024-03-06T07:20:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 --2- 172.21.15.26:0/893119824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c40c3a70 0x7fb8c4072680 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 --2- 172.21.15.26:0/893119824 >> v2:172.21.15.117:3300/0 conn(0x7fb8c406d750 0x7fb8c40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 -- 172.21.15.26:0/893119824 >> 172.21.15.26:0/893119824 conn(0x7fb8c40c5560 msgr2=0x7fb8c40c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 -- 172.21.15.26:0/893119824 shutdown_connections 2024-03-06T07:20:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.660+0000 7fb8cc465640 1 -- 172.21.15.26:0/893119824 wait complete. 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.661+0000 7fb8cc465640 1 Processor -- start 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.661+0000 7fb8cc465640 1 -- start start 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8cc465640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c406d750 0x7fb8c406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8cc465640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c406db30 0x7fb8c40ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8cc465640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8c40c3a70 0x7fb8c41ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8cc465640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4072d40 con 0x7fb8c40c3a70 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8cc465640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4072bc0 con 0x7fb8c406db30 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8cc465640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4072ec0 con 0x7fb8c406d750 2024-03-06T07:20:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8c40c3a70 0x7fb8c41ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8c40c3a70 0x7fb8c41ea570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45734/0 (socket says 172.21.15.26:45734) 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 -- 172.21.15.26:0/3996640438 learned_addr learned my addr 172.21.15.26:0/3996640438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8ca201640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c406d750 0x7fb8c406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8c9a00640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c406db30 0x7fb8c40ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 -- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c406d750 msgr2=0x7fb8c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c406d750 0x7fb8c406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 -- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c406db30 msgr2=0x7fb8c40ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c406db30 0x7fb8c40ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 -- 172.21.15.26:0/3996640438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c414dad0 con 0x7fb8c40c3a70 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8ca201640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c406d750 0x7fb8c406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8caa02640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8c40c3a70 0x7fb8c41ea570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400c710 tx=0x7fb8b400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.662+0000 7fb8c2ffd640 1 -- 172.21.15.26:0/3996640438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4017070 con 0x7fb8c40c3a70 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.663+0000 7fb8c2ffd640 1 -- 172.21.15.26:0/3996640438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b4003da0 con 0x7fb8c40c3a70 2024-03-06T07:20:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.663+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c41ee850 con 0x7fb8c40c3a70 2024-03-06T07:20:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.663+0000 7fb8c2ffd640 1 -- 172.21.15.26:0/3996640438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4005290 con 0x7fb8c40c3a70 2024-03-06T07:20:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.663+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c4154830 con 0x7fb8c40c3a70 2024-03-06T07:20:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.663+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c40c6670 con 0x7fb8c40c3a70 2024-03-06T07:20:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.664+0000 7fb8c2ffd640 1 -- 172.21.15.26:0/3996640438 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8b401a070 con 0x7fb8c40c3a70 2024-03-06T07:20:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.664+0000 7fb8c2ffd640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8b006b4d0 0x7fb8b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.664+0000 7fb8c2ffd640 1 -- 172.21.15.26:0/3996640438 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb8b4089a30 con 0x7fb8c40c3a70 2024-03-06T07:20:38.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.665+0000 7fb8ca201640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8b006b4d0 0x7fb8b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.665+0000 7fb8ca201640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8b006b4d0 0x7fb8b006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009560 tx=0x7fb8bc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.669+0000 7fb8c2ffd640 1 -- 172.21.15.26:0/3996640438 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb8c40c6670 con 0x7fb8c40c3a70 2024-03-06T07:20:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.849+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8c404f330 con 0x7fb8c40c3a70 2024-03-06T07:20:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.850+0000 7fb8c2ffd640 1 -- 172.21.15.26:0/3996640438 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8b400ce20 con 0x7fb8c40c3a70 2024-03-06T07:20:38.852 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8b006b4d0 msgr2=0x7fb8b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8b006b4d0 0x7fb8b006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009560 tx=0x7fb8bc009190 comp rx=0 tx=0).stop 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8c40c3a70 msgr2=0x7fb8c41ea570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8c40c3a70 0x7fb8c41ea570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400c710 tx=0x7fb8b400cbd0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 shutdown_connections 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8b006b4d0 0x7fb8b006d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8c40c3a70 0x7fb8c41ea570 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c406db30 0x7fb8c40ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 --2- 172.21.15.26:0/3996640438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c406d750 0x7fb8c406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 >> 172.21.15.26:0/3996640438 conn(0x7fb8c40c5560 msgr2=0x7fb8c40c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 shutdown_connections 2024-03-06T07:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.852+0000 7fb8cc465640 1 -- 172.21.15.26:0/3996640438 wait complete. 2024-03-06T07:20:38.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.968+0000 7f52180fb640 1 Processor -- start 2024-03-06T07:20:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.968+0000 7f52180fb640 1 -- start start 2024-03-06T07:20:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.968+0000 7f52180fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c57d0 0x7f5210155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.968+0000 7f52180fb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5210152dd0 con 0x7f52100c5370 2024-03-06T07:20:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.968+0000 7f52180fb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5210158b00 con 0x7f52101529f0 2024-03-06T07:20:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.968+0000 7f52180fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5210159710 con 0x7f52100c57d0 2024-03-06T07:20:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5215696640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52100c5370 0x7f5210155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44372/0 (socket says 172.21.15.26:44372) 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5215696640 1 -- 172.21.15.26:0/1525403277 learned_addr learned my addr 172.21.15.26:0/1525403277 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5216698640 1 --2- 172.21.15.26:0/1525403277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c57d0 0x7f5210155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5214e95640 1 -- 172.21.15.26:0/1525403277 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986800229 0 0) 0x7f5210152dd0 con 0x7f52100c5370 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5214e95640 1 -- 172.21.15.26:0/1525403277 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51fc003520 con 0x7f52100c5370 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5214e95640 1 -- 172.21.15.26:0/1525403277 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973599157 0 0) 0x7f5210158b00 con 0x7f52101529f0 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5214e95640 1 -- 172.21.15.26:0/1525403277 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5210152dd0 con 0x7f52101529f0 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5216698640 1 -- 172.21.15.26:0/1525403277 >> v1:172.21.15.117:6789/0 conn(0x7f52101529f0 legacy=0x7f52100c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5216698640 1 -- 172.21.15.26:0/1525403277 >> v1:172.21.15.26:6789/0 conn(0x7f52100c5370 legacy=0x7f5210155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.969+0000 7f5216698640 1 -- 172.21.15.26:0/1525403277 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5210158de0 con 0x7f52100c57d0 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f5216698640 1 --2- 172.21.15.26:0/1525403277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c57d0 0x7f5210155c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5208009dc0 tx=0x7f520802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b5f9896f4f4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f5214e95640 1 -- 172.21.15.26:0/1525403277 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208039070 con 0x7f52100c57d0 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f5214e95640 1 -- 172.21.15.26:0/1525403277 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520802cc40 con 0x7f52100c57d0 2024-03-06T07:20:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f5214e95640 1 -- 172.21.15.26:0/1525403277 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208034440 con 0x7f52100c57d0 2024-03-06T07:20:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f52180fb640 1 -- 172.21.15.26:0/1525403277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c57d0 msgr2=0x7f5210155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f52180fb640 1 --2- 172.21.15.26:0/1525403277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c57d0 0x7f5210155c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5208009dc0 tx=0x7f520802be90 comp rx=0 tx=0).stop 2024-03-06T07:20:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f52180fb640 1 -- 172.21.15.26:0/1525403277 shutdown_connections 2024-03-06T07:20:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f52180fb640 1 --2- 172.21.15.26:0/1525403277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c57d0 0x7f5210155c40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.970+0000 7f52180fb640 1 -- 172.21.15.26:0/1525403277 >> 172.21.15.26:0/1525403277 conn(0x7f52100ac580 msgr2=0x7f52100ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.971+0000 7f52180fb640 1 -- 172.21.15.26:0/1525403277 shutdown_connections 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.971+0000 7f52180fb640 1 -- 172.21.15.26:0/1525403277 wait complete. 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 Processor -- start 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 -- start start 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c5370 0x7f521014c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52100c57d0 0x7f5210157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52101529f0 0x7f5210148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100c68b0 con 0x7f52100c57d0 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100c9000 con 0x7f52101529f0 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f52180fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52101593a0 con 0x7f52100c5370 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f5215696640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52100c57d0 0x7f5210157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f5215696640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52100c57d0 0x7f5210157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45736/0 (socket says 172.21.15.26:45736) 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f5215696640 1 -- 172.21.15.26:0/2231002084 learned_addr learned my addr 172.21.15.26:0/2231002084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f5215e97640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c5370 0x7f521014c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.972+0000 7f5216698640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52101529f0 0x7f5210148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f5215696640 1 -- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c5370 msgr2=0x7f521014c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f5215696640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c5370 0x7f521014c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f5215696640 1 -- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52101529f0 msgr2=0x7f5210148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f5215696640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52101529f0 0x7f5210148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f5215696640 1 -- 172.21.15.26:0/2231002084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52101eb7a0 con 0x7f52100c57d0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f5215696640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52100c57d0 0x7f5210157590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f520000c530 tx=0x7f520000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f52067fc640 1 -- 172.21.15.26:0/2231002084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200017070 con 0x7f52100c57d0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f52067fc640 1 -- 172.21.15.26:0/2231002084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200005ce0 con 0x7f52100c57d0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52101eba80 con 0x7f52100c57d0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f52067fc640 1 -- 172.21.15.26:0/2231002084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200007270 con 0x7f52100c57d0 2024-03-06T07:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.973+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5210154c90 con 0x7f52100c57d0 2024-03-06T07:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.974+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521004f330 con 0x7f52100c57d0 2024-03-06T07:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.975+0000 7f52067fc640 1 -- 172.21.15.26:0/2231002084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f520001a070 con 0x7f52100c57d0 2024-03-06T07:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.975+0000 7f52067fc640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.975+0000 7f52067fc640 1 -- 172.21.15.26:0/2231002084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5200089790 con 0x7f52100c57d0 2024-03-06T07:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.975+0000 7f5215e97640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.975+0000 7f5215e97640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f520c007fd0 tx=0x7f520c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:38.979+0000 7f52067fc640 1 -- 172.21.15.26:0/2231002084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5200057d10 con 0x7f52100c57d0 2024-03-06T07:20:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.160+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52100b0940 con 0x7f52100c57d0 2024-03-06T07:20:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.160+0000 7f52067fc640 1 -- 172.21.15.26:0/2231002084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f520005bdb0 con 0x7f52100c57d0 2024-03-06T07:20:39.162 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 msgr2=0x7f51f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f520c007fd0 tx=0x7f520c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52100c57d0 msgr2=0x7f5210157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52100c57d0 0x7f5210157590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f520000c530 tx=0x7f520000c9f0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 shutdown_connections 2024-03-06T07:20:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52101529f0 0x7f5210148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52100c57d0 0x7f5210157590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 --2- 172.21.15.26:0/2231002084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52100c5370 0x7f521014c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 >> 172.21.15.26:0/2231002084 conn(0x7f52100ac580 msgr2=0x7f52100c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 shutdown_connections 2024-03-06T07:20:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.162+0000 7f52180fb640 1 -- 172.21.15.26:0/2231002084 wait complete. 2024-03-06T07:20:39.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.277+0000 7f70149d0640 1 Processor -- start 2024-03-06T07:20:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.277+0000 7f70149d0640 1 -- start start 2024-03-06T07:20:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f70149d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f70100a8290 0x7f70101539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f70149d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100c4740 0x7f70100701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f70149d0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70100c62a0 con 0x7f70100c69a0 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f70149d0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7010153f00 con 0x7f70100a8290 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f70149d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70100706e0 con 0x7f70100c4740 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f700e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f70100a8290 0x7f70101539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f700e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f70100a8290 0x7f70101539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45740/0 (socket says 172.21.15.26:45740) 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f700e59c640 1 -- 172.21.15.26:0/2794480649 learned_addr learned my addr 172.21.15.26:0/2794480649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.278+0000 7f700dd9b640 1 --2- 172.21.15.26:0/2794480649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100c4740 0x7f70100701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.279+0000 7f700e59c640 1 -- 172.21.15.26:0/2794480649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100c4740 msgr2=0x7f70100701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.279+0000 7f700e59c640 1 --2- 172.21.15.26:0/2794480649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100c4740 0x7f70100701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.279+0000 7f700e59c640 1 -- 172.21.15.26:0/2794480649 >> v1:172.21.15.117:6789/0 conn(0x7f70100c69a0 legacy=0x7f7010072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.279+0000 7f700e59c640 1 -- 172.21.15.26:0/2794480649 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70100c4c00 con 0x7f70100a8290 2024-03-06T07:20:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.279+0000 7f700e59c640 1 --2- 172.21.15.26:0/2794480649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100a8290 0x7f70101539c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7000009870 tx=0x7f700002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a277afd5b595c80e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.279+0000 7f700d59a640 1 -- 172.21.15.26:0/2794480649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000317c0 con 0x7f70100a8290 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f700d59a640 1 -- 172.21.15.26:0/2794480649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7000037070 con 0x7f70100a8290 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f700d59a640 1 -- 172.21.15.26:0/2794480649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000031b80 con 0x7f70100a8290 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 -- 172.21.15.26:0/2794480649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100a8290 msgr2=0x7f70101539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 --2- 172.21.15.26:0/2794480649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100a8290 0x7f70101539c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7000009870 tx=0x7f700002b860 comp rx=0 tx=0).stop 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 -- 172.21.15.26:0/2794480649 shutdown_connections 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 --2- 172.21.15.26:0/2794480649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100c4740 0x7f70100701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 --2- 172.21.15.26:0/2794480649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100a8290 0x7f70101539c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 -- 172.21.15.26:0/2794480649 >> 172.21.15.26:0/2794480649 conn(0x7f70100c5560 msgr2=0x7f70100c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 -- 172.21.15.26:0/2794480649 shutdown_connections 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.280+0000 7f70149d0640 1 -- 172.21.15.26:0/2794480649 wait complete. 2024-03-06T07:20:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.281+0000 7f70149d0640 1 Processor -- start 2024-03-06T07:20:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.281+0000 7f70149d0640 1 -- start start 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f70149d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100a8290 0x7f701006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f70149d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100c4740 0x7f7010146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f70149d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70100c69a0 0x7f7010147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f70149d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701006fc80 con 0x7f70100c4740 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f70149d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701006f9a0 con 0x7f70100c69a0 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f70149d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7010072d90 con 0x7f70100a8290 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100c4740 0x7f7010146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100c4740 0x7f7010146ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45746/0 (socket says 172.21.15.26:45746) 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 -- 172.21.15.26:0/3530935386 learned_addr learned my addr 172.21.15.26:0/3530935386 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700e59c640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100a8290 0x7f701006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700ed9d640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70100c69a0 0x7f7010147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 -- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100a8290 msgr2=0x7f701006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100a8290 0x7f701006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 -- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70100c69a0 msgr2=0x7f7010147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70100c69a0 0x7f7010147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.282+0000 7f700dd9b640 1 -- 172.21.15.26:0/3530935386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7010150550 con 0x7f70100c4740 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.283+0000 7f700dd9b640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100c4740 0x7f7010146ec0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f700400e9e0 tx=0x7f700400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.283+0000 7f6ffeffd640 1 -- 172.21.15.26:0/3530935386 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004017070 con 0x7f70100c4740 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.283+0000 7f6ffeffd640 1 -- 172.21.15.26:0/3530935386 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700400c9d0 con 0x7f70100c4740 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.283+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70101ee670 con 0x7f70100c4740 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.283+0000 7f6ffeffd640 1 -- 172.21.15.26:0/3530935386 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400cdc0 con 0x7f70100c4740 2024-03-06T07:20:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.283+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7010154590 con 0x7f70100c4740 2024-03-06T07:20:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.284+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701004f330 con 0x7f70100c4740 2024-03-06T07:20:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.284+0000 7f6ffeffd640 1 -- 172.21.15.26:0/3530935386 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f700401a070 con 0x7f70100c4740 2024-03-06T07:20:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.284+0000 7f6ffeffd640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ff006b4d0 0x7f6ff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.285+0000 7f6ffeffd640 1 -- 172.21.15.26:0/3530935386 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f700408a7b0 con 0x7f70100c4740 2024-03-06T07:20:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.285+0000 7f700e59c640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ff006b4d0 0x7f6ff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.285+0000 7f700e59c640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ff006b4d0 0x7f6ff006d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7000036000 tx=0x7f700003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.289+0000 7f6ffeffd640 1 -- 172.21.15.26:0/3530935386 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7004058d30 con 0x7f70100c4740 2024-03-06T07:20:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.469+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f70100c8480 con 0x7f70100c4740 2024-03-06T07:20:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.469+0000 7f6ffeffd640 1 -- 172.21.15.26:0/3530935386 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f700405cdd0 con 0x7f70100c4740 2024-03-06T07:20:39.471 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ff006b4d0 msgr2=0x7f6ff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ff006b4d0 0x7f6ff006d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7000036000 tx=0x7f700003d040 comp rx=0 tx=0).stop 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100c4740 msgr2=0x7f7010146ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100c4740 0x7f7010146ec0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f700400e9e0 tx=0x7f700400eea0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 shutdown_connections 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ff006b4d0 0x7f6ff006d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70100c69a0 0x7f7010147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70100c4740 0x7f7010146ec0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 --2- 172.21.15.26:0/3530935386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70100a8290 0x7f701006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 >> 172.21.15.26:0/3530935386 conn(0x7f70100c5560 msgr2=0x7f70101527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 shutdown_connections 2024-03-06T07:20:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.471+0000 7f70149d0640 1 -- 172.21.15.26:0/3530935386 wait complete. 2024-03-06T07:20:39.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop' 2024-03-06T07:20:39.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.584+0000 7fdd45981640 1 Processor -- start 2024-03-06T07:20:39.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.584+0000 7fdd45981640 1 -- start start 2024-03-06T07:20:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.584+0000 7fdd45981640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd45981640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd400c5330 0x7fdd400c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd45981640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd40156180 con 0x7fdd400c8a00 2024-03-06T07:20:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd45981640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd401587b0 con 0x7fdd400c5330 2024-03-06T07:20:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd45981640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd40159630 con 0x7fdd40152920 2024-03-06T07:20:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd400c5330 0x7fdd400c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd400c5330 0x7fdd400c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45760/0 (socket says 172.21.15.26:45760) 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3f7fe640 1 -- 172.21.15.26:0/394649419 learned_addr learned my addr 172.21.15.26:0/394649419 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3effd640 1 --2- 172.21.15.26:0/394649419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3f7fe640 1 -- 172.21.15.26:0/394649419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 msgr2=0x7fdd40155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3f7fe640 1 --2- 172.21.15.26:0/394649419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3f7fe640 1 -- 172.21.15.26:0/394649419 >> v1:172.21.15.117:6789/0 conn(0x7fdd400c8a00 legacy=0x7fdd40158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3e7fc640 1 -- 172.21.15.26:0/394649419 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558585283 0 0) 0x7fdd28001110 con 0x7fdd400c8a00 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.585+0000 7fdd3f7fe640 1 -- 172.21.15.26:0/394649419 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400c7760 con 0x7fdd400c5330 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd3f7fe640 1 --2- 172.21.15.26:0/394649419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c5330 0x7fdd400c57c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009e50 tx=0x7fdd3802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b5f940b12860ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd3e7fc640 1 -- 172.21.15.26:0/394649419 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3802ca70 con 0x7fdd400c5330 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd3e7fc640 1 -- 172.21.15.26:0/394649419 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3802cc10 con 0x7fdd400c5330 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd3e7fc640 1 -- 172.21.15.26:0/394649419 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd380343f0 con 0x7fdd400c5330 2024-03-06T07:20:39.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd45981640 1 -- 172.21.15.26:0/394649419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c5330 msgr2=0x7fdd400c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd45981640 1 --2- 172.21.15.26:0/394649419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c5330 0x7fdd400c57c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009e50 tx=0x7fdd3802bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd45981640 1 -- 172.21.15.26:0/394649419 shutdown_connections 2024-03-06T07:20:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd45981640 1 --2- 172.21.15.26:0/394649419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c5330 0x7fdd400c57c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd45981640 1 --2- 172.21.15.26:0/394649419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.586+0000 7fdd45981640 1 -- 172.21.15.26:0/394649419 >> 172.21.15.26:0/394649419 conn(0x7fdd400ac560 msgr2=0x7fdd400ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.587+0000 7fdd45981640 1 -- 172.21.15.26:0/394649419 shutdown_connections 2024-03-06T07:20:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.587+0000 7fdd45981640 1 -- 172.21.15.26:0/394649419 wait complete. 2024-03-06T07:20:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 Processor -- start 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 -- start start 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd400c5330 0x7fdd4014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c8a00 0x7fdd40148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd40158a00 con 0x7fdd400c8a00 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd400c7d20 con 0x7fdd400c5330 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd45981640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4006d030 con 0x7fdd40152920 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.588+0000 7fdd367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c8a00 0x7fdd40148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c8a00 0x7fdd40148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45772/0 (socket says 172.21.15.26:45772) 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 -- 172.21.15.26:0/1330475777 learned_addr learned my addr 172.21.15.26:0/1330475777 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd3f7fe640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd3effd640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd400c5330 0x7fdd4014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 -- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 msgr2=0x7fdd40149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 -- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd400c5330 msgr2=0x7fdd4014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd400c5330 0x7fdd4014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 -- 172.21.15.26:0/1330475777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd401497c0 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd367fc640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c8a00 0x7fdd40148d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800c410 tx=0x7fdd2800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd3cff9640 1 -- 172.21.15.26:0/1330475777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28017070 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd3cff9640 1 -- 172.21.15.26:0/1330475777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd280035f0 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd401ef170 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd3cff9640 1 -- 172.21.15.26:0/1330475777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd280039e0 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.589+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd40155690 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.590+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd400c7390 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.591+0000 7fdd3cff9640 1 -- 172.21.15.26:0/1330475777 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd2801a070 con 0x7fdd400c8a00 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.591+0000 7fdd3cff9640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd2006b500 0x7fdd2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.591+0000 7fdd3cff9640 1 -- 172.21.15.26:0/1330475777 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdd280898a0 con 0x7fdd400c8a00 2024-03-06T07:20:39.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.591+0000 7fdd3effd640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd2006b500 0x7fdd2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.592+0000 7fdd3effd640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd2006b500 0x7fdd2006d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009aa0 tx=0x7fdd30009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.595+0000 7fdd3cff9640 1 -- 172.21.15.26:0/1330475777 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdd400c7390 con 0x7fdd400c8a00 2024-03-06T07:20:39.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:39.846+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]} v 0) v1 -- 0x7fdd4004f330 con 0x7fdd400c8a00 2024-03-06T07:20:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.700+0000 7fdd3cff9640 1 -- 172.21.15.26:0/1330475777 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]}]=0 v54) v1 ==== 146+0+0 (secure 0 0 0) 0x7fdd40156180 con 0x7fdd400c8a00 2024-03-06T07:20:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd2006b500 msgr2=0x7fdd2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd2006b500 0x7fdd2006d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009aa0 tx=0x7fdd30009190 comp rx=0 tx=0).stop 2024-03-06T07:20:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c8a00 msgr2=0x7fdd40148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c8a00 0x7fdd40148d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800c410 tx=0x7fdd2800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 shutdown_connections 2024-03-06T07:20:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd2006b500 0x7fdd2006d9c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd40152920 0x7fdd40149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd400c8a00 0x7fdd40148d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 --2- 172.21.15.26:0/1330475777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd400c5330 0x7fdd4014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 >> 172.21.15.26:0/1330475777 conn(0x7fdd400ac560 msgr2=0x7fdd40151880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.703+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 shutdown_connections 2024-03-06T07:20:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.704+0000 7fdd45981640 1 -- 172.21.15.26:0/1330475777 wait complete. 2024-03-06T07:20:40.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:20:40.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.829+0000 7f983318c640 1 Processor -- start 2024-03-06T07:20:40.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.829+0000 7f983318c640 1 -- start start 2024-03-06T07:20:40.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f983318c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f982c06b3a0 0x7f982c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f983318c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c15fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f983318c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982c06b800 con 0x7f982c06afc0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f983318c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982c158790 con 0x7f982c06b3a0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f983318c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0c5840 con 0x7f982c159160 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f982bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f982c06b3a0 0x7f982c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f9830f28640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f982c06afc0 0x7f982c0c2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44380/0 (socket says 172.21.15.26:44380) 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f9830f28640 1 -- 172.21.15.26:0/312751707 learned_addr learned my addr 172.21.15.26:0/312751707 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f9831729640 1 --2- 172.21.15.26:0/312751707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c15fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f982bfff640 1 -- 172.21.15.26:0/312751707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 msgr2=0x7f982c15fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f982bfff640 1 --2- 172.21.15.26:0/312751707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c15fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f982bfff640 1 -- 172.21.15.26:0/312751707 >> v1:172.21.15.26:6789/0 conn(0x7f982c06afc0 legacy=0x7f982c0c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f982bfff640 1 -- 172.21.15.26:0/312751707 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0c5b20 con 0x7f982c06b3a0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.830+0000 7f982b7fe640 1 -- 172.21.15.26:0/312751707 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263248727 0 0) 0x7f982c06b800 con 0x7f982c06afc0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.831+0000 7f982bfff640 1 --2- 172.21.15.26:0/312751707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c158c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f98180097a0 tx=0x7f981802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8412f0fbdecd319 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.831+0000 7f982b7fe640 1 -- 172.21.15.26:0/312751707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180315a0 con 0x7f982c06b3a0 2024-03-06T07:20:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.831+0000 7f982b7fe640 1 -- 172.21.15.26:0/312751707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9818037070 con 0x7f982c06b3a0 2024-03-06T07:20:40.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.831+0000 7f982b7fe640 1 -- 172.21.15.26:0/312751707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818031960 con 0x7f982c06b3a0 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.831+0000 7f983318c640 1 -- 172.21.15.26:0/312751707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 msgr2=0x7f982c158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.831+0000 7f983318c640 1 --2- 172.21.15.26:0/312751707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c158c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f98180097a0 tx=0x7f981802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.832+0000 7f983318c640 1 -- 172.21.15.26:0/312751707 shutdown_connections 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.832+0000 7f983318c640 1 --2- 172.21.15.26:0/312751707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c158c20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.832+0000 7f983318c640 1 --2- 172.21.15.26:0/312751707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c15fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.832+0000 7f983318c640 1 -- 172.21.15.26:0/312751707 >> 172.21.15.26:0/312751707 conn(0x7f982c0c4220 msgr2=0x7f982c0c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.832+0000 7f983318c640 1 -- 172.21.15.26:0/312751707 shutdown_connections 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.832+0000 7f983318c640 1 -- 172.21.15.26:0/312751707 wait complete. 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 Processor -- start 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 -- start start 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f982c06afc0 0x7f982c14d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c1eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c1eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c1595f0 con 0x7f982c06b3a0 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0c60e0 con 0x7f982c06afc0 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.833+0000 7f983318c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982c155220 con 0x7f982c159160 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c1eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c1eadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45792/0 (socket says 172.21.15.26:45792) 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 -- 172.21.15.26:0/1147472956 learned_addr learned my addr 172.21.15.26:0/1147472956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f9831729640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c1eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f9830f28640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f982c06afc0 0x7f982c14d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 -- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 msgr2=0x7f982c1eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c1eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 -- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f982c06afc0 msgr2=0x7f982c14d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f982c06afc0 0x7f982c14d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 -- 172.21.15.26:0/1147472956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c1ee3b0 con 0x7f982c06b3a0 2024-03-06T07:20:40.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f982bfff640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c1eadc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f98180096f0 tx=0x7f98180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f98297fa640 1 -- 172.21.15.26:0/1147472956 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818032630 con 0x7f982c06b3a0 2024-03-06T07:20:40.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f98297fa640 1 -- 172.21.15.26:0/1147472956 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180327f0 con 0x7f982c06b3a0 2024-03-06T07:20:40.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982c0a9080 con 0x7f982c06b3a0 2024-03-06T07:20:40.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f98297fa640 1 -- 172.21.15.26:0/1147472956 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818032be0 con 0x7f982c06b3a0 2024-03-06T07:20:40.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.834+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982c1e15d0 con 0x7f982c06b3a0 2024-03-06T07:20:40.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.835+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982c04f330 con 0x7f982c06b3a0 2024-03-06T07:20:40.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.836+0000 7f98297fa640 1 -- 172.21.15.26:0/1147472956 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f981803b3f0 con 0x7f982c06b3a0 2024-03-06T07:20:40.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.836+0000 7f98297fa640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f981006b4d0 0x7f981006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.836+0000 7f98297fa640 1 -- 172.21.15.26:0/1147472956 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f981802f030 con 0x7f982c06b3a0 2024-03-06T07:20:40.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.836+0000 7f9830f28640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f981006b4d0 0x7f981006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.837+0000 7f9830f28640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f981006b4d0 0x7f981006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9820000f60 tx=0x7f9820007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:40.840+0000 7f98297fa640 1 -- 172.21.15.26:0/1147472956 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9818040070 con 0x7f982c06b3a0 2024-03-06T07:20:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.020+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f982c0c6730 con 0x7f982c06b3a0 2024-03-06T07:20:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.023+0000 7f98297fa640 1 -- 172.21.15.26:0/1147472956 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v11)=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7f981807a680 con 0x7f982c06b3a0 2024-03-06T07:20:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.025+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f981006b4d0 msgr2=0x7f981006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.025+0000 7f983318c640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f981006b4d0 0x7f981006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9820000f60 tx=0x7f9820007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.025+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 msgr2=0x7f982c1eadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.025+0000 7f983318c640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c1eadc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f98180096f0 tx=0x7f98180046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 shutdown_connections 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f981006b4d0 0x7f981006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f982c159160 0x7f982c1eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c06b3a0 0x7f982c1eadc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 --2- 172.21.15.26:0/1147472956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f982c06afc0 0x7f982c14d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 >> 172.21.15.26:0/1147472956 conn(0x7f982c0c4220 msgr2=0x7f982c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 shutdown_connections 2024-03-06T07:20:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.026+0000 7f983318c640 1 -- 172.21.15.26:0/1147472956 wait complete. 2024-03-06T07:20:41.038 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:20:41.038 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:20:41.087 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:20:41.142 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:20:41.142 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:20:41.142 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:41.157 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:20:41.157 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:41.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:41.200 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:41.212 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:20:41.212 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:41.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.432+0000 7ff63dc81640 1 Processor -- start 2024-03-06T07:20:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.432+0000 7ff63dc81640 1 -- start start 2024-03-06T07:20:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.433+0000 7ff63dc81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6380c5480 0x7ff638155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.433+0000 7ff63dc81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6380c5cc0 con 0x7ff6381586f0 2024-03-06T07:20:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.433+0000 7ff63dc81640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6381591c0 con 0x7ff6380c5860 2024-03-06T07:20:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.433+0000 7ff63dc81640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63806df40 con 0x7ff6380c5480 2024-03-06T07:20:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.433+0000 7ff6377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6380c5480 0x7ff638155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.433+0000 7ff637fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff6381586f0 0x7ff63815f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44394/0 (socket says 172.21.15.26:44394) 2024-03-06T07:20:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.433+0000 7ff637fff640 1 -- 172.21.15.26:0/3489530511 learned_addr learned my addr 172.21.15.26:0/3489530511 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.434+0000 7ff6377fe640 1 -- 172.21.15.26:0/3489530511 >> v1:172.21.15.117:6789/0 conn(0x7ff6380c5860 legacy=0x7ff63806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.434+0000 7ff6377fe640 1 -- 172.21.15.26:0/3489530511 >> v1:172.21.15.26:6789/0 conn(0x7ff6381586f0 legacy=0x7ff63815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.434+0000 7ff6377fe640 1 -- 172.21.15.26:0/3489530511 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6381594a0 con 0x7ff6380c5480 2024-03-06T07:20:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.434+0000 7ff6377fe640 1 --2- 172.21.15.26:0/3489530511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6380c5480 0x7ff638155ba0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff628009d90 tx=0x7ff62802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade84642211ad4a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.434+0000 7ff6367fc640 1 -- 172.21.15.26:0/3489530511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628039070 con 0x7ff6380c5480 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff6367fc640 1 -- 172.21.15.26:0/3489530511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62802cc60 con 0x7ff6380c5480 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff6367fc640 1 -- 172.21.15.26:0/3489530511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628034660 con 0x7ff6380c5480 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff63dc81640 1 -- 172.21.15.26:0/3489530511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6380c5480 msgr2=0x7ff638155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3489530511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6380c5480 0x7ff638155ba0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff628009d90 tx=0x7ff62802beb0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff63dc81640 1 -- 172.21.15.26:0/3489530511 shutdown_connections 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3489530511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6380c5480 0x7ff638155ba0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff63dc81640 1 -- 172.21.15.26:0/3489530511 >> 172.21.15.26:0/3489530511 conn(0x7ff63806b1a0 msgr2=0x7ff63806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff63dc81640 1 -- 172.21.15.26:0/3489530511 shutdown_connections 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.435+0000 7ff63dc81640 1 -- 172.21.15.26:0/3489530511 wait complete. 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.436+0000 7ff63dc81640 1 Processor -- start 2024-03-06T07:20:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.436+0000 7ff63dc81640 1 -- start start 2024-03-06T07:20:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff63dc81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6380c5480 0x7ff6381ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff63dc81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6380c5860 0x7ff638153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff63dc81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6381586f0 0x7ff63814f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff63dc81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff638155680 con 0x7ff6381586f0 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff63dc81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63806fdb0 con 0x7ff6380c5860 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff63dc81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff638152a20 con 0x7ff6380c5480 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6381586f0 0x7ff63814f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6381586f0 0x7ff63814f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45806/0 (socket says 172.21.15.26:45806) 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 -- 172.21.15.26:0/3831790910 learned_addr learned my addr 172.21.15.26:0/3831790910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff636ffd640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6380c5860 0x7ff638153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff6377fe640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6380c5480 0x7ff6381ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 -- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6380c5480 msgr2=0x7ff6381ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6380c5480 0x7ff6381ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 -- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6380c5860 msgr2=0x7ff638153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6380c5860 0x7ff638153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.437+0000 7ff637fff640 1 -- 172.21.15.26:0/3831790910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380ac600 con 0x7ff6381586f0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.438+0000 7ff637fff640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6381586f0 0x7ff63814f550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c3f0 tx=0x7ff63000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.438+0000 7ff61ffff640 1 -- 172.21.15.26:0/3831790910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630017070 con 0x7ff6381586f0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.438+0000 7ff61ffff640 1 -- 172.21.15.26:0/3831790910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff630005ce0 con 0x7ff6381586f0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.438+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6380ac8e0 con 0x7ff6381586f0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.438+0000 7ff61ffff640 1 -- 172.21.15.26:0/3831790910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6300072f0 con 0x7ff6381586f0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.438+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6380acea0 con 0x7ff6381586f0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.439+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff638075300 con 0x7ff6381586f0 2024-03-06T07:20:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.439+0000 7ff61ffff640 1 -- 172.21.15.26:0/3831790910 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff63001a070 con 0x7ff6381586f0 2024-03-06T07:20:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.439+0000 7ff61ffff640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff62406b4d0 0x7ff62406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.440+0000 7ff61ffff640 1 -- 172.21.15.26:0/3831790910 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff630089790 con 0x7ff6381586f0 2024-03-06T07:20:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.440+0000 7ff6377fe640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff62406b4d0 0x7ff62406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.440+0000 7ff6377fe640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff62406b4d0 0x7ff62406d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff628036000 tx=0x7ff628030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.444+0000 7ff61ffff640 1 -- 172.21.15.26:0/3831790910 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff638075300 con 0x7ff6381586f0 2024-03-06T07:20:41.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.626+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff63804f330 con 0x7ff6381586f0 2024-03-06T07:20:41.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.627+0000 7ff61ffff640 1 -- 172.21.15.26:0/3831790910 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7ff630057c80 con 0x7ff6381586f0 2024-03-06T07:20:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:41.629 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:41.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff62406b4d0 msgr2=0x7ff62406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff62406b4d0 0x7ff62406d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff628036000 tx=0x7ff628030000 comp rx=0 tx=0).stop 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6381586f0 msgr2=0x7ff63814f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6381586f0 0x7ff63814f550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c3f0 tx=0x7ff63000c8b0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 shutdown_connections 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff62406b4d0 0x7ff62406d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6381586f0 0x7ff63814f550 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6380c5860 0x7ff638153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 --2- 172.21.15.26:0/3831790910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6380c5480 0x7ff6381ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 >> 172.21.15.26:0/3831790910 conn(0x7ff63806b1a0 msgr2=0x7ff6380c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 shutdown_connections 2024-03-06T07:20:41.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.630+0000 7ff63dc81640 1 -- 172.21.15.26:0/3831790910 wait complete. 2024-03-06T07:20:41.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:41.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.747+0000 7f493678c640 1 Processor -- start 2024-03-06T07:20:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.747+0000 7f493678c640 1 -- start start 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f493678c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49300c5370 0x7f49300c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f493678c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4930155530 0x7f49301559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f493678c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4930152dd0 con 0x7f49301529f0 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f493678c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4930159710 con 0x7f4930155530 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f493678c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c68b0 con 0x7f49300c5370 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f4934d29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4930155530 0x7f49301559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f492ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f49301529f0 0x7f49300c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44406/0 (socket says 172.21.15.26:44406) 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f492ffff640 1 -- 172.21.15.26:0/1813031114 learned_addr learned my addr 172.21.15.26:0/1813031114 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f492f7fe640 1 --2- 172.21.15.26:0/1813031114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49300c5370 0x7f49300c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f4934d29640 1 -- 172.21.15.26:0/1813031114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49300c5370 msgr2=0x7f49300c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f4934d29640 1 --2- 172.21.15.26:0/1813031114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49300c5370 0x7f49300c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f4934d29640 1 -- 172.21.15.26:0/1813031114 >> v1:172.21.15.26:6789/0 conn(0x7f49301529f0 legacy=0x7f49300c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f4934d29640 1 -- 172.21.15.26:0/1813031114 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930158b00 con 0x7f4930155530 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.748+0000 7f492effd640 1 -- 172.21.15.26:0/1813031114 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491137423 0 0) 0x7f4930152dd0 con 0x7f49301529f0 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.749+0000 7f4934d29640 1 --2- 172.21.15.26:0/1813031114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301559c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4928009d90 tx=0x7f492802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4db45475bd91868 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.749+0000 7f492effd640 1 -- 172.21.15.26:0/1813031114 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928039070 con 0x7f4930155530 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.749+0000 7f492effd640 1 -- 172.21.15.26:0/1813031114 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492802cbc0 con 0x7f4930155530 2024-03-06T07:20:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.749+0000 7f492effd640 1 -- 172.21.15.26:0/1813031114 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928034440 con 0x7f4930155530 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 -- 172.21.15.26:0/1813031114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 msgr2=0x7f49301559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 --2- 172.21.15.26:0/1813031114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301559c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4928009d90 tx=0x7f492802be10 comp rx=0 tx=0).stop 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 -- 172.21.15.26:0/1813031114 shutdown_connections 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 --2- 172.21.15.26:0/1813031114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301559c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 --2- 172.21.15.26:0/1813031114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49300c5370 0x7f49300c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 -- 172.21.15.26:0/1813031114 >> 172.21.15.26:0/1813031114 conn(0x7f49300ac580 msgr2=0x7f49300ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 -- 172.21.15.26:0/1813031114 shutdown_connections 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.750+0000 7f493678c640 1 -- 172.21.15.26:0/1813031114 wait complete. 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.751+0000 7f493678c640 1 Processor -- start 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.751+0000 7f493678c640 1 -- start start 2024-03-06T07:20:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f493678c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49300c5370 0x7f4930157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f493678c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49301529f0 0x7f49301ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f493678c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f493678c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301590c0 con 0x7f4930155530 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f493678c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c9000 con 0x7f49300c5370 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f493678c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49301593a0 con 0x7f49301529f0 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301eedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45826/0 (socket says 172.21.15.26:45826) 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 -- 172.21.15.26:0/1143276002 learned_addr learned my addr 172.21.15.26:0/1143276002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f492ffff640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49300c5370 0x7f4930157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f492f7fe640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49301529f0 0x7f49301ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 -- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49301529f0 msgr2=0x7f49301ee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49301529f0 0x7f49301ee8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 -- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49300c5370 msgr2=0x7f4930157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49300c5370 0x7f4930157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.752+0000 7f4934d29640 1 -- 172.21.15.26:0/1143276002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300bee80 con 0x7f4930155530 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.753+0000 7f4934d29640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301eedf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4928009eb0 tx=0x7f4928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.753+0000 7f492cff9640 1 -- 172.21.15.26:0/1143276002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928044020 con 0x7f4930155530 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.753+0000 7f492cff9640 1 -- 172.21.15.26:0/1143276002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492803f470 con 0x7f4930155530 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.753+0000 7f492cff9640 1 -- 172.21.15.26:0/1143276002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928039040 con 0x7f4930155530 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.753+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300c48c0 con 0x7f4930155530 2024-03-06T07:20:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.753+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49300bf120 con 0x7f4930155530 2024-03-06T07:20:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.754+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493004f330 con 0x7f4930155530 2024-03-06T07:20:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.754+0000 7f492cff9640 1 -- 172.21.15.26:0/1143276002 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4928047020 con 0x7f4930155530 2024-03-06T07:20:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.755+0000 7f492cff9640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f491c06b500 0x7f491c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.755+0000 7f492cff9640 1 -- 172.21.15.26:0/1143276002 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49280acc80 con 0x7f4930155530 2024-03-06T07:20:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.755+0000 7f492ffff640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f491c06b500 0x7f491c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.755+0000 7f492ffff640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f491c06b500 0x7f491c06d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4924000f60 tx=0x7f4924007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.759+0000 7f492cff9640 1 -- 172.21.15.26:0/1143276002 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f492804e070 con 0x7f4930155530 2024-03-06T07:20:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.943+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49300c4af0 con 0x7f4930155530 2024-03-06T07:20:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.943+0000 7f492cff9640 1 -- 172.21.15.26:0/1143276002 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f492807b170 con 0x7f4930155530 2024-03-06T07:20:41.945 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f491c06b500 msgr2=0x7f491c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f491c06b500 0x7f491c06d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4924000f60 tx=0x7f4924007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 msgr2=0x7f49301eedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301eedf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4928009eb0 tx=0x7f4928004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 shutdown_connections 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f491c06b500 0x7f491c06d9c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930155530 0x7f49301eedf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49301529f0 0x7f49301ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 --2- 172.21.15.26:0/1143276002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49300c5370 0x7f4930157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 >> 172.21.15.26:0/1143276002 conn(0x7f49300ac580 msgr2=0x7f493006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 shutdown_connections 2024-03-06T07:20:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:41.945+0000 7f493678c640 1 -- 172.21.15.26:0/1143276002 wait complete. 2024-03-06T07:20:41.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.061+0000 7f49c8cfb640 1 Processor -- start 2024-03-06T07:20:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.061+0000 7f49c8cfb640 1 -- start start 2024-03-06T07:20:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c8cfb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c40be150 0x7f49c4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c8cfb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40c75e0 con 0x7f49c4158a10 2024-03-06T07:20:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c8cfb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40c78c0 con 0x7f49c414bcc0 2024-03-06T07:20:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c8cfb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40c7f10 con 0x7f49c40be150 2024-03-06T07:20:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f49c4158a10 0x7f49c4158e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44416/0 (socket says 172.21.15.26:44416) 2024-03-06T07:20:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c2d9d640 1 -- 172.21.15.26:0/307936161 learned_addr learned my addr 172.21.15.26:0/307936161 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c1d9b640 1 --2- 172.21.15.26:0/307936161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c40be150 0x7f49c4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c159a640 1 -- 172.21.15.26:0/307936161 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873340406 0 0) 0x7f49c40c75e0 con 0x7f49c4158a10 2024-03-06T07:20:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c159a640 1 -- 172.21.15.26:0/307936161 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b0003520 con 0x7f49c4158a10 2024-03-06T07:20:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c159a640 1 -- 172.21.15.26:0/307936161 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026540545 0 0) 0x7f49c40c78c0 con 0x7f49c414bcc0 2024-03-06T07:20:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c159a640 1 -- 172.21.15.26:0/307936161 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c40c75e0 con 0x7f49c414bcc0 2024-03-06T07:20:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c1d9b640 1 -- 172.21.15.26:0/307936161 >> v1:172.21.15.117:6789/0 conn(0x7f49c414bcc0 legacy=0x7f49c40c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c1d9b640 1 -- 172.21.15.26:0/307936161 >> v1:172.21.15.26:6789/0 conn(0x7f49c4158a10 legacy=0x7f49c4158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.062+0000 7f49c1d9b640 1 -- 172.21.15.26:0/307936161 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40c7ba0 con 0x7f49c40be150 2024-03-06T07:20:42.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.063+0000 7f49c1d9b640 1 --2- 172.21.15.26:0/307936161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c40be150 0x7f49c4155a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0097d0 tx=0x7f49ac02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4501045b77e70a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.063+0000 7f49c159a640 1 -- 172.21.15.26:0/307936161 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac034070 con 0x7f49c40be150 2024-03-06T07:20:42.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.063+0000 7f49c159a640 1 -- 172.21.15.26:0/307936161 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac030880 con 0x7f49c40be150 2024-03-06T07:20:42.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.063+0000 7f49c159a640 1 -- 172.21.15.26:0/307936161 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac030c30 con 0x7f49c40be150 2024-03-06T07:20:42.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.064+0000 7f49c8cfb640 1 -- 172.21.15.26:0/307936161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c40be150 msgr2=0x7f49c4155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.064+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/307936161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c40be150 0x7f49c4155a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0097d0 tx=0x7f49ac02b9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.064+0000 7f49c8cfb640 1 -- 172.21.15.26:0/307936161 shutdown_connections 2024-03-06T07:20:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.064+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/307936161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c40be150 0x7f49c4155a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.064+0000 7f49c8cfb640 1 -- 172.21.15.26:0/307936161 >> 172.21.15.26:0/307936161 conn(0x7f49c40ac600 msgr2=0x7f49c40aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.064+0000 7f49c8cfb640 1 -- 172.21.15.26:0/307936161 shutdown_connections 2024-03-06T07:20:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.064+0000 7f49c8cfb640 1 -- 172.21.15.26:0/307936161 wait complete. 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.065+0000 7f49c8cfb640 1 Processor -- start 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.065+0000 7f49c8cfb640 1 -- start start 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c8cfb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c40be150 0x7f49c40c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c8cfb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c414bcc0 0x7f49c40c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c8cfb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c4158a10 0x7f49c40c2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c8cfb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40c5ae0 con 0x7f49c4158a10 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c8cfb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c406d1d0 con 0x7f49c40be150 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c8cfb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40c5c60 con 0x7f49c414bcc0 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c4158a10 0x7f49c40c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c4158a10 0x7f49c40c2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45838/0 (socket says 172.21.15.26:45838) 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 -- 172.21.15.26:0/3151661164 learned_addr learned my addr 172.21.15.26:0/3151661164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c259c640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c40be150 0x7f49c40c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c1d9b640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c414bcc0 0x7f49c40c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 -- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c414bcc0 msgr2=0x7f49c40c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c414bcc0 0x7f49c40c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 -- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c40be150 msgr2=0x7f49c40c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c40be150 0x7f49c40c0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.066+0000 7f49c2d9d640 1 -- 172.21.15.26:0/3151661164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40c48c0 con 0x7f49c4158a10 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.067+0000 7f49c2d9d640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c4158a10 0x7f49c40c2060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49b400cad0 tx=0x7f49b400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.067+0000 7f49baffd640 1 -- 172.21.15.26:0/3151661164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4017070 con 0x7f49c4158a10 2024-03-06T07:20:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.067+0000 7f49baffd640 1 -- 172.21.15.26:0/3151661164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400ac50 con 0x7f49c4158a10 2024-03-06T07:20:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.067+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c41eae60 con 0x7f49c4158a10 2024-03-06T07:20:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.067+0000 7f49baffd640 1 -- 172.21.15.26:0/3151661164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4012400 con 0x7f49c4158a10 2024-03-06T07:20:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.067+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c41eb420 con 0x7f49c4158a10 2024-03-06T07:20:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.068+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c41eac50 con 0x7f49c4158a10 2024-03-06T07:20:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.068+0000 7f49baffd640 1 -- 172.21.15.26:0/3151661164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49b401a070 con 0x7f49c4158a10 2024-03-06T07:20:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.069+0000 7f49baffd640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49a406b4d0 0x7f49a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.069+0000 7f49baffd640 1 -- 172.21.15.26:0/3151661164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49b408a1b0 con 0x7f49c4158a10 2024-03-06T07:20:42.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.069+0000 7f49c259c640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49a406b4d0 0x7f49a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.069+0000 7f49c259c640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49a406b4d0 0x7f49a406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007fd0 tx=0x7f49bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.073+0000 7f49baffd640 1 -- 172.21.15.26:0/3151661164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49b4058730 con 0x7f49c4158a10 2024-03-06T07:20:42.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.253+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49c404f330 con 0x7f49c4158a10 2024-03-06T07:20:42.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.253+0000 7f49baffd640 1 -- 172.21.15.26:0/3151661164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49b405c7d0 con 0x7f49c4158a10 2024-03-06T07:20:42.255 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49a406b4d0 msgr2=0x7f49a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49a406b4d0 0x7f49a406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007fd0 tx=0x7f49bc009040 comp rx=0 tx=0).stop 2024-03-06T07:20:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c4158a10 msgr2=0x7f49c40c2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c4158a10 0x7f49c40c2060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49b400cad0 tx=0x7f49b400cf90 comp rx=0 tx=0).stop 2024-03-06T07:20:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 shutdown_connections 2024-03-06T07:20:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49a406b4d0 0x7f49a406d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c4158a10 0x7f49c40c2060 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c414bcc0 0x7f49c40c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 --2- 172.21.15.26:0/3151661164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c40be150 0x7f49c40c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 >> 172.21.15.26:0/3151661164 conn(0x7f49c40ac600 msgr2=0x7f49c40c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 shutdown_connections 2024-03-06T07:20:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.255+0000 7f49c8cfb640 1 -- 172.21.15.26:0/3151661164 wait complete. 2024-03-06T07:20:42.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:42.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.370+0000 7f412094b640 1 Processor -- start 2024-03-06T07:20:42.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.370+0000 7f412094b640 1 -- start start 2024-03-06T07:20:42.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.371+0000 7f412094b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f411c14f250 0x7f411c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.371+0000 7f412094b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c14f630 con 0x7f411c158aa0 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.371+0000 7f412094b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c0c76c0 con 0x7f411c0c8960 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.371+0000 7f412094b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0c7470 con 0x7f411c14f250 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.371+0000 7f411a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f411c14f250 0x7f411c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.371+0000 7f411ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f411c158aa0 0x7f411c158ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44426/0 (socket says 172.21.15.26:44426) 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.371+0000 7f411ad9d640 1 -- 172.21.15.26:0/1046216644 learned_addr learned my addr 172.21.15.26:0/1046216644 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.372+0000 7f411a59c640 1 -- 172.21.15.26:0/1046216644 >> v1:172.21.15.117:6789/0 conn(0x7f411c0c8960 legacy=0x7f411c0c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.372+0000 7f411a59c640 1 -- 172.21.15.26:0/1046216644 >> v1:172.21.15.26:6789/0 conn(0x7f411c158aa0 legacy=0x7f411c158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.372+0000 7f4119d9b640 1 -- 172.21.15.26:0/1046216644 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422944401 0 0) 0x7f411c0c76c0 con 0x7f411c0c8960 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.372+0000 7f411a59c640 1 -- 172.21.15.26:0/1046216644 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0c79a0 con 0x7f411c14f250 2024-03-06T07:20:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.372+0000 7f411a59c640 1 --2- 172.21.15.26:0/1046216644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c14f250 0x7f411c155bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f411c0c75f0 tx=0x7f410c02b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcd91c28f04b4a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f4119d9b640 1 -- 172.21.15.26:0/1046216644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c037070 con 0x7f411c14f250 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f4119d9b640 1 -- 172.21.15.26:0/1046216644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c030710 con 0x7f411c14f250 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f4119d9b640 1 -- 172.21.15.26:0/1046216644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c02be60 con 0x7f411c14f250 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f412094b640 1 -- 172.21.15.26:0/1046216644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c14f250 msgr2=0x7f411c155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f412094b640 1 --2- 172.21.15.26:0/1046216644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c14f250 0x7f411c155bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f411c0c75f0 tx=0x7f410c02b660 comp rx=0 tx=0).stop 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f412094b640 1 -- 172.21.15.26:0/1046216644 shutdown_connections 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f412094b640 1 --2- 172.21.15.26:0/1046216644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c14f250 0x7f411c155bc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f412094b640 1 -- 172.21.15.26:0/1046216644 >> 172.21.15.26:0/1046216644 conn(0x7f411c0ac610 msgr2=0x7f411c0aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f412094b640 1 -- 172.21.15.26:0/1046216644 shutdown_connections 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.373+0000 7f412094b640 1 -- 172.21.15.26:0/1046216644 wait complete. 2024-03-06T07:20:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.374+0000 7f412094b640 1 Processor -- start 2024-03-06T07:20:42.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.374+0000 7f412094b640 1 -- start start 2024-03-06T07:20:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f412094b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c0c8960 0x7f411c14be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f412094b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c14f250 0x7f411c0c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f412094b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c158aa0 0x7f411c0c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f412094b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0be310 con 0x7f411c158aa0 2024-03-06T07:20:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f412094b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0c7f60 con 0x7f411c0c8960 2024-03-06T07:20:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f412094b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1595e0 con 0x7f411c14f250 2024-03-06T07:20:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c158aa0 0x7f411c0c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c158aa0 0x7f411c0c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45862/0 (socket says 172.21.15.26:45862) 2024-03-06T07:20:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 -- 172.21.15.26:0/3907074108 learned_addr learned my addr 172.21.15.26:0/3907074108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411a59c640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c0c8960 0x7f411c14be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f4113fff640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c14f250 0x7f411c0c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 -- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c14f250 msgr2=0x7f411c0c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c14f250 0x7f411c0c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 -- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c0c8960 msgr2=0x7f411c14be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c0c8960 0x7f411c14be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.375+0000 7f411ad9d640 1 -- 172.21.15.26:0/3907074108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c1eb840 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.376+0000 7f411ad9d640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c158aa0 0x7f411c0c3a70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f411400caf0 tx=0x7f41140020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.376+0000 7f4112ffd640 1 -- 172.21.15.26:0/3907074108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114018070 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.376+0000 7f4112ffd640 1 -- 172.21.15.26:0/3907074108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411400a760 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.376+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c0ad8b0 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.376+0000 7f4112ffd640 1 -- 172.21.15.26:0/3907074108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400ab50 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.376+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c0ade70 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.377+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c0ae070 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.377+0000 7f4112ffd640 1 -- 172.21.15.26:0/3907074108 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f411401b070 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.378+0000 7f4112ffd640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40fc06b4d0 0x7f40fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.378+0000 7f4112ffd640 1 -- 172.21.15.26:0/3907074108 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4114089c30 con 0x7f411c158aa0 2024-03-06T07:20:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.378+0000 7f411a59c640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40fc06b4d0 0x7f40fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.378+0000 7f411a59c640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40fc06b4d0 0x7f40fc06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.382+0000 7f4112ffd640 1 -- 172.21.15.26:0/3907074108 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f411c0ae070 con 0x7f411c158aa0 2024-03-06T07:20:42.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.564+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f411c0ad6a0 con 0x7f411c158aa0 2024-03-06T07:20:42.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.565+0000 7f4112ffd640 1 -- 172.21.15.26:0/3907074108 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41140581b0 con 0x7f411c158aa0 2024-03-06T07:20:42.566 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40fc06b4d0 msgr2=0x7f40fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40fc06b4d0 0x7f40fc06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c008030 comp rx=0 tx=0).stop 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c158aa0 msgr2=0x7f411c0c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c158aa0 0x7f411c0c3a70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f411400caf0 tx=0x7f41140020f0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 shutdown_connections 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40fc06b4d0 0x7f40fc06d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c158aa0 0x7f411c0c3a70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c14f250 0x7f411c0c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 --2- 172.21.15.26:0/3907074108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c0c8960 0x7f411c14be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 >> 172.21.15.26:0/3907074108 conn(0x7f411c0ac610 msgr2=0x7f411c155490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 shutdown_connections 2024-03-06T07:20:42.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.567+0000 7f412094b640 1 -- 172.21.15.26:0/3907074108 wait complete. 2024-03-06T07:20:42.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.682+0000 7f71ae55e640 1 Processor -- start 2024-03-06T07:20:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.682+0000 7f71ae55e640 1 -- start start 2024-03-06T07:20:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.683+0000 7f71ae55e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a814f580 0x7f71a8153420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.683+0000 7f71ae55e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a80c2d80 con 0x7f71a8155520 2024-03-06T07:20:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.683+0000 7f71ae55e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a80c3a00 con 0x7f71a806dca0 2024-03-06T07:20:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.683+0000 7f71ae55e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c2b30 con 0x7f71a814f580 2024-03-06T07:20:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.683+0000 7f71acafb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f71a8155520 0x7f71a81559b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44436/0 (socket says 172.21.15.26:44436) 2024-03-06T07:20:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.683+0000 7f71acafb640 1 -- 172.21.15.26:0/2663061189 learned_addr learned my addr 172.21.15.26:0/2663061189 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.683+0000 7f71a7fff640 1 --2- 172.21.15.26:0/2663061189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a814f580 0x7f71a8153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a6ffd640 1 -- 172.21.15.26:0/2663061189 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437847955 0 0) 0x7f71a80c2d80 con 0x7f71a8155520 2024-03-06T07:20:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a6ffd640 1 -- 172.21.15.26:0/2663061189 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7190003520 con 0x7f71a8155520 2024-03-06T07:20:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a6ffd640 1 -- 172.21.15.26:0/2663061189 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491997040 0 0) 0x7f71a80c3a00 con 0x7f71a806dca0 2024-03-06T07:20:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a6ffd640 1 -- 172.21.15.26:0/2663061189 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a80c2d80 con 0x7f71a806dca0 2024-03-06T07:20:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a7fff640 1 -- 172.21.15.26:0/2663061189 >> v1:172.21.15.117:6789/0 conn(0x7f71a806dca0 legacy=0x7f71a806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a7fff640 1 -- 172.21.15.26:0/2663061189 >> v1:172.21.15.26:6789/0 conn(0x7f71a8155520 legacy=0x7f71a81559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a7fff640 1 -- 172.21.15.26:0/2663061189 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a80a7170 con 0x7f71a814f580 2024-03-06T07:20:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.684+0000 7f71a7fff640 1 --2- 172.21.15.26:0/2663061189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a814f580 0x7f71a8153420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71980097d0 tx=0x7f719802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7287b1680b033dc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71a6ffd640 1 -- 172.21.15.26:0/2663061189 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198031770 con 0x7f71a814f580 2024-03-06T07:20:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71a6ffd640 1 -- 172.21.15.26:0/2663061189 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198037070 con 0x7f71a814f580 2024-03-06T07:20:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71a6ffd640 1 -- 172.21.15.26:0/2663061189 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198031b30 con 0x7f71a814f580 2024-03-06T07:20:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71ae55e640 1 -- 172.21.15.26:0/2663061189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a814f580 msgr2=0x7f71a8153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2663061189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a814f580 0x7f71a8153420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71980097d0 tx=0x7f719802b810 comp rx=0 tx=0).stop 2024-03-06T07:20:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71ae55e640 1 -- 172.21.15.26:0/2663061189 shutdown_connections 2024-03-06T07:20:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2663061189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a814f580 0x7f71a8153420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71ae55e640 1 -- 172.21.15.26:0/2663061189 >> 172.21.15.26:0/2663061189 conn(0x7f71a80a94d0 msgr2=0x7f71a80a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71ae55e640 1 -- 172.21.15.26:0/2663061189 shutdown_connections 2024-03-06T07:20:42.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.685+0000 7f71ae55e640 1 -- 172.21.15.26:0/2663061189 wait complete. 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.686+0000 7f71ae55e640 1 Processor -- start 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71ae55e640 1 -- start start 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71ae55e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a806dca0 0x7f71a81df1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71ae55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a814f580 0x7f71a814dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71ae55e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71a8155520 0x7f71a81e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71ae55e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c3060 con 0x7f71a814f580 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71ae55e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c3d70 con 0x7f71a8155520 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71ae55e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81560c0 con 0x7f71a806dca0 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a814f580 0x7f71a814dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a814f580 0x7f71a814dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45866/0 (socket says 172.21.15.26:45866) 2024-03-06T07:20:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71a77fe640 1 -- 172.21.15.26:0/2541289329 learned_addr learned my addr 172.21.15.26:0/2541289329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71acafb640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71a8155520 0x7f71a81e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.687+0000 7f71a7fff640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a806dca0 0x7f71a81df1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a77fe640 1 -- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a806dca0 msgr2=0x7f71a81df1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a77fe640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a806dca0 0x7f71a81df1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a77fe640 1 -- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71a8155520 msgr2=0x7f71a81e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a77fe640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71a8155520 0x7f71a81e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a77fe640 1 -- 172.21.15.26:0/2541289329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a81e9930 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a77fe640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a814f580 0x7f71a814dd00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c460 tx=0x7f719c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a4ff9640 1 -- 172.21.15.26:0/2541289329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c017070 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a4ff9640 1 -- 172.21.15.26:0/2541289329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c0052f0 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71a4ff9640 1 -- 172.21.15.26:0/2541289329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c003830 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a81e8c30 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.688+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a814d990 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.689+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a8071700 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.689+0000 7f71a4ff9640 1 -- 172.21.15.26:0/2541289329 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f719c01a070 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.690+0000 7f71a4ff9640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.690+0000 7f71a4ff9640 1 -- 172.21.15.26:0/2541289329 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f719c089790 con 0x7f71a814f580 2024-03-06T07:20:42.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.690+0000 7f71a7fff640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.690+0000 7f71a7fff640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f719406b4d0 0x7f719406d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7198036000 tx=0x7f719803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.694+0000 7f71a4ff9640 1 -- 172.21.15.26:0/2541289329 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f71a8071700 con 0x7f71a814f580 2024-03-06T07:20:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.873+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f71a804f330 con 0x7f71a814f580 2024-03-06T07:20:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.873+0000 7f71a4ff9640 1 -- 172.21.15.26:0/2541289329 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f719c057d10 con 0x7f71a814f580 2024-03-06T07:20:42.875 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f719406b4d0 msgr2=0x7f719406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f719406b4d0 0x7f719406d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7198036000 tx=0x7f719803d040 comp rx=0 tx=0).stop 2024-03-06T07:20:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a814f580 msgr2=0x7f71a814dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a814f580 0x7f71a814dd00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c460 tx=0x7f719c00c920 comp rx=0 tx=0).stop 2024-03-06T07:20:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 shutdown_connections 2024-03-06T07:20:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71a8155520 0x7f71a81e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a814f580 0x7f71a814dd00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 --2- 172.21.15.26:0/2541289329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71a806dca0 0x7f71a81df1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.875+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 >> 172.21.15.26:0/2541289329 conn(0x7f71a80a94d0 msgr2=0x7f71a8155990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.876+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 shutdown_connections 2024-03-06T07:20:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.876+0000 7f71ae55e640 1 -- 172.21.15.26:0/2541289329 wait complete. 2024-03-06T07:20:42.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.989+0000 7fae4ac9a640 1 Processor -- start 2024-03-06T07:20:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.989+0000 7fae4ac9a640 1 -- start start 2024-03-06T07:20:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.990+0000 7fae4ac9a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae440c69f0 0x7fae440c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.990+0000 7fae4ac9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae44155130 con 0x7fae440c8950 2024-03-06T07:20:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.990+0000 7fae4ac9a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440ad480 con 0x7fae441558d0 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.990+0000 7fae4ac9a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae44155440 con 0x7fae440c69f0 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.990+0000 7fae43fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae440c8950 0x7fae440acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44450/0 (socket says 172.21.15.26:44450) 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.990+0000 7fae43fff640 1 -- 172.21.15.26:0/2798565899 learned_addr learned my addr 172.21.15.26:0/2798565899 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.990+0000 7fae48a36640 1 --2- 172.21.15.26:0/2798565899 >> v2:172.21.15.117:3300/0 conn(0x7fae440c69f0 0x7fae440c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.991+0000 7fae437fe640 1 -- 172.21.15.26:0/2798565899 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737635019 0 0) 0x7fae38001110 con 0x7fae440c8950 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.991+0000 7fae437fe640 1 -- 172.21.15.26:0/2798565899 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae2c003520 con 0x7fae440c8950 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.991+0000 7fae48a36640 1 -- 172.21.15.26:0/2798565899 >> v1:172.21.15.117:6789/0 conn(0x7fae441558d0 legacy=0x7fae4415fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.991+0000 7fae48a36640 1 -- 172.21.15.26:0/2798565899 >> v1:172.21.15.26:6789/0 conn(0x7fae440c8950 legacy=0x7fae440acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.991+0000 7fae48a36640 1 -- 172.21.15.26:0/2798565899 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440c5d50 con 0x7fae440c69f0 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.991+0000 7fae48a36640 1 --2- 172.21.15.26:0/2798565899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c69f0 0x7fae440c6dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009d90 tx=0x7fae3c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1964e2d3f9bd41 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae437fe640 1 -- 172.21.15.26:0/2798565899 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c039070 con 0x7fae440c69f0 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae437fe640 1 -- 172.21.15.26:0/2798565899 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c02cc60 con 0x7fae440c69f0 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae437fe640 1 -- 172.21.15.26:0/2798565899 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c034640 con 0x7fae440c69f0 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae4ac9a640 1 -- 172.21.15.26:0/2798565899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c69f0 msgr2=0x7fae440c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/2798565899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c69f0 0x7fae440c6dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009d90 tx=0x7fae3c02beb0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae4ac9a640 1 -- 172.21.15.26:0/2798565899 shutdown_connections 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/2798565899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c69f0 0x7fae440c6dd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae4ac9a640 1 -- 172.21.15.26:0/2798565899 >> 172.21.15.26:0/2798565899 conn(0x7fae440cae70 msgr2=0x7fae440cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae4ac9a640 1 -- 172.21.15.26:0/2798565899 shutdown_connections 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.992+0000 7fae4ac9a640 1 -- 172.21.15.26:0/2798565899 wait complete. 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.993+0000 7fae4ac9a640 1 Processor -- start 2024-03-06T07:20:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.993+0000 7fae4ac9a640 1 -- start start 2024-03-06T07:20:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae4ac9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440c69f0 0x7fae44158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae4ac9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c8950 0x7fae44157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae4ac9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae441558d0 0x7fae441edf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae4ac9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44152a20 con 0x7fae441558d0 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae4ac9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440c6310 con 0x7fae440c8950 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae4ac9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae44152ba0 con 0x7fae440c69f0 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae49237640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae441558d0 0x7fae441edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae49237640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae441558d0 0x7fae441edf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45878/0 (socket says 172.21.15.26:45878) 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae49237640 1 -- 172.21.15.26:0/714955168 learned_addr learned my addr 172.21.15.26:0/714955168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae48a36640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440c69f0 0x7fae44158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.994+0000 7fae43fff640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c8950 0x7fae44157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae49237640 1 -- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440c69f0 msgr2=0x7fae44158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae49237640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440c69f0 0x7fae44158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae49237640 1 -- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c8950 msgr2=0x7fae44157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae49237640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c8950 0x7fae44157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae49237640 1 -- 172.21.15.26:0/714955168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440c2e10 con 0x7fae441558d0 2024-03-06T07:20:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae49237640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae441558d0 0x7fae441edf60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c400 tx=0x7fae3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae417fa640 1 -- 172.21.15.26:0/714955168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34017070 con 0x7fae441558d0 2024-03-06T07:20:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae417fa640 1 -- 172.21.15.26:0/714955168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae340052d0 con 0x7fae441558d0 2024-03-06T07:20:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440c30f0 con 0x7fae441558d0 2024-03-06T07:20:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae417fa640 1 -- 172.21.15.26:0/714955168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34003810 con 0x7fae441558d0 2024-03-06T07:20:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.995+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae44159340 con 0x7fae441558d0 2024-03-06T07:20:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.996+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4404f330 con 0x7fae441558d0 2024-03-06T07:20:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.996+0000 7fae417fa640 1 -- 172.21.15.26:0/714955168 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae3401a070 con 0x7fae441558d0 2024-03-06T07:20:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.997+0000 7fae417fa640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae3006b500 0x7fae3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.997+0000 7fae417fa640 1 -- 172.21.15.26:0/714955168 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae34089770 con 0x7fae441558d0 2024-03-06T07:20:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.997+0000 7fae48a36640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae3006b500 0x7fae3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:42.997+0000 7fae48a36640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae3006b500 0x7fae3006d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae3c036000 tx=0x7fae3c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.001+0000 7fae417fa640 1 -- 172.21.15.26:0/714955168 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae34057cf0 con 0x7fae441558d0 2024-03-06T07:20:43.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.194+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae440ca650 con 0x7fae441558d0 2024-03-06T07:20:43.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.194+0000 7fae417fa640 1 -- 172.21.15.26:0/714955168 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae3405bd90 con 0x7fae441558d0 2024-03-06T07:20:43.196 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae3006b500 msgr2=0x7fae3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae3006b500 0x7fae3006d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae3c036000 tx=0x7fae3c030000 comp rx=0 tx=0).stop 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae441558d0 msgr2=0x7fae441edf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae441558d0 0x7fae441edf60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c400 tx=0x7fae3400c8c0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 shutdown_connections 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae3006b500 0x7fae3006d9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae441558d0 0x7fae441edf60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440c8950 0x7fae44157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 --2- 172.21.15.26:0/714955168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440c69f0 0x7fae44158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 >> 172.21.15.26:0/714955168 conn(0x7fae440cae70 msgr2=0x7fae440acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 shutdown_connections 2024-03-06T07:20:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.196+0000 7fae4ac9a640 1 -- 172.21.15.26:0/714955168 wait complete. 2024-03-06T07:20:43.208 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-06T07:20:43.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:43.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-06T07:20:43.236 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:20:43.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:43.237 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-06T07:20:43.291 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:20:43.291 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:43.307 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:20:43.307 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:43.363 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:20:43.363 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:43.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.519+0000 7f4a241d6640 1 Processor -- start 2024-03-06T07:20:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.519+0000 7f4a241d6640 1 -- start start 2024-03-06T07:20:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a241d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a241d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a1c1592d0 0x7f4a1c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a241d6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c072630 con 0x7f4a1c06da40 2024-03-06T07:20:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a241d6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c06de20 con 0x7f4a1c1592d0 2024-03-06T07:20:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a241d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c072910 con 0x7f4a1c0c6780 2024-03-06T07:20:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a22773640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a1c1592d0 0x7f4a1c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a21f72640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4a1c06da40 0x7f4a1c155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44456/0 (socket says 172.21.15.26:44456) 2024-03-06T07:20:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a21f72640 1 -- 172.21.15.26:0/3471558948 learned_addr learned my addr 172.21.15.26:0/3471558948 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a21771640 1 --2- 172.21.15.26:0/3471558948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a22773640 1 -- 172.21.15.26:0/3471558948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 msgr2=0x7f4a1c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a22773640 1 --2- 172.21.15.26:0/3471558948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a22773640 1 -- 172.21.15.26:0/3471558948 >> v1:172.21.15.26:6789/0 conn(0x7f4a1c06da40 legacy=0x7f4a1c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.520+0000 7f4a22773640 1 -- 172.21.15.26:0/3471558948 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c072bf0 con 0x7f4a1c1592d0 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.521+0000 7f4a22773640 1 --2- 172.21.15.26:0/3471558948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c0c3280 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009d90 tx=0x7f4a0c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac688795539b6fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.521+0000 7f4a20f70640 1 -- 172.21.15.26:0/3471558948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c039070 con 0x7f4a1c1592d0 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.521+0000 7f4a20f70640 1 -- 172.21.15.26:0/3471558948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c02cbc0 con 0x7f4a1c1592d0 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.521+0000 7f4a20f70640 1 -- 172.21.15.26:0/3471558948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c034440 con 0x7f4a1c1592d0 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 -- 172.21.15.26:0/3471558948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 msgr2=0x7f4a1c0c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 --2- 172.21.15.26:0/3471558948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c0c3280 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009d90 tx=0x7f4a0c02be10 comp rx=0 tx=0).stop 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 -- 172.21.15.26:0/3471558948 shutdown_connections 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 --2- 172.21.15.26:0/3471558948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c0c3280 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 --2- 172.21.15.26:0/3471558948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 -- 172.21.15.26:0/3471558948 >> 172.21.15.26:0/3471558948 conn(0x7f4a1c0aa630 msgr2=0x7f4a1c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 -- 172.21.15.26:0/3471558948 shutdown_connections 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.522+0000 7f4a241d6640 1 -- 172.21.15.26:0/3471558948 wait complete. 2024-03-06T07:20:43.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.523+0000 7f4a241d6640 1 Processor -- start 2024-03-06T07:20:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.523+0000 7f4a241d6640 1 -- start start 2024-03-06T07:20:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1c06da40 0x7f4a1c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c1ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c1eb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0c3940 con 0x7f4a1c1592d0 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0c37c0 con 0x7f4a1c06da40 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0c3ac0 con 0x7f4a1c0c6780 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c1eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c1eb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45904/0 (socket says 172.21.15.26:45904) 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 -- 172.21.15.26:0/763058358 learned_addr learned my addr 172.21.15.26:0/763058358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a21f72640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1c06da40 0x7f4a1c06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a21771640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c1ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 -- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 msgr2=0x7f4a1c1ead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c1ead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 -- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1c06da40 msgr2=0x7f4a1c06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1c06da40 0x7f4a1c06f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 -- 172.21.15.26:0/763058358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0ae5c0 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a22773640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c1eb280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009eb0 tx=0x7f4a0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a1a7fc640 1 -- 172.21.15.26:0/763058358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c039050 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a1a7fc640 1 -- 172.21.15.26:0/763058358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c045070 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0ae8a0 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a1a7fc640 1 -- 172.21.15.26:0/763058358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c009450 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.524+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c06fae0 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.525+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c04f330 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.526+0000 7f4a1a7fc640 1 -- 172.21.15.26:0/763058358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a0c03e850 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.526+0000 7f4a1a7fc640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.526+0000 7f4a1a7fc640 1 -- 172.21.15.26:0/763058358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4a0c0ad330 con 0x7f4a1c1592d0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.526+0000 7f4a21f72640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.527+0000 7f4a21f72640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c070060 tx=0x7f4a14007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.530+0000 7f4a1a7fc640 1 -- 172.21.15.26:0/763058358 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4a0c07b930 con 0x7f4a1c1592d0 2024-03-06T07:20:43.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.711+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a1c073240 con 0x7f4a1c1592d0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.712+0000 7f4a1a7fc640 1 -- 172.21.15.26:0/763058358 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f4a0c07f9d0 con 0x7f4a1c1592d0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:43.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 msgr2=0x7f4a0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c070060 tx=0x7f4a14007e80 comp rx=0 tx=0).stop 2024-03-06T07:20:43.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 msgr2=0x7f4a1c1eb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c1eb280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009eb0 tx=0x7f4a0c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 shutdown_connections 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1c1592d0 0x7f4a1c1eb280 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a1c0c6780 0x7f4a1c1ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 --2- 172.21.15.26:0/763058358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1c06da40 0x7f4a1c06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 >> 172.21.15.26:0/763058358 conn(0x7f4a1c0aa630 msgr2=0x7f4a1c0c5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 shutdown_connections 2024-03-06T07:20:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.714+0000 7f4a241d6640 1 -- 172.21.15.26:0/763058358 wait complete. 2024-03-06T07:20:43.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:43.727 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.830+0000 7fecc7dd7640 1 Processor -- start 2024-03-06T07:20:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.830+0000 7fecc7dd7640 1 -- start start 2024-03-06T07:20:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc7dd7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fecc00c5990 0x7fecc00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc7dd7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fecc006c900 0x7fecc00a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc7dd7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc00c3ef0 con 0x7fecc00ab170 2024-03-06T07:20:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc7dd7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc01556d0 con 0x7fecc00c5990 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc7dd7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc01559b0 con 0x7fecc006c900 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc5b73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fecc00c5990 0x7fecc00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc5b73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fecc00c5990 0x7fecc00c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45920/0 (socket says 172.21.15.26:45920) 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc5b73640 1 -- 172.21.15.26:0/41331045 learned_addr learned my addr 172.21.15.26:0/41331045 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.831+0000 7fecc6374640 1 --2- 172.21.15.26:0/41331045 >> v2:172.21.15.117:3300/0 conn(0x7fecc006c900 0x7fecc00a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.832+0000 7fecc5b73640 1 -- 172.21.15.26:0/41331045 >> v2:172.21.15.117:3300/0 conn(0x7fecc006c900 msgr2=0x7fecc00a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.832+0000 7fecc5b73640 1 --2- 172.21.15.26:0/41331045 >> v2:172.21.15.117:3300/0 conn(0x7fecc006c900 0x7fecc00a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.832+0000 7fecc5b73640 1 -- 172.21.15.26:0/41331045 >> v1:172.21.15.117:6789/0 conn(0x7fecc00ab170 legacy=0x7fecc006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.832+0000 7fecc5b73640 1 -- 172.21.15.26:0/41331045 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0155c90 con 0x7fecc00c5990 2024-03-06T07:20:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.832+0000 7fecc5b73640 1 --2- 172.21.15.26:0/41331045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc00c5990 0x7fecc00c0ab0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009870 tx=0x7fecb802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cf563f1514e3f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.832+0000 7fecc4b71640 1 -- 172.21.15.26:0/41331045 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8034070 con 0x7fecc00c5990 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc4b71640 1 -- 172.21.15.26:0/41331045 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8030860 con 0x7fecc00c5990 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc4b71640 1 -- 172.21.15.26:0/41331045 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8030c30 con 0x7fecc00c5990 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 -- 172.21.15.26:0/41331045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc00c5990 msgr2=0x7fecc00c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/41331045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc00c5990 0x7fecc00c0ab0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009870 tx=0x7fecb802b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 -- 172.21.15.26:0/41331045 shutdown_connections 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/41331045 >> v2:172.21.15.117:3300/0 conn(0x7fecc006c900 0x7fecc00a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/41331045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc00c5990 0x7fecc00c0ab0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 -- 172.21.15.26:0/41331045 >> 172.21.15.26:0/41331045 conn(0x7fecc00c6710 msgr2=0x7fecc00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 -- 172.21.15.26:0/41331045 shutdown_connections 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.833+0000 7fecc7dd7640 1 -- 172.21.15.26:0/41331045 wait complete. 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.834+0000 7fecc7dd7640 1 Processor -- start 2024-03-06T07:20:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.834+0000 7fecc7dd7640 1 -- start start 2024-03-06T07:20:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc7dd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc006c900 0x7fecc0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc7dd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecc00ab170 0x7fecc00c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc7dd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecc00c5990 0x7fecc0146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc7dd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc0156610 con 0x7fecc006c900 2024-03-06T07:20:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc7dd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc0156490 con 0x7fecc00c5990 2024-03-06T07:20:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc7dd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00c3af0 con 0x7fecc00ab170 2024-03-06T07:20:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc006c900 0x7fecc0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc006c900 0x7fecc0154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45930/0 (socket says 172.21.15.26:45930) 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 -- 172.21.15.26:0/1709087339 learned_addr learned my addr 172.21.15.26:0/1709087339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc6374640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecc00c5990 0x7fecc0146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5372640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecc00ab170 0x7fecc00c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 -- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecc00ab170 msgr2=0x7fecc00c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecc00ab170 0x7fecc00c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 -- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecc00c5990 msgr2=0x7fecc0146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecc00c5990 0x7fecc0146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 -- 172.21.15.26:0/1709087339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0155160 con 0x7fecc006c900 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.835+0000 7fecc5b73640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc006c900 0x7fecc0154c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fecb8000c00 tx=0x7fecb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.836+0000 7fecbe7fc640 1 -- 172.21.15.26:0/1709087339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8034040 con 0x7fecc006c900 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.836+0000 7fecbe7fc640 1 -- 172.21.15.26:0/1709087339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8039d90 con 0x7fecc006c900 2024-03-06T07:20:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.836+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc014e260 con 0x7fecc006c900 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.836+0000 7fecbe7fc640 1 -- 172.21.15.26:0/1709087339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8043630 con 0x7fecc006c900 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.836+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc014e820 con 0x7fecc006c900 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.837+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc004f330 con 0x7fecc006c900 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.837+0000 7fecbe7fc640 1 -- 172.21.15.26:0/1709087339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecb804a060 con 0x7fecc006c900 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.837+0000 7fecbe7fc640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecac06b500 0x7fecac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.838+0000 7fecbe7fc640 1 -- 172.21.15.26:0/1709087339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fecb80ac350 con 0x7fecc006c900 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.838+0000 7fecc5372640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecac06b500 0x7fecac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.838+0000 7fecc5372640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecac06b500 0x7fecac06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fecc01e8940 tx=0x7fecb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:43.842+0000 7fecbe7fc640 1 -- 172.21.15.26:0/1709087339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fecb803e070 con 0x7fecc006c900 2024-03-06T07:20:44.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.024+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecc00c1740 con 0x7fecc006c900 2024-03-06T07:20:44.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.025+0000 7fecbe7fc640 1 -- 172.21.15.26:0/1709087339 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecb807a8d0 con 0x7fecc006c900 2024-03-06T07:20:44.026 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecac06b500 msgr2=0x7fecac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecac06b500 0x7fecac06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fecc01e8940 tx=0x7fecb4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc006c900 msgr2=0x7fecc0154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc006c900 0x7fecc0154c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fecb8000c00 tx=0x7fecb80046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 shutdown_connections 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecac06b500 0x7fecac06d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecc00c5990 0x7fecc0146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecc00ab170 0x7fecc00c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 --2- 172.21.15.26:0/1709087339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc006c900 0x7fecc0154c20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 >> 172.21.15.26:0/1709087339 conn(0x7fecc00c6710 msgr2=0x7fecc00aa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 shutdown_connections 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.027+0000 7fecc7dd7640 1 -- 172.21.15.26:0/1709087339 wait complete. 2024-03-06T07:20:44.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:44.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.142+0000 7f2167b5c640 1 Processor -- start 2024-03-06T07:20:44.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.142+0000 7f2167b5c640 1 -- start start 2024-03-06T07:20:44.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f2167b5c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21601529f0 0x7f21600c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f2167b5c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f21600c57d0 0x7f2160155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f2167b5c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2160152dd0 con 0x7f21600c5370 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f2167b5c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2160159710 con 0x7f21601529f0 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f2167b5c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2160156180 con 0x7f21600c57d0 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21658f8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21601529f0 0x7f21600c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21650f7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21600c5370 0x7f2160155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44464/0 (socket says 172.21.15.26:44464) 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21650f7640 1 -- 172.21.15.26:0/460080926 learned_addr learned my addr 172.21.15.26:0/460080926 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21660f9640 1 --2- 172.21.15.26:0/460080926 >> v2:172.21.15.117:3300/0 conn(0x7f21600c57d0 0x7f2160155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21658f8640 1 -- 172.21.15.26:0/460080926 >> v2:172.21.15.117:3300/0 conn(0x7f21600c57d0 msgr2=0x7f2160155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:44.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21658f8640 1 --2- 172.21.15.26:0/460080926 >> v2:172.21.15.117:3300/0 conn(0x7f21600c57d0 0x7f2160155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21658f8640 1 -- 172.21.15.26:0/460080926 >> v1:172.21.15.26:6789/0 conn(0x7f21600c5370 legacy=0x7f2160155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21658f8640 1 -- 172.21.15.26:0/460080926 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160158b00 con 0x7f21601529f0 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.143+0000 7f21648f6640 1 -- 172.21.15.26:0/460080926 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575409737 0 0) 0x7f2160152dd0 con 0x7f21600c5370 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.144+0000 7f21658f8640 1 --2- 172.21.15.26:0/460080926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f21600c88f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21600735c0 tx=0x7f215802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c31c72f7f5236fc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.144+0000 7f21648f6640 1 -- 172.21.15.26:0/460080926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21580315a0 con 0x7f21601529f0 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.144+0000 7f21648f6640 1 -- 172.21.15.26:0/460080926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158037070 con 0x7f21601529f0 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.144+0000 7f21648f6640 1 -- 172.21.15.26:0/460080926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158031960 con 0x7f21601529f0 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 -- 172.21.15.26:0/460080926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 msgr2=0x7f21600c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 --2- 172.21.15.26:0/460080926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f21600c88f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21600735c0 tx=0x7f215802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 -- 172.21.15.26:0/460080926 shutdown_connections 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 --2- 172.21.15.26:0/460080926 >> v2:172.21.15.117:3300/0 conn(0x7f21600c57d0 0x7f2160155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 --2- 172.21.15.26:0/460080926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f21600c88f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 -- 172.21.15.26:0/460080926 >> 172.21.15.26:0/460080926 conn(0x7f21600ac580 msgr2=0x7f21600ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 -- 172.21.15.26:0/460080926 shutdown_connections 2024-03-06T07:20:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.145+0000 7f2167b5c640 1 -- 172.21.15.26:0/460080926 wait complete. 2024-03-06T07:20:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.146+0000 7f2167b5c640 1 Processor -- start 2024-03-06T07:20:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.146+0000 7f2167b5c640 1 -- start start 2024-03-06T07:20:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f2167b5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21600c5370 0x7f2160157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f2167b5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21600c57d0 0x7f2160156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f2167b5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f2160154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f2167b5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601590c0 con 0x7f21601529f0 2024-03-06T07:20:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f2167b5c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600c68b0 con 0x7f21600c57d0 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f2167b5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21601593a0 con 0x7f21600c5370 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f2160154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f2160154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45954/0 (socket says 172.21.15.26:45954) 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 -- 172.21.15.26:0/1176805212 learned_addr learned my addr 172.21.15.26:0/1176805212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21658f8640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21600c5370 0x7f2160157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21650f7640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21600c57d0 0x7f2160156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 -- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21600c5370 msgr2=0x7f2160157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21600c5370 0x7f2160157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 -- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21600c57d0 msgr2=0x7f2160156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21600c57d0 0x7f2160156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 -- 172.21.15.26:0/1176805212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21601ea740 con 0x7f21601529f0 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f21660f9640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f2160154280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2150009dd0 tx=0x7f215000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f215e7fc640 1 -- 172.21.15.26:0/1176805212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150018020 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f215e7fc640 1 -- 172.21.15.26:0/1176805212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2150014480 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.147+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21601ea9b0 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.148+0000 7f215e7fc640 1 -- 172.21.15.26:0/1176805212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150014870 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.148+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21601547c0 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.148+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2160149910 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.149+0000 7f215e7fc640 1 -- 172.21.15.26:0/1176805212 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2150005d10 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.149+0000 7f215e7fc640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f214c06b4d0 0x7f214c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.149+0000 7f215e7fc640 1 -- 172.21.15.26:0/1176805212 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f215008af10 con 0x7f21601529f0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.149+0000 7f21658f8640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f214c06b4d0 0x7f214c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.150+0000 7f21658f8640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f214c06b4d0 0x7f214c06d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2158036000 tx=0x7f215803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.153+0000 7f215e7fc640 1 -- 172.21.15.26:0/1176805212 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2160149910 con 0x7f21601529f0 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.334+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f216004f330 con 0x7f21601529f0 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.334+0000 7f215e7fc640 1 -- 172.21.15.26:0/1176805212 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2150022070 con 0x7f21601529f0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f214c06b4d0 msgr2=0x7f214c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f214c06b4d0 0x7f214c06d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2158036000 tx=0x7f215803d040 comp rx=0 tx=0).stop 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 msgr2=0x7f2160154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f2160154280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2150009dd0 tx=0x7f215000c490 comp rx=0 tx=0).stop 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 shutdown_connections 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f214c06b4d0 0x7f214c06d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21601529f0 0x7f2160154280 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21600c57d0 0x7f2160156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 --2- 172.21.15.26:0/1176805212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21600c5370 0x7f2160157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 >> 172.21.15.26:0/1176805212 conn(0x7f21600ac580 msgr2=0x7f2160155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 shutdown_connections 2024-03-06T07:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.336+0000 7f2167b5c640 1 -- 172.21.15.26:0/1176805212 wait complete. 2024-03-06T07:20:44.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:44.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.451+0000 7f2ca59e9640 1 Processor -- start 2024-03-06T07:20:44.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.451+0000 7f2ca59e9640 1 -- start start 2024-03-06T07:20:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2ca59e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ca01529f0 0x7f2ca00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2ca59e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ca00c5370 0x7f2ca00c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2ca59e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca0152dd0 con 0x7f2ca0155530 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2ca59e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca0159710 con 0x7f2ca01529f0 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2ca59e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca00c68b0 con 0x7f2ca00c5370 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ca01529f0 0x7f2ca00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2ca0155530 0x7f2ca01559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44468/0 (socket says 172.21.15.26:44468) 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9f7fe640 1 -- 172.21.15.26:0/2608565182 learned_addr learned my addr 172.21.15.26:0/2608565182 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9e7fc640 1 --2- 172.21.15.26:0/2608565182 >> v2:172.21.15.117:3300/0 conn(0x7f2ca00c5370 0x7f2ca00c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9effd640 1 -- 172.21.15.26:0/2608565182 >> v2:172.21.15.117:3300/0 conn(0x7f2ca00c5370 msgr2=0x7f2ca00c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9effd640 1 --2- 172.21.15.26:0/2608565182 >> v2:172.21.15.117:3300/0 conn(0x7f2ca00c5370 0x7f2ca00c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9effd640 1 -- 172.21.15.26:0/2608565182 >> v1:172.21.15.26:6789/0 conn(0x7f2ca0155530 legacy=0x7f2ca01559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9dffb640 1 -- 172.21.15.26:0/2608565182 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816222540 0 0) 0x7f2ca0152dd0 con 0x7f2ca0155530 2024-03-06T07:20:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.452+0000 7f2c9effd640 1 -- 172.21.15.26:0/2608565182 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0158b00 con 0x7f2ca01529f0 2024-03-06T07:20:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.453+0000 7f2c9effd640 1 --2- 172.21.15.26:0/2608565182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca01529f0 0x7f2ca00c88f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2ca008c9b0 tx=0x7f2c9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c1d62e18a33768 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.453+0000 7f2c9dffb640 1 -- 172.21.15.26:0/2608565182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980315a0 con 0x7f2ca01529f0 2024-03-06T07:20:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.453+0000 7f2c9dffb640 1 -- 172.21.15.26:0/2608565182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98037070 con 0x7f2ca01529f0 2024-03-06T07:20:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.453+0000 7f2c9dffb640 1 -- 172.21.15.26:0/2608565182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98031960 con 0x7f2ca01529f0 2024-03-06T07:20:44.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 -- 172.21.15.26:0/2608565182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca01529f0 msgr2=0x7f2ca00c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/2608565182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca01529f0 0x7f2ca00c88f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2ca008c9b0 tx=0x7f2c9802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 -- 172.21.15.26:0/2608565182 shutdown_connections 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/2608565182 >> v2:172.21.15.117:3300/0 conn(0x7f2ca00c5370 0x7f2ca00c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/2608565182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca01529f0 0x7f2ca00c88f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 -- 172.21.15.26:0/2608565182 >> 172.21.15.26:0/2608565182 conn(0x7f2ca00ac580 msgr2=0x7f2ca00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 -- 172.21.15.26:0/2608565182 shutdown_connections 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.454+0000 7f2ca59e9640 1 -- 172.21.15.26:0/2608565182 wait complete. 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.455+0000 7f2ca59e9640 1 Processor -- start 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.455+0000 7f2ca59e9640 1 -- start start 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2ca59e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca00c5370 0x7f2ca0156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2ca59e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca01529f0 0x7f2ca0154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2ca59e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca0155530 0x7f2ca01ed150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2ca59e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca01593a0 con 0x7f2ca00c5370 2024-03-06T07:20:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2ca59e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca01590c0 con 0x7f2ca01529f0 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2ca59e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca01560d0 con 0x7f2ca0155530 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca00c5370 0x7f2ca0156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca00c5370 0x7f2ca0156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45980/0 (socket says 172.21.15.26:45980) 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 -- 172.21.15.26:0/1134620506 learned_addr learned my addr 172.21.15.26:0/1134620506 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9e7fc640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca01529f0 0x7f2ca0154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9f7fe640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca0155530 0x7f2ca01ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 -- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca0155530 msgr2=0x7f2ca01ed150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca0155530 0x7f2ca01ed150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 -- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca01529f0 msgr2=0x7f2ca0154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca01529f0 0x7f2ca0154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 -- 172.21.15.26:0/1134620506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0154d80 con 0x7f2ca00c5370 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9f7fe640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca0155530 0x7f2ca01ed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.456+0000 7f2c9effd640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca00c5370 0x7f2ca0156540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009890 tx=0x7f2c980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.457+0000 7f2c87fff640 1 -- 172.21.15.26:0/1134620506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98032630 con 0x7f2ca00c5370 2024-03-06T07:20:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.457+0000 7f2c87fff640 1 -- 172.21.15.26:0/1134620506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c980327f0 con 0x7f2ca00c5370 2024-03-06T07:20:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.457+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca0156c40 con 0x7f2ca00c5370 2024-03-06T07:20:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.457+0000 7f2c87fff640 1 -- 172.21.15.26:0/1134620506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98032be0 con 0x7f2ca00c5370 2024-03-06T07:20:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.457+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca0157200 con 0x7f2ca00c5370 2024-03-06T07:20:44.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.457+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca004f330 con 0x7f2ca00c5370 2024-03-06T07:20:44.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.458+0000 7f2c87fff640 1 -- 172.21.15.26:0/1134620506 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c9804c020 con 0x7f2ca00c5370 2024-03-06T07:20:44.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.458+0000 7f2c87fff640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.458+0000 7f2c87fff640 1 -- 172.21.15.26:0/1134620506 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2c9802f020 con 0x7f2ca00c5370 2024-03-06T07:20:44.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.459+0000 7f2c9e7fc640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.459+0000 7f2c9e7fc640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00c1830 tx=0x7f2c94009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.463+0000 7f2c87fff640 1 -- 172.21.15.26:0/1134620506 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ca0154d80 con 0x7f2ca00c5370 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.646+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ca00c7bf0 con 0x7f2ca00c5370 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.646+0000 7f2c87fff640 1 -- 172.21.15.26:0/1134620506 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c98040070 con 0x7f2ca00c5370 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c8c06b4d0 msgr2=0x7f2c8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00c1830 tx=0x7f2c94009190 comp rx=0 tx=0).stop 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca00c5370 msgr2=0x7f2ca0156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca00c5370 0x7f2ca0156540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009890 tx=0x7f2c980046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 shutdown_connections 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca0155530 0x7f2ca01ed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca01529f0 0x7f2ca0154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 --2- 172.21.15.26:0/1134620506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca00c5370 0x7f2ca0156540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 >> 172.21.15.26:0/1134620506 conn(0x7f2ca00ac580 msgr2=0x7f2ca006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 shutdown_connections 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.648+0000 7f2ca59e9640 1 -- 172.21.15.26:0/1134620506 wait complete. 2024-03-06T07:20:44.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:44.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.763+0000 7f3ed7e66640 1 Processor -- start 2024-03-06T07:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.763+0000 7f3ed7e66640 1 -- start start 2024-03-06T07:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.764+0000 7f3ed7e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ed0072b30 0x7f3ed00c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.764+0000 7f3ed7e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed015cdd0 con 0x7f3ed0156bb0 2024-03-06T07:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.764+0000 7f3ed7e66640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed015d0b0 con 0x7f3ed0072680 2024-03-06T07:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.764+0000 7f3ed7e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed015d390 con 0x7f3ed0072b30 2024-03-06T07:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.764+0000 7f3ed5c02640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3ed0156bb0 0x7f3ed0159f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44470/0 (socket says 172.21.15.26:44470) 2024-03-06T07:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.764+0000 7f3ed6403640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ed0072b30 0x7f3ed00c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.764+0000 7f3ed5c02640 1 -- 172.21.15.26:0/2677014419 learned_addr learned my addr 172.21.15.26:0/2677014419 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed4c00640 1 -- 172.21.15.26:0/2677014419 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064179164 0 0) 0x7f3ed015cdd0 con 0x7f3ed0156bb0 2024-03-06T07:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed4c00640 1 -- 172.21.15.26:0/2677014419 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eb8003520 con 0x7f3ed0156bb0 2024-03-06T07:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed4c00640 1 -- 172.21.15.26:0/2677014419 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585238767 0 0) 0x7f3ed015d0b0 con 0x7f3ed0072680 2024-03-06T07:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed6403640 1 -- 172.21.15.26:0/2677014419 >> v1:172.21.15.117:6789/0 conn(0x7f3ed0072680 legacy=0x7f3ed00c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed6403640 1 -- 172.21.15.26:0/2677014419 >> v1:172.21.15.26:6789/0 conn(0x7f3ed0156bb0 legacy=0x7f3ed0159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed6403640 1 -- 172.21.15.26:0/2677014419 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed015d670 con 0x7f3ed0072b30 2024-03-06T07:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed6403640 1 --2- 172.21.15.26:0/2677014419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0072b30 0x7f3ed00c7390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009d90 tx=0x7f3ec002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b3498ef878c07b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed4c00640 1 -- 172.21.15.26:0/2677014419 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0039070 con 0x7f3ed0072b30 2024-03-06T07:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.765+0000 7f3ed4c00640 1 -- 172.21.15.26:0/2677014419 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec002cc60 con 0x7f3ed0072b30 2024-03-06T07:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed4c00640 1 -- 172.21.15.26:0/2677014419 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0034660 con 0x7f3ed0072b30 2024-03-06T07:20:44.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2677014419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0072b30 msgr2=0x7f3ed00c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2677014419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0072b30 0x7f3ed00c7390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009d90 tx=0x7f3ec002beb0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2677014419 shutdown_connections 2024-03-06T07:20:44.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2677014419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0072b30 0x7f3ed00c7390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2677014419 >> 172.21.15.26:0/2677014419 conn(0x7f3ed00c5560 msgr2=0x7f3ed00c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2677014419 shutdown_connections 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.766+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2677014419 wait complete. 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.767+0000 7f3ed7e66640 1 Processor -- start 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.767+0000 7f3ed7e66640 1 -- start start 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed7e66640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed0072680 0x7f3ed00ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed7e66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0072b30 0x7f3ed00acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed7e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0156bb0 0x7f3ed01ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed7e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed015d950 con 0x7f3ed0156bb0 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed7e66640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed015caa0 con 0x7f3ed0072b30 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed7e66640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00c8410 con 0x7f3ed0072680 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed6403640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0156bb0 0x7f3ed01ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed6403640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0156bb0 0x7f3ed01ebf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45990/0 (socket says 172.21.15.26:45990) 2024-03-06T07:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed6403640 1 -- 172.21.15.26:0/2791744695 learned_addr learned my addr 172.21.15.26:0/2791744695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed5401640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0072b30 0x7f3ed00acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.768+0000 7f3ed5c02640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed0072680 0x7f3ed00ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed6403640 1 -- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed0072680 msgr2=0x7f3ed00ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed6403640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed0072680 0x7f3ed00ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed6403640 1 -- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0072b30 msgr2=0x7f3ed00acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed6403640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0072b30 0x7f3ed00acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed6403640 1 -- 172.21.15.26:0/2791744695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00abec0 con 0x7f3ed0156bb0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed6403640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0156bb0 0x7f3ed01ebf60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0037040 tx=0x7f3ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ece7fc640 1 -- 172.21.15.26:0/2791744695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0039050 con 0x7f3ed0156bb0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ece7fc640 1 -- 172.21.15.26:0/2791744695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec003a040 con 0x7f3ed0156bb0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed006f130 con 0x7f3ed0156bb0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ece7fc640 1 -- 172.21.15.26:0/2791744695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0009450 con 0x7f3ed0156bb0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.769+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed0159280 con 0x7f3ed0156bb0 2024-03-06T07:20:44.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.770+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed004f330 con 0x7f3ed0156bb0 2024-03-06T07:20:44.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.770+0000 7f3ece7fc640 1 -- 172.21.15.26:0/2791744695 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ec0047020 con 0x7f3ed0156bb0 2024-03-06T07:20:44.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.771+0000 7f3ece7fc640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.771+0000 7f3ece7fc640 1 -- 172.21.15.26:0/2791744695 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ec00ac570 con 0x7f3ed0156bb0 2024-03-06T07:20:44.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.771+0000 7f3ed5c02640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.771+0000 7f3ed5c02640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005880 tx=0x7f3ec8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.775+0000 7f3ece7fc640 1 -- 172.21.15.26:0/2791744695 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ec004e020 con 0x7f3ed0156bb0 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.957+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3ed00c87d0 con 0x7f3ed0156bb0 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.957+0000 7f3ece7fc640 1 -- 172.21.15.26:0/2791744695 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3ec007aaf0 con 0x7f3ed0156bb0 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ebc06b500 msgr2=0x7f3ebc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005880 tx=0x7f3ec8009040 comp rx=0 tx=0).stop 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0156bb0 msgr2=0x7f3ed01ebf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0156bb0 0x7f3ed01ebf60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0037040 tx=0x7f3ec0004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 shutdown_connections 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed0156bb0 0x7f3ed01ebf60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0072b30 0x7f3ed00acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 --2- 172.21.15.26:0/2791744695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed0072680 0x7f3ed00ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 >> 172.21.15.26:0/2791744695 conn(0x7f3ed00c5560 msgr2=0x7f3ed0156100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 shutdown_connections 2024-03-06T07:20:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:44.960+0000 7f3ed7e66640 1 -- 172.21.15.26:0/2791744695 wait complete. 2024-03-06T07:20:44.973 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.074+0000 7f8f4a1d0640 1 Processor -- start 2024-03-06T07:20:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.074+0000 7f8f4a1d0640 1 -- start start 2024-03-06T07:20:45.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.074+0000 7f8f4a1d0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f440bfe90 0x7f8f44155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.074+0000 7f8f4a1d0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f440c7200 con 0x7f8f441522d0 2024-03-06T07:20:45.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.074+0000 7f8f4a1d0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f44159590 con 0x7f8f440c6d70 2024-03-06T07:20:45.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.074+0000 7f8f4a1d0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f44158710 con 0x7f8f440bfe90 2024-03-06T07:20:45.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f43fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f441522d0 0x7f8f4415faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44484/0 (socket says 172.21.15.26:44484) 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f43fff640 1 -- 172.21.15.26:0/460021693 learned_addr learned my addr 172.21.15.26:0/460021693 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f437fe640 1 --2- 172.21.15.26:0/460021693 >> v2:172.21.15.117:3300/0 conn(0x7f8f440bfe90 0x7f8f44155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f427fc640 1 -- 172.21.15.26:0/460021693 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477667893 0 0) 0x7f8f3c001490 con 0x7f8f441522d0 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f427fc640 1 -- 172.21.15.26:0/460021693 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f2c003520 con 0x7f8f441522d0 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f437fe640 1 -- 172.21.15.26:0/460021693 >> v1:172.21.15.117:6789/0 conn(0x7f8f440c6d70 legacy=0x7f8f44158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f437fe640 1 -- 172.21.15.26:0/460021693 >> v1:172.21.15.26:6789/0 conn(0x7f8f441522d0 legacy=0x7f8f4415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.075+0000 7f8f437fe640 1 -- 172.21.15.26:0/460021693 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440c59d0 con 0x7f8f440bfe90 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f437fe640 1 --2- 172.21.15.26:0/460021693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 0x7f8f44155120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f340097d0 tx=0x7f8f3402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a49b823d28a0306 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f427fc640 1 -- 172.21.15.26:0/460021693 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34035020 con 0x7f8f440bfe90 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f427fc640 1 -- 172.21.15.26:0/460021693 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34031830 con 0x7f8f440bfe90 2024-03-06T07:20:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f427fc640 1 -- 172.21.15.26:0/460021693 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34031be0 con 0x7f8f440bfe90 2024-03-06T07:20:45.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/460021693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 msgr2=0x7f8f44155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/460021693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 0x7f8f44155120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f340097d0 tx=0x7f8f3402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/460021693 shutdown_connections 2024-03-06T07:20:45.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/460021693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 0x7f8f44155120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/460021693 >> 172.21.15.26:0/460021693 conn(0x7f8f440acaa0 msgr2=0x7f8f440aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:45.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/460021693 shutdown_connections 2024-03-06T07:20:45.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.076+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/460021693 wait complete. 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.077+0000 7f8f4a1d0640 1 Processor -- start 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f4a1d0640 1 -- start start 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f4a1d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 0x7f8f441eb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f4a1d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440c6d70 0x7f8f441ee130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f4a1d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f441522d0 0x7f8f440aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f4a1d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44158960 con 0x7f8f440c6d70 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f4a1d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440c5f90 con 0x7f8f440bfe90 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f4a1d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440ca700 con 0x7f8f441522d0 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440c6d70 0x7f8f441ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440c6d70 0x7f8f441ee130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46000/0 (socket says 172.21.15.26:46000) 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.078+0000 7f8f42ffd640 1 -- 172.21.15.26:0/2147304205 learned_addr learned my addr 172.21.15.26:0/2147304205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f43fff640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f441522d0 0x7f8f440aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f437fe640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 0x7f8f441eb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f42ffd640 1 -- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f441522d0 msgr2=0x7f8f440aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f42ffd640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f441522d0 0x7f8f440aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f42ffd640 1 -- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 msgr2=0x7f8f441eb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f42ffd640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 0x7f8f441eb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f42ffd640 1 -- 172.21.15.26:0/2147304205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f441ebb10 con 0x7f8f440c6d70 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f42ffd640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440c6d70 0x7f8f441ee130 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c400 tx=0x7f8f3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f2bfff640 1 -- 172.21.15.26:0/2147304205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380052d0 con 0x7f8f440c6d70 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f2bfff640 1 -- 172.21.15.26:0/2147304205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38005490 con 0x7f8f440c6d70 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f441ee670 con 0x7f8f440c6d70 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f2bfff640 1 -- 172.21.15.26:0/2147304205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38007260 con 0x7f8f440c6d70 2024-03-06T07:20:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.079+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f441eadd0 con 0x7f8f440c6d70 2024-03-06T07:20:45.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.080+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f441eabc0 con 0x7f8f440c6d70 2024-03-06T07:20:45.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.081+0000 7f8f2bfff640 1 -- 172.21.15.26:0/2147304205 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f38007400 con 0x7f8f440c6d70 2024-03-06T07:20:45.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.081+0000 7f8f2bfff640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f3006b4d0 0x7f8f3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.081+0000 7f8f2bfff640 1 -- 172.21.15.26:0/2147304205 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8f38089910 con 0x7f8f440c6d70 2024-03-06T07:20:45.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.081+0000 7f8f437fe640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f3006b4d0 0x7f8f3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.082+0000 7f8f437fe640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f3006b4d0 0x7f8f3006d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f34036000 tx=0x7f8f34008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.085+0000 7f8f2bfff640 1 -- 172.21.15.26:0/2147304205 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f38057e90 con 0x7f8f440c6d70 2024-03-06T07:20:45.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.277+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8f4404f330 con 0x7f8f440c6d70 2024-03-06T07:20:45.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.278+0000 7f8f2bfff640 1 -- 172.21.15.26:0/2147304205 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8f3805bf30 con 0x7f8f440c6d70 2024-03-06T07:20:45.280 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f3006b4d0 msgr2=0x7f8f3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f3006b4d0 0x7f8f3006d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f34036000 tx=0x7f8f34008030 comp rx=0 tx=0).stop 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440c6d70 msgr2=0x7f8f441ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440c6d70 0x7f8f441ee130 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c400 tx=0x7f8f3800c8c0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 shutdown_connections 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f3006b4d0 0x7f8f3006d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f441522d0 0x7f8f440aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440c6d70 0x7f8f441ee130 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 --2- 172.21.15.26:0/2147304205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f440bfe90 0x7f8f441eb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 >> 172.21.15.26:0/2147304205 conn(0x7f8f440acaa0 msgr2=0x7f8f440ab1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 shutdown_connections 2024-03-06T07:20:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.280+0000 7f8f4a1d0640 1 -- 172.21.15.26:0/2147304205 wait complete. 2024-03-06T07:20:45.292 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-06T07:20:45.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:45.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-06T07:20:45.321 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:20:45.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:45.321 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-06T07:20:45.385 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:20:45.385 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:45.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.486+0000 7fc29bd9a640 1 Processor -- start 2024-03-06T07:20:45.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.486+0000 7fc29bd9a640 1 -- start start 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29bd9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c5330 0x7fc2940c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29bd9a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc2940c88b0 0x7fc2941586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29bd9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2940c7dd0 0x7fc294158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29bd9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2941559e0 con 0x7fc2940c7dd0 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29bd9a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc294155110 con 0x7fc2940c88b0 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29bd9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2940c8f40 con 0x7fc2940c5330 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2940c7dd0 0x7fc294158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2940c7dd0 0x7fc294158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46016/0 (socket says 172.21.15.26:46016) 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 -- 172.21.15.26:0/616942721 learned_addr learned my addr 172.21.15.26:0/616942721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc299335640 1 --2- 172.21.15.26:0/616942721 >> v2:172.21.15.117:3300/0 conn(0x7fc2940c88b0 0x7fc2941586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc299b36640 1 --2- 172.21.15.26:0/616942721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c5330 0x7fc2940c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 -- 172.21.15.26:0/616942721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c5330 msgr2=0x7fc2940c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:45.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 --2- 172.21.15.26:0/616942721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c5330 0x7fc2940c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 -- 172.21.15.26:0/616942721 >> v2:172.21.15.117:3300/0 conn(0x7fc2940c88b0 msgr2=0x7fc2941586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:45.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 --2- 172.21.15.26:0/616942721 >> v2:172.21.15.117:3300/0 conn(0x7fc2940c88b0 0x7fc2941586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.487+0000 7fc29a337640 1 -- 172.21.15.26:0/616942721 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294155390 con 0x7fc2940c7dd0 2024-03-06T07:20:45.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.488+0000 7fc29a337640 1 --2- 172.21.15.26:0/616942721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c7dd0 0x7fc294158be0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc29408c940 tx=0x7fc28c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05edf6307c00450 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.488+0000 7fc298b34640 1 -- 172.21.15.26:0/616942721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c02c9e0 con 0x7fc2940c7dd0 2024-03-06T07:20:45.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.488+0000 7fc298b34640 1 -- 172.21.15.26:0/616942721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c02cb80 con 0x7fc2940c7dd0 2024-03-06T07:20:45.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.488+0000 7fc298b34640 1 -- 172.21.15.26:0/616942721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0343f0 con 0x7fc2940c7dd0 2024-03-06T07:20:45.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.488+0000 7fc29bd9a640 1 -- 172.21.15.26:0/616942721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c7dd0 msgr2=0x7fc294158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.488+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/616942721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c7dd0 0x7fc294158be0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc29408c940 tx=0x7fc28c02be40 comp rx=0 tx=0).stop 2024-03-06T07:20:45.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.489+0000 7fc29bd9a640 1 -- 172.21.15.26:0/616942721 shutdown_connections 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.489+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/616942721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c7dd0 0x7fc294158be0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.489+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/616942721 >> v2:172.21.15.117:3300/0 conn(0x7fc2940c88b0 0x7fc2941586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.489+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/616942721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c5330 0x7fc2940c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.489+0000 7fc29bd9a640 1 -- 172.21.15.26:0/616942721 >> 172.21.15.26:0/616942721 conn(0x7fc2940ac560 msgr2=0x7fc2940ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.489+0000 7fc29bd9a640 1 -- 172.21.15.26:0/616942721 shutdown_connections 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.489+0000 7fc29bd9a640 1 -- 172.21.15.26:0/616942721 wait complete. 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 Processor -- start 2024-03-06T07:20:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 -- start start 2024-03-06T07:20:45.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c5330 0x7fc2940c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2940c7dd0 0x7fc29414d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c88b0 0x7fc29414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2941529d0 con 0x7fc2940c5330 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29406d090 con 0x7fc2940c7dd0 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc29bd9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc294152b50 con 0x7fc2940c88b0 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc299b36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c5330 0x7fc2940c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc299b36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c5330 0x7fc2940c3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46024/0 (socket says 172.21.15.26:46024) 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.490+0000 7fc299b36640 1 -- 172.21.15.26:0/1236842010 learned_addr learned my addr 172.21.15.26:0/1236842010 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc29a337640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c88b0 0x7fc29414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc299335640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2940c7dd0 0x7fc29414d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc299b36640 1 -- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c88b0 msgr2=0x7fc29414d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc299b36640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c88b0 0x7fc29414d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc299b36640 1 -- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2940c7dd0 msgr2=0x7fc29414d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc299b36640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2940c7dd0 0x7fc29414d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc299b36640 1 -- 172.21.15.26:0/1236842010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29414df00 con 0x7fc2940c5330 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc299b36640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c5330 0x7fc2940c3120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e8a0 tx=0x7fc29000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc28a7fc640 1 -- 172.21.15.26:0/1236842010 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290017070 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc28a7fc640 1 -- 172.21.15.26:0/1236842010 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29000ee90 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc28a7fc640 1 -- 172.21.15.26:0/1236842010 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000cb80 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29414e1e0 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.491+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2941521d0 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.492+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29404f330 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.492+0000 7fc28a7fc640 1 -- 172.21.15.26:0/1236842010 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc29001a070 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.493+0000 7fc28a7fc640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc27806b4d0 0x7fc27806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.493+0000 7fc28a7fc640 1 -- 172.21.15.26:0/1236842010 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc29008a5a0 con 0x7fc2940c5330 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.493+0000 7fc299335640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc27806b4d0 0x7fc27806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.494+0000 7fc299335640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc27806b4d0 0x7fc27806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc29414de30 tx=0x7fc284009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.497+0000 7fc28a7fc640 1 -- 172.21.15.26:0/1236842010 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc290058b20 con 0x7fc2940c5330 2024-03-06T07:20:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.716+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc29406cdc0 con 0x7fc2940c5330 2024-03-06T07:20:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.716+0000 7fc28a7fc640 1 -- 172.21.15.26:0/1236842010 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7fc29005cbc0 con 0x7fc2940c5330 2024-03-06T07:20:45.718 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:45.718 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc27806b4d0 msgr2=0x7fc27806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc27806b4d0 0x7fc27806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc29414de30 tx=0x7fc284009190 comp rx=0 tx=0).stop 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c5330 msgr2=0x7fc2940c3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c5330 0x7fc2940c3120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e8a0 tx=0x7fc29000ed60 comp rx=0 tx=0).stop 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 shutdown_connections 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc27806b4d0 0x7fc27806d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2940c88b0 0x7fc29414d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2940c7dd0 0x7fc29414d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 --2- 172.21.15.26:0/1236842010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2940c5330 0x7fc2940c3120 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 >> 172.21.15.26:0/1236842010 conn(0x7fc2940ac560 msgr2=0x7fc2940c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 shutdown_connections 2024-03-06T07:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.719+0000 7fc29bd9a640 1 -- 172.21.15.26:0/1236842010 wait complete. 2024-03-06T07:20:45.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.831+0000 7fea957f7640 1 Processor -- start 2024-03-06T07:20:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.831+0000 7fea957f7640 1 -- start start 2024-03-06T07:20:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea957f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fea901506b0 0x7fea90153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea957f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea957f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea900acc00 0x7fea900c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea957f7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea90156450 con 0x7fea900acc00 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea957f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea90156a10 con 0x7fea901506b0 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea957f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea90070800 con 0x7fea90156ea0 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea8f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea900acc00 0x7fea900c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea8f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea900acc00 0x7fea900c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46032/0 (socket says 172.21.15.26:46032) 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea8f7fe640 1 -- 172.21.15.26:0/1848673911 learned_addr learned my addr 172.21.15.26:0/1848673911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea8effd640 1 --2- 172.21.15.26:0/1848673911 >> v2:172.21.15.117:3300/0 conn(0x7fea901506b0 0x7fea90153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.832+0000 7fea8e7fc640 1 --2- 172.21.15.26:0/1848673911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8f7fe640 1 -- 172.21.15.26:0/1848673911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 msgr2=0x7fea900702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8f7fe640 1 --2- 172.21.15.26:0/1848673911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea900702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8f7fe640 1 -- 172.21.15.26:0/1848673911 >> v2:172.21.15.117:3300/0 conn(0x7fea901506b0 msgr2=0x7fea90153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8f7fe640 1 --2- 172.21.15.26:0/1848673911 >> v2:172.21.15.117:3300/0 conn(0x7fea901506b0 0x7fea90153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8f7fe640 1 -- 172.21.15.26:0/1848673911 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90070aa0 con 0x7fea900acc00 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8f7fe640 1 --2- 172.21.15.26:0/1848673911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c54a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fea900c97e0 tx=0x7fea8802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dd82115997f784 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8dffb640 1 -- 172.21.15.26:0/1848673911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88035020 con 0x7fea900acc00 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8dffb640 1 -- 172.21.15.26:0/1848673911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88031870 con 0x7fea900acc00 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.833+0000 7fea8dffb640 1 -- 172.21.15.26:0/1848673911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88031c40 con 0x7fea900acc00 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 -- 172.21.15.26:0/1848673911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 msgr2=0x7fea900c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 --2- 172.21.15.26:0/1848673911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c54a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fea900c97e0 tx=0x7fea8802b910 comp rx=0 tx=0).stop 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 -- 172.21.15.26:0/1848673911 shutdown_connections 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 --2- 172.21.15.26:0/1848673911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea900702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 --2- 172.21.15.26:0/1848673911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c54a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 --2- 172.21.15.26:0/1848673911 >> v2:172.21.15.117:3300/0 conn(0x7fea901506b0 0x7fea90153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 -- 172.21.15.26:0/1848673911 >> 172.21.15.26:0/1848673911 conn(0x7fea9006b1a0 msgr2=0x7fea9006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:45.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 -- 172.21.15.26:0/1848673911 shutdown_connections 2024-03-06T07:20:45.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.834+0000 7fea957f7640 1 -- 172.21.15.26:0/1848673911 wait complete. 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.835+0000 7fea957f7640 1 Processor -- start 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.835+0000 7fea957f7640 1 -- start start 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea957f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea957f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea901506b0 0x7fea9013d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea957f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea9013dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea957f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900ad070 con 0x7fea900acc00 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea957f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901566a0 con 0x7fea901506b0 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea957f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900ad1f0 con 0x7fea90156ea0 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46038/0 (socket says 172.21.15.26:46038) 2024-03-06T07:20:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 -- 172.21.15.26:0/3004548369 learned_addr learned my addr 172.21.15.26:0/3004548369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8f7fe640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea9013dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8e7fc640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea901506b0 0x7fea9013d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 -- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 msgr2=0x7fea9013dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea9013dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 -- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea901506b0 msgr2=0x7fea9013d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea901506b0 0x7fea9013d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 -- 172.21.15.26:0/3004548369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90155600 con 0x7fea900acc00 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.836+0000 7fea8effd640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c1d60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea8000eed0 tx=0x7fea8000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.837+0000 7fea877fe640 1 -- 172.21.15.26:0/3004548369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80018020 con 0x7fea900acc00 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.837+0000 7fea877fe640 1 -- 172.21.15.26:0/3004548369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea80014400 con 0x7fea900acc00 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.837+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea901558e0 con 0x7fea900acc00 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.837+0000 7fea877fe640 1 -- 172.21.15.26:0/3004548369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800147d0 con 0x7fea900acc00 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.837+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea90155ea0 con 0x7fea900acc00 2024-03-06T07:20:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.838+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9004f330 con 0x7fea900acc00 2024-03-06T07:20:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.838+0000 7fea877fe640 1 -- 172.21.15.26:0/3004548369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea80005d10 con 0x7fea900acc00 2024-03-06T07:20:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.838+0000 7fea877fe640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea7006b4d0 0x7fea7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.838+0000 7fea877fe640 1 -- 172.21.15.26:0/3004548369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea8008aa60 con 0x7fea900acc00 2024-03-06T07:20:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.839+0000 7fea8e7fc640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea7006b4d0 0x7fea7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.839+0000 7fea8e7fc640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea7006b4d0 0x7fea7006d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea9014a0c0 tx=0x7fea78009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:45.843+0000 7fea877fe640 1 -- 172.21.15.26:0/3004548369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea80022070 con 0x7fea900acc00 2024-03-06T07:20:46.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.026+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea900700a0 con 0x7fea900acc00 2024-03-06T07:20:46.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.027+0000 7fea877fe640 1 -- 172.21.15.26:0/3004548369 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fea80058fe0 con 0x7fea900acc00 2024-03-06T07:20:46.030 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:46.030 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea7006b4d0 msgr2=0x7fea7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea7006b4d0 0x7fea7006d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea9014a0c0 tx=0x7fea78009190 comp rx=0 tx=0).stop 2024-03-06T07:20:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 msgr2=0x7fea900c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c1d60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea8000eed0 tx=0x7fea8000b480 comp rx=0 tx=0).stop 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 shutdown_connections 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fea7006b4d0 0x7fea7006d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea90156ea0 0x7fea9013dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea901506b0 0x7fea9013d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 --2- 172.21.15.26:0/3004548369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea900acc00 0x7fea900c1d60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 >> 172.21.15.26:0/3004548369 conn(0x7fea9006b1a0 msgr2=0x7fea9014cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 shutdown_connections 2024-03-06T07:20:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.030+0000 7fea957f7640 1 -- 172.21.15.26:0/3004548369 wait complete. 2024-03-06T07:20:46.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:19:59.660648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:00.133540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:03.850046+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:46.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.146+0000 7fbf239e9640 1 Processor -- start 2024-03-06T07:20:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.146+0000 7fbf239e9640 1 -- start start 2024-03-06T07:20:46.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.147+0000 7fbf239e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c06ca70 0x7fbf1c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.147+0000 7fbf239e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0b0450 con 0x7fbf1c0b0070 2024-03-06T07:20:46.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.147+0000 7fbf239e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c1560e0 con 0x7fbf1c155c30 2024-03-06T07:20:46.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.147+0000 7fbf239e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c06c020 con 0x7fbf1c06ca70 2024-03-06T07:20:46.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.147+0000 7fbf21785640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbf1c0b0070 0x7fbf1c0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44492/0 (socket says 172.21.15.26:44492) 2024-03-06T07:20:46.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.147+0000 7fbf21785640 1 -- 172.21.15.26:0/1515967811 learned_addr learned my addr 172.21.15.26:0/1515967811 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.147+0000 7fbf21f86640 1 --2- 172.21.15.26:0/1515967811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c06ca70 0x7fbf1c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf13fff640 1 -- 172.21.15.26:0/1515967811 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762390522 0 0) 0x7fbf18001110 con 0x7fbf1c0b0070 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf13fff640 1 -- 172.21.15.26:0/1515967811 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf08003520 con 0x7fbf1c0b0070 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf13fff640 1 -- 172.21.15.26:0/1515967811 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306905836 0 0) 0x7fbf1c1560e0 con 0x7fbf1c155c30 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf13fff640 1 -- 172.21.15.26:0/1515967811 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf18001110 con 0x7fbf1c155c30 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf21f86640 1 -- 172.21.15.26:0/1515967811 >> v1:172.21.15.117:6789/0 conn(0x7fbf1c155c30 legacy=0x7fbf1c0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf21f86640 1 -- 172.21.15.26:0/1515967811 >> v1:172.21.15.26:6789/0 conn(0x7fbf1c0b0070 legacy=0x7fbf1c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf21f86640 1 -- 172.21.15.26:0/1515967811 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0c77f0 con 0x7fbf1c06ca70 2024-03-06T07:20:46.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.148+0000 7fbf21f86640 1 --2- 172.21.15.26:0/1515967811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c06ca70 0x7fbf1c14e800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c1562b0 tx=0x7fbf1402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=491d378715e10ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf13fff640 1 -- 172.21.15.26:0/1515967811 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14039070 con 0x7fbf1c06ca70 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf13fff640 1 -- 172.21.15.26:0/1515967811 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1402cc40 con 0x7fbf1c06ca70 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf13fff640 1 -- 172.21.15.26:0/1515967811 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14034440 con 0x7fbf1c06ca70 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf239e9640 1 -- 172.21.15.26:0/1515967811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c06ca70 msgr2=0x7fbf1c14e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf239e9640 1 --2- 172.21.15.26:0/1515967811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c06ca70 0x7fbf1c14e800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c1562b0 tx=0x7fbf1402be90 comp rx=0 tx=0).stop 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf239e9640 1 -- 172.21.15.26:0/1515967811 shutdown_connections 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf239e9640 1 --2- 172.21.15.26:0/1515967811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c06ca70 0x7fbf1c14e800 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf239e9640 1 -- 172.21.15.26:0/1515967811 >> 172.21.15.26:0/1515967811 conn(0x7fbf1c0ac590 msgr2=0x7fbf1c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf239e9640 1 -- 172.21.15.26:0/1515967811 shutdown_connections 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.149+0000 7fbf239e9640 1 -- 172.21.15.26:0/1515967811 wait complete. 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.150+0000 7fbf239e9640 1 Processor -- start 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.150+0000 7fbf239e9640 1 -- start start 2024-03-06T07:20:46.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf239e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c06ca70 0x7fbf1c153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf239e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c0b0070 0x7fbf1c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf239e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1c155c30 0x7fbf1c14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf239e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c06c550 con 0x7fbf1c155c30 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf239e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c06c270 con 0x7fbf1c06ca70 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf239e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c06da80 con 0x7fbf1c0b0070 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1c155c30 0x7fbf1c14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1c155c30 0x7fbf1c14d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46044/0 (socket says 172.21.15.26:46044) 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 -- 172.21.15.26:0/3772578152 learned_addr learned my addr 172.21.15.26:0/3772578152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21785640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c06ca70 0x7fbf1c153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf20f84640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c0b0070 0x7fbf1c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 -- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c0b0070 msgr2=0x7fbf1c14d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c0b0070 0x7fbf1c14d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 -- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c06ca70 msgr2=0x7fbf1c153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c06ca70 0x7fbf1c153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.151+0000 7fbf21f86640 1 -- 172.21.15.26:0/3772578152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c1ee050 con 0x7fbf1c155c30 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.152+0000 7fbf21f86640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1c155c30 0x7fbf1c14d750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009eb0 tx=0x7fbf14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.152+0000 7fbf11ffb640 1 -- 172.21.15.26:0/3772578152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14039050 con 0x7fbf1c155c30 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.152+0000 7fbf11ffb640 1 -- 172.21.15.26:0/3772578152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf14045070 con 0x7fbf1c155c30 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.152+0000 7fbf11ffb640 1 -- 172.21.15.26:0/3772578152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14009450 con 0x7fbf1c155c30 2024-03-06T07:20:46.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.152+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c1ee330 con 0x7fbf1c155c30 2024-03-06T07:20:46.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.152+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c1ee8f0 con 0x7fbf1c155c30 2024-03-06T07:20:46.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.153+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c04f330 con 0x7fbf1c155c30 2024-03-06T07:20:46.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.153+0000 7fbf11ffb640 1 -- 172.21.15.26:0/3772578152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf14040670 con 0x7fbf1c155c30 2024-03-06T07:20:46.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.154+0000 7fbf11ffb640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf0006b500 0x7fbf0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.154+0000 7fbf11ffb640 1 -- 172.21.15.26:0/3772578152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbf140acf30 con 0x7fbf1c155c30 2024-03-06T07:20:46.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.154+0000 7fbf21785640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf0006b500 0x7fbf0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.154+0000 7fbf21785640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf0006b500 0x7fbf0006d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf18000f60 tx=0x7fbf18007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.158+0000 7fbf11ffb640 1 -- 172.21.15.26:0/3772578152 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbf1407b550 con 0x7fbf1c155c30 2024-03-06T07:20:46.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.336+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf1c06d860 con 0x7fbf1c155c30 2024-03-06T07:20:46.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.337+0000 7fbf11ffb640 1 -- 172.21.15.26:0/3772578152 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf1407f5f0 con 0x7fbf1c155c30 2024-03-06T07:20:46.339 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf0006b500 msgr2=0x7fbf0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf0006b500 0x7fbf0006d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf18000f60 tx=0x7fbf18007d70 comp rx=0 tx=0).stop 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1c155c30 msgr2=0x7fbf1c14d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1c155c30 0x7fbf1c14d750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009eb0 tx=0x7fbf14004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 shutdown_connections 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf0006b500 0x7fbf0006d9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1c155c30 0x7fbf1c14d750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c0b0070 0x7fbf1c14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 --2- 172.21.15.26:0/3772578152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c06ca70 0x7fbf1c153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 >> 172.21.15.26:0/3772578152 conn(0x7fbf1c0ac590 msgr2=0x7fbf1c0ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 shutdown_connections 2024-03-06T07:20:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.339+0000 7fbf239e9640 1 -- 172.21.15.26:0/3772578152 wait complete. 2024-03-06T07:20:46.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:46.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.454+0000 7f409da88640 1 Processor -- start 2024-03-06T07:20:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.454+0000 7f409da88640 1 -- start start 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f409da88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40980c67f0 0x7f4098155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f409da88640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40980c4ef0 0x7f4098155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f409da88640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4098152cb0 con 0x7f40981528d0 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f409da88640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4098155ca0 con 0x7f40980c4ef0 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f409da88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40980c38b0 con 0x7f40980c67f0 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4096ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40980c67f0 0x7f4098155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4097fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40980c4ef0 0x7f4098155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4097fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40980c4ef0 0x7f4098155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:46216/0 (socket says 172.21.15.26:46216) 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f40977fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f40981528d0 0x7f40980724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:47290/0 (socket says 172.21.15.26:47290) 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4096ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40980c67f0 0x7f4098155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:49688/0 (socket says 172.21.15.26:49688) 2024-03-06T07:20:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4097fff640 1 -- 172.21.15.26:0/2879607962 learned_addr learned my addr 172.21.15.26:0/2879607962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4097fff640 1 -- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40980c67f0 msgr2=0x7f4098155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4097fff640 1 --2- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40980c67f0 0x7f4098155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4097fff640 1 -- 172.21.15.26:0/2879607962 >> v1:172.21.15.117:6789/0 conn(0x7f40981528d0 legacy=0x7f40980724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4097fff640 1 -- 172.21.15.26:0/2879607962 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40981594f0 con 0x7f40980c4ef0 2024-03-06T07:20:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f40967fc640 1 -- 172.21.15.26:0/2879607962 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559219354 0 0) 0x7f4098152cb0 con 0x7f40981528d0 2024-03-06T07:20:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.455+0000 7f4096ffd640 1 --2- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40980c67f0 0x7f4098155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.456+0000 7f4097fff640 1 --2- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098155760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4090009e50 tx=0x7f409002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41064ae3042517ea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.456+0000 7f40967fc640 1 -- 172.21.15.26:0/2879607962 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409002cb90 con 0x7f40980c4ef0 2024-03-06T07:20:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.456+0000 7f40967fc640 1 -- 172.21.15.26:0/2879607962 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409002cd30 con 0x7f40980c4ef0 2024-03-06T07:20:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.456+0000 7f40967fc640 1 -- 172.21.15.26:0/2879607962 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090034610 con 0x7f40980c4ef0 2024-03-06T07:20:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 -- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 msgr2=0x7f4098155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 --2- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098155760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4090009e50 tx=0x7f409002bff0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 -- 172.21.15.26:0/2879607962 shutdown_connections 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 --2- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098155760 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 --2- 172.21.15.26:0/2879607962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40980c67f0 0x7f4098155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 -- 172.21.15.26:0/2879607962 >> 172.21.15.26:0/2879607962 conn(0x7f40980aa6a0 msgr2=0x7f40980aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 -- 172.21.15.26:0/2879607962 shutdown_connections 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.457+0000 7f409da88640 1 -- 172.21.15.26:0/2879607962 wait complete. 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.458+0000 7f409da88640 1 Processor -- start 2024-03-06T07:20:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.458+0000 7f409da88640 1 -- start start 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f409da88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f409da88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40980c67f0 0x7f409814b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f409da88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40981528d0 0x7f40981e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f409da88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40980a8250 con 0x7f40980c67f0 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f409da88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40980c3390 con 0x7f40980c4ef0 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f409da88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40981591a0 con 0x7f40981528d0 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40980c67f0 0x7f409814b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40980c67f0 0x7f409814b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46056/0 (socket says 172.21.15.26:46056) 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 -- 172.21.15.26:0/3030491039 learned_addr learned my addr 172.21.15.26:0/3030491039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f40977fe640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4097fff640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40981528d0 0x7f40981e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 -- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40981528d0 msgr2=0x7f40981e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40981528d0 0x7f40981e79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 -- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 msgr2=0x7f4098151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 -- 172.21.15.26:0/3030491039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098153310 con 0x7f40980c67f0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4097fff640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40981528d0 0x7f40981e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f40977fe640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f4096ffd640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40980c67f0 0x7f409814b110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ebe0 tx=0x7f408c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f407ffff640 1 -- 172.21.15.26:0/3030491039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c017070 con 0x7f40980c67f0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f407ffff640 1 -- 172.21.15.26:0/3030491039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00cc30 con 0x7f40980c67f0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f407ffff640 1 -- 172.21.15.26:0/3030491039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c013470 con 0x7f40980c67f0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.459+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40981ee730 con 0x7f40980c67f0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.460+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4098153030 con 0x7f40980c67f0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.460+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4098158fc0 con 0x7f40980c67f0 2024-03-06T07:20:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.461+0000 7f407ffff640 1 -- 172.21.15.26:0/3030491039 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f408c01a070 con 0x7f40980c67f0 2024-03-06T07:20:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.461+0000 7f407ffff640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f408406b4d0 0x7f408406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.461+0000 7f407ffff640 1 -- 172.21.15.26:0/3030491039 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f408c08a390 con 0x7f40980c67f0 2024-03-06T07:20:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.461+0000 7f40977fe640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f408406b4d0 0x7f408406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.462+0000 7f40977fe640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f408406b4d0 0x7f408406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4088000f60 tx=0x7f4088007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.466+0000 7f407ffff640 1 -- 172.21.15.26:0/3030491039 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4098158fc0 con 0x7f40980c67f0 2024-03-06T07:20:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.645+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f409804f330 con 0x7f40980c67f0 2024-03-06T07:20:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.645+0000 7f407ffff640 1 -- 172.21.15.26:0/3030491039 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f408c058910 con 0x7f40980c67f0 2024-03-06T07:20:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f408406b4d0 msgr2=0x7f408406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f408406b4d0 0x7f408406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4088000f60 tx=0x7f4088007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40980c67f0 msgr2=0x7f409814b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40980c67f0 0x7f409814b110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ebe0 tx=0x7f408c00c520 comp rx=0 tx=0).stop 2024-03-06T07:20:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 shutdown_connections 2024-03-06T07:20:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f408406b4d0 0x7f408406d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40981528d0 0x7f40981e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40980c67f0 0x7f409814b110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 --2- 172.21.15.26:0/3030491039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40980c4ef0 0x7f4098151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 >> 172.21.15.26:0/3030491039 conn(0x7f40980aa6a0 msgr2=0x7f40980724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 shutdown_connections 2024-03-06T07:20:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.647+0000 7f409da88640 1 -- 172.21.15.26:0/3030491039 wait complete. 2024-03-06T07:20:46.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.762+0000 7f792ab6b640 1 Processor -- start 2024-03-06T07:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f792ab6b640 1 -- start start 2024-03-06T07:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f792ab6b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f79240c3a70 0x7f79240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f792ab6b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7924156060 con 0x7f792406db30 2024-03-06T07:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f792ab6b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79240a8430 con 0x7f792406d750 2024-03-06T07:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f792ab6b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7924155b30 con 0x7f79240c3a70 2024-03-06T07:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f7929108640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f792406db30 0x7f79240c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44498/0 (socket says 172.21.15.26:44498) 2024-03-06T07:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f7929108640 1 -- 172.21.15.26:0/1357302080 learned_addr learned my addr 172.21.15.26:0/1357302080 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.763+0000 7f7923fff640 1 --2- 172.21.15.26:0/1357302080 >> v2:172.21.15.117:3300/0 conn(0x7f79240c3a70 0x7f79240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.764+0000 7f79237fe640 1 -- 172.21.15.26:0/1357302080 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713101538 0 0) 0x7f7924156060 con 0x7f792406db30 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.764+0000 7f79237fe640 1 -- 172.21.15.26:0/1357302080 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7914003520 con 0x7f792406db30 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.764+0000 7f79237fe640 1 -- 172.21.15.26:0/1357302080 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724013840 0 0) 0x7f79240a8430 con 0x7f792406d750 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.764+0000 7f79237fe640 1 -- 172.21.15.26:0/1357302080 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7924156060 con 0x7f792406d750 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.764+0000 7f7923fff640 1 -- 172.21.15.26:0/1357302080 >> v1:172.21.15.117:6789/0 conn(0x7f792406d750 legacy=0x7f7924072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.764+0000 7f7923fff640 1 -- 172.21.15.26:0/1357302080 >> v1:172.21.15.26:6789/0 conn(0x7f792406db30 legacy=0x7f79240c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.764+0000 7f7923fff640 1 -- 172.21.15.26:0/1357302080 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924155d80 con 0x7f79240c3a70 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f7923fff640 1 --2- 172.21.15.26:0/1357302080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 0x7f79240732a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79100097a0 tx=0x7f791002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7251f1d8fb36d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f79237fe640 1 -- 172.21.15.26:0/1357302080 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79100315a0 con 0x7f79240c3a70 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f79237fe640 1 -- 172.21.15.26:0/1357302080 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7910037070 con 0x7f79240c3a70 2024-03-06T07:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f79237fe640 1 -- 172.21.15.26:0/1357302080 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910031960 con 0x7f79240c3a70 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f792ab6b640 1 -- 172.21.15.26:0/1357302080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 msgr2=0x7f79240732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1357302080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 0x7f79240732a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79100097a0 tx=0x7f791002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f792ab6b640 1 -- 172.21.15.26:0/1357302080 shutdown_connections 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1357302080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 0x7f79240732a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f792ab6b640 1 -- 172.21.15.26:0/1357302080 >> 172.21.15.26:0/1357302080 conn(0x7f79240c5560 msgr2=0x7f79240c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f792ab6b640 1 -- 172.21.15.26:0/1357302080 shutdown_connections 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.765+0000 7f792ab6b640 1 -- 172.21.15.26:0/1357302080 wait complete. 2024-03-06T07:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.766+0000 7f792ab6b640 1 Processor -- start 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.766+0000 7f792ab6b640 1 -- start start 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f792ab6b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f792406d750 0x7f792414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f792ab6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f792406db30 0x7f79240acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f792ab6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 0x7f79240ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f792ab6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79240c2720 con 0x7f792406db30 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f792ab6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79240c25a0 con 0x7f79240c3a70 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f792ab6b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79240a7cd0 con 0x7f792406d750 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f7923fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f792406db30 0x7f79240acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f7923fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f792406db30 0x7f79240acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46066/0 (socket says 172.21.15.26:46066) 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f7923fff640 1 -- 172.21.15.26:0/1923961213 learned_addr learned my addr 172.21.15.26:0/1923961213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f7929108640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 0x7f79240ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.767+0000 7f7928907640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f792406d750 0x7f792414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f7923fff640 1 -- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f792406d750 msgr2=0x7f792414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f7923fff640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f792406d750 0x7f792414d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f7923fff640 1 -- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 msgr2=0x7f79240ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f7923fff640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 0x7f79240ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f7923fff640 1 -- 172.21.15.26:0/1923961213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792414b210 con 0x7f792406db30 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f7923fff640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f792406db30 0x7f79240acbc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7910034040 tx=0x7f79100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f79217fa640 1 -- 172.21.15.26:0/1923961213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910032630 con 0x7f792406db30 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f79217fa640 1 -- 172.21.15.26:0/1923961213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79100327f0 con 0x7f792406db30 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79241eda60 con 0x7f792406db30 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f79217fa640 1 -- 172.21.15.26:0/1923961213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910032be0 con 0x7f792406db30 2024-03-06T07:20:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.768+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79241ee020 con 0x7f792406db30 2024-03-06T07:20:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.769+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792404f330 con 0x7f792406db30 2024-03-06T07:20:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.769+0000 7f79217fa640 1 -- 172.21.15.26:0/1923961213 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f791004e440 con 0x7f792406db30 2024-03-06T07:20:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.770+0000 7f79217fa640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f790806b4d0 0x7f790806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.770+0000 7f79217fa640 1 -- 172.21.15.26:0/1923961213 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f79100ac0a0 con 0x7f792406db30 2024-03-06T07:20:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.770+0000 7f7928907640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f790806b4d0 0x7f790806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.770+0000 7f7928907640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f790806b4d0 0x7f790806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7918007fd0 tx=0x7f7918009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.774+0000 7f79217fa640 1 -- 172.21.15.26:0/1923961213 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7910040070 con 0x7f792406db30 2024-03-06T07:20:46.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.954+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7924074700 con 0x7f792406db30 2024-03-06T07:20:46.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.955+0000 7f79217fa640 1 -- 172.21.15.26:0/1923961213 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f791007a510 con 0x7f792406db30 2024-03-06T07:20:46.957 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:20:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.956+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f790806b4d0 msgr2=0x7f790806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.956+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f790806b4d0 0x7f790806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7918007fd0 tx=0x7f7918009040 comp rx=0 tx=0).stop 2024-03-06T07:20:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.956+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f792406db30 msgr2=0x7f79240acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.956+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f792406db30 0x7f79240acbc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7910034040 tx=0x7f79100046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 shutdown_connections 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f790806b4d0 0x7f790806d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79240c3a70 0x7f79240ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f792406db30 0x7f79240acbc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 --2- 172.21.15.26:0/1923961213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f792406d750 0x7f792414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 >> 172.21.15.26:0/1923961213 conn(0x7f79240c5560 msgr2=0x7f7924072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 shutdown_connections 2024-03-06T07:20:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:46.957+0000 7f792ab6b640 1 -- 172.21.15.26:0/1923961213 wait complete. 2024-03-06T07:20:46.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.072+0000 7fe9ba6c3640 1 Processor -- start 2024-03-06T07:20:47.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.072+0000 7fe9ba6c3640 1 -- start start 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.072+0000 7fe9ba6c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b41529f0 0x7fe9b40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.072+0000 7fe9ba6c3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe9b40c5370 0x7fe9b40c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9ba6c3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b4152dd0 con 0x7fe9b4155530 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9ba6c3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b4159710 con 0x7fe9b40c5370 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9ba6c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40c68b0 con 0x7fe9b41529f0 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b41529f0 0x7fe9b40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe9b40c5370 0x7fe9b40c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b41529f0 0x7fe9b40c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:36772/0 (socket says 172.21.15.26:36772) 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b8c60640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe9b4155530 0x7fe9b41559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:54162/0 (socket says 172.21.15.26:54162) 2024-03-06T07:20:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe9b40c5370 0x7fe9b40c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43940/0 (socket says 172.21.15.26:43940) 2024-03-06T07:20:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b3fff640 1 -- 172.21.15.26:0/2736631215 learned_addr learned my addr 172.21.15.26:0/2736631215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b3fff640 1 -- 172.21.15.26:0/2736631215 >> v2:172.21.15.117:3300/0 conn(0x7fe9b40c5370 msgr2=0x7fe9b40c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b3fff640 1 --2- 172.21.15.26:0/2736631215 >> v2:172.21.15.117:3300/0 conn(0x7fe9b40c5370 0x7fe9b40c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b3fff640 1 -- 172.21.15.26:0/2736631215 >> v1:172.21.15.117:6789/0 conn(0x7fe9b4155530 legacy=0x7fe9b41559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.073+0000 7fe9b3fff640 1 -- 172.21.15.26:0/2736631215 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b4158b00 con 0x7fe9b41529f0 2024-03-06T07:20:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.074+0000 7fe9b3fff640 1 --2- 172.21.15.26:0/2736631215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b41529f0 0x7fe9b40c88f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009870 tx=0x7fe9a802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a8bbf6d2f7c94b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.074+0000 7fe9b2ffd640 1 -- 172.21.15.26:0/2736631215 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8035020 con 0x7fe9b41529f0 2024-03-06T07:20:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.074+0000 7fe9b2ffd640 1 -- 172.21.15.26:0/2736631215 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8031640 con 0x7fe9b41529f0 2024-03-06T07:20:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.074+0000 7fe9b2ffd640 1 -- 172.21.15.26:0/2736631215 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a802d040 con 0x7fe9b41529f0 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.074+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/2736631215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b41529f0 msgr2=0x7fe9b40c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.074+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/2736631215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b41529f0 0x7fe9b40c88f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009870 tx=0x7fe9a802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.075+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/2736631215 shutdown_connections 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.075+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/2736631215 >> v2:172.21.15.117:3300/0 conn(0x7fe9b40c5370 0x7fe9b40c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.075+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/2736631215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b41529f0 0x7fe9b40c88f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.075+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/2736631215 >> 172.21.15.26:0/2736631215 conn(0x7fe9b40ac580 msgr2=0x7fe9b40ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.075+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/2736631215 shutdown_connections 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.075+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/2736631215 wait complete. 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 Processor -- start 2024-03-06T07:20:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 -- start start 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b40c5370 0x7fe9b41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b41529f0 0x7fe9b4156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9b4155530 0x7fe9b4156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b41593a0 con 0x7fe9b41529f0 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b41590c0 con 0x7fe9b4155530 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.076+0000 7fe9ba6c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b41560d0 con 0x7fe9b40c5370 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b41529f0 0x7fe9b4156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b41529f0 0x7fe9b4156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58290/0 (socket says 172.21.15.26:58290) 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 -- 172.21.15.26:0/1872168474 learned_addr learned my addr 172.21.15.26:0/1872168474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b3fff640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b40c5370 0x7fe9b41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b8c60640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9b4155530 0x7fe9b4156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 -- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b40c5370 msgr2=0x7fe9b41543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b40c5370 0x7fe9b41543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 -- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9b4155530 msgr2=0x7fe9b4156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9b4155530 0x7fe9b4156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 -- 172.21.15.26:0/1872168474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b41eaf50 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b3fff640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b40c5370 0x7fe9b41543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b37fe640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b41529f0 0x7fe9b4156980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e9b0 tx=0x7fe9a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b0ff9640 1 -- 172.21.15.26:0/1872168474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4017070 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b0ff9640 1 -- 172.21.15.26:0/1872168474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a400c9a0 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b41eb230 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9b0ff9640 1 -- 172.21.15.26:0/1872168474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400cd90 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.077+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b41eb7f0 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.078+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b404f330 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.079+0000 7fe9b0ff9640 1 -- 172.21.15.26:0/1872168474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9a401a070 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.079+0000 7fe9b0ff9640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe9a006b4d0 0x7fe9a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.079+0000 7fe9b0ff9640 1 -- 172.21.15.26:0/1872168474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9a408a790 con 0x7fe9b41529f0 2024-03-06T07:20:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.079+0000 7fe9b3fff640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe9a006b4d0 0x7fe9a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.080+0000 7fe9b3fff640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe9a006b4d0 0x7fe9a006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8036000 tx=0x7fe9a803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.083+0000 7fe9b0ff9640 1 -- 172.21.15.26:0/1872168474 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe9a4058d10 con 0x7fe9b41529f0 2024-03-06T07:20:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.262+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe9b4068e50 con 0x7fe9b41529f0 2024-03-06T07:20:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.262+0000 7fe9b0ff9640 1 -- 172.21.15.26:0/1872168474 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe9a405cdb0 con 0x7fe9b41529f0 2024-03-06T07:20:47.264 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe9a006b4d0 msgr2=0x7fe9a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe9a006b4d0 0x7fe9a006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8036000 tx=0x7fe9a803c040 comp rx=0 tx=0).stop 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b41529f0 msgr2=0x7fe9b4156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b41529f0 0x7fe9b4156980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e9b0 tx=0x7fe9a400ee70 comp rx=0 tx=0).stop 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 shutdown_connections 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe9a006b4d0 0x7fe9a006d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9b4155530 0x7fe9b4156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b41529f0 0x7fe9b4156980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 --2- 172.21.15.26:0/1872168474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9b40c5370 0x7fe9b41543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 >> 172.21.15.26:0/1872168474 conn(0x7fe9b40ac580 msgr2=0x7fe9b40ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 shutdown_connections 2024-03-06T07:20:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.264+0000 7fe9ba6c3640 1 -- 172.21.15.26:0/1872168474 wait complete. 2024-03-06T07:20:47.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.377+0000 7f4033edf640 1 Processor -- start 2024-03-06T07:20:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.377+0000 7f4033edf640 1 -- start start 2024-03-06T07:20:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4033edf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f402c06da40 0x7f402c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4033edf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f402c0c6780 0x7f402c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4033edf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c156050 con 0x7f402c1592d0 2024-03-06T07:20:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4033edf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c06de20 con 0x7f402c06da40 2024-03-06T07:20:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4033edf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c072630 con 0x7f402c0c6780 2024-03-06T07:20:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4031c7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f402c06da40 0x7f402c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f403147a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f402c0c6780 0x7f402c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4031c7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f402c06da40 0x7f402c155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58298/0 (socket says 172.21.15.26:58298) 2024-03-06T07:20:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4031c7b640 1 -- 172.21.15.26:0/1839640191 learned_addr learned my addr 172.21.15.26:0/1839640191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4030c79640 1 -- 172.21.15.26:0/1839640191 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475995101 0 0) 0x7f4028001490 con 0x7f402c1592d0 2024-03-06T07:20:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4031c7b640 1 -- 172.21.15.26:0/1839640191 >> v2:172.21.15.117:3300/0 conn(0x7f402c0c6780 msgr2=0x7f402c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:47.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4031c7b640 1 --2- 172.21.15.26:0/1839640191 >> v2:172.21.15.117:3300/0 conn(0x7f402c0c6780 0x7f402c158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4031c7b640 1 -- 172.21.15.26:0/1839640191 >> v1:172.21.15.117:6789/0 conn(0x7f402c1592d0 legacy=0x7f402c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.378+0000 7f4031c7b640 1 -- 172.21.15.26:0/1839640191 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c072910 con 0x7f402c06da40 2024-03-06T07:20:47.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.379+0000 7f4031c7b640 1 --2- 172.21.15.26:0/1839640191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c155b10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f402c0c7120 tx=0x7f402402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31323d3aee0ee39 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.379+0000 7f4030c79640 1 -- 172.21.15.26:0/1839640191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024035020 con 0x7f402c06da40 2024-03-06T07:20:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.379+0000 7f4030c79640 1 -- 172.21.15.26:0/1839640191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4024031870 con 0x7f402c06da40 2024-03-06T07:20:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.379+0000 7f4030c79640 1 -- 172.21.15.26:0/1839640191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024031c40 con 0x7f402c06da40 2024-03-06T07:20:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 -- 172.21.15.26:0/1839640191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 msgr2=0x7f402c155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 --2- 172.21.15.26:0/1839640191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c155b10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f402c0c7120 tx=0x7f402402b910 comp rx=0 tx=0).stop 2024-03-06T07:20:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 -- 172.21.15.26:0/1839640191 shutdown_connections 2024-03-06T07:20:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 --2- 172.21.15.26:0/1839640191 >> v2:172.21.15.117:3300/0 conn(0x7f402c0c6780 0x7f402c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 --2- 172.21.15.26:0/1839640191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c155b10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 -- 172.21.15.26:0/1839640191 >> 172.21.15.26:0/1839640191 conn(0x7f402c0aa630 msgr2=0x7f402c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 -- 172.21.15.26:0/1839640191 shutdown_connections 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.380+0000 7f4033edf640 1 -- 172.21.15.26:0/1839640191 wait complete. 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 Processor -- start 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 -- start start 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c1eec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f402c0c6780 0x7f402c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c1592d0 0x7f402c0ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0c3990 con 0x7f402c06da40 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c158890 con 0x7f402c1592d0 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.381+0000 7f4033edf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0c3b10 con 0x7f402c0c6780 2024-03-06T07:20:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c1eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c1eec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58300/0 (socket says 172.21.15.26:58300) 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 -- 172.21.15.26:0/3623737459 learned_addr learned my addr 172.21.15.26:0/3623737459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f403247c640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c1592d0 0x7f402c0ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f403147a640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f402c0c6780 0x7f402c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 -- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f402c0c6780 msgr2=0x7f402c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f402c0c6780 0x7f402c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 -- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c1592d0 msgr2=0x7f402c0ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c1592d0 0x7f402c0ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 -- 172.21.15.26:0/3623737459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c0aeb50 con 0x7f402c06da40 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4031c7b640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c1eec20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4024000c00 tx=0x7f40240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f40227fc640 1 -- 172.21.15.26:0/3623737459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024041070 con 0x7f402c06da40 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f40227fc640 1 -- 172.21.15.26:0/3623737459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402403ad00 con 0x7f402c06da40 2024-03-06T07:20:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f40227fc640 1 -- 172.21.15.26:0/3623737459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024035040 con 0x7f402c06da40 2024-03-06T07:20:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c1ee140 con 0x7f402c06da40 2024-03-06T07:20:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.382+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c154700 con 0x7f402c06da40 2024-03-06T07:20:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.383+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c04f330 con 0x7f402c06da40 2024-03-06T07:20:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.384+0000 7f40227fc640 1 -- 172.21.15.26:0/3623737459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4024038020 con 0x7f402c06da40 2024-03-06T07:20:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.384+0000 7f40227fc640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f401006b4d0 0x7f401006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.384+0000 7f40227fc640 1 -- 172.21.15.26:0/3623737459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f40240ac280 con 0x7f402c06da40 2024-03-06T07:20:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.384+0000 7f403147a640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f401006b4d0 0x7f401006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.385+0000 7f403147a640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f401006b4d0 0x7f401006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f401c009b40 tx=0x7f401c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.388+0000 7f40227fc640 1 -- 172.21.15.26:0/3623737459 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f402407a770 con 0x7f402c06da40 2024-03-06T07:20:47.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.608+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f402c073240 con 0x7f402c06da40 2024-03-06T07:20:47.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.609+0000 7f40227fc640 1 -- 172.21.15.26:0/3623737459 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7f402407e810 con 0x7f402c06da40 2024-03-06T07:20:47.610 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.610+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f401006b4d0 msgr2=0x7f401006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.610+0000 7f4033edf640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f401006b4d0 0x7f401006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f401c009b40 tx=0x7f401c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.610+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 msgr2=0x7f402c1eec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.610+0000 7f4033edf640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c1eec20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4024000c00 tx=0x7f40240046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 shutdown_connections 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f401006b4d0 0x7f401006d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c1592d0 0x7f402c0ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f402c0c6780 0x7f402c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 --2- 172.21.15.26:0/3623737459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c06da40 0x7f402c1eec20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 >> 172.21.15.26:0/3623737459 conn(0x7f402c0aa630 msgr2=0x7f402c0c7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 shutdown_connections 2024-03-06T07:20:47.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.611+0000 7f4033edf640 1 -- 172.21.15.26:0/3623737459 wait complete. 2024-03-06T07:20:47.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:47.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.723+0000 7f011c5ac640 1 Processor -- start 2024-03-06T07:20:47.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.723+0000 7f011c5ac640 1 -- start start 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f011c5ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0114152920 0x7f0114155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f011c5ac640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01140c2620 0x7f01140c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f011c5ac640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011406afc0 con 0x7f01140c2190 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f011c5ac640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0114156160 con 0x7f01140c2620 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f011c5ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011406b2a0 con 0x7f0114152920 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f011ab49640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01140c2620 0x7f01140c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f0119b47640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01140c2190 0x7f01140c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43918/0 (socket says 172.21.15.26:43918) 2024-03-06T07:20:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f0119b47640 1 -- 172.21.15.26:0/4069250622 learned_addr learned my addr 172.21.15.26:0/4069250622 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.724+0000 7f011a348640 1 --2- 172.21.15.26:0/4069250622 >> v2:172.21.15.117:3300/0 conn(0x7f0114152920 0x7f0114155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.725+0000 7f011ab49640 1 -- 172.21.15.26:0/4069250622 >> v2:172.21.15.117:3300/0 conn(0x7f0114152920 msgr2=0x7f0114155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.725+0000 7f011ab49640 1 --2- 172.21.15.26:0/4069250622 >> v2:172.21.15.117:3300/0 conn(0x7f0114152920 0x7f0114155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.725+0000 7f011ab49640 1 -- 172.21.15.26:0/4069250622 >> v1:172.21.15.26:6789/0 conn(0x7f01140c2190 legacy=0x7f01140c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.725+0000 7f0119346640 1 -- 172.21.15.26:0/4069250622 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657631303 0 0) 0x7f011406afc0 con 0x7f01140c2190 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.725+0000 7f011ab49640 1 -- 172.21.15.26:0/4069250622 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011406b580 con 0x7f01140c2620 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.725+0000 7f011ab49640 1 --2- 172.21.15.26:0/4069250622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01140c2620 0x7f01140c1990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0114070a90 tx=0x7f010c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d9608dd940a430 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.725+0000 7f0119346640 1 -- 172.21.15.26:0/4069250622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c039070 con 0x7f01140c2620 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f0119346640 1 -- 172.21.15.26:0/4069250622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c02cc80 con 0x7f01140c2620 2024-03-06T07:20:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f0119346640 1 -- 172.21.15.26:0/4069250622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c034660 con 0x7f01140c2620 2024-03-06T07:20:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 -- 172.21.15.26:0/4069250622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01140c2620 msgr2=0x7f01140c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 --2- 172.21.15.26:0/4069250622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01140c2620 0x7f01140c1990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0114070a90 tx=0x7f010c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 -- 172.21.15.26:0/4069250622 shutdown_connections 2024-03-06T07:20:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 --2- 172.21.15.26:0/4069250622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01140c2620 0x7f01140c1990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 --2- 172.21.15.26:0/4069250622 >> v2:172.21.15.117:3300/0 conn(0x7f0114152920 0x7f0114155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 -- 172.21.15.26:0/4069250622 >> 172.21.15.26:0/4069250622 conn(0x7f01140c4290 msgr2=0x7f01140c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 -- 172.21.15.26:0/4069250622 shutdown_connections 2024-03-06T07:20:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.726+0000 7f011c5ac640 1 -- 172.21.15.26:0/4069250622 wait complete. 2024-03-06T07:20:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.727+0000 7f011c5ac640 1 Processor -- start 2024-03-06T07:20:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.727+0000 7f011c5ac640 1 -- start start 2024-03-06T07:20:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011c5ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01140c2190 0x7f0114072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011c5ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01140c2620 0x7f0114147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011c5ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0114152920 0x7f0114147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011c5ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01140a7100 con 0x7f0114152920 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011c5ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01140c0d60 con 0x7f01140c2190 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011c5ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01140a7280 con 0x7f01140c2620 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0114152920 0x7f0114147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0114152920 0x7f0114147c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58318/0 (socket says 172.21.15.26:58318) 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 -- 172.21.15.26:0/2805103445 learned_addr learned my addr 172.21.15.26:0/2805103445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f0119b47640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01140c2620 0x7f0114147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011a348640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01140c2190 0x7f0114072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 -- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01140c2620 msgr2=0x7f0114147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01140c2620 0x7f0114147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 -- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01140c2190 msgr2=0x7f0114072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01140c2190 0x7f0114072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.728+0000 7f011ab49640 1 -- 172.21.15.26:0/2805103445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0114151d90 con 0x7f0114152920 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.729+0000 7f011ab49640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0114152920 0x7f0114147c60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f010c009e80 tx=0x7f010c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.729+0000 7f0112ffd640 1 -- 172.21.15.26:0/2805103445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c044020 con 0x7f0114152920 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.729+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0114156710 con 0x7f0114152920 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.729+0000 7f0112ffd640 1 -- 172.21.15.26:0/2805103445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c03a040 con 0x7f0114152920 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.729+0000 7f0112ffd640 1 -- 172.21.15.26:0/2805103445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c039050 con 0x7f0114152920 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.729+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0114072ac0 con 0x7f0114152920 2024-03-06T07:20:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.730+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0114073620 con 0x7f0114152920 2024-03-06T07:20:47.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.730+0000 7f0112ffd640 1 -- 172.21.15.26:0/2805103445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f010c003b90 con 0x7f0114152920 2024-03-06T07:20:47.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.731+0000 7f0112ffd640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b4d0 0x7f010006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.731+0000 7f0112ffd640 1 -- 172.21.15.26:0/2805103445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f010c0acb30 con 0x7f0114152920 2024-03-06T07:20:47.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.731+0000 7f011a348640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b4d0 0x7f010006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.731+0000 7f011a348640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b4d0 0x7f010006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0104009aa0 tx=0x7f0104009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.735+0000 7f0112ffd640 1 -- 172.21.15.26:0/2805103445 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0114073620 con 0x7f0114152920 2024-03-06T07:20:47.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.917+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f011404f330 con 0x7f0114152920 2024-03-06T07:20:47.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.918+0000 7f0112ffd640 1 -- 172.21.15.26:0/2805103445 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f010c04e020 con 0x7f0114152920 2024-03-06T07:20:47.920 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:47.920 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b4d0 msgr2=0x7f010006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b4d0 0x7f010006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0104009aa0 tx=0x7f0104009190 comp rx=0 tx=0).stop 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0114152920 msgr2=0x7f0114147c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0114152920 0x7f0114147c60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f010c009e80 tx=0x7f010c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 shutdown_connections 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b4d0 0x7f010006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0114152920 0x7f0114147c60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01140c2620 0x7f0114147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 --2- 172.21.15.26:0/2805103445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01140c2190 0x7f0114072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 >> 172.21.15.26:0/2805103445 conn(0x7f01140c4290 msgr2=0x7f01140c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 shutdown_connections 2024-03-06T07:20:47.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:47.920+0000 7f011c5ac640 1 -- 172.21.15.26:0/2805103445 wait complete. 2024-03-06T07:20:47.933 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:20:47.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.033+0000 7f1462a84640 1 Processor -- start 2024-03-06T07:20:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.033+0000 7f1462a84640 1 -- start start 2024-03-06T07:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.033+0000 7f1462a84640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f145c072e70 0x7f145c0bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.033+0000 7f1462a84640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145c06c8e0 con 0x7f145c073250 2024-03-06T07:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.033+0000 7f1462a84640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145c06ce80 con 0x7f145c06e490 2024-03-06T07:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.033+0000 7f1462a84640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145c0c1b00 con 0x7f145c072e70 2024-03-06T07:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f145bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f145c073250 0x7f145c0c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43924/0 (socket says 172.21.15.26:43924) 2024-03-06T07:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f145bfff640 1 -- 172.21.15.26:0/2918440843 learned_addr learned my addr 172.21.15.26:0/2918440843 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f1460820640 1 --2- 172.21.15.26:0/2918440843 >> v2:172.21.15.117:3300/0 conn(0x7f145c072e70 0x7f145c0bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f145b7fe640 1 -- 172.21.15.26:0/2918440843 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906543619 0 0) 0x7f145c06c8e0 con 0x7f145c073250 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f145b7fe640 1 -- 172.21.15.26:0/2918440843 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003520 con 0x7f145c073250 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f1460820640 1 -- 172.21.15.26:0/2918440843 >> v1:172.21.15.117:6789/0 conn(0x7f145c06e490 legacy=0x7f145c15fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f145b7fe640 1 -- 172.21.15.26:0/2918440843 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209443125 0 0) 0x7f145c06ce80 con 0x7f145c06e490 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f1460820640 1 -- 172.21.15.26:0/2918440843 >> v1:172.21.15.26:6789/0 conn(0x7f145c073250 legacy=0x7f145c0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.034+0000 7f1460820640 1 -- 172.21.15.26:0/2918440843 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c06cba0 con 0x7f145c072e70 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.035+0000 7f1460820640 1 --2- 172.21.15.26:0/2918440843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c072e70 0x7f145c0bd510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f14500097a0 tx=0x7f145002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f546675b6d4180ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.035+0000 7f145b7fe640 1 -- 172.21.15.26:0/2918440843 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450035020 con 0x7f145c072e70 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.035+0000 7f145b7fe640 1 -- 172.21.15.26:0/2918440843 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450031830 con 0x7f145c072e70 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.035+0000 7f145b7fe640 1 -- 172.21.15.26:0/2918440843 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450031be0 con 0x7f145c072e70 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.035+0000 7f1462a84640 1 -- 172.21.15.26:0/2918440843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c072e70 msgr2=0x7f145c0bd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.035+0000 7f1462a84640 1 --2- 172.21.15.26:0/2918440843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c072e70 0x7f145c0bd510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f14500097a0 tx=0x7f145002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.036+0000 7f1462a84640 1 -- 172.21.15.26:0/2918440843 shutdown_connections 2024-03-06T07:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.036+0000 7f1462a84640 1 --2- 172.21.15.26:0/2918440843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c072e70 0x7f145c0bd510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.036+0000 7f1462a84640 1 -- 172.21.15.26:0/2918440843 >> 172.21.15.26:0/2918440843 conn(0x7f145c06b1a0 msgr2=0x7f145c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.036+0000 7f1462a84640 1 -- 172.21.15.26:0/2918440843 shutdown_connections 2024-03-06T07:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.036+0000 7f1462a84640 1 -- 172.21.15.26:0/2918440843 wait complete. 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.037+0000 7f1462a84640 1 Processor -- start 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.037+0000 7f1462a84640 1 -- start start 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.037+0000 7f1462a84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c06e490 0x7f145c0abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.037+0000 7f1462a84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f145c072e70 0x7f145c0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.037+0000 7f1462a84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f145c073250 0x7f145c146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.037+0000 7f1462a84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0c6760 con 0x7f145c072e70 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.037+0000 7f1462a84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0c4a00 con 0x7f145c06e490 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f1462a84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c154080 con 0x7f145c073250 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f145c072e70 0x7f145c0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f145c072e70 0x7f145c0aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58322/0 (socket says 172.21.15.26:58322) 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 -- 172.21.15.26:0/2513631410 learned_addr learned my addr 172.21.15.26:0/2513631410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f1460820640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c06e490 0x7f145c0abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f1461021640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f145c073250 0x7f145c146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 -- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f145c073250 msgr2=0x7f145c146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f145c073250 0x7f145c146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 -- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c06e490 msgr2=0x7f145c0abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c06e490 0x7f145c0abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 -- 172.21.15.26:0/2513631410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0a78f0 con 0x7f145c072e70 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f145bfff640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f145c072e70 0x7f145c0aad30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1448001eb0 tx=0x7f144800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f14597fa640 1 -- 172.21.15.26:0/2513631410 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448017070 con 0x7f145c072e70 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f14597fa640 1 -- 172.21.15.26:0/2513631410 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448005d00 con 0x7f145c072e70 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f14597fa640 1 -- 172.21.15.26:0/2513631410 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448007310 con 0x7f145c072e70 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c0a7bd0 con 0x7f145c072e70 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.038+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c0c0dc0 con 0x7f145c072e70 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.039+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145c04f330 con 0x7f145c072e70 2024-03-06T07:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.040+0000 7f14597fa640 1 -- 172.21.15.26:0/2513631410 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f144801a070 con 0x7f145c072e70 2024-03-06T07:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.040+0000 7f14597fa640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f144006b4d0 0x7f144006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.040+0000 7f14597fa640 1 -- 172.21.15.26:0/2513631410 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f14480897d0 con 0x7f145c072e70 2024-03-06T07:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.040+0000 7f1460820640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f144006b4d0 0x7f144006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.041+0000 7f1460820640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f144006b4d0 0x7f144006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1450036000 tx=0x7f1450008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.044+0000 7f14597fa640 1 -- 172.21.15.26:0/2513631410 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1448057d50 con 0x7f145c072e70 2024-03-06T07:20:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.267+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f145c06ce80 con 0x7f145c072e70 2024-03-06T07:20:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.268+0000 7f14597fa640 1 -- 172.21.15.26:0/2513631410 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7f144805bdf0 con 0x7f145c072e70 2024-03-06T07:20:48.270 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:48.270 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f144006b4d0 msgr2=0x7f144006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f144006b4d0 0x7f144006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1450036000 tx=0x7f1450008030 comp rx=0 tx=0).stop 2024-03-06T07:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f145c072e70 msgr2=0x7f145c0aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f145c072e70 0x7f145c0aad30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1448001eb0 tx=0x7f144800c820 comp rx=0 tx=0).stop 2024-03-06T07:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 shutdown_connections 2024-03-06T07:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f144006b4d0 0x7f144006d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f145c073250 0x7f145c146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f145c072e70 0x7f145c0aad30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 --2- 172.21.15.26:0/2513631410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f145c06e490 0x7f145c0abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 >> 172.21.15.26:0/2513631410 conn(0x7f145c06b1a0 msgr2=0x7f145c0bc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 shutdown_connections 2024-03-06T07:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.270+0000 7f1462a84640 1 -- 172.21.15.26:0/2513631410 wait complete. 2024-03-06T07:20:48.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.383+0000 7f6843efb640 1 Processor -- start 2024-03-06T07:20:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.383+0000 7f6843efb640 1 -- start start 2024-03-06T07:20:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6843efb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f683c06da40 0x7f683c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6843efb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6843efb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c156050 con 0x7f683c0c6780 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6843efb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c06de20 con 0x7f683c06da40 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6843efb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c072630 con 0x7f683c0c7380 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6841c97640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f683c06da40 0x7f683c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6841c97640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f683c06da40 0x7f683c155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58328/0 (socket says 172.21.15.26:58328) 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6841c97640 1 -- 172.21.15.26:0/1054461636 learned_addr learned my addr 172.21.15.26:0/1054461636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6842498640 1 --2- 172.21.15.26:0/1054461636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6841c97640 1 -- 172.21.15.26:0/1054461636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 msgr2=0x7f683c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6841c97640 1 --2- 172.21.15.26:0/1054461636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6841c97640 1 -- 172.21.15.26:0/1054461636 >> v1:172.21.15.117:6789/0 conn(0x7f683c0c6780 legacy=0x7f683c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.384+0000 7f6841c97640 1 -- 172.21.15.26:0/1054461636 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c072910 con 0x7f683c06da40 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.385+0000 7f6841c97640 1 --2- 172.21.15.26:0/1054461636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c06da40 0x7f683c155b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6834009870 tx=0x7f683402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dee8df379e448 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.385+0000 7f6840c95640 1 -- 172.21.15.26:0/1054461636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834035020 con 0x7f683c06da40 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.385+0000 7f6840c95640 1 -- 172.21.15.26:0/1054461636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6834031640 con 0x7f683c06da40 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.385+0000 7f6840c95640 1 -- 172.21.15.26:0/1054461636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683402d040 con 0x7f683c06da40 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 -- 172.21.15.26:0/1054461636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c06da40 msgr2=0x7f683c155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 --2- 172.21.15.26:0/1054461636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c06da40 0x7f683c155b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6834009870 tx=0x7f683402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 -- 172.21.15.26:0/1054461636 shutdown_connections 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 --2- 172.21.15.26:0/1054461636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 --2- 172.21.15.26:0/1054461636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c06da40 0x7f683c155b10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 -- 172.21.15.26:0/1054461636 >> 172.21.15.26:0/1054461636 conn(0x7f683c0aa630 msgr2=0x7f683c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 -- 172.21.15.26:0/1054461636 shutdown_connections 2024-03-06T07:20:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.386+0000 7f6843efb640 1 -- 172.21.15.26:0/1054461636 wait complete. 2024-03-06T07:20:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.387+0000 7f6843efb640 1 Processor -- start 2024-03-06T07:20:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.387+0000 7f6843efb640 1 -- start start 2024-03-06T07:20:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c06da40 0x7f683c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0c6780 0x7f683c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c1eea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c1594a0 con 0x7f683c0c6780 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c158890 con 0x7f683c06da40 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c37c0 con 0x7f683c0c7380 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0c6780 0x7f683c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0c6780 0x7f683c154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58344/0 (socket says 172.21.15.26:58344) 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 -- 172.21.15.26:0/1981793681 learned_addr learned my addr 172.21.15.26:0/1981793681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6842498640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c1eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841c97640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c06da40 0x7f683c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 -- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 msgr2=0x7f683c1eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c1eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 -- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c06da40 msgr2=0x7f683c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c06da40 0x7f683c156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 -- 172.21.15.26:0/1981793681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c0adef0 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6841496640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0c6780 0x7f683c154310 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f683000c430 tx=0x7f683000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f683a7fc640 1 -- 172.21.15.26:0/1981793681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68300052d0 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f683a7fc640 1 -- 172.21.15.26:0/1981793681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6830005490 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c0ae440 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f683a7fc640 1 -- 172.21.15.26:0/1981793681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830007260 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.388+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c1ee250 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.389+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c04f330 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.390+0000 7f683a7fc640 1 -- 172.21.15.26:0/1981793681 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6830007400 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.390+0000 7f683a7fc640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682806b4d0 0x7f682806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.390+0000 7f683a7fc640 1 -- 172.21.15.26:0/1981793681 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f68300898e0 con 0x7f683c0c6780 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.390+0000 7f6841c97640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682806b4d0 0x7f682806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.391+0000 7f6841c97640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682806b4d0 0x7f682806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6834036000 tx=0x7f683403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.394+0000 7f683a7fc640 1 -- 172.21.15.26:0/1981793681 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f683c156050 con 0x7f683c0c6780 2024-03-06T07:20:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.615+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f683c0aec10 con 0x7f683c0c6780 2024-03-06T07:20:48.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.615+0000 7f683a7fc640 1 -- 172.21.15.26:0/1981793681 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4973 (secure 0 0 0) 0x7f683005ce20 con 0x7f683c0c6780 2024-03-06T07:20:48.617 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:48.617 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/327946761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":327946761},{"type":"v1","addr":"172.21.15.117:6836","nonce":327946761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1582279868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1582279868},{"type":"v1","addr":"172.21.15.26:6834","nonce":1582279868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/703312428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":703312428},{"type":"v1","addr":"172.21.15.26:6835","nonce":703312428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:01.171153+0000","modified":"2024-03-06T07:20:17.459497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3090618513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3090618513},{"type":"v1","addr":"172.21.15.117:6837","nonce":3090618513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-06T07:20:48.617 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-03-06T07:20:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682806b4d0 msgr2=0x7f682806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682806b4d0 0x7f682806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6834036000 tx=0x7f683403c040 comp rx=0 tx=0).stop 2024-03-06T07:20:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0c6780 msgr2=0x7f683c154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0c6780 0x7f683c154310 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f683000c430 tx=0x7f683000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 shutdown_connections 2024-03-06T07:20:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682806b4d0 0x7f682806d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0c7380 0x7f683c1eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0c6780 0x7f683c154310 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 --2- 172.21.15.26:0/1981793681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c06da40 0x7f683c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 >> 172.21.15.26:0/1981793681 conn(0x7f683c0aa630 msgr2=0x7f683c0c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 shutdown_connections 2024-03-06T07:20:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.617+0000 7f6843efb640 1 -- 172.21.15.26:0/1981793681 wait complete. 2024-03-06T07:20:48.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:48.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.731+0000 7f24b6684640 1 Processor -- start 2024-03-06T07:20:48.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.731+0000 7f24b6684640 1 -- start start 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b6684640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24b00c7380 0x7f24b00c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b6684640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b0072630 con 0x7f24b00c6780 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b6684640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b0072910 con 0x7f24b006da40 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b6684640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b006de20 con 0x7f24b00c7380 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b4c21640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24b00c7380 0x7f24b00c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24af7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f24b00c6780 0x7f24b0158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43930/0 (socket says 172.21.15.26:43930) 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24af7fe640 1 -- 172.21.15.26:0/4072208036 learned_addr learned my addr 172.21.15.26:0/4072208036 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b4c21640 1 -- 172.21.15.26:0/4072208036 >> v1:172.21.15.117:6789/0 conn(0x7f24b006da40 legacy=0x7f24b0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b4c21640 1 -- 172.21.15.26:0/4072208036 >> v1:172.21.15.26:6789/0 conn(0x7f24b00c6780 legacy=0x7f24b0158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.732+0000 7f24b4c21640 1 -- 172.21.15.26:0/4072208036 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0072bf0 con 0x7f24b00c7380 2024-03-06T07:20:48.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b4c21640 1 --2- 172.21.15.26:0/4072208036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b00c3280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009d90 tx=0x7f24a802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb58cf5039c53e32 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24aeffd640 1 -- 172.21.15.26:0/4072208036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8039070 con 0x7f24b00c7380 2024-03-06T07:20:48.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24aeffd640 1 -- 172.21.15.26:0/4072208036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a802cc60 con 0x7f24b00c7380 2024-03-06T07:20:48.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24aeffd640 1 -- 172.21.15.26:0/4072208036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8034660 con 0x7f24b00c7380 2024-03-06T07:20:48.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b6684640 1 -- 172.21.15.26:0/4072208036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 msgr2=0x7f24b00c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b6684640 1 --2- 172.21.15.26:0/4072208036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b00c3280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009d90 tx=0x7f24a802beb0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b6684640 1 -- 172.21.15.26:0/4072208036 shutdown_connections 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b6684640 1 --2- 172.21.15.26:0/4072208036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b00c3280 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b6684640 1 -- 172.21.15.26:0/4072208036 >> 172.21.15.26:0/4072208036 conn(0x7f24b00aa630 msgr2=0x7f24b00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b6684640 1 -- 172.21.15.26:0/4072208036 shutdown_connections 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.733+0000 7f24b6684640 1 -- 172.21.15.26:0/4072208036 wait complete. 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.734+0000 7f24b6684640 1 Processor -- start 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b6684640 1 -- start start 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b6684640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24b006da40 0x7f24b01ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b6684640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b00c6780 0x7f24b01531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b6684640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b0150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b6684640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00c37c0 con 0x7f24b00c7380 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b6684640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01594a0 con 0x7f24b00c6780 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b6684640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00c3940 con 0x7f24b006da40 2024-03-06T07:20:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b4c21640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b0150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b4c21640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b0150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58364/0 (socket says 172.21.15.26:58364) 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.735+0000 7f24b4c21640 1 -- 172.21.15.26:0/2111147254 learned_addr learned my addr 172.21.15.26:0/2111147254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24affff640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24b006da40 0x7f24b01ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24af7fe640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b00c6780 0x7f24b01531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b4c21640 1 -- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24b006da40 msgr2=0x7f24b01ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b4c21640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24b006da40 0x7f24b01ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b4c21640 1 -- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b00c6780 msgr2=0x7f24b01531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b4c21640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b00c6780 0x7f24b01531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b4c21640 1 -- 172.21.15.26:0/2111147254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b01eecd0 con 0x7f24b00c7380 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24af7fe640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b00c6780 0x7f24b01531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b4c21640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b0150d00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009eb0 tx=0x7f24a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24acff9640 1 -- 172.21.15.26:0/2111147254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8039050 con 0x7f24b00c7380 2024-03-06T07:20:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24acff9640 1 -- 172.21.15.26:0/2111147254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8045070 con 0x7f24b00c7380 2024-03-06T07:20:48.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b00adb00 con 0x7f24b00c7380 2024-03-06T07:20:48.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24acff9640 1 -- 172.21.15.26:0/2111147254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a803e850 con 0x7f24b00c7380 2024-03-06T07:20:48.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.736+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b00ae0c0 con 0x7f24b00c7380 2024-03-06T07:20:48.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.737+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b004f330 con 0x7f24b00c7380 2024-03-06T07:20:48.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.738+0000 7f24acff9640 1 -- 172.21.15.26:0/2111147254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24a803e9f0 con 0x7f24b00c7380 2024-03-06T07:20:48.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.738+0000 7f24acff9640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f249c06b4d0 0x7f249c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.738+0000 7f24acff9640 1 -- 172.21.15.26:0/2111147254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f24a80ad190 con 0x7f24b00c7380 2024-03-06T07:20:48.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.738+0000 7f24affff640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f249c06b4d0 0x7f249c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.739+0000 7f24affff640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f249c06b4d0 0x7f249c06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a40016e0 tx=0x7f24a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.742+0000 7f24acff9640 1 -- 172.21.15.26:0/2111147254 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f24a807bf40 con 0x7f24b00c7380 2024-03-06T07:20:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.924+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24b0158580 con 0x7f24b00c7380 2024-03-06T07:20:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.926+0000 7f24acff9640 1 -- 172.21.15.26:0/2111147254 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f24a807b840 con 0x7f24b00c7380 2024-03-06T07:20:48.928 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:48.928 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:06.210219+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:19:59.660648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:00.133540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:03.850046+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f249c06b4d0 msgr2=0x7f249c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f249c06b4d0 0x7f249c06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a40016e0 tx=0x7f24a4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 msgr2=0x7f24b0150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b0150d00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009eb0 tx=0x7f24a8004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 shutdown_connections 2024-03-06T07:20:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f249c06b4d0 0x7f249c06d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b00c7380 0x7f24b0150d00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b00c6780 0x7f24b01531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 --2- 172.21.15.26:0/2111147254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24b006da40 0x7f24b01ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 >> 172.21.15.26:0/2111147254 conn(0x7f24b00aa630 msgr2=0x7f24b00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 shutdown_connections 2024-03-06T07:20:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:48.928+0000 7f24b6684640 1 -- 172.21.15.26:0/2111147254 wait complete. 2024-03-06T07:20:48.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:20:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.041+0000 7f11a2108640 1 Processor -- start 2024-03-06T07:20:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.041+0000 7f11a2108640 1 -- start start 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f11a2108640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f119c0c6d70 0x7f119c160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f11a2108640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c0c71e0 con 0x7f119c155120 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f11a2108640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c159590 con 0x7f119c0bfe90 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f11a2108640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c158710 con 0x7f119c0c6d70 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f119c155120 0x7f119c1522d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43934/0 (socket says 172.21.15.26:43934) 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119affd640 1 -- 172.21.15.26:0/2270890407 learned_addr learned my addr 172.21.15.26:0/2270890407 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119bfff640 1 --2- 172.21.15.26:0/2270890407 >> v2:172.21.15.117:3300/0 conn(0x7f119c0c6d70 0x7f119c160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119a7fc640 1 -- 172.21.15.26:0/2270890407 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099760908 0 0) 0x7f119c0c71e0 con 0x7f119c155120 2024-03-06T07:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119a7fc640 1 -- 172.21.15.26:0/2270890407 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1184003520 con 0x7f119c155120 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119a7fc640 1 -- 172.21.15.26:0/2270890407 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744396501 0 0) 0x7f119c159590 con 0x7f119c0bfe90 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119a7fc640 1 -- 172.21.15.26:0/2270890407 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f119c0c71e0 con 0x7f119c0bfe90 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119bfff640 1 -- 172.21.15.26:0/2270890407 >> v1:172.21.15.117:6789/0 conn(0x7f119c0bfe90 legacy=0x7f119c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.042+0000 7f119bfff640 1 -- 172.21.15.26:0/2270890407 >> v1:172.21.15.26:6789/0 conn(0x7f119c155120 legacy=0x7f119c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.043+0000 7f119bfff640 1 -- 172.21.15.26:0/2270890407 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c0c59d0 con 0x7f119c0c6d70 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.043+0000 7f119bfff640 1 --2- 172.21.15.26:0/2270890407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 0x7f119c160ab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f119c158890 tx=0x7f119402be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2111f163be73d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.043+0000 7f119a7fc640 1 -- 172.21.15.26:0/2270890407 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194039070 con 0x7f119c0c6d70 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.043+0000 7f119a7fc640 1 -- 172.21.15.26:0/2270890407 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119402cc40 con 0x7f119c0c6d70 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.043+0000 7f119a7fc640 1 -- 172.21.15.26:0/2270890407 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194034440 con 0x7f119c0c6d70 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.044+0000 7f11a2108640 1 -- 172.21.15.26:0/2270890407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 msgr2=0x7f119c160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.044+0000 7f11a2108640 1 --2- 172.21.15.26:0/2270890407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 0x7f119c160ab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f119c158890 tx=0x7f119402be90 comp rx=0 tx=0).stop 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.044+0000 7f11a2108640 1 -- 172.21.15.26:0/2270890407 shutdown_connections 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.044+0000 7f11a2108640 1 --2- 172.21.15.26:0/2270890407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 0x7f119c160ab0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.044+0000 7f11a2108640 1 -- 172.21.15.26:0/2270890407 >> 172.21.15.26:0/2270890407 conn(0x7f119c0acaa0 msgr2=0x7f119c0aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.044+0000 7f11a2108640 1 -- 172.21.15.26:0/2270890407 shutdown_connections 2024-03-06T07:20:49.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.044+0000 7f11a2108640 1 -- 172.21.15.26:0/2270890407 wait complete. 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.045+0000 7f11a2108640 1 Processor -- start 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.045+0000 7f11a2108640 1 -- start start 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f11a2108640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f119c0bfe90 0x7f119c1eba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f11a2108640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 0x7f119c0abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f11a2108640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f119c155120 0x7f119c0aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f11a2108640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c158960 con 0x7f119c0bfe90 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f11a2108640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0c5f90 con 0x7f119c0c6d70 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f11a2108640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0ca700 con 0x7f119c155120 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f119c0bfe90 0x7f119c1eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f119c0bfe90 0x7f119c1eba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58370/0 (socket says 172.21.15.26:58370) 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 -- 172.21.15.26:0/2924030674 learned_addr learned my addr 172.21.15.26:0/2924030674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119affd640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 0x7f119c0abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119bfff640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f119c155120 0x7f119c0aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 -- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f119c155120 msgr2=0x7f119c0aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f119c155120 0x7f119c0aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 -- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 msgr2=0x7f119c0abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 0x7f119c0abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 -- 172.21.15.26:0/2924030674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c06c540 con 0x7f119c0bfe90 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.046+0000 7f119b7fe640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f119c0bfe90 0x7f119c1eba10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c490 tx=0x7f118c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.047+0000 7f1183fff640 1 -- 172.21.15.26:0/2924030674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0052d0 con 0x7f119c0bfe90 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.047+0000 7f1183fff640 1 -- 172.21.15.26:0/2924030674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c005470 con 0x7f119c0bfe90 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.047+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c06c820 con 0x7f119c0bfe90 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.047+0000 7f1183fff640 1 -- 172.21.15.26:0/2924030674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c012870 con 0x7f119c0bfe90 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.047+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c0ac200 con 0x7f119c0bfe90 2024-03-06T07:20:49.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.048+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c04f330 con 0x7f119c0bfe90 2024-03-06T07:20:49.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.048+0000 7f1183fff640 1 -- 172.21.15.26:0/2924030674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f118c012a10 con 0x7f119c0bfe90 2024-03-06T07:20:49.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.048+0000 7f1183fff640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f118806b4d0 0x7f118806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.049+0000 7f1183fff640 1 -- 172.21.15.26:0/2924030674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f118c089fe0 con 0x7f119c0bfe90 2024-03-06T07:20:49.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.049+0000 7f119affd640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f118806b4d0 0x7f118806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.049+0000 7f119affd640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f118806b4d0 0x7f118806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11900011e0 tx=0x7f1190007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.053+0000 7f1183fff640 1 -- 172.21.15.26:0/2924030674 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f118c058560 con 0x7f119c0bfe90 2024-03-06T07:20:49.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.270+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f119c0c0270 con 0x7f119c0bfe90 2024-03-06T07:20:49.272 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:20:49.271+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:20:49.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.272+0000 7f1183fff640 1 -- 172.21.15.26:0/2924030674 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f118c05c600 con 0x7f119c0bfe90 2024-03-06T07:20:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:20:49.276 INFO:tasks.ceph.mds.d.smithi117.stderr: -4631> 2024-03-06T07:19:55.422+0000 7fea3dc42640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-06T07:20:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.276+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f118806b4d0 msgr2=0x7f118806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.276+0000 7f11a2108640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f118806b4d0 0x7f118806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11900011e0 tx=0x7f1190007d30 comp rx=0 tx=0).stop 2024-03-06T07:20:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.276+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f119c0bfe90 msgr2=0x7f119c1eba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.276+0000 7f11a2108640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f119c0bfe90 0x7f119c1eba10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c490 tx=0x7f118c00c950 comp rx=0 tx=0).stop 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.276+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 shutdown_connections 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.276+0000 7f11a2108640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f118806b4d0 0x7f118806d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.276+0000 7f11a2108640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f119c155120 0x7f119c0aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.277+0000 7f11a2108640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0c6d70 0x7f119c0abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.277+0000 7f11a2108640 1 --2- 172.21.15.26:0/2924030674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f119c0bfe90 0x7f119c1eba10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.277+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 >> 172.21.15.26:0/2924030674 conn(0x7f119c0acaa0 msgr2=0x7f119c0ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.277+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 shutdown_connections 2024-03-06T07:20:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.277+0000 7f11a2108640 1 -- 172.21.15.26:0/2924030674 wait complete. 2024-03-06T07:20:49.280 INFO:tasks.ceph.mds.d.smithi117.stderr: -3405> 2024-03-06T07:20:06.432+0000 7fea37c36640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:20:49.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:20:49.310 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:20:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.396+0000 7f6f9257a640 1 Processor -- start 2024-03-06T07:20:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.396+0000 7f6f9257a640 1 -- start start 2024-03-06T07:20:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f9257a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f8c1527f0 0x7f6f8c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f9257a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c1596f0 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f9257a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c0c82a0 con 0x7f6f8c0c88f0 2024-03-06T07:20:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f9257a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8c0c9130 con 0x7f6f8c1527f0 2024-03-06T07:20:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f8bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f8c1527f0 0x7f6f8c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f8bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f8c1527f0 0x7f6f8c0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58380/0 (socket says 172.21.15.26:58380) 2024-03-06T07:20:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f8bfff640 1 -- 172.21.15.26:0/3438727860 learned_addr learned my addr 172.21.15.26:0/3438727860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f90b17640 1 --1- 172.21.15.26:0/3438727860 >> v1:172.21.15.26:6789/0 conn(0x7f6f8c0c58b0 0x7f6f8c15fb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43942/0 (socket says 172.21.15.26:43942) 2024-03-06T07:20:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f8bfff640 1 -- 172.21.15.26:0/3438727860 >> v1:172.21.15.117:6789/0 conn(0x7f6f8c0c88f0 legacy=0x7f6f8c155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f8bfff640 1 -- 172.21.15.26:0/3438727860 >> v1:172.21.15.26:6789/0 conn(0x7f6f8c0c58b0 legacy=0x7f6f8c15fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.397+0000 7f6f8bfff640 1 -- 172.21.15.26:0/3438727860 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c158930 con 0x7f6f8c1527f0 2024-03-06T07:20:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.398+0000 7f6f8bfff640 1 --2- 172.21.15.26:0/3438727860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c0c5370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009870 tx=0x7f6f8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b2c32e2d8e3e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.398+0000 7f6f8affd640 1 -- 172.21.15.26:0/3438727860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80035020 con 0x7f6f8c1527f0 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.398+0000 7f6f8affd640 1 -- 172.21.15.26:0/3438727860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f80031870 con 0x7f6f8c1527f0 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.398+0000 7f6f8affd640 1 -- 172.21.15.26:0/3438727860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80031c40 con 0x7f6f8c1527f0 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.399+0000 7f6f9257a640 1 -- 172.21.15.26:0/3438727860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c1527f0 msgr2=0x7f6f8c0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.399+0000 7f6f9257a640 1 --2- 172.21.15.26:0/3438727860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c0c5370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009870 tx=0x7f6f8002b910 comp rx=0 tx=0).stop 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.399+0000 7f6f9257a640 1 -- 172.21.15.26:0/3438727860 shutdown_connections 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.399+0000 7f6f9257a640 1 --2- 172.21.15.26:0/3438727860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c0c5370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.399+0000 7f6f9257a640 1 -- 172.21.15.26:0/3438727860 >> 172.21.15.26:0/3438727860 conn(0x7f6f8c0ac580 msgr2=0x7f6f8c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.399+0000 7f6f9257a640 1 -- 172.21.15.26:0/3438727860 shutdown_connections 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.399+0000 7f6f9257a640 1 -- 172.21.15.26:0/3438727860 wait complete. 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.400+0000 7f6f9257a640 1 Processor -- start 2024-03-06T07:20:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.400+0000 7f6f9257a640 1 -- start start 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f9257a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c0c58b0 0x7f6f8c1ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f9257a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8c0c88f0 0x7f6f8c1473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f9257a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f9257a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c0c6b90 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f9257a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c159260 con 0x7f6f8c1527f0 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f9257a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c158ef0 con 0x7f6f8c0c88f0 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c0c58b0 0x7f6f8c1ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c0c58b0 0x7f6f8c1ee470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58384/0 (socket says 172.21.15.26:58384) 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 -- 172.21.15.26:0/1258766640 learned_addr learned my addr 172.21.15.26:0/1258766640 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f90b17640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8b7fe640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8c0c88f0 0x7f6f8c1473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 -- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8c0c88f0 msgr2=0x7f6f8c1473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8c0c88f0 0x7f6f8c1473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 -- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c1527f0 msgr2=0x7f6f8c147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 -- 172.21.15.26:0/1258766640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c0c46d0 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f90b17640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8b7fe640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8c0c88f0 0x7f6f8c1473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f8bfff640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c0c58b0 0x7f6f8c1ee470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000c00 tx=0x7f6f800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f88ff9640 1 -- 172.21.15.26:0/1258766640 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80040070 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f88ff9640 1 -- 172.21.15.26:0/1258766640 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8003a420 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.401+0000 7f6f88ff9640 1 -- 172.21.15.26:0/1258766640 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80035040 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.402+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c0c49b0 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.402+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c147e50 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.402+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c1ef090 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.403+0000 7f6f88ff9640 1 -- 172.21.15.26:0/1258766640 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f8004b060 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.403+0000 7f6f88ff9640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f7806b4d0 0x7f6f7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.403+0000 7f6f88ff9640 1 -- 172.21.15.26:0/1258766640 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f6f800ac540 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.403+0000 7f6f8b7fe640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f7806b4d0 0x7f6f7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.404+0000 7f6f8b7fe640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f7806b4d0 0x7f6f7806d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c150c50 tx=0x7f6f7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.408+0000 7f6f88ff9640 1 -- 172.21.15.26:0/1258766640 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f8c1ef090 con 0x7f6f8c0c58b0 2024-03-06T07:20:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:49.625+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6f8c04f330 con 0x7f6f8c0c58b0 2024-03-06T07:20:50.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.280+0000 7f6f88ff9640 1 -- 172.21.15.26:0/1258766640 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6f8007afd0 con 0x7f6f8c0c58b0 2024-03-06T07:20:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f7806b4d0 msgr2=0x7f6f7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f7806b4d0 0x7f6f7806d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c150c50 tx=0x7f6f7c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c0c58b0 msgr2=0x7f6f8c1ee470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c0c58b0 0x7f6f8c1ee470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000c00 tx=0x7f6f800046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 shutdown_connections 2024-03-06T07:20:50.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f7806b4d0 0x7f6f7806d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c1527f0 0x7f6f8c147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8c0c88f0 0x7f6f8c1473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 --2- 172.21.15.26:0/1258766640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f8c0c58b0 0x7f6f8c1ee470 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 >> 172.21.15.26:0/1258766640 conn(0x7f6f8c0ac580 msgr2=0x7f6f8c150010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 shutdown_connections 2024-03-06T07:20:50.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.284+0000 7f6f9257a640 1 -- 172.21.15.26:0/1258766640 wait complete. 2024-03-06T07:20:50.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:20:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.408+0000 7f7c7caf4640 1 Processor -- start 2024-03-06T07:20:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.408+0000 7f7c7caf4640 1 -- start start 2024-03-06T07:20:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.408+0000 7f7c7caf4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 0x7f7c780c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7caf4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780c5400 con 0x7f7c780c7930 2024-03-06T07:20:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7caf4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780c56e0 con 0x7f7c78152870 2024-03-06T07:20:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7caf4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7806d320 con 0x7f7c780c88e0 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c76d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7c780c7930 0x7f7c7806bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43944/0 (socket says 172.21.15.26:43944) 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c76d9d640 1 -- 172.21.15.26:0/1365873082 learned_addr learned my addr 172.21.15.26:0/1365873082 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742496848 0 0) 0x7f7c780c5400 con 0x7f7c780c7930 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c60003520 con 0x7f7c780c7930 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 0x7f7c780c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1866629036 0 0) 0x7f7c60003520 con 0x7f7c780c7930 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 msgr2=0x7f7c780c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 0x7f7c780c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 >> v1:172.21.15.117:6789/0 conn(0x7f7c78152870 legacy=0x7f7c78155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c780c59c0 con 0x7f7c780c7930 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f7c700019f0 con 0x7f7c780c7930 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 >> v1:172.21.15.26:6789/0 conn(0x7f7c780c7930 legacy=0x7f7c7806bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c600039e0 0x7f7c60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 0x7f7c6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7806d320 con 0x7f7c60004800 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c60005520 con 0x7f7c600039e0 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6000bf00 con 0x7f7c780c88e0 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.409+0000 7f7c7659c640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c600039e0 0x7f7c60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c76d9d640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 0x7f7c6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c75d9b640 1 -- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 msgr2=0x7f7c6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 0x7f7c6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c75d9b640 1 -- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c600039e0 msgr2=0x7f7c60003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c600039e0 0x7f7c60003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c75d9b640 1 -- 172.21.15.26:0/1365873082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60003520 con 0x7f7c60004800 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c76d9d640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c780c88e0 0x7f7c6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.410+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c60004c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00ba00 tx=0x7f7c60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67daab71eaf297ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c036020 con 0x7f7c60004800 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c032620 con 0x7f7c60004800 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7559a640 1 -- 172.21.15.26:0/1365873082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00b600 con 0x7f7c60004800 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7caf4640 1 -- 172.21.15.26:0/1365873082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 msgr2=0x7f7c60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7caf4640 1 --2- 172.21.15.26:0/1365873082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c60004c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00ba00 tx=0x7f7c60005830 comp rx=0 tx=0).stop 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c76d9d640 1 -- 172.21.15.26:0/1365873082 reap_dead start 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7caf4640 1 -- 172.21.15.26:0/1365873082 shutdown_connections 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7caf4640 1 -- 172.21.15.26:0/1365873082 >> 172.21.15.26:0/1365873082 conn(0x7f7c780ac5f0 msgr2=0x7f7c780aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7caf4640 1 -- 172.21.15.26:0/1365873082 shutdown_connections 2024-03-06T07:20:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.411+0000 7f7c7caf4640 1 -- 172.21.15.26:0/1365873082 wait complete. 2024-03-06T07:20:50.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.412+0000 7f7c7caf4640 1 Processor -- start 2024-03-06T07:20:50.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.412+0000 7f7c7caf4640 1 -- start start 2024-03-06T07:20:50.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7caf4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c781ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7caf4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c78148cb0 0x7f7c780c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7caf4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c78141da0 0x7f7c78147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7caf4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780c9470 con 0x7f7c60004800 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7caf4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7806b8d0 con 0x7f7c78148cb0 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7caf4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780c95f0 con 0x7f7c78141da0 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c781ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c781ebe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58390/0 (socket says 172.21.15.26:58390) 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 -- 172.21.15.26:0/3534929500 learned_addr learned my addr 172.21.15.26:0/3534929500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c76d9d640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c78141da0 0x7f7c78147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c78148cb0 0x7f7c780c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 -- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c78141da0 msgr2=0x7f7c78147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c78141da0 0x7f7c78147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 -- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c78148cb0 msgr2=0x7f7c780c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c78148cb0 0x7f7c780c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c7659c640 1 -- 172.21.15.26:0/3534929500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c780c3510 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c76d9d640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c78141da0 0x7f7c78147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.413+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c78148cb0 0x7f7c780c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.414+0000 7f7c7659c640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c781ebe70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800f850 tx=0x7f7c6800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.414+0000 7f7c66ffd640 1 -- 172.21.15.26:0/3534929500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800fe40 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.414+0000 7f7c66ffd640 1 -- 172.21.15.26:0/3534929500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c68018070 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.414+0000 7f7c66ffd640 1 -- 172.21.15.26:0/3534929500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800d6d0 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.414+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c78145ef0 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.414+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c781464b0 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.415+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c78069460 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.415+0000 7f7c66ffd640 1 -- 172.21.15.26:0/3534929500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c6800d870 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.416+0000 7f7c66ffd640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c5806b500 0x7f7c5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.416+0000 7f7c66ffd640 1 -- 172.21.15.26:0/3534929500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f7c6808a450 con 0x7f7c60004800 2024-03-06T07:20:50.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.416+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c5806b500 0x7f7c5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.416+0000 7f7c75d9b640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c5806b500 0x7f7c5806d9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c780c8d20 tx=0x7f7c6c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.420+0000 7f7c66ffd640 1 -- 172.21.15.26:0/3534929500 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c78069460 con 0x7f7c60004800 2024-03-06T07:20:50.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:50.603+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7c7804f330 con 0x7f7c60004800 2024-03-06T07:20:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.281+0000 7f7c66ffd640 1 -- 172.21.15.26:0/3534929500 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7c68058e50 con 0x7f7c60004800 2024-03-06T07:20:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c5806b500 msgr2=0x7f7c5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c5806b500 0x7f7c5806d9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c780c8d20 tx=0x7f7c6c030000 comp rx=0 tx=0).stop 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 msgr2=0x7f7c781ebe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c781ebe70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800f850 tx=0x7f7c6800fd10 comp rx=0 tx=0).stop 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 shutdown_connections 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c5806b500 0x7f7c5806d9c0 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c780c8d20 tx=0x7f7c6c030000 comp rx=0 tx=0).stop 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c78141da0 0x7f7c78147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c78148cb0 0x7f7c780c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 --2- 172.21.15.26:0/3534929500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60004800 0x7f7c781ebe70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 >> 172.21.15.26:0/3534929500 conn(0x7f7c780ac5f0 msgr2=0x7f7c780ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 shutdown_connections 2024-03-06T07:20:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.286+0000 7f7c7caf4640 1 -- 172.21.15.26:0/3534929500 wait complete. 2024-03-06T07:20:51.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:20:51.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.412+0000 7fc95535e640 1 Processor -- start 2024-03-06T07:20:51.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.412+0000 7fc95535e640 1 -- start start 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc95535e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9500c53f0 0x7fc95006b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc95535e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc95006b700 0x7fc9500c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc95535e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95013e070 con 0x7fc95013dc90 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc95535e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95006c190 con 0x7fc95006b700 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc95535e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95006bde0 con 0x7fc9500c53f0 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc95006b700 0x7fc9500c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc95013dc90 0x7fc950159230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43948/0 (socket says 172.21.15.26:43948) 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94effd640 1 -- 172.21.15.26:0/1690796640 learned_addr learned my addr 172.21.15.26:0/1690796640 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94e7fc640 1 --2- 172.21.15.26:0/1690796640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9500c53f0 0x7fc95006b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94f7fe640 1 -- 172.21.15.26:0/1690796640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9500c53f0 msgr2=0x7fc95006b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94f7fe640 1 --2- 172.21.15.26:0/1690796640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9500c53f0 0x7fc95006b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94f7fe640 1 -- 172.21.15.26:0/1690796640 >> v1:172.21.15.26:6789/0 conn(0x7fc95013dc90 legacy=0x7fc950159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94f7fe640 1 -- 172.21.15.26:0/1690796640 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95013e420 con 0x7fc95006b700 2024-03-06T07:20:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.413+0000 7fc94e7fc640 1 --2- 172.21.15.26:0/1690796640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9500c53f0 0x7fc95006b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.414+0000 7fc94f7fe640 1 --2- 172.21.15.26:0/1690796640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95006b700 0x7fc9500c69f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc948009d90 tx=0x7fc94802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c835995f4e79df server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.414+0000 7fc94dffb640 1 -- 172.21.15.26:0/1690796640 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948039070 con 0x7fc95006b700 2024-03-06T07:20:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.414+0000 7fc94dffb640 1 -- 172.21.15.26:0/1690796640 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94802cbc0 con 0x7fc95006b700 2024-03-06T07:20:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.414+0000 7fc94dffb640 1 -- 172.21.15.26:0/1690796640 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948034440 con 0x7fc95006b700 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 -- 172.21.15.26:0/1690796640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95006b700 msgr2=0x7fc9500c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 --2- 172.21.15.26:0/1690796640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95006b700 0x7fc9500c69f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc948009d90 tx=0x7fc94802be10 comp rx=0 tx=0).stop 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 -- 172.21.15.26:0/1690796640 shutdown_connections 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 --2- 172.21.15.26:0/1690796640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95006b700 0x7fc9500c69f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 --2- 172.21.15.26:0/1690796640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9500c53f0 0x7fc95006b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 -- 172.21.15.26:0/1690796640 >> 172.21.15.26:0/1690796640 conn(0x7fc9500caa70 msgr2=0x7fc9500cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 -- 172.21.15.26:0/1690796640 shutdown_connections 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.415+0000 7fc95535e640 1 -- 172.21.15.26:0/1690796640 wait complete. 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.416+0000 7fc95535e640 1 Processor -- start 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.416+0000 7fc95535e640 1 -- start start 2024-03-06T07:20:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc95535e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95006b700 0x7fc950157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc95535e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9500c53f0 0x7fc950156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc95535e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95013dc90 0x7fc9501568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc95535e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500c61f0 con 0x7fc95013dc90 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc95535e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95013e9e0 con 0x7fc9500c53f0 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc95535e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500c8970 con 0x7fc95006b700 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95013dc90 0x7fc9501568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95013dc90 0x7fc9501568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58412/0 (socket says 172.21.15.26:58412) 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 -- 172.21.15.26:0/3434684859 learned_addr learned my addr 172.21.15.26:0/3434684859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94effd640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95006b700 0x7fc950157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94e7fc640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9500c53f0 0x7fc950156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 -- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95006b700 msgr2=0x7fc950157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95006b700 0x7fc950157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 -- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9500c53f0 msgr2=0x7fc950156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9500c53f0 0x7fc950156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 -- 172.21.15.26:0/3434684859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950137850 con 0x7fc95013dc90 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.417+0000 7fc94f7fe640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95013dc90 0x7fc9501568a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc948009eb0 tx=0x7fc948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.418+0000 7fc9377fe640 1 -- 172.21.15.26:0/3434684859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948039050 con 0x7fc95013dc90 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.418+0000 7fc9377fe640 1 -- 172.21.15.26:0/3434684859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc948045070 con 0x7fc95013dc90 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.418+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9501ee780 con 0x7fc95013dc90 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.418+0000 7fc9377fe640 1 -- 172.21.15.26:0/3434684859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94803e830 con 0x7fc95013dc90 2024-03-06T07:20:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.418+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc950158420 con 0x7fc95013dc90 2024-03-06T07:20:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.419+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95004f330 con 0x7fc95013dc90 2024-03-06T07:20:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.419+0000 7fc9377fe640 1 -- 172.21.15.26:0/3434684859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc94803e9d0 con 0x7fc95013dc90 2024-03-06T07:20:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.419+0000 7fc9377fe640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc93c06b500 0x7fc93c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.419+0000 7fc9377fe640 1 -- 172.21.15.26:0/3434684859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6412+0+0 (secure 0 0 0) 0x7fc9480ad410 con 0x7fc95013dc90 2024-03-06T07:20:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.420+0000 7fc94effd640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc93c06b500 0x7fc93c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.420+0000 7fc94effd640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc93c06b500 0x7fc93c06d9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc940000f60 tx=0x7fc940009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.424+0000 7fc9377fe640 1 -- 172.21.15.26:0/3434684859 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc948032e20 con 0x7fc95013dc90 2024-03-06T07:20:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:51.607+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc95006bb70 con 0x7fc95013dc90 2024-03-06T07:20:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.283+0000 7fc9377fe640 1 -- 172.21.15.26:0/3434684859 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc94807c180 con 0x7fc95013dc90 2024-03-06T07:20:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:20:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc93c06b500 msgr2=0x7fc93c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc93c06b500 0x7fc93c06d9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc940000f60 tx=0x7fc940009040 comp rx=0 tx=0).stop 2024-03-06T07:20:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95013dc90 msgr2=0x7fc9501568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95013dc90 0x7fc9501568a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc948009eb0 tx=0x7fc948004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 shutdown_connections 2024-03-06T07:20:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc93c06b500 0x7fc93c06d9c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95013dc90 0x7fc9501568a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9500c53f0 0x7fc950156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 --2- 172.21.15.26:0/3434684859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95006b700 0x7fc950157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 >> 172.21.15.26:0/3434684859 conn(0x7fc9500caa70 msgr2=0x7fc95013bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 shutdown_connections 2024-03-06T07:20:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.285+0000 7fc95535e640 1 -- 172.21.15.26:0/3434684859 wait complete. 2024-03-06T07:20:52.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-06T07:20:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.407+0000 7f60b553e640 1 Processor -- start 2024-03-06T07:20:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.408+0000 7f60b553e640 1 -- start start 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.408+0000 7f60b553e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60b0152920 0x7f60b0155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.408+0000 7f60b553e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b00c2900 0x7f60b015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.408+0000 7f60b553e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b0159740 con 0x7f60b0155eb0 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.408+0000 7f60b553e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b01551d0 con 0x7f60b0152920 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.408+0000 7f60b553e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00c8ab0 con 0x7f60b00c2900 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60af7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b00c2900 0x7f60b015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60aeffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60b0152920 0x7f60b0155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60ae7fc640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f60b0155eb0 0x7f60b00c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:54224/0 (socket says 172.21.15.26:54224) 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60aeffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60b0152920 0x7f60b0155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44050/0 (socket says 172.21.15.26:44050) 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60ae7fc640 1 -- 172.21.15.26:0/1502623951 learned_addr learned my addr 172.21.15.26:0/1502623951 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60adffb640 1 -- 172.21.15.26:0/1502623951 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298398776 0 0) 0x7f60b0159740 con 0x7f60b0155eb0 2024-03-06T07:20:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60adffb640 1 -- 172.21.15.26:0/1502623951 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f609c003520 con 0x7f60b0155eb0 2024-03-06T07:20:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60af7fe640 1 -- 172.21.15.26:0/1502623951 >> v2:172.21.15.117:3300/0 conn(0x7f60b0152920 msgr2=0x7f60b0155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60af7fe640 1 --2- 172.21.15.26:0/1502623951 >> v2:172.21.15.117:3300/0 conn(0x7f60b0152920 0x7f60b0155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60af7fe640 1 -- 172.21.15.26:0/1502623951 >> v1:172.21.15.117:6789/0 conn(0x7f60b0155eb0 legacy=0x7f60b00c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60af7fe640 1 -- 172.21.15.26:0/1502623951 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b00c8d90 con 0x7f60b00c2900 2024-03-06T07:20:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.409+0000 7f60aeffd640 1 --2- 172.21.15.26:0/1502623951 >> v2:172.21.15.117:3300/0 conn(0x7f60b0152920 0x7f60b0155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.410+0000 7f60af7fe640 1 --2- 172.21.15.26:0/1502623951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b00c2900 0x7f60b015fb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009d90 tx=0x7f60a002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37bce05bc368ab11 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.410+0000 7f60adffb640 1 -- 172.21.15.26:0/1502623951 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0039070 con 0x7f60b00c2900 2024-03-06T07:20:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.410+0000 7f60adffb640 1 -- 172.21.15.26:0/1502623951 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a002cbc0 con 0x7f60b00c2900 2024-03-06T07:20:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.410+0000 7f60adffb640 1 -- 172.21.15.26:0/1502623951 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0034440 con 0x7f60b00c2900 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 -- 172.21.15.26:0/1502623951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b00c2900 msgr2=0x7f60b015fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 --2- 172.21.15.26:0/1502623951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b00c2900 0x7f60b015fb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009d90 tx=0x7f60a002be10 comp rx=0 tx=0).stop 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 -- 172.21.15.26:0/1502623951 shutdown_connections 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 --2- 172.21.15.26:0/1502623951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b00c2900 0x7f60b015fb50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 --2- 172.21.15.26:0/1502623951 >> v2:172.21.15.117:3300/0 conn(0x7f60b0152920 0x7f60b0155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 -- 172.21.15.26:0/1502623951 >> 172.21.15.26:0/1502623951 conn(0x7f60b00aa5c0 msgr2=0x7f60b00aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 -- 172.21.15.26:0/1502623951 shutdown_connections 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.411+0000 7f60b553e640 1 -- 172.21.15.26:0/1502623951 wait complete. 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.412+0000 7f60b553e640 1 Processor -- start 2024-03-06T07:20:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.412+0000 7f60b553e640 1 -- start start 2024-03-06T07:20:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60b553e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60b00c2900 0x7f60b006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60b553e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b0152920 0x7f60b01581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60b553e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b0155eb0 0x7f60b01586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60b553e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00c82a0 con 0x7f60b0152920 2024-03-06T07:20:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60b553e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0155450 con 0x7f60b00c2900 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60b553e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00c8420 con 0x7f60b0155eb0 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b0152920 0x7f60b01581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b0152920 0x7f60b01581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58424/0 (socket says 172.21.15.26:58424) 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 -- 172.21.15.26:0/2265282382 learned_addr learned my addr 172.21.15.26:0/2265282382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60aeffd640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60b00c2900 0x7f60b006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60af7fe640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b0155eb0 0x7f60b01586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 -- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b0155eb0 msgr2=0x7f60b01586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b0155eb0 0x7f60b01586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 -- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60b00c2900 msgr2=0x7f60b006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60b00c2900 0x7f60b006c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 -- 172.21.15.26:0/2265282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b01568e0 con 0x7f60b0152920 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60ae7fc640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b0152920 0x7f60b01581a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f609800c390 tx=0x7f609800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.413+0000 7f60a77fe640 1 -- 172.21.15.26:0/2265282382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098017070 con 0x7f60b0152920 2024-03-06T07:20:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.414+0000 7f60a77fe640 1 -- 172.21.15.26:0/2265282382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6098005ce0 con 0x7f60b0152920 2024-03-06T07:20:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.414+0000 7f60a77fe640 1 -- 172.21.15.26:0/2265282382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60980072f0 con 0x7f60b0152920 2024-03-06T07:20:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.414+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b0156bc0 con 0x7f60b0152920 2024-03-06T07:20:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.414+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b006c880 con 0x7f60b0152920 2024-03-06T07:20:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.415+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b004f330 con 0x7f60b0152920 2024-03-06T07:20:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.415+0000 7f60a77fe640 1 -- 172.21.15.26:0/2265282382 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f609801a070 con 0x7f60b0152920 2024-03-06T07:20:52.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.415+0000 7f60a77fe640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f609006b4d0 0x7f609006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.415+0000 7f60a77fe640 1 -- 172.21.15.26:0/2265282382 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6098089610 con 0x7f60b0152920 2024-03-06T07:20:52.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.416+0000 7f60aeffd640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f609006b4d0 0x7f609006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.416+0000 7f60aeffd640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f609006b4d0 0x7f609006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60b00addd0 tx=0x7f60a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.420+0000 7f60a77fe640 1 -- 172.21.15.26:0/2265282382 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6098058490 con 0x7f60b0152920 2024-03-06T07:20:52.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:52.599+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60b00c1fd0 con 0x7f60b0152920 2024-03-06T07:20:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.297+0000 7f60a77fe640 1 -- 172.21.15.26:0/2265282382 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v29) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6098057d90 con 0x7f60b0152920 2024-03-06T07:20:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data_ec' removed 2024-03-06T07:20:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.301+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f609006b4d0 msgr2=0x7f609006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.301+0000 7f60b553e640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f609006b4d0 0x7f609006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60b00addd0 tx=0x7f60a8008040 comp rx=0 tx=0).stop 2024-03-06T07:20:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.301+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b0152920 msgr2=0x7f60b01581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.301+0000 7f60b553e640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b0152920 0x7f60b01581a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f609800c390 tx=0x7f609800c850 comp rx=0 tx=0).stop 2024-03-06T07:20:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.302+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 shutdown_connections 2024-03-06T07:20:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.302+0000 7f60b553e640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f609006b4d0 0x7f609006d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.302+0000 7f60b553e640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60b0155eb0 0x7f60b01586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.302+0000 7f60b553e640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b0152920 0x7f60b01581a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.302+0000 7f60b553e640 1 --2- 172.21.15.26:0/2265282382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60b00c2900 0x7f60b006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.302+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 >> 172.21.15.26:0/2265282382 conn(0x7f60b00aa5c0 msgr2=0x7f60b00aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.303+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 shutdown_connections 2024-03-06T07:20:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.303+0000 7f60b553e640 1 -- 172.21.15.26:0/2265282382 wait complete. 2024-03-06T07:20:53.319 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:20:53.319 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:20:53.319 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:20:53.320 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:53.320 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:20:53.320 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:20:53.320 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:20:53.320 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:53.320 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:20:53.320 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:20:53.321 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:20:53.321 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:53.321 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:20:53.321 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:20:53.321 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:20:53.321 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:53.331 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:20:53.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:20:53.333 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:20:53.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:20:53.335 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:20:53.335 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:20:53.337 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:20:53.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:20:53.374 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:20:53.375 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:53.376 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:20:53.376 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:53.379 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:20:53.379 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:53.380 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:20:53.381 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:53.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:20:53.458 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:20:53.459 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:20:53.461 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:20:53.461 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:20:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.494+0000 7ff12f15c640 1 Processor -- start 2024-03-06T07:20:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12f15c640 1 -- start start 2024-03-06T07:20:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12f15c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1280c8950 0x7ff1280c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12f15c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1281558d0 0x7ff128155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12f15c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff128155130 con 0x7ff1280c69f0 2024-03-06T07:20:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12f15c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff128155440 con 0x7ff1280c8950 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12f15c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280ad480 con 0x7ff1281558d0 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff127fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1280c8950 0x7ff1280c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12cef8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1280c69f0 0x7ff1280acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43974/0 (socket says 172.21.15.26:43974) 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.495+0000 7ff12cef8640 1 -- 172.21.15.26:0/1283490437 learned_addr learned my addr 172.21.15.26:0/1283490437 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.496+0000 7ff12d6f9640 1 --2- 172.21.15.26:0/1283490437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1281558d0 0x7ff128155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.496+0000 7ff127fff640 1 -- 172.21.15.26:0/1283490437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1281558d0 msgr2=0x7ff128155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.496+0000 7ff127fff640 1 --2- 172.21.15.26:0/1283490437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1281558d0 0x7ff128155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.496+0000 7ff127fff640 1 -- 172.21.15.26:0/1283490437 >> v1:172.21.15.26:6789/0 conn(0x7ff1280c69f0 legacy=0x7ff1280acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.496+0000 7ff127fff640 1 -- 172.21.15.26:0/1283490437 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280c5d50 con 0x7ff1280c8950 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff127fff640 1 --2- 172.21.15.26:0/1283490437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1280c8950 0x7ff1280c6dd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1140097a0 tx=0x7ff11402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5783ec304e0077e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff1277fe640 1 -- 172.21.15.26:0/1283490437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114031780 con 0x7ff1280c8950 2024-03-06T07:20:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff1277fe640 1 -- 172.21.15.26:0/1283490437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114037070 con 0x7ff1280c8950 2024-03-06T07:20:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff1277fe640 1 -- 172.21.15.26:0/1283490437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114031b40 con 0x7ff1280c8950 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 -- 172.21.15.26:0/1283490437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1280c8950 msgr2=0x7ff1280c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1283490437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1280c8950 0x7ff1280c6dd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1140097a0 tx=0x7ff11402b820 comp rx=0 tx=0).stop 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 -- 172.21.15.26:0/1283490437 shutdown_connections 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1283490437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1280c8950 0x7ff1280c6dd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1283490437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1281558d0 0x7ff128155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 -- 172.21.15.26:0/1283490437 >> 172.21.15.26:0/1283490437 conn(0x7ff1280cae70 msgr2=0x7ff1280cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 -- 172.21.15.26:0/1283490437 shutdown_connections 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.497+0000 7ff12f15c640 1 -- 172.21.15.26:0/1283490437 wait complete. 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.498+0000 7ff12f15c640 1 Processor -- start 2024-03-06T07:20:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.498+0000 7ff12f15c640 1 -- start start 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12f15c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1280c69f0 0x7ff1281496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12f15c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1280c8950 0x7ff12814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12f15c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1281558d0 0x7ff128148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12f15c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff128152a20 con 0x7ff1281558d0 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12f15c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280c6310 con 0x7ff1280c8950 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12f15c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff128152ba0 con 0x7ff1280c69f0 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12d6f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1281558d0 0x7ff128148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.499+0000 7ff12d6f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1281558d0 0x7ff128148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58466/0 (socket says 172.21.15.26:58466) 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12d6f9640 1 -- 172.21.15.26:0/1506656569 learned_addr learned my addr 172.21.15.26:0/1506656569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12cef8640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1280c69f0 0x7ff1281496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff127fff640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1280c8950 0x7ff12814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12d6f9640 1 -- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1280c69f0 msgr2=0x7ff1281496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12d6f9640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1280c69f0 0x7ff1281496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12d6f9640 1 -- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1280c8950 msgr2=0x7ff12814ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12d6f9640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1280c8950 0x7ff12814ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12d6f9640 1 -- 172.21.15.26:0/1506656569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12814d320 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12d6f9640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1281558d0 0x7ff128148ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff120007e40 tx=0x7ff12000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff1257fa640 1 -- 172.21.15.26:0/1506656569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120014620 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff1257fa640 1 -- 172.21.15.26:0/1506656569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff120019070 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff128159010 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff1257fa640 1 -- 172.21.15.26:0/1506656569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120014a20 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.500+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12814cb20 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.501+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff128159530 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.502+0000 7ff1257fa640 1 -- 172.21.15.26:0/1506656569 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff120014bc0 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.502+0000 7ff1257fa640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff10c06b4d0 0x7ff10c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.502+0000 7ff1257fa640 1 -- 172.21.15.26:0/1506656569 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff12008a830 con 0x7ff1281558d0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.502+0000 7ff12cef8640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff10c06b4d0 0x7ff10c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.503+0000 7ff12cef8640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff10c06b4d0 0x7ff10c06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff11c007fd0 tx=0x7ff11c007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.506+0000 7ff1257fa640 1 -- 172.21.15.26:0/1506656569 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff1200591d0 con 0x7ff1281558d0 2024-03-06T07:20:53.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:53.702+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff1280ca650 con 0x7ff1281558d0 2024-03-06T07:20:54.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.306+0000 7ff1257fa640 1 -- 172.21.15.26:0/1506656569 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff12005d270 con 0x7ff1281558d0 2024-03-06T07:20:54.308 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff10c06b4d0 msgr2=0x7ff10c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff10c06b4d0 0x7ff10c06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff11c007fd0 tx=0x7ff11c007e80 comp rx=0 tx=0).stop 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1281558d0 msgr2=0x7ff128148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1281558d0 0x7ff128148ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff120007e40 tx=0x7ff12000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 shutdown_connections 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff10c06b4d0 0x7ff10c06d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1281558d0 0x7ff128148ab0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1280c8950 0x7ff12814ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 --2- 172.21.15.26:0/1506656569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1280c69f0 0x7ff1281496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 >> 172.21.15.26:0/1506656569 conn(0x7ff1280cae70 msgr2=0x7ff1281517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 shutdown_connections 2024-03-06T07:20:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.308+0000 7ff12f15c640 1 -- 172.21.15.26:0/1506656569 wait complete. 2024-03-06T07:20:54.323 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.425+0000 7f1d04615640 1 Processor -- start 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.425+0000 7f1d04615640 1 -- start start 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.425+0000 7f1d04615640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cfc1592d0 0x7f1cfc1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.425+0000 7f1d04615640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1cfc0ac260 0x7f1cfc155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.425+0000 7f1d04615640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc152d40 con 0x7f1cfc152960 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.425+0000 7f1d04615640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc066e30 con 0x7f1cfc1592d0 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.425+0000 7f1d04615640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc155d50 con 0x7f1cfc0ac260 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d01bb0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cfc1592d0 0x7f1cfc1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d01bb0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cfc1592d0 0x7f1cfc1552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58470/0 (socket says 172.21.15.26:58470) 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d01bb0640 1 -- 172.21.15.26:0/4233739296 learned_addr learned my addr 172.21.15.26:0/4233739296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:54.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d02bb2640 1 --2- 172.21.15.26:0/4233739296 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc0ac260 0x7f1cfc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d01bb0640 1 -- 172.21.15.26:0/4233739296 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc0ac260 msgr2=0x7f1cfc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d01bb0640 1 --2- 172.21.15.26:0/4233739296 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc0ac260 0x7f1cfc155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d01bb0640 1 -- 172.21.15.26:0/4233739296 >> v1:172.21.15.117:6789/0 conn(0x7f1cfc152960 legacy=0x7f1cfc06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d01bb0640 1 -- 172.21.15.26:0/4233739296 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc0c1b50 con 0x7f1cfc1592d0 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.426+0000 7f1d02bb2640 1 --2- 172.21.15.26:0/4233739296 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc0ac260 0x7f1cfc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d01bb0640 1 --2- 172.21.15.26:0/4233739296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc1592d0 0x7f1cfc1552d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc06b380 tx=0x7f1cf002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f5d0b73adeff47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d013af640 1 -- 172.21.15.26:0/4233739296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0034070 con 0x7f1cfc1592d0 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d013af640 1 -- 172.21.15.26:0/4233739296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0030860 con 0x7f1cfc1592d0 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d013af640 1 -- 172.21.15.26:0/4233739296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0030c30 con 0x7f1cfc1592d0 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 -- 172.21.15.26:0/4233739296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc1592d0 msgr2=0x7f1cfc1552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 --2- 172.21.15.26:0/4233739296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc1592d0 0x7f1cfc1552d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc06b380 tx=0x7f1cf002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 -- 172.21.15.26:0/4233739296 shutdown_connections 2024-03-06T07:20:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 --2- 172.21.15.26:0/4233739296 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc0ac260 0x7f1cfc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 --2- 172.21.15.26:0/4233739296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc1592d0 0x7f1cfc1552d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 -- 172.21.15.26:0/4233739296 >> 172.21.15.26:0/4233739296 conn(0x7f1cfc0c70b0 msgr2=0x7f1cfc0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 -- 172.21.15.26:0/4233739296 shutdown_connections 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.427+0000 7f1d04615640 1 -- 172.21.15.26:0/4233739296 wait complete. 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.428+0000 7f1d04615640 1 Processor -- start 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d04615640 1 -- start start 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d04615640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0ac260 0x7f1cfc14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d04615640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc152960 0x7f1cfc071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d04615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc0724d0 0x7f1cfc14bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d04615640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc159760 con 0x7f1cfc0724d0 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d04615640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc0672c0 con 0x7f1cfc152960 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d04615640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc06e3a0 con 0x7f1cfc0ac260 2024-03-06T07:20:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d02bb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc0724d0 0x7f1cfc14bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d02bb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc0724d0 0x7f1cfc14bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58476/0 (socket says 172.21.15.26:58476) 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.429+0000 7f1d02bb2640 1 -- 172.21.15.26:0/1191028579 learned_addr learned my addr 172.21.15.26:0/1191028579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d023b1640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0ac260 0x7f1cfc14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d01bb0640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc152960 0x7f1cfc071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d02bb2640 1 -- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0ac260 msgr2=0x7f1cfc14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d02bb2640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0ac260 0x7f1cfc14b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d02bb2640 1 -- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc152960 msgr2=0x7f1cfc071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d02bb2640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc152960 0x7f1cfc071f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d02bb2640 1 -- 172.21.15.26:0/1191028579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc1ebd90 con 0x7f1cfc0724d0 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d02bb2640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc0724d0 0x7f1cfc14bdf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c7f0 tx=0x7f1cec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1cfaffd640 1 -- 172.21.15.26:0/1191028579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec015070 con 0x7f1cfc0724d0 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1cfaffd640 1 -- 172.21.15.26:0/1191028579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec0077c0 con 0x7f1cfc0724d0 2024-03-06T07:20:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc1ec070 con 0x7f1cfc0724d0 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1cfaffd640 1 -- 172.21.15.26:0/1191028579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec007bf0 con 0x7f1cfc0724d0 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.430+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc1543e0 con 0x7f1cfc0724d0 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.431+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc0c2c70 con 0x7f1cfc0724d0 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.432+0000 7f1cfaffd640 1 -- 172.21.15.26:0/1191028579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cec018070 con 0x7f1cfc0724d0 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.432+0000 7f1cfaffd640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ce806b500 0x7f1ce806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.432+0000 7f1cfaffd640 1 -- 172.21.15.26:0/1191028579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1cec08a0b0 con 0x7f1cfc0724d0 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.432+0000 7f1d023b1640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ce806b500 0x7f1ce806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.433+0000 7f1d023b1640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ce806b500 0x7f1ce806d9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007fd0 tx=0x7f1cf4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.436+0000 7f1cfaffd640 1 -- 172.21.15.26:0/1191028579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1cfc0c2c70 con 0x7f1cfc0724d0 2024-03-06T07:20:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.691+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1cfc04f330 con 0x7f1cfc0724d0 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.691+0000 7f1cfaffd640 1 -- 172.21.15.26:0/1191028579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6065 (secure 0 0 0) 0x7f1cec059180 con 0x7f1cfc0724d0 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:20:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:20:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:20:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mds", 2024-03-06T07:20:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAQGehlIW++MBAA9EiZKQMaqvnB+pVFz4gt1w==", 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mds" 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mgr", 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAQGehlNne+MBAALYLFgg8JmO9Sr7lqf5+oRg==", 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-osd", 2024-03-06T07:20:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAQGehl7n2+MBAABFBs2aZixfcrsgkDwp/eTw==", 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-osd" 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd", 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAQGehlUYW+MBAAgiHfr/THrkcSM6QQ165cQQ==", 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAQGehlz4u+MBAAo3bNjGj1NbyzxzqxCgea4w==", 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rgw", 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAQGehlhZK+MBAAKaafj1vbLKH3+x8R/zwcdQ==", 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:20:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:20:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:20:54.708 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:20:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ce806b500 msgr2=0x7f1ce806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ce806b500 0x7f1ce806d9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007fd0 tx=0x7f1cf4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc0724d0 msgr2=0x7f1cfc14bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc0724d0 0x7f1cfc14bdf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c7f0 tx=0x7f1cec00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 shutdown_connections 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ce806b500 0x7f1ce806d9c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc0724d0 0x7f1cfc14bdf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc152960 0x7f1cfc071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 --2- 172.21.15.26:0/1191028579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0ac260 0x7f1cfc14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 >> 172.21.15.26:0/1191028579 conn(0x7f1cfc0c70b0 msgr2=0x7f1cfc06bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 shutdown_connections 2024-03-06T07:20:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.693+0000 7f1d04615640 1 -- 172.21.15.26:0/1191028579 wait complete. 2024-03-06T07:20:54.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-06T07:20:54.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.812+0000 7efea535c640 1 Processor -- start 2024-03-06T07:20:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.812+0000 7efea535c640 1 -- start start 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efea535c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efea0152b40 0x7efea00c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efea535c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea00733f0 con 0x7efea00a7ff0 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efea535c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea00c82c0 con 0x7efea00724d0 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efea535c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea0072d70 con 0x7efea0152b40 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efe9effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efea0152b40 0x7efea00c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efe9f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efea00a7ff0 0x7efea015fb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43988/0 (socket says 172.21.15.26:43988) 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efe9f7fe640 1 -- 172.21.15.26:0/3708539435 learned_addr learned my addr 172.21.15.26:0/3708539435 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efe9effd640 1 -- 172.21.15.26:0/3708539435 >> v1:172.21.15.117:6789/0 conn(0x7efea00724d0 legacy=0x7efea01551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efe9effd640 1 -- 172.21.15.26:0/3708539435 >> v1:172.21.15.26:6789/0 conn(0x7efea00a7ff0 legacy=0x7efea015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.813+0000 7efe9effd640 1 -- 172.21.15.26:0/3708539435 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea006fd30 con 0x7efea0152b40 2024-03-06T07:20:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.814+0000 7efe9effd640 1 --2- 172.21.15.26:0/3708539435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea00c8810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe900097a0 tx=0x7efe9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d8e664dc99a1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.814+0000 7efe9e7fc640 1 -- 172.21.15.26:0/3708539435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90035020 con 0x7efea0152b40 2024-03-06T07:20:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.814+0000 7efe9e7fc640 1 -- 172.21.15.26:0/3708539435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe90031830 con 0x7efea0152b40 2024-03-06T07:20:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.814+0000 7efe9e7fc640 1 -- 172.21.15.26:0/3708539435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90031c00 con 0x7efea0152b40 2024-03-06T07:20:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.814+0000 7efea535c640 1 -- 172.21.15.26:0/3708539435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 msgr2=0x7efea00c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.814+0000 7efea535c640 1 --2- 172.21.15.26:0/3708539435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea00c8810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe900097a0 tx=0x7efe9002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.815+0000 7efea535c640 1 -- 172.21.15.26:0/3708539435 shutdown_connections 2024-03-06T07:20:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.815+0000 7efea535c640 1 --2- 172.21.15.26:0/3708539435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea00c8810 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.815+0000 7efea535c640 1 -- 172.21.15.26:0/3708539435 >> 172.21.15.26:0/3708539435 conn(0x7efea006b970 msgr2=0x7efea00aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.815+0000 7efea535c640 1 -- 172.21.15.26:0/3708539435 shutdown_connections 2024-03-06T07:20:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.815+0000 7efea535c640 1 -- 172.21.15.26:0/3708539435 wait complete. 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 Processor -- start 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 -- start start 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea00724d0 0x7efea01ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea00a7ff0 0x7efea00ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c8d50 con 0x7efea0152b40 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00704d0 con 0x7efea00724d0 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efea535c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c8ed0 con 0x7efea00a7ff0 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea014a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58482/0 (socket says 172.21.15.26:58482) 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.816+0000 7efe9f7fe640 1 -- 172.21.15.26:0/4058027360 learned_addr learned my addr 172.21.15.26:0/4058027360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe97fff640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea00a7ff0 0x7efea00ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9effd640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea00724d0 0x7efea01ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9f7fe640 1 -- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea00a7ff0 msgr2=0x7efea00ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9f7fe640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea00a7ff0 0x7efea00ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9f7fe640 1 -- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea00724d0 msgr2=0x7efea01ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9f7fe640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea00724d0 0x7efea01ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9f7fe640 1 -- 172.21.15.26:0/4058027360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00adc70 con 0x7efea0152b40 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9effd640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea00724d0 0x7efea01ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe9f7fe640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea014a390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efe9800ca80 tx=0x7efe9800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe977fe640 1 -- 172.21.15.26:0/4058027360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98017070 con 0x7efea0152b40 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe977fe640 1 -- 172.21.15.26:0/4058027360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9800ac30 con 0x7efea0152b40 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efe977fe640 1 -- 172.21.15.26:0/4058027360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98012600 con 0x7efea0152b40 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea014b070 con 0x7efea0152b40 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.817+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea00abe70 con 0x7efea0152b40 2024-03-06T07:20:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.818+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea0075300 con 0x7efea0152b40 2024-03-06T07:20:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.818+0000 7efe977fe640 1 -- 172.21.15.26:0/4058027360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe9801a070 con 0x7efea0152b40 2024-03-06T07:20:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.819+0000 7efe977fe640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe8006b4d0 0x7efe8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.819+0000 7efe977fe640 1 -- 172.21.15.26:0/4058027360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efe98089c80 con 0x7efea0152b40 2024-03-06T07:20:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.819+0000 7efe9effd640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe8006b4d0 0x7efe8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.819+0000 7efe9effd640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe8006b4d0 0x7efe8006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe90036000 tx=0x7efe90008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:54.823+0000 7efe977fe640 1 -- 172.21.15.26:0/4058027360 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efea0075300 con 0x7efea0152b40 2024-03-06T07:20:55.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.071+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7efea004f330 con 0x7efea0152b40 2024-03-06T07:20:55.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.073+0000 7efe977fe640 1 -- 172.21.15.26:0/4058027360 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7efe9800ae20 con 0x7efea0152b40 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe8006b4d0 msgr2=0x7efe8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe8006b4d0 0x7efe8006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe90036000 tx=0x7efe90008030 comp rx=0 tx=0).stop 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 msgr2=0x7efea014a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea014a390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efe9800ca80 tx=0x7efe9800cf40 comp rx=0 tx=0).stop 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 shutdown_connections 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe8006b4d0 0x7efe8006d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0152b40 0x7efea014a390 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea00a7ff0 0x7efea00ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 --2- 172.21.15.26:0/4058027360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea00724d0 0x7efea01ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 >> 172.21.15.26:0/4058027360 conn(0x7efea006b970 msgr2=0x7efea0151630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 shutdown_connections 2024-03-06T07:20:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.075+0000 7efea535c640 1 -- 172.21.15.26:0/4058027360 wait complete. 2024-03-06T07:20:55.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-06T07:20:55.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.189+0000 7fd8b2233640 1 Processor -- start 2024-03-06T07:20:55.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8b2233640 1 -- start start 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8b2233640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8ac0c6ab0 0x7fd8ac1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8b2233640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac1561a0 0x7fd8ac0a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8b2233640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac0a9990 con 0x7fd8ac0aa430 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8b2233640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac155d10 con 0x7fd8ac0c6ab0 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8b2233640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac1556c0 con 0x7fd8ac1561a0 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8abfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd8ac0aa430 0x7fd8ac14f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43996/0 (socket says 172.21.15.26:43996) 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8ab7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8ac0c6ab0 0x7fd8ac1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.190+0000 7fd8abfff640 1 -- 172.21.15.26:0/2774680652 learned_addr learned my addr 172.21.15.26:0/2774680652 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.191+0000 7fd8aaffd640 1 --2- 172.21.15.26:0/2774680652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac1561a0 0x7fd8ac0a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.191+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/2774680652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac1561a0 msgr2=0x7fd8ac0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.191+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/2774680652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac1561a0 0x7fd8ac0a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.191+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/2774680652 >> v1:172.21.15.26:6789/0 conn(0x7fd8ac0aa430 legacy=0x7fd8ac14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.191+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/2774680652 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac1559a0 con 0x7fd8ac0c6ab0 2024-03-06T07:20:55.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.191+0000 7fd8aa7fc640 1 -- 172.21.15.26:0/2774680652 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726791036 0 0) 0x7fd8ac0a9990 con 0x7fd8ac0aa430 2024-03-06T07:20:55.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.191+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/2774680652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac0c6ab0 0x7fd8ac1529e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0097a0 tx=0x7fd89c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85838106e33708d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8aa7fc640 1 -- 172.21.15.26:0/2774680652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0315a0 con 0x7fd8ac0c6ab0 2024-03-06T07:20:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8aa7fc640 1 -- 172.21.15.26:0/2774680652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c037070 con 0x7fd8ac0c6ab0 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8aa7fc640 1 -- 172.21.15.26:0/2774680652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c031960 con 0x7fd8ac0c6ab0 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 -- 172.21.15.26:0/2774680652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac0c6ab0 msgr2=0x7fd8ac1529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 --2- 172.21.15.26:0/2774680652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac0c6ab0 0x7fd8ac1529e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0097a0 tx=0x7fd89c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 -- 172.21.15.26:0/2774680652 shutdown_connections 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 --2- 172.21.15.26:0/2774680652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac1561a0 0x7fd8ac0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 --2- 172.21.15.26:0/2774680652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac0c6ab0 0x7fd8ac1529e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 -- 172.21.15.26:0/2774680652 >> 172.21.15.26:0/2774680652 conn(0x7fd8ac06b1a0 msgr2=0x7fd8ac06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 -- 172.21.15.26:0/2774680652 shutdown_connections 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.192+0000 7fd8b2233640 1 -- 172.21.15.26:0/2774680652 wait complete. 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.193+0000 7fd8b2233640 1 Processor -- start 2024-03-06T07:20:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8b2233640 1 -- start start 2024-03-06T07:20:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8b2233640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac0aa430 0x7fd8ac14ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8b2233640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8ac0c6ab0 0x7fd8ac1504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8b2233640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1561a0 0x7fd8ac0713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8b2233640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac14f820 con 0x7fd8ac1561a0 2024-03-06T07:20:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8b2233640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac0c2a90 con 0x7fd8ac0c6ab0 2024-03-06T07:20:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8b2233640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac143990 con 0x7fd8ac0aa430 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1561a0 0x7fd8ac0713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.194+0000 7fd8abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1561a0 0x7fd8ac0713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58496/0 (socket says 172.21.15.26:58496) 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8abfff640 1 -- 172.21.15.26:0/1521932480 learned_addr learned my addr 172.21.15.26:0/1521932480 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8aaffd640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8ac0c6ab0 0x7fd8ac1504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac0aa430 0x7fd8ac14ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8abfff640 1 -- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac0aa430 msgr2=0x7fd8ac14ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8abfff640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac0aa430 0x7fd8ac14ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8abfff640 1 -- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8ac0c6ab0 msgr2=0x7fd8ac1504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8abfff640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8ac0c6ab0 0x7fd8ac1504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8abfff640 1 -- 172.21.15.26:0/1521932480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac150b20 con 0x7fd8ac1561a0 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac0aa430 0x7fd8ac14ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8abfff640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1561a0 0x7fd8ac0713a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c730 tx=0x7fd8a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8a3fff640 1 -- 172.21.15.26:0/1521932480 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4018070 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8a3fff640 1 -- 172.21.15.26:0/1521932480 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a400ce40 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac143be0 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8a3fff640 1 -- 172.21.15.26:0/1521932480 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400a860 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.195+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac1441a0 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.196+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac0aab20 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.197+0000 7fd8a3fff640 1 -- 172.21.15.26:0/1521932480 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8a401b070 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.197+0000 7fd8a3fff640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd88c06b4d0 0x7fd88c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.197+0000 7fd8a3fff640 1 -- 172.21.15.26:0/1521932480 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd8a40895a0 con 0x7fd8ac1561a0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.197+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd88c06b4d0 0x7fd88c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.198+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd88c06b4d0 0x7fd88c06d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd89c036000 tx=0x7fd89c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.202+0000 7fd8a3fff640 1 -- 172.21.15.26:0/1521932480 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd8ac0aab20 con 0x7fd8ac1561a0 2024-03-06T07:20:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.453+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd8ac04f330 con 0x7fd8ac1561a0 2024-03-06T07:20:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.456+0000 7fd8a3fff640 1 -- 172.21.15.26:0/1521932480 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd8a4057f80 con 0x7fd8ac1561a0 2024-03-06T07:20:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd88c06b4d0 msgr2=0x7fd88c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd88c06b4d0 0x7fd88c06d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd89c036000 tx=0x7fd89c008030 comp rx=0 tx=0).stop 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1561a0 msgr2=0x7fd8ac0713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1561a0 0x7fd8ac0713a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c730 tx=0x7fd8a400cbf0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 shutdown_connections 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd88c06b4d0 0x7fd88c06d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1561a0 0x7fd8ac0713a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8ac0c6ab0 0x7fd8ac1504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 --2- 172.21.15.26:0/1521932480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8ac0aa430 0x7fd8ac14ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 >> 172.21.15.26:0/1521932480 conn(0x7fd8ac06b1a0 msgr2=0x7fd8ac0c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 shutdown_connections 2024-03-06T07:20:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.458+0000 7fd8b2233640 1 -- 172.21.15.26:0/1521932480 wait complete. 2024-03-06T07:20:55.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-06T07:20:55.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.576+0000 7f209e26b640 1 Processor -- start 2024-03-06T07:20:55.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.576+0000 7f209e26b640 1 -- start start 2024-03-06T07:20:55.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209e26b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20980c8950 0x7f20980c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209e26b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20981558d0 0x7f2098155d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209e26b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2098155130 con 0x7f20980c69f0 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209e26b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2098155440 con 0x7f20981558d0 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209e26b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20980ad480 con 0x7f20980c8950 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209c808640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20981558d0 0x7f2098155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209c808640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20981558d0 0x7f2098155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58498/0 (socket says 172.21.15.26:58498) 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.577+0000 7f209c808640 1 -- 172.21.15.26:0/1703979267 learned_addr learned my addr 172.21.15.26:0/1703979267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.578+0000 7f20977fe640 1 --2- 172.21.15.26:0/1703979267 >> v2:172.21.15.117:3300/0 conn(0x7f20980c8950 0x7f20980c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.578+0000 7f209c808640 1 -- 172.21.15.26:0/1703979267 >> v2:172.21.15.117:3300/0 conn(0x7f20980c8950 msgr2=0x7f20980c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.578+0000 7f209c808640 1 --2- 172.21.15.26:0/1703979267 >> v2:172.21.15.117:3300/0 conn(0x7f20980c8950 0x7f20980c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.578+0000 7f209c808640 1 -- 172.21.15.26:0/1703979267 >> v1:172.21.15.117:6789/0 conn(0x7f20980c69f0 legacy=0x7f20980acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.578+0000 7f209c808640 1 -- 172.21.15.26:0/1703979267 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980c5d50 con 0x7f20981558d0 2024-03-06T07:20:55.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.579+0000 7f209c808640 1 --2- 172.21.15.26:0/1703979267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981558d0 0x7f2098155d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f20980c7430 tx=0x7f209002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5c07ff34fa4d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.579+0000 7f2096ffd640 1 -- 172.21.15.26:0/1703979267 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209002ca80 con 0x7f20981558d0 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.579+0000 7f2096ffd640 1 -- 172.21.15.26:0/1703979267 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209002cc20 con 0x7f20981558d0 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.579+0000 7f2096ffd640 1 -- 172.21.15.26:0/1703979267 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900343f0 con 0x7f20981558d0 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 -- 172.21.15.26:0/1703979267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981558d0 msgr2=0x7f2098155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 --2- 172.21.15.26:0/1703979267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981558d0 0x7f2098155d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f20980c7430 tx=0x7f209002bee0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 -- 172.21.15.26:0/1703979267 shutdown_connections 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 --2- 172.21.15.26:0/1703979267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981558d0 0x7f2098155d00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 --2- 172.21.15.26:0/1703979267 >> v2:172.21.15.117:3300/0 conn(0x7f20980c8950 0x7f20980c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 -- 172.21.15.26:0/1703979267 >> 172.21.15.26:0/1703979267 conn(0x7f20980cae70 msgr2=0x7f20980cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 -- 172.21.15.26:0/1703979267 shutdown_connections 2024-03-06T07:20:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.580+0000 7f209e26b640 1 -- 172.21.15.26:0/1703979267 wait complete. 2024-03-06T07:20:55.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.581+0000 7f209e26b640 1 Processor -- start 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.581+0000 7f209e26b640 1 -- start start 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f209e26b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20980c69f0 0x7f20980c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f209e26b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20980c8950 0x7f20981ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f209e26b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20981558d0 0x7f2098153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f209e26b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098152a20 con 0x7f20980c8950 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f209e26b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20980c6310 con 0x7f20981558d0 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f209e26b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2098152ba0 con 0x7f20980c69f0 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20980c8950 0x7f20981ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20980c8950 0x7f20981ee2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58502/0 (socket says 172.21.15.26:58502) 2024-03-06T07:20:55.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 -- 172.21.15.26:0/1120040749 learned_addr learned my addr 172.21.15.26:0/1120040749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 -- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20980c69f0 msgr2=0x7f20980c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f209c808640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20981558d0 0x7f2098153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f2097fff640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20980c69f0 0x7f20980c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20980c69f0 0x7f20980c1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 -- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20981558d0 msgr2=0x7f2098153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20981558d0 0x7f2098153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 -- 172.21.15.26:0/1120040749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980c2fb0 con 0x7f20980c8950 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f20977fe640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20980c8950 0x7f20981ee2b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f208800e870 tx=0x7f208800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.582+0000 7f2094ff9640 1 -- 172.21.15.26:0/1120040749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088017070 con 0x7f20980c8950 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.583+0000 7f2094ff9640 1 -- 172.21.15.26:0/1120040749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800c940 con 0x7f20980c8950 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.583+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20981eeb80 con 0x7f20980c8950 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.583+0000 7f2094ff9640 1 -- 172.21.15.26:0/1120040749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800cd30 con 0x7f20980c8950 2024-03-06T07:20:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.583+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20980c47f0 con 0x7f20980c8950 2024-03-06T07:20:55.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.584+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209804f330 con 0x7f20980c8950 2024-03-06T07:20:55.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.584+0000 7f2094ff9640 1 -- 172.21.15.26:0/1120040749 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f208801a070 con 0x7f20980c8950 2024-03-06T07:20:55.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.585+0000 7f2094ff9640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.585+0000 7f2094ff9640 1 -- 172.21.15.26:0/1120040749 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2088089f00 con 0x7f20980c8950 2024-03-06T07:20:55.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.585+0000 7f2097fff640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.586+0000 7f2097fff640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f208406b4d0 0x7f208406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f208c001500 tx=0x7f208c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.589+0000 7f2094ff9640 1 -- 172.21.15.26:0/1120040749 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2088058f40 con 0x7f20980c8950 2024-03-06T07:20:55.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.839+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f20980ca650 con 0x7f20980c8950 2024-03-06T07:20:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.841+0000 7f2094ff9640 1 -- 172.21.15.26:0/1120040749 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2088058840 con 0x7f20980c8950 2024-03-06T07:20:55.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f208406b4d0 msgr2=0x7f208406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f208406b4d0 0x7f208406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f208c001500 tx=0x7f208c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20980c8950 msgr2=0x7f20981ee2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20980c8950 0x7f20981ee2b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f208800e870 tx=0x7f208800ed30 comp rx=0 tx=0).stop 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 shutdown_connections 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20981558d0 0x7f2098153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20980c8950 0x7f20981ee2b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 --2- 172.21.15.26:0/1120040749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20980c69f0 0x7f20980c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 >> 172.21.15.26:0/1120040749 conn(0x7f20980cae70 msgr2=0x7f2098151720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 shutdown_connections 2024-03-06T07:20:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.843+0000 7f209e26b640 1 -- 172.21.15.26:0/1120040749 wait complete. 2024-03-06T07:20:55.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-06T07:20:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.956+0000 7f9ba8466640 1 Processor -- start 2024-03-06T07:20:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.957+0000 7f9ba8466640 1 -- start start 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.957+0000 7f9ba8466640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c7170 0x7f9ba00c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.957+0000 7f9ba8466640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00a8390 con 0x7f9ba006ff10 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba8466640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00730f0 con 0x7f9ba00c6c40 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba8466640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c7620 con 0x7f9ba00c7170 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba6202640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ba006ff10 0x7f9ba0153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44004/0 (socket says 172.21.15.26:44004) 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba6202640 1 -- 172.21.15.26:0/314394799 learned_addr learned my addr 172.21.15.26:0/314394799 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba5a01640 1 --2- 172.21.15.26:0/314394799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c7170 0x7f9ba00c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba5200640 1 -- 172.21.15.26:0/314394799 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286239405 0 0) 0x7f9ba00a8390 con 0x7f9ba006ff10 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba5200640 1 -- 172.21.15.26:0/314394799 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b8c003520 con 0x7f9ba006ff10 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba5a01640 1 -- 172.21.15.26:0/314394799 >> v1:172.21.15.117:6789/0 conn(0x7f9ba00c6c40 legacy=0x7f9ba015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba5200640 1 -- 172.21.15.26:0/314394799 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322247018 0 0) 0x7f9ba00730f0 con 0x7f9ba00c6c40 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba5a01640 1 -- 172.21.15.26:0/314394799 >> v1:172.21.15.26:6789/0 conn(0x7f9ba006ff10 legacy=0x7f9ba0153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.958+0000 7f9ba5a01640 1 -- 172.21.15.26:0/314394799 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba00733d0 con 0x7f9ba00c7170 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.959+0000 7f9ba5a01640 1 --2- 172.21.15.26:0/314394799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c7170 0x7f9ba00c8810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0097d0 tx=0x7f9b9c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fecb47072c1f3629 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.959+0000 7f9ba5200640 1 -- 172.21.15.26:0/314394799 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c031770 con 0x7f9ba00c7170 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.959+0000 7f9ba5200640 1 -- 172.21.15.26:0/314394799 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c037070 con 0x7f9ba00c7170 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.959+0000 7f9ba5200640 1 -- 172.21.15.26:0/314394799 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c031b10 con 0x7f9ba00c7170 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.960+0000 7f9ba8466640 1 -- 172.21.15.26:0/314394799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c7170 msgr2=0x7f9ba00c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.960+0000 7f9ba8466640 1 --2- 172.21.15.26:0/314394799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c7170 0x7f9ba00c8810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0097d0 tx=0x7f9b9c02b810 comp rx=0 tx=0).stop 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.960+0000 7f9ba8466640 1 -- 172.21.15.26:0/314394799 shutdown_connections 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.960+0000 7f9ba8466640 1 --2- 172.21.15.26:0/314394799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c7170 0x7f9ba00c8810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.960+0000 7f9ba8466640 1 -- 172.21.15.26:0/314394799 >> 172.21.15.26:0/314394799 conn(0x7f9ba006b970 msgr2=0x7f9ba00aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.960+0000 7f9ba8466640 1 -- 172.21.15.26:0/314394799 shutdown_connections 2024-03-06T07:20:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.960+0000 7f9ba8466640 1 -- 172.21.15.26:0/314394799 wait complete. 2024-03-06T07:20:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.961+0000 7f9ba8466640 1 Processor -- start 2024-03-06T07:20:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.961+0000 7f9ba8466640 1 -- start start 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba8466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ba006ff10 0x7f9ba01eb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba8466640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c6c40 0x7f9ba01560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba8466640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba00c7170 0x7f9ba00c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba8466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01544b0 con 0x7f9ba006ff10 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba8466640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba0154330 con 0x7f9ba00c7170 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba8466640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00702f0 con 0x7f9ba00c6c40 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba6202640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ba006ff10 0x7f9ba01eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba6202640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ba006ff10 0x7f9ba01eb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58514/0 (socket says 172.21.15.26:58514) 2024-03-06T07:20:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba6202640 1 -- 172.21.15.26:0/2422408263 learned_addr learned my addr 172.21.15.26:0/2422408263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba5a01640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c6c40 0x7f9ba01560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.962+0000 7f9ba6a03640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba00c7170 0x7f9ba00c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba6202640 1 -- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c6c40 msgr2=0x7f9ba01560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba6202640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c6c40 0x7f9ba01560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba6202640 1 -- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba00c7170 msgr2=0x7f9ba00c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba6202640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba00c7170 0x7f9ba00c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba6202640 1 -- 172.21.15.26:0/2422408263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0156ef0 con 0x7f9ba006ff10 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba6a03640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba00c7170 0x7f9ba00c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba5a01640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c6c40 0x7f9ba01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba6202640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ba006ff10 0x7f9ba01eb980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c390 tx=0x7f9b9000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9b96ffd640 1 -- 172.21.15.26:0/2422408263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90017070 con 0x7f9ba006ff10 2024-03-06T07:20:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9b96ffd640 1 -- 172.21.15.26:0/2422408263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90005ce0 con 0x7f9ba006ff10 2024-03-06T07:20:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba01571d0 con 0x7f9ba006ff10 2024-03-06T07:20:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9b96ffd640 1 -- 172.21.15.26:0/2422408263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900072f0 con 0x7f9ba006ff10 2024-03-06T07:20:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.963+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba01e8570 con 0x7f9ba006ff10 2024-03-06T07:20:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.964+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba004f330 con 0x7f9ba006ff10 2024-03-06T07:20:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.964+0000 7f9b96ffd640 1 -- 172.21.15.26:0/2422408263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b9001a070 con 0x7f9ba006ff10 2024-03-06T07:20:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.965+0000 7f9b96ffd640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b8406b4d0 0x7f9b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.965+0000 7f9b96ffd640 1 -- 172.21.15.26:0/2422408263 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9b900893c0 con 0x7f9ba006ff10 2024-03-06T07:20:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.965+0000 7f9ba5a01640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b8406b4d0 0x7f9b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.966+0000 7f9ba5a01640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b8406b4d0 0x7f9b8406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c036000 tx=0x7f9b9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:55.969+0000 7f9b96ffd640 1 -- 172.21.15.26:0/2422408263 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b90058490 con 0x7f9ba006ff10 2024-03-06T07:20:56.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.221+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9ba006fa80 con 0x7f9ba006ff10 2024-03-06T07:20:56.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.223+0000 7f9b96ffd640 1 -- 172.21.15.26:0/2422408263 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9b90057d90 con 0x7f9ba006ff10 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b8406b4d0 msgr2=0x7f9b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b8406b4d0 0x7f9b8406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c036000 tx=0x7f9b9c008030 comp rx=0 tx=0).stop 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ba006ff10 msgr2=0x7f9ba01eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ba006ff10 0x7f9ba01eb980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c390 tx=0x7f9b9000c850 comp rx=0 tx=0).stop 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 shutdown_connections 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b8406b4d0 0x7f9b8406d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba00c7170 0x7f9ba00c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ba00c6c40 0x7f9ba01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 --2- 172.21.15.26:0/2422408263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ba006ff10 0x7f9ba01eb980 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.225+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 >> 172.21.15.26:0/2422408263 conn(0x7f9ba006b970 msgr2=0x7f9ba01539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:56.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.226+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 shutdown_connections 2024-03-06T07:20:56.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.226+0000 7f9ba8466640 1 -- 172.21.15.26:0/2422408263 wait complete. 2024-03-06T07:20:56.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-06T07:20:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.341+0000 7fa395088640 1 Processor -- start 2024-03-06T07:20:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.341+0000 7fa395088640 1 -- start start 2024-03-06T07:20:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa395088640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3900c1150 0x7fa39006b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa395088640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3900c53d0 0x7fa3900c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa395088640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa390159520 0x7fa3901552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa395088640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa390159090 con 0x7fa3900c53d0 2024-03-06T07:20:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa395088640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39013e420 con 0x7fa390159520 2024-03-06T07:20:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa395088640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa390158a40 con 0x7fa3900c1150 2024-03-06T07:20:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3900c53d0 0x7fa3900c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3900c53d0 0x7fa3900c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58516/0 (socket says 172.21.15.26:58516) 2024-03-06T07:20:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 -- 172.21.15.26:0/1112991167 learned_addr learned my addr 172.21.15.26:0/1112991167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 -- 172.21.15.26:0/1112991167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3900c1150 msgr2=0x7fa39006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 --2- 172.21.15.26:0/1112991167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3900c1150 0x7fa39006b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 -- 172.21.15.26:0/1112991167 >> v2:172.21.15.117:3300/0 conn(0x7fa390159520 msgr2=0x7fa3901552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 --2- 172.21.15.26:0/1112991167 >> v2:172.21.15.117:3300/0 conn(0x7fa390159520 0x7fa3901552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.342+0000 7fa38e59c640 1 -- 172.21.15.26:0/1112991167 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa390158d20 con 0x7fa3900c53d0 2024-03-06T07:20:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.343+0000 7fa38e59c640 1 --2- 172.21.15.26:0/1112991167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900c8950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa39006b100 tx=0x7fa38402b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef96a7eb59d589 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.343+0000 7fa38dd9b640 1 -- 172.21.15.26:0/1112991167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384031760 con 0x7fa3900c53d0 2024-03-06T07:20:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.343+0000 7fa38dd9b640 1 -- 172.21.15.26:0/1112991167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384037070 con 0x7fa3900c53d0 2024-03-06T07:20:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.343+0000 7fa38dd9b640 1 -- 172.21.15.26:0/1112991167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384031b20 con 0x7fa3900c53d0 2024-03-06T07:20:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.343+0000 7fa395088640 1 -- 172.21.15.26:0/1112991167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 msgr2=0x7fa3900c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.343+0000 7fa395088640 1 --2- 172.21.15.26:0/1112991167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900c8950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa39006b100 tx=0x7fa38402b800 comp rx=0 tx=0).stop 2024-03-06T07:20:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.344+0000 7fa395088640 1 -- 172.21.15.26:0/1112991167 shutdown_connections 2024-03-06T07:20:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.344+0000 7fa395088640 1 --2- 172.21.15.26:0/1112991167 >> v2:172.21.15.117:3300/0 conn(0x7fa390159520 0x7fa3901552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.344+0000 7fa395088640 1 --2- 172.21.15.26:0/1112991167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900c8950 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.344+0000 7fa395088640 1 --2- 172.21.15.26:0/1112991167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3900c1150 0x7fa39006b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.344+0000 7fa395088640 1 -- 172.21.15.26:0/1112991167 >> 172.21.15.26:0/1112991167 conn(0x7fa3900ac680 msgr2=0x7fa3900aca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.344+0000 7fa395088640 1 -- 172.21.15.26:0/1112991167 shutdown_connections 2024-03-06T07:20:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.344+0000 7fa395088640 1 -- 172.21.15.26:0/1112991167 wait complete. 2024-03-06T07:20:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.345+0000 7fa395088640 1 Processor -- start 2024-03-06T07:20:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.345+0000 7fa395088640 1 -- start start 2024-03-06T07:20:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3900c1150 0x7fa3901eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa390159520 0x7fa390152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39013e7d0 con 0x7fa3900c53d0 2024-03-06T07:20:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900c1530 con 0x7fa3900c1150 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39013e950 con 0x7fa390159520 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58528/0 (socket says 172.21.15.26:58528) 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 -- 172.21.15.26:0/2665460595 learned_addr learned my addr 172.21.15.26:0/2665460595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38f59e640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa390159520 0x7fa390152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 -- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa390159520 msgr2=0x7fa390152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa390159520 0x7fa390152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 -- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3900c1150 msgr2=0x7fa3901eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38ed9d640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3900c1150 0x7fa3901eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3900c1150 0x7fa3901eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 -- 172.21.15.26:0/2665460595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900ad770 con 0x7fa3900c53d0 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38ed9d640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3900c1150 0x7fa3901eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa38e59c640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900afd20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa3840090b0 tx=0x7fa3840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa3777fe640 1 -- 172.21.15.26:0/2665460595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38403b800 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa3777fe640 1 -- 172.21.15.26:0/2665460595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38403b9a0 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39013a1f0 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa3777fe640 1 -- 172.21.15.26:0/2665460595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38403bd70 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.346+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3900adcf0 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.347+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390152ed0 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.348+0000 7fa3777fe640 1 -- 172.21.15.26:0/2665460595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa38404c060 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.348+0000 7fa3777fe640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa37c06b4d0 0x7fa37c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.349+0000 7fa3777fe640 1 -- 172.21.15.26:0/2665460595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa3840ab440 con 0x7fa3900c53d0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.349+0000 7fa38ed9d640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa37c06b4d0 0x7fa37c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.349+0000 7fa38ed9d640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa37c06b4d0 0x7fa37c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.353+0000 7fa3777fe640 1 -- 172.21.15.26:0/2665460595 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa384040070 con 0x7fa3900c53d0 2024-03-06T07:20:56.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.602+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa39004f330 con 0x7fa3900c53d0 2024-03-06T07:20:56.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.604+0000 7fa3777fe640 1 -- 172.21.15.26:0/2665460595 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa38407a510 con 0x7fa3900c53d0 2024-03-06T07:20:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa37c06b4d0 msgr2=0x7fa37c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa37c06b4d0 0x7fa37c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380008040 comp rx=0 tx=0).stop 2024-03-06T07:20:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 msgr2=0x7fa3900afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900afd20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa3840090b0 tx=0x7fa3840046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 shutdown_connections 2024-03-06T07:20:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa37c06b4d0 0x7fa37c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa390159520 0x7fa390152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3900c53d0 0x7fa3900afd20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 --2- 172.21.15.26:0/2665460595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3900c1150 0x7fa3901eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 >> 172.21.15.26:0/2665460595 conn(0x7fa3900ac680 msgr2=0x7fa3900aa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 shutdown_connections 2024-03-06T07:20:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.606+0000 7fa395088640 1 -- 172.21.15.26:0/2665460595 wait complete. 2024-03-06T07:20:56.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-06T07:20:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.719+0000 7f7feb99a640 1 Processor -- start 2024-03-06T07:20:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.719+0000 7f7feb99a640 1 -- start start 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7feb99a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe4155120 0x7f7fe40c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7feb99a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe4159590 con 0x7f7fe40bfe90 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7feb99a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe40c59d0 con 0x7f7fe41522d0 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7feb99a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe4158710 con 0x7f7fe4155120 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7fe8f35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe4155120 0x7f7fe40c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7fe9736640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7fe40bfe90 0x7f7fe4158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44020/0 (socket says 172.21.15.26:44020) 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7fe9736640 1 -- 172.21.15.26:0/2152337107 learned_addr learned my addr 172.21.15.26:0/2152337107 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7fe8f35640 1 -- 172.21.15.26:0/2152337107 >> v1:172.21.15.117:6789/0 conn(0x7f7fe41522d0 legacy=0x7f7fe415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7fe8f35640 1 -- 172.21.15.26:0/2152337107 >> v1:172.21.15.26:6789/0 conn(0x7f7fe40bfe90 legacy=0x7f7fe4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7fe8f35640 1 -- 172.21.15.26:0/2152337107 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe40c5cb0 con 0x7f7fe4155120 2024-03-06T07:20:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.720+0000 7f7fdbfff640 1 -- 172.21.15.26:0/2152337107 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525359747 0 0) 0x7f7fe4159590 con 0x7f7fe40bfe90 2024-03-06T07:20:56.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.721+0000 7f7fe8f35640 1 --2- 172.21.15.26:0/2152337107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe4155120 0x7f7fe40c6d70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4158890 tx=0x7f7fd402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84318af190a8060 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.721+0000 7f7fdbfff640 1 -- 172.21.15.26:0/2152337107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4035020 con 0x7f7fe4155120 2024-03-06T07:20:56.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.721+0000 7f7fdbfff640 1 -- 172.21.15.26:0/2152337107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd4031800 con 0x7f7fe4155120 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.721+0000 7f7fdbfff640 1 -- 172.21.15.26:0/2152337107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4031bd0 con 0x7f7fe4155120 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.722+0000 7f7feb99a640 1 -- 172.21.15.26:0/2152337107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe4155120 msgr2=0x7f7fe40c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.722+0000 7f7feb99a640 1 --2- 172.21.15.26:0/2152337107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe4155120 0x7f7fe40c6d70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4158890 tx=0x7f7fd402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.722+0000 7f7feb99a640 1 -- 172.21.15.26:0/2152337107 shutdown_connections 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.722+0000 7f7feb99a640 1 --2- 172.21.15.26:0/2152337107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe4155120 0x7f7fe40c6d70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.722+0000 7f7feb99a640 1 -- 172.21.15.26:0/2152337107 >> 172.21.15.26:0/2152337107 conn(0x7f7fe40acaa0 msgr2=0x7f7fe40aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.722+0000 7f7feb99a640 1 -- 172.21.15.26:0/2152337107 shutdown_connections 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.722+0000 7f7feb99a640 1 -- 172.21.15.26:0/2152337107 wait complete. 2024-03-06T07:20:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.723+0000 7f7feb99a640 1 Processor -- start 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.723+0000 7f7feb99a640 1 -- start start 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7feb99a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe40bfe90 0x7f7fe40afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7feb99a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe41522d0 0x7f7fe41508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7feb99a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe4155120 0x7f7fe41e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7feb99a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40ca700 con 0x7f7fe40bfe90 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7feb99a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4158960 con 0x7f7fe41522d0 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7feb99a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40ca880 con 0x7f7fe4155120 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe40bfe90 0x7f7fe40afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe40bfe90 0x7f7fe40afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58544/0 (socket says 172.21.15.26:58544) 2024-03-06T07:20:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 -- 172.21.15.26:0/1874712164 learned_addr learned my addr 172.21.15.26:0/1874712164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 -- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe4155120 msgr2=0x7f7fe41e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe4155120 0x7f7fe41e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 -- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe41522d0 msgr2=0x7f7fe41508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe41522d0 0x7f7fe41508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 -- 172.21.15.26:0/1874712164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe41ea820 con 0x7f7fe40bfe90 2024-03-06T07:20:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fe9736640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe40bfe90 0x7f7fe40afd40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c4a0 tx=0x7f7fdc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fd9ffb640 1 -- 172.21.15.26:0/1874712164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc018070 con 0x7f7fe40bfe90 2024-03-06T07:20:56.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fd9ffb640 1 -- 172.21.15.26:0/1874712164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc0074e0 con 0x7f7fe40bfe90 2024-03-06T07:20:56.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe406b060 con 0x7f7fe40bfe90 2024-03-06T07:20:56.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.724+0000 7f7fd9ffb640 1 -- 172.21.15.26:0/1874712164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc004a20 con 0x7f7fe40bfe90 2024-03-06T07:20:56.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.725+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe406b620 con 0x7f7fe40bfe90 2024-03-06T07:20:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.725+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe404f330 con 0x7f7fe40bfe90 2024-03-06T07:20:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.726+0000 7f7fd9ffb640 1 -- 172.21.15.26:0/1874712164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fdc01b070 con 0x7f7fe40bfe90 2024-03-06T07:20:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.726+0000 7f7fd9ffb640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc806b4d0 0x7f7fc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.726+0000 7f7fd9ffb640 1 -- 172.21.15.26:0/1874712164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7fdc089290 con 0x7f7fe40bfe90 2024-03-06T07:20:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.726+0000 7f7fe8f35640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc806b4d0 0x7f7fc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.727+0000 7f7fe8f35640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc806b4d0 0x7f7fc806d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4036000 tx=0x7f7fd4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.730+0000 7f7fd9ffb640 1 -- 172.21.15.26:0/1874712164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7fdc058360 con 0x7f7fe40bfe90 2024-03-06T07:20:56.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.978+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f7fe40ac430 con 0x7f7fe40bfe90 2024-03-06T07:20:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.980+0000 7f7fd9ffb640 1 -- 172.21.15.26:0/1874712164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7fdc057c60 con 0x7f7fe40bfe90 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc806b4d0 msgr2=0x7f7fc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc806b4d0 0x7f7fc806d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4036000 tx=0x7f7fd4030000 comp rx=0 tx=0).stop 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe40bfe90 msgr2=0x7f7fe40afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe40bfe90 0x7f7fe40afd40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c4a0 tx=0x7f7fdc00c960 comp rx=0 tx=0).stop 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 shutdown_connections 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc806b4d0 0x7f7fc806d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe4155120 0x7f7fe41e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe41522d0 0x7f7fe41508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 --2- 172.21.15.26:0/1874712164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe40bfe90 0x7f7fe40afd40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 >> 172.21.15.26:0/1874712164 conn(0x7f7fe40acaa0 msgr2=0x7f7fe40ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 shutdown_connections 2024-03-06T07:20:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:56.982+0000 7f7feb99a640 1 -- 172.21.15.26:0/1874712164 wait complete. 2024-03-06T07:20:56.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.095+0000 7f973082a640 1 Processor -- start 2024-03-06T07:20:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.095+0000 7f973082a640 1 -- start start 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.095+0000 7f973082a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f972c0c25c0 0x7f972c06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.095+0000 7f973082a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972c0a9530 con 0x7f972c06afc0 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.095+0000 7f973082a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972c0a9810 con 0x7f972c0c20b0 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.095+0000 7f973082a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972c0a9af0 con 0x7f972c0c25c0 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.096+0000 7f972ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f972c0c25c0 0x7f972c06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.096+0000 7f972a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f972c06afc0 0x7f972c153180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57880/0 (socket says 172.21.15.26:57880) 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.096+0000 7f972a59c640 1 -- 172.21.15.26:0/3430791241 learned_addr learned my addr 172.21.15.26:0/3430791241 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.096+0000 7f972ad9d640 1 -- 172.21.15.26:0/3430791241 >> v1:172.21.15.117:6789/0 conn(0x7f972c0c20b0 legacy=0x7f972c153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.096+0000 7f972ad9d640 1 -- 172.21.15.26:0/3430791241 >> v1:172.21.15.26:6789/0 conn(0x7f972c06afc0 legacy=0x7f972c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.096+0000 7f972ad9d640 1 -- 172.21.15.26:0/3430791241 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c0a9dd0 con 0x7f972c0c25c0 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.096+0000 7f972959a640 1 -- 172.21.15.26:0/3430791241 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526542179 0 0) 0x7f972c0a9530 con 0x7f972c06afc0 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f972ad9d640 1 --2- 172.21.15.26:0/3430791241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c06b3a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f971c009d90 tx=0x7f971c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d281a00731c411 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f972959a640 1 -- 172.21.15.26:0/3430791241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c039070 con 0x7f972c0c25c0 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f972959a640 1 -- 172.21.15.26:0/3430791241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c02cb40 con 0x7f972c0c25c0 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f972959a640 1 -- 172.21.15.26:0/3430791241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c034440 con 0x7f972c0c25c0 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f973082a640 1 -- 172.21.15.26:0/3430791241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 msgr2=0x7f972c06b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f973082a640 1 --2- 172.21.15.26:0/3430791241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c06b3a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f971c009d90 tx=0x7f971c02bd90 comp rx=0 tx=0).stop 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f973082a640 1 -- 172.21.15.26:0/3430791241 shutdown_connections 2024-03-06T07:20:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f973082a640 1 --2- 172.21.15.26:0/3430791241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c06b3a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f973082a640 1 -- 172.21.15.26:0/3430791241 >> 172.21.15.26:0/3430791241 conn(0x7f972c0c76b0 msgr2=0x7f972c0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f973082a640 1 -- 172.21.15.26:0/3430791241 shutdown_connections 2024-03-06T07:20:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.097+0000 7f973082a640 1 -- 172.21.15.26:0/3430791241 wait complete. 2024-03-06T07:20:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 Processor -- start 2024-03-06T07:20:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 -- start start 2024-03-06T07:20:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c06afc0 0x7f972c1ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c0c20b0 0x7f972c1de970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c1deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c150b00 con 0x7f972c0c25c0 2024-03-06T07:20:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c150980 con 0x7f972c06afc0 2024-03-06T07:20:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f973082a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c0a7d50 con 0x7f972c0c20b0 2024-03-06T07:20:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f972ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c1deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f972ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c1deeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44234/0 (socket says 172.21.15.26:44234) 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.099+0000 7f972ad9d640 1 -- 172.21.15.26:0/2088296911 learned_addr learned my addr 172.21.15.26:0/2088296911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f972ad9d640 1 -- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c0c20b0 msgr2=0x7f972c1de970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f972ad9d640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c0c20b0 0x7f972c1de970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f972ad9d640 1 -- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c06afc0 msgr2=0x7f972c1ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f972ad9d640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c06afc0 0x7f972c1ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f972ad9d640 1 -- 172.21.15.26:0/2088296911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c147b30 con 0x7f972c0c25c0 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f972ad9d640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c1deeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f971c037040 tx=0x7f971c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f971affd640 1 -- 172.21.15.26:0/2088296911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c039050 con 0x7f972c0c25c0 2024-03-06T07:20:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f971affd640 1 -- 172.21.15.26:0/2088296911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c03a040 con 0x7f972c0c25c0 2024-03-06T07:20:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f971affd640 1 -- 172.21.15.26:0/2088296911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c003990 con 0x7f972c0c25c0 2024-03-06T07:20:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c1de210 con 0x7f972c0c25c0 2024-03-06T07:20:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.100+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c1dcf40 con 0x7f972c0c25c0 2024-03-06T07:20:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.101+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c04f330 con 0x7f972c0c25c0 2024-03-06T07:20:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.101+0000 7f971affd640 1 -- 172.21.15.26:0/2088296911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f971c047030 con 0x7f972c0c25c0 2024-03-06T07:20:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.102+0000 7f971affd640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f970c06b4d0 0x7f970c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.102+0000 7f971affd640 1 -- 172.21.15.26:0/2088296911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f971c0ac4e0 con 0x7f972c0c25c0 2024-03-06T07:20:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.102+0000 7f972a59c640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f970c06b4d0 0x7f970c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.103+0000 7f972a59c640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f970c06b4d0 0x7f970c06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9724007fd0 tx=0x7f9724007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.106+0000 7f971affd640 1 -- 172.21.15.26:0/2088296911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f971c04e020 con 0x7f972c0c25c0 2024-03-06T07:20:57.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.288+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f972c0c57b0 con 0x7f972c0c25c0 2024-03-06T07:20:57.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.289+0000 7f971affd640 1 -- 172.21.15.26:0/2088296911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+13176 (secure 0 0 0) 0x7f971c07b5b0 con 0x7f972c0c25c0 2024-03-06T07:20:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:57.290 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:20:54.312003+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:57.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.290+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f970c06b4d0 msgr2=0x7f970c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.290+0000 7f973082a640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f970c06b4d0 0x7f970c06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9724007fd0 tx=0x7f9724007920 comp rx=0 tx=0).stop 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.290+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 msgr2=0x7f972c1deeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.290+0000 7f973082a640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c1deeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f971c037040 tx=0x7f971c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 shutdown_connections 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f970c06b4d0 0x7f970c06d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972c0c25c0 0x7f972c1deeb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c0c20b0 0x7f972c1de970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 --2- 172.21.15.26:0/2088296911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c06afc0 0x7f972c1ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 >> 172.21.15.26:0/2088296911 conn(0x7f972c0c76b0 msgr2=0x7f972c14e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 shutdown_connections 2024-03-06T07:20:57.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.291+0000 7f973082a640 1 -- 172.21.15.26:0/2088296911 wait complete. 2024-03-06T07:20:57.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:20:57.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:57.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.406+0000 7f4f4224e640 1 Processor -- start 2024-03-06T07:20:57.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.406+0000 7f4f4224e640 1 -- start start 2024-03-06T07:20:57.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.407+0000 7f4f4224e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f3c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.407+0000 7f4f4224e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3c152cb0 con 0x7f4f3c1528d0 2024-03-06T07:20:57.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.407+0000 7f4f4224e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3c0c38b0 con 0x7f4f3c0c67f0 2024-03-06T07:20:57.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.407+0000 7f4f4224e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1594f0 con 0x7f4f3c0c4ef0 2024-03-06T07:20:57.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.407+0000 7f4f3b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4f3c1528d0 0x7f4f3c0724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57886/0 (socket says 172.21.15.26:57886) 2024-03-06T07:20:57.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.407+0000 7f4f3b7fe640 1 -- 172.21.15.26:0/1757437436 learned_addr learned my addr 172.21.15.26:0/1757437436 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:57.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3bfff640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f3c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193600340 0 0) 0x7f4f3c152cb0 con 0x7f4f3c1528d0 2024-03-06T07:20:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f24003520 con 0x7f4f3c1528d0 2024-03-06T07:20:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213671957 0 0) 0x7f4f3c0c38b0 con 0x7f4f3c0c67f0 2024-03-06T07:20:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f3c152cb0 con 0x7f4f3c0c67f0 2024-03-06T07:20:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799995839 0 0) 0x7f4f24003520 con 0x7f4f3c1528d0 2024-03-06T07:20:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 msgr2=0x7f4f3c155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f3c155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 >> v1:172.21.15.117:6789/0 conn(0x7f4f3c0c67f0 legacy=0x7f4f3c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c0a7f70 con 0x7f4f3c1528d0 2024-03-06T07:20:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3bfff640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f3c155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4f2c001670 con 0x7f4f3c1528d0 2024-03-06T07:20:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 >> v1:172.21.15.26:6789/0 conn(0x7f4f3c1528d0 legacy=0x7f4f3c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24003b70 0x7f4f24003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24004800 0x7f4f24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1594f0 con 0x7f4f24003b70 2024-03-06T07:20:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2400b9f0 con 0x7f4f24004800 2024-03-06T07:20:57.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2400bbf0 con 0x7f4f3c0c4ef0 2024-03-06T07:20:57.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3b7fe640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24003b70 0x7f4f24003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3affd640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24004800 0x7f4f24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3bfff640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3b7fe640 1 -- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 msgr2=0x7f4f240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:57.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3b7fe640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3b7fe640 1 -- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24004800 msgr2=0x7f4f24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:57.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3b7fe640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24004800 0x7f4f24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3b7fe640 1 -- 172.21.15.26:0/1757437436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f24003520 con 0x7f4f24003b70 2024-03-06T07:20:57.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.408+0000 7f4f3bfff640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f3c0c4ef0 0x7f4f240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:57.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.409+0000 7f4f3b7fe640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24003b70 0x7f4f24003f70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c007fd0 tx=0x7f4f24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5e19520e2b07c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.409+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c007ce0 con 0x7f4f24003b70 2024-03-06T07:20:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.409+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c030070 con 0x7f4f24003b70 2024-03-06T07:20:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.409+0000 7f4f3a7fc640 1 -- 172.21.15.26:0/1757437436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c005d50 con 0x7f4f24003b70 2024-03-06T07:20:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.410+0000 7f4f4224e640 1 -- 172.21.15.26:0/1757437436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24003b70 msgr2=0x7f4f24003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.410+0000 7f4f4224e640 1 --2- 172.21.15.26:0/1757437436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24003b70 0x7f4f24003f70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c007fd0 tx=0x7f4f24005830 comp rx=0 tx=0).stop 2024-03-06T07:20:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.410+0000 7f4f3bfff640 1 -- 172.21.15.26:0/1757437436 reap_dead start 2024-03-06T07:20:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.410+0000 7f4f4224e640 1 -- 172.21.15.26:0/1757437436 shutdown_connections 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.410+0000 7f4f4224e640 1 -- 172.21.15.26:0/1757437436 >> 172.21.15.26:0/1757437436 conn(0x7f4f3c0aa6a0 msgr2=0x7f4f3c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.410+0000 7f4f4224e640 1 -- 172.21.15.26:0/1757437436 shutdown_connections 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.410+0000 7f4f4224e640 1 -- 172.21.15.26:0/1757437436 wait complete. 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.411+0000 7f4f4224e640 1 Processor -- start 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.411+0000 7f4f4224e640 1 -- start start 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f4224e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f24003b70 0x7f4f3c0ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f4224e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f3c154480 0x7f4f3c1ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f4224e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c1eea10 0x7f4f3c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f4224e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1591a0 con 0x7f4f3c1eea10 2024-03-06T07:20:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f4224e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c0a8250 con 0x7f4f3c154480 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f4224e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c072be0 con 0x7f4f24003b70 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c1eea10 0x7f4f3c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c1eea10 0x7f4f3c151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44262/0 (socket says 172.21.15.26:44262) 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 -- 172.21.15.26:0/3243430173 learned_addr learned my addr 172.21.15.26:0/3243430173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3b7fe640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f24003b70 0x7f4f3c0ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3affd640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f3c154480 0x7f4f3c1ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 -- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f24003b70 msgr2=0x7f4f3c0ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f24003b70 0x7f4f3c0ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 -- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f3c154480 msgr2=0x7f4f3c1ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f3c154480 0x7f4f3c1ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 -- 172.21.15.26:0/3243430173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c0ac550 con 0x7f4f3c1eea10 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.412+0000 7f4f3bfff640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c1eea10 0x7f4f3c151140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c155e70 tx=0x7f4f34009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.413+0000 7f4f23fff640 1 -- 172.21.15.26:0/3243430173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3401a020 con 0x7f4f3c1eea10 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.413+0000 7f4f23fff640 1 -- 172.21.15.26:0/3243430173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f34011d70 con 0x7f4f3c1eea10 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.413+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c0adc30 con 0x7f4f3c1eea10 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.413+0000 7f4f23fff640 1 -- 172.21.15.26:0/3243430173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34016690 con 0x7f4f3c1eea10 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.413+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c0ae1f0 con 0x7f4f3c1eea10 2024-03-06T07:20:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.413+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c04f330 con 0x7f4f3c1eea10 2024-03-06T07:20:57.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.414+0000 7f4f23fff640 1 -- 172.21.15.26:0/3243430173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f34005d10 con 0x7f4f3c1eea10 2024-03-06T07:20:57.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.414+0000 7f4f23fff640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4f2806b500 0x7f4f2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.414+0000 7f4f23fff640 1 -- 172.21.15.26:0/3243430173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4f3408ab80 con 0x7f4f3c1eea10 2024-03-06T07:20:57.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.415+0000 7f4f3b7fe640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4f2806b500 0x7f4f2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.415+0000 7f4f3b7fe640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4f2806b500 0x7f4f2806d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c034000 tx=0x7f4f2c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.419+0000 7f4f23fff640 1 -- 172.21.15.26:0/3243430173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4f34024030 con 0x7f4f3c1eea10 2024-03-06T07:20:57.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.600+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f3c073120 con 0x7f4f3c1eea10 2024-03-06T07:20:57.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.600+0000 7f4f23fff640 1 -- 172.21.15.26:0/3243430173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f34059c50 con 0x7f4f3c1eea10 2024-03-06T07:20:57.602 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4f2806b500 msgr2=0x7f4f2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4f2806b500 0x7f4f2806d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c034000 tx=0x7f4f2c007000 comp rx=0 tx=0).stop 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c1eea10 msgr2=0x7f4f3c151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c1eea10 0x7f4f3c151140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c155e70 tx=0x7f4f34009f90 comp rx=0 tx=0).stop 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 shutdown_connections 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4f2806b500 0x7f4f2806d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c1eea10 0x7f4f3c151140 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f3c154480 0x7f4f3c1ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 --2- 172.21.15.26:0/3243430173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f24003b70 0x7f4f3c0ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 >> 172.21.15.26:0/3243430173 conn(0x7f4f3c0aa6a0 msgr2=0x7f4f3c06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 shutdown_connections 2024-03-06T07:20:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.602+0000 7f4f4224e640 1 -- 172.21.15.26:0/3243430173 wait complete. 2024-03-06T07:20:57.614 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:20:57.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.713+0000 7fe46a9a0640 1 Processor -- start 2024-03-06T07:20:57.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe46a9a0640 1 -- start start 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe46a9a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4640aa180 0x7fe46414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe46a9a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe46414f140 0x7fe4640c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe46a9a0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe464155bb0 con 0x7fe4641550c0 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe46a9a0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4640c4610 con 0x7fe4640aa180 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe46a9a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46414f5f0 con 0x7fe46414f140 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe4637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4640aa180 0x7fe46414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe4637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4640aa180 0x7fe46414ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44270/0 (socket says 172.21.15.26:44270) 2024-03-06T07:20:57.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.714+0000 7fe4637fe640 1 -- 172.21.15.26:0/3532862389 learned_addr learned my addr 172.21.15.26:0/3532862389 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.715+0000 7fe468f3d640 1 --2- 172.21.15.26:0/3532862389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe46414f140 0x7fe4640c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.715+0000 7fe4637fe640 1 -- 172.21.15.26:0/3532862389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe46414f140 msgr2=0x7fe4640c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.715+0000 7fe4637fe640 1 --2- 172.21.15.26:0/3532862389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe46414f140 0x7fe4640c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.715+0000 7fe4637fe640 1 -- 172.21.15.26:0/3532862389 >> v1:172.21.15.117:6789/0 conn(0x7fe4641550c0 legacy=0x7fe4641554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.715+0000 7fe4637fe640 1 -- 172.21.15.26:0/3532862389 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640aa660 con 0x7fe4640aa180 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.715+0000 7fe468f3d640 1 --2- 172.21.15.26:0/3532862389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe46414f140 0x7fe4640c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.715+0000 7fe4637fe640 1 --2- 172.21.15.26:0/3532862389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4640aa180 0x7fe46414ec00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe458009870 tx=0x7fe45802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04dfd1dbf14fb3e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe462ffd640 1 -- 172.21.15.26:0/3532862389 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458038070 con 0x7fe4640aa180 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe462ffd640 1 -- 172.21.15.26:0/3532862389 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458031740 con 0x7fe4640aa180 2024-03-06T07:20:57.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe462ffd640 1 -- 172.21.15.26:0/3532862389 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45802f050 con 0x7fe4640aa180 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 -- 172.21.15.26:0/3532862389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4640aa180 msgr2=0x7fe46414ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/3532862389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4640aa180 0x7fe46414ec00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe458009870 tx=0x7fe45802b7e0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 -- 172.21.15.26:0/3532862389 shutdown_connections 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/3532862389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe46414f140 0x7fe4640c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/3532862389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4640aa180 0x7fe46414ec00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 -- 172.21.15.26:0/3532862389 >> 172.21.15.26:0/3532862389 conn(0x7fe4640c6710 msgr2=0x7fe4640c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 -- 172.21.15.26:0/3532862389 shutdown_connections 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.716+0000 7fe46a9a0640 1 -- 172.21.15.26:0/3532862389 wait complete. 2024-03-06T07:20:57.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.717+0000 7fe46a9a0640 1 Processor -- start 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.717+0000 7fe46a9a0640 1 -- start start 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe46a9a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4640aa180 0x7fe464147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe46a9a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46414f140 0x7fe4640a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe46a9a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4641550c0 0x7fe4640a8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe46a9a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640c1310 con 0x7fe4641550c0 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe46a9a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640c4860 con 0x7fe46414f140 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe46a9a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640c1490 con 0x7fe4640aa180 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4641550c0 0x7fe4640a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4641550c0 0x7fe4640a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44282/0 (socket says 172.21.15.26:44282) 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 -- 172.21.15.26:0/1478839979 learned_addr learned my addr 172.21.15.26:0/1478839979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe463fff640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4640aa180 0x7fe464147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe4637fe640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46414f140 0x7fe4640a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 -- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4640aa180 msgr2=0x7fe464147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4640aa180 0x7fe464147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 -- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46414f140 msgr2=0x7fe4640a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46414f140 0x7fe4640a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.718+0000 7fe468f3d640 1 -- 172.21.15.26:0/1478839979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4641dcd90 con 0x7fe4641550c0 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.719+0000 7fe468f3d640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4641550c0 0x7fe4640a8080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00ce90 tx=0x7fe45c00e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.719+0000 7fe460ff9640 1 -- 172.21.15.26:0/1478839979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00c2e0 con 0x7fe4641550c0 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.719+0000 7fe460ff9640 1 -- 172.21.15.26:0/1478839979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c00ebd0 con 0x7fe4641550c0 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.719+0000 7fe460ff9640 1 -- 172.21.15.26:0/1478839979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00a470 con 0x7fe4641550c0 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.719+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4641ddc50 con 0x7fe4641550c0 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.719+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4641dbb30 con 0x7fe4641550c0 2024-03-06T07:20:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.720+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46404f330 con 0x7fe4641550c0 2024-03-06T07:20:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.720+0000 7fe460ff9640 1 -- 172.21.15.26:0/1478839979 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe45c00a610 con 0x7fe4641550c0 2024-03-06T07:20:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.721+0000 7fe460ff9640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe45006b670 0x7fe45006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.721+0000 7fe460ff9640 1 -- 172.21.15.26:0/1478839979 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe45c08c4f0 con 0x7fe4641550c0 2024-03-06T07:20:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.721+0000 7fe463fff640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe45006b670 0x7fe45006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.721+0000 7fe463fff640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe45006b670 0x7fe45006db30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.725+0000 7fe460ff9640 1 -- 172.21.15.26:0/1478839979 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe45c05b520 con 0x7fe4641550c0 2024-03-06T07:20:57.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.910+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe464068450 con 0x7fe4641550c0 2024-03-06T07:20:57.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.910+0000 7fe460ff9640 1 -- 172.21.15.26:0/1478839979 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe45c05ae20 con 0x7fe4641550c0 2024-03-06T07:20:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:20:57.913 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:20:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:20:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:20:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:20:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:20:57.913 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:20:57.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe45006b670 msgr2=0x7fe45006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe45006b670 0x7fe45006db30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454008040 comp rx=0 tx=0).stop 2024-03-06T07:20:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4641550c0 msgr2=0x7fe4640a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4641550c0 0x7fe4640a8080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00ce90 tx=0x7fe45c00e420 comp rx=0 tx=0).stop 2024-03-06T07:20:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 shutdown_connections 2024-03-06T07:20:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe45006b670 0x7fe45006db30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4641550c0 0x7fe4640a8080 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46414f140 0x7fe4640a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 --2- 172.21.15.26:0/1478839979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4640aa180 0x7fe464147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 >> 172.21.15.26:0/1478839979 conn(0x7fe4640c6710 msgr2=0x7fe4640c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 shutdown_connections 2024-03-06T07:20:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:57.912+0000 7fe46a9a0640 1 -- 172.21.15.26:0/1478839979 wait complete. 2024-03-06T07:20:57.926 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:20:57.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.030+0000 7fdfa9f79640 1 Processor -- start 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.030+0000 7fdfa9f79640 1 -- start start 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa9f79640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdfa406fe20 0x7fdfa4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa9f79640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfa4158780 0x7fdfa4158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa9f79640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa4156150 con 0x7fdfa40c6780 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa9f79640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa40a7d70 con 0x7fdfa4158780 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa9f79640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa40a8050 con 0x7fdfa406fe20 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfa4158780 0x7fdfa4158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfa4158780 0x7fdfa4158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44294/0 (socket says 172.21.15.26:44294) 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa3fff640 1 -- 172.21.15.26:0/3063431669 learned_addr learned my addr 172.21.15.26:0/3063431669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa3fff640 1 -- 172.21.15.26:0/3063431669 >> v2:172.21.15.117:3300/0 conn(0x7fdfa406fe20 msgr2=0x7fdfa4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa3fff640 1 --2- 172.21.15.26:0/3063431669 >> v2:172.21.15.117:3300/0 conn(0x7fdfa406fe20 0x7fdfa4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa3fff640 1 -- 172.21.15.26:0/3063431669 >> v1:172.21.15.117:6789/0 conn(0x7fdfa40c6780 legacy=0x7fdfa4072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.031+0000 7fdfa3fff640 1 -- 172.21.15.26:0/3063431669 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa40a8330 con 0x7fdfa4158780 2024-03-06T07:20:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.032+0000 7fdfa3fff640 1 --2- 172.21.15.26:0/3063431669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa4158780 0x7fdfa4158bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4071dd0 tx=0x7fdf9c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978e3154925ae6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.032+0000 7fdfa27fc640 1 -- 172.21.15.26:0/3063431669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c02c9f0 con 0x7fdfa4158780 2024-03-06T07:20:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.032+0000 7fdfa27fc640 1 -- 172.21.15.26:0/3063431669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c02cb90 con 0x7fdfa4158780 2024-03-06T07:20:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.032+0000 7fdfa27fc640 1 -- 172.21.15.26:0/3063431669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0343f0 con 0x7fdfa4158780 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 -- 172.21.15.26:0/3063431669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa4158780 msgr2=0x7fdfa4158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/3063431669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa4158780 0x7fdfa4158bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4071dd0 tx=0x7fdf9c02be50 comp rx=0 tx=0).stop 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 -- 172.21.15.26:0/3063431669 shutdown_connections 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/3063431669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa4158780 0x7fdfa4158bd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/3063431669 >> v2:172.21.15.117:3300/0 conn(0x7fdfa406fe20 0x7fdfa4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 -- 172.21.15.26:0/3063431669 >> 172.21.15.26:0/3063431669 conn(0x7fdfa40aa630 msgr2=0x7fdfa40aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 -- 172.21.15.26:0/3063431669 shutdown_connections 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.033+0000 7fdfa9f79640 1 -- 172.21.15.26:0/3063431669 wait complete. 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.034+0000 7fdfa9f79640 1 Processor -- start 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.034+0000 7fdfa9f79640 1 -- start start 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa9f79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa406fe20 0x7fdfa414ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa9f79640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfa40c6780 0x7fdfa41e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa9f79640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa4158780 0x7fdfa4157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa9f79640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4070200 con 0x7fdfa406fe20 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa9f79640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40c3600 con 0x7fdfa4158780 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa9f79640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4072c20 con 0x7fdfa40c6780 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa406fe20 0x7fdfa414ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa406fe20 0x7fdfa414ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44298/0 (socket says 172.21.15.26:44298) 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 -- 172.21.15.26:0/2169811931 learned_addr learned my addr 172.21.15.26:0/2169811931 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa3fff640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa4158780 0x7fdfa4157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 -- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfa40c6780 msgr2=0x7fdfa41e09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa2ffd640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfa40c6780 0x7fdfa41e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfa40c6780 0x7fdfa41e09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 -- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa4158780 msgr2=0x7fdfa4157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa4158780 0x7fdfa4157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 -- 172.21.15.26:0/2169811931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa414f4c0 con 0x7fdfa406fe20 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.035+0000 7fdfa37fe640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa406fe20 0x7fdfa414ef80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b4a0 tx=0x7fdf9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.036+0000 7fdf8bfff640 1 -- 172.21.15.26:0/2169811931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94014070 con 0x7fdfa406fe20 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.036+0000 7fdf8bfff640 1 -- 172.21.15.26:0/2169811931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf94010610 con 0x7fdfa406fe20 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.036+0000 7fdf8bfff640 1 -- 172.21.15.26:0/2169811931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94010a00 con 0x7fdfa406fe20 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.036+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa41ebb80 con 0x7fdfa406fe20 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.036+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa414eca0 con 0x7fdfa406fe20 2024-03-06T07:20:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.037+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa404f330 con 0x7fdfa406fe20 2024-03-06T07:20:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.037+0000 7fdf8bfff640 1 -- 172.21.15.26:0/2169811931 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf94017070 con 0x7fdfa406fe20 2024-03-06T07:20:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.037+0000 7fdf8bfff640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf9006b4d0 0x7fdf9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.038+0000 7fdf8bfff640 1 -- 172.21.15.26:0/2169811931 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdf940899d0 con 0x7fdfa406fe20 2024-03-06T07:20:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.038+0000 7fdfa2ffd640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf9006b4d0 0x7fdf9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.038+0000 7fdfa2ffd640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf9006b4d0 0x7fdf9006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40ad790 tx=0x7fdf98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.042+0000 7fdf8bfff640 1 -- 172.21.15.26:0/2169811931 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf94058aa0 con 0x7fdfa406fe20 2024-03-06T07:20:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.222+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdfa40c5160 con 0x7fdfa406fe20 2024-03-06T07:20:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.983+0000 7fdf8bfff640 1 -- 172.21.15.26:0/2169811931 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdf940583a0 con 0x7fdfa406fe20 2024-03-06T07:20:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf9006b4d0 msgr2=0x7fdf9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf9006b4d0 0x7fdf9006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40ad790 tx=0x7fdf98009040 comp rx=0 tx=0).stop 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa406fe20 msgr2=0x7fdfa414ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa406fe20 0x7fdfa414ef80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b4a0 tx=0x7fdf9400b960 comp rx=0 tx=0).stop 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 shutdown_connections 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf9006b4d0 0x7fdf9006d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa4158780 0x7fdfa4157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfa40c6780 0x7fdfa41e09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 --2- 172.21.15.26:0/2169811931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa406fe20 0x7fdfa414ef80 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b4a0 tx=0x7fdf9400b960 comp rx=0 tx=0).stop 2024-03-06T07:20:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 >> 172.21.15.26:0/2169811931 conn(0x7fdfa40aa630 msgr2=0x7fdfa40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 shutdown_connections 2024-03-06T07:20:58.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:58.986+0000 7fdfa9f79640 1 -- 172.21.15.26:0/2169811931 wait complete. 2024-03-06T07:20:59.005 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:20:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.118+0000 7f12404fb640 1 Processor -- start 2024-03-06T07:20:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.118+0000 7f12404fb640 1 -- start start 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f12404fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380c5990 0x7f12380c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f12404fb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12380ab170 0x7f123806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f12404fb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380c3ef0 con 0x7f123806c730 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f12404fb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123806cbe0 con 0x7f12380ab170 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f12404fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12381556d0 con 0x7f12380c5990 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123da96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12380ab170 0x7f123806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123ea98640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f123806c730 0x7f12381523f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57902/0 (socket says 172.21.15.26:57902) 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123ea98640 1 -- 172.21.15.26:0/2852737907 learned_addr learned my addr 172.21.15.26:0/2852737907 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123e297640 1 --2- 172.21.15.26:0/2852737907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380c5990 0x7f12380c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123da96640 1 -- 172.21.15.26:0/2852737907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380c5990 msgr2=0x7f12380c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:59.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123da96640 1 --2- 172.21.15.26:0/2852737907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380c5990 0x7f12380c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123da96640 1 -- 172.21.15.26:0/2852737907 >> v1:172.21.15.26:6789/0 conn(0x7f123806c730 legacy=0x7f12381523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123da96640 1 -- 172.21.15.26:0/2852737907 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12381559b0 con 0x7f12380ab170 2024-03-06T07:20:59.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123d295640 1 -- 172.21.15.26:0/2852737907 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019852629 0 0) 0x7f12380c3ef0 con 0x7f123806c730 2024-03-06T07:20:59.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.119+0000 7f123e297640 1 --2- 172.21.15.26:0/2852737907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380c5990 0x7f12380c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.120+0000 7f123da96640 1 --2- 172.21.15.26:0/2852737907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380ab170 0x7f123806c1f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12280097a0 tx=0x7f122802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e32936b9c0f41d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.120+0000 7f123d295640 1 -- 172.21.15.26:0/2852737907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12280315a0 con 0x7f12380ab170 2024-03-06T07:20:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.120+0000 7f123d295640 1 -- 172.21.15.26:0/2852737907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1228037070 con 0x7f12380ab170 2024-03-06T07:20:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.120+0000 7f123d295640 1 -- 172.21.15.26:0/2852737907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228031960 con 0x7f12380ab170 2024-03-06T07:20:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 -- 172.21.15.26:0/2852737907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380ab170 msgr2=0x7f123806c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 --2- 172.21.15.26:0/2852737907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380ab170 0x7f123806c1f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12280097a0 tx=0x7f122802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 -- 172.21.15.26:0/2852737907 shutdown_connections 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 --2- 172.21.15.26:0/2852737907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380ab170 0x7f123806c1f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 --2- 172.21.15.26:0/2852737907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380c5990 0x7f12380c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 -- 172.21.15.26:0/2852737907 >> 172.21.15.26:0/2852737907 conn(0x7f12380c6710 msgr2=0x7f12380c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 -- 172.21.15.26:0/2852737907 shutdown_connections 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.121+0000 7f12404fb640 1 -- 172.21.15.26:0/2852737907 wait complete. 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 Processor -- start 2024-03-06T07:20:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 -- start start 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f123806c730 0x7f12380a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380ab170 0x7f12380a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380c5990 0x7f12380a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1238156490 con 0x7f12380c5990 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1238155f70 con 0x7f123806c730 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.122+0000 7f12404fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1238156610 con 0x7f12380ab170 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380c5990 0x7f12380a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380c5990 0x7f12380a6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44312/0 (socket says 172.21.15.26:44312) 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 -- 172.21.15.26:0/2682395884 learned_addr learned my addr 172.21.15.26:0/2682395884 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123da96640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380ab170 0x7f12380a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 -- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380ab170 msgr2=0x7f12380a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380ab170 0x7f12380a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 -- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f123806c730 msgr2=0x7f12380a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123e297640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f123806c730 0x7f12380a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f123806c730 0x7f12380a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 -- 172.21.15.26:0/2682395884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238150820 con 0x7f12380c5990 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f123ea98640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380c5990 0x7f12380a6870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f123000c7a0 tx=0x7f123000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f122effd640 1 -- 172.21.15.26:0/2682395884 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230018070 con 0x7f12380c5990 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f122effd640 1 -- 172.21.15.26:0/2682395884 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123000a640 con 0x7f12380c5990 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1238150b00 con 0x7f12380c5990 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f122effd640 1 -- 172.21.15.26:0/2682395884 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000aab0 con 0x7f12380c5990 2024-03-06T07:20:59.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.123+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12380a7840 con 0x7f12380c5990 2024-03-06T07:20:59.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.124+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12380c1530 con 0x7f12380c5990 2024-03-06T07:20:59.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.125+0000 7f122effd640 1 -- 172.21.15.26:0/2682395884 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f123001b070 con 0x7f12380c5990 2024-03-06T07:20:59.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.125+0000 7f122effd640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f121c06b500 0x7f121c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.126+0000 7f122effd640 1 -- 172.21.15.26:0/2682395884 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1230089ab0 con 0x7f12380c5990 2024-03-06T07:20:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.126+0000 7f123e297640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f121c06b500 0x7f121c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.126+0000 7f123e297640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f121c06b500 0x7f121c06d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f123400b840 tx=0x7f1234008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:59.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.129+0000 7f122effd640 1 -- 172.21.15.26:0/2682395884 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f12380c1530 con 0x7f12380c5990 2024-03-06T07:20:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.309+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f123804f330 con 0x7f12380c5990 2024-03-06T07:20:59.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.990+0000 7f122effd640 1 -- 172.21.15.26:0/2682395884 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7f12300589f0 con 0x7f12380c5990 2024-03-06T07:20:59.992 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f121c06b500 msgr2=0x7f121c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f121c06b500 0x7f121c06d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f123400b840 tx=0x7f1234008040 comp rx=0 tx=0).stop 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380c5990 msgr2=0x7f12380a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380c5990 0x7f12380a6870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f123000c7a0 tx=0x7f123000cc60 comp rx=0 tx=0).stop 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 shutdown_connections 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f121c06b500 0x7f121c06d9c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12380c5990 0x7f12380a6870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12380ab170 0x7f12380a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 --2- 172.21.15.26:0/2682395884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f123806c730 0x7f12380a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 >> 172.21.15.26:0/2682395884 conn(0x7f12380c6710 msgr2=0x7f12381457d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 shutdown_connections 2024-03-06T07:20:59.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:20:59.993+0000 7f12404fb640 1 -- 172.21.15.26:0/2682395884 wait complete. 2024-03-06T07:21:00.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:21:00.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.126+0000 7f6f56ad4640 1 Processor -- start 2024-03-06T07:21:00.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.126+0000 7f6f56ad4640 1 -- start start 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f56ad4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f50155cf0 0x7f6f500aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f56ad4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f500c1f90 con 0x7f6f500a8ff0 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f56ad4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f500c11d0 con 0x7f6f500a6290 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f56ad4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5014f850 con 0x7f6f50155cf0 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f55071640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f50155cf0 0x7f6f500aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f54870640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f500a8ff0 0x7f6f500ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57908/0 (socket says 172.21.15.26:57908) 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f54870640 1 -- 172.21.15.26:0/569427198 learned_addr learned my addr 172.21.15.26:0/569427198 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f55071640 1 -- 172.21.15.26:0/569427198 >> v1:172.21.15.117:6789/0 conn(0x7f6f500a6290 legacy=0x7f6f500a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f55071640 1 -- 172.21.15.26:0/569427198 >> v1:172.21.15.26:6789/0 conn(0x7f6f500a8ff0 legacy=0x7f6f500ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f55071640 1 -- 172.21.15.26:0/569427198 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f500c14b0 con 0x7f6f50155cf0 2024-03-06T07:21:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.127+0000 7f6f4f7fe640 1 -- 172.21.15.26:0/569427198 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695033314 0 0) 0x7f6f500c1f90 con 0x7f6f500a8ff0 2024-03-06T07:21:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.128+0000 7f6f55071640 1 --2- 172.21.15.26:0/569427198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f500aa530 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009d90 tx=0x7f6f4802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5857f85c916b49 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.128+0000 7f6f4f7fe640 1 -- 172.21.15.26:0/569427198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48039070 con 0x7f6f50155cf0 2024-03-06T07:21:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.128+0000 7f6f4f7fe640 1 -- 172.21.15.26:0/569427198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4802cb40 con 0x7f6f50155cf0 2024-03-06T07:21:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.128+0000 7f6f4f7fe640 1 -- 172.21.15.26:0/569427198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48034440 con 0x7f6f50155cf0 2024-03-06T07:21:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.128+0000 7f6f56ad4640 1 -- 172.21.15.26:0/569427198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 msgr2=0x7f6f500aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.129+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/569427198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f500aa530 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009d90 tx=0x7f6f4802bd90 comp rx=0 tx=0).stop 2024-03-06T07:21:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.129+0000 7f6f56ad4640 1 -- 172.21.15.26:0/569427198 shutdown_connections 2024-03-06T07:21:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.129+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/569427198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f500aa530 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.129+0000 7f6f56ad4640 1 -- 172.21.15.26:0/569427198 >> 172.21.15.26:0/569427198 conn(0x7f6f5006afc0 msgr2=0x7f6f5006b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.129+0000 7f6f56ad4640 1 -- 172.21.15.26:0/569427198 shutdown_connections 2024-03-06T07:21:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.129+0000 7f6f56ad4640 1 -- 172.21.15.26:0/569427198 wait complete. 2024-03-06T07:21:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.130+0000 7f6f56ad4640 1 Processor -- start 2024-03-06T07:21:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.130+0000 7f6f56ad4640 1 -- start start 2024-03-06T07:21:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.130+0000 7f6f56ad4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f500a8ff0 0x7f6f50144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.130+0000 7f6f56ad4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f5014e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.130+0000 7f6f56ad4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f500c6040 0x7f6f5014ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f56ad4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500c1a70 con 0x7f6f50155cf0 2024-03-06T07:21:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f56ad4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50156260 con 0x7f6f500a8ff0 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f56ad4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500ab900 con 0x7f6f500c6040 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f5014e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f5014e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44332/0 (socket says 172.21.15.26:44332) 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 -- 172.21.15.26:0/1120935153 learned_addr learned my addr 172.21.15.26:0/1120935153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 -- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f500c6040 msgr2=0x7f6f5014ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f500c6040 0x7f6f5014ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f54870640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f500a8ff0 0x7f6f50144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 -- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f500a8ff0 msgr2=0x7f6f50144da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f500a8ff0 0x7f6f50144da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 -- 172.21.15.26:0/1120935153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50071940 con 0x7f6f50155cf0 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4ffff640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f5014e8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b420 tx=0x7f6f3c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4d7fa640 1 -- 172.21.15.26:0/1120935153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c011830 con 0x7f6f50155cf0 2024-03-06T07:21:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4d7fa640 1 -- 172.21.15.26:0/1120935153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c016070 con 0x7f6f50155cf0 2024-03-06T07:21:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f500c5270 con 0x7f6f50155cf0 2024-03-06T07:21:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f4d7fa640 1 -- 172.21.15.26:0/1120935153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c011c10 con 0x7f6f50155cf0 2024-03-06T07:21:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.131+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f50153560 con 0x7f6f50155cf0 2024-03-06T07:21:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.132+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5004f330 con 0x7f6f50155cf0 2024-03-06T07:21:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.133+0000 7f6f4d7fa640 1 -- 172.21.15.26:0/1120935153 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f3c01b460 con 0x7f6f50155cf0 2024-03-06T07:21:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.133+0000 7f6f4d7fa640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f3406b4d0 0x7f6f3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.134+0000 7f6f4d7fa640 1 -- 172.21.15.26:0/1120935153 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f6f3c089e80 con 0x7f6f50155cf0 2024-03-06T07:21:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.134+0000 7f6f54870640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f3406b4d0 0x7f6f3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.134+0000 7f6f54870640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f3406b4d0 0x7f6f3406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f44007fd0 tx=0x7f6f44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.138+0000 7f6f4d7fa640 1 -- 172.21.15.26:0/1120935153 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f3c020020 con 0x7f6f50155cf0 2024-03-06T07:21:00.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.358+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6f500c24e0 con 0x7f6f50155cf0 2024-03-06T07:21:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.992+0000 7f6f4d7fa640 1 -- 172.21.15.26:0/1120935153 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6f3c058b30 con 0x7f6f50155cf0 2024-03-06T07:21:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 5 and data pool 6 2024-03-06T07:21:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f3406b4d0 msgr2=0x7f6f3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f3406b4d0 0x7f6f3406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f44007fd0 tx=0x7f6f44009040 comp rx=0 tx=0).stop 2024-03-06T07:21:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 msgr2=0x7f6f5014e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f5014e8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b420 tx=0x7f6f3c00b8e0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 shutdown_connections 2024-03-06T07:21:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f3406b4d0 0x7f6f3406d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f500c6040 0x7f6f5014ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f50155cf0 0x7f6f5014e8c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 --2- 172.21.15.26:0/1120935153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f500a8ff0 0x7f6f50144da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.995+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 >> 172.21.15.26:0/1120935153 conn(0x7f6f5006afc0 msgr2=0x7f6f500c3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.996+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 shutdown_connections 2024-03-06T07:21:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:00.996+0000 7f6f56ad4640 1 -- 172.21.15.26:0/1120935153 wait complete. 2024-03-06T07:21:01.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:01.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.130+0000 7f7362caf640 1 Processor -- start 2024-03-06T07:21:01.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.130+0000 7f7362caf640 1 -- start start 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f7362caf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c5330 0x7f735c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f7362caf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f735c158db0 0x7f735c15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f7362caf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c155110 con 0x7f735c0c88b0 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f7362caf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c1559e0 con 0x7f735c158db0 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f7362caf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0c8f40 con 0x7f735c0c5330 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f736124c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f735c158db0 0x7f735c15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f736124c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f735c158db0 0x7f735c15fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44340/0 (socket says 172.21.15.26:44340) 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f736124c640 1 -- 172.21.15.26:0/1707015847 learned_addr learned my addr 172.21.15.26:0/1707015847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f736124c640 1 -- 172.21.15.26:0/1707015847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c5330 msgr2=0x7f735c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.131+0000 7f736124c640 1 --2- 172.21.15.26:0/1707015847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c5330 0x7f735c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.132+0000 7f736124c640 1 -- 172.21.15.26:0/1707015847 >> v1:172.21.15.117:6789/0 conn(0x7f735c0c88b0 legacy=0x7f735c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.132+0000 7f736124c640 1 -- 172.21.15.26:0/1707015847 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c155390 con 0x7f735c158db0 2024-03-06T07:21:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.132+0000 7f736124c640 1 --2- 172.21.15.26:0/1707015847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c158db0 0x7f735c15fa30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7344009870 tx=0x7f734402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e618228f5cace87d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.132+0000 7f735b7fe640 1 -- 172.21.15.26:0/1707015847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73440317c0 con 0x7f735c158db0 2024-03-06T07:21:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.132+0000 7f735b7fe640 1 -- 172.21.15.26:0/1707015847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344037070 con 0x7f735c158db0 2024-03-06T07:21:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.132+0000 7f735b7fe640 1 -- 172.21.15.26:0/1707015847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344031b80 con 0x7f735c158db0 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 -- 172.21.15.26:0/1707015847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c158db0 msgr2=0x7f735c15fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 --2- 172.21.15.26:0/1707015847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c158db0 0x7f735c15fa30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7344009870 tx=0x7f734402b860 comp rx=0 tx=0).stop 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 -- 172.21.15.26:0/1707015847 shutdown_connections 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 --2- 172.21.15.26:0/1707015847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c158db0 0x7f735c15fa30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 --2- 172.21.15.26:0/1707015847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c5330 0x7f735c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 -- 172.21.15.26:0/1707015847 >> 172.21.15.26:0/1707015847 conn(0x7f735c0ac560 msgr2=0x7f735c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 -- 172.21.15.26:0/1707015847 shutdown_connections 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.133+0000 7f7362caf640 1 -- 172.21.15.26:0/1707015847 wait complete. 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.134+0000 7f7362caf640 1 Processor -- start 2024-03-06T07:21:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.134+0000 7f7362caf640 1 -- start start 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7362caf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c0c5330 0x7f735c14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7362caf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c88b0 0x7f735c156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7362caf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735c14c3a0 0x7f735c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7362caf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c1529d0 con 0x7f735c0c5330 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7362caf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c06d090 con 0x7f735c14c3a0 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7362caf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c152b50 con 0x7f735c0c88b0 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7360a4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c0c5330 0x7f735c14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7360a4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c0c5330 0x7f735c14e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44342/0 (socket says 172.21.15.26:44342) 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7360a4b640 1 -- 172.21.15.26:0/3063777164 learned_addr learned my addr 172.21.15.26:0/3063777164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f735bfff640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c88b0 0x7f735c156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f736124c640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735c14c3a0 0x7f735c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7360a4b640 1 -- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c88b0 msgr2=0x7f735c156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7360a4b640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c88b0 0x7f735c156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7360a4b640 1 -- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735c14c3a0 msgr2=0x7f735c156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.135+0000 7f7360a4b640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735c14c3a0 0x7f735c156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f7360a4b640 1 -- 172.21.15.26:0/3063777164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c157b80 con 0x7f735c0c5330 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f736124c640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735c14c3a0 0x7f735c156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f7360a4b640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c0c5330 0x7f735c14e030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f735400ba50 tx=0x7f735400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f73597fa640 1 -- 172.21.15.26:0/3063777164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354019070 con 0x7f735c0c5330 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f73597fa640 1 -- 172.21.15.26:0/3063777164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735400cc00 con 0x7f735c0c5330 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c1eba20 con 0x7f735c0c5330 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f73597fa640 1 -- 172.21.15.26:0/3063777164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354014440 con 0x7f735c0c5330 2024-03-06T07:21:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.136+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c158120 con 0x7f735c0c5330 2024-03-06T07:21:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.137+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c14e5d0 con 0x7f735c0c5330 2024-03-06T07:21:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.137+0000 7f73597fa640 1 -- 172.21.15.26:0/3063777164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f735401c070 con 0x7f735c0c5330 2024-03-06T07:21:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.138+0000 7f73597fa640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.138+0000 7f73597fa640 1 -- 172.21.15.26:0/3063777164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f735408a030 con 0x7f735c0c5330 2024-03-06T07:21:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.138+0000 7f735bfff640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.139+0000 7f735bfff640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f733c06b4d0 0x7f733c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f735c14f3b0 tx=0x7f734c007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.142+0000 7f73597fa640 1 -- 172.21.15.26:0/3063777164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7354058760 con 0x7f735c0c5330 2024-03-06T07:21:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.325+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f735c04f330 con 0x7f735c0c5330 2024-03-06T07:21:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.326+0000 7f73597fa640 1 -- 172.21.15.26:0/3063777164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16967 (secure 0 0 0) 0x7f735405c800 con 0x7f735c0c5330 2024-03-06T07:21:01.333 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:01.333 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:00.359630+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:58.241645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:59.328635+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f733c06b4d0 msgr2=0x7f733c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f733c06b4d0 0x7f733c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f735c14f3b0 tx=0x7f734c007e30 comp rx=0 tx=0).stop 2024-03-06T07:21:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c0c5330 msgr2=0x7f735c14e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c0c5330 0x7f735c14e030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f735400ba50 tx=0x7f735400bf10 comp rx=0 tx=0).stop 2024-03-06T07:21:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 shutdown_connections 2024-03-06T07:21:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735c14c3a0 0x7f735c156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f735c0c88b0 0x7f735c156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 --2- 172.21.15.26:0/3063777164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c0c5330 0x7f735c14e030 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 >> 172.21.15.26:0/3063777164 conn(0x7f735c0ac560 msgr2=0x7f735c0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 shutdown_connections 2024-03-06T07:21:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.328+0000 7f7362caf640 1 -- 172.21.15.26:0/3063777164 wait complete. 2024-03-06T07:21:01.340 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:01.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.440+0000 7fb97a53e640 1 Processor -- start 2024-03-06T07:21:01.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.440+0000 7fb97a53e640 1 -- start start 2024-03-06T07:21:01.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb97a53e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb974150710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb97a53e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9740c3d60 0x7fb9740c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb97a53e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb974156f90 con 0x7fb974072720 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb97a53e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9740c38d0 con 0x7fb9740c3d60 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb97a53e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb974157270 con 0x7fb974150290 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb9737fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb974150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb978adb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9740c3d60 0x7fb9740c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb9737fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb974150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:55706/0 (socket says 172.21.15.26:55706) 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb978adb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9740c3d60 0x7fb9740c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51980/0 (socket says 172.21.15.26:51980) 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb973fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb974072720 0x7fb9741564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38794/0 (socket says 172.21.15.26:38794) 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.441+0000 7fb9737fe640 1 -- 172.21.15.26:0/800161498 learned_addr learned my addr 172.21.15.26:0/800161498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.442+0000 7fb9737fe640 1 -- 172.21.15.26:0/800161498 >> v2:172.21.15.117:3300/0 conn(0x7fb9740c3d60 msgr2=0x7fb9740c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.442+0000 7fb9737fe640 1 --2- 172.21.15.26:0/800161498 >> v2:172.21.15.117:3300/0 conn(0x7fb9740c3d60 0x7fb9740c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.442+0000 7fb9737fe640 1 -- 172.21.15.26:0/800161498 >> v1:172.21.15.117:6789/0 conn(0x7fb974072720 legacy=0x7fb9741564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.442+0000 7fb9737fe640 1 -- 172.21.15.26:0/800161498 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740c3280 con 0x7fb974150290 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb9737fe640 1 --2- 172.21.15.26:0/800161498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb974150710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9640098a0 tx=0x7fb96402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409e0ee68f518b02 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb972ffd640 1 -- 172.21.15.26:0/800161498 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964035020 con 0x7fb974150290 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb972ffd640 1 -- 172.21.15.26:0/800161498 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb964031640 con 0x7fb974150290 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb972ffd640 1 -- 172.21.15.26:0/800161498 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96402d040 con 0x7fb974150290 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 -- 172.21.15.26:0/800161498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 msgr2=0x7fb974150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 --2- 172.21.15.26:0/800161498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb974150710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9640098a0 tx=0x7fb96402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 -- 172.21.15.26:0/800161498 shutdown_connections 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 --2- 172.21.15.26:0/800161498 >> v2:172.21.15.117:3300/0 conn(0x7fb9740c3d60 0x7fb9740c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 --2- 172.21.15.26:0/800161498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb974150710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 -- 172.21.15.26:0/800161498 >> 172.21.15.26:0/800161498 conn(0x7fb9740aa630 msgr2=0x7fb9740aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 -- 172.21.15.26:0/800161498 shutdown_connections 2024-03-06T07:21:01.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.443+0000 7fb97a53e640 1 -- 172.21.15.26:0/800161498 wait complete. 2024-03-06T07:21:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.444+0000 7fb97a53e640 1 Processor -- start 2024-03-06T07:21:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb97a53e640 1 -- start start 2024-03-06T07:21:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb97a53e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb974072720 0x7fb9740c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb97a53e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9740c3d60 0x7fb9741542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb97a53e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb9740c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb97a53e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974156bd0 con 0x7fb974072720 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb97a53e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97406f8f0 con 0x7fb9740c3d60 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb97a53e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb974156d50 con 0x7fb974150290 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb973fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb974072720 0x7fb9740c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb973fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb974072720 0x7fb9740c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44356/0 (socket says 172.21.15.26:44356) 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.445+0000 7fb973fff640 1 -- 172.21.15.26:0/4280235110 learned_addr learned my addr 172.21.15.26:0/4280235110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb9737fe640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9740c3d60 0x7fb9741542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb978adb640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb9740c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb973fff640 1 -- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 msgr2=0x7fb9740c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb973fff640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb9740c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb973fff640 1 -- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9740c3d60 msgr2=0x7fb9741542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb973fff640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9740c3d60 0x7fb9741542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb973fff640 1 -- 172.21.15.26:0/4280235110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb974146160 con 0x7fb974072720 2024-03-06T07:21:01.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb973fff640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb974072720 0x7fb9740c1f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb96800c430 tx=0x7fb96800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb970ff9640 1 -- 172.21.15.26:0/4280235110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9680074e0 con 0x7fb974072720 2024-03-06T07:21:01.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb970ff9640 1 -- 172.21.15.26:0/4280235110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb968007680 con 0x7fb974072720 2024-03-06T07:21:01.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb970ff9640 1 -- 172.21.15.26:0/4280235110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968004a00 con 0x7fb974072720 2024-03-06T07:21:01.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9741eab90 con 0x7fb974072720 2024-03-06T07:21:01.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.446+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9741eb150 con 0x7fb974072720 2024-03-06T07:21:01.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.447+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9741ea980 con 0x7fb974072720 2024-03-06T07:21:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.448+0000 7fb970ff9640 1 -- 172.21.15.26:0/4280235110 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb96801a020 con 0x7fb974072720 2024-03-06T07:21:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.448+0000 7fb970ff9640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.448+0000 7fb970ff9640 1 -- 172.21.15.26:0/4280235110 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb968089b10 con 0x7fb974072720 2024-03-06T07:21:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.448+0000 7fb9737fe640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.449+0000 7fb9737fe640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb96006b4d0 0x7fb96006d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb964036000 tx=0x7fb96403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.452+0000 7fb970ff9640 1 -- 172.21.15.26:0/4280235110 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb968058890 con 0x7fb974072720 2024-03-06T07:21:01.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.634+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb974070440 con 0x7fb974072720 2024-03-06T07:21:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.635+0000 7fb970ff9640 1 -- 172.21.15.26:0/4280235110 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16967 (secure 0 0 0) 0x7fb968058190 con 0x7fb974072720 2024-03-06T07:21:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:01.637 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:00.359630+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:58.241645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:59.328635+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb96006b4d0 msgr2=0x7fb96006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb96006b4d0 0x7fb96006d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb964036000 tx=0x7fb96403c040 comp rx=0 tx=0).stop 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb974072720 msgr2=0x7fb9740c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb974072720 0x7fb9740c1f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb96800c430 tx=0x7fb96800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 shutdown_connections 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb974150290 0x7fb9740c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9740c3d60 0x7fb9741542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 --2- 172.21.15.26:0/4280235110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb974072720 0x7fb9740c1f20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 >> 172.21.15.26:0/4280235110 conn(0x7fb9740aa630 msgr2=0x7fb9740a9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 shutdown_connections 2024-03-06T07:21:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.637+0000 7fb97a53e640 1 -- 172.21.15.26:0/4280235110 wait complete. 2024-03-06T07:21:01.650 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:21:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.752+0000 7ffba455c640 1 Processor -- start 2024-03-06T07:21:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.752+0000 7ffba455c640 1 -- start start 2024-03-06T07:21:01.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba455c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0a8290 0x7ffb9c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba455c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb9c0c4740 0x7ffb9c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba455c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9c0c62a0 con 0x7ffb9c0c69a0 2024-03-06T07:21:01.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba455c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c153f00 con 0x7ffb9c0c4740 2024-03-06T07:21:01.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba455c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0706e0 con 0x7ffb9c0a8290 2024-03-06T07:21:01.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba1af7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb9c0c4740 0x7ffb9c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba1af7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb9c0c4740 0x7ffb9c0701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44366/0 (socket says 172.21.15.26:44366) 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba1af7640 1 -- 172.21.15.26:0/522338493 learned_addr learned my addr 172.21.15.26:0/522338493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba1af7640 1 -- 172.21.15.26:0/522338493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0a8290 msgr2=0x7ffb9c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.753+0000 7ffba1af7640 1 --2- 172.21.15.26:0/522338493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0a8290 0x7ffb9c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.754+0000 7ffba1af7640 1 -- 172.21.15.26:0/522338493 >> v1:172.21.15.117:6789/0 conn(0x7ffb9c0c69a0 legacy=0x7ffb9c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.754+0000 7ffba1af7640 1 -- 172.21.15.26:0/522338493 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c0c4c00 con 0x7ffb9c0c4740 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.754+0000 7ffba1af7640 1 --2- 172.21.15.26:0/522338493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0c4740 0x7ffb9c0701a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009870 tx=0x7ffb9002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ad935f3e97dab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.754+0000 7ffba12f6640 1 -- 172.21.15.26:0/522338493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90038070 con 0x7ffb9c0c4740 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.754+0000 7ffba12f6640 1 -- 172.21.15.26:0/522338493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb90031740 con 0x7ffb9c0c4740 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.754+0000 7ffba12f6640 1 -- 172.21.15.26:0/522338493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9002f050 con 0x7ffb9c0c4740 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 -- 172.21.15.26:0/522338493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0c4740 msgr2=0x7ffb9c0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 --2- 172.21.15.26:0/522338493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0c4740 0x7ffb9c0701a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009870 tx=0x7ffb9002b7e0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 -- 172.21.15.26:0/522338493 shutdown_connections 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 --2- 172.21.15.26:0/522338493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0c4740 0x7ffb9c0701a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 --2- 172.21.15.26:0/522338493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0a8290 0x7ffb9c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 -- 172.21.15.26:0/522338493 >> 172.21.15.26:0/522338493 conn(0x7ffb9c0c5560 msgr2=0x7ffb9c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 -- 172.21.15.26:0/522338493 shutdown_connections 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.755+0000 7ffba455c640 1 -- 172.21.15.26:0/522338493 wait complete. 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.756+0000 7ffba455c640 1 Processor -- start 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.756+0000 7ffba455c640 1 -- start start 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba455c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0a8290 0x7ffb9c1ec470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba455c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0c4740 0x7ffb9c1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba455c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c0c69a0 0x7ffb9c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba455c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c06fc80 con 0x7ffb9c0a8290 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba455c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c06f9a0 con 0x7ffb9c0c69a0 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba455c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c072d90 con 0x7ffb9c0c4740 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0a8290 0x7ffb9c1ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0a8290 0x7ffb9c1ec470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44370/0 (socket says 172.21.15.26:44370) 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 -- 172.21.15.26:0/1165069554 learned_addr learned my addr 172.21.15.26:0/1165069554 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba2af9640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c0c69a0 0x7ffb9c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba1af7640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0c4740 0x7ffb9c1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 -- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0c4740 msgr2=0x7ffb9c1560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0c4740 0x7ffb9c1560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 -- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c0c69a0 msgr2=0x7ffb9c151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c0c69a0 0x7ffb9c151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 -- 172.21.15.26:0/1165069554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c156620 con 0x7ffb9c0a8290 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba2af9640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c0c69a0 0x7ffb9c151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba1af7640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0c4740 0x7ffb9c1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.757+0000 7ffba22f8640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0a8290 0x7ffb9c1ec470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400b4a0 tx=0x7ffb9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.758+0000 7ffb9affd640 1 -- 172.21.15.26:0/1165069554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94014070 con 0x7ffb9c0a8290 2024-03-06T07:21:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.758+0000 7ffb9affd640 1 -- 172.21.15.26:0/1165069554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94010610 con 0x7ffb9c0a8290 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.758+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c155560 con 0x7ffb9c0a8290 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.758+0000 7ffb9affd640 1 -- 172.21.15.26:0/1165069554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94010a00 con 0x7ffb9c0a8290 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.758+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c155b20 con 0x7ffb9c0a8290 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.759+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c04f330 con 0x7ffb9c0a8290 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.759+0000 7ffb9affd640 1 -- 172.21.15.26:0/1165069554 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb94017070 con 0x7ffb9c0a8290 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.759+0000 7ffb9affd640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb8806b500 0x7ffb8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.760+0000 7ffb9affd640 1 -- 172.21.15.26:0/1165069554 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb94089ce0 con 0x7ffb9c0a8290 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.760+0000 7ffba1af7640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb8806b500 0x7ffb8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.760+0000 7ffba1af7640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb8806b500 0x7ffb8806d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb90035000 tx=0x7ffb90008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.764+0000 7ffb9affd640 1 -- 172.21.15.26:0/1165069554 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffb94058a60 con 0x7ffb9c0a8290 2024-03-06T07:21:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:01.983+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffb9c0c8480 con 0x7ffb9c0a8290 2024-03-06T07:21:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.007+0000 7ffb9affd640 1 -- 172.21.15.26:0/1165069554 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffb94058360 con 0x7ffb9c0a8290 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb8806b500 msgr2=0x7ffb8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb8806b500 0x7ffb8806d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb90035000 tx=0x7ffb90008030 comp rx=0 tx=0).stop 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0a8290 msgr2=0x7ffb9c1ec470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0a8290 0x7ffb9c1ec470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400b4a0 tx=0x7ffb9400b960 comp rx=0 tx=0).stop 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 shutdown_connections 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb8806b500 0x7ffb8806d9c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c0c69a0 0x7ffb9c151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb9c0c4740 0x7ffb9c1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 --2- 172.21.15.26:0/1165069554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb9c0a8290 0x7ffb9c1ec470 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 >> 172.21.15.26:0/1165069554 conn(0x7ffb9c0c5560 msgr2=0x7ffb9c1dc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 shutdown_connections 2024-03-06T07:21:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.010+0000 7ffba455c640 1 -- 172.21.15.26:0/1165069554 wait complete. 2024-03-06T07:21:02.023 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:21:02.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:21:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 Processor -- start 2024-03-06T07:21:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 -- start start 2024-03-06T07:21:02.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83f00c5880 0x7f83f00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83f00c5c60 0x7f83f00c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f006f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f00723b0 con 0x7f83f00c5c60 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f00728e0 con 0x7f83f00c5880 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.126+0000 7f83f6d32640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f0072600 con 0x7f83f00c7720 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83f00c5c60 0x7f83f00c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83f52cf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f006f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83f00c5c60 0x7f83f00c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44380/0 (socket says 172.21.15.26:44380) 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 -- 172.21.15.26:0/954531626 learned_addr learned my addr 172.21.15.26:0/954531626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83f4ace640 1 --2- 172.21.15.26:0/954531626 >> v2:172.21.15.117:3300/0 conn(0x7f83f00c5880 0x7f83f00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 -- 172.21.15.26:0/954531626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 msgr2=0x7f83f006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 --2- 172.21.15.26:0/954531626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f006f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 -- 172.21.15.26:0/954531626 >> v2:172.21.15.117:3300/0 conn(0x7f83f00c5880 msgr2=0x7f83f00c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 --2- 172.21.15.26:0/954531626 >> v2:172.21.15.117:3300/0 conn(0x7f83f00c5880 0x7f83f00c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83effff640 1 -- 172.21.15.26:0/954531626 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f0153af0 con 0x7f83f00c5c60 2024-03-06T07:21:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.127+0000 7f83f52cf640 1 --2- 172.21.15.26:0/954531626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f006f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83effff640 1 --2- 172.21.15.26:0/954531626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5c60 0x7f83f00c71e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f83f0070f80 tx=0x7f83e402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97439bfed260fb6a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83ef7fe640 1 -- 172.21.15.26:0/954531626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4038070 con 0x7f83f00c5c60 2024-03-06T07:21:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83ef7fe640 1 -- 172.21.15.26:0/954531626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e402f050 con 0x7f83f00c5c60 2024-03-06T07:21:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83ef7fe640 1 -- 172.21.15.26:0/954531626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4031970 con 0x7f83f00c5c60 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 -- 172.21.15.26:0/954531626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5c60 msgr2=0x7f83f00c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 --2- 172.21.15.26:0/954531626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5c60 0x7f83f00c71e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f83f0070f80 tx=0x7f83e402b7f0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 -- 172.21.15.26:0/954531626 shutdown_connections 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 --2- 172.21.15.26:0/954531626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f006f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 --2- 172.21.15.26:0/954531626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5c60 0x7f83f00c71e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 --2- 172.21.15.26:0/954531626 >> v2:172.21.15.117:3300/0 conn(0x7f83f00c5880 0x7f83f00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 -- 172.21.15.26:0/954531626 >> 172.21.15.26:0/954531626 conn(0x7f83f00aa630 msgr2=0x7f83f00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 -- 172.21.15.26:0/954531626 shutdown_connections 2024-03-06T07:21:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.128+0000 7f83f6d32640 1 -- 172.21.15.26:0/954531626 wait complete. 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.129+0000 7f83f6d32640 1 Processor -- start 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f6d32640 1 -- start start 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f6d32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5880 0x7f83f00c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f6d32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83f00c5c60 0x7f83f0148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f6d32640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f0149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f6d32640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f0073190 con 0x7f83f00c5880 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f6d32640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00c6780 con 0x7f83f00c5c60 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f6d32640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f0073310 con 0x7f83f00c7720 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f4ace640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5880 0x7f83f00c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f4ace640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5880 0x7f83f00c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44394/0 (socket says 172.21.15.26:44394) 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.130+0000 7f83f4ace640 1 -- 172.21.15.26:0/3263835204 learned_addr learned my addr 172.21.15.26:0/3263835204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83effff640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83f00c5c60 0x7f83f0148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f52cf640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f0149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f4ace640 1 -- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 msgr2=0x7f83f0149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f4ace640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f0149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f4ace640 1 -- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83f00c5c60 msgr2=0x7f83f0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f4ace640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83f00c5c60 0x7f83f0148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f4ace640 1 -- 172.21.15.26:0/3263835204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00ae3b0 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f4ace640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5880 0x7f83f00c1f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f83e800efc0 tx=0x7f83e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83ed7fa640 1 -- 172.21.15.26:0/3263835204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8018020 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83ed7fa640 1 -- 172.21.15.26:0/3263835204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8014480 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f006e010 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83ed7fa640 1 -- 172.21.15.26:0/3263835204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8014850 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.131+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f01ee990 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.132+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f00ae1b0 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.133+0000 7f83ed7fa640 1 -- 172.21.15.26:0/3263835204 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83e8005d10 con 0x7f83f00c5880 2024-03-06T07:21:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.133+0000 7f83ed7fa640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f83dc06b4d0 0x7f83dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.133+0000 7f83ed7fa640 1 -- 172.21.15.26:0/3263835204 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e808af90 con 0x7f83f00c5880 2024-03-06T07:21:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.133+0000 7f83effff640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f83dc06b4d0 0x7f83dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.134+0000 7f83effff640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f83dc06b4d0 0x7f83dc06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f83e4035000 tx=0x7f83e4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.137+0000 7f83ed7fa640 1 -- 172.21.15.26:0/3263835204 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83e8022030 con 0x7f83f00c5880 2024-03-06T07:21:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:02.357+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f83f004f330 con 0x7f83f00c5880 2024-03-06T07:21:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.012+0000 7f83ed7fa640 1 -- 172.21.15.26:0/3263835204 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83e8010040 con 0x7f83f00c5880 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.014+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f83dc06b4d0 msgr2=0x7f83dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.014+0000 7f83f6d32640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f83dc06b4d0 0x7f83dc06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f83e4035000 tx=0x7f83e4008030 comp rx=0 tx=0).stop 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.014+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5880 msgr2=0x7f83f00c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.014+0000 7f83f6d32640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5880 0x7f83f00c1f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f83e800efc0 tx=0x7f83e800c490 comp rx=0 tx=0).stop 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 shutdown_connections 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f83dc06b4d0 0x7f83dc06d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83f00c7720 0x7f83f0149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83f00c5c60 0x7f83f0148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 --2- 172.21.15.26:0/3263835204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83f00c5880 0x7f83f00c1f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 >> 172.21.15.26:0/3263835204 conn(0x7f83f00aa630 msgr2=0x7f83f0073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 shutdown_connections 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.015+0000 7f83f6d32640 1 -- 172.21.15.26:0/3263835204 wait complete. 2024-03-06T07:21:03.029 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:21:03.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:03.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.134+0000 7f61dc5f6640 1 Processor -- start 2024-03-06T07:21:03.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.134+0000 7f61dc5f6640 1 -- start start 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dc5f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dc5f6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61d4159460 0x7f61d40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dc5f6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d40c7760 con 0x7f61d4152920 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dc5f6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d41587b0 con 0x7f61d4159460 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dc5f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40c7a40 con 0x7f61d40c8a00 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dab93640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61d4159460 0x7f61d40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61da392640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61d4152920 0x7f61d4155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57924/0 (socket says 172.21.15.26:57924) 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61da392640 1 -- 172.21.15.26:0/2287566262 learned_addr learned my addr 172.21.15.26:0/2287566262 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61d9b91640 1 --2- 172.21.15.26:0/2287566262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dab93640 1 -- 172.21.15.26:0/2287566262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 msgr2=0x7f61d4158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:03.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dab93640 1 --2- 172.21.15.26:0/2287566262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dab93640 1 -- 172.21.15.26:0/2287566262 >> v1:172.21.15.26:6789/0 conn(0x7f61d4152920 legacy=0x7f61d4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61dab93640 1 -- 172.21.15.26:0/2287566262 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40c7d20 con 0x7f61d4159460 2024-03-06T07:21:03.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.135+0000 7f61d9390640 1 -- 172.21.15.26:0/2287566262 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593170399 0 0) 0x7f61d40c7760 con 0x7f61d4152920 2024-03-06T07:21:03.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.136+0000 7f61dab93640 1 --2- 172.21.15.26:0/2287566262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4159460 0x7f61d40c5330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009d90 tx=0x7f61c402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac5afab060d0b62 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.136+0000 7f61d9390640 1 -- 172.21.15.26:0/2287566262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4039070 con 0x7f61d4159460 2024-03-06T07:21:03.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.136+0000 7f61d9390640 1 -- 172.21.15.26:0/2287566262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c402cbc0 con 0x7f61d4159460 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.136+0000 7f61d9390640 1 -- 172.21.15.26:0/2287566262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4034440 con 0x7f61d4159460 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 -- 172.21.15.26:0/2287566262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4159460 msgr2=0x7f61d40c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/2287566262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4159460 0x7f61d40c5330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009d90 tx=0x7f61c402be10 comp rx=0 tx=0).stop 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 -- 172.21.15.26:0/2287566262 shutdown_connections 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/2287566262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4159460 0x7f61d40c5330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/2287566262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 -- 172.21.15.26:0/2287566262 >> 172.21.15.26:0/2287566262 conn(0x7f61d40ac560 msgr2=0x7f61d40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 -- 172.21.15.26:0/2287566262 shutdown_connections 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.137+0000 7f61dc5f6640 1 -- 172.21.15.26:0/2287566262 wait complete. 2024-03-06T07:21:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.138+0000 7f61dc5f6640 1 Processor -- start 2024-03-06T07:21:03.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.138+0000 7f61dc5f6640 1 -- start start 2024-03-06T07:21:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dc5f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dc5f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4152920 0x7f61d40c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dc5f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4158280 0x7f61d40c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dc5f6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40c7390 con 0x7f61d4158280 2024-03-06T07:21:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dc5f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406d030 con 0x7f61d4152920 2024-03-06T07:21:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dc5f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40c7510 con 0x7f61d40c8a00 2024-03-06T07:21:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4158280 0x7f61d40c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4158280 0x7f61d40c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44414/0 (socket says 172.21.15.26:44414) 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 -- 172.21.15.26:0/1852254658 learned_addr learned my addr 172.21.15.26:0/1852254658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61da392640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61d9b91640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4152920 0x7f61d40c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 -- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 msgr2=0x7f61d4156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 -- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4152920 msgr2=0x7f61d40c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4152920 0x7f61d40c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 -- 172.21.15.26:0/1852254658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40c4920 con 0x7f61d4158280 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.139+0000 7f61dab93640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4158280 0x7f61d40c2490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009d90 tx=0x7f61c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.140+0000 7f61d2ffd640 1 -- 172.21.15.26:0/1852254658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4044020 con 0x7f61d4158280 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.140+0000 7f61d2ffd640 1 -- 172.21.15.26:0/1852254658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c403a040 con 0x7f61d4158280 2024-03-06T07:21:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.140+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d4157c50 con 0x7f61d4158280 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.140+0000 7f61d2ffd640 1 -- 172.21.15.26:0/1852254658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4039050 con 0x7f61d4158280 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.140+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d4157690 con 0x7f61d4158280 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.141+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d404f330 con 0x7f61d4158280 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.141+0000 7f61d2ffd640 1 -- 172.21.15.26:0/1852254658 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61c4003bb0 con 0x7f61d4158280 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.141+0000 7f61d2ffd640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61c006b4d0 0x7f61c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.141+0000 7f61d2ffd640 1 -- 172.21.15.26:0/1852254658 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61c40ac810 con 0x7f61d4158280 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.142+0000 7f61da392640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61c006b4d0 0x7f61c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.142+0000 7f61da392640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61c006b4d0 0x7f61c006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f61cc000f60 tx=0x7f61cc007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.146+0000 7f61d2ffd640 1 -- 172.21.15.26:0/1852254658 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f61c404e060 con 0x7f61d4158280 2024-03-06T07:21:03.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.365+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61d406cda0 con 0x7f61d4158280 2024-03-06T07:21:03.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.366+0000 7f61d2ffd640 1 -- 172.21.15.26:0/1852254658 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f61d40c4920 con 0x7f61d4158280 2024-03-06T07:21:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:03.369 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:02.358562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/2470383395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2470383395},{"type":"v1","addr":"172.21.15.117:6836","nonce":2470383395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5209,"qdb_cluster":[5209]},"id":2}]} 2024-03-06T07:21:03.369 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-03-06T07:21:03.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61c006b4d0 msgr2=0x7f61c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61c006b4d0 0x7f61c006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f61cc000f60 tx=0x7f61cc007d50 comp rx=0 tx=0).stop 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4158280 msgr2=0x7f61d40c2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4158280 0x7f61d40c2490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009d90 tx=0x7f61c4004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 shutdown_connections 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61c006b4d0 0x7f61c006d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61d4158280 0x7f61d40c2490 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4152920 0x7f61d40c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 --2- 172.21.15.26:0/1852254658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d40c8a00 0x7f61d4156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 >> 172.21.15.26:0/1852254658 conn(0x7f61d40ac560 msgr2=0x7f61d40ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 shutdown_connections 2024-03-06T07:21:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.368+0000 7f61dc5f6640 1 -- 172.21.15.26:0/1852254658 wait complete. 2024-03-06T07:21:03.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:03.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.480+0000 7ff4a389a640 1 Processor -- start 2024-03-06T07:21:03.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.480+0000 7ff4a389a640 1 -- start start 2024-03-06T07:21:03.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a389a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a389a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff49c0c36e0 0x7ff49c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a389a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c158800 con 0x7ff49c0c3280 2024-03-06T07:21:03.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a389a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c0c3b10 con 0x7ff49c0c36e0 2024-03-06T07:21:03.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a389a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c158ae0 con 0x7ff49c06d9f0 2024-03-06T07:21:03.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1636640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1e37640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff49c0c36e0 0x7ff49c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1636640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c0a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:55762/0 (socket says 172.21.15.26:55762) 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1636640 1 -- 172.21.15.26:0/3995297195 learned_addr learned my addr 172.21.15.26:0/3995297195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a0e35640 1 --1- 172.21.15.26:0/3995297195 >> v1:172.21.15.117:6789/0 conn(0x7ff49c0c3280 0x7ff49c1551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38824/0 (socket says 172.21.15.26:38824) 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1636640 1 -- 172.21.15.26:0/3995297195 >> v2:172.21.15.117:3300/0 conn(0x7ff49c0c36e0 msgr2=0x7ff49c1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1636640 1 --2- 172.21.15.26:0/3995297195 >> v2:172.21.15.117:3300/0 conn(0x7ff49c0c36e0 0x7ff49c1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1636640 1 -- 172.21.15.26:0/3995297195 >> v1:172.21.15.117:6789/0 conn(0x7ff49c0c3280 legacy=0x7ff49c1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1636640 1 -- 172.21.15.26:0/3995297195 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c158dc0 con 0x7ff49c06d9f0 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.481+0000 7ff4a1e37640 1 --2- 172.21.15.26:0/3995297195 >> v2:172.21.15.117:3300/0 conn(0x7ff49c0c36e0 0x7ff49c1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.482+0000 7ff4a1636640 1 --2- 172.21.15.26:0/3995297195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c0a7cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff494009870 tx=0x7ff49402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5a67de354bb036 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.482+0000 7ff49bfff640 1 -- 172.21.15.26:0/3995297195 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494035020 con 0x7ff49c06d9f0 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.482+0000 7ff49bfff640 1 -- 172.21.15.26:0/3995297195 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494031640 con 0x7ff49c06d9f0 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.482+0000 7ff49bfff640 1 -- 172.21.15.26:0/3995297195 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49402d040 con 0x7ff49c06d9f0 2024-03-06T07:21:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 -- 172.21.15.26:0/3995297195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 msgr2=0x7ff49c0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 --2- 172.21.15.26:0/3995297195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c0a7cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff494009870 tx=0x7ff49402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 -- 172.21.15.26:0/3995297195 shutdown_connections 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 --2- 172.21.15.26:0/3995297195 >> v2:172.21.15.117:3300/0 conn(0x7ff49c0c36e0 0x7ff49c1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 --2- 172.21.15.26:0/3995297195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c0a7cd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 -- 172.21.15.26:0/3995297195 >> 172.21.15.26:0/3995297195 conn(0x7ff49c0aa630 msgr2=0x7ff49c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 -- 172.21.15.26:0/3995297195 shutdown_connections 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.483+0000 7ff4a389a640 1 -- 172.21.15.26:0/3995297195 wait complete. 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.484+0000 7ff4a389a640 1 Processor -- start 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.484+0000 7ff4a389a640 1 -- start start 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a389a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c14bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a389a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff49c0c3280 0x7ff49c1ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a389a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff49c0c36e0 0x7ff49c14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a389a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c155e30 con 0x7ff49c0c36e0 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a389a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c159380 con 0x7ff49c0c3280 2024-03-06T07:21:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a389a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c06ddd0 con 0x7ff49c06d9f0 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff49c0c36e0 0x7ff49c14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff49c0c36e0 0x7ff49c14dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44420/0 (socket says 172.21.15.26:44420) 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 -- 172.21.15.26:0/580821758 learned_addr learned my addr 172.21.15.26:0/580821758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1636640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c14bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a0e35640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff49c0c3280 0x7ff49c1ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 -- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 msgr2=0x7ff49c14bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c14bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 -- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff49c0c3280 msgr2=0x7ff49c1ee180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff49c0c3280 0x7ff49c1ee180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 -- 172.21.15.26:0/580821758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c14b8b0 con 0x7ff49c0c36e0 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1636640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c14bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a1e37640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff49c0c36e0 0x7ff49c14dc60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009e50 tx=0x7ff48c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff499ffb640 1 -- 172.21.15.26:0/580821758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c018030 con 0x7ff49c0c36e0 2024-03-06T07:21:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff499ffb640 1 -- 172.21.15.26:0/580821758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c00cd90 con 0x7ff49c0c36e0 2024-03-06T07:21:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.485+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c1eea50 con 0x7ff49c0c36e0 2024-03-06T07:21:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.486+0000 7ff499ffb640 1 -- 172.21.15.26:0/580821758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c003bb0 con 0x7ff49c0c36e0 2024-03-06T07:21:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.486+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c1530d0 con 0x7ff49c0c36e0 2024-03-06T07:21:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.487+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c04f330 con 0x7ff49c0c36e0 2024-03-06T07:21:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.487+0000 7ff499ffb640 1 -- 172.21.15.26:0/580821758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff48c01b070 con 0x7ff49c0c36e0 2024-03-06T07:21:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.488+0000 7ff499ffb640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48806b500 0x7ff48806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.488+0000 7ff499ffb640 1 -- 172.21.15.26:0/580821758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff48c08ac50 con 0x7ff49c0c36e0 2024-03-06T07:21:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.488+0000 7ff4a1636640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48806b500 0x7ff48806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.488+0000 7ff4a1636640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48806b500 0x7ff48806d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff494036000 tx=0x7ff49403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.492+0000 7ff499ffb640 1 -- 172.21.15.26:0/580821758 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff48c010040 con 0x7ff49c0c36e0 2024-03-06T07:21:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.674+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff49c06cf40 con 0x7ff49c0c36e0 2024-03-06T07:21:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.675+0000 7ff499ffb640 1 -- 172.21.15.26:0/580821758 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16954 (secure 0 0 0) 0x7ff48c059290 con 0x7ff49c0c36e0 2024-03-06T07:21:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:03.678 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:02.004441+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:58.241645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:59.328635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48806b500 msgr2=0x7ff48806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48806b500 0x7ff48806d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff494036000 tx=0x7ff49403c040 comp rx=0 tx=0).stop 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff49c0c36e0 msgr2=0x7ff49c14dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff49c0c36e0 0x7ff49c14dc60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009e50 tx=0x7ff48c00c490 comp rx=0 tx=0).stop 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 shutdown_connections 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48806b500 0x7ff48806d9c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff49c0c36e0 0x7ff49c14dc60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff49c0c3280 0x7ff49c1ee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 --2- 172.21.15.26:0/580821758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff49c06d9f0 0x7ff49c14bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 >> 172.21.15.26:0/580821758 conn(0x7ff49c0aa630 msgr2=0x7ff49c06c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 shutdown_connections 2024-03-06T07:21:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.677+0000 7ff4a389a640 1 -- 172.21.15.26:0/580821758 wait complete. 2024-03-06T07:21:03.691 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:21:03.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65c90d640 1 Processor -- start 2024-03-06T07:21:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65c90d640 1 -- start start 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65c90d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc65806d9f0 0x7fc6580a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65c90d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc6580c3710 0x7fc6581558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65c90d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc658158800 con 0x7fc6580c3280 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65c90d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc658158ae0 con 0x7fc65806d9f0 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65c90d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc658158dc0 con 0x7fc6580c3710 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.790+0000 7fc65659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc65806d9f0 0x7fc6580a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc65659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc65806d9f0 0x7fc6580a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44430/0 (socket says 172.21.15.26:44430) 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc65659c640 1 -- 172.21.15.26:0/14889395 learned_addr learned my addr 172.21.15.26:0/14889395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc656d9d640 1 --2- 172.21.15.26:0/14889395 >> v2:172.21.15.117:3300/0 conn(0x7fc6580c3710 0x7fc6581558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc65659c640 1 -- 172.21.15.26:0/14889395 >> v2:172.21.15.117:3300/0 conn(0x7fc6580c3710 msgr2=0x7fc6581558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc65659c640 1 --2- 172.21.15.26:0/14889395 >> v2:172.21.15.117:3300/0 conn(0x7fc6580c3710 0x7fc6581558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc65659c640 1 -- 172.21.15.26:0/14889395 >> v1:172.21.15.117:6789/0 conn(0x7fc6580c3280 legacy=0x7fc6581551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc65659c640 1 -- 172.21.15.26:0/14889395 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6581590a0 con 0x7fc65806d9f0 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.791+0000 7fc65659c640 1 --2- 172.21.15.26:0/14889395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc65806d9f0 0x7fc6580a7cd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc658159760 tx=0x7fc64802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d151f29370ea493 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc655d9b640 1 -- 172.21.15.26:0/14889395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648035020 con 0x7fc65806d9f0 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc655d9b640 1 -- 172.21.15.26:0/14889395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc648031870 con 0x7fc65806d9f0 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc655d9b640 1 -- 172.21.15.26:0/14889395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648031c40 con 0x7fc65806d9f0 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 -- 172.21.15.26:0/14889395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc65806d9f0 msgr2=0x7fc6580a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 --2- 172.21.15.26:0/14889395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc65806d9f0 0x7fc6580a7cd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc658159760 tx=0x7fc64802b910 comp rx=0 tx=0).stop 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 -- 172.21.15.26:0/14889395 shutdown_connections 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 --2- 172.21.15.26:0/14889395 >> v2:172.21.15.117:3300/0 conn(0x7fc6580c3710 0x7fc6581558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 --2- 172.21.15.26:0/14889395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc65806d9f0 0x7fc6580a7cd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 -- 172.21.15.26:0/14889395 >> 172.21.15.26:0/14889395 conn(0x7fc6580aa630 msgr2=0x7fc6580aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 -- 172.21.15.26:0/14889395 shutdown_connections 2024-03-06T07:21:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.792+0000 7fc65c90d640 1 -- 172.21.15.26:0/14889395 wait complete. 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.793+0000 7fc65c90d640 1 Processor -- start 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65c90d640 1 -- start start 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65c90d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc65806d9f0 0x7fc65806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65c90d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6580c3280 0x7fc65814b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65c90d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6580c3710 0x7fc65814b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65c90d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65806ddd0 con 0x7fc6580c3280 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65c90d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc658155e30 con 0x7fc65806d9f0 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65c90d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65806df50 con 0x7fc6580c3710 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc64ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6580c3280 0x7fc65814b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc64ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6580c3280 0x7fc65814b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44444/0 (socket says 172.21.15.26:44444) 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc64ffff640 1 -- 172.21.15.26:0/1181568905 learned_addr learned my addr 172.21.15.26:0/1181568905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc65659c640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc65806d9f0 0x7fc65806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.794+0000 7fc656d9d640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6580c3710 0x7fc65814b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64ffff640 1 -- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6580c3710 msgr2=0x7fc65814b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64ffff640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6580c3710 0x7fc65814b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64ffff640 1 -- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc65806d9f0 msgr2=0x7fc65806c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64ffff640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc65806d9f0 0x7fc65806c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64ffff640 1 -- 172.21.15.26:0/1181568905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6581ee990 con 0x7fc6580c3280 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64ffff640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6580c3280 0x7fc65814b1d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c400 tx=0x7fc64000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64effd640 1 -- 172.21.15.26:0/1181568905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400052d0 con 0x7fc6580c3280 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64effd640 1 -- 172.21.15.26:0/1181568905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc640005490 con 0x7fc6580c3280 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc658070420 con 0x7fc6580c3280 2024-03-06T07:21:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc64effd640 1 -- 172.21.15.26:0/1181568905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640007260 con 0x7fc6580c3280 2024-03-06T07:21:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.795+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65814bf90 con 0x7fc6580c3280 2024-03-06T07:21:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.796+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6581eee90 con 0x7fc6580c3280 2024-03-06T07:21:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.797+0000 7fc64effd640 1 -- 172.21.15.26:0/1181568905 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc640007400 con 0x7fc6580c3280 2024-03-06T07:21:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.797+0000 7fc64effd640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.797+0000 7fc64effd640 1 -- 172.21.15.26:0/1181568905 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc6400897d0 con 0x7fc6580c3280 2024-03-06T07:21:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.797+0000 7fc65659c640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.797+0000 7fc65659c640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc648036000 tx=0x7fc64803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.801+0000 7fc64effd640 1 -- 172.21.15.26:0/1181568905 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc6581eee90 con 0x7fc6580c3280 2024-03-06T07:21:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.980+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc65814c190 con 0x7fc63806b4d0 2024-03-06T07:21:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.987+0000 7fc64effd640 1 -- 172.21.15.26:0/1181568905 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246709 (secure 0 0 0) 0x7fc6480843d0 con 0x7fc63806b4d0 2024-03-06T07:21:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc63806b4d0 msgr2=0x7fc63806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc648036000 tx=0x7fc64803c040 comp rx=0 tx=0).stop 2024-03-06T07:21:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6580c3280 msgr2=0x7fc65814b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6580c3280 0x7fc65814b1d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c400 tx=0x7fc64000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 shutdown_connections 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6580c3710 0x7fc65814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6580c3280 0x7fc65814b1d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 --2- 172.21.15.26:0/1181568905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc65806d9f0 0x7fc65806c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 >> 172.21.15.26:0/1181568905 conn(0x7fc6580aa630 msgr2=0x7fc6580c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 shutdown_connections 2024-03-06T07:21:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:03.989+0000 7fc65c90d640 1 -- 172.21.15.26:0/1181568905 wait complete. 2024-03-06T07:21:04.003 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-06T07:21:03.042861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":3168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":3244032,"data_stored":2483024,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.058153"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999702+0000","last_change":"2024-03-06T07:20:59.999702+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999702+0000","last_undegraded":"2024-03-06T07:20:59.999702+0000","last_fullsized":"2024-03-06T07:20:59.999702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998904+0000","last_change":"2024-03-06T07:20:59.998904+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998904+0000","last_undegraded":"2024-03-06T07:20:59.998904+0000","last_fullsized":"2024-03-06T07:20:59.998904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003003+0000","last_change":"2024-03-06T07:21:00.003003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003003+0000","last_undegraded":"2024-03-06T07:21:00.003003+0000","last_fullsized":"2024-03-06T07:21:00.003003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997726+0000","last_change":"2024-03-06T07:20:59.997726+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997726+0000","last_undegraded":"2024-03-06T07:20:59.997726+0000","last_fullsized":"2024-03-06T07:20:59.997726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996704+0000","last_change":"2024-03-06T07:20:59.996704+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996704+0000","last_undegraded":"2024-03-06T07:20:59.996704+0000","last_fullsized":"2024-03-06T07:20:59.996704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001349+0000","last_change":"2024-03-06T07:21:00.001349+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001349+0000","last_undegraded":"2024-03-06T07:21:00.001349+0000","last_fullsized":"2024-03-06T07:21:00.001349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001027+0000","last_change":"2024-03-06T07:21:00.001027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001027+0000","last_undegraded":"2024-03-06T07:21:00.001027+0000","last_fullsized":"2024-03-06T07:21:00.001027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997302+0000","last_change":"2024-03-06T07:20:59.997302+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997302+0000","last_undegraded":"2024-03-06T07:20:59.997302+0000","last_fullsized":"2024-03-06T07:20:59.997302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003260+0000","last_change":"2024-03-06T07:21:00.003260+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003260+0000","last_undegraded":"2024-03-06T07:21:00.003260+0000","last_fullsized":"2024-03-06T07:21:00.003260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002815+0000","last_change":"2024-03-06T07:21:00.002815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002815+0000","last_undegraded":"2024-03-06T07:21:00.002815+0000","last_fullsized":"2024-03-06T07:21:00.002815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004749+0000","last_change":"2024-03-06T07:21:00.004749+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004749+0000","last_undegraded":"2024-03-06T07:21:00.004749+0000","last_fullsized":"2024-03-06T07:21:00.004749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003163+0000","last_change":"2024-03-06T07:21:00.003163+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003163+0000","last_undegraded":"2024-03-06T07:21:00.003163+0000","last_fullsized":"2024-03-06T07:21:00.003163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003644+0000","last_change":"2024-03-06T07:21:00.003644+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003644+0000","last_undegraded":"2024-03-06T07:21:00.003644+0000","last_fullsized":"2024-03-06T07:21:00.003644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001459+0000","last_change":"2024-03-06T07:21:00.001459+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001459+0000","last_undegraded":"2024-03-06T07:21:00.001459+0000","last_fullsized":"2024-03-06T07:21:00.001459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000599+0000","last_change":"2024-03-06T07:21:00.000599+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000599+0000","last_undegraded":"2024-03-06T07:21:00.000599+0000","last_fullsized":"2024-03-06T07:21:00.000599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998031+0000","last_change":"2024-03-06T07:20:59.998031+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998031+0000","last_undegraded":"2024-03-06T07:20:59.998031+0000","last_fullsized":"2024-03-06T07:20:59.998031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998032+0000","last_change":"2024-03-06T07:20:59.998032+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998032+0000","last_undegraded":"2024-03-06T07:20:59.998032+0000","last_fullsized":"2024-03-06T07:20:59.998032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003193+0000","last_change":"2024-03-06T07:21:00.003193+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003193+0000","last_undegraded":"2024-03-06T07:21:00.003193+0000","last_fullsized":"2024-03-06T07:21:00.003193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997357+0000","last_change":"2024-03-06T07:20:59.997357+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997357+0000","last_undegraded":"2024-03-06T07:20:59.997357+0000","last_fullsized":"2024-03-06T07:20:59.997357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003259+0000","last_change":"2024-03-06T07:21:00.003259+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003259+0000","last_undegraded":"2024-03-06T07:21:00.003259+0000","last_fullsized":"2024-03-06T07:21:00.003259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003247+0000","last_change":"2024-03-06T07:21:00.003247+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003247+0000","last_undegraded":"2024-03-06T07:21:00.003247+0000","last_fullsized":"2024-03-06T07:21:00.003247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000143+0000","last_change":"2024-03-06T07:21:00.000143+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000143+0000","last_undegraded":"2024-03-06T07:21:00.000143+0000","last_fullsized":"2024-03-06T07:21:00.000143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997546+0000","last_change":"2024-03-06T07:20:59.997546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997546+0000","last_undegraded":"2024-03-06T07:20:59.997546+0000","last_fullsized":"2024-03-06T07:20:59.997546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998183+0000","last_change":"2024-03-06T07:20:59.998183+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998183+0000","last_undegraded":"2024-03-06T07:20:59.998183+0000","last_fullsized":"2024-03-06T07:20:59.998183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003168+0000","last_change":"2024-03-06T07:21:00.003168+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003168+0000","last_undegraded":"2024-03-06T07:21:00.003168+0000","last_fullsized":"2024-03-06T07:21:00.003168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002177+0000","last_change":"2024-03-06T07:21:00.002177+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002177+0000","last_undegraded":"2024-03-06T07:21:00.002177+0000","last_fullsized":"2024-03-06T07:21:00.002177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003051+0000","last_change":"2024-03-06T07:21:00.003051+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003051+0000","last_undegraded":"2024-03-06T07:21:00.003051+0000","last_fullsized":"2024-03-06T07:21:00.003051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000938+0000","last_change":"2024-03-06T07:21:00.000938+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000938+0000","last_undegraded":"2024-03-06T07:21:00.000938+0000","last_fullsized":"2024-03-06T07:21:00.000938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002465+0000","last_change":"2024-03-06T07:21:00.002465+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002465+0000","last_undegraded":"2024-03-06T07:21:00.002465+0000","last_fullsized":"2024-03-06T07:21:00.002465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001989+0000","last_change":"2024-03-06T07:21:00.001989+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001989+0000","last_undegraded":"2024-03-06T07:21:00.001989+0000","last_fullsized":"2024-03-06T07:21:00.001989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997070+0000","last_change":"2024-03-06T07:20:59.997070+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997070+0000","last_undegraded":"2024-03-06T07:20:59.997070+0000","last_fullsized":"2024-03-06T07:20:59.997070+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002027+0000","last_change":"2024-03-06T07:21:00.002027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002027+0000","last_undegraded":"2024-03-06T07:21:00.002027+0000","last_fullsized":"2024-03-06T07:21:00.002027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997095+0000","last_change":"2024-03-06T07:20:59.997095+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997095+0000","last_undegraded":"2024-03-06T07:20:59.997095+0000","last_fullsized":"2024-03-06T07:20:59.997095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018235+0000","last_change":"2024-03-06T07:21:00.010567+0000","last_active":"2024-03-06T07:21:00.018235+0000","last_peered":"2024-03-06T07:21:00.018235+0000","last_clean":"2024-03-06T07:21:00.018235+0000","last_became_active":"2024-03-06T07:21:00.010277+0000","last_became_peered":"2024-03-06T07:21:00.010277+0000","last_unstale":"2024-03-06T07:21:00.018235+0000","last_undegraded":"2024-03-06T07:21:00.018235+0000","last_fullsized":"2024-03-06T07:21:00.018235+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.435165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001748+0000","last_change":"2024-03-06T07:21:00.001748+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001748+0000","last_undegraded":"2024-03-06T07:21:00.001748+0000","last_fullsized":"2024-03-06T07:21:00.001748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.013052+0000","last_change":"2024-03-06T07:21:00.011675+0000","last_active":"2024-03-06T07:21:00.013052+0000","last_peered":"2024-03-06T07:21:00.013052+0000","last_clean":"2024-03-06T07:21:00.013052+0000","last_became_active":"2024-03-06T07:21:00.011344+0000","last_became_peered":"2024-03-06T07:21:00.011344+0000","last_unstale":"2024-03-06T07:21:00.013052+0000","last_undegraded":"2024-03-06T07:21:00.013052+0000","last_fullsized":"2024-03-06T07:21:00.013052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:04.618247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000216+0000","last_change":"2024-03-06T07:21:00.000216+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000216+0000","last_undegraded":"2024-03-06T07:21:00.000216+0000","last_fullsized":"2024-03-06T07:21:00.000216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016876+0000","last_change":"2024-03-06T07:21:00.012691+0000","last_active":"2024-03-06T07:21:00.016876+0000","last_peered":"2024-03-06T07:21:00.016876+0000","last_clean":"2024-03-06T07:21:00.016876+0000","last_became_active":"2024-03-06T07:21:00.012404+0000","last_became_peered":"2024-03-06T07:21:00.012404+0000","last_unstale":"2024-03-06T07:21:00.016876+0000","last_undegraded":"2024-03-06T07:21:00.016876+0000","last_fullsized":"2024-03-06T07:21:00.016876+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000111+0000","last_change":"2024-03-06T07:21:00.000111+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000111+0000","last_undegraded":"2024-03-06T07:21:00.000111+0000","last_fullsized":"2024-03-06T07:21:00.000111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018440+0000","last_change":"2024-03-06T07:21:00.011238+0000","last_active":"2024-03-06T07:21:00.018440+0000","last_peered":"2024-03-06T07:21:00.018440+0000","last_clean":"2024-03-06T07:21:00.018440+0000","last_became_active":"2024-03-06T07:21:00.010202+0000","last_became_peered":"2024-03-06T07:21:00.010202+0000","last_unstale":"2024-03-06T07:21:00.018440+0000","last_undegraded":"2024-03-06T07:21:00.018440+0000","last_fullsized":"2024-03-06T07:21:00.018440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:23.626033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.010108+0000","last_change":"2024-03-06T07:21:00.010108+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.010108+0000","last_undegraded":"2024-03-06T07:21:00.010108+0000","last_fullsized":"2024-03-06T07:21:00.010108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018243+0000","last_change":"2024-03-06T07:21:00.008004+0000","last_active":"2024-03-06T07:21:00.018243+0000","last_peered":"2024-03-06T07:21:00.018243+0000","last_clean":"2024-03-06T07:21:00.018243+0000","last_became_active":"2024-03-06T07:21:00.007716+0000","last_became_peered":"2024-03-06T07:21:00.007716+0000","last_unstale":"2024-03-06T07:21:00.018243+0000","last_undegraded":"2024-03-06T07:21:00.018243+0000","last_fullsized":"2024-03-06T07:21:00.018243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000136+0000","last_change":"2024-03-06T07:21:00.000136+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000136+0000","last_undegraded":"2024-03-06T07:21:00.000136+0000","last_fullsized":"2024-03-06T07:21:00.000136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010724+0000","last_change":"2024-03-06T07:21:00.006417+0000","last_active":"2024-03-06T07:21:00.010724+0000","last_peered":"2024-03-06T07:21:00.010724+0000","last_clean":"2024-03-06T07:21:00.010724+0000","last_became_active":"2024-03-06T07:21:00.006164+0000","last_became_peered":"2024-03-06T07:21:00.006164+0000","last_unstale":"2024-03-06T07:21:00.010724+0000","last_undegraded":"2024-03-06T07:21:00.010724+0000","last_fullsized":"2024-03-06T07:21:00.010724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:32.480280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000567+0000","last_change":"2024-03-06T07:21:00.000567+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000567+0000","last_undegraded":"2024-03-06T07:21:00.000567+0000","last_fullsized":"2024-03-06T07:21:00.000567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.019321+0000","last_change":"2024-03-06T07:21:00.018959+0000","last_active":"2024-03-06T07:21:00.019321+0000","last_peered":"2024-03-06T07:21:00.019321+0000","last_clean":"2024-03-06T07:21:00.019321+0000","last_became_active":"2024-03-06T07:21:00.018641+0000","last_became_peered":"2024-03-06T07:21:00.018641+0000","last_unstale":"2024-03-06T07:21:00.019321+0000","last_undegraded":"2024-03-06T07:21:00.019321+0000","last_fullsized":"2024-03-06T07:21:00.019321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998815+0000","last_change":"2024-03-06T07:20:59.998815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998815+0000","last_undegraded":"2024-03-06T07:20:59.998815+0000","last_fullsized":"2024-03-06T07:20:59.998815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018484+0000","last_change":"2024-03-06T07:21:00.012306+0000","last_active":"2024-03-06T07:21:00.018484+0000","last_peered":"2024-03-06T07:21:00.018484+0000","last_clean":"2024-03-06T07:21:00.018484+0000","last_became_active":"2024-03-06T07:21:00.012030+0000","last_became_peered":"2024-03-06T07:21:00.012030+0000","last_unstale":"2024-03-06T07:21:00.018484+0000","last_undegraded":"2024-03-06T07:21:00.018484+0000","last_fullsized":"2024-03-06T07:21:00.018484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:34.501662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997276+0000","last_change":"2024-03-06T07:20:59.997276+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997276+0000","last_undegraded":"2024-03-06T07:20:59.997276+0000","last_fullsized":"2024-03-06T07:20:59.997276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017570+0000","last_change":"2024-03-06T07:21:00.015472+0000","last_active":"2024-03-06T07:21:00.017570+0000","last_peered":"2024-03-06T07:21:00.017570+0000","last_clean":"2024-03-06T07:21:00.017570+0000","last_became_active":"2024-03-06T07:21:00.015206+0000","last_became_peered":"2024-03-06T07:21:00.015206+0000","last_unstale":"2024-03-06T07:21:00.017570+0000","last_undegraded":"2024-03-06T07:21:00.017570+0000","last_fullsized":"2024-03-06T07:21:00.017570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:12.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996607+0000","last_change":"2024-03-06T07:20:59.996607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996607+0000","last_undegraded":"2024-03-06T07:20:59.996607+0000","last_fullsized":"2024-03-06T07:20:59.996607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018385+0000","last_change":"2024-03-06T07:21:00.011966+0000","last_active":"2024-03-06T07:21:00.018385+0000","last_peered":"2024-03-06T07:21:00.018385+0000","last_clean":"2024-03-06T07:21:00.018385+0000","last_became_active":"2024-03-06T07:21:00.011703+0000","last_became_peered":"2024-03-06T07:21:00.011703+0000","last_unstale":"2024-03-06T07:21:00.018385+0000","last_undegraded":"2024-03-06T07:21:00.018385+0000","last_fullsized":"2024-03-06T07:21:00.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:12.743167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004577+0000","last_change":"2024-03-06T07:21:00.004577+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004577+0000","last_undegraded":"2024-03-06T07:21:00.004577+0000","last_fullsized":"2024-03-06T07:21:00.004577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018708+0000","last_change":"2024-03-06T07:21:00.017403+0000","last_active":"2024-03-06T07:21:00.018708+0000","last_peered":"2024-03-06T07:21:00.018708+0000","last_clean":"2024-03-06T07:21:00.018708+0000","last_became_active":"2024-03-06T07:21:00.017171+0000","last_became_peered":"2024-03-06T07:21:00.017171+0000","last_unstale":"2024-03-06T07:21:00.018708+0000","last_undegraded":"2024-03-06T07:21:00.018708+0000","last_fullsized":"2024-03-06T07:21:00.018708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999477+0000","last_change":"2024-03-06T07:20:59.999477+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999477+0000","last_undegraded":"2024-03-06T07:20:59.999477+0000","last_fullsized":"2024-03-06T07:20:59.999477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011916+0000","last_change":"2024-03-06T07:21:00.007232+0000","last_active":"2024-03-06T07:21:00.011916+0000","last_peered":"2024-03-06T07:21:00.011916+0000","last_clean":"2024-03-06T07:21:00.011916+0000","last_became_active":"2024-03-06T07:21:00.006787+0000","last_became_peered":"2024-03-06T07:21:00.006787+0000","last_unstale":"2024-03-06T07:21:00.011916+0000","last_undegraded":"2024-03-06T07:21:00.011916+0000","last_fullsized":"2024-03-06T07:21:00.011916+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.009601+0000","last_change":"2024-03-06T07:21:00.009601+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.009601+0000","last_undegraded":"2024-03-06T07:21:00.009601+0000","last_fullsized":"2024-03-06T07:21:00.009601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017495+0000","last_change":"2024-03-06T07:21:00.014388+0000","last_active":"2024-03-06T07:21:00.017495+0000","last_peered":"2024-03-06T07:21:00.017495+0000","last_clean":"2024-03-06T07:21:00.017495+0000","last_became_active":"2024-03-06T07:21:00.014118+0000","last_became_peered":"2024-03-06T07:21:00.014118+0000","last_unstale":"2024-03-06T07:21:00.017495+0000","last_undegraded":"2024-03-06T07:21:00.017495+0000","last_fullsized":"2024-03-06T07:21:00.017495+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:16.342176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999388+0000","last_change":"2024-03-06T07:20:59.999388+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999388+0000","last_undegraded":"2024-03-06T07:20:59.999388+0000","last_fullsized":"2024-03-06T07:20:59.999388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017297+0000","last_change":"2024-03-06T07:21:00.014221+0000","last_active":"2024-03-06T07:21:00.017297+0000","last_peered":"2024-03-06T07:21:00.017297+0000","last_clean":"2024-03-06T07:21:00.017297+0000","last_became_active":"2024-03-06T07:21:00.013921+0000","last_became_peered":"2024-03-06T07:21:00.013921+0000","last_unstale":"2024-03-06T07:21:00.017297+0000","last_undegraded":"2024-03-06T07:21:00.017297+0000","last_fullsized":"2024-03-06T07:21:00.017297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996517+0000","last_change":"2024-03-06T07:20:59.996517+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996517+0000","last_undegraded":"2024-03-06T07:20:59.996517+0000","last_fullsized":"2024-03-06T07:20:59.996517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015389+0000","last_change":"2024-03-06T07:21:00.010506+0000","last_active":"2024-03-06T07:21:00.015389+0000","last_peered":"2024-03-06T07:21:00.015389+0000","last_clean":"2024-03-06T07:21:00.015389+0000","last_became_active":"2024-03-06T07:21:00.010172+0000","last_became_peered":"2024-03-06T07:21:00.010172+0000","last_unstale":"2024-03-06T07:21:00.015389+0000","last_undegraded":"2024-03-06T07:21:00.015389+0000","last_fullsized":"2024-03-06T07:21:00.015389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:46.431814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998304+0000","last_change":"2024-03-06T07:20:59.998304+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998304+0000","last_undegraded":"2024-03-06T07:20:59.998304+0000","last_fullsized":"2024-03-06T07:20:59.998304+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.009278+0000","last_change":"2024-03-06T07:21:00.005310+0000","last_active":"2024-03-06T07:21:00.009278+0000","last_peered":"2024-03-06T07:21:00.009278+0000","last_clean":"2024-03-06T07:21:00.009278+0000","last_became_active":"2024-03-06T07:21:00.004999+0000","last_became_peered":"2024-03-06T07:21:00.004999+0000","last_unstale":"2024-03-06T07:21:00.009278+0000","last_undegraded":"2024-03-06T07:21:00.009278+0000","last_fullsized":"2024-03-06T07:21:00.009278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999103+0000","last_change":"2024-03-06T07:20:59.999103+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999103+0000","last_undegraded":"2024-03-06T07:20:59.999103+0000","last_fullsized":"2024-03-06T07:20:59.999103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018327+0000","last_change":"2024-03-06T07:21:00.011969+0000","last_active":"2024-03-06T07:21:00.018327+0000","last_peered":"2024-03-06T07:21:00.018327+0000","last_clean":"2024-03-06T07:21:00.018327+0000","last_became_active":"2024-03-06T07:21:00.011741+0000","last_became_peered":"2024-03-06T07:21:00.011741+0000","last_unstale":"2024-03-06T07:21:00.018327+0000","last_undegraded":"2024-03-06T07:21:00.018327+0000","last_fullsized":"2024-03-06T07:21:00.018327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:12.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004368+0000","last_change":"2024-03-06T07:21:00.004368+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004368+0000","last_undegraded":"2024-03-06T07:21:00.004368+0000","last_fullsized":"2024-03-06T07:21:00.004368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018480+0000","last_change":"2024-03-06T07:21:00.013456+0000","last_active":"2024-03-06T07:21:00.018480+0000","last_peered":"2024-03-06T07:21:00.018480+0000","last_clean":"2024-03-06T07:21:00.018480+0000","last_became_active":"2024-03-06T07:21:00.013184+0000","last_became_peered":"2024-03-06T07:21:00.013184+0000","last_unstale":"2024-03-06T07:21:00.018480+0000","last_undegraded":"2024-03-06T07:21:00.018480+0000","last_fullsized":"2024-03-06T07:21:00.018480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000640+0000","last_change":"2024-03-06T07:21:00.000640+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000640+0000","last_undegraded":"2024-03-06T07:21:00.000640+0000","last_fullsized":"2024-03-06T07:21:00.000640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005305+0000","last_change":"2024-03-06T07:21:00.003577+0000","last_active":"2024-03-06T07:21:00.005305+0000","last_peered":"2024-03-06T07:21:00.005305+0000","last_clean":"2024-03-06T07:21:00.005305+0000","last_became_active":"2024-03-06T07:21:00.003294+0000","last_became_peered":"2024-03-06T07:21:00.003294+0000","last_unstale":"2024-03-06T07:21:00.005305+0000","last_undegraded":"2024-03-06T07:21:00.005305+0000","last_fullsized":"2024-03-06T07:21:00.005305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:02.580999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996923+0000","last_change":"2024-03-06T07:20:59.996923+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996923+0000","last_undegraded":"2024-03-06T07:20:59.996923+0000","last_fullsized":"2024-03-06T07:20:59.996923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016900+0000","last_change":"2024-03-06T07:21:00.012331+0000","last_active":"2024-03-06T07:21:00.016900+0000","last_peered":"2024-03-06T07:21:00.016900+0000","last_clean":"2024-03-06T07:21:00.016900+0000","last_became_active":"2024-03-06T07:21:00.011994+0000","last_became_peered":"2024-03-06T07:21:00.011994+0000","last_unstale":"2024-03-06T07:21:00.016900+0000","last_undegraded":"2024-03-06T07:21:00.016900+0000","last_fullsized":"2024-03-06T07:21:00.016900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996058+0000","last_change":"2024-03-06T07:20:59.996058+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996058+0000","last_undegraded":"2024-03-06T07:20:59.996058+0000","last_fullsized":"2024-03-06T07:20:59.996058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011233+0000","last_change":"2024-03-06T07:21:00.008859+0000","last_active":"2024-03-06T07:21:00.011233+0000","last_peered":"2024-03-06T07:21:00.011233+0000","last_clean":"2024-03-06T07:21:00.011233+0000","last_became_active":"2024-03-06T07:21:00.008565+0000","last_became_peered":"2024-03-06T07:21:00.008565+0000","last_unstale":"2024-03-06T07:21:00.011233+0000","last_undegraded":"2024-03-06T07:21:00.011233+0000","last_fullsized":"2024-03-06T07:21:00.011233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:03.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002934+0000","last_change":"2024-03-06T07:21:00.002934+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002934+0000","last_undegraded":"2024-03-06T07:21:00.002934+0000","last_fullsized":"2024-03-06T07:21:00.002934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010375+0000","last_change":"2024-03-06T07:21:00.007836+0000","last_active":"2024-03-06T07:21:00.010375+0000","last_peered":"2024-03-06T07:21:00.010375+0000","last_clean":"2024-03-06T07:21:00.010375+0000","last_became_active":"2024-03-06T07:21:00.006276+0000","last_became_peered":"2024-03-06T07:21:00.006276+0000","last_unstale":"2024-03-06T07:21:00.010375+0000","last_undegraded":"2024-03-06T07:21:00.010375+0000","last_fullsized":"2024-03-06T07:21:00.010375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:26.980322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999890+0000","last_change":"2024-03-06T07:20:59.999890+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999890+0000","last_undegraded":"2024-03-06T07:20:59.999890+0000","last_fullsized":"2024-03-06T07:20:59.999890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018160+0000","last_change":"2024-03-06T07:21:00.011278+0000","last_active":"2024-03-06T07:21:00.018160+0000","last_peered":"2024-03-06T07:21:00.018160+0000","last_clean":"2024-03-06T07:21:00.018160+0000","last_became_active":"2024-03-06T07:21:00.011018+0000","last_became_peered":"2024-03-06T07:21:00.011018+0000","last_unstale":"2024-03-06T07:21:00.018160+0000","last_undegraded":"2024-03-06T07:21:00.018160+0000","last_fullsized":"2024-03-06T07:21:00.018160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997087+0000","last_change":"2024-03-06T07:20:59.997087+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997087+0000","last_undegraded":"2024-03-06T07:20:59.997087+0000","last_fullsized":"2024-03-06T07:20:59.997087+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.014862+0000","last_change":"2024-03-06T07:21:00.013699+0000","last_active":"2024-03-06T07:21:00.014862+0000","last_peered":"2024-03-06T07:21:00.014862+0000","last_clean":"2024-03-06T07:21:00.014862+0000","last_became_active":"2024-03-06T07:21:00.013360+0000","last_became_peered":"2024-03-06T07:21:00.013360+0000","last_unstale":"2024-03-06T07:21:00.014862+0000","last_undegraded":"2024-03-06T07:21:00.014862+0000","last_fullsized":"2024-03-06T07:21:00.014862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:06.072955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001334+0000","last_change":"2024-03-06T07:21:00.001334+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001334+0000","last_undegraded":"2024-03-06T07:21:00.001334+0000","last_fullsized":"2024-03-06T07:21:00.001334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018669+0000","last_change":"2024-03-06T07:21:00.015827+0000","last_active":"2024-03-06T07:21:00.018669+0000","last_peered":"2024-03-06T07:21:00.018669+0000","last_clean":"2024-03-06T07:21:00.018669+0000","last_became_active":"2024-03-06T07:21:00.015565+0000","last_became_peered":"2024-03-06T07:21:00.015565+0000","last_unstale":"2024-03-06T07:21:00.018669+0000","last_undegraded":"2024-03-06T07:21:00.018669+0000","last_fullsized":"2024-03-06T07:21:00.018669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:44.990676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996060+0000","last_change":"2024-03-06T07:20:59.996060+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996060+0000","last_undegraded":"2024-03-06T07:20:59.996060+0000","last_fullsized":"2024-03-06T07:20:59.996060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011205+0000","last_change":"2024-03-06T07:21:00.009047+0000","last_active":"2024-03-06T07:21:00.011205+0000","last_peered":"2024-03-06T07:21:00.011205+0000","last_clean":"2024-03-06T07:21:00.011205+0000","last_became_active":"2024-03-06T07:21:00.008803+0000","last_became_peered":"2024-03-06T07:21:00.008803+0000","last_unstale":"2024-03-06T07:21:00.011205+0000","last_undegraded":"2024-03-06T07:21:00.011205+0000","last_fullsized":"2024-03-06T07:21:00.011205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:17.432587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998464+0000","last_change":"2024-03-06T07:20:59.998464+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998464+0000","last_undegraded":"2024-03-06T07:20:59.998464+0000","last_fullsized":"2024-03-06T07:20:59.998464+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016624+0000","last_change":"2024-03-06T07:21:00.015433+0000","last_active":"2024-03-06T07:21:00.016624+0000","last_peered":"2024-03-06T07:21:00.016624+0000","last_clean":"2024-03-06T07:21:00.016624+0000","last_became_active":"2024-03-06T07:21:00.015156+0000","last_became_peered":"2024-03-06T07:21:00.015156+0000","last_unstale":"2024-03-06T07:21:00.016624+0000","last_undegraded":"2024-03-06T07:21:00.016624+0000","last_fullsized":"2024-03-06T07:21:00.016624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.005409+0000","last_change":"2024-03-06T07:21:00.005409+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.005409+0000","last_undegraded":"2024-03-06T07:21:00.005409+0000","last_fullsized":"2024-03-06T07:21:00.005409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017074+0000","last_change":"2024-03-06T07:21:00.016544+0000","last_active":"2024-03-06T07:21:00.017074+0000","last_peered":"2024-03-06T07:21:00.017074+0000","last_clean":"2024-03-06T07:21:00.017074+0000","last_became_active":"2024-03-06T07:21:00.015894+0000","last_became_peered":"2024-03-06T07:21:00.015894+0000","last_unstale":"2024-03-06T07:21:00.017074+0000","last_undegraded":"2024-03-06T07:21:00.017074+0000","last_fullsized":"2024-03-06T07:21:00.017074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.008546+0000","last_change":"2024-03-06T07:21:00.008546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.008546+0000","last_undegraded":"2024-03-06T07:21:00.008546+0000","last_fullsized":"2024-03-06T07:21:00.008546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015354+0000","last_change":"2024-03-06T07:21:00.010211+0000","last_active":"2024-03-06T07:21:00.015354+0000","last_peered":"2024-03-06T07:21:00.015354+0000","last_clean":"2024-03-06T07:21:00.015354+0000","last_became_active":"2024-03-06T07:21:00.009915+0000","last_became_peered":"2024-03-06T07:21:00.009915+0000","last_unstale":"2024-03-06T07:21:00.015354+0000","last_undegraded":"2024-03-06T07:21:00.015354+0000","last_fullsized":"2024-03-06T07:21:00.015354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001003+0000","last_change":"2024-03-06T07:21:00.001003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001003+0000","last_undegraded":"2024-03-06T07:21:00.001003+0000","last_fullsized":"2024-03-06T07:21:00.001003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018391+0000","last_change":"2024-03-06T07:21:00.011094+0000","last_active":"2024-03-06T07:21:00.018391+0000","last_peered":"2024-03-06T07:21:00.018391+0000","last_clean":"2024-03-06T07:21:00.018391+0000","last_became_active":"2024-03-06T07:21:00.010457+0000","last_became_peered":"2024-03-06T07:21:00.010457+0000","last_unstale":"2024-03-06T07:21:00.018391+0000","last_undegraded":"2024-03-06T07:21:00.018391+0000","last_fullsized":"2024-03-06T07:21:00.018391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:41.550434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996256+0000","last_change":"2024-03-06T07:20:59.996256+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996256+0000","last_undegraded":"2024-03-06T07:20:59.996256+0000","last_fullsized":"2024-03-06T07:20:59.996256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017529+0000","last_change":"2024-03-06T07:21:00.014362+0000","last_active":"2024-03-06T07:21:00.017529+0000","last_peered":"2024-03-06T07:21:00.017529+0000","last_clean":"2024-03-06T07:21:00.017529+0000","last_became_active":"2024-03-06T07:21:00.014072+0000","last_became_peered":"2024-03-06T07:21:00.014072+0000","last_unstale":"2024-03-06T07:21:00.017529+0000","last_undegraded":"2024-03-06T07:21:00.017529+0000","last_fullsized":"2024-03-06T07:21:00.017529+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:12.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003607+0000","last_change":"2024-03-06T07:21:00.003607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003607+0000","last_undegraded":"2024-03-06T07:21:00.003607+0000","last_fullsized":"2024-03-06T07:21:00.003607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005867+0000","last_change":"2024-03-06T07:21:00.002103+0000","last_active":"2024-03-06T07:21:00.005867+0000","last_peered":"2024-03-06T07:21:00.005867+0000","last_clean":"2024-03-06T07:21:00.005867+0000","last_became_active":"2024-03-06T07:21:00.001791+0000","last_became_peered":"2024-03-06T07:21:00.001791+0000","last_unstale":"2024-03-06T07:21:00.005867+0000","last_undegraded":"2024-03-06T07:21:00.005867+0000","last_fullsized":"2024-03-06T07:21:00.005867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:20:59.995260+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:59.995260+0000","last_peered":"2024-03-06T07:20:59.995260+0000","last_clean":"2024-03-06T07:20:59.995260+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:59.995260+0000","last_undegraded":"2024-03-06T07:20:59.995260+0000","last_fullsized":"2024-03-06T07:20:59.995260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:04.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:04.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.142+0000 7f26e2f13640 1 Processor -- start 2024-03-06T07:21:04.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.142+0000 7f26e2f13640 1 -- start start 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e2f13640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26dc0c6780 0x7f26dc158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e2f13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc1592d0 0x7f26dc0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e2f13640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26dc072630 con 0x7f26dc06da40 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e2f13640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26dc06de20 con 0x7f26dc0c6780 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e2f13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc072910 con 0x7f26dc1592d0 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e14b0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc1592d0 0x7f26dc0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e0caf640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f26dc06da40 0x7f26dc155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38838/0 (socket says 172.21.15.26:38838) 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e0caf640 1 -- 172.21.15.26:0/2952404146 learned_addr learned my addr 172.21.15.26:0/2952404146 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26dbfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26dc0c6780 0x7f26dc158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26db7fe640 1 -- 172.21.15.26:0/2952404146 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215820993 0 0) 0x7f26dc072630 con 0x7f26dc06da40 2024-03-06T07:21:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e14b0640 1 -- 172.21.15.26:0/2952404146 >> v2:172.21.15.117:3300/0 conn(0x7f26dc0c6780 msgr2=0x7f26dc158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e14b0640 1 --2- 172.21.15.26:0/2952404146 >> v2:172.21.15.117:3300/0 conn(0x7f26dc0c6780 0x7f26dc158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e14b0640 1 -- 172.21.15.26:0/2952404146 >> v1:172.21.15.117:6789/0 conn(0x7f26dc06da40 legacy=0x7f26dc155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.143+0000 7f26e14b0640 1 -- 172.21.15.26:0/2952404146 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc072bf0 con 0x7f26dc1592d0 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.144+0000 7f26e14b0640 1 --2- 172.21.15.26:0/2952404146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc1592d0 0x7f26dc0c3280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009d90 tx=0x7f26d402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc27cabd404c202 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.144+0000 7f26db7fe640 1 -- 172.21.15.26:0/2952404146 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4039070 con 0x7f26dc1592d0 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.144+0000 7f26db7fe640 1 -- 172.21.15.26:0/2952404146 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d402cbc0 con 0x7f26dc1592d0 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.144+0000 7f26db7fe640 1 -- 172.21.15.26:0/2952404146 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4034440 con 0x7f26dc1592d0 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 -- 172.21.15.26:0/2952404146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc1592d0 msgr2=0x7f26dc0c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 --2- 172.21.15.26:0/2952404146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc1592d0 0x7f26dc0c3280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009d90 tx=0x7f26d402be10 comp rx=0 tx=0).stop 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 -- 172.21.15.26:0/2952404146 shutdown_connections 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 --2- 172.21.15.26:0/2952404146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc1592d0 0x7f26dc0c3280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 --2- 172.21.15.26:0/2952404146 >> v2:172.21.15.117:3300/0 conn(0x7f26dc0c6780 0x7f26dc158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 -- 172.21.15.26:0/2952404146 >> 172.21.15.26:0/2952404146 conn(0x7f26dc0aa630 msgr2=0x7f26dc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:04.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 -- 172.21.15.26:0/2952404146 shutdown_connections 2024-03-06T07:21:04.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.145+0000 7f26e2f13640 1 -- 172.21.15.26:0/2952404146 wait complete. 2024-03-06T07:21:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.146+0000 7f26e2f13640 1 Processor -- start 2024-03-06T07:21:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.146+0000 7f26e2f13640 1 -- start start 2024-03-06T07:21:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e2f13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26dc06da40 0x7f26dc14aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e2f13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc0c6780 0x7f26dc06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e2f13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc1592d0 0x7f26dc0a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e2f13640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc0c3940 con 0x7f26dc1592d0 2024-03-06T07:21:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e2f13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc0c37c0 con 0x7f26dc06da40 2024-03-06T07:21:04.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e2f13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc0c3ac0 con 0x7f26dc0c6780 2024-03-06T07:21:04.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e14b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc1592d0 0x7f26dc0a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e14b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc1592d0 0x7f26dc0a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44460/0 (socket says 172.21.15.26:44460) 2024-03-06T07:21:04.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e14b0640 1 -- 172.21.15.26:0/928204068 learned_addr learned my addr 172.21.15.26:0/928204068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:04.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26dbfff640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc0c6780 0x7f26dc06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e0caf640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26dc06da40 0x7f26dc14aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26dbfff640 1 -- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26dc06da40 msgr2=0x7f26dc14aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26dbfff640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26dc06da40 0x7f26dc14aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26dbfff640 1 -- 172.21.15.26:0/928204068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc1592d0 msgr2=0x7f26dc0a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26dbfff640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc1592d0 0x7f26dc0a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26dbfff640 1 -- 172.21.15.26:0/928204068 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc1eeec0 con 0x7f26dc0c6780 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e14b0640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc1592d0 0x7f26dc0a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26e0caf640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26dc06da40 0x7f26dc14aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.147+0000 7f26dbfff640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc0c6780 0x7f26dc06f6b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26d000e8a0 tx=0x7f26d000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.148+0000 7f26d97fa640 1 -- 172.21.15.26:0/928204068 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0017070 con 0x7f26dc0c6780 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.148+0000 7f26d97fa640 1 -- 172.21.15.26:0/928204068 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d000c940 con 0x7f26dc0c6780 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.148+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc0700d0 con 0x7f26dc0c6780 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.148+0000 7f26d97fa640 1 -- 172.21.15.26:0/928204068 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0009e60 con 0x7f26dc0c6780 2024-03-06T07:21:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.148+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc0a9ca0 con 0x7f26dc0c6780 2024-03-06T07:21:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.149+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc06e020 con 0x7f26dc0c6780 2024-03-06T07:21:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.149+0000 7f26d97fa640 1 -- 172.21.15.26:0/928204068 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26d001a070 con 0x7f26dc0c6780 2024-03-06T07:21:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.149+0000 7f26d97fa640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26c806b4d0 0x7f26c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.150+0000 7f26d97fa640 1 -- 172.21.15.26:0/928204068 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f26d008a0e0 con 0x7f26dc0c6780 2024-03-06T07:21:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.150+0000 7f26e0caf640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26c806b4d0 0x7f26c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.150+0000 7f26e0caf640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26c806b4d0 0x7f26c806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26cc007fd0 tx=0x7f26cc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.154+0000 7f26d97fa640 1 -- 172.21.15.26:0/928204068 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f26dc06e020 con 0x7f26dc0c6780 2024-03-06T07:21:04.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.332+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26dc1ee960 con 0x7f26c806b4d0 2024-03-06T07:21:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.344+0000 7f26d97fa640 1 -- 172.21.15.26:0/928204068 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246709 (secure 0 0 0) 0x7f26dc1ee960 con 0x7f26c806b4d0 2024-03-06T07:21:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:04.350 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:04.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26c806b4d0 msgr2=0x7f26c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26c806b4d0 0x7f26c806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26cc007fd0 tx=0x7f26cc007de0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc0c6780 msgr2=0x7f26dc06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc0c6780 0x7f26dc06f6b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26d000e8a0 tx=0x7f26d000ed60 comp rx=0 tx=0).stop 2024-03-06T07:21:04.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 shutdown_connections 2024-03-06T07:21:04.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26c806b4d0 0x7f26c806d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26dc1592d0 0x7f26dc0a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26dc0c6780 0x7f26dc06f6b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 --2- 172.21.15.26:0/928204068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26dc06da40 0x7f26dc14aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 >> 172.21.15.26:0/928204068 conn(0x7f26dc0aa630 msgr2=0x7f26dc0c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:04.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 shutdown_connections 2024-03-06T07:21:04.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.347+0000 7f26e2f13640 1 -- 172.21.15.26:0/928204068 wait complete. 2024-03-06T07:21:04.360 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-06T07:21:03.042861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":3168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":3244032,"data_stored":2483024,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.058153"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999702+0000","last_change":"2024-03-06T07:20:59.999702+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999702+0000","last_undegraded":"2024-03-06T07:20:59.999702+0000","last_fullsized":"2024-03-06T07:20:59.999702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998904+0000","last_change":"2024-03-06T07:20:59.998904+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998904+0000","last_undegraded":"2024-03-06T07:20:59.998904+0000","last_fullsized":"2024-03-06T07:20:59.998904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003003+0000","last_change":"2024-03-06T07:21:00.003003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003003+0000","last_undegraded":"2024-03-06T07:21:00.003003+0000","last_fullsized":"2024-03-06T07:21:00.003003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997726+0000","last_change":"2024-03-06T07:20:59.997726+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997726+0000","last_undegraded":"2024-03-06T07:20:59.997726+0000","last_fullsized":"2024-03-06T07:20:59.997726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996704+0000","last_change":"2024-03-06T07:20:59.996704+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996704+0000","last_undegraded":"2024-03-06T07:20:59.996704+0000","last_fullsized":"2024-03-06T07:20:59.996704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001349+0000","last_change":"2024-03-06T07:21:00.001349+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001349+0000","last_undegraded":"2024-03-06T07:21:00.001349+0000","last_fullsized":"2024-03-06T07:21:00.001349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001027+0000","last_change":"2024-03-06T07:21:00.001027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001027+0000","last_undegraded":"2024-03-06T07:21:00.001027+0000","last_fullsized":"2024-03-06T07:21:00.001027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997302+0000","last_change":"2024-03-06T07:20:59.997302+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997302+0000","last_undegraded":"2024-03-06T07:20:59.997302+0000","last_fullsized":"2024-03-06T07:20:59.997302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003260+0000","last_change":"2024-03-06T07:21:00.003260+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003260+0000","last_undegraded":"2024-03-06T07:21:00.003260+0000","last_fullsized":"2024-03-06T07:21:00.003260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002815+0000","last_change":"2024-03-06T07:21:00.002815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002815+0000","last_undegraded":"2024-03-06T07:21:00.002815+0000","last_fullsized":"2024-03-06T07:21:00.002815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004749+0000","last_change":"2024-03-06T07:21:00.004749+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004749+0000","last_undegraded":"2024-03-06T07:21:00.004749+0000","last_fullsized":"2024-03-06T07:21:00.004749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003163+0000","last_change":"2024-03-06T07:21:00.003163+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003163+0000","last_undegraded":"2024-03-06T07:21:00.003163+0000","last_fullsized":"2024-03-06T07:21:00.003163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003644+0000","last_change":"2024-03-06T07:21:00.003644+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003644+0000","last_undegraded":"2024-03-06T07:21:00.003644+0000","last_fullsized":"2024-03-06T07:21:00.003644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001459+0000","last_change":"2024-03-06T07:21:00.001459+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001459+0000","last_undegraded":"2024-03-06T07:21:00.001459+0000","last_fullsized":"2024-03-06T07:21:00.001459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000599+0000","last_change":"2024-03-06T07:21:00.000599+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000599+0000","last_undegraded":"2024-03-06T07:21:00.000599+0000","last_fullsized":"2024-03-06T07:21:00.000599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998031+0000","last_change":"2024-03-06T07:20:59.998031+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998031+0000","last_undegraded":"2024-03-06T07:20:59.998031+0000","last_fullsized":"2024-03-06T07:20:59.998031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998032+0000","last_change":"2024-03-06T07:20:59.998032+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998032+0000","last_undegraded":"2024-03-06T07:20:59.998032+0000","last_fullsized":"2024-03-06T07:20:59.998032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003193+0000","last_change":"2024-03-06T07:21:00.003193+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003193+0000","last_undegraded":"2024-03-06T07:21:00.003193+0000","last_fullsized":"2024-03-06T07:21:00.003193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997357+0000","last_change":"2024-03-06T07:20:59.997357+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997357+0000","last_undegraded":"2024-03-06T07:20:59.997357+0000","last_fullsized":"2024-03-06T07:20:59.997357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003259+0000","last_change":"2024-03-06T07:21:00.003259+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003259+0000","last_undegraded":"2024-03-06T07:21:00.003259+0000","last_fullsized":"2024-03-06T07:21:00.003259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003247+0000","last_change":"2024-03-06T07:21:00.003247+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003247+0000","last_undegraded":"2024-03-06T07:21:00.003247+0000","last_fullsized":"2024-03-06T07:21:00.003247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000143+0000","last_change":"2024-03-06T07:21:00.000143+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000143+0000","last_undegraded":"2024-03-06T07:21:00.000143+0000","last_fullsized":"2024-03-06T07:21:00.000143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997546+0000","last_change":"2024-03-06T07:20:59.997546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997546+0000","last_undegraded":"2024-03-06T07:20:59.997546+0000","last_fullsized":"2024-03-06T07:20:59.997546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998183+0000","last_change":"2024-03-06T07:20:59.998183+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998183+0000","last_undegraded":"2024-03-06T07:20:59.998183+0000","last_fullsized":"2024-03-06T07:20:59.998183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003168+0000","last_change":"2024-03-06T07:21:00.003168+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003168+0000","last_undegraded":"2024-03-06T07:21:00.003168+0000","last_fullsized":"2024-03-06T07:21:00.003168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002177+0000","last_change":"2024-03-06T07:21:00.002177+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002177+0000","last_undegraded":"2024-03-06T07:21:00.002177+0000","last_fullsized":"2024-03-06T07:21:00.002177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003051+0000","last_change":"2024-03-06T07:21:00.003051+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003051+0000","last_undegraded":"2024-03-06T07:21:00.003051+0000","last_fullsized":"2024-03-06T07:21:00.003051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000938+0000","last_change":"2024-03-06T07:21:00.000938+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000938+0000","last_undegraded":"2024-03-06T07:21:00.000938+0000","last_fullsized":"2024-03-06T07:21:00.000938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002465+0000","last_change":"2024-03-06T07:21:00.002465+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002465+0000","last_undegraded":"2024-03-06T07:21:00.002465+0000","last_fullsized":"2024-03-06T07:21:00.002465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001989+0000","last_change":"2024-03-06T07:21:00.001989+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001989+0000","last_undegraded":"2024-03-06T07:21:00.001989+0000","last_fullsized":"2024-03-06T07:21:00.001989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997070+0000","last_change":"2024-03-06T07:20:59.997070+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997070+0000","last_undegraded":"2024-03-06T07:20:59.997070+0000","last_fullsized":"2024-03-06T07:20:59.997070+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002027+0000","last_change":"2024-03-06T07:21:00.002027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002027+0000","last_undegraded":"2024-03-06T07:21:00.002027+0000","last_fullsized":"2024-03-06T07:21:00.002027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997095+0000","last_change":"2024-03-06T07:20:59.997095+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997095+0000","last_undegraded":"2024-03-06T07:20:59.997095+0000","last_fullsized":"2024-03-06T07:20:59.997095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018235+0000","last_change":"2024-03-06T07:21:00.010567+0000","last_active":"2024-03-06T07:21:00.018235+0000","last_peered":"2024-03-06T07:21:00.018235+0000","last_clean":"2024-03-06T07:21:00.018235+0000","last_became_active":"2024-03-06T07:21:00.010277+0000","last_became_peered":"2024-03-06T07:21:00.010277+0000","last_unstale":"2024-03-06T07:21:00.018235+0000","last_undegraded":"2024-03-06T07:21:00.018235+0000","last_fullsized":"2024-03-06T07:21:00.018235+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.435165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001748+0000","last_change":"2024-03-06T07:21:00.001748+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001748+0000","last_undegraded":"2024-03-06T07:21:00.001748+0000","last_fullsized":"2024-03-06T07:21:00.001748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.013052+0000","last_change":"2024-03-06T07:21:00.011675+0000","last_active":"2024-03-06T07:21:00.013052+0000","last_peered":"2024-03-06T07:21:00.013052+0000","last_clean":"2024-03-06T07:21:00.013052+0000","last_became_active":"2024-03-06T07:21:00.011344+0000","last_became_peered":"2024-03-06T07:21:00.011344+0000","last_unstale":"2024-03-06T07:21:00.013052+0000","last_undegraded":"2024-03-06T07:21:00.013052+0000","last_fullsized":"2024-03-06T07:21:00.013052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:04.618247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000216+0000","last_change":"2024-03-06T07:21:00.000216+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000216+0000","last_undegraded":"2024-03-06T07:21:00.000216+0000","last_fullsized":"2024-03-06T07:21:00.000216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016876+0000","last_change":"2024-03-06T07:21:00.012691+0000","last_active":"2024-03-06T07:21:00.016876+0000","last_peered":"2024-03-06T07:21:00.016876+0000","last_clean":"2024-03-06T07:21:00.016876+0000","last_became_active":"2024-03-06T07:21:00.012404+0000","last_became_peered":"2024-03-06T07:21:00.012404+0000","last_unstale":"2024-03-06T07:21:00.016876+0000","last_undegraded":"2024-03-06T07:21:00.016876+0000","last_fullsized":"2024-03-06T07:21:00.016876+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000111+0000","last_change":"2024-03-06T07:21:00.000111+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000111+0000","last_undegraded":"2024-03-06T07:21:00.000111+0000","last_fullsized":"2024-03-06T07:21:00.000111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018440+0000","last_change":"2024-03-06T07:21:00.011238+0000","last_active":"2024-03-06T07:21:00.018440+0000","last_peered":"2024-03-06T07:21:00.018440+0000","last_clean":"2024-03-06T07:21:00.018440+0000","last_became_active":"2024-03-06T07:21:00.010202+0000","last_became_peered":"2024-03-06T07:21:00.010202+0000","last_unstale":"2024-03-06T07:21:00.018440+0000","last_undegraded":"2024-03-06T07:21:00.018440+0000","last_fullsized":"2024-03-06T07:21:00.018440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:23.626033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.010108+0000","last_change":"2024-03-06T07:21:00.010108+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.010108+0000","last_undegraded":"2024-03-06T07:21:00.010108+0000","last_fullsized":"2024-03-06T07:21:00.010108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018243+0000","last_change":"2024-03-06T07:21:00.008004+0000","last_active":"2024-03-06T07:21:00.018243+0000","last_peered":"2024-03-06T07:21:00.018243+0000","last_clean":"2024-03-06T07:21:00.018243+0000","last_became_active":"2024-03-06T07:21:00.007716+0000","last_became_peered":"2024-03-06T07:21:00.007716+0000","last_unstale":"2024-03-06T07:21:00.018243+0000","last_undegraded":"2024-03-06T07:21:00.018243+0000","last_fullsized":"2024-03-06T07:21:00.018243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000136+0000","last_change":"2024-03-06T07:21:00.000136+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000136+0000","last_undegraded":"2024-03-06T07:21:00.000136+0000","last_fullsized":"2024-03-06T07:21:00.000136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010724+0000","last_change":"2024-03-06T07:21:00.006417+0000","last_active":"2024-03-06T07:21:00.010724+0000","last_peered":"2024-03-06T07:21:00.010724+0000","last_clean":"2024-03-06T07:21:00.010724+0000","last_became_active":"2024-03-06T07:21:00.006164+0000","last_became_peered":"2024-03-06T07:21:00.006164+0000","last_unstale":"2024-03-06T07:21:00.010724+0000","last_undegraded":"2024-03-06T07:21:00.010724+0000","last_fullsized":"2024-03-06T07:21:00.010724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:32.480280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000567+0000","last_change":"2024-03-06T07:21:00.000567+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000567+0000","last_undegraded":"2024-03-06T07:21:00.000567+0000","last_fullsized":"2024-03-06T07:21:00.000567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.019321+0000","last_change":"2024-03-06T07:21:00.018959+0000","last_active":"2024-03-06T07:21:00.019321+0000","last_peered":"2024-03-06T07:21:00.019321+0000","last_clean":"2024-03-06T07:21:00.019321+0000","last_became_active":"2024-03-06T07:21:00.018641+0000","last_became_peered":"2024-03-06T07:21:00.018641+0000","last_unstale":"2024-03-06T07:21:00.019321+0000","last_undegraded":"2024-03-06T07:21:00.019321+0000","last_fullsized":"2024-03-06T07:21:00.019321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998815+0000","last_change":"2024-03-06T07:20:59.998815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998815+0000","last_undegraded":"2024-03-06T07:20:59.998815+0000","last_fullsized":"2024-03-06T07:20:59.998815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018484+0000","last_change":"2024-03-06T07:21:00.012306+0000","last_active":"2024-03-06T07:21:00.018484+0000","last_peered":"2024-03-06T07:21:00.018484+0000","last_clean":"2024-03-06T07:21:00.018484+0000","last_became_active":"2024-03-06T07:21:00.012030+0000","last_became_peered":"2024-03-06T07:21:00.012030+0000","last_unstale":"2024-03-06T07:21:00.018484+0000","last_undegraded":"2024-03-06T07:21:00.018484+0000","last_fullsized":"2024-03-06T07:21:00.018484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:34.501662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997276+0000","last_change":"2024-03-06T07:20:59.997276+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997276+0000","last_undegraded":"2024-03-06T07:20:59.997276+0000","last_fullsized":"2024-03-06T07:20:59.997276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017570+0000","last_change":"2024-03-06T07:21:00.015472+0000","last_active":"2024-03-06T07:21:00.017570+0000","last_peered":"2024-03-06T07:21:00.017570+0000","last_clean":"2024-03-06T07:21:00.017570+0000","last_became_active":"2024-03-06T07:21:00.015206+0000","last_became_peered":"2024-03-06T07:21:00.015206+0000","last_unstale":"2024-03-06T07:21:00.017570+0000","last_undegraded":"2024-03-06T07:21:00.017570+0000","last_fullsized":"2024-03-06T07:21:00.017570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:12.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996607+0000","last_change":"2024-03-06T07:20:59.996607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996607+0000","last_undegraded":"2024-03-06T07:20:59.996607+0000","last_fullsized":"2024-03-06T07:20:59.996607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018385+0000","last_change":"2024-03-06T07:21:00.011966+0000","last_active":"2024-03-06T07:21:00.018385+0000","last_peered":"2024-03-06T07:21:00.018385+0000","last_clean":"2024-03-06T07:21:00.018385+0000","last_became_active":"2024-03-06T07:21:00.011703+0000","last_became_peered":"2024-03-06T07:21:00.011703+0000","last_unstale":"2024-03-06T07:21:00.018385+0000","last_undegraded":"2024-03-06T07:21:00.018385+0000","last_fullsized":"2024-03-06T07:21:00.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:12.743167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004577+0000","last_change":"2024-03-06T07:21:00.004577+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004577+0000","last_undegraded":"2024-03-06T07:21:00.004577+0000","last_fullsized":"2024-03-06T07:21:00.004577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018708+0000","last_change":"2024-03-06T07:21:00.017403+0000","last_active":"2024-03-06T07:21:00.018708+0000","last_peered":"2024-03-06T07:21:00.018708+0000","last_clean":"2024-03-06T07:21:00.018708+0000","last_became_active":"2024-03-06T07:21:00.017171+0000","last_became_peered":"2024-03-06T07:21:00.017171+0000","last_unstale":"2024-03-06T07:21:00.018708+0000","last_undegraded":"2024-03-06T07:21:00.018708+0000","last_fullsized":"2024-03-06T07:21:00.018708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999477+0000","last_change":"2024-03-06T07:20:59.999477+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999477+0000","last_undegraded":"2024-03-06T07:20:59.999477+0000","last_fullsized":"2024-03-06T07:20:59.999477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011916+0000","last_change":"2024-03-06T07:21:00.007232+0000","last_active":"2024-03-06T07:21:00.011916+0000","last_peered":"2024-03-06T07:21:00.011916+0000","last_clean":"2024-03-06T07:21:00.011916+0000","last_became_active":"2024-03-06T07:21:00.006787+0000","last_became_peered":"2024-03-06T07:21:00.006787+0000","last_unstale":"2024-03-06T07:21:00.011916+0000","last_undegraded":"2024-03-06T07:21:00.011916+0000","last_fullsized":"2024-03-06T07:21:00.011916+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.009601+0000","last_change":"2024-03-06T07:21:00.009601+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.009601+0000","last_undegraded":"2024-03-06T07:21:00.009601+0000","last_fullsized":"2024-03-06T07:21:00.009601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017495+0000","last_change":"2024-03-06T07:21:00.014388+0000","last_active":"2024-03-06T07:21:00.017495+0000","last_peered":"2024-03-06T07:21:00.017495+0000","last_clean":"2024-03-06T07:21:00.017495+0000","last_became_active":"2024-03-06T07:21:00.014118+0000","last_became_peered":"2024-03-06T07:21:00.014118+0000","last_unstale":"2024-03-06T07:21:00.017495+0000","last_undegraded":"2024-03-06T07:21:00.017495+0000","last_fullsized":"2024-03-06T07:21:00.017495+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:16.342176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999388+0000","last_change":"2024-03-06T07:20:59.999388+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999388+0000","last_undegraded":"2024-03-06T07:20:59.999388+0000","last_fullsized":"2024-03-06T07:20:59.999388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017297+0000","last_change":"2024-03-06T07:21:00.014221+0000","last_active":"2024-03-06T07:21:00.017297+0000","last_peered":"2024-03-06T07:21:00.017297+0000","last_clean":"2024-03-06T07:21:00.017297+0000","last_became_active":"2024-03-06T07:21:00.013921+0000","last_became_peered":"2024-03-06T07:21:00.013921+0000","last_unstale":"2024-03-06T07:21:00.017297+0000","last_undegraded":"2024-03-06T07:21:00.017297+0000","last_fullsized":"2024-03-06T07:21:00.017297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996517+0000","last_change":"2024-03-06T07:20:59.996517+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996517+0000","last_undegraded":"2024-03-06T07:20:59.996517+0000","last_fullsized":"2024-03-06T07:20:59.996517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015389+0000","last_change":"2024-03-06T07:21:00.010506+0000","last_active":"2024-03-06T07:21:00.015389+0000","last_peered":"2024-03-06T07:21:00.015389+0000","last_clean":"2024-03-06T07:21:00.015389+0000","last_became_active":"2024-03-06T07:21:00.010172+0000","last_became_peered":"2024-03-06T07:21:00.010172+0000","last_unstale":"2024-03-06T07:21:00.015389+0000","last_undegraded":"2024-03-06T07:21:00.015389+0000","last_fullsized":"2024-03-06T07:21:00.015389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:46.431814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998304+0000","last_change":"2024-03-06T07:20:59.998304+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998304+0000","last_undegraded":"2024-03-06T07:20:59.998304+0000","last_fullsized":"2024-03-06T07:20:59.998304+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.009278+0000","last_change":"2024-03-06T07:21:00.005310+0000","last_active":"2024-03-06T07:21:00.009278+0000","last_peered":"2024-03-06T07:21:00.009278+0000","last_clean":"2024-03-06T07:21:00.009278+0000","last_became_active":"2024-03-06T07:21:00.004999+0000","last_became_peered":"2024-03-06T07:21:00.004999+0000","last_unstale":"2024-03-06T07:21:00.009278+0000","last_undegraded":"2024-03-06T07:21:00.009278+0000","last_fullsized":"2024-03-06T07:21:00.009278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999103+0000","last_change":"2024-03-06T07:20:59.999103+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999103+0000","last_undegraded":"2024-03-06T07:20:59.999103+0000","last_fullsized":"2024-03-06T07:20:59.999103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018327+0000","last_change":"2024-03-06T07:21:00.011969+0000","last_active":"2024-03-06T07:21:00.018327+0000","last_peered":"2024-03-06T07:21:00.018327+0000","last_clean":"2024-03-06T07:21:00.018327+0000","last_became_active":"2024-03-06T07:21:00.011741+0000","last_became_peered":"2024-03-06T07:21:00.011741+0000","last_unstale":"2024-03-06T07:21:00.018327+0000","last_undegraded":"2024-03-06T07:21:00.018327+0000","last_fullsized":"2024-03-06T07:21:00.018327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:12.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004368+0000","last_change":"2024-03-06T07:21:00.004368+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004368+0000","last_undegraded":"2024-03-06T07:21:00.004368+0000","last_fullsized":"2024-03-06T07:21:00.004368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018480+0000","last_change":"2024-03-06T07:21:00.013456+0000","last_active":"2024-03-06T07:21:00.018480+0000","last_peered":"2024-03-06T07:21:00.018480+0000","last_clean":"2024-03-06T07:21:00.018480+0000","last_became_active":"2024-03-06T07:21:00.013184+0000","last_became_peered":"2024-03-06T07:21:00.013184+0000","last_unstale":"2024-03-06T07:21:00.018480+0000","last_undegraded":"2024-03-06T07:21:00.018480+0000","last_fullsized":"2024-03-06T07:21:00.018480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000640+0000","last_change":"2024-03-06T07:21:00.000640+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000640+0000","last_undegraded":"2024-03-06T07:21:00.000640+0000","last_fullsized":"2024-03-06T07:21:00.000640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005305+0000","last_change":"2024-03-06T07:21:00.003577+0000","last_active":"2024-03-06T07:21:00.005305+0000","last_peered":"2024-03-06T07:21:00.005305+0000","last_clean":"2024-03-06T07:21:00.005305+0000","last_became_active":"2024-03-06T07:21:00.003294+0000","last_became_peered":"2024-03-06T07:21:00.003294+0000","last_unstale":"2024-03-06T07:21:00.005305+0000","last_undegraded":"2024-03-06T07:21:00.005305+0000","last_fullsized":"2024-03-06T07:21:00.005305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:02.580999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996923+0000","last_change":"2024-03-06T07:20:59.996923+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996923+0000","last_undegraded":"2024-03-06T07:20:59.996923+0000","last_fullsized":"2024-03-06T07:20:59.996923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016900+0000","last_change":"2024-03-06T07:21:00.012331+0000","last_active":"2024-03-06T07:21:00.016900+0000","last_peered":"2024-03-06T07:21:00.016900+0000","last_clean":"2024-03-06T07:21:00.016900+0000","last_became_active":"2024-03-06T07:21:00.011994+0000","last_became_peered":"2024-03-06T07:21:00.011994+0000","last_unstale":"2024-03-06T07:21:00.016900+0000","last_undegraded":"2024-03-06T07:21:00.016900+0000","last_fullsized":"2024-03-06T07:21:00.016900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996058+0000","last_change":"2024-03-06T07:20:59.996058+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996058+0000","last_undegraded":"2024-03-06T07:20:59.996058+0000","last_fullsized":"2024-03-06T07:20:59.996058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011233+0000","last_change":"2024-03-06T07:21:00.008859+0000","last_active":"2024-03-06T07:21:00.011233+0000","last_peered":"2024-03-06T07:21:00.011233+0000","last_clean":"2024-03-06T07:21:00.011233+0000","last_became_active":"2024-03-06T07:21:00.008565+0000","last_became_peered":"2024-03-06T07:21:00.008565+0000","last_unstale":"2024-03-06T07:21:00.011233+0000","last_undegraded":"2024-03-06T07:21:00.011233+0000","last_fullsized":"2024-03-06T07:21:00.011233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:03.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002934+0000","last_change":"2024-03-06T07:21:00.002934+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002934+0000","last_undegraded":"2024-03-06T07:21:00.002934+0000","last_fullsized":"2024-03-06T07:21:00.002934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010375+0000","last_change":"2024-03-06T07:21:00.007836+0000","last_active":"2024-03-06T07:21:00.010375+0000","last_peered":"2024-03-06T07:21:00.010375+0000","last_clean":"2024-03-06T07:21:00.010375+0000","last_became_active":"2024-03-06T07:21:00.006276+0000","last_became_peered":"2024-03-06T07:21:00.006276+0000","last_unstale":"2024-03-06T07:21:00.010375+0000","last_undegraded":"2024-03-06T07:21:00.010375+0000","last_fullsized":"2024-03-06T07:21:00.010375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:26.980322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999890+0000","last_change":"2024-03-06T07:20:59.999890+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999890+0000","last_undegraded":"2024-03-06T07:20:59.999890+0000","last_fullsized":"2024-03-06T07:20:59.999890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018160+0000","last_change":"2024-03-06T07:21:00.011278+0000","last_active":"2024-03-06T07:21:00.018160+0000","last_peered":"2024-03-06T07:21:00.018160+0000","last_clean":"2024-03-06T07:21:00.018160+0000","last_became_active":"2024-03-06T07:21:00.011018+0000","last_became_peered":"2024-03-06T07:21:00.011018+0000","last_unstale":"2024-03-06T07:21:00.018160+0000","last_undegraded":"2024-03-06T07:21:00.018160+0000","last_fullsized":"2024-03-06T07:21:00.018160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997087+0000","last_change":"2024-03-06T07:20:59.997087+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997087+0000","last_undegraded":"2024-03-06T07:20:59.997087+0000","last_fullsized":"2024-03-06T07:20:59.997087+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.014862+0000","last_change":"2024-03-06T07:21:00.013699+0000","last_active":"2024-03-06T07:21:00.014862+0000","last_peered":"2024-03-06T07:21:00.014862+0000","last_clean":"2024-03-06T07:21:00.014862+0000","last_became_active":"2024-03-06T07:21:00.013360+0000","last_became_peered":"2024-03-06T07:21:00.013360+0000","last_unstale":"2024-03-06T07:21:00.014862+0000","last_undegraded":"2024-03-06T07:21:00.014862+0000","last_fullsized":"2024-03-06T07:21:00.014862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:06.072955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001334+0000","last_change":"2024-03-06T07:21:00.001334+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001334+0000","last_undegraded":"2024-03-06T07:21:00.001334+0000","last_fullsized":"2024-03-06T07:21:00.001334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018669+0000","last_change":"2024-03-06T07:21:00.015827+0000","last_active":"2024-03-06T07:21:00.018669+0000","last_peered":"2024-03-06T07:21:00.018669+0000","last_clean":"2024-03-06T07:21:00.018669+0000","last_became_active":"2024-03-06T07:21:00.015565+0000","last_became_peered":"2024-03-06T07:21:00.015565+0000","last_unstale":"2024-03-06T07:21:00.018669+0000","last_undegraded":"2024-03-06T07:21:00.018669+0000","last_fullsized":"2024-03-06T07:21:00.018669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:44.990676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996060+0000","last_change":"2024-03-06T07:20:59.996060+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996060+0000","last_undegraded":"2024-03-06T07:20:59.996060+0000","last_fullsized":"2024-03-06T07:20:59.996060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011205+0000","last_change":"2024-03-06T07:21:00.009047+0000","last_active":"2024-03-06T07:21:00.011205+0000","last_peered":"2024-03-06T07:21:00.011205+0000","last_clean":"2024-03-06T07:21:00.011205+0000","last_became_active":"2024-03-06T07:21:00.008803+0000","last_became_peered":"2024-03-06T07:21:00.008803+0000","last_unstale":"2024-03-06T07:21:00.011205+0000","last_undegraded":"2024-03-06T07:21:00.011205+0000","last_fullsized":"2024-03-06T07:21:00.011205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:17.432587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998464+0000","last_change":"2024-03-06T07:20:59.998464+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998464+0000","last_undegraded":"2024-03-06T07:20:59.998464+0000","last_fullsized":"2024-03-06T07:20:59.998464+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016624+0000","last_change":"2024-03-06T07:21:00.015433+0000","last_active":"2024-03-06T07:21:00.016624+0000","last_peered":"2024-03-06T07:21:00.016624+0000","last_clean":"2024-03-06T07:21:00.016624+0000","last_became_active":"2024-03-06T07:21:00.015156+0000","last_became_peered":"2024-03-06T07:21:00.015156+0000","last_unstale":"2024-03-06T07:21:00.016624+0000","last_undegraded":"2024-03-06T07:21:00.016624+0000","last_fullsized":"2024-03-06T07:21:00.016624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.005409+0000","last_change":"2024-03-06T07:21:00.005409+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.005409+0000","last_undegraded":"2024-03-06T07:21:00.005409+0000","last_fullsized":"2024-03-06T07:21:00.005409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017074+0000","last_change":"2024-03-06T07:21:00.016544+0000","last_active":"2024-03-06T07:21:00.017074+0000","last_peered":"2024-03-06T07:21:00.017074+0000","last_clean":"2024-03-06T07:21:00.017074+0000","last_became_active":"2024-03-06T07:21:00.015894+0000","last_became_peered":"2024-03-06T07:21:00.015894+0000","last_unstale":"2024-03-06T07:21:00.017074+0000","last_undegraded":"2024-03-06T07:21:00.017074+0000","last_fullsized":"2024-03-06T07:21:00.017074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.008546+0000","last_change":"2024-03-06T07:21:00.008546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.008546+0000","last_undegraded":"2024-03-06T07:21:00.008546+0000","last_fullsized":"2024-03-06T07:21:00.008546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015354+0000","last_change":"2024-03-06T07:21:00.010211+0000","last_active":"2024-03-06T07:21:00.015354+0000","last_peered":"2024-03-06T07:21:00.015354+0000","last_clean":"2024-03-06T07:21:00.015354+0000","last_became_active":"2024-03-06T07:21:00.009915+0000","last_became_peered":"2024-03-06T07:21:00.009915+0000","last_unstale":"2024-03-06T07:21:00.015354+0000","last_undegraded":"2024-03-06T07:21:00.015354+0000","last_fullsized":"2024-03-06T07:21:00.015354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001003+0000","last_change":"2024-03-06T07:21:00.001003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001003+0000","last_undegraded":"2024-03-06T07:21:00.001003+0000","last_fullsized":"2024-03-06T07:21:00.001003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018391+0000","last_change":"2024-03-06T07:21:00.011094+0000","last_active":"2024-03-06T07:21:00.018391+0000","last_peered":"2024-03-06T07:21:00.018391+0000","last_clean":"2024-03-06T07:21:00.018391+0000","last_became_active":"2024-03-06T07:21:00.010457+0000","last_became_peered":"2024-03-06T07:21:00.010457+0000","last_unstale":"2024-03-06T07:21:00.018391+0000","last_undegraded":"2024-03-06T07:21:00.018391+0000","last_fullsized":"2024-03-06T07:21:00.018391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:41.550434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996256+0000","last_change":"2024-03-06T07:20:59.996256+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996256+0000","last_undegraded":"2024-03-06T07:20:59.996256+0000","last_fullsized":"2024-03-06T07:20:59.996256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017529+0000","last_change":"2024-03-06T07:21:00.014362+0000","last_active":"2024-03-06T07:21:00.017529+0000","last_peered":"2024-03-06T07:21:00.017529+0000","last_clean":"2024-03-06T07:21:00.017529+0000","last_became_active":"2024-03-06T07:21:00.014072+0000","last_became_peered":"2024-03-06T07:21:00.014072+0000","last_unstale":"2024-03-06T07:21:00.017529+0000","last_undegraded":"2024-03-06T07:21:00.017529+0000","last_fullsized":"2024-03-06T07:21:00.017529+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:12.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003607+0000","last_change":"2024-03-06T07:21:00.003607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003607+0000","last_undegraded":"2024-03-06T07:21:00.003607+0000","last_fullsized":"2024-03-06T07:21:00.003607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005867+0000","last_change":"2024-03-06T07:21:00.002103+0000","last_active":"2024-03-06T07:21:00.005867+0000","last_peered":"2024-03-06T07:21:00.005867+0000","last_clean":"2024-03-06T07:21:00.005867+0000","last_became_active":"2024-03-06T07:21:00.001791+0000","last_became_peered":"2024-03-06T07:21:00.001791+0000","last_unstale":"2024-03-06T07:21:00.005867+0000","last_undegraded":"2024-03-06T07:21:00.005867+0000","last_fullsized":"2024-03-06T07:21:00.005867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:20:59.995260+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:59.995260+0000","last_peered":"2024-03-06T07:20:59.995260+0000","last_clean":"2024-03-06T07:20:59.995260+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:59.995260+0000","last_undegraded":"2024-03-06T07:20:59.995260+0000","last_fullsized":"2024-03-06T07:20:59.995260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:04.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.498+0000 7efd2dd0c640 1 Processor -- start 2024-03-06T07:21:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd2dd0c640 1 -- start start 2024-03-06T07:21:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd2dd0c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2814f360 0x7efd28155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd2dd0c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280c73d0 con 0x7efd28158b60 2024-03-06T07:21:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd2dd0c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280be2e0 con 0x7efd280c88c0 2024-03-06T07:21:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd2dd0c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd28159700 con 0x7efd2814f360 2024-03-06T07:21:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd27fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd28158b60 0x7efd28158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57928/0 (socket says 172.21.15.26:57928) 2024-03-06T07:21:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd27fff640 1 -- 172.21.15.26:0/937529825 learned_addr learned my addr 172.21.15.26:0/937529825 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.499+0000 7efd277fe640 1 --2- 172.21.15.26:0/937529825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2814f360 0x7efd28155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.500+0000 7efd267fc640 1 -- 172.21.15.26:0/937529825 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918194622 0 0) 0x7efd280c73d0 con 0x7efd28158b60 2024-03-06T07:21:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.500+0000 7efd267fc640 1 -- 172.21.15.26:0/937529825 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd10003520 con 0x7efd28158b60 2024-03-06T07:21:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.500+0000 7efd267fc640 1 -- 172.21.15.26:0/937529825 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21843337 0 0) 0x7efd280be2e0 con 0x7efd280c88c0 2024-03-06T07:21:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.500+0000 7efd277fe640 1 -- 172.21.15.26:0/937529825 >> v1:172.21.15.117:6789/0 conn(0x7efd280c88c0 legacy=0x7efd280c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.500+0000 7efd277fe640 1 -- 172.21.15.26:0/937529825 >> v1:172.21.15.26:6789/0 conn(0x7efd28158b60 legacy=0x7efd28158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.500+0000 7efd277fe640 1 -- 172.21.15.26:0/937529825 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280c7670 con 0x7efd2814f360 2024-03-06T07:21:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd277fe640 1 --2- 172.21.15.26:0/937529825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2814f360 0x7efd28155c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd180097d0 tx=0x7efd1802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506c161c8e9052d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd267fc640 1 -- 172.21.15.26:0/937529825 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18035020 con 0x7efd2814f360 2024-03-06T07:21:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd267fc640 1 -- 172.21.15.26:0/937529825 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18031800 con 0x7efd2814f360 2024-03-06T07:21:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd267fc640 1 -- 172.21.15.26:0/937529825 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18031bd0 con 0x7efd2814f360 2024-03-06T07:21:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd2dd0c640 1 -- 172.21.15.26:0/937529825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2814f360 msgr2=0x7efd28155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/937529825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2814f360 0x7efd28155c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd180097d0 tx=0x7efd1802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd2dd0c640 1 -- 172.21.15.26:0/937529825 shutdown_connections 2024-03-06T07:21:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/937529825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2814f360 0x7efd28155c10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd2dd0c640 1 -- 172.21.15.26:0/937529825 >> 172.21.15.26:0/937529825 conn(0x7efd280ac570 msgr2=0x7efd280ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd2dd0c640 1 -- 172.21.15.26:0/937529825 shutdown_connections 2024-03-06T07:21:04.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.501+0000 7efd2dd0c640 1 -- 172.21.15.26:0/937529825 wait complete. 2024-03-06T07:21:04.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.502+0000 7efd2dd0c640 1 Processor -- start 2024-03-06T07:21:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd2dd0c640 1 -- start start 2024-03-06T07:21:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd2dd0c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd280c88c0 0x7efd280c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd2dd0c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd2814f360 0x7efd2814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd2dd0c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd28158b60 0x7efd2814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd2dd0c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280c7c30 con 0x7efd28158b60 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd2dd0c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280c5a50 con 0x7efd2814f360 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd2dd0c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd280c5bd0 con 0x7efd280c88c0 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd28158b60 0x7efd2814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd28158b60 0x7efd2814c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44474/0 (socket says 172.21.15.26:44474) 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.503+0000 7efd27fff640 1 -- 172.21.15.26:0/471935626 learned_addr learned my addr 172.21.15.26:0/471935626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd277fe640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd280c88c0 0x7efd280c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd26ffd640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd2814f360 0x7efd2814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd27fff640 1 -- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd280c88c0 msgr2=0x7efd280c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd27fff640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd280c88c0 0x7efd280c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd27fff640 1 -- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd2814f360 msgr2=0x7efd2814bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd27fff640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd2814f360 0x7efd2814bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd27fff640 1 -- 172.21.15.26:0/471935626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd281ec240 con 0x7efd28158b60 2024-03-06T07:21:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd27fff640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd28158b60 0x7efd2814c1b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efd2000cb40 tx=0x7efd2000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd0ffff640 1 -- 172.21.15.26:0/471935626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20017070 con 0x7efd28158b60 2024-03-06T07:21:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd0ffff640 1 -- 172.21.15.26:0/471935626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2000aca0 con 0x7efd28158b60 2024-03-06T07:21:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd0ffff640 1 -- 172.21.15.26:0/471935626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20012620 con 0x7efd28158b60 2024-03-06T07:21:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd280c2f00 con 0x7efd28158b60 2024-03-06T07:21:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.504+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd281ec4e0 con 0x7efd28158b60 2024-03-06T07:21:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.505+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2804f330 con 0x7efd28158b60 2024-03-06T07:21:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.506+0000 7efd0ffff640 1 -- 172.21.15.26:0/471935626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd2001a070 con 0x7efd28158b60 2024-03-06T07:21:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.506+0000 7efd0ffff640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.506+0000 7efd0ffff640 1 -- 172.21.15.26:0/471935626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd2008a0e0 con 0x7efd28158b60 2024-03-06T07:21:04.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.506+0000 7efd277fe640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.507+0000 7efd277fe640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efd18036000 tx=0x7efd18008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.510+0000 7efd0ffff640 1 -- 172.21.15.26:0/471935626 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd20058810 con 0x7efd28158b60 2024-03-06T07:21:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.689+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd280c8d50 con 0x7efd1406b4d0 2024-03-06T07:21:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.695+0000 7efd0ffff640 1 -- 172.21.15.26:0/471935626 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246709 (secure 0 0 0) 0x7efd280c8d50 con 0x7efd1406b4d0 2024-03-06T07:21:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd1406b4d0 msgr2=0x7efd1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efd18036000 tx=0x7efd18008030 comp rx=0 tx=0).stop 2024-03-06T07:21:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd28158b60 msgr2=0x7efd2814c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd28158b60 0x7efd2814c1b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efd2000cb40 tx=0x7efd2000a060 comp rx=0 tx=0).stop 2024-03-06T07:21:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 shutdown_connections 2024-03-06T07:21:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd28158b60 0x7efd2814c1b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd2814f360 0x7efd2814bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 --2- 172.21.15.26:0/471935626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd280c88c0 0x7efd280c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 >> 172.21.15.26:0/471935626 conn(0x7efd280ac570 msgr2=0x7efd28154240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 shutdown_connections 2024-03-06T07:21:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.698+0000 7efd2dd0c640 1 -- 172.21.15.26:0/471935626 wait complete. 2024-03-06T07:21:04.711 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-06T07:21:03.042861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":3168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":3244032,"data_stored":2483024,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.058153"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999702+0000","last_change":"2024-03-06T07:20:59.999702+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999702+0000","last_undegraded":"2024-03-06T07:20:59.999702+0000","last_fullsized":"2024-03-06T07:20:59.999702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998904+0000","last_change":"2024-03-06T07:20:59.998904+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998904+0000","last_undegraded":"2024-03-06T07:20:59.998904+0000","last_fullsized":"2024-03-06T07:20:59.998904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003003+0000","last_change":"2024-03-06T07:21:00.003003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003003+0000","last_undegraded":"2024-03-06T07:21:00.003003+0000","last_fullsized":"2024-03-06T07:21:00.003003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997726+0000","last_change":"2024-03-06T07:20:59.997726+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997726+0000","last_undegraded":"2024-03-06T07:20:59.997726+0000","last_fullsized":"2024-03-06T07:20:59.997726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996704+0000","last_change":"2024-03-06T07:20:59.996704+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996704+0000","last_undegraded":"2024-03-06T07:20:59.996704+0000","last_fullsized":"2024-03-06T07:20:59.996704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001349+0000","last_change":"2024-03-06T07:21:00.001349+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001349+0000","last_undegraded":"2024-03-06T07:21:00.001349+0000","last_fullsized":"2024-03-06T07:21:00.001349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001027+0000","last_change":"2024-03-06T07:21:00.001027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001027+0000","last_undegraded":"2024-03-06T07:21:00.001027+0000","last_fullsized":"2024-03-06T07:21:00.001027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997302+0000","last_change":"2024-03-06T07:20:59.997302+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997302+0000","last_undegraded":"2024-03-06T07:20:59.997302+0000","last_fullsized":"2024-03-06T07:20:59.997302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003260+0000","last_change":"2024-03-06T07:21:00.003260+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003260+0000","last_undegraded":"2024-03-06T07:21:00.003260+0000","last_fullsized":"2024-03-06T07:21:00.003260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002815+0000","last_change":"2024-03-06T07:21:00.002815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002815+0000","last_undegraded":"2024-03-06T07:21:00.002815+0000","last_fullsized":"2024-03-06T07:21:00.002815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004749+0000","last_change":"2024-03-06T07:21:00.004749+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004749+0000","last_undegraded":"2024-03-06T07:21:00.004749+0000","last_fullsized":"2024-03-06T07:21:00.004749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003163+0000","last_change":"2024-03-06T07:21:00.003163+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003163+0000","last_undegraded":"2024-03-06T07:21:00.003163+0000","last_fullsized":"2024-03-06T07:21:00.003163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003644+0000","last_change":"2024-03-06T07:21:00.003644+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003644+0000","last_undegraded":"2024-03-06T07:21:00.003644+0000","last_fullsized":"2024-03-06T07:21:00.003644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001459+0000","last_change":"2024-03-06T07:21:00.001459+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001459+0000","last_undegraded":"2024-03-06T07:21:00.001459+0000","last_fullsized":"2024-03-06T07:21:00.001459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000599+0000","last_change":"2024-03-06T07:21:00.000599+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000599+0000","last_undegraded":"2024-03-06T07:21:00.000599+0000","last_fullsized":"2024-03-06T07:21:00.000599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998031+0000","last_change":"2024-03-06T07:20:59.998031+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998031+0000","last_undegraded":"2024-03-06T07:20:59.998031+0000","last_fullsized":"2024-03-06T07:20:59.998031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998032+0000","last_change":"2024-03-06T07:20:59.998032+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998032+0000","last_undegraded":"2024-03-06T07:20:59.998032+0000","last_fullsized":"2024-03-06T07:20:59.998032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003193+0000","last_change":"2024-03-06T07:21:00.003193+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003193+0000","last_undegraded":"2024-03-06T07:21:00.003193+0000","last_fullsized":"2024-03-06T07:21:00.003193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997357+0000","last_change":"2024-03-06T07:20:59.997357+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997357+0000","last_undegraded":"2024-03-06T07:20:59.997357+0000","last_fullsized":"2024-03-06T07:20:59.997357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003259+0000","last_change":"2024-03-06T07:21:00.003259+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003259+0000","last_undegraded":"2024-03-06T07:21:00.003259+0000","last_fullsized":"2024-03-06T07:21:00.003259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003247+0000","last_change":"2024-03-06T07:21:00.003247+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003247+0000","last_undegraded":"2024-03-06T07:21:00.003247+0000","last_fullsized":"2024-03-06T07:21:00.003247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000143+0000","last_change":"2024-03-06T07:21:00.000143+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000143+0000","last_undegraded":"2024-03-06T07:21:00.000143+0000","last_fullsized":"2024-03-06T07:21:00.000143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997546+0000","last_change":"2024-03-06T07:20:59.997546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997546+0000","last_undegraded":"2024-03-06T07:20:59.997546+0000","last_fullsized":"2024-03-06T07:20:59.997546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998183+0000","last_change":"2024-03-06T07:20:59.998183+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998183+0000","last_undegraded":"2024-03-06T07:20:59.998183+0000","last_fullsized":"2024-03-06T07:20:59.998183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003168+0000","last_change":"2024-03-06T07:21:00.003168+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003168+0000","last_undegraded":"2024-03-06T07:21:00.003168+0000","last_fullsized":"2024-03-06T07:21:00.003168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002177+0000","last_change":"2024-03-06T07:21:00.002177+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002177+0000","last_undegraded":"2024-03-06T07:21:00.002177+0000","last_fullsized":"2024-03-06T07:21:00.002177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003051+0000","last_change":"2024-03-06T07:21:00.003051+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003051+0000","last_undegraded":"2024-03-06T07:21:00.003051+0000","last_fullsized":"2024-03-06T07:21:00.003051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000938+0000","last_change":"2024-03-06T07:21:00.000938+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000938+0000","last_undegraded":"2024-03-06T07:21:00.000938+0000","last_fullsized":"2024-03-06T07:21:00.000938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002465+0000","last_change":"2024-03-06T07:21:00.002465+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002465+0000","last_undegraded":"2024-03-06T07:21:00.002465+0000","last_fullsized":"2024-03-06T07:21:00.002465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001989+0000","last_change":"2024-03-06T07:21:00.001989+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001989+0000","last_undegraded":"2024-03-06T07:21:00.001989+0000","last_fullsized":"2024-03-06T07:21:00.001989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997070+0000","last_change":"2024-03-06T07:20:59.997070+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997070+0000","last_undegraded":"2024-03-06T07:20:59.997070+0000","last_fullsized":"2024-03-06T07:20:59.997070+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002027+0000","last_change":"2024-03-06T07:21:00.002027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002027+0000","last_undegraded":"2024-03-06T07:21:00.002027+0000","last_fullsized":"2024-03-06T07:21:00.002027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997095+0000","last_change":"2024-03-06T07:20:59.997095+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997095+0000","last_undegraded":"2024-03-06T07:20:59.997095+0000","last_fullsized":"2024-03-06T07:20:59.997095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018235+0000","last_change":"2024-03-06T07:21:00.010567+0000","last_active":"2024-03-06T07:21:00.018235+0000","last_peered":"2024-03-06T07:21:00.018235+0000","last_clean":"2024-03-06T07:21:00.018235+0000","last_became_active":"2024-03-06T07:21:00.010277+0000","last_became_peered":"2024-03-06T07:21:00.010277+0000","last_unstale":"2024-03-06T07:21:00.018235+0000","last_undegraded":"2024-03-06T07:21:00.018235+0000","last_fullsized":"2024-03-06T07:21:00.018235+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.435165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001748+0000","last_change":"2024-03-06T07:21:00.001748+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001748+0000","last_undegraded":"2024-03-06T07:21:00.001748+0000","last_fullsized":"2024-03-06T07:21:00.001748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.013052+0000","last_change":"2024-03-06T07:21:00.011675+0000","last_active":"2024-03-06T07:21:00.013052+0000","last_peered":"2024-03-06T07:21:00.013052+0000","last_clean":"2024-03-06T07:21:00.013052+0000","last_became_active":"2024-03-06T07:21:00.011344+0000","last_became_peered":"2024-03-06T07:21:00.011344+0000","last_unstale":"2024-03-06T07:21:00.013052+0000","last_undegraded":"2024-03-06T07:21:00.013052+0000","last_fullsized":"2024-03-06T07:21:00.013052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:04.618247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000216+0000","last_change":"2024-03-06T07:21:00.000216+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000216+0000","last_undegraded":"2024-03-06T07:21:00.000216+0000","last_fullsized":"2024-03-06T07:21:00.000216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016876+0000","last_change":"2024-03-06T07:21:00.012691+0000","last_active":"2024-03-06T07:21:00.016876+0000","last_peered":"2024-03-06T07:21:00.016876+0000","last_clean":"2024-03-06T07:21:00.016876+0000","last_became_active":"2024-03-06T07:21:00.012404+0000","last_became_peered":"2024-03-06T07:21:00.012404+0000","last_unstale":"2024-03-06T07:21:00.016876+0000","last_undegraded":"2024-03-06T07:21:00.016876+0000","last_fullsized":"2024-03-06T07:21:00.016876+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000111+0000","last_change":"2024-03-06T07:21:00.000111+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000111+0000","last_undegraded":"2024-03-06T07:21:00.000111+0000","last_fullsized":"2024-03-06T07:21:00.000111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018440+0000","last_change":"2024-03-06T07:21:00.011238+0000","last_active":"2024-03-06T07:21:00.018440+0000","last_peered":"2024-03-06T07:21:00.018440+0000","last_clean":"2024-03-06T07:21:00.018440+0000","last_became_active":"2024-03-06T07:21:00.010202+0000","last_became_peered":"2024-03-06T07:21:00.010202+0000","last_unstale":"2024-03-06T07:21:00.018440+0000","last_undegraded":"2024-03-06T07:21:00.018440+0000","last_fullsized":"2024-03-06T07:21:00.018440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:23.626033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.010108+0000","last_change":"2024-03-06T07:21:00.010108+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.010108+0000","last_undegraded":"2024-03-06T07:21:00.010108+0000","last_fullsized":"2024-03-06T07:21:00.010108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018243+0000","last_change":"2024-03-06T07:21:00.008004+0000","last_active":"2024-03-06T07:21:00.018243+0000","last_peered":"2024-03-06T07:21:00.018243+0000","last_clean":"2024-03-06T07:21:00.018243+0000","last_became_active":"2024-03-06T07:21:00.007716+0000","last_became_peered":"2024-03-06T07:21:00.007716+0000","last_unstale":"2024-03-06T07:21:00.018243+0000","last_undegraded":"2024-03-06T07:21:00.018243+0000","last_fullsized":"2024-03-06T07:21:00.018243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000136+0000","last_change":"2024-03-06T07:21:00.000136+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000136+0000","last_undegraded":"2024-03-06T07:21:00.000136+0000","last_fullsized":"2024-03-06T07:21:00.000136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010724+0000","last_change":"2024-03-06T07:21:00.006417+0000","last_active":"2024-03-06T07:21:00.010724+0000","last_peered":"2024-03-06T07:21:00.010724+0000","last_clean":"2024-03-06T07:21:00.010724+0000","last_became_active":"2024-03-06T07:21:00.006164+0000","last_became_peered":"2024-03-06T07:21:00.006164+0000","last_unstale":"2024-03-06T07:21:00.010724+0000","last_undegraded":"2024-03-06T07:21:00.010724+0000","last_fullsized":"2024-03-06T07:21:00.010724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:32.480280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000567+0000","last_change":"2024-03-06T07:21:00.000567+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000567+0000","last_undegraded":"2024-03-06T07:21:00.000567+0000","last_fullsized":"2024-03-06T07:21:00.000567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.019321+0000","last_change":"2024-03-06T07:21:00.018959+0000","last_active":"2024-03-06T07:21:00.019321+0000","last_peered":"2024-03-06T07:21:00.019321+0000","last_clean":"2024-03-06T07:21:00.019321+0000","last_became_active":"2024-03-06T07:21:00.018641+0000","last_became_peered":"2024-03-06T07:21:00.018641+0000","last_unstale":"2024-03-06T07:21:00.019321+0000","last_undegraded":"2024-03-06T07:21:00.019321+0000","last_fullsized":"2024-03-06T07:21:00.019321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998815+0000","last_change":"2024-03-06T07:20:59.998815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998815+0000","last_undegraded":"2024-03-06T07:20:59.998815+0000","last_fullsized":"2024-03-06T07:20:59.998815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018484+0000","last_change":"2024-03-06T07:21:00.012306+0000","last_active":"2024-03-06T07:21:00.018484+0000","last_peered":"2024-03-06T07:21:00.018484+0000","last_clean":"2024-03-06T07:21:00.018484+0000","last_became_active":"2024-03-06T07:21:00.012030+0000","last_became_peered":"2024-03-06T07:21:00.012030+0000","last_unstale":"2024-03-06T07:21:00.018484+0000","last_undegraded":"2024-03-06T07:21:00.018484+0000","last_fullsized":"2024-03-06T07:21:00.018484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:34.501662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997276+0000","last_change":"2024-03-06T07:20:59.997276+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997276+0000","last_undegraded":"2024-03-06T07:20:59.997276+0000","last_fullsized":"2024-03-06T07:20:59.997276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017570+0000","last_change":"2024-03-06T07:21:00.015472+0000","last_active":"2024-03-06T07:21:00.017570+0000","last_peered":"2024-03-06T07:21:00.017570+0000","last_clean":"2024-03-06T07:21:00.017570+0000","last_became_active":"2024-03-06T07:21:00.015206+0000","last_became_peered":"2024-03-06T07:21:00.015206+0000","last_unstale":"2024-03-06T07:21:00.017570+0000","last_undegraded":"2024-03-06T07:21:00.017570+0000","last_fullsized":"2024-03-06T07:21:00.017570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:12.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996607+0000","last_change":"2024-03-06T07:20:59.996607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996607+0000","last_undegraded":"2024-03-06T07:20:59.996607+0000","last_fullsized":"2024-03-06T07:20:59.996607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018385+0000","last_change":"2024-03-06T07:21:00.011966+0000","last_active":"2024-03-06T07:21:00.018385+0000","last_peered":"2024-03-06T07:21:00.018385+0000","last_clean":"2024-03-06T07:21:00.018385+0000","last_became_active":"2024-03-06T07:21:00.011703+0000","last_became_peered":"2024-03-06T07:21:00.011703+0000","last_unstale":"2024-03-06T07:21:00.018385+0000","last_undegraded":"2024-03-06T07:21:00.018385+0000","last_fullsized":"2024-03-06T07:21:00.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:12.743167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004577+0000","last_change":"2024-03-06T07:21:00.004577+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004577+0000","last_undegraded":"2024-03-06T07:21:00.004577+0000","last_fullsized":"2024-03-06T07:21:00.004577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018708+0000","last_change":"2024-03-06T07:21:00.017403+0000","last_active":"2024-03-06T07:21:00.018708+0000","last_peered":"2024-03-06T07:21:00.018708+0000","last_clean":"2024-03-06T07:21:00.018708+0000","last_became_active":"2024-03-06T07:21:00.017171+0000","last_became_peered":"2024-03-06T07:21:00.017171+0000","last_unstale":"2024-03-06T07:21:00.018708+0000","last_undegraded":"2024-03-06T07:21:00.018708+0000","last_fullsized":"2024-03-06T07:21:00.018708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999477+0000","last_change":"2024-03-06T07:20:59.999477+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999477+0000","last_undegraded":"2024-03-06T07:20:59.999477+0000","last_fullsized":"2024-03-06T07:20:59.999477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011916+0000","last_change":"2024-03-06T07:21:00.007232+0000","last_active":"2024-03-06T07:21:00.011916+0000","last_peered":"2024-03-06T07:21:00.011916+0000","last_clean":"2024-03-06T07:21:00.011916+0000","last_became_active":"2024-03-06T07:21:00.006787+0000","last_became_peered":"2024-03-06T07:21:00.006787+0000","last_unstale":"2024-03-06T07:21:00.011916+0000","last_undegraded":"2024-03-06T07:21:00.011916+0000","last_fullsized":"2024-03-06T07:21:00.011916+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.009601+0000","last_change":"2024-03-06T07:21:00.009601+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.009601+0000","last_undegraded":"2024-03-06T07:21:00.009601+0000","last_fullsized":"2024-03-06T07:21:00.009601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017495+0000","last_change":"2024-03-06T07:21:00.014388+0000","last_active":"2024-03-06T07:21:00.017495+0000","last_peered":"2024-03-06T07:21:00.017495+0000","last_clean":"2024-03-06T07:21:00.017495+0000","last_became_active":"2024-03-06T07:21:00.014118+0000","last_became_peered":"2024-03-06T07:21:00.014118+0000","last_unstale":"2024-03-06T07:21:00.017495+0000","last_undegraded":"2024-03-06T07:21:00.017495+0000","last_fullsized":"2024-03-06T07:21:00.017495+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:16.342176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999388+0000","last_change":"2024-03-06T07:20:59.999388+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999388+0000","last_undegraded":"2024-03-06T07:20:59.999388+0000","last_fullsized":"2024-03-06T07:20:59.999388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017297+0000","last_change":"2024-03-06T07:21:00.014221+0000","last_active":"2024-03-06T07:21:00.017297+0000","last_peered":"2024-03-06T07:21:00.017297+0000","last_clean":"2024-03-06T07:21:00.017297+0000","last_became_active":"2024-03-06T07:21:00.013921+0000","last_became_peered":"2024-03-06T07:21:00.013921+0000","last_unstale":"2024-03-06T07:21:00.017297+0000","last_undegraded":"2024-03-06T07:21:00.017297+0000","last_fullsized":"2024-03-06T07:21:00.017297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996517+0000","last_change":"2024-03-06T07:20:59.996517+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996517+0000","last_undegraded":"2024-03-06T07:20:59.996517+0000","last_fullsized":"2024-03-06T07:20:59.996517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015389+0000","last_change":"2024-03-06T07:21:00.010506+0000","last_active":"2024-03-06T07:21:00.015389+0000","last_peered":"2024-03-06T07:21:00.015389+0000","last_clean":"2024-03-06T07:21:00.015389+0000","last_became_active":"2024-03-06T07:21:00.010172+0000","last_became_peered":"2024-03-06T07:21:00.010172+0000","last_unstale":"2024-03-06T07:21:00.015389+0000","last_undegraded":"2024-03-06T07:21:00.015389+0000","last_fullsized":"2024-03-06T07:21:00.015389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:46.431814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998304+0000","last_change":"2024-03-06T07:20:59.998304+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998304+0000","last_undegraded":"2024-03-06T07:20:59.998304+0000","last_fullsized":"2024-03-06T07:20:59.998304+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.009278+0000","last_change":"2024-03-06T07:21:00.005310+0000","last_active":"2024-03-06T07:21:00.009278+0000","last_peered":"2024-03-06T07:21:00.009278+0000","last_clean":"2024-03-06T07:21:00.009278+0000","last_became_active":"2024-03-06T07:21:00.004999+0000","last_became_peered":"2024-03-06T07:21:00.004999+0000","last_unstale":"2024-03-06T07:21:00.009278+0000","last_undegraded":"2024-03-06T07:21:00.009278+0000","last_fullsized":"2024-03-06T07:21:00.009278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999103+0000","last_change":"2024-03-06T07:20:59.999103+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999103+0000","last_undegraded":"2024-03-06T07:20:59.999103+0000","last_fullsized":"2024-03-06T07:20:59.999103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018327+0000","last_change":"2024-03-06T07:21:00.011969+0000","last_active":"2024-03-06T07:21:00.018327+0000","last_peered":"2024-03-06T07:21:00.018327+0000","last_clean":"2024-03-06T07:21:00.018327+0000","last_became_active":"2024-03-06T07:21:00.011741+0000","last_became_peered":"2024-03-06T07:21:00.011741+0000","last_unstale":"2024-03-06T07:21:00.018327+0000","last_undegraded":"2024-03-06T07:21:00.018327+0000","last_fullsized":"2024-03-06T07:21:00.018327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:12.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004368+0000","last_change":"2024-03-06T07:21:00.004368+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004368+0000","last_undegraded":"2024-03-06T07:21:00.004368+0000","last_fullsized":"2024-03-06T07:21:00.004368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018480+0000","last_change":"2024-03-06T07:21:00.013456+0000","last_active":"2024-03-06T07:21:00.018480+0000","last_peered":"2024-03-06T07:21:00.018480+0000","last_clean":"2024-03-06T07:21:00.018480+0000","last_became_active":"2024-03-06T07:21:00.013184+0000","last_became_peered":"2024-03-06T07:21:00.013184+0000","last_unstale":"2024-03-06T07:21:00.018480+0000","last_undegraded":"2024-03-06T07:21:00.018480+0000","last_fullsized":"2024-03-06T07:21:00.018480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000640+0000","last_change":"2024-03-06T07:21:00.000640+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000640+0000","last_undegraded":"2024-03-06T07:21:00.000640+0000","last_fullsized":"2024-03-06T07:21:00.000640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005305+0000","last_change":"2024-03-06T07:21:00.003577+0000","last_active":"2024-03-06T07:21:00.005305+0000","last_peered":"2024-03-06T07:21:00.005305+0000","last_clean":"2024-03-06T07:21:00.005305+0000","last_became_active":"2024-03-06T07:21:00.003294+0000","last_became_peered":"2024-03-06T07:21:00.003294+0000","last_unstale":"2024-03-06T07:21:00.005305+0000","last_undegraded":"2024-03-06T07:21:00.005305+0000","last_fullsized":"2024-03-06T07:21:00.005305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:02.580999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996923+0000","last_change":"2024-03-06T07:20:59.996923+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996923+0000","last_undegraded":"2024-03-06T07:20:59.996923+0000","last_fullsized":"2024-03-06T07:20:59.996923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016900+0000","last_change":"2024-03-06T07:21:00.012331+0000","last_active":"2024-03-06T07:21:00.016900+0000","last_peered":"2024-03-06T07:21:00.016900+0000","last_clean":"2024-03-06T07:21:00.016900+0000","last_became_active":"2024-03-06T07:21:00.011994+0000","last_became_peered":"2024-03-06T07:21:00.011994+0000","last_unstale":"2024-03-06T07:21:00.016900+0000","last_undegraded":"2024-03-06T07:21:00.016900+0000","last_fullsized":"2024-03-06T07:21:00.016900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996058+0000","last_change":"2024-03-06T07:20:59.996058+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996058+0000","last_undegraded":"2024-03-06T07:20:59.996058+0000","last_fullsized":"2024-03-06T07:20:59.996058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011233+0000","last_change":"2024-03-06T07:21:00.008859+0000","last_active":"2024-03-06T07:21:00.011233+0000","last_peered":"2024-03-06T07:21:00.011233+0000","last_clean":"2024-03-06T07:21:00.011233+0000","last_became_active":"2024-03-06T07:21:00.008565+0000","last_became_peered":"2024-03-06T07:21:00.008565+0000","last_unstale":"2024-03-06T07:21:00.011233+0000","last_undegraded":"2024-03-06T07:21:00.011233+0000","last_fullsized":"2024-03-06T07:21:00.011233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:03.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002934+0000","last_change":"2024-03-06T07:21:00.002934+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002934+0000","last_undegraded":"2024-03-06T07:21:00.002934+0000","last_fullsized":"2024-03-06T07:21:00.002934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010375+0000","last_change":"2024-03-06T07:21:00.007836+0000","last_active":"2024-03-06T07:21:00.010375+0000","last_peered":"2024-03-06T07:21:00.010375+0000","last_clean":"2024-03-06T07:21:00.010375+0000","last_became_active":"2024-03-06T07:21:00.006276+0000","last_became_peered":"2024-03-06T07:21:00.006276+0000","last_unstale":"2024-03-06T07:21:00.010375+0000","last_undegraded":"2024-03-06T07:21:00.010375+0000","last_fullsized":"2024-03-06T07:21:00.010375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:26.980322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999890+0000","last_change":"2024-03-06T07:20:59.999890+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999890+0000","last_undegraded":"2024-03-06T07:20:59.999890+0000","last_fullsized":"2024-03-06T07:20:59.999890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018160+0000","last_change":"2024-03-06T07:21:00.011278+0000","last_active":"2024-03-06T07:21:00.018160+0000","last_peered":"2024-03-06T07:21:00.018160+0000","last_clean":"2024-03-06T07:21:00.018160+0000","last_became_active":"2024-03-06T07:21:00.011018+0000","last_became_peered":"2024-03-06T07:21:00.011018+0000","last_unstale":"2024-03-06T07:21:00.018160+0000","last_undegraded":"2024-03-06T07:21:00.018160+0000","last_fullsized":"2024-03-06T07:21:00.018160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997087+0000","last_change":"2024-03-06T07:20:59.997087+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997087+0000","last_undegraded":"2024-03-06T07:20:59.997087+0000","last_fullsized":"2024-03-06T07:20:59.997087+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.014862+0000","last_change":"2024-03-06T07:21:00.013699+0000","last_active":"2024-03-06T07:21:00.014862+0000","last_peered":"2024-03-06T07:21:00.014862+0000","last_clean":"2024-03-06T07:21:00.014862+0000","last_became_active":"2024-03-06T07:21:00.013360+0000","last_became_peered":"2024-03-06T07:21:00.013360+0000","last_unstale":"2024-03-06T07:21:00.014862+0000","last_undegraded":"2024-03-06T07:21:00.014862+0000","last_fullsized":"2024-03-06T07:21:00.014862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:06.072955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001334+0000","last_change":"2024-03-06T07:21:00.001334+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001334+0000","last_undegraded":"2024-03-06T07:21:00.001334+0000","last_fullsized":"2024-03-06T07:21:00.001334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018669+0000","last_change":"2024-03-06T07:21:00.015827+0000","last_active":"2024-03-06T07:21:00.018669+0000","last_peered":"2024-03-06T07:21:00.018669+0000","last_clean":"2024-03-06T07:21:00.018669+0000","last_became_active":"2024-03-06T07:21:00.015565+0000","last_became_peered":"2024-03-06T07:21:00.015565+0000","last_unstale":"2024-03-06T07:21:00.018669+0000","last_undegraded":"2024-03-06T07:21:00.018669+0000","last_fullsized":"2024-03-06T07:21:00.018669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:44.990676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996060+0000","last_change":"2024-03-06T07:20:59.996060+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996060+0000","last_undegraded":"2024-03-06T07:20:59.996060+0000","last_fullsized":"2024-03-06T07:20:59.996060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011205+0000","last_change":"2024-03-06T07:21:00.009047+0000","last_active":"2024-03-06T07:21:00.011205+0000","last_peered":"2024-03-06T07:21:00.011205+0000","last_clean":"2024-03-06T07:21:00.011205+0000","last_became_active":"2024-03-06T07:21:00.008803+0000","last_became_peered":"2024-03-06T07:21:00.008803+0000","last_unstale":"2024-03-06T07:21:00.011205+0000","last_undegraded":"2024-03-06T07:21:00.011205+0000","last_fullsized":"2024-03-06T07:21:00.011205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:17.432587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998464+0000","last_change":"2024-03-06T07:20:59.998464+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998464+0000","last_undegraded":"2024-03-06T07:20:59.998464+0000","last_fullsized":"2024-03-06T07:20:59.998464+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016624+0000","last_change":"2024-03-06T07:21:00.015433+0000","last_active":"2024-03-06T07:21:00.016624+0000","last_peered":"2024-03-06T07:21:00.016624+0000","last_clean":"2024-03-06T07:21:00.016624+0000","last_became_active":"2024-03-06T07:21:00.015156+0000","last_became_peered":"2024-03-06T07:21:00.015156+0000","last_unstale":"2024-03-06T07:21:00.016624+0000","last_undegraded":"2024-03-06T07:21:00.016624+0000","last_fullsized":"2024-03-06T07:21:00.016624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.005409+0000","last_change":"2024-03-06T07:21:00.005409+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.005409+0000","last_undegraded":"2024-03-06T07:21:00.005409+0000","last_fullsized":"2024-03-06T07:21:00.005409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017074+0000","last_change":"2024-03-06T07:21:00.016544+0000","last_active":"2024-03-06T07:21:00.017074+0000","last_peered":"2024-03-06T07:21:00.017074+0000","last_clean":"2024-03-06T07:21:00.017074+0000","last_became_active":"2024-03-06T07:21:00.015894+0000","last_became_peered":"2024-03-06T07:21:00.015894+0000","last_unstale":"2024-03-06T07:21:00.017074+0000","last_undegraded":"2024-03-06T07:21:00.017074+0000","last_fullsized":"2024-03-06T07:21:00.017074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.008546+0000","last_change":"2024-03-06T07:21:00.008546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.008546+0000","last_undegraded":"2024-03-06T07:21:00.008546+0000","last_fullsized":"2024-03-06T07:21:00.008546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015354+0000","last_change":"2024-03-06T07:21:00.010211+0000","last_active":"2024-03-06T07:21:00.015354+0000","last_peered":"2024-03-06T07:21:00.015354+0000","last_clean":"2024-03-06T07:21:00.015354+0000","last_became_active":"2024-03-06T07:21:00.009915+0000","last_became_peered":"2024-03-06T07:21:00.009915+0000","last_unstale":"2024-03-06T07:21:00.015354+0000","last_undegraded":"2024-03-06T07:21:00.015354+0000","last_fullsized":"2024-03-06T07:21:00.015354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001003+0000","last_change":"2024-03-06T07:21:00.001003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001003+0000","last_undegraded":"2024-03-06T07:21:00.001003+0000","last_fullsized":"2024-03-06T07:21:00.001003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018391+0000","last_change":"2024-03-06T07:21:00.011094+0000","last_active":"2024-03-06T07:21:00.018391+0000","last_peered":"2024-03-06T07:21:00.018391+0000","last_clean":"2024-03-06T07:21:00.018391+0000","last_became_active":"2024-03-06T07:21:00.010457+0000","last_became_peered":"2024-03-06T07:21:00.010457+0000","last_unstale":"2024-03-06T07:21:00.018391+0000","last_undegraded":"2024-03-06T07:21:00.018391+0000","last_fullsized":"2024-03-06T07:21:00.018391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:41.550434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996256+0000","last_change":"2024-03-06T07:20:59.996256+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996256+0000","last_undegraded":"2024-03-06T07:20:59.996256+0000","last_fullsized":"2024-03-06T07:20:59.996256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017529+0000","last_change":"2024-03-06T07:21:00.014362+0000","last_active":"2024-03-06T07:21:00.017529+0000","last_peered":"2024-03-06T07:21:00.017529+0000","last_clean":"2024-03-06T07:21:00.017529+0000","last_became_active":"2024-03-06T07:21:00.014072+0000","last_became_peered":"2024-03-06T07:21:00.014072+0000","last_unstale":"2024-03-06T07:21:00.017529+0000","last_undegraded":"2024-03-06T07:21:00.017529+0000","last_fullsized":"2024-03-06T07:21:00.017529+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:12.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003607+0000","last_change":"2024-03-06T07:21:00.003607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003607+0000","last_undegraded":"2024-03-06T07:21:00.003607+0000","last_fullsized":"2024-03-06T07:21:00.003607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005867+0000","last_change":"2024-03-06T07:21:00.002103+0000","last_active":"2024-03-06T07:21:00.005867+0000","last_peered":"2024-03-06T07:21:00.005867+0000","last_clean":"2024-03-06T07:21:00.005867+0000","last_became_active":"2024-03-06T07:21:00.001791+0000","last_became_peered":"2024-03-06T07:21:00.001791+0000","last_unstale":"2024-03-06T07:21:00.005867+0000","last_undegraded":"2024-03-06T07:21:00.005867+0000","last_fullsized":"2024-03-06T07:21:00.005867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:20:59.995260+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:59.995260+0000","last_peered":"2024-03-06T07:20:59.995260+0000","last_clean":"2024-03-06T07:20:59.995260+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:59.995260+0000","last_undegraded":"2024-03-06T07:20:59.995260+0000","last_fullsized":"2024-03-06T07:20:59.995260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:04.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-03-06T07:21:04.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.999702+0000', 'last_change': '2024-03-06T07:20:59.999702+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.999702+0000', 'last_undegraded': '2024-03-06T07:20:59.999702+0000', 'last_fullsized': '2024-03-06T07:20:59.999702+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.998904+0000', 'last_change': '2024-03-06T07:20:59.998904+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.998904+0000', 'last_undegraded': '2024-03-06T07:20:59.998904+0000', 'last_fullsized': '2024-03-06T07:20:59.998904+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003003+0000', 'last_change': '2024-03-06T07:21:00.003003+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003003+0000', 'last_undegraded': '2024-03-06T07:21:00.003003+0000', 'last_fullsized': '2024-03-06T07:21:00.003003+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997726+0000', 'last_change': '2024-03-06T07:20:59.997726+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997726+0000', 'last_undegraded': '2024-03-06T07:20:59.997726+0000', 'last_fullsized': '2024-03-06T07:20:59.997726+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.996704+0000', 'last_change': '2024-03-06T07:20:59.996704+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.996704+0000', 'last_undegraded': '2024-03-06T07:20:59.996704+0000', 'last_fullsized': '2024-03-06T07:20:59.996704+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.001349+0000', 'last_change': '2024-03-06T07:21:00.001349+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.001349+0000', 'last_undegraded': '2024-03-06T07:21:00.001349+0000', 'last_fullsized': '2024-03-06T07:21:00.001349+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-03-06T07:21:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.001027+0000', 'last_change': '2024-03-06T07:21:00.001027+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.001027+0000', 'last_undegraded': '2024-03-06T07:21:00.001027+0000', 'last_fullsized': '2024-03-06T07:21:00.001027+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-03-06T07:21:04.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997302+0000', 'last_change': '2024-03-06T07:20:59.997302+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997302+0000', 'last_undegraded': '2024-03-06T07:20:59.997302+0000', 'last_fullsized': '2024-03-06T07:20:59.997302+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-03-06T07:21:04.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003260+0000', 'last_change': '2024-03-06T07:21:00.003260+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003260+0000', 'last_undegraded': '2024-03-06T07:21:00.003260+0000', 'last_fullsized': '2024-03-06T07:21:00.003260+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-03-06T07:21:04.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.002815+0000', 'last_change': '2024-03-06T07:21:00.002815+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.002815+0000', 'last_undegraded': '2024-03-06T07:21:00.002815+0000', 'last_fullsized': '2024-03-06T07:21:00.002815+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-03-06T07:21:04.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.004749+0000', 'last_change': '2024-03-06T07:21:00.004749+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.004749+0000', 'last_undegraded': '2024-03-06T07:21:00.004749+0000', 'last_fullsized': '2024-03-06T07:21:00.004749+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-03-06T07:21:04.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003163+0000', 'last_change': '2024-03-06T07:21:00.003163+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003163+0000', 'last_undegraded': '2024-03-06T07:21:00.003163+0000', 'last_fullsized': '2024-03-06T07:21:00.003163+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-03-06T07:21:04.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003644+0000', 'last_change': '2024-03-06T07:21:00.003644+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003644+0000', 'last_undegraded': '2024-03-06T07:21:00.003644+0000', 'last_fullsized': '2024-03-06T07:21:00.003644+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-03-06T07:21:04.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.001459+0000', 'last_change': '2024-03-06T07:21:00.001459+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.001459+0000', 'last_undegraded': '2024-03-06T07:21:00.001459+0000', 'last_fullsized': '2024-03-06T07:21:00.001459+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:04.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-03-06T07:21:04.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000599+0000', 'last_change': '2024-03-06T07:21:00.000599+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000599+0000', 'last_undegraded': '2024-03-06T07:21:00.000599+0000', 'last_fullsized': '2024-03-06T07:21:00.000599+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-03-06T07:21:04.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.998031+0000', 'last_change': '2024-03-06T07:20:59.998031+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.998031+0000', 'last_undegraded': '2024-03-06T07:20:59.998031+0000', 'last_fullsized': '2024-03-06T07:20:59.998031+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-03-06T07:21:04.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.998032+0000', 'last_change': '2024-03-06T07:20:59.998032+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.998032+0000', 'last_undegraded': '2024-03-06T07:20:59.998032+0000', 'last_fullsized': '2024-03-06T07:20:59.998032+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-03-06T07:21:04.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003193+0000', 'last_change': '2024-03-06T07:21:00.003193+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003193+0000', 'last_undegraded': '2024-03-06T07:21:00.003193+0000', 'last_fullsized': '2024-03-06T07:21:00.003193+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:04.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-03-06T07:21:04.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997357+0000', 'last_change': '2024-03-06T07:20:59.997357+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997357+0000', 'last_undegraded': '2024-03-06T07:20:59.997357+0000', 'last_fullsized': '2024-03-06T07:20:59.997357+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:04.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-03-06T07:21:04.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003259+0000', 'last_change': '2024-03-06T07:21:00.003259+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003259+0000', 'last_undegraded': '2024-03-06T07:21:00.003259+0000', 'last_fullsized': '2024-03-06T07:21:00.003259+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:04.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-03-06T07:21:04.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003247+0000', 'last_change': '2024-03-06T07:21:00.003247+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003247+0000', 'last_undegraded': '2024-03-06T07:21:00.003247+0000', 'last_fullsized': '2024-03-06T07:21:00.003247+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-03-06T07:21:04.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000143+0000', 'last_change': '2024-03-06T07:21:00.000143+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000143+0000', 'last_undegraded': '2024-03-06T07:21:00.000143+0000', 'last_fullsized': '2024-03-06T07:21:00.000143+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:04.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-03-06T07:21:04.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997546+0000', 'last_change': '2024-03-06T07:20:59.997546+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997546+0000', 'last_undegraded': '2024-03-06T07:20:59.997546+0000', 'last_fullsized': '2024-03-06T07:20:59.997546+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-03-06T07:21:04.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.998183+0000', 'last_change': '2024-03-06T07:20:59.998183+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.998183+0000', 'last_undegraded': '2024-03-06T07:20:59.998183+0000', 'last_fullsized': '2024-03-06T07:20:59.998183+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:04.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-03-06T07:21:04.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003168+0000', 'last_change': '2024-03-06T07:21:00.003168+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003168+0000', 'last_undegraded': '2024-03-06T07:21:00.003168+0000', 'last_fullsized': '2024-03-06T07:21:00.003168+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:04.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-03-06T07:21:04.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.002177+0000', 'last_change': '2024-03-06T07:21:00.002177+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.002177+0000', 'last_undegraded': '2024-03-06T07:21:00.002177+0000', 'last_fullsized': '2024-03-06T07:21:00.002177+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:04.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-03-06T07:21:04.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003051+0000', 'last_change': '2024-03-06T07:21:00.003051+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003051+0000', 'last_undegraded': '2024-03-06T07:21:00.003051+0000', 'last_fullsized': '2024-03-06T07:21:00.003051+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:04.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-03-06T07:21:04.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000938+0000', 'last_change': '2024-03-06T07:21:00.000938+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000938+0000', 'last_undegraded': '2024-03-06T07:21:00.000938+0000', 'last_fullsized': '2024-03-06T07:21:00.000938+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-03-06T07:21:04.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.002465+0000', 'last_change': '2024-03-06T07:21:00.002465+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.002465+0000', 'last_undegraded': '2024-03-06T07:21:00.002465+0000', 'last_fullsized': '2024-03-06T07:21:00.002465+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:04.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-03-06T07:21:04.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.001989+0000', 'last_change': '2024-03-06T07:21:00.001989+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.001989+0000', 'last_undegraded': '2024-03-06T07:21:00.001989+0000', 'last_fullsized': '2024-03-06T07:21:00.001989+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:04.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-03-06T07:21:04.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997070+0000', 'last_change': '2024-03-06T07:20:59.997070+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997070+0000', 'last_undegraded': '2024-03-06T07:20:59.997070+0000', 'last_fullsized': '2024-03-06T07:20:59.997070+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-03-06T07:21:04.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.002027+0000', 'last_change': '2024-03-06T07:21:00.002027+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.002027+0000', 'last_undegraded': '2024-03-06T07:21:00.002027+0000', 'last_fullsized': '2024-03-06T07:21:00.002027+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:04.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-03-06T07:21:04.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997095+0000', 'last_change': '2024-03-06T07:20:59.997095+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997095+0000', 'last_undegraded': '2024-03-06T07:20:59.997095+0000', 'last_fullsized': '2024-03-06T07:20:59.997095+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-03-06T07:21:04.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.001748+0000', 'last_change': '2024-03-06T07:21:00.001748+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.001748+0000', 'last_undegraded': '2024-03-06T07:21:00.001748+0000', 'last_fullsized': '2024-03-06T07:21:00.001748+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:04.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-03-06T07:21:04.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000216+0000', 'last_change': '2024-03-06T07:21:00.000216+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000216+0000', 'last_undegraded': '2024-03-06T07:21:00.000216+0000', 'last_fullsized': '2024-03-06T07:21:00.000216+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:04.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-03-06T07:21:04.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000111+0000', 'last_change': '2024-03-06T07:21:00.000111+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000111+0000', 'last_undegraded': '2024-03-06T07:21:00.000111+0000', 'last_fullsized': '2024-03-06T07:21:00.000111+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-03-06T07:21:04.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.010108+0000', 'last_change': '2024-03-06T07:21:00.010108+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.010108+0000', 'last_undegraded': '2024-03-06T07:21:00.010108+0000', 'last_fullsized': '2024-03-06T07:21:00.010108+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-06T07:21:04.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000136+0000', 'last_change': '2024-03-06T07:21:00.000136+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000136+0000', 'last_undegraded': '2024-03-06T07:21:00.000136+0000', 'last_fullsized': '2024-03-06T07:21:00.000136+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-03-06T07:21:04.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000567+0000', 'last_change': '2024-03-06T07:21:00.000567+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000567+0000', 'last_undegraded': '2024-03-06T07:21:00.000567+0000', 'last_fullsized': '2024-03-06T07:21:00.000567+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:04.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-03-06T07:21:04.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.998815+0000', 'last_change': '2024-03-06T07:20:59.998815+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.998815+0000', 'last_undegraded': '2024-03-06T07:20:59.998815+0000', 'last_fullsized': '2024-03-06T07:20:59.998815+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-03-06T07:21:04.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997276+0000', 'last_change': '2024-03-06T07:20:59.997276+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997276+0000', 'last_undegraded': '2024-03-06T07:20:59.997276+0000', 'last_fullsized': '2024-03-06T07:20:59.997276+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:04.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-03-06T07:21:04.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.996607+0000', 'last_change': '2024-03-06T07:20:59.996607+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.996607+0000', 'last_undegraded': '2024-03-06T07:20:59.996607+0000', 'last_fullsized': '2024-03-06T07:20:59.996607+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-03-06T07:21:04.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.004577+0000', 'last_change': '2024-03-06T07:21:00.004577+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.004577+0000', 'last_undegraded': '2024-03-06T07:21:00.004577+0000', 'last_fullsized': '2024-03-06T07:21:00.004577+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-03-06T07:21:04.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.999477+0000', 'last_change': '2024-03-06T07:20:59.999477+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.999477+0000', 'last_undegraded': '2024-03-06T07:20:59.999477+0000', 'last_fullsized': '2024-03-06T07:20:59.999477+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:04.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-03-06T07:21:04.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.009601+0000', 'last_change': '2024-03-06T07:21:00.009601+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.009601+0000', 'last_undegraded': '2024-03-06T07:21:00.009601+0000', 'last_fullsized': '2024-03-06T07:21:00.009601+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-03-06T07:21:04.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.999388+0000', 'last_change': '2024-03-06T07:20:59.999388+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.999388+0000', 'last_undegraded': '2024-03-06T07:20:59.999388+0000', 'last_fullsized': '2024-03-06T07:20:59.999388+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-03-06T07:21:04.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.996517+0000', 'last_change': '2024-03-06T07:20:59.996517+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.996517+0000', 'last_undegraded': '2024-03-06T07:20:59.996517+0000', 'last_fullsized': '2024-03-06T07:20:59.996517+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-03-06T07:21:04.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.998304+0000', 'last_change': '2024-03-06T07:20:59.998304+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.998304+0000', 'last_undegraded': '2024-03-06T07:20:59.998304+0000', 'last_fullsized': '2024-03-06T07:20:59.998304+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-03-06T07:21:04.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.999103+0000', 'last_change': '2024-03-06T07:20:59.999103+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.999103+0000', 'last_undegraded': '2024-03-06T07:20:59.999103+0000', 'last_fullsized': '2024-03-06T07:20:59.999103+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-06T07:21:04.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.004368+0000', 'last_change': '2024-03-06T07:21:00.004368+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.004368+0000', 'last_undegraded': '2024-03-06T07:21:00.004368+0000', 'last_fullsized': '2024-03-06T07:21:00.004368+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-03-06T07:21:04.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.000640+0000', 'last_change': '2024-03-06T07:21:00.000640+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.000640+0000', 'last_undegraded': '2024-03-06T07:21:00.000640+0000', 'last_fullsized': '2024-03-06T07:21:00.000640+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:04.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-03-06T07:21:04.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.996923+0000', 'last_change': '2024-03-06T07:20:59.996923+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.996923+0000', 'last_undegraded': '2024-03-06T07:20:59.996923+0000', 'last_fullsized': '2024-03-06T07:20:59.996923+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:04.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-03-06T07:21:04.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.996058+0000', 'last_change': '2024-03-06T07:20:59.996058+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.996058+0000', 'last_undegraded': '2024-03-06T07:20:59.996058+0000', 'last_fullsized': '2024-03-06T07:20:59.996058+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-03-06T07:21:04.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.002934+0000', 'last_change': '2024-03-06T07:21:00.002934+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.002934+0000', 'last_undegraded': '2024-03-06T07:21:00.002934+0000', 'last_fullsized': '2024-03-06T07:21:00.002934+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-06T07:21:04.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.999890+0000', 'last_change': '2024-03-06T07:20:59.999890+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.999890+0000', 'last_undegraded': '2024-03-06T07:20:59.999890+0000', 'last_fullsized': '2024-03-06T07:20:59.999890+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-03-06T07:21:04.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.997087+0000', 'last_change': '2024-03-06T07:20:59.997087+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.997087+0000', 'last_undegraded': '2024-03-06T07:20:59.997087+0000', 'last_fullsized': '2024-03-06T07:20:59.997087+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-03-06T07:21:04.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.001334+0000', 'last_change': '2024-03-06T07:21:00.001334+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.001334+0000', 'last_undegraded': '2024-03-06T07:21:00.001334+0000', 'last_fullsized': '2024-03-06T07:21:00.001334+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:04.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-03-06T07:21:04.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.996060+0000', 'last_change': '2024-03-06T07:20:59.996060+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.996060+0000', 'last_undegraded': '2024-03-06T07:20:59.996060+0000', 'last_fullsized': '2024-03-06T07:20:59.996060+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-03-06T07:21:04.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.998464+0000', 'last_change': '2024-03-06T07:20:59.998464+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.998464+0000', 'last_undegraded': '2024-03-06T07:20:59.998464+0000', 'last_fullsized': '2024-03-06T07:20:59.998464+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:04.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-03-06T07:21:04.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.005409+0000', 'last_change': '2024-03-06T07:21:00.005409+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.005409+0000', 'last_undegraded': '2024-03-06T07:21:00.005409+0000', 'last_fullsized': '2024-03-06T07:21:00.005409+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:04.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-03-06T07:21:04.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.008546+0000', 'last_change': '2024-03-06T07:21:00.008546+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.008546+0000', 'last_undegraded': '2024-03-06T07:21:00.008546+0000', 'last_fullsized': '2024-03-06T07:21:00.008546+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:04.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-03-06T07:21:04.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.001003+0000', 'last_change': '2024-03-06T07:21:00.001003+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.001003+0000', 'last_undegraded': '2024-03-06T07:21:00.001003+0000', 'last_fullsized': '2024-03-06T07:21:00.001003+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:04.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-03-06T07:21:04.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:20:59.996256+0000', 'last_change': '2024-03-06T07:20:59.996256+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:20:59.996256+0000', 'last_undegraded': '2024-03-06T07:20:59.996256+0000', 'last_fullsized': '2024-03-06T07:20:59.996256+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:04.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-06T07:21:04.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:00.003607+0000', 'last_change': '2024-03-06T07:21:00.003607+0000', 'last_active': '2024-03-06T07:20:59.984090+0000', 'last_peered': '2024-03-06T07:20:59.984090+0000', 'last_clean': '2024-03-06T07:20:59.984090+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:00.003607+0000', 'last_undegraded': '2024-03-06T07:21:00.003607+0000', 'last_fullsized': '2024-03-06T07:21:00.003607+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'last_clean_scrub_stamp': '2024-03-06T07:20:59.984090+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:04.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.849+0000 7effbafd7640 1 Processor -- start 2024-03-06T07:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.849+0000 7effbafd7640 1 -- start start 2024-03-06T07:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effbafd7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effb40c5480 0x7effb4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effbafd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effbafd7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb40c5cc0 con 0x7effb40c5860 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effbafd7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb406df40 con 0x7effb40c5480 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effbafd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb406fdb0 con 0x7effb41586f0 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effb8d73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effb40c5480 0x7effb4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effb8d73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effb40c5480 0x7effb4155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44480/0 (socket says 172.21.15.26:44480) 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effb8d73640 1 -- 172.21.15.26:0/3611587188 learned_addr learned my addr 172.21.15.26:0/3611587188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.850+0000 7effb9574640 1 --2- 172.21.15.26:0/3611587188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.851+0000 7effb8d73640 1 -- 172.21.15.26:0/3611587188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 msgr2=0x7effb4158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.851+0000 7effb8d73640 1 --2- 172.21.15.26:0/3611587188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.851+0000 7effb8d73640 1 -- 172.21.15.26:0/3611587188 >> v1:172.21.15.117:6789/0 conn(0x7effb40c5860 legacy=0x7effb406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.851+0000 7effb8d73640 1 -- 172.21.15.26:0/3611587188 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb41591c0 con 0x7effb40c5480 2024-03-06T07:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.851+0000 7effb8d73640 1 --2- 172.21.15.26:0/3611587188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb4155ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7effac009870 tx=0x7effac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5206ce8531ce9a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effb37fe640 1 -- 172.21.15.26:0/3611587188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac035020 con 0x7effb40c5480 2024-03-06T07:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effb37fe640 1 -- 172.21.15.26:0/3611587188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac031640 con 0x7effb40c5480 2024-03-06T07:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effb37fe640 1 -- 172.21.15.26:0/3611587188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac02d040 con 0x7effb40c5480 2024-03-06T07:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 -- 172.21.15.26:0/3611587188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 msgr2=0x7effb4155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 --2- 172.21.15.26:0/3611587188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb4155ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7effac009870 tx=0x7effac02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 -- 172.21.15.26:0/3611587188 shutdown_connections 2024-03-06T07:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 --2- 172.21.15.26:0/3611587188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 --2- 172.21.15.26:0/3611587188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb4155ba0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 -- 172.21.15.26:0/3611587188 >> 172.21.15.26:0/3611587188 conn(0x7effb406b1a0 msgr2=0x7effb406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 -- 172.21.15.26:0/3611587188 shutdown_connections 2024-03-06T07:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.852+0000 7effbafd7640 1 -- 172.21.15.26:0/3611587188 wait complete. 2024-03-06T07:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.853+0000 7effbafd7640 1 Processor -- start 2024-03-06T07:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.853+0000 7effbafd7640 1 -- start start 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effbafd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb41566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effbafd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb40c5860 0x7effb4150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effbafd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effbafd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb4155680 con 0x7effb40c5480 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effbafd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb41553a0 con 0x7effb40c5860 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effbafd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb4152a20 con 0x7effb41586f0 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb41566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb41566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44490/0 (socket says 172.21.15.26:44490) 2024-03-06T07:21:04.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 -- 172.21.15.26:0/1196960306 learned_addr learned my addr 172.21.15.26:0/1196960306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb3fff640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb40c5860 0x7effb4150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb9574640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 -- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 msgr2=0x7effb4151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 -- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb40c5860 msgr2=0x7effb4150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb40c5860 0x7effb4150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.854+0000 7effb8d73640 1 -- 172.21.15.26:0/1196960306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40c4df0 con 0x7effb40c5480 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.855+0000 7effb8d73640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb41566b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7effac02b930 tx=0x7effac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.855+0000 7effb17fa640 1 -- 172.21.15.26:0/1196960306 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac040070 con 0x7effb40c5480 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.855+0000 7effb17fa640 1 -- 172.21.15.26:0/1196960306 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac03e070 con 0x7effb40c5480 2024-03-06T07:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.855+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb40c4b10 con 0x7effb40c5480 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.855+0000 7effb17fa640 1 -- 172.21.15.26:0/1196960306 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac035040 con 0x7effb40c5480 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.855+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb4150510 con 0x7effb40c5480 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.856+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb404f330 con 0x7effb40c5480 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.856+0000 7effb17fa640 1 -- 172.21.15.26:0/1196960306 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effac038020 con 0x7effb40c5480 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.856+0000 7effb17fa640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7eff9406b4d0 0x7eff9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.857+0000 7effb17fa640 1 -- 172.21.15.26:0/1196960306 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7effac0abb50 con 0x7effb40c5480 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.857+0000 7effb3fff640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7eff9406b4d0 0x7eff9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.857+0000 7effb3fff640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7eff9406b4d0 0x7eff9406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7effb40c5170 tx=0x7effa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:04.861+0000 7effb17fa640 1 -- 172.21.15.26:0/1196960306 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7effac07a280 con 0x7effb40c5480 2024-03-06T07:21:05.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.118+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7effb406e0c0 con 0x7effb40c5480 2024-03-06T07:21:05.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.118+0000 7effb17fa640 1 -- 172.21.15.26:0/1196960306 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1581 (secure 0 0 0) 0x7effac07e320 con 0x7effb40c5480 2024-03-06T07:21:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:05.121 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":37,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709595,"num_in_osds":8,"osd_in_since":1709709586,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":241631232,"bytes_avail":772852482048,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5209}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-03-06T07:20:57.041789+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:21:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7eff9406b4d0 msgr2=0x7eff9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7eff9406b4d0 0x7eff9406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7effb40c5170 tx=0x7effa4009040 comp rx=0 tx=0).stop 2024-03-06T07:21:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 msgr2=0x7effb41566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb41566b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7effac02b930 tx=0x7effac0046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 shutdown_connections 2024-03-06T07:21:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7eff9406b4d0 0x7eff9406d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb41586f0 0x7effb4151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb40c5860 0x7effb4150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 --2- 172.21.15.26:0/1196960306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb40c5480 0x7effb41566b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 >> 172.21.15.26:0/1196960306 conn(0x7effb406b1a0 msgr2=0x7effb40c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.120+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 shutdown_connections 2024-03-06T07:21:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.121+0000 7effbafd7640 1 -- 172.21.15.26:0/1196960306 wait complete. 2024-03-06T07:21:05.134 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:21:05.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.233+0000 7f06a495e640 1 Processor -- start 2024-03-06T07:21:05.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.233+0000 7f06a495e640 1 -- start start 2024-03-06T07:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.233+0000 7f06a495e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f06a0152920 0x7f06a0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.233+0000 7f06a495e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a00c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.233+0000 7f06a495e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a0156180 con 0x7f06a00c8a00 2024-03-06T07:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f06a495e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a01587b0 con 0x7f06a0152920 2024-03-06T07:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f06a495e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a0159630 con 0x7f06a00c5330 2024-03-06T07:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069dd9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f06a00c8a00 0x7f06a0158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57942/0 (socket says 172.21.15.26:57942) 2024-03-06T07:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069dd9b640 1 -- 172.21.15.26:0/1105858406 learned_addr learned my addr 172.21.15.26:0/1105858406 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069ed9d640 1 --2- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a00c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069e59c640 1 --2- 172.21.15.26:0/1105858406 >> v2:172.21.15.117:3300/0 conn(0x7f06a0152920 0x7f06a0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069d59a640 1 -- 172.21.15.26:0/1105858406 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938024992 0 0) 0x7f0688001110 con 0x7f06a00c8a00 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069d59a640 1 -- 172.21.15.26:0/1105858406 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f068c003520 con 0x7f06a00c8a00 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069e59c640 1 -- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 msgr2=0x7f06a00c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069e59c640 1 --2- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a00c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069e59c640 1 -- 172.21.15.26:0/1105858406 >> v1:172.21.15.26:6789/0 conn(0x7f06a00c8a00 legacy=0x7f06a0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.234+0000 7f069e59c640 1 -- 172.21.15.26:0/1105858406 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00c7760 con 0x7f06a0152920 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.235+0000 7f069e59c640 1 --2- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 0x7f06a0155c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f06980097a0 tx=0x7f069802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c089c8722bcf88 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.235+0000 7f069d59a640 1 -- 172.21.15.26:0/1105858406 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06980315a0 con 0x7f06a0152920 2024-03-06T07:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.235+0000 7f069d59a640 1 -- 172.21.15.26:0/1105858406 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0698037070 con 0x7f06a0152920 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.235+0000 7f069d59a640 1 -- 172.21.15.26:0/1105858406 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698031940 con 0x7f06a0152920 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.235+0000 7f06a495e640 1 -- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 msgr2=0x7f06a0155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.235+0000 7f06a495e640 1 --2- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 0x7f06a0155c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f06980097a0 tx=0x7f069802b840 comp rx=0 tx=0).stop 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.236+0000 7f06a495e640 1 -- 172.21.15.26:0/1105858406 shutdown_connections 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.236+0000 7f06a495e640 1 --2- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a00c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.236+0000 7f06a495e640 1 --2- 172.21.15.26:0/1105858406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 0x7f06a0155c40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.236+0000 7f06a495e640 1 -- 172.21.15.26:0/1105858406 >> 172.21.15.26:0/1105858406 conn(0x7f06a00ac560 msgr2=0x7f06a00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.236+0000 7f06a495e640 1 -- 172.21.15.26:0/1105858406 shutdown_connections 2024-03-06T07:21:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.236+0000 7f06a495e640 1 -- 172.21.15.26:0/1105858406 wait complete. 2024-03-06T07:21:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.237+0000 7f06a495e640 1 Processor -- start 2024-03-06T07:21:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.237+0000 7f06a495e640 1 -- start start 2024-03-06T07:21:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.237+0000 7f06a495e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a01eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.237+0000 7f06a495e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06a00c8a00 0x7f06a014b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f06a495e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 0x7f06a0157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f06a495e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a0158a00 con 0x7f06a00c8a00 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f06a495e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00c7d20 con 0x7f06a0152920 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f06a495e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a006d030 con 0x7f06a00c5330 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06a00c8a00 0x7f06a014b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06a00c8a00 0x7f06a014b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44492/0 (socket says 172.21.15.26:44492) 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 -- 172.21.15.26:0/4082151454 learned_addr learned my addr 172.21.15.26:0/4082151454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069e59c640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a01eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069ed9d640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 0x7f06a0157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 -- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 msgr2=0x7f06a01eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a01eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 -- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 msgr2=0x7f06a0157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 0x7f06a0157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 -- 172.21.15.26:0/4082151454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a014fed0 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f069dd9b640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06a00c8a00 0x7f06a014b760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f068800c3a0 tx=0x7f068800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f0696ffd640 1 -- 172.21.15.26:0/4082151454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688017070 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f0696ffd640 1 -- 172.21.15.26:0/4082151454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688005ce0 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a0155790 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f0696ffd640 1 -- 172.21.15.26:0/4082151454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688007310 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.238+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a0155d50 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.239+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a01eabf0 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.240+0000 7f0696ffd640 1 -- 172.21.15.26:0/4082151454 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f068801a070 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.240+0000 7f0696ffd640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f068006b4d0 0x7f068006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.240+0000 7f0696ffd640 1 -- 172.21.15.26:0/4082151454 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0688089660 con 0x7f06a00c8a00 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.240+0000 7f069e59c640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f068006b4d0 0x7f068006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.241+0000 7f069e59c640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f068006b4d0 0x7f068006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0698036000 tx=0x7f0698008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.244+0000 7f0696ffd640 1 -- 172.21.15.26:0/4082151454 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f06a0156180 con 0x7f06a00c8a00 2024-03-06T07:21:05.315 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:21:05.314+0000 7f91a74e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.421+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06a004f330 con 0x7f068006b4d0 2024-03-06T07:21:05.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.427+0000 7f0696ffd640 1 -- 172.21.15.26:0/4082151454 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246709 (secure 0 0 0) 0x7f06a004f330 con 0x7f068006b4d0 2024-03-06T07:21:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:05.434 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:05.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f068006b4d0 msgr2=0x7f068006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f068006b4d0 0x7f068006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0698036000 tx=0x7f0698008030 comp rx=0 tx=0).stop 2024-03-06T07:21:05.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06a00c8a00 msgr2=0x7f06a014b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06a00c8a00 0x7f06a014b760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f068800c3a0 tx=0x7f068800c860 comp rx=0 tx=0).stop 2024-03-06T07:21:05.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 shutdown_connections 2024-03-06T07:21:05.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f068006b4d0 0x7f068006d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a0152920 0x7f06a0157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06a00c8a00 0x7f06a014b760 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 --2- 172.21.15.26:0/4082151454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a00c5330 0x7f06a01eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 >> 172.21.15.26:0/4082151454 conn(0x7f06a00ac560 msgr2=0x7f06a00bdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 shutdown_connections 2024-03-06T07:21:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:05.430+0000 7f06a495e640 1 -- 172.21.15.26:0/4082151454 wait complete. 2024-03-06T07:21:05.444 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-03-06T07:21:05.043128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":3168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":3244032,"data_stored":2483024,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000983"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999702+0000","last_change":"2024-03-06T07:20:59.999702+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999702+0000","last_undegraded":"2024-03-06T07:20:59.999702+0000","last_fullsized":"2024-03-06T07:20:59.999702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998904+0000","last_change":"2024-03-06T07:20:59.998904+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998904+0000","last_undegraded":"2024-03-06T07:20:59.998904+0000","last_fullsized":"2024-03-06T07:20:59.998904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003003+0000","last_change":"2024-03-06T07:21:00.003003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003003+0000","last_undegraded":"2024-03-06T07:21:00.003003+0000","last_fullsized":"2024-03-06T07:21:00.003003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997726+0000","last_change":"2024-03-06T07:20:59.997726+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997726+0000","last_undegraded":"2024-03-06T07:20:59.997726+0000","last_fullsized":"2024-03-06T07:20:59.997726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996704+0000","last_change":"2024-03-06T07:20:59.996704+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996704+0000","last_undegraded":"2024-03-06T07:20:59.996704+0000","last_fullsized":"2024-03-06T07:20:59.996704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001349+0000","last_change":"2024-03-06T07:21:00.001349+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001349+0000","last_undegraded":"2024-03-06T07:21:00.001349+0000","last_fullsized":"2024-03-06T07:21:00.001349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001027+0000","last_change":"2024-03-06T07:21:00.001027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001027+0000","last_undegraded":"2024-03-06T07:21:00.001027+0000","last_fullsized":"2024-03-06T07:21:00.001027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997302+0000","last_change":"2024-03-06T07:20:59.997302+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997302+0000","last_undegraded":"2024-03-06T07:20:59.997302+0000","last_fullsized":"2024-03-06T07:20:59.997302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003260+0000","last_change":"2024-03-06T07:21:00.003260+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003260+0000","last_undegraded":"2024-03-06T07:21:00.003260+0000","last_fullsized":"2024-03-06T07:21:00.003260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002815+0000","last_change":"2024-03-06T07:21:00.002815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002815+0000","last_undegraded":"2024-03-06T07:21:00.002815+0000","last_fullsized":"2024-03-06T07:21:00.002815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004749+0000","last_change":"2024-03-06T07:21:00.004749+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004749+0000","last_undegraded":"2024-03-06T07:21:00.004749+0000","last_fullsized":"2024-03-06T07:21:00.004749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003163+0000","last_change":"2024-03-06T07:21:00.003163+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003163+0000","last_undegraded":"2024-03-06T07:21:00.003163+0000","last_fullsized":"2024-03-06T07:21:00.003163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003644+0000","last_change":"2024-03-06T07:21:00.003644+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003644+0000","last_undegraded":"2024-03-06T07:21:00.003644+0000","last_fullsized":"2024-03-06T07:21:00.003644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001459+0000","last_change":"2024-03-06T07:21:00.001459+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001459+0000","last_undegraded":"2024-03-06T07:21:00.001459+0000","last_fullsized":"2024-03-06T07:21:00.001459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000599+0000","last_change":"2024-03-06T07:21:00.000599+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000599+0000","last_undegraded":"2024-03-06T07:21:00.000599+0000","last_fullsized":"2024-03-06T07:21:00.000599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998031+0000","last_change":"2024-03-06T07:20:59.998031+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998031+0000","last_undegraded":"2024-03-06T07:20:59.998031+0000","last_fullsized":"2024-03-06T07:20:59.998031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998032+0000","last_change":"2024-03-06T07:20:59.998032+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998032+0000","last_undegraded":"2024-03-06T07:20:59.998032+0000","last_fullsized":"2024-03-06T07:20:59.998032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003193+0000","last_change":"2024-03-06T07:21:00.003193+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003193+0000","last_undegraded":"2024-03-06T07:21:00.003193+0000","last_fullsized":"2024-03-06T07:21:00.003193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997357+0000","last_change":"2024-03-06T07:20:59.997357+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997357+0000","last_undegraded":"2024-03-06T07:20:59.997357+0000","last_fullsized":"2024-03-06T07:20:59.997357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003259+0000","last_change":"2024-03-06T07:21:00.003259+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003259+0000","last_undegraded":"2024-03-06T07:21:00.003259+0000","last_fullsized":"2024-03-06T07:21:00.003259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003247+0000","last_change":"2024-03-06T07:21:00.003247+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003247+0000","last_undegraded":"2024-03-06T07:21:00.003247+0000","last_fullsized":"2024-03-06T07:21:00.003247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000143+0000","last_change":"2024-03-06T07:21:00.000143+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000143+0000","last_undegraded":"2024-03-06T07:21:00.000143+0000","last_fullsized":"2024-03-06T07:21:00.000143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997546+0000","last_change":"2024-03-06T07:20:59.997546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997546+0000","last_undegraded":"2024-03-06T07:20:59.997546+0000","last_fullsized":"2024-03-06T07:20:59.997546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998183+0000","last_change":"2024-03-06T07:20:59.998183+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998183+0000","last_undegraded":"2024-03-06T07:20:59.998183+0000","last_fullsized":"2024-03-06T07:20:59.998183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003168+0000","last_change":"2024-03-06T07:21:00.003168+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003168+0000","last_undegraded":"2024-03-06T07:21:00.003168+0000","last_fullsized":"2024-03-06T07:21:00.003168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002177+0000","last_change":"2024-03-06T07:21:00.002177+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002177+0000","last_undegraded":"2024-03-06T07:21:00.002177+0000","last_fullsized":"2024-03-06T07:21:00.002177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003051+0000","last_change":"2024-03-06T07:21:00.003051+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003051+0000","last_undegraded":"2024-03-06T07:21:00.003051+0000","last_fullsized":"2024-03-06T07:21:00.003051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000938+0000","last_change":"2024-03-06T07:21:00.000938+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000938+0000","last_undegraded":"2024-03-06T07:21:00.000938+0000","last_fullsized":"2024-03-06T07:21:00.000938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002465+0000","last_change":"2024-03-06T07:21:00.002465+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002465+0000","last_undegraded":"2024-03-06T07:21:00.002465+0000","last_fullsized":"2024-03-06T07:21:00.002465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001989+0000","last_change":"2024-03-06T07:21:00.001989+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001989+0000","last_undegraded":"2024-03-06T07:21:00.001989+0000","last_fullsized":"2024-03-06T07:21:00.001989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997070+0000","last_change":"2024-03-06T07:20:59.997070+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997070+0000","last_undegraded":"2024-03-06T07:20:59.997070+0000","last_fullsized":"2024-03-06T07:20:59.997070+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002027+0000","last_change":"2024-03-06T07:21:00.002027+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002027+0000","last_undegraded":"2024-03-06T07:21:00.002027+0000","last_fullsized":"2024-03-06T07:21:00.002027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997095+0000","last_change":"2024-03-06T07:20:59.997095+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997095+0000","last_undegraded":"2024-03-06T07:20:59.997095+0000","last_fullsized":"2024-03-06T07:20:59.997095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018235+0000","last_change":"2024-03-06T07:21:00.010567+0000","last_active":"2024-03-06T07:21:00.018235+0000","last_peered":"2024-03-06T07:21:00.018235+0000","last_clean":"2024-03-06T07:21:00.018235+0000","last_became_active":"2024-03-06T07:21:00.010277+0000","last_became_peered":"2024-03-06T07:21:00.010277+0000","last_unstale":"2024-03-06T07:21:00.018235+0000","last_undegraded":"2024-03-06T07:21:00.018235+0000","last_fullsized":"2024-03-06T07:21:00.018235+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.435165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001748+0000","last_change":"2024-03-06T07:21:00.001748+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001748+0000","last_undegraded":"2024-03-06T07:21:00.001748+0000","last_fullsized":"2024-03-06T07:21:00.001748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.013052+0000","last_change":"2024-03-06T07:21:00.011675+0000","last_active":"2024-03-06T07:21:00.013052+0000","last_peered":"2024-03-06T07:21:00.013052+0000","last_clean":"2024-03-06T07:21:00.013052+0000","last_became_active":"2024-03-06T07:21:00.011344+0000","last_became_peered":"2024-03-06T07:21:00.011344+0000","last_unstale":"2024-03-06T07:21:00.013052+0000","last_undegraded":"2024-03-06T07:21:00.013052+0000","last_fullsized":"2024-03-06T07:21:00.013052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:04.618247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000216+0000","last_change":"2024-03-06T07:21:00.000216+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000216+0000","last_undegraded":"2024-03-06T07:21:00.000216+0000","last_fullsized":"2024-03-06T07:21:00.000216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016876+0000","last_change":"2024-03-06T07:21:00.012691+0000","last_active":"2024-03-06T07:21:00.016876+0000","last_peered":"2024-03-06T07:21:00.016876+0000","last_clean":"2024-03-06T07:21:00.016876+0000","last_became_active":"2024-03-06T07:21:00.012404+0000","last_became_peered":"2024-03-06T07:21:00.012404+0000","last_unstale":"2024-03-06T07:21:00.016876+0000","last_undegraded":"2024-03-06T07:21:00.016876+0000","last_fullsized":"2024-03-06T07:21:00.016876+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000111+0000","last_change":"2024-03-06T07:21:00.000111+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000111+0000","last_undegraded":"2024-03-06T07:21:00.000111+0000","last_fullsized":"2024-03-06T07:21:00.000111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018440+0000","last_change":"2024-03-06T07:21:00.011238+0000","last_active":"2024-03-06T07:21:00.018440+0000","last_peered":"2024-03-06T07:21:00.018440+0000","last_clean":"2024-03-06T07:21:00.018440+0000","last_became_active":"2024-03-06T07:21:00.010202+0000","last_became_peered":"2024-03-06T07:21:00.010202+0000","last_unstale":"2024-03-06T07:21:00.018440+0000","last_undegraded":"2024-03-06T07:21:00.018440+0000","last_fullsized":"2024-03-06T07:21:00.018440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:23.626033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.010108+0000","last_change":"2024-03-06T07:21:00.010108+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.010108+0000","last_undegraded":"2024-03-06T07:21:00.010108+0000","last_fullsized":"2024-03-06T07:21:00.010108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018243+0000","last_change":"2024-03-06T07:21:00.008004+0000","last_active":"2024-03-06T07:21:00.018243+0000","last_peered":"2024-03-06T07:21:00.018243+0000","last_clean":"2024-03-06T07:21:00.018243+0000","last_became_active":"2024-03-06T07:21:00.007716+0000","last_became_peered":"2024-03-06T07:21:00.007716+0000","last_unstale":"2024-03-06T07:21:00.018243+0000","last_undegraded":"2024-03-06T07:21:00.018243+0000","last_fullsized":"2024-03-06T07:21:00.018243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000136+0000","last_change":"2024-03-06T07:21:00.000136+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000136+0000","last_undegraded":"2024-03-06T07:21:00.000136+0000","last_fullsized":"2024-03-06T07:21:00.000136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010724+0000","last_change":"2024-03-06T07:21:00.006417+0000","last_active":"2024-03-06T07:21:00.010724+0000","last_peered":"2024-03-06T07:21:00.010724+0000","last_clean":"2024-03-06T07:21:00.010724+0000","last_became_active":"2024-03-06T07:21:00.006164+0000","last_became_peered":"2024-03-06T07:21:00.006164+0000","last_unstale":"2024-03-06T07:21:00.010724+0000","last_undegraded":"2024-03-06T07:21:00.010724+0000","last_fullsized":"2024-03-06T07:21:00.010724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:32.480280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000567+0000","last_change":"2024-03-06T07:21:00.000567+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000567+0000","last_undegraded":"2024-03-06T07:21:00.000567+0000","last_fullsized":"2024-03-06T07:21:00.000567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.019321+0000","last_change":"2024-03-06T07:21:00.018959+0000","last_active":"2024-03-06T07:21:00.019321+0000","last_peered":"2024-03-06T07:21:00.019321+0000","last_clean":"2024-03-06T07:21:00.019321+0000","last_became_active":"2024-03-06T07:21:00.018641+0000","last_became_peered":"2024-03-06T07:21:00.018641+0000","last_unstale":"2024-03-06T07:21:00.019321+0000","last_undegraded":"2024-03-06T07:21:00.019321+0000","last_fullsized":"2024-03-06T07:21:00.019321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998815+0000","last_change":"2024-03-06T07:20:59.998815+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998815+0000","last_undegraded":"2024-03-06T07:20:59.998815+0000","last_fullsized":"2024-03-06T07:20:59.998815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018484+0000","last_change":"2024-03-06T07:21:00.012306+0000","last_active":"2024-03-06T07:21:00.018484+0000","last_peered":"2024-03-06T07:21:00.018484+0000","last_clean":"2024-03-06T07:21:00.018484+0000","last_became_active":"2024-03-06T07:21:00.012030+0000","last_became_peered":"2024-03-06T07:21:00.012030+0000","last_unstale":"2024-03-06T07:21:00.018484+0000","last_undegraded":"2024-03-06T07:21:00.018484+0000","last_fullsized":"2024-03-06T07:21:00.018484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:34.501662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997276+0000","last_change":"2024-03-06T07:20:59.997276+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997276+0000","last_undegraded":"2024-03-06T07:20:59.997276+0000","last_fullsized":"2024-03-06T07:20:59.997276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017570+0000","last_change":"2024-03-06T07:21:00.015472+0000","last_active":"2024-03-06T07:21:00.017570+0000","last_peered":"2024-03-06T07:21:00.017570+0000","last_clean":"2024-03-06T07:21:00.017570+0000","last_became_active":"2024-03-06T07:21:00.015206+0000","last_became_peered":"2024-03-06T07:21:00.015206+0000","last_unstale":"2024-03-06T07:21:00.017570+0000","last_undegraded":"2024-03-06T07:21:00.017570+0000","last_fullsized":"2024-03-06T07:21:00.017570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:12.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996607+0000","last_change":"2024-03-06T07:20:59.996607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996607+0000","last_undegraded":"2024-03-06T07:20:59.996607+0000","last_fullsized":"2024-03-06T07:20:59.996607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018385+0000","last_change":"2024-03-06T07:21:00.011966+0000","last_active":"2024-03-06T07:21:00.018385+0000","last_peered":"2024-03-06T07:21:00.018385+0000","last_clean":"2024-03-06T07:21:00.018385+0000","last_became_active":"2024-03-06T07:21:00.011703+0000","last_became_peered":"2024-03-06T07:21:00.011703+0000","last_unstale":"2024-03-06T07:21:00.018385+0000","last_undegraded":"2024-03-06T07:21:00.018385+0000","last_fullsized":"2024-03-06T07:21:00.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:12.743167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004577+0000","last_change":"2024-03-06T07:21:00.004577+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004577+0000","last_undegraded":"2024-03-06T07:21:00.004577+0000","last_fullsized":"2024-03-06T07:21:00.004577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018708+0000","last_change":"2024-03-06T07:21:00.017403+0000","last_active":"2024-03-06T07:21:00.018708+0000","last_peered":"2024-03-06T07:21:00.018708+0000","last_clean":"2024-03-06T07:21:00.018708+0000","last_became_active":"2024-03-06T07:21:00.017171+0000","last_became_peered":"2024-03-06T07:21:00.017171+0000","last_unstale":"2024-03-06T07:21:00.018708+0000","last_undegraded":"2024-03-06T07:21:00.018708+0000","last_fullsized":"2024-03-06T07:21:00.018708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999477+0000","last_change":"2024-03-06T07:20:59.999477+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999477+0000","last_undegraded":"2024-03-06T07:20:59.999477+0000","last_fullsized":"2024-03-06T07:20:59.999477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011916+0000","last_change":"2024-03-06T07:21:00.007232+0000","last_active":"2024-03-06T07:21:00.011916+0000","last_peered":"2024-03-06T07:21:00.011916+0000","last_clean":"2024-03-06T07:21:00.011916+0000","last_became_active":"2024-03-06T07:21:00.006787+0000","last_became_peered":"2024-03-06T07:21:00.006787+0000","last_unstale":"2024-03-06T07:21:00.011916+0000","last_undegraded":"2024-03-06T07:21:00.011916+0000","last_fullsized":"2024-03-06T07:21:00.011916+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.009601+0000","last_change":"2024-03-06T07:21:00.009601+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.009601+0000","last_undegraded":"2024-03-06T07:21:00.009601+0000","last_fullsized":"2024-03-06T07:21:00.009601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017495+0000","last_change":"2024-03-06T07:21:00.014388+0000","last_active":"2024-03-06T07:21:00.017495+0000","last_peered":"2024-03-06T07:21:00.017495+0000","last_clean":"2024-03-06T07:21:00.017495+0000","last_became_active":"2024-03-06T07:21:00.014118+0000","last_became_peered":"2024-03-06T07:21:00.014118+0000","last_unstale":"2024-03-06T07:21:00.017495+0000","last_undegraded":"2024-03-06T07:21:00.017495+0000","last_fullsized":"2024-03-06T07:21:00.017495+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:16.342176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999388+0000","last_change":"2024-03-06T07:20:59.999388+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999388+0000","last_undegraded":"2024-03-06T07:20:59.999388+0000","last_fullsized":"2024-03-06T07:20:59.999388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017297+0000","last_change":"2024-03-06T07:21:00.014221+0000","last_active":"2024-03-06T07:21:00.017297+0000","last_peered":"2024-03-06T07:21:00.017297+0000","last_clean":"2024-03-06T07:21:00.017297+0000","last_became_active":"2024-03-06T07:21:00.013921+0000","last_became_peered":"2024-03-06T07:21:00.013921+0000","last_unstale":"2024-03-06T07:21:00.017297+0000","last_undegraded":"2024-03-06T07:21:00.017297+0000","last_fullsized":"2024-03-06T07:21:00.017297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996517+0000","last_change":"2024-03-06T07:20:59.996517+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996517+0000","last_undegraded":"2024-03-06T07:20:59.996517+0000","last_fullsized":"2024-03-06T07:20:59.996517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015389+0000","last_change":"2024-03-06T07:21:00.010506+0000","last_active":"2024-03-06T07:21:00.015389+0000","last_peered":"2024-03-06T07:21:00.015389+0000","last_clean":"2024-03-06T07:21:00.015389+0000","last_became_active":"2024-03-06T07:21:00.010172+0000","last_became_peered":"2024-03-06T07:21:00.010172+0000","last_unstale":"2024-03-06T07:21:00.015389+0000","last_undegraded":"2024-03-06T07:21:00.015389+0000","last_fullsized":"2024-03-06T07:21:00.015389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:46.431814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998304+0000","last_change":"2024-03-06T07:20:59.998304+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998304+0000","last_undegraded":"2024-03-06T07:20:59.998304+0000","last_fullsized":"2024-03-06T07:20:59.998304+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.009278+0000","last_change":"2024-03-06T07:21:00.005310+0000","last_active":"2024-03-06T07:21:00.009278+0000","last_peered":"2024-03-06T07:21:00.009278+0000","last_clean":"2024-03-06T07:21:00.009278+0000","last_became_active":"2024-03-06T07:21:00.004999+0000","last_became_peered":"2024-03-06T07:21:00.004999+0000","last_unstale":"2024-03-06T07:21:00.009278+0000","last_undegraded":"2024-03-06T07:21:00.009278+0000","last_fullsized":"2024-03-06T07:21:00.009278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999103+0000","last_change":"2024-03-06T07:20:59.999103+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999103+0000","last_undegraded":"2024-03-06T07:20:59.999103+0000","last_fullsized":"2024-03-06T07:20:59.999103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018327+0000","last_change":"2024-03-06T07:21:00.011969+0000","last_active":"2024-03-06T07:21:00.018327+0000","last_peered":"2024-03-06T07:21:00.018327+0000","last_clean":"2024-03-06T07:21:00.018327+0000","last_became_active":"2024-03-06T07:21:00.011741+0000","last_became_peered":"2024-03-06T07:21:00.011741+0000","last_unstale":"2024-03-06T07:21:00.018327+0000","last_undegraded":"2024-03-06T07:21:00.018327+0000","last_fullsized":"2024-03-06T07:21:00.018327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:12.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.004368+0000","last_change":"2024-03-06T07:21:00.004368+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.004368+0000","last_undegraded":"2024-03-06T07:21:00.004368+0000","last_fullsized":"2024-03-06T07:21:00.004368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018480+0000","last_change":"2024-03-06T07:21:00.013456+0000","last_active":"2024-03-06T07:21:00.018480+0000","last_peered":"2024-03-06T07:21:00.018480+0000","last_clean":"2024-03-06T07:21:00.018480+0000","last_became_active":"2024-03-06T07:21:00.013184+0000","last_became_peered":"2024-03-06T07:21:00.013184+0000","last_unstale":"2024-03-06T07:21:00.018480+0000","last_undegraded":"2024-03-06T07:21:00.018480+0000","last_fullsized":"2024-03-06T07:21:00.018480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.000640+0000","last_change":"2024-03-06T07:21:00.000640+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.000640+0000","last_undegraded":"2024-03-06T07:21:00.000640+0000","last_fullsized":"2024-03-06T07:21:00.000640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005305+0000","last_change":"2024-03-06T07:21:00.003577+0000","last_active":"2024-03-06T07:21:00.005305+0000","last_peered":"2024-03-06T07:21:00.005305+0000","last_clean":"2024-03-06T07:21:00.005305+0000","last_became_active":"2024-03-06T07:21:00.003294+0000","last_became_peered":"2024-03-06T07:21:00.003294+0000","last_unstale":"2024-03-06T07:21:00.005305+0000","last_undegraded":"2024-03-06T07:21:00.005305+0000","last_fullsized":"2024-03-06T07:21:00.005305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:02.580999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996923+0000","last_change":"2024-03-06T07:20:59.996923+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996923+0000","last_undegraded":"2024-03-06T07:20:59.996923+0000","last_fullsized":"2024-03-06T07:20:59.996923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016900+0000","last_change":"2024-03-06T07:21:00.012331+0000","last_active":"2024-03-06T07:21:00.016900+0000","last_peered":"2024-03-06T07:21:00.016900+0000","last_clean":"2024-03-06T07:21:00.016900+0000","last_became_active":"2024-03-06T07:21:00.011994+0000","last_became_peered":"2024-03-06T07:21:00.011994+0000","last_unstale":"2024-03-06T07:21:00.016900+0000","last_undegraded":"2024-03-06T07:21:00.016900+0000","last_fullsized":"2024-03-06T07:21:00.016900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996058+0000","last_change":"2024-03-06T07:20:59.996058+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996058+0000","last_undegraded":"2024-03-06T07:20:59.996058+0000","last_fullsized":"2024-03-06T07:20:59.996058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011233+0000","last_change":"2024-03-06T07:21:00.008859+0000","last_active":"2024-03-06T07:21:00.011233+0000","last_peered":"2024-03-06T07:21:00.011233+0000","last_clean":"2024-03-06T07:21:00.011233+0000","last_became_active":"2024-03-06T07:21:00.008565+0000","last_became_peered":"2024-03-06T07:21:00.008565+0000","last_unstale":"2024-03-06T07:21:00.011233+0000","last_undegraded":"2024-03-06T07:21:00.011233+0000","last_fullsized":"2024-03-06T07:21:00.011233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:03.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.002934+0000","last_change":"2024-03-06T07:21:00.002934+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.002934+0000","last_undegraded":"2024-03-06T07:21:00.002934+0000","last_fullsized":"2024-03-06T07:21:00.002934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.010375+0000","last_change":"2024-03-06T07:21:00.007836+0000","last_active":"2024-03-06T07:21:00.010375+0000","last_peered":"2024-03-06T07:21:00.010375+0000","last_clean":"2024-03-06T07:21:00.010375+0000","last_became_active":"2024-03-06T07:21:00.006276+0000","last_became_peered":"2024-03-06T07:21:00.006276+0000","last_unstale":"2024-03-06T07:21:00.010375+0000","last_undegraded":"2024-03-06T07:21:00.010375+0000","last_fullsized":"2024-03-06T07:21:00.010375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:26.980322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.999890+0000","last_change":"2024-03-06T07:20:59.999890+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.999890+0000","last_undegraded":"2024-03-06T07:20:59.999890+0000","last_fullsized":"2024-03-06T07:20:59.999890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018160+0000","last_change":"2024-03-06T07:21:00.011278+0000","last_active":"2024-03-06T07:21:00.018160+0000","last_peered":"2024-03-06T07:21:00.018160+0000","last_clean":"2024-03-06T07:21:00.018160+0000","last_became_active":"2024-03-06T07:21:00.011018+0000","last_became_peered":"2024-03-06T07:21:00.011018+0000","last_unstale":"2024-03-06T07:21:00.018160+0000","last_undegraded":"2024-03-06T07:21:00.018160+0000","last_fullsized":"2024-03-06T07:21:00.018160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.997087+0000","last_change":"2024-03-06T07:20:59.997087+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.997087+0000","last_undegraded":"2024-03-06T07:20:59.997087+0000","last_fullsized":"2024-03-06T07:20:59.997087+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.014862+0000","last_change":"2024-03-06T07:21:00.013699+0000","last_active":"2024-03-06T07:21:00.014862+0000","last_peered":"2024-03-06T07:21:00.014862+0000","last_clean":"2024-03-06T07:21:00.014862+0000","last_became_active":"2024-03-06T07:21:00.013360+0000","last_became_peered":"2024-03-06T07:21:00.013360+0000","last_unstale":"2024-03-06T07:21:00.014862+0000","last_undegraded":"2024-03-06T07:21:00.014862+0000","last_fullsized":"2024-03-06T07:21:00.014862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:06.072955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001334+0000","last_change":"2024-03-06T07:21:00.001334+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001334+0000","last_undegraded":"2024-03-06T07:21:00.001334+0000","last_fullsized":"2024-03-06T07:21:00.001334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018669+0000","last_change":"2024-03-06T07:21:00.015827+0000","last_active":"2024-03-06T07:21:00.018669+0000","last_peered":"2024-03-06T07:21:00.018669+0000","last_clean":"2024-03-06T07:21:00.018669+0000","last_became_active":"2024-03-06T07:21:00.015565+0000","last_became_peered":"2024-03-06T07:21:00.015565+0000","last_unstale":"2024-03-06T07:21:00.018669+0000","last_undegraded":"2024-03-06T07:21:00.018669+0000","last_fullsized":"2024-03-06T07:21:00.018669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:44.990676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996060+0000","last_change":"2024-03-06T07:20:59.996060+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996060+0000","last_undegraded":"2024-03-06T07:20:59.996060+0000","last_fullsized":"2024-03-06T07:20:59.996060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.011205+0000","last_change":"2024-03-06T07:21:00.009047+0000","last_active":"2024-03-06T07:21:00.011205+0000","last_peered":"2024-03-06T07:21:00.011205+0000","last_clean":"2024-03-06T07:21:00.011205+0000","last_became_active":"2024-03-06T07:21:00.008803+0000","last_became_peered":"2024-03-06T07:21:00.008803+0000","last_unstale":"2024-03-06T07:21:00.011205+0000","last_undegraded":"2024-03-06T07:21:00.011205+0000","last_fullsized":"2024-03-06T07:21:00.011205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:17.432587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.998464+0000","last_change":"2024-03-06T07:20:59.998464+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.998464+0000","last_undegraded":"2024-03-06T07:20:59.998464+0000","last_fullsized":"2024-03-06T07:20:59.998464+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.016624+0000","last_change":"2024-03-06T07:21:00.015433+0000","last_active":"2024-03-06T07:21:00.016624+0000","last_peered":"2024-03-06T07:21:00.016624+0000","last_clean":"2024-03-06T07:21:00.016624+0000","last_became_active":"2024-03-06T07:21:00.015156+0000","last_became_peered":"2024-03-06T07:21:00.015156+0000","last_unstale":"2024-03-06T07:21:00.016624+0000","last_undegraded":"2024-03-06T07:21:00.016624+0000","last_fullsized":"2024-03-06T07:21:00.016624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.005409+0000","last_change":"2024-03-06T07:21:00.005409+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.005409+0000","last_undegraded":"2024-03-06T07:21:00.005409+0000","last_fullsized":"2024-03-06T07:21:00.005409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017074+0000","last_change":"2024-03-06T07:21:00.016544+0000","last_active":"2024-03-06T07:21:00.017074+0000","last_peered":"2024-03-06T07:21:00.017074+0000","last_clean":"2024-03-06T07:21:00.017074+0000","last_became_active":"2024-03-06T07:21:00.015894+0000","last_became_peered":"2024-03-06T07:21:00.015894+0000","last_unstale":"2024-03-06T07:21:00.017074+0000","last_undegraded":"2024-03-06T07:21:00.017074+0000","last_fullsized":"2024-03-06T07:21:00.017074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.008546+0000","last_change":"2024-03-06T07:21:00.008546+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.008546+0000","last_undegraded":"2024-03-06T07:21:00.008546+0000","last_fullsized":"2024-03-06T07:21:00.008546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.015354+0000","last_change":"2024-03-06T07:21:00.010211+0000","last_active":"2024-03-06T07:21:00.015354+0000","last_peered":"2024-03-06T07:21:00.015354+0000","last_clean":"2024-03-06T07:21:00.015354+0000","last_became_active":"2024-03-06T07:21:00.009915+0000","last_became_peered":"2024-03-06T07:21:00.009915+0000","last_unstale":"2024-03-06T07:21:00.015354+0000","last_undegraded":"2024-03-06T07:21:00.015354+0000","last_fullsized":"2024-03-06T07:21:00.015354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:25:12.857843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.001003+0000","last_change":"2024-03-06T07:21:00.001003+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.001003+0000","last_undegraded":"2024-03-06T07:21:00.001003+0000","last_fullsized":"2024-03-06T07:21:00.001003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.018391+0000","last_change":"2024-03-06T07:21:00.011094+0000","last_active":"2024-03-06T07:21:00.018391+0000","last_peered":"2024-03-06T07:21:00.018391+0000","last_clean":"2024-03-06T07:21:00.018391+0000","last_became_active":"2024-03-06T07:21:00.010457+0000","last_became_peered":"2024-03-06T07:21:00.010457+0000","last_unstale":"2024-03-06T07:21:00.018391+0000","last_undegraded":"2024-03-06T07:21:00.018391+0000","last_fullsized":"2024-03-06T07:21:00.018391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:41.550434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:20:59.996256+0000","last_change":"2024-03-06T07:20:59.996256+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:20:59.996256+0000","last_undegraded":"2024-03-06T07:20:59.996256+0000","last_fullsized":"2024-03-06T07:20:59.996256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.017529+0000","last_change":"2024-03-06T07:21:00.014362+0000","last_active":"2024-03-06T07:21:00.017529+0000","last_peered":"2024-03-06T07:21:00.017529+0000","last_clean":"2024-03-06T07:21:00.017529+0000","last_became_active":"2024-03-06T07:21:00.014072+0000","last_became_peered":"2024-03-06T07:21:00.014072+0000","last_unstale":"2024-03-06T07:21:00.017529+0000","last_undegraded":"2024-03-06T07:21:00.017529+0000","last_fullsized":"2024-03-06T07:21:00.017529+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:12.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-03-06T07:21:00.003607+0000","last_change":"2024-03-06T07:21:00.003607+0000","last_active":"2024-03-06T07:20:59.984090+0000","last_peered":"2024-03-06T07:20:59.984090+0000","last_clean":"2024-03-06T07:20:59.984090+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:00.003607+0000","last_undegraded":"2024-03-06T07:21:00.003607+0000","last_fullsized":"2024-03-06T07:21:00.003607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:21:00.005867+0000","last_change":"2024-03-06T07:21:00.002103+0000","last_active":"2024-03-06T07:21:00.005867+0000","last_peered":"2024-03-06T07:21:00.005867+0000","last_clean":"2024-03-06T07:21:00.005867+0000","last_became_active":"2024-03-06T07:21:00.001791+0000","last_became_peered":"2024-03-06T07:21:00.001791+0000","last_unstale":"2024-03-06T07:21:00.005867+0000","last_undegraded":"2024-03-06T07:21:00.005867+0000","last_fullsized":"2024-03-06T07:21:00.005867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-06T07:20:59.995260+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:20:59.995260+0000","last_peered":"2024-03-06T07:20:59.995260+0000","last_clean":"2024-03-06T07:20:59.995260+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:20:59.995260+0000","last_undegraded":"2024-03-06T07:20:59.995260+0000","last_fullsized":"2024-03-06T07:20:59.995260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":162786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":753154,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:08.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:08.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.564+0000 7f3a801f7640 1 Processor -- start 2024-03-06T07:21:08.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.564+0000 7f3a801f7640 1 -- start start 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a801f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a7806da40 0x7f3a78155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a801f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a78158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a801f7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a78156050 con 0x7f3a781592d0 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a801f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7806de20 con 0x7f3a7806da40 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a801f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78072630 con 0x7f3a780c6780 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a7df93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a7806da40 0x7f3a78155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a7d792640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a78158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a7df93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a7806da40 0x7f3a78155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60354/0 (socket says 172.21.15.26:60354) 2024-03-06T07:21:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a7e794640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3a781592d0 0x7f3a780c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60920/0 (socket says 172.21.15.26:60920) 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a7df93640 1 -- 172.21.15.26:0/3686873033 learned_addr learned my addr 172.21.15.26:0/3686873033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.565+0000 7f3a7d792640 1 --2- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a78158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:45592/0 (socket says 172.21.15.26:45592) 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.566+0000 7f3a7df93640 1 -- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 msgr2=0x7f3a78158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.566+0000 7f3a7df93640 1 --2- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a78158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.566+0000 7f3a7df93640 1 -- 172.21.15.26:0/3686873033 >> v1:172.21.15.117:6789/0 conn(0x7f3a781592d0 legacy=0x7f3a780c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.566+0000 7f3a7df93640 1 -- 172.21.15.26:0/3686873033 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a78072910 con 0x7f3a7806da40 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.566+0000 7f3a7d792640 1 --2- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a78158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.566+0000 7f3a7df93640 1 --2- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78155b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009870 tx=0x7f3a7002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e8adcdde32ad9bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a7cf91640 1 -- 172.21.15.26:0/3686873033 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70035020 con 0x7f3a7806da40 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a7cf91640 1 -- 172.21.15.26:0/3686873033 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a70031640 con 0x7f3a7806da40 2024-03-06T07:21:08.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a7cf91640 1 -- 172.21.15.26:0/3686873033 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7002d040 con 0x7f3a7806da40 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 -- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 msgr2=0x7f3a78155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78155b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009870 tx=0x7f3a7002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 -- 172.21.15.26:0/3686873033 shutdown_connections 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a78158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3686873033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78155b10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 -- 172.21.15.26:0/3686873033 >> 172.21.15.26:0/3686873033 conn(0x7f3a780aa630 msgr2=0x7f3a780aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 -- 172.21.15.26:0/3686873033 shutdown_connections 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.567+0000 7f3a801f7640 1 -- 172.21.15.26:0/3686873033 wait complete. 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.568+0000 7f3a801f7640 1 Processor -- start 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.568+0000 7f3a801f7640 1 -- start start 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a801f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a801f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a780a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a801f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a781592d0 0x7f3a78156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a801f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780c3990 con 0x7f3a781592d0 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a801f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78158890 con 0x7f3a7806da40 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a801f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780c3b10 con 0x7f3a780c6780 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a7e794640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a781592d0 0x7f3a78156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a7e794640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a781592d0 0x7f3a78156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36862/0 (socket says 172.21.15.26:36862) 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a7df93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a7e794640 1 -- 172.21.15.26:0/3560666491 learned_addr learned my addr 172.21.15.26:0/3560666491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.569+0000 7f3a7d792640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a780a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a7e794640 1 -- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 msgr2=0x7f3a780a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a7e794640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a780a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a7e794640 1 -- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 msgr2=0x7f3a78158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a7e794640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a7e794640 1 -- 172.21.15.26:0/3560666491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780ab790 con 0x7f3a781592d0 2024-03-06T07:21:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a7df93640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a7e794640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a781592d0 0x7f3a78156e00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c730 tx=0x7f3a6800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a767fc640 1 -- 172.21.15.26:0/3560666491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68017070 con 0x7f3a781592d0 2024-03-06T07:21:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a767fc640 1 -- 172.21.15.26:0/3560666491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800ce40 con 0x7f3a781592d0 2024-03-06T07:21:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a78157be0 con 0x7f3a781592d0 2024-03-06T07:21:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a767fc640 1 -- 172.21.15.26:0/3560666491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800a7e0 con 0x7f3a781592d0 2024-03-06T07:21:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.570+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a780a9dd0 con 0x7f3a781592d0 2024-03-06T07:21:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.571+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a780ab960 con 0x7f3a781592d0 2024-03-06T07:21:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.572+0000 7f3a767fc640 1 -- 172.21.15.26:0/3560666491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a6801a070 con 0x7f3a781592d0 2024-03-06T07:21:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.572+0000 7f3a767fc640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a6406b4d0 0x7f3a6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.572+0000 7f3a767fc640 1 -- 172.21.15.26:0/3560666491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a68089810 con 0x7f3a781592d0 2024-03-06T07:21:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.572+0000 7f3a7df93640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a6406b4d0 0x7f3a6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.573+0000 7f3a7df93640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a6406b4d0 0x7f3a6406d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3a70036000 tx=0x7f3a7003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.576+0000 7f3a767fc640 1 -- 172.21.15.26:0/3560666491 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a78156050 con 0x7f3a781592d0 2024-03-06T07:21:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.754+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a7804f330 con 0x7f3a6406b4d0 2024-03-06T07:21:08.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.760+0000 7f3a767fc640 1 -- 172.21.15.26:0/3560666491 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252032 (secure 0 0 0) 0x7f3a70085890 con 0x7f3a6406b4d0 2024-03-06T07:21:08.765 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:08.767 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:08.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a6406b4d0 msgr2=0x7f3a6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a6406b4d0 0x7f3a6406d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3a70036000 tx=0x7f3a7003c040 comp rx=0 tx=0).stop 2024-03-06T07:21:08.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a781592d0 msgr2=0x7f3a78156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a781592d0 0x7f3a78156e00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c730 tx=0x7f3a6800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 shutdown_connections 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a6406b4d0 0x7f3a6406d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a781592d0 0x7f3a78156e00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a780c6780 0x7f3a780a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 --2- 172.21.15.26:0/3560666491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7806da40 0x7f3a78158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.763+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 >> 172.21.15.26:0/3560666491 conn(0x7f3a780aa630 msgr2=0x7f3a780c3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.764+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 shutdown_connections 2024-03-06T07:21:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.764+0000 7f3a801f7640 1 -- 172.21.15.26:0/3560666491 wait complete. 2024-03-06T07:21:08.777 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-03-06T07:21:07.043404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236340,"kb_used_data":3520,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738380,"statfs":{"total":773094113280,"available":772852101120,"internally_reserved":0,"allocated":3604480,"data_stored":2652052,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.048459"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304827+0000","last_change":"2024-03-06T07:21:01.015437+0000","last_active":"2024-03-06T07:21:03.304827+0000","last_peered":"2024-03-06T07:21:03.304827+0000","last_clean":"2024-03-06T07:21:03.304827+0000","last_became_active":"2024-03-06T07:21:01.015170+0000","last_became_peered":"2024-03-06T07:21:01.015170+0000","last_unstale":"2024-03-06T07:21:03.304827+0000","last_undegraded":"2024-03-06T07:21:03.304827+0000","last_fullsized":"2024-03-06T07:21:03.304827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:41.264943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304598+0000","last_change":"2024-03-06T07:21:01.014212+0000","last_active":"2024-03-06T07:21:03.304598+0000","last_peered":"2024-03-06T07:21:03.304598+0000","last_clean":"2024-03-06T07:21:03.304598+0000","last_became_active":"2024-03-06T07:21:01.013947+0000","last_became_peered":"2024-03-06T07:21:01.013947+0000","last_unstale":"2024-03-06T07:21:03.304598+0000","last_undegraded":"2024-03-06T07:21:03.304598+0000","last_fullsized":"2024-03-06T07:21:03.304598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:35:28.400753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203684+0000","last_change":"2024-03-06T07:21:01.018803+0000","last_active":"2024-03-06T07:21:02.203684+0000","last_peered":"2024-03-06T07:21:02.203684+0000","last_clean":"2024-03-06T07:21:02.203684+0000","last_became_active":"2024-03-06T07:21:01.018501+0000","last_became_peered":"2024-03-06T07:21:01.018501+0000","last_unstale":"2024-03-06T07:21:02.203684+0000","last_undegraded":"2024-03-06T07:21:02.203684+0000","last_fullsized":"2024-03-06T07:21:02.203684+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:52.879221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010213+0000","last_change":"2024-03-06T07:21:01.010748+0000","last_active":"2024-03-06T07:21:02.010213+0000","last_peered":"2024-03-06T07:21:02.010213+0000","last_clean":"2024-03-06T07:21:02.010213+0000","last_became_active":"2024-03-06T07:21:01.010460+0000","last_became_peered":"2024-03-06T07:21:01.010460+0000","last_unstale":"2024-03-06T07:21:02.010213+0000","last_undegraded":"2024-03-06T07:21:02.010213+0000","last_fullsized":"2024-03-06T07:21:02.010213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:05.620348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.009926+0000","last_change":"2024-03-06T07:21:01.021734+0000","last_active":"2024-03-06T07:21:02.009926+0000","last_peered":"2024-03-06T07:21:02.009926+0000","last_clean":"2024-03-06T07:21:02.009926+0000","last_became_active":"2024-03-06T07:21:01.016853+0000","last_became_peered":"2024-03-06T07:21:01.016853+0000","last_unstale":"2024-03-06T07:21:02.009926+0000","last_undegraded":"2024-03-06T07:21:02.009926+0000","last_fullsized":"2024-03-06T07:21:02.009926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:48.437266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.303950+0000","last_change":"2024-03-06T07:21:01.008669+0000","last_active":"2024-03-06T07:21:03.303950+0000","last_peered":"2024-03-06T07:21:03.303950+0000","last_clean":"2024-03-06T07:21:03.303950+0000","last_became_active":"2024-03-06T07:21:01.008414+0000","last_became_peered":"2024-03-06T07:21:01.008414+0000","last_unstale":"2024-03-06T07:21:03.303950+0000","last_undegraded":"2024-03-06T07:21:03.303950+0000","last_fullsized":"2024-03-06T07:21:03.303950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:57.516671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010836+0000","last_change":"2024-03-06T07:21:01.022748+0000","last_active":"2024-03-06T07:21:02.010836+0000","last_peered":"2024-03-06T07:21:02.010836+0000","last_clean":"2024-03-06T07:21:02.010836+0000","last_became_active":"2024-03-06T07:21:01.022490+0000","last_became_peered":"2024-03-06T07:21:01.022490+0000","last_unstale":"2024-03-06T07:21:02.010836+0000","last_undegraded":"2024-03-06T07:21:02.010836+0000","last_fullsized":"2024-03-06T07:21:02.010836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:46.618359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304148+0000","last_change":"2024-03-06T07:21:01.021170+0000","last_active":"2024-03-06T07:21:03.304148+0000","last_peered":"2024-03-06T07:21:03.304148+0000","last_clean":"2024-03-06T07:21:03.304148+0000","last_became_active":"2024-03-06T07:21:01.020412+0000","last_became_peered":"2024-03-06T07:21:01.020412+0000","last_unstale":"2024-03-06T07:21:03.304148+0000","last_undegraded":"2024-03-06T07:21:03.304148+0000","last_fullsized":"2024-03-06T07:21:03.304148+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:47.082638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203871+0000","last_change":"2024-03-06T07:21:01.020258+0000","last_active":"2024-03-06T07:21:02.203871+0000","last_peered":"2024-03-06T07:21:02.203871+0000","last_clean":"2024-03-06T07:21:02.203871+0000","last_became_active":"2024-03-06T07:21:01.019069+0000","last_became_peered":"2024-03-06T07:21:01.019069+0000","last_unstale":"2024-03-06T07:21:02.203871+0000","last_undegraded":"2024-03-06T07:21:02.203871+0000","last_fullsized":"2024-03-06T07:21:02.203871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:47.847882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011107+0000","last_change":"2024-03-06T07:21:01.027335+0000","last_active":"2024-03-06T07:21:02.011107+0000","last_peered":"2024-03-06T07:21:02.011107+0000","last_clean":"2024-03-06T07:21:02.011107+0000","last_became_active":"2024-03-06T07:21:01.027073+0000","last_became_peered":"2024-03-06T07:21:01.027073+0000","last_unstale":"2024-03-06T07:21:02.011107+0000","last_undegraded":"2024-03-06T07:21:02.011107+0000","last_fullsized":"2024-03-06T07:21:02.011107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:50:05.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011477+0000","last_change":"2024-03-06T07:21:01.028780+0000","last_active":"2024-03-06T07:21:02.011477+0000","last_peered":"2024-03-06T07:21:02.011477+0000","last_clean":"2024-03-06T07:21:02.011477+0000","last_became_active":"2024-03-06T07:21:01.027943+0000","last_became_peered":"2024-03-06T07:21:01.027943+0000","last_unstale":"2024-03-06T07:21:02.011477+0000","last_undegraded":"2024-03-06T07:21:02.011477+0000","last_fullsized":"2024-03-06T07:21:02.011477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:21.855614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203648+0000","last_change":"2024-03-06T07:21:01.012656+0000","last_active":"2024-03-06T07:21:02.203648+0000","last_peered":"2024-03-06T07:21:02.203648+0000","last_clean":"2024-03-06T07:21:02.203648+0000","last_became_active":"2024-03-06T07:21:01.012360+0000","last_became_peered":"2024-03-06T07:21:01.012360+0000","last_unstale":"2024-03-06T07:21:02.203648+0000","last_undegraded":"2024-03-06T07:21:02.203648+0000","last_fullsized":"2024-03-06T07:21:02.203648+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:04.791562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011050+0000","last_change":"2024-03-06T07:21:01.021907+0000","last_active":"2024-03-06T07:21:02.011050+0000","last_peered":"2024-03-06T07:21:02.011050+0000","last_clean":"2024-03-06T07:21:02.011050+0000","last_became_active":"2024-03-06T07:21:01.021648+0000","last_became_peered":"2024-03-06T07:21:01.021648+0000","last_unstale":"2024-03-06T07:21:02.011050+0000","last_undegraded":"2024-03-06T07:21:02.011050+0000","last_fullsized":"2024-03-06T07:21:02.011050+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:45:09.701793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304709+0000","last_change":"2024-03-06T07:21:01.018800+0000","last_active":"2024-03-06T07:21:03.304709+0000","last_peered":"2024-03-06T07:21:03.304709+0000","last_clean":"2024-03-06T07:21:03.304709+0000","last_became_active":"2024-03-06T07:21:01.018521+0000","last_became_peered":"2024-03-06T07:21:01.018521+0000","last_unstale":"2024-03-06T07:21:03.304709+0000","last_undegraded":"2024-03-06T07:21:03.304709+0000","last_fullsized":"2024-03-06T07:21:03.304709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:45.992777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011452+0000","last_change":"2024-03-06T07:21:01.013155+0000","last_active":"2024-03-06T07:21:02.011452+0000","last_peered":"2024-03-06T07:21:02.011452+0000","last_clean":"2024-03-06T07:21:02.011452+0000","last_became_active":"2024-03-06T07:21:01.012869+0000","last_became_peered":"2024-03-06T07:21:01.012869+0000","last_unstale":"2024-03-06T07:21:02.011452+0000","last_undegraded":"2024-03-06T07:21:02.011452+0000","last_fullsized":"2024-03-06T07:21:02.011452+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:53:32.534135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304058+0000","last_change":"2024-03-06T07:21:01.020718+0000","last_active":"2024-03-06T07:21:03.304058+0000","last_peered":"2024-03-06T07:21:03.304058+0000","last_clean":"2024-03-06T07:21:03.304058+0000","last_became_active":"2024-03-06T07:21:01.020449+0000","last_became_peered":"2024-03-06T07:21:01.020449+0000","last_unstale":"2024-03-06T07:21:03.304058+0000","last_undegraded":"2024-03-06T07:21:03.304058+0000","last_fullsized":"2024-03-06T07:21:03.304058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:47.847882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305307+0000","last_change":"2024-03-06T07:21:01.021486+0000","last_active":"2024-03-06T07:21:03.305307+0000","last_peered":"2024-03-06T07:21:03.305307+0000","last_clean":"2024-03-06T07:21:03.305307+0000","last_became_active":"2024-03-06T07:21:01.021221+0000","last_became_peered":"2024-03-06T07:21:01.021221+0000","last_unstale":"2024-03-06T07:21:03.305307+0000","last_undegraded":"2024-03-06T07:21:03.305307+0000","last_fullsized":"2024-03-06T07:21:03.305307+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:50:05.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305095+0000","last_change":"2024-03-06T07:21:01.024503+0000","last_active":"2024-03-06T07:21:03.305095+0000","last_peered":"2024-03-06T07:21:03.305095+0000","last_clean":"2024-03-06T07:21:03.305095+0000","last_became_active":"2024-03-06T07:21:01.024218+0000","last_became_peered":"2024-03-06T07:21:01.024218+0000","last_unstale":"2024-03-06T07:21:03.305095+0000","last_undegraded":"2024-03-06T07:21:03.305095+0000","last_fullsized":"2024-03-06T07:21:03.305095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:51:35.794855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010321+0000","last_change":"2024-03-06T07:21:01.004129+0000","last_active":"2024-03-06T07:21:02.010321+0000","last_peered":"2024-03-06T07:21:02.010321+0000","last_clean":"2024-03-06T07:21:02.010321+0000","last_became_active":"2024-03-06T07:21:01.003842+0000","last_became_peered":"2024-03-06T07:21:01.003842+0000","last_unstale":"2024-03-06T07:21:02.010321+0000","last_undegraded":"2024-03-06T07:21:02.010321+0000","last_fullsized":"2024-03-06T07:21:02.010321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:07.709990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304917+0000","last_change":"2024-03-06T07:21:01.023815+0000","last_active":"2024-03-06T07:21:03.304917+0000","last_peered":"2024-03-06T07:21:03.304917+0000","last_clean":"2024-03-06T07:21:03.304917+0000","last_became_active":"2024-03-06T07:21:01.023548+0000","last_became_peered":"2024-03-06T07:21:01.023548+0000","last_unstale":"2024-03-06T07:21:03.304917+0000","last_undegraded":"2024-03-06T07:21:03.304917+0000","last_fullsized":"2024-03-06T07:21:03.304917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:13.991090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010951+0000","last_change":"2024-03-06T07:21:01.018527+0000","last_active":"2024-03-06T07:21:02.010951+0000","last_peered":"2024-03-06T07:21:02.010951+0000","last_clean":"2024-03-06T07:21:02.010951+0000","last_became_active":"2024-03-06T07:21:01.018195+0000","last_became_peered":"2024-03-06T07:21:01.018195+0000","last_unstale":"2024-03-06T07:21:02.010951+0000","last_undegraded":"2024-03-06T07:21:02.010951+0000","last_fullsized":"2024-03-06T07:21:02.010951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:59.941710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305299+0000","last_change":"2024-03-06T07:21:01.013596+0000","last_active":"2024-03-06T07:21:03.305299+0000","last_peered":"2024-03-06T07:21:03.305299+0000","last_clean":"2024-03-06T07:21:03.305299+0000","last_became_active":"2024-03-06T07:21:01.013323+0000","last_became_peered":"2024-03-06T07:21:01.013323+0000","last_unstale":"2024-03-06T07:21:03.305299+0000","last_undegraded":"2024-03-06T07:21:03.305299+0000","last_fullsized":"2024-03-06T07:21:03.305299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:13.991090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304128+0000","last_change":"2024-03-06T07:21:01.002791+0000","last_active":"2024-03-06T07:21:03.304128+0000","last_peered":"2024-03-06T07:21:03.304128+0000","last_clean":"2024-03-06T07:21:03.304128+0000","last_became_active":"2024-03-06T07:21:01.002524+0000","last_became_peered":"2024-03-06T07:21:01.002524+0000","last_unstale":"2024-03-06T07:21:03.304128+0000","last_undegraded":"2024-03-06T07:21:03.304128+0000","last_fullsized":"2024-03-06T07:21:03.304128+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:45.728944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.009901+0000","last_change":"2024-03-06T07:21:01.011756+0000","last_active":"2024-03-06T07:21:02.009901+0000","last_peered":"2024-03-06T07:21:02.009901+0000","last_clean":"2024-03-06T07:21:02.009901+0000","last_became_active":"2024-03-06T07:21:01.010829+0000","last_became_peered":"2024-03-06T07:21:01.010829+0000","last_unstale":"2024-03-06T07:21:02.009901+0000","last_undegraded":"2024-03-06T07:21:02.009901+0000","last_fullsized":"2024-03-06T07:21:02.009901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:47.433915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304847+0000","last_change":"2024-03-06T07:21:01.018103+0000","last_active":"2024-03-06T07:21:03.304847+0000","last_peered":"2024-03-06T07:21:03.304847+0000","last_clean":"2024-03-06T07:21:03.304847+0000","last_became_active":"2024-03-06T07:21:01.017815+0000","last_became_peered":"2024-03-06T07:21:01.017815+0000","last_unstale":"2024-03-06T07:21:03.304847+0000","last_undegraded":"2024-03-06T07:21:03.304847+0000","last_fullsized":"2024-03-06T07:21:03.304847+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:17.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304568+0000","last_change":"2024-03-06T07:21:01.024013+0000","last_active":"2024-03-06T07:21:03.304568+0000","last_peered":"2024-03-06T07:21:03.304568+0000","last_clean":"2024-03-06T07:21:03.304568+0000","last_became_active":"2024-03-06T07:21:01.023769+0000","last_became_peered":"2024-03-06T07:21:01.023769+0000","last_unstale":"2024-03-06T07:21:03.304568+0000","last_undegraded":"2024-03-06T07:21:03.304568+0000","last_fullsized":"2024-03-06T07:21:03.304568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:04.791562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304973+0000","last_change":"2024-03-06T07:21:01.017164+0000","last_active":"2024-03-06T07:21:03.304973+0000","last_peered":"2024-03-06T07:21:03.304973+0000","last_clean":"2024-03-06T07:21:03.304973+0000","last_became_active":"2024-03-06T07:21:01.016798+0000","last_became_peered":"2024-03-06T07:21:01.016798+0000","last_unstale":"2024-03-06T07:21:03.304973+0000","last_undegraded":"2024-03-06T07:21:03.304973+0000","last_fullsized":"2024-03-06T07:21:03.304973+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:13.153890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011226+0000","last_change":"2024-03-06T07:21:01.026619+0000","last_active":"2024-03-06T07:21:02.011226+0000","last_peered":"2024-03-06T07:21:02.011226+0000","last_clean":"2024-03-06T07:21:02.011226+0000","last_became_active":"2024-03-06T07:21:01.026368+0000","last_became_peered":"2024-03-06T07:21:01.026368+0000","last_unstale":"2024-03-06T07:21:02.011226+0000","last_undegraded":"2024-03-06T07:21:02.011226+0000","last_fullsized":"2024-03-06T07:21:02.011226+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:30:54.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304705+0000","last_change":"2024-03-06T07:21:01.016772+0000","last_active":"2024-03-06T07:21:03.304705+0000","last_peered":"2024-03-06T07:21:03.304705+0000","last_clean":"2024-03-06T07:21:03.304705+0000","last_became_active":"2024-03-06T07:21:01.016484+0000","last_became_peered":"2024-03-06T07:21:01.016484+0000","last_unstale":"2024-03-06T07:21:03.304705+0000","last_undegraded":"2024-03-06T07:21:03.304705+0000","last_fullsized":"2024-03-06T07:21:03.304705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:08.272929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304368+0000","last_change":"2024-03-06T07:21:01.018218+0000","last_active":"2024-03-06T07:21:03.304368+0000","last_peered":"2024-03-06T07:21:03.304368+0000","last_clean":"2024-03-06T07:21:03.304368+0000","last_became_active":"2024-03-06T07:21:01.017889+0000","last_became_peered":"2024-03-06T07:21:01.017889+0000","last_unstale":"2024-03-06T07:21:03.304368+0000","last_undegraded":"2024-03-06T07:21:03.304368+0000","last_fullsized":"2024-03-06T07:21:03.304368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:45.992777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304322+0000","last_change":"2024-03-06T07:21:01.007221+0000","last_active":"2024-03-06T07:21:03.304322+0000","last_peered":"2024-03-06T07:21:03.304322+0000","last_clean":"2024-03-06T07:21:03.304322+0000","last_became_active":"2024-03-06T07:21:01.006724+0000","last_became_peered":"2024-03-06T07:21:01.006724+0000","last_unstale":"2024-03-06T07:21:03.304322+0000","last_undegraded":"2024-03-06T07:21:03.304322+0000","last_fullsized":"2024-03-06T07:21:03.304322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:21.248617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.303933+0000","last_change":"2024-03-06T07:21:01.005337+0000","last_active":"2024-03-06T07:21:03.303933+0000","last_peered":"2024-03-06T07:21:03.303933+0000","last_clean":"2024-03-06T07:21:03.303933+0000","last_became_active":"2024-03-06T07:21:01.004455+0000","last_became_peered":"2024-03-06T07:21:01.004455+0000","last_unstale":"2024-03-06T07:21:03.303933+0000","last_undegraded":"2024-03-06T07:21:03.303933+0000","last_fullsized":"2024-03-06T07:21:03.303933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:21.082035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304519+0000","last_change":"2024-03-06T07:21:01.020092+0000","last_active":"2024-03-06T07:21:03.304519+0000","last_peered":"2024-03-06T07:21:03.304519+0000","last_clean":"2024-03-06T07:21:03.304519+0000","last_became_active":"2024-03-06T07:21:01.019796+0000","last_became_peered":"2024-03-06T07:21:01.019796+0000","last_unstale":"2024-03-06T07:21:03.304519+0000","last_undegraded":"2024-03-06T07:21:03.304519+0000","last_fullsized":"2024-03-06T07:21:03.304519+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:46.618359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304929+0000","last_change":"2024-03-06T07:21:00.010567+0000","last_active":"2024-03-06T07:21:03.304929+0000","last_peered":"2024-03-06T07:21:03.304929+0000","last_clean":"2024-03-06T07:21:03.304929+0000","last_became_active":"2024-03-06T07:21:00.010277+0000","last_became_peered":"2024-03-06T07:21:00.010277+0000","last_unstale":"2024-03-06T07:21:03.304929+0000","last_undegraded":"2024-03-06T07:21:03.304929+0000","last_fullsized":"2024-03-06T07:21:03.304929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:06.707889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304588+0000","last_change":"2024-03-06T07:21:01.024191+0000","last_active":"2024-03-06T07:21:03.304588+0000","last_peered":"2024-03-06T07:21:03.304588+0000","last_clean":"2024-03-06T07:21:03.304588+0000","last_became_active":"2024-03-06T07:21:01.023879+0000","last_became_peered":"2024-03-06T07:21:01.023879+0000","last_unstale":"2024-03-06T07:21:03.304588+0000","last_undegraded":"2024-03-06T07:21:03.304588+0000","last_fullsized":"2024-03-06T07:21:03.304588+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:52.879221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203749+0000","last_change":"2024-03-06T07:21:00.011675+0000","last_active":"2024-03-06T07:21:02.203749+0000","last_peered":"2024-03-06T07:21:02.203749+0000","last_clean":"2024-03-06T07:21:02.203749+0000","last_became_active":"2024-03-06T07:21:00.011344+0000","last_became_peered":"2024-03-06T07:21:00.011344+0000","last_unstale":"2024-03-06T07:21:02.203749+0000","last_undegraded":"2024-03-06T07:21:02.203749+0000","last_fullsized":"2024-03-06T07:21:02.203749+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:44.433486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305681+0000","last_change":"2024-03-06T07:21:01.021422+0000","last_active":"2024-03-06T07:21:03.305681+0000","last_peered":"2024-03-06T07:21:03.305681+0000","last_clean":"2024-03-06T07:21:03.305681+0000","last_became_active":"2024-03-06T07:21:01.020807+0000","last_became_peered":"2024-03-06T07:21:01.020807+0000","last_unstale":"2024-03-06T07:21:03.305681+0000","last_undegraded":"2024-03-06T07:21:03.305681+0000","last_fullsized":"2024-03-06T07:21:03.305681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:21.082035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304720+0000","last_change":"2024-03-06T07:21:00.012691+0000","last_active":"2024-03-06T07:21:03.304720+0000","last_peered":"2024-03-06T07:21:03.304720+0000","last_clean":"2024-03-06T07:21:03.304720+0000","last_became_active":"2024-03-06T07:21:00.012404+0000","last_became_peered":"2024-03-06T07:21:00.012404+0000","last_unstale":"2024-03-06T07:21:03.304720+0000","last_undegraded":"2024-03-06T07:21:03.304720+0000","last_fullsized":"2024-03-06T07:21:03.304720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:12.743167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304754+0000","last_change":"2024-03-06T07:21:01.018225+0000","last_active":"2024-03-06T07:21:03.304754+0000","last_peered":"2024-03-06T07:21:03.304754+0000","last_clean":"2024-03-06T07:21:03.304754+0000","last_became_active":"2024-03-06T07:21:01.017865+0000","last_became_peered":"2024-03-06T07:21:01.017865+0000","last_unstale":"2024-03-06T07:21:03.304754+0000","last_undegraded":"2024-03-06T07:21:03.304754+0000","last_fullsized":"2024-03-06T07:21:03.304754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:46.292046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305459+0000","last_change":"2024-03-06T07:21:00.011238+0000","last_active":"2024-03-06T07:21:03.305459+0000","last_peered":"2024-03-06T07:21:03.305459+0000","last_clean":"2024-03-06T07:21:03.305459+0000","last_became_active":"2024-03-06T07:21:00.010202+0000","last_became_peered":"2024-03-06T07:21:00.010202+0000","last_unstale":"2024-03-06T07:21:03.305459+0000","last_undegraded":"2024-03-06T07:21:03.305459+0000","last_fullsized":"2024-03-06T07:21:03.305459+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:46.431814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203470+0000","last_change":"2024-03-06T07:21:01.008952+0000","last_active":"2024-03-06T07:21:02.203470+0000","last_peered":"2024-03-06T07:21:02.203470+0000","last_clean":"2024-03-06T07:21:02.203470+0000","last_became_active":"2024-03-06T07:21:01.008678+0000","last_became_peered":"2024-03-06T07:21:01.008678+0000","last_unstale":"2024-03-06T07:21:02.203470+0000","last_undegraded":"2024-03-06T07:21:02.203470+0000","last_fullsized":"2024-03-06T07:21:02.203470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:57.516671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011143+0000","last_change":"2024-03-06T07:21:00.008004+0000","last_active":"2024-03-06T07:21:02.011143+0000","last_peered":"2024-03-06T07:21:02.011143+0000","last_clean":"2024-03-06T07:21:02.011143+0000","last_became_active":"2024-03-06T07:21:00.007716+0000","last_became_peered":"2024-03-06T07:21:00.007716+0000","last_unstale":"2024-03-06T07:21:02.011143+0000","last_undegraded":"2024-03-06T07:21:02.011143+0000","last_fullsized":"2024-03-06T07:21:02.011143+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:16.342176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011143+0000","last_change":"2024-03-06T07:21:01.016347+0000","last_active":"2024-03-06T07:21:02.011143+0000","last_peered":"2024-03-06T07:21:02.011143+0000","last_clean":"2024-03-06T07:21:02.011143+0000","last_became_active":"2024-03-06T07:21:01.016050+0000","last_became_peered":"2024-03-06T07:21:01.016050+0000","last_unstale":"2024-03-06T07:21:02.011143+0000","last_undegraded":"2024-03-06T07:21:02.011143+0000","last_fullsized":"2024-03-06T07:21:02.011143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:47.082638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304372+0000","last_change":"2024-03-06T07:21:00.006417+0000","last_active":"2024-03-06T07:21:03.304372+0000","last_peered":"2024-03-06T07:21:03.304372+0000","last_clean":"2024-03-06T07:21:03.304372+0000","last_became_active":"2024-03-06T07:21:00.006164+0000","last_became_peered":"2024-03-06T07:21:00.006164+0000","last_unstale":"2024-03-06T07:21:03.304372+0000","last_undegraded":"2024-03-06T07:21:03.304372+0000","last_fullsized":"2024-03-06T07:21:03.304372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:44.990676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305159+0000","last_change":"2024-03-06T07:21:01.021349+0000","last_active":"2024-03-06T07:21:03.305159+0000","last_peered":"2024-03-06T07:21:03.305159+0000","last_clean":"2024-03-06T07:21:03.305159+0000","last_became_active":"2024-03-06T07:21:01.020827+0000","last_became_peered":"2024-03-06T07:21:01.020827+0000","last_unstale":"2024-03-06T07:21:03.305159+0000","last_undegraded":"2024-03-06T07:21:03.305159+0000","last_fullsized":"2024-03-06T07:21:03.305159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:17.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304332+0000","last_change":"2024-03-06T07:21:00.018959+0000","last_active":"2024-03-06T07:21:03.304332+0000","last_peered":"2024-03-06T07:21:03.304332+0000","last_clean":"2024-03-06T07:21:03.304332+0000","last_became_active":"2024-03-06T07:21:00.018641+0000","last_became_peered":"2024-03-06T07:21:00.018641+0000","last_unstale":"2024-03-06T07:21:03.304332+0000","last_undegraded":"2024-03-06T07:21:03.304332+0000","last_fullsized":"2024-03-06T07:21:03.304332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:05.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304008+0000","last_change":"2024-03-06T07:21:01.007239+0000","last_active":"2024-03-06T07:21:03.304008+0000","last_peered":"2024-03-06T07:21:03.304008+0000","last_clean":"2024-03-06T07:21:03.304008+0000","last_became_active":"2024-03-06T07:21:01.006964+0000","last_became_peered":"2024-03-06T07:21:01.006964+0000","last_unstale":"2024-03-06T07:21:03.304008+0000","last_undegraded":"2024-03-06T07:21:03.304008+0000","last_fullsized":"2024-03-06T07:21:03.304008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:46.279575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304722+0000","last_change":"2024-03-06T07:21:00.012306+0000","last_active":"2024-03-06T07:21:03.304722+0000","last_peered":"2024-03-06T07:21:03.304722+0000","last_clean":"2024-03-06T07:21:03.304722+0000","last_became_active":"2024-03-06T07:21:00.012030+0000","last_became_peered":"2024-03-06T07:21:00.012030+0000","last_unstale":"2024-03-06T07:21:03.304722+0000","last_undegraded":"2024-03-06T07:21:03.304722+0000","last_fullsized":"2024-03-06T07:21:03.304722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.435165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010239+0000","last_change":"2024-03-06T07:21:01.015364+0000","last_active":"2024-03-06T07:21:02.010239+0000","last_peered":"2024-03-06T07:21:02.010239+0000","last_clean":"2024-03-06T07:21:02.010239+0000","last_became_active":"2024-03-06T07:21:01.015031+0000","last_became_peered":"2024-03-06T07:21:01.015031+0000","last_unstale":"2024-03-06T07:21:02.010239+0000","last_undegraded":"2024-03-06T07:21:02.010239+0000","last_fullsized":"2024-03-06T07:21:02.010239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:37.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304839+0000","last_change":"2024-03-06T07:21:00.015472+0000","last_active":"2024-03-06T07:21:03.304839+0000","last_peered":"2024-03-06T07:21:03.304839+0000","last_clean":"2024-03-06T07:21:03.304839+0000","last_became_active":"2024-03-06T07:21:00.015206+0000","last_became_peered":"2024-03-06T07:21:00.015206+0000","last_unstale":"2024-03-06T07:21:03.304839+0000","last_undegraded":"2024-03-06T07:21:03.304839+0000","last_fullsized":"2024-03-06T07:21:03.304839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:56.514570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304895+0000","last_change":"2024-03-06T07:21:01.022598+0000","last_active":"2024-03-06T07:21:03.304895+0000","last_peered":"2024-03-06T07:21:03.304895+0000","last_clean":"2024-03-06T07:21:03.304895+0000","last_became_active":"2024-03-06T07:21:01.022332+0000","last_became_peered":"2024-03-06T07:21:01.022332+0000","last_unstale":"2024-03-06T07:21:03.304895+0000","last_undegraded":"2024-03-06T07:21:03.304895+0000","last_fullsized":"2024-03-06T07:21:03.304895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:45:09.701793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011447+0000","last_change":"2024-03-06T07:21:00.011966+0000","last_active":"2024-03-06T07:21:02.011447+0000","last_peered":"2024-03-06T07:21:02.011447+0000","last_clean":"2024-03-06T07:21:02.011447+0000","last_became_active":"2024-03-06T07:21:00.011703+0000","last_became_peered":"2024-03-06T07:21:00.011703+0000","last_unstale":"2024-03-06T07:21:02.011447+0000","last_undegraded":"2024-03-06T07:21:02.011447+0000","last_fullsized":"2024-03-06T07:21:02.011447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:12.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203637+0000","last_change":"2024-03-06T07:21:01.021354+0000","last_active":"2024-03-06T07:21:02.203637+0000","last_peered":"2024-03-06T07:21:02.203637+0000","last_clean":"2024-03-06T07:21:02.203637+0000","last_became_active":"2024-03-06T07:21:01.021063+0000","last_became_peered":"2024-03-06T07:21:01.021063+0000","last_unstale":"2024-03-06T07:21:02.203637+0000","last_undegraded":"2024-03-06T07:21:02.203637+0000","last_fullsized":"2024-03-06T07:21:02.203637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:53.929858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011785+0000","last_change":"2024-03-06T07:21:00.017403+0000","last_active":"2024-03-06T07:21:02.011785+0000","last_peered":"2024-03-06T07:21:02.011785+0000","last_clean":"2024-03-06T07:21:02.011785+0000","last_became_active":"2024-03-06T07:21:00.017171+0000","last_became_peered":"2024-03-06T07:21:00.017171+0000","last_unstale":"2024-03-06T07:21:02.011785+0000","last_undegraded":"2024-03-06T07:21:02.011785+0000","last_fullsized":"2024-03-06T07:21:02.011785+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:12.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305668+0000","last_change":"2024-03-06T07:21:01.012275+0000","last_active":"2024-03-06T07:21:03.305668+0000","last_peered":"2024-03-06T07:21:03.305668+0000","last_clean":"2024-03-06T07:21:03.305668+0000","last_became_active":"2024-03-06T07:21:01.011591+0000","last_became_peered":"2024-03-06T07:21:01.011591+0000","last_unstale":"2024-03-06T07:21:03.305668+0000","last_undegraded":"2024-03-06T07:21:03.305668+0000","last_fullsized":"2024-03-06T07:21:03.305668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:51:35.794855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304512+0000","last_change":"2024-03-06T07:21:00.007232+0000","last_active":"2024-03-06T07:21:03.304512+0000","last_peered":"2024-03-06T07:21:03.304512+0000","last_clean":"2024-03-06T07:21:03.304512+0000","last_became_active":"2024-03-06T07:21:00.006787+0000","last_became_peered":"2024-03-06T07:21:00.006787+0000","last_unstale":"2024-03-06T07:21:03.304512+0000","last_undegraded":"2024-03-06T07:21:03.304512+0000","last_fullsized":"2024-03-06T07:21:03.304512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:20.079934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203471+0000","last_change":"2024-03-06T07:21:01.018392+0000","last_active":"2024-03-06T07:21:02.203471+0000","last_peered":"2024-03-06T07:21:02.203471+0000","last_clean":"2024-03-06T07:21:02.203471+0000","last_became_active":"2024-03-06T07:21:01.018048+0000","last_became_peered":"2024-03-06T07:21:01.018048+0000","last_unstale":"2024-03-06T07:21:02.203471+0000","last_undegraded":"2024-03-06T07:21:02.203471+0000","last_fullsized":"2024-03-06T07:21:02.203471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:21.082035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304397+0000","last_change":"2024-03-06T07:21:00.014388+0000","last_active":"2024-03-06T07:21:03.304397+0000","last_peered":"2024-03-06T07:21:03.304397+0000","last_clean":"2024-03-06T07:21:03.304397+0000","last_became_active":"2024-03-06T07:21:00.014118+0000","last_became_peered":"2024-03-06T07:21:00.014118+0000","last_unstale":"2024-03-06T07:21:03.304397+0000","last_undegraded":"2024-03-06T07:21:03.304397+0000","last_fullsized":"2024-03-06T07:21:03.304397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:44:15.041359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304145+0000","last_change":"2024-03-06T07:21:01.015510+0000","last_active":"2024-03-06T07:21:03.304145+0000","last_peered":"2024-03-06T07:21:03.304145+0000","last_clean":"2024-03-06T07:21:03.304145+0000","last_became_active":"2024-03-06T07:21:01.015220+0000","last_became_peered":"2024-03-06T07:21:01.015220+0000","last_unstale":"2024-03-06T07:21:03.304145+0000","last_undegraded":"2024-03-06T07:21:03.304145+0000","last_fullsized":"2024-03-06T07:21:03.304145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:50:05.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.303885+0000","last_change":"2024-03-06T07:21:00.014221+0000","last_active":"2024-03-06T07:21:03.303885+0000","last_peered":"2024-03-06T07:21:03.303885+0000","last_clean":"2024-03-06T07:21:03.303885+0000","last_became_active":"2024-03-06T07:21:00.013921+0000","last_became_peered":"2024-03-06T07:21:00.013921+0000","last_unstale":"2024-03-06T07:21:03.303885+0000","last_undegraded":"2024-03-06T07:21:03.303885+0000","last_fullsized":"2024-03-06T07:21:03.303885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:32.480280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304326+0000","last_change":"2024-03-06T07:21:01.005609+0000","last_active":"2024-03-06T07:21:03.304326+0000","last_peered":"2024-03-06T07:21:03.304326+0000","last_clean":"2024-03-06T07:21:03.304326+0000","last_became_active":"2024-03-06T07:21:01.005316+0000","last_became_peered":"2024-03-06T07:21:01.005316+0000","last_unstale":"2024-03-06T07:21:03.304326+0000","last_undegraded":"2024-03-06T07:21:03.304326+0000","last_fullsized":"2024-03-06T07:21:03.304326+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:12.491905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304194+0000","last_change":"2024-03-06T07:21:00.010506+0000","last_active":"2024-03-06T07:21:03.304194+0000","last_peered":"2024-03-06T07:21:03.304194+0000","last_clean":"2024-03-06T07:21:03.304194+0000","last_became_active":"2024-03-06T07:21:00.010172+0000","last_became_peered":"2024-03-06T07:21:00.010172+0000","last_unstale":"2024-03-06T07:21:03.304194+0000","last_undegraded":"2024-03-06T07:21:03.304194+0000","last_fullsized":"2024-03-06T07:21:03.304194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:04.618247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304150+0000","last_change":"2024-03-06T07:21:01.016295+0000","last_active":"2024-03-06T07:21:03.304150+0000","last_peered":"2024-03-06T07:21:03.304150+0000","last_clean":"2024-03-06T07:21:03.304150+0000","last_became_active":"2024-03-06T07:21:01.015944+0000","last_became_peered":"2024-03-06T07:21:01.015944+0000","last_unstale":"2024-03-06T07:21:03.304150+0000","last_undegraded":"2024-03-06T07:21:03.304150+0000","last_fullsized":"2024-03-06T07:21:03.304150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:30:54.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":22,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304047+0000","last_change":"2024-03-06T07:21:00.005310+0000","last_active":"2024-03-06T07:21:03.304047+0000","last_peered":"2024-03-06T07:21:03.304047+0000","last_clean":"2024-03-06T07:21:03.304047+0000","last_became_active":"2024-03-06T07:21:00.004999+0000","last_became_peered":"2024-03-06T07:21:00.004999+0000","last_unstale":"2024-03-06T07:21:03.304047+0000","last_undegraded":"2024-03-06T07:21:03.304047+0000","last_fullsized":"2024-03-06T07:21:03.304047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:12.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304539+0000","last_change":"2024-03-06T07:21:01.018972+0000","last_active":"2024-03-06T07:21:03.304539+0000","last_peered":"2024-03-06T07:21:03.304539+0000","last_clean":"2024-03-06T07:21:03.304539+0000","last_became_active":"2024-03-06T07:21:01.018692+0000","last_became_peered":"2024-03-06T07:21:01.018692+0000","last_unstale":"2024-03-06T07:21:03.304539+0000","last_undegraded":"2024-03-06T07:21:03.304539+0000","last_fullsized":"2024-03-06T07:21:03.304539+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:53:32.534135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011670+0000","last_change":"2024-03-06T07:21:00.011969+0000","last_active":"2024-03-06T07:21:02.011670+0000","last_peered":"2024-03-06T07:21:02.011670+0000","last_clean":"2024-03-06T07:21:02.011670+0000","last_became_active":"2024-03-06T07:21:00.011741+0000","last_became_peered":"2024-03-06T07:21:00.011741+0000","last_unstale":"2024-03-06T07:21:02.011670+0000","last_undegraded":"2024-03-06T07:21:02.011670+0000","last_fullsized":"2024-03-06T07:21:02.011670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:56.514570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011624+0000","last_change":"2024-03-06T07:21:01.025736+0000","last_active":"2024-03-06T07:21:02.011624+0000","last_peered":"2024-03-06T07:21:02.011624+0000","last_clean":"2024-03-06T07:21:02.011624+0000","last_became_active":"2024-03-06T07:21:01.025405+0000","last_became_peered":"2024-03-06T07:21:01.025405+0000","last_unstale":"2024-03-06T07:21:02.011624+0000","last_undegraded":"2024-03-06T07:21:02.011624+0000","last_fullsized":"2024-03-06T07:21:02.011624+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:35:28.400753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305708+0000","last_change":"2024-03-06T07:21:00.013456+0000","last_active":"2024-03-06T07:21:03.305708+0000","last_peered":"2024-03-06T07:21:03.305708+0000","last_clean":"2024-03-06T07:21:03.305708+0000","last_became_active":"2024-03-06T07:21:00.013184+0000","last_became_peered":"2024-03-06T07:21:00.013184+0000","last_unstale":"2024-03-06T07:21:03.305708+0000","last_undegraded":"2024-03-06T07:21:03.305708+0000","last_fullsized":"2024-03-06T07:21:03.305708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:26:58.643185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305655+0000","last_change":"2024-03-06T07:21:01.012367+0000","last_active":"2024-03-06T07:21:03.305655+0000","last_peered":"2024-03-06T07:21:03.305655+0000","last_clean":"2024-03-06T07:21:03.305655+0000","last_became_active":"2024-03-06T07:21:01.011523+0000","last_became_peered":"2024-03-06T07:21:01.011523+0000","last_unstale":"2024-03-06T07:21:03.305655+0000","last_undegraded":"2024-03-06T07:21:03.305655+0000","last_fullsized":"2024-03-06T07:21:03.305655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:13.153890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010323+0000","last_change":"2024-03-06T07:21:00.003577+0000","last_active":"2024-03-06T07:21:02.010323+0000","last_peered":"2024-03-06T07:21:02.010323+0000","last_clean":"2024-03-06T07:21:02.010323+0000","last_became_active":"2024-03-06T07:21:00.003294+0000","last_became_peered":"2024-03-06T07:21:00.003294+0000","last_unstale":"2024-03-06T07:21:02.010323+0000","last_undegraded":"2024-03-06T07:21:02.010323+0000","last_fullsized":"2024-03-06T07:21:02.010323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:06.256880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010069+0000","last_change":"2024-03-06T07:21:01.012608+0000","last_active":"2024-03-06T07:21:02.010069+0000","last_peered":"2024-03-06T07:21:02.010069+0000","last_clean":"2024-03-06T07:21:02.010069+0000","last_became_active":"2024-03-06T07:21:01.012252+0000","last_became_peered":"2024-03-06T07:21:01.012252+0000","last_unstale":"2024-03-06T07:21:02.010069+0000","last_undegraded":"2024-03-06T07:21:02.010069+0000","last_fullsized":"2024-03-06T07:21:02.010069+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:26:59.645286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304286+0000","last_change":"2024-03-06T07:21:00.012331+0000","last_active":"2024-03-06T07:21:03.304286+0000","last_peered":"2024-03-06T07:21:03.304286+0000","last_clean":"2024-03-06T07:21:03.304286+0000","last_became_active":"2024-03-06T07:21:00.011994+0000","last_became_peered":"2024-03-06T07:21:00.011994+0000","last_unstale":"2024-03-06T07:21:03.304286+0000","last_undegraded":"2024-03-06T07:21:03.304286+0000","last_fullsized":"2024-03-06T07:21:03.304286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:12.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304191+0000","last_change":"2024-03-06T07:21:01.003446+0000","last_active":"2024-03-06T07:21:03.304191+0000","last_peered":"2024-03-06T07:21:03.304191+0000","last_clean":"2024-03-06T07:21:03.304191+0000","last_became_active":"2024-03-06T07:21:01.003192+0000","last_became_peered":"2024-03-06T07:21:01.003192+0000","last_unstale":"2024-03-06T07:21:03.304191+0000","last_undegraded":"2024-03-06T07:21:03.304191+0000","last_fullsized":"2024-03-06T07:21:03.304191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:32.742525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.010007+0000","last_change":"2024-03-06T07:21:00.008859+0000","last_active":"2024-03-06T07:21:02.010007+0000","last_peered":"2024-03-06T07:21:02.010007+0000","last_clean":"2024-03-06T07:21:02.010007+0000","last_became_active":"2024-03-06T07:21:00.008565+0000","last_became_peered":"2024-03-06T07:21:00.008565+0000","last_unstale":"2024-03-06T07:21:02.010007+0000","last_undegraded":"2024-03-06T07:21:02.010007+0000","last_fullsized":"2024-03-06T07:21:02.010007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:55.226211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203567+0000","last_change":"2024-03-06T07:21:01.018736+0000","last_active":"2024-03-06T07:21:02.203567+0000","last_peered":"2024-03-06T07:21:02.203567+0000","last_clean":"2024-03-06T07:21:02.203567+0000","last_became_active":"2024-03-06T07:21:01.018428+0000","last_became_peered":"2024-03-06T07:21:01.018428+0000","last_unstale":"2024-03-06T07:21:02.203567+0000","last_undegraded":"2024-03-06T07:21:02.203567+0000","last_fullsized":"2024-03-06T07:21:02.203567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:44:16.043460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.009949+0000","last_change":"2024-03-06T07:21:00.007836+0000","last_active":"2024-03-06T07:21:02.009949+0000","last_peered":"2024-03-06T07:21:02.009949+0000","last_clean":"2024-03-06T07:21:02.009949+0000","last_became_active":"2024-03-06T07:21:00.006276+0000","last_became_peered":"2024-03-06T07:21:00.006276+0000","last_unstale":"2024-03-06T07:21:02.009949+0000","last_undegraded":"2024-03-06T07:21:02.009949+0000","last_fullsized":"2024-03-06T07:21:02.009949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:39.230614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011061+0000","last_change":"2024-03-06T07:21:01.019290+0000","last_active":"2024-03-06T07:21:02.011061+0000","last_peered":"2024-03-06T07:21:02.011061+0000","last_clean":"2024-03-06T07:21:02.011061+0000","last_became_active":"2024-03-06T07:21:01.019004+0000","last_became_peered":"2024-03-06T07:21:01.019004+0000","last_unstale":"2024-03-06T07:21:02.011061+0000","last_undegraded":"2024-03-06T07:21:02.011061+0000","last_fullsized":"2024-03-06T07:21:02.011061+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:46.279575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304282+0000","last_change":"2024-03-06T07:21:00.011278+0000","last_active":"2024-03-06T07:21:03.304282+0000","last_peered":"2024-03-06T07:21:03.304282+0000","last_clean":"2024-03-06T07:21:03.304282+0000","last_became_active":"2024-03-06T07:21:00.011018+0000","last_became_peered":"2024-03-06T07:21:00.011018+0000","last_unstale":"2024-03-06T07:21:03.304282+0000","last_undegraded":"2024-03-06T07:21:03.304282+0000","last_fullsized":"2024-03-06T07:21:03.304282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:12.743167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304355+0000","last_change":"2024-03-06T07:21:01.010955+0000","last_active":"2024-03-06T07:21:03.304355+0000","last_peered":"2024-03-06T07:21:03.304355+0000","last_clean":"2024-03-06T07:21:03.304355+0000","last_became_active":"2024-03-06T07:21:01.010656+0000","last_became_peered":"2024-03-06T07:21:01.010656+0000","last_unstale":"2024-03-06T07:21:03.304355+0000","last_undegraded":"2024-03-06T07:21:03.304355+0000","last_fullsized":"2024-03-06T07:21:03.304355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:21.855614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.009797+0000","last_change":"2024-03-06T07:21:00.013699+0000","last_active":"2024-03-06T07:21:02.009797+0000","last_peered":"2024-03-06T07:21:02.009797+0000","last_clean":"2024-03-06T07:21:02.009797+0000","last_became_active":"2024-03-06T07:21:00.013360+0000","last_became_peered":"2024-03-06T07:21:00.013360+0000","last_unstale":"2024-03-06T07:21:02.009797+0000","last_undegraded":"2024-03-06T07:21:02.009797+0000","last_fullsized":"2024-03-06T07:21:02.009797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:35.817770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304200+0000","last_change":"2024-03-06T07:21:01.023289+0000","last_active":"2024-03-06T07:21:03.304200+0000","last_peered":"2024-03-06T07:21:03.304200+0000","last_clean":"2024-03-06T07:21:03.304200+0000","last_became_active":"2024-03-06T07:21:01.023028+0000","last_became_peered":"2024-03-06T07:21:01.023028+0000","last_unstale":"2024-03-06T07:21:03.304200+0000","last_undegraded":"2024-03-06T07:21:03.304200+0000","last_fullsized":"2024-03-06T07:21:03.304200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:44:16.043460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011751+0000","last_change":"2024-03-06T07:21:00.015827+0000","last_active":"2024-03-06T07:21:02.011751+0000","last_peered":"2024-03-06T07:21:02.011751+0000","last_clean":"2024-03-06T07:21:02.011751+0000","last_became_active":"2024-03-06T07:21:00.015565+0000","last_became_peered":"2024-03-06T07:21:00.015565+0000","last_unstale":"2024-03-06T07:21:02.011751+0000","last_undegraded":"2024-03-06T07:21:02.011751+0000","last_fullsized":"2024-03-06T07:21:02.011751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:20.079934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.305147+0000","last_change":"2024-03-06T07:21:01.023120+0000","last_active":"2024-03-06T07:21:03.305147+0000","last_peered":"2024-03-06T07:21:03.305147+0000","last_clean":"2024-03-06T07:21:03.305147+0000","last_became_active":"2024-03-06T07:21:01.022861+0000","last_became_peered":"2024-03-06T07:21:01.022861+0000","last_unstale":"2024-03-06T07:21:03.305147+0000","last_undegraded":"2024-03-06T07:21:03.305147+0000","last_fullsized":"2024-03-06T07:21:03.305147+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:46.279575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.009884+0000","last_change":"2024-03-06T07:21:00.009047+0000","last_active":"2024-03-06T07:21:02.009884+0000","last_peered":"2024-03-06T07:21:02.009884+0000","last_clean":"2024-03-06T07:21:02.009884+0000","last_became_active":"2024-03-06T07:21:00.008803+0000","last_became_peered":"2024-03-06T07:21:00.008803+0000","last_unstale":"2024-03-06T07:21:02.009884+0000","last_undegraded":"2024-03-06T07:21:02.009884+0000","last_fullsized":"2024-03-06T07:21:02.009884+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:31.829832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304569+0000","last_change":"2024-03-06T07:21:01.017203+0000","last_active":"2024-03-06T07:21:03.304569+0000","last_peered":"2024-03-06T07:21:03.304569+0000","last_clean":"2024-03-06T07:21:03.304569+0000","last_became_active":"2024-03-06T07:21:01.016912+0000","last_became_peered":"2024-03-06T07:21:01.016912+0000","last_unstale":"2024-03-06T07:21:03.304569+0000","last_undegraded":"2024-03-06T07:21:03.304569+0000","last_fullsized":"2024-03-06T07:21:03.304569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:27.503785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203754+0000","last_change":"2024-03-06T07:21:00.015433+0000","last_active":"2024-03-06T07:21:02.203754+0000","last_peered":"2024-03-06T07:21:02.203754+0000","last_clean":"2024-03-06T07:21:02.203754+0000","last_became_active":"2024-03-06T07:21:00.015156+0000","last_became_peered":"2024-03-06T07:21:00.015156+0000","last_unstale":"2024-03-06T07:21:02.203754+0000","last_undegraded":"2024-03-06T07:21:02.203754+0000","last_fullsized":"2024-03-06T07:21:02.203754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:15.259656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203772+0000","last_change":"2024-03-06T07:21:01.011117+0000","last_active":"2024-03-06T07:21:02.203772+0000","last_peered":"2024-03-06T07:21:02.203772+0000","last_clean":"2024-03-06T07:21:02.203772+0000","last_became_active":"2024-03-06T07:21:01.010846+0000","last_became_peered":"2024-03-06T07:21:01.010846+0000","last_unstale":"2024-03-06T07:21:02.203772+0000","last_undegraded":"2024-03-06T07:21:02.203772+0000","last_fullsized":"2024-03-06T07:21:02.203772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:50.174079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.203729+0000","last_change":"2024-03-06T07:21:00.016544+0000","last_active":"2024-03-06T07:21:02.203729+0000","last_peered":"2024-03-06T07:21:02.203729+0000","last_clean":"2024-03-06T07:21:02.203729+0000","last_became_active":"2024-03-06T07:21:00.015894+0000","last_became_peered":"2024-03-06T07:21:00.015894+0000","last_unstale":"2024-03-06T07:21:02.203729+0000","last_undegraded":"2024-03-06T07:21:02.203729+0000","last_fullsized":"2024-03-06T07:21:02.203729+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:32.480280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.303951+0000","last_change":"2024-03-06T07:21:01.020796+0000","last_active":"2024-03-06T07:21:03.303951+0000","last_peered":"2024-03-06T07:21:03.303951+0000","last_clean":"2024-03-06T07:21:03.303951+0000","last_became_active":"2024-03-06T07:21:01.020536+0000","last_became_peered":"2024-03-06T07:21:01.020536+0000","last_unstale":"2024-03-06T07:21:03.303951+0000","last_undegraded":"2024-03-06T07:21:03.303951+0000","last_fullsized":"2024-03-06T07:21:03.303951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:53.929858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304457+0000","last_change":"2024-03-06T07:21:00.010211+0000","last_active":"2024-03-06T07:21:03.304457+0000","last_peered":"2024-03-06T07:21:03.304457+0000","last_clean":"2024-03-06T07:21:03.304457+0000","last_became_active":"2024-03-06T07:21:00.009915+0000","last_became_peered":"2024-03-06T07:21:00.009915+0000","last_unstale":"2024-03-06T07:21:03.304457+0000","last_undegraded":"2024-03-06T07:21:03.304457+0000","last_fullsized":"2024-03-06T07:21:03.304457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:36.362835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304715+0000","last_change":"2024-03-06T07:21:01.009218+0000","last_active":"2024-03-06T07:21:03.304715+0000","last_peered":"2024-03-06T07:21:03.304715+0000","last_clean":"2024-03-06T07:21:03.304715+0000","last_became_active":"2024-03-06T07:21:01.008907+0000","last_became_peered":"2024-03-06T07:21:01.008907+0000","last_unstale":"2024-03-06T07:21:03.304715+0000","last_undegraded":"2024-03-06T07:21:03.304715+0000","last_fullsized":"2024-03-06T07:21:03.304715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:08.272929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304882+0000","last_change":"2024-03-06T07:21:00.011094+0000","last_active":"2024-03-06T07:21:03.304882+0000","last_peered":"2024-03-06T07:21:03.304882+0000","last_clean":"2024-03-06T07:21:03.304882+0000","last_became_active":"2024-03-06T07:21:00.010457+0000","last_became_peered":"2024-03-06T07:21:00.010457+0000","last_unstale":"2024-03-06T07:21:03.304882+0000","last_undegraded":"2024-03-06T07:21:03.304882+0000","last_fullsized":"2024-03-06T07:21:03.304882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:04.618247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304541+0000","last_change":"2024-03-06T07:21:01.023643+0000","last_active":"2024-03-06T07:21:03.304541+0000","last_peered":"2024-03-06T07:21:03.304541+0000","last_clean":"2024-03-06T07:21:03.304541+0000","last_became_active":"2024-03-06T07:21:01.023367+0000","last_became_peered":"2024-03-06T07:21:01.023367+0000","last_unstale":"2024-03-06T07:21:03.304541+0000","last_undegraded":"2024-03-06T07:21:03.304541+0000","last_fullsized":"2024-03-06T07:21:03.304541+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:59.941710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304188+0000","last_change":"2024-03-06T07:21:00.014362+0000","last_active":"2024-03-06T07:21:03.304188+0000","last_peered":"2024-03-06T07:21:03.304188+0000","last_clean":"2024-03-06T07:21:03.304188+0000","last_became_active":"2024-03-06T07:21:00.014072+0000","last_became_peered":"2024-03-06T07:21:00.014072+0000","last_unstale":"2024-03-06T07:21:03.304188+0000","last_undegraded":"2024-03-06T07:21:03.304188+0000","last_fullsized":"2024-03-06T07:21:03.304188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:52.927757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:02.011568+0000","last_change":"2024-03-06T07:21:01.025979+0000","last_active":"2024-03-06T07:21:02.011568+0000","last_peered":"2024-03-06T07:21:02.011568+0000","last_clean":"2024-03-06T07:21:02.011568+0000","last_became_active":"2024-03-06T07:21:01.025667+0000","last_became_peered":"2024-03-06T07:21:01.025667+0000","last_unstale":"2024-03-06T07:21:02.011568+0000","last_undegraded":"2024-03-06T07:21:02.011568+0000","last_fullsized":"2024-03-06T07:21:02.011568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:59.984090+0000","last_clean_scrub_stamp":"2024-03-06T07:20:59.984090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:27.503785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304540+0000","last_change":"2024-03-06T07:21:00.002103+0000","last_active":"2024-03-06T07:21:03.304540+0000","last_peered":"2024-03-06T07:21:03.304540+0000","last_clean":"2024-03-06T07:21:03.304540+0000","last_became_active":"2024-03-06T07:21:00.001791+0000","last_became_peered":"2024-03-06T07:21:00.001791+0000","last_unstale":"2024-03-06T07:21:03.304540+0000","last_undegraded":"2024-03-06T07:21:03.304540+0000","last_fullsized":"2024-03-06T07:21:03.304540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:58.981989+0000","last_clean_scrub_stamp":"2024-03-06T07:20:58.981989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:07.270828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-06T07:21:03.304166+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:03.304166+0000","last_peered":"2024-03-06T07:21:03.304166+0000","last_clean":"2024-03-06T07:21:03.304166+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:03.304166+0000","last_undegraded":"2024-03-06T07:21:03.304166+0000","last_fullsized":"2024-03-06T07:21:03.304166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":775362,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":183424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":183916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":184958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":183382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":430080,"data_stored":183402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":773714,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:08.781 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:21:08.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:21:08.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.924+0000 7f8e3ff7a640 1 Processor -- start 2024-03-06T07:21:08.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3ff7a640 1 -- start start 2024-03-06T07:21:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3ff7a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e3806afc0 0x7f8e3806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3ff7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3ff7a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e380c5840 con 0x7f8e380c2580 2024-03-06T07:21:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3ff7a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e38158790 con 0x7f8e3806afc0 2024-03-06T07:21:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3ff7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380c5b20 con 0x7f8e380c2120 2024-03-06T07:21:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3dd16640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e3806afc0 0x7f8e3806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3dd16640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e3806afc0 0x7f8e3806b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36876/0 (socket says 172.21.15.26:36876) 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3dd16640 1 -- 172.21.15.26:0/467085172 learned_addr learned my addr 172.21.15.26:0/467085172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.925+0000 7f8e3d515640 1 --2- 172.21.15.26:0/467085172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.926+0000 7f8e3dd16640 1 -- 172.21.15.26:0/467085172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 msgr2=0x7f8e38158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.926+0000 7f8e3dd16640 1 --2- 172.21.15.26:0/467085172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.926+0000 7f8e3dd16640 1 -- 172.21.15.26:0/467085172 >> v1:172.21.15.117:6789/0 conn(0x7f8e380c2580 legacy=0x7f8e38159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.926+0000 7f8e3dd16640 1 -- 172.21.15.26:0/467085172 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380c5e00 con 0x7f8e3806afc0 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.926+0000 7f8e3d515640 1 --2- 172.21.15.26:0/467085172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.926+0000 7f8e3dd16640 1 --2- 172.21.15.26:0/467085172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e3806afc0 0x7f8e3806b3a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009870 tx=0x7f8e3402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6065902bd6239a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3cd14640 1 -- 172.21.15.26:0/467085172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34035020 con 0x7f8e3806afc0 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3cd14640 1 -- 172.21.15.26:0/467085172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e34031640 con 0x7f8e3806afc0 2024-03-06T07:21:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3cd14640 1 -- 172.21.15.26:0/467085172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402d040 con 0x7f8e3806afc0 2024-03-06T07:21:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/467085172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e3806afc0 msgr2=0x7f8e3806b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/467085172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e3806afc0 0x7f8e3806b3a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009870 tx=0x7f8e3402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/467085172 shutdown_connections 2024-03-06T07:21:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/467085172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/467085172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e3806afc0 0x7f8e3806b3a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/467085172 >> 172.21.15.26:0/467085172 conn(0x7f8e380c4220 msgr2=0x7f8e380c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/467085172 shutdown_connections 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.927+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/467085172 wait complete. 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.928+0000 7f8e3ff7a640 1 Processor -- start 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.928+0000 7f8e3ff7a640 1 -- start start 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3ff7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3806afc0 0x7f8e381e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3ff7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3ff7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e380c2580 0x7f8e3814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3ff7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380c2b30 con 0x7f8e380c2580 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3ff7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e38155220 con 0x7f8e3806afc0 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3ff7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380c2cb0 con 0x7f8e380c2120 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3e517640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e380c2580 0x7f8e3814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3806afc0 0x7f8e381e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3d515640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3806afc0 0x7f8e381e1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60368/0 (socket says 172.21.15.26:60368) 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 -- 172.21.15.26:0/2942851179 learned_addr learned my addr 172.21.15.26:0/2942851179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3e517640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e380c2580 0x7f8e3814b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36892/0 (socket says 172.21.15.26:36892) 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 -- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 msgr2=0x7f8e38156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 -- 172.21.15.26:0/2942851179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e380c2580 msgr2=0x7f8e3814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e380c2580 0x7f8e3814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3dd16640 1 -- 172.21.15.26:0/2942851179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380731c0 con 0x7f8e3806afc0 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.929+0000 7f8e3e517640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e380c2580 0x7f8e3814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.930+0000 7f8e3dd16640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3806afc0 0x7f8e381e1e40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8e3402b930 tx=0x7f8e340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.930+0000 7f8e2e7fc640 1 -- 172.21.15.26:0/2942851179 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3403aa60 con 0x7f8e3806afc0 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.930+0000 7f8e2e7fc640 1 -- 172.21.15.26:0/2942851179 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3403ac00 con 0x7f8e3806afc0 2024-03-06T07:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.930+0000 7f8e2e7fc640 1 -- 172.21.15.26:0/2942851179 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e340443f0 con 0x7f8e3806afc0 2024-03-06T07:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.930+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e381e1520 con 0x7f8e3806afc0 2024-03-06T07:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.930+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e381e2380 con 0x7f8e3806afc0 2024-03-06T07:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.931+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e38067ad0 con 0x7f8e3806afc0 2024-03-06T07:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.931+0000 7f8e2e7fc640 1 -- 172.21.15.26:0/2942851179 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e34038020 con 0x7f8e3806afc0 2024-03-06T07:21:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.932+0000 7f8e2e7fc640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e1c06b4d0 0x7f8e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.932+0000 7f8e2e7fc640 1 -- 172.21.15.26:0/2942851179 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e340abfb0 con 0x7f8e3806afc0 2024-03-06T07:21:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.932+0000 7f8e3d515640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e1c06b4d0 0x7f8e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.932+0000 7f8e3d515640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e1c06b4d0 0x7f8e1c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e3806fae0 tx=0x7f8e28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:08.940+0000 7f8e2e7fc640 1 -- 172.21.15.26:0/2942851179 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e38067ad0 con 0x7f8e3806afc0 2024-03-06T07:21:09.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.193+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8e3804f330 con 0x7f8e3806afc0 2024-03-06T07:21:09.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.197+0000 7f8e2e7fc640 1 -- 172.21.15.26:0/2942851179 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8e3403f070 con 0x7f8e3806afc0 2024-03-06T07:21:09.199 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:21:09.199 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:21:09.199 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:21:09.200 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:21:09.200 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:21:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:21:09.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.198+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e1c06b4d0 msgr2=0x7f8e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.198+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e1c06b4d0 0x7f8e1c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e3806fae0 tx=0x7f8e28008040 comp rx=0 tx=0).stop 2024-03-06T07:21:09.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3806afc0 msgr2=0x7f8e381e1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3806afc0 0x7f8e381e1e40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8e3402b930 tx=0x7f8e340046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 shutdown_connections 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e1c06b4d0 0x7f8e1c06d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e380c2580 0x7f8e3814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e380c2120 0x7f8e38156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 --2- 172.21.15.26:0/2942851179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3806afc0 0x7f8e381e1e40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 >> 172.21.15.26:0/2942851179 conn(0x7f8e380c4220 msgr2=0x7f8e38158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 shutdown_connections 2024-03-06T07:21:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.199+0000 7f8e3ff7a640 1 -- 172.21.15.26:0/2942851179 wait complete. 2024-03-06T07:21:09.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.311+0000 7f469e847640 1 Processor -- start 2024-03-06T07:21:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.311+0000 7f469e847640 1 -- start start 2024-03-06T07:21:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f469e847640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46980c3280 0x7f46980c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f469e847640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46981551e0 0x7f4698155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f469e847640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4698158800 con 0x7f469806d9f0 2024-03-06T07:21:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f469e847640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4698158ae0 con 0x7f46980c3280 2024-03-06T07:21:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f469e847640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4698158dc0 con 0x7f46981551e0 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f46977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46980c3280 0x7f46980c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f46977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46980c3280 0x7f46980c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36902/0 (socket says 172.21.15.26:36902) 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f46977fe640 1 -- 172.21.15.26:0/2105129317 learned_addr learned my addr 172.21.15.26:0/2105129317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f469cde4640 1 --2- 172.21.15.26:0/2105129317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46981551e0 0x7f4698155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f46977fe640 1 -- 172.21.15.26:0/2105129317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46981551e0 msgr2=0x7f4698155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f46977fe640 1 --2- 172.21.15.26:0/2105129317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46981551e0 0x7f4698155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f46977fe640 1 -- 172.21.15.26:0/2105129317 >> v1:172.21.15.117:6789/0 conn(0x7f469806d9f0 legacy=0x7f46980a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.312+0000 7f46977fe640 1 -- 172.21.15.26:0/2105129317 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46981590a0 con 0x7f46980c3280 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.313+0000 7f46977fe640 1 --2- 172.21.15.26:0/2105129317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46980c3280 0x7f46980c3710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f468802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554d179a9b5e4041 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.313+0000 7f4696ffd640 1 -- 172.21.15.26:0/2105129317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688035020 con 0x7f46980c3280 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.313+0000 7f4696ffd640 1 -- 172.21.15.26:0/2105129317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688031640 con 0x7f46980c3280 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.313+0000 7f4696ffd640 1 -- 172.21.15.26:0/2105129317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468802d040 con 0x7f46980c3280 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 -- 172.21.15.26:0/2105129317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46980c3280 msgr2=0x7f46980c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 --2- 172.21.15.26:0/2105129317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46980c3280 0x7f46980c3710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f468802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 -- 172.21.15.26:0/2105129317 shutdown_connections 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 --2- 172.21.15.26:0/2105129317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46981551e0 0x7f4698155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 --2- 172.21.15.26:0/2105129317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46980c3280 0x7f46980c3710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 -- 172.21.15.26:0/2105129317 >> 172.21.15.26:0/2105129317 conn(0x7f46980aa630 msgr2=0x7f46980aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 -- 172.21.15.26:0/2105129317 shutdown_connections 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.314+0000 7f469e847640 1 -- 172.21.15.26:0/2105129317 wait complete. 2024-03-06T07:21:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.315+0000 7f469e847640 1 Processor -- start 2024-03-06T07:21:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.315+0000 7f469e847640 1 -- start start 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469e847640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469806d9f0 0x7f469814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469e847640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46980c3280 0x7f4698157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469e847640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981551e0 0x7f469814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469e847640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469806df50 con 0x7f46981551e0 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469e847640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469806ddd0 con 0x7f469806d9f0 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469e847640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980c6910 con 0x7f46980c3280 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981551e0 0x7f469814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981551e0 0x7f469814e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36908/0 (socket says 172.21.15.26:36908) 2024-03-06T07:21:09.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f46977fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46980c3280 0x7f4698157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 -- 172.21.15.26:0/3317342517 learned_addr learned my addr 172.21.15.26:0/3317342517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f4697fff640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469806d9f0 0x7f469814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 -- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46980c3280 msgr2=0x7f4698157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46980c3280 0x7f4698157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 -- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469806d9f0 msgr2=0x7f469814f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469806d9f0 0x7f469814f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f469cde4640 1 -- 172.21.15.26:0/3317342517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46981e15d0 con 0x7f46981551e0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.316+0000 7f46977fe640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46980c3280 0x7f4698157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.317+0000 7f469cde4640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981551e0 0x7f469814e640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f469000ec40 tx=0x7f469000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.317+0000 7f4694ff9640 1 -- 172.21.15.26:0/3317342517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690018020 con 0x7f46981551e0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.317+0000 7f4694ff9640 1 -- 172.21.15.26:0/3317342517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469000ee50 con 0x7f46981551e0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.317+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46980adde0 con 0x7f46981551e0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.317+0000 7f4694ff9640 1 -- 172.21.15.26:0/3317342517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900146c0 con 0x7f46981551e0 2024-03-06T07:21:09.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.317+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46980ae3a0 con 0x7f46981551e0 2024-03-06T07:21:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.318+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469804f330 con 0x7f46981551e0 2024-03-06T07:21:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.318+0000 7f4694ff9640 1 -- 172.21.15.26:0/3317342517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4690005d10 con 0x7f46981551e0 2024-03-06T07:21:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.318+0000 7f4694ff9640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f467806b500 0x7f467806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.319+0000 7f4694ff9640 1 -- 172.21.15.26:0/3317342517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f469008aaf0 con 0x7f46981551e0 2024-03-06T07:21:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.319+0000 7f4697fff640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f467806b500 0x7f467806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.319+0000 7f4697fff640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f467806b500 0x7f467806d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f46980c37c0 tx=0x7f4680009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.323+0000 7f4694ff9640 1 -- 172.21.15.26:0/3317342517 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4690022070 con 0x7f46981551e0 2024-03-06T07:21:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.540+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4698073240 con 0x7f46981551e0 2024-03-06T07:21:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.541+0000 7f4694ff9640 1 -- 172.21.15.26:0/3317342517 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f46981e15d0 con 0x7f46981551e0 2024-03-06T07:21:09.544 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:09.544 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:02.358562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/2470383395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2470383395},{"type":"v1","addr":"172.21.15.117:6836","nonce":2470383395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5209,"qdb_cluster":[5209]},"id":2}]} 2024-03-06T07:21:09.544 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f467806b500 msgr2=0x7f467806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f467806b500 0x7f467806d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f46980c37c0 tx=0x7f4680009040 comp rx=0 tx=0).stop 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981551e0 msgr2=0x7f469814e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981551e0 0x7f469814e640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f469000ec40 tx=0x7f469000c490 comp rx=0 tx=0).stop 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 shutdown_connections 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f467806b500 0x7f467806d9c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981551e0 0x7f469814e640 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46980c3280 0x7f4698157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 --2- 172.21.15.26:0/3317342517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469806d9f0 0x7f469814f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 >> 172.21.15.26:0/3317342517 conn(0x7f46980aa630 msgr2=0x7f46980c7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 shutdown_connections 2024-03-06T07:21:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:09.543+0000 7f469e847640 1 -- 172.21.15.26:0/3317342517 wait complete. 2024-03-06T07:21:09.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:00.359941+0000', 'modified': '2024-03-06T07:21:02.358562+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5209}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5209': {'gid': 5209, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/2470383395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2470383395}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 2470383395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5209, 'qdb_cluster': [5209]} 2024-03-06T07:21:09.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:09.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:21:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.063+0000 7f83a9965640 1 Processor -- start 2024-03-06T07:21:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.063+0000 7f83a9965640 1 -- start start 2024-03-06T07:21:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a9965640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83a406ff10 0x7f83a4153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a9965640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c7170 0x7f83a4154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a9965640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a40a8390 con 0x7f83a40c6c40 2024-03-06T07:21:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a9965640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a40c7620 con 0x7f83a406ff10 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a9965640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a40730f0 con 0x7f83a40c7170 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a37fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f83a40c6c40 0x7f83a40c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60942/0 (socket says 172.21.15.26:60942) 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a37fe640 1 -- 172.21.15.26:0/2026430278 learned_addr learned my addr 172.21.15.26:0/2026430278 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a2ffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83a406ff10 0x7f83a4153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.064+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c7170 0x7f83a4154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.065+0000 7f83a1ffb640 1 -- 172.21.15.26:0/2026430278 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587437116 0 0) 0x7f83a40a8390 con 0x7f83a40c6c40 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.065+0000 7f83a1ffb640 1 -- 172.21.15.26:0/2026430278 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8390003520 con 0x7f83a40c6c40 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.065+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c7170 msgr2=0x7f83a4154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.065+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c7170 0x7f83a4154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.065+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2026430278 >> v1:172.21.15.117:6789/0 conn(0x7f83a40c6c40 legacy=0x7f83a40c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.065+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2026430278 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a40733d0 con 0x7f83a406ff10 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.065+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a406ff10 0x7f83a4153c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f839c0097a0 tx=0x7f839c02b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=658a47041c51438e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a1ffb640 1 -- 172.21.15.26:0/2026430278 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c038070 con 0x7f83a406ff10 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a1ffb640 1 -- 172.21.15.26:0/2026430278 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c031700 con 0x7f83a406ff10 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a1ffb640 1 -- 172.21.15.26:0/2026430278 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c031ab0 con 0x7f83a406ff10 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 -- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a406ff10 msgr2=0x7f83a4153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 --2- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a406ff10 0x7f83a4153c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f839c0097a0 tx=0x7f839c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 -- 172.21.15.26:0/2026430278 shutdown_connections 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 --2- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c7170 0x7f83a4154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 --2- 172.21.15.26:0/2026430278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a406ff10 0x7f83a4153c20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 -- 172.21.15.26:0/2026430278 >> 172.21.15.26:0/2026430278 conn(0x7f83a406b970 msgr2=0x7f83a40aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 -- 172.21.15.26:0/2026430278 shutdown_connections 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.066+0000 7f83a9965640 1 -- 172.21.15.26:0/2026430278 wait complete. 2024-03-06T07:21:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.067+0000 7f83a9965640 1 Processor -- start 2024-03-06T07:21:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.067+0000 7f83a9965640 1 -- start start 2024-03-06T07:21:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a9965640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a406ff10 0x7f83a40ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a9965640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40c6c40 0x7f83a4149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a9965640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40c7170 0x7f83a41549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a9965640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4070470 con 0x7f83a40c6c40 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a9965640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a40702f0 con 0x7f83a40c7170 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a9965640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a40c6860 con 0x7f83a406ff10 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40c6c40 0x7f83a4149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40c6c40 0x7f83a4149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36916/0 (socket says 172.21.15.26:36916) 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a27fc640 1 -- 172.21.15.26:0/2820747383 learned_addr learned my addr 172.21.15.26:0/2820747383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a406ff10 0x7f83a40ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.068+0000 7f83a37fe640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40c7170 0x7f83a41549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a27fc640 1 -- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a406ff10 msgr2=0x7f83a40ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a406ff10 0x7f83a40ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a27fc640 1 -- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40c7170 msgr2=0x7f83a41549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40c7170 0x7f83a41549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a27fc640 1 -- 172.21.15.26:0/2820747383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a414d6a0 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40c6c40 0x7f83a4149630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e840 tx=0x7f838c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f839bfff640 1 -- 172.21.15.26:0/2820747383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c017070 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f839bfff640 1 -- 172.21.15.26:0/2820747383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c00c940 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a41e9830 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f839bfff640 1 -- 172.21.15.26:0/2820747383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c00cd10 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.069+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a40bdc20 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.070+0000 7f839bfff640 1 -- 172.21.15.26:0/2820747383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f838c01a070 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.071+0000 7f839bfff640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f838406b4d0 0x7f838406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.071+0000 7f839bfff640 1 -- 172.21.15.26:0/2820747383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f838c08a0c0 con 0x7f83a40c6c40 2024-03-06T07:21:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.071+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f838406b4d0 0x7f838406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.071+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f838406b4d0 0x7f838406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f839c035000 tx=0x7f839c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 Processor -- start 2024-03-06T07:21:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 -- start start 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d189ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a1330 0x5598d189ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d189f4b0 0x5598d189b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d18056b0 con 0x5598d18a1330 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d17aea60 con 0x5598d189f4b0 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.072+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598d1801750 con 0x5598d18a0f50 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a1330 0x5598d189ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d189ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a1330 0x5598d189ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36932/0 (socket says 172.21.15.26:36932) 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d189ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:45660/0 (socket says 172.21.15.26:45660) 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a27fc640 1 -- 172.21.15.26:0/1584798592 learned_addr learned my addr 172.21.15.26:0/1584798592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a37fe640 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d189f4b0 0x5598d189b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 -- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d189f4b0 msgr2=0x5598d189b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d189f4b0 0x5598d189b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 -- 172.21.15.26:0/1584798592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a1330 msgr2=0x5598d189ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a1330 0x5598d189ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 -- 172.21.15.26:0/1584798592 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598d17ccf30 con 0x5598d18a0f50 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d189ea30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f839c033000 tx=0x7f839c030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556019ea5b0bdb75 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83997fa640 1 -- 172.21.15.26:0/1584798592 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c04d020 con 0x5598d18a0f50 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83997fa640 1 -- 172.21.15.26:0/1584798592 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c004220 con 0x5598d18a0f50 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.073+0000 7f83997fa640 1 -- 172.21.15.26:0/1584798592 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c038040 con 0x5598d18a0f50 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 -- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 msgr2=0x5598d189ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d189ea30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f839c033000 tx=0x7f839c030000 comp rx=0 tx=0).stop 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 -- 172.21.15.26:0/1584798592 shutdown_connections 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d189f4b0 0x5598d189b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a1330 0x5598d189ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 --2- 172.21.15.26:0/1584798592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d189ea30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 -- 172.21.15.26:0/1584798592 >> 172.21.15.26:0/1584798592 conn(0x5598d189d7d0 msgr2=0x5598d18c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 -- 172.21.15.26:0/1584798592 shutdown_connections 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.074+0000 7f83ac62d740 1 -- 172.21.15.26:0/1584798592 wait complete. 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 -- start start 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 Processor -- start 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d18c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18a1330 0x5598d18c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18c1b90 0x5598d18c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d18c3230 con 0x5598d18c1b90 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d17c1ea0 con 0x5598d18a1330 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598d18c2c70 con 0x5598d18a0f50 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83ac62d740 10 client.0 _ll_register_callbacks cb 0x5598d17c1870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d18c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18c1b90 0x5598d18c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d18c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:45672/0 (socket says 172.21.15.26:45672) 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18c1b90 0x5598d18c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36938/0 (socket says 172.21.15.26:36938) 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2600998617 learned_addr learned my addr 172.21.15.26:0/2600998617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.075+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18a1330 0x5598d18c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18a1330 msgr2=0x5598d18c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18a1330 0x5598d18c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2600998617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18c1b90 msgr2=0x5598d18c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18c1b90 0x5598d18c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2600998617 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598d17c2270 con 0x5598d18a0f50 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d18c4020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f839c0098c0 tx=0x7f839c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15643ddd7aa2921f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83827fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-03-06T07:21:10.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83827fc640 1 -- 172.21.15.26:0/2600998617 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c04d040 con 0x5598d18a0f50 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83827fc640 1 -- 172.21.15.26:0/2600998617 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f839c03d980 con 0x5598d18a0f50 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f839c004220 con 0x5598d18a0f50 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83827fc640 1 -- 172.21.15.26:0/2600998617 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f839c03db80 con 0x5598d18a0f50 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83ac62d740 10 client.5024 fetch_fsmap learned FSMap version 16 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5598d151a2d0 con 0x5598d18a0f50 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83827fc640 1 -- 172.21.15.26:0/2600998617 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c004220 con 0x5598d18a0f50 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.076+0000 7f83827fc640 1 -- 172.21.15.26:0/2600998617 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 16) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f839c032070 con 0x5598d18a0f50 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83ac62d740 10 client.5024 fetch_fsmap finished waiting for FSMap version 16 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83ac62d740 10 client.5024 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83ac62d740 20 client.5024 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d18f19b0 0x5598d18fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83ac62d740 4 client.5024 mds_command: new command op to 5209 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 --> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598d1812bf0 con 0x5598d18f19b0 2024-03-06T07:21:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d18f19b0 0x5598d18fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83827fc640 10 client.5024 ms_handle_connect on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.077+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d18f19b0 0x5598d18fc4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954c9e475d26bbdb server_cookie=e79eadf7a6e7afc3 in_seq=0 out_seq=0 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83827fc640 1 -- 172.21.15.26:0/2600998617 <== mds.0 v2:172.21.15.117:6834/2470383395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f838c081e50 con 0x5598d18f19b0 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83827fc640 10 client.5024 handle_command_reply: tid=0 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 client.5024 shutdown 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 msgr2=0x5598d18c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d18c4020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f839c0098c0 tx=0x7f839c004730 comp rx=0 tx=0).stop 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 shutdown_connections 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18c1b90 0x5598d18c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d18f19b0 0x5598d18fc4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18a1330 0x5598d18c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83827fc640 0 client.5024 ms_handle_reset on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2600998617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a0f50 0x5598d18c4020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 >> 172.21.15.26:0/2600998617 conn(0x5598d189f4b0 msgr2=0x5598d18ddb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 shutdown_connections 2024-03-06T07:21:10.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 1 -- 172.21.15.26:0/2600998617 wait complete. 2024-03-06T07:21:10.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.078+0000 7f83ac62d740 20 client.5024 trim_cache size 0 max 16384 2024-03-06T07:21:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.095+0000 7f83ac62d740 1 Processor -- start 2024-03-06T07:21:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.095+0000 7f83ac62d740 1 -- start start 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d18eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d18bc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83ac62d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a41f4b10 con 0x5598d18a0f50 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a41ec9b0 con 0x5598d18c1b90 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a41f4540 con 0x5598d18a1330 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d18eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d18eb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36950/0 (socket says 172.21.15.26:36950) 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 -- 172.21.15.26:0/3780225915 learned_addr learned my addr 172.21.15.26:0/3780225915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a37fe640 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d18bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a27fc640 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 -- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 msgr2=0x5598d18ddb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 -- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 msgr2=0x5598d18bc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d18bc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 -- 172.21.15.26:0/3780225915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598d18c19c0 con 0x5598d18a0f50 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.096+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d18eb060 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f839c03dfd0 tx=0x7f839c00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bdd0d3d2bc9e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f8398ff9640 1 -- 172.21.15.26:0/3780225915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c041040 con 0x5598d18a0f50 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f8398ff9640 1 -- 172.21.15.26:0/3780225915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c057070 con 0x5598d18a0f50 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f8398ff9640 1 -- 172.21.15.26:0/3780225915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c04d040 con 0x5598d18a0f50 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 -- 172.21.15.26:0/3780225915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 msgr2=0x5598d18eb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d18eb060 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f839c03dfd0 tx=0x7f839c00b050 comp rx=0 tx=0).stop 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 -- 172.21.15.26:0/3780225915 shutdown_connections 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d18bc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 --2- 172.21.15.26:0/3780225915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d18eb060 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 -- 172.21.15.26:0/3780225915 >> 172.21.15.26:0/3780225915 conn(0x5598d189f4b0 msgr2=0x5598d18c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 -- 172.21.15.26:0/3780225915 shutdown_connections 2024-03-06T07:21:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.097+0000 7f83ac62d740 1 -- 172.21.15.26:0/3780225915 wait complete. 2024-03-06T07:21:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 -- start start 2024-03-06T07:21:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 Processor -- start 2024-03-06T07:21:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d189adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d194a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d18c5eb0 con 0x5598d18a0f50 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d18ee710 con 0x5598d18c1b90 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598d18f0e00 con 0x5598d18a1330 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.098+0000 7f83ac62d740 10 client.0 _ll_register_callbacks cb 0x7f83900036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d189adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d189adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36962/0 (socket says 172.21.15.26:36962) 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2495227277 learned_addr learned my addr 172.21.15.26:0/2495227277 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a37fe640 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d194a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 msgr2=0x5598d18ddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 msgr2=0x5598d194a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d194a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 -- 172.21.15.26:0/2495227277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598d15e46b0 con 0x5598d18a0f50 2024-03-06T07:21:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83a2ffd640 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d189adb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f839c054040 tx=0x7f839c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8329eb705dc3f9e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.099+0000 7f83827fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.101+0000 7f83827fc640 1 -- 172.21.15.26:0/2495227277 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c06d060 con 0x5598d18a0f50 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.101+0000 7f83827fc640 1 -- 172.21.15.26:0/2495227277 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f839c065680 con 0x5598d18a0f50 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.101+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f839c057070 con 0x5598d18a0f50 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.101+0000 7f83827fc640 1 -- 172.21.15.26:0/2495227277 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f839c073050 con 0x5598d18a0f50 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.101+0000 7f83ac62d740 10 client.5406 fetch_fsmap learned FSMap version 16 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.101+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5598d1901180 con 0x5598d18a0f50 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.101+0000 7f83827fc640 1 -- 172.21.15.26:0/2495227277 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c057070 con 0x5598d18a0f50 2024-03-06T07:21:10.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83827fc640 1 -- 172.21.15.26:0/2495227277 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 16) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f839c076020 con 0x5598d18a0f50 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83ac62d740 10 client.5406 fetch_fsmap finished waiting for FSMap version 16 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83ac62d740 10 client.5406 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83ac62d740 20 client.5406 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d194a700 0x5598d18e0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83ac62d740 4 client.5406 mds_command: new command op to 5209 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 --> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598d180cf40 con 0x5598d194a700 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d194a700 0x5598d18e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83827fc640 10 client.5406 ms_handle_connect on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83a27fc640 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d194a700 0x5598d18e0370 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22e3441e61c4929 server_cookie=a145d791db82772c in_seq=0 out_seq=0 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83827fc640 1 -- 172.21.15.26:0/2495227277 <== mds.0 v2:172.21.15.117:6834/2470383395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5598d180cf40 con 0x5598d194a700 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83827fc640 10 client.5406 handle_command_reply: tid=0 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.102+0000 7f83ac62d740 1 client.5406 shutdown 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 msgr2=0x5598d189adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d189adb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f839c054040 tx=0x7f839c031000 comp rx=0 tx=0).stop 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 shutdown_connections 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x5598d194a700 0x5598d18e0370 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598d18c1b90 0x5598d194a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5598d18a1330 0x5598d18ddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 --2- 172.21.15.26:0/2495227277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d18a0f50 0x5598d189adb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83827fc640 0 client.5406 ms_handle_reset on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 >> 172.21.15.26:0/2495227277 conn(0x5598d189f4b0 msgr2=0x5598d18a1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 shutdown_connections 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 1 -- 172.21.15.26:0/2495227277 wait complete. 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83ac62d740 20 client.5406 trim_cache size 0 max 16384 2024-03-06T07:21:10.109 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5209, 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 9.1104748820000001, 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 16, 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 37, 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 37, 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 16.642648933 2024-03-06T07:21:10.110 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:21:10.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f838406b4d0 msgr2=0x7f838406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f838406b4d0 0x7f838406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f839c035000 tx=0x7f839c008030 comp rx=0 tx=0).stop 2024-03-06T07:21:10.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40c6c40 msgr2=0x7f83a4149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40c6c40 0x7f83a4149630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e840 tx=0x7f838c00ed00 comp rx=0 tx=0).stop 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 shutdown_connections 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f838406b4d0 0x7f838406d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40c7170 0x7f83a41549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40c6c40 0x7f83a4149630 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 --2- 172.21.15.26:0/2820747383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a406ff10 0x7f83a40ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 >> 172.21.15.26:0/2820747383 conn(0x7f83a406b970 msgr2=0x7f83a4150bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 shutdown_connections 2024-03-06T07:21:10.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.103+0000 7f83a9965640 1 -- 172.21.15.26:0/2820747383 wait complete. 2024-03-06T07:21:10.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.215+0000 7f1279878640 1 Processor -- start 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.215+0000 7f1279878640 1 -- start start 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1279878640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12740c8a00 0x7f1274158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1279878640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1274159460 0x7f12740c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1279878640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12740c7760 con 0x7f1274152920 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1279878640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12741587b0 con 0x7f12740c8a00 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1279878640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12740c7a40 con 0x7f1274159460 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f12727fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12740c8a00 0x7f1274158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1272ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1274152920 0x7f1274155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56428/0 (socket says 172.21.15.26:56428) 2024-03-06T07:21:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1272ffd640 1 -- 172.21.15.26:0/664001467 learned_addr learned my addr 172.21.15.26:0/664001467 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f12737fe640 1 --2- 172.21.15.26:0/664001467 >> v2:172.21.15.117:3300/0 conn(0x7f1274159460 0x7f12740c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f12727fc640 1 -- 172.21.15.26:0/664001467 >> v2:172.21.15.117:3300/0 conn(0x7f1274159460 msgr2=0x7f12740c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f12727fc640 1 --2- 172.21.15.26:0/664001467 >> v2:172.21.15.117:3300/0 conn(0x7f1274159460 0x7f12740c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f12727fc640 1 -- 172.21.15.26:0/664001467 >> v1:172.21.15.26:6789/0 conn(0x7f1274152920 legacy=0x7f1274155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f12727fc640 1 -- 172.21.15.26:0/664001467 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12740c7d20 con 0x7f12740c8a00 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f1271ffb640 1 -- 172.21.15.26:0/664001467 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913000993 0 0) 0x7f12740c7760 con 0x7f1274152920 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.216+0000 7f12737fe640 1 --2- 172.21.15.26:0/664001467 >> v2:172.21.15.117:3300/0 conn(0x7f1274159460 0x7f12740c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.217+0000 7f12727fc640 1 --2- 172.21.15.26:0/664001467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f1274158f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f12740cba60 tx=0x7f125c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85a9e77a94c00fb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.217+0000 7f1271ffb640 1 -- 172.21.15.26:0/664001467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c0315a0 con 0x7f12740c8a00 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.217+0000 7f1271ffb640 1 -- 172.21.15.26:0/664001467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c037070 con 0x7f12740c8a00 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.217+0000 7f1271ffb640 1 -- 172.21.15.26:0/664001467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c031960 con 0x7f12740c8a00 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 -- 172.21.15.26:0/664001467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 msgr2=0x7f1274158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 --2- 172.21.15.26:0/664001467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f1274158f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f12740cba60 tx=0x7f125c02b840 comp rx=0 tx=0).stop 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 -- 172.21.15.26:0/664001467 shutdown_connections 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 --2- 172.21.15.26:0/664001467 >> v2:172.21.15.117:3300/0 conn(0x7f1274159460 0x7f12740c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 --2- 172.21.15.26:0/664001467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f1274158f20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 -- 172.21.15.26:0/664001467 >> 172.21.15.26:0/664001467 conn(0x7f12740ac560 msgr2=0x7f12740ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 -- 172.21.15.26:0/664001467 shutdown_connections 2024-03-06T07:21:10.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.218+0000 7f1279878640 1 -- 172.21.15.26:0/664001467 wait complete. 2024-03-06T07:21:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.219+0000 7f1279878640 1 Processor -- start 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.219+0000 7f1279878640 1 -- start start 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f12740c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1274152920 0x7f1274146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12740aaef0 0x7f1274147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12740c7390 con 0x7f12740c8a00 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127406d030 con 0x7f12740aaef0 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12740c7510 con 0x7f1274152920 2024-03-06T07:21:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f12740c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f12740c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36980/0 (socket says 172.21.15.26:36980) 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 -- 172.21.15.26:0/1358247153 learned_addr learned my addr 172.21.15.26:0/1358247153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f12727fc640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1274152920 0x7f1274146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f12737fe640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12740aaef0 0x7f1274147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 -- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1274152920 msgr2=0x7f1274146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1274152920 0x7f1274146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 -- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12740aaef0 msgr2=0x7f1274147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12740aaef0 0x7f1274147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 -- 172.21.15.26:0/1358247153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12740abf00 con 0x7f12740c8a00 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f12737fe640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12740aaef0 0x7f1274147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1272ffd640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f12740c9130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f126c00c4d0 tx=0x7f126c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f126bfff640 1 -- 172.21.15.26:0/1358247153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c018070 con 0x7f12740c8a00 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f126bfff640 1 -- 172.21.15.26:0/1358247153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c007500 con 0x7f12740c8a00 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f126bfff640 1 -- 172.21.15.26:0/1358247153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c004a20 con 0x7f12740c8a00 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1274147ea0 con 0x7f12740c8a00 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.220+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1274147650 con 0x7f12740c8a00 2024-03-06T07:21:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.221+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1274147c20 con 0x7f12740c8a00 2024-03-06T07:21:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.222+0000 7f126bfff640 1 -- 172.21.15.26:0/1358247153 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f126c01b070 con 0x7f12740c8a00 2024-03-06T07:21:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.222+0000 7f126bfff640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f125406b4d0 0x7f125406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.222+0000 7f126bfff640 1 -- 172.21.15.26:0/1358247153 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f126c089500 con 0x7f12740c8a00 2024-03-06T07:21:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.222+0000 7f12727fc640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f125406b4d0 0x7f125406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.223+0000 7f12727fc640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f125406b4d0 0x7f125406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f125c036000 tx=0x7f125c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.227+0000 7f126bfff640 1 -- 172.21.15.26:0/1358247153 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1274147c20 con 0x7f12740c8a00 2024-03-06T07:21:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.445+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f127404f330 con 0x7f12740c8a00 2024-03-06T07:21:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.445+0000 7f126bfff640 1 -- 172.21.15.26:0/1358247153 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f126c057c30 con 0x7f12740c8a00 2024-03-06T07:21:10.449 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:10.449 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:02.358562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/2470383395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2470383395},{"type":"v1","addr":"172.21.15.117:6836","nonce":2470383395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5209,"qdb_cluster":[5209]},"id":2}]} 2024-03-06T07:21:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-03-06T07:21:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f125406b4d0 msgr2=0x7f125406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f125406b4d0 0x7f125406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f125c036000 tx=0x7f125c03d040 comp rx=0 tx=0).stop 2024-03-06T07:21:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 msgr2=0x7f12740c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f12740c9130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f126c00c4d0 tx=0x7f126c00c990 comp rx=0 tx=0).stop 2024-03-06T07:21:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 shutdown_connections 2024-03-06T07:21:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f125406b4d0 0x7f125406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12740aaef0 0x7f1274147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1274152920 0x7f1274146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 --2- 172.21.15.26:0/1358247153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12740c8a00 0x7f12740c9130 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 >> 172.21.15.26:0/1358247153 conn(0x7f12740ac560 msgr2=0x7f12740ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 shutdown_connections 2024-03-06T07:21:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.447+0000 7f1279878640 1 -- 172.21.15.26:0/1358247153 wait complete. 2024-03-06T07:21:10.462 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 16, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:00.359941+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5209": { "addr": "172.21.15.117:6836/2470383395", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2470383395, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 2470383395, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5209, "incarnation": 14, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:02.358562+0000", "qdb_cluster": [ 5209 ], "qdb_leader": 5209, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5209 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/2730813301", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 2730813301, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2730813301, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4910, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6833/3831184543", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3831184543, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3831184543, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5202, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2694023942", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2694023942, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2694023942, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5205, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:10.462 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:10.462 INFO:tasks.cephfs.test_failover:current = [{'gid': 5202, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/3831184543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3831184543}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3831184543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}, {'gid': 5209, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/2470383395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2470383395}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 2470383395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5205, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2694023942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2694023942}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2694023942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 4910, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}] 2024-03-06T07:21:10.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs 2024-03-06T07:21:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.564+0000 7fb4420fb640 1 Processor -- start 2024-03-06T07:21:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.564+0000 7fb4420fb640 1 -- start start 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb4420fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb4420fb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c0c7760 con 0x7fb43c152920 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb4420fb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c0c7a40 con 0x7fb43c159460 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb4420fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c1587b0 con 0x7fb43c0c8a00 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb43c152920 0x7fb43c155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56434/0 (socket says 172.21.15.26:56434) 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43b7fe640 1 -- 172.21.15.26:0/2597757016 learned_addr learned my addr 172.21.15.26:0/2597757016 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43affd640 1 --2- 172.21.15.26:0/2597757016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43a7fc640 1 -- 172.21.15.26:0/2597757016 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625267329 0 0) 0x7fb43c0c7760 con 0x7fb43c152920 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43a7fc640 1 -- 172.21.15.26:0/2597757016 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb424003520 con 0x7fb43c152920 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43a7fc640 1 -- 172.21.15.26:0/2597757016 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620519046 0 0) 0x7fb43c0c7a40 con 0x7fb43c159460 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43a7fc640 1 -- 172.21.15.26:0/2597757016 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb43c0c7760 con 0x7fb43c159460 2024-03-06T07:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43affd640 1 -- 172.21.15.26:0/2597757016 >> v1:172.21.15.117:6789/0 conn(0x7fb43c159460 legacy=0x7fb43c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43affd640 1 -- 172.21.15.26:0/2597757016 >> v1:172.21.15.26:6789/0 conn(0x7fb43c152920 legacy=0x7fb43c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.565+0000 7fb43affd640 1 -- 172.21.15.26:0/2597757016 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0c7d20 con 0x7fb43c0c8a00 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.566+0000 7fb43affd640 1 --2- 172.21.15.26:0/2597757016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c158f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4300097d0 tx=0x7fb43002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe13110d3340ad28 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.566+0000 7fb43a7fc640 1 -- 172.21.15.26:0/2597757016 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430035020 con 0x7fb43c0c8a00 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.566+0000 7fb43a7fc640 1 -- 172.21.15.26:0/2597757016 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb430031800 con 0x7fb43c0c8a00 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.566+0000 7fb43a7fc640 1 -- 172.21.15.26:0/2597757016 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430031bb0 con 0x7fb43c0c8a00 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.567+0000 7fb4420fb640 1 -- 172.21.15.26:0/2597757016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 msgr2=0x7fb43c158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.567+0000 7fb4420fb640 1 --2- 172.21.15.26:0/2597757016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c158f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4300097d0 tx=0x7fb43002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.567+0000 7fb4420fb640 1 -- 172.21.15.26:0/2597757016 shutdown_connections 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.567+0000 7fb4420fb640 1 --2- 172.21.15.26:0/2597757016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c158f20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.567+0000 7fb4420fb640 1 -- 172.21.15.26:0/2597757016 >> 172.21.15.26:0/2597757016 conn(0x7fb43c0ac560 msgr2=0x7fb43c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.567+0000 7fb4420fb640 1 -- 172.21.15.26:0/2597757016 shutdown_connections 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.567+0000 7fb4420fb640 1 -- 172.21.15.26:0/2597757016 wait complete. 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.568+0000 7fb4420fb640 1 Processor -- start 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.568+0000 7fb4420fb640 1 -- start start 2024-03-06T07:21:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb4420fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c14be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb4420fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43c152920 0x7fb43c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb4420fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c14ad80 0x7fb43c14b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb4420fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0c5a40 con 0x7fb43c152920 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb4420fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c06d030 con 0x7fb43c14ad80 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb4420fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0c5bc0 con 0x7fb43c0c8a00 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43c152920 0x7fb43c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43c152920 0x7fb43c1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36982/0 (socket says 172.21.15.26:36982) 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 -- 172.21.15.26:0/3752576372 learned_addr learned my addr 172.21.15.26:0/3752576372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43bfff640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c14ad80 0x7fb43c14b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43b7fe640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c14be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 -- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 msgr2=0x7fb43c14be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c14be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 -- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c14ad80 msgr2=0x7fb43c14b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c14ad80 0x7fb43c14b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 -- 172.21.15.26:0/3752576372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c14b750 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43bfff640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c14ad80 0x7fb43c14b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.569+0000 7fb43affd640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43c152920 0x7fb43c1578a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb4300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.570+0000 7fb423fff640 1 -- 172.21.15.26:0/3752576372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430041070 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.570+0000 7fb423fff640 1 -- 172.21.15.26:0/3752576372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb430031e30 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.570+0000 7fb423fff640 1 -- 172.21.15.26:0/3752576372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430035040 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.570+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c14bb20 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.570+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c157de0 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.571+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c067910 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.571+0000 7fb423fff640 1 -- 172.21.15.26:0/3752576372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb430038020 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.571+0000 7fb423fff640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb42806b500 0x7fb42806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.571+0000 7fb423fff640 1 -- 172.21.15.26:0/3752576372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4300abc20 con 0x7fb43c152920 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.572+0000 7fb43b7fe640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb42806b500 0x7fb42806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.572+0000 7fb43b7fe640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb42806b500 0x7fb42806d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb42c000f60 tx=0x7fb42c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.576+0000 7fb423fff640 1 -- 172.21.15.26:0/3752576372 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb43c067910 con 0x7fb43c152920 2024-03-06T07:21:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.757+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fb43c1eebb0 con 0x7fb43c152920 2024-03-06T07:21:10.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.760+0000 7fb423fff640 1 -- 172.21.15.26:0/3752576372 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v12)=0 v12) v1 ==== 116+0+0 (secure 0 0 0) 0x7fb43007a350 con 0x7fb43c152920 2024-03-06T07:21:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb42806b500 msgr2=0x7fb42806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb42806b500 0x7fb42806d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb42c000f60 tx=0x7fb42c007d50 comp rx=0 tx=0).stop 2024-03-06T07:21:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43c152920 msgr2=0x7fb43c1578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43c152920 0x7fb43c1578a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb4300046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 shutdown_connections 2024-03-06T07:21:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb42806b500 0x7fb42806d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c14ad80 0x7fb43c14b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43c152920 0x7fb43c1578a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 --2- 172.21.15.26:0/3752576372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0c8a00 0x7fb43c14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 >> 172.21.15.26:0/3752576372 conn(0x7fb43c0ac560 msgr2=0x7fb43c151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 shutdown_connections 2024-03-06T07:21:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.762+0000 7fb4420fb640 1 -- 172.21.15.26:0/3752576372 wait complete. 2024-03-06T07:21:10.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:10.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.879+0000 7fe43c1bc640 1 Processor -- start 2024-03-06T07:21:10.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.879+0000 7fe43c1bc640 1 -- start start 2024-03-06T07:21:10.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43c1bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4340c3710 0x7fe4341558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43c1bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe434158800 con 0x7fe4340c3280 2024-03-06T07:21:10.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43c1bc640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe434158ae0 con 0x7fe43406d9f0 2024-03-06T07:21:10.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43c1bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe434158dc0 con 0x7fe4340c3710 2024-03-06T07:21:10.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43a759640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4340c3710 0x7fe4341558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe439757640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe4340c3280 0x7fe4341551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56436/0 (socket says 172.21.15.26:56436) 2024-03-06T07:21:10.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe439757640 1 -- 172.21.15.26:0/3190383970 learned_addr learned my addr 172.21.15.26:0/3190383970 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:10.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43a759640 1 -- 172.21.15.26:0/3190383970 >> v1:172.21.15.117:6789/0 conn(0x7fe43406d9f0 legacy=0x7fe4340a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43a759640 1 -- 172.21.15.26:0/3190383970 >> v1:172.21.15.26:6789/0 conn(0x7fe4340c3280 legacy=0x7fe4341551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe438f56640 1 -- 172.21.15.26:0/3190383970 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053908238 0 0) 0x7fe434158800 con 0x7fe4340c3280 2024-03-06T07:21:10.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.880+0000 7fe43a759640 1 -- 172.21.15.26:0/3190383970 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4341590a0 con 0x7fe4340c3710 2024-03-06T07:21:10.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.881+0000 7fe43a759640 1 --2- 172.21.15.26:0/3190383970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3710 0x7fe4341558f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009d90 tx=0x7fe42c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ef1e5f87ac6ff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.881+0000 7fe438f56640 1 -- 172.21.15.26:0/3190383970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c039070 con 0x7fe4340c3710 2024-03-06T07:21:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.881+0000 7fe438f56640 1 -- 172.21.15.26:0/3190383970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c02cc60 con 0x7fe4340c3710 2024-03-06T07:21:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.881+0000 7fe438f56640 1 -- 172.21.15.26:0/3190383970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c034660 con 0x7fe4340c3710 2024-03-06T07:21:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.882+0000 7fe43c1bc640 1 -- 172.21.15.26:0/3190383970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3710 msgr2=0x7fe4341558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.882+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/3190383970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3710 0x7fe4341558f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009d90 tx=0x7fe42c02beb0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.882+0000 7fe43c1bc640 1 -- 172.21.15.26:0/3190383970 shutdown_connections 2024-03-06T07:21:10.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.882+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/3190383970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3710 0x7fe4341558f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.882+0000 7fe43c1bc640 1 -- 172.21.15.26:0/3190383970 >> 172.21.15.26:0/3190383970 conn(0x7fe4340aa630 msgr2=0x7fe4340aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.882+0000 7fe43c1bc640 1 -- 172.21.15.26:0/3190383970 shutdown_connections 2024-03-06T07:21:10.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.882+0000 7fe43c1bc640 1 -- 172.21.15.26:0/3190383970 wait complete. 2024-03-06T07:21:10.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 Processor -- start 2024-03-06T07:21:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 -- start start 2024-03-06T07:21:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe43406d9f0 0x7fe4341dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3280 0x7fe43406f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4340c3710 0x7fe43414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43406ddd0 con 0x7fe4340c3280 2024-03-06T07:21:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434155e30 con 0x7fe4340c3710 2024-03-06T07:21:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe43c1bc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43406df50 con 0x7fe43406d9f0 2024-03-06T07:21:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.883+0000 7fe439757640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3280 0x7fe43406f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3280 0x7fe43406f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37000/0 (socket says 172.21.15.26:37000) 2024-03-06T07:21:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 -- 172.21.15.26:0/1684412176 learned_addr learned my addr 172.21.15.26:0/1684412176 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439f58640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe43406d9f0 0x7fe4341dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe43a759640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4340c3710 0x7fe43414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 -- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe43406d9f0 msgr2=0x7fe4341dfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe43406d9f0 0x7fe4341dfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 -- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4340c3710 msgr2=0x7fe43414f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4340c3710 0x7fe43414f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 -- 172.21.15.26:0/1684412176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43406d190 con 0x7fe4340c3280 2024-03-06T07:21:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe439757640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3280 0x7fe43406f390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c4d0 tx=0x7fe42400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe42a7fc640 1 -- 172.21.15.26:0/1684412176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424018070 con 0x7fe4340c3280 2024-03-06T07:21:10.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe42a7fc640 1 -- 172.21.15.26:0/1684412176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240074e0 con 0x7fe4340c3280 2024-03-06T07:21:10.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe42a7fc640 1 -- 172.21.15.26:0/1684412176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424004a20 con 0x7fe4340c3280 2024-03-06T07:21:10.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4341ebb90 con 0x7fe4340c3280 2024-03-06T07:21:10.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.884+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4340ae480 con 0x7fe4340c3280 2024-03-06T07:21:10.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.885+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe434069460 con 0x7fe4340c3280 2024-03-06T07:21:10.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.885+0000 7fe42a7fc640 1 -- 172.21.15.26:0/1684412176 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe42401b070 con 0x7fe4340c3280 2024-03-06T07:21:10.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.886+0000 7fe42a7fc640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe41806b4d0 0x7fe41806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.886+0000 7fe42a7fc640 1 -- 172.21.15.26:0/1684412176 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4240895b0 con 0x7fe4340c3280 2024-03-06T07:21:10.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.886+0000 7fe439f58640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe41806b4d0 0x7fe41806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.886+0000 7fe439f58640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe41806b4d0 0x7fe41806d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe434151b50 tx=0x7fe430009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:10.890+0000 7fe42a7fc640 1 -- 172.21.15.26:0/1684412176 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe434069460 con 0x7fe4340c3280 2024-03-06T07:21:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.108+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe43404f330 con 0x7fe4340c3280 2024-03-06T07:21:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.108+0000 7fe42a7fc640 1 -- 172.21.15.26:0/1684412176 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4983 (secure 0 0 0) 0x7fe424057ce0 con 0x7fe4340c3280 2024-03-06T07:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:02.358562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/2470383395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2470383395},{"type":"v1","addr":"172.21.15.117:6836","nonce":2470383395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5209,"qdb_cluster":[5209]},"id":2}]} 2024-03-06T07:21:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe41806b4d0 msgr2=0x7fe41806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe41806b4d0 0x7fe41806d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe434151b50 tx=0x7fe430009040 comp rx=0 tx=0).stop 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3280 msgr2=0x7fe43406f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3280 0x7fe43406f390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c4d0 tx=0x7fe42400c990 comp rx=0 tx=0).stop 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 shutdown_connections 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe41806b4d0 0x7fe41806d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4340c3710 0x7fe43414f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4340c3280 0x7fe43406f390 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 --2- 172.21.15.26:0/1684412176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe43406d9f0 0x7fe4341dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 >> 172.21.15.26:0/1684412176 conn(0x7fe4340aa630 msgr2=0x7fe4340c7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 shutdown_connections 2024-03-06T07:21:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.110+0000 7fe43c1bc640 1 -- 172.21.15.26:0/1684412176 wait complete. 2024-03-06T07:21:11.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:11.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:11.527 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:00.359941+0000', 'modified': '2024-03-06T07:21:02.358562+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5209}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5209': {'gid': 5209, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/2470383395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2470383395}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 2470383395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5209, 'qdb_cluster': [5209]} 2024-03-06T07:21:11.527 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:11.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:21:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.628+0000 7fd547283640 1 Processor -- start 2024-03-06T07:21:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd547283640 1 -- start start 2024-03-06T07:21:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd547283640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5400c8950 0x7fd5400c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd547283640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5401558d0 0x7fd540155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd547283640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd540155130 con 0x7fd5400c69f0 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd547283640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd540155440 con 0x7fd5400c8950 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd547283640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400ad480 con 0x7fd5401558d0 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd545820640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5401558d0 0x7fd540155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.629+0000 7fd54481e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5400c8950 0x7fd5400c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.630+0000 7fd54501f640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd5400c69f0 0x7fd5400acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60970/0 (socket says 172.21.15.26:60970) 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.630+0000 7fd54501f640 1 -- 172.21.15.26:0/1021284527 learned_addr learned my addr 172.21.15.26:0/1021284527 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.630+0000 7fd54481e640 1 -- 172.21.15.26:0/1021284527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5401558d0 msgr2=0x7fd540155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.630+0000 7fd54481e640 1 --2- 172.21.15.26:0/1021284527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5401558d0 0x7fd540155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.630+0000 7fd54481e640 1 -- 172.21.15.26:0/1021284527 >> v1:172.21.15.117:6789/0 conn(0x7fd5400c69f0 legacy=0x7fd5400acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.630+0000 7fd54481e640 1 -- 172.21.15.26:0/1021284527 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400c5d50 con 0x7fd5400c8950 2024-03-06T07:21:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.630+0000 7fd537fff640 1 -- 172.21.15.26:0/1021284527 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983902410 0 0) 0x7fd538001110 con 0x7fd5400c69f0 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd54481e640 1 --2- 172.21.15.26:0/1021284527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400c6dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5300097a0 tx=0x7fd53002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569487408826798d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd537fff640 1 -- 172.21.15.26:0/1021284527 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300315a0 con 0x7fd5400c8950 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd537fff640 1 -- 172.21.15.26:0/1021284527 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530037070 con 0x7fd5400c8950 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd537fff640 1 -- 172.21.15.26:0/1021284527 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530031960 con 0x7fd5400c8950 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 -- 172.21.15.26:0/1021284527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 msgr2=0x7fd5400c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 --2- 172.21.15.26:0/1021284527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400c6dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5300097a0 tx=0x7fd53002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 -- 172.21.15.26:0/1021284527 shutdown_connections 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 --2- 172.21.15.26:0/1021284527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5401558d0 0x7fd540155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 --2- 172.21.15.26:0/1021284527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400c6dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 -- 172.21.15.26:0/1021284527 >> 172.21.15.26:0/1021284527 conn(0x7fd5400cae70 msgr2=0x7fd5400cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 -- 172.21.15.26:0/1021284527 shutdown_connections 2024-03-06T07:21:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.631+0000 7fd547283640 1 -- 172.21.15.26:0/1021284527 wait complete. 2024-03-06T07:21:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.632+0000 7fd547283640 1 Processor -- start 2024-03-06T07:21:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.632+0000 7fd547283640 1 -- start start 2024-03-06T07:21:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd547283640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5400c69f0 0x7fd540154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd547283640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd547283640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5401558d0 0x7fd54014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd547283640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540152a20 con 0x7fd5401558d0 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd547283640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400c6310 con 0x7fd5400c8950 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd547283640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd540152ba0 con 0x7fd5400c69f0 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd545820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5401558d0 0x7fd54014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd54501f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5400c69f0 0x7fd540154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd545820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5401558d0 0x7fd54014c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37008/0 (socket says 172.21.15.26:37008) 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd545820640 1 -- 172.21.15.26:0/150485750 learned_addr learned my addr 172.21.15.26:0/150485750 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.633+0000 7fd54481e640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd545820640 1 -- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5400c69f0 msgr2=0x7fd540154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd545820640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5400c69f0 0x7fd540154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd545820640 1 -- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 msgr2=0x7fd5400ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd545820640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd545820640 1 -- 172.21.15.26:0/150485750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5401ea690 con 0x7fd5401558d0 2024-03-06T07:21:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd54481e640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd545820640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5401558d0 0x7fd54014c400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00efc0 tx=0x7fd53c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd535ffb640 1 -- 172.21.15.26:0/150485750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c018020 con 0x7fd5401558d0 2024-03-06T07:21:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd535ffb640 1 -- 172.21.15.26:0/150485750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c014480 con 0x7fd5401558d0 2024-03-06T07:21:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5401ea970 con 0x7fd5401558d0 2024-03-06T07:21:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd535ffb640 1 -- 172.21.15.26:0/150485750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c014870 con 0x7fd5401558d0 2024-03-06T07:21:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.634+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5401eaf30 con 0x7fd5401558d0 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.635+0000 7fd535ffb640 1 -- 172.21.15.26:0/150485750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd53c005d10 con 0x7fd5401558d0 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.636+0000 7fd535ffb640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd52406b4d0 0x7fd52406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.636+0000 7fd535ffb640 1 -- 172.21.15.26:0/150485750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd53c08acd0 con 0x7fd5401558d0 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.636+0000 7fd54501f640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd52406b4d0 0x7fd52406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.636+0000 7fd54501f640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd52406b4d0 0x7fd52406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd538007fd0 tx=0x7fd538007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 Processor -- start 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 -- start start 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7acd0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd3330 0x558d7acd0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd14b0 0x558d7accd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7ac376b0 con 0x558d7acd2f50 2024-03-06T07:21:11.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7abe0a60 con 0x558d7acd3330 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.637+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d7ac33750 con 0x558d7acd14b0 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7acd0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7acd0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37018/0 (socket says 172.21.15.26:37018) 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 -- 172.21.15.26:0/4125186900 learned_addr learned my addr 172.21.15.26:0/4125186900 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54481e640 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd3330 0x558d7acd0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd545820640 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd14b0 0x558d7accd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 -- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd14b0 msgr2=0x558d7accd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd14b0 0x558d7accd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 -- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd3330 msgr2=0x558d7acd0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd3330 0x558d7acd0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 -- 172.21.15.26:0/4125186900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d7abfef30 con 0x558d7acd2f50 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd54501f640 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7acd0a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd5380077e0 tx=0x7fd538013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db42e3af9bdc5246 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd52affd640 1 -- 172.21.15.26:0/4125186900 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538026070 con 0x558d7acd2f50 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd52affd640 1 -- 172.21.15.26:0/4125186900 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538001cf0 con 0x558d7acd2f50 2024-03-06T07:21:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.638+0000 7fd52affd640 1 -- 172.21.15.26:0/4125186900 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538013d70 con 0x558d7acd2f50 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 -- 172.21.15.26:0/4125186900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 msgr2=0x558d7acd0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7acd0a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd5380077e0 tx=0x7fd538013000 comp rx=0 tx=0).stop 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 -- 172.21.15.26:0/4125186900 shutdown_connections 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd14b0 0x558d7accd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd3330 0x558d7acd0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 --2- 172.21.15.26:0/4125186900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7acd0a30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 -- 172.21.15.26:0/4125186900 >> 172.21.15.26:0/4125186900 conn(0x558d7accf7d0 msgr2=0x558d7acf43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 -- 172.21.15.26:0/4125186900 shutdown_connections 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 -- 172.21.15.26:0/4125186900 wait complete. 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 -- start start 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.639+0000 7fd549e5f740 1 Processor -- start 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7ad22f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7acf3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7ad20640 0x558d7ad0b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd549e5f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7acf5230 con 0x558d7acd2f50 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7abf3ea0 con 0x558d7ad20640 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d7acf4c70 con 0x558d7acd3330 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd549e5f740 10 client.0 _ll_register_callbacks cb 0x558d7abf3870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd54501f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7ad22f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd54501f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7ad22f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37020/0 (socket says 172.21.15.26:37020) 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd54501f640 1 -- 172.21.15.26:0/1383812129 learned_addr learned my addr 172.21.15.26:0/1383812129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd54481e640 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7acf3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.640+0000 7fd545820640 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7ad20640 0x558d7ad0b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd54501f640 1 -- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 msgr2=0x558d7acf3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd54501f640 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7acf3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd54501f640 1 -- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7ad20640 msgr2=0x558d7ad0b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd54501f640 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7ad20640 0x558d7ad0b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd54501f640 1 -- 172.21.15.26:0/1383812129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d7abf4270 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd54501f640 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7ad22f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x558d7ad125d0 tx=0x7fd538000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146a671e5986c0e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd51ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd51ffff640 1 -- 172.21.15.26:0/1383812129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538026070 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd51ffff640 1 -- 172.21.15.26:0/1383812129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd538021a70 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd538001cf0 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd51ffff640 1 -- 172.21.15.26:0/1383812129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd538039050 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 10 client.5439 fetch_fsmap learned FSMap version 16 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558d7a94c2d0 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd51ffff640 1 -- 172.21.15.26:0/1383812129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538001cf0 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd51ffff640 1 -- 172.21.15.26:0/1383812129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 16) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd53803c020 con 0x558d7acd2f50 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 10 client.5439 fetch_fsmap finished waiting for FSMap version 16 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 10 client.5439 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 20 client.5439 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7acf5f30 0x558d7ad2e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 4 client.5439 mds_command: new command op to 5209 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.641+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 --> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d7ac44bf0 con 0x558d7acf5f30 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.642+0000 7fd54481e640 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7acf5f30 0x558d7ad2e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.642+0000 7fd54481e640 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7acf5f30 0x558d7ad2e5a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65331d8268e1bda0 server_cookie=6c67d0b875d506d in_seq=0 out_seq=0 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.642+0000 7fd51ffff640 10 client.5439 ms_handle_connect on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd51ffff640 1 -- 172.21.15.26:0/1383812129 <== mds.0 v2:172.21.15.117:6834/2470383395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558d7ac44bf0 con 0x558d7acf5f30 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd51ffff640 10 client.5439 handle_command_reply: tid=0 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 client.5439 shutdown 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 msgr2=0x558d7ad22f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7ad22f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x558d7ad125d0 tx=0x7fd538000ed0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 shutdown_connections 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7acf5f30 0x558d7ad2e5a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7ad20640 0x558d7ad0b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7acf3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd51ffff640 0 client.5439 ms_handle_reset on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 --2- 172.21.15.26:0/1383812129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd2f50 0x558d7ad22f00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 >> 172.21.15.26:0/1383812129 conn(0x558d7acd14b0 msgr2=0x558d7ad0fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 shutdown_connections 2024-03-06T07:21:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 1 -- 172.21.15.26:0/1383812129 wait complete. 2024-03-06T07:21:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.643+0000 7fd549e5f740 20 client.5439 trim_cache size 0 max 16384 2024-03-06T07:21:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 Processor -- start 2024-03-06T07:21:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 -- start start 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7ad335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd3330 0x558d7acf76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acf5f30 0x558d7acf8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401f6e20 con 0x558d7acd3330 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401f6220 con 0x558d7acd2f50 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401f6820 con 0x558d7acf5f30 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.660+0000 7fd54481e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd3330 0x558d7acf76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd3330 0x558d7acf76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37026/0 (socket says 172.21.15.26:37026) 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 -- 172.21.15.26:0/2842460731 learned_addr learned my addr 172.21.15.26:0/2842460731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54501f640 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7ad335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd545820640 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acf5f30 0x558d7acf8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 -- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acf5f30 msgr2=0x558d7acf8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acf5f30 0x558d7acf8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 -- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 msgr2=0x558d7ad335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7ad335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 -- 172.21.15.26:0/2842460731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d7aa166b0 con 0x558d7acd3330 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54501f640 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7ad335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd545820640 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acf5f30 0x558d7acf8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd54481e640 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd3330 0x558d7acf76f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd5300097a0 tx=0x7fd530030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fadf1ec6b4d64d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd52a7fc640 1 -- 172.21.15.26:0/2842460731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530053070 con 0x558d7acd3330 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd52a7fc640 1 -- 172.21.15.26:0/2842460731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530037040 con 0x558d7acd3330 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.661+0000 7fd52a7fc640 1 -- 172.21.15.26:0/2842460731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300044b0 con 0x558d7acd3330 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 -- 172.21.15.26:0/2842460731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd3330 msgr2=0x558d7acf76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd3330 0x558d7acf76f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd5300097a0 tx=0x7fd530030000 comp rx=0 tx=0).stop 2024-03-06T07:21:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 -- 172.21.15.26:0/2842460731 shutdown_connections 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acf5f30 0x558d7acf8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acd3330 0x558d7acf76f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 --2- 172.21.15.26:0/2842460731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7ad335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 -- 172.21.15.26:0/2842460731 >> 172.21.15.26:0/2842460731 conn(0x558d7acd14b0 msgr2=0x558d7acce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 -- 172.21.15.26:0/2842460731 shutdown_connections 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 -- 172.21.15.26:0/2842460731 wait complete. 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 -- start start 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.662+0000 7fd549e5f740 1 Processor -- start 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7acd1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7ad7ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd549e5f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acf5f30 0x558d7ad7bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd549e5f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7ad0c2f0 con 0x558d7acf5f30 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7ad1fe40 con 0x558d7acd2f50 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd549e5f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d7acf6d70 con 0x558d7acd3330 2024-03-06T07:21:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd549e5f740 10 client.0 _ll_register_callbacks cb 0x7fd53c0556a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acf5f30 0x558d7ad7bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acf5f30 0x558d7ad7bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37040/0 (socket says 172.21.15.26:37040) 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 -- 172.21.15.26:0/300443367 learned_addr learned my addr 172.21.15.26:0/300443367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd54481e640 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7ad7ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd54501f640 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7acd1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 -- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 msgr2=0x558d7ad7ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7ad7ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 -- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 msgr2=0x558d7acd1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7acd1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.663+0000 7fd545820640 1 -- 172.21.15.26:0/300443367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d7ad132f0 con 0x558d7acf5f30 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd545820640 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acf5f30 0x558d7ad7bc30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd53c059390 tx=0x7fd53c0591d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3f8dc04fe92bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd51ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:11.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd51ffff640 1 -- 172.21.15.26:0/300443367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c058a50 con 0x558d7acf5f30 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd51ffff640 1 -- 172.21.15.26:0/300443367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd53c05f430 con 0x558d7acf5f30 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd530037040 con 0x558d7acf5f30 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd51ffff640 1 -- 172.21.15.26:0/300443367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd53c05f650 con 0x558d7acf5f30 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 10 client.5445 fetch_fsmap learned FSMap version 16 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558d7acf7f80 con 0x558d7acf5f30 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd51ffff640 1 -- 172.21.15.26:0/300443367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530037040 con 0x558d7acf5f30 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd51ffff640 1 -- 172.21.15.26:0/300443367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 16) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd53c055150 con 0x558d7acf5f30 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 10 client.5445 fetch_fsmap finished waiting for FSMap version 16 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 10 client.5445 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 20 client.5445 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7ad20640 0x558d7ad81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 4 client.5445 mds_command: new command op to 5209 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 --> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558d7ac3ef40 con 0x558d7ad20640 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.664+0000 7fd54501f640 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7ad20640 0x558d7ad81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd54501f640 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7ad20640 0x558d7ad81f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31e2f24b8dc549e server_cookie=82f6a5dd922e9c68 in_seq=0 out_seq=0 2024-03-06T07:21:11.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd51ffff640 10 client.5445 ms_handle_connect on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd51ffff640 1 -- 172.21.15.26:0/300443367 <== mds.0 v2:172.21.15.117:6834/2470383395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x558d7ac3ef40 con 0x558d7ad20640 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd51ffff640 10 client.5445 handle_command_reply: tid=0 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 client.5445 shutdown 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acf5f30 msgr2=0x558d7ad7bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acf5f30 0x558d7ad7bc30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd53c059390 tx=0x7fd53c0591d0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 shutdown_connections 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d7acf5f30 0x558d7ad7bc30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558d7acd3330 0x558d7ad7ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:6834/2470383395,v1:172.21.15.117:6836/2470383395] conn(0x558d7ad20640 0x558d7ad81f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 --2- 172.21.15.26:0/300443367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7acd2f50 0x558d7acd1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd51ffff640 0 client.5445 ms_handle_reset on v2:172.21.15.117:6834/2470383395 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 >> 172.21.15.26:0/300443367 conn(0x558d7acd14b0 msgr2=0x558d7acf76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 shutdown_connections 2024-03-06T07:21:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 1 -- 172.21.15.26:0/300443367 wait complete. 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.665+0000 7fd549e5f740 20 client.5445 trim_cache size 0 max 16384 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5209, 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.673056229, 2024-03-06T07:21:11.675 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 16, 2024-03-06T07:21:11.676 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 37, 2024-03-06T07:21:11.676 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 37, 2024-03-06T07:21:11.676 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 18.205231179999998 2024-03-06T07:21:11.676 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd52406b4d0 msgr2=0x7fd52406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd52406b4d0 0x7fd52406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd538007fd0 tx=0x7fd538007de0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5401558d0 msgr2=0x7fd54014c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5401558d0 0x7fd54014c400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00efc0 tx=0x7fd53c00c490 comp rx=0 tx=0).stop 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 shutdown_connections 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd52406b4d0 0x7fd52406d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5401558d0 0x7fd54014c400 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5400c8950 0x7fd5400ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 --2- 172.21.15.26:0/150485750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5400c69f0 0x7fd540154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 >> 172.21.15.26:0/150485750 conn(0x7fd5400cae70 msgr2=0x7fd540150ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 shutdown_connections 2024-03-06T07:21:11.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:11.666+0000 7fd547283640 1 -- 172.21.15.26:0/150485750 wait complete. 2024-03-06T07:21:11.682 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 16, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:00.359941+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5209": { "addr": "172.21.15.117:6836/2470383395", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2470383395, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 2470383395, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5209, "incarnation": 14, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:02.358562+0000", "qdb_cluster": [ 5209 ], "qdb_leader": 5209, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5209 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/2730813301", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 2730813301, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2730813301, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4910, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6833/3831184543", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3831184543, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3831184543, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5202, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2694023942", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2694023942, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2694023942, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5205, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:11.684 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 2, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:21:11.684 INFO:tasks.cephfs.test_failover:current = [{'gid': 5202, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/3831184543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3831184543}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3831184543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}, {'gid': 5209, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/2470383395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2470383395}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 2470383395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5205, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2694023942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2694023942}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2694023942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 4910, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}] 2024-03-06T07:21:11.684 DEBUG:tasks.cephfs.test_failover:2 != -1 2024-03-06T07:21:11.684 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:21:14.115 INFO:tasks.ceph.mds.b.smithi117.stderr: -356> 2024-03-06T07:21:05.314+0000 7f91a74e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:14.141 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:21:16.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:16.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.797+0000 7f779de9d640 1 Processor -- start 2024-03-06T07:21:16.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.797+0000 7f779de9d640 1 -- start start 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f779de9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7798072f70 0x7f7798154570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f779de9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f77980c6560 0x7f7798154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f779de9d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77980bdc30 con 0x7f77980bbf80 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f779de9d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980bd690 con 0x7f7798072f70 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f779de9d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980bd950 con 0x7f77980c6560 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f77977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7798072f70 0x7f7798154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f77977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7798072f70 0x7f7798154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37044/0 (socket says 172.21.15.26:37044) 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f77977fe640 1 -- 172.21.15.26:0/3121430379 learned_addr learned my addr 172.21.15.26:0/3121430379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:16.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f7796ffd640 1 --2- 172.21.15.26:0/3121430379 >> v2:172.21.15.117:3300/0 conn(0x7f77980c6560 0x7f7798154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:16.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f77977fe640 1 -- 172.21.15.26:0/3121430379 >> v2:172.21.15.117:3300/0 conn(0x7f77980c6560 msgr2=0x7f7798154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:16.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f77977fe640 1 --2- 172.21.15.26:0/3121430379 >> v2:172.21.15.117:3300/0 conn(0x7f77980c6560 0x7f7798154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:16.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f77977fe640 1 -- 172.21.15.26:0/3121430379 >> v1:172.21.15.117:6789/0 conn(0x7f77980bbf80 legacy=0x7f7798156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:16.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.798+0000 7f77977fe640 1 -- 172.21.15.26:0/3121430379 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980c5540 con 0x7f7798072f70 2024-03-06T07:21:16.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.799+0000 7f77977fe640 1 --2- 172.21.15.26:0/3121430379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f7798154570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f77980c4650 tx=0x7f778802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2509140c5a9bbf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:16.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.799+0000 7f77967fc640 1 -- 172.21.15.26:0/3121430379 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788035020 con 0x7f7798072f70 2024-03-06T07:21:16.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.799+0000 7f77967fc640 1 -- 172.21.15.26:0/3121430379 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788031870 con 0x7f7798072f70 2024-03-06T07:21:16.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.799+0000 7f77967fc640 1 -- 172.21.15.26:0/3121430379 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788031c40 con 0x7f7798072f70 2024-03-06T07:21:16.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 -- 172.21.15.26:0/3121430379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 msgr2=0x7f7798154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:16.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 --2- 172.21.15.26:0/3121430379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f7798154570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f77980c4650 tx=0x7f778802b910 comp rx=0 tx=0).stop 2024-03-06T07:21:16.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 -- 172.21.15.26:0/3121430379 shutdown_connections 2024-03-06T07:21:16.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 --2- 172.21.15.26:0/3121430379 >> v2:172.21.15.117:3300/0 conn(0x7f77980c6560 0x7f7798154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:16.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 --2- 172.21.15.26:0/3121430379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f7798154570 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:16.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 -- 172.21.15.26:0/3121430379 >> 172.21.15.26:0/3121430379 conn(0x7f779806b1a0 msgr2=0x7f779806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:16.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 -- 172.21.15.26:0/3121430379 shutdown_connections 2024-03-06T07:21:16.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.800+0000 7f779de9d640 1 -- 172.21.15.26:0/3121430379 wait complete. 2024-03-06T07:21:16.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.801+0000 7f779de9d640 1 Processor -- start 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.801+0000 7f779de9d640 1 -- start start 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f779de9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f77981e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f779de9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77980bbf80 0x7f77980aa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f779de9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77980c6560 0x7f77980a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f779de9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798073350 con 0x7f7798072f70 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f779de9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980bba60 con 0x7f77980bbf80 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f779de9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77980734d0 con 0x7f77980c6560 2024-03-06T07:21:16.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f77981e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f77981e9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37058/0 (socket says 172.21.15.26:37058) 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 -- 172.21.15.26:0/3650781793 learned_addr learned my addr 172.21.15.26:0/3650781793 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 -- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77980c6560 msgr2=0x7f77980a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f7796ffd640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77980bbf80 0x7f77980aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77980c6560 0x7f77980a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 -- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77980bbf80 msgr2=0x7f77980aa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77980bbf80 0x7f77980aa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 -- 172.21.15.26:0/3650781793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980ac1c0 con 0x7f7798072f70 2024-03-06T07:21:16.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f7796ffd640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77980bbf80 0x7f77980aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:16.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.802+0000 7f77977fe640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f77981e9b70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7788000c00 tx=0x7f77880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:16.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.803+0000 7f777ffff640 1 -- 172.21.15.26:0/3650781793 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788040070 con 0x7f7798072f70 2024-03-06T07:21:16.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.803+0000 7f777ffff640 1 -- 172.21.15.26:0/3650781793 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788030d00 con 0x7f7798072f70 2024-03-06T07:21:16.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.803+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77980ac4a0 con 0x7f7798072f70 2024-03-06T07:21:16.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.803+0000 7f777ffff640 1 -- 172.21.15.26:0/3650781793 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788035040 con 0x7f7798072f70 2024-03-06T07:21:16.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.803+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77980a6ae0 con 0x7f7798072f70 2024-03-06T07:21:16.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.804+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77980acd30 con 0x7f7798072f70 2024-03-06T07:21:16.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.804+0000 7f777ffff640 1 -- 172.21.15.26:0/3650781793 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f778804b060 con 0x7f7798072f70 2024-03-06T07:21:16.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.804+0000 7f777ffff640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f778406b500 0x7f778406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:16.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.805+0000 7f777ffff640 1 -- 172.21.15.26:0/3650781793 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f77880ac200 con 0x7f7798072f70 2024-03-06T07:21:16.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.805+0000 7f7796ffd640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f778406b500 0x7f778406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:16.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.805+0000 7f7796ffd640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f778406b500 0x7f778406d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f778c006fd0 tx=0x7f778c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:16.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:16.809+0000 7f777ffff640 1 -- 172.21.15.26:0/3650781793 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f77980acd30 con 0x7f7798072f70 2024-03-06T07:21:17.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.031+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f779804f330 con 0x7f7798072f70 2024-03-06T07:21:17.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.031+0000 7f777ffff640 1 -- 172.21.15.26:0/3650781793 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4976 (secure 0 0 0) 0x7f778807ae10 con 0x7f7798072f70 2024-03-06T07:21:17.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:17.034 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:15.118831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":2,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-03-06T07:21:17.035 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-03-06T07:21:17.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f778406b500 msgr2=0x7f778406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:17.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f778406b500 0x7f778406d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f778c006fd0 tx=0x7f778c008040 comp rx=0 tx=0).stop 2024-03-06T07:21:17.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 msgr2=0x7f77981e9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f77981e9b70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7788000c00 tx=0x7f77880046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 shutdown_connections 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f778406b500 0x7f778406d9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77980c6560 0x7f77980a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77980bbf80 0x7f77980aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 --2- 172.21.15.26:0/3650781793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7798072f70 0x7f77981e9b70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 >> 172.21.15.26:0/3650781793 conn(0x7f779806b1a0 msgr2=0x7f77980c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 shutdown_connections 2024-03-06T07:21:17.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:17.034+0000 7f779de9d640 1 -- 172.21.15.26:0/3650781793 wait complete. 2024-03-06T07:21:17.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:00.359941+0000', 'modified': '2024-03-06T07:21:15.118831+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 38, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4910': {'gid': 4910, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:21:17.450 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4910:up:rejoin 2024-03-06T07:21:18.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:18.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.563+0000 7f52a453e640 1 Processor -- start 2024-03-06T07:21:18.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.563+0000 7f52a453e640 1 -- start start 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a453e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c06d9f0 0x7f529c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a453e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f529c0c3280 0x7f529c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a453e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f529c1551e0 0x7f529c155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a453e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c158800 con 0x7f529c0c3280 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a453e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c158ae0 con 0x7f529c1551e0 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a453e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c158dc0 con 0x7f529c06d9f0 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f529c0c3280 0x7f529c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f529c0c3280 0x7f529c0c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47254/0 (socket says 172.21.15.26:47254) 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 -- 172.21.15.26:0/1606490299 learned_addr learned my addr 172.21.15.26:0/1606490299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a22da640 1 --2- 172.21.15.26:0/1606490299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c06d9f0 0x7f529c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a2adb640 1 --2- 172.21.15.26:0/1606490299 >> v2:172.21.15.117:3300/0 conn(0x7f529c1551e0 0x7f529c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 -- 172.21.15.26:0/1606490299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c06d9f0 msgr2=0x7f529c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 --2- 172.21.15.26:0/1606490299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c06d9f0 0x7f529c0a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 -- 172.21.15.26:0/1606490299 >> v2:172.21.15.117:3300/0 conn(0x7f529c1551e0 msgr2=0x7f529c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 --2- 172.21.15.26:0/1606490299 >> v2:172.21.15.117:3300/0 conn(0x7f529c1551e0 0x7f529c155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.564+0000 7f52a1ad9640 1 -- 172.21.15.26:0/1606490299 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c1590a0 con 0x7f529c0c3280 2024-03-06T07:21:18.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.565+0000 7f52a1ad9640 1 --2- 172.21.15.26:0/1606490299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c0c3280 0x7f529c0c36e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f529c0c3130 tx=0x7f528c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96d63ae73a3d6ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.565+0000 7f52a12d8640 1 -- 172.21.15.26:0/1606490299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c02e070 con 0x7f529c0c3280 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.565+0000 7f52a12d8640 1 -- 172.21.15.26:0/1606490299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c037070 con 0x7f529c0c3280 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.565+0000 7f52a12d8640 1 -- 172.21.15.26:0/1606490299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c031a00 con 0x7f529c0c3280 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 -- 172.21.15.26:0/1606490299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c0c3280 msgr2=0x7f529c0c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 --2- 172.21.15.26:0/1606490299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c0c3280 0x7f529c0c36e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f529c0c3130 tx=0x7f528c02b880 comp rx=0 tx=0).stop 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 -- 172.21.15.26:0/1606490299 shutdown_connections 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 --2- 172.21.15.26:0/1606490299 >> v2:172.21.15.117:3300/0 conn(0x7f529c1551e0 0x7f529c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 --2- 172.21.15.26:0/1606490299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c0c3280 0x7f529c0c36e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 --2- 172.21.15.26:0/1606490299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c06d9f0 0x7f529c0a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 -- 172.21.15.26:0/1606490299 >> 172.21.15.26:0/1606490299 conn(0x7f529c0aa630 msgr2=0x7f529c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 -- 172.21.15.26:0/1606490299 shutdown_connections 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.566+0000 7f52a453e640 1 -- 172.21.15.26:0/1606490299 wait complete. 2024-03-06T07:21:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.567+0000 7f52a453e640 1 Processor -- start 2024-03-06T07:21:18.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.567+0000 7f52a453e640 1 -- start start 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a453e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c06d9f0 0x7f529c06f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a453e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529c0c3280 0x7f529c1e1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a453e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c1551e0 0x7f529c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a453e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c06df50 con 0x7f529c06d9f0 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a453e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c06ddd0 con 0x7f529c0c3280 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a453e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0a8210 con 0x7f529c1551e0 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c06d9f0 0x7f529c06f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c06d9f0 0x7f529c06f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47256/0 (socket says 172.21.15.26:47256) 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 -- 172.21.15.26:0/1385149367 learned_addr learned my addr 172.21.15.26:0/1385149367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a2adb640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c1551e0 0x7f529c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a1ad9640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529c0c3280 0x7f529c1e1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 -- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c1551e0 msgr2=0x7f529c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c1551e0 0x7f529c151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 -- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529c0c3280 msgr2=0x7f529c1e1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529c0c3280 0x7f529c1e1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 -- 172.21.15.26:0/1385149367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c1eb7d0 con 0x7f529c06d9f0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f52a22da640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c06d9f0 0x7f529c06f390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f529800e8a0 tx=0x7f529800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f5292ffd640 1 -- 172.21.15.26:0/1385149367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298017070 con 0x7f529c06d9f0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.568+0000 7f5292ffd640 1 -- 172.21.15.26:0/1385149367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529800c940 con 0x7f529c06d9f0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.569+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c1ebab0 con 0x7f529c06d9f0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.569+0000 7f5292ffd640 1 -- 172.21.15.26:0/1385149367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800cd30 con 0x7f529c06d9f0 2024-03-06T07:21:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.569+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c06f8d0 con 0x7f529c06d9f0 2024-03-06T07:21:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.570+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c075300 con 0x7f529c06d9f0 2024-03-06T07:21:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.570+0000 7f5292ffd640 1 -- 172.21.15.26:0/1385149367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f529801a070 con 0x7f529c06d9f0 2024-03-06T07:21:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.570+0000 7f5292ffd640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528006b4d0 0x7f528006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.570+0000 7f5292ffd640 1 -- 172.21.15.26:0/1385149367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f529808a580 con 0x7f529c06d9f0 2024-03-06T07:21:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.571+0000 7f52a1ad9640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528006b4d0 0x7f528006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.571+0000 7f52a1ad9640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528006b4d0 0x7f528006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f528c036000 tx=0x7f528c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.575+0000 7f5292ffd640 1 -- 172.21.15.26:0/1385149367 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f529c075300 con 0x7f529c06d9f0 2024-03-06T07:21:18.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.797+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f529c04f330 con 0x7f529c06d9f0 2024-03-06T07:21:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.797+0000 7f5292ffd640 1 -- 172.21.15.26:0/1385149367 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f5298058bd0 con 0x7f529c06d9f0 2024-03-06T07:21:18.800 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:18.800 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:17.117874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":2,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4910,"qdb_cluster":[4910]},"id":2}]} 2024-03-06T07:21:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-03-06T07:21:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528006b4d0 msgr2=0x7f528006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528006b4d0 0x7f528006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f528c036000 tx=0x7f528c03d040 comp rx=0 tx=0).stop 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c06d9f0 msgr2=0x7f529c06f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c06d9f0 0x7f529c06f390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f529800e8a0 tx=0x7f529800ed60 comp rx=0 tx=0).stop 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 shutdown_connections 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528006b4d0 0x7f528006d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529c1551e0 0x7f529c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529c0c3280 0x7f529c1e1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 --2- 172.21.15.26:0/1385149367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f529c06d9f0 0x7f529c06f390 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 >> 172.21.15.26:0/1385149367 conn(0x7f529c0aa630 msgr2=0x7f529c155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 shutdown_connections 2024-03-06T07:21:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:18.800+0000 7f52a453e640 1 -- 172.21.15.26:0/1385149367 wait complete. 2024-03-06T07:21:19.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:00.359941+0000', 'modified': '2024-03-06T07:21:17.117874+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 38, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4910': {'gid': 4910, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4910, 'qdb_cluster': [4910]} 2024-03-06T07:21:19.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:19.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:21:19.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.320+0000 7fb9b3881640 1 Processor -- start 2024-03-06T07:21:19.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.320+0000 7fb9b3881640 1 -- start start 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b3881640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b3881640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac0733f0 con 0x7fb9ac152720 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b3881640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac155ef0 con 0x7fb9ac072cd0 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b3881640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac155ca0 con 0x7fb9ac06f550 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b161d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb9ac152720 0x7fb9ac0c77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55100/0 (socket says 172.21.15.26:55100) 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b161d640 1 -- 172.21.15.26:0/4100533565 learned_addr learned my addr 172.21.15.26:0/4100533565 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/4100533565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937771277 0 0) 0x7fb9ac0733f0 con 0x7fb9ac152720 2024-03-06T07:21:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb998003520 con 0x7fb9ac152720 2024-03-06T07:21:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046424431 0 0) 0x7fb9ac155ef0 con 0x7fb9ac072cd0 2024-03-06T07:21:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ac0733f0 con 0x7fb9ac072cd0 2024-03-06T07:21:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208214512 0 0) 0x7fb998003520 con 0x7fb9ac152720 2024-03-06T07:21:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b0e1c640 1 -- 172.21.15.26:0/4100533565 >> v1:172.21.15.117:6789/0 conn(0x7fb9ac072cd0 legacy=0x7fb9ac06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b0e1c640 1 -- 172.21.15.26:0/4100533565 >> v1:172.21.15.26:6789/0 conn(0x7fb9ac152720 legacy=0x7fb9ac0c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.321+0000 7fb9b0e1c640 1 -- 172.21.15.26:0/4100533565 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac1588e0 con 0x7fb9ac06f550 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.322+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/4100533565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1593c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac155e20 tx=0x7fb99c02b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e600ace2d4b8e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.322+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c037070 con 0x7fb9ac06f550 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.322+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c030510 con 0x7fb9ac06f550 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.322+0000 7fb9a3fff640 1 -- 172.21.15.26:0/4100533565 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8001670 con 0x7fb9ac06f550 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.323+0000 7fb9b3881640 1 -- 172.21.15.26:0/4100533565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 msgr2=0x7fb9ac1593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.323+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4100533565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1593c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac155e20 tx=0x7fb99c02b760 comp rx=0 tx=0).stop 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.323+0000 7fb9b3881640 1 -- 172.21.15.26:0/4100533565 shutdown_connections 2024-03-06T07:21:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.323+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4100533565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1593c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.323+0000 7fb9b3881640 1 -- 172.21.15.26:0/4100533565 >> 172.21.15.26:0/4100533565 conn(0x7fb9ac06ea40 msgr2=0x7fb9ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.323+0000 7fb9b3881640 1 -- 172.21.15.26:0/4100533565 shutdown_connections 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.323+0000 7fb9b3881640 1 -- 172.21.15.26:0/4100533565 wait complete. 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.324+0000 7fb9b3881640 1 Processor -- start 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.324+0000 7fb9b3881640 1 -- start start 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.324+0000 7fb9b3881640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1ebc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b3881640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac072cd0 0x7fb9ac1edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b3881640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac152720 0x7fb9ac1ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b3881640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac158ea0 con 0x7fb9ac152720 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b3881640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1561d0 con 0x7fb9ac072cd0 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b3881640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0c7ef0 con 0x7fb9ac06f550 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac152720 0x7fb9ac1ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac152720 0x7fb9ac1ee4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47264/0 (socket says 172.21.15.26:47264) 2024-03-06T07:21:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/4241062700 learned_addr learned my addr 172.21.15.26:0/4241062700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b161d640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1ebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac072cd0 0x7fb9ac1edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 msgr2=0x7fb9ac1ebc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1ebc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac072cd0 msgr2=0x7fb9ac1edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac072cd0 0x7fb9ac1edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/4241062700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac1eea20 con 0x7fb9ac152720 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac152720 0x7fb9ac1ee4e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4002230 tx=0x7fb9a400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9a1ffb640 1 -- 172.21.15.26:0/4241062700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4003b70 con 0x7fb9ac152720 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9a1ffb640 1 -- 172.21.15.26:0/4241062700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a4003d10 con 0x7fb9ac152720 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac0c9160 con 0x7fb9ac152720 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.325+0000 7fb9a1ffb640 1 -- 172.21.15.26:0/4241062700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40052d0 con 0x7fb9ac152720 2024-03-06T07:21:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.326+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac153e00 con 0x7fb9ac152720 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.327+0000 7fb9a1ffb640 1 -- 172.21.15.26:0/4241062700 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9a4005680 con 0x7fb9ac152720 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.327+0000 7fb9a1ffb640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb99006b500 0x7fb99006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.328+0000 7fb9a1ffb640 1 -- 172.21.15.26:0/4241062700 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb9a408a330 con 0x7fb9ac152720 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.328+0000 7fb9b161d640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb99006b500 0x7fb99006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.328+0000 7fb9b161d640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb99006b500 0x7fb99006d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80011e0 tx=0x7fb9a8001670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 Processor -- start 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 -- start start 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658991f50 0x55965898fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658992330 0x55965898ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589904b0 0x55965898c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596588f66b0 con 0x559658992330 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965889fa60 con 0x5596589904b0 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596588f2750 con 0x559658991f50 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b0e1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658992330 0x55965898ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b161d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658991f50 0x55965898fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b0e1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658992330 0x55965898ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47266/0 (socket says 172.21.15.26:47266) 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b161d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658991f50 0x55965898fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:38538/0 (socket says 172.21.15.26:38538) 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b0e1c640 1 -- 172.21.15.26:0/2316667113 learned_addr learned my addr 172.21.15.26:0/2316667113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.329+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589904b0 0x55965898c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb9b161d640 1 -- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589904b0 msgr2=0x55965898c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb9b161d640 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589904b0 0x55965898c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb9b161d640 1 -- 172.21.15.26:0/2316667113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658992330 msgr2=0x55965898ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb9b161d640 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658992330 0x55965898ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb9b161d640 1 -- 172.21.15.26:0/2316667113 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596588bdf30 con 0x559658991f50 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658992330 0x55965898ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb9b161d640 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658991f50 0x55965898fa30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8005880 tx=0x7fb9a8012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac34008e636e3c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb996ffd640 1 -- 172.21.15.26:0/2316667113 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8025070 con 0x559658991f50 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb996ffd640 1 -- 172.21.15.26:0/2316667113 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a801b050 con 0x559658991f50 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.330+0000 7fb996ffd640 1 -- 172.21.15.26:0/2316667113 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8020410 con 0x559658991f50 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 -- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658991f50 msgr2=0x55965898fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658991f50 0x55965898fa30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8005880 tx=0x7fb9a8012000 comp rx=0 tx=0).stop 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 -- 172.21.15.26:0/2316667113 shutdown_connections 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589904b0 0x55965898c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658992330 0x55965898ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 --2- 172.21.15.26:0/2316667113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658991f50 0x55965898fa30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 -- 172.21.15.26:0/2316667113 >> 172.21.15.26:0/2316667113 conn(0x55965898e7d0 msgr2=0x5596589b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 -- 172.21.15.26:0/2316667113 shutdown_connections 2024-03-06T07:21:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 -- 172.21.15.26:0/2316667113 wait complete. 2024-03-06T07:21:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 -- start start 2024-03-06T07:21:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.331+0000 7fb9b6467740 1 Processor -- start 2024-03-06T07:21:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559658991f50 0x5596589b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x5596589b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596589b5f10 0x5596589add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b6467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596589b4230 con 0x5596589b5f10 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596588b2ea0 con 0x559658991f50 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596589b3c70 con 0x559658992330 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b6467740 10 client.0 _ll_register_callbacks cb 0x5596588b2870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b1e1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596589b5f10 0x5596589add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b1e1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596589b5f10 0x5596589add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47270/0 (socket says 172.21.15.26:47270) 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/3872708701 learned_addr learned my addr 172.21.15.26:0/3872708701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b161d640 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559658991f50 0x5596589b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x5596589b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 msgr2=0x5596589b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x5596589b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559658991f50 msgr2=0x5596589b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.332+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559658991f50 0x5596589b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b1e1e640 1 -- 172.21.15.26:0/3872708701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596588b3270 con 0x5596589b5f10 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596589b5f10 0x5596589add20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4054df0 tx=0x7fb9a405caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f90c3be5404cb40 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb98bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb98bfff640 1 -- 172.21.15.26:0/3872708701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40701d0 con 0x5596589b5f10 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb98bfff640 1 -- 172.21.15.26:0/3872708701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb9a4070370 con 0x5596589b5f10 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a801b050 con 0x5596589b5f10 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb98bfff640 1 -- 172.21.15.26:0/3872708701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a4057de0 con 0x5596589b5f10 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 10 client.5472 fetch_fsmap learned FSMap version 21 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55965860b2d0 con 0x5596589b5f10 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb98bfff640 1 -- 172.21.15.26:0/3872708701 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a801b050 con 0x5596589b5f10 2024-03-06T07:21:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb98bfff640 1 -- 172.21.15.26:0/3872708701 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fb9a4057f80 con 0x5596589b5f10 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 10 client.5472 fetch_fsmap finished waiting for FSMap version 21 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 10 client.5472 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 20 client.5472 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x5596589ce590 0x5596589ed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 4 client.5472 mds_command: new command op to 4910 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.333+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 --> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559658903bf0 con 0x5596589ce590 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.334+0000 7fb9b161d640 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x5596589ce590 0x5596589ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.334+0000 7fb9b161d640 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x5596589ce590 0x5596589ed570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59a4cd053ec61c6 server_cookie=3652266a3d68554e in_seq=0 out_seq=0 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.334+0000 7fb98bfff640 10 client.5472 ms_handle_connect on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.334+0000 7fb98bfff640 1 -- 172.21.15.26:0/3872708701 <== mds.0 v2:172.21.15.117:6835/2730813301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559658903bf0 con 0x5596589ce590 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.334+0000 7fb98bfff640 10 client.5472 handle_command_reply: tid=0 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 client.5472 shutdown 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596589b5f10 msgr2=0x5596589add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596589b5f10 0x5596589add20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4054df0 tx=0x7fb9a405caf0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 shutdown_connections 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x5596589ce590 0x5596589ed570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596589b5f10 0x5596589add20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x5596589b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb98bfff640 0 client.5472 ms_handle_reset on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3872708701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559658991f50 0x5596589b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 >> 172.21.15.26:0/3872708701 conn(0x5596589904b0 msgr2=0x5596589afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 shutdown_connections 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 1 -- 172.21.15.26:0/3872708701 wait complete. 2024-03-06T07:21:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.335+0000 7fb9b6467740 20 client.5472 trim_cache size 0 max 16384 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.351+0000 7fb9b6467740 1 Processor -- start 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.351+0000 7fb9b6467740 1 -- start start 2024-03-06T07:21:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x55965898fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x55965898d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x5596589b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b6467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1f6d20 con 0x559658991f50 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1eb610 con 0x5596589b5f10 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1f6720 con 0x559658992330 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x55965898fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x55965898fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47282/0 (socket says 172.21.15.26:47282) 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 -- 172.21.15.26:0/3740424794 learned_addr learned my addr 172.21.15.26:0/3740424794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x5596589b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x55965898d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 -- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 msgr2=0x55965898d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x55965898d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 -- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 msgr2=0x5596589b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x5596589b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.352+0000 7fb9b161d640 1 -- 172.21.15.26:0/3740424794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596586d56b0 con 0x559658991f50 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b161d640 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x55965898fef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8020000 tx=0x7fb9a8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f40e2473c8c60e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9967fc640 1 -- 172.21.15.26:0/3740424794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8037070 con 0x559658991f50 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9967fc640 1 -- 172.21.15.26:0/3740424794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a801b070 con 0x559658991f50 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9967fc640 1 -- 172.21.15.26:0/3740424794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8025040 con 0x559658991f50 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 -- 172.21.15.26:0/3740424794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 msgr2=0x55965898fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x55965898fef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8020000 tx=0x7fb9a8014000 comp rx=0 tx=0).stop 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 -- 172.21.15.26:0/3740424794 shutdown_connections 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x5596589b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x55965898d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 --2- 172.21.15.26:0/3740424794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x55965898fef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 -- 172.21.15.26:0/3740424794 >> 172.21.15.26:0/3740424794 conn(0x5596589904b0 msgr2=0x5596589d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 -- 172.21.15.26:0/3740424794 shutdown_connections 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.353+0000 7fb9b6467740 1 -- 172.21.15.26:0/3740424794 wait complete. 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.354+0000 7fb9b6467740 1 -- start start 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.354+0000 7fb9b6467740 1 Processor -- start 2024-03-06T07:21:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x5596589b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x559658a3e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x559658a3e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596589e2450 con 0x559658991f50 2024-03-06T07:21:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596589ce410 con 0x5596589b5f10 2024-03-06T07:21:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596589b46a0 con 0x559658992330 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 10 client.0 _ll_register_callbacks cb 0x7fb9a40585a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x5596589b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x5596589b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47294/0 (socket says 172.21.15.26:47294) 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 -- 172.21.15.26:0/654987338 learned_addr learned my addr 172.21.15.26:0/654987338 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b1e1e640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x559658a3e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x559658a3e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 -- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 msgr2=0x559658a3e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x559658a3e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 -- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 msgr2=0x559658a3e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x559658a3e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 -- 172.21.15.26:0/654987338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596589b08d0 con 0x559658991f50 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x559658a3e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b161d640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x5596589b5860 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8014fd0 tx=0x7fb9a8001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe9f634f29eacb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb98bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb98bfff640 1 -- 172.21.15.26:0/654987338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8042060 con 0x559658991f50 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb98bfff640 1 -- 172.21.15.26:0/654987338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb9a80328b0 con 0x559658991f50 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.355+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a801b070 con 0x559658991f50 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb98bfff640 1 -- 172.21.15.26:0/654987338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a8025030 con 0x559658991f50 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 10 client.5478 fetch_fsmap learned FSMap version 21 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x559658a3eee0 con 0x559658991f50 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb98bfff640 1 -- 172.21.15.26:0/654987338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a801b070 con 0x559658991f50 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb98bfff640 1 -- 172.21.15.26:0/654987338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fb9a803a030 con 0x559658991f50 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 10 client.5478 fetch_fsmap finished waiting for FSMap version 21 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 10 client.5478 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 20 client.5478 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x559658a3b940 0x5596589daf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 4 client.5478 mds_command: new command op to 4910 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 --> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5596588fdf40 con 0x559658a3b940 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x559658a3b940 0x5596589daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb9b0e1c640 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x559658a3b940 0x5596589daf00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145448e901aca4db server_cookie=ad56dab88d0a61d3 in_seq=0 out_seq=0 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.356+0000 7fb98bfff640 10 client.5478 ms_handle_connect on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb98bfff640 1 -- 172.21.15.26:0/654987338 <== mds.0 v2:172.21.15.117:6835/2730813301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5596588fdf40 con 0x559658a3b940 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb98bfff640 10 client.5478 handle_command_reply: tid=0 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 client.5478 shutdown 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 msgr2=0x5596589b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x5596589b5860 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8014fd0 tx=0x7fb9a8001150 comp rx=0 tx=0).stop 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 shutdown_connections 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x559658a3b940 0x5596589daf00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596589b5f10 0x559658a3e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559658992330 0x559658a3e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb98bfff640 0 client.5478 ms_handle_reset on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 --2- 172.21.15.26:0/654987338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559658991f50 0x5596589b5860 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 >> 172.21.15.26:0/654987338 conn(0x5596589904b0 msgr2=0x559658987e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 shutdown_connections 2024-03-06T07:21:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 1 -- 172.21.15.26:0/654987338 wait complete. 2024-03-06T07:21:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b6467740 20 client.5478 trim_cache size 0 max 16384 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4910, 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.2458989230000004, 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-03-06T07:21:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 38, 2024-03-06T07:21:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 38, 2024-03-06T07:21:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 25.896784803999999 2024-03-06T07:21:19.369 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb99006b500 msgr2=0x7fb99006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb99006b500 0x7fb99006d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80011e0 tx=0x7fb9a8001670 comp rx=0 tx=0).stop 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac152720 msgr2=0x7fb9ac1ee4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac152720 0x7fb9ac1ee4e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4002230 tx=0x7fb9a400cb90 comp rx=0 tx=0).stop 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 shutdown_connections 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb99006b500 0x7fb99006d9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac152720 0x7fb9ac1ee4e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac072cd0 0x7fb9ac1edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 --2- 172.21.15.26:0/4241062700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9ac06f550 0x7fb9ac1ebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.357+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 >> 172.21.15.26:0/4241062700 conn(0x7fb9ac06ea40 msgr2=0x7fb9ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.358+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 shutdown_connections 2024-03-06T07:21:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.358+0000 7fb9b3881640 1 -- 172.21.15.26:0/4241062700 wait complete. 2024-03-06T07:21:19.371 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 21, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:00.359941+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 21, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4910": { "addr": "172.21.15.117:6837/2730813301", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 2730813301, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2730813301, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4910, "incarnation": 18, "join_fscid": 2, "name": "d", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:17.117874+0000", "qdb_cluster": [ 4910 ], "qdb_leader": 4910, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4910 } } } ], "standbys": [ { "addr": "172.21.15.26:6833/3831184543", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3831184543, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3831184543, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5202, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2694023942", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2694023942, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2694023942, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5205, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6836/3245601481", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3245601481, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3245601481, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5398, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:19.373 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 2, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:21:19.373 INFO:tasks.cephfs.test_failover:current = [{'gid': 5202, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/3831184543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3831184543}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3831184543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}, {'gid': 5398, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3245601481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3245601481}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3245601481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 19}, {'gid': 5205, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2694023942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2694023942}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2694023942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 4910, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:21:19.373 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:21:19.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:21:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.474+0000 7ff54a113640 1 Processor -- start 2024-03-06T07:21:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.474+0000 7ff54a113640 1 -- start start 2024-03-06T07:21:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.474+0000 7ff54a113640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5441527f0 0x7ff5440c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.474+0000 7ff54a113640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5440c88f0 0x7ff544155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.474+0000 7ff54a113640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff5440c58b0 0x7ff544155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff54a113640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5440c9130 con 0x7ff5441527f0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff54a113640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff544159260 con 0x7ff5440c58b0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff54a113640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440c6b90 con 0x7ff5440c88f0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5441527f0 0x7ff5440c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5441527f0 0x7ff5440c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47296/0 (socket says 172.21.15.26:47296) 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 -- 172.21.15.26:0/4185047830 learned_addr learned my addr 172.21.15.26:0/4185047830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff543fff640 1 --2- 172.21.15.26:0/4185047830 >> v2:172.21.15.117:3300/0 conn(0x7ff5440c58b0 0x7ff544155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff542ffd640 1 --2- 172.21.15.26:0/4185047830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5440c88f0 0x7ff544155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 -- 172.21.15.26:0/4185047830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5440c88f0 msgr2=0x7ff544155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 --2- 172.21.15.26:0/4185047830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5440c88f0 0x7ff544155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 -- 172.21.15.26:0/4185047830 >> v2:172.21.15.117:3300/0 conn(0x7ff5440c58b0 msgr2=0x7ff544155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 --2- 172.21.15.26:0/4185047830 >> v2:172.21.15.117:3300/0 conn(0x7ff5440c58b0 0x7ff544155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.475+0000 7ff5437fe640 1 -- 172.21.15.26:0/4185047830 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5441596f0 con 0x7ff5441527f0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff5437fe640 1 --2- 172.21.15.26:0/4185047830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5441527f0 0x7ff5440c5370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff54408cb10 tx=0x7ff53802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a741cc63049e765f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff5427fc640 1 -- 172.21.15.26:0/4185047830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538038070 con 0x7ff5441527f0 2024-03-06T07:21:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff5427fc640 1 -- 172.21.15.26:0/4185047830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538031740 con 0x7ff5441527f0 2024-03-06T07:21:19.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff5427fc640 1 -- 172.21.15.26:0/4185047830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53802f050 con 0x7ff5441527f0 2024-03-06T07:21:19.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 -- 172.21.15.26:0/4185047830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5441527f0 msgr2=0x7ff5440c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 --2- 172.21.15.26:0/4185047830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5441527f0 0x7ff5440c5370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff54408cb10 tx=0x7ff53802b7e0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 -- 172.21.15.26:0/4185047830 shutdown_connections 2024-03-06T07:21:19.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 --2- 172.21.15.26:0/4185047830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5440c88f0 0x7ff544155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 --2- 172.21.15.26:0/4185047830 >> v2:172.21.15.117:3300/0 conn(0x7ff5440c58b0 0x7ff544155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 --2- 172.21.15.26:0/4185047830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5441527f0 0x7ff5440c5370 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 -- 172.21.15.26:0/4185047830 >> 172.21.15.26:0/4185047830 conn(0x7ff5440ac580 msgr2=0x7ff5440ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 -- 172.21.15.26:0/4185047830 shutdown_connections 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.476+0000 7ff54a113640 1 -- 172.21.15.26:0/4185047830 wait complete. 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.477+0000 7ff54a113640 1 Processor -- start 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff54a113640 1 -- start start 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff54a113640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5440c58b0 0x7ff5440ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff54a113640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5440c88f0 0x7ff544153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff54a113640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5441527f0 0x7ff544153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff54a113640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff544158c10 con 0x7ff5440c88f0 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff54a113640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff544158930 con 0x7ff5440c58b0 2024-03-06T07:21:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff54a113640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff544158ef0 con 0x7ff5441527f0 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5440c88f0 0x7ff544153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5440c88f0 0x7ff544153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47312/0 (socket says 172.21.15.26:47312) 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.478+0000 7ff542ffd640 1 -- 172.21.15.26:0/3553430063 learned_addr learned my addr 172.21.15.26:0/3553430063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff5437fe640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5440c58b0 0x7ff5440ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff543fff640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5441527f0 0x7ff544153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff542ffd640 1 -- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5441527f0 msgr2=0x7ff544153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff542ffd640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5441527f0 0x7ff544153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff542ffd640 1 -- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5440c58b0 msgr2=0x7ff5440ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff542ffd640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5440c58b0 0x7ff5440ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff542ffd640 1 -- 172.21.15.26:0/3553430063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440be2b0 con 0x7ff5440c88f0 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff543fff640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5441527f0 0x7ff544153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff542ffd640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5440c88f0 0x7ff544153770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff53400e870 tx=0x7ff53400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff52bfff640 1 -- 172.21.15.26:0/3553430063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534017070 con 0x7ff5440c88f0 2024-03-06T07:21:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff52bfff640 1 -- 172.21.15.26:0/3553430063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53400c940 con 0x7ff5440c88f0 2024-03-06T07:21:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5441edf00 con 0x7ff5440c88f0 2024-03-06T07:21:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff52bfff640 1 -- 172.21.15.26:0/3553430063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53400cd30 con 0x7ff5440c88f0 2024-03-06T07:21:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.479+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5441ee4c0 con 0x7ff5440c88f0 2024-03-06T07:21:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.480+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54404f330 con 0x7ff5440c88f0 2024-03-06T07:21:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.480+0000 7ff52bfff640 1 -- 172.21.15.26:0/3553430063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff53401a070 con 0x7ff5440c88f0 2024-03-06T07:21:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.481+0000 7ff52bfff640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff53006b4d0 0x7ff53006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.481+0000 7ff52bfff640 1 -- 172.21.15.26:0/3553430063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff53408a4c0 con 0x7ff5440c88f0 2024-03-06T07:21:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.481+0000 7ff5437fe640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff53006b4d0 0x7ff53006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.481+0000 7ff5437fe640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff53006b4d0 0x7ff53006d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff538035000 tx=0x7ff538008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.485+0000 7ff52bfff640 1 -- 172.21.15.26:0/3553430063 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff534058ba0 con 0x7ff5440c88f0 2024-03-06T07:21:19.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.668+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7ff5440c5410 con 0x7ff5440c88f0 2024-03-06T07:21:19.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.671+0000 7ff52bfff640 1 -- 172.21.15.26:0/3553430063 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v13) v1 ==== 96+0+0 (secure 0 0 0) 0x7ff53405cc40 con 0x7ff5440c88f0 2024-03-06T07:21:19.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff53006b4d0 msgr2=0x7ff53006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff53006b4d0 0x7ff53006d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff538035000 tx=0x7ff538008030 comp rx=0 tx=0).stop 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5440c88f0 msgr2=0x7ff544153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5440c88f0 0x7ff544153770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff53400e870 tx=0x7ff53400ed30 comp rx=0 tx=0).stop 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 shutdown_connections 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff53006b4d0 0x7ff53006d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5441527f0 0x7ff544153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5440c88f0 0x7ff544153770 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 --2- 172.21.15.26:0/3553430063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5440c58b0 0x7ff5440ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 >> 172.21.15.26:0/3553430063 conn(0x7ff5440ac580 msgr2=0x7ff5441517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 shutdown_connections 2024-03-06T07:21:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.673+0000 7ff54a113640 1 -- 172.21.15.26:0/3553430063 wait complete. 2024-03-06T07:21:19.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.785+0000 7f4834650640 1 Processor -- start 2024-03-06T07:21:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.785+0000 7f4834650640 1 -- start start 2024-03-06T07:21:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4834650640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f482c0c88c0 0x7f482c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4834650640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f482c158b60 0x7f482c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4834650640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0c73d0 con 0x7f482c14f360 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4834650640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0c8dd0 con 0x7f482c0c88c0 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4834650640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0be2e0 con 0x7f482c158b60 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4831beb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f482c0c88c0 0x7f482c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4831beb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f482c0c88c0 0x7f482c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47324/0 (socket says 172.21.15.26:47324) 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.786+0000 7f4831beb640 1 -- 172.21.15.26:0/2555003817 learned_addr learned my addr 172.21.15.26:0/2555003817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.787+0000 7f4832bed640 1 --2- 172.21.15.26:0/2555003817 >> v2:172.21.15.117:3300/0 conn(0x7f482c158b60 0x7f482c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.787+0000 7f4831beb640 1 -- 172.21.15.26:0/2555003817 >> v2:172.21.15.117:3300/0 conn(0x7f482c158b60 msgr2=0x7f482c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.787+0000 7f4831beb640 1 --2- 172.21.15.26:0/2555003817 >> v2:172.21.15.117:3300/0 conn(0x7f482c158b60 0x7f482c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.787+0000 7f4831beb640 1 -- 172.21.15.26:0/2555003817 >> v1:172.21.15.117:6789/0 conn(0x7f482c14f360 legacy=0x7f482c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.787+0000 7f4831beb640 1 -- 172.21.15.26:0/2555003817 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0c7670 con 0x7f482c0c88c0 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4831beb640 1 --2- 172.21.15.26:0/2555003817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c0c88c0 0x7f482c155c10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f482c152d60 tx=0x7f481c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19fe3e509d9c8c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f48313ea640 1 -- 172.21.15.26:0/2555003817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c035020 con 0x7f482c0c88c0 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f48313ea640 1 -- 172.21.15.26:0/2555003817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c031870 con 0x7f482c0c88c0 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f48313ea640 1 -- 172.21.15.26:0/2555003817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c031c40 con 0x7f482c0c88c0 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 -- 172.21.15.26:0/2555003817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c0c88c0 msgr2=0x7f482c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 --2- 172.21.15.26:0/2555003817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c0c88c0 0x7f482c155c10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f482c152d60 tx=0x7f481c02b910 comp rx=0 tx=0).stop 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 -- 172.21.15.26:0/2555003817 shutdown_connections 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 --2- 172.21.15.26:0/2555003817 >> v2:172.21.15.117:3300/0 conn(0x7f482c158b60 0x7f482c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 --2- 172.21.15.26:0/2555003817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c0c88c0 0x7f482c155c10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 -- 172.21.15.26:0/2555003817 >> 172.21.15.26:0/2555003817 conn(0x7f482c0ac570 msgr2=0x7f482c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 -- 172.21.15.26:0/2555003817 shutdown_connections 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.788+0000 7f4834650640 1 -- 172.21.15.26:0/2555003817 wait complete. 2024-03-06T07:21:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 Processor -- start 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 -- start start 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f482c0c88c0 0x7f482c0c0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c14f360 0x7f482c0bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482c158b60 0x7f482c0bf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0c5a50 con 0x7f482c14f360 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0c7c30 con 0x7f482c158b60 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4834650640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0c5bd0 con 0x7f482c0c88c0 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4831beb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c14f360 0x7f482c0bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4831beb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c14f360 0x7f482c0bf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47330/0 (socket says 172.21.15.26:47330) 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4831beb640 1 -- 172.21.15.26:0/2189885470 learned_addr learned my addr 172.21.15.26:0/2189885470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f48323ec640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f482c0c88c0 0x7f482c0c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.790+0000 7f4832bed640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482c158b60 0x7f482c0bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4831beb640 1 -- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f482c0c88c0 msgr2=0x7f482c0c0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4831beb640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f482c0c88c0 0x7f482c0c0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4831beb640 1 -- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482c158b60 msgr2=0x7f482c0bf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4831beb640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482c158b60 0x7f482c0bf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4831beb640 1 -- 172.21.15.26:0/2189885470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0be620 con 0x7f482c14f360 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4831beb640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c14f360 0x7f482c0bf3e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f481c02b950 tx=0x7f481c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4822ffd640 1 -- 172.21.15.26:0/2189885470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c040070 con 0x7f482c14f360 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4822ffd640 1 -- 172.21.15.26:0/2189885470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c030d00 con 0x7f482c14f360 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c0be900 con 0x7f482c14f360 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4822ffd640 1 -- 172.21.15.26:0/2189885470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c035040 con 0x7f482c14f360 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.791+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c0beec0 con 0x7f482c14f360 2024-03-06T07:21:19.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.792+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c04f330 con 0x7f482c14f360 2024-03-06T07:21:19.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.792+0000 7f4822ffd640 1 -- 172.21.15.26:0/2189885470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f481c04b060 con 0x7f482c14f360 2024-03-06T07:21:19.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.793+0000 7f4822ffd640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f481006b4d0 0x7f481006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.793+0000 7f4822ffd640 1 -- 172.21.15.26:0/2189885470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f481c0abea0 con 0x7f482c14f360 2024-03-06T07:21:19.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.793+0000 7f48323ec640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f481006b4d0 0x7f481006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.794+0000 7f48323ec640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f481006b4d0 0x7f481006d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f482c0c8280 tx=0x7f4828009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:19.797+0000 7f4822ffd640 1 -- 172.21.15.26:0/2189885470 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f481c07a580 con 0x7f482c14f360 2024-03-06T07:21:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.019+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f482c14e9c0 con 0x7f482c14f360 2024-03-06T07:21:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.019+0000 7f4822ffd640 1 -- 172.21.15.26:0/2189885470 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f481c07e620 con 0x7f482c14f360 2024-03-06T07:21:20.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:20.022 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:17.117874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":2,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4910,"qdb_cluster":[4910]},"id":2}]} 2024-03-06T07:21:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-03-06T07:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f481006b4d0 msgr2=0x7f481006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f481006b4d0 0x7f481006d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f482c0c8280 tx=0x7f4828009040 comp rx=0 tx=0).stop 2024-03-06T07:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c14f360 msgr2=0x7f482c0bf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c14f360 0x7f482c0bf3e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f481c02b950 tx=0x7f481c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 shutdown_connections 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f481006b4d0 0x7f481006d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482c158b60 0x7f482c0bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482c14f360 0x7f482c0bf3e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 --2- 172.21.15.26:0/2189885470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f482c0c88c0 0x7f482c0c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 >> 172.21.15.26:0/2189885470 conn(0x7f482c0ac570 msgr2=0x7f482c14cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 shutdown_connections 2024-03-06T07:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.022+0000 7f4834650640 1 -- 172.21.15.26:0/2189885470 wait complete. 2024-03-06T07:21:20.317 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:21:20.315+0000 7f14111e5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:20.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:00.359941+0000', 'modified': '2024-03-06T07:21:17.117874+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 38, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4910': {'gid': 4910, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4910, 'qdb_cluster': [4910]} 2024-03-06T07:21:20.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:20.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:21:20.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.542+0000 7f7f2680e640 1 Processor -- start 2024-03-06T07:21:20.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.542+0000 7f7f2680e640 1 -- start start 2024-03-06T07:21:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.542+0000 7f7f2680e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7f2006c240 0x7f7f200c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.542+0000 7f7f2680e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f200ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.542+0000 7f7f2680e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f200abae0 con 0x7f7f200ab690 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.542+0000 7f7f2680e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f201553c0 con 0x7f7f2006c240 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.542+0000 7f7f2680e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201556a0 con 0x7f7f200ac2b0 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f200ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7f2006c240 0x7f7f200c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f24dab640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f7f200ab690 0x7f7f201588a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:59532/0 (socket says 172.21.15.26:59532) 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7f2006c240 0x7f7f200c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:50660/0 (socket says 172.21.15.26:50660) 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f200ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:38596/0 (socket says 172.21.15.26:38596) 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f24dab640 1 -- 172.21.15.26:0/2596476523 learned_addr learned my addr 172.21.15.26:0/2596476523 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1ffff640 1 -- 172.21.15.26:0/2596476523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 msgr2=0x7f7f200ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/2596476523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f200ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1ffff640 1 -- 172.21.15.26:0/2596476523 >> v1:172.21.15.117:6789/0 conn(0x7f7f200ab690 legacy=0x7f7f201588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1ffff640 1 -- 172.21.15.26:0/2596476523 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f20155980 con 0x7f7f2006c240 2024-03-06T07:21:20.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.543+0000 7f7f1effd640 1 -- 172.21.15.26:0/2596476523 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509246269 0 0) 0x7f7f200abae0 con 0x7f7f200ab690 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/2596476523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2006c240 0x7f7f200c4250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f100097a0 tx=0x7f7f1002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02bb529ca599e0a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f1effd640 1 -- 172.21.15.26:0/2596476523 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100315a0 con 0x7f7f2006c240 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f1effd640 1 -- 172.21.15.26:0/2596476523 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10037070 con 0x7f7f2006c240 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f1effd640 1 -- 172.21.15.26:0/2596476523 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10031960 con 0x7f7f2006c240 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 -- 172.21.15.26:0/2596476523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2006c240 msgr2=0x7f7f200c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 --2- 172.21.15.26:0/2596476523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2006c240 0x7f7f200c4250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f100097a0 tx=0x7f7f1002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 -- 172.21.15.26:0/2596476523 shutdown_connections 2024-03-06T07:21:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 --2- 172.21.15.26:0/2596476523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f200ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 --2- 172.21.15.26:0/2596476523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2006c240 0x7f7f200c4250 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 -- 172.21.15.26:0/2596476523 >> 172.21.15.26:0/2596476523 conn(0x7f7f200c7670 msgr2=0x7f7f200c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 -- 172.21.15.26:0/2596476523 shutdown_connections 2024-03-06T07:21:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.544+0000 7f7f2680e640 1 -- 172.21.15.26:0/2596476523 wait complete. 2024-03-06T07:21:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.545+0000 7f7f2680e640 1 Processor -- start 2024-03-06T07:21:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f2680e640 1 -- start start 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f2680e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2006c240 0x7f7f20159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f2680e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f200ab690 0x7f7f201576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f2680e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f20151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f2680e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f20156180 con 0x7f7f2006c240 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f2680e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f200c1d90 con 0x7f7f200ab690 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f2680e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f200c4830 con 0x7f7f200ac2b0 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2006c240 0x7f7f20159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f1f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f200ab690 0x7f7f201576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2006c240 0x7f7f20159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47340/0 (socket says 172.21.15.26:47340) 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3673986565 learned_addr learned my addr 172.21.15.26:0/3673986565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.546+0000 7f7f24dab640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f20151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1f7fe640 1 -- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 msgr2=0x7f7f20151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f20151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1f7fe640 1 -- 172.21.15.26:0/3673986565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2006c240 msgr2=0x7f7f20159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2006c240 0x7f7f20159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1f7fe640 1 -- 172.21.15.26:0/3673986565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f201566a0 con 0x7f7f200ab690 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f200ab690 0x7f7f201576d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e870 tx=0x7f7f1400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1cff9640 1 -- 172.21.15.26:0/3673986565 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14017070 con 0x7f7f200ab690 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1cff9640 1 -- 172.21.15.26:0/3673986565 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1400ee60 con 0x7f7f200ab690 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f200c32e0 con 0x7f7f200ab690 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f1cff9640 1 -- 172.21.15.26:0/3673986565 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400cb60 con 0x7f7f200ab690 2024-03-06T07:21:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.547+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f200c35c0 con 0x7f7f200ab690 2024-03-06T07:21:20.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.549+0000 7f7f1cff9640 1 -- 172.21.15.26:0/3673986565 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f1401a070 con 0x7f7f200ab690 2024-03-06T07:21:20.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.549+0000 7f7f1cff9640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7f0c06b4d0 0x7f7f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.549+0000 7f7f1cff9640 1 -- 172.21.15.26:0/3673986565 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7f1408a1a0 con 0x7f7f200ab690 2024-03-06T07:21:20.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.549+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7f0c06b4d0 0x7f7f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.550+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7f0c06b4d0 0x7f7f0c06d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f1002b880 tx=0x7f7f10008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.550+0000 7f7f293f4740 1 Processor -- start 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.550+0000 7f7f293f4740 1 -- start start 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8411f50 0x55f1b840fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b840ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b84104b0 0x55f1b840c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f293f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b83766b0 con 0x55f1b84104b0 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b831fa60 con 0x55f1b8412330 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1b8372750 con 0x55f1b8411f50 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b84104b0 0x55f1b840c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b84104b0 0x55f1b840c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47350/0 (socket says 172.21.15.26:47350) 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 -- 172.21.15.26:0/1455790029 learned_addr learned my addr 172.21.15.26:0/1455790029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8411f50 0x55f1b840fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b840ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 -- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8411f50 msgr2=0x55f1b840fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8411f50 0x55f1b840fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 -- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 msgr2=0x55f1b840ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b840ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 -- 172.21.15.26:0/1455790029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1b833df30 con 0x55f1b84104b0 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f24dab640 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b84104b0 0x55f1b840c680 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001ff0 tx=0x7f7f18007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ea4c8beee9847e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.551+0000 7f7f05ffb640 1 -- 172.21.15.26:0/1455790029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f180133f0 con 0x55f1b84104b0 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f05ffb640 1 -- 172.21.15.26:0/1455790029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f18018070 con 0x55f1b84104b0 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f05ffb640 1 -- 172.21.15.26:0/1455790029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18013790 con 0x55f1b84104b0 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 -- 172.21.15.26:0/1455790029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b84104b0 msgr2=0x55f1b840c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b84104b0 0x55f1b840c680 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001ff0 tx=0x7f7f18007410 comp rx=0 tx=0).stop 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 -- 172.21.15.26:0/1455790029 shutdown_connections 2024-03-06T07:21:20.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b84104b0 0x55f1b840c680 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b840ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1455790029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8411f50 0x55f1b840fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 -- 172.21.15.26:0/1455790029 >> 172.21.15.26:0/1455790029 conn(0x55f1b840e7d0 msgr2=0x55f1b84333c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 -- 172.21.15.26:0/1455790029 shutdown_connections 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.552+0000 7f7f293f4740 1 -- 172.21.15.26:0/1455790029 wait complete. 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.553+0000 7f7f293f4740 1 -- start start 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.553+0000 7f7f293f4740 1 Processor -- start 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.553+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8434600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b8451a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b844aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b8434230 con 0x55f1b8411f50 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b8332ea0 con 0x55f1b8435c70 2024-03-06T07:21:20.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1b8433c70 con 0x55f1b8412330 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 10 client.0 _ll_register_callbacks cb 0x55f1b8332870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8434600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8434600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47352/0 (socket says 172.21.15.26:47352) 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3434262780 learned_addr learned my addr 172.21.15.26:0/3434262780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b8451a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f24dab640 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b844aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 msgr2=0x55f1b8451a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b8451a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 msgr2=0x55f1b844aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b844aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3434262780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1b8333270 con 0x55f1b8411f50 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8434600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7f10003890 tx=0x7f7f1002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98137dc36effcc8c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7efeffd640 1 -- 172.21.15.26:0/3434262780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10031cb0 con 0x55f1b8411f50 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7efeffd640 1 -- 172.21.15.26:0/3434262780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7f10031e50 con 0x55f1b8411f50 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f18018070 con 0x55f1b8411f50 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7efeffd640 1 -- 172.21.15.26:0/3434262780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f10032430 con 0x55f1b8411f50 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 10 client.5499 fetch_fsmap learned FSMap version 21 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.554+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55f1b808b2d0 con 0x55f1b8411f50 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7efeffd640 1 -- 172.21.15.26:0/3434262780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f18018070 con 0x55f1b8411f50 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7efeffd640 1 -- 172.21.15.26:0/3434262780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f7f1005c050 con 0x55f1b8411f50 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f293f4740 10 client.5499 fetch_fsmap finished waiting for FSMap version 21 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f293f4740 10 client.5499 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f293f4740 20 client.5499 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b845f140 0x55f1b846d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f293f4740 4 client.5499 mds_command: new command op to 4910 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 --> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1b8383bf0 con 0x55f1b845f140 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b845f140 0x55f1b846d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7efeffd640 10 client.5499 ms_handle_connect on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.555+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b845f140 0x55f1b846d5d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b11a2171915a18 server_cookie=594f70137751c258 in_seq=0 out_seq=0 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7efeffd640 1 -- 172.21.15.26:0/3434262780 <== mds.0 v2:172.21.15.117:6835/2730813301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f1b8383bf0 con 0x55f1b845f140 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7efeffd640 10 client.5499 handle_command_reply: tid=0 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 client.5499 shutdown 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 msgr2=0x55f1b8434600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8434600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7f10003890 tx=0x7f7f1002e000 comp rx=0 tx=0).stop 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 shutdown_connections 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b845f140 0x55f1b846d5d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b844aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b8451a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7efeffd640 0 client.5499 ms_handle_reset on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:20.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3434262780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8434600 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 >> 172.21.15.26:0/3434262780 conn(0x55f1b84104b0 msgr2=0x55f1b840c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 shutdown_connections 2024-03-06T07:21:20.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 1 -- 172.21.15.26:0/3434262780 wait complete. 2024-03-06T07:21:20.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.556+0000 7f7f293f4740 20 client.5499 trim_cache size 0 max 16384 2024-03-06T07:21:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.573+0000 7f7f293f4740 1 Processor -- start 2024-03-06T07:21:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.573+0000 7f7f293f4740 1 -- start start 2024-03-06T07:21:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.573+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b840b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.573+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b8436550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.573+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8435c70 0x55f1b8436a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f293f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201f6dd0 con 0x55f1b8411f50 2024-03-06T07:21:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201f61d0 con 0x55f1b8412330 2024-03-06T07:21:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201f67d0 con 0x55f1b8435c70 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b840b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b840b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47354/0 (socket says 172.21.15.26:47354) 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3878003299 learned_addr learned my addr 172.21.15.26:0/3878003299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b8436550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f24dab640 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8435c70 0x55f1b8436a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8435c70 msgr2=0x55f1b8436a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8435c70 0x55f1b8436a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 msgr2=0x55f1b8436550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b8436550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 -- 172.21.15.26:0/3878003299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1b8434970 con 0x55f1b8411f50 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b8436550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f24dab640 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8435c70 0x55f1b8436a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b840b510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f10003860 tx=0x7f7f10041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170301b1b0bcecb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f057fa640 1 -- 172.21.15.26:0/3878003299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10031cb0 con 0x55f1b8411f50 2024-03-06T07:21:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f057fa640 1 -- 172.21.15.26:0/3878003299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10037040 con 0x55f1b8411f50 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.574+0000 7f7f057fa640 1 -- 172.21.15.26:0/3878003299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100327c0 con 0x55f1b8411f50 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 -- 172.21.15.26:0/3878003299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 msgr2=0x55f1b840b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b840b510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f10003860 tx=0x7f7f10041000 comp rx=0 tx=0).stop 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 -- 172.21.15.26:0/3878003299 shutdown_connections 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8435c70 0x55f1b8436a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8412330 0x55f1b8436550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 --2- 172.21.15.26:0/3878003299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b840b510 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 -- 172.21.15.26:0/3878003299 >> 172.21.15.26:0/3878003299 conn(0x55f1b84104b0 msgr2=0x55f1b8407ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 -- 172.21.15.26:0/3878003299 shutdown_connections 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 -- 172.21.15.26:0/3878003299 wait complete. 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 -- start start 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.575+0000 7f7f293f4740 1 Processor -- start 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8436880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b84be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f293f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b84bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f293f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b8462690 con 0x55f1b8411f50 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b845efc0 con 0x55f1b8435c70 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f293f4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1b84360c0 con 0x55f1b8412330 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f293f4740 10 client.0 _ll_register_callbacks cb 0x7f7f10003770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8436880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8436880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47360/0 (socket says 172.21.15.26:47360) 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f1ffff640 1 -- 172.21.15.26:0/1890251507 learned_addr learned my addr 172.21.15.26:0/1890251507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b84be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.576+0000 7f7f24dab640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b84bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f1ffff640 1 -- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 msgr2=0x55f1b84be0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b84be0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f1ffff640 1 -- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 msgr2=0x55f1b84bb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b84bb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f1ffff640 1 -- 172.21.15.26:0/1890251507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1b81556b0 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b84be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f1ffff640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8436880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f10003ef0 tx=0x7f7f10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f043847200749d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7efeffd640 1 -- 172.21.15.26:0/1890251507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100329d0 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7efeffd640 1 -- 172.21.15.26:0/1890251507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7f10068ca0 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f10037040 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7efeffd640 1 -- 172.21.15.26:0/1890251507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f1006f050 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 10 client.5505 fetch_fsmap learned FSMap version 21 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55f1b84c2130 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7efeffd640 1 -- 172.21.15.26:0/1890251507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10037040 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7efeffd640 1 -- 172.21.15.26:0/1890251507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f7f10072020 con 0x55f1b8411f50 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 10 client.5505 fetch_fsmap finished waiting for FSMap version 21 2024-03-06T07:21:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 10 client.5505 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 20 client.5505 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b84bb810 0x55f1b8462f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 4 client.5505 mds_command: new command op to 4910 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.577+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 --> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f1b837df40 con 0x55f1b84bb810 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b84bb810 0x55f1b8462f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f1f7fe640 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b84bb810 0x55f1b8462f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ba7fe4e35ef356 server_cookie=c67eb07034ebac78 in_seq=0 out_seq=0 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7efeffd640 10 client.5505 ms_handle_connect on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7efeffd640 1 -- 172.21.15.26:0/1890251507 <== mds.0 v2:172.21.15.117:6835/2730813301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f1b837df40 con 0x55f1b84bb810 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7efeffd640 10 client.5505 handle_command_reply: tid=0 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 client.5505 shutdown 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 msgr2=0x55f1b8436880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8436880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f10003ef0 tx=0x7f7f10004730 comp rx=0 tx=0).stop 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 shutdown_connections 2024-03-06T07:21:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x55f1b84bb810 0x55f1b8462f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b8435c70 0x55f1b84bb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f1b8412330 0x55f1b84be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 --2- 172.21.15.26:0/1890251507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1b8411f50 0x55f1b8436880 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7efeffd640 0 client.5505 ms_handle_reset on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 >> 172.21.15.26:0/1890251507 conn(0x55f1b84104b0 msgr2=0x55f1b840b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 shutdown_connections 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 1 -- 172.21.15.26:0/1890251507 wait complete. 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.578+0000 7f7f293f4740 20 client.5505 trim_cache size 0 max 16384 2024-03-06T07:21:20.588 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4910, 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 6.4673861639999997, 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 38, 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 38, 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 27.118271977999999 2024-03-06T07:21:20.589 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:21:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7f0c06b4d0 msgr2=0x7f7f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7f0c06b4d0 0x7f7f0c06d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f1002b880 tx=0x7f7f10008030 comp rx=0 tx=0).stop 2024-03-06T07:21:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f200ab690 msgr2=0x7f7f201576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f200ab690 0x7f7f201576d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e870 tx=0x7f7f1400ed30 comp rx=0 tx=0).stop 2024-03-06T07:21:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 shutdown_connections 2024-03-06T07:21:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7f0c06b4d0 0x7f7f0c06d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f200ac2b0 0x7f7f20151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f200ab690 0x7f7f201576d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 --2- 172.21.15.26:0/3673986565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2006c240 0x7f7f20159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 >> 172.21.15.26:0/3673986565 conn(0x7f7f200c7670 msgr2=0x7f7f2014edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 shutdown_connections 2024-03-06T07:21:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:20.579+0000 7f7f2680e640 1 -- 172.21.15.26:0/3673986565 wait complete. 2024-03-06T07:21:20.592 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 21, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:00.359941+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 21, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4910": { "addr": "172.21.15.117:6837/2730813301", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 2730813301, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2730813301, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4910, "incarnation": 18, "join_fscid": 2, "name": "d", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:17.117874+0000", "qdb_cluster": [ 4910 ], "qdb_leader": 4910, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4910 } } } ], "standbys": [ { "addr": "172.21.15.26:6833/3831184543", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3831184543, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3831184543, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5202, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2694023942", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2694023942, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2694023942, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5205, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6836/3245601481", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3245601481, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3245601481, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5398, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:20.593 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:21:20.593 INFO:tasks.cephfs.test_failover:current = [{'gid': 5202, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/3831184543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3831184543}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3831184543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}, {'gid': 5398, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3245601481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3245601481}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3245601481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 19}, {'gid': 5205, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2694023942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2694023942}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2694023942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 4910, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:21:20.593 DEBUG:tasks.cephfs.test_failover:-1 != 2 2024-03-06T07:21:20.593 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:21:25.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.706+0000 7f2453c47640 1 Processor -- start 2024-03-06T07:21:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f2453c47640 1 -- start start 2024-03-06T07:21:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f2453c47640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f244c150580 0x7f244c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f2453c47640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c1320 0x7f244c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f2453c47640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c153a60 con 0x7f244c0c6000 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f2453c47640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c153d40 con 0x7f244c150580 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f2453c47640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c06c2c0 con 0x7f244c0c1320 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f24519e3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f244c150580 0x7f244c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f24519e3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f244c150580 0x7f244c06dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47368/0 (socket says 172.21.15.26:47368) 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.707+0000 7f24519e3640 1 -- 172.21.15.26:0/2647657498 learned_addr learned my addr 172.21.15.26:0/2647657498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.708+0000 7f24511e2640 1 --2- 172.21.15.26:0/2647657498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c1320 0x7f244c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.708+0000 7f24519e3640 1 -- 172.21.15.26:0/2647657498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c1320 msgr2=0x7f244c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.708+0000 7f24519e3640 1 --2- 172.21.15.26:0/2647657498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c1320 0x7f244c06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.708+0000 7f24519e3640 1 -- 172.21.15.26:0/2647657498 >> v1:172.21.15.117:6789/0 conn(0x7f244c0c6000 legacy=0x7f244c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.708+0000 7f24519e3640 1 -- 172.21.15.26:0/2647657498 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c06c5a0 con 0x7f244c150580 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f24519e3640 1 --2- 172.21.15.26:0/2647657498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c06dd10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2448009870 tx=0x7f244802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc3fd8ffb6a578f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f24509e1640 1 -- 172.21.15.26:0/2647657498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448035020 con 0x7f244c150580 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f24509e1640 1 -- 172.21.15.26:0/2647657498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2448031640 con 0x7f244c150580 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f24509e1640 1 -- 172.21.15.26:0/2647657498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244802d040 con 0x7f244c150580 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f2453c47640 1 -- 172.21.15.26:0/2647657498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 msgr2=0x7f244c06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f2453c47640 1 --2- 172.21.15.26:0/2647657498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c06dd10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2448009870 tx=0x7f244802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f2453c47640 1 -- 172.21.15.26:0/2647657498 shutdown_connections 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f2453c47640 1 --2- 172.21.15.26:0/2647657498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c1320 0x7f244c06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f2453c47640 1 --2- 172.21.15.26:0/2647657498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c06dd10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.709+0000 7f2453c47640 1 -- 172.21.15.26:0/2647657498 >> 172.21.15.26:0/2647657498 conn(0x7f244c0c6710 msgr2=0x7f244c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.710+0000 7f2453c47640 1 -- 172.21.15.26:0/2647657498 shutdown_connections 2024-03-06T07:21:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.710+0000 7f2453c47640 1 -- 172.21.15.26:0/2647657498 wait complete. 2024-03-06T07:21:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 Processor -- start 2024-03-06T07:21:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 -- start start 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244c0c1320 0x7f244c1ea240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c6000 0x7f244c1496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0c0dc0 con 0x7f244c150580 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0c0c40 con 0x7f244c0c1320 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f2453c47640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c156dc0 con 0x7f244c0c6000 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f24521e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f24521e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47372/0 (socket says 172.21.15.26:47372) 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f24521e4640 1 -- 172.21.15.26:0/2801937054 learned_addr learned my addr 172.21.15.26:0/2801937054 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.711+0000 7f24511e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c6000 0x7f244c1496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24519e3640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244c0c1320 0x7f244c1ea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24521e4640 1 -- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c6000 msgr2=0x7f244c1496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24521e4640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c6000 0x7f244c1496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24521e4640 1 -- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244c0c1320 msgr2=0x7f244c1ea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24521e4640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244c0c1320 0x7f244c1ea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24521e4640 1 -- 172.21.15.26:0/2801937054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c1eabd0 con 0x7f244c150580 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24521e4640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c148aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f244400c710 tx=0x7f244400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24427fc640 1 -- 172.21.15.26:0/2801937054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444016020 con 0x7f244c150580 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24427fc640 1 -- 172.21.15.26:0/2801937054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444006d50 con 0x7f244c150580 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c0a6b10 con 0x7f244c150580 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f24427fc640 1 -- 172.21.15.26:0/2801937054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444003bb0 con 0x7f244c150580 2024-03-06T07:21:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.712+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c0a63d0 con 0x7f244c150580 2024-03-06T07:21:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.713+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c04f330 con 0x7f244c150580 2024-03-06T07:21:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.714+0000 7f24427fc640 1 -- 172.21.15.26:0/2801937054 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f244400a850 con 0x7f244c150580 2024-03-06T07:21:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.714+0000 7f24427fc640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243006b4d0 0x7f243006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.714+0000 7f24427fc640 1 -- 172.21.15.26:0/2801937054 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2444089e70 con 0x7f244c150580 2024-03-06T07:21:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.714+0000 7f24519e3640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243006b4d0 0x7f243006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.715+0000 7f24519e3640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243006b4d0 0x7f243006d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2448036000 tx=0x7f244803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:25.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.719+0000 7f24427fc640 1 -- 172.21.15.26:0/2801937054 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2444020020 con 0x7f244c150580 2024-03-06T07:21:25.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.937+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f244c0c5de0 con 0x7f244c150580 2024-03-06T07:21:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.937+0000 7f24427fc640 1 -- 172.21.15.26:0/2801937054 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4985 (secure 0 0 0) 0x7f244400ce20 con 0x7f244c150580 2024-03-06T07:21:25.943 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:25.944 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:20.118951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4910,"qdb_cluster":[4910]},"id":2}]} 2024-03-06T07:21:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 22 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243006b4d0 msgr2=0x7f243006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243006b4d0 0x7f243006d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2448036000 tx=0x7f244803c040 comp rx=0 tx=0).stop 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 msgr2=0x7f244c148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c148aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f244400c710 tx=0x7f244400cbd0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 shutdown_connections 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243006b4d0 0x7f243006d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f244c150580 0x7f244c148aa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f244c0c6000 0x7f244c1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 --2- 172.21.15.26:0/2801937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244c0c1320 0x7f244c1ea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 >> 172.21.15.26:0/2801937054 conn(0x7f244c0c6710 msgr2=0x7f244c152690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 shutdown_connections 2024-03-06T07:21:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:25.939+0000 7f2453c47640 1 -- 172.21.15.26:0/2801937054 wait complete. 2024-03-06T07:21:26.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:00.359941+0000', 'modified': '2024-03-06T07:21:20.118951+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 38, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4910': {'gid': 4910, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4910, 'qdb_cluster': [4910]} 2024-03-06T07:21:26.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:26.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:21:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.458+0000 7f6676c81640 1 Processor -- start 2024-03-06T07:21:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.458+0000 7f6676c81640 1 -- start start 2024-03-06T07:21:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6676c81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f667006e700 0x7f667006eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6676c81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700a5f60 0x7f6670156770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6676c81640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6670156cb0 0x7f6670072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6676c81640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6670153280 con 0x7f667006e700 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6676c81640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6670153560 con 0x7f6670156cb0 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6676c81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6670153840 con 0x7f66700a5f60 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f667006e700 0x7f667006eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f667006e700 0x7f667006eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47388/0 (socket says 172.21.15.26:47388) 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 -- 172.21.15.26:0/3271880315 learned_addr learned my addr 172.21.15.26:0/3271880315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f666ffff640 1 --2- 172.21.15.26:0/3271880315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700a5f60 0x7f6670156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f667521e640 1 --2- 172.21.15.26:0/3271880315 >> v2:172.21.15.117:3300/0 conn(0x7f6670156cb0 0x7f6670072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 -- 172.21.15.26:0/3271880315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700a5f60 msgr2=0x7f6670156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 --2- 172.21.15.26:0/3271880315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700a5f60 0x7f6670156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 -- 172.21.15.26:0/3271880315 >> v2:172.21.15.117:3300/0 conn(0x7f6670156cb0 msgr2=0x7f6670072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 --2- 172.21.15.26:0/3271880315 >> v2:172.21.15.117:3300/0 conn(0x7f6670156cb0 0x7f6670072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f6674a1d640 1 -- 172.21.15.26:0/3271880315 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670153b20 con 0x7f667006e700 2024-03-06T07:21:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.459+0000 7f667521e640 1 --2- 172.21.15.26:0/3271880315 >> v2:172.21.15.117:3300/0 conn(0x7f6670156cb0 0x7f6670072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6674a1d640 1 --2- 172.21.15.26:0/3271880315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f667006e700 0x7f667006eae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f66700c34a0 tx=0x7f666402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02295493c66d7ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f666f7fe640 1 -- 172.21.15.26:0/3271880315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666402e070 con 0x7f667006e700 2024-03-06T07:21:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f666f7fe640 1 -- 172.21.15.26:0/3271880315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6664037070 con 0x7f667006e700 2024-03-06T07:21:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f666f7fe640 1 -- 172.21.15.26:0/3271880315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664031a00 con 0x7f667006e700 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6676c81640 1 -- 172.21.15.26:0/3271880315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f667006e700 msgr2=0x7f667006eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6676c81640 1 --2- 172.21.15.26:0/3271880315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f667006e700 0x7f667006eae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f66700c34a0 tx=0x7f666402b880 comp rx=0 tx=0).stop 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6676c81640 1 -- 172.21.15.26:0/3271880315 shutdown_connections 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6676c81640 1 --2- 172.21.15.26:0/3271880315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700a5f60 0x7f6670156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6676c81640 1 --2- 172.21.15.26:0/3271880315 >> v2:172.21.15.117:3300/0 conn(0x7f6670156cb0 0x7f6670072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6676c81640 1 --2- 172.21.15.26:0/3271880315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f667006e700 0x7f667006eae0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.460+0000 7f6676c81640 1 -- 172.21.15.26:0/3271880315 >> 172.21.15.26:0/3271880315 conn(0x7f66700c6af0 msgr2=0x7f66700c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.461+0000 7f6676c81640 1 -- 172.21.15.26:0/3271880315 shutdown_connections 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.461+0000 7f6676c81640 1 -- 172.21.15.26:0/3271880315 wait complete. 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 Processor -- start 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 -- start start 2024-03-06T07:21:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f667006e700 0x7f66700a7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66700a5f60 0x7f6670148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6670156cb0 0x7f6670149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6670152f30 con 0x7f66700a5f60 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66700c67e0 con 0x7f6670156cb0 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f6676c81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66700a63e0 con 0x7f667006e700 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.462+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66700a5f60 0x7f6670148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66700a5f60 0x7f6670148d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47398/0 (socket says 172.21.15.26:47398) 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 -- 172.21.15.26:0/3386024489 learned_addr learned my addr 172.21.15.26:0/3386024489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f6674a1d640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f667006e700 0x7f66700a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f667521e640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6670156cb0 0x7f6670149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 -- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f667006e700 msgr2=0x7f66700a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f667006e700 0x7f66700a7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 -- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6670156cb0 msgr2=0x7f6670149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6670156cb0 0x7f6670149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 -- 172.21.15.26:0/3386024489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66700a85a0 con 0x7f66700a5f60 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666ffff640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66700a5f60 0x7f6670148d00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f665c00e8c0 tx=0x7f665c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666d7fa640 1 -- 172.21.15.26:0/3386024489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c017070 con 0x7f66700a5f60 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666d7fa640 1 -- 172.21.15.26:0/3386024489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00c940 con 0x7f66700a5f60 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667014dfa0 con 0x7f66700a5f60 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f666d7fa640 1 -- 172.21.15.26:0/3386024489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00cd10 con 0x7f66700a5f60 2024-03-06T07:21:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.463+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667014e560 con 0x7f66700a5f60 2024-03-06T07:21:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.465+0000 7f666d7fa640 1 -- 172.21.15.26:0/3386024489 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f665c01a070 con 0x7f66700a5f60 2024-03-06T07:21:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.465+0000 7f666d7fa640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f665406b4d0 0x7f665406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.465+0000 7f666d7fa640 1 -- 172.21.15.26:0/3386024489 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f665c08a690 con 0x7f66700a5f60 2024-03-06T07:21:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.465+0000 7f6674a1d640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f665406b4d0 0x7f665406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.466+0000 7f6674a1d640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f665406b4d0 0x7f665406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6664036000 tx=0x7f666403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.466+0000 7f6679863740 1 Processor -- start 2024-03-06T07:21:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.466+0000 7f6679863740 1 -- start start 2024-03-06T07:21:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffabc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f6679863740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffabc5550 0x560ffac0ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabbf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f6679863740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ffab296b0 con 0x560ffabc5550 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ffaad2a60 con 0x560ffabff420 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ffab25750 con 0x560ffac0e800 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffabc5550 0x560ffac0ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f667521e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabbf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffabc5550 0x560ffac0ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47402/0 (socket says 172.21.15.26:47402) 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 -- 172.21.15.26:0/3027922383 learned_addr learned my addr 172.21.15.26:0/3027922383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f6674a1d640 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffabc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 -- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 msgr2=0x560ffabc5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffabc5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 -- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 msgr2=0x560ffabbf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabbf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 -- 172.21.15.26:0/3027922383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ffaaf0f30 con 0x560ffabc5550 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f667521e640 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabbf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.467+0000 7f666ffff640 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffabc5550 0x560ffac0ebe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f665c088940 tx=0x7f665c05f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2cc0f3bfae1a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f665a7fc640 1 -- 172.21.15.26:0/3027922383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c021050 con 0x560ffabc5550 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f665a7fc640 1 -- 172.21.15.26:0/3027922383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c05cd70 con 0x560ffabc5550 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f665a7fc640 1 -- 172.21.15.26:0/3027922383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c061330 con 0x560ffabc5550 2024-03-06T07:21:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 -- 172.21.15.26:0/3027922383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffabc5550 msgr2=0x560ffac0ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffabc5550 0x560ffac0ebe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f665c088940 tx=0x7f665c05f050 comp rx=0 tx=0).stop 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 -- 172.21.15.26:0/3027922383 shutdown_connections 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabbf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffabc5550 0x560ffac0ebe0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 --2- 172.21.15.26:0/3027922383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffabc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 -- 172.21.15.26:0/3027922383 >> 172.21.15.26:0/3027922383 conn(0x560ffabc17d0 msgr2=0x560ffabe63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 -- 172.21.15.26:0/3027922383 shutdown_connections 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.468+0000 7f6679863740 1 -- 172.21.15.26:0/3027922383 wait complete. 2024-03-06T07:21:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.469+0000 7f6679863740 1 -- start start 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.469+0000 7f6679863740 1 Processor -- start 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabc0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffac15c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6679863740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac16190 0x560ffabe2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6679863740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ffac03590 con 0x560ffac16190 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ffaae5ea0 con 0x560ffabff420 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ffabe6db0 con 0x560ffac0e800 2024-03-06T07:21:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6679863740 10 client.0 _ll_register_callbacks cb 0x560ffaae5870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac16190 0x560ffabe2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f666ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffac15c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac16190 0x560ffabe2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47406/0 (socket says 172.21.15.26:47406) 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 -- 172.21.15.26:0/450527664 learned_addr learned my addr 172.21.15.26:0/450527664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f666ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffac15c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:38690/0 (socket says 172.21.15.26:38690) 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f6674a1d640 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabc0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 -- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 msgr2=0x560ffac15c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffac15c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 -- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 msgr2=0x560ffabc0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabc0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f667521e640 1 -- 172.21.15.26:0/450527664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ffaae6270 con 0x560ffac16190 2024-03-06T07:21:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.470+0000 7f666ffff640 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffac15c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f667521e640 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac16190 0x560ffabe2a80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6668004660 tx=0x7f6668007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9a0a0861acb3c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f664f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f664f7fe640 1 -- 172.21.15.26:0/450527664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666801a070 con 0x560ffac16190 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f664f7fe640 1 -- 172.21.15.26:0/450527664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6668016630 con 0x560ffac16190 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f665c05cd70 con 0x560ffac16190 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f664f7fe640 1 -- 172.21.15.26:0/450527664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6668025050 con 0x560ffac16190 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 10 client.5523 fetch_fsmap learned FSMap version 22 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560ffa83e2d0 con 0x560ffac16190 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f664f7fe640 1 -- 172.21.15.26:0/450527664 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c05cd70 con 0x560ffac16190 2024-03-06T07:21:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f664f7fe640 1 -- 172.21.15.26:0/450527664 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 22) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6668028020 con 0x560ffac16190 2024-03-06T07:21:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 10 client.5523 fetch_fsmap finished waiting for FSMap version 22 2024-03-06T07:21:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 10 client.5523 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:21:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 20 client.5523 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac0a4f0 0x560ffac0a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 4 client.5523 mds_command: new command op to 4910 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 --> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ffab36bf0 con 0x560ffac0a4f0 2024-03-06T07:21:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.471+0000 7f6674a1d640 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac0a4f0 0x560ffac0a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.472+0000 7f664f7fe640 10 client.5523 ms_handle_connect on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.472+0000 7f6674a1d640 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac0a4f0 0x560ffac0a8d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4775de8df9278b8 server_cookie=a09ea5361c69158e in_seq=0 out_seq=0 2024-03-06T07:21:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.472+0000 7f664f7fe640 1 -- 172.21.15.26:0/450527664 <== mds.0 v2:172.21.15.117:6835/2730813301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560ffab36bf0 con 0x560ffac0a4f0 2024-03-06T07:21:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.472+0000 7f664f7fe640 10 client.5523 handle_command_reply: tid=0 2024-03-06T07:21:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.472+0000 7f6679863740 1 client.5523 shutdown 2024-03-06T07:21:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac16190 msgr2=0x560ffabe2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac16190 0x560ffabe2a80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6668004660 tx=0x7f6668007700 comp rx=0 tx=0).stop 2024-03-06T07:21:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 shutdown_connections 2024-03-06T07:21:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac0a4f0 0x560ffac0a8d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac16190 0x560ffabe2a80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac0e800 0x560ffac15c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 --2- 172.21.15.26:0/450527664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabc0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f664f7fe640 0 client.5523 ms_handle_reset on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 >> 172.21.15.26:0/450527664 conn(0x560ffabc5550 msgr2=0x560ffabe66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 shutdown_connections 2024-03-06T07:21:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 1 -- 172.21.15.26:0/450527664 wait complete. 2024-03-06T07:21:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.473+0000 7f6679863740 20 client.5523 trim_cache size 0 max 16384 2024-03-06T07:21:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 Processor -- start 2024-03-06T07:21:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 -- start start 2024-03-06T07:21:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabe6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffabffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66701f3900 con 0x560ffac0e800 2024-03-06T07:21:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667006e350 con 0x560ffabff420 2024-03-06T07:21:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667006d750 con 0x560ffac16190 2024-03-06T07:21:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffabffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffabffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47422/0 (socket says 172.21.15.26:47422) 2024-03-06T07:21:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f666ffff640 1 -- 172.21.15.26:0/1642011662 learned_addr learned my addr 172.21.15.26:0/1642011662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f667521e640 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac00080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.490+0000 7f6674a1d640 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabe6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f666ffff640 1 -- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 msgr2=0x560ffac00080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f666ffff640 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac00080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f666ffff640 1 -- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 msgr2=0x560ffabe6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f666ffff640 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabe6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f666ffff640 1 -- 172.21.15.26:0/1642011662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ffa9086b0 con 0x560ffac0e800 2024-03-06T07:21:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f666ffff640 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffabffb40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f665c08a010 tx=0x7f665c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57d0da554251e3a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6659ffb640 1 -- 172.21.15.26:0/1642011662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c054f70 con 0x560ffac0e800 2024-03-06T07:21:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6659ffb640 1 -- 172.21.15.26:0/1642011662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c061330 con 0x560ffac0e800 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6659ffb640 1 -- 172.21.15.26:0/1642011662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c06cbe0 con 0x560ffac0e800 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6679863740 1 -- 172.21.15.26:0/1642011662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 msgr2=0x560ffabffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6679863740 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffabffb40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f665c08a010 tx=0x7f665c014000 comp rx=0 tx=0).stop 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6679863740 1 -- 172.21.15.26:0/1642011662 shutdown_connections 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6679863740 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6679863740 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffabffb40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6679863740 1 --2- 172.21.15.26:0/1642011662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffabe6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.491+0000 7f6679863740 1 -- 172.21.15.26:0/1642011662 >> 172.21.15.26:0/1642011662 conn(0x560ffabc5550 msgr2=0x560ffabbe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.492+0000 7f6679863740 1 -- 172.21.15.26:0/1642011662 shutdown_connections 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.492+0000 7f6679863740 1 -- 172.21.15.26:0/1642011662 wait complete. 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.492+0000 7f6679863740 1 -- start start 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.492+0000 7f6679863740 1 Processor -- start 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffac29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6679863740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffac04670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6679863740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6679863740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ffac0a370 con 0x560ffac0e800 2024-03-06T07:21:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667006dd50 con 0x560ffabff420 2024-03-06T07:21:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6679863740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ffabc2ac0 con 0x560ffac16190 2024-03-06T07:21:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6679863740 10 client.0 _ll_register_callbacks cb 0x7f6668004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffac04670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffac04670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47426/0 (socket says 172.21.15.26:47426) 2024-03-06T07:21:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 -- 172.21.15.26:0/767636876 learned_addr learned my addr 172.21.15.26:0/767636876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f667521e640 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f6674a1d640 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffac29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 -- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 msgr2=0x560ffac72620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac72620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 -- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 msgr2=0x560ffac29080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffac29080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 -- 172.21.15.26:0/767636876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ffac0aa00 con 0x560ffac0e800 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f666ffff640 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffac04670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f665c08a010 tx=0x7f665c00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7194834a541ecfc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.493+0000 7f664f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f664f7fe640 1 -- 172.21.15.26:0/767636876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c021070 con 0x560ffac0e800 2024-03-06T07:21:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f664f7fe640 1 -- 172.21.15.26:0/767636876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f665c058320 con 0x560ffac0e800 2024-03-06T07:21:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f665c061330 con 0x560ffac0e800 2024-03-06T07:21:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f664f7fe640 1 -- 172.21.15.26:0/767636876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f665c054f70 con 0x560ffac0e800 2024-03-06T07:21:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 10 client.5529 fetch_fsmap learned FSMap version 22 2024-03-06T07:21:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560ffac0afc0 con 0x560ffac0e800 2024-03-06T07:21:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f664f7fe640 1 -- 172.21.15.26:0/767636876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c061330 con 0x560ffac0e800 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f664f7fe640 1 -- 172.21.15.26:0/767636876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 22) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f665c054a20 con 0x560ffac0e800 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 10 client.5529 fetch_fsmap finished waiting for FSMap version 22 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 10 client.5529 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 20 client.5529 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac72b60 0x560ffabffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 4 client.5529 mds_command: new command op to 4910 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 --> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560ffab30f40 con 0x560ffac72b60 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.494+0000 7f6674a1d640 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac72b60 0x560ffabffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f664f7fe640 10 client.5529 ms_handle_connect on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6674a1d640 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac72b60 0x560ffabffb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6b45da99c2bb31 server_cookie=1659083c04f36deb in_seq=0 out_seq=0 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f664f7fe640 1 -- 172.21.15.26:0/767636876 <== mds.0 v2:172.21.15.117:6835/2730813301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560ffab30f40 con 0x560ffac72b60 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f664f7fe640 10 client.5529 handle_command_reply: tid=0 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 client.5529 shutdown 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 msgr2=0x560ffac04670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffac04670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f665c08a010 tx=0x7f665c00efd0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 shutdown_connections 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:6835/2730813301,v1:172.21.15.117:6837/2730813301] conn(0x560ffac72b60 0x560ffabffb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x560ffac16190 0x560ffac72620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ffac0e800 0x560ffac04670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f664f7fe640 0 client.5529 ms_handle_reset on v2:172.21.15.117:6835/2730813301 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 --2- 172.21.15.26:0/767636876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ffabff420 0x560ffac29080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 >> 172.21.15.26:0/767636876 conn(0x560ffabc5550 msgr2=0x560ffac02560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 shutdown_connections 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 1 -- 172.21.15.26:0/767636876 wait complete. 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6679863740 20 client.5529 trim_cache size 0 max 16384 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:21:26.513 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4910, 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 12.384028948999999, 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 22, 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 38, 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 38, 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 33.034914909000001 2024-03-06T07:21:26.514 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:21:26.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f665406b4d0 msgr2=0x7f665406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6676c81640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f665406b4d0 0x7f665406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6664036000 tx=0x7f666403d040 comp rx=0 tx=0).stop 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66700a5f60 msgr2=0x7f6670148d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.495+0000 7f6676c81640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66700a5f60 0x7f6670148d00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f665c00e8c0 tx=0x7f665c00ed80 comp rx=0 tx=0).stop 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 shutdown_connections 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f665406b4d0 0x7f665406d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6670156cb0 0x7f6670149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66700a5f60 0x7f6670148d00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 --2- 172.21.15.26:0/3386024489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f667006e700 0x7f66700a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 >> 172.21.15.26:0/3386024489 conn(0x7f66700c6af0 msgr2=0x7f66700c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 shutdown_connections 2024-03-06T07:21:26.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.496+0000 7f6676c81640 1 -- 172.21.15.26:0/3386024489 wait complete. 2024-03-06T07:21:26.517 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:00.359941+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4910": { "addr": "172.21.15.117:6837/2730813301", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 2730813301, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2730813301, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4910, "incarnation": 18, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:20.118951+0000", "qdb_cluster": [ 4910 ], "qdb_leader": 4910, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4910 } } } ], "standbys": [ { "addr": "172.21.15.26:6833/3831184543", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3831184543, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3831184543, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5202, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2694023942", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2694023942, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2694023942, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5205, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6836/3245601481", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3245601481, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3245601481, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5398, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:26.518 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:21:26.518 INFO:tasks.cephfs.test_failover:current = [{'gid': 5202, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/3831184543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3831184543}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3831184543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}, {'gid': 5398, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3245601481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3245601481}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3245601481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 19}, {'gid': 5205, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2694023942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2694023942}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2694023942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 4910, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.117:6837/2730813301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 2730813301}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2730813301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:21:26.518 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:21:26.518 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:21:26.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:26.569 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:26.620 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:21:26.620 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:21:26.620 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:26.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:21:26.636 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:21:26.636 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:26.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:21:26.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.775+0000 7fe746015640 1 Processor -- start 2024-03-06T07:21:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.775+0000 7fe746015640 1 -- start start 2024-03-06T07:21:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe746015640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe740155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe746015640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7400c73d0 con 0x7fe74014f360 2024-03-06T07:21:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe746015640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7400be2e0 con 0x7fe7400c88c0 2024-03-06T07:21:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe746015640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe740159700 con 0x7fe740159270 2024-03-06T07:21:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe73f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe74014f360 0x7fe7400c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55106/0 (socket says 172.21.15.26:55106) 2024-03-06T07:21:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe73f7fe640 1 -- 172.21.15.26:0/1613410725 learned_addr learned my addr 172.21.15.26:0/1613410725 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:26.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe73ffff640 1 --2- 172.21.15.26:0/1613410725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe740155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.776+0000 7fe73e7fc640 1 -- 172.21.15.26:0/1613410725 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207667911 0 0) 0x7fe7400c73d0 con 0x7fe74014f360 2024-03-06T07:21:26.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.777+0000 7fe73e7fc640 1 -- 172.21.15.26:0/1613410725 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe728003520 con 0x7fe74014f360 2024-03-06T07:21:26.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.777+0000 7fe73e7fc640 1 -- 172.21.15.26:0/1613410725 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405851237 0 0) 0x7fe7400be2e0 con 0x7fe7400c88c0 2024-03-06T07:21:26.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.777+0000 7fe73e7fc640 1 -- 172.21.15.26:0/1613410725 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7400c73d0 con 0x7fe7400c88c0 2024-03-06T07:21:26.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.777+0000 7fe73ffff640 1 -- 172.21.15.26:0/1613410725 >> v1:172.21.15.117:6789/0 conn(0x7fe7400c88c0 legacy=0x7fe740158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.777+0000 7fe73ffff640 1 -- 172.21.15.26:0/1613410725 >> v1:172.21.15.26:6789/0 conn(0x7fe74014f360 legacy=0x7fe7400c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.777+0000 7fe73ffff640 1 -- 172.21.15.26:0/1613410725 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400c7670 con 0x7fe740159270 2024-03-06T07:21:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.777+0000 7fe73ffff640 1 --2- 172.21.15.26:0/1613410725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe740155c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe738009dc0 tx=0x7fe73802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e951f4d0631b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe73e7fc640 1 -- 172.21.15.26:0/1613410725 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738039070 con 0x7fe740159270 2024-03-06T07:21:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe73e7fc640 1 -- 172.21.15.26:0/1613410725 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73802cc40 con 0x7fe740159270 2024-03-06T07:21:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe73e7fc640 1 -- 172.21.15.26:0/1613410725 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738034440 con 0x7fe740159270 2024-03-06T07:21:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe746015640 1 -- 172.21.15.26:0/1613410725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 msgr2=0x7fe740155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe746015640 1 --2- 172.21.15.26:0/1613410725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe740155c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe738009dc0 tx=0x7fe73802be90 comp rx=0 tx=0).stop 2024-03-06T07:21:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe746015640 1 -- 172.21.15.26:0/1613410725 shutdown_connections 2024-03-06T07:21:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe746015640 1 --2- 172.21.15.26:0/1613410725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe740155c10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe746015640 1 -- 172.21.15.26:0/1613410725 >> 172.21.15.26:0/1613410725 conn(0x7fe7400ac570 msgr2=0x7fe7400ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe746015640 1 -- 172.21.15.26:0/1613410725 shutdown_connections 2024-03-06T07:21:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.778+0000 7fe746015640 1 -- 172.21.15.26:0/1613410725 wait complete. 2024-03-06T07:21:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.779+0000 7fe746015640 1 Processor -- start 2024-03-06T07:21:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.779+0000 7fe746015640 1 -- start start 2024-03-06T07:21:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe746015640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7400c88c0 0x7fe7400aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe746015640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe74014f360 0x7fe74014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe746015640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe74014aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe746015640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400c7c30 con 0x7fe7400c88c0 2024-03-06T07:21:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe746015640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400c5a50 con 0x7fe74014f360 2024-03-06T07:21:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe746015640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400c5bd0 con 0x7fe740159270 2024-03-06T07:21:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7400c88c0 0x7fe7400aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7400c88c0 0x7fe7400aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47438/0 (socket says 172.21.15.26:47438) 2024-03-06T07:21:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 -- 172.21.15.26:0/826541990 learned_addr learned my addr 172.21.15.26:0/826541990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73ffff640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe74014aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73effd640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe74014f360 0x7fe74014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 -- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 msgr2=0x7fe74014aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe74014aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 -- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe74014f360 msgr2=0x7fe74014a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe74014f360 0x7fe74014a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.780+0000 7fe73f7fe640 1 -- 172.21.15.26:0/826541990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74014b2d0 con 0x7fe7400c88c0 2024-03-06T07:21:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.781+0000 7fe73f7fe640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7400c88c0 0x7fe7400aae50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe73000c500 tx=0x7fe73000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.781+0000 7fe727fff640 1 -- 172.21.15.26:0/826541990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730017070 con 0x7fe7400c88c0 2024-03-06T07:21:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.781+0000 7fe727fff640 1 -- 172.21.15.26:0/826541990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe730005ce0 con 0x7fe7400c88c0 2024-03-06T07:21:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.781+0000 7fe727fff640 1 -- 172.21.15.26:0/826541990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7300072f0 con 0x7fe7400c88c0 2024-03-06T07:21:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.781+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74014aff0 con 0x7fe7400c88c0 2024-03-06T07:21:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.781+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7400c2470 con 0x7fe7400c88c0 2024-03-06T07:21:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.782+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74004f330 con 0x7fe7400c88c0 2024-03-06T07:21:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.782+0000 7fe727fff640 1 -- 172.21.15.26:0/826541990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe73001a070 con 0x7fe7400c88c0 2024-03-06T07:21:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.782+0000 7fe727fff640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.783+0000 7fe727fff640 1 -- 172.21.15.26:0/826541990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe730089680 con 0x7fe7400c88c0 2024-03-06T07:21:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.783+0000 7fe73effd640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.783+0000 7fe73effd640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe72c06b4d0 0x7fe72c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe734007fd0 tx=0x7fe734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:26.787+0000 7fe727fff640 1 -- 172.21.15.26:0/826541990 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe730057d60 con 0x7fe7400c88c0 2024-03-06T07:21:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.009+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7400c8d50 con 0x7fe7400c88c0 2024-03-06T07:21:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.010+0000 7fe727fff640 1 -- 172.21.15.26:0/826541990 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4985 (secure 0 0 0) 0x7fe73005be00 con 0x7fe7400c88c0 2024-03-06T07:21:27.013 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:27.013 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:20.118951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4910,"qdb_cluster":[4910]},"id":2}]} 2024-03-06T07:21:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 22 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe72c06b4d0 msgr2=0x7fe72c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe72c06b4d0 0x7fe72c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe734007fd0 tx=0x7fe734009040 comp rx=0 tx=0).stop 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7400c88c0 msgr2=0x7fe7400aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7400c88c0 0x7fe7400aae50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe73000c500 tx=0x7fe73000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 shutdown_connections 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe740159270 0x7fe74014aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe74014f360 0x7fe74014a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 --2- 172.21.15.26:0/826541990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7400c88c0 0x7fe7400aae50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 >> 172.21.15.26:0/826541990 conn(0x7fe7400ac570 msgr2=0x7fe74014bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 shutdown_connections 2024-03-06T07:21:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.012+0000 7fe746015640 1 -- 172.21.15.26:0/826541990 wait complete. 2024-03-06T07:21:27.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.125+0000 7f4bafbbc640 1 Processor -- start 2024-03-06T07:21:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.125+0000 7f4bafbbc640 1 -- start start 2024-03-06T07:21:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bafbbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba8152520 0x7f4ba806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bafbbc640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ba8155c80 0x7f4ba80c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bafbbc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ba80720a0 0x7f4ba806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bafbbc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80c1dc0 con 0x7f4ba80720a0 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bafbbc640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80c1ae0 con 0x7f4ba8155c80 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bafbbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba806e450 con 0x7f4ba8152520 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ba80720a0 0x7f4ba806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ba80720a0 0x7f4ba806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36838/0 (socket says 172.21.15.26:36838) 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 -- 172.21.15.26:0/860453333 learned_addr learned my addr 172.21.15.26:0/860453333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bad157640 1 --2- 172.21.15.26:0/860453333 >> v2:172.21.15.117:3300/0 conn(0x7f4ba8155c80 0x7f4ba80c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bad958640 1 --2- 172.21.15.26:0/860453333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba8152520 0x7f4ba806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 -- 172.21.15.26:0/860453333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba8152520 msgr2=0x7f4ba806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 --2- 172.21.15.26:0/860453333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba8152520 0x7f4ba806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 -- 172.21.15.26:0/860453333 >> v2:172.21.15.117:3300/0 conn(0x7f4ba8155c80 msgr2=0x7f4ba80c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 --2- 172.21.15.26:0/860453333 >> v2:172.21.15.117:3300/0 conn(0x7f4ba8155c80 0x7f4ba80c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.126+0000 7f4bae159640 1 -- 172.21.15.26:0/860453333 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba806e730 con 0x7f4ba80720a0 2024-03-06T07:21:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.127+0000 7f4bae159640 1 --2- 172.21.15.26:0/860453333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba80720a0 0x7f4ba806afc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80c5230 tx=0x7f4ba002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38320e8fdc183ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.127+0000 7f4bac956640 1 -- 172.21.15.26:0/860453333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba002ca70 con 0x7f4ba80720a0 2024-03-06T07:21:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.127+0000 7f4bac956640 1 -- 172.21.15.26:0/860453333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba002cc10 con 0x7f4ba80720a0 2024-03-06T07:21:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.127+0000 7f4bac956640 1 -- 172.21.15.26:0/860453333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00343f0 con 0x7f4ba80720a0 2024-03-06T07:21:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 -- 172.21.15.26:0/860453333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba80720a0 msgr2=0x7f4ba806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/860453333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba80720a0 0x7f4ba806afc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80c5230 tx=0x7f4ba002bed0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 -- 172.21.15.26:0/860453333 shutdown_connections 2024-03-06T07:21:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/860453333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba80720a0 0x7f4ba806afc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/860453333 >> v2:172.21.15.117:3300/0 conn(0x7f4ba8155c80 0x7f4ba80c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/860453333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba8152520 0x7f4ba806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 -- 172.21.15.26:0/860453333 >> 172.21.15.26:0/860453333 conn(0x7f4ba80c76b0 msgr2=0x7f4ba80c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 -- 172.21.15.26:0/860453333 shutdown_connections 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.128+0000 7f4bafbbc640 1 -- 172.21.15.26:0/860453333 wait complete. 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 Processor -- start 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 -- start start 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80720a0 0x7f4ba80a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8152520 0x7f4ba80ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba8155c80 0x7f4ba80acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8152900 con 0x7f4ba8155c80 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80c17f0 con 0x7f4ba8152520 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.129+0000 7f4bafbbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8152a80 con 0x7f4ba80720a0 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba8155c80 0x7f4ba80acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba8155c80 0x7f4ba80acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36842/0 (socket says 172.21.15.26:36842) 2024-03-06T07:21:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 -- 172.21.15.26:0/723231068 learned_addr learned my addr 172.21.15.26:0/723231068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bad157640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8152520 0x7f4ba80ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bad958640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80720a0 0x7f4ba80a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 -- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80720a0 msgr2=0x7f4ba80a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80720a0 0x7f4ba80a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 -- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8152520 msgr2=0x7f4ba80ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8152520 0x7f4ba80ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 -- 172.21.15.26:0/723231068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba81eb5e0 con 0x7f4ba8155c80 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bae159640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba8155c80 0x7f4ba80acb40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4ba002c4e0 tx=0x7f4ba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4b9e7fc640 1 -- 172.21.15.26:0/723231068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0043020 con 0x7f4ba8155c80 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4b9e7fc640 1 -- 172.21.15.26:0/723231068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba003e450 con 0x7f4ba8155c80 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba81eb8c0 con 0x7f4ba8155c80 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4b9e7fc640 1 -- 172.21.15.26:0/723231068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0009510 con 0x7f4ba8155c80 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.130+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba80a7130 con 0x7f4ba8155c80 2024-03-06T07:21:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.131+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba80a7330 con 0x7f4ba8155c80 2024-03-06T07:21:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.132+0000 7f4b9e7fc640 1 -- 172.21.15.26:0/723231068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ba0046020 con 0x7f4ba8155c80 2024-03-06T07:21:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.132+0000 7f4b9e7fc640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b8c06b500 0x7f4b8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.132+0000 7f4b9e7fc640 1 -- 172.21.15.26:0/723231068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4ba00ac620 con 0x7f4ba8155c80 2024-03-06T07:21:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.132+0000 7f4bad958640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b8c06b500 0x7f4b8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.133+0000 7f4bad958640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b8c06b500 0x7f4b8c06d9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009aa0 tx=0x7f4ba4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.136+0000 7f4b9e7fc640 1 -- 172.21.15.26:0/723231068 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4ba80a7330 con 0x7f4ba8155c80 2024-03-06T07:21:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.316+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ba804f330 con 0x7f4ba8155c80 2024-03-06T07:21:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.318+0000 7f4b9e7fc640 1 -- 172.21.15.26:0/723231068 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17085 (secure 0 0 0) 0x7f4ba004d070 con 0x7f4ba8155c80 2024-03-06T07:21:27.320 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:27.321 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":38,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:14.107392+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:58.241645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:59.328635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/2470383395":"2024-03-07T07:21:14.107365+0000","172.21.15.117:6834/2470383395":"2024-03-07T07:21:14.107365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.319+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b8c06b500 msgr2=0x7f4b8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.319+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b8c06b500 0x7f4b8c06d9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009aa0 tx=0x7f4ba4009190 comp rx=0 tx=0).stop 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.319+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba8155c80 msgr2=0x7f4ba80acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.319+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba8155c80 0x7f4ba80acb40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4ba002c4e0 tx=0x7f4ba0004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 shutdown_connections 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b8c06b500 0x7f4b8c06d9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba8155c80 0x7f4ba80acb40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8152520 0x7f4ba80ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 --2- 172.21.15.26:0/723231068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80720a0 0x7f4ba80a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 >> 172.21.15.26:0/723231068 conn(0x7f4ba80c76b0 msgr2=0x7f4ba80c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 shutdown_connections 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.320+0000 7f4bafbbc640 1 -- 172.21.15.26:0/723231068 wait complete. 2024-03-06T07:21:27.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:58.241645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:59.328635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:27.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:27.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.435+0000 7f698cb56640 1 Processor -- start 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.435+0000 7f698cb56640 1 -- start start 2024-03-06T07:21:27.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698cb56640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69881528d0 0x7f69880724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698cb56640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6988152cb0 con 0x7f69880c67f0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698cb56640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69880c38b0 con 0x7f6988072a10 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698cb56640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69881594f0 con 0x7f69881528d0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69881528d0 0x7f69880724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f6986d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f69880c67f0 0x7f698815fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45736/0 (socket says 172.21.15.26:45736) 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f6986d9d640 1 -- 172.21.15.26:0/1350310996 learned_addr learned my addr 172.21.15.26:0/1350310996 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698659c640 1 -- 172.21.15.26:0/1350310996 >> v1:172.21.15.117:6789/0 conn(0x7f6988072a10 legacy=0x7f6988155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698659c640 1 -- 172.21.15.26:0/1350310996 >> v1:172.21.15.26:6789/0 conn(0x7f69880c67f0 legacy=0x7f698815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f698659c640 1 -- 172.21.15.26:0/1350310996 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880a7f70 con 0x7f69881528d0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.436+0000 7f6985d9b640 1 -- 172.21.15.26:0/1350310996 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615217064 0 0) 0x7f6988152cb0 con 0x7f69880c67f0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f698659c640 1 --2- 172.21.15.26:0/1350310996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69881528d0 0x7f69880724d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f69780097a0 tx=0x7f697802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a37341d13b7378 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f6985d9b640 1 -- 172.21.15.26:0/1350310996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697802f810 con 0x7f69881528d0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f6985d9b640 1 -- 172.21.15.26:0/1350310996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697802f9b0 con 0x7f69881528d0 2024-03-06T07:21:27.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f6985d9b640 1 -- 172.21.15.26:0/1350310996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697802fd80 con 0x7f69881528d0 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f698cb56640 1 -- 172.21.15.26:0/1350310996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69881528d0 msgr2=0x7f69880724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f698cb56640 1 --2- 172.21.15.26:0/1350310996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69881528d0 0x7f69880724d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f69780097a0 tx=0x7f697802b9f0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f698cb56640 1 -- 172.21.15.26:0/1350310996 shutdown_connections 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f698cb56640 1 --2- 172.21.15.26:0/1350310996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69881528d0 0x7f69880724d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.437+0000 7f698cb56640 1 -- 172.21.15.26:0/1350310996 >> 172.21.15.26:0/1350310996 conn(0x7f69880aa6a0 msgr2=0x7f69880aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.438+0000 7f698cb56640 1 -- 172.21.15.26:0/1350310996 shutdown_connections 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.438+0000 7f698cb56640 1 -- 172.21.15.26:0/1350310996 wait complete. 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 Processor -- start 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 -- start start 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988072a10 0x7f698806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69880c67f0 0x7f6988147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69881528d0 0x7f6988147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69881591a0 con 0x7f6988072a10 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880a8250 con 0x7f69880c67f0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698cb56640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880c4ef0 con 0x7f69881528d0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988072a10 0x7f698806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988072a10 0x7f698806ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36864/0 (socket says 172.21.15.26:36864) 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.439+0000 7f698659c640 1 -- 172.21.15.26:0/1899272513 learned_addr learned my addr 172.21.15.26:0/1899272513 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f697ffff640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69880c67f0 0x7f6988147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f6986d9d640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69881528d0 0x7f6988147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698659c640 1 -- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69881528d0 msgr2=0x7f6988147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698659c640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69881528d0 0x7f6988147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698659c640 1 -- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69880c67f0 msgr2=0x7f6988147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698659c640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69880c67f0 0x7f6988147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698659c640 1 -- 172.21.15.26:0/1899272513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69881ec200 con 0x7f6988072a10 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f697ffff640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69880c67f0 0x7f6988147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f6986d9d640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69881528d0 0x7f6988147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698659c640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988072a10 0x7f698806ff50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f69780098c0 tx=0x7f6978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f697effd640 1 -- 172.21.15.26:0/1899272513 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978047070 con 0x7f6988072a10 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f697effd640 1 -- 172.21.15.26:0/1899272513 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6978037c60 con 0x7f6988072a10 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69881edfe0 con 0x7f6988072a10 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f697effd640 1 -- 172.21.15.26:0/1899272513 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697802fe60 con 0x7f6988072a10 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.440+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69881eedc0 con 0x7f6988072a10 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.441+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698804f330 con 0x7f6988072a10 2024-03-06T07:21:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.441+0000 7f697effd640 1 -- 172.21.15.26:0/1899272513 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f697804a070 con 0x7f6988072a10 2024-03-06T07:21:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.442+0000 7f697effd640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f696806b500 0x7f696806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.442+0000 7f697effd640 1 -- 172.21.15.26:0/1899272513 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f69780ac0f0 con 0x7f6988072a10 2024-03-06T07:21:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.442+0000 7f697ffff640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f696806b500 0x7f696806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.442+0000 7f697ffff640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f696806b500 0x7f696806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6970007fd0 tx=0x7f6970009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.446+0000 7f697effd640 1 -- 172.21.15.26:0/1899272513 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6978033030 con 0x7f6988072a10 2024-03-06T07:21:27.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.627+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6988073120 con 0x7f6988072a10 2024-03-06T07:21:27.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.627+0000 7f697effd640 1 -- 172.21.15.26:0/1899272513 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f697807ad90 con 0x7f6988072a10 2024-03-06T07:21:27.630 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f696806b500 msgr2=0x7f696806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f696806b500 0x7f696806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6970007fd0 tx=0x7f6970009040 comp rx=0 tx=0).stop 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988072a10 msgr2=0x7f698806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988072a10 0x7f698806ff50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f69780098c0 tx=0x7f6978004730 comp rx=0 tx=0).stop 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 shutdown_connections 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f696806b500 0x7f696806d9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69881528d0 0x7f6988147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69880c67f0 0x7f6988147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 --2- 172.21.15.26:0/1899272513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988072a10 0x7f698806ff50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 >> 172.21.15.26:0/1899272513 conn(0x7f69880aa6a0 msgr2=0x7f69880c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 shutdown_connections 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.629+0000 7f698cb56640 1 -- 172.21.15.26:0/1899272513 wait complete. 2024-03-06T07:21:27.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:21:27.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.745+0000 7f7e1e9ac640 1 Processor -- start 2024-03-06T07:21:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.745+0000 7f7e1e9ac640 1 -- start start 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1e9ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e180ab170 0x7f7e180c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1e9ac640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e180c3ef0 con 0x7f7e1806c900 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1e9ac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e181556d0 con 0x7f7e180c5990 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1e9ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e181559b0 con 0x7f7e180ab170 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e177fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7e1806c900 0x7f7e180a9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45752/0 (socket says 172.21.15.26:45752) 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e177fe640 1 -- 172.21.15.26:0/1301520710 learned_addr learned my addr 172.21.15.26:0/1301520710 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1cf49640 1 --2- 172.21.15.26:0/1301520710 >> v2:172.21.15.117:3300/0 conn(0x7f7e180ab170 0x7f7e180c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e16ffd640 1 -- 172.21.15.26:0/1301520710 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205139859 0 0) 0x7f7e180c3ef0 con 0x7f7e1806c900 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e16ffd640 1 -- 172.21.15.26:0/1301520710 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e00003520 con 0x7f7e1806c900 2024-03-06T07:21:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e16ffd640 1 -- 172.21.15.26:0/1301520710 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30194239 0 0) 0x7f7e181556d0 con 0x7f7e180c5990 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e16ffd640 1 -- 172.21.15.26:0/1301520710 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e180c3ef0 con 0x7f7e180c5990 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1cf49640 1 -- 172.21.15.26:0/1301520710 >> v1:172.21.15.117:6789/0 conn(0x7f7e180c5990 legacy=0x7f7e1806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1cf49640 1 -- 172.21.15.26:0/1301520710 >> v1:172.21.15.26:6789/0 conn(0x7f7e1806c900 legacy=0x7f7e180a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.746+0000 7f7e1cf49640 1 -- 172.21.15.26:0/1301520710 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e18155c90 con 0x7f7e180ab170 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.747+0000 7f7e1cf49640 1 --2- 172.21.15.26:0/1301520710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180ab170 0x7f7e180c0ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009dc0 tx=0x7f7e1002bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffa844e901eb4060 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.747+0000 7f7e16ffd640 1 -- 172.21.15.26:0/1301520710 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10039070 con 0x7f7e180ab170 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.747+0000 7f7e16ffd640 1 -- 172.21.15.26:0/1301520710 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1002ccd0 con 0x7f7e180ab170 2024-03-06T07:21:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.747+0000 7f7e16ffd640 1 -- 172.21.15.26:0/1301520710 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10034660 con 0x7f7e180ab170 2024-03-06T07:21:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.748+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/1301520710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180ab170 msgr2=0x7f7e180c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.748+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/1301520710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180ab170 0x7f7e180c0ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009dc0 tx=0x7f7e1002bf20 comp rx=0 tx=0).stop 2024-03-06T07:21:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.748+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/1301520710 shutdown_connections 2024-03-06T07:21:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.748+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/1301520710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180ab170 0x7f7e180c0ab0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.748+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/1301520710 >> 172.21.15.26:0/1301520710 conn(0x7f7e180c6710 msgr2=0x7f7e180c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.748+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/1301520710 shutdown_connections 2024-03-06T07:21:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.748+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/1301520710 wait complete. 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.749+0000 7f7e1e9ac640 1 Processor -- start 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.749+0000 7f7e1e9ac640 1 -- start start 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e1e9ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e1806c900 0x7f7e1814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e1e9ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e180ab170 0x7f7e181e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e1e9ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180c5990 0x7f7e181e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e1e9ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e18156610 con 0x7f7e180ab170 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e1e9ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e18156490 con 0x7f7e180c5990 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e1e9ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180c3af0 con 0x7f7e1806c900 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e180ab170 0x7f7e181e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e180ab170 0x7f7e181e5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36878/0 (socket says 172.21.15.26:36878) 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 -- 172.21.15.26:0/4090573288 learned_addr learned my addr 172.21.15.26:0/4090573288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e17fff640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e1806c900 0x7f7e1814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e1cf49640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180c5990 0x7f7e181e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 -- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e1806c900 msgr2=0x7f7e1814f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e1806c900 0x7f7e1814f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 -- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180c5990 msgr2=0x7f7e181e5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180c5990 0x7f7e181e5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 -- 172.21.15.26:0/4090573288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e181ebfa0 con 0x7f7e180ab170 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.750+0000 7f7e177fe640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e180ab170 0x7f7e181e5ab0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c500 tx=0x7f7e0800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.751+0000 7f7e14ff9640 1 -- 172.21.15.26:0/4090573288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08017070 con 0x7f7e180ab170 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.751+0000 7f7e14ff9640 1 -- 172.21.15.26:0/4090573288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e08005ce0 con 0x7f7e180ab170 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.751+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e181ec280 con 0x7f7e180ab170 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.751+0000 7f7e14ff9640 1 -- 172.21.15.26:0/4090573288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080072f0 con 0x7f7e180ab170 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.751+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1814f680 con 0x7f7e180ab170 2024-03-06T07:21:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.752+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e180c3900 con 0x7f7e180ab170 2024-03-06T07:21:27.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.752+0000 7f7e14ff9640 1 -- 172.21.15.26:0/4090573288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e0801a070 con 0x7f7e180ab170 2024-03-06T07:21:27.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.752+0000 7f7e14ff9640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e0406b4d0 0x7f7e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.753+0000 7f7e14ff9640 1 -- 172.21.15.26:0/4090573288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7e08089650 con 0x7f7e180ab170 2024-03-06T07:21:27.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.753+0000 7f7e17fff640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e0406b4d0 0x7f7e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.753+0000 7f7e17fff640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e0406b4d0 0x7f7e0406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c007fd0 tx=0x7f7e0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.757+0000 7f7e14ff9640 1 -- 172.21.15.26:0/4090573288 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7e08057d30 con 0x7f7e180ab170 2024-03-06T07:21:27.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.936+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e1804f330 con 0x7f7e180ab170 2024-03-06T07:21:27.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.936+0000 7f7e14ff9640 1 -- 172.21.15.26:0/4090573288 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e0805bdd0 con 0x7f7e180ab170 2024-03-06T07:21:27.939 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:21:27.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e0406b4d0 msgr2=0x7f7e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e0406b4d0 0x7f7e0406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c007fd0 tx=0x7f7e0c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:27.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e180ab170 msgr2=0x7f7e181e5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e180ab170 0x7f7e181e5ab0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c500 tx=0x7f7e0800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 shutdown_connections 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e0406b4d0 0x7f7e0406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e180c5990 0x7f7e181e5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e180ab170 0x7f7e181e5ab0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 --2- 172.21.15.26:0/4090573288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e1806c900 0x7f7e1814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 >> 172.21.15.26:0/4090573288 conn(0x7f7e180c6710 msgr2=0x7f7e1814be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 shutdown_connections 2024-03-06T07:21:27.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:27.938+0000 7f7e1e9ac640 1 -- 172.21.15.26:0/4090573288 wait complete. 2024-03-06T07:21:27.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:21:28.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.053+0000 7fad5bb13640 1 Processor -- start 2024-03-06T07:21:28.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.053+0000 7fad5bb13640 1 -- start start 2024-03-06T07:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad5bb13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad540b0070 0x7fad54155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad5bb13640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fad5406ca70 0x7fad540c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad5bb13640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad540b0450 con 0x7fad540c5380 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad5bb13640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5406c020 con 0x7fad5406ca70 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad5bb13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad540c77f0 con 0x7fad540b0070 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad5a0b0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fad540c5380 0x7fad5414e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45758/0 (socket says 172.21.15.26:45758) 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad5a0b0640 1 -- 172.21.15.26:0/1091905914 learned_addr learned my addr 172.21.15.26:0/1091905914 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad598af640 1 --2- 172.21.15.26:0/1091905914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad540b0070 0x7fad54155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad590ae640 1 --2- 172.21.15.26:0/1091905914 >> v2:172.21.15.117:3300/0 conn(0x7fad5406ca70 0x7fad540c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad598af640 1 -- 172.21.15.26:0/1091905914 >> v2:172.21.15.117:3300/0 conn(0x7fad5406ca70 msgr2=0x7fad540c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad598af640 1 --2- 172.21.15.26:0/1091905914 >> v2:172.21.15.117:3300/0 conn(0x7fad5406ca70 0x7fad540c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad588ad640 1 -- 172.21.15.26:0/1091905914 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709044226 0 0) 0x7fad4c001490 con 0x7fad540c5380 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad598af640 1 -- 172.21.15.26:0/1091905914 >> v1:172.21.15.26:6789/0 conn(0x7fad540c5380 legacy=0x7fad5414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad598af640 1 -- 172.21.15.26:0/1091905914 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540c7ad0 con 0x7fad540b0070 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.054+0000 7fad590ae640 1 --2- 172.21.15.26:0/1091905914 >> v2:172.21.15.117:3300/0 conn(0x7fad5406ca70 0x7fad540c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.055+0000 7fad598af640 1 --2- 172.21.15.26:0/1091905914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad540b0070 0x7fad54155c30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad500097a0 tx=0x7fad5002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d5a503b7331942 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.055+0000 7fad588ad640 1 -- 172.21.15.26:0/1091905914 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500315a0 con 0x7fad540b0070 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.055+0000 7fad588ad640 1 -- 172.21.15.26:0/1091905914 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad50037070 con 0x7fad540b0070 2024-03-06T07:21:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.055+0000 7fad588ad640 1 -- 172.21.15.26:0/1091905914 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50031960 con 0x7fad540b0070 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 -- 172.21.15.26:0/1091905914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad540b0070 msgr2=0x7fad54155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 --2- 172.21.15.26:0/1091905914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad540b0070 0x7fad54155c30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad500097a0 tx=0x7fad5002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 -- 172.21.15.26:0/1091905914 shutdown_connections 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 --2- 172.21.15.26:0/1091905914 >> v2:172.21.15.117:3300/0 conn(0x7fad5406ca70 0x7fad540c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 --2- 172.21.15.26:0/1091905914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad540b0070 0x7fad54155c30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 -- 172.21.15.26:0/1091905914 >> 172.21.15.26:0/1091905914 conn(0x7fad540ac590 msgr2=0x7fad540ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 -- 172.21.15.26:0/1091905914 shutdown_connections 2024-03-06T07:21:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.056+0000 7fad5bb13640 1 -- 172.21.15.26:0/1091905914 wait complete. 2024-03-06T07:21:28.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.057+0000 7fad5bb13640 1 Processor -- start 2024-03-06T07:21:28.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.057+0000 7fad5bb13640 1 -- start start 2024-03-06T07:21:28.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.057+0000 7fad5bb13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5406ca70 0x7fad540c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.057+0000 7fad5bb13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad540b0070 0x7fad541eb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5bb13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad540c5380 0x7fad54154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5bb13640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5406cef0 con 0x7fad540c5380 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5bb13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5406c550 con 0x7fad540b0070 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5bb13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5406da80 con 0x7fad5406ca70 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad540c5380 0x7fad54154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad540c5380 0x7fad54154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36892/0 (socket says 172.21.15.26:36892) 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 -- 172.21.15.26:0/904091288 learned_addr learned my addr 172.21.15.26:0/904091288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad598af640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5406ca70 0x7fad540c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad590ae640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad540b0070 0x7fad541eb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 -- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5406ca70 msgr2=0x7fad540c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5406ca70 0x7fad540c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 -- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad540b0070 msgr2=0x7fad541eb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad540b0070 0x7fad541eb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 -- 172.21.15.26:0/904091288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540c12c0 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5a0b0640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad540c5380 0x7fad54154620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00c730 tx=0x7fad4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad4a7fc640 1 -- 172.21.15.26:0/904091288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c018070 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad4a7fc640 1 -- 172.21.15.26:0/904091288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c00ce40 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad4a7fc640 1 -- 172.21.15.26:0/904091288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00a860 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad540c4430 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.058+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad540c1560 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.059+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5404f330 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.060+0000 7fad4a7fc640 1 -- 172.21.15.26:0/904091288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad4c00aa00 con 0x7fad540c5380 2024-03-06T07:21:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.060+0000 7fad4a7fc640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b4d0 0x7fad3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.060+0000 7fad4a7fc640 1 -- 172.21.15.26:0/904091288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fad4c089920 con 0x7fad540c5380 2024-03-06T07:21:28.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.060+0000 7fad598af640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b4d0 0x7fad3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.061+0000 7fad598af640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b4d0 0x7fad3806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fad50036000 tx=0x7fad5003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.064+0000 7fad4a7fc640 1 -- 172.21.15.26:0/904091288 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fad540b0450 con 0x7fad540c5380 2024-03-06T07:21:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.244+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad540af690 con 0x7fad540c5380 2024-03-06T07:21:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.245+0000 7fad4a7fc640 1 -- 172.21.15.26:0/904091288 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad4c057f80 con 0x7fad540c5380 2024-03-06T07:21:28.247 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:21:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b4d0 msgr2=0x7fad3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b4d0 0x7fad3806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fad50036000 tx=0x7fad5003d040 comp rx=0 tx=0).stop 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad540c5380 msgr2=0x7fad54154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad540c5380 0x7fad54154620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00c730 tx=0x7fad4c00cbf0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 shutdown_connections 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b4d0 0x7fad3806d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad540c5380 0x7fad54154620 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad540b0070 0x7fad541eb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 --2- 172.21.15.26:0/904091288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5406ca70 0x7fad540c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 >> 172.21.15.26:0/904091288 conn(0x7fad540ac590 msgr2=0x7fad540ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 shutdown_connections 2024-03-06T07:21:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.247+0000 7fad5bb13640 1 -- 172.21.15.26:0/904091288 wait complete. 2024-03-06T07:21:28.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.359+0000 7f03bdd35640 1 Processor -- start 2024-03-06T07:21:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.359+0000 7f03bdd35640 1 -- start start 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03bdd35640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f03b80c7380 0x7f03b80c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03bdd35640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b8072630 con 0x7f03b806da40 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03bdd35640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b8072910 con 0x7f03b80c6780 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03bdd35640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b806de20 con 0x7f03b80c7380 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03b806da40 0x7f03b8155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45762/0 (socket says 172.21.15.26:45762) 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b77fe640 1 -- 172.21.15.26:0/2962307351 learned_addr learned my addr 172.21.15.26:0/2962307351 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b7fff640 1 --2- 172.21.15.26:0/2962307351 >> v2:172.21.15.117:3300/0 conn(0x7f03b80c7380 0x7f03b80c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b67fc640 1 -- 172.21.15.26:0/2962307351 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787359080 0 0) 0x7f03b8072630 con 0x7f03b806da40 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b67fc640 1 -- 172.21.15.26:0/2962307351 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a0003520 con 0x7f03b806da40 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b7fff640 1 -- 172.21.15.26:0/2962307351 >> v1:172.21.15.117:6789/0 conn(0x7f03b80c6780 legacy=0x7f03b8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b67fc640 1 -- 172.21.15.26:0/2962307351 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099940582 0 0) 0x7f03b8072910 con 0x7f03b80c6780 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.360+0000 7f03b7fff640 1 -- 172.21.15.26:0/2962307351 >> v1:172.21.15.26:6789/0 conn(0x7f03b806da40 legacy=0x7f03b8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.361+0000 7f03b7fff640 1 -- 172.21.15.26:0/2962307351 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8072bf0 con 0x7f03b80c7380 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.361+0000 7f03b7fff640 1 --2- 172.21.15.26:0/2962307351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b80c3280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009d90 tx=0x7f03b002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba0b284f7ce69e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.361+0000 7f03b67fc640 1 -- 172.21.15.26:0/2962307351 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0039070 con 0x7f03b80c7380 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.361+0000 7f03b67fc640 1 -- 172.21.15.26:0/2962307351 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b002cc60 con 0x7f03b80c7380 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.361+0000 7f03b67fc640 1 -- 172.21.15.26:0/2962307351 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0034640 con 0x7f03b80c7380 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.362+0000 7f03bdd35640 1 -- 172.21.15.26:0/2962307351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 msgr2=0x7f03b80c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.362+0000 7f03bdd35640 1 --2- 172.21.15.26:0/2962307351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b80c3280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009d90 tx=0x7f03b002beb0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.362+0000 7f03bdd35640 1 -- 172.21.15.26:0/2962307351 shutdown_connections 2024-03-06T07:21:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.362+0000 7f03bdd35640 1 --2- 172.21.15.26:0/2962307351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b80c3280 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.362+0000 7f03bdd35640 1 -- 172.21.15.26:0/2962307351 >> 172.21.15.26:0/2962307351 conn(0x7f03b80aa630 msgr2=0x7f03b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.362+0000 7f03bdd35640 1 -- 172.21.15.26:0/2962307351 shutdown_connections 2024-03-06T07:21:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.362+0000 7f03bdd35640 1 -- 172.21.15.26:0/2962307351 wait complete. 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.363+0000 7f03bdd35640 1 Processor -- start 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.363+0000 7f03bdd35640 1 -- start start 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03bdd35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b806da40 0x7f03b814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03bdd35640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b80c6780 0x7f03b8156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03bdd35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b81568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03bdd35640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c37c0 con 0x7f03b80c7380 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03bdd35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b81594a0 con 0x7f03b806da40 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03bdd35640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c3940 con 0x7f03b80c6780 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b81568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b81568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36894/0 (socket says 172.21.15.26:36894) 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 -- 172.21.15.26:0/271129307 learned_addr learned my addr 172.21.15.26:0/271129307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b77fe640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b806da40 0x7f03b814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b6ffd640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b80c6780 0x7f03b8156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 -- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b80c6780 msgr2=0x7f03b8156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b80c6780 0x7f03b8156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 -- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b806da40 msgr2=0x7f03b814deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b806da40 0x7f03b814deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.364+0000 7f03b7fff640 1 -- 172.21.15.26:0/271129307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8149eb0 con 0x7f03b80c7380 2024-03-06T07:21:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.365+0000 7f03b7fff640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b81568d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f03b0037040 tx=0x7f03b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.365+0000 7f039ffff640 1 -- 172.21.15.26:0/271129307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0039050 con 0x7f03b80c7380 2024-03-06T07:21:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.365+0000 7f039ffff640 1 -- 172.21.15.26:0/271129307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0045070 con 0x7f03b80c7380 2024-03-06T07:21:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.365+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b81eed20 con 0x7f03b80c7380 2024-03-06T07:21:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.365+0000 7f039ffff640 1 -- 172.21.15.26:0/271129307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0009450 con 0x7f03b80c7380 2024-03-06T07:21:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.365+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b806e700 con 0x7f03b80c7380 2024-03-06T07:21:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.366+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b804f330 con 0x7f03b80c7380 2024-03-06T07:21:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.366+0000 7f039ffff640 1 -- 172.21.15.26:0/271129307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03b003e830 con 0x7f03b80c7380 2024-03-06T07:21:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.367+0000 7f039ffff640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b4d0 0x7f03a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.367+0000 7f039ffff640 1 -- 172.21.15.26:0/271129307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f03b00ad2c0 con 0x7f03b80c7380 2024-03-06T07:21:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.367+0000 7f03b77fe640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b4d0 0x7f03a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.367+0000 7f03b77fe640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b4d0 0x7f03a406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005880 tx=0x7f03a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.371+0000 7f039ffff640 1 -- 172.21.15.26:0/271129307 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f03b007ba30 con 0x7f03b80c7380 2024-03-06T07:21:28.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.588+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03b8073240 con 0x7f03b80c7380 2024-03-06T07:21:28.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.589+0000 7f039ffff640 1 -- 172.21.15.26:0/271129307 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4985 (secure 0 0 0) 0x7f03b007fad0 con 0x7f03b80c7380 2024-03-06T07:21:28.592 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:28.593 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:20.118951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4910,"qdb_cluster":[4910]},"id":2}]} 2024-03-06T07:21:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 22 2024-03-06T07:21:28.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b4d0 msgr2=0x7f03a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b4d0 0x7f03a406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005880 tx=0x7f03a8009040 comp rx=0 tx=0).stop 2024-03-06T07:21:28.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 msgr2=0x7f03b81568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b81568d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f03b0037040 tx=0x7f03b0004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:28.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 shutdown_connections 2024-03-06T07:21:28.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b4d0 0x7f03a406d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b81568d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b80c6780 0x7f03b8156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 --2- 172.21.15.26:0/271129307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b806da40 0x7f03b814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 >> 172.21.15.26:0/271129307 conn(0x7f03b80aa630 msgr2=0x7f03b80c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 shutdown_connections 2024-03-06T07:21:28.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.591+0000 7f03bdd35640 1 -- 172.21.15.26:0/271129307 wait complete. 2024-03-06T07:21:28.605 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.707+0000 7fb19bd43640 1 Processor -- start 2024-03-06T07:21:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.707+0000 7fb19bd43640 1 -- start start 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19bd43640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1940aa180 0x7fb19414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19bd43640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb19414f140 0x7fb1940c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19bd43640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb194155bb0 con 0x7fb1941550c0 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19bd43640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1940c4610 con 0x7fb1940aa180 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19bd43640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19414f5f0 con 0x7fb19414f140 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb199adf640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb1941550c0 0x7fb1941554a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:45750/0 (socket says 172.21.15.26:45750) 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb1992de640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1940aa180 0x7fb19414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb199adf640 1 -- 172.21.15.26:0/2044668684 learned_addr learned my addr 172.21.15.26:0/2044668684 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19a2e0640 1 --2- 172.21.15.26:0/2044668684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb19414f140 0x7fb1940c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb198add640 1 -- 172.21.15.26:0/2044668684 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756848320 0 0) 0x7fb194155bb0 con 0x7fb1941550c0 2024-03-06T07:21:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb198add640 1 -- 172.21.15.26:0/2044668684 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb180003520 con 0x7fb1941550c0 2024-03-06T07:21:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19a2e0640 1 -- 172.21.15.26:0/2044668684 >> v2:172.21.15.117:3300/0 conn(0x7fb1940aa180 msgr2=0x7fb19414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19a2e0640 1 --2- 172.21.15.26:0/2044668684 >> v2:172.21.15.117:3300/0 conn(0x7fb1940aa180 0x7fb19414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19a2e0640 1 -- 172.21.15.26:0/2044668684 >> v1:172.21.15.117:6789/0 conn(0x7fb1941550c0 legacy=0x7fb1941554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb19a2e0640 1 -- 172.21.15.26:0/2044668684 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940aa660 con 0x7fb19414f140 2024-03-06T07:21:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.708+0000 7fb1992de640 1 --2- 172.21.15.26:0/2044668684 >> v2:172.21.15.117:3300/0 conn(0x7fb1940aa180 0x7fb19414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.709+0000 7fb19a2e0640 1 --2- 172.21.15.26:0/2044668684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb19414f140 0x7fb1940c5060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00aea0 tx=0x7fb18c02ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300962162fe850ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.709+0000 7fb198add640 1 -- 172.21.15.26:0/2044668684 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c02da20 con 0x7fb19414f140 2024-03-06T07:21:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.709+0000 7fb198add640 1 -- 172.21.15.26:0/2044668684 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c02dbc0 con 0x7fb19414f140 2024-03-06T07:21:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.709+0000 7fb198add640 1 -- 172.21.15.26:0/2044668684 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c008440 con 0x7fb19414f140 2024-03-06T07:21:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 -- 172.21.15.26:0/2044668684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb19414f140 msgr2=0x7fb1940c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 --2- 172.21.15.26:0/2044668684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb19414f140 0x7fb1940c5060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00aea0 tx=0x7fb18c02ce80 comp rx=0 tx=0).stop 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 -- 172.21.15.26:0/2044668684 shutdown_connections 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 --2- 172.21.15.26:0/2044668684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb19414f140 0x7fb1940c5060 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 --2- 172.21.15.26:0/2044668684 >> v2:172.21.15.117:3300/0 conn(0x7fb1940aa180 0x7fb19414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 -- 172.21.15.26:0/2044668684 >> 172.21.15.26:0/2044668684 conn(0x7fb1940c6710 msgr2=0x7fb1940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 -- 172.21.15.26:0/2044668684 shutdown_connections 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.710+0000 7fb19bd43640 1 -- 172.21.15.26:0/2044668684 wait complete. 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 Processor -- start 2024-03-06T07:21:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 -- start start 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1940aa180 0x7fb1940bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19414f140 0x7fb1940a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1941550c0 0x7fb1940a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940c1310 con 0x7fb19414f140 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940c4860 con 0x7fb1940aa180 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb19bd43640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940c1490 con 0x7fb1941550c0 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.711+0000 7fb1992de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19414f140 0x7fb1940a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19414f140 0x7fb1940a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36910/0 (socket says 172.21.15.26:36910) 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 -- 172.21.15.26:0/525497713 learned_addr learned my addr 172.21.15.26:0/525497713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb199adf640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1940aa180 0x7fb1940bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb19a2e0640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1941550c0 0x7fb1940a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 -- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1941550c0 msgr2=0x7fb1940a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1941550c0 0x7fb1940a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 -- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1940aa180 msgr2=0x7fb1940bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1940aa180 0x7fb1940bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 -- 172.21.15.26:0/525497713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19406d2d0 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb1992de640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19414f140 0x7fb1940a6430 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e930 tx=0x7fb18400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb18a7fc640 1 -- 172.21.15.26:0/525497713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184017070 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb18a7fc640 1 -- 172.21.15.26:0/525497713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18400ca00 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb18a7fc640 1 -- 172.21.15.26:0/525497713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400cdf0 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19406d5b0 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.712+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19406db70 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.713+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1940acab0 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.714+0000 7fb18a7fc640 1 -- 172.21.15.26:0/525497713 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb18401a070 con 0x7fb19414f140 2024-03-06T07:21:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.714+0000 7fb18a7fc640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb17806b4d0 0x7fb17806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.714+0000 7fb18a7fc640 1 -- 172.21.15.26:0/525497713 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb18408a300 con 0x7fb19414f140 2024-03-06T07:21:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.714+0000 7fb199adf640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb17806b4d0 0x7fb17806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.715+0000 7fb199adf640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb17806b4d0 0x7fb17806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb190005880 tx=0x7fb190009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.718+0000 7fb18a7fc640 1 -- 172.21.15.26:0/525497713 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb1940acab0 con 0x7fb19414f140 2024-03-06T07:21:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.899+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb19404f330 con 0x7fb19414f140 2024-03-06T07:21:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.900+0000 7fb18a7fc640 1 -- 172.21.15.26:0/525497713 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17085 (secure 0 0 0) 0x7fb1840588c0 con 0x7fb19414f140 2024-03-06T07:21:28.903 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:28.903 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":38,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:14.107392+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:58.241645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:59.328635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/2470383395":"2024-03-07T07:21:14.107365+0000","172.21.15.117:6834/2470383395":"2024-03-07T07:21:14.107365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.902+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb17806b4d0 msgr2=0x7fb17806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.902+0000 7fb19bd43640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb17806b4d0 0x7fb17806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb190005880 tx=0x7fb190009040 comp rx=0 tx=0).stop 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.902+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19414f140 msgr2=0x7fb1940a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.902+0000 7fb19bd43640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19414f140 0x7fb1940a6430 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e930 tx=0x7fb18400edf0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.902+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 shutdown_connections 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.903+0000 7fb19bd43640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb17806b4d0 0x7fb17806d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.903+0000 7fb19bd43640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1941550c0 0x7fb1940a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.903+0000 7fb19bd43640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19414f140 0x7fb1940a6430 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.903+0000 7fb19bd43640 1 --2- 172.21.15.26:0/525497713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1940aa180 0x7fb1940bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.903+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 >> 172.21.15.26:0/525497713 conn(0x7fb1940c6710 msgr2=0x7fb194153180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.903+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 shutdown_connections 2024-03-06T07:21:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:28.903+0000 7fb19bd43640 1 -- 172.21.15.26:0/525497713 wait complete. 2024-03-06T07:21:28.916 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:21:28.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 Processor -- start 2024-03-06T07:21:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 -- start start 2024-03-06T07:21:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01f406da40 0x7f01f4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f40c6780 0x7f01f4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01f41592d0 0x7f01f40c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f406de20 con 0x7f01f406da40 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f4156050 con 0x7f01f41592d0 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.016+0000 7f01f94e0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4072630 con 0x7f01f40c6780 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01f406da40 0x7f01f4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01f41592d0 0x7f01f40c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01f406da40 0x7f01f4155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36924/0 (socket says 172.21.15.26:36924) 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f27fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f40c6780 0x7f01f4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 -- 172.21.15.26:0/3851556762 learned_addr learned my addr 172.21.15.26:0/3851556762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 -- 172.21.15.26:0/3851556762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f40c6780 msgr2=0x7f01f4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 --2- 172.21.15.26:0/3851556762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f40c6780 0x7f01f4158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 -- 172.21.15.26:0/3851556762 >> v2:172.21.15.117:3300/0 conn(0x7f01f41592d0 msgr2=0x7f01f40c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 --2- 172.21.15.26:0/3851556762 >> v2:172.21.15.117:3300/0 conn(0x7f01f41592d0 0x7f01f40c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f2ffd640 1 -- 172.21.15.26:0/3851556762 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4072910 con 0x7f01f406da40 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.017+0000 7f01f37fe640 1 --2- 172.21.15.26:0/3851556762 >> v2:172.21.15.117:3300/0 conn(0x7f01f41592d0 0x7f01f40c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f2ffd640 1 --2- 172.21.15.26:0/3851556762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f406da40 0x7f01f4155b10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01f40c46e0 tx=0x7f01e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1635c36a6e8a54b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f1ffb640 1 -- 172.21.15.26:0/3851556762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e402e070 con 0x7f01f406da40 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f1ffb640 1 -- 172.21.15.26:0/3851556762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4037070 con 0x7f01f406da40 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f1ffb640 1 -- 172.21.15.26:0/3851556762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4031a00 con 0x7f01f406da40 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 -- 172.21.15.26:0/3851556762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f406da40 msgr2=0x7f01f4155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 --2- 172.21.15.26:0/3851556762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f406da40 0x7f01f4155b10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01f40c46e0 tx=0x7f01e402b880 comp rx=0 tx=0).stop 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 -- 172.21.15.26:0/3851556762 shutdown_connections 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 --2- 172.21.15.26:0/3851556762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f40c6780 0x7f01f4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 --2- 172.21.15.26:0/3851556762 >> v2:172.21.15.117:3300/0 conn(0x7f01f41592d0 0x7f01f40c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 --2- 172.21.15.26:0/3851556762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f406da40 0x7f01f4155b10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 -- 172.21.15.26:0/3851556762 >> 172.21.15.26:0/3851556762 conn(0x7f01f40aa630 msgr2=0x7f01f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 -- 172.21.15.26:0/3851556762 shutdown_connections 2024-03-06T07:21:29.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.018+0000 7f01f94e0640 1 -- 172.21.15.26:0/3851556762 wait complete. 2024-03-06T07:21:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 Processor -- start 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 -- start start 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f406da40 0x7f01f4157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f40c6780 0x7f01f41eea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f41592d0 0x7f01f40a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40c37c0 con 0x7f01f41592d0 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4158890 con 0x7f01f40c6780 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f94e0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40c3940 con 0x7f01f406da40 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f41592d0 0x7f01f40a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f41592d0 0x7f01f40a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36928/0 (socket says 172.21.15.26:36928) 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f37fe640 1 -- 172.21.15.26:0/2321532100 learned_addr learned my addr 172.21.15.26:0/2321532100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f2ffd640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f406da40 0x7f01f4157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.020+0000 7f01f27fc640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f40c6780 0x7f01f41eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f37fe640 1 -- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f406da40 msgr2=0x7f01f4157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f37fe640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f406da40 0x7f01f4157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f37fe640 1 -- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f40c6780 msgr2=0x7f01f41eea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f37fe640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f40c6780 0x7f01f41eea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f37fe640 1 -- 172.21.15.26:0/2321532100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4151a40 con 0x7f01f41592d0 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f2ffd640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f406da40 0x7f01f4157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f27fc640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f40c6780 0x7f01f41eea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f37fe640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f41592d0 0x7f01f40a9c00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00efc0 tx=0x7f01ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01db7fe640 1 -- 172.21.15.26:0/2321532100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec018030 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01db7fe640 1 -- 172.21.15.26:0/2321532100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00cd30 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4151d20 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01db7fe640 1 -- 172.21.15.26:0/2321532100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec003bb0 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.021+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f41ee520 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.022+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f414fda0 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.022+0000 7f01db7fe640 1 -- 172.21.15.26:0/2321532100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01ec01b070 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.023+0000 7f01db7fe640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f01e006b500 0x7f01e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.023+0000 7f01db7fe640 1 -- 172.21.15.26:0/2321532100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f01ec08abd0 con 0x7f01f41592d0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.023+0000 7f01f2ffd640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f01e006b500 0x7f01e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.023+0000 7f01f2ffd640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f01e006b500 0x7f01e006d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01e4036000 tx=0x7f01e403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.028+0000 7f01db7fe640 1 -- 172.21.15.26:0/2321532100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f01f414fda0 con 0x7f01f41592d0 2024-03-06T07:21:29.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.247+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01f404f330 con 0x7f01f41592d0 2024-03-06T07:21:29.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.248+0000 7f01db7fe640 1 -- 172.21.15.26:0/2321532100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4985 (secure 0 0 0) 0x7f01ec010040 con 0x7f01f41592d0 2024-03-06T07:21:29.251 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:29.251 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:20.118951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4910,"qdb_cluster":[4910]},"id":2}]} 2024-03-06T07:21:29.251 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 22 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f01e006b500 msgr2=0x7f01e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f01e006b500 0x7f01e006d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01e4036000 tx=0x7f01e403d040 comp rx=0 tx=0).stop 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f41592d0 msgr2=0x7f01f40a9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f41592d0 0x7f01f40a9c00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00efc0 tx=0x7f01ec00c490 comp rx=0 tx=0).stop 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 shutdown_connections 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f01e006b500 0x7f01e006d9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01f41592d0 0x7f01f40a9c00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f40c6780 0x7f01f41eea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 --2- 172.21.15.26:0/2321532100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f406da40 0x7f01f4157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 >> 172.21.15.26:0/2321532100 conn(0x7f01f40aa630 msgr2=0x7f01f40c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 shutdown_connections 2024-03-06T07:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.250+0000 7f01f94e0640 1 -- 172.21.15.26:0/2321532100 wait complete. 2024-03-06T07:21:29.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.362+0000 7f6d99bbf640 1 Processor -- start 2024-03-06T07:21:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.362+0000 7f6d99bbf640 1 -- start start 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d99bbf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d940c88c0 0x7f6d94155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d99bbf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d940c73d0 con 0x7f6d94158b60 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d99bbf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d940be2e0 con 0x7f6d9414f360 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d99bbf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d94159700 con 0x7f6d940c88c0 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d92ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d940c88c0 0x7f6d94155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d93fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6d94158b60 0x7f6d94158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45770/0 (socket says 172.21.15.26:45770) 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d93fff640 1 -- 172.21.15.26:0/681573333 learned_addr learned my addr 172.21.15.26:0/681573333 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.363+0000 7f6d927fc640 1 -- 172.21.15.26:0/681573333 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707406528 0 0) 0x7f6d940be2e0 con 0x7f6d9414f360 2024-03-06T07:21:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.364+0000 7f6d92ffd640 1 -- 172.21.15.26:0/681573333 >> v1:172.21.15.117:6789/0 conn(0x7f6d9414f360 legacy=0x7f6d940c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.364+0000 7f6d92ffd640 1 -- 172.21.15.26:0/681573333 >> v1:172.21.15.26:6789/0 conn(0x7f6d94158b60 legacy=0x7f6d94158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.364+0000 7f6d92ffd640 1 -- 172.21.15.26:0/681573333 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940c7670 con 0x7f6d940c88c0 2024-03-06T07:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.364+0000 7f6d92ffd640 1 --2- 172.21.15.26:0/681573333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d940c88c0 0x7f6d94155c10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6d880097a0 tx=0x7f6d8802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdee2855c588bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.364+0000 7f6d927fc640 1 -- 172.21.15.26:0/681573333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88037070 con 0x7f6d940c88c0 2024-03-06T07:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.364+0000 7f6d927fc640 1 -- 172.21.15.26:0/681573333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d88030710 con 0x7f6d940c88c0 2024-03-06T07:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.364+0000 7f6d927fc640 1 -- 172.21.15.26:0/681573333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88030ae0 con 0x7f6d940c88c0 2024-03-06T07:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.365+0000 7f6d99bbf640 1 -- 172.21.15.26:0/681573333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d940c88c0 msgr2=0x7f6d94155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.365+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/681573333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d940c88c0 0x7f6d94155c10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6d880097a0 tx=0x7f6d8802b710 comp rx=0 tx=0).stop 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.365+0000 7f6d99bbf640 1 -- 172.21.15.26:0/681573333 shutdown_connections 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.365+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/681573333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d940c88c0 0x7f6d94155c10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.365+0000 7f6d99bbf640 1 -- 172.21.15.26:0/681573333 >> 172.21.15.26:0/681573333 conn(0x7f6d940ac570 msgr2=0x7f6d940ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.365+0000 7f6d99bbf640 1 -- 172.21.15.26:0/681573333 shutdown_connections 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.365+0000 7f6d99bbf640 1 -- 172.21.15.26:0/681573333 wait complete. 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.366+0000 7f6d99bbf640 1 Processor -- start 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.366+0000 7f6d99bbf640 1 -- start start 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d99bbf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d940c88c0 0x7f6d940be4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d99bbf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9414f360 0x7f6d94153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d99bbf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d94158b60 0x7f6d94151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d99bbf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c7c30 con 0x7f6d9414f360 2024-03-06T07:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d99bbf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c5a50 con 0x7f6d940c88c0 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d99bbf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c5bd0 con 0x7f6d94158b60 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9414f360 0x7f6d94153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9414f360 0x7f6d94153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36944/0 (socket says 172.21.15.26:36944) 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 -- 172.21.15.26:0/552919387 learned_addr learned my addr 172.21.15.26:0/552919387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d937fe640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d940c88c0 0x7f6d940be4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d93fff640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d94158b60 0x7f6d94151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 -- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d94158b60 msgr2=0x7f6d94151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d94158b60 0x7f6d94151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 -- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d940c88c0 msgr2=0x7f6d940be4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d940c88c0 0x7f6d940be4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 -- 172.21.15.26:0/552919387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d941514c0 con 0x7f6d9414f360 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.367+0000 7f6d92ffd640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9414f360 0x7f6d94153fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6d880098c0 tx=0x7f6d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.368+0000 7f6d7bfff640 1 -- 172.21.15.26:0/552919387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88042070 con 0x7f6d9414f360 2024-03-06T07:21:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.368+0000 7f6d7bfff640 1 -- 172.21.15.26:0/552919387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d88032a40 con 0x7f6d9414f360 2024-03-06T07:21:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.368+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9414f950 con 0x7f6d9414f360 2024-03-06T07:21:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.368+0000 7f6d7bfff640 1 -- 172.21.15.26:0/552919387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88037040 con 0x7f6d9414f360 2024-03-06T07:21:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.368+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d94154500 con 0x7f6d9414f360 2024-03-06T07:21:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.369+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9404f330 con 0x7f6d9414f360 2024-03-06T07:21:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.369+0000 7f6d7bfff640 1 -- 172.21.15.26:0/552919387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d88045070 con 0x7f6d9414f360 2024-03-06T07:21:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.369+0000 7f6d7bfff640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d8006b500 0x7f6d8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.369+0000 7f6d7bfff640 1 -- 172.21.15.26:0/552919387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6d880ac0c0 con 0x7f6d9414f360 2024-03-06T07:21:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.370+0000 7f6d937fe640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d8006b500 0x7f6d8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.370+0000 7f6d937fe640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d8006b500 0x7f6d8006d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000f60 tx=0x7f6d84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.375+0000 7f6d7bfff640 1 -- 172.21.15.26:0/552919387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6d8807a8c0 con 0x7f6d9414f360 2024-03-06T07:21:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.591+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d940c8d50 con 0x7f6d9414f360 2024-03-06T07:21:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.592+0000 7f6d7bfff640 1 -- 172.21.15.26:0/552919387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4985 (secure 0 0 0) 0x7f6d8807e960 con 0x7f6d9414f360 2024-03-06T07:21:29.595 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:29.595 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:00.359941+0000","modified":"2024-03-06T07:21:20.118951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4910":{"gid":4910,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/2730813301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2730813301},{"type":"v1","addr":"172.21.15.117:6837","nonce":2730813301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4910,"qdb_cluster":[4910]},"id":2}]} 2024-03-06T07:21:29.595 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 22 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d8006b500 msgr2=0x7f6d8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d8006b500 0x7f6d8006d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000f60 tx=0x7f6d84009040 comp rx=0 tx=0).stop 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9414f360 msgr2=0x7f6d94153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9414f360 0x7f6d94153fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6d880098c0 tx=0x7f6d880046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 shutdown_connections 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d8006b500 0x7f6d8006d9c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d94158b60 0x7f6d94151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9414f360 0x7f6d94153fc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 --2- 172.21.15.26:0/552919387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d940c88c0 0x7f6d940be4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 >> 172.21.15.26:0/552919387 conn(0x7f6d940ac570 msgr2=0x7f6d9414d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 shutdown_connections 2024-03-06T07:21:29.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.594+0000 7f6d99bbf640 1 -- 172.21.15.26:0/552919387 wait complete. 2024-03-06T07:21:29.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.707+0000 7f69a1202640 1 Processor -- start 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.707+0000 7f69a1202640 1 -- start start 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f69a1202640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f699c06cc00 0x7f699c155990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f69a1202640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c0c52f0 con 0x7f699c0c9240 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f69a1202640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c0c55d0 con 0x7f699c14bad0 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f69a1202640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699c0c58b0 con 0x7f699c06cc00 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f699b59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f699c06cc00 0x7f699c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f699a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f699c0c9240 0x7f699c0c7830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45780/0 (socket says 172.21.15.26:45780) 2024-03-06T07:21:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f699a59c640 1 -- 172.21.15.26:0/652677845 learned_addr learned my addr 172.21.15.26:0/652677845 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f699b59e640 1 -- 172.21.15.26:0/652677845 >> v1:172.21.15.117:6789/0 conn(0x7f699c14bad0 legacy=0x7f699c0cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f699b59e640 1 -- 172.21.15.26:0/652677845 >> v1:172.21.15.26:6789/0 conn(0x7f699c0c9240 legacy=0x7f699c0c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.708+0000 7f699b59e640 1 -- 172.21.15.26:0/652677845 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0c7310 con 0x7f699c06cc00 2024-03-06T07:21:29.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.709+0000 7f699b59e640 1 --2- 172.21.15.26:0/652677845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06cc00 0x7f699c155990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f698c009d90 tx=0x7f698c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceb9f512ea697a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.709+0000 7f6999d9b640 1 -- 172.21.15.26:0/652677845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c039070 con 0x7f699c06cc00 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.709+0000 7f6999d9b640 1 -- 172.21.15.26:0/652677845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c02cc60 con 0x7f699c06cc00 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.709+0000 7f6999d9b640 1 -- 172.21.15.26:0/652677845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c034660 con 0x7f699c06cc00 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.710+0000 7f69a1202640 1 -- 172.21.15.26:0/652677845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06cc00 msgr2=0x7f699c155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.710+0000 7f69a1202640 1 --2- 172.21.15.26:0/652677845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06cc00 0x7f699c155990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f698c009d90 tx=0x7f698c02beb0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.710+0000 7f69a1202640 1 -- 172.21.15.26:0/652677845 shutdown_connections 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.710+0000 7f69a1202640 1 --2- 172.21.15.26:0/652677845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06cc00 0x7f699c155990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.710+0000 7f69a1202640 1 -- 172.21.15.26:0/652677845 >> 172.21.15.26:0/652677845 conn(0x7f699c072b10 msgr2=0x7f699c072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.710+0000 7f69a1202640 1 -- 172.21.15.26:0/652677845 shutdown_connections 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.710+0000 7f69a1202640 1 -- 172.21.15.26:0/652677845 wait complete. 2024-03-06T07:21:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.711+0000 7f69a1202640 1 Processor -- start 2024-03-06T07:21:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.711+0000 7f69a1202640 1 -- start start 2024-03-06T07:21:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f69a1202640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c06cc00 0x7f699c0c0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f69a1202640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c0c9240 0x7f699c13f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f69a1202640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c14bad0 0x7f699c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f69a1202640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0cb600 con 0x7f699c0c9240 2024-03-06T07:21:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f69a1202640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0cb480 con 0x7f699c06cc00 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f69a1202640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0c7f40 con 0x7f699c14bad0 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c0c9240 0x7f699c13f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c0c9240 0x7f699c13f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36956/0 (socket says 172.21.15.26:36956) 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 -- 172.21.15.26:0/2301131175 learned_addr learned my addr 172.21.15.26:0/2301131175 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699ad9d640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c06cc00 0x7f699c0c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699b59e640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c14bad0 0x7f699c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 -- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c14bad0 msgr2=0x7f699c13f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c14bad0 0x7f699c13f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 -- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c06cc00 msgr2=0x7f699c0c0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c06cc00 0x7f699c0c0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 -- 172.21.15.26:0/2301131175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0c37f0 con 0x7f699c0c9240 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699b59e640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c14bad0 0x7f699c13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f699a59c640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c0c9240 0x7f699c13f0b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f699000c4d0 tx=0x7f699000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.712+0000 7f69837fe640 1 -- 172.21.15.26:0/2301131175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990017070 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.713+0000 7f69837fe640 1 -- 172.21.15.26:0/2301131175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6990005ce0 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.713+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c0c3ad0 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.713+0000 7f69837fe640 1 -- 172.21.15.26:0/2301131175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990007270 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.713+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c0c0a40 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.714+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c0684b0 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.714+0000 7f69837fe640 1 -- 172.21.15.26:0/2301131175 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f699001a070 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.714+0000 7f69837fe640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.714+0000 7f69837fe640 1 -- 172.21.15.26:0/2301131175 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f69900895d0 con 0x7f699c0c9240 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.715+0000 7f699ad9d640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.715+0000 7f699ad9d640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6994001780 tx=0x7f6994009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.719+0000 7f69837fe640 1 -- 172.21.15.26:0/2301131175 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6990057cb0 con 0x7f699c0c9240 2024-03-06T07:21:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.899+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f699c0693c0 con 0x7f699c0c9240 2024-03-06T07:21:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.900+0000 7f69837fe640 1 -- 172.21.15.26:0/2301131175 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17085 (secure 0 0 0) 0x7f699005bd50 con 0x7f699c0c9240 2024-03-06T07:21:29.903 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:29.903 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":38,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:14.107392+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:58.241645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:59.328635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/2470383395":"2024-03-07T07:21:14.107365+0000","172.21.15.117:6834/2470383395":"2024-03-07T07:21:14.107365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 msgr2=0x7f698806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6994001780 tx=0x7f6994009040 comp rx=0 tx=0).stop 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c0c9240 msgr2=0x7f699c13f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c0c9240 0x7f699c13f0b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f699000c4d0 tx=0x7f699000c990 comp rx=0 tx=0).stop 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 shutdown_connections 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c14bad0 0x7f699c13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c0c9240 0x7f699c13f0b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 --2- 172.21.15.26:0/2301131175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c06cc00 0x7f699c0c0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 >> 172.21.15.26:0/2301131175 conn(0x7f699c072b10 msgr2=0x7f699c0c1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 shutdown_connections 2024-03-06T07:21:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:29.902+0000 7f69a1202640 1 -- 172.21.15.26:0/2301131175 wait complete. 2024-03-06T07:21:29.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:21:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.016+0000 7fa8c9aa4640 1 Processor -- start 2024-03-06T07:21:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.016+0000 7fa8c9aa4640 1 -- start start 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c9aa4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c406f550 0x7fa8c41593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c9aa4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa8c4072cd0 0x7fa8c406fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c9aa4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40733f0 con 0x7fa8c4152720 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c9aa4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c4155ca0 con 0x7fa8c4072cd0 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c9aa4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41561d0 con 0x7fa8c406f550 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c3fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa8c4072cd0 0x7fa8c406fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c37fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa8c4152720 0x7fa8c40c77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:45786/0 (socket says 172.21.15.26:45786) 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c406f550 0x7fa8c41593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c37fe640 1 -- 172.21.15.26:0/1028113938 learned_addr learned my addr 172.21.15.26:0/1028113938 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c2ffd640 1 -- 172.21.15.26:0/1028113938 >> v2:172.21.15.117:3300/0 conn(0x7fa8c4072cd0 msgr2=0x7fa8c406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c2ffd640 1 --2- 172.21.15.26:0/1028113938 >> v2:172.21.15.117:3300/0 conn(0x7fa8c4072cd0 0x7fa8c406fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c27fc640 1 -- 172.21.15.26:0/1028113938 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017101793 0 0) 0x7fa8c40733f0 con 0x7fa8c4152720 2024-03-06T07:21:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c2ffd640 1 -- 172.21.15.26:0/1028113938 >> v1:172.21.15.117:6789/0 conn(0x7fa8c4152720 legacy=0x7fa8c40c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c2ffd640 1 -- 172.21.15.26:0/1028113938 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4155ef0 con 0x7fa8c406f550 2024-03-06T07:21:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c2ffd640 1 --2- 172.21.15.26:0/1028113938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c406f550 0x7fa8c41593c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80097a0 tx=0x7fa8b802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884f50557d0b5094 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c27fc640 1 -- 172.21.15.26:0/1028113938 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80315a0 con 0x7fa8c406f550 2024-03-06T07:21:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c27fc640 1 -- 172.21.15.26:0/1028113938 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b8037070 con 0x7fa8c406f550 2024-03-06T07:21:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.018+0000 7fa8c27fc640 1 -- 172.21.15.26:0/1028113938 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8031960 con 0x7fa8c406f550 2024-03-06T07:21:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/1028113938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c406f550 msgr2=0x7fa8c41593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/1028113938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c406f550 0x7fa8c41593c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80097a0 tx=0x7fa8b802b840 comp rx=0 tx=0).stop 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/1028113938 shutdown_connections 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/1028113938 >> v2:172.21.15.117:3300/0 conn(0x7fa8c4072cd0 0x7fa8c406fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/1028113938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c406f550 0x7fa8c41593c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/1028113938 >> 172.21.15.26:0/1028113938 conn(0x7fa8c406ea40 msgr2=0x7fa8c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/1028113938 shutdown_connections 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.019+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/1028113938 wait complete. 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.020+0000 7fa8c9aa4640 1 Processor -- start 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.020+0000 7fa8c9aa4640 1 -- start start 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8c406f550 0x7fa8c4153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c4072cd0 0x7fa8c4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c4152720 0x7fa8c4154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4158ea0 con 0x7fa8c4072cd0 2024-03-06T07:21:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4158bc0 con 0x7fa8c406f550 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40c7ef0 con 0x7fa8c4152720 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c4072cd0 0x7fa8c4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c4072cd0 0x7fa8c4154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36964/0 (socket says 172.21.15.26:36964) 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 -- 172.21.15.26:0/3955799068 learned_addr learned my addr 172.21.15.26:0/3955799068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c37fe640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8c406f550 0x7fa8c4153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c3fff640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c4152720 0x7fa8c4154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 -- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c4152720 msgr2=0x7fa8c4154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c4152720 0x7fa8c4154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 -- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8c406f550 msgr2=0x7fa8c4153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8c406f550 0x7fa8c4153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 -- 172.21.15.26:0/3955799068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c414b500 con 0x7fa8c4072cd0 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c2ffd640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c4072cd0 0x7fa8c4154400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80097a0 tx=0x7fa8b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8abfff640 1 -- 172.21.15.26:0/3955799068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8004230 con 0x7fa8c4072cd0 2024-03-06T07:21:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8abfff640 1 -- 172.21.15.26:0/3955799068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80043d0 con 0x7fa8c4072cd0 2024-03-06T07:21:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c41eb6e0 con 0x7fa8c4072cd0 2024-03-06T07:21:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8abfff640 1 -- 172.21.15.26:0/3955799068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003850 con 0x7fa8c4072cd0 2024-03-06T07:21:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.021+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c40c3790 con 0x7fa8c4072cd0 2024-03-06T07:21:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.022+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c404f330 con 0x7fa8c4072cd0 2024-03-06T07:21:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.023+0000 7fa8abfff640 1 -- 172.21.15.26:0/3955799068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8b80039f0 con 0x7fa8c4072cd0 2024-03-06T07:21:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.023+0000 7fa8abfff640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.023+0000 7fa8abfff640 1 -- 172.21.15.26:0/3955799068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa8b802f030 con 0x7fa8c4072cd0 2024-03-06T07:21:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.023+0000 7fa8c37fe640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.024+0000 7fa8c37fe640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4007fd0 tx=0x7fa8b4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.028+0000 7fa8abfff640 1 -- 172.21.15.26:0/3955799068 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa8b8041070 con 0x7fa8c4072cd0 2024-03-06T07:21:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.251+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8c41513e0 con 0x7fa8c4072cd0 2024-03-06T07:21:30.253 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:21:30.252+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:21:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.253+0000 7fa8abfff640 1 -- 172.21.15.26:0/3955799068 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8b807a840 con 0x7fa8c4072cd0 2024-03-06T07:21:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 msgr2=0x7fa8b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4007fd0 tx=0x7fa8b4007de0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c4072cd0 msgr2=0x7fa8c4154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c4072cd0 0x7fa8c4154400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80097a0 tx=0x7fa8b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 shutdown_connections 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8c4152720 0x7fa8c4154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8c4072cd0 0x7fa8c4154400 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 --2- 172.21.15.26:0/3955799068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8c406f550 0x7fa8c4153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 >> 172.21.15.26:0/3955799068 conn(0x7fa8c406ea40 msgr2=0x7fa8c414bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 shutdown_connections 2024-03-06T07:21:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.255+0000 7fa8c9aa4640 1 -- 172.21.15.26:0/3955799068 wait complete. 2024-03-06T07:21:30.258 INFO:tasks.ceph.mds.d.smithi117.stderr: -401> 2024-03-06T07:21:20.315+0000 7f14111e5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:30.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:21:30.282 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:21:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.370+0000 7f26af22a640 1 Processor -- start 2024-03-06T07:21:30.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.370+0000 7f26af22a640 1 -- start start 2024-03-06T07:21:30.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26af22a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26a80c4930 0x7f26a80c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26af22a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 0x7f26a80c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26af22a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a8155300 con 0x7f26a80c21c0 2024-03-06T07:21:30.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26af22a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a81555e0 con 0x7f26a80c4930 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26af22a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a81558c0 con 0x7f26a80c4d10 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26acfc6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26a80c4930 0x7f26a80c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26acfc6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26a80c4930 0x7f26a80c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36972/0 (socket says 172.21.15.26:36972) 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26acfc6640 1 -- 172.21.15.26:0/1135334790 learned_addr learned my addr 172.21.15.26:0/1135334790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26acfc6640 1 -- 172.21.15.26:0/1135334790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 msgr2=0x7f26a80c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26acfc6640 1 --2- 172.21.15.26:0/1135334790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 0x7f26a80c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26acfc6640 1 -- 172.21.15.26:0/1135334790 >> v1:172.21.15.117:6789/0 conn(0x7f26a80c21c0 legacy=0x7f26a81587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.371+0000 7f26acfc6640 1 -- 172.21.15.26:0/1135334790 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a8155ba0 con 0x7f26a80c4930 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26acfc6640 1 --2- 172.21.15.26:0/1135334790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a80c7ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f269c009870 tx=0x7f269c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9315f2803f723d10 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26a77fe640 1 -- 172.21.15.26:0/1135334790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0317c0 con 0x7f26a80c4930 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26a77fe640 1 -- 172.21.15.26:0/1135334790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c037070 con 0x7f26a80c4930 2024-03-06T07:21:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26a77fe640 1 -- 172.21.15.26:0/1135334790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c031b80 con 0x7f26a80c4930 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26af22a640 1 -- 172.21.15.26:0/1135334790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 msgr2=0x7f26a80c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26af22a640 1 --2- 172.21.15.26:0/1135334790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a80c7ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f269c009870 tx=0x7f269c02b860 comp rx=0 tx=0).stop 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26af22a640 1 -- 172.21.15.26:0/1135334790 shutdown_connections 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26af22a640 1 --2- 172.21.15.26:0/1135334790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 0x7f26a80c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26af22a640 1 --2- 172.21.15.26:0/1135334790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a80c7ab0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26af22a640 1 -- 172.21.15.26:0/1135334790 >> 172.21.15.26:0/1135334790 conn(0x7f26a80aa5c0 msgr2=0x7f26a80aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.372+0000 7f26af22a640 1 -- 172.21.15.26:0/1135334790 shutdown_connections 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.373+0000 7f26af22a640 1 -- 172.21.15.26:0/1135334790 wait complete. 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 Processor -- start 2024-03-06T07:21:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 -- start start 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a80c21c0 0x7f26a81588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a81e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 0x7f26a81e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80c2ac0 con 0x7f26a80c4930 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a81560c0 con 0x7f26a80c21c0 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.374+0000 7f26af22a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80c2c40 con 0x7f26a80c4d10 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a81e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a81e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36978/0 (socket says 172.21.15.26:36978) 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 -- 172.21.15.26:0/614115353 learned_addr learned my addr 172.21.15.26:0/614115353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26acfc6640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a80c21c0 0x7f26a81588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26ad7c7640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 0x7f26a81e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 -- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 msgr2=0x7f26a81e0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 0x7f26a81e0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 -- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a80c21c0 msgr2=0x7f26a81588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a80c21c0 0x7f26a81588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 -- 172.21.15.26:0/614115353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a81dffe0 con 0x7f26a80c4930 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26acfc6640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a80c21c0 0x7f26a81588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a7fff640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a81e06a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f269400b440 tx=0x7f269400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a57fa640 1 -- 172.21.15.26:0/614115353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694015020 con 0x7f26a80c4930 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a57fa640 1 -- 172.21.15.26:0/614115353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26940118b0 con 0x7f26a80c4930 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a81515d0 con 0x7f26a80c4930 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26a57fa640 1 -- 172.21.15.26:0/614115353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694011ca0 con 0x7f26a80c4930 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.375+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a81ebc00 con 0x7f26a80c4930 2024-03-06T07:21:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.376+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a804f330 con 0x7f26a80c4930 2024-03-06T07:21:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.377+0000 7f26a57fa640 1 -- 172.21.15.26:0/614115353 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2694018020 con 0x7f26a80c4930 2024-03-06T07:21:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.377+0000 7f26a57fa640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f268c06b4d0 0x7f268c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.377+0000 7f26a57fa640 1 -- 172.21.15.26:0/614115353 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f269408a480 con 0x7f26a80c4930 2024-03-06T07:21:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.378+0000 7f26acfc6640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f268c06b4d0 0x7f268c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.378+0000 7f26acfc6640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f268c06b4d0 0x7f268c06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f269c036000 tx=0x7f269c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.382+0000 7f26a57fa640 1 -- 172.21.15.26:0/614115353 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f269401f020 con 0x7f26a80c4930 2024-03-06T07:21:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:30.601+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f26a80a9430 con 0x7f26a80c4930 2024-03-06T07:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.254+0000 7f26a57fa640 1 -- 172.21.15.26:0/614115353 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f269400d040 con 0x7f26a80c4930 2024-03-06T07:21:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.256+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f268c06b4d0 msgr2=0x7f268c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.256+0000 7f26af22a640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f268c06b4d0 0x7f268c06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f269c036000 tx=0x7f269c03d040 comp rx=0 tx=0).stop 2024-03-06T07:21:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.256+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 msgr2=0x7f26a81e06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.256+0000 7f26af22a640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a81e06a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f269400b440 tx=0x7f269400b900 comp rx=0 tx=0).stop 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 shutdown_connections 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f268c06b4d0 0x7f268c06d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26a80c4d10 0x7f26a81e0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a80c4930 0x7f26a81e06a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 --2- 172.21.15.26:0/614115353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a80c21c0 0x7f26a81588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 >> 172.21.15.26:0/614115353 conn(0x7f26a80aa5c0 msgr2=0x7f26a80c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 shutdown_connections 2024-03-06T07:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.257+0000 7f26af22a640 1 -- 172.21.15.26:0/614115353 wait complete. 2024-03-06T07:21:31.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:21:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.380+0000 7f12e87d7640 1 Processor -- start 2024-03-06T07:21:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.380+0000 7f12e87d7640 1 -- start start 2024-03-06T07:21:31.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.381+0000 7f12e87d7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12e014f180 0x7f12e00c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:31.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.381+0000 7f12e87d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e01566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:31.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.381+0000 7f12e87d7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00c8f60 con 0x7f12e00ca000 2024-03-06T07:21:31.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.381+0000 7f12e87d7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e0156c00 con 0x7f12e014f180 2024-03-06T07:21:31.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.381+0000 7f12e87d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00c9240 con 0x7f12e01551d0 2024-03-06T07:21:31.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e5d72640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e01566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:31.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e5d72640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e01566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:44730/0 (socket says 172.21.15.26:44730) 2024-03-06T07:21:31.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e6573640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12e014f180 0x7f12e00c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e5d72640 1 -- 172.21.15.26:0/2671918977 learned_addr learned my addr 172.21.15.26:0/2671918977 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e5d72640 1 -- 172.21.15.26:0/2671918977 >> v2:172.21.15.117:3300/0 conn(0x7f12e014f180 msgr2=0x7f12e00c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e5d72640 1 --2- 172.21.15.26:0/2671918977 >> v2:172.21.15.117:3300/0 conn(0x7f12e014f180 0x7f12e00c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e5d72640 1 -- 172.21.15.26:0/2671918977 >> v1:172.21.15.117:6789/0 conn(0x7f12e00ca000 legacy=0x7f12e006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e5d72640 1 -- 172.21.15.26:0/2671918977 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00c9520 con 0x7f12e01551d0 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.382+0000 7f12e6573640 1 --2- 172.21.15.26:0/2671918977 >> v2:172.21.15.117:3300/0 conn(0x7f12e014f180 0x7f12e00c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.383+0000 7f12e5d72640 1 --2- 172.21.15.26:0/2671918977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e01566c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009870 tx=0x7f12d002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45907ea90c9665ce server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.383+0000 7f12e5571640 1 -- 172.21.15.26:0/2671918977 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0034070 con 0x7f12e01551d0 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.383+0000 7f12e5571640 1 -- 172.21.15.26:0/2671918977 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d0030840 con 0x7f12e01551d0 2024-03-06T07:21:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.383+0000 7f12e5571640 1 -- 172.21.15.26:0/2671918977 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0030c10 con 0x7f12e01551d0 2024-03-06T07:21:31.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 -- 172.21.15.26:0/2671918977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 msgr2=0x7f12e01566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2671918977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e01566c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009870 tx=0x7f12d002b980 comp rx=0 tx=0).stop 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 -- 172.21.15.26:0/2671918977 shutdown_connections 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2671918977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e01566c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2671918977 >> v2:172.21.15.117:3300/0 conn(0x7f12e014f180 0x7f12e00c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 -- 172.21.15.26:0/2671918977 >> 172.21.15.26:0/2671918977 conn(0x7f12e0072d20 msgr2=0x7f12e00ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 -- 172.21.15.26:0/2671918977 shutdown_connections 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.384+0000 7f12e87d7640 1 -- 172.21.15.26:0/2671918977 wait complete. 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.385+0000 7f12e87d7640 1 Processor -- start 2024-03-06T07:21:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.385+0000 7f12e87d7640 1 -- start start 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.385+0000 7f12e87d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e00ca000 0x7f12e00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e87d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e014f180 0x7f12e014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e87d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e87d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e014f560 con 0x7f12e00ca000 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e87d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00c9ae0 con 0x7f12e014f180 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e87d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e014f6e0 con 0x7f12e01551d0 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e00ca000 0x7f12e00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e00ca000 0x7f12e00aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36990/0 (socket says 172.21.15.26:36990) 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 -- 172.21.15.26:0/2493290248 learned_addr learned my addr 172.21.15.26:0/2493290248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e5d72640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e014f180 0x7f12e014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6d74640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 -- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 msgr2=0x7f12e014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 -- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e014f180 msgr2=0x7f12e014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e014f180 0x7f12e014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 -- 172.21.15.26:0/2493290248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e01e2360 con 0x7f12e00ca000 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6d74640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e6573640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e00ca000 0x7f12e00aed60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00ec10 tx=0x7f12dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12d6ffd640 1 -- 172.21.15.26:0/2493290248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc017070 con 0x7f12e00ca000 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12d6ffd640 1 -- 172.21.15.26:0/2493290248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc00cc30 con 0x7f12e00ca000 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e01518a0 con 0x7f12e00ca000 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12d6ffd640 1 -- 172.21.15.26:0/2493290248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc013470 con 0x7f12e00ca000 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.386+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e00af2a0 con 0x7f12e00ca000 2024-03-06T07:21:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.387+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e004f330 con 0x7f12e00ca000 2024-03-06T07:21:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.388+0000 7f12d6ffd640 1 -- 172.21.15.26:0/2493290248 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12dc01a070 con 0x7f12e00ca000 2024-03-06T07:21:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.388+0000 7f12d6ffd640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12c406b4d0 0x7f12c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.388+0000 7f12d6ffd640 1 -- 172.21.15.26:0/2493290248 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f12dc08a600 con 0x7f12e00ca000 2024-03-06T07:21:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.388+0000 7f12e5d72640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12c406b4d0 0x7f12c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.389+0000 7f12e5d72640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12c406b4d0 0x7f12c406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f12d0036000 tx=0x7f12d003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:31.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.392+0000 7f12d6ffd640 1 -- 172.21.15.26:0/2493290248 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f12dc058c80 con 0x7f12e00ca000 2024-03-06T07:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:31.572+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12e00afe70 con 0x7f12e00ca000 2024-03-06T07:21:32.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.255+0000 7f12d6ffd640 1 -- 172.21.15.26:0/2493290248 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7f12dc05cd20 con 0x7f12e00ca000 2024-03-06T07:21:32.257 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12c406b4d0 msgr2=0x7f12c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12c406b4d0 0x7f12c406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f12d0036000 tx=0x7f12d003b040 comp rx=0 tx=0).stop 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e00ca000 msgr2=0x7f12e00aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e00ca000 0x7f12e00aed60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00ec10 tx=0x7f12dc00c520 comp rx=0 tx=0).stop 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 shutdown_connections 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12c406b4d0 0x7f12c406d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12e01551d0 0x7f12e014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e014f180 0x7f12e014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 --2- 172.21.15.26:0/2493290248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e00ca000 0x7f12e00aed60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 >> 172.21.15.26:0/2493290248 conn(0x7f12e0072d20 msgr2=0x7f12e00c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 shutdown_connections 2024-03-06T07:21:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.259+0000 7f12e87d7640 1 -- 172.21.15.26:0/2493290248 wait complete. 2024-03-06T07:21:32.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:21:32.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.386+0000 7fe2ad963640 1 Processor -- start 2024-03-06T07:21:32.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.386+0000 7fe2ad963640 1 -- start start 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.387+0000 7fe2ad963640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe2a813dc90 0x7fe2a8159230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.387+0000 7fe2ad963640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a813e070 con 0x7fe2a80c53f0 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.387+0000 7fe2ad963640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a813e420 con 0x7fe2a80c69f0 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.387+0000 7fe2ad963640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a806c190 con 0x7fe2a813dc90 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.387+0000 7fe2a67fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe2a80c53f0 0x7fe2a806b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45786/0 (socket says 172.21.15.26:45786) 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.387+0000 7fe2a67fc640 1 -- 172.21.15.26:0/2476868221 learned_addr learned my addr 172.21.15.26:0/2476868221 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.387+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/2476868221 >> v2:172.21.15.117:3300/0 conn(0x7fe2a813dc90 0x7fe2a8159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208969523 0 0) 0x7fe2a813e070 con 0x7fe2a80c53f0 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe294003520 con 0x7fe2a80c53f0 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2297287687 0 0) 0x7fe294003520 con 0x7fe2a80c53f0 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 >> v2:172.21.15.117:3300/0 conn(0x7fe2a813dc90 msgr2=0x7fe2a8159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 --2- 172.21.15.26:0/2476868221 >> v2:172.21.15.117:3300/0 conn(0x7fe2a813dc90 0x7fe2a8159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 >> v1:172.21.15.117:6789/0 conn(0x7fe2a80c69f0 legacy=0x7fe2a815fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a813e700 con 0x7fe2a80c53f0 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe290001670 con 0x7fe2a80c53f0 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 >> v1:172.21.15.26:6789/0 conn(0x7fe2a80c53f0 legacy=0x7fe2a806b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe294003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2940043b0 0x7fe294004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe294005120 0x7fe29400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29400c030 con 0x7fe2940039b0 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29400c1e0 con 0x7fe294005120 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29400c3e0 con 0x7fe2940043b0 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe294003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a67fc640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2940043b0 0x7fe294004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a77fe640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe294005120 0x7fe29400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a6ffd640 1 -- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2940043b0 msgr2=0x7fe294004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2940043b0 0x7fe294004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a6ffd640 1 -- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe294005120 msgr2=0x7fe29400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe294005120 0x7fe29400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a6ffd640 1 -- 172.21.15.26:0/2476868221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe294003520 con 0x7fe2940039b0 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a77fe640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe294005120 0x7fe29400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.388+0000 7fe2a77fe640 1 -- 172.21.15.26:0/2476868221 reap_dead start 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.389+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe294003d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ac0 tx=0x7fe294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e87e9a05feb0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.389+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0039070 con 0x7fe2940039b0 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.389+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a002c6b0 con 0x7fe2940039b0 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.389+0000 7fe2a5ffb640 1 -- 172.21.15.26:0/2476868221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0009690 con 0x7fe2940039b0 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.390+0000 7fe2ad963640 1 -- 172.21.15.26:0/2476868221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 msgr2=0x7fe294003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.390+0000 7fe2ad963640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe294003d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ac0 tx=0x7fe294005830 comp rx=0 tx=0).stop 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.390+0000 7fe2ad963640 1 -- 172.21.15.26:0/2476868221 shutdown_connections 2024-03-06T07:21:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.390+0000 7fe2ad963640 1 --2- 172.21.15.26:0/2476868221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe294003d90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.390+0000 7fe2ad963640 1 -- 172.21.15.26:0/2476868221 >> 172.21.15.26:0/2476868221 conn(0x7fe2a80caa70 msgr2=0x7fe2a80cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:32.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.390+0000 7fe2ad963640 1 -- 172.21.15.26:0/2476868221 shutdown_connections 2024-03-06T07:21:32.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.390+0000 7fe2ad963640 1 -- 172.21.15.26:0/2476868221 wait complete. 2024-03-06T07:21:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.391+0000 7fe2ad963640 1 Processor -- start 2024-03-06T07:21:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.391+0000 7fe2ad963640 1 -- start start 2024-03-06T07:21:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2ad963640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe2a8136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2ad963640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2a8155b70 0x7fe2a80ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2ad963640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2a80adee0 0x7fe2a81edba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2ad963640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80c61f0 con 0x7fe2940039b0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2ad963640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a806bde0 con 0x7fe2a8155b70 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2ad963640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80c8970 con 0x7fe2a80adee0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe2a8136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe2a8136930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37022/0 (socket says 172.21.15.26:37022) 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 -- 172.21.15.26:0/3976463745 learned_addr learned my addr 172.21.15.26:0/3976463745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 -- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2a80adee0 msgr2=0x7fe2a81edba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2a80adee0 0x7fe2a81edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 -- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2a8155b70 msgr2=0x7fe2a80ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2a8155b70 0x7fe2a80ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 -- 172.21.15.26:0/3976463745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80aef80 con 0x7fe2940039b0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2a6ffd640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe2a8136930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0037040 tx=0x7fe2a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe29ffff640 1 -- 172.21.15.26:0/3976463745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0042020 con 0x7fe2940039b0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe29ffff640 1 -- 172.21.15.26:0/3976463745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a003a040 con 0x7fe2940039b0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe29ffff640 1 -- 172.21.15.26:0/3976463745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0039070 con 0x7fe2940039b0 2024-03-06T07:21:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.392+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a80af260 con 0x7fe2940039b0 2024-03-06T07:21:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.393+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a80b0120 con 0x7fe2940039b0 2024-03-06T07:21:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.393+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a806c2d0 con 0x7fe2940039b0 2024-03-06T07:21:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.394+0000 7fe29ffff640 1 -- 172.21.15.26:0/3976463745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2a0045020 con 0x7fe2940039b0 2024-03-06T07:21:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.394+0000 7fe29ffff640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe28806b4d0 0x7fe28806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.394+0000 7fe29ffff640 1 -- 172.21.15.26:0/3976463745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fe2a00ab5a0 con 0x7fe2940039b0 2024-03-06T07:21:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.395+0000 7fe2a67fc640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe28806b4d0 0x7fe28806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.395+0000 7fe2a67fc640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe28806b4d0 0x7fe28806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe290001eb0 tx=0x7fe290001150 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:32.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.399+0000 7fe29ffff640 1 -- 172.21.15.26:0/3976463745 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe2a806c2d0 con 0x7fe2940039b0 2024-03-06T07:21:32.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:32.578+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2a804f330 con 0x7fe2940039b0 2024-03-06T07:21:33.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.257+0000 7fe29ffff640 1 -- 172.21.15.26:0/3976463745 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe2a004c070 con 0x7fe2940039b0 2024-03-06T07:21:33.259 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:21:33.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe28806b4d0 msgr2=0x7fe28806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe28806b4d0 0x7fe28806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe290001eb0 tx=0x7fe290001150 comp rx=0 tx=0).stop 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 msgr2=0x7fe2a8136930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe2a8136930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0037040 tx=0x7fe2a0004730 comp rx=0 tx=0).stop 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 shutdown_connections 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe28806b4d0 0x7fe28806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2a80adee0 0x7fe2a81edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2a8155b70 0x7fe2a80ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 --2- 172.21.15.26:0/3976463745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2940039b0 0x7fe2a8136930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 >> 172.21.15.26:0/3976463745 conn(0x7fe2a80caa70 msgr2=0x7fe2a813ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 shutdown_connections 2024-03-06T07:21:33.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.260+0000 7fe2ad963640 1 -- 172.21.15.26:0/3976463745 wait complete. 2024-03-06T07:21:33.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:21:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.383+0000 7f019b876640 1 Processor -- start 2024-03-06T07:21:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.383+0000 7f019b876640 1 -- start start 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f019b876640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01940ab310 0x7f01940ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f019b876640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01941538c0 0x7f01941494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f019b876640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0194154330 con 0x7f01940a97c0 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f019b876640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940c46a0 con 0x7f01941538c0 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f019b876640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0194153dc0 con 0x7f01940ab310 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0198e11640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01941538c0 0x7f01941494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0198e11640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01941538c0 0x7f01941494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37036/0 (socket says 172.21.15.26:37036) 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0199e13640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01940a97c0 0x7f01940c4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45792/0 (socket says 172.21.15.26:45792) 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0198e11640 1 -- 172.21.15.26:0/500571514 learned_addr learned my addr 172.21.15.26:0/500571514 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0199612640 1 --2- 172.21.15.26:0/500571514 >> v2:172.21.15.117:3300/0 conn(0x7f01940ab310 0x7f01940ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0198e11640 1 -- 172.21.15.26:0/500571514 >> v2:172.21.15.117:3300/0 conn(0x7f01940ab310 msgr2=0x7f01940ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0198e11640 1 --2- 172.21.15.26:0/500571514 >> v2:172.21.15.117:3300/0 conn(0x7f01940ab310 0x7f01940ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0198e11640 1 -- 172.21.15.26:0/500571514 >> v1:172.21.15.26:6789/0 conn(0x7f01940a97c0 legacy=0x7f01940c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0198e11640 1 -- 172.21.15.26:0/500571514 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940c5d30 con 0x7f01941538c0 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.384+0000 7f0193fff640 1 -- 172.21.15.26:0/500571514 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19510691 0 0) 0x7f0194154330 con 0x7f01940a97c0 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.385+0000 7f0199612640 1 --2- 172.21.15.26:0/500571514 >> v2:172.21.15.117:3300/0 conn(0x7f01940ab310 0x7f01940ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.385+0000 7f0198e11640 1 --2- 172.21.15.26:0/500571514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01941538c0 0x7f01941494a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f01940a9550 tx=0x7f018802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c57c76d1cc231 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.385+0000 7f0193fff640 1 -- 172.21.15.26:0/500571514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188035020 con 0x7f01941538c0 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.385+0000 7f0193fff640 1 -- 172.21.15.26:0/500571514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0188031870 con 0x7f01941538c0 2024-03-06T07:21:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.385+0000 7f0193fff640 1 -- 172.21.15.26:0/500571514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188031c40 con 0x7f01941538c0 2024-03-06T07:21:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 -- 172.21.15.26:0/500571514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01941538c0 msgr2=0x7f01941494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 --2- 172.21.15.26:0/500571514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01941538c0 0x7f01941494a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f01940a9550 tx=0x7f018802b910 comp rx=0 tx=0).stop 2024-03-06T07:21:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 -- 172.21.15.26:0/500571514 shutdown_connections 2024-03-06T07:21:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 --2- 172.21.15.26:0/500571514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01941538c0 0x7f01941494a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 --2- 172.21.15.26:0/500571514 >> v2:172.21.15.117:3300/0 conn(0x7f01940ab310 0x7f01940ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 -- 172.21.15.26:0/500571514 >> 172.21.15.26:0/500571514 conn(0x7f01940c6710 msgr2=0x7f01940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 -- 172.21.15.26:0/500571514 shutdown_connections 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.386+0000 7f019b876640 1 -- 172.21.15.26:0/500571514 wait complete. 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.387+0000 7f019b876640 1 Processor -- start 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.387+0000 7f019b876640 1 -- start start 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f019b876640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01940a97c0 0x7f019406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f019b876640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01940ab310 0x7f01941eab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f019b876640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01941538c0 0x7f01941ee840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f019b876640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940abc30 con 0x7f01940ab310 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f019b876640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940c62f0 con 0x7f01941538c0 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f019b876640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01940c5240 con 0x7f01940a97c0 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01940ab310 0x7f01941eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01940ab310 0x7f01941eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37046/0 (socket says 172.21.15.26:37046) 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 -- 172.21.15.26:0/2876766907 learned_addr learned my addr 172.21.15.26:0/2876766907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0199e13640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01941538c0 0x7f01941ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 -- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01940a97c0 msgr2=0x7f019406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0199612640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01940a97c0 0x7f019406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01940a97c0 0x7f019406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 -- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01941538c0 msgr2=0x7f01941ee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01941538c0 0x7f01941ee840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 -- 172.21.15.26:0/2876766907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01941ee080 con 0x7f01940ab310 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0199e13640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01941538c0 0x7f01941ee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0199612640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01940a97c0 0x7f019406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.388+0000 7f0198e11640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01940ab310 0x7f01941eab10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f018802b950 tx=0x7f01880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.389+0000 7f0191ffb640 1 -- 172.21.15.26:0/2876766907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188040070 con 0x7f01940ab310 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.389+0000 7f0191ffb640 1 -- 172.21.15.26:0/2876766907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018803a420 con 0x7f01940ab310 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.389+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019406d320 con 0x7f01940ab310 2024-03-06T07:21:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.389+0000 7f0191ffb640 1 -- 172.21.15.26:0/2876766907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188035040 con 0x7f01940ab310 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.389+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019406dee0 con 0x7f01940ab310 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.390+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019404f330 con 0x7f01940ab310 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.390+0000 7f0191ffb640 1 -- 172.21.15.26:0/2876766907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f018804b060 con 0x7f01940ab310 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.391+0000 7f0191ffb640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f018006b500 0x7f018006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.391+0000 7f0191ffb640 1 -- 172.21.15.26:0/2876766907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f01880abf90 con 0x7f01940ab310 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.391+0000 7f0199612640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f018006b500 0x7f018006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.391+0000 7f0199612640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f018006b500 0x7f018006d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f01941ebc40 tx=0x7f018c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.395+0000 7f0191ffb640 1 -- 172.21.15.26:0/2876766907 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f018807afc0 con 0x7f01940ab310 2024-03-06T07:21:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.588+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f01940aa580 con 0x7f01940ab310 2024-03-06T07:21:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.591+0000 7f0191ffb640 1 -- 172.21.15.26:0/2876766907 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f018807a8c0 con 0x7f01940ab310 2024-03-06T07:21:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f018006b500 msgr2=0x7f018006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f018006b500 0x7f018006d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f01941ebc40 tx=0x7f018c008040 comp rx=0 tx=0).stop 2024-03-06T07:21:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01940ab310 msgr2=0x7f01941eab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01940ab310 0x7f01941eab10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f018802b950 tx=0x7f01880046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 shutdown_connections 2024-03-06T07:21:33.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f018006b500 0x7f018006d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01941538c0 0x7f01941ee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01940ab310 0x7f01941eab10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 --2- 172.21.15.26:0/2876766907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01940a97c0 0x7f019406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 >> 172.21.15.26:0/2876766907 conn(0x7f01940c6710 msgr2=0x7f01940c2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:33.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 shutdown_connections 2024-03-06T07:21:33.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.593+0000 7f019b876640 1 -- 172.21.15.26:0/2876766907 wait complete. 2024-03-06T07:21:33.605 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:21:33.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.707+0000 7fe0a6271640 1 Processor -- start 2024-03-06T07:21:33.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.707+0000 7fe0a6271640 1 -- start start 2024-03-06T07:21:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a6271640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe0a0152730 0x7fe0a00c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a6271640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0a00c3750 0x7fe0a0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a6271640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a0152b10 con 0x7fe0a00c6710 2024-03-06T07:21:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a6271640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a0156110 con 0x7fe0a00c3750 2024-03-06T07:21:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a6271640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a0158ad0 con 0x7fe0a0152730 2024-03-06T07:21:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a480e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0a00c3750 0x7fe0a0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a480e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0a00c3750 0x7fe0a0155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37052/0 (socket says 172.21.15.26:37052) 2024-03-06T07:21:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a480e640 1 -- 172.21.15.26:0/3841282318 learned_addr learned my addr 172.21.15.26:0/3841282318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a480e640 1 -- 172.21.15.26:0/3841282318 >> v2:172.21.15.117:3300/0 conn(0x7fe0a0152730 msgr2=0x7fe0a00c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a480e640 1 --2- 172.21.15.26:0/3841282318 >> v2:172.21.15.117:3300/0 conn(0x7fe0a0152730 0x7fe0a00c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a480e640 1 -- 172.21.15.26:0/3841282318 >> v1:172.21.15.117:6789/0 conn(0x7fe0a00c6710 legacy=0x7fe0a01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.708+0000 7fe0a480e640 1 -- 172.21.15.26:0/3841282318 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0158db0 con 0x7fe0a00c3750 2024-03-06T07:21:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.709+0000 7fe0a480e640 1 --2- 172.21.15.26:0/3841282318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a00c3750 0x7fe0a0155bd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0073290 tx=0x7fe09802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d9b0240a1261e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.709+0000 7fe09effd640 1 -- 172.21.15.26:0/3841282318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09802c9f0 con 0x7fe0a00c3750 2024-03-06T07:21:33.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.709+0000 7fe09effd640 1 -- 172.21.15.26:0/3841282318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09802cb90 con 0x7fe0a00c3750 2024-03-06T07:21:33.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.709+0000 7fe09effd640 1 -- 172.21.15.26:0/3841282318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980343f0 con 0x7fe0a00c3750 2024-03-06T07:21:33.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 -- 172.21.15.26:0/3841282318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a00c3750 msgr2=0x7fe0a0155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3841282318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a00c3750 0x7fe0a0155bd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0073290 tx=0x7fe09802be50 comp rx=0 tx=0).stop 2024-03-06T07:21:33.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 -- 172.21.15.26:0/3841282318 shutdown_connections 2024-03-06T07:21:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3841282318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a00c3750 0x7fe0a0155bd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3841282318 >> v2:172.21.15.117:3300/0 conn(0x7fe0a0152730 0x7fe0a00c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 -- 172.21.15.26:0/3841282318 >> 172.21.15.26:0/3841282318 conn(0x7fe0a00aa5c0 msgr2=0x7fe0a00aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 -- 172.21.15.26:0/3841282318 shutdown_connections 2024-03-06T07:21:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.710+0000 7fe0a6271640 1 -- 172.21.15.26:0/3841282318 wait complete. 2024-03-06T07:21:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.711+0000 7fe0a6271640 1 Processor -- start 2024-03-06T07:21:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.711+0000 7fe0a6271640 1 -- start start 2024-03-06T07:21:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a6271640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a00c3750 0x7fe0a01eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a6271640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a00c6710 0x7fe0a0149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a6271640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a0152730 0x7fe0a01eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a6271640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a01586f0 con 0x7fe0a0152730 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a6271640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0159370 con 0x7fe0a00c6710 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a6271640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0158870 con 0x7fe0a00c3750 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a0152730 0x7fe0a01eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a0152730 0x7fe0a01eaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37060/0 (socket says 172.21.15.26:37060) 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 -- 172.21.15.26:0/3909781946 learned_addr learned my addr 172.21.15.26:0/3909781946 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe09f7fe640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a00c6710 0x7fe0a0149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 -- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a00c3750 msgr2=0x7fe0a01eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe09ffff640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a00c3750 0x7fe0a01eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a00c3750 0x7fe0a01eeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 -- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a00c6710 msgr2=0x7fe0a0149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a00c6710 0x7fe0a0149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 -- 172.21.15.26:0/3909781946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a006c840 con 0x7fe0a0152730 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe09f7fe640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a00c6710 0x7fe0a0149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a480e640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a0152730 0x7fe0a01eaf80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe098036040 tx=0x7fe098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe09cff9640 1 -- 172.21.15.26:0/3909781946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098043020 con 0x7fe0a0152730 2024-03-06T07:21:33.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe09cff9640 1 -- 172.21.15.26:0/3909781946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09803e450 con 0x7fe0a0152730 2024-03-06T07:21:33.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a0151b20 con 0x7fe0a0152730 2024-03-06T07:21:33.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.712+0000 7fe09cff9640 1 -- 172.21.15.26:0/3909781946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098009510 con 0x7fe0a0152730 2024-03-06T07:21:33.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.713+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a0150cf0 con 0x7fe0a0152730 2024-03-06T07:21:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.714+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a004f330 con 0x7fe0a0152730 2024-03-06T07:21:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.714+0000 7fe09cff9640 1 -- 172.21.15.26:0/3909781946 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe098046020 con 0x7fe0a0152730 2024-03-06T07:21:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.714+0000 7fe09cff9640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08c06b4d0 0x7fe08c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.715+0000 7fe09cff9640 1 -- 172.21.15.26:0/3909781946 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe0980ac970 con 0x7fe0a0152730 2024-03-06T07:21:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.715+0000 7fe09ffff640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08c06b4d0 0x7fe08c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.715+0000 7fe09ffff640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08c06b4d0 0x7fe08c06d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe09400a810 tx=0x7fe094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.719+0000 7fe09cff9640 1 -- 172.21.15.26:0/3909781946 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe09804d030 con 0x7fe0a0152730 2024-03-06T07:21:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.900+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7fe0a00c34f0 con 0x7fe0a0152730 2024-03-06T07:21:33.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.900+0000 7fe09cff9640 1 -- 172.21.15.26:0/3909781946 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v14) v1 ==== 96+0+0 (secure 0 0 0) 0x7fe09807b9a0 con 0x7fe0a0152730 2024-03-06T07:21:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08c06b4d0 msgr2=0x7fe08c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08c06b4d0 0x7fe08c06d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe09400a810 tx=0x7fe094008040 comp rx=0 tx=0).stop 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a0152730 msgr2=0x7fe0a01eaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a0152730 0x7fe0a01eaf80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe098036040 tx=0x7fe098004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 shutdown_connections 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08c06b4d0 0x7fe08c06d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a0152730 0x7fe0a01eaf80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a00c6710 0x7fe0a0149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 --2- 172.21.15.26:0/3909781946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a00c3750 0x7fe0a01eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 >> 172.21.15.26:0/3909781946 conn(0x7fe0a00aa5c0 msgr2=0x7fe0a0153850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 shutdown_connections 2024-03-06T07:21:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:33.902+0000 7fe0a6271640 1 -- 172.21.15.26:0/3909781946 wait complete. 2024-03-06T07:21:33.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop' 2024-03-06T07:21:34.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.014+0000 7f336869d640 1 Processor -- start 2024-03-06T07:21:34.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.014+0000 7f336869d640 1 -- start start 2024-03-06T07:21:34.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f336869d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33601592d0 0x7f33601552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f336869d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3360152d40 con 0x7f33600ac260 2024-03-06T07:21:34.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f336869d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33600c1b50 con 0x7f3360152960 2024-03-06T07:21:34.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f336869d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3360066e30 con 0x7f33601592d0 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3366c3a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f33600ac260 0x7f3360155810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45796/0 (socket says 172.21.15.26:45796) 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3366c3a640 1 -- 172.21.15.26:0/1541038074 learned_addr learned my addr 172.21.15.26:0/1541038074 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365c38640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33601592d0 0x7f33601552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259597728 0 0) 0x7f3360152d40 con 0x7f33600ac260 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f334c003520 con 0x7f33600ac260 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961365350 0 0) 0x7f334c003520 con 0x7f33600ac260 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33601592d0 msgr2=0x7f33601552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33601592d0 0x7f33601552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 >> v1:172.21.15.117:6789/0 conn(0x7f3360152960 legacy=0x7f336006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600c1dc0 con 0x7f33600ac260 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f33580019f0 con 0x7f33600ac260 2024-03-06T07:21:34.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 >> v1:172.21.15.26:6789/0 conn(0x7f33600ac260 legacy=0x7f3360155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.015+0000 7f3365437640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c0039e0 0x7f334c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3365437640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f334c004390 0x7f334c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3365437640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f334c004f80 0x7f334c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3360066e30 con 0x7f334c004f80 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c00bf00 con 0x7f334c0039e0 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c00c080 con 0x7f334c004390 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3366c3a640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f334c004f80 0x7f334c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3366439640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c0039e0 0x7f334c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3365c38640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f334c004390 0x7f334c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3366c3a640 1 -- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f334c004390 msgr2=0x7f334c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3366c3a640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f334c004390 0x7f334c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3366c3a640 1 -- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c0039e0 msgr2=0x7f334c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3366c3a640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c0039e0 0x7f334c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.016+0000 7f3366c3a640 1 -- 172.21.15.26:0/1541038074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c003520 con 0x7f334c004f80 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f3366c3a640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f334c004f80 0x7f334c00b9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f335800a850 tx=0x7f334c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0deca4081fa5f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800a4a0 con 0x7f334c004f80 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3358034650 con 0x7f334c004f80 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f3365437640 1 -- 172.21.15.26:0/1541038074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358034a40 con 0x7f334c004f80 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f336869d640 1 -- 172.21.15.26:0/1541038074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f334c004f80 msgr2=0x7f334c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f336869d640 1 --2- 172.21.15.26:0/1541038074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f334c004f80 0x7f334c00b9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f335800a850 tx=0x7f334c005830 comp rx=0 tx=0).stop 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f3366c3a640 1 -- 172.21.15.26:0/1541038074 reap_dead start 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f336869d640 1 -- 172.21.15.26:0/1541038074 shutdown_connections 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f336869d640 1 -- 172.21.15.26:0/1541038074 >> 172.21.15.26:0/1541038074 conn(0x7f33600c70b0 msgr2=0x7f33600c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f336869d640 1 -- 172.21.15.26:0/1541038074 shutdown_connections 2024-03-06T07:21:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.017+0000 7f336869d640 1 -- 172.21.15.26:0/1541038074 wait complete. 2024-03-06T07:21:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 Processor -- start 2024-03-06T07:21:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 -- start start 2024-03-06T07:21:34.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c004f80 0x7f3360121ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3360071f10 0x7f336014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f336014fc50 0x7f33601534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336006e3a0 con 0x7f3360071f10 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3360159760 con 0x7f334c004f80 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f336869d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336006e520 con 0x7f336014fc50 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f3365c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3360071f10 0x7f336014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f3365c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3360071f10 0x7f336014f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37074/0 (socket says 172.21.15.26:37074) 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f3365c38640 1 -- 172.21.15.26:0/3428917787 learned_addr learned my addr 172.21.15.26:0/3428917787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f3366439640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c004f80 0x7f3360121ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.019+0000 7f3366c3a640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f336014fc50 0x7f33601534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3365c38640 1 -- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f336014fc50 msgr2=0x7f33601534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3365c38640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f336014fc50 0x7f33601534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3365c38640 1 -- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c004f80 msgr2=0x7f3360121ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3365c38640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c004f80 0x7f3360121ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3365c38640 1 -- 172.21.15.26:0/3428917787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33601211a0 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3366439640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c004f80 0x7f3360121ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3365c38640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3360071f10 0x7f336014f710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f33500119a0 tx=0x7f3350011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3356ffd640 1 -- 172.21.15.26:0/3428917787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335000faf0 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3356ffd640 1 -- 172.21.15.26:0/3428917787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335000fc90 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f3356ffd640 1 -- 172.21.15.26:0/3428917787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350014440 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33601500d0 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.020+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33600c40d0 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.021+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336004f330 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.021+0000 7f3356ffd640 1 -- 172.21.15.26:0/3428917787 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3350012040 con 0x7f3360071f10 2024-03-06T07:21:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.022+0000 7f3356ffd640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b500 0x7f334406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.022+0000 7f3356ffd640 1 -- 172.21.15.26:0/3428917787 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f335008a7a0 con 0x7f3360071f10 2024-03-06T07:21:34.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.022+0000 7f3366439640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b500 0x7f334406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.022+0000 7f3366439640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b500 0x7f334406d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f335c003700 tx=0x7f335c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.026+0000 7f3356ffd640 1 -- 172.21.15.26:0/3428917787 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3350019030 con 0x7f3360071f10 2024-03-06T07:21:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:34.275+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]} v 0) v1 -- 0x7f3360156270 con 0x7f3360071f10 2024-03-06T07:21:35.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.263+0000 7f3356ffd640 1 -- 172.21.15.26:0/3428917787 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]}]=0 v102) v1 ==== 143+0+0 (secure 0 0 0) 0x7f33500590e0 con 0x7f3360071f10 2024-03-06T07:21:35.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.265+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b500 msgr2=0x7f334406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.265+0000 7f336869d640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b500 0x7f334406d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f335c003700 tx=0x7f335c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:35.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.265+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3360071f10 msgr2=0x7f336014f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.265+0000 7f336869d640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3360071f10 0x7f336014f710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f33500119a0 tx=0x7f3350011e60 comp rx=0 tx=0).stop 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 shutdown_connections 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f334406b500 0x7f334406d9c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f336014fc50 0x7f33601534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3360071f10 0x7f336014f710 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 --2- 172.21.15.26:0/3428917787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f334c004f80 0x7f3360121ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 >> 172.21.15.26:0/3428917787 conn(0x7f33600c70b0 msgr2=0x7f33600aa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 shutdown_connections 2024-03-06T07:21:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.266+0000 7f336869d640 1 -- 172.21.15.26:0/3428917787 wait complete. 2024-03-06T07:21:35.280 INFO:tasks.cephfs_test_runner:That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. ... ok 2024-03-06T07:21:35.281 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_last_resort (tasks.cephfs.test_failover.TestClusterAffinity) That a standby with mds_join_fs set to another fs is still used if necessary. 2024-03-06T07:21:35.281 INFO:tasks.cephfs_test_runner:test_join_fs_last_resort (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:21:35.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.387+0000 7f4ccfd7a640 1 Processor -- start 2024-03-06T07:21:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.387+0000 7f4ccfd7a640 1 -- start start 2024-03-06T07:21:35.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4ccfd7a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4cc81527b0 0x7f4cc8159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4ccfd7a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4cc80c88b0 0x7f4cc80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4ccfd7a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc80c7c60 con 0x7f4cc80c7180 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4ccfd7a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc8159640 con 0x7f4cc80c88b0 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4ccfd7a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc80c8060 con 0x7f4cc81527b0 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4cce317640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4cc80c88b0 0x7f4cc80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4ccd315640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4cc80c7180 0x7f4cc8155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45798/0 (socket says 172.21.15.26:45798) 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.388+0000 7f4ccd315640 1 -- 172.21.15.26:0/1581295063 learned_addr learned my addr 172.21.15.26:0/1581295063 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4ccdb16640 1 --2- 172.21.15.26:0/1581295063 >> v2:172.21.15.117:3300/0 conn(0x7f4cc81527b0 0x7f4cc8159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4cce317640 1 -- 172.21.15.26:0/1581295063 >> v2:172.21.15.117:3300/0 conn(0x7f4cc81527b0 msgr2=0x7f4cc8159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4cce317640 1 --2- 172.21.15.26:0/1581295063 >> v2:172.21.15.117:3300/0 conn(0x7f4cc81527b0 0x7f4cc8159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4cce317640 1 -- 172.21.15.26:0/1581295063 >> v1:172.21.15.26:6789/0 conn(0x7f4cc80c7180 legacy=0x7f4cc8155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4cce317640 1 -- 172.21.15.26:0/1581295063 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80c76a0 con 0x7f4cc80c88b0 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4cccb14640 1 -- 172.21.15.26:0/1581295063 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912921717 0 0) 0x7f4cbc001110 con 0x7f4cc80c7180 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4ccdb16640 1 --2- 172.21.15.26:0/1581295063 >> v2:172.21.15.117:3300/0 conn(0x7f4cc81527b0 0x7f4cc8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4cce317640 1 --2- 172.21.15.26:0/1581295063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc80c88b0 0x7f4cc80c5330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80cba80 tx=0x7f4cb802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b887e582e1bdaeb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.389+0000 7f4cccb14640 1 -- 172.21.15.26:0/1581295063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8039070 con 0x7f4cc80c88b0 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4cccb14640 1 -- 172.21.15.26:0/1581295063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb802cbf0 con 0x7f4cc80c88b0 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4cccb14640 1 -- 172.21.15.26:0/1581295063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8034440 con 0x7f4cc80c88b0 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/1581295063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc80c88b0 msgr2=0x7f4cc80c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/1581295063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc80c88b0 0x7f4cc80c5330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80cba80 tx=0x7f4cb802be40 comp rx=0 tx=0).stop 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/1581295063 shutdown_connections 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/1581295063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc80c88b0 0x7f4cc80c5330 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/1581295063 >> v2:172.21.15.117:3300/0 conn(0x7f4cc81527b0 0x7f4cc8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/1581295063 >> 172.21.15.26:0/1581295063 conn(0x7f4cc80ac560 msgr2=0x7f4cc80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/1581295063 shutdown_connections 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.390+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/1581295063 wait complete. 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.391+0000 7f4ccfd7a640 1 Processor -- start 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.391+0000 7f4ccfd7a640 1 -- start start 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccfd7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc80c7180 0x7f4cc81eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccfd7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc80c88b0 0x7f4cc81ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccfd7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc81527b0 0x7f4cc80bf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccfd7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8155600 con 0x7f4cc81527b0 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccfd7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8155320 con 0x7f4cc80c7180 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccfd7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8152b90 con 0x7f4cc80c88b0 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc81527b0 0x7f4cc80bf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc81527b0 0x7f4cc80bf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37086/0 (socket says 172.21.15.26:37086) 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 -- 172.21.15.26:0/2211690044 learned_addr learned my addr 172.21.15.26:0/2211690044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccdb16640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc80c7180 0x7f4cc81eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccd315640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc80c88b0 0x7f4cc81ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 -- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc80c88b0 msgr2=0x7f4cc81ebf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc80c88b0 0x7f4cc81ebf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 -- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc80c7180 msgr2=0x7f4cc81eaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc80c7180 0x7f4cc81eaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 -- 172.21.15.26:0/2211690044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc81ec4c0 con 0x7f4cc81527b0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4ccd315640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc80c88b0 0x7f4cc81ebf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.392+0000 7f4cce317640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc81527b0 0x7f4cc80bf7d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009e80 tx=0x7f4cb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.393+0000 7f4cc67fc640 1 -- 172.21.15.26:0/2211690044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8044020 con 0x7f4cc81527b0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.393+0000 7f4cc67fc640 1 -- 172.21.15.26:0/2211690044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb803a040 con 0x7f4cc81527b0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.393+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc80be730 con 0x7f4cc81527b0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.393+0000 7f4cc67fc640 1 -- 172.21.15.26:0/2211690044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8039050 con 0x7f4cc81527b0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.393+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc80becf0 con 0x7f4cc81527b0 2024-03-06T07:21:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.394+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc80be4c0 con 0x7f4cc81527b0 2024-03-06T07:21:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.394+0000 7f4cc67fc640 1 -- 172.21.15.26:0/2211690044 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cb8003b90 con 0x7f4cc81527b0 2024-03-06T07:21:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.395+0000 7f4cc67fc640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cb406b4d0 0x7f4cb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.395+0000 7f4cc67fc640 1 -- 172.21.15.26:0/2211690044 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4cb80ac8c0 con 0x7f4cc81527b0 2024-03-06T07:21:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.395+0000 7f4ccdb16640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cb406b4d0 0x7f4cb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.395+0000 7f4ccdb16640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cb406b4d0 0x7f4cb406d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000aac0 tx=0x7f4cc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.399+0000 7f4cc67fc640 1 -- 172.21.15.26:0/2211690044 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4cb804e020 con 0x7f4cc81527b0 2024-03-06T07:21:35.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.581+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cc806ce00 con 0x7f4cc81527b0 2024-03-06T07:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.582+0000 7f4cc67fc640 1 -- 172.21.15.26:0/2211690044 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+13439 (secure 0 0 0) 0x7f4cb807b8f0 con 0x7f4cc81527b0 2024-03-06T07:21:35.584 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:35.584 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:33.256365+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/2730813301":"2024-03-07T07:21:30.252424+0000","172.21.15.117:6835/2730813301":"2024-03-07T07:21:30.252424+0000","172.21.15.117:6836/2470383395":"2024-03-07T07:21:14.107365+0000","172.21.15.117:6834/2470383395":"2024-03-07T07:21:14.107365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cb406b4d0 msgr2=0x7f4cb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cb406b4d0 0x7f4cb406d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000aac0 tx=0x7f4cc0008040 comp rx=0 tx=0).stop 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc81527b0 msgr2=0x7f4cc80bf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc81527b0 0x7f4cc80bf7d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009e80 tx=0x7f4cb8004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 shutdown_connections 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cb406b4d0 0x7f4cb406d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cc81527b0 0x7f4cc80bf7d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc80c88b0 0x7f4cc81ebf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 --2- 172.21.15.26:0/2211690044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc80c7180 0x7f4cc81eaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 >> 172.21.15.26:0/2211690044 conn(0x7f4cc80ac560 msgr2=0x7f4cc80ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 shutdown_connections 2024-03-06T07:21:35.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.584+0000 7f4ccfd7a640 1 -- 172.21.15.26:0/2211690044 wait complete. 2024-03-06T07:21:35.597 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:35.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.700+0000 7f87eb58c640 1 Processor -- start 2024-03-06T07:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.700+0000 7f87eb58c640 1 -- start start 2024-03-06T07:21:35.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.700+0000 7f87eb58c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87e40be150 0x7f87e4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.700+0000 7f87eb58c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e40c75e0 con 0x7f87e4158a10 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.700+0000 7f87eb58c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e40c78c0 con 0x7f87e414bcc0 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.700+0000 7f87eb58c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e40c7f10 con 0x7f87e40be150 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.701+0000 7f87e8b27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87e40be150 0x7f87e4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.701+0000 7f87e9b29640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87e4158a10 0x7f87e4158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45812/0 (socket says 172.21.15.26:45812) 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.701+0000 7f87e9b29640 1 -- 172.21.15.26:0/1532667508 learned_addr learned my addr 172.21.15.26:0/1532667508 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.701+0000 7f87e8b27640 1 -- 172.21.15.26:0/1532667508 >> v1:172.21.15.117:6789/0 conn(0x7f87e414bcc0 legacy=0x7f87e40c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.701+0000 7f87e8b27640 1 -- 172.21.15.26:0/1532667508 >> v1:172.21.15.26:6789/0 conn(0x7f87e4158a10 legacy=0x7f87e4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.701+0000 7f87e8b27640 1 -- 172.21.15.26:0/1532667508 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40c7ba0 con 0x7f87e40be150 2024-03-06T07:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87e8b27640 1 --2- 172.21.15.26:0/1532667508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40be150 0x7f87e4155a00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f87d40097a0 tx=0x7f87d402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc67d39cd759305 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87dbfff640 1 -- 172.21.15.26:0/1532667508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d402f810 con 0x7f87e40be150 2024-03-06T07:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87dbfff640 1 -- 172.21.15.26:0/1532667508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d402f9b0 con 0x7f87e40be150 2024-03-06T07:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87dbfff640 1 -- 172.21.15.26:0/1532667508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d402fd80 con 0x7f87e40be150 2024-03-06T07:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87eb58c640 1 -- 172.21.15.26:0/1532667508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40be150 msgr2=0x7f87e4155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87eb58c640 1 --2- 172.21.15.26:0/1532667508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40be150 0x7f87e4155a00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f87d40097a0 tx=0x7f87d402b9f0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87eb58c640 1 -- 172.21.15.26:0/1532667508 shutdown_connections 2024-03-06T07:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87eb58c640 1 --2- 172.21.15.26:0/1532667508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40be150 0x7f87e4155a00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87eb58c640 1 -- 172.21.15.26:0/1532667508 >> 172.21.15.26:0/1532667508 conn(0x7f87e40ac600 msgr2=0x7f87e40aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87eb58c640 1 -- 172.21.15.26:0/1532667508 shutdown_connections 2024-03-06T07:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.702+0000 7f87eb58c640 1 -- 172.21.15.26:0/1532667508 wait complete. 2024-03-06T07:21:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 Processor -- start 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 -- start start 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e40be150 0x7f87e414b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87e414bcc0 0x7f87e414a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e4158a10 0x7f87e414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c5ae0 con 0x7f87e4158a10 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e406d1d0 con 0x7f87e40be150 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87eb58c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c5c60 con 0x7f87e414bcc0 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87e9b29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e4158a10 0x7f87e414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87e9b29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e4158a10 0x7f87e414ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37094/0 (socket says 172.21.15.26:37094) 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.704+0000 7f87e9b29640 1 -- 172.21.15.26:0/4007459688 learned_addr learned my addr 172.21.15.26:0/4007459688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9328640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e40be150 0x7f87e414b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e8b27640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87e414bcc0 0x7f87e414a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9b29640 1 -- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87e414bcc0 msgr2=0x7f87e414a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9b29640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87e414bcc0 0x7f87e414a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9b29640 1 -- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e40be150 msgr2=0x7f87e414b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9b29640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e40be150 0x7f87e414b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9b29640 1 -- 172.21.15.26:0/4007459688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e41ec0e0 con 0x7f87e4158a10 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9328640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e40be150 0x7f87e414b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e8b27640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87e414bcc0 0x7f87e414a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87e9b29640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e4158a10 0x7f87e414ad50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00c3a0 tx=0x7f87dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87d9ffb640 1 -- 172.21.15.26:0/4007459688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc017070 con 0x7f87e4158a10 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87d9ffb640 1 -- 172.21.15.26:0/4007459688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc005ce0 con 0x7f87e4158a10 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e40c10d0 con 0x7f87e4158a10 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87d9ffb640 1 -- 172.21.15.26:0/4007459688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0072d0 con 0x7f87e4158a10 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.705+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e40be5e0 con 0x7f87e4158a10 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.706+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e404f330 con 0x7f87e4158a10 2024-03-06T07:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.706+0000 7f87d9ffb640 1 -- 172.21.15.26:0/4007459688 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87dc01a070 con 0x7f87e4158a10 2024-03-06T07:21:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.707+0000 7f87d9ffb640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.707+0000 7f87d9ffb640 1 -- 172.21.15.26:0/4007459688 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f87dc089230 con 0x7f87e4158a10 2024-03-06T07:21:35.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.707+0000 7f87e9328640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.708+0000 7f87e9328640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87e0001d70 tx=0x7f87e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.711+0000 7f87d9ffb640 1 -- 172.21.15.26:0/4007459688 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f87dc057c10 con 0x7f87e4158a10 2024-03-06T07:21:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.894+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87e40c1680 con 0x7f87e4158a10 2024-03-06T07:21:35.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.895+0000 7f87d9ffb640 1 -- 172.21.15.26:0/4007459688 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87dc05bcb0 con 0x7f87e4158a10 2024-03-06T07:21:35.897 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 msgr2=0x7f87c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87e0001d70 tx=0x7f87e0009040 comp rx=0 tx=0).stop 2024-03-06T07:21:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e4158a10 msgr2=0x7f87e414ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e4158a10 0x7f87e414ad50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00c3a0 tx=0x7f87dc00c860 comp rx=0 tx=0).stop 2024-03-06T07:21:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 shutdown_connections 2024-03-06T07:21:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e4158a10 0x7f87e414ad50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87e414bcc0 0x7f87e414a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 --2- 172.21.15.26:0/4007459688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e40be150 0x7f87e414b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 >> 172.21.15.26:0/4007459688 conn(0x7f87e40ac600 msgr2=0x7f87e4158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:35.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 shutdown_connections 2024-03-06T07:21:35.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:35.897+0000 7f87eb58c640 1 -- 172.21.15.26:0/4007459688 wait complete. 2024-03-06T07:21:35.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort' 2024-03-06T07:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.009+0000 7f6e8ac47640 1 Processor -- start 2024-03-06T07:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.009+0000 7f6e8ac47640 1 -- start start 2024-03-06T07:21:36.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e8ac47640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e8406da40 0x7f6e84155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e8ac47640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e840c6780 0x7f6e84158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e8ac47640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e84156050 con 0x7f6e841592d0 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e8ac47640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8406de20 con 0x7f6e840c6780 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e8ac47640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84072630 con 0x7f6e8406da40 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e83fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e840c6780 0x7f6e84158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e891e4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e841592d0 0x7f6e840c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45816/0 (socket says 172.21.15.26:45816) 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e891e4640 1 -- 172.21.15.26:0/2305838999 learned_addr learned my addr 172.21.15.26:0/2305838999 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e889e3640 1 --2- 172.21.15.26:0/2305838999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e8406da40 0x7f6e84155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e83fff640 1 -- 172.21.15.26:0/2305838999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e8406da40 msgr2=0x7f6e84155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e83fff640 1 --2- 172.21.15.26:0/2305838999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e8406da40 0x7f6e84155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e83fff640 1 -- 172.21.15.26:0/2305838999 >> v1:172.21.15.26:6789/0 conn(0x7f6e841592d0 legacy=0x7f6e840c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e83fff640 1 -- 172.21.15.26:0/2305838999 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e84072910 con 0x7f6e840c6780 2024-03-06T07:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.010+0000 7f6e837fe640 1 -- 172.21.15.26:0/2305838999 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688346774 0 0) 0x7f6e74001110 con 0x7f6e841592d0 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.011+0000 7f6e83fff640 1 --2- 172.21.15.26:0/2305838999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e840c6780 0x7f6e84158d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6e780097a0 tx=0x7f6e7802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da2edbb50c28ae server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.011+0000 7f6e837fe640 1 -- 172.21.15.26:0/2305838999 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780315a0 con 0x7f6e840c6780 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.011+0000 7f6e837fe640 1 -- 172.21.15.26:0/2305838999 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e78037070 con 0x7f6e840c6780 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.011+0000 7f6e837fe640 1 -- 172.21.15.26:0/2305838999 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78031960 con 0x7f6e840c6780 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 -- 172.21.15.26:0/2305838999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e840c6780 msgr2=0x7f6e84158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/2305838999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e840c6780 0x7f6e84158d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6e780097a0 tx=0x7f6e7802b840 comp rx=0 tx=0).stop 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 -- 172.21.15.26:0/2305838999 shutdown_connections 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/2305838999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e840c6780 0x7f6e84158d90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/2305838999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e8406da40 0x7f6e84155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 -- 172.21.15.26:0/2305838999 >> 172.21.15.26:0/2305838999 conn(0x7f6e840aa630 msgr2=0x7f6e840aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 -- 172.21.15.26:0/2305838999 shutdown_connections 2024-03-06T07:21:36.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.012+0000 7f6e8ac47640 1 -- 172.21.15.26:0/2305838999 wait complete. 2024-03-06T07:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.013+0000 7f6e8ac47640 1 Processor -- start 2024-03-06T07:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.013+0000 7f6e8ac47640 1 -- start start 2024-03-06T07:21:36.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e8ac47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8406da40 0x7f6e841edf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e8ac47640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e840c6780 0x7f6e841ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e8ac47640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e841592d0 0x7f6e841ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e8ac47640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840c3990 con 0x7f6e841592d0 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e8ac47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84158890 con 0x7f6e8406da40 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e8ac47640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840c3b10 con 0x7f6e840c6780 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e841592d0 0x7f6e841ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e841592d0 0x7f6e841ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37112/0 (socket says 172.21.15.26:37112) 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 -- 172.21.15.26:0/1030857972 learned_addr learned my addr 172.21.15.26:0/1030857972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e83fff640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e840c6780 0x7f6e841ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e889e3640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8406da40 0x7f6e841edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 -- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e840c6780 msgr2=0x7f6e841ebdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e840c6780 0x7f6e841ebdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 -- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8406da40 msgr2=0x7f6e841edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8406da40 0x7f6e841edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 -- 172.21.15.26:0/1030857972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840ac5d0 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e889e3640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8406da40 0x7f6e841edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.014+0000 7f6e891e4640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e841592d0 0x7f6e841ec310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c3c0 tx=0x7f6e7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.015+0000 7f6e817fa640 1 -- 172.21.15.26:0/1030857972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740074e0 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.015+0000 7f6e817fa640 1 -- 172.21.15.26:0/1030857972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74007680 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.015+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e841eea10 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.015+0000 7f6e817fa640 1 -- 172.21.15.26:0/1030857972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74004a20 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.015+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8406c9c0 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.016+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e841debe0 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.016+0000 7f6e817fa640 1 -- 172.21.15.26:0/1030857972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e74011070 con 0x7f6e841592d0 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.016+0000 7f6e817fa640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e7006b4d0 0x7f6e7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.017+0000 7f6e817fa640 1 -- 172.21.15.26:0/1030857972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6e74089b90 con 0x7f6e841592d0 2024-03-06T07:21:36.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.017+0000 7f6e889e3640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e7006b4d0 0x7f6e7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.017+0000 7f6e889e3640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e7006b4d0 0x7f6e7006d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e8406ccf0 tx=0x7f6e7c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.021+0000 7f6e817fa640 1 -- 172.21.15.26:0/1030857972 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6e84156050 con 0x7f6e841592d0 2024-03-06T07:21:36.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:36.270+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]} v 0) v1 -- 0x7f6e8404f330 con 0x7f6e841592d0 2024-03-06T07:21:37.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.268+0000 7f6e817fa640 1 -- 172.21.15.26:0/1030857972 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]}]=0 v104) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6e74058bc0 con 0x7f6e841592d0 2024-03-06T07:21:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e7006b4d0 msgr2=0x7f6e7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e7006b4d0 0x7f6e7006d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e8406ccf0 tx=0x7f6e7c00b040 comp rx=0 tx=0).stop 2024-03-06T07:21:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e841592d0 msgr2=0x7f6e841ec310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e841592d0 0x7f6e841ec310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c3c0 tx=0x7f6e7400c880 comp rx=0 tx=0).stop 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 shutdown_connections 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e7006b4d0 0x7f6e7006d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e841592d0 0x7f6e841ec310 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e840c6780 0x7f6e841ebdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 --2- 172.21.15.26:0/1030857972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8406da40 0x7f6e841edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 >> 172.21.15.26:0/1030857972 conn(0x7f6e840aa630 msgr2=0x7f6e84154900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.271+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 shutdown_connections 2024-03-06T07:21:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.272+0000 7f6e8ac47640 1 -- 172.21.15.26:0/1030857972 wait complete. 2024-03-06T07:21:37.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:21:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.396+0000 7faa75478640 1 Processor -- start 2024-03-06T07:21:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.396+0000 7faa75478640 1 -- start start 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa75478640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa700c67f0 0x7faa70155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa75478640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa700c4ef0 0x7faa70155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa75478640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa70152cb0 con 0x7faa701528d0 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa75478640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa70155ca0 con 0x7faa700c4ef0 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa75478640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700c38b0 con 0x7faa700c67f0 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa700c4ef0 0x7faa70155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa701528d0 0x7faa700724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36476/0 (socket says 172.21.15.26:36476) 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6effd640 1 -- 172.21.15.26:0/2944049698 learned_addr learned my addr 172.21.15.26:0/2944049698 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa67fff640 1 --2- 172.21.15.26:0/2944049698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa700c67f0 0x7faa70155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6f7fe640 1 -- 172.21.15.26:0/2944049698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa700c67f0 msgr2=0x7faa70155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6f7fe640 1 --2- 172.21.15.26:0/2944049698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa700c67f0 0x7faa70155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6f7fe640 1 -- 172.21.15.26:0/2944049698 >> v1:172.21.15.26:6789/0 conn(0x7faa701528d0 legacy=0x7faa700724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6f7fe640 1 -- 172.21.15.26:0/2944049698 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa701594f0 con 0x7faa700c4ef0 2024-03-06T07:21:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.397+0000 7faa6e7fc640 1 -- 172.21.15.26:0/2944049698 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947474065 0 0) 0x7faa70152cb0 con 0x7faa701528d0 2024-03-06T07:21:37.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.398+0000 7faa6f7fe640 1 --2- 172.21.15.26:0/2944049698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c4ef0 0x7faa70155760 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faa68009d90 tx=0x7faa6802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19627931ba7bfa25 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:37.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.398+0000 7faa6e7fc640 1 -- 172.21.15.26:0/2944049698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68039070 con 0x7faa700c4ef0 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.398+0000 7faa6e7fc640 1 -- 172.21.15.26:0/2944049698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6802cce0 con 0x7faa700c4ef0 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.398+0000 7faa6e7fc640 1 -- 172.21.15.26:0/2944049698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68034660 con 0x7faa700c4ef0 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 -- 172.21.15.26:0/2944049698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c4ef0 msgr2=0x7faa70155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 --2- 172.21.15.26:0/2944049698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c4ef0 0x7faa70155760 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faa68009d90 tx=0x7faa6802bf30 comp rx=0 tx=0).stop 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 -- 172.21.15.26:0/2944049698 shutdown_connections 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 --2- 172.21.15.26:0/2944049698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c4ef0 0x7faa70155760 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 --2- 172.21.15.26:0/2944049698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa700c67f0 0x7faa70155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 -- 172.21.15.26:0/2944049698 >> 172.21.15.26:0/2944049698 conn(0x7faa700aa6a0 msgr2=0x7faa700aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 -- 172.21.15.26:0/2944049698 shutdown_connections 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.399+0000 7faa75478640 1 -- 172.21.15.26:0/2944049698 wait complete. 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.400+0000 7faa75478640 1 Processor -- start 2024-03-06T07:21:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.400+0000 7faa75478640 1 -- start start 2024-03-06T07:21:37.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.400+0000 7faa75478640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa700c4ef0 0x7faa70156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa75478640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c67f0 0x7faa70157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa75478640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa701528d0 0x7faa701e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa75478640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700a8250 con 0x7faa700c67f0 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa75478640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700c3390 con 0x7faa700c4ef0 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa75478640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa701591a0 con 0x7faa701528d0 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c67f0 0x7faa70157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c67f0 0x7faa70157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45844/0 (socket says 172.21.15.26:45844) 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 -- 172.21.15.26:0/748750895 learned_addr learned my addr 172.21.15.26:0/748750895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa6f7fe640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa701528d0 0x7faa701e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa6effd640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa700c4ef0 0x7faa70156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 -- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa701528d0 msgr2=0x7faa701e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa701528d0 0x7faa701e7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 -- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa700c4ef0 msgr2=0x7faa70156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa700c4ef0 0x7faa70156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 -- 172.21.15.26:0/748750895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa701eed00 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa6f7fe640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa701528d0 0x7faa701e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa6effd640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa700c4ef0 0x7faa70156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa67fff640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c67f0 0x7faa70157430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faa5800d800 tx=0x7faa5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa677fe640 1 -- 172.21.15.26:0/748750895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800b6c0 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa677fe640 1 -- 172.21.15.26:0/748750895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58019070 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7014d2f0 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa677fe640 1 -- 172.21.15.26:0/748750895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800ba80 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.401+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7014cca0 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.403+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7004f330 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.403+0000 7faa677fe640 1 -- 172.21.15.26:0/748750895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa5800bc20 con 0x7faa700c67f0 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.403+0000 7faa677fe640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa5006b4d0 0x7faa5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.403+0000 7faa677fe640 1 -- 172.21.15.26:0/748750895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7faa5808a5d0 con 0x7faa700c67f0 2024-03-06T07:21:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.404+0000 7faa6effd640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa5006b4d0 0x7faa5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.404+0000 7faa6effd640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa5006b4d0 0x7faa5006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa60000f60 tx=0x7faa60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:37.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.408+0000 7faa677fe640 1 -- 172.21.15.26:0/748750895 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa58059560 con 0x7faa700c67f0 2024-03-06T07:21:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.593+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa70155670 con 0x7faa700c67f0 2024-03-06T07:21:37.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.597+0000 7faa677fe640 1 -- 172.21.15.26:0/748750895 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa58058e60 con 0x7faa700c67f0 2024-03-06T07:21:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa5006b4d0 msgr2=0x7faa5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa5006b4d0 0x7faa5006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa60000f60 tx=0x7faa60009040 comp rx=0 tx=0).stop 2024-03-06T07:21:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c67f0 msgr2=0x7faa70157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c67f0 0x7faa70157430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faa5800d800 tx=0x7faa5800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 shutdown_connections 2024-03-06T07:21:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa5006b4d0 0x7faa5006d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa701528d0 0x7faa701e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa700c67f0 0x7faa70157430 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 --2- 172.21.15.26:0/748750895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa700c4ef0 0x7faa70156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 >> 172.21.15.26:0/748750895 conn(0x7faa700aa6a0 msgr2=0x7faa700c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 shutdown_connections 2024-03-06T07:21:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.599+0000 7faa75478640 1 -- 172.21.15.26:0/748750895 wait complete. 2024-03-06T07:21:37.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:21:37.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:37.657 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:37.699 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:21:37.699 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:21:37.700 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:37.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:21:37.714 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:21:37.714 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:37.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:21:37.757 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:21:37.757 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:37.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.857+0000 7f95aed9a640 1 Processor -- start 2024-03-06T07:21:37.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.857+0000 7f95aed9a640 1 -- start start 2024-03-06T07:21:37.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95aed9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95a8072720 0x7f95a81564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95aed9a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95a80c3d60 0x7f95a80c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95aed9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a8156f90 con 0x7f95a8156a00 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95aed9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a80c38d0 con 0x7f95a8072720 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95aed9a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a8157270 con 0x7f95a80c3d60 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95acb36640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95a8072720 0x7f95a81564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95a7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95a8156a00 0x7f95a8150290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36492/0 (socket says 172.21.15.26:36492) 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95a7fff640 1 -- 172.21.15.26:0/4079004236 learned_addr learned my addr 172.21.15.26:0/4079004236 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95ad337640 1 --2- 172.21.15.26:0/4079004236 >> v2:172.21.15.117:3300/0 conn(0x7f95a80c3d60 0x7f95a80c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95acb36640 1 -- 172.21.15.26:0/4079004236 >> v2:172.21.15.117:3300/0 conn(0x7f95a80c3d60 msgr2=0x7f95a80c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95acb36640 1 --2- 172.21.15.26:0/4079004236 >> v2:172.21.15.117:3300/0 conn(0x7f95a80c3d60 0x7f95a80c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95acb36640 1 -- 172.21.15.26:0/4079004236 >> v1:172.21.15.26:6789/0 conn(0x7f95a8156a00 legacy=0x7f95a8150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95a77fe640 1 -- 172.21.15.26:0/4079004236 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505405948 0 0) 0x7f95a8156f90 con 0x7f95a8156a00 2024-03-06T07:21:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.858+0000 7f95acb36640 1 -- 172.21.15.26:0/4079004236 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80c3280 con 0x7f95a8072720 2024-03-06T07:21:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.859+0000 7f95acb36640 1 --2- 172.21.15.26:0/4079004236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8072720 0x7f95a81564c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f95a80ab560 tx=0x7f95a002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b0c39d416537 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.859+0000 7f95a77fe640 1 -- 172.21.15.26:0/4079004236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0039070 con 0x7f95a8072720 2024-03-06T07:21:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.859+0000 7f95a77fe640 1 -- 172.21.15.26:0/4079004236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a002cbf0 con 0x7f95a8072720 2024-03-06T07:21:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.859+0000 7f95a77fe640 1 -- 172.21.15.26:0/4079004236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0034440 con 0x7f95a8072720 2024-03-06T07:21:37.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 -- 172.21.15.26:0/4079004236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8072720 msgr2=0x7f95a81564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 --2- 172.21.15.26:0/4079004236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8072720 0x7f95a81564c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f95a80ab560 tx=0x7f95a002be40 comp rx=0 tx=0).stop 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 -- 172.21.15.26:0/4079004236 shutdown_connections 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 --2- 172.21.15.26:0/4079004236 >> v2:172.21.15.117:3300/0 conn(0x7f95a80c3d60 0x7f95a80c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 --2- 172.21.15.26:0/4079004236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8072720 0x7f95a81564c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 -- 172.21.15.26:0/4079004236 >> 172.21.15.26:0/4079004236 conn(0x7f95a80aa630 msgr2=0x7f95a80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 -- 172.21.15.26:0/4079004236 shutdown_connections 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.860+0000 7f95aed9a640 1 -- 172.21.15.26:0/4079004236 wait complete. 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.861+0000 7f95aed9a640 1 Processor -- start 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.861+0000 7f95aed9a640 1 -- start start 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95aed9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a8072720 0x7f95a8153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95aed9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80c3d60 0x7f95a8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95aed9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8156a00 0x7f95a81521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95aed9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a81509a0 con 0x7f95a8156a00 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95aed9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a806f8f0 con 0x7f95a8072720 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95aed9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8150b20 con 0x7f95a80c3d60 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8156a00 0x7f95a81521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8156a00 0x7f95a81521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45864/0 (socket says 172.21.15.26:45864) 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 -- 172.21.15.26:0/2003264605 learned_addr learned my addr 172.21.15.26:0/2003264605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95acb36640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a8072720 0x7f95a8153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95a7fff640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80c3d60 0x7f95a8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 -- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80c3d60 msgr2=0x7f95a8155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80c3d60 0x7f95a8155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 -- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a8072720 msgr2=0x7f95a8153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a8072720 0x7f95a8153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 -- 172.21.15.26:0/2003264605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8155fd0 con 0x7f95a8156a00 2024-03-06T07:21:37.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95a7fff640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80c3d60 0x7f95a8155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:37.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95ad337640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8156a00 0x7f95a81521e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f959800c540 tx=0x7f959800ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:37.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.862+0000 7f95a57fa640 1 -- 172.21.15.26:0/2003264605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598015070 con 0x7f95a8156a00 2024-03-06T07:21:37.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.863+0000 7f95a57fa640 1 -- 172.21.15.26:0/2003264605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598004220 con 0x7f95a8156a00 2024-03-06T07:21:37.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.863+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a81e8bb0 con 0x7f95a8156a00 2024-03-06T07:21:37.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.863+0000 7f95a57fa640 1 -- 172.21.15.26:0/2003264605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598003610 con 0x7f95a8156a00 2024-03-06T07:21:37.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.863+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a81e9170 con 0x7f95a8156a00 2024-03-06T07:21:37.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.864+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a81e9370 con 0x7f95a8156a00 2024-03-06T07:21:37.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.864+0000 7f95a57fa640 1 -- 172.21.15.26:0/2003264605 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95980037b0 con 0x7f95a8156a00 2024-03-06T07:21:37.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.864+0000 7f95a57fa640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959406b4d0 0x7f959406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:37.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.865+0000 7f95a57fa640 1 -- 172.21.15.26:0/2003264605 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f959808a230 con 0x7f95a8156a00 2024-03-06T07:21:37.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.865+0000 7f95acb36640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959406b4d0 0x7f959406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:37.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.865+0000 7f95acb36640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959406b4d0 0x7f959406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95a0036000 tx=0x7f95a0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:37.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:37.869+0000 7f95a57fa640 1 -- 172.21.15.26:0/2003264605 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95a81e9370 con 0x7f95a8156a00 2024-03-06T07:21:38.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.087+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95a804f330 con 0x7f95a8156a00 2024-03-06T07:21:38.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.088+0000 7f95a57fa640 1 -- 172.21.15.26:0/2003264605 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f9598059140 con 0x7f95a8156a00 2024-03-06T07:21:38.090 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:38.090 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3831184543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3831184543},{"type":"v1","addr":"172.21.15.26:6833","nonce":3831184543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2694023942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2694023942},{"type":"v1","addr":"172.21.15.26:6835","nonce":2694023942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5398,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3245601481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3245601481},{"type":"v1","addr":"172.21.15.117:6836","nonce":3245601481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3957788054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3957788054},{"type":"v1","addr":"172.21.15.117:6837","nonce":3957788054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[]} 2024-03-06T07:21:38.090 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 24 2024-03-06T07:21:38.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959406b4d0 msgr2=0x7f959406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959406b4d0 0x7f959406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95a0036000 tx=0x7f95a0030000 comp rx=0 tx=0).stop 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8156a00 msgr2=0x7f95a81521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8156a00 0x7f95a81521e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f959800c540 tx=0x7f959800ca00 comp rx=0 tx=0).stop 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 shutdown_connections 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959406b4d0 0x7f959406d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a8156a00 0x7f95a81521e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80c3d60 0x7f95a8155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 --2- 172.21.15.26:0/2003264605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a8072720 0x7f95a8153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 >> 172.21.15.26:0/2003264605 conn(0x7f95a80aa630 msgr2=0x7f95a80a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 shutdown_connections 2024-03-06T07:21:38.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.090+0000 7f95aed9a640 1 -- 172.21.15.26:0/2003264605 wait complete. 2024-03-06T07:21:38.103 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:21:38.103 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:21:38.103 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:21:38.103 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.104 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:21:38.104 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:21:38.104 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:21:38.104 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.104 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:21:38.104 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:21:38.105 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:21:38.105 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.105 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:21:38.105 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:21:38.105 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:21:38.105 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.112 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:21:38.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:21:38.113 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:21:38.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:21:38.116 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:21:38.116 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:21:38.117 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:21:38.117 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:21:38.155 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:21:38.155 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.156 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:21:38.156 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.159 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:21:38.159 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.160 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:21:38.160 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:21:38.233 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:21:38.233 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:21:38.244 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:21:38.244 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:21:38.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.265+0000 7f62d30d4640 1 Processor -- start 2024-03-06T07:21:38.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.265+0000 7f62d30d4640 1 -- start start 2024-03-06T07:21:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d30d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc0c0fa0 0x7f62cc153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d30d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62cc06d990 0x7f62cc06de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d30d4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc070550 con 0x7f62cc072f00 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d30d4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc06e360 con 0x7f62cc06d990 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d30d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc070830 con 0x7f62cc0c0fa0 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d0e70640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f62cc072f00 0x7f62cc0c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36502/0 (socket says 172.21.15.26:36502) 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d0e70640 1 -- 172.21.15.26:0/3505926832 learned_addr learned my addr 172.21.15.26:0/3505926832 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62d1671640 1 --2- 172.21.15.26:0/3505926832 >> v2:172.21.15.26:3300/0 conn(0x7f62cc06d990 0x7f62cc06de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.266+0000 7f62cbfff640 1 --2- 172.21.15.26:0/3505926832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc0c0fa0 0x7f62cc153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.267+0000 7f62cb7fe640 1 -- 172.21.15.26:0/3505926832 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135004158 0 0) 0x7f62cc070550 con 0x7f62cc072f00 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.267+0000 7f62cb7fe640 1 -- 172.21.15.26:0/3505926832 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62bc003520 con 0x7f62cc072f00 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.267+0000 7f62d1671640 1 -- 172.21.15.26:0/3505926832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc0c0fa0 msgr2=0x7f62cc153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.267+0000 7f62d1671640 1 --2- 172.21.15.26:0/3505926832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc0c0fa0 0x7f62cc153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.267+0000 7f62d1671640 1 -- 172.21.15.26:0/3505926832 >> v1:172.21.15.26:6789/0 conn(0x7f62cc072f00 legacy=0x7f62cc0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.267+0000 7f62d1671640 1 -- 172.21.15.26:0/3505926832 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc070b10 con 0x7f62cc06d990 2024-03-06T07:21:38.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.267+0000 7f62d1671640 1 --2- 172.21.15.26:0/3505926832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc06de20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009d90 tx=0x7f62c402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f7c9d3b3e7e4b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62cb7fe640 1 -- 172.21.15.26:0/3505926832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4039070 con 0x7f62cc06d990 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62cb7fe640 1 -- 172.21.15.26:0/3505926832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c402cc60 con 0x7f62cc06d990 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62cb7fe640 1 -- 172.21.15.26:0/3505926832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4034640 con 0x7f62cc06d990 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 -- 172.21.15.26:0/3505926832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 msgr2=0x7f62cc06de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 --2- 172.21.15.26:0/3505926832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc06de20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009d90 tx=0x7f62c402beb0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 -- 172.21.15.26:0/3505926832 shutdown_connections 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 --2- 172.21.15.26:0/3505926832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc06de20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 --2- 172.21.15.26:0/3505926832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc0c0fa0 0x7f62cc153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 -- 172.21.15.26:0/3505926832 >> 172.21.15.26:0/3505926832 conn(0x7f62cc06b1a0 msgr2=0x7f62cc06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 -- 172.21.15.26:0/3505926832 shutdown_connections 2024-03-06T07:21:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.268+0000 7f62d30d4640 1 -- 172.21.15.26:0/3505926832 wait complete. 2024-03-06T07:21:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.269+0000 7f62d30d4640 1 Processor -- start 2024-03-06T07:21:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.270+0000 7f62d30d4640 1 -- start start 2024-03-06T07:21:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.270+0000 7f62d30d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.270+0000 7f62d30d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc072f00 0x7f62cc148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.270+0000 7f62d30d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62cc0c0fa0 0x7f62cc1456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d30d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0c70d0 con 0x7f62cc06d990 2024-03-06T07:21:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d30d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0c6f50 con 0x7f62cc0c0fa0 2024-03-06T07:21:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d30d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc070df0 con 0x7f62cc072f00 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d1671640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62cc0c0fa0 0x7f62cc1456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc14e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45902/0 (socket says 172.21.15.26:45902) 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 -- 172.21.15.26:0/2617258930 learned_addr learned my addr 172.21.15.26:0/2617258930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62cbfff640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc072f00 0x7f62cc148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 -- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc072f00 msgr2=0x7f62cc148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc072f00 0x7f62cc148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 -- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62cc0c0fa0 msgr2=0x7f62cc1456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62cc0c0fa0 0x7f62cc1456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 -- 172.21.15.26:0/2617258930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc156e30 con 0x7f62cc06d990 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.271+0000 7f62d0e70640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc14e650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c430 tx=0x7f62c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.272+0000 7f62c97fa640 1 -- 172.21.15.26:0/2617258930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0017070 con 0x7f62cc06d990 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.273+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc156b50 con 0x7f62cc06d990 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.273+0000 7f62c97fa640 1 -- 172.21.15.26:0/2617258930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0005ce0 con 0x7f62cc06d990 2024-03-06T07:21:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.273+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc155c60 con 0x7f62cc06d990 2024-03-06T07:21:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.273+0000 7f62c97fa640 1 -- 172.21.15.26:0/2617258930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00072d0 con 0x7f62cc06d990 2024-03-06T07:21:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.274+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc04f330 con 0x7f62cc06d990 2024-03-06T07:21:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.274+0000 7f62c97fa640 1 -- 172.21.15.26:0/2617258930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62c001a070 con 0x7f62cc06d990 2024-03-06T07:21:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.274+0000 7f62c97fa640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62b006b4d0 0x7f62b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.274+0000 7f62c97fa640 1 -- 172.21.15.26:0/2617258930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f62c0089420 con 0x7f62cc06d990 2024-03-06T07:21:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.275+0000 7f62cbfff640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62b006b4d0 0x7f62b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.276+0000 7f62cbfff640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62b006b4d0 0x7f62b006d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0754e0 tx=0x7f62b801a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.281+0000 7f62c97fa640 1 -- 172.21.15.26:0/2617258930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f62c0057cf0 con 0x7f62cc06d990 2024-03-06T07:21:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:38.473+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f62cc0abf40 con 0x7f62cc06d990 2024-03-06T07:21:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.285+0000 7f62c97fa640 1 -- 172.21.15.26:0/2617258930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) v1 ==== 97+0+0 (secure 0 0 0) 0x7f62c005bd90 con 0x7f62cc06d990 2024-03-06T07:21:39.287 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:21:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62b006b4d0 msgr2=0x7f62b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62b006b4d0 0x7f62b006d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0754e0 tx=0x7f62b801a040 comp rx=0 tx=0).stop 2024-03-06T07:21:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 msgr2=0x7f62cc14e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc14e650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c430 tx=0x7f62c000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 shutdown_connections 2024-03-06T07:21:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62b006b4d0 0x7f62b006d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62cc0c0fa0 0x7f62cc1456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62cc072f00 0x7f62cc148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 --2- 172.21.15.26:0/2617258930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62cc06d990 0x7f62cc14e650 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 >> 172.21.15.26:0/2617258930 conn(0x7f62cc06b1a0 msgr2=0x7f62cc0c1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 shutdown_connections 2024-03-06T07:21:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.288+0000 7f62d30d4640 1 -- 172.21.15.26:0/2617258930 wait complete. 2024-03-06T07:21:39.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:21:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.410+0000 7fc6015a1640 1 Processor -- start 2024-03-06T07:21:39.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.410+0000 7fc6015a1640 1 -- start start 2024-03-06T07:21:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.411+0000 7fc6015a1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5fc06e490 0x7fc5fc160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.411+0000 7fc6015a1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc0bdc20 con 0x7fc5fc072e70 2024-03-06T07:21:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.411+0000 7fc6015a1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc06c8e0 con 0x7fc5fc073250 2024-03-06T07:21:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.411+0000 7fc6015a1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5fc06ce80 con 0x7fc5fc06e490 2024-03-06T07:21:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.411+0000 7fc5fb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5fc06e490 0x7fc5fc160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.411+0000 7fc5faffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc5fc072e70 0x7fc5fc0bd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36508/0 (socket says 172.21.15.26:36508) 2024-03-06T07:21:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.411+0000 7fc5faffd640 1 -- 172.21.15.26:0/3698776050 learned_addr learned my addr 172.21.15.26:0/3698776050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.412+0000 7fc5fb7fe640 1 -- 172.21.15.26:0/3698776050 >> v1:172.21.15.117:6789/0 conn(0x7fc5fc073250 legacy=0x7fc5fc0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.412+0000 7fc5fb7fe640 1 -- 172.21.15.26:0/3698776050 >> v1:172.21.15.26:6789/0 conn(0x7fc5fc072e70 legacy=0x7fc5fc0bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.412+0000 7fc5fb7fe640 1 -- 172.21.15.26:0/3698776050 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc0c1b00 con 0x7fc5fc06e490 2024-03-06T07:21:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.412+0000 7fc5f9ffb640 1 -- 172.21.15.26:0/3698776050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210295013 0 0) 0x7fc5fc0bdc20 con 0x7fc5fc072e70 2024-03-06T07:21:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.412+0000 7fc5fb7fe640 1 --2- 172.21.15.26:0/3698776050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc06e490 0x7fc5fc160cf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009d90 tx=0x7fc5ec02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc3d34d85be293f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc5f9ffb640 1 -- 172.21.15.26:0/3698776050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec039070 con 0x7fc5fc06e490 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc5f9ffb640 1 -- 172.21.15.26:0/3698776050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec02cbd0 con 0x7fc5fc06e490 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc5f9ffb640 1 -- 172.21.15.26:0/3698776050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec034440 con 0x7fc5fc06e490 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc6015a1640 1 -- 172.21.15.26:0/3698776050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc06e490 msgr2=0x7fc5fc160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3698776050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc06e490 0x7fc5fc160cf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009d90 tx=0x7fc5ec02be20 comp rx=0 tx=0).stop 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc6015a1640 1 -- 172.21.15.26:0/3698776050 shutdown_connections 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3698776050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc06e490 0x7fc5fc160cf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc6015a1640 1 -- 172.21.15.26:0/3698776050 >> 172.21.15.26:0/3698776050 conn(0x7fc5fc06b1a0 msgr2=0x7fc5fc06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc6015a1640 1 -- 172.21.15.26:0/3698776050 shutdown_connections 2024-03-06T07:21:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.413+0000 7fc6015a1640 1 -- 172.21.15.26:0/3698776050 wait complete. 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.414+0000 7fc6015a1640 1 Processor -- start 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.414+0000 7fc6015a1640 1 -- start start 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc6015a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5fc06e490 0x7fc5fc1ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc6015a1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5fc072e70 0x7fc5fc157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc6015a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc073250 0x7fc5fc157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc6015a1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0c3ca0 con 0x7fc5fc073250 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc6015a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0c3b20 con 0x7fc5fc06e490 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc6015a1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0c6760 con 0x7fc5fc072e70 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc5fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc073250 0x7fc5fc157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc5fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc073250 0x7fc5fc157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45910/0 (socket says 172.21.15.26:45910) 2024-03-06T07:21:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.415+0000 7fc5fb7fe640 1 -- 172.21.15.26:0/3515695127 learned_addr learned my addr 172.21.15.26:0/3515695127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5faffd640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5fc06e490 0x7fc5fc1ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fa7fc640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5fc072e70 0x7fc5fc157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fb7fe640 1 -- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5fc072e70 msgr2=0x7fc5fc157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fb7fe640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5fc072e70 0x7fc5fc157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fb7fe640 1 -- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5fc06e490 msgr2=0x7fc5fc1ee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fb7fe640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5fc06e490 0x7fc5fc1ee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fb7fe640 1 -- 172.21.15.26:0/3515695127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc0723a0 con 0x7fc5fc073250 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5faffd640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5fc06e490 0x7fc5fc1ee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fa7fc640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5fc072e70 0x7fc5fc157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5fb7fe640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc073250 0x7fc5fc157f60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009eb0 tx=0x7fc5ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5e37fe640 1 -- 172.21.15.26:0/3515695127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec044030 con 0x7fc5fc073250 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5e37fe640 1 -- 172.21.15.26:0/3515695127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec02c9d0 con 0x7fc5fc073250 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc0c3070 con 0x7fc5fc073250 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc5e37fe640 1 -- 172.21.15.26:0/3515695127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec039040 con 0x7fc5fc073250 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.416+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc0bbff0 con 0x7fc5fc073250 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.417+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5fc04f330 con 0x7fc5fc073250 2024-03-06T07:21:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.418+0000 7fc5e37fe640 1 -- 172.21.15.26:0/3515695127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5ec047070 con 0x7fc5fc073250 2024-03-06T07:21:39.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.418+0000 7fc5e37fe640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc5e806b4d0 0x7fc5e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.418+0000 7fc5e37fe640 1 -- 172.21.15.26:0/3515695127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc5ec0ac880 con 0x7fc5fc073250 2024-03-06T07:21:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.418+0000 7fc5faffd640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc5e806b4d0 0x7fc5e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.419+0000 7fc5faffd640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc5e806b4d0 0x7fc5e806d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000f60 tx=0x7fc5f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.423+0000 7fc5e37fe640 1 -- 172.21.15.26:0/3515695127 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5ec04e070 con 0x7fc5fc073250 2024-03-06T07:21:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.674+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc5fc0c6340 con 0x7fc5fc073250 2024-03-06T07:21:39.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.674+0000 7fc5e37fe640 1 -- 172.21.15.26:0/3515695127 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+4724 (secure 0 0 0) 0x7fc5ec07ba70 con 0x7fc5fc073250 2024-03-06T07:21:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:39.676 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:21:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:21:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:21:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:21:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:21:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.682 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.683 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.684 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:21:39.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:21:39.686 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:21:39.688 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc5e806b4d0 msgr2=0x7fc5e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc5e806b4d0 0x7fc5e806d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000f60 tx=0x7fc5f4009040 comp rx=0 tx=0).stop 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc073250 msgr2=0x7fc5fc157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc073250 0x7fc5fc157f60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009eb0 tx=0x7fc5ec004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 shutdown_connections 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc5e806b4d0 0x7fc5e806d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5fc073250 0x7fc5fc157f60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5fc072e70 0x7fc5fc157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 --2- 172.21.15.26:0/3515695127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5fc06e490 0x7fc5fc1ee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 >> 172.21.15.26:0/3515695127 conn(0x7fc5fc06b1a0 msgr2=0x7fc5fc1509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 shutdown_connections 2024-03-06T07:21:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.676+0000 7fc6015a1640 1 -- 172.21.15.26:0/3515695127 wait complete. 2024-03-06T07:21:39.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.791+0000 7f3188e81640 1 Processor -- start 2024-03-06T07:21:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.791+0000 7f3188e81640 1 -- start start 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f3188e81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318414f790 0x7f31841529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f3188e81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31840c1aa0 0x7f31840c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f3188e81640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840c28d0 con 0x7f31840c5c60 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f3188e81640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31840c2680 con 0x7f31840c1aa0 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f3188e81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31840c2bb0 con 0x7f318414f790 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f317bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31840c1aa0 0x7f31840c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f317bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31840c1aa0 0x7f31840c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45914/0 (socket says 172.21.15.26:45914) 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f317bfff640 1 -- 172.21.15.26:0/3654852870 learned_addr learned my addr 172.21.15.26:0/3654852870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f317bfff640 1 -- 172.21.15.26:0/3654852870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318414f790 msgr2=0x7f31841529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f317bfff640 1 --2- 172.21.15.26:0/3654852870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318414f790 0x7f31841529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f317bfff640 1 -- 172.21.15.26:0/3654852870 >> v1:172.21.15.117:6789/0 conn(0x7f31840c5c60 legacy=0x7f3184155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:39.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.792+0000 7f317bfff640 1 -- 172.21.15.26:0/3654852870 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840c2e90 con 0x7f31840c1aa0 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.793+0000 7f317bfff640 1 --2- 172.21.15.26:0/3654852870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c1aa0 0x7f31840c5720 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f316c009870 tx=0x7f316c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75b476755efb5c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.793+0000 7f3181d9b640 1 -- 172.21.15.26:0/3654852870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c035020 con 0x7f31840c1aa0 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.793+0000 7f3181d9b640 1 -- 172.21.15.26:0/3654852870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c031640 con 0x7f31840c1aa0 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.793+0000 7f3181d9b640 1 -- 172.21.15.26:0/3654852870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c02d040 con 0x7f31840c1aa0 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 -- 172.21.15.26:0/3654852870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c1aa0 msgr2=0x7f31840c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 --2- 172.21.15.26:0/3654852870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c1aa0 0x7f31840c5720 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f316c009870 tx=0x7f316c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 -- 172.21.15.26:0/3654852870 shutdown_connections 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 --2- 172.21.15.26:0/3654852870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c1aa0 0x7f31840c5720 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 --2- 172.21.15.26:0/3654852870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318414f790 0x7f31841529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 -- 172.21.15.26:0/3654852870 >> 172.21.15.26:0/3654852870 conn(0x7f31840c6710 msgr2=0x7f31840c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:39.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 -- 172.21.15.26:0/3654852870 shutdown_connections 2024-03-06T07:21:39.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.794+0000 7f3188e81640 1 -- 172.21.15.26:0/3654852870 wait complete. 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.795+0000 7f3188e81640 1 Processor -- start 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.795+0000 7f3188e81640 1 -- start start 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f3188e81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31840c1aa0 0x7f3184072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f3188e81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c5c60 0x7f31840c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f3188e81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f318414f790 0x7f31841ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f3188e81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318406cc00 con 0x7f31840c5c60 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f3188e81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31840c0d60 con 0x7f318414f790 2024-03-06T07:21:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f3188e81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318406cd80 con 0x7f31840c1aa0 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c5c60 0x7f31840c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c5c60 0x7f31840c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45916/0 (socket says 172.21.15.26:45916) 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 -- 172.21.15.26:0/2361236308 learned_addr learned my addr 172.21.15.26:0/2361236308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f318259c640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31840c1aa0 0x7f3184072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f3182d9d640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f318414f790 0x7f31841ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 -- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31840c1aa0 msgr2=0x7f3184072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31840c1aa0 0x7f3184072ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 -- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f318414f790 msgr2=0x7f31841ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f318414f790 0x7f31841ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 -- 172.21.15.26:0/2361236308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840c52c0 con 0x7f31840c5c60 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317bfff640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c5c60 0x7f31840c4880 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317affd640 1 -- 172.21.15.26:0/2361236308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c03ac60 con 0x7f31840c5c60 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.796+0000 7f317affd640 1 -- 172.21.15.26:0/2361236308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c03ae20 con 0x7f31840c5c60 2024-03-06T07:21:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.797+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3184151d10 con 0x7f31840c5c60 2024-03-06T07:21:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.797+0000 7f317affd640 1 -- 172.21.15.26:0/2361236308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c044630 con 0x7f31840c5c60 2024-03-06T07:21:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.797+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31840c4dc0 con 0x7f31840c5c60 2024-03-06T07:21:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.798+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31841ebec0 con 0x7f31840c5c60 2024-03-06T07:21:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.798+0000 7f317affd640 1 -- 172.21.15.26:0/2361236308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f316c038020 con 0x7f31840c5c60 2024-03-06T07:21:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.798+0000 7f317affd640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b500 0x7f316406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.798+0000 7f317affd640 1 -- 172.21.15.26:0/2361236308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f316c0abd90 con 0x7f31840c5c60 2024-03-06T07:21:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.799+0000 7f318259c640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b500 0x7f316406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.799+0000 7f318259c640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b500 0x7f316406d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f317400b810 tx=0x7f3174008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.803+0000 7f317affd640 1 -- 172.21.15.26:0/2361236308 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f316c03f070 con 0x7f31840c5c60 2024-03-06T07:21:39.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.988+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f318404f330 con 0x7f31840c5c60 2024-03-06T07:21:39.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.989+0000 7f317affd640 1 -- 172.21.15.26:0/2361236308 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13176 (secure 0 0 0) 0x7f316c07ae60 con 0x7f31840c5c60 2024-03-06T07:21:39.991 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:39.991 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:39.284755+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:39.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b500 msgr2=0x7f316406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b500 0x7f316406d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f317400b810 tx=0x7f3174008040 comp rx=0 tx=0).stop 2024-03-06T07:21:39.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c5c60 msgr2=0x7f31840c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c5c60 0x7f31840c4880 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 shutdown_connections 2024-03-06T07:21:39.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b500 0x7f316406d9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f318414f790 0x7f31841ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c5c60 0x7f31840c4880 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 --2- 172.21.15.26:0/2361236308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31840c1aa0 0x7f3184072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 >> 172.21.15.26:0/2361236308 conn(0x7f31840c6710 msgr2=0x7f31840c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 shutdown_connections 2024-03-06T07:21:39.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:39.991+0000 7f3188e81640 1 -- 172.21.15.26:0/2361236308 wait complete. 2024-03-06T07:21:40.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:40.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:40.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.106+0000 7f4a2aa6f640 1 Processor -- start 2024-03-06T07:21:40.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.106+0000 7f4a2aa6f640 1 -- start start 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.107+0000 7f4a2aa6f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4a240a8290 0x7f4a241539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.107+0000 7f4a2aa6f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240c62a0 con 0x7f4a240c4740 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.107+0000 7f4a2aa6f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240c4c00 con 0x7f4a240c69a0 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.107+0000 7f4a2aa6f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a24153f00 con 0x7f4a240a8290 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.107+0000 7f4a23fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4a240c4740 0x7f4a240701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36510/0 (socket says 172.21.15.26:36510) 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.107+0000 7f4a23fff640 1 -- 172.21.15.26:0/2058497091 learned_addr learned my addr 172.21.15.26:0/2058497091 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a2880b640 1 --2- 172.21.15.26:0/2058497091 >> v2:172.21.15.117:3300/0 conn(0x7f4a240a8290 0x7f4a241539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388543264 0 0) 0x7f4a240c62a0 con 0x7f4a240c4740 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a14003520 con 0x7f4a240c4740 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3230624673 0 0) 0x7f4a14003520 con 0x7f4a240c4740 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 >> v2:172.21.15.117:3300/0 conn(0x7f4a240a8290 msgr2=0x7f4a241539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 --2- 172.21.15.26:0/2058497091 >> v2:172.21.15.117:3300/0 conn(0x7f4a240a8290 0x7f4a241539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 >> v1:172.21.15.117:6789/0 conn(0x7f4a240c69a0 legacy=0x7f4a24072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240c4ee0 con 0x7f4a240c4740 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4a10001670 con 0x7f4a240c4740 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 >> v1:172.21.15.26:6789/0 conn(0x7f4a240c4740 legacy=0x7f4a240701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a140039b0 0x7f4a14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a14004800 0x7f4a14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a140052e0 0x7f4a1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24153f00 con 0x7f4a14004800 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1400c1b0 con 0x7f4a140052e0 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1400c3b0 con 0x7f4a140039b0 2024-03-06T07:21:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a23fff640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a14004800 0x7f4a14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a2900c640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a140052e0 0x7f4a1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a2880b640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a140039b0 0x7f4a14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a23fff640 1 -- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a140039b0 msgr2=0x7f4a14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a23fff640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a140039b0 0x7f4a14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a23fff640 1 -- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a140052e0 msgr2=0x7f4a1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a23fff640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a140052e0 0x7f4a1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a23fff640 1 -- 172.21.15.26:0/2058497091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14003520 con 0x7f4a14004800 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a2900c640 1 -- 172.21.15.26:0/2058497091 reap_dead start 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.108+0000 7f4a2880b640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a140039b0 0x7f4a14003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.109+0000 7f4a23fff640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a14004800 0x7f4a14004c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c210 tx=0x7f4a14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95f6f6feba0584e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.109+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10007d10 con 0x7f4a14004800 2024-03-06T07:21:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.109+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10030070 con 0x7f4a14004800 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.109+0000 7f4a237fe640 1 -- 172.21.15.26:0/2058497091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10005d90 con 0x7f4a14004800 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.110+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/2058497091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a14004800 msgr2=0x7f4a14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.110+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a14004800 0x7f4a14004c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c210 tx=0x7f4a14005830 comp rx=0 tx=0).stop 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.110+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/2058497091 shutdown_connections 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.110+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/2058497091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a14004800 0x7f4a14004c80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.110+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/2058497091 >> 172.21.15.26:0/2058497091 conn(0x7f4a240c5560 msgr2=0x7f4a240c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.110+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/2058497091 shutdown_connections 2024-03-06T07:21:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.110+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/2058497091 wait complete. 2024-03-06T07:21:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.111+0000 7f4a2aa6f640 1 Processor -- start 2024-03-06T07:21:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.111+0000 7f4a2aa6f640 1 -- start start 2024-03-06T07:21:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a14004800 0x7f4a241e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2406d2e0 0x7f4a24156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a240ac200 0x7f4a240ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c6df0 con 0x7f4a2406d2e0 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2406fc80 con 0x7f4a240ac200 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c6f70 con 0x7f4a14004800 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2406d2e0 0x7f4a24156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2406d2e0 0x7f4a24156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45926/0 (socket says 172.21.15.26:45926) 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 -- 172.21.15.26:0/140411799 learned_addr learned my addr 172.21.15.26:0/140411799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2900c640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a240ac200 0x7f4a240ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 -- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a14004800 msgr2=0x7f4a241e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a14004800 0x7f4a241e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 -- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a240ac200 msgr2=0x7f4a240ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a240ac200 0x7f4a240ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 -- 172.21.15.26:0/140411799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a241558a0 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2900c640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a240ac200 0x7f4a240ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a23fff640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2406d2e0 0x7f4a24156c20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c210 tx=0x7f4a240c50b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a217fa640 1 -- 172.21.15.26:0/140411799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10045070 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a217fa640 1 -- 172.21.15.26:0/140411799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10000fe0 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a24155b80 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a217fa640 1 -- 172.21.15.26:0/140411799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10004c30 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.112+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a24156140 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.113+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a24067ad0 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.114+0000 7f4a217fa640 1 -- 172.21.15.26:0/140411799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a10007070 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.114+0000 7f4a217fa640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.114+0000 7f4a217fa640 1 -- 172.21.15.26:0/140411799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4a100aac20 con 0x7f4a2406d2e0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.114+0000 7f4a2880b640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.115+0000 7f4a2880b640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a24069cc0 tx=0x7f4a18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.119+0000 7f4a217fa640 1 -- 172.21.15.26:0/140411799 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4a24067ad0 con 0x7f4a2406d2e0 2024-03-06T07:21:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.297+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a2406e4d0 con 0x7f4a2406d2e0 2024-03-06T07:21:40.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.297+0000 7f4a217fa640 1 -- 172.21.15.26:0/140411799 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a2406e4d0 con 0x7f4a2406d2e0 2024-03-06T07:21:40.299 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:21:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 msgr2=0x7f4a0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a24069cc0 tx=0x7f4a18008040 comp rx=0 tx=0).stop 2024-03-06T07:21:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2406d2e0 msgr2=0x7f4a24156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2406d2e0 0x7f4a24156c20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c210 tx=0x7f4a240c50b0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 shutdown_connections 2024-03-06T07:21:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4a0806b4d0 0x7f4a0806d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a240ac200 0x7f4a240ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2406d2e0 0x7f4a24156c20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 --2- 172.21.15.26:0/140411799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a14004800 0x7f4a241e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.299+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 >> 172.21.15.26:0/140411799 conn(0x7f4a240c5560 msgr2=0x7f4a240c3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.300+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 shutdown_connections 2024-03-06T07:21:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.300+0000 7f4a2aa6f640 1 -- 172.21.15.26:0/140411799 wait complete. 2024-03-06T07:21:40.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:21:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.412+0000 7f0602d93640 1 Processor -- start 2024-03-06T07:21:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.412+0000 7f0602d93640 1 -- start start 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0602d93640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05fc152960 0x7f05fc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0602d93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc1592d0 0x7f05fc1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0602d93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc152d40 con 0x7f05fc0ac260 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0602d93640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc066e30 con 0x7f05fc152960 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0602d93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0c1dc0 con 0x7f05fc1592d0 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0600b2f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05fc152960 0x7f05fc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0601330640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f05fc0ac260 0x7f05fc155810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36516/0 (socket says 172.21.15.26:36516) 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0600b2f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05fc152960 0x7f05fc06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45938/0 (socket says 172.21.15.26:45938) 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0601330640 1 -- 172.21.15.26:0/1692231299 learned_addr learned my addr 172.21.15.26:0/1692231299 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f05fbfff640 1 --2- 172.21.15.26:0/1692231299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc1592d0 0x7f05fc1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0600b2f640 1 -- 172.21.15.26:0/1692231299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc1592d0 msgr2=0x7f05fc1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.413+0000 7f0600b2f640 1 --2- 172.21.15.26:0/1692231299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc1592d0 0x7f05fc1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.414+0000 7f0600b2f640 1 -- 172.21.15.26:0/1692231299 >> v1:172.21.15.26:6789/0 conn(0x7f05fc0ac260 legacy=0x7f05fc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.414+0000 7f05fb7fe640 1 -- 172.21.15.26:0/1692231299 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960779146 0 0) 0x7f05fc152d40 con 0x7f05fc0ac260 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.414+0000 7f0600b2f640 1 -- 172.21.15.26:0/1692231299 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0c1b50 con 0x7f05fc152960 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.414+0000 7f0600b2f640 1 --2- 172.21.15.26:0/1692231299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc06dc90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009d90 tx=0x7f05f402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13c3bdece0fd293 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.414+0000 7f05fb7fe640 1 -- 172.21.15.26:0/1692231299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4039070 con 0x7f05fc152960 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.414+0000 7f05fb7fe640 1 -- 172.21.15.26:0/1692231299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f402cc50 con 0x7f05fc152960 2024-03-06T07:21:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.414+0000 7f05fb7fe640 1 -- 172.21.15.26:0/1692231299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4034440 con 0x7f05fc152960 2024-03-06T07:21:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 -- 172.21.15.26:0/1692231299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 msgr2=0x7f05fc06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 --2- 172.21.15.26:0/1692231299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc06dc90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009d90 tx=0x7f05f402bea0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 -- 172.21.15.26:0/1692231299 shutdown_connections 2024-03-06T07:21:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 --2- 172.21.15.26:0/1692231299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc1592d0 0x7f05fc1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 --2- 172.21.15.26:0/1692231299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc06dc90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 -- 172.21.15.26:0/1692231299 >> 172.21.15.26:0/1692231299 conn(0x7f05fc0c70b0 msgr2=0x7f05fc0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 -- 172.21.15.26:0/1692231299 shutdown_connections 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.415+0000 7f0602d93640 1 -- 172.21.15.26:0/1692231299 wait complete. 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.416+0000 7f0602d93640 1 Processor -- start 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.416+0000 7f0602d93640 1 -- start start 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0602d93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc0ac260 0x7f05fc071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0602d93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc14d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0602d93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc158450 0x7f05fc14dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0602d93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0c20a0 con 0x7f05fc152960 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0602d93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc159760 con 0x7f05fc0ac260 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0602d93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0c2220 con 0x7f05fc158450 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc14d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc14d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45952/0 (socket says 172.21.15.26:45952) 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 -- 172.21.15.26:0/2333338525 learned_addr learned my addr 172.21.15.26:0/2333338525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0601330640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc158450 0x7f05fc14dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f0600b2f640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc0ac260 0x7f05fc071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 -- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc158450 msgr2=0x7f05fc14dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc158450 0x7f05fc14dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 -- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc0ac260 msgr2=0x7f05fc071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc0ac260 0x7f05fc071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 -- 172.21.15.26:0/2333338525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc14e250 con 0x7f05fc152960 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.417+0000 7f05fbfff640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc14d7d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f05f000d620 tx=0x7f05f000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.418+0000 7f05f97fa640 1 -- 172.21.15.26:0/2333338525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000b620 con 0x7f05fc152960 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.418+0000 7f05f97fa640 1 -- 172.21.15.26:0/2333338525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f0019070 con 0x7f05fc152960 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.418+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc1eb300 con 0x7f05fc152960 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.418+0000 7f05f97fa640 1 -- 172.21.15.26:0/2333338525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000ba00 con 0x7f05fc152960 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.418+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc1eb8c0 con 0x7f05fc152960 2024-03-06T07:21:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.419+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc04f330 con 0x7f05fc152960 2024-03-06T07:21:40.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.419+0000 7f05f97fa640 1 -- 172.21.15.26:0/2333338525 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05f000bba0 con 0x7f05fc152960 2024-03-06T07:21:40.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.419+0000 7f05f97fa640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e806b4d0 0x7f05e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.420+0000 7f05f97fa640 1 -- 172.21.15.26:0/2333338525 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f05f008a4e0 con 0x7f05fc152960 2024-03-06T07:21:40.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.420+0000 7f0600b2f640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e806b4d0 0x7f05e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.420+0000 7f0600b2f640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e806b4d0 0x7f05e806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f05f4036000 tx=0x7f05f4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.424+0000 7f05f97fa640 1 -- 172.21.15.26:0/2333338525 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f05f0059490 con 0x7f05fc152960 2024-03-06T07:21:40.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.608+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f05fc156270 con 0x7f05fc152960 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.608+0000 7f05f97fa640 1 -- 172.21.15.26:0/2333338525 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) v1 ==== 84+0+65 (secure 0 0 0) 0x7f05f0058d90 con 0x7f05fc152960 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:40.610 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e806b4d0 msgr2=0x7f05e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e806b4d0 0x7f05e806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f05f4036000 tx=0x7f05f4030000 comp rx=0 tx=0).stop 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 msgr2=0x7f05fc14d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc14d7d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f05f000d620 tx=0x7f05f000dae0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 shutdown_connections 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e806b4d0 0x7f05e806d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05fc158450 0x7f05fc14dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc152960 0x7f05fc14d7d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 --2- 172.21.15.26:0/2333338525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc0ac260 0x7f05fc071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 >> 172.21.15.26:0/2333338525 conn(0x7f05fc0c70b0 msgr2=0x7f05fc150ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 shutdown_connections 2024-03-06T07:21:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.610+0000 7f0602d93640 1 -- 172.21.15.26:0/2333338525 wait complete. 2024-03-06T07:21:40.622 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:21:40.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:21:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.726+0000 7f084cab5640 1 Processor -- start 2024-03-06T07:21:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.726+0000 7f084cab5640 1 -- start start 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f084cab5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08480c88e0 0x7f08480c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f084cab5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08480c7930 0x7f084806bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f084cab5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08480c5400 con 0x7f0848152870 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f084cab5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084806d320 con 0x7f08480c7930 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f084cab5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084806b8d0 con 0x7f08480c88e0 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f0846d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08480c7930 0x7f084806bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f084659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0848152870 0x7f0848155bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36518/0 (socket says 172.21.15.26:36518) 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f084659c640 1 -- 172.21.15.26:0/3766899954 learned_addr learned my addr 172.21.15.26:0/3766899954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f0846d9d640 1 -- 172.21.15.26:0/3766899954 >> v2:172.21.15.117:3300/0 conn(0x7f08480c88e0 msgr2=0x7f08480c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f083ffff640 1 --2- 172.21.15.26:0/3766899954 >> v2:172.21.15.117:3300/0 conn(0x7f08480c88e0 0x7f08480c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f0845d9b640 1 -- 172.21.15.26:0/3766899954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407025173 0 0) 0x7f08480c5400 con 0x7f0848152870 2024-03-06T07:21:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f0846d9d640 1 --2- 172.21.15.26:0/3766899954 >> v2:172.21.15.117:3300/0 conn(0x7f08480c88e0 0x7f08480c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f0846d9d640 1 -- 172.21.15.26:0/3766899954 >> v1:172.21.15.26:6789/0 conn(0x7f0848152870 legacy=0x7f0848155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.727+0000 7f0846d9d640 1 -- 172.21.15.26:0/3766899954 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480c56e0 con 0x7f08480c7930 2024-03-06T07:21:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.728+0000 7f0846d9d640 1 --2- 172.21.15.26:0/3766899954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c7930 0x7f084806bd60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f08480ad5b0 tx=0x7f084002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eef98b62c6900a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.728+0000 7f0845d9b640 1 -- 172.21.15.26:0/3766899954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840039070 con 0x7f08480c7930 2024-03-06T07:21:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.728+0000 7f0845d9b640 1 -- 172.21.15.26:0/3766899954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084002cbf0 con 0x7f08480c7930 2024-03-06T07:21:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.728+0000 7f0845d9b640 1 -- 172.21.15.26:0/3766899954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840034440 con 0x7f08480c7930 2024-03-06T07:21:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 -- 172.21.15.26:0/3766899954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c7930 msgr2=0x7f084806bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 --2- 172.21.15.26:0/3766899954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c7930 0x7f084806bd60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f08480ad5b0 tx=0x7f084002be40 comp rx=0 tx=0).stop 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 -- 172.21.15.26:0/3766899954 shutdown_connections 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 --2- 172.21.15.26:0/3766899954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c7930 0x7f084806bd60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 --2- 172.21.15.26:0/3766899954 >> v2:172.21.15.117:3300/0 conn(0x7f08480c88e0 0x7f08480c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 -- 172.21.15.26:0/3766899954 >> 172.21.15.26:0/3766899954 conn(0x7f08480ac5f0 msgr2=0x7f08480aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 -- 172.21.15.26:0/3766899954 shutdown_connections 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.729+0000 7f084cab5640 1 -- 172.21.15.26:0/3766899954 wait complete. 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.730+0000 7f084cab5640 1 Processor -- start 2024-03-06T07:21:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.730+0000 7f084cab5640 1 -- start start 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480c7930 0x7f08480afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c88e0 0x7f08480af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0848152870 0x7f08481eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480c9470 con 0x7f08480c88e0 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480c5ca0 con 0x7f08480c7930 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480c95f0 con 0x7f0848152870 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c88e0 0x7f08480af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c88e0 0x7f08480af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45972/0 (socket says 172.21.15.26:45972) 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 -- 172.21.15.26:0/277859881 learned_addr learned my addr 172.21.15.26:0/277859881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084659c640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480c7930 0x7f08480afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 -- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0848152870 msgr2=0x7f08481eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f0846d9d640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0848152870 0x7f08481eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0848152870 0x7f08481eb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 -- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480c7930 msgr2=0x7f08480afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480c7930 0x7f08480afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 -- 172.21.15.26:0/277859881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480b0740 con 0x7f08480c88e0 2024-03-06T07:21:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084659c640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480c7930 0x7f08480afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083ffff640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c88e0 0x7f08480af300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f083000c820 tx=0x7f083000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083effd640 1 -- 172.21.15.26:0/277859881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830015070 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083effd640 1 -- 172.21.15.26:0/277859881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830004220 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08480ae860 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f083effd640 1 -- 172.21.15.26:0/277859881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083000ce10 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.731+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08480aeb00 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.733+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084804f330 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.733+0000 7f083effd640 1 -- 172.21.15.26:0/277859881 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0830007830 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.733+0000 7f083effd640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f082806b4d0 0x7f082806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.733+0000 7f083effd640 1 -- 172.21.15.26:0/277859881 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0830089de0 con 0x7f08480c88e0 2024-03-06T07:21:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.734+0000 7f084659c640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f082806b4d0 0x7f082806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.734+0000 7f084659c640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f082806b4d0 0x7f082806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0838000f60 tx=0x7f0838009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.738+0000 7f083effd640 1 -- 172.21.15.26:0/277859881 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0830058eb0 con 0x7f08480c88e0 2024-03-06T07:21:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:40.919+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f08480c16d0 con 0x7f08480c88e0 2024-03-06T07:21:41.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.290+0000 7f083effd640 1 -- 172.21.15.26:0/277859881 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f08300587b0 con 0x7f08480c88e0 2024-03-06T07:21:41.292 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:21:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.292+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f082806b4d0 msgr2=0x7f082806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.292+0000 7f084cab5640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f082806b4d0 0x7f082806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0838000f60 tx=0x7f0838009040 comp rx=0 tx=0).stop 2024-03-06T07:21:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.292+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c88e0 msgr2=0x7f08480af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.292+0000 7f084cab5640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c88e0 0x7f08480af300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f083000c820 tx=0x7f083000cce0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 shutdown_connections 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f082806b4d0 0x7f082806d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0848152870 0x7f08481eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08480c88e0 0x7f08480af300 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 --2- 172.21.15.26:0/277859881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480c7930 0x7f08480afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 >> 172.21.15.26:0/277859881 conn(0x7f08480ac5f0 msgr2=0x7f08480ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 shutdown_connections 2024-03-06T07:21:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.293+0000 7f084cab5640 1 -- 172.21.15.26:0/277859881 wait complete. 2024-03-06T07:21:41.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:21:41.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:41.401 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.421+0000 7f0f3050c640 1 Processor -- start 2024-03-06T07:21:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.421+0000 7f0f3050c640 1 -- start start 2024-03-06T07:21:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f3050c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f281528d0 0x7f0f280724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f3050c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f28072a10 0x7f0f28155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f3050c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f28152cb0 con 0x7f0f280c67f0 2024-03-06T07:21:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f3050c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f280c38b0 con 0x7f0f281528d0 2024-03-06T07:21:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f3050c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281594f0 con 0x7f0f28072a10 2024-03-06T07:21:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f2e2a8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f281528d0 0x7f0f280724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f2e2a8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f281528d0 0x7f0f280724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45974/0 (socket says 172.21.15.26:45974) 2024-03-06T07:21:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.422+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/3670413032 learned_addr learned my addr 172.21.15.26:0/3670413032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.423+0000 7f0f2daa7640 1 --2- 172.21.15.26:0/3670413032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f28072a10 0x7f0f28155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.423+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/3670413032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f28072a10 msgr2=0x7f0f28155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.423+0000 7f0f2e2a8640 1 --2- 172.21.15.26:0/3670413032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f28072a10 0x7f0f28155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.423+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/3670413032 >> v1:172.21.15.117:6789/0 conn(0x7f0f280c67f0 legacy=0x7f0f28155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.423+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/3670413032 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280a7f70 con 0x7f0f281528d0 2024-03-06T07:21:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f2e2a8640 1 --2- 172.21.15.26:0/3670413032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f281528d0 0x7f0f280724d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009870 tx=0x7f0f2402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572273f3514c7edd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f2d2a6640 1 -- 172.21.15.26:0/3670413032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2402f840 con 0x7f0f281528d0 2024-03-06T07:21:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f2d2a6640 1 -- 172.21.15.26:0/3670413032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2402f9e0 con 0x7f0f281528d0 2024-03-06T07:21:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f2d2a6640 1 -- 172.21.15.26:0/3670413032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2402fdb0 con 0x7f0f281528d0 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f3050c640 1 -- 172.21.15.26:0/3670413032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f281528d0 msgr2=0x7f0f280724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f3050c640 1 --2- 172.21.15.26:0/3670413032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f281528d0 0x7f0f280724d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009870 tx=0x7f0f2402ba20 comp rx=0 tx=0).stop 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f3050c640 1 -- 172.21.15.26:0/3670413032 shutdown_connections 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f3050c640 1 --2- 172.21.15.26:0/3670413032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f28072a10 0x7f0f28155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f3050c640 1 --2- 172.21.15.26:0/3670413032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f281528d0 0x7f0f280724d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.424+0000 7f0f3050c640 1 -- 172.21.15.26:0/3670413032 >> 172.21.15.26:0/3670413032 conn(0x7f0f280aa6a0 msgr2=0x7f0f280aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.425+0000 7f0f3050c640 1 -- 172.21.15.26:0/3670413032 shutdown_connections 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.425+0000 7f0f3050c640 1 -- 172.21.15.26:0/3670413032 wait complete. 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 Processor -- start 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 -- start start 2024-03-06T07:21:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f28072a10 0x7f0f281569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f280c67f0 0x7f0f280ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f281528d0 0x7f0f28158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281591a0 con 0x7f0f28072a10 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280a8250 con 0x7f0f280c67f0 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f3050c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280c4ef0 con 0x7f0f281528d0 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f2e2a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f28072a10 0x7f0f281569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f2e2a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f28072a10 0x7f0f281569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45982/0 (socket says 172.21.15.26:45982) 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.426+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/1167208151 learned_addr learned my addr 172.21.15.26:0/1167208151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2daa7640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f280c67f0 0x7f0f280ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2eaa9640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f281528d0 0x7f0f28158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f281528d0 msgr2=0x7f0f28158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2e2a8640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f281528d0 0x7f0f28158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f280c67f0 msgr2=0x7f0f280ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2e2a8640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f280c67f0 0x7f0f280ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2e2a8640 1 -- 172.21.15.26:0/1167208151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f28156f30 con 0x7f0f28072a10 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f2e2a8640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f28072a10 0x7f0f281569f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009870 tx=0x7f0f24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f1effd640 1 -- 172.21.15.26:0/1167208151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24037bc0 con 0x7f0f28072a10 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f1effd640 1 -- 172.21.15.26:0/1167208151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f24046070 con 0x7f0f28072a10 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f281eb680 con 0x7f0f28072a10 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f1effd640 1 -- 172.21.15.26:0/1167208151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24041400 con 0x7f0f28072a10 2024-03-06T07:21:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.427+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f280ad9e0 con 0x7f0f28072a10 2024-03-06T07:21:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.428+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2806c1d0 con 0x7f0f28072a10 2024-03-06T07:21:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.429+0000 7f0f1effd640 1 -- 172.21.15.26:0/1167208151 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f24040070 con 0x7f0f28072a10 2024-03-06T07:21:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.429+0000 7f0f1effd640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.429+0000 7f0f1effd640 1 -- 172.21.15.26:0/1167208151 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0f240ac6b0 con 0x7f0f28072a10 2024-03-06T07:21:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.430+0000 7f0f2daa7640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.430+0000 7f0f2daa7640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ad5a0 tx=0x7f0f18009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.434+0000 7f0f1effd640 1 -- 172.21.15.26:0/1167208151 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f2806c1d0 con 0x7f0f28072a10 2024-03-06T07:21:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:41.615+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0f2804f330 con 0x7f0f28072a10 2024-03-06T07:21:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.292+0000 7f0f1effd640 1 -- 172.21.15.26:0/1167208151 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0f24033030 con 0x7f0f28072a10 2024-03-06T07:21:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0c06b4d0 msgr2=0x7f0f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ad5a0 tx=0x7f0f18009190 comp rx=0 tx=0).stop 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f28072a10 msgr2=0x7f0f281569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f28072a10 0x7f0f281569f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009870 tx=0x7f0f24004730 comp rx=0 tx=0).stop 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 shutdown_connections 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f281528d0 0x7f0f28158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f280c67f0 0x7f0f280ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 --2- 172.21.15.26:0/1167208151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f28072a10 0x7f0f281569f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.296+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 >> 172.21.15.26:0/1167208151 conn(0x7f0f280aa6a0 msgr2=0x7f0f280c80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.302+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 shutdown_connections 2024-03-06T07:21:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.305+0000 7f0f3050c640 1 -- 172.21.15.26:0/1167208151 wait complete. 2024-03-06T07:21:42.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:21:42.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.438+0000 7fb2bd8c3640 1 Processor -- start 2024-03-06T07:21:42.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.438+0000 7fb2bd8c3640 1 -- start start 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2bd8c3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2b80c3280 0x7fb2b80c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2bd8c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b8158800 con 0x7fb2b806d9f0 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2bd8c3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b8158ae0 con 0x7fb2b81551e0 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2bd8c3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b8158dc0 con 0x7fb2b80c3280 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b67fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2b80c3280 0x7fb2b80c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb2b806d9f0 0x7fb2b80a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36520/0 (socket says 172.21.15.26:36520) 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b67fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2b80c3280 0x7fb2b80c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45996/0 (socket says 172.21.15.26:45996) 2024-03-06T07:21:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b6ffd640 1 -- 172.21.15.26:0/2137548709 learned_addr learned my addr 172.21.15.26:0/2137548709 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:42.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b67fc640 1 -- 172.21.15.26:0/2137548709 >> v1:172.21.15.117:6789/0 conn(0x7fb2b81551e0 legacy=0x7fb2b81555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b67fc640 1 -- 172.21.15.26:0/2137548709 >> v1:172.21.15.26:6789/0 conn(0x7fb2b806d9f0 legacy=0x7fb2b80a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b67fc640 1 -- 172.21.15.26:0/2137548709 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b81590a0 con 0x7fb2b80c3280 2024-03-06T07:21:42.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.439+0000 7fb2b5ffb640 1 -- 172.21.15.26:0/2137548709 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775362152 0 0) 0x7fb2b8158800 con 0x7fb2b806d9f0 2024-03-06T07:21:42.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.440+0000 7fb2b67fc640 1 --2- 172.21.15.26:0/2137548709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b80c3280 0x7fb2b80c3710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0097a0 tx=0x7fb2ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6150232c262122 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:42.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.440+0000 7fb2b5ffb640 1 -- 172.21.15.26:0/2137548709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac035020 con 0x7fb2b80c3280 2024-03-06T07:21:42.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.440+0000 7fb2b5ffb640 1 -- 172.21.15.26:0/2137548709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac031830 con 0x7fb2b80c3280 2024-03-06T07:21:42.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.440+0000 7fb2b5ffb640 1 -- 172.21.15.26:0/2137548709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac031c00 con 0x7fb2b80c3280 2024-03-06T07:21:42.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.441+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/2137548709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b80c3280 msgr2=0x7fb2b80c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.441+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/2137548709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b80c3280 0x7fb2b80c3710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0097a0 tx=0x7fb2ac02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.441+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/2137548709 shutdown_connections 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.441+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/2137548709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b80c3280 0x7fb2b80c3710 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.441+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/2137548709 >> 172.21.15.26:0/2137548709 conn(0x7fb2b80aa630 msgr2=0x7fb2b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.441+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/2137548709 shutdown_connections 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.441+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/2137548709 wait complete. 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.442+0000 7fb2bd8c3640 1 Processor -- start 2024-03-06T07:21:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.442+0000 7fb2bd8c3640 1 -- start start 2024-03-06T07:21:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b806d9f0 0x7fb2b814f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b80c3280 0x7fb2b81ee110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2b81551e0 0x7fb2b8140a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b806df50 con 0x7fb2b806d9f0 2024-03-06T07:21:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b806ddd0 con 0x7fb2b80c3280 2024-03-06T07:21:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c6910 con 0x7fb2b81551e0 2024-03-06T07:21:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b806d9f0 0x7fb2b814f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b806d9f0 0x7fb2b814f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46008/0 (socket says 172.21.15.26:46008) 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 -- 172.21.15.26:0/3660829026 learned_addr learned my addr 172.21.15.26:0/3660829026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b67fc640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b80c3280 0x7fb2b81ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b77fe640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2b81551e0 0x7fb2b8140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 -- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2b81551e0 msgr2=0x7fb2b8140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2b81551e0 0x7fb2b8140a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 -- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b80c3280 msgr2=0x7fb2b81ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b80c3280 0x7fb2b81ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 -- 172.21.15.26:0/3660829026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b81eeb90 con 0x7fb2b806d9f0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2b6ffd640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b806d9f0 0x7fb2b814f340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000c4d0 tx=0x7fb2b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb29ffff640 1 -- 172.21.15.26:0/3660829026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0018070 con 0x7fb2b806d9f0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb29ffff640 1 -- 172.21.15.26:0/3660829026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b00074e0 con 0x7fb2b806d9f0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80ab7a0 con 0x7fb2b806d9f0 2024-03-06T07:21:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb29ffff640 1 -- 172.21.15.26:0/3660829026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0004a20 con 0x7fb2b806d9f0 2024-03-06T07:21:42.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.443+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b814efd0 con 0x7fb2b806d9f0 2024-03-06T07:21:42.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.445+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b8068fa0 con 0x7fb2b806d9f0 2024-03-06T07:21:42.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.445+0000 7fb29ffff640 1 -- 172.21.15.26:0/3660829026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2b001b070 con 0x7fb2b806d9f0 2024-03-06T07:21:42.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.445+0000 7fb29ffff640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a406b500 0x7fb2a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.445+0000 7fb29ffff640 1 -- 172.21.15.26:0/3660829026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fb2b0089630 con 0x7fb2b806d9f0 2024-03-06T07:21:42.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.446+0000 7fb2b67fc640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a406b500 0x7fb2a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.446+0000 7fb2b67fc640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a406b500 0x7fb2a406d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac036000 tx=0x7fb2ac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.450+0000 7fb29ffff640 1 -- 172.21.15.26:0/3660829026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb2b8068fa0 con 0x7fb2b806d9f0 2024-03-06T07:21:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:42.667+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb2b804f330 con 0x7fb2b806d9f0 2024-03-06T07:21:43.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.314+0000 7fb29ffff640 1 -- 172.21.15.26:0/3660829026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v26) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb2b0058380 con 0x7fb2b806d9f0 2024-03-06T07:21:43.316 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 7 and data pool 8 2024-03-06T07:21:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.326+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a406b500 msgr2=0x7fb2a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.326+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a406b500 0x7fb2a406d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac036000 tx=0x7fb2ac008030 comp rx=0 tx=0).stop 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.326+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b806d9f0 msgr2=0x7fb2b814f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.326+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b806d9f0 0x7fb2b814f340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000c4d0 tx=0x7fb2b000c990 comp rx=0 tx=0).stop 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.327+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 shutdown_connections 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.327+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a406b500 0x7fb2a406d9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.327+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2b81551e0 0x7fb2b8140a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.327+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b80c3280 0x7fb2b81ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.327+0000 7fb2bd8c3640 1 --2- 172.21.15.26:0/3660829026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b806d9f0 0x7fb2b814f340 secure :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000c4d0 tx=0x7fb2b000c990 comp rx=0 tx=0).stop 2024-03-06T07:21:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.327+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 >> 172.21.15.26:0/3660829026 conn(0x7fb2b80aa630 msgr2=0x7fb2b81532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.330+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 shutdown_connections 2024-03-06T07:21:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.331+0000 7fb2bd8c3640 1 -- 172.21.15.26:0/3660829026 wait complete. 2024-03-06T07:21:43.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.455+0000 7fe3f6b48640 1 Processor -- start 2024-03-06T07:21:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.455+0000 7fe3f6b48640 1 -- start start 2024-03-06T07:21:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3f6b48640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3f00c3710 0x7fe3f01558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3f6b48640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f0158800 con 0x7fe3f006d9f0 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3f6b48640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f0158ae0 con 0x7fe3f00c3280 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3f6b48640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f0158dc0 con 0x7fe3f00c3710 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3f48e4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe3f006d9f0 0x7fe3f00a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36530/0 (socket says 172.21.15.26:36530) 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3f48e4640 1 -- 172.21.15.26:0/2016749657 learned_addr learned my addr 172.21.15.26:0/2016749657 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3f50e5640 1 --2- 172.21.15.26:0/2016749657 >> v2:172.21.15.117:3300/0 conn(0x7fe3f00c3710 0x7fe3f01558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524201684 0 0) 0x7fe3f0158800 con 0x7fe3f006d9f0 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e0003520 con 0x7fe3f006d9f0 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4186705927 0 0) 0x7fe3e0003520 con 0x7fe3f006d9f0 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 >> v2:172.21.15.117:3300/0 conn(0x7fe3f00c3710 msgr2=0x7fe3f01558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 --2- 172.21.15.26:0/2016749657 >> v2:172.21.15.117:3300/0 conn(0x7fe3f00c3710 0x7fe3f01558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 >> v1:172.21.15.117:6789/0 conn(0x7fe3f00c3280 legacy=0x7fe3f01551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f01590a0 con 0x7fe3f006d9f0 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe3e4001670 con 0x7fe3f006d9f0 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 >> v1:172.21.15.26:6789/0 conn(0x7fe3f006d9f0 legacy=0x7fe3f00a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.456+0000 7fe3ef7fe640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3ef7fe640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e00051c0 0x7fe3e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f0158dc0 con 0x7fe3e00039b0 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e000c160 con 0x7fe3e00051c0 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e000c360 con 0x7fe3e0004800 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f48e4640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3effff640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f50e5640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e00051c0 0x7fe3e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f48e4640 1 -- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0004800 msgr2=0x7fe3e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:43.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f48e4640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f48e4640 1 -- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e00051c0 msgr2=0x7fe3e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:43.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f48e4640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e00051c0 0x7fe3e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f48e4640 1 -- 172.21.15.26:0/2016749657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0003520 con 0x7fe3e00039b0 2024-03-06T07:21:43.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3f50e5640 1 -- 172.21.15.26:0/2016749657 reap_dead start 2024-03-06T07:21:43.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.457+0000 7fe3effff640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:43.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f48e4640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c240 tx=0x7fe3e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26fbad340e8ffdc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4007d10 con 0x7fe3e00039b0 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4030070 con 0x7fe3e00039b0 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3ef7fe640 1 -- 172.21.15.26:0/2016749657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4005d90 con 0x7fe3e00039b0 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f6b48640 1 -- 172.21.15.26:0/2016749657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e00039b0 msgr2=0x7fe3e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c240 tx=0x7fe3e0005830 comp rx=0 tx=0).stop 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f6b48640 1 -- 172.21.15.26:0/2016749657 shutdown_connections 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/2016749657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f6b48640 1 -- 172.21.15.26:0/2016749657 >> 172.21.15.26:0/2016749657 conn(0x7fe3f00aa630 msgr2=0x7fe3f00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f6b48640 1 -- 172.21.15.26:0/2016749657 shutdown_connections 2024-03-06T07:21:43.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.458+0000 7fe3f6b48640 1 -- 172.21.15.26:0/2016749657 wait complete. 2024-03-06T07:21:43.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.459+0000 7fe3f6b48640 1 Processor -- start 2024-03-06T07:21:43.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f6b48640 1 -- start start 2024-03-06T07:21:43.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f6b48640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e00039b0 0x7fe3f01eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f6b48640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f00ab620 0x7fe3f006f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f6b48640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f014dc20 0x7fe3f00acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f6b48640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f006df50 con 0x7fe3f014dc20 2024-03-06T07:21:43.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f6b48640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f006ddd0 con 0x7fe3f00ab620 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f6b48640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00c6910 con 0x7fe3e00039b0 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f50e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f014dc20 0x7fe3f00acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f50e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f014dc20 0x7fe3f00acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46014/0 (socket says 172.21.15.26:46014) 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.460+0000 7fe3f50e5640 1 -- 172.21.15.26:0/1687349280 learned_addr learned my addr 172.21.15.26:0/1687349280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3effff640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f00ab620 0x7fe3f006f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f48e4640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e00039b0 0x7fe3f01eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f50e5640 1 -- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e00039b0 msgr2=0x7fe3f01eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f50e5640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e00039b0 0x7fe3f01eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f50e5640 1 -- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f00ab620 msgr2=0x7fe3f006f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f50e5640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f00ab620 0x7fe3f006f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f50e5640 1 -- 172.21.15.26:0/1687349280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0070250 con 0x7fe3f014dc20 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f50e5640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f014dc20 0x7fe3f00acd40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007c20 tx=0x7fe3e80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3ed7fa640 1 -- 172.21.15.26:0/1687349280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8007e90 con 0x7fe3f014dc20 2024-03-06T07:21:43.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3ed7fa640 1 -- 172.21.15.26:0/1687349280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e801b070 con 0x7fe3f014dc20 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f014ac90 con 0x7fe3f014dc20 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3ed7fa640 1 -- 172.21.15.26:0/1687349280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8011de0 con 0x7fe3f014dc20 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.461+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f014a6d0 con 0x7fe3f014dc20 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.462+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f004f330 con 0x7fe3f014dc20 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.462+0000 7fe3ed7fa640 1 -- 172.21.15.26:0/1687349280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3e8003990 con 0x7fe3f014dc20 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.463+0000 7fe3ed7fa640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3d406b6a0 0x7fe3d406db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.463+0000 7fe3ed7fa640 1 -- 172.21.15.26:0/1687349280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3e808a9f0 con 0x7fe3f014dc20 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.463+0000 7fe3f48e4640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3d406b6a0 0x7fe3d406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.464+0000 7fe3f48e4640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3d406b6a0 0x7fe3d406db60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4034000 tx=0x7fe3e4039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.467+0000 7fe3ed7fa640 1 -- 172.21.15.26:0/1687349280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe3f0070250 con 0x7fe3f014dc20 2024-03-06T07:21:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.649+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3f0073240 con 0x7fe3f014dc20 2024-03-06T07:21:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.650+0000 7fe3ed7fa640 1 -- 172.21.15.26:0/1687349280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16967 (secure 0 0 0) 0x7fe3e808a070 con 0x7fe3f014dc20 2024-03-06T07:21:43.652 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:43.652 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:42.668840+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3d406b6a0 msgr2=0x7fe3d406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3d406b6a0 0x7fe3d406db60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4034000 tx=0x7fe3e4039040 comp rx=0 tx=0).stop 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f014dc20 msgr2=0x7fe3f00acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f014dc20 0x7fe3f00acd40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007c20 tx=0x7fe3e80115a0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 shutdown_connections 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe3d406b6a0 0x7fe3d406db60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f014dc20 0x7fe3f00acd40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f00ab620 0x7fe3f006f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 --2- 172.21.15.26:0/1687349280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e00039b0 0x7fe3f01eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 >> 172.21.15.26:0/1687349280 conn(0x7fe3f00aa630 msgr2=0x7fe3f00c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 shutdown_connections 2024-03-06T07:21:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.652+0000 7fe3f6b48640 1 -- 172.21.15.26:0/1687349280 wait complete. 2024-03-06T07:21:43.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.764+0000 7f75cc75e640 1 Processor -- start 2024-03-06T07:21:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.765+0000 7f75cc75e640 1 -- start start 2024-03-06T07:21:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.765+0000 7f75cc75e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75c4152920 0x7f75c4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.765+0000 7f75cc75e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c4158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.765+0000 7f75cc75e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c4156180 con 0x7f75c4159460 2024-03-06T07:21:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.765+0000 7f75cc75e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c41587b0 con 0x7f75c4152920 2024-03-06T07:21:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.765+0000 7f75cc75e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40c7760 con 0x7f75c40c8a00 2024-03-06T07:21:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75ca4fa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75c4152920 0x7f75c4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75cacfb640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f75c4159460 0x7f75c40c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:56342/0 (socket says 172.21.15.26:56342) 2024-03-06T07:21:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75cacfb640 1 -- 172.21.15.26:0/3122179551 learned_addr learned my addr 172.21.15.26:0/3122179551 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75c9cf9640 1 --2- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c4158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75ca4fa640 1 -- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 msgr2=0x7f75c4158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75ca4fa640 1 --2- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c4158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75ca4fa640 1 -- 172.21.15.26:0/3122179551 >> v1:172.21.15.117:6789/0 conn(0x7f75c4159460 legacy=0x7f75c40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75ca4fa640 1 -- 172.21.15.26:0/3122179551 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c40c7a40 con 0x7f75c4152920 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75c94f8640 1 -- 172.21.15.26:0/3122179551 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942837495 0 0) 0x7f75b4001490 con 0x7f75c4159460 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.766+0000 7f75c9cf9640 1 --2- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.767+0000 7f75ca4fa640 1 --2- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 0x7f75c4155c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097a0 tx=0x7f75bc02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d22e4e9e743535 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.767+0000 7f75c94f8640 1 -- 172.21.15.26:0/3122179551 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc031780 con 0x7f75c4152920 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.767+0000 7f75c94f8640 1 -- 172.21.15.26:0/3122179551 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc037070 con 0x7f75c4152920 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.767+0000 7f75c94f8640 1 -- 172.21.15.26:0/3122179551 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc031b40 con 0x7f75c4152920 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 -- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 msgr2=0x7f75c4155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 --2- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 0x7f75c4155c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097a0 tx=0x7f75bc02b820 comp rx=0 tx=0).stop 2024-03-06T07:21:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 -- 172.21.15.26:0/3122179551 shutdown_connections 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 --2- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 --2- 172.21.15.26:0/3122179551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 0x7f75c4155c40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 -- 172.21.15.26:0/3122179551 >> 172.21.15.26:0/3122179551 conn(0x7f75c40ac560 msgr2=0x7f75c40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 -- 172.21.15.26:0/3122179551 shutdown_connections 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.768+0000 7f75cc75e640 1 -- 172.21.15.26:0/3122179551 wait complete. 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 Processor -- start 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 -- start start 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c41eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 0x7f75c414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c414d5a0 0x7f75c414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c406d030 con 0x7f75c414d5a0 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c4158a00 con 0x7f75c4152920 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.769+0000 7f75cc75e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40c5a40 con 0x7f75c40c8a00 2024-03-06T07:21:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c414d5a0 0x7f75c414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c414d5a0 0x7f75c414da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46028/0 (socket says 172.21.15.26:46028) 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 -- 172.21.15.26:0/12078886 learned_addr learned my addr 172.21.15.26:0/12078886 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75c9cf9640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 0x7f75c414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75ca4fa640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c41eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 -- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 msgr2=0x7f75c41eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c41eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 -- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 msgr2=0x7f75c414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 0x7f75c414b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 -- 172.21.15.26:0/12078886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c40cb160 con 0x7f75c414d5a0 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cacfb640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c414d5a0 0x7f75c414da30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c7b0 tx=0x7f75b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75c2ffd640 1 -- 172.21.15.26:0/12078886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4018070 con 0x7f75c414d5a0 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75c2ffd640 1 -- 172.21.15.26:0/12078886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b400a640 con 0x7f75c414d5a0 2024-03-06T07:21:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c414e7d0 con 0x7f75c414d5a0 2024-03-06T07:21:43.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75c2ffd640 1 -- 172.21.15.26:0/12078886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400ce60 con 0x7f75c414d5a0 2024-03-06T07:21:43.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.770+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c4157040 con 0x7f75c414d5a0 2024-03-06T07:21:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.771+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c404f330 con 0x7f75c414d5a0 2024-03-06T07:21:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.772+0000 7f75c2ffd640 1 -- 172.21.15.26:0/12078886 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75b401b070 con 0x7f75c414d5a0 2024-03-06T07:21:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.772+0000 7f75c2ffd640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75b006b500 0x7f75b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.772+0000 7f75c2ffd640 1 -- 172.21.15.26:0/12078886 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75b40897a0 con 0x7f75c414d5a0 2024-03-06T07:21:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.773+0000 7f75ca4fa640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75b006b500 0x7f75b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.773+0000 7f75ca4fa640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75b006b500 0x7f75b006d9c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75bc036000 tx=0x7f75bc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.777+0000 7f75c2ffd640 1 -- 172.21.15.26:0/12078886 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f75c4156180 con 0x7f75c414d5a0 2024-03-06T07:21:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.959+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75c40ca3e0 con 0x7f75c414d5a0 2024-03-06T07:21:43.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.960+0000 7f75c2ffd640 1 -- 172.21.15.26:0/12078886 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16967 (secure 0 0 0) 0x7f75b4057e40 con 0x7f75c414d5a0 2024-03-06T07:21:43.962 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:43.962 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:42.668840+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75b006b500 msgr2=0x7f75b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75b006b500 0x7f75b006d9c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75bc036000 tx=0x7f75bc008030 comp rx=0 tx=0).stop 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c414d5a0 msgr2=0x7f75c414da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c414d5a0 0x7f75c414da30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c7b0 tx=0x7f75b400cc70 comp rx=0 tx=0).stop 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 shutdown_connections 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f75b006b500 0x7f75b006d9c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c414d5a0 0x7f75c414da30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4152920 0x7f75c414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 --2- 172.21.15.26:0/12078886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75c40c8a00 0x7f75c41eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 >> 172.21.15.26:0/12078886 conn(0x7f75c40ac560 msgr2=0x7f75c40c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 shutdown_connections 2024-03-06T07:21:43.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:43.962+0000 7f75cc75e640 1 -- 172.21.15.26:0/12078886 wait complete. 2024-03-06T07:21:43.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:21:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.078+0000 7f4c2e7c7640 1 Processor -- start 2024-03-06T07:21:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.078+0000 7f4c2e7c7640 1 -- start start 2024-03-06T07:21:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c2e7c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4c2806d750 0x7f4c280732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c2e7c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c280c3a70 0x7f4c28072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c2e7c7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c28156060 con 0x7f4c2806db30 2024-03-06T07:21:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c2e7c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c28155b30 con 0x7f4c280c3a70 2024-03-06T07:21:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c2e7c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c280a8430 con 0x7f4c2806d750 2024-03-06T07:21:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c277fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c280c3a70 0x7f4c28072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c277fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c280c3a70 0x7f4c28072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46034/0 (socket says 172.21.15.26:46034) 2024-03-06T07:21:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c277fe640 1 -- 172.21.15.26:0/2185456358 learned_addr learned my addr 172.21.15.26:0/2185456358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c277fe640 1 -- 172.21.15.26:0/2185456358 >> v2:172.21.15.117:3300/0 conn(0x7f4c2806d750 msgr2=0x7f4c280732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.079+0000 7f4c27fff640 1 --2- 172.21.15.26:0/2185456358 >> v2:172.21.15.117:3300/0 conn(0x7f4c2806d750 0x7f4c280732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.080+0000 7f4c277fe640 1 --2- 172.21.15.26:0/2185456358 >> v2:172.21.15.117:3300/0 conn(0x7f4c2806d750 0x7f4c280732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.080+0000 7f4c277fe640 1 -- 172.21.15.26:0/2185456358 >> v1:172.21.15.117:6789/0 conn(0x7f4c2806db30 legacy=0x7f4c280c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.080+0000 7f4c277fe640 1 -- 172.21.15.26:0/2185456358 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28155d80 con 0x7f4c280c3a70 2024-03-06T07:21:44.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.080+0000 7f4c277fe640 1 --2- 172.21.15.26:0/2185456358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c28072680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4c280c5440 tx=0x7f4c1802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a97cc093d5de2c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.080+0000 7f4c26ffd640 1 -- 172.21.15.26:0/2185456358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18038070 con 0x7f4c280c3a70 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.080+0000 7f4c26ffd640 1 -- 172.21.15.26:0/2185456358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1802f050 con 0x7f4c280c3a70 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.080+0000 7f4c26ffd640 1 -- 172.21.15.26:0/2185456358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18031970 con 0x7f4c280c3a70 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2185456358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 msgr2=0x7f4c28072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2185456358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c28072680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4c280c5440 tx=0x7f4c1802b7f0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2185456358 shutdown_connections 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2185456358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c28072680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2185456358 >> v2:172.21.15.117:3300/0 conn(0x7f4c2806d750 0x7f4c280732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2185456358 >> 172.21.15.26:0/2185456358 conn(0x7f4c280c5560 msgr2=0x7f4c280c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2185456358 shutdown_connections 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.081+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2185456358 wait complete. 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.082+0000 7f4c2e7c7640 1 Processor -- start 2024-03-06T07:21:44.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.082+0000 7f4c2e7c7640 1 -- start start 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2e7c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2806d750 0x7f4c281541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2e7c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2806db30 0x7f4c280c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2e7c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c280c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2e7c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28072bc0 con 0x7f4c280c3a70 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2e7c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c280c27b0 con 0x7f4c2806db30 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2e7c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28072d40 con 0x7f4c2806d750 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c280c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c280c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46040/0 (socket says 172.21.15.26:46040) 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 -- 172.21.15.26:0/2664032136 learned_addr learned my addr 172.21.15.26:0/2664032136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c277fe640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2806db30 0x7f4c280c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c27fff640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2806d750 0x7f4c281541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 -- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2806d750 msgr2=0x7f4c281541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2806d750 0x7f4c281541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 -- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2806db30 msgr2=0x7f4c280c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2806db30 0x7f4c280c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.083+0000 7f4c2cd64640 1 -- 172.21.15.26:0/2664032136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c280aa4e0 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.084+0000 7f4c2cd64640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c280c80d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000b960 tx=0x7f4c2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.084+0000 7f4c24ff9640 1 -- 172.21.15.26:0/2664032136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20019070 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.084+0000 7f4c24ff9640 1 -- 172.21.15.26:0/2664032136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2000cb20 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.084+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c280c94d0 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.084+0000 7f4c24ff9640 1 -- 172.21.15.26:0/2664032136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20014440 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.084+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c281eb3f0 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.085+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2804f330 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.085+0000 7f4c24ff9640 1 -- 172.21.15.26:0/2664032136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c2001c070 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.086+0000 7f4c24ff9640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0806b4d0 0x7f4c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.086+0000 7f4c24ff9640 1 -- 172.21.15.26:0/2664032136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c20089ec0 con 0x7f4c280c3a70 2024-03-06T07:21:44.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.086+0000 7f4c27fff640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0806b4d0 0x7f4c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.086+0000 7f4c27fff640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0806b4d0 0x7f4c0806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c2806d190 tx=0x7f4c10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.090+0000 7f4c24ff9640 1 -- 172.21.15.26:0/2664032136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4c200585f0 con 0x7f4c280c3a70 2024-03-06T07:21:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:44.311+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c280c2030 con 0x7f4c280c3a70 2024-03-06T07:21:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.313+0000 7f4c24ff9640 1 -- 172.21.15.26:0/2664032136 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4c2005c690 con 0x7f4c280c3a70 2024-03-06T07:21:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0806b4d0 msgr2=0x7f4c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0806b4d0 0x7f4c0806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c2806d190 tx=0x7f4c10008040 comp rx=0 tx=0).stop 2024-03-06T07:21:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 msgr2=0x7f4c280c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c280c80d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000b960 tx=0x7f4c2000be20 comp rx=0 tx=0).stop 2024-03-06T07:21:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 shutdown_connections 2024-03-06T07:21:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c0806b4d0 0x7f4c0806d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c280c3a70 0x7f4c280c80d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2806db30 0x7f4c280c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 --2- 172.21.15.26:0/2664032136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2806d750 0x7f4c281541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 >> 172.21.15.26:0/2664032136 conn(0x7f4c280c5560 msgr2=0x7f4c280a94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:45.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 shutdown_connections 2024-03-06T07:21:45.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.315+0000 7f4c2e7c7640 1 -- 172.21.15.26:0/2664032136 wait complete. 2024-03-06T07:21:45.329 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:21:45.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:21:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.435+0000 7f1ae3713640 1 Processor -- start 2024-03-06T07:21:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.435+0000 7f1ae3713640 1 -- start start 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae3713640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1adc0c4740 0x7f1adc1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae3713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c69a0 0x7f1adc072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae3713640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1adc0c62a0 con 0x7f1adc0a8290 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae3713640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc153f00 con 0x7f1adc0c4740 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae3713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc0c4c00 con 0x7f1adc0c69a0 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae0cae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1adc0c4740 0x7f1adc1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae0cae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1adc0c4740 0x7f1adc1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46042/0 (socket says 172.21.15.26:46042) 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae0cae640 1 -- 172.21.15.26:0/3758054755 learned_addr learned my addr 172.21.15.26:0/3758054755 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae1cb0640 1 --2- 172.21.15.26:0/3758054755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c69a0 0x7f1adc072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae0cae640 1 -- 172.21.15.26:0/3758054755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c69a0 msgr2=0x7f1adc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:45.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae0cae640 1 --2- 172.21.15.26:0/3758054755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c69a0 0x7f1adc072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae0cae640 1 -- 172.21.15.26:0/3758054755 >> v1:172.21.15.117:6789/0 conn(0x7f1adc0a8290 legacy=0x7f1adc0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:45.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.436+0000 7f1ae0cae640 1 -- 172.21.15.26:0/3758054755 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0c4ee0 con 0x7f1adc0c4740 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.437+0000 7f1ae0cae640 1 --2- 172.21.15.26:0/3758054755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0c4740 0x7f1adc1539c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009870 tx=0x7f1acc02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d655839db739bc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.437+0000 7f1ad3fff640 1 -- 172.21.15.26:0/3758054755 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc038070 con 0x7f1adc0c4740 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.437+0000 7f1ad3fff640 1 -- 172.21.15.26:0/3758054755 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc031740 con 0x7f1adc0c4740 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.437+0000 7f1ad3fff640 1 -- 172.21.15.26:0/3758054755 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc02f050 con 0x7f1adc0c4740 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.437+0000 7f1ae3713640 1 -- 172.21.15.26:0/3758054755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0c4740 msgr2=0x7f1adc1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.437+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3758054755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0c4740 0x7f1adc1539c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009870 tx=0x7f1acc02b7e0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.438+0000 7f1ae3713640 1 -- 172.21.15.26:0/3758054755 shutdown_connections 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.438+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3758054755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c69a0 0x7f1adc072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.438+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3758054755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0c4740 0x7f1adc1539c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.438+0000 7f1ae3713640 1 -- 172.21.15.26:0/3758054755 >> 172.21.15.26:0/3758054755 conn(0x7f1adc0c5560 msgr2=0x7f1adc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.438+0000 7f1ae3713640 1 -- 172.21.15.26:0/3758054755 shutdown_connections 2024-03-06T07:21:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.438+0000 7f1ae3713640 1 -- 172.21.15.26:0/3758054755 wait complete. 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.439+0000 7f1ae3713640 1 Processor -- start 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.439+0000 7f1ae3713640 1 -- start start 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae3713640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0a8290 0x7f1adc1dfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae3713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c4740 0x7f1adc1ec400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae3713640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0c69a0 0x7f1adc1ec940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae3713640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc0c6e30 con 0x7f1adc0a8290 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae3713640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc06fc80 con 0x7f1adc0c69a0 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae3713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc072bc0 con 0x7f1adc0c4740 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0a8290 0x7f1adc1dfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0a8290 0x7f1adc1dfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46048/0 (socket says 172.21.15.26:46048) 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 -- 172.21.15.26:0/3627345558 learned_addr learned my addr 172.21.15.26:0/3627345558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae0cae640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c4740 0x7f1adc1ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae1cb0640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0c69a0 0x7f1adc1ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 -- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c4740 msgr2=0x7f1adc1ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c4740 0x7f1adc1ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 -- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0c69a0 msgr2=0x7f1adc1ec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0c69a0 0x7f1adc1ec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 -- 172.21.15.26:0/3627345558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc14bac0 con 0x7f1adc0a8290 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae0cae640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c4740 0x7f1adc1ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae1cb0640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0c69a0 0x7f1adc1ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.440+0000 7f1ae14af640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0a8290 0x7f1adc1dfa00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c430 tx=0x7f1ad800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.441+0000 7f1ad1ffb640 1 -- 172.21.15.26:0/3627345558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011790 con 0x7f1adc0a8290 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.441+0000 7f1ad1ffb640 1 -- 172.21.15.26:0/3627345558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad8017070 con 0x7f1adc0a8290 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.441+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc14d480 con 0x7f1adc0a8290 2024-03-06T07:21:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.441+0000 7f1ad1ffb640 1 -- 172.21.15.26:0/3627345558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011b70 con 0x7f1adc0a8290 2024-03-06T07:21:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.441+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc14da40 con 0x7f1adc0a8290 2024-03-06T07:21:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.442+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc04f330 con 0x7f1adc0a8290 2024-03-06T07:21:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.442+0000 7f1ad1ffb640 1 -- 172.21.15.26:0/3627345558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ad8011d10 con 0x7f1adc0a8290 2024-03-06T07:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.443+0000 7f1ad1ffb640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ac006b4d0 0x7f1ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.443+0000 7f1ad1ffb640 1 -- 172.21.15.26:0/3627345558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ad808a1c0 con 0x7f1adc0a8290 2024-03-06T07:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.443+0000 7f1ae0cae640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ac006b4d0 0x7f1ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.444+0000 7f1ae0cae640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ac006b4d0 0x7f1ac006d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1acc035000 tx=0x7f1acc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.447+0000 7f1ad1ffb640 1 -- 172.21.15.26:0/3627345558 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1ad8058f40 con 0x7f1adc0a8290 2024-03-06T07:21:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:45.676+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1adc0c8480 con 0x7f1adc0a8290 2024-03-06T07:21:46.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.320+0000 7f1ad1ffb640 1 -- 172.21.15.26:0/3627345558 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1ad8058840 con 0x7f1adc0a8290 2024-03-06T07:21:46.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ac006b4d0 msgr2=0x7f1ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ac006b4d0 0x7f1ac006d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1acc035000 tx=0x7f1acc008030 comp rx=0 tx=0).stop 2024-03-06T07:21:46.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0a8290 msgr2=0x7f1adc1dfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0a8290 0x7f1adc1dfa00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c430 tx=0x7f1ad800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 shutdown_connections 2024-03-06T07:21:46.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ac006b4d0 0x7f1ac006d990 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1acc035000 tx=0x7f1acc008030 comp rx=0 tx=0).stop 2024-03-06T07:21:46.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0c69a0 0x7f1adc1ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc0c4740 0x7f1adc1ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 --2- 172.21.15.26:0/3627345558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1adc0a8290 0x7f1adc1dfa00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 >> 172.21.15.26:0/3627345558 conn(0x7f1adc0c5560 msgr2=0x7f1adc0c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:46.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.322+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 shutdown_connections 2024-03-06T07:21:46.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.323+0000 7f1ae3713640 1 -- 172.21.15.26:0/3627345558 wait complete. 2024-03-06T07:21:46.336 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:21:46.337 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.436+0000 7f4ee0a15640 1 Processor -- start 2024-03-06T07:21:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.436+0000 7f4ee0a15640 1 -- start start 2024-03-06T07:21:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.437+0000 7f4ee0a15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 0x7f4edc155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.437+0000 7f4ee0a15640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0c5cc0 con 0x7f4edc0c5860 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.437+0000 7f4ee0a15640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc1591c0 con 0x7f4edc1586f0 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.437+0000 7f4ee0a15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06df40 con 0x7f4edc0c5480 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.437+0000 7f4ed9d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4edc0c5860 0x7f4edc06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36534/0 (socket says 172.21.15.26:36534) 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.437+0000 7f4ed9d9b640 1 -- 172.21.15.26:0/1982791074 learned_addr learned my addr 172.21.15.26:0/1982791074 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4eda59c640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 0x7f4edc155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238004459 0 0) 0x7f4edc0c5cc0 con 0x7f4edc0c5860 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec4003520 con 0x7f4edc0c5860 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 364976772 0 0) 0x7f4ec4003520 con 0x7f4edc0c5860 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 msgr2=0x7f4edc155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 0x7f4edc155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 >> v1:172.21.15.117:6789/0 conn(0x7f4edc1586f0 legacy=0x7f4edc15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc1594a0 con 0x7f4edc0c5860 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4ed0001670 con 0x7f4edc0c5860 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 >> v1:172.21.15.26:6789/0 conn(0x7f4edc0c5860 legacy=0x7f4edc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec40039e0 0x7f4ec4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 0x7f4ec4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec4004d40 0x7f4ec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06df40 con 0x7f4ec40039e0 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400baf0 con 0x7f4ec4004d40 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400bcf0 con 0x7f4edc0c5480 2024-03-06T07:21:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4eda59c640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec40039e0 0x7f4ec4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4ed9d9b640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 0x7f4ec4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4edad9d640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec4004d40 0x7f4ec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4eda59c640 1 -- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 msgr2=0x7f4ec4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4eda59c640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc0c5480 0x7f4ec4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4eda59c640 1 -- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec4004d40 msgr2=0x7f4ec40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4eda59c640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec4004d40 0x7f4ec40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.438+0000 7f4eda59c640 1 -- 172.21.15.26:0/1982791074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4003520 con 0x7f4ec40039e0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.439+0000 7f4eda59c640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec40039e0 0x7f4ec4003e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009fc0 tx=0x7f4ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc85ef570836134 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.439+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc039070 con 0x7f4ec40039e0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.439+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc02cb40 con 0x7f4ec40039e0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.439+0000 7f4ed959a640 1 -- 172.21.15.26:0/1982791074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc009c10 con 0x7f4ec40039e0 2024-03-06T07:21:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.440+0000 7f4ee0a15640 1 -- 172.21.15.26:0/1982791074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec40039e0 msgr2=0x7f4ec4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.440+0000 7f4ee0a15640 1 --2- 172.21.15.26:0/1982791074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec40039e0 0x7f4ec4003e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009fc0 tx=0x7f4ec4005830 comp rx=0 tx=0).stop 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.440+0000 7f4edad9d640 1 -- 172.21.15.26:0/1982791074 reap_dead start 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.440+0000 7f4ee0a15640 1 -- 172.21.15.26:0/1982791074 shutdown_connections 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.440+0000 7f4ee0a15640 1 -- 172.21.15.26:0/1982791074 >> 172.21.15.26:0/1982791074 conn(0x7f4edc06b1a0 msgr2=0x7f4edc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.440+0000 7f4ee0a15640 1 -- 172.21.15.26:0/1982791074 shutdown_connections 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.440+0000 7f4ee0a15640 1 -- 172.21.15.26:0/1982791074 wait complete. 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.441+0000 7f4ee0a15640 1 Processor -- start 2024-03-06T07:21:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.441+0000 7f4ee0a15640 1 -- start start 2024-03-06T07:21:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4ee0a15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec40039e0 0x7f4edc1ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4ee0a15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc14c130 0x7f4edc151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4ee0a15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4edc1eb3f0 0x7f4edc1eb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4ee0a15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc155680 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4ee0a15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06fdb0 con 0x7f4ec40039e0 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4ee0a15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc152a20 con 0x7f4edc14c130 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4edc1eb3f0 0x7f4edc1eb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4edc1eb3f0 0x7f4edc1eb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46068/0 (socket says 172.21.15.26:46068) 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 -- 172.21.15.26:0/281572323 learned_addr learned my addr 172.21.15.26:0/281572323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4ed9d9b640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc14c130 0x7f4edc151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4eda59c640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec40039e0 0x7f4edc1ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 -- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc14c130 msgr2=0x7f4edc151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc14c130 0x7f4edc151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 -- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec40039e0 msgr2=0x7f4edc1ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec40039e0 0x7f4edc1ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.442+0000 7f4edad9d640 1 -- 172.21.15.26:0/281572323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc1ee910 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.443+0000 7f4edad9d640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4edc1eb3f0 0x7f4edc1eb870 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed400e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.443+0000 7f4ecaffd640 1 -- 172.21.15.26:0/281572323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4019070 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.443+0000 7f4ecaffd640 1 -- 172.21.15.26:0/281572323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400ee70 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.443+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc1eebf0 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.443+0000 7f4ecaffd640 1 -- 172.21.15.26:0/281572323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40035f0 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.443+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1ef1b0 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.444+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc04f330 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.444+0000 7f4ecaffd640 1 -- 172.21.15.26:0/281572323 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ed401c070 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.445+0000 7f4ecaffd640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ebc06b500 0x7f4ebc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.445+0000 7f4ecaffd640 1 -- 172.21.15.26:0/281572323 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ed40895f0 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.445+0000 7f4eda59c640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ebc06b500 0x7f4ebc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.445+0000 7f4eda59c640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ebc06b500 0x7f4ebc06d9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc036000 tx=0x7f4ecc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.449+0000 7f4ecaffd640 1 -- 172.21.15.26:0/281572323 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4ed4057c90 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.667+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4edc06d4b0 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.667+0000 7f4ecaffd640 1 -- 172.21.15.26:0/281572323 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4977 (secure 0 0 0) 0x7f4ed405bd30 con 0x7f4edc1eb3f0 2024-03-06T07:21:46.669 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:46.669 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1383022235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1383022235},{"type":"v1","addr":"172.21.15.26:6834","nonce":1383022235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:21:45.677934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/440323369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":440323369},{"type":"v1","addr":"172.21.15.26:6835","nonce":440323369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":3}]} 2024-03-06T07:21:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-03-06T07:21:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ebc06b500 msgr2=0x7f4ebc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ebc06b500 0x7f4ebc06d9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc036000 tx=0x7f4ecc02e000 comp rx=0 tx=0).stop 2024-03-06T07:21:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4edc1eb3f0 msgr2=0x7f4edc1eb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4edc1eb3f0 0x7f4edc1eb870 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed400e670 comp rx=0 tx=0).stop 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 shutdown_connections 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ebc06b500 0x7f4ebc06d9c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4edc1eb3f0 0x7f4edc1eb870 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4edc14c130 0x7f4edc151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 --2- 172.21.15.26:0/281572323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec40039e0 0x7f4edc1ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 >> 172.21.15.26:0/281572323 conn(0x7f4edc06b1a0 msgr2=0x7f4edc1537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 shutdown_connections 2024-03-06T07:21:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.670+0000 7f4ee0a15640 1 -- 172.21.15.26:0/281572323 wait complete. 2024-03-06T07:21:46.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.783+0000 7f6400688640 1 Processor -- start 2024-03-06T07:21:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.783+0000 7f6400688640 1 -- start start 2024-03-06T07:21:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f6400688640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f6400688640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63f8155a40 0x7f63f8155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f6400688640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f8156410 con 0x7f63f814f790 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f6400688640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f80c2680 con 0x7f63f8155a40 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f6400688640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80c28d0 con 0x7f63f80c1aa0 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fec25640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63f8155a40 0x7f63f8155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fe424640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f63f814f790 0x7f63f80c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36542/0 (socket says 172.21.15.26:36542) 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fec25640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63f8155a40 0x7f63f8155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46078/0 (socket says 172.21.15.26:46078) 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fe424640 1 -- 172.21.15.26:0/4170353514 learned_addr learned my addr 172.21.15.26:0/4170353514 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fec25640 1 -- 172.21.15.26:0/4170353514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 msgr2=0x7f63f81529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fdc23640 1 --2- 172.21.15.26:0/4170353514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fec25640 1 --2- 172.21.15.26:0/4170353514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f81529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fec25640 1 -- 172.21.15.26:0/4170353514 >> v1:172.21.15.26:6789/0 conn(0x7f63f814f790 legacy=0x7f63f80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fd422640 1 -- 172.21.15.26:0/4170353514 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298037981 0 0) 0x7f63f8156410 con 0x7f63f814f790 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fec25640 1 -- 172.21.15.26:0/4170353514 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80c2bb0 con 0x7f63f8155a40 2024-03-06T07:21:46.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.784+0000 7f63fdc23640 1 --2- 172.21.15.26:0/4170353514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.785+0000 7f63fec25640 1 --2- 172.21.15.26:0/4170353514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f8155a40 0x7f63f8155ed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f63f000bfa0 tx=0x7f63f002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd3be2d6d88a0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.785+0000 7f63fd422640 1 -- 172.21.15.26:0/4170353514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f002ea40 con 0x7f63f8155a40 2024-03-06T07:21:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.785+0000 7f63fd422640 1 -- 172.21.15.26:0/4170353514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f002ebe0 con 0x7f63f8155a40 2024-03-06T07:21:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.785+0000 7f63fd422640 1 -- 172.21.15.26:0/4170353514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0009540 con 0x7f63f8155a40 2024-03-06T07:21:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.785+0000 7f6400688640 1 -- 172.21.15.26:0/4170353514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f8155a40 msgr2=0x7f63f8155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.785+0000 7f6400688640 1 --2- 172.21.15.26:0/4170353514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f8155a40 0x7f63f8155ed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f63f000bfa0 tx=0x7f63f002e0a0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.786+0000 7f6400688640 1 -- 172.21.15.26:0/4170353514 shutdown_connections 2024-03-06T07:21:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.786+0000 7f6400688640 1 --2- 172.21.15.26:0/4170353514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f8155a40 0x7f63f8155ed0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.786+0000 7f6400688640 1 --2- 172.21.15.26:0/4170353514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.786+0000 7f6400688640 1 -- 172.21.15.26:0/4170353514 >> 172.21.15.26:0/4170353514 conn(0x7f63f80c6710 msgr2=0x7f63f80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.786+0000 7f6400688640 1 -- 172.21.15.26:0/4170353514 shutdown_connections 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.786+0000 7f6400688640 1 -- 172.21.15.26:0/4170353514 wait complete. 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 Processor -- start 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 -- start start 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f80730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f814f790 0x7f63f8071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8155a40 0x7f63f8150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80c0d60 con 0x7f63f814f790 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8155520 con 0x7f63f8155a40 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.787+0000 7f6400688640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80c5e30 con 0x7f63f80c1aa0 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f814f790 0x7f63f8071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f814f790 0x7f63f8071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46082/0 (socket says 172.21.15.26:46082) 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 -- 172.21.15.26:0/3319784552 learned_addr learned my addr 172.21.15.26:0/3319784552 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fe424640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f80730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fec25640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8155a40 0x7f63f8150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 -- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 msgr2=0x7f63f80730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f80730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 -- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8155a40 msgr2=0x7f63f8150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8155a40 0x7f63f8150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 -- 172.21.15.26:0/3319784552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f81ea860 con 0x7f63f814f790 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fe424640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f80730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63fdc23640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f814f790 0x7f63f8071320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c7f0 tx=0x7f63e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63eeffd640 1 -- 172.21.15.26:0/3319784552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8015070 con 0x7f63f814f790 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63eeffd640 1 -- 172.21.15.26:0/3319784552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8007a30 con 0x7f63f814f790 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f81eab40 con 0x7f63f814f790 2024-03-06T07:21:46.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f63eeffd640 1 -- 172.21.15.26:0/3319784552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8007e00 con 0x7f63f814f790 2024-03-06T07:21:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.788+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f8151fb0 con 0x7f63f814f790 2024-03-06T07:21:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.789+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f804f330 con 0x7f63f814f790 2024-03-06T07:21:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.790+0000 7f63eeffd640 1 -- 172.21.15.26:0/3319784552 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63e8018070 con 0x7f63f814f790 2024-03-06T07:21:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.791+0000 7f63eeffd640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f63dc06b4d0 0x7f63dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.791+0000 7f63eeffd640 1 -- 172.21.15.26:0/3319784552 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63e808a2b0 con 0x7f63f814f790 2024-03-06T07:21:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.791+0000 7f63fe424640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f63dc06b4d0 0x7f63dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.792+0000 7f63fe424640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f63dc06b4d0 0x7f63dc06d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000f60 tx=0x7f63f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.795+0000 7f63eeffd640 1 -- 172.21.15.26:0/3319784552 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f63e8059030 con 0x7f63f814f790 2024-03-06T07:21:46.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.984+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63f80c1f40 con 0x7f63f814f790 2024-03-06T07:21:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.985+0000 7f63eeffd640 1 -- 172.21.15.26:0/3319784552 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16954 (secure 0 0 0) 0x7f63e8058930 con 0x7f63f814f790 2024-03-06T07:21:46.987 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:46.987 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":50,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:44.311940+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f63dc06b4d0 msgr2=0x7f63dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f63dc06b4d0 0x7f63dc06d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000f60 tx=0x7f63f4009040 comp rx=0 tx=0).stop 2024-03-06T07:21:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f814f790 msgr2=0x7f63f8071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f814f790 0x7f63f8071320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c7f0 tx=0x7f63e800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 shutdown_connections 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f63dc06b4d0 0x7f63dc06d990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8155a40 0x7f63f8150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63f814f790 0x7f63f8071320 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 --2- 172.21.15.26:0/3319784552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f80c1aa0 0x7f63f80730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 >> 172.21.15.26:0/3319784552 conn(0x7f63f80c6710 msgr2=0x7f63f80c42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 shutdown_connections 2024-03-06T07:21:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:46.987+0000 7f6400688640 1 -- 172.21.15.26:0/3319784552 wait complete. 2024-03-06T07:21:47.001 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:21:47.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:47.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.100+0000 7f410298a640 1 Processor -- start 2024-03-06T07:21:47.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.101+0000 7f410298a640 1 -- start start 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.101+0000 7f410298a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40fc0c5480 0x7f40fc155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.101+0000 7f410298a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc1586f0 0x7f40fc158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.101+0000 7f410298a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc0c5cc0 con 0x7f40fc0c5860 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.101+0000 7f410298a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc06df40 con 0x7f40fc0c5480 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.101+0000 7f410298a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06fdb0 con 0x7f40fc1586f0 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fbfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40fc0c5480 0x7f40fc155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f4100f27640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc1586f0 0x7f40fc158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fbfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40fc0c5480 0x7f40fc155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34092/0 (socket says 172.21.15.26:34092) 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fbfff640 1 -- 172.21.15.26:0/1470321803 learned_addr learned my addr 172.21.15.26:0/1470321803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fb7fe640 1 --1- 172.21.15.26:0/1470321803 >> v1:172.21.15.117:6789/0 conn(0x7f40fc0c5860 0x7f40fc06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38770/0 (socket says 172.21.15.26:38770) 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fbfff640 1 -- 172.21.15.26:0/1470321803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc1586f0 msgr2=0x7f40fc158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fbfff640 1 --2- 172.21.15.26:0/1470321803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc1586f0 0x7f40fc158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fbfff640 1 -- 172.21.15.26:0/1470321803 >> v1:172.21.15.117:6789/0 conn(0x7f40fc0c5860 legacy=0x7f40fc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.102+0000 7f40fbfff640 1 -- 172.21.15.26:0/1470321803 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc1591c0 con 0x7f40fc0c5480 2024-03-06T07:21:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.103+0000 7f40fbfff640 1 --2- 172.21.15.26:0/1470321803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc155ba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009870 tx=0x7f40ec02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8aefec42ef7a10e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.103+0000 7f40faffd640 1 -- 172.21.15.26:0/1470321803 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec035020 con 0x7f40fc0c5480 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.103+0000 7f40faffd640 1 -- 172.21.15.26:0/1470321803 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec031640 con 0x7f40fc0c5480 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.103+0000 7f40faffd640 1 -- 172.21.15.26:0/1470321803 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec02d040 con 0x7f40fc0c5480 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.103+0000 7f410298a640 1 -- 172.21.15.26:0/1470321803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 msgr2=0x7f40fc155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.103+0000 7f410298a640 1 --2- 172.21.15.26:0/1470321803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc155ba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009870 tx=0x7f40ec02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.104+0000 7f410298a640 1 -- 172.21.15.26:0/1470321803 shutdown_connections 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.104+0000 7f410298a640 1 --2- 172.21.15.26:0/1470321803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc1586f0 0x7f40fc158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.104+0000 7f410298a640 1 --2- 172.21.15.26:0/1470321803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc155ba0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.104+0000 7f410298a640 1 -- 172.21.15.26:0/1470321803 >> 172.21.15.26:0/1470321803 conn(0x7f40fc06b1a0 msgr2=0x7f40fc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.104+0000 7f410298a640 1 -- 172.21.15.26:0/1470321803 shutdown_connections 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.104+0000 7f410298a640 1 -- 172.21.15.26:0/1470321803 wait complete. 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 Processor -- start 2024-03-06T07:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 -- start start 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc0c5860 0x7f40fc150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40fc1586f0 0x7f40fc151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc155680 con 0x7f40fc1586f0 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc1553a0 con 0x7f40fc0c5480 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.105+0000 7f410298a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc152a20 con 0x7f40fc0c5860 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40fc1586f0 0x7f40fc151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40fc1586f0 0x7f40fc151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49930/0 (socket says 172.21.15.26:49930) 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 -- 172.21.15.26:0/1639640982 learned_addr learned my addr 172.21.15.26:0/1639640982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f40fbfff640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f40fb7fe640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc0c5860 0x7f40fc150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 -- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc0c5860 msgr2=0x7f40fc150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc0c5860 0x7f40fc150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 -- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 msgr2=0x7f40fc1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 -- 172.21.15.26:0/1639640982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc0c4df0 con 0x7f40fc1586f0 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f40fbfff640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f4100f27640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40fc1586f0 0x7f40fc151250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f40f400ec40 tx=0x7f40f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f40f8ff9640 1 -- 172.21.15.26:0/1639640982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4018020 con 0x7f40fc1586f0 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f40f8ff9640 1 -- 172.21.15.26:0/1639640982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f400ee50 con 0x7f40fc1586f0 2024-03-06T07:21:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc0c4b10 con 0x7f40fc1586f0 2024-03-06T07:21:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f40f8ff9640 1 -- 172.21.15.26:0/1639640982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40146c0 con 0x7f40fc1586f0 2024-03-06T07:21:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.106+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc150510 con 0x7f40fc1586f0 2024-03-06T07:21:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.107+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc04f330 con 0x7f40fc1586f0 2024-03-06T07:21:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.108+0000 7f40f8ff9640 1 -- 172.21.15.26:0/1639640982 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40f4005d10 con 0x7f40fc1586f0 2024-03-06T07:21:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.109+0000 7f40f8ff9640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40e806b4d0 0x7f40e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.109+0000 7f40f8ff9640 1 -- 172.21.15.26:0/1639640982 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40f408b000 con 0x7f40fc1586f0 2024-03-06T07:21:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.109+0000 7f40fbfff640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40e806b4d0 0x7f40e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.109+0000 7f40fbfff640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40e806b4d0 0x7f40e806d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40ec036000 tx=0x7f40ec03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.113+0000 7f40f8ff9640 1 -- 172.21.15.26:0/1639640982 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40f4022030 con 0x7f40fc1586f0 2024-03-06T07:21:47.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.291+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40fc06e0c0 con 0x7f40e806b4d0 2024-03-06T07:21:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.297+0000 7f40f8ff9640 1 -- 172.21.15.26:0/1639640982 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7f40fc06e0c0 con 0x7f40e806b4d0 2024-03-06T07:21:47.305 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40e806b4d0 msgr2=0x7f40e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40e806b4d0 0x7f40e806d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40ec036000 tx=0x7f40ec03c040 comp rx=0 tx=0).stop 2024-03-06T07:21:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40fc1586f0 msgr2=0x7f40fc151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40fc1586f0 0x7f40fc151250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f40f400ec40 tx=0x7f40f400c490 comp rx=0 tx=0).stop 2024-03-06T07:21:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 shutdown_connections 2024-03-06T07:21:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f40e806b4d0 0x7f40e806d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40fc1586f0 0x7f40fc151250 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40fc0c5860 0x7f40fc150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 --2- 172.21.15.26:0/1639640982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40fc0c5480 0x7f40fc1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 >> 172.21.15.26:0/1639640982 conn(0x7f40fc06b1a0 msgr2=0x7f40fc0c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 shutdown_connections 2024-03-06T07:21:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.300+0000 7f410298a640 1 -- 172.21.15.26:0/1639640982 wait complete. 2024-03-06T07:21:47.313 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-03-06T07:21:47.051463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237648,"kb_used_data":4808,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737072,"statfs":{"total":773094113280,"available":772850761728,"internally_reserved":0,"allocated":4923392,"data_stored":3281580,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759424"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608964+0000","last_change":"2024-03-06T07:21:43.317811+0000","last_active":"2024-03-06T07:21:44.608964+0000","last_peered":"2024-03-06T07:21:44.608964+0000","last_clean":"2024-03-06T07:21:44.608964+0000","last_became_active":"2024-03-06T07:21:43.317509+0000","last_became_peered":"2024-03-06T07:21:43.317509+0000","last_unstale":"2024-03-06T07:21:44.608964+0000","last_undegraded":"2024-03-06T07:21:44.608964+0000","last_fullsized":"2024-03-06T07:21:44.608964+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335820+0000","last_change":"2024-03-06T07:21:43.317801+0000","last_active":"2024-03-06T07:21:44.335820+0000","last_peered":"2024-03-06T07:21:44.335820+0000","last_clean":"2024-03-06T07:21:44.335820+0000","last_became_active":"2024-03-06T07:21:43.317522+0000","last_became_peered":"2024-03-06T07:21:43.317522+0000","last_unstale":"2024-03-06T07:21:44.335820+0000","last_undegraded":"2024-03-06T07:21:44.335820+0000","last_fullsized":"2024-03-06T07:21:44.335820+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333258+0000","last_change":"2024-03-06T07:21:43.328071+0000","last_active":"2024-03-06T07:21:44.333258+0000","last_peered":"2024-03-06T07:21:44.333258+0000","last_clean":"2024-03-06T07:21:44.333258+0000","last_became_active":"2024-03-06T07:21:43.327781+0000","last_became_peered":"2024-03-06T07:21:43.327781+0000","last_unstale":"2024-03-06T07:21:44.333258+0000","last_undegraded":"2024-03-06T07:21:44.333258+0000","last_fullsized":"2024-03-06T07:21:44.333258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335595+0000","last_change":"2024-03-06T07:21:43.326286+0000","last_active":"2024-03-06T07:21:44.335595+0000","last_peered":"2024-03-06T07:21:44.335595+0000","last_clean":"2024-03-06T07:21:44.335595+0000","last_became_active":"2024-03-06T07:21:43.326031+0000","last_became_peered":"2024-03-06T07:21:43.326031+0000","last_unstale":"2024-03-06T07:21:44.335595+0000","last_undegraded":"2024-03-06T07:21:44.335595+0000","last_fullsized":"2024-03-06T07:21:44.335595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:21.645487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318644+0000","last_change":"2024-03-06T07:21:43.321810+0000","last_active":"2024-03-06T07:21:44.318644+0000","last_peered":"2024-03-06T07:21:44.318644+0000","last_clean":"2024-03-06T07:21:44.318644+0000","last_became_active":"2024-03-06T07:21:43.321538+0000","last_became_peered":"2024-03-06T07:21:43.321538+0000","last_unstale":"2024-03-06T07:21:44.318644+0000","last_undegraded":"2024-03-06T07:21:44.318644+0000","last_fullsized":"2024-03-06T07:21:44.318644+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:10.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608702+0000","last_change":"2024-03-06T07:21:43.318713+0000","last_active":"2024-03-06T07:21:44.608702+0000","last_peered":"2024-03-06T07:21:44.608702+0000","last_clean":"2024-03-06T07:21:44.608702+0000","last_became_active":"2024-03-06T07:21:43.318385+0000","last_became_peered":"2024-03-06T07:21:43.318385+0000","last_unstale":"2024-03-06T07:21:44.608702+0000","last_undegraded":"2024-03-06T07:21:44.608702+0000","last_fullsized":"2024-03-06T07:21:44.608702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332693+0000","last_change":"2024-03-06T07:21:43.327099+0000","last_active":"2024-03-06T07:21:44.332693+0000","last_peered":"2024-03-06T07:21:44.332693+0000","last_clean":"2024-03-06T07:21:44.332693+0000","last_became_active":"2024-03-06T07:21:43.326851+0000","last_became_peered":"2024-03-06T07:21:43.326851+0000","last_unstale":"2024-03-06T07:21:44.332693+0000","last_undegraded":"2024-03-06T07:21:44.332693+0000","last_fullsized":"2024-03-06T07:21:44.332693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317670+0000","last_change":"2024-03-06T07:21:43.325897+0000","last_active":"2024-03-06T07:21:44.317670+0000","last_peered":"2024-03-06T07:21:44.317670+0000","last_clean":"2024-03-06T07:21:44.317670+0000","last_became_active":"2024-03-06T07:21:43.325647+0000","last_became_peered":"2024-03-06T07:21:43.325647+0000","last_unstale":"2024-03-06T07:21:44.317670+0000","last_undegraded":"2024-03-06T07:21:44.317670+0000","last_fullsized":"2024-03-06T07:21:44.317670+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318058+0000","last_change":"2024-03-06T07:21:43.330337+0000","last_active":"2024-03-06T07:21:44.318058+0000","last_peered":"2024-03-06T07:21:44.318058+0000","last_clean":"2024-03-06T07:21:44.318058+0000","last_became_active":"2024-03-06T07:21:43.330053+0000","last_became_peered":"2024-03-06T07:21:43.330053+0000","last_unstale":"2024-03-06T07:21:44.318058+0000","last_undegraded":"2024-03-06T07:21:44.318058+0000","last_fullsized":"2024-03-06T07:21:44.318058+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608988+0000","last_change":"2024-03-06T07:21:43.325100+0000","last_active":"2024-03-06T07:21:44.608988+0000","last_peered":"2024-03-06T07:21:44.608988+0000","last_clean":"2024-03-06T07:21:44.608988+0000","last_became_active":"2024-03-06T07:21:43.324812+0000","last_became_peered":"2024-03-06T07:21:43.324812+0000","last_unstale":"2024-03-06T07:21:44.608988+0000","last_undegraded":"2024-03-06T07:21:44.608988+0000","last_fullsized":"2024-03-06T07:21:44.608988+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:11.442368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608633+0000","last_change":"2024-03-06T07:21:43.322188+0000","last_active":"2024-03-06T07:21:44.608633+0000","last_peered":"2024-03-06T07:21:44.608633+0000","last_clean":"2024-03-06T07:21:44.608633+0000","last_became_active":"2024-03-06T07:21:43.321685+0000","last_became_peered":"2024-03-06T07:21:43.321685+0000","last_unstale":"2024-03-06T07:21:44.608633+0000","last_undegraded":"2024-03-06T07:21:44.608633+0000","last_fullsized":"2024-03-06T07:21:44.608633+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:54.799043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318167+0000","last_change":"2024-03-06T07:21:43.321952+0000","last_active":"2024-03-06T07:21:44.318167+0000","last_peered":"2024-03-06T07:21:44.318167+0000","last_clean":"2024-03-06T07:21:44.318167+0000","last_became_active":"2024-03-06T07:21:43.321579+0000","last_became_peered":"2024-03-06T07:21:43.321579+0000","last_unstale":"2024-03-06T07:21:44.318167+0000","last_undegraded":"2024-03-06T07:21:44.318167+0000","last_fullsized":"2024-03-06T07:21:44.318167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:04.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608940+0000","last_change":"2024-03-06T07:21:43.321277+0000","last_active":"2024-03-06T07:21:44.608940+0000","last_peered":"2024-03-06T07:21:44.608940+0000","last_clean":"2024-03-06T07:21:44.608940+0000","last_became_active":"2024-03-06T07:21:43.320973+0000","last_became_peered":"2024-03-06T07:21:43.320973+0000","last_unstale":"2024-03-06T07:21:44.608940+0000","last_undegraded":"2024-03-06T07:21:44.608940+0000","last_fullsized":"2024-03-06T07:21:44.608940+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:15.957299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332780+0000","last_change":"2024-03-06T07:21:43.329915+0000","last_active":"2024-03-06T07:21:44.332780+0000","last_peered":"2024-03-06T07:21:44.332780+0000","last_clean":"2024-03-06T07:21:44.332780+0000","last_became_active":"2024-03-06T07:21:43.328775+0000","last_became_peered":"2024-03-06T07:21:43.328775+0000","last_unstale":"2024-03-06T07:21:44.332780+0000","last_undegraded":"2024-03-06T07:21:44.332780+0000","last_fullsized":"2024-03-06T07:21:44.332780+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335398+0000","last_change":"2024-03-06T07:21:43.321137+0000","last_active":"2024-03-06T07:21:44.335398+0000","last_peered":"2024-03-06T07:21:44.335398+0000","last_clean":"2024-03-06T07:21:44.335398+0000","last_became_active":"2024-03-06T07:21:43.320820+0000","last_became_peered":"2024-03-06T07:21:43.320820+0000","last_unstale":"2024-03-06T07:21:44.335398+0000","last_undegraded":"2024-03-06T07:21:44.335398+0000","last_fullsized":"2024-03-06T07:21:44.335398+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:52.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608712+0000","last_change":"2024-03-06T07:21:43.320518+0000","last_active":"2024-03-06T07:21:44.608712+0000","last_peered":"2024-03-06T07:21:44.608712+0000","last_clean":"2024-03-06T07:21:44.608712+0000","last_became_active":"2024-03-06T07:21:43.320178+0000","last_became_peered":"2024-03-06T07:21:43.320178+0000","last_unstale":"2024-03-06T07:21:44.608712+0000","last_undegraded":"2024-03-06T07:21:44.608712+0000","last_fullsized":"2024-03-06T07:21:44.608712+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333101+0000","last_change":"2024-03-06T07:21:43.324279+0000","last_active":"2024-03-06T07:21:44.333101+0000","last_peered":"2024-03-06T07:21:44.333101+0000","last_clean":"2024-03-06T07:21:44.333101+0000","last_became_active":"2024-03-06T07:21:43.323499+0000","last_became_peered":"2024-03-06T07:21:43.323499+0000","last_unstale":"2024-03-06T07:21:44.333101+0000","last_undegraded":"2024-03-06T07:21:44.333101+0000","last_fullsized":"2024-03-06T07:21:44.333101+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335836+0000","last_change":"2024-03-06T07:21:43.318202+0000","last_active":"2024-03-06T07:21:44.335836+0000","last_peered":"2024-03-06T07:21:44.335836+0000","last_clean":"2024-03-06T07:21:44.335836+0000","last_became_active":"2024-03-06T07:21:43.317929+0000","last_became_peered":"2024-03-06T07:21:43.317929+0000","last_unstale":"2024-03-06T07:21:44.335836+0000","last_undegraded":"2024-03-06T07:21:44.335836+0000","last_fullsized":"2024-03-06T07:21:44.335836+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333309+0000","last_change":"2024-03-06T07:21:43.324265+0000","last_active":"2024-03-06T07:21:44.333309+0000","last_peered":"2024-03-06T07:21:44.333309+0000","last_clean":"2024-03-06T07:21:44.333309+0000","last_became_active":"2024-03-06T07:21:43.323877+0000","last_became_peered":"2024-03-06T07:21:43.323877+0000","last_unstale":"2024-03-06T07:21:44.333309+0000","last_undegraded":"2024-03-06T07:21:44.333309+0000","last_fullsized":"2024-03-06T07:21:44.333309+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609270+0000","last_change":"2024-03-06T07:21:43.321580+0000","last_active":"2024-03-06T07:21:44.609270+0000","last_peered":"2024-03-06T07:21:44.609270+0000","last_clean":"2024-03-06T07:21:44.609270+0000","last_became_active":"2024-03-06T07:21:43.319775+0000","last_became_peered":"2024-03-06T07:21:43.319775+0000","last_unstale":"2024-03-06T07:21:44.609270+0000","last_undegraded":"2024-03-06T07:21:44.609270+0000","last_fullsized":"2024-03-06T07:21:44.609270+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:32.802384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609293+0000","last_change":"2024-03-06T07:21:43.322797+0000","last_active":"2024-03-06T07:21:44.609293+0000","last_peered":"2024-03-06T07:21:44.609293+0000","last_clean":"2024-03-06T07:21:44.609293+0000","last_became_active":"2024-03-06T07:21:43.322505+0000","last_became_peered":"2024-03-06T07:21:43.322505+0000","last_unstale":"2024-03-06T07:21:44.609293+0000","last_undegraded":"2024-03-06T07:21:44.609293+0000","last_fullsized":"2024-03-06T07:21:44.609293+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317902+0000","last_change":"2024-03-06T07:21:43.326058+0000","last_active":"2024-03-06T07:21:44.317902+0000","last_peered":"2024-03-06T07:21:44.317902+0000","last_clean":"2024-03-06T07:21:44.317902+0000","last_became_active":"2024-03-06T07:21:43.325799+0000","last_became_peered":"2024-03-06T07:21:43.325799+0000","last_unstale":"2024-03-06T07:21:44.317902+0000","last_undegraded":"2024-03-06T07:21:44.317902+0000","last_fullsized":"2024-03-06T07:21:44.317902+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608601+0000","last_change":"2024-03-06T07:21:43.322671+0000","last_active":"2024-03-06T07:21:44.608601+0000","last_peered":"2024-03-06T07:21:44.608601+0000","last_clean":"2024-03-06T07:21:44.608601+0000","last_became_active":"2024-03-06T07:21:43.322376+0000","last_became_peered":"2024-03-06T07:21:43.322376+0000","last_unstale":"2024-03-06T07:21:44.608601+0000","last_undegraded":"2024-03-06T07:21:44.608601+0000","last_fullsized":"2024-03-06T07:21:44.608601+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:15.049663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332658+0000","last_change":"2024-03-06T07:21:43.327014+0000","last_active":"2024-03-06T07:21:44.332658+0000","last_peered":"2024-03-06T07:21:44.332658+0000","last_clean":"2024-03-06T07:21:44.332658+0000","last_became_active":"2024-03-06T07:21:43.326759+0000","last_became_peered":"2024-03-06T07:21:43.326759+0000","last_unstale":"2024-03-06T07:21:44.332658+0000","last_undegraded":"2024-03-06T07:21:44.332658+0000","last_fullsized":"2024-03-06T07:21:44.332658+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333245+0000","last_change":"2024-03-06T07:21:43.323199+0000","last_active":"2024-03-06T07:21:44.333245+0000","last_peered":"2024-03-06T07:21:44.333245+0000","last_clean":"2024-03-06T07:21:44.333245+0000","last_became_active":"2024-03-06T07:21:43.322835+0000","last_became_peered":"2024-03-06T07:21:43.322835+0000","last_unstale":"2024-03-06T07:21:44.333245+0000","last_undegraded":"2024-03-06T07:21:44.333245+0000","last_fullsized":"2024-03-06T07:21:44.333245+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317893+0000","last_change":"2024-03-06T07:21:43.318509+0000","last_active":"2024-03-06T07:21:44.317893+0000","last_peered":"2024-03-06T07:21:44.317893+0000","last_clean":"2024-03-06T07:21:44.317893+0000","last_became_active":"2024-03-06T07:21:43.318177+0000","last_became_peered":"2024-03-06T07:21:43.318177+0000","last_unstale":"2024-03-06T07:21:44.317893+0000","last_undegraded":"2024-03-06T07:21:44.317893+0000","last_fullsized":"2024-03-06T07:21:44.317893+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609147+0000","last_change":"2024-03-06T07:21:43.332733+0000","last_active":"2024-03-06T07:21:44.609147+0000","last_peered":"2024-03-06T07:21:44.609147+0000","last_clean":"2024-03-06T07:21:44.609147+0000","last_became_active":"2024-03-06T07:21:43.331512+0000","last_became_peered":"2024-03-06T07:21:43.331512+0000","last_unstale":"2024-03-06T07:21:44.609147+0000","last_undegraded":"2024-03-06T07:21:44.609147+0000","last_fullsized":"2024-03-06T07:21:44.609147+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609342+0000","last_change":"2024-03-06T07:21:43.321054+0000","last_active":"2024-03-06T07:21:44.609342+0000","last_peered":"2024-03-06T07:21:44.609342+0000","last_clean":"2024-03-06T07:21:44.609342+0000","last_became_active":"2024-03-06T07:21:43.319435+0000","last_became_peered":"2024-03-06T07:21:43.319435+0000","last_unstale":"2024-03-06T07:21:44.609342+0000","last_undegraded":"2024-03-06T07:21:44.609342+0000","last_fullsized":"2024-03-06T07:21:44.609342+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332809+0000","last_change":"2024-03-06T07:21:43.328812+0000","last_active":"2024-03-06T07:21:44.332809+0000","last_peered":"2024-03-06T07:21:44.332809+0000","last_clean":"2024-03-06T07:21:44.332809+0000","last_became_active":"2024-03-06T07:21:43.328366+0000","last_became_peered":"2024-03-06T07:21:43.328366+0000","last_unstale":"2024-03-06T07:21:44.332809+0000","last_undegraded":"2024-03-06T07:21:44.332809+0000","last_fullsized":"2024-03-06T07:21:44.332809+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:48.381105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335423+0000","last_change":"2024-03-06T07:21:43.321155+0000","last_active":"2024-03-06T07:21:44.335423+0000","last_peered":"2024-03-06T07:21:44.335423+0000","last_clean":"2024-03-06T07:21:44.335423+0000","last_became_active":"2024-03-06T07:21:43.320747+0000","last_became_peered":"2024-03-06T07:21:43.320747+0000","last_unstale":"2024-03-06T07:21:44.335423+0000","last_undegraded":"2024-03-06T07:21:44.335423+0000","last_fullsized":"2024-03-06T07:21:44.335423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:42.424597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332779+0000","last_change":"2024-03-06T07:21:43.326550+0000","last_active":"2024-03-06T07:21:44.332779+0000","last_peered":"2024-03-06T07:21:44.332779+0000","last_clean":"2024-03-06T07:21:44.332779+0000","last_became_active":"2024-03-06T07:21:43.326254+0000","last_became_peered":"2024-03-06T07:21:43.326254+0000","last_unstale":"2024-03-06T07:21:44.332779+0000","last_undegraded":"2024-03-06T07:21:44.332779+0000","last_fullsized":"2024-03-06T07:21:44.332779+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447732+0000","last_change":"2024-03-06T07:21:43.322773+0000","last_active":"2024-03-06T07:21:44.447732+0000","last_peered":"2024-03-06T07:21:44.447732+0000","last_clean":"2024-03-06T07:21:44.447732+0000","last_became_active":"2024-03-06T07:21:43.322525+0000","last_became_peered":"2024-03-06T07:21:43.322525+0000","last_unstale":"2024-03-06T07:21:44.447732+0000","last_undegraded":"2024-03-06T07:21:44.447732+0000","last_fullsized":"2024-03-06T07:21:44.447732+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335210+0000","last_change":"2024-03-06T07:21:43.319837+0000","last_active":"2024-03-06T07:21:44.335210+0000","last_peered":"2024-03-06T07:21:44.335210+0000","last_clean":"2024-03-06T07:21:44.335210+0000","last_became_active":"2024-03-06T07:21:43.319564+0000","last_became_peered":"2024-03-06T07:21:43.319564+0000","last_unstale":"2024-03-06T07:21:44.335210+0000","last_undegraded":"2024-03-06T07:21:44.335210+0000","last_fullsized":"2024-03-06T07:21:44.335210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609434+0000","last_change":"2024-03-06T07:21:42.310584+0000","last_active":"2024-03-06T07:21:44.609434+0000","last_peered":"2024-03-06T07:21:44.609434+0000","last_clean":"2024-03-06T07:21:44.609434+0000","last_became_active":"2024-03-06T07:21:42.310317+0000","last_became_peered":"2024-03-06T07:21:42.310317+0000","last_unstale":"2024-03-06T07:21:44.609434+0000","last_undegraded":"2024-03-06T07:21:44.609434+0000","last_fullsized":"2024-03-06T07:21:44.609434+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447956+0000","last_change":"2024-03-06T07:21:43.323883+0000","last_active":"2024-03-06T07:21:44.447956+0000","last_peered":"2024-03-06T07:21:44.447956+0000","last_clean":"2024-03-06T07:21:44.447956+0000","last_became_active":"2024-03-06T07:21:43.323613+0000","last_became_peered":"2024-03-06T07:21:43.323613+0000","last_unstale":"2024-03-06T07:21:44.447956+0000","last_undegraded":"2024-03-06T07:21:44.447956+0000","last_fullsized":"2024-03-06T07:21:44.447956+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609091+0000","last_change":"2024-03-06T07:21:42.304199+0000","last_active":"2024-03-06T07:21:44.609091+0000","last_peered":"2024-03-06T07:21:44.609091+0000","last_clean":"2024-03-06T07:21:44.609091+0000","last_became_active":"2024-03-06T07:21:42.303802+0000","last_became_peered":"2024-03-06T07:21:42.303802+0000","last_unstale":"2024-03-06T07:21:44.609091+0000","last_undegraded":"2024-03-06T07:21:44.609091+0000","last_fullsized":"2024-03-06T07:21:44.609091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609109+0000","last_change":"2024-03-06T07:21:43.323512+0000","last_active":"2024-03-06T07:21:44.609109+0000","last_peered":"2024-03-06T07:21:44.609109+0000","last_clean":"2024-03-06T07:21:44.609109+0000","last_became_active":"2024-03-06T07:21:43.323216+0000","last_became_peered":"2024-03-06T07:21:43.323216+0000","last_unstale":"2024-03-06T07:21:44.609109+0000","last_undegraded":"2024-03-06T07:21:44.609109+0000","last_fullsized":"2024-03-06T07:21:44.609109+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:31.821522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609201+0000","last_change":"2024-03-06T07:21:42.305711+0000","last_active":"2024-03-06T07:21:44.609201+0000","last_peered":"2024-03-06T07:21:44.609201+0000","last_clean":"2024-03-06T07:21:44.609201+0000","last_became_active":"2024-03-06T07:21:42.305431+0000","last_became_peered":"2024-03-06T07:21:42.305431+0000","last_unstale":"2024-03-06T07:21:44.609201+0000","last_undegraded":"2024-03-06T07:21:44.609201+0000","last_fullsized":"2024-03-06T07:21:44.609201+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317784+0000","last_change":"2024-03-06T07:21:43.317966+0000","last_active":"2024-03-06T07:21:44.317784+0000","last_peered":"2024-03-06T07:21:44.317784+0000","last_clean":"2024-03-06T07:21:44.317784+0000","last_became_active":"2024-03-06T07:21:43.317700+0000","last_became_peered":"2024-03-06T07:21:43.317700+0000","last_unstale":"2024-03-06T07:21:44.317784+0000","last_undegraded":"2024-03-06T07:21:44.317784+0000","last_fullsized":"2024-03-06T07:21:44.317784+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447944+0000","last_change":"2024-03-06T07:21:42.315045+0000","last_active":"2024-03-06T07:21:44.447944+0000","last_peered":"2024-03-06T07:21:44.447944+0000","last_clean":"2024-03-06T07:21:44.447944+0000","last_became_active":"2024-03-06T07:21:42.314768+0000","last_became_peered":"2024-03-06T07:21:42.314768+0000","last_unstale":"2024-03-06T07:21:44.447944+0000","last_undegraded":"2024-03-06T07:21:44.447944+0000","last_fullsized":"2024-03-06T07:21:44.447944+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:14.953476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318579+0000","last_change":"2024-03-06T07:21:43.324372+0000","last_active":"2024-03-06T07:21:44.318579+0000","last_peered":"2024-03-06T07:21:44.318579+0000","last_clean":"2024-03-06T07:21:44.318579+0000","last_became_active":"2024-03-06T07:21:43.324097+0000","last_became_peered":"2024-03-06T07:21:43.324097+0000","last_unstale":"2024-03-06T07:21:44.318579+0000","last_undegraded":"2024-03-06T07:21:44.318579+0000","last_fullsized":"2024-03-06T07:21:44.318579+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:49:40.932248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332776+0000","last_change":"2024-03-06T07:21:42.317614+0000","last_active":"2024-03-06T07:21:44.332776+0000","last_peered":"2024-03-06T07:21:44.332776+0000","last_clean":"2024-03-06T07:21:44.332776+0000","last_became_active":"2024-03-06T07:21:42.317094+0000","last_became_peered":"2024-03-06T07:21:42.317094+0000","last_unstale":"2024-03-06T07:21:44.332776+0000","last_undegraded":"2024-03-06T07:21:44.332776+0000","last_fullsized":"2024-03-06T07:21:44.332776+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:15.739191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335623+0000","last_change":"2024-03-06T07:21:43.318727+0000","last_active":"2024-03-06T07:21:44.335623+0000","last_peered":"2024-03-06T07:21:44.335623+0000","last_clean":"2024-03-06T07:21:44.335623+0000","last_became_active":"2024-03-06T07:21:43.318440+0000","last_became_peered":"2024-03-06T07:21:43.318440+0000","last_unstale":"2024-03-06T07:21:44.335623+0000","last_undegraded":"2024-03-06T07:21:44.335623+0000","last_fullsized":"2024-03-06T07:21:44.335623+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318489+0000","last_change":"2024-03-06T07:21:42.307726+0000","last_active":"2024-03-06T07:21:44.318489+0000","last_peered":"2024-03-06T07:21:44.318489+0000","last_clean":"2024-03-06T07:21:44.318489+0000","last_became_active":"2024-03-06T07:21:42.307445+0000","last_became_peered":"2024-03-06T07:21:42.307445+0000","last_unstale":"2024-03-06T07:21:44.318489+0000","last_undegraded":"2024-03-06T07:21:44.318489+0000","last_fullsized":"2024-03-06T07:21:44.318489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318063+0000","last_change":"2024-03-06T07:21:43.318372+0000","last_active":"2024-03-06T07:21:44.318063+0000","last_peered":"2024-03-06T07:21:44.318063+0000","last_clean":"2024-03-06T07:21:44.318063+0000","last_became_active":"2024-03-06T07:21:43.318047+0000","last_became_peered":"2024-03-06T07:21:43.318047+0000","last_unstale":"2024-03-06T07:21:44.318063+0000","last_undegraded":"2024-03-06T07:21:44.318063+0000","last_fullsized":"2024-03-06T07:21:44.318063+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609065+0000","last_change":"2024-03-06T07:21:42.310677+0000","last_active":"2024-03-06T07:21:44.609065+0000","last_peered":"2024-03-06T07:21:44.609065+0000","last_clean":"2024-03-06T07:21:44.609065+0000","last_became_active":"2024-03-06T07:21:42.310375+0000","last_became_peered":"2024-03-06T07:21:42.310375+0000","last_unstale":"2024-03-06T07:21:44.609065+0000","last_undegraded":"2024-03-06T07:21:44.609065+0000","last_fullsized":"2024-03-06T07:21:44.609065+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:13.837451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335829+0000","last_change":"2024-03-06T07:21:43.325740+0000","last_active":"2024-03-06T07:21:44.335829+0000","last_peered":"2024-03-06T07:21:44.335829+0000","last_clean":"2024-03-06T07:21:44.335829+0000","last_became_active":"2024-03-06T07:21:43.325425+0000","last_became_peered":"2024-03-06T07:21:43.325425+0000","last_unstale":"2024-03-06T07:21:44.335829+0000","last_undegraded":"2024-03-06T07:21:44.335829+0000","last_fullsized":"2024-03-06T07:21:44.335829+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332650+0000","last_change":"2024-03-06T07:21:42.319786+0000","last_active":"2024-03-06T07:21:44.332650+0000","last_peered":"2024-03-06T07:21:44.332650+0000","last_clean":"2024-03-06T07:21:44.332650+0000","last_became_active":"2024-03-06T07:21:42.319430+0000","last_became_peered":"2024-03-06T07:21:42.319430+0000","last_unstale":"2024-03-06T07:21:44.332650+0000","last_undegraded":"2024-03-06T07:21:44.332650+0000","last_fullsized":"2024-03-06T07:21:44.332650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:09.719754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318319+0000","last_change":"2024-03-06T07:21:43.330805+0000","last_active":"2024-03-06T07:21:44.318319+0000","last_peered":"2024-03-06T07:21:44.318319+0000","last_clean":"2024-03-06T07:21:44.318319+0000","last_became_active":"2024-03-06T07:21:43.330267+0000","last_became_peered":"2024-03-06T07:21:43.330267+0000","last_unstale":"2024-03-06T07:21:44.318319+0000","last_undegraded":"2024-03-06T07:21:44.318319+0000","last_fullsized":"2024-03-06T07:21:44.318319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:56.716578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335767+0000","last_change":"2024-03-06T07:21:42.312213+0000","last_active":"2024-03-06T07:21:44.335767+0000","last_peered":"2024-03-06T07:21:44.335767+0000","last_clean":"2024-03-06T07:21:44.335767+0000","last_became_active":"2024-03-06T07:21:42.311878+0000","last_became_peered":"2024-03-06T07:21:42.311878+0000","last_unstale":"2024-03-06T07:21:44.335767+0000","last_undegraded":"2024-03-06T07:21:44.335767+0000","last_fullsized":"2024-03-06T07:21:44.335767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447790+0000","last_change":"2024-03-06T07:21:43.320122+0000","last_active":"2024-03-06T07:21:44.447790+0000","last_peered":"2024-03-06T07:21:44.447790+0000","last_clean":"2024-03-06T07:21:44.447790+0000","last_became_active":"2024-03-06T07:21:43.319799+0000","last_became_peered":"2024-03-06T07:21:43.319799+0000","last_unstale":"2024-03-06T07:21:44.447790+0000","last_undegraded":"2024-03-06T07:21:44.447790+0000","last_fullsized":"2024-03-06T07:21:44.447790+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318501+0000","last_change":"2024-03-06T07:21:42.319681+0000","last_active":"2024-03-06T07:21:44.318501+0000","last_peered":"2024-03-06T07:21:44.318501+0000","last_clean":"2024-03-06T07:21:44.318501+0000","last_became_active":"2024-03-06T07:21:42.319452+0000","last_became_peered":"2024-03-06T07:21:42.319452+0000","last_unstale":"2024-03-06T07:21:44.318501+0000","last_undegraded":"2024-03-06T07:21:44.318501+0000","last_fullsized":"2024-03-06T07:21:44.318501+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:35.308790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333038+0000","last_change":"2024-03-06T07:21:43.323490+0000","last_active":"2024-03-06T07:21:44.333038+0000","last_peered":"2024-03-06T07:21:44.333038+0000","last_clean":"2024-03-06T07:21:44.333038+0000","last_became_active":"2024-03-06T07:21:43.323088+0000","last_became_peered":"2024-03-06T07:21:43.323088+0000","last_unstale":"2024-03-06T07:21:44.333038+0000","last_undegraded":"2024-03-06T07:21:44.333038+0000","last_fullsized":"2024-03-06T07:21:44.333038+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335765+0000","last_change":"2024-03-06T07:21:42.312732+0000","last_active":"2024-03-06T07:21:44.335765+0000","last_peered":"2024-03-06T07:21:44.335765+0000","last_clean":"2024-03-06T07:21:44.335765+0000","last_became_active":"2024-03-06T07:21:42.312462+0000","last_became_peered":"2024-03-06T07:21:42.312462+0000","last_unstale":"2024-03-06T07:21:44.335765+0000","last_undegraded":"2024-03-06T07:21:44.335765+0000","last_fullsized":"2024-03-06T07:21:44.335765+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318716+0000","last_change":"2024-03-06T07:21:43.320022+0000","last_active":"2024-03-06T07:21:44.318716+0000","last_peered":"2024-03-06T07:21:44.318716+0000","last_clean":"2024-03-06T07:21:44.318716+0000","last_became_active":"2024-03-06T07:21:43.319716+0000","last_became_peered":"2024-03-06T07:21:43.319716+0000","last_unstale":"2024-03-06T07:21:44.318716+0000","last_undegraded":"2024-03-06T07:21:44.318716+0000","last_fullsized":"2024-03-06T07:21:44.318716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:50.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318087+0000","last_change":"2024-03-06T07:21:42.308298+0000","last_active":"2024-03-06T07:21:44.318087+0000","last_peered":"2024-03-06T07:21:44.318087+0000","last_clean":"2024-03-06T07:21:44.318087+0000","last_became_active":"2024-03-06T07:21:42.308008+0000","last_became_peered":"2024-03-06T07:21:42.308008+0000","last_unstale":"2024-03-06T07:21:44.318087+0000","last_undegraded":"2024-03-06T07:21:44.318087+0000","last_fullsized":"2024-03-06T07:21:44.318087+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:51.723259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447774+0000","last_change":"2024-03-06T07:21:43.323228+0000","last_active":"2024-03-06T07:21:44.447774+0000","last_peered":"2024-03-06T07:21:44.447774+0000","last_clean":"2024-03-06T07:21:44.447774+0000","last_became_active":"2024-03-06T07:21:43.322951+0000","last_became_peered":"2024-03-06T07:21:43.322951+0000","last_unstale":"2024-03-06T07:21:44.447774+0000","last_undegraded":"2024-03-06T07:21:44.447774+0000","last_fullsized":"2024-03-06T07:21:44.447774+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332880+0000","last_change":"2024-03-06T07:21:42.321872+0000","last_active":"2024-03-06T07:21:44.332880+0000","last_peered":"2024-03-06T07:21:44.332880+0000","last_clean":"2024-03-06T07:21:44.332880+0000","last_became_active":"2024-03-06T07:21:42.321384+0000","last_became_peered":"2024-03-06T07:21:42.321384+0000","last_unstale":"2024-03-06T07:21:44.332880+0000","last_undegraded":"2024-03-06T07:21:44.332880+0000","last_fullsized":"2024-03-06T07:21:44.332880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333396+0000","last_change":"2024-03-06T07:21:43.324244+0000","last_active":"2024-03-06T07:21:44.333396+0000","last_peered":"2024-03-06T07:21:44.333396+0000","last_clean":"2024-03-06T07:21:44.333396+0000","last_became_active":"2024-03-06T07:21:43.323549+0000","last_became_peered":"2024-03-06T07:21:43.323549+0000","last_unstale":"2024-03-06T07:21:44.333396+0000","last_undegraded":"2024-03-06T07:21:44.333396+0000","last_fullsized":"2024-03-06T07:21:44.333396+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609239+0000","last_change":"2024-03-06T07:21:42.318308+0000","last_active":"2024-03-06T07:21:44.609239+0000","last_peered":"2024-03-06T07:21:44.609239+0000","last_clean":"2024-03-06T07:21:44.609239+0000","last_became_active":"2024-03-06T07:21:42.318080+0000","last_became_peered":"2024-03-06T07:21:42.318080+0000","last_unstale":"2024-03-06T07:21:44.609239+0000","last_undegraded":"2024-03-06T07:21:44.609239+0000","last_fullsized":"2024-03-06T07:21:44.609239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317946+0000","last_change":"2024-03-06T07:21:43.331853+0000","last_active":"2024-03-06T07:21:44.317946+0000","last_peered":"2024-03-06T07:21:44.317946+0000","last_clean":"2024-03-06T07:21:44.317946+0000","last_became_active":"2024-03-06T07:21:43.331560+0000","last_became_peered":"2024-03-06T07:21:43.331560+0000","last_unstale":"2024-03-06T07:21:44.317946+0000","last_undegraded":"2024-03-06T07:21:44.317946+0000","last_fullsized":"2024-03-06T07:21:44.317946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335375+0000","last_change":"2024-03-06T07:21:42.309167+0000","last_active":"2024-03-06T07:21:44.335375+0000","last_peered":"2024-03-06T07:21:44.335375+0000","last_clean":"2024-03-06T07:21:44.335375+0000","last_became_active":"2024-03-06T07:21:42.308882+0000","last_became_peered":"2024-03-06T07:21:42.308882+0000","last_unstale":"2024-03-06T07:21:44.335375+0000","last_undegraded":"2024-03-06T07:21:44.335375+0000","last_fullsized":"2024-03-06T07:21:44.335375+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333062+0000","last_change":"2024-03-06T07:21:43.324536+0000","last_active":"2024-03-06T07:21:44.333062+0000","last_peered":"2024-03-06T07:21:44.333062+0000","last_clean":"2024-03-06T07:21:44.333062+0000","last_became_active":"2024-03-06T07:21:43.323818+0000","last_became_peered":"2024-03-06T07:21:43.323818+0000","last_unstale":"2024-03-06T07:21:44.333062+0000","last_undegraded":"2024-03-06T07:21:44.333062+0000","last_fullsized":"2024-03-06T07:21:44.333062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317773+0000","last_change":"2024-03-06T07:21:42.308460+0000","last_active":"2024-03-06T07:21:44.317773+0000","last_peered":"2024-03-06T07:21:44.317773+0000","last_clean":"2024-03-06T07:21:44.317773+0000","last_became_active":"2024-03-06T07:21:42.307461+0000","last_became_peered":"2024-03-06T07:21:42.307461+0000","last_unstale":"2024-03-06T07:21:44.317773+0000","last_undegraded":"2024-03-06T07:21:44.317773+0000","last_fullsized":"2024-03-06T07:21:44.317773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:34.346180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317733+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:44.317733+0000","last_peered":"2024-03-06T07:21:44.317733+0000","last_clean":"2024-03-06T07:21:44.317733+0000","last_became_active":"2024-03-06T07:21:43.316848+0000","last_became_peered":"2024-03-06T07:21:43.316848+0000","last_unstale":"2024-03-06T07:21:44.317733+0000","last_undegraded":"2024-03-06T07:21:44.317733+0000","last_fullsized":"2024-03-06T07:21:44.317733+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333219+0000","last_change":"2024-03-06T07:21:42.319998+0000","last_active":"2024-03-06T07:21:44.333219+0000","last_peered":"2024-03-06T07:21:44.333219+0000","last_clean":"2024-03-06T07:21:44.333219+0000","last_became_active":"2024-03-06T07:21:42.319529+0000","last_became_peered":"2024-03-06T07:21:42.319529+0000","last_unstale":"2024-03-06T07:21:44.333219+0000","last_undegraded":"2024-03-06T07:21:44.333219+0000","last_fullsized":"2024-03-06T07:21:44.333219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608621+0000","last_change":"2024-03-06T07:21:43.327074+0000","last_active":"2024-03-06T07:21:44.608621+0000","last_peered":"2024-03-06T07:21:44.608621+0000","last_clean":"2024-03-06T07:21:44.608621+0000","last_became_active":"2024-03-06T07:21:43.326797+0000","last_became_peered":"2024-03-06T07:21:43.326797+0000","last_unstale":"2024-03-06T07:21:44.608621+0000","last_undegraded":"2024-03-06T07:21:44.608621+0000","last_fullsized":"2024-03-06T07:21:44.608621+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608792+0000","last_change":"2024-03-06T07:21:42.310454+0000","last_active":"2024-03-06T07:21:44.608792+0000","last_peered":"2024-03-06T07:21:44.608792+0000","last_clean":"2024-03-06T07:21:44.608792+0000","last_became_active":"2024-03-06T07:21:42.310172+0000","last_became_peered":"2024-03-06T07:21:42.310172+0000","last_unstale":"2024-03-06T07:21:44.608792+0000","last_undegraded":"2024-03-06T07:21:44.608792+0000","last_fullsized":"2024-03-06T07:21:44.608792+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:34.182536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318530+0000","last_change":"2024-03-06T07:21:43.324398+0000","last_active":"2024-03-06T07:21:44.318530+0000","last_peered":"2024-03-06T07:21:44.318530+0000","last_clean":"2024-03-06T07:21:44.318530+0000","last_became_active":"2024-03-06T07:21:43.324115+0000","last_became_peered":"2024-03-06T07:21:43.324115+0000","last_unstale":"2024-03-06T07:21:44.318530+0000","last_undegraded":"2024-03-06T07:21:44.318530+0000","last_fullsized":"2024-03-06T07:21:44.318530+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:36.888023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609426+0000","last_change":"2024-03-06T07:21:42.312390+0000","last_active":"2024-03-06T07:21:44.609426+0000","last_peered":"2024-03-06T07:21:44.609426+0000","last_clean":"2024-03-06T07:21:44.609426+0000","last_became_active":"2024-03-06T07:21:42.312079+0000","last_became_peered":"2024-03-06T07:21:42.312079+0000","last_unstale":"2024-03-06T07:21:44.609426+0000","last_undegraded":"2024-03-06T07:21:44.609426+0000","last_fullsized":"2024-03-06T07:21:44.609426+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609086+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:44.609086+0000","last_peered":"2024-03-06T07:21:44.609086+0000","last_clean":"2024-03-06T07:21:44.609086+0000","last_became_active":"2024-03-06T07:21:43.316857+0000","last_became_peered":"2024-03-06T07:21:43.316857+0000","last_unstale":"2024-03-06T07:21:44.609086+0000","last_undegraded":"2024-03-06T07:21:44.609086+0000","last_fullsized":"2024-03-06T07:21:44.609086+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:48.715734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609389+0000","last_change":"2024-03-06T07:21:42.311972+0000","last_active":"2024-03-06T07:21:44.609389+0000","last_peered":"2024-03-06T07:21:44.609389+0000","last_clean":"2024-03-06T07:21:44.609389+0000","last_became_active":"2024-03-06T07:21:42.311640+0000","last_became_peered":"2024-03-06T07:21:42.311640+0000","last_unstale":"2024-03-06T07:21:44.609389+0000","last_undegraded":"2024-03-06T07:21:44.609389+0000","last_fullsized":"2024-03-06T07:21:44.609389+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:22.568258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447949+0000","last_change":"2024-03-06T07:21:43.320374+0000","last_active":"2024-03-06T07:21:44.447949+0000","last_peered":"2024-03-06T07:21:44.447949+0000","last_clean":"2024-03-06T07:21:44.447949+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:44.447949+0000","last_undegraded":"2024-03-06T07:21:44.447949+0000","last_fullsized":"2024-03-06T07:21:44.447949+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609053+0000","last_change":"2024-03-06T07:21:42.309901+0000","last_active":"2024-03-06T07:21:44.609053+0000","last_peered":"2024-03-06T07:21:44.609053+0000","last_clean":"2024-03-06T07:21:44.609053+0000","last_became_active":"2024-03-06T07:21:42.309532+0000","last_became_peered":"2024-03-06T07:21:42.309532+0000","last_unstale":"2024-03-06T07:21:44.609053+0000","last_undegraded":"2024-03-06T07:21:44.609053+0000","last_fullsized":"2024-03-06T07:21:44.609053+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609397+0000","last_change":"2024-03-06T07:21:43.320820+0000","last_active":"2024-03-06T07:21:44.609397+0000","last_peered":"2024-03-06T07:21:44.609397+0000","last_clean":"2024-03-06T07:21:44.609397+0000","last_became_active":"2024-03-06T07:21:43.319225+0000","last_became_peered":"2024-03-06T07:21:43.319225+0000","last_unstale":"2024-03-06T07:21:44.609397+0000","last_undegraded":"2024-03-06T07:21:44.609397+0000","last_fullsized":"2024-03-06T07:21:44.609397+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317589+0000","last_change":"2024-03-06T07:21:42.307676+0000","last_active":"2024-03-06T07:21:44.317589+0000","last_peered":"2024-03-06T07:21:44.317589+0000","last_clean":"2024-03-06T07:21:44.317589+0000","last_became_active":"2024-03-06T07:21:42.307388+0000","last_became_peered":"2024-03-06T07:21:42.307388+0000","last_unstale":"2024-03-06T07:21:44.317589+0000","last_undegraded":"2024-03-06T07:21:44.317589+0000","last_fullsized":"2024-03-06T07:21:44.317589+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:47.711911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333072+0000","last_change":"2024-03-06T07:21:43.329841+0000","last_active":"2024-03-06T07:21:44.333072+0000","last_peered":"2024-03-06T07:21:44.333072+0000","last_clean":"2024-03-06T07:21:44.333072+0000","last_became_active":"2024-03-06T07:21:43.329526+0000","last_became_peered":"2024-03-06T07:21:43.329526+0000","last_unstale":"2024-03-06T07:21:44.333072+0000","last_undegraded":"2024-03-06T07:21:44.333072+0000","last_fullsized":"2024-03-06T07:21:44.333072+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332814+0000","last_change":"2024-03-06T07:21:42.318168+0000","last_active":"2024-03-06T07:21:44.332814+0000","last_peered":"2024-03-06T07:21:44.332814+0000","last_clean":"2024-03-06T07:21:44.332814+0000","last_became_active":"2024-03-06T07:21:42.317866+0000","last_became_peered":"2024-03-06T07:21:42.317866+0000","last_unstale":"2024-03-06T07:21:44.332814+0000","last_undegraded":"2024-03-06T07:21:44.332814+0000","last_fullsized":"2024-03-06T07:21:44.332814+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:49.054000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609107+0000","last_change":"2024-03-06T07:21:43.316009+0000","last_active":"2024-03-06T07:21:44.609107+0000","last_peered":"2024-03-06T07:21:44.609107+0000","last_clean":"2024-03-06T07:21:44.609107+0000","last_became_active":"2024-03-06T07:21:43.315699+0000","last_became_peered":"2024-03-06T07:21:43.315699+0000","last_unstale":"2024-03-06T07:21:44.609107+0000","last_undegraded":"2024-03-06T07:21:44.609107+0000","last_fullsized":"2024-03-06T07:21:44.609107+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:52.769462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608468+0000","last_change":"2024-03-06T07:21:42.311690+0000","last_active":"2024-03-06T07:21:44.608468+0000","last_peered":"2024-03-06T07:21:44.608468+0000","last_clean":"2024-03-06T07:21:44.608468+0000","last_became_active":"2024-03-06T07:21:42.311417+0000","last_became_peered":"2024-03-06T07:21:42.311417+0000","last_unstale":"2024-03-06T07:21:44.608468+0000","last_undegraded":"2024-03-06T07:21:44.608468+0000","last_fullsized":"2024-03-06T07:21:44.608468+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.094878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447749+0000","last_change":"2024-03-06T07:21:43.322229+0000","last_active":"2024-03-06T07:21:44.447749+0000","last_peered":"2024-03-06T07:21:44.447749+0000","last_clean":"2024-03-06T07:21:44.447749+0000","last_became_active":"2024-03-06T07:21:43.321934+0000","last_became_peered":"2024-03-06T07:21:43.321934+0000","last_unstale":"2024-03-06T07:21:44.447749+0000","last_undegraded":"2024-03-06T07:21:44.447749+0000","last_fullsized":"2024-03-06T07:21:44.447749+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609427+0000","last_change":"2024-03-06T07:21:42.321714+0000","last_active":"2024-03-06T07:21:44.609427+0000","last_peered":"2024-03-06T07:21:44.609427+0000","last_clean":"2024-03-06T07:21:44.609427+0000","last_became_active":"2024-03-06T07:21:42.321444+0000","last_became_peered":"2024-03-06T07:21:42.321444+0000","last_unstale":"2024-03-06T07:21:44.609427+0000","last_undegraded":"2024-03-06T07:21:44.609427+0000","last_fullsized":"2024-03-06T07:21:44.609427+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609148+0000","last_change":"2024-03-06T07:21:43.316403+0000","last_active":"2024-03-06T07:21:44.609148+0000","last_peered":"2024-03-06T07:21:44.609148+0000","last_clean":"2024-03-06T07:21:44.609148+0000","last_became_active":"2024-03-06T07:21:43.315416+0000","last_became_peered":"2024-03-06T07:21:43.315416+0000","last_unstale":"2024-03-06T07:21:44.609148+0000","last_undegraded":"2024-03-06T07:21:44.609148+0000","last_fullsized":"2024-03-06T07:21:44.609148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:32.706894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335773+0000","last_change":"2024-03-06T07:21:42.311790+0000","last_active":"2024-03-06T07:21:44.335773+0000","last_peered":"2024-03-06T07:21:44.335773+0000","last_clean":"2024-03-06T07:21:44.335773+0000","last_became_active":"2024-03-06T07:21:42.311500+0000","last_became_peered":"2024-03-06T07:21:42.311500+0000","last_unstale":"2024-03-06T07:21:44.335773+0000","last_undegraded":"2024-03-06T07:21:44.335773+0000","last_fullsized":"2024-03-06T07:21:44.335773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317822+0000","last_change":"2024-03-06T07:21:43.319244+0000","last_active":"2024-03-06T07:21:44.317822+0000","last_peered":"2024-03-06T07:21:44.317822+0000","last_clean":"2024-03-06T07:21:44.317822+0000","last_became_active":"2024-03-06T07:21:43.318994+0000","last_became_peered":"2024-03-06T07:21:43.318994+0000","last_unstale":"2024-03-06T07:21:44.317822+0000","last_undegraded":"2024-03-06T07:21:44.317822+0000","last_fullsized":"2024-03-06T07:21:44.317822+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333013+0000","last_change":"2024-03-06T07:21:42.315718+0000","last_active":"2024-03-06T07:21:44.333013+0000","last_peered":"2024-03-06T07:21:44.333013+0000","last_clean":"2024-03-06T07:21:44.333013+0000","last_became_active":"2024-03-06T07:21:42.315402+0000","last_became_peered":"2024-03-06T07:21:42.315402+0000","last_unstale":"2024-03-06T07:21:44.333013+0000","last_undegraded":"2024-03-06T07:21:44.333013+0000","last_fullsized":"2024-03-06T07:21:44.333013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332903+0000","last_change":"2024-03-06T07:21:43.323135+0000","last_active":"2024-03-06T07:21:44.332903+0000","last_peered":"2024-03-06T07:21:44.332903+0000","last_clean":"2024-03-06T07:21:44.332903+0000","last_became_active":"2024-03-06T07:21:43.322814+0000","last_became_peered":"2024-03-06T07:21:43.322814+0000","last_unstale":"2024-03-06T07:21:44.332903+0000","last_undegraded":"2024-03-06T07:21:44.332903+0000","last_fullsized":"2024-03-06T07:21:44.332903+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:24.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317877+0000","last_change":"2024-03-06T07:21:42.308451+0000","last_active":"2024-03-06T07:21:44.317877+0000","last_peered":"2024-03-06T07:21:44.317877+0000","last_clean":"2024-03-06T07:21:44.317877+0000","last_became_active":"2024-03-06T07:21:42.308167+0000","last_became_peered":"2024-03-06T07:21:42.308167+0000","last_unstale":"2024-03-06T07:21:44.317877+0000","last_undegraded":"2024-03-06T07:21:44.317877+0000","last_fullsized":"2024-03-06T07:21:44.317877+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:31.703072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447915+0000","last_change":"2024-03-06T07:21:43.324622+0000","last_active":"2024-03-06T07:21:44.447915+0000","last_peered":"2024-03-06T07:21:44.447915+0000","last_clean":"2024-03-06T07:21:44.447915+0000","last_became_active":"2024-03-06T07:21:43.324350+0000","last_became_peered":"2024-03-06T07:21:43.324350+0000","last_unstale":"2024-03-06T07:21:44.447915+0000","last_undegraded":"2024-03-06T07:21:44.447915+0000","last_fullsized":"2024-03-06T07:21:44.447915+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317714+0000","last_change":"2024-03-06T07:21:42.309246+0000","last_active":"2024-03-06T07:21:44.317714+0000","last_peered":"2024-03-06T07:21:44.317714+0000","last_clean":"2024-03-06T07:21:44.317714+0000","last_became_active":"2024-03-06T07:21:42.308990+0000","last_became_peered":"2024-03-06T07:21:42.308990+0000","last_unstale":"2024-03-06T07:21:44.317714+0000","last_undegraded":"2024-03-06T07:21:44.317714+0000","last_fullsized":"2024-03-06T07:21:44.317714+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:51.765639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332957+0000","last_change":"2024-03-06T07:21:43.329441+0000","last_active":"2024-03-06T07:21:44.332957+0000","last_peered":"2024-03-06T07:21:44.332957+0000","last_clean":"2024-03-06T07:21:44.332957+0000","last_became_active":"2024-03-06T07:21:43.327694+0000","last_became_peered":"2024-03-06T07:21:43.327694+0000","last_unstale":"2024-03-06T07:21:44.332957+0000","last_undegraded":"2024-03-06T07:21:44.332957+0000","last_fullsized":"2024-03-06T07:21:44.332957+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318424+0000","last_change":"2024-03-06T07:21:42.309614+0000","last_active":"2024-03-06T07:21:44.318424+0000","last_peered":"2024-03-06T07:21:44.318424+0000","last_clean":"2024-03-06T07:21:44.318424+0000","last_became_active":"2024-03-06T07:21:42.309319+0000","last_became_peered":"2024-03-06T07:21:42.309319+0000","last_unstale":"2024-03-06T07:21:44.318424+0000","last_undegraded":"2024-03-06T07:21:44.318424+0000","last_fullsized":"2024-03-06T07:21:44.318424+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:28.385954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318213+0000","last_change":"2024-03-06T07:21:43.321711+0000","last_active":"2024-03-06T07:21:44.318213+0000","last_peered":"2024-03-06T07:21:44.318213+0000","last_clean":"2024-03-06T07:21:44.318213+0000","last_became_active":"2024-03-06T07:21:43.321434+0000","last_became_peered":"2024-03-06T07:21:43.321434+0000","last_unstale":"2024-03-06T07:21:44.318213+0000","last_undegraded":"2024-03-06T07:21:44.318213+0000","last_fullsized":"2024-03-06T07:21:44.318213+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:06.150276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447853+0000","last_change":"2024-03-06T07:21:42.310672+0000","last_active":"2024-03-06T07:21:44.447853+0000","last_peered":"2024-03-06T07:21:44.447853+0000","last_clean":"2024-03-06T07:21:44.447853+0000","last_became_active":"2024-03-06T07:21:42.310366+0000","last_became_peered":"2024-03-06T07:21:42.310366+0000","last_unstale":"2024-03-06T07:21:44.447853+0000","last_undegraded":"2024-03-06T07:21:44.447853+0000","last_fullsized":"2024-03-06T07:21:44.447853+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:24.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609137+0000","last_change":"2024-03-06T07:21:43.319233+0000","last_active":"2024-03-06T07:21:44.609137+0000","last_peered":"2024-03-06T07:21:44.609137+0000","last_clean":"2024-03-06T07:21:44.609137+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:44.609137+0000","last_undegraded":"2024-03-06T07:21:44.609137+0000","last_fullsized":"2024-03-06T07:21:44.609137+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.334573+0000","last_change":"2024-03-06T07:21:42.322036+0000","last_active":"2024-03-06T07:21:44.334573+0000","last_peered":"2024-03-06T07:21:44.334573+0000","last_clean":"2024-03-06T07:21:44.334573+0000","last_became_active":"2024-03-06T07:21:42.321385+0000","last_became_peered":"2024-03-06T07:21:42.321385+0000","last_unstale":"2024-03-06T07:21:44.334573+0000","last_undegraded":"2024-03-06T07:21:44.334573+0000","last_fullsized":"2024-03-06T07:21:44.334573+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335159+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:44.335159+0000","last_peered":"2024-03-06T07:21:44.335159+0000","last_clean":"2024-03-06T07:21:44.335159+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:44.335159+0000","last_undegraded":"2024-03-06T07:21:44.335159+0000","last_fullsized":"2024-03-06T07:21:44.335159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":854168,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":602112,"data_stored":264328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":264258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":610304,"data_stored":265310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":593920,"data_stored":258596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":602112,"data_stored":263836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":655360,"data_stored":847408,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:47.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:47.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.453+0000 7fb516ef4640 1 Processor -- start 2024-03-06T07:21:47.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.453+0000 7fb516ef4640 1 -- start start 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.454+0000 7fb516ef4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb51014f360 0x7fb510155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.454+0000 7fb516ef4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb510158b60 0x7fb510158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.454+0000 7fb516ef4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100c73d0 con 0x7fb5100c88c0 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.454+0000 7fb516ef4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100c8dd0 con 0x7fb51014f360 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.454+0000 7fb516ef4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100be2e0 con 0x7fb510158b60 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb515491640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb510158b60 0x7fb510158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb515491640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb510158b60 0x7fb510158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:49356/0 (socket says 172.21.15.26:49356) 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb515491640 1 -- 172.21.15.26:0/2278858045 learned_addr learned my addr 172.21.15.26:0/2278858045 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb514c90640 1 --2- 172.21.15.26:0/2278858045 >> v2:172.21.15.117:3300/0 conn(0x7fb51014f360 0x7fb510155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb514c90640 1 -- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb510158b60 msgr2=0x7fb510158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb514c90640 1 --2- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb510158b60 0x7fb510158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb514c90640 1 -- 172.21.15.26:0/2278858045 >> v1:172.21.15.117:6789/0 conn(0x7fb5100c88c0 legacy=0x7fb5100c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb514c90640 1 -- 172.21.15.26:0/2278858045 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100c7670 con 0x7fb51014f360 2024-03-06T07:21:47.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.455+0000 7fb515491640 1 --2- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb510158b60 0x7fb510158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.456+0000 7fb514c90640 1 --2- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51014f360 0x7fb510155c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb5080097a0 tx=0x7fb50802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db61910e577a41a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.456+0000 7fb50f7fe640 1 -- 172.21.15.26:0/2278858045 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5080315a0 con 0x7fb51014f360 2024-03-06T07:21:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.456+0000 7fb50f7fe640 1 -- 172.21.15.26:0/2278858045 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb508037070 con 0x7fb51014f360 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.456+0000 7fb50f7fe640 1 -- 172.21.15.26:0/2278858045 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508031960 con 0x7fb51014f360 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.456+0000 7fb516ef4640 1 -- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51014f360 msgr2=0x7fb510155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.457+0000 7fb516ef4640 1 --2- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51014f360 0x7fb510155c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb5080097a0 tx=0x7fb50802b840 comp rx=0 tx=0).stop 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.457+0000 7fb516ef4640 1 -- 172.21.15.26:0/2278858045 shutdown_connections 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.457+0000 7fb516ef4640 1 --2- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb510158b60 0x7fb510158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.457+0000 7fb516ef4640 1 --2- 172.21.15.26:0/2278858045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51014f360 0x7fb510155c10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.457+0000 7fb516ef4640 1 -- 172.21.15.26:0/2278858045 >> 172.21.15.26:0/2278858045 conn(0x7fb5100ac570 msgr2=0x7fb5100ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.457+0000 7fb516ef4640 1 -- 172.21.15.26:0/2278858045 shutdown_connections 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.457+0000 7fb516ef4640 1 -- 172.21.15.26:0/2278858045 wait complete. 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.458+0000 7fb516ef4640 1 Processor -- start 2024-03-06T07:21:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.458+0000 7fb516ef4640 1 -- start start 2024-03-06T07:21:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5100c88c0 0x7fb51014bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb51014f360 0x7fb5100af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb510158b60 0x7fb5100bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100c5a50 con 0x7fb51014f360 2024-03-06T07:21:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100c7c30 con 0x7fb510158b60 2024-03-06T07:21:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100c5bd0 con 0x7fb5100c88c0 2024-03-06T07:21:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb51014f360 0x7fb5100af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb51014f360 0x7fb5100af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49942/0 (socket says 172.21.15.26:49942) 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 -- 172.21.15.26:0/3807306640 learned_addr learned my addr 172.21.15.26:0/3807306640 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 -- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5100c88c0 msgr2=0x7fb51014bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb514c90640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5100c88c0 0x7fb51014bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb515491640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb510158b60 0x7fb5100bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5100c88c0 0x7fb51014bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 -- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb510158b60 msgr2=0x7fb5100bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb510158b60 0x7fb5100bfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 -- 172.21.15.26:0/3807306640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51014bbf0 con 0x7fb51014f360 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50ffff640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb51014f360 0x7fb5100af8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c440 tx=0x7fb50400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50d7fa640 1 -- 172.21.15.26:0/3807306640 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5040052d0 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50d7fa640 1 -- 172.21.15.26:0/3807306640 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504005470 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5101eb880 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb50d7fa640 1 -- 172.21.15.26:0/3807306640 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504007260 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.459+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5100ad430 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.460+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5100ad630 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.461+0000 7fb50d7fa640 1 -- 172.21.15.26:0/3807306640 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5040074c0 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.461+0000 7fb50d7fa640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.461+0000 7fb50d7fa640 1 -- 172.21.15.26:0/3807306640 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb504089750 con 0x7fb51014f360 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.462+0000 7fb514c90640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.462+0000 7fb514c90640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb508036000 tx=0x7fb508008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.466+0000 7fb50d7fa640 1 -- 172.21.15.26:0/3807306640 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb5040584d0 con 0x7fb51014f360 2024-03-06T07:21:47.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.816+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb51004f330 con 0x7fb4fc06b4d0 2024-03-06T07:21:47.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.823+0000 7fb50d7fa640 1 -- 172.21.15.26:0/3807306640 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7fb51004f330 con 0x7fb4fc06b4d0 2024-03-06T07:21:47.827 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:47.828 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:47.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb4fc06b4d0 msgr2=0x7fb4fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb508036000 tx=0x7fb508008030 comp rx=0 tx=0).stop 2024-03-06T07:21:47.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb51014f360 msgr2=0x7fb5100af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb51014f360 0x7fb5100af8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c440 tx=0x7fb50400c900 comp rx=0 tx=0).stop 2024-03-06T07:21:47.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 shutdown_connections 2024-03-06T07:21:47.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb510158b60 0x7fb5100bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb51014f360 0x7fb5100af8f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 --2- 172.21.15.26:0/3807306640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5100c88c0 0x7fb51014bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 >> 172.21.15.26:0/3807306640 conn(0x7fb5100ac570 msgr2=0x7fb5100ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 shutdown_connections 2024-03-06T07:21:47.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.826+0000 7fb516ef4640 1 -- 172.21.15.26:0/3807306640 wait complete. 2024-03-06T07:21:47.838 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-03-06T07:21:47.051463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237648,"kb_used_data":4808,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737072,"statfs":{"total":773094113280,"available":772850761728,"internally_reserved":0,"allocated":4923392,"data_stored":3281580,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759424"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608964+0000","last_change":"2024-03-06T07:21:43.317811+0000","last_active":"2024-03-06T07:21:44.608964+0000","last_peered":"2024-03-06T07:21:44.608964+0000","last_clean":"2024-03-06T07:21:44.608964+0000","last_became_active":"2024-03-06T07:21:43.317509+0000","last_became_peered":"2024-03-06T07:21:43.317509+0000","last_unstale":"2024-03-06T07:21:44.608964+0000","last_undegraded":"2024-03-06T07:21:44.608964+0000","last_fullsized":"2024-03-06T07:21:44.608964+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335820+0000","last_change":"2024-03-06T07:21:43.317801+0000","last_active":"2024-03-06T07:21:44.335820+0000","last_peered":"2024-03-06T07:21:44.335820+0000","last_clean":"2024-03-06T07:21:44.335820+0000","last_became_active":"2024-03-06T07:21:43.317522+0000","last_became_peered":"2024-03-06T07:21:43.317522+0000","last_unstale":"2024-03-06T07:21:44.335820+0000","last_undegraded":"2024-03-06T07:21:44.335820+0000","last_fullsized":"2024-03-06T07:21:44.335820+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333258+0000","last_change":"2024-03-06T07:21:43.328071+0000","last_active":"2024-03-06T07:21:44.333258+0000","last_peered":"2024-03-06T07:21:44.333258+0000","last_clean":"2024-03-06T07:21:44.333258+0000","last_became_active":"2024-03-06T07:21:43.327781+0000","last_became_peered":"2024-03-06T07:21:43.327781+0000","last_unstale":"2024-03-06T07:21:44.333258+0000","last_undegraded":"2024-03-06T07:21:44.333258+0000","last_fullsized":"2024-03-06T07:21:44.333258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335595+0000","last_change":"2024-03-06T07:21:43.326286+0000","last_active":"2024-03-06T07:21:44.335595+0000","last_peered":"2024-03-06T07:21:44.335595+0000","last_clean":"2024-03-06T07:21:44.335595+0000","last_became_active":"2024-03-06T07:21:43.326031+0000","last_became_peered":"2024-03-06T07:21:43.326031+0000","last_unstale":"2024-03-06T07:21:44.335595+0000","last_undegraded":"2024-03-06T07:21:44.335595+0000","last_fullsized":"2024-03-06T07:21:44.335595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:21.645487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318644+0000","last_change":"2024-03-06T07:21:43.321810+0000","last_active":"2024-03-06T07:21:44.318644+0000","last_peered":"2024-03-06T07:21:44.318644+0000","last_clean":"2024-03-06T07:21:44.318644+0000","last_became_active":"2024-03-06T07:21:43.321538+0000","last_became_peered":"2024-03-06T07:21:43.321538+0000","last_unstale":"2024-03-06T07:21:44.318644+0000","last_undegraded":"2024-03-06T07:21:44.318644+0000","last_fullsized":"2024-03-06T07:21:44.318644+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:10.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608702+0000","last_change":"2024-03-06T07:21:43.318713+0000","last_active":"2024-03-06T07:21:44.608702+0000","last_peered":"2024-03-06T07:21:44.608702+0000","last_clean":"2024-03-06T07:21:44.608702+0000","last_became_active":"2024-03-06T07:21:43.318385+0000","last_became_peered":"2024-03-06T07:21:43.318385+0000","last_unstale":"2024-03-06T07:21:44.608702+0000","last_undegraded":"2024-03-06T07:21:44.608702+0000","last_fullsized":"2024-03-06T07:21:44.608702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332693+0000","last_change":"2024-03-06T07:21:43.327099+0000","last_active":"2024-03-06T07:21:44.332693+0000","last_peered":"2024-03-06T07:21:44.332693+0000","last_clean":"2024-03-06T07:21:44.332693+0000","last_became_active":"2024-03-06T07:21:43.326851+0000","last_became_peered":"2024-03-06T07:21:43.326851+0000","last_unstale":"2024-03-06T07:21:44.332693+0000","last_undegraded":"2024-03-06T07:21:44.332693+0000","last_fullsized":"2024-03-06T07:21:44.332693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317670+0000","last_change":"2024-03-06T07:21:43.325897+0000","last_active":"2024-03-06T07:21:44.317670+0000","last_peered":"2024-03-06T07:21:44.317670+0000","last_clean":"2024-03-06T07:21:44.317670+0000","last_became_active":"2024-03-06T07:21:43.325647+0000","last_became_peered":"2024-03-06T07:21:43.325647+0000","last_unstale":"2024-03-06T07:21:44.317670+0000","last_undegraded":"2024-03-06T07:21:44.317670+0000","last_fullsized":"2024-03-06T07:21:44.317670+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318058+0000","last_change":"2024-03-06T07:21:43.330337+0000","last_active":"2024-03-06T07:21:44.318058+0000","last_peered":"2024-03-06T07:21:44.318058+0000","last_clean":"2024-03-06T07:21:44.318058+0000","last_became_active":"2024-03-06T07:21:43.330053+0000","last_became_peered":"2024-03-06T07:21:43.330053+0000","last_unstale":"2024-03-06T07:21:44.318058+0000","last_undegraded":"2024-03-06T07:21:44.318058+0000","last_fullsized":"2024-03-06T07:21:44.318058+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608988+0000","last_change":"2024-03-06T07:21:43.325100+0000","last_active":"2024-03-06T07:21:44.608988+0000","last_peered":"2024-03-06T07:21:44.608988+0000","last_clean":"2024-03-06T07:21:44.608988+0000","last_became_active":"2024-03-06T07:21:43.324812+0000","last_became_peered":"2024-03-06T07:21:43.324812+0000","last_unstale":"2024-03-06T07:21:44.608988+0000","last_undegraded":"2024-03-06T07:21:44.608988+0000","last_fullsized":"2024-03-06T07:21:44.608988+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:11.442368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608633+0000","last_change":"2024-03-06T07:21:43.322188+0000","last_active":"2024-03-06T07:21:44.608633+0000","last_peered":"2024-03-06T07:21:44.608633+0000","last_clean":"2024-03-06T07:21:44.608633+0000","last_became_active":"2024-03-06T07:21:43.321685+0000","last_became_peered":"2024-03-06T07:21:43.321685+0000","last_unstale":"2024-03-06T07:21:44.608633+0000","last_undegraded":"2024-03-06T07:21:44.608633+0000","last_fullsized":"2024-03-06T07:21:44.608633+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:54.799043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318167+0000","last_change":"2024-03-06T07:21:43.321952+0000","last_active":"2024-03-06T07:21:44.318167+0000","last_peered":"2024-03-06T07:21:44.318167+0000","last_clean":"2024-03-06T07:21:44.318167+0000","last_became_active":"2024-03-06T07:21:43.321579+0000","last_became_peered":"2024-03-06T07:21:43.321579+0000","last_unstale":"2024-03-06T07:21:44.318167+0000","last_undegraded":"2024-03-06T07:21:44.318167+0000","last_fullsized":"2024-03-06T07:21:44.318167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:04.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608940+0000","last_change":"2024-03-06T07:21:43.321277+0000","last_active":"2024-03-06T07:21:44.608940+0000","last_peered":"2024-03-06T07:21:44.608940+0000","last_clean":"2024-03-06T07:21:44.608940+0000","last_became_active":"2024-03-06T07:21:43.320973+0000","last_became_peered":"2024-03-06T07:21:43.320973+0000","last_unstale":"2024-03-06T07:21:44.608940+0000","last_undegraded":"2024-03-06T07:21:44.608940+0000","last_fullsized":"2024-03-06T07:21:44.608940+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:15.957299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332780+0000","last_change":"2024-03-06T07:21:43.329915+0000","last_active":"2024-03-06T07:21:44.332780+0000","last_peered":"2024-03-06T07:21:44.332780+0000","last_clean":"2024-03-06T07:21:44.332780+0000","last_became_active":"2024-03-06T07:21:43.328775+0000","last_became_peered":"2024-03-06T07:21:43.328775+0000","last_unstale":"2024-03-06T07:21:44.332780+0000","last_undegraded":"2024-03-06T07:21:44.332780+0000","last_fullsized":"2024-03-06T07:21:44.332780+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335398+0000","last_change":"2024-03-06T07:21:43.321137+0000","last_active":"2024-03-06T07:21:44.335398+0000","last_peered":"2024-03-06T07:21:44.335398+0000","last_clean":"2024-03-06T07:21:44.335398+0000","last_became_active":"2024-03-06T07:21:43.320820+0000","last_became_peered":"2024-03-06T07:21:43.320820+0000","last_unstale":"2024-03-06T07:21:44.335398+0000","last_undegraded":"2024-03-06T07:21:44.335398+0000","last_fullsized":"2024-03-06T07:21:44.335398+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:52.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608712+0000","last_change":"2024-03-06T07:21:43.320518+0000","last_active":"2024-03-06T07:21:44.608712+0000","last_peered":"2024-03-06T07:21:44.608712+0000","last_clean":"2024-03-06T07:21:44.608712+0000","last_became_active":"2024-03-06T07:21:43.320178+0000","last_became_peered":"2024-03-06T07:21:43.320178+0000","last_unstale":"2024-03-06T07:21:44.608712+0000","last_undegraded":"2024-03-06T07:21:44.608712+0000","last_fullsized":"2024-03-06T07:21:44.608712+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333101+0000","last_change":"2024-03-06T07:21:43.324279+0000","last_active":"2024-03-06T07:21:44.333101+0000","last_peered":"2024-03-06T07:21:44.333101+0000","last_clean":"2024-03-06T07:21:44.333101+0000","last_became_active":"2024-03-06T07:21:43.323499+0000","last_became_peered":"2024-03-06T07:21:43.323499+0000","last_unstale":"2024-03-06T07:21:44.333101+0000","last_undegraded":"2024-03-06T07:21:44.333101+0000","last_fullsized":"2024-03-06T07:21:44.333101+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335836+0000","last_change":"2024-03-06T07:21:43.318202+0000","last_active":"2024-03-06T07:21:44.335836+0000","last_peered":"2024-03-06T07:21:44.335836+0000","last_clean":"2024-03-06T07:21:44.335836+0000","last_became_active":"2024-03-06T07:21:43.317929+0000","last_became_peered":"2024-03-06T07:21:43.317929+0000","last_unstale":"2024-03-06T07:21:44.335836+0000","last_undegraded":"2024-03-06T07:21:44.335836+0000","last_fullsized":"2024-03-06T07:21:44.335836+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333309+0000","last_change":"2024-03-06T07:21:43.324265+0000","last_active":"2024-03-06T07:21:44.333309+0000","last_peered":"2024-03-06T07:21:44.333309+0000","last_clean":"2024-03-06T07:21:44.333309+0000","last_became_active":"2024-03-06T07:21:43.323877+0000","last_became_peered":"2024-03-06T07:21:43.323877+0000","last_unstale":"2024-03-06T07:21:44.333309+0000","last_undegraded":"2024-03-06T07:21:44.333309+0000","last_fullsized":"2024-03-06T07:21:44.333309+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609270+0000","last_change":"2024-03-06T07:21:43.321580+0000","last_active":"2024-03-06T07:21:44.609270+0000","last_peered":"2024-03-06T07:21:44.609270+0000","last_clean":"2024-03-06T07:21:44.609270+0000","last_became_active":"2024-03-06T07:21:43.319775+0000","last_became_peered":"2024-03-06T07:21:43.319775+0000","last_unstale":"2024-03-06T07:21:44.609270+0000","last_undegraded":"2024-03-06T07:21:44.609270+0000","last_fullsized":"2024-03-06T07:21:44.609270+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:32.802384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609293+0000","last_change":"2024-03-06T07:21:43.322797+0000","last_active":"2024-03-06T07:21:44.609293+0000","last_peered":"2024-03-06T07:21:44.609293+0000","last_clean":"2024-03-06T07:21:44.609293+0000","last_became_active":"2024-03-06T07:21:43.322505+0000","last_became_peered":"2024-03-06T07:21:43.322505+0000","last_unstale":"2024-03-06T07:21:44.609293+0000","last_undegraded":"2024-03-06T07:21:44.609293+0000","last_fullsized":"2024-03-06T07:21:44.609293+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317902+0000","last_change":"2024-03-06T07:21:43.326058+0000","last_active":"2024-03-06T07:21:44.317902+0000","last_peered":"2024-03-06T07:21:44.317902+0000","last_clean":"2024-03-06T07:21:44.317902+0000","last_became_active":"2024-03-06T07:21:43.325799+0000","last_became_peered":"2024-03-06T07:21:43.325799+0000","last_unstale":"2024-03-06T07:21:44.317902+0000","last_undegraded":"2024-03-06T07:21:44.317902+0000","last_fullsized":"2024-03-06T07:21:44.317902+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608601+0000","last_change":"2024-03-06T07:21:43.322671+0000","last_active":"2024-03-06T07:21:44.608601+0000","last_peered":"2024-03-06T07:21:44.608601+0000","last_clean":"2024-03-06T07:21:44.608601+0000","last_became_active":"2024-03-06T07:21:43.322376+0000","last_became_peered":"2024-03-06T07:21:43.322376+0000","last_unstale":"2024-03-06T07:21:44.608601+0000","last_undegraded":"2024-03-06T07:21:44.608601+0000","last_fullsized":"2024-03-06T07:21:44.608601+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:15.049663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332658+0000","last_change":"2024-03-06T07:21:43.327014+0000","last_active":"2024-03-06T07:21:44.332658+0000","last_peered":"2024-03-06T07:21:44.332658+0000","last_clean":"2024-03-06T07:21:44.332658+0000","last_became_active":"2024-03-06T07:21:43.326759+0000","last_became_peered":"2024-03-06T07:21:43.326759+0000","last_unstale":"2024-03-06T07:21:44.332658+0000","last_undegraded":"2024-03-06T07:21:44.332658+0000","last_fullsized":"2024-03-06T07:21:44.332658+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333245+0000","last_change":"2024-03-06T07:21:43.323199+0000","last_active":"2024-03-06T07:21:44.333245+0000","last_peered":"2024-03-06T07:21:44.333245+0000","last_clean":"2024-03-06T07:21:44.333245+0000","last_became_active":"2024-03-06T07:21:43.322835+0000","last_became_peered":"2024-03-06T07:21:43.322835+0000","last_unstale":"2024-03-06T07:21:44.333245+0000","last_undegraded":"2024-03-06T07:21:44.333245+0000","last_fullsized":"2024-03-06T07:21:44.333245+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317893+0000","last_change":"2024-03-06T07:21:43.318509+0000","last_active":"2024-03-06T07:21:44.317893+0000","last_peered":"2024-03-06T07:21:44.317893+0000","last_clean":"2024-03-06T07:21:44.317893+0000","last_became_active":"2024-03-06T07:21:43.318177+0000","last_became_peered":"2024-03-06T07:21:43.318177+0000","last_unstale":"2024-03-06T07:21:44.317893+0000","last_undegraded":"2024-03-06T07:21:44.317893+0000","last_fullsized":"2024-03-06T07:21:44.317893+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609147+0000","last_change":"2024-03-06T07:21:43.332733+0000","last_active":"2024-03-06T07:21:44.609147+0000","last_peered":"2024-03-06T07:21:44.609147+0000","last_clean":"2024-03-06T07:21:44.609147+0000","last_became_active":"2024-03-06T07:21:43.331512+0000","last_became_peered":"2024-03-06T07:21:43.331512+0000","last_unstale":"2024-03-06T07:21:44.609147+0000","last_undegraded":"2024-03-06T07:21:44.609147+0000","last_fullsized":"2024-03-06T07:21:44.609147+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609342+0000","last_change":"2024-03-06T07:21:43.321054+0000","last_active":"2024-03-06T07:21:44.609342+0000","last_peered":"2024-03-06T07:21:44.609342+0000","last_clean":"2024-03-06T07:21:44.609342+0000","last_became_active":"2024-03-06T07:21:43.319435+0000","last_became_peered":"2024-03-06T07:21:43.319435+0000","last_unstale":"2024-03-06T07:21:44.609342+0000","last_undegraded":"2024-03-06T07:21:44.609342+0000","last_fullsized":"2024-03-06T07:21:44.609342+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332809+0000","last_change":"2024-03-06T07:21:43.328812+0000","last_active":"2024-03-06T07:21:44.332809+0000","last_peered":"2024-03-06T07:21:44.332809+0000","last_clean":"2024-03-06T07:21:44.332809+0000","last_became_active":"2024-03-06T07:21:43.328366+0000","last_became_peered":"2024-03-06T07:21:43.328366+0000","last_unstale":"2024-03-06T07:21:44.332809+0000","last_undegraded":"2024-03-06T07:21:44.332809+0000","last_fullsized":"2024-03-06T07:21:44.332809+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:48.381105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335423+0000","last_change":"2024-03-06T07:21:43.321155+0000","last_active":"2024-03-06T07:21:44.335423+0000","last_peered":"2024-03-06T07:21:44.335423+0000","last_clean":"2024-03-06T07:21:44.335423+0000","last_became_active":"2024-03-06T07:21:43.320747+0000","last_became_peered":"2024-03-06T07:21:43.320747+0000","last_unstale":"2024-03-06T07:21:44.335423+0000","last_undegraded":"2024-03-06T07:21:44.335423+0000","last_fullsized":"2024-03-06T07:21:44.335423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:42.424597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332779+0000","last_change":"2024-03-06T07:21:43.326550+0000","last_active":"2024-03-06T07:21:44.332779+0000","last_peered":"2024-03-06T07:21:44.332779+0000","last_clean":"2024-03-06T07:21:44.332779+0000","last_became_active":"2024-03-06T07:21:43.326254+0000","last_became_peered":"2024-03-06T07:21:43.326254+0000","last_unstale":"2024-03-06T07:21:44.332779+0000","last_undegraded":"2024-03-06T07:21:44.332779+0000","last_fullsized":"2024-03-06T07:21:44.332779+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447732+0000","last_change":"2024-03-06T07:21:43.322773+0000","last_active":"2024-03-06T07:21:44.447732+0000","last_peered":"2024-03-06T07:21:44.447732+0000","last_clean":"2024-03-06T07:21:44.447732+0000","last_became_active":"2024-03-06T07:21:43.322525+0000","last_became_peered":"2024-03-06T07:21:43.322525+0000","last_unstale":"2024-03-06T07:21:44.447732+0000","last_undegraded":"2024-03-06T07:21:44.447732+0000","last_fullsized":"2024-03-06T07:21:44.447732+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335210+0000","last_change":"2024-03-06T07:21:43.319837+0000","last_active":"2024-03-06T07:21:44.335210+0000","last_peered":"2024-03-06T07:21:44.335210+0000","last_clean":"2024-03-06T07:21:44.335210+0000","last_became_active":"2024-03-06T07:21:43.319564+0000","last_became_peered":"2024-03-06T07:21:43.319564+0000","last_unstale":"2024-03-06T07:21:44.335210+0000","last_undegraded":"2024-03-06T07:21:44.335210+0000","last_fullsized":"2024-03-06T07:21:44.335210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609434+0000","last_change":"2024-03-06T07:21:42.310584+0000","last_active":"2024-03-06T07:21:44.609434+0000","last_peered":"2024-03-06T07:21:44.609434+0000","last_clean":"2024-03-06T07:21:44.609434+0000","last_became_active":"2024-03-06T07:21:42.310317+0000","last_became_peered":"2024-03-06T07:21:42.310317+0000","last_unstale":"2024-03-06T07:21:44.609434+0000","last_undegraded":"2024-03-06T07:21:44.609434+0000","last_fullsized":"2024-03-06T07:21:44.609434+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447956+0000","last_change":"2024-03-06T07:21:43.323883+0000","last_active":"2024-03-06T07:21:44.447956+0000","last_peered":"2024-03-06T07:21:44.447956+0000","last_clean":"2024-03-06T07:21:44.447956+0000","last_became_active":"2024-03-06T07:21:43.323613+0000","last_became_peered":"2024-03-06T07:21:43.323613+0000","last_unstale":"2024-03-06T07:21:44.447956+0000","last_undegraded":"2024-03-06T07:21:44.447956+0000","last_fullsized":"2024-03-06T07:21:44.447956+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609091+0000","last_change":"2024-03-06T07:21:42.304199+0000","last_active":"2024-03-06T07:21:44.609091+0000","last_peered":"2024-03-06T07:21:44.609091+0000","last_clean":"2024-03-06T07:21:44.609091+0000","last_became_active":"2024-03-06T07:21:42.303802+0000","last_became_peered":"2024-03-06T07:21:42.303802+0000","last_unstale":"2024-03-06T07:21:44.609091+0000","last_undegraded":"2024-03-06T07:21:44.609091+0000","last_fullsized":"2024-03-06T07:21:44.609091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609109+0000","last_change":"2024-03-06T07:21:43.323512+0000","last_active":"2024-03-06T07:21:44.609109+0000","last_peered":"2024-03-06T07:21:44.609109+0000","last_clean":"2024-03-06T07:21:44.609109+0000","last_became_active":"2024-03-06T07:21:43.323216+0000","last_became_peered":"2024-03-06T07:21:43.323216+0000","last_unstale":"2024-03-06T07:21:44.609109+0000","last_undegraded":"2024-03-06T07:21:44.609109+0000","last_fullsized":"2024-03-06T07:21:44.609109+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:31.821522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609201+0000","last_change":"2024-03-06T07:21:42.305711+0000","last_active":"2024-03-06T07:21:44.609201+0000","last_peered":"2024-03-06T07:21:44.609201+0000","last_clean":"2024-03-06T07:21:44.609201+0000","last_became_active":"2024-03-06T07:21:42.305431+0000","last_became_peered":"2024-03-06T07:21:42.305431+0000","last_unstale":"2024-03-06T07:21:44.609201+0000","last_undegraded":"2024-03-06T07:21:44.609201+0000","last_fullsized":"2024-03-06T07:21:44.609201+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317784+0000","last_change":"2024-03-06T07:21:43.317966+0000","last_active":"2024-03-06T07:21:44.317784+0000","last_peered":"2024-03-06T07:21:44.317784+0000","last_clean":"2024-03-06T07:21:44.317784+0000","last_became_active":"2024-03-06T07:21:43.317700+0000","last_became_peered":"2024-03-06T07:21:43.317700+0000","last_unstale":"2024-03-06T07:21:44.317784+0000","last_undegraded":"2024-03-06T07:21:44.317784+0000","last_fullsized":"2024-03-06T07:21:44.317784+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447944+0000","last_change":"2024-03-06T07:21:42.315045+0000","last_active":"2024-03-06T07:21:44.447944+0000","last_peered":"2024-03-06T07:21:44.447944+0000","last_clean":"2024-03-06T07:21:44.447944+0000","last_became_active":"2024-03-06T07:21:42.314768+0000","last_became_peered":"2024-03-06T07:21:42.314768+0000","last_unstale":"2024-03-06T07:21:44.447944+0000","last_undegraded":"2024-03-06T07:21:44.447944+0000","last_fullsized":"2024-03-06T07:21:44.447944+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:14.953476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318579+0000","last_change":"2024-03-06T07:21:43.324372+0000","last_active":"2024-03-06T07:21:44.318579+0000","last_peered":"2024-03-06T07:21:44.318579+0000","last_clean":"2024-03-06T07:21:44.318579+0000","last_became_active":"2024-03-06T07:21:43.324097+0000","last_became_peered":"2024-03-06T07:21:43.324097+0000","last_unstale":"2024-03-06T07:21:44.318579+0000","last_undegraded":"2024-03-06T07:21:44.318579+0000","last_fullsized":"2024-03-06T07:21:44.318579+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:49:40.932248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332776+0000","last_change":"2024-03-06T07:21:42.317614+0000","last_active":"2024-03-06T07:21:44.332776+0000","last_peered":"2024-03-06T07:21:44.332776+0000","last_clean":"2024-03-06T07:21:44.332776+0000","last_became_active":"2024-03-06T07:21:42.317094+0000","last_became_peered":"2024-03-06T07:21:42.317094+0000","last_unstale":"2024-03-06T07:21:44.332776+0000","last_undegraded":"2024-03-06T07:21:44.332776+0000","last_fullsized":"2024-03-06T07:21:44.332776+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:15.739191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335623+0000","last_change":"2024-03-06T07:21:43.318727+0000","last_active":"2024-03-06T07:21:44.335623+0000","last_peered":"2024-03-06T07:21:44.335623+0000","last_clean":"2024-03-06T07:21:44.335623+0000","last_became_active":"2024-03-06T07:21:43.318440+0000","last_became_peered":"2024-03-06T07:21:43.318440+0000","last_unstale":"2024-03-06T07:21:44.335623+0000","last_undegraded":"2024-03-06T07:21:44.335623+0000","last_fullsized":"2024-03-06T07:21:44.335623+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318489+0000","last_change":"2024-03-06T07:21:42.307726+0000","last_active":"2024-03-06T07:21:44.318489+0000","last_peered":"2024-03-06T07:21:44.318489+0000","last_clean":"2024-03-06T07:21:44.318489+0000","last_became_active":"2024-03-06T07:21:42.307445+0000","last_became_peered":"2024-03-06T07:21:42.307445+0000","last_unstale":"2024-03-06T07:21:44.318489+0000","last_undegraded":"2024-03-06T07:21:44.318489+0000","last_fullsized":"2024-03-06T07:21:44.318489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318063+0000","last_change":"2024-03-06T07:21:43.318372+0000","last_active":"2024-03-06T07:21:44.318063+0000","last_peered":"2024-03-06T07:21:44.318063+0000","last_clean":"2024-03-06T07:21:44.318063+0000","last_became_active":"2024-03-06T07:21:43.318047+0000","last_became_peered":"2024-03-06T07:21:43.318047+0000","last_unstale":"2024-03-06T07:21:44.318063+0000","last_undegraded":"2024-03-06T07:21:44.318063+0000","last_fullsized":"2024-03-06T07:21:44.318063+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609065+0000","last_change":"2024-03-06T07:21:42.310677+0000","last_active":"2024-03-06T07:21:44.609065+0000","last_peered":"2024-03-06T07:21:44.609065+0000","last_clean":"2024-03-06T07:21:44.609065+0000","last_became_active":"2024-03-06T07:21:42.310375+0000","last_became_peered":"2024-03-06T07:21:42.310375+0000","last_unstale":"2024-03-06T07:21:44.609065+0000","last_undegraded":"2024-03-06T07:21:44.609065+0000","last_fullsized":"2024-03-06T07:21:44.609065+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:13.837451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335829+0000","last_change":"2024-03-06T07:21:43.325740+0000","last_active":"2024-03-06T07:21:44.335829+0000","last_peered":"2024-03-06T07:21:44.335829+0000","last_clean":"2024-03-06T07:21:44.335829+0000","last_became_active":"2024-03-06T07:21:43.325425+0000","last_became_peered":"2024-03-06T07:21:43.325425+0000","last_unstale":"2024-03-06T07:21:44.335829+0000","last_undegraded":"2024-03-06T07:21:44.335829+0000","last_fullsized":"2024-03-06T07:21:44.335829+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332650+0000","last_change":"2024-03-06T07:21:42.319786+0000","last_active":"2024-03-06T07:21:44.332650+0000","last_peered":"2024-03-06T07:21:44.332650+0000","last_clean":"2024-03-06T07:21:44.332650+0000","last_became_active":"2024-03-06T07:21:42.319430+0000","last_became_peered":"2024-03-06T07:21:42.319430+0000","last_unstale":"2024-03-06T07:21:44.332650+0000","last_undegraded":"2024-03-06T07:21:44.332650+0000","last_fullsized":"2024-03-06T07:21:44.332650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:09.719754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318319+0000","last_change":"2024-03-06T07:21:43.330805+0000","last_active":"2024-03-06T07:21:44.318319+0000","last_peered":"2024-03-06T07:21:44.318319+0000","last_clean":"2024-03-06T07:21:44.318319+0000","last_became_active":"2024-03-06T07:21:43.330267+0000","last_became_peered":"2024-03-06T07:21:43.330267+0000","last_unstale":"2024-03-06T07:21:44.318319+0000","last_undegraded":"2024-03-06T07:21:44.318319+0000","last_fullsized":"2024-03-06T07:21:44.318319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:56.716578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335767+0000","last_change":"2024-03-06T07:21:42.312213+0000","last_active":"2024-03-06T07:21:44.335767+0000","last_peered":"2024-03-06T07:21:44.335767+0000","last_clean":"2024-03-06T07:21:44.335767+0000","last_became_active":"2024-03-06T07:21:42.311878+0000","last_became_peered":"2024-03-06T07:21:42.311878+0000","last_unstale":"2024-03-06T07:21:44.335767+0000","last_undegraded":"2024-03-06T07:21:44.335767+0000","last_fullsized":"2024-03-06T07:21:44.335767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447790+0000","last_change":"2024-03-06T07:21:43.320122+0000","last_active":"2024-03-06T07:21:44.447790+0000","last_peered":"2024-03-06T07:21:44.447790+0000","last_clean":"2024-03-06T07:21:44.447790+0000","last_became_active":"2024-03-06T07:21:43.319799+0000","last_became_peered":"2024-03-06T07:21:43.319799+0000","last_unstale":"2024-03-06T07:21:44.447790+0000","last_undegraded":"2024-03-06T07:21:44.447790+0000","last_fullsized":"2024-03-06T07:21:44.447790+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318501+0000","last_change":"2024-03-06T07:21:42.319681+0000","last_active":"2024-03-06T07:21:44.318501+0000","last_peered":"2024-03-06T07:21:44.318501+0000","last_clean":"2024-03-06T07:21:44.318501+0000","last_became_active":"2024-03-06T07:21:42.319452+0000","last_became_peered":"2024-03-06T07:21:42.319452+0000","last_unstale":"2024-03-06T07:21:44.318501+0000","last_undegraded":"2024-03-06T07:21:44.318501+0000","last_fullsized":"2024-03-06T07:21:44.318501+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:35.308790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333038+0000","last_change":"2024-03-06T07:21:43.323490+0000","last_active":"2024-03-06T07:21:44.333038+0000","last_peered":"2024-03-06T07:21:44.333038+0000","last_clean":"2024-03-06T07:21:44.333038+0000","last_became_active":"2024-03-06T07:21:43.323088+0000","last_became_peered":"2024-03-06T07:21:43.323088+0000","last_unstale":"2024-03-06T07:21:44.333038+0000","last_undegraded":"2024-03-06T07:21:44.333038+0000","last_fullsized":"2024-03-06T07:21:44.333038+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335765+0000","last_change":"2024-03-06T07:21:42.312732+0000","last_active":"2024-03-06T07:21:44.335765+0000","last_peered":"2024-03-06T07:21:44.335765+0000","last_clean":"2024-03-06T07:21:44.335765+0000","last_became_active":"2024-03-06T07:21:42.312462+0000","last_became_peered":"2024-03-06T07:21:42.312462+0000","last_unstale":"2024-03-06T07:21:44.335765+0000","last_undegraded":"2024-03-06T07:21:44.335765+0000","last_fullsized":"2024-03-06T07:21:44.335765+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318716+0000","last_change":"2024-03-06T07:21:43.320022+0000","last_active":"2024-03-06T07:21:44.318716+0000","last_peered":"2024-03-06T07:21:44.318716+0000","last_clean":"2024-03-06T07:21:44.318716+0000","last_became_active":"2024-03-06T07:21:43.319716+0000","last_became_peered":"2024-03-06T07:21:43.319716+0000","last_unstale":"2024-03-06T07:21:44.318716+0000","last_undegraded":"2024-03-06T07:21:44.318716+0000","last_fullsized":"2024-03-06T07:21:44.318716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:50.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318087+0000","last_change":"2024-03-06T07:21:42.308298+0000","last_active":"2024-03-06T07:21:44.318087+0000","last_peered":"2024-03-06T07:21:44.318087+0000","last_clean":"2024-03-06T07:21:44.318087+0000","last_became_active":"2024-03-06T07:21:42.308008+0000","last_became_peered":"2024-03-06T07:21:42.308008+0000","last_unstale":"2024-03-06T07:21:44.318087+0000","last_undegraded":"2024-03-06T07:21:44.318087+0000","last_fullsized":"2024-03-06T07:21:44.318087+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:51.723259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447774+0000","last_change":"2024-03-06T07:21:43.323228+0000","last_active":"2024-03-06T07:21:44.447774+0000","last_peered":"2024-03-06T07:21:44.447774+0000","last_clean":"2024-03-06T07:21:44.447774+0000","last_became_active":"2024-03-06T07:21:43.322951+0000","last_became_peered":"2024-03-06T07:21:43.322951+0000","last_unstale":"2024-03-06T07:21:44.447774+0000","last_undegraded":"2024-03-06T07:21:44.447774+0000","last_fullsized":"2024-03-06T07:21:44.447774+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332880+0000","last_change":"2024-03-06T07:21:42.321872+0000","last_active":"2024-03-06T07:21:44.332880+0000","last_peered":"2024-03-06T07:21:44.332880+0000","last_clean":"2024-03-06T07:21:44.332880+0000","last_became_active":"2024-03-06T07:21:42.321384+0000","last_became_peered":"2024-03-06T07:21:42.321384+0000","last_unstale":"2024-03-06T07:21:44.332880+0000","last_undegraded":"2024-03-06T07:21:44.332880+0000","last_fullsized":"2024-03-06T07:21:44.332880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333396+0000","last_change":"2024-03-06T07:21:43.324244+0000","last_active":"2024-03-06T07:21:44.333396+0000","last_peered":"2024-03-06T07:21:44.333396+0000","last_clean":"2024-03-06T07:21:44.333396+0000","last_became_active":"2024-03-06T07:21:43.323549+0000","last_became_peered":"2024-03-06T07:21:43.323549+0000","last_unstale":"2024-03-06T07:21:44.333396+0000","last_undegraded":"2024-03-06T07:21:44.333396+0000","last_fullsized":"2024-03-06T07:21:44.333396+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609239+0000","last_change":"2024-03-06T07:21:42.318308+0000","last_active":"2024-03-06T07:21:44.609239+0000","last_peered":"2024-03-06T07:21:44.609239+0000","last_clean":"2024-03-06T07:21:44.609239+0000","last_became_active":"2024-03-06T07:21:42.318080+0000","last_became_peered":"2024-03-06T07:21:42.318080+0000","last_unstale":"2024-03-06T07:21:44.609239+0000","last_undegraded":"2024-03-06T07:21:44.609239+0000","last_fullsized":"2024-03-06T07:21:44.609239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317946+0000","last_change":"2024-03-06T07:21:43.331853+0000","last_active":"2024-03-06T07:21:44.317946+0000","last_peered":"2024-03-06T07:21:44.317946+0000","last_clean":"2024-03-06T07:21:44.317946+0000","last_became_active":"2024-03-06T07:21:43.331560+0000","last_became_peered":"2024-03-06T07:21:43.331560+0000","last_unstale":"2024-03-06T07:21:44.317946+0000","last_undegraded":"2024-03-06T07:21:44.317946+0000","last_fullsized":"2024-03-06T07:21:44.317946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335375+0000","last_change":"2024-03-06T07:21:42.309167+0000","last_active":"2024-03-06T07:21:44.335375+0000","last_peered":"2024-03-06T07:21:44.335375+0000","last_clean":"2024-03-06T07:21:44.335375+0000","last_became_active":"2024-03-06T07:21:42.308882+0000","last_became_peered":"2024-03-06T07:21:42.308882+0000","last_unstale":"2024-03-06T07:21:44.335375+0000","last_undegraded":"2024-03-06T07:21:44.335375+0000","last_fullsized":"2024-03-06T07:21:44.335375+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333062+0000","last_change":"2024-03-06T07:21:43.324536+0000","last_active":"2024-03-06T07:21:44.333062+0000","last_peered":"2024-03-06T07:21:44.333062+0000","last_clean":"2024-03-06T07:21:44.333062+0000","last_became_active":"2024-03-06T07:21:43.323818+0000","last_became_peered":"2024-03-06T07:21:43.323818+0000","last_unstale":"2024-03-06T07:21:44.333062+0000","last_undegraded":"2024-03-06T07:21:44.333062+0000","last_fullsized":"2024-03-06T07:21:44.333062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317773+0000","last_change":"2024-03-06T07:21:42.308460+0000","last_active":"2024-03-06T07:21:44.317773+0000","last_peered":"2024-03-06T07:21:44.317773+0000","last_clean":"2024-03-06T07:21:44.317773+0000","last_became_active":"2024-03-06T07:21:42.307461+0000","last_became_peered":"2024-03-06T07:21:42.307461+0000","last_unstale":"2024-03-06T07:21:44.317773+0000","last_undegraded":"2024-03-06T07:21:44.317773+0000","last_fullsized":"2024-03-06T07:21:44.317773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:34.346180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317733+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:44.317733+0000","last_peered":"2024-03-06T07:21:44.317733+0000","last_clean":"2024-03-06T07:21:44.317733+0000","last_became_active":"2024-03-06T07:21:43.316848+0000","last_became_peered":"2024-03-06T07:21:43.316848+0000","last_unstale":"2024-03-06T07:21:44.317733+0000","last_undegraded":"2024-03-06T07:21:44.317733+0000","last_fullsized":"2024-03-06T07:21:44.317733+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333219+0000","last_change":"2024-03-06T07:21:42.319998+0000","last_active":"2024-03-06T07:21:44.333219+0000","last_peered":"2024-03-06T07:21:44.333219+0000","last_clean":"2024-03-06T07:21:44.333219+0000","last_became_active":"2024-03-06T07:21:42.319529+0000","last_became_peered":"2024-03-06T07:21:42.319529+0000","last_unstale":"2024-03-06T07:21:44.333219+0000","last_undegraded":"2024-03-06T07:21:44.333219+0000","last_fullsized":"2024-03-06T07:21:44.333219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608621+0000","last_change":"2024-03-06T07:21:43.327074+0000","last_active":"2024-03-06T07:21:44.608621+0000","last_peered":"2024-03-06T07:21:44.608621+0000","last_clean":"2024-03-06T07:21:44.608621+0000","last_became_active":"2024-03-06T07:21:43.326797+0000","last_became_peered":"2024-03-06T07:21:43.326797+0000","last_unstale":"2024-03-06T07:21:44.608621+0000","last_undegraded":"2024-03-06T07:21:44.608621+0000","last_fullsized":"2024-03-06T07:21:44.608621+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608792+0000","last_change":"2024-03-06T07:21:42.310454+0000","last_active":"2024-03-06T07:21:44.608792+0000","last_peered":"2024-03-06T07:21:44.608792+0000","last_clean":"2024-03-06T07:21:44.608792+0000","last_became_active":"2024-03-06T07:21:42.310172+0000","last_became_peered":"2024-03-06T07:21:42.310172+0000","last_unstale":"2024-03-06T07:21:44.608792+0000","last_undegraded":"2024-03-06T07:21:44.608792+0000","last_fullsized":"2024-03-06T07:21:44.608792+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:34.182536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318530+0000","last_change":"2024-03-06T07:21:43.324398+0000","last_active":"2024-03-06T07:21:44.318530+0000","last_peered":"2024-03-06T07:21:44.318530+0000","last_clean":"2024-03-06T07:21:44.318530+0000","last_became_active":"2024-03-06T07:21:43.324115+0000","last_became_peered":"2024-03-06T07:21:43.324115+0000","last_unstale":"2024-03-06T07:21:44.318530+0000","last_undegraded":"2024-03-06T07:21:44.318530+0000","last_fullsized":"2024-03-06T07:21:44.318530+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:36.888023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609426+0000","last_change":"2024-03-06T07:21:42.312390+0000","last_active":"2024-03-06T07:21:44.609426+0000","last_peered":"2024-03-06T07:21:44.609426+0000","last_clean":"2024-03-06T07:21:44.609426+0000","last_became_active":"2024-03-06T07:21:42.312079+0000","last_became_peered":"2024-03-06T07:21:42.312079+0000","last_unstale":"2024-03-06T07:21:44.609426+0000","last_undegraded":"2024-03-06T07:21:44.609426+0000","last_fullsized":"2024-03-06T07:21:44.609426+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609086+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:44.609086+0000","last_peered":"2024-03-06T07:21:44.609086+0000","last_clean":"2024-03-06T07:21:44.609086+0000","last_became_active":"2024-03-06T07:21:43.316857+0000","last_became_peered":"2024-03-06T07:21:43.316857+0000","last_unstale":"2024-03-06T07:21:44.609086+0000","last_undegraded":"2024-03-06T07:21:44.609086+0000","last_fullsized":"2024-03-06T07:21:44.609086+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:48.715734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609389+0000","last_change":"2024-03-06T07:21:42.311972+0000","last_active":"2024-03-06T07:21:44.609389+0000","last_peered":"2024-03-06T07:21:44.609389+0000","last_clean":"2024-03-06T07:21:44.609389+0000","last_became_active":"2024-03-06T07:21:42.311640+0000","last_became_peered":"2024-03-06T07:21:42.311640+0000","last_unstale":"2024-03-06T07:21:44.609389+0000","last_undegraded":"2024-03-06T07:21:44.609389+0000","last_fullsized":"2024-03-06T07:21:44.609389+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:22.568258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447949+0000","last_change":"2024-03-06T07:21:43.320374+0000","last_active":"2024-03-06T07:21:44.447949+0000","last_peered":"2024-03-06T07:21:44.447949+0000","last_clean":"2024-03-06T07:21:44.447949+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:44.447949+0000","last_undegraded":"2024-03-06T07:21:44.447949+0000","last_fullsized":"2024-03-06T07:21:44.447949+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609053+0000","last_change":"2024-03-06T07:21:42.309901+0000","last_active":"2024-03-06T07:21:44.609053+0000","last_peered":"2024-03-06T07:21:44.609053+0000","last_clean":"2024-03-06T07:21:44.609053+0000","last_became_active":"2024-03-06T07:21:42.309532+0000","last_became_peered":"2024-03-06T07:21:42.309532+0000","last_unstale":"2024-03-06T07:21:44.609053+0000","last_undegraded":"2024-03-06T07:21:44.609053+0000","last_fullsized":"2024-03-06T07:21:44.609053+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609397+0000","last_change":"2024-03-06T07:21:43.320820+0000","last_active":"2024-03-06T07:21:44.609397+0000","last_peered":"2024-03-06T07:21:44.609397+0000","last_clean":"2024-03-06T07:21:44.609397+0000","last_became_active":"2024-03-06T07:21:43.319225+0000","last_became_peered":"2024-03-06T07:21:43.319225+0000","last_unstale":"2024-03-06T07:21:44.609397+0000","last_undegraded":"2024-03-06T07:21:44.609397+0000","last_fullsized":"2024-03-06T07:21:44.609397+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317589+0000","last_change":"2024-03-06T07:21:42.307676+0000","last_active":"2024-03-06T07:21:44.317589+0000","last_peered":"2024-03-06T07:21:44.317589+0000","last_clean":"2024-03-06T07:21:44.317589+0000","last_became_active":"2024-03-06T07:21:42.307388+0000","last_became_peered":"2024-03-06T07:21:42.307388+0000","last_unstale":"2024-03-06T07:21:44.317589+0000","last_undegraded":"2024-03-06T07:21:44.317589+0000","last_fullsized":"2024-03-06T07:21:44.317589+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:47.711911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333072+0000","last_change":"2024-03-06T07:21:43.329841+0000","last_active":"2024-03-06T07:21:44.333072+0000","last_peered":"2024-03-06T07:21:44.333072+0000","last_clean":"2024-03-06T07:21:44.333072+0000","last_became_active":"2024-03-06T07:21:43.329526+0000","last_became_peered":"2024-03-06T07:21:43.329526+0000","last_unstale":"2024-03-06T07:21:44.333072+0000","last_undegraded":"2024-03-06T07:21:44.333072+0000","last_fullsized":"2024-03-06T07:21:44.333072+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332814+0000","last_change":"2024-03-06T07:21:42.318168+0000","last_active":"2024-03-06T07:21:44.332814+0000","last_peered":"2024-03-06T07:21:44.332814+0000","last_clean":"2024-03-06T07:21:44.332814+0000","last_became_active":"2024-03-06T07:21:42.317866+0000","last_became_peered":"2024-03-06T07:21:42.317866+0000","last_unstale":"2024-03-06T07:21:44.332814+0000","last_undegraded":"2024-03-06T07:21:44.332814+0000","last_fullsized":"2024-03-06T07:21:44.332814+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:49.054000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609107+0000","last_change":"2024-03-06T07:21:43.316009+0000","last_active":"2024-03-06T07:21:44.609107+0000","last_peered":"2024-03-06T07:21:44.609107+0000","last_clean":"2024-03-06T07:21:44.609107+0000","last_became_active":"2024-03-06T07:21:43.315699+0000","last_became_peered":"2024-03-06T07:21:43.315699+0000","last_unstale":"2024-03-06T07:21:44.609107+0000","last_undegraded":"2024-03-06T07:21:44.609107+0000","last_fullsized":"2024-03-06T07:21:44.609107+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:52.769462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.608468+0000","last_change":"2024-03-06T07:21:42.311690+0000","last_active":"2024-03-06T07:21:44.608468+0000","last_peered":"2024-03-06T07:21:44.608468+0000","last_clean":"2024-03-06T07:21:44.608468+0000","last_became_active":"2024-03-06T07:21:42.311417+0000","last_became_peered":"2024-03-06T07:21:42.311417+0000","last_unstale":"2024-03-06T07:21:44.608468+0000","last_undegraded":"2024-03-06T07:21:44.608468+0000","last_fullsized":"2024-03-06T07:21:44.608468+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.094878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447749+0000","last_change":"2024-03-06T07:21:43.322229+0000","last_active":"2024-03-06T07:21:44.447749+0000","last_peered":"2024-03-06T07:21:44.447749+0000","last_clean":"2024-03-06T07:21:44.447749+0000","last_became_active":"2024-03-06T07:21:43.321934+0000","last_became_peered":"2024-03-06T07:21:43.321934+0000","last_unstale":"2024-03-06T07:21:44.447749+0000","last_undegraded":"2024-03-06T07:21:44.447749+0000","last_fullsized":"2024-03-06T07:21:44.447749+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609427+0000","last_change":"2024-03-06T07:21:42.321714+0000","last_active":"2024-03-06T07:21:44.609427+0000","last_peered":"2024-03-06T07:21:44.609427+0000","last_clean":"2024-03-06T07:21:44.609427+0000","last_became_active":"2024-03-06T07:21:42.321444+0000","last_became_peered":"2024-03-06T07:21:42.321444+0000","last_unstale":"2024-03-06T07:21:44.609427+0000","last_undegraded":"2024-03-06T07:21:44.609427+0000","last_fullsized":"2024-03-06T07:21:44.609427+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609148+0000","last_change":"2024-03-06T07:21:43.316403+0000","last_active":"2024-03-06T07:21:44.609148+0000","last_peered":"2024-03-06T07:21:44.609148+0000","last_clean":"2024-03-06T07:21:44.609148+0000","last_became_active":"2024-03-06T07:21:43.315416+0000","last_became_peered":"2024-03-06T07:21:43.315416+0000","last_unstale":"2024-03-06T07:21:44.609148+0000","last_undegraded":"2024-03-06T07:21:44.609148+0000","last_fullsized":"2024-03-06T07:21:44.609148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:32.706894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335773+0000","last_change":"2024-03-06T07:21:42.311790+0000","last_active":"2024-03-06T07:21:44.335773+0000","last_peered":"2024-03-06T07:21:44.335773+0000","last_clean":"2024-03-06T07:21:44.335773+0000","last_became_active":"2024-03-06T07:21:42.311500+0000","last_became_peered":"2024-03-06T07:21:42.311500+0000","last_unstale":"2024-03-06T07:21:44.335773+0000","last_undegraded":"2024-03-06T07:21:44.335773+0000","last_fullsized":"2024-03-06T07:21:44.335773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317822+0000","last_change":"2024-03-06T07:21:43.319244+0000","last_active":"2024-03-06T07:21:44.317822+0000","last_peered":"2024-03-06T07:21:44.317822+0000","last_clean":"2024-03-06T07:21:44.317822+0000","last_became_active":"2024-03-06T07:21:43.318994+0000","last_became_peered":"2024-03-06T07:21:43.318994+0000","last_unstale":"2024-03-06T07:21:44.317822+0000","last_undegraded":"2024-03-06T07:21:44.317822+0000","last_fullsized":"2024-03-06T07:21:44.317822+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.333013+0000","last_change":"2024-03-06T07:21:42.315718+0000","last_active":"2024-03-06T07:21:44.333013+0000","last_peered":"2024-03-06T07:21:44.333013+0000","last_clean":"2024-03-06T07:21:44.333013+0000","last_became_active":"2024-03-06T07:21:42.315402+0000","last_became_peered":"2024-03-06T07:21:42.315402+0000","last_unstale":"2024-03-06T07:21:44.333013+0000","last_undegraded":"2024-03-06T07:21:44.333013+0000","last_fullsized":"2024-03-06T07:21:44.333013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332903+0000","last_change":"2024-03-06T07:21:43.323135+0000","last_active":"2024-03-06T07:21:44.332903+0000","last_peered":"2024-03-06T07:21:44.332903+0000","last_clean":"2024-03-06T07:21:44.332903+0000","last_became_active":"2024-03-06T07:21:43.322814+0000","last_became_peered":"2024-03-06T07:21:43.322814+0000","last_unstale":"2024-03-06T07:21:44.332903+0000","last_undegraded":"2024-03-06T07:21:44.332903+0000","last_fullsized":"2024-03-06T07:21:44.332903+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:24.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317877+0000","last_change":"2024-03-06T07:21:42.308451+0000","last_active":"2024-03-06T07:21:44.317877+0000","last_peered":"2024-03-06T07:21:44.317877+0000","last_clean":"2024-03-06T07:21:44.317877+0000","last_became_active":"2024-03-06T07:21:42.308167+0000","last_became_peered":"2024-03-06T07:21:42.308167+0000","last_unstale":"2024-03-06T07:21:44.317877+0000","last_undegraded":"2024-03-06T07:21:44.317877+0000","last_fullsized":"2024-03-06T07:21:44.317877+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:31.703072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447915+0000","last_change":"2024-03-06T07:21:43.324622+0000","last_active":"2024-03-06T07:21:44.447915+0000","last_peered":"2024-03-06T07:21:44.447915+0000","last_clean":"2024-03-06T07:21:44.447915+0000","last_became_active":"2024-03-06T07:21:43.324350+0000","last_became_peered":"2024-03-06T07:21:43.324350+0000","last_unstale":"2024-03-06T07:21:44.447915+0000","last_undegraded":"2024-03-06T07:21:44.447915+0000","last_fullsized":"2024-03-06T07:21:44.447915+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.317714+0000","last_change":"2024-03-06T07:21:42.309246+0000","last_active":"2024-03-06T07:21:44.317714+0000","last_peered":"2024-03-06T07:21:44.317714+0000","last_clean":"2024-03-06T07:21:44.317714+0000","last_became_active":"2024-03-06T07:21:42.308990+0000","last_became_peered":"2024-03-06T07:21:42.308990+0000","last_unstale":"2024-03-06T07:21:44.317714+0000","last_undegraded":"2024-03-06T07:21:44.317714+0000","last_fullsized":"2024-03-06T07:21:44.317714+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:51.765639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.332957+0000","last_change":"2024-03-06T07:21:43.329441+0000","last_active":"2024-03-06T07:21:44.332957+0000","last_peered":"2024-03-06T07:21:44.332957+0000","last_clean":"2024-03-06T07:21:44.332957+0000","last_became_active":"2024-03-06T07:21:43.327694+0000","last_became_peered":"2024-03-06T07:21:43.327694+0000","last_unstale":"2024-03-06T07:21:44.332957+0000","last_undegraded":"2024-03-06T07:21:44.332957+0000","last_fullsized":"2024-03-06T07:21:44.332957+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318424+0000","last_change":"2024-03-06T07:21:42.309614+0000","last_active":"2024-03-06T07:21:44.318424+0000","last_peered":"2024-03-06T07:21:44.318424+0000","last_clean":"2024-03-06T07:21:44.318424+0000","last_became_active":"2024-03-06T07:21:42.309319+0000","last_became_peered":"2024-03-06T07:21:42.309319+0000","last_unstale":"2024-03-06T07:21:44.318424+0000","last_undegraded":"2024-03-06T07:21:44.318424+0000","last_fullsized":"2024-03-06T07:21:44.318424+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:28.385954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.318213+0000","last_change":"2024-03-06T07:21:43.321711+0000","last_active":"2024-03-06T07:21:44.318213+0000","last_peered":"2024-03-06T07:21:44.318213+0000","last_clean":"2024-03-06T07:21:44.318213+0000","last_became_active":"2024-03-06T07:21:43.321434+0000","last_became_peered":"2024-03-06T07:21:43.321434+0000","last_unstale":"2024-03-06T07:21:44.318213+0000","last_undegraded":"2024-03-06T07:21:44.318213+0000","last_fullsized":"2024-03-06T07:21:44.318213+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:06.150276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.447853+0000","last_change":"2024-03-06T07:21:42.310672+0000","last_active":"2024-03-06T07:21:44.447853+0000","last_peered":"2024-03-06T07:21:44.447853+0000","last_clean":"2024-03-06T07:21:44.447853+0000","last_became_active":"2024-03-06T07:21:42.310366+0000","last_became_peered":"2024-03-06T07:21:42.310366+0000","last_unstale":"2024-03-06T07:21:44.447853+0000","last_undegraded":"2024-03-06T07:21:44.447853+0000","last_fullsized":"2024-03-06T07:21:44.447853+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:24.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.609137+0000","last_change":"2024-03-06T07:21:43.319233+0000","last_active":"2024-03-06T07:21:44.609137+0000","last_peered":"2024-03-06T07:21:44.609137+0000","last_clean":"2024-03-06T07:21:44.609137+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:44.609137+0000","last_undegraded":"2024-03-06T07:21:44.609137+0000","last_fullsized":"2024-03-06T07:21:44.609137+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.334573+0000","last_change":"2024-03-06T07:21:42.322036+0000","last_active":"2024-03-06T07:21:44.334573+0000","last_peered":"2024-03-06T07:21:44.334573+0000","last_clean":"2024-03-06T07:21:44.334573+0000","last_became_active":"2024-03-06T07:21:42.321385+0000","last_became_peered":"2024-03-06T07:21:42.321385+0000","last_unstale":"2024-03-06T07:21:44.334573+0000","last_undegraded":"2024-03-06T07:21:44.334573+0000","last_fullsized":"2024-03-06T07:21:44.334573+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.335159+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:44.335159+0000","last_peered":"2024-03-06T07:21:44.335159+0000","last_clean":"2024-03-06T07:21:44.335159+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:44.335159+0000","last_undegraded":"2024-03-06T07:21:44.335159+0000","last_fullsized":"2024-03-06T07:21:44.335159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":854168,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":602112,"data_stored":264328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":264258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":610304,"data_stored":265310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":593920,"data_stored":258596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":602112,"data_stored":263836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":655360,"data_stored":847408,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:47.842 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:21:47.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:21:47.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.989+0000 7f22681f6640 1 Processor -- start 2024-03-06T07:21:47.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.989+0000 7f22681f6640 1 -- start start 2024-03-06T07:21:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.989+0000 7f22681f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 0x7f2260155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.989+0000 7f22681f6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22600c5380 0x7f22600c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.989+0000 7f22681f6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600b0450 con 0x7f226006ca70 2024-03-06T07:21:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.989+0000 7f22681f6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226006c020 con 0x7f22600c5380 2024-03-06T07:21:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.989+0000 7f22681f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22600c77f0 con 0x7f22600b0070 2024-03-06T07:21:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2265791640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f226006ca70 0x7f22600c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38336/0 (socket says 172.21.15.26:38336) 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2265791640 1 -- 172.21.15.26:0/3650887721 learned_addr learned my addr 172.21.15.26:0/3650887721 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2265f92640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 0x7f2260155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2266793640 1 --2- 172.21.15.26:0/3650887721 >> v2:172.21.15.117:3300/0 conn(0x7f22600c5380 0x7f22600c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946527579 0 0) 0x7f2250001110 con 0x7f226006ca70 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f224c003520 con 0x7f226006ca70 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 839898837 0 0) 0x7f224c003520 con 0x7f226006ca70 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 msgr2=0x7f2260155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 0x7f2260155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 >> v2:172.21.15.117:3300/0 conn(0x7f22600c5380 msgr2=0x7f22600c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 --2- 172.21.15.26:0/3650887721 >> v2:172.21.15.117:3300/0 conn(0x7f22600c5380 0x7f22600c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600c7ad0 con 0x7f226006ca70 2024-03-06T07:21:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f2250001840 con 0x7f226006ca70 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2265f92640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 0x7f2260155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 >> v1:172.21.15.26:6789/0 conn(0x7f226006ca70 legacy=0x7f22600c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c004800 0x7f224c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 0x7f224c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226006c020 con 0x7f224c004800 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22600c77f0 con 0x7f224c0039b0 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c00b9c0 con 0x7f22600b0070 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2266793640 1 --2- 172.21.15.26:0/3650887721 >> v2:172.21.15.117:3300/0 conn(0x7f22600c5380 0x7f22600c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.990+0000 7f2265791640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c004800 0x7f224c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265f92640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2266793640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 0x7f224c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265791640 1 -- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 msgr2=0x7f224c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265791640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600b0070 0x7f224c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265791640 1 -- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c0039b0 msgr2=0x7f224c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265791640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265791640 1 -- 172.21.15.26:0/3650887721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c003520 con 0x7f224c004800 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265f92640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.991+0000 7f2265791640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c004800 0x7f224c004c80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f225000c3e0 tx=0x7f224c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a082dd6cdd2538b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225000c0e0 con 0x7f224c004800 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2250030070 con 0x7f224c004800 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f2264f90640 1 -- 172.21.15.26:0/3650887721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500058d0 con 0x7f224c004800 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f22681f6640 1 -- 172.21.15.26:0/3650887721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c004800 msgr2=0x7f224c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f22681f6640 1 --2- 172.21.15.26:0/3650887721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c004800 0x7f224c004c80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f225000c3e0 tx=0x7f224c005830 comp rx=0 tx=0).stop 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f2266793640 1 -- 172.21.15.26:0/3650887721 reap_dead start 2024-03-06T07:21:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f22681f6640 1 -- 172.21.15.26:0/3650887721 shutdown_connections 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f22681f6640 1 -- 172.21.15.26:0/3650887721 >> 172.21.15.26:0/3650887721 conn(0x7f22600ac590 msgr2=0x7f22600ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f22681f6640 1 -- 172.21.15.26:0/3650887721 shutdown_connections 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.992+0000 7f22681f6640 1 -- 172.21.15.26:0/3650887721 wait complete. 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.993+0000 7f22681f6640 1 Processor -- start 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.993+0000 7f22681f6640 1 -- start start 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f22681f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2260154280 0x7f226014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f22681f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226014ab10 0x7f226014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f22681f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600c1d10 0x7f22600c2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f22681f6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226006cef0 con 0x7f2260154280 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f22681f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226006c550 con 0x7f226014ab10 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f22681f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226006da80 con 0x7f22600c1d10 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2260154280 0x7f226014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2260154280 0x7f226014e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49950/0 (socket says 172.21.15.26:49950) 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 -- 172.21.15.26:0/2477435811 learned_addr learned my addr 172.21.15.26:0/2477435811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265791640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226014ab10 0x7f226014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2266793640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600c1d10 0x7f22600c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 -- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600c1d10 msgr2=0x7f22600c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600c1d10 0x7f22600c2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 -- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226014ab10 msgr2=0x7f226014af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226014ab10 0x7f226014af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265f92640 1 -- 172.21.15.26:0/2477435811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22601570a0 con 0x7f2260154280 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2266793640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600c1d10 0x7f22600c2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.994+0000 7f2265791640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226014ab10 0x7f226014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.995+0000 7f2265f92640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2260154280 0x7f226014e390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f225c011560 tx=0x7f225c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.995+0000 7f22567fc640 1 -- 172.21.15.26:0/2477435811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c015870 con 0x7f2260154280 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.995+0000 7f22567fc640 1 -- 172.21.15.26:0/2477435811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c015a10 con 0x7f2260154280 2024-03-06T07:21:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.995+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2260157380 con 0x7f2260154280 2024-03-06T07:21:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.995+0000 7f22567fc640 1 -- 172.21.15.26:0/2477435811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c015de0 con 0x7f2260154280 2024-03-06T07:21:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.995+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22600bddc0 con 0x7f2260154280 2024-03-06T07:21:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.996+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226004f330 con 0x7f2260154280 2024-03-06T07:21:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.996+0000 7f22567fc640 1 -- 172.21.15.26:0/2477435811 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f225c01d400 con 0x7f2260154280 2024-03-06T07:21:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.997+0000 7f22567fc640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f224406b4d0 0x7f224406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.997+0000 7f22567fc640 1 -- 172.21.15.26:0/2477435811 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f225c08ab20 con 0x7f2260154280 2024-03-06T07:21:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.997+0000 7f2265791640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f224406b4d0 0x7f224406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:47.997+0000 7f2265791640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f224406b4d0 0x7f224406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f225000c3b0 tx=0x7f2250007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.001+0000 7f22567fc640 1 -- 172.21.15.26:0/2477435811 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f225c019030 con 0x7f2260154280 2024-03-06T07:21:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.251+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f226006d860 con 0x7f2260154280 2024-03-06T07:21:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.252+0000 7f22567fc640 1 -- 172.21.15.26:0/2477435811 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f225c059810 con 0x7f2260154280 2024-03-06T07:21:48.254 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:21:48.254 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:21:48.254 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:21:48.254 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:21:48.254 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:21:48.255 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:21:48.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.255+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f224406b4d0 msgr2=0x7f224406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.255+0000 7f22681f6640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f224406b4d0 0x7f224406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f225000c3b0 tx=0x7f2250007000 comp rx=0 tx=0).stop 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.255+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2260154280 msgr2=0x7f226014e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2260154280 0x7f226014e390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f225c011560 tx=0x7f225c011a20 comp rx=0 tx=0).stop 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 shutdown_connections 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f224406b4d0 0x7f224406d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22600c1d10 0x7f22600c2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226014ab10 0x7f226014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 --2- 172.21.15.26:0/2477435811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2260154280 0x7f226014e390 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 >> 172.21.15.26:0/2477435811 conn(0x7f22600ac590 msgr2=0x7f22600ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 shutdown_connections 2024-03-06T07:21:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.256+0000 7f22681f6640 1 -- 172.21.15.26:0/2477435811 wait complete. 2024-03-06T07:21:48.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:48.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.374+0000 7fdb525f7640 1 Processor -- start 2024-03-06T07:21:48.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.374+0000 7fdb525f7640 1 -- start start 2024-03-06T07:21:48.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.374+0000 7fdb525f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb4c159270 0x7fdb4c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.374+0000 7fdb525f7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c0c73d0 con 0x7fdb4c0c88c0 2024-03-06T07:21:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.374+0000 7fdb525f7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c0be2e0 con 0x7fdb4c14f360 2024-03-06T07:21:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.374+0000 7fdb525f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4c159700 con 0x7fdb4c159270 2024-03-06T07:21:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdb4c0c88c0 0x7fdb4c158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38350/0 (socket says 172.21.15.26:38350) 2024-03-06T07:21:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4b7fe640 1 -- 172.21.15.26:0/582262764 learned_addr learned my addr 172.21.15.26:0/582262764 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228664742 0 0) 0x7fdb4c0c73d0 con 0x7fdb4c0c88c0 2024-03-06T07:21:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb50b94640 1 --2- 172.21.15.26:0/582262764 >> v2:172.21.15.117:3300/0 conn(0x7fdb4c159270 0x7fdb4c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb34003520 con 0x7fdb4c0c88c0 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496685525 0 0) 0x7fdb34003520 con 0x7fdb4c0c88c0 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 >> v2:172.21.15.117:3300/0 conn(0x7fdb4c159270 msgr2=0x7fdb4c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 --2- 172.21.15.26:0/582262764 >> v2:172.21.15.117:3300/0 conn(0x7fdb4c159270 0x7fdb4c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 >> v1:172.21.15.117:6789/0 conn(0x7fdb4c14f360 legacy=0x7fdb4c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c0c7670 con 0x7fdb4c0c88c0 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fdb40001670 con 0x7fdb4c0c88c0 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 >> v1:172.21.15.26:6789/0 conn(0x7fdb4c0c88c0 legacy=0x7fdb4c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb34004800 0x7fdb34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb340051c0 0x7fdb3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c159700 con 0x7fdb340039b0 2024-03-06T07:21:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3400c160 con 0x7fdb34004800 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3400c360 con 0x7fdb340051c0 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.375+0000 7fdb4bfff640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4b7fe640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb34004800 0x7fdb34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb50b94640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb340051c0 0x7fdb3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4bfff640 1 -- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb340051c0 msgr2=0x7fdb3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4bfff640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb340051c0 0x7fdb3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4bfff640 1 -- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb34004800 msgr2=0x7fdb34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4bfff640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb34004800 0x7fdb34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4bfff640 1 -- 172.21.15.26:0/582262764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34003520 con 0x7fdb340039b0 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4b7fe640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb34004800 0x7fdb34004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb50b94640 1 -- 172.21.15.26:0/582262764 reap_dead start 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.376+0000 7fdb4bfff640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb34003dd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c001750 tx=0x7fdb34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43202f1adfa7ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c001370 con 0x7fdb340039b0 2024-03-06T07:21:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c0052d0 con 0x7fdb340039b0 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb4affd640 1 -- 172.21.15.26:0/582262764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c003830 con 0x7fdb340039b0 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb525f7640 1 -- 172.21.15.26:0/582262764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 msgr2=0x7fdb34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb525f7640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb34003dd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c001750 tx=0x7fdb34005830 comp rx=0 tx=0).stop 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb525f7640 1 -- 172.21.15.26:0/582262764 shutdown_connections 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb525f7640 1 --2- 172.21.15.26:0/582262764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb34003dd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb525f7640 1 -- 172.21.15.26:0/582262764 >> 172.21.15.26:0/582262764 conn(0x7fdb4c0ac570 msgr2=0x7fdb4c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb525f7640 1 -- 172.21.15.26:0/582262764 shutdown_connections 2024-03-06T07:21:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.377+0000 7fdb525f7640 1 -- 172.21.15.26:0/582262764 wait complete. 2024-03-06T07:21:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.379+0000 7fdb525f7640 1 Processor -- start 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.379+0000 7fdb525f7640 1 -- start start 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb525f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb4c1eee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb525f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c0c0f70 0x7fdb4c154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb525f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c14bb70 0x7fdb4c14bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb525f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c0c7c30 con 0x7fdb340039b0 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb525f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c155c10 con 0x7fdb4c14bb70 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb525f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c155d90 con 0x7fdb4c0c0f70 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb4c1eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb4c1eee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49960/0 (socket says 172.21.15.26:49960) 2024-03-06T07:21:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 -- 172.21.15.26:0/3791409298 learned_addr learned my addr 172.21.15.26:0/3791409298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb50b94640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c14bb70 0x7fdb4c14bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4b7fe640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c0c0f70 0x7fdb4c154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 -- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c0c0f70 msgr2=0x7fdb4c154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c0c0f70 0x7fdb4c154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 -- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c14bb70 msgr2=0x7fdb4c14bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c14bb70 0x7fdb4c14bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 -- 172.21.15.26:0/3791409298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c1ef340 con 0x7fdb340039b0 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4b7fe640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c0c0f70 0x7fdb4c154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb50b94640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c14bb70 0x7fdb4c14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb4bfff640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb4c1eee00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c000c00 tx=0x7fdb3c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb48ff9640 1 -- 172.21.15.26:0/3791409298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c041020 con 0x7fdb340039b0 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.380+0000 7fdb48ff9640 1 -- 172.21.15.26:0/3791409298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c005480 con 0x7fdb340039b0 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.381+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c0ad770 con 0x7fdb340039b0 2024-03-06T07:21:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.381+0000 7fdb48ff9640 1 -- 172.21.15.26:0/3791409298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c007a30 con 0x7fdb340039b0 2024-03-06T07:21:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.381+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c1ec390 con 0x7fdb340039b0 2024-03-06T07:21:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.382+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c04f330 con 0x7fdb340039b0 2024-03-06T07:21:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.382+0000 7fdb48ff9640 1 -- 172.21.15.26:0/3791409298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb3c044020 con 0x7fdb340039b0 2024-03-06T07:21:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.383+0000 7fdb48ff9640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3806b500 0x7fdb3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.383+0000 7fdb48ff9640 1 -- 172.21.15.26:0/3791409298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb3c0abe10 con 0x7fdb340039b0 2024-03-06T07:21:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.383+0000 7fdb4b7fe640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3806b500 0x7fdb3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.384+0000 7fdb4b7fe640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3806b500 0x7fdb3806d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdb40001cd0 tx=0x7fdb40001e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.387+0000 7fdb48ff9640 1 -- 172.21.15.26:0/3791409298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb3c0af030 con 0x7fdb340039b0 2024-03-06T07:21:48.603 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-03-06T07:21:48.602+0000 7f11bce8a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:48.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.632+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb4c0c8d50 con 0x7fdb340039b0 2024-03-06T07:21:48.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.633+0000 7fdb48ff9640 1 -- 172.21.15.26:0/3791409298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4977 (secure 0 0 0) 0x7fdb3c04b030 con 0x7fdb340039b0 2024-03-06T07:21:48.634 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:48.634 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1383022235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1383022235},{"type":"v1","addr":"172.21.15.26:6834","nonce":1383022235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:21:45.677934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/440323369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":440323369},{"type":"v1","addr":"172.21.15.26:6835","nonce":440323369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":3}]} 2024-03-06T07:21:48.635 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3806b500 msgr2=0x7fdb3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3806b500 0x7fdb3806d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdb40001cd0 tx=0x7fdb40001e20 comp rx=0 tx=0).stop 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 msgr2=0x7fdb4c1eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb4c1eee00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c000c00 tx=0x7fdb3c001120 comp rx=0 tx=0).stop 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 shutdown_connections 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb3806b500 0x7fdb3806d9c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c14bb70 0x7fdb4c14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c0c0f70 0x7fdb4c154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 --2- 172.21.15.26:0/3791409298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb340039b0 0x7fdb4c1eee00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 >> 172.21.15.26:0/3791409298 conn(0x7fdb4c0ac570 msgr2=0x7fdb4c0ab800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 shutdown_connections 2024-03-06T07:21:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:48.635+0000 7fdb525f7640 1 -- 172.21.15.26:0/3791409298 wait complete. 2024-03-06T07:21:49.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:42.669148+0000', 'modified': '2024-03-06T07:21:45.677934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5697': {'gid': 5697, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/440323369', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 440323369}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 440323369}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5697, 'qdb_cluster': [5697]} 2024-03-06T07:21:49.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:49.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.154+0000 7fc84504e640 1 Processor -- start 2024-03-06T07:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.154+0000 7fc84504e640 1 -- start start 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.154+0000 7fc84504e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc840152b40 0x7fc8400c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.154+0000 7fc84504e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8400a7ff0 0x7fc840155900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.154+0000 7fc84504e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400733f0 con 0x7fc8400724d0 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.154+0000 7fc84504e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc840072d70 con 0x7fc840152b40 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.154+0000 7fc84504e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400c82c0 con 0x7fc8400a7ff0 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83ed9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc840152b40 0x7fc8400c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83f59e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8400a7ff0 0x7fc840155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83e59c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc8400724d0 0x7fc8401551f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38792/0 (socket says 172.21.15.26:38792) 2024-03-06T07:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83ed9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc840152b40 0x7fc8400c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34220/0 (socket says 172.21.15.26:34220) 2024-03-06T07:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83e59c640 1 -- 172.21.15.26:0/3070039980 learned_addr learned my addr 172.21.15.26:0/3070039980 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83ed9d640 1 -- 172.21.15.26:0/3070039980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8400a7ff0 msgr2=0x7fc840155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/3070039980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8400a7ff0 0x7fc840155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83ed9d640 1 -- 172.21.15.26:0/3070039980 >> v1:172.21.15.117:6789/0 conn(0x7fc8400724d0 legacy=0x7fc8401551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83dd9b640 1 -- 172.21.15.26:0/3070039980 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395390440 0 0) 0x7fc8400733f0 con 0x7fc8400724d0 2024-03-06T07:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.155+0000 7fc83ed9d640 1 -- 172.21.15.26:0/3070039980 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84006fd30 con 0x7fc840152b40 2024-03-06T07:21:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.156+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/3070039980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc840152b40 0x7fc8400c8810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097a0 tx=0x7fc83802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ade781abbc48fda5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc83dd9b640 1 -- 172.21.15.26:0/3070039980 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8380315a0 con 0x7fc840152b40 2024-03-06T07:21:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc83dd9b640 1 -- 172.21.15.26:0/3070039980 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838037070 con 0x7fc840152b40 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc83dd9b640 1 -- 172.21.15.26:0/3070039980 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838031960 con 0x7fc840152b40 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 -- 172.21.15.26:0/3070039980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc840152b40 msgr2=0x7fc8400c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 --2- 172.21.15.26:0/3070039980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc840152b40 0x7fc8400c8810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097a0 tx=0x7fc83802b840 comp rx=0 tx=0).stop 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 -- 172.21.15.26:0/3070039980 shutdown_connections 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 --2- 172.21.15.26:0/3070039980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8400a7ff0 0x7fc840155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 --2- 172.21.15.26:0/3070039980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc840152b40 0x7fc8400c8810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 -- 172.21.15.26:0/3070039980 >> 172.21.15.26:0/3070039980 conn(0x7fc84006b970 msgr2=0x7fc8400aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 -- 172.21.15.26:0/3070039980 shutdown_connections 2024-03-06T07:21:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.157+0000 7fc84504e640 1 -- 172.21.15.26:0/3070039980 wait complete. 2024-03-06T07:21:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.159+0000 7fc84504e640 1 Processor -- start 2024-03-06T07:21:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.159+0000 7fc84504e640 1 -- start start 2024-03-06T07:21:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.160+0000 7fc84504e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8400724d0 0x7fc840153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.160+0000 7fc84504e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8400a7ff0 0x7fc8401ee220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.160+0000 7fc84504e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc840152b40 0x7fc84006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.160+0000 7fc84504e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400c8d50 con 0x7fc8400a7ff0 2024-03-06T07:21:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.160+0000 7fc84504e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc840070490 con 0x7fc8400724d0 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.160+0000 7fc84504e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400c8ed0 con 0x7fc840152b40 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8400a7ff0 0x7fc8401ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8400a7ff0 0x7fc8401ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49968/0 (socket says 172.21.15.26:49968) 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 -- 172.21.15.26:0/3615246087 learned_addr learned my addr 172.21.15.26:0/3615246087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8400724d0 0x7fc840153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83f59e640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc840152b40 0x7fc84006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 -- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc840152b40 msgr2=0x7fc84006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc840152b40 0x7fc84006c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 -- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8400724d0 msgr2=0x7fc840153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8400724d0 0x7fc840153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 -- 172.21.15.26:0/3615246087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8401530d0 con 0x7fc8400a7ff0 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc83e59c640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8400a7ff0 0x7fc8401ee220 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc83400c7c0 tx=0x7fc83400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc8277fe640 1 -- 172.21.15.26:0/3615246087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834018070 con 0x7fc8400a7ff0 2024-03-06T07:21:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc8277fe640 1 -- 172.21.15.26:0/3615246087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83400a640 con 0x7fc8400a7ff0 2024-03-06T07:21:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8401eeb60 con 0x7fc8400a7ff0 2024-03-06T07:21:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc8277fe640 1 -- 172.21.15.26:0/3615246087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83400aa30 con 0x7fc8400a7ff0 2024-03-06T07:21:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.161+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84006e960 con 0x7fc8400a7ff0 2024-03-06T07:21:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.163+0000 7fc8277fe640 1 -- 172.21.15.26:0/3615246087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc83401b070 con 0x7fc8400a7ff0 2024-03-06T07:21:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.163+0000 7fc8277fe640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82c06b4d0 0x7fc82c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.163+0000 7fc8277fe640 1 -- 172.21.15.26:0/3615246087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc834089910 con 0x7fc8400a7ff0 2024-03-06T07:21:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.163+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82c06b4d0 0x7fc82c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.164+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82c06b4d0 0x7fc82c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc838036000 tx=0x7fc83803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.164+0000 7fc847c3b740 1 Processor -- start 2024-03-06T07:21:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.164+0000 7fc847c3b740 1 -- start start 2024-03-06T07:21:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x56106076fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x56106076ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5610607704b0 0x56106076c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc847c3b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610606d66b0 con 0x561060771f50 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106067fa60 con 0x561060772330 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610606d2750 con 0x5610607704b0 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x56106076fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x56106076fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49980/0 (socket says 172.21.15.26:49980) 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 -- 172.21.15.26:0/128594904 learned_addr learned my addr 172.21.15.26:0/128594904 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83e59c640 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x56106076ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83f59e640 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5610607704b0 0x56106076c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 -- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5610607704b0 msgr2=0x56106076c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5610607704b0 0x56106076c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 -- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 msgr2=0x56106076ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x56106076ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83ed9d640 1 -- 172.21.15.26:0/128594904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56106069df30 con 0x561060771f50 2024-03-06T07:21:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.165+0000 7fc83e59c640 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x56106076ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x56106076fa30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc838034040 tx=0x7fc838030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37db88d97ca764b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc824ff9640 1 -- 172.21.15.26:0/128594904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838054070 con 0x561060771f50 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc824ff9640 1 -- 172.21.15.26:0/128594904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838037040 con 0x561060771f50 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc824ff9640 1 -- 172.21.15.26:0/128594904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838032bc0 con 0x561060771f50 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc847c3b740 1 -- 172.21.15.26:0/128594904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 msgr2=0x56106076fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc847c3b740 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x56106076fa30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc838034040 tx=0x7fc838030000 comp rx=0 tx=0).stop 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc847c3b740 1 -- 172.21.15.26:0/128594904 shutdown_connections 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc847c3b740 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5610607704b0 0x56106076c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc847c3b740 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x56106076ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc847c3b740 1 --2- 172.21.15.26:0/128594904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x56106076fa30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.166+0000 7fc847c3b740 1 -- 172.21.15.26:0/128594904 >> 172.21.15.26:0/128594904 conn(0x56106076e7d0 msgr2=0x5610607933c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.167+0000 7fc847c3b740 1 -- 172.21.15.26:0/128594904 shutdown_connections 2024-03-06T07:21:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.167+0000 7fc847c3b740 1 -- 172.21.15.26:0/128594904 wait complete. 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.167+0000 7fc847c3b740 1 -- start start 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.167+0000 7fc847c3b740 1 Processor -- start 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x561060798a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x561060794700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x5610607aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc847c3b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561060794230 con 0x561060771f50 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561060692ea0 con 0x561060772330 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561060793c70 con 0x561060797a30 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x561060798a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc847c3b740 10 client.0 _ll_register_callbacks cb 0x561060692870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x561060798a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49986/0 (socket says 172.21.15.26:49986) 2024-03-06T07:21:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 -- 172.21.15.26:0/244714461 learned_addr learned my addr 172.21.15.26:0/244714461 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83f59e640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x5610607aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83e59c640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x561060794700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 -- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 msgr2=0x5610607aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x5610607aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 -- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 msgr2=0x561060794700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x561060794700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 -- 172.21.15.26:0/244714461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561060693270 con 0x561060771f50 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83e59c640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x561060794700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x561060798a80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc838009560 tx=0x7fc838004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3929e9e8fb476e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.168+0000 7fc81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc81dffb640 1 -- 172.21.15.26:0/244714461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83805d060 con 0x561060771f50 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc81dffb640 1 -- 172.21.15.26:0/244714461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8380456f0 con 0x561060771f50 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc838037040 con 0x561060771f50 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc81dffb640 1 -- 172.21.15.26:0/244714461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc838063050 con 0x561060771f50 2024-03-06T07:21:49.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 10 client.5841 fetch_fsmap learned FSMap version 29 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5610603eb2d0 con 0x561060771f50 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc81dffb640 1 -- 172.21.15.26:0/244714461 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838037040 con 0x561060771f50 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc81dffb640 1 -- 172.21.15.26:0/244714461 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fc838066020 con 0x561060771f50 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 10 client.5841 fetch_fsmap finished waiting for FSMap version 29 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 10 client.5841 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 20 client.5841 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x5610607c24e0 0x5610607c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 4 client.5841 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 --> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610606e3bf0 con 0x5610607c24e0 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.169+0000 7fc83e59c640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x5610607c24e0 0x5610607c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc81dffb640 10 client.5841 ms_handle_connect on v2:172.21.15.26:6833/440323369 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc83e59c640 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x5610607c24e0 0x5610607c28c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f5ed47db837c8d server_cookie=ccadaa17884093bb in_seq=0 out_seq=0 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc81dffb640 1 -- 172.21.15.26:0/244714461 <== mds.0 v2:172.21.15.26:6833/440323369 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5610606e3bf0 con 0x5610607c24e0 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc81dffb640 10 client.5841 handle_command_reply: tid=0 2024-03-06T07:21:49.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 client.5841 shutdown 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 msgr2=0x561060798a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x561060798a80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc838009560 tx=0x7fc838004730 comp rx=0 tx=0).stop 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 shutdown_connections 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x5610607c24e0 0x5610607c28c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x5610607aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060772330 0x561060794700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc81dffb640 0 client.5841 ms_handle_reset on v2:172.21.15.26:6833/440323369 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 --2- 172.21.15.26:0/244714461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060771f50 0x561060798a80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 >> 172.21.15.26:0/244714461 conn(0x5610607704b0 msgr2=0x5610607aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 shutdown_connections 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.170+0000 7fc847c3b740 1 -- 172.21.15.26:0/244714461 wait complete. 2024-03-06T07:21:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.171+0000 7fc847c3b740 20 client.5841 trim_cache size 0 max 16384 2024-03-06T07:21:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.187+0000 7fc847c3b740 1 Processor -- start 2024-03-06T07:21:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.187+0000 7fc847c3b740 1 -- start start 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 0x56106076d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x561060796ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x561060797020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc847c3b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401f6d00 con 0x561060772330 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401eb4a0 con 0x561060771f50 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401f6700 con 0x561060797a30 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x561060796ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x561060796ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49998/0 (socket says 172.21.15.26:49998) 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 -- 172.21.15.26:0/701839985 learned_addr learned my addr 172.21.15.26:0/701839985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 -- 172.21.15.26:0/701839985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 msgr2=0x561060797020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 --2- 172.21.15.26:0/701839985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x561060797020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 -- 172.21.15.26:0/701839985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 msgr2=0x56106076d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 --2- 172.21.15.26:0/701839985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 0x56106076d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 -- 172.21.15.26:0/701839985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610604b56b0 con 0x561060772330 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.188+0000 7fc83e59c640 1 --2- 172.21.15.26:0/701839985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x561060796ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc8340019a0 tx=0x7fc834001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c574911eb9f7e958 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc81ffff640 1 -- 172.21.15.26:0/701839985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834086070 con 0x561060772330 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc81ffff640 1 -- 172.21.15.26:0/701839985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834056290 con 0x561060772330 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc81ffff640 1 -- 172.21.15.26:0/701839985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834060210 con 0x561060772330 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 -- 172.21.15.26:0/701839985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 msgr2=0x561060796ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 --2- 172.21.15.26:0/701839985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x561060796ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc8340019a0 tx=0x7fc834001250 comp rx=0 tx=0).stop 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 -- 172.21.15.26:0/701839985 shutdown_connections 2024-03-06T07:21:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 --2- 172.21.15.26:0/701839985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x561060797020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 --2- 172.21.15.26:0/701839985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x561060796ae0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 --2- 172.21.15.26:0/701839985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 0x56106076d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 -- 172.21.15.26:0/701839985 >> 172.21.15.26:0/701839985 conn(0x5610607704b0 msgr2=0x561060793360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 -- 172.21.15.26:0/701839985 shutdown_connections 2024-03-06T07:21:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.189+0000 7fc847c3b740 1 -- 172.21.15.26:0/701839985 wait complete. 2024-03-06T07:21:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.190+0000 7fc847c3b740 1 -- start start 2024-03-06T07:21:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.190+0000 7fc847c3b740 1 Processor -- start 2024-03-06T07:21:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 0x5610607d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x5610607c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc847c3b740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x56106081d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc847c3b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610607bf400 con 0x561060772330 2024-03-06T07:21:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561060794230 con 0x561060771f50 2024-03-06T07:21:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc847c3b740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610607aae00 con 0x561060797a30 2024-03-06T07:21:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc847c3b740 10 client.0 _ll_register_callbacks cb 0x7fc838032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x5610607c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x5610607c3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50004/0 (socket says 172.21.15.26:50004) 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 -- 172.21.15.26:0/895087886 learned_addr learned my addr 172.21.15.26:0/895087886 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 0x5610607d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83f59e640 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x56106081d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 -- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 msgr2=0x56106081d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x56106081d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 -- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 msgr2=0x5610607d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 0x5610607d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 -- 172.21.15.26:0/895087886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610607c8e80 con 0x561060772330 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc83e59c640 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x5610607c3d40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc834058150 tx=0x7fc834009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a219898decd15be server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.191+0000 7fc81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc81dffb640 1 -- 172.21.15.26:0/895087886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83407b060 con 0x561060772330 2024-03-06T07:21:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc81dffb640 1 -- 172.21.15.26:0/895087886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc834060210 con 0x561060772330 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc834056290 con 0x561060772330 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc81dffb640 1 -- 172.21.15.26:0/895087886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc834081050 con 0x561060772330 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 10 client.5847 fetch_fsmap learned FSMap version 29 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5610607d1ef0 con 0x561060772330 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc81dffb640 1 -- 172.21.15.26:0/895087886 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834056290 con 0x561060772330 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc81dffb640 1 -- 172.21.15.26:0/895087886 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fc8340811f0 con 0x561060772330 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 10 client.5847 fetch_fsmap finished waiting for FSMap version 29 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 10 client.5847 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 20 client.5847 populate_metadata read hostname 'smithi026' 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x56106081ae60 0x561060820d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 4 client.5847 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 --> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5610606ddf40 con 0x56106081ae60 2024-03-06T07:21:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x56106081ae60 0x561060820d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.192+0000 7fc81dffb640 10 client.5847 ms_handle_connect on v2:172.21.15.26:6833/440323369 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc83ed9d640 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x56106081ae60 0x561060820d40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b265c0e3a6fd00 server_cookie=9287cc67d8dcafe9 in_seq=0 out_seq=0 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc81dffb640 1 -- 172.21.15.26:0/895087886 <== mds.0 v2:172.21.15.26:6833/440323369 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5610606ddf40 con 0x56106081ae60 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc81dffb640 10 client.5847 handle_command_reply: tid=0 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 client.5847 shutdown 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 msgr2=0x5610607c3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x5610607c3d40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc834058150 tx=0x7fc834009040 comp rx=0 tx=0).stop 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 shutdown_connections 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:6833/440323369,v1:172.21.15.26:6835/440323369] conn(0x56106081ae60 0x561060820d40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561060797a30 0x56106081d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561060772330 0x5610607c3d40 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 --2- 172.21.15.26:0/895087886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561060771f50 0x5610607d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 >> 172.21.15.26:0/895087886 conn(0x5610607704b0 msgr2=0x56106076d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc81dffb640 0 client.5847 ms_handle_reset on v2:172.21.15.26:6833/440323369 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 shutdown_connections 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 1 -- 172.21.15.26:0/895087886 wait complete. 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc847c3b740 20 client.5847 trim_cache size 0 max 16384 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5697, 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.8802952839999998, 2024-03-06T07:21:49.201 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 29, 2024-03-06T07:21:49.202 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 50, 2024-03-06T07:21:49.202 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 50, 2024-03-06T07:21:49.202 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.959835314999999 2024-03-06T07:21:49.202 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82c06b4d0 msgr2=0x7fc82c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc84504e640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82c06b4d0 0x7fc82c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc838036000 tx=0x7fc83803d040 comp rx=0 tx=0).stop 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8400a7ff0 msgr2=0x7fc8401ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.193+0000 7fc84504e640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8400a7ff0 0x7fc8401ee220 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc83400c7c0 tx=0x7fc83400cc80 comp rx=0 tx=0).stop 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 shutdown_connections 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82c06b4d0 0x7fc82c06d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc840152b40 0x7fc84006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8400a7ff0 0x7fc8401ee220 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 --2- 172.21.15.26:0/3615246087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8400724d0 0x7fc840153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 >> 172.21.15.26:0/3615246087 conn(0x7fc84006b970 msgr2=0x7fc8400c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 shutdown_connections 2024-03-06T07:21:49.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.194+0000 7fc84504e640 1 -- 172.21.15.26:0/3615246087 wait complete. 2024-03-06T07:21:49.206 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:49.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.306+0000 7fb77ae4e640 1 Processor -- start 2024-03-06T07:21:49.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.306+0000 7fb77ae4e640 1 -- start start 2024-03-06T07:21:49.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb77ae4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb7740c88c0 0x7fb774155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb77ae4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7740c73d0 con 0x7fb77414f360 2024-03-06T07:21:49.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb77ae4e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7740be2e0 con 0x7fb774158b60 2024-03-06T07:21:49.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb77ae4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb774159700 con 0x7fb7740c88c0 2024-03-06T07:21:49.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb773fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb7740c88c0 0x7fb774155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb778bea640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb77414f360 0x7fb7740c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38352/0 (socket says 172.21.15.26:38352) 2024-03-06T07:21:49.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb778bea640 1 -- 172.21.15.26:0/754112287 learned_addr learned my addr 172.21.15.26:0/754112287 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:49.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb773fff640 1 -- 172.21.15.26:0/754112287 >> v1:172.21.15.117:6789/0 conn(0x7fb774158b60 legacy=0x7fb774158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb773fff640 1 -- 172.21.15.26:0/754112287 >> v1:172.21.15.26:6789/0 conn(0x7fb77414f360 legacy=0x7fb7740c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.307+0000 7fb773fff640 1 -- 172.21.15.26:0/754112287 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740c7670 con 0x7fb7740c88c0 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.308+0000 7fb773fff640 1 --2- 172.21.15.26:0/754112287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7740c88c0 0x7fb774155c10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb7600097a0 tx=0x7fb76002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c302ed1cde6b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.308+0000 7fb7737fe640 1 -- 172.21.15.26:0/754112287 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7600315a0 con 0x7fb7740c88c0 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.308+0000 7fb7737fe640 1 -- 172.21.15.26:0/754112287 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb760037070 con 0x7fb7740c88c0 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.308+0000 7fb7737fe640 1 -- 172.21.15.26:0/754112287 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760031960 con 0x7fb7740c88c0 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.309+0000 7fb77ae4e640 1 -- 172.21.15.26:0/754112287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7740c88c0 msgr2=0x7fb774155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.309+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/754112287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7740c88c0 0x7fb774155c10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb7600097a0 tx=0x7fb76002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.309+0000 7fb77ae4e640 1 -- 172.21.15.26:0/754112287 shutdown_connections 2024-03-06T07:21:49.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.309+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/754112287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7740c88c0 0x7fb774155c10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.309+0000 7fb77ae4e640 1 -- 172.21.15.26:0/754112287 >> 172.21.15.26:0/754112287 conn(0x7fb7740ac570 msgr2=0x7fb7740ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.309+0000 7fb77ae4e640 1 -- 172.21.15.26:0/754112287 shutdown_connections 2024-03-06T07:21:49.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.309+0000 7fb77ae4e640 1 -- 172.21.15.26:0/754112287 wait complete. 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.310+0000 7fb77ae4e640 1 Processor -- start 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.310+0000 7fb77ae4e640 1 -- start start 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb77ae4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7740c88c0 0x7fb7740be760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb77ae4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb77414f360 0x7fb7741ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb77ae4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb774158b60 0x7fb7741eea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb77ae4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740c7c30 con 0x7fb774158b60 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb77ae4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740c5a50 con 0x7fb77414f360 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb77ae4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740c5bd0 con 0x7fb7740c88c0 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb774158b60 0x7fb7741eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb774158b60 0x7fb7741eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50020/0 (socket says 172.21.15.26:50020) 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 -- 172.21.15.26:0/597451050 learned_addr learned my addr 172.21.15.26:0/597451050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb773fff640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb77414f360 0x7fb7741ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb778bea640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7740c88c0 0x7fb7740be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 -- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7740c88c0 msgr2=0x7fb7740be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7740c88c0 0x7fb7740be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 -- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb77414f360 msgr2=0x7fb7741ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb77414f360 0x7fb7741ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.311+0000 7fb7793eb640 1 -- 172.21.15.26:0/597451050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740c11c0 con 0x7fb774158b60 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.312+0000 7fb7793eb640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb774158b60 0x7fb7741eea10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00b970 tx=0x7fb76c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.312+0000 7fb7717fa640 1 -- 172.21.15.26:0/597451050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c019070 con 0x7fb774158b60 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.312+0000 7fb7717fa640 1 -- 172.21.15.26:0/597451050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c00cae0 con 0x7fb774158b60 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.312+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7740bf0a0 con 0x7fb774158b60 2024-03-06T07:21:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.312+0000 7fb7717fa640 1 -- 172.21.15.26:0/597451050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c014440 con 0x7fb774158b60 2024-03-06T07:21:49.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.312+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7740c0de0 con 0x7fb774158b60 2024-03-06T07:21:49.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.313+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77404f330 con 0x7fb774158b60 2024-03-06T07:21:49.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.313+0000 7fb7717fa640 1 -- 172.21.15.26:0/597451050 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb76c01c070 con 0x7fb774158b60 2024-03-06T07:21:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.314+0000 7fb7717fa640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb75806b4d0 0x7fb75806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.314+0000 7fb7717fa640 1 -- 172.21.15.26:0/597451050 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb76c089e40 con 0x7fb774158b60 2024-03-06T07:21:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.314+0000 7fb778bea640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb75806b4d0 0x7fb75806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.314+0000 7fb778bea640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb75806b4d0 0x7fb75806d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb768007fd0 tx=0x7fb768007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.322+0000 7fb7717fa640 1 -- 172.21.15.26:0/597451050 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb76c058570 con 0x7fb774158b60 2024-03-06T07:21:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.541+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7740c8d50 con 0x7fb774158b60 2024-03-06T07:21:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.542+0000 7fb7717fa640 1 -- 172.21.15.26:0/597451050 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4977 (secure 0 0 0) 0x7fb76c05c610 con 0x7fb774158b60 2024-03-06T07:21:49.544 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:49.544 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1383022235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1383022235},{"type":"v1","addr":"172.21.15.26:6834","nonce":1383022235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:21:45.677934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/440323369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":440323369},{"type":"v1","addr":"172.21.15.26:6835","nonce":440323369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":3}]} 2024-03-06T07:21:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-03-06T07:21:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb75806b4d0 msgr2=0x7fb75806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb75806b4d0 0x7fb75806d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb768007fd0 tx=0x7fb768007e80 comp rx=0 tx=0).stop 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb774158b60 msgr2=0x7fb7741eea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb774158b60 0x7fb7741eea10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00b970 tx=0x7fb76c00be30 comp rx=0 tx=0).stop 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 shutdown_connections 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb75806b4d0 0x7fb75806d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb774158b60 0x7fb7741eea10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb77414f360 0x7fb7741ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 --2- 172.21.15.26:0/597451050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7740c88c0 0x7fb7740be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 >> 172.21.15.26:0/597451050 conn(0x7fb7740ac570 msgr2=0x7fb77414d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 shutdown_connections 2024-03-06T07:21:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.544+0000 7fb77ae4e640 1 -- 172.21.15.26:0/597451050 wait complete. 2024-03-06T07:21:49.557 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 3, "epoch": 29, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:42.669148+0000", "damaged": [], "data_pools": [ 8 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5697": { "addr": "172.21.15.26:6835/440323369", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 440323369, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 440323369, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5697, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 7, "modified": "2024-03-06T07:21:45.677934+0000", "qdb_cluster": [ 5697 ], "qdb_leader": 5697, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5697 } } } ], "standbys": [ { "addr": "172.21.15.26:6834/1383022235", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 1383022235, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 1383022235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5234, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/357874491", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 357874491, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 357874491, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5240, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6836/3928098986", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3928098986, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3928098986, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5617, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:49.557 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:49.557 INFO:tasks.cephfs.test_failover:current = [{'gid': 5234, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/1383022235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1383022235}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1383022235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 25}, {'gid': 5617, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3928098986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3928098986}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3928098986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 25}, {'gid': 5697, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/440323369', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 440323369}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 440323369}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5240, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/357874491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 357874491}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 357874491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 25}] 2024-03-06T07:21:49.557 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.a mds_join_fs cephfs2 2024-03-06T07:21:49.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.660+0000 7fa492948640 1 Processor -- start 2024-03-06T07:21:49.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.660+0000 7fa492948640 1 -- start start 2024-03-06T07:21:49.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa492948640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa492948640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa48c06c0b0 0x7fa48c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa492948640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48c155260 con 0x7fa48c1527f0 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa492948640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48c0c8f70 con 0x7fa48c06c0b0 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa492948640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c155540 con 0x7fa48c0c7370 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa490ee5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa48c06c0b0 0x7fa48c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa48bfff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa48c1527f0 0x7fa48c0c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38814/0 (socket says 172.21.15.26:38814) 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.661+0000 7fa48bfff640 1 -- 172.21.15.26:0/3775615896 learned_addr learned my addr 172.21.15.26:0/3775615896 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.662+0000 7fa48affd640 1 -- 172.21.15.26:0/3775615896 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806763913 0 0) 0x7fa48c155260 con 0x7fa48c1527f0 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.662+0000 7fa48affd640 1 -- 172.21.15.26:0/3775615896 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa478003520 con 0x7fa48c1527f0 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.662+0000 7fa490ee5640 1 -- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 msgr2=0x7fa48c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.662+0000 7fa490ee5640 1 --2- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.662+0000 7fa490ee5640 1 -- 172.21.15.26:0/3775615896 >> v1:172.21.15.117:6789/0 conn(0x7fa48c1527f0 legacy=0x7fa48c0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.662+0000 7fa490ee5640 1 -- 172.21.15.26:0/3775615896 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c155820 con 0x7fa48c06c0b0 2024-03-06T07:21:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.662+0000 7fa48b7fe640 1 --2- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa490ee5640 1 --2- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 0x7fa48c06c4e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0c90f0 tx=0x7fa48402bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=491e45ad2068fc9b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa48affd640 1 -- 172.21.15.26:0/3775615896 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484039070 con 0x7fa48c06c0b0 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa48affd640 1 -- 172.21.15.26:0/3775615896 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48402cb20 con 0x7fa48c06c0b0 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa48affd640 1 -- 172.21.15.26:0/3775615896 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484034440 con 0x7fa48c06c0b0 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 -- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 msgr2=0x7fa48c06c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 --2- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 0x7fa48c06c4e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0c90f0 tx=0x7fa48402bd70 comp rx=0 tx=0).stop 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 -- 172.21.15.26:0/3775615896 shutdown_connections 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 --2- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 0x7fa48c06c4e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 --2- 172.21.15.26:0/3775615896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 -- 172.21.15.26:0/3775615896 >> 172.21.15.26:0/3775615896 conn(0x7fa48c0ac4f0 msgr2=0x7fa48c0ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 -- 172.21.15.26:0/3775615896 shutdown_connections 2024-03-06T07:21:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.663+0000 7fa492948640 1 -- 172.21.15.26:0/3775615896 wait complete. 2024-03-06T07:21:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.664+0000 7fa492948640 1 Processor -- start 2024-03-06T07:21:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa492948640 1 -- start start 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa492948640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 0x7fa48c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa492948640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa492948640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c1527f0 0x7fa48c149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa492948640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c06ca20 con 0x7fa48c1527f0 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa492948640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0be1a0 con 0x7fa48c06c0b0 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa492948640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c06cba0 con 0x7fa48c0c7370 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa490ee5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c1527f0 0x7fa48c149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa490ee5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c1527f0 0x7fa48c149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50036/0 (socket says 172.21.15.26:50036) 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.665+0000 7fa490ee5640 1 -- 172.21.15.26:0/676356630 learned_addr learned my addr 172.21.15.26:0/676356630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa48b7fe640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa48bfff640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 0x7fa48c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa490ee5640 1 -- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 msgr2=0x7fa48c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa490ee5640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa490ee5640 1 -- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 msgr2=0x7fa48c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa490ee5640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 0x7fa48c0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa490ee5640 1 -- 172.21.15.26:0/676356630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c154aa0 con 0x7fa48c1527f0 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa490ee5640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c1527f0 0x7fa48c149ad0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e80 tx=0x7fa484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa488ff9640 1 -- 172.21.15.26:0/676356630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484039050 con 0x7fa48c1527f0 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa488ff9640 1 -- 172.21.15.26:0/676356630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa484045070 con 0x7fa48c1527f0 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c0afcf0 con 0x7fa48c1527f0 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa488ff9640 1 -- 172.21.15.26:0/676356630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484009450 con 0x7fa48c1527f0 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.666+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c0b0210 con 0x7fa48c1527f0 2024-03-06T07:21:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.667+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c04f330 con 0x7fa48c1527f0 2024-03-06T07:21:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.668+0000 7fa488ff9640 1 -- 172.21.15.26:0/676356630 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa48403e850 con 0x7fa48c1527f0 2024-03-06T07:21:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.668+0000 7fa488ff9640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa46c06b4d0 0x7fa46c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.668+0000 7fa488ff9640 1 -- 172.21.15.26:0/676356630 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa4840ad150 con 0x7fa48c1527f0 2024-03-06T07:21:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.668+0000 7fa48bfff640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa46c06b4d0 0x7fa46c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.669+0000 7fa48bfff640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa46c06b4d0 0x7fa46c06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005880 tx=0x7fa47c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.673+0000 7fa488ff9640 1 -- 172.21.15.26:0/676356630 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa48c0afcf0 con 0x7fa48c1527f0 2024-03-06T07:21:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.855+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fa48c06d860 con 0x7fa48c1527f0 2024-03-06T07:21:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.858+0000 7fa488ff9640 1 -- 172.21.15.26:0/676356630 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v16)=0 v16) v1 ==== 117+0+0 (secure 0 0 0) 0x7fa48c154aa0 con 0x7fa48c1527f0 2024-03-06T07:21:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.860+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa46c06b4d0 msgr2=0x7fa46c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.860+0000 7fa492948640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa46c06b4d0 0x7fa46c06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005880 tx=0x7fa47c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.860+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c1527f0 msgr2=0x7fa48c149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.860+0000 7fa492948640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c1527f0 0x7fa48c149ad0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e80 tx=0x7fa484004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 shutdown_connections 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa46c06b4d0 0x7fa46c06d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c1527f0 0x7fa48c149ad0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48c0c7370 0x7fa48c0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 --2- 172.21.15.26:0/676356630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c06c0b0 0x7fa48c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 >> 172.21.15.26:0/676356630 conn(0x7fa48c0ac4f0 msgr2=0x7fa48c14fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 shutdown_connections 2024-03-06T07:21:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.861+0000 7fa492948640 1 -- 172.21.15.26:0/676356630 wait complete. 2024-03-06T07:21:49.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs2 2024-03-06T07:21:49.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.976+0000 7fe38b240640 1 Processor -- start 2024-03-06T07:21:49.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.976+0000 7fe38b240640 1 -- start start 2024-03-06T07:21:49.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.977+0000 7fe38b240640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 0x7fe3840c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.977+0000 7fe38b240640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe384073580 con 0x7fe3840c7f60 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.977+0000 7fe38b240640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3840701f0 con 0x7fe3841505f0 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.977+0000 7fe38b240640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe384070b20 con 0x7fe384156ef0 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.977+0000 7fe383fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe3840c7f60 0x7fe3840c6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38366/0 (socket says 172.21.15.26:38366) 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.977+0000 7fe383fff640 1 -- 172.21.15.26:0/2867555813 learned_addr learned my addr 172.21.15.26:0/2867555813 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3897dd640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 0x7fe3840c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200294501 0 0) 0x7fe384073580 con 0x7fe3840c7f60 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe374003520 con 0x7fe3840c7f60 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216351891 0 0) 0x7fe374003520 con 0x7fe3840c7f60 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 msgr2=0x7fe3840c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 0x7fe3840c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 >> v1:172.21.15.117:6789/0 conn(0x7fe3841505f0 legacy=0x7fe384072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840704d0 con 0x7fe3840c7f60 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe370001670 con 0x7fe3840c7f60 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 >> v1:172.21.15.26:6789/0 conn(0x7fe3840c7f60 legacy=0x7fe3840c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3740039e0 0x7fe374003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 0x7fe374004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe374004d40 0x7fe3740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe384070b20 con 0x7fe3740039e0 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37400fe80 con 0x7fe374004d40 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe374010080 con 0x7fe384156ef0 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe388fdc640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3740039e0 0x7fe374003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe383fff640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 0x7fe374004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe3897dd640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe374004d40 0x7fe3740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe388fdc640 1 -- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 msgr2=0x7fe374004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe388fdc640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe384156ef0 0x7fe374004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.978+0000 7fe388fdc640 1 -- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe374004d40 msgr2=0x7fe3740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.979+0000 7fe388fdc640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe374004d40 0x7fe3740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.979+0000 7fe388fdc640 1 -- 172.21.15.26:0/2867555813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe374003520 con 0x7fe3740039e0 2024-03-06T07:21:49.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.979+0000 7fe388fdc640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3740039e0 0x7fe374003e00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe378001720 tx=0x7fe374005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a236ea7900434a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.979+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378001370 con 0x7fe3740039e0 2024-03-06T07:21:49.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.979+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3780052d0 con 0x7fe3740039e0 2024-03-06T07:21:49.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.979+0000 7fe3837fe640 1 -- 172.21.15.26:0/2867555813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378003830 con 0x7fe3740039e0 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.980+0000 7fe38b240640 1 -- 172.21.15.26:0/2867555813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3740039e0 msgr2=0x7fe374003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.980+0000 7fe38b240640 1 --2- 172.21.15.26:0/2867555813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3740039e0 0x7fe374003e00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe378001720 tx=0x7fe374005830 comp rx=0 tx=0).stop 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.980+0000 7fe3897dd640 1 -- 172.21.15.26:0/2867555813 reap_dead start 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.980+0000 7fe38b240640 1 -- 172.21.15.26:0/2867555813 shutdown_connections 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.980+0000 7fe38b240640 1 -- 172.21.15.26:0/2867555813 >> 172.21.15.26:0/2867555813 conn(0x7fe3840aa5c0 msgr2=0x7fe3840aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.980+0000 7fe38b240640 1 -- 172.21.15.26:0/2867555813 shutdown_connections 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.980+0000 7fe38b240640 1 -- 172.21.15.26:0/2867555813 wait complete. 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.981+0000 7fe38b240640 1 Processor -- start 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.981+0000 7fe38b240640 1 -- start start 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe38b240640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740039e0 0x7fe3841e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe38b240640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38414ab50 0x7fe3840ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe38b240640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38414f030 0x7fe38414f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe38b240640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe384072c20 con 0x7fe38414ab50 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe38b240640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38406fec0 con 0x7fe38414f030 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe38b240640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe384072da0 con 0x7fe3740039e0 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38414ab50 0x7fe3840ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38414ab50 0x7fe3840ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50056/0 (socket says 172.21.15.26:50056) 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 -- 172.21.15.26:0/3088193655 learned_addr learned my addr 172.21.15.26:0/3088193655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe388fdc640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740039e0 0x7fe3841e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe3897dd640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38414f030 0x7fe38414f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 -- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740039e0 msgr2=0x7fe3841e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740039e0 0x7fe3841e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 -- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38414f030 msgr2=0x7fe38414f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38414f030 0x7fe38414f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.982+0000 7fe383fff640 1 -- 172.21.15.26:0/3088193655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38414d880 con 0x7fe38414ab50 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.983+0000 7fe383fff640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38414ab50 0x7fe3840ae110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe3700011e0 tx=0x7fe370007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.983+0000 7fe3817fa640 1 -- 172.21.15.26:0/3088193655 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000f960 con 0x7fe38414ab50 2024-03-06T07:21:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.983+0000 7fe3817fa640 1 -- 172.21.15.26:0/3088193655 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe370018070 con 0x7fe38414ab50 2024-03-06T07:21:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.983+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38414db60 con 0x7fe38414ab50 2024-03-06T07:21:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.983+0000 7fe3817fa640 1 -- 172.21.15.26:0/3088193655 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000fd60 con 0x7fe38414ab50 2024-03-06T07:21:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.983+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38414e120 con 0x7fe38414ab50 2024-03-06T07:21:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.984+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3840a9c40 con 0x7fe38414ab50 2024-03-06T07:21:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.984+0000 7fe3817fa640 1 -- 172.21.15.26:0/3088193655 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe37001d460 con 0x7fe38414ab50 2024-03-06T07:21:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.985+0000 7fe3817fa640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe36806b500 0x7fe36806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.985+0000 7fe3817fa640 1 -- 172.21.15.26:0/3088193655 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe37008a3d0 con 0x7fe38414ab50 2024-03-06T07:21:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.985+0000 7fe388fdc640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe36806b500 0x7fe36806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.986+0000 7fe388fdc640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe36806b500 0x7fe36806d9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe378033000 tx=0x7fe378009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:49.989+0000 7fe3817fa640 1 -- 172.21.15.26:0/3088193655 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe3840a9c40 con 0x7fe38414ab50 2024-03-06T07:21:50.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.170+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fe38404f330 con 0x7fe38414ab50 2024-03-06T07:21:50.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.173+0000 7fe3817fa640 1 -- 172.21.15.26:0/3088193655 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v17)=0 v17) v1 ==== 117+0+0 (secure 0 0 0) 0x7fe3700589a0 con 0x7fe38414ab50 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe36806b500 msgr2=0x7fe36806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe36806b500 0x7fe36806d9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe378033000 tx=0x7fe378009030 comp rx=0 tx=0).stop 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38414ab50 msgr2=0x7fe3840ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38414ab50 0x7fe3840ae110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe3700011e0 tx=0x7fe370007720 comp rx=0 tx=0).stop 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 shutdown_connections 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe36806b500 0x7fe36806d9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38414f030 0x7fe38414f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38414ab50 0x7fe3840ae110 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 --2- 172.21.15.26:0/3088193655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740039e0 0x7fe3841e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 >> 172.21.15.26:0/3088193655 conn(0x7fe3840aa5c0 msgr2=0x7fe38414fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 shutdown_connections 2024-03-06T07:21:50.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.175+0000 7fe38b240640 1 -- 172.21.15.26:0/3088193655 wait complete. 2024-03-06T07:21:50.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.b mds_join_fs cephfs2 2024-03-06T07:21:50.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.291+0000 7f3118c15640 1 Processor -- start 2024-03-06T07:21:50.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.291+0000 7f3118c15640 1 -- start start 2024-03-06T07:21:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3118c15640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31141505f0 0x7f3114156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3118c15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31140c6710 0x7f31140c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3118c15640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3114073580 con 0x7f31140c7f60 2024-03-06T07:21:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3118c15640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3114070b20 con 0x7f31141505f0 2024-03-06T07:21:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3118c15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140701f0 con 0x7f31140c6710 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f311259c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31141505f0 0x7f3114156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3112d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31140c6710 0x7f31140c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3111d9b640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f31140c7f60 0x7f3114072510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38832/0 (socket says 172.21.15.26:38832) 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f311259c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31141505f0 0x7f3114156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34308/0 (socket says 172.21.15.26:34308) 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3112d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31140c6710 0x7f31140c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:49544/0 (socket says 172.21.15.26:49544) 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3111d9b640 1 -- 172.21.15.26:0/406971197 learned_addr learned my addr 172.21.15.26:0/406971197 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.292+0000 7f3112d9d640 1 -- 172.21.15.26:0/406971197 >> v2:172.21.15.117:3300/0 conn(0x7f31141505f0 msgr2=0x7f3114156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.293+0000 7f3112d9d640 1 --2- 172.21.15.26:0/406971197 >> v2:172.21.15.117:3300/0 conn(0x7f31141505f0 0x7f3114156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.293+0000 7f3112d9d640 1 -- 172.21.15.26:0/406971197 >> v1:172.21.15.117:6789/0 conn(0x7f31140c7f60 legacy=0x7f3114072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.293+0000 7f3112d9d640 1 -- 172.21.15.26:0/406971197 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31140704d0 con 0x7f31140c6710 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.293+0000 7f311159a640 1 -- 172.21.15.26:0/406971197 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936885112 0 0) 0x7f3114073580 con 0x7f31140c7f60 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.293+0000 7f311259c640 1 --2- 172.21.15.26:0/406971197 >> v2:172.21.15.117:3300/0 conn(0x7f31141505f0 0x7f3114156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.293+0000 7f3112d9d640 1 --2- 172.21.15.26:0/406971197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31140c6710 0x7f31140c6ba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f31040097a0 tx=0x7f310402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd7c7fb3fb8011d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f311159a640 1 -- 172.21.15.26:0/406971197 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104035020 con 0x7f31140c6710 2024-03-06T07:21:50.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f311159a640 1 -- 172.21.15.26:0/406971197 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104031830 con 0x7f31140c6710 2024-03-06T07:21:50.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f311159a640 1 -- 172.21.15.26:0/406971197 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104031c00 con 0x7f31140c6710 2024-03-06T07:21:50.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 -- 172.21.15.26:0/406971197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31140c6710 msgr2=0x7f31140c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 --2- 172.21.15.26:0/406971197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31140c6710 0x7f31140c6ba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f31040097a0 tx=0x7f310402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 -- 172.21.15.26:0/406971197 shutdown_connections 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 --2- 172.21.15.26:0/406971197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31140c6710 0x7f31140c6ba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 --2- 172.21.15.26:0/406971197 >> v2:172.21.15.117:3300/0 conn(0x7f31141505f0 0x7f3114156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 -- 172.21.15.26:0/406971197 >> 172.21.15.26:0/406971197 conn(0x7f31140aa5c0 msgr2=0x7f31140aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 -- 172.21.15.26:0/406971197 shutdown_connections 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.294+0000 7f3118c15640 1 -- 172.21.15.26:0/406971197 wait complete. 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.295+0000 7f3118c15640 1 Processor -- start 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.295+0000 7f3118c15640 1 -- start start 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3118c15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31140c6710 0x7f3114151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3118c15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31140c7f60 0x7f31141ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3118c15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31141505f0 0x7f31140c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3118c15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3114072c20 con 0x7f31140c6710 2024-03-06T07:21:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3118c15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311406fec0 con 0x7f31140c7f60 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3118c15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3114072da0 con 0x7f31141505f0 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f311259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31140c6710 0x7f3114151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f311259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31140c6710 0x7f3114151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50070/0 (socket says 172.21.15.26:50070) 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f311259c640 1 -- 172.21.15.26:0/1931986691 learned_addr learned my addr 172.21.15.26:0/1931986691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3112d9d640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31141505f0 0x7f31140c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.296+0000 7f3111d9b640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31140c7f60 0x7f31141ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f311259c640 1 -- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31141505f0 msgr2=0x7f31140c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f311259c640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31141505f0 0x7f31140c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f311259c640 1 -- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31140c7f60 msgr2=0x7f31141ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f311259c640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31140c7f60 0x7f31141ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f311259c640 1 -- 172.21.15.26:0/1931986691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31141ec7d0 con 0x7f31140c6710 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f311259c640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31140c6710 0x7f3114151790 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f310c00efc0 tx=0x7f310c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f310affd640 1 -- 172.21.15.26:0/1931986691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c018020 con 0x7f31140c6710 2024-03-06T07:21:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f310affd640 1 -- 172.21.15.26:0/1931986691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c014680 con 0x7f31140c6710 2024-03-06T07:21:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31141541a0 con 0x7f31140c6710 2024-03-06T07:21:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f310affd640 1 -- 172.21.15.26:0/1931986691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c014a50 con 0x7f31140c6710 2024-03-06T07:21:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.297+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3114154460 con 0x7f31140c6710 2024-03-06T07:21:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.298+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311404f330 con 0x7f31140c6710 2024-03-06T07:21:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.298+0000 7f310affd640 1 -- 172.21.15.26:0/1931986691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f310c005d10 con 0x7f31140c6710 2024-03-06T07:21:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.299+0000 7f310affd640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30f406b500 0x7f30f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.299+0000 7f310affd640 1 -- 172.21.15.26:0/1931986691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f310c08afc0 con 0x7f31140c6710 2024-03-06T07:21:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.299+0000 7f3111d9b640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30f406b500 0x7f30f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.299+0000 7f3111d9b640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30f406b500 0x7f30f406d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31140c22c0 tx=0x7f30fc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.307+0000 7f310affd640 1 -- 172.21.15.26:0/1931986691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f310c022030 con 0x7f31140c6710 2024-03-06T07:21:50.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.491+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f31140c2e50 con 0x7f31140c6710 2024-03-06T07:21:50.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.494+0000 7f310affd640 1 -- 172.21.15.26:0/1931986691 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v18)=0 v18) v1 ==== 117+0+0 (secure 0 0 0) 0x7f310c010040 con 0x7f31140c6710 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30f406b500 msgr2=0x7f30f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30f406b500 0x7f30f406d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31140c22c0 tx=0x7f30fc007de0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31140c6710 msgr2=0x7f3114151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31140c6710 0x7f3114151790 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f310c00efc0 tx=0x7f310c00c490 comp rx=0 tx=0).stop 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 shutdown_connections 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f30f406b500 0x7f30f406d9c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31141505f0 0x7f31140c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31140c7f60 0x7f31141ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 --2- 172.21.15.26:0/1931986691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31140c6710 0x7f3114151790 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 >> 172.21.15.26:0/1931986691 conn(0x7f31140aa5c0 msgr2=0x7f31140a8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:50.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 shutdown_connections 2024-03-06T07:21:50.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.496+0000 7f3118c15640 1 -- 172.21.15.26:0/1931986691 wait complete. 2024-03-06T07:21:50.509 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:50.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.610+0000 7fb60a393640 1 Processor -- start 2024-03-06T07:21:50.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.610+0000 7fb60a393640 1 -- start start 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb60a393640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6040c5480 0x7fb604155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb60a393640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb6040c5860 0x7fb60406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb60a393640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6040c5cc0 con 0x7fb6041586f0 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb60a393640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60406df40 con 0x7fb6040c5860 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb60a393640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6041591c0 con 0x7fb6040c5480 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb6037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb6040c5860 0x7fb60406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb608930640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb6041586f0 0x7fb6040724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38382/0 (socket says 172.21.15.26:38382) 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb608930640 1 -- 172.21.15.26:0/300144053 learned_addr learned my addr 172.21.15.26:0/300144053 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb603fff640 1 --2- 172.21.15.26:0/300144053 >> v2:172.21.15.117:3300/0 conn(0x7fb6040c5480 0x7fb604155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb6037fe640 1 -- 172.21.15.26:0/300144053 >> v2:172.21.15.117:3300/0 conn(0x7fb6040c5480 msgr2=0x7fb604155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb6037fe640 1 --2- 172.21.15.26:0/300144053 >> v2:172.21.15.117:3300/0 conn(0x7fb6040c5480 0x7fb604155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb6037fe640 1 -- 172.21.15.26:0/300144053 >> v1:172.21.15.26:6789/0 conn(0x7fb6041586f0 legacy=0x7fb6040724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.611+0000 7fb6037fe640 1 -- 172.21.15.26:0/300144053 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6041594a0 con 0x7fb6040c5860 2024-03-06T07:21:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.612+0000 7fb6037fe640 1 --2- 172.21.15.26:0/300144053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6040c5860 0x7fb60406f6a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb6040c1cf0 tx=0x7fb5f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f976ba51ddada3ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.612+0000 7fb602ffd640 1 -- 172.21.15.26:0/300144053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80315a0 con 0x7fb6040c5860 2024-03-06T07:21:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.612+0000 7fb602ffd640 1 -- 172.21.15.26:0/300144053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f8037070 con 0x7fb6040c5860 2024-03-06T07:21:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.612+0000 7fb602ffd640 1 -- 172.21.15.26:0/300144053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8031960 con 0x7fb6040c5860 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 -- 172.21.15.26:0/300144053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6040c5860 msgr2=0x7fb60406f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 --2- 172.21.15.26:0/300144053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6040c5860 0x7fb60406f6a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb6040c1cf0 tx=0x7fb5f802b840 comp rx=0 tx=0).stop 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 -- 172.21.15.26:0/300144053 shutdown_connections 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 --2- 172.21.15.26:0/300144053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6040c5860 0x7fb60406f6a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 --2- 172.21.15.26:0/300144053 >> v2:172.21.15.117:3300/0 conn(0x7fb6040c5480 0x7fb604155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 -- 172.21.15.26:0/300144053 >> 172.21.15.26:0/300144053 conn(0x7fb60406b1a0 msgr2=0x7fb60406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 -- 172.21.15.26:0/300144053 shutdown_connections 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.613+0000 7fb60a393640 1 -- 172.21.15.26:0/300144053 wait complete. 2024-03-06T07:21:50.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.614+0000 7fb60a393640 1 Processor -- start 2024-03-06T07:21:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.615+0000 7fb60a393640 1 -- start start 2024-03-06T07:21:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb60a393640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6040c5480 0x7fb60414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb60a393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6040c5860 0x7fb604154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb60a393640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6041586f0 0x7fb6040ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb60a393640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604152a20 con 0x7fb6041586f0 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb60a393640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604155680 con 0x7fb6040c5480 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb60a393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb604155050 con 0x7fb6040c5860 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6041586f0 0x7fb6040ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6041586f0 0x7fb6040ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50092/0 (socket says 172.21.15.26:50092) 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 -- 172.21.15.26:0/771040782 learned_addr learned my addr 172.21.15.26:0/771040782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb6037fe640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6040c5860 0x7fb604154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb603fff640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6040c5480 0x7fb60414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 -- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6040c5860 msgr2=0x7fb604154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6040c5860 0x7fb604154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 -- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6040c5480 msgr2=0x7fb60414f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6040c5480 0x7fb60414f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 -- 172.21.15.26:0/771040782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040ac400 con 0x7fb6041586f0 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb603fff640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6040c5480 0x7fb60414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.616+0000 7fb608930640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6041586f0 0x7fb6040ac940 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00ca70 tx=0x7fb5fc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.617+0000 7fb600ff9640 1 -- 172.21.15.26:0/771040782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc017070 con 0x7fb6041586f0 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.617+0000 7fb600ff9640 1 -- 172.21.15.26:0/771040782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc00abd0 con 0x7fb6041586f0 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.617+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb604150430 con 0x7fb6041586f0 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.617+0000 7fb600ff9640 1 -- 172.21.15.26:0/771040782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc012400 con 0x7fb6041586f0 2024-03-06T07:21:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.617+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6041509b0 con 0x7fb6041586f0 2024-03-06T07:21:50.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.618+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60404f330 con 0x7fb6041586f0 2024-03-06T07:21:50.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.618+0000 7fb600ff9640 1 -- 172.21.15.26:0/771040782 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5fc01a070 con 0x7fb6041586f0 2024-03-06T07:21:50.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.618+0000 7fb600ff9640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f006b500 0x7fb5f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.619+0000 7fb600ff9640 1 -- 172.21.15.26:0/771040782 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb5fc089eb0 con 0x7fb6041586f0 2024-03-06T07:21:50.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.619+0000 7fb603fff640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f006b500 0x7fb5f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.619+0000 7fb603fff640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f006b500 0x7fb5f006d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb6041549b0 tx=0x7fb5f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.627+0000 7fb600ff9640 1 -- 172.21.15.26:0/771040782 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb5fc0585b0 con 0x7fb6041586f0 2024-03-06T07:21:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.811+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb60406e0c0 con 0x7fb6041586f0 2024-03-06T07:21:50.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.813+0000 7fb600ff9640 1 -- 172.21.15.26:0/771040782 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16994 (secure 0 0 0) 0x7fb5fc05c650 con 0x7fb6041586f0 2024-03-06T07:21:50.815 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:50.815 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":52,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:50.335595+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f006b500 msgr2=0x7fb5f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f006b500 0x7fb5f006d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb6041549b0 tx=0x7fb5f4008040 comp rx=0 tx=0).stop 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6041586f0 msgr2=0x7fb6040ac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6041586f0 0x7fb6040ac940 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00ca70 tx=0x7fb5fc00cf30 comp rx=0 tx=0).stop 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 shutdown_connections 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f006b500 0x7fb5f006d9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6041586f0 0x7fb6040ac940 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6040c5860 0x7fb604154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 --2- 172.21.15.26:0/771040782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6040c5480 0x7fb60414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 >> 172.21.15.26:0/771040782 conn(0x7fb60406b1a0 msgr2=0x7fb60406fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 shutdown_connections 2024-03-06T07:21:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.815+0000 7fb60a393640 1 -- 172.21.15.26:0/771040782 wait complete. 2024-03-06T07:21:50.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:21:40.938361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:21:41.634747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:50.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:50.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.931+0000 7fbab926f640 1 Processor -- start 2024-03-06T07:21:50.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.931+0000 7fbab926f640 1 -- start start 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab926f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbab406d800 0x7fbab4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab926f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbab40a5eb0 0x7fbab40c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab926f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4155f70 con 0x7fbab4152920 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab926f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab4156250 con 0x7fbab406d800 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab926f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab40c49a0 con 0x7fbab40a5eb0 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbab406d800 0x7fbab4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbab406d800 0x7fbab4152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50104/0 (socket says 172.21.15.26:50104) 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab27fc640 1 -- 172.21.15.26:0/1734874619 learned_addr learned my addr 172.21.15.26:0/1734874619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab37fe640 1 --2- 172.21.15.26:0/1734874619 >> v2:172.21.15.117:3300/0 conn(0x7fbab40a5eb0 0x7fbab40c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab27fc640 1 -- 172.21.15.26:0/1734874619 >> v2:172.21.15.117:3300/0 conn(0x7fbab40a5eb0 msgr2=0x7fbab40c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab27fc640 1 --2- 172.21.15.26:0/1734874619 >> v2:172.21.15.117:3300/0 conn(0x7fbab40a5eb0 0x7fbab40c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab27fc640 1 -- 172.21.15.26:0/1734874619 >> v1:172.21.15.117:6789/0 conn(0x7fbab4152920 legacy=0x7fbab40bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab27fc640 1 -- 172.21.15.26:0/1734874619 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40c4c80 con 0x7fbab406d800 2024-03-06T07:21:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.932+0000 7fbab37fe640 1 --2- 172.21.15.26:0/1734874619 >> v2:172.21.15.117:3300/0 conn(0x7fbab40a5eb0 0x7fbab40c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.933+0000 7fbab27fc640 1 --2- 172.21.15.26:0/1734874619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab406d800 0x7fbab4152d00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbab4068270 tx=0x7fba9c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c3271d536e1f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.933+0000 7fbab1ffb640 1 -- 172.21.15.26:0/1734874619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c035020 con 0x7fbab406d800 2024-03-06T07:21:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.933+0000 7fbab1ffb640 1 -- 172.21.15.26:0/1734874619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c031870 con 0x7fbab406d800 2024-03-06T07:21:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.933+0000 7fbab1ffb640 1 -- 172.21.15.26:0/1734874619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c031c40 con 0x7fbab406d800 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 -- 172.21.15.26:0/1734874619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab406d800 msgr2=0x7fbab4152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 --2- 172.21.15.26:0/1734874619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab406d800 0x7fbab4152d00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbab4068270 tx=0x7fba9c02b910 comp rx=0 tx=0).stop 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 -- 172.21.15.26:0/1734874619 shutdown_connections 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 --2- 172.21.15.26:0/1734874619 >> v2:172.21.15.117:3300/0 conn(0x7fbab40a5eb0 0x7fbab40c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 --2- 172.21.15.26:0/1734874619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab406d800 0x7fbab4152d00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 -- 172.21.15.26:0/1734874619 >> 172.21.15.26:0/1734874619 conn(0x7fbab406cb60 msgr2=0x7fbab40a86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 -- 172.21.15.26:0/1734874619 shutdown_connections 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.934+0000 7fbab926f640 1 -- 172.21.15.26:0/1734874619 wait complete. 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.935+0000 7fbab926f640 1 Processor -- start 2024-03-06T07:21:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.935+0000 7fbab926f640 1 -- start start 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab926f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab406d800 0x7fbab4155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab926f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab40a5eb0 0x7fbab40c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab926f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab4152920 0x7fbab4150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab926f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40735b0 con 0x7fbab4152920 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab926f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4073430 con 0x7fbab40a5eb0 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab926f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab406e3f0 con 0x7fbab406d800 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab4152920 0x7fbab4150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab4152920 0x7fbab4150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50106/0 (socket says 172.21.15.26:50106) 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 -- 172.21.15.26:0/2032099084 learned_addr learned my addr 172.21.15.26:0/2032099084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab27fc640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab40a5eb0 0x7fbab40c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab2ffd640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab406d800 0x7fbab4155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 -- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab406d800 msgr2=0x7fbab4155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab406d800 0x7fbab4155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 -- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab40a5eb0 msgr2=0x7fbab40c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab40a5eb0 0x7fbab40c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 -- 172.21.15.26:0/2032099084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4070db0 con 0x7fbab4152920 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab37fe640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab4152920 0x7fbab4150c00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00cef0 tx=0x7fbaac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbaab7fe640 1 -- 172.21.15.26:0/2032099084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac01a070 con 0x7fbab4152920 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbaab7fe640 1 -- 172.21.15.26:0/2032099084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac00d860 con 0x7fbab4152920 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.936+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab4154800 con 0x7fbab4152920 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.937+0000 7fbaab7fe640 1 -- 172.21.15.26:0/2032099084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac00dc50 con 0x7fbab4152920 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.937+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab414b670 con 0x7fbab4152920 2024-03-06T07:21:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.937+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab414a6d0 con 0x7fbab4152920 2024-03-06T07:21:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.938+0000 7fbaab7fe640 1 -- 172.21.15.26:0/2032099084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaac01d070 con 0x7fbab4152920 2024-03-06T07:21:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.938+0000 7fbaab7fe640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba9406b500 0x7fba9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.938+0000 7fbaab7fe640 1 -- 172.21.15.26:0/2032099084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbaac08ac40 con 0x7fbab4152920 2024-03-06T07:21:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.939+0000 7fbab2ffd640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba9406b500 0x7fba9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.939+0000 7fbab2ffd640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba9406b500 0x7fba9406d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004ea0 tx=0x7fbaa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:50.943+0000 7fbaab7fe640 1 -- 172.21.15.26:0/2032099084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbaac0599c0 con 0x7fbab4152920 2024-03-06T07:21:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.151+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbab404f330 con 0x7fbab4152920 2024-03-06T07:21:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.152+0000 7fbaab7fe640 1 -- 172.21.15.26:0/2032099084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbaac0592c0 con 0x7fbab4152920 2024-03-06T07:21:51.153 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:21:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba9406b500 msgr2=0x7fba9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba9406b500 0x7fba9406d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004ea0 tx=0x7fbaa4009040 comp rx=0 tx=0).stop 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab4152920 msgr2=0x7fbab4150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab4152920 0x7fbab4150c00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00cef0 tx=0x7fbaac00a710 comp rx=0 tx=0).stop 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 shutdown_connections 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba9406b500 0x7fba9406d9c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab4152920 0x7fbab4150c00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab40a5eb0 0x7fbab40c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 --2- 172.21.15.26:0/2032099084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab406d800 0x7fbab4155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 >> 172.21.15.26:0/2032099084 conn(0x7fbab406cb60 msgr2=0x7fbab4071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 shutdown_connections 2024-03-06T07:21:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.154+0000 7fbab926f640 1 -- 172.21.15.26:0/2032099084 wait complete. 2024-03-06T07:21:51.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:21:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.280+0000 7f001f088640 1 Processor -- start 2024-03-06T07:21:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.280+0000 7f001f088640 1 -- start start 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f001f088640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0018155d30 0x7f00180be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f001f088640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00180c81b0 con 0x7f00181528e0 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f001f088640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00180c8490 con 0x7f00180724b0 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f001f088640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00180c6710 con 0x7f0018155d30 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f000ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0018155d30 0x7f00180be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f001ce24640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f00181528e0 0x7f00180c7160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38392/0 (socket says 172.21.15.26:38392) 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f000ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0018155d30 0x7f00180be8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50120/0 (socket says 172.21.15.26:50120) 2024-03-06T07:21:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.281+0000 7f001ce24640 1 -- 172.21.15.26:0/3434571889 learned_addr learned my addr 172.21.15.26:0/3434571889 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.282+0000 7f000ffff640 1 -- 172.21.15.26:0/3434571889 >> v1:172.21.15.117:6789/0 conn(0x7f00180724b0 legacy=0x7f0018070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.282+0000 7f000ffff640 1 -- 172.21.15.26:0/3434571889 >> v1:172.21.15.26:6789/0 conn(0x7f00181528e0 legacy=0x7f00180c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.282+0000 7f000ffff640 1 -- 172.21.15.26:0/3434571889 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180c6960 con 0x7f0018155d30 2024-03-06T07:21:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.282+0000 7f000f7fe640 1 -- 172.21.15.26:0/3434571889 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756190509 0 0) 0x7f00180c81b0 con 0x7f00181528e0 2024-03-06T07:21:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.282+0000 7f000ffff640 1 --2- 172.21.15.26:0/3434571889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0018155d30 0x7f00180be8c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f00080097a0 tx=0x7f000802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61895ea37d623db server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.282+0000 7f000f7fe640 1 -- 172.21.15.26:0/3434571889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008034070 con 0x7f0018155d30 2024-03-06T07:21:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f000f7fe640 1 -- 172.21.15.26:0/3434571889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008030610 con 0x7f0018155d30 2024-03-06T07:21:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f000f7fe640 1 -- 172.21.15.26:0/3434571889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080309e0 con 0x7f0018155d30 2024-03-06T07:21:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f001f088640 1 -- 172.21.15.26:0/3434571889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0018155d30 msgr2=0x7f00180be8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f001f088640 1 --2- 172.21.15.26:0/3434571889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0018155d30 0x7f00180be8c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f00080097a0 tx=0x7f000802b960 comp rx=0 tx=0).stop 2024-03-06T07:21:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f001f088640 1 -- 172.21.15.26:0/3434571889 shutdown_connections 2024-03-06T07:21:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f001f088640 1 --2- 172.21.15.26:0/3434571889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0018155d30 0x7f00180be8c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f001f088640 1 -- 172.21.15.26:0/3434571889 >> 172.21.15.26:0/3434571889 conn(0x7f00180c5310 msgr2=0x7f00180c56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f001f088640 1 -- 172.21.15.26:0/3434571889 shutdown_connections 2024-03-06T07:21:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.283+0000 7f001f088640 1 -- 172.21.15.26:0/3434571889 wait complete. 2024-03-06T07:21:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.284+0000 7f001f088640 1 Processor -- start 2024-03-06T07:21:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.284+0000 7f001f088640 1 -- start start 2024-03-06T07:21:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001f088640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00180724b0 0x7f00180c0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001f088640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00181528e0 0x7f00181deac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001f088640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0018155d30 0x7f00181dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001f088640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00181561c0 con 0x7f00180724b0 2024-03-06T07:21:51.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001f088640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018152cc0 con 0x7f0018155d30 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001f088640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180bee00 con 0x7f00181528e0 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00180724b0 0x7f00180c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00180724b0 0x7f00180c0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50130/0 (socket says 172.21.15.26:50130) 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 -- 172.21.15.26:0/3001589009 learned_addr learned my addr 172.21.15.26:0/3001589009 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f000ffff640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00181528e0 0x7f00181deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 -- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00181528e0 msgr2=0x7f00181deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001d625640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0018155d30 0x7f00181dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00181528e0 0x7f00181deac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 -- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0018155d30 msgr2=0x7f00181dfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0018155d30 0x7f00181dfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.285+0000 7f001ce24640 1 -- 172.21.15.26:0/3001589009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0018154b50 con 0x7f00180724b0 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.286+0000 7f001ce24640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00180724b0 0x7f00180c0500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f001400c3f0 tx=0x7f001400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.286+0000 7f000d7fa640 1 -- 172.21.15.26:0/3001589009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00140074e0 con 0x7f00180724b0 2024-03-06T07:21:51.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.286+0000 7f000d7fa640 1 -- 172.21.15.26:0/3001589009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00140047e0 con 0x7f00180724b0 2024-03-06T07:21:51.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.286+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00181df540 con 0x7f00180724b0 2024-03-06T07:21:51.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.286+0000 7f000d7fa640 1 -- 172.21.15.26:0/3001589009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014004c10 con 0x7f00180724b0 2024-03-06T07:21:51.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.286+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00180c0d30 con 0x7f00180724b0 2024-03-06T07:21:51.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.287+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00180c0aa0 con 0x7f00180724b0 2024-03-06T07:21:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.287+0000 7f000d7fa640 1 -- 172.21.15.26:0/3001589009 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0014011070 con 0x7f00180724b0 2024-03-06T07:21:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.288+0000 7f000d7fa640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f000406b4d0 0x7f000406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.288+0000 7f000d7fa640 1 -- 172.21.15.26:0/3001589009 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0014089ed0 con 0x7f00180724b0 2024-03-06T07:21:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.288+0000 7f000ffff640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f000406b4d0 0x7f000406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.288+0000 7f000ffff640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f000406b4d0 0x7f000406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0008036000 tx=0x7f000803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.292+0000 7f000d7fa640 1 -- 172.21.15.26:0/3001589009 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f001401a2b0 con 0x7f00180724b0 2024-03-06T07:21:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.477+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f001804f330 con 0x7f00180724b0 2024-03-06T07:21:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.478+0000 7f000d7fa640 1 -- 172.21.15.26:0/3001589009 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0014058c20 con 0x7f00180724b0 2024-03-06T07:21:51.479 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:21:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f000406b4d0 msgr2=0x7f000406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f000406b4d0 0x7f000406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0008036000 tx=0x7f000803b040 comp rx=0 tx=0).stop 2024-03-06T07:21:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00180724b0 msgr2=0x7f00180c0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00180724b0 0x7f00180c0500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f001400c3f0 tx=0x7f001400c8b0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 shutdown_connections 2024-03-06T07:21:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f000406b4d0 0x7f000406d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0018155d30 0x7f00181dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00181528e0 0x7f00181deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 --2- 172.21.15.26:0/3001589009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00180724b0 0x7f00180c0500 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.479+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 >> 172.21.15.26:0/3001589009 conn(0x7f00180c5310 msgr2=0x7f0018072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.480+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 shutdown_connections 2024-03-06T07:21:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.480+0000 7f001f088640 1 -- 172.21.15.26:0/3001589009 wait complete. 2024-03-06T07:21:51.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:21:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.596+0000 7f72d33eb640 1 Processor -- start 2024-03-06T07:21:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.596+0000 7f72d33eb640 1 -- start start 2024-03-06T07:21:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.597+0000 7f72d33eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f72cc14ef70 0x7f72cc06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.597+0000 7f72d33eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc0c2790 con 0x7f72cc0c1820 2024-03-06T07:21:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.597+0000 7f72d33eb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc06cf40 con 0x7f72cc06cad0 2024-03-06T07:21:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.597+0000 7f72d33eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc0c3090 con 0x7f72cc14ef70 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.597+0000 7f72d0986640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f72cc0c1820 0x7f72cc0c2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38404/0 (socket says 172.21.15.26:38404) 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.597+0000 7f72d0986640 1 -- 172.21.15.26:0/1575403972 learned_addr learned my addr 172.21.15.26:0/1575403972 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.597+0000 7f72d1187640 1 --2- 172.21.15.26:0/1575403972 >> v2:172.21.15.117:3300/0 conn(0x7f72cc14ef70 0x7f72cc06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.598+0000 7f72c3fff640 1 -- 172.21.15.26:0/1575403972 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423922446 0 0) 0x7f72cc0c2790 con 0x7f72cc0c1820 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.598+0000 7f72c3fff640 1 -- 172.21.15.26:0/1575403972 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b8003520 con 0x7f72cc0c1820 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.598+0000 7f72d1187640 1 -- 172.21.15.26:0/1575403972 >> v1:172.21.15.117:6789/0 conn(0x7f72cc06cad0 legacy=0x7f72cc154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.598+0000 7f72d1187640 1 -- 172.21.15.26:0/1575403972 >> v1:172.21.15.26:6789/0 conn(0x7f72cc0c1820 legacy=0x7f72cc0c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.598+0000 7f72d1187640 1 -- 172.21.15.26:0/1575403972 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0c1540 con 0x7f72cc14ef70 2024-03-06T07:21:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72d1187640 1 --2- 172.21.15.26:0/1575403972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc14ef70 0x7f72cc06c1f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f72c80097a0 tx=0x7f72c802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5c2266583cc3df server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72c3fff640 1 -- 172.21.15.26:0/1575403972 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8035020 con 0x7f72cc14ef70 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72c3fff640 1 -- 172.21.15.26:0/1575403972 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8031830 con 0x7f72cc14ef70 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72c3fff640 1 -- 172.21.15.26:0/1575403972 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8031c00 con 0x7f72cc14ef70 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72d33eb640 1 -- 172.21.15.26:0/1575403972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc14ef70 msgr2=0x7f72cc06c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72d33eb640 1 --2- 172.21.15.26:0/1575403972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc14ef70 0x7f72cc06c1f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f72c80097a0 tx=0x7f72c802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72d33eb640 1 -- 172.21.15.26:0/1575403972 shutdown_connections 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72d33eb640 1 --2- 172.21.15.26:0/1575403972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc14ef70 0x7f72cc06c1f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.599+0000 7f72d33eb640 1 -- 172.21.15.26:0/1575403972 >> 172.21.15.26:0/1575403972 conn(0x7f72cc0c6710 msgr2=0x7f72cc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.600+0000 7f72d33eb640 1 -- 172.21.15.26:0/1575403972 shutdown_connections 2024-03-06T07:21:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.600+0000 7f72d33eb640 1 -- 172.21.15.26:0/1575403972 wait complete. 2024-03-06T07:21:51.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.601+0000 7f72d33eb640 1 Processor -- start 2024-03-06T07:21:51.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.601+0000 7f72d33eb640 1 -- start start 2024-03-06T07:21:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.602+0000 7f72d33eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72cc06cad0 0x7f72cc0aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.602+0000 7f72d33eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc0c1820 0x7f72cc0a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.602+0000 7f72d33eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72cc14ef70 0x7f72cc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.602+0000 7f72d33eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc155750 con 0x7f72cc14ef70 2024-03-06T07:21:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.602+0000 7f72d33eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc14f350 con 0x7f72cc0c1820 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.602+0000 7f72d33eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc155340 con 0x7f72cc06cad0 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.602+0000 7f72d1988640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72cc14ef70 0x7f72cc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72cc14ef70 0x7f72cc14da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50136/0 (socket says 172.21.15.26:50136) 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 -- 172.21.15.26:0/155410166 learned_addr learned my addr 172.21.15.26:0/155410166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d0986640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc0c1820 0x7f72cc0a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1187640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72cc06cad0 0x7f72cc0aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 -- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72cc06cad0 msgr2=0x7f72cc0aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72cc06cad0 0x7f72cc0aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 -- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc0c1820 msgr2=0x7f72cc0a7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc0c1820 0x7f72cc0a7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 -- 172.21.15.26:0/155410166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc1e6cc0 con 0x7f72cc14ef70 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d0986640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc0c1820 0x7f72cc0a7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1187640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72cc06cad0 0x7f72cc0aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d1988640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72cc14ef70 0x7f72cc14da10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c7d0 tx=0x7f72c400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72c1ffb640 1 -- 172.21.15.26:0/155410166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4003b70 con 0x7f72cc14ef70 2024-03-06T07:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72c1ffb640 1 -- 172.21.15.26:0/155410166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4003d30 con 0x7f72cc14ef70 2024-03-06T07:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc1e6f80 con 0x7f72cc14ef70 2024-03-06T07:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.603+0000 7f72c1ffb640 1 -- 172.21.15.26:0/155410166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400aa90 con 0x7f72cc14ef70 2024-03-06T07:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.604+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1e7540 con 0x7f72cc14ef70 2024-03-06T07:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.605+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc1e4960 con 0x7f72cc14ef70 2024-03-06T07:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.605+0000 7f72c1ffb640 1 -- 172.21.15.26:0/155410166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72c400ac30 con 0x7f72cc14ef70 2024-03-06T07:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.605+0000 7f72c1ffb640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.605+0000 7f72c1ffb640 1 -- 172.21.15.26:0/155410166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f72c408a260 con 0x7f72cc14ef70 2024-03-06T07:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.605+0000 7f72d1187640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.606+0000 7f72d1187640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f72b006b4d0 0x7f72b006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f72c8036000 tx=0x7f72c803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.609+0000 7f72c1ffb640 1 -- 172.21.15.26:0/155410166 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f72cc06cf40 con 0x7f72cc14ef70 2024-03-06T07:21:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.805+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72cc04f330 con 0x7f72cc14ef70 2024-03-06T07:21:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.805+0000 7f72c1ffb640 1 -- 172.21.15.26:0/155410166 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72c4058fb0 con 0x7f72cc14ef70 2024-03-06T07:21:51.807 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f72b006b4d0 msgr2=0x7f72b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f72b006b4d0 0x7f72b006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f72c8036000 tx=0x7f72c803c040 comp rx=0 tx=0).stop 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72cc14ef70 msgr2=0x7f72cc14da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72cc14ef70 0x7f72cc14da10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c7d0 tx=0x7f72c400cc90 comp rx=0 tx=0).stop 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 shutdown_connections 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72cc14ef70 0x7f72cc14da10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc0c1820 0x7f72cc0a7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 --2- 172.21.15.26:0/155410166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72cc06cad0 0x7f72cc0aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 >> 172.21.15.26:0/155410166 conn(0x7f72cc0c6710 msgr2=0x7f72cc0c3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 shutdown_connections 2024-03-06T07:21:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.808+0000 7f72d33eb640 1 -- 172.21.15.26:0/155410166 wait complete. 2024-03-06T07:21:51.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:21:51.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.923+0000 7f747b888640 1 Processor -- start 2024-03-06T07:21:51.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.924+0000 7f747b888640 1 -- start start 2024-03-06T07:21:51.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.924+0000 7f747b888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 0x7f74740c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.924+0000 7f747b888640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74741572f0 con 0x7f74741506b0 2024-03-06T07:21:51.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.924+0000 7f747b888640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74740ad310 con 0x7f7474153990 2024-03-06T07:21:51.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.924+0000 7f747b888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7474156450 con 0x7f7474156ea0 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.924+0000 7f7479624640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f74741506b0 0x7f74740702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38406/0 (socket says 172.21.15.26:38406) 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.924+0000 7f7479624640 1 -- 172.21.15.26:0/1928968347 learned_addr learned my addr 172.21.15.26:0/1928968347 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7479e25640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 0x7f74740c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220631839 0 0) 0x7f74741572f0 con 0x7f74741506b0 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f745c003520 con 0x7f74741506b0 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749741540 0 0) 0x7f74740ad310 con 0x7f7474153990 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74741572f0 con 0x7f7474153990 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657784 0 0) 0x7f745c003520 con 0x7f74741506b0 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 msgr2=0x7f74740c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 0x7f74740c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 >> v1:172.21.15.117:6789/0 conn(0x7f7474153990 legacy=0x7f74740acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7474070aa0 con 0x7f74741506b0 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f746c001690 con 0x7f74741506b0 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7479e25640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 0x7f74740c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 >> v1:172.21.15.26:6789/0 conn(0x7f74741506b0 legacy=0x7f74740702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f745c003b70 0x7f745c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f745c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 0x7f745c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7474156450 con 0x7f745c004800 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c00b9f0 con 0x7f745c003b70 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c00bbf0 con 0x7f7474156ea0 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7479624640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f745c003b70 0x7f745c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7478e23640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f745c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7479e25640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 0x7f745c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.925+0000 7f7478e23640 1 -- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 msgr2=0x7f745c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.926+0000 7f7478e23640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7474156ea0 0x7f745c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.926+0000 7f7478e23640 1 -- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f745c003b70 msgr2=0x7f745c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.926+0000 7f7478e23640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f745c003b70 0x7f745c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.926+0000 7f7478e23640 1 -- 172.21.15.26:0/1928968347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c003520 con 0x7f745c004800 2024-03-06T07:21:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f7478e23640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f745c004c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74680018a0 tx=0x7f745c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9635990ed8d350f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468037070 con 0x7f745c004800 2024-03-06T07:21:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74680052d0 con 0x7f745c004800 2024-03-06T07:21:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f7473fff640 1 -- 172.21.15.26:0/1928968347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74680014f0 con 0x7f745c004800 2024-03-06T07:21:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f747b888640 1 -- 172.21.15.26:0/1928968347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 msgr2=0x7f745c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f747b888640 1 --2- 172.21.15.26:0/1928968347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f745c004c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74680018a0 tx=0x7f745c005830 comp rx=0 tx=0).stop 2024-03-06T07:21:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f7479e25640 1 -- 172.21.15.26:0/1928968347 reap_dead start 2024-03-06T07:21:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f747b888640 1 -- 172.21.15.26:0/1928968347 shutdown_connections 2024-03-06T07:21:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f747b888640 1 -- 172.21.15.26:0/1928968347 >> 172.21.15.26:0/1928968347 conn(0x7f747406b1a0 msgr2=0x7f747406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f747b888640 1 -- 172.21.15.26:0/1928968347 shutdown_connections 2024-03-06T07:21:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.927+0000 7f747b888640 1 -- 172.21.15.26:0/1928968347 wait complete. 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.928+0000 7f747b888640 1 Processor -- start 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f747b888640 1 -- start start 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f747b888640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f74741e9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f747b888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740a8860 0x7f74740aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f747b888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ab220 0x7f7474149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f747b888640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74741566a0 con 0x7f745c004800 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f747b888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7474156a10 con 0x7f74740ab220 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f747b888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74740c54a0 con 0x7f74740a8860 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f7479624640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f74741e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f7479624640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f74741e9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50158/0 (socket says 172.21.15.26:50158) 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.929+0000 7f7479624640 1 -- 172.21.15.26:0/1729419527 learned_addr learned my addr 172.21.15.26:0/1729419527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7479624640 1 -- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740a8860 msgr2=0x7f74740aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7479624640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740a8860 0x7f74740aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7479624640 1 -- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ab220 msgr2=0x7f7474149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7479624640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ab220 0x7f7474149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7479624640 1 -- 172.21.15.26:0/1729419527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747414cea0 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7479624640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f74741e9520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f746c0072c0 tx=0x7f746c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7471ffb640 1 -- 172.21.15.26:0/1729419527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c00f940 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7471ffb640 1 -- 172.21.15.26:0/1729419527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c018070 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f7471ffb640 1 -- 172.21.15.26:0/1729419527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c00fd40 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747414d180 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.930+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747414d460 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.931+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7474150f90 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.931+0000 7f7471ffb640 1 -- 172.21.15.26:0/1729419527 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f746c01d460 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.932+0000 7f7471ffb640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f746006b500 0x7f746006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.932+0000 7f7471ffb640 1 -- 172.21.15.26:0/1729419527 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f746c08a250 con 0x7f745c004800 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.932+0000 7f7478e23640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f746006b500 0x7f746006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.933+0000 7f7478e23640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f746006b500 0x7f746006d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7468034000 tx=0x7f7468009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:51.936+0000 7f7471ffb640 1 -- 172.21.15.26:0/1729419527 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7474150f90 con 0x7f745c004800 2024-03-06T07:21:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.118+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f747404f330 con 0x7f745c004800 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.118+0000 7f7471ffb640 1 -- 172.21.15.26:0/1729419527 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v52) v1 ==== 84+0+207 (secure 0 0 0) 0x7f746c058fa0 con 0x7f745c004800 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 7, 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_metadata" 2024-03-06T07:21:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:21:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:21:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 8, 2024-03-06T07:21:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_data" 2024-03-06T07:21:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:21:52.121 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f746006b500 msgr2=0x7f746006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f746006b500 0x7f746006d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7468034000 tx=0x7f7468009030 comp rx=0 tx=0).stop 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 msgr2=0x7f74741e9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f74741e9520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f746c0072c0 tx=0x7f746c007780 comp rx=0 tx=0).stop 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 shutdown_connections 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f746006b500 0x7f746006d9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ab220 0x7f7474149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740a8860 0x7f74740aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 --2- 172.21.15.26:0/1729419527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f745c004800 0x7f74741e9520 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 >> 172.21.15.26:0/1729419527 conn(0x7f747406b1a0 msgr2=0x7f7474151550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 shutdown_connections 2024-03-06T07:21:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.120+0000 7f747b888640 1 -- 172.21.15.26:0/1729419527 wait complete. 2024-03-06T07:21:52.132 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-06T07:21:52.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-06T07:21:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.236+0000 7f04adced640 1 Processor -- start 2024-03-06T07:21:52.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.236+0000 7f04adced640 1 -- start start 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04adced640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f04a8072fc0 0x7f04a80724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04adced640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a80c6220 con 0x7f04a80c5e40 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04adced640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a8070210 con 0x7f04a8072a10 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04adced640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a80704d0 con 0x7f04a8072fc0 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f04a80c5e40 0x7f04a8153940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38412/0 (socket says 172.21.15.26:38412) 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a77fe640 1 -- 172.21.15.26:0/3864897235 learned_addr learned my addr 172.21.15.26:0/3864897235 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297790030 0 0) 0x7f04a80c6220 con 0x7f04a80c5e40 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a6ffd640 1 --2- 172.21.15.26:0/3864897235 >> v2:172.21.15.117:3300/0 conn(0x7f04a8072fc0 0x7f04a80724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0494003520 con 0x7f04a80c5e40 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 388322902 0 0) 0x7f0494003520 con 0x7f04a80c5e40 2024-03-06T07:21:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 >> v2:172.21.15.117:3300/0 conn(0x7f04a8072fc0 msgr2=0x7f04a80724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 --2- 172.21.15.26:0/3864897235 >> v2:172.21.15.117:3300/0 conn(0x7f04a8072fc0 0x7f04a80724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 >> v1:172.21.15.117:6789/0 conn(0x7f04a8072a10 legacy=0x7f04a80c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a806f610 con 0x7f04a80c5e40 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f0498001670 con 0x7f04a80c5e40 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 >> v1:172.21.15.26:6789/0 conn(0x7f04a80c5e40 legacy=0x7f04a8153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f0494003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0494004800 0x7f0494004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0494005210 0x7f049400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80704d0 con 0x7f0494003a00 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049400c160 con 0x7f0494005210 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.237+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049400c360 con 0x7f0494004800 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a77fe640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f0494003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a6ffd640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0494004800 0x7f0494004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a7fff640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0494005210 0x7f049400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a77fe640 1 -- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0494004800 msgr2=0x7f0494004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a77fe640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0494004800 0x7f0494004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a77fe640 1 -- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0494005210 msgr2=0x7f049400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a77fe640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0494005210 0x7f049400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a77fe640 1 -- 172.21.15.26:0/3864897235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494003520 con 0x7f0494003a00 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.238+0000 7f04a7fff640 1 -- 172.21.15.26:0/3864897235 reap_dead start 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04a77fe640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f0494003e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f049800c160 tx=0x7f0494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6cc23e5ed7456 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498007c60 con 0x7f0494003a00 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498030070 con 0x7f0494003a00 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04a67fc640 1 -- 172.21.15.26:0/3864897235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498005d90 con 0x7f0494003a00 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04adced640 1 -- 172.21.15.26:0/3864897235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 msgr2=0x7f0494003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04adced640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f0494003e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f049800c160 tx=0x7f0494005830 comp rx=0 tx=0).stop 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04adced640 1 -- 172.21.15.26:0/3864897235 shutdown_connections 2024-03-06T07:21:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04adced640 1 --2- 172.21.15.26:0/3864897235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f0494003e20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04adced640 1 -- 172.21.15.26:0/3864897235 >> 172.21.15.26:0/3864897235 conn(0x7f04a80c88f0 msgr2=0x7f04a80c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04adced640 1 -- 172.21.15.26:0/3864897235 shutdown_connections 2024-03-06T07:21:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.239+0000 7f04adced640 1 -- 172.21.15.26:0/3864897235 wait complete. 2024-03-06T07:21:52.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.240+0000 7f04adced640 1 Processor -- start 2024-03-06T07:21:52.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04adced640 1 -- start start 2024-03-06T07:21:52.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04adced640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f04a806c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04adced640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a80a7ff0 0x7f04a8154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04adced640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a814b180 0x7f04a81ee510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04adced640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80724d0 con 0x7f0494003a00 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04adced640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a806fbd0 con 0x7f04a814b180 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04adced640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a8072650 con 0x7f04a80a7ff0 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f04a806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f04a806c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50174/0 (socket says 172.21.15.26:50174) 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.241+0000 7f04a77fe640 1 -- 172.21.15.26:0/3034787169 learned_addr learned my addr 172.21.15.26:0/3034787169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04a77fe640 1 -- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a80a7ff0 msgr2=0x7f04a8154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04a77fe640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a80a7ff0 0x7f04a8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04a77fe640 1 -- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a814b180 msgr2=0x7f04a81ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04a77fe640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a814b180 0x7f04a81ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04a77fe640 1 -- 172.21.15.26:0/3034787169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a81edd30 con 0x7f0494003a00 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04a77fe640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f04a806c960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f049800c160 tx=0x7f0498001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f049ffff640 1 -- 172.21.15.26:0/3034787169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498045070 con 0x7f0494003a00 2024-03-06T07:21:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f049ffff640 1 -- 172.21.15.26:0/3034787169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04980047e0 con 0x7f0494003a00 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f049ffff640 1 -- 172.21.15.26:0/3034787169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498007c60 con 0x7f0494003a00 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a8154630 con 0x7f0494003a00 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.242+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a8156ea0 con 0x7f0494003a00 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.243+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a804f330 con 0x7f0494003a00 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.243+0000 7f049ffff640 1 -- 172.21.15.26:0/3034787169 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0498004bd0 con 0x7f0494003a00 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.244+0000 7f049ffff640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048806b500 0x7f048806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.244+0000 7f049ffff640 1 -- 172.21.15.26:0/3034787169 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f04980ab050 con 0x7f0494003a00 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.244+0000 7f04a6ffd640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048806b500 0x7f048806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.245+0000 7f04a6ffd640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048806b500 0x7f048806d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0490006e80 tx=0x7f0490006df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.248+0000 7f049ffff640 1 -- 172.21.15.26:0/3034787169 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0498079da0 con 0x7f0494003a00 2024-03-06T07:21:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:52.603+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f04a806f3f0 con 0x7f0494003a00 2024-03-06T07:21:53.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.348+0000 7f049ffff640 1 -- 172.21.15.26:0/3034787169 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v53) v1 ==== 140+0+0 (secure 0 0 0) 0x7f04980796a0 con 0x7f0494003a00 2024-03-06T07:21:53.350 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_metadata' created 2024-03-06T07:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048806b500 msgr2=0x7f048806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048806b500 0x7f048806d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0490006e80 tx=0x7f0490006df0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 msgr2=0x7f04a806c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f04a806c960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f049800c160 tx=0x7f0498001970 comp rx=0 tx=0).stop 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 shutdown_connections 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048806b500 0x7f048806d9c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a814b180 0x7f04a81ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a80a7ff0 0x7f04a8154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 --2- 172.21.15.26:0/3034787169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0494003a00 0x7f04a806c960 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.350+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 >> 172.21.15.26:0/3034787169 conn(0x7f04a80c88f0 msgr2=0x7f04a8151080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.351+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 shutdown_connections 2024-03-06T07:21:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.351+0000 7f04adced640 1 -- 172.21.15.26:0/3034787169 wait complete. 2024-03-06T07:21:53.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:21:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.478+0000 7f987daed640 1 Processor -- start 2024-03-06T07:21:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.478+0000 7f987daed640 1 -- start start 2024-03-06T07:21:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.479+0000 7f987daed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98781539c0 0x7f9878072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.479+0000 7f987daed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98780c62a0 con 0x7f98780c4740 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.479+0000 7f987daed640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98780c4c00 con 0x7f98780a8290 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.479+0000 7f987daed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98780c4ee0 con 0x7f98781539c0 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.479+0000 7f986ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f98780c4740 0x7f98780c69a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38422/0 (socket says 172.21.15.26:38422) 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.479+0000 7f986ffff640 1 -- 172.21.15.26:0/3824150958 learned_addr learned my addr 172.21.15.26:0/3824150958 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9877fff640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98781539c0 0x7f9878072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248562865 0 0) 0x7f98780c62a0 con 0x7f98780c4740 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9864003520 con 0x7f98780c4740 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295544337 0 0) 0x7f98780c4c00 con 0x7f98780a8290 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98780c62a0 con 0x7f98780a8290 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483499771 0 0) 0x7f9864003520 con 0x7f98780c4740 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98781539c0 msgr2=0x7f9878072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98781539c0 0x7f9878072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 >> v1:172.21.15.117:6789/0 conn(0x7f98780a8290 legacy=0x7f98780701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987806f9a0 con 0x7f98780c4740 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f9860001670 con 0x7f98780c4740 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 >> v1:172.21.15.26:6789/0 conn(0x7f98780c4740 legacy=0x7f98780c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9864003b70 0x7f9864003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9864004800 0x7f9864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98640051c0 0x7f986400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98780c4ee0 con 0x7f98640051c0 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986400c0f0 con 0x7f9864003b70 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986400c2f0 con 0x7f9864004800 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9877fff640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98781539c0 0x7f9878072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f9877fff640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98640051c0 0x7f986400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f986ffff640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9864004800 0x7f9864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.480+0000 7f98777fe640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9864003b70 0x7f9864003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9877fff640 1 -- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9864004800 msgr2=0x7f9864004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9877fff640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9864004800 0x7f9864004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9877fff640 1 -- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9864003b70 msgr2=0x7f9864003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9877fff640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9864003b70 0x7f9864003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9877fff640 1 -- 172.21.15.26:0/3824150958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864003520 con 0x7f98640051c0 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9877fff640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98640051c0 0x7f986400b9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9878153da0 tx=0x7f9864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfa6f0611384209 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870009d40 con 0x7f98640051c0 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987002ccf0 con 0x7f98640051c0 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.481+0000 7f9876ffd640 1 -- 172.21.15.26:0/3824150958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870003bb0 con 0x7f98640051c0 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.482+0000 7f987daed640 1 -- 172.21.15.26:0/3824150958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98640051c0 msgr2=0x7f986400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.482+0000 7f987daed640 1 --2- 172.21.15.26:0/3824150958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98640051c0 0x7f986400b9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9878153da0 tx=0x7f9864005830 comp rx=0 tx=0).stop 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.482+0000 7f9877fff640 1 -- 172.21.15.26:0/3824150958 reap_dead start 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.482+0000 7f987daed640 1 -- 172.21.15.26:0/3824150958 shutdown_connections 2024-03-06T07:21:53.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.482+0000 7f987daed640 1 -- 172.21.15.26:0/3824150958 >> 172.21.15.26:0/3824150958 conn(0x7f98780c5560 msgr2=0x7f98780c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:53.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.482+0000 7f987daed640 1 -- 172.21.15.26:0/3824150958 shutdown_connections 2024-03-06T07:21:53.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.482+0000 7f987daed640 1 -- 172.21.15.26:0/3824150958 wait complete. 2024-03-06T07:21:53.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.483+0000 7f987daed640 1 Processor -- start 2024-03-06T07:21:53.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.483+0000 7f987daed640 1 -- start start 2024-03-06T07:21:53.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f987daed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640051c0 0x7f9878151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f987daed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98781555b0 0x7f987814dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f987daed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9878152300 0x7f98780ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f987daed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9878072680 con 0x7f98781555b0 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f987daed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9878153e50 con 0x7f9878152300 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f987daed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9878072800 con 0x7f98640051c0 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98781555b0 0x7f987814dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98781555b0 0x7f987814dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50192/0 (socket says 172.21.15.26:50192) 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 -- 172.21.15.26:0/3644692943 learned_addr learned my addr 172.21.15.26:0/3644692943 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f9877fff640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9878152300 0x7f98780ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f98777fe640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640051c0 0x7f9878151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 -- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640051c0 msgr2=0x7f9878151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640051c0 0x7f9878151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 -- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9878152300 msgr2=0x7f98780ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9878152300 0x7f98780ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 -- 172.21.15.26:0/3644692943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780ac9c0 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f9877fff640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9878152300 0x7f98780ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f98777fe640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640051c0 0x7f9878151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.484+0000 7f986ffff640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98781555b0 0x7f987814dba0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9860001b20 tx=0x7f9860007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.485+0000 7f9874ff9640 1 -- 172.21.15.26:0/3644692943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000f750 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.485+0000 7f9874ff9640 1 -- 172.21.15.26:0/3644692943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9860018070 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.485+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98781570b0 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.485+0000 7f9874ff9640 1 -- 172.21.15.26:0/3644692943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000fb30 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.485+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98780ac1c0 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.486+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987806efa0 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.486+0000 7f9874ff9640 1 -- 172.21.15.26:0/3644692943 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f986000fcd0 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.487+0000 7f9874ff9640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f985806b500 0x7f985806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.487+0000 7f9874ff9640 1 -- 172.21.15.26:0/3644692943 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f986008a4b0 con 0x7f98781555b0 2024-03-06T07:21:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.487+0000 7f98777fe640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f985806b500 0x7f985806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.488+0000 7f98777fe640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f985806b500 0x7f985806d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f98680010e0 tx=0x7f9868009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.492+0000 7f9874ff9640 1 -- 172.21.15.26:0/3644692943 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f987806efa0 con 0x7f98781555b0 2024-03-06T07:21:53.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:53.688+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9878075300 con 0x7f98781555b0 2024-03-06T07:21:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.353+0000 7f9874ff9640 1 -- 172.21.15.26:0/3644692943 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v54) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9878075300 con 0x7f98781555b0 2024-03-06T07:21:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_data' created 2024-03-06T07:21:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f985806b500 msgr2=0x7f985806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f985806b500 0x7f985806d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f98680010e0 tx=0x7f9868009040 comp rx=0 tx=0).stop 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98781555b0 msgr2=0x7f987814dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98781555b0 0x7f987814dba0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9860001b20 tx=0x7f9860007430 comp rx=0 tx=0).stop 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 shutdown_connections 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f985806b500 0x7f985806d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9878152300 0x7f98780ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98781555b0 0x7f987814dba0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 --2- 172.21.15.26:0/3644692943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640051c0 0x7f9878151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 >> 172.21.15.26:0/3644692943 conn(0x7f98780c5560 msgr2=0x7f98780a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 shutdown_connections 2024-03-06T07:21:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.355+0000 7f987daed640 1 -- 172.21.15.26:0/3644692943 wait complete. 2024-03-06T07:21:54.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-06T07:21:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.521+0000 7fd503555640 1 Processor -- start 2024-03-06T07:21:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.522+0000 7fd503555640 1 -- start start 2024-03-06T07:21:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.522+0000 7fd503555640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4fc1527f0 0x7fd4fc0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.522+0000 7fd503555640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc155260 con 0x7fd4fc0c7370 2024-03-06T07:21:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.522+0000 7fd503555640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc155540 con 0x7fd4fc0c7800 2024-03-06T07:21:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.522+0000 7fd503555640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c7be0 con 0x7fd4fc1527f0 2024-03-06T07:21:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd500af0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd4fc0c7370 0x7fd4fc06c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38424/0 (socket says 172.21.15.26:38424) 2024-03-06T07:21:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd500af0640 1 -- 172.21.15.26:0/3593428337 learned_addr learned my addr 172.21.15.26:0/3593428337 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd5012f1640 1 --2- 172.21.15.26:0/3593428337 >> v2:172.21.15.117:3300/0 conn(0x7fd4fc1527f0 0x7fd4fc0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083471174 0 0) 0x7fd4fc155260 con 0x7fd4fc0c7370 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e4003520 con 0x7fd4fc0c7370 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3791117457 0 0) 0x7fd4e4003520 con 0x7fd4fc0c7370 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 >> v2:172.21.15.117:3300/0 conn(0x7fd4fc1527f0 msgr2=0x7fd4fc0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 --2- 172.21.15.26:0/3593428337 >> v2:172.21.15.117:3300/0 conn(0x7fd4fc1527f0 0x7fd4fc0c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 >> v1:172.21.15.117:6789/0 conn(0x7fd4fc0c7800 legacy=0x7fd4fc06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc155820 con 0x7fd4fc0c7370 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd4f0001670 con 0x7fd4fc0c7370 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 >> v1:172.21.15.26:6789/0 conn(0x7fd4fc0c7370 legacy=0x7fd4fc06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4e40039b0 0x7fd4e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e4004800 0x7fd4e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e40052e0 0x7fd4e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c7be0 con 0x7fd4e40039b0 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e400c1b0 con 0x7fd4e40052e0 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e400c3b0 con 0x7fd4e4004800 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.523+0000 7fd5012f1640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4e40039b0 0x7fd4e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd501af2640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e40052e0 0x7fd4e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd500af0640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e4004800 0x7fd4e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd5012f1640 1 -- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e4004800 msgr2=0x7fd4e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd5012f1640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e4004800 0x7fd4e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd5012f1640 1 -- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e40052e0 msgr2=0x7fd4e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd5012f1640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e40052e0 0x7fd4e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd5012f1640 1 -- 172.21.15.26:0/3593428337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e4003520 con 0x7fd4e40039b0 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd501af2640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e40052e0 0x7fd4e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd501af2640 1 -- 172.21.15.26:0/3593428337 reap_dead start 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.524+0000 7fd5012f1640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4e40039b0 0x7fd4e4003dd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009a90 tx=0x7fd4e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4111271b15f96825 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4039070 con 0x7fd4e40039b0 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f402c680 con 0x7fd4e40039b0 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd4fbfff640 1 -- 172.21.15.26:0/3593428337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4009690 con 0x7fd4e40039b0 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd503555640 1 -- 172.21.15.26:0/3593428337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4e40039b0 msgr2=0x7fd4e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd503555640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4e40039b0 0x7fd4e4003dd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009a90 tx=0x7fd4e4005830 comp rx=0 tx=0).stop 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd503555640 1 -- 172.21.15.26:0/3593428337 shutdown_connections 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd503555640 1 --2- 172.21.15.26:0/3593428337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4e40039b0 0x7fd4e4003dd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd503555640 1 -- 172.21.15.26:0/3593428337 >> 172.21.15.26:0/3593428337 conn(0x7fd4fc0ac4f0 msgr2=0x7fd4fc0ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd503555640 1 -- 172.21.15.26:0/3593428337 shutdown_connections 2024-03-06T07:21:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.525+0000 7fd503555640 1 -- 172.21.15.26:0/3593428337 wait complete. 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.527+0000 7fd503555640 1 Processor -- start 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.527+0000 7fd503555640 1 -- start start 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd503555640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e40039b0 0x7fd4fc0c0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd503555640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc14f960 0x7fd4fc147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd503555640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc0b03b0 0x7fd4fc147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd503555640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c8860 con 0x7fd4fc14f960 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd503555640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0be1a0 con 0x7fd4fc0b03b0 2024-03-06T07:21:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd503555640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c89e0 con 0x7fd4e40039b0 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc14f960 0x7fd4fc147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc14f960 0x7fd4fc147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50198/0 (socket says 172.21.15.26:50198) 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 -- 172.21.15.26:0/684628234 learned_addr learned my addr 172.21.15.26:0/684628234 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd501af2640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc0b03b0 0x7fd4fc147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd5012f1640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e40039b0 0x7fd4fc0c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 -- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e40039b0 msgr2=0x7fd4fc0c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e40039b0 0x7fd4fc0c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 -- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc0b03b0 msgr2=0x7fd4fc147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc0b03b0 0x7fd4fc147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 -- 172.21.15.26:0/684628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc14a4d0 con 0x7fd4fc14f960 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.528+0000 7fd500af0640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc14f960 0x7fd4fc147ab0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000e730 tx=0x7fd4f000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.529+0000 7fd4f9ffb640 1 -- 172.21.15.26:0/684628234 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000ee40 con 0x7fd4fc14f960 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.529+0000 7fd4f9ffb640 1 -- 172.21.15.26:0/684628234 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f0018070 con 0x7fd4fc14f960 2024-03-06T07:21:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.529+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc14d980 con 0x7fd4fc14f960 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.529+0000 7fd4f9ffb640 1 -- 172.21.15.26:0/684628234 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000ca90 con 0x7fd4fc14f960 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.529+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc14df40 con 0x7fd4fc14f960 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.530+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc04f330 con 0x7fd4fc14f960 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.530+0000 7fd4f9ffb640 1 -- 172.21.15.26:0/684628234 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4f000cc30 con 0x7fd4fc14f960 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.531+0000 7fd4f9ffb640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.531+0000 7fd4f9ffb640 1 -- 172.21.15.26:0/684628234 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6913+0+0 (secure 0 0 0) 0x7fd4f008a460 con 0x7fd4fc14f960 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.531+0000 7fd5012f1640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.531+0000 7fd5012f1640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4036000 tx=0x7fd4f4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.535+0000 7fd4f9ffb640 1 -- 172.21.15.26:0/684628234 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd4f0058eb0 con 0x7fd4fc14f960 2024-03-06T07:21:54.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:54.754+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7fd4fc14c850 con 0x7fd4fc14f960 2024-03-06T07:21:55.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.366+0000 7fd4f9ffb640 1 -- 172.21.15.26:0/684628234 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 9 and data pool 10 v30) v1 ==== 176+0+0 (secure 0 0 0) 0x7fd4f00587b0 con 0x7fd4fc14f960 2024-03-06T07:21:55.368 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 9 and data pool 10 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 msgr2=0x7fd4e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4036000 tx=0x7fd4f4030000 comp rx=0 tx=0).stop 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc14f960 msgr2=0x7fd4fc147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc14f960 0x7fd4fc147ab0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000e730 tx=0x7fd4f000ebf0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 shutdown_connections 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc0b03b0 0x7fd4fc147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc14f960 0x7fd4fc147ab0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 --2- 172.21.15.26:0/684628234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e40039b0 0x7fd4fc0c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 >> 172.21.15.26:0/684628234 conn(0x7fd4fc0ac4f0 msgr2=0x7fd4fc0aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 shutdown_connections 2024-03-06T07:21:55.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.371+0000 7fd503555640 1 -- 172.21.15.26:0/684628234 wait complete. 2024-03-06T07:21:55.390 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:55.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.490+0000 7fc33cb55640 1 Processor -- start 2024-03-06T07:21:55.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.490+0000 7fc33cb55640 1 -- start start 2024-03-06T07:21:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc33cb55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc33814f360 0x7fc338155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc33cb55640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc3380c88c0 0x7fc3380c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc33cb55640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3380c5880 0x7fc338158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc33cb55640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380c8dd0 con 0x7fc3380c5880 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc33cb55640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380c73d0 con 0x7fc3380c88c0 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc33cb55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380be2e0 con 0x7fc33814f360 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3380c5880 0x7fc338158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3380c5880 0x7fc338158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50204/0 (socket says 172.21.15.26:50204) 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 -- 172.21.15.26:0/3404890011 learned_addr learned my addr 172.21.15.26:0/3404890011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc33659c640 1 --2- 172.21.15.26:0/3404890011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc33814f360 0x7fc338155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc32ffff640 1 --2- 172.21.15.26:0/3404890011 >> v2:172.21.15.117:3300/0 conn(0x7fc3380c88c0 0x7fc3380c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 -- 172.21.15.26:0/3404890011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc33814f360 msgr2=0x7fc338155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 --2- 172.21.15.26:0/3404890011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc33814f360 0x7fc338155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 -- 172.21.15.26:0/3404890011 >> v2:172.21.15.117:3300/0 conn(0x7fc3380c88c0 msgr2=0x7fc3380c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 --2- 172.21.15.26:0/3404890011 >> v2:172.21.15.117:3300/0 conn(0x7fc3380c88c0 0x7fc3380c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.491+0000 7fc336d9d640 1 -- 172.21.15.26:0/3404890011 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380c7670 con 0x7fc3380c5880 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.492+0000 7fc336d9d640 1 --2- 172.21.15.26:0/3404890011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3380c5880 0x7fc338158b60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc3380cbac0 tx=0x7fc33002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c174526671f0bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.492+0000 7fc335d9b640 1 -- 172.21.15.26:0/3404890011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33002c9e0 con 0x7fc3380c5880 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.492+0000 7fc335d9b640 1 -- 172.21.15.26:0/3404890011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33002cb80 con 0x7fc3380c5880 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.492+0000 7fc335d9b640 1 -- 172.21.15.26:0/3404890011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3300343f0 con 0x7fc3380c5880 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 -- 172.21.15.26:0/3404890011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3380c5880 msgr2=0x7fc338158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 --2- 172.21.15.26:0/3404890011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3380c5880 0x7fc338158b60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc3380cbac0 tx=0x7fc33002be40 comp rx=0 tx=0).stop 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 -- 172.21.15.26:0/3404890011 shutdown_connections 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 --2- 172.21.15.26:0/3404890011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3380c5880 0x7fc338158b60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 --2- 172.21.15.26:0/3404890011 >> v2:172.21.15.117:3300/0 conn(0x7fc3380c88c0 0x7fc3380c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 --2- 172.21.15.26:0/3404890011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc33814f360 0x7fc338155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 -- 172.21.15.26:0/3404890011 >> 172.21.15.26:0/3404890011 conn(0x7fc3380ac570 msgr2=0x7fc3380ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 -- 172.21.15.26:0/3404890011 shutdown_connections 2024-03-06T07:21:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.493+0000 7fc33cb55640 1 -- 172.21.15.26:0/3404890011 wait complete. 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.494+0000 7fc33cb55640 1 Processor -- start 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.494+0000 7fc33cb55640 1 -- start start 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3380c5880 0x7fc3381ee780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c88c0 0x7fc3380ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33814f360 0x7fc338156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381590a0 con 0x7fc33814f360 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380c7c30 con 0x7fc3380c5880 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc338159220 con 0x7fc3380c88c0 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33814f360 0x7fc338156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33814f360 0x7fc338156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50210/0 (socket says 172.21.15.26:50210) 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 -- 172.21.15.26:0/1001211428 learned_addr learned my addr 172.21.15.26:0/1001211428 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc32ffff640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c88c0 0x7fc3380ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33659c640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3380c5880 0x7fc3381ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 -- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c88c0 msgr2=0x7fc3380ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c88c0 0x7fc3380ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 -- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3380c5880 msgr2=0x7fc3381ee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3380c5880 0x7fc3381ee780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 -- 172.21.15.26:0/1001211428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380b0560 con 0x7fc33814f360 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc336d9d640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33814f360 0x7fc338156ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc330036040 tx=0x7fc330004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc32effd640 1 -- 172.21.15.26:0/1001211428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330043020 con 0x7fc33814f360 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc32effd640 1 -- 172.21.15.26:0/1001211428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33003e450 con 0x7fc33814f360 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3380ac2d0 con 0x7fc33814f360 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc32effd640 1 -- 172.21.15.26:0/1001211428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330009510 con 0x7fc33814f360 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.495+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc338157d60 con 0x7fc33814f360 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.497+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33804f330 con 0x7fc33814f360 2024-03-06T07:21:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.497+0000 7fc32effd640 1 -- 172.21.15.26:0/1001211428 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc330046020 con 0x7fc33814f360 2024-03-06T07:21:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.497+0000 7fc32effd640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b4d0 0x7fc31806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.497+0000 7fc32effd640 1 -- 172.21.15.26:0/1001211428 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7015+0+0 (secure 0 0 0) 0x7fc3300acf00 con 0x7fc33814f360 2024-03-06T07:21:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.498+0000 7fc33659c640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b4d0 0x7fc31806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.498+0000 7fc33659c640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b4d0 0x7fc31806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc328006fd0 tx=0x7fc328008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.502+0000 7fc32effd640 1 -- 172.21.15.26:0/1001211428 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc33004d030 con 0x7fc33814f360 2024-03-06T07:21:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.817+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc33806cda0 con 0x7fc33814f360 2024-03-06T07:21:55.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.818+0000 7fc32effd640 1 -- 172.21.15.26:0/1001211428 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+20731 (secure 0 0 0) 0x7fc33007ba10 con 0x7fc33814f360 2024-03-06T07:21:55.858 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:55.858 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":55,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:54.755514+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:55.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b4d0 msgr2=0x7fc31806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b4d0 0x7fc31806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc328006fd0 tx=0x7fc328008040 comp rx=0 tx=0).stop 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33814f360 msgr2=0x7fc338156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33814f360 0x7fc338156ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc330036040 tx=0x7fc330004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 shutdown_connections 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b4d0 0x7fc31806d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33814f360 0x7fc338156ea0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c88c0 0x7fc3380ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 --2- 172.21.15.26:0/1001211428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3380c5880 0x7fc3381ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 >> 172.21.15.26:0/1001211428 conn(0x7fc3380ac570 msgr2=0x7fc3380ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 shutdown_connections 2024-03-06T07:21:55.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.821+0000 7fc33cb55640 1 -- 172.21.15.26:0/1001211428 wait complete. 2024-03-06T07:21:55.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.962+0000 7f69457c9640 1 Processor -- start 2024-03-06T07:21:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.962+0000 7f69457c9640 1 -- start start 2024-03-06T07:21:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f69457c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6940159100 0x7f69400c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f69457c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69400c7c60 con 0x7f69401527b0 2024-03-06T07:21:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f69457c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69400c8060 con 0x7f69400c7180 2024-03-06T07:21:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f69457c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400c8df0 con 0x7f6940159100 2024-03-06T07:21:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f69401527b0 0x7f6940155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38436/0 (socket says 172.21.15.26:38436) 2024-03-06T07:21:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693effd640 1 -- 172.21.15.26:0/264479040 learned_addr learned my addr 172.21.15.26:0/264479040 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693f7fe640 1 --2- 172.21.15.26:0/264479040 >> v2:172.21.15.117:3300/0 conn(0x7f6940159100 0x7f69400c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693dffb640 1 -- 172.21.15.26:0/264479040 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556821052 0 0) 0x7f6938001110 con 0x7f69401527b0 2024-03-06T07:21:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693dffb640 1 -- 172.21.15.26:0/264479040 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f692c003520 con 0x7f69401527b0 2024-03-06T07:21:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693f7fe640 1 -- 172.21.15.26:0/264479040 >> v1:172.21.15.117:6789/0 conn(0x7f69400c7180 legacy=0x7f69400c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693dffb640 1 -- 172.21.15.26:0/264479040 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059990505 0 0) 0x7f69400c8060 con 0x7f69400c7180 2024-03-06T07:21:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693f7fe640 1 -- 172.21.15.26:0/264479040 >> v1:172.21.15.26:6789/0 conn(0x7f69401527b0 legacy=0x7f6940155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.963+0000 7f693f7fe640 1 -- 172.21.15.26:0/264479040 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400c76a0 con 0x7f6940159100 2024-03-06T07:21:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.964+0000 7f693f7fe640 1 --2- 172.21.15.26:0/264479040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f69400c88b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6930009d90 tx=0x7f693002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d002e8c39e01bfc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.964+0000 7f693dffb640 1 -- 172.21.15.26:0/264479040 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930039070 con 0x7f6940159100 2024-03-06T07:21:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.964+0000 7f693dffb640 1 -- 172.21.15.26:0/264479040 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693002cc60 con 0x7f6940159100 2024-03-06T07:21:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.964+0000 7f693dffb640 1 -- 172.21.15.26:0/264479040 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930034640 con 0x7f6940159100 2024-03-06T07:21:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.965+0000 7f69457c9640 1 -- 172.21.15.26:0/264479040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 msgr2=0x7f69400c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.965+0000 7f69457c9640 1 --2- 172.21.15.26:0/264479040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f69400c88b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6930009d90 tx=0x7f693002beb0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.965+0000 7f69457c9640 1 -- 172.21.15.26:0/264479040 shutdown_connections 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.965+0000 7f69457c9640 1 --2- 172.21.15.26:0/264479040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f69400c88b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.965+0000 7f69457c9640 1 -- 172.21.15.26:0/264479040 >> 172.21.15.26:0/264479040 conn(0x7f69400ac560 msgr2=0x7f69400ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.965+0000 7f69457c9640 1 -- 172.21.15.26:0/264479040 shutdown_connections 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.965+0000 7f69457c9640 1 -- 172.21.15.26:0/264479040 wait complete. 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.967+0000 7f69457c9640 1 Processor -- start 2024-03-06T07:21:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.967+0000 7f69457c9640 1 -- start start 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f69457c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69400c7180 0x7f6940157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f69457c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69401527b0 0x7f69401ee620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f69457c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f6940154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f69457c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940155600 con 0x7f69400c7180 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f69457c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940155320 con 0x7f6940159100 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f69457c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6940152b90 con 0x7f69401527b0 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f693effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69400c7180 0x7f6940157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f693effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69400c7180 0x7f6940157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50222/0 (socket says 172.21.15.26:50222) 2024-03-06T07:21:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f693effd640 1 -- 172.21.15.26:0/3172751962 learned_addr learned my addr 172.21.15.26:0/3172751962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.968+0000 7f693f7fe640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f6940154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693e7fc640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69401527b0 0x7f69401ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693effd640 1 -- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69401527b0 msgr2=0x7f69401ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693effd640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69401527b0 0x7f69401ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693effd640 1 -- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 msgr2=0x7f6940154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693effd640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f6940154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693effd640 1 -- 172.21.15.26:0/3172751962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940157f10 con 0x7f69400c7180 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693f7fe640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f6940154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f693effd640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69400c7180 0x7f6940157170 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f693800c760 tx=0x7f693800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f69377fe640 1 -- 172.21.15.26:0/3172751962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938017070 con 0x7f69400c7180 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f69377fe640 1 -- 172.21.15.26:0/3172751962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6938005ad0 con 0x7f69400c7180 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6940156690 con 0x7f69400c7180 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f69377fe640 1 -- 172.21.15.26:0/3172751962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938007e20 con 0x7f69400c7180 2024-03-06T07:21:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.969+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6940156c50 con 0x7f69400c7180 2024-03-06T07:21:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.970+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694004f330 con 0x7f69400c7180 2024-03-06T07:21:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.970+0000 7f69377fe640 1 -- 172.21.15.26:0/3172751962 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f693801a070 con 0x7f69400c7180 2024-03-06T07:21:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.971+0000 7f69377fe640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692006b4d0 0x7f692006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.971+0000 7f69377fe640 1 -- 172.21.15.26:0/3172751962 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f6938089b10 con 0x7f69400c7180 2024-03-06T07:21:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.971+0000 7f693e7fc640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692006b4d0 0x7f692006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.971+0000 7f693e7fc640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692006b4d0 0x7f692006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f69400c5ae0 tx=0x7f6928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:55.975+0000 7f69377fe640 1 -- 172.21.15.26:0/3172751962 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f69400c7c60 con 0x7f69400c7180 2024-03-06T07:21:56.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.156+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f694006ce00 con 0x7f69400c7180 2024-03-06T07:21:56.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.157+0000 7f69377fe640 1 -- 172.21.15.26:0/3172751962 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+20731 (secure 0 0 0) 0x7f6938058470 con 0x7f69400c7180 2024-03-06T07:21:56.159 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:56.159 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":55,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:54.755514+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:56.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692006b4d0 msgr2=0x7f692006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692006b4d0 0x7f692006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f69400c5ae0 tx=0x7f6928009040 comp rx=0 tx=0).stop 2024-03-06T07:21:56.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69400c7180 msgr2=0x7f6940157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69400c7180 0x7f6940157170 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f693800c760 tx=0x7f693800cc20 comp rx=0 tx=0).stop 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 shutdown_connections 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692006b4d0 0x7f692006d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6940159100 0x7f6940154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69401527b0 0x7f69401ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 --2- 172.21.15.26:0/3172751962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69400c7180 0x7f6940157170 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.159+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 >> 172.21.15.26:0/3172751962 conn(0x7f69400ac560 msgr2=0x7f6940150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.160+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 shutdown_connections 2024-03-06T07:21:56.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.160+0000 7f69457c9640 1 -- 172.21.15.26:0/3172751962 wait complete. 2024-03-06T07:21:56.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-06T07:21:56.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.274+0000 7f8b76ac3640 1 Processor -- start 2024-03-06T07:21:56.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.274+0000 7f8b76ac3640 1 -- start start 2024-03-06T07:21:56.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b76ac3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8b700a8290 0x7f8b701539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b76ac3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700c62a0 con 0x7f8b700c4740 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b76ac3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700c4c00 con 0x7f8b700c69a0 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b76ac3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b70153f00 con 0x7f8b700a8290 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b6ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b700c4740 0x7f8b700701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38442/0 (socket says 172.21.15.26:38442) 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1023536055 learned_addr learned my addr 172.21.15.26:0/1023536055 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b7485f640 1 --2- 172.21.15.26:0/1023536055 >> v2:172.21.15.117:3300/0 conn(0x7f8b700a8290 0x7f8b701539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590756977 0 0) 0x7f8b700c62a0 con 0x7f8b700c4740 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.275+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b60003520 con 0x7f8b700c4740 2024-03-06T07:21:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572332246 0 0) 0x7f8b60003520 con 0x7f8b700c4740 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 >> v2:172.21.15.117:3300/0 conn(0x7f8b700a8290 msgr2=0x7f8b701539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 --2- 172.21.15.26:0/1023536055 >> v2:172.21.15.117:3300/0 conn(0x7f8b700a8290 0x7f8b701539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 >> v1:172.21.15.117:6789/0 conn(0x7f8b700c69a0 legacy=0x7f8b70072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700c4ee0 con 0x7f8b700c4740 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f8b5c001670 con 0x7f8b700c4740 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 >> v1:172.21.15.26:6789/0 conn(0x7f8b700c4740 legacy=0x7f8b700701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b600039b0 0x7f8b60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b60004800 0x7f8b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b600051c0 0x7f8b6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b70153f00 con 0x7f8b60004800 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6000c110 con 0x7f8b600051c0 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6000c310 con 0x7f8b600039b0 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b60004800 0x7f8b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b7485f640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b600039b0 0x7f8b60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b75060640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b600051c0 0x7f8b6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b600039b0 msgr2=0x7f8b60003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b600039b0 0x7f8b60003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b600051c0 msgr2=0x7f8b6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b600051c0 0x7f8b6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1023536055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60003520 con 0x7f8b60004800 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.276+0000 7f8b75060640 1 -- 172.21.15.26:0/1023536055 reap_dead start 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.277+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b60004800 0x7f8b60004c80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c210 tx=0x7f8b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd16e522dfe2fadc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.277+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c007ce0 con 0x7f8b60004800 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.277+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c030070 con 0x7f8b60004800 2024-03-06T07:21:56.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.277+0000 7f8b6f7fe640 1 -- 172.21.15.26:0/1023536055 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c005d70 con 0x7f8b60004800 2024-03-06T07:21:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.278+0000 7f8b76ac3640 1 -- 172.21.15.26:0/1023536055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b60004800 msgr2=0x7f8b60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.278+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b60004800 0x7f8b60004c80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c210 tx=0x7f8b60005830 comp rx=0 tx=0).stop 2024-03-06T07:21:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.278+0000 7f8b76ac3640 1 -- 172.21.15.26:0/1023536055 shutdown_connections 2024-03-06T07:21:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.278+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/1023536055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b60004800 0x7f8b60004c80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.278+0000 7f8b76ac3640 1 -- 172.21.15.26:0/1023536055 >> 172.21.15.26:0/1023536055 conn(0x7f8b700c5560 msgr2=0x7f8b700c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.278+0000 7f8b76ac3640 1 -- 172.21.15.26:0/1023536055 shutdown_connections 2024-03-06T07:21:56.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.278+0000 7f8b76ac3640 1 -- 172.21.15.26:0/1023536055 wait complete. 2024-03-06T07:21:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.279+0000 7f8b76ac3640 1 Processor -- start 2024-03-06T07:21:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.279+0000 7f8b76ac3640 1 -- start start 2024-03-06T07:21:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b60004800 0x7f8b701ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b701dcd80 0x7f8b701dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b701ebe40 0x7f8b701ec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c6df0 con 0x7f8b701dcd80 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7006fc80 con 0x7f8b60004800 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c6f70 con 0x7f8b701ebe40 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b701dcd80 0x7f8b701dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b701dcd80 0x7f8b701dd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50236/0 (socket says 172.21.15.26:50236) 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 -- 172.21.15.26:0/81676843 learned_addr learned my addr 172.21.15.26:0/81676843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 -- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b701ebe40 msgr2=0x7f8b701ec2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b75060640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b701ebe40 0x7f8b701ec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b7485f640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b60004800 0x7f8b701ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b701ebe40 0x7f8b701ec2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 -- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b60004800 msgr2=0x7f8b701ddc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b60004800 0x7f8b701ddc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 -- 172.21.15.26:0/81676843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b701dd720 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b701dcd80 0x7f8b701dd1e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c210 tx=0x7f8b5c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6d7fa640 1 -- 172.21.15.26:0/81676843 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c045070 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6d7fa640 1 -- 172.21.15.26:0/81676843 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c004bf0 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700ae540 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b6d7fa640 1 -- 172.21.15.26:0/81676843 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c001040 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.280+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700ae260 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.282+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7004f330 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.282+0000 7f8b6d7fa640 1 -- 172.21.15.26:0/81676843 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b5c007070 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.282+0000 7f8b6d7fa640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b5406b500 0x7f8b5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.282+0000 7f8b6d7fa640 1 -- 172.21.15.26:0/81676843 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f8b5c0ab670 con 0x7f8b701dcd80 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.283+0000 7f8b7485f640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b5406b500 0x7f8b5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.283+0000 7f8b7485f640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b5406b500 0x7f8b5406d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009a60 tx=0x7f8b64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.286+0000 7f8b6d7fa640 1 -- 172.21.15.26:0/81676843 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8b701dd720 con 0x7f8b701dcd80 2024-03-06T07:21:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:56.512+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8b700c61e0 con 0x7f8b701dcd80 2024-03-06T07:21:57.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.372+0000 7f8b6d7fa640 1 -- 172.21.15.26:0/81676843 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 119+0+0 (secure 0 0 0) 0x7f8b5c079f50 con 0x7f8b701dcd80 2024-03-06T07:21:57.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.374+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b5406b500 msgr2=0x7f8b5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.374+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b5406b500 0x7f8b5406d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009a60 tx=0x7f8b64008040 comp rx=0 tx=0).stop 2024-03-06T07:21:57.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.374+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b701dcd80 msgr2=0x7f8b701dd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.374+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b701dcd80 0x7f8b701dd1e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c210 tx=0x7f8b5c02e070 comp rx=0 tx=0).stop 2024-03-06T07:21:57.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 shutdown_connections 2024-03-06T07:21:57.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b5406b500 0x7f8b5406d9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b701ebe40 0x7f8b701ec2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b701dcd80 0x7f8b701dd1e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 --2- 172.21.15.26:0/81676843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b60004800 0x7f8b701ddc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 >> 172.21.15.26:0/81676843 conn(0x7f8b700c5560 msgr2=0x7f8b700c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 shutdown_connections 2024-03-06T07:21:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.375+0000 7f8b76ac3640 1 -- 172.21.15.26:0/81676843 wait complete. 2024-03-06T07:21:57.387 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:21:57.387 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-06T07:21:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.493+0000 7fd915bb5640 1 Processor -- start 2024-03-06T07:21:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd915bb5640 1 -- start start 2024-03-06T07:21:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd915bb5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9100b0070 0x7fd910155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd915bb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd9100c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd915bb5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9100b0450 con 0x7fd91006ca70 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd915bb5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91006c020 con 0x7fd9100b0070 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd915bb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91006cf20 con 0x7fd9100c5380 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd90f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9100b0070 0x7fd910155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd90f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9100b0070 0x7fd910155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57822/0 (socket says 172.21.15.26:57822) 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd90f7fe640 1 -- 172.21.15.26:0/126934131 learned_addr learned my addr 172.21.15.26:0/126934131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.494+0000 7fd90ffff640 1 --2- 172.21.15.26:0/126934131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd9100c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.495+0000 7fd90f7fe640 1 -- 172.21.15.26:0/126934131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 msgr2=0x7fd9100c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.495+0000 7fd90f7fe640 1 --2- 172.21.15.26:0/126934131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd9100c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.495+0000 7fd90f7fe640 1 -- 172.21.15.26:0/126934131 >> v1:172.21.15.117:6789/0 conn(0x7fd91006ca70 legacy=0x7fd9100c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.495+0000 7fd90f7fe640 1 -- 172.21.15.26:0/126934131 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100c77f0 con 0x7fd9100b0070 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.495+0000 7fd90f7fe640 1 --2- 172.21.15.26:0/126934131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b0070 0x7fd910155c30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd904009870 tx=0x7fd90402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656163ce243af3c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd90e7fc640 1 -- 172.21.15.26:0/126934131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904035020 con 0x7fd9100b0070 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd90e7fc640 1 -- 172.21.15.26:0/126934131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd904031640 con 0x7fd9100b0070 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd90e7fc640 1 -- 172.21.15.26:0/126934131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90402d040 con 0x7fd9100b0070 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 -- 172.21.15.26:0/126934131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b0070 msgr2=0x7fd910155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 --2- 172.21.15.26:0/126934131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b0070 0x7fd910155c30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd904009870 tx=0x7fd90402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 -- 172.21.15.26:0/126934131 shutdown_connections 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 --2- 172.21.15.26:0/126934131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd9100c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 --2- 172.21.15.26:0/126934131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b0070 0x7fd910155c30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 -- 172.21.15.26:0/126934131 >> 172.21.15.26:0/126934131 conn(0x7fd9100ac590 msgr2=0x7fd9100ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 -- 172.21.15.26:0/126934131 shutdown_connections 2024-03-06T07:21:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.496+0000 7fd915bb5640 1 -- 172.21.15.26:0/126934131 wait complete. 2024-03-06T07:21:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.497+0000 7fd915bb5640 1 Processor -- start 2024-03-06T07:21:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.497+0000 7fd915bb5640 1 -- start start 2024-03-06T07:21:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd915bb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd91006ca70 0x7fd9101ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd915bb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9100b0070 0x7fd91014ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd915bb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd91014bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd915bb5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91006c550 con 0x7fd91006ca70 2024-03-06T07:21:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd915bb5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91006c270 con 0x7fd9100b0070 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd915bb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91006da80 con 0x7fd9100c5380 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd91006ca70 0x7fd9101ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd91006ca70 0x7fd9101ec3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57834/0 (socket says 172.21.15.26:57834) 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd90f7fe640 1 -- 172.21.15.26:0/2664275409 learned_addr learned my addr 172.21.15.26:0/2664275409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd90effd640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9100b0070 0x7fd91014ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd90ffff640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd91014bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd90f7fe640 1 -- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 msgr2=0x7fd91014bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.498+0000 7fd90f7fe640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd91014bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd90f7fe640 1 -- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9100b0070 msgr2=0x7fd91014ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd90f7fe640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9100b0070 0x7fd91014ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd90f7fe640 1 -- 172.21.15.26:0/2664275409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100addc0 con 0x7fd91006ca70 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd90f7fe640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd91006ca70 0x7fd9101ec3b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd9040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd8f7fff640 1 -- 172.21.15.26:0/2664275409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904040070 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd8f7fff640 1 -- 172.21.15.26:0/2664275409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9040399f0 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd8f7fff640 1 -- 172.21.15.26:0/2664275409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904035040 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9101546c0 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.499+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd910153900 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.500+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9100678a0 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.500+0000 7fd8f7fff640 1 -- 172.21.15.26:0/2664275409 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd90404b060 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.501+0000 7fd8f7fff640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8fc06b4d0 0x7fd8fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.501+0000 7fd8f7fff640 1 -- 172.21.15.26:0/2664275409 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7fd9040abdf0 con 0x7fd91006ca70 2024-03-06T07:21:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.501+0000 7fd90effd640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8fc06b4d0 0x7fd8fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.501+0000 7fd90effd640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8fc06b4d0 0x7fd8fc06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd9100c16f0 tx=0x7fd900009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.505+0000 7fd8f7fff640 1 -- 172.21.15.26:0/2664275409 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd9100678a0 con 0x7fd91006ca70 2024-03-06T07:21:57.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:57.724+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd91006bda0 con 0x7fd91006ca70 2024-03-06T07:21:58.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.373+0000 7fd8f7fff640 1 -- 172.21.15.26:0/2664275409 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd91006bda0 con 0x7fd91006ca70 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8fc06b4d0 msgr2=0x7fd8fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8fc06b4d0 0x7fd8fc06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd9100c16f0 tx=0x7fd900009040 comp rx=0 tx=0).stop 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd91006ca70 msgr2=0x7fd9101ec3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd91006ca70 0x7fd9101ec3b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd9040046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 shutdown_connections 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8fc06b4d0 0x7fd8fc06d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9100c5380 0x7fd91014bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9100b0070 0x7fd91014ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 --2- 172.21.15.26:0/2664275409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd91006ca70 0x7fd9101ec3b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 >> 172.21.15.26:0/2664275409 conn(0x7fd9100ac590 msgr2=0x7fd9100c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 shutdown_connections 2024-03-06T07:21:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.375+0000 7fd915bb5640 1 -- 172.21.15.26:0/2664275409 wait complete. 2024-03-06T07:21:58.388 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-06T07:21:58.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.488+0000 7fbce55ac640 1 Processor -- start 2024-03-06T07:21:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.488+0000 7fbce55ac640 1 -- start start 2024-03-06T07:21:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbce55ac640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbce00c88c0 0x7fbce0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbce55ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbce0158b60 0x7fbce0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbce55ac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce00c73d0 con 0x7fbce014f360 2024-03-06T07:21:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbce55ac640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce00c8dd0 con 0x7fbce00c88c0 2024-03-06T07:21:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbce55ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce00be2e0 con 0x7fbce0158b60 2024-03-06T07:21:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcde7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbce00c88c0 0x7fbce0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcde7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbce00c88c0 0x7fbce0155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57846/0 (socket says 172.21.15.26:57846) 2024-03-06T07:21:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcde7fc640 1 -- 172.21.15.26:0/1107669169 learned_addr learned my addr 172.21.15.26:0/1107669169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcdf7fe640 1 --2- 172.21.15.26:0/1107669169 >> v2:172.21.15.117:3300/0 conn(0x7fbce0158b60 0x7fbce0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcde7fc640 1 -- 172.21.15.26:0/1107669169 >> v2:172.21.15.117:3300/0 conn(0x7fbce0158b60 msgr2=0x7fbce0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcde7fc640 1 --2- 172.21.15.26:0/1107669169 >> v2:172.21.15.117:3300/0 conn(0x7fbce0158b60 0x7fbce0158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcde7fc640 1 -- 172.21.15.26:0/1107669169 >> v1:172.21.15.117:6789/0 conn(0x7fbce014f360 legacy=0x7fbce00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.489+0000 7fbcde7fc640 1 -- 172.21.15.26:0/1107669169 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00c7670 con 0x7fbce00c88c0 2024-03-06T07:21:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.490+0000 7fbcde7fc640 1 --2- 172.21.15.26:0/1107669169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce0155c10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbce0152d60 tx=0x7fbcc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89e67724cbcfdb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.490+0000 7fbcddffb640 1 -- 172.21.15.26:0/1107669169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8035020 con 0x7fbce00c88c0 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.490+0000 7fbcddffb640 1 -- 172.21.15.26:0/1107669169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8031870 con 0x7fbce00c88c0 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.490+0000 7fbcddffb640 1 -- 172.21.15.26:0/1107669169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8031c40 con 0x7fbce00c88c0 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 -- 172.21.15.26:0/1107669169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 msgr2=0x7fbce0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1107669169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce0155c10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbce0152d60 tx=0x7fbcc802b910 comp rx=0 tx=0).stop 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 -- 172.21.15.26:0/1107669169 shutdown_connections 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1107669169 >> v2:172.21.15.117:3300/0 conn(0x7fbce0158b60 0x7fbce0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1107669169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce0155c10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 -- 172.21.15.26:0/1107669169 >> 172.21.15.26:0/1107669169 conn(0x7fbce00ac570 msgr2=0x7fbce00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 -- 172.21.15.26:0/1107669169 shutdown_connections 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.491+0000 7fbce55ac640 1 -- 172.21.15.26:0/1107669169 wait complete. 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.492+0000 7fbce55ac640 1 Processor -- start 2024-03-06T07:21:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.492+0000 7fbce55ac640 1 -- start start 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbce55ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce00ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbce55ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce014f360 0x7fbce00adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbce55ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce0158b60 0x7fbce01ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbce55ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c5a50 con 0x7fbce00c88c0 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbce55ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c7c30 con 0x7fbce0158b60 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbce55ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c5bd0 con 0x7fbce014f360 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce00ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce00ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57850/0 (socket says 172.21.15.26:57850) 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 -- 172.21.15.26:0/1084222676 learned_addr learned my addr 172.21.15.26:0/1084222676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdf7fe640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce0158b60 0x7fbce01ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcde7fc640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce014f360 0x7fbce00adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 -- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce014f360 msgr2=0x7fbce00adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce014f360 0x7fbce00adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 -- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce0158b60 msgr2=0x7fbce01ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce0158b60 0x7fbce01ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.493+0000 7fbcdeffd640 1 -- 172.21.15.26:0/1084222676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce014bba0 con 0x7fbce00c88c0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.494+0000 7fbcdeffd640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce00ad660 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000c400 tx=0x7fbcd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.494+0000 7fbcd77fe640 1 -- 172.21.15.26:0/1084222676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00052d0 con 0x7fbce00c88c0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.494+0000 7fbcd77fe640 1 -- 172.21.15.26:0/1084222676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd0005490 con 0x7fbce00c88c0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.494+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce0152b10 con 0x7fbce00c88c0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.494+0000 7fbcd77fe640 1 -- 172.21.15.26:0/1084222676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0007260 con 0x7fbce00c88c0 2024-03-06T07:21:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.494+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce014be40 con 0x7fbce00c88c0 2024-03-06T07:21:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.495+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce004f330 con 0x7fbce00c88c0 2024-03-06T07:21:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.495+0000 7fbcd77fe640 1 -- 172.21.15.26:0/1084222676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcd0007400 con 0x7fbce00c88c0 2024-03-06T07:21:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.496+0000 7fbcd77fe640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc006b500 0x7fbcc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.496+0000 7fbcd77fe640 1 -- 172.21.15.26:0/1084222676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7fbcd0089ac0 con 0x7fbce00c88c0 2024-03-06T07:21:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.496+0000 7fbcde7fc640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc006b500 0x7fbcc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.496+0000 7fbcde7fc640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc006b500 0x7fbcc006d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036000 tx=0x7fbcc803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.500+0000 7fbcd77fe640 1 -- 172.21.15.26:0/1084222676 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbcd0057e60 con 0x7fbce00c88c0 2024-03-06T07:21:58.605 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:21:58.603+0000 7f9723f83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:58.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.733+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbce006cda0 con 0x7fbce00c88c0 2024-03-06T07:21:58.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.734+0000 7fbcd77fe640 1 -- 172.21.15.26:0/1084222676 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6256 (secure 0 0 0) 0x7fbcd005ce20 con 0x7fbce00c88c0 2024-03-06T07:21:58.736 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.26:6834/1383022235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1383022235},{"type":"v1","addr":"172.21.15.26:6834","nonce":1383022235}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:21:45.677934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/440323369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":440323369},{"type":"v1","addr":"172.21.15.26:6835","nonce":440323369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:21:58.736 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 34 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc006b500 msgr2=0x7fbcc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc006b500 0x7fbcc006d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036000 tx=0x7fbcc803c040 comp rx=0 tx=0).stop 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 msgr2=0x7fbce00ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce00ad660 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000c400 tx=0x7fbcd000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 shutdown_connections 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc006b500 0x7fbcc006d9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce0158b60 0x7fbce01ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce014f360 0x7fbce00adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 --2- 172.21.15.26:0/1084222676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce00c88c0 0x7fbce00ad660 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 >> 172.21.15.26:0/1084222676 conn(0x7fbce00ac570 msgr2=0x7fbce014cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:58.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 shutdown_connections 2024-03-06T07:21:58.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.736+0000 7fbce55ac640 1 -- 172.21.15.26:0/1084222676 wait complete. 2024-03-06T07:21:58.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:58.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.852+0000 7f877b65c640 1 Processor -- start 2024-03-06T07:21:58.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.853+0000 7f877b65c640 1 -- start start 2024-03-06T07:21:58.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.853+0000 7f877b65c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87740c7180 0x7f8774159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.853+0000 7f877b65c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87740c88b0 0x7f87740c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.853+0000 7f877b65c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87740c7c60 con 0x7f87741527b0 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.853+0000 7f877b65c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8774159640 con 0x7f87740c7180 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.853+0000 7f877b65c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740c8060 con 0x7f87740c88b0 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8778bf7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87740c7180 0x7f8774159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8779bf9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87740c88b0 0x7f87740c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8778bf7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87740c7180 0x7f8774159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:46790/0 (socket says 172.21.15.26:46790) 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f87793f8640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f87741527b0 0x7f8774155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:43860/0 (socket says 172.21.15.26:43860) 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8778bf7640 1 -- 172.21.15.26:0/805215623 learned_addr learned my addr 172.21.15.26:0/805215623 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8778bf7640 1 -- 172.21.15.26:0/805215623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87740c88b0 msgr2=0x7f87740c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8778bf7640 1 --2- 172.21.15.26:0/805215623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87740c88b0 0x7f87740c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8778bf7640 1 -- 172.21.15.26:0/805215623 >> v1:172.21.15.117:6789/0 conn(0x7f87741527b0 legacy=0x7f8774155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.854+0000 7f8778bf7640 1 -- 172.21.15.26:0/805215623 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740c76a0 con 0x7f87740c7180 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f8778bf7640 1 --2- 172.21.15.26:0/805215623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c7180 0x7f8774159100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8768009870 tx=0x7f876802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c1d7dfc981c2a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f8773fff640 1 -- 172.21.15.26:0/805215623 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768035020 con 0x7f87740c7180 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f8773fff640 1 -- 172.21.15.26:0/805215623 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768031640 con 0x7f87740c7180 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f8773fff640 1 -- 172.21.15.26:0/805215623 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876802d040 con 0x7f87740c7180 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f877b65c640 1 -- 172.21.15.26:0/805215623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c7180 msgr2=0x7f8774159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f877b65c640 1 --2- 172.21.15.26:0/805215623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c7180 0x7f8774159100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8768009870 tx=0x7f876802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f877b65c640 1 -- 172.21.15.26:0/805215623 shutdown_connections 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f877b65c640 1 --2- 172.21.15.26:0/805215623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87740c88b0 0x7f87740c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f877b65c640 1 --2- 172.21.15.26:0/805215623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c7180 0x7f8774159100 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.855+0000 7f877b65c640 1 -- 172.21.15.26:0/805215623 >> 172.21.15.26:0/805215623 conn(0x7f87740ac560 msgr2=0x7f87740ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:58.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.856+0000 7f877b65c640 1 -- 172.21.15.26:0/805215623 shutdown_connections 2024-03-06T07:21:58.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.856+0000 7f877b65c640 1 -- 172.21.15.26:0/805215623 wait complete. 2024-03-06T07:21:58.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 Processor -- start 2024-03-06T07:21:58.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 -- start start 2024-03-06T07:21:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87740c7180 0x7f877414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c88b0 0x7f877414bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87741527b0 0x7f87741ee450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8774155600 con 0x7f87740c7180 2024-03-06T07:21:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8774155320 con 0x7f87740c88b0 2024-03-06T07:21:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f877b65c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740c8d60 con 0x7f87741527b0 2024-03-06T07:21:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f87793f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87740c7180 0x7f877414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f87793f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87740c7180 0x7f877414ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57854/0 (socket says 172.21.15.26:57854) 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.857+0000 7f87793f8640 1 -- 172.21.15.26:0/3216466842 learned_addr learned my addr 172.21.15.26:0/3216466842 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f8779bf9640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87741527b0 0x7f87741ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f8778bf7640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c88b0 0x7f877414bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f87793f8640 1 -- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87741527b0 msgr2=0x7f87741ee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f87793f8640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87741527b0 0x7f87741ee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f87793f8640 1 -- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c88b0 msgr2=0x7f877414bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f87793f8640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c88b0 0x7f877414bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f87793f8640 1 -- 172.21.15.26:0/3216466842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740c16b0 con 0x7f87740c7180 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f87793f8640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87740c7180 0x7f877414ba40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f876400c760 tx=0x7f876400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f8771ffb640 1 -- 172.21.15.26:0/3216466842 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764017070 con 0x7f87740c7180 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f8771ffb640 1 -- 172.21.15.26:0/3216466842 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876400a640 con 0x7f87740c7180 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87740c10f0 con 0x7f87740c7180 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f8771ffb640 1 -- 172.21.15.26:0/3216466842 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400aa10 con 0x7f87740c7180 2024-03-06T07:21:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.858+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87740c2b30 con 0x7f87740c7180 2024-03-06T07:21:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.859+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877404f330 con 0x7f87740c7180 2024-03-06T07:21:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.860+0000 7f8771ffb640 1 -- 172.21.15.26:0/3216466842 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f876401a070 con 0x7f87740c7180 2024-03-06T07:21:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.860+0000 7f8771ffb640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876006b4d0 0x7f876006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.860+0000 7f8771ffb640 1 -- 172.21.15.26:0/3216466842 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f8764089cf0 con 0x7f87740c7180 2024-03-06T07:21:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.860+0000 7f8778bf7640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876006b4d0 0x7f876006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.861+0000 7f8778bf7640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876006b4d0 0x7f876006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8768036000 tx=0x7f876803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:58.864+0000 7f8771ffb640 1 -- 172.21.15.26:0/3216466842 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f876408e070 con 0x7f87740c7180 2024-03-06T07:21:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.049+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f877406ce00 con 0x7f87740c7180 2024-03-06T07:21:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.051+0000 7f8771ffb640 1 -- 172.21.15.26:0/3216466842 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+20718 (secure 0 0 0) 0x7f87740c7c60 con 0x7f87740c7180 2024-03-06T07:21:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:59.053 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":56,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:21:56.354998+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876006b4d0 msgr2=0x7f876006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876006b4d0 0x7f876006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8768036000 tx=0x7f876803c040 comp rx=0 tx=0).stop 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87740c7180 msgr2=0x7f877414ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87740c7180 0x7f877414ba40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f876400c760 tx=0x7f876400cc20 comp rx=0 tx=0).stop 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 shutdown_connections 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876006b4d0 0x7f876006d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87741527b0 0x7f87741ee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87740c88b0 0x7f877414bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 --2- 172.21.15.26:0/3216466842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87740c7180 0x7f877414ba40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 >> 172.21.15.26:0/3216466842 conn(0x7f87740ac560 msgr2=0x7f8774158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 shutdown_connections 2024-03-06T07:21:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.053+0000 7f877b65c640 1 -- 172.21.15.26:0/3216466842 wait complete. 2024-03-06T07:21:59.065 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:21:59.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:59.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.169+0000 7ff7c3913640 1 Processor -- start 2024-03-06T07:21:59.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.169+0000 7ff7c3913640 1 -- start start 2024-03-06T07:21:59.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c3913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c3913640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7bc0c88c0 0x7ff7bc0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c3913640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7bc0c73d0 con 0x7ff7bc0c5880 2024-03-06T07:21:59.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c3913640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7bc0c8dd0 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c3913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0be2e0 con 0x7ff7bc14f360 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c0eae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7bc0c88c0 0x7ff7bc0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c0eae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7bc0c88c0 0x7ff7bc0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57870/0 (socket says 172.21.15.26:57870) 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c0eae640 1 -- 172.21.15.26:0/1350486209 learned_addr learned my addr 172.21.15.26:0/1350486209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.170+0000 7ff7c16af640 1 --2- 172.21.15.26:0/1350486209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.171+0000 7ff7c16af640 1 -- 172.21.15.26:0/1350486209 >> v2:172.21.15.26:3300/0 conn(0x7ff7bc0c88c0 msgr2=0x7ff7bc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.171+0000 7ff7c16af640 1 --2- 172.21.15.26:0/1350486209 >> v2:172.21.15.26:3300/0 conn(0x7ff7bc0c88c0 0x7ff7bc0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.171+0000 7ff7affff640 1 -- 172.21.15.26:0/1350486209 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999979469 0 0) 0x7ff7bc0c73d0 con 0x7ff7bc0c5880 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.171+0000 7ff7c16af640 1 -- 172.21.15.26:0/1350486209 >> v1:172.21.15.117:6789/0 conn(0x7ff7bc0c5880 legacy=0x7ff7bc158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.171+0000 7ff7c16af640 1 -- 172.21.15.26:0/1350486209 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0c7670 con 0x7ff7bc14f360 2024-03-06T07:21:59.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.171+0000 7ff7c0eae640 1 --2- 172.21.15.26:0/1350486209 >> v2:172.21.15.26:3300/0 conn(0x7ff7bc0c88c0 0x7ff7bc0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:59.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.171+0000 7ff7c16af640 1 --2- 172.21.15.26:0/1350486209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc155c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80097a0 tx=0x7ff7b802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d0337284832f0b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7affff640 1 -- 172.21.15.26:0/1350486209 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8031780 con 0x7ff7bc14f360 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7affff640 1 -- 172.21.15.26:0/1350486209 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b8037070 con 0x7ff7bc14f360 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7affff640 1 -- 172.21.15.26:0/1350486209 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8031b40 con 0x7ff7bc14f360 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 -- 172.21.15.26:0/1350486209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 msgr2=0x7ff7bc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1350486209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc155c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80097a0 tx=0x7ff7b802b820 comp rx=0 tx=0).stop 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 -- 172.21.15.26:0/1350486209 shutdown_connections 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1350486209 >> v2:172.21.15.26:3300/0 conn(0x7ff7bc0c88c0 0x7ff7bc0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1350486209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc155c10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 -- 172.21.15.26:0/1350486209 >> 172.21.15.26:0/1350486209 conn(0x7ff7bc0ac570 msgr2=0x7ff7bc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 -- 172.21.15.26:0/1350486209 shutdown_connections 2024-03-06T07:21:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.172+0000 7ff7c3913640 1 -- 172.21.15.26:0/1350486209 wait complete. 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.174+0000 7ff7c3913640 1 Processor -- start 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.174+0000 7ff7c3913640 1 -- start start 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c3913640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7bc0c5880 0x7ff7bc1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c3913640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7bc0c88c0 0x7ff7bc1ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c3913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc1e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c3913640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc159270 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c3913640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0c7c30 con 0x7ff7bc0c5880 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c3913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc1593f0 con 0x7ff7bc14f360 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7bc0c88c0 0x7ff7bc1ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7bc0c88c0 0x7ff7bc1ebd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57872/0 (socket says 172.21.15.26:57872) 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c16af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7bc0c5880 0x7ff7bc1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 -- 172.21.15.26:0/1583370088 learned_addr learned my addr 172.21.15.26:0/1583370088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c1eb0640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc1e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 -- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 msgr2=0x7ff7bc1e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc1e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 -- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7bc0c5880 msgr2=0x7ff7bc1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7bc0c5880 0x7ff7bc1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 -- 172.21.15.26:0/1583370088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0c17e0 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7c0eae640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7bc0c88c0 0x7ff7bc1ebd10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000e9d0 tx=0x7ff7b000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.175+0000 7ff7adffb640 1 -- 172.21.15.26:0/1583370088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0017070 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.176+0000 7ff7adffb640 1 -- 172.21.15.26:0/1583370088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b000ca60 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.176+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc1eb550 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.176+0000 7ff7adffb640 1 -- 172.21.15.26:0/1583370088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000ce10 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.176+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc1ee220 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.177+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc14d0c0 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.177+0000 7ff7adffb640 1 -- 172.21.15.26:0/1583370088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7b001a070 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.178+0000 7ff7adffb640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7a006b500 0x7ff7a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.178+0000 7ff7adffb640 1 -- 172.21.15.26:0/1583370088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7ff7b008a560 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.178+0000 7ff7c16af640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7a006b500 0x7ff7a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.178+0000 7ff7c16af640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7a006b500 0x7ff7a006d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8036000 tx=0x7ff7b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.182+0000 7ff7adffb640 1 -- 172.21.15.26:0/1583370088 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7bc14d0c0 con 0x7ff7bc0c88c0 2024-03-06T07:21:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.365+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7bc1eb300 con 0x7ff7a006b500 2024-03-06T07:21:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.388+0000 7ff7adffb640 1 -- 172.21.15.26:0/1583370088 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485262 (secure 0 0 0) 0x7ff7bc1eb300 con 0x7ff7a006b500 2024-03-06T07:21:59.395 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:59.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7a006b500 msgr2=0x7ff7a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7a006b500 0x7ff7a006d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8036000 tx=0x7ff7b8008030 comp rx=0 tx=0).stop 2024-03-06T07:21:59.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7bc0c88c0 msgr2=0x7ff7bc1ebd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7bc0c88c0 0x7ff7bc1ebd10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000e9d0 tx=0x7ff7b000ee90 comp rx=0 tx=0).stop 2024-03-06T07:21:59.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 shutdown_connections 2024-03-06T07:21:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7a006b500 0x7ff7a006d9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7bc14f360 0x7ff7bc1e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7bc0c88c0 0x7ff7bc1ebd10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 --2- 172.21.15.26:0/1583370088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7bc0c5880 0x7ff7bc1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 >> 172.21.15.26:0/1583370088 conn(0x7ff7bc0ac570 msgr2=0x7ff7bc14e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 shutdown_connections 2024-03-06T07:21:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.392+0000 7ff7c3913640 1 -- 172.21.15.26:0/1583370088 wait complete. 2024-03-06T07:21:59.404 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-06T07:21:59.053948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237996,"kb_used_data":5180,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736724,"statfs":{"total":773094113280,"available":772850405376,"internally_reserved":0,"allocated":5304320,"data_stored":3492128,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703896"},"pg_stats":[{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359676+0000","last_change":"2024-03-06T07:21:54.359676+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359676+0000","last_undegraded":"2024-03-06T07:21:54.359676+0000","last_fullsized":"2024-03-06T07:21:54.359676+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372914+0000","last_change":"2024-03-06T07:21:43.317811+0000","last_active":"2024-03-06T07:21:54.372914+0000","last_peered":"2024-03-06T07:21:54.372914+0000","last_clean":"2024-03-06T07:21:54.372914+0000","last_became_active":"2024-03-06T07:21:43.317509+0000","last_became_peered":"2024-03-06T07:21:43.317509+0000","last_unstale":"2024-03-06T07:21:54.372914+0000","last_undegraded":"2024-03-06T07:21:54.372914+0000","last_fullsized":"2024-03-06T07:21:54.372914+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364197+0000","last_change":"2024-03-06T07:21:54.364197+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364197+0000","last_undegraded":"2024-03-06T07:21:54.364197+0000","last_fullsized":"2024-03-06T07:21:54.364197+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357765+0000","last_change":"2024-03-06T07:21:43.317801+0000","last_active":"2024-03-06T07:21:54.357765+0000","last_peered":"2024-03-06T07:21:54.357765+0000","last_clean":"2024-03-06T07:21:54.357765+0000","last_became_active":"2024-03-06T07:21:43.317522+0000","last_became_peered":"2024-03-06T07:21:43.317522+0000","last_unstale":"2024-03-06T07:21:54.357765+0000","last_undegraded":"2024-03-06T07:21:54.357765+0000","last_fullsized":"2024-03-06T07:21:54.357765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.368156+0000","last_change":"2024-03-06T07:21:54.368156+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.368156+0000","last_undegraded":"2024-03-06T07:21:54.368156+0000","last_fullsized":"2024-03-06T07:21:54.368156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369212+0000","last_change":"2024-03-06T07:21:43.328071+0000","last_active":"2024-03-06T07:21:54.369212+0000","last_peered":"2024-03-06T07:21:54.369212+0000","last_clean":"2024-03-06T07:21:54.369212+0000","last_became_active":"2024-03-06T07:21:43.327781+0000","last_became_peered":"2024-03-06T07:21:43.327781+0000","last_unstale":"2024-03-06T07:21:54.369212+0000","last_undegraded":"2024-03-06T07:21:54.369212+0000","last_fullsized":"2024-03-06T07:21:54.369212+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365661+0000","last_change":"2024-03-06T07:21:54.365661+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365661+0000","last_undegraded":"2024-03-06T07:21:54.365661+0000","last_fullsized":"2024-03-06T07:21:54.365661+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357083+0000","last_change":"2024-03-06T07:21:43.326286+0000","last_active":"2024-03-06T07:21:54.357083+0000","last_peered":"2024-03-06T07:21:54.357083+0000","last_clean":"2024-03-06T07:21:54.357083+0000","last_became_active":"2024-03-06T07:21:43.326031+0000","last_became_peered":"2024-03-06T07:21:43.326031+0000","last_unstale":"2024-03-06T07:21:54.357083+0000","last_undegraded":"2024-03-06T07:21:54.357083+0000","last_fullsized":"2024-03-06T07:21:54.357083+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:21.645487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361397+0000","last_change":"2024-03-06T07:21:54.361397+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361397+0000","last_undegraded":"2024-03-06T07:21:54.361397+0000","last_fullsized":"2024-03-06T07:21:54.361397+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369745+0000","last_change":"2024-03-06T07:21:43.321810+0000","last_active":"2024-03-06T07:21:54.369745+0000","last_peered":"2024-03-06T07:21:54.369745+0000","last_clean":"2024-03-06T07:21:54.369745+0000","last_became_active":"2024-03-06T07:21:43.321538+0000","last_became_peered":"2024-03-06T07:21:43.321538+0000","last_unstale":"2024-03-06T07:21:54.369745+0000","last_undegraded":"2024-03-06T07:21:54.369745+0000","last_fullsized":"2024-03-06T07:21:54.369745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:10.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359185+0000","last_change":"2024-03-06T07:21:54.359185+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359185+0000","last_undegraded":"2024-03-06T07:21:54.359185+0000","last_fullsized":"2024-03-06T07:21:54.359185+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368491+0000","last_change":"2024-03-06T07:21:43.318713+0000","last_active":"2024-03-06T07:21:54.368491+0000","last_peered":"2024-03-06T07:21:54.368491+0000","last_clean":"2024-03-06T07:21:54.368491+0000","last_became_active":"2024-03-06T07:21:43.318385+0000","last_became_peered":"2024-03-06T07:21:43.318385+0000","last_unstale":"2024-03-06T07:21:54.368491+0000","last_undegraded":"2024-03-06T07:21:54.368491+0000","last_fullsized":"2024-03-06T07:21:54.368491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361827+0000","last_change":"2024-03-06T07:21:54.361827+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361827+0000","last_undegraded":"2024-03-06T07:21:54.361827+0000","last_fullsized":"2024-03-06T07:21:54.361827+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368595+0000","last_change":"2024-03-06T07:21:43.327099+0000","last_active":"2024-03-06T07:21:54.368595+0000","last_peered":"2024-03-06T07:21:54.368595+0000","last_clean":"2024-03-06T07:21:54.368595+0000","last_became_active":"2024-03-06T07:21:43.326851+0000","last_became_peered":"2024-03-06T07:21:43.326851+0000","last_unstale":"2024-03-06T07:21:54.368595+0000","last_undegraded":"2024-03-06T07:21:54.368595+0000","last_fullsized":"2024-03-06T07:21:54.368595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360705+0000","last_change":"2024-03-06T07:21:54.360705+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360705+0000","last_undegraded":"2024-03-06T07:21:54.360705+0000","last_fullsized":"2024-03-06T07:21:54.360705+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358329+0000","last_change":"2024-03-06T07:21:43.325897+0000","last_active":"2024-03-06T07:21:54.358329+0000","last_peered":"2024-03-06T07:21:54.358329+0000","last_clean":"2024-03-06T07:21:54.358329+0000","last_became_active":"2024-03-06T07:21:43.325647+0000","last_became_peered":"2024-03-06T07:21:43.325647+0000","last_unstale":"2024-03-06T07:21:54.358329+0000","last_undegraded":"2024-03-06T07:21:54.358329+0000","last_fullsized":"2024-03-06T07:21:54.358329+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365629+0000","last_change":"2024-03-06T07:21:54.365629+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365629+0000","last_undegraded":"2024-03-06T07:21:54.365629+0000","last_fullsized":"2024-03-06T07:21:54.365629+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359703+0000","last_change":"2024-03-06T07:21:43.330337+0000","last_active":"2024-03-06T07:21:54.359703+0000","last_peered":"2024-03-06T07:21:54.359703+0000","last_clean":"2024-03-06T07:21:54.359703+0000","last_became_active":"2024-03-06T07:21:43.330053+0000","last_became_peered":"2024-03-06T07:21:43.330053+0000","last_unstale":"2024-03-06T07:21:54.359703+0000","last_undegraded":"2024-03-06T07:21:54.359703+0000","last_fullsized":"2024-03-06T07:21:54.359703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358522+0000","last_change":"2024-03-06T07:21:54.358522+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358522+0000","last_undegraded":"2024-03-06T07:21:54.358522+0000","last_fullsized":"2024-03-06T07:21:54.358522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370017+0000","last_change":"2024-03-06T07:21:43.325100+0000","last_active":"2024-03-06T07:21:54.370017+0000","last_peered":"2024-03-06T07:21:54.370017+0000","last_clean":"2024-03-06T07:21:54.370017+0000","last_became_active":"2024-03-06T07:21:43.324812+0000","last_became_peered":"2024-03-06T07:21:43.324812+0000","last_unstale":"2024-03-06T07:21:54.370017+0000","last_undegraded":"2024-03-06T07:21:54.370017+0000","last_fullsized":"2024-03-06T07:21:54.370017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:11.442368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359265+0000","last_change":"2024-03-06T07:21:54.359265+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359265+0000","last_undegraded":"2024-03-06T07:21:54.359265+0000","last_fullsized":"2024-03-06T07:21:54.359265+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366884+0000","last_change":"2024-03-06T07:21:43.322188+0000","last_active":"2024-03-06T07:21:54.366884+0000","last_peered":"2024-03-06T07:21:54.366884+0000","last_clean":"2024-03-06T07:21:54.366884+0000","last_became_active":"2024-03-06T07:21:43.321685+0000","last_became_peered":"2024-03-06T07:21:43.321685+0000","last_unstale":"2024-03-06T07:21:54.366884+0000","last_undegraded":"2024-03-06T07:21:54.366884+0000","last_fullsized":"2024-03-06T07:21:54.366884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:54.799043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359570+0000","last_change":"2024-03-06T07:21:54.359570+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359570+0000","last_undegraded":"2024-03-06T07:21:54.359570+0000","last_fullsized":"2024-03-06T07:21:54.359570+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359503+0000","last_change":"2024-03-06T07:21:43.321952+0000","last_active":"2024-03-06T07:21:54.359503+0000","last_peered":"2024-03-06T07:21:54.359503+0000","last_clean":"2024-03-06T07:21:54.359503+0000","last_became_active":"2024-03-06T07:21:43.321579+0000","last_became_peered":"2024-03-06T07:21:43.321579+0000","last_unstale":"2024-03-06T07:21:54.359503+0000","last_undegraded":"2024-03-06T07:21:54.359503+0000","last_fullsized":"2024-03-06T07:21:54.359503+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:04.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359648+0000","last_change":"2024-03-06T07:21:54.359648+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359648+0000","last_undegraded":"2024-03-06T07:21:54.359648+0000","last_fullsized":"2024-03-06T07:21:54.359648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368337+0000","last_change":"2024-03-06T07:21:43.321277+0000","last_active":"2024-03-06T07:21:54.368337+0000","last_peered":"2024-03-06T07:21:54.368337+0000","last_clean":"2024-03-06T07:21:54.368337+0000","last_became_active":"2024-03-06T07:21:43.320973+0000","last_became_peered":"2024-03-06T07:21:43.320973+0000","last_unstale":"2024-03-06T07:21:54.368337+0000","last_undegraded":"2024-03-06T07:21:54.368337+0000","last_fullsized":"2024-03-06T07:21:54.368337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:15.957299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362294+0000","last_change":"2024-03-06T07:21:54.362294+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362294+0000","last_undegraded":"2024-03-06T07:21:54.362294+0000","last_fullsized":"2024-03-06T07:21:54.362294+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364199+0000","last_change":"2024-03-06T07:21:50.373762+0000","last_active":"2024-03-06T07:21:54.364199+0000","last_peered":"2024-03-06T07:21:54.364199+0000","last_clean":"2024-03-06T07:21:54.364199+0000","last_became_active":"2024-03-06T07:21:50.373521+0000","last_became_peered":"2024-03-06T07:21:50.373521+0000","last_unstale":"2024-03-06T07:21:54.364199+0000","last_undegraded":"2024-03-06T07:21:54.364199+0000","last_fullsized":"2024-03-06T07:21:54.364199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359359+0000","last_change":"2024-03-06T07:21:54.359359+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359359+0000","last_undegraded":"2024-03-06T07:21:54.359359+0000","last_fullsized":"2024-03-06T07:21:54.359359+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357693+0000","last_change":"2024-03-06T07:21:43.321137+0000","last_active":"2024-03-06T07:21:54.357693+0000","last_peered":"2024-03-06T07:21:54.357693+0000","last_clean":"2024-03-06T07:21:54.357693+0000","last_became_active":"2024-03-06T07:21:43.320820+0000","last_became_peered":"2024-03-06T07:21:43.320820+0000","last_unstale":"2024-03-06T07:21:54.357693+0000","last_undegraded":"2024-03-06T07:21:54.357693+0000","last_fullsized":"2024-03-06T07:21:54.357693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:52.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359065+0000","last_change":"2024-03-06T07:21:54.359065+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359065+0000","last_undegraded":"2024-03-06T07:21:54.359065+0000","last_fullsized":"2024-03-06T07:21:54.359065+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366975+0000","last_change":"2024-03-06T07:21:43.320518+0000","last_active":"2024-03-06T07:21:54.366975+0000","last_peered":"2024-03-06T07:21:54.366975+0000","last_clean":"2024-03-06T07:21:54.366975+0000","last_became_active":"2024-03-06T07:21:43.320178+0000","last_became_peered":"2024-03-06T07:21:43.320178+0000","last_unstale":"2024-03-06T07:21:54.366975+0000","last_undegraded":"2024-03-06T07:21:54.366975+0000","last_fullsized":"2024-03-06T07:21:54.366975+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360235+0000","last_change":"2024-03-06T07:21:54.360235+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360235+0000","last_undegraded":"2024-03-06T07:21:54.360235+0000","last_fullsized":"2024-03-06T07:21:54.360235+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370182+0000","last_change":"2024-03-06T07:21:43.324279+0000","last_active":"2024-03-06T07:21:54.370182+0000","last_peered":"2024-03-06T07:21:54.370182+0000","last_clean":"2024-03-06T07:21:54.370182+0000","last_became_active":"2024-03-06T07:21:43.323499+0000","last_became_peered":"2024-03-06T07:21:43.323499+0000","last_unstale":"2024-03-06T07:21:54.370182+0000","last_undegraded":"2024-03-06T07:21:54.370182+0000","last_fullsized":"2024-03-06T07:21:54.370182+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359045+0000","last_change":"2024-03-06T07:21:54.359045+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359045+0000","last_undegraded":"2024-03-06T07:21:54.359045+0000","last_fullsized":"2024-03-06T07:21:54.359045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357861+0000","last_change":"2024-03-06T07:21:43.318202+0000","last_active":"2024-03-06T07:21:54.357861+0000","last_peered":"2024-03-06T07:21:54.357861+0000","last_clean":"2024-03-06T07:21:54.357861+0000","last_became_active":"2024-03-06T07:21:43.317929+0000","last_became_peered":"2024-03-06T07:21:43.317929+0000","last_unstale":"2024-03-06T07:21:54.357861+0000","last_undegraded":"2024-03-06T07:21:54.357861+0000","last_fullsized":"2024-03-06T07:21:54.357861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367783+0000","last_change":"2024-03-06T07:21:54.367783+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367783+0000","last_undegraded":"2024-03-06T07:21:54.367783+0000","last_fullsized":"2024-03-06T07:21:54.367783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369274+0000","last_change":"2024-03-06T07:21:43.324265+0000","last_active":"2024-03-06T07:21:54.369274+0000","last_peered":"2024-03-06T07:21:54.369274+0000","last_clean":"2024-03-06T07:21:54.369274+0000","last_became_active":"2024-03-06T07:21:43.323877+0000","last_became_peered":"2024-03-06T07:21:43.323877+0000","last_unstale":"2024-03-06T07:21:54.369274+0000","last_undegraded":"2024-03-06T07:21:54.369274+0000","last_fullsized":"2024-03-06T07:21:54.369274+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367102+0000","last_change":"2024-03-06T07:21:54.367102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367102+0000","last_undegraded":"2024-03-06T07:21:54.367102+0000","last_fullsized":"2024-03-06T07:21:54.367102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365314+0000","last_change":"2024-03-06T07:21:43.321580+0000","last_active":"2024-03-06T07:21:54.365314+0000","last_peered":"2024-03-06T07:21:54.365314+0000","last_clean":"2024-03-06T07:21:54.365314+0000","last_became_active":"2024-03-06T07:21:43.319775+0000","last_became_peered":"2024-03-06T07:21:43.319775+0000","last_unstale":"2024-03-06T07:21:54.365314+0000","last_undegraded":"2024-03-06T07:21:54.365314+0000","last_fullsized":"2024-03-06T07:21:54.365314+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:32.802384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358875+0000","last_change":"2024-03-06T07:21:54.358875+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358875+0000","last_undegraded":"2024-03-06T07:21:54.358875+0000","last_fullsized":"2024-03-06T07:21:54.358875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365089+0000","last_change":"2024-03-06T07:21:43.322797+0000","last_active":"2024-03-06T07:21:54.365089+0000","last_peered":"2024-03-06T07:21:54.365089+0000","last_clean":"2024-03-06T07:21:54.365089+0000","last_became_active":"2024-03-06T07:21:43.322505+0000","last_became_peered":"2024-03-06T07:21:43.322505+0000","last_unstale":"2024-03-06T07:21:54.365089+0000","last_undegraded":"2024-03-06T07:21:54.365089+0000","last_fullsized":"2024-03-06T07:21:54.365089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359127+0000","last_change":"2024-03-06T07:21:54.359127+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359127+0000","last_undegraded":"2024-03-06T07:21:54.359127+0000","last_fullsized":"2024-03-06T07:21:54.359127+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358833+0000","last_change":"2024-03-06T07:21:43.326058+0000","last_active":"2024-03-06T07:21:54.358833+0000","last_peered":"2024-03-06T07:21:54.358833+0000","last_clean":"2024-03-06T07:21:54.358833+0000","last_became_active":"2024-03-06T07:21:43.325799+0000","last_became_peered":"2024-03-06T07:21:43.325799+0000","last_unstale":"2024-03-06T07:21:54.358833+0000","last_undegraded":"2024-03-06T07:21:54.358833+0000","last_fullsized":"2024-03-06T07:21:54.358833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357760+0000","last_change":"2024-03-06T07:21:54.357760+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357760+0000","last_undegraded":"2024-03-06T07:21:54.357760+0000","last_fullsized":"2024-03-06T07:21:54.357760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365258+0000","last_change":"2024-03-06T07:21:43.322671+0000","last_active":"2024-03-06T07:21:54.365258+0000","last_peered":"2024-03-06T07:21:54.365258+0000","last_clean":"2024-03-06T07:21:54.365258+0000","last_became_active":"2024-03-06T07:21:43.322376+0000","last_became_peered":"2024-03-06T07:21:43.322376+0000","last_unstale":"2024-03-06T07:21:54.365258+0000","last_undegraded":"2024-03-06T07:21:54.365258+0000","last_fullsized":"2024-03-06T07:21:54.365258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:15.049663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358331+0000","last_change":"2024-03-06T07:21:54.358331+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358331+0000","last_undegraded":"2024-03-06T07:21:54.358331+0000","last_fullsized":"2024-03-06T07:21:54.358331+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364803+0000","last_change":"2024-03-06T07:21:43.327014+0000","last_active":"2024-03-06T07:21:54.364803+0000","last_peered":"2024-03-06T07:21:54.364803+0000","last_clean":"2024-03-06T07:21:54.364803+0000","last_became_active":"2024-03-06T07:21:43.326759+0000","last_became_peered":"2024-03-06T07:21:43.326759+0000","last_unstale":"2024-03-06T07:21:54.364803+0000","last_undegraded":"2024-03-06T07:21:54.364803+0000","last_fullsized":"2024-03-06T07:21:54.364803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366485+0000","last_change":"2024-03-06T07:21:54.366485+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366485+0000","last_undegraded":"2024-03-06T07:21:54.366485+0000","last_fullsized":"2024-03-06T07:21:54.366485+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370315+0000","last_change":"2024-03-06T07:21:43.323199+0000","last_active":"2024-03-06T07:21:54.370315+0000","last_peered":"2024-03-06T07:21:54.370315+0000","last_clean":"2024-03-06T07:21:54.370315+0000","last_became_active":"2024-03-06T07:21:43.322835+0000","last_became_peered":"2024-03-06T07:21:43.322835+0000","last_unstale":"2024-03-06T07:21:54.370315+0000","last_undegraded":"2024-03-06T07:21:54.370315+0000","last_fullsized":"2024-03-06T07:21:54.370315+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.369128+0000","last_change":"2024-03-06T07:21:54.369128+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.369128+0000","last_undegraded":"2024-03-06T07:21:54.369128+0000","last_fullsized":"2024-03-06T07:21:54.369128+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357960+0000","last_change":"2024-03-06T07:21:43.318509+0000","last_active":"2024-03-06T07:21:54.357960+0000","last_peered":"2024-03-06T07:21:54.357960+0000","last_clean":"2024-03-06T07:21:54.357960+0000","last_became_active":"2024-03-06T07:21:43.318177+0000","last_became_peered":"2024-03-06T07:21:43.318177+0000","last_unstale":"2024-03-06T07:21:54.357960+0000","last_undegraded":"2024-03-06T07:21:54.357960+0000","last_fullsized":"2024-03-06T07:21:54.357960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361035+0000","last_change":"2024-03-06T07:21:54.361035+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361035+0000","last_undegraded":"2024-03-06T07:21:54.361035+0000","last_fullsized":"2024-03-06T07:21:54.361035+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369672+0000","last_change":"2024-03-06T07:21:43.332733+0000","last_active":"2024-03-06T07:21:54.369672+0000","last_peered":"2024-03-06T07:21:54.369672+0000","last_clean":"2024-03-06T07:21:54.369672+0000","last_became_active":"2024-03-06T07:21:43.331512+0000","last_became_peered":"2024-03-06T07:21:43.331512+0000","last_unstale":"2024-03-06T07:21:54.369672+0000","last_undegraded":"2024-03-06T07:21:54.369672+0000","last_fullsized":"2024-03-06T07:21:54.369672+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363616+0000","last_change":"2024-03-06T07:21:54.363616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363616+0000","last_undegraded":"2024-03-06T07:21:54.363616+0000","last_fullsized":"2024-03-06T07:21:54.363616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365453+0000","last_change":"2024-03-06T07:21:43.321054+0000","last_active":"2024-03-06T07:21:54.365453+0000","last_peered":"2024-03-06T07:21:54.365453+0000","last_clean":"2024-03-06T07:21:54.365453+0000","last_became_active":"2024-03-06T07:21:43.319435+0000","last_became_peered":"2024-03-06T07:21:43.319435+0000","last_unstale":"2024-03-06T07:21:54.365453+0000","last_undegraded":"2024-03-06T07:21:54.365453+0000","last_fullsized":"2024-03-06T07:21:54.365453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362601+0000","last_change":"2024-03-06T07:21:54.362601+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362601+0000","last_undegraded":"2024-03-06T07:21:54.362601+0000","last_fullsized":"2024-03-06T07:21:54.362601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364825+0000","last_change":"2024-03-06T07:21:43.328812+0000","last_active":"2024-03-06T07:21:54.364825+0000","last_peered":"2024-03-06T07:21:54.364825+0000","last_clean":"2024-03-06T07:21:54.364825+0000","last_became_active":"2024-03-06T07:21:43.328366+0000","last_became_peered":"2024-03-06T07:21:43.328366+0000","last_unstale":"2024-03-06T07:21:54.364825+0000","last_undegraded":"2024-03-06T07:21:54.364825+0000","last_fullsized":"2024-03-06T07:21:54.364825+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:48.381105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362773+0000","last_change":"2024-03-06T07:21:54.362773+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362773+0000","last_undegraded":"2024-03-06T07:21:54.362773+0000","last_fullsized":"2024-03-06T07:21:54.362773+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356882+0000","last_change":"2024-03-06T07:21:43.321155+0000","last_active":"2024-03-06T07:21:54.356882+0000","last_peered":"2024-03-06T07:21:54.356882+0000","last_clean":"2024-03-06T07:21:54.356882+0000","last_became_active":"2024-03-06T07:21:43.320747+0000","last_became_peered":"2024-03-06T07:21:43.320747+0000","last_unstale":"2024-03-06T07:21:54.356882+0000","last_undegraded":"2024-03-06T07:21:54.356882+0000","last_fullsized":"2024-03-06T07:21:54.356882+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:42.424597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358521+0000","last_change":"2024-03-06T07:21:54.358521+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358521+0000","last_undegraded":"2024-03-06T07:21:54.358521+0000","last_fullsized":"2024-03-06T07:21:54.358521+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368700+0000","last_change":"2024-03-06T07:21:43.326550+0000","last_active":"2024-03-06T07:21:54.368700+0000","last_peered":"2024-03-06T07:21:54.368700+0000","last_clean":"2024-03-06T07:21:54.368700+0000","last_became_active":"2024-03-06T07:21:43.326254+0000","last_became_peered":"2024-03-06T07:21:43.326254+0000","last_unstale":"2024-03-06T07:21:54.368700+0000","last_undegraded":"2024-03-06T07:21:54.368700+0000","last_fullsized":"2024-03-06T07:21:54.368700+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357560+0000","last_change":"2024-03-06T07:21:54.357560+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357560+0000","last_undegraded":"2024-03-06T07:21:54.357560+0000","last_fullsized":"2024-03-06T07:21:54.357560+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360266+0000","last_change":"2024-03-06T07:21:43.322773+0000","last_active":"2024-03-06T07:21:54.360266+0000","last_peered":"2024-03-06T07:21:54.360266+0000","last_clean":"2024-03-06T07:21:54.360266+0000","last_became_active":"2024-03-06T07:21:43.322525+0000","last_became_peered":"2024-03-06T07:21:43.322525+0000","last_unstale":"2024-03-06T07:21:54.360266+0000","last_undegraded":"2024-03-06T07:21:54.360266+0000","last_fullsized":"2024-03-06T07:21:54.360266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362616+0000","last_change":"2024-03-06T07:21:54.362616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362616+0000","last_undegraded":"2024-03-06T07:21:54.362616+0000","last_fullsized":"2024-03-06T07:21:54.362616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376862+0000","last_change":"2024-03-06T07:21:54.366905+0000","last_active":"2024-03-06T07:21:54.376862+0000","last_peered":"2024-03-06T07:21:54.376862+0000","last_clean":"2024-03-06T07:21:54.376862+0000","last_became_active":"2024-03-06T07:21:54.366627+0000","last_became_peered":"2024-03-06T07:21:54.366627+0000","last_unstale":"2024-03-06T07:21:54.376862+0000","last_undegraded":"2024-03-06T07:21:54.376862+0000","last_fullsized":"2024-03-06T07:21:54.376862+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356615+0000","last_change":"2024-03-06T07:21:43.319837+0000","last_active":"2024-03-06T07:21:54.356615+0000","last_peered":"2024-03-06T07:21:54.356615+0000","last_clean":"2024-03-06T07:21:54.356615+0000","last_became_active":"2024-03-06T07:21:43.319564+0000","last_became_peered":"2024-03-06T07:21:43.319564+0000","last_unstale":"2024-03-06T07:21:54.356615+0000","last_undegraded":"2024-03-06T07:21:54.356615+0000","last_fullsized":"2024-03-06T07:21:54.356615+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365243+0000","last_change":"2024-03-06T07:21:42.310584+0000","last_active":"2024-03-06T07:21:54.365243+0000","last_peered":"2024-03-06T07:21:54.365243+0000","last_clean":"2024-03-06T07:21:54.365243+0000","last_became_active":"2024-03-06T07:21:42.310317+0000","last_became_peered":"2024-03-06T07:21:42.310317+0000","last_unstale":"2024-03-06T07:21:54.365243+0000","last_undegraded":"2024-03-06T07:21:54.365243+0000","last_fullsized":"2024-03-06T07:21:54.365243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362416+0000","last_change":"2024-03-06T07:21:54.362416+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362416+0000","last_undegraded":"2024-03-06T07:21:54.362416+0000","last_fullsized":"2024-03-06T07:21:54.362416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380180+0000","last_change":"2024-03-06T07:21:54.377542+0000","last_active":"2024-03-06T07:21:54.380180+0000","last_peered":"2024-03-06T07:21:54.380180+0000","last_clean":"2024-03-06T07:21:54.380180+0000","last_became_active":"2024-03-06T07:21:54.377250+0000","last_became_peered":"2024-03-06T07:21:54.377250+0000","last_unstale":"2024-03-06T07:21:54.380180+0000","last_undegraded":"2024-03-06T07:21:54.380180+0000","last_fullsized":"2024-03-06T07:21:54.380180+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:36.750007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359472+0000","last_change":"2024-03-06T07:21:43.323883+0000","last_active":"2024-03-06T07:21:54.359472+0000","last_peered":"2024-03-06T07:21:54.359472+0000","last_clean":"2024-03-06T07:21:54.359472+0000","last_became_active":"2024-03-06T07:21:43.323613+0000","last_became_peered":"2024-03-06T07:21:43.323613+0000","last_unstale":"2024-03-06T07:21:54.359472+0000","last_undegraded":"2024-03-06T07:21:54.359472+0000","last_fullsized":"2024-03-06T07:21:54.359472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370119+0000","last_change":"2024-03-06T07:21:42.304199+0000","last_active":"2024-03-06T07:21:54.370119+0000","last_peered":"2024-03-06T07:21:54.370119+0000","last_clean":"2024-03-06T07:21:54.370119+0000","last_became_active":"2024-03-06T07:21:42.303802+0000","last_became_peered":"2024-03-06T07:21:42.303802+0000","last_unstale":"2024-03-06T07:21:54.370119+0000","last_undegraded":"2024-03-06T07:21:54.370119+0000","last_fullsized":"2024-03-06T07:21:54.370119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362959+0000","last_change":"2024-03-06T07:21:54.362959+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362959+0000","last_undegraded":"2024-03-06T07:21:54.362959+0000","last_fullsized":"2024-03-06T07:21:54.362959+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376955+0000","last_change":"2024-03-06T07:21:54.368687+0000","last_active":"2024-03-06T07:21:54.376955+0000","last_peered":"2024-03-06T07:21:54.376955+0000","last_clean":"2024-03-06T07:21:54.376955+0000","last_became_active":"2024-03-06T07:21:54.368106+0000","last_became_peered":"2024-03-06T07:21:54.368106+0000","last_unstale":"2024-03-06T07:21:54.376955+0000","last_undegraded":"2024-03-06T07:21:54.376955+0000","last_fullsized":"2024-03-06T07:21:54.376955+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364372+0000","last_change":"2024-03-06T07:21:43.323512+0000","last_active":"2024-03-06T07:21:54.364372+0000","last_peered":"2024-03-06T07:21:54.364372+0000","last_clean":"2024-03-06T07:21:54.364372+0000","last_became_active":"2024-03-06T07:21:43.323216+0000","last_became_peered":"2024-03-06T07:21:43.323216+0000","last_unstale":"2024-03-06T07:21:54.364372+0000","last_undegraded":"2024-03-06T07:21:54.364372+0000","last_fullsized":"2024-03-06T07:21:54.364372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:31.821522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369735+0000","last_change":"2024-03-06T07:21:42.305711+0000","last_active":"2024-03-06T07:21:54.369735+0000","last_peered":"2024-03-06T07:21:54.369735+0000","last_clean":"2024-03-06T07:21:54.369735+0000","last_became_active":"2024-03-06T07:21:42.305431+0000","last_became_peered":"2024-03-06T07:21:42.305431+0000","last_unstale":"2024-03-06T07:21:54.369735+0000","last_undegraded":"2024-03-06T07:21:54.369735+0000","last_fullsized":"2024-03-06T07:21:54.369735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357254+0000","last_change":"2024-03-06T07:21:54.357254+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357254+0000","last_undegraded":"2024-03-06T07:21:54.357254+0000","last_fullsized":"2024-03-06T07:21:54.357254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374713+0000","last_change":"2024-03-06T07:21:54.370174+0000","last_active":"2024-03-06T07:21:54.374713+0000","last_peered":"2024-03-06T07:21:54.374713+0000","last_clean":"2024-03-06T07:21:54.374713+0000","last_became_active":"2024-03-06T07:21:54.369897+0000","last_became_peered":"2024-03-06T07:21:54.369897+0000","last_unstale":"2024-03-06T07:21:54.374713+0000","last_undegraded":"2024-03-06T07:21:54.374713+0000","last_fullsized":"2024-03-06T07:21:54.374713+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359478+0000","last_change":"2024-03-06T07:21:43.317966+0000","last_active":"2024-03-06T07:21:54.359478+0000","last_peered":"2024-03-06T07:21:54.359478+0000","last_clean":"2024-03-06T07:21:54.359478+0000","last_became_active":"2024-03-06T07:21:43.317700+0000","last_became_peered":"2024-03-06T07:21:43.317700+0000","last_unstale":"2024-03-06T07:21:54.359478+0000","last_undegraded":"2024-03-06T07:21:54.359478+0000","last_fullsized":"2024-03-06T07:21:54.359478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359768+0000","last_change":"2024-03-06T07:21:42.315045+0000","last_active":"2024-03-06T07:21:54.359768+0000","last_peered":"2024-03-06T07:21:54.359768+0000","last_clean":"2024-03-06T07:21:54.359768+0000","last_became_active":"2024-03-06T07:21:42.314768+0000","last_became_peered":"2024-03-06T07:21:42.314768+0000","last_unstale":"2024-03-06T07:21:54.359768+0000","last_undegraded":"2024-03-06T07:21:54.359768+0000","last_fullsized":"2024-03-06T07:21:54.359768+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:14.953476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360237+0000","last_change":"2024-03-06T07:21:54.360237+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360237+0000","last_undegraded":"2024-03-06T07:21:54.360237+0000","last_fullsized":"2024-03-06T07:21:54.360237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379825+0000","last_change":"2024-03-06T07:21:54.375783+0000","last_active":"2024-03-06T07:21:54.379825+0000","last_peered":"2024-03-06T07:21:54.379825+0000","last_clean":"2024-03-06T07:21:54.379825+0000","last_became_active":"2024-03-06T07:21:54.375473+0000","last_became_peered":"2024-03-06T07:21:54.375473+0000","last_unstale":"2024-03-06T07:21:54.379825+0000","last_undegraded":"2024-03-06T07:21:54.379825+0000","last_fullsized":"2024-03-06T07:21:54.379825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:08.612677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370306+0000","last_change":"2024-03-06T07:21:50.343971+0000","last_active":"2024-03-06T07:21:54.370306+0000","last_peered":"2024-03-06T07:21:54.370306+0000","last_clean":"2024-03-06T07:21:54.370306+0000","last_became_active":"2024-03-06T07:21:50.343658+0000","last_became_peered":"2024-03-06T07:21:50.343658+0000","last_unstale":"2024-03-06T07:21:54.370306+0000","last_undegraded":"2024-03-06T07:21:54.370306+0000","last_fullsized":"2024-03-06T07:21:54.370306+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364922+0000","last_change":"2024-03-06T07:21:42.317614+0000","last_active":"2024-03-06T07:21:54.364922+0000","last_peered":"2024-03-06T07:21:54.364922+0000","last_clean":"2024-03-06T07:21:54.364922+0000","last_became_active":"2024-03-06T07:21:42.317094+0000","last_became_peered":"2024-03-06T07:21:42.317094+0000","last_unstale":"2024-03-06T07:21:54.364922+0000","last_undegraded":"2024-03-06T07:21:54.364922+0000","last_fullsized":"2024-03-06T07:21:54.364922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:15.739191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364680+0000","last_change":"2024-03-06T07:21:54.364680+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364680+0000","last_undegraded":"2024-03-06T07:21:54.364680+0000","last_fullsized":"2024-03-06T07:21:54.364680+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377152+0000","last_change":"2024-03-06T07:21:54.375914+0000","last_active":"2024-03-06T07:21:54.377152+0000","last_peered":"2024-03-06T07:21:54.377152+0000","last_clean":"2024-03-06T07:21:54.377152+0000","last_became_active":"2024-03-06T07:21:54.375619+0000","last_became_peered":"2024-03-06T07:21:54.375619+0000","last_unstale":"2024-03-06T07:21:54.377152+0000","last_undegraded":"2024-03-06T07:21:54.377152+0000","last_fullsized":"2024-03-06T07:21:54.377152+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:26.719365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358617+0000","last_change":"2024-03-06T07:21:43.318727+0000","last_active":"2024-03-06T07:21:54.358617+0000","last_peered":"2024-03-06T07:21:54.358617+0000","last_clean":"2024-03-06T07:21:54.358617+0000","last_became_active":"2024-03-06T07:21:43.318440+0000","last_became_peered":"2024-03-06T07:21:43.318440+0000","last_unstale":"2024-03-06T07:21:54.358617+0000","last_undegraded":"2024-03-06T07:21:54.358617+0000","last_fullsized":"2024-03-06T07:21:54.358617+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370222+0000","last_change":"2024-03-06T07:21:42.307726+0000","last_active":"2024-03-06T07:21:54.370222+0000","last_peered":"2024-03-06T07:21:54.370222+0000","last_clean":"2024-03-06T07:21:54.370222+0000","last_became_active":"2024-03-06T07:21:42.307445+0000","last_became_peered":"2024-03-06T07:21:42.307445+0000","last_unstale":"2024-03-06T07:21:54.370222+0000","last_undegraded":"2024-03-06T07:21:54.370222+0000","last_fullsized":"2024-03-06T07:21:54.370222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366515+0000","last_change":"2024-03-06T07:21:54.366515+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366515+0000","last_undegraded":"2024-03-06T07:21:54.366515+0000","last_fullsized":"2024-03-06T07:21:54.366515+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376980+0000","last_change":"2024-03-06T07:21:54.370955+0000","last_active":"2024-03-06T07:21:54.376980+0000","last_peered":"2024-03-06T07:21:54.376980+0000","last_clean":"2024-03-06T07:21:54.376980+0000","last_became_active":"2024-03-06T07:21:54.370680+0000","last_became_peered":"2024-03-06T07:21:54.370680+0000","last_unstale":"2024-03-06T07:21:54.376980+0000","last_undegraded":"2024-03-06T07:21:54.376980+0000","last_fullsized":"2024-03-06T07:21:54.376980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359348+0000","last_change":"2024-03-06T07:21:43.318372+0000","last_active":"2024-03-06T07:21:54.359348+0000","last_peered":"2024-03-06T07:21:54.359348+0000","last_clean":"2024-03-06T07:21:54.359348+0000","last_became_active":"2024-03-06T07:21:43.318047+0000","last_became_peered":"2024-03-06T07:21:43.318047+0000","last_unstale":"2024-03-06T07:21:54.359348+0000","last_undegraded":"2024-03-06T07:21:54.359348+0000","last_fullsized":"2024-03-06T07:21:54.359348+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364556+0000","last_change":"2024-03-06T07:21:42.310677+0000","last_active":"2024-03-06T07:21:54.364556+0000","last_peered":"2024-03-06T07:21:54.364556+0000","last_clean":"2024-03-06T07:21:54.364556+0000","last_became_active":"2024-03-06T07:21:42.310375+0000","last_became_peered":"2024-03-06T07:21:42.310375+0000","last_unstale":"2024-03-06T07:21:54.364556+0000","last_undegraded":"2024-03-06T07:21:54.364556+0000","last_fullsized":"2024-03-06T07:21:54.364556+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:13.837451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358658+0000","last_change":"2024-03-06T07:21:54.358658+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358658+0000","last_undegraded":"2024-03-06T07:21:54.358658+0000","last_fullsized":"2024-03-06T07:21:54.358658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379656+0000","last_change":"2024-03-06T07:21:54.374914+0000","last_active":"2024-03-06T07:21:54.379656+0000","last_peered":"2024-03-06T07:21:54.379656+0000","last_clean":"2024-03-06T07:21:54.379656+0000","last_became_active":"2024-03-06T07:21:54.374570+0000","last_became_peered":"2024-03-06T07:21:54.374570+0000","last_unstale":"2024-03-06T07:21:54.379656+0000","last_undegraded":"2024-03-06T07:21:54.379656+0000","last_fullsized":"2024-03-06T07:21:54.379656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:05:03.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358220+0000","last_change":"2024-03-06T07:21:43.325740+0000","last_active":"2024-03-06T07:21:54.358220+0000","last_peered":"2024-03-06T07:21:54.358220+0000","last_clean":"2024-03-06T07:21:54.358220+0000","last_became_active":"2024-03-06T07:21:43.325425+0000","last_became_peered":"2024-03-06T07:21:43.325425+0000","last_unstale":"2024-03-06T07:21:54.358220+0000","last_undegraded":"2024-03-06T07:21:54.358220+0000","last_fullsized":"2024-03-06T07:21:54.358220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364895+0000","last_change":"2024-03-06T07:21:42.319786+0000","last_active":"2024-03-06T07:21:54.364895+0000","last_peered":"2024-03-06T07:21:54.364895+0000","last_clean":"2024-03-06T07:21:54.364895+0000","last_became_active":"2024-03-06T07:21:42.319430+0000","last_became_peered":"2024-03-06T07:21:42.319430+0000","last_unstale":"2024-03-06T07:21:54.364895+0000","last_undegraded":"2024-03-06T07:21:54.364895+0000","last_fullsized":"2024-03-06T07:21:54.364895+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:09.719754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360176+0000","last_change":"2024-03-06T07:21:54.360176+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360176+0000","last_undegraded":"2024-03-06T07:21:54.360176+0000","last_fullsized":"2024-03-06T07:21:54.360176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369499+0000","last_change":"2024-03-06T07:21:54.367800+0000","last_active":"2024-03-06T07:21:54.369499+0000","last_peered":"2024-03-06T07:21:54.369499+0000","last_clean":"2024-03-06T07:21:54.369499+0000","last_became_active":"2024-03-06T07:21:54.367468+0000","last_became_peered":"2024-03-06T07:21:54.367468+0000","last_unstale":"2024-03-06T07:21:54.369499+0000","last_undegraded":"2024-03-06T07:21:54.369499+0000","last_fullsized":"2024-03-06T07:21:54.369499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.330805+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.330267+0000","last_became_peered":"2024-03-06T07:21:43.330267+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:56.716578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358318+0000","last_change":"2024-03-06T07:21:42.312213+0000","last_active":"2024-03-06T07:21:54.358318+0000","last_peered":"2024-03-06T07:21:54.358318+0000","last_clean":"2024-03-06T07:21:54.358318+0000","last_became_active":"2024-03-06T07:21:42.311878+0000","last_became_peered":"2024-03-06T07:21:42.311878+0000","last_unstale":"2024-03-06T07:21:54.358318+0000","last_undegraded":"2024-03-06T07:21:54.358318+0000","last_fullsized":"2024-03-06T07:21:54.358318+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358914+0000","last_change":"2024-03-06T07:21:54.358914+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358914+0000","last_undegraded":"2024-03-06T07:21:54.358914+0000","last_fullsized":"2024-03-06T07:21:54.358914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380052+0000","last_change":"2024-03-06T07:21:54.377646+0000","last_active":"2024-03-06T07:21:54.380052+0000","last_peered":"2024-03-06T07:21:54.380052+0000","last_clean":"2024-03-06T07:21:54.380052+0000","last_became_active":"2024-03-06T07:21:54.377219+0000","last_became_peered":"2024-03-06T07:21:54.377219+0000","last_unstale":"2024-03-06T07:21:54.380052+0000","last_undegraded":"2024-03-06T07:21:54.380052+0000","last_fullsized":"2024-03-06T07:21:54.380052+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:59.768227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:43.320122+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:43.319799+0000","last_became_peered":"2024-03-06T07:21:43.319799+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370295+0000","last_change":"2024-03-06T07:21:42.319681+0000","last_active":"2024-03-06T07:21:54.370295+0000","last_peered":"2024-03-06T07:21:54.370295+0000","last_clean":"2024-03-06T07:21:54.370295+0000","last_became_active":"2024-03-06T07:21:42.319452+0000","last_became_peered":"2024-03-06T07:21:42.319452+0000","last_unstale":"2024-03-06T07:21:54.370295+0000","last_undegraded":"2024-03-06T07:21:54.370295+0000","last_fullsized":"2024-03-06T07:21:54.370295+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:35.308790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362688+0000","last_change":"2024-03-06T07:21:54.362688+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362688+0000","last_undegraded":"2024-03-06T07:21:54.362688+0000","last_fullsized":"2024-03-06T07:21:54.362688+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368199+0000","last_change":"2024-03-06T07:21:54.367115+0000","last_active":"2024-03-06T07:21:54.368199+0000","last_peered":"2024-03-06T07:21:54.368199+0000","last_clean":"2024-03-06T07:21:54.368199+0000","last_became_active":"2024-03-06T07:21:54.366564+0000","last_became_peered":"2024-03-06T07:21:54.366564+0000","last_unstale":"2024-03-06T07:21:54.368199+0000","last_undegraded":"2024-03-06T07:21:54.368199+0000","last_fullsized":"2024-03-06T07:21:54.368199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365114+0000","last_change":"2024-03-06T07:21:43.323490+0000","last_active":"2024-03-06T07:21:54.365114+0000","last_peered":"2024-03-06T07:21:54.365114+0000","last_clean":"2024-03-06T07:21:54.365114+0000","last_became_active":"2024-03-06T07:21:43.323088+0000","last_became_peered":"2024-03-06T07:21:43.323088+0000","last_unstale":"2024-03-06T07:21:54.365114+0000","last_undegraded":"2024-03-06T07:21:54.365114+0000","last_fullsized":"2024-03-06T07:21:54.365114+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358795+0000","last_change":"2024-03-06T07:21:42.312732+0000","last_active":"2024-03-06T07:21:54.358795+0000","last_peered":"2024-03-06T07:21:54.358795+0000","last_clean":"2024-03-06T07:21:54.358795+0000","last_became_active":"2024-03-06T07:21:42.312462+0000","last_became_peered":"2024-03-06T07:21:42.312462+0000","last_unstale":"2024-03-06T07:21:54.358795+0000","last_undegraded":"2024-03-06T07:21:54.358795+0000","last_fullsized":"2024-03-06T07:21:54.358795+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363414+0000","last_change":"2024-03-06T07:21:54.363414+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363414+0000","last_undegraded":"2024-03-06T07:21:54.363414+0000","last_fullsized":"2024-03-06T07:21:54.363414+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367522+0000","last_change":"2024-03-06T07:21:54.365558+0000","last_active":"2024-03-06T07:21:54.367522+0000","last_peered":"2024-03-06T07:21:54.367522+0000","last_clean":"2024-03-06T07:21:54.367522+0000","last_became_active":"2024-03-06T07:21:54.365272+0000","last_became_peered":"2024-03-06T07:21:54.365272+0000","last_unstale":"2024-03-06T07:21:54.367522+0000","last_undegraded":"2024-03-06T07:21:54.367522+0000","last_fullsized":"2024-03-06T07:21:54.367522+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369842+0000","last_change":"2024-03-06T07:21:43.320022+0000","last_active":"2024-03-06T07:21:54.369842+0000","last_peered":"2024-03-06T07:21:54.369842+0000","last_clean":"2024-03-06T07:21:54.369842+0000","last_became_active":"2024-03-06T07:21:43.319716+0000","last_became_peered":"2024-03-06T07:21:43.319716+0000","last_unstale":"2024-03-06T07:21:54.369842+0000","last_undegraded":"2024-03-06T07:21:54.369842+0000","last_fullsized":"2024-03-06T07:21:54.369842+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:50.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359596+0000","last_change":"2024-03-06T07:21:42.308298+0000","last_active":"2024-03-06T07:21:54.359596+0000","last_peered":"2024-03-06T07:21:54.359596+0000","last_clean":"2024-03-06T07:21:54.359596+0000","last_became_active":"2024-03-06T07:21:42.308008+0000","last_became_peered":"2024-03-06T07:21:42.308008+0000","last_unstale":"2024-03-06T07:21:54.359596+0000","last_undegraded":"2024-03-06T07:21:54.359596+0000","last_fullsized":"2024-03-06T07:21:54.359596+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:51.723259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361630+0000","last_change":"2024-03-06T07:21:54.361630+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361630+0000","last_undegraded":"2024-03-06T07:21:54.361630+0000","last_fullsized":"2024-03-06T07:21:54.361630+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376550+0000","last_change":"2024-03-06T07:21:54.375937+0000","last_active":"2024-03-06T07:21:54.376550+0000","last_peered":"2024-03-06T07:21:54.376550+0000","last_clean":"2024-03-06T07:21:54.376550+0000","last_became_active":"2024-03-06T07:21:54.375648+0000","last_became_peered":"2024-03-06T07:21:54.375648+0000","last_unstale":"2024-03-06T07:21:54.376550+0000","last_undegraded":"2024-03-06T07:21:54.376550+0000","last_fullsized":"2024-03-06T07:21:54.376550+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359583+0000","last_change":"2024-03-06T07:21:43.323228+0000","last_active":"2024-03-06T07:21:54.359583+0000","last_peered":"2024-03-06T07:21:54.359583+0000","last_clean":"2024-03-06T07:21:54.359583+0000","last_became_active":"2024-03-06T07:21:43.322951+0000","last_became_peered":"2024-03-06T07:21:43.322951+0000","last_unstale":"2024-03-06T07:21:54.359583+0000","last_undegraded":"2024-03-06T07:21:54.359583+0000","last_fullsized":"2024-03-06T07:21:54.359583+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365013+0000","last_change":"2024-03-06T07:21:42.321872+0000","last_active":"2024-03-06T07:21:54.365013+0000","last_peered":"2024-03-06T07:21:54.365013+0000","last_clean":"2024-03-06T07:21:54.365013+0000","last_became_active":"2024-03-06T07:21:42.321384+0000","last_became_peered":"2024-03-06T07:21:42.321384+0000","last_unstale":"2024-03-06T07:21:54.365013+0000","last_undegraded":"2024-03-06T07:21:54.365013+0000","last_fullsized":"2024-03-06T07:21:54.365013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358625+0000","last_change":"2024-03-06T07:21:54.358625+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358625+0000","last_undegraded":"2024-03-06T07:21:54.358625+0000","last_fullsized":"2024-03-06T07:21:54.358625+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375601+0000","last_change":"2024-03-06T07:21:54.373488+0000","last_active":"2024-03-06T07:21:54.375601+0000","last_peered":"2024-03-06T07:21:54.375601+0000","last_clean":"2024-03-06T07:21:54.375601+0000","last_became_active":"2024-03-06T07:21:54.373225+0000","last_became_peered":"2024-03-06T07:21:54.373225+0000","last_unstale":"2024-03-06T07:21:54.375601+0000","last_undegraded":"2024-03-06T07:21:54.375601+0000","last_fullsized":"2024-03-06T07:21:54.375601+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369356+0000","last_change":"2024-03-06T07:21:43.324244+0000","last_active":"2024-03-06T07:21:54.369356+0000","last_peered":"2024-03-06T07:21:54.369356+0000","last_clean":"2024-03-06T07:21:54.369356+0000","last_became_active":"2024-03-06T07:21:43.323549+0000","last_became_peered":"2024-03-06T07:21:43.323549+0000","last_unstale":"2024-03-06T07:21:54.369356+0000","last_undegraded":"2024-03-06T07:21:54.369356+0000","last_fullsized":"2024-03-06T07:21:54.369356+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364546+0000","last_change":"2024-03-06T07:21:42.318308+0000","last_active":"2024-03-06T07:21:54.364546+0000","last_peered":"2024-03-06T07:21:54.364546+0000","last_clean":"2024-03-06T07:21:54.364546+0000","last_became_active":"2024-03-06T07:21:42.318080+0000","last_became_peered":"2024-03-06T07:21:42.318080+0000","last_unstale":"2024-03-06T07:21:54.364546+0000","last_undegraded":"2024-03-06T07:21:54.364546+0000","last_fullsized":"2024-03-06T07:21:54.364546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360511+0000","last_change":"2024-03-06T07:21:54.360511+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360511+0000","last_undegraded":"2024-03-06T07:21:54.360511+0000","last_fullsized":"2024-03-06T07:21:54.360511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378458+0000","last_change":"2024-03-06T07:21:54.377690+0000","last_active":"2024-03-06T07:21:54.378458+0000","last_peered":"2024-03-06T07:21:54.378458+0000","last_clean":"2024-03-06T07:21:54.378458+0000","last_became_active":"2024-03-06T07:21:54.377408+0000","last_became_peered":"2024-03-06T07:21:54.377408+0000","last_unstale":"2024-03-06T07:21:54.378458+0000","last_undegraded":"2024-03-06T07:21:54.378458+0000","last_fullsized":"2024-03-06T07:21:54.378458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:31.244976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359542+0000","last_change":"2024-03-06T07:21:43.331853+0000","last_active":"2024-03-06T07:21:54.359542+0000","last_peered":"2024-03-06T07:21:54.359542+0000","last_clean":"2024-03-06T07:21:54.359542+0000","last_became_active":"2024-03-06T07:21:43.331560+0000","last_became_peered":"2024-03-06T07:21:43.331560+0000","last_unstale":"2024-03-06T07:21:54.359542+0000","last_undegraded":"2024-03-06T07:21:54.359542+0000","last_fullsized":"2024-03-06T07:21:54.359542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357780+0000","last_change":"2024-03-06T07:21:42.309167+0000","last_active":"2024-03-06T07:21:54.357780+0000","last_peered":"2024-03-06T07:21:54.357780+0000","last_clean":"2024-03-06T07:21:54.357780+0000","last_became_active":"2024-03-06T07:21:42.308882+0000","last_became_peered":"2024-03-06T07:21:42.308882+0000","last_unstale":"2024-03-06T07:21:54.357780+0000","last_undegraded":"2024-03-06T07:21:54.357780+0000","last_fullsized":"2024-03-06T07:21:54.357780+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366074+0000","last_change":"2024-03-06T07:21:54.366074+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366074+0000","last_undegraded":"2024-03-06T07:21:54.366074+0000","last_fullsized":"2024-03-06T07:21:54.366074+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378021+0000","last_change":"2024-03-06T07:21:54.377427+0000","last_active":"2024-03-06T07:21:54.378021+0000","last_peered":"2024-03-06T07:21:54.378021+0000","last_clean":"2024-03-06T07:21:54.378021+0000","last_became_active":"2024-03-06T07:21:54.377183+0000","last_became_peered":"2024-03-06T07:21:54.377183+0000","last_unstale":"2024-03-06T07:21:54.378021+0000","last_undegraded":"2024-03-06T07:21:54.378021+0000","last_fullsized":"2024-03-06T07:21:54.378021+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.324536+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.323818+0000","last_became_peered":"2024-03-06T07:21:43.323818+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358547+0000","last_change":"2024-03-06T07:21:42.308460+0000","last_active":"2024-03-06T07:21:54.358547+0000","last_peered":"2024-03-06T07:21:54.358547+0000","last_clean":"2024-03-06T07:21:54.358547+0000","last_became_active":"2024-03-06T07:21:42.307461+0000","last_became_peered":"2024-03-06T07:21:42.307461+0000","last_unstale":"2024-03-06T07:21:54.358547+0000","last_undegraded":"2024-03-06T07:21:54.358547+0000","last_fullsized":"2024-03-06T07:21:54.358547+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:34.346180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364356+0000","last_change":"2024-03-06T07:21:54.364356+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364356+0000","last_undegraded":"2024-03-06T07:21:54.364356+0000","last_fullsized":"2024-03-06T07:21:54.364356+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367923+0000","last_change":"2024-03-06T07:21:54.366324+0000","last_active":"2024-03-06T07:21:54.367923+0000","last_peered":"2024-03-06T07:21:54.367923+0000","last_clean":"2024-03-06T07:21:54.367923+0000","last_became_active":"2024-03-06T07:21:54.366064+0000","last_became_peered":"2024-03-06T07:21:54.366064+0000","last_unstale":"2024-03-06T07:21:54.367923+0000","last_undegraded":"2024-03-06T07:21:54.367923+0000","last_fullsized":"2024-03-06T07:21:54.367923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358427+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.358427+0000","last_peered":"2024-03-06T07:21:54.358427+0000","last_clean":"2024-03-06T07:21:54.358427+0000","last_became_active":"2024-03-06T07:21:43.316848+0000","last_became_peered":"2024-03-06T07:21:43.316848+0000","last_unstale":"2024-03-06T07:21:54.358427+0000","last_undegraded":"2024-03-06T07:21:54.358427+0000","last_fullsized":"2024-03-06T07:21:54.358427+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370393+0000","last_change":"2024-03-06T07:21:42.319998+0000","last_active":"2024-03-06T07:21:54.370393+0000","last_peered":"2024-03-06T07:21:54.370393+0000","last_clean":"2024-03-06T07:21:54.370393+0000","last_became_active":"2024-03-06T07:21:42.319529+0000","last_became_peered":"2024-03-06T07:21:42.319529+0000","last_unstale":"2024-03-06T07:21:54.370393+0000","last_undegraded":"2024-03-06T07:21:54.370393+0000","last_fullsized":"2024-03-06T07:21:54.370393+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361900+0000","last_change":"2024-03-06T07:21:54.361900+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361900+0000","last_undegraded":"2024-03-06T07:21:54.361900+0000","last_fullsized":"2024-03-06T07:21:54.361900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374081+0000","last_change":"2024-03-06T07:21:54.370046+0000","last_active":"2024-03-06T07:21:54.374081+0000","last_peered":"2024-03-06T07:21:54.374081+0000","last_clean":"2024-03-06T07:21:54.374081+0000","last_became_active":"2024-03-06T07:21:54.369791+0000","last_became_peered":"2024-03-06T07:21:54.369791+0000","last_unstale":"2024-03-06T07:21:54.374081+0000","last_undegraded":"2024-03-06T07:21:54.374081+0000","last_fullsized":"2024-03-06T07:21:54.374081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:09.373432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367145+0000","last_change":"2024-03-06T07:21:43.327074+0000","last_active":"2024-03-06T07:21:54.367145+0000","last_peered":"2024-03-06T07:21:54.367145+0000","last_clean":"2024-03-06T07:21:54.367145+0000","last_became_active":"2024-03-06T07:21:43.326797+0000","last_became_peered":"2024-03-06T07:21:43.326797+0000","last_unstale":"2024-03-06T07:21:54.367145+0000","last_undegraded":"2024-03-06T07:21:54.367145+0000","last_fullsized":"2024-03-06T07:21:54.367145+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370126+0000","last_change":"2024-03-06T07:21:42.310454+0000","last_active":"2024-03-06T07:21:54.370126+0000","last_peered":"2024-03-06T07:21:54.370126+0000","last_clean":"2024-03-06T07:21:54.370126+0000","last_became_active":"2024-03-06T07:21:42.310172+0000","last_became_peered":"2024-03-06T07:21:42.310172+0000","last_unstale":"2024-03-06T07:21:54.370126+0000","last_undegraded":"2024-03-06T07:21:54.370126+0000","last_fullsized":"2024-03-06T07:21:54.370126+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:34.182536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360981+0000","last_change":"2024-03-06T07:21:54.360981+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360981+0000","last_undegraded":"2024-03-06T07:21:54.360981+0000","last_fullsized":"2024-03-06T07:21:54.360981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376984+0000","last_change":"2024-03-06T07:21:54.368897+0000","last_active":"2024-03-06T07:21:54.376984+0000","last_peered":"2024-03-06T07:21:54.376984+0000","last_clean":"2024-03-06T07:21:54.376984+0000","last_became_active":"2024-03-06T07:21:54.368398+0000","last_became_peered":"2024-03-06T07:21:54.368398+0000","last_unstale":"2024-03-06T07:21:54.376984+0000","last_undegraded":"2024-03-06T07:21:54.376984+0000","last_fullsized":"2024-03-06T07:21:54.376984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369221+0000","last_change":"2024-03-06T07:21:43.324398+0000","last_active":"2024-03-06T07:21:54.369221+0000","last_peered":"2024-03-06T07:21:54.369221+0000","last_clean":"2024-03-06T07:21:54.369221+0000","last_became_active":"2024-03-06T07:21:43.324115+0000","last_became_peered":"2024-03-06T07:21:43.324115+0000","last_unstale":"2024-03-06T07:21:54.369221+0000","last_undegraded":"2024-03-06T07:21:54.369221+0000","last_fullsized":"2024-03-06T07:21:54.369221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:36.888023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365848+0000","last_change":"2024-03-06T07:21:42.312390+0000","last_active":"2024-03-06T07:21:54.365848+0000","last_peered":"2024-03-06T07:21:54.365848+0000","last_clean":"2024-03-06T07:21:54.365848+0000","last_became_active":"2024-03-06T07:21:42.312079+0000","last_became_peered":"2024-03-06T07:21:42.312079+0000","last_unstale":"2024-03-06T07:21:54.365848+0000","last_undegraded":"2024-03-06T07:21:54.365848+0000","last_fullsized":"2024-03-06T07:21:54.365848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364238+0000","last_change":"2024-03-06T07:21:54.364238+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364238+0000","last_undegraded":"2024-03-06T07:21:54.364238+0000","last_fullsized":"2024-03-06T07:21:54.364238+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372534+0000","last_change":"2024-03-06T07:21:54.370872+0000","last_active":"2024-03-06T07:21:54.372534+0000","last_peered":"2024-03-06T07:21:54.372534+0000","last_clean":"2024-03-06T07:21:54.372534+0000","last_became_active":"2024-03-06T07:21:54.370589+0000","last_became_peered":"2024-03-06T07:21:54.370589+0000","last_unstale":"2024-03-06T07:21:54.372534+0000","last_undegraded":"2024-03-06T07:21:54.372534+0000","last_fullsized":"2024-03-06T07:21:54.372534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:36.087274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368542+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.368542+0000","last_peered":"2024-03-06T07:21:54.368542+0000","last_clean":"2024-03-06T07:21:54.368542+0000","last_became_active":"2024-03-06T07:21:43.316857+0000","last_became_peered":"2024-03-06T07:21:43.316857+0000","last_unstale":"2024-03-06T07:21:54.368542+0000","last_undegraded":"2024-03-06T07:21:54.368542+0000","last_fullsized":"2024-03-06T07:21:54.368542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:48.715734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365550+0000","last_change":"2024-03-06T07:21:42.311972+0000","last_active":"2024-03-06T07:21:54.365550+0000","last_peered":"2024-03-06T07:21:54.365550+0000","last_clean":"2024-03-06T07:21:54.365550+0000","last_became_active":"2024-03-06T07:21:42.311640+0000","last_became_peered":"2024-03-06T07:21:42.311640+0000","last_unstale":"2024-03-06T07:21:54.365550+0000","last_undegraded":"2024-03-06T07:21:54.365550+0000","last_fullsized":"2024-03-06T07:21:54.365550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:22.568258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363210+0000","last_change":"2024-03-06T07:21:54.363210+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363210+0000","last_undegraded":"2024-03-06T07:21:54.363210+0000","last_fullsized":"2024-03-06T07:21:54.363210+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375823+0000","last_change":"2024-03-06T07:21:54.372740+0000","last_active":"2024-03-06T07:21:54.375823+0000","last_peered":"2024-03-06T07:21:54.375823+0000","last_clean":"2024-03-06T07:21:54.375823+0000","last_became_active":"2024-03-06T07:21:54.371525+0000","last_became_peered":"2024-03-06T07:21:54.371525+0000","last_unstale":"2024-03-06T07:21:54.375823+0000","last_undegraded":"2024-03-06T07:21:54.375823+0000","last_fullsized":"2024-03-06T07:21:54.375823+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359675+0000","last_change":"2024-03-06T07:21:43.320374+0000","last_active":"2024-03-06T07:21:54.359675+0000","last_peered":"2024-03-06T07:21:54.359675+0000","last_clean":"2024-03-06T07:21:54.359675+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.359675+0000","last_undegraded":"2024-03-06T07:21:54.359675+0000","last_fullsized":"2024-03-06T07:21:54.359675+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368677+0000","last_change":"2024-03-06T07:21:42.309901+0000","last_active":"2024-03-06T07:21:54.368677+0000","last_peered":"2024-03-06T07:21:54.368677+0000","last_clean":"2024-03-06T07:21:54.368677+0000","last_became_active":"2024-03-06T07:21:42.309532+0000","last_became_peered":"2024-03-06T07:21:42.309532+0000","last_unstale":"2024-03-06T07:21:54.368677+0000","last_undegraded":"2024-03-06T07:21:54.368677+0000","last_fullsized":"2024-03-06T07:21:54.368677+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361638+0000","last_change":"2024-03-06T07:21:54.361638+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361638+0000","last_undegraded":"2024-03-06T07:21:54.361638+0000","last_fullsized":"2024-03-06T07:21:54.361638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377134+0000","last_change":"2024-03-06T07:21:54.375852+0000","last_active":"2024-03-06T07:21:54.377134+0000","last_peered":"2024-03-06T07:21:54.377134+0000","last_clean":"2024-03-06T07:21:54.377134+0000","last_became_active":"2024-03-06T07:21:54.375600+0000","last_became_peered":"2024-03-06T07:21:54.375600+0000","last_unstale":"2024-03-06T07:21:54.377134+0000","last_undegraded":"2024-03-06T07:21:54.377134+0000","last_fullsized":"2024-03-06T07:21:54.377134+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:40.575618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365686+0000","last_change":"2024-03-06T07:21:43.320820+0000","last_active":"2024-03-06T07:21:54.365686+0000","last_peered":"2024-03-06T07:21:54.365686+0000","last_clean":"2024-03-06T07:21:54.365686+0000","last_became_active":"2024-03-06T07:21:43.319225+0000","last_became_peered":"2024-03-06T07:21:43.319225+0000","last_unstale":"2024-03-06T07:21:54.365686+0000","last_undegraded":"2024-03-06T07:21:54.365686+0000","last_fullsized":"2024-03-06T07:21:54.365686+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358917+0000","last_change":"2024-03-06T07:21:42.307676+0000","last_active":"2024-03-06T07:21:54.358917+0000","last_peered":"2024-03-06T07:21:54.358917+0000","last_clean":"2024-03-06T07:21:54.358917+0000","last_became_active":"2024-03-06T07:21:42.307388+0000","last_became_peered":"2024-03-06T07:21:42.307388+0000","last_unstale":"2024-03-06T07:21:54.358917+0000","last_undegraded":"2024-03-06T07:21:54.358917+0000","last_fullsized":"2024-03-06T07:21:54.358917+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:47.711911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361080+0000","last_change":"2024-03-06T07:21:54.361080+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361080+0000","last_undegraded":"2024-03-06T07:21:54.361080+0000","last_fullsized":"2024-03-06T07:21:54.361080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380012+0000","last_change":"2024-03-06T07:21:54.374823+0000","last_active":"2024-03-06T07:21:54.380012+0000","last_peered":"2024-03-06T07:21:54.380012+0000","last_clean":"2024-03-06T07:21:54.380012+0000","last_became_active":"2024-03-06T07:21:54.374515+0000","last_became_peered":"2024-03-06T07:21:54.374515+0000","last_unstale":"2024-03-06T07:21:54.380012+0000","last_undegraded":"2024-03-06T07:21:54.380012+0000","last_fullsized":"2024-03-06T07:21:54.380012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369756+0000","last_change":"2024-03-06T07:21:43.329841+0000","last_active":"2024-03-06T07:21:54.369756+0000","last_peered":"2024-03-06T07:21:54.369756+0000","last_clean":"2024-03-06T07:21:54.369756+0000","last_became_active":"2024-03-06T07:21:43.329526+0000","last_became_peered":"2024-03-06T07:21:43.329526+0000","last_unstale":"2024-03-06T07:21:54.369756+0000","last_undegraded":"2024-03-06T07:21:54.369756+0000","last_fullsized":"2024-03-06T07:21:54.369756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368782+0000","last_change":"2024-03-06T07:21:42.318168+0000","last_active":"2024-03-06T07:21:54.368782+0000","last_peered":"2024-03-06T07:21:54.368782+0000","last_clean":"2024-03-06T07:21:54.368782+0000","last_became_active":"2024-03-06T07:21:42.317866+0000","last_became_peered":"2024-03-06T07:21:42.317866+0000","last_unstale":"2024-03-06T07:21:54.368782+0000","last_undegraded":"2024-03-06T07:21:54.368782+0000","last_fullsized":"2024-03-06T07:21:54.368782+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:49.054000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361933+0000","last_change":"2024-03-06T07:21:54.361933+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361933+0000","last_undegraded":"2024-03-06T07:21:54.361933+0000","last_fullsized":"2024-03-06T07:21:54.361933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374694+0000","last_change":"2024-03-06T07:21:54.369742+0000","last_active":"2024-03-06T07:21:54.374694+0000","last_peered":"2024-03-06T07:21:54.374694+0000","last_clean":"2024-03-06T07:21:54.374694+0000","last_became_active":"2024-03-06T07:21:54.368872+0000","last_became_peered":"2024-03-06T07:21:54.368872+0000","last_unstale":"2024-03-06T07:21:54.374694+0000","last_undegraded":"2024-03-06T07:21:54.374694+0000","last_fullsized":"2024-03-06T07:21:54.374694+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373011+0000","last_change":"2024-03-06T07:21:43.316009+0000","last_active":"2024-03-06T07:21:54.373011+0000","last_peered":"2024-03-06T07:21:54.373011+0000","last_clean":"2024-03-06T07:21:54.373011+0000","last_became_active":"2024-03-06T07:21:43.315699+0000","last_became_peered":"2024-03-06T07:21:43.315699+0000","last_unstale":"2024-03-06T07:21:54.373011+0000","last_undegraded":"2024-03-06T07:21:54.373011+0000","last_fullsized":"2024-03-06T07:21:54.373011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:52.769462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365150+0000","last_change":"2024-03-06T07:21:42.311690+0000","last_active":"2024-03-06T07:21:54.365150+0000","last_peered":"2024-03-06T07:21:54.365150+0000","last_clean":"2024-03-06T07:21:54.365150+0000","last_became_active":"2024-03-06T07:21:42.311417+0000","last_became_peered":"2024-03-06T07:21:42.311417+0000","last_unstale":"2024-03-06T07:21:54.365150+0000","last_undegraded":"2024-03-06T07:21:54.365150+0000","last_fullsized":"2024-03-06T07:21:54.365150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.094878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362069+0000","last_change":"2024-03-06T07:21:54.362069+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362069+0000","last_undegraded":"2024-03-06T07:21:54.362069+0000","last_fullsized":"2024-03-06T07:21:54.362069+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378181+0000","last_change":"2024-03-06T07:21:54.377550+0000","last_active":"2024-03-06T07:21:54.378181+0000","last_peered":"2024-03-06T07:21:54.378181+0000","last_clean":"2024-03-06T07:21:54.378181+0000","last_became_active":"2024-03-06T07:21:54.377271+0000","last_became_peered":"2024-03-06T07:21:54.377271+0000","last_unstale":"2024-03-06T07:21:54.378181+0000","last_undegraded":"2024-03-06T07:21:54.378181+0000","last_fullsized":"2024-03-06T07:21:54.378181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360360+0000","last_change":"2024-03-06T07:21:43.322229+0000","last_active":"2024-03-06T07:21:54.360360+0000","last_peered":"2024-03-06T07:21:54.360360+0000","last_clean":"2024-03-06T07:21:54.360360+0000","last_became_active":"2024-03-06T07:21:43.321934+0000","last_became_peered":"2024-03-06T07:21:43.321934+0000","last_unstale":"2024-03-06T07:21:54.360360+0000","last_undegraded":"2024-03-06T07:21:54.360360+0000","last_fullsized":"2024-03-06T07:21:54.360360+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365379+0000","last_change":"2024-03-06T07:21:42.321714+0000","last_active":"2024-03-06T07:21:54.365379+0000","last_peered":"2024-03-06T07:21:54.365379+0000","last_clean":"2024-03-06T07:21:54.365379+0000","last_became_active":"2024-03-06T07:21:42.321444+0000","last_became_peered":"2024-03-06T07:21:42.321444+0000","last_unstale":"2024-03-06T07:21:54.365379+0000","last_undegraded":"2024-03-06T07:21:54.365379+0000","last_fullsized":"2024-03-06T07:21:54.365379+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361491+0000","last_change":"2024-03-06T07:21:54.361491+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361491+0000","last_undegraded":"2024-03-06T07:21:54.361491+0000","last_fullsized":"2024-03-06T07:21:54.361491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373855+0000","last_change":"2024-03-06T07:21:54.371666+0000","last_active":"2024-03-06T07:21:54.373855+0000","last_peered":"2024-03-06T07:21:54.373855+0000","last_clean":"2024-03-06T07:21:54.373855+0000","last_became_active":"2024-03-06T07:21:54.371394+0000","last_became_peered":"2024-03-06T07:21:54.371394+0000","last_unstale":"2024-03-06T07:21:54.373855+0000","last_undegraded":"2024-03-06T07:21:54.373855+0000","last_fullsized":"2024-03-06T07:21:54.373855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:15.173188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368609+0000","last_change":"2024-03-06T07:21:43.316403+0000","last_active":"2024-03-06T07:21:54.368609+0000","last_peered":"2024-03-06T07:21:54.368609+0000","last_clean":"2024-03-06T07:21:54.368609+0000","last_became_active":"2024-03-06T07:21:43.315416+0000","last_became_peered":"2024-03-06T07:21:43.315416+0000","last_unstale":"2024-03-06T07:21:54.368609+0000","last_undegraded":"2024-03-06T07:21:54.368609+0000","last_fullsized":"2024-03-06T07:21:54.368609+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:32.706894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:42.311790+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:42.311500+0000","last_became_peered":"2024-03-06T07:21:42.311500+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358738+0000","last_change":"2024-03-06T07:21:54.358738+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358738+0000","last_undegraded":"2024-03-06T07:21:54.358738+0000","last_fullsized":"2024-03-06T07:21:54.358738+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374655+0000","last_change":"2024-03-06T07:21:54.369593+0000","last_active":"2024-03-06T07:21:54.374655+0000","last_peered":"2024-03-06T07:21:54.374655+0000","last_clean":"2024-03-06T07:21:54.374655+0000","last_became_active":"2024-03-06T07:21:54.369336+0000","last_became_peered":"2024-03-06T07:21:54.369336+0000","last_unstale":"2024-03-06T07:21:54.374655+0000","last_undegraded":"2024-03-06T07:21:54.374655+0000","last_fullsized":"2024-03-06T07:21:54.374655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359566+0000","last_change":"2024-03-06T07:21:43.319244+0000","last_active":"2024-03-06T07:21:54.359566+0000","last_peered":"2024-03-06T07:21:54.359566+0000","last_clean":"2024-03-06T07:21:54.359566+0000","last_became_active":"2024-03-06T07:21:43.318994+0000","last_became_peered":"2024-03-06T07:21:43.318994+0000","last_unstale":"2024-03-06T07:21:54.359566+0000","last_undegraded":"2024-03-06T07:21:54.359566+0000","last_fullsized":"2024-03-06T07:21:54.359566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369178+0000","last_change":"2024-03-06T07:21:42.315718+0000","last_active":"2024-03-06T07:21:54.369178+0000","last_peered":"2024-03-06T07:21:54.369178+0000","last_clean":"2024-03-06T07:21:54.369178+0000","last_became_active":"2024-03-06T07:21:42.315402+0000","last_became_peered":"2024-03-06T07:21:42.315402+0000","last_unstale":"2024-03-06T07:21:54.369178+0000","last_undegraded":"2024-03-06T07:21:54.369178+0000","last_fullsized":"2024-03-06T07:21:54.369178+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365483+0000","last_change":"2024-03-06T07:21:54.365483+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365483+0000","last_undegraded":"2024-03-06T07:21:54.365483+0000","last_fullsized":"2024-03-06T07:21:54.365483+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379803+0000","last_change":"2024-03-06T07:21:54.375429+0000","last_active":"2024-03-06T07:21:54.379803+0000","last_peered":"2024-03-06T07:21:54.379803+0000","last_clean":"2024-03-06T07:21:54.379803+0000","last_became_active":"2024-03-06T07:21:54.375117+0000","last_became_peered":"2024-03-06T07:21:54.375117+0000","last_unstale":"2024-03-06T07:21:54.379803+0000","last_undegraded":"2024-03-06T07:21:54.379803+0000","last_fullsized":"2024-03-06T07:21:54.379803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369090+0000","last_change":"2024-03-06T07:21:43.323135+0000","last_active":"2024-03-06T07:21:54.369090+0000","last_peered":"2024-03-06T07:21:54.369090+0000","last_clean":"2024-03-06T07:21:54.369090+0000","last_became_active":"2024-03-06T07:21:43.322814+0000","last_became_peered":"2024-03-06T07:21:43.322814+0000","last_unstale":"2024-03-06T07:21:54.369090+0000","last_undegraded":"2024-03-06T07:21:54.369090+0000","last_fullsized":"2024-03-06T07:21:54.369090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:24.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359662+0000","last_change":"2024-03-06T07:21:42.308451+0000","last_active":"2024-03-06T07:21:54.359662+0000","last_peered":"2024-03-06T07:21:54.359662+0000","last_clean":"2024-03-06T07:21:54.359662+0000","last_became_active":"2024-03-06T07:21:42.308167+0000","last_became_peered":"2024-03-06T07:21:42.308167+0000","last_unstale":"2024-03-06T07:21:54.359662+0000","last_undegraded":"2024-03-06T07:21:54.359662+0000","last_fullsized":"2024-03-06T07:21:54.359662+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:31.703072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362102+0000","last_change":"2024-03-06T07:21:54.362102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362102+0000","last_undegraded":"2024-03-06T07:21:54.362102+0000","last_fullsized":"2024-03-06T07:21:54.362102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376688+0000","last_change":"2024-03-06T07:21:54.375869+0000","last_active":"2024-03-06T07:21:54.376688+0000","last_peered":"2024-03-06T07:21:54.376688+0000","last_clean":"2024-03-06T07:21:54.376688+0000","last_became_active":"2024-03-06T07:21:54.375586+0000","last_became_peered":"2024-03-06T07:21:54.375586+0000","last_unstale":"2024-03-06T07:21:54.376688+0000","last_undegraded":"2024-03-06T07:21:54.376688+0000","last_fullsized":"2024-03-06T07:21:54.376688+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:02.230230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356824+0000","last_change":"2024-03-06T07:21:43.324622+0000","last_active":"2024-03-06T07:21:54.356824+0000","last_peered":"2024-03-06T07:21:54.356824+0000","last_clean":"2024-03-06T07:21:54.356824+0000","last_became_active":"2024-03-06T07:21:43.324350+0000","last_became_peered":"2024-03-06T07:21:43.324350+0000","last_unstale":"2024-03-06T07:21:54.356824+0000","last_undegraded":"2024-03-06T07:21:54.356824+0000","last_fullsized":"2024-03-06T07:21:54.356824+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359015+0000","last_change":"2024-03-06T07:21:42.309246+0000","last_active":"2024-03-06T07:21:54.359015+0000","last_peered":"2024-03-06T07:21:54.359015+0000","last_clean":"2024-03-06T07:21:54.359015+0000","last_became_active":"2024-03-06T07:21:42.308990+0000","last_became_peered":"2024-03-06T07:21:42.308990+0000","last_unstale":"2024-03-06T07:21:54.359015+0000","last_undegraded":"2024-03-06T07:21:54.359015+0000","last_fullsized":"2024-03-06T07:21:54.359015+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:51.765639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358506+0000","last_change":"2024-03-06T07:21:54.358506+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358506+0000","last_undegraded":"2024-03-06T07:21:54.358506+0000","last_fullsized":"2024-03-06T07:21:54.358506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377281+0000","last_change":"2024-03-06T07:21:54.371835+0000","last_active":"2024-03-06T07:21:54.377281+0000","last_peered":"2024-03-06T07:21:54.377281+0000","last_clean":"2024-03-06T07:21:54.377281+0000","last_became_active":"2024-03-06T07:21:54.371518+0000","last_became_peered":"2024-03-06T07:21:54.371518+0000","last_unstale":"2024-03-06T07:21:54.377281+0000","last_undegraded":"2024-03-06T07:21:54.377281+0000","last_fullsized":"2024-03-06T07:21:54.377281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369611+0000","last_change":"2024-03-06T07:21:43.329441+0000","last_active":"2024-03-06T07:21:54.369611+0000","last_peered":"2024-03-06T07:21:54.369611+0000","last_clean":"2024-03-06T07:21:54.369611+0000","last_became_active":"2024-03-06T07:21:43.327694+0000","last_became_peered":"2024-03-06T07:21:43.327694+0000","last_unstale":"2024-03-06T07:21:54.369611+0000","last_undegraded":"2024-03-06T07:21:54.369611+0000","last_fullsized":"2024-03-06T07:21:54.369611+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369282+0000","last_change":"2024-03-06T07:21:42.309614+0000","last_active":"2024-03-06T07:21:54.369282+0000","last_peered":"2024-03-06T07:21:54.369282+0000","last_clean":"2024-03-06T07:21:54.369282+0000","last_became_active":"2024-03-06T07:21:42.309319+0000","last_became_peered":"2024-03-06T07:21:42.309319+0000","last_unstale":"2024-03-06T07:21:54.369282+0000","last_undegraded":"2024-03-06T07:21:54.369282+0000","last_fullsized":"2024-03-06T07:21:54.369282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:28.385954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360823+0000","last_change":"2024-03-06T07:21:54.360823+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360823+0000","last_undegraded":"2024-03-06T07:21:54.360823+0000","last_fullsized":"2024-03-06T07:21:54.360823+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379806+0000","last_change":"2024-03-06T07:21:54.375555+0000","last_active":"2024-03-06T07:21:54.379806+0000","last_peered":"2024-03-06T07:21:54.379806+0000","last_clean":"2024-03-06T07:21:54.379806+0000","last_became_active":"2024-03-06T07:21:54.375251+0000","last_became_peered":"2024-03-06T07:21:54.375251+0000","last_unstale":"2024-03-06T07:21:54.379806+0000","last_undegraded":"2024-03-06T07:21:54.379806+0000","last_fullsized":"2024-03-06T07:21:54.379806+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:01.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359852+0000","last_change":"2024-03-06T07:21:43.321711+0000","last_active":"2024-03-06T07:21:54.359852+0000","last_peered":"2024-03-06T07:21:54.359852+0000","last_clean":"2024-03-06T07:21:54.359852+0000","last_became_active":"2024-03-06T07:21:43.321434+0000","last_became_peered":"2024-03-06T07:21:43.321434+0000","last_unstale":"2024-03-06T07:21:54.359852+0000","last_undegraded":"2024-03-06T07:21:54.359852+0000","last_fullsized":"2024-03-06T07:21:54.359852+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:06.150276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360450+0000","last_change":"2024-03-06T07:21:42.310672+0000","last_active":"2024-03-06T07:21:54.360450+0000","last_peered":"2024-03-06T07:21:54.360450+0000","last_clean":"2024-03-06T07:21:54.360450+0000","last_became_active":"2024-03-06T07:21:42.310366+0000","last_became_peered":"2024-03-06T07:21:42.310366+0000","last_unstale":"2024-03-06T07:21:54.360450+0000","last_undegraded":"2024-03-06T07:21:54.360450+0000","last_fullsized":"2024-03-06T07:21:54.360450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:24.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361405+0000","last_change":"2024-03-06T07:21:54.361405+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361405+0000","last_undegraded":"2024-03-06T07:21:54.361405+0000","last_fullsized":"2024-03-06T07:21:54.361405+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367559+0000","last_change":"2024-03-06T07:21:54.366502+0000","last_active":"2024-03-06T07:21:54.367559+0000","last_peered":"2024-03-06T07:21:54.367559+0000","last_clean":"2024-03-06T07:21:54.367559+0000","last_became_active":"2024-03-06T07:21:54.366066+0000","last_became_peered":"2024-03-06T07:21:54.366066+0000","last_unstale":"2024-03-06T07:21:54.367559+0000","last_undegraded":"2024-03-06T07:21:54.367559+0000","last_fullsized":"2024-03-06T07:21:54.367559+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364470+0000","last_change":"2024-03-06T07:21:43.319233+0000","last_active":"2024-03-06T07:21:54.364470+0000","last_peered":"2024-03-06T07:21:54.364470+0000","last_clean":"2024-03-06T07:21:54.364470+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.364470+0000","last_undegraded":"2024-03-06T07:21:54.364470+0000","last_fullsized":"2024-03-06T07:21:54.364470+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369843+0000","last_change":"2024-03-06T07:21:42.322036+0000","last_active":"2024-03-06T07:21:54.369843+0000","last_peered":"2024-03-06T07:21:54.369843+0000","last_clean":"2024-03-06T07:21:54.369843+0000","last_became_active":"2024-03-06T07:21:42.321385+0000","last_became_peered":"2024-03-06T07:21:42.321385+0000","last_unstale":"2024-03-06T07:21:54.369843+0000","last_undegraded":"2024-03-06T07:21:54.369843+0000","last_fullsized":"2024-03-06T07:21:54.369843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356636+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:54.356636+0000","last_peered":"2024-03-06T07:21:54.356636+0000","last_clean":"2024-03-06T07:21:54.356636+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:54.356636+0000","last_undegraded":"2024-03-06T07:21:54.356636+0000","last_fullsized":"2024-03-06T07:21:54.356636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":881653,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":651264,"data_stored":291813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":291161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":284171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":292795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":643072,"data_stored":285201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":291321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":874013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:59.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:59.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.544+0000 7f4da920e640 1 Processor -- start 2024-03-06T07:21:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.544+0000 7f4da920e640 1 -- start start 2024-03-06T07:21:59.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da920e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4da4153990 0x7f4da4156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da920e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4da40acc00 0x7f4da40c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da920e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da4070aa0 con 0x7f4da41506b0 2024-03-06T07:21:59.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da920e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da4156450 con 0x7f4da4153990 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da920e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da4156a10 con 0x7f4da40acc00 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4d9bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4da4153990 0x7f4da4156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4da41506b0 0x7f4da40702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56804/0 (socket says 172.21.15.26:56804) 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da2d9d640 1 -- 172.21.15.26:0/1325550598 learned_addr learned my addr 172.21.15.26:0/1325550598 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da359e640 1 --2- 172.21.15.26:0/1325550598 >> v2:172.21.15.117:3300/0 conn(0x7f4da40acc00 0x7f4da40c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4d9bfff640 1 -- 172.21.15.26:0/1325550598 >> v2:172.21.15.117:3300/0 conn(0x7f4da40acc00 msgr2=0x7f4da40c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4d9bfff640 1 --2- 172.21.15.26:0/1325550598 >> v2:172.21.15.117:3300/0 conn(0x7f4da40acc00 0x7f4da40c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4d9bfff640 1 -- 172.21.15.26:0/1325550598 >> v1:172.21.15.26:6789/0 conn(0x7f4da41506b0 legacy=0x7f4da40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4da259c640 1 -- 172.21.15.26:0/1325550598 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217645364 0 0) 0x7f4da4070aa0 con 0x7f4da41506b0 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.545+0000 7f4d9bfff640 1 -- 172.21.15.26:0/1325550598 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4070d80 con 0x7f4da4153990 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4d9bfff640 1 --2- 172.21.15.26:0/1325550598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da4153990 0x7f4da4156ea0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4da40735e0 tx=0x7f4d8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9d138726665aac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da259c640 1 -- 172.21.15.26:0/1325550598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c035020 con 0x7f4da4153990 2024-03-06T07:21:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da259c640 1 -- 172.21.15.26:0/1325550598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c031830 con 0x7f4da4153990 2024-03-06T07:21:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da259c640 1 -- 172.21.15.26:0/1325550598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c031c00 con 0x7f4da4153990 2024-03-06T07:21:59.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da920e640 1 -- 172.21.15.26:0/1325550598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da4153990 msgr2=0x7f4da4156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da920e640 1 --2- 172.21.15.26:0/1325550598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da4153990 0x7f4da4156ea0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4da40735e0 tx=0x7f4d8c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da920e640 1 -- 172.21.15.26:0/1325550598 shutdown_connections 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da920e640 1 --2- 172.21.15.26:0/1325550598 >> v2:172.21.15.117:3300/0 conn(0x7f4da40acc00 0x7f4da40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.546+0000 7f4da920e640 1 --2- 172.21.15.26:0/1325550598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da4153990 0x7f4da4156ea0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.547+0000 7f4da920e640 1 -- 172.21.15.26:0/1325550598 >> 172.21.15.26:0/1325550598 conn(0x7f4da406b1a0 msgr2=0x7f4da406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.547+0000 7f4da920e640 1 -- 172.21.15.26:0/1325550598 shutdown_connections 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.547+0000 7f4da920e640 1 -- 172.21.15.26:0/1325550598 wait complete. 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 Processor -- start 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 -- start start 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da40acc00 0x7f4da4154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da41506b0 0x7f4da41e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da4153990 0x7f4da40c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40c5b60 con 0x7f4da41506b0 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40c59e0 con 0x7f4da4153990 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.548+0000 7f4da920e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40c5ce0 con 0x7f4da40acc00 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da41506b0 0x7f4da41e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da41506b0 0x7f4da41e8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57884/0 (socket says 172.21.15.26:57884) 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 -- 172.21.15.26:0/4052647547 learned_addr learned my addr 172.21.15.26:0/4052647547 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4da2d9d640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da40acc00 0x7f4da4154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4da359e640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da4153990 0x7f4da40c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 -- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da40acc00 msgr2=0x7f4da4154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da40acc00 0x7f4da4154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 -- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da4153990 msgr2=0x7f4da40c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da4153990 0x7f4da40c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 -- 172.21.15.26:0/4052647547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4154790 con 0x7f4da41506b0 2024-03-06T07:21:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9bfff640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da41506b0 0x7f4da41e8fd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0096f0 tx=0x7f4d8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9b7fe640 1 -- 172.21.15.26:0/4052647547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c035040 con 0x7f4da41506b0 2024-03-06T07:21:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9b7fe640 1 -- 172.21.15.26:0/4052647547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c004230 con 0x7f4da41506b0 2024-03-06T07:21:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da4152c10 con 0x7f4da41506b0 2024-03-06T07:21:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4d9b7fe640 1 -- 172.21.15.26:0/4052647547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c03a640 con 0x7f4da41506b0 2024-03-06T07:21:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.549+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da4152930 con 0x7f4da41506b0 2024-03-06T07:21:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.550+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da404f330 con 0x7f4da41506b0 2024-03-06T07:21:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.551+0000 7f4d9b7fe640 1 -- 172.21.15.26:0/4052647547 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d8c038020 con 0x7f4da41506b0 2024-03-06T07:21:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.552+0000 7f4d9b7fe640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d8406b4d0 0x7f4d8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.552+0000 7f4d9b7fe640 1 -- 172.21.15.26:0/4052647547 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f4d8c0ac270 con 0x7f4da41506b0 2024-03-06T07:21:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.552+0000 7f4da2d9d640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d8406b4d0 0x7f4d8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.552+0000 7f4da2d9d640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d8406b4d0 0x7f4d8406d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000f60 tx=0x7f4d94007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.556+0000 7f4d9b7fe640 1 -- 172.21.15.26:0/4052647547 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d8c07a590 con 0x7f4da41506b0 2024-03-06T07:21:59.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.736+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4da414e240 con 0x7f4d8406b4d0 2024-03-06T07:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.747+0000 7f4d9b7fe640 1 -- 172.21.15.26:0/4052647547 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485262 (secure 0 0 0) 0x7f4da414e240 con 0x7f4d8406b4d0 2024-03-06T07:21:59.754 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:21:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:21:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d8406b4d0 msgr2=0x7f4d8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d8406b4d0 0x7f4d8406d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000f60 tx=0x7f4d94007d50 comp rx=0 tx=0).stop 2024-03-06T07:21:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da41506b0 msgr2=0x7f4da41e8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da41506b0 0x7f4da41e8fd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0096f0 tx=0x7f4d8c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 shutdown_connections 2024-03-06T07:21:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d8406b4d0 0x7f4d8406d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da4153990 0x7f4da40c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da41506b0 0x7f4da41e8fd0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 --2- 172.21.15.26:0/4052647547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da40acc00 0x7f4da4154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 >> 172.21.15.26:0/4052647547 conn(0x7f4da406b1a0 msgr2=0x7f4da4152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 shutdown_connections 2024-03-06T07:21:59.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.751+0000 7f4da920e640 1 -- 172.21.15.26:0/4052647547 wait complete. 2024-03-06T07:21:59.763 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-06T07:21:59.053948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237996,"kb_used_data":5180,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736724,"statfs":{"total":773094113280,"available":772850405376,"internally_reserved":0,"allocated":5304320,"data_stored":3492128,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703896"},"pg_stats":[{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359676+0000","last_change":"2024-03-06T07:21:54.359676+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359676+0000","last_undegraded":"2024-03-06T07:21:54.359676+0000","last_fullsized":"2024-03-06T07:21:54.359676+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372914+0000","last_change":"2024-03-06T07:21:43.317811+0000","last_active":"2024-03-06T07:21:54.372914+0000","last_peered":"2024-03-06T07:21:54.372914+0000","last_clean":"2024-03-06T07:21:54.372914+0000","last_became_active":"2024-03-06T07:21:43.317509+0000","last_became_peered":"2024-03-06T07:21:43.317509+0000","last_unstale":"2024-03-06T07:21:54.372914+0000","last_undegraded":"2024-03-06T07:21:54.372914+0000","last_fullsized":"2024-03-06T07:21:54.372914+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364197+0000","last_change":"2024-03-06T07:21:54.364197+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364197+0000","last_undegraded":"2024-03-06T07:21:54.364197+0000","last_fullsized":"2024-03-06T07:21:54.364197+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357765+0000","last_change":"2024-03-06T07:21:43.317801+0000","last_active":"2024-03-06T07:21:54.357765+0000","last_peered":"2024-03-06T07:21:54.357765+0000","last_clean":"2024-03-06T07:21:54.357765+0000","last_became_active":"2024-03-06T07:21:43.317522+0000","last_became_peered":"2024-03-06T07:21:43.317522+0000","last_unstale":"2024-03-06T07:21:54.357765+0000","last_undegraded":"2024-03-06T07:21:54.357765+0000","last_fullsized":"2024-03-06T07:21:54.357765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.368156+0000","last_change":"2024-03-06T07:21:54.368156+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.368156+0000","last_undegraded":"2024-03-06T07:21:54.368156+0000","last_fullsized":"2024-03-06T07:21:54.368156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369212+0000","last_change":"2024-03-06T07:21:43.328071+0000","last_active":"2024-03-06T07:21:54.369212+0000","last_peered":"2024-03-06T07:21:54.369212+0000","last_clean":"2024-03-06T07:21:54.369212+0000","last_became_active":"2024-03-06T07:21:43.327781+0000","last_became_peered":"2024-03-06T07:21:43.327781+0000","last_unstale":"2024-03-06T07:21:54.369212+0000","last_undegraded":"2024-03-06T07:21:54.369212+0000","last_fullsized":"2024-03-06T07:21:54.369212+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365661+0000","last_change":"2024-03-06T07:21:54.365661+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365661+0000","last_undegraded":"2024-03-06T07:21:54.365661+0000","last_fullsized":"2024-03-06T07:21:54.365661+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357083+0000","last_change":"2024-03-06T07:21:43.326286+0000","last_active":"2024-03-06T07:21:54.357083+0000","last_peered":"2024-03-06T07:21:54.357083+0000","last_clean":"2024-03-06T07:21:54.357083+0000","last_became_active":"2024-03-06T07:21:43.326031+0000","last_became_peered":"2024-03-06T07:21:43.326031+0000","last_unstale":"2024-03-06T07:21:54.357083+0000","last_undegraded":"2024-03-06T07:21:54.357083+0000","last_fullsized":"2024-03-06T07:21:54.357083+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:21.645487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361397+0000","last_change":"2024-03-06T07:21:54.361397+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361397+0000","last_undegraded":"2024-03-06T07:21:54.361397+0000","last_fullsized":"2024-03-06T07:21:54.361397+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369745+0000","last_change":"2024-03-06T07:21:43.321810+0000","last_active":"2024-03-06T07:21:54.369745+0000","last_peered":"2024-03-06T07:21:54.369745+0000","last_clean":"2024-03-06T07:21:54.369745+0000","last_became_active":"2024-03-06T07:21:43.321538+0000","last_became_peered":"2024-03-06T07:21:43.321538+0000","last_unstale":"2024-03-06T07:21:54.369745+0000","last_undegraded":"2024-03-06T07:21:54.369745+0000","last_fullsized":"2024-03-06T07:21:54.369745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:10.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359185+0000","last_change":"2024-03-06T07:21:54.359185+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359185+0000","last_undegraded":"2024-03-06T07:21:54.359185+0000","last_fullsized":"2024-03-06T07:21:54.359185+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368491+0000","last_change":"2024-03-06T07:21:43.318713+0000","last_active":"2024-03-06T07:21:54.368491+0000","last_peered":"2024-03-06T07:21:54.368491+0000","last_clean":"2024-03-06T07:21:54.368491+0000","last_became_active":"2024-03-06T07:21:43.318385+0000","last_became_peered":"2024-03-06T07:21:43.318385+0000","last_unstale":"2024-03-06T07:21:54.368491+0000","last_undegraded":"2024-03-06T07:21:54.368491+0000","last_fullsized":"2024-03-06T07:21:54.368491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361827+0000","last_change":"2024-03-06T07:21:54.361827+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361827+0000","last_undegraded":"2024-03-06T07:21:54.361827+0000","last_fullsized":"2024-03-06T07:21:54.361827+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368595+0000","last_change":"2024-03-06T07:21:43.327099+0000","last_active":"2024-03-06T07:21:54.368595+0000","last_peered":"2024-03-06T07:21:54.368595+0000","last_clean":"2024-03-06T07:21:54.368595+0000","last_became_active":"2024-03-06T07:21:43.326851+0000","last_became_peered":"2024-03-06T07:21:43.326851+0000","last_unstale":"2024-03-06T07:21:54.368595+0000","last_undegraded":"2024-03-06T07:21:54.368595+0000","last_fullsized":"2024-03-06T07:21:54.368595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360705+0000","last_change":"2024-03-06T07:21:54.360705+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360705+0000","last_undegraded":"2024-03-06T07:21:54.360705+0000","last_fullsized":"2024-03-06T07:21:54.360705+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358329+0000","last_change":"2024-03-06T07:21:43.325897+0000","last_active":"2024-03-06T07:21:54.358329+0000","last_peered":"2024-03-06T07:21:54.358329+0000","last_clean":"2024-03-06T07:21:54.358329+0000","last_became_active":"2024-03-06T07:21:43.325647+0000","last_became_peered":"2024-03-06T07:21:43.325647+0000","last_unstale":"2024-03-06T07:21:54.358329+0000","last_undegraded":"2024-03-06T07:21:54.358329+0000","last_fullsized":"2024-03-06T07:21:54.358329+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365629+0000","last_change":"2024-03-06T07:21:54.365629+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365629+0000","last_undegraded":"2024-03-06T07:21:54.365629+0000","last_fullsized":"2024-03-06T07:21:54.365629+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359703+0000","last_change":"2024-03-06T07:21:43.330337+0000","last_active":"2024-03-06T07:21:54.359703+0000","last_peered":"2024-03-06T07:21:54.359703+0000","last_clean":"2024-03-06T07:21:54.359703+0000","last_became_active":"2024-03-06T07:21:43.330053+0000","last_became_peered":"2024-03-06T07:21:43.330053+0000","last_unstale":"2024-03-06T07:21:54.359703+0000","last_undegraded":"2024-03-06T07:21:54.359703+0000","last_fullsized":"2024-03-06T07:21:54.359703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358522+0000","last_change":"2024-03-06T07:21:54.358522+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358522+0000","last_undegraded":"2024-03-06T07:21:54.358522+0000","last_fullsized":"2024-03-06T07:21:54.358522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370017+0000","last_change":"2024-03-06T07:21:43.325100+0000","last_active":"2024-03-06T07:21:54.370017+0000","last_peered":"2024-03-06T07:21:54.370017+0000","last_clean":"2024-03-06T07:21:54.370017+0000","last_became_active":"2024-03-06T07:21:43.324812+0000","last_became_peered":"2024-03-06T07:21:43.324812+0000","last_unstale":"2024-03-06T07:21:54.370017+0000","last_undegraded":"2024-03-06T07:21:54.370017+0000","last_fullsized":"2024-03-06T07:21:54.370017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:11.442368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359265+0000","last_change":"2024-03-06T07:21:54.359265+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359265+0000","last_undegraded":"2024-03-06T07:21:54.359265+0000","last_fullsized":"2024-03-06T07:21:54.359265+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366884+0000","last_change":"2024-03-06T07:21:43.322188+0000","last_active":"2024-03-06T07:21:54.366884+0000","last_peered":"2024-03-06T07:21:54.366884+0000","last_clean":"2024-03-06T07:21:54.366884+0000","last_became_active":"2024-03-06T07:21:43.321685+0000","last_became_peered":"2024-03-06T07:21:43.321685+0000","last_unstale":"2024-03-06T07:21:54.366884+0000","last_undegraded":"2024-03-06T07:21:54.366884+0000","last_fullsized":"2024-03-06T07:21:54.366884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:54.799043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359570+0000","last_change":"2024-03-06T07:21:54.359570+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359570+0000","last_undegraded":"2024-03-06T07:21:54.359570+0000","last_fullsized":"2024-03-06T07:21:54.359570+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359503+0000","last_change":"2024-03-06T07:21:43.321952+0000","last_active":"2024-03-06T07:21:54.359503+0000","last_peered":"2024-03-06T07:21:54.359503+0000","last_clean":"2024-03-06T07:21:54.359503+0000","last_became_active":"2024-03-06T07:21:43.321579+0000","last_became_peered":"2024-03-06T07:21:43.321579+0000","last_unstale":"2024-03-06T07:21:54.359503+0000","last_undegraded":"2024-03-06T07:21:54.359503+0000","last_fullsized":"2024-03-06T07:21:54.359503+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:04.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359648+0000","last_change":"2024-03-06T07:21:54.359648+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359648+0000","last_undegraded":"2024-03-06T07:21:54.359648+0000","last_fullsized":"2024-03-06T07:21:54.359648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368337+0000","last_change":"2024-03-06T07:21:43.321277+0000","last_active":"2024-03-06T07:21:54.368337+0000","last_peered":"2024-03-06T07:21:54.368337+0000","last_clean":"2024-03-06T07:21:54.368337+0000","last_became_active":"2024-03-06T07:21:43.320973+0000","last_became_peered":"2024-03-06T07:21:43.320973+0000","last_unstale":"2024-03-06T07:21:54.368337+0000","last_undegraded":"2024-03-06T07:21:54.368337+0000","last_fullsized":"2024-03-06T07:21:54.368337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:15.957299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362294+0000","last_change":"2024-03-06T07:21:54.362294+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362294+0000","last_undegraded":"2024-03-06T07:21:54.362294+0000","last_fullsized":"2024-03-06T07:21:54.362294+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364199+0000","last_change":"2024-03-06T07:21:50.373762+0000","last_active":"2024-03-06T07:21:54.364199+0000","last_peered":"2024-03-06T07:21:54.364199+0000","last_clean":"2024-03-06T07:21:54.364199+0000","last_became_active":"2024-03-06T07:21:50.373521+0000","last_became_peered":"2024-03-06T07:21:50.373521+0000","last_unstale":"2024-03-06T07:21:54.364199+0000","last_undegraded":"2024-03-06T07:21:54.364199+0000","last_fullsized":"2024-03-06T07:21:54.364199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359359+0000","last_change":"2024-03-06T07:21:54.359359+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359359+0000","last_undegraded":"2024-03-06T07:21:54.359359+0000","last_fullsized":"2024-03-06T07:21:54.359359+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357693+0000","last_change":"2024-03-06T07:21:43.321137+0000","last_active":"2024-03-06T07:21:54.357693+0000","last_peered":"2024-03-06T07:21:54.357693+0000","last_clean":"2024-03-06T07:21:54.357693+0000","last_became_active":"2024-03-06T07:21:43.320820+0000","last_became_peered":"2024-03-06T07:21:43.320820+0000","last_unstale":"2024-03-06T07:21:54.357693+0000","last_undegraded":"2024-03-06T07:21:54.357693+0000","last_fullsized":"2024-03-06T07:21:54.357693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:52.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359065+0000","last_change":"2024-03-06T07:21:54.359065+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359065+0000","last_undegraded":"2024-03-06T07:21:54.359065+0000","last_fullsized":"2024-03-06T07:21:54.359065+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366975+0000","last_change":"2024-03-06T07:21:43.320518+0000","last_active":"2024-03-06T07:21:54.366975+0000","last_peered":"2024-03-06T07:21:54.366975+0000","last_clean":"2024-03-06T07:21:54.366975+0000","last_became_active":"2024-03-06T07:21:43.320178+0000","last_became_peered":"2024-03-06T07:21:43.320178+0000","last_unstale":"2024-03-06T07:21:54.366975+0000","last_undegraded":"2024-03-06T07:21:54.366975+0000","last_fullsized":"2024-03-06T07:21:54.366975+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360235+0000","last_change":"2024-03-06T07:21:54.360235+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360235+0000","last_undegraded":"2024-03-06T07:21:54.360235+0000","last_fullsized":"2024-03-06T07:21:54.360235+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370182+0000","last_change":"2024-03-06T07:21:43.324279+0000","last_active":"2024-03-06T07:21:54.370182+0000","last_peered":"2024-03-06T07:21:54.370182+0000","last_clean":"2024-03-06T07:21:54.370182+0000","last_became_active":"2024-03-06T07:21:43.323499+0000","last_became_peered":"2024-03-06T07:21:43.323499+0000","last_unstale":"2024-03-06T07:21:54.370182+0000","last_undegraded":"2024-03-06T07:21:54.370182+0000","last_fullsized":"2024-03-06T07:21:54.370182+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359045+0000","last_change":"2024-03-06T07:21:54.359045+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359045+0000","last_undegraded":"2024-03-06T07:21:54.359045+0000","last_fullsized":"2024-03-06T07:21:54.359045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357861+0000","last_change":"2024-03-06T07:21:43.318202+0000","last_active":"2024-03-06T07:21:54.357861+0000","last_peered":"2024-03-06T07:21:54.357861+0000","last_clean":"2024-03-06T07:21:54.357861+0000","last_became_active":"2024-03-06T07:21:43.317929+0000","last_became_peered":"2024-03-06T07:21:43.317929+0000","last_unstale":"2024-03-06T07:21:54.357861+0000","last_undegraded":"2024-03-06T07:21:54.357861+0000","last_fullsized":"2024-03-06T07:21:54.357861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367783+0000","last_change":"2024-03-06T07:21:54.367783+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367783+0000","last_undegraded":"2024-03-06T07:21:54.367783+0000","last_fullsized":"2024-03-06T07:21:54.367783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369274+0000","last_change":"2024-03-06T07:21:43.324265+0000","last_active":"2024-03-06T07:21:54.369274+0000","last_peered":"2024-03-06T07:21:54.369274+0000","last_clean":"2024-03-06T07:21:54.369274+0000","last_became_active":"2024-03-06T07:21:43.323877+0000","last_became_peered":"2024-03-06T07:21:43.323877+0000","last_unstale":"2024-03-06T07:21:54.369274+0000","last_undegraded":"2024-03-06T07:21:54.369274+0000","last_fullsized":"2024-03-06T07:21:54.369274+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367102+0000","last_change":"2024-03-06T07:21:54.367102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367102+0000","last_undegraded":"2024-03-06T07:21:54.367102+0000","last_fullsized":"2024-03-06T07:21:54.367102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365314+0000","last_change":"2024-03-06T07:21:43.321580+0000","last_active":"2024-03-06T07:21:54.365314+0000","last_peered":"2024-03-06T07:21:54.365314+0000","last_clean":"2024-03-06T07:21:54.365314+0000","last_became_active":"2024-03-06T07:21:43.319775+0000","last_became_peered":"2024-03-06T07:21:43.319775+0000","last_unstale":"2024-03-06T07:21:54.365314+0000","last_undegraded":"2024-03-06T07:21:54.365314+0000","last_fullsized":"2024-03-06T07:21:54.365314+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:32.802384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358875+0000","last_change":"2024-03-06T07:21:54.358875+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358875+0000","last_undegraded":"2024-03-06T07:21:54.358875+0000","last_fullsized":"2024-03-06T07:21:54.358875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365089+0000","last_change":"2024-03-06T07:21:43.322797+0000","last_active":"2024-03-06T07:21:54.365089+0000","last_peered":"2024-03-06T07:21:54.365089+0000","last_clean":"2024-03-06T07:21:54.365089+0000","last_became_active":"2024-03-06T07:21:43.322505+0000","last_became_peered":"2024-03-06T07:21:43.322505+0000","last_unstale":"2024-03-06T07:21:54.365089+0000","last_undegraded":"2024-03-06T07:21:54.365089+0000","last_fullsized":"2024-03-06T07:21:54.365089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359127+0000","last_change":"2024-03-06T07:21:54.359127+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359127+0000","last_undegraded":"2024-03-06T07:21:54.359127+0000","last_fullsized":"2024-03-06T07:21:54.359127+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358833+0000","last_change":"2024-03-06T07:21:43.326058+0000","last_active":"2024-03-06T07:21:54.358833+0000","last_peered":"2024-03-06T07:21:54.358833+0000","last_clean":"2024-03-06T07:21:54.358833+0000","last_became_active":"2024-03-06T07:21:43.325799+0000","last_became_peered":"2024-03-06T07:21:43.325799+0000","last_unstale":"2024-03-06T07:21:54.358833+0000","last_undegraded":"2024-03-06T07:21:54.358833+0000","last_fullsized":"2024-03-06T07:21:54.358833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357760+0000","last_change":"2024-03-06T07:21:54.357760+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357760+0000","last_undegraded":"2024-03-06T07:21:54.357760+0000","last_fullsized":"2024-03-06T07:21:54.357760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365258+0000","last_change":"2024-03-06T07:21:43.322671+0000","last_active":"2024-03-06T07:21:54.365258+0000","last_peered":"2024-03-06T07:21:54.365258+0000","last_clean":"2024-03-06T07:21:54.365258+0000","last_became_active":"2024-03-06T07:21:43.322376+0000","last_became_peered":"2024-03-06T07:21:43.322376+0000","last_unstale":"2024-03-06T07:21:54.365258+0000","last_undegraded":"2024-03-06T07:21:54.365258+0000","last_fullsized":"2024-03-06T07:21:54.365258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:15.049663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358331+0000","last_change":"2024-03-06T07:21:54.358331+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358331+0000","last_undegraded":"2024-03-06T07:21:54.358331+0000","last_fullsized":"2024-03-06T07:21:54.358331+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364803+0000","last_change":"2024-03-06T07:21:43.327014+0000","last_active":"2024-03-06T07:21:54.364803+0000","last_peered":"2024-03-06T07:21:54.364803+0000","last_clean":"2024-03-06T07:21:54.364803+0000","last_became_active":"2024-03-06T07:21:43.326759+0000","last_became_peered":"2024-03-06T07:21:43.326759+0000","last_unstale":"2024-03-06T07:21:54.364803+0000","last_undegraded":"2024-03-06T07:21:54.364803+0000","last_fullsized":"2024-03-06T07:21:54.364803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366485+0000","last_change":"2024-03-06T07:21:54.366485+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366485+0000","last_undegraded":"2024-03-06T07:21:54.366485+0000","last_fullsized":"2024-03-06T07:21:54.366485+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370315+0000","last_change":"2024-03-06T07:21:43.323199+0000","last_active":"2024-03-06T07:21:54.370315+0000","last_peered":"2024-03-06T07:21:54.370315+0000","last_clean":"2024-03-06T07:21:54.370315+0000","last_became_active":"2024-03-06T07:21:43.322835+0000","last_became_peered":"2024-03-06T07:21:43.322835+0000","last_unstale":"2024-03-06T07:21:54.370315+0000","last_undegraded":"2024-03-06T07:21:54.370315+0000","last_fullsized":"2024-03-06T07:21:54.370315+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.369128+0000","last_change":"2024-03-06T07:21:54.369128+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.369128+0000","last_undegraded":"2024-03-06T07:21:54.369128+0000","last_fullsized":"2024-03-06T07:21:54.369128+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357960+0000","last_change":"2024-03-06T07:21:43.318509+0000","last_active":"2024-03-06T07:21:54.357960+0000","last_peered":"2024-03-06T07:21:54.357960+0000","last_clean":"2024-03-06T07:21:54.357960+0000","last_became_active":"2024-03-06T07:21:43.318177+0000","last_became_peered":"2024-03-06T07:21:43.318177+0000","last_unstale":"2024-03-06T07:21:54.357960+0000","last_undegraded":"2024-03-06T07:21:54.357960+0000","last_fullsized":"2024-03-06T07:21:54.357960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361035+0000","last_change":"2024-03-06T07:21:54.361035+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361035+0000","last_undegraded":"2024-03-06T07:21:54.361035+0000","last_fullsized":"2024-03-06T07:21:54.361035+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369672+0000","last_change":"2024-03-06T07:21:43.332733+0000","last_active":"2024-03-06T07:21:54.369672+0000","last_peered":"2024-03-06T07:21:54.369672+0000","last_clean":"2024-03-06T07:21:54.369672+0000","last_became_active":"2024-03-06T07:21:43.331512+0000","last_became_peered":"2024-03-06T07:21:43.331512+0000","last_unstale":"2024-03-06T07:21:54.369672+0000","last_undegraded":"2024-03-06T07:21:54.369672+0000","last_fullsized":"2024-03-06T07:21:54.369672+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363616+0000","last_change":"2024-03-06T07:21:54.363616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363616+0000","last_undegraded":"2024-03-06T07:21:54.363616+0000","last_fullsized":"2024-03-06T07:21:54.363616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365453+0000","last_change":"2024-03-06T07:21:43.321054+0000","last_active":"2024-03-06T07:21:54.365453+0000","last_peered":"2024-03-06T07:21:54.365453+0000","last_clean":"2024-03-06T07:21:54.365453+0000","last_became_active":"2024-03-06T07:21:43.319435+0000","last_became_peered":"2024-03-06T07:21:43.319435+0000","last_unstale":"2024-03-06T07:21:54.365453+0000","last_undegraded":"2024-03-06T07:21:54.365453+0000","last_fullsized":"2024-03-06T07:21:54.365453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362601+0000","last_change":"2024-03-06T07:21:54.362601+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362601+0000","last_undegraded":"2024-03-06T07:21:54.362601+0000","last_fullsized":"2024-03-06T07:21:54.362601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364825+0000","last_change":"2024-03-06T07:21:43.328812+0000","last_active":"2024-03-06T07:21:54.364825+0000","last_peered":"2024-03-06T07:21:54.364825+0000","last_clean":"2024-03-06T07:21:54.364825+0000","last_became_active":"2024-03-06T07:21:43.328366+0000","last_became_peered":"2024-03-06T07:21:43.328366+0000","last_unstale":"2024-03-06T07:21:54.364825+0000","last_undegraded":"2024-03-06T07:21:54.364825+0000","last_fullsized":"2024-03-06T07:21:54.364825+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:48.381105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362773+0000","last_change":"2024-03-06T07:21:54.362773+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362773+0000","last_undegraded":"2024-03-06T07:21:54.362773+0000","last_fullsized":"2024-03-06T07:21:54.362773+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356882+0000","last_change":"2024-03-06T07:21:43.321155+0000","last_active":"2024-03-06T07:21:54.356882+0000","last_peered":"2024-03-06T07:21:54.356882+0000","last_clean":"2024-03-06T07:21:54.356882+0000","last_became_active":"2024-03-06T07:21:43.320747+0000","last_became_peered":"2024-03-06T07:21:43.320747+0000","last_unstale":"2024-03-06T07:21:54.356882+0000","last_undegraded":"2024-03-06T07:21:54.356882+0000","last_fullsized":"2024-03-06T07:21:54.356882+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:42.424597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358521+0000","last_change":"2024-03-06T07:21:54.358521+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358521+0000","last_undegraded":"2024-03-06T07:21:54.358521+0000","last_fullsized":"2024-03-06T07:21:54.358521+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368700+0000","last_change":"2024-03-06T07:21:43.326550+0000","last_active":"2024-03-06T07:21:54.368700+0000","last_peered":"2024-03-06T07:21:54.368700+0000","last_clean":"2024-03-06T07:21:54.368700+0000","last_became_active":"2024-03-06T07:21:43.326254+0000","last_became_peered":"2024-03-06T07:21:43.326254+0000","last_unstale":"2024-03-06T07:21:54.368700+0000","last_undegraded":"2024-03-06T07:21:54.368700+0000","last_fullsized":"2024-03-06T07:21:54.368700+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357560+0000","last_change":"2024-03-06T07:21:54.357560+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357560+0000","last_undegraded":"2024-03-06T07:21:54.357560+0000","last_fullsized":"2024-03-06T07:21:54.357560+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360266+0000","last_change":"2024-03-06T07:21:43.322773+0000","last_active":"2024-03-06T07:21:54.360266+0000","last_peered":"2024-03-06T07:21:54.360266+0000","last_clean":"2024-03-06T07:21:54.360266+0000","last_became_active":"2024-03-06T07:21:43.322525+0000","last_became_peered":"2024-03-06T07:21:43.322525+0000","last_unstale":"2024-03-06T07:21:54.360266+0000","last_undegraded":"2024-03-06T07:21:54.360266+0000","last_fullsized":"2024-03-06T07:21:54.360266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362616+0000","last_change":"2024-03-06T07:21:54.362616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362616+0000","last_undegraded":"2024-03-06T07:21:54.362616+0000","last_fullsized":"2024-03-06T07:21:54.362616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376862+0000","last_change":"2024-03-06T07:21:54.366905+0000","last_active":"2024-03-06T07:21:54.376862+0000","last_peered":"2024-03-06T07:21:54.376862+0000","last_clean":"2024-03-06T07:21:54.376862+0000","last_became_active":"2024-03-06T07:21:54.366627+0000","last_became_peered":"2024-03-06T07:21:54.366627+0000","last_unstale":"2024-03-06T07:21:54.376862+0000","last_undegraded":"2024-03-06T07:21:54.376862+0000","last_fullsized":"2024-03-06T07:21:54.376862+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356615+0000","last_change":"2024-03-06T07:21:43.319837+0000","last_active":"2024-03-06T07:21:54.356615+0000","last_peered":"2024-03-06T07:21:54.356615+0000","last_clean":"2024-03-06T07:21:54.356615+0000","last_became_active":"2024-03-06T07:21:43.319564+0000","last_became_peered":"2024-03-06T07:21:43.319564+0000","last_unstale":"2024-03-06T07:21:54.356615+0000","last_undegraded":"2024-03-06T07:21:54.356615+0000","last_fullsized":"2024-03-06T07:21:54.356615+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365243+0000","last_change":"2024-03-06T07:21:42.310584+0000","last_active":"2024-03-06T07:21:54.365243+0000","last_peered":"2024-03-06T07:21:54.365243+0000","last_clean":"2024-03-06T07:21:54.365243+0000","last_became_active":"2024-03-06T07:21:42.310317+0000","last_became_peered":"2024-03-06T07:21:42.310317+0000","last_unstale":"2024-03-06T07:21:54.365243+0000","last_undegraded":"2024-03-06T07:21:54.365243+0000","last_fullsized":"2024-03-06T07:21:54.365243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362416+0000","last_change":"2024-03-06T07:21:54.362416+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362416+0000","last_undegraded":"2024-03-06T07:21:54.362416+0000","last_fullsized":"2024-03-06T07:21:54.362416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380180+0000","last_change":"2024-03-06T07:21:54.377542+0000","last_active":"2024-03-06T07:21:54.380180+0000","last_peered":"2024-03-06T07:21:54.380180+0000","last_clean":"2024-03-06T07:21:54.380180+0000","last_became_active":"2024-03-06T07:21:54.377250+0000","last_became_peered":"2024-03-06T07:21:54.377250+0000","last_unstale":"2024-03-06T07:21:54.380180+0000","last_undegraded":"2024-03-06T07:21:54.380180+0000","last_fullsized":"2024-03-06T07:21:54.380180+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:36.750007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359472+0000","last_change":"2024-03-06T07:21:43.323883+0000","last_active":"2024-03-06T07:21:54.359472+0000","last_peered":"2024-03-06T07:21:54.359472+0000","last_clean":"2024-03-06T07:21:54.359472+0000","last_became_active":"2024-03-06T07:21:43.323613+0000","last_became_peered":"2024-03-06T07:21:43.323613+0000","last_unstale":"2024-03-06T07:21:54.359472+0000","last_undegraded":"2024-03-06T07:21:54.359472+0000","last_fullsized":"2024-03-06T07:21:54.359472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370119+0000","last_change":"2024-03-06T07:21:42.304199+0000","last_active":"2024-03-06T07:21:54.370119+0000","last_peered":"2024-03-06T07:21:54.370119+0000","last_clean":"2024-03-06T07:21:54.370119+0000","last_became_active":"2024-03-06T07:21:42.303802+0000","last_became_peered":"2024-03-06T07:21:42.303802+0000","last_unstale":"2024-03-06T07:21:54.370119+0000","last_undegraded":"2024-03-06T07:21:54.370119+0000","last_fullsized":"2024-03-06T07:21:54.370119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362959+0000","last_change":"2024-03-06T07:21:54.362959+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362959+0000","last_undegraded":"2024-03-06T07:21:54.362959+0000","last_fullsized":"2024-03-06T07:21:54.362959+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376955+0000","last_change":"2024-03-06T07:21:54.368687+0000","last_active":"2024-03-06T07:21:54.376955+0000","last_peered":"2024-03-06T07:21:54.376955+0000","last_clean":"2024-03-06T07:21:54.376955+0000","last_became_active":"2024-03-06T07:21:54.368106+0000","last_became_peered":"2024-03-06T07:21:54.368106+0000","last_unstale":"2024-03-06T07:21:54.376955+0000","last_undegraded":"2024-03-06T07:21:54.376955+0000","last_fullsized":"2024-03-06T07:21:54.376955+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364372+0000","last_change":"2024-03-06T07:21:43.323512+0000","last_active":"2024-03-06T07:21:54.364372+0000","last_peered":"2024-03-06T07:21:54.364372+0000","last_clean":"2024-03-06T07:21:54.364372+0000","last_became_active":"2024-03-06T07:21:43.323216+0000","last_became_peered":"2024-03-06T07:21:43.323216+0000","last_unstale":"2024-03-06T07:21:54.364372+0000","last_undegraded":"2024-03-06T07:21:54.364372+0000","last_fullsized":"2024-03-06T07:21:54.364372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:31.821522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369735+0000","last_change":"2024-03-06T07:21:42.305711+0000","last_active":"2024-03-06T07:21:54.369735+0000","last_peered":"2024-03-06T07:21:54.369735+0000","last_clean":"2024-03-06T07:21:54.369735+0000","last_became_active":"2024-03-06T07:21:42.305431+0000","last_became_peered":"2024-03-06T07:21:42.305431+0000","last_unstale":"2024-03-06T07:21:54.369735+0000","last_undegraded":"2024-03-06T07:21:54.369735+0000","last_fullsized":"2024-03-06T07:21:54.369735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357254+0000","last_change":"2024-03-06T07:21:54.357254+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357254+0000","last_undegraded":"2024-03-06T07:21:54.357254+0000","last_fullsized":"2024-03-06T07:21:54.357254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374713+0000","last_change":"2024-03-06T07:21:54.370174+0000","last_active":"2024-03-06T07:21:54.374713+0000","last_peered":"2024-03-06T07:21:54.374713+0000","last_clean":"2024-03-06T07:21:54.374713+0000","last_became_active":"2024-03-06T07:21:54.369897+0000","last_became_peered":"2024-03-06T07:21:54.369897+0000","last_unstale":"2024-03-06T07:21:54.374713+0000","last_undegraded":"2024-03-06T07:21:54.374713+0000","last_fullsized":"2024-03-06T07:21:54.374713+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359478+0000","last_change":"2024-03-06T07:21:43.317966+0000","last_active":"2024-03-06T07:21:54.359478+0000","last_peered":"2024-03-06T07:21:54.359478+0000","last_clean":"2024-03-06T07:21:54.359478+0000","last_became_active":"2024-03-06T07:21:43.317700+0000","last_became_peered":"2024-03-06T07:21:43.317700+0000","last_unstale":"2024-03-06T07:21:54.359478+0000","last_undegraded":"2024-03-06T07:21:54.359478+0000","last_fullsized":"2024-03-06T07:21:54.359478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359768+0000","last_change":"2024-03-06T07:21:42.315045+0000","last_active":"2024-03-06T07:21:54.359768+0000","last_peered":"2024-03-06T07:21:54.359768+0000","last_clean":"2024-03-06T07:21:54.359768+0000","last_became_active":"2024-03-06T07:21:42.314768+0000","last_became_peered":"2024-03-06T07:21:42.314768+0000","last_unstale":"2024-03-06T07:21:54.359768+0000","last_undegraded":"2024-03-06T07:21:54.359768+0000","last_fullsized":"2024-03-06T07:21:54.359768+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:14.953476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360237+0000","last_change":"2024-03-06T07:21:54.360237+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360237+0000","last_undegraded":"2024-03-06T07:21:54.360237+0000","last_fullsized":"2024-03-06T07:21:54.360237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379825+0000","last_change":"2024-03-06T07:21:54.375783+0000","last_active":"2024-03-06T07:21:54.379825+0000","last_peered":"2024-03-06T07:21:54.379825+0000","last_clean":"2024-03-06T07:21:54.379825+0000","last_became_active":"2024-03-06T07:21:54.375473+0000","last_became_peered":"2024-03-06T07:21:54.375473+0000","last_unstale":"2024-03-06T07:21:54.379825+0000","last_undegraded":"2024-03-06T07:21:54.379825+0000","last_fullsized":"2024-03-06T07:21:54.379825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:08.612677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370306+0000","last_change":"2024-03-06T07:21:50.343971+0000","last_active":"2024-03-06T07:21:54.370306+0000","last_peered":"2024-03-06T07:21:54.370306+0000","last_clean":"2024-03-06T07:21:54.370306+0000","last_became_active":"2024-03-06T07:21:50.343658+0000","last_became_peered":"2024-03-06T07:21:50.343658+0000","last_unstale":"2024-03-06T07:21:54.370306+0000","last_undegraded":"2024-03-06T07:21:54.370306+0000","last_fullsized":"2024-03-06T07:21:54.370306+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364922+0000","last_change":"2024-03-06T07:21:42.317614+0000","last_active":"2024-03-06T07:21:54.364922+0000","last_peered":"2024-03-06T07:21:54.364922+0000","last_clean":"2024-03-06T07:21:54.364922+0000","last_became_active":"2024-03-06T07:21:42.317094+0000","last_became_peered":"2024-03-06T07:21:42.317094+0000","last_unstale":"2024-03-06T07:21:54.364922+0000","last_undegraded":"2024-03-06T07:21:54.364922+0000","last_fullsized":"2024-03-06T07:21:54.364922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:15.739191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364680+0000","last_change":"2024-03-06T07:21:54.364680+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364680+0000","last_undegraded":"2024-03-06T07:21:54.364680+0000","last_fullsized":"2024-03-06T07:21:54.364680+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377152+0000","last_change":"2024-03-06T07:21:54.375914+0000","last_active":"2024-03-06T07:21:54.377152+0000","last_peered":"2024-03-06T07:21:54.377152+0000","last_clean":"2024-03-06T07:21:54.377152+0000","last_became_active":"2024-03-06T07:21:54.375619+0000","last_became_peered":"2024-03-06T07:21:54.375619+0000","last_unstale":"2024-03-06T07:21:54.377152+0000","last_undegraded":"2024-03-06T07:21:54.377152+0000","last_fullsized":"2024-03-06T07:21:54.377152+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:26.719365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358617+0000","last_change":"2024-03-06T07:21:43.318727+0000","last_active":"2024-03-06T07:21:54.358617+0000","last_peered":"2024-03-06T07:21:54.358617+0000","last_clean":"2024-03-06T07:21:54.358617+0000","last_became_active":"2024-03-06T07:21:43.318440+0000","last_became_peered":"2024-03-06T07:21:43.318440+0000","last_unstale":"2024-03-06T07:21:54.358617+0000","last_undegraded":"2024-03-06T07:21:54.358617+0000","last_fullsized":"2024-03-06T07:21:54.358617+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370222+0000","last_change":"2024-03-06T07:21:42.307726+0000","last_active":"2024-03-06T07:21:54.370222+0000","last_peered":"2024-03-06T07:21:54.370222+0000","last_clean":"2024-03-06T07:21:54.370222+0000","last_became_active":"2024-03-06T07:21:42.307445+0000","last_became_peered":"2024-03-06T07:21:42.307445+0000","last_unstale":"2024-03-06T07:21:54.370222+0000","last_undegraded":"2024-03-06T07:21:54.370222+0000","last_fullsized":"2024-03-06T07:21:54.370222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366515+0000","last_change":"2024-03-06T07:21:54.366515+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366515+0000","last_undegraded":"2024-03-06T07:21:54.366515+0000","last_fullsized":"2024-03-06T07:21:54.366515+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376980+0000","last_change":"2024-03-06T07:21:54.370955+0000","last_active":"2024-03-06T07:21:54.376980+0000","last_peered":"2024-03-06T07:21:54.376980+0000","last_clean":"2024-03-06T07:21:54.376980+0000","last_became_active":"2024-03-06T07:21:54.370680+0000","last_became_peered":"2024-03-06T07:21:54.370680+0000","last_unstale":"2024-03-06T07:21:54.376980+0000","last_undegraded":"2024-03-06T07:21:54.376980+0000","last_fullsized":"2024-03-06T07:21:54.376980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359348+0000","last_change":"2024-03-06T07:21:43.318372+0000","last_active":"2024-03-06T07:21:54.359348+0000","last_peered":"2024-03-06T07:21:54.359348+0000","last_clean":"2024-03-06T07:21:54.359348+0000","last_became_active":"2024-03-06T07:21:43.318047+0000","last_became_peered":"2024-03-06T07:21:43.318047+0000","last_unstale":"2024-03-06T07:21:54.359348+0000","last_undegraded":"2024-03-06T07:21:54.359348+0000","last_fullsized":"2024-03-06T07:21:54.359348+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364556+0000","last_change":"2024-03-06T07:21:42.310677+0000","last_active":"2024-03-06T07:21:54.364556+0000","last_peered":"2024-03-06T07:21:54.364556+0000","last_clean":"2024-03-06T07:21:54.364556+0000","last_became_active":"2024-03-06T07:21:42.310375+0000","last_became_peered":"2024-03-06T07:21:42.310375+0000","last_unstale":"2024-03-06T07:21:54.364556+0000","last_undegraded":"2024-03-06T07:21:54.364556+0000","last_fullsized":"2024-03-06T07:21:54.364556+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:13.837451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358658+0000","last_change":"2024-03-06T07:21:54.358658+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358658+0000","last_undegraded":"2024-03-06T07:21:54.358658+0000","last_fullsized":"2024-03-06T07:21:54.358658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379656+0000","last_change":"2024-03-06T07:21:54.374914+0000","last_active":"2024-03-06T07:21:54.379656+0000","last_peered":"2024-03-06T07:21:54.379656+0000","last_clean":"2024-03-06T07:21:54.379656+0000","last_became_active":"2024-03-06T07:21:54.374570+0000","last_became_peered":"2024-03-06T07:21:54.374570+0000","last_unstale":"2024-03-06T07:21:54.379656+0000","last_undegraded":"2024-03-06T07:21:54.379656+0000","last_fullsized":"2024-03-06T07:21:54.379656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:05:03.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358220+0000","last_change":"2024-03-06T07:21:43.325740+0000","last_active":"2024-03-06T07:21:54.358220+0000","last_peered":"2024-03-06T07:21:54.358220+0000","last_clean":"2024-03-06T07:21:54.358220+0000","last_became_active":"2024-03-06T07:21:43.325425+0000","last_became_peered":"2024-03-06T07:21:43.325425+0000","last_unstale":"2024-03-06T07:21:54.358220+0000","last_undegraded":"2024-03-06T07:21:54.358220+0000","last_fullsized":"2024-03-06T07:21:54.358220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364895+0000","last_change":"2024-03-06T07:21:42.319786+0000","last_active":"2024-03-06T07:21:54.364895+0000","last_peered":"2024-03-06T07:21:54.364895+0000","last_clean":"2024-03-06T07:21:54.364895+0000","last_became_active":"2024-03-06T07:21:42.319430+0000","last_became_peered":"2024-03-06T07:21:42.319430+0000","last_unstale":"2024-03-06T07:21:54.364895+0000","last_undegraded":"2024-03-06T07:21:54.364895+0000","last_fullsized":"2024-03-06T07:21:54.364895+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:09.719754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360176+0000","last_change":"2024-03-06T07:21:54.360176+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360176+0000","last_undegraded":"2024-03-06T07:21:54.360176+0000","last_fullsized":"2024-03-06T07:21:54.360176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369499+0000","last_change":"2024-03-06T07:21:54.367800+0000","last_active":"2024-03-06T07:21:54.369499+0000","last_peered":"2024-03-06T07:21:54.369499+0000","last_clean":"2024-03-06T07:21:54.369499+0000","last_became_active":"2024-03-06T07:21:54.367468+0000","last_became_peered":"2024-03-06T07:21:54.367468+0000","last_unstale":"2024-03-06T07:21:54.369499+0000","last_undegraded":"2024-03-06T07:21:54.369499+0000","last_fullsized":"2024-03-06T07:21:54.369499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.330805+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.330267+0000","last_became_peered":"2024-03-06T07:21:43.330267+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:56.716578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358318+0000","last_change":"2024-03-06T07:21:42.312213+0000","last_active":"2024-03-06T07:21:54.358318+0000","last_peered":"2024-03-06T07:21:54.358318+0000","last_clean":"2024-03-06T07:21:54.358318+0000","last_became_active":"2024-03-06T07:21:42.311878+0000","last_became_peered":"2024-03-06T07:21:42.311878+0000","last_unstale":"2024-03-06T07:21:54.358318+0000","last_undegraded":"2024-03-06T07:21:54.358318+0000","last_fullsized":"2024-03-06T07:21:54.358318+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358914+0000","last_change":"2024-03-06T07:21:54.358914+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358914+0000","last_undegraded":"2024-03-06T07:21:54.358914+0000","last_fullsized":"2024-03-06T07:21:54.358914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380052+0000","last_change":"2024-03-06T07:21:54.377646+0000","last_active":"2024-03-06T07:21:54.380052+0000","last_peered":"2024-03-06T07:21:54.380052+0000","last_clean":"2024-03-06T07:21:54.380052+0000","last_became_active":"2024-03-06T07:21:54.377219+0000","last_became_peered":"2024-03-06T07:21:54.377219+0000","last_unstale":"2024-03-06T07:21:54.380052+0000","last_undegraded":"2024-03-06T07:21:54.380052+0000","last_fullsized":"2024-03-06T07:21:54.380052+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:59.768227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:43.320122+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:43.319799+0000","last_became_peered":"2024-03-06T07:21:43.319799+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370295+0000","last_change":"2024-03-06T07:21:42.319681+0000","last_active":"2024-03-06T07:21:54.370295+0000","last_peered":"2024-03-06T07:21:54.370295+0000","last_clean":"2024-03-06T07:21:54.370295+0000","last_became_active":"2024-03-06T07:21:42.319452+0000","last_became_peered":"2024-03-06T07:21:42.319452+0000","last_unstale":"2024-03-06T07:21:54.370295+0000","last_undegraded":"2024-03-06T07:21:54.370295+0000","last_fullsized":"2024-03-06T07:21:54.370295+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:35.308790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362688+0000","last_change":"2024-03-06T07:21:54.362688+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362688+0000","last_undegraded":"2024-03-06T07:21:54.362688+0000","last_fullsized":"2024-03-06T07:21:54.362688+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368199+0000","last_change":"2024-03-06T07:21:54.367115+0000","last_active":"2024-03-06T07:21:54.368199+0000","last_peered":"2024-03-06T07:21:54.368199+0000","last_clean":"2024-03-06T07:21:54.368199+0000","last_became_active":"2024-03-06T07:21:54.366564+0000","last_became_peered":"2024-03-06T07:21:54.366564+0000","last_unstale":"2024-03-06T07:21:54.368199+0000","last_undegraded":"2024-03-06T07:21:54.368199+0000","last_fullsized":"2024-03-06T07:21:54.368199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365114+0000","last_change":"2024-03-06T07:21:43.323490+0000","last_active":"2024-03-06T07:21:54.365114+0000","last_peered":"2024-03-06T07:21:54.365114+0000","last_clean":"2024-03-06T07:21:54.365114+0000","last_became_active":"2024-03-06T07:21:43.323088+0000","last_became_peered":"2024-03-06T07:21:43.323088+0000","last_unstale":"2024-03-06T07:21:54.365114+0000","last_undegraded":"2024-03-06T07:21:54.365114+0000","last_fullsized":"2024-03-06T07:21:54.365114+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358795+0000","last_change":"2024-03-06T07:21:42.312732+0000","last_active":"2024-03-06T07:21:54.358795+0000","last_peered":"2024-03-06T07:21:54.358795+0000","last_clean":"2024-03-06T07:21:54.358795+0000","last_became_active":"2024-03-06T07:21:42.312462+0000","last_became_peered":"2024-03-06T07:21:42.312462+0000","last_unstale":"2024-03-06T07:21:54.358795+0000","last_undegraded":"2024-03-06T07:21:54.358795+0000","last_fullsized":"2024-03-06T07:21:54.358795+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363414+0000","last_change":"2024-03-06T07:21:54.363414+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363414+0000","last_undegraded":"2024-03-06T07:21:54.363414+0000","last_fullsized":"2024-03-06T07:21:54.363414+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367522+0000","last_change":"2024-03-06T07:21:54.365558+0000","last_active":"2024-03-06T07:21:54.367522+0000","last_peered":"2024-03-06T07:21:54.367522+0000","last_clean":"2024-03-06T07:21:54.367522+0000","last_became_active":"2024-03-06T07:21:54.365272+0000","last_became_peered":"2024-03-06T07:21:54.365272+0000","last_unstale":"2024-03-06T07:21:54.367522+0000","last_undegraded":"2024-03-06T07:21:54.367522+0000","last_fullsized":"2024-03-06T07:21:54.367522+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369842+0000","last_change":"2024-03-06T07:21:43.320022+0000","last_active":"2024-03-06T07:21:54.369842+0000","last_peered":"2024-03-06T07:21:54.369842+0000","last_clean":"2024-03-06T07:21:54.369842+0000","last_became_active":"2024-03-06T07:21:43.319716+0000","last_became_peered":"2024-03-06T07:21:43.319716+0000","last_unstale":"2024-03-06T07:21:54.369842+0000","last_undegraded":"2024-03-06T07:21:54.369842+0000","last_fullsized":"2024-03-06T07:21:54.369842+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:50.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359596+0000","last_change":"2024-03-06T07:21:42.308298+0000","last_active":"2024-03-06T07:21:54.359596+0000","last_peered":"2024-03-06T07:21:54.359596+0000","last_clean":"2024-03-06T07:21:54.359596+0000","last_became_active":"2024-03-06T07:21:42.308008+0000","last_became_peered":"2024-03-06T07:21:42.308008+0000","last_unstale":"2024-03-06T07:21:54.359596+0000","last_undegraded":"2024-03-06T07:21:54.359596+0000","last_fullsized":"2024-03-06T07:21:54.359596+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:51.723259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361630+0000","last_change":"2024-03-06T07:21:54.361630+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361630+0000","last_undegraded":"2024-03-06T07:21:54.361630+0000","last_fullsized":"2024-03-06T07:21:54.361630+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376550+0000","last_change":"2024-03-06T07:21:54.375937+0000","last_active":"2024-03-06T07:21:54.376550+0000","last_peered":"2024-03-06T07:21:54.376550+0000","last_clean":"2024-03-06T07:21:54.376550+0000","last_became_active":"2024-03-06T07:21:54.375648+0000","last_became_peered":"2024-03-06T07:21:54.375648+0000","last_unstale":"2024-03-06T07:21:54.376550+0000","last_undegraded":"2024-03-06T07:21:54.376550+0000","last_fullsized":"2024-03-06T07:21:54.376550+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359583+0000","last_change":"2024-03-06T07:21:43.323228+0000","last_active":"2024-03-06T07:21:54.359583+0000","last_peered":"2024-03-06T07:21:54.359583+0000","last_clean":"2024-03-06T07:21:54.359583+0000","last_became_active":"2024-03-06T07:21:43.322951+0000","last_became_peered":"2024-03-06T07:21:43.322951+0000","last_unstale":"2024-03-06T07:21:54.359583+0000","last_undegraded":"2024-03-06T07:21:54.359583+0000","last_fullsized":"2024-03-06T07:21:54.359583+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365013+0000","last_change":"2024-03-06T07:21:42.321872+0000","last_active":"2024-03-06T07:21:54.365013+0000","last_peered":"2024-03-06T07:21:54.365013+0000","last_clean":"2024-03-06T07:21:54.365013+0000","last_became_active":"2024-03-06T07:21:42.321384+0000","last_became_peered":"2024-03-06T07:21:42.321384+0000","last_unstale":"2024-03-06T07:21:54.365013+0000","last_undegraded":"2024-03-06T07:21:54.365013+0000","last_fullsized":"2024-03-06T07:21:54.365013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358625+0000","last_change":"2024-03-06T07:21:54.358625+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358625+0000","last_undegraded":"2024-03-06T07:21:54.358625+0000","last_fullsized":"2024-03-06T07:21:54.358625+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375601+0000","last_change":"2024-03-06T07:21:54.373488+0000","last_active":"2024-03-06T07:21:54.375601+0000","last_peered":"2024-03-06T07:21:54.375601+0000","last_clean":"2024-03-06T07:21:54.375601+0000","last_became_active":"2024-03-06T07:21:54.373225+0000","last_became_peered":"2024-03-06T07:21:54.373225+0000","last_unstale":"2024-03-06T07:21:54.375601+0000","last_undegraded":"2024-03-06T07:21:54.375601+0000","last_fullsized":"2024-03-06T07:21:54.375601+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369356+0000","last_change":"2024-03-06T07:21:43.324244+0000","last_active":"2024-03-06T07:21:54.369356+0000","last_peered":"2024-03-06T07:21:54.369356+0000","last_clean":"2024-03-06T07:21:54.369356+0000","last_became_active":"2024-03-06T07:21:43.323549+0000","last_became_peered":"2024-03-06T07:21:43.323549+0000","last_unstale":"2024-03-06T07:21:54.369356+0000","last_undegraded":"2024-03-06T07:21:54.369356+0000","last_fullsized":"2024-03-06T07:21:54.369356+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364546+0000","last_change":"2024-03-06T07:21:42.318308+0000","last_active":"2024-03-06T07:21:54.364546+0000","last_peered":"2024-03-06T07:21:54.364546+0000","last_clean":"2024-03-06T07:21:54.364546+0000","last_became_active":"2024-03-06T07:21:42.318080+0000","last_became_peered":"2024-03-06T07:21:42.318080+0000","last_unstale":"2024-03-06T07:21:54.364546+0000","last_undegraded":"2024-03-06T07:21:54.364546+0000","last_fullsized":"2024-03-06T07:21:54.364546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360511+0000","last_change":"2024-03-06T07:21:54.360511+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360511+0000","last_undegraded":"2024-03-06T07:21:54.360511+0000","last_fullsized":"2024-03-06T07:21:54.360511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378458+0000","last_change":"2024-03-06T07:21:54.377690+0000","last_active":"2024-03-06T07:21:54.378458+0000","last_peered":"2024-03-06T07:21:54.378458+0000","last_clean":"2024-03-06T07:21:54.378458+0000","last_became_active":"2024-03-06T07:21:54.377408+0000","last_became_peered":"2024-03-06T07:21:54.377408+0000","last_unstale":"2024-03-06T07:21:54.378458+0000","last_undegraded":"2024-03-06T07:21:54.378458+0000","last_fullsized":"2024-03-06T07:21:54.378458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:31.244976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359542+0000","last_change":"2024-03-06T07:21:43.331853+0000","last_active":"2024-03-06T07:21:54.359542+0000","last_peered":"2024-03-06T07:21:54.359542+0000","last_clean":"2024-03-06T07:21:54.359542+0000","last_became_active":"2024-03-06T07:21:43.331560+0000","last_became_peered":"2024-03-06T07:21:43.331560+0000","last_unstale":"2024-03-06T07:21:54.359542+0000","last_undegraded":"2024-03-06T07:21:54.359542+0000","last_fullsized":"2024-03-06T07:21:54.359542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357780+0000","last_change":"2024-03-06T07:21:42.309167+0000","last_active":"2024-03-06T07:21:54.357780+0000","last_peered":"2024-03-06T07:21:54.357780+0000","last_clean":"2024-03-06T07:21:54.357780+0000","last_became_active":"2024-03-06T07:21:42.308882+0000","last_became_peered":"2024-03-06T07:21:42.308882+0000","last_unstale":"2024-03-06T07:21:54.357780+0000","last_undegraded":"2024-03-06T07:21:54.357780+0000","last_fullsized":"2024-03-06T07:21:54.357780+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366074+0000","last_change":"2024-03-06T07:21:54.366074+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366074+0000","last_undegraded":"2024-03-06T07:21:54.366074+0000","last_fullsized":"2024-03-06T07:21:54.366074+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378021+0000","last_change":"2024-03-06T07:21:54.377427+0000","last_active":"2024-03-06T07:21:54.378021+0000","last_peered":"2024-03-06T07:21:54.378021+0000","last_clean":"2024-03-06T07:21:54.378021+0000","last_became_active":"2024-03-06T07:21:54.377183+0000","last_became_peered":"2024-03-06T07:21:54.377183+0000","last_unstale":"2024-03-06T07:21:54.378021+0000","last_undegraded":"2024-03-06T07:21:54.378021+0000","last_fullsized":"2024-03-06T07:21:54.378021+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.324536+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.323818+0000","last_became_peered":"2024-03-06T07:21:43.323818+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358547+0000","last_change":"2024-03-06T07:21:42.308460+0000","last_active":"2024-03-06T07:21:54.358547+0000","last_peered":"2024-03-06T07:21:54.358547+0000","last_clean":"2024-03-06T07:21:54.358547+0000","last_became_active":"2024-03-06T07:21:42.307461+0000","last_became_peered":"2024-03-06T07:21:42.307461+0000","last_unstale":"2024-03-06T07:21:54.358547+0000","last_undegraded":"2024-03-06T07:21:54.358547+0000","last_fullsized":"2024-03-06T07:21:54.358547+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:34.346180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364356+0000","last_change":"2024-03-06T07:21:54.364356+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364356+0000","last_undegraded":"2024-03-06T07:21:54.364356+0000","last_fullsized":"2024-03-06T07:21:54.364356+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367923+0000","last_change":"2024-03-06T07:21:54.366324+0000","last_active":"2024-03-06T07:21:54.367923+0000","last_peered":"2024-03-06T07:21:54.367923+0000","last_clean":"2024-03-06T07:21:54.367923+0000","last_became_active":"2024-03-06T07:21:54.366064+0000","last_became_peered":"2024-03-06T07:21:54.366064+0000","last_unstale":"2024-03-06T07:21:54.367923+0000","last_undegraded":"2024-03-06T07:21:54.367923+0000","last_fullsized":"2024-03-06T07:21:54.367923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358427+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.358427+0000","last_peered":"2024-03-06T07:21:54.358427+0000","last_clean":"2024-03-06T07:21:54.358427+0000","last_became_active":"2024-03-06T07:21:43.316848+0000","last_became_peered":"2024-03-06T07:21:43.316848+0000","last_unstale":"2024-03-06T07:21:54.358427+0000","last_undegraded":"2024-03-06T07:21:54.358427+0000","last_fullsized":"2024-03-06T07:21:54.358427+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370393+0000","last_change":"2024-03-06T07:21:42.319998+0000","last_active":"2024-03-06T07:21:54.370393+0000","last_peered":"2024-03-06T07:21:54.370393+0000","last_clean":"2024-03-06T07:21:54.370393+0000","last_became_active":"2024-03-06T07:21:42.319529+0000","last_became_peered":"2024-03-06T07:21:42.319529+0000","last_unstale":"2024-03-06T07:21:54.370393+0000","last_undegraded":"2024-03-06T07:21:54.370393+0000","last_fullsized":"2024-03-06T07:21:54.370393+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361900+0000","last_change":"2024-03-06T07:21:54.361900+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361900+0000","last_undegraded":"2024-03-06T07:21:54.361900+0000","last_fullsized":"2024-03-06T07:21:54.361900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374081+0000","last_change":"2024-03-06T07:21:54.370046+0000","last_active":"2024-03-06T07:21:54.374081+0000","last_peered":"2024-03-06T07:21:54.374081+0000","last_clean":"2024-03-06T07:21:54.374081+0000","last_became_active":"2024-03-06T07:21:54.369791+0000","last_became_peered":"2024-03-06T07:21:54.369791+0000","last_unstale":"2024-03-06T07:21:54.374081+0000","last_undegraded":"2024-03-06T07:21:54.374081+0000","last_fullsized":"2024-03-06T07:21:54.374081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:09.373432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367145+0000","last_change":"2024-03-06T07:21:43.327074+0000","last_active":"2024-03-06T07:21:54.367145+0000","last_peered":"2024-03-06T07:21:54.367145+0000","last_clean":"2024-03-06T07:21:54.367145+0000","last_became_active":"2024-03-06T07:21:43.326797+0000","last_became_peered":"2024-03-06T07:21:43.326797+0000","last_unstale":"2024-03-06T07:21:54.367145+0000","last_undegraded":"2024-03-06T07:21:54.367145+0000","last_fullsized":"2024-03-06T07:21:54.367145+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370126+0000","last_change":"2024-03-06T07:21:42.310454+0000","last_active":"2024-03-06T07:21:54.370126+0000","last_peered":"2024-03-06T07:21:54.370126+0000","last_clean":"2024-03-06T07:21:54.370126+0000","last_became_active":"2024-03-06T07:21:42.310172+0000","last_became_peered":"2024-03-06T07:21:42.310172+0000","last_unstale":"2024-03-06T07:21:54.370126+0000","last_undegraded":"2024-03-06T07:21:54.370126+0000","last_fullsized":"2024-03-06T07:21:54.370126+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:34.182536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360981+0000","last_change":"2024-03-06T07:21:54.360981+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360981+0000","last_undegraded":"2024-03-06T07:21:54.360981+0000","last_fullsized":"2024-03-06T07:21:54.360981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376984+0000","last_change":"2024-03-06T07:21:54.368897+0000","last_active":"2024-03-06T07:21:54.376984+0000","last_peered":"2024-03-06T07:21:54.376984+0000","last_clean":"2024-03-06T07:21:54.376984+0000","last_became_active":"2024-03-06T07:21:54.368398+0000","last_became_peered":"2024-03-06T07:21:54.368398+0000","last_unstale":"2024-03-06T07:21:54.376984+0000","last_undegraded":"2024-03-06T07:21:54.376984+0000","last_fullsized":"2024-03-06T07:21:54.376984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369221+0000","last_change":"2024-03-06T07:21:43.324398+0000","last_active":"2024-03-06T07:21:54.369221+0000","last_peered":"2024-03-06T07:21:54.369221+0000","last_clean":"2024-03-06T07:21:54.369221+0000","last_became_active":"2024-03-06T07:21:43.324115+0000","last_became_peered":"2024-03-06T07:21:43.324115+0000","last_unstale":"2024-03-06T07:21:54.369221+0000","last_undegraded":"2024-03-06T07:21:54.369221+0000","last_fullsized":"2024-03-06T07:21:54.369221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:36.888023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365848+0000","last_change":"2024-03-06T07:21:42.312390+0000","last_active":"2024-03-06T07:21:54.365848+0000","last_peered":"2024-03-06T07:21:54.365848+0000","last_clean":"2024-03-06T07:21:54.365848+0000","last_became_active":"2024-03-06T07:21:42.312079+0000","last_became_peered":"2024-03-06T07:21:42.312079+0000","last_unstale":"2024-03-06T07:21:54.365848+0000","last_undegraded":"2024-03-06T07:21:54.365848+0000","last_fullsized":"2024-03-06T07:21:54.365848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364238+0000","last_change":"2024-03-06T07:21:54.364238+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364238+0000","last_undegraded":"2024-03-06T07:21:54.364238+0000","last_fullsized":"2024-03-06T07:21:54.364238+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372534+0000","last_change":"2024-03-06T07:21:54.370872+0000","last_active":"2024-03-06T07:21:54.372534+0000","last_peered":"2024-03-06T07:21:54.372534+0000","last_clean":"2024-03-06T07:21:54.372534+0000","last_became_active":"2024-03-06T07:21:54.370589+0000","last_became_peered":"2024-03-06T07:21:54.370589+0000","last_unstale":"2024-03-06T07:21:54.372534+0000","last_undegraded":"2024-03-06T07:21:54.372534+0000","last_fullsized":"2024-03-06T07:21:54.372534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:36.087274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368542+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.368542+0000","last_peered":"2024-03-06T07:21:54.368542+0000","last_clean":"2024-03-06T07:21:54.368542+0000","last_became_active":"2024-03-06T07:21:43.316857+0000","last_became_peered":"2024-03-06T07:21:43.316857+0000","last_unstale":"2024-03-06T07:21:54.368542+0000","last_undegraded":"2024-03-06T07:21:54.368542+0000","last_fullsized":"2024-03-06T07:21:54.368542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:48.715734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365550+0000","last_change":"2024-03-06T07:21:42.311972+0000","last_active":"2024-03-06T07:21:54.365550+0000","last_peered":"2024-03-06T07:21:54.365550+0000","last_clean":"2024-03-06T07:21:54.365550+0000","last_became_active":"2024-03-06T07:21:42.311640+0000","last_became_peered":"2024-03-06T07:21:42.311640+0000","last_unstale":"2024-03-06T07:21:54.365550+0000","last_undegraded":"2024-03-06T07:21:54.365550+0000","last_fullsized":"2024-03-06T07:21:54.365550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:22.568258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363210+0000","last_change":"2024-03-06T07:21:54.363210+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363210+0000","last_undegraded":"2024-03-06T07:21:54.363210+0000","last_fullsized":"2024-03-06T07:21:54.363210+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375823+0000","last_change":"2024-03-06T07:21:54.372740+0000","last_active":"2024-03-06T07:21:54.375823+0000","last_peered":"2024-03-06T07:21:54.375823+0000","last_clean":"2024-03-06T07:21:54.375823+0000","last_became_active":"2024-03-06T07:21:54.371525+0000","last_became_peered":"2024-03-06T07:21:54.371525+0000","last_unstale":"2024-03-06T07:21:54.375823+0000","last_undegraded":"2024-03-06T07:21:54.375823+0000","last_fullsized":"2024-03-06T07:21:54.375823+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359675+0000","last_change":"2024-03-06T07:21:43.320374+0000","last_active":"2024-03-06T07:21:54.359675+0000","last_peered":"2024-03-06T07:21:54.359675+0000","last_clean":"2024-03-06T07:21:54.359675+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.359675+0000","last_undegraded":"2024-03-06T07:21:54.359675+0000","last_fullsized":"2024-03-06T07:21:54.359675+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368677+0000","last_change":"2024-03-06T07:21:42.309901+0000","last_active":"2024-03-06T07:21:54.368677+0000","last_peered":"2024-03-06T07:21:54.368677+0000","last_clean":"2024-03-06T07:21:54.368677+0000","last_became_active":"2024-03-06T07:21:42.309532+0000","last_became_peered":"2024-03-06T07:21:42.309532+0000","last_unstale":"2024-03-06T07:21:54.368677+0000","last_undegraded":"2024-03-06T07:21:54.368677+0000","last_fullsized":"2024-03-06T07:21:54.368677+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361638+0000","last_change":"2024-03-06T07:21:54.361638+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361638+0000","last_undegraded":"2024-03-06T07:21:54.361638+0000","last_fullsized":"2024-03-06T07:21:54.361638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377134+0000","last_change":"2024-03-06T07:21:54.375852+0000","last_active":"2024-03-06T07:21:54.377134+0000","last_peered":"2024-03-06T07:21:54.377134+0000","last_clean":"2024-03-06T07:21:54.377134+0000","last_became_active":"2024-03-06T07:21:54.375600+0000","last_became_peered":"2024-03-06T07:21:54.375600+0000","last_unstale":"2024-03-06T07:21:54.377134+0000","last_undegraded":"2024-03-06T07:21:54.377134+0000","last_fullsized":"2024-03-06T07:21:54.377134+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:40.575618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365686+0000","last_change":"2024-03-06T07:21:43.320820+0000","last_active":"2024-03-06T07:21:54.365686+0000","last_peered":"2024-03-06T07:21:54.365686+0000","last_clean":"2024-03-06T07:21:54.365686+0000","last_became_active":"2024-03-06T07:21:43.319225+0000","last_became_peered":"2024-03-06T07:21:43.319225+0000","last_unstale":"2024-03-06T07:21:54.365686+0000","last_undegraded":"2024-03-06T07:21:54.365686+0000","last_fullsized":"2024-03-06T07:21:54.365686+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358917+0000","last_change":"2024-03-06T07:21:42.307676+0000","last_active":"2024-03-06T07:21:54.358917+0000","last_peered":"2024-03-06T07:21:54.358917+0000","last_clean":"2024-03-06T07:21:54.358917+0000","last_became_active":"2024-03-06T07:21:42.307388+0000","last_became_peered":"2024-03-06T07:21:42.307388+0000","last_unstale":"2024-03-06T07:21:54.358917+0000","last_undegraded":"2024-03-06T07:21:54.358917+0000","last_fullsized":"2024-03-06T07:21:54.358917+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:47.711911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361080+0000","last_change":"2024-03-06T07:21:54.361080+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361080+0000","last_undegraded":"2024-03-06T07:21:54.361080+0000","last_fullsized":"2024-03-06T07:21:54.361080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380012+0000","last_change":"2024-03-06T07:21:54.374823+0000","last_active":"2024-03-06T07:21:54.380012+0000","last_peered":"2024-03-06T07:21:54.380012+0000","last_clean":"2024-03-06T07:21:54.380012+0000","last_became_active":"2024-03-06T07:21:54.374515+0000","last_became_peered":"2024-03-06T07:21:54.374515+0000","last_unstale":"2024-03-06T07:21:54.380012+0000","last_undegraded":"2024-03-06T07:21:54.380012+0000","last_fullsized":"2024-03-06T07:21:54.380012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369756+0000","last_change":"2024-03-06T07:21:43.329841+0000","last_active":"2024-03-06T07:21:54.369756+0000","last_peered":"2024-03-06T07:21:54.369756+0000","last_clean":"2024-03-06T07:21:54.369756+0000","last_became_active":"2024-03-06T07:21:43.329526+0000","last_became_peered":"2024-03-06T07:21:43.329526+0000","last_unstale":"2024-03-06T07:21:54.369756+0000","last_undegraded":"2024-03-06T07:21:54.369756+0000","last_fullsized":"2024-03-06T07:21:54.369756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368782+0000","last_change":"2024-03-06T07:21:42.318168+0000","last_active":"2024-03-06T07:21:54.368782+0000","last_peered":"2024-03-06T07:21:54.368782+0000","last_clean":"2024-03-06T07:21:54.368782+0000","last_became_active":"2024-03-06T07:21:42.317866+0000","last_became_peered":"2024-03-06T07:21:42.317866+0000","last_unstale":"2024-03-06T07:21:54.368782+0000","last_undegraded":"2024-03-06T07:21:54.368782+0000","last_fullsized":"2024-03-06T07:21:54.368782+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:49.054000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361933+0000","last_change":"2024-03-06T07:21:54.361933+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361933+0000","last_undegraded":"2024-03-06T07:21:54.361933+0000","last_fullsized":"2024-03-06T07:21:54.361933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374694+0000","last_change":"2024-03-06T07:21:54.369742+0000","last_active":"2024-03-06T07:21:54.374694+0000","last_peered":"2024-03-06T07:21:54.374694+0000","last_clean":"2024-03-06T07:21:54.374694+0000","last_became_active":"2024-03-06T07:21:54.368872+0000","last_became_peered":"2024-03-06T07:21:54.368872+0000","last_unstale":"2024-03-06T07:21:54.374694+0000","last_undegraded":"2024-03-06T07:21:54.374694+0000","last_fullsized":"2024-03-06T07:21:54.374694+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373011+0000","last_change":"2024-03-06T07:21:43.316009+0000","last_active":"2024-03-06T07:21:54.373011+0000","last_peered":"2024-03-06T07:21:54.373011+0000","last_clean":"2024-03-06T07:21:54.373011+0000","last_became_active":"2024-03-06T07:21:43.315699+0000","last_became_peered":"2024-03-06T07:21:43.315699+0000","last_unstale":"2024-03-06T07:21:54.373011+0000","last_undegraded":"2024-03-06T07:21:54.373011+0000","last_fullsized":"2024-03-06T07:21:54.373011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:52.769462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365150+0000","last_change":"2024-03-06T07:21:42.311690+0000","last_active":"2024-03-06T07:21:54.365150+0000","last_peered":"2024-03-06T07:21:54.365150+0000","last_clean":"2024-03-06T07:21:54.365150+0000","last_became_active":"2024-03-06T07:21:42.311417+0000","last_became_peered":"2024-03-06T07:21:42.311417+0000","last_unstale":"2024-03-06T07:21:54.365150+0000","last_undegraded":"2024-03-06T07:21:54.365150+0000","last_fullsized":"2024-03-06T07:21:54.365150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.094878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362069+0000","last_change":"2024-03-06T07:21:54.362069+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362069+0000","last_undegraded":"2024-03-06T07:21:54.362069+0000","last_fullsized":"2024-03-06T07:21:54.362069+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378181+0000","last_change":"2024-03-06T07:21:54.377550+0000","last_active":"2024-03-06T07:21:54.378181+0000","last_peered":"2024-03-06T07:21:54.378181+0000","last_clean":"2024-03-06T07:21:54.378181+0000","last_became_active":"2024-03-06T07:21:54.377271+0000","last_became_peered":"2024-03-06T07:21:54.377271+0000","last_unstale":"2024-03-06T07:21:54.378181+0000","last_undegraded":"2024-03-06T07:21:54.378181+0000","last_fullsized":"2024-03-06T07:21:54.378181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360360+0000","last_change":"2024-03-06T07:21:43.322229+0000","last_active":"2024-03-06T07:21:54.360360+0000","last_peered":"2024-03-06T07:21:54.360360+0000","last_clean":"2024-03-06T07:21:54.360360+0000","last_became_active":"2024-03-06T07:21:43.321934+0000","last_became_peered":"2024-03-06T07:21:43.321934+0000","last_unstale":"2024-03-06T07:21:54.360360+0000","last_undegraded":"2024-03-06T07:21:54.360360+0000","last_fullsized":"2024-03-06T07:21:54.360360+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365379+0000","last_change":"2024-03-06T07:21:42.321714+0000","last_active":"2024-03-06T07:21:54.365379+0000","last_peered":"2024-03-06T07:21:54.365379+0000","last_clean":"2024-03-06T07:21:54.365379+0000","last_became_active":"2024-03-06T07:21:42.321444+0000","last_became_peered":"2024-03-06T07:21:42.321444+0000","last_unstale":"2024-03-06T07:21:54.365379+0000","last_undegraded":"2024-03-06T07:21:54.365379+0000","last_fullsized":"2024-03-06T07:21:54.365379+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361491+0000","last_change":"2024-03-06T07:21:54.361491+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361491+0000","last_undegraded":"2024-03-06T07:21:54.361491+0000","last_fullsized":"2024-03-06T07:21:54.361491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373855+0000","last_change":"2024-03-06T07:21:54.371666+0000","last_active":"2024-03-06T07:21:54.373855+0000","last_peered":"2024-03-06T07:21:54.373855+0000","last_clean":"2024-03-06T07:21:54.373855+0000","last_became_active":"2024-03-06T07:21:54.371394+0000","last_became_peered":"2024-03-06T07:21:54.371394+0000","last_unstale":"2024-03-06T07:21:54.373855+0000","last_undegraded":"2024-03-06T07:21:54.373855+0000","last_fullsized":"2024-03-06T07:21:54.373855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:15.173188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368609+0000","last_change":"2024-03-06T07:21:43.316403+0000","last_active":"2024-03-06T07:21:54.368609+0000","last_peered":"2024-03-06T07:21:54.368609+0000","last_clean":"2024-03-06T07:21:54.368609+0000","last_became_active":"2024-03-06T07:21:43.315416+0000","last_became_peered":"2024-03-06T07:21:43.315416+0000","last_unstale":"2024-03-06T07:21:54.368609+0000","last_undegraded":"2024-03-06T07:21:54.368609+0000","last_fullsized":"2024-03-06T07:21:54.368609+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:32.706894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:42.311790+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:42.311500+0000","last_became_peered":"2024-03-06T07:21:42.311500+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358738+0000","last_change":"2024-03-06T07:21:54.358738+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358738+0000","last_undegraded":"2024-03-06T07:21:54.358738+0000","last_fullsized":"2024-03-06T07:21:54.358738+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374655+0000","last_change":"2024-03-06T07:21:54.369593+0000","last_active":"2024-03-06T07:21:54.374655+0000","last_peered":"2024-03-06T07:21:54.374655+0000","last_clean":"2024-03-06T07:21:54.374655+0000","last_became_active":"2024-03-06T07:21:54.369336+0000","last_became_peered":"2024-03-06T07:21:54.369336+0000","last_unstale":"2024-03-06T07:21:54.374655+0000","last_undegraded":"2024-03-06T07:21:54.374655+0000","last_fullsized":"2024-03-06T07:21:54.374655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359566+0000","last_change":"2024-03-06T07:21:43.319244+0000","last_active":"2024-03-06T07:21:54.359566+0000","last_peered":"2024-03-06T07:21:54.359566+0000","last_clean":"2024-03-06T07:21:54.359566+0000","last_became_active":"2024-03-06T07:21:43.318994+0000","last_became_peered":"2024-03-06T07:21:43.318994+0000","last_unstale":"2024-03-06T07:21:54.359566+0000","last_undegraded":"2024-03-06T07:21:54.359566+0000","last_fullsized":"2024-03-06T07:21:54.359566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369178+0000","last_change":"2024-03-06T07:21:42.315718+0000","last_active":"2024-03-06T07:21:54.369178+0000","last_peered":"2024-03-06T07:21:54.369178+0000","last_clean":"2024-03-06T07:21:54.369178+0000","last_became_active":"2024-03-06T07:21:42.315402+0000","last_became_peered":"2024-03-06T07:21:42.315402+0000","last_unstale":"2024-03-06T07:21:54.369178+0000","last_undegraded":"2024-03-06T07:21:54.369178+0000","last_fullsized":"2024-03-06T07:21:54.369178+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365483+0000","last_change":"2024-03-06T07:21:54.365483+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365483+0000","last_undegraded":"2024-03-06T07:21:54.365483+0000","last_fullsized":"2024-03-06T07:21:54.365483+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379803+0000","last_change":"2024-03-06T07:21:54.375429+0000","last_active":"2024-03-06T07:21:54.379803+0000","last_peered":"2024-03-06T07:21:54.379803+0000","last_clean":"2024-03-06T07:21:54.379803+0000","last_became_active":"2024-03-06T07:21:54.375117+0000","last_became_peered":"2024-03-06T07:21:54.375117+0000","last_unstale":"2024-03-06T07:21:54.379803+0000","last_undegraded":"2024-03-06T07:21:54.379803+0000","last_fullsized":"2024-03-06T07:21:54.379803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369090+0000","last_change":"2024-03-06T07:21:43.323135+0000","last_active":"2024-03-06T07:21:54.369090+0000","last_peered":"2024-03-06T07:21:54.369090+0000","last_clean":"2024-03-06T07:21:54.369090+0000","last_became_active":"2024-03-06T07:21:43.322814+0000","last_became_peered":"2024-03-06T07:21:43.322814+0000","last_unstale":"2024-03-06T07:21:54.369090+0000","last_undegraded":"2024-03-06T07:21:54.369090+0000","last_fullsized":"2024-03-06T07:21:54.369090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:24.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359662+0000","last_change":"2024-03-06T07:21:42.308451+0000","last_active":"2024-03-06T07:21:54.359662+0000","last_peered":"2024-03-06T07:21:54.359662+0000","last_clean":"2024-03-06T07:21:54.359662+0000","last_became_active":"2024-03-06T07:21:42.308167+0000","last_became_peered":"2024-03-06T07:21:42.308167+0000","last_unstale":"2024-03-06T07:21:54.359662+0000","last_undegraded":"2024-03-06T07:21:54.359662+0000","last_fullsized":"2024-03-06T07:21:54.359662+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:31.703072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362102+0000","last_change":"2024-03-06T07:21:54.362102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362102+0000","last_undegraded":"2024-03-06T07:21:54.362102+0000","last_fullsized":"2024-03-06T07:21:54.362102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376688+0000","last_change":"2024-03-06T07:21:54.375869+0000","last_active":"2024-03-06T07:21:54.376688+0000","last_peered":"2024-03-06T07:21:54.376688+0000","last_clean":"2024-03-06T07:21:54.376688+0000","last_became_active":"2024-03-06T07:21:54.375586+0000","last_became_peered":"2024-03-06T07:21:54.375586+0000","last_unstale":"2024-03-06T07:21:54.376688+0000","last_undegraded":"2024-03-06T07:21:54.376688+0000","last_fullsized":"2024-03-06T07:21:54.376688+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:02.230230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356824+0000","last_change":"2024-03-06T07:21:43.324622+0000","last_active":"2024-03-06T07:21:54.356824+0000","last_peered":"2024-03-06T07:21:54.356824+0000","last_clean":"2024-03-06T07:21:54.356824+0000","last_became_active":"2024-03-06T07:21:43.324350+0000","last_became_peered":"2024-03-06T07:21:43.324350+0000","last_unstale":"2024-03-06T07:21:54.356824+0000","last_undegraded":"2024-03-06T07:21:54.356824+0000","last_fullsized":"2024-03-06T07:21:54.356824+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359015+0000","last_change":"2024-03-06T07:21:42.309246+0000","last_active":"2024-03-06T07:21:54.359015+0000","last_peered":"2024-03-06T07:21:54.359015+0000","last_clean":"2024-03-06T07:21:54.359015+0000","last_became_active":"2024-03-06T07:21:42.308990+0000","last_became_peered":"2024-03-06T07:21:42.308990+0000","last_unstale":"2024-03-06T07:21:54.359015+0000","last_undegraded":"2024-03-06T07:21:54.359015+0000","last_fullsized":"2024-03-06T07:21:54.359015+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:51.765639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358506+0000","last_change":"2024-03-06T07:21:54.358506+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358506+0000","last_undegraded":"2024-03-06T07:21:54.358506+0000","last_fullsized":"2024-03-06T07:21:54.358506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377281+0000","last_change":"2024-03-06T07:21:54.371835+0000","last_active":"2024-03-06T07:21:54.377281+0000","last_peered":"2024-03-06T07:21:54.377281+0000","last_clean":"2024-03-06T07:21:54.377281+0000","last_became_active":"2024-03-06T07:21:54.371518+0000","last_became_peered":"2024-03-06T07:21:54.371518+0000","last_unstale":"2024-03-06T07:21:54.377281+0000","last_undegraded":"2024-03-06T07:21:54.377281+0000","last_fullsized":"2024-03-06T07:21:54.377281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369611+0000","last_change":"2024-03-06T07:21:43.329441+0000","last_active":"2024-03-06T07:21:54.369611+0000","last_peered":"2024-03-06T07:21:54.369611+0000","last_clean":"2024-03-06T07:21:54.369611+0000","last_became_active":"2024-03-06T07:21:43.327694+0000","last_became_peered":"2024-03-06T07:21:43.327694+0000","last_unstale":"2024-03-06T07:21:54.369611+0000","last_undegraded":"2024-03-06T07:21:54.369611+0000","last_fullsized":"2024-03-06T07:21:54.369611+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369282+0000","last_change":"2024-03-06T07:21:42.309614+0000","last_active":"2024-03-06T07:21:54.369282+0000","last_peered":"2024-03-06T07:21:54.369282+0000","last_clean":"2024-03-06T07:21:54.369282+0000","last_became_active":"2024-03-06T07:21:42.309319+0000","last_became_peered":"2024-03-06T07:21:42.309319+0000","last_unstale":"2024-03-06T07:21:54.369282+0000","last_undegraded":"2024-03-06T07:21:54.369282+0000","last_fullsized":"2024-03-06T07:21:54.369282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:28.385954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360823+0000","last_change":"2024-03-06T07:21:54.360823+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360823+0000","last_undegraded":"2024-03-06T07:21:54.360823+0000","last_fullsized":"2024-03-06T07:21:54.360823+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379806+0000","last_change":"2024-03-06T07:21:54.375555+0000","last_active":"2024-03-06T07:21:54.379806+0000","last_peered":"2024-03-06T07:21:54.379806+0000","last_clean":"2024-03-06T07:21:54.379806+0000","last_became_active":"2024-03-06T07:21:54.375251+0000","last_became_peered":"2024-03-06T07:21:54.375251+0000","last_unstale":"2024-03-06T07:21:54.379806+0000","last_undegraded":"2024-03-06T07:21:54.379806+0000","last_fullsized":"2024-03-06T07:21:54.379806+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:01.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359852+0000","last_change":"2024-03-06T07:21:43.321711+0000","last_active":"2024-03-06T07:21:54.359852+0000","last_peered":"2024-03-06T07:21:54.359852+0000","last_clean":"2024-03-06T07:21:54.359852+0000","last_became_active":"2024-03-06T07:21:43.321434+0000","last_became_peered":"2024-03-06T07:21:43.321434+0000","last_unstale":"2024-03-06T07:21:54.359852+0000","last_undegraded":"2024-03-06T07:21:54.359852+0000","last_fullsized":"2024-03-06T07:21:54.359852+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:06.150276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360450+0000","last_change":"2024-03-06T07:21:42.310672+0000","last_active":"2024-03-06T07:21:54.360450+0000","last_peered":"2024-03-06T07:21:54.360450+0000","last_clean":"2024-03-06T07:21:54.360450+0000","last_became_active":"2024-03-06T07:21:42.310366+0000","last_became_peered":"2024-03-06T07:21:42.310366+0000","last_unstale":"2024-03-06T07:21:54.360450+0000","last_undegraded":"2024-03-06T07:21:54.360450+0000","last_fullsized":"2024-03-06T07:21:54.360450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:24.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361405+0000","last_change":"2024-03-06T07:21:54.361405+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361405+0000","last_undegraded":"2024-03-06T07:21:54.361405+0000","last_fullsized":"2024-03-06T07:21:54.361405+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367559+0000","last_change":"2024-03-06T07:21:54.366502+0000","last_active":"2024-03-06T07:21:54.367559+0000","last_peered":"2024-03-06T07:21:54.367559+0000","last_clean":"2024-03-06T07:21:54.367559+0000","last_became_active":"2024-03-06T07:21:54.366066+0000","last_became_peered":"2024-03-06T07:21:54.366066+0000","last_unstale":"2024-03-06T07:21:54.367559+0000","last_undegraded":"2024-03-06T07:21:54.367559+0000","last_fullsized":"2024-03-06T07:21:54.367559+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364470+0000","last_change":"2024-03-06T07:21:43.319233+0000","last_active":"2024-03-06T07:21:54.364470+0000","last_peered":"2024-03-06T07:21:54.364470+0000","last_clean":"2024-03-06T07:21:54.364470+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.364470+0000","last_undegraded":"2024-03-06T07:21:54.364470+0000","last_fullsized":"2024-03-06T07:21:54.364470+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369843+0000","last_change":"2024-03-06T07:21:42.322036+0000","last_active":"2024-03-06T07:21:54.369843+0000","last_peered":"2024-03-06T07:21:54.369843+0000","last_clean":"2024-03-06T07:21:54.369843+0000","last_became_active":"2024-03-06T07:21:42.321385+0000","last_became_peered":"2024-03-06T07:21:42.321385+0000","last_unstale":"2024-03-06T07:21:54.369843+0000","last_undegraded":"2024-03-06T07:21:54.369843+0000","last_fullsized":"2024-03-06T07:21:54.369843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356636+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:54.356636+0000","last_peered":"2024-03-06T07:21:54.356636+0000","last_clean":"2024-03-06T07:21:54.356636+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:54.356636+0000","last_undegraded":"2024-03-06T07:21:54.356636+0000","last_fullsized":"2024-03-06T07:21:54.356636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":881653,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":651264,"data_stored":291813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":291161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":284171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":292795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":643072,"data_stored":285201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":291321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":874013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:59.771 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.903+0000 7f6c03455640 1 Processor -- start 2024-03-06T07:21:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.903+0000 7f6c03455640 1 -- start start 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c03455640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6bfc1527b0 0x7f6bfc159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c03455640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c7180 0x7f6bfc0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c03455640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc155b20 0x7f6bfc0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c03455640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc159640 con 0x7f6bfc1527b0 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c03455640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c8df0 con 0x7f6bfc0c7180 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c03455640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c7c60 con 0x7f6bfc155b20 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c011f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6bfc1527b0 0x7f6bfc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c011f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6bfc1527b0 0x7f6bfc159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57886/0 (socket says 172.21.15.26:57886) 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c011f1640 1 -- 172.21.15.26:0/1346772330 learned_addr learned my addr 172.21.15.26:0/1346772330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c009f0640 1 --2- 172.21.15.26:0/1346772330 >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c7180 0x7f6bfc0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c019f2640 1 --2- 172.21.15.26:0/1346772330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc155b20 0x7f6bfc0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c011f1640 1 -- 172.21.15.26:0/1346772330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc155b20 msgr2=0x7f6bfc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.904+0000 7f6c011f1640 1 --2- 172.21.15.26:0/1346772330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc155b20 0x7f6bfc0c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.905+0000 7f6c011f1640 1 -- 172.21.15.26:0/1346772330 >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c7180 msgr2=0x7f6bfc0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.905+0000 7f6c011f1640 1 --2- 172.21.15.26:0/1346772330 >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c7180 0x7f6bfc0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.905+0000 7f6c011f1640 1 -- 172.21.15.26:0/1346772330 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0c8060 con 0x7f6bfc1527b0 2024-03-06T07:21:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.905+0000 7f6c011f1640 1 --2- 172.21.15.26:0/1346772330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc1527b0 0x7f6bfc159100 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc08cac0 tx=0x7f6bec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5319b7fe2e978542 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.905+0000 7f6bf3fff640 1 -- 172.21.15.26:0/1346772330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec02e070 con 0x7f6bfc1527b0 2024-03-06T07:21:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.905+0000 7f6bf3fff640 1 -- 172.21.15.26:0/1346772330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec037070 con 0x7f6bfc1527b0 2024-03-06T07:21:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.905+0000 7f6bf3fff640 1 -- 172.21.15.26:0/1346772330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec031a00 con 0x7f6bfc1527b0 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 -- 172.21.15.26:0/1346772330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc1527b0 msgr2=0x7f6bfc159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 --2- 172.21.15.26:0/1346772330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc1527b0 0x7f6bfc159100 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc08cac0 tx=0x7f6bec02b880 comp rx=0 tx=0).stop 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 -- 172.21.15.26:0/1346772330 shutdown_connections 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 --2- 172.21.15.26:0/1346772330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc155b20 0x7f6bfc0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 --2- 172.21.15.26:0/1346772330 >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c7180 0x7f6bfc0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 --2- 172.21.15.26:0/1346772330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc1527b0 0x7f6bfc159100 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 -- 172.21.15.26:0/1346772330 >> 172.21.15.26:0/1346772330 conn(0x7f6bfc0ac560 msgr2=0x7f6bfc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 -- 172.21.15.26:0/1346772330 shutdown_connections 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.906+0000 7f6c03455640 1 -- 172.21.15.26:0/1346772330 wait complete. 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.907+0000 7f6c03455640 1 Processor -- start 2024-03-06T07:21:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.907+0000 7f6c03455640 1 -- start start 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c03455640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0c7180 0x7f6bfc154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c03455640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc1527b0 0x7f6bfc157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c03455640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc155b20 0x7f6bfc0c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c03455640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc155320 con 0x7f6bfc0c7180 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c03455640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c7980 con 0x7f6bfc155b20 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c03455640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc155600 con 0x7f6bfc1527b0 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0c7180 0x7f6bfc154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0c7180 0x7f6bfc154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57890/0 (socket says 172.21.15.26:57890) 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 -- 172.21.15.26:0/3461133405 learned_addr learned my addr 172.21.15.26:0/3461133405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c009f0640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc1527b0 0x7f6bfc157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c019f2640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc155b20 0x7f6bfc0c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 -- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc1527b0 msgr2=0x7f6bfc157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc1527b0 0x7f6bfc157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 -- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc155b20 msgr2=0x7f6bfc0c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc155b20 0x7f6bfc0c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.908+0000 7f6c011f1640 1 -- 172.21.15.26:0/3461133405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc153b80 con 0x7f6bfc0c7180 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.909+0000 7f6c011f1640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0c7180 0x7f6bfc154080 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000c00 tx=0x7f6bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.909+0000 7f6bf1ffb640 1 -- 172.21.15.26:0/3461133405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec032670 con 0x7f6bfc0c7180 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.909+0000 7f6bf1ffb640 1 -- 172.21.15.26:0/3461133405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec032810 con 0x7f6bfc0c7180 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.909+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc154b40 con 0x7f6bfc0c7180 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.909+0000 7f6bf1ffb640 1 -- 172.21.15.26:0/3461133405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec032c00 con 0x7f6bfc0c7180 2024-03-06T07:21:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.909+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc157c60 con 0x7f6bfc0c7180 2024-03-06T07:21:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.910+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc1539b0 con 0x7f6bfc0c7180 2024-03-06T07:21:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.911+0000 7f6bf1ffb640 1 -- 172.21.15.26:0/3461133405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bec04c060 con 0x7f6bfc0c7180 2024-03-06T07:21:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.911+0000 7f6bf1ffb640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b4d0 0x7f6be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.911+0000 7f6bf1ffb640 1 -- 172.21.15.26:0/3461133405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f6bec0abdc0 con 0x7f6bfc0c7180 2024-03-06T07:21:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.911+0000 7f6c009f0640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b4d0 0x7f6be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.911+0000 7f6c009f0640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b4d0 0x7f6be006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0be470 tx=0x7f6bf8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:21:59.915+0000 7f6bf1ffb640 1 -- 172.21.15.26:0/3461133405 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6bec040070 con 0x7f6bfc0c7180 2024-03-06T07:22:00.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.113+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bfc155060 con 0x7f6be006b4d0 2024-03-06T07:22:00.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.124+0000 7f6bf1ffb640 1 -- 172.21.15.26:0/3461133405 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485262 (secure 0 0 0) 0x7f6bf8011a30 con 0x7f6be006b4d0 2024-03-06T07:22:00.126 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b4d0 msgr2=0x7f6be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b4d0 0x7f6be006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0be470 tx=0x7f6bf8009190 comp rx=0 tx=0).stop 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0c7180 msgr2=0x7f6bfc154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0c7180 0x7f6bfc154080 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000c00 tx=0x7f6bec0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 shutdown_connections 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b4d0 0x7f6be006d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc155b20 0x7f6bfc0c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bfc1527b0 0x7f6bfc157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 --2- 172.21.15.26:0/3461133405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0c7180 0x7f6bfc154080 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 >> 172.21.15.26:0/3461133405 conn(0x7f6bfc0ac560 msgr2=0x7f6bfc150a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 shutdown_connections 2024-03-06T07:22:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.128+0000 7f6c03455640 1 -- 172.21.15.26:0/3461133405 wait complete. 2024-03-06T07:22:00.141 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-06T07:21:59.053948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237996,"kb_used_data":5180,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736724,"statfs":{"total":773094113280,"available":772850405376,"internally_reserved":0,"allocated":5304320,"data_stored":3492128,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703896"},"pg_stats":[{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359676+0000","last_change":"2024-03-06T07:21:54.359676+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359676+0000","last_undegraded":"2024-03-06T07:21:54.359676+0000","last_fullsized":"2024-03-06T07:21:54.359676+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372914+0000","last_change":"2024-03-06T07:21:43.317811+0000","last_active":"2024-03-06T07:21:54.372914+0000","last_peered":"2024-03-06T07:21:54.372914+0000","last_clean":"2024-03-06T07:21:54.372914+0000","last_became_active":"2024-03-06T07:21:43.317509+0000","last_became_peered":"2024-03-06T07:21:43.317509+0000","last_unstale":"2024-03-06T07:21:54.372914+0000","last_undegraded":"2024-03-06T07:21:54.372914+0000","last_fullsized":"2024-03-06T07:21:54.372914+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364197+0000","last_change":"2024-03-06T07:21:54.364197+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364197+0000","last_undegraded":"2024-03-06T07:21:54.364197+0000","last_fullsized":"2024-03-06T07:21:54.364197+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357765+0000","last_change":"2024-03-06T07:21:43.317801+0000","last_active":"2024-03-06T07:21:54.357765+0000","last_peered":"2024-03-06T07:21:54.357765+0000","last_clean":"2024-03-06T07:21:54.357765+0000","last_became_active":"2024-03-06T07:21:43.317522+0000","last_became_peered":"2024-03-06T07:21:43.317522+0000","last_unstale":"2024-03-06T07:21:54.357765+0000","last_undegraded":"2024-03-06T07:21:54.357765+0000","last_fullsized":"2024-03-06T07:21:54.357765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.368156+0000","last_change":"2024-03-06T07:21:54.368156+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.368156+0000","last_undegraded":"2024-03-06T07:21:54.368156+0000","last_fullsized":"2024-03-06T07:21:54.368156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369212+0000","last_change":"2024-03-06T07:21:43.328071+0000","last_active":"2024-03-06T07:21:54.369212+0000","last_peered":"2024-03-06T07:21:54.369212+0000","last_clean":"2024-03-06T07:21:54.369212+0000","last_became_active":"2024-03-06T07:21:43.327781+0000","last_became_peered":"2024-03-06T07:21:43.327781+0000","last_unstale":"2024-03-06T07:21:54.369212+0000","last_undegraded":"2024-03-06T07:21:54.369212+0000","last_fullsized":"2024-03-06T07:21:54.369212+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365661+0000","last_change":"2024-03-06T07:21:54.365661+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365661+0000","last_undegraded":"2024-03-06T07:21:54.365661+0000","last_fullsized":"2024-03-06T07:21:54.365661+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357083+0000","last_change":"2024-03-06T07:21:43.326286+0000","last_active":"2024-03-06T07:21:54.357083+0000","last_peered":"2024-03-06T07:21:54.357083+0000","last_clean":"2024-03-06T07:21:54.357083+0000","last_became_active":"2024-03-06T07:21:43.326031+0000","last_became_peered":"2024-03-06T07:21:43.326031+0000","last_unstale":"2024-03-06T07:21:54.357083+0000","last_undegraded":"2024-03-06T07:21:54.357083+0000","last_fullsized":"2024-03-06T07:21:54.357083+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:21.645487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361397+0000","last_change":"2024-03-06T07:21:54.361397+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361397+0000","last_undegraded":"2024-03-06T07:21:54.361397+0000","last_fullsized":"2024-03-06T07:21:54.361397+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369745+0000","last_change":"2024-03-06T07:21:43.321810+0000","last_active":"2024-03-06T07:21:54.369745+0000","last_peered":"2024-03-06T07:21:54.369745+0000","last_clean":"2024-03-06T07:21:54.369745+0000","last_became_active":"2024-03-06T07:21:43.321538+0000","last_became_peered":"2024-03-06T07:21:43.321538+0000","last_unstale":"2024-03-06T07:21:54.369745+0000","last_undegraded":"2024-03-06T07:21:54.369745+0000","last_fullsized":"2024-03-06T07:21:54.369745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:10.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359185+0000","last_change":"2024-03-06T07:21:54.359185+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359185+0000","last_undegraded":"2024-03-06T07:21:54.359185+0000","last_fullsized":"2024-03-06T07:21:54.359185+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368491+0000","last_change":"2024-03-06T07:21:43.318713+0000","last_active":"2024-03-06T07:21:54.368491+0000","last_peered":"2024-03-06T07:21:54.368491+0000","last_clean":"2024-03-06T07:21:54.368491+0000","last_became_active":"2024-03-06T07:21:43.318385+0000","last_became_peered":"2024-03-06T07:21:43.318385+0000","last_unstale":"2024-03-06T07:21:54.368491+0000","last_undegraded":"2024-03-06T07:21:54.368491+0000","last_fullsized":"2024-03-06T07:21:54.368491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361827+0000","last_change":"2024-03-06T07:21:54.361827+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361827+0000","last_undegraded":"2024-03-06T07:21:54.361827+0000","last_fullsized":"2024-03-06T07:21:54.361827+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368595+0000","last_change":"2024-03-06T07:21:43.327099+0000","last_active":"2024-03-06T07:21:54.368595+0000","last_peered":"2024-03-06T07:21:54.368595+0000","last_clean":"2024-03-06T07:21:54.368595+0000","last_became_active":"2024-03-06T07:21:43.326851+0000","last_became_peered":"2024-03-06T07:21:43.326851+0000","last_unstale":"2024-03-06T07:21:54.368595+0000","last_undegraded":"2024-03-06T07:21:54.368595+0000","last_fullsized":"2024-03-06T07:21:54.368595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360705+0000","last_change":"2024-03-06T07:21:54.360705+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360705+0000","last_undegraded":"2024-03-06T07:21:54.360705+0000","last_fullsized":"2024-03-06T07:21:54.360705+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358329+0000","last_change":"2024-03-06T07:21:43.325897+0000","last_active":"2024-03-06T07:21:54.358329+0000","last_peered":"2024-03-06T07:21:54.358329+0000","last_clean":"2024-03-06T07:21:54.358329+0000","last_became_active":"2024-03-06T07:21:43.325647+0000","last_became_peered":"2024-03-06T07:21:43.325647+0000","last_unstale":"2024-03-06T07:21:54.358329+0000","last_undegraded":"2024-03-06T07:21:54.358329+0000","last_fullsized":"2024-03-06T07:21:54.358329+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365629+0000","last_change":"2024-03-06T07:21:54.365629+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365629+0000","last_undegraded":"2024-03-06T07:21:54.365629+0000","last_fullsized":"2024-03-06T07:21:54.365629+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359703+0000","last_change":"2024-03-06T07:21:43.330337+0000","last_active":"2024-03-06T07:21:54.359703+0000","last_peered":"2024-03-06T07:21:54.359703+0000","last_clean":"2024-03-06T07:21:54.359703+0000","last_became_active":"2024-03-06T07:21:43.330053+0000","last_became_peered":"2024-03-06T07:21:43.330053+0000","last_unstale":"2024-03-06T07:21:54.359703+0000","last_undegraded":"2024-03-06T07:21:54.359703+0000","last_fullsized":"2024-03-06T07:21:54.359703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358522+0000","last_change":"2024-03-06T07:21:54.358522+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358522+0000","last_undegraded":"2024-03-06T07:21:54.358522+0000","last_fullsized":"2024-03-06T07:21:54.358522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370017+0000","last_change":"2024-03-06T07:21:43.325100+0000","last_active":"2024-03-06T07:21:54.370017+0000","last_peered":"2024-03-06T07:21:54.370017+0000","last_clean":"2024-03-06T07:21:54.370017+0000","last_became_active":"2024-03-06T07:21:43.324812+0000","last_became_peered":"2024-03-06T07:21:43.324812+0000","last_unstale":"2024-03-06T07:21:54.370017+0000","last_undegraded":"2024-03-06T07:21:54.370017+0000","last_fullsized":"2024-03-06T07:21:54.370017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:11.442368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359265+0000","last_change":"2024-03-06T07:21:54.359265+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359265+0000","last_undegraded":"2024-03-06T07:21:54.359265+0000","last_fullsized":"2024-03-06T07:21:54.359265+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366884+0000","last_change":"2024-03-06T07:21:43.322188+0000","last_active":"2024-03-06T07:21:54.366884+0000","last_peered":"2024-03-06T07:21:54.366884+0000","last_clean":"2024-03-06T07:21:54.366884+0000","last_became_active":"2024-03-06T07:21:43.321685+0000","last_became_peered":"2024-03-06T07:21:43.321685+0000","last_unstale":"2024-03-06T07:21:54.366884+0000","last_undegraded":"2024-03-06T07:21:54.366884+0000","last_fullsized":"2024-03-06T07:21:54.366884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:54.799043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359570+0000","last_change":"2024-03-06T07:21:54.359570+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359570+0000","last_undegraded":"2024-03-06T07:21:54.359570+0000","last_fullsized":"2024-03-06T07:21:54.359570+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359503+0000","last_change":"2024-03-06T07:21:43.321952+0000","last_active":"2024-03-06T07:21:54.359503+0000","last_peered":"2024-03-06T07:21:54.359503+0000","last_clean":"2024-03-06T07:21:54.359503+0000","last_became_active":"2024-03-06T07:21:43.321579+0000","last_became_peered":"2024-03-06T07:21:43.321579+0000","last_unstale":"2024-03-06T07:21:54.359503+0000","last_undegraded":"2024-03-06T07:21:54.359503+0000","last_fullsized":"2024-03-06T07:21:54.359503+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:04.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359648+0000","last_change":"2024-03-06T07:21:54.359648+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359648+0000","last_undegraded":"2024-03-06T07:21:54.359648+0000","last_fullsized":"2024-03-06T07:21:54.359648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368337+0000","last_change":"2024-03-06T07:21:43.321277+0000","last_active":"2024-03-06T07:21:54.368337+0000","last_peered":"2024-03-06T07:21:54.368337+0000","last_clean":"2024-03-06T07:21:54.368337+0000","last_became_active":"2024-03-06T07:21:43.320973+0000","last_became_peered":"2024-03-06T07:21:43.320973+0000","last_unstale":"2024-03-06T07:21:54.368337+0000","last_undegraded":"2024-03-06T07:21:54.368337+0000","last_fullsized":"2024-03-06T07:21:54.368337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:15.957299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362294+0000","last_change":"2024-03-06T07:21:54.362294+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362294+0000","last_undegraded":"2024-03-06T07:21:54.362294+0000","last_fullsized":"2024-03-06T07:21:54.362294+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364199+0000","last_change":"2024-03-06T07:21:50.373762+0000","last_active":"2024-03-06T07:21:54.364199+0000","last_peered":"2024-03-06T07:21:54.364199+0000","last_clean":"2024-03-06T07:21:54.364199+0000","last_became_active":"2024-03-06T07:21:50.373521+0000","last_became_peered":"2024-03-06T07:21:50.373521+0000","last_unstale":"2024-03-06T07:21:54.364199+0000","last_undegraded":"2024-03-06T07:21:54.364199+0000","last_fullsized":"2024-03-06T07:21:54.364199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359359+0000","last_change":"2024-03-06T07:21:54.359359+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359359+0000","last_undegraded":"2024-03-06T07:21:54.359359+0000","last_fullsized":"2024-03-06T07:21:54.359359+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357693+0000","last_change":"2024-03-06T07:21:43.321137+0000","last_active":"2024-03-06T07:21:54.357693+0000","last_peered":"2024-03-06T07:21:54.357693+0000","last_clean":"2024-03-06T07:21:54.357693+0000","last_became_active":"2024-03-06T07:21:43.320820+0000","last_became_peered":"2024-03-06T07:21:43.320820+0000","last_unstale":"2024-03-06T07:21:54.357693+0000","last_undegraded":"2024-03-06T07:21:54.357693+0000","last_fullsized":"2024-03-06T07:21:54.357693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:52.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359065+0000","last_change":"2024-03-06T07:21:54.359065+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359065+0000","last_undegraded":"2024-03-06T07:21:54.359065+0000","last_fullsized":"2024-03-06T07:21:54.359065+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366975+0000","last_change":"2024-03-06T07:21:43.320518+0000","last_active":"2024-03-06T07:21:54.366975+0000","last_peered":"2024-03-06T07:21:54.366975+0000","last_clean":"2024-03-06T07:21:54.366975+0000","last_became_active":"2024-03-06T07:21:43.320178+0000","last_became_peered":"2024-03-06T07:21:43.320178+0000","last_unstale":"2024-03-06T07:21:54.366975+0000","last_undegraded":"2024-03-06T07:21:54.366975+0000","last_fullsized":"2024-03-06T07:21:54.366975+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360235+0000","last_change":"2024-03-06T07:21:54.360235+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360235+0000","last_undegraded":"2024-03-06T07:21:54.360235+0000","last_fullsized":"2024-03-06T07:21:54.360235+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370182+0000","last_change":"2024-03-06T07:21:43.324279+0000","last_active":"2024-03-06T07:21:54.370182+0000","last_peered":"2024-03-06T07:21:54.370182+0000","last_clean":"2024-03-06T07:21:54.370182+0000","last_became_active":"2024-03-06T07:21:43.323499+0000","last_became_peered":"2024-03-06T07:21:43.323499+0000","last_unstale":"2024-03-06T07:21:54.370182+0000","last_undegraded":"2024-03-06T07:21:54.370182+0000","last_fullsized":"2024-03-06T07:21:54.370182+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359045+0000","last_change":"2024-03-06T07:21:54.359045+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359045+0000","last_undegraded":"2024-03-06T07:21:54.359045+0000","last_fullsized":"2024-03-06T07:21:54.359045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357861+0000","last_change":"2024-03-06T07:21:43.318202+0000","last_active":"2024-03-06T07:21:54.357861+0000","last_peered":"2024-03-06T07:21:54.357861+0000","last_clean":"2024-03-06T07:21:54.357861+0000","last_became_active":"2024-03-06T07:21:43.317929+0000","last_became_peered":"2024-03-06T07:21:43.317929+0000","last_unstale":"2024-03-06T07:21:54.357861+0000","last_undegraded":"2024-03-06T07:21:54.357861+0000","last_fullsized":"2024-03-06T07:21:54.357861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367783+0000","last_change":"2024-03-06T07:21:54.367783+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367783+0000","last_undegraded":"2024-03-06T07:21:54.367783+0000","last_fullsized":"2024-03-06T07:21:54.367783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369274+0000","last_change":"2024-03-06T07:21:43.324265+0000","last_active":"2024-03-06T07:21:54.369274+0000","last_peered":"2024-03-06T07:21:54.369274+0000","last_clean":"2024-03-06T07:21:54.369274+0000","last_became_active":"2024-03-06T07:21:43.323877+0000","last_became_peered":"2024-03-06T07:21:43.323877+0000","last_unstale":"2024-03-06T07:21:54.369274+0000","last_undegraded":"2024-03-06T07:21:54.369274+0000","last_fullsized":"2024-03-06T07:21:54.369274+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367102+0000","last_change":"2024-03-06T07:21:54.367102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367102+0000","last_undegraded":"2024-03-06T07:21:54.367102+0000","last_fullsized":"2024-03-06T07:21:54.367102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365314+0000","last_change":"2024-03-06T07:21:43.321580+0000","last_active":"2024-03-06T07:21:54.365314+0000","last_peered":"2024-03-06T07:21:54.365314+0000","last_clean":"2024-03-06T07:21:54.365314+0000","last_became_active":"2024-03-06T07:21:43.319775+0000","last_became_peered":"2024-03-06T07:21:43.319775+0000","last_unstale":"2024-03-06T07:21:54.365314+0000","last_undegraded":"2024-03-06T07:21:54.365314+0000","last_fullsized":"2024-03-06T07:21:54.365314+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:32.802384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358875+0000","last_change":"2024-03-06T07:21:54.358875+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358875+0000","last_undegraded":"2024-03-06T07:21:54.358875+0000","last_fullsized":"2024-03-06T07:21:54.358875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365089+0000","last_change":"2024-03-06T07:21:43.322797+0000","last_active":"2024-03-06T07:21:54.365089+0000","last_peered":"2024-03-06T07:21:54.365089+0000","last_clean":"2024-03-06T07:21:54.365089+0000","last_became_active":"2024-03-06T07:21:43.322505+0000","last_became_peered":"2024-03-06T07:21:43.322505+0000","last_unstale":"2024-03-06T07:21:54.365089+0000","last_undegraded":"2024-03-06T07:21:54.365089+0000","last_fullsized":"2024-03-06T07:21:54.365089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359127+0000","last_change":"2024-03-06T07:21:54.359127+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359127+0000","last_undegraded":"2024-03-06T07:21:54.359127+0000","last_fullsized":"2024-03-06T07:21:54.359127+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358833+0000","last_change":"2024-03-06T07:21:43.326058+0000","last_active":"2024-03-06T07:21:54.358833+0000","last_peered":"2024-03-06T07:21:54.358833+0000","last_clean":"2024-03-06T07:21:54.358833+0000","last_became_active":"2024-03-06T07:21:43.325799+0000","last_became_peered":"2024-03-06T07:21:43.325799+0000","last_unstale":"2024-03-06T07:21:54.358833+0000","last_undegraded":"2024-03-06T07:21:54.358833+0000","last_fullsized":"2024-03-06T07:21:54.358833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357760+0000","last_change":"2024-03-06T07:21:54.357760+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357760+0000","last_undegraded":"2024-03-06T07:21:54.357760+0000","last_fullsized":"2024-03-06T07:21:54.357760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365258+0000","last_change":"2024-03-06T07:21:43.322671+0000","last_active":"2024-03-06T07:21:54.365258+0000","last_peered":"2024-03-06T07:21:54.365258+0000","last_clean":"2024-03-06T07:21:54.365258+0000","last_became_active":"2024-03-06T07:21:43.322376+0000","last_became_peered":"2024-03-06T07:21:43.322376+0000","last_unstale":"2024-03-06T07:21:54.365258+0000","last_undegraded":"2024-03-06T07:21:54.365258+0000","last_fullsized":"2024-03-06T07:21:54.365258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:15.049663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358331+0000","last_change":"2024-03-06T07:21:54.358331+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358331+0000","last_undegraded":"2024-03-06T07:21:54.358331+0000","last_fullsized":"2024-03-06T07:21:54.358331+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364803+0000","last_change":"2024-03-06T07:21:43.327014+0000","last_active":"2024-03-06T07:21:54.364803+0000","last_peered":"2024-03-06T07:21:54.364803+0000","last_clean":"2024-03-06T07:21:54.364803+0000","last_became_active":"2024-03-06T07:21:43.326759+0000","last_became_peered":"2024-03-06T07:21:43.326759+0000","last_unstale":"2024-03-06T07:21:54.364803+0000","last_undegraded":"2024-03-06T07:21:54.364803+0000","last_fullsized":"2024-03-06T07:21:54.364803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366485+0000","last_change":"2024-03-06T07:21:54.366485+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366485+0000","last_undegraded":"2024-03-06T07:21:54.366485+0000","last_fullsized":"2024-03-06T07:21:54.366485+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370315+0000","last_change":"2024-03-06T07:21:43.323199+0000","last_active":"2024-03-06T07:21:54.370315+0000","last_peered":"2024-03-06T07:21:54.370315+0000","last_clean":"2024-03-06T07:21:54.370315+0000","last_became_active":"2024-03-06T07:21:43.322835+0000","last_became_peered":"2024-03-06T07:21:43.322835+0000","last_unstale":"2024-03-06T07:21:54.370315+0000","last_undegraded":"2024-03-06T07:21:54.370315+0000","last_fullsized":"2024-03-06T07:21:54.370315+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.369128+0000","last_change":"2024-03-06T07:21:54.369128+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.369128+0000","last_undegraded":"2024-03-06T07:21:54.369128+0000","last_fullsized":"2024-03-06T07:21:54.369128+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357960+0000","last_change":"2024-03-06T07:21:43.318509+0000","last_active":"2024-03-06T07:21:54.357960+0000","last_peered":"2024-03-06T07:21:54.357960+0000","last_clean":"2024-03-06T07:21:54.357960+0000","last_became_active":"2024-03-06T07:21:43.318177+0000","last_became_peered":"2024-03-06T07:21:43.318177+0000","last_unstale":"2024-03-06T07:21:54.357960+0000","last_undegraded":"2024-03-06T07:21:54.357960+0000","last_fullsized":"2024-03-06T07:21:54.357960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361035+0000","last_change":"2024-03-06T07:21:54.361035+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361035+0000","last_undegraded":"2024-03-06T07:21:54.361035+0000","last_fullsized":"2024-03-06T07:21:54.361035+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369672+0000","last_change":"2024-03-06T07:21:43.332733+0000","last_active":"2024-03-06T07:21:54.369672+0000","last_peered":"2024-03-06T07:21:54.369672+0000","last_clean":"2024-03-06T07:21:54.369672+0000","last_became_active":"2024-03-06T07:21:43.331512+0000","last_became_peered":"2024-03-06T07:21:43.331512+0000","last_unstale":"2024-03-06T07:21:54.369672+0000","last_undegraded":"2024-03-06T07:21:54.369672+0000","last_fullsized":"2024-03-06T07:21:54.369672+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363616+0000","last_change":"2024-03-06T07:21:54.363616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363616+0000","last_undegraded":"2024-03-06T07:21:54.363616+0000","last_fullsized":"2024-03-06T07:21:54.363616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365453+0000","last_change":"2024-03-06T07:21:43.321054+0000","last_active":"2024-03-06T07:21:54.365453+0000","last_peered":"2024-03-06T07:21:54.365453+0000","last_clean":"2024-03-06T07:21:54.365453+0000","last_became_active":"2024-03-06T07:21:43.319435+0000","last_became_peered":"2024-03-06T07:21:43.319435+0000","last_unstale":"2024-03-06T07:21:54.365453+0000","last_undegraded":"2024-03-06T07:21:54.365453+0000","last_fullsized":"2024-03-06T07:21:54.365453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362601+0000","last_change":"2024-03-06T07:21:54.362601+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362601+0000","last_undegraded":"2024-03-06T07:21:54.362601+0000","last_fullsized":"2024-03-06T07:21:54.362601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364825+0000","last_change":"2024-03-06T07:21:43.328812+0000","last_active":"2024-03-06T07:21:54.364825+0000","last_peered":"2024-03-06T07:21:54.364825+0000","last_clean":"2024-03-06T07:21:54.364825+0000","last_became_active":"2024-03-06T07:21:43.328366+0000","last_became_peered":"2024-03-06T07:21:43.328366+0000","last_unstale":"2024-03-06T07:21:54.364825+0000","last_undegraded":"2024-03-06T07:21:54.364825+0000","last_fullsized":"2024-03-06T07:21:54.364825+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:48.381105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362773+0000","last_change":"2024-03-06T07:21:54.362773+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362773+0000","last_undegraded":"2024-03-06T07:21:54.362773+0000","last_fullsized":"2024-03-06T07:21:54.362773+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356882+0000","last_change":"2024-03-06T07:21:43.321155+0000","last_active":"2024-03-06T07:21:54.356882+0000","last_peered":"2024-03-06T07:21:54.356882+0000","last_clean":"2024-03-06T07:21:54.356882+0000","last_became_active":"2024-03-06T07:21:43.320747+0000","last_became_peered":"2024-03-06T07:21:43.320747+0000","last_unstale":"2024-03-06T07:21:54.356882+0000","last_undegraded":"2024-03-06T07:21:54.356882+0000","last_fullsized":"2024-03-06T07:21:54.356882+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:42.424597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358521+0000","last_change":"2024-03-06T07:21:54.358521+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358521+0000","last_undegraded":"2024-03-06T07:21:54.358521+0000","last_fullsized":"2024-03-06T07:21:54.358521+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368700+0000","last_change":"2024-03-06T07:21:43.326550+0000","last_active":"2024-03-06T07:21:54.368700+0000","last_peered":"2024-03-06T07:21:54.368700+0000","last_clean":"2024-03-06T07:21:54.368700+0000","last_became_active":"2024-03-06T07:21:43.326254+0000","last_became_peered":"2024-03-06T07:21:43.326254+0000","last_unstale":"2024-03-06T07:21:54.368700+0000","last_undegraded":"2024-03-06T07:21:54.368700+0000","last_fullsized":"2024-03-06T07:21:54.368700+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357560+0000","last_change":"2024-03-06T07:21:54.357560+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357560+0000","last_undegraded":"2024-03-06T07:21:54.357560+0000","last_fullsized":"2024-03-06T07:21:54.357560+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360266+0000","last_change":"2024-03-06T07:21:43.322773+0000","last_active":"2024-03-06T07:21:54.360266+0000","last_peered":"2024-03-06T07:21:54.360266+0000","last_clean":"2024-03-06T07:21:54.360266+0000","last_became_active":"2024-03-06T07:21:43.322525+0000","last_became_peered":"2024-03-06T07:21:43.322525+0000","last_unstale":"2024-03-06T07:21:54.360266+0000","last_undegraded":"2024-03-06T07:21:54.360266+0000","last_fullsized":"2024-03-06T07:21:54.360266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362616+0000","last_change":"2024-03-06T07:21:54.362616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362616+0000","last_undegraded":"2024-03-06T07:21:54.362616+0000","last_fullsized":"2024-03-06T07:21:54.362616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376862+0000","last_change":"2024-03-06T07:21:54.366905+0000","last_active":"2024-03-06T07:21:54.376862+0000","last_peered":"2024-03-06T07:21:54.376862+0000","last_clean":"2024-03-06T07:21:54.376862+0000","last_became_active":"2024-03-06T07:21:54.366627+0000","last_became_peered":"2024-03-06T07:21:54.366627+0000","last_unstale":"2024-03-06T07:21:54.376862+0000","last_undegraded":"2024-03-06T07:21:54.376862+0000","last_fullsized":"2024-03-06T07:21:54.376862+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356615+0000","last_change":"2024-03-06T07:21:43.319837+0000","last_active":"2024-03-06T07:21:54.356615+0000","last_peered":"2024-03-06T07:21:54.356615+0000","last_clean":"2024-03-06T07:21:54.356615+0000","last_became_active":"2024-03-06T07:21:43.319564+0000","last_became_peered":"2024-03-06T07:21:43.319564+0000","last_unstale":"2024-03-06T07:21:54.356615+0000","last_undegraded":"2024-03-06T07:21:54.356615+0000","last_fullsized":"2024-03-06T07:21:54.356615+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365243+0000","last_change":"2024-03-06T07:21:42.310584+0000","last_active":"2024-03-06T07:21:54.365243+0000","last_peered":"2024-03-06T07:21:54.365243+0000","last_clean":"2024-03-06T07:21:54.365243+0000","last_became_active":"2024-03-06T07:21:42.310317+0000","last_became_peered":"2024-03-06T07:21:42.310317+0000","last_unstale":"2024-03-06T07:21:54.365243+0000","last_undegraded":"2024-03-06T07:21:54.365243+0000","last_fullsized":"2024-03-06T07:21:54.365243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362416+0000","last_change":"2024-03-06T07:21:54.362416+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362416+0000","last_undegraded":"2024-03-06T07:21:54.362416+0000","last_fullsized":"2024-03-06T07:21:54.362416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380180+0000","last_change":"2024-03-06T07:21:54.377542+0000","last_active":"2024-03-06T07:21:54.380180+0000","last_peered":"2024-03-06T07:21:54.380180+0000","last_clean":"2024-03-06T07:21:54.380180+0000","last_became_active":"2024-03-06T07:21:54.377250+0000","last_became_peered":"2024-03-06T07:21:54.377250+0000","last_unstale":"2024-03-06T07:21:54.380180+0000","last_undegraded":"2024-03-06T07:21:54.380180+0000","last_fullsized":"2024-03-06T07:21:54.380180+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:36.750007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359472+0000","last_change":"2024-03-06T07:21:43.323883+0000","last_active":"2024-03-06T07:21:54.359472+0000","last_peered":"2024-03-06T07:21:54.359472+0000","last_clean":"2024-03-06T07:21:54.359472+0000","last_became_active":"2024-03-06T07:21:43.323613+0000","last_became_peered":"2024-03-06T07:21:43.323613+0000","last_unstale":"2024-03-06T07:21:54.359472+0000","last_undegraded":"2024-03-06T07:21:54.359472+0000","last_fullsized":"2024-03-06T07:21:54.359472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370119+0000","last_change":"2024-03-06T07:21:42.304199+0000","last_active":"2024-03-06T07:21:54.370119+0000","last_peered":"2024-03-06T07:21:54.370119+0000","last_clean":"2024-03-06T07:21:54.370119+0000","last_became_active":"2024-03-06T07:21:42.303802+0000","last_became_peered":"2024-03-06T07:21:42.303802+0000","last_unstale":"2024-03-06T07:21:54.370119+0000","last_undegraded":"2024-03-06T07:21:54.370119+0000","last_fullsized":"2024-03-06T07:21:54.370119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362959+0000","last_change":"2024-03-06T07:21:54.362959+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362959+0000","last_undegraded":"2024-03-06T07:21:54.362959+0000","last_fullsized":"2024-03-06T07:21:54.362959+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376955+0000","last_change":"2024-03-06T07:21:54.368687+0000","last_active":"2024-03-06T07:21:54.376955+0000","last_peered":"2024-03-06T07:21:54.376955+0000","last_clean":"2024-03-06T07:21:54.376955+0000","last_became_active":"2024-03-06T07:21:54.368106+0000","last_became_peered":"2024-03-06T07:21:54.368106+0000","last_unstale":"2024-03-06T07:21:54.376955+0000","last_undegraded":"2024-03-06T07:21:54.376955+0000","last_fullsized":"2024-03-06T07:21:54.376955+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364372+0000","last_change":"2024-03-06T07:21:43.323512+0000","last_active":"2024-03-06T07:21:54.364372+0000","last_peered":"2024-03-06T07:21:54.364372+0000","last_clean":"2024-03-06T07:21:54.364372+0000","last_became_active":"2024-03-06T07:21:43.323216+0000","last_became_peered":"2024-03-06T07:21:43.323216+0000","last_unstale":"2024-03-06T07:21:54.364372+0000","last_undegraded":"2024-03-06T07:21:54.364372+0000","last_fullsized":"2024-03-06T07:21:54.364372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:31.821522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369735+0000","last_change":"2024-03-06T07:21:42.305711+0000","last_active":"2024-03-06T07:21:54.369735+0000","last_peered":"2024-03-06T07:21:54.369735+0000","last_clean":"2024-03-06T07:21:54.369735+0000","last_became_active":"2024-03-06T07:21:42.305431+0000","last_became_peered":"2024-03-06T07:21:42.305431+0000","last_unstale":"2024-03-06T07:21:54.369735+0000","last_undegraded":"2024-03-06T07:21:54.369735+0000","last_fullsized":"2024-03-06T07:21:54.369735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357254+0000","last_change":"2024-03-06T07:21:54.357254+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357254+0000","last_undegraded":"2024-03-06T07:21:54.357254+0000","last_fullsized":"2024-03-06T07:21:54.357254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374713+0000","last_change":"2024-03-06T07:21:54.370174+0000","last_active":"2024-03-06T07:21:54.374713+0000","last_peered":"2024-03-06T07:21:54.374713+0000","last_clean":"2024-03-06T07:21:54.374713+0000","last_became_active":"2024-03-06T07:21:54.369897+0000","last_became_peered":"2024-03-06T07:21:54.369897+0000","last_unstale":"2024-03-06T07:21:54.374713+0000","last_undegraded":"2024-03-06T07:21:54.374713+0000","last_fullsized":"2024-03-06T07:21:54.374713+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359478+0000","last_change":"2024-03-06T07:21:43.317966+0000","last_active":"2024-03-06T07:21:54.359478+0000","last_peered":"2024-03-06T07:21:54.359478+0000","last_clean":"2024-03-06T07:21:54.359478+0000","last_became_active":"2024-03-06T07:21:43.317700+0000","last_became_peered":"2024-03-06T07:21:43.317700+0000","last_unstale":"2024-03-06T07:21:54.359478+0000","last_undegraded":"2024-03-06T07:21:54.359478+0000","last_fullsized":"2024-03-06T07:21:54.359478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359768+0000","last_change":"2024-03-06T07:21:42.315045+0000","last_active":"2024-03-06T07:21:54.359768+0000","last_peered":"2024-03-06T07:21:54.359768+0000","last_clean":"2024-03-06T07:21:54.359768+0000","last_became_active":"2024-03-06T07:21:42.314768+0000","last_became_peered":"2024-03-06T07:21:42.314768+0000","last_unstale":"2024-03-06T07:21:54.359768+0000","last_undegraded":"2024-03-06T07:21:54.359768+0000","last_fullsized":"2024-03-06T07:21:54.359768+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:14.953476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360237+0000","last_change":"2024-03-06T07:21:54.360237+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360237+0000","last_undegraded":"2024-03-06T07:21:54.360237+0000","last_fullsized":"2024-03-06T07:21:54.360237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379825+0000","last_change":"2024-03-06T07:21:54.375783+0000","last_active":"2024-03-06T07:21:54.379825+0000","last_peered":"2024-03-06T07:21:54.379825+0000","last_clean":"2024-03-06T07:21:54.379825+0000","last_became_active":"2024-03-06T07:21:54.375473+0000","last_became_peered":"2024-03-06T07:21:54.375473+0000","last_unstale":"2024-03-06T07:21:54.379825+0000","last_undegraded":"2024-03-06T07:21:54.379825+0000","last_fullsized":"2024-03-06T07:21:54.379825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:08.612677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370306+0000","last_change":"2024-03-06T07:21:50.343971+0000","last_active":"2024-03-06T07:21:54.370306+0000","last_peered":"2024-03-06T07:21:54.370306+0000","last_clean":"2024-03-06T07:21:54.370306+0000","last_became_active":"2024-03-06T07:21:50.343658+0000","last_became_peered":"2024-03-06T07:21:50.343658+0000","last_unstale":"2024-03-06T07:21:54.370306+0000","last_undegraded":"2024-03-06T07:21:54.370306+0000","last_fullsized":"2024-03-06T07:21:54.370306+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364922+0000","last_change":"2024-03-06T07:21:42.317614+0000","last_active":"2024-03-06T07:21:54.364922+0000","last_peered":"2024-03-06T07:21:54.364922+0000","last_clean":"2024-03-06T07:21:54.364922+0000","last_became_active":"2024-03-06T07:21:42.317094+0000","last_became_peered":"2024-03-06T07:21:42.317094+0000","last_unstale":"2024-03-06T07:21:54.364922+0000","last_undegraded":"2024-03-06T07:21:54.364922+0000","last_fullsized":"2024-03-06T07:21:54.364922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:15.739191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364680+0000","last_change":"2024-03-06T07:21:54.364680+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364680+0000","last_undegraded":"2024-03-06T07:21:54.364680+0000","last_fullsized":"2024-03-06T07:21:54.364680+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377152+0000","last_change":"2024-03-06T07:21:54.375914+0000","last_active":"2024-03-06T07:21:54.377152+0000","last_peered":"2024-03-06T07:21:54.377152+0000","last_clean":"2024-03-06T07:21:54.377152+0000","last_became_active":"2024-03-06T07:21:54.375619+0000","last_became_peered":"2024-03-06T07:21:54.375619+0000","last_unstale":"2024-03-06T07:21:54.377152+0000","last_undegraded":"2024-03-06T07:21:54.377152+0000","last_fullsized":"2024-03-06T07:21:54.377152+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:26.719365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358617+0000","last_change":"2024-03-06T07:21:43.318727+0000","last_active":"2024-03-06T07:21:54.358617+0000","last_peered":"2024-03-06T07:21:54.358617+0000","last_clean":"2024-03-06T07:21:54.358617+0000","last_became_active":"2024-03-06T07:21:43.318440+0000","last_became_peered":"2024-03-06T07:21:43.318440+0000","last_unstale":"2024-03-06T07:21:54.358617+0000","last_undegraded":"2024-03-06T07:21:54.358617+0000","last_fullsized":"2024-03-06T07:21:54.358617+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370222+0000","last_change":"2024-03-06T07:21:42.307726+0000","last_active":"2024-03-06T07:21:54.370222+0000","last_peered":"2024-03-06T07:21:54.370222+0000","last_clean":"2024-03-06T07:21:54.370222+0000","last_became_active":"2024-03-06T07:21:42.307445+0000","last_became_peered":"2024-03-06T07:21:42.307445+0000","last_unstale":"2024-03-06T07:21:54.370222+0000","last_undegraded":"2024-03-06T07:21:54.370222+0000","last_fullsized":"2024-03-06T07:21:54.370222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366515+0000","last_change":"2024-03-06T07:21:54.366515+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366515+0000","last_undegraded":"2024-03-06T07:21:54.366515+0000","last_fullsized":"2024-03-06T07:21:54.366515+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376980+0000","last_change":"2024-03-06T07:21:54.370955+0000","last_active":"2024-03-06T07:21:54.376980+0000","last_peered":"2024-03-06T07:21:54.376980+0000","last_clean":"2024-03-06T07:21:54.376980+0000","last_became_active":"2024-03-06T07:21:54.370680+0000","last_became_peered":"2024-03-06T07:21:54.370680+0000","last_unstale":"2024-03-06T07:21:54.376980+0000","last_undegraded":"2024-03-06T07:21:54.376980+0000","last_fullsized":"2024-03-06T07:21:54.376980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359348+0000","last_change":"2024-03-06T07:21:43.318372+0000","last_active":"2024-03-06T07:21:54.359348+0000","last_peered":"2024-03-06T07:21:54.359348+0000","last_clean":"2024-03-06T07:21:54.359348+0000","last_became_active":"2024-03-06T07:21:43.318047+0000","last_became_peered":"2024-03-06T07:21:43.318047+0000","last_unstale":"2024-03-06T07:21:54.359348+0000","last_undegraded":"2024-03-06T07:21:54.359348+0000","last_fullsized":"2024-03-06T07:21:54.359348+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364556+0000","last_change":"2024-03-06T07:21:42.310677+0000","last_active":"2024-03-06T07:21:54.364556+0000","last_peered":"2024-03-06T07:21:54.364556+0000","last_clean":"2024-03-06T07:21:54.364556+0000","last_became_active":"2024-03-06T07:21:42.310375+0000","last_became_peered":"2024-03-06T07:21:42.310375+0000","last_unstale":"2024-03-06T07:21:54.364556+0000","last_undegraded":"2024-03-06T07:21:54.364556+0000","last_fullsized":"2024-03-06T07:21:54.364556+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:13.837451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358658+0000","last_change":"2024-03-06T07:21:54.358658+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358658+0000","last_undegraded":"2024-03-06T07:21:54.358658+0000","last_fullsized":"2024-03-06T07:21:54.358658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379656+0000","last_change":"2024-03-06T07:21:54.374914+0000","last_active":"2024-03-06T07:21:54.379656+0000","last_peered":"2024-03-06T07:21:54.379656+0000","last_clean":"2024-03-06T07:21:54.379656+0000","last_became_active":"2024-03-06T07:21:54.374570+0000","last_became_peered":"2024-03-06T07:21:54.374570+0000","last_unstale":"2024-03-06T07:21:54.379656+0000","last_undegraded":"2024-03-06T07:21:54.379656+0000","last_fullsized":"2024-03-06T07:21:54.379656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:05:03.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358220+0000","last_change":"2024-03-06T07:21:43.325740+0000","last_active":"2024-03-06T07:21:54.358220+0000","last_peered":"2024-03-06T07:21:54.358220+0000","last_clean":"2024-03-06T07:21:54.358220+0000","last_became_active":"2024-03-06T07:21:43.325425+0000","last_became_peered":"2024-03-06T07:21:43.325425+0000","last_unstale":"2024-03-06T07:21:54.358220+0000","last_undegraded":"2024-03-06T07:21:54.358220+0000","last_fullsized":"2024-03-06T07:21:54.358220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364895+0000","last_change":"2024-03-06T07:21:42.319786+0000","last_active":"2024-03-06T07:21:54.364895+0000","last_peered":"2024-03-06T07:21:54.364895+0000","last_clean":"2024-03-06T07:21:54.364895+0000","last_became_active":"2024-03-06T07:21:42.319430+0000","last_became_peered":"2024-03-06T07:21:42.319430+0000","last_unstale":"2024-03-06T07:21:54.364895+0000","last_undegraded":"2024-03-06T07:21:54.364895+0000","last_fullsized":"2024-03-06T07:21:54.364895+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:09.719754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360176+0000","last_change":"2024-03-06T07:21:54.360176+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360176+0000","last_undegraded":"2024-03-06T07:21:54.360176+0000","last_fullsized":"2024-03-06T07:21:54.360176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369499+0000","last_change":"2024-03-06T07:21:54.367800+0000","last_active":"2024-03-06T07:21:54.369499+0000","last_peered":"2024-03-06T07:21:54.369499+0000","last_clean":"2024-03-06T07:21:54.369499+0000","last_became_active":"2024-03-06T07:21:54.367468+0000","last_became_peered":"2024-03-06T07:21:54.367468+0000","last_unstale":"2024-03-06T07:21:54.369499+0000","last_undegraded":"2024-03-06T07:21:54.369499+0000","last_fullsized":"2024-03-06T07:21:54.369499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.330805+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.330267+0000","last_became_peered":"2024-03-06T07:21:43.330267+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:56.716578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358318+0000","last_change":"2024-03-06T07:21:42.312213+0000","last_active":"2024-03-06T07:21:54.358318+0000","last_peered":"2024-03-06T07:21:54.358318+0000","last_clean":"2024-03-06T07:21:54.358318+0000","last_became_active":"2024-03-06T07:21:42.311878+0000","last_became_peered":"2024-03-06T07:21:42.311878+0000","last_unstale":"2024-03-06T07:21:54.358318+0000","last_undegraded":"2024-03-06T07:21:54.358318+0000","last_fullsized":"2024-03-06T07:21:54.358318+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358914+0000","last_change":"2024-03-06T07:21:54.358914+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358914+0000","last_undegraded":"2024-03-06T07:21:54.358914+0000","last_fullsized":"2024-03-06T07:21:54.358914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380052+0000","last_change":"2024-03-06T07:21:54.377646+0000","last_active":"2024-03-06T07:21:54.380052+0000","last_peered":"2024-03-06T07:21:54.380052+0000","last_clean":"2024-03-06T07:21:54.380052+0000","last_became_active":"2024-03-06T07:21:54.377219+0000","last_became_peered":"2024-03-06T07:21:54.377219+0000","last_unstale":"2024-03-06T07:21:54.380052+0000","last_undegraded":"2024-03-06T07:21:54.380052+0000","last_fullsized":"2024-03-06T07:21:54.380052+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:59.768227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:43.320122+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:43.319799+0000","last_became_peered":"2024-03-06T07:21:43.319799+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370295+0000","last_change":"2024-03-06T07:21:42.319681+0000","last_active":"2024-03-06T07:21:54.370295+0000","last_peered":"2024-03-06T07:21:54.370295+0000","last_clean":"2024-03-06T07:21:54.370295+0000","last_became_active":"2024-03-06T07:21:42.319452+0000","last_became_peered":"2024-03-06T07:21:42.319452+0000","last_unstale":"2024-03-06T07:21:54.370295+0000","last_undegraded":"2024-03-06T07:21:54.370295+0000","last_fullsized":"2024-03-06T07:21:54.370295+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:35.308790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362688+0000","last_change":"2024-03-06T07:21:54.362688+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362688+0000","last_undegraded":"2024-03-06T07:21:54.362688+0000","last_fullsized":"2024-03-06T07:21:54.362688+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368199+0000","last_change":"2024-03-06T07:21:54.367115+0000","last_active":"2024-03-06T07:21:54.368199+0000","last_peered":"2024-03-06T07:21:54.368199+0000","last_clean":"2024-03-06T07:21:54.368199+0000","last_became_active":"2024-03-06T07:21:54.366564+0000","last_became_peered":"2024-03-06T07:21:54.366564+0000","last_unstale":"2024-03-06T07:21:54.368199+0000","last_undegraded":"2024-03-06T07:21:54.368199+0000","last_fullsized":"2024-03-06T07:21:54.368199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365114+0000","last_change":"2024-03-06T07:21:43.323490+0000","last_active":"2024-03-06T07:21:54.365114+0000","last_peered":"2024-03-06T07:21:54.365114+0000","last_clean":"2024-03-06T07:21:54.365114+0000","last_became_active":"2024-03-06T07:21:43.323088+0000","last_became_peered":"2024-03-06T07:21:43.323088+0000","last_unstale":"2024-03-06T07:21:54.365114+0000","last_undegraded":"2024-03-06T07:21:54.365114+0000","last_fullsized":"2024-03-06T07:21:54.365114+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358795+0000","last_change":"2024-03-06T07:21:42.312732+0000","last_active":"2024-03-06T07:21:54.358795+0000","last_peered":"2024-03-06T07:21:54.358795+0000","last_clean":"2024-03-06T07:21:54.358795+0000","last_became_active":"2024-03-06T07:21:42.312462+0000","last_became_peered":"2024-03-06T07:21:42.312462+0000","last_unstale":"2024-03-06T07:21:54.358795+0000","last_undegraded":"2024-03-06T07:21:54.358795+0000","last_fullsized":"2024-03-06T07:21:54.358795+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363414+0000","last_change":"2024-03-06T07:21:54.363414+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363414+0000","last_undegraded":"2024-03-06T07:21:54.363414+0000","last_fullsized":"2024-03-06T07:21:54.363414+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367522+0000","last_change":"2024-03-06T07:21:54.365558+0000","last_active":"2024-03-06T07:21:54.367522+0000","last_peered":"2024-03-06T07:21:54.367522+0000","last_clean":"2024-03-06T07:21:54.367522+0000","last_became_active":"2024-03-06T07:21:54.365272+0000","last_became_peered":"2024-03-06T07:21:54.365272+0000","last_unstale":"2024-03-06T07:21:54.367522+0000","last_undegraded":"2024-03-06T07:21:54.367522+0000","last_fullsized":"2024-03-06T07:21:54.367522+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369842+0000","last_change":"2024-03-06T07:21:43.320022+0000","last_active":"2024-03-06T07:21:54.369842+0000","last_peered":"2024-03-06T07:21:54.369842+0000","last_clean":"2024-03-06T07:21:54.369842+0000","last_became_active":"2024-03-06T07:21:43.319716+0000","last_became_peered":"2024-03-06T07:21:43.319716+0000","last_unstale":"2024-03-06T07:21:54.369842+0000","last_undegraded":"2024-03-06T07:21:54.369842+0000","last_fullsized":"2024-03-06T07:21:54.369842+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:50.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359596+0000","last_change":"2024-03-06T07:21:42.308298+0000","last_active":"2024-03-06T07:21:54.359596+0000","last_peered":"2024-03-06T07:21:54.359596+0000","last_clean":"2024-03-06T07:21:54.359596+0000","last_became_active":"2024-03-06T07:21:42.308008+0000","last_became_peered":"2024-03-06T07:21:42.308008+0000","last_unstale":"2024-03-06T07:21:54.359596+0000","last_undegraded":"2024-03-06T07:21:54.359596+0000","last_fullsized":"2024-03-06T07:21:54.359596+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:51.723259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361630+0000","last_change":"2024-03-06T07:21:54.361630+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361630+0000","last_undegraded":"2024-03-06T07:21:54.361630+0000","last_fullsized":"2024-03-06T07:21:54.361630+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376550+0000","last_change":"2024-03-06T07:21:54.375937+0000","last_active":"2024-03-06T07:21:54.376550+0000","last_peered":"2024-03-06T07:21:54.376550+0000","last_clean":"2024-03-06T07:21:54.376550+0000","last_became_active":"2024-03-06T07:21:54.375648+0000","last_became_peered":"2024-03-06T07:21:54.375648+0000","last_unstale":"2024-03-06T07:21:54.376550+0000","last_undegraded":"2024-03-06T07:21:54.376550+0000","last_fullsized":"2024-03-06T07:21:54.376550+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359583+0000","last_change":"2024-03-06T07:21:43.323228+0000","last_active":"2024-03-06T07:21:54.359583+0000","last_peered":"2024-03-06T07:21:54.359583+0000","last_clean":"2024-03-06T07:21:54.359583+0000","last_became_active":"2024-03-06T07:21:43.322951+0000","last_became_peered":"2024-03-06T07:21:43.322951+0000","last_unstale":"2024-03-06T07:21:54.359583+0000","last_undegraded":"2024-03-06T07:21:54.359583+0000","last_fullsized":"2024-03-06T07:21:54.359583+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365013+0000","last_change":"2024-03-06T07:21:42.321872+0000","last_active":"2024-03-06T07:21:54.365013+0000","last_peered":"2024-03-06T07:21:54.365013+0000","last_clean":"2024-03-06T07:21:54.365013+0000","last_became_active":"2024-03-06T07:21:42.321384+0000","last_became_peered":"2024-03-06T07:21:42.321384+0000","last_unstale":"2024-03-06T07:21:54.365013+0000","last_undegraded":"2024-03-06T07:21:54.365013+0000","last_fullsized":"2024-03-06T07:21:54.365013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358625+0000","last_change":"2024-03-06T07:21:54.358625+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358625+0000","last_undegraded":"2024-03-06T07:21:54.358625+0000","last_fullsized":"2024-03-06T07:21:54.358625+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375601+0000","last_change":"2024-03-06T07:21:54.373488+0000","last_active":"2024-03-06T07:21:54.375601+0000","last_peered":"2024-03-06T07:21:54.375601+0000","last_clean":"2024-03-06T07:21:54.375601+0000","last_became_active":"2024-03-06T07:21:54.373225+0000","last_became_peered":"2024-03-06T07:21:54.373225+0000","last_unstale":"2024-03-06T07:21:54.375601+0000","last_undegraded":"2024-03-06T07:21:54.375601+0000","last_fullsized":"2024-03-06T07:21:54.375601+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369356+0000","last_change":"2024-03-06T07:21:43.324244+0000","last_active":"2024-03-06T07:21:54.369356+0000","last_peered":"2024-03-06T07:21:54.369356+0000","last_clean":"2024-03-06T07:21:54.369356+0000","last_became_active":"2024-03-06T07:21:43.323549+0000","last_became_peered":"2024-03-06T07:21:43.323549+0000","last_unstale":"2024-03-06T07:21:54.369356+0000","last_undegraded":"2024-03-06T07:21:54.369356+0000","last_fullsized":"2024-03-06T07:21:54.369356+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364546+0000","last_change":"2024-03-06T07:21:42.318308+0000","last_active":"2024-03-06T07:21:54.364546+0000","last_peered":"2024-03-06T07:21:54.364546+0000","last_clean":"2024-03-06T07:21:54.364546+0000","last_became_active":"2024-03-06T07:21:42.318080+0000","last_became_peered":"2024-03-06T07:21:42.318080+0000","last_unstale":"2024-03-06T07:21:54.364546+0000","last_undegraded":"2024-03-06T07:21:54.364546+0000","last_fullsized":"2024-03-06T07:21:54.364546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360511+0000","last_change":"2024-03-06T07:21:54.360511+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360511+0000","last_undegraded":"2024-03-06T07:21:54.360511+0000","last_fullsized":"2024-03-06T07:21:54.360511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378458+0000","last_change":"2024-03-06T07:21:54.377690+0000","last_active":"2024-03-06T07:21:54.378458+0000","last_peered":"2024-03-06T07:21:54.378458+0000","last_clean":"2024-03-06T07:21:54.378458+0000","last_became_active":"2024-03-06T07:21:54.377408+0000","last_became_peered":"2024-03-06T07:21:54.377408+0000","last_unstale":"2024-03-06T07:21:54.378458+0000","last_undegraded":"2024-03-06T07:21:54.378458+0000","last_fullsized":"2024-03-06T07:21:54.378458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:31.244976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359542+0000","last_change":"2024-03-06T07:21:43.331853+0000","last_active":"2024-03-06T07:21:54.359542+0000","last_peered":"2024-03-06T07:21:54.359542+0000","last_clean":"2024-03-06T07:21:54.359542+0000","last_became_active":"2024-03-06T07:21:43.331560+0000","last_became_peered":"2024-03-06T07:21:43.331560+0000","last_unstale":"2024-03-06T07:21:54.359542+0000","last_undegraded":"2024-03-06T07:21:54.359542+0000","last_fullsized":"2024-03-06T07:21:54.359542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357780+0000","last_change":"2024-03-06T07:21:42.309167+0000","last_active":"2024-03-06T07:21:54.357780+0000","last_peered":"2024-03-06T07:21:54.357780+0000","last_clean":"2024-03-06T07:21:54.357780+0000","last_became_active":"2024-03-06T07:21:42.308882+0000","last_became_peered":"2024-03-06T07:21:42.308882+0000","last_unstale":"2024-03-06T07:21:54.357780+0000","last_undegraded":"2024-03-06T07:21:54.357780+0000","last_fullsized":"2024-03-06T07:21:54.357780+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366074+0000","last_change":"2024-03-06T07:21:54.366074+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366074+0000","last_undegraded":"2024-03-06T07:21:54.366074+0000","last_fullsized":"2024-03-06T07:21:54.366074+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378021+0000","last_change":"2024-03-06T07:21:54.377427+0000","last_active":"2024-03-06T07:21:54.378021+0000","last_peered":"2024-03-06T07:21:54.378021+0000","last_clean":"2024-03-06T07:21:54.378021+0000","last_became_active":"2024-03-06T07:21:54.377183+0000","last_became_peered":"2024-03-06T07:21:54.377183+0000","last_unstale":"2024-03-06T07:21:54.378021+0000","last_undegraded":"2024-03-06T07:21:54.378021+0000","last_fullsized":"2024-03-06T07:21:54.378021+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.324536+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.323818+0000","last_became_peered":"2024-03-06T07:21:43.323818+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358547+0000","last_change":"2024-03-06T07:21:42.308460+0000","last_active":"2024-03-06T07:21:54.358547+0000","last_peered":"2024-03-06T07:21:54.358547+0000","last_clean":"2024-03-06T07:21:54.358547+0000","last_became_active":"2024-03-06T07:21:42.307461+0000","last_became_peered":"2024-03-06T07:21:42.307461+0000","last_unstale":"2024-03-06T07:21:54.358547+0000","last_undegraded":"2024-03-06T07:21:54.358547+0000","last_fullsized":"2024-03-06T07:21:54.358547+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:34.346180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364356+0000","last_change":"2024-03-06T07:21:54.364356+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364356+0000","last_undegraded":"2024-03-06T07:21:54.364356+0000","last_fullsized":"2024-03-06T07:21:54.364356+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367923+0000","last_change":"2024-03-06T07:21:54.366324+0000","last_active":"2024-03-06T07:21:54.367923+0000","last_peered":"2024-03-06T07:21:54.367923+0000","last_clean":"2024-03-06T07:21:54.367923+0000","last_became_active":"2024-03-06T07:21:54.366064+0000","last_became_peered":"2024-03-06T07:21:54.366064+0000","last_unstale":"2024-03-06T07:21:54.367923+0000","last_undegraded":"2024-03-06T07:21:54.367923+0000","last_fullsized":"2024-03-06T07:21:54.367923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358427+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.358427+0000","last_peered":"2024-03-06T07:21:54.358427+0000","last_clean":"2024-03-06T07:21:54.358427+0000","last_became_active":"2024-03-06T07:21:43.316848+0000","last_became_peered":"2024-03-06T07:21:43.316848+0000","last_unstale":"2024-03-06T07:21:54.358427+0000","last_undegraded":"2024-03-06T07:21:54.358427+0000","last_fullsized":"2024-03-06T07:21:54.358427+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370393+0000","last_change":"2024-03-06T07:21:42.319998+0000","last_active":"2024-03-06T07:21:54.370393+0000","last_peered":"2024-03-06T07:21:54.370393+0000","last_clean":"2024-03-06T07:21:54.370393+0000","last_became_active":"2024-03-06T07:21:42.319529+0000","last_became_peered":"2024-03-06T07:21:42.319529+0000","last_unstale":"2024-03-06T07:21:54.370393+0000","last_undegraded":"2024-03-06T07:21:54.370393+0000","last_fullsized":"2024-03-06T07:21:54.370393+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361900+0000","last_change":"2024-03-06T07:21:54.361900+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361900+0000","last_undegraded":"2024-03-06T07:21:54.361900+0000","last_fullsized":"2024-03-06T07:21:54.361900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374081+0000","last_change":"2024-03-06T07:21:54.370046+0000","last_active":"2024-03-06T07:21:54.374081+0000","last_peered":"2024-03-06T07:21:54.374081+0000","last_clean":"2024-03-06T07:21:54.374081+0000","last_became_active":"2024-03-06T07:21:54.369791+0000","last_became_peered":"2024-03-06T07:21:54.369791+0000","last_unstale":"2024-03-06T07:21:54.374081+0000","last_undegraded":"2024-03-06T07:21:54.374081+0000","last_fullsized":"2024-03-06T07:21:54.374081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:09.373432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367145+0000","last_change":"2024-03-06T07:21:43.327074+0000","last_active":"2024-03-06T07:21:54.367145+0000","last_peered":"2024-03-06T07:21:54.367145+0000","last_clean":"2024-03-06T07:21:54.367145+0000","last_became_active":"2024-03-06T07:21:43.326797+0000","last_became_peered":"2024-03-06T07:21:43.326797+0000","last_unstale":"2024-03-06T07:21:54.367145+0000","last_undegraded":"2024-03-06T07:21:54.367145+0000","last_fullsized":"2024-03-06T07:21:54.367145+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370126+0000","last_change":"2024-03-06T07:21:42.310454+0000","last_active":"2024-03-06T07:21:54.370126+0000","last_peered":"2024-03-06T07:21:54.370126+0000","last_clean":"2024-03-06T07:21:54.370126+0000","last_became_active":"2024-03-06T07:21:42.310172+0000","last_became_peered":"2024-03-06T07:21:42.310172+0000","last_unstale":"2024-03-06T07:21:54.370126+0000","last_undegraded":"2024-03-06T07:21:54.370126+0000","last_fullsized":"2024-03-06T07:21:54.370126+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:34.182536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360981+0000","last_change":"2024-03-06T07:21:54.360981+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360981+0000","last_undegraded":"2024-03-06T07:21:54.360981+0000","last_fullsized":"2024-03-06T07:21:54.360981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376984+0000","last_change":"2024-03-06T07:21:54.368897+0000","last_active":"2024-03-06T07:21:54.376984+0000","last_peered":"2024-03-06T07:21:54.376984+0000","last_clean":"2024-03-06T07:21:54.376984+0000","last_became_active":"2024-03-06T07:21:54.368398+0000","last_became_peered":"2024-03-06T07:21:54.368398+0000","last_unstale":"2024-03-06T07:21:54.376984+0000","last_undegraded":"2024-03-06T07:21:54.376984+0000","last_fullsized":"2024-03-06T07:21:54.376984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369221+0000","last_change":"2024-03-06T07:21:43.324398+0000","last_active":"2024-03-06T07:21:54.369221+0000","last_peered":"2024-03-06T07:21:54.369221+0000","last_clean":"2024-03-06T07:21:54.369221+0000","last_became_active":"2024-03-06T07:21:43.324115+0000","last_became_peered":"2024-03-06T07:21:43.324115+0000","last_unstale":"2024-03-06T07:21:54.369221+0000","last_undegraded":"2024-03-06T07:21:54.369221+0000","last_fullsized":"2024-03-06T07:21:54.369221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:36.888023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365848+0000","last_change":"2024-03-06T07:21:42.312390+0000","last_active":"2024-03-06T07:21:54.365848+0000","last_peered":"2024-03-06T07:21:54.365848+0000","last_clean":"2024-03-06T07:21:54.365848+0000","last_became_active":"2024-03-06T07:21:42.312079+0000","last_became_peered":"2024-03-06T07:21:42.312079+0000","last_unstale":"2024-03-06T07:21:54.365848+0000","last_undegraded":"2024-03-06T07:21:54.365848+0000","last_fullsized":"2024-03-06T07:21:54.365848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364238+0000","last_change":"2024-03-06T07:21:54.364238+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364238+0000","last_undegraded":"2024-03-06T07:21:54.364238+0000","last_fullsized":"2024-03-06T07:21:54.364238+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372534+0000","last_change":"2024-03-06T07:21:54.370872+0000","last_active":"2024-03-06T07:21:54.372534+0000","last_peered":"2024-03-06T07:21:54.372534+0000","last_clean":"2024-03-06T07:21:54.372534+0000","last_became_active":"2024-03-06T07:21:54.370589+0000","last_became_peered":"2024-03-06T07:21:54.370589+0000","last_unstale":"2024-03-06T07:21:54.372534+0000","last_undegraded":"2024-03-06T07:21:54.372534+0000","last_fullsized":"2024-03-06T07:21:54.372534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:36.087274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368542+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.368542+0000","last_peered":"2024-03-06T07:21:54.368542+0000","last_clean":"2024-03-06T07:21:54.368542+0000","last_became_active":"2024-03-06T07:21:43.316857+0000","last_became_peered":"2024-03-06T07:21:43.316857+0000","last_unstale":"2024-03-06T07:21:54.368542+0000","last_undegraded":"2024-03-06T07:21:54.368542+0000","last_fullsized":"2024-03-06T07:21:54.368542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:48.715734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365550+0000","last_change":"2024-03-06T07:21:42.311972+0000","last_active":"2024-03-06T07:21:54.365550+0000","last_peered":"2024-03-06T07:21:54.365550+0000","last_clean":"2024-03-06T07:21:54.365550+0000","last_became_active":"2024-03-06T07:21:42.311640+0000","last_became_peered":"2024-03-06T07:21:42.311640+0000","last_unstale":"2024-03-06T07:21:54.365550+0000","last_undegraded":"2024-03-06T07:21:54.365550+0000","last_fullsized":"2024-03-06T07:21:54.365550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:22.568258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363210+0000","last_change":"2024-03-06T07:21:54.363210+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363210+0000","last_undegraded":"2024-03-06T07:21:54.363210+0000","last_fullsized":"2024-03-06T07:21:54.363210+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375823+0000","last_change":"2024-03-06T07:21:54.372740+0000","last_active":"2024-03-06T07:21:54.375823+0000","last_peered":"2024-03-06T07:21:54.375823+0000","last_clean":"2024-03-06T07:21:54.375823+0000","last_became_active":"2024-03-06T07:21:54.371525+0000","last_became_peered":"2024-03-06T07:21:54.371525+0000","last_unstale":"2024-03-06T07:21:54.375823+0000","last_undegraded":"2024-03-06T07:21:54.375823+0000","last_fullsized":"2024-03-06T07:21:54.375823+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359675+0000","last_change":"2024-03-06T07:21:43.320374+0000","last_active":"2024-03-06T07:21:54.359675+0000","last_peered":"2024-03-06T07:21:54.359675+0000","last_clean":"2024-03-06T07:21:54.359675+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.359675+0000","last_undegraded":"2024-03-06T07:21:54.359675+0000","last_fullsized":"2024-03-06T07:21:54.359675+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368677+0000","last_change":"2024-03-06T07:21:42.309901+0000","last_active":"2024-03-06T07:21:54.368677+0000","last_peered":"2024-03-06T07:21:54.368677+0000","last_clean":"2024-03-06T07:21:54.368677+0000","last_became_active":"2024-03-06T07:21:42.309532+0000","last_became_peered":"2024-03-06T07:21:42.309532+0000","last_unstale":"2024-03-06T07:21:54.368677+0000","last_undegraded":"2024-03-06T07:21:54.368677+0000","last_fullsized":"2024-03-06T07:21:54.368677+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361638+0000","last_change":"2024-03-06T07:21:54.361638+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361638+0000","last_undegraded":"2024-03-06T07:21:54.361638+0000","last_fullsized":"2024-03-06T07:21:54.361638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377134+0000","last_change":"2024-03-06T07:21:54.375852+0000","last_active":"2024-03-06T07:21:54.377134+0000","last_peered":"2024-03-06T07:21:54.377134+0000","last_clean":"2024-03-06T07:21:54.377134+0000","last_became_active":"2024-03-06T07:21:54.375600+0000","last_became_peered":"2024-03-06T07:21:54.375600+0000","last_unstale":"2024-03-06T07:21:54.377134+0000","last_undegraded":"2024-03-06T07:21:54.377134+0000","last_fullsized":"2024-03-06T07:21:54.377134+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:40.575618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365686+0000","last_change":"2024-03-06T07:21:43.320820+0000","last_active":"2024-03-06T07:21:54.365686+0000","last_peered":"2024-03-06T07:21:54.365686+0000","last_clean":"2024-03-06T07:21:54.365686+0000","last_became_active":"2024-03-06T07:21:43.319225+0000","last_became_peered":"2024-03-06T07:21:43.319225+0000","last_unstale":"2024-03-06T07:21:54.365686+0000","last_undegraded":"2024-03-06T07:21:54.365686+0000","last_fullsized":"2024-03-06T07:21:54.365686+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358917+0000","last_change":"2024-03-06T07:21:42.307676+0000","last_active":"2024-03-06T07:21:54.358917+0000","last_peered":"2024-03-06T07:21:54.358917+0000","last_clean":"2024-03-06T07:21:54.358917+0000","last_became_active":"2024-03-06T07:21:42.307388+0000","last_became_peered":"2024-03-06T07:21:42.307388+0000","last_unstale":"2024-03-06T07:21:54.358917+0000","last_undegraded":"2024-03-06T07:21:54.358917+0000","last_fullsized":"2024-03-06T07:21:54.358917+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:47.711911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361080+0000","last_change":"2024-03-06T07:21:54.361080+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361080+0000","last_undegraded":"2024-03-06T07:21:54.361080+0000","last_fullsized":"2024-03-06T07:21:54.361080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380012+0000","last_change":"2024-03-06T07:21:54.374823+0000","last_active":"2024-03-06T07:21:54.380012+0000","last_peered":"2024-03-06T07:21:54.380012+0000","last_clean":"2024-03-06T07:21:54.380012+0000","last_became_active":"2024-03-06T07:21:54.374515+0000","last_became_peered":"2024-03-06T07:21:54.374515+0000","last_unstale":"2024-03-06T07:21:54.380012+0000","last_undegraded":"2024-03-06T07:21:54.380012+0000","last_fullsized":"2024-03-06T07:21:54.380012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369756+0000","last_change":"2024-03-06T07:21:43.329841+0000","last_active":"2024-03-06T07:21:54.369756+0000","last_peered":"2024-03-06T07:21:54.369756+0000","last_clean":"2024-03-06T07:21:54.369756+0000","last_became_active":"2024-03-06T07:21:43.329526+0000","last_became_peered":"2024-03-06T07:21:43.329526+0000","last_unstale":"2024-03-06T07:21:54.369756+0000","last_undegraded":"2024-03-06T07:21:54.369756+0000","last_fullsized":"2024-03-06T07:21:54.369756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368782+0000","last_change":"2024-03-06T07:21:42.318168+0000","last_active":"2024-03-06T07:21:54.368782+0000","last_peered":"2024-03-06T07:21:54.368782+0000","last_clean":"2024-03-06T07:21:54.368782+0000","last_became_active":"2024-03-06T07:21:42.317866+0000","last_became_peered":"2024-03-06T07:21:42.317866+0000","last_unstale":"2024-03-06T07:21:54.368782+0000","last_undegraded":"2024-03-06T07:21:54.368782+0000","last_fullsized":"2024-03-06T07:21:54.368782+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:49.054000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361933+0000","last_change":"2024-03-06T07:21:54.361933+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361933+0000","last_undegraded":"2024-03-06T07:21:54.361933+0000","last_fullsized":"2024-03-06T07:21:54.361933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374694+0000","last_change":"2024-03-06T07:21:54.369742+0000","last_active":"2024-03-06T07:21:54.374694+0000","last_peered":"2024-03-06T07:21:54.374694+0000","last_clean":"2024-03-06T07:21:54.374694+0000","last_became_active":"2024-03-06T07:21:54.368872+0000","last_became_peered":"2024-03-06T07:21:54.368872+0000","last_unstale":"2024-03-06T07:21:54.374694+0000","last_undegraded":"2024-03-06T07:21:54.374694+0000","last_fullsized":"2024-03-06T07:21:54.374694+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373011+0000","last_change":"2024-03-06T07:21:43.316009+0000","last_active":"2024-03-06T07:21:54.373011+0000","last_peered":"2024-03-06T07:21:54.373011+0000","last_clean":"2024-03-06T07:21:54.373011+0000","last_became_active":"2024-03-06T07:21:43.315699+0000","last_became_peered":"2024-03-06T07:21:43.315699+0000","last_unstale":"2024-03-06T07:21:54.373011+0000","last_undegraded":"2024-03-06T07:21:54.373011+0000","last_fullsized":"2024-03-06T07:21:54.373011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:52.769462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365150+0000","last_change":"2024-03-06T07:21:42.311690+0000","last_active":"2024-03-06T07:21:54.365150+0000","last_peered":"2024-03-06T07:21:54.365150+0000","last_clean":"2024-03-06T07:21:54.365150+0000","last_became_active":"2024-03-06T07:21:42.311417+0000","last_became_peered":"2024-03-06T07:21:42.311417+0000","last_unstale":"2024-03-06T07:21:54.365150+0000","last_undegraded":"2024-03-06T07:21:54.365150+0000","last_fullsized":"2024-03-06T07:21:54.365150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.094878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362069+0000","last_change":"2024-03-06T07:21:54.362069+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362069+0000","last_undegraded":"2024-03-06T07:21:54.362069+0000","last_fullsized":"2024-03-06T07:21:54.362069+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378181+0000","last_change":"2024-03-06T07:21:54.377550+0000","last_active":"2024-03-06T07:21:54.378181+0000","last_peered":"2024-03-06T07:21:54.378181+0000","last_clean":"2024-03-06T07:21:54.378181+0000","last_became_active":"2024-03-06T07:21:54.377271+0000","last_became_peered":"2024-03-06T07:21:54.377271+0000","last_unstale":"2024-03-06T07:21:54.378181+0000","last_undegraded":"2024-03-06T07:21:54.378181+0000","last_fullsized":"2024-03-06T07:21:54.378181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360360+0000","last_change":"2024-03-06T07:21:43.322229+0000","last_active":"2024-03-06T07:21:54.360360+0000","last_peered":"2024-03-06T07:21:54.360360+0000","last_clean":"2024-03-06T07:21:54.360360+0000","last_became_active":"2024-03-06T07:21:43.321934+0000","last_became_peered":"2024-03-06T07:21:43.321934+0000","last_unstale":"2024-03-06T07:21:54.360360+0000","last_undegraded":"2024-03-06T07:21:54.360360+0000","last_fullsized":"2024-03-06T07:21:54.360360+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365379+0000","last_change":"2024-03-06T07:21:42.321714+0000","last_active":"2024-03-06T07:21:54.365379+0000","last_peered":"2024-03-06T07:21:54.365379+0000","last_clean":"2024-03-06T07:21:54.365379+0000","last_became_active":"2024-03-06T07:21:42.321444+0000","last_became_peered":"2024-03-06T07:21:42.321444+0000","last_unstale":"2024-03-06T07:21:54.365379+0000","last_undegraded":"2024-03-06T07:21:54.365379+0000","last_fullsized":"2024-03-06T07:21:54.365379+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361491+0000","last_change":"2024-03-06T07:21:54.361491+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361491+0000","last_undegraded":"2024-03-06T07:21:54.361491+0000","last_fullsized":"2024-03-06T07:21:54.361491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373855+0000","last_change":"2024-03-06T07:21:54.371666+0000","last_active":"2024-03-06T07:21:54.373855+0000","last_peered":"2024-03-06T07:21:54.373855+0000","last_clean":"2024-03-06T07:21:54.373855+0000","last_became_active":"2024-03-06T07:21:54.371394+0000","last_became_peered":"2024-03-06T07:21:54.371394+0000","last_unstale":"2024-03-06T07:21:54.373855+0000","last_undegraded":"2024-03-06T07:21:54.373855+0000","last_fullsized":"2024-03-06T07:21:54.373855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:15.173188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368609+0000","last_change":"2024-03-06T07:21:43.316403+0000","last_active":"2024-03-06T07:21:54.368609+0000","last_peered":"2024-03-06T07:21:54.368609+0000","last_clean":"2024-03-06T07:21:54.368609+0000","last_became_active":"2024-03-06T07:21:43.315416+0000","last_became_peered":"2024-03-06T07:21:43.315416+0000","last_unstale":"2024-03-06T07:21:54.368609+0000","last_undegraded":"2024-03-06T07:21:54.368609+0000","last_fullsized":"2024-03-06T07:21:54.368609+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:32.706894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:42.311790+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:42.311500+0000","last_became_peered":"2024-03-06T07:21:42.311500+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358738+0000","last_change":"2024-03-06T07:21:54.358738+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358738+0000","last_undegraded":"2024-03-06T07:21:54.358738+0000","last_fullsized":"2024-03-06T07:21:54.358738+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374655+0000","last_change":"2024-03-06T07:21:54.369593+0000","last_active":"2024-03-06T07:21:54.374655+0000","last_peered":"2024-03-06T07:21:54.374655+0000","last_clean":"2024-03-06T07:21:54.374655+0000","last_became_active":"2024-03-06T07:21:54.369336+0000","last_became_peered":"2024-03-06T07:21:54.369336+0000","last_unstale":"2024-03-06T07:21:54.374655+0000","last_undegraded":"2024-03-06T07:21:54.374655+0000","last_fullsized":"2024-03-06T07:21:54.374655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359566+0000","last_change":"2024-03-06T07:21:43.319244+0000","last_active":"2024-03-06T07:21:54.359566+0000","last_peered":"2024-03-06T07:21:54.359566+0000","last_clean":"2024-03-06T07:21:54.359566+0000","last_became_active":"2024-03-06T07:21:43.318994+0000","last_became_peered":"2024-03-06T07:21:43.318994+0000","last_unstale":"2024-03-06T07:21:54.359566+0000","last_undegraded":"2024-03-06T07:21:54.359566+0000","last_fullsized":"2024-03-06T07:21:54.359566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369178+0000","last_change":"2024-03-06T07:21:42.315718+0000","last_active":"2024-03-06T07:21:54.369178+0000","last_peered":"2024-03-06T07:21:54.369178+0000","last_clean":"2024-03-06T07:21:54.369178+0000","last_became_active":"2024-03-06T07:21:42.315402+0000","last_became_peered":"2024-03-06T07:21:42.315402+0000","last_unstale":"2024-03-06T07:21:54.369178+0000","last_undegraded":"2024-03-06T07:21:54.369178+0000","last_fullsized":"2024-03-06T07:21:54.369178+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365483+0000","last_change":"2024-03-06T07:21:54.365483+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365483+0000","last_undegraded":"2024-03-06T07:21:54.365483+0000","last_fullsized":"2024-03-06T07:21:54.365483+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379803+0000","last_change":"2024-03-06T07:21:54.375429+0000","last_active":"2024-03-06T07:21:54.379803+0000","last_peered":"2024-03-06T07:21:54.379803+0000","last_clean":"2024-03-06T07:21:54.379803+0000","last_became_active":"2024-03-06T07:21:54.375117+0000","last_became_peered":"2024-03-06T07:21:54.375117+0000","last_unstale":"2024-03-06T07:21:54.379803+0000","last_undegraded":"2024-03-06T07:21:54.379803+0000","last_fullsized":"2024-03-06T07:21:54.379803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369090+0000","last_change":"2024-03-06T07:21:43.323135+0000","last_active":"2024-03-06T07:21:54.369090+0000","last_peered":"2024-03-06T07:21:54.369090+0000","last_clean":"2024-03-06T07:21:54.369090+0000","last_became_active":"2024-03-06T07:21:43.322814+0000","last_became_peered":"2024-03-06T07:21:43.322814+0000","last_unstale":"2024-03-06T07:21:54.369090+0000","last_undegraded":"2024-03-06T07:21:54.369090+0000","last_fullsized":"2024-03-06T07:21:54.369090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:24.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359662+0000","last_change":"2024-03-06T07:21:42.308451+0000","last_active":"2024-03-06T07:21:54.359662+0000","last_peered":"2024-03-06T07:21:54.359662+0000","last_clean":"2024-03-06T07:21:54.359662+0000","last_became_active":"2024-03-06T07:21:42.308167+0000","last_became_peered":"2024-03-06T07:21:42.308167+0000","last_unstale":"2024-03-06T07:21:54.359662+0000","last_undegraded":"2024-03-06T07:21:54.359662+0000","last_fullsized":"2024-03-06T07:21:54.359662+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:31.703072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362102+0000","last_change":"2024-03-06T07:21:54.362102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362102+0000","last_undegraded":"2024-03-06T07:21:54.362102+0000","last_fullsized":"2024-03-06T07:21:54.362102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376688+0000","last_change":"2024-03-06T07:21:54.375869+0000","last_active":"2024-03-06T07:21:54.376688+0000","last_peered":"2024-03-06T07:21:54.376688+0000","last_clean":"2024-03-06T07:21:54.376688+0000","last_became_active":"2024-03-06T07:21:54.375586+0000","last_became_peered":"2024-03-06T07:21:54.375586+0000","last_unstale":"2024-03-06T07:21:54.376688+0000","last_undegraded":"2024-03-06T07:21:54.376688+0000","last_fullsized":"2024-03-06T07:21:54.376688+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:02.230230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356824+0000","last_change":"2024-03-06T07:21:43.324622+0000","last_active":"2024-03-06T07:21:54.356824+0000","last_peered":"2024-03-06T07:21:54.356824+0000","last_clean":"2024-03-06T07:21:54.356824+0000","last_became_active":"2024-03-06T07:21:43.324350+0000","last_became_peered":"2024-03-06T07:21:43.324350+0000","last_unstale":"2024-03-06T07:21:54.356824+0000","last_undegraded":"2024-03-06T07:21:54.356824+0000","last_fullsized":"2024-03-06T07:21:54.356824+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359015+0000","last_change":"2024-03-06T07:21:42.309246+0000","last_active":"2024-03-06T07:21:54.359015+0000","last_peered":"2024-03-06T07:21:54.359015+0000","last_clean":"2024-03-06T07:21:54.359015+0000","last_became_active":"2024-03-06T07:21:42.308990+0000","last_became_peered":"2024-03-06T07:21:42.308990+0000","last_unstale":"2024-03-06T07:21:54.359015+0000","last_undegraded":"2024-03-06T07:21:54.359015+0000","last_fullsized":"2024-03-06T07:21:54.359015+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:51.765639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358506+0000","last_change":"2024-03-06T07:21:54.358506+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358506+0000","last_undegraded":"2024-03-06T07:21:54.358506+0000","last_fullsized":"2024-03-06T07:21:54.358506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377281+0000","last_change":"2024-03-06T07:21:54.371835+0000","last_active":"2024-03-06T07:21:54.377281+0000","last_peered":"2024-03-06T07:21:54.377281+0000","last_clean":"2024-03-06T07:21:54.377281+0000","last_became_active":"2024-03-06T07:21:54.371518+0000","last_became_peered":"2024-03-06T07:21:54.371518+0000","last_unstale":"2024-03-06T07:21:54.377281+0000","last_undegraded":"2024-03-06T07:21:54.377281+0000","last_fullsized":"2024-03-06T07:21:54.377281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369611+0000","last_change":"2024-03-06T07:21:43.329441+0000","last_active":"2024-03-06T07:21:54.369611+0000","last_peered":"2024-03-06T07:21:54.369611+0000","last_clean":"2024-03-06T07:21:54.369611+0000","last_became_active":"2024-03-06T07:21:43.327694+0000","last_became_peered":"2024-03-06T07:21:43.327694+0000","last_unstale":"2024-03-06T07:21:54.369611+0000","last_undegraded":"2024-03-06T07:21:54.369611+0000","last_fullsized":"2024-03-06T07:21:54.369611+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369282+0000","last_change":"2024-03-06T07:21:42.309614+0000","last_active":"2024-03-06T07:21:54.369282+0000","last_peered":"2024-03-06T07:21:54.369282+0000","last_clean":"2024-03-06T07:21:54.369282+0000","last_became_active":"2024-03-06T07:21:42.309319+0000","last_became_peered":"2024-03-06T07:21:42.309319+0000","last_unstale":"2024-03-06T07:21:54.369282+0000","last_undegraded":"2024-03-06T07:21:54.369282+0000","last_fullsized":"2024-03-06T07:21:54.369282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:28.385954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360823+0000","last_change":"2024-03-06T07:21:54.360823+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360823+0000","last_undegraded":"2024-03-06T07:21:54.360823+0000","last_fullsized":"2024-03-06T07:21:54.360823+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379806+0000","last_change":"2024-03-06T07:21:54.375555+0000","last_active":"2024-03-06T07:21:54.379806+0000","last_peered":"2024-03-06T07:21:54.379806+0000","last_clean":"2024-03-06T07:21:54.379806+0000","last_became_active":"2024-03-06T07:21:54.375251+0000","last_became_peered":"2024-03-06T07:21:54.375251+0000","last_unstale":"2024-03-06T07:21:54.379806+0000","last_undegraded":"2024-03-06T07:21:54.379806+0000","last_fullsized":"2024-03-06T07:21:54.379806+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:01.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359852+0000","last_change":"2024-03-06T07:21:43.321711+0000","last_active":"2024-03-06T07:21:54.359852+0000","last_peered":"2024-03-06T07:21:54.359852+0000","last_clean":"2024-03-06T07:21:54.359852+0000","last_became_active":"2024-03-06T07:21:43.321434+0000","last_became_peered":"2024-03-06T07:21:43.321434+0000","last_unstale":"2024-03-06T07:21:54.359852+0000","last_undegraded":"2024-03-06T07:21:54.359852+0000","last_fullsized":"2024-03-06T07:21:54.359852+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:06.150276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360450+0000","last_change":"2024-03-06T07:21:42.310672+0000","last_active":"2024-03-06T07:21:54.360450+0000","last_peered":"2024-03-06T07:21:54.360450+0000","last_clean":"2024-03-06T07:21:54.360450+0000","last_became_active":"2024-03-06T07:21:42.310366+0000","last_became_peered":"2024-03-06T07:21:42.310366+0000","last_unstale":"2024-03-06T07:21:54.360450+0000","last_undegraded":"2024-03-06T07:21:54.360450+0000","last_fullsized":"2024-03-06T07:21:54.360450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:24.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361405+0000","last_change":"2024-03-06T07:21:54.361405+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361405+0000","last_undegraded":"2024-03-06T07:21:54.361405+0000","last_fullsized":"2024-03-06T07:21:54.361405+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367559+0000","last_change":"2024-03-06T07:21:54.366502+0000","last_active":"2024-03-06T07:21:54.367559+0000","last_peered":"2024-03-06T07:21:54.367559+0000","last_clean":"2024-03-06T07:21:54.367559+0000","last_became_active":"2024-03-06T07:21:54.366066+0000","last_became_peered":"2024-03-06T07:21:54.366066+0000","last_unstale":"2024-03-06T07:21:54.367559+0000","last_undegraded":"2024-03-06T07:21:54.367559+0000","last_fullsized":"2024-03-06T07:21:54.367559+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364470+0000","last_change":"2024-03-06T07:21:43.319233+0000","last_active":"2024-03-06T07:21:54.364470+0000","last_peered":"2024-03-06T07:21:54.364470+0000","last_clean":"2024-03-06T07:21:54.364470+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.364470+0000","last_undegraded":"2024-03-06T07:21:54.364470+0000","last_fullsized":"2024-03-06T07:21:54.364470+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369843+0000","last_change":"2024-03-06T07:21:42.322036+0000","last_active":"2024-03-06T07:21:54.369843+0000","last_peered":"2024-03-06T07:21:54.369843+0000","last_clean":"2024-03-06T07:21:54.369843+0000","last_became_active":"2024-03-06T07:21:42.321385+0000","last_became_peered":"2024-03-06T07:21:42.321385+0000","last_unstale":"2024-03-06T07:21:54.369843+0000","last_undegraded":"2024-03-06T07:21:54.369843+0000","last_fullsized":"2024-03-06T07:21:54.369843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356636+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:54.356636+0000","last_peered":"2024-03-06T07:21:54.356636+0000","last_clean":"2024-03-06T07:21:54.356636+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:54.356636+0000","last_undegraded":"2024-03-06T07:21:54.356636+0000","last_fullsized":"2024-03-06T07:21:54.356636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":881653,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":651264,"data_stored":291813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":291161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":284171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":292795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":643072,"data_stored":285201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":291321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":874013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:00.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359676+0000', 'last_change': '2024-03-06T07:21:54.359676+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359676+0000', 'last_undegraded': '2024-03-06T07:21:54.359676+0000', 'last_fullsized': '2024-03-06T07:21:54.359676+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.364197+0000', 'last_change': '2024-03-06T07:21:54.364197+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.364197+0000', 'last_undegraded': '2024-03-06T07:21:54.364197+0000', 'last_fullsized': '2024-03-06T07:21:54.364197+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.368156+0000', 'last_change': '2024-03-06T07:21:54.368156+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.368156+0000', 'last_undegraded': '2024-03-06T07:21:54.368156+0000', 'last_fullsized': '2024-03-06T07:21:54.368156+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.365661+0000', 'last_change': '2024-03-06T07:21:54.365661+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.365661+0000', 'last_undegraded': '2024-03-06T07:21:54.365661+0000', 'last_fullsized': '2024-03-06T07:21:54.365661+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-03-06T07:22:00.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361397+0000', 'last_change': '2024-03-06T07:21:54.361397+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361397+0000', 'last_undegraded': '2024-03-06T07:21:54.361397+0000', 'last_fullsized': '2024-03-06T07:21:54.361397+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359185+0000', 'last_change': '2024-03-06T07:21:54.359185+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359185+0000', 'last_undegraded': '2024-03-06T07:21:54.359185+0000', 'last_fullsized': '2024-03-06T07:21:54.359185+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361827+0000', 'last_change': '2024-03-06T07:21:54.361827+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361827+0000', 'last_undegraded': '2024-03-06T07:21:54.361827+0000', 'last_fullsized': '2024-03-06T07:21:54.361827+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.360705+0000', 'last_change': '2024-03-06T07:21:54.360705+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.360705+0000', 'last_undegraded': '2024-03-06T07:21:54.360705+0000', 'last_fullsized': '2024-03-06T07:21:54.360705+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.365629+0000', 'last_change': '2024-03-06T07:21:54.365629+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.365629+0000', 'last_undegraded': '2024-03-06T07:21:54.365629+0000', 'last_fullsized': '2024-03-06T07:21:54.365629+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-03-06T07:22:00.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358522+0000', 'last_change': '2024-03-06T07:21:54.358522+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358522+0000', 'last_undegraded': '2024-03-06T07:21:54.358522+0000', 'last_fullsized': '2024-03-06T07:21:54.358522+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359265+0000', 'last_change': '2024-03-06T07:21:54.359265+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359265+0000', 'last_undegraded': '2024-03-06T07:21:54.359265+0000', 'last_fullsized': '2024-03-06T07:21:54.359265+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359570+0000', 'last_change': '2024-03-06T07:21:54.359570+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359570+0000', 'last_undegraded': '2024-03-06T07:21:54.359570+0000', 'last_fullsized': '2024-03-06T07:21:54.359570+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359648+0000', 'last_change': '2024-03-06T07:21:54.359648+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359648+0000', 'last_undegraded': '2024-03-06T07:21:54.359648+0000', 'last_fullsized': '2024-03-06T07:21:54.359648+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362294+0000', 'last_change': '2024-03-06T07:21:54.362294+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362294+0000', 'last_undegraded': '2024-03-06T07:21:54.362294+0000', 'last_fullsized': '2024-03-06T07:21:54.362294+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-03-06T07:22:00.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359359+0000', 'last_change': '2024-03-06T07:21:54.359359+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359359+0000', 'last_undegraded': '2024-03-06T07:21:54.359359+0000', 'last_fullsized': '2024-03-06T07:21:54.359359+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359065+0000', 'last_change': '2024-03-06T07:21:54.359065+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359065+0000', 'last_undegraded': '2024-03-06T07:21:54.359065+0000', 'last_fullsized': '2024-03-06T07:21:54.359065+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.360235+0000', 'last_change': '2024-03-06T07:21:54.360235+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.360235+0000', 'last_undegraded': '2024-03-06T07:21:54.360235+0000', 'last_fullsized': '2024-03-06T07:21:54.360235+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359045+0000', 'last_change': '2024-03-06T07:21:54.359045+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359045+0000', 'last_undegraded': '2024-03-06T07:21:54.359045+0000', 'last_fullsized': '2024-03-06T07:21:54.359045+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.367783+0000', 'last_change': '2024-03-06T07:21:54.367783+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.367783+0000', 'last_undegraded': '2024-03-06T07:21:54.367783+0000', 'last_fullsized': '2024-03-06T07:21:54.367783+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-03-06T07:22:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.367102+0000', 'last_change': '2024-03-06T07:21:54.367102+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.367102+0000', 'last_undegraded': '2024-03-06T07:21:54.367102+0000', 'last_fullsized': '2024-03-06T07:21:54.367102+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358875+0000', 'last_change': '2024-03-06T07:21:54.358875+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358875+0000', 'last_undegraded': '2024-03-06T07:21:54.358875+0000', 'last_fullsized': '2024-03-06T07:21:54.358875+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.359127+0000', 'last_change': '2024-03-06T07:21:54.359127+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.359127+0000', 'last_undegraded': '2024-03-06T07:21:54.359127+0000', 'last_fullsized': '2024-03-06T07:21:54.359127+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.357760+0000', 'last_change': '2024-03-06T07:21:54.357760+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.357760+0000', 'last_undegraded': '2024-03-06T07:21:54.357760+0000', 'last_fullsized': '2024-03-06T07:21:54.357760+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358331+0000', 'last_change': '2024-03-06T07:21:54.358331+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358331+0000', 'last_undegraded': '2024-03-06T07:21:54.358331+0000', 'last_fullsized': '2024-03-06T07:21:54.358331+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.366485+0000', 'last_change': '2024-03-06T07:21:54.366485+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.366485+0000', 'last_undegraded': '2024-03-06T07:21:54.366485+0000', 'last_fullsized': '2024-03-06T07:21:54.366485+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-03-06T07:22:00.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.369128+0000', 'last_change': '2024-03-06T07:21:54.369128+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.369128+0000', 'last_undegraded': '2024-03-06T07:21:54.369128+0000', 'last_fullsized': '2024-03-06T07:21:54.369128+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361035+0000', 'last_change': '2024-03-06T07:21:54.361035+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361035+0000', 'last_undegraded': '2024-03-06T07:21:54.361035+0000', 'last_fullsized': '2024-03-06T07:21:54.361035+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.363616+0000', 'last_change': '2024-03-06T07:21:54.363616+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.363616+0000', 'last_undegraded': '2024-03-06T07:21:54.363616+0000', 'last_fullsized': '2024-03-06T07:21:54.363616+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362601+0000', 'last_change': '2024-03-06T07:21:54.362601+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362601+0000', 'last_undegraded': '2024-03-06T07:21:54.362601+0000', 'last_fullsized': '2024-03-06T07:21:54.362601+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362773+0000', 'last_change': '2024-03-06T07:21:54.362773+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362773+0000', 'last_undegraded': '2024-03-06T07:21:54.362773+0000', 'last_fullsized': '2024-03-06T07:21:54.362773+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-03-06T07:22:00.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358521+0000', 'last_change': '2024-03-06T07:21:54.358521+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358521+0000', 'last_undegraded': '2024-03-06T07:21:54.358521+0000', 'last_fullsized': '2024-03-06T07:21:54.358521+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.357560+0000', 'last_change': '2024-03-06T07:21:54.357560+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.357560+0000', 'last_undegraded': '2024-03-06T07:21:54.357560+0000', 'last_fullsized': '2024-03-06T07:21:54.357560+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362616+0000', 'last_change': '2024-03-06T07:21:54.362616+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362616+0000', 'last_undegraded': '2024-03-06T07:21:54.362616+0000', 'last_fullsized': '2024-03-06T07:21:54.362616+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362416+0000', 'last_change': '2024-03-06T07:21:54.362416+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362416+0000', 'last_undegraded': '2024-03-06T07:21:54.362416+0000', 'last_fullsized': '2024-03-06T07:21:54.362416+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362959+0000', 'last_change': '2024-03-06T07:21:54.362959+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362959+0000', 'last_undegraded': '2024-03-06T07:21:54.362959+0000', 'last_fullsized': '2024-03-06T07:21:54.362959+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-03-06T07:22:00.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.357254+0000', 'last_change': '2024-03-06T07:21:54.357254+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.357254+0000', 'last_undegraded': '2024-03-06T07:21:54.357254+0000', 'last_fullsized': '2024-03-06T07:21:54.357254+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.360237+0000', 'last_change': '2024-03-06T07:21:54.360237+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.360237+0000', 'last_undegraded': '2024-03-06T07:21:54.360237+0000', 'last_fullsized': '2024-03-06T07:21:54.360237+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.364680+0000', 'last_change': '2024-03-06T07:21:54.364680+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.364680+0000', 'last_undegraded': '2024-03-06T07:21:54.364680+0000', 'last_fullsized': '2024-03-06T07:21:54.364680+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.366515+0000', 'last_change': '2024-03-06T07:21:54.366515+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.366515+0000', 'last_undegraded': '2024-03-06T07:21:54.366515+0000', 'last_fullsized': '2024-03-06T07:21:54.366515+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358658+0000', 'last_change': '2024-03-06T07:21:54.358658+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358658+0000', 'last_undegraded': '2024-03-06T07:21:54.358658+0000', 'last_fullsized': '2024-03-06T07:21:54.358658+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.360176+0000', 'last_change': '2024-03-06T07:21:54.360176+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.360176+0000', 'last_undegraded': '2024-03-06T07:21:54.360176+0000', 'last_fullsized': '2024-03-06T07:21:54.360176+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358914+0000', 'last_change': '2024-03-06T07:21:54.358914+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358914+0000', 'last_undegraded': '2024-03-06T07:21:54.358914+0000', 'last_fullsized': '2024-03-06T07:21:54.358914+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362688+0000', 'last_change': '2024-03-06T07:21:54.362688+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362688+0000', 'last_undegraded': '2024-03-06T07:21:54.362688+0000', 'last_fullsized': '2024-03-06T07:21:54.362688+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.363414+0000', 'last_change': '2024-03-06T07:21:54.363414+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.363414+0000', 'last_undegraded': '2024-03-06T07:21:54.363414+0000', 'last_fullsized': '2024-03-06T07:21:54.363414+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361630+0000', 'last_change': '2024-03-06T07:21:54.361630+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361630+0000', 'last_undegraded': '2024-03-06T07:21:54.361630+0000', 'last_fullsized': '2024-03-06T07:21:54.361630+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358625+0000', 'last_change': '2024-03-06T07:21:54.358625+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358625+0000', 'last_undegraded': '2024-03-06T07:21:54.358625+0000', 'last_fullsized': '2024-03-06T07:21:54.358625+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-03-06T07:22:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.360511+0000', 'last_change': '2024-03-06T07:21:54.360511+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.360511+0000', 'last_undegraded': '2024-03-06T07:21:54.360511+0000', 'last_fullsized': '2024-03-06T07:21:54.360511+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.366074+0000', 'last_change': '2024-03-06T07:21:54.366074+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.366074+0000', 'last_undegraded': '2024-03-06T07:21:54.366074+0000', 'last_fullsized': '2024-03-06T07:21:54.366074+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.364356+0000', 'last_change': '2024-03-06T07:21:54.364356+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.364356+0000', 'last_undegraded': '2024-03-06T07:21:54.364356+0000', 'last_fullsized': '2024-03-06T07:21:54.364356+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361900+0000', 'last_change': '2024-03-06T07:21:54.361900+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361900+0000', 'last_undegraded': '2024-03-06T07:21:54.361900+0000', 'last_fullsized': '2024-03-06T07:21:54.361900+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.360981+0000', 'last_change': '2024-03-06T07:21:54.360981+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.360981+0000', 'last_undegraded': '2024-03-06T07:21:54.360981+0000', 'last_fullsized': '2024-03-06T07:21:54.360981+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.364238+0000', 'last_change': '2024-03-06T07:21:54.364238+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.364238+0000', 'last_undegraded': '2024-03-06T07:21:54.364238+0000', 'last_fullsized': '2024-03-06T07:21:54.364238+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.363210+0000', 'last_change': '2024-03-06T07:21:54.363210+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.363210+0000', 'last_undegraded': '2024-03-06T07:21:54.363210+0000', 'last_fullsized': '2024-03-06T07:21:54.363210+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361638+0000', 'last_change': '2024-03-06T07:21:54.361638+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361638+0000', 'last_undegraded': '2024-03-06T07:21:54.361638+0000', 'last_fullsized': '2024-03-06T07:21:54.361638+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361080+0000', 'last_change': '2024-03-06T07:21:54.361080+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361080+0000', 'last_undegraded': '2024-03-06T07:21:54.361080+0000', 'last_fullsized': '2024-03-06T07:21:54.361080+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361933+0000', 'last_change': '2024-03-06T07:21:54.361933+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361933+0000', 'last_undegraded': '2024-03-06T07:21:54.361933+0000', 'last_fullsized': '2024-03-06T07:21:54.361933+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362069+0000', 'last_change': '2024-03-06T07:21:54.362069+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362069+0000', 'last_undegraded': '2024-03-06T07:21:54.362069+0000', 'last_fullsized': '2024-03-06T07:21:54.362069+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-03-06T07:22:00.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361491+0000', 'last_change': '2024-03-06T07:21:54.361491+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361491+0000', 'last_undegraded': '2024-03-06T07:21:54.361491+0000', 'last_fullsized': '2024-03-06T07:21:54.361491+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358738+0000', 'last_change': '2024-03-06T07:21:54.358738+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358738+0000', 'last_undegraded': '2024-03-06T07:21:54.358738+0000', 'last_fullsized': '2024-03-06T07:21:54.358738+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.365483+0000', 'last_change': '2024-03-06T07:21:54.365483+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.365483+0000', 'last_undegraded': '2024-03-06T07:21:54.365483+0000', 'last_fullsized': '2024-03-06T07:21:54.365483+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.362102+0000', 'last_change': '2024-03-06T07:21:54.362102+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.362102+0000', 'last_undegraded': '2024-03-06T07:21:54.362102+0000', 'last_fullsized': '2024-03-06T07:21:54.362102+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.358506+0000', 'last_change': '2024-03-06T07:21:54.358506+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.358506+0000', 'last_undegraded': '2024-03-06T07:21:54.358506+0000', 'last_fullsized': '2024-03-06T07:21:54.358506+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-03-06T07:22:00.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.360823+0000', 'last_change': '2024-03-06T07:21:54.360823+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.360823+0000', 'last_undegraded': '2024-03-06T07:21:54.360823+0000', 'last_fullsized': '2024-03-06T07:21:54.360823+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-03-06T07:22:00.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:54.361405+0000', 'last_change': '2024-03-06T07:21:54.361405+0000', 'last_active': '2024-03-06T07:21:54.349222+0000', 'last_peered': '2024-03-06T07:21:54.349222+0000', 'last_clean': '2024-03-06T07:21:54.349222+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:54.361405+0000', 'last_undegraded': '2024-03-06T07:21:54.361405+0000', 'last_fullsized': '2024-03-06T07:21:54.361405+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:54.349222+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:00.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:22:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.280+0000 7f2fa3447640 1 Processor -- start 2024-03-06T07:22:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa3447640 1 -- start start 2024-03-06T07:22:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa3447640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c1506b0 0x7f2f9c153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa3447640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f9c156ea0 0x7f2f9c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa3447640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9c070800 con 0x7f2f9c0acc00 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa3447640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9c157320 con 0x7f2f9c156ea0 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa3447640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c156450 con 0x7f2f9c1506b0 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa11e3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c1506b0 0x7f2f9c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.281+0000 7f2fa09e2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f9c156ea0 0x7f2f9c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.282+0000 7f2fa11e3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c1506b0 0x7f2f9c153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:48604/0 (socket says 172.21.15.26:48604) 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.282+0000 7f2fa11e3640 1 -- 172.21.15.26:0/2895372528 learned_addr learned my addr 172.21.15.26:0/2895372528 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.282+0000 7f2fa09e2640 1 -- 172.21.15.26:0/2895372528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c1506b0 msgr2=0x7f2f9c153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.282+0000 7f2fa09e2640 1 --2- 172.21.15.26:0/2895372528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c1506b0 0x7f2f9c153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.282+0000 7f2fa09e2640 1 -- 172.21.15.26:0/2895372528 >> v1:172.21.15.117:6789/0 conn(0x7f2f9c0acc00 legacy=0x7f2f9c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.282+0000 7f2fa09e2640 1 -- 172.21.15.26:0/2895372528 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c070aa0 con 0x7f2f9c156ea0 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2fa09e2640 1 --2- 172.21.15.26:0/2895372528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c156ea0 0x7f2f9c0702c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00dbc0 tx=0x7f2f8c02fcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9ba19bb2afeb1e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2f93fff640 1 -- 172.21.15.26:0/2895372528 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c030860 con 0x7f2f9c156ea0 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2f93fff640 1 -- 172.21.15.26:0/2895372528 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c030a00 con 0x7f2f9c156ea0 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2f93fff640 1 -- 172.21.15.26:0/2895372528 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c030dd0 con 0x7f2f9c156ea0 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2fa3447640 1 -- 172.21.15.26:0/2895372528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c156ea0 msgr2=0x7f2f9c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2895372528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c156ea0 0x7f2f9c0702c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00dbc0 tx=0x7f2f8c02fcc0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2fa3447640 1 -- 172.21.15.26:0/2895372528 shutdown_connections 2024-03-06T07:22:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2895372528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c156ea0 0x7f2f9c0702c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2895372528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c1506b0 0x7f2f9c153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.283+0000 7f2fa3447640 1 -- 172.21.15.26:0/2895372528 >> 172.21.15.26:0/2895372528 conn(0x7f2f9c06b1a0 msgr2=0x7f2f9c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.284+0000 7f2fa3447640 1 -- 172.21.15.26:0/2895372528 shutdown_connections 2024-03-06T07:22:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.284+0000 7f2fa3447640 1 -- 172.21.15.26:0/2895372528 wait complete. 2024-03-06T07:22:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 Processor -- start 2024-03-06T07:22:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 -- start start 2024-03-06T07:22:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c0acc00 0x7f2f9c1ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f9c1506b0 0x7f2f9c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c156ea0 0x7f2f9c153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c1566a0 con 0x7f2f9c1506b0 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c156a10 con 0x7f2f9c0acc00 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa3447640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0c5bb0 con 0x7f2f9c156ea0 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.285+0000 7f2fa09e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f9c1506b0 0x7f2f9c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f9c1506b0 0x7f2f9c06d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57898/0 (socket says 172.21.15.26:57898) 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 -- 172.21.15.26:0/2053239407 learned_addr learned my addr 172.21.15.26:0/2053239407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa19e4640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c156ea0 0x7f2f9c153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa11e3640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c0acc00 0x7f2f9c1ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 -- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c156ea0 msgr2=0x7f2f9c153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c156ea0 0x7f2f9c153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 -- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c0acc00 msgr2=0x7f2f9c1ea0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c0acc00 0x7f2f9c1ea0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 -- 172.21.15.26:0/2053239407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c0c47d0 con 0x7f2f9c1506b0 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa11e3640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c0acc00 0x7f2f9c1ea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa09e2640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f9c1506b0 0x7f2f9c06d4c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c03a040 tx=0x7f2f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2f91ffb640 1 -- 172.21.15.26:0/2053239407 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c048020 con 0x7f2f9c1506b0 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2f91ffb640 1 -- 172.21.15.26:0/2053239407 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c00bc10 con 0x7f2f9c1506b0 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9c0c4ab0 con 0x7f2f9c1506b0 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2f91ffb640 1 -- 172.21.15.26:0/2053239407 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c006e10 con 0x7f2f9c1506b0 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.286+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c06da00 con 0x7f2f9c1506b0 2024-03-06T07:22:00.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.287+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9c154a50 con 0x7f2f9c1506b0 2024-03-06T07:22:00.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.288+0000 7f2f91ffb640 1 -- 172.21.15.26:0/2053239407 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f8c04b020 con 0x7f2f9c1506b0 2024-03-06T07:22:00.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.289+0000 7f2f91ffb640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f8006b500 0x7f2f8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.289+0000 7f2f91ffb640 1 -- 172.21.15.26:0/2053239407 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f2f8c0b0dc0 con 0x7f2f9c1506b0 2024-03-06T07:22:00.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.289+0000 7f2fa11e3640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f8006b500 0x7f2f8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.289+0000 7f2fa11e3640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f8006b500 0x7f2f8006d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009fc0 tx=0x7f2f98009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.297+0000 7f2f91ffb640 1 -- 172.21.15.26:0/2053239407 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f8c052030 con 0x7f2f9c1506b0 2024-03-06T07:22:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.548+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2f9c04f330 con 0x7f2f9c1506b0 2024-03-06T07:22:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.548+0000 7f2f91ffb640 1 -- 172.21.15.26:0/2053239407 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1557 (secure 0 0 0) 0x7f2f8c07f030 con 0x7f2f9c1506b0 2024-03-06T07:22:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":135,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709595,"num_in_osds":8,"osd_in_since":1709709586,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129},{"state_name":"creating+peering","count":64}],"num_pgs":193,"num_pools":5,"num_objects":24,"data_bytes":592778,"bytes_used":243707904,"bytes_avail":772850405376,"bytes_total":773094113280,"inactive_pgs_ratio":0.33160620927810669},"fsmap":{"epoch":34,"id":3,"up":1,"in":1,"max":1,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":5617},{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5697}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-03-06T07:21:31.047700+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:22:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f8006b500 msgr2=0x7f2f8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f8006b500 0x7f2f8006d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009fc0 tx=0x7f2f98009240 comp rx=0 tx=0).stop 2024-03-06T07:22:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f9c1506b0 msgr2=0x7f2f9c06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f9c1506b0 0x7f2f9c06d4c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c03a040 tx=0x7f2f8c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 shutdown_connections 2024-03-06T07:22:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f8006b500 0x7f2f8006d9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f9c156ea0 0x7f2f9c153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f9c1506b0 0x7f2f9c06d4c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 --2- 172.21.15.26:0/2053239407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f9c0acc00 0x7f2f9c1ea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 >> 172.21.15.26:0/2053239407 conn(0x7f2f9c06b1a0 msgr2=0x7f2f9c14f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 shutdown_connections 2024-03-06T07:22:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.550+0000 7f2fa3447640 1 -- 172.21.15.26:0/2053239407 wait complete. 2024-03-06T07:22:00.562 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:22:00.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.662+0000 7f9e16c2a640 1 Processor -- start 2024-03-06T07:22:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.662+0000 7f9e16c2a640 1 -- start start 2024-03-06T07:22:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e16c2a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e100c5480 0x7f9e10155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e16c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e10158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e16c2a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e100c5cc0 con 0x7f9e100c5860 2024-03-06T07:22:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e16c2a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1006df40 con 0x7f9e100c5480 2024-03-06T07:22:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e16c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1006fdb0 con 0x7f9e101586f0 2024-03-06T07:22:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e149c6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e100c5480 0x7f9e10155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e0ffff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9e100c5860 0x7f9e1006f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:43890/0 (socket says 172.21.15.26:43890) 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e151c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e10158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e0ffff640 1 -- 172.21.15.26:0/357509339 learned_addr learned my addr 172.21.15.26:0/357509339 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e151c7640 1 -- 172.21.15.26:0/357509339 >> v2:172.21.15.117:3300/0 conn(0x7f9e100c5480 msgr2=0x7f9e10155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e151c7640 1 --2- 172.21.15.26:0/357509339 >> v2:172.21.15.117:3300/0 conn(0x7f9e100c5480 0x7f9e10155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e151c7640 1 -- 172.21.15.26:0/357509339 >> v1:172.21.15.117:6789/0 conn(0x7f9e100c5860 legacy=0x7f9e1006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e0f7fe640 1 -- 172.21.15.26:0/357509339 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906743934 0 0) 0x7f9e100c5cc0 con 0x7f9e100c5860 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e151c7640 1 -- 172.21.15.26:0/357509339 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e101591c0 con 0x7f9e101586f0 2024-03-06T07:22:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.663+0000 7f9e149c6640 1 --2- 172.21.15.26:0/357509339 >> v2:172.21.15.117:3300/0 conn(0x7f9e100c5480 0x7f9e10155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.664+0000 7f9e151c7640 1 --2- 172.21.15.26:0/357509339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e10158b80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097a0 tx=0x7f9e0002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260fd05918306f11 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.664+0000 7f9e0f7fe640 1 -- 172.21.15.26:0/357509339 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e000315a0 con 0x7f9e101586f0 2024-03-06T07:22:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.664+0000 7f9e0f7fe640 1 -- 172.21.15.26:0/357509339 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e00037070 con 0x7f9e101586f0 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.664+0000 7f9e0f7fe640 1 -- 172.21.15.26:0/357509339 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00031960 con 0x7f9e101586f0 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 -- 172.21.15.26:0/357509339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 msgr2=0x7f9e10158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/357509339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e10158b80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097a0 tx=0x7f9e0002b840 comp rx=0 tx=0).stop 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 -- 172.21.15.26:0/357509339 shutdown_connections 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/357509339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e10158b80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/357509339 >> v2:172.21.15.117:3300/0 conn(0x7f9e100c5480 0x7f9e10155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 -- 172.21.15.26:0/357509339 >> 172.21.15.26:0/357509339 conn(0x7f9e1006b1a0 msgr2=0x7f9e1006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 -- 172.21.15.26:0/357509339 shutdown_connections 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.665+0000 7f9e16c2a640 1 -- 172.21.15.26:0/357509339 wait complete. 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.666+0000 7f9e16c2a640 1 Processor -- start 2024-03-06T07:22:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.666+0000 7f9e16c2a640 1 -- start start 2024-03-06T07:22:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e16c2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e100c5480 0x7f9e100c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e16c2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e100c5860 0x7f9e100701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e16c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e100ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e16c2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10155680 con 0x7f9e100c5860 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e16c2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e101553a0 con 0x7f9e100c5480 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e16c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10152a20 con 0x7f9e101586f0 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e100c5480 0x7f9e100c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e0ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e100c5860 0x7f9e100701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e151c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e100ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e100c5480 0x7f9e100c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:46896/0 (socket says 172.21.15.26:46896) 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 -- 172.21.15.26:0/4213591857 learned_addr learned my addr 172.21.15.26:0/4213591857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 -- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 msgr2=0x7f9e100ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e100ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 -- 172.21.15.26:0/4213591857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e100c5860 msgr2=0x7f9e100701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e100c5860 0x7f9e100701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e149c6640 1 -- 172.21.15.26:0/4213591857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100aa810 con 0x7f9e100c5480 2024-03-06T07:22:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.667+0000 7f9e0ffff640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e100c5860 0x7f9e100701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.668+0000 7f9e149c6640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e100c5480 0x7f9e100c4550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800efc0 tx=0x7f9e0800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.668+0000 7f9e0d7fa640 1 -- 172.21.15.26:0/4213591857 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08018020 con 0x7f9e100c5480 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.668+0000 7f9e0d7fa640 1 -- 172.21.15.26:0/4213591857 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e08014680 con 0x7f9e100c5480 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.668+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e100aaaf0 con 0x7f9e100c5480 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.668+0000 7f9e0d7fa640 1 -- 172.21.15.26:0/4213591857 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08014a50 con 0x7f9e100c5480 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.668+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e100c3d50 con 0x7f9e100c5480 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.669+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1004f330 con 0x7f9e100c5480 2024-03-06T07:22:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.669+0000 7f9e0d7fa640 1 -- 172.21.15.26:0/4213591857 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e08005d10 con 0x7f9e100c5480 2024-03-06T07:22:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.669+0000 7f9e0d7fa640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9dfc06b4d0 0x7f9dfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.670+0000 7f9e0d7fa640 1 -- 172.21.15.26:0/4213591857 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f9e0808b370 con 0x7f9e100c5480 2024-03-06T07:22:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.670+0000 7f9e0ffff640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9dfc06b4d0 0x7f9dfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.670+0000 7f9e0ffff640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9dfc06b4d0 0x7f9dfc06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e100aa380 tx=0x7f9e04007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.674+0000 7f9e0d7fa640 1 -- 172.21.15.26:0/4213591857 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e08022030 con 0x7f9e100c5480 2024-03-06T07:22:00.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.852+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e100aa5c0 con 0x7f9dfc06b4d0 2024-03-06T07:22:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.863+0000 7f9e0d7fa640 1 -- 172.21.15.26:0/4213591857 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485262 (secure 0 0 0) 0x7f9e100aa5c0 con 0x7f9dfc06b4d0 2024-03-06T07:22:00.870 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:00.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9dfc06b4d0 msgr2=0x7f9dfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9dfc06b4d0 0x7f9dfc06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e100aa380 tx=0x7f9e04007d50 comp rx=0 tx=0).stop 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e100c5480 msgr2=0x7f9e100c4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e100c5480 0x7f9e100c4550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800efc0 tx=0x7f9e0800c490 comp rx=0 tx=0).stop 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 shutdown_connections 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9dfc06b4d0 0x7f9dfc06d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101586f0 0x7f9e100ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e100c5860 0x7f9e100701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 --2- 172.21.15.26:0/4213591857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e100c5480 0x7f9e100c4550 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 >> 172.21.15.26:0/4213591857 conn(0x7f9e1006b1a0 msgr2=0x7f9e1006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 shutdown_connections 2024-03-06T07:22:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:00.867+0000 7f9e16c2a640 1 -- 172.21.15.26:0/4213591857 wait complete. 2024-03-06T07:22:00.883 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-06T07:21:59.053948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237996,"kb_used_data":5180,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736724,"statfs":{"total":773094113280,"available":772850405376,"internally_reserved":0,"allocated":5304320,"data_stored":3492128,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703896"},"pg_stats":[{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359676+0000","last_change":"2024-03-06T07:21:54.359676+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359676+0000","last_undegraded":"2024-03-06T07:21:54.359676+0000","last_fullsized":"2024-03-06T07:21:54.359676+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372914+0000","last_change":"2024-03-06T07:21:43.317811+0000","last_active":"2024-03-06T07:21:54.372914+0000","last_peered":"2024-03-06T07:21:54.372914+0000","last_clean":"2024-03-06T07:21:54.372914+0000","last_became_active":"2024-03-06T07:21:43.317509+0000","last_became_peered":"2024-03-06T07:21:43.317509+0000","last_unstale":"2024-03-06T07:21:54.372914+0000","last_undegraded":"2024-03-06T07:21:54.372914+0000","last_fullsized":"2024-03-06T07:21:54.372914+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364197+0000","last_change":"2024-03-06T07:21:54.364197+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364197+0000","last_undegraded":"2024-03-06T07:21:54.364197+0000","last_fullsized":"2024-03-06T07:21:54.364197+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357765+0000","last_change":"2024-03-06T07:21:43.317801+0000","last_active":"2024-03-06T07:21:54.357765+0000","last_peered":"2024-03-06T07:21:54.357765+0000","last_clean":"2024-03-06T07:21:54.357765+0000","last_became_active":"2024-03-06T07:21:43.317522+0000","last_became_peered":"2024-03-06T07:21:43.317522+0000","last_unstale":"2024-03-06T07:21:54.357765+0000","last_undegraded":"2024-03-06T07:21:54.357765+0000","last_fullsized":"2024-03-06T07:21:54.357765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.368156+0000","last_change":"2024-03-06T07:21:54.368156+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.368156+0000","last_undegraded":"2024-03-06T07:21:54.368156+0000","last_fullsized":"2024-03-06T07:21:54.368156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369212+0000","last_change":"2024-03-06T07:21:43.328071+0000","last_active":"2024-03-06T07:21:54.369212+0000","last_peered":"2024-03-06T07:21:54.369212+0000","last_clean":"2024-03-06T07:21:54.369212+0000","last_became_active":"2024-03-06T07:21:43.327781+0000","last_became_peered":"2024-03-06T07:21:43.327781+0000","last_unstale":"2024-03-06T07:21:54.369212+0000","last_undegraded":"2024-03-06T07:21:54.369212+0000","last_fullsized":"2024-03-06T07:21:54.369212+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365661+0000","last_change":"2024-03-06T07:21:54.365661+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365661+0000","last_undegraded":"2024-03-06T07:21:54.365661+0000","last_fullsized":"2024-03-06T07:21:54.365661+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357083+0000","last_change":"2024-03-06T07:21:43.326286+0000","last_active":"2024-03-06T07:21:54.357083+0000","last_peered":"2024-03-06T07:21:54.357083+0000","last_clean":"2024-03-06T07:21:54.357083+0000","last_became_active":"2024-03-06T07:21:43.326031+0000","last_became_peered":"2024-03-06T07:21:43.326031+0000","last_unstale":"2024-03-06T07:21:54.357083+0000","last_undegraded":"2024-03-06T07:21:54.357083+0000","last_fullsized":"2024-03-06T07:21:54.357083+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:21.645487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361397+0000","last_change":"2024-03-06T07:21:54.361397+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361397+0000","last_undegraded":"2024-03-06T07:21:54.361397+0000","last_fullsized":"2024-03-06T07:21:54.361397+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369745+0000","last_change":"2024-03-06T07:21:43.321810+0000","last_active":"2024-03-06T07:21:54.369745+0000","last_peered":"2024-03-06T07:21:54.369745+0000","last_clean":"2024-03-06T07:21:54.369745+0000","last_became_active":"2024-03-06T07:21:43.321538+0000","last_became_peered":"2024-03-06T07:21:43.321538+0000","last_unstale":"2024-03-06T07:21:54.369745+0000","last_undegraded":"2024-03-06T07:21:54.369745+0000","last_fullsized":"2024-03-06T07:21:54.369745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:10.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359185+0000","last_change":"2024-03-06T07:21:54.359185+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359185+0000","last_undegraded":"2024-03-06T07:21:54.359185+0000","last_fullsized":"2024-03-06T07:21:54.359185+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368491+0000","last_change":"2024-03-06T07:21:43.318713+0000","last_active":"2024-03-06T07:21:54.368491+0000","last_peered":"2024-03-06T07:21:54.368491+0000","last_clean":"2024-03-06T07:21:54.368491+0000","last_became_active":"2024-03-06T07:21:43.318385+0000","last_became_peered":"2024-03-06T07:21:43.318385+0000","last_unstale":"2024-03-06T07:21:54.368491+0000","last_undegraded":"2024-03-06T07:21:54.368491+0000","last_fullsized":"2024-03-06T07:21:54.368491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361827+0000","last_change":"2024-03-06T07:21:54.361827+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361827+0000","last_undegraded":"2024-03-06T07:21:54.361827+0000","last_fullsized":"2024-03-06T07:21:54.361827+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368595+0000","last_change":"2024-03-06T07:21:43.327099+0000","last_active":"2024-03-06T07:21:54.368595+0000","last_peered":"2024-03-06T07:21:54.368595+0000","last_clean":"2024-03-06T07:21:54.368595+0000","last_became_active":"2024-03-06T07:21:43.326851+0000","last_became_peered":"2024-03-06T07:21:43.326851+0000","last_unstale":"2024-03-06T07:21:54.368595+0000","last_undegraded":"2024-03-06T07:21:54.368595+0000","last_fullsized":"2024-03-06T07:21:54.368595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360705+0000","last_change":"2024-03-06T07:21:54.360705+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360705+0000","last_undegraded":"2024-03-06T07:21:54.360705+0000","last_fullsized":"2024-03-06T07:21:54.360705+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358329+0000","last_change":"2024-03-06T07:21:43.325897+0000","last_active":"2024-03-06T07:21:54.358329+0000","last_peered":"2024-03-06T07:21:54.358329+0000","last_clean":"2024-03-06T07:21:54.358329+0000","last_became_active":"2024-03-06T07:21:43.325647+0000","last_became_peered":"2024-03-06T07:21:43.325647+0000","last_unstale":"2024-03-06T07:21:54.358329+0000","last_undegraded":"2024-03-06T07:21:54.358329+0000","last_fullsized":"2024-03-06T07:21:54.358329+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365629+0000","last_change":"2024-03-06T07:21:54.365629+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365629+0000","last_undegraded":"2024-03-06T07:21:54.365629+0000","last_fullsized":"2024-03-06T07:21:54.365629+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359703+0000","last_change":"2024-03-06T07:21:43.330337+0000","last_active":"2024-03-06T07:21:54.359703+0000","last_peered":"2024-03-06T07:21:54.359703+0000","last_clean":"2024-03-06T07:21:54.359703+0000","last_became_active":"2024-03-06T07:21:43.330053+0000","last_became_peered":"2024-03-06T07:21:43.330053+0000","last_unstale":"2024-03-06T07:21:54.359703+0000","last_undegraded":"2024-03-06T07:21:54.359703+0000","last_fullsized":"2024-03-06T07:21:54.359703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358522+0000","last_change":"2024-03-06T07:21:54.358522+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358522+0000","last_undegraded":"2024-03-06T07:21:54.358522+0000","last_fullsized":"2024-03-06T07:21:54.358522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370017+0000","last_change":"2024-03-06T07:21:43.325100+0000","last_active":"2024-03-06T07:21:54.370017+0000","last_peered":"2024-03-06T07:21:54.370017+0000","last_clean":"2024-03-06T07:21:54.370017+0000","last_became_active":"2024-03-06T07:21:43.324812+0000","last_became_peered":"2024-03-06T07:21:43.324812+0000","last_unstale":"2024-03-06T07:21:54.370017+0000","last_undegraded":"2024-03-06T07:21:54.370017+0000","last_fullsized":"2024-03-06T07:21:54.370017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:11.442368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359265+0000","last_change":"2024-03-06T07:21:54.359265+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359265+0000","last_undegraded":"2024-03-06T07:21:54.359265+0000","last_fullsized":"2024-03-06T07:21:54.359265+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366884+0000","last_change":"2024-03-06T07:21:43.322188+0000","last_active":"2024-03-06T07:21:54.366884+0000","last_peered":"2024-03-06T07:21:54.366884+0000","last_clean":"2024-03-06T07:21:54.366884+0000","last_became_active":"2024-03-06T07:21:43.321685+0000","last_became_peered":"2024-03-06T07:21:43.321685+0000","last_unstale":"2024-03-06T07:21:54.366884+0000","last_undegraded":"2024-03-06T07:21:54.366884+0000","last_fullsized":"2024-03-06T07:21:54.366884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:54.799043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359570+0000","last_change":"2024-03-06T07:21:54.359570+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359570+0000","last_undegraded":"2024-03-06T07:21:54.359570+0000","last_fullsized":"2024-03-06T07:21:54.359570+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359503+0000","last_change":"2024-03-06T07:21:43.321952+0000","last_active":"2024-03-06T07:21:54.359503+0000","last_peered":"2024-03-06T07:21:54.359503+0000","last_clean":"2024-03-06T07:21:54.359503+0000","last_became_active":"2024-03-06T07:21:43.321579+0000","last_became_peered":"2024-03-06T07:21:43.321579+0000","last_unstale":"2024-03-06T07:21:54.359503+0000","last_undegraded":"2024-03-06T07:21:54.359503+0000","last_fullsized":"2024-03-06T07:21:54.359503+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:04.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359648+0000","last_change":"2024-03-06T07:21:54.359648+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359648+0000","last_undegraded":"2024-03-06T07:21:54.359648+0000","last_fullsized":"2024-03-06T07:21:54.359648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368337+0000","last_change":"2024-03-06T07:21:43.321277+0000","last_active":"2024-03-06T07:21:54.368337+0000","last_peered":"2024-03-06T07:21:54.368337+0000","last_clean":"2024-03-06T07:21:54.368337+0000","last_became_active":"2024-03-06T07:21:43.320973+0000","last_became_peered":"2024-03-06T07:21:43.320973+0000","last_unstale":"2024-03-06T07:21:54.368337+0000","last_undegraded":"2024-03-06T07:21:54.368337+0000","last_fullsized":"2024-03-06T07:21:54.368337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:15.957299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362294+0000","last_change":"2024-03-06T07:21:54.362294+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362294+0000","last_undegraded":"2024-03-06T07:21:54.362294+0000","last_fullsized":"2024-03-06T07:21:54.362294+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364199+0000","last_change":"2024-03-06T07:21:50.373762+0000","last_active":"2024-03-06T07:21:54.364199+0000","last_peered":"2024-03-06T07:21:54.364199+0000","last_clean":"2024-03-06T07:21:54.364199+0000","last_became_active":"2024-03-06T07:21:50.373521+0000","last_became_peered":"2024-03-06T07:21:50.373521+0000","last_unstale":"2024-03-06T07:21:54.364199+0000","last_undegraded":"2024-03-06T07:21:54.364199+0000","last_fullsized":"2024-03-06T07:21:54.364199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359359+0000","last_change":"2024-03-06T07:21:54.359359+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359359+0000","last_undegraded":"2024-03-06T07:21:54.359359+0000","last_fullsized":"2024-03-06T07:21:54.359359+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357693+0000","last_change":"2024-03-06T07:21:43.321137+0000","last_active":"2024-03-06T07:21:54.357693+0000","last_peered":"2024-03-06T07:21:54.357693+0000","last_clean":"2024-03-06T07:21:54.357693+0000","last_became_active":"2024-03-06T07:21:43.320820+0000","last_became_peered":"2024-03-06T07:21:43.320820+0000","last_unstale":"2024-03-06T07:21:54.357693+0000","last_undegraded":"2024-03-06T07:21:54.357693+0000","last_fullsized":"2024-03-06T07:21:54.357693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:52.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359065+0000","last_change":"2024-03-06T07:21:54.359065+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359065+0000","last_undegraded":"2024-03-06T07:21:54.359065+0000","last_fullsized":"2024-03-06T07:21:54.359065+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.366975+0000","last_change":"2024-03-06T07:21:43.320518+0000","last_active":"2024-03-06T07:21:54.366975+0000","last_peered":"2024-03-06T07:21:54.366975+0000","last_clean":"2024-03-06T07:21:54.366975+0000","last_became_active":"2024-03-06T07:21:43.320178+0000","last_became_peered":"2024-03-06T07:21:43.320178+0000","last_unstale":"2024-03-06T07:21:54.366975+0000","last_undegraded":"2024-03-06T07:21:54.366975+0000","last_fullsized":"2024-03-06T07:21:54.366975+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360235+0000","last_change":"2024-03-06T07:21:54.360235+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360235+0000","last_undegraded":"2024-03-06T07:21:54.360235+0000","last_fullsized":"2024-03-06T07:21:54.360235+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370182+0000","last_change":"2024-03-06T07:21:43.324279+0000","last_active":"2024-03-06T07:21:54.370182+0000","last_peered":"2024-03-06T07:21:54.370182+0000","last_clean":"2024-03-06T07:21:54.370182+0000","last_became_active":"2024-03-06T07:21:43.323499+0000","last_became_peered":"2024-03-06T07:21:43.323499+0000","last_unstale":"2024-03-06T07:21:54.370182+0000","last_undegraded":"2024-03-06T07:21:54.370182+0000","last_fullsized":"2024-03-06T07:21:54.370182+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359045+0000","last_change":"2024-03-06T07:21:54.359045+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359045+0000","last_undegraded":"2024-03-06T07:21:54.359045+0000","last_fullsized":"2024-03-06T07:21:54.359045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357861+0000","last_change":"2024-03-06T07:21:43.318202+0000","last_active":"2024-03-06T07:21:54.357861+0000","last_peered":"2024-03-06T07:21:54.357861+0000","last_clean":"2024-03-06T07:21:54.357861+0000","last_became_active":"2024-03-06T07:21:43.317929+0000","last_became_peered":"2024-03-06T07:21:43.317929+0000","last_unstale":"2024-03-06T07:21:54.357861+0000","last_undegraded":"2024-03-06T07:21:54.357861+0000","last_fullsized":"2024-03-06T07:21:54.357861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367783+0000","last_change":"2024-03-06T07:21:54.367783+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367783+0000","last_undegraded":"2024-03-06T07:21:54.367783+0000","last_fullsized":"2024-03-06T07:21:54.367783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369274+0000","last_change":"2024-03-06T07:21:43.324265+0000","last_active":"2024-03-06T07:21:54.369274+0000","last_peered":"2024-03-06T07:21:54.369274+0000","last_clean":"2024-03-06T07:21:54.369274+0000","last_became_active":"2024-03-06T07:21:43.323877+0000","last_became_peered":"2024-03-06T07:21:43.323877+0000","last_unstale":"2024-03-06T07:21:54.369274+0000","last_undegraded":"2024-03-06T07:21:54.369274+0000","last_fullsized":"2024-03-06T07:21:54.369274+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.367102+0000","last_change":"2024-03-06T07:21:54.367102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.367102+0000","last_undegraded":"2024-03-06T07:21:54.367102+0000","last_fullsized":"2024-03-06T07:21:54.367102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365314+0000","last_change":"2024-03-06T07:21:43.321580+0000","last_active":"2024-03-06T07:21:54.365314+0000","last_peered":"2024-03-06T07:21:54.365314+0000","last_clean":"2024-03-06T07:21:54.365314+0000","last_became_active":"2024-03-06T07:21:43.319775+0000","last_became_peered":"2024-03-06T07:21:43.319775+0000","last_unstale":"2024-03-06T07:21:54.365314+0000","last_undegraded":"2024-03-06T07:21:54.365314+0000","last_fullsized":"2024-03-06T07:21:54.365314+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:32.802384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358875+0000","last_change":"2024-03-06T07:21:54.358875+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358875+0000","last_undegraded":"2024-03-06T07:21:54.358875+0000","last_fullsized":"2024-03-06T07:21:54.358875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365089+0000","last_change":"2024-03-06T07:21:43.322797+0000","last_active":"2024-03-06T07:21:54.365089+0000","last_peered":"2024-03-06T07:21:54.365089+0000","last_clean":"2024-03-06T07:21:54.365089+0000","last_became_active":"2024-03-06T07:21:43.322505+0000","last_became_peered":"2024-03-06T07:21:43.322505+0000","last_unstale":"2024-03-06T07:21:54.365089+0000","last_undegraded":"2024-03-06T07:21:54.365089+0000","last_fullsized":"2024-03-06T07:21:54.365089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.359127+0000","last_change":"2024-03-06T07:21:54.359127+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.359127+0000","last_undegraded":"2024-03-06T07:21:54.359127+0000","last_fullsized":"2024-03-06T07:21:54.359127+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358833+0000","last_change":"2024-03-06T07:21:43.326058+0000","last_active":"2024-03-06T07:21:54.358833+0000","last_peered":"2024-03-06T07:21:54.358833+0000","last_clean":"2024-03-06T07:21:54.358833+0000","last_became_active":"2024-03-06T07:21:43.325799+0000","last_became_peered":"2024-03-06T07:21:43.325799+0000","last_unstale":"2024-03-06T07:21:54.358833+0000","last_undegraded":"2024-03-06T07:21:54.358833+0000","last_fullsized":"2024-03-06T07:21:54.358833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357760+0000","last_change":"2024-03-06T07:21:54.357760+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357760+0000","last_undegraded":"2024-03-06T07:21:54.357760+0000","last_fullsized":"2024-03-06T07:21:54.357760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365258+0000","last_change":"2024-03-06T07:21:43.322671+0000","last_active":"2024-03-06T07:21:54.365258+0000","last_peered":"2024-03-06T07:21:54.365258+0000","last_clean":"2024-03-06T07:21:54.365258+0000","last_became_active":"2024-03-06T07:21:43.322376+0000","last_became_peered":"2024-03-06T07:21:43.322376+0000","last_unstale":"2024-03-06T07:21:54.365258+0000","last_undegraded":"2024-03-06T07:21:54.365258+0000","last_fullsized":"2024-03-06T07:21:54.365258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:15.049663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358331+0000","last_change":"2024-03-06T07:21:54.358331+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358331+0000","last_undegraded":"2024-03-06T07:21:54.358331+0000","last_fullsized":"2024-03-06T07:21:54.358331+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364803+0000","last_change":"2024-03-06T07:21:43.327014+0000","last_active":"2024-03-06T07:21:54.364803+0000","last_peered":"2024-03-06T07:21:54.364803+0000","last_clean":"2024-03-06T07:21:54.364803+0000","last_became_active":"2024-03-06T07:21:43.326759+0000","last_became_peered":"2024-03-06T07:21:43.326759+0000","last_unstale":"2024-03-06T07:21:54.364803+0000","last_undegraded":"2024-03-06T07:21:54.364803+0000","last_fullsized":"2024-03-06T07:21:54.364803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366485+0000","last_change":"2024-03-06T07:21:54.366485+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366485+0000","last_undegraded":"2024-03-06T07:21:54.366485+0000","last_fullsized":"2024-03-06T07:21:54.366485+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370315+0000","last_change":"2024-03-06T07:21:43.323199+0000","last_active":"2024-03-06T07:21:54.370315+0000","last_peered":"2024-03-06T07:21:54.370315+0000","last_clean":"2024-03-06T07:21:54.370315+0000","last_became_active":"2024-03-06T07:21:43.322835+0000","last_became_peered":"2024-03-06T07:21:43.322835+0000","last_unstale":"2024-03-06T07:21:54.370315+0000","last_undegraded":"2024-03-06T07:21:54.370315+0000","last_fullsized":"2024-03-06T07:21:54.370315+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.369128+0000","last_change":"2024-03-06T07:21:54.369128+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.369128+0000","last_undegraded":"2024-03-06T07:21:54.369128+0000","last_fullsized":"2024-03-06T07:21:54.369128+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357960+0000","last_change":"2024-03-06T07:21:43.318509+0000","last_active":"2024-03-06T07:21:54.357960+0000","last_peered":"2024-03-06T07:21:54.357960+0000","last_clean":"2024-03-06T07:21:54.357960+0000","last_became_active":"2024-03-06T07:21:43.318177+0000","last_became_peered":"2024-03-06T07:21:43.318177+0000","last_unstale":"2024-03-06T07:21:54.357960+0000","last_undegraded":"2024-03-06T07:21:54.357960+0000","last_fullsized":"2024-03-06T07:21:54.357960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361035+0000","last_change":"2024-03-06T07:21:54.361035+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361035+0000","last_undegraded":"2024-03-06T07:21:54.361035+0000","last_fullsized":"2024-03-06T07:21:54.361035+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369672+0000","last_change":"2024-03-06T07:21:43.332733+0000","last_active":"2024-03-06T07:21:54.369672+0000","last_peered":"2024-03-06T07:21:54.369672+0000","last_clean":"2024-03-06T07:21:54.369672+0000","last_became_active":"2024-03-06T07:21:43.331512+0000","last_became_peered":"2024-03-06T07:21:43.331512+0000","last_unstale":"2024-03-06T07:21:54.369672+0000","last_undegraded":"2024-03-06T07:21:54.369672+0000","last_fullsized":"2024-03-06T07:21:54.369672+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363616+0000","last_change":"2024-03-06T07:21:54.363616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363616+0000","last_undegraded":"2024-03-06T07:21:54.363616+0000","last_fullsized":"2024-03-06T07:21:54.363616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365453+0000","last_change":"2024-03-06T07:21:43.321054+0000","last_active":"2024-03-06T07:21:54.365453+0000","last_peered":"2024-03-06T07:21:54.365453+0000","last_clean":"2024-03-06T07:21:54.365453+0000","last_became_active":"2024-03-06T07:21:43.319435+0000","last_became_peered":"2024-03-06T07:21:43.319435+0000","last_unstale":"2024-03-06T07:21:54.365453+0000","last_undegraded":"2024-03-06T07:21:54.365453+0000","last_fullsized":"2024-03-06T07:21:54.365453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362601+0000","last_change":"2024-03-06T07:21:54.362601+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362601+0000","last_undegraded":"2024-03-06T07:21:54.362601+0000","last_fullsized":"2024-03-06T07:21:54.362601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364825+0000","last_change":"2024-03-06T07:21:43.328812+0000","last_active":"2024-03-06T07:21:54.364825+0000","last_peered":"2024-03-06T07:21:54.364825+0000","last_clean":"2024-03-06T07:21:54.364825+0000","last_became_active":"2024-03-06T07:21:43.328366+0000","last_became_peered":"2024-03-06T07:21:43.328366+0000","last_unstale":"2024-03-06T07:21:54.364825+0000","last_undegraded":"2024-03-06T07:21:54.364825+0000","last_fullsized":"2024-03-06T07:21:54.364825+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:48.381105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362773+0000","last_change":"2024-03-06T07:21:54.362773+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362773+0000","last_undegraded":"2024-03-06T07:21:54.362773+0000","last_fullsized":"2024-03-06T07:21:54.362773+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356882+0000","last_change":"2024-03-06T07:21:43.321155+0000","last_active":"2024-03-06T07:21:54.356882+0000","last_peered":"2024-03-06T07:21:54.356882+0000","last_clean":"2024-03-06T07:21:54.356882+0000","last_became_active":"2024-03-06T07:21:43.320747+0000","last_became_peered":"2024-03-06T07:21:43.320747+0000","last_unstale":"2024-03-06T07:21:54.356882+0000","last_undegraded":"2024-03-06T07:21:54.356882+0000","last_fullsized":"2024-03-06T07:21:54.356882+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:42.424597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358521+0000","last_change":"2024-03-06T07:21:54.358521+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358521+0000","last_undegraded":"2024-03-06T07:21:54.358521+0000","last_fullsized":"2024-03-06T07:21:54.358521+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368700+0000","last_change":"2024-03-06T07:21:43.326550+0000","last_active":"2024-03-06T07:21:54.368700+0000","last_peered":"2024-03-06T07:21:54.368700+0000","last_clean":"2024-03-06T07:21:54.368700+0000","last_became_active":"2024-03-06T07:21:43.326254+0000","last_became_peered":"2024-03-06T07:21:43.326254+0000","last_unstale":"2024-03-06T07:21:54.368700+0000","last_undegraded":"2024-03-06T07:21:54.368700+0000","last_fullsized":"2024-03-06T07:21:54.368700+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357560+0000","last_change":"2024-03-06T07:21:54.357560+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357560+0000","last_undegraded":"2024-03-06T07:21:54.357560+0000","last_fullsized":"2024-03-06T07:21:54.357560+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360266+0000","last_change":"2024-03-06T07:21:43.322773+0000","last_active":"2024-03-06T07:21:54.360266+0000","last_peered":"2024-03-06T07:21:54.360266+0000","last_clean":"2024-03-06T07:21:54.360266+0000","last_became_active":"2024-03-06T07:21:43.322525+0000","last_became_peered":"2024-03-06T07:21:43.322525+0000","last_unstale":"2024-03-06T07:21:54.360266+0000","last_undegraded":"2024-03-06T07:21:54.360266+0000","last_fullsized":"2024-03-06T07:21:54.360266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362616+0000","last_change":"2024-03-06T07:21:54.362616+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362616+0000","last_undegraded":"2024-03-06T07:21:54.362616+0000","last_fullsized":"2024-03-06T07:21:54.362616+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376862+0000","last_change":"2024-03-06T07:21:54.366905+0000","last_active":"2024-03-06T07:21:54.376862+0000","last_peered":"2024-03-06T07:21:54.376862+0000","last_clean":"2024-03-06T07:21:54.376862+0000","last_became_active":"2024-03-06T07:21:54.366627+0000","last_became_peered":"2024-03-06T07:21:54.366627+0000","last_unstale":"2024-03-06T07:21:54.376862+0000","last_undegraded":"2024-03-06T07:21:54.376862+0000","last_fullsized":"2024-03-06T07:21:54.376862+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356615+0000","last_change":"2024-03-06T07:21:43.319837+0000","last_active":"2024-03-06T07:21:54.356615+0000","last_peered":"2024-03-06T07:21:54.356615+0000","last_clean":"2024-03-06T07:21:54.356615+0000","last_became_active":"2024-03-06T07:21:43.319564+0000","last_became_peered":"2024-03-06T07:21:43.319564+0000","last_unstale":"2024-03-06T07:21:54.356615+0000","last_undegraded":"2024-03-06T07:21:54.356615+0000","last_fullsized":"2024-03-06T07:21:54.356615+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365243+0000","last_change":"2024-03-06T07:21:42.310584+0000","last_active":"2024-03-06T07:21:54.365243+0000","last_peered":"2024-03-06T07:21:54.365243+0000","last_clean":"2024-03-06T07:21:54.365243+0000","last_became_active":"2024-03-06T07:21:42.310317+0000","last_became_peered":"2024-03-06T07:21:42.310317+0000","last_unstale":"2024-03-06T07:21:54.365243+0000","last_undegraded":"2024-03-06T07:21:54.365243+0000","last_fullsized":"2024-03-06T07:21:54.365243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362416+0000","last_change":"2024-03-06T07:21:54.362416+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362416+0000","last_undegraded":"2024-03-06T07:21:54.362416+0000","last_fullsized":"2024-03-06T07:21:54.362416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380180+0000","last_change":"2024-03-06T07:21:54.377542+0000","last_active":"2024-03-06T07:21:54.380180+0000","last_peered":"2024-03-06T07:21:54.380180+0000","last_clean":"2024-03-06T07:21:54.380180+0000","last_became_active":"2024-03-06T07:21:54.377250+0000","last_became_peered":"2024-03-06T07:21:54.377250+0000","last_unstale":"2024-03-06T07:21:54.380180+0000","last_undegraded":"2024-03-06T07:21:54.380180+0000","last_fullsized":"2024-03-06T07:21:54.380180+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:36.750007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359472+0000","last_change":"2024-03-06T07:21:43.323883+0000","last_active":"2024-03-06T07:21:54.359472+0000","last_peered":"2024-03-06T07:21:54.359472+0000","last_clean":"2024-03-06T07:21:54.359472+0000","last_became_active":"2024-03-06T07:21:43.323613+0000","last_became_peered":"2024-03-06T07:21:43.323613+0000","last_unstale":"2024-03-06T07:21:54.359472+0000","last_undegraded":"2024-03-06T07:21:54.359472+0000","last_fullsized":"2024-03-06T07:21:54.359472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370119+0000","last_change":"2024-03-06T07:21:42.304199+0000","last_active":"2024-03-06T07:21:54.370119+0000","last_peered":"2024-03-06T07:21:54.370119+0000","last_clean":"2024-03-06T07:21:54.370119+0000","last_became_active":"2024-03-06T07:21:42.303802+0000","last_became_peered":"2024-03-06T07:21:42.303802+0000","last_unstale":"2024-03-06T07:21:54.370119+0000","last_undegraded":"2024-03-06T07:21:54.370119+0000","last_fullsized":"2024-03-06T07:21:54.370119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362959+0000","last_change":"2024-03-06T07:21:54.362959+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362959+0000","last_undegraded":"2024-03-06T07:21:54.362959+0000","last_fullsized":"2024-03-06T07:21:54.362959+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376955+0000","last_change":"2024-03-06T07:21:54.368687+0000","last_active":"2024-03-06T07:21:54.376955+0000","last_peered":"2024-03-06T07:21:54.376955+0000","last_clean":"2024-03-06T07:21:54.376955+0000","last_became_active":"2024-03-06T07:21:54.368106+0000","last_became_peered":"2024-03-06T07:21:54.368106+0000","last_unstale":"2024-03-06T07:21:54.376955+0000","last_undegraded":"2024-03-06T07:21:54.376955+0000","last_fullsized":"2024-03-06T07:21:54.376955+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364372+0000","last_change":"2024-03-06T07:21:43.323512+0000","last_active":"2024-03-06T07:21:54.364372+0000","last_peered":"2024-03-06T07:21:54.364372+0000","last_clean":"2024-03-06T07:21:54.364372+0000","last_became_active":"2024-03-06T07:21:43.323216+0000","last_became_peered":"2024-03-06T07:21:43.323216+0000","last_unstale":"2024-03-06T07:21:54.364372+0000","last_undegraded":"2024-03-06T07:21:54.364372+0000","last_fullsized":"2024-03-06T07:21:54.364372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:31.821522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369735+0000","last_change":"2024-03-06T07:21:42.305711+0000","last_active":"2024-03-06T07:21:54.369735+0000","last_peered":"2024-03-06T07:21:54.369735+0000","last_clean":"2024-03-06T07:21:54.369735+0000","last_became_active":"2024-03-06T07:21:42.305431+0000","last_became_peered":"2024-03-06T07:21:42.305431+0000","last_unstale":"2024-03-06T07:21:54.369735+0000","last_undegraded":"2024-03-06T07:21:54.369735+0000","last_fullsized":"2024-03-06T07:21:54.369735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.357254+0000","last_change":"2024-03-06T07:21:54.357254+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.357254+0000","last_undegraded":"2024-03-06T07:21:54.357254+0000","last_fullsized":"2024-03-06T07:21:54.357254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374713+0000","last_change":"2024-03-06T07:21:54.370174+0000","last_active":"2024-03-06T07:21:54.374713+0000","last_peered":"2024-03-06T07:21:54.374713+0000","last_clean":"2024-03-06T07:21:54.374713+0000","last_became_active":"2024-03-06T07:21:54.369897+0000","last_became_peered":"2024-03-06T07:21:54.369897+0000","last_unstale":"2024-03-06T07:21:54.374713+0000","last_undegraded":"2024-03-06T07:21:54.374713+0000","last_fullsized":"2024-03-06T07:21:54.374713+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359478+0000","last_change":"2024-03-06T07:21:43.317966+0000","last_active":"2024-03-06T07:21:54.359478+0000","last_peered":"2024-03-06T07:21:54.359478+0000","last_clean":"2024-03-06T07:21:54.359478+0000","last_became_active":"2024-03-06T07:21:43.317700+0000","last_became_peered":"2024-03-06T07:21:43.317700+0000","last_unstale":"2024-03-06T07:21:54.359478+0000","last_undegraded":"2024-03-06T07:21:54.359478+0000","last_fullsized":"2024-03-06T07:21:54.359478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359768+0000","last_change":"2024-03-06T07:21:42.315045+0000","last_active":"2024-03-06T07:21:54.359768+0000","last_peered":"2024-03-06T07:21:54.359768+0000","last_clean":"2024-03-06T07:21:54.359768+0000","last_became_active":"2024-03-06T07:21:42.314768+0000","last_became_peered":"2024-03-06T07:21:42.314768+0000","last_unstale":"2024-03-06T07:21:54.359768+0000","last_undegraded":"2024-03-06T07:21:54.359768+0000","last_fullsized":"2024-03-06T07:21:54.359768+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:14.953476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360237+0000","last_change":"2024-03-06T07:21:54.360237+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360237+0000","last_undegraded":"2024-03-06T07:21:54.360237+0000","last_fullsized":"2024-03-06T07:21:54.360237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379825+0000","last_change":"2024-03-06T07:21:54.375783+0000","last_active":"2024-03-06T07:21:54.379825+0000","last_peered":"2024-03-06T07:21:54.379825+0000","last_clean":"2024-03-06T07:21:54.379825+0000","last_became_active":"2024-03-06T07:21:54.375473+0000","last_became_peered":"2024-03-06T07:21:54.375473+0000","last_unstale":"2024-03-06T07:21:54.379825+0000","last_undegraded":"2024-03-06T07:21:54.379825+0000","last_fullsized":"2024-03-06T07:21:54.379825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:08.612677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370306+0000","last_change":"2024-03-06T07:21:50.343971+0000","last_active":"2024-03-06T07:21:54.370306+0000","last_peered":"2024-03-06T07:21:54.370306+0000","last_clean":"2024-03-06T07:21:54.370306+0000","last_became_active":"2024-03-06T07:21:50.343658+0000","last_became_peered":"2024-03-06T07:21:50.343658+0000","last_unstale":"2024-03-06T07:21:54.370306+0000","last_undegraded":"2024-03-06T07:21:54.370306+0000","last_fullsized":"2024-03-06T07:21:54.370306+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364922+0000","last_change":"2024-03-06T07:21:42.317614+0000","last_active":"2024-03-06T07:21:54.364922+0000","last_peered":"2024-03-06T07:21:54.364922+0000","last_clean":"2024-03-06T07:21:54.364922+0000","last_became_active":"2024-03-06T07:21:42.317094+0000","last_became_peered":"2024-03-06T07:21:42.317094+0000","last_unstale":"2024-03-06T07:21:54.364922+0000","last_undegraded":"2024-03-06T07:21:54.364922+0000","last_fullsized":"2024-03-06T07:21:54.364922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:15.739191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364680+0000","last_change":"2024-03-06T07:21:54.364680+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364680+0000","last_undegraded":"2024-03-06T07:21:54.364680+0000","last_fullsized":"2024-03-06T07:21:54.364680+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377152+0000","last_change":"2024-03-06T07:21:54.375914+0000","last_active":"2024-03-06T07:21:54.377152+0000","last_peered":"2024-03-06T07:21:54.377152+0000","last_clean":"2024-03-06T07:21:54.377152+0000","last_became_active":"2024-03-06T07:21:54.375619+0000","last_became_peered":"2024-03-06T07:21:54.375619+0000","last_unstale":"2024-03-06T07:21:54.377152+0000","last_undegraded":"2024-03-06T07:21:54.377152+0000","last_fullsized":"2024-03-06T07:21:54.377152+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:26.719365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358617+0000","last_change":"2024-03-06T07:21:43.318727+0000","last_active":"2024-03-06T07:21:54.358617+0000","last_peered":"2024-03-06T07:21:54.358617+0000","last_clean":"2024-03-06T07:21:54.358617+0000","last_became_active":"2024-03-06T07:21:43.318440+0000","last_became_peered":"2024-03-06T07:21:43.318440+0000","last_unstale":"2024-03-06T07:21:54.358617+0000","last_undegraded":"2024-03-06T07:21:54.358617+0000","last_fullsized":"2024-03-06T07:21:54.358617+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370222+0000","last_change":"2024-03-06T07:21:42.307726+0000","last_active":"2024-03-06T07:21:54.370222+0000","last_peered":"2024-03-06T07:21:54.370222+0000","last_clean":"2024-03-06T07:21:54.370222+0000","last_became_active":"2024-03-06T07:21:42.307445+0000","last_became_peered":"2024-03-06T07:21:42.307445+0000","last_unstale":"2024-03-06T07:21:54.370222+0000","last_undegraded":"2024-03-06T07:21:54.370222+0000","last_fullsized":"2024-03-06T07:21:54.370222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366515+0000","last_change":"2024-03-06T07:21:54.366515+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366515+0000","last_undegraded":"2024-03-06T07:21:54.366515+0000","last_fullsized":"2024-03-06T07:21:54.366515+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376980+0000","last_change":"2024-03-06T07:21:54.370955+0000","last_active":"2024-03-06T07:21:54.376980+0000","last_peered":"2024-03-06T07:21:54.376980+0000","last_clean":"2024-03-06T07:21:54.376980+0000","last_became_active":"2024-03-06T07:21:54.370680+0000","last_became_peered":"2024-03-06T07:21:54.370680+0000","last_unstale":"2024-03-06T07:21:54.376980+0000","last_undegraded":"2024-03-06T07:21:54.376980+0000","last_fullsized":"2024-03-06T07:21:54.376980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359348+0000","last_change":"2024-03-06T07:21:43.318372+0000","last_active":"2024-03-06T07:21:54.359348+0000","last_peered":"2024-03-06T07:21:54.359348+0000","last_clean":"2024-03-06T07:21:54.359348+0000","last_became_active":"2024-03-06T07:21:43.318047+0000","last_became_peered":"2024-03-06T07:21:43.318047+0000","last_unstale":"2024-03-06T07:21:54.359348+0000","last_undegraded":"2024-03-06T07:21:54.359348+0000","last_fullsized":"2024-03-06T07:21:54.359348+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364556+0000","last_change":"2024-03-06T07:21:42.310677+0000","last_active":"2024-03-06T07:21:54.364556+0000","last_peered":"2024-03-06T07:21:54.364556+0000","last_clean":"2024-03-06T07:21:54.364556+0000","last_became_active":"2024-03-06T07:21:42.310375+0000","last_became_peered":"2024-03-06T07:21:42.310375+0000","last_unstale":"2024-03-06T07:21:54.364556+0000","last_undegraded":"2024-03-06T07:21:54.364556+0000","last_fullsized":"2024-03-06T07:21:54.364556+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:13.837451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358658+0000","last_change":"2024-03-06T07:21:54.358658+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358658+0000","last_undegraded":"2024-03-06T07:21:54.358658+0000","last_fullsized":"2024-03-06T07:21:54.358658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379656+0000","last_change":"2024-03-06T07:21:54.374914+0000","last_active":"2024-03-06T07:21:54.379656+0000","last_peered":"2024-03-06T07:21:54.379656+0000","last_clean":"2024-03-06T07:21:54.379656+0000","last_became_active":"2024-03-06T07:21:54.374570+0000","last_became_peered":"2024-03-06T07:21:54.374570+0000","last_unstale":"2024-03-06T07:21:54.379656+0000","last_undegraded":"2024-03-06T07:21:54.379656+0000","last_fullsized":"2024-03-06T07:21:54.379656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:05:03.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358220+0000","last_change":"2024-03-06T07:21:43.325740+0000","last_active":"2024-03-06T07:21:54.358220+0000","last_peered":"2024-03-06T07:21:54.358220+0000","last_clean":"2024-03-06T07:21:54.358220+0000","last_became_active":"2024-03-06T07:21:43.325425+0000","last_became_peered":"2024-03-06T07:21:43.325425+0000","last_unstale":"2024-03-06T07:21:54.358220+0000","last_undegraded":"2024-03-06T07:21:54.358220+0000","last_fullsized":"2024-03-06T07:21:54.358220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364895+0000","last_change":"2024-03-06T07:21:42.319786+0000","last_active":"2024-03-06T07:21:54.364895+0000","last_peered":"2024-03-06T07:21:54.364895+0000","last_clean":"2024-03-06T07:21:54.364895+0000","last_became_active":"2024-03-06T07:21:42.319430+0000","last_became_peered":"2024-03-06T07:21:42.319430+0000","last_unstale":"2024-03-06T07:21:54.364895+0000","last_undegraded":"2024-03-06T07:21:54.364895+0000","last_fullsized":"2024-03-06T07:21:54.364895+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:09.719754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360176+0000","last_change":"2024-03-06T07:21:54.360176+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360176+0000","last_undegraded":"2024-03-06T07:21:54.360176+0000","last_fullsized":"2024-03-06T07:21:54.360176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369499+0000","last_change":"2024-03-06T07:21:54.367800+0000","last_active":"2024-03-06T07:21:54.369499+0000","last_peered":"2024-03-06T07:21:54.369499+0000","last_clean":"2024-03-06T07:21:54.369499+0000","last_became_active":"2024-03-06T07:21:54.367468+0000","last_became_peered":"2024-03-06T07:21:54.367468+0000","last_unstale":"2024-03-06T07:21:54.369499+0000","last_undegraded":"2024-03-06T07:21:54.369499+0000","last_fullsized":"2024-03-06T07:21:54.369499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.330805+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.330267+0000","last_became_peered":"2024-03-06T07:21:43.330267+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:56.716578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358318+0000","last_change":"2024-03-06T07:21:42.312213+0000","last_active":"2024-03-06T07:21:54.358318+0000","last_peered":"2024-03-06T07:21:54.358318+0000","last_clean":"2024-03-06T07:21:54.358318+0000","last_became_active":"2024-03-06T07:21:42.311878+0000","last_became_peered":"2024-03-06T07:21:42.311878+0000","last_unstale":"2024-03-06T07:21:54.358318+0000","last_undegraded":"2024-03-06T07:21:54.358318+0000","last_fullsized":"2024-03-06T07:21:54.358318+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358914+0000","last_change":"2024-03-06T07:21:54.358914+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358914+0000","last_undegraded":"2024-03-06T07:21:54.358914+0000","last_fullsized":"2024-03-06T07:21:54.358914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380052+0000","last_change":"2024-03-06T07:21:54.377646+0000","last_active":"2024-03-06T07:21:54.380052+0000","last_peered":"2024-03-06T07:21:54.380052+0000","last_clean":"2024-03-06T07:21:54.380052+0000","last_became_active":"2024-03-06T07:21:54.377219+0000","last_became_peered":"2024-03-06T07:21:54.377219+0000","last_unstale":"2024-03-06T07:21:54.380052+0000","last_undegraded":"2024-03-06T07:21:54.380052+0000","last_fullsized":"2024-03-06T07:21:54.380052+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:59.768227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:43.320122+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:43.319799+0000","last_became_peered":"2024-03-06T07:21:43.319799+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370295+0000","last_change":"2024-03-06T07:21:42.319681+0000","last_active":"2024-03-06T07:21:54.370295+0000","last_peered":"2024-03-06T07:21:54.370295+0000","last_clean":"2024-03-06T07:21:54.370295+0000","last_became_active":"2024-03-06T07:21:42.319452+0000","last_became_peered":"2024-03-06T07:21:42.319452+0000","last_unstale":"2024-03-06T07:21:54.370295+0000","last_undegraded":"2024-03-06T07:21:54.370295+0000","last_fullsized":"2024-03-06T07:21:54.370295+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:35.308790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362688+0000","last_change":"2024-03-06T07:21:54.362688+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362688+0000","last_undegraded":"2024-03-06T07:21:54.362688+0000","last_fullsized":"2024-03-06T07:21:54.362688+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368199+0000","last_change":"2024-03-06T07:21:54.367115+0000","last_active":"2024-03-06T07:21:54.368199+0000","last_peered":"2024-03-06T07:21:54.368199+0000","last_clean":"2024-03-06T07:21:54.368199+0000","last_became_active":"2024-03-06T07:21:54.366564+0000","last_became_peered":"2024-03-06T07:21:54.366564+0000","last_unstale":"2024-03-06T07:21:54.368199+0000","last_undegraded":"2024-03-06T07:21:54.368199+0000","last_fullsized":"2024-03-06T07:21:54.368199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365114+0000","last_change":"2024-03-06T07:21:43.323490+0000","last_active":"2024-03-06T07:21:54.365114+0000","last_peered":"2024-03-06T07:21:54.365114+0000","last_clean":"2024-03-06T07:21:54.365114+0000","last_became_active":"2024-03-06T07:21:43.323088+0000","last_became_peered":"2024-03-06T07:21:43.323088+0000","last_unstale":"2024-03-06T07:21:54.365114+0000","last_undegraded":"2024-03-06T07:21:54.365114+0000","last_fullsized":"2024-03-06T07:21:54.365114+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358795+0000","last_change":"2024-03-06T07:21:42.312732+0000","last_active":"2024-03-06T07:21:54.358795+0000","last_peered":"2024-03-06T07:21:54.358795+0000","last_clean":"2024-03-06T07:21:54.358795+0000","last_became_active":"2024-03-06T07:21:42.312462+0000","last_became_peered":"2024-03-06T07:21:42.312462+0000","last_unstale":"2024-03-06T07:21:54.358795+0000","last_undegraded":"2024-03-06T07:21:54.358795+0000","last_fullsized":"2024-03-06T07:21:54.358795+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363414+0000","last_change":"2024-03-06T07:21:54.363414+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363414+0000","last_undegraded":"2024-03-06T07:21:54.363414+0000","last_fullsized":"2024-03-06T07:21:54.363414+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367522+0000","last_change":"2024-03-06T07:21:54.365558+0000","last_active":"2024-03-06T07:21:54.367522+0000","last_peered":"2024-03-06T07:21:54.367522+0000","last_clean":"2024-03-06T07:21:54.367522+0000","last_became_active":"2024-03-06T07:21:54.365272+0000","last_became_peered":"2024-03-06T07:21:54.365272+0000","last_unstale":"2024-03-06T07:21:54.367522+0000","last_undegraded":"2024-03-06T07:21:54.367522+0000","last_fullsized":"2024-03-06T07:21:54.367522+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369842+0000","last_change":"2024-03-06T07:21:43.320022+0000","last_active":"2024-03-06T07:21:54.369842+0000","last_peered":"2024-03-06T07:21:54.369842+0000","last_clean":"2024-03-06T07:21:54.369842+0000","last_became_active":"2024-03-06T07:21:43.319716+0000","last_became_peered":"2024-03-06T07:21:43.319716+0000","last_unstale":"2024-03-06T07:21:54.369842+0000","last_undegraded":"2024-03-06T07:21:54.369842+0000","last_fullsized":"2024-03-06T07:21:54.369842+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:50.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359596+0000","last_change":"2024-03-06T07:21:42.308298+0000","last_active":"2024-03-06T07:21:54.359596+0000","last_peered":"2024-03-06T07:21:54.359596+0000","last_clean":"2024-03-06T07:21:54.359596+0000","last_became_active":"2024-03-06T07:21:42.308008+0000","last_became_peered":"2024-03-06T07:21:42.308008+0000","last_unstale":"2024-03-06T07:21:54.359596+0000","last_undegraded":"2024-03-06T07:21:54.359596+0000","last_fullsized":"2024-03-06T07:21:54.359596+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:51.723259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361630+0000","last_change":"2024-03-06T07:21:54.361630+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361630+0000","last_undegraded":"2024-03-06T07:21:54.361630+0000","last_fullsized":"2024-03-06T07:21:54.361630+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376550+0000","last_change":"2024-03-06T07:21:54.375937+0000","last_active":"2024-03-06T07:21:54.376550+0000","last_peered":"2024-03-06T07:21:54.376550+0000","last_clean":"2024-03-06T07:21:54.376550+0000","last_became_active":"2024-03-06T07:21:54.375648+0000","last_became_peered":"2024-03-06T07:21:54.375648+0000","last_unstale":"2024-03-06T07:21:54.376550+0000","last_undegraded":"2024-03-06T07:21:54.376550+0000","last_fullsized":"2024-03-06T07:21:54.376550+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359583+0000","last_change":"2024-03-06T07:21:43.323228+0000","last_active":"2024-03-06T07:21:54.359583+0000","last_peered":"2024-03-06T07:21:54.359583+0000","last_clean":"2024-03-06T07:21:54.359583+0000","last_became_active":"2024-03-06T07:21:43.322951+0000","last_became_peered":"2024-03-06T07:21:43.322951+0000","last_unstale":"2024-03-06T07:21:54.359583+0000","last_undegraded":"2024-03-06T07:21:54.359583+0000","last_fullsized":"2024-03-06T07:21:54.359583+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365013+0000","last_change":"2024-03-06T07:21:42.321872+0000","last_active":"2024-03-06T07:21:54.365013+0000","last_peered":"2024-03-06T07:21:54.365013+0000","last_clean":"2024-03-06T07:21:54.365013+0000","last_became_active":"2024-03-06T07:21:42.321384+0000","last_became_peered":"2024-03-06T07:21:42.321384+0000","last_unstale":"2024-03-06T07:21:54.365013+0000","last_undegraded":"2024-03-06T07:21:54.365013+0000","last_fullsized":"2024-03-06T07:21:54.365013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358625+0000","last_change":"2024-03-06T07:21:54.358625+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358625+0000","last_undegraded":"2024-03-06T07:21:54.358625+0000","last_fullsized":"2024-03-06T07:21:54.358625+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375601+0000","last_change":"2024-03-06T07:21:54.373488+0000","last_active":"2024-03-06T07:21:54.375601+0000","last_peered":"2024-03-06T07:21:54.375601+0000","last_clean":"2024-03-06T07:21:54.375601+0000","last_became_active":"2024-03-06T07:21:54.373225+0000","last_became_peered":"2024-03-06T07:21:54.373225+0000","last_unstale":"2024-03-06T07:21:54.375601+0000","last_undegraded":"2024-03-06T07:21:54.375601+0000","last_fullsized":"2024-03-06T07:21:54.375601+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369356+0000","last_change":"2024-03-06T07:21:43.324244+0000","last_active":"2024-03-06T07:21:54.369356+0000","last_peered":"2024-03-06T07:21:54.369356+0000","last_clean":"2024-03-06T07:21:54.369356+0000","last_became_active":"2024-03-06T07:21:43.323549+0000","last_became_peered":"2024-03-06T07:21:43.323549+0000","last_unstale":"2024-03-06T07:21:54.369356+0000","last_undegraded":"2024-03-06T07:21:54.369356+0000","last_fullsized":"2024-03-06T07:21:54.369356+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364546+0000","last_change":"2024-03-06T07:21:42.318308+0000","last_active":"2024-03-06T07:21:54.364546+0000","last_peered":"2024-03-06T07:21:54.364546+0000","last_clean":"2024-03-06T07:21:54.364546+0000","last_became_active":"2024-03-06T07:21:42.318080+0000","last_became_peered":"2024-03-06T07:21:42.318080+0000","last_unstale":"2024-03-06T07:21:54.364546+0000","last_undegraded":"2024-03-06T07:21:54.364546+0000","last_fullsized":"2024-03-06T07:21:54.364546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360511+0000","last_change":"2024-03-06T07:21:54.360511+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360511+0000","last_undegraded":"2024-03-06T07:21:54.360511+0000","last_fullsized":"2024-03-06T07:21:54.360511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378458+0000","last_change":"2024-03-06T07:21:54.377690+0000","last_active":"2024-03-06T07:21:54.378458+0000","last_peered":"2024-03-06T07:21:54.378458+0000","last_clean":"2024-03-06T07:21:54.378458+0000","last_became_active":"2024-03-06T07:21:54.377408+0000","last_became_peered":"2024-03-06T07:21:54.377408+0000","last_unstale":"2024-03-06T07:21:54.378458+0000","last_undegraded":"2024-03-06T07:21:54.378458+0000","last_fullsized":"2024-03-06T07:21:54.378458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:31.244976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359542+0000","last_change":"2024-03-06T07:21:43.331853+0000","last_active":"2024-03-06T07:21:54.359542+0000","last_peered":"2024-03-06T07:21:54.359542+0000","last_clean":"2024-03-06T07:21:54.359542+0000","last_became_active":"2024-03-06T07:21:43.331560+0000","last_became_peered":"2024-03-06T07:21:43.331560+0000","last_unstale":"2024-03-06T07:21:54.359542+0000","last_undegraded":"2024-03-06T07:21:54.359542+0000","last_fullsized":"2024-03-06T07:21:54.359542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.357780+0000","last_change":"2024-03-06T07:21:42.309167+0000","last_active":"2024-03-06T07:21:54.357780+0000","last_peered":"2024-03-06T07:21:54.357780+0000","last_clean":"2024-03-06T07:21:54.357780+0000","last_became_active":"2024-03-06T07:21:42.308882+0000","last_became_peered":"2024-03-06T07:21:42.308882+0000","last_unstale":"2024-03-06T07:21:54.357780+0000","last_undegraded":"2024-03-06T07:21:54.357780+0000","last_fullsized":"2024-03-06T07:21:54.357780+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.366074+0000","last_change":"2024-03-06T07:21:54.366074+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.366074+0000","last_undegraded":"2024-03-06T07:21:54.366074+0000","last_fullsized":"2024-03-06T07:21:54.366074+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378021+0000","last_change":"2024-03-06T07:21:54.377427+0000","last_active":"2024-03-06T07:21:54.378021+0000","last_peered":"2024-03-06T07:21:54.378021+0000","last_clean":"2024-03-06T07:21:54.378021+0000","last_became_active":"2024-03-06T07:21:54.377183+0000","last_became_peered":"2024-03-06T07:21:54.377183+0000","last_unstale":"2024-03-06T07:21:54.378021+0000","last_undegraded":"2024-03-06T07:21:54.378021+0000","last_fullsized":"2024-03-06T07:21:54.378021+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:02.096882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370105+0000","last_change":"2024-03-06T07:21:43.324536+0000","last_active":"2024-03-06T07:21:54.370105+0000","last_peered":"2024-03-06T07:21:54.370105+0000","last_clean":"2024-03-06T07:21:54.370105+0000","last_became_active":"2024-03-06T07:21:43.323818+0000","last_became_peered":"2024-03-06T07:21:43.323818+0000","last_unstale":"2024-03-06T07:21:54.370105+0000","last_undegraded":"2024-03-06T07:21:54.370105+0000","last_fullsized":"2024-03-06T07:21:54.370105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358547+0000","last_change":"2024-03-06T07:21:42.308460+0000","last_active":"2024-03-06T07:21:54.358547+0000","last_peered":"2024-03-06T07:21:54.358547+0000","last_clean":"2024-03-06T07:21:54.358547+0000","last_became_active":"2024-03-06T07:21:42.307461+0000","last_became_peered":"2024-03-06T07:21:42.307461+0000","last_unstale":"2024-03-06T07:21:54.358547+0000","last_undegraded":"2024-03-06T07:21:54.358547+0000","last_fullsized":"2024-03-06T07:21:54.358547+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:34.346180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364356+0000","last_change":"2024-03-06T07:21:54.364356+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364356+0000","last_undegraded":"2024-03-06T07:21:54.364356+0000","last_fullsized":"2024-03-06T07:21:54.364356+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367923+0000","last_change":"2024-03-06T07:21:54.366324+0000","last_active":"2024-03-06T07:21:54.367923+0000","last_peered":"2024-03-06T07:21:54.367923+0000","last_clean":"2024-03-06T07:21:54.367923+0000","last_became_active":"2024-03-06T07:21:54.366064+0000","last_became_peered":"2024-03-06T07:21:54.366064+0000","last_unstale":"2024-03-06T07:21:54.367923+0000","last_undegraded":"2024-03-06T07:21:54.367923+0000","last_fullsized":"2024-03-06T07:21:54.367923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:55.616266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358427+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.358427+0000","last_peered":"2024-03-06T07:21:54.358427+0000","last_clean":"2024-03-06T07:21:54.358427+0000","last_became_active":"2024-03-06T07:21:43.316848+0000","last_became_peered":"2024-03-06T07:21:43.316848+0000","last_unstale":"2024-03-06T07:21:54.358427+0000","last_undegraded":"2024-03-06T07:21:54.358427+0000","last_fullsized":"2024-03-06T07:21:54.358427+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370393+0000","last_change":"2024-03-06T07:21:42.319998+0000","last_active":"2024-03-06T07:21:54.370393+0000","last_peered":"2024-03-06T07:21:54.370393+0000","last_clean":"2024-03-06T07:21:54.370393+0000","last_became_active":"2024-03-06T07:21:42.319529+0000","last_became_peered":"2024-03-06T07:21:42.319529+0000","last_unstale":"2024-03-06T07:21:54.370393+0000","last_undegraded":"2024-03-06T07:21:54.370393+0000","last_fullsized":"2024-03-06T07:21:54.370393+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361900+0000","last_change":"2024-03-06T07:21:54.361900+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361900+0000","last_undegraded":"2024-03-06T07:21:54.361900+0000","last_fullsized":"2024-03-06T07:21:54.361900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374081+0000","last_change":"2024-03-06T07:21:54.370046+0000","last_active":"2024-03-06T07:21:54.374081+0000","last_peered":"2024-03-06T07:21:54.374081+0000","last_clean":"2024-03-06T07:21:54.374081+0000","last_became_active":"2024-03-06T07:21:54.369791+0000","last_became_peered":"2024-03-06T07:21:54.369791+0000","last_unstale":"2024-03-06T07:21:54.374081+0000","last_undegraded":"2024-03-06T07:21:54.374081+0000","last_fullsized":"2024-03-06T07:21:54.374081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:09.373432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367145+0000","last_change":"2024-03-06T07:21:43.327074+0000","last_active":"2024-03-06T07:21:54.367145+0000","last_peered":"2024-03-06T07:21:54.367145+0000","last_clean":"2024-03-06T07:21:54.367145+0000","last_became_active":"2024-03-06T07:21:43.326797+0000","last_became_peered":"2024-03-06T07:21:43.326797+0000","last_unstale":"2024-03-06T07:21:54.367145+0000","last_undegraded":"2024-03-06T07:21:54.367145+0000","last_fullsized":"2024-03-06T07:21:54.367145+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.370126+0000","last_change":"2024-03-06T07:21:42.310454+0000","last_active":"2024-03-06T07:21:54.370126+0000","last_peered":"2024-03-06T07:21:54.370126+0000","last_clean":"2024-03-06T07:21:54.370126+0000","last_became_active":"2024-03-06T07:21:42.310172+0000","last_became_peered":"2024-03-06T07:21:42.310172+0000","last_unstale":"2024-03-06T07:21:54.370126+0000","last_undegraded":"2024-03-06T07:21:54.370126+0000","last_fullsized":"2024-03-06T07:21:54.370126+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:34.182536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360981+0000","last_change":"2024-03-06T07:21:54.360981+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360981+0000","last_undegraded":"2024-03-06T07:21:54.360981+0000","last_fullsized":"2024-03-06T07:21:54.360981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376984+0000","last_change":"2024-03-06T07:21:54.368897+0000","last_active":"2024-03-06T07:21:54.376984+0000","last_peered":"2024-03-06T07:21:54.376984+0000","last_clean":"2024-03-06T07:21:54.376984+0000","last_became_active":"2024-03-06T07:21:54.368398+0000","last_became_peered":"2024-03-06T07:21:54.368398+0000","last_unstale":"2024-03-06T07:21:54.376984+0000","last_undegraded":"2024-03-06T07:21:54.376984+0000","last_fullsized":"2024-03-06T07:21:54.376984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:54.030393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369221+0000","last_change":"2024-03-06T07:21:43.324398+0000","last_active":"2024-03-06T07:21:54.369221+0000","last_peered":"2024-03-06T07:21:54.369221+0000","last_clean":"2024-03-06T07:21:54.369221+0000","last_became_active":"2024-03-06T07:21:43.324115+0000","last_became_peered":"2024-03-06T07:21:43.324115+0000","last_unstale":"2024-03-06T07:21:54.369221+0000","last_undegraded":"2024-03-06T07:21:54.369221+0000","last_fullsized":"2024-03-06T07:21:54.369221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:36.888023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365848+0000","last_change":"2024-03-06T07:21:42.312390+0000","last_active":"2024-03-06T07:21:54.365848+0000","last_peered":"2024-03-06T07:21:54.365848+0000","last_clean":"2024-03-06T07:21:54.365848+0000","last_became_active":"2024-03-06T07:21:42.312079+0000","last_became_peered":"2024-03-06T07:21:42.312079+0000","last_unstale":"2024-03-06T07:21:54.365848+0000","last_undegraded":"2024-03-06T07:21:54.365848+0000","last_fullsized":"2024-03-06T07:21:54.365848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.364238+0000","last_change":"2024-03-06T07:21:54.364238+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.364238+0000","last_undegraded":"2024-03-06T07:21:54.364238+0000","last_fullsized":"2024-03-06T07:21:54.364238+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.372534+0000","last_change":"2024-03-06T07:21:54.370872+0000","last_active":"2024-03-06T07:21:54.372534+0000","last_peered":"2024-03-06T07:21:54.372534+0000","last_clean":"2024-03-06T07:21:54.372534+0000","last_became_active":"2024-03-06T07:21:54.370589+0000","last_became_peered":"2024-03-06T07:21:54.370589+0000","last_unstale":"2024-03-06T07:21:54.372534+0000","last_undegraded":"2024-03-06T07:21:54.372534+0000","last_fullsized":"2024-03-06T07:21:54.372534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:36.087274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368542+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:54.368542+0000","last_peered":"2024-03-06T07:21:54.368542+0000","last_clean":"2024-03-06T07:21:54.368542+0000","last_became_active":"2024-03-06T07:21:43.316857+0000","last_became_peered":"2024-03-06T07:21:43.316857+0000","last_unstale":"2024-03-06T07:21:54.368542+0000","last_undegraded":"2024-03-06T07:21:54.368542+0000","last_fullsized":"2024-03-06T07:21:54.368542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:48.715734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365550+0000","last_change":"2024-03-06T07:21:42.311972+0000","last_active":"2024-03-06T07:21:54.365550+0000","last_peered":"2024-03-06T07:21:54.365550+0000","last_clean":"2024-03-06T07:21:54.365550+0000","last_became_active":"2024-03-06T07:21:42.311640+0000","last_became_peered":"2024-03-06T07:21:42.311640+0000","last_unstale":"2024-03-06T07:21:54.365550+0000","last_undegraded":"2024-03-06T07:21:54.365550+0000","last_fullsized":"2024-03-06T07:21:54.365550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:22.568258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.363210+0000","last_change":"2024-03-06T07:21:54.363210+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.363210+0000","last_undegraded":"2024-03-06T07:21:54.363210+0000","last_fullsized":"2024-03-06T07:21:54.363210+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.375823+0000","last_change":"2024-03-06T07:21:54.372740+0000","last_active":"2024-03-06T07:21:54.375823+0000","last_peered":"2024-03-06T07:21:54.375823+0000","last_clean":"2024-03-06T07:21:54.375823+0000","last_became_active":"2024-03-06T07:21:54.371525+0000","last_became_peered":"2024-03-06T07:21:54.371525+0000","last_unstale":"2024-03-06T07:21:54.375823+0000","last_undegraded":"2024-03-06T07:21:54.375823+0000","last_fullsized":"2024-03-06T07:21:54.375823+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:11.681237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359675+0000","last_change":"2024-03-06T07:21:43.320374+0000","last_active":"2024-03-06T07:21:54.359675+0000","last_peered":"2024-03-06T07:21:54.359675+0000","last_clean":"2024-03-06T07:21:54.359675+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.359675+0000","last_undegraded":"2024-03-06T07:21:54.359675+0000","last_fullsized":"2024-03-06T07:21:54.359675+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368677+0000","last_change":"2024-03-06T07:21:42.309901+0000","last_active":"2024-03-06T07:21:54.368677+0000","last_peered":"2024-03-06T07:21:54.368677+0000","last_clean":"2024-03-06T07:21:54.368677+0000","last_became_active":"2024-03-06T07:21:42.309532+0000","last_became_peered":"2024-03-06T07:21:42.309532+0000","last_unstale":"2024-03-06T07:21:54.368677+0000","last_undegraded":"2024-03-06T07:21:54.368677+0000","last_fullsized":"2024-03-06T07:21:54.368677+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361638+0000","last_change":"2024-03-06T07:21:54.361638+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361638+0000","last_undegraded":"2024-03-06T07:21:54.361638+0000","last_fullsized":"2024-03-06T07:21:54.361638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377134+0000","last_change":"2024-03-06T07:21:54.375852+0000","last_active":"2024-03-06T07:21:54.377134+0000","last_peered":"2024-03-06T07:21:54.377134+0000","last_clean":"2024-03-06T07:21:54.377134+0000","last_became_active":"2024-03-06T07:21:54.375600+0000","last_became_peered":"2024-03-06T07:21:54.375600+0000","last_unstale":"2024-03-06T07:21:54.377134+0000","last_undegraded":"2024-03-06T07:21:54.377134+0000","last_fullsized":"2024-03-06T07:21:54.377134+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:40.575618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365686+0000","last_change":"2024-03-06T07:21:43.320820+0000","last_active":"2024-03-06T07:21:54.365686+0000","last_peered":"2024-03-06T07:21:54.365686+0000","last_clean":"2024-03-06T07:21:54.365686+0000","last_became_active":"2024-03-06T07:21:43.319225+0000","last_became_peered":"2024-03-06T07:21:43.319225+0000","last_unstale":"2024-03-06T07:21:54.365686+0000","last_undegraded":"2024-03-06T07:21:54.365686+0000","last_fullsized":"2024-03-06T07:21:54.365686+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.358917+0000","last_change":"2024-03-06T07:21:42.307676+0000","last_active":"2024-03-06T07:21:54.358917+0000","last_peered":"2024-03-06T07:21:54.358917+0000","last_clean":"2024-03-06T07:21:54.358917+0000","last_became_active":"2024-03-06T07:21:42.307388+0000","last_became_peered":"2024-03-06T07:21:42.307388+0000","last_unstale":"2024-03-06T07:21:54.358917+0000","last_undegraded":"2024-03-06T07:21:54.358917+0000","last_fullsized":"2024-03-06T07:21:54.358917+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:47.711911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361080+0000","last_change":"2024-03-06T07:21:54.361080+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361080+0000","last_undegraded":"2024-03-06T07:21:54.361080+0000","last_fullsized":"2024-03-06T07:21:54.361080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.380012+0000","last_change":"2024-03-06T07:21:54.374823+0000","last_active":"2024-03-06T07:21:54.380012+0000","last_peered":"2024-03-06T07:21:54.380012+0000","last_clean":"2024-03-06T07:21:54.380012+0000","last_became_active":"2024-03-06T07:21:54.374515+0000","last_became_peered":"2024-03-06T07:21:54.374515+0000","last_unstale":"2024-03-06T07:21:54.380012+0000","last_undegraded":"2024-03-06T07:21:54.380012+0000","last_fullsized":"2024-03-06T07:21:54.380012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369756+0000","last_change":"2024-03-06T07:21:43.329841+0000","last_active":"2024-03-06T07:21:54.369756+0000","last_peered":"2024-03-06T07:21:54.369756+0000","last_clean":"2024-03-06T07:21:54.369756+0000","last_became_active":"2024-03-06T07:21:43.329526+0000","last_became_peered":"2024-03-06T07:21:43.329526+0000","last_unstale":"2024-03-06T07:21:54.369756+0000","last_undegraded":"2024-03-06T07:21:54.369756+0000","last_fullsized":"2024-03-06T07:21:54.369756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368782+0000","last_change":"2024-03-06T07:21:42.318168+0000","last_active":"2024-03-06T07:21:54.368782+0000","last_peered":"2024-03-06T07:21:54.368782+0000","last_clean":"2024-03-06T07:21:54.368782+0000","last_became_active":"2024-03-06T07:21:42.317866+0000","last_became_peered":"2024-03-06T07:21:42.317866+0000","last_unstale":"2024-03-06T07:21:54.368782+0000","last_undegraded":"2024-03-06T07:21:54.368782+0000","last_fullsized":"2024-03-06T07:21:54.368782+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:49.054000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361933+0000","last_change":"2024-03-06T07:21:54.361933+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361933+0000","last_undegraded":"2024-03-06T07:21:54.361933+0000","last_fullsized":"2024-03-06T07:21:54.361933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374694+0000","last_change":"2024-03-06T07:21:54.369742+0000","last_active":"2024-03-06T07:21:54.374694+0000","last_peered":"2024-03-06T07:21:54.374694+0000","last_clean":"2024-03-06T07:21:54.374694+0000","last_became_active":"2024-03-06T07:21:54.368872+0000","last_became_peered":"2024-03-06T07:21:54.368872+0000","last_unstale":"2024-03-06T07:21:54.374694+0000","last_undegraded":"2024-03-06T07:21:54.374694+0000","last_fullsized":"2024-03-06T07:21:54.374694+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:14.086987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373011+0000","last_change":"2024-03-06T07:21:43.316009+0000","last_active":"2024-03-06T07:21:54.373011+0000","last_peered":"2024-03-06T07:21:54.373011+0000","last_clean":"2024-03-06T07:21:54.373011+0000","last_became_active":"2024-03-06T07:21:43.315699+0000","last_became_peered":"2024-03-06T07:21:43.315699+0000","last_unstale":"2024-03-06T07:21:54.373011+0000","last_undegraded":"2024-03-06T07:21:54.373011+0000","last_fullsized":"2024-03-06T07:21:54.373011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:52.769462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365150+0000","last_change":"2024-03-06T07:21:42.311690+0000","last_active":"2024-03-06T07:21:54.365150+0000","last_peered":"2024-03-06T07:21:54.365150+0000","last_clean":"2024-03-06T07:21:54.365150+0000","last_became_active":"2024-03-06T07:21:42.311417+0000","last_became_peered":"2024-03-06T07:21:42.311417+0000","last_unstale":"2024-03-06T07:21:54.365150+0000","last_undegraded":"2024-03-06T07:21:54.365150+0000","last_fullsized":"2024-03-06T07:21:54.365150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.094878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362069+0000","last_change":"2024-03-06T07:21:54.362069+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362069+0000","last_undegraded":"2024-03-06T07:21:54.362069+0000","last_fullsized":"2024-03-06T07:21:54.362069+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.378181+0000","last_change":"2024-03-06T07:21:54.377550+0000","last_active":"2024-03-06T07:21:54.378181+0000","last_peered":"2024-03-06T07:21:54.378181+0000","last_clean":"2024-03-06T07:21:54.378181+0000","last_became_active":"2024-03-06T07:21:54.377271+0000","last_became_peered":"2024-03-06T07:21:54.377271+0000","last_unstale":"2024-03-06T07:21:54.378181+0000","last_undegraded":"2024-03-06T07:21:54.378181+0000","last_fullsized":"2024-03-06T07:21:54.378181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:48.675040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360360+0000","last_change":"2024-03-06T07:21:43.322229+0000","last_active":"2024-03-06T07:21:54.360360+0000","last_peered":"2024-03-06T07:21:54.360360+0000","last_clean":"2024-03-06T07:21:54.360360+0000","last_became_active":"2024-03-06T07:21:43.321934+0000","last_became_peered":"2024-03-06T07:21:43.321934+0000","last_unstale":"2024-03-06T07:21:54.360360+0000","last_undegraded":"2024-03-06T07:21:54.360360+0000","last_fullsized":"2024-03-06T07:21:54.360360+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.365379+0000","last_change":"2024-03-06T07:21:42.321714+0000","last_active":"2024-03-06T07:21:54.365379+0000","last_peered":"2024-03-06T07:21:54.365379+0000","last_clean":"2024-03-06T07:21:54.365379+0000","last_became_active":"2024-03-06T07:21:42.321444+0000","last_became_peered":"2024-03-06T07:21:42.321444+0000","last_unstale":"2024-03-06T07:21:54.365379+0000","last_undegraded":"2024-03-06T07:21:54.365379+0000","last_fullsized":"2024-03-06T07:21:54.365379+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361491+0000","last_change":"2024-03-06T07:21:54.361491+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361491+0000","last_undegraded":"2024-03-06T07:21:54.361491+0000","last_fullsized":"2024-03-06T07:21:54.361491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.373855+0000","last_change":"2024-03-06T07:21:54.371666+0000","last_active":"2024-03-06T07:21:54.373855+0000","last_peered":"2024-03-06T07:21:54.373855+0000","last_clean":"2024-03-06T07:21:54.373855+0000","last_became_active":"2024-03-06T07:21:54.371394+0000","last_became_peered":"2024-03-06T07:21:54.371394+0000","last_unstale":"2024-03-06T07:21:54.373855+0000","last_undegraded":"2024-03-06T07:21:54.373855+0000","last_fullsized":"2024-03-06T07:21:54.373855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:15.173188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.368609+0000","last_change":"2024-03-06T07:21:43.316403+0000","last_active":"2024-03-06T07:21:54.368609+0000","last_peered":"2024-03-06T07:21:54.368609+0000","last_clean":"2024-03-06T07:21:54.368609+0000","last_became_active":"2024-03-06T07:21:43.315416+0000","last_became_peered":"2024-03-06T07:21:43.315416+0000","last_unstale":"2024-03-06T07:21:54.368609+0000","last_undegraded":"2024-03-06T07:21:54.368609+0000","last_fullsized":"2024-03-06T07:21:54.368609+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:32.706894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359364+0000","last_change":"2024-03-06T07:21:42.311790+0000","last_active":"2024-03-06T07:21:54.359364+0000","last_peered":"2024-03-06T07:21:54.359364+0000","last_clean":"2024-03-06T07:21:54.359364+0000","last_became_active":"2024-03-06T07:21:42.311500+0000","last_became_peered":"2024-03-06T07:21:42.311500+0000","last_unstale":"2024-03-06T07:21:54.359364+0000","last_undegraded":"2024-03-06T07:21:54.359364+0000","last_fullsized":"2024-03-06T07:21:54.359364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358738+0000","last_change":"2024-03-06T07:21:54.358738+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358738+0000","last_undegraded":"2024-03-06T07:21:54.358738+0000","last_fullsized":"2024-03-06T07:21:54.358738+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.374655+0000","last_change":"2024-03-06T07:21:54.369593+0000","last_active":"2024-03-06T07:21:54.374655+0000","last_peered":"2024-03-06T07:21:54.374655+0000","last_clean":"2024-03-06T07:21:54.374655+0000","last_became_active":"2024-03-06T07:21:54.369336+0000","last_became_peered":"2024-03-06T07:21:54.369336+0000","last_unstale":"2024-03-06T07:21:54.374655+0000","last_undegraded":"2024-03-06T07:21:54.374655+0000","last_fullsized":"2024-03-06T07:21:54.374655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:35.519893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359566+0000","last_change":"2024-03-06T07:21:43.319244+0000","last_active":"2024-03-06T07:21:54.359566+0000","last_peered":"2024-03-06T07:21:54.359566+0000","last_clean":"2024-03-06T07:21:54.359566+0000","last_became_active":"2024-03-06T07:21:43.318994+0000","last_became_peered":"2024-03-06T07:21:43.318994+0000","last_unstale":"2024-03-06T07:21:54.359566+0000","last_undegraded":"2024-03-06T07:21:54.359566+0000","last_fullsized":"2024-03-06T07:21:54.359566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369178+0000","last_change":"2024-03-06T07:21:42.315718+0000","last_active":"2024-03-06T07:21:54.369178+0000","last_peered":"2024-03-06T07:21:54.369178+0000","last_clean":"2024-03-06T07:21:54.369178+0000","last_became_active":"2024-03-06T07:21:42.315402+0000","last_became_peered":"2024-03-06T07:21:42.315402+0000","last_unstale":"2024-03-06T07:21:54.369178+0000","last_undegraded":"2024-03-06T07:21:54.369178+0000","last_fullsized":"2024-03-06T07:21:54.369178+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.365483+0000","last_change":"2024-03-06T07:21:54.365483+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.365483+0000","last_undegraded":"2024-03-06T07:21:54.365483+0000","last_fullsized":"2024-03-06T07:21:54.365483+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379803+0000","last_change":"2024-03-06T07:21:54.375429+0000","last_active":"2024-03-06T07:21:54.379803+0000","last_peered":"2024-03-06T07:21:54.379803+0000","last_clean":"2024-03-06T07:21:54.379803+0000","last_became_active":"2024-03-06T07:21:54.375117+0000","last_became_peered":"2024-03-06T07:21:54.375117+0000","last_unstale":"2024-03-06T07:21:54.379803+0000","last_undegraded":"2024-03-06T07:21:54.379803+0000","last_fullsized":"2024-03-06T07:21:54.379803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:42.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369090+0000","last_change":"2024-03-06T07:21:43.323135+0000","last_active":"2024-03-06T07:21:54.369090+0000","last_peered":"2024-03-06T07:21:54.369090+0000","last_clean":"2024-03-06T07:21:54.369090+0000","last_became_active":"2024-03-06T07:21:43.322814+0000","last_became_peered":"2024-03-06T07:21:43.322814+0000","last_unstale":"2024-03-06T07:21:54.369090+0000","last_undegraded":"2024-03-06T07:21:54.369090+0000","last_fullsized":"2024-03-06T07:21:54.369090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:24.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359662+0000","last_change":"2024-03-06T07:21:42.308451+0000","last_active":"2024-03-06T07:21:54.359662+0000","last_peered":"2024-03-06T07:21:54.359662+0000","last_clean":"2024-03-06T07:21:54.359662+0000","last_became_active":"2024-03-06T07:21:42.308167+0000","last_became_peered":"2024-03-06T07:21:42.308167+0000","last_unstale":"2024-03-06T07:21:54.359662+0000","last_undegraded":"2024-03-06T07:21:54.359662+0000","last_fullsized":"2024-03-06T07:21:54.359662+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:31.703072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.362102+0000","last_change":"2024-03-06T07:21:54.362102+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.362102+0000","last_undegraded":"2024-03-06T07:21:54.362102+0000","last_fullsized":"2024-03-06T07:21:54.362102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.376688+0000","last_change":"2024-03-06T07:21:54.375869+0000","last_active":"2024-03-06T07:21:54.376688+0000","last_peered":"2024-03-06T07:21:54.376688+0000","last_clean":"2024-03-06T07:21:54.376688+0000","last_became_active":"2024-03-06T07:21:54.375586+0000","last_became_peered":"2024-03-06T07:21:54.375586+0000","last_unstale":"2024-03-06T07:21:54.376688+0000","last_undegraded":"2024-03-06T07:21:54.376688+0000","last_fullsized":"2024-03-06T07:21:54.376688+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:02.230230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356824+0000","last_change":"2024-03-06T07:21:43.324622+0000","last_active":"2024-03-06T07:21:54.356824+0000","last_peered":"2024-03-06T07:21:54.356824+0000","last_clean":"2024-03-06T07:21:54.356824+0000","last_became_active":"2024-03-06T07:21:43.324350+0000","last_became_peered":"2024-03-06T07:21:43.324350+0000","last_unstale":"2024-03-06T07:21:54.356824+0000","last_undegraded":"2024-03-06T07:21:54.356824+0000","last_fullsized":"2024-03-06T07:21:54.356824+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359015+0000","last_change":"2024-03-06T07:21:42.309246+0000","last_active":"2024-03-06T07:21:54.359015+0000","last_peered":"2024-03-06T07:21:54.359015+0000","last_clean":"2024-03-06T07:21:54.359015+0000","last_became_active":"2024-03-06T07:21:42.308990+0000","last_became_peered":"2024-03-06T07:21:42.308990+0000","last_unstale":"2024-03-06T07:21:54.359015+0000","last_undegraded":"2024-03-06T07:21:54.359015+0000","last_fullsized":"2024-03-06T07:21:54.359015+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:51.765639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.358506+0000","last_change":"2024-03-06T07:21:54.358506+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.358506+0000","last_undegraded":"2024-03-06T07:21:54.358506+0000","last_fullsized":"2024-03-06T07:21:54.358506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.377281+0000","last_change":"2024-03-06T07:21:54.371835+0000","last_active":"2024-03-06T07:21:54.377281+0000","last_peered":"2024-03-06T07:21:54.377281+0000","last_clean":"2024-03-06T07:21:54.377281+0000","last_became_active":"2024-03-06T07:21:54.371518+0000","last_became_peered":"2024-03-06T07:21:54.371518+0000","last_unstale":"2024-03-06T07:21:54.377281+0000","last_undegraded":"2024-03-06T07:21:54.377281+0000","last_fullsized":"2024-03-06T07:21:54.377281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:34.160892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369611+0000","last_change":"2024-03-06T07:21:43.329441+0000","last_active":"2024-03-06T07:21:54.369611+0000","last_peered":"2024-03-06T07:21:54.369611+0000","last_clean":"2024-03-06T07:21:54.369611+0000","last_became_active":"2024-03-06T07:21:43.327694+0000","last_became_peered":"2024-03-06T07:21:43.327694+0000","last_unstale":"2024-03-06T07:21:54.369611+0000","last_undegraded":"2024-03-06T07:21:54.369611+0000","last_fullsized":"2024-03-06T07:21:54.369611+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369282+0000","last_change":"2024-03-06T07:21:42.309614+0000","last_active":"2024-03-06T07:21:54.369282+0000","last_peered":"2024-03-06T07:21:54.369282+0000","last_clean":"2024-03-06T07:21:54.369282+0000","last_became_active":"2024-03-06T07:21:42.309319+0000","last_became_peered":"2024-03-06T07:21:42.309319+0000","last_unstale":"2024-03-06T07:21:54.369282+0000","last_undegraded":"2024-03-06T07:21:54.369282+0000","last_fullsized":"2024-03-06T07:21:54.369282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:28.385954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.360823+0000","last_change":"2024-03-06T07:21:54.360823+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.360823+0000","last_undegraded":"2024-03-06T07:21:54.360823+0000","last_fullsized":"2024-03-06T07:21:54.360823+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.379806+0000","last_change":"2024-03-06T07:21:54.375555+0000","last_active":"2024-03-06T07:21:54.379806+0000","last_peered":"2024-03-06T07:21:54.379806+0000","last_clean":"2024-03-06T07:21:54.379806+0000","last_became_active":"2024-03-06T07:21:54.375251+0000","last_became_peered":"2024-03-06T07:21:54.375251+0000","last_unstale":"2024-03-06T07:21:54.379806+0000","last_undegraded":"2024-03-06T07:21:54.379806+0000","last_fullsized":"2024-03-06T07:21:54.379806+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:01.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.359852+0000","last_change":"2024-03-06T07:21:43.321711+0000","last_active":"2024-03-06T07:21:54.359852+0000","last_peered":"2024-03-06T07:21:54.359852+0000","last_clean":"2024-03-06T07:21:54.359852+0000","last_became_active":"2024-03-06T07:21:43.321434+0000","last_became_peered":"2024-03-06T07:21:43.321434+0000","last_unstale":"2024-03-06T07:21:54.359852+0000","last_undegraded":"2024-03-06T07:21:54.359852+0000","last_fullsized":"2024-03-06T07:21:54.359852+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:06.150276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.360450+0000","last_change":"2024-03-06T07:21:42.310672+0000","last_active":"2024-03-06T07:21:54.360450+0000","last_peered":"2024-03-06T07:21:54.360450+0000","last_clean":"2024-03-06T07:21:54.360450+0000","last_became_active":"2024-03-06T07:21:42.310366+0000","last_became_peered":"2024-03-06T07:21:42.310366+0000","last_unstale":"2024-03-06T07:21:54.360450+0000","last_undegraded":"2024-03-06T07:21:54.360450+0000","last_fullsized":"2024-03-06T07:21:54.360450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:24.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-03-06T07:21:54.361405+0000","last_change":"2024-03-06T07:21:54.361405+0000","last_active":"2024-03-06T07:21:54.349222+0000","last_peered":"2024-03-06T07:21:54.349222+0000","last_clean":"2024-03-06T07:21:54.349222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:54.361405+0000","last_undegraded":"2024-03-06T07:21:54.361405+0000","last_fullsized":"2024-03-06T07:21:54.361405+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.367559+0000","last_change":"2024-03-06T07:21:54.366502+0000","last_active":"2024-03-06T07:21:54.367559+0000","last_peered":"2024-03-06T07:21:54.367559+0000","last_clean":"2024-03-06T07:21:54.367559+0000","last_became_active":"2024-03-06T07:21:54.366066+0000","last_became_peered":"2024-03-06T07:21:54.366066+0000","last_unstale":"2024-03-06T07:21:54.367559+0000","last_undegraded":"2024-03-06T07:21:54.367559+0000","last_fullsized":"2024-03-06T07:21:54.367559+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.651422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.364470+0000","last_change":"2024-03-06T07:21:43.319233+0000","last_active":"2024-03-06T07:21:54.364470+0000","last_peered":"2024-03-06T07:21:54.364470+0000","last_clean":"2024-03-06T07:21:54.364470+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:54.364470+0000","last_undegraded":"2024-03-06T07:21:54.364470+0000","last_fullsized":"2024-03-06T07:21:54.364470+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.369843+0000","last_change":"2024-03-06T07:21:42.322036+0000","last_active":"2024-03-06T07:21:54.369843+0000","last_peered":"2024-03-06T07:21:54.369843+0000","last_clean":"2024-03-06T07:21:54.369843+0000","last_became_active":"2024-03-06T07:21:42.321385+0000","last_became_peered":"2024-03-06T07:21:42.321385+0000","last_unstale":"2024-03-06T07:21:54.369843+0000","last_undegraded":"2024-03-06T07:21:54.369843+0000","last_fullsized":"2024-03-06T07:21:54.369843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-06T07:21:54.356636+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:54.356636+0000","last_peered":"2024-03-06T07:21:54.356636+0000","last_clean":"2024-03-06T07:21:54.356636+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:54.356636+0000","last_undegraded":"2024-03-06T07:21:54.356636+0000","last_fullsized":"2024-03-06T07:21:54.356636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":881653,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":651264,"data_stored":291813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":291161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":284171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":292795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":643072,"data_stored":285201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":291321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":874013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:03.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.991+0000 7fed2c071640 1 Processor -- start 2024-03-06T07:22:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.991+0000 7fed2c071640 1 -- start start 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2c071640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fed24155ba0 0x7fed240c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2c071640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed241591c0 con 0x7fed241586f0 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2c071640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed241594a0 con 0x7fed240c5480 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2c071640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2406df40 con 0x7fed24155ba0 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2a60e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fed241586f0 0x7fed2415f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56818/0 (socket says 172.21.15.26:56818) 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2a60e640 1 -- 172.21.15.26:0/3268487791 learned_addr learned my addr 172.21.15.26:0/3268487791 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2960c640 1 --2- 172.21.15.26:0/3268487791 >> v2:172.21.15.117:3300/0 conn(0x7fed24155ba0 0x7fed240c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed28e0b640 1 -- 172.21.15.26:0/3268487791 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570245905 0 0) 0x7fed241591c0 con 0x7fed241586f0 2024-03-06T07:22:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed28e0b640 1 -- 172.21.15.26:0/3268487791 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed10003520 con 0x7fed241586f0 2024-03-06T07:22:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed28e0b640 1 -- 172.21.15.26:0/3268487791 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964792628 0 0) 0x7fed241594a0 con 0x7fed240c5480 2024-03-06T07:22:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed28e0b640 1 -- 172.21.15.26:0/3268487791 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed241591c0 con 0x7fed240c5480 2024-03-06T07:22:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2960c640 1 -- 172.21.15.26:0/3268487791 >> v1:172.21.15.117:6789/0 conn(0x7fed240c5480 legacy=0x7fed2406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2960c640 1 -- 172.21.15.26:0/3268487791 >> v1:172.21.15.26:6789/0 conn(0x7fed241586f0 legacy=0x7fed2415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.992+0000 7fed2960c640 1 -- 172.21.15.26:0/3268487791 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed241553a0 con 0x7fed24155ba0 2024-03-06T07:22:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.993+0000 7fed2960c640 1 --2- 172.21.15.26:0/3268487791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed24155ba0 0x7fed240c5860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed200097a0 tx=0x7fed2002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34dfca2e4d26f67a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.993+0000 7fed28e0b640 1 -- 172.21.15.26:0/3268487791 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20035020 con 0x7fed24155ba0 2024-03-06T07:22:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.993+0000 7fed28e0b640 1 -- 172.21.15.26:0/3268487791 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed20031830 con 0x7fed24155ba0 2024-03-06T07:22:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.993+0000 7fed28e0b640 1 -- 172.21.15.26:0/3268487791 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031c00 con 0x7fed24155ba0 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.994+0000 7fed2c071640 1 -- 172.21.15.26:0/3268487791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed24155ba0 msgr2=0x7fed240c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.994+0000 7fed2c071640 1 --2- 172.21.15.26:0/3268487791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed24155ba0 0x7fed240c5860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed200097a0 tx=0x7fed2002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.994+0000 7fed2c071640 1 -- 172.21.15.26:0/3268487791 shutdown_connections 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.994+0000 7fed2c071640 1 --2- 172.21.15.26:0/3268487791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed24155ba0 0x7fed240c5860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.994+0000 7fed2c071640 1 -- 172.21.15.26:0/3268487791 >> 172.21.15.26:0/3268487791 conn(0x7fed2406b1a0 msgr2=0x7fed2406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.994+0000 7fed2c071640 1 -- 172.21.15.26:0/3268487791 shutdown_connections 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.994+0000 7fed2c071640 1 -- 172.21.15.26:0/3268487791 wait complete. 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.995+0000 7fed2c071640 1 Processor -- start 2024-03-06T07:22:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.995+0000 7fed2c071640 1 -- start start 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2c071640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed240c5480 0x7fed2414bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2c071640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed24155ba0 0x7fed240c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2c071640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed241586f0 0x7fed241ee140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2c071640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24156000 con 0x7fed240c5480 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2c071640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2406fdb0 con 0x7fed241586f0 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2c071640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed24152a20 con 0x7fed24155ba0 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed240c5480 0x7fed2414bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed240c5480 0x7fed2414bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57914/0 (socket says 172.21.15.26:57914) 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 -- 172.21.15.26:0/1928175704 learned_addr learned my addr 172.21.15.26:0/1928175704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2a60e640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed241586f0 0x7fed241ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed2960c640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed24155ba0 0x7fed240c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 -- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed24155ba0 msgr2=0x7fed240c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed24155ba0 0x7fed240c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 -- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed241586f0 msgr2=0x7fed241ee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed241586f0 0x7fed241ee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 -- 172.21.15.26:0/1928175704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2414c180 con 0x7fed240c5480 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed29e0d640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed240c5480 0x7fed2414bc40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fed1400c530 tx=0x7fed1400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.996+0000 7fed1a7fc640 1 -- 172.21.15.26:0/1928175704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140052d0 con 0x7fed240c5480 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.997+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed241ebaa0 con 0x7fed240c5480 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.997+0000 7fed1a7fc640 1 -- 172.21.15.26:0/1928175704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14005490 con 0x7fed240c5480 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.997+0000 7fed1a7fc640 1 -- 172.21.15.26:0/1928175704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14012870 con 0x7fed240c5480 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.997+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed241eace0 con 0x7fed240c5480 2024-03-06T07:22:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.998+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2404f330 con 0x7fed240c5480 2024-03-06T07:22:04.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.998+0000 7fed1a7fc640 1 -- 172.21.15.26:0/1928175704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed14012a10 con 0x7fed240c5480 2024-03-06T07:22:04.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.998+0000 7fed1a7fc640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.999+0000 7fed1a7fc640 1 -- 172.21.15.26:0/1928175704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7fed1408a210 con 0x7fed240c5480 2024-03-06T07:22:04.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.999+0000 7fed2960c640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:03.999+0000 7fed2960c640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fed20036000 tx=0x7fed20008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.003+0000 7fed1a7fc640 1 -- 172.21.15.26:0/1928175704 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed14058530 con 0x7fed240c5480 2024-03-06T07:22:04.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.184+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed2406e0c0 con 0x7fed0806b4d0 2024-03-06T07:22:04.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.195+0000 7fed1a7fc640 1 -- 172.21.15.26:0/1928175704 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490559 (secure 0 0 0) 0x7fed2406e0c0 con 0x7fed0806b4d0 2024-03-06T07:22:04.202 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:04.205 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.199+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 msgr2=0x7fed0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.199+0000 7fed2c071640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fed20036000 tx=0x7fed20008030 comp rx=0 tx=0).stop 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.199+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed240c5480 msgr2=0x7fed2414bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.199+0000 7fed2c071640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed240c5480 0x7fed2414bc40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fed1400c530 tx=0x7fed1400c9f0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 shutdown_connections 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed241586f0 0x7fed241ee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed24155ba0 0x7fed240c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 --2- 172.21.15.26:0/1928175704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed240c5480 0x7fed2414bc40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 >> 172.21.15.26:0/1928175704 conn(0x7fed2406b1a0 msgr2=0x7fed2406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 shutdown_connections 2024-03-06T07:22:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.200+0000 7fed2c071640 1 -- 172.21.15.26:0/1928175704 wait complete. 2024-03-06T07:22:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-06T07:22:03.054907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":375,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238292,"kb_used_data":5472,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736428,"statfs":{"total":773094113280,"available":772850102272,"internally_reserved":0,"allocated":5603328,"data_stored":3645184,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001695"},"pg_stats":[{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427035+0000","last_change":"2024-03-06T07:21:55.368683+0000","last_active":"2024-03-06T07:21:57.427035+0000","last_peered":"2024-03-06T07:21:57.427035+0000","last_clean":"2024-03-06T07:21:57.427035+0000","last_became_active":"2024-03-06T07:21:55.368401+0000","last_became_peered":"2024-03-06T07:21:55.368401+0000","last_unstale":"2024-03-06T07:21:57.427035+0000","last_undegraded":"2024-03-06T07:21:57.427035+0000","last_fullsized":"2024-03-06T07:21:57.427035+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:03.305740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359222+0000","last_change":"2024-03-06T07:21:43.317811+0000","last_active":"2024-03-06T07:21:56.359222+0000","last_peered":"2024-03-06T07:21:56.359222+0000","last_clean":"2024-03-06T07:21:56.359222+0000","last_became_active":"2024-03-06T07:21:43.317509+0000","last_became_peered":"2024-03-06T07:21:43.317509+0000","last_unstale":"2024-03-06T07:21:56.359222+0000","last_undegraded":"2024-03-06T07:21:56.359222+0000","last_fullsized":"2024-03-06T07:21:56.359222+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427388+0000","last_change":"2024-03-06T07:21:55.381597+0000","last_active":"2024-03-06T07:21:57.427388+0000","last_peered":"2024-03-06T07:21:57.427388+0000","last_clean":"2024-03-06T07:21:57.427388+0000","last_became_active":"2024-03-06T07:21:55.381325+0000","last_became_peered":"2024-03-06T07:21:55.381325+0000","last_unstale":"2024-03-06T07:21:57.427388+0000","last_undegraded":"2024-03-06T07:21:57.427388+0000","last_fullsized":"2024-03-06T07:21:57.427388+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:24:00.439098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427748+0000","last_change":"2024-03-06T07:21:43.317801+0000","last_active":"2024-03-06T07:21:57.427748+0000","last_peered":"2024-03-06T07:21:57.427748+0000","last_clean":"2024-03-06T07:21:57.427748+0000","last_became_active":"2024-03-06T07:21:43.317522+0000","last_became_peered":"2024-03-06T07:21:43.317522+0000","last_unstale":"2024-03-06T07:21:57.427748+0000","last_undegraded":"2024-03-06T07:21:57.427748+0000","last_fullsized":"2024-03-06T07:21:57.427748+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359039+0000","last_change":"2024-03-06T07:21:55.378779+0000","last_active":"2024-03-06T07:21:56.359039+0000","last_peered":"2024-03-06T07:21:56.359039+0000","last_clean":"2024-03-06T07:21:56.359039+0000","last_became_active":"2024-03-06T07:21:55.378515+0000","last_became_peered":"2024-03-06T07:21:55.378515+0000","last_unstale":"2024-03-06T07:21:56.359039+0000","last_undegraded":"2024-03-06T07:21:56.359039+0000","last_fullsized":"2024-03-06T07:21:56.359039+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:16.178688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427288+0000","last_change":"2024-03-06T07:21:43.328071+0000","last_active":"2024-03-06T07:21:57.427288+0000","last_peered":"2024-03-06T07:21:57.427288+0000","last_clean":"2024-03-06T07:21:57.427288+0000","last_became_active":"2024-03-06T07:21:43.327781+0000","last_became_peered":"2024-03-06T07:21:43.327781+0000","last_unstale":"2024-03-06T07:21:57.427288+0000","last_undegraded":"2024-03-06T07:21:57.427288+0000","last_fullsized":"2024-03-06T07:21:57.427288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359012+0000","last_change":"2024-03-06T07:21:55.367538+0000","last_active":"2024-03-06T07:21:56.359012+0000","last_peered":"2024-03-06T07:21:56.359012+0000","last_clean":"2024-03-06T07:21:56.359012+0000","last_became_active":"2024-03-06T07:21:55.367292+0000","last_became_peered":"2024-03-06T07:21:55.367292+0000","last_unstale":"2024-03-06T07:21:56.359012+0000","last_undegraded":"2024-03-06T07:21:56.359012+0000","last_fullsized":"2024-03-06T07:21:56.359012+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:31:50.197261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426810+0000","last_change":"2024-03-06T07:21:43.326286+0000","last_active":"2024-03-06T07:21:57.426810+0000","last_peered":"2024-03-06T07:21:57.426810+0000","last_clean":"2024-03-06T07:21:57.426810+0000","last_became_active":"2024-03-06T07:21:43.326031+0000","last_became_peered":"2024-03-06T07:21:43.326031+0000","last_unstale":"2024-03-06T07:21:57.426810+0000","last_undegraded":"2024-03-06T07:21:57.426810+0000","last_fullsized":"2024-03-06T07:21:57.426810+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:21.645487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380392+0000","last_change":"2024-03-06T07:21:55.378393+0000","last_active":"2024-03-06T07:21:56.380392+0000","last_peered":"2024-03-06T07:21:56.380392+0000","last_clean":"2024-03-06T07:21:56.380392+0000","last_became_active":"2024-03-06T07:21:55.378098+0000","last_became_peered":"2024-03-06T07:21:55.378098+0000","last_unstale":"2024-03-06T07:21:56.380392+0000","last_undegraded":"2024-03-06T07:21:56.380392+0000","last_fullsized":"2024-03-06T07:21:56.380392+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:22.765884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427073+0000","last_change":"2024-03-06T07:21:43.321810+0000","last_active":"2024-03-06T07:21:57.427073+0000","last_peered":"2024-03-06T07:21:57.427073+0000","last_clean":"2024-03-06T07:21:57.427073+0000","last_became_active":"2024-03-06T07:21:43.321538+0000","last_became_peered":"2024-03-06T07:21:43.321538+0000","last_unstale":"2024-03-06T07:21:57.427073+0000","last_undegraded":"2024-03-06T07:21:57.427073+0000","last_fullsized":"2024-03-06T07:21:57.427073+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:10.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377530+0000","last_change":"2024-03-06T07:21:55.381816+0000","last_active":"2024-03-06T07:21:56.377530+0000","last_peered":"2024-03-06T07:21:56.377530+0000","last_clean":"2024-03-06T07:21:56.377530+0000","last_became_active":"2024-03-06T07:21:55.381558+0000","last_became_peered":"2024-03-06T07:21:55.381558+0000","last_unstale":"2024-03-06T07:21:56.377530+0000","last_undegraded":"2024-03-06T07:21:56.377530+0000","last_fullsized":"2024-03-06T07:21:56.377530+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:03.235730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358431+0000","last_change":"2024-03-06T07:21:43.318713+0000","last_active":"2024-03-06T07:21:56.358431+0000","last_peered":"2024-03-06T07:21:56.358431+0000","last_clean":"2024-03-06T07:21:56.358431+0000","last_became_active":"2024-03-06T07:21:43.318385+0000","last_became_peered":"2024-03-06T07:21:43.318385+0000","last_unstale":"2024-03-06T07:21:56.358431+0000","last_undegraded":"2024-03-06T07:21:56.358431+0000","last_fullsized":"2024-03-06T07:21:56.358431+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362644+0000","last_change":"2024-03-06T07:21:55.385774+0000","last_active":"2024-03-06T07:21:56.362644+0000","last_peered":"2024-03-06T07:21:56.362644+0000","last_clean":"2024-03-06T07:21:56.362644+0000","last_became_active":"2024-03-06T07:21:55.384802+0000","last_became_peered":"2024-03-06T07:21:55.384802+0000","last_unstale":"2024-03-06T07:21:56.362644+0000","last_undegraded":"2024-03-06T07:21:56.362644+0000","last_fullsized":"2024-03-06T07:21:56.362644+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:02.802467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426661+0000","last_change":"2024-03-06T07:21:43.327099+0000","last_active":"2024-03-06T07:21:57.426661+0000","last_peered":"2024-03-06T07:21:57.426661+0000","last_clean":"2024-03-06T07:21:57.426661+0000","last_became_active":"2024-03-06T07:21:43.326851+0000","last_became_peered":"2024-03-06T07:21:43.326851+0000","last_unstale":"2024-03-06T07:21:57.426661+0000","last_undegraded":"2024-03-06T07:21:57.426661+0000","last_fullsized":"2024-03-06T07:21:57.426661+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426446+0000","last_change":"2024-03-06T07:21:55.379926+0000","last_active":"2024-03-06T07:21:57.426446+0000","last_peered":"2024-03-06T07:21:57.426446+0000","last_clean":"2024-03-06T07:21:57.426446+0000","last_became_active":"2024-03-06T07:21:55.379494+0000","last_became_peered":"2024-03-06T07:21:55.379494+0000","last_unstale":"2024-03-06T07:21:57.426446+0000","last_undegraded":"2024-03-06T07:21:57.426446+0000","last_fullsized":"2024-03-06T07:21:57.426446+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:26.299060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.425946+0000","last_change":"2024-03-06T07:21:43.325897+0000","last_active":"2024-03-06T07:21:57.425946+0000","last_peered":"2024-03-06T07:21:57.425946+0000","last_clean":"2024-03-06T07:21:57.425946+0000","last_became_active":"2024-03-06T07:21:43.325647+0000","last_became_peered":"2024-03-06T07:21:43.325647+0000","last_unstale":"2024-03-06T07:21:57.425946+0000","last_undegraded":"2024-03-06T07:21:57.425946+0000","last_fullsized":"2024-03-06T07:21:57.425946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427536+0000","last_change":"2024-03-06T07:21:55.383225+0000","last_active":"2024-03-06T07:21:57.427536+0000","last_peered":"2024-03-06T07:21:57.427536+0000","last_clean":"2024-03-06T07:21:57.427536+0000","last_became_active":"2024-03-06T07:21:55.382928+0000","last_became_peered":"2024-03-06T07:21:55.382928+0000","last_unstale":"2024-03-06T07:21:57.427536+0000","last_undegraded":"2024-03-06T07:21:57.427536+0000","last_fullsized":"2024-03-06T07:21:57.427536+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:55.035893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426614+0000","last_change":"2024-03-06T07:21:43.330337+0000","last_active":"2024-03-06T07:21:57.426614+0000","last_peered":"2024-03-06T07:21:57.426614+0000","last_clean":"2024-03-06T07:21:57.426614+0000","last_became_active":"2024-03-06T07:21:43.330053+0000","last_became_peered":"2024-03-06T07:21:43.330053+0000","last_unstale":"2024-03-06T07:21:57.426614+0000","last_undegraded":"2024-03-06T07:21:57.426614+0000","last_fullsized":"2024-03-06T07:21:57.426614+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377894+0000","last_change":"2024-03-06T07:21:55.381101+0000","last_active":"2024-03-06T07:21:56.377894+0000","last_peered":"2024-03-06T07:21:56.377894+0000","last_clean":"2024-03-06T07:21:56.377894+0000","last_became_active":"2024-03-06T07:21:55.380812+0000","last_became_peered":"2024-03-06T07:21:55.380812+0000","last_unstale":"2024-03-06T07:21:56.377894+0000","last_undegraded":"2024-03-06T07:21:56.377894+0000","last_fullsized":"2024-03-06T07:21:56.377894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:02.410442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359107+0000","last_change":"2024-03-06T07:21:43.325100+0000","last_active":"2024-03-06T07:21:56.359107+0000","last_peered":"2024-03-06T07:21:56.359107+0000","last_clean":"2024-03-06T07:21:56.359107+0000","last_became_active":"2024-03-06T07:21:43.324812+0000","last_became_peered":"2024-03-06T07:21:43.324812+0000","last_unstale":"2024-03-06T07:21:56.359107+0000","last_undegraded":"2024-03-06T07:21:56.359107+0000","last_fullsized":"2024-03-06T07:21:56.359107+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:11.442368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427360+0000","last_change":"2024-03-06T07:21:55.375012+0000","last_active":"2024-03-06T07:21:57.427360+0000","last_peered":"2024-03-06T07:21:57.427360+0000","last_clean":"2024-03-06T07:21:57.427360+0000","last_became_active":"2024-03-06T07:21:55.374733+0000","last_became_peered":"2024-03-06T07:21:55.374733+0000","last_unstale":"2024-03-06T07:21:57.427360+0000","last_undegraded":"2024-03-06T07:21:57.427360+0000","last_fullsized":"2024-03-06T07:21:57.427360+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:37.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380988+0000","last_change":"2024-03-06T07:21:43.322188+0000","last_active":"2024-03-06T07:21:56.380988+0000","last_peered":"2024-03-06T07:21:56.380988+0000","last_clean":"2024-03-06T07:21:56.380988+0000","last_became_active":"2024-03-06T07:21:43.321685+0000","last_became_peered":"2024-03-06T07:21:43.321685+0000","last_unstale":"2024-03-06T07:21:56.380988+0000","last_undegraded":"2024-03-06T07:21:56.380988+0000","last_fullsized":"2024-03-06T07:21:56.380988+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:54.799043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427593+0000","last_change":"2024-03-06T07:21:55.372234+0000","last_active":"2024-03-06T07:21:57.427593+0000","last_peered":"2024-03-06T07:21:57.427593+0000","last_clean":"2024-03-06T07:21:57.427593+0000","last_became_active":"2024-03-06T07:21:55.371951+0000","last_became_peered":"2024-03-06T07:21:55.371951+0000","last_unstale":"2024-03-06T07:21:57.427593+0000","last_undegraded":"2024-03-06T07:21:57.427593+0000","last_fullsized":"2024-03-06T07:21:57.427593+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:45.481910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427541+0000","last_change":"2024-03-06T07:21:43.321952+0000","last_active":"2024-03-06T07:21:57.427541+0000","last_peered":"2024-03-06T07:21:57.427541+0000","last_clean":"2024-03-06T07:21:57.427541+0000","last_became_active":"2024-03-06T07:21:43.321579+0000","last_became_peered":"2024-03-06T07:21:43.321579+0000","last_unstale":"2024-03-06T07:21:57.427541+0000","last_undegraded":"2024-03-06T07:21:57.427541+0000","last_fullsized":"2024-03-06T07:21:57.427541+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:04.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426446+0000","last_change":"2024-03-06T07:21:55.364543+0000","last_active":"2024-03-06T07:21:57.426446+0000","last_peered":"2024-03-06T07:21:57.426446+0000","last_clean":"2024-03-06T07:21:57.426446+0000","last_became_active":"2024-03-06T07:21:55.364212+0000","last_became_peered":"2024-03-06T07:21:55.364212+0000","last_unstale":"2024-03-06T07:21:57.426446+0000","last_undegraded":"2024-03-06T07:21:57.426446+0000","last_fullsized":"2024-03-06T07:21:57.426446+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:50.467606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358385+0000","last_change":"2024-03-06T07:21:43.321277+0000","last_active":"2024-03-06T07:21:56.358385+0000","last_peered":"2024-03-06T07:21:56.358385+0000","last_clean":"2024-03-06T07:21:56.358385+0000","last_became_active":"2024-03-06T07:21:43.320973+0000","last_became_peered":"2024-03-06T07:21:43.320973+0000","last_unstale":"2024-03-06T07:21:56.358385+0000","last_undegraded":"2024-03-06T07:21:56.358385+0000","last_fullsized":"2024-03-06T07:21:56.358385+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:15.957299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380282+0000","last_change":"2024-03-06T07:21:55.376130+0000","last_active":"2024-03-06T07:21:56.380282+0000","last_peered":"2024-03-06T07:21:56.380282+0000","last_clean":"2024-03-06T07:21:56.380282+0000","last_became_active":"2024-03-06T07:21:55.375862+0000","last_became_peered":"2024-03-06T07:21:55.375862+0000","last_unstale":"2024-03-06T07:21:56.380282+0000","last_undegraded":"2024-03-06T07:21:56.380282+0000","last_fullsized":"2024-03-06T07:21:56.380282+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:21.868917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362056+0000","last_change":"2024-03-06T07:21:50.373762+0000","last_active":"2024-03-06T07:21:56.362056+0000","last_peered":"2024-03-06T07:21:56.362056+0000","last_clean":"2024-03-06T07:21:56.362056+0000","last_became_active":"2024-03-06T07:21:50.373521+0000","last_became_peered":"2024-03-06T07:21:50.373521+0000","last_unstale":"2024-03-06T07:21:56.362056+0000","last_undegraded":"2024-03-06T07:21:56.362056+0000","last_fullsized":"2024-03-06T07:21:56.362056+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:40.261647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377939+0000","last_change":"2024-03-06T07:21:55.387156+0000","last_active":"2024-03-06T07:21:56.377939+0000","last_peered":"2024-03-06T07:21:56.377939+0000","last_clean":"2024-03-06T07:21:56.377939+0000","last_became_active":"2024-03-06T07:21:55.386892+0000","last_became_peered":"2024-03-06T07:21:55.386892+0000","last_unstale":"2024-03-06T07:21:56.377939+0000","last_undegraded":"2024-03-06T07:21:56.377939+0000","last_fullsized":"2024-03-06T07:21:56.377939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:10.249527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426533+0000","last_change":"2024-03-06T07:21:43.321137+0000","last_active":"2024-03-06T07:21:57.426533+0000","last_peered":"2024-03-06T07:21:57.426533+0000","last_clean":"2024-03-06T07:21:57.426533+0000","last_became_active":"2024-03-06T07:21:43.320820+0000","last_became_peered":"2024-03-06T07:21:43.320820+0000","last_unstale":"2024-03-06T07:21:57.426533+0000","last_undegraded":"2024-03-06T07:21:57.426533+0000","last_fullsized":"2024-03-06T07:21:57.426533+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:52.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426551+0000","last_change":"2024-03-06T07:21:55.372956+0000","last_active":"2024-03-06T07:21:57.426551+0000","last_peered":"2024-03-06T07:21:57.426551+0000","last_clean":"2024-03-06T07:21:57.426551+0000","last_became_active":"2024-03-06T07:21:55.371896+0000","last_became_peered":"2024-03-06T07:21:55.371896+0000","last_unstale":"2024-03-06T07:21:57.426551+0000","last_undegraded":"2024-03-06T07:21:57.426551+0000","last_fullsized":"2024-03-06T07:21:57.426551+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:02.410442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358045+0000","last_change":"2024-03-06T07:21:43.320518+0000","last_active":"2024-03-06T07:21:56.358045+0000","last_peered":"2024-03-06T07:21:56.358045+0000","last_clean":"2024-03-06T07:21:56.358045+0000","last_became_active":"2024-03-06T07:21:43.320178+0000","last_became_peered":"2024-03-06T07:21:43.320178+0000","last_unstale":"2024-03-06T07:21:56.358045+0000","last_undegraded":"2024-03-06T07:21:56.358045+0000","last_fullsized":"2024-03-06T07:21:56.358045+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362992+0000","last_change":"2024-03-06T07:21:55.376765+0000","last_active":"2024-03-06T07:21:56.362992+0000","last_peered":"2024-03-06T07:21:56.362992+0000","last_clean":"2024-03-06T07:21:56.362992+0000","last_became_active":"2024-03-06T07:21:55.376475+0000","last_became_peered":"2024-03-06T07:21:55.376475+0000","last_unstale":"2024-03-06T07:21:56.362992+0000","last_undegraded":"2024-03-06T07:21:56.362992+0000","last_fullsized":"2024-03-06T07:21:56.362992+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:00.129312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427738+0000","last_change":"2024-03-06T07:21:43.324279+0000","last_active":"2024-03-06T07:21:57.427738+0000","last_peered":"2024-03-06T07:21:57.427738+0000","last_clean":"2024-03-06T07:21:57.427738+0000","last_became_active":"2024-03-06T07:21:43.323499+0000","last_became_peered":"2024-03-06T07:21:43.323499+0000","last_unstale":"2024-03-06T07:21:57.427738+0000","last_undegraded":"2024-03-06T07:21:57.427738+0000","last_fullsized":"2024-03-06T07:21:57.427738+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427226+0000","last_change":"2024-03-06T07:21:55.377543+0000","last_active":"2024-03-06T07:21:57.427226+0000","last_peered":"2024-03-06T07:21:57.427226+0000","last_clean":"2024-03-06T07:21:57.427226+0000","last_became_active":"2024-03-06T07:21:55.377293+0000","last_became_peered":"2024-03-06T07:21:55.377293+0000","last_unstale":"2024-03-06T07:21:57.427226+0000","last_undegraded":"2024-03-06T07:21:57.427226+0000","last_fullsized":"2024-03-06T07:21:57.427226+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:25.966338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427782+0000","last_change":"2024-03-06T07:21:43.318202+0000","last_active":"2024-03-06T07:21:57.427782+0000","last_peered":"2024-03-06T07:21:57.427782+0000","last_clean":"2024-03-06T07:21:57.427782+0000","last_became_active":"2024-03-06T07:21:43.317929+0000","last_became_peered":"2024-03-06T07:21:43.317929+0000","last_unstale":"2024-03-06T07:21:57.427782+0000","last_undegraded":"2024-03-06T07:21:57.427782+0000","last_fullsized":"2024-03-06T07:21:57.427782+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359039+0000","last_change":"2024-03-06T07:21:55.374902+0000","last_active":"2024-03-06T07:21:56.359039+0000","last_peered":"2024-03-06T07:21:56.359039+0000","last_clean":"2024-03-06T07:21:56.359039+0000","last_became_active":"2024-03-06T07:21:55.374557+0000","last_became_peered":"2024-03-06T07:21:55.374557+0000","last_unstale":"2024-03-06T07:21:56.359039+0000","last_undegraded":"2024-03-06T07:21:56.359039+0000","last_fullsized":"2024-03-06T07:21:56.359039+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:12.686737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427369+0000","last_change":"2024-03-06T07:21:43.324265+0000","last_active":"2024-03-06T07:21:57.427369+0000","last_peered":"2024-03-06T07:21:57.427369+0000","last_clean":"2024-03-06T07:21:57.427369+0000","last_became_active":"2024-03-06T07:21:43.323877+0000","last_became_peered":"2024-03-06T07:21:43.323877+0000","last_unstale":"2024-03-06T07:21:57.427369+0000","last_undegraded":"2024-03-06T07:21:57.427369+0000","last_fullsized":"2024-03-06T07:21:57.427369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:56.043649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427483+0000","last_change":"2024-03-06T07:21:55.384160+0000","last_active":"2024-03-06T07:21:57.427483+0000","last_peered":"2024-03-06T07:21:57.427483+0000","last_clean":"2024-03-06T07:21:57.427483+0000","last_became_active":"2024-03-06T07:21:55.383305+0000","last_became_peered":"2024-03-06T07:21:55.383305+0000","last_unstale":"2024-03-06T07:21:57.427483+0000","last_undegraded":"2024-03-06T07:21:57.427483+0000","last_fullsized":"2024-03-06T07:21:57.427483+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:36.525393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.363050+0000","last_change":"2024-03-06T07:21:43.321580+0000","last_active":"2024-03-06T07:21:56.363050+0000","last_peered":"2024-03-06T07:21:56.363050+0000","last_clean":"2024-03-06T07:21:56.363050+0000","last_became_active":"2024-03-06T07:21:43.319775+0000","last_became_peered":"2024-03-06T07:21:43.319775+0000","last_unstale":"2024-03-06T07:21:56.363050+0000","last_undegraded":"2024-03-06T07:21:56.363050+0000","last_fullsized":"2024-03-06T07:21:56.363050+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:32.802384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426061+0000","last_change":"2024-03-06T07:21:55.375487+0000","last_active":"2024-03-06T07:21:57.426061+0000","last_peered":"2024-03-06T07:21:57.426061+0000","last_clean":"2024-03-06T07:21:57.426061+0000","last_became_active":"2024-03-06T07:21:55.375244+0000","last_became_peered":"2024-03-06T07:21:55.375244+0000","last_unstale":"2024-03-06T07:21:57.426061+0000","last_undegraded":"2024-03-06T07:21:57.426061+0000","last_fullsized":"2024-03-06T07:21:57.426061+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:26.299060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362888+0000","last_change":"2024-03-06T07:21:43.322797+0000","last_active":"2024-03-06T07:21:56.362888+0000","last_peered":"2024-03-06T07:21:56.362888+0000","last_clean":"2024-03-06T07:21:56.362888+0000","last_became_active":"2024-03-06T07:21:43.322505+0000","last_became_peered":"2024-03-06T07:21:43.322505+0000","last_unstale":"2024-03-06T07:21:56.362888+0000","last_undegraded":"2024-03-06T07:21:56.362888+0000","last_fullsized":"2024-03-06T07:21:56.362888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426071+0000","last_change":"2024-03-06T07:21:55.373687+0000","last_active":"2024-03-06T07:21:57.426071+0000","last_peered":"2024-03-06T07:21:57.426071+0000","last_clean":"2024-03-06T07:21:57.426071+0000","last_became_active":"2024-03-06T07:21:55.373418+0000","last_became_peered":"2024-03-06T07:21:55.373418+0000","last_unstale":"2024-03-06T07:21:57.426071+0000","last_undegraded":"2024-03-06T07:21:57.426071+0000","last_fullsized":"2024-03-06T07:21:57.426071+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:10:53.122484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427018+0000","last_change":"2024-03-06T07:21:43.326058+0000","last_active":"2024-03-06T07:21:57.427018+0000","last_peered":"2024-03-06T07:21:57.427018+0000","last_clean":"2024-03-06T07:21:57.427018+0000","last_became_active":"2024-03-06T07:21:43.325799+0000","last_became_peered":"2024-03-06T07:21:43.325799+0000","last_unstale":"2024-03-06T07:21:57.427018+0000","last_undegraded":"2024-03-06T07:21:57.427018+0000","last_fullsized":"2024-03-06T07:21:57.427018+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377814+0000","last_change":"2024-03-06T07:21:55.376114+0000","last_active":"2024-03-06T07:21:56.377814+0000","last_peered":"2024-03-06T07:21:56.377814+0000","last_clean":"2024-03-06T07:21:56.377814+0000","last_became_active":"2024-03-06T07:21:55.375759+0000","last_became_peered":"2024-03-06T07:21:55.375759+0000","last_unstale":"2024-03-06T07:21:56.377814+0000","last_undegraded":"2024-03-06T07:21:56.377814+0000","last_fullsized":"2024-03-06T07:21:56.377814+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:31.023717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380441+0000","last_change":"2024-03-06T07:21:43.322671+0000","last_active":"2024-03-06T07:21:56.380441+0000","last_peered":"2024-03-06T07:21:56.380441+0000","last_clean":"2024-03-06T07:21:56.380441+0000","last_became_active":"2024-03-06T07:21:43.322376+0000","last_became_peered":"2024-03-06T07:21:43.322376+0000","last_unstale":"2024-03-06T07:21:56.380441+0000","last_undegraded":"2024-03-06T07:21:56.380441+0000","last_fullsized":"2024-03-06T07:21:56.380441+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:15.049663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426755+0000","last_change":"2024-03-06T07:21:55.375462+0000","last_active":"2024-03-06T07:21:57.426755+0000","last_peered":"2024-03-06T07:21:57.426755+0000","last_clean":"2024-03-06T07:21:57.426755+0000","last_became_active":"2024-03-06T07:21:55.375214+0000","last_became_peered":"2024-03-06T07:21:55.375214+0000","last_unstale":"2024-03-06T07:21:57.426755+0000","last_undegraded":"2024-03-06T07:21:57.426755+0000","last_fullsized":"2024-03-06T07:21:57.426755+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:25.479299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426131+0000","last_change":"2024-03-06T07:21:43.327014+0000","last_active":"2024-03-06T07:21:57.426131+0000","last_peered":"2024-03-06T07:21:57.426131+0000","last_clean":"2024-03-06T07:21:57.426131+0000","last_became_active":"2024-03-06T07:21:43.326759+0000","last_became_peered":"2024-03-06T07:21:43.326759+0000","last_unstale":"2024-03-06T07:21:57.426131+0000","last_undegraded":"2024-03-06T07:21:57.426131+0000","last_fullsized":"2024-03-06T07:21:57.426131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427382+0000","last_change":"2024-03-06T07:21:55.385098+0000","last_active":"2024-03-06T07:21:57.427382+0000","last_peered":"2024-03-06T07:21:57.427382+0000","last_clean":"2024-03-06T07:21:57.427382+0000","last_became_active":"2024-03-06T07:21:55.384416+0000","last_became_peered":"2024-03-06T07:21:55.384416+0000","last_unstale":"2024-03-06T07:21:57.427382+0000","last_undegraded":"2024-03-06T07:21:57.427382+0000","last_fullsized":"2024-03-06T07:21:57.427382+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:00.667138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427852+0000","last_change":"2024-03-06T07:21:43.323199+0000","last_active":"2024-03-06T07:21:57.427852+0000","last_peered":"2024-03-06T07:21:57.427852+0000","last_clean":"2024-03-06T07:21:57.427852+0000","last_became_active":"2024-03-06T07:21:43.322835+0000","last_became_peered":"2024-03-06T07:21:43.322835+0000","last_unstale":"2024-03-06T07:21:57.427852+0000","last_undegraded":"2024-03-06T07:21:57.427852+0000","last_fullsized":"2024-03-06T07:21:57.427852+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358592+0000","last_change":"2024-03-06T07:21:55.383496+0000","last_active":"2024-03-06T07:21:56.358592+0000","last_peered":"2024-03-06T07:21:56.358592+0000","last_clean":"2024-03-06T07:21:56.358592+0000","last_became_active":"2024-03-06T07:21:55.383214+0000","last_became_peered":"2024-03-06T07:21:55.383214+0000","last_unstale":"2024-03-06T07:21:56.358592+0000","last_undegraded":"2024-03-06T07:21:56.358592+0000","last_fullsized":"2024-03-06T07:21:56.358592+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:18.208270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426188+0000","last_change":"2024-03-06T07:21:43.318509+0000","last_active":"2024-03-06T07:21:57.426188+0000","last_peered":"2024-03-06T07:21:57.426188+0000","last_clean":"2024-03-06T07:21:57.426188+0000","last_became_active":"2024-03-06T07:21:43.318177+0000","last_became_peered":"2024-03-06T07:21:43.318177+0000","last_unstale":"2024-03-06T07:21:57.426188+0000","last_undegraded":"2024-03-06T07:21:57.426188+0000","last_fullsized":"2024-03-06T07:21:57.426188+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:57.560184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378320+0000","last_change":"2024-03-06T07:21:55.378412+0000","last_active":"2024-03-06T07:21:56.378320+0000","last_peered":"2024-03-06T07:21:56.378320+0000","last_clean":"2024-03-06T07:21:56.378320+0000","last_became_active":"2024-03-06T07:21:55.378103+0000","last_became_peered":"2024-03-06T07:21:55.378103+0000","last_unstale":"2024-03-06T07:21:56.378320+0000","last_undegraded":"2024-03-06T07:21:56.378320+0000","last_fullsized":"2024-03-06T07:21:56.378320+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:16.339384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359199+0000","last_change":"2024-03-06T07:21:43.332733+0000","last_active":"2024-03-06T07:21:56.359199+0000","last_peered":"2024-03-06T07:21:56.359199+0000","last_clean":"2024-03-06T07:21:56.359199+0000","last_became_active":"2024-03-06T07:21:43.331512+0000","last_became_peered":"2024-03-06T07:21:43.331512+0000","last_unstale":"2024-03-06T07:21:56.359199+0000","last_undegraded":"2024-03-06T07:21:56.359199+0000","last_fullsized":"2024-03-06T07:21:56.359199+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378221+0000","last_change":"2024-03-06T07:21:55.377695+0000","last_active":"2024-03-06T07:21:56.378221+0000","last_peered":"2024-03-06T07:21:56.378221+0000","last_clean":"2024-03-06T07:21:56.378221+0000","last_became_active":"2024-03-06T07:21:55.376652+0000","last_became_peered":"2024-03-06T07:21:55.376652+0000","last_unstale":"2024-03-06T07:21:56.378221+0000","last_undegraded":"2024-03-06T07:21:56.378221+0000","last_fullsized":"2024-03-06T07:21:56.378221+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:03.305740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.363149+0000","last_change":"2024-03-06T07:21:43.321054+0000","last_active":"2024-03-06T07:21:56.363149+0000","last_peered":"2024-03-06T07:21:56.363149+0000","last_clean":"2024-03-06T07:21:56.363149+0000","last_became_active":"2024-03-06T07:21:43.319435+0000","last_became_peered":"2024-03-06T07:21:43.319435+0000","last_unstale":"2024-03-06T07:21:56.363149+0000","last_undegraded":"2024-03-06T07:21:56.363149+0000","last_fullsized":"2024-03-06T07:21:56.363149+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:52.397876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426134+0000","last_change":"2024-03-06T07:21:55.380858+0000","last_active":"2024-03-06T07:21:57.426134+0000","last_peered":"2024-03-06T07:21:57.426134+0000","last_clean":"2024-03-06T07:21:57.426134+0000","last_became_active":"2024-03-06T07:21:55.380524+0000","last_became_peered":"2024-03-06T07:21:55.380524+0000","last_unstale":"2024-03-06T07:21:57.426134+0000","last_undegraded":"2024-03-06T07:21:57.426134+0000","last_fullsized":"2024-03-06T07:21:57.426134+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:14.131143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426679+0000","last_change":"2024-03-06T07:21:43.328812+0000","last_active":"2024-03-06T07:21:57.426679+0000","last_peered":"2024-03-06T07:21:57.426679+0000","last_clean":"2024-03-06T07:21:57.426679+0000","last_became_active":"2024-03-06T07:21:43.328366+0000","last_became_peered":"2024-03-06T07:21:43.328366+0000","last_unstale":"2024-03-06T07:21:57.426679+0000","last_undegraded":"2024-03-06T07:21:57.426679+0000","last_fullsized":"2024-03-06T07:21:57.426679+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:48.381105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.425962+0000","last_change":"2024-03-06T07:21:55.368513+0000","last_active":"2024-03-06T07:21:57.425962+0000","last_peered":"2024-03-06T07:21:57.425962+0000","last_clean":"2024-03-06T07:21:57.425962+0000","last_became_active":"2024-03-06T07:21:55.368278+0000","last_became_peered":"2024-03-06T07:21:55.368278+0000","last_unstale":"2024-03-06T07:21:57.425962+0000","last_undegraded":"2024-03-06T07:21:57.425962+0000","last_fullsized":"2024-03-06T07:21:57.425962+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:58.084878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426639+0000","last_change":"2024-03-06T07:21:43.321155+0000","last_active":"2024-03-06T07:21:57.426639+0000","last_peered":"2024-03-06T07:21:57.426639+0000","last_clean":"2024-03-06T07:21:57.426639+0000","last_became_active":"2024-03-06T07:21:43.320747+0000","last_became_peered":"2024-03-06T07:21:43.320747+0000","last_unstale":"2024-03-06T07:21:57.426639+0000","last_undegraded":"2024-03-06T07:21:57.426639+0000","last_fullsized":"2024-03-06T07:21:57.426639+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:42.424597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426568+0000","last_change":"2024-03-06T07:21:55.383099+0000","last_active":"2024-03-06T07:21:57.426568+0000","last_peered":"2024-03-06T07:21:57.426568+0000","last_clean":"2024-03-06T07:21:57.426568+0000","last_became_active":"2024-03-06T07:21:55.382786+0000","last_became_peered":"2024-03-06T07:21:55.382786+0000","last_unstale":"2024-03-06T07:21:57.426568+0000","last_undegraded":"2024-03-06T07:21:57.426568+0000","last_fullsized":"2024-03-06T07:21:57.426568+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:31.023717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426695+0000","last_change":"2024-03-06T07:21:43.326550+0000","last_active":"2024-03-06T07:21:57.426695+0000","last_peered":"2024-03-06T07:21:57.426695+0000","last_clean":"2024-03-06T07:21:57.426695+0000","last_became_active":"2024-03-06T07:21:43.326254+0000","last_became_peered":"2024-03-06T07:21:43.326254+0000","last_unstale":"2024-03-06T07:21:57.426695+0000","last_undegraded":"2024-03-06T07:21:57.426695+0000","last_fullsized":"2024-03-06T07:21:57.426695+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377806+0000","last_change":"2024-03-06T07:21:55.377610+0000","last_active":"2024-03-06T07:21:56.377806+0000","last_peered":"2024-03-06T07:21:56.377806+0000","last_clean":"2024-03-06T07:21:56.377806+0000","last_became_active":"2024-03-06T07:21:55.377344+0000","last_became_peered":"2024-03-06T07:21:55.377344+0000","last_unstale":"2024-03-06T07:21:56.377806+0000","last_undegraded":"2024-03-06T07:21:56.377806+0000","last_fullsized":"2024-03-06T07:21:56.377806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:50.467606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377698+0000","last_change":"2024-03-06T07:21:43.322773+0000","last_active":"2024-03-06T07:21:56.377698+0000","last_peered":"2024-03-06T07:21:56.377698+0000","last_clean":"2024-03-06T07:21:56.377698+0000","last_became_active":"2024-03-06T07:21:43.322525+0000","last_became_peered":"2024-03-06T07:21:43.322525+0000","last_unstale":"2024-03-06T07:21:56.377698+0000","last_undegraded":"2024-03-06T07:21:56.377698+0000","last_fullsized":"2024-03-06T07:21:56.377698+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362197+0000","last_change":"2024-03-06T07:21:55.379206+0000","last_active":"2024-03-06T07:21:56.362197+0000","last_peered":"2024-03-06T07:21:56.362197+0000","last_clean":"2024-03-06T07:21:56.362197+0000","last_became_active":"2024-03-06T07:21:55.378938+0000","last_became_peered":"2024-03-06T07:21:55.378938+0000","last_unstale":"2024-03-06T07:21:56.362197+0000","last_undegraded":"2024-03-06T07:21:56.362197+0000","last_fullsized":"2024-03-06T07:21:56.362197+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:36.525393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426072+0000","last_change":"2024-03-06T07:21:54.366905+0000","last_active":"2024-03-06T07:21:57.426072+0000","last_peered":"2024-03-06T07:21:57.426072+0000","last_clean":"2024-03-06T07:21:57.426072+0000","last_became_active":"2024-03-06T07:21:54.366627+0000","last_became_peered":"2024-03-06T07:21:54.366627+0000","last_unstale":"2024-03-06T07:21:57.426072+0000","last_undegraded":"2024-03-06T07:21:57.426072+0000","last_fullsized":"2024-03-06T07:21:57.426072+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:07.096142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426183+0000","last_change":"2024-03-06T07:21:43.319837+0000","last_active":"2024-03-06T07:21:57.426183+0000","last_peered":"2024-03-06T07:21:57.426183+0000","last_clean":"2024-03-06T07:21:57.426183+0000","last_became_active":"2024-03-06T07:21:43.319564+0000","last_became_peered":"2024-03-06T07:21:43.319564+0000","last_unstale":"2024-03-06T07:21:57.426183+0000","last_undegraded":"2024-03-06T07:21:57.426183+0000","last_fullsized":"2024-03-06T07:21:57.426183+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:35.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":30,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362958+0000","last_change":"2024-03-06T07:21:42.310584+0000","last_active":"2024-03-06T07:21:56.362958+0000","last_peered":"2024-03-06T07:21:56.362958+0000","last_clean":"2024-03-06T07:21:56.362958+0000","last_became_active":"2024-03-06T07:21:42.310317+0000","last_became_peered":"2024-03-06T07:21:42.310317+0000","last_unstale":"2024-03-06T07:21:56.362958+0000","last_undegraded":"2024-03-06T07:21:56.362958+0000","last_fullsized":"2024-03-06T07:21:56.362958+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358529+0000","last_change":"2024-03-06T07:21:55.374014+0000","last_active":"2024-03-06T07:21:56.358529+0000","last_peered":"2024-03-06T07:21:56.358529+0000","last_clean":"2024-03-06T07:21:56.358529+0000","last_became_active":"2024-03-06T07:21:55.373763+0000","last_became_peered":"2024-03-06T07:21:55.373763+0000","last_unstale":"2024-03-06T07:21:56.358529+0000","last_undegraded":"2024-03-06T07:21:56.358529+0000","last_fullsized":"2024-03-06T07:21:56.358529+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:16.531191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"55'8","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426304+0000","last_change":"2024-03-06T07:21:54.377542+0000","last_active":"2024-03-06T07:21:57.426304+0000","last_peered":"2024-03-06T07:21:57.426304+0000","last_clean":"2024-03-06T07:21:57.426304+0000","last_became_active":"2024-03-06T07:21:54.377250+0000","last_became_peered":"2024-03-06T07:21:54.377250+0000","last_unstale":"2024-03-06T07:21:57.426304+0000","last_undegraded":"2024-03-06T07:21:57.426304+0000","last_fullsized":"2024-03-06T07:21:57.426304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:27.009793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377792+0000","last_change":"2024-03-06T07:21:43.323883+0000","last_active":"2024-03-06T07:21:56.377792+0000","last_peered":"2024-03-06T07:21:56.377792+0000","last_clean":"2024-03-06T07:21:56.377792+0000","last_became_active":"2024-03-06T07:21:43.323613+0000","last_became_peered":"2024-03-06T07:21:43.323613+0000","last_unstale":"2024-03-06T07:21:56.377792+0000","last_undegraded":"2024-03-06T07:21:56.377792+0000","last_fullsized":"2024-03-06T07:21:56.377792+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:43.186546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":27,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359111+0000","last_change":"2024-03-06T07:21:42.304199+0000","last_active":"2024-03-06T07:21:56.359111+0000","last_peered":"2024-03-06T07:21:56.359111+0000","last_clean":"2024-03-06T07:21:56.359111+0000","last_became_active":"2024-03-06T07:21:42.303802+0000","last_became_peered":"2024-03-06T07:21:42.303802+0000","last_unstale":"2024-03-06T07:21:56.359111+0000","last_undegraded":"2024-03-06T07:21:56.359111+0000","last_fullsized":"2024-03-06T07:21:56.359111+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426075+0000","last_change":"2024-03-06T07:21:55.385184+0000","last_active":"2024-03-06T07:21:57.426075+0000","last_peered":"2024-03-06T07:21:57.426075+0000","last_clean":"2024-03-06T07:21:57.426075+0000","last_became_active":"2024-03-06T07:21:55.383692+0000","last_became_peered":"2024-03-06T07:21:55.383692+0000","last_unstale":"2024-03-06T07:21:57.426075+0000","last_undegraded":"2024-03-06T07:21:57.426075+0000","last_fullsized":"2024-03-06T07:21:57.426075+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:32.250475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426661+0000","last_change":"2024-03-06T07:21:54.368687+0000","last_active":"2024-03-06T07:21:57.426661+0000","last_peered":"2024-03-06T07:21:57.426661+0000","last_clean":"2024-03-06T07:21:57.426661+0000","last_became_active":"2024-03-06T07:21:54.368106+0000","last_became_peered":"2024-03-06T07:21:54.368106+0000","last_unstale":"2024-03-06T07:21:57.426661+0000","last_undegraded":"2024-03-06T07:21:57.426661+0000","last_fullsized":"2024-03-06T07:21:57.426661+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:32.049609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362337+0000","last_change":"2024-03-06T07:21:43.323512+0000","last_active":"2024-03-06T07:21:56.362337+0000","last_peered":"2024-03-06T07:21:56.362337+0000","last_clean":"2024-03-06T07:21:56.362337+0000","last_became_active":"2024-03-06T07:21:43.323216+0000","last_became_peered":"2024-03-06T07:21:43.323216+0000","last_unstale":"2024-03-06T07:21:56.362337+0000","last_undegraded":"2024-03-06T07:21:56.362337+0000","last_fullsized":"2024-03-06T07:21:56.362337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:31.821522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":28,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359100+0000","last_change":"2024-03-06T07:21:42.305711+0000","last_active":"2024-03-06T07:21:56.359100+0000","last_peered":"2024-03-06T07:21:56.359100+0000","last_clean":"2024-03-06T07:21:56.359100+0000","last_became_active":"2024-03-06T07:21:42.305431+0000","last_became_peered":"2024-03-06T07:21:42.305431+0000","last_unstale":"2024-03-06T07:21:56.359100+0000","last_undegraded":"2024-03-06T07:21:56.359100+0000","last_fullsized":"2024-03-06T07:21:56.359100+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377619+0000","last_change":"2024-03-06T07:21:55.375089+0000","last_active":"2024-03-06T07:21:56.377619+0000","last_peered":"2024-03-06T07:21:56.377619+0000","last_clean":"2024-03-06T07:21:56.377619+0000","last_became_active":"2024-03-06T07:21:55.374527+0000","last_became_peered":"2024-03-06T07:21:55.374527+0000","last_unstale":"2024-03-06T07:21:56.377619+0000","last_undegraded":"2024-03-06T07:21:56.377619+0000","last_fullsized":"2024-03-06T07:21:56.377619+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:10:53.122484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377615+0000","last_change":"2024-03-06T07:21:54.370174+0000","last_active":"2024-03-06T07:21:56.377615+0000","last_peered":"2024-03-06T07:21:56.377615+0000","last_clean":"2024-03-06T07:21:56.377615+0000","last_became_active":"2024-03-06T07:21:54.369897+0000","last_became_peered":"2024-03-06T07:21:54.369897+0000","last_unstale":"2024-03-06T07:21:56.377615+0000","last_undegraded":"2024-03-06T07:21:56.377615+0000","last_fullsized":"2024-03-06T07:21:56.377615+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:08.612677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426674+0000","last_change":"2024-03-06T07:21:43.317966+0000","last_active":"2024-03-06T07:21:57.426674+0000","last_peered":"2024-03-06T07:21:57.426674+0000","last_clean":"2024-03-06T07:21:57.426674+0000","last_became_active":"2024-03-06T07:21:43.317700+0000","last_became_peered":"2024-03-06T07:21:43.317700+0000","last_unstale":"2024-03-06T07:21:57.426674+0000","last_undegraded":"2024-03-06T07:21:57.426674+0000","last_fullsized":"2024-03-06T07:21:57.426674+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":28,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378016+0000","last_change":"2024-03-06T07:21:42.315045+0000","last_active":"2024-03-06T07:21:56.378016+0000","last_peered":"2024-03-06T07:21:56.378016+0000","last_clean":"2024-03-06T07:21:56.378016+0000","last_became_active":"2024-03-06T07:21:42.314768+0000","last_became_peered":"2024-03-06T07:21:42.314768+0000","last_unstale":"2024-03-06T07:21:56.378016+0000","last_undegraded":"2024-03-06T07:21:56.378016+0000","last_fullsized":"2024-03-06T07:21:56.378016+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:14.953476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362823+0000","last_change":"2024-03-06T07:21:55.380069+0000","last_active":"2024-03-06T07:21:56.362823+0000","last_peered":"2024-03-06T07:21:56.362823+0000","last_clean":"2024-03-06T07:21:56.362823+0000","last_became_active":"2024-03-06T07:21:55.379752+0000","last_became_peered":"2024-03-06T07:21:55.379752+0000","last_unstale":"2024-03-06T07:21:56.362823+0000","last_undegraded":"2024-03-06T07:21:56.362823+0000","last_fullsized":"2024-03-06T07:21:56.362823+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:33.055109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"55'3","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427136+0000","last_change":"2024-03-06T07:21:54.375783+0000","last_active":"2024-03-06T07:21:57.427136+0000","last_peered":"2024-03-06T07:21:57.427136+0000","last_clean":"2024-03-06T07:21:57.427136+0000","last_became_active":"2024-03-06T07:21:54.375473+0000","last_became_peered":"2024-03-06T07:21:54.375473+0000","last_unstale":"2024-03-06T07:21:57.427136+0000","last_undegraded":"2024-03-06T07:21:57.427136+0000","last_fullsized":"2024-03-06T07:21:57.427136+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:31:49.191761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":29,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427367+0000","last_change":"2024-03-06T07:21:50.343971+0000","last_active":"2024-03-06T07:21:57.427367+0000","last_peered":"2024-03-06T07:21:57.427367+0000","last_clean":"2024-03-06T07:21:57.427367+0000","last_became_active":"2024-03-06T07:21:50.343658+0000","last_became_peered":"2024-03-06T07:21:50.343658+0000","last_unstale":"2024-03-06T07:21:57.427367+0000","last_undegraded":"2024-03-06T07:21:57.427367+0000","last_fullsized":"2024-03-06T07:21:57.427367+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":27,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427308+0000","last_change":"2024-03-06T07:21:42.317614+0000","last_active":"2024-03-06T07:21:57.427308+0000","last_peered":"2024-03-06T07:21:57.427308+0000","last_clean":"2024-03-06T07:21:57.427308+0000","last_became_active":"2024-03-06T07:21:42.317094+0000","last_became_peered":"2024-03-06T07:21:42.317094+0000","last_unstale":"2024-03-06T07:21:57.427308+0000","last_undegraded":"2024-03-06T07:21:57.427308+0000","last_fullsized":"2024-03-06T07:21:57.427308+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:15.739191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427248+0000","last_change":"2024-03-06T07:21:55.383370+0000","last_active":"2024-03-06T07:21:57.427248+0000","last_peered":"2024-03-06T07:21:57.427248+0000","last_clean":"2024-03-06T07:21:57.427248+0000","last_became_active":"2024-03-06T07:21:55.383114+0000","last_became_peered":"2024-03-06T07:21:55.383114+0000","last_unstale":"2024-03-06T07:21:57.427248+0000","last_undegraded":"2024-03-06T07:21:57.427248+0000","last_fullsized":"2024-03-06T07:21:57.427248+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:55.244540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"55'6","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358815+0000","last_change":"2024-03-06T07:21:54.375914+0000","last_active":"2024-03-06T07:21:56.358815+0000","last_peered":"2024-03-06T07:21:56.358815+0000","last_clean":"2024-03-06T07:21:56.358815+0000","last_became_active":"2024-03-06T07:21:54.375619+0000","last_became_peered":"2024-03-06T07:21:54.375619+0000","last_unstale":"2024-03-06T07:21:56.358815+0000","last_undegraded":"2024-03-06T07:21:56.358815+0000","last_fullsized":"2024-03-06T07:21:56.358815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:59.661638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426976+0000","last_change":"2024-03-06T07:21:43.318727+0000","last_active":"2024-03-06T07:21:57.426976+0000","last_peered":"2024-03-06T07:21:57.426976+0000","last_clean":"2024-03-06T07:21:57.426976+0000","last_became_active":"2024-03-06T07:21:43.318440+0000","last_became_peered":"2024-03-06T07:21:43.318440+0000","last_unstale":"2024-03-06T07:21:57.426976+0000","last_undegraded":"2024-03-06T07:21:57.426976+0000","last_fullsized":"2024-03-06T07:21:57.426976+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:58.320938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":27,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426194+0000","last_change":"2024-03-06T07:21:42.307726+0000","last_active":"2024-03-06T07:21:57.426194+0000","last_peered":"2024-03-06T07:21:57.426194+0000","last_clean":"2024-03-06T07:21:57.426194+0000","last_became_active":"2024-03-06T07:21:42.307445+0000","last_became_peered":"2024-03-06T07:21:42.307445+0000","last_unstale":"2024-03-06T07:21:57.426194+0000","last_undegraded":"2024-03-06T07:21:57.426194+0000","last_fullsized":"2024-03-06T07:21:57.426194+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358761+0000","last_change":"2024-03-06T07:21:55.371539+0000","last_active":"2024-03-06T07:21:56.358761+0000","last_peered":"2024-03-06T07:21:56.358761+0000","last_clean":"2024-03-06T07:21:56.358761+0000","last_became_active":"2024-03-06T07:21:55.371114+0000","last_became_peered":"2024-03-06T07:21:55.371114+0000","last_unstale":"2024-03-06T07:21:56.358761+0000","last_undegraded":"2024-03-06T07:21:56.358761+0000","last_fullsized":"2024-03-06T07:21:56.358761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:43.535618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427572+0000","last_change":"2024-03-06T07:21:54.370955+0000","last_active":"2024-03-06T07:21:57.427572+0000","last_peered":"2024-03-06T07:21:57.427572+0000","last_clean":"2024-03-06T07:21:57.427572+0000","last_became_active":"2024-03-06T07:21:54.370680+0000","last_became_peered":"2024-03-06T07:21:54.370680+0000","last_unstale":"2024-03-06T07:21:57.427572+0000","last_undegraded":"2024-03-06T07:21:57.427572+0000","last_fullsized":"2024-03-06T07:21:57.427572+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:42.530118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426358+0000","last_change":"2024-03-06T07:21:43.318372+0000","last_active":"2024-03-06T07:21:57.426358+0000","last_peered":"2024-03-06T07:21:57.426358+0000","last_clean":"2024-03-06T07:21:57.426358+0000","last_became_active":"2024-03-06T07:21:43.318047+0000","last_became_peered":"2024-03-06T07:21:43.318047+0000","last_unstale":"2024-03-06T07:21:57.426358+0000","last_undegraded":"2024-03-06T07:21:57.426358+0000","last_fullsized":"2024-03-06T07:21:57.426358+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362199+0000","last_change":"2024-03-06T07:21:42.310677+0000","last_active":"2024-03-06T07:21:56.362199+0000","last_peered":"2024-03-06T07:21:56.362199+0000","last_clean":"2024-03-06T07:21:56.362199+0000","last_became_active":"2024-03-06T07:21:42.310375+0000","last_became_peered":"2024-03-06T07:21:42.310375+0000","last_unstale":"2024-03-06T07:21:56.362199+0000","last_undegraded":"2024-03-06T07:21:56.362199+0000","last_fullsized":"2024-03-06T07:21:56.362199+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:13.837451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426380+0000","last_change":"2024-03-06T07:21:55.374692+0000","last_active":"2024-03-06T07:21:57.426380+0000","last_peered":"2024-03-06T07:21:57.426380+0000","last_clean":"2024-03-06T07:21:57.426380+0000","last_became_active":"2024-03-06T07:21:55.374389+0000","last_became_peered":"2024-03-06T07:21:55.374389+0000","last_unstale":"2024-03-06T07:21:57.426380+0000","last_undegraded":"2024-03-06T07:21:57.426380+0000","last_fullsized":"2024-03-06T07:21:57.426380+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:47.260471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427036+0000","last_change":"2024-03-06T07:21:54.374914+0000","last_active":"2024-03-06T07:21:57.427036+0000","last_peered":"2024-03-06T07:21:57.427036+0000","last_clean":"2024-03-06T07:21:57.427036+0000","last_became_active":"2024-03-06T07:21:54.374570+0000","last_became_peered":"2024-03-06T07:21:54.374570+0000","last_unstale":"2024-03-06T07:21:57.427036+0000","last_undegraded":"2024-03-06T07:21:57.427036+0000","last_fullsized":"2024-03-06T07:21:57.427036+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:51.314141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427651+0000","last_change":"2024-03-06T07:21:43.325740+0000","last_active":"2024-03-06T07:21:57.427651+0000","last_peered":"2024-03-06T07:21:57.427651+0000","last_clean":"2024-03-06T07:21:57.427651+0000","last_became_active":"2024-03-06T07:21:43.325425+0000","last_became_peered":"2024-03-06T07:21:43.325425+0000","last_unstale":"2024-03-06T07:21:57.427651+0000","last_undegraded":"2024-03-06T07:21:57.427651+0000","last_fullsized":"2024-03-06T07:21:57.427651+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426150+0000","last_change":"2024-03-06T07:21:42.319786+0000","last_active":"2024-03-06T07:21:57.426150+0000","last_peered":"2024-03-06T07:21:57.426150+0000","last_clean":"2024-03-06T07:21:57.426150+0000","last_became_active":"2024-03-06T07:21:42.319430+0000","last_became_peered":"2024-03-06T07:21:42.319430+0000","last_unstale":"2024-03-06T07:21:57.426150+0000","last_undegraded":"2024-03-06T07:21:57.426150+0000","last_fullsized":"2024-03-06T07:21:57.426150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:09.719754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362623+0000","last_change":"2024-03-06T07:21:55.378287+0000","last_active":"2024-03-06T07:21:56.362623+0000","last_peered":"2024-03-06T07:21:56.362623+0000","last_clean":"2024-03-06T07:21:56.362623+0000","last_became_active":"2024-03-06T07:21:55.376674+0000","last_became_peered":"2024-03-06T07:21:55.376674+0000","last_unstale":"2024-03-06T07:21:56.362623+0000","last_undegraded":"2024-03-06T07:21:56.362623+0000","last_fullsized":"2024-03-06T07:21:56.362623+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:43.524816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426148+0000","last_change":"2024-03-06T07:21:54.367800+0000","last_active":"2024-03-06T07:21:57.426148+0000","last_peered":"2024-03-06T07:21:57.426148+0000","last_clean":"2024-03-06T07:21:57.426148+0000","last_became_active":"2024-03-06T07:21:54.367468+0000","last_became_peered":"2024-03-06T07:21:54.367468+0000","last_unstale":"2024-03-06T07:21:57.426148+0000","last_undegraded":"2024-03-06T07:21:57.426148+0000","last_fullsized":"2024-03-06T07:21:57.426148+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:10:52.116984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426457+0000","last_change":"2024-03-06T07:21:43.330805+0000","last_active":"2024-03-06T07:21:57.426457+0000","last_peered":"2024-03-06T07:21:57.426457+0000","last_clean":"2024-03-06T07:21:57.426457+0000","last_became_active":"2024-03-06T07:21:43.330267+0000","last_became_peered":"2024-03-06T07:21:43.330267+0000","last_unstale":"2024-03-06T07:21:57.426457+0000","last_undegraded":"2024-03-06T07:21:57.426457+0000","last_fullsized":"2024-03-06T07:21:57.426457+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:56.716578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427734+0000","last_change":"2024-03-06T07:21:42.312213+0000","last_active":"2024-03-06T07:21:57.427734+0000","last_peered":"2024-03-06T07:21:57.427734+0000","last_clean":"2024-03-06T07:21:57.427734+0000","last_became_active":"2024-03-06T07:21:42.311878+0000","last_became_peered":"2024-03-06T07:21:42.311878+0000","last_unstale":"2024-03-06T07:21:57.427734+0000","last_undegraded":"2024-03-06T07:21:57.427734+0000","last_fullsized":"2024-03-06T07:21:57.427734+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426894+0000","last_change":"2024-03-06T07:21:55.384447+0000","last_active":"2024-03-06T07:21:57.426894+0000","last_peered":"2024-03-06T07:21:57.426894+0000","last_clean":"2024-03-06T07:21:57.426894+0000","last_became_active":"2024-03-06T07:21:55.384168+0000","last_became_peered":"2024-03-06T07:21:55.384168+0000","last_unstale":"2024-03-06T07:21:57.426894+0000","last_undegraded":"2024-03-06T07:21:57.426894+0000","last_fullsized":"2024-03-06T07:21:57.426894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:41.567995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.361974+0000","last_change":"2024-03-06T07:21:54.377646+0000","last_active":"2024-03-06T07:21:56.361974+0000","last_peered":"2024-03-06T07:21:56.361974+0000","last_clean":"2024-03-06T07:21:56.361974+0000","last_became_active":"2024-03-06T07:21:54.377219+0000","last_became_peered":"2024-03-06T07:21:54.377219+0000","last_unstale":"2024-03-06T07:21:56.361974+0000","last_undegraded":"2024-03-06T07:21:56.361974+0000","last_fullsized":"2024-03-06T07:21:56.361974+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:03.821956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377754+0000","last_change":"2024-03-06T07:21:43.320122+0000","last_active":"2024-03-06T07:21:56.377754+0000","last_peered":"2024-03-06T07:21:56.377754+0000","last_clean":"2024-03-06T07:21:56.377754+0000","last_became_active":"2024-03-06T07:21:43.319799+0000","last_became_peered":"2024-03-06T07:21:43.319799+0000","last_unstale":"2024-03-06T07:21:56.377754+0000","last_undegraded":"2024-03-06T07:21:56.377754+0000","last_fullsized":"2024-03-06T07:21:56.377754+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:20.192482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427634+0000","last_change":"2024-03-06T07:21:42.319681+0000","last_active":"2024-03-06T07:21:57.427634+0000","last_peered":"2024-03-06T07:21:57.427634+0000","last_clean":"2024-03-06T07:21:57.427634+0000","last_became_active":"2024-03-06T07:21:42.319452+0000","last_became_peered":"2024-03-06T07:21:42.319452+0000","last_unstale":"2024-03-06T07:21:57.427634+0000","last_undegraded":"2024-03-06T07:21:57.427634+0000","last_fullsized":"2024-03-06T07:21:57.427634+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:35.308790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380229+0000","last_change":"2024-03-06T07:21:55.371609+0000","last_active":"2024-03-06T07:21:56.380229+0000","last_peered":"2024-03-06T07:21:56.380229+0000","last_clean":"2024-03-06T07:21:56.380229+0000","last_became_active":"2024-03-06T07:21:55.371311+0000","last_became_peered":"2024-03-06T07:21:55.371311+0000","last_unstale":"2024-03-06T07:21:56.380229+0000","last_undegraded":"2024-03-06T07:21:56.380229+0000","last_fullsized":"2024-03-06T07:21:56.380229+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:26.899267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426566+0000","last_change":"2024-03-06T07:21:54.367115+0000","last_active":"2024-03-06T07:21:57.426566+0000","last_peered":"2024-03-06T07:21:57.426566+0000","last_clean":"2024-03-06T07:21:57.426566+0000","last_became_active":"2024-03-06T07:21:54.366564+0000","last_became_peered":"2024-03-06T07:21:54.366564+0000","last_unstale":"2024-03-06T07:21:57.426566+0000","last_undegraded":"2024-03-06T07:21:57.426566+0000","last_fullsized":"2024-03-06T07:21:57.426566+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:09.244027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426495+0000","last_change":"2024-03-06T07:21:43.323490+0000","last_active":"2024-03-06T07:21:57.426495+0000","last_peered":"2024-03-06T07:21:57.426495+0000","last_clean":"2024-03-06T07:21:57.426495+0000","last_became_active":"2024-03-06T07:21:43.323088+0000","last_became_peered":"2024-03-06T07:21:43.323088+0000","last_unstale":"2024-03-06T07:21:57.426495+0000","last_undegraded":"2024-03-06T07:21:57.426495+0000","last_fullsized":"2024-03-06T07:21:57.426495+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:42.977899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":29,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427136+0000","last_change":"2024-03-06T07:21:42.312732+0000","last_active":"2024-03-06T07:21:57.427136+0000","last_peered":"2024-03-06T07:21:57.427136+0000","last_clean":"2024-03-06T07:21:57.427136+0000","last_became_active":"2024-03-06T07:21:42.312462+0000","last_became_peered":"2024-03-06T07:21:42.312462+0000","last_unstale":"2024-03-06T07:21:57.427136+0000","last_undegraded":"2024-03-06T07:21:57.427136+0000","last_fullsized":"2024-03-06T07:21:57.427136+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358023+0000","last_change":"2024-03-06T07:21:55.374781+0000","last_active":"2024-03-06T07:21:56.358023+0000","last_peered":"2024-03-06T07:21:56.358023+0000","last_clean":"2024-03-06T07:21:56.358023+0000","last_became_active":"2024-03-06T07:21:55.374201+0000","last_became_peered":"2024-03-06T07:21:55.374201+0000","last_unstale":"2024-03-06T07:21:56.358023+0000","last_undegraded":"2024-03-06T07:21:56.358023+0000","last_fullsized":"2024-03-06T07:21:56.358023+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:08.101642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426842+0000","last_change":"2024-03-06T07:21:54.365558+0000","last_active":"2024-03-06T07:21:57.426842+0000","last_peered":"2024-03-06T07:21:57.426842+0000","last_clean":"2024-03-06T07:21:57.426842+0000","last_became_active":"2024-03-06T07:21:54.365272+0000","last_became_peered":"2024-03-06T07:21:54.365272+0000","last_unstale":"2024-03-06T07:21:57.426842+0000","last_undegraded":"2024-03-06T07:21:57.426842+0000","last_fullsized":"2024-03-06T07:21:57.426842+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:01.404942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427180+0000","last_change":"2024-03-06T07:21:43.320022+0000","last_active":"2024-03-06T07:21:57.427180+0000","last_peered":"2024-03-06T07:21:57.427180+0000","last_clean":"2024-03-06T07:21:57.427180+0000","last_became_active":"2024-03-06T07:21:43.319716+0000","last_became_peered":"2024-03-06T07:21:43.319716+0000","last_unstale":"2024-03-06T07:21:57.427180+0000","last_undegraded":"2024-03-06T07:21:57.427180+0000","last_fullsized":"2024-03-06T07:21:57.427180+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:50.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":32,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427550+0000","last_change":"2024-03-06T07:21:42.308298+0000","last_active":"2024-03-06T07:21:57.427550+0000","last_peered":"2024-03-06T07:21:57.427550+0000","last_clean":"2024-03-06T07:21:57.427550+0000","last_became_active":"2024-03-06T07:21:42.308008+0000","last_became_peered":"2024-03-06T07:21:42.308008+0000","last_unstale":"2024-03-06T07:21:57.427550+0000","last_undegraded":"2024-03-06T07:21:57.427550+0000","last_fullsized":"2024-03-06T07:21:57.427550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:51.723259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427531+0000","last_change":"2024-03-06T07:21:55.367324+0000","last_active":"2024-03-06T07:21:57.427531+0000","last_peered":"2024-03-06T07:21:57.427531+0000","last_clean":"2024-03-06T07:21:57.427531+0000","last_became_active":"2024-03-06T07:21:55.367064+0000","last_became_peered":"2024-03-06T07:21:55.367064+0000","last_unstale":"2024-03-06T07:21:57.427531+0000","last_undegraded":"2024-03-06T07:21:57.427531+0000","last_fullsized":"2024-03-06T07:21:57.427531+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:25.479299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427051+0000","last_change":"2024-03-06T07:21:54.375937+0000","last_active":"2024-03-06T07:21:57.427051+0000","last_peered":"2024-03-06T07:21:57.427051+0000","last_clean":"2024-03-06T07:21:57.427051+0000","last_became_active":"2024-03-06T07:21:54.375648+0000","last_became_peered":"2024-03-06T07:21:54.375648+0000","last_unstale":"2024-03-06T07:21:57.427051+0000","last_undegraded":"2024-03-06T07:21:57.427051+0000","last_fullsized":"2024-03-06T07:21:57.427051+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:02.230230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377855+0000","last_change":"2024-03-06T07:21:43.323228+0000","last_active":"2024-03-06T07:21:56.377855+0000","last_peered":"2024-03-06T07:21:56.377855+0000","last_clean":"2024-03-06T07:21:56.377855+0000","last_became_active":"2024-03-06T07:21:43.322951+0000","last_became_peered":"2024-03-06T07:21:43.322951+0000","last_unstale":"2024-03-06T07:21:56.377855+0000","last_undegraded":"2024-03-06T07:21:56.377855+0000","last_fullsized":"2024-03-06T07:21:56.377855+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:15.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":28,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427404+0000","last_change":"2024-03-06T07:21:42.321872+0000","last_active":"2024-03-06T07:21:57.427404+0000","last_peered":"2024-03-06T07:21:57.427404+0000","last_clean":"2024-03-06T07:21:57.427404+0000","last_became_active":"2024-03-06T07:21:42.321384+0000","last_became_peered":"2024-03-06T07:21:42.321384+0000","last_unstale":"2024-03-06T07:21:57.427404+0000","last_undegraded":"2024-03-06T07:21:57.427404+0000","last_fullsized":"2024-03-06T07:21:57.427404+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:30.817699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427491+0000","last_change":"2024-03-06T07:21:55.366192+0000","last_active":"2024-03-06T07:21:57.427491+0000","last_peered":"2024-03-06T07:21:57.427491+0000","last_clean":"2024-03-06T07:21:57.427491+0000","last_became_active":"2024-03-06T07:21:55.365910+0000","last_became_peered":"2024-03-06T07:21:55.365910+0000","last_unstale":"2024-03-06T07:21:57.427491+0000","last_undegraded":"2024-03-06T07:21:57.427491+0000","last_fullsized":"2024-03-06T07:21:57.427491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:16.339384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427108+0000","last_change":"2024-03-06T07:21:54.373488+0000","last_active":"2024-03-06T07:21:57.427108+0000","last_peered":"2024-03-06T07:21:57.427108+0000","last_clean":"2024-03-06T07:21:57.427108+0000","last_became_active":"2024-03-06T07:21:54.373225+0000","last_became_peered":"2024-03-06T07:21:54.373225+0000","last_unstale":"2024-03-06T07:21:57.427108+0000","last_undegraded":"2024-03-06T07:21:57.427108+0000","last_fullsized":"2024-03-06T07:21:57.427108+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:49.462106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427491+0000","last_change":"2024-03-06T07:21:43.324244+0000","last_active":"2024-03-06T07:21:57.427491+0000","last_peered":"2024-03-06T07:21:57.427491+0000","last_clean":"2024-03-06T07:21:57.427491+0000","last_became_active":"2024-03-06T07:21:43.323549+0000","last_became_peered":"2024-03-06T07:21:43.323549+0000","last_unstale":"2024-03-06T07:21:57.427491+0000","last_undegraded":"2024-03-06T07:21:57.427491+0000","last_fullsized":"2024-03-06T07:21:57.427491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:48.071319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362702+0000","last_change":"2024-03-06T07:21:42.318308+0000","last_active":"2024-03-06T07:21:56.362702+0000","last_peered":"2024-03-06T07:21:56.362702+0000","last_clean":"2024-03-06T07:21:56.362702+0000","last_became_active":"2024-03-06T07:21:42.318080+0000","last_became_peered":"2024-03-06T07:21:42.318080+0000","last_unstale":"2024-03-06T07:21:56.362702+0000","last_undegraded":"2024-03-06T07:21:56.362702+0000","last_fullsized":"2024-03-06T07:21:56.362702+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:03.158929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362904+0000","last_change":"2024-03-06T07:21:55.374911+0000","last_active":"2024-03-06T07:21:56.362904+0000","last_peered":"2024-03-06T07:21:56.362904+0000","last_clean":"2024-03-06T07:21:56.362904+0000","last_became_active":"2024-03-06T07:21:55.373801+0000","last_became_peered":"2024-03-06T07:21:55.373801+0000","last_unstale":"2024-03-06T07:21:56.362904+0000","last_undegraded":"2024-03-06T07:21:56.362904+0000","last_fullsized":"2024-03-06T07:21:56.362904+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:08.101642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"55'4","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359218+0000","last_change":"2024-03-06T07:21:54.377690+0000","last_active":"2024-03-06T07:21:56.359218+0000","last_peered":"2024-03-06T07:21:56.359218+0000","last_clean":"2024-03-06T07:21:56.359218+0000","last_became_active":"2024-03-06T07:21:54.377408+0000","last_became_peered":"2024-03-06T07:21:54.377408+0000","last_unstale":"2024-03-06T07:21:56.359218+0000","last_undegraded":"2024-03-06T07:21:56.359218+0000","last_fullsized":"2024-03-06T07:21:56.359218+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:54.239040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426276+0000","last_change":"2024-03-06T07:21:43.331853+0000","last_active":"2024-03-06T07:21:57.426276+0000","last_peered":"2024-03-06T07:21:57.426276+0000","last_clean":"2024-03-06T07:21:57.426276+0000","last_became_active":"2024-03-06T07:21:43.331560+0000","last_became_peered":"2024-03-06T07:21:43.331560+0000","last_unstale":"2024-03-06T07:21:57.426276+0000","last_undegraded":"2024-03-06T07:21:57.426276+0000","last_fullsized":"2024-03-06T07:21:57.426276+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426619+0000","last_change":"2024-03-06T07:21:42.309167+0000","last_active":"2024-03-06T07:21:57.426619+0000","last_peered":"2024-03-06T07:21:57.426619+0000","last_clean":"2024-03-06T07:21:57.426619+0000","last_became_active":"2024-03-06T07:21:42.308882+0000","last_became_peered":"2024-03-06T07:21:42.308882+0000","last_unstale":"2024-03-06T07:21:57.426619+0000","last_undegraded":"2024-03-06T07:21:57.426619+0000","last_fullsized":"2024-03-06T07:21:57.426619+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427528+0000","last_change":"2024-03-06T07:21:55.389235+0000","last_active":"2024-03-06T07:21:57.427528+0000","last_peered":"2024-03-06T07:21:57.427528+0000","last_clean":"2024-03-06T07:21:57.427528+0000","last_became_active":"2024-03-06T07:21:55.388954+0000","last_became_peered":"2024-03-06T07:21:55.388954+0000","last_unstale":"2024-03-06T07:21:57.427528+0000","last_undegraded":"2024-03-06T07:21:57.427528+0000","last_fullsized":"2024-03-06T07:21:57.427528+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:15.092487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427614+0000","last_change":"2024-03-06T07:21:54.377427+0000","last_active":"2024-03-06T07:21:57.427614+0000","last_peered":"2024-03-06T07:21:57.427614+0000","last_clean":"2024-03-06T07:21:57.427614+0000","last_became_active":"2024-03-06T07:21:54.377183+0000","last_became_peered":"2024-03-06T07:21:54.377183+0000","last_unstale":"2024-03-06T07:21:57.427614+0000","last_undegraded":"2024-03-06T07:21:57.427614+0000","last_fullsized":"2024-03-06T07:21:57.427614+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:30.018217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427736+0000","last_change":"2024-03-06T07:21:43.324536+0000","last_active":"2024-03-06T07:21:57.427736+0000","last_peered":"2024-03-06T07:21:57.427736+0000","last_clean":"2024-03-06T07:21:57.427736+0000","last_became_active":"2024-03-06T07:21:43.323818+0000","last_became_peered":"2024-03-06T07:21:43.323818+0000","last_unstale":"2024-03-06T07:21:57.427736+0000","last_undegraded":"2024-03-06T07:21:57.427736+0000","last_fullsized":"2024-03-06T07:21:57.427736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":34,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426143+0000","last_change":"2024-03-06T07:21:42.308460+0000","last_active":"2024-03-06T07:21:57.426143+0000","last_peered":"2024-03-06T07:21:57.426143+0000","last_clean":"2024-03-06T07:21:57.426143+0000","last_became_active":"2024-03-06T07:21:42.307461+0000","last_became_peered":"2024-03-06T07:21:42.307461+0000","last_unstale":"2024-03-06T07:21:57.426143+0000","last_undegraded":"2024-03-06T07:21:57.426143+0000","last_fullsized":"2024-03-06T07:21:57.426143+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:34.346180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358135+0000","last_change":"2024-03-06T07:21:55.379027+0000","last_active":"2024-03-06T07:21:56.358135+0000","last_peered":"2024-03-06T07:21:56.358135+0000","last_clean":"2024-03-06T07:21:56.358135+0000","last_became_active":"2024-03-06T07:21:55.378797+0000","last_became_peered":"2024-03-06T07:21:55.378797+0000","last_unstale":"2024-03-06T07:21:56.358135+0000","last_undegraded":"2024-03-06T07:21:56.358135+0000","last_fullsized":"2024-03-06T07:21:56.358135+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:00.129312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426148+0000","last_change":"2024-03-06T07:21:54.366324+0000","last_active":"2024-03-06T07:21:57.426148+0000","last_peered":"2024-03-06T07:21:57.426148+0000","last_clean":"2024-03-06T07:21:57.426148+0000","last_became_active":"2024-03-06T07:21:54.366064+0000","last_became_peered":"2024-03-06T07:21:54.366064+0000","last_unstale":"2024-03-06T07:21:57.426148+0000","last_undegraded":"2024-03-06T07:21:57.426148+0000","last_fullsized":"2024-03-06T07:21:57.426148+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:49.462106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426045+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:57.426045+0000","last_peered":"2024-03-06T07:21:57.426045+0000","last_clean":"2024-03-06T07:21:57.426045+0000","last_became_active":"2024-03-06T07:21:43.316848+0000","last_became_peered":"2024-03-06T07:21:43.316848+0000","last_unstale":"2024-03-06T07:21:57.426045+0000","last_undegraded":"2024-03-06T07:21:57.426045+0000","last_fullsized":"2024-03-06T07:21:57.426045+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.108399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":27,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427908+0000","last_change":"2024-03-06T07:21:42.319998+0000","last_active":"2024-03-06T07:21:57.427908+0000","last_peered":"2024-03-06T07:21:57.427908+0000","last_clean":"2024-03-06T07:21:57.427908+0000","last_became_active":"2024-03-06T07:21:42.319529+0000","last_became_peered":"2024-03-06T07:21:42.319529+0000","last_unstale":"2024-03-06T07:21:57.427908+0000","last_undegraded":"2024-03-06T07:21:57.427908+0000","last_fullsized":"2024-03-06T07:21:57.427908+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380594+0000","last_change":"2024-03-06T07:21:55.378645+0000","last_active":"2024-03-06T07:21:56.380594+0000","last_peered":"2024-03-06T07:21:56.380594+0000","last_clean":"2024-03-06T07:21:56.380594+0000","last_became_active":"2024-03-06T07:21:55.378383+0000","last_became_peered":"2024-03-06T07:21:55.378383+0000","last_unstale":"2024-03-06T07:21:56.380594+0000","last_undegraded":"2024-03-06T07:21:56.380594+0000","last_fullsized":"2024-03-06T07:21:56.380594+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:40.657177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.357982+0000","last_change":"2024-03-06T07:21:54.370046+0000","last_active":"2024-03-06T07:21:56.357982+0000","last_peered":"2024-03-06T07:21:56.357982+0000","last_clean":"2024-03-06T07:21:56.357982+0000","last_became_active":"2024-03-06T07:21:54.369791+0000","last_became_peered":"2024-03-06T07:21:54.369791+0000","last_unstale":"2024-03-06T07:21:56.357982+0000","last_undegraded":"2024-03-06T07:21:56.357982+0000","last_fullsized":"2024-03-06T07:21:56.357982+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:13.125643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358181+0000","last_change":"2024-03-06T07:21:43.327074+0000","last_active":"2024-03-06T07:21:56.358181+0000","last_peered":"2024-03-06T07:21:56.358181+0000","last_clean":"2024-03-06T07:21:56.358181+0000","last_became_active":"2024-03-06T07:21:43.326797+0000","last_became_peered":"2024-03-06T07:21:43.326797+0000","last_unstale":"2024-03-06T07:21:56.358181+0000","last_undegraded":"2024-03-06T07:21:56.358181+0000","last_fullsized":"2024-03-06T07:21:56.358181+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:25.767310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":28,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.381059+0000","last_change":"2024-03-06T07:21:42.310454+0000","last_active":"2024-03-06T07:21:56.381059+0000","last_peered":"2024-03-06T07:21:56.381059+0000","last_clean":"2024-03-06T07:21:56.381059+0000","last_became_active":"2024-03-06T07:21:42.310172+0000","last_became_peered":"2024-03-06T07:21:42.310172+0000","last_unstale":"2024-03-06T07:21:56.381059+0000","last_undegraded":"2024-03-06T07:21:56.381059+0000","last_fullsized":"2024-03-06T07:21:56.381059+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:34.182536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.361963+0000","last_change":"2024-03-06T07:21:55.373097+0000","last_active":"2024-03-06T07:21:56.361963+0000","last_peered":"2024-03-06T07:21:56.361963+0000","last_clean":"2024-03-06T07:21:56.361963+0000","last_became_active":"2024-03-06T07:21:55.372801+0000","last_became_peered":"2024-03-06T07:21:55.372801+0000","last_unstale":"2024-03-06T07:21:56.361963+0000","last_undegraded":"2024-03-06T07:21:56.361963+0000","last_fullsized":"2024-03-06T07:21:56.361963+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:24:00.439098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426712+0000","last_change":"2024-03-06T07:21:54.368897+0000","last_active":"2024-03-06T07:21:57.426712+0000","last_peered":"2024-03-06T07:21:57.426712+0000","last_clean":"2024-03-06T07:21:57.426712+0000","last_became_active":"2024-03-06T07:21:54.368398+0000","last_became_peered":"2024-03-06T07:21:54.368398+0000","last_unstale":"2024-03-06T07:21:57.426712+0000","last_undegraded":"2024-03-06T07:21:57.426712+0000","last_fullsized":"2024-03-06T07:21:57.426712+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:59.123812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426119+0000","last_change":"2024-03-06T07:21:43.324398+0000","last_active":"2024-03-06T07:21:57.426119+0000","last_peered":"2024-03-06T07:21:57.426119+0000","last_clean":"2024-03-06T07:21:57.426119+0000","last_became_active":"2024-03-06T07:21:43.324115+0000","last_became_peered":"2024-03-06T07:21:43.324115+0000","last_unstale":"2024-03-06T07:21:57.426119+0000","last_undegraded":"2024-03-06T07:21:57.426119+0000","last_fullsized":"2024-03-06T07:21:57.426119+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:36.888023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":27,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.363084+0000","last_change":"2024-03-06T07:21:42.312390+0000","last_active":"2024-03-06T07:21:56.363084+0000","last_peered":"2024-03-06T07:21:56.363084+0000","last_clean":"2024-03-06T07:21:56.363084+0000","last_became_active":"2024-03-06T07:21:42.312079+0000","last_became_peered":"2024-03-06T07:21:42.312079+0000","last_unstale":"2024-03-06T07:21:56.363084+0000","last_undegraded":"2024-03-06T07:21:56.363084+0000","last_fullsized":"2024-03-06T07:21:56.363084+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:27.595361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426075+0000","last_change":"2024-03-06T07:21:55.389138+0000","last_active":"2024-03-06T07:21:57.426075+0000","last_peered":"2024-03-06T07:21:57.426075+0000","last_clean":"2024-03-06T07:21:57.426075+0000","last_became_active":"2024-03-06T07:21:55.388872+0000","last_became_peered":"2024-03-06T07:21:55.388872+0000","last_unstale":"2024-03-06T07:21:57.426075+0000","last_undegraded":"2024-03-06T07:21:57.426075+0000","last_fullsized":"2024-03-06T07:21:57.426075+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:27.724865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426936+0000","last_change":"2024-03-06T07:21:54.370872+0000","last_active":"2024-03-06T07:21:57.426936+0000","last_peered":"2024-03-06T07:21:57.426936+0000","last_clean":"2024-03-06T07:21:57.426936+0000","last_became_active":"2024-03-06T07:21:54.370589+0000","last_became_peered":"2024-03-06T07:21:54.370589+0000","last_unstale":"2024-03-06T07:21:57.426936+0000","last_undegraded":"2024-03-06T07:21:57.426936+0000","last_fullsized":"2024-03-06T07:21:57.426936+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:30.018217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358423+0000","last_change":"2024-03-06T07:21:43.317119+0000","last_active":"2024-03-06T07:21:56.358423+0000","last_peered":"2024-03-06T07:21:56.358423+0000","last_clean":"2024-03-06T07:21:56.358423+0000","last_became_active":"2024-03-06T07:21:43.316857+0000","last_became_peered":"2024-03-06T07:21:43.316857+0000","last_unstale":"2024-03-06T07:21:56.358423+0000","last_undegraded":"2024-03-06T07:21:56.358423+0000","last_fullsized":"2024-03-06T07:21:56.358423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:48.715734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.363181+0000","last_change":"2024-03-06T07:21:42.311972+0000","last_active":"2024-03-06T07:21:56.363181+0000","last_peered":"2024-03-06T07:21:56.363181+0000","last_clean":"2024-03-06T07:21:56.363181+0000","last_became_active":"2024-03-06T07:21:42.311640+0000","last_became_peered":"2024-03-06T07:21:42.311640+0000","last_unstale":"2024-03-06T07:21:56.363181+0000","last_undegraded":"2024-03-06T07:21:56.363181+0000","last_fullsized":"2024-03-06T07:21:56.363181+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:22.568258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378311+0000","last_change":"2024-03-06T07:21:55.377504+0000","last_active":"2024-03-06T07:21:56.378311+0000","last_peered":"2024-03-06T07:21:56.378311+0000","last_clean":"2024-03-06T07:21:56.378311+0000","last_became_active":"2024-03-06T07:21:55.376965+0000","last_became_peered":"2024-03-06T07:21:55.376965+0000","last_unstale":"2024-03-06T07:21:56.378311+0000","last_undegraded":"2024-03-06T07:21:56.378311+0000","last_fullsized":"2024-03-06T07:21:56.378311+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:37.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427147+0000","last_change":"2024-03-06T07:21:54.372740+0000","last_active":"2024-03-06T07:21:57.427147+0000","last_peered":"2024-03-06T07:21:57.427147+0000","last_clean":"2024-03-06T07:21:57.427147+0000","last_became_active":"2024-03-06T07:21:54.371525+0000","last_became_peered":"2024-03-06T07:21:54.371525+0000","last_unstale":"2024-03-06T07:21:57.427147+0000","last_undegraded":"2024-03-06T07:21:57.427147+0000","last_fullsized":"2024-03-06T07:21:57.427147+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:01.404942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377965+0000","last_change":"2024-03-06T07:21:43.320374+0000","last_active":"2024-03-06T07:21:56.377965+0000","last_peered":"2024-03-06T07:21:56.377965+0000","last_clean":"2024-03-06T07:21:56.377965+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:56.377965+0000","last_undegraded":"2024-03-06T07:21:56.377965+0000","last_fullsized":"2024-03-06T07:21:56.377965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.609144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":28,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358523+0000","last_change":"2024-03-06T07:21:42.309901+0000","last_active":"2024-03-06T07:21:56.358523+0000","last_peered":"2024-03-06T07:21:56.358523+0000","last_clean":"2024-03-06T07:21:56.358523+0000","last_became_active":"2024-03-06T07:21:42.309532+0000","last_became_peered":"2024-03-06T07:21:42.309532+0000","last_unstale":"2024-03-06T07:21:56.358523+0000","last_undegraded":"2024-03-06T07:21:56.358523+0000","last_fullsized":"2024-03-06T07:21:56.358523+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:08.807101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.363176+0000","last_change":"2024-03-06T07:21:55.378537+0000","last_active":"2024-03-06T07:21:56.363176+0000","last_peered":"2024-03-06T07:21:56.363176+0000","last_clean":"2024-03-06T07:21:56.363176+0000","last_became_active":"2024-03-06T07:21:55.378276+0000","last_became_peered":"2024-03-06T07:21:55.378276+0000","last_unstale":"2024-03-06T07:21:56.363176+0000","last_undegraded":"2024-03-06T07:21:56.363176+0000","last_fullsized":"2024-03-06T07:21:56.363176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:16.531191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358848+0000","last_change":"2024-03-06T07:21:54.375852+0000","last_active":"2024-03-06T07:21:56.358848+0000","last_peered":"2024-03-06T07:21:56.358848+0000","last_clean":"2024-03-06T07:21:56.358848+0000","last_became_active":"2024-03-06T07:21:54.375600+0000","last_became_peered":"2024-03-06T07:21:54.375600+0000","last_unstale":"2024-03-06T07:21:56.358848+0000","last_undegraded":"2024-03-06T07:21:56.358848+0000","last_fullsized":"2024-03-06T07:21:56.358848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:59.433599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362740+0000","last_change":"2024-03-06T07:21:43.320820+0000","last_active":"2024-03-06T07:21:56.362740+0000","last_peered":"2024-03-06T07:21:56.362740+0000","last_clean":"2024-03-06T07:21:56.362740+0000","last_became_active":"2024-03-06T07:21:43.319225+0000","last_became_peered":"2024-03-06T07:21:43.319225+0000","last_unstale":"2024-03-06T07:21:56.362740+0000","last_undegraded":"2024-03-06T07:21:56.362740+0000","last_fullsized":"2024-03-06T07:21:56.362740+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:25.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427080+0000","last_change":"2024-03-06T07:21:42.307676+0000","last_active":"2024-03-06T07:21:57.427080+0000","last_peered":"2024-03-06T07:21:57.427080+0000","last_clean":"2024-03-06T07:21:57.427080+0000","last_became_active":"2024-03-06T07:21:42.307388+0000","last_became_peered":"2024-03-06T07:21:42.307388+0000","last_unstale":"2024-03-06T07:21:57.427080+0000","last_undegraded":"2024-03-06T07:21:57.427080+0000","last_fullsized":"2024-03-06T07:21:57.427080+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:47.711911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427017+0000","last_change":"2024-03-06T07:21:55.381123+0000","last_active":"2024-03-06T07:21:57.427017+0000","last_peered":"2024-03-06T07:21:57.427017+0000","last_clean":"2024-03-06T07:21:57.427017+0000","last_became_active":"2024-03-06T07:21:55.380453+0000","last_became_peered":"2024-03-06T07:21:55.380453+0000","last_unstale":"2024-03-06T07:21:57.427017+0000","last_undegraded":"2024-03-06T07:21:57.427017+0000","last_fullsized":"2024-03-06T07:21:57.427017+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:47.260471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378445+0000","last_change":"2024-03-06T07:21:54.374823+0000","last_active":"2024-03-06T07:21:56.378445+0000","last_peered":"2024-03-06T07:21:56.378445+0000","last_clean":"2024-03-06T07:21:56.378445+0000","last_became_active":"2024-03-06T07:21:54.374515+0000","last_became_peered":"2024-03-06T07:21:54.374515+0000","last_unstale":"2024-03-06T07:21:56.378445+0000","last_undegraded":"2024-03-06T07:21:56.378445+0000","last_fullsized":"2024-03-06T07:21:56.378445+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:15.525691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427781+0000","last_change":"2024-03-06T07:21:43.329841+0000","last_active":"2024-03-06T07:21:57.427781+0000","last_peered":"2024-03-06T07:21:57.427781+0000","last_clean":"2024-03-06T07:21:57.427781+0000","last_became_active":"2024-03-06T07:21:43.329526+0000","last_became_peered":"2024-03-06T07:21:43.329526+0000","last_unstale":"2024-03-06T07:21:57.427781+0000","last_undegraded":"2024-03-06T07:21:57.427781+0000","last_fullsized":"2024-03-06T07:21:57.427781+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:31.466822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426790+0000","last_change":"2024-03-06T07:21:42.318168+0000","last_active":"2024-03-06T07:21:57.426790+0000","last_peered":"2024-03-06T07:21:57.426790+0000","last_clean":"2024-03-06T07:21:57.426790+0000","last_became_active":"2024-03-06T07:21:42.317866+0000","last_became_peered":"2024-03-06T07:21:42.317866+0000","last_unstale":"2024-03-06T07:21:57.426790+0000","last_undegraded":"2024-03-06T07:21:57.426790+0000","last_fullsized":"2024-03-06T07:21:57.426790+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:49.054000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380943+0000","last_change":"2024-03-06T07:21:55.369781+0000","last_active":"2024-03-06T07:21:56.380943+0000","last_peered":"2024-03-06T07:21:56.380943+0000","last_clean":"2024-03-06T07:21:56.380943+0000","last_became_active":"2024-03-06T07:21:55.369515+0000","last_became_peered":"2024-03-06T07:21:55.369515+0000","last_unstale":"2024-03-06T07:21:56.380943+0000","last_undegraded":"2024-03-06T07:21:56.380943+0000","last_fullsized":"2024-03-06T07:21:56.380943+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:35.630075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"56'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378461+0000","last_change":"2024-03-06T07:21:54.369742+0000","last_active":"2024-03-06T07:21:56.378461+0000","last_peered":"2024-03-06T07:21:56.378461+0000","last_clean":"2024-03-06T07:21:56.378461+0000","last_became_active":"2024-03-06T07:21:54.368872+0000","last_became_peered":"2024-03-06T07:21:54.368872+0000","last_unstale":"2024-03-06T07:21:56.378461+0000","last_undegraded":"2024-03-06T07:21:56.378461+0000","last_fullsized":"2024-03-06T07:21:56.378461+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:07.096142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.359307+0000","last_change":"2024-03-06T07:21:43.316009+0000","last_active":"2024-03-06T07:21:56.359307+0000","last_peered":"2024-03-06T07:21:56.359307+0000","last_clean":"2024-03-06T07:21:56.359307+0000","last_became_active":"2024-03-06T07:21:43.315699+0000","last_became_peered":"2024-03-06T07:21:43.315699+0000","last_unstale":"2024-03-06T07:21:56.359307+0000","last_undegraded":"2024-03-06T07:21:56.359307+0000","last_fullsized":"2024-03-06T07:21:56.359307+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:52.769462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.380475+0000","last_change":"2024-03-06T07:21:42.311690+0000","last_active":"2024-03-06T07:21:56.380475+0000","last_peered":"2024-03-06T07:21:56.380475+0000","last_clean":"2024-03-06T07:21:56.380475+0000","last_became_active":"2024-03-06T07:21:42.311417+0000","last_became_peered":"2024-03-06T07:21:42.311417+0000","last_unstale":"2024-03-06T07:21:56.380475+0000","last_undegraded":"2024-03-06T07:21:56.380475+0000","last_fullsized":"2024-03-06T07:21:56.380475+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.094878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362403+0000","last_change":"2024-03-06T07:21:55.378491+0000","last_active":"2024-03-06T07:21:56.362403+0000","last_peered":"2024-03-06T07:21:56.362403+0000","last_clean":"2024-03-06T07:21:56.362403+0000","last_became_active":"2024-03-06T07:21:55.376381+0000","last_became_peered":"2024-03-06T07:21:55.376381+0000","last_unstale":"2024-03-06T07:21:56.362403+0000","last_undegraded":"2024-03-06T07:21:56.362403+0000","last_fullsized":"2024-03-06T07:21:56.362403+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:15.092487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.425961+0000","last_change":"2024-03-06T07:21:54.377550+0000","last_active":"2024-03-06T07:21:57.425961+0000","last_peered":"2024-03-06T07:21:57.425961+0000","last_clean":"2024-03-06T07:21:57.425961+0000","last_became_active":"2024-03-06T07:21:54.377271+0000","last_became_peered":"2024-03-06T07:21:54.377271+0000","last_unstale":"2024-03-06T07:21:57.425961+0000","last_undegraded":"2024-03-06T07:21:57.425961+0000","last_fullsized":"2024-03-06T07:21:57.425961+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:10:52.116984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377908+0000","last_change":"2024-03-06T07:21:43.322229+0000","last_active":"2024-03-06T07:21:56.377908+0000","last_peered":"2024-03-06T07:21:56.377908+0000","last_clean":"2024-03-06T07:21:56.377908+0000","last_became_active":"2024-03-06T07:21:43.321934+0000","last_became_peered":"2024-03-06T07:21:43.321934+0000","last_unstale":"2024-03-06T07:21:56.377908+0000","last_undegraded":"2024-03-06T07:21:56.377908+0000","last_fullsized":"2024-03-06T07:21:56.377908+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:29.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":28,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.363103+0000","last_change":"2024-03-06T07:21:42.321714+0000","last_active":"2024-03-06T07:21:56.363103+0000","last_peered":"2024-03-06T07:21:56.363103+0000","last_clean":"2024-03-06T07:21:56.363103+0000","last_became_active":"2024-03-06T07:21:42.321444+0000","last_became_peered":"2024-03-06T07:21:42.321444+0000","last_unstale":"2024-03-06T07:21:56.363103+0000","last_undegraded":"2024-03-06T07:21:56.363103+0000","last_fullsized":"2024-03-06T07:21:56.363103+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:09.704068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426064+0000","last_change":"2024-03-06T07:21:55.367907+0000","last_active":"2024-03-06T07:21:57.426064+0000","last_peered":"2024-03-06T07:21:57.426064+0000","last_clean":"2024-03-06T07:21:57.426064+0000","last_became_active":"2024-03-06T07:21:55.367679+0000","last_became_peered":"2024-03-06T07:21:55.367679+0000","last_unstale":"2024-03-06T07:21:57.426064+0000","last_undegraded":"2024-03-06T07:21:57.426064+0000","last_fullsized":"2024-03-06T07:21:57.426064+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:17.026034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427201+0000","last_change":"2024-03-06T07:21:54.371666+0000","last_active":"2024-03-06T07:21:57.427201+0000","last_peered":"2024-03-06T07:21:57.427201+0000","last_clean":"2024-03-06T07:21:57.427201+0000","last_became_active":"2024-03-06T07:21:54.371394+0000","last_became_peered":"2024-03-06T07:21:54.371394+0000","last_unstale":"2024-03-06T07:21:57.427201+0000","last_undegraded":"2024-03-06T07:21:57.427201+0000","last_fullsized":"2024-03-06T07:21:57.427201+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:28.029299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.358402+0000","last_change":"2024-03-06T07:21:43.316403+0000","last_active":"2024-03-06T07:21:56.358402+0000","last_peered":"2024-03-06T07:21:56.358402+0000","last_clean":"2024-03-06T07:21:56.358402+0000","last_became_active":"2024-03-06T07:21:43.315416+0000","last_became_peered":"2024-03-06T07:21:43.315416+0000","last_unstale":"2024-03-06T07:21:56.358402+0000","last_undegraded":"2024-03-06T07:21:56.358402+0000","last_fullsized":"2024-03-06T07:21:56.358402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:32.706894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427250+0000","last_change":"2024-03-06T07:21:42.311790+0000","last_active":"2024-03-06T07:21:57.427250+0000","last_peered":"2024-03-06T07:21:57.427250+0000","last_clean":"2024-03-06T07:21:57.427250+0000","last_became_active":"2024-03-06T07:21:42.311500+0000","last_became_peered":"2024-03-06T07:21:42.311500+0000","last_unstale":"2024-03-06T07:21:57.427250+0000","last_undegraded":"2024-03-06T07:21:57.427250+0000","last_fullsized":"2024-03-06T07:21:57.427250+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:24.763487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377582+0000","last_change":"2024-03-06T07:21:55.375304+0000","last_active":"2024-03-06T07:21:56.377582+0000","last_peered":"2024-03-06T07:21:56.377582+0000","last_clean":"2024-03-06T07:21:56.377582+0000","last_became_active":"2024-03-06T07:21:55.374827+0000","last_became_peered":"2024-03-06T07:21:55.374827+0000","last_unstale":"2024-03-06T07:21:56.377582+0000","last_undegraded":"2024-03-06T07:21:56.377582+0000","last_fullsized":"2024-03-06T07:21:56.377582+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:29.034799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.377749+0000","last_change":"2024-03-06T07:21:54.369593+0000","last_active":"2024-03-06T07:21:56.377749+0000","last_peered":"2024-03-06T07:21:56.377749+0000","last_clean":"2024-03-06T07:21:56.377749+0000","last_became_active":"2024-03-06T07:21:54.369336+0000","last_became_peered":"2024-03-06T07:21:54.369336+0000","last_unstale":"2024-03-06T07:21:56.377749+0000","last_undegraded":"2024-03-06T07:21:56.377749+0000","last_fullsized":"2024-03-06T07:21:56.377749+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:32.049609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426761+0000","last_change":"2024-03-06T07:21:43.319244+0000","last_active":"2024-03-06T07:21:57.426761+0000","last_peered":"2024-03-06T07:21:57.426761+0000","last_clean":"2024-03-06T07:21:57.426761+0000","last_became_active":"2024-03-06T07:21:43.318994+0000","last_became_peered":"2024-03-06T07:21:43.318994+0000","last_unstale":"2024-03-06T07:21:57.426761+0000","last_undegraded":"2024-03-06T07:21:57.426761+0000","last_fullsized":"2024-03-06T07:21:57.426761+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:50.744474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427284+0000","last_change":"2024-03-06T07:21:42.315718+0000","last_active":"2024-03-06T07:21:57.427284+0000","last_peered":"2024-03-06T07:21:57.427284+0000","last_clean":"2024-03-06T07:21:57.427284+0000","last_became_active":"2024-03-06T07:21:42.315402+0000","last_became_peered":"2024-03-06T07:21:42.315402+0000","last_unstale":"2024-03-06T07:21:57.427284+0000","last_undegraded":"2024-03-06T07:21:57.427284+0000","last_fullsized":"2024-03-06T07:21:57.427284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:46.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427667+0000","last_change":"2024-03-06T07:21:55.384421+0000","last_active":"2024-03-06T07:21:57.427667+0000","last_peered":"2024-03-06T07:21:57.427667+0000","last_clean":"2024-03-06T07:21:57.427667+0000","last_became_active":"2024-03-06T07:21:55.384088+0000","last_became_peered":"2024-03-06T07:21:55.384088+0000","last_unstale":"2024-03-06T07:21:57.427667+0000","last_undegraded":"2024-03-06T07:21:57.427667+0000","last_fullsized":"2024-03-06T07:21:57.427667+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:35.166392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426841+0000","last_change":"2024-03-06T07:21:54.375429+0000","last_active":"2024-03-06T07:21:57.426841+0000","last_peered":"2024-03-06T07:21:57.426841+0000","last_clean":"2024-03-06T07:21:57.426841+0000","last_became_active":"2024-03-06T07:21:54.375117+0000","last_became_peered":"2024-03-06T07:21:54.375117+0000","last_unstale":"2024-03-06T07:21:57.426841+0000","last_undegraded":"2024-03-06T07:21:57.426841+0000","last_fullsized":"2024-03-06T07:21:57.426841+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:15.525691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427201+0000","last_change":"2024-03-06T07:21:43.323135+0000","last_active":"2024-03-06T07:21:57.427201+0000","last_peered":"2024-03-06T07:21:57.427201+0000","last_clean":"2024-03-06T07:21:57.427201+0000","last_became_active":"2024-03-06T07:21:43.322814+0000","last_became_peered":"2024-03-06T07:21:43.322814+0000","last_unstale":"2024-03-06T07:21:57.427201+0000","last_undegraded":"2024-03-06T07:21:57.427201+0000","last_fullsized":"2024-03-06T07:21:57.427201+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:24.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426975+0000","last_change":"2024-03-06T07:21:42.308451+0000","last_active":"2024-03-06T07:21:57.426975+0000","last_peered":"2024-03-06T07:21:57.426975+0000","last_clean":"2024-03-06T07:21:57.426975+0000","last_became_active":"2024-03-06T07:21:42.308167+0000","last_became_peered":"2024-03-06T07:21:42.308167+0000","last_unstale":"2024-03-06T07:21:57.426975+0000","last_undegraded":"2024-03-06T07:21:57.426975+0000","last_fullsized":"2024-03-06T07:21:57.426975+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:31.703072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362829+0000","last_change":"2024-03-06T07:21:55.379453+0000","last_active":"2024-03-06T07:21:56.362829+0000","last_peered":"2024-03-06T07:21:56.362829+0000","last_clean":"2024-03-06T07:21:56.362829+0000","last_became_active":"2024-03-06T07:21:55.379207+0000","last_became_peered":"2024-03-06T07:21:55.379207+0000","last_unstale":"2024-03-06T07:21:56.362829+0000","last_undegraded":"2024-03-06T07:21:56.362829+0000","last_fullsized":"2024-03-06T07:21:56.362829+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:09.618177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427309+0000","last_change":"2024-03-06T07:21:54.375869+0000","last_active":"2024-03-06T07:21:57.427309+0000","last_peered":"2024-03-06T07:21:57.427309+0000","last_clean":"2024-03-06T07:21:57.427309+0000","last_became_active":"2024-03-06T07:21:54.375586+0000","last_became_peered":"2024-03-06T07:21:54.375586+0000","last_unstale":"2024-03-06T07:21:57.427309+0000","last_undegraded":"2024-03-06T07:21:57.427309+0000","last_fullsized":"2024-03-06T07:21:57.427309+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:02.300240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378122+0000","last_change":"2024-03-06T07:21:43.324622+0000","last_active":"2024-03-06T07:21:56.378122+0000","last_peered":"2024-03-06T07:21:56.378122+0000","last_clean":"2024-03-06T07:21:56.378122+0000","last_became_active":"2024-03-06T07:21:43.324350+0000","last_became_peered":"2024-03-06T07:21:43.324350+0000","last_unstale":"2024-03-06T07:21:56.378122+0000","last_undegraded":"2024-03-06T07:21:56.378122+0000","last_fullsized":"2024-03-06T07:21:56.378122+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:02.073150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427100+0000","last_change":"2024-03-06T07:21:42.309246+0000","last_active":"2024-03-06T07:21:57.427100+0000","last_peered":"2024-03-06T07:21:57.427100+0000","last_clean":"2024-03-06T07:21:57.427100+0000","last_became_active":"2024-03-06T07:21:42.308990+0000","last_became_peered":"2024-03-06T07:21:42.308990+0000","last_unstale":"2024-03-06T07:21:57.427100+0000","last_undegraded":"2024-03-06T07:21:57.427100+0000","last_fullsized":"2024-03-06T07:21:57.427100+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:51.765639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426695+0000","last_change":"2024-03-06T07:21:55.382093+0000","last_active":"2024-03-06T07:21:57.426695+0000","last_peered":"2024-03-06T07:21:57.426695+0000","last_clean":"2024-03-06T07:21:57.426695+0000","last_became_active":"2024-03-06T07:21:55.381823+0000","last_became_peered":"2024-03-06T07:21:55.381823+0000","last_unstale":"2024-03-06T07:21:57.426695+0000","last_undegraded":"2024-03-06T07:21:57.426695+0000","last_fullsized":"2024-03-06T07:21:57.426695+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:10.249527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378273+0000","last_change":"2024-03-06T07:21:54.371835+0000","last_active":"2024-03-06T07:21:56.378273+0000","last_peered":"2024-03-06T07:21:56.378273+0000","last_clean":"2024-03-06T07:21:56.378273+0000","last_became_active":"2024-03-06T07:21:54.371518+0000","last_became_peered":"2024-03-06T07:21:54.371518+0000","last_unstale":"2024-03-06T07:21:56.378273+0000","last_undegraded":"2024-03-06T07:21:56.378273+0000","last_fullsized":"2024-03-06T07:21:56.378273+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:21:59.123812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427692+0000","last_change":"2024-03-06T07:21:43.329441+0000","last_active":"2024-03-06T07:21:57.427692+0000","last_peered":"2024-03-06T07:21:57.427692+0000","last_clean":"2024-03-06T07:21:57.427692+0000","last_became_active":"2024-03-06T07:21:43.327694+0000","last_became_peered":"2024-03-06T07:21:43.327694+0000","last_unstale":"2024-03-06T07:21:57.427692+0000","last_undegraded":"2024-03-06T07:21:57.427692+0000","last_fullsized":"2024-03-06T07:21:57.427692+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:03.034493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426312+0000","last_change":"2024-03-06T07:21:42.309614+0000","last_active":"2024-03-06T07:21:57.426312+0000","last_peered":"2024-03-06T07:21:57.426312+0000","last_clean":"2024-03-06T07:21:57.426312+0000","last_became_active":"2024-03-06T07:21:42.309319+0000","last_became_peered":"2024-03-06T07:21:42.309319+0000","last_unstale":"2024-03-06T07:21:57.426312+0000","last_undegraded":"2024-03-06T07:21:57.426312+0000","last_fullsized":"2024-03-06T07:21:57.426312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:28.385954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362588+0000","last_change":"2024-03-06T07:21:55.390300+0000","last_active":"2024-03-06T07:21:56.362588+0000","last_peered":"2024-03-06T07:21:56.362588+0000","last_clean":"2024-03-06T07:21:56.362588+0000","last_became_active":"2024-03-06T07:21:55.389757+0000","last_became_peered":"2024-03-06T07:21:55.389757+0000","last_unstale":"2024-03-06T07:21:56.362588+0000","last_undegraded":"2024-03-06T07:21:56.362588+0000","last_fullsized":"2024-03-06T07:21:56.362588+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:35.166392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427656+0000","last_change":"2024-03-06T07:21:54.375555+0000","last_active":"2024-03-06T07:21:57.427656+0000","last_peered":"2024-03-06T07:21:57.427656+0000","last_clean":"2024-03-06T07:21:57.427656+0000","last_became_active":"2024-03-06T07:21:54.375251+0000","last_became_peered":"2024-03-06T07:21:54.375251+0000","last_unstale":"2024-03-06T07:21:57.427656+0000","last_undegraded":"2024-03-06T07:21:57.427656+0000","last_fullsized":"2024-03-06T07:21:57.427656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:17.202770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426764+0000","last_change":"2024-03-06T07:21:43.321711+0000","last_active":"2024-03-06T07:21:57.426764+0000","last_peered":"2024-03-06T07:21:57.426764+0000","last_clean":"2024-03-06T07:21:57.426764+0000","last_became_active":"2024-03-06T07:21:43.321434+0000","last_became_peered":"2024-03-06T07:21:43.321434+0000","last_unstale":"2024-03-06T07:21:57.426764+0000","last_undegraded":"2024-03-06T07:21:57.426764+0000","last_fullsized":"2024-03-06T07:21:57.426764+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:06.150276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":26,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.378058+0000","last_change":"2024-03-06T07:21:42.310672+0000","last_active":"2024-03-06T07:21:56.378058+0000","last_peered":"2024-03-06T07:21:56.378058+0000","last_clean":"2024-03-06T07:21:56.378058+0000","last_became_active":"2024-03-06T07:21:42.310366+0000","last_became_peered":"2024-03-06T07:21:42.310366+0000","last_unstale":"2024-03-06T07:21:56.378058+0000","last_undegraded":"2024-03-06T07:21:56.378058+0000","last_fullsized":"2024-03-06T07:21:56.378058+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:24.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362556+0000","last_change":"2024-03-06T07:21:55.375875+0000","last_active":"2024-03-06T07:21:56.362556+0000","last_peered":"2024-03-06T07:21:56.362556+0000","last_clean":"2024-03-06T07:21:56.362556+0000","last_became_active":"2024-03-06T07:21:55.375611+0000","last_became_peered":"2024-03-06T07:21:55.375611+0000","last_unstale":"2024-03-06T07:21:56.362556+0000","last_undegraded":"2024-03-06T07:21:56.362556+0000","last_fullsized":"2024-03-06T07:21:56.362556+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:54.349222+0000","last_clean_scrub_stamp":"2024-03-06T07:21:54.349222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:55.035893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.425811+0000","last_change":"2024-03-06T07:21:54.366502+0000","last_active":"2024-03-06T07:21:57.425811+0000","last_peered":"2024-03-06T07:21:57.425811+0000","last_clean":"2024-03-06T07:21:57.425811+0000","last_became_active":"2024-03-06T07:21:54.366066+0000","last_became_peered":"2024-03-06T07:21:54.366066+0000","last_unstale":"2024-03-06T07:21:57.425811+0000","last_undegraded":"2024-03-06T07:21:57.425811+0000","last_fullsized":"2024-03-06T07:21:57.425811+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:53.343722+0000","last_clean_scrub_stamp":"2024-03-06T07:21:53.343722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:28.029299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":24,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:56.362131+0000","last_change":"2024-03-06T07:21:43.319233+0000","last_active":"2024-03-06T07:21:56.362131+0000","last_peered":"2024-03-06T07:21:56.362131+0000","last_clean":"2024-03-06T07:21:56.362131+0000","last_became_active":"2024-03-06T07:21:43.318838+0000","last_became_peered":"2024-03-06T07:21:43.318838+0000","last_unstale":"2024-03-06T07:21:56.362131+0000","last_undegraded":"2024-03-06T07:21:56.362131+0000","last_fullsized":"2024-03-06T07:21:56.362131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.291229+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.291229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:47.590404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":27,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.427787+0000","last_change":"2024-03-06T07:21:42.322036+0000","last_active":"2024-03-06T07:21:57.427787+0000","last_peered":"2024-03-06T07:21:57.427787+0000","last_clean":"2024-03-06T07:21:57.427787+0000","last_became_active":"2024-03-06T07:21:42.321385+0000","last_became_peered":"2024-03-06T07:21:42.321385+0000","last_unstale":"2024-03-06T07:21:57.427787+0000","last_undegraded":"2024-03-06T07:21:57.427787+0000","last_fullsized":"2024-03-06T07:21:57.427787+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.287406+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.287406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:31.798561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-03-06T07:21:57.426238+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:21:57.426238+0000","last_peered":"2024-03-06T07:21:57.426238+0000","last_clean":"2024-03-06T07:21:57.426238+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:21:57.426238+0000","last_undegraded":"2024-03-06T07:21:57.426238+0000","last_fullsized":"2024-03-06T07:21:57.426238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":898119,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":307537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":306873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":309045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":684032,"data_stored":308485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":679936,"data_stored":308463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":309387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":741376,"data_stored":897275,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:04.219 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:22:04.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 3:0 2024-03-06T07:22:04.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.357+0000 7f85a4caf640 1 Processor -- start 2024-03-06T07:22:04.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f85a4caf640 1 -- start start 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f85a4caf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a00c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f85a4caf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a0155150 con 0x7f85a00c27b0 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f85a4caf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a01553d0 con 0x7f85a0158720 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f85a4caf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0155d00 con 0x7f85a0152740 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f8597fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f85a00c27b0 0x7f85a006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56826/0 (socket says 172.21.15.26:56826) 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f8597fff640 1 -- 172.21.15.26:0/4244826613 learned_addr learned my addr 172.21.15.26:0/4244826613 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.358+0000 7f859e59c640 1 --2- 172.21.15.26:0/4244826613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a00c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.359+0000 7f859dd9b640 1 -- 172.21.15.26:0/4244826613 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044008239 0 0) 0x7f85a01553d0 con 0x7f85a0158720 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.359+0000 7f859dd9b640 1 -- 172.21.15.26:0/4244826613 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f858c003520 con 0x7f85a0158720 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.359+0000 7f859dd9b640 1 -- 172.21.15.26:0/4244826613 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49775751 0 0) 0x7f8588001110 con 0x7f85a00c27b0 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.359+0000 7f859dd9b640 1 -- 172.21.15.26:0/4244826613 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a01553d0 con 0x7f85a00c27b0 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.359+0000 7f859e59c640 1 -- 172.21.15.26:0/4244826613 >> v1:172.21.15.117:6789/0 conn(0x7f85a0158720 legacy=0x7f85a0158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.359+0000 7f859e59c640 1 -- 172.21.15.26:0/4244826613 >> v1:172.21.15.26:6789/0 conn(0x7f85a00c27b0 legacy=0x7f85a006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.359+0000 7f859e59c640 1 -- 172.21.15.26:0/4244826613 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a01556b0 con 0x7f85a0152740 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f859e59c640 1 --2- 172.21.15.26:0/4244826613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a00c20b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85900097d0 tx=0x7f859002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299ea1093442d597 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f859dd9b640 1 -- 172.21.15.26:0/4244826613 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590035020 con 0x7f85a0152740 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f859dd9b640 1 -- 172.21.15.26:0/4244826613 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8590031800 con 0x7f85a0152740 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f859dd9b640 1 -- 172.21.15.26:0/4244826613 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590031bd0 con 0x7f85a0152740 2024-03-06T07:22:04.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f85a4caf640 1 -- 172.21.15.26:0/4244826613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 msgr2=0x7f85a00c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f85a4caf640 1 --2- 172.21.15.26:0/4244826613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a00c20b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85900097d0 tx=0x7f859002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f85a4caf640 1 -- 172.21.15.26:0/4244826613 shutdown_connections 2024-03-06T07:22:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f85a4caf640 1 --2- 172.21.15.26:0/4244826613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a00c20b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f85a4caf640 1 -- 172.21.15.26:0/4244826613 >> 172.21.15.26:0/4244826613 conn(0x7f85a00c41b0 msgr2=0x7f85a00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f85a4caf640 1 -- 172.21.15.26:0/4244826613 shutdown_connections 2024-03-06T07:22:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.360+0000 7f85a4caf640 1 -- 172.21.15.26:0/4244826613 wait complete. 2024-03-06T07:22:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 Processor -- start 2024-03-06T07:22:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 -- start start 2024-03-06T07:22:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a00c27b0 0x7f85a0150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a014e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0158720 0x7f85a01eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a006be80 con 0x7f85a0158720 2024-03-06T07:22:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0159280 con 0x7f85a00c27b0 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f85a4caf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0152b20 con 0x7f85a0152740 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f859ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0158720 0x7f85a01eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f859ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0158720 0x7f85a01eabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57930/0 (socket says 172.21.15.26:57930) 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f859ed9d640 1 -- 172.21.15.26:0/3520857987 learned_addr learned my addr 172.21.15.26:0/3520857987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f8597fff640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a014e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.362+0000 7f859e59c640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a00c27b0 0x7f85a0150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f859ed9d640 1 -- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 msgr2=0x7f85a014e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f859ed9d640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a014e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f859ed9d640 1 -- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a00c27b0 msgr2=0x7f85a0150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f859ed9d640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a00c27b0 0x7f85a0150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f859ed9d640 1 -- 172.21.15.26:0/3520857987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a014f560 con 0x7f85a0158720 2024-03-06T07:22:04.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f859ed9d640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0158720 0x7f85a01eabd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f859800c700 tx=0x7f859800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f8596ffd640 1 -- 172.21.15.26:0/3520857987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598003b70 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f8596ffd640 1 -- 172.21.15.26:0/3520857987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598003d30 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a0150780 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f8596ffd640 1 -- 172.21.15.26:0/3520857987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800aa90 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.363+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a01ee670 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.364+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a0072160 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.365+0000 7f8596ffd640 1 -- 172.21.15.26:0/3520857987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f859800ac30 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.365+0000 7f8596ffd640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.365+0000 7f8596ffd640 1 -- 172.21.15.26:0/3520857987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f859808a470 con 0x7f85a0158720 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.365+0000 7f859e59c640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.365+0000 7f859e59c640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f858006b4d0 0x7f858006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8590036000 tx=0x7f8590008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.369+0000 7f8596ffd640 1 -- 172.21.15.26:0/3520857987 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f85a0072160 con 0x7f85a0158720 2024-03-06T07:22:04.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.620+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "3:0"} v 0) v1 -- 0x7f85a006e920 con 0x7f85a0158720 2024-03-06T07:22:04.623 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:22:04.621+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.622+0000 7f8596ffd640 1 -- 172.21.15.26:0/3520857987 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "3:0"}]=0 failed mds gid 5697 v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8598058810 con 0x7f85a0158720 2024-03-06T07:22:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:failed mds gid 5697 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f858006b4d0 msgr2=0x7f858006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f858006b4d0 0x7f858006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8590036000 tx=0x7f8590008030 comp rx=0 tx=0).stop 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0158720 msgr2=0x7f85a01eabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0158720 0x7f85a01eabd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f859800c700 tx=0x7f859800cbc0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 shutdown_connections 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f858006b4d0 0x7f858006d990 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8590036000 tx=0x7f8590008030 comp rx=0 tx=0).stop 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0158720 0x7f85a01eabd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a0152740 0x7f85a014e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 --2- 172.21.15.26:0/3520857987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a00c27b0 0x7f85a0150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 >> 172.21.15.26:0/3520857987 conn(0x7f85a00c41b0 msgr2=0x7f85a0152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 shutdown_connections 2024-03-06T07:22:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.625+0000 7f85a4caf640 1 -- 172.21.15.26:0/3520857987 wait complete. 2024-03-06T07:22:04.628 INFO:tasks.ceph.mds.c.smithi026.stderr: -573> 2024-03-06T07:21:48.602+0000 7f11bce8a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:04.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:04.655 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:22:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.742+0000 7fa9486ed640 1 Processor -- start 2024-03-06T07:22:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.742+0000 7fa9486ed640 1 -- start start 2024-03-06T07:22:04.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa9486ed640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa94014f790 0x7fa9401529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa9486ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa940155a40 0x7fa940155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa9486ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9401563d0 con 0x7fa9400c1aa0 2024-03-06T07:22:04.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa9486ed640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400c2680 con 0x7fa94014f790 2024-03-06T07:22:04.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa9486ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400c28d0 con 0x7fa940155a40 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa945c88640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa9400c1aa0 0x7fa9400c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56844/0 (socket says 172.21.15.26:56844) 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa945c88640 1 -- 172.21.15.26:0/972924296 learned_addr learned my addr 172.21.15.26:0/972924296 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa946489640 1 --2- 172.21.15.26:0/972924296 >> v2:172.21.15.117:3300/0 conn(0x7fa94014f790 0x7fa9401529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa946c8a640 1 --2- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa940155a40 0x7fa940155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa945487640 1 -- 172.21.15.26:0/972924296 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633958112 0 0) 0x7fa9401563d0 con 0x7fa9400c1aa0 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa945487640 1 -- 172.21.15.26:0/972924296 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa92c003520 con 0x7fa9400c1aa0 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa946489640 1 -- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa940155a40 msgr2=0x7fa940155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa946489640 1 --2- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa940155a40 0x7fa940155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa946489640 1 -- 172.21.15.26:0/972924296 >> v1:172.21.15.26:6789/0 conn(0x7fa9400c1aa0 legacy=0x7fa9400c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.743+0000 7fa946489640 1 -- 172.21.15.26:0/972924296 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400c2bb0 con 0x7fa94014f790 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.744+0000 7fa946489640 1 --2- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa94014f790 0x7fa9401529d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0097a0 tx=0x7fa93c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a045c14986152d56 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.744+0000 7fa945487640 1 -- 172.21.15.26:0/972924296 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c035020 con 0x7fa94014f790 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.744+0000 7fa945487640 1 -- 172.21.15.26:0/972924296 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c031830 con 0x7fa94014f790 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.744+0000 7fa945487640 1 -- 172.21.15.26:0/972924296 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c031be0 con 0x7fa94014f790 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 -- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa94014f790 msgr2=0x7fa9401529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 --2- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa94014f790 0x7fa9401529d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0097a0 tx=0x7fa93c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 -- 172.21.15.26:0/972924296 shutdown_connections 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 --2- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa940155a40 0x7fa940155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 --2- 172.21.15.26:0/972924296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa94014f790 0x7fa9401529d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 -- 172.21.15.26:0/972924296 >> 172.21.15.26:0/972924296 conn(0x7fa9400c6710 msgr2=0x7fa9400c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 -- 172.21.15.26:0/972924296 shutdown_connections 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.745+0000 7fa9486ed640 1 -- 172.21.15.26:0/972924296 wait complete. 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 Processor -- start 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 -- start start 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9400c1aa0 0x7fa9401de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa94014f790 0x7fa9401dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa940155a40 0x7fa940147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400c0d60 con 0x7fa940155a40 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa940155520 con 0x7fa9400c1aa0 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.746+0000 7fa9486ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400c5e30 con 0x7fa94014f790 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa940155a40 0x7fa940147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa940155a40 0x7fa940147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57954/0 (socket says 172.21.15.26:57954) 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 -- 172.21.15.26:0/1372768690 learned_addr learned my addr 172.21.15.26:0/1372768690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946489640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9400c1aa0 0x7fa9401de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa945c88640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa94014f790 0x7fa9401dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 -- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa94014f790 msgr2=0x7fa9401dbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa94014f790 0x7fa9401dbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 -- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9400c1aa0 msgr2=0x7fa9401de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9400c1aa0 0x7fa9401de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 -- 172.21.15.26:0/1372768690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9401510b0 con 0x7fa940155a40 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa946c8a640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa940155a40 0x7fa940147960 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa93800efc0 tx=0x7fa93800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa936ffd640 1 -- 172.21.15.26:0/1372768690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938018020 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa936ffd640 1 -- 172.21.15.26:0/1372768690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938014480 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa940150e10 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa936ffd640 1 -- 172.21.15.26:0/1372768690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938014870 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.747+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa940155220 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.748+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa940150860 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.749+0000 7fa936ffd640 1 -- 172.21.15.26:0/1372768690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa938005d10 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.749+0000 7fa936ffd640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa92406b4d0 0x7fa92406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.749+0000 7fa936ffd640 1 -- 172.21.15.26:0/1372768690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7101+0+0 (secure 0 0 0) 0x7fa93808b040 con 0x7fa940155a40 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.749+0000 7fa946489640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa92406b4d0 0x7fa92406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.750+0000 7fa946489640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa92406b4d0 0x7fa92406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa93c036000 tx=0x7fa93c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.753+0000 7fa936ffd640 1 -- 172.21.15.26:0/1372768690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa938022070 con 0x7fa940155a40 2024-03-06T07:22:04.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.974+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa940152210 con 0x7fa940155a40 2024-03-06T07:22:04.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.974+0000 7fa936ffd640 1 -- 172.21.15.26:0/1372768690 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5462 (secure 0 0 0) 0x7fa938010040 con 0x7fa940155a40 2024-03-06T07:22:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:22:04.626219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5234":{"gid":5234,"name":"a","rank":0,"incarnation":36,"state":"up:replay","state_seq":6,"addr":"172.21.15.26:6834/1383022235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1383022235},{"type":"v1","addr":"172.21.15.26:6834","nonce":1383022235}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 36 2024-03-06T07:22:04.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa92406b4d0 msgr2=0x7fa92406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa92406b4d0 0x7fa92406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa93c036000 tx=0x7fa93c008030 comp rx=0 tx=0).stop 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa940155a40 msgr2=0x7fa940147960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa940155a40 0x7fa940147960 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa93800efc0 tx=0x7fa93800c490 comp rx=0 tx=0).stop 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 shutdown_connections 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa92406b4d0 0x7fa92406d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa940155a40 0x7fa940147960 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa94014f790 0x7fa9401dbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 --2- 172.21.15.26:0/1372768690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9400c1aa0 0x7fa9401de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 >> 172.21.15.26:0/1372768690 conn(0x7fa9400c6710 msgr2=0x7fa9400a7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 shutdown_connections 2024-03-06T07:22:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:04.977+0000 7fa9486ed640 1 -- 172.21.15.26:0/1372768690 wait complete. 2024-03-06T07:22:04.989 INFO:tasks.cephfs.test_failover:Waiting for former active to reclaim its spot 2024-03-06T07:22:04.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-03-06T07:22:05.138 INFO:teuthology.orchestra.run.smithi026.stdout:{"mds_beacon_grace":"15.000000"} 2024-03-06T07:22:05.138 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-03-06T07:22:05.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.238+0000 7fbdc0d48640 1 Processor -- start 2024-03-06T07:22:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.238+0000 7fbdc0d48640 1 -- start start 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdc0d48640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbdbc152920 0x7fbdbc155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdc0d48640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc0c8a00 0x7fbdbc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdc0d48640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdbc159460 0x7fbdbc0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdc0d48640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc1587b0 con 0x7fbdbc159460 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdc0d48640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc156180 con 0x7fbdbc152920 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdc0d48640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0c7760 con 0x7fbdbc0c8a00 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdbad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdbc159460 0x7fbdbc0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdbad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdbc159460 0x7fbdbc0c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57966/0 (socket says 172.21.15.26:57966) 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdbad9d640 1 -- 172.21.15.26:0/4178565562 learned_addr learned my addr 172.21.15.26:0/4178565562 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdb9d9b640 1 --2- 172.21.15.26:0/4178565562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc0c8a00 0x7fbdbc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.239+0000 7fbdba59c640 1 --2- 172.21.15.26:0/4178565562 >> v2:172.21.15.117:3300/0 conn(0x7fbdbc152920 0x7fbdbc155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdbad9d640 1 -- 172.21.15.26:0/4178565562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc0c8a00 msgr2=0x7fbdbc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdbad9d640 1 --2- 172.21.15.26:0/4178565562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc0c8a00 0x7fbdbc158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdbad9d640 1 -- 172.21.15.26:0/4178565562 >> v2:172.21.15.117:3300/0 conn(0x7fbdbc152920 msgr2=0x7fbdbc155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdbad9d640 1 --2- 172.21.15.26:0/4178565562 >> v2:172.21.15.117:3300/0 conn(0x7fbdbc152920 0x7fbdbc155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdbad9d640 1 -- 172.21.15.26:0/4178565562 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc0c7a40 con 0x7fbdbc159460 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdba59c640 1 --2- 172.21.15.26:0/4178565562 >> v2:172.21.15.117:3300/0 conn(0x7fbdbc152920 0x7fbdbc155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdb9d9b640 1 --2- 172.21.15.26:0/4178565562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc0c8a00 0x7fbdbc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdbad9d640 1 --2- 172.21.15.26:0/4178565562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc159460 0x7fbdbc0c5330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0be490 tx=0x7fbdb402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23344203eae386eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdb959a640 1 -- 172.21.15.26:0/4178565562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb402c9e0 con 0x7fbdbc159460 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdb959a640 1 -- 172.21.15.26:0/4178565562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb402cb80 con 0x7fbdbc159460 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.240+0000 7fbdb959a640 1 -- 172.21.15.26:0/4178565562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40343f0 con 0x7fbdbc159460 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 -- 172.21.15.26:0/4178565562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc159460 msgr2=0x7fbdbc0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/4178565562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc159460 0x7fbdbc0c5330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0be490 tx=0x7fbdb402be40 comp rx=0 tx=0).stop 2024-03-06T07:22:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 -- 172.21.15.26:0/4178565562 shutdown_connections 2024-03-06T07:22:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/4178565562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc0c8a00 0x7fbdbc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/4178565562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc159460 0x7fbdbc0c5330 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/4178565562 >> v2:172.21.15.117:3300/0 conn(0x7fbdbc152920 0x7fbdbc155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 -- 172.21.15.26:0/4178565562 >> 172.21.15.26:0/4178565562 conn(0x7fbdbc0ac560 msgr2=0x7fbdbc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 -- 172.21.15.26:0/4178565562 shutdown_connections 2024-03-06T07:22:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.241+0000 7fbdc0d48640 1 -- 172.21.15.26:0/4178565562 wait complete. 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.242+0000 7fbdc0d48640 1 Processor -- start 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.242+0000 7fbdc0d48640 1 -- start start 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdc0d48640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdbc0c8a00 0x7fbdbc0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdc0d48640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc152920 0x7fbdbc156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdc0d48640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc1568a0 0x7fbdbc0bca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdc0d48640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc06d030 con 0x7fbdbc152920 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdc0d48640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc158a00 con 0x7fbdbc0c8a00 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdc0d48640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0c7390 con 0x7fbdbc1568a0 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc152920 0x7fbdbc156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc152920 0x7fbdbc156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57980/0 (socket says 172.21.15.26:57980) 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 -- 172.21.15.26:0/107856690 learned_addr learned my addr 172.21.15.26:0/107856690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdba59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdbc0c8a00 0x7fbdbc0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdbad9d640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc1568a0 0x7fbdbc0bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 -- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc1568a0 msgr2=0x7fbdbc0bca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc1568a0 0x7fbdbc0bca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 -- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdbc0c8a00 msgr2=0x7fbdbc0ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdbc0c8a00 0x7fbdbc0ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdb9d9b640 1 -- 172.21.15.26:0/107856690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc158560 con 0x7fbdbc152920 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.243+0000 7fbdba59c640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdbc0c8a00 0x7fbdbc0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.244+0000 7fbdb9d9b640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc152920 0x7fbdbc156360 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000e8e0 tx=0x7fbdb000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.244+0000 7fbdaaffd640 1 -- 172.21.15.26:0/107856690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0017070 con 0x7fbdbc152920 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.244+0000 7fbdaaffd640 1 -- 172.21.15.26:0/107856690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb000c8e0 con 0x7fbdbc152920 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.244+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc14eb90 con 0x7fbdbc152920 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.244+0000 7fbdaaffd640 1 -- 172.21.15.26:0/107856690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000ccb0 con 0x7fbdbc152920 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.244+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc1eb6f0 con 0x7fbdbc152920 2024-03-06T07:22:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.245+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdbc04f330 con 0x7fbdbc152920 2024-03-06T07:22:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.245+0000 7fbdaaffd640 1 -- 172.21.15.26:0/107856690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdb001a070 con 0x7fbdbc152920 2024-03-06T07:22:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.245+0000 7fbdaaffd640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.246+0000 7fbdaaffd640 1 -- 172.21.15.26:0/107856690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7101+0+0 (secure 0 0 0) 0x7fbdb008a5c0 con 0x7fbdbc152920 2024-03-06T07:22:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.246+0000 7fbdba59c640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.246+0000 7fbdba59c640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009560 tx=0x7fbdac009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.250+0000 7fbdaaffd640 1 -- 172.21.15.26:0/107856690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbdb00587e0 con 0x7fbdbc152920 2024-03-06T07:22:05.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.470+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdbc06c6e0 con 0x7fbdbc152920 2024-03-06T07:22:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.470+0000 7fbdaaffd640 1 -- 172.21.15.26:0/107856690 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5462 (secure 0 0 0) 0x7fbdb005c880 con 0x7fbdbc152920 2024-03-06T07:22:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:05.472 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:22:04.626219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5234":{"gid":5234,"name":"a","rank":0,"incarnation":36,"state":"up:replay","state_seq":6,"addr":"172.21.15.26:6834/1383022235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1383022235},{"type":"v1","addr":"172.21.15.26:6834","nonce":1383022235}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:22:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 36 2024-03-06T07:22:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd9c06b4d0 msgr2=0x7fbd9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009560 tx=0x7fbdac009190 comp rx=0 tx=0).stop 2024-03-06T07:22:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc152920 msgr2=0x7fbdbc156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc152920 0x7fbdbc156360 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000e8e0 tx=0x7fbdb000eda0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 shutdown_connections 2024-03-06T07:22:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdbc1568a0 0x7fbdbc0bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdbc152920 0x7fbdbc156360 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 --2- 172.21.15.26:0/107856690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdbc0c8a00 0x7fbdbc0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 >> 172.21.15.26:0/107856690 conn(0x7fbdbc0ac560 msgr2=0x7fbdbc14ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 shutdown_connections 2024-03-06T07:22:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:05.472+0000 7fbdc0d48640 1 -- 172.21.15.26:0/107856690 wait complete. 2024-03-06T07:22:05.485 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-03-06T07:22:07.683 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:22:10.485 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:10.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.598+0000 7f61f4bd0640 1 Processor -- start 2024-03-06T07:22:10.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.598+0000 7f61f4bd0640 1 -- start start 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61f4bd0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61f0072720 0x7f61f01564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61f4bd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f0156a00 0x7f61f0150290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61f4bd0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f0156f90 con 0x7f61f01507d0 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61f4bd0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f00c38d0 con 0x7f61f0072720 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61f4bd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0157270 con 0x7f61f0156a00 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61ee59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61f0072720 0x7f61f01564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61eed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61f01507d0 0x7f61f00c3d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47634/0 (socket says 172.21.15.26:47634) 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61eed9d640 1 -- 172.21.15.26:0/1828546951 learned_addr learned my addr 172.21.15.26:0/1828546951 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61edd9b640 1 --2- 172.21.15.26:0/1828546951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f0156a00 0x7f61f0150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61ee59c640 1 -- 172.21.15.26:0/1828546951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f0156a00 msgr2=0x7f61f0150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61ee59c640 1 --2- 172.21.15.26:0/1828546951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f0156a00 0x7f61f0150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61ee59c640 1 -- 172.21.15.26:0/1828546951 >> v1:172.21.15.26:6789/0 conn(0x7f61f01507d0 legacy=0x7f61f00c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61ee59c640 1 -- 172.21.15.26:0/1828546951 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00c3280 con 0x7f61f0072720 2024-03-06T07:22:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.599+0000 7f61ed59a640 1 -- 172.21.15.26:0/1828546951 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909617875 0 0) 0x7f61f0156f90 con 0x7f61f01507d0 2024-03-06T07:22:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.600+0000 7f61ee59c640 1 --2- 172.21.15.26:0/1828546951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01564c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f61e00097d0 tx=0x7f61e002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed9e57d4b6ce642 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.600+0000 7f61ed59a640 1 -- 172.21.15.26:0/1828546951 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0039070 con 0x7f61f0072720 2024-03-06T07:22:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.600+0000 7f61ed59a640 1 -- 172.21.15.26:0/1828546951 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e00316f0 con 0x7f61f0072720 2024-03-06T07:22:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.600+0000 7f61ed59a640 1 -- 172.21.15.26:0/1828546951 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0031ac0 con 0x7f61f0072720 2024-03-06T07:22:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1828546951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 msgr2=0x7f61f01564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1828546951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01564c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f61e00097d0 tx=0x7f61e002b790 comp rx=0 tx=0).stop 2024-03-06T07:22:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1828546951 shutdown_connections 2024-03-06T07:22:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1828546951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f0156a00 0x7f61f0150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1828546951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01564c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1828546951 >> 172.21.15.26:0/1828546951 conn(0x7f61f00aa630 msgr2=0x7f61f00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1828546951 shutdown_connections 2024-03-06T07:22:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.601+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1828546951 wait complete. 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.602+0000 7f61f4bd0640 1 Processor -- start 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.602+0000 7f61f4bd0640 1 -- start start 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61f4bd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61f4bd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f01507d0 0x7f61f0152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61f4bd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0156a00 0x7f61f01ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61f4bd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0072b00 con 0x7f61f0072720 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61f4bd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f006f8f0 con 0x7f61f0156a00 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61f4bd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0072c80 con 0x7f61f01507d0 2024-03-06T07:22:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50722/0 (socket says 172.21.15.26:50722) 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 -- 172.21.15.26:0/1701418380 learned_addr learned my addr 172.21.15.26:0/1701418380 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61edd9b640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f01507d0 0x7f61f0152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61eed9d640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0156a00 0x7f61f01ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 -- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f01507d0 msgr2=0x7f61f0152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f01507d0 0x7f61f0152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 -- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0156a00 msgr2=0x7f61f01ec7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0156a00 0x7f61f01ec7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 -- 172.21.15.26:0/1701418380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00adec0 con 0x7f61f0072720 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.603+0000 7f61ee59c640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01e8b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f61e0000c00 tx=0x7f61e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.604+0000 7f61deffd640 1 -- 172.21.15.26:0/1701418380 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0043070 con 0x7f61f0072720 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.604+0000 7f61deffd640 1 -- 172.21.15.26:0/1701418380 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e0034480 con 0x7f61f0072720 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.604+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f00ae1a0 con 0x7f61f0072720 2024-03-06T07:22:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.604+0000 7f61deffd640 1 -- 172.21.15.26:0/1701418380 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0039040 con 0x7f61f0072720 2024-03-06T07:22:10.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.604+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f00ae760 con 0x7f61f0072720 2024-03-06T07:22:10.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.605+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f004f330 con 0x7f61f0072720 2024-03-06T07:22:10.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.605+0000 7f61deffd640 1 -- 172.21.15.26:0/1701418380 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61e0046070 con 0x7f61f0072720 2024-03-06T07:22:10.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.606+0000 7f61deffd640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61d006b4d0 0x7f61d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:10.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.606+0000 7f61deffd640 1 -- 172.21.15.26:0/1701418380 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f61e00ac210 con 0x7f61f0072720 2024-03-06T07:22:10.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.606+0000 7f61edd9b640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61d006b4d0 0x7f61d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:10.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.606+0000 7f61edd9b640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61d006b4d0 0x7f61d006d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f61f0156830 tx=0x7f61e4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:10.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.610+0000 7f61deffd640 1 -- 172.21.15.26:0/1701418380 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f61e00316c0 con 0x7f61f0072720 2024-03-06T07:22:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.830+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61f014a450 con 0x7f61f0072720 2024-03-06T07:22:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.831+0000 7f61deffd640 1 -- 172.21.15.26:0/1701418380 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6260 (secure 0 0 0) 0x7f61e007a460 con 0x7f61f0072720 2024-03-06T07:22:10.833 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:10.833 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:22:10.655453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5414":{"gid":5414,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/2696722008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2696722008},{"type":"v1","addr":"172.21.15.26:6835","nonce":2696722008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5414,"qdb_cluster":[5414]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:22:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 43 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61d006b4d0 msgr2=0x7f61d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61d006b4d0 0x7f61d006d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f61f0156830 tx=0x7f61e4009190 comp rx=0 tx=0).stop 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 msgr2=0x7f61f01e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01e8b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f61e0000c00 tx=0x7f61e00046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 shutdown_connections 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61d006b4d0 0x7f61d006d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0156a00 0x7f61f01ec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61f01507d0 0x7f61f0152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 --2- 172.21.15.26:0/1701418380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f0072720 0x7f61f01e8b00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 >> 172.21.15.26:0/1701418380 conn(0x7f61f00aa630 msgr2=0x7f61f014bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 shutdown_connections 2024-03-06T07:22:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:10.833+0000 7f61f4bd0640 1 -- 172.21.15.26:0/1701418380 wait complete. 2024-03-06T07:22:10.845 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:22:10.846 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:22:10.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:10.886 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:10.926 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:22:10.927 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:22:10.927 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:10.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:22:10.942 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:22:10.943 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:10.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:22:10.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:11.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.081+0000 7f37dd15e640 1 Processor -- start 2024-03-06T07:22:11.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.081+0000 7f37dd15e640 1 -- start start 2024-03-06T07:22:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37dd15e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f37d8159270 0x7f37d8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37dd15e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d80c73d0 con 0x7f37d814f360 2024-03-06T07:22:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37dd15e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d80be2e0 con 0x7f37d80c88c0 2024-03-06T07:22:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37dd15e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d8159700 con 0x7f37d8159270 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37d6d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f37d814f360 0x7f37d80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47646/0 (socket says 172.21.15.26:47646) 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37d6d9d640 1 -- 172.21.15.26:0/2655512395 learned_addr learned my addr 172.21.15.26:0/2655512395 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37d759e640 1 --2- 172.21.15.26:0/2655512395 >> v2:172.21.15.117:3300/0 conn(0x7f37d8159270 0x7f37d8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37d5d9b640 1 -- 172.21.15.26:0/2655512395 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958398609 0 0) 0x7f37d80c73d0 con 0x7f37d814f360 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.082+0000 7f37d5d9b640 1 -- 172.21.15.26:0/2655512395 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c4003520 con 0x7f37d814f360 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.083+0000 7f37d759e640 1 -- 172.21.15.26:0/2655512395 >> v1:172.21.15.117:6789/0 conn(0x7f37d80c88c0 legacy=0x7f37d8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.083+0000 7f37d5d9b640 1 -- 172.21.15.26:0/2655512395 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509046180 0 0) 0x7f37d80be2e0 con 0x7f37d80c88c0 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.083+0000 7f37d759e640 1 -- 172.21.15.26:0/2655512395 >> v1:172.21.15.26:6789/0 conn(0x7f37d814f360 legacy=0x7f37d80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.083+0000 7f37d759e640 1 -- 172.21.15.26:0/2655512395 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80c7670 con 0x7f37d8159270 2024-03-06T07:22:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.083+0000 7f37d759e640 1 --2- 172.21.15.26:0/2655512395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 0x7f37d8155c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009d90 tx=0x7f37d002bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819cd59938a559b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37d5d9b640 1 -- 172.21.15.26:0/2655512395 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0039070 con 0x7f37d8159270 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37d5d9b640 1 -- 172.21.15.26:0/2655512395 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d002c7d0 con 0x7f37d8159270 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37d5d9b640 1 -- 172.21.15.26:0/2655512395 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8001670 con 0x7f37d8159270 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37dd15e640 1 -- 172.21.15.26:0/2655512395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 msgr2=0x7f37d8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37dd15e640 1 --2- 172.21.15.26:0/2655512395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 0x7f37d8155c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009d90 tx=0x7f37d002bca0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37dd15e640 1 -- 172.21.15.26:0/2655512395 shutdown_connections 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37dd15e640 1 --2- 172.21.15.26:0/2655512395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 0x7f37d8155c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37dd15e640 1 -- 172.21.15.26:0/2655512395 >> 172.21.15.26:0/2655512395 conn(0x7f37d80ac570 msgr2=0x7f37d80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37dd15e640 1 -- 172.21.15.26:0/2655512395 shutdown_connections 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.084+0000 7f37dd15e640 1 -- 172.21.15.26:0/2655512395 wait complete. 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 Processor -- start 2024-03-06T07:22:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 -- start start 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37d80c88c0 0x7f37d80afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37d814f360 0x7f37d814a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 0x7f37d81481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80c7c30 con 0x7f37d80c88c0 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80c5a50 con 0x7f37d8159270 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37dd15e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80c5bd0 con 0x7f37d814f360 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37d6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37d80c88c0 0x7f37d80afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37d6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37d80c88c0 0x7f37d80afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50726/0 (socket says 172.21.15.26:50726) 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.086+0000 7f37d6d9d640 1 -- 172.21.15.26:0/3232437065 learned_addr learned my addr 172.21.15.26:0/3232437065 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d759e640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 0x7f37d81481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d659c640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37d814f360 0x7f37d814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d6d9d640 1 -- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37d814f360 msgr2=0x7f37d814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d6d9d640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37d814f360 0x7f37d814a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d6d9d640 1 -- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 msgr2=0x7f37d81481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d6d9d640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 0x7f37d81481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d6d9d640 1 -- 172.21.15.26:0/3232437065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80bf660 con 0x7f37d80c88c0 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37d6d9d640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37d80c88c0 0x7f37d80afd50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c7d0 tx=0x7f37c800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37cf7fe640 1 -- 172.21.15.26:0/3232437065 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8018070 con 0x7f37d80c88c0 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37cf7fe640 1 -- 172.21.15.26:0/3232437065 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c8007500 con 0x7f37d80c88c0 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d80bf0a0 con 0x7f37d80c88c0 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37cf7fe640 1 -- 172.21.15.26:0/3232437065 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8004a20 con 0x7f37d80c88c0 2024-03-06T07:22:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.087+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d814a8a0 con 0x7f37d80c88c0 2024-03-06T07:22:11.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.088+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d804f330 con 0x7f37d80c88c0 2024-03-06T07:22:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.089+0000 7f37cf7fe640 1 -- 172.21.15.26:0/3232437065 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37c801b070 con 0x7f37d80c88c0 2024-03-06T07:22:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.089+0000 7f37cf7fe640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37b806b4d0 0x7f37b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.089+0000 7f37cf7fe640 1 -- 172.21.15.26:0/3232437065 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f37c8089d10 con 0x7f37d80c88c0 2024-03-06T07:22:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.089+0000 7f37d659c640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37b806b4d0 0x7f37b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.090+0000 7f37d659c640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37b806b4d0 0x7f37b806d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f37d80c16a0 tx=0x7f37c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.094+0000 7f37cf7fe640 1 -- 172.21.15.26:0/3232437065 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f37c8058000 con 0x7f37d80c88c0 2024-03-06T07:22:11.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.388+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37d80c9510 con 0x7f37d80c88c0 2024-03-06T07:22:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.389+0000 7f37cf7fe640 1 -- 172.21.15.26:0/3232437065 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6260 (secure 0 0 0) 0x7f37c805c0a0 con 0x7f37d80c88c0 2024-03-06T07:22:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:11.391 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:22:10.655453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5414":{"gid":5414,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/2696722008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2696722008},{"type":"v1","addr":"172.21.15.26:6835","nonce":2696722008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5414,"qdb_cluster":[5414]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:22:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 43 2024-03-06T07:22:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37b806b4d0 msgr2=0x7f37b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37b806b4d0 0x7f37b806d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f37d80c16a0 tx=0x7f37c0009040 comp rx=0 tx=0).stop 2024-03-06T07:22:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37d80c88c0 msgr2=0x7f37d80afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37d80c88c0 0x7f37d80afd50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c7d0 tx=0x7f37c800cc90 comp rx=0 tx=0).stop 2024-03-06T07:22:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 shutdown_connections 2024-03-06T07:22:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37b806b4d0 0x7f37b806d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8159270 0x7f37d81481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37d814f360 0x7f37d814a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 --2- 172.21.15.26:0/3232437065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37d80c88c0 0x7f37d80afd50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 >> 172.21.15.26:0/3232437065 conn(0x7f37d80ac570 msgr2=0x7f37d8158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 shutdown_connections 2024-03-06T07:22:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.391+0000 7f37dd15e640 1 -- 172.21.15.26:0/3232437065 wait complete. 2024-03-06T07:22:11.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:11.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:11.464 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.508+0000 7f8d94ff7640 1 Processor -- start 2024-03-06T07:22:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.508+0000 7f8d94ff7640 1 -- start start 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d94ff7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d900c88c0 0x7f8d90155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d94ff7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d90158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d94ff7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d900c73d0 con 0x7f8d9014f360 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d94ff7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d900c8dd0 con 0x7f8d900c88c0 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d94ff7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900be2e0 con 0x7f8d90158b60 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d900c88c0 0x7f8d90155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d900c88c0 0x7f8d90155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50736/0 (socket says 172.21.15.26:50736) 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8dd9b640 1 -- 172.21.15.26:0/1104080279 learned_addr learned my addr 172.21.15.26:0/1104080279 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8ed9d640 1 --2- 172.21.15.26:0/1104080279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d90158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8dd9b640 1 -- 172.21.15.26:0/1104080279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 msgr2=0x7f8d90158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8dd9b640 1 --2- 172.21.15.26:0/1104080279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d90158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8dd9b640 1 -- 172.21.15.26:0/1104080279 >> v1:172.21.15.117:6789/0 conn(0x7f8d9014f360 legacy=0x7f8d900c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.509+0000 7f8d8dd9b640 1 -- 172.21.15.26:0/1104080279 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900c7670 con 0x7f8d900c88c0 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.510+0000 7f8d8dd9b640 1 --2- 172.21.15.26:0/1104080279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90155c10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009870 tx=0x7f8d8002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cae32d9ee61a6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.510+0000 7f8d8d59a640 1 -- 172.21.15.26:0/1104080279 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80035020 con 0x7f8d900c88c0 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.510+0000 7f8d8d59a640 1 -- 172.21.15.26:0/1104080279 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80031640 con 0x7f8d900c88c0 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.510+0000 7f8d8d59a640 1 -- 172.21.15.26:0/1104080279 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8002d040 con 0x7f8d900c88c0 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 -- 172.21.15.26:0/1104080279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 msgr2=0x7f8d90155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/1104080279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90155c10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009870 tx=0x7f8d8002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 -- 172.21.15.26:0/1104080279 shutdown_connections 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/1104080279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d90158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/1104080279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90155c10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 -- 172.21.15.26:0/1104080279 >> 172.21.15.26:0/1104080279 conn(0x7f8d900ac570 msgr2=0x7f8d900ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 -- 172.21.15.26:0/1104080279 shutdown_connections 2024-03-06T07:22:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.511+0000 7f8d94ff7640 1 -- 172.21.15.26:0/1104080279 wait complete. 2024-03-06T07:22:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.512+0000 7f8d94ff7640 1 Processor -- start 2024-03-06T07:22:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.512+0000 7f8d94ff7640 1 -- start start 2024-03-06T07:22:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d94ff7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d94ff7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9014f360 0x7f8d9014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d94ff7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d901ead10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d94ff7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900c5a50 con 0x7f8d900c88c0 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d94ff7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900c7c30 con 0x7f8d9014f360 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d94ff7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900c5bd0 con 0x7f8d90158b60 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50752/0 (socket says 172.21.15.26:50752) 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 -- 172.21.15.26:0/2405413592 learned_addr learned my addr 172.21.15.26:0/2405413592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8ed9d640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d901ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8dd9b640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9014f360 0x7f8d9014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 -- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 msgr2=0x7f8d901ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d901ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 -- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9014f360 msgr2=0x7f8d9014de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9014f360 0x7f8d9014de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 -- 172.21.15.26:0/2405413592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90152590 con 0x7f8d900c88c0 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d8e59c640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90158420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c3f0 tx=0x7f8d7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.513+0000 7f8d86ffd640 1 -- 172.21.15.26:0/2405413592 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780052d0 con 0x7f8d900c88c0 2024-03-06T07:22:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.514+0000 7f8d86ffd640 1 -- 172.21.15.26:0/2405413592 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d78005490 con 0x7f8d900c88c0 2024-03-06T07:22:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.514+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d900c1b30 con 0x7f8d900c88c0 2024-03-06T07:22:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.514+0000 7f8d86ffd640 1 -- 172.21.15.26:0/2405413592 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78007260 con 0x7f8d900c88c0 2024-03-06T07:22:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.514+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9014e380 con 0x7f8d900c88c0 2024-03-06T07:22:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.515+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9004f330 con 0x7f8d900c88c0 2024-03-06T07:22:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.515+0000 7f8d86ffd640 1 -- 172.21.15.26:0/2405413592 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d78007400 con 0x7f8d900c88c0 2024-03-06T07:22:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.515+0000 7f8d86ffd640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8d7006b4d0 0x7f8d7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.516+0000 7f8d86ffd640 1 -- 172.21.15.26:0/2405413592 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f8d78089b30 con 0x7f8d900c88c0 2024-03-06T07:22:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.516+0000 7f8d8dd9b640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8d7006b4d0 0x7f8d7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.516+0000 7f8d8dd9b640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8d7006b4d0 0x7f8d7006d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8d80036000 tx=0x7f8d8003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.520+0000 7f8d86ffd640 1 -- 172.21.15.26:0/2405413592 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8d78057e20 con 0x7f8d900c88c0 2024-03-06T07:22:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.702+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d90155ed0 con 0x7f8d900c88c0 2024-03-06T07:22:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.703+0000 7f8d86ffd640 1 -- 172.21.15.26:0/2405413592 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+20975 (secure 0 0 0) 0x7f8d7805bec0 con 0x7f8d900c88c0 2024-03-06T07:22:11.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:11.705 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:07.641811+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6832/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6835/440323369":"2024-03-07T07:22:04.621666+0000","172.21.15.26:6833/440323369":"2024-03-07T07:22:04.621666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8d7006b4d0 msgr2=0x7f8d7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8d7006b4d0 0x7f8d7006d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8d80036000 tx=0x7f8d8003c040 comp rx=0 tx=0).stop 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 msgr2=0x7f8d90158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90158420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c3f0 tx=0x7f8d7800c8b0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 shutdown_connections 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8d7006b4d0 0x7f8d7006d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d90158b60 0x7f8d901ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9014f360 0x7f8d9014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 --2- 172.21.15.26:0/2405413592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d900c88c0 0x7f8d90158420 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 >> 172.21.15.26:0/2405413592 conn(0x7f8d900ac570 msgr2=0x7f8d900c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 shutdown_connections 2024-03-06T07:22:11.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.705+0000 7f8d94ff7640 1 -- 172.21.15.26:0/2405413592 wait complete. 2024-03-06T07:22:11.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:21:40.938361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:21:41.634747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:21:52.622857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:21:53.707058+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:11.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.820+0000 7f2ecb3d7640 1 Processor -- start 2024-03-06T07:22:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.821+0000 7f2ecb3d7640 1 -- start start 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ecb3d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ec40a8290 0x7f2ec41539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ecb3d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40c69a0 0x7f2ec4072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ecb3d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec40c62a0 con 0x7f2ec40c4740 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ecb3d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec4153f00 con 0x7f2ec40a8290 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ecb3d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40c4c00 con 0x7f2ec40c69a0 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec9173640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ec40a8290 0x7f2ec41539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec8972640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2ec40c4740 0x7f2ec40701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47658/0 (socket says 172.21.15.26:47658) 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec9173640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ec40a8290 0x7f2ec41539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50754/0 (socket says 172.21.15.26:50754) 2024-03-06T07:22:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec8972640 1 -- 172.21.15.26:0/2443758368 learned_addr learned my addr 172.21.15.26:0/2443758368 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec9974640 1 --2- 172.21.15.26:0/2443758368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40c69a0 0x7f2ec4072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec9173640 1 -- 172.21.15.26:0/2443758368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40c69a0 msgr2=0x7f2ec4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec9173640 1 --2- 172.21.15.26:0/2443758368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40c69a0 0x7f2ec4072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec9173640 1 -- 172.21.15.26:0/2443758368 >> v1:172.21.15.26:6789/0 conn(0x7f2ec40c4740 legacy=0x7f2ec40701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ebbfff640 1 -- 172.21.15.26:0/2443758368 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909341336 0 0) 0x7f2ec40c62a0 con 0x7f2ec40c4740 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.822+0000 7f2ec9173640 1 -- 172.21.15.26:0/2443758368 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40c4ee0 con 0x7f2ec40a8290 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.823+0000 7f2ec9173640 1 --2- 172.21.15.26:0/2443758368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40a8290 0x7f2ec41539c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00097a0 tx=0x7f2ec002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fed49830758bcc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.823+0000 7f2ebbfff640 1 -- 172.21.15.26:0/2443758368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0038070 con 0x7f2ec40a8290 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.823+0000 7f2ebbfff640 1 -- 172.21.15.26:0/2443758368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec0031700 con 0x7f2ec40a8290 2024-03-06T07:22:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.823+0000 7f2ebbfff640 1 -- 172.21.15.26:0/2443758368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0031ad0 con 0x7f2ec40a8290 2024-03-06T07:22:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/2443758368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40a8290 msgr2=0x7f2ec41539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/2443758368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40a8290 0x7f2ec41539c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00097a0 tx=0x7f2ec002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/2443758368 shutdown_connections 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/2443758368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40c69a0 0x7f2ec4072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/2443758368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40a8290 0x7f2ec41539c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/2443758368 >> 172.21.15.26:0/2443758368 conn(0x7f2ec40c5560 msgr2=0x7f2ec40c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/2443758368 shutdown_connections 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.824+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/2443758368 wait complete. 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.825+0000 7f2ecb3d7640 1 Processor -- start 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.825+0000 7f2ecb3d7640 1 -- start start 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ecb3d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40a8290 0x7f2ec41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ecb3d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40c4740 0x7f2ec41ee130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ecb3d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec40c69a0 0x7f2ec41ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ecb3d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40c6e30 con 0x7f2ec40c4740 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ecb3d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec406fc80 con 0x7f2ec40c69a0 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ecb3d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4072bc0 con 0x7f2ec40a8290 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40c4740 0x7f2ec41ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40c4740 0x7f2ec41ee130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50762/0 (socket says 172.21.15.26:50762) 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 -- 172.21.15.26:0/946984607 learned_addr learned my addr 172.21.15.26:0/946984607 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec9173640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40a8290 0x7f2ec41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec9974640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec40c69a0 0x7f2ec41ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 -- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40a8290 msgr2=0x7f2ec41566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40a8290 0x7f2ec41566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 -- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec40c69a0 msgr2=0x7f2ec41ee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec40c69a0 0x7f2ec41ee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 -- 172.21.15.26:0/946984607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec406da40 con 0x7f2ec40c4740 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.826+0000 7f2ec8972640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40c4740 0x7f2ec41ee130 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c4d0 tx=0x7f2eb400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.827+0000 7f2eb9ffb640 1 -- 172.21.15.26:0/946984607 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4018070 con 0x7f2ec40c4740 2024-03-06T07:22:11.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.827+0000 7f2eb9ffb640 1 -- 172.21.15.26:0/946984607 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4007500 con 0x7f2ec40c4740 2024-03-06T07:22:11.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.827+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec406dd20 con 0x7f2ec40c4740 2024-03-06T07:22:11.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.827+0000 7f2eb9ffb640 1 -- 172.21.15.26:0/946984607 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4004a20 con 0x7f2ec40c4740 2024-03-06T07:22:11.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.827+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec40ae520 con 0x7f2ec40c4740 2024-03-06T07:22:11.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.828+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec404f330 con 0x7f2ec40c4740 2024-03-06T07:22:11.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.828+0000 7f2eb9ffb640 1 -- 172.21.15.26:0/946984607 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2eb401b070 con 0x7f2ec40c4740 2024-03-06T07:22:11.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.828+0000 7f2eb9ffb640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.829+0000 7f2eb9ffb640 1 -- 172.21.15.26:0/946984607 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f2eb4089990 con 0x7f2ec40c4740 2024-03-06T07:22:11.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.829+0000 7f2ec9173640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.829+0000 7f2ec9173640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0035000 tx=0x7f2ec0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:11.833+0000 7f2eb9ffb640 1 -- 172.21.15.26:0/946984607 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2eb4057c80 con 0x7f2ec40c4740 2024-03-06T07:22:12.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.051+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ec40c7930 con 0x7f2ec40c4740 2024-03-06T07:22:12.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.052+0000 7f2eb9ffb640 1 -- 172.21.15.26:0/946984607 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2eb405bd20 con 0x7f2ec40c4740 2024-03-06T07:22:12.054 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:22:12.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 msgr2=0x7f2ea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0035000 tx=0x7f2ec0008030 comp rx=0 tx=0).stop 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40c4740 msgr2=0x7f2ec41ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40c4740 0x7f2ec41ee130 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c4d0 tx=0x7f2eb400c990 comp rx=0 tx=0).stop 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 shutdown_connections 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec40c69a0 0x7f2ec41ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec40c4740 0x7f2ec41ee130 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 --2- 172.21.15.26:0/946984607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec40a8290 0x7f2ec41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 >> 172.21.15.26:0/946984607 conn(0x7f2ec40c5560 msgr2=0x7f2ec40c3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 shutdown_connections 2024-03-06T07:22:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.054+0000 7f2ecb3d7640 1 -- 172.21.15.26:0/946984607 wait complete. 2024-03-06T07:22:12.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:22:12.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.186+0000 7f0124aeb640 1 Processor -- start 2024-03-06T07:22:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.186+0000 7f0124aeb640 1 -- start start 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f0124aeb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01200c25a0 0x7f01200732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f0124aeb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0120156060 con 0x7f01200c3a70 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f0124aeb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01200a8430 con 0x7f012006d750 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f0124aeb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0120155b30 con 0x7f01200c25a0 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f011ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01200c25a0 0x7f01200732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f011dd9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01200c3a70 0x7f01200c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47664/0 (socket says 172.21.15.26:47664) 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f011dd9b640 1 -- 172.21.15.26:0/674825667 learned_addr learned my addr 172.21.15.26:0/674825667 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f011ed9d640 1 -- 172.21.15.26:0/674825667 >> v1:172.21.15.117:6789/0 conn(0x7f012006d750 legacy=0x7f0120072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f011d59a640 1 -- 172.21.15.26:0/674825667 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713089621 0 0) 0x7f01200a8430 con 0x7f012006d750 2024-03-06T07:22:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f011ed9d640 1 -- 172.21.15.26:0/674825667 >> v1:172.21.15.26:6789/0 conn(0x7f01200c3a70 legacy=0x7f01200c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.187+0000 7f011ed9d640 1 -- 172.21.15.26:0/674825667 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120155d80 con 0x7f01200c25a0 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.188+0000 7f011ed9d640 1 --2- 172.21.15.26:0/674825667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01200c25a0 0x7f01200732a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f01180097a0 tx=0x7f011802b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7fe0a856ef86e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.188+0000 7f011d59a640 1 -- 172.21.15.26:0/674825667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118037070 con 0x7f01200c25a0 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.188+0000 7f011d59a640 1 -- 172.21.15.26:0/674825667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0118030710 con 0x7f01200c25a0 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.188+0000 7f011d59a640 1 -- 172.21.15.26:0/674825667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118030ae0 con 0x7f01200c25a0 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.189+0000 7f0124aeb640 1 -- 172.21.15.26:0/674825667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01200c25a0 msgr2=0x7f01200732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.189+0000 7f0124aeb640 1 --2- 172.21.15.26:0/674825667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01200c25a0 0x7f01200732a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f01180097a0 tx=0x7f011802b680 comp rx=0 tx=0).stop 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.189+0000 7f0124aeb640 1 -- 172.21.15.26:0/674825667 shutdown_connections 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.189+0000 7f0124aeb640 1 --2- 172.21.15.26:0/674825667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01200c25a0 0x7f01200732a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.189+0000 7f0124aeb640 1 -- 172.21.15.26:0/674825667 >> 172.21.15.26:0/674825667 conn(0x7f01200c5560 msgr2=0x7f01200c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.189+0000 7f0124aeb640 1 -- 172.21.15.26:0/674825667 shutdown_connections 2024-03-06T07:22:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.189+0000 7f0124aeb640 1 -- 172.21.15.26:0/674825667 wait complete. 2024-03-06T07:22:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.191+0000 7f0124aeb640 1 Processor -- start 2024-03-06T07:22:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.191+0000 7f0124aeb640 1 -- start start 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f0124aeb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f012006d750 0x7f012014d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f0124aeb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01200c3a70 0x7f0120157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f0124aeb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120158120 0x7f01200a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f0124aeb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012006dcb0 con 0x7f0120158120 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f0124aeb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012006db30 con 0x7f012006d750 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f0124aeb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012006de30 con 0x7f01200c3a70 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120158120 0x7f01200a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120158120 0x7f01200a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50776/0 (socket says 172.21.15.26:50776) 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f012006d750 0x7f012014d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 -- 172.21.15.26:0/4210224785 learned_addr learned my addr 172.21.15.26:0/4210224785 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011dd9b640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01200c3a70 0x7f0120157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 -- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01200c3a70 msgr2=0x7f0120157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01200c3a70 0x7f0120157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 -- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f012006d750 msgr2=0x7f012014d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f012006d750 0x7f012014d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011ed9d640 1 -- 172.21.15.26:0/4210224785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012014dee0 con 0x7f0120158120 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.192+0000 7f011e59c640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f012006d750 0x7f012014d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.193+0000 7f011ed9d640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120158120 0x7f01200a8430 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f01180098c0 tx=0x7f01180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.193+0000 7f010effd640 1 -- 172.21.15.26:0/4210224785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118042070 con 0x7f0120158120 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.193+0000 7f010effd640 1 -- 172.21.15.26:0/4210224785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180329c0 con 0x7f0120158120 2024-03-06T07:22:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.193+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01200a9710 con 0x7f0120158120 2024-03-06T07:22:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.193+0000 7f010effd640 1 -- 172.21.15.26:0/4210224785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118037040 con 0x7f0120158120 2024-03-06T07:22:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.193+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01200a99d0 con 0x7f0120158120 2024-03-06T07:22:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.194+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012004f330 con 0x7f0120158120 2024-03-06T07:22:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.195+0000 7f010effd640 1 -- 172.21.15.26:0/4210224785 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0118045070 con 0x7f0120158120 2024-03-06T07:22:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.195+0000 7f010effd640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b500 0x7f010006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.195+0000 7f010effd640 1 -- 172.21.15.26:0/4210224785 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f01180ac9b0 con 0x7f0120158120 2024-03-06T07:22:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.195+0000 7f011e59c640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b500 0x7f010006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.196+0000 7f011e59c640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b500 0x7f010006d9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f01100012e0 tx=0x7f0110009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.199+0000 7f010effd640 1 -- 172.21.15.26:0/4210224785 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f012014dee0 con 0x7f0120158120 2024-03-06T07:22:12.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.378+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01200c1fc0 con 0x7f0120158120 2024-03-06T07:22:12.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.379+0000 7f010effd640 1 -- 172.21.15.26:0/4210224785 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f011807ad30 con 0x7f0120158120 2024-03-06T07:22:12.381 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b500 msgr2=0x7f010006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b500 0x7f010006d9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f01100012e0 tx=0x7f0110009040 comp rx=0 tx=0).stop 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120158120 msgr2=0x7f01200a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120158120 0x7f01200a8430 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f01180098c0 tx=0x7f01180046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 shutdown_connections 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f010006b500 0x7f010006d9c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120158120 0x7f01200a8430 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01200c3a70 0x7f0120157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 --2- 172.21.15.26:0/4210224785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f012006d750 0x7f012014d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 >> 172.21.15.26:0/4210224785 conn(0x7f01200c5560 msgr2=0x7f01200c8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 shutdown_connections 2024-03-06T07:22:12.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.381+0000 7f0124aeb640 1 -- 172.21.15.26:0/4210224785 wait complete. 2024-03-06T07:22:12.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:22:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.497+0000 7f4441088640 1 Processor -- start 2024-03-06T07:22:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.497+0000 7f4441088640 1 -- start start 2024-03-06T07:22:12.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f4441088640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f443c06d750 0x7f443c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f4441088640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0c3a70 0x7f443c072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f4441088640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443c156060 con 0x7f443c06db30 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f4441088640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443c155b30 con 0x7f443c06d750 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f4441088640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c0a8430 con 0x7f443c0c3a70 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443b59e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f443c06db30 0x7f443c0c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47666/0 (socket says 172.21.15.26:47666) 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443b59e640 1 -- 172.21.15.26:0/156172704 learned_addr learned my addr 172.21.15.26:0/156172704 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443a59c640 1 --2- 172.21.15.26:0/156172704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0c3a70 0x7f443c072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443ad9d640 1 --2- 172.21.15.26:0/156172704 >> v2:172.21.15.117:3300/0 conn(0x7f443c06d750 0x7f443c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f4439d9b640 1 -- 172.21.15.26:0/156172704 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434404138 0 0) 0x7f443c156060 con 0x7f443c06db30 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f4439d9b640 1 -- 172.21.15.26:0/156172704 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4424003520 con 0x7f443c06db30 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443a59c640 1 -- 172.21.15.26:0/156172704 >> v2:172.21.15.117:3300/0 conn(0x7f443c06d750 msgr2=0x7f443c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443a59c640 1 --2- 172.21.15.26:0/156172704 >> v2:172.21.15.117:3300/0 conn(0x7f443c06d750 0x7f443c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443a59c640 1 -- 172.21.15.26:0/156172704 >> v1:172.21.15.26:6789/0 conn(0x7f443c06db30 legacy=0x7f443c0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443a59c640 1 -- 172.21.15.26:0/156172704 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c155d80 con 0x7f443c0c3a70 2024-03-06T07:22:12.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.498+0000 7f443ad9d640 1 --2- 172.21.15.26:0/156172704 >> v2:172.21.15.117:3300/0 conn(0x7f443c06d750 0x7f443c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.499+0000 7f443a59c640 1 --2- 172.21.15.26:0/156172704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0c3a70 0x7f443c072680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44300097a0 tx=0x7f443002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e3a00a1cb52a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.499+0000 7f4439d9b640 1 -- 172.21.15.26:0/156172704 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430038070 con 0x7f443c0c3a70 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.499+0000 7f4439d9b640 1 -- 172.21.15.26:0/156172704 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4430031700 con 0x7f443c0c3a70 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.499+0000 7f4439d9b640 1 -- 172.21.15.26:0/156172704 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430031ad0 con 0x7f443c0c3a70 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 -- 172.21.15.26:0/156172704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0c3a70 msgr2=0x7f443c072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 --2- 172.21.15.26:0/156172704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0c3a70 0x7f443c072680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44300097a0 tx=0x7f443002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 -- 172.21.15.26:0/156172704 shutdown_connections 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 --2- 172.21.15.26:0/156172704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0c3a70 0x7f443c072680 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 --2- 172.21.15.26:0/156172704 >> v2:172.21.15.117:3300/0 conn(0x7f443c06d750 0x7f443c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 -- 172.21.15.26:0/156172704 >> 172.21.15.26:0/156172704 conn(0x7f443c0c5560 msgr2=0x7f443c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 -- 172.21.15.26:0/156172704 shutdown_connections 2024-03-06T07:22:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.500+0000 7f4441088640 1 -- 172.21.15.26:0/156172704 wait complete. 2024-03-06T07:22:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.501+0000 7f4441088640 1 Processor -- start 2024-03-06T07:22:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.501+0000 7f4441088640 1 -- start start 2024-03-06T07:22:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f4441088640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c06d750 0x7f443c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f4441088640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c06db30 0x7f443c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f4441088640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0c3a70 0x7f443c0ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f4441088640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c072bc0 con 0x7f443c0c3a70 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f4441088640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c0c27f0 con 0x7f443c06d750 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f4441088640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c072d40 con 0x7f443c06db30 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0c3a70 0x7f443c0ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c06d750 0x7f443c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0c3a70 0x7f443c0ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50784/0 (socket says 172.21.15.26:50784) 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c06db30 0x7f443c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 -- 172.21.15.26:0/3223631448 learned_addr learned my addr 172.21.15.26:0/3223631448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 -- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c06db30 msgr2=0x7f443c14e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c06db30 0x7f443c14e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 -- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c06d750 msgr2=0x7f443c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c06d750 0x7f443c152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.502+0000 7f443b59e640 1 -- 172.21.15.26:0/3223631448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c1ea400 con 0x7f443c0c3a70 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.503+0000 7f443b59e640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0c3a70 0x7f443c0ac5a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f443400cad0 tx=0x7f443400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.503+0000 7f44237fe640 1 -- 172.21.15.26:0/3223631448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434017070 con 0x7f443c0c3a70 2024-03-06T07:22:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.503+0000 7f44237fe640 1 -- 172.21.15.26:0/3223631448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443400ac30 con 0x7f443c0c3a70 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.503+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c1519c0 con 0x7f443c0c3a70 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.503+0000 7f44237fe640 1 -- 172.21.15.26:0/3223631448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434012400 con 0x7f443c0c3a70 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.503+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c1ee700 con 0x7f443c0c3a70 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.504+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c04f330 con 0x7f443c0c3a70 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.504+0000 7f44237fe640 1 -- 172.21.15.26:0/3223631448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f443401a070 con 0x7f443c0c3a70 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.505+0000 7f44237fe640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f442806b4d0 0x7f442806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.505+0000 7f44237fe640 1 -- 172.21.15.26:0/3223631448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f443408a480 con 0x7f443c0c3a70 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.505+0000 7f443ad9d640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f442806b4d0 0x7f442806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.505+0000 7f443ad9d640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f442806b4d0 0x7f442806d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f442c009560 tx=0x7f442c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.509+0000 7f44237fe640 1 -- 172.21.15.26:0/3223631448 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4434058770 con 0x7f443c0c3a70 2024-03-06T07:22:12.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.690+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f443c0c6ac0 con 0x7f443c0c3a70 2024-03-06T07:22:12.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.690+0000 7f44237fe640 1 -- 172.21.15.26:0/3223631448 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f443405c810 con 0x7f443c0c3a70 2024-03-06T07:22:12.692 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:22:12.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f442806b4d0 msgr2=0x7f442806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f442806b4d0 0x7f442806d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f442c009560 tx=0x7f442c009190 comp rx=0 tx=0).stop 2024-03-06T07:22:12.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0c3a70 msgr2=0x7f443c0ac5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0c3a70 0x7f443c0ac5a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f443400cad0 tx=0x7f443400cf90 comp rx=0 tx=0).stop 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 shutdown_connections 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f442806b4d0 0x7f442806d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0c3a70 0x7f443c0ac5a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c06db30 0x7f443c14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 --2- 172.21.15.26:0/3223631448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c06d750 0x7f443c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 >> 172.21.15.26:0/3223631448 conn(0x7f443c0c5560 msgr2=0x7f443c0c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 shutdown_connections 2024-03-06T07:22:12.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.692+0000 7f4441088640 1 -- 172.21.15.26:0/3223631448 wait complete. 2024-03-06T07:22:12.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:22:12.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.808+0000 7fb33e113640 1 Processor -- start 2024-03-06T07:22:12.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.808+0000 7fb33e113640 1 -- start start 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb33e113640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3380c3a70 0x7fb3380732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb33e113640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3380c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb33e113640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb338156060 con 0x7fb33806d750 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb33e113640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb338155b30 con 0x7fb3380c3a70 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb33e113640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380a8430 con 0x7fb33806db30 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb336ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3380c3a70 0x7fb3380732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb3377fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb33806d750 0x7fb338072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47668/0 (socket says 172.21.15.26:47668) 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb3377fe640 1 -- 172.21.15.26:0/3770149827 learned_addr learned my addr 172.21.15.26:0/3770149827 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb337fff640 1 --2- 172.21.15.26:0/3770149827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3380c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb336ffd640 1 -- 172.21.15.26:0/3770149827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 msgr2=0x7fb3380c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb336ffd640 1 --2- 172.21.15.26:0/3770149827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3380c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb336ffd640 1 -- 172.21.15.26:0/3770149827 >> v1:172.21.15.26:6789/0 conn(0x7fb33806d750 legacy=0x7fb338072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb336ffd640 1 -- 172.21.15.26:0/3770149827 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb338155d80 con 0x7fb3380c3a70 2024-03-06T07:22:12.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.809+0000 7fb3367fc640 1 -- 172.21.15.26:0/3770149827 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201985979 0 0) 0x7fb338156060 con 0x7fb33806d750 2024-03-06T07:22:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.810+0000 7fb336ffd640 1 --2- 172.21.15.26:0/3770149827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3380c3a70 0x7fb3380732a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0097a0 tx=0x7fb32c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79603dc5d56ff1fb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.810+0000 7fb3367fc640 1 -- 172.21.15.26:0/3770149827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c038070 con 0x7fb3380c3a70 2024-03-06T07:22:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.810+0000 7fb3367fc640 1 -- 172.21.15.26:0/3770149827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c031700 con 0x7fb3380c3a70 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.810+0000 7fb3367fc640 1 -- 172.21.15.26:0/3770149827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c031ad0 con 0x7fb3380c3a70 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 -- 172.21.15.26:0/3770149827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3380c3a70 msgr2=0x7fb3380732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 --2- 172.21.15.26:0/3770149827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3380c3a70 0x7fb3380732a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0097a0 tx=0x7fb32c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 -- 172.21.15.26:0/3770149827 shutdown_connections 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 --2- 172.21.15.26:0/3770149827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3380c3a70 0x7fb3380732a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 --2- 172.21.15.26:0/3770149827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3380c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 -- 172.21.15.26:0/3770149827 >> 172.21.15.26:0/3770149827 conn(0x7fb3380c5560 msgr2=0x7fb3380c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 -- 172.21.15.26:0/3770149827 shutdown_connections 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.811+0000 7fb33e113640 1 -- 172.21.15.26:0/3770149827 wait complete. 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.812+0000 7fb33e113640 1 Processor -- start 2024-03-06T07:22:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.812+0000 7fb33e113640 1 -- start start 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33806d750 0x7fb33814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3381e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3380c3a70 0x7fb3381e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380a7e50 con 0x7fb33806d750 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380a7cd0 con 0x7fb3380c3a70 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380a7fd0 con 0x7fb33806db30 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33806d750 0x7fb33814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33806d750 0x7fb33814d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50800/0 (socket says 172.21.15.26:50800) 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 -- 172.21.15.26:0/3145304479 learned_addr learned my addr 172.21.15.26:0/3145304479 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb337fff640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3380c3a70 0x7fb3381e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb336ffd640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3381e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 -- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 msgr2=0x7fb3381e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3381e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 -- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3380c3a70 msgr2=0x7fb3381e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3380c3a70 0x7fb3381e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 -- 172.21.15.26:0/3145304479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380c8d30 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb3377fe640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33806d750 0x7fb33814d080 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c430 tx=0x7fb32800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb31ffff640 1 -- 172.21.15.26:0/3145304479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280074e0 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb31ffff640 1 -- 172.21.15.26:0/3145304479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3280047e0 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb31ffff640 1 -- 172.21.15.26:0/3145304479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328004bd0 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3380c8470 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.813+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3380c8a50 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.814+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33804f330 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.815+0000 7fb31ffff640 1 -- 172.21.15.26:0/3145304479 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb328011070 con 0x7fb33806d750 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.815+0000 7fb31ffff640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.815+0000 7fb31ffff640 1 -- 172.21.15.26:0/3145304479 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7fb32808a270 con 0x7fb33806d750 2024-03-06T07:22:12.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.815+0000 7fb336ffd640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.816+0000 7fb336ffd640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb32406b4d0 0x7fb32406d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb32c035000 tx=0x7fb32c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.820+0000 7fb31ffff640 1 -- 172.21.15.26:0/3145304479 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb3280584d0 con 0x7fb33806d750 2024-03-06T07:22:13.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.999+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb338152410 con 0x7fb33806d750 2024-03-06T07:22:13.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:12.999+0000 7fb31ffff640 1 -- 172.21.15.26:0/3145304479 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7fb32805c570 con 0x7fb33806d750 2024-03-06T07:22:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:22:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb32406b4d0 msgr2=0x7fb32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb32406b4d0 0x7fb32406d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb32c035000 tx=0x7fb32c008030 comp rx=0 tx=0).stop 2024-03-06T07:22:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33806d750 msgr2=0x7fb33814d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33806d750 0x7fb33814d080 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c430 tx=0x7fb32800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 shutdown_connections 2024-03-06T07:22:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3380c3a70 0x7fb3381e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb33806db30 0x7fb3381e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 --2- 172.21.15.26:0/3145304479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33806d750 0x7fb33814d080 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 >> 172.21.15.26:0/3145304479 conn(0x7fb3380c5560 msgr2=0x7fb33806fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 shutdown_connections 2024-03-06T07:22:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.001+0000 7fb33e113640 1 -- 172.21.15.26:0/3145304479 wait complete. 2024-03-06T07:22:13.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:22:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.117+0000 7fe682e88640 1 Processor -- start 2024-03-06T07:22:13.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.117+0000 7fe682e88640 1 -- start start 2024-03-06T07:22:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.117+0000 7fe682e88640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe67c155c30 0x7fe67c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.117+0000 7fe682e88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.117+0000 7fe682e88640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0b0450 con 0x7fe67c0b0070 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.117+0000 7fe682e88640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c06cfb0 con 0x7fe67c155c30 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.117+0000 7fe682e88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c06c020 con 0x7fe67c0c5380 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe67c155c30 0x7fe67c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe67c155c30 0x7fe67c06ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50810/0 (socket says 172.21.15.26:50810) 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67bfff640 1 -- 172.21.15.26:0/2355869079 learned_addr learned my addr 172.21.15.26:0/2355869079 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe681425640 1 --2- 172.21.15.26:0/2355869079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67bfff640 1 -- 172.21.15.26:0/2355869079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 msgr2=0x7fe67c0c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67bfff640 1 --2- 172.21.15.26:0/2355869079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67b7fe640 1 -- 172.21.15.26:0/2355869079 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7460936 0 0) 0x7fe674001490 con 0x7fe67c0b0070 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67bfff640 1 -- 172.21.15.26:0/2355869079 >> v1:172.21.15.117:6789/0 conn(0x7fe67c0b0070 legacy=0x7fe67c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.118+0000 7fe67bfff640 1 -- 172.21.15.26:0/2355869079 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c1560e0 con 0x7fe67c155c30 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.119+0000 7fe67bfff640 1 --2- 172.21.15.26:0/2355869079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c155c30 0x7fe67c06ca70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe664009870 tx=0x7fe66402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b7ccd234f2ff68 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.119+0000 7fe67b7fe640 1 -- 172.21.15.26:0/2355869079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664035020 con 0x7fe67c155c30 2024-03-06T07:22:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.119+0000 7fe67b7fe640 1 -- 172.21.15.26:0/2355869079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe664031640 con 0x7fe67c155c30 2024-03-06T07:22:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.119+0000 7fe67b7fe640 1 -- 172.21.15.26:0/2355869079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66402d040 con 0x7fe67c155c30 2024-03-06T07:22:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.119+0000 7fe682e88640 1 -- 172.21.15.26:0/2355869079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c155c30 msgr2=0x7fe67c06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.119+0000 7fe682e88640 1 --2- 172.21.15.26:0/2355869079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c155c30 0x7fe67c06ca70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe664009870 tx=0x7fe66402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.120+0000 7fe682e88640 1 -- 172.21.15.26:0/2355869079 shutdown_connections 2024-03-06T07:22:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.120+0000 7fe682e88640 1 --2- 172.21.15.26:0/2355869079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.120+0000 7fe682e88640 1 --2- 172.21.15.26:0/2355869079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c155c30 0x7fe67c06ca70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.120+0000 7fe682e88640 1 -- 172.21.15.26:0/2355869079 >> 172.21.15.26:0/2355869079 conn(0x7fe67c0ac590 msgr2=0x7fe67c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.120+0000 7fe682e88640 1 -- 172.21.15.26:0/2355869079 shutdown_connections 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.120+0000 7fe682e88640 1 -- 172.21.15.26:0/2355869079 wait complete. 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.121+0000 7fe682e88640 1 Processor -- start 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.121+0000 7fe682e88640 1 -- start start 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.121+0000 7fe682e88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c0b0070 0x7fe67c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe682e88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe682e88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c155c30 0x7fe67c0c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe682e88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c06c270 con 0x7fe67c0b0070 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe682e88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0c7ad0 con 0x7fe67c155c30 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe682e88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c06c550 con 0x7fe67c0c5380 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c0b0070 0x7fe67c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c0b0070 0x7fe67c158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50820/0 (socket says 172.21.15.26:50820) 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 -- 172.21.15.26:0/473080391 learned_addr learned my addr 172.21.15.26:0/473080391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe67bfff640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe681425640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c155c30 0x7fe67c0c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 -- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 msgr2=0x7fe67c0c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 -- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c155c30 msgr2=0x7fe67c0c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c155c30 0x7fe67c0c28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 -- 172.21.15.26:0/473080391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c1570c0 con 0x7fe67c0b0070 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe680c24640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c0b0070 0x7fe67c158ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe67400c820 tx=0x7fe67400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe6797fa640 1 -- 172.21.15.26:0/473080391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674017070 con 0x7fe67c0b0070 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe6797fa640 1 -- 172.21.15.26:0/473080391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe674003da0 con 0x7fe67c0b0070 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c157360 con 0x7fe67c0b0070 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.122+0000 7fe6797fa640 1 -- 172.21.15.26:0/473080391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674005290 con 0x7fe67c0b0070 2024-03-06T07:22:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.123+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c157920 con 0x7fe67c0b0070 2024-03-06T07:22:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.124+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c04f330 con 0x7fe67c0b0070 2024-03-06T07:22:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.124+0000 7fe6797fa640 1 -- 172.21.15.26:0/473080391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe67401a070 con 0x7fe67c0b0070 2024-03-06T07:22:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.124+0000 7fe6797fa640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe65c06b4d0 0x7fe65c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.124+0000 7fe6797fa640 1 -- 172.21.15.26:0/473080391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7fe674089d40 con 0x7fe67c0b0070 2024-03-06T07:22:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.125+0000 7fe67bfff640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe65c06b4d0 0x7fe65c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.125+0000 7fe67bfff640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe65c06b4d0 0x7fe65c06d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe664036000 tx=0x7fe66403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.129+0000 7fe6797fa640 1 -- 172.21.15.26:0/473080391 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe67c0b0450 con 0x7fe67c0b0070 2024-03-06T07:22:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.368+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fe67c06d860 con 0x7fe67c0b0070 2024-03-06T07:22:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.369+0000 7fe6797fa640 1 -- 172.21.15.26:0/473080391 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v58) v1 ==== 101+0+11 (secure 0 0 0) 0x7fe674058030 con 0x7fe67c0b0070 2024-03-06T07:22:13.370 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:22:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe65c06b4d0 msgr2=0x7fe65c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe65c06b4d0 0x7fe65c06d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe664036000 tx=0x7fe66403c040 comp rx=0 tx=0).stop 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c0b0070 msgr2=0x7fe67c158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c0b0070 0x7fe67c158ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe67400c820 tx=0x7fe67400cce0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 shutdown_connections 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe65c06b4d0 0x7fe65c06d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c155c30 0x7fe67c0c28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe67c0c5380 0x7fe67c0c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 --2- 172.21.15.26:0/473080391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe67c0b0070 0x7fe67c158ec0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 >> 172.21.15.26:0/473080391 conn(0x7fe67c0ac590 msgr2=0x7fe67c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 shutdown_connections 2024-03-06T07:22:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.371+0000 7fe682e88640 1 -- 172.21.15.26:0/473080391 wait complete. 2024-03-06T07:22:13.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.483+0000 7f1d2797a640 1 Processor -- start 2024-03-06T07:22:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.483+0000 7f1d2797a640 1 -- start start 2024-03-06T07:22:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d2797a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1d200c5860 0x7f1d201609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d2797a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d201591c0 con 0x7f1d20155ba0 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d2797a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d201594a0 con 0x7f1d200c5480 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d2797a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2006df40 con 0x7f1d200c5860 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d24f15640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1d20155ba0 0x7f1d201586f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47676/0 (socket says 172.21.15.26:47676) 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d24f15640 1 -- 172.21.15.26:0/4202133407 learned_addr learned my addr 172.21.15.26:0/4202133407 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d25f17640 1 --2- 172.21.15.26:0/4202133407 >> v2:172.21.15.117:3300/0 conn(0x7f1d200c5860 0x7f1d201609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d17fff640 1 -- 172.21.15.26:0/4202133407 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773933037 0 0) 0x7f1d201591c0 con 0x7f1d20155ba0 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d17fff640 1 -- 172.21.15.26:0/4202133407 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d0c003520 con 0x7f1d20155ba0 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d17fff640 1 -- 172.21.15.26:0/4202133407 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133701421 0 0) 0x7f1d201594a0 con 0x7f1d200c5480 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d17fff640 1 -- 172.21.15.26:0/4202133407 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d201591c0 con 0x7f1d200c5480 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d25f17640 1 -- 172.21.15.26:0/4202133407 >> v1:172.21.15.117:6789/0 conn(0x7f1d200c5480 legacy=0x7f1d2006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d25f17640 1 -- 172.21.15.26:0/4202133407 >> v1:172.21.15.26:6789/0 conn(0x7f1d20155ba0 legacy=0x7f1d201586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.484+0000 7f1d25f17640 1 -- 172.21.15.26:0/4202133407 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d201553a0 con 0x7f1d200c5860 2024-03-06T07:22:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.485+0000 7f1d25f17640 1 --2- 172.21.15.26:0/4202133407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5860 0x7f1d201609e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009d90 tx=0x7f1d1802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7efce89520218638 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.485+0000 7f1d17fff640 1 -- 172.21.15.26:0/4202133407 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18039070 con 0x7f1d200c5860 2024-03-06T07:22:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.485+0000 7f1d17fff640 1 -- 172.21.15.26:0/4202133407 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1802cc60 con 0x7f1d200c5860 2024-03-06T07:22:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.485+0000 7f1d17fff640 1 -- 172.21.15.26:0/4202133407 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18034640 con 0x7f1d200c5860 2024-03-06T07:22:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.486+0000 7f1d2797a640 1 -- 172.21.15.26:0/4202133407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5860 msgr2=0x7f1d201609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.486+0000 7f1d2797a640 1 --2- 172.21.15.26:0/4202133407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5860 0x7f1d201609e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009d90 tx=0x7f1d1802beb0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.486+0000 7f1d2797a640 1 -- 172.21.15.26:0/4202133407 shutdown_connections 2024-03-06T07:22:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.486+0000 7f1d2797a640 1 --2- 172.21.15.26:0/4202133407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5860 0x7f1d201609e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.486+0000 7f1d2797a640 1 -- 172.21.15.26:0/4202133407 >> 172.21.15.26:0/4202133407 conn(0x7f1d2006b1a0 msgr2=0x7f1d2006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.486+0000 7f1d2797a640 1 -- 172.21.15.26:0/4202133407 shutdown_connections 2024-03-06T07:22:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.486+0000 7f1d2797a640 1 -- 172.21.15.26:0/4202133407 wait complete. 2024-03-06T07:22:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.488+0000 7f1d2797a640 1 Processor -- start 2024-03-06T07:22:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.488+0000 7f1d2797a640 1 -- start start 2024-03-06T07:22:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d2797a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5480 0x7f1d20153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d2797a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d200c5860 0x7f1d20154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d2797a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d20155ba0 0x7f1d2014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d2797a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200c5cf0 con 0x7f1d200c5860 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d2797a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2006fdb0 con 0x7f1d200c5480 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d2797a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20156000 con 0x7f1d20155ba0 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d200c5860 0x7f1d20154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d200c5860 0x7f1d20154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50824/0 (socket says 172.21.15.26:50824) 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 -- 172.21.15.26:0/1269354297 learned_addr learned my addr 172.21.15.26:0/1269354297 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d25f17640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d20155ba0 0x7f1d2014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d25716640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5480 0x7f1d20153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 -- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d20155ba0 msgr2=0x7f1d2014d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d20155ba0 0x7f1d2014d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 -- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5480 msgr2=0x7f1d20153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5480 0x7f1d20153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 -- 172.21.15.26:0/1269354297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d201511a0 con 0x7f1d200c5860 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.489+0000 7f1d24f15640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d200c5860 0x7f1d20154a20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c410 tx=0x7f1d1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.490+0000 7f1d15ffb640 1 -- 172.21.15.26:0/1269354297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10017070 con 0x7f1d200c5860 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.490+0000 7f1d15ffb640 1 -- 172.21.15.26:0/1269354297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d10005ce0 con 0x7f1d200c5860 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.490+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d20154f60 con 0x7f1d200c5860 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.490+0000 7f1d15ffb640 1 -- 172.21.15.26:0/1269354297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d100072f0 con 0x7f1d200c5860 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.490+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d20151720 con 0x7f1d200c5860 2024-03-06T07:22:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.491+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2004f330 con 0x7f1d200c5860 2024-03-06T07:22:13.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.492+0000 7f1d15ffb640 1 -- 172.21.15.26:0/1269354297 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d1001a070 con 0x7f1d200c5860 2024-03-06T07:22:13.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.492+0000 7f1d15ffb640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d0406b4d0 0x7f1d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.492+0000 7f1d15ffb640 1 -- 172.21.15.26:0/1269354297 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f1d10089b30 con 0x7f1d200c5860 2024-03-06T07:22:13.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.492+0000 7f1d25716640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d0406b4d0 0x7f1d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.493+0000 7f1d25716640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d0406b4d0 0x7f1d0406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c007fd0 tx=0x7f1d1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.496+0000 7f1d15ffb640 1 -- 172.21.15.26:0/1269354297 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1d10057ee0 con 0x7f1d200c5860 2024-03-06T07:22:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.767+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d2006e0c0 con 0x7f1d200c5860 2024-03-06T07:22:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.768+0000 7f1d15ffb640 1 -- 172.21.15.26:0/1269354297 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6260 (secure 0 0 0) 0x7f1d1005bf80 con 0x7f1d200c5860 2024-03-06T07:22:13.769 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:13.770 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:22:10.655453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5414":{"gid":5414,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/2696722008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2696722008},{"type":"v1","addr":"172.21.15.26:6835","nonce":2696722008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5414,"qdb_cluster":[5414]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:22:13.770 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 43 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d0406b4d0 msgr2=0x7f1d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d0406b4d0 0x7f1d0406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c007fd0 tx=0x7f1d1c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d200c5860 msgr2=0x7f1d20154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d200c5860 0x7f1d20154a20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c410 tx=0x7f1d1000c8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 shutdown_connections 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d0406b4d0 0x7f1d0406d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d20155ba0 0x7f1d2014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d200c5860 0x7f1d20154a20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 --2- 172.21.15.26:0/1269354297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d200c5480 0x7f1d20153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 >> 172.21.15.26:0/1269354297 conn(0x7f1d2006b1a0 msgr2=0x7f1d2006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 shutdown_connections 2024-03-06T07:22:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.770+0000 7f1d2797a640 1 -- 172.21.15.26:0/1269354297 wait complete. 2024-03-06T07:22:13.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:13.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.883+0000 7f25da9c9640 1 Processor -- start 2024-03-06T07:22:13.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.883+0000 7f25da9c9640 1 -- start start 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25da9c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25d41527b0 0x7f25d4159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25da9c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25d40c7180 0x7f25d40c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25da9c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d40c7c60 con 0x7f25d4155b20 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25da9c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d4159640 con 0x7f25d41527b0 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25da9c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d40c8df0 con 0x7f25d40c7180 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25d41527b0 0x7f25d4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d8f66640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f25d4155b20 0x7f25d40c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47682/0 (socket says 172.21.15.26:47682) 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d8f66640 1 -- 172.21.15.26:0/685327053 learned_addr learned my addr 172.21.15.26:0/685327053 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d37fe640 1 --2- 172.21.15.26:0/685327053 >> v2:172.21.15.117:3300/0 conn(0x7f25d40c7180 0x7f25d40c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d3fff640 1 -- 172.21.15.26:0/685327053 >> v2:172.21.15.117:3300/0 conn(0x7f25d40c7180 msgr2=0x7f25d40c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d3fff640 1 --2- 172.21.15.26:0/685327053 >> v2:172.21.15.117:3300/0 conn(0x7f25d40c7180 0x7f25d40c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d3fff640 1 -- 172.21.15.26:0/685327053 >> v1:172.21.15.26:6789/0 conn(0x7f25d4155b20 legacy=0x7f25d40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.884+0000 7f25d2ffd640 1 -- 172.21.15.26:0/685327053 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293852757 0 0) 0x7f25cc001490 con 0x7f25d4155b20 2024-03-06T07:22:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.885+0000 7f25d3fff640 1 -- 172.21.15.26:0/685327053 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d40c8060 con 0x7f25d41527b0 2024-03-06T07:22:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.885+0000 7f25d3fff640 1 --2- 172.21.15.26:0/685327053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d41527b0 0x7f25d4159100 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f25d408cac0 tx=0x7f25c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fb959bda4b8c69 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.885+0000 7f25d2ffd640 1 -- 172.21.15.26:0/685327053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c80315a0 con 0x7f25d41527b0 2024-03-06T07:22:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.885+0000 7f25d2ffd640 1 -- 172.21.15.26:0/685327053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c8037070 con 0x7f25d41527b0 2024-03-06T07:22:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.885+0000 7f25d2ffd640 1 -- 172.21.15.26:0/685327053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8031960 con 0x7f25d41527b0 2024-03-06T07:22:13.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 -- 172.21.15.26:0/685327053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d41527b0 msgr2=0x7f25d4159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 --2- 172.21.15.26:0/685327053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d41527b0 0x7f25d4159100 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f25d408cac0 tx=0x7f25c802b840 comp rx=0 tx=0).stop 2024-03-06T07:22:13.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 -- 172.21.15.26:0/685327053 shutdown_connections 2024-03-06T07:22:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 --2- 172.21.15.26:0/685327053 >> v2:172.21.15.117:3300/0 conn(0x7f25d40c7180 0x7f25d40c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 --2- 172.21.15.26:0/685327053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d41527b0 0x7f25d4159100 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 -- 172.21.15.26:0/685327053 >> 172.21.15.26:0/685327053 conn(0x7f25d40ac560 msgr2=0x7f25d40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 -- 172.21.15.26:0/685327053 shutdown_connections 2024-03-06T07:22:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.886+0000 7f25da9c9640 1 -- 172.21.15.26:0/685327053 wait complete. 2024-03-06T07:22:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.888+0000 7f25da9c9640 1 Processor -- start 2024-03-06T07:22:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.888+0000 7f25da9c9640 1 -- start start 2024-03-06T07:22:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25da9c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d40c7180 0x7f25d41eec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25da9c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25d41527b0 0x7f25d41eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25da9c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25d4155b20 0x7f25d4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25da9c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4155320 con 0x7f25d40c7180 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25da9c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40c7980 con 0x7f25d4155b20 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25da9c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4155600 con 0x7f25d41527b0 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d40c7180 0x7f25d41eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d40c7180 0x7f25d41eec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50846/0 (socket says 172.21.15.26:50846) 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25d3fff640 1 -- 172.21.15.26:0/289876777 learned_addr learned my addr 172.21.15.26:0/289876777 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25d37fe640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25d41527b0 0x7f25d41eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.889+0000 7f25d8f66640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25d4155b20 0x7f25d4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d3fff640 1 -- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25d41527b0 msgr2=0x7f25d41eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d3fff640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25d41527b0 0x7f25d41eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d3fff640 1 -- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25d4155b20 msgr2=0x7f25d4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d3fff640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25d4155b20 0x7f25d4154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d3fff640 1 -- 172.21.15.26:0/289876777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d41574b0 con 0x7f25d40c7180 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d3fff640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d40c7180 0x7f25d41eec80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f25c8034040 tx=0x7f25c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d0ff9640 1 -- 172.21.15.26:0/289876777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8004230 con 0x7f25d40c7180 2024-03-06T07:22:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d0ff9640 1 -- 172.21.15.26:0/289876777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c80043d0 con 0x7f25d40c7180 2024-03-06T07:22:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d41ebc30 con 0x7f25d40c7180 2024-03-06T07:22:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25d0ff9640 1 -- 172.21.15.26:0/289876777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8003850 con 0x7f25d40c7180 2024-03-06T07:22:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.890+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d41587e0 con 0x7f25d40c7180 2024-03-06T07:22:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.891+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d404f330 con 0x7f25d40c7180 2024-03-06T07:22:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.892+0000 7f25d0ff9640 1 -- 172.21.15.26:0/289876777 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25c80039f0 con 0x7f25d40c7180 2024-03-06T07:22:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.892+0000 7f25d0ff9640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.892+0000 7f25d0ff9640 1 -- 172.21.15.26:0/289876777 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f25c802f030 con 0x7f25d40c7180 2024-03-06T07:22:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.892+0000 7f25d37fe640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.893+0000 7f25d37fe640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c006b4d0 0x7f25c006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f25d40ac960 tx=0x7f25c4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:13.896+0000 7f25d0ff9640 1 -- 172.21.15.26:0/289876777 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f25d41574b0 con 0x7f25d40c7180 2024-03-06T07:22:14.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.079+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25d414fdb0 con 0x7f25d40c7180 2024-03-06T07:22:14.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.081+0000 7f25d0ff9640 1 -- 172.21.15.26:0/289876777 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+20975 (secure 0 0 0) 0x7f25c8041070 con 0x7f25d40c7180 2024-03-06T07:22:14.082 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:14.083 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:07.641811+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6832/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6835/440323369":"2024-03-07T07:22:04.621666+0000","172.21.15.26:6833/440323369":"2024-03-07T07:22:04.621666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:14.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c006b4d0 msgr2=0x7f25c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c006b4d0 0x7f25c006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f25d40ac960 tx=0x7f25c4009190 comp rx=0 tx=0).stop 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d40c7180 msgr2=0x7f25d41eec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d40c7180 0x7f25d41eec80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f25c8034040 tx=0x7f25c80046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 shutdown_connections 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25d4155b20 0x7f25d4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25d41527b0 0x7f25d41eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 --2- 172.21.15.26:0/289876777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d40c7180 0x7f25d41eec80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 >> 172.21.15.26:0/289876777 conn(0x7f25d40ac560 msgr2=0x7f25d40aad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 shutdown_connections 2024-03-06T07:22:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.083+0000 7f25da9c9640 1 -- 172.21.15.26:0/289876777 wait complete. 2024-03-06T07:22:14.096 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:22:14.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.196+0000 7fce6d864640 1 Processor -- start 2024-03-06T07:22:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.196+0000 7fce6d864640 1 -- start start 2024-03-06T07:22:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce6d864640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce68155c20 0x7fce680c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce6d864640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce680c17c0 0x7fce6815fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce6d864640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6806afc0 con 0x7fce68152920 2024-03-06T07:22:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce6d864640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6806b2a0 con 0x7fce68155c20 2024-03-06T07:22:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce6d864640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6806b580 con 0x7fce680c17c0 2024-03-06T07:22:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce677fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce680c17c0 0x7fce6815fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce667fc640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce68155c20 0x7fce680c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce677fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce680c17c0 0x7fce6815fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:55224/0 (socket says 172.21.15.26:55224) 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce677fe640 1 -- 172.21.15.26:0/321274560 learned_addr learned my addr 172.21.15.26:0/321274560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce66ffd640 1 --1- 172.21.15.26:0/321274560 >> v1:172.21.15.117:6789/0 conn(0x7fce68152920 0x7fce680c2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:33134/0 (socket says 172.21.15.26:33134) 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce677fe640 1 -- 172.21.15.26:0/321274560 >> v2:172.21.15.117:3300/0 conn(0x7fce68155c20 msgr2=0x7fce680c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.197+0000 7fce677fe640 1 --2- 172.21.15.26:0/321274560 >> v2:172.21.15.117:3300/0 conn(0x7fce68155c20 0x7fce680c1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.198+0000 7fce677fe640 1 -- 172.21.15.26:0/321274560 >> v1:172.21.15.117:6789/0 conn(0x7fce68152920 legacy=0x7fce680c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.198+0000 7fce677fe640 1 -- 172.21.15.26:0/321274560 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6806b860 con 0x7fce680c17c0 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.198+0000 7fce677fe640 1 --2- 172.21.15.26:0/321274560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce680c17c0 0x7fce6815fb20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce60009e50 tx=0x7fce6002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1626d3e145a5fc0f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.198+0000 7fce65ffb640 1 -- 172.21.15.26:0/321274560 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6002cb00 con 0x7fce680c17c0 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.198+0000 7fce65ffb640 1 -- 172.21.15.26:0/321274560 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6002cca0 con 0x7fce680c17c0 2024-03-06T07:22:14.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.198+0000 7fce65ffb640 1 -- 172.21.15.26:0/321274560 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60034610 con 0x7fce680c17c0 2024-03-06T07:22:14.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 -- 172.21.15.26:0/321274560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce680c17c0 msgr2=0x7fce6815fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 --2- 172.21.15.26:0/321274560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce680c17c0 0x7fce6815fb20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce60009e50 tx=0x7fce6002bf60 comp rx=0 tx=0).stop 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 -- 172.21.15.26:0/321274560 shutdown_connections 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 --2- 172.21.15.26:0/321274560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce680c17c0 0x7fce6815fb20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 --2- 172.21.15.26:0/321274560 >> v2:172.21.15.117:3300/0 conn(0x7fce68155c20 0x7fce680c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 -- 172.21.15.26:0/321274560 >> 172.21.15.26:0/321274560 conn(0x7fce680c4290 msgr2=0x7fce680c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 -- 172.21.15.26:0/321274560 shutdown_connections 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.199+0000 7fce6d864640 1 -- 172.21.15.26:0/321274560 wait complete. 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.201+0000 7fce6d864640 1 Processor -- start 2024-03-06T07:22:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.201+0000 7fce6d864640 1 -- start start 2024-03-06T07:22:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce6d864640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce680c17c0 0x7fce681eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce6d864640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce68152920 0x7fce681eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce6d864640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce68155c20 0x7fce681df210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce6d864640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce680c28a0 con 0x7fce680c17c0 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce6d864640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce681560b0 con 0x7fce68152920 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce6d864640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce680c2a20 con 0x7fce68155c20 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce680c17c0 0x7fce681eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce680c17c0 0x7fce681eb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50850/0 (socket says 172.21.15.26:50850) 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce667fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce68152920 0x7fce681eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce66ffd640 1 -- 172.21.15.26:0/1430787208 learned_addr learned my addr 172.21.15.26:0/1430787208 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.202+0000 7fce677fe640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce68155c20 0x7fce681df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce66ffd640 1 -- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce68155c20 msgr2=0x7fce681df210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce66ffd640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce68155c20 0x7fce681df210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce66ffd640 1 -- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce68152920 msgr2=0x7fce681eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce66ffd640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce68152920 0x7fce681eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce66ffd640 1 -- 172.21.15.26:0/1430787208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce681ebb00 con 0x7fce680c17c0 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce667fc640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce68152920 0x7fce681eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:14.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce677fe640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce68155c20 0x7fce681df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:14.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce66ffd640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce680c17c0 0x7fce681eb080 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fce5000c400 tx=0x7fce5000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce5ffff640 1 -- 172.21.15.26:0/1430787208 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500074e0 con 0x7fce680c17c0 2024-03-06T07:22:14.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce5ffff640 1 -- 172.21.15.26:0/1430787208 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50007680 con 0x7fce680c17c0 2024-03-06T07:22:14.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce680ab0e0 con 0x7fce680c17c0 2024-03-06T07:22:14.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce5ffff640 1 -- 172.21.15.26:0/1430787208 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50004a00 con 0x7fce680c17c0 2024-03-06T07:22:14.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.203+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce681e13b0 con 0x7fce680c17c0 2024-03-06T07:22:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.204+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce681ec000 con 0x7fce680c17c0 2024-03-06T07:22:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.205+0000 7fce5ffff640 1 -- 172.21.15.26:0/1430787208 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce5001a020 con 0x7fce680c17c0 2024-03-06T07:22:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.205+0000 7fce5ffff640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce4806b4d0 0x7fce4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.205+0000 7fce5ffff640 1 -- 172.21.15.26:0/1430787208 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7fce50089e30 con 0x7fce680c17c0 2024-03-06T07:22:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.205+0000 7fce667fc640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce4806b4d0 0x7fce4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.206+0000 7fce667fc640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce4806b4d0 0x7fce4806d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fce68159800 tx=0x7fce58009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.210+0000 7fce5ffff640 1 -- 172.21.15.26:0/1430787208 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fce681ec000 con 0x7fce680c17c0 2024-03-06T07:22:14.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.456+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce681e15b0 con 0x7fce680c17c0 2024-03-06T07:22:14.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.457+0000 7fce5ffff640 1 -- 172.21.15.26:0/1430787208 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6260 (secure 0 0 0) 0x7fce681e15b0 con 0x7fce680c17c0 2024-03-06T07:22:14.458 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:14.459 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:22:10.655453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5414":{"gid":5414,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/2696722008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2696722008},{"type":"v1","addr":"172.21.15.26:6835","nonce":2696722008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5414,"qdb_cluster":[5414]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:22:14.459 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 43 2024-03-06T07:22:14.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce4806b4d0 msgr2=0x7fce4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce4806b4d0 0x7fce4806d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fce68159800 tx=0x7fce58009190 comp rx=0 tx=0).stop 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce680c17c0 msgr2=0x7fce681eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce680c17c0 0x7fce681eb080 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fce5000c400 tx=0x7fce5000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 shutdown_connections 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce4806b4d0 0x7fce4806d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce68155c20 0x7fce681df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce68152920 0x7fce681eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 --2- 172.21.15.26:0/1430787208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce680c17c0 0x7fce681eb080 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 >> 172.21.15.26:0/1430787208 conn(0x7fce680c4290 msgr2=0x7fce6814d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 shutdown_connections 2024-03-06T07:22:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.459+0000 7fce6d864640 1 -- 172.21.15.26:0/1430787208 wait complete. 2024-03-06T07:22:14.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.571+0000 7f6edebf4640 1 Processor -- start 2024-03-06T07:22:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.571+0000 7f6edebf4640 1 -- start start 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6edebf4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ed80c88c0 0x7f6ed8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6edebf4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ed8158b60 0x7f6ed8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6edebf4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80c73d0 con 0x7f6ed814f360 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6edebf4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80c8dd0 con 0x7f6ed80c88c0 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6edebf4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80be2e0 con 0x7f6ed8158b60 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6ed7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ed80c88c0 0x7f6ed8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6ed7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ed80c88c0 0x7f6ed8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50866/0 (socket says 172.21.15.26:50866) 2024-03-06T07:22:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6ed7fff640 1 -- 172.21.15.26:0/643231456 learned_addr learned my addr 172.21.15.26:0/643231456 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6edd191640 1 --2- 172.21.15.26:0/643231456 >> v2:172.21.15.117:3300/0 conn(0x7f6ed8158b60 0x7f6ed8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6ed7fff640 1 -- 172.21.15.26:0/643231456 >> v2:172.21.15.117:3300/0 conn(0x7f6ed8158b60 msgr2=0x7f6ed8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6ed7fff640 1 --2- 172.21.15.26:0/643231456 >> v2:172.21.15.117:3300/0 conn(0x7f6ed8158b60 0x7f6ed8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6ed7fff640 1 -- 172.21.15.26:0/643231456 >> v1:172.21.15.117:6789/0 conn(0x7f6ed814f360 legacy=0x7f6ed80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.572+0000 7f6ed7fff640 1 -- 172.21.15.26:0/643231456 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80c7670 con 0x7f6ed80c88c0 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.573+0000 7f6ed7fff640 1 --2- 172.21.15.26:0/643231456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed8155c10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8152d60 tx=0x7f6ec002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a3fc488b8361ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.573+0000 7f6ed77fe640 1 -- 172.21.15.26:0/643231456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0035020 con 0x7f6ed80c88c0 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.573+0000 7f6ed77fe640 1 -- 172.21.15.26:0/643231456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec0031870 con 0x7f6ed80c88c0 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.573+0000 7f6ed77fe640 1 -- 172.21.15.26:0/643231456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0031c40 con 0x7f6ed80c88c0 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 -- 172.21.15.26:0/643231456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 msgr2=0x7f6ed8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 --2- 172.21.15.26:0/643231456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed8155c10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8152d60 tx=0x7f6ec002b910 comp rx=0 tx=0).stop 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 -- 172.21.15.26:0/643231456 shutdown_connections 2024-03-06T07:22:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 --2- 172.21.15.26:0/643231456 >> v2:172.21.15.117:3300/0 conn(0x7f6ed8158b60 0x7f6ed8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 --2- 172.21.15.26:0/643231456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed8155c10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 -- 172.21.15.26:0/643231456 >> 172.21.15.26:0/643231456 conn(0x7f6ed80ac570 msgr2=0x7f6ed80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 -- 172.21.15.26:0/643231456 shutdown_connections 2024-03-06T07:22:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.574+0000 7f6edebf4640 1 -- 172.21.15.26:0/643231456 wait complete. 2024-03-06T07:22:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.576+0000 7f6edebf4640 1 Processor -- start 2024-03-06T07:22:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.576+0000 7f6edebf4640 1 -- start start 2024-03-06T07:22:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edebf4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed81ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edebf4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed814f360 0x7f6ed80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edebf4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed8158b60 0x7f6ed80ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edebf4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80c5a50 con 0x7f6ed80c88c0 2024-03-06T07:22:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edebf4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80c7c30 con 0x7f6ed814f360 2024-03-06T07:22:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edebf4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80c5bd0 con 0x7f6ed8158b60 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed81ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed81ee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50874/0 (socket says 172.21.15.26:50874) 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 -- 172.21.15.26:0/3868165700 learned_addr learned my addr 172.21.15.26:0/3868165700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6ed7fff640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed814f360 0x7f6ed80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edd191640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed8158b60 0x7f6ed80ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 -- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed8158b60 msgr2=0x7f6ed80ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed8158b60 0x7f6ed80ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 -- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed814f360 msgr2=0x7f6ed80ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed814f360 0x7f6ed80ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.577+0000 7f6edc990640 1 -- 172.21.15.26:0/3868165700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80ad6d0 con 0x7f6ed80c88c0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.578+0000 7f6edc990640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed81ee330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000c7c0 tx=0x7f6ed000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.578+0000 7f6ed57fa640 1 -- 172.21.15.26:0/3868165700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0016020 con 0x7f6ed80c88c0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.578+0000 7f6ed57fa640 1 -- 172.21.15.26:0/3868165700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed0006d50 con 0x7f6ed80c88c0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.578+0000 7f6ed57fa640 1 -- 172.21.15.26:0/3868165700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0003bb0 con 0x7f6ed80c88c0 2024-03-06T07:22:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.578+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed80adac0 con 0x7f6ed80c88c0 2024-03-06T07:22:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.578+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed80add80 con 0x7f6ed80c88c0 2024-03-06T07:22:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.579+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed8069460 con 0x7f6ed80c88c0 2024-03-06T07:22:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.579+0000 7f6ed57fa640 1 -- 172.21.15.26:0/3868165700 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ed000a850 con 0x7f6ed80c88c0 2024-03-06T07:22:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.580+0000 7f6ed57fa640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.580+0000 7f6ed57fa640 1 -- 172.21.15.26:0/3868165700 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f6ed008a3a0 con 0x7f6ed80c88c0 2024-03-06T07:22:14.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.580+0000 7f6ed7fff640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.580+0000 7f6ed7fff640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6eb806b4d0 0x7f6eb806d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0036000 tx=0x7f6ec003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.588+0000 7f6ed57fa640 1 -- 172.21.15.26:0/3868165700 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ed8069460 con 0x7f6ed80c88c0 2024-03-06T07:22:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.834+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ed80acbd0 con 0x7f6ed80c88c0 2024-03-06T07:22:14.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.834+0000 7f6ed57fa640 1 -- 172.21.15.26:0/3868165700 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6260 (secure 0 0 0) 0x7f6ed0020020 con 0x7f6ed80c88c0 2024-03-06T07:22:14.836 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:14.836 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":6,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:42.669148+0000","modified":"2024-03-06T07:22:10.655453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5414":{"gid":5414,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/2696722008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2696722008},{"type":"v1","addr":"172.21.15.26:6835","nonce":2696722008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5414,"qdb_cluster":[5414]},"id":3},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:21:57.725575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5617":{"gid":5617,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6836/3928098986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3928098986},{"type":"v1","addr":"172.21.15.117:6836","nonce":3928098986}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5617,"qdb_cluster":[5617]},"id":4}]} 2024-03-06T07:22:14.836 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 43 2024-03-06T07:22:14.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6eb806b4d0 msgr2=0x7f6eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6eb806b4d0 0x7f6eb806d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0036000 tx=0x7f6ec003c040 comp rx=0 tx=0).stop 2024-03-06T07:22:14.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 msgr2=0x7f6ed81ee330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed81ee330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000c7c0 tx=0x7f6ed000cc80 comp rx=0 tx=0).stop 2024-03-06T07:22:14.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 shutdown_connections 2024-03-06T07:22:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed8158b60 0x7f6ed80ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed814f360 0x7f6ed80ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 --2- 172.21.15.26:0/3868165700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed80c88c0 0x7f6ed81ee330 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.836+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 >> 172.21.15.26:0/3868165700 conn(0x7f6ed80ac570 msgr2=0x7f6ed80cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.837+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 shutdown_connections 2024-03-06T07:22:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.837+0000 7f6edebf4640 1 -- 172.21.15.26:0/3868165700 wait complete. 2024-03-06T07:22:14.849 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.949+0000 7efe470fa640 1 Processor -- start 2024-03-06T07:22:14.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.949+0000 7efe470fa640 1 -- start start 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.949+0000 7efe470fa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efe400c88c0 0x7efe40155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.949+0000 7efe470fa640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe400c73d0 con 0x7efe40158b60 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.949+0000 7efe470fa640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe400be2e0 con 0x7efe4014f360 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe470fa640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe40159700 con 0x7efe400c88c0 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe3ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efe400c88c0 0x7efe40155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe45697640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efe40158b60 0x7efe40158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47692/0 (socket says 172.21.15.26:47692) 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe45697640 1 -- 172.21.15.26:0/2593690702 learned_addr learned my addr 172.21.15.26:0/2593690702 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe3ffff640 1 -- 172.21.15.26:0/2593690702 >> v1:172.21.15.117:6789/0 conn(0x7efe4014f360 legacy=0x7efe400c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe3ffff640 1 -- 172.21.15.26:0/2593690702 >> v1:172.21.15.26:6789/0 conn(0x7efe40158b60 legacy=0x7efe40158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe3ffff640 1 -- 172.21.15.26:0/2593690702 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe400c7670 con 0x7efe400c88c0 2024-03-06T07:22:14.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.950+0000 7efe3f7fe640 1 -- 172.21.15.26:0/2593690702 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911997310 0 0) 0x7efe400c73d0 con 0x7efe40158b60 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.951+0000 7efe3ffff640 1 --2- 172.21.15.26:0/2593690702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe400c88c0 0x7efe40155c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccae41ce5628fa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.951+0000 7efe3f7fe640 1 -- 172.21.15.26:0/2593690702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c035020 con 0x7efe400c88c0 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.951+0000 7efe3f7fe640 1 -- 172.21.15.26:0/2593690702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c031830 con 0x7efe400c88c0 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.951+0000 7efe3f7fe640 1 -- 172.21.15.26:0/2593690702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c031c00 con 0x7efe400c88c0 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.952+0000 7efe470fa640 1 -- 172.21.15.26:0/2593690702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe400c88c0 msgr2=0x7efe40155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.952+0000 7efe470fa640 1 --2- 172.21.15.26:0/2593690702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe400c88c0 0x7efe40155c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.952+0000 7efe470fa640 1 -- 172.21.15.26:0/2593690702 shutdown_connections 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.952+0000 7efe470fa640 1 --2- 172.21.15.26:0/2593690702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe400c88c0 0x7efe40155c10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.952+0000 7efe470fa640 1 -- 172.21.15.26:0/2593690702 >> 172.21.15.26:0/2593690702 conn(0x7efe400ac570 msgr2=0x7efe400ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.952+0000 7efe470fa640 1 -- 172.21.15.26:0/2593690702 shutdown_connections 2024-03-06T07:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.952+0000 7efe470fa640 1 -- 172.21.15.26:0/2593690702 wait complete. 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.953+0000 7efe470fa640 1 Processor -- start 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.953+0000 7efe470fa640 1 -- start start 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe470fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe400c88c0 0x7efe401df200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe470fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe4014f360 0x7efe400c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe470fa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe40158b60 0x7efe400bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe470fa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400c7c30 con 0x7efe4014f360 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe470fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400c5a50 con 0x7efe400c88c0 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe470fa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400c5bd0 con 0x7efe40158b60 2024-03-06T07:22:14.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe4014f360 0x7efe400c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe4014f360 0x7efe400c0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50894/0 (socket says 172.21.15.26:50894) 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 -- 172.21.15.26:0/482952073 learned_addr learned my addr 172.21.15.26:0/482952073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe44e96640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe400c88c0 0x7efe401df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe45697640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe40158b60 0x7efe400bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 -- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe40158b60 msgr2=0x7efe400bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe40158b60 0x7efe400bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 -- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe400c88c0 msgr2=0x7efe401df200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe400c88c0 0x7efe401df200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 -- 172.21.15.26:0/482952073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe401df740 con 0x7efe4014f360 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.954+0000 7efe3ffff640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe4014f360 0x7efe400c0f50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.955+0000 7efe3d7fa640 1 -- 172.21.15.26:0/482952073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c035050 con 0x7efe4014f360 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.955+0000 7efe3d7fa640 1 -- 172.21.15.26:0/482952073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c004230 con 0x7efe4014f360 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.955+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe40154830 con 0x7efe4014f360 2024-03-06T07:22:14.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.955+0000 7efe3d7fa640 1 -- 172.21.15.26:0/482952073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c03b740 con 0x7efe4014f360 2024-03-06T07:22:14.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.955+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe400be620 con 0x7efe4014f360 2024-03-06T07:22:14.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.956+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe401552c0 con 0x7efe4014f360 2024-03-06T07:22:14.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.957+0000 7efe3d7fa640 1 -- 172.21.15.26:0/482952073 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe2c038020 con 0x7efe4014f360 2024-03-06T07:22:14.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.957+0000 7efe3d7fa640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe2406b4d0 0x7efe2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.957+0000 7efe3d7fa640 1 -- 172.21.15.26:0/482952073 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7efe2c0ac470 con 0x7efe4014f360 2024-03-06T07:22:14.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.957+0000 7efe44e96640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe2406b4d0 0x7efe2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.958+0000 7efe44e96640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe2406b4d0 0x7efe2406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe34000f60 tx=0x7efe34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:14.961+0000 7efe3d7fa640 1 -- 172.21.15.26:0/482952073 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efe2c040070 con 0x7efe4014f360 2024-03-06T07:22:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.143+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe4004f330 con 0x7efe4014f360 2024-03-06T07:22:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.145+0000 7efe3d7fa640 1 -- 172.21.15.26:0/482952073 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+20975 (secure 0 0 0) 0x7efe2c07a760 con 0x7efe4014f360 2024-03-06T07:22:15.147 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:15.147 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:07.641811+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:40.938361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:41.634747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6832/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6835/440323369":"2024-03-07T07:22:04.621666+0000","172.21.15.26:6833/440323369":"2024-03-07T07:22:04.621666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:15.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe2406b4d0 msgr2=0x7efe2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe2406b4d0 0x7efe2406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe34000f60 tx=0x7efe34009040 comp rx=0 tx=0).stop 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe4014f360 msgr2=0x7efe400c0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe4014f360 0x7efe400c0f50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 shutdown_connections 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe2406b4d0 0x7efe2406d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe40158b60 0x7efe400bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe4014f360 0x7efe400c0f50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 --2- 172.21.15.26:0/482952073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe400c88c0 0x7efe401df200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 >> 172.21.15.26:0/482952073 conn(0x7efe400ac570 msgr2=0x7efe400c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 shutdown_connections 2024-03-06T07:22:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.147+0000 7efe470fa640 1 -- 172.21.15.26:0/482952073 wait complete. 2024-03-06T07:22:15.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:22:15.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.261+0000 7fc621c88640 1 Processor -- start 2024-03-06T07:22:15.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.261+0000 7fc621c88640 1 -- start start 2024-03-06T07:22:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.261+0000 7fc621c88640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc61c152b40 0x7fc61c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.261+0000 7fc621c88640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc61c0724d0 0x7fc61c1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.261+0000 7fc621c88640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61c0733f0 con 0x7fc61c0a7ff0 2024-03-06T07:22:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.261+0000 7fc621c88640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61c072d70 con 0x7fc61c152b40 2024-03-06T07:22:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.261+0000 7fc621c88640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61c0c82c0 con 0x7fc61c0724d0 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc61c152b40 0x7fc61c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc61c152b40 0x7fc61c0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50906/0 (socket says 172.21.15.26:50906) 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61b7fe640 1 -- 172.21.15.26:0/2372681071 learned_addr learned my addr 172.21.15.26:0/2372681071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc61c0a7ff0 0x7fc61c155730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47708/0 (socket says 172.21.15.26:47708) 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61affd640 1 --2- 172.21.15.26:0/2372681071 >> v2:172.21.15.117:3300/0 conn(0x7fc61c0724d0 0x7fc61c1551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61b7fe640 1 -- 172.21.15.26:0/2372681071 >> v2:172.21.15.117:3300/0 conn(0x7fc61c0724d0 msgr2=0x7fc61c1551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61b7fe640 1 --2- 172.21.15.26:0/2372681071 >> v2:172.21.15.117:3300/0 conn(0x7fc61c0724d0 0x7fc61c1551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61b7fe640 1 -- 172.21.15.26:0/2372681071 >> v1:172.21.15.26:6789/0 conn(0x7fc61c0a7ff0 legacy=0x7fc61c155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.262+0000 7fc61b7fe640 1 -- 172.21.15.26:0/2372681071 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c06fd30 con 0x7fc61c152b40 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc61b7fe640 1 --2- 172.21.15.26:0/2372681071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c0c8810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0c5330 tx=0x7fc60c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c8b71fd5bc125d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc61a7fc640 1 -- 172.21.15.26:0/2372681071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c035020 con 0x7fc61c152b40 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc61a7fc640 1 -- 172.21.15.26:0/2372681071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60c031870 con 0x7fc61c152b40 2024-03-06T07:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc61a7fc640 1 -- 172.21.15.26:0/2372681071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c031c40 con 0x7fc61c152b40 2024-03-06T07:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 -- 172.21.15.26:0/2372681071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 msgr2=0x7fc61c0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 --2- 172.21.15.26:0/2372681071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c0c8810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0c5330 tx=0x7fc60c02b910 comp rx=0 tx=0).stop 2024-03-06T07:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 -- 172.21.15.26:0/2372681071 shutdown_connections 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 --2- 172.21.15.26:0/2372681071 >> v2:172.21.15.117:3300/0 conn(0x7fc61c0724d0 0x7fc61c1551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 --2- 172.21.15.26:0/2372681071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c0c8810 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 -- 172.21.15.26:0/2372681071 >> 172.21.15.26:0/2372681071 conn(0x7fc61c06b970 msgr2=0x7fc61c0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 -- 172.21.15.26:0/2372681071 shutdown_connections 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.263+0000 7fc621c88640 1 -- 172.21.15.26:0/2372681071 wait complete. 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.264+0000 7fc621c88640 1 Processor -- start 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc621c88640 1 -- start start 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc621c88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc61c0724d0 0x7fc61c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc621c88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc61c0a7ff0 0x7fc61c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc621c88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c14b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc621c88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c0c8d50 con 0x7fc61c152b40 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc621c88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c070490 con 0x7fc61c0724d0 2024-03-06T07:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc621c88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c0c8ed0 con 0x7fc61c0a7ff0 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.265+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c14b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50912/0 (socket says 172.21.15.26:50912) 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61bfff640 1 -- 172.21.15.26:0/1305242417 learned_addr learned my addr 172.21.15.26:0/1305242417 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61b7fe640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc61c0724d0 0x7fc61c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61affd640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc61c0a7ff0 0x7fc61c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61bfff640 1 -- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc61c0a7ff0 msgr2=0x7fc61c14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61bfff640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc61c0a7ff0 0x7fc61c14afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61bfff640 1 -- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc61c0724d0 msgr2=0x7fc61c06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61bfff640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc61c0724d0 0x7fc61c06d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61bfff640 1 -- 172.21.15.26:0/1305242417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c0a9590 con 0x7fc61c152b40 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61b7fe640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc61c0724d0 0x7fc61c06d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc61bfff640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c14b520 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c760 tx=0x7fc61400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc603fff640 1 -- 172.21.15.26:0/1305242417 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614017070 con 0x7fc61c152b40 2024-03-06T07:22:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc603fff640 1 -- 172.21.15.26:0/1305242417 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61400a640 con 0x7fc61c152b40 2024-03-06T07:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61c0a9870 con 0x7fc61c152b40 2024-03-06T07:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc603fff640 1 -- 172.21.15.26:0/1305242417 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61400a9f0 con 0x7fc61c152b40 2024-03-06T07:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.266+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61c06da00 con 0x7fc61c152b40 2024-03-06T07:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.267+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61c04f330 con 0x7fc61c152b40 2024-03-06T07:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.268+0000 7fc603fff640 1 -- 172.21.15.26:0/1305242417 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc61401a070 con 0x7fc61c152b40 2024-03-06T07:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.268+0000 7fc603fff640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.268+0000 7fc603fff640 1 -- 172.21.15.26:0/1305242417 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7187+0+0 (secure 0 0 0) 0x7fc614089c90 con 0x7fc61c152b40 2024-03-06T07:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.268+0000 7fc61b7fe640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.269+0000 7fc61b7fe640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc60c036000 tx=0x7fc60c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.273+0000 7fc603fff640 1 -- 172.21.15.26:0/1305242417 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc614057ef0 con 0x7fc61c152b40 2024-03-06T07:22:15.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.496+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc61c1563c0 con 0x7fc61c152b40 2024-03-06T07:22:15.498 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:22:15.496+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.497+0000 7fc603fff640 1 -- 172.21.15.26:0/1305242417 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc61405bf90 con 0x7fc61c152b40 2024-03-06T07:22:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:22:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.499+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 msgr2=0x7fc60806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.499+0000 7fc621c88640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc60c036000 tx=0x7fc60c03c040 comp rx=0 tx=0).stop 2024-03-06T07:22:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.499+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 msgr2=0x7fc61c14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.499+0000 7fc621c88640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c14b520 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c760 tx=0x7fc61400cc20 comp rx=0 tx=0).stop 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.499+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 shutdown_connections 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.500+0000 7fc621c88640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.500+0000 7fc621c88640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc61c152b40 0x7fc61c14b520 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.500+0000 7fc621c88640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc61c0a7ff0 0x7fc61c14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.500+0000 7fc621c88640 1 --2- 172.21.15.26:0/1305242417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc61c0724d0 0x7fc61c06d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.500+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 >> 172.21.15.26:0/1305242417 conn(0x7fc61c06b970 msgr2=0x7fc61c14f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.500+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 shutdown_connections 2024-03-06T07:22:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.500+0000 7fc621c88640 1 -- 172.21.15.26:0/1305242417 wait complete. 2024-03-06T07:22:15.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:22:15.530 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:22:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.620+0000 7f669e5d0640 1 Processor -- start 2024-03-06T07:22:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.620+0000 7f669e5d0640 1 -- start start 2024-03-06T07:22:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f669e5d0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f669806bff0 0x7f66980c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f669e5d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66980c7210 0x7f66980c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f669e5d0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66980b04b0 con 0x7f66980b00d0 2024-03-06T07:22:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f669e5d0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6698156160 con 0x7f66980c7210 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f669e5d0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66980c5710 con 0x7f669806bff0 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f669cb6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66980c7210 0x7f66980c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f6697fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f66980b00d0 0x7f669806b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47730/0 (socket says 172.21.15.26:47730) 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f6697fff640 1 -- 172.21.15.26:0/4021000760 learned_addr learned my addr 172.21.15.26:0/4021000760 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.621+0000 7f66977fe640 1 --2- 172.21.15.26:0/4021000760 >> v2:172.21.15.117:3300/0 conn(0x7f669806bff0 0x7f66980c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.622+0000 7f669cb6d640 1 -- 172.21.15.26:0/4021000760 >> v2:172.21.15.117:3300/0 conn(0x7f669806bff0 msgr2=0x7f66980c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.622+0000 7f669cb6d640 1 --2- 172.21.15.26:0/4021000760 >> v2:172.21.15.117:3300/0 conn(0x7f669806bff0 0x7f66980c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.622+0000 7f669cb6d640 1 -- 172.21.15.26:0/4021000760 >> v1:172.21.15.26:6789/0 conn(0x7f66980b00d0 legacy=0x7f669806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.622+0000 7f669cb6d640 1 -- 172.21.15.26:0/4021000760 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669806d210 con 0x7f66980c7210 2024-03-06T07:22:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.622+0000 7f6696ffd640 1 -- 172.21.15.26:0/4021000760 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307727736 0 0) 0x7f66980b04b0 con 0x7f66980b00d0 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.622+0000 7f669cb6d640 1 --2- 172.21.15.26:0/4021000760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66980c7210 0x7f66980c7680 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f669806ce20 tx=0x7f669002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74711ddf8b32112 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f6696ffd640 1 -- 172.21.15.26:0/4021000760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690039070 con 0x7f66980c7210 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f6696ffd640 1 -- 172.21.15.26:0/4021000760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669002cc80 con 0x7f66980c7210 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f6696ffd640 1 -- 172.21.15.26:0/4021000760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690034660 con 0x7f66980c7210 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 -- 172.21.15.26:0/4021000760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66980c7210 msgr2=0x7f66980c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 --2- 172.21.15.26:0/4021000760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66980c7210 0x7f66980c7680 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f669806ce20 tx=0x7f669002bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 -- 172.21.15.26:0/4021000760 shutdown_connections 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 --2- 172.21.15.26:0/4021000760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66980c7210 0x7f66980c7680 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 --2- 172.21.15.26:0/4021000760 >> v2:172.21.15.117:3300/0 conn(0x7f669806bff0 0x7f66980c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 -- 172.21.15.26:0/4021000760 >> 172.21.15.26:0/4021000760 conn(0x7f66980ac810 msgr2=0x7f66980acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 -- 172.21.15.26:0/4021000760 shutdown_connections 2024-03-06T07:22:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.623+0000 7f669e5d0640 1 -- 172.21.15.26:0/4021000760 wait complete. 2024-03-06T07:22:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.624+0000 7f669e5d0640 1 Processor -- start 2024-03-06T07:22:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.624+0000 7f669e5d0640 1 -- start start 2024-03-06T07:22:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f669e5d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669806bff0 0x7f66981eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f669e5d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66980b00d0 0x7f66980aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f669e5d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66980c7210 0x7f669814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f669e5d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c5ba0 con 0x7f669806bff0 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f669e5d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6698152c50 con 0x7f66980b00d0 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f669e5d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c5d20 con 0x7f66980c7210 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f6697fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669806bff0 0x7f66981eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f6697fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669806bff0 0x7f66981eebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50934/0 (socket says 172.21.15.26:50934) 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f6697fff640 1 -- 172.21.15.26:0/3161511774 learned_addr learned my addr 172.21.15.26:0/3161511774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f66977fe640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66980b00d0 0x7f66980aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.625+0000 7f669cb6d640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66980c7210 0x7f669814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6697fff640 1 -- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66980c7210 msgr2=0x7f669814e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6697fff640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66980c7210 0x7f669814e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6697fff640 1 -- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66980b00d0 msgr2=0x7f66980aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6697fff640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66980b00d0 0x7f66980aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6697fff640 1 -- 172.21.15.26:0/3161511774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66980c4ae0 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6697fff640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669806bff0 0x7f66981eebf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f668800c4d0 tx=0x7f668800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6694ff9640 1 -- 172.21.15.26:0/3161511774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688018070 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6694ff9640 1 -- 172.21.15.26:0/3161511774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688007500 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66980c4dc0 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f6694ff9640 1 -- 172.21.15.26:0/3161511774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688004a20 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.626+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66980af0e0 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.627+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6698158490 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.627+0000 7f6694ff9640 1 -- 172.21.15.26:0/3161511774 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f668801b070 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.628+0000 7f6694ff9640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f668406b4d0 0x7f668406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.628+0000 7f6694ff9640 1 -- 172.21.15.26:0/3161511774 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7273+0+0 (secure 0 0 0) 0x7f6688089a20 con 0x7f669806bff0 2024-03-06T07:22:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.628+0000 7f66977fe640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f668406b4d0 0x7f668406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.628+0000 7f66977fe640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f668406b4d0 0x7f668406d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6698151e90 tx=0x7f668c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.632+0000 7f6694ff9640 1 -- 172.21.15.26:0/3161511774 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6688057cc0 con 0x7f669806bff0 2024-03-06T07:22:15.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:15.851+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6698075300 con 0x7f669806bff0 2024-03-06T07:22:16.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.499+0000 7f6694ff9640 1 -- 172.21.15.26:0/3161511774 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6698075300 con 0x7f669806bff0 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f668406b4d0 msgr2=0x7f668406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f668406b4d0 0x7f668406d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6698151e90 tx=0x7f668c009190 comp rx=0 tx=0).stop 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669806bff0 msgr2=0x7f66981eebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669806bff0 0x7f66981eebf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f668800c4d0 tx=0x7f668800c990 comp rx=0 tx=0).stop 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 shutdown_connections 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f668406b4d0 0x7f668406d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66980c7210 0x7f669814e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66980b00d0 0x7f66980aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 --2- 172.21.15.26:0/3161511774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669806bff0 0x7f66981eebf0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 >> 172.21.15.26:0/3161511774 conn(0x7f66980ac810 msgr2=0x7f66980acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:16.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 shutdown_connections 2024-03-06T07:22:16.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.501+0000 7f669e5d0640 1 -- 172.21.15.26:0/3161511774 wait complete. 2024-03-06T07:22:16.509 INFO:tasks.ceph.mds.b.smithi117.stderr: -588> 2024-03-06T07:21:58.603+0000 7f9723f83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:16.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:22:16.535 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:22:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f37a22c2640 1 Processor -- start 2024-03-06T07:22:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f37a22c2640 1 -- start start 2024-03-06T07:22:16.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f37a22c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f379c0c2900 0x7f379c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f37a22c2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c159740 con 0x7f379c152920 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f37a22c2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c1551d0 con 0x7f379c0c80d0 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f37a22c2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379c0c8ab0 con 0x7f379c0c2900 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f379b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f379c0c2900 0x7f379c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f379bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f379c152920 0x7f379c155970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47746/0 (socket says 172.21.15.26:47746) 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.667+0000 7f379bfff640 1 -- 172.21.15.26:0/139156261 learned_addr learned my addr 172.21.15.26:0/139156261 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.668+0000 7f379b7fe640 1 -- 172.21.15.26:0/139156261 >> v1:172.21.15.117:6789/0 conn(0x7f379c0c80d0 legacy=0x7f379c15fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.668+0000 7f379b7fe640 1 -- 172.21.15.26:0/139156261 >> v1:172.21.15.26:6789/0 conn(0x7f379c152920 legacy=0x7f379c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.668+0000 7f379b7fe640 1 -- 172.21.15.26:0/139156261 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0c8d90 con 0x7f379c0c2900 2024-03-06T07:22:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.668+0000 7f379b7fe640 1 --2- 172.21.15.26:0/139156261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c2900 0x7f379c0c7b90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f37900097a0 tx=0x7f379002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be097814c7324531 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f379affd640 1 -- 172.21.15.26:0/139156261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37900315a0 con 0x7f379c0c2900 2024-03-06T07:22:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f379affd640 1 -- 172.21.15.26:0/139156261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3790037070 con 0x7f379c0c2900 2024-03-06T07:22:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f379affd640 1 -- 172.21.15.26:0/139156261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790031960 con 0x7f379c0c2900 2024-03-06T07:22:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f37a22c2640 1 -- 172.21.15.26:0/139156261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c2900 msgr2=0x7f379c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f37a22c2640 1 --2- 172.21.15.26:0/139156261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c2900 0x7f379c0c7b90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f37900097a0 tx=0x7f379002b840 comp rx=0 tx=0).stop 2024-03-06T07:22:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f37a22c2640 1 -- 172.21.15.26:0/139156261 shutdown_connections 2024-03-06T07:22:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f37a22c2640 1 --2- 172.21.15.26:0/139156261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c2900 0x7f379c0c7b90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f37a22c2640 1 -- 172.21.15.26:0/139156261 >> 172.21.15.26:0/139156261 conn(0x7f379c0aa5c0 msgr2=0x7f379c0aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f37a22c2640 1 -- 172.21.15.26:0/139156261 shutdown_connections 2024-03-06T07:22:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.669+0000 7f37a22c2640 1 -- 172.21.15.26:0/139156261 wait complete. 2024-03-06T07:22:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.670+0000 7f37a22c2640 1 Processor -- start 2024-03-06T07:22:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a22c2640 1 -- start start 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a22c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0c2900 0x7f379c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a22c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c80d0 0x7f379c1eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a22c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c152920 0x7f379c06c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a22c2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c156080 con 0x7f379c0c80d0 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a22c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c155450 con 0x7f379c0c2900 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a22c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0c86c0 con 0x7f379c152920 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f379b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c80d0 0x7f379c1eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f379b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c80d0 0x7f379c1eb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50952/0 (socket says 172.21.15.26:50952) 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f379b7fe640 1 -- 172.21.15.26:0/1300284067 learned_addr learned my addr 172.21.15.26:0/1300284067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f379bfff640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0c2900 0x7f379c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.671+0000 7f37a085f640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c152920 0x7f379c06c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f379b7fe640 1 -- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c152920 msgr2=0x7f379c06c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f379b7fe640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c152920 0x7f379c06c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f379b7fe640 1 -- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0c2900 msgr2=0x7f379c151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f379b7fe640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0c2900 0x7f379c151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f379b7fe640 1 -- 172.21.15.26:0/1300284067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c152480 con 0x7f379c0c80d0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f379bfff640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0c2900 0x7f379c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f379b7fe640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c80d0 0x7f379c1eb6f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f37900098c0 tx=0x7f37900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f3798ff9640 1 -- 172.21.15.26:0/1300284067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790032840 con 0x7f379c0c80d0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f3798ff9640 1 -- 172.21.15.26:0/1300284067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37900329e0 con 0x7f379c0c80d0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c1e2010 con 0x7f379c0c80d0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f3798ff9640 1 -- 172.21.15.26:0/1300284067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790032dd0 con 0x7f379c0c80d0 2024-03-06T07:22:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.672+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c1e25d0 con 0x7f379c0c80d0 2024-03-06T07:22:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.673+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c04f330 con 0x7f379c0c80d0 2024-03-06T07:22:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.674+0000 7f3798ff9640 1 -- 172.21.15.26:0/1300284067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f379004d3f0 con 0x7f379c0c80d0 2024-03-06T07:22:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.674+0000 7f3798ff9640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f378806b4d0 0x7f378806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.674+0000 7f3798ff9640 1 -- 172.21.15.26:0/1300284067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7359+0+0 (secure 0 0 0) 0x7f379002f030 con 0x7f379c0c80d0 2024-03-06T07:22:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.674+0000 7f379bfff640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f378806b4d0 0x7f378806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.675+0000 7f379bfff640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f378806b4d0 0x7f378806d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f378c000f60 tx=0x7f378c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.678+0000 7f3798ff9640 1 -- 172.21.15.26:0/1300284067 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3790040070 con 0x7f379c0c80d0 2024-03-06T07:22:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:16.859+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f379c0c4050 con 0x7f379c0c80d0 2024-03-06T07:22:17.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.685+0000 7f3798ff9640 1 -- 172.21.15.26:0/1300284067 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) v1 ==== 181+0+0 (secure 0 0 0) 0x7f379c1551d0 con 0x7f379c0c80d0 2024-03-06T07:22:17.686 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f378806b4d0 msgr2=0x7f378806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f378806b4d0 0x7f378806d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f378c000f60 tx=0x7f378c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c80d0 msgr2=0x7f379c1eb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c80d0 0x7f379c1eb6f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f37900098c0 tx=0x7f37900046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 shutdown_connections 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f378806b4d0 0x7f378806d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c152920 0x7f379c06c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f379c0c80d0 0x7f379c1eb6f0 secure :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f37900098c0 tx=0x7f37900046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 --2- 172.21.15.26:0/1300284067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0c2900 0x7f379c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.688+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 >> 172.21.15.26:0/1300284067 conn(0x7f379c0aa5c0 msgr2=0x7f379c0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.689+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 shutdown_connections 2024-03-06T07:22:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.689+0000 7f37a22c2640 1 -- 172.21.15.26:0/1300284067 wait complete. 2024-03-06T07:22:17.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:22:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.814+0000 7ff73729a640 1 Processor -- start 2024-03-06T07:22:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.814+0000 7ff73729a640 1 -- start start 2024-03-06T07:22:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff73729a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7300c7470 0x7ff730155d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff73729a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 0x7ff7300c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff73729a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff730156260 con 0x7ff730152860 2024-03-06T07:22:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff73729a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff730158da0 con 0x7ff7300c7470 2024-03-06T07:22:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff73729a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300c7a10 con 0x7ff730159230 2024-03-06T07:22:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff735036640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff730152860 0x7ff7300c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55036/0 (socket says 172.21.15.26:55036) 2024-03-06T07:22:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff735036640 1 -- 172.21.15.26:0/1283037895 learned_addr learned my addr 172.21.15.26:0/1283037895 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508368063 0 0) 0x7ff72c001110 con 0x7ff730152860 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff71c003520 con 0x7ff730152860 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff734835640 1 --2- 172.21.15.26:0/1283037895 >> v2:172.21.15.117:3300/0 conn(0x7ff7300c7470 0x7ff730155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.815+0000 7ff735837640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 0x7ff7300c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275913574 0 0) 0x7ff71c003520 con 0x7ff730152860 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 msgr2=0x7ff7300c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 0x7ff7300c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 >> v2:172.21.15.117:3300/0 conn(0x7ff7300c7470 msgr2=0x7ff730155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 --2- 172.21.15.26:0/1283037895 >> v2:172.21.15.117:3300/0 conn(0x7ff7300c7470 0x7ff730155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300c7cf0 con 0x7ff730152860 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7ff72c001840 con 0x7ff730152860 2024-03-06T07:22:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 >> v1:172.21.15.26:6789/0 conn(0x7ff730152860 legacy=0x7ff7300c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff71c0039b0 0x7ff71c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 0x7ff71c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71c004d40 0x7ff71c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730158da0 con 0x7ff71c004d40 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300c7a10 con 0x7ff71c0039b0 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c00bb20 con 0x7ff730159230 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff735837640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71c004d40 0x7ff71c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff735036640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff71c0039b0 0x7ff71c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff734835640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 0x7ff71c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff735837640 1 -- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 msgr2=0x7ff71c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff735837640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff730159230 0x7ff71c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff735837640 1 -- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff71c0039b0 msgr2=0x7ff71c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff735837640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff71c0039b0 0x7ff71c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.816+0000 7ff735837640 1 -- 172.21.15.26:0/1283037895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c003520 con 0x7ff71c004d40 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff735837640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71c004d40 0x7ff71c0051c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff728007360 tx=0x7ff71c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b613fcfadc43fdd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728039070 con 0x7ff71c004d40 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72802ccd0 con 0x7ff71c004d40 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff727fff640 1 -- 172.21.15.26:0/1283037895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728009be0 con 0x7ff71c004d40 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff73729a640 1 -- 172.21.15.26:0/1283037895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71c004d40 msgr2=0x7ff71c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff73729a640 1 --2- 172.21.15.26:0/1283037895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71c004d40 0x7ff71c0051c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff728007360 tx=0x7ff71c005830 comp rx=0 tx=0).stop 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff735837640 1 -- 172.21.15.26:0/1283037895 reap_dead start 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff73729a640 1 -- 172.21.15.26:0/1283037895 shutdown_connections 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.817+0000 7ff73729a640 1 -- 172.21.15.26:0/1283037895 >> 172.21.15.26:0/1283037895 conn(0x7ff7300ac630 msgr2=0x7ff7300aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.818+0000 7ff73729a640 1 -- 172.21.15.26:0/1283037895 shutdown_connections 2024-03-06T07:22:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.818+0000 7ff73729a640 1 -- 172.21.15.26:0/1283037895 wait complete. 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 Processor -- start 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 -- start start 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff71c004d40 0x7ff73014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730157e40 0x7ff73014ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73014bfa0 0x7ff7301ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7301596c0 con 0x7ff73014bfa0 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730152c40 con 0x7ff730157e40 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.819+0000 7ff73729a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff730159840 con 0x7ff71c004d40 2024-03-06T07:22:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73014bfa0 0x7ff7301ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73014bfa0 0x7ff7301ed1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41396/0 (socket says 172.21.15.26:41396) 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 -- 172.21.15.26:0/3550322847 learned_addr learned my addr 172.21.15.26:0/3550322847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735036640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff71c004d40 0x7ff73014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff734835640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730157e40 0x7ff73014ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 -- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff71c004d40 msgr2=0x7ff73014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff71c004d40 0x7ff73014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 -- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730157e40 msgr2=0x7ff73014ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730157e40 0x7ff73014ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 -- 172.21.15.26:0/3550322847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73014e060 con 0x7ff73014bfa0 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff734835640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730157e40 0x7ff73014ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735036640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff71c004d40 0x7ff73014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff735837640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73014bfa0 0x7ff7301ed1c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff728037040 tx=0x7ff728004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff725ffb640 1 -- 172.21.15.26:0/3550322847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728039030 con 0x7ff73014bfa0 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff725ffb640 1 -- 172.21.15.26:0/3550322847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728043070 con 0x7ff73014bfa0 2024-03-06T07:22:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7301ec720 con 0x7ff73014bfa0 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff725ffb640 1 -- 172.21.15.26:0/3550322847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728009be0 con 0x7ff73014bfa0 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.820+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7301ec9c0 con 0x7ff73014bfa0 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.821+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7300ad3d0 con 0x7ff73014bfa0 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.822+0000 7ff725ffb640 1 -- 172.21.15.26:0/3550322847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff72803d7d0 con 0x7ff73014bfa0 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.822+0000 7ff725ffb640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71406b500 0x7ff71406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.822+0000 7ff725ffb640 1 -- 172.21.15.26:0/3550322847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6921+0+0 (secure 0 0 0) 0x7ff7280ac990 con 0x7ff73014bfa0 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.822+0000 7ff735036640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71406b500 0x7ff71406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.823+0000 7ff735036640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71406b500 0x7ff71406d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff72c001eb0 tx=0x7ff72c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:17.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:17.827+0000 7ff725ffb640 1 -- 172.21.15.26:0/3550322847 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7300ad3d0 con 0x7ff73014bfa0 2024-03-06T07:22:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.018+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff73004f330 con 0x7ff73014bfa0 2024-03-06T07:22:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.685+0000 7ff725ffb640 1 -- 172.21.15.26:0/3550322847 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff7280aae20 con 0x7ff73014bfa0 2024-03-06T07:22:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:22:18.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71406b500 msgr2=0x7ff71406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71406b500 0x7ff71406d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff72c001eb0 tx=0x7ff72c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73014bfa0 msgr2=0x7ff7301ed1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73014bfa0 0x7ff7301ed1c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff728037040 tx=0x7ff728004ab0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 shutdown_connections 2024-03-06T07:22:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff71406b500 0x7ff71406d9c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73014bfa0 0x7ff7301ed1c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730157e40 0x7ff73014ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 --2- 172.21.15.26:0/3550322847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff71c004d40 0x7ff73014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 >> 172.21.15.26:0/3550322847 conn(0x7ff7300ac630 msgr2=0x7ff7300c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:18.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 shutdown_connections 2024-03-06T07:22:18.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.687+0000 7ff73729a640 1 -- 172.21.15.26:0/3550322847 wait complete. 2024-03-06T07:22:18.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.809+0000 7f8a53a88640 1 Processor -- start 2024-03-06T07:22:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.809+0000 7f8a53a88640 1 -- start start 2024-03-06T07:22:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a53a88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a53a88640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a4c0c3d60 0x7f8a4c0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a53a88640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c156f90 con 0x7f8a4c156a00 2024-03-06T07:22:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a53a88640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4c0c38d0 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a53a88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c157270 con 0x7f8a4c072720 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a52025640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a4c0c3d60 0x7f8a4c0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a52025640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a4c0c3d60 0x7f8a4c0c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41398/0 (socket says 172.21.15.26:41398) 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.810+0000 7f8a52025640 1 -- 172.21.15.26:0/3502193485 learned_addr learned my addr 172.21.15.26:0/3502193485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.811+0000 7f8a51824640 1 --2- 172.21.15.26:0/3502193485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.811+0000 7f8a52025640 1 -- 172.21.15.26:0/3502193485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 msgr2=0x7f8a4c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.811+0000 7f8a52025640 1 --2- 172.21.15.26:0/3502193485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.811+0000 7f8a52025640 1 -- 172.21.15.26:0/3502193485 >> v1:172.21.15.117:6789/0 conn(0x7f8a4c156a00 legacy=0x7f8a4c150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.811+0000 7f8a52025640 1 -- 172.21.15.26:0/3502193485 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0c3280 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.811+0000 7f8a51824640 1 --2- 172.21.15.26:0/3502193485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.811+0000 7f8a52025640 1 --2- 172.21.15.26:0/3502193485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c0c41f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009e80 tx=0x7f8a4802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727d536939ead6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a50822640 1 -- 172.21.15.26:0/3502193485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4802c9c0 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a50822640 1 -- 172.21.15.26:0/3502193485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4802cb60 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a50822640 1 -- 172.21.15.26:0/3502193485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a480343f0 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 -- 172.21.15.26:0/3502193485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 msgr2=0x7f8a4c0c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 --2- 172.21.15.26:0/3502193485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c0c41f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009e80 tx=0x7f8a4802be20 comp rx=0 tx=0).stop 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 -- 172.21.15.26:0/3502193485 shutdown_connections 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 --2- 172.21.15.26:0/3502193485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c0c41f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 --2- 172.21.15.26:0/3502193485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 -- 172.21.15.26:0/3502193485 >> 172.21.15.26:0/3502193485 conn(0x7f8a4c0aa630 msgr2=0x7f8a4c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 -- 172.21.15.26:0/3502193485 shutdown_connections 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.812+0000 7f8a53a88640 1 -- 172.21.15.26:0/3502193485 wait complete. 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.813+0000 7f8a53a88640 1 Processor -- start 2024-03-06T07:22:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.813+0000 7f8a53a88640 1 -- start start 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a53a88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a53a88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c1e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a53a88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c156a00 0x7f8a4c1e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a53a88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c1509a0 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a53a88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c06f8f0 con 0x7f8a4c156a00 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a53a88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c150b20 con 0x7f8a4c072720 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c1e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c1e8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41412/0 (socket says 172.21.15.26:41412) 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 -- 172.21.15.26:0/2242610210 learned_addr learned my addr 172.21.15.26:0/2242610210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a52025640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c156a00 0x7f8a4c1e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51824640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 -- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 msgr2=0x7f8a4c1535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 -- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c156a00 msgr2=0x7f8a4c1e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c156a00 0x7f8a4c1e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a51023640 1 -- 172.21.15.26:0/2242610210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c1e8920 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.814+0000 7f8a52025640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c156a00 0x7f8a4c1e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.815+0000 7f8a51023640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c1e8f70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00b420 tx=0x7f8a3c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.815+0000 7f8a427fc640 1 -- 172.21.15.26:0/2242610210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c011830 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.815+0000 7f8a427fc640 1 -- 172.21.15.26:0/2242610210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c016070 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.815+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c0acfc0 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.815+0000 7f8a427fc640 1 -- 172.21.15.26:0/2242610210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c011c10 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.815+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c0ad960 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.816+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c0adba0 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.816+0000 7f8a427fc640 1 -- 172.21.15.26:0/2242610210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a3c01b460 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.817+0000 7f8a427fc640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3006b500 0x7f8a3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.817+0000 7f8a427fc640 1 -- 172.21.15.26:0/2242610210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f8a3c089ef0 con 0x7f8a4c0c3d60 2024-03-06T07:22:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.817+0000 7f8a51824640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3006b500 0x7f8a3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.817+0000 7f8a51824640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3006b500 0x7f8a3006d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400a990 tx=0x7f8a44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:18.821+0000 7f8a427fc640 1 -- 172.21.15.26:0/2242610210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8a4c0adba0 con 0x7f8a4c0c3d60 2024-03-06T07:22:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.004+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a4c04f330 con 0x7f8a4c0c3d60 2024-03-06T07:22:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.005+0000 7f8a427fc640 1 -- 172.21.15.26:0/2242610210 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17419 (secure 0 0 0) 0x7f8a3c020020 con 0x7f8a4c0c3d60 2024-03-06T07:22:19.008 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:19.008 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":62,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:18.685508+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.26:6833/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.117:6836/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.117:6834/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.26:6834/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6832/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6835/440323369":"2024-03-07T07:22:04.621666+0000","172.21.15.26:6833/440323369":"2024-03-07T07:22:04.621666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3006b500 msgr2=0x7f8a3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3006b500 0x7f8a3006d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400a990 tx=0x7f8a44008040 comp rx=0 tx=0).stop 2024-03-06T07:22:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 msgr2=0x7f8a4c1e8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c1e8f70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00b420 tx=0x7f8a3c00b8e0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 shutdown_connections 2024-03-06T07:22:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3006b500 0x7f8a3006d9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c156a00 0x7f8a4c1e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c0c3d60 0x7f8a4c1e8f70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 --2- 172.21.15.26:0/2242610210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c072720 0x7f8a4c1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 >> 172.21.15.26:0/2242610210 conn(0x7f8a4c0aa630 msgr2=0x7f8a4c0a9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 shutdown_connections 2024-03-06T07:22:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.007+0000 7f8a53a88640 1 -- 172.21.15.26:0/2242610210 wait complete. 2024-03-06T07:22:19.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:21:52.622857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:21:53.707058+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:19.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:19.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.130+0000 7fb12ae2a640 1 Processor -- start 2024-03-06T07:22:19.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.130+0000 7fb12ae2a640 1 -- start start 2024-03-06T07:22:19.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb12ae2a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1240a8290 0x7fb1241539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb12ae2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240c69a0 0x7fb124072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb12ae2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1240c62a0 con 0x7fb1240c4740 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb12ae2a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb124153f00 con 0x7fb1240a8290 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb12ae2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240c4c00 con 0x7fb1240c69a0 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb123fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb1240c4740 0x7fb1240701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55044/0 (socket says 172.21.15.26:55044) 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb123fff640 1 -- 172.21.15.26:0/3233304115 learned_addr learned my addr 172.21.15.26:0/3233304115 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb1293c7640 1 --2- 172.21.15.26:0/3233304115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240c69a0 0x7fb124072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb128bc6640 1 --2- 172.21.15.26:0/3233304115 >> v2:172.21.15.117:3300/0 conn(0x7fb1240a8290 0x7fb1241539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb1237fe640 1 -- 172.21.15.26:0/3233304115 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918475649 0 0) 0x7fb1240c62a0 con 0x7fb1240c4740 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.131+0000 7fb1237fe640 1 -- 172.21.15.26:0/3233304115 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb110003520 con 0x7fb1240c4740 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.132+0000 7fb1237fe640 1 -- 172.21.15.26:0/3233304115 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2906697376 0 0) 0x7fb110003520 con 0x7fb1240c4740 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.132+0000 7fb1293c7640 1 -- 172.21.15.26:0/3233304115 >> v2:172.21.15.117:3300/0 conn(0x7fb1240a8290 msgr2=0x7fb1241539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.132+0000 7fb1293c7640 1 --2- 172.21.15.26:0/3233304115 >> v2:172.21.15.117:3300/0 conn(0x7fb1240a8290 0x7fb1241539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.132+0000 7fb1293c7640 1 -- 172.21.15.26:0/3233304115 >> v1:172.21.15.26:6789/0 conn(0x7fb1240c4740 legacy=0x7fb1240701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.132+0000 7fb1293c7640 1 -- 172.21.15.26:0/3233304115 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240c4ee0 con 0x7fb1240c69a0 2024-03-06T07:22:19.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.132+0000 7fb128bc6640 1 --2- 172.21.15.26:0/3233304115 >> v2:172.21.15.117:3300/0 conn(0x7fb1240a8290 0x7fb1241539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb1293c7640 1 --2- 172.21.15.26:0/3233304115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240c69a0 0x7fb124072680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009d90 tx=0x7fb11c02bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44226f4b50e8d92a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb1237fe640 1 -- 172.21.15.26:0/3233304115 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c039070 con 0x7fb1240c69a0 2024-03-06T07:22:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb1237fe640 1 -- 172.21.15.26:0/3233304115 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c02c780 con 0x7fb1240c69a0 2024-03-06T07:22:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb1237fe640 1 -- 172.21.15.26:0/3233304115 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c001670 con 0x7fb1240c69a0 2024-03-06T07:22:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3233304115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240c69a0 msgr2=0x7fb124072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3233304115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240c69a0 0x7fb124072680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009d90 tx=0x7fb11c02bc50 comp rx=0 tx=0).stop 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3233304115 shutdown_connections 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3233304115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240c69a0 0x7fb124072680 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3233304115 >> v2:172.21.15.117:3300/0 conn(0x7fb1240a8290 0x7fb1241539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3233304115 >> 172.21.15.26:0/3233304115 conn(0x7fb1240c5560 msgr2=0x7fb1240c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3233304115 shutdown_connections 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.133+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3233304115 wait complete. 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.134+0000 7fb12ae2a640 1 Processor -- start 2024-03-06T07:22:19.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.134+0000 7fb12ae2a640 1 -- start start 2024-03-06T07:22:19.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb12ae2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240a8290 0x7fb12406d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb12ae2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1240c4740 0x7fb12414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb12ae2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240c69a0 0x7fb12414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb12ae2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240c6e30 con 0x7fb1240c69a0 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb12ae2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12406fc80 con 0x7fb1240c4740 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb12ae2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb124072bc0 con 0x7fb1240a8290 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb1293c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240c69a0 0x7fb12414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb1293c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240c69a0 0x7fb12414e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41422/0 (socket says 172.21.15.26:41422) 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb1293c7640 1 -- 172.21.15.26:0/3583595835 learned_addr learned my addr 172.21.15.26:0/3583595835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb128bc6640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240a8290 0x7fb12406d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.135+0000 7fb123fff640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1240c4740 0x7fb12414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1293c7640 1 -- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240a8290 msgr2=0x7fb12406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1293c7640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240a8290 0x7fb12406d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1293c7640 1 -- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1240c4740 msgr2=0x7fb12414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1293c7640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1240c4740 0x7fb12414dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1293c7640 1 -- 172.21.15.26:0/3583595835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124154680 con 0x7fb1240c69a0 2024-03-06T07:22:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1293c7640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240c69a0 0x7fb12414e220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009d90 tx=0x7fb11c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1217fa640 1 -- 172.21.15.26:0/3583595835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c039050 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1217fa640 1 -- 172.21.15.26:0/3583595835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c045070 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb124154920 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb1217fa640 1 -- 172.21.15.26:0/3583595835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c009450 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.136+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12406dbc0 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.137+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12404f330 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.137+0000 7fb1217fa640 1 -- 172.21.15.26:0/3583595835 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb11c04a460 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.138+0000 7fb1217fa640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb10406b4d0 0x7fb10406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.138+0000 7fb1217fa640 1 -- 172.21.15.26:0/3583595835 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fb11c0acbb0 con 0x7fb1240c69a0 2024-03-06T07:22:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.138+0000 7fb128bc6640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb10406b4d0 0x7fb10406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.138+0000 7fb128bc6640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb10406b4d0 0x7fb10406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb114009560 tx=0x7fb114008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.142+0000 7fb1217fa640 1 -- 172.21.15.26:0/3583595835 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb11c07b7b0 con 0x7fb1240c69a0 2024-03-06T07:22:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.321+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1240c30b0 con 0x7fb1240c69a0 2024-03-06T07:22:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.322+0000 7fb1217fa640 1 -- 172.21.15.26:0/3583595835 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb11c07b0b0 con 0x7fb1240c69a0 2024-03-06T07:22:19.324 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:22:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.323+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb10406b4d0 msgr2=0x7fb10406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.323+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb10406b4d0 0x7fb10406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb114009560 tx=0x7fb114008040 comp rx=0 tx=0).stop 2024-03-06T07:22:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.323+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240c69a0 msgr2=0x7fb12414e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.323+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240c69a0 0x7fb12414e220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009d90 tx=0x7fb11c004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 shutdown_connections 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb10406b4d0 0x7fb10406d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240c69a0 0x7fb12414e220 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1240c4740 0x7fb12414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 --2- 172.21.15.26:0/3583595835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1240a8290 0x7fb12406d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 >> 172.21.15.26:0/3583595835 conn(0x7fb1240c5560 msgr2=0x7fb1241dc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 shutdown_connections 2024-03-06T07:22:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.324+0000 7fb12ae2a640 1 -- 172.21.15.26:0/3583595835 wait complete. 2024-03-06T07:22:19.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:22:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.439+0000 7fb54ff93640 1 Processor -- start 2024-03-06T07:22:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.439+0000 7fb54ff93640 1 -- start start 2024-03-06T07:22:19.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.440+0000 7fb54ff93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 0x7fb548155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.440+0000 7fb54ff93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb548073290 0x7fb548155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.440+0000 7fb54ff93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb548152db0 con 0x7fb5481529d0 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.440+0000 7fb54ff93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480c6dd0 con 0x7fb548073290 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.440+0000 7fb54ff93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c6940 con 0x7fb548072cd0 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.440+0000 7fb54dd2f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb5481529d0 0x7fb5480c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55060/0 (socket says 172.21.15.26:55060) 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.440+0000 7fb54dd2f640 1 -- 172.21.15.26:0/1153658897 learned_addr learned my addr 172.21.15.26:0/1153658897 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689073403 0 0) 0x7fb544001110 con 0x7fb5481529d0 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb534003520 con 0x7fb5481529d0 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 0x7fb548155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54e530640 1 --2- 172.21.15.26:0/1153658897 >> v2:172.21.15.117:3300/0 conn(0x7fb548073290 0x7fb548155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668720073 0 0) 0x7fb534003520 con 0x7fb5481529d0 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 msgr2=0x7fb548155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 0x7fb548155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 >> v2:172.21.15.117:3300/0 conn(0x7fb548073290 msgr2=0x7fb548155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 --2- 172.21.15.26:0/1153658897 >> v2:172.21.15.117:3300/0 conn(0x7fb548073290 0x7fb548155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5481590f0 con 0x7fb5481529d0 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fb544001840 con 0x7fb5481529d0 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 >> v1:172.21.15.26:6789/0 conn(0x7fb5481529d0 legacy=0x7fb5480c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 0x7fb5340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb534004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb534004d40 0x7fb5340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c6dd0 con 0x7fb534003f30 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c6940 con 0x7fb534004d40 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53400baa0 con 0x7fb548072cd0 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb534004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54dd2f640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 0x7fb5340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54e530640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb534004d40 0x7fb5340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54d52e640 1 -- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 msgr2=0x7fb5340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb548072cd0 0x7fb5340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54d52e640 1 -- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb534004d40 msgr2=0x7fb5340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb534004d40 0x7fb5340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.441+0000 7fb54d52e640 1 -- 172.21.15.26:0/1153658897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534003520 con 0x7fb534003f30 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.442+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb534004800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb538009ac0 tx=0x7fb534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d963d4ceefa24b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.442+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538039070 con 0x7fb534003f30 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.442+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53802c6b0 con 0x7fb534003f30 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.442+0000 7fb54cd2d640 1 -- 172.21.15.26:0/1153658897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538009690 con 0x7fb534003f30 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.443+0000 7fb54ff93640 1 -- 172.21.15.26:0/1153658897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 msgr2=0x7fb534004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.443+0000 7fb54ff93640 1 --2- 172.21.15.26:0/1153658897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb534004800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb538009ac0 tx=0x7fb534005830 comp rx=0 tx=0).stop 2024-03-06T07:22:19.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.443+0000 7fb54e530640 1 -- 172.21.15.26:0/1153658897 reap_dead start 2024-03-06T07:22:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.443+0000 7fb54ff93640 1 -- 172.21.15.26:0/1153658897 shutdown_connections 2024-03-06T07:22:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.443+0000 7fb54ff93640 1 -- 172.21.15.26:0/1153658897 >> 172.21.15.26:0/1153658897 conn(0x7fb5480aa630 msgr2=0x7fb5480aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.443+0000 7fb54ff93640 1 -- 172.21.15.26:0/1153658897 shutdown_connections 2024-03-06T07:22:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.443+0000 7fb54ff93640 1 -- 172.21.15.26:0/1153658897 wait complete. 2024-03-06T07:22:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.444+0000 7fb54ff93640 1 Processor -- start 2024-03-06T07:22:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.444+0000 7fb54ff93640 1 -- start start 2024-03-06T07:22:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb54814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5480ab930 0x7fb54814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481ee660 0x7fb5481eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548159370 con 0x7fb534003f30 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548072720 con 0x7fb5481ee660 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480723b0 con 0x7fb5480ab930 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb54814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb54814deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41440/0 (socket says 172.21.15.26:41440) 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 -- 172.21.15.26:0/1763663694 learned_addr learned my addr 172.21.15.26:0/1763663694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54e530640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481ee660 0x7fb5481eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5480ab930 0x7fb54814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 -- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5480ab930 msgr2=0x7fb54814e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5480ab930 0x7fb54814e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 -- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481ee660 msgr2=0x7fb5481eeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481ee660 0x7fb5481eeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 -- 172.21.15.26:0/1763663694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480abdc0 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54e530640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481ee660 0x7fb5481eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5480ab930 0x7fb54814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54dd2f640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb54814deb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb544007340 tx=0x7fb544007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb53e7fc640 1 -- 172.21.15.26:0/1763663694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54400f9c0 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb53e7fc640 1 -- 172.21.15.26:0/1763663694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544018070 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5480a86e0 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb53e7fc640 1 -- 172.21.15.26:0/1763663694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54400fd80 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.445+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480a8ca0 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.446+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54804f330 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.447+0000 7fb53e7fc640 1 -- 172.21.15.26:0/1763663694 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb54401d460 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.447+0000 7fb53e7fc640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb52c06b4d0 0x7fb52c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.447+0000 7fb53e7fc640 1 -- 172.21.15.26:0/1763663694 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fb54408a610 con 0x7fb534003f30 2024-03-06T07:22:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.447+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb52c06b4d0 0x7fb52c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.448+0000 7fb54d52e640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb52c06b4d0 0x7fb52c06d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb538036000 tx=0x7fb538030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.452+0000 7fb53e7fc640 1 -- 172.21.15.26:0/1763663694 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb5481590f0 con 0x7fb534003f30 2024-03-06T07:22:19.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.631+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5480ab2a0 con 0x7fb534003f30 2024-03-06T07:22:19.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.632+0000 7fb53e7fc640 1 -- 172.21.15.26:0/1763663694 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v62) v1 ==== 105+0+11 (secure 0 0 0) 0x7fb548152db0 con 0x7fb534003f30 2024-03-06T07:22:19.634 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:22:19.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.633+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb52c06b4d0 msgr2=0x7fb52c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.633+0000 7fb54ff93640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb52c06b4d0 0x7fb52c06d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb538036000 tx=0x7fb538030000 comp rx=0 tx=0).stop 2024-03-06T07:22:19.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.633+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 msgr2=0x7fb54814deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.633+0000 7fb54ff93640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb54814deb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb544007340 tx=0x7fb544007800 comp rx=0 tx=0).stop 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.633+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 shutdown_connections 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.634+0000 7fb54ff93640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb52c06b4d0 0x7fb52c06d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.634+0000 7fb54ff93640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481ee660 0x7fb5481eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.634+0000 7fb54ff93640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5480ab930 0x7fb54814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.634+0000 7fb54ff93640 1 --2- 172.21.15.26:0/1763663694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb534003f30 0x7fb54814deb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.634+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 >> 172.21.15.26:0/1763663694 conn(0x7fb5480aa630 msgr2=0x7fb54814ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.634+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 shutdown_connections 2024-03-06T07:22:19.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.634+0000 7fb54ff93640 1 -- 172.21.15.26:0/1763663694 wait complete. 2024-03-06T07:22:19.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:22:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.749+0000 7f6ffa439640 1 Processor -- start 2024-03-06T07:22:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ffa439640 1 -- start start 2024-03-06T07:22:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ffa439640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40b0070 0x7f6ff4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ffa439640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ff40c5380 0x7f6ff40c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ffa439640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40b0450 con 0x7f6ff406ca70 2024-03-06T07:22:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ffa439640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff406c020 con 0x7f6ff40c5380 2024-03-06T07:22:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ffa439640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40c77f0 con 0x7f6ff40b0070 2024-03-06T07:22:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ff89d6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ff40c5380 0x7f6ff40c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40b0070 0x7f6ff4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ff89d6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ff40c5380 0x7f6ff40c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56490/0 (socket says 172.21.15.26:56490) 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ff37fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6ff406ca70 0x7f6ff40c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:49338/0 (socket says 172.21.15.26:49338) 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.750+0000 7f6ff89d6640 1 -- 172.21.15.26:0/1787163440 learned_addr learned my addr 172.21.15.26:0/1787163440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.751+0000 7f6ff89d6640 1 -- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40b0070 msgr2=0x7f6ff4155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.751+0000 7f6ff89d6640 1 --2- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40b0070 0x7f6ff4155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.751+0000 7f6ff89d6640 1 -- 172.21.15.26:0/1787163440 >> v1:172.21.15.117:6789/0 conn(0x7f6ff406ca70 legacy=0x7f6ff40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.751+0000 7f6ff89d6640 1 -- 172.21.15.26:0/1787163440 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40c7ad0 con 0x7f6ff40c5380 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.751+0000 7f6ff3fff640 1 --2- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40b0070 0x7f6ff4155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ff89d6640 1 --2- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40c5380 0x7f6ff40c57d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009e50 tx=0x7f6fec02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674d3fd9e402b539 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ff2ffd640 1 -- 172.21.15.26:0/1787163440 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec02ca70 con 0x7f6ff40c5380 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ff2ffd640 1 -- 172.21.15.26:0/1787163440 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec02cc10 con 0x7f6ff40c5380 2024-03-06T07:22:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ff2ffd640 1 -- 172.21.15.26:0/1787163440 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0343f0 con 0x7f6ff40c5380 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 -- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40c5380 msgr2=0x7f6ff40c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 --2- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40c5380 0x7f6ff40c57d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009e50 tx=0x7f6fec02bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 -- 172.21.15.26:0/1787163440 shutdown_connections 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 --2- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40c5380 0x7f6ff40c57d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 --2- 172.21.15.26:0/1787163440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40b0070 0x7f6ff4155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 -- 172.21.15.26:0/1787163440 >> 172.21.15.26:0/1787163440 conn(0x7f6ff40ac590 msgr2=0x7f6ff40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 -- 172.21.15.26:0/1787163440 shutdown_connections 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.752+0000 7f6ffa439640 1 -- 172.21.15.26:0/1787163440 wait complete. 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.753+0000 7f6ffa439640 1 Processor -- start 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ffa439640 1 -- start start 2024-03-06T07:22:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ffa439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff406ca70 0x7f6ff414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ffa439640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40b0070 0x7f6ff40c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ffa439640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40c5380 0x7f6ff40c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ffa439640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff406cef0 con 0x7f6ff406ca70 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ffa439640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff406c550 con 0x7f6ff40b0070 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ffa439640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff406da80 con 0x7f6ff40c5380 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff406ca70 0x7f6ff414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff406ca70 0x7f6ff414a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41442/0 (socket says 172.21.15.26:41442) 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ff3fff640 1 -- 172.21.15.26:0/2878338295 learned_addr learned my addr 172.21.15.26:0/2878338295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ff89d6640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40c5380 0x7f6ff40c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.754+0000 7f6ff37fe640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40b0070 0x7f6ff40c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff3fff640 1 -- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40c5380 msgr2=0x7f6ff40c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff3fff640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40c5380 0x7f6ff40c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff3fff640 1 -- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40b0070 msgr2=0x7f6ff40c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff3fff640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40b0070 0x7f6ff40c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff3fff640 1 -- 172.21.15.26:0/2878338295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff41ea870 con 0x7f6ff406ca70 2024-03-06T07:22:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff3fff640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff406ca70 0x7f6ff414a0e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400ebe0 tx=0x7f6fe400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff0ff9640 1 -- 172.21.15.26:0/2878338295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4017070 con 0x7f6ff406ca70 2024-03-06T07:22:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff0ff9640 1 -- 172.21.15.26:0/2878338295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe400cc30 con 0x7f6ff406ca70 2024-03-06T07:22:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff41eab50 con 0x7f6ff406ca70 2024-03-06T07:22:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ff0ff9640 1 -- 172.21.15.26:0/2878338295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4013470 con 0x7f6ff406ca70 2024-03-06T07:22:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.755+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff41eb110 con 0x7f6ff406ca70 2024-03-06T07:22:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.756+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff404f330 con 0x7f6ff406ca70 2024-03-06T07:22:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.757+0000 7f6ff0ff9640 1 -- 172.21.15.26:0/2878338295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fe401a070 con 0x7f6ff406ca70 2024-03-06T07:22:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.757+0000 7f6ff0ff9640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fe006b4d0 0x7f6fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.757+0000 7f6ff0ff9640 1 -- 172.21.15.26:0/2878338295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f6fe408a7b0 con 0x7f6ff406ca70 2024-03-06T07:22:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.757+0000 7f6ff37fe640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fe006b4d0 0x7f6fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.758+0000 7f6ff37fe640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fe006b4d0 0x7f6fe006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40be410 tx=0x7f6fe8007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.761+0000 7f6ff0ff9640 1 -- 172.21.15.26:0/2878338295 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6fe4058c70 con 0x7f6ff406ca70 2024-03-06T07:22:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.941+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f6ff406d860 con 0x7f6ff406ca70 2024-03-06T07:22:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.941+0000 7f6ff0ff9640 1 -- 172.21.15.26:0/2878338295 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v62) v1 ==== 101+0+11 (secure 0 0 0) 0x7f6fe405cd10 con 0x7f6ff406ca70 2024-03-06T07:22:19.944 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fe006b4d0 msgr2=0x7f6fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fe006b4d0 0x7f6fe006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40be410 tx=0x7f6fe8007e80 comp rx=0 tx=0).stop 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff406ca70 msgr2=0x7f6ff414a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff406ca70 0x7f6ff414a0e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400ebe0 tx=0x7f6fe400c520 comp rx=0 tx=0).stop 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 shutdown_connections 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fe006b4d0 0x7f6fe006d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff40c5380 0x7f6ff40c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff40b0070 0x7f6ff40c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 --2- 172.21.15.26:0/2878338295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff406ca70 0x7f6ff414a0e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 >> 172.21.15.26:0/2878338295 conn(0x7f6ff40ac590 msgr2=0x7f6ff40c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 shutdown_connections 2024-03-06T07:22:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:19.943+0000 7f6ffa439640 1 -- 172.21.15.26:0/2878338295 wait complete. 2024-03-06T07:22:19.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.055+0000 7f033629d640 1 Processor -- start 2024-03-06T07:22:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.055+0000 7f033629d640 1 -- start start 2024-03-06T07:22:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033629d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300c8a00 0x7f0330158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033629d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0330159460 0x7f03300c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033629d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03300c7760 con 0x7f0330152920 2024-03-06T07:22:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033629d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03301587b0 con 0x7f0330159460 2024-03-06T07:22:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033629d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03300c7a40 con 0x7f03300c8a00 2024-03-06T07:22:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033483a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0330159460 0x7f03300c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300c8a00 0x7f0330158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f032ffff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0330152920 0x7f0330155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:49354/0 (socket says 172.21.15.26:49354) 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f032ffff640 1 -- 172.21.15.26:0/143487048 learned_addr learned my addr 172.21.15.26:0/143487048 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033483a640 1 --2- 172.21.15.26:0/143487048 >> v2:172.21.15.26:3300/0 conn(0x7f0330159460 0x7f03300c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41444/0 (socket says 172.21.15.26:41444) 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033483a640 1 -- 172.21.15.26:0/143487048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300c8a00 msgr2=0x7f0330158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033483a640 1 --2- 172.21.15.26:0/143487048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300c8a00 0x7f0330158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f032effd640 1 -- 172.21.15.26:0/143487048 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139653562 0 0) 0x7f03300c7760 con 0x7f0330152920 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033483a640 1 -- 172.21.15.26:0/143487048 >> v1:172.21.15.117:6789/0 conn(0x7f0330152920 legacy=0x7f0330155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.056+0000 7f033483a640 1 -- 172.21.15.26:0/143487048 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300c7d20 con 0x7f0330159460 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.057+0000 7f033483a640 1 --2- 172.21.15.26:0/143487048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0330159460 0x7f03300c5330 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0328009d90 tx=0x7f032802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a065ef40875b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.057+0000 7f032effd640 1 -- 172.21.15.26:0/143487048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328039070 con 0x7f0330159460 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f032effd640 1 -- 172.21.15.26:0/143487048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032802cbc0 con 0x7f0330159460 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f032effd640 1 -- 172.21.15.26:0/143487048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328034440 con 0x7f0330159460 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 -- 172.21.15.26:0/143487048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0330159460 msgr2=0x7f03300c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 --2- 172.21.15.26:0/143487048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0330159460 0x7f03300c5330 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0328009d90 tx=0x7f032802be10 comp rx=0 tx=0).stop 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 -- 172.21.15.26:0/143487048 shutdown_connections 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 --2- 172.21.15.26:0/143487048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0330159460 0x7f03300c5330 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 --2- 172.21.15.26:0/143487048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300c8a00 0x7f0330158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 -- 172.21.15.26:0/143487048 >> 172.21.15.26:0/143487048 conn(0x7f03300ac560 msgr2=0x7f03300ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 -- 172.21.15.26:0/143487048 shutdown_connections 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.058+0000 7f033629d640 1 -- 172.21.15.26:0/143487048 wait complete. 2024-03-06T07:22:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.059+0000 7f033629d640 1 Processor -- start 2024-03-06T07:22:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.059+0000 7f033629d640 1 -- start start 2024-03-06T07:22:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f033629d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03300c8a00 0x7f033014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f033629d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0330152920 0x7f033014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f033629d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0330147d00 0x7f03300aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f033629d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03300c7390 con 0x7f03300c8a00 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f033629d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033006d030 con 0x7f0330152920 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f033629d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03300c7510 con 0x7f0330147d00 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f032ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03300c8a00 0x7f033014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f032ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03300c8a00 0x7f033014ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41460/0 (socket says 172.21.15.26:41460) 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f032ffff640 1 -- 172.21.15.26:0/2913765071 learned_addr learned my addr 172.21.15.26:0/2913765071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f033483a640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0330147d00 0x7f03300aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.060+0000 7f032f7fe640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0330152920 0x7f033014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032ffff640 1 -- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0330147d00 msgr2=0x7f03300aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032ffff640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0330147d00 0x7f03300aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032ffff640 1 -- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0330152920 msgr2=0x7f033014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032ffff640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0330152920 0x7f033014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032ffff640 1 -- 172.21.15.26:0/2913765071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330147ab0 con 0x7f03300c8a00 2024-03-06T07:22:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032ffff640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03300c8a00 0x7f033014ce30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f032000c4d0 tx=0x7f032000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032cff9640 1 -- 172.21.15.26:0/2913765071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320018070 con 0x7f03300c8a00 2024-03-06T07:22:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032cff9640 1 -- 172.21.15.26:0/2913765071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320007500 con 0x7f03300c8a00 2024-03-06T07:22:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03301477d0 con 0x7f03300c8a00 2024-03-06T07:22:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f032cff9640 1 -- 172.21.15.26:0/2913765071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320004a20 con 0x7f03300c8a00 2024-03-06T07:22:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.061+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0330149810 con 0x7f03300c8a00 2024-03-06T07:22:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.062+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0330149a10 con 0x7f03300c8a00 2024-03-06T07:22:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.062+0000 7f032cff9640 1 -- 172.21.15.26:0/2913765071 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f032001b070 con 0x7f03300c8a00 2024-03-06T07:22:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.063+0000 7f032cff9640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.063+0000 7f032cff9640 1 -- 172.21.15.26:0/2913765071 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f03200896f0 con 0x7f03300c8a00 2024-03-06T07:22:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.063+0000 7f032f7fe640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.063+0000 7f032f7fe640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f033014c810 tx=0x7f0318009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.067+0000 7f032cff9640 1 -- 172.21.15.26:0/2913765071 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0320057cc0 con 0x7f03300c8a00 2024-03-06T07:22:20.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.285+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f033004f330 con 0x7f03300c8a00 2024-03-06T07:22:20.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.286+0000 7f032cff9640 1 -- 172.21.15.26:0/2913765071 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f032005bd60 con 0x7f03300c8a00 2024-03-06T07:22:20.289 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:20.289 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1256717910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1256717910},{"type":"v1","addr":"172.21.15.117:6836","nonce":1256717910}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4276915425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4276915425},{"type":"v1","addr":"172.21.15.26:6835","nonce":4276915425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:22:19.506857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5240":{"gid":5240,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5240,"qdb_cluster":[5240]},"id":4}]} 2024-03-06T07:22:20.289 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2024-03-06T07:22:20.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f031006b4d0 msgr2=0x7f031006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f033014c810 tx=0x7f0318009190 comp rx=0 tx=0).stop 2024-03-06T07:22:20.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03300c8a00 msgr2=0x7f033014ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03300c8a00 0x7f033014ce30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f032000c4d0 tx=0x7f032000c990 comp rx=0 tx=0).stop 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 shutdown_connections 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0330147d00 0x7f03300aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0330152920 0x7f033014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 --2- 172.21.15.26:0/2913765071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03300c8a00 0x7f033014ce30 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 >> 172.21.15.26:0/2913765071 conn(0x7f03300ac560 msgr2=0x7f033014b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 shutdown_connections 2024-03-06T07:22:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.288+0000 7f033629d640 1 -- 172.21.15.26:0/2913765071 wait complete. 2024-03-06T07:22:20.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.400+0000 7fdf5d476640 1 Processor -- start 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.400+0000 7fdf5d476640 1 -- start start 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.401+0000 7fdf5d476640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf580c5480 0x7fdf58155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.401+0000 7fdf5d476640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf580c5cc0 con 0x7fdf580c5860 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.401+0000 7fdf5d476640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf581591c0 con 0x7fdf581586f0 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.401+0000 7fdf5d476640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5806df40 con 0x7fdf580c5480 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.401+0000 7fdf567fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf580c5860 0x7fdf5806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55070/0 (socket says 172.21.15.26:55070) 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.401+0000 7fdf567fc640 1 -- 172.21.15.26:0/3509547140 learned_addr learned my addr 172.21.15.26:0/3509547140 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.401+0000 7fdf56ffd640 1 --2- 172.21.15.26:0/3509547140 >> v2:172.21.15.117:3300/0 conn(0x7fdf580c5480 0x7fdf58155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf56ffd640 1 -- 172.21.15.26:0/3509547140 >> v1:172.21.15.117:6789/0 conn(0x7fdf581586f0 legacy=0x7fdf5815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf55ffb640 1 -- 172.21.15.26:0/3509547140 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585226153 0 0) 0x7fdf580c5cc0 con 0x7fdf580c5860 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf56ffd640 1 -- 172.21.15.26:0/3509547140 >> v1:172.21.15.26:6789/0 conn(0x7fdf580c5860 legacy=0x7fdf5806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf56ffd640 1 -- 172.21.15.26:0/3509547140 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf581594a0 con 0x7fdf580c5480 2024-03-06T07:22:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf56ffd640 1 --2- 172.21.15.26:0/3509547140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5480 0x7fdf58155ba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009d90 tx=0x7fdf5002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390d6b4fdbcbb2f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf55ffb640 1 -- 172.21.15.26:0/3509547140 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50039070 con 0x7fdf580c5480 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf55ffb640 1 -- 172.21.15.26:0/3509547140 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5002cc60 con 0x7fdf580c5480 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.402+0000 7fdf55ffb640 1 -- 172.21.15.26:0/3509547140 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50034660 con 0x7fdf580c5480 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.403+0000 7fdf5d476640 1 -- 172.21.15.26:0/3509547140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5480 msgr2=0x7fdf58155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.403+0000 7fdf5d476640 1 --2- 172.21.15.26:0/3509547140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5480 0x7fdf58155ba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009d90 tx=0x7fdf5002beb0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.403+0000 7fdf5d476640 1 -- 172.21.15.26:0/3509547140 shutdown_connections 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.403+0000 7fdf5d476640 1 --2- 172.21.15.26:0/3509547140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5480 0x7fdf58155ba0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.403+0000 7fdf5d476640 1 -- 172.21.15.26:0/3509547140 >> 172.21.15.26:0/3509547140 conn(0x7fdf5806b1a0 msgr2=0x7fdf5806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.403+0000 7fdf5d476640 1 -- 172.21.15.26:0/3509547140 shutdown_connections 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.403+0000 7fdf5d476640 1 -- 172.21.15.26:0/3509547140 wait complete. 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.404+0000 7fdf5d476640 1 Processor -- start 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.404+0000 7fdf5d476640 1 -- start start 2024-03-06T07:22:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf5d476640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf580c5480 0x7fdf580c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf5d476640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5860 0x7fdf581534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf5d476640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf581586f0 0x7fdf58153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf5d476640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58155680 con 0x7fdf580c5480 2024-03-06T07:22:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf5d476640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5806fdb0 con 0x7fdf580c5860 2024-03-06T07:22:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf5d476640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58152a20 con 0x7fdf581586f0 2024-03-06T07:22:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf580c5480 0x7fdf580c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf580c5480 0x7fdf580c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41470/0 (socket says 172.21.15.26:41470) 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 -- 172.21.15.26:0/1980633118 learned_addr learned my addr 172.21.15.26:0/1980633118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf567fc640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5860 0x7fdf581534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf577fe640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf581586f0 0x7fdf58153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 -- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf581586f0 msgr2=0x7fdf58153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf581586f0 0x7fdf58153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 -- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5860 msgr2=0x7fdf581534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5860 0x7fdf581534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 -- 172.21.15.26:0/1980633118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580ad750 con 0x7fdf580c5480 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf567fc640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5860 0x7fdf581534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.405+0000 7fdf56ffd640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf580c5480 0x7fdf580c8990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdf50037040 tx=0x7fdf50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.406+0000 7fdf4f7fe640 1 -- 172.21.15.26:0/1980633118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50039050 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.406+0000 7fdf4f7fe640 1 -- 172.21.15.26:0/1980633118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5003a040 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.406+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581eb9f0 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.406+0000 7fdf4f7fe640 1 -- 172.21.15.26:0/1980633118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50003bd0 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.406+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf581eb710 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.407+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5804f330 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.407+0000 7fdf4f7fe640 1 -- 172.21.15.26:0/1980633118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf50047030 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.408+0000 7fdf4f7fe640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.408+0000 7fdf4f7fe640 1 -- 172.21.15.26:0/1980633118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fdf500ac920 con 0x7fdf580c5480 2024-03-06T07:22:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.408+0000 7fdf567fc640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.408+0000 7fdf567fc640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf3806b4d0 0x7fdf3806d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf40007f70 tx=0x7fdf40007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.412+0000 7fdf4f7fe640 1 -- 172.21.15.26:0/1980633118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf5004e020 con 0x7fdf580c5480 2024-03-06T07:22:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.595+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf5806e0c0 con 0x7fdf580c5480 2024-03-06T07:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.596+0000 7fdf4f7fe640 1 -- 172.21.15.26:0/1980633118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17419 (secure 0 0 0) 0x7fdf5007aef0 con 0x7fdf580c5480 2024-03-06T07:22:20.599 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:20.599 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":62,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:18.685508+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.26:6833/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.117:6836/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.117:6834/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.26:6834/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6832/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6835/440323369":"2024-03-07T07:22:04.621666+0000","172.21.15.26:6833/440323369":"2024-03-07T07:22:04.621666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf3806b4d0 msgr2=0x7fdf3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf3806b4d0 0x7fdf3806d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf40007f70 tx=0x7fdf40007f00 comp rx=0 tx=0).stop 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf580c5480 msgr2=0x7fdf580c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf580c5480 0x7fdf580c8990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdf50037040 tx=0x7fdf50004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 shutdown_connections 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf581586f0 0x7fdf58153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf580c5860 0x7fdf581534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 --2- 172.21.15.26:0/1980633118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf580c5480 0x7fdf580c8990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 >> 172.21.15.26:0/1980633118 conn(0x7fdf5806b1a0 msgr2=0x7fdf5806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 shutdown_connections 2024-03-06T07:22:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.598+0000 7fdf5d476640 1 -- 172.21.15.26:0/1980633118 wait complete. 2024-03-06T07:22:20.612 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-06T07:22:20.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.714+0000 7f2056393640 1 Processor -- start 2024-03-06T07:22:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.714+0000 7f2056393640 1 -- start start 2024-03-06T07:22:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.714+0000 7f2056393640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2050155ba0 0x7f20500c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.714+0000 7f2056393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20501586f0 0x7f2050158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.714+0000 7f2056393640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20501591c0 con 0x7f20500c5480 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.714+0000 7f2056393640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205006df40 con 0x7f2050155ba0 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2056393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205006fdb0 con 0x7f20501586f0 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2054930640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20501586f0 0x7f2050158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f204f7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2050155ba0 0x7f20500c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2054930640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20501586f0 0x7f2050158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:47694/0 (socket says 172.21.15.26:47694) 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2054930640 1 -- 172.21.15.26:0/1092581736 learned_addr learned my addr 172.21.15.26:0/1092581736 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2054930640 1 -- 172.21.15.26:0/1092581736 >> v2:172.21.15.117:3300/0 conn(0x7f2050155ba0 msgr2=0x7f20500c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2054930640 1 --2- 172.21.15.26:0/1092581736 >> v2:172.21.15.117:3300/0 conn(0x7f2050155ba0 0x7f20500c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2054930640 1 -- 172.21.15.26:0/1092581736 >> v1:172.21.15.117:6789/0 conn(0x7f20500c5480 legacy=0x7f205006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.715+0000 7f2054930640 1 -- 172.21.15.26:0/1092581736 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20501594a0 con 0x7f20501586f0 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2054930640 1 --2- 172.21.15.26:0/1092581736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20501586f0 0x7f2050158b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2048009e50 tx=0x7f204802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b99d9d01503794 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f204effd640 1 -- 172.21.15.26:0/1092581736 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204802ca70 con 0x7f20501586f0 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f204effd640 1 -- 172.21.15.26:0/1092581736 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204802cc10 con 0x7f20501586f0 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f204effd640 1 -- 172.21.15.26:0/1092581736 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20480343f0 con 0x7f20501586f0 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2056393640 1 -- 172.21.15.26:0/1092581736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20501586f0 msgr2=0x7f2050158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2056393640 1 --2- 172.21.15.26:0/1092581736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20501586f0 0x7f2050158b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2048009e50 tx=0x7f204802bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2056393640 1 -- 172.21.15.26:0/1092581736 shutdown_connections 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2056393640 1 --2- 172.21.15.26:0/1092581736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20501586f0 0x7f2050158b80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2056393640 1 --2- 172.21.15.26:0/1092581736 >> v2:172.21.15.117:3300/0 conn(0x7f2050155ba0 0x7f20500c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2056393640 1 -- 172.21.15.26:0/1092581736 >> 172.21.15.26:0/1092581736 conn(0x7f205006b1a0 msgr2=0x7f205006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.716+0000 7f2056393640 1 -- 172.21.15.26:0/1092581736 shutdown_connections 2024-03-06T07:22:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.717+0000 7f2056393640 1 -- 172.21.15.26:0/1092581736 wait complete. 2024-03-06T07:22:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 Processor -- start 2024-03-06T07:22:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 -- start start 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20500c5480 0x7f20501de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2050155ba0 0x7f205014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20501586f0 0x7f20501eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050155680 con 0x7f20500c5480 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050156000 con 0x7f20501586f0 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f2056393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2050152a20 con 0x7f2050155ba0 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.718+0000 7f204ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20500c5480 0x7f20501de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20500c5480 0x7f20501de6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41486/0 (socket says 172.21.15.26:41486) 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 -- 172.21.15.26:0/308116853 learned_addr learned my addr 172.21.15.26:0/308116853 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f2054930640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20501586f0 0x7f20501eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204f7fe640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2050155ba0 0x7f205014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 -- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2050155ba0 msgr2=0x7f205014f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2050155ba0 0x7f205014f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 -- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20501586f0 msgr2=0x7f20501eb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20501586f0 0x7f20501eb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 -- 172.21.15.26:0/308116853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050151cc0 con 0x7f20500c5480 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204ffff640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20500c5480 0x7f20501de6b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f204000c3f0 tx=0x7f204000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204cff9640 1 -- 172.21.15.26:0/308116853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400052d0 con 0x7f20500c5480 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204cff9640 1 -- 172.21.15.26:0/308116853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040005490 con 0x7f20500c5480 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20501eaec0 con 0x7f20500c5480 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f204cff9640 1 -- 172.21.15.26:0/308116853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040007260 con 0x7f20500c5480 2024-03-06T07:22:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.719+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2050157800 con 0x7f20500c5480 2024-03-06T07:22:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.720+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205004f330 con 0x7f20500c5480 2024-03-06T07:22:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.721+0000 7f204cff9640 1 -- 172.21.15.26:0/308116853 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2040007400 con 0x7f20500c5480 2024-03-06T07:22:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.721+0000 7f204cff9640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f203c06b4d0 0x7f203c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.721+0000 7f204cff9640 1 -- 172.21.15.26:0/308116853 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f2040089910 con 0x7f20500c5480 2024-03-06T07:22:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.721+0000 7f204f7fe640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f203c06b4d0 0x7f203c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.721+0000 7f204f7fe640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f203c06b4d0 0x7f203c06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20501532f0 tx=0x7f2044009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.725+0000 7f204cff9640 1 -- 172.21.15.26:0/308116853 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2050151cc0 con 0x7f20500c5480 2024-03-06T07:22:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.949+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f205006ba70 con 0x7f20500c5480 2024-03-06T07:22:20.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.949+0000 7f204cff9640 1 -- 172.21.15.26:0/308116853 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f204005ce20 con 0x7f20500c5480 2024-03-06T07:22:20.952 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:20.953 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1256717910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1256717910},{"type":"v1","addr":"172.21.15.117:6836","nonce":1256717910}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4276915425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4276915425},{"type":"v1","addr":"172.21.15.26:6835","nonce":4276915425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:22:19.506857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5240":{"gid":5240,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5240,"qdb_cluster":[5240]},"id":4}]} 2024-03-06T07:22:20.953 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2024-03-06T07:22:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f203c06b4d0 msgr2=0x7f203c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f203c06b4d0 0x7f203c06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20501532f0 tx=0x7f2044009190 comp rx=0 tx=0).stop 2024-03-06T07:22:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20500c5480 msgr2=0x7f20501de6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20500c5480 0x7f20501de6b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f204000c3f0 tx=0x7f204000c8b0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 shutdown_connections 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f203c06b4d0 0x7f203c06d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20501586f0 0x7f20501eb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2050155ba0 0x7f205014f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 --2- 172.21.15.26:0/308116853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20500c5480 0x7f20501de6b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 >> 172.21.15.26:0/308116853 conn(0x7f205006b1a0 msgr2=0x7f20500c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 shutdown_connections 2024-03-06T07:22:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:20.952+0000 7f2056393640 1 -- 172.21.15.26:0/308116853 wait complete. 2024-03-06T07:22:20.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.063+0000 7f7981372640 1 Processor -- start 2024-03-06T07:22:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.063+0000 7f7981372640 1 -- start start 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f7981372640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c152920 0x7f797c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f7981372640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c06afc0 con 0x7f797c0c25f0 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f7981372640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c06b2a0 con 0x7f797c0c2190 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f7981372640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c156160 con 0x7f797c152920 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f797b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f797c0c25f0 0x7f797c15fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55082/0 (socket says 172.21.15.26:55082) 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f797b7fe640 1 -- 172.21.15.26:0/1469240702 learned_addr learned my addr 172.21.15.26:0/1469240702 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f797affd640 1 --2- 172.21.15.26:0/1469240702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c152920 0x7f797c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f7979ffb640 1 -- 172.21.15.26:0/1469240702 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422897225 0 0) 0x7f797c06afc0 con 0x7f797c0c25f0 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f7979ffb640 1 -- 172.21.15.26:0/1469240702 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7964003520 con 0x7f797c0c25f0 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.064+0000 7f7979ffb640 1 -- 172.21.15.26:0/1469240702 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325565823 0 0) 0x7f797c06b2a0 con 0x7f797c0c2190 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.065+0000 7f797affd640 1 -- 172.21.15.26:0/1469240702 >> v1:172.21.15.117:6789/0 conn(0x7f797c0c2190 legacy=0x7f797c0c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.065+0000 7f797affd640 1 -- 172.21.15.26:0/1469240702 >> v1:172.21.15.26:6789/0 conn(0x7f797c0c25f0 legacy=0x7f797c15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.065+0000 7f797affd640 1 -- 172.21.15.26:0/1469240702 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c06b580 con 0x7f797c152920 2024-03-06T07:22:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.065+0000 7f797affd640 1 --2- 172.21.15.26:0/1469240702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c152920 0x7f797c155c20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f796c0097d0 tx=0x7f796c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d257291967caa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.065+0000 7f7979ffb640 1 -- 172.21.15.26:0/1469240702 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c035020 con 0x7f797c152920 2024-03-06T07:22:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.065+0000 7f7979ffb640 1 -- 172.21.15.26:0/1469240702 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c031890 con 0x7f797c152920 2024-03-06T07:22:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.065+0000 7f7979ffb640 1 -- 172.21.15.26:0/1469240702 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c031c40 con 0x7f797c152920 2024-03-06T07:22:21.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.066+0000 7f7981372640 1 -- 172.21.15.26:0/1469240702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c152920 msgr2=0x7f797c155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.066+0000 7f7981372640 1 --2- 172.21.15.26:0/1469240702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c152920 0x7f797c155c20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f796c0097d0 tx=0x7f796c02b930 comp rx=0 tx=0).stop 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.066+0000 7f7981372640 1 -- 172.21.15.26:0/1469240702 shutdown_connections 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.066+0000 7f7981372640 1 --2- 172.21.15.26:0/1469240702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c152920 0x7f797c155c20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.066+0000 7f7981372640 1 -- 172.21.15.26:0/1469240702 >> 172.21.15.26:0/1469240702 conn(0x7f797c0c4290 msgr2=0x7f797c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.066+0000 7f7981372640 1 -- 172.21.15.26:0/1469240702 shutdown_connections 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.066+0000 7f7981372640 1 -- 172.21.15.26:0/1469240702 wait complete. 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.067+0000 7f7981372640 1 Processor -- start 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.067+0000 7f7981372640 1 -- start start 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f7981372640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c0c2190 0x7f797c0725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f7981372640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c25f0 0x7f797c147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f7981372640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c152920 0x7f797c1eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f7981372640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0a7100 con 0x7f797c152920 2024-03-06T07:22:21.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f7981372640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0c0d60 con 0x7f797c0c25f0 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f7981372640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0a7280 con 0x7f797c0c2190 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c152920 0x7f797c1eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c152920 0x7f797c1eabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41502/0 (socket says 172.21.15.26:41502) 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 -- 172.21.15.26:0/1085673582 learned_addr learned my addr 172.21.15.26:0/1085673582 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797affd640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c0c2190 0x7f797c0725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797a7fc640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c25f0 0x7f797c147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 -- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c0c2190 msgr2=0x7f797c0725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c0c2190 0x7f797c0725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 -- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c25f0 msgr2=0x7f797c147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c25f0 0x7f797c147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.068+0000 7f797b7fe640 1 -- 172.21.15.26:0/1085673582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c06f3e0 con 0x7f797c152920 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.069+0000 7f797b7fe640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c152920 0x7f797c1eabd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f797400cb40 tx=0x7f797400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.069+0000 7f79637fe640 1 -- 172.21.15.26:0/1085673582 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974017070 con 0x7f797c152920 2024-03-06T07:22:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.069+0000 7f79637fe640 1 -- 172.21.15.26:0/1085673582 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400aca0 con 0x7f797c152920 2024-03-06T07:22:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.069+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c0aa7b0 con 0x7f797c152920 2024-03-06T07:22:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.069+0000 7f79637fe640 1 -- 172.21.15.26:0/1085673582 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974012620 con 0x7f797c152920 2024-03-06T07:22:21.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.069+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c1eefe0 con 0x7f797c152920 2024-03-06T07:22:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.070+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c04f330 con 0x7f797c152920 2024-03-06T07:22:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.070+0000 7f79637fe640 1 -- 172.21.15.26:0/1085673582 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f797401a070 con 0x7f797c152920 2024-03-06T07:22:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.070+0000 7f79637fe640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f796806b4d0 0x7f796806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.070+0000 7f79637fe640 1 -- 172.21.15.26:0/1085673582 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f797408a210 con 0x7f797c152920 2024-03-06T07:22:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.071+0000 7f797affd640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f796806b4d0 0x7f796806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.071+0000 7f797affd640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f796806b4d0 0x7f796806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f796c036000 tx=0x7f796c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.075+0000 7f79637fe640 1 -- 172.21.15.26:0/1085673582 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f79740587e0 con 0x7f797c152920 2024-03-06T07:22:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.293+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f797c068460 con 0x7f797c152920 2024-03-06T07:22:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.294+0000 7f79637fe640 1 -- 172.21.15.26:0/1085673582 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f797405c880 con 0x7f797c152920 2024-03-06T07:22:21.296 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:21.296 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1256717910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1256717910},{"type":"v1","addr":"172.21.15.117:6836","nonce":1256717910}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4276915425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4276915425},{"type":"v1","addr":"172.21.15.26:6835","nonce":4276915425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:54.755920+0000","modified":"2024-03-06T07:22:19.506857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5240":{"gid":5240,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/357874491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":357874491},{"type":"v1","addr":"172.21.15.117:6837","nonce":357874491}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5240,"qdb_cluster":[5240]},"id":4}]} 2024-03-06T07:22:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2024-03-06T07:22:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f796806b4d0 msgr2=0x7f796806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f796806b4d0 0x7f796806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f796c036000 tx=0x7f796c03c040 comp rx=0 tx=0).stop 2024-03-06T07:22:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c152920 msgr2=0x7f797c1eabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c152920 0x7f797c1eabd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f797400cb40 tx=0x7f797400a060 comp rx=0 tx=0).stop 2024-03-06T07:22:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 shutdown_connections 2024-03-06T07:22:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f796806b4d0 0x7f796806d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c152920 0x7f797c1eabd0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c25f0 0x7f797c147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 --2- 172.21.15.26:0/1085673582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c0c2190 0x7f797c0725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.295+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 >> 172.21.15.26:0/1085673582 conn(0x7f797c0c4290 msgr2=0x7f797c14abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.296+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 shutdown_connections 2024-03-06T07:22:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.296+0000 7f7981372640 1 -- 172.21.15.26:0/1085673582 wait complete. 2024-03-06T07:22:21.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:21.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.408+0000 7f1183a6b640 1 Processor -- start 2024-03-06T07:22:21.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.408+0000 7f1183a6b640 1 -- start start 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1183a6b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f117c152920 0x7f117c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1183a6b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c0c5330 0x7f117c0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1183a6b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c156180 con 0x7f117c0c8a00 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1183a6b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c1587b0 con 0x7f117c152920 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1183a6b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c159630 con 0x7f117c0c5330 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1181807640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f117c152920 0x7f117c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1181006640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f117c0c8a00 0x7f117c158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55086/0 (socket says 172.21.15.26:55086) 2024-03-06T07:22:21.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1181006640 1 -- 172.21.15.26:0/3388095621 learned_addr learned my addr 172.21.15.26:0/3388095621 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:21.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1182008640 1 --2- 172.21.15.26:0/3388095621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c0c5330 0x7f117c0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1181807640 1 -- 172.21.15.26:0/3388095621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c0c5330 msgr2=0x7f117c0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:21.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1181807640 1 --2- 172.21.15.26:0/3388095621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c0c5330 0x7f117c0c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1181807640 1 -- 172.21.15.26:0/3388095621 >> v1:172.21.15.26:6789/0 conn(0x7f117c0c8a00 legacy=0x7f117c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.409+0000 7f1181807640 1 -- 172.21.15.26:0/3388095621 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0c7760 con 0x7f117c152920 2024-03-06T07:22:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.410+0000 7f1181807640 1 --2- 172.21.15.26:0/3388095621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c152920 0x7f117c155c40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f116c009d90 tx=0x7f116c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3ac9d38a46e460 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.410+0000 7f1180805640 1 -- 172.21.15.26:0/3388095621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c039070 con 0x7f117c152920 2024-03-06T07:22:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.410+0000 7f1180805640 1 -- 172.21.15.26:0/3388095621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c02cbc0 con 0x7f117c152920 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.410+0000 7f1180805640 1 -- 172.21.15.26:0/3388095621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c034440 con 0x7f117c152920 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 -- 172.21.15.26:0/3388095621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c152920 msgr2=0x7f117c155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3388095621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c152920 0x7f117c155c40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f116c009d90 tx=0x7f116c02be10 comp rx=0 tx=0).stop 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 -- 172.21.15.26:0/3388095621 shutdown_connections 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3388095621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c0c5330 0x7f117c0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3388095621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c152920 0x7f117c155c40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 -- 172.21.15.26:0/3388095621 >> 172.21.15.26:0/3388095621 conn(0x7f117c0ac560 msgr2=0x7f117c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 -- 172.21.15.26:0/3388095621 shutdown_connections 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.411+0000 7f1183a6b640 1 -- 172.21.15.26:0/3388095621 wait complete. 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.412+0000 7f1183a6b640 1 Processor -- start 2024-03-06T07:22:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.412+0000 7f1183a6b640 1 -- start start 2024-03-06T07:22:21.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1183a6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c0c5330 0x7f117c157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1183a6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f117c0c8a00 0x7f117c147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1183a6b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c152920 0x7f117c148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1183a6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c158a00 con 0x7f117c0c5330 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1183a6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0c7d20 con 0x7f117c0c8a00 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1183a6b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c06d030 con 0x7f117c152920 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c0c5330 0x7f117c157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c0c5330 0x7f117c157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41524/0 (socket says 172.21.15.26:41524) 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 -- 172.21.15.26:0/3499746775 learned_addr learned my addr 172.21.15.26:0/3499746775 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181006640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f117c0c8a00 0x7f117c147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1182008640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c152920 0x7f117c148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 -- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c152920 msgr2=0x7f117c148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c152920 0x7f117c148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 -- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f117c0c8a00 msgr2=0x7f117c147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f117c0c8a00 0x7f117c147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 -- 172.21.15.26:0/3499746775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0c4860 con 0x7f117c0c5330 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f1181807640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c0c5330 0x7f117c157870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f116c037040 tx=0x7f116c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f117a7fc640 1 -- 172.21.15.26:0/3499746775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c039050 con 0x7f117c0c5330 2024-03-06T07:22:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f117a7fc640 1 -- 172.21.15.26:0/3499746775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c045070 con 0x7f117c0c5330 2024-03-06T07:22:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.413+0000 7f117a7fc640 1 -- 172.21.15.26:0/3499746775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c03e850 con 0x7f117c0c5330 2024-03-06T07:22:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.414+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c1558f0 con 0x7f117c0c5330 2024-03-06T07:22:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.414+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c148580 con 0x7f117c0c5330 2024-03-06T07:22:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.415+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c04f330 con 0x7f117c0c5330 2024-03-06T07:22:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.415+0000 7f117a7fc640 1 -- 172.21.15.26:0/3499746775 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f116c03e9f0 con 0x7f117c0c5330 2024-03-06T07:22:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.415+0000 7f117a7fc640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f116806b4d0 0x7f116806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.415+0000 7f117a7fc640 1 -- 172.21.15.26:0/3499746775 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f116c0ad2e0 con 0x7f117c0c5330 2024-03-06T07:22:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.416+0000 7f1181006640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f116806b4d0 0x7f116806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.416+0000 7f1181006640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f116806b4d0 0x7f116806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f117c14b8c0 tx=0x7f1170007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.420+0000 7f117a7fc640 1 -- 172.21.15.26:0/3499746775 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f116c07b930 con 0x7f117c0c5330 2024-03-06T07:22:21.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.606+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f117c06cda0 con 0x7f117c0c5330 2024-03-06T07:22:21.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.607+0000 7f117a7fc640 1 -- 172.21.15.26:0/3499746775 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17419 (secure 0 0 0) 0x7f116c07f9d0 con 0x7f117c0c5330 2024-03-06T07:22:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:21.610 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":62,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:18.685508+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:52.622857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:53.707058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.26:6833/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.117:6836/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.117:6834/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.26:6834/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6832/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6835/440323369":"2024-03-07T07:22:04.621666+0000","172.21.15.26:6833/440323369":"2024-03-07T07:22:04.621666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f116806b4d0 msgr2=0x7f116806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f116806b4d0 0x7f116806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f117c14b8c0 tx=0x7f1170007e80 comp rx=0 tx=0).stop 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c0c5330 msgr2=0x7f117c157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c0c5330 0x7f117c157870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f116c037040 tx=0x7f116c004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 shutdown_connections 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f116806b4d0 0x7f116806d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f117c152920 0x7f117c148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f117c0c8a00 0x7f117c147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 --2- 172.21.15.26:0/3499746775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c0c5330 0x7f117c157870 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 >> 172.21.15.26:0/3499746775 conn(0x7f117c0ac560 msgr2=0x7f117c0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 shutdown_connections 2024-03-06T07:22:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.609+0000 7f1183a6b640 1 -- 172.21.15.26:0/3499746775 wait complete. 2024-03-06T07:22:21.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-06T07:22:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.723+0000 7fe0a44d4640 1 Processor -- start 2024-03-06T07:22:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.724+0000 7fe0a44d4640 1 -- start start 2024-03-06T07:22:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.724+0000 7fe0a44d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.724+0000 7fe0a44d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe09c1561a0 0x7fe09c0a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.724+0000 7fe0a44d4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe09c0aa430 0x7fe09c14f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.724+0000 7fe0a44d4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c155d10 con 0x7fe09c1561a0 2024-03-06T07:22:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.724+0000 7fe0a44d4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c0a9990 con 0x7fe09c0aa430 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.724+0000 7fe0a44d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1556c0 con 0x7fe09c0c6ab0 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe09c1561a0 0x7fe09c0a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe09c1561a0 0x7fe09c0a9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41532/0 (socket says 172.21.15.26:41532) 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 -- 172.21.15.26:0/4002587635 learned_addr learned my addr 172.21.15.26:0/4002587635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a2a71640 1 --2- 172.21.15.26:0/4002587635 >> v2:172.21.15.117:3300/0 conn(0x7fe09c0aa430 0x7fe09c14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a2270640 1 --2- 172.21.15.26:0/4002587635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 -- 172.21.15.26:0/4002587635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 msgr2=0x7fe09c1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 --2- 172.21.15.26:0/4002587635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 -- 172.21.15.26:0/4002587635 >> v2:172.21.15.117:3300/0 conn(0x7fe09c0aa430 msgr2=0x7fe09c14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 --2- 172.21.15.26:0/4002587635 >> v2:172.21.15.117:3300/0 conn(0x7fe09c0aa430 0x7fe09c14f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.725+0000 7fe0a1a6f640 1 -- 172.21.15.26:0/4002587635 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c1559a0 con 0x7fe09c1561a0 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a1a6f640 1 --2- 172.21.15.26:0/4002587635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c1561a0 0x7fe09c0a9ef0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a9300 tx=0x7fe08c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17415bf7aaf4d732 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a126e640 1 -- 172.21.15.26:0/4002587635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c038070 con 0x7fe09c1561a0 2024-03-06T07:22:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a126e640 1 -- 172.21.15.26:0/4002587635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c02f050 con 0x7fe09c1561a0 2024-03-06T07:22:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a126e640 1 -- 172.21.15.26:0/4002587635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c031970 con 0x7fe09c1561a0 2024-03-06T07:22:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 -- 172.21.15.26:0/4002587635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c1561a0 msgr2=0x7fe09c0a9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/4002587635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c1561a0 0x7fe09c0a9ef0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a9300 tx=0x7fe08c02b7f0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 -- 172.21.15.26:0/4002587635 shutdown_connections 2024-03-06T07:22:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/4002587635 >> v2:172.21.15.117:3300/0 conn(0x7fe09c0aa430 0x7fe09c14f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/4002587635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c1561a0 0x7fe09c0a9ef0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/4002587635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 -- 172.21.15.26:0/4002587635 >> 172.21.15.26:0/4002587635 conn(0x7fe09c06b1a0 msgr2=0x7fe09c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 -- 172.21.15.26:0/4002587635 shutdown_connections 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.726+0000 7fe0a44d4640 1 -- 172.21.15.26:0/4002587635 wait complete. 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.727+0000 7fe0a44d4640 1 Processor -- start 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.727+0000 7fe0a44d4640 1 -- start start 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a44d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c0aa430 0x7fe09c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a44d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a44d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09c1561a0 0x7fe09c1e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a44d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c2c10 con 0x7fe09c0aa430 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a44d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c2a90 con 0x7fe09c1561a0 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a44d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c6e90 con 0x7fe09c0c6ab0 2024-03-06T07:22:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c0aa430 0x7fe09c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c0aa430 0x7fe09c151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41538/0 (socket says 172.21.15.26:41538) 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 -- 172.21.15.26:0/2676564586 learned_addr learned my addr 172.21.15.26:0/2676564586 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a1a6f640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2a71640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09c1561a0 0x7fe09c1e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 -- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 msgr2=0x7fe09c1e89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1e89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 -- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09c1561a0 msgr2=0x7fe09c1e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09c1561a0 0x7fe09c1e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.728+0000 7fe0a2270640 1 -- 172.21.15.26:0/2676564586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c1ebeb0 con 0x7fe09c0aa430 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.729+0000 7fe0a2270640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c0aa430 0x7fe09c151020 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe09800e8a0 tx=0x7fe09800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.729+0000 7fe092ffd640 1 -- 172.21.15.26:0/2676564586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098017070 con 0x7fe09c0aa430 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.729+0000 7fe092ffd640 1 -- 172.21.15.26:0/2676564586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09800c940 con 0x7fe09c0aa430 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.729+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c1ec190 con 0x7fe09c0aa430 2024-03-06T07:22:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.729+0000 7fe092ffd640 1 -- 172.21.15.26:0/2676564586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09800cd30 con 0x7fe09c0aa430 2024-03-06T07:22:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.729+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1ec750 con 0x7fe09c0aa430 2024-03-06T07:22:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.730+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c04f330 con 0x7fe09c0aa430 2024-03-06T07:22:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.730+0000 7fe092ffd640 1 -- 172.21.15.26:0/2676564586 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe09801a070 con 0x7fe09c0aa430 2024-03-06T07:22:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.730+0000 7fe092ffd640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08006b4d0 0x7fe08006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.731+0000 7fe092ffd640 1 -- 172.21.15.26:0/2676564586 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fe09808a680 con 0x7fe09c0aa430 2024-03-06T07:22:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.731+0000 7fe0a1a6f640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08006b4d0 0x7fe08006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.731+0000 7fe0a1a6f640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08006b4d0 0x7fe08006d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe08c035000 tx=0x7fe08c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.735+0000 7fe092ffd640 1 -- 172.21.15.26:0/2676564586 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe098058bb0 con 0x7fe09c0aa430 2024-03-06T07:22:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.957+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7fe09c0c26b0 con 0x7fe09c0aa430 2024-03-06T07:22:21.960 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:22:21.958+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.959+0000 7fe092ffd640 1 -- 172.21.15.26:0/2676564586 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 163+0+0 (secure 0 0 0) 0x7fe09805cc50 con 0x7fe09c0aa430 2024-03-06T07:22:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:22:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.961+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08006b4d0 msgr2=0x7fe08006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08006b4d0 0x7fe08006d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe08c035000 tx=0x7fe08c008030 comp rx=0 tx=0).stop 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c0aa430 msgr2=0x7fe09c151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c0aa430 0x7fe09c151020 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe09800e8a0 tx=0x7fe09800ed60 comp rx=0 tx=0).stop 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 shutdown_connections 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe08006b4d0 0x7fe08006d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09c1561a0 0x7fe09c1e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09c0c6ab0 0x7fe09c1e89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 --2- 172.21.15.26:0/2676564586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c0aa430 0x7fe09c151020 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 >> 172.21.15.26:0/2676564586 conn(0x7fe09c06b1a0 msgr2=0x7fe09c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 shutdown_connections 2024-03-06T07:22:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:21.962+0000 7fe0a44d4640 1 -- 172.21.15.26:0/2676564586 wait complete. 2024-03-06T07:22:21.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-06T07:22:21.990 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:22:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.076+0000 7fe56073a640 1 Processor -- start 2024-03-06T07:22:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.076+0000 7fe56073a640 1 -- start start 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe56073a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5580ab310 0x7fe5580ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe56073a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5581538c0 0x7fe5581494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe56073a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe558154330 con 0x7fe5580a97c0 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe56073a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5580c46a0 con 0x7fe5580ab310 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe56073a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe558153dc0 con 0x7fe5581538c0 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55e4d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5580ab310 0x7fe5580ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55e4d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5580ab310 0x7fe5580ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41542/0 (socket says 172.21.15.26:41542) 2024-03-06T07:22:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55e4d6640 1 -- 172.21.15.26:0/2896197330 learned_addr learned my addr 172.21.15.26:0/2896197330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55dcd5640 1 --2- 172.21.15.26:0/2896197330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5581538c0 0x7fe5581494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55e4d6640 1 -- 172.21.15.26:0/2896197330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5581538c0 msgr2=0x7fe5581494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55e4d6640 1 --2- 172.21.15.26:0/2896197330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5581538c0 0x7fe5581494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55e4d6640 1 -- 172.21.15.26:0/2896197330 >> v1:172.21.15.117:6789/0 conn(0x7fe5580a97c0 legacy=0x7fe5580c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.077+0000 7fe55e4d6640 1 -- 172.21.15.26:0/2896197330 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5580c5d30 con 0x7fe5580ab310 2024-03-06T07:22:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.078+0000 7fe55e4d6640 1 --2- 172.21.15.26:0/2896197330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580ab6f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe5580c4820 tx=0x7fe55002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208d6eba488bd474 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.078+0000 7fe55d4d4640 1 -- 172.21.15.26:0/2896197330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55002c9c0 con 0x7fe5580ab310 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.078+0000 7fe55d4d4640 1 -- 172.21.15.26:0/2896197330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55002cb60 con 0x7fe5580ab310 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.078+0000 7fe55d4d4640 1 -- 172.21.15.26:0/2896197330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500343f0 con 0x7fe5580ab310 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 -- 172.21.15.26:0/2896197330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 msgr2=0x7fe5580ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 --2- 172.21.15.26:0/2896197330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580ab6f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe5580c4820 tx=0x7fe55002be20 comp rx=0 tx=0).stop 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 -- 172.21.15.26:0/2896197330 shutdown_connections 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 --2- 172.21.15.26:0/2896197330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5581538c0 0x7fe5581494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 --2- 172.21.15.26:0/2896197330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580ab6f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 -- 172.21.15.26:0/2896197330 >> 172.21.15.26:0/2896197330 conn(0x7fe5580c6710 msgr2=0x7fe5580c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 -- 172.21.15.26:0/2896197330 shutdown_connections 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.079+0000 7fe56073a640 1 -- 172.21.15.26:0/2896197330 wait complete. 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.080+0000 7fe56073a640 1 Processor -- start 2024-03-06T07:22:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.080+0000 7fe56073a640 1 -- start start 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5580a97c0 0x7fe558071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5581538c0 0x7fe5581e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580abc30 con 0x7fe5580ab310 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580c62f0 con 0x7fe5581538c0 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580c5240 con 0x7fe5580a97c0 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41558/0 (socket says 172.21.15.26:41558) 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 -- 172.21.15.26:0/2975999413 learned_addr learned my addr 172.21.15.26:0/2975999413 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55e4d6640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5580a97c0 0x7fe558071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55ecd7640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5581538c0 0x7fe5581e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 -- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5580a97c0 msgr2=0x7fe558071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5580a97c0 0x7fe558071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 -- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5581538c0 msgr2=0x7fe5581e9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5581538c0 0x7fe5581e9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 -- 172.21.15.26:0/2975999413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5580703f0 con 0x7fe5580ab310 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55ecd7640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5581538c0 0x7fe5581e9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55e4d6640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5580a97c0 0x7fe558071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe55dcd5640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580728f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00d620 tx=0x7fe54c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe556ffd640 1 -- 172.21.15.26:0/2975999413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00b620 con 0x7fe5580ab310 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe556ffd640 1 -- 172.21.15.26:0/2975999413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c019070 con 0x7fe5580ab310 2024-03-06T07:22:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe556ffd640 1 -- 172.21.15.26:0/2975999413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00ba00 con 0x7fe5580ab310 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5580706d0 con 0x7fe5580ab310 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.081+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55814a5e0 con 0x7fe5580ab310 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.082+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe558068e00 con 0x7fe5580ab310 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.083+0000 7fe556ffd640 1 -- 172.21.15.26:0/2975999413 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe54c00bba0 con 0x7fe5580ab310 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.083+0000 7fe556ffd640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe54406b500 0x7fe54406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.083+0000 7fe556ffd640 1 -- 172.21.15.26:0/2975999413 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6535+0+0 (secure 0 0 0) 0x7fe54c08a8b0 con 0x7fe5580ab310 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.083+0000 7fe55e4d6640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe54406b500 0x7fe54406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.084+0000 7fe55e4d6640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe54406b500 0x7fe54406d9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe550035000 tx=0x7fe550030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.088+0000 7fe556ffd640 1 -- 172.21.15.26:0/2975999413 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe558068e00 con 0x7fe5580ab310 2024-03-06T07:22:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.305+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe5580c5a70 con 0x7fe5580ab310 2024-03-06T07:22:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.960+0000 7fe556ffd640 1 -- 172.21.15.26:0/2975999413 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe54c059480 con 0x7fe5580ab310 2024-03-06T07:22:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe54406b500 msgr2=0x7fe54406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe54406b500 0x7fe54406d9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe550035000 tx=0x7fe550030000 comp rx=0 tx=0).stop 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 msgr2=0x7fe5580728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580728f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00d620 tx=0x7fe54c00dae0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 shutdown_connections 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe54406b500 0x7fe54406d9c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5581538c0 0x7fe5581e9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5580ab310 0x7fe5580728f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 --2- 172.21.15.26:0/2975999413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5580a97c0 0x7fe558071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 >> 172.21.15.26:0/2975999413 conn(0x7fe5580c6710 msgr2=0x7fe558151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 shutdown_connections 2024-03-06T07:22:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:22.963+0000 7fe56073a640 1 -- 172.21.15.26:0/2975999413 wait complete. 2024-03-06T07:22:22.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-06T07:22:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.084+0000 7f79ea90d640 1 Processor -- start 2024-03-06T07:22:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.085+0000 7f79ea90d640 1 -- start start 2024-03-06T07:22:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.085+0000 7f79ea90d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f79e406f2d0 0x7f79e415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.085+0000 7f79ea90d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e4072620 con 0x7f79e40c7f10 2024-03-06T07:22:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.085+0000 7f79ea90d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e4072900 con 0x7f79e40c8720 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.085+0000 7f79ea90d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e4072be0 con 0x7f79e406f2d0 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.085+0000 7f79e8eaa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f79e406f2d0 0x7f79e415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.085+0000 7f79e3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f79e40c7f10 0x7f79e40c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55088/0 (socket says 172.21.15.26:55088) 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.086+0000 7f79e3fff640 1 -- 172.21.15.26:0/2977067797 learned_addr learned my addr 172.21.15.26:0/2977067797 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.086+0000 7f79e8eaa640 1 -- 172.21.15.26:0/2977067797 >> v1:172.21.15.117:6789/0 conn(0x7f79e40c8720 legacy=0x7f79e406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.086+0000 7f79e8eaa640 1 -- 172.21.15.26:0/2977067797 >> v1:172.21.15.26:6789/0 conn(0x7f79e40c7f10 legacy=0x7f79e40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.086+0000 7f79e8eaa640 1 -- 172.21.15.26:0/2977067797 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e4072ec0 con 0x7f79e406f2d0 2024-03-06T07:22:23.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.086+0000 7f79e37fe640 1 -- 172.21.15.26:0/2977067797 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337220590 0 0) 0x7f79e4072620 con 0x7f79e40c7f10 2024-03-06T07:22:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79e8eaa640 1 --2- 172.21.15.26:0/2977067797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e415dd10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009d90 tx=0x7f79dc02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a3c32c4e6e907 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79e37fe640 1 -- 172.21.15.26:0/2977067797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc039070 con 0x7f79e406f2d0 2024-03-06T07:22:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79e37fe640 1 -- 172.21.15.26:0/2977067797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc02cb40 con 0x7f79e406f2d0 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79e37fe640 1 -- 172.21.15.26:0/2977067797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc034440 con 0x7f79e406f2d0 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79ea90d640 1 -- 172.21.15.26:0/2977067797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 msgr2=0x7f79e415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79ea90d640 1 --2- 172.21.15.26:0/2977067797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e415dd10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009d90 tx=0x7f79dc02bd90 comp rx=0 tx=0).stop 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79ea90d640 1 -- 172.21.15.26:0/2977067797 shutdown_connections 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79ea90d640 1 --2- 172.21.15.26:0/2977067797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e415dd10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79ea90d640 1 -- 172.21.15.26:0/2977067797 >> 172.21.15.26:0/2977067797 conn(0x7f79e40aa5c0 msgr2=0x7f79e40aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79ea90d640 1 -- 172.21.15.26:0/2977067797 shutdown_connections 2024-03-06T07:22:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.087+0000 7f79ea90d640 1 -- 172.21.15.26:0/2977067797 wait complete. 2024-03-06T07:22:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.088+0000 7f79ea90d640 1 Processor -- start 2024-03-06T07:22:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79ea90d640 1 -- start start 2024-03-06T07:22:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79ea90d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79ea90d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e40c7f10 0x7f79e4150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79ea90d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40c8720 0x7f79e406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79ea90d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40c8470 con 0x7f79e406f2d0 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79ea90d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40c82f0 con 0x7f79e40c8720 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79ea90d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415e250 con 0x7f79e40c7f10 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e414f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41584/0 (socket says 172.21.15.26:41584) 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79e3fff640 1 -- 172.21.15.26:0/1319763873 learned_addr learned my addr 172.21.15.26:0/1319763873 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.089+0000 7f79e3fff640 1 -- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e40c7f10 msgr2=0x7f79e4150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e3fff640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e40c7f10 0x7f79e4150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e3fff640 1 -- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40c8720 msgr2=0x7f79e406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e3fff640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40c8720 0x7f79e406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e3fff640 1 -- 172.21.15.26:0/1319763873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40c2820 con 0x7f79e406f2d0 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e3fff640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e414f630 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c4d0 tx=0x7f79d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e17fa640 1 -- 172.21.15.26:0/1319763873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4018070 con 0x7f79e406f2d0 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e17fa640 1 -- 172.21.15.26:0/1319763873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d4007500 con 0x7f79e406f2d0 2024-03-06T07:22:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e415dd40 con 0x7f79e406f2d0 2024-03-06T07:22:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79e17fa640 1 -- 172.21.15.26:0/1319763873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4004a20 con 0x7f79e406f2d0 2024-03-06T07:22:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.090+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e4151450 con 0x7f79e406f2d0 2024-03-06T07:22:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.091+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e404f330 con 0x7f79e406f2d0 2024-03-06T07:22:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.091+0000 7f79e17fa640 1 -- 172.21.15.26:0/1319763873 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79d401b070 con 0x7f79e406f2d0 2024-03-06T07:22:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.092+0000 7f79e17fa640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79c406b4d0 0x7f79c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.092+0000 7f79e17fa640 1 -- 172.21.15.26:0/1319763873 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f79d40897f0 con 0x7f79e406f2d0 2024-03-06T07:22:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.092+0000 7f79dbfff640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79c406b4d0 0x7f79c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.092+0000 7f79dbfff640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79c406b4d0 0x7f79c406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f79cc003cc0 tx=0x7f79cc0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.096+0000 7f79e17fa640 1 -- 172.21.15.26:0/1319763873 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f79d40583c0 con 0x7f79e406f2d0 2024-03-06T07:22:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.277+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79e40c3530 con 0x7f79e406f2d0 2024-03-06T07:22:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.965+0000 7f79e17fa640 1 -- 172.21.15.26:0/1319763873 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v64) v1 ==== 184+0+0 (secure 0 0 0) 0x7f79d4057cc0 con 0x7f79e406f2d0 2024-03-06T07:22:23.968 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_metadata' removed 2024-03-06T07:22:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79c406b4d0 msgr2=0x7f79c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79c406b4d0 0x7f79c406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f79cc003cc0 tx=0x7f79cc0079b0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 msgr2=0x7f79e414f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e414f630 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c4d0 tx=0x7f79d400c990 comp rx=0 tx=0).stop 2024-03-06T07:22:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 shutdown_connections 2024-03-06T07:22:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f79c406b4d0 0x7f79c406d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40c8720 0x7f79e406da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e40c7f10 0x7f79e4150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 --2- 172.21.15.26:0/1319763873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e406f2d0 0x7f79e414f630 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.968+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 >> 172.21.15.26:0/1319763873 conn(0x7f79e40aa5c0 msgr2=0x7f79e40aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.971+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 shutdown_connections 2024-03-06T07:22:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:23.971+0000 7f79ea90d640 1 -- 172.21.15.26:0/1319763873 wait complete. 2024-03-06T07:22:23.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.095+0000 7f1f9209a640 1 Processor -- start 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.095+0000 7f1f9209a640 1 -- start start 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f9209a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f8c0a9830 0x7f1f8c151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f9209a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c153820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f9209a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c06e850 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f9209a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c06d2b0 con 0x7f1f8c0a9830 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f9209a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c06d590 con 0x7f1f8c153390 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f8c0a9830 0x7f1f8c151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f8c0a9830 0x7f1f8c151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41588/0 (socket says 172.21.15.26:41588) 2024-03-06T07:22:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8affd640 1 -- 172.21.15.26:0/2870342502 learned_addr learned my addr 172.21.15.26:0/2870342502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8bfff640 1 --2- 172.21.15.26:0/2870342502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8affd640 1 -- 172.21.15.26:0/2870342502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 msgr2=0x7f1f8c153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8affd640 1 --2- 172.21.15.26:0/2870342502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8affd640 1 -- 172.21.15.26:0/2870342502 >> v1:172.21.15.117:6789/0 conn(0x7f1f8c0c22b0 legacy=0x7f1f8c14ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.096+0000 7f1f8affd640 1 -- 172.21.15.26:0/2870342502 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c06d870 con 0x7f1f8c0a9830 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.097+0000 7f1f8affd640 1 --2- 172.21.15.26:0/2870342502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c151e10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000bb40 tx=0x7f1f8002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5817b5c698e07fd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.097+0000 7f1f8a7fc640 1 -- 172.21.15.26:0/2870342502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8003b070 con 0x7f1f8c0a9830 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.097+0000 7f1f8a7fc640 1 -- 172.21.15.26:0/2870342502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80036440 con 0x7f1f8c0a9830 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.097+0000 7f1f8a7fc640 1 -- 172.21.15.26:0/2870342502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80036810 con 0x7f1f8c0a9830 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 -- 172.21.15.26:0/2870342502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0a9830 msgr2=0x7f1f8c151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 --2- 172.21.15.26:0/2870342502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c151e10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000bb40 tx=0x7f1f8002e150 comp rx=0 tx=0).stop 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 -- 172.21.15.26:0/2870342502 shutdown_connections 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 --2- 172.21.15.26:0/2870342502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 --2- 172.21.15.26:0/2870342502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c151e10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 -- 172.21.15.26:0/2870342502 >> 172.21.15.26:0/2870342502 conn(0x7f1f8c0c65b0 msgr2=0x7f1f8c0c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 -- 172.21.15.26:0/2870342502 shutdown_connections 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.098+0000 7f1f9209a640 1 -- 172.21.15.26:0/2870342502 wait complete. 2024-03-06T07:22:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.099+0000 7f1f9209a640 1 Processor -- start 2024-03-06T07:22:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.099+0000 7f1f9209a640 1 -- start start 2024-03-06T07:22:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f9209a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f9209a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0c22b0 0x7f1f8c14bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f9209a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c0be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f9209a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c14f7e0 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f9209a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c14f660 con 0x7f1f8c0a9830 2024-03-06T07:22:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f9209a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0c2690 con 0x7f1f8c153390 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0c22b0 0x7f1f8c14bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0c22b0 0x7f1f8c14bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41598/0 (socket says 172.21.15.26:41598) 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 -- 172.21.15.26:0/4174971614 learned_addr learned my addr 172.21.15.26:0/4174971614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8bfff640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c0be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8b7fe640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 -- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 msgr2=0x7f1f8c0be890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c0be890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 -- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c0a9830 msgr2=0x7f1f8c14dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c14dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 -- 172.21.15.26:0/4174971614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0bfe70 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8b7fe640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.100+0000 7f1f8affd640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0c22b0 0x7f1f8c14bb60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000b2c0 tx=0x7f1f800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.101+0000 7f1f73fff640 1 -- 172.21.15.26:0/4174971614 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80045020 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.101+0000 7f1f73fff640 1 -- 172.21.15.26:0/4174971614 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80041620 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.101+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c0c0150 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.101+0000 7f1f73fff640 1 -- 172.21.15.26:0/4174971614 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8003b040 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.101+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c0c4d20 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.102+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c04f330 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.102+0000 7f1f73fff640 1 -- 172.21.15.26:0/4174971614 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f80048020 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.102+0000 7f1f73fff640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f7806b4d0 0x7f1f7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.103+0000 7f1f73fff640 1 -- 172.21.15.26:0/4174971614 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6095+0+0 (secure 0 0 0) 0x7f1f800aec40 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.103+0000 7f1f8b7fe640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f7806b4d0 0x7f1f7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.103+0000 7f1f8b7fe640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f7806b4d0 0x7f1f7806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c001680 tx=0x7f1f7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:24.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.107+0000 7f1f73fff640 1 -- 172.21.15.26:0/4174971614 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f8004f030 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.295+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f8c0c0870 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.966+0000 7f1f73fff640 1 -- 172.21.15.26:0/4174971614 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v65) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1f8007d950 con 0x7f1f8c0c22b0 2024-03-06T07:22:24.969 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_data' removed 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f7806b4d0 msgr2=0x7f1f7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f7806b4d0 0x7f1f7806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c001680 tx=0x7f1f7c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0c22b0 msgr2=0x7f1f8c14bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0c22b0 0x7f1f8c14bb60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000b2c0 tx=0x7f1f800046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 shutdown_connections 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f7806b4d0 0x7f1f7806d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f8c153390 0x7f1f8c0be890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0c22b0 0x7f1f8c14bb60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 --2- 172.21.15.26:0/4174971614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c0a9830 0x7f1f8c14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 >> 172.21.15.26:0/4174971614 conn(0x7f1f8c0c65b0 msgr2=0x7f1f8c152390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 shutdown_connections 2024-03-06T07:22:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:24.970+0000 7f1f9209a640 1 -- 172.21.15.26:0/4174971614 wait complete. 2024-03-06T07:22:24.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:22:25.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.093+0000 7fdd61ec9640 1 Processor -- start 2024-03-06T07:22:25.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.093+0000 7fdd61ec9640 1 -- start start 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd61ec9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd5c14ef70 0x7fdd5c06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd61ec9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd5c06cad0 0x7fdd5c154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd61ec9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c0c2790 con 0x7fdd5c0c1820 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd61ec9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c0c3090 con 0x7fdd5c06cad0 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd61ec9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c14f350 con 0x7fdd5c14ef70 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd5c06cad0 0x7fdd5c154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdd5c0c1820 0x7fdd5c0c2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55096/0 (socket says 172.21.15.26:55096) 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5affd640 1 -- 172.21.15.26:0/2894782049 learned_addr learned my addr 172.21.15.26:0/2894782049 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5b7fe640 1 --2- 172.21.15.26:0/2894782049 >> v2:172.21.15.117:3300/0 conn(0x7fdd5c14ef70 0x7fdd5c06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5bfff640 1 -- 172.21.15.26:0/2894782049 >> v2:172.21.15.117:3300/0 conn(0x7fdd5c14ef70 msgr2=0x7fdd5c06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5bfff640 1 --2- 172.21.15.26:0/2894782049 >> v2:172.21.15.117:3300/0 conn(0x7fdd5c14ef70 0x7fdd5c06c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5bfff640 1 -- 172.21.15.26:0/2894782049 >> v1:172.21.15.26:6789/0 conn(0x7fdd5c0c1820 legacy=0x7fdd5c0c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5bfff640 1 -- 172.21.15.26:0/2894782049 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c06cf40 con 0x7fdd5c06cad0 2024-03-06T07:22:25.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5a7fc640 1 -- 172.21.15.26:0/2894782049 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486241591 0 0) 0x7fdd5c0c2790 con 0x7fdd5c0c1820 2024-03-06T07:22:25.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.094+0000 7fdd5b7fe640 1 --2- 172.21.15.26:0/2894782049 >> v2:172.21.15.117:3300/0 conn(0x7fdd5c14ef70 0x7fdd5c06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:25.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.095+0000 7fdd5bfff640 1 --2- 172.21.15.26:0/2894782049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c154c30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c06e5d0 tx=0x7fdd5402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149ff390cfdff1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.095+0000 7fdd5a7fc640 1 -- 172.21.15.26:0/2894782049 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54039070 con 0x7fdd5c06cad0 2024-03-06T07:22:25.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.095+0000 7fdd5a7fc640 1 -- 172.21.15.26:0/2894782049 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5402cbf0 con 0x7fdd5c06cad0 2024-03-06T07:22:25.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.095+0000 7fdd5a7fc640 1 -- 172.21.15.26:0/2894782049 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54034440 con 0x7fdd5c06cad0 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 -- 172.21.15.26:0/2894782049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 msgr2=0x7fdd5c154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/2894782049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c154c30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c06e5d0 tx=0x7fdd5402be40 comp rx=0 tx=0).stop 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 -- 172.21.15.26:0/2894782049 shutdown_connections 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/2894782049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c154c30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/2894782049 >> v2:172.21.15.117:3300/0 conn(0x7fdd5c14ef70 0x7fdd5c06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 -- 172.21.15.26:0/2894782049 >> 172.21.15.26:0/2894782049 conn(0x7fdd5c0c6710 msgr2=0x7fdd5c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 -- 172.21.15.26:0/2894782049 shutdown_connections 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.096+0000 7fdd61ec9640 1 -- 172.21.15.26:0/2894782049 wait complete. 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.097+0000 7fdd61ec9640 1 Processor -- start 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.097+0000 7fdd61ec9640 1 -- start start 2024-03-06T07:22:25.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd61ec9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c0717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd61ec9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd5c0c1820 0x7fdd5c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd61ec9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd5c14ef70 0x7fdd5c1ec810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd61ec9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c155750 con 0x7fdd5c06cad0 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd61ec9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0c24b0 con 0x7fdd5c14ef70 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd61ec9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c155170 con 0x7fdd5c0c1820 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c0717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c0717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41614/0 (socket says 172.21.15.26:41614) 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 -- 172.21.15.26:0/516617902 learned_addr learned my addr 172.21.15.26:0/516617902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5affd640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd5c0c1820 0x7fdd5c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 -- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd5c0c1820 msgr2=0x7fdd5c151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd5c0c1820 0x7fdd5c151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 -- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd5c14ef70 msgr2=0x7fdd5c1ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5bfff640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd5c14ef70 0x7fdd5c1ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd5c14ef70 0x7fdd5c1ec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 -- 172.21.15.26:0/516617902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c1454a0 con 0x7fdd5c06cad0 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5bfff640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd5c14ef70 0x7fdd5c1ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.098+0000 7fdd5b7fe640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c0717e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00d800 tx=0x7fdd4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.099+0000 7fdd43fff640 1 -- 172.21.15.26:0/516617902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004030 con 0x7fdd5c06cad0 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.099+0000 7fdd43fff640 1 -- 172.21.15.26:0/516617902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0041d0 con 0x7fdd5c06cad0 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.099+0000 7fdd43fff640 1 -- 172.21.15.26:0/516617902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00b6c0 con 0x7fdd5c06cad0 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.099+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c145780 con 0x7fdd5c06cad0 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.099+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c151e90 con 0x7fdd5c06cad0 2024-03-06T07:22:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.100+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c04f330 con 0x7fdd5c06cad0 2024-03-06T07:22:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.100+0000 7fdd43fff640 1 -- 172.21.15.26:0/516617902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd4c00b860 con 0x7fdd5c06cad0 2024-03-06T07:22:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.100+0000 7fdd43fff640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd4806b4d0 0x7fdd4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.101+0000 7fdd43fff640 1 -- 172.21.15.26:0/516617902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fdd4c08a440 con 0x7fdd5c06cad0 2024-03-06T07:22:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.101+0000 7fdd5affd640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd4806b4d0 0x7fdd4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.101+0000 7fdd5affd640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd4806b4d0 0x7fdd4806d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdd50007fd0 tx=0x7fdd50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.105+0000 7fdd43fff640 1 -- 172.21.15.26:0/516617902 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdd4c059370 con 0x7fdd5c06cad0 2024-03-06T07:22:25.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.289+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdd5c073380 con 0x7fdd5c06cad0 2024-03-06T07:22:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.292+0000 7fdd43fff640 1 -- 172.21.15.26:0/516617902 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd4c058c70 con 0x7fdd5c06cad0 2024-03-06T07:22:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.294+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd4806b4d0 msgr2=0x7fdd4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd4806b4d0 0x7fdd4806d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdd50007fd0 tx=0x7fdd50009040 comp rx=0 tx=0).stop 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 msgr2=0x7fdd5c0717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c0717e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00d800 tx=0x7fdd4c00dcc0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 shutdown_connections 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdd4806b4d0 0x7fdd4806d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd5c14ef70 0x7fdd5c1ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd5c0c1820 0x7fdd5c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 --2- 172.21.15.26:0/516617902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd5c06cad0 0x7fdd5c0717e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 >> 172.21.15.26:0/516617902 conn(0x7fdd5c0c6710 msgr2=0x7fdd5c0c3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 shutdown_connections 2024-03-06T07:22:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.295+0000 7fdd61ec9640 1 -- 172.21.15.26:0/516617902 wait complete. 2024-03-06T07:22:25.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:22:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.409+0000 7faa31671640 1 Processor -- start 2024-03-06T07:22:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.409+0000 7faa31671640 1 -- start start 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa31671640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 0x7faa2c0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa31671640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa2c0c7210 0x7faa2c0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa31671640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c0b04b0 con 0x7faa2c0b00d0 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa31671640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c156160 con 0x7faa2c0c7210 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa31671640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0c5710 con 0x7faa2c06bff0 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa2affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa2c0b00d0 0x7faa2c06b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55104/0 (socket says 172.21.15.26:55104) 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa2affd640 1 -- 172.21.15.26:0/2609239525 learned_addr learned my addr 172.21.15.26:0/2609239525 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558263642 0 0) 0x7faa2c0b04b0 con 0x7faa2c0b00d0 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa2a7fc640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 0x7faa2c0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa14003520 con 0x7faa2c0b00d0 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.410+0000 7faa2b7fe640 1 --2- 172.21.15.26:0/2609239525 >> v2:172.21.15.117:3300/0 conn(0x7faa2c0c7210 0x7faa2c0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3231199079 0 0) 0x7faa14003520 con 0x7faa2c0b00d0 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 msgr2=0x7faa2c0c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 0x7faa2c0c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 >> v2:172.21.15.117:3300/0 conn(0x7faa2c0c7210 msgr2=0x7faa2c0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 --2- 172.21.15.26:0/2609239525 >> v2:172.21.15.117:3300/0 conn(0x7faa2c0c7210 0x7faa2c0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c06d210 con 0x7faa2c0b00d0 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7faa24001670 con 0x7faa2c0b00d0 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 >> v1:172.21.15.26:6789/0 conn(0x7faa2c0b00d0 legacy=0x7faa2c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 0x7faa140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa14003f30 0x7faa14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa14004d40 0x7faa140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c156160 con 0x7faa14003f30 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0c5710 con 0x7faa14004d40 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1400b9c0 con 0x7faa2c06bff0 2024-03-06T07:22:25.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2a7fc640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa14003f30 0x7faa14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2affd640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 0x7faa140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2b7fe640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa14004d40 0x7faa140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2a7fc640 1 -- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 msgr2=0x7faa140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2a7fc640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c06bff0 0x7faa140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2a7fc640 1 -- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa14004d40 msgr2=0x7faa140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2a7fc640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa14004d40 0x7faa140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.411+0000 7faa2a7fc640 1 -- 172.21.15.26:0/2609239525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14003520 con 0x7faa14003f30 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.412+0000 7faa2a7fc640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa14003f30 0x7faa14004800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa20009ac0 tx=0x7faa14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926ce2ef8b9c8978 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.412+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20039070 con 0x7faa14003f30 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.412+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2002c6b0 con 0x7faa14003f30 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.412+0000 7faa29ffb640 1 -- 172.21.15.26:0/2609239525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20009690 con 0x7faa14003f30 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.413+0000 7faa31671640 1 -- 172.21.15.26:0/2609239525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa14003f30 msgr2=0x7faa14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.413+0000 7faa31671640 1 --2- 172.21.15.26:0/2609239525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa14003f30 0x7faa14004800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa20009ac0 tx=0x7faa14005830 comp rx=0 tx=0).stop 2024-03-06T07:22:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.413+0000 7faa2b7fe640 1 -- 172.21.15.26:0/2609239525 reap_dead start 2024-03-06T07:22:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.413+0000 7faa31671640 1 -- 172.21.15.26:0/2609239525 shutdown_connections 2024-03-06T07:22:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.413+0000 7faa31671640 1 -- 172.21.15.26:0/2609239525 >> 172.21.15.26:0/2609239525 conn(0x7faa2c0ac810 msgr2=0x7faa2c0acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.413+0000 7faa31671640 1 -- 172.21.15.26:0/2609239525 shutdown_connections 2024-03-06T07:22:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.413+0000 7faa31671640 1 -- 172.21.15.26:0/2609239525 wait complete. 2024-03-06T07:22:25.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.414+0000 7faa31671640 1 Processor -- start 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.414+0000 7faa31671640 1 -- start start 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa31671640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa14003f30 0x7faa2c0ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa31671640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0c4750 0x7faa2c0c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa31671640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2c0c29d0 0x7faa2c14c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa31671640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0c5ba0 con 0x7faa2c0c29d0 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa31671640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c152c50 con 0x7faa2c0c4750 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa31671640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0c5d20 con 0x7faa14003f30 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2c0c29d0 0x7faa2c14c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2c0c29d0 0x7faa2c14c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41618/0 (socket says 172.21.15.26:41618) 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 -- 172.21.15.26:0/1465449726 learned_addr learned my addr 172.21.15.26:0/1465449726 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2a7fc640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0c4750 0x7faa2c0c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2affd640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa14003f30 0x7faa2c0ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 -- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa14003f30 msgr2=0x7faa2c0ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa14003f30 0x7faa2c0ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 -- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0c4750 msgr2=0x7faa2c0c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0c4750 0x7faa2c0c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 -- 172.21.15.26:0/1465449726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0caa60 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2a7fc640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0c4750 0x7faa2c0c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa2b7fe640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2c0c29d0 0x7faa2c14c090 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa1c011bf0 tx=0x7faa1c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa137fe640 1 -- 172.21.15.26:0/1465449726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00fe10 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa137fe640 1 -- 172.21.15.26:0/1465449726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c015430 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c1ebc20 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.415+0000 7faa137fe640 1 -- 172.21.15.26:0/1465449726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c015820 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.416+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c0cb000 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.416+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c04f330 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.417+0000 7faa137fe640 1 -- 172.21.15.26:0/1465449726 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa1c0159c0 con 0x7faa2c0c29d0 2024-03-06T07:22:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.417+0000 7faa137fe640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa1806b4d0 0x7faa1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.417+0000 7faa137fe640 1 -- 172.21.15.26:0/1465449726 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7faa1c08afe0 con 0x7faa2c0c29d0 2024-03-06T07:22:25.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.417+0000 7faa2affd640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa1806b4d0 0x7faa1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.418+0000 7faa2affd640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa1806b4d0 0x7faa1806d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa240014f0 tx=0x7faa24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.422+0000 7faa137fe640 1 -- 172.21.15.26:0/1465449726 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa1c019030 con 0x7faa2c0c29d0 2024-03-06T07:22:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.602+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7faa2c06ce50 con 0x7faa2c0c29d0 2024-03-06T07:22:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.605+0000 7faa137fe640 1 -- 172.21.15.26:0/1465449726 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v20) v1 ==== 96+0+0 (secure 0 0 0) 0x7faa1c010040 con 0x7faa2c0c29d0 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa1806b4d0 msgr2=0x7faa1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa1806b4d0 0x7faa1806d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa240014f0 tx=0x7faa24009040 comp rx=0 tx=0).stop 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2c0c29d0 msgr2=0x7faa2c14c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2c0c29d0 0x7faa2c14c090 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa1c011bf0 tx=0x7faa1c00f580 comp rx=0 tx=0).stop 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 shutdown_connections 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa1806b4d0 0x7faa1806d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2c0c29d0 0x7faa2c14c090 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0c4750 0x7faa2c0c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 --2- 172.21.15.26:0/1465449726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa14003f30 0x7faa2c0ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 >> 172.21.15.26:0/1465449726 conn(0x7faa2c0ac810 msgr2=0x7faa2c1de830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 shutdown_connections 2024-03-06T07:22:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.607+0000 7faa31671640 1 -- 172.21.15.26:0/1465449726 wait complete. 2024-03-06T07:22:25.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:22:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.722+0000 7f69427f7640 1 Processor -- start 2024-03-06T07:22:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.722+0000 7f69427f7640 1 -- start start 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.722+0000 7f69427f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f693c1527f0 0x7f693c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f69427f7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693c1596f0 con 0x7f693c0c88f0 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f69427f7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693c0c82a0 con 0x7f693c0c58b0 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f69427f7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693c0c9130 con 0x7f693c1527f0 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f693b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f693c0c88f0 0x7f693c155210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55112/0 (socket says 172.21.15.26:55112) 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f693bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f693c1527f0 0x7f693c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f693b7fe640 1 -- 172.21.15.26:0/583470445 learned_addr learned my addr 172.21.15.26:0/583470445 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f693bfff640 1 -- 172.21.15.26:0/583470445 >> v1:172.21.15.117:6789/0 conn(0x7f693c0c58b0 legacy=0x7f693c15fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f693bfff640 1 -- 172.21.15.26:0/583470445 >> v1:172.21.15.26:6789/0 conn(0x7f693c0c88f0 legacy=0x7f693c155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f693affd640 1 -- 172.21.15.26:0/583470445 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893349727 0 0) 0x7f693c1596f0 con 0x7f693c0c88f0 2024-03-06T07:22:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.723+0000 7f693bfff640 1 -- 172.21.15.26:0/583470445 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c158930 con 0x7f693c1527f0 2024-03-06T07:22:25.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f693bfff640 1 --2- 172.21.15.26:0/583470445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c0c5370 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f692c0097a0 tx=0x7f692c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66b7ef01d0b3fdd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f693affd640 1 -- 172.21.15.26:0/583470445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c038070 con 0x7f693c1527f0 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f693affd640 1 -- 172.21.15.26:0/583470445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c031700 con 0x7f693c1527f0 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f693affd640 1 -- 172.21.15.26:0/583470445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c031ad0 con 0x7f693c1527f0 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f69427f7640 1 -- 172.21.15.26:0/583470445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 msgr2=0x7f693c0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f69427f7640 1 --2- 172.21.15.26:0/583470445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c0c5370 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f692c0097a0 tx=0x7f692c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f69427f7640 1 -- 172.21.15.26:0/583470445 shutdown_connections 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.724+0000 7f69427f7640 1 --2- 172.21.15.26:0/583470445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c0c5370 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.725+0000 7f69427f7640 1 -- 172.21.15.26:0/583470445 >> 172.21.15.26:0/583470445 conn(0x7f693c0ac580 msgr2=0x7f693c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.725+0000 7f69427f7640 1 -- 172.21.15.26:0/583470445 shutdown_connections 2024-03-06T07:22:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.725+0000 7f69427f7640 1 -- 172.21.15.26:0/583470445 wait complete. 2024-03-06T07:22:25.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 Processor -- start 2024-03-06T07:22:25.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 -- start start 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f693c0c58b0 0x7f693c0abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693c0c88f0 0x7f693c148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c0c6b90 con 0x7f693c1527f0 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c159260 con 0x7f693c0c58b0 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.726+0000 7f69427f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693c158ef0 con 0x7f693c0c88f0 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c148760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41644/0 (socket says 172.21.15.26:41644) 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 -- 172.21.15.26:0/63245967 learned_addr learned my addr 172.21.15.26:0/63245967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f693b7fe640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693c0c88f0 0x7f693c148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 -- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693c0c88f0 msgr2=0x7f693c148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f693bfff640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f693c0c58b0 0x7f693c0abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693c0c88f0 0x7f693c148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 -- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f693c0c58b0 msgr2=0x7f693c0abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f693c0c58b0 0x7f693c0abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 -- 172.21.15.26:0/63245967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c148fa0 con 0x7f693c1527f0 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6940d94640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c148760 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f693400b970 tx=0x7f693400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6938ff9640 1 -- 172.21.15.26:0/63245967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934019070 con 0x7f693c1527f0 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6938ff9640 1 -- 172.21.15.26:0/63245967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693400cb40 con 0x7f693c1527f0 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c156020 con 0x7f693c1527f0 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f6938ff9640 1 -- 172.21.15.26:0/63245967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934014440 con 0x7f693c1527f0 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.727+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c14d930 con 0x7f693c1527f0 2024-03-06T07:22:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.728+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693c155e10 con 0x7f693c1527f0 2024-03-06T07:22:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.729+0000 7f6938ff9640 1 -- 172.21.15.26:0/63245967 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f693401c070 con 0x7f693c1527f0 2024-03-06T07:22:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.729+0000 7f6938ff9640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692806b4d0 0x7f692806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.729+0000 7f6938ff9640 1 -- 172.21.15.26:0/63245967 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f6934089dc0 con 0x7f693c1527f0 2024-03-06T07:22:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.729+0000 7f693bfff640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692806b4d0 0x7f692806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.730+0000 7f693bfff640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692806b4d0 0x7f692806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f692c035000 tx=0x7f692c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.733+0000 7f6938ff9640 1 -- 172.21.15.26:0/63245967 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6934058cf0 con 0x7f693c1527f0 2024-03-06T07:22:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.915+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7f693c04f330 con 0x7f693c1527f0 2024-03-06T07:22:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.918+0000 7f6938ff9640 1 -- 172.21.15.26:0/63245967 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v21) v1 ==== 96+0+0 (secure 0 0 0) 0x7f69340585f0 con 0x7f693c1527f0 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692806b4d0 msgr2=0x7f692806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692806b4d0 0x7f692806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f692c035000 tx=0x7f692c008030 comp rx=0 tx=0).stop 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 msgr2=0x7f693c148760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c148760 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f693400b970 tx=0x7f693400be30 comp rx=0 tx=0).stop 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 shutdown_connections 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f692806b4d0 0x7f692806d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f693c1527f0 0x7f693c148760 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693c0c88f0 0x7f693c148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 --2- 172.21.15.26:0/63245967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f693c0c58b0 0x7f693c0abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 >> 172.21.15.26:0/63245967 conn(0x7f693c0ac580 msgr2=0x7f693c1517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 shutdown_connections 2024-03-06T07:22:25.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:25.920+0000 7f69427f7640 1 -- 172.21.15.26:0/63245967 wait complete. 2024-03-06T07:22:25.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.a mds_join_fs 2024-03-06T07:22:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.037+0000 7fa705181640 1 Processor -- start 2024-03-06T07:22:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.037+0000 7fa705181640 1 -- start start 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa705181640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7000b0070 0x7fa700155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa705181640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7000b0450 con 0x7fa70006ca70 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa705181640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70006cf20 con 0x7fa7000c5380 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa705181640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70006c020 con 0x7fa7000b0070 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa6fed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7000b0070 0x7fa700155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa6fe59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa70006ca70 0x7fa7000c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55114/0 (socket says 172.21.15.26:55114) 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa6fe59c640 1 -- 172.21.15.26:0/3728240678 learned_addr learned my addr 172.21.15.26:0/3728240678 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa6fed9d640 1 -- 172.21.15.26:0/3728240678 >> v1:172.21.15.117:6789/0 conn(0x7fa7000c5380 legacy=0x7fa70014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa6fed9d640 1 -- 172.21.15.26:0/3728240678 >> v1:172.21.15.26:6789/0 conn(0x7fa70006ca70 legacy=0x7fa7000c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.038+0000 7fa6fed9d640 1 -- 172.21.15.26:0/3728240678 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000c77f0 con 0x7fa7000b0070 2024-03-06T07:22:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.039+0000 7fa6fed9d640 1 --2- 172.21.15.26:0/3728240678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa700155c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80097a0 tx=0x7fa6f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11dfb3c4a3dfbfb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.039+0000 7fa6fdd9b640 1 -- 172.21.15.26:0/3728240678 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80315a0 con 0x7fa7000b0070 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.039+0000 7fa6fdd9b640 1 -- 172.21.15.26:0/3728240678 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f8037070 con 0x7fa7000b0070 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.039+0000 7fa6fdd9b640 1 -- 172.21.15.26:0/3728240678 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8031960 con 0x7fa7000b0070 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.040+0000 7fa705181640 1 -- 172.21.15.26:0/3728240678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 msgr2=0x7fa700155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.040+0000 7fa705181640 1 --2- 172.21.15.26:0/3728240678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa700155c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80097a0 tx=0x7fa6f802b840 comp rx=0 tx=0).stop 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.040+0000 7fa705181640 1 -- 172.21.15.26:0/3728240678 shutdown_connections 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.040+0000 7fa705181640 1 --2- 172.21.15.26:0/3728240678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa700155c30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.040+0000 7fa705181640 1 -- 172.21.15.26:0/3728240678 >> 172.21.15.26:0/3728240678 conn(0x7fa7000ac590 msgr2=0x7fa7000ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.040+0000 7fa705181640 1 -- 172.21.15.26:0/3728240678 shutdown_connections 2024-03-06T07:22:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.040+0000 7fa705181640 1 -- 172.21.15.26:0/3728240678 wait complete. 2024-03-06T07:22:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.041+0000 7fa705181640 1 Processor -- start 2024-03-06T07:22:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.041+0000 7fa705181640 1 -- start start 2024-03-06T07:22:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa705181640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70006ca70 0x7fa700156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa705181640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa7001542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa705181640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7000c5380 0x7fa7001eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa705181640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70006c550 con 0x7fa7000b0070 2024-03-06T07:22:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa705181640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70006c270 con 0x7fa7000c5380 2024-03-06T07:22:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa705181640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70006da80 con 0x7fa70006ca70 2024-03-06T07:22:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa7001542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa7001542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41658/0 (socket says 172.21.15.26:41658) 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 -- 172.21.15.26:0/3122448017 learned_addr learned my addr 172.21.15.26:0/3122448017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fed9d640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70006ca70 0x7fa700156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6ff59e640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7000c5380 0x7fa7001eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 -- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70006ca70 msgr2=0x7fa700156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70006ca70 0x7fa700156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 -- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7000c5380 msgr2=0x7fa7001eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7000c5380 0x7fa7001eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 -- 172.21.15.26:0/3122448017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa700156640 con 0x7fa7000b0070 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fed9d640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70006ca70 0x7fa700156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6ff59e640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7000c5380 0x7fa7001eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6fe59c640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa7001542e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c3c0 tx=0x7fa6e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6f77fe640 1 -- 172.21.15.26:0/3122448017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8017070 con 0x7fa7000b0070 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6f77fe640 1 -- 172.21.15.26:0/3122448017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8005ce0 con 0x7fa7000b0070 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa700156920 con 0x7fa7000b0070 2024-03-06T07:22:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.042+0000 7fa6f77fe640 1 -- 172.21.15.26:0/3122448017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80072f0 con 0x7fa7000b0070 2024-03-06T07:22:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.043+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa700154820 con 0x7fa7000b0070 2024-03-06T07:22:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.043+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70004f330 con 0x7fa7000b0070 2024-03-06T07:22:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.044+0000 7fa6f77fe640 1 -- 172.21.15.26:0/3122448017 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6e801a070 con 0x7fa7000b0070 2024-03-06T07:22:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.044+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6e006b4d0 0x7fa6e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.044+0000 7fa6f77fe640 1 -- 172.21.15.26:0/3122448017 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fa6e8089530 con 0x7fa7000b0070 2024-03-06T07:22:26.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.044+0000 7fa6fed9d640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6e006b4d0 0x7fa6e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.045+0000 7fa6fed9d640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6e006b4d0 0x7fa6e006d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8036000 tx=0x7fa6f8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.048+0000 7fa6f77fe640 1 -- 172.21.15.26:0/3122448017 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7000b0450 con 0x7fa7000b0070 2024-03-06T07:22:26.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.232+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"} v 0) v1 -- 0x7fa70006d860 con 0x7fa7000b0070 2024-03-06T07:22:26.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.235+0000 7fa6f77fe640 1 -- 172.21.15.26:0/3122448017 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"}]=0 v22) v1 ==== 96+0+0 (secure 0 0 0) 0x7fa6e8058460 con 0x7fa7000b0070 2024-03-06T07:22:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6e006b4d0 msgr2=0x7fa6e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6e006b4d0 0x7fa6e006d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8036000 tx=0x7fa6f8008030 comp rx=0 tx=0).stop 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 msgr2=0x7fa7001542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa7001542e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c3c0 tx=0x7fa6e800c880 comp rx=0 tx=0).stop 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 shutdown_connections 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6e006b4d0 0x7fa6e006d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7000c5380 0x7fa7001eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7000b0070 0x7fa7001542e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 --2- 172.21.15.26:0/3122448017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70006ca70 0x7fa700156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 >> 172.21.15.26:0/3122448017 conn(0x7fa7000ac590 msgr2=0x7fa7000ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 shutdown_connections 2024-03-06T07:22:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.237+0000 7fa705181640 1 -- 172.21.15.26:0/3122448017 wait complete. 2024-03-06T07:22:26.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort' 2024-03-06T07:22:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.349+0000 7f666ceb5640 1 Processor -- start 2024-03-06T07:22:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.349+0000 7f666ceb5640 1 -- start start 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.349+0000 7f666ceb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66681506b0 0x7f6668153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f666ceb5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66680acc00 0x7f66680c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f666ceb5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6668070aa0 con 0x7f6668156ea0 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f666ceb5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6668157320 con 0x7f66680acc00 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f666ceb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6668156450 con 0x7f66681506b0 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f6666d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66680acc00 0x7f66680c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f6666d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66680acc00 0x7f66680c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41668/0 (socket says 172.21.15.26:41668) 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f6666d9d640 1 -- 172.21.15.26:0/365455314 learned_addr learned my addr 172.21.15.26:0/365455314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f666659c640 1 --2- 172.21.15.26:0/365455314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66681506b0 0x7f6668153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f6666d9d640 1 -- 172.21.15.26:0/365455314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66681506b0 msgr2=0x7f6668153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f6666d9d640 1 --2- 172.21.15.26:0/365455314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66681506b0 0x7f6668153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f6666d9d640 1 -- 172.21.15.26:0/365455314 >> v1:172.21.15.117:6789/0 conn(0x7f6668156ea0 legacy=0x7f66680702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.350+0000 7f6666d9d640 1 -- 172.21.15.26:0/365455314 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6668070d80 con 0x7f66680acc00 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f6666d9d640 1 --2- 172.21.15.26:0/365455314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66680acc00 0x7f66680c54a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6660009e50 tx=0x7f666002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f883b84f3dd60c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f6665d9b640 1 -- 172.21.15.26:0/365455314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666002cb00 con 0x7f66680acc00 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f6665d9b640 1 -- 172.21.15.26:0/365455314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666002cca0 con 0x7f66680acc00 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f6665d9b640 1 -- 172.21.15.26:0/365455314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660034610 con 0x7f66680acc00 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f666ceb5640 1 -- 172.21.15.26:0/365455314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66680acc00 msgr2=0x7f66680c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f666ceb5640 1 --2- 172.21.15.26:0/365455314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66680acc00 0x7f66680c54a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6660009e50 tx=0x7f666002bf60 comp rx=0 tx=0).stop 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f666ceb5640 1 -- 172.21.15.26:0/365455314 shutdown_connections 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f666ceb5640 1 --2- 172.21.15.26:0/365455314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66680acc00 0x7f66680c54a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f666ceb5640 1 --2- 172.21.15.26:0/365455314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66681506b0 0x7f6668153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f666ceb5640 1 -- 172.21.15.26:0/365455314 >> 172.21.15.26:0/365455314 conn(0x7f666806b1a0 msgr2=0x7f666806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.351+0000 7f666ceb5640 1 -- 172.21.15.26:0/365455314 shutdown_connections 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.352+0000 7f666ceb5640 1 -- 172.21.15.26:0/365455314 wait complete. 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 Processor -- start 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 -- start start 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66680acc00 0x7f66680c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66681506b0 0x7f66681e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668156ea0 0x7f66681e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66680ad090 con 0x7f66681506b0 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668156a10 con 0x7f6668156ea0 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f666ceb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66680ad210 con 0x7f66680acc00 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.353+0000 7f665ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66681506b0 0x7f66681e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66681506b0 0x7f66681e89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41678/0 (socket says 172.21.15.26:41678) 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 -- 172.21.15.26:0/3660133616 learned_addr learned my addr 172.21.15.26:0/3660133616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f666659c640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66680acc00 0x7f66680c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f6666d9d640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668156ea0 0x7f66681e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 -- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66680acc00 msgr2=0x7f66680c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66680acc00 0x7f66680c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 -- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668156ea0 msgr2=0x7f66681e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668156ea0 0x7f66681e8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 -- 172.21.15.26:0/3660133616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66681ece20 con 0x7f66681506b0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f666659c640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66680acc00 0x7f66680c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f6666d9d640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668156ea0 0x7f66681e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665ffff640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66681506b0 0x7f66681e89e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f665000c470 tx=0x7f665000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665effd640 1 -- 172.21.15.26:0/3660133616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66500052d0 con 0x7f66681506b0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665effd640 1 -- 172.21.15.26:0/3660133616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6650005490 con 0x7f66681506b0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66681ecb40 con 0x7f66681506b0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f665effd640 1 -- 172.21.15.26:0/3660133616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650007260 con 0x7f66681506b0 2024-03-06T07:22:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.354+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6668152ca0 con 0x7f66681506b0 2024-03-06T07:22:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.355+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666804f330 con 0x7f66681506b0 2024-03-06T07:22:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.356+0000 7f665effd640 1 -- 172.21.15.26:0/3660133616 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6650007400 con 0x7f66681506b0 2024-03-06T07:22:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.356+0000 7f665effd640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.356+0000 7f665effd640 1 -- 172.21.15.26:0/3660133616 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f6650089650 con 0x7f66681506b0 2024-03-06T07:22:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.356+0000 7f666659c640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.357+0000 7f666659c640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f664806b4d0 0x7f664806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6658009b90 tx=0x7f6658008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.361+0000 7f665effd640 1 -- 172.21.15.26:0/3660133616 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6650058500 con 0x7f66681506b0 2024-03-06T07:22:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.619+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]} v 0) v1 -- 0x7f66680700a0 con 0x7f66681506b0 2024-03-06T07:22:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.972+0000 7f665effd640 1 -- 172.21.15.26:0/3660133616 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]}]=0 v152) v1 ==== 150+0+0 (secure 0 0 0) 0x7f6650057e00 con 0x7f66681506b0 2024-03-06T07:22:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f664806b4d0 msgr2=0x7f664806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f664806b4d0 0x7f664806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6658009b90 tx=0x7f6658008040 comp rx=0 tx=0).stop 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66681506b0 msgr2=0x7f66681e89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66681506b0 0x7f66681e89e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f665000c470 tx=0x7f665000c930 comp rx=0 tx=0).stop 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 shutdown_connections 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668156ea0 0x7f66681e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66681506b0 0x7f66681e89e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 --2- 172.21.15.26:0/3660133616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66680acc00 0x7f66680c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 >> 172.21.15.26:0/3660133616 conn(0x7f666806b1a0 msgr2=0x7f66680702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 shutdown_connections 2024-03-06T07:22:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:26.975+0000 7f666ceb5640 1 -- 172.21.15.26:0/3660133616 wait complete. 2024-03-06T07:22:26.989 INFO:tasks.cephfs_test_runner:That a standby with mds_join_fs set to another fs is still used if necessary. ... ok 2024-03-06T07:22:26.989 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_last_resort_refused (tasks.cephfs.test_failover.TestClusterAffinity) That a standby with mds_join_fs set to another fs is not used if refuse_standby_for_another_fs is set. 2024-03-06T07:22:26.989 INFO:tasks.cephfs_test_runner:test_join_fs_last_resort_refused (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:22:26.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.090+0000 7fb6aed65640 1 Processor -- start 2024-03-06T07:22:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.090+0000 7fb6aed65640 1 -- start start 2024-03-06T07:22:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6aed65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 0x7fb6a80c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6aed65640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6a81551e0 0x7fb6a8155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6aed65640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a8158800 con 0x7fb6a806d9f0 2024-03-06T07:22:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6aed65640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a8158ae0 con 0x7fb6a81551e0 2024-03-06T07:22:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6aed65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8158dc0 con 0x7fb6a80c3280 2024-03-06T07:22:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6acb01640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb6a806d9f0 0x7fb6a80a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42692/0 (socket says 172.21.15.26:42692) 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6acb01640 1 -- 172.21.15.26:0/3738794910 learned_addr learned my addr 172.21.15.26:0/3738794910 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a7fff640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 0x7fb6a80c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6ad302640 1 --2- 172.21.15.26:0/3738794910 >> v2:172.21.15.117:3300/0 conn(0x7fb6a81551e0 0x7fb6a8155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908271006 0 0) 0x7fb6a8158800 con 0x7fb6a806d9f0 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb690003520 con 0x7fb6a806d9f0 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1029841048 0 0) 0x7fb690003520 con 0x7fb6a806d9f0 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 msgr2=0x7fb6a80c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 0x7fb6a80c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 >> v2:172.21.15.117:3300/0 conn(0x7fb6a81551e0 msgr2=0x7fb6a8155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 --2- 172.21.15.26:0/3738794910 >> v2:172.21.15.117:3300/0 conn(0x7fb6a81551e0 0x7fb6a8155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a81590a0 con 0x7fb6a806d9f0 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.091+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fb6a00019f0 con 0x7fb6a806d9f0 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 >> v1:172.21.15.26:6789/0 conn(0x7fb6a806d9f0 legacy=0x7fb6a80a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a77fe640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6900039b0 0x7fb690003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a77fe640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb690004800 0x7fb690004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a77fe640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 0x7fb6900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8158ae0 con 0x7fb6900039b0 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8158dc0 con 0x7fb690004800 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69000ba10 con 0x7fb6a80c3280 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6acb01640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6900039b0 0x7fb690003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6a7fff640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb690004800 0x7fb690004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6ad302640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 0x7fb6900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6acb01640 1 -- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 msgr2=0x7fb6900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6acb01640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 0x7fb6900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6acb01640 1 -- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb690004800 msgr2=0x7fb690004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6acb01640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb690004800 0x7fb690004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6acb01640 1 -- 172.21.15.26:0/3738794910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690003520 con 0x7fb6900039b0 2024-03-06T07:22:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.092+0000 7fb6ad302640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80c3280 0x7fb6900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6acb01640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6900039b0 0x7fb690003dd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000a7d0 tx=0x7fb690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bd76e8649db4c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000a3f0 con 0x7fb6900039b0 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0034630 con 0x7fb6900039b0 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6a77fe640 1 -- 172.21.15.26:0/3738794910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0034a40 con 0x7fb6900039b0 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6aed65640 1 -- 172.21.15.26:0/3738794910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6900039b0 msgr2=0x7fb690003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6aed65640 1 --2- 172.21.15.26:0/3738794910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6900039b0 0x7fb690003dd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000a7d0 tx=0x7fb690005830 comp rx=0 tx=0).stop 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6ad302640 1 -- 172.21.15.26:0/3738794910 reap_dead start 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6aed65640 1 -- 172.21.15.26:0/3738794910 shutdown_connections 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6aed65640 1 -- 172.21.15.26:0/3738794910 >> 172.21.15.26:0/3738794910 conn(0x7fb6a80aa630 msgr2=0x7fb6a80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6aed65640 1 -- 172.21.15.26:0/3738794910 shutdown_connections 2024-03-06T07:22:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.093+0000 7fb6aed65640 1 -- 172.21.15.26:0/3738794910 wait complete. 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.094+0000 7fb6aed65640 1 Processor -- start 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6aed65640 1 -- start start 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6aed65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6900039b0 0x7fb6a80701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6aed65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80ac2d0 0x7fb6a806f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6aed65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6a81449f0 0x7fb6a8144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6aed65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a806df50 con 0x7fb6a81449f0 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6aed65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a806ddd0 con 0x7fb6900039b0 2024-03-06T07:22:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6aed65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80c6910 con 0x7fb6a80ac2d0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6ad302640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6a81449f0 0x7fb6a8144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6ad302640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6a81449f0 0x7fb6a8144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48914/0 (socket says 172.21.15.26:48914) 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.095+0000 7fb6ad302640 1 -- 172.21.15.26:0/1126218583 learned_addr learned my addr 172.21.15.26:0/1126218583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6acb01640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6900039b0 0x7fb6a80701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6a7fff640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80ac2d0 0x7fb6a806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6ad302640 1 -- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80ac2d0 msgr2=0x7fb6a806f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6ad302640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80ac2d0 0x7fb6a806f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6ad302640 1 -- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6900039b0 msgr2=0x7fb6a80701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6ad302640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6900039b0 0x7fb6a80701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6ad302640 1 -- 172.21.15.26:0/1126218583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a81453b0 con 0x7fb6a81449f0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6ad302640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6a81449f0 0x7fb6a8144e70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80c44e0 tx=0x7fb698011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6a57fa640 1 -- 172.21.15.26:0/1126218583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698019020 con 0x7fb6a81449f0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6a57fa640 1 -- 172.21.15.26:0/1126218583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb698003610 con 0x7fb6a81449f0 2024-03-06T07:22:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a806f8c0 con 0x7fb6a81449f0 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6a57fa640 1 -- 172.21.15.26:0/1126218583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698003a00 con 0x7fb6a81449f0 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.096+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a80abaa0 con 0x7fb6a81449f0 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.097+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a804f330 con 0x7fb6a81449f0 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.098+0000 7fb6a57fa640 1 -- 172.21.15.26:0/1126218583 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb69801c020 con 0x7fb6a81449f0 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.098+0000 7fb6a57fa640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb69406b4d0 0x7fb69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.098+0000 7fb6a57fa640 1 -- 172.21.15.26:0/1126218583 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fb69808a160 con 0x7fb6a81449f0 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.098+0000 7fb6acb01640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb69406b4d0 0x7fb69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.099+0000 7fb6acb01640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb69406b4d0 0x7fb69406d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0034000 tx=0x7fb6a0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.103+0000 7fb6a57fa640 1 -- 172.21.15.26:0/1126218583 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb698023030 con 0x7fb6a81449f0 2024-03-06T07:22:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.284+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6a8073240 con 0x7fb6a81449f0 2024-03-06T07:22:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.285+0000 7fb6a57fa640 1 -- 172.21.15.26:0/1126218583 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13826 (secure 0 0 0) 0x7fb698058a40 con 0x7fb6a81449f0 2024-03-06T07:22:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:27.287 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:24.966708+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6835/357874491":"2024-03-07T07:22:21.959390+0000","172.21.15.26:6835/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.26:6833/2696722008":"2024-03-07T07:22:15.497069+0000","172.21.15.117:6836/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.117:6834/3928098986":"2024-03-07T07:22:16.501832+0000","172.21.15.26:6834/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6832/1383022235":"2024-03-07T07:22:07.641783+0000","172.21.15.26:6835/440323369":"2024-03-07T07:22:04.621666+0000","172.21.15.117:6837/357874491":"2024-03-07T07:22:21.959390+0000","172.21.15.26:6833/440323369":"2024-03-07T07:22:04.621666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:27.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.286+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb69406b4d0 msgr2=0x7fb69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb69406b4d0 0x7fb69406d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0034000 tx=0x7fb6a0009030 comp rx=0 tx=0).stop 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6a81449f0 msgr2=0x7fb6a8144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6a81449f0 0x7fb6a8144e70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80c44e0 tx=0x7fb698011920 comp rx=0 tx=0).stop 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 shutdown_connections 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb69406b4d0 0x7fb69406d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6a81449f0 0x7fb6a8144e70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6a80ac2d0 0x7fb6a806f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 --2- 172.21.15.26:0/1126218583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6900039b0 0x7fb6a80701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 >> 172.21.15.26:0/1126218583 conn(0x7fb6a80aa630 msgr2=0x7fb6a81555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 shutdown_connections 2024-03-06T07:22:27.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.287+0000 7fb6aed65640 1 -- 172.21.15.26:0/1126218583 wait complete. 2024-03-06T07:22:27.300 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:27.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.402+0000 7feff180e640 1 Processor -- start 2024-03-06T07:22:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.402+0000 7feff180e640 1 -- start start 2024-03-06T07:22:27.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7feff180e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fefec06ca70 0x7fefec14e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7feff180e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec0b0450 con 0x7fefec0b0070 2024-03-06T07:22:27.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7feff180e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec1560e0 con 0x7fefec155c30 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7feff180e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefec06c020 con 0x7fefec06ca70 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefeaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fefec0b0070 0x7fefec0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42702/0 (socket says 172.21.15.26:42702) 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefeaffd640 1 -- 172.21.15.26:0/1900334788 learned_addr learned my addr 172.21.15.26:0/1900334788 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256661276 0 0) 0x7fefdc001490 con 0x7fefec0b0070 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd8003520 con 0x7fefec0b0070 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/1900334788 >> v2:172.21.15.117:3300/0 conn(0x7fefec06ca70 0x7fefec14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3623567586 0 0) 0x7fefd8003520 con 0x7fefec0b0070 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 >> v2:172.21.15.117:3300/0 conn(0x7fefec06ca70 msgr2=0x7fefec14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefe9ffb640 1 --2- 172.21.15.26:0/1900334788 >> v2:172.21.15.117:3300/0 conn(0x7fefec06ca70 0x7fefec14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.403+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 >> v1:172.21.15.117:6789/0 conn(0x7fefec155c30 legacy=0x7fefec0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0c77f0 con 0x7fefec0b0070 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fefdc001bc0 con 0x7fefec0b0070 2024-03-06T07:22:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 >> v1:172.21.15.26:6789/0 conn(0x7fefec0b0070 legacy=0x7fefec0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80039b0 0x7fefd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8004800 0x7fefd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd80051c0 0x7fefd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec06c020 con 0x7fefd80051c0 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd800c160 con 0x7fefd8004800 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd800c360 con 0x7fefd80039b0 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd80051c0 0x7fefd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefea7fc640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8004800 0x7fefd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefeaffd640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80039b0 0x7fefd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefeb7fe640 1 -- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80039b0 msgr2=0x7fefd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80039b0 0x7fefd8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefeb7fe640 1 -- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8004800 msgr2=0x7fefd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8004800 0x7fefd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.404+0000 7fefeb7fe640 1 -- 172.21.15.26:0/1900334788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8003520 con 0x7fefd80051c0 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7fefeb7fe640 1 -- 172.21.15.26:0/1900334788 reap_dead start 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd80051c0 0x7fefd800b9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009ac0 tx=0x7fefd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c1b846214d7a5d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4009690 con 0x7fefd80051c0 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4004020 con 0x7fefd80051c0 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7fefe9ffb640 1 -- 172.21.15.26:0/1900334788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4004410 con 0x7fefd80051c0 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7feff180e640 1 -- 172.21.15.26:0/1900334788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd80051c0 msgr2=0x7fefd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7feff180e640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd80051c0 0x7fefd800b9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009ac0 tx=0x7fefd8005830 comp rx=0 tx=0).stop 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7feff180e640 1 -- 172.21.15.26:0/1900334788 shutdown_connections 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7feff180e640 1 --2- 172.21.15.26:0/1900334788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd80051c0 0x7fefd800b9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7feff180e640 1 -- 172.21.15.26:0/1900334788 >> 172.21.15.26:0/1900334788 conn(0x7fefec0ac590 msgr2=0x7fefec0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7feff180e640 1 -- 172.21.15.26:0/1900334788 shutdown_connections 2024-03-06T07:22:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.405+0000 7feff180e640 1 -- 172.21.15.26:0/1900334788 wait complete. 2024-03-06T07:22:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 Processor -- start 2024-03-06T07:22:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 -- start start 2024-03-06T07:22:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80051c0 0x7fefec0c1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefec151ef0 0x7fefec0bf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec158370 0x7fefec0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec06c550 con 0x7fefec158370 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec06c270 con 0x7fefec151ef0 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7feff180e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec06da80 con 0x7fefd80051c0 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.407+0000 7fefeb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec158370 0x7fefec0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec158370 0x7fefec0bea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48938/0 (socket says 172.21.15.26:48938) 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 -- 172.21.15.26:0/926917519 learned_addr learned my addr 172.21.15.26:0/926917519 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefea7fc640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefec151ef0 0x7fefec0bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeaffd640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80051c0 0x7fefec0c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 -- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80051c0 msgr2=0x7fefec0c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80051c0 0x7fefec0c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 -- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefec151ef0 msgr2=0x7fefec0bf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefec151ef0 0x7fefec0bf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 -- 172.21.15.26:0/926917519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec1eb530 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeaffd640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80051c0 0x7fefec0c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefeb7fe640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec158370 0x7fefec0bea90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fefe402c380 tx=0x7fefe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefe3fff640 1 -- 172.21.15.26:0/926917519 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe402cbd0 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefe3fff640 1 -- 172.21.15.26:0/926917519 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe402cd90 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec151c10 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7fefe3fff640 1 -- 172.21.15.26:0/926917519 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4003850 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.408+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec159550 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.409+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec04f330 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.410+0000 7fefe3fff640 1 -- 172.21.15.26:0/926917519 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefe40039f0 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.410+0000 7fefe3fff640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefcc06b500 0x7fefcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.410+0000 7fefe3fff640 1 -- 172.21.15.26:0/926917519 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fefe40aaf90 con 0x7fefec158370 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.410+0000 7fefeaffd640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefcc06b500 0x7fefcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.411+0000 7fefeaffd640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefcc06b500 0x7fefcc06d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fefec0bdaf0 tx=0x7fefdc006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.414+0000 7fefe3fff640 1 -- 172.21.15.26:0/926917519 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fefec1eb530 con 0x7fefec158370 2024-03-06T07:22:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.594+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fefec06d860 con 0x7fefec158370 2024-03-06T07:22:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.594+0000 7fefe3fff640 1 -- 172.21.15.26:0/926917519 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fefe4079e30 con 0x7fefec158370 2024-03-06T07:22:27.597 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:22:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefcc06b500 msgr2=0x7fefcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefcc06b500 0x7fefcc06d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fefec0bdaf0 tx=0x7fefdc006040 comp rx=0 tx=0).stop 2024-03-06T07:22:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec158370 msgr2=0x7fefec0bea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec158370 0x7fefec0bea90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fefe402c380 tx=0x7fefe4004730 comp rx=0 tx=0).stop 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 shutdown_connections 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefcc06b500 0x7fefcc06d9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec158370 0x7fefec0bea90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefec151ef0 0x7fefec0bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 --2- 172.21.15.26:0/926917519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefd80051c0 0x7fefec0c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 >> 172.21.15.26:0/926917519 conn(0x7fefec0ac590 msgr2=0x7fefec0c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 shutdown_connections 2024-03-06T07:22:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.596+0000 7feff180e640 1 -- 172.21.15.26:0/926917519 wait complete. 2024-03-06T07:22:27.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused' 2024-03-06T07:22:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.709+0000 7fa3c080e640 1 Processor -- start 2024-03-06T07:22:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.709+0000 7fa3c080e640 1 -- start start 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3c080e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3bc0c88c0 0x7fa3bc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3c080e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc158b60 0x7fa3bc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3c080e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc0c73d0 con 0x7fa3bc14f360 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3c080e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc0c8dd0 con 0x7fa3bc0c88c0 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3c080e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0be2e0 con 0x7fa3bc158b60 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3b9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3bc0c88c0 0x7fa3bc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3ba59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa3bc14f360 0x7fa3bc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42718/0 (socket says 172.21.15.26:42718) 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3ba59c640 1 -- 172.21.15.26:0/2094536901 learned_addr learned my addr 172.21.15.26:0/2094536901 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3bad9d640 1 --2- 172.21.15.26:0/2094536901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc158b60 0x7fa3bc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3b9d9b640 1 -- 172.21.15.26:0/2094536901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc158b60 msgr2=0x7fa3bc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.710+0000 7fa3b9d9b640 1 --2- 172.21.15.26:0/2094536901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc158b60 0x7fa3bc158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3b9d9b640 1 -- 172.21.15.26:0/2094536901 >> v1:172.21.15.26:6789/0 conn(0x7fa3bc14f360 legacy=0x7fa3bc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3b9d9b640 1 -- 172.21.15.26:0/2094536901 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0c7670 con 0x7fa3bc0c88c0 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3b959a640 1 -- 172.21.15.26:0/2094536901 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018405288 0 0) 0x7fa3bc0c73d0 con 0x7fa3bc14f360 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3bad9d640 1 --2- 172.21.15.26:0/2094536901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc158b60 0x7fa3bc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3b9d9b640 1 --2- 172.21.15.26:0/2094536901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc0c88c0 0x7fa3bc155c10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097a0 tx=0x7fa3a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabc553d1a128d23 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3b959a640 1 -- 172.21.15.26:0/2094536901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40315a0 con 0x7fa3bc0c88c0 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3b959a640 1 -- 172.21.15.26:0/2094536901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a4037070 con 0x7fa3bc0c88c0 2024-03-06T07:22:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.711+0000 7fa3b959a640 1 -- 172.21.15.26:0/2094536901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4031960 con 0x7fa3bc0c88c0 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 -- 172.21.15.26:0/2094536901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc0c88c0 msgr2=0x7fa3bc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 --2- 172.21.15.26:0/2094536901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc0c88c0 0x7fa3bc155c10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097a0 tx=0x7fa3a402b840 comp rx=0 tx=0).stop 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 -- 172.21.15.26:0/2094536901 shutdown_connections 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 --2- 172.21.15.26:0/2094536901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc0c88c0 0x7fa3bc155c10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 --2- 172.21.15.26:0/2094536901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc158b60 0x7fa3bc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 -- 172.21.15.26:0/2094536901 >> 172.21.15.26:0/2094536901 conn(0x7fa3bc0ac570 msgr2=0x7fa3bc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 -- 172.21.15.26:0/2094536901 shutdown_connections 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.712+0000 7fa3c080e640 1 -- 172.21.15.26:0/2094536901 wait complete. 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.713+0000 7fa3c080e640 1 Processor -- start 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.713+0000 7fa3c080e640 1 -- start start 2024-03-06T07:22:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3c080e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc0c88c0 0x7fa3bc157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3c080e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3bc14f360 0x7fa3bc151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3c080e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc158b60 0x7fa3bc151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3c080e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0c5a50 con 0x7fa3bc158b60 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3c080e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0c7c30 con 0x7fa3bc14f360 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3c080e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0c5bd0 con 0x7fa3bc0c88c0 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc158b60 0x7fa3bc151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc158b60 0x7fa3bc151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48954/0 (socket says 172.21.15.26:48954) 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 -- 172.21.15.26:0/1374935343 learned_addr learned my addr 172.21.15.26:0/1374935343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3b9d9b640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3bc14f360 0x7fa3bc151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3ba59c640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc0c88c0 0x7fa3bc157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 -- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc0c88c0 msgr2=0x7fa3bc157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc0c88c0 0x7fa3bc157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 -- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3bc14f360 msgr2=0x7fa3bc151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3bc14f360 0x7fa3bc151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.714+0000 7fa3bad9d640 1 -- 172.21.15.26:0/1374935343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc1eae50 con 0x7fa3bc158b60 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.715+0000 7fa3bad9d640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc158b60 0x7fa3bc151ca0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00cef0 tx=0x7fa3ac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.715+0000 7fa3b2ffd640 1 -- 172.21.15.26:0/1374935343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac01a070 con 0x7fa3bc158b60 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.715+0000 7fa3b2ffd640 1 -- 172.21.15.26:0/1374935343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac00d860 con 0x7fa3bc158b60 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.715+0000 7fa3b2ffd640 1 -- 172.21.15.26:0/1374935343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00dc50 con 0x7fa3bc158b60 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.715+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc1ee1b0 con 0x7fa3bc158b60 2024-03-06T07:22:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.715+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc1ee770 con 0x7fa3bc158b60 2024-03-06T07:22:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.716+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc04f330 con 0x7fa3bc158b60 2024-03-06T07:22:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.716+0000 7fa3b2ffd640 1 -- 172.21.15.26:0/1374935343 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3ac01d070 con 0x7fa3bc158b60 2024-03-06T07:22:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.717+0000 7fa3b2ffd640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa39c06b4d0 0x7fa39c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.717+0000 7fa3b2ffd640 1 -- 172.21.15.26:0/1374935343 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fa3ac08aa00 con 0x7fa3bc158b60 2024-03-06T07:22:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.717+0000 7fa3ba59c640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa39c06b4d0 0x7fa39c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.717+0000 7fa3ba59c640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa39c06b4d0 0x7fa39c06d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4007fd0 tx=0x7fa3b4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.721+0000 7fa3b2ffd640 1 -- 172.21.15.26:0/1374935343 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa3ac059310 con 0x7fa3bc158b60 2024-03-06T07:22:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:27.973+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]} v 0) v1 -- 0x7fa3bc06cda0 con 0x7fa3bc158b60 2024-03-06T07:22:28.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.978+0000 7fa3b2ffd640 1 -- 172.21.15.26:0/1374935343 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]}]=0 v154) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa3ac05d3b0 con 0x7fa3bc158b60 2024-03-06T07:22:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa39c06b4d0 msgr2=0x7fa39c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa39c06b4d0 0x7fa39c06d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4007fd0 tx=0x7fa3b4007de0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc158b60 msgr2=0x7fa3bc151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc158b60 0x7fa3bc151ca0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00cef0 tx=0x7fa3ac00a710 comp rx=0 tx=0).stop 2024-03-06T07:22:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 shutdown_connections 2024-03-06T07:22:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa39c06b4d0 0x7fa39c06d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3bc158b60 0x7fa3bc151ca0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3bc14f360 0x7fa3bc151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 --2- 172.21.15.26:0/1374935343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3bc0c88c0 0x7fa3bc157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 >> 172.21.15.26:0/1374935343 conn(0x7fa3bc0ac570 msgr2=0x7fa3bc158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 shutdown_connections 2024-03-06T07:22:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:28.981+0000 7fa3c080e640 1 -- 172.21.15.26:0/1374935343 wait complete. 2024-03-06T07:22:28.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:22:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.105+0000 7f49b39f6640 1 Processor -- start 2024-03-06T07:22:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b39f6640 1 -- start start 2024-03-06T07:22:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b39f6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49ac0b00d0 0x7f49ac0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b39f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 0x7f49ac0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b39f6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ac0b04b0 con 0x7f49ac06b8e0 2024-03-06T07:22:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b39f6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac156160 con 0x7f49ac0b00d0 2024-03-06T07:22:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b39f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0c5710 con 0x7f49ac0c7210 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b0f91640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f49ac06b8e0 0x7f49ac06bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42724/0 (socket says 172.21.15.26:42724) 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.106+0000 7f49b0f91640 1 -- 172.21.15.26:0/4017399237 learned_addr learned my addr 172.21.15.26:0/4017399237 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49b1f93640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 0x7f49ac0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49b1792640 1 --2- 172.21.15.26:0/4017399237 >> v2:172.21.15.117:3300/0 conn(0x7f49ac0b00d0 0x7f49ac0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190403050 0 0) 0x7f49ac0b04b0 con 0x7f49ac06b8e0 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4998003520 con 0x7f49ac06b8e0 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560769481 0 0) 0x7f4998003520 con 0x7f49ac06b8e0 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 msgr2=0x7f49ac0c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 0x7f49ac0c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 >> v2:172.21.15.117:3300/0 conn(0x7f49ac0b00d0 msgr2=0x7f49ac0c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 --2- 172.21.15.26:0/4017399237 >> v2:172.21.15.117:3300/0 conn(0x7f49ac0b00d0 0x7f49ac0c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac06d210 con 0x7f49ac06b8e0 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f499c001690 con 0x7f49ac06b8e0 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 >> v1:172.21.15.26:6789/0 conn(0x7f49ac06b8e0 legacy=0x7f49ac06bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980039b0 0x7f4998003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 0x7f4998004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4998004d40 0x7f49980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac156160 con 0x7f4998004d40 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0c5710 con 0x7f49980039b0 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499800b9c0 con 0x7f49ac0c7210 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49b1f93640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4998004d40 0x7f49980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49b1792640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980039b0 0x7f4998003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.107+0000 7f49b0f91640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 0x7f4998004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49b1f93640 1 -- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 msgr2=0x7f4998004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49b1f93640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac0c7210 0x7f4998004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49b1f93640 1 -- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980039b0 msgr2=0x7f4998003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49b1f93640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980039b0 0x7f4998003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49b1f93640 1 -- 172.21.15.26:0/4017399237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4998003520 con 0x7f4998004d40 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49b1792640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980039b0 0x7f4998003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49b1f93640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4998004d40 0x7f49980051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e170 tx=0x7f4998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2d017396c44b4e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.108+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4039070 con 0x7f4998004d40 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a402cbf0 con 0x7f4998004d40 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49a3fff640 1 -- 172.21.15.26:0/4017399237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4009c10 con 0x7f4998004d40 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49b39f6640 1 -- 172.21.15.26:0/4017399237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4998004d40 msgr2=0x7f49980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49b39f6640 1 --2- 172.21.15.26:0/4017399237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4998004d40 0x7f49980051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e170 tx=0x7f4998005830 comp rx=0 tx=0).stop 2024-03-06T07:22:29.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49b1f93640 1 -- 172.21.15.26:0/4017399237 reap_dead start 2024-03-06T07:22:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49b39f6640 1 -- 172.21.15.26:0/4017399237 shutdown_connections 2024-03-06T07:22:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49b39f6640 1 -- 172.21.15.26:0/4017399237 >> 172.21.15.26:0/4017399237 conn(0x7f49ac0ac810 msgr2=0x7f49ac0acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49b39f6640 1 -- 172.21.15.26:0/4017399237 shutdown_connections 2024-03-06T07:22:29.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.109+0000 7f49b39f6640 1 -- 172.21.15.26:0/4017399237 wait complete. 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.110+0000 7f49b39f6640 1 Processor -- start 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.110+0000 7f49b39f6640 1 -- start start 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b39f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49ac1ee3e0 0x7f49ac1ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b39f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac157390 0x7f49ac157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b39f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac1eac00 0x7f49ac1eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b39f6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0c5ba0 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b39f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac152c50 con 0x7f49ac1eac00 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b39f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0c5d20 con 0x7f49ac157390 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b1792640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49ac1ee3e0 0x7f49ac1ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b1792640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49ac1ee3e0 0x7f49ac1ee7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48972/0 (socket says 172.21.15.26:48972) 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b1792640 1 -- 172.21.15.26:0/3178157204 learned_addr learned my addr 172.21.15.26:0/3178157204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b0f91640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac157390 0x7f49ac157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.111+0000 7f49b1f93640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac1eac00 0x7f49ac1eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b1792640 1 -- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac157390 msgr2=0x7f49ac157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b1792640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac157390 0x7f49ac157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b1792640 1 -- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac1eac00 msgr2=0x7f49ac1eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b1792640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac1eac00 0x7f49ac1eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b1792640 1 -- 172.21.15.26:0/3178157204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac14fcb0 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b0f91640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac157390 0x7f49ac157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b1f93640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac1eac00 0x7f49ac1eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b1792640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49ac1ee3e0 0x7f49ac1ee7c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f49a8011cd0 tx=0x7f49a800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49a1ffb640 1 -- 172.21.15.26:0/3178157204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a800fd20 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49a1ffb640 1 -- 172.21.15.26:0/3178157204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8014440 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac1eeef0 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49a1ffb640 1 -- 172.21.15.26:0/3178157204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8014830 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.112+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac158ae0 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.113+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac04f330 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.114+0000 7f49a1ffb640 1 -- 172.21.15.26:0/3178157204 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49a8012040 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.114+0000 7f49a1ffb640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f499006b4d0 0x7f499006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.114+0000 7f49a1ffb640 1 -- 172.21.15.26:0/3178157204 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f49a808aab0 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.114+0000 7f49b0f91640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f499006b4d0 0x7f499006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.115+0000 7f49b0f91640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f499006b4d0 0x7f499006d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f49ac155760 tx=0x7f499c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.118+0000 7f49a1ffb640 1 -- 172.21.15.26:0/3178157204 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49a8019030 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.298+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f49ac158ce0 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.301+0000 7f49a1ffb640 1 -- 172.21.15.26:0/3178157204 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v23)=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f49a8059390 con 0x7f49ac1ee3e0 2024-03-06T07:22:29.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f499006b4d0 msgr2=0x7f499006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f499006b4d0 0x7f499006d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f49ac155760 tx=0x7f499c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:29.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49ac1ee3e0 msgr2=0x7f49ac1ee7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49ac1ee3e0 0x7f49ac1ee7c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f49a8011cd0 tx=0x7f49a800f610 comp rx=0 tx=0).stop 2024-03-06T07:22:29.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 shutdown_connections 2024-03-06T07:22:29.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f499006b4d0 0x7f499006d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac1eac00 0x7f49ac1eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49ac157390 0x7f49ac157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 --2- 172.21.15.26:0/3178157204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49ac1ee3e0 0x7f49ac1ee7c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 >> 172.21.15.26:0/3178157204 conn(0x7f49ac0ac810 msgr2=0x7f49ac0ab430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 shutdown_connections 2024-03-06T07:22:29.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.303+0000 7f49b39f6640 1 -- 172.21.15.26:0/3178157204 wait complete. 2024-03-06T07:22:29.316 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:22:29.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:29.358 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:29.414 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:22:29.415 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:22:29.415 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:29.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:22:29.430 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:22:29.430 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:29.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:22:29.447 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:22:29.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:29.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.569+0000 7f934fe29640 1 Processor -- start 2024-03-06T07:22:29.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.569+0000 7f934fe29640 1 -- start start 2024-03-06T07:22:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934fe29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f934814f790 0x7f93481529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934fe29640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f93480c1aa0 0x7f93480c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934fe29640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93480c28d0 con 0x7f93480c5c60 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934fe29640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93480c2680 con 0x7f934814f790 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934fe29640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93480c2bb0 con 0x7f93480c1aa0 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934dbc5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f934814f790 0x7f93481529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934dbc5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f934814f790 0x7f93481529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48986/0 (socket says 172.21.15.26:48986) 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934dbc5640 1 -- 172.21.15.26:0/970705854 learned_addr learned my addr 172.21.15.26:0/970705854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934dbc5640 1 -- 172.21.15.26:0/970705854 >> v2:172.21.15.117:3300/0 conn(0x7f93480c1aa0 msgr2=0x7f93480c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934d3c4640 1 --2- 172.21.15.26:0/970705854 >> v2:172.21.15.117:3300/0 conn(0x7f93480c1aa0 0x7f93480c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934dbc5640 1 --2- 172.21.15.26:0/970705854 >> v2:172.21.15.117:3300/0 conn(0x7f93480c1aa0 0x7f93480c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934dbc5640 1 -- 172.21.15.26:0/970705854 >> v1:172.21.15.117:6789/0 conn(0x7f93480c5c60 legacy=0x7f9348155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934dbc5640 1 -- 172.21.15.26:0/970705854 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480c2e90 con 0x7f934814f790 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.570+0000 7f934d3c4640 1 --2- 172.21.15.26:0/970705854 >> v2:172.21.15.117:3300/0 conn(0x7f93480c1aa0 0x7f93480c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.571+0000 7f934dbc5640 1 --2- 172.21.15.26:0/970705854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f934814f790 0x7f93481529d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9348070280 tx=0x7f934002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc5aeabf7894335 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.571+0000 7f934cbc3640 1 -- 172.21.15.26:0/970705854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340035020 con 0x7f934814f790 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.571+0000 7f934cbc3640 1 -- 172.21.15.26:0/970705854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340031870 con 0x7f934814f790 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.571+0000 7f934cbc3640 1 -- 172.21.15.26:0/970705854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031c40 con 0x7f934814f790 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 -- 172.21.15.26:0/970705854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f934814f790 msgr2=0x7f93481529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 --2- 172.21.15.26:0/970705854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f934814f790 0x7f93481529d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9348070280 tx=0x7f934002b910 comp rx=0 tx=0).stop 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 -- 172.21.15.26:0/970705854 shutdown_connections 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 --2- 172.21.15.26:0/970705854 >> v2:172.21.15.117:3300/0 conn(0x7f93480c1aa0 0x7f93480c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 --2- 172.21.15.26:0/970705854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f934814f790 0x7f93481529d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 -- 172.21.15.26:0/970705854 >> 172.21.15.26:0/970705854 conn(0x7f93480c6710 msgr2=0x7f93480c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 -- 172.21.15.26:0/970705854 shutdown_connections 2024-03-06T07:22:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.572+0000 7f934fe29640 1 -- 172.21.15.26:0/970705854 wait complete. 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.573+0000 7f934fe29640 1 Processor -- start 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.573+0000 7f934fe29640 1 -- start start 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934fe29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93480c1aa0 0x7f9348150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934fe29640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93480c5c60 0x7f93481475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934fe29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934814f790 0x7f93481549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934fe29640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934806cc00 con 0x7f93480c1aa0 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934fe29640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c0d60 con 0x7f934814f790 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934fe29640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934806cd80 con 0x7f93480c5c60 2024-03-06T07:22:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93480c1aa0 0x7f9348150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93480c1aa0 0x7f9348150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49000/0 (socket says 172.21.15.26:49000) 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 -- 172.21.15.26:0/647378100 learned_addr learned my addr 172.21.15.26:0/647378100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 -- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93480c5c60 msgr2=0x7f93481475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93480c5c60 0x7f93481475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 -- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934814f790 msgr2=0x7f93481549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934e3c6640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934814f790 0x7f93481549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934814f790 0x7f93481549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 -- 172.21.15.26:0/647378100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348151300 con 0x7f93480c1aa0 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934e3c6640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934814f790 0x7f93481549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f934dbc5640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93480c1aa0 0x7f9348150c90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.574+0000 7f93467fc640 1 -- 172.21.15.26:0/647378100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340040070 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.575+0000 7f93467fc640 1 -- 172.21.15.26:0/647378100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934003a420 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.575+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9348151570 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.575+0000 7f93467fc640 1 -- 172.21.15.26:0/647378100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340035040 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.575+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9348151b30 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.576+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934804f330 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.576+0000 7f93467fc640 1 -- 172.21.15.26:0/647378100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f934004b060 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.576+0000 7f93467fc640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f933406b500 0x7f933406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.577+0000 7f93467fc640 1 -- 172.21.15.26:0/647378100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f93400ac100 con 0x7f93480c1aa0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.577+0000 7f934d3c4640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f933406b500 0x7f933406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.577+0000 7f934d3c4640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f933406b500 0x7f933406d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f933c006fd0 tx=0x7f933c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.581+0000 7f93467fc640 1 -- 172.21.15.26:0/647378100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f934007afa0 con 0x7f93480c1aa0 2024-03-06T07:22:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.799+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93480c1f40 con 0x7f93480c1aa0 2024-03-06T07:22:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.800+0000 7f93467fc640 1 -- 172.21.15.26:0/647378100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f934007a8a0 con 0x7f93480c1aa0 2024-03-06T07:22:29.803 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:29.803 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1256717910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1256717910},{"type":"v1","addr":"172.21.15.117:6836","nonce":1256717910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":5570,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3917286845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3917286845},{"type":"v1","addr":"172.21.15.117:6837","nonce":3917286845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1777543557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1777543557},{"type":"v1","addr":"172.21.15.26:6834","nonce":1777543557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4276915425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4276915425},{"type":"v1","addr":"172.21.15.26:6835","nonce":4276915425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[]} 2024-03-06T07:22:29.803 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 51 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f933406b500 msgr2=0x7f933406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f933406b500 0x7f933406d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f933c006fd0 tx=0x7f933c008040 comp rx=0 tx=0).stop 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93480c1aa0 msgr2=0x7f9348150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93480c1aa0 0x7f9348150c90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 shutdown_connections 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f933406b500 0x7f933406d9c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934814f790 0x7f93481549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93480c5c60 0x7f93481475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 --2- 172.21.15.26:0/647378100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93480c1aa0 0x7f9348150c90 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 >> 172.21.15.26:0/647378100 conn(0x7f93480c6710 msgr2=0x7f9348155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 shutdown_connections 2024-03-06T07:22:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.802+0000 7f934fe29640 1 -- 172.21.15.26:0/647378100 wait complete. 2024-03-06T07:22:29.815 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:22:29.816 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:22:29.816 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:22:29.816 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:29.816 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:22:29.816 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:22:29.816 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:22:29.817 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:29.817 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:22:29.817 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:22:29.817 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:22:29.817 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:29.817 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:22:29.817 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:22:29.818 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:22:29.818 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:29.827 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:22:29.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:22:29.829 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:22:29.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:22:29.830 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:22:29.830 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:22:29.831 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:22:29.831 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:22:29.870 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:22:29.870 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:29.871 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:22:29.871 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:29.872 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:22:29.872 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:29.873 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:22:29.873 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:29.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:22:29.946 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:22:29.947 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:22:29.958 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:22:29.958 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:22:29.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.976+0000 7fa6e3839640 1 Processor -- start 2024-03-06T07:22:29.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.976+0000 7fa6e3839640 1 -- start start 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.976+0000 7fa6e3839640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e3839640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6dc0c64f0 0x7fa6dc0c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e3839640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc0c8e30 con 0x7fa6dc0c8950 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e3839640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc152ce0 con 0x7fa6dc0c64f0 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e3839640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0c9110 con 0x7fa6dc152900 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e15d5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e1dd6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6dc0c64f0 0x7fa6dc0c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e15d5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:53332/0 (socket says 172.21.15.26:53332) 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e15d5640 1 -- 172.21.15.26:0/2549409143 learned_addr learned my addr 172.21.15.26:0/2549409143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e15d5640 1 -- 172.21.15.26:0/2549409143 >> v2:172.21.15.117:3300/0 conn(0x7fa6dc0c64f0 msgr2=0x7fa6dc0c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e15d5640 1 --2- 172.21.15.26:0/2549409143 >> v2:172.21.15.117:3300/0 conn(0x7fa6dc0c64f0 0x7fa6dc0c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e15d5640 1 -- 172.21.15.26:0/2549409143 >> v1:172.21.15.117:6789/0 conn(0x7fa6dc0c8950 legacy=0x7fa6dc0bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e15d5640 1 -- 172.21.15.26:0/2549409143 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0c93f0 con 0x7fa6dc152900 2024-03-06T07:22:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6e1dd6640 1 --2- 172.21.15.26:0/2549409143 >> v2:172.21.15.117:3300/0 conn(0x7fa6dc0c64f0 0x7fa6dc0c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.977+0000 7fa6d3fff640 1 -- 172.21.15.26:0/2549409143 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982360341 0 0) 0x7fa6dc0c8e30 con 0x7fa6dc0c8950 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.978+0000 7fa6e15d5640 1 --2- 172.21.15.26:0/2549409143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc155240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009870 tx=0x7fa6d802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3364b30162f177c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.978+0000 7fa6d3fff640 1 -- 172.21.15.26:0/2549409143 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80317c0 con 0x7fa6dc152900 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.978+0000 7fa6d3fff640 1 -- 172.21.15.26:0/2549409143 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d8037070 con 0x7fa6dc152900 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.978+0000 7fa6d3fff640 1 -- 172.21.15.26:0/2549409143 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8031b80 con 0x7fa6dc152900 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 -- 172.21.15.26:0/2549409143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 msgr2=0x7fa6dc155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 --2- 172.21.15.26:0/2549409143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc155240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009870 tx=0x7fa6d802b860 comp rx=0 tx=0).stop 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 -- 172.21.15.26:0/2549409143 shutdown_connections 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 --2- 172.21.15.26:0/2549409143 >> v2:172.21.15.117:3300/0 conn(0x7fa6dc0c64f0 0x7fa6dc0c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 --2- 172.21.15.26:0/2549409143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc155240 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 -- 172.21.15.26:0/2549409143 >> 172.21.15.26:0/2549409143 conn(0x7fa6dc0ac860 msgr2=0x7fa6dc0acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 -- 172.21.15.26:0/2549409143 shutdown_connections 2024-03-06T07:22:29.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.979+0000 7fa6e3839640 1 -- 172.21.15.26:0/2549409143 wait complete. 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.980+0000 7fa6e3839640 1 Processor -- start 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.980+0000 7fa6e3839640 1 -- start start 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e3839640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6dc0c64f0 0x7fa6dc148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e3839640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0c8950 0x7fa6dc0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e3839640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc1eb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e3839640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0c0180 con 0x7fa6dc0c8950 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e3839640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0c0000 con 0x7fa6dc0c64f0 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e3839640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0c55f0 con 0x7fa6dc152900 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e0dd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0c8950 0x7fa6dc0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e0dd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0c8950 0x7fa6dc0ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49038/0 (socket says 172.21.15.26:49038) 2024-03-06T07:22:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e15d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6dc0c64f0 0x7fa6dc148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.981+0000 7fa6e0dd4640 1 -- 172.21.15.26:0/3514706112 learned_addr learned my addr 172.21.15.26:0/3514706112 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e1dd6640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc1eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e0dd4640 1 -- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 msgr2=0x7fa6dc1eb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e0dd4640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc1eb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e0dd4640 1 -- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6dc0c64f0 msgr2=0x7fa6dc148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e0dd4640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6dc0c64f0 0x7fa6dc148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e0dd4640 1 -- 172.21.15.26:0/3514706112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc1ee730 con 0x7fa6dc0c8950 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e0dd4640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0c8950 0x7fa6dc0ab8b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c490 tx=0x7fa6cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6d1ffb640 1 -- 172.21.15.26:0/3514706112 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc005480 con 0x7fa6dc0c8950 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6d1ffb640 1 -- 172.21.15.26:0/3514706112 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc0035f0 con 0x7fa6dc0c8950 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6d1ffb640 1 -- 172.21.15.26:0/3514706112 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0039c0 con 0x7fa6dc0c8950 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc1eea10 con 0x7fa6dc0c8950 2024-03-06T07:22:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.982+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc1eefd0 con 0x7fa6dc0c8950 2024-03-06T07:22:29.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.983+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc04f330 con 0x7fa6dc0c8950 2024-03-06T07:22:29.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.983+0000 7fa6d1ffb640 1 -- 172.21.15.26:0/3514706112 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6cc003b60 con 0x7fa6dc0c8950 2024-03-06T07:22:29.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.984+0000 7fa6d1ffb640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6c006b670 0x7fa6c006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.984+0000 7fa6d1ffb640 1 -- 172.21.15.26:0/3514706112 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fa6cc089d30 con 0x7fa6dc0c8950 2024-03-06T07:22:29.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.984+0000 7fa6e15d5640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6c006b670 0x7fa6c006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.984+0000 7fa6e15d5640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6c006b670 0x7fa6c006db30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8036000 tx=0x7fa6d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:29.988+0000 7fa6d1ffb640 1 -- 172.21.15.26:0/3514706112 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa6cc058c60 con 0x7fa6dc0c8950 2024-03-06T07:22:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:30.185+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa6dc0ac460 con 0x7fa6dc0c8950 2024-03-06T07:22:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:30.999+0000 7fa6d1ffb640 1 -- 172.21.15.26:0/3514706112 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa6cc058560 con 0x7fa6dc0c8950 2024-03-06T07:22:31.002 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:22:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6c006b670 msgr2=0x7fa6c006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6c006b670 0x7fa6c006db30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8036000 tx=0x7fa6d8008030 comp rx=0 tx=0).stop 2024-03-06T07:22:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0c8950 msgr2=0x7fa6dc0ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0c8950 0x7fa6dc0ab8b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c490 tx=0x7fa6cc00c950 comp rx=0 tx=0).stop 2024-03-06T07:22:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 shutdown_connections 2024-03-06T07:22:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa6c006b670 0x7fa6c006db30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6dc152900 0x7fa6dc1eb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0c8950 0x7fa6dc0ab8b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 --2- 172.21.15.26:0/3514706112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6dc0c64f0 0x7fa6dc148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 >> 172.21.15.26:0/3514706112 conn(0x7fa6dc0ac860 msgr2=0x7fa6dc073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 shutdown_connections 2024-03-06T07:22:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.002+0000 7fa6e3839640 1 -- 172.21.15.26:0/3514706112 wait complete. 2024-03-06T07:22:31.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:22:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.123+0000 7f57285d6640 1 Processor -- start 2024-03-06T07:22:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.123+0000 7f57285d6640 1 -- start start 2024-03-06T07:22:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.124+0000 7f57285d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 0x7f5720155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.124+0000 7f57285d6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57200c73d0 con 0x7f5720158b60 2024-03-06T07:22:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.124+0000 7f57285d6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57200be2e0 con 0x7f57200c88c0 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.124+0000 7f57285d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5720159700 con 0x7f572014f360 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.124+0000 7f5726b73640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5720158b60 0x7f5720158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42738/0 (socket says 172.21.15.26:42738) 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.124+0000 7f5726b73640 1 -- 172.21.15.26:0/4117477657 learned_addr learned my addr 172.21.15.26:0/4117477657 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5726372640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 0x7f5720155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012248986 0 0) 0x7f57200c73d0 con 0x7f5720158b60 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5708003520 con 0x7f5720158b60 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2102281934 0 0) 0x7f5708003520 con 0x7f5720158b60 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 msgr2=0x7f5720155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 0x7f5720155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 >> v1:172.21.15.117:6789/0 conn(0x7f57200c88c0 legacy=0x7f57200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200c7670 con 0x7f5720158b60 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f57100019f0 con 0x7f5720158b60 2024-03-06T07:22:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 >> v1:172.21.15.26:6789/0 conn(0x7f5720158b60 legacy=0x7f5720158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57080039e0 0x7f5708003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5708004800 0x7f5708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 0x7f5708005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5720159700 con 0x7f5708004800 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570800bb50 con 0x7f57080039e0 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570800bd50 con 0x7f572014f360 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725b71640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5708004800 0x7f5708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5726372640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57080039e0 0x7f5708003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5726b73640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 0x7f5708005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725b71640 1 -- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 msgr2=0x7f5708005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725b71640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f572014f360 0x7f5708005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725b71640 1 -- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57080039e0 msgr2=0x7f5708003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725b71640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57080039e0 0x7f5708003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.125+0000 7f5725b71640 1 -- 172.21.15.26:0/4117477657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5708003520 con 0x7f5708004800 2024-03-06T07:22:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.126+0000 7f5725b71640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5708004800 0x7f5708004c80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5714001720 tx=0x7f5708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5780d22436ee8bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.126+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714001370 con 0x7f5708004800 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.126+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57140052d0 con 0x7f5708004800 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.126+0000 7f5725370640 1 -- 172.21.15.26:0/4117477657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714003830 con 0x7f5708004800 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.127+0000 7f57285d6640 1 -- 172.21.15.26:0/4117477657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5708004800 msgr2=0x7f5708004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.127+0000 7f57285d6640 1 --2- 172.21.15.26:0/4117477657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5708004800 0x7f5708004c80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5714001720 tx=0x7f5708005830 comp rx=0 tx=0).stop 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.127+0000 7f5726b73640 1 -- 172.21.15.26:0/4117477657 reap_dead start 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.127+0000 7f57285d6640 1 -- 172.21.15.26:0/4117477657 shutdown_connections 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.127+0000 7f57285d6640 1 -- 172.21.15.26:0/4117477657 >> 172.21.15.26:0/4117477657 conn(0x7f57200ac570 msgr2=0x7f57200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.127+0000 7f57285d6640 1 -- 172.21.15.26:0/4117477657 shutdown_connections 2024-03-06T07:22:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.127+0000 7f57285d6640 1 -- 172.21.15.26:0/4117477657 wait complete. 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.128+0000 7f57285d6640 1 Processor -- start 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.128+0000 7f57285d6640 1 -- start start 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f57285d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5708004800 0x7f5720157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f57285d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57201e0f50 0x7f57201eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f57285d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57201ef110 0x7f572014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f57285d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c7c30 con 0x7f57201e0f50 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f57285d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c5a50 con 0x7f57201ef110 2024-03-06T07:22:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f57285d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c5bd0 con 0x7f5708004800 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57201e0f50 0x7f57201eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57201e0f50 0x7f57201eebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49062/0 (socket says 172.21.15.26:49062) 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 -- 172.21.15.26:0/1247841865 learned_addr learned my addr 172.21.15.26:0/1247841865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5726372640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5708004800 0x7f5720157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5726b73640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57201ef110 0x7f572014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 -- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5708004800 msgr2=0x7f5720157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5708004800 0x7f5720157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 -- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57201ef110 msgr2=0x7f572014bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57201ef110 0x7f572014bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 -- 172.21.15.26:0/1247841865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200c3290 con 0x7f57201e0f50 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f5725b71640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57201e0f50 0x7f57201eebd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5714008fd0 tx=0x7f5714001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f571effd640 1 -- 172.21.15.26:0/1247841865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714041070 con 0x7f57201e0f50 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f571effd640 1 -- 172.21.15.26:0/1247841865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5714005480 con 0x7f57201e0f50 2024-03-06T07:22:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.129+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57201ef590 con 0x7f57201e0f50 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.130+0000 7f571effd640 1 -- 172.21.15.26:0/1247841865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714007a30 con 0x7f57201e0f50 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.130+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57201dfc20 con 0x7f57201e0f50 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.130+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57201dfa10 con 0x7f57201e0f50 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.131+0000 7f571effd640 1 -- 172.21.15.26:0/1247841865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5714044070 con 0x7f57201e0f50 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.131+0000 7f571effd640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f570c06b500 0x7f570c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.131+0000 7f571effd640 1 -- 172.21.15.26:0/1247841865 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f57140abbf0 con 0x7f57201e0f50 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.131+0000 7f5726372640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f570c06b500 0x7f570c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.132+0000 7f5726372640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f570c06b500 0x7f570c06d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5718009980 tx=0x7f5718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.136+0000 7f571effd640 1 -- 172.21.15.26:0/1247841865 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f571407ac30 con 0x7f57201e0f50 2024-03-06T07:22:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.384+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f57201dfdf0 con 0x7f57201e0f50 2024-03-06T07:22:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.385+0000 7f571effd640 1 -- 172.21.15.26:0/1247841865 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f57201dfdf0 con 0x7f57201e0f50 2024-03-06T07:22:31.387 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:22:31.389 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:22:31.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.391 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:22:31.393 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:22:31.395 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:22:31.396 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:22:31.399 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:22:31.400 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:22:31.400 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:22:31.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:22:31.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:31.400 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:22:31.400 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f570c06b500 msgr2=0x7f570c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f570c06b500 0x7f570c06d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5718009980 tx=0x7f5718008040 comp rx=0 tx=0).stop 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57201e0f50 msgr2=0x7f57201eebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57201e0f50 0x7f57201eebd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5714008fd0 tx=0x7f5714001120 comp rx=0 tx=0).stop 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 shutdown_connections 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f570c06b500 0x7f570c06d9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57201ef110 0x7f572014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57201e0f50 0x7f57201eebd0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 --2- 172.21.15.26:0/1247841865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5708004800 0x7f5720157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 >> 172.21.15.26:0/1247841865 conn(0x7f57200ac570 msgr2=0x7f57200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 shutdown_connections 2024-03-06T07:22:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.387+0000 7f57285d6640 1 -- 172.21.15.26:0/1247841865 wait complete. 2024-03-06T07:22:31.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:31.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.502+0000 7f6baacaf640 1 Processor -- start 2024-03-06T07:22:31.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.502+0000 7f6baacaf640 1 -- start start 2024-03-06T07:22:31.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6baacaf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ba40c3280 0x7f6ba40c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6baacaf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba4158800 con 0x7f6ba406d9f0 2024-03-06T07:22:31.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6baacaf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba4158ae0 con 0x7f6ba41551e0 2024-03-06T07:22:31.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6baacaf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba4158dc0 con 0x7f6ba40c3280 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6ba3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ba40c3280 0x7f6ba40c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6ba8a4b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6ba406d9f0 0x7f6ba40a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42752/0 (socket says 172.21.15.26:42752) 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6ba8a4b640 1 -- 172.21.15.26:0/556716451 learned_addr learned my addr 172.21.15.26:0/556716451 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6ba3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ba40c3280 0x7f6ba40c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49076/0 (socket says 172.21.15.26:49076) 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6ba3fff640 1 -- 172.21.15.26:0/556716451 >> v1:172.21.15.117:6789/0 conn(0x7f6ba41551e0 legacy=0x7f6ba41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6ba3fff640 1 -- 172.21.15.26:0/556716451 >> v1:172.21.15.26:6789/0 conn(0x7f6ba406d9f0 legacy=0x7f6ba40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.503+0000 7f6ba3fff640 1 -- 172.21.15.26:0/556716451 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba41590a0 con 0x7f6ba40c3280 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6ba3fff640 1 --2- 172.21.15.26:0/556716451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba40c3710 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6b900097a0 tx=0x7f6b9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce92b9134b21de85 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6ba37fe640 1 -- 172.21.15.26:0/556716451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b900315a0 con 0x7f6ba40c3280 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6ba37fe640 1 -- 172.21.15.26:0/556716451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b90037070 con 0x7f6ba40c3280 2024-03-06T07:22:31.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6ba37fe640 1 -- 172.21.15.26:0/556716451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90031960 con 0x7f6ba40c3280 2024-03-06T07:22:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6baacaf640 1 -- 172.21.15.26:0/556716451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 msgr2=0x7f6ba40c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6baacaf640 1 --2- 172.21.15.26:0/556716451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba40c3710 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6b900097a0 tx=0x7f6b9002b840 comp rx=0 tx=0).stop 2024-03-06T07:22:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6baacaf640 1 -- 172.21.15.26:0/556716451 shutdown_connections 2024-03-06T07:22:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.504+0000 7f6baacaf640 1 --2- 172.21.15.26:0/556716451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba40c3710 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.505+0000 7f6baacaf640 1 -- 172.21.15.26:0/556716451 >> 172.21.15.26:0/556716451 conn(0x7f6ba40aa630 msgr2=0x7f6ba40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.505+0000 7f6baacaf640 1 -- 172.21.15.26:0/556716451 shutdown_connections 2024-03-06T07:22:31.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.505+0000 7f6baacaf640 1 -- 172.21.15.26:0/556716451 wait complete. 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 Processor -- start 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 -- start start 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba406d9f0 0x7f6ba406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba4157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41551e0 0x7f6ba414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba406df50 con 0x7f6ba40c3280 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba406ddd0 con 0x7f6ba41551e0 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6baacaf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40c6910 con 0x7f6ba406d9f0 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba4157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.506+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba4157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49078/0 (socket says 172.21.15.26:49078) 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba3fff640 1 -- 172.21.15.26:0/1612021313 learned_addr learned my addr 172.21.15.26:0/1612021313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba8a4b640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba406d9f0 0x7f6ba406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba924c640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41551e0 0x7f6ba414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba3fff640 1 -- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba406d9f0 msgr2=0x7f6ba406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba3fff640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba406d9f0 0x7f6ba406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba3fff640 1 -- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41551e0 msgr2=0x7f6ba414ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba3fff640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41551e0 0x7f6ba414ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba3fff640 1 -- 172.21.15.26:0/1612021313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba406e610 con 0x7f6ba40c3280 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba924c640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41551e0 0x7f6ba414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba3fff640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba4157610 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000c00 tx=0x7f6b900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba17fa640 1 -- 172.21.15.26:0/1612021313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90032840 con 0x7f6ba40c3280 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba406e8b0 con 0x7f6ba40c3280 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba17fa640 1 -- 172.21.15.26:0/1612021313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b900329e0 con 0x7f6ba40c3280 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6ba17fa640 1 -- 172.21.15.26:0/1612021313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90032dd0 con 0x7f6ba40c3280 2024-03-06T07:22:31.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.507+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba406ee70 con 0x7f6ba40c3280 2024-03-06T07:22:31.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.508+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba404f330 con 0x7f6ba40c3280 2024-03-06T07:22:31.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.509+0000 7f6ba17fa640 1 -- 172.21.15.26:0/1612021313 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b9004d3f0 con 0x7f6ba40c3280 2024-03-06T07:22:31.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.509+0000 7f6ba17fa640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b8806b4d0 0x7f6b8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.509+0000 7f6ba17fa640 1 -- 172.21.15.26:0/1612021313 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6b9002f030 con 0x7f6ba40c3280 2024-03-06T07:22:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.509+0000 7f6ba8a4b640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b8806b4d0 0x7f6b8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.510+0000 7f6ba8a4b640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b8806b4d0 0x7f6b8806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000f60 tx=0x7f6b98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.513+0000 7f6ba17fa640 1 -- 172.21.15.26:0/1612021313 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6b90040070 con 0x7f6ba40c3280 2024-03-06T07:22:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.698+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ba4072b30 con 0x7f6ba40c3280 2024-03-06T07:22:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.699+0000 7f6ba17fa640 1 -- 172.21.15.26:0/1612021313 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+13177 (secure 0 0 0) 0x7f6b9007a600 con 0x7f6ba40c3280 2024-03-06T07:22:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:31.701 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":70,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:30.999631+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b8806b4d0 msgr2=0x7f6b8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b8806b4d0 0x7f6b8806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000f60 tx=0x7f6b98009040 comp rx=0 tx=0).stop 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 msgr2=0x7f6ba4157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba4157610 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000c00 tx=0x7f6b900046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 shutdown_connections 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b8806b4d0 0x7f6b8806d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41551e0 0x7f6ba414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba40c3280 0x7f6ba4157610 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 --2- 172.21.15.26:0/1612021313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba406d9f0 0x7f6ba406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 >> 172.21.15.26:0/1612021313 conn(0x7f6ba40aa630 msgr2=0x7f6ba40c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 shutdown_connections 2024-03-06T07:22:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.701+0000 7f6baacaf640 1 -- 172.21.15.26:0/1612021313 wait complete. 2024-03-06T07:22:31.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:31.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.817+0000 7fb7929a2640 1 Processor -- start 2024-03-06T07:22:31.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.818+0000 7fb7929a2640 1 -- start start 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.818+0000 7fb7929a2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c152a20 0x7fb78c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.818+0000 7fb7929a2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb78c0c8d50 0x7fb78c06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.818+0000 7fb7929a2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c1552c0 con 0x7fb78c06afc0 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.818+0000 7fb7929a2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78c0c6d30 con 0x7fb78c0c8d50 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.818+0000 7fb7929a2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c155580 con 0x7fb78c152a20 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7919a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c152a20 0x7fb78c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7921a1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb78c0c8d50 0x7fb78c06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7921a1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb78c0c8d50 0x7fb78c06b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38846/0 (socket says 172.21.15.26:38846) 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb79119f640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb78c06afc0 0x7fb78c0c8640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:39240/0 (socket says 172.21.15.26:39240) 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7919a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c152a20 0x7fb78c1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:53400/0 (socket says 172.21.15.26:53400) 2024-03-06T07:22:31.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7921a1640 1 -- 172.21.15.26:0/3223656665 learned_addr learned my addr 172.21.15.26:0/3223656665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:31.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7921a1640 1 -- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c152a20 msgr2=0x7fb78c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7921a1640 1 --2- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c152a20 0x7fb78c1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7921a1640 1 -- 172.21.15.26:0/3223656665 >> v1:172.21.15.117:6789/0 conn(0x7fb78c06afc0 legacy=0x7fb78c0c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb79099e640 1 -- 172.21.15.26:0/3223656665 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792983629 0 0) 0x7fb78c1552c0 con 0x7fb78c06afc0 2024-03-06T07:22:31.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7921a1640 1 -- 172.21.15.26:0/3223656665 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c155860 con 0x7fb78c0c8d50 2024-03-06T07:22:31.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.819+0000 7fb7919a0640 1 --2- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c152a20 0x7fb78c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:31.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.820+0000 7fb7921a1640 1 --2- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c0c8d50 0x7fb78c06b420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb784009e50 tx=0x7fb78402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313a2fe18152f17a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.820+0000 7fb79099e640 1 -- 172.21.15.26:0/3223656665 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78402c9e0 con 0x7fb78c0c8d50 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.820+0000 7fb79099e640 1 -- 172.21.15.26:0/3223656665 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78402cb80 con 0x7fb78c0c8d50 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.820+0000 7fb79099e640 1 -- 172.21.15.26:0/3223656665 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7840343f0 con 0x7fb78c0c8d50 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 -- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c0c8d50 msgr2=0x7fb78c06b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 --2- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c0c8d50 0x7fb78c06b420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb784009e50 tx=0x7fb78402be40 comp rx=0 tx=0).stop 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 -- 172.21.15.26:0/3223656665 shutdown_connections 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 --2- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c0c8d50 0x7fb78c06b420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 --2- 172.21.15.26:0/3223656665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c152a20 0x7fb78c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 -- 172.21.15.26:0/3223656665 >> 172.21.15.26:0/3223656665 conn(0x7fb78c0ca740 msgr2=0x7fb78c0cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 -- 172.21.15.26:0/3223656665 shutdown_connections 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.821+0000 7fb7929a2640 1 -- 172.21.15.26:0/3223656665 wait complete. 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.822+0000 7fb7929a2640 1 Processor -- start 2024-03-06T07:22:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.822+0000 7fb7929a2640 1 -- start start 2024-03-06T07:22:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb78c06afc0 0x7fb78c1eea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c152a20 0x7fb78c0c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c0c1fc0 0x7fb78c1eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c156060 con 0x7fb78c06afc0 2024-03-06T07:22:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c152e00 con 0x7fb78c152a20 2024-03-06T07:22:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1eefc0 con 0x7fb78c0c1fc0 2024-03-06T07:22:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb78c06afc0 0x7fb78c1eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb78c06afc0 0x7fb78c1eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49092/0 (socket says 172.21.15.26:49092) 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 -- 172.21.15.26:0/1485699520 learned_addr learned my addr 172.21.15.26:0/1485699520 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7921a1640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c0c1fc0 0x7fb78c1eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb79119f640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c152a20 0x7fb78c0c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 -- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c0c1fc0 msgr2=0x7fb78c1eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c0c1fc0 0x7fb78c1eb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 -- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c152a20 msgr2=0x7fb78c0c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c152a20 0x7fb78c0c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 -- 172.21.15.26:0/1485699520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c1e0200 con 0x7fb78c06afc0 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7919a0640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb78c06afc0 0x7fb78c1eea80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb78800e990 tx=0x7fb78800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7827fc640 1 -- 172.21.15.26:0/1485699520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788017070 con 0x7fb78c06afc0 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7827fc640 1 -- 172.21.15.26:0/1485699520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78800c9e0 con 0x7fb78c06afc0 2024-03-06T07:22:31.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c1e04e0 con 0x7fb78c06afc0 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7827fc640 1 -- 172.21.15.26:0/1485699520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78800cdb0 con 0x7fb78c06afc0 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.823+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c1eab00 con 0x7fb78c06afc0 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.824+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c1e08d0 con 0x7fb78c06afc0 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.825+0000 7fb7827fc640 1 -- 172.21.15.26:0/1485699520 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb78801a070 con 0x7fb78c06afc0 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.825+0000 7fb7827fc640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb77006b500 0x7fb77006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.825+0000 7fb7827fc640 1 -- 172.21.15.26:0/1485699520 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb78808a1a0 con 0x7fb78c06afc0 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.825+0000 7fb79119f640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb77006b500 0x7fb77006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.826+0000 7fb79119f640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb77006b500 0x7fb77006d9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb78c158220 tx=0x7fb77c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:31.829+0000 7fb7827fc640 1 -- 172.21.15.26:0/1485699520 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb788058b80 con 0x7fb78c06afc0 2024-03-06T07:22:32.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.008+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb78c0684b0 con 0x7fb78c06afc0 2024-03-06T07:22:32.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.008+0000 7fb7827fc640 1 -- 172.21.15.26:0/1485699520 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb78805cc20 con 0x7fb78c06afc0 2024-03-06T07:22:32.011 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:22:32.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb77006b500 msgr2=0x7fb77006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb77006b500 0x7fb77006d9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb78c158220 tx=0x7fb77c007d50 comp rx=0 tx=0).stop 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb78c06afc0 msgr2=0x7fb78c1eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb78c06afc0 0x7fb78c1eea80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb78800e990 tx=0x7fb78800ee50 comp rx=0 tx=0).stop 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 shutdown_connections 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb77006b500 0x7fb77006d9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb78c0c1fc0 0x7fb78c1eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb78c152a20 0x7fb78c0c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 --2- 172.21.15.26:0/1485699520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb78c06afc0 0x7fb78c1eea80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 >> 172.21.15.26:0/1485699520 conn(0x7fb78c0ca740 msgr2=0x7fb78c0cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 shutdown_connections 2024-03-06T07:22:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.010+0000 7fb7929a2640 1 -- 172.21.15.26:0/1485699520 wait complete. 2024-03-06T07:22:32.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:22:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.122+0000 7fc320cdf640 1 Processor -- start 2024-03-06T07:22:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.122+0000 7fc320cdf640 1 -- start start 2024-03-06T07:22:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc320cdf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c1527b0 0x7fc31c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc320cdf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc31c0c88b0 0x7fc31c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc320cdf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31c0c7c60 con 0x7fc31c0c7180 2024-03-06T07:22:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc320cdf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31c159640 con 0x7fc31c0c88b0 2024-03-06T07:22:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc320cdf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c0c8060 con 0x7fc31c1527b0 2024-03-06T07:22:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc31ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc31c0c88b0 0x7fc31c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc31ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc31c0c88b0 0x7fc31c0c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49096/0 (socket says 172.21.15.26:49096) 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc31ad9d640 1 -- 172.21.15.26:0/2617613752 learned_addr learned my addr 172.21.15.26:0/2617613752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.123+0000 7fc31a59c640 1 --2- 172.21.15.26:0/2617613752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c1527b0 0x7fc31c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31ad9d640 1 -- 172.21.15.26:0/2617613752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c1527b0 msgr2=0x7fc31c159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31ad9d640 1 --2- 172.21.15.26:0/2617613752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c1527b0 0x7fc31c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31ad9d640 1 -- 172.21.15.26:0/2617613752 >> v1:172.21.15.117:6789/0 conn(0x7fc31c0c7180 legacy=0x7fc31c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31ad9d640 1 -- 172.21.15.26:0/2617613752 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c0c76a0 con 0x7fc31c0c88b0 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31ad9d640 1 --2- 172.21.15.26:0/2617613752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c88b0 0x7fc31c0c5330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc314009e50 tx=0x7fc31402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d64fefc2dc5e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31959a640 1 -- 172.21.15.26:0/2617613752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31402c9e0 con 0x7fc31c0c88b0 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31959a640 1 -- 172.21.15.26:0/2617613752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31402cb80 con 0x7fc31c0c88b0 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.124+0000 7fc31959a640 1 -- 172.21.15.26:0/2617613752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3140343f0 con 0x7fc31c0c88b0 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 -- 172.21.15.26:0/2617613752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c88b0 msgr2=0x7fc31c0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2617613752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c88b0 0x7fc31c0c5330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc314009e50 tx=0x7fc31402be40 comp rx=0 tx=0).stop 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 -- 172.21.15.26:0/2617613752 shutdown_connections 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2617613752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c88b0 0x7fc31c0c5330 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2617613752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c1527b0 0x7fc31c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 -- 172.21.15.26:0/2617613752 >> 172.21.15.26:0/2617613752 conn(0x7fc31c0ac560 msgr2=0x7fc31c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 -- 172.21.15.26:0/2617613752 shutdown_connections 2024-03-06T07:22:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.125+0000 7fc320cdf640 1 -- 172.21.15.26:0/2617613752 wait complete. 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.126+0000 7fc320cdf640 1 Processor -- start 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.126+0000 7fc320cdf640 1 -- start start 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc320cdf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c7180 0x7fc31c1ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc320cdf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c0c88b0 0x7fc31c0c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc320cdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31c1527b0 0x7fc31c0c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc320cdf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c155600 con 0x7fc31c0c7180 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc320cdf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c155320 con 0x7fc31c1527b0 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc320cdf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c0c8d60 con 0x7fc31c0c88b0 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c7180 0x7fc31c1ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c7180 0x7fc31c1ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49100/0 (socket says 172.21.15.26:49100) 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31a59c640 1 -- 172.21.15.26:0/2426288515 learned_addr learned my addr 172.21.15.26:0/2426288515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31ad9d640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31c1527b0 0x7fc31c0c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc319d9b640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c0c88b0 0x7fc31c0c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31a59c640 1 -- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c0c88b0 msgr2=0x7fc31c0c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31a59c640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c0c88b0 0x7fc31c0c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31a59c640 1 -- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31c1527b0 msgr2=0x7fc31c0c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.127+0000 7fc31a59c640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31c1527b0 0x7fc31c0c08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.128+0000 7fc31a59c640 1 -- 172.21.15.26:0/2426288515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c158500 con 0x7fc31c0c7180 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.128+0000 7fc31a59c640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c7180 0x7fc31c1ee010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c7f0 tx=0x7fc30c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.128+0000 7fc30affd640 1 -- 172.21.15.26:0/2426288515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c015070 con 0x7fc31c0c7180 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.128+0000 7fc30affd640 1 -- 172.21.15.26:0/2426288515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c004240 con 0x7fc31c0c7180 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.128+0000 7fc30affd640 1 -- 172.21.15.26:0/2426288515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c007620 con 0x7fc31c0c7180 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.128+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c1ee8e0 con 0x7fc31c0c7180 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.128+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c157fc0 con 0x7fc31c0c7180 2024-03-06T07:22:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.129+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31c04f330 con 0x7fc31c0c7180 2024-03-06T07:22:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.129+0000 7fc30affd640 1 -- 172.21.15.26:0/2426288515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc30c0079d0 con 0x7fc31c0c7180 2024-03-06T07:22:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.130+0000 7fc30affd640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.130+0000 7fc30affd640 1 -- 172.21.15.26:0/2426288515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc30c08a0a0 con 0x7fc31c0c7180 2024-03-06T07:22:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.130+0000 7fc319d9b640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.131+0000 7fc319d9b640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc31c1eeea0 tx=0x7fc310008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.134+0000 7fc30affd640 1 -- 172.21.15.26:0/2426288515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc30c0590e0 con 0x7fc31c0c7180 2024-03-06T07:22:32.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.315+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc31c06ce00 con 0x7fc31c0c7180 2024-03-06T07:22:32.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.315+0000 7fc30affd640 1 -- 172.21.15.26:0/2426288515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc30c0589e0 con 0x7fc31c0c7180 2024-03-06T07:22:32.318 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:32.318 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:22:32.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:32.318 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:22:32.318 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:22:32.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:32.319 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:22:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2fc06b4d0 msgr2=0x7fc2fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc31c1eeea0 tx=0x7fc310008040 comp rx=0 tx=0).stop 2024-03-06T07:22:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c7180 msgr2=0x7fc31c1ee010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c7180 0x7fc31c1ee010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c7f0 tx=0x7fc30c00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 shutdown_connections 2024-03-06T07:22:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31c1527b0 0x7fc31c0c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31c0c88b0 0x7fc31c0c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 --2- 172.21.15.26:0/2426288515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31c0c7180 0x7fc31c1ee010 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 >> 172.21.15.26:0/2426288515 conn(0x7fc31c0ac560 msgr2=0x7fc31c0c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 shutdown_connections 2024-03-06T07:22:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.317+0000 7fc320cdf640 1 -- 172.21.15.26:0/2426288515 wait complete. 2024-03-06T07:22:32.330 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:22:32.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:22:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.433+0000 7fd47d1a1640 1 Processor -- start 2024-03-06T07:22:32.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.433+0000 7fd47d1a1640 1 -- start start 2024-03-06T07:22:32.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.434+0000 7fd47d1a1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 0x7fd4781566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.434+0000 7fd47d1a1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4780c8f60 con 0x7fd4780c8980 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.434+0000 7fd47d1a1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4780c9240 con 0x7fd47814f180 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.434+0000 7fd47d1a1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd478156c00 con 0x7fd4781551d0 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.434+0000 7fd47659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd4780c8980 0x7fd47806c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42768/0 (socket says 172.21.15.26:42768) 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.434+0000 7fd47659c640 1 -- 172.21.15.26:0/1831060517 learned_addr learned my addr 172.21.15.26:0/1831060517 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.434+0000 7fd47759e640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 0x7fd4781566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546175336 0 0) 0x7fd4780c8f60 con 0x7fd4780c8980 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd460003520 con 0x7fd4780c8980 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953241115 0 0) 0x7fd4780c9240 con 0x7fd47814f180 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4780c8f60 con 0x7fd47814f180 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2408984432 0 0) 0x7fd460003520 con 0x7fd4780c8980 2024-03-06T07:22:32.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 msgr2=0x7fd4781566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 0x7fd4781566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 >> v1:172.21.15.117:6789/0 conn(0x7fd47814f180 legacy=0x7fd4780ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780c9520 con 0x7fd4780c8980 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd46c001670 con 0x7fd4780c8980 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 >> v1:172.21.15.26:6789/0 conn(0x7fd4780c8980 legacy=0x7fd47806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd460003b70 0x7fd460003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 0x7fd460004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd460004d40 0x7fd4600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478156c00 con 0x7fd460003b70 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46000ba70 con 0x7fd460004d40 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46000bc20 con 0x7fd4781551d0 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd476d9d640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd460003b70 0x7fd460003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd47659c640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 0x7fd460004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd47759e640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd460004d40 0x7fd4600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd476d9d640 1 -- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 msgr2=0x7fd460004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd476d9d640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781551d0 0x7fd460004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd476d9d640 1 -- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd460004d40 msgr2=0x7fd4600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd476d9d640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd460004d40 0x7fd4600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.435+0000 7fd476d9d640 1 -- 172.21.15.26:0/1831060517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460003520 con 0x7fd460003b70 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.436+0000 7fd476d9d640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd460003b70 0x7fd460003f70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd4680018d0 tx=0x7fd460005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaf604254f5f2e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.436+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468037070 con 0x7fd460003b70 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.436+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4680052d0 con 0x7fd460003b70 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.436+0000 7fd475d9b640 1 -- 172.21.15.26:0/1831060517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680014f0 con 0x7fd460003b70 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.437+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1831060517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd460003b70 msgr2=0x7fd460003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.437+0000 7fd47d1a1640 1 --2- 172.21.15.26:0/1831060517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd460003b70 0x7fd460003f70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd4680018d0 tx=0x7fd460005830 comp rx=0 tx=0).stop 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.437+0000 7fd47759e640 1 -- 172.21.15.26:0/1831060517 reap_dead start 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.437+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1831060517 shutdown_connections 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.437+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1831060517 >> 172.21.15.26:0/1831060517 conn(0x7fd478072d20 msgr2=0x7fd4780ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.437+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1831060517 shutdown_connections 2024-03-06T07:22:32.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.437+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1831060517 wait complete. 2024-03-06T07:22:32.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.438+0000 7fd47d1a1640 1 Processor -- start 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.438+0000 7fd47d1a1640 1 -- start start 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47d1a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd478158610 0x7fd478150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47d1a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4781de6c0 0x7fd4781deb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47d1a1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781df080 0x7fd4781edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47d1a1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47814f560 con 0x7fd478158610 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47d1a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780c9ae0 con 0x7fd4781de6c0 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47d1a1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47814f6e0 con 0x7fd4781df080 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd478158610 0x7fd478150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd478158610 0x7fd478150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49110/0 (socket says 172.21.15.26:49110) 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 -- 172.21.15.26:0/1373472885 learned_addr learned my addr 172.21.15.26:0/1373472885 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47759e640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781df080 0x7fd4781edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47659c640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4781de6c0 0x7fd4781deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 -- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781df080 msgr2=0x7fd4781edbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781df080 0x7fd4781edbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 -- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4781de6c0 msgr2=0x7fd4781deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4781de6c0 0x7fd4781deb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 -- 172.21.15.26:0/1373472885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478158f50 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47659c640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4781de6c0 0x7fd4781deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd476d9d640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd478158610 0x7fd478150270 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd4680018d0 tx=0x7fd4680012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd45f7fe640 1 -- 172.21.15.26:0/1373472885 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468042020 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd45f7fe640 1 -- 172.21.15.26:0/1373472885 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468038040 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4781507b0 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.439+0000 7fd45f7fe640 1 -- 172.21.15.26:0/1373472885 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468037070 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.440+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4781e13c0 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.440+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47804f330 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.441+0000 7fd45f7fe640 1 -- 172.21.15.26:0/1373472885 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd468045020 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.441+0000 7fd45f7fe640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd46406b500 0x7fd46406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.441+0000 7fd45f7fe640 1 -- 172.21.15.26:0/1373472885 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd4680ab8d0 con 0x7fd478158610 2024-03-06T07:22:32.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.441+0000 7fd47659c640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd46406b500 0x7fd46406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.442+0000 7fd47659c640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd46406b500 0x7fd46406d9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd47814ea10 tx=0x7fd46c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.446+0000 7fd45f7fe640 1 -- 172.21.15.26:0/1373472885 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd46804c030 con 0x7fd478158610 2024-03-06T07:22:32.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:32.625+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd4780afe70 con 0x7fd478158610 2024-03-06T07:22:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.010+0000 7fd45f7fe640 1 -- 172.21.15.26:0/1373472885 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd46807a900 con 0x7fd478158610 2024-03-06T07:22:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:22:33.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd46406b500 msgr2=0x7fd46406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd46406b500 0x7fd46406d9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd47814ea10 tx=0x7fd46c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:33.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd478158610 msgr2=0x7fd478150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd478158610 0x7fd478150270 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd4680018d0 tx=0x7fd4680012d0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 shutdown_connections 2024-03-06T07:22:33.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd46406b500 0x7fd46406d9c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4781df080 0x7fd4781edbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4781de6c0 0x7fd4781deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 --2- 172.21.15.26:0/1373472885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd478158610 0x7fd478150270 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 >> 172.21.15.26:0/1373472885 conn(0x7fd478072d20 msgr2=0x7fd478154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:33.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.012+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 shutdown_connections 2024-03-06T07:22:33.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.013+0000 7fd47d1a1640 1 -- 172.21.15.26:0/1373472885 wait complete. 2024-03-06T07:22:33.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:22:33.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 Processor -- start 2024-03-06T07:22:33.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 -- start start 2024-03-06T07:22:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c1529f0 0x7fd51c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd51c0c5370 0x7fd51c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd51c155530 0x7fd51c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c159710 con 0x7fd51c155530 2024-03-06T07:22:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0c68b0 con 0x7fd51c0c5370 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.136+0000 7fd522684640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c152dd0 con 0x7fd51c1529f0 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd51c155530 0x7fd51c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd51c155530 0x7fd51c1559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49116/0 (socket says 172.21.15.26:49116) 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 -- 172.21.15.26:0/1119892912 learned_addr learned my addr 172.21.15.26:0/1119892912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd51bfff640 1 --2- 172.21.15.26:0/1119892912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c1529f0 0x7fd51c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd51b7fe640 1 --2- 172.21.15.26:0/1119892912 >> v2:172.21.15.117:3300/0 conn(0x7fd51c0c5370 0x7fd51c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 -- 172.21.15.26:0/1119892912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c1529f0 msgr2=0x7fd51c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 --2- 172.21.15.26:0/1119892912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c1529f0 0x7fd51c0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 -- 172.21.15.26:0/1119892912 >> v2:172.21.15.117:3300/0 conn(0x7fd51c0c5370 msgr2=0x7fd51c0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 --2- 172.21.15.26:0/1119892912 >> v2:172.21.15.117:3300/0 conn(0x7fd51c0c5370 0x7fd51c0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd520c21640 1 -- 172.21.15.26:0/1119892912 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c158b00 con 0x7fd51c155530 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.137+0000 7fd51b7fe640 1 --2- 172.21.15.26:0/1119892912 >> v2:172.21.15.117:3300/0 conn(0x7fd51c0c5370 0x7fd51c0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd520c21640 1 --2- 172.21.15.26:0/1119892912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c155530 0x7fd51c1559c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0cbac0 tx=0x7fd51402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19acceb6d9bbc1e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd51affd640 1 -- 172.21.15.26:0/1119892912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51402c9e0 con 0x7fd51c155530 2024-03-06T07:22:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd51affd640 1 -- 172.21.15.26:0/1119892912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51402cb80 con 0x7fd51c155530 2024-03-06T07:22:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd51affd640 1 -- 172.21.15.26:0/1119892912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140343f0 con 0x7fd51c155530 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd522684640 1 -- 172.21.15.26:0/1119892912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c155530 msgr2=0x7fd51c1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd522684640 1 --2- 172.21.15.26:0/1119892912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c155530 0x7fd51c1559c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0cbac0 tx=0x7fd51402be40 comp rx=0 tx=0).stop 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd522684640 1 -- 172.21.15.26:0/1119892912 shutdown_connections 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd522684640 1 --2- 172.21.15.26:0/1119892912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c155530 0x7fd51c1559c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd522684640 1 --2- 172.21.15.26:0/1119892912 >> v2:172.21.15.117:3300/0 conn(0x7fd51c0c5370 0x7fd51c0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd522684640 1 --2- 172.21.15.26:0/1119892912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c1529f0 0x7fd51c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.138+0000 7fd522684640 1 -- 172.21.15.26:0/1119892912 >> 172.21.15.26:0/1119892912 conn(0x7fd51c0ac580 msgr2=0x7fd51c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.139+0000 7fd522684640 1 -- 172.21.15.26:0/1119892912 shutdown_connections 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.139+0000 7fd522684640 1 -- 172.21.15.26:0/1119892912 wait complete. 2024-03-06T07:22:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 Processor -- start 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 -- start start 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c0c5370 0x7fd51c1e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51c1529f0 0x7fd51c1eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c155530 0x7fd51c14e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1593a0 con 0x7fd51c0c5370 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1590c0 con 0x7fd51c1529f0 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd522684640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c158740 con 0x7fd51c155530 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.140+0000 7fd51bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c0c5370 0x7fd51c1e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c0c5370 0x7fd51c1e8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49120/0 (socket says 172.21.15.26:49120) 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 -- 172.21.15.26:0/3855261631 learned_addr learned my addr 172.21.15.26:0/3855261631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 -- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c155530 msgr2=0x7fd51c14e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c155530 0x7fd51c14e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 -- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51c1529f0 msgr2=0x7fd51c1eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51c1529f0 0x7fd51c1eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 -- 172.21.15.26:0/3855261631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c14efd0 con 0x7fd51c0c5370 2024-03-06T07:22:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd51bfff640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c0c5370 0x7fd51c1e8b60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00e9b0 tx=0x7fd50c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd518ff9640 1 -- 172.21.15.26:0/3855261631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c017070 con 0x7fd51c0c5370 2024-03-06T07:22:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd518ff9640 1 -- 172.21.15.26:0/3855261631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c00c9a0 con 0x7fd51c0c5370 2024-03-06T07:22:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd518ff9640 1 -- 172.21.15.26:0/3855261631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00cd90 con 0x7fd51c0c5370 2024-03-06T07:22:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c14f2b0 con 0x7fd51c0c5370 2024-03-06T07:22:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.141+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c14d940 con 0x7fd51c0c5370 2024-03-06T07:22:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.142+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c148f50 con 0x7fd51c0c5370 2024-03-06T07:22:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.143+0000 7fd518ff9640 1 -- 172.21.15.26:0/3855261631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd50c01a070 con 0x7fd51c0c5370 2024-03-06T07:22:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.143+0000 7fd518ff9640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd50806b4d0 0x7fd50806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.143+0000 7fd518ff9640 1 -- 172.21.15.26:0/3855261631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fd50c08a4b0 con 0x7fd51c0c5370 2024-03-06T07:22:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.143+0000 7fd51b7fe640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd50806b4d0 0x7fd50806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.144+0000 7fd51b7fe640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd50806b4d0 0x7fd50806d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd51c1508f0 tx=0x7fd510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:33.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.148+0000 7fd518ff9640 1 -- 172.21.15.26:0/3855261631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd50c059410 con 0x7fd51c0c5370 2024-03-06T07:22:33.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:33.330+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd51c04f330 con 0x7fd51c0c5370 2024-03-06T07:22:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.020+0000 7fd518ff9640 1 -- 172.21.15.26:0/3855261631 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd50c058d10 con 0x7fd51c0c5370 2024-03-06T07:22:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:22:34.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.023+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd50806b4d0 msgr2=0x7fd50806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.023+0000 7fd522684640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd50806b4d0 0x7fd50806d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd51c1508f0 tx=0x7fd510008040 comp rx=0 tx=0).stop 2024-03-06T07:22:34.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.023+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c0c5370 msgr2=0x7fd51c1e8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.023+0000 7fd522684640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c0c5370 0x7fd51c1e8b60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00e9b0 tx=0x7fd50c00ee70 comp rx=0 tx=0).stop 2024-03-06T07:22:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.024+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 shutdown_connections 2024-03-06T07:22:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.024+0000 7fd522684640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd50806b4d0 0x7fd50806d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.024+0000 7fd522684640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd51c155530 0x7fd51c14e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.024+0000 7fd522684640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51c1529f0 0x7fd51c1eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.024+0000 7fd522684640 1 --2- 172.21.15.26:0/3855261631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd51c0c5370 0x7fd51c1e8b60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.024+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 >> 172.21.15.26:0/3855261631 conn(0x7fd51c0ac580 msgr2=0x7fd51c06c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.025+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 shutdown_connections 2024-03-06T07:22:34.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.025+0000 7fd522684640 1 -- 172.21.15.26:0/3855261631 wait complete. 2024-03-06T07:22:34.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:22:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.159+0000 7f9e039c9640 1 Processor -- start 2024-03-06T07:22:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.159+0000 7f9e039c9640 1 -- start start 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e039c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9dfc155b60 0x7f9dfc1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e039c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc1560b0 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e039c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc14f580 con 0x7f9dfc14f1a0 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e039c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c8dc0 con 0x7f9dfc155b60 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e01f66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9dfc155b60 0x7f9dfc1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e00f64640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9dfc0c88b0 0x7f9dfc158ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42770/0 (socket says 172.21.15.26:42770) 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e00f64640 1 -- 172.21.15.26:0/3493202248 learned_addr learned my addr 172.21.15.26:0/3493202248 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.160+0000 7f9e01f66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9dfc155b60 0x7f9dfc1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49134/0 (socket says 172.21.15.26:49134) 2024-03-06T07:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9e01f66640 1 -- 172.21.15.26:0/3493202248 >> v1:172.21.15.117:6789/0 conn(0x7f9dfc14f1a0 legacy=0x7f9dfc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9e01f66640 1 -- 172.21.15.26:0/3493202248 >> v1:172.21.15.26:6789/0 conn(0x7f9dfc0c88b0 legacy=0x7f9dfc158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9e01f66640 1 -- 172.21.15.26:0/3493202248 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0c78d0 con 0x7f9dfc155b60 2024-03-06T07:22:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9df3fff640 1 -- 172.21.15.26:0/3493202248 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857998773 0 0) 0x7f9dfc1560b0 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9e01f66640 1 --2- 172.21.15.26:0/3493202248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc155b60 0x7f9dfc1591c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009d90 tx=0x7f9df802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6155a128f474f67 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9df3fff640 1 -- 172.21.15.26:0/3493202248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8039070 con 0x7f9dfc155b60 2024-03-06T07:22:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9df3fff640 1 -- 172.21.15.26:0/3493202248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df802cbd0 con 0x7f9dfc155b60 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.161+0000 7f9df3fff640 1 -- 172.21.15.26:0/3493202248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8034440 con 0x7f9dfc155b60 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.162+0000 7f9e039c9640 1 -- 172.21.15.26:0/3493202248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc155b60 msgr2=0x7f9dfc1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.162+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3493202248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc155b60 0x7f9dfc1591c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009d90 tx=0x7f9df802be20 comp rx=0 tx=0).stop 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.162+0000 7f9e039c9640 1 -- 172.21.15.26:0/3493202248 shutdown_connections 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.162+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3493202248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc155b60 0x7f9dfc1591c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.162+0000 7f9e039c9640 1 -- 172.21.15.26:0/3493202248 >> 172.21.15.26:0/3493202248 conn(0x7f9dfc0728d0 msgr2=0x7f9dfc072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.162+0000 7f9e039c9640 1 -- 172.21.15.26:0/3493202248 shutdown_connections 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.162+0000 7f9e039c9640 1 -- 172.21.15.26:0/3493202248 wait complete. 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.163+0000 7f9e039c9640 1 Processor -- start 2024-03-06T07:22:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.163+0000 7f9e039c9640 1 -- start start 2024-03-06T07:22:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e039c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc0c88b0 0x7f9dfc1eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e039c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dfc14f1a0 0x7f9dfc0ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e039c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dfc155b60 0x7f9dfc0c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e039c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c7e90 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e039c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c8170 con 0x7f9dfc14f1a0 2024-03-06T07:22:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e039c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c5a40 con 0x7f9dfc155b60 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc0c88b0 0x7f9dfc1eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc0c88b0 0x7f9dfc1eb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49136/0 (socket says 172.21.15.26:49136) 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 -- 172.21.15.26:0/3776080561 learned_addr learned my addr 172.21.15.26:0/3776080561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01f66640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dfc155b60 0x7f9dfc0c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e00f64640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dfc14f1a0 0x7f9dfc0ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 -- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dfc155b60 msgr2=0x7f9dfc0c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dfc155b60 0x7f9dfc0c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 -- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dfc14f1a0 msgr2=0x7f9dfc0ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dfc14f1a0 0x7f9dfc0ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 -- 172.21.15.26:0/3776080561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0ade70 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e00f64640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dfc14f1a0 0x7f9dfc0ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01f66640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dfc155b60 0x7f9dfc0c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.164+0000 7f9e01765640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc0c88b0 0x7f9dfc1eb050 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9df400b3f0 tx=0x7f9df400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.165+0000 7f9df1ffb640 1 -- 172.21.15.26:0/3776080561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4015020 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.165+0000 7f9df1ffb640 1 -- 172.21.15.26:0/3776080561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4011830 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.165+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc0ae150 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.165+0000 7f9df1ffb640 1 -- 172.21.15.26:0/3776080561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4011c20 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.165+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc0ae710 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.166+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc158480 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.166+0000 7f9df1ffb640 1 -- 172.21.15.26:0/3776080561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9df4018020 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.166+0000 7f9df1ffb640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9de006b4d0 0x7f9de006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.167+0000 7f9df1ffb640 1 -- 172.21.15.26:0/3776080561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f9df4089f80 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.167+0000 7f9e00f64640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9de006b4d0 0x7f9de006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.167+0000 7f9e00f64640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9de006b4d0 0x7f9de006d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0cba70 tx=0x7f9dec007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.171+0000 7f9df1ffb640 1 -- 172.21.15.26:0/3776080561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9dfc158480 con 0x7f9dfc0c88b0 2024-03-06T07:22:34.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:34.393+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9dfc04f330 con 0x7f9dfc0c88b0 2024-03-06T07:22:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.025+0000 7f9df1ffb640 1 -- 172.21.15.26:0/3776080561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9df401f030 con 0x7f9dfc0c88b0 2024-03-06T07:22:35.028 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 11 and data pool 12 2024-03-06T07:22:35.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.031+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9de006b4d0 msgr2=0x7f9de006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.031+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9de006b4d0 0x7f9de006d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0cba70 tx=0x7f9dec007ca0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.031+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc0c88b0 msgr2=0x7f9dfc1eb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.031+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc0c88b0 0x7f9dfc1eb050 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9df400b3f0 tx=0x7f9df400b8b0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.033+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 shutdown_connections 2024-03-06T07:22:35.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.033+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9de006b4d0 0x7f9de006d990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.033+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dfc155b60 0x7f9dfc0c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.033+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dfc14f1a0 0x7f9dfc0ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.033+0000 7f9e039c9640 1 --2- 172.21.15.26:0/3776080561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dfc0c88b0 0x7f9dfc1eb050 secure :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9df400b3f0 tx=0x7f9df400b8b0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.033+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 >> 172.21.15.26:0/3776080561 conn(0x7f9dfc0728d0 msgr2=0x7f9dfc153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.039+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 shutdown_connections 2024-03-06T07:22:35.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.039+0000 7f9e039c9640 1 -- 172.21.15.26:0/3776080561 wait complete. 2024-03-06T07:22:35.060 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.163+0000 7f5ae83e2640 1 Processor -- start 2024-03-06T07:22:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.163+0000 7f5ae83e2640 1 -- start start 2024-03-06T07:22:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae83e2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5ae006d9f0 0x7f5ae00a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae83e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae83e2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ae01551e0 0x7f5ae0155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae83e2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae0158800 con 0x7f5ae01551e0 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae83e2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae0158ae0 con 0x7f5ae006d9f0 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae83e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0158dc0 con 0x7f5ae00c3280 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ae01551e0 0x7f5ae0155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ae01551e0 0x7f5ae0155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49150/0 (socket says 172.21.15.26:49150) 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 -- 172.21.15.26:0/2581182647 learned_addr learned my addr 172.21.15.26:0/2581182647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae617e640 1 --2- 172.21.15.26:0/2581182647 >> v2:172.21.15.117:3300/0 conn(0x7f5ae006d9f0 0x7f5ae00a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae597d640 1 --2- 172.21.15.26:0/2581182647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 -- 172.21.15.26:0/2581182647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 msgr2=0x7f5ae00c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 --2- 172.21.15.26:0/2581182647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 -- 172.21.15.26:0/2581182647 >> v2:172.21.15.117:3300/0 conn(0x7f5ae006d9f0 msgr2=0x7f5ae00a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 --2- 172.21.15.26:0/2581182647 >> v2:172.21.15.117:3300/0 conn(0x7f5ae006d9f0 0x7f5ae00a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae697f640 1 -- 172.21.15.26:0/2581182647 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae01590a0 con 0x7f5ae01551e0 2024-03-06T07:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.164+0000 7f5ae617e640 1 --2- 172.21.15.26:0/2581182647 >> v2:172.21.15.117:3300/0 conn(0x7f5ae006d9f0 0x7f5ae00a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.165+0000 7f5ae697f640 1 --2- 172.21.15.26:0/2581182647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae0155610 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00c9820 tx=0x7f5ad802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dff502d51e1a88 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.165+0000 7f5ae517c640 1 -- 172.21.15.26:0/2581182647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad802c9e0 con 0x7f5ae01551e0 2024-03-06T07:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.165+0000 7f5ae517c640 1 -- 172.21.15.26:0/2581182647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad802cb80 con 0x7f5ae01551e0 2024-03-06T07:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.165+0000 7f5ae517c640 1 -- 172.21.15.26:0/2581182647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80343f0 con 0x7f5ae01551e0 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2581182647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 msgr2=0x7f5ae0155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2581182647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae0155610 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00c9820 tx=0x7f5ad802be40 comp rx=0 tx=0).stop 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2581182647 shutdown_connections 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2581182647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2581182647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae0155610 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2581182647 >> v2:172.21.15.117:3300/0 conn(0x7f5ae006d9f0 0x7f5ae00a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2581182647 >> 172.21.15.26:0/2581182647 conn(0x7f5ae00aa630 msgr2=0x7f5ae00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2581182647 shutdown_connections 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.166+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2581182647 wait complete. 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.167+0000 7f5ae83e2640 1 Processor -- start 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.167+0000 7f5ae83e2640 1 -- start start 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae006d9f0 0x7f5ae014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae01ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae006df50 con 0x7f5ae01551e0 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae006ddd0 con 0x7f5ae006d9f0 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00a8210 con 0x7f5ae00c3280 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae01ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae01ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49152/0 (socket says 172.21.15.26:49152) 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 -- 172.21.15.26:0/2928840744 learned_addr learned my addr 172.21.15.26:0/2928840744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae617e640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae006d9f0 0x7f5ae014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae597d640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 -- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 msgr2=0x7f5ae00a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 -- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae006d9f0 msgr2=0x7f5ae014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae006d9f0 0x7f5ae014ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 -- 172.21.15.26:0/2928840744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae00ab9d0 con 0x7f5ae01551e0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae697f640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae01ee0c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5ad802c450 tx=0x7f5ad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ad6ffd640 1 -- 172.21.15.26:0/2928840744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8043020 con 0x7f5ae01551e0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ad6ffd640 1 -- 172.21.15.26:0/2928840744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad803e450 con 0x7f5ae01551e0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ad6ffd640 1 -- 172.21.15.26:0/2928840744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8009510 con 0x7f5ae01551e0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae00abcb0 con 0x7f5ae01551e0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.168+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae00ac270 con 0x7f5ae01551e0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.169+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae004f330 con 0x7f5ae01551e0 2024-03-06T07:22:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.170+0000 7f5ad6ffd640 1 -- 172.21.15.26:0/2928840744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ad8046020 con 0x7f5ae01551e0 2024-03-06T07:22:35.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.170+0000 7f5ad6ffd640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ac406b4d0 0x7f5ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.170+0000 7f5ad6ffd640 1 -- 172.21.15.26:0/2928840744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ad80ac940 con 0x7f5ae01551e0 2024-03-06T07:22:35.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.170+0000 7f5ae617e640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ac406b4d0 0x7f5ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.171+0000 7f5ae617e640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ac406b4d0 0x7f5ac406d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006fd0 tx=0x7f5adc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.175+0000 7f5ad6ffd640 1 -- 172.21.15.26:0/2928840744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ad804d030 con 0x7f5ae01551e0 2024-03-06T07:22:35.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.358+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae0073240 con 0x7f5ae01551e0 2024-03-06T07:22:35.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.360+0000 7f5ad6ffd640 1 -- 172.21.15.26:0/2928840744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16970 (secure 0 0 0) 0x7f5ad807b070 con 0x7f5ae01551e0 2024-03-06T07:22:35.362 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:35.363 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:34.395337+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ac406b4d0 msgr2=0x7f5ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ac406b4d0 0x7f5ac406d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006fd0 tx=0x7f5adc008040 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 msgr2=0x7f5ae01ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae01ee0c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5ad802c450 tx=0x7f5ad8004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 shutdown_connections 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ac406b4d0 0x7f5ac406d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae01551e0 0x7f5ae01ee0c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ae00c3280 0x7f5ae00a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 --2- 172.21.15.26:0/2928840744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae006d9f0 0x7f5ae014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 >> 172.21.15.26:0/2928840744 conn(0x7f5ae00aa630 msgr2=0x7f5ae01555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 shutdown_connections 2024-03-06T07:22:35.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.362+0000 7f5ae83e2640 1 -- 172.21.15.26:0/2928840744 wait complete. 2024-03-06T07:22:35.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:35.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdec2223640 1 Processor -- start 2024-03-06T07:22:35.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdec2223640 1 -- start start 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdec2223640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdebc06e520 0x7fdebc072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdec2223640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdebc071310 0x7fdebc14f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdec2223640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc0731f0 con 0x7fdebc155350 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdec2223640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc071c10 con 0x7fdebc071310 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdec2223640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc071ef0 con 0x7fdebc06e520 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.475+0000 7fdebbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdebc071310 0x7fdebc14f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdebc071310 0x7fdebc14f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49158/0 (socket says 172.21.15.26:49158) 2024-03-06T07:22:35.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdebc155350 0x7fdebc0c20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42776/0 (socket says 172.21.15.26:42776) 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebbfff640 1 -- 172.21.15.26:0/2711273713 learned_addr learned my addr 172.21.15.26:0/2711273713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebb7fe640 1 --2- 172.21.15.26:0/2711273713 >> v2:172.21.15.117:3300/0 conn(0x7fdebc06e520 0x7fdebc072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebbfff640 1 -- 172.21.15.26:0/2711273713 >> v2:172.21.15.117:3300/0 conn(0x7fdebc06e520 msgr2=0x7fdebc072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebbfff640 1 --2- 172.21.15.26:0/2711273713 >> v2:172.21.15.117:3300/0 conn(0x7fdebc06e520 0x7fdebc072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebbfff640 1 -- 172.21.15.26:0/2711273713 >> v1:172.21.15.26:6789/0 conn(0x7fdebc155350 legacy=0x7fdebc0c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdeba7fc640 1 -- 172.21.15.26:0/2711273713 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158801966 0 0) 0x7fdebc0731f0 con 0x7fdebc155350 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.476+0000 7fdebbfff640 1 -- 172.21.15.26:0/2711273713 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0721d0 con 0x7fdebc071310 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdebbfff640 1 --2- 172.21.15.26:0/2711273713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc071310 0x7fdebc14f120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdebc070e60 tx=0x7fdeb402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ecd67ddc3b4446 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdeba7fc640 1 -- 172.21.15.26:0/2711273713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb402ea90 con 0x7fdebc071310 2024-03-06T07:22:35.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdeba7fc640 1 -- 172.21.15.26:0/2711273713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb402ec30 con 0x7fdebc071310 2024-03-06T07:22:35.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdeba7fc640 1 -- 172.21.15.26:0/2711273713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4009600 con 0x7fdebc071310 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 -- 172.21.15.26:0/2711273713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc071310 msgr2=0x7fdebc14f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 --2- 172.21.15.26:0/2711273713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc071310 0x7fdebc14f120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdebc070e60 tx=0x7fdeb402e0f0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 -- 172.21.15.26:0/2711273713 shutdown_connections 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 --2- 172.21.15.26:0/2711273713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc071310 0x7fdebc14f120 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 --2- 172.21.15.26:0/2711273713 >> v2:172.21.15.117:3300/0 conn(0x7fdebc06e520 0x7fdebc072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 -- 172.21.15.26:0/2711273713 >> 172.21.15.26:0/2711273713 conn(0x7fdebc0c76b0 msgr2=0x7fdebc0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 -- 172.21.15.26:0/2711273713 shutdown_connections 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.477+0000 7fdec2223640 1 -- 172.21.15.26:0/2711273713 wait complete. 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.478+0000 7fdec2223640 1 Processor -- start 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdec2223640 1 -- start start 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdec2223640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc06e520 0x7fdebc149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdec2223640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc071310 0x7fdebc143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdec2223640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdebc155350 0x7fdebc143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdec2223640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc06e900 con 0x7fdebc06e520 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdec2223640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc072790 con 0x7fdebc071310 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdec2223640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc06ea80 con 0x7fdebc155350 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdebb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc06e520 0x7fdebc149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdebb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc06e520 0x7fdebc149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49160/0 (socket says 172.21.15.26:49160) 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdebb7fe640 1 -- 172.21.15.26:0/4062640786 learned_addr learned my addr 172.21.15.26:0/4062640786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdebaffd640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc071310 0x7fdebc143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.479+0000 7fdebbfff640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdebc155350 0x7fdebc143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdebb7fe640 1 -- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdebc155350 msgr2=0x7fdebc143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdebb7fe640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdebc155350 0x7fdebc143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdebb7fe640 1 -- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc071310 msgr2=0x7fdebc143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdebb7fe640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc071310 0x7fdebc143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdebb7fe640 1 -- 172.21.15.26:0/4062640786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc1e9bf0 con 0x7fdebc06e520 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdebaffd640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc071310 0x7fdebc143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdebb7fe640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc06e520 0x7fdebc149fd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00d830 tx=0x7fdeac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdea3fff640 1 -- 172.21.15.26:0/4062640786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00b6c0 con 0x7fdebc06e520 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdea3fff640 1 -- 172.21.15.26:0/4062640786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac019070 con 0x7fdebc06e520 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdea3fff640 1 -- 172.21.15.26:0/4062640786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00ba80 con 0x7fdebc06e520 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc1e9ed0 con 0x7fdebc06e520 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.480+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc1ea490 con 0x7fdebc06e520 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.481+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc04f330 con 0x7fdebc06e520 2024-03-06T07:22:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.481+0000 7fdea3fff640 1 -- 172.21.15.26:0/4062640786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdeac01e460 con 0x7fdebc06e520 2024-03-06T07:22:35.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.482+0000 7fdea3fff640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdea806b500 0x7fdea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.482+0000 7fdea3fff640 1 -- 172.21.15.26:0/4062640786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdeac08a9a0 con 0x7fdebc06e520 2024-03-06T07:22:35.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.482+0000 7fdebaffd640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdea806b500 0x7fdea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.482+0000 7fdebaffd640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdea806b500 0x7fdea806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0a8ec0 tx=0x7fdeb0007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.486+0000 7fdea3fff640 1 -- 172.21.15.26:0/4062640786 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdeac059720 con 0x7fdebc06e520 2024-03-06T07:22:35.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.665+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdebc146660 con 0x7fdebc06e520 2024-03-06T07:22:35.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.666+0000 7fdea3fff640 1 -- 172.21.15.26:0/4062640786 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16970 (secure 0 0 0) 0x7fdeac059020 con 0x7fdebc06e520 2024-03-06T07:22:35.669 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:35.669 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:34.395337+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:35.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdea806b500 msgr2=0x7fdea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdea806b500 0x7fdea806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0a8ec0 tx=0x7fdeb0007ca0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc06e520 msgr2=0x7fdebc149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc06e520 0x7fdebc149fd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00d830 tx=0x7fdeac00dcf0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 shutdown_connections 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdea806b500 0x7fdea806d9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdebc155350 0x7fdebc143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc071310 0x7fdebc143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 --2- 172.21.15.26:0/4062640786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdebc06e520 0x7fdebc149fd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.668+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 >> 172.21.15.26:0/4062640786 conn(0x7fdebc0c76b0 msgr2=0x7fdebc1558c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.669+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 shutdown_connections 2024-03-06T07:22:35.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.669+0000 7fdec2223640 1 -- 172.21.15.26:0/4062640786 wait complete. 2024-03-06T07:22:35.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:22:35.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.784+0000 7f08312c9640 1 Processor -- start 2024-03-06T07:22:35.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.784+0000 7f08312c9640 1 -- start start 2024-03-06T07:22:35.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f08312c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f082c0c5990 0x7f082c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f08312c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f082c0ab170 0x7f082c06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f08312c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c06cba0 con 0x7f082c06c730 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f08312c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c0c3ef0 con 0x7f082c0ab170 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f08312c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c1556d0 con 0x7f082c0c5990 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082a7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f082c0ab170 0x7f082c06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082a7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f082c0ab170 0x7f082c06c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49166/0 (socket says 172.21.15.26:49166) 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f082c06c730 0x7f082c1523f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42788/0 (socket says 172.21.15.26:42788) 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082a7fc640 1 -- 172.21.15.26:0/2120121824 learned_addr learned my addr 172.21.15.26:0/2120121824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082affd640 1 --2- 172.21.15.26:0/2120121824 >> v2:172.21.15.117:3300/0 conn(0x7f082c0c5990 0x7f082c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082a7fc640 1 -- 172.21.15.26:0/2120121824 >> v2:172.21.15.117:3300/0 conn(0x7f082c0c5990 msgr2=0x7f082c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082a7fc640 1 --2- 172.21.15.26:0/2120121824 >> v2:172.21.15.117:3300/0 conn(0x7f082c0c5990 0x7f082c0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082a7fc640 1 -- 172.21.15.26:0/2120121824 >> v1:172.21.15.26:6789/0 conn(0x7f082c06c730 legacy=0x7f082c1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.785+0000 7f082a7fc640 1 -- 172.21.15.26:0/2120121824 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c1559b0 con 0x7f082c0ab170 2024-03-06T07:22:35.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.786+0000 7f082a7fc640 1 --2- 172.21.15.26:0/2120121824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0ab170 0x7f082c06c1f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f082c0c12a0 tx=0x7f081402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf765dbe0a32b837 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.786+0000 7f0829ffb640 1 -- 172.21.15.26:0/2120121824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814034070 con 0x7f082c0ab170 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.786+0000 7f0829ffb640 1 -- 172.21.15.26:0/2120121824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814030860 con 0x7f082c0ab170 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.786+0000 7f0829ffb640 1 -- 172.21.15.26:0/2120121824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814030c30 con 0x7f082c0ab170 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 -- 172.21.15.26:0/2120121824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0ab170 msgr2=0x7f082c06c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 --2- 172.21.15.26:0/2120121824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0ab170 0x7f082c06c1f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f082c0c12a0 tx=0x7f081402b9a0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 -- 172.21.15.26:0/2120121824 shutdown_connections 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 --2- 172.21.15.26:0/2120121824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0ab170 0x7f082c06c1f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 --2- 172.21.15.26:0/2120121824 >> v2:172.21.15.117:3300/0 conn(0x7f082c0c5990 0x7f082c0c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 -- 172.21.15.26:0/2120121824 >> 172.21.15.26:0/2120121824 conn(0x7f082c0c6710 msgr2=0x7f082c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 -- 172.21.15.26:0/2120121824 shutdown_connections 2024-03-06T07:22:35.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.787+0000 7f08312c9640 1 -- 172.21.15.26:0/2120121824 wait complete. 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.788+0000 7f08312c9640 1 Processor -- start 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.788+0000 7f08312c9640 1 -- start start 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f08312c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c06c730 0x7f082c0a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f08312c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c0ab170 0x7f082c148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f08312c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0c5990 0x7f082c149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f08312c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c156490 con 0x7f082c0c5990 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f08312c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c155f70 con 0x7f082c06c730 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f08312c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c156610 con 0x7f082c0ab170 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0c5990 0x7f082c149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0c5990 0x7f082c149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49176/0 (socket says 172.21.15.26:49176) 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 -- 172.21.15.26:0/1483885280 learned_addr learned my addr 172.21.15.26:0/1483885280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082a7fc640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c0ab170 0x7f082c148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082affd640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c06c730 0x7f082c0a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 -- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c0ab170 msgr2=0x7f082c148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c0ab170 0x7f082c148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 -- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c06c730 msgr2=0x7f082c0a6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c06c730 0x7f082c0a6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 -- 172.21.15.26:0/1483885280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c149330 con 0x7f082c0c5990 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082affd640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c06c730 0x7f082c0a6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f082b7fe640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0c5990 0x7f082c149e00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f081c00c780 tx=0x7f081c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.789+0000 7f08237fe640 1 -- 172.21.15.26:0/1483885280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c017070 con 0x7f082c0c5990 2024-03-06T07:22:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.790+0000 7f08237fe640 1 -- 172.21.15.26:0/1483885280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c00a640 con 0x7f082c0c5990 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.790+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c151b10 con 0x7f082c0c5990 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.790+0000 7f08237fe640 1 -- 172.21.15.26:0/1483885280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c00aa30 con 0x7f082c0c5990 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.790+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c151030 con 0x7f082c0c5990 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.790+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c0a9740 con 0x7f082c0c5990 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.791+0000 7f08237fe640 1 -- 172.21.15.26:0/1483885280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f081c01a070 con 0x7f082c0c5990 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.791+0000 7f08237fe640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f080c06b4d0 0x7f080c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.792+0000 7f08237fe640 1 -- 172.21.15.26:0/1483885280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f081c0898a0 con 0x7f082c0c5990 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.792+0000 7f082affd640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f080c06b4d0 0x7f080c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.792+0000 7f082affd640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f080c06b4d0 0x7f080c06d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f082c1e8900 tx=0x7f0824008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:35.796+0000 7f08237fe640 1 -- 172.21.15.26:0/1483885280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f082c0a9740 con 0x7f082c0c5990 2024-03-06T07:22:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.014+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f082c04f330 con 0x7f082c0c5990 2024-03-06T07:22:36.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.026+0000 7f08237fe640 1 -- 172.21.15.26:0/1483885280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7f081c057f40 con 0x7f082c0c5990 2024-03-06T07:22:36.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.028+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f080c06b4d0 msgr2=0x7f080c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.028+0000 7f08312c9640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f080c06b4d0 0x7f080c06d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f082c1e8900 tx=0x7f0824008040 comp rx=0 tx=0).stop 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.028+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0c5990 msgr2=0x7f082c149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.028+0000 7f08312c9640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0c5990 0x7f082c149e00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f081c00c780 tx=0x7f081c00cc40 comp rx=0 tx=0).stop 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 shutdown_connections 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f080c06b4d0 0x7f080c06d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c0c5990 0x7f082c149e00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c0ab170 0x7f082c148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 --2- 172.21.15.26:0/1483885280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c06c730 0x7f082c0a6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 >> 172.21.15.26:0/1483885280 conn(0x7f082c0c6710 msgr2=0x7f082c0a9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 shutdown_connections 2024-03-06T07:22:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.029+0000 7f08312c9640 1 -- 172.21.15.26:0/1483885280 wait complete. 2024-03-06T07:22:36.043 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:22:36.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:22:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.145+0000 7f9460615640 1 Processor -- start 2024-03-06T07:22:36.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.145+0000 7f9460615640 1 -- start start 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f9460615640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94580c67f0 0x7f9458155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f9460615640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94580c4ef0 0x7f9458155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f9460615640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9458152cb0 con 0x7f94581528d0 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f9460615640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9458155ca0 con 0x7f94580c67f0 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f9460615640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580c38b0 con 0x7f94580c4ef0 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945dbb0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94580c67f0 0x7f9458155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945dbb0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94580c67f0 0x7f9458155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49186/0 (socket says 172.21.15.26:49186) 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945dbb0640 1 -- 172.21.15.26:0/180101365 learned_addr learned my addr 172.21.15.26:0/180101365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945ebb2640 1 --2- 172.21.15.26:0/180101365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94580c4ef0 0x7f9458155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945dbb0640 1 -- 172.21.15.26:0/180101365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94580c4ef0 msgr2=0x7f9458155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945dbb0640 1 --2- 172.21.15.26:0/180101365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94580c4ef0 0x7f9458155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945dbb0640 1 -- 172.21.15.26:0/180101365 >> v1:172.21.15.117:6789/0 conn(0x7f94581528d0 legacy=0x7f94580724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.146+0000 7f945dbb0640 1 -- 172.21.15.26:0/180101365 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94581594f0 con 0x7f94580c67f0 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.147+0000 7f945dbb0640 1 --2- 172.21.15.26:0/180101365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f9458155220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9448009870 tx=0x7f944802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9520b043d5a8242b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.147+0000 7f945d3af640 1 -- 172.21.15.26:0/180101365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944802f840 con 0x7f94580c67f0 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.147+0000 7f945d3af640 1 -- 172.21.15.26:0/180101365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944802f9e0 con 0x7f94580c67f0 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.147+0000 7f945d3af640 1 -- 172.21.15.26:0/180101365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944802fdb0 con 0x7f94580c67f0 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 -- 172.21.15.26:0/180101365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 msgr2=0x7f9458155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 --2- 172.21.15.26:0/180101365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f9458155220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9448009870 tx=0x7f944802ba20 comp rx=0 tx=0).stop 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 -- 172.21.15.26:0/180101365 shutdown_connections 2024-03-06T07:22:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 --2- 172.21.15.26:0/180101365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94580c4ef0 0x7f9458155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 --2- 172.21.15.26:0/180101365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f9458155220 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 -- 172.21.15.26:0/180101365 >> 172.21.15.26:0/180101365 conn(0x7f94580aa6a0 msgr2=0x7f94580aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 -- 172.21.15.26:0/180101365 shutdown_connections 2024-03-06T07:22:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.148+0000 7f9460615640 1 -- 172.21.15.26:0/180101365 wait complete. 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.149+0000 7f9460615640 1 Processor -- start 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.149+0000 7f9460615640 1 -- start start 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f9460615640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94580c4ef0 0x7f9458158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f9460615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f94581eabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f9460615640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94581528d0 0x7f94580702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f9460615640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580a8250 con 0x7f94580c67f0 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f9460615640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580c3390 con 0x7f94580c4ef0 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f9460615640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94581591a0 con 0x7f94581528d0 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f94581eabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f94581eabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49190/0 (socket says 172.21.15.26:49190) 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 -- 172.21.15.26:0/3520483828 learned_addr learned my addr 172.21.15.26:0/3520483828 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945e3b1640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94580c4ef0 0x7f9458158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945ebb2640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94581528d0 0x7f94580702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 -- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94581528d0 msgr2=0x7f94580702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94581528d0 0x7f94580702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 -- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94580c4ef0 msgr2=0x7f9458158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94580c4ef0 0x7f9458158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 -- 172.21.15.26:0/3520483828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945814b5b0 con 0x7f94580c67f0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.150+0000 7f945dbb0640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f94581eabe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9448000c00 tx=0x7f9448004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.151+0000 7f944effd640 1 -- 172.21.15.26:0/3520483828 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448037bc0 con 0x7f94580c67f0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.151+0000 7f944effd640 1 -- 172.21.15.26:0/3520483828 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448046070 con 0x7f94580c67f0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.151+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945814b850 con 0x7f94580c67f0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.151+0000 7f944effd640 1 -- 172.21.15.26:0/3520483828 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448041400 con 0x7f94580c67f0 2024-03-06T07:22:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.151+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94581ee4d0 con 0x7f94580c67f0 2024-03-06T07:22:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.152+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945804f330 con 0x7f94580c67f0 2024-03-06T07:22:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.152+0000 7f944effd640 1 -- 172.21.15.26:0/3520483828 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9448040070 con 0x7f94580c67f0 2024-03-06T07:22:36.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.152+0000 7f944effd640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f943c06b4d0 0x7f943c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.152+0000 7f944effd640 1 -- 172.21.15.26:0/3520483828 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94480ac830 con 0x7f94580c67f0 2024-03-06T07:22:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.152+0000 7f945e3b1640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f943c06b4d0 0x7f943c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.153+0000 7f945e3b1640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f943c06b4d0 0x7f943c06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9454001e70 tx=0x7f9454009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.157+0000 7f944effd640 1 -- 172.21.15.26:0/3520483828 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9448033030 con 0x7f94580c67f0 2024-03-06T07:22:36.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:36.379+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f94581ee6a0 con 0x7f94580c67f0 2024-03-06T07:22:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.028+0000 7f944effd640 1 -- 172.21.15.26:0/3520483828 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f944807af60 con 0x7f94580c67f0 2024-03-06T07:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f943c06b4d0 msgr2=0x7f943c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f943c06b4d0 0x7f943c06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9454001e70 tx=0x7f9454009040 comp rx=0 tx=0).stop 2024-03-06T07:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 msgr2=0x7f94581eabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f94581eabe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9448000c00 tx=0x7f9448004730 comp rx=0 tx=0).stop 2024-03-06T07:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 shutdown_connections 2024-03-06T07:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f943c06b4d0 0x7f943c06d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94581528d0 0x7f94580702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94580c67f0 0x7f94581eabe0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 --2- 172.21.15.26:0/3520483828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94580c4ef0 0x7f9458158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.030+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 >> 172.21.15.26:0/3520483828 conn(0x7f94580aa6a0 msgr2=0x7f945806cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.031+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 shutdown_connections 2024-03-06T07:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.031+0000 7f9460615640 1 -- 172.21.15.26:0/3520483828 wait complete. 2024-03-06T07:22:37.046 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:22:37.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.152+0000 7f2e92fa0640 1 Processor -- start 2024-03-06T07:22:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.152+0000 7f2e92fa0640 1 -- start start 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e92fa0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e8c14ba90 0x7f2e8c1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e92fa0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c14be70 con 0x7f2e8c158a40 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e92fa0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c0cad30 con 0x7f2e8c0c8690 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e92fa0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c0cb010 con 0x7f2e8c14ba90 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e91f9e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e8c14ba90 0x7f2e8c1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e9279f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2e8c158a40 0x7f2e8c158eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47114/0 (socket says 172.21.15.26:47114) 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e91f9e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e8c14ba90 0x7f2e8c1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34158/0 (socket says 172.21.15.26:34158) 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e9279f640 1 -- 172.21.15.26:0/2162151794 learned_addr learned my addr 172.21.15.26:0/2162151794 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e90f9c640 1 -- 172.21.15.26:0/2162151794 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273478998 0 0) 0x7f2e84001490 con 0x7f2e8c158a40 2024-03-06T07:22:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e90f9c640 1 -- 172.21.15.26:0/2162151794 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e78003520 con 0x7f2e8c158a40 2024-03-06T07:22:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e90f9c640 1 -- 172.21.15.26:0/2162151794 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130545359 0 0) 0x7f2e8c0cad30 con 0x7f2e8c0c8690 2024-03-06T07:22:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e90f9c640 1 -- 172.21.15.26:0/2162151794 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e84001490 con 0x7f2e8c0c8690 2024-03-06T07:22:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e91f9e640 1 -- 172.21.15.26:0/2162151794 >> v1:172.21.15.117:6789/0 conn(0x7f2e8c0c8690 legacy=0x7f2e8c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e91f9e640 1 -- 172.21.15.26:0/2162151794 >> v1:172.21.15.26:6789/0 conn(0x7f2e8c158a40 legacy=0x7f2e8c158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.153+0000 7f2e91f9e640 1 -- 172.21.15.26:0/2162151794 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0cb2f0 con 0x7f2e8c14ba90 2024-03-06T07:22:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.154+0000 7f2e91f9e640 1 --2- 172.21.15.26:0/2162151794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c1559b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2e880097d0 tx=0x7f2e8802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f39bf7213c2b38 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.154+0000 7f2e90f9c640 1 -- 172.21.15.26:0/2162151794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88034070 con 0x7f2e8c14ba90 2024-03-06T07:22:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.154+0000 7f2e90f9c640 1 -- 172.21.15.26:0/2162151794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88030610 con 0x7f2e8c14ba90 2024-03-06T07:22:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.154+0000 7f2e90f9c640 1 -- 172.21.15.26:0/2162151794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880309e0 con 0x7f2e8c14ba90 2024-03-06T07:22:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.155+0000 7f2e92fa0640 1 -- 172.21.15.26:0/2162151794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 msgr2=0x7f2e8c1559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.155+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/2162151794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c1559b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2e880097d0 tx=0x7f2e8802b960 comp rx=0 tx=0).stop 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.155+0000 7f2e92fa0640 1 -- 172.21.15.26:0/2162151794 shutdown_connections 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.155+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/2162151794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c1559b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.155+0000 7f2e92fa0640 1 -- 172.21.15.26:0/2162151794 >> 172.21.15.26:0/2162151794 conn(0x7f2e8c0728d0 msgr2=0x7f2e8c072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.155+0000 7f2e92fa0640 1 -- 172.21.15.26:0/2162151794 shutdown_connections 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.155+0000 7f2e92fa0640 1 -- 172.21.15.26:0/2162151794 wait complete. 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.156+0000 7f2e92fa0640 1 Processor -- start 2024-03-06T07:22:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.156+0000 7f2e92fa0640 1 -- start start 2024-03-06T07:22:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c0c8690 0x7f2e8c1492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c158a40 0x7f2e8c0aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0c5820 con 0x7f2e8c14ba90 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c06cf60 con 0x7f2e8c0c8690 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0c59a0 con 0x7f2e8c158a40 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c142b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34168/0 (socket says 172.21.15.26:34168) 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 -- 172.21.15.26:0/1301072903 learned_addr learned my addr 172.21.15.26:0/1301072903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e91f9e640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c0c8690 0x7f2e8c1492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9279f640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c158a40 0x7f2e8c0aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 -- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c158a40 msgr2=0x7f2e8c0aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c158a40 0x7f2e8c0aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 -- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c0c8690 msgr2=0x7f2e8c1492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c0c8690 0x7f2e8c1492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 -- 172.21.15.26:0/1301072903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0ac090 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9279f640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c158a40 0x7f2e8c0aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e9179d640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c142b60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c500 tx=0x7f2e7c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e827fc640 1 -- 172.21.15.26:0/1301072903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0052d0 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e827fc640 1 -- 172.21.15.26:0/1301072903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c005490 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e827fc640 1 -- 172.21.15.26:0/1301072903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c012870 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c14b000 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.157+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c1ea7c0 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.158+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c1ea2d0 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.159+0000 7f2e827fc640 1 -- 172.21.15.26:0/1301072903 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e7c012a10 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.159+0000 7f2e827fc640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e7006b4d0 0x7f2e7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.159+0000 7f2e827fc640 1 -- 172.21.15.26:0/1301072903 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e7c089ff0 con 0x7f2e8c14ba90 2024-03-06T07:22:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.159+0000 7f2e91f9e640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e7006b4d0 0x7f2e7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.160+0000 7f2e91f9e640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e7006b4d0 0x7f2e7006d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2e88036000 tx=0x7f2e8803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.164+0000 7f2e827fc640 1 -- 172.21.15.26:0/1301072903 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2e8c0ac090 con 0x7f2e8c14ba90 2024-03-06T07:22:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.384+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e8c06cd40 con 0x7f2e8c14ba90 2024-03-06T07:22:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.385+0000 7f2e827fc640 1 -- 172.21.15.26:0/1301072903 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4979 (secure 0 0 0) 0x7f2e7c058cf0 con 0x7f2e8c14ba90 2024-03-06T07:22:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:37.388 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:36.381549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6339":{"gid":6339,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/218606650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":218606650},{"type":"v1","addr":"172.21.15.117:6836","nonce":218606650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6339,"qdb_cluster":[6339]},"id":5}]} 2024-03-06T07:22:37.388 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 55 2024-03-06T07:22:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e7006b4d0 msgr2=0x7f2e7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e7006b4d0 0x7f2e7006d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2e88036000 tx=0x7f2e8803b040 comp rx=0 tx=0).stop 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 msgr2=0x7f2e8c142b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c142b60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c500 tx=0x7f2e7c00c9c0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 shutdown_connections 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e7006b4d0 0x7f2e7006d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c158a40 0x7f2e8c0aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e8c14ba90 0x7f2e8c142b60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 --2- 172.21.15.26:0/1301072903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c0c8690 0x7f2e8c1492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.386+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 >> 172.21.15.26:0/1301072903 conn(0x7f2e8c0728d0 msgr2=0x7f2e8c072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.387+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 shutdown_connections 2024-03-06T07:22:37.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.387+0000 7f2e92fa0640 1 -- 172.21.15.26:0/1301072903 wait complete. 2024-03-06T07:22:37.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:37.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.498+0000 7f94db6b5640 1 Processor -- start 2024-03-06T07:22:37.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94db6b5640 1 -- start start 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94db6b5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94d40c7180 0x7f94d4159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94db6b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c88b0 0x7f94d40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94db6b5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d40c7c60 con 0x7f94d41527b0 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94db6b5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d4159640 con 0x7f94d40c7180 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94db6b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40c8060 con 0x7f94d40c88b0 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94d8c50640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94d40c7180 0x7f94d4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94d8c50640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94d40c7180 0x7f94d4159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:54620/0 (socket says 172.21.15.26:54620) 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94d8c50640 1 -- 172.21.15.26:0/3687154195 learned_addr learned my addr 172.21.15.26:0/3687154195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94d9451640 1 --1- 172.21.15.26:0/3687154195 >> v1:172.21.15.117:6789/0 conn(0x7f94d41527b0 0x7f94d4155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:40146/0 (socket says 172.21.15.26:40146) 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.499+0000 7f94d9c52640 1 --2- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c88b0 0x7f94d40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.500+0000 7f94d8c50640 1 -- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c88b0 msgr2=0x7f94d40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.500+0000 7f94d8c50640 1 --2- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c88b0 0x7f94d40c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.500+0000 7f94d8c50640 1 -- 172.21.15.26:0/3687154195 >> v1:172.21.15.117:6789/0 conn(0x7f94d41527b0 legacy=0x7f94d4155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.500+0000 7f94d8c50640 1 -- 172.21.15.26:0/3687154195 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d40c76a0 con 0x7f94d40c7180 2024-03-06T07:22:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.500+0000 7f94cbfff640 1 -- 172.21.15.26:0/3687154195 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499837409 0 0) 0x7f94d0001110 con 0x7f94d41527b0 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94d8c50640 1 --2- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c7180 0x7f94d4159100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009870 tx=0x7f94c402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce9aa59adba1afc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94cbfff640 1 -- 172.21.15.26:0/3687154195 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4035020 con 0x7f94d40c7180 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94cbfff640 1 -- 172.21.15.26:0/3687154195 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c4031640 con 0x7f94d40c7180 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94cbfff640 1 -- 172.21.15.26:0/3687154195 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c402d040 con 0x7f94d40c7180 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 -- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c7180 msgr2=0x7f94d4159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c7180 0x7f94d4159100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009870 tx=0x7f94c402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 -- 172.21.15.26:0/3687154195 shutdown_connections 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c88b0 0x7f94d40c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3687154195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c7180 0x7f94d4159100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 -- 172.21.15.26:0/3687154195 >> 172.21.15.26:0/3687154195 conn(0x7f94d40ac560 msgr2=0x7f94d40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 -- 172.21.15.26:0/3687154195 shutdown_connections 2024-03-06T07:22:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.501+0000 7f94db6b5640 1 -- 172.21.15.26:0/3687154195 wait complete. 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.502+0000 7f94db6b5640 1 Processor -- start 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.502+0000 7f94db6b5640 1 -- start start 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94db6b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c7180 0x7f94d41ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94db6b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c88b0 0x7f94d41ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94db6b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d41527b0 0x7f94d41ec740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94db6b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4155600 con 0x7f94d41527b0 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94db6b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4155320 con 0x7f94d40c88b0 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94db6b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40c8d60 con 0x7f94d40c7180 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d9c52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d41527b0 0x7f94d41ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d9451640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c7180 0x7f94d41ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d8c50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c88b0 0x7f94d41ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d9c52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d41527b0 0x7f94d41ec740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34184/0 (socket says 172.21.15.26:34184) 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d9451640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c7180 0x7f94d41ee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:44542/0 (socket says 172.21.15.26:44542) 2024-03-06T07:22:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d9c52640 1 -- 172.21.15.26:0/3067636385 learned_addr learned my addr 172.21.15.26:0/3067636385 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d8c50640 1 -- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c7180 msgr2=0x7f94d41ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d8c50640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c7180 0x7f94d41ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d8c50640 1 -- 172.21.15.26:0/3067636385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d41527b0 msgr2=0x7f94d41ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d8c50640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d41527b0 0x7f94d41ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d8c50640 1 -- 172.21.15.26:0/3067636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d414f8d0 con 0x7f94d40c88b0 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.503+0000 7f94d9c52640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d41527b0 0x7f94d41ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.504+0000 7f94d8c50640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c88b0 0x7f94d41ee880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94d40cbab0 tx=0x7f94c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.504+0000 7f94c9ffb640 1 -- 172.21.15.26:0/3067636385 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4040070 con 0x7f94d40c88b0 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.504+0000 7f94c9ffb640 1 -- 172.21.15.26:0/3067636385 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c40399f0 con 0x7f94d40c88b0 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.504+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d41eef50 con 0x7f94d40c88b0 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.504+0000 7f94c9ffb640 1 -- 172.21.15.26:0/3067636385 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4035040 con 0x7f94d40c88b0 2024-03-06T07:22:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.504+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d40c0e60 con 0x7f94d40c88b0 2024-03-06T07:22:37.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.505+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d404f330 con 0x7f94d40c88b0 2024-03-06T07:22:37.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.505+0000 7f94c9ffb640 1 -- 172.21.15.26:0/3067636385 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94c404b060 con 0x7f94d40c88b0 2024-03-06T07:22:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.506+0000 7f94c9ffb640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94b806b4d0 0x7f94b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.506+0000 7f94c9ffb640 1 -- 172.21.15.26:0/3067636385 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94c40abaa0 con 0x7f94d40c88b0 2024-03-06T07:22:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.506+0000 7f94d9451640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94b806b4d0 0x7f94b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.506+0000 7f94d9451640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94b806b4d0 0x7f94b806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f94d0007fd0 tx=0x7f94d0007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.511+0000 7f94c9ffb640 1 -- 172.21.15.26:0/3067636385 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f94c407a0b0 con 0x7f94d40c88b0 2024-03-06T07:22:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.693+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94d4158c40 con 0x7f94d40c88b0 2024-03-06T07:22:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.695+0000 7f94c9ffb640 1 -- 172.21.15.26:0/3067636385 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16957 (secure 0 0 0) 0x7f94c407e150 con 0x7f94d40c88b0 2024-03-06T07:22:37.697 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:37.698 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":74,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:36.025110+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94b806b4d0 msgr2=0x7f94b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94b806b4d0 0x7f94b806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f94d0007fd0 tx=0x7f94d0007de0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c88b0 msgr2=0x7f94d41ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c88b0 0x7f94d41ee880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94d40cbab0 tx=0x7f94c40046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 shutdown_connections 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94b806b4d0 0x7f94b806d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d41527b0 0x7f94d41ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94d40c88b0 0x7f94d41ee880 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 --2- 172.21.15.26:0/3067636385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94d40c7180 0x7f94d41ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 >> 172.21.15.26:0/3067636385 conn(0x7f94d40ac560 msgr2=0x7f94d4158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 shutdown_connections 2024-03-06T07:22:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.697+0000 7f94db6b5640 1 -- 172.21.15.26:0/3067636385 wait complete. 2024-03-06T07:22:37.710 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:22:37.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.809+0000 7f90cc00f640 1 Processor -- start 2024-03-06T07:22:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.809+0000 7f90cc00f640 1 -- start start 2024-03-06T07:22:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90cc00f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c4150290 0x7f90c4150710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90cc00f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c4156f90 con 0x7f90c4072720 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90cc00f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c4157270 con 0x7f90c40c3d60 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90cc00f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40c38d0 con 0x7f90c4150290 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90c9dab640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f90c4072720 0x7f90c41564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47124/0 (socket says 172.21.15.26:47124) 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90c9dab640 1 -- 172.21.15.26:0/2403512954 learned_addr learned my addr 172.21.15.26:0/2403512954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90c95aa640 1 --2- 172.21.15.26:0/2403512954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c4150290 0x7f90c4150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.810+0000 7f90c8da9640 1 -- 172.21.15.26:0/2403512954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038483262 0 0) 0x7f90c4156f90 con 0x7f90c4072720 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.811+0000 7f90c8da9640 1 -- 172.21.15.26:0/2403512954 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90ac003520 con 0x7f90c4072720 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.811+0000 7f90c8da9640 1 -- 172.21.15.26:0/2403512954 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976281491 0 0) 0x7f90c4157270 con 0x7f90c40c3d60 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.811+0000 7f90c8da9640 1 -- 172.21.15.26:0/2403512954 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c4156f90 con 0x7f90c40c3d60 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.811+0000 7f90c95aa640 1 -- 172.21.15.26:0/2403512954 >> v1:172.21.15.117:6789/0 conn(0x7f90c40c3d60 legacy=0x7f90c40c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.811+0000 7f90c95aa640 1 -- 172.21.15.26:0/2403512954 >> v1:172.21.15.26:6789/0 conn(0x7f90c4072720 legacy=0x7f90c41564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.811+0000 7f90c95aa640 1 -- 172.21.15.26:0/2403512954 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40c3280 con 0x7f90c4150290 2024-03-06T07:22:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.811+0000 7f90c95aa640 1 --2- 172.21.15.26:0/2403512954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c4150290 0x7f90c4150710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009800 tx=0x7f90b802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec22db8a186ed83e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90c8da9640 1 -- 172.21.15.26:0/2403512954 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8035020 con 0x7f90c4150290 2024-03-06T07:22:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90c8da9640 1 -- 172.21.15.26:0/2403512954 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b8031800 con 0x7f90c4150290 2024-03-06T07:22:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90c8da9640 1 -- 172.21.15.26:0/2403512954 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8031bd0 con 0x7f90c4150290 2024-03-06T07:22:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90cc00f640 1 -- 172.21.15.26:0/2403512954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c4150290 msgr2=0x7f90c4150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90cc00f640 1 --2- 172.21.15.26:0/2403512954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c4150290 0x7f90c4150710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009800 tx=0x7f90b802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90cc00f640 1 -- 172.21.15.26:0/2403512954 shutdown_connections 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90cc00f640 1 --2- 172.21.15.26:0/2403512954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c4150290 0x7f90c4150710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90cc00f640 1 -- 172.21.15.26:0/2403512954 >> 172.21.15.26:0/2403512954 conn(0x7f90c40aa630 msgr2=0x7f90c40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90cc00f640 1 -- 172.21.15.26:0/2403512954 shutdown_connections 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.812+0000 7f90cc00f640 1 -- 172.21.15.26:0/2403512954 wait complete. 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.813+0000 7f90cc00f640 1 Processor -- start 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.813+0000 7f90cc00f640 1 -- start start 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90cc00f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90c4072720 0x7f90c40ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90cc00f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c40c3d60 0x7f90c41491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90cc00f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c4150290 0x7f90c40ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90cc00f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40c4900 con 0x7f90c4150290 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90cc00f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c406f8f0 con 0x7f90c4072720 2024-03-06T07:22:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90cc00f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c41e9480 con 0x7f90c40c3d60 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c4150290 0x7f90c40ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c4150290 0x7f90c40ab930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34188/0 (socket says 172.21.15.26:34188) 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 -- 172.21.15.26:0/3067962926 learned_addr learned my addr 172.21.15.26:0/3067962926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90c95aa640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c40c3d60 0x7f90c41491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90c9dab640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90c4072720 0x7f90c40ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 -- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c40c3d60 msgr2=0x7f90c41491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c40c3d60 0x7f90c41491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 -- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90c4072720 msgr2=0x7f90c40ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90c4072720 0x7f90c40ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 -- 172.21.15.26:0/3067962926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c41ece60 con 0x7f90c4150290 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.814+0000 7f90ca5ac640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c4150290 0x7f90c40ab930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f90b4002230 tx=0x7f90b400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.815+0000 7f90c27fc640 1 -- 172.21.15.26:0/3067962926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4003b70 con 0x7f90c4150290 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.815+0000 7f90c27fc640 1 -- 172.21.15.26:0/3067962926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4003d10 con 0x7f90c4150290 2024-03-06T07:22:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.815+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c41e8e30 con 0x7f90c4150290 2024-03-06T07:22:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.815+0000 7f90c27fc640 1 -- 172.21.15.26:0/3067962926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40052d0 con 0x7f90c4150290 2024-03-06T07:22:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.815+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c41e8b40 con 0x7f90c4150290 2024-03-06T07:22:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.816+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c40734c0 con 0x7f90c4150290 2024-03-06T07:22:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.816+0000 7f90c27fc640 1 -- 172.21.15.26:0/3067962926 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90b4005680 con 0x7f90c4150290 2024-03-06T07:22:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.816+0000 7f90c27fc640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.817+0000 7f90c27fc640 1 -- 172.21.15.26:0/3067962926 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90b408a400 con 0x7f90c4150290 2024-03-06T07:22:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.817+0000 7f90c9dab640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.817+0000 7f90c9dab640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f90b006b4d0 0x7f90b006d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000f60 tx=0x7f90bc007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.821+0000 7f90c27fc640 1 -- 172.21.15.26:0/3067962926 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90c40734c0 con 0x7f90c4150290 2024-03-06T07:22:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:37.998+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90c404f330 con 0x7f90b006b4d0 2024-03-06T07:22:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.005+0000 7f90c27fc640 1 -- 172.21.15.26:0/3067962926 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252074 (secure 0 0 0) 0x7f90c404f330 con 0x7f90b006b4d0 2024-03-06T07:22:38.013 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:38.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.007+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f90b006b4d0 msgr2=0x7f90b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.007+0000 7f90cc00f640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f90b006b4d0 0x7f90b006d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000f60 tx=0x7f90bc007d30 comp rx=0 tx=0).stop 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.007+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c4150290 msgr2=0x7f90c40ab930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.007+0000 7f90cc00f640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c4150290 0x7f90c40ab930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f90b4002230 tx=0x7f90b400cb90 comp rx=0 tx=0).stop 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 shutdown_connections 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90c4150290 0x7f90c40ab930 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90c40c3d60 0x7f90c41491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 --2- 172.21.15.26:0/3067962926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90c4072720 0x7f90c40ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 >> 172.21.15.26:0/3067962926 conn(0x7f90c40aa630 msgr2=0x7f90c4156710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 shutdown_connections 2024-03-06T07:22:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.008+0000 7f90cc00f640 1 -- 172.21.15.26:0/3067962926 wait complete. 2024-03-06T07:22:38.021 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-03-06T07:22:37.061963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239828,"kb_used_data":7008,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734892,"statfs":{"total":773094113280,"available":772848529408,"internally_reserved":0,"allocated":7176192,"data_stored":4489896,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.045000"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.031188+0000","last_change":"2024-03-06T07:22:35.025691+0000","last_active":"2024-03-06T07:22:35.031188+0000","last_peered":"2024-03-06T07:22:35.031188+0000","last_clean":"2024-03-06T07:22:35.031188+0000","last_became_active":"2024-03-06T07:22:35.025394+0000","last_became_peered":"2024-03-06T07:22:35.025394+0000","last_unstale":"2024-03-06T07:22:35.031188+0000","last_undegraded":"2024-03-06T07:22:35.031188+0000","last_fullsized":"2024-03-06T07:22:35.031188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.050056+0000","last_change":"2024-03-06T07:22:35.048775+0000","last_active":"2024-03-06T07:22:35.050056+0000","last_peered":"2024-03-06T07:22:35.050056+0000","last_clean":"2024-03-06T07:22:35.050056+0000","last_became_active":"2024-03-06T07:22:35.048476+0000","last_became_peered":"2024-03-06T07:22:35.048476+0000","last_unstale":"2024-03-06T07:22:35.050056+0000","last_undegraded":"2024-03-06T07:22:35.050056+0000","last_fullsized":"2024-03-06T07:22:35.050056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039858+0000","last_change":"2024-03-06T07:22:35.037727+0000","last_active":"2024-03-06T07:22:35.039858+0000","last_peered":"2024-03-06T07:22:35.039858+0000","last_clean":"2024-03-06T07:22:35.039858+0000","last_became_active":"2024-03-06T07:22:35.037421+0000","last_became_peered":"2024-03-06T07:22:35.037421+0000","last_unstale":"2024-03-06T07:22:35.039858+0000","last_undegraded":"2024-03-06T07:22:35.039858+0000","last_fullsized":"2024-03-06T07:22:35.039858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042882+0000","last_change":"2024-03-06T07:22:35.035949+0000","last_active":"2024-03-06T07:22:35.042882+0000","last_peered":"2024-03-06T07:22:35.042882+0000","last_clean":"2024-03-06T07:22:35.042882+0000","last_became_active":"2024-03-06T07:22:35.035687+0000","last_became_peered":"2024-03-06T07:22:35.035687+0000","last_unstale":"2024-03-06T07:22:35.042882+0000","last_undegraded":"2024-03-06T07:22:35.042882+0000","last_fullsized":"2024-03-06T07:22:35.042882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:52.174644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037996+0000","last_change":"2024-03-06T07:22:35.033081+0000","last_active":"2024-03-06T07:22:35.037996+0000","last_peered":"2024-03-06T07:22:35.037996+0000","last_clean":"2024-03-06T07:22:35.037996+0000","last_became_active":"2024-03-06T07:22:35.032799+0000","last_became_peered":"2024-03-06T07:22:35.032799+0000","last_unstale":"2024-03-06T07:22:35.037996+0000","last_undegraded":"2024-03-06T07:22:35.037996+0000","last_fullsized":"2024-03-06T07:22:35.037996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:50.041891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044539+0000","last_change":"2024-03-06T07:22:35.041976+0000","last_active":"2024-03-06T07:22:35.044539+0000","last_peered":"2024-03-06T07:22:35.044539+0000","last_clean":"2024-03-06T07:22:35.044539+0000","last_became_active":"2024-03-06T07:22:35.041546+0000","last_became_peered":"2024-03-06T07:22:35.041546+0000","last_unstale":"2024-03-06T07:22:35.044539+0000","last_undegraded":"2024-03-06T07:22:35.044539+0000","last_fullsized":"2024-03-06T07:22:35.044539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039303+0000","last_change":"2024-03-06T07:22:35.038320+0000","last_active":"2024-03-06T07:22:35.039303+0000","last_peered":"2024-03-06T07:22:35.039303+0000","last_clean":"2024-03-06T07:22:35.039303+0000","last_became_active":"2024-03-06T07:22:35.038043+0000","last_became_peered":"2024-03-06T07:22:35.038043+0000","last_unstale":"2024-03-06T07:22:35.039303+0000","last_undegraded":"2024-03-06T07:22:35.039303+0000","last_fullsized":"2024-03-06T07:22:35.039303+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:27.011065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.041743+0000","last_change":"2024-03-06T07:22:35.040651+0000","last_active":"2024-03-06T07:22:35.041743+0000","last_peered":"2024-03-06T07:22:35.041743+0000","last_clean":"2024-03-06T07:22:35.041743+0000","last_became_active":"2024-03-06T07:22:35.040382+0000","last_became_peered":"2024-03-06T07:22:35.040382+0000","last_unstale":"2024-03-06T07:22:35.041743+0000","last_undegraded":"2024-03-06T07:22:35.041743+0000","last_fullsized":"2024-03-06T07:22:35.041743+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:16.188352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040761+0000","last_change":"2024-03-06T07:22:35.035524+0000","last_active":"2024-03-06T07:22:35.040761+0000","last_peered":"2024-03-06T07:22:35.040761+0000","last_clean":"2024-03-06T07:22:35.040761+0000","last_became_active":"2024-03-06T07:22:35.035252+0000","last_became_peered":"2024-03-06T07:22:35.035252+0000","last_unstale":"2024-03-06T07:22:35.040761+0000","last_undegraded":"2024-03-06T07:22:35.040761+0000","last_fullsized":"2024-03-06T07:22:35.040761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:10.686677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044689+0000","last_change":"2024-03-06T07:22:35.040188+0000","last_active":"2024-03-06T07:22:35.044689+0000","last_peered":"2024-03-06T07:22:35.044689+0000","last_clean":"2024-03-06T07:22:35.044689+0000","last_became_active":"2024-03-06T07:22:35.039868+0000","last_became_peered":"2024-03-06T07:22:35.039868+0000","last_unstale":"2024-03-06T07:22:35.044689+0000","last_undegraded":"2024-03-06T07:22:35.044689+0000","last_fullsized":"2024-03-06T07:22:35.044689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:04.009924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.050474+0000","last_change":"2024-03-06T07:22:35.049739+0000","last_active":"2024-03-06T07:22:35.050474+0000","last_peered":"2024-03-06T07:22:35.050474+0000","last_clean":"2024-03-06T07:22:35.050474+0000","last_became_active":"2024-03-06T07:22:35.049118+0000","last_became_peered":"2024-03-06T07:22:35.049118+0000","last_unstale":"2024-03-06T07:22:35.050474+0000","last_undegraded":"2024-03-06T07:22:35.050474+0000","last_fullsized":"2024-03-06T07:22:35.050474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042924+0000","last_change":"2024-03-06T07:22:35.038831+0000","last_active":"2024-03-06T07:22:35.042924+0000","last_peered":"2024-03-06T07:22:35.042924+0000","last_clean":"2024-03-06T07:22:35.042924+0000","last_became_active":"2024-03-06T07:22:35.038550+0000","last_became_peered":"2024-03-06T07:22:35.038550+0000","last_unstale":"2024-03-06T07:22:35.042924+0000","last_undegraded":"2024-03-06T07:22:35.042924+0000","last_fullsized":"2024-03-06T07:22:35.042924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:48.293048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039490+0000","last_change":"2024-03-06T07:22:35.037819+0000","last_active":"2024-03-06T07:22:35.039490+0000","last_peered":"2024-03-06T07:22:35.039490+0000","last_clean":"2024-03-06T07:22:35.039490+0000","last_became_active":"2024-03-06T07:22:35.037519+0000","last_became_peered":"2024-03-06T07:22:35.037519+0000","last_unstale":"2024-03-06T07:22:35.039490+0000","last_undegraded":"2024-03-06T07:22:35.039490+0000","last_fullsized":"2024-03-06T07:22:35.039490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:34.907499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.041507+0000","last_change":"2024-03-06T07:22:35.040052+0000","last_active":"2024-03-06T07:22:35.041507+0000","last_peered":"2024-03-06T07:22:35.041507+0000","last_clean":"2024-03-06T07:22:35.041507+0000","last_became_active":"2024-03-06T07:22:35.039794+0000","last_became_peered":"2024-03-06T07:22:35.039794+0000","last_unstale":"2024-03-06T07:22:35.041507+0000","last_undegraded":"2024-03-06T07:22:35.041507+0000","last_fullsized":"2024-03-06T07:22:35.041507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038208+0000","last_change":"2024-03-06T07:22:35.034108+0000","last_active":"2024-03-06T07:22:35.038208+0000","last_peered":"2024-03-06T07:22:35.038208+0000","last_clean":"2024-03-06T07:22:35.038208+0000","last_became_active":"2024-03-06T07:22:35.033823+0000","last_became_peered":"2024-03-06T07:22:35.033823+0000","last_unstale":"2024-03-06T07:22:35.038208+0000","last_undegraded":"2024-03-06T07:22:35.038208+0000","last_fullsized":"2024-03-06T07:22:35.038208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:55.082141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044523+0000","last_change":"2024-03-06T07:22:35.037317+0000","last_active":"2024-03-06T07:22:35.044523+0000","last_peered":"2024-03-06T07:22:35.044523+0000","last_clean":"2024-03-06T07:22:35.044523+0000","last_became_active":"2024-03-06T07:22:35.033185+0000","last_became_peered":"2024-03-06T07:22:35.033185+0000","last_unstale":"2024-03-06T07:22:35.044523+0000","last_undegraded":"2024-03-06T07:22:35.044523+0000","last_fullsized":"2024-03-06T07:22:35.044523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037970+0000","last_change":"2024-03-06T07:22:35.035073+0000","last_active":"2024-03-06T07:22:35.037970+0000","last_peered":"2024-03-06T07:22:35.037970+0000","last_clean":"2024-03-06T07:22:35.037970+0000","last_became_active":"2024-03-06T07:22:35.034739+0000","last_became_peered":"2024-03-06T07:22:35.034739+0000","last_unstale":"2024-03-06T07:22:35.037970+0000","last_undegraded":"2024-03-06T07:22:35.037970+0000","last_fullsized":"2024-03-06T07:22:35.037970+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:55:11.913435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.033258+0000","last_change":"2024-03-06T07:22:35.032202+0000","last_active":"2024-03-06T07:22:35.033258+0000","last_peered":"2024-03-06T07:22:35.033258+0000","last_clean":"2024-03-06T07:22:35.033258+0000","last_became_active":"2024-03-06T07:22:35.031858+0000","last_became_peered":"2024-03-06T07:22:35.031858+0000","last_unstale":"2024-03-06T07:22:35.033258+0000","last_undegraded":"2024-03-06T07:22:35.033258+0000","last_fullsized":"2024-03-06T07:22:35.033258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046626+0000","last_change":"2024-03-06T07:22:35.041928+0000","last_active":"2024-03-06T07:22:35.046626+0000","last_peered":"2024-03-06T07:22:35.046626+0000","last_clean":"2024-03-06T07:22:35.046626+0000","last_became_active":"2024-03-06T07:22:35.041672+0000","last_became_peered":"2024-03-06T07:22:35.041672+0000","last_unstale":"2024-03-06T07:22:35.046626+0000","last_undegraded":"2024-03-06T07:22:35.046626+0000","last_fullsized":"2024-03-06T07:22:35.046626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038191+0000","last_change":"2024-03-06T07:22:35.036178+0000","last_active":"2024-03-06T07:22:35.038191+0000","last_peered":"2024-03-06T07:22:35.038191+0000","last_clean":"2024-03-06T07:22:35.038191+0000","last_became_active":"2024-03-06T07:22:35.035874+0000","last_became_peered":"2024-03-06T07:22:35.035874+0000","last_unstale":"2024-03-06T07:22:35.038191+0000","last_undegraded":"2024-03-06T07:22:35.038191+0000","last_fullsized":"2024-03-06T07:22:35.038191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:53.794103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.036528+0000","last_change":"2024-03-06T07:22:35.031787+0000","last_active":"2024-03-06T07:22:35.036528+0000","last_peered":"2024-03-06T07:22:35.036528+0000","last_clean":"2024-03-06T07:22:35.036528+0000","last_became_active":"2024-03-06T07:22:35.031524+0000","last_became_peered":"2024-03-06T07:22:35.031524+0000","last_unstale":"2024-03-06T07:22:35.036528+0000","last_undegraded":"2024-03-06T07:22:35.036528+0000","last_fullsized":"2024-03-06T07:22:35.036528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:42.898690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040976+0000","last_change":"2024-03-06T07:22:35.039311+0000","last_active":"2024-03-06T07:22:35.040976+0000","last_peered":"2024-03-06T07:22:35.040976+0000","last_clean":"2024-03-06T07:22:35.040976+0000","last_became_active":"2024-03-06T07:22:35.038971+0000","last_became_peered":"2024-03-06T07:22:35.038971+0000","last_unstale":"2024-03-06T07:22:35.040976+0000","last_undegraded":"2024-03-06T07:22:35.040976+0000","last_fullsized":"2024-03-06T07:22:35.040976+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:24:40.102058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.036808+0000","last_change":"2024-03-06T07:22:35.031978+0000","last_active":"2024-03-06T07:22:35.036808+0000","last_peered":"2024-03-06T07:22:35.036808+0000","last_clean":"2024-03-06T07:22:35.036808+0000","last_became_active":"2024-03-06T07:22:35.031711+0000","last_became_peered":"2024-03-06T07:22:35.031711+0000","last_unstale":"2024-03-06T07:22:35.036808+0000","last_undegraded":"2024-03-06T07:22:35.036808+0000","last_fullsized":"2024-03-06T07:22:35.036808+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:08.697758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.048359+0000","last_change":"2024-03-06T07:22:35.045711+0000","last_active":"2024-03-06T07:22:35.048359+0000","last_peered":"2024-03-06T07:22:35.048359+0000","last_clean":"2024-03-06T07:22:35.048359+0000","last_became_active":"2024-03-06T07:22:35.041242+0000","last_became_peered":"2024-03-06T07:22:35.041242+0000","last_unstale":"2024-03-06T07:22:35.048359+0000","last_undegraded":"2024-03-06T07:22:35.048359+0000","last_fullsized":"2024-03-06T07:22:35.048359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:43:41.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044446+0000","last_change":"2024-03-06T07:22:35.042301+0000","last_active":"2024-03-06T07:22:35.044446+0000","last_peered":"2024-03-06T07:22:35.044446+0000","last_clean":"2024-03-06T07:22:35.044446+0000","last_became_active":"2024-03-06T07:22:35.041172+0000","last_became_peered":"2024-03-06T07:22:35.041172+0000","last_unstale":"2024-03-06T07:22:35.044446+0000","last_undegraded":"2024-03-06T07:22:35.044446+0000","last_fullsized":"2024-03-06T07:22:35.044446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042090+0000","last_change":"2024-03-06T07:22:35.038117+0000","last_active":"2024-03-06T07:22:35.042090+0000","last_peered":"2024-03-06T07:22:35.042090+0000","last_clean":"2024-03-06T07:22:35.042090+0000","last_became_active":"2024-03-06T07:22:35.037812+0000","last_became_peered":"2024-03-06T07:22:35.037812+0000","last_unstale":"2024-03-06T07:22:35.042090+0000","last_undegraded":"2024-03-06T07:22:35.042090+0000","last_fullsized":"2024-03-06T07:22:35.042090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:18.407646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040449+0000","last_change":"2024-03-06T07:22:35.033425+0000","last_active":"2024-03-06T07:22:35.040449+0000","last_peered":"2024-03-06T07:22:35.040449+0000","last_clean":"2024-03-06T07:22:35.040449+0000","last_became_active":"2024-03-06T07:22:35.032784+0000","last_became_peered":"2024-03-06T07:22:35.032784+0000","last_unstale":"2024-03-06T07:22:35.040449+0000","last_undegraded":"2024-03-06T07:22:35.040449+0000","last_fullsized":"2024-03-06T07:22:35.040449+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038171+0000","last_change":"2024-03-06T07:22:35.033473+0000","last_active":"2024-03-06T07:22:35.038171+0000","last_peered":"2024-03-06T07:22:35.038171+0000","last_clean":"2024-03-06T07:22:35.038171+0000","last_became_active":"2024-03-06T07:22:35.032407+0000","last_became_peered":"2024-03-06T07:22:35.032407+0000","last_unstale":"2024-03-06T07:22:35.038171+0000","last_undegraded":"2024-03-06T07:22:35.038171+0000","last_fullsized":"2024-03-06T07:22:35.038171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:21.244078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049427+0000","last_change":"2024-03-06T07:22:35.048127+0000","last_active":"2024-03-06T07:22:35.049427+0000","last_peered":"2024-03-06T07:22:35.049427+0000","last_clean":"2024-03-06T07:22:35.049427+0000","last_became_active":"2024-03-06T07:22:35.047869+0000","last_became_peered":"2024-03-06T07:22:35.047869+0000","last_unstale":"2024-03-06T07:22:35.049427+0000","last_undegraded":"2024-03-06T07:22:35.049427+0000","last_fullsized":"2024-03-06T07:22:35.049427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044448+0000","last_change":"2024-03-06T07:22:35.038257+0000","last_active":"2024-03-06T07:22:35.044448+0000","last_peered":"2024-03-06T07:22:35.044448+0000","last_clean":"2024-03-06T07:22:35.044448+0000","last_became_active":"2024-03-06T07:22:35.037969+0000","last_became_peered":"2024-03-06T07:22:35.037969+0000","last_unstale":"2024-03-06T07:22:35.044448+0000","last_undegraded":"2024-03-06T07:22:35.044448+0000","last_fullsized":"2024-03-06T07:22:35.044448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:43:41.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.033107+0000","last_change":"2024-03-06T07:22:35.032129+0000","last_active":"2024-03-06T07:22:35.033107+0000","last_peered":"2024-03-06T07:22:35.033107+0000","last_clean":"2024-03-06T07:22:35.033107+0000","last_became_active":"2024-03-06T07:22:35.027787+0000","last_became_peered":"2024-03-06T07:22:35.027787+0000","last_unstale":"2024-03-06T07:22:35.033107+0000","last_undegraded":"2024-03-06T07:22:35.033107+0000","last_fullsized":"2024-03-06T07:22:35.033107+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044483+0000","last_change":"2024-03-06T07:22:35.036989+0000","last_active":"2024-03-06T07:22:35.044483+0000","last_peered":"2024-03-06T07:22:35.044483+0000","last_clean":"2024-03-06T07:22:35.044483+0000","last_became_active":"2024-03-06T07:22:35.032803+0000","last_became_peered":"2024-03-06T07:22:35.032803+0000","last_unstale":"2024-03-06T07:22:35.044483+0000","last_undegraded":"2024-03-06T07:22:35.044483+0000","last_fullsized":"2024-03-06T07:22:35.044483+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.029382+0000","last_change":"2024-03-06T07:22:35.025981+0000","last_active":"2024-03-06T07:22:35.029382+0000","last_peered":"2024-03-06T07:22:35.029382+0000","last_clean":"2024-03-06T07:22:35.029382+0000","last_became_active":"2024-03-06T07:22:35.025721+0000","last_became_peered":"2024-03-06T07:22:35.025721+0000","last_unstale":"2024-03-06T07:22:35.029382+0000","last_undegraded":"2024-03-06T07:22:35.029382+0000","last_fullsized":"2024-03-06T07:22:35.029382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049700+0000","last_change":"2024-03-06T07:22:34.043747+0000","last_active":"2024-03-06T07:22:35.049700+0000","last_peered":"2024-03-06T07:22:35.049700+0000","last_clean":"2024-03-06T07:22:35.049700+0000","last_became_active":"2024-03-06T07:22:34.042367+0000","last_became_peered":"2024-03-06T07:22:34.042367+0000","last_unstale":"2024-03-06T07:22:35.049700+0000","last_undegraded":"2024-03-06T07:22:35.049700+0000","last_fullsized":"2024-03-06T07:22:35.049700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:33.138544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038190+0000","last_change":"2024-03-06T07:22:35.032593+0000","last_active":"2024-03-06T07:22:35.038190+0000","last_peered":"2024-03-06T07:22:35.038190+0000","last_clean":"2024-03-06T07:22:35.038190+0000","last_became_active":"2024-03-06T07:22:35.031651+0000","last_became_peered":"2024-03-06T07:22:35.031651+0000","last_unstale":"2024-03-06T07:22:35.038190+0000","last_undegraded":"2024-03-06T07:22:35.038190+0000","last_fullsized":"2024-03-06T07:22:35.038190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:48.293048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046357+0000","last_change":"2024-03-06T07:22:34.038702+0000","last_active":"2024-03-06T07:22:35.046357+0000","last_peered":"2024-03-06T07:22:35.046357+0000","last_clean":"2024-03-06T07:22:35.046357+0000","last_became_active":"2024-03-06T07:22:34.038165+0000","last_became_peered":"2024-03-06T07:22:34.038165+0000","last_unstale":"2024-03-06T07:22:35.046357+0000","last_undegraded":"2024-03-06T07:22:35.046357+0000","last_fullsized":"2024-03-06T07:22:35.046357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039617+0000","last_change":"2024-03-06T07:22:35.035774+0000","last_active":"2024-03-06T07:22:35.039617+0000","last_peered":"2024-03-06T07:22:35.039617+0000","last_clean":"2024-03-06T07:22:35.039617+0000","last_became_active":"2024-03-06T07:22:35.035468+0000","last_became_peered":"2024-03-06T07:22:35.035468+0000","last_unstale":"2024-03-06T07:22:35.039617+0000","last_undegraded":"2024-03-06T07:22:35.039617+0000","last_fullsized":"2024-03-06T07:22:35.039617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:49.912487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046668+0000","last_change":"2024-03-06T07:22:34.042434+0000","last_active":"2024-03-06T07:22:35.046668+0000","last_peered":"2024-03-06T07:22:35.046668+0000","last_clean":"2024-03-06T07:22:35.046668+0000","last_became_active":"2024-03-06T07:22:34.042093+0000","last_became_peered":"2024-03-06T07:22:34.042093+0000","last_unstale":"2024-03-06T07:22:35.046668+0000","last_undegraded":"2024-03-06T07:22:35.046668+0000","last_fullsized":"2024-03-06T07:22:35.046668+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.036685+0000","last_change":"2024-03-06T07:22:35.032012+0000","last_active":"2024-03-06T07:22:35.036685+0000","last_peered":"2024-03-06T07:22:35.036685+0000","last_clean":"2024-03-06T07:22:35.036685+0000","last_became_active":"2024-03-06T07:22:35.031728+0000","last_became_peered":"2024-03-06T07:22:35.031728+0000","last_unstale":"2024-03-06T07:22:35.036685+0000","last_undegraded":"2024-03-06T07:22:35.036685+0000","last_fullsized":"2024-03-06T07:22:35.036685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:32.785444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046950+0000","last_change":"2024-03-06T07:22:34.038862+0000","last_active":"2024-03-06T07:22:35.046950+0000","last_peered":"2024-03-06T07:22:35.046950+0000","last_clean":"2024-03-06T07:22:35.046950+0000","last_became_active":"2024-03-06T07:22:34.038590+0000","last_became_peered":"2024-03-06T07:22:34.038590+0000","last_unstale":"2024-03-06T07:22:35.046950+0000","last_undegraded":"2024-03-06T07:22:35.046950+0000","last_fullsized":"2024-03-06T07:22:35.046950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:15.997587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040991+0000","last_change":"2024-03-06T07:22:35.036832+0000","last_active":"2024-03-06T07:22:35.040991+0000","last_peered":"2024-03-06T07:22:35.040991+0000","last_clean":"2024-03-06T07:22:35.040991+0000","last_became_active":"2024-03-06T07:22:35.036392+0000","last_became_peered":"2024-03-06T07:22:35.036392+0000","last_unstale":"2024-03-06T07:22:35.040991+0000","last_undegraded":"2024-03-06T07:22:35.040991+0000","last_fullsized":"2024-03-06T07:22:35.040991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.045898+0000","last_change":"2024-03-06T07:22:34.037848+0000","last_active":"2024-03-06T07:22:35.045898+0000","last_peered":"2024-03-06T07:22:35.045898+0000","last_clean":"2024-03-06T07:22:35.045898+0000","last_became_active":"2024-03-06T07:22:34.037572+0000","last_became_peered":"2024-03-06T07:22:34.037572+0000","last_unstale":"2024-03-06T07:22:35.045898+0000","last_undegraded":"2024-03-06T07:22:35.045898+0000","last_fullsized":"2024-03-06T07:22:35.045898+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:15.748729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.043769+0000","last_change":"2024-03-06T07:22:35.040723+0000","last_active":"2024-03-06T07:22:35.043769+0000","last_peered":"2024-03-06T07:22:35.043769+0000","last_clean":"2024-03-06T07:22:35.043769+0000","last_became_active":"2024-03-06T07:22:35.036544+0000","last_became_peered":"2024-03-06T07:22:35.036544+0000","last_unstale":"2024-03-06T07:22:35.043769+0000","last_undegraded":"2024-03-06T07:22:35.043769+0000","last_fullsized":"2024-03-06T07:22:35.043769+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.045551+0000","last_change":"2024-03-06T07:22:34.040746+0000","last_active":"2024-03-06T07:22:35.045551+0000","last_peered":"2024-03-06T07:22:35.045551+0000","last_clean":"2024-03-06T07:22:35.045551+0000","last_became_active":"2024-03-06T07:22:34.040466+0000","last_became_peered":"2024-03-06T07:22:34.040466+0000","last_unstale":"2024-03-06T07:22:35.045551+0000","last_undegraded":"2024-03-06T07:22:35.045551+0000","last_fullsized":"2024-03-06T07:22:35.045551+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:35.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039576+0000","last_change":"2024-03-06T07:22:35.035207+0000","last_active":"2024-03-06T07:22:35.039576+0000","last_peered":"2024-03-06T07:22:35.039576+0000","last_clean":"2024-03-06T07:22:35.039576+0000","last_became_active":"2024-03-06T07:22:35.034887+0000","last_became_peered":"2024-03-06T07:22:35.034887+0000","last_unstale":"2024-03-06T07:22:35.039576+0000","last_undegraded":"2024-03-06T07:22:35.039576+0000","last_fullsized":"2024-03-06T07:22:35.039576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:42.073401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.028684+0000","last_change":"2024-03-06T07:22:34.042833+0000","last_active":"2024-03-06T07:22:35.028684+0000","last_peered":"2024-03-06T07:22:35.028684+0000","last_clean":"2024-03-06T07:22:35.028684+0000","last_became_active":"2024-03-06T07:22:34.042557+0000","last_became_peered":"2024-03-06T07:22:34.042557+0000","last_unstale":"2024-03-06T07:22:35.028684+0000","last_undegraded":"2024-03-06T07:22:35.028684+0000","last_fullsized":"2024-03-06T07:22:35.028684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038879+0000","last_change":"2024-03-06T07:22:35.028293+0000","last_active":"2024-03-06T07:22:35.038879+0000","last_peered":"2024-03-06T07:22:35.038879+0000","last_clean":"2024-03-06T07:22:35.038879+0000","last_became_active":"2024-03-06T07:22:35.028017+0000","last_became_peered":"2024-03-06T07:22:35.028017+0000","last_unstale":"2024-03-06T07:22:35.038879+0000","last_undegraded":"2024-03-06T07:22:35.038879+0000","last_fullsized":"2024-03-06T07:22:35.038879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020218+0000","last_change":"2024-03-06T07:22:34.037660+0000","last_active":"2024-03-06T07:22:35.020218+0000","last_peered":"2024-03-06T07:22:35.020218+0000","last_clean":"2024-03-06T07:22:35.020218+0000","last_became_active":"2024-03-06T07:22:34.037347+0000","last_became_peered":"2024-03-06T07:22:34.037347+0000","last_unstale":"2024-03-06T07:22:35.020218+0000","last_undegraded":"2024-03-06T07:22:35.020218+0000","last_fullsized":"2024-03-06T07:22:35.020218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:43.483410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.048335+0000","last_change":"2024-03-06T07:22:35.045241+0000","last_active":"2024-03-06T07:22:35.048335+0000","last_peered":"2024-03-06T07:22:35.048335+0000","last_clean":"2024-03-06T07:22:35.048335+0000","last_became_active":"2024-03-06T07:22:35.044953+0000","last_became_peered":"2024-03-06T07:22:35.044953+0000","last_unstale":"2024-03-06T07:22:35.048335+0000","last_undegraded":"2024-03-06T07:22:35.048335+0000","last_fullsized":"2024-03-06T07:22:35.048335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.019518+0000","last_change":"2024-03-06T07:22:34.038501+0000","last_active":"2024-03-06T07:22:35.019518+0000","last_peered":"2024-03-06T07:22:35.019518+0000","last_clean":"2024-03-06T07:22:35.019518+0000","last_became_active":"2024-03-06T07:22:34.038238+0000","last_became_peered":"2024-03-06T07:22:34.038238+0000","last_unstale":"2024-03-06T07:22:35.019518+0000","last_undegraded":"2024-03-06T07:22:35.019518+0000","last_fullsized":"2024-03-06T07:22:35.019518+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.041991+0000","last_change":"2024-03-06T07:22:35.035913+0000","last_active":"2024-03-06T07:22:35.041991+0000","last_peered":"2024-03-06T07:22:35.041991+0000","last_clean":"2024-03-06T07:22:35.041991+0000","last_became_active":"2024-03-06T07:22:35.035658+0000","last_became_peered":"2024-03-06T07:22:35.035658+0000","last_unstale":"2024-03-06T07:22:35.041991+0000","last_undegraded":"2024-03-06T07:22:35.041991+0000","last_fullsized":"2024-03-06T07:22:35.041991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:19.634384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.029096+0000","last_change":"2024-03-06T07:22:34.040689+0000","last_active":"2024-03-06T07:22:35.029096+0000","last_peered":"2024-03-06T07:22:35.029096+0000","last_clean":"2024-03-06T07:22:35.029096+0000","last_became_active":"2024-03-06T07:22:34.040410+0000","last_became_peered":"2024-03-06T07:22:34.040410+0000","last_unstale":"2024-03-06T07:22:35.029096+0000","last_undegraded":"2024-03-06T07:22:35.029096+0000","last_fullsized":"2024-03-06T07:22:35.029096+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.281130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.034305+0000","last_change":"2024-03-06T07:22:35.032615+0000","last_active":"2024-03-06T07:22:35.034305+0000","last_peered":"2024-03-06T07:22:35.034305+0000","last_clean":"2024-03-06T07:22:35.034305+0000","last_became_active":"2024-03-06T07:22:35.031167+0000","last_became_peered":"2024-03-06T07:22:35.031167+0000","last_unstale":"2024-03-06T07:22:35.034305+0000","last_undegraded":"2024-03-06T07:22:35.034305+0000","last_fullsized":"2024-03-06T07:22:35.034305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.050647+0000","last_change":"2024-03-06T07:22:34.038406+0000","last_active":"2024-03-06T07:22:35.050647+0000","last_peered":"2024-03-06T07:22:35.050647+0000","last_clean":"2024-03-06T07:22:35.050647+0000","last_became_active":"2024-03-06T07:22:34.038040+0000","last_became_peered":"2024-03-06T07:22:34.038040+0000","last_unstale":"2024-03-06T07:22:35.050647+0000","last_undegraded":"2024-03-06T07:22:35.050647+0000","last_fullsized":"2024-03-06T07:22:35.050647+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:26.063950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040485+0000","last_change":"2024-03-06T07:22:35.038459+0000","last_active":"2024-03-06T07:22:35.040485+0000","last_peered":"2024-03-06T07:22:35.040485+0000","last_clean":"2024-03-06T07:22:35.040485+0000","last_became_active":"2024-03-06T07:22:35.038185+0000","last_became_peered":"2024-03-06T07:22:35.038185+0000","last_unstale":"2024-03-06T07:22:35.040485+0000","last_undegraded":"2024-03-06T07:22:35.040485+0000","last_fullsized":"2024-03-06T07:22:35.040485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:05.629298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049070+0000","last_change":"2024-03-06T07:22:34.037962+0000","last_active":"2024-03-06T07:22:35.049070+0000","last_peered":"2024-03-06T07:22:35.049070+0000","last_clean":"2024-03-06T07:22:35.049070+0000","last_became_active":"2024-03-06T07:22:34.037635+0000","last_became_peered":"2024-03-06T07:22:34.037635+0000","last_unstale":"2024-03-06T07:22:35.049070+0000","last_undegraded":"2024-03-06T07:22:35.049070+0000","last_fullsized":"2024-03-06T07:22:35.049070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042905+0000","last_change":"2024-03-06T07:22:35.036674+0000","last_active":"2024-03-06T07:22:35.042905+0000","last_peered":"2024-03-06T07:22:35.042905+0000","last_clean":"2024-03-06T07:22:35.042905+0000","last_became_active":"2024-03-06T07:22:35.036400+0000","last_became_peered":"2024-03-06T07:22:35.036400+0000","last_unstale":"2024-03-06T07:22:35.042905+0000","last_undegraded":"2024-03-06T07:22:35.042905+0000","last_fullsized":"2024-03-06T07:22:35.042905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:55.948724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046183+0000","last_change":"2024-03-06T07:22:34.041311+0000","last_active":"2024-03-06T07:22:35.046183+0000","last_peered":"2024-03-06T07:22:35.046183+0000","last_clean":"2024-03-06T07:22:35.046183+0000","last_became_active":"2024-03-06T07:22:34.040940+0000","last_became_peered":"2024-03-06T07:22:34.040940+0000","last_unstale":"2024-03-06T07:22:35.046183+0000","last_undegraded":"2024-03-06T07:22:35.046183+0000","last_fullsized":"2024-03-06T07:22:35.046183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:24.924844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.024503+0000","last_change":"2024-03-06T07:22:35.022953+0000","last_active":"2024-03-06T07:22:35.024503+0000","last_peered":"2024-03-06T07:22:35.024503+0000","last_clean":"2024-03-06T07:22:35.024503+0000","last_became_active":"2024-03-06T07:22:35.022681+0000","last_became_peered":"2024-03-06T07:22:35.022681+0000","last_unstale":"2024-03-06T07:22:35.024503+0000","last_undegraded":"2024-03-06T07:22:35.024503+0000","last_fullsized":"2024-03-06T07:22:35.024503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020602+0000","last_change":"2024-03-06T07:22:34.037840+0000","last_active":"2024-03-06T07:22:35.020602+0000","last_peered":"2024-03-06T07:22:35.020602+0000","last_clean":"2024-03-06T07:22:35.020602+0000","last_became_active":"2024-03-06T07:22:34.037556+0000","last_became_peered":"2024-03-06T07:22:34.037556+0000","last_unstale":"2024-03-06T07:22:35.020602+0000","last_undegraded":"2024-03-06T07:22:35.020602+0000","last_fullsized":"2024-03-06T07:22:35.020602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038768+0000","last_change":"2024-03-06T07:22:35.034503+0000","last_active":"2024-03-06T07:22:35.038768+0000","last_peered":"2024-03-06T07:22:35.038768+0000","last_clean":"2024-03-06T07:22:35.038768+0000","last_became_active":"2024-03-06T07:22:35.034230+0000","last_became_peered":"2024-03-06T07:22:35.034230+0000","last_unstale":"2024-03-06T07:22:35.038768+0000","last_undegraded":"2024-03-06T07:22:35.038768+0000","last_fullsized":"2024-03-06T07:22:35.038768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:05.629298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020787+0000","last_change":"2024-03-06T07:22:34.039959+0000","last_active":"2024-03-06T07:22:35.020787+0000","last_peered":"2024-03-06T07:22:35.020787+0000","last_clean":"2024-03-06T07:22:35.020787+0000","last_became_active":"2024-03-06T07:22:34.039689+0000","last_became_peered":"2024-03-06T07:22:34.039689+0000","last_unstale":"2024-03-06T07:22:35.020787+0000","last_undegraded":"2024-03-06T07:22:35.020787+0000","last_fullsized":"2024-03-06T07:22:35.020787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:23.420842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038920+0000","last_change":"2024-03-06T07:22:35.032290+0000","last_active":"2024-03-06T07:22:35.038920+0000","last_peered":"2024-03-06T07:22:35.038920+0000","last_clean":"2024-03-06T07:22:35.038920+0000","last_became_active":"2024-03-06T07:22:35.032048+0000","last_became_peered":"2024-03-06T07:22:35.032048+0000","last_unstale":"2024-03-06T07:22:35.038920+0000","last_undegraded":"2024-03-06T07:22:35.038920+0000","last_fullsized":"2024-03-06T07:22:35.038920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:21.230955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.051866+0000","last_change":"2024-03-06T07:22:34.043149+0000","last_active":"2024-03-06T07:22:35.051866+0000","last_peered":"2024-03-06T07:22:35.051866+0000","last_clean":"2024-03-06T07:22:35.051866+0000","last_became_active":"2024-03-06T07:22:34.042888+0000","last_became_peered":"2024-03-06T07:22:34.042888+0000","last_unstale":"2024-03-06T07:22:35.051866+0000","last_undegraded":"2024-03-06T07:22:35.051866+0000","last_fullsized":"2024-03-06T07:22:35.051866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.035965+0000","last_change":"2024-03-06T07:22:35.034984+0000","last_active":"2024-03-06T07:22:35.035965+0000","last_peered":"2024-03-06T07:22:35.035965+0000","last_clean":"2024-03-06T07:22:35.035965+0000","last_became_active":"2024-03-06T07:22:35.031303+0000","last_became_peered":"2024-03-06T07:22:35.031303+0000","last_unstale":"2024-03-06T07:22:35.035965+0000","last_undegraded":"2024-03-06T07:22:35.035965+0000","last_fullsized":"2024-03-06T07:22:35.035965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046601+0000","last_change":"2024-03-06T07:22:34.044320+0000","last_active":"2024-03-06T07:22:35.046601+0000","last_peered":"2024-03-06T07:22:35.046601+0000","last_clean":"2024-03-06T07:22:35.046601+0000","last_became_active":"2024-03-06T07:22:34.043832+0000","last_became_peered":"2024-03-06T07:22:34.043832+0000","last_unstale":"2024-03-06T07:22:35.046601+0000","last_undegraded":"2024-03-06T07:22:35.046601+0000","last_fullsized":"2024-03-06T07:22:35.046601+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037938+0000","last_change":"2024-03-06T07:22:35.034203+0000","last_active":"2024-03-06T07:22:35.037938+0000","last_peered":"2024-03-06T07:22:35.037938+0000","last_clean":"2024-03-06T07:22:35.037938+0000","last_became_active":"2024-03-06T07:22:35.033903+0000","last_became_peered":"2024-03-06T07:22:35.033903+0000","last_unstale":"2024-03-06T07:22:35.037938+0000","last_undegraded":"2024-03-06T07:22:35.037938+0000","last_fullsized":"2024-03-06T07:22:35.037938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:19:07.387824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.045282+0000","last_change":"2024-03-06T07:22:34.036176+0000","last_active":"2024-03-06T07:22:35.045282+0000","last_peered":"2024-03-06T07:22:35.045282+0000","last_clean":"2024-03-06T07:22:35.045282+0000","last_became_active":"2024-03-06T07:22:34.035867+0000","last_became_peered":"2024-03-06T07:22:34.035867+0000","last_unstale":"2024-03-06T07:22:35.045282+0000","last_undegraded":"2024-03-06T07:22:35.045282+0000","last_fullsized":"2024-03-06T07:22:35.045282+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:28.336495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040955+0000","last_change":"2024-03-06T07:22:35.038497+0000","last_active":"2024-03-06T07:22:35.040955+0000","last_peered":"2024-03-06T07:22:35.040955+0000","last_clean":"2024-03-06T07:22:35.040955+0000","last_became_active":"2024-03-06T07:22:35.038189+0000","last_became_peered":"2024-03-06T07:22:35.038189+0000","last_unstale":"2024-03-06T07:22:35.040955+0000","last_undegraded":"2024-03-06T07:22:35.040955+0000","last_fullsized":"2024-03-06T07:22:35.040955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:40.330097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049723+0000","last_change":"2024-03-06T07:22:34.038423+0000","last_active":"2024-03-06T07:22:35.049723+0000","last_peered":"2024-03-06T07:22:35.049723+0000","last_clean":"2024-03-06T07:22:35.049723+0000","last_became_active":"2024-03-06T07:22:34.038126+0000","last_became_peered":"2024-03-06T07:22:34.038126+0000","last_unstale":"2024-03-06T07:22:35.049723+0000","last_undegraded":"2024-03-06T07:22:35.049723+0000","last_fullsized":"2024-03-06T07:22:35.049723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:43.441029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046669+0000","last_change":"2024-03-06T07:22:35.042660+0000","last_active":"2024-03-06T07:22:35.046669+0000","last_peered":"2024-03-06T07:22:35.046669+0000","last_clean":"2024-03-06T07:22:35.046669+0000","last_became_active":"2024-03-06T07:22:35.042327+0000","last_became_peered":"2024-03-06T07:22:35.042327+0000","last_unstale":"2024-03-06T07:22:35.046669+0000","last_undegraded":"2024-03-06T07:22:35.046669+0000","last_fullsized":"2024-03-06T07:22:35.046669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:43:41.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020834+0000","last_change":"2024-03-06T07:22:34.035505+0000","last_active":"2024-03-06T07:22:35.020834+0000","last_peered":"2024-03-06T07:22:35.020834+0000","last_clean":"2024-03-06T07:22:35.020834+0000","last_became_active":"2024-03-06T07:22:34.035257+0000","last_became_peered":"2024-03-06T07:22:34.035257+0000","last_unstale":"2024-03-06T07:22:35.020834+0000","last_undegraded":"2024-03-06T07:22:35.020834+0000","last_fullsized":"2024-03-06T07:22:35.020834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040969+0000","last_change":"2024-03-06T07:22:35.032174+0000","last_active":"2024-03-06T07:22:35.040969+0000","last_peered":"2024-03-06T07:22:35.040969+0000","last_clean":"2024-03-06T07:22:35.040969+0000","last_became_active":"2024-03-06T07:22:35.031869+0000","last_became_peered":"2024-03-06T07:22:35.031869+0000","last_unstale":"2024-03-06T07:22:35.040969+0000","last_undegraded":"2024-03-06T07:22:35.040969+0000","last_fullsized":"2024-03-06T07:22:35.040969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:18.407646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046790+0000","last_change":"2024-03-06T07:22:34.037475+0000","last_active":"2024-03-06T07:22:35.046790+0000","last_peered":"2024-03-06T07:22:35.046790+0000","last_clean":"2024-03-06T07:22:35.046790+0000","last_became_active":"2024-03-06T07:22:34.037206+0000","last_became_peered":"2024-03-06T07:22:34.037206+0000","last_unstale":"2024-03-06T07:22:35.046790+0000","last_undegraded":"2024-03-06T07:22:35.046790+0000","last_fullsized":"2024-03-06T07:22:35.046790+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044530+0000","last_change":"2024-03-06T07:22:35.042271+0000","last_active":"2024-03-06T07:22:35.044530+0000","last_peered":"2024-03-06T07:22:35.044530+0000","last_clean":"2024-03-06T07:22:35.044530+0000","last_became_active":"2024-03-06T07:22:35.036189+0000","last_became_peered":"2024-03-06T07:22:35.036189+0000","last_unstale":"2024-03-06T07:22:35.044530+0000","last_undegraded":"2024-03-06T07:22:35.044530+0000","last_fullsized":"2024-03-06T07:22:35.044530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.025415+0000","last_change":"2024-03-06T07:22:34.036505+0000","last_active":"2024-03-06T07:22:35.025415+0000","last_peered":"2024-03-06T07:22:35.025415+0000","last_clean":"2024-03-06T07:22:35.025415+0000","last_became_active":"2024-03-06T07:22:34.036198+0000","last_became_peered":"2024-03-06T07:22:34.036198+0000","last_unstale":"2024-03-06T07:22:35.025415+0000","last_undegraded":"2024-03-06T07:22:35.025415+0000","last_fullsized":"2024-03-06T07:22:35.025415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038901+0000","last_change":"2024-03-06T07:22:35.031196+0000","last_active":"2024-03-06T07:22:35.038901+0000","last_peered":"2024-03-06T07:22:35.038901+0000","last_clean":"2024-03-06T07:22:35.038901+0000","last_became_active":"2024-03-06T07:22:35.030767+0000","last_became_peered":"2024-03-06T07:22:35.030767+0000","last_unstale":"2024-03-06T07:22:35.038901+0000","last_undegraded":"2024-03-06T07:22:35.038901+0000","last_fullsized":"2024-03-06T07:22:35.038901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.025346+0000","last_change":"2024-03-06T07:22:34.034863+0000","last_active":"2024-03-06T07:22:35.025346+0000","last_peered":"2024-03-06T07:22:35.025346+0000","last_clean":"2024-03-06T07:22:35.025346+0000","last_became_active":"2024-03-06T07:22:34.034577+0000","last_became_peered":"2024-03-06T07:22:34.034577+0000","last_unstale":"2024-03-06T07:22:35.025346+0000","last_undegraded":"2024-03-06T07:22:35.025346+0000","last_fullsized":"2024-03-06T07:22:35.025346+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040765+0000","last_change":"2024-03-06T07:22:35.036664+0000","last_active":"2024-03-06T07:22:35.040765+0000","last_peered":"2024-03-06T07:22:35.040765+0000","last_clean":"2024-03-06T07:22:35.040765+0000","last_became_active":"2024-03-06T07:22:35.036379+0000","last_became_peered":"2024-03-06T07:22:35.036379+0000","last_unstale":"2024-03-06T07:22:35.040765+0000","last_undegraded":"2024-03-06T07:22:35.040765+0000","last_fullsized":"2024-03-06T07:22:35.040765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:42.968699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046261+0000","last_change":"2024-03-06T07:22:34.044993+0000","last_active":"2024-03-06T07:22:35.046261+0000","last_peered":"2024-03-06T07:22:35.046261+0000","last_clean":"2024-03-06T07:22:35.046261+0000","last_became_active":"2024-03-06T07:22:34.044795+0000","last_became_peered":"2024-03-06T07:22:34.044795+0000","last_unstale":"2024-03-06T07:22:35.046261+0000","last_undegraded":"2024-03-06T07:22:35.046261+0000","last_fullsized":"2024-03-06T07:22:35.046261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.256142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038745+0000","last_change":"2024-03-06T07:22:35.031505+0000","last_active":"2024-03-06T07:22:35.038745+0000","last_peered":"2024-03-06T07:22:35.038745+0000","last_clean":"2024-03-06T07:22:35.038745+0000","last_became_active":"2024-03-06T07:22:35.031241+0000","last_became_peered":"2024-03-06T07:22:35.031241+0000","last_unstale":"2024-03-06T07:22:35.038745+0000","last_undegraded":"2024-03-06T07:22:35.038745+0000","last_fullsized":"2024-03-06T07:22:35.038745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.028828+0000","last_change":"2024-03-06T07:22:34.044453+0000","last_active":"2024-03-06T07:22:35.028828+0000","last_peered":"2024-03-06T07:22:35.028828+0000","last_clean":"2024-03-06T07:22:35.028828+0000","last_became_active":"2024-03-06T07:22:34.044190+0000","last_became_peered":"2024-03-06T07:22:34.044190+0000","last_unstale":"2024-03-06T07:22:35.028828+0000","last_undegraded":"2024-03-06T07:22:35.028828+0000","last_fullsized":"2024-03-06T07:22:35.028828+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039595+0000","last_change":"2024-03-06T07:22:35.034792+0000","last_active":"2024-03-06T07:22:35.039595+0000","last_peered":"2024-03-06T07:22:35.039595+0000","last_clean":"2024-03-06T07:22:35.039595+0000","last_became_active":"2024-03-06T07:22:35.034503+0000","last_became_peered":"2024-03-06T07:22:35.034503+0000","last_unstale":"2024-03-06T07:22:35.039595+0000","last_undegraded":"2024-03-06T07:22:35.039595+0000","last_fullsized":"2024-03-06T07:22:35.039595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:30.130566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020885+0000","last_change":"2024-03-06T07:22:34.037188+0000","last_active":"2024-03-06T07:22:35.020885+0000","last_peered":"2024-03-06T07:22:35.020885+0000","last_clean":"2024-03-06T07:22:35.020885+0000","last_became_active":"2024-03-06T07:22:34.036918+0000","last_became_peered":"2024-03-06T07:22:34.036918+0000","last_unstale":"2024-03-06T07:22:35.020885+0000","last_undegraded":"2024-03-06T07:22:35.020885+0000","last_fullsized":"2024-03-06T07:22:35.020885+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038162+0000","last_change":"2024-03-06T07:22:35.033263+0000","last_active":"2024-03-06T07:22:35.038162+0000","last_peered":"2024-03-06T07:22:35.038162+0000","last_clean":"2024-03-06T07:22:35.038162+0000","last_became_active":"2024-03-06T07:22:35.033005+0000","last_became_peered":"2024-03-06T07:22:35.033005+0000","last_unstale":"2024-03-06T07:22:35.038162+0000","last_undegraded":"2024-03-06T07:22:35.038162+0000","last_fullsized":"2024-03-06T07:22:35.038162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:04.009924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020331+0000","last_change":"2024-03-06T07:22:34.035735+0000","last_active":"2024-03-06T07:22:35.020331+0000","last_peered":"2024-03-06T07:22:35.020331+0000","last_clean":"2024-03-06T07:22:35.020331+0000","last_became_active":"2024-03-06T07:22:34.035459+0000","last_became_peered":"2024-03-06T07:22:34.035459+0000","last_unstale":"2024-03-06T07:22:35.020331+0000","last_undegraded":"2024-03-06T07:22:35.020331+0000","last_fullsized":"2024-03-06T07:22:35.020331+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039257+0000","last_change":"2024-03-06T07:22:35.038111+0000","last_active":"2024-03-06T07:22:35.039257+0000","last_peered":"2024-03-06T07:22:35.039257+0000","last_clean":"2024-03-06T07:22:35.039257+0000","last_became_active":"2024-03-06T07:22:35.037779+0000","last_became_peered":"2024-03-06T07:22:35.037779+0000","last_unstale":"2024-03-06T07:22:35.039257+0000","last_undegraded":"2024-03-06T07:22:35.039257+0000","last_fullsized":"2024-03-06T07:22:35.039257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:18.894659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.017935+0000","last_change":"2024-03-06T07:22:34.039910+0000","last_active":"2024-03-06T07:22:35.017935+0000","last_peered":"2024-03-06T07:22:35.017935+0000","last_clean":"2024-03-06T07:22:35.017935+0000","last_became_active":"2024-03-06T07:22:34.039246+0000","last_became_peered":"2024-03-06T07:22:34.039246+0000","last_unstale":"2024-03-06T07:22:35.017935+0000","last_undegraded":"2024-03-06T07:22:35.017935+0000","last_fullsized":"2024-03-06T07:22:35.017935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:41.961694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037669+0000","last_change":"2024-03-06T07:22:35.027657+0000","last_active":"2024-03-06T07:22:35.037669+0000","last_peered":"2024-03-06T07:22:35.037669+0000","last_clean":"2024-03-06T07:22:35.037669+0000","last_became_active":"2024-03-06T07:22:35.027404+0000","last_became_peered":"2024-03-06T07:22:35.027404+0000","last_unstale":"2024-03-06T07:22:35.037669+0000","last_undegraded":"2024-03-06T07:22:35.037669+0000","last_fullsized":"2024-03-06T07:22:35.037669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.022151+0000","last_change":"2024-03-06T07:22:34.039209+0000","last_active":"2024-03-06T07:22:35.022151+0000","last_peered":"2024-03-06T07:22:35.022151+0000","last_clean":"2024-03-06T07:22:35.022151+0000","last_became_active":"2024-03-06T07:22:34.038921+0000","last_became_peered":"2024-03-06T07:22:34.038921+0000","last_unstale":"2024-03-06T07:22:35.022151+0000","last_undegraded":"2024-03-06T07:22:35.022151+0000","last_fullsized":"2024-03-06T07:22:35.022151+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:12.359435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040218+0000","last_change":"2024-03-06T07:22:35.038010+0000","last_active":"2024-03-06T07:22:35.040218+0000","last_peered":"2024-03-06T07:22:35.040218+0000","last_clean":"2024-03-06T07:22:35.040218+0000","last_became_active":"2024-03-06T07:22:35.037717+0000","last_became_peered":"2024-03-06T07:22:35.037717+0000","last_unstale":"2024-03-06T07:22:35.040218+0000","last_undegraded":"2024-03-06T07:22:35.040218+0000","last_fullsized":"2024-03-06T07:22:35.040218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:21.230955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.019874+0000","last_change":"2024-03-06T07:22:34.039679+0000","last_active":"2024-03-06T07:22:35.019874+0000","last_peered":"2024-03-06T07:22:35.019874+0000","last_clean":"2024-03-06T07:22:35.019874+0000","last_became_active":"2024-03-06T07:22:34.039092+0000","last_became_peered":"2024-03-06T07:22:34.039092+0000","last_unstale":"2024-03-06T07:22:35.019874+0000","last_undegraded":"2024-03-06T07:22:35.019874+0000","last_fullsized":"2024-03-06T07:22:35.019874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.029321+0000","last_change":"2024-03-06T07:22:35.026900+0000","last_active":"2024-03-06T07:22:35.029321+0000","last_peered":"2024-03-06T07:22:35.029321+0000","last_clean":"2024-03-06T07:22:35.029321+0000","last_became_active":"2024-03-06T07:22:35.026626+0000","last_became_peered":"2024-03-06T07:22:35.026626+0000","last_unstale":"2024-03-06T07:22:35.029321+0000","last_undegraded":"2024-03-06T07:22:35.029321+0000","last_fullsized":"2024-03-06T07:22:35.029321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.021043+0000","last_change":"2024-03-06T07:22:34.039382+0000","last_active":"2024-03-06T07:22:35.021043+0000","last_peered":"2024-03-06T07:22:35.021043+0000","last_clean":"2024-03-06T07:22:35.021043+0000","last_became_active":"2024-03-06T07:22:34.039088+0000","last_became_peered":"2024-03-06T07:22:34.039088+0000","last_unstale":"2024-03-06T07:22:35.021043+0000","last_undegraded":"2024-03-06T07:22:35.021043+0000","last_fullsized":"2024-03-06T07:22:35.021043+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040476+0000","last_change":"2024-03-06T07:22:35.033671+0000","last_active":"2024-03-06T07:22:35.040476+0000","last_peered":"2024-03-06T07:22:35.040476+0000","last_clean":"2024-03-06T07:22:35.040476+0000","last_became_active":"2024-03-06T07:22:35.033167+0000","last_became_peered":"2024-03-06T07:22:35.033167+0000","last_unstale":"2024-03-06T07:22:35.040476+0000","last_undegraded":"2024-03-06T07:22:35.040476+0000","last_fullsized":"2024-03-06T07:22:35.040476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.026424+0000","last_change":"2024-03-06T07:22:34.037026+0000","last_active":"2024-03-06T07:22:35.026424+0000","last_peered":"2024-03-06T07:22:35.026424+0000","last_clean":"2024-03-06T07:22:35.026424+0000","last_became_active":"2024-03-06T07:22:34.036742+0000","last_became_peered":"2024-03-06T07:22:34.036742+0000","last_unstale":"2024-03-06T07:22:35.026424+0000","last_undegraded":"2024-03-06T07:22:35.026424+0000","last_fullsized":"2024-03-06T07:22:35.026424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":270,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.017876+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:22:35.017876+0000","last_peered":"2024-03-06T07:22:35.017876+0000","last_clean":"2024-03-06T07:22:35.017876+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:22:35.017876+0000","last_undegraded":"2024-03-06T07:22:35.017876+0000","last_fullsized":"2024-03-06T07:22:35.017876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":1004013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":1004013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:38.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.161+0000 7f54307c7640 1 Processor -- start 2024-03-06T07:22:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.161+0000 7f54307c7640 1 -- start start 2024-03-06T07:22:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.161+0000 7f54307c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542814f650 0x7f5428071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f54307c7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5428072440 con 0x7f5428152c20 2024-03-06T07:22:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f54307c7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54280c2b10 con 0x7f54280c3150 2024-03-06T07:22:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f54307c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5428152140 con 0x7f542814f650 2024-03-06T07:22:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542ed64640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5428152c20 0x7f542815c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47140/0 (socket says 172.21.15.26:47140) 2024-03-06T07:22:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542ed64640 1 -- 172.21.15.26:0/3173875396 learned_addr learned my addr 172.21.15.26:0/3173875396 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542e563640 1 --2- 172.21.15.26:0/3173875396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542814f650 0x7f5428071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542d561640 1 -- 172.21.15.26:0/3173875396 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125162621 0 0) 0x7f5420001490 con 0x7f5428152c20 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542d561640 1 -- 172.21.15.26:0/3173875396 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5414003520 con 0x7f5428152c20 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542d561640 1 -- 172.21.15.26:0/3173875396 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150238286 0 0) 0x7f54280c2b10 con 0x7f54280c3150 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542d561640 1 -- 172.21.15.26:0/3173875396 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5420001490 con 0x7f54280c3150 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542e563640 1 -- 172.21.15.26:0/3173875396 >> v1:172.21.15.117:6789/0 conn(0x7f54280c3150 legacy=0x7f54280c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542e563640 1 -- 172.21.15.26:0/3173875396 >> v1:172.21.15.26:6789/0 conn(0x7f5428152c20 legacy=0x7f542815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.162+0000 7f542e563640 1 -- 172.21.15.26:0/3173875396 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5428152420 con 0x7f542814f650 2024-03-06T07:22:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.163+0000 7f542e563640 1 --2- 172.21.15.26:0/3173875396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542814f650 0x7f5428071880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f54240097d0 tx=0x7f542402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352580998da944e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.163+0000 7f542d561640 1 -- 172.21.15.26:0/3173875396 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424035020 con 0x7f542814f650 2024-03-06T07:22:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.163+0000 7f542d561640 1 -- 172.21.15.26:0/3173875396 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424031890 con 0x7f542814f650 2024-03-06T07:22:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.163+0000 7f542d561640 1 -- 172.21.15.26:0/3173875396 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424031c60 con 0x7f542814f650 2024-03-06T07:22:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.164+0000 7f54307c7640 1 -- 172.21.15.26:0/3173875396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542814f650 msgr2=0x7f5428071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.164+0000 7f54307c7640 1 --2- 172.21.15.26:0/3173875396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542814f650 0x7f5428071880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f54240097d0 tx=0x7f542402b930 comp rx=0 tx=0).stop 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.164+0000 7f54307c7640 1 -- 172.21.15.26:0/3173875396 shutdown_connections 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.164+0000 7f54307c7640 1 --2- 172.21.15.26:0/3173875396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542814f650 0x7f5428071880 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.164+0000 7f54307c7640 1 -- 172.21.15.26:0/3173875396 >> 172.21.15.26:0/3173875396 conn(0x7f54280c4290 msgr2=0x7f54280c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.164+0000 7f54307c7640 1 -- 172.21.15.26:0/3173875396 shutdown_connections 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.164+0000 7f54307c7640 1 -- 172.21.15.26:0/3173875396 wait complete. 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.165+0000 7f54307c7640 1 Processor -- start 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.165+0000 7f54307c7640 1 -- start start 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.165+0000 7f54307c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54280c3150 0x7f54281482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.165+0000 7f54307c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542814f650 0x7f5428155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.165+0000 7f54307c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5428152c20 0x7f5428144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.165+0000 7f54307c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280c40c0 con 0x7f542814f650 2024-03-06T07:22:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f54307c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280c3f40 con 0x7f54280c3150 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f54307c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54280c5da0 con 0x7f5428152c20 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542814f650 0x7f5428155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542814f650 0x7f5428155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34204/0 (socket says 172.21.15.26:34204) 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 -- 172.21.15.26:0/2488146025 learned_addr learned my addr 172.21.15.26:0/2488146025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542e563640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54280c3150 0x7f54281482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542ed64640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5428152c20 0x7f5428144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 -- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5428152c20 msgr2=0x7f5428144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5428152c20 0x7f5428144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 -- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54280c3150 msgr2=0x7f54281482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54280c3150 0x7f54281482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 -- 172.21.15.26:0/2488146025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542814a710 con 0x7f542814f650 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f542dd62640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542814f650 0x7f5428155d20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f541800c500 tx=0x7f541800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f541effd640 1 -- 172.21.15.26:0/2488146025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180052d0 con 0x7f542814f650 2024-03-06T07:22:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f541effd640 1 -- 172.21.15.26:0/2488146025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418005490 con 0x7f542814f650 2024-03-06T07:22:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f541effd640 1 -- 172.21.15.26:0/2488146025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418012870 con 0x7f542814f650 2024-03-06T07:22:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542814c810 con 0x7f542814f650 2024-03-06T07:22:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.166+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5428149d10 con 0x7f542814f650 2024-03-06T07:22:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.167+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542804f330 con 0x7f542814f650 2024-03-06T07:22:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.168+0000 7f541effd640 1 -- 172.21.15.26:0/2488146025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5418012a10 con 0x7f542814f650 2024-03-06T07:22:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.168+0000 7f541effd640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f540c06b4d0 0x7f540c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.168+0000 7f541effd640 1 -- 172.21.15.26:0/2488146025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5418089f60 con 0x7f542814f650 2024-03-06T07:22:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.168+0000 7f542e563640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f540c06b4d0 0x7f540c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.169+0000 7f542e563640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f540c06b4d0 0x7f540c06d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5424036000 tx=0x7f5424008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.173+0000 7f541effd640 1 -- 172.21.15.26:0/2488146025 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5418058610 con 0x7f542814f650 2024-03-06T07:22:38.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.351+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5428068460 con 0x7f540c06b4d0 2024-03-06T07:22:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.358+0000 7f541effd640 1 -- 172.21.15.26:0/2488146025 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252074 (secure 0 0 0) 0x7f54240878c0 con 0x7f540c06b4d0 2024-03-06T07:22:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f540c06b4d0 msgr2=0x7f540c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f540c06b4d0 0x7f540c06d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5424036000 tx=0x7f5424008030 comp rx=0 tx=0).stop 2024-03-06T07:22:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542814f650 msgr2=0x7f5428155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542814f650 0x7f5428155d20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f541800c500 tx=0x7f541800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 shutdown_connections 2024-03-06T07:22:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f540c06b4d0 0x7f540c06d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5428152c20 0x7f5428144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542814f650 0x7f5428155d20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 --2- 172.21.15.26:0/2488146025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54280c3150 0x7f54281482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 >> 172.21.15.26:0/2488146025 conn(0x7f54280c4290 msgr2=0x7f5428071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 shutdown_connections 2024-03-06T07:22:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.360+0000 7f54307c7640 1 -- 172.21.15.26:0/2488146025 wait complete. 2024-03-06T07:22:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-03-06T07:22:37.061963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239828,"kb_used_data":7008,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734892,"statfs":{"total":773094113280,"available":772848529408,"internally_reserved":0,"allocated":7176192,"data_stored":4489896,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.045000"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.031188+0000","last_change":"2024-03-06T07:22:35.025691+0000","last_active":"2024-03-06T07:22:35.031188+0000","last_peered":"2024-03-06T07:22:35.031188+0000","last_clean":"2024-03-06T07:22:35.031188+0000","last_became_active":"2024-03-06T07:22:35.025394+0000","last_became_peered":"2024-03-06T07:22:35.025394+0000","last_unstale":"2024-03-06T07:22:35.031188+0000","last_undegraded":"2024-03-06T07:22:35.031188+0000","last_fullsized":"2024-03-06T07:22:35.031188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.050056+0000","last_change":"2024-03-06T07:22:35.048775+0000","last_active":"2024-03-06T07:22:35.050056+0000","last_peered":"2024-03-06T07:22:35.050056+0000","last_clean":"2024-03-06T07:22:35.050056+0000","last_became_active":"2024-03-06T07:22:35.048476+0000","last_became_peered":"2024-03-06T07:22:35.048476+0000","last_unstale":"2024-03-06T07:22:35.050056+0000","last_undegraded":"2024-03-06T07:22:35.050056+0000","last_fullsized":"2024-03-06T07:22:35.050056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039858+0000","last_change":"2024-03-06T07:22:35.037727+0000","last_active":"2024-03-06T07:22:35.039858+0000","last_peered":"2024-03-06T07:22:35.039858+0000","last_clean":"2024-03-06T07:22:35.039858+0000","last_became_active":"2024-03-06T07:22:35.037421+0000","last_became_peered":"2024-03-06T07:22:35.037421+0000","last_unstale":"2024-03-06T07:22:35.039858+0000","last_undegraded":"2024-03-06T07:22:35.039858+0000","last_fullsized":"2024-03-06T07:22:35.039858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042882+0000","last_change":"2024-03-06T07:22:35.035949+0000","last_active":"2024-03-06T07:22:35.042882+0000","last_peered":"2024-03-06T07:22:35.042882+0000","last_clean":"2024-03-06T07:22:35.042882+0000","last_became_active":"2024-03-06T07:22:35.035687+0000","last_became_peered":"2024-03-06T07:22:35.035687+0000","last_unstale":"2024-03-06T07:22:35.042882+0000","last_undegraded":"2024-03-06T07:22:35.042882+0000","last_fullsized":"2024-03-06T07:22:35.042882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:52.174644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037996+0000","last_change":"2024-03-06T07:22:35.033081+0000","last_active":"2024-03-06T07:22:35.037996+0000","last_peered":"2024-03-06T07:22:35.037996+0000","last_clean":"2024-03-06T07:22:35.037996+0000","last_became_active":"2024-03-06T07:22:35.032799+0000","last_became_peered":"2024-03-06T07:22:35.032799+0000","last_unstale":"2024-03-06T07:22:35.037996+0000","last_undegraded":"2024-03-06T07:22:35.037996+0000","last_fullsized":"2024-03-06T07:22:35.037996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:50.041891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044539+0000","last_change":"2024-03-06T07:22:35.041976+0000","last_active":"2024-03-06T07:22:35.044539+0000","last_peered":"2024-03-06T07:22:35.044539+0000","last_clean":"2024-03-06T07:22:35.044539+0000","last_became_active":"2024-03-06T07:22:35.041546+0000","last_became_peered":"2024-03-06T07:22:35.041546+0000","last_unstale":"2024-03-06T07:22:35.044539+0000","last_undegraded":"2024-03-06T07:22:35.044539+0000","last_fullsized":"2024-03-06T07:22:35.044539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039303+0000","last_change":"2024-03-06T07:22:35.038320+0000","last_active":"2024-03-06T07:22:35.039303+0000","last_peered":"2024-03-06T07:22:35.039303+0000","last_clean":"2024-03-06T07:22:35.039303+0000","last_became_active":"2024-03-06T07:22:35.038043+0000","last_became_peered":"2024-03-06T07:22:35.038043+0000","last_unstale":"2024-03-06T07:22:35.039303+0000","last_undegraded":"2024-03-06T07:22:35.039303+0000","last_fullsized":"2024-03-06T07:22:35.039303+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:27.011065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.041743+0000","last_change":"2024-03-06T07:22:35.040651+0000","last_active":"2024-03-06T07:22:35.041743+0000","last_peered":"2024-03-06T07:22:35.041743+0000","last_clean":"2024-03-06T07:22:35.041743+0000","last_became_active":"2024-03-06T07:22:35.040382+0000","last_became_peered":"2024-03-06T07:22:35.040382+0000","last_unstale":"2024-03-06T07:22:35.041743+0000","last_undegraded":"2024-03-06T07:22:35.041743+0000","last_fullsized":"2024-03-06T07:22:35.041743+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:16.188352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040761+0000","last_change":"2024-03-06T07:22:35.035524+0000","last_active":"2024-03-06T07:22:35.040761+0000","last_peered":"2024-03-06T07:22:35.040761+0000","last_clean":"2024-03-06T07:22:35.040761+0000","last_became_active":"2024-03-06T07:22:35.035252+0000","last_became_peered":"2024-03-06T07:22:35.035252+0000","last_unstale":"2024-03-06T07:22:35.040761+0000","last_undegraded":"2024-03-06T07:22:35.040761+0000","last_fullsized":"2024-03-06T07:22:35.040761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:10.686677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044689+0000","last_change":"2024-03-06T07:22:35.040188+0000","last_active":"2024-03-06T07:22:35.044689+0000","last_peered":"2024-03-06T07:22:35.044689+0000","last_clean":"2024-03-06T07:22:35.044689+0000","last_became_active":"2024-03-06T07:22:35.039868+0000","last_became_peered":"2024-03-06T07:22:35.039868+0000","last_unstale":"2024-03-06T07:22:35.044689+0000","last_undegraded":"2024-03-06T07:22:35.044689+0000","last_fullsized":"2024-03-06T07:22:35.044689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:04.009924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.050474+0000","last_change":"2024-03-06T07:22:35.049739+0000","last_active":"2024-03-06T07:22:35.050474+0000","last_peered":"2024-03-06T07:22:35.050474+0000","last_clean":"2024-03-06T07:22:35.050474+0000","last_became_active":"2024-03-06T07:22:35.049118+0000","last_became_peered":"2024-03-06T07:22:35.049118+0000","last_unstale":"2024-03-06T07:22:35.050474+0000","last_undegraded":"2024-03-06T07:22:35.050474+0000","last_fullsized":"2024-03-06T07:22:35.050474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042924+0000","last_change":"2024-03-06T07:22:35.038831+0000","last_active":"2024-03-06T07:22:35.042924+0000","last_peered":"2024-03-06T07:22:35.042924+0000","last_clean":"2024-03-06T07:22:35.042924+0000","last_became_active":"2024-03-06T07:22:35.038550+0000","last_became_peered":"2024-03-06T07:22:35.038550+0000","last_unstale":"2024-03-06T07:22:35.042924+0000","last_undegraded":"2024-03-06T07:22:35.042924+0000","last_fullsized":"2024-03-06T07:22:35.042924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:48.293048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039490+0000","last_change":"2024-03-06T07:22:35.037819+0000","last_active":"2024-03-06T07:22:35.039490+0000","last_peered":"2024-03-06T07:22:35.039490+0000","last_clean":"2024-03-06T07:22:35.039490+0000","last_became_active":"2024-03-06T07:22:35.037519+0000","last_became_peered":"2024-03-06T07:22:35.037519+0000","last_unstale":"2024-03-06T07:22:35.039490+0000","last_undegraded":"2024-03-06T07:22:35.039490+0000","last_fullsized":"2024-03-06T07:22:35.039490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:34.907499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.041507+0000","last_change":"2024-03-06T07:22:35.040052+0000","last_active":"2024-03-06T07:22:35.041507+0000","last_peered":"2024-03-06T07:22:35.041507+0000","last_clean":"2024-03-06T07:22:35.041507+0000","last_became_active":"2024-03-06T07:22:35.039794+0000","last_became_peered":"2024-03-06T07:22:35.039794+0000","last_unstale":"2024-03-06T07:22:35.041507+0000","last_undegraded":"2024-03-06T07:22:35.041507+0000","last_fullsized":"2024-03-06T07:22:35.041507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038208+0000","last_change":"2024-03-06T07:22:35.034108+0000","last_active":"2024-03-06T07:22:35.038208+0000","last_peered":"2024-03-06T07:22:35.038208+0000","last_clean":"2024-03-06T07:22:35.038208+0000","last_became_active":"2024-03-06T07:22:35.033823+0000","last_became_peered":"2024-03-06T07:22:35.033823+0000","last_unstale":"2024-03-06T07:22:35.038208+0000","last_undegraded":"2024-03-06T07:22:35.038208+0000","last_fullsized":"2024-03-06T07:22:35.038208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:55.082141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044523+0000","last_change":"2024-03-06T07:22:35.037317+0000","last_active":"2024-03-06T07:22:35.044523+0000","last_peered":"2024-03-06T07:22:35.044523+0000","last_clean":"2024-03-06T07:22:35.044523+0000","last_became_active":"2024-03-06T07:22:35.033185+0000","last_became_peered":"2024-03-06T07:22:35.033185+0000","last_unstale":"2024-03-06T07:22:35.044523+0000","last_undegraded":"2024-03-06T07:22:35.044523+0000","last_fullsized":"2024-03-06T07:22:35.044523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037970+0000","last_change":"2024-03-06T07:22:35.035073+0000","last_active":"2024-03-06T07:22:35.037970+0000","last_peered":"2024-03-06T07:22:35.037970+0000","last_clean":"2024-03-06T07:22:35.037970+0000","last_became_active":"2024-03-06T07:22:35.034739+0000","last_became_peered":"2024-03-06T07:22:35.034739+0000","last_unstale":"2024-03-06T07:22:35.037970+0000","last_undegraded":"2024-03-06T07:22:35.037970+0000","last_fullsized":"2024-03-06T07:22:35.037970+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:55:11.913435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.033258+0000","last_change":"2024-03-06T07:22:35.032202+0000","last_active":"2024-03-06T07:22:35.033258+0000","last_peered":"2024-03-06T07:22:35.033258+0000","last_clean":"2024-03-06T07:22:35.033258+0000","last_became_active":"2024-03-06T07:22:35.031858+0000","last_became_peered":"2024-03-06T07:22:35.031858+0000","last_unstale":"2024-03-06T07:22:35.033258+0000","last_undegraded":"2024-03-06T07:22:35.033258+0000","last_fullsized":"2024-03-06T07:22:35.033258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046626+0000","last_change":"2024-03-06T07:22:35.041928+0000","last_active":"2024-03-06T07:22:35.046626+0000","last_peered":"2024-03-06T07:22:35.046626+0000","last_clean":"2024-03-06T07:22:35.046626+0000","last_became_active":"2024-03-06T07:22:35.041672+0000","last_became_peered":"2024-03-06T07:22:35.041672+0000","last_unstale":"2024-03-06T07:22:35.046626+0000","last_undegraded":"2024-03-06T07:22:35.046626+0000","last_fullsized":"2024-03-06T07:22:35.046626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038191+0000","last_change":"2024-03-06T07:22:35.036178+0000","last_active":"2024-03-06T07:22:35.038191+0000","last_peered":"2024-03-06T07:22:35.038191+0000","last_clean":"2024-03-06T07:22:35.038191+0000","last_became_active":"2024-03-06T07:22:35.035874+0000","last_became_peered":"2024-03-06T07:22:35.035874+0000","last_unstale":"2024-03-06T07:22:35.038191+0000","last_undegraded":"2024-03-06T07:22:35.038191+0000","last_fullsized":"2024-03-06T07:22:35.038191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:53.794103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.036528+0000","last_change":"2024-03-06T07:22:35.031787+0000","last_active":"2024-03-06T07:22:35.036528+0000","last_peered":"2024-03-06T07:22:35.036528+0000","last_clean":"2024-03-06T07:22:35.036528+0000","last_became_active":"2024-03-06T07:22:35.031524+0000","last_became_peered":"2024-03-06T07:22:35.031524+0000","last_unstale":"2024-03-06T07:22:35.036528+0000","last_undegraded":"2024-03-06T07:22:35.036528+0000","last_fullsized":"2024-03-06T07:22:35.036528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:08:42.898690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040976+0000","last_change":"2024-03-06T07:22:35.039311+0000","last_active":"2024-03-06T07:22:35.040976+0000","last_peered":"2024-03-06T07:22:35.040976+0000","last_clean":"2024-03-06T07:22:35.040976+0000","last_became_active":"2024-03-06T07:22:35.038971+0000","last_became_peered":"2024-03-06T07:22:35.038971+0000","last_unstale":"2024-03-06T07:22:35.040976+0000","last_undegraded":"2024-03-06T07:22:35.040976+0000","last_fullsized":"2024-03-06T07:22:35.040976+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:24:40.102058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.036808+0000","last_change":"2024-03-06T07:22:35.031978+0000","last_active":"2024-03-06T07:22:35.036808+0000","last_peered":"2024-03-06T07:22:35.036808+0000","last_clean":"2024-03-06T07:22:35.036808+0000","last_became_active":"2024-03-06T07:22:35.031711+0000","last_became_peered":"2024-03-06T07:22:35.031711+0000","last_unstale":"2024-03-06T07:22:35.036808+0000","last_undegraded":"2024-03-06T07:22:35.036808+0000","last_fullsized":"2024-03-06T07:22:35.036808+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:08.697758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.048359+0000","last_change":"2024-03-06T07:22:35.045711+0000","last_active":"2024-03-06T07:22:35.048359+0000","last_peered":"2024-03-06T07:22:35.048359+0000","last_clean":"2024-03-06T07:22:35.048359+0000","last_became_active":"2024-03-06T07:22:35.041242+0000","last_became_peered":"2024-03-06T07:22:35.041242+0000","last_unstale":"2024-03-06T07:22:35.048359+0000","last_undegraded":"2024-03-06T07:22:35.048359+0000","last_fullsized":"2024-03-06T07:22:35.048359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:43:41.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044446+0000","last_change":"2024-03-06T07:22:35.042301+0000","last_active":"2024-03-06T07:22:35.044446+0000","last_peered":"2024-03-06T07:22:35.044446+0000","last_clean":"2024-03-06T07:22:35.044446+0000","last_became_active":"2024-03-06T07:22:35.041172+0000","last_became_peered":"2024-03-06T07:22:35.041172+0000","last_unstale":"2024-03-06T07:22:35.044446+0000","last_undegraded":"2024-03-06T07:22:35.044446+0000","last_fullsized":"2024-03-06T07:22:35.044446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042090+0000","last_change":"2024-03-06T07:22:35.038117+0000","last_active":"2024-03-06T07:22:35.042090+0000","last_peered":"2024-03-06T07:22:35.042090+0000","last_clean":"2024-03-06T07:22:35.042090+0000","last_became_active":"2024-03-06T07:22:35.037812+0000","last_became_peered":"2024-03-06T07:22:35.037812+0000","last_unstale":"2024-03-06T07:22:35.042090+0000","last_undegraded":"2024-03-06T07:22:35.042090+0000","last_fullsized":"2024-03-06T07:22:35.042090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:18.407646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040449+0000","last_change":"2024-03-06T07:22:35.033425+0000","last_active":"2024-03-06T07:22:35.040449+0000","last_peered":"2024-03-06T07:22:35.040449+0000","last_clean":"2024-03-06T07:22:35.040449+0000","last_became_active":"2024-03-06T07:22:35.032784+0000","last_became_peered":"2024-03-06T07:22:35.032784+0000","last_unstale":"2024-03-06T07:22:35.040449+0000","last_undegraded":"2024-03-06T07:22:35.040449+0000","last_fullsized":"2024-03-06T07:22:35.040449+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038171+0000","last_change":"2024-03-06T07:22:35.033473+0000","last_active":"2024-03-06T07:22:35.038171+0000","last_peered":"2024-03-06T07:22:35.038171+0000","last_clean":"2024-03-06T07:22:35.038171+0000","last_became_active":"2024-03-06T07:22:35.032407+0000","last_became_peered":"2024-03-06T07:22:35.032407+0000","last_unstale":"2024-03-06T07:22:35.038171+0000","last_undegraded":"2024-03-06T07:22:35.038171+0000","last_fullsized":"2024-03-06T07:22:35.038171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:21.244078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049427+0000","last_change":"2024-03-06T07:22:35.048127+0000","last_active":"2024-03-06T07:22:35.049427+0000","last_peered":"2024-03-06T07:22:35.049427+0000","last_clean":"2024-03-06T07:22:35.049427+0000","last_became_active":"2024-03-06T07:22:35.047869+0000","last_became_peered":"2024-03-06T07:22:35.047869+0000","last_unstale":"2024-03-06T07:22:35.049427+0000","last_undegraded":"2024-03-06T07:22:35.049427+0000","last_fullsized":"2024-03-06T07:22:35.049427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044448+0000","last_change":"2024-03-06T07:22:35.038257+0000","last_active":"2024-03-06T07:22:35.044448+0000","last_peered":"2024-03-06T07:22:35.044448+0000","last_clean":"2024-03-06T07:22:35.044448+0000","last_became_active":"2024-03-06T07:22:35.037969+0000","last_became_peered":"2024-03-06T07:22:35.037969+0000","last_unstale":"2024-03-06T07:22:35.044448+0000","last_undegraded":"2024-03-06T07:22:35.044448+0000","last_fullsized":"2024-03-06T07:22:35.044448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:43:41.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.033107+0000","last_change":"2024-03-06T07:22:35.032129+0000","last_active":"2024-03-06T07:22:35.033107+0000","last_peered":"2024-03-06T07:22:35.033107+0000","last_clean":"2024-03-06T07:22:35.033107+0000","last_became_active":"2024-03-06T07:22:35.027787+0000","last_became_peered":"2024-03-06T07:22:35.027787+0000","last_unstale":"2024-03-06T07:22:35.033107+0000","last_undegraded":"2024-03-06T07:22:35.033107+0000","last_fullsized":"2024-03-06T07:22:35.033107+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044483+0000","last_change":"2024-03-06T07:22:35.036989+0000","last_active":"2024-03-06T07:22:35.044483+0000","last_peered":"2024-03-06T07:22:35.044483+0000","last_clean":"2024-03-06T07:22:35.044483+0000","last_became_active":"2024-03-06T07:22:35.032803+0000","last_became_peered":"2024-03-06T07:22:35.032803+0000","last_unstale":"2024-03-06T07:22:35.044483+0000","last_undegraded":"2024-03-06T07:22:35.044483+0000","last_fullsized":"2024-03-06T07:22:35.044483+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.029382+0000","last_change":"2024-03-06T07:22:35.025981+0000","last_active":"2024-03-06T07:22:35.029382+0000","last_peered":"2024-03-06T07:22:35.029382+0000","last_clean":"2024-03-06T07:22:35.029382+0000","last_became_active":"2024-03-06T07:22:35.025721+0000","last_became_peered":"2024-03-06T07:22:35.025721+0000","last_unstale":"2024-03-06T07:22:35.029382+0000","last_undegraded":"2024-03-06T07:22:35.029382+0000","last_fullsized":"2024-03-06T07:22:35.029382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049700+0000","last_change":"2024-03-06T07:22:34.043747+0000","last_active":"2024-03-06T07:22:35.049700+0000","last_peered":"2024-03-06T07:22:35.049700+0000","last_clean":"2024-03-06T07:22:35.049700+0000","last_became_active":"2024-03-06T07:22:34.042367+0000","last_became_peered":"2024-03-06T07:22:34.042367+0000","last_unstale":"2024-03-06T07:22:35.049700+0000","last_undegraded":"2024-03-06T07:22:35.049700+0000","last_fullsized":"2024-03-06T07:22:35.049700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:33.138544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038190+0000","last_change":"2024-03-06T07:22:35.032593+0000","last_active":"2024-03-06T07:22:35.038190+0000","last_peered":"2024-03-06T07:22:35.038190+0000","last_clean":"2024-03-06T07:22:35.038190+0000","last_became_active":"2024-03-06T07:22:35.031651+0000","last_became_peered":"2024-03-06T07:22:35.031651+0000","last_unstale":"2024-03-06T07:22:35.038190+0000","last_undegraded":"2024-03-06T07:22:35.038190+0000","last_fullsized":"2024-03-06T07:22:35.038190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:48.293048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046357+0000","last_change":"2024-03-06T07:22:34.038702+0000","last_active":"2024-03-06T07:22:35.046357+0000","last_peered":"2024-03-06T07:22:35.046357+0000","last_clean":"2024-03-06T07:22:35.046357+0000","last_became_active":"2024-03-06T07:22:34.038165+0000","last_became_peered":"2024-03-06T07:22:34.038165+0000","last_unstale":"2024-03-06T07:22:35.046357+0000","last_undegraded":"2024-03-06T07:22:35.046357+0000","last_fullsized":"2024-03-06T07:22:35.046357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039617+0000","last_change":"2024-03-06T07:22:35.035774+0000","last_active":"2024-03-06T07:22:35.039617+0000","last_peered":"2024-03-06T07:22:35.039617+0000","last_clean":"2024-03-06T07:22:35.039617+0000","last_became_active":"2024-03-06T07:22:35.035468+0000","last_became_peered":"2024-03-06T07:22:35.035468+0000","last_unstale":"2024-03-06T07:22:35.039617+0000","last_undegraded":"2024-03-06T07:22:35.039617+0000","last_fullsized":"2024-03-06T07:22:35.039617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:49.912487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046668+0000","last_change":"2024-03-06T07:22:34.042434+0000","last_active":"2024-03-06T07:22:35.046668+0000","last_peered":"2024-03-06T07:22:35.046668+0000","last_clean":"2024-03-06T07:22:35.046668+0000","last_became_active":"2024-03-06T07:22:34.042093+0000","last_became_peered":"2024-03-06T07:22:34.042093+0000","last_unstale":"2024-03-06T07:22:35.046668+0000","last_undegraded":"2024-03-06T07:22:35.046668+0000","last_fullsized":"2024-03-06T07:22:35.046668+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.036685+0000","last_change":"2024-03-06T07:22:35.032012+0000","last_active":"2024-03-06T07:22:35.036685+0000","last_peered":"2024-03-06T07:22:35.036685+0000","last_clean":"2024-03-06T07:22:35.036685+0000","last_became_active":"2024-03-06T07:22:35.031728+0000","last_became_peered":"2024-03-06T07:22:35.031728+0000","last_unstale":"2024-03-06T07:22:35.036685+0000","last_undegraded":"2024-03-06T07:22:35.036685+0000","last_fullsized":"2024-03-06T07:22:35.036685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:32.785444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046950+0000","last_change":"2024-03-06T07:22:34.038862+0000","last_active":"2024-03-06T07:22:35.046950+0000","last_peered":"2024-03-06T07:22:35.046950+0000","last_clean":"2024-03-06T07:22:35.046950+0000","last_became_active":"2024-03-06T07:22:34.038590+0000","last_became_peered":"2024-03-06T07:22:34.038590+0000","last_unstale":"2024-03-06T07:22:35.046950+0000","last_undegraded":"2024-03-06T07:22:35.046950+0000","last_fullsized":"2024-03-06T07:22:35.046950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:15.997587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040991+0000","last_change":"2024-03-06T07:22:35.036832+0000","last_active":"2024-03-06T07:22:35.040991+0000","last_peered":"2024-03-06T07:22:35.040991+0000","last_clean":"2024-03-06T07:22:35.040991+0000","last_became_active":"2024-03-06T07:22:35.036392+0000","last_became_peered":"2024-03-06T07:22:35.036392+0000","last_unstale":"2024-03-06T07:22:35.040991+0000","last_undegraded":"2024-03-06T07:22:35.040991+0000","last_fullsized":"2024-03-06T07:22:35.040991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.045898+0000","last_change":"2024-03-06T07:22:34.037848+0000","last_active":"2024-03-06T07:22:35.045898+0000","last_peered":"2024-03-06T07:22:35.045898+0000","last_clean":"2024-03-06T07:22:35.045898+0000","last_became_active":"2024-03-06T07:22:34.037572+0000","last_became_peered":"2024-03-06T07:22:34.037572+0000","last_unstale":"2024-03-06T07:22:35.045898+0000","last_undegraded":"2024-03-06T07:22:35.045898+0000","last_fullsized":"2024-03-06T07:22:35.045898+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:15.748729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.043769+0000","last_change":"2024-03-06T07:22:35.040723+0000","last_active":"2024-03-06T07:22:35.043769+0000","last_peered":"2024-03-06T07:22:35.043769+0000","last_clean":"2024-03-06T07:22:35.043769+0000","last_became_active":"2024-03-06T07:22:35.036544+0000","last_became_peered":"2024-03-06T07:22:35.036544+0000","last_unstale":"2024-03-06T07:22:35.043769+0000","last_undegraded":"2024-03-06T07:22:35.043769+0000","last_fullsized":"2024-03-06T07:22:35.043769+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.045551+0000","last_change":"2024-03-06T07:22:34.040746+0000","last_active":"2024-03-06T07:22:35.045551+0000","last_peered":"2024-03-06T07:22:35.045551+0000","last_clean":"2024-03-06T07:22:35.045551+0000","last_became_active":"2024-03-06T07:22:34.040466+0000","last_became_peered":"2024-03-06T07:22:34.040466+0000","last_unstale":"2024-03-06T07:22:35.045551+0000","last_undegraded":"2024-03-06T07:22:35.045551+0000","last_fullsized":"2024-03-06T07:22:35.045551+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:35.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039576+0000","last_change":"2024-03-06T07:22:35.035207+0000","last_active":"2024-03-06T07:22:35.039576+0000","last_peered":"2024-03-06T07:22:35.039576+0000","last_clean":"2024-03-06T07:22:35.039576+0000","last_became_active":"2024-03-06T07:22:35.034887+0000","last_became_peered":"2024-03-06T07:22:35.034887+0000","last_unstale":"2024-03-06T07:22:35.039576+0000","last_undegraded":"2024-03-06T07:22:35.039576+0000","last_fullsized":"2024-03-06T07:22:35.039576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:42.073401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.028684+0000","last_change":"2024-03-06T07:22:34.042833+0000","last_active":"2024-03-06T07:22:35.028684+0000","last_peered":"2024-03-06T07:22:35.028684+0000","last_clean":"2024-03-06T07:22:35.028684+0000","last_became_active":"2024-03-06T07:22:34.042557+0000","last_became_peered":"2024-03-06T07:22:34.042557+0000","last_unstale":"2024-03-06T07:22:35.028684+0000","last_undegraded":"2024-03-06T07:22:35.028684+0000","last_fullsized":"2024-03-06T07:22:35.028684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038879+0000","last_change":"2024-03-06T07:22:35.028293+0000","last_active":"2024-03-06T07:22:35.038879+0000","last_peered":"2024-03-06T07:22:35.038879+0000","last_clean":"2024-03-06T07:22:35.038879+0000","last_became_active":"2024-03-06T07:22:35.028017+0000","last_became_peered":"2024-03-06T07:22:35.028017+0000","last_unstale":"2024-03-06T07:22:35.038879+0000","last_undegraded":"2024-03-06T07:22:35.038879+0000","last_fullsized":"2024-03-06T07:22:35.038879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020218+0000","last_change":"2024-03-06T07:22:34.037660+0000","last_active":"2024-03-06T07:22:35.020218+0000","last_peered":"2024-03-06T07:22:35.020218+0000","last_clean":"2024-03-06T07:22:35.020218+0000","last_became_active":"2024-03-06T07:22:34.037347+0000","last_became_peered":"2024-03-06T07:22:34.037347+0000","last_unstale":"2024-03-06T07:22:35.020218+0000","last_undegraded":"2024-03-06T07:22:35.020218+0000","last_fullsized":"2024-03-06T07:22:35.020218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:43.483410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.048335+0000","last_change":"2024-03-06T07:22:35.045241+0000","last_active":"2024-03-06T07:22:35.048335+0000","last_peered":"2024-03-06T07:22:35.048335+0000","last_clean":"2024-03-06T07:22:35.048335+0000","last_became_active":"2024-03-06T07:22:35.044953+0000","last_became_peered":"2024-03-06T07:22:35.044953+0000","last_unstale":"2024-03-06T07:22:35.048335+0000","last_undegraded":"2024-03-06T07:22:35.048335+0000","last_fullsized":"2024-03-06T07:22:35.048335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.019518+0000","last_change":"2024-03-06T07:22:34.038501+0000","last_active":"2024-03-06T07:22:35.019518+0000","last_peered":"2024-03-06T07:22:35.019518+0000","last_clean":"2024-03-06T07:22:35.019518+0000","last_became_active":"2024-03-06T07:22:34.038238+0000","last_became_peered":"2024-03-06T07:22:34.038238+0000","last_unstale":"2024-03-06T07:22:35.019518+0000","last_undegraded":"2024-03-06T07:22:35.019518+0000","last_fullsized":"2024-03-06T07:22:35.019518+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.041991+0000","last_change":"2024-03-06T07:22:35.035913+0000","last_active":"2024-03-06T07:22:35.041991+0000","last_peered":"2024-03-06T07:22:35.041991+0000","last_clean":"2024-03-06T07:22:35.041991+0000","last_became_active":"2024-03-06T07:22:35.035658+0000","last_became_peered":"2024-03-06T07:22:35.035658+0000","last_unstale":"2024-03-06T07:22:35.041991+0000","last_undegraded":"2024-03-06T07:22:35.041991+0000","last_fullsized":"2024-03-06T07:22:35.041991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:19.634384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.029096+0000","last_change":"2024-03-06T07:22:34.040689+0000","last_active":"2024-03-06T07:22:35.029096+0000","last_peered":"2024-03-06T07:22:35.029096+0000","last_clean":"2024-03-06T07:22:35.029096+0000","last_became_active":"2024-03-06T07:22:34.040410+0000","last_became_peered":"2024-03-06T07:22:34.040410+0000","last_unstale":"2024-03-06T07:22:35.029096+0000","last_undegraded":"2024-03-06T07:22:35.029096+0000","last_fullsized":"2024-03-06T07:22:35.029096+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.281130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.034305+0000","last_change":"2024-03-06T07:22:35.032615+0000","last_active":"2024-03-06T07:22:35.034305+0000","last_peered":"2024-03-06T07:22:35.034305+0000","last_clean":"2024-03-06T07:22:35.034305+0000","last_became_active":"2024-03-06T07:22:35.031167+0000","last_became_peered":"2024-03-06T07:22:35.031167+0000","last_unstale":"2024-03-06T07:22:35.034305+0000","last_undegraded":"2024-03-06T07:22:35.034305+0000","last_fullsized":"2024-03-06T07:22:35.034305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.050647+0000","last_change":"2024-03-06T07:22:34.038406+0000","last_active":"2024-03-06T07:22:35.050647+0000","last_peered":"2024-03-06T07:22:35.050647+0000","last_clean":"2024-03-06T07:22:35.050647+0000","last_became_active":"2024-03-06T07:22:34.038040+0000","last_became_peered":"2024-03-06T07:22:34.038040+0000","last_unstale":"2024-03-06T07:22:35.050647+0000","last_undegraded":"2024-03-06T07:22:35.050647+0000","last_fullsized":"2024-03-06T07:22:35.050647+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:26.063950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040485+0000","last_change":"2024-03-06T07:22:35.038459+0000","last_active":"2024-03-06T07:22:35.040485+0000","last_peered":"2024-03-06T07:22:35.040485+0000","last_clean":"2024-03-06T07:22:35.040485+0000","last_became_active":"2024-03-06T07:22:35.038185+0000","last_became_peered":"2024-03-06T07:22:35.038185+0000","last_unstale":"2024-03-06T07:22:35.040485+0000","last_undegraded":"2024-03-06T07:22:35.040485+0000","last_fullsized":"2024-03-06T07:22:35.040485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:05.629298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049070+0000","last_change":"2024-03-06T07:22:34.037962+0000","last_active":"2024-03-06T07:22:35.049070+0000","last_peered":"2024-03-06T07:22:35.049070+0000","last_clean":"2024-03-06T07:22:35.049070+0000","last_became_active":"2024-03-06T07:22:34.037635+0000","last_became_peered":"2024-03-06T07:22:34.037635+0000","last_unstale":"2024-03-06T07:22:35.049070+0000","last_undegraded":"2024-03-06T07:22:35.049070+0000","last_fullsized":"2024-03-06T07:22:35.049070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.042905+0000","last_change":"2024-03-06T07:22:35.036674+0000","last_active":"2024-03-06T07:22:35.042905+0000","last_peered":"2024-03-06T07:22:35.042905+0000","last_clean":"2024-03-06T07:22:35.042905+0000","last_became_active":"2024-03-06T07:22:35.036400+0000","last_became_peered":"2024-03-06T07:22:35.036400+0000","last_unstale":"2024-03-06T07:22:35.042905+0000","last_undegraded":"2024-03-06T07:22:35.042905+0000","last_fullsized":"2024-03-06T07:22:35.042905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:55.948724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046183+0000","last_change":"2024-03-06T07:22:34.041311+0000","last_active":"2024-03-06T07:22:35.046183+0000","last_peered":"2024-03-06T07:22:35.046183+0000","last_clean":"2024-03-06T07:22:35.046183+0000","last_became_active":"2024-03-06T07:22:34.040940+0000","last_became_peered":"2024-03-06T07:22:34.040940+0000","last_unstale":"2024-03-06T07:22:35.046183+0000","last_undegraded":"2024-03-06T07:22:35.046183+0000","last_fullsized":"2024-03-06T07:22:35.046183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:24.924844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.024503+0000","last_change":"2024-03-06T07:22:35.022953+0000","last_active":"2024-03-06T07:22:35.024503+0000","last_peered":"2024-03-06T07:22:35.024503+0000","last_clean":"2024-03-06T07:22:35.024503+0000","last_became_active":"2024-03-06T07:22:35.022681+0000","last_became_peered":"2024-03-06T07:22:35.022681+0000","last_unstale":"2024-03-06T07:22:35.024503+0000","last_undegraded":"2024-03-06T07:22:35.024503+0000","last_fullsized":"2024-03-06T07:22:35.024503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020602+0000","last_change":"2024-03-06T07:22:34.037840+0000","last_active":"2024-03-06T07:22:35.020602+0000","last_peered":"2024-03-06T07:22:35.020602+0000","last_clean":"2024-03-06T07:22:35.020602+0000","last_became_active":"2024-03-06T07:22:34.037556+0000","last_became_peered":"2024-03-06T07:22:34.037556+0000","last_unstale":"2024-03-06T07:22:35.020602+0000","last_undegraded":"2024-03-06T07:22:35.020602+0000","last_fullsized":"2024-03-06T07:22:35.020602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038768+0000","last_change":"2024-03-06T07:22:35.034503+0000","last_active":"2024-03-06T07:22:35.038768+0000","last_peered":"2024-03-06T07:22:35.038768+0000","last_clean":"2024-03-06T07:22:35.038768+0000","last_became_active":"2024-03-06T07:22:35.034230+0000","last_became_peered":"2024-03-06T07:22:35.034230+0000","last_unstale":"2024-03-06T07:22:35.038768+0000","last_undegraded":"2024-03-06T07:22:35.038768+0000","last_fullsized":"2024-03-06T07:22:35.038768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:05.629298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020787+0000","last_change":"2024-03-06T07:22:34.039959+0000","last_active":"2024-03-06T07:22:35.020787+0000","last_peered":"2024-03-06T07:22:35.020787+0000","last_clean":"2024-03-06T07:22:35.020787+0000","last_became_active":"2024-03-06T07:22:34.039689+0000","last_became_peered":"2024-03-06T07:22:34.039689+0000","last_unstale":"2024-03-06T07:22:35.020787+0000","last_undegraded":"2024-03-06T07:22:35.020787+0000","last_fullsized":"2024-03-06T07:22:35.020787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:23.420842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038920+0000","last_change":"2024-03-06T07:22:35.032290+0000","last_active":"2024-03-06T07:22:35.038920+0000","last_peered":"2024-03-06T07:22:35.038920+0000","last_clean":"2024-03-06T07:22:35.038920+0000","last_became_active":"2024-03-06T07:22:35.032048+0000","last_became_peered":"2024-03-06T07:22:35.032048+0000","last_unstale":"2024-03-06T07:22:35.038920+0000","last_undegraded":"2024-03-06T07:22:35.038920+0000","last_fullsized":"2024-03-06T07:22:35.038920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:21.230955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.051866+0000","last_change":"2024-03-06T07:22:34.043149+0000","last_active":"2024-03-06T07:22:35.051866+0000","last_peered":"2024-03-06T07:22:35.051866+0000","last_clean":"2024-03-06T07:22:35.051866+0000","last_became_active":"2024-03-06T07:22:34.042888+0000","last_became_peered":"2024-03-06T07:22:34.042888+0000","last_unstale":"2024-03-06T07:22:35.051866+0000","last_undegraded":"2024-03-06T07:22:35.051866+0000","last_fullsized":"2024-03-06T07:22:35.051866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.035965+0000","last_change":"2024-03-06T07:22:35.034984+0000","last_active":"2024-03-06T07:22:35.035965+0000","last_peered":"2024-03-06T07:22:35.035965+0000","last_clean":"2024-03-06T07:22:35.035965+0000","last_became_active":"2024-03-06T07:22:35.031303+0000","last_became_peered":"2024-03-06T07:22:35.031303+0000","last_unstale":"2024-03-06T07:22:35.035965+0000","last_undegraded":"2024-03-06T07:22:35.035965+0000","last_fullsized":"2024-03-06T07:22:35.035965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046601+0000","last_change":"2024-03-06T07:22:34.044320+0000","last_active":"2024-03-06T07:22:35.046601+0000","last_peered":"2024-03-06T07:22:35.046601+0000","last_clean":"2024-03-06T07:22:35.046601+0000","last_became_active":"2024-03-06T07:22:34.043832+0000","last_became_peered":"2024-03-06T07:22:34.043832+0000","last_unstale":"2024-03-06T07:22:35.046601+0000","last_undegraded":"2024-03-06T07:22:35.046601+0000","last_fullsized":"2024-03-06T07:22:35.046601+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037938+0000","last_change":"2024-03-06T07:22:35.034203+0000","last_active":"2024-03-06T07:22:35.037938+0000","last_peered":"2024-03-06T07:22:35.037938+0000","last_clean":"2024-03-06T07:22:35.037938+0000","last_became_active":"2024-03-06T07:22:35.033903+0000","last_became_peered":"2024-03-06T07:22:35.033903+0000","last_unstale":"2024-03-06T07:22:35.037938+0000","last_undegraded":"2024-03-06T07:22:35.037938+0000","last_fullsized":"2024-03-06T07:22:35.037938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:19:07.387824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.045282+0000","last_change":"2024-03-06T07:22:34.036176+0000","last_active":"2024-03-06T07:22:35.045282+0000","last_peered":"2024-03-06T07:22:35.045282+0000","last_clean":"2024-03-06T07:22:35.045282+0000","last_became_active":"2024-03-06T07:22:34.035867+0000","last_became_peered":"2024-03-06T07:22:34.035867+0000","last_unstale":"2024-03-06T07:22:35.045282+0000","last_undegraded":"2024-03-06T07:22:35.045282+0000","last_fullsized":"2024-03-06T07:22:35.045282+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:28.336495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040955+0000","last_change":"2024-03-06T07:22:35.038497+0000","last_active":"2024-03-06T07:22:35.040955+0000","last_peered":"2024-03-06T07:22:35.040955+0000","last_clean":"2024-03-06T07:22:35.040955+0000","last_became_active":"2024-03-06T07:22:35.038189+0000","last_became_peered":"2024-03-06T07:22:35.038189+0000","last_unstale":"2024-03-06T07:22:35.040955+0000","last_undegraded":"2024-03-06T07:22:35.040955+0000","last_fullsized":"2024-03-06T07:22:35.040955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:40.330097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.049723+0000","last_change":"2024-03-06T07:22:34.038423+0000","last_active":"2024-03-06T07:22:35.049723+0000","last_peered":"2024-03-06T07:22:35.049723+0000","last_clean":"2024-03-06T07:22:35.049723+0000","last_became_active":"2024-03-06T07:22:34.038126+0000","last_became_peered":"2024-03-06T07:22:34.038126+0000","last_unstale":"2024-03-06T07:22:35.049723+0000","last_undegraded":"2024-03-06T07:22:35.049723+0000","last_fullsized":"2024-03-06T07:22:35.049723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:43.441029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046669+0000","last_change":"2024-03-06T07:22:35.042660+0000","last_active":"2024-03-06T07:22:35.046669+0000","last_peered":"2024-03-06T07:22:35.046669+0000","last_clean":"2024-03-06T07:22:35.046669+0000","last_became_active":"2024-03-06T07:22:35.042327+0000","last_became_peered":"2024-03-06T07:22:35.042327+0000","last_unstale":"2024-03-06T07:22:35.046669+0000","last_undegraded":"2024-03-06T07:22:35.046669+0000","last_fullsized":"2024-03-06T07:22:35.046669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:43:41.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020834+0000","last_change":"2024-03-06T07:22:34.035505+0000","last_active":"2024-03-06T07:22:35.020834+0000","last_peered":"2024-03-06T07:22:35.020834+0000","last_clean":"2024-03-06T07:22:35.020834+0000","last_became_active":"2024-03-06T07:22:34.035257+0000","last_became_peered":"2024-03-06T07:22:34.035257+0000","last_unstale":"2024-03-06T07:22:35.020834+0000","last_undegraded":"2024-03-06T07:22:35.020834+0000","last_fullsized":"2024-03-06T07:22:35.020834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040969+0000","last_change":"2024-03-06T07:22:35.032174+0000","last_active":"2024-03-06T07:22:35.040969+0000","last_peered":"2024-03-06T07:22:35.040969+0000","last_clean":"2024-03-06T07:22:35.040969+0000","last_became_active":"2024-03-06T07:22:35.031869+0000","last_became_peered":"2024-03-06T07:22:35.031869+0000","last_unstale":"2024-03-06T07:22:35.040969+0000","last_undegraded":"2024-03-06T07:22:35.040969+0000","last_fullsized":"2024-03-06T07:22:35.040969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:18.407646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046790+0000","last_change":"2024-03-06T07:22:34.037475+0000","last_active":"2024-03-06T07:22:35.046790+0000","last_peered":"2024-03-06T07:22:35.046790+0000","last_clean":"2024-03-06T07:22:35.046790+0000","last_became_active":"2024-03-06T07:22:34.037206+0000","last_became_peered":"2024-03-06T07:22:34.037206+0000","last_unstale":"2024-03-06T07:22:35.046790+0000","last_undegraded":"2024-03-06T07:22:35.046790+0000","last_fullsized":"2024-03-06T07:22:35.046790+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.044530+0000","last_change":"2024-03-06T07:22:35.042271+0000","last_active":"2024-03-06T07:22:35.044530+0000","last_peered":"2024-03-06T07:22:35.044530+0000","last_clean":"2024-03-06T07:22:35.044530+0000","last_became_active":"2024-03-06T07:22:35.036189+0000","last_became_peered":"2024-03-06T07:22:35.036189+0000","last_unstale":"2024-03-06T07:22:35.044530+0000","last_undegraded":"2024-03-06T07:22:35.044530+0000","last_fullsized":"2024-03-06T07:22:35.044530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.025415+0000","last_change":"2024-03-06T07:22:34.036505+0000","last_active":"2024-03-06T07:22:35.025415+0000","last_peered":"2024-03-06T07:22:35.025415+0000","last_clean":"2024-03-06T07:22:35.025415+0000","last_became_active":"2024-03-06T07:22:34.036198+0000","last_became_peered":"2024-03-06T07:22:34.036198+0000","last_unstale":"2024-03-06T07:22:35.025415+0000","last_undegraded":"2024-03-06T07:22:35.025415+0000","last_fullsized":"2024-03-06T07:22:35.025415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038901+0000","last_change":"2024-03-06T07:22:35.031196+0000","last_active":"2024-03-06T07:22:35.038901+0000","last_peered":"2024-03-06T07:22:35.038901+0000","last_clean":"2024-03-06T07:22:35.038901+0000","last_became_active":"2024-03-06T07:22:35.030767+0000","last_became_peered":"2024-03-06T07:22:35.030767+0000","last_unstale":"2024-03-06T07:22:35.038901+0000","last_undegraded":"2024-03-06T07:22:35.038901+0000","last_fullsized":"2024-03-06T07:22:35.038901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.025346+0000","last_change":"2024-03-06T07:22:34.034863+0000","last_active":"2024-03-06T07:22:35.025346+0000","last_peered":"2024-03-06T07:22:35.025346+0000","last_clean":"2024-03-06T07:22:35.025346+0000","last_became_active":"2024-03-06T07:22:34.034577+0000","last_became_peered":"2024-03-06T07:22:34.034577+0000","last_unstale":"2024-03-06T07:22:35.025346+0000","last_undegraded":"2024-03-06T07:22:35.025346+0000","last_fullsized":"2024-03-06T07:22:35.025346+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040765+0000","last_change":"2024-03-06T07:22:35.036664+0000","last_active":"2024-03-06T07:22:35.040765+0000","last_peered":"2024-03-06T07:22:35.040765+0000","last_clean":"2024-03-06T07:22:35.040765+0000","last_became_active":"2024-03-06T07:22:35.036379+0000","last_became_peered":"2024-03-06T07:22:35.036379+0000","last_unstale":"2024-03-06T07:22:35.040765+0000","last_undegraded":"2024-03-06T07:22:35.040765+0000","last_fullsized":"2024-03-06T07:22:35.040765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:42.968699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.046261+0000","last_change":"2024-03-06T07:22:34.044993+0000","last_active":"2024-03-06T07:22:35.046261+0000","last_peered":"2024-03-06T07:22:35.046261+0000","last_clean":"2024-03-06T07:22:35.046261+0000","last_became_active":"2024-03-06T07:22:34.044795+0000","last_became_peered":"2024-03-06T07:22:34.044795+0000","last_unstale":"2024-03-06T07:22:35.046261+0000","last_undegraded":"2024-03-06T07:22:35.046261+0000","last_fullsized":"2024-03-06T07:22:35.046261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.256142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038745+0000","last_change":"2024-03-06T07:22:35.031505+0000","last_active":"2024-03-06T07:22:35.038745+0000","last_peered":"2024-03-06T07:22:35.038745+0000","last_clean":"2024-03-06T07:22:35.038745+0000","last_became_active":"2024-03-06T07:22:35.031241+0000","last_became_peered":"2024-03-06T07:22:35.031241+0000","last_unstale":"2024-03-06T07:22:35.038745+0000","last_undegraded":"2024-03-06T07:22:35.038745+0000","last_fullsized":"2024-03-06T07:22:35.038745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.028828+0000","last_change":"2024-03-06T07:22:34.044453+0000","last_active":"2024-03-06T07:22:35.028828+0000","last_peered":"2024-03-06T07:22:35.028828+0000","last_clean":"2024-03-06T07:22:35.028828+0000","last_became_active":"2024-03-06T07:22:34.044190+0000","last_became_peered":"2024-03-06T07:22:34.044190+0000","last_unstale":"2024-03-06T07:22:35.028828+0000","last_undegraded":"2024-03-06T07:22:35.028828+0000","last_fullsized":"2024-03-06T07:22:35.028828+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039595+0000","last_change":"2024-03-06T07:22:35.034792+0000","last_active":"2024-03-06T07:22:35.039595+0000","last_peered":"2024-03-06T07:22:35.039595+0000","last_clean":"2024-03-06T07:22:35.039595+0000","last_became_active":"2024-03-06T07:22:35.034503+0000","last_became_peered":"2024-03-06T07:22:35.034503+0000","last_unstale":"2024-03-06T07:22:35.039595+0000","last_undegraded":"2024-03-06T07:22:35.039595+0000","last_fullsized":"2024-03-06T07:22:35.039595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:30.130566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020885+0000","last_change":"2024-03-06T07:22:34.037188+0000","last_active":"2024-03-06T07:22:35.020885+0000","last_peered":"2024-03-06T07:22:35.020885+0000","last_clean":"2024-03-06T07:22:35.020885+0000","last_became_active":"2024-03-06T07:22:34.036918+0000","last_became_peered":"2024-03-06T07:22:34.036918+0000","last_unstale":"2024-03-06T07:22:35.020885+0000","last_undegraded":"2024-03-06T07:22:35.020885+0000","last_fullsized":"2024-03-06T07:22:35.020885+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.038162+0000","last_change":"2024-03-06T07:22:35.033263+0000","last_active":"2024-03-06T07:22:35.038162+0000","last_peered":"2024-03-06T07:22:35.038162+0000","last_clean":"2024-03-06T07:22:35.038162+0000","last_became_active":"2024-03-06T07:22:35.033005+0000","last_became_peered":"2024-03-06T07:22:35.033005+0000","last_unstale":"2024-03-06T07:22:35.038162+0000","last_undegraded":"2024-03-06T07:22:35.038162+0000","last_fullsized":"2024-03-06T07:22:35.038162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:04.009924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.020331+0000","last_change":"2024-03-06T07:22:34.035735+0000","last_active":"2024-03-06T07:22:35.020331+0000","last_peered":"2024-03-06T07:22:35.020331+0000","last_clean":"2024-03-06T07:22:35.020331+0000","last_became_active":"2024-03-06T07:22:34.035459+0000","last_became_peered":"2024-03-06T07:22:34.035459+0000","last_unstale":"2024-03-06T07:22:35.020331+0000","last_undegraded":"2024-03-06T07:22:35.020331+0000","last_fullsized":"2024-03-06T07:22:35.020331+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.039257+0000","last_change":"2024-03-06T07:22:35.038111+0000","last_active":"2024-03-06T07:22:35.039257+0000","last_peered":"2024-03-06T07:22:35.039257+0000","last_clean":"2024-03-06T07:22:35.039257+0000","last_became_active":"2024-03-06T07:22:35.037779+0000","last_became_peered":"2024-03-06T07:22:35.037779+0000","last_unstale":"2024-03-06T07:22:35.039257+0000","last_undegraded":"2024-03-06T07:22:35.039257+0000","last_fullsized":"2024-03-06T07:22:35.039257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:18.894659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.017935+0000","last_change":"2024-03-06T07:22:34.039910+0000","last_active":"2024-03-06T07:22:35.017935+0000","last_peered":"2024-03-06T07:22:35.017935+0000","last_clean":"2024-03-06T07:22:35.017935+0000","last_became_active":"2024-03-06T07:22:34.039246+0000","last_became_peered":"2024-03-06T07:22:34.039246+0000","last_unstale":"2024-03-06T07:22:35.017935+0000","last_undegraded":"2024-03-06T07:22:35.017935+0000","last_fullsized":"2024-03-06T07:22:35.017935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:41.961694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.037669+0000","last_change":"2024-03-06T07:22:35.027657+0000","last_active":"2024-03-06T07:22:35.037669+0000","last_peered":"2024-03-06T07:22:35.037669+0000","last_clean":"2024-03-06T07:22:35.037669+0000","last_became_active":"2024-03-06T07:22:35.027404+0000","last_became_peered":"2024-03-06T07:22:35.027404+0000","last_unstale":"2024-03-06T07:22:35.037669+0000","last_undegraded":"2024-03-06T07:22:35.037669+0000","last_fullsized":"2024-03-06T07:22:35.037669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.022151+0000","last_change":"2024-03-06T07:22:34.039209+0000","last_active":"2024-03-06T07:22:35.022151+0000","last_peered":"2024-03-06T07:22:35.022151+0000","last_clean":"2024-03-06T07:22:35.022151+0000","last_became_active":"2024-03-06T07:22:34.038921+0000","last_became_peered":"2024-03-06T07:22:34.038921+0000","last_unstale":"2024-03-06T07:22:35.022151+0000","last_undegraded":"2024-03-06T07:22:35.022151+0000","last_fullsized":"2024-03-06T07:22:35.022151+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:12.359435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040218+0000","last_change":"2024-03-06T07:22:35.038010+0000","last_active":"2024-03-06T07:22:35.040218+0000","last_peered":"2024-03-06T07:22:35.040218+0000","last_clean":"2024-03-06T07:22:35.040218+0000","last_became_active":"2024-03-06T07:22:35.037717+0000","last_became_peered":"2024-03-06T07:22:35.037717+0000","last_unstale":"2024-03-06T07:22:35.040218+0000","last_undegraded":"2024-03-06T07:22:35.040218+0000","last_fullsized":"2024-03-06T07:22:35.040218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:21.230955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.019874+0000","last_change":"2024-03-06T07:22:34.039679+0000","last_active":"2024-03-06T07:22:35.019874+0000","last_peered":"2024-03-06T07:22:35.019874+0000","last_clean":"2024-03-06T07:22:35.019874+0000","last_became_active":"2024-03-06T07:22:34.039092+0000","last_became_peered":"2024-03-06T07:22:34.039092+0000","last_unstale":"2024-03-06T07:22:35.019874+0000","last_undegraded":"2024-03-06T07:22:35.019874+0000","last_fullsized":"2024-03-06T07:22:35.019874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.029321+0000","last_change":"2024-03-06T07:22:35.026900+0000","last_active":"2024-03-06T07:22:35.029321+0000","last_peered":"2024-03-06T07:22:35.029321+0000","last_clean":"2024-03-06T07:22:35.029321+0000","last_became_active":"2024-03-06T07:22:35.026626+0000","last_became_peered":"2024-03-06T07:22:35.026626+0000","last_unstale":"2024-03-06T07:22:35.029321+0000","last_undegraded":"2024-03-06T07:22:35.029321+0000","last_fullsized":"2024-03-06T07:22:35.029321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.021043+0000","last_change":"2024-03-06T07:22:34.039382+0000","last_active":"2024-03-06T07:22:35.021043+0000","last_peered":"2024-03-06T07:22:35.021043+0000","last_clean":"2024-03-06T07:22:35.021043+0000","last_became_active":"2024-03-06T07:22:34.039088+0000","last_became_peered":"2024-03-06T07:22:34.039088+0000","last_unstale":"2024-03-06T07:22:35.021043+0000","last_undegraded":"2024-03-06T07:22:35.021043+0000","last_fullsized":"2024-03-06T07:22:35.021043+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.040476+0000","last_change":"2024-03-06T07:22:35.033671+0000","last_active":"2024-03-06T07:22:35.040476+0000","last_peered":"2024-03-06T07:22:35.040476+0000","last_clean":"2024-03-06T07:22:35.040476+0000","last_became_active":"2024-03-06T07:22:35.033167+0000","last_became_peered":"2024-03-06T07:22:35.033167+0000","last_unstale":"2024-03-06T07:22:35.040476+0000","last_undegraded":"2024-03-06T07:22:35.040476+0000","last_fullsized":"2024-03-06T07:22:35.040476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.026424+0000","last_change":"2024-03-06T07:22:34.037026+0000","last_active":"2024-03-06T07:22:35.026424+0000","last_peered":"2024-03-06T07:22:35.026424+0000","last_clean":"2024-03-06T07:22:35.026424+0000","last_became_active":"2024-03-06T07:22:34.036742+0000","last_became_peered":"2024-03-06T07:22:34.036742+0000","last_unstale":"2024-03-06T07:22:35.026424+0000","last_undegraded":"2024-03-06T07:22:35.026424+0000","last_fullsized":"2024-03-06T07:22:35.026424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":270,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.017876+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:22:35.017876+0000","last_peered":"2024-03-06T07:22:35.017876+0000","last_clean":"2024-03-06T07:22:35.017876+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:22:35.017876+0000","last_undegraded":"2024-03-06T07:22:35.017876+0000","last_fullsized":"2024-03-06T07:22:35.017876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":1004013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":413645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":1004013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:38.377 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:22:38.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:22:38.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.519+0000 7f80bedd0640 1 Processor -- start 2024-03-06T07:22:38.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.519+0000 7f80bedd0640 1 -- start start 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80bedd0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80b81529d0 0x7f80b80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80bedd0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80b80c57e0 0x7f80b8155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80bedd0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b8152db0 con 0x7f80b8072cd0 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80bedd0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80c6dd0 con 0x7f80b80c57e0 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80bedd0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80c6940 con 0x7f80b81529d0 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80b80c57e0 0x7f80b8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80b80c57e0 0x7f80b8155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34216/0 (socket says 172.21.15.26:34216) 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80b7fff640 1 -- 172.21.15.26:0/2691895436 learned_addr learned my addr 172.21.15.26:0/2691895436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.520+0000 7f80bcb6c640 1 --2- 172.21.15.26:0/2691895436 >> v2:172.21.15.117:3300/0 conn(0x7f80b81529d0 0x7f80b80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.521+0000 7f80b7fff640 1 -- 172.21.15.26:0/2691895436 >> v2:172.21.15.117:3300/0 conn(0x7f80b81529d0 msgr2=0x7f80b80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.521+0000 7f80b7fff640 1 --2- 172.21.15.26:0/2691895436 >> v2:172.21.15.117:3300/0 conn(0x7f80b81529d0 0x7f80b80c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.521+0000 7f80b7fff640 1 -- 172.21.15.26:0/2691895436 >> v1:172.21.15.117:6789/0 conn(0x7f80b8072cd0 legacy=0x7f80b8155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.521+0000 7f80b7fff640 1 -- 172.21.15.26:0/2691895436 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b81590f0 con 0x7f80b80c57e0 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.521+0000 7f80b7fff640 1 --2- 172.21.15.26:0/2691895436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b80c57e0 0x7f80b8155120 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f80b80c98d0 tx=0x7f80a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cf1fcb2b26667a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80b77fe640 1 -- 172.21.15.26:0/2691895436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a402e070 con 0x7f80b80c57e0 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80b77fe640 1 -- 172.21.15.26:0/2691895436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4037070 con 0x7f80b80c57e0 2024-03-06T07:22:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80b77fe640 1 -- 172.21.15.26:0/2691895436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4031a00 con 0x7f80b80c57e0 2024-03-06T07:22:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 -- 172.21.15.26:0/2691895436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b80c57e0 msgr2=0x7f80b8155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 --2- 172.21.15.26:0/2691895436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b80c57e0 0x7f80b8155120 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f80b80c98d0 tx=0x7f80a402b880 comp rx=0 tx=0).stop 2024-03-06T07:22:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 -- 172.21.15.26:0/2691895436 shutdown_connections 2024-03-06T07:22:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 --2- 172.21.15.26:0/2691895436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b80c57e0 0x7f80b8155120 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 --2- 172.21.15.26:0/2691895436 >> v2:172.21.15.117:3300/0 conn(0x7f80b81529d0 0x7f80b80c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 -- 172.21.15.26:0/2691895436 >> 172.21.15.26:0/2691895436 conn(0x7f80b80aa630 msgr2=0x7f80b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 -- 172.21.15.26:0/2691895436 shutdown_connections 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.522+0000 7f80bedd0640 1 -- 172.21.15.26:0/2691895436 wait complete. 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.523+0000 7f80bedd0640 1 Processor -- start 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.523+0000 7f80bedd0640 1 -- start start 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bedd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b8072cd0 0x7f80b81eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bedd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b80c57e0 0x7f80b806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bedd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b81529d0 0x7f80b81eab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bedd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8072720 con 0x7f80b8072cd0 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bedd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8155d70 con 0x7f80b80c57e0 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bedd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8159370 con 0x7f80b81529d0 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b8072cd0 0x7f80b81eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b8072cd0 0x7f80b81eebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34230/0 (socket says 172.21.15.26:34230) 2024-03-06T07:22:38.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 -- 172.21.15.26:0/3555470436 learned_addr learned my addr 172.21.15.26:0/3555470436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80b7fff640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b80c57e0 0x7f80b806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bd36d640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b81529d0 0x7f80b81eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 -- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b81529d0 msgr2=0x7f80b81eab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b81529d0 0x7f80b81eab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 -- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b80c57e0 msgr2=0x7f80b806e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b80c57e0 0x7f80b806e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.524+0000 7f80bcb6c640 1 -- 172.21.15.26:0/3555470436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80ae4a0 con 0x7f80b8072cd0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.525+0000 7f80bcb6c640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b8072cd0 0x7f80b81eebc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00e870 tx=0x7f80ac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.525+0000 7f80b57fa640 1 -- 172.21.15.26:0/3555470436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac017070 con 0x7f80b8072cd0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.525+0000 7f80b57fa640 1 -- 172.21.15.26:0/3555470436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac00c940 con 0x7f80b8072cd0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.525+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b80ae780 con 0x7f80b8072cd0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.525+0000 7f80b57fa640 1 -- 172.21.15.26:0/3555470436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00cd30 con 0x7f80b8072cd0 2024-03-06T07:22:38.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.525+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b81ee3c0 con 0x7f80b8072cd0 2024-03-06T07:22:38.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.526+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b804f330 con 0x7f80b8072cd0 2024-03-06T07:22:38.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.526+0000 7f80b57fa640 1 -- 172.21.15.26:0/3555470436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80ac01a070 con 0x7f80b8072cd0 2024-03-06T07:22:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.527+0000 7f80b57fa640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f809c06b4d0 0x7f809c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.527+0000 7f80b57fa640 1 -- 172.21.15.26:0/3555470436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80ac08a5a0 con 0x7f80b8072cd0 2024-03-06T07:22:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.527+0000 7f80b7fff640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f809c06b4d0 0x7f809c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.527+0000 7f80b7fff640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f809c06b4d0 0x7f809c06d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f80a4036000 tx=0x7f80a403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.531+0000 7f80b57fa640 1 -- 172.21.15.26:0/3555470436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f80ac058ba0 con 0x7f80b8072cd0 2024-03-06T07:22:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.781+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f80b80c55c0 con 0x7f80b8072cd0 2024-03-06T07:22:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.782+0000 7f80b57fa640 1 -- 172.21.15.26:0/3555470436 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f80ac05cc40 con 0x7f80b8072cd0 2024-03-06T07:22:38.785 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:22:38.785 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:22:38.785 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:22:38.785 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:22:38.785 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:22:38.786 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:22:38.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.784+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f809c06b4d0 msgr2=0x7f809c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.784+0000 7f80bedd0640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f809c06b4d0 0x7f809c06d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f80a4036000 tx=0x7f80a403d040 comp rx=0 tx=0).stop 2024-03-06T07:22:38.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.784+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b8072cd0 msgr2=0x7f80b81eebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.784+0000 7f80bedd0640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b8072cd0 0x7f80b81eebc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00e870 tx=0x7f80ac00ed30 comp rx=0 tx=0).stop 2024-03-06T07:22:38.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.784+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 shutdown_connections 2024-03-06T07:22:38.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.784+0000 7f80bedd0640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f809c06b4d0 0x7f809c06d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.785+0000 7f80bedd0640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b81529d0 0x7f80b81eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.785+0000 7f80bedd0640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b80c57e0 0x7f80b806e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.785+0000 7f80bedd0640 1 --2- 172.21.15.26:0/3555470436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b8072cd0 0x7f80b81eebc0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.785+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 >> 172.21.15.26:0/3555470436 conn(0x7f80b80aa630 msgr2=0x7f80b80a9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.785+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 shutdown_connections 2024-03-06T07:22:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.785+0000 7f80bedd0640 1 -- 172.21.15.26:0/3555470436 wait complete. 2024-03-06T07:22:38.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.897+0000 7f2b7201c640 1 Processor -- start 2024-03-06T07:22:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.897+0000 7f2b7201c640 1 -- start start 2024-03-06T07:22:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b7201c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b6c14f310 0x7f2b6c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b7201c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6c155bb0 con 0x7f2b6c0aa180 2024-03-06T07:22:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b7201c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6c0aa660 con 0x7f2b6c1550c0 2024-03-06T07:22:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b7201c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6c0c4610 con 0x7f2b6c14f310 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2b6c0aa180 0x7f2b6c14ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47156/0 (socket says 172.21.15.26:47156) 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6affd640 1 -- 172.21.15.26:0/1327282220 learned_addr learned my addr 172.21.15.26:0/1327282220 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6bfff640 1 --2- 172.21.15.26:0/1327282220 >> v2:172.21.15.117:3300/0 conn(0x7f2b6c14f310 0x7f2b6c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6a7fc640 1 -- 172.21.15.26:0/1327282220 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484793473 0 0) 0x7f2b6c155bb0 con 0x7f2b6c0aa180 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6a7fc640 1 -- 172.21.15.26:0/1327282220 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b54003440 con 0x7f2b6c0aa180 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6bfff640 1 -- 172.21.15.26:0/1327282220 >> v1:172.21.15.117:6789/0 conn(0x7f2b6c1550c0 legacy=0x7f2b6c1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6bfff640 1 -- 172.21.15.26:0/1327282220 >> v1:172.21.15.26:6789/0 conn(0x7f2b6c0aa180 legacy=0x7f2b6c14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6a7fc640 1 -- 172.21.15.26:0/1327282220 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632939692 0 0) 0x7f2b6c0aa660 con 0x7f2b6c1550c0 2024-03-06T07:22:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.898+0000 7f2b6bfff640 1 -- 172.21.15.26:0/1327282220 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c0aa940 con 0x7f2b6c14f310 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.899+0000 7f2b6bfff640 1 --2- 172.21.15.26:0/1327282220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c14f310 0x7f2b6c0c5060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009d90 tx=0x7f2b6402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54ad2e871915bfb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.899+0000 7f2b6a7fc640 1 -- 172.21.15.26:0/1327282220 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64039070 con 0x7f2b6c14f310 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.899+0000 7f2b6a7fc640 1 -- 172.21.15.26:0/1327282220 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6402cbd0 con 0x7f2b6c14f310 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.899+0000 7f2b6a7fc640 1 -- 172.21.15.26:0/1327282220 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64034440 con 0x7f2b6c14f310 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.900+0000 7f2b7201c640 1 -- 172.21.15.26:0/1327282220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c14f310 msgr2=0x7f2b6c0c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.900+0000 7f2b7201c640 1 --2- 172.21.15.26:0/1327282220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c14f310 0x7f2b6c0c5060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009d90 tx=0x7f2b6402be20 comp rx=0 tx=0).stop 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.900+0000 7f2b7201c640 1 -- 172.21.15.26:0/1327282220 shutdown_connections 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.900+0000 7f2b7201c640 1 --2- 172.21.15.26:0/1327282220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c14f310 0x7f2b6c0c5060 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.900+0000 7f2b7201c640 1 -- 172.21.15.26:0/1327282220 >> 172.21.15.26:0/1327282220 conn(0x7f2b6c0c6710 msgr2=0x7f2b6c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.900+0000 7f2b7201c640 1 -- 172.21.15.26:0/1327282220 shutdown_connections 2024-03-06T07:22:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.900+0000 7f2b7201c640 1 -- 172.21.15.26:0/1327282220 wait complete. 2024-03-06T07:22:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.901+0000 7f2b7201c640 1 Processor -- start 2024-03-06T07:22:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.901+0000 7f2b7201c640 1 -- start start 2024-03-06T07:22:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b7201c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c0aa180 0x7f2b6c071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b7201c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b6c14f310 0x7f2b6c06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b7201c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c1550c0 0x7f2b6c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b7201c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c0c1490 con 0x7f2b6c0aa180 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b7201c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c0c1310 con 0x7f2b6c1550c0 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b7201c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c0c3210 con 0x7f2b6c14f310 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c0aa180 0x7f2b6c071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c0aa180 0x7f2b6c071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34238/0 (socket says 172.21.15.26:34238) 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 -- 172.21.15.26:0/3717795969 learned_addr learned my addr 172.21.15.26:0/3717795969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6bfff640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c1550c0 0x7f2b6c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6affd640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b6c14f310 0x7f2b6c06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 -- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b6c14f310 msgr2=0x7f2b6c06e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b6c14f310 0x7f2b6c06e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 -- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c1550c0 msgr2=0x7f2b6c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c1550c0 0x7f2b6c070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6b7fe640 1 -- 172.21.15.26:0/3717795969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c0a8020 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.902+0000 7f2b6bfff640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c1550c0 0x7f2b6c070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.903+0000 7f2b6b7fe640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c0aa180 0x7f2b6c071ed0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c410 tx=0x7f2b5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.903+0000 7f2b53fff640 1 -- 172.21.15.26:0/3717795969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c017070 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.903+0000 7f2b53fff640 1 -- 172.21.15.26:0/3717795969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c0035f0 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.903+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6c0a7780 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.903+0000 7f2b53fff640 1 -- 172.21.15.26:0/3717795969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0039e0 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.903+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c0a7d40 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.904+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c04f330 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.904+0000 7f2b53fff640 1 -- 172.21.15.26:0/3717795969 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b5c01a070 con 0x7f2b6c0aa180 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.905+0000 7f2b53fff640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b5806b500 0x7f2b5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.905+0000 7f2b53fff640 1 -- 172.21.15.26:0/3717795969 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b5c089790 con 0x7f2b6c0aa180 2024-03-06T07:22:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.905+0000 7f2b6affd640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b5806b500 0x7f2b5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.905+0000 7f2b6affd640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b5806b500 0x7f2b5806d9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0a5dc0 tx=0x7f2b60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:38.909+0000 7f2b53fff640 1 -- 172.21.15.26:0/3717795969 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2b5c057da0 con 0x7f2b6c0aa180 2024-03-06T07:22:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.128+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b6c0a5ef0 con 0x7f2b6c0aa180 2024-03-06T07:22:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.129+0000 7f2b53fff640 1 -- 172.21.15.26:0/3717795969 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4979 (secure 0 0 0) 0x7f2b5c05be40 con 0x7f2b6c0aa180 2024-03-06T07:22:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:36.381549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6339":{"gid":6339,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/218606650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":218606650},{"type":"v1","addr":"172.21.15.117:6836","nonce":218606650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6339,"qdb_cluster":[6339]},"id":5}]} 2024-03-06T07:22:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 55 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b5806b500 msgr2=0x7f2b5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b5806b500 0x7f2b5806d9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0a5dc0 tx=0x7f2b60009040 comp rx=0 tx=0).stop 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c0aa180 msgr2=0x7f2b6c071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c0aa180 0x7f2b6c071ed0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c410 tx=0x7f2b5c00c8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 shutdown_connections 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b5806b500 0x7f2b5806d9c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b6c1550c0 0x7f2b6c070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b6c14f310 0x7f2b6c06e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 --2- 172.21.15.26:0/3717795969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c0aa180 0x7f2b6c071ed0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 >> 172.21.15.26:0/3717795969 conn(0x7f2b6c0c6710 msgr2=0x7f2b6c14bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 shutdown_connections 2024-03-06T07:22:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.131+0000 7f2b7201c640 1 -- 172.21.15.26:0/3717795969 wait complete. 2024-03-06T07:22:39.547 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:22:34.395648+0000', 'modified': '2024-03-06T07:22:36.381549+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6339}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6339': {'gid': 6339, 'name': 'b', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/218606650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 218606650}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 218606650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6339, 'qdb_cluster': [6339]} 2024-03-06T07:22:39.547 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:22:39.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:22:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb7eafb640 1 Processor -- start 2024-03-06T07:22:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb7eafb640 1 -- start start 2024-03-06T07:22:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb7eafb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffb78072ef0 0x7ffb78156d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb7eafb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb780732d0 0x7ffb781572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb7eafb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb78070160 con 0x7ffb780c7270 2024-03-06T07:22:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb7eafb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb780c7030 con 0x7ffb780732d0 2024-03-06T07:22:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb7eafb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb780c6a70 con 0x7ffb78072ef0 2024-03-06T07:22:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.651+0000 7ffb77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb780732d0 0x7ffb781572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb7d098640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffb780c7270 0x7ffb780c36b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47160/0 (socket says 172.21.15.26:47160) 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb780732d0 0x7ffb781572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34252/0 (socket says 172.21.15.26:34252) 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb7d098640 1 -- 172.21.15.26:0/2654311108 learned_addr learned my addr 172.21.15.26:0/2654311108 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb7c897640 1 --2- 172.21.15.26:0/2654311108 >> v2:172.21.15.117:3300/0 conn(0x7ffb78072ef0 0x7ffb78156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb77fff640 1 -- 172.21.15.26:0/2654311108 >> v2:172.21.15.117:3300/0 conn(0x7ffb78072ef0 msgr2=0x7ffb78156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb77fff640 1 --2- 172.21.15.26:0/2654311108 >> v2:172.21.15.117:3300/0 conn(0x7ffb78072ef0 0x7ffb78156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb77fff640 1 -- 172.21.15.26:0/2654311108 >> v1:172.21.15.26:6789/0 conn(0x7ffb780c7270 legacy=0x7ffb780c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb77fff640 1 -- 172.21.15.26:0/2654311108 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780c6d50 con 0x7ffb780732d0 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.652+0000 7ffb777fe640 1 -- 172.21.15.26:0/2654311108 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073893331 0 0) 0x7ffb70001490 con 0x7ffb780c7270 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb77fff640 1 --2- 172.21.15.26:0/2654311108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780732d0 0x7ffb781572b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffb7806bd00 tx=0x7ffb6402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6d07f16298427d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb777fe640 1 -- 172.21.15.26:0/2654311108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640315a0 con 0x7ffb780732d0 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb777fe640 1 -- 172.21.15.26:0/2654311108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb64037070 con 0x7ffb780732d0 2024-03-06T07:22:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb777fe640 1 -- 172.21.15.26:0/2654311108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64031960 con 0x7ffb780732d0 2024-03-06T07:22:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 -- 172.21.15.26:0/2654311108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780732d0 msgr2=0x7ffb781572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/2654311108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780732d0 0x7ffb781572b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffb7806bd00 tx=0x7ffb6402b840 comp rx=0 tx=0).stop 2024-03-06T07:22:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 -- 172.21.15.26:0/2654311108 shutdown_connections 2024-03-06T07:22:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/2654311108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780732d0 0x7ffb781572b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/2654311108 >> v2:172.21.15.117:3300/0 conn(0x7ffb78072ef0 0x7ffb78156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 -- 172.21.15.26:0/2654311108 >> 172.21.15.26:0/2654311108 conn(0x7ffb7806ea40 msgr2=0x7ffb7806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 -- 172.21.15.26:0/2654311108 shutdown_connections 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.653+0000 7ffb7eafb640 1 -- 172.21.15.26:0/2654311108 wait complete. 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.654+0000 7ffb7eafb640 1 Processor -- start 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.654+0000 7ffb7eafb640 1 -- start start 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7eafb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb78072ef0 0x7ffb780ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7eafb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780c7270 0x7ffb780a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7eafb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780ad830 0x7ffb780a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7eafb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7806fc40 con 0x7ffb780ad830 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7eafb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7806f960 con 0x7ffb780c7270 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7eafb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb78070360 con 0x7ffb78072ef0 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7d098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780ad830 0x7ffb780a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7d098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780ad830 0x7ffb780a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34256/0 (socket says 172.21.15.26:34256) 2024-03-06T07:22:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7d098640 1 -- 172.21.15.26:0/3649390060 learned_addr learned my addr 172.21.15.26:0/3649390060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb78072ef0 0x7ffb780ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.655+0000 7ffb77fff640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780c7270 0x7ffb780a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7d098640 1 -- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb78072ef0 msgr2=0x7ffb780ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7d098640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb78072ef0 0x7ffb780ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7d098640 1 -- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780c7270 msgr2=0x7ffb780a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7d098640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780c7270 0x7ffb780a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7d098640 1 -- 172.21.15.26:0/3649390060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780beb90 con 0x7ffb780ad830 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7d098640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780ad830 0x7ffb780a7220 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000c780 tx=0x7ffb7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb757fa640 1 -- 172.21.15.26:0/3649390060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70018070 con 0x7ffb780ad830 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb757fa640 1 -- 172.21.15.26:0/3649390060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7000ce30 con 0x7ffb780ad830 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb780a7b20 con 0x7ffb780ad830 2024-03-06T07:22:39.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb757fa640 1 -- 172.21.15.26:0/3649390060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000a840 con 0x7ffb780ad830 2024-03-06T07:22:39.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.656+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb781e95c0 con 0x7ffb780ad830 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.657+0000 7ffb757fa640 1 -- 172.21.15.26:0/3649390060 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb7001b070 con 0x7ffb780ad830 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.658+0000 7ffb757fa640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b4d0 0x7ffb5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.658+0000 7ffb757fa640 1 -- 172.21.15.26:0/3649390060 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb700897a0 con 0x7ffb780ad830 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.658+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b4d0 0x7ffb5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.658+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b4d0 0x7ffb5c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009b40 tx=0x7ffb6c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 Processor -- start 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 -- start start 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e6f50 0x55cd929e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e7330 0x55cd929e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e54b0 0x55cd929e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9294b6b0 con 0x55cd929e7330 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd928f4a60 con 0x55cd929e6f50 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.659+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd92947750 con 0x55cd929e54b0 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e7330 0x55cd929e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb7c897640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e6f50 0x55cd929e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e7330 0x55cd929e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34266/0 (socket says 172.21.15.26:34266) 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 -- 172.21.15.26:0/3594604872 learned_addr learned my addr 172.21.15.26:0/3594604872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e6f50 0x55cd929e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:54710/0 (socket says 172.21.15.26:54710) 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb7d098640 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e54b0 0x55cd929e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 -- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e54b0 msgr2=0x55cd929e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e54b0 0x55cd929e1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 -- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e6f50 msgr2=0x55cd929e4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e6f50 0x55cd929e4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 -- 172.21.15.26:0/3594604872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd92912f30 con 0x55cd929e7330 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb7d098640 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e54b0 0x55cd929e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e6f50 0x55cd929e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb77fff640 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e7330 0x55cd929e4f70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ffb64034040 tx=0x7ffb64008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add87ad1076f32ed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb627fc640 1 -- 172.21.15.26:0/3594604872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6404a070 con 0x55cd929e7330 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb627fc640 1 -- 172.21.15.26:0/3594604872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb64037040 con 0x55cd929e7330 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.660+0000 7ffb627fc640 1 -- 172.21.15.26:0/3594604872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64031a80 con 0x55cd929e7330 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 -- 172.21.15.26:0/3594604872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e7330 msgr2=0x55cd929e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e7330 0x55cd929e4f70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ffb64034040 tx=0x7ffb64008030 comp rx=0 tx=0).stop 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 -- 172.21.15.26:0/3594604872 shutdown_connections 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e54b0 0x55cd929e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e7330 0x55cd929e4f70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3594604872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e6f50 0x55cd929e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 -- 172.21.15.26:0/3594604872 >> 172.21.15.26:0/3594604872 conn(0x55cd929e37d0 msgr2=0x55cd92a083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 -- 172.21.15.26:0/3594604872 shutdown_connections 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 -- 172.21.15.26:0/3594604872 wait complete. 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.661+0000 7ffb817c4740 1 -- start start 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 1 Processor -- start 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd92a26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd92a374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd92a34010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd92a09230 con 0x55cd92a37a10 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd92907ea0 con 0x55cd929e7330 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd92a08c70 con 0x55cd929e6f50 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb817c4740 10 client.0 _ll_register_callbacks cb 0x55cd92907870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:22:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb7d098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd92a34010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb7d098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd92a34010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34272/0 (socket says 172.21.15.26:34272) 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb7d098640 1 -- 172.21.15.26:0/2061982722 learned_addr learned my addr 172.21.15.26:0/2061982722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb77fff640 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd92a374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.662+0000 7ffb7c897640 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd92a26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb7d098640 1 -- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 msgr2=0x55cd92a26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb7d098640 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd92a26a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb7d098640 1 -- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 msgr2=0x55cd92a374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb7d098640 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd92a374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb7d098640 1 -- 172.21.15.26:0/2061982722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd92908270 con 0x55cd92a37a10 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb7d098640 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd92a34010 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffb70087bd0 tx=0x7ffb7005c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb7f45225343c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb577fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb577fe640 1 -- 172.21.15.26:0/2061982722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70057520 con 0x55cd92a37a10 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb577fe640 1 -- 172.21.15.26:0/2061982722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb70082d70 con 0x55cd92a37a10 2024-03-06T07:22:39.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb64037040 con 0x55cd92a37a10 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb577fe640 1 -- 172.21.15.26:0/2061982722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb7006fbb0 con 0x55cd92a37a10 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 10 client.6471 fetch_fsmap learned FSMap version 55 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55cd926602d0 con 0x55cd92a37a10 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb577fe640 1 -- 172.21.15.26:0/2061982722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb64037040 con 0x55cd92a37a10 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb577fe640 1 -- 172.21.15.26:0/2061982722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) v1 ==== 3314+0+0 (secure 0 0 0) 0x7ffb70053c20 con 0x55cd92a37a10 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 10 client.6471 fetch_fsmap finished waiting for FSMap version 55 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 10 client.6471 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 20 client.6471 populate_metadata read hostname 'smithi026' 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd92a06c00 0x55cd92a42540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 4 client.6471 mds_command: new command op to 6339 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.663+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 --> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd92958bf0 con 0x55cd92a06c00 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.664+0000 7ffb7c897640 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd92a06c00 0x55cd92a42540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.664+0000 7ffb7c897640 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd92a06c00 0x55cd92a42540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3243369dec615a87 server_cookie=5813f2542ac6aadf in_seq=0 out_seq=0 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.664+0000 7ffb577fe640 10 client.6471 ms_handle_connect on v2:172.21.15.117:6834/218606650 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb577fe640 1 -- 172.21.15.26:0/2061982722 <== mds.0 v2:172.21.15.117:6834/218606650 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cd92958bf0 con 0x55cd92a06c00 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb577fe640 10 client.6471 handle_command_reply: tid=0 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 client.6471 shutdown 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 msgr2=0x55cd92a34010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd92a34010 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffb70087bd0 tx=0x7ffb7005c080 comp rx=0 tx=0).stop 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 shutdown_connections 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd92a06c00 0x55cd92a42540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd92a34010 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd92a374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 --2- 172.21.15.26:0/2061982722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd92a26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb577fe640 0 client.6471 ms_handle_reset on v2:172.21.15.117:6834/218606650 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 >> 172.21.15.26:0/2061982722 conn(0x55cd929e54b0 msgr2=0x55cd929e58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 shutdown_connections 2024-03-06T07:22:39.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 1 -- 172.21.15.26:0/2061982722 wait complete. 2024-03-06T07:22:39.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.665+0000 7ffb817c4740 20 client.6471 trim_cache size 0 max 16384 2024-03-06T07:22:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.682+0000 7ffb817c4740 1 Processor -- start 2024-03-06T07:22:39.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.682+0000 7ffb817c4740 1 -- start start 2024-03-06T07:22:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd929e0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd929e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd929e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb817c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781ec6a0 con 0x55cd92a37a10 2024-03-06T07:22:39.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb78150510 con 0x55cd929e7330 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781ec0a0 con 0x55cd929e6f50 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd929e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd929e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34278/0 (socket says 172.21.15.26:34278) 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 -- 172.21.15.26:0/1087036863 learned_addr learned my addr 172.21.15.26:0/1087036863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb77fff640 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd929e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7c897640 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd929e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 -- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 msgr2=0x55cd929e0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd929e0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 -- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 msgr2=0x55cd929e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd929e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 -- 172.21.15.26:0/1087036863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9272a6b0 con 0x55cd92a37a10 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7c897640 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd929e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb7d098640 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd929e8070 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ffb70054850 tx=0x7ffb7006c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f59abd053132ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb61ffb640 1 -- 172.21.15.26:0/1087036863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70057320 con 0x55cd92a37a10 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb61ffb640 1 -- 172.21.15.26:0/1087036863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb700574c0 con 0x55cd92a37a10 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.683+0000 7ffb61ffb640 1 -- 172.21.15.26:0/1087036863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70056fb0 con 0x55cd92a37a10 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 -- 172.21.15.26:0/1087036863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 msgr2=0x55cd929e8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd929e8070 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ffb70054850 tx=0x7ffb7006c000 comp rx=0 tx=0).stop 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 -- 172.21.15.26:0/1087036863 shutdown_connections 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd92a37a10 0x55cd929e8070 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd929e7330 0x55cd929e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 --2- 172.21.15.26:0/1087036863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e6f50 0x55cd929e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 -- 172.21.15.26:0/1087036863 >> 172.21.15.26:0/1087036863 conn(0x55cd929e54b0 msgr2=0x55cd929dcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 -- 172.21.15.26:0/1087036863 shutdown_connections 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 -- 172.21.15.26:0/1087036863 wait complete. 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.684+0000 7ffb817c4740 1 -- start start 2024-03-06T07:22:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 1 Processor -- start 2024-03-06T07:22:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e6f50 0x55cd92a93a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e7330 0x55cd92a46e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd92a37a10 0x55cd92a90c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd92a338b0 con 0x55cd929e6f50 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd92a09830 con 0x55cd92a37a10 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd92a0c340 con 0x55cd929e7330 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb817c4740 10 client.0 _ll_register_callbacks cb 0x7ffb70057fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb7c897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e6f50 0x55cd92a93a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb7c897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e6f50 0x55cd92a93a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34292/0 (socket says 172.21.15.26:34292) 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb7c897640 1 -- 172.21.15.26:0/3422621964 learned_addr learned my addr 172.21.15.26:0/3422621964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb77fff640 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e7330 0x55cd92a46e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.685+0000 7ffb7d098640 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd92a37a10 0x55cd92a90c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb7c897640 1 -- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e7330 msgr2=0x55cd92a46e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e7330 0x55cd92a46e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb7c897640 1 -- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd92a37a10 msgr2=0x55cd92a90c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd92a37a10 0x55cd92a90c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb7c897640 1 -- 172.21.15.26:0/3422621964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd92a0a3c0 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb7c897640 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e6f50 0x55cd92a93a70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c011690 tx=0x7ffb6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c177351def5dff server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb577fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb577fe640 1 -- 172.21.15.26:0/3422621964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c004480 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb577fe640 1 -- 172.21.15.26:0/3422621964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb6c003da0 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb700574c0 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb577fe640 1 -- 172.21.15.26:0/3422621964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb6c03c050 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 10 client.6477 fetch_fsmap learned FSMap version 55 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55cd929e5f10 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb577fe640 1 -- 172.21.15.26:0/3422621964 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb700574c0 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb577fe640 1 -- 172.21.15.26:0/3422621964 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) v1 ==== 3314+0+0 (secure 0 0 0) 0x7ffb6c03f020 con 0x55cd929e6f50 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 10 client.6477 fetch_fsmap finished waiting for FSMap version 55 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 10 client.6477 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:22:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 20 client.6477 populate_metadata read hostname 'smithi026' 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd929e7820 0x55cd929e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 4 client.6477 mds_command: new command op to 6339 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.686+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 --> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cd92952f40 con 0x55cd929e7820 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb77fff640 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd929e7820 0x55cd929e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb77fff640 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd929e7820 0x55cd929e7c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60f4424b04c0dc2 server_cookie=2815b669329f21c4 in_seq=0 out_seq=0 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb577fe640 10 client.6477 ms_handle_connect on v2:172.21.15.117:6834/218606650 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb577fe640 1 -- 172.21.15.26:0/3422621964 <== mds.0 v2:172.21.15.117:6834/218606650 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55cd92952f40 con 0x55cd929e7820 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb577fe640 10 client.6477 handle_command_reply: tid=0 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 client.6477 shutdown 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e6f50 msgr2=0x55cd92a93a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e6f50 0x55cd92a93a70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c011690 tx=0x7ffb6c004730 comp rx=0 tx=0).stop 2024-03-06T07:22:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 shutdown_connections 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:6834/218606650,v1:172.21.15.117:6836/218606650] conn(0x55cd929e7820 0x55cd929e7c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd92a37a10 0x55cd92a90c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd929e7330 0x55cd92a46e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb577fe640 0 client.6477 ms_handle_reset on v2:172.21.15.117:6834/218606650 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 --2- 172.21.15.26:0/3422621964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd929e6f50 0x55cd92a93a70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 >> 172.21.15.26:0/3422621964 conn(0x55cd929e54b0 msgr2=0x55cd929e2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 shutdown_connections 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 1 -- 172.21.15.26:0/3422621964 wait complete. 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.687+0000 7ffb817c4740 20 client.6477 trim_cache size 0 max 16384 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:22:39.697 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6339, 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.6631735450000003, 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 55, 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 74, 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 74, 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 9.7298828210000003 2024-03-06T07:22:39.698 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:22:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b4d0 msgr2=0x7ffb5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b4d0 0x7ffb5c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009b40 tx=0x7ffb6c009190 comp rx=0 tx=0).stop 2024-03-06T07:22:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780ad830 msgr2=0x7ffb780a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780ad830 0x7ffb780a7220 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000c780 tx=0x7ffb7000cc40 comp rx=0 tx=0).stop 2024-03-06T07:22:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 shutdown_connections 2024-03-06T07:22:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b4d0 0x7ffb5c06d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb780ad830 0x7ffb780a7220 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780c7270 0x7ffb780a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 --2- 172.21.15.26:0/3649390060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb78072ef0 0x7ffb780ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 >> 172.21.15.26:0/3649390060 conn(0x7ffb7806ea40 msgr2=0x7ffb780c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 shutdown_connections 2024-03-06T07:22:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.688+0000 7ffb7eafb640 1 -- 172.21.15.26:0/3649390060 wait complete. 2024-03-06T07:22:39.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.800+0000 7f82d8b93640 1 Processor -- start 2024-03-06T07:22:39.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.800+0000 7f82d8b93640 1 -- start start 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d8b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d406afc0 0x7f82d406b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d8b93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f82d40c20b0 0x7f82d4153180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d8b93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d4153dd0 con 0x7f82d40c25c0 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d8b93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d40a9530 con 0x7f82d40c20b0 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d8b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40a9810 con 0x7f82d406afc0 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f82d40c25c0 0x7f82d41536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47168/0 (socket says 172.21.15.26:47168) 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d2d9d640 1 -- 172.21.15.26:0/2560143429 learned_addr learned my addr 172.21.15.26:0/2560143429 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d259c640 1 --2- 172.21.15.26:0/2560143429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d406afc0 0x7f82d406b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d1d9b640 1 --2- 172.21.15.26:0/2560143429 >> v2:172.21.15.117:3300/0 conn(0x7f82d40c20b0 0x7f82d4153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d159a640 1 -- 172.21.15.26:0/2560143429 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412060624 0 0) 0x7f82d4153dd0 con 0x7f82d40c25c0 2024-03-06T07:22:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d259c640 1 -- 172.21.15.26:0/2560143429 >> v2:172.21.15.117:3300/0 conn(0x7f82d40c20b0 msgr2=0x7f82d4153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d259c640 1 --2- 172.21.15.26:0/2560143429 >> v2:172.21.15.117:3300/0 conn(0x7f82d40c20b0 0x7f82d4153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d259c640 1 -- 172.21.15.26:0/2560143429 >> v1:172.21.15.26:6789/0 conn(0x7f82d40c25c0 legacy=0x7f82d41536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.801+0000 7f82d259c640 1 -- 172.21.15.26:0/2560143429 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d40a9af0 con 0x7f82d406afc0 2024-03-06T07:22:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.802+0000 7f82d259c640 1 --2- 172.21.15.26:0/2560143429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d406afc0 0x7f82d406b3a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f82c40097a0 tx=0x7f82c402b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90849c5fa2ba4e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.802+0000 7f82d159a640 1 -- 172.21.15.26:0/2560143429 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4038070 con 0x7f82d406afc0 2024-03-06T07:22:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.802+0000 7f82d159a640 1 -- 172.21.15.26:0/2560143429 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4031700 con 0x7f82d406afc0 2024-03-06T07:22:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.802+0000 7f82d159a640 1 -- 172.21.15.26:0/2560143429 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4031ad0 con 0x7f82d406afc0 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 -- 172.21.15.26:0/2560143429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d406afc0 msgr2=0x7f82d406b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 --2- 172.21.15.26:0/2560143429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d406afc0 0x7f82d406b3a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f82c40097a0 tx=0x7f82c402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 -- 172.21.15.26:0/2560143429 shutdown_connections 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 --2- 172.21.15.26:0/2560143429 >> v2:172.21.15.117:3300/0 conn(0x7f82d40c20b0 0x7f82d4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 --2- 172.21.15.26:0/2560143429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d406afc0 0x7f82d406b3a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 -- 172.21.15.26:0/2560143429 >> 172.21.15.26:0/2560143429 conn(0x7f82d40c76b0 msgr2=0x7f82d40c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 -- 172.21.15.26:0/2560143429 shutdown_connections 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.803+0000 7f82d8b93640 1 -- 172.21.15.26:0/2560143429 wait complete. 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.804+0000 7f82d8b93640 1 Processor -- start 2024-03-06T07:22:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.804+0000 7f82d8b93640 1 -- start start 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d406afc0 0x7f82d41ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d40c20b0 0x7f82d40c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d40c25c0 0x7f82d40c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d4150980 con 0x7f82d406afc0 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40aa0b0 con 0x7f82d40c25c0 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d4150b00 con 0x7f82d40c20b0 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d406afc0 0x7f82d41ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d406afc0 0x7f82d41ebca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34306/0 (socket says 172.21.15.26:34306) 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 -- 172.21.15.26:0/3028473 learned_addr learned my addr 172.21.15.26:0/3028473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d1d9b640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d40c20b0 0x7f82d40c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d2d9d640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d40c25c0 0x7f82d40c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 -- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d40c20b0 msgr2=0x7f82d40c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d40c20b0 0x7f82d40c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 -- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d40c25c0 msgr2=0x7f82d40c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d40c25c0 0x7f82d40c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 -- 172.21.15.26:0/3028473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d414f500 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d259c640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d406afc0 0x7f82d41ebca0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f82c4036040 tx=0x7f82c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82c2ffd640 1 -- 172.21.15.26:0/3028473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4042070 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82c2ffd640 1 -- 172.21.15.26:0/3028473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c403d890 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d414cf90 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82c2ffd640 1 -- 172.21.15.26:0/3028473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4038040 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.805+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d41eb4a0 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.806+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d404f330 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.807+0000 7f82c2ffd640 1 -- 172.21.15.26:0/3028473 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82c403b070 con 0x7f82d406afc0 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.807+0000 7f82c2ffd640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.807+0000 7f82c2ffd640 1 -- 172.21.15.26:0/3028473 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f82c40ab950 con 0x7f82d406afc0 2024-03-06T07:22:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.807+0000 7f82d1d9b640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.808+0000 7f82d1d9b640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f82b406b4d0 0x7f82b406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f82d4152580 tx=0x7f82c8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:39.812+0000 7f82c2ffd640 1 -- 172.21.15.26:0/3028473 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f82c4030070 con 0x7f82d406afc0 2024-03-06T07:22:40.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.029+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82d40c1a50 con 0x7f82d406afc0 2024-03-06T07:22:40.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.030+0000 7f82c2ffd640 1 -- 172.21.15.26:0/3028473 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4979 (secure 0 0 0) 0x7f82c407a1a0 con 0x7f82d406afc0 2024-03-06T07:22:40.033 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:40.033 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:36.381549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6339":{"gid":6339,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/218606650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":218606650},{"type":"v1","addr":"172.21.15.117:6836","nonce":218606650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6339,"qdb_cluster":[6339]},"id":5}]} 2024-03-06T07:22:40.033 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 55 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f82b406b4d0 msgr2=0x7f82b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f82b406b4d0 0x7f82b406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f82d4152580 tx=0x7f82c8009190 comp rx=0 tx=0).stop 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d406afc0 msgr2=0x7f82d41ebca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d406afc0 0x7f82d41ebca0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f82c4036040 tx=0x7f82c40046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 shutdown_connections 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d40c25c0 0x7f82d40c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82d40c20b0 0x7f82d40c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 --2- 172.21.15.26:0/3028473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d406afc0 0x7f82d41ebca0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 >> 172.21.15.26:0/3028473 conn(0x7f82d40c76b0 msgr2=0x7f82d406e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:40.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 shutdown_connections 2024-03-06T07:22:40.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.032+0000 7f82d8b93640 1 -- 172.21.15.26:0/3028473 wait complete. 2024-03-06T07:22:40.046 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 5, "epoch": 55, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 5, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:22:34.395648+0000", "damaged": [], "data_pools": [ 12 ], "enabled": true, "epoch": 55, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6339": { "addr": "172.21.15.117:6836/218606650", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 218606650, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 218606650, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6339, "incarnation": 53, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 11, "modified": "2024-03-06T07:22:36.381549+0000", "qdb_cluster": [ 6339 ], "qdb_leader": 6339, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6339 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/378552977", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 378552977, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 378552977, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 52, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5618, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6833/3180689366", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3180689366, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3180689366, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 52, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6324, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/3035454765", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 3035454765, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 3035454765, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 52, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6327, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:22:40.047 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:22:40.047 INFO:tasks.cephfs.test_failover:current = [{'gid': 6324, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/3180689366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3180689366}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3180689366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 52}, {'gid': 6339, 'name': 'b', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/218606650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 218606650}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 218606650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6327, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/3035454765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3035454765}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3035454765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 52}, {'gid': 5618, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/378552977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 378552977}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 378552977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 52}] 2024-03-06T07:22:40.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs2 2024-03-06T07:22:40.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.149+0000 7fcb94481640 1 Processor -- start 2024-03-06T07:22:40.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.149+0000 7fcb94481640 1 -- start start 2024-03-06T07:22:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb94481640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb94481640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c072630 con 0x7fcb8c0c6780 2024-03-06T07:22:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb94481640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c072910 con 0x7fcb8c06da40 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb94481640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c06de20 con 0x7fcb8c0c7380 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb91a1c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcb8c0c6780 0x7fcb8c158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47180/0 (socket says 172.21.15.26:47180) 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb91a1c640 1 -- 172.21.15.26:0/2969724005 learned_addr learned my addr 172.21.15.26:0/2969724005 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb92a1e640 1 --2- 172.21.15.26:0/2969724005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb9121b640 1 -- 172.21.15.26:0/2969724005 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210441652 0 0) 0x7fcb8c072630 con 0x7fcb8c0c6780 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb9121b640 1 -- 172.21.15.26:0/2969724005 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb74003520 con 0x7fcb8c0c6780 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb9121b640 1 -- 172.21.15.26:0/2969724005 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372213986 0 0) 0x7fcb8c072910 con 0x7fcb8c06da40 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb9121b640 1 -- 172.21.15.26:0/2969724005 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb8c072630 con 0x7fcb8c06da40 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb92a1e640 1 -- 172.21.15.26:0/2969724005 >> v1:172.21.15.117:6789/0 conn(0x7fcb8c06da40 legacy=0x7fcb8c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb92a1e640 1 -- 172.21.15.26:0/2969724005 >> v1:172.21.15.26:6789/0 conn(0x7fcb8c0c6780 legacy=0x7fcb8c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.150+0000 7fcb92a1e640 1 -- 172.21.15.26:0/2969724005 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c072bf0 con 0x7fcb8c0c7380 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.151+0000 7fcb92a1e640 1 --2- 172.21.15.26:0/2969724005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c0c3280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009dc0 tx=0x7fcb7c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7812fef7edf85f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.151+0000 7fcb9121b640 1 -- 172.21.15.26:0/2969724005 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c039070 con 0x7fcb8c0c7380 2024-03-06T07:22:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.151+0000 7fcb9121b640 1 -- 172.21.15.26:0/2969724005 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c02cc40 con 0x7fcb8c0c7380 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.151+0000 7fcb9121b640 1 -- 172.21.15.26:0/2969724005 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c034440 con 0x7fcb8c0c7380 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.152+0000 7fcb94481640 1 -- 172.21.15.26:0/2969724005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 msgr2=0x7fcb8c0c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.152+0000 7fcb94481640 1 --2- 172.21.15.26:0/2969724005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c0c3280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009dc0 tx=0x7fcb7c02be90 comp rx=0 tx=0).stop 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.152+0000 7fcb94481640 1 -- 172.21.15.26:0/2969724005 shutdown_connections 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.152+0000 7fcb94481640 1 --2- 172.21.15.26:0/2969724005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c0c3280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.152+0000 7fcb94481640 1 -- 172.21.15.26:0/2969724005 >> 172.21.15.26:0/2969724005 conn(0x7fcb8c0aa630 msgr2=0x7fcb8c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.152+0000 7fcb94481640 1 -- 172.21.15.26:0/2969724005 shutdown_connections 2024-03-06T07:22:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.152+0000 7fcb94481640 1 -- 172.21.15.26:0/2969724005 wait complete. 2024-03-06T07:22:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.153+0000 7fcb94481640 1 Processor -- start 2024-03-06T07:22:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.153+0000 7fcb94481640 1 -- start start 2024-03-06T07:22:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb94481640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb8c06da40 0x7fcb8c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb94481640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c0c6780 0x7fcb8c1ee210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb94481640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c1ee750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb94481640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0c37c0 con 0x7fcb8c06da40 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb94481640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1594a0 con 0x7fcb8c0c6780 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb94481640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0c3940 con 0x7fcb8c0c7380 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb8c06da40 0x7fcb8c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb8c06da40 0x7fcb8c14ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34316/0 (socket says 172.21.15.26:34316) 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 -- 172.21.15.26:0/776387281 learned_addr learned my addr 172.21.15.26:0/776387281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb91a1c640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c0c6780 0x7fcb8c1ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb92a1e640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c1ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 -- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 msgr2=0x7fcb8c1ee750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c1ee750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 -- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c0c6780 msgr2=0x7fcb8c1ee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c0c6780 0x7fcb8c1ee210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 -- 172.21.15.26:0/776387281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0ab870 con 0x7fcb8c06da40 2024-03-06T07:22:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb9221d640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb8c06da40 0x7fcb8c14ffd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c460 tx=0x7fcb8400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.154+0000 7fcb8affd640 1 -- 172.21.15.26:0/776387281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84017070 con 0x7fcb8c06da40 2024-03-06T07:22:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.155+0000 7fcb8affd640 1 -- 172.21.15.26:0/776387281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb840052f0 con 0x7fcb8c06da40 2024-03-06T07:22:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.155+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c0abb50 con 0x7fcb8c06da40 2024-03-06T07:22:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.155+0000 7fcb8affd640 1 -- 172.21.15.26:0/776387281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84003830 con 0x7fcb8c06da40 2024-03-06T07:22:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.155+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c150dc0 con 0x7fcb8c06da40 2024-03-06T07:22:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.155+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c04f330 con 0x7fcb8c06da40 2024-03-06T07:22:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.156+0000 7fcb8affd640 1 -- 172.21.15.26:0/776387281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb8401a070 con 0x7fcb8c06da40 2024-03-06T07:22:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.156+0000 7fcb8affd640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7806b500 0x7fcb7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.157+0000 7fcb8affd640 1 -- 172.21.15.26:0/776387281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb840895e0 con 0x7fcb8c06da40 2024-03-06T07:22:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.157+0000 7fcb91a1c640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7806b500 0x7fcb7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.157+0000 7fcb91a1c640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7806b500 0x7fcb7806d9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c154f40 tx=0x7fcb80007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.161+0000 7fcb8affd640 1 -- 172.21.15.26:0/776387281 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb84057d10 con 0x7fcb8c06da40 2024-03-06T07:22:40.310 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:22:40.308+0000 7f7f9c337640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.340+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fcb8c073240 con 0x7fcb8c06da40 2024-03-06T07:22:40.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.343+0000 7fcb8affd640 1 -- 172.21.15.26:0/776387281 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v24)=0 v24) v1 ==== 117+0+0 (secure 0 0 0) 0x7fcb8405bdb0 con 0x7fcb8c06da40 2024-03-06T07:22:40.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7806b500 msgr2=0x7fcb7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7806b500 0x7fcb7806d9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c154f40 tx=0x7fcb80007d30 comp rx=0 tx=0).stop 2024-03-06T07:22:40.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb8c06da40 msgr2=0x7fcb8c14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb8c06da40 0x7fcb8c14ffd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c460 tx=0x7fcb8400c920 comp rx=0 tx=0).stop 2024-03-06T07:22:40.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 shutdown_connections 2024-03-06T07:22:40.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7806b500 0x7fcb7806d9c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c0c7380 0x7fcb8c1ee750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c0c6780 0x7fcb8c1ee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 --2- 172.21.15.26:0/776387281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb8c06da40 0x7fcb8c14ffd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 >> 172.21.15.26:0/776387281 conn(0x7fcb8c0aa630 msgr2=0x7fcb8c0c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:40.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 shutdown_connections 2024-03-06T07:22:40.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.345+0000 7fcb94481640 1 -- 172.21.15.26:0/776387281 wait complete. 2024-03-06T07:22:40.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.a mds_join_fs cephfs2 2024-03-06T07:22:40.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.460+0000 7f611909d640 1 Processor -- start 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.460+0000 7f611909d640 1 -- start start 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.460+0000 7f611909d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f61140a6290 0x7f61140a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.460+0000 7f611909d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6114155cf0 0x7f61140aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.460+0000 7f611909d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61140aa8a0 con 0x7f61140a8ff0 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.460+0000 7f611909d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611414f850 con 0x7f6114155cf0 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.460+0000 7f611909d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6114156260 con 0x7f61140a6290 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f611359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6114155cf0 0x7f61140aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f6112d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61140a8ff0 0x7f61140ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47196/0 (socket says 172.21.15.26:47196) 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f6112d9d640 1 -- 172.21.15.26:0/382879616 learned_addr learned my addr 172.21.15.26:0/382879616 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f610bfff640 1 --2- 172.21.15.26:0/382879616 >> v2:172.21.15.117:3300/0 conn(0x7f61140a6290 0x7f61140a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f611359e640 1 -- 172.21.15.26:0/382879616 >> v2:172.21.15.117:3300/0 conn(0x7f61140a6290 msgr2=0x7f61140a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f611359e640 1 --2- 172.21.15.26:0/382879616 >> v2:172.21.15.117:3300/0 conn(0x7f61140a6290 0x7f61140a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f611359e640 1 -- 172.21.15.26:0/382879616 >> v1:172.21.15.26:6789/0 conn(0x7f61140a8ff0 legacy=0x7f61140ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.461+0000 7f611359e640 1 -- 172.21.15.26:0/382879616 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140c1f90 con 0x7f6114155cf0 2024-03-06T07:22:40.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611359e640 1 --2- 172.21.15.26:0/382879616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6114155cf0 0x7f61140aa360 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f611406e560 tx=0x7f610c02dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26a85a7f533134c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611259c640 1 -- 172.21.15.26:0/382879616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c02e940 con 0x7f6114155cf0 2024-03-06T07:22:40.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611259c640 1 -- 172.21.15.26:0/382879616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c02eae0 con 0x7f6114155cf0 2024-03-06T07:22:40.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611259c640 1 -- 172.21.15.26:0/382879616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c009540 con 0x7f6114155cf0 2024-03-06T07:22:40.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 -- 172.21.15.26:0/382879616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6114155cf0 msgr2=0x7f61140aa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 --2- 172.21.15.26:0/382879616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6114155cf0 0x7f61140aa360 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f611406e560 tx=0x7f610c02dfa0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 -- 172.21.15.26:0/382879616 shutdown_connections 2024-03-06T07:22:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 --2- 172.21.15.26:0/382879616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6114155cf0 0x7f61140aa360 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 --2- 172.21.15.26:0/382879616 >> v2:172.21.15.117:3300/0 conn(0x7f61140a6290 0x7f61140a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 -- 172.21.15.26:0/382879616 >> 172.21.15.26:0/382879616 conn(0x7f611406afc0 msgr2=0x7f611406b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 -- 172.21.15.26:0/382879616 shutdown_connections 2024-03-06T07:22:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.462+0000 7f611909d640 1 -- 172.21.15.26:0/382879616 wait complete. 2024-03-06T07:22:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 Processor -- start 2024-03-06T07:22:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 -- start start 2024-03-06T07:22:40.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61140a8ff0 0x7f6114146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6114155cf0 0x7f6114148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611414ff90 0x7f61141478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61140c1790 con 0x7f611414ff90 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61140c14b0 con 0x7f6114155cf0 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611909d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61140c1a70 con 0x7f61140a8ff0 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611414ff90 0x7f61141478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611414ff90 0x7f61141478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34332/0 (socket says 172.21.15.26:34332) 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.464+0000 7f611359e640 1 -- 172.21.15.26:0/3885104374 learned_addr learned my addr 172.21.15.26:0/3885104374 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f6112d9d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61140a8ff0 0x7f6114146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f610bfff640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6114155cf0 0x7f6114148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611359e640 1 -- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61140a8ff0 msgr2=0x7f6114146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611359e640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61140a8ff0 0x7f6114146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611359e640 1 -- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6114155cf0 msgr2=0x7f6114148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611359e640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6114155cf0 0x7f6114148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611359e640 1 -- 172.21.15.26:0/3885104374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114146b10 con 0x7f611414ff90 2024-03-06T07:22:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611359e640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611414ff90 0x7f61141478d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f610c038040 tx=0x7f610c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f610b7fe640 1 -- 172.21.15.26:0/3885104374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c045020 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f610b7fe640 1 -- 172.21.15.26:0/3885104374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c040450 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6114146df0 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f610b7fe640 1 -- 172.21.15.26:0/3885104374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c040840 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.465+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6114149240 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.466+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611404f330 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.467+0000 7f610b7fe640 1 -- 172.21.15.26:0/3885104374 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f610c048020 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.467+0000 7f610b7fe640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60f406b4d0 0x7f60f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.467+0000 7f610b7fe640 1 -- 172.21.15.26:0/3885104374 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f610c0aebe0 con 0x7f611414ff90 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.467+0000 7f6112d9d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60f406b4d0 0x7f60f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.468+0000 7f6112d9d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60f406b4d0 0x7f60f406d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6104000f60 tx=0x7f6104007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.471+0000 7f610b7fe640 1 -- 172.21.15.26:0/3885104374 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f610c04f030 con 0x7f611414ff90 2024-03-06T07:22:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.653+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f61140c24e0 con 0x7f611414ff90 2024-03-06T07:22:40.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.656+0000 7f610b7fe640 1 -- 172.21.15.26:0/3885104374 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v25)=0 v25) v1 ==== 117+0+0 (secure 0 0 0) 0x7f610c07d310 con 0x7f611414ff90 2024-03-06T07:22:40.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60f406b4d0 msgr2=0x7f60f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60f406b4d0 0x7f60f406d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6104000f60 tx=0x7f6104007e80 comp rx=0 tx=0).stop 2024-03-06T07:22:40.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611414ff90 msgr2=0x7f61141478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611414ff90 0x7f61141478d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f610c038040 tx=0x7f610c004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:40.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 shutdown_connections 2024-03-06T07:22:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60f406b4d0 0x7f60f406d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611414ff90 0x7f61141478d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6114155cf0 0x7f6114148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 --2- 172.21.15.26:0/3885104374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61140a8ff0 0x7f6114146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 >> 172.21.15.26:0/3885104374 conn(0x7f611406afc0 msgr2=0x7f6114153b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 shutdown_connections 2024-03-06T07:22:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.658+0000 7f611909d640 1 -- 172.21.15.26:0/3885104374 wait complete. 2024-03-06T07:22:40.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.c mds_join_fs cephfs2 2024-03-06T07:22:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.776+0000 7ff8be3b5640 1 Processor -- start 2024-03-06T07:22:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.776+0000 7ff8be3b5640 1 -- start start 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8be3b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8be3b5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8b8158db0 0x7ff8b815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8be3b5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b8155110 con 0x7ff8b80c88b0 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8be3b5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b81559e0 con 0x7ff8b8158db0 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8be3b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b80c8f40 con 0x7ff8b80c5330 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8bc952640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8b8158db0 0x7ff8b815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8b77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff8b80c88b0 0x7ff8b81586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47204/0 (socket says 172.21.15.26:47204) 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8bc952640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8b8158db0 0x7ff8b815fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34334/0 (socket says 172.21.15.26:34334) 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8bc952640 1 -- 172.21.15.26:0/1073679041 learned_addr learned my addr 172.21.15.26:0/1073679041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8b7fff640 1 --2- 172.21.15.26:0/1073679041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8bc952640 1 -- 172.21.15.26:0/1073679041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 msgr2=0x7ff8b80c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.777+0000 7ff8bc952640 1 --2- 172.21.15.26:0/1073679041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b80c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8bc952640 1 -- 172.21.15.26:0/1073679041 >> v1:172.21.15.26:6789/0 conn(0x7ff8b80c88b0 legacy=0x7ff8b81586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1073679041 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625348971 0 0) 0x7ff8a8001110 con 0x7ff8b80c88b0 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8bc952640 1 -- 172.21.15.26:0/1073679041 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8155390 con 0x7ff8b8158db0 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8b7fff640 1 --2- 172.21.15.26:0/1073679041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b80c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8bc952640 1 --2- 172.21.15.26:0/1073679041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b8158db0 0x7ff8b815fa30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009e50 tx=0x7ff8b002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4509ef3a0a7f728 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1073679041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b002ca70 con 0x7ff8b8158db0 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1073679041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b002cc10 con 0x7ff8b8158db0 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.778+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1073679041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00343f0 con 0x7ff8b8158db0 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 -- 172.21.15.26:0/1073679041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b8158db0 msgr2=0x7ff8b815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/1073679041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b8158db0 0x7ff8b815fa30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009e50 tx=0x7ff8b002bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 -- 172.21.15.26:0/1073679041 shutdown_connections 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/1073679041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b8158db0 0x7ff8b815fa30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/1073679041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b80c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 -- 172.21.15.26:0/1073679041 >> 172.21.15.26:0/1073679041 conn(0x7ff8b80ac560 msgr2=0x7ff8b80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 -- 172.21.15.26:0/1073679041 shutdown_connections 2024-03-06T07:22:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.779+0000 7ff8be3b5640 1 -- 172.21.15.26:0/1073679041 wait complete. 2024-03-06T07:22:40.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.780+0000 7ff8be3b5640 1 Processor -- start 2024-03-06T07:22:40.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.780+0000 7ff8be3b5640 1 -- start start 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8be3b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8be3b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b80c88b0 0x7ff8b80be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8be3b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b81eefa0 0x7ff8b81eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8be3b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b81529d0 con 0x7ff8b80c88b0 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8be3b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b806d090 con 0x7ff8b81eefa0 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8be3b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8152b50 con 0x7ff8b80c5330 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b80c88b0 0x7ff8b80be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b80c88b0 0x7ff8b80be9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34346/0 (socket says 172.21.15.26:34346) 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 -- 172.21.15.26:0/279999196 learned_addr learned my addr 172.21.15.26:0/279999196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8bc952640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b81eefa0 0x7ff8b81eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b7fff640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 -- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 msgr2=0x7ff8b814bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b814bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 -- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b81eefa0 msgr2=0x7ff8b81eb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b81eefa0 0x7ff8b81eb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.781+0000 7ff8b77fe640 1 -- 172.21.15.26:0/279999196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b81576e0 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.782+0000 7ff8b77fe640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b80c88b0 0x7ff8b80be9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c750 tx=0x7ff8a800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.782+0000 7ff8b4ff9640 1 -- 172.21.15.26:0/279999196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8018070 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.782+0000 7ff8b4ff9640 1 -- 172.21.15.26:0/279999196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a80070d0 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.782+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b81ef430 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.782+0000 7ff8b4ff9640 1 -- 172.21.15.26:0/279999196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80074c0 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.782+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b8158550 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.783+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b8067910 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.783+0000 7ff8b4ff9640 1 -- 172.21.15.26:0/279999196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8a801b070 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.784+0000 7ff8b4ff9640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.784+0000 7ff8b4ff9640 1 -- 172.21.15.26:0/279999196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8a8089670 con 0x7ff8b80c88b0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.784+0000 7ff8b7fff640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:40.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.784+0000 7ff8b7fff640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8a406b4d0 0x7ff8a406d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009560 tx=0x7ff8ac009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:40.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.788+0000 7ff8b4ff9640 1 -- 172.21.15.26:0/279999196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff8b8067910 con 0x7ff8b80c88b0 2024-03-06T07:22:40.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.968+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7ff8b8075300 con 0x7ff8b80c88b0 2024-03-06T07:22:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.971+0000 7ff8b4ff9640 1 -- 172.21.15.26:0/279999196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v26)=0 v26) v1 ==== 117+0+0 (secure 0 0 0) 0x7ff8b8075300 con 0x7ff8b80c88b0 2024-03-06T07:22:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8a406b4d0 msgr2=0x7ff8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8a406b4d0 0x7ff8a406d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009560 tx=0x7ff8ac009190 comp rx=0 tx=0).stop 2024-03-06T07:22:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b80c88b0 msgr2=0x7ff8b80be9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b80c88b0 0x7ff8b80be9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c750 tx=0x7ff8a800cc10 comp rx=0 tx=0).stop 2024-03-06T07:22:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 shutdown_connections 2024-03-06T07:22:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b81eefa0 0x7ff8b81eb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b80c88b0 0x7ff8b80be9c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 --2- 172.21.15.26:0/279999196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80c5330 0x7ff8b814bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 >> 172.21.15.26:0/279999196 conn(0x7ff8b80ac560 msgr2=0x7ff8b81549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 shutdown_connections 2024-03-06T07:22:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:40.973+0000 7ff8be3b5640 1 -- 172.21.15.26:0/279999196 wait complete. 2024-03-06T07:22:40.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:41.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.086+0000 7fc77fec0640 1 Processor -- start 2024-03-06T07:22:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.086+0000 7fc77fec0640 1 -- start start 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77fec0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7780c6780 0x7fc778158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77fec0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7781592d0 0x7fc7780c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77fec0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc778072630 con 0x7fc77806da40 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77fec0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77806de20 con 0x7fc7781592d0 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77fec0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc778072910 con 0x7fc7780c6780 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77e45d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7781592d0 0x7fc7780c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77e45d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7781592d0 0x7fc7780c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34362/0 (socket says 172.21.15.26:34362) 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77e45d640 1 -- 172.21.15.26:0/594648136 learned_addr learned my addr 172.21.15.26:0/594648136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77d45b640 1 --2- 172.21.15.26:0/594648136 >> v2:172.21.15.117:3300/0 conn(0x7fc7780c6780 0x7fc778158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77e45d640 1 -- 172.21.15.26:0/594648136 >> v2:172.21.15.117:3300/0 conn(0x7fc7780c6780 msgr2=0x7fc778158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77e45d640 1 --2- 172.21.15.26:0/594648136 >> v2:172.21.15.117:3300/0 conn(0x7fc7780c6780 0x7fc778158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77cc5a640 1 -- 172.21.15.26:0/594648136 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715039607 0 0) 0x7fc778072630 con 0x7fc77806da40 2024-03-06T07:22:41.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77e45d640 1 -- 172.21.15.26:0/594648136 >> v1:172.21.15.117:6789/0 conn(0x7fc77806da40 legacy=0x7fc778155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.087+0000 7fc77e45d640 1 -- 172.21.15.26:0/594648136 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778072bf0 con 0x7fc7781592d0 2024-03-06T07:22:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.088+0000 7fc77e45d640 1 --2- 172.21.15.26:0/594648136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7781592d0 0x7fc7780c3280 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc7780c7120 tx=0x7fc77002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19da402c27edae89 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.088+0000 7fc77cc5a640 1 -- 172.21.15.26:0/594648136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77002ca80 con 0x7fc7781592d0 2024-03-06T07:22:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.088+0000 7fc77cc5a640 1 -- 172.21.15.26:0/594648136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77002cc20 con 0x7fc7781592d0 2024-03-06T07:22:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.088+0000 7fc77cc5a640 1 -- 172.21.15.26:0/594648136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700343f0 con 0x7fc7781592d0 2024-03-06T07:22:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.088+0000 7fc77fec0640 1 -- 172.21.15.26:0/594648136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7781592d0 msgr2=0x7fc7780c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.088+0000 7fc77fec0640 1 --2- 172.21.15.26:0/594648136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7781592d0 0x7fc7780c3280 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc7780c7120 tx=0x7fc77002bee0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.089+0000 7fc77fec0640 1 -- 172.21.15.26:0/594648136 shutdown_connections 2024-03-06T07:22:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.089+0000 7fc77fec0640 1 --2- 172.21.15.26:0/594648136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7781592d0 0x7fc7780c3280 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.089+0000 7fc77fec0640 1 --2- 172.21.15.26:0/594648136 >> v2:172.21.15.117:3300/0 conn(0x7fc7780c6780 0x7fc778158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.089+0000 7fc77fec0640 1 -- 172.21.15.26:0/594648136 >> 172.21.15.26:0/594648136 conn(0x7fc7780aa630 msgr2=0x7fc7780aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.089+0000 7fc77fec0640 1 -- 172.21.15.26:0/594648136 shutdown_connections 2024-03-06T07:22:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.089+0000 7fc77fec0640 1 -- 172.21.15.26:0/594648136 wait complete. 2024-03-06T07:22:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 Processor -- start 2024-03-06T07:22:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 -- start start 2024-03-06T07:22:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc77806da40 0x7fc7780adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7780c6780 0x7fc778157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7781592d0 0x7fc7781ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780c3940 con 0x7fc7780c6780 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780c37c0 con 0x7fc7781592d0 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77fec0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780c3ac0 con 0x7fc77806da40 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77d45b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7780c6780 0x7fc778157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77d45b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7780c6780 0x7fc778157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34374/0 (socket says 172.21.15.26:34374) 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.090+0000 7fc77d45b640 1 -- 172.21.15.26:0/3949013512 learned_addr learned my addr 172.21.15.26:0/3949013512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77e45d640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7781592d0 0x7fc7781ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77dc5c640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc77806da40 0x7fc7780adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77d45b640 1 -- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc77806da40 msgr2=0x7fc7780adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77d45b640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc77806da40 0x7fc7780adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77d45b640 1 -- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7781592d0 msgr2=0x7fc7781ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77d45b640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7781592d0 0x7fc7781ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77d45b640 1 -- 172.21.15.26:0/3949013512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7780ae740 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77d45b640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7780c6780 0x7fc778157740 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc76800e8a0 tx=0x7fc76800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc76e7fc640 1 -- 172.21.15.26:0/3949013512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768017070 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc76e7fc640 1 -- 172.21.15.26:0/3949013512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76800c940 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7780aea20 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc76e7fc640 1 -- 172.21.15.26:0/3949013512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76800cd30 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.091+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7780ad660 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.092+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77804f330 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.093+0000 7fc76e7fc640 1 -- 172.21.15.26:0/3949013512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc76801a070 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.093+0000 7fc76e7fc640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc75c06b500 0x7fc75c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.093+0000 7fc76e7fc640 1 -- 172.21.15.26:0/3949013512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc76808a510 con 0x7fc7780c6780 2024-03-06T07:22:41.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.093+0000 7fc77dc5c640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc75c06b500 0x7fc75c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.094+0000 7fc77dc5c640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc75c06b500 0x7fc75c06d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc774000f60 tx=0x7fc774009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.097+0000 7fc76e7fc640 1 -- 172.21.15.26:0/3949013512 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc768058bb0 con 0x7fc7780c6780 2024-03-06T07:22:41.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.279+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc778073240 con 0x7fc7780c6780 2024-03-06T07:22:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.280+0000 7fc76e7fc640 1 -- 172.21.15.26:0/3949013512 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16957 (secure 0 0 0) 0x7fc76805cc50 con 0x7fc7780c6780 2024-03-06T07:22:41.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:41.283 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":74,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:36.025110+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:41.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc75c06b500 msgr2=0x7fc75c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc75c06b500 0x7fc75c06d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc774000f60 tx=0x7fc774009040 comp rx=0 tx=0).stop 2024-03-06T07:22:41.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7780c6780 msgr2=0x7fc778157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7780c6780 0x7fc778157740 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc76800e8a0 tx=0x7fc76800ed60 comp rx=0 tx=0).stop 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 shutdown_connections 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc75c06b500 0x7fc75c06d9c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7781592d0 0x7fc7781ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7780c6780 0x7fc778157740 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 --2- 172.21.15.26:0/3949013512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc77806da40 0x7fc7780adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 >> 172.21.15.26:0/3949013512 conn(0x7fc7780aa630 msgr2=0x7fc7780aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 shutdown_connections 2024-03-06T07:22:41.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.282+0000 7fc77fec0640 1 -- 172.21.15.26:0/3949013512 wait complete. 2024-03-06T07:22:41.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:22:32.646079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:22:33.350567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:41.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.399+0000 7fb86704e640 1 Processor -- start 2024-03-06T07:22:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.399+0000 7fb86704e640 1 -- start start 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.399+0000 7fb86704e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8600c5370 0x7fb8600c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.399+0000 7fb86704e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb860155530 0x7fb8601559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb86704e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb860152dd0 con 0x7fb8601529f0 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb86704e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb860159710 con 0x7fb860155530 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb86704e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600c68b0 con 0x7fb8600c5370 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb864dea640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8601529f0 0x7fb8600c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47220/0 (socket says 172.21.15.26:47220) 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb864dea640 1 -- 172.21.15.26:0/2520806574 learned_addr learned my addr 172.21.15.26:0/2520806574 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb8655eb640 1 --2- 172.21.15.26:0/2520806574 >> v2:172.21.15.117:3300/0 conn(0x7fb860155530 0x7fb8601559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb85ffff640 1 --2- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8600c5370 0x7fb8600c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb85f7fe640 1 -- 172.21.15.26:0/2520806574 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584390350 0 0) 0x7fb860152dd0 con 0x7fb8601529f0 2024-03-06T07:22:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb85f7fe640 1 -- 172.21.15.26:0/2520806574 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb850003520 con 0x7fb8601529f0 2024-03-06T07:22:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb8655eb640 1 -- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8600c5370 msgr2=0x7fb8600c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb8655eb640 1 --2- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8600c5370 0x7fb8600c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb8655eb640 1 -- 172.21.15.26:0/2520806574 >> v1:172.21.15.26:6789/0 conn(0x7fb8601529f0 legacy=0x7fb8600c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb8655eb640 1 -- 172.21.15.26:0/2520806574 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860158b00 con 0x7fb860155530 2024-03-06T07:22:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.400+0000 7fb85ffff640 1 --2- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8600c5370 0x7fb8600c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.401+0000 7fb8655eb640 1 --2- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb860155530 0x7fb8601559c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb858009d90 tx=0x7fb85802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff01fb6bba54951 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.401+0000 7fb85f7fe640 1 -- 172.21.15.26:0/2520806574 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858039070 con 0x7fb860155530 2024-03-06T07:22:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.401+0000 7fb85f7fe640 1 -- 172.21.15.26:0/2520806574 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85802cbc0 con 0x7fb860155530 2024-03-06T07:22:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.401+0000 7fb85f7fe640 1 -- 172.21.15.26:0/2520806574 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858034440 con 0x7fb860155530 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 -- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb860155530 msgr2=0x7fb8601559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 --2- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb860155530 0x7fb8601559c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb858009d90 tx=0x7fb85802be10 comp rx=0 tx=0).stop 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 -- 172.21.15.26:0/2520806574 shutdown_connections 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 --2- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb860155530 0x7fb8601559c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 --2- 172.21.15.26:0/2520806574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8600c5370 0x7fb8600c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 -- 172.21.15.26:0/2520806574 >> 172.21.15.26:0/2520806574 conn(0x7fb8600ac580 msgr2=0x7fb8600ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 -- 172.21.15.26:0/2520806574 shutdown_connections 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.402+0000 7fb86704e640 1 -- 172.21.15.26:0/2520806574 wait complete. 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.403+0000 7fb86704e640 1 Processor -- start 2024-03-06T07:22:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.403+0000 7fb86704e640 1 -- start start 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.403+0000 7fb86704e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8600c5370 0x7fb8600ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.403+0000 7fb86704e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8601529f0 0x7fb8600ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.403+0000 7fb86704e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb860155530 0x7fb8600c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb86704e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601590c0 con 0x7fb860155530 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb86704e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600c9000 con 0x7fb8600c5370 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb86704e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601593a0 con 0x7fb8601529f0 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb860155530 0x7fb8600c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb860155530 0x7fb8600c0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34386/0 (socket says 172.21.15.26:34386) 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 -- 172.21.15.26:0/439263569 learned_addr learned my addr 172.21.15.26:0/439263569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb864dea640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8600c5370 0x7fb8600ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb85ffff640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8601529f0 0x7fb8600ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 -- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8601529f0 msgr2=0x7fb8600ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8601529f0 0x7fb8600ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 -- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8600c5370 msgr2=0x7fb8600ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8600c5370 0x7fb8600ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 -- 172.21.15.26:0/439263569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8600be1e0 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb8655eb640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb860155530 0x7fb8600c0a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb858009d90 tx=0x7fb858004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb85d7fa640 1 -- 172.21.15.26:0/439263569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858044020 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb85d7fa640 1 -- 172.21.15.26:0/439263569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85803f470 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8600b0510 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb85d7fa640 1 -- 172.21.15.26:0/439263569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858039040 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.404+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8600bfd90 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.405+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86004f330 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.406+0000 7fb85d7fa640 1 -- 172.21.15.26:0/439263569 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb858047020 con 0x7fb860155530 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.406+0000 7fb85d7fa640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb84406b500 0x7fb84406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.406+0000 7fb85d7fa640 1 -- 172.21.15.26:0/439263569 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8580aca00 con 0x7fb860155530 2024-03-06T07:22:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.406+0000 7fb864dea640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb84406b500 0x7fb84406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.407+0000 7fb864dea640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb84406b500 0x7fb84406d9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb854005880 tx=0x7fb854009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.410+0000 7fb85d7fa640 1 -- 172.21.15.26:0/439263569 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb85804e070 con 0x7fb860155530 2024-03-06T07:22:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.592+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb86014fe90 con 0x7fb860155530 2024-03-06T07:22:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.592+0000 7fb85d7fa640 1 -- 172.21.15.26:0/439263569 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb85807b310 con 0x7fb860155530 2024-03-06T07:22:41.595 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb84406b500 msgr2=0x7fb84406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb84406b500 0x7fb84406d9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb854005880 tx=0x7fb854009040 comp rx=0 tx=0).stop 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb860155530 msgr2=0x7fb8600c0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb860155530 0x7fb8600c0a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb858009d90 tx=0x7fb858004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 shutdown_connections 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb84406b500 0x7fb84406d9c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb860155530 0x7fb8600c0a50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8601529f0 0x7fb8600ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 --2- 172.21.15.26:0/439263569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8600c5370 0x7fb8600ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 >> 172.21.15.26:0/439263569 conn(0x7fb8600ac580 msgr2=0x7fb8600ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 shutdown_connections 2024-03-06T07:22:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.594+0000 7fb86704e640 1 -- 172.21.15.26:0/439263569 wait complete. 2024-03-06T07:22:41.606 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:41.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:22:41.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.709+0000 7fb368884640 1 Processor -- start 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.709+0000 7fb368884640 1 -- start start 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb368884640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3640c4740 0x7fb3641539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb368884640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb368884640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3640c62a0 con 0x7fb3640a8290 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb368884640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb364153f00 con 0x7fb3640c4740 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb368884640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640c4c00 con 0x7fb3640c69a0 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb35bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3640c4740 0x7fb3641539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb36259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb3640a8290 0x7fb3640701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47232/0 (socket says 172.21.15.26:47232) 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb35bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3640c4740 0x7fb3641539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34392/0 (socket says 172.21.15.26:34392) 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb36259c640 1 -- 172.21.15.26:0/3450898499 learned_addr learned my addr 172.21.15.26:0/3450898499 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.710+0000 7fb362d9d640 1 --2- 172.21.15.26:0/3450898499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.711+0000 7fb35bfff640 1 -- 172.21.15.26:0/3450898499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 msgr2=0x7fb364072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.711+0000 7fb35bfff640 1 --2- 172.21.15.26:0/3450898499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.711+0000 7fb35bfff640 1 -- 172.21.15.26:0/3450898499 >> v1:172.21.15.26:6789/0 conn(0x7fb3640a8290 legacy=0x7fb3640701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.711+0000 7fb361d9b640 1 -- 172.21.15.26:0/3450898499 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258725888 0 0) 0x7fb3640c62a0 con 0x7fb3640a8290 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.711+0000 7fb35bfff640 1 -- 172.21.15.26:0/3450898499 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3640c4ee0 con 0x7fb3640c4740 2024-03-06T07:22:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.711+0000 7fb362d9d640 1 --2- 172.21.15.26:0/3450898499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.711+0000 7fb35bfff640 1 --2- 172.21.15.26:0/3450898499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb3641539c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097a0 tx=0x7fb34c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559cf7a21218d93e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb361d9b640 1 -- 172.21.15.26:0/3450898499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c038070 con 0x7fb3640c4740 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb361d9b640 1 -- 172.21.15.26:0/3450898499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c031700 con 0x7fb3640c4740 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb361d9b640 1 -- 172.21.15.26:0/3450898499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c031ad0 con 0x7fb3640c4740 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 -- 172.21.15.26:0/3450898499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 msgr2=0x7fb3641539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 --2- 172.21.15.26:0/3450898499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb3641539c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097a0 tx=0x7fb34c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 -- 172.21.15.26:0/3450898499 shutdown_connections 2024-03-06T07:22:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 --2- 172.21.15.26:0/3450898499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb3641539c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 --2- 172.21.15.26:0/3450898499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 -- 172.21.15.26:0/3450898499 >> 172.21.15.26:0/3450898499 conn(0x7fb3640c5560 msgr2=0x7fb3640c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 -- 172.21.15.26:0/3450898499 shutdown_connections 2024-03-06T07:22:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.712+0000 7fb368884640 1 -- 172.21.15.26:0/3450898499 wait complete. 2024-03-06T07:22:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.713+0000 7fb368884640 1 Processor -- start 2024-03-06T07:22:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb368884640 1 -- start start 2024-03-06T07:22:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb368884640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3640a8290 0x7fb3640c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb368884640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb364155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb368884640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb368884640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640c6e30 con 0x7fb3640c4740 2024-03-06T07:22:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb368884640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36406fc80 con 0x7fb3640a8290 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb368884640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb364072bc0 con 0x7fb3640c69a0 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb35bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb364155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb35bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb364155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34400/0 (socket says 172.21.15.26:34400) 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb35bfff640 1 -- 172.21.15.26:0/3576890902 learned_addr learned my addr 172.21.15.26:0/3576890902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb36259c640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3640a8290 0x7fb3640c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.714+0000 7fb362d9d640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35bfff640 1 -- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 msgr2=0x7fb364156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35bfff640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35bfff640 1 -- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3640a8290 msgr2=0x7fb3640c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35bfff640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3640a8290 0x7fb3640c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35bfff640 1 -- 172.21.15.26:0/3576890902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36414be30 con 0x7fb3640c4740 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35bfff640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb364155b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb34c036040 tx=0x7fb34c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35affd640 1 -- 172.21.15.26:0/3576890902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c042070 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35affd640 1 -- 172.21.15.26:0/3576890902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c0339b0 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3641eb540 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb35affd640 1 -- 172.21.15.26:0/3576890902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c038040 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.715+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36414c1e0 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.716+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3641eb330 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.716+0000 7fb35affd640 1 -- 172.21.15.26:0/3576890902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb34c03b070 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.717+0000 7fb35affd640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb34406b4d0 0x7fb34406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.717+0000 7fb35affd640 1 -- 172.21.15.26:0/3576890902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb34c0abb70 con 0x7fb3640c4740 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.717+0000 7fb36259c640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb34406b4d0 0x7fb34406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.717+0000 7fb36259c640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb34406b4d0 0x7fb34406d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb354000f60 tx=0x7fb354007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:41.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.721+0000 7fb35affd640 1 -- 172.21.15.26:0/3576890902 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb34c030070 con 0x7fb3640c4740 2024-03-06T07:22:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.900+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb36404f330 con 0x7fb3640c4740 2024-03-06T07:22:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.901+0000 7fb35affd640 1 -- 172.21.15.26:0/3576890902 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb34c07a220 con 0x7fb3640c4740 2024-03-06T07:22:41.904 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb34406b4d0 msgr2=0x7fb34406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb34406b4d0 0x7fb34406d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb354000f60 tx=0x7fb354007d50 comp rx=0 tx=0).stop 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 msgr2=0x7fb364155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb364155b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb34c036040 tx=0x7fb34c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 shutdown_connections 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb34406b4d0 0x7fb34406d990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3640c69a0 0x7fb364156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3640c4740 0x7fb364155b40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 --2- 172.21.15.26:0/3576890902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3640a8290 0x7fb3640c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 >> 172.21.15.26:0/3576890902 conn(0x7fb3640c5560 msgr2=0x7fb3640c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 shutdown_connections 2024-03-06T07:22:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:41.903+0000 7fb368884640 1 -- 172.21.15.26:0/3576890902 wait complete. 2024-03-06T07:22:41.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:22:42.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.023+0000 7ff1a16eb640 1 Processor -- start 2024-03-06T07:22:42.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.023+0000 7ff1a16eb640 1 -- start start 2024-03-06T07:22:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff1a16eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff19c0c5990 0x7ff19c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff1a16eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff1a16eb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0c3ef0 con 0x7ff19c06c730 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff1a16eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c06cbe0 con 0x7ff19c0c5990 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff1a16eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1556d0 con 0x7ff19c0ab170 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19affd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff19c0c5990 0x7ff19c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19a7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19affd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff19c0c5990 0x7ff19c0c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:54858/0 (socket says 172.21.15.26:54858) 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19affd640 1 -- 172.21.15.26:0/4267343859 learned_addr learned my addr 172.21.15.26:0/4267343859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19b7fe640 1 --1- 172.21.15.26:0/4267343859 >> v1:172.21.15.117:6789/0 conn(0x7ff19c06c730 0x7ff19c1523f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:40182/0 (socket says 172.21.15.26:40182) 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19affd640 1 -- 172.21.15.26:0/4267343859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 msgr2=0x7ff19c06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19affd640 1 --2- 172.21.15.26:0/4267343859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c06c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19affd640 1 -- 172.21.15.26:0/4267343859 >> v1:172.21.15.117:6789/0 conn(0x7ff19c06c730 legacy=0x7ff19c1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.024+0000 7ff19affd640 1 -- 172.21.15.26:0/4267343859 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c1559b0 con 0x7ff19c0c5990 2024-03-06T07:22:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.025+0000 7ff19affd640 1 --2- 172.21.15.26:0/4267343859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 0x7ff19c0c0ab0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff194009870 tx=0x7ff19402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f213fcf834508a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.025+0000 7ff199ffb640 1 -- 172.21.15.26:0/4267343859 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194035020 con 0x7ff19c0c5990 2024-03-06T07:22:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.025+0000 7ff199ffb640 1 -- 172.21.15.26:0/4267343859 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194031640 con 0x7ff19c0c5990 2024-03-06T07:22:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.025+0000 7ff199ffb640 1 -- 172.21.15.26:0/4267343859 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19402d040 con 0x7ff19c0c5990 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 -- 172.21.15.26:0/4267343859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 msgr2=0x7ff19c0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/4267343859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 0x7ff19c0c0ab0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff194009870 tx=0x7ff19402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 -- 172.21.15.26:0/4267343859 shutdown_connections 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/4267343859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/4267343859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 0x7ff19c0c0ab0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 -- 172.21.15.26:0/4267343859 >> 172.21.15.26:0/4267343859 conn(0x7ff19c0c6710 msgr2=0x7ff19c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 -- 172.21.15.26:0/4267343859 shutdown_connections 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.026+0000 7ff1a16eb640 1 -- 172.21.15.26:0/4267343859 wait complete. 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.027+0000 7ff1a16eb640 1 Processor -- start 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.027+0000 7ff1a16eb640 1 -- start start 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.027+0000 7ff1a16eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c06c730 0x7ff19c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.027+0000 7ff1a16eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c0a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.027+0000 7ff1a16eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 0x7ff19c0a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.027+0000 7ff1a16eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c156490 con 0x7ff19c06c730 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.027+0000 7ff1a16eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c155f70 con 0x7ff19c0c5990 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff1a16eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c156610 con 0x7ff19c0ab170 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c06c730 0x7ff19c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c06c730 0x7ff19c151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34404/0 (socket says 172.21.15.26:34404) 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 -- 172.21.15.26:0/922188710 learned_addr learned my addr 172.21.15.26:0/922188710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19b7fe640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 0x7ff19c0a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19a7fc640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c0a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 -- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 msgr2=0x7ff19c0a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c0a7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 -- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 msgr2=0x7ff19c0a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 0x7ff19c0a8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 -- 172.21.15.26:0/922188710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0a8560 con 0x7ff19c06c730 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19a7fc640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c0a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff19affd640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c06c730 0x7ff19c151ae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff19402b930 tx=0x7ff1940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff1937fe640 1 -- 172.21.15.26:0/922188710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194040070 con 0x7ff19c06c730 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff1937fe640 1 -- 172.21.15.26:0/922188710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19403e070 con 0x7ff19c06c730 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c1ecc40 con 0x7ff19c06c730 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff1937fe640 1 -- 172.21.15.26:0/922188710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194035040 con 0x7ff19c06c730 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.028+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c1db6e0 con 0x7ff19c06c730 2024-03-06T07:22:42.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.029+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c04f330 con 0x7ff19c06c730 2024-03-06T07:22:42.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.030+0000 7ff1937fe640 1 -- 172.21.15.26:0/922188710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff194038020 con 0x7ff19c06c730 2024-03-06T07:22:42.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.030+0000 7ff1937fe640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b500 0x7ff17c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.030+0000 7ff1937fe640 1 -- 172.21.15.26:0/922188710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1940aba90 con 0x7ff19c06c730 2024-03-06T07:22:42.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.030+0000 7ff19a7fc640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b500 0x7ff17c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.031+0000 7ff19a7fc640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b500 0x7ff17c06d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff19c1ebdb0 tx=0x7ff184009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.035+0000 7ff1937fe640 1 -- 172.21.15.26:0/922188710 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff19407a140 con 0x7ff19c06c730 2024-03-06T07:22:42.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.217+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff19c0c1740 con 0x7ff19c06c730 2024-03-06T07:22:42.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.217+0000 7ff1937fe640 1 -- 172.21.15.26:0/922188710 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff19407e1e0 con 0x7ff19c06c730 2024-03-06T07:22:42.220 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:22:42.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b500 msgr2=0x7ff17c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b500 0x7ff17c06d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff19c1ebdb0 tx=0x7ff184009190 comp rx=0 tx=0).stop 2024-03-06T07:22:42.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c06c730 msgr2=0x7ff19c151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c06c730 0x7ff19c151ae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff19402b930 tx=0x7ff1940046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 shutdown_connections 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b500 0x7ff17c06d9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c0c5990 0x7ff19c0a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0ab170 0x7ff19c0a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 --2- 172.21.15.26:0/922188710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c06c730 0x7ff19c151ae0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 >> 172.21.15.26:0/922188710 conn(0x7ff19c0c6710 msgr2=0x7ff19c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.219+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 shutdown_connections 2024-03-06T07:22:42.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.220+0000 7ff1a16eb640 1 -- 172.21.15.26:0/922188710 wait complete. 2024-03-06T07:22:42.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:22:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.331+0000 7fbdcf9c9640 1 Processor -- start 2024-03-06T07:22:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.331+0000 7fbdcf9c9640 1 -- start start 2024-03-06T07:22:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdcf9c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdc81571f0 0x7fbdc8153a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdcf9c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbdc80c7d10 0x7fbdc80c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdcf9c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc80c81a0 con 0x7fbdc8072720 2024-03-06T07:22:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdcf9c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc81564c0 con 0x7fbdc81571f0 2024-03-06T07:22:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdcf9c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80c8440 con 0x7fbdc80c7d10 2024-03-06T07:22:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdccf64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdc81571f0 0x7fbdc8153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdccf64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdc81571f0 0x7fbdc8153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34406/0 (socket says 172.21.15.26:34406) 2024-03-06T07:22:42.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdccf64640 1 -- 172.21.15.26:0/513663705 learned_addr learned my addr 172.21.15.26:0/513663705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:42.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdcdf66640 1 --2- 172.21.15.26:0/513663705 >> v2:172.21.15.117:3300/0 conn(0x7fbdc80c7d10 0x7fbdc80c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdccf64640 1 -- 172.21.15.26:0/513663705 >> v2:172.21.15.117:3300/0 conn(0x7fbdc80c7d10 msgr2=0x7fbdc80c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:42.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdccf64640 1 --2- 172.21.15.26:0/513663705 >> v2:172.21.15.117:3300/0 conn(0x7fbdc80c7d10 0x7fbdc80c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdccf64640 1 -- 172.21.15.26:0/513663705 >> v1:172.21.15.117:6789/0 conn(0x7fbdc8072720 legacy=0x7fbdc80c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.332+0000 7fbdccf64640 1 -- 172.21.15.26:0/513663705 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8156710 con 0x7fbdc81571f0 2024-03-06T07:22:42.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.333+0000 7fbdccf64640 1 --2- 172.21.15.26:0/513663705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc81571f0 0x7fbdc8153a00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8156640 tx=0x7fbdbc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62edc16ac2b51601 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.333+0000 7fbdc7fff640 1 -- 172.21.15.26:0/513663705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc02e070 con 0x7fbdc81571f0 2024-03-06T07:22:42.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.333+0000 7fbdc7fff640 1 -- 172.21.15.26:0/513663705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc037070 con 0x7fbdc81571f0 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.333+0000 7fbdc7fff640 1 -- 172.21.15.26:0/513663705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc031a00 con 0x7fbdc81571f0 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/513663705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc81571f0 msgr2=0x7fbdc8153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/513663705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc81571f0 0x7fbdc8153a00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8156640 tx=0x7fbdbc02b880 comp rx=0 tx=0).stop 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/513663705 shutdown_connections 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/513663705 >> v2:172.21.15.117:3300/0 conn(0x7fbdc80c7d10 0x7fbdc80c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/513663705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc81571f0 0x7fbdc8153a00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/513663705 >> 172.21.15.26:0/513663705 conn(0x7fbdc80aa6a0 msgr2=0x7fbdc80aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/513663705 shutdown_connections 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.334+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/513663705 wait complete. 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.335+0000 7fbdcf9c9640 1 Processor -- start 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.335+0000 7fbdcf9c9640 1 -- start start 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcf9c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc8072720 0x7fbdc80c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcf9c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdc80c7d10 0x7fbdc806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcf9c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdc81571f0 0x7fbdc806c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcf9c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c4ed0 con 0x7fbdc8072720 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcf9c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc8156cd0 con 0x7fbdc80c7d10 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcf9c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c5050 con 0x7fbdc81571f0 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcd765640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc8072720 0x7fbdc80c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcd765640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc8072720 0x7fbdc80c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34416/0 (socket says 172.21.15.26:34416) 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcdf66640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdc81571f0 0x7fbdc806c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdccf64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdc80c7d10 0x7fbdc806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcd765640 1 -- 172.21.15.26:0/3828507148 learned_addr learned my addr 172.21.15.26:0/3828507148 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcdf66640 1 -- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdc80c7d10 msgr2=0x7fbdc806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcdf66640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdc80c7d10 0x7fbdc806c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcdf66640 1 -- 172.21.15.26:0/3828507148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc8072720 msgr2=0x7fbdc80c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcdf66640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc8072720 0x7fbdc80c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcdf66640 1 -- 172.21.15.26:0/3828507148 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc81eae30 con 0x7fbdc81571f0 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdcd765640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc8072720 0x7fbdc80c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.336+0000 7fbdccf64640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdc80c7d10 0x7fbdc806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.337+0000 7fbdcdf66640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdc81571f0 0x7fbdc806c610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e7f0 tx=0x7fbdb800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.337+0000 7fbdc5ffb640 1 -- 172.21.15.26:0/3828507148 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8017070 con 0x7fbdc81571f0 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.337+0000 7fbdc5ffb640 1 -- 172.21.15.26:0/3828507148 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb800c940 con 0x7fbdc81571f0 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.337+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc81eb110 con 0x7fbdc81571f0 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.337+0000 7fbdc5ffb640 1 -- 172.21.15.26:0/3828507148 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800cd10 con 0x7fbdc81571f0 2024-03-06T07:22:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.337+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc81eb6d0 con 0x7fbdc81571f0 2024-03-06T07:22:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.338+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc80c2530 con 0x7fbdc81571f0 2024-03-06T07:22:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.338+0000 7fbdc5ffb640 1 -- 172.21.15.26:0/3828507148 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdb801a070 con 0x7fbdc81571f0 2024-03-06T07:22:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.339+0000 7fbdc5ffb640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbdb406b4d0 0x7fbdb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.339+0000 7fbdc5ffb640 1 -- 172.21.15.26:0/3828507148 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbdb808a540 con 0x7fbdc81571f0 2024-03-06T07:22:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.339+0000 7fbdcd765640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbdb406b4d0 0x7fbdb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.339+0000 7fbdcd765640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbdb406b4d0 0x7fbdb406d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0007fd0 tx=0x7fbdc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.344+0000 7fbdc5ffb640 1 -- 172.21.15.26:0/3828507148 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbdb8058c70 con 0x7fbdc81571f0 2024-03-06T07:22:42.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.525+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbdc804f330 con 0x7fbdc81571f0 2024-03-06T07:22:42.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.525+0000 7fbdc5ffb640 1 -- 172.21.15.26:0/3828507148 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+209 (secure 0 0 0) 0x7fbdb805cd10 con 0x7fbdc81571f0 2024-03-06T07:22:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:42.528 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:22:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:22:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 11, 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_metadata" 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 12, 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_data" 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:22:42.529 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:22:42.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbdb406b4d0 msgr2=0x7fbdb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbdb406b4d0 0x7fbdb406d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0007fd0 tx=0x7fbdc0009040 comp rx=0 tx=0).stop 2024-03-06T07:22:42.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdc81571f0 msgr2=0x7fbdc806c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdc81571f0 0x7fbdc806c610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e7f0 tx=0x7fbdb800ecb0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 shutdown_connections 2024-03-06T07:22:42.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbdb406b4d0 0x7fbdb406d990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdc81571f0 0x7fbdc806c610 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdc80c7d10 0x7fbdc806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 --2- 172.21.15.26:0/3828507148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc8072720 0x7fbdc80c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 >> 172.21.15.26:0/3828507148 conn(0x7fbdc80aa6a0 msgr2=0x7fbdc81539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:42.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 shutdown_connections 2024-03-06T07:22:42.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.527+0000 7fbdcf9c9640 1 -- 172.21.15.26:0/3828507148 wait complete. 2024-03-06T07:22:42.540 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-06T07:22:42.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-06T07:22:42.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.643+0000 7f14d2b65640 1 Processor -- start 2024-03-06T07:22:42.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.643+0000 7f14d2b65640 1 -- start start 2024-03-06T07:22:42.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.644+0000 7f14d2b65640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14cc0c88e0 0x7f14cc0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.644+0000 7f14d2b65640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc0c5400 con 0x7f14cc0c7930 2024-03-06T07:22:42.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.644+0000 7f14d2b65640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc0c56e0 con 0x7f14cc152870 2024-03-06T07:22:42.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.644+0000 7f14d2b65640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14cc06d320 con 0x7f14cc0c88e0 2024-03-06T07:22:42.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.644+0000 7f14d1102640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f14cc0c7930 0x7f14cc06bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47248/0 (socket says 172.21.15.26:47248) 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.644+0000 7f14d1102640 1 -- 172.21.15.26:0/489422763 learned_addr learned my addr 172.21.15.26:0/489422763 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.644+0000 7f14cbfff640 1 --2- 172.21.15.26:0/489422763 >> v2:172.21.15.117:3300/0 conn(0x7f14cc0c88e0 0x7f14cc0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.645+0000 7f14cb7fe640 1 -- 172.21.15.26:0/489422763 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913819259 0 0) 0x7f14cc0c5400 con 0x7f14cc0c7930 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.645+0000 7f14cb7fe640 1 -- 172.21.15.26:0/489422763 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14bc003520 con 0x7f14cc0c7930 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.645+0000 7f14cbfff640 1 -- 172.21.15.26:0/489422763 >> v1:172.21.15.117:6789/0 conn(0x7f14cc152870 legacy=0x7f14cc155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.645+0000 7f14cbfff640 1 -- 172.21.15.26:0/489422763 >> v1:172.21.15.26:6789/0 conn(0x7f14cc0c7930 legacy=0x7f14cc06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.645+0000 7f14cb7fe640 1 -- 172.21.15.26:0/489422763 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328321205 0 0) 0x7f14cc0c56e0 con 0x7f14cc152870 2024-03-06T07:22:42.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.645+0000 7f14cbfff640 1 -- 172.21.15.26:0/489422763 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0c59c0 con 0x7f14cc0c88e0 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14cbfff640 1 --2- 172.21.15.26:0/489422763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 0x7f14cc0c73f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14b80097a0 tx=0x7f14b802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ab183a46dba9bc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14cb7fe640 1 -- 172.21.15.26:0/489422763 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8035020 con 0x7f14cc0c88e0 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14cb7fe640 1 -- 172.21.15.26:0/489422763 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b8031830 con 0x7f14cc0c88e0 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14cb7fe640 1 -- 172.21.15.26:0/489422763 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8031be0 con 0x7f14cc0c88e0 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14d2b65640 1 -- 172.21.15.26:0/489422763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 msgr2=0x7f14cc0c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14d2b65640 1 --2- 172.21.15.26:0/489422763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 0x7f14cc0c73f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14b80097a0 tx=0x7f14b802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14d2b65640 1 -- 172.21.15.26:0/489422763 shutdown_connections 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14d2b65640 1 --2- 172.21.15.26:0/489422763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 0x7f14cc0c73f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14d2b65640 1 -- 172.21.15.26:0/489422763 >> 172.21.15.26:0/489422763 conn(0x7f14cc0ac5f0 msgr2=0x7f14cc0aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14d2b65640 1 -- 172.21.15.26:0/489422763 shutdown_connections 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.646+0000 7f14d2b65640 1 -- 172.21.15.26:0/489422763 wait complete. 2024-03-06T07:22:42.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.647+0000 7f14d2b65640 1 Processor -- start 2024-03-06T07:22:42.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.647+0000 7f14d2b65640 1 -- start start 2024-03-06T07:22:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d2b65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14cc0c7930 0x7f14cc0c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d2b65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 0x7f14cc1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d2b65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14cc152870 0x7f14cc1ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d2b65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0c9470 con 0x7f14cc152870 2024-03-06T07:22:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d2b65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc06b8d0 con 0x7f14cc0c88e0 2024-03-06T07:22:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d2b65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0c95f0 con 0x7f14cc0c7930 2024-03-06T07:22:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d1102640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14cc152870 0x7f14cc1ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d1102640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14cc152870 0x7f14cc1ee3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34428/0 (socket says 172.21.15.26:34428) 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d1102640 1 -- 172.21.15.26:0/2083370883 learned_addr learned my addr 172.21.15.26:0/2083370883 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14cbfff640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 0x7f14cc1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.648+0000 7f14d0901640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14cc0c7930 0x7f14cc0c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d1102640 1 -- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14cc0c7930 msgr2=0x7f14cc0c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d1102640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14cc0c7930 0x7f14cc0c2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d1102640 1 -- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 msgr2=0x7f14cc1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d1102640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 0x7f14cc1558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d1102640 1 -- 172.21.15.26:0/2083370883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc1eed00 con 0x7f14cc152870 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d1102640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14cc152870 0x7f14cc1ee3c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f14c400ca50 tx=0x7f14c400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14c97fa640 1 -- 172.21.15.26:0/2083370883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4017070 con 0x7f14cc152870 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14c97fa640 1 -- 172.21.15.26:0/2083370883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c400ab50 con 0x7f14cc152870 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc1550b0 con 0x7f14cc152870 2024-03-06T07:22:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14c97fa640 1 -- 172.21.15.26:0/2083370883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4012400 con 0x7f14cc152870 2024-03-06T07:22:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.649+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc155df0 con 0x7f14cc152870 2024-03-06T07:22:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.650+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc069460 con 0x7f14cc152870 2024-03-06T07:22:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.650+0000 7f14c97fa640 1 -- 172.21.15.26:0/2083370883 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14c401a070 con 0x7f14cc152870 2024-03-06T07:22:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.651+0000 7f14c97fa640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14b006b4d0 0x7f14b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.651+0000 7f14c97fa640 1 -- 172.21.15.26:0/2083370883 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14c4089fc0 con 0x7f14cc152870 2024-03-06T07:22:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.651+0000 7f14d0901640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14b006b4d0 0x7f14b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.651+0000 7f14d0901640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14b006b4d0 0x7f14b006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000f60 tx=0x7f14c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.655+0000 7f14c97fa640 1 -- 172.21.15.26:0/2083370883 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14cc069460 con 0x7f14cc152870 2024-03-06T07:22:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:42.835+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f14cc0c64d0 con 0x7f14cc152870 2024-03-06T07:22:43.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.349+0000 7f14c97fa640 1 -- 172.21.15.26:0/2083370883 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v75) v1 ==== 140+0+0 (secure 0 0 0) 0x7f14cc0c64d0 con 0x7f14cc152870 2024-03-06T07:22:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_metadata' created 2024-03-06T07:22:43.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14b006b4d0 msgr2=0x7f14b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14b006b4d0 0x7f14b006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000f60 tx=0x7f14c0009040 comp rx=0 tx=0).stop 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14cc152870 msgr2=0x7f14cc1ee3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14cc152870 0x7f14cc1ee3c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f14c400ca50 tx=0x7f14c400cf10 comp rx=0 tx=0).stop 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 shutdown_connections 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14b006b4d0 0x7f14b006d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14cc152870 0x7f14cc1ee3c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc0c88e0 0x7f14cc1558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 --2- 172.21.15.26:0/2083370883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14cc0c7930 0x7f14cc0c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 >> 172.21.15.26:0/2083370883 conn(0x7f14cc0ac5f0 msgr2=0x7f14cc0aa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 shutdown_connections 2024-03-06T07:22:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.353+0000 7f14d2b65640 1 -- 172.21.15.26:0/2083370883 wait complete. 2024-03-06T07:22:43.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:22:43.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.477+0000 7fbb9c06d640 1 Processor -- start 2024-03-06T07:22:43.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.477+0000 7fbb9c06d640 1 -- start start 2024-03-06T07:22:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9c06d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbb940c5370 0x7fbb940c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9c06d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb94155530 0x7fbb941559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9c06d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb94152dd0 con 0x7fbb941529f0 2024-03-06T07:22:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9c06d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb94159710 con 0x7fbb940c5370 2024-03-06T07:22:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9c06d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940c68b0 con 0x7fbb94155530 2024-03-06T07:22:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb99e09640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb941529f0 0x7fbb940c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47262/0 (socket says 172.21.15.26:47262) 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb99e09640 1 -- 172.21.15.26:0/2571276016 learned_addr learned my addr 172.21.15.26:0/2571276016 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb99608640 1 --2- 172.21.15.26:0/2571276016 >> v2:172.21.15.117:3300/0 conn(0x7fbb940c5370 0x7fbb940c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9a60a640 1 --2- 172.21.15.26:0/2571276016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb94155530 0x7fbb941559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb98e07640 1 -- 172.21.15.26:0/2571276016 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002369071 0 0) 0x7fbb94152dd0 con 0x7fbb941529f0 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb98e07640 1 -- 172.21.15.26:0/2571276016 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb7c003520 con 0x7fbb941529f0 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9a60a640 1 -- 172.21.15.26:0/2571276016 >> v2:172.21.15.117:3300/0 conn(0x7fbb940c5370 msgr2=0x7fbb940c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9a60a640 1 --2- 172.21.15.26:0/2571276016 >> v2:172.21.15.117:3300/0 conn(0x7fbb940c5370 0x7fbb940c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9a60a640 1 -- 172.21.15.26:0/2571276016 >> v1:172.21.15.26:6789/0 conn(0x7fbb941529f0 legacy=0x7fbb940c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb9a60a640 1 -- 172.21.15.26:0/2571276016 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94158b00 con 0x7fbb94155530 2024-03-06T07:22:43.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.478+0000 7fbb99608640 1 --2- 172.21.15.26:0/2571276016 >> v2:172.21.15.117:3300/0 conn(0x7fbb940c5370 0x7fbb940c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.479+0000 7fbb9a60a640 1 --2- 172.21.15.26:0/2571276016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb94155530 0x7fbb941559c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009d90 tx=0x7fbb8402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed05d804817a028 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.479+0000 7fbb98e07640 1 -- 172.21.15.26:0/2571276016 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84039070 con 0x7fbb94155530 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.479+0000 7fbb98e07640 1 -- 172.21.15.26:0/2571276016 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8402cbc0 con 0x7fbb94155530 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.479+0000 7fbb98e07640 1 -- 172.21.15.26:0/2571276016 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84034440 con 0x7fbb94155530 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2571276016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb94155530 msgr2=0x7fbb941559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2571276016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb94155530 0x7fbb941559c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009d90 tx=0x7fbb8402be10 comp rx=0 tx=0).stop 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2571276016 shutdown_connections 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2571276016 >> v2:172.21.15.117:3300/0 conn(0x7fbb940c5370 0x7fbb940c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2571276016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb94155530 0x7fbb941559c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2571276016 >> 172.21.15.26:0/2571276016 conn(0x7fbb940ac580 msgr2=0x7fbb940ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2571276016 shutdown_connections 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.480+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2571276016 wait complete. 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.481+0000 7fbb9c06d640 1 Processor -- start 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.481+0000 7fbb9c06d640 1 -- start start 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9c06d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb940c5370 0x7fbb940ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9c06d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb941529f0 0x7fbb941ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9c06d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb94155530 0x7fbb941eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9c06d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941590c0 con 0x7fbb94155530 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9c06d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940c9000 con 0x7fbb941529f0 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9c06d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941593a0 con 0x7fbb940c5370 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb94155530 0x7fbb941eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb94155530 0x7fbb941eaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34438/0 (socket says 172.21.15.26:34438) 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 -- 172.21.15.26:0/2551128074 learned_addr learned my addr 172.21.15.26:0/2551128074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb99608640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb941529f0 0x7fbb941ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb99e09640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb940c5370 0x7fbb940ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 -- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb940c5370 msgr2=0x7fbb940ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb940c5370 0x7fbb940ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 -- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb941529f0 msgr2=0x7fbb941ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb941529f0 0x7fbb941ea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 -- 172.21.15.26:0/2551128074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94149e40 con 0x7fbb94155530 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb99608640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb941529f0 0x7fbb941ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.482+0000 7fbb9a60a640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb94155530 0x7fbb941eaf20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009d90 tx=0x7fbb84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.483+0000 7fbb927fc640 1 -- 172.21.15.26:0/2551128074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84039050 con 0x7fbb94155530 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.483+0000 7fbb927fc640 1 -- 172.21.15.26:0/2551128074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb84045070 con 0x7fbb94155530 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.483+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb94149030 con 0x7fbb94155530 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.483+0000 7fbb927fc640 1 -- 172.21.15.26:0/2551128074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84009450 con 0x7fbb94155530 2024-03-06T07:22:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.483+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb940abb60 con 0x7fbb94155530 2024-03-06T07:22:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.483+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb94148d90 con 0x7fbb94155530 2024-03-06T07:22:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.484+0000 7fbb927fc640 1 -- 172.21.15.26:0/2551128074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb8403e850 con 0x7fbb94155530 2024-03-06T07:22:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.485+0000 7fbb927fc640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8006b500 0x7fbb8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.485+0000 7fbb927fc640 1 -- 172.21.15.26:0/2551128074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6472+0+0 (secure 0 0 0) 0x7fbb840ac910 con 0x7fbb94155530 2024-03-06T07:22:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.485+0000 7fbb99e09640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8006b500 0x7fbb8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.485+0000 7fbb99e09640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8006b500 0x7fbb8006d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005880 tx=0x7fbb8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.489+0000 7fbb927fc640 1 -- 172.21.15.26:0/2551128074 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb8404e020 con 0x7fbb94155530 2024-03-06T07:22:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:43.670+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbb941ee860 con 0x7fbb94155530 2024-03-06T07:22:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.354+0000 7fbb927fc640 1 -- 172.21.15.26:0/2551128074 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v76) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbb8407aed0 con 0x7fbb94155530 2024-03-06T07:22:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_data' created 2024-03-06T07:22:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.357+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8006b500 msgr2=0x7fbb8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8006b500 0x7fbb8006d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005880 tx=0x7fbb8c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb94155530 msgr2=0x7fbb941eaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb94155530 0x7fbb941eaf20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009d90 tx=0x7fbb84004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 shutdown_connections 2024-03-06T07:22:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8006b500 0x7fbb8006d9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb94155530 0x7fbb941eaf20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb941529f0 0x7fbb941ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 --2- 172.21.15.26:0/2551128074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb940c5370 0x7fbb940ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.358+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 >> 172.21.15.26:0/2551128074 conn(0x7fbb940ac580 msgr2=0x7fbb94150b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.359+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 shutdown_connections 2024-03-06T07:22:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.360+0000 7fbb9c06d640 1 -- 172.21.15.26:0/2551128074 wait complete. 2024-03-06T07:22:44.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-06T07:22:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.495+0000 7f8392532640 1 Processor -- start 2024-03-06T07:22:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.495+0000 7f8392532640 1 -- start start 2024-03-06T07:22:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8392532640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f838c0c3280 0x7f838c0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8392532640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c1551e0 0x7f838c155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8392532640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c158800 con 0x7f838c06d9f0 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8392532640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c158ae0 con 0x7f838c0c3280 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8392532640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c158dc0 con 0x7f838c1551e0 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838b7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f838c0c3280 0x7f838c0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8390acf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c1551e0 0x7f838c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838b7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f838c0c3280 0x7f838c0c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:54916/0 (socket says 172.21.15.26:54916) 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838b7fe640 1 -- 172.21.15.26:0/976744079 learned_addr learned my addr 172.21.15.26:0/976744079 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8390acf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c1551e0 0x7f838c155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:44780/0 (socket says 172.21.15.26:44780) 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838bfff640 1 --1- 172.21.15.26:0/976744079 >> v1:172.21.15.117:6789/0 conn(0x7f838c06d9f0 0x7f838c0a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:40206/0 (socket says 172.21.15.26:40206) 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838b7fe640 1 -- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c1551e0 msgr2=0x7f838c155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838b7fe640 1 --2- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c1551e0 0x7f838c155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838b7fe640 1 -- 172.21.15.26:0/976744079 >> v1:172.21.15.117:6789/0 conn(0x7f838c06d9f0 legacy=0x7f838c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f838b7fe640 1 -- 172.21.15.26:0/976744079 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c1590a0 con 0x7f838c0c3280 2024-03-06T07:22:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.496+0000 7f8390acf640 1 --2- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c1551e0 0x7f838c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.497+0000 7f838b7fe640 1 --2- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c0c3710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f837c009870 tx=0x7f837c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5724e298109ea7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.497+0000 7f838affd640 1 -- 172.21.15.26:0/976744079 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c035020 con 0x7f838c0c3280 2024-03-06T07:22:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.497+0000 7f838affd640 1 -- 172.21.15.26:0/976744079 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c031640 con 0x7f838c0c3280 2024-03-06T07:22:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.497+0000 7f838affd640 1 -- 172.21.15.26:0/976744079 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c02d040 con 0x7f838c0c3280 2024-03-06T07:22:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 -- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 msgr2=0x7f838c0c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 --2- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c0c3710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f837c009870 tx=0x7f837c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 -- 172.21.15.26:0/976744079 shutdown_connections 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 --2- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c1551e0 0x7f838c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 --2- 172.21.15.26:0/976744079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c0c3710 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 -- 172.21.15.26:0/976744079 >> 172.21.15.26:0/976744079 conn(0x7f838c0aa630 msgr2=0x7f838c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 -- 172.21.15.26:0/976744079 shutdown_connections 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.498+0000 7f8392532640 1 -- 172.21.15.26:0/976744079 wait complete. 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.499+0000 7f8392532640 1 Processor -- start 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.499+0000 7f8392532640 1 -- start start 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8392532640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c06d9f0 0x7f838c1edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8392532640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c14e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8392532640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838c1551e0 0x7f838c14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8392532640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c06df50 con 0x7f838c1551e0 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8392532640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c06ddd0 con 0x7f838c0c3280 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8392532640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0c6910 con 0x7f838c06d9f0 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838c1551e0 0x7f838c14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f838bfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c06d9f0 0x7f838c1edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838c1551e0 0x7f838c14eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34444/0 (socket says 172.21.15.26:34444) 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f838b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c14e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 -- 172.21.15.26:0/2010928341 learned_addr learned my addr 172.21.15.26:0/2010928341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 -- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c06d9f0 msgr2=0x7f838c1edec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c06d9f0 0x7f838c1edec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 -- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 msgr2=0x7f838c14e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c14e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f8390acf640 1 -- 172.21.15.26:0/2010928341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c1ee8b0 con 0x7f838c1551e0 2024-03-06T07:22:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f838bfff640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c06d9f0 0x7f838c1edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.500+0000 7f838b7fe640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c14e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.501+0000 7f8390acf640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838c1551e0 0x7f838c14eaf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f838400efc0 tx=0x7f838400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.501+0000 7f8388ff9640 1 -- 172.21.15.26:0/2010928341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384018030 con 0x7f838c1551e0 2024-03-06T07:22:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.501+0000 7f8388ff9640 1 -- 172.21.15.26:0/2010928341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838400cd30 con 0x7f838c1551e0 2024-03-06T07:22:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.501+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c14f330 con 0x7f838c1551e0 2024-03-06T07:22:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.501+0000 7f8388ff9640 1 -- 172.21.15.26:0/2010928341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384003bb0 con 0x7f838c1551e0 2024-03-06T07:22:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.501+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c0703c0 con 0x7f838c1551e0 2024-03-06T07:22:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.502+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c04f330 con 0x7f838c1551e0 2024-03-06T07:22:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.502+0000 7f8388ff9640 1 -- 172.21.15.26:0/2010928341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f838401b070 con 0x7f838c1551e0 2024-03-06T07:22:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.503+0000 7f8388ff9640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.503+0000 7f8388ff9640 1 -- 172.21.15.26:0/2010928341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f838408ae70 con 0x7f838c1551e0 2024-03-06T07:22:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.503+0000 7f838bfff640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.503+0000 7f838bfff640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8380007fd0 tx=0x7f8380007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.508+0000 7f8388ff9640 1 -- 172.21.15.26:0/2010928341 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8384010040 con 0x7f838c1551e0 2024-03-06T07:22:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:44.726+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f838c073240 con 0x7f838c1551e0 2024-03-06T07:22:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.368+0000 7f8388ff9640 1 -- 172.21.15.26:0/2010928341 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 13 and data pool 14 v56) v1 ==== 177+0+0 (secure 0 0 0) 0x7f83840592b0 con 0x7f838c1551e0 2024-03-06T07:22:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 13 and data pool 14 2024-03-06T07:22:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.378+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f837806b4d0 msgr2=0x7f837806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.378+0000 7f8392532640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8380007fd0 tx=0x7f8380007e80 comp rx=0 tx=0).stop 2024-03-06T07:22:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.378+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838c1551e0 msgr2=0x7f838c14eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.378+0000 7f8392532640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838c1551e0 0x7f838c14eaf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f838400efc0 tx=0x7f838400c490 comp rx=0 tx=0).stop 2024-03-06T07:22:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.379+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 shutdown_connections 2024-03-06T07:22:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.379+0000 7f8392532640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.379+0000 7f8392532640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838c1551e0 0x7f838c14eaf0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.379+0000 7f8392532640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f838c0c3280 0x7f838c14e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.379+0000 7f8392532640 1 --2- 172.21.15.26:0/2010928341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f838c06d9f0 0x7f838c1edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.379+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 >> 172.21.15.26:0/2010928341 conn(0x7f838c0aa630 msgr2=0x7f838c0c7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.380+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 shutdown_connections 2024-03-06T07:22:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.380+0000 7f8392532640 1 -- 172.21.15.26:0/2010928341 wait complete. 2024-03-06T07:22:45.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:45.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.504+0000 7f296f083640 1 Processor -- start 2024-03-06T07:22:45.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.504+0000 7f296f083640 1 -- start start 2024-03-06T07:22:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296f083640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29680c6000 0x7f2968156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296f083640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2968153a60 con 0x7f29680c1320 2024-03-06T07:22:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296f083640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2968153d40 con 0x7f2968150580 2024-03-06T07:22:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296f083640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296806c2c0 con 0x7f29680c6000 2024-03-06T07:22:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f2967fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f29680c1320 0x7f29681566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47270/0 (socket says 172.21.15.26:47270) 2024-03-06T07:22:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f2967fff640 1 -- 172.21.15.26:0/1041728231 learned_addr learned my addr 172.21.15.26:0/1041728231 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:45.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296d620640 1 --2- 172.21.15.26:0/1041728231 >> v2:172.21.15.117:3300/0 conn(0x7f29680c6000 0x7f2968156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f29677fe640 1 -- 172.21.15.26:0/1041728231 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095095766 0 0) 0x7f2968153a60 con 0x7f29680c1320 2024-03-06T07:22:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f29677fe640 1 -- 172.21.15.26:0/1041728231 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2954003520 con 0x7f29680c1320 2024-03-06T07:22:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296d620640 1 -- 172.21.15.26:0/1041728231 >> v1:172.21.15.117:6789/0 conn(0x7f2968150580 legacy=0x7f296806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f29677fe640 1 -- 172.21.15.26:0/1041728231 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172546956 0 0) 0x7f2968153d40 con 0x7f2968150580 2024-03-06T07:22:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296d620640 1 -- 172.21.15.26:0/1041728231 >> v1:172.21.15.26:6789/0 conn(0x7f29680c1320 legacy=0x7f29681566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.505+0000 7f296d620640 1 -- 172.21.15.26:0/1041728231 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296806c5a0 con 0x7f29680c6000 2024-03-06T07:22:45.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.506+0000 7f296d620640 1 --2- 172.21.15.26:0/1041728231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29680c6000 0x7f2968156dc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f29580097a0 tx=0x7f295802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b47f175335c50b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.506+0000 7f29677fe640 1 -- 172.21.15.26:0/1041728231 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958035020 con 0x7f29680c6000 2024-03-06T07:22:45.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.506+0000 7f29677fe640 1 -- 172.21.15.26:0/1041728231 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2958031830 con 0x7f29680c6000 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.506+0000 7f29677fe640 1 -- 172.21.15.26:0/1041728231 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958031be0 con 0x7f29680c6000 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.507+0000 7f296f083640 1 -- 172.21.15.26:0/1041728231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29680c6000 msgr2=0x7f2968156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.507+0000 7f296f083640 1 --2- 172.21.15.26:0/1041728231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29680c6000 0x7f2968156dc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f29580097a0 tx=0x7f295802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.507+0000 7f296f083640 1 -- 172.21.15.26:0/1041728231 shutdown_connections 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.507+0000 7f296f083640 1 --2- 172.21.15.26:0/1041728231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29680c6000 0x7f2968156dc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.507+0000 7f296f083640 1 -- 172.21.15.26:0/1041728231 >> 172.21.15.26:0/1041728231 conn(0x7f29680c6710 msgr2=0x7f29680c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.507+0000 7f296f083640 1 -- 172.21.15.26:0/1041728231 shutdown_connections 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.507+0000 7f296f083640 1 -- 172.21.15.26:0/1041728231 wait complete. 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.508+0000 7f296f083640 1 Processor -- start 2024-03-06T07:22:45.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.508+0000 7f296f083640 1 -- start start 2024-03-06T07:22:45.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296f083640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29680c1320 0x7f29681516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296f083640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680c6000 0x7f29681e8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296f083640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968150580 0x7f2968070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296f083640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296806e420 con 0x7f29680c1320 2024-03-06T07:22:45.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296f083640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2968150960 con 0x7f2968150580 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296f083640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296806e5a0 con 0x7f29680c6000 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29680c1320 0x7f29681516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29680c1320 0x7f29681516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34450/0 (socket says 172.21.15.26:34450) 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 -- 172.21.15.26:0/2819127333 learned_addr learned my addr 172.21.15.26:0/2819127333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f2967fff640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680c6000 0x7f29681e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296d620640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968150580 0x7f2968070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 -- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680c6000 msgr2=0x7f29681e8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680c6000 0x7f29681e8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 -- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968150580 msgr2=0x7f2968070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968150580 0x7f2968070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296ce1f640 1 -- 172.21.15.26:0/2819127333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29681e5f40 con 0x7f29680c1320 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f2967fff640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680c6000 0x7f29681e8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.509+0000 7f296d620640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968150580 0x7f2968070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:45.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.510+0000 7f296ce1f640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29680c1320 0x7f29681516a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f296000c7f0 tx=0x7f296000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.510+0000 7f29657fa640 1 -- 172.21.15.26:0/2819127333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960017070 con 0x7f29680c1320 2024-03-06T07:22:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.510+0000 7f29657fa640 1 -- 172.21.15.26:0/2819127333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2960003da0 con 0x7f29680c1320 2024-03-06T07:22:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.510+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2968152780 con 0x7f29680c1320 2024-03-06T07:22:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.510+0000 7f29657fa640 1 -- 172.21.15.26:0/2819127333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960005290 con 0x7f29680c1320 2024-03-06T07:22:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.510+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2968152a40 con 0x7f29680c1320 2024-03-06T07:22:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.511+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296804f330 con 0x7f29680c1320 2024-03-06T07:22:45.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.511+0000 7f29657fa640 1 -- 172.21.15.26:0/2819127333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f296001a070 con 0x7f29680c1320 2024-03-06T07:22:45.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.512+0000 7f29657fa640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f294806b4d0 0x7f294806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.512+0000 7f29657fa640 1 -- 172.21.15.26:0/2819127333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f2960089ca0 con 0x7f29680c1320 2024-03-06T07:22:45.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.512+0000 7f2967fff640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f294806b4d0 0x7f294806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.512+0000 7f2967fff640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f294806b4d0 0x7f294806d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f29680722d0 tx=0x7f2950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.516+0000 7f29657fa640 1 -- 172.21.15.26:0/2819127333 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2960058000 con 0x7f29680c1320 2024-03-06T07:22:45.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.698+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29680c5de0 con 0x7f29680c1320 2024-03-06T07:22:45.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.699+0000 7f29657fa640 1 -- 172.21.15.26:0/2819127333 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+20694 (secure 0 0 0) 0x7f296005c0a0 con 0x7f29680c1320 2024-03-06T07:22:45.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:45.701 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:44.728473+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f294806b4d0 msgr2=0x7f294806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f294806b4d0 0x7f294806d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f29680722d0 tx=0x7f2950009040 comp rx=0 tx=0).stop 2024-03-06T07:22:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29680c1320 msgr2=0x7f29681516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29680c1320 0x7f29681516a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f296000c7f0 tx=0x7f296000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 shutdown_connections 2024-03-06T07:22:45.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f294806b4d0 0x7f294806d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968150580 0x7f2968070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680c6000 0x7f29681e8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 --2- 172.21.15.26:0/2819127333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29680c1320 0x7f29681516a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 >> 172.21.15.26:0/2819127333 conn(0x7f29680c6710 msgr2=0x7f29680c4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 shutdown_connections 2024-03-06T07:22:45.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.701+0000 7f296f083640 1 -- 172.21.15.26:0/2819127333 wait complete. 2024-03-06T07:22:45.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:45.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.814+0000 7ff479335640 1 Processor -- start 2024-03-06T07:22:45.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.814+0000 7ff479335640 1 -- start start 2024-03-06T07:22:45.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff479335640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474152520 0x7ff47406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff479335640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff474155c80 0x7ff4740c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff479335640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4740720a0 0x7ff47406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff479335640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4740c1dc0 con 0x7ff4740720a0 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff479335640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4740c1ae0 con 0x7ff474155c80 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff479335640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47406e450 con 0x7ff474152520 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4740720a0 0x7ff47406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4740720a0 0x7ff47406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34462/0 (socket says 172.21.15.26:34462) 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 -- 172.21.15.26:0/4070656998 learned_addr learned my addr 172.21.15.26:0/4070656998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4727fc640 1 --2- 172.21.15.26:0/4070656998 >> v2:172.21.15.117:3300/0 conn(0x7ff474155c80 0x7ff4740c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff472ffd640 1 --2- 172.21.15.26:0/4070656998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474152520 0x7ff47406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 -- 172.21.15.26:0/4070656998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474152520 msgr2=0x7ff47406ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 --2- 172.21.15.26:0/4070656998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474152520 0x7ff47406ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 -- 172.21.15.26:0/4070656998 >> v2:172.21.15.117:3300/0 conn(0x7ff474155c80 msgr2=0x7ff4740c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 --2- 172.21.15.26:0/4070656998 >> v2:172.21.15.117:3300/0 conn(0x7ff474155c80 0x7ff4740c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4737fe640 1 -- 172.21.15.26:0/4070656998 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47406e730 con 0x7ff4740720a0 2024-03-06T07:22:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.815+0000 7ff4727fc640 1 --2- 172.21.15.26:0/4070656998 >> v2:172.21.15.117:3300/0 conn(0x7ff474155c80 0x7ff4740c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.816+0000 7ff4737fe640 1 --2- 172.21.15.26:0/4070656998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff47406afc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff4740c5230 tx=0x7ff46402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e6ec3677a3a376 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.816+0000 7ff471ffb640 1 -- 172.21.15.26:0/4070656998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46402ca70 con 0x7ff4740720a0 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.816+0000 7ff471ffb640 1 -- 172.21.15.26:0/4070656998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46402cc10 con 0x7ff4740720a0 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.816+0000 7ff471ffb640 1 -- 172.21.15.26:0/4070656998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640343f0 con 0x7ff4740720a0 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 -- 172.21.15.26:0/4070656998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 msgr2=0x7ff47406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 --2- 172.21.15.26:0/4070656998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff47406afc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff4740c5230 tx=0x7ff46402bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 -- 172.21.15.26:0/4070656998 shutdown_connections 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 --2- 172.21.15.26:0/4070656998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff47406afc0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 --2- 172.21.15.26:0/4070656998 >> v2:172.21.15.117:3300/0 conn(0x7ff474155c80 0x7ff4740c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 --2- 172.21.15.26:0/4070656998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474152520 0x7ff47406ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 -- 172.21.15.26:0/4070656998 >> 172.21.15.26:0/4070656998 conn(0x7ff4740c76b0 msgr2=0x7ff4740c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 -- 172.21.15.26:0/4070656998 shutdown_connections 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.817+0000 7ff479335640 1 -- 172.21.15.26:0/4070656998 wait complete. 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.818+0000 7ff479335640 1 Processor -- start 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.818+0000 7ff479335640 1 -- start start 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff479335640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff474153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff479335640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff474152520 0x7ff47414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff479335640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474155c80 0x7ff47414fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff479335640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474152900 con 0x7ff4740720a0 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff479335640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c17f0 con 0x7ff474152520 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff479335640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff474152a80 con 0x7ff474155c80 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff474153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff474153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34478/0 (socket says 172.21.15.26:34478) 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 -- 172.21.15.26:0/2116613134 learned_addr learned my addr 172.21.15.26:0/2116613134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff4737fe640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474155c80 0x7ff47414fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff4727fc640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff474152520 0x7ff47414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 -- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474155c80 msgr2=0x7ff47414fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474155c80 0x7ff47414fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 -- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff474152520 msgr2=0x7ff47414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff474152520 0x7ff47414f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 -- 172.21.15.26:0/2116613134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4741e1e80 con 0x7ff4740720a0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.819+0000 7ff472ffd640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff474153ed0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00e8a0 tx=0x7ff46c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.820+0000 7ff46b7fe640 1 -- 172.21.15.26:0/2116613134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c017070 con 0x7ff4740720a0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.820+0000 7ff46b7fe640 1 -- 172.21.15.26:0/2116613134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00c940 con 0x7ff4740720a0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.820+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4740c5ce0 con 0x7ff4740720a0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.820+0000 7ff46b7fe640 1 -- 172.21.15.26:0/2116613134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00cd30 con 0x7ff4740720a0 2024-03-06T07:22:45.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.820+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4741e2120 con 0x7ff4740720a0 2024-03-06T07:22:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.821+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47404f330 con 0x7ff4740720a0 2024-03-06T07:22:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.821+0000 7ff46b7fe640 1 -- 172.21.15.26:0/2116613134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff46c01a070 con 0x7ff4740720a0 2024-03-06T07:22:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.821+0000 7ff46b7fe640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.822+0000 7ff46b7fe640 1 -- 172.21.15.26:0/2116613134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6957+0+0 (secure 0 0 0) 0x7ff46c08a430 con 0x7ff4740720a0 2024-03-06T07:22:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.822+0000 7ff4727fc640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.822+0000 7ff4727fc640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45406b4d0 0x7ff45406d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009560 tx=0x7ff45c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:45.826+0000 7ff46b7fe640 1 -- 172.21.15.26:0/2116613134 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff46c058810 con 0x7ff4740720a0 2024-03-06T07:22:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.007+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4740c6d30 con 0x7ff4740720a0 2024-03-06T07:22:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.008+0000 7ff46b7fe640 1 -- 172.21.15.26:0/2116613134 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+20694 (secure 0 0 0) 0x7ff46c05c8b0 con 0x7ff4740720a0 2024-03-06T07:22:46.010 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:46.010 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:44.728473+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45406b4d0 msgr2=0x7ff45406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45406b4d0 0x7ff45406d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009560 tx=0x7ff45c009190 comp rx=0 tx=0).stop 2024-03-06T07:22:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 msgr2=0x7ff474153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff474153ed0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00e8a0 tx=0x7ff46c00ed60 comp rx=0 tx=0).stop 2024-03-06T07:22:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 shutdown_connections 2024-03-06T07:22:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff474155c80 0x7ff47414fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff474152520 0x7ff47414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 --2- 172.21.15.26:0/2116613134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4740720a0 0x7ff474153ed0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 >> 172.21.15.26:0/2116613134 conn(0x7ff4740c76b0 msgr2=0x7ff47406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 shutdown_connections 2024-03-06T07:22:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.010+0000 7ff479335640 1 -- 172.21.15.26:0/2116613134 wait complete. 2024-03-06T07:22:46.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-06T07:22:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.126+0000 7f7bc541c640 1 Processor -- start 2024-03-06T07:22:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.126+0000 7f7bc541c640 1 -- start start 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bc541c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7bc006c900 0x7f7bc00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bc541c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00ab170 0x7f7bc00a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bc541c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc00c3ef0 con 0x7f7bc00c5990 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bc541c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc01556d0 con 0x7f7bc006c900 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bc541c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc01559b0 con 0x7f7bc00ab170 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bb67fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7bc006c900 0x7f7bc00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bb67fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7bc006c900 0x7f7bc00c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34484/0 (socket says 172.21.15.26:34484) 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bbeffd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f7bc00c5990 0x7f7bc006c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:40228/0 (socket says 172.21.15.26:40228) 2024-03-06T07:22:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bbf7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00ab170 0x7f7bc00a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bb67fc640 1 -- 172.21.15.26:0/1225351911 learned_addr learned my addr 172.21.15.26:0/1225351911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bb67fc640 1 -- 172.21.15.26:0/1225351911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00ab170 msgr2=0x7f7bc00a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bb67fc640 1 --2- 172.21.15.26:0/1225351911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00ab170 0x7f7bc00a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bb67fc640 1 -- 172.21.15.26:0/1225351911 >> v1:172.21.15.117:6789/0 conn(0x7f7bc00c5990 legacy=0x7f7bc006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bbe7fc640 1 -- 172.21.15.26:0/1225351911 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632538228 0 0) 0x7f7bc00c3ef0 con 0x7f7bc00c5990 2024-03-06T07:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.127+0000 7f7bb67fc640 1 -- 172.21.15.26:0/1225351911 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc0155c90 con 0x7f7bc006c900 2024-03-06T07:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.128+0000 7f7bb67fc640 1 --2- 172.21.15.26:0/1225351911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c0ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009870 tx=0x7f7ba802db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f397d9ad1e5f7ab server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.128+0000 7f7bbe7fc640 1 -- 172.21.15.26:0/1225351911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba803b070 con 0x7f7bc006c900 2024-03-06T07:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.128+0000 7f7bbe7fc640 1 -- 172.21.15.26:0/1225351911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba802e940 con 0x7f7bc006c900 2024-03-06T07:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.128+0000 7f7bbe7fc640 1 -- 172.21.15.26:0/1225351911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba802ed10 con 0x7f7bc006c900 2024-03-06T07:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.128+0000 7f7bc541c640 1 -- 172.21.15.26:0/1225351911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 msgr2=0x7f7bc00c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.128+0000 7f7bc541c640 1 --2- 172.21.15.26:0/1225351911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c0ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009870 tx=0x7f7ba802db90 comp rx=0 tx=0).stop 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.129+0000 7f7bc541c640 1 -- 172.21.15.26:0/1225351911 shutdown_connections 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.129+0000 7f7bc541c640 1 --2- 172.21.15.26:0/1225351911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00ab170 0x7f7bc00a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.129+0000 7f7bc541c640 1 --2- 172.21.15.26:0/1225351911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c0ab0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.129+0000 7f7bc541c640 1 -- 172.21.15.26:0/1225351911 >> 172.21.15.26:0/1225351911 conn(0x7f7bc00c6710 msgr2=0x7f7bc00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.129+0000 7f7bc541c640 1 -- 172.21.15.26:0/1225351911 shutdown_connections 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.129+0000 7f7bc541c640 1 -- 172.21.15.26:0/1225351911 wait complete. 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 Processor -- start 2024-03-06T07:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 -- start start 2024-03-06T07:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc00ab170 0x7f7bc00bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00c5990 0x7f7bc01ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0156610 con 0x7f7bc006c900 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0156490 con 0x7f7bc00ab170 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.130+0000 7f7bc541c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc00c3af0 con 0x7f7bc00c5990 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34500/0 (socket says 172.21.15.26:34500) 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 -- 172.21.15.26:0/433353906 learned_addr learned my addr 172.21.15.26:0/433353906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbf7fe640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00c5990 0x7f7bc01ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bb67fc640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc00ab170 0x7f7bc00bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 -- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00c5990 msgr2=0x7f7bc01ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00c5990 0x7f7bc01ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 -- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc00ab170 msgr2=0x7f7bc00bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc00ab170 0x7f7bc00bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 -- 172.21.15.26:0/433353906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00c33e0 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bbeffd640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c1960 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c430 tx=0x7f7bb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bb7fff640 1 -- 172.21.15.26:0/433353906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00074e0 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bb7fff640 1 -- 172.21.15.26:0/433353906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb00047e0 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc00c36c0 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bb7fff640 1 -- 172.21.15.26:0/433353906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0004bd0 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.131+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc0155120 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.132+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc0075300 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.133+0000 7f7bb7fff640 1 -- 172.21.15.26:0/433353906 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bb0011070 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.133+0000 7f7bb7fff640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ba006b500 0x7f7ba006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.133+0000 7f7bb7fff640 1 -- 172.21.15.26:0/433353906 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f7bb008a160 con 0x7f7bc006c900 2024-03-06T07:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.133+0000 7f7bb67fc640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ba006b500 0x7f7ba006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.133+0000 7f7bb67fc640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ba006b500 0x7f7ba006d9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8038000 tx=0x7f7ba8032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.137+0000 7f7bb7fff640 1 -- 172.21.15.26:0/433353906 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7bc0075300 con 0x7f7bc006c900 2024-03-06T07:22:46.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.355+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7bc01ee070 con 0x7f7bc006c900 2024-03-06T07:22:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.369+0000 7f7bb7fff640 1 -- 172.21.15.26:0/433353906 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 119+0+0 (secure 0 0 0) 0x7f7bb0058540 con 0x7f7bc006c900 2024-03-06T07:22:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ba006b500 msgr2=0x7f7ba006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ba006b500 0x7f7ba006d9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8038000 tx=0x7f7ba8032000 comp rx=0 tx=0).stop 2024-03-06T07:22:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 msgr2=0x7f7bc00c1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c1960 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c430 tx=0x7f7bb000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 shutdown_connections 2024-03-06T07:22:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ba006b500 0x7f7ba006d9c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc00c5990 0x7f7bc01ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc00ab170 0x7f7bc00bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 --2- 172.21.15.26:0/433353906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc006c900 0x7f7bc00c1960 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.371+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 >> 172.21.15.26:0/433353906 conn(0x7f7bc00c6710 msgr2=0x7f7bc006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.372+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 shutdown_connections 2024-03-06T07:22:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.372+0000 7f7bc541c640 1 -- 172.21.15.26:0/433353906 wait complete. 2024-03-06T07:22:46.385 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:22:46.385 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-06T07:22:46.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.491+0000 7f6169f7a640 1 Processor -- start 2024-03-06T07:22:46.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.491+0000 7f6169f7a640 1 -- start start 2024-03-06T07:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6169f7a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6164152770 0x7f61640c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6169f7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f61640c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6169f7a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6164155ed0 con 0x7f61640c12a0 2024-03-06T07:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6169f7a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640c6a00 con 0x7f6164152770 2024-03-06T07:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6169f7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c6c30 con 0x7f61640c7710 2024-03-06T07:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6162ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61640c12a0 0x7f616406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47274/0 (socket says 172.21.15.26:47274) 2024-03-06T07:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6162ffd640 1 -- 172.21.15.26:0/1177537013 learned_addr learned my addr 172.21.15.26:0/1177537013 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f61637fe640 1 --2- 172.21.15.26:0/1177537013 >> v2:172.21.15.117:3300/0 conn(0x7f6164152770 0x7f61640c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.492+0000 7f6163fff640 1 --2- 172.21.15.26:0/1177537013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f61640c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.493+0000 7f61627fc640 1 -- 172.21.15.26:0/1177537013 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467630752 0 0) 0x7f6164155ed0 con 0x7f61640c12a0 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.493+0000 7f61627fc640 1 -- 172.21.15.26:0/1177537013 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614c003520 con 0x7f61640c12a0 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.493+0000 7f6163fff640 1 -- 172.21.15.26:0/1177537013 >> v2:172.21.15.117:3300/0 conn(0x7f6164152770 msgr2=0x7f61640c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.493+0000 7f6163fff640 1 --2- 172.21.15.26:0/1177537013 >> v2:172.21.15.117:3300/0 conn(0x7f6164152770 0x7f61640c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.493+0000 7f6163fff640 1 -- 172.21.15.26:0/1177537013 >> v1:172.21.15.26:6789/0 conn(0x7f61640c12a0 legacy=0x7f616406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.493+0000 7f6163fff640 1 -- 172.21.15.26:0/1177537013 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640c6f10 con 0x7f61640c7710 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.493+0000 7f61637fe640 1 --2- 172.21.15.26:0/1177537013 >> v2:172.21.15.117:3300/0 conn(0x7f6164152770 0x7f61640c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6163fff640 1 --2- 172.21.15.26:0/1177537013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f61640c7b80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f615c009d90 tx=0x7f615c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d0ae297771b72ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f61627fc640 1 -- 172.21.15.26:0/1177537013 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c039070 con 0x7f61640c7710 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f61627fc640 1 -- 172.21.15.26:0/1177537013 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c02cc50 con 0x7f61640c7710 2024-03-06T07:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f61627fc640 1 -- 172.21.15.26:0/1177537013 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c034440 con 0x7f61640c7710 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 -- 172.21.15.26:0/1177537013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 msgr2=0x7f61640c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1177537013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f61640c7b80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f615c009d90 tx=0x7f615c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 -- 172.21.15.26:0/1177537013 shutdown_connections 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1177537013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f61640c7b80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1177537013 >> v2:172.21.15.117:3300/0 conn(0x7f6164152770 0x7f61640c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 -- 172.21.15.26:0/1177537013 >> 172.21.15.26:0/1177537013 conn(0x7f61640a94d0 msgr2=0x7f61640a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 -- 172.21.15.26:0/1177537013 shutdown_connections 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.494+0000 7f6169f7a640 1 -- 172.21.15.26:0/1177537013 wait complete. 2024-03-06T07:22:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.495+0000 7f6169f7a640 1 Processor -- start 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f6169f7a640 1 -- start start 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f6169f7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640c12a0 0x7f6164153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f6169f7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f6164141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f6169f7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164152770 0x7f61641ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f6169f7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164155ba0 con 0x7f61640c12a0 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f6169f7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164152b50 con 0x7f6164152770 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f6169f7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616406b6d0 con 0x7f61640c7710 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.496+0000 7f61637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640c12a0 0x7f6164153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640c12a0 0x7f6164153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34512/0 (socket says 172.21.15.26:34512) 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 -- 172.21.15.26:0/1273954713 learned_addr learned my addr 172.21.15.26:0/1273954713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f6163fff640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164152770 0x7f61641ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f6162ffd640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f6164141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 -- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 msgr2=0x7f6164141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f6164141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 -- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164152770 msgr2=0x7f61641ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164152770 0x7f61641ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 -- 172.21.15.26:0/1273954713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616414c500 con 0x7f61640c12a0 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f61637fe640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640c12a0 0x7f6164153000 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f615400e960 tx=0x7f615400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f614bfff640 1 -- 172.21.15.26:0/1273954713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154017070 con 0x7f61640c12a0 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f614bfff640 1 -- 172.21.15.26:0/1273954713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615400ca00 con 0x7f61640c12a0 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f614bfff640 1 -- 172.21.15.26:0/1273954713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615400cdd0 con 0x7f61640c12a0 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616414c220 con 0x7f61640c12a0 2024-03-06T07:22:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.497+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61640aa9d0 con 0x7f61640c12a0 2024-03-06T07:22:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.498+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616404f330 con 0x7f61640c12a0 2024-03-06T07:22:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.499+0000 7f614bfff640 1 -- 172.21.15.26:0/1273954713 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f615401a070 con 0x7f61640c12a0 2024-03-06T07:22:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.499+0000 7f614bfff640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f615006b4d0 0x7f615006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.499+0000 7f614bfff640 1 -- 172.21.15.26:0/1273954713 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f615408a860 con 0x7f61640c12a0 2024-03-06T07:22:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.500+0000 7f6162ffd640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f615006b4d0 0x7f615006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.500+0000 7f6162ffd640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f615006b4d0 0x7f615006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f61640aae50 tx=0x7f6158009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.504+0000 7f614bfff640 1 -- 172.21.15.26:0/1273954713 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f616414c500 con 0x7f61640c12a0 2024-03-06T07:22:46.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:46.724+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f61640ac980 con 0x7f61640c12a0 2024-03-06T07:22:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.378+0000 7f614bfff640 1 -- 172.21.15.26:0/1273954713 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6154058bc0 con 0x7f61640c12a0 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f615006b4d0 msgr2=0x7f615006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f615006b4d0 0x7f615006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f61640aae50 tx=0x7f6158009040 comp rx=0 tx=0).stop 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640c12a0 msgr2=0x7f6164153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640c12a0 0x7f6164153000 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f615400e960 tx=0x7f615400ee20 comp rx=0 tx=0).stop 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 shutdown_connections 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f615006b4d0 0x7f615006d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164152770 0x7f61641ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61640c7710 0x7f6164141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 --2- 172.21.15.26:0/1273954713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640c12a0 0x7f6164153000 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 >> 172.21.15.26:0/1273954713 conn(0x7f61640a94d0 msgr2=0x7f616414f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 shutdown_connections 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.381+0000 7f6169f7a640 1 -- 172.21.15.26:0/1273954713 wait complete. 2024-03-06T07:22:47.393 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-06T07:22:47.394 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:47.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.496+0000 7fdde0be9640 1 Processor -- start 2024-03-06T07:22:47.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.496+0000 7fdde0be9640 1 -- start start 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdde0be9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdddc14f360 0x7fdddc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdde0be9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdddc158b60 0x7fdddc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdde0be9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc0c73d0 con 0x7fdddc0c88c0 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdde0be9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0c8dd0 con 0x7fdddc158b60 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdde0be9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0be2e0 con 0x7fdddc14f360 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdddad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdddc158b60 0x7fdddc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdddad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdddc158b60 0x7fdddc158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49146/0 (socket says 172.21.15.26:49146) 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdddad9d640 1 -- 172.21.15.26:0/3193492755 learned_addr learned my addr 172.21.15.26:0/3193492755 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fddd3fff640 1 --1- 172.21.15.26:0/3193492755 >> v1:172.21.15.26:6789/0 conn(0x7fdddc0c88c0 0x7fdddc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57494/0 (socket says 172.21.15.26:57494) 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fddda59c640 1 --2- 172.21.15.26:0/3193492755 >> v2:172.21.15.117:3300/0 conn(0x7fdddc14f360 0x7fdddc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdddad9d640 1 -- 172.21.15.26:0/3193492755 >> v2:172.21.15.117:3300/0 conn(0x7fdddc14f360 msgr2=0x7fdddc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdddad9d640 1 --2- 172.21.15.26:0/3193492755 >> v2:172.21.15.117:3300/0 conn(0x7fdddc14f360 0x7fdddc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdddad9d640 1 -- 172.21.15.26:0/3193492755 >> v1:172.21.15.26:6789/0 conn(0x7fdddc0c88c0 legacy=0x7fdddc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fdddad9d640 1 -- 172.21.15.26:0/3193492755 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0c7670 con 0x7fdddc158b60 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.497+0000 7fddd9d9b640 1 -- 172.21.15.26:0/3193492755 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783255029 0 0) 0x7fdddc0c73d0 con 0x7fdddc0c88c0 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.498+0000 7fdddad9d640 1 --2- 172.21.15.26:0/3193492755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc158b60 0x7fdddc158ff0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdddc08c840 tx=0x7fddd402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776fa2ae268614b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.498+0000 7fddd9d9b640 1 -- 172.21.15.26:0/3193492755 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd402ca80 con 0x7fdddc158b60 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.498+0000 7fddd9d9b640 1 -- 172.21.15.26:0/3193492755 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd402cc20 con 0x7fdddc158b60 2024-03-06T07:22:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.498+0000 7fddd9d9b640 1 -- 172.21.15.26:0/3193492755 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40343f0 con 0x7fdddc158b60 2024-03-06T07:22:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 -- 172.21.15.26:0/3193492755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc158b60 msgr2=0x7fdddc158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 --2- 172.21.15.26:0/3193492755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc158b60 0x7fdddc158ff0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdddc08c840 tx=0x7fddd402bee0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 -- 172.21.15.26:0/3193492755 shutdown_connections 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 --2- 172.21.15.26:0/3193492755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc158b60 0x7fdddc158ff0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 --2- 172.21.15.26:0/3193492755 >> v2:172.21.15.117:3300/0 conn(0x7fdddc14f360 0x7fdddc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 -- 172.21.15.26:0/3193492755 >> 172.21.15.26:0/3193492755 conn(0x7fdddc0ac570 msgr2=0x7fdddc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 -- 172.21.15.26:0/3193492755 shutdown_connections 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.499+0000 7fdde0be9640 1 -- 172.21.15.26:0/3193492755 wait complete. 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.501+0000 7fdde0be9640 1 Processor -- start 2024-03-06T07:22:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.501+0000 7fdde0be9640 1 -- start start 2024-03-06T07:22:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fdde0be9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc0c88c0 0x7fdddc1ead00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fdde0be9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdddc14f360 0x7fdddc1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fdde0be9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdddc158b60 0x7fdddc0c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fdde0be9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0c5a50 con 0x7fdddc0c88c0 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fdde0be9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0c7c30 con 0x7fdddc14f360 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fdde0be9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0c5bd0 con 0x7fdddc158b60 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc0c88c0 0x7fdddc1ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc0c88c0 0x7fdddc1ead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49154/0 (socket says 172.21.15.26:49154) 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 -- 172.21.15.26:0/1053995278 learned_addr learned my addr 172.21.15.26:0/1053995278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddd3fff640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdddc14f360 0x7fdddc1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fdddad9d640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdddc158b60 0x7fdddc0c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 -- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdddc158b60 msgr2=0x7fdddc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdddc158b60 0x7fdddc0c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 -- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdddc14f360 msgr2=0x7fdddc1510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdddc14f360 0x7fdddc1510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.502+0000 7fddda59c640 1 -- 172.21.15.26:0/1053995278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0c0cb0 con 0x7fdddc0c88c0 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.503+0000 7fddda59c640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc0c88c0 0x7fdddc1ead00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00e870 tx=0x7fddcc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.503+0000 7fddd2ffd640 1 -- 172.21.15.26:0/1053995278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc017070 con 0x7fdddc0c88c0 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.503+0000 7fddd2ffd640 1 -- 172.21.15.26:0/1053995278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc00c940 con 0x7fdddc0c88c0 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.503+0000 7fddd2ffd640 1 -- 172.21.15.26:0/1053995278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc00cd30 con 0x7fdddc0c88c0 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.503+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc0c0f90 con 0x7fdddc0c88c0 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.503+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc151610 con 0x7fdddc0c88c0 2024-03-06T07:22:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.504+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc04f330 con 0x7fdddc0c88c0 2024-03-06T07:22:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.504+0000 7fddd2ffd640 1 -- 172.21.15.26:0/1053995278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddcc01a070 con 0x7fdddc0c88c0 2024-03-06T07:22:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.505+0000 7fddd2ffd640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fddbc06b4d0 0x7fddbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.505+0000 7fddd2ffd640 1 -- 172.21.15.26:0/1053995278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fddcc08a820 con 0x7fdddc0c88c0 2024-03-06T07:22:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.505+0000 7fddd3fff640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fddbc06b4d0 0x7fddbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.506+0000 7fddd3fff640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fddbc06b4d0 0x7fddbc06d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdddc14e1d0 tx=0x7fddc4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.509+0000 7fddd2ffd640 1 -- 172.21.15.26:0/1053995278 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fddcc058c00 con 0x7fdddc0c88c0 2024-03-06T07:22:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.727+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdddc06cda0 con 0x7fdddc0c88c0 2024-03-06T07:22:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.727+0000 7fddd2ffd640 1 -- 172.21.15.26:0/1053995278 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+6259 (secure 0 0 0) 0x7fddcc05cca0 con 0x7fdddc0c88c0 2024-03-06T07:22:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:47.729 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:36.381549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6339":{"gid":6339,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/218606650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":218606650},{"type":"v1","addr":"172.21.15.117:6836","nonce":218606650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6339,"qdb_cluster":[6339]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:22:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 58 2024-03-06T07:22:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fddbc06b4d0 msgr2=0x7fddbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fddbc06b4d0 0x7fddbc06d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdddc14e1d0 tx=0x7fddc4007de0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc0c88c0 msgr2=0x7fdddc1ead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc0c88c0 0x7fdddc1ead00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00e870 tx=0x7fddcc00ed30 comp rx=0 tx=0).stop 2024-03-06T07:22:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 shutdown_connections 2024-03-06T07:22:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fddbc06b4d0 0x7fddbc06d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdddc158b60 0x7fdddc0c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdddc14f360 0x7fdddc1510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 --2- 172.21.15.26:0/1053995278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdddc0c88c0 0x7fdddc1ead00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.729+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 >> 172.21.15.26:0/1053995278 conn(0x7fdddc0ac570 msgr2=0x7fdddc0c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.730+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 shutdown_connections 2024-03-06T07:22:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.730+0000 7fdde0be9640 1 -- 172.21.15.26:0/1053995278 wait complete. 2024-03-06T07:22:47.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:47.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.843+0000 7f52a9e40640 1 Processor -- start 2024-03-06T07:22:47.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.843+0000 7f52a9e40640 1 -- start start 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a9e40640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52a40c88c0 0x7f52a4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a9e40640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f52a4158b60 0x7f52a4158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a9e40640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a40c73d0 con 0x7f52a414f360 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a9e40640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a40c8dd0 con 0x7f52a40c88c0 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a9e40640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a40be2e0 con 0x7f52a4158b60 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52a40c88c0 0x7f52a4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52a40c88c0 0x7f52a4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49156/0 (socket says 172.21.15.26:49156) 2024-03-06T07:22:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a2ffd640 1 -- 172.21.15.26:0/4235980499 learned_addr learned my addr 172.21.15.26:0/4235980499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.844+0000 7f52a3fff640 1 --2- 172.21.15.26:0/4235980499 >> v2:172.21.15.117:3300/0 conn(0x7f52a4158b60 0x7f52a4158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.845+0000 7f52a2ffd640 1 -- 172.21.15.26:0/4235980499 >> v2:172.21.15.117:3300/0 conn(0x7f52a4158b60 msgr2=0x7f52a4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.845+0000 7f52a2ffd640 1 --2- 172.21.15.26:0/4235980499 >> v2:172.21.15.117:3300/0 conn(0x7f52a4158b60 0x7f52a4158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.845+0000 7f52a2ffd640 1 -- 172.21.15.26:0/4235980499 >> v1:172.21.15.117:6789/0 conn(0x7f52a414f360 legacy=0x7f52a40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.845+0000 7f52a27fc640 1 -- 172.21.15.26:0/4235980499 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311569267 0 0) 0x7f52a40c73d0 con 0x7f52a414f360 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.845+0000 7f52a2ffd640 1 -- 172.21.15.26:0/4235980499 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40c7670 con 0x7f52a40c88c0 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.845+0000 7f52a3fff640 1 --2- 172.21.15.26:0/4235980499 >> v2:172.21.15.117:3300/0 conn(0x7f52a4158b60 0x7f52a4158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.845+0000 7f52a2ffd640 1 --2- 172.21.15.26:0/4235980499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a4155c10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f52a4152d60 tx=0x7f528c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507bf2f6b129b7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a27fc640 1 -- 172.21.15.26:0/4235980499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c02ca80 con 0x7f52a40c88c0 2024-03-06T07:22:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a27fc640 1 -- 172.21.15.26:0/4235980499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c02cc20 con 0x7f52a40c88c0 2024-03-06T07:22:47.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a27fc640 1 -- 172.21.15.26:0/4235980499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0343f0 con 0x7f52a40c88c0 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 -- 172.21.15.26:0/4235980499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 msgr2=0x7f52a4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 --2- 172.21.15.26:0/4235980499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a4155c10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f52a4152d60 tx=0x7f528c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 -- 172.21.15.26:0/4235980499 shutdown_connections 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 --2- 172.21.15.26:0/4235980499 >> v2:172.21.15.117:3300/0 conn(0x7f52a4158b60 0x7f52a4158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 --2- 172.21.15.26:0/4235980499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a4155c10 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 -- 172.21.15.26:0/4235980499 >> 172.21.15.26:0/4235980499 conn(0x7f52a40ac570 msgr2=0x7f52a40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 -- 172.21.15.26:0/4235980499 shutdown_connections 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.846+0000 7f52a9e40640 1 -- 172.21.15.26:0/4235980499 wait complete. 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.848+0000 7f52a9e40640 1 Processor -- start 2024-03-06T07:22:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.848+0000 7f52a9e40640 1 -- start start 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a9e40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a41ee620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a9e40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a414f360 0x7f52a4148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a9e40640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a4158b60 0x7f52a4148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a9e40640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40c5a50 con 0x7f52a40c88c0 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a9e40640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40c7c30 con 0x7f52a414f360 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a9e40640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40c5bd0 con 0x7f52a4158b60 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a41ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a41ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49162/0 (socket says 172.21.15.26:49162) 2024-03-06T07:22:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a37fe640 1 -- 172.21.15.26:0/1436854415 learned_addr learned my addr 172.21.15.26:0/1436854415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a2ffd640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a414f360 0x7f52a4148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.849+0000 7f52a3fff640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a4158b60 0x7f52a4148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a37fe640 1 -- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a4158b60 msgr2=0x7f52a4148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a37fe640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a4158b60 0x7f52a4148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a37fe640 1 -- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a414f360 msgr2=0x7f52a4148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a37fe640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a414f360 0x7f52a4148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a37fe640 1 -- 172.21.15.26:0/1436854415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a4152b80 con 0x7f52a40c88c0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a37fe640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a41ee620 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f529400c410 tx=0x7f529400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f529bfff640 1 -- 172.21.15.26:0/1436854415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294017070 con 0x7f52a40c88c0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f529bfff640 1 -- 172.21.15.26:0/1436854415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52940035f0 con 0x7f52a40c88c0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a41eee60 con 0x7f52a40c88c0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f529bfff640 1 -- 172.21.15.26:0/1436854415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294003a20 con 0x7f52a40c88c0 2024-03-06T07:22:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.850+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a41eeb60 con 0x7f52a40c88c0 2024-03-06T07:22:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.851+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a414eea0 con 0x7f52a40c88c0 2024-03-06T07:22:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.852+0000 7f529bfff640 1 -- 172.21.15.26:0/1436854415 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f529401a070 con 0x7f52a40c88c0 2024-03-06T07:22:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.852+0000 7f529bfff640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528406b4d0 0x7f528406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.852+0000 7f529bfff640 1 -- 172.21.15.26:0/1436854415 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f5294089930 con 0x7f52a40c88c0 2024-03-06T07:22:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.852+0000 7f52a2ffd640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528406b4d0 0x7f528406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.853+0000 7f52a2ffd640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528406b4d0 0x7f528406d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f528c035000 tx=0x7f528c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:47.857+0000 7f529bfff640 1 -- 172.21.15.26:0/1436854415 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5294057d10 con 0x7f52a40c88c0 2024-03-06T07:22:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.041+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52a404f330 con 0x7f52a40c88c0 2024-03-06T07:22:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.042+0000 7f529bfff640 1 -- 172.21.15.26:0/1436854415 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+20681 (secure 0 0 0) 0x7f529405bdb0 con 0x7f52a40c88c0 2024-03-06T07:22:48.044 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:48.044 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:46.369612+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.044+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528406b4d0 msgr2=0x7f528406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.044+0000 7f52a9e40640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528406b4d0 0x7f528406d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f528c035000 tx=0x7f528c030000 comp rx=0 tx=0).stop 2024-03-06T07:22:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.044+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 msgr2=0x7f52a41ee620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.044+0000 7f52a9e40640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a41ee620 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f529400c410 tx=0x7f529400c810 comp rx=0 tx=0).stop 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 shutdown_connections 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f528406b4d0 0x7f528406d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a4158b60 0x7f52a4148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a414f360 0x7f52a4148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 --2- 172.21.15.26:0/1436854415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a40c88c0 0x7f52a41ee620 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 >> 172.21.15.26:0/1436854415 conn(0x7f52a40ac570 msgr2=0x7f52a40ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 shutdown_connections 2024-03-06T07:22:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.045+0000 7f52a9e40640 1 -- 172.21.15.26:0/1436854415 wait complete. 2024-03-06T07:22:48.057 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:22:48.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.164+0000 7f3188cc2640 1 Processor -- start 2024-03-06T07:22:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.164+0000 7f3188cc2640 1 -- start start 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.164+0000 7f3188cc2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31840c7180 0x7f3184159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.164+0000 7f3188cc2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840c7c60 con 0x7f31840c88b0 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.164+0000 7f3188cc2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840c8060 con 0x7f31841527b0 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.164+0000 7f3188cc2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3184159640 con 0x7f31840c7180 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f3182d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f31840c88b0 0x7f31840c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57500/0 (socket says 172.21.15.26:57500) 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f3182d9d640 1 -- 172.21.15.26:0/637549049 learned_addr learned my addr 172.21.15.26:0/637549049 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f3181d9b640 1 --2- 172.21.15.26:0/637549049 >> v2:172.21.15.117:3300/0 conn(0x7f31840c7180 0x7f3184159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f318159a640 1 -- 172.21.15.26:0/637549049 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361483905 0 0) 0x7f31840c8060 con 0x7f31841527b0 2024-03-06T07:22:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f318159a640 1 -- 172.21.15.26:0/637549049 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3170003520 con 0x7f31841527b0 2024-03-06T07:22:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f318159a640 1 -- 172.21.15.26:0/637549049 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397849303 0 0) 0x7f317c001110 con 0x7f31840c88b0 2024-03-06T07:22:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f318159a640 1 -- 172.21.15.26:0/637549049 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31840c8060 con 0x7f31840c88b0 2024-03-06T07:22:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f3181d9b640 1 -- 172.21.15.26:0/637549049 >> v1:172.21.15.117:6789/0 conn(0x7f31841527b0 legacy=0x7f3184155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f3181d9b640 1 -- 172.21.15.26:0/637549049 >> v1:172.21.15.26:6789/0 conn(0x7f31840c88b0 legacy=0x7f31840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.165+0000 7f3181d9b640 1 -- 172.21.15.26:0/637549049 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840c76a0 con 0x7f31840c7180 2024-03-06T07:22:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.166+0000 7f3181d9b640 1 --2- 172.21.15.26:0/637549049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 0x7f3184159100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32192fa35ea92673 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.166+0000 7f318159a640 1 -- 172.21.15.26:0/637549049 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c035020 con 0x7f31840c7180 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.166+0000 7f318159a640 1 -- 172.21.15.26:0/637549049 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c031830 con 0x7f31840c7180 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.166+0000 7f318159a640 1 -- 172.21.15.26:0/637549049 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c031c00 con 0x7f31840c7180 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.167+0000 7f3188cc2640 1 -- 172.21.15.26:0/637549049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 msgr2=0x7f3184159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.167+0000 7f3188cc2640 1 --2- 172.21.15.26:0/637549049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 0x7f3184159100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.167+0000 7f3188cc2640 1 -- 172.21.15.26:0/637549049 shutdown_connections 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.167+0000 7f3188cc2640 1 --2- 172.21.15.26:0/637549049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 0x7f3184159100 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.167+0000 7f3188cc2640 1 -- 172.21.15.26:0/637549049 >> 172.21.15.26:0/637549049 conn(0x7f31840ac560 msgr2=0x7f31840ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.167+0000 7f3188cc2640 1 -- 172.21.15.26:0/637549049 shutdown_connections 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.167+0000 7f3188cc2640 1 -- 172.21.15.26:0/637549049 wait complete. 2024-03-06T07:22:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.168+0000 7f3188cc2640 1 Processor -- start 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.168+0000 7f3188cc2640 1 -- start start 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3188cc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 0x7f31841577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3188cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c88b0 0x7f31841e0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3188cc2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31841527b0 0x7f318414d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3188cc2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184155600 con 0x7f31840c88b0 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3188cc2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184155320 con 0x7f31840c7180 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3188cc2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3184152b90 con 0x7f31841527b0 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c88b0 0x7f31841e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c88b0 0x7f31841e0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49174/0 (socket says 172.21.15.26:49174) 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 -- 172.21.15.26:0/2305477002 learned_addr learned my addr 172.21.15.26:0/2305477002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f318259c640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 0x7f31841577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3182d9d640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31841527b0 0x7f318414d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 -- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31841527b0 msgr2=0x7f318414d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31841527b0 0x7f318414d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 -- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 msgr2=0x7f31841577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 0x7f31841577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 -- 172.21.15.26:0/2305477002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31841ded60 con 0x7f31840c88b0 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.169+0000 7f3181d9b640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c88b0 0x7f31841e0130 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f316c02bee0 tx=0x7f316c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.170+0000 7f317affd640 1 -- 172.21.15.26:0/2305477002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c035040 con 0x7f31840c88b0 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.170+0000 7f317affd640 1 -- 172.21.15.26:0/2305477002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c03b920 con 0x7f31840c88b0 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.170+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31841df040 con 0x7f31840c88b0 2024-03-06T07:22:48.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.170+0000 7f317affd640 1 -- 172.21.15.26:0/2305477002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c03bd10 con 0x7f31840c88b0 2024-03-06T07:22:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.170+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31841df600 con 0x7f31840c88b0 2024-03-06T07:22:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.171+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318404f330 con 0x7f31840c88b0 2024-03-06T07:22:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.172+0000 7f317affd640 1 -- 172.21.15.26:0/2305477002 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f316c038020 con 0x7f31840c88b0 2024-03-06T07:22:48.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.172+0000 7f317affd640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b4d0 0x7f316406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.172+0000 7f317affd640 1 -- 172.21.15.26:0/2305477002 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f316c0ac680 con 0x7f31840c88b0 2024-03-06T07:22:48.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.172+0000 7f318259c640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b4d0 0x7f316406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.173+0000 7f318259c640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b4d0 0x7f316406d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f31840bde20 tx=0x7f3174009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.177+0000 7f317affd640 1 -- 172.21.15.26:0/2305477002 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f316c07a9e0 con 0x7f31840c88b0 2024-03-06T07:22:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.395+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f318406ce00 con 0x7f316406b4d0 2024-03-06T07:22:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.406+0000 7f317affd640 1 -- 172.21.15.26:0/2305477002 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485419 (secure 0 0 0) 0x7f318406ce00 con 0x7f316406b4d0 2024-03-06T07:22:48.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b4d0 msgr2=0x7f316406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b4d0 0x7f316406d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f31840bde20 tx=0x7f3174009040 comp rx=0 tx=0).stop 2024-03-06T07:22:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c88b0 msgr2=0x7f31841e0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c88b0 0x7f31841e0130 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f316c02bee0 tx=0x7f316c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 shutdown_connections 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f316406b4d0 0x7f316406d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31841527b0 0x7f318414d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31840c88b0 0x7f31841e0130 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 --2- 172.21.15.26:0/2305477002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31840c7180 0x7f31841577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.410+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 >> 172.21.15.26:0/2305477002 conn(0x7f31840ac560 msgr2=0x7f31840c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.411+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 shutdown_connections 2024-03-06T07:22:48.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.411+0000 7f3188cc2640 1 -- 172.21.15.26:0/2305477002 wait complete. 2024-03-06T07:22:48.423 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-03-06T07:22:47.063892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244432,"kb_used_data":7444,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730288,"statfs":{"total":773094113280,"available":772843814912,"internally_reserved":0,"allocated":7622656,"data_stored":4714746,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000767"},"pg_stats":[{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361902+0000","last_change":"2024-03-06T07:22:44.361902+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361902+0000","last_undegraded":"2024-03-06T07:22:44.361902+0000","last_fullsized":"2024-03-06T07:22:44.361902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365039+0000","last_change":"2024-03-06T07:22:35.025691+0000","last_active":"2024-03-06T07:22:44.365039+0000","last_peered":"2024-03-06T07:22:44.365039+0000","last_clean":"2024-03-06T07:22:44.365039+0000","last_became_active":"2024-03-06T07:22:35.025394+0000","last_became_peered":"2024-03-06T07:22:35.025394+0000","last_unstale":"2024-03-06T07:22:44.365039+0000","last_undegraded":"2024-03-06T07:22:44.365039+0000","last_fullsized":"2024-03-06T07:22:44.365039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366504+0000","last_change":"2024-03-06T07:22:44.366504+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366504+0000","last_undegraded":"2024-03-06T07:22:44.366504+0000","last_fullsized":"2024-03-06T07:22:44.366504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373511+0000","last_change":"2024-03-06T07:22:35.048775+0000","last_active":"2024-03-06T07:22:44.373511+0000","last_peered":"2024-03-06T07:22:44.373511+0000","last_clean":"2024-03-06T07:22:44.373511+0000","last_became_active":"2024-03-06T07:22:35.048476+0000","last_became_peered":"2024-03-06T07:22:35.048476+0000","last_unstale":"2024-03-06T07:22:44.373511+0000","last_undegraded":"2024-03-06T07:22:44.373511+0000","last_fullsized":"2024-03-06T07:22:44.373511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:41.704388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362319+0000","last_change":"2024-03-06T07:22:44.362319+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362319+0000","last_undegraded":"2024-03-06T07:22:44.362319+0000","last_fullsized":"2024-03-06T07:22:44.362319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359053+0000","last_change":"2024-03-06T07:22:35.037727+0000","last_active":"2024-03-06T07:22:44.359053+0000","last_peered":"2024-03-06T07:22:44.359053+0000","last_clean":"2024-03-06T07:22:44.359053+0000","last_became_active":"2024-03-06T07:22:35.037421+0000","last_became_peered":"2024-03-06T07:22:35.037421+0000","last_unstale":"2024-03-06T07:22:44.359053+0000","last_undegraded":"2024-03-06T07:22:44.359053+0000","last_fullsized":"2024-03-06T07:22:44.359053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362908+0000","last_change":"2024-03-06T07:22:44.362908+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362908+0000","last_undegraded":"2024-03-06T07:22:44.362908+0000","last_fullsized":"2024-03-06T07:22:44.362908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365569+0000","last_change":"2024-03-06T07:22:35.035949+0000","last_active":"2024-03-06T07:22:44.365569+0000","last_peered":"2024-03-06T07:22:44.365569+0000","last_clean":"2024-03-06T07:22:44.365569+0000","last_became_active":"2024-03-06T07:22:35.035687+0000","last_became_peered":"2024-03-06T07:22:35.035687+0000","last_unstale":"2024-03-06T07:22:44.365569+0000","last_undegraded":"2024-03-06T07:22:44.365569+0000","last_fullsized":"2024-03-06T07:22:44.365569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362767+0000","last_change":"2024-03-06T07:22:44.362767+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362767+0000","last_undegraded":"2024-03-06T07:22:44.362767+0000","last_fullsized":"2024-03-06T07:22:44.362767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365712+0000","last_change":"2024-03-06T07:22:35.033081+0000","last_active":"2024-03-06T07:22:44.365712+0000","last_peered":"2024-03-06T07:22:44.365712+0000","last_clean":"2024-03-06T07:22:44.365712+0000","last_became_active":"2024-03-06T07:22:35.032799+0000","last_became_peered":"2024-03-06T07:22:35.032799+0000","last_unstale":"2024-03-06T07:22:44.365712+0000","last_undegraded":"2024-03-06T07:22:44.365712+0000","last_fullsized":"2024-03-06T07:22:44.365712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.187775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365929+0000","last_change":"2024-03-06T07:22:44.365929+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365929+0000","last_undegraded":"2024-03-06T07:22:44.365929+0000","last_fullsized":"2024-03-06T07:22:44.365929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364635+0000","last_change":"2024-03-06T07:22:35.041976+0000","last_active":"2024-03-06T07:22:44.364635+0000","last_peered":"2024-03-06T07:22:44.364635+0000","last_clean":"2024-03-06T07:22:44.364635+0000","last_became_active":"2024-03-06T07:22:35.041546+0000","last_became_peered":"2024-03-06T07:22:35.041546+0000","last_unstale":"2024-03-06T07:22:44.364635+0000","last_undegraded":"2024-03-06T07:22:44.364635+0000","last_fullsized":"2024-03-06T07:22:44.364635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360301+0000","last_change":"2024-03-06T07:22:44.360301+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360301+0000","last_undegraded":"2024-03-06T07:22:44.360301+0000","last_fullsized":"2024-03-06T07:22:44.360301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365335+0000","last_change":"2024-03-06T07:22:35.038320+0000","last_active":"2024-03-06T07:22:44.365335+0000","last_peered":"2024-03-06T07:22:44.365335+0000","last_clean":"2024-03-06T07:22:44.365335+0000","last_became_active":"2024-03-06T07:22:35.038043+0000","last_became_peered":"2024-03-06T07:22:35.038043+0000","last_unstale":"2024-03-06T07:22:44.365335+0000","last_undegraded":"2024-03-06T07:22:44.365335+0000","last_fullsized":"2024-03-06T07:22:44.365335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366763+0000","last_change":"2024-03-06T07:22:44.366763+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366763+0000","last_undegraded":"2024-03-06T07:22:44.366763+0000","last_fullsized":"2024-03-06T07:22:44.366763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368165+0000","last_change":"2024-03-06T07:22:35.040651+0000","last_active":"2024-03-06T07:22:44.368165+0000","last_peered":"2024-03-06T07:22:44.368165+0000","last_clean":"2024-03-06T07:22:44.368165+0000","last_became_active":"2024-03-06T07:22:35.040382+0000","last_became_peered":"2024-03-06T07:22:35.040382+0000","last_unstale":"2024-03-06T07:22:44.368165+0000","last_undegraded":"2024-03-06T07:22:44.368165+0000","last_fullsized":"2024-03-06T07:22:44.368165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:54.755446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360342+0000","last_change":"2024-03-06T07:22:44.360342+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360342+0000","last_undegraded":"2024-03-06T07:22:44.360342+0000","last_fullsized":"2024-03-06T07:22:44.360342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361779+0000","last_change":"2024-03-06T07:22:35.035524+0000","last_active":"2024-03-06T07:22:44.361779+0000","last_peered":"2024-03-06T07:22:44.361779+0000","last_clean":"2024-03-06T07:22:44.361779+0000","last_became_active":"2024-03-06T07:22:35.035252+0000","last_became_peered":"2024-03-06T07:22:35.035252+0000","last_unstale":"2024-03-06T07:22:44.361779+0000","last_undegraded":"2024-03-06T07:22:44.361779+0000","last_fullsized":"2024-03-06T07:22:44.361779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364257+0000","last_change":"2024-03-06T07:22:44.364257+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364257+0000","last_undegraded":"2024-03-06T07:22:44.364257+0000","last_fullsized":"2024-03-06T07:22:44.364257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365193+0000","last_change":"2024-03-06T07:22:35.040188+0000","last_active":"2024-03-06T07:22:44.365193+0000","last_peered":"2024-03-06T07:22:44.365193+0000","last_clean":"2024-03-06T07:22:44.365193+0000","last_became_active":"2024-03-06T07:22:35.039868+0000","last_became_peered":"2024-03-06T07:22:35.039868+0000","last_unstale":"2024-03-06T07:22:44.365193+0000","last_undegraded":"2024-03-06T07:22:44.365193+0000","last_fullsized":"2024-03-06T07:22:44.365193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359274+0000","last_change":"2024-03-06T07:22:44.359274+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359274+0000","last_undegraded":"2024-03-06T07:22:44.359274+0000","last_fullsized":"2024-03-06T07:22:44.359274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375229+0000","last_change":"2024-03-06T07:22:35.049739+0000","last_active":"2024-03-06T07:22:44.375229+0000","last_peered":"2024-03-06T07:22:44.375229+0000","last_clean":"2024-03-06T07:22:44.375229+0000","last_became_active":"2024-03-06T07:22:35.049118+0000","last_became_peered":"2024-03-06T07:22:35.049118+0000","last_unstale":"2024-03-06T07:22:44.375229+0000","last_undegraded":"2024-03-06T07:22:44.375229+0000","last_fullsized":"2024-03-06T07:22:44.375229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:49.784740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362088+0000","last_change":"2024-03-06T07:22:44.362088+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362088+0000","last_undegraded":"2024-03-06T07:22:44.362088+0000","last_fullsized":"2024-03-06T07:22:44.362088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365663+0000","last_change":"2024-03-06T07:22:35.038831+0000","last_active":"2024-03-06T07:22:44.365663+0000","last_peered":"2024-03-06T07:22:44.365663+0000","last_clean":"2024-03-06T07:22:44.365663+0000","last_became_active":"2024-03-06T07:22:35.038550+0000","last_became_peered":"2024-03-06T07:22:35.038550+0000","last_unstale":"2024-03-06T07:22:44.365663+0000","last_undegraded":"2024-03-06T07:22:44.365663+0000","last_fullsized":"2024-03-06T07:22:44.365663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362839+0000","last_change":"2024-03-06T07:22:44.362839+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362839+0000","last_undegraded":"2024-03-06T07:22:44.362839+0000","last_fullsized":"2024-03-06T07:22:44.362839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365819+0000","last_change":"2024-03-06T07:22:35.037819+0000","last_active":"2024-03-06T07:22:44.365819+0000","last_peered":"2024-03-06T07:22:44.365819+0000","last_clean":"2024-03-06T07:22:44.365819+0000","last_became_active":"2024-03-06T07:22:35.037519+0000","last_became_peered":"2024-03-06T07:22:35.037519+0000","last_unstale":"2024-03-06T07:22:44.365819+0000","last_undegraded":"2024-03-06T07:22:44.365819+0000","last_fullsized":"2024-03-06T07:22:44.365819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:49.281137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.369795+0000","last_change":"2024-03-06T07:22:44.369795+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.369795+0000","last_undegraded":"2024-03-06T07:22:44.369795+0000","last_fullsized":"2024-03-06T07:22:44.369795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364770+0000","last_change":"2024-03-06T07:22:35.040052+0000","last_active":"2024-03-06T07:22:44.364770+0000","last_peered":"2024-03-06T07:22:44.364770+0000","last_clean":"2024-03-06T07:22:44.364770+0000","last_became_active":"2024-03-06T07:22:35.039794+0000","last_became_peered":"2024-03-06T07:22:35.039794+0000","last_unstale":"2024-03-06T07:22:44.364770+0000","last_undegraded":"2024-03-06T07:22:44.364770+0000","last_fullsized":"2024-03-06T07:22:44.364770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364008+0000","last_change":"2024-03-06T07:22:44.364008+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364008+0000","last_undegraded":"2024-03-06T07:22:44.364008+0000","last_fullsized":"2024-03-06T07:22:44.364008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366659+0000","last_change":"2024-03-06T07:22:35.034108+0000","last_active":"2024-03-06T07:22:44.366659+0000","last_peered":"2024-03-06T07:22:44.366659+0000","last_clean":"2024-03-06T07:22:44.366659+0000","last_became_active":"2024-03-06T07:22:35.033823+0000","last_became_peered":"2024-03-06T07:22:35.033823+0000","last_unstale":"2024-03-06T07:22:44.366659+0000","last_undegraded":"2024-03-06T07:22:44.366659+0000","last_fullsized":"2024-03-06T07:22:44.366659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:04.872840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359980+0000","last_change":"2024-03-06T07:22:44.359980+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359980+0000","last_undegraded":"2024-03-06T07:22:44.359980+0000","last_fullsized":"2024-03-06T07:22:44.359980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363762+0000","last_change":"2024-03-06T07:22:35.037317+0000","last_active":"2024-03-06T07:22:44.363762+0000","last_peered":"2024-03-06T07:22:44.363762+0000","last_clean":"2024-03-06T07:22:44.363762+0000","last_became_active":"2024-03-06T07:22:35.033185+0000","last_became_peered":"2024-03-06T07:22:35.033185+0000","last_unstale":"2024-03-06T07:22:44.363762+0000","last_undegraded":"2024-03-06T07:22:44.363762+0000","last_fullsized":"2024-03-06T07:22:44.363762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359992+0000","last_change":"2024-03-06T07:22:44.359992+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359992+0000","last_undegraded":"2024-03-06T07:22:44.359992+0000","last_fullsized":"2024-03-06T07:22:44.359992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365447+0000","last_change":"2024-03-06T07:22:35.035073+0000","last_active":"2024-03-06T07:22:44.365447+0000","last_peered":"2024-03-06T07:22:44.365447+0000","last_clean":"2024-03-06T07:22:44.365447+0000","last_became_active":"2024-03-06T07:22:35.034739+0000","last_became_peered":"2024-03-06T07:22:35.034739+0000","last_unstale":"2024-03-06T07:22:44.365447+0000","last_undegraded":"2024-03-06T07:22:44.365447+0000","last_fullsized":"2024-03-06T07:22:44.365447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:12.718069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363938+0000","last_change":"2024-03-06T07:22:44.363938+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363938+0000","last_undegraded":"2024-03-06T07:22:44.363938+0000","last_fullsized":"2024-03-06T07:22:44.363938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359224+0000","last_change":"2024-03-06T07:22:35.032202+0000","last_active":"2024-03-06T07:22:44.359224+0000","last_peered":"2024-03-06T07:22:44.359224+0000","last_clean":"2024-03-06T07:22:44.359224+0000","last_became_active":"2024-03-06T07:22:35.031858+0000","last_became_peered":"2024-03-06T07:22:35.031858+0000","last_unstale":"2024-03-06T07:22:44.359224+0000","last_undegraded":"2024-03-06T07:22:44.359224+0000","last_fullsized":"2024-03-06T07:22:44.359224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364868+0000","last_change":"2024-03-06T07:22:44.364868+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364868+0000","last_undegraded":"2024-03-06T07:22:44.364868+0000","last_fullsized":"2024-03-06T07:22:44.364868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364576+0000","last_change":"2024-03-06T07:22:35.041928+0000","last_active":"2024-03-06T07:22:44.364576+0000","last_peered":"2024-03-06T07:22:44.364576+0000","last_clean":"2024-03-06T07:22:44.364576+0000","last_became_active":"2024-03-06T07:22:35.041672+0000","last_became_peered":"2024-03-06T07:22:35.041672+0000","last_unstale":"2024-03-06T07:22:44.364576+0000","last_undegraded":"2024-03-06T07:22:44.364576+0000","last_fullsized":"2024-03-06T07:22:44.364576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362840+0000","last_change":"2024-03-06T07:22:44.362840+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362840+0000","last_undegraded":"2024-03-06T07:22:44.362840+0000","last_fullsized":"2024-03-06T07:22:44.362840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368275+0000","last_change":"2024-03-06T07:22:35.036178+0000","last_active":"2024-03-06T07:22:44.368275+0000","last_peered":"2024-03-06T07:22:44.368275+0000","last_clean":"2024-03-06T07:22:44.368275+0000","last_became_active":"2024-03-06T07:22:35.035874+0000","last_became_peered":"2024-03-06T07:22:35.035874+0000","last_unstale":"2024-03-06T07:22:44.368275+0000","last_undegraded":"2024-03-06T07:22:44.368275+0000","last_fullsized":"2024-03-06T07:22:44.368275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.764602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364726+0000","last_change":"2024-03-06T07:22:44.364726+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364726+0000","last_undegraded":"2024-03-06T07:22:44.364726+0000","last_fullsized":"2024-03-06T07:22:44.364726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361583+0000","last_change":"2024-03-06T07:22:35.031787+0000","last_active":"2024-03-06T07:22:44.361583+0000","last_peered":"2024-03-06T07:22:44.361583+0000","last_clean":"2024-03-06T07:22:44.361583+0000","last_became_active":"2024-03-06T07:22:35.031524+0000","last_became_peered":"2024-03-06T07:22:35.031524+0000","last_unstale":"2024-03-06T07:22:44.361583+0000","last_undegraded":"2024-03-06T07:22:44.361583+0000","last_fullsized":"2024-03-06T07:22:44.361583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361747+0000","last_change":"2024-03-06T07:22:44.361747+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361747+0000","last_undegraded":"2024-03-06T07:22:44.361747+0000","last_fullsized":"2024-03-06T07:22:44.361747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363506+0000","last_change":"2024-03-06T07:22:35.039311+0000","last_active":"2024-03-06T07:22:44.363506+0000","last_peered":"2024-03-06T07:22:44.363506+0000","last_clean":"2024-03-06T07:22:44.363506+0000","last_became_active":"2024-03-06T07:22:35.038971+0000","last_became_peered":"2024-03-06T07:22:35.038971+0000","last_unstale":"2024-03-06T07:22:44.363506+0000","last_undegraded":"2024-03-06T07:22:44.363506+0000","last_fullsized":"2024-03-06T07:22:44.363506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:42.465427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359016+0000","last_change":"2024-03-06T07:22:44.359016+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359016+0000","last_undegraded":"2024-03-06T07:22:44.359016+0000","last_fullsized":"2024-03-06T07:22:44.359016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361143+0000","last_change":"2024-03-06T07:22:35.031978+0000","last_active":"2024-03-06T07:22:44.361143+0000","last_peered":"2024-03-06T07:22:44.361143+0000","last_clean":"2024-03-06T07:22:44.361143+0000","last_became_active":"2024-03-06T07:22:35.031711+0000","last_became_peered":"2024-03-06T07:22:35.031711+0000","last_unstale":"2024-03-06T07:22:44.361143+0000","last_undegraded":"2024-03-06T07:22:44.361143+0000","last_fullsized":"2024-03-06T07:22:44.361143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:17.004593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359118+0000","last_change":"2024-03-06T07:22:44.359118+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359118+0000","last_undegraded":"2024-03-06T07:22:44.359118+0000","last_fullsized":"2024-03-06T07:22:44.359118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375412+0000","last_change":"2024-03-06T07:22:35.045711+0000","last_active":"2024-03-06T07:22:44.375412+0000","last_peered":"2024-03-06T07:22:44.375412+0000","last_clean":"2024-03-06T07:22:44.375412+0000","last_became_active":"2024-03-06T07:22:35.041242+0000","last_became_peered":"2024-03-06T07:22:35.041242+0000","last_unstale":"2024-03-06T07:22:44.375412+0000","last_undegraded":"2024-03-06T07:22:44.375412+0000","last_fullsized":"2024-03-06T07:22:44.375412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:16.266604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364653+0000","last_change":"2024-03-06T07:22:44.364653+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364653+0000","last_undegraded":"2024-03-06T07:22:44.364653+0000","last_fullsized":"2024-03-06T07:22:44.364653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374041+0000","last_change":"2024-03-06T07:22:35.042301+0000","last_active":"2024-03-06T07:22:44.374041+0000","last_peered":"2024-03-06T07:22:44.374041+0000","last_clean":"2024-03-06T07:22:44.374041+0000","last_became_active":"2024-03-06T07:22:35.041172+0000","last_became_peered":"2024-03-06T07:22:35.041172+0000","last_unstale":"2024-03-06T07:22:44.374041+0000","last_undegraded":"2024-03-06T07:22:44.374041+0000","last_fullsized":"2024-03-06T07:22:44.374041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:58.974636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363518+0000","last_change":"2024-03-06T07:22:44.363518+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363518+0000","last_undegraded":"2024-03-06T07:22:44.363518+0000","last_fullsized":"2024-03-06T07:22:44.363518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365279+0000","last_change":"2024-03-06T07:22:35.038117+0000","last_active":"2024-03-06T07:22:44.365279+0000","last_peered":"2024-03-06T07:22:44.365279+0000","last_clean":"2024-03-06T07:22:44.365279+0000","last_became_active":"2024-03-06T07:22:35.037812+0000","last_became_peered":"2024-03-06T07:22:35.037812+0000","last_unstale":"2024-03-06T07:22:44.365279+0000","last_undegraded":"2024-03-06T07:22:44.365279+0000","last_fullsized":"2024-03-06T07:22:44.365279+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363196+0000","last_change":"2024-03-06T07:22:44.363196+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363196+0000","last_undegraded":"2024-03-06T07:22:44.363196+0000","last_fullsized":"2024-03-06T07:22:44.363196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368368+0000","last_change":"2024-03-06T07:22:35.033425+0000","last_active":"2024-03-06T07:22:44.368368+0000","last_peered":"2024-03-06T07:22:44.368368+0000","last_clean":"2024-03-06T07:22:44.368368+0000","last_became_active":"2024-03-06T07:22:35.032784+0000","last_became_peered":"2024-03-06T07:22:35.032784+0000","last_unstale":"2024-03-06T07:22:44.368368+0000","last_undegraded":"2024-03-06T07:22:44.368368+0000","last_fullsized":"2024-03-06T07:22:44.368368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361026+0000","last_change":"2024-03-06T07:22:44.361026+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361026+0000","last_undegraded":"2024-03-06T07:22:44.361026+0000","last_fullsized":"2024-03-06T07:22:44.361026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368367+0000","last_change":"2024-03-06T07:22:35.033473+0000","last_active":"2024-03-06T07:22:44.368367+0000","last_peered":"2024-03-06T07:22:44.368367+0000","last_clean":"2024-03-06T07:22:44.368367+0000","last_became_active":"2024-03-06T07:22:35.032407+0000","last_became_peered":"2024-03-06T07:22:35.032407+0000","last_unstale":"2024-03-06T07:22:44.368367+0000","last_undegraded":"2024-03-06T07:22:44.368367+0000","last_fullsized":"2024-03-06T07:22:44.368367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:39.792272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363072+0000","last_change":"2024-03-06T07:22:44.363072+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363072+0000","last_undegraded":"2024-03-06T07:22:44.363072+0000","last_fullsized":"2024-03-06T07:22:44.363072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370470+0000","last_change":"2024-03-06T07:22:35.048127+0000","last_active":"2024-03-06T07:22:44.370470+0000","last_peered":"2024-03-06T07:22:44.370470+0000","last_clean":"2024-03-06T07:22:44.370470+0000","last_became_active":"2024-03-06T07:22:35.047869+0000","last_became_peered":"2024-03-06T07:22:35.047869+0000","last_unstale":"2024-03-06T07:22:44.370470+0000","last_undegraded":"2024-03-06T07:22:44.370470+0000","last_fullsized":"2024-03-06T07:22:44.370470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.375194+0000","last_change":"2024-03-06T07:22:44.375194+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.375194+0000","last_undegraded":"2024-03-06T07:22:44.375194+0000","last_fullsized":"2024-03-06T07:22:44.375194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363056+0000","last_change":"2024-03-06T07:22:35.038257+0000","last_active":"2024-03-06T07:22:44.363056+0000","last_peered":"2024-03-06T07:22:44.363056+0000","last_clean":"2024-03-06T07:22:44.363056+0000","last_became_active":"2024-03-06T07:22:35.037969+0000","last_became_peered":"2024-03-06T07:22:35.037969+0000","last_unstale":"2024-03-06T07:22:44.363056+0000","last_undegraded":"2024-03-06T07:22:44.363056+0000","last_fullsized":"2024-03-06T07:22:44.363056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362412+0000","last_change":"2024-03-06T07:22:44.362412+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362412+0000","last_undegraded":"2024-03-06T07:22:44.362412+0000","last_fullsized":"2024-03-06T07:22:44.362412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358712+0000","last_change":"2024-03-06T07:22:35.032129+0000","last_active":"2024-03-06T07:22:44.358712+0000","last_peered":"2024-03-06T07:22:44.358712+0000","last_clean":"2024-03-06T07:22:44.358712+0000","last_became_active":"2024-03-06T07:22:35.027787+0000","last_became_peered":"2024-03-06T07:22:35.027787+0000","last_unstale":"2024-03-06T07:22:44.358712+0000","last_undegraded":"2024-03-06T07:22:44.358712+0000","last_fullsized":"2024-03-06T07:22:44.358712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:27.011065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359560+0000","last_change":"2024-03-06T07:22:44.359560+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359560+0000","last_undegraded":"2024-03-06T07:22:44.359560+0000","last_fullsized":"2024-03-06T07:22:44.359560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363854+0000","last_change":"2024-03-06T07:22:35.036989+0000","last_active":"2024-03-06T07:22:44.363854+0000","last_peered":"2024-03-06T07:22:44.363854+0000","last_clean":"2024-03-06T07:22:44.363854+0000","last_became_active":"2024-03-06T07:22:35.032803+0000","last_became_peered":"2024-03-06T07:22:35.032803+0000","last_unstale":"2024-03-06T07:22:44.363854+0000","last_undegraded":"2024-03-06T07:22:44.363854+0000","last_fullsized":"2024-03-06T07:22:44.363854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365367+0000","last_change":"2024-03-06T07:22:44.365367+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365367+0000","last_undegraded":"2024-03-06T07:22:44.365367+0000","last_fullsized":"2024-03-06T07:22:44.365367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373250+0000","last_change":"2024-03-06T07:22:44.372693+0000","last_active":"2024-03-06T07:22:44.373250+0000","last_peered":"2024-03-06T07:22:44.373250+0000","last_clean":"2024-03-06T07:22:44.373250+0000","last_became_active":"2024-03-06T07:22:44.372375+0000","last_became_peered":"2024-03-06T07:22:44.372375+0000","last_unstale":"2024-03-06T07:22:44.373250+0000","last_undegraded":"2024-03-06T07:22:44.373250+0000","last_fullsized":"2024-03-06T07:22:44.373250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358464+0000","last_change":"2024-03-06T07:22:35.025981+0000","last_active":"2024-03-06T07:22:44.358464+0000","last_peered":"2024-03-06T07:22:44.358464+0000","last_clean":"2024-03-06T07:22:44.358464+0000","last_became_active":"2024-03-06T07:22:35.025721+0000","last_became_peered":"2024-03-06T07:22:35.025721+0000","last_unstale":"2024-03-06T07:22:44.358464+0000","last_undegraded":"2024-03-06T07:22:44.358464+0000","last_fullsized":"2024-03-06T07:22:44.358464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:18.894659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365555+0000","last_change":"2024-03-06T07:22:34.043747+0000","last_active":"2024-03-06T07:22:44.365555+0000","last_peered":"2024-03-06T07:22:44.365555+0000","last_clean":"2024-03-06T07:22:44.365555+0000","last_became_active":"2024-03-06T07:22:34.042367+0000","last_became_peered":"2024-03-06T07:22:34.042367+0000","last_unstale":"2024-03-06T07:22:44.365555+0000","last_undegraded":"2024-03-06T07:22:44.365555+0000","last_fullsized":"2024-03-06T07:22:44.365555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:33.138544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363123+0000","last_change":"2024-03-06T07:22:44.363123+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363123+0000","last_undegraded":"2024-03-06T07:22:44.363123+0000","last_fullsized":"2024-03-06T07:22:44.363123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384852+0000","last_change":"2024-03-06T07:22:44.380429+0000","last_active":"2024-03-06T07:22:44.384852+0000","last_peered":"2024-03-06T07:22:44.384852+0000","last_clean":"2024-03-06T07:22:44.384852+0000","last_became_active":"2024-03-06T07:22:44.380170+0000","last_became_peered":"2024-03-06T07:22:44.380170+0000","last_unstale":"2024-03-06T07:22:44.384852+0000","last_undegraded":"2024-03-06T07:22:44.384852+0000","last_fullsized":"2024-03-06T07:22:44.384852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:53.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370159+0000","last_change":"2024-03-06T07:22:35.032593+0000","last_active":"2024-03-06T07:22:44.370159+0000","last_peered":"2024-03-06T07:22:44.370159+0000","last_clean":"2024-03-06T07:22:44.370159+0000","last_became_active":"2024-03-06T07:22:35.031651+0000","last_became_peered":"2024-03-06T07:22:35.031651+0000","last_unstale":"2024-03-06T07:22:44.370159+0000","last_undegraded":"2024-03-06T07:22:44.370159+0000","last_fullsized":"2024-03-06T07:22:44.370159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364514+0000","last_change":"2024-03-06T07:22:34.038702+0000","last_active":"2024-03-06T07:22:44.364514+0000","last_peered":"2024-03-06T07:22:44.364514+0000","last_clean":"2024-03-06T07:22:44.364514+0000","last_became_active":"2024-03-06T07:22:34.038165+0000","last_became_peered":"2024-03-06T07:22:34.038165+0000","last_unstale":"2024-03-06T07:22:44.364514+0000","last_undegraded":"2024-03-06T07:22:44.364514+0000","last_fullsized":"2024-03-06T07:22:44.364514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362338+0000","last_change":"2024-03-06T07:22:44.362338+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362338+0000","last_undegraded":"2024-03-06T07:22:44.362338+0000","last_fullsized":"2024-03-06T07:22:44.362338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.381941+0000","last_change":"2024-03-06T07:22:44.375256+0000","last_active":"2024-03-06T07:22:44.381941+0000","last_peered":"2024-03-06T07:22:44.381941+0000","last_clean":"2024-03-06T07:22:44.381941+0000","last_became_active":"2024-03-06T07:22:44.374987+0000","last_became_peered":"2024-03-06T07:22:44.374987+0000","last_unstale":"2024-03-06T07:22:44.381941+0000","last_undegraded":"2024-03-06T07:22:44.381941+0000","last_fullsized":"2024-03-06T07:22:44.381941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:32.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361466+0000","last_change":"2024-03-06T07:22:35.035774+0000","last_active":"2024-03-06T07:22:44.361466+0000","last_peered":"2024-03-06T07:22:44.361466+0000","last_clean":"2024-03-06T07:22:44.361466+0000","last_became_active":"2024-03-06T07:22:35.035468+0000","last_became_peered":"2024-03-06T07:22:35.035468+0000","last_unstale":"2024-03-06T07:22:44.361466+0000","last_undegraded":"2024-03-06T07:22:44.361466+0000","last_fullsized":"2024-03-06T07:22:44.361466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:25.144870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368550+0000","last_change":"2024-03-06T07:22:34.042434+0000","last_active":"2024-03-06T07:22:44.368550+0000","last_peered":"2024-03-06T07:22:44.368550+0000","last_clean":"2024-03-06T07:22:44.368550+0000","last_became_active":"2024-03-06T07:22:34.042093+0000","last_became_peered":"2024-03-06T07:22:34.042093+0000","last_unstale":"2024-03-06T07:22:44.368550+0000","last_undegraded":"2024-03-06T07:22:44.368550+0000","last_fullsized":"2024-03-06T07:22:44.368550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364352+0000","last_change":"2024-03-06T07:22:44.364352+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364352+0000","last_undegraded":"2024-03-06T07:22:44.364352+0000","last_fullsized":"2024-03-06T07:22:44.364352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375700+0000","last_change":"2024-03-06T07:22:44.373849+0000","last_active":"2024-03-06T07:22:44.375700+0000","last_peered":"2024-03-06T07:22:44.375700+0000","last_clean":"2024-03-06T07:22:44.375700+0000","last_became_active":"2024-03-06T07:22:44.373135+0000","last_became_peered":"2024-03-06T07:22:44.373135+0000","last_unstale":"2024-03-06T07:22:44.375700+0000","last_undegraded":"2024-03-06T07:22:44.375700+0000","last_fullsized":"2024-03-06T07:22:44.375700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:52.622331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361651+0000","last_change":"2024-03-06T07:22:35.032012+0000","last_active":"2024-03-06T07:22:44.361651+0000","last_peered":"2024-03-06T07:22:44.361651+0000","last_clean":"2024-03-06T07:22:44.361651+0000","last_became_active":"2024-03-06T07:22:35.031728+0000","last_became_peered":"2024-03-06T07:22:35.031728+0000","last_unstale":"2024-03-06T07:22:44.361651+0000","last_undegraded":"2024-03-06T07:22:44.361651+0000","last_fullsized":"2024-03-06T07:22:44.361651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364137+0000","last_change":"2024-03-06T07:22:34.038862+0000","last_active":"2024-03-06T07:22:44.364137+0000","last_peered":"2024-03-06T07:22:44.364137+0000","last_clean":"2024-03-06T07:22:44.364137+0000","last_became_active":"2024-03-06T07:22:34.038590+0000","last_became_peered":"2024-03-06T07:22:34.038590+0000","last_unstale":"2024-03-06T07:22:44.364137+0000","last_undegraded":"2024-03-06T07:22:44.364137+0000","last_fullsized":"2024-03-06T07:22:44.364137+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:15.997587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362381+0000","last_change":"2024-03-06T07:22:44.362381+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362381+0000","last_undegraded":"2024-03-06T07:22:44.362381+0000","last_fullsized":"2024-03-06T07:22:44.362381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376768+0000","last_change":"2024-03-06T07:22:44.375296+0000","last_active":"2024-03-06T07:22:44.376768+0000","last_peered":"2024-03-06T07:22:44.376768+0000","last_clean":"2024-03-06T07:22:44.376768+0000","last_became_active":"2024-03-06T07:22:44.375028+0000","last_became_peered":"2024-03-06T07:22:44.375028+0000","last_unstale":"2024-03-06T07:22:44.376768+0000","last_undegraded":"2024-03-06T07:22:44.376768+0000","last_fullsized":"2024-03-06T07:22:44.376768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.786058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376180+0000","last_change":"2024-03-06T07:22:35.036832+0000","last_active":"2024-03-06T07:22:44.376180+0000","last_peered":"2024-03-06T07:22:44.376180+0000","last_clean":"2024-03-06T07:22:44.376180+0000","last_became_active":"2024-03-06T07:22:35.036392+0000","last_became_peered":"2024-03-06T07:22:35.036392+0000","last_unstale":"2024-03-06T07:22:44.376180+0000","last_undegraded":"2024-03-06T07:22:44.376180+0000","last_fullsized":"2024-03-06T07:22:44.376180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:13.614365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361685+0000","last_change":"2024-03-06T07:22:34.037848+0000","last_active":"2024-03-06T07:22:44.361685+0000","last_peered":"2024-03-06T07:22:44.361685+0000","last_clean":"2024-03-06T07:22:44.361685+0000","last_became_active":"2024-03-06T07:22:34.037572+0000","last_became_peered":"2024-03-06T07:22:34.037572+0000","last_unstale":"2024-03-06T07:22:44.361685+0000","last_undegraded":"2024-03-06T07:22:44.361685+0000","last_fullsized":"2024-03-06T07:22:44.361685+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:15.748729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364294+0000","last_change":"2024-03-06T07:22:44.364294+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364294+0000","last_undegraded":"2024-03-06T07:22:44.364294+0000","last_fullsized":"2024-03-06T07:22:44.364294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379327+0000","last_change":"2024-03-06T07:22:44.375707+0000","last_active":"2024-03-06T07:22:44.379327+0000","last_peered":"2024-03-06T07:22:44.379327+0000","last_clean":"2024-03-06T07:22:44.379327+0000","last_became_active":"2024-03-06T07:22:44.375510+0000","last_became_peered":"2024-03-06T07:22:44.375510+0000","last_unstale":"2024-03-06T07:22:44.379327+0000","last_undegraded":"2024-03-06T07:22:44.379327+0000","last_fullsized":"2024-03-06T07:22:44.379327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:57.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375276+0000","last_change":"2024-03-06T07:22:35.040723+0000","last_active":"2024-03-06T07:22:44.375276+0000","last_peered":"2024-03-06T07:22:44.375276+0000","last_clean":"2024-03-06T07:22:44.375276+0000","last_became_active":"2024-03-06T07:22:35.036544+0000","last_became_peered":"2024-03-06T07:22:35.036544+0000","last_unstale":"2024-03-06T07:22:44.375276+0000","last_undegraded":"2024-03-06T07:22:44.375276+0000","last_fullsized":"2024-03-06T07:22:44.375276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:44.185335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366731+0000","last_change":"2024-03-06T07:22:34.040746+0000","last_active":"2024-03-06T07:22:44.366731+0000","last_peered":"2024-03-06T07:22:44.366731+0000","last_clean":"2024-03-06T07:22:44.366731+0000","last_became_active":"2024-03-06T07:22:34.040466+0000","last_became_peered":"2024-03-06T07:22:34.040466+0000","last_unstale":"2024-03-06T07:22:44.366731+0000","last_undegraded":"2024-03-06T07:22:44.366731+0000","last_fullsized":"2024-03-06T07:22:44.366731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:35.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367578+0000","last_change":"2024-03-06T07:22:44.367578+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367578+0000","last_undegraded":"2024-03-06T07:22:44.367578+0000","last_fullsized":"2024-03-06T07:22:44.367578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384804+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384804+0000","last_peered":"2024-03-06T07:22:44.384804+0000","last_clean":"2024-03-06T07:22:44.384804+0000","last_became_active":"2024-03-06T07:22:44.380134+0000","last_became_peered":"2024-03-06T07:22:44.380134+0000","last_unstale":"2024-03-06T07:22:44.384804+0000","last_undegraded":"2024-03-06T07:22:44.384804+0000","last_fullsized":"2024-03-06T07:22:44.384804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:13.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361680+0000","last_change":"2024-03-06T07:22:35.035207+0000","last_active":"2024-03-06T07:22:44.361680+0000","last_peered":"2024-03-06T07:22:44.361680+0000","last_clean":"2024-03-06T07:22:44.361680+0000","last_became_active":"2024-03-06T07:22:35.034887+0000","last_became_peered":"2024-03-06T07:22:35.034887+0000","last_unstale":"2024-03-06T07:22:44.361680+0000","last_undegraded":"2024-03-06T07:22:44.361680+0000","last_fullsized":"2024-03-06T07:22:44.361680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370135+0000","last_change":"2024-03-06T07:22:34.042833+0000","last_active":"2024-03-06T07:22:44.370135+0000","last_peered":"2024-03-06T07:22:44.370135+0000","last_clean":"2024-03-06T07:22:44.370135+0000","last_became_active":"2024-03-06T07:22:34.042557+0000","last_became_peered":"2024-03-06T07:22:34.042557+0000","last_unstale":"2024-03-06T07:22:44.370135+0000","last_undegraded":"2024-03-06T07:22:44.370135+0000","last_fullsized":"2024-03-06T07:22:44.370135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.372269+0000","last_change":"2024-03-06T07:22:44.372269+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.372269+0000","last_undegraded":"2024-03-06T07:22:44.372269+0000","last_fullsized":"2024-03-06T07:22:44.372269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379060+0000","last_change":"2024-03-06T07:22:44.371322+0000","last_active":"2024-03-06T07:22:44.379060+0000","last_peered":"2024-03-06T07:22:44.379060+0000","last_clean":"2024-03-06T07:22:44.379060+0000","last_became_active":"2024-03-06T07:22:44.371058+0000","last_became_peered":"2024-03-06T07:22:44.371058+0000","last_unstale":"2024-03-06T07:22:44.379060+0000","last_undegraded":"2024-03-06T07:22:44.379060+0000","last_fullsized":"2024-03-06T07:22:44.379060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:25.531656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364553+0000","last_change":"2024-03-06T07:22:35.028293+0000","last_active":"2024-03-06T07:22:44.364553+0000","last_peered":"2024-03-06T07:22:44.364553+0000","last_clean":"2024-03-06T07:22:44.364553+0000","last_became_active":"2024-03-06T07:22:35.028017+0000","last_became_peered":"2024-03-06T07:22:35.028017+0000","last_unstale":"2024-03-06T07:22:44.364553+0000","last_undegraded":"2024-03-06T07:22:44.364553+0000","last_fullsized":"2024-03-06T07:22:44.364553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:36.890851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369543+0000","last_change":"2024-03-06T07:22:34.037660+0000","last_active":"2024-03-06T07:22:44.369543+0000","last_peered":"2024-03-06T07:22:44.369543+0000","last_clean":"2024-03-06T07:22:44.369543+0000","last_became_active":"2024-03-06T07:22:34.037347+0000","last_became_peered":"2024-03-06T07:22:34.037347+0000","last_unstale":"2024-03-06T07:22:44.369543+0000","last_undegraded":"2024-03-06T07:22:44.369543+0000","last_fullsized":"2024-03-06T07:22:44.369543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:43.483410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362846+0000","last_change":"2024-03-06T07:22:44.362846+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362846+0000","last_undegraded":"2024-03-06T07:22:44.362846+0000","last_fullsized":"2024-03-06T07:22:44.362846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374719+0000","last_change":"2024-03-06T07:22:44.370568+0000","last_active":"2024-03-06T07:22:44.374719+0000","last_peered":"2024-03-06T07:22:44.374719+0000","last_clean":"2024-03-06T07:22:44.374719+0000","last_became_active":"2024-03-06T07:22:44.370050+0000","last_became_peered":"2024-03-06T07:22:44.370050+0000","last_unstale":"2024-03-06T07:22:44.374719+0000","last_undegraded":"2024-03-06T07:22:44.374719+0000","last_fullsized":"2024-03-06T07:22:44.374719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:46.225047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375512+0000","last_change":"2024-03-06T07:22:35.045241+0000","last_active":"2024-03-06T07:22:44.375512+0000","last_peered":"2024-03-06T07:22:44.375512+0000","last_clean":"2024-03-06T07:22:44.375512+0000","last_became_active":"2024-03-06T07:22:35.044953+0000","last_became_peered":"2024-03-06T07:22:35.044953+0000","last_unstale":"2024-03-06T07:22:44.375512+0000","last_undegraded":"2024-03-06T07:22:44.375512+0000","last_fullsized":"2024-03-06T07:22:44.375512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365128+0000","last_change":"2024-03-06T07:22:34.038501+0000","last_active":"2024-03-06T07:22:44.365128+0000","last_peered":"2024-03-06T07:22:44.365128+0000","last_clean":"2024-03-06T07:22:44.365128+0000","last_became_active":"2024-03-06T07:22:34.038238+0000","last_became_peered":"2024-03-06T07:22:34.038238+0000","last_unstale":"2024-03-06T07:22:44.365128+0000","last_undegraded":"2024-03-06T07:22:44.365128+0000","last_fullsized":"2024-03-06T07:22:44.365128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363829+0000","last_change":"2024-03-06T07:22:44.363829+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363829+0000","last_undegraded":"2024-03-06T07:22:44.363829+0000","last_fullsized":"2024-03-06T07:22:44.363829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379031+0000","last_change":"2024-03-06T07:22:44.377341+0000","last_active":"2024-03-06T07:22:44.379031+0000","last_peered":"2024-03-06T07:22:44.379031+0000","last_clean":"2024-03-06T07:22:44.379031+0000","last_became_active":"2024-03-06T07:22:44.377070+0000","last_became_peered":"2024-03-06T07:22:44.377070+0000","last_unstale":"2024-03-06T07:22:44.379031+0000","last_undegraded":"2024-03-06T07:22:44.379031+0000","last_fullsized":"2024-03-06T07:22:44.379031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:28.968579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365969+0000","last_change":"2024-03-06T07:22:35.035913+0000","last_active":"2024-03-06T07:22:44.365969+0000","last_peered":"2024-03-06T07:22:44.365969+0000","last_clean":"2024-03-06T07:22:44.365969+0000","last_became_active":"2024-03-06T07:22:35.035658+0000","last_became_peered":"2024-03-06T07:22:35.035658+0000","last_unstale":"2024-03-06T07:22:44.365969+0000","last_undegraded":"2024-03-06T07:22:44.365969+0000","last_fullsized":"2024-03-06T07:22:44.365969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:55.082141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365373+0000","last_change":"2024-03-06T07:22:34.040689+0000","last_active":"2024-03-06T07:22:44.365373+0000","last_peered":"2024-03-06T07:22:44.365373+0000","last_clean":"2024-03-06T07:22:44.365373+0000","last_became_active":"2024-03-06T07:22:34.040410+0000","last_became_peered":"2024-03-06T07:22:34.040410+0000","last_unstale":"2024-03-06T07:22:44.365373+0000","last_undegraded":"2024-03-06T07:22:44.365373+0000","last_fullsized":"2024-03-06T07:22:44.365373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.281130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361564+0000","last_change":"2024-03-06T07:22:44.361564+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361564+0000","last_undegraded":"2024-03-06T07:22:44.361564+0000","last_fullsized":"2024-03-06T07:22:44.361564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384844+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384844+0000","last_peered":"2024-03-06T07:22:44.384844+0000","last_clean":"2024-03-06T07:22:44.384844+0000","last_became_active":"2024-03-06T07:22:44.380123+0000","last_became_peered":"2024-03-06T07:22:44.380123+0000","last_unstale":"2024-03-06T07:22:44.384844+0000","last_undegraded":"2024-03-06T07:22:44.384844+0000","last_fullsized":"2024-03-06T07:22:44.384844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:59.118936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364927+0000","last_change":"2024-03-06T07:22:35.032615+0000","last_active":"2024-03-06T07:22:44.364927+0000","last_peered":"2024-03-06T07:22:44.364927+0000","last_clean":"2024-03-06T07:22:44.364927+0000","last_became_active":"2024-03-06T07:22:35.031167+0000","last_became_peered":"2024-03-06T07:22:35.031167+0000","last_unstale":"2024-03-06T07:22:44.364927+0000","last_undegraded":"2024-03-06T07:22:44.364927+0000","last_fullsized":"2024-03-06T07:22:44.364927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369335+0000","last_change":"2024-03-06T07:22:34.038406+0000","last_active":"2024-03-06T07:22:44.369335+0000","last_peered":"2024-03-06T07:22:44.369335+0000","last_clean":"2024-03-06T07:22:44.369335+0000","last_became_active":"2024-03-06T07:22:34.038040+0000","last_became_peered":"2024-03-06T07:22:34.038040+0000","last_unstale":"2024-03-06T07:22:44.369335+0000","last_undegraded":"2024-03-06T07:22:44.369335+0000","last_fullsized":"2024-03-06T07:22:44.369335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:26.063950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359017+0000","last_change":"2024-03-06T07:22:44.359017+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359017+0000","last_undegraded":"2024-03-06T07:22:44.359017+0000","last_fullsized":"2024-03-06T07:22:44.359017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379693+0000","last_change":"2024-03-06T07:22:44.377976+0000","last_active":"2024-03-06T07:22:44.379693+0000","last_peered":"2024-03-06T07:22:44.379693+0000","last_clean":"2024-03-06T07:22:44.379693+0000","last_became_active":"2024-03-06T07:22:44.377659+0000","last_became_peered":"2024-03-06T07:22:44.377659+0000","last_unstale":"2024-03-06T07:22:44.379693+0000","last_undegraded":"2024-03-06T07:22:44.379693+0000","last_fullsized":"2024-03-06T07:22:44.379693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:42.020403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359010+0000","last_change":"2024-03-06T07:22:35.038459+0000","last_active":"2024-03-06T07:22:44.359010+0000","last_peered":"2024-03-06T07:22:44.359010+0000","last_clean":"2024-03-06T07:22:44.359010+0000","last_became_active":"2024-03-06T07:22:35.038185+0000","last_became_peered":"2024-03-06T07:22:35.038185+0000","last_unstale":"2024-03-06T07:22:44.359010+0000","last_undegraded":"2024-03-06T07:22:44.359010+0000","last_fullsized":"2024-03-06T07:22:44.359010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364863+0000","last_change":"2024-03-06T07:22:34.037962+0000","last_active":"2024-03-06T07:22:44.364863+0000","last_peered":"2024-03-06T07:22:44.364863+0000","last_clean":"2024-03-06T07:22:44.364863+0000","last_became_active":"2024-03-06T07:22:34.037635+0000","last_became_peered":"2024-03-06T07:22:34.037635+0000","last_unstale":"2024-03-06T07:22:44.364863+0000","last_undegraded":"2024-03-06T07:22:44.364863+0000","last_fullsized":"2024-03-06T07:22:44.364863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362391+0000","last_change":"2024-03-06T07:22:44.362391+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362391+0000","last_undegraded":"2024-03-06T07:22:44.362391+0000","last_fullsized":"2024-03-06T07:22:44.362391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382003+0000","last_change":"2024-03-06T07:22:44.377361+0000","last_active":"2024-03-06T07:22:44.382003+0000","last_peered":"2024-03-06T07:22:44.382003+0000","last_clean":"2024-03-06T07:22:44.382003+0000","last_became_active":"2024-03-06T07:22:44.377101+0000","last_became_peered":"2024-03-06T07:22:44.377101+0000","last_unstale":"2024-03-06T07:22:44.382003+0000","last_undegraded":"2024-03-06T07:22:44.382003+0000","last_fullsized":"2024-03-06T07:22:44.382003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:13.917348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365752+0000","last_change":"2024-03-06T07:22:35.036674+0000","last_active":"2024-03-06T07:22:44.365752+0000","last_peered":"2024-03-06T07:22:44.365752+0000","last_clean":"2024-03-06T07:22:44.365752+0000","last_became_active":"2024-03-06T07:22:35.036400+0000","last_became_peered":"2024-03-06T07:22:35.036400+0000","last_unstale":"2024-03-06T07:22:44.365752+0000","last_undegraded":"2024-03-06T07:22:44.365752+0000","last_fullsized":"2024-03-06T07:22:44.365752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365049+0000","last_change":"2024-03-06T07:22:34.041311+0000","last_active":"2024-03-06T07:22:44.365049+0000","last_peered":"2024-03-06T07:22:44.365049+0000","last_clean":"2024-03-06T07:22:44.365049+0000","last_became_active":"2024-03-06T07:22:34.040940+0000","last_became_peered":"2024-03-06T07:22:34.040940+0000","last_unstale":"2024-03-06T07:22:44.365049+0000","last_undegraded":"2024-03-06T07:22:44.365049+0000","last_fullsized":"2024-03-06T07:22:44.365049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:24.924844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362009+0000","last_change":"2024-03-06T07:22:44.362009+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362009+0000","last_undegraded":"2024-03-06T07:22:44.362009+0000","last_fullsized":"2024-03-06T07:22:44.362009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379734+0000","last_change":"2024-03-06T07:22:44.378302+0000","last_active":"2024-03-06T07:22:44.379734+0000","last_peered":"2024-03-06T07:22:44.379734+0000","last_clean":"2024-03-06T07:22:44.379734+0000","last_became_active":"2024-03-06T07:22:44.377994+0000","last_became_peered":"2024-03-06T07:22:44.377994+0000","last_unstale":"2024-03-06T07:22:44.379734+0000","last_undegraded":"2024-03-06T07:22:44.379734+0000","last_fullsized":"2024-03-06T07:22:44.379734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:35.747227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363188+0000","last_change":"2024-03-06T07:22:35.022953+0000","last_active":"2024-03-06T07:22:44.363188+0000","last_peered":"2024-03-06T07:22:44.363188+0000","last_clean":"2024-03-06T07:22:44.363188+0000","last_became_active":"2024-03-06T07:22:35.022681+0000","last_became_peered":"2024-03-06T07:22:35.022681+0000","last_unstale":"2024-03-06T07:22:44.363188+0000","last_undegraded":"2024-03-06T07:22:44.363188+0000","last_fullsized":"2024-03-06T07:22:44.363188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363145+0000","last_change":"2024-03-06T07:22:34.037840+0000","last_active":"2024-03-06T07:22:44.363145+0000","last_peered":"2024-03-06T07:22:44.363145+0000","last_clean":"2024-03-06T07:22:44.363145+0000","last_became_active":"2024-03-06T07:22:34.037556+0000","last_became_peered":"2024-03-06T07:22:34.037556+0000","last_unstale":"2024-03-06T07:22:44.363145+0000","last_undegraded":"2024-03-06T07:22:44.363145+0000","last_fullsized":"2024-03-06T07:22:44.363145+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359660+0000","last_change":"2024-03-06T07:22:44.359660+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359660+0000","last_undegraded":"2024-03-06T07:22:44.359660+0000","last_fullsized":"2024-03-06T07:22:44.359660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379220+0000","last_change":"2024-03-06T07:22:44.372722+0000","last_active":"2024-03-06T07:22:44.379220+0000","last_peered":"2024-03-06T07:22:44.379220+0000","last_clean":"2024-03-06T07:22:44.379220+0000","last_became_active":"2024-03-06T07:22:44.372451+0000","last_became_peered":"2024-03-06T07:22:44.372451+0000","last_unstale":"2024-03-06T07:22:44.379220+0000","last_undegraded":"2024-03-06T07:22:44.379220+0000","last_fullsized":"2024-03-06T07:22:44.379220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:15.672728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366071+0000","last_change":"2024-03-06T07:22:35.034503+0000","last_active":"2024-03-06T07:22:44.366071+0000","last_peered":"2024-03-06T07:22:44.366071+0000","last_clean":"2024-03-06T07:22:44.366071+0000","last_became_active":"2024-03-06T07:22:35.034230+0000","last_became_peered":"2024-03-06T07:22:35.034230+0000","last_unstale":"2024-03-06T07:22:44.366071+0000","last_undegraded":"2024-03-06T07:22:44.366071+0000","last_fullsized":"2024-03-06T07:22:44.366071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365219+0000","last_change":"2024-03-06T07:22:34.039959+0000","last_active":"2024-03-06T07:22:44.365219+0000","last_peered":"2024-03-06T07:22:44.365219+0000","last_clean":"2024-03-06T07:22:44.365219+0000","last_became_active":"2024-03-06T07:22:34.039689+0000","last_became_peered":"2024-03-06T07:22:34.039689+0000","last_unstale":"2024-03-06T07:22:44.365219+0000","last_undegraded":"2024-03-06T07:22:44.365219+0000","last_fullsized":"2024-03-06T07:22:44.365219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:23.420842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364356+0000","last_change":"2024-03-06T07:22:44.364356+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364356+0000","last_undegraded":"2024-03-06T07:22:44.364356+0000","last_fullsized":"2024-03-06T07:22:44.364356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.372909+0000","last_change":"2024-03-06T07:22:44.371938+0000","last_active":"2024-03-06T07:22:44.372909+0000","last_peered":"2024-03-06T07:22:44.372909+0000","last_clean":"2024-03-06T07:22:44.372909+0000","last_became_active":"2024-03-06T07:22:44.371634+0000","last_became_peered":"2024-03-06T07:22:44.371634+0000","last_unstale":"2024-03-06T07:22:44.372909+0000","last_undegraded":"2024-03-06T07:22:44.372909+0000","last_fullsized":"2024-03-06T07:22:44.372909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364645+0000","last_change":"2024-03-06T07:22:35.032290+0000","last_active":"2024-03-06T07:22:44.364645+0000","last_peered":"2024-03-06T07:22:44.364645+0000","last_clean":"2024-03-06T07:22:44.364645+0000","last_became_active":"2024-03-06T07:22:35.032048+0000","last_became_peered":"2024-03-06T07:22:35.032048+0000","last_unstale":"2024-03-06T07:22:44.364645+0000","last_undegraded":"2024-03-06T07:22:44.364645+0000","last_fullsized":"2024-03-06T07:22:44.364645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:25.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375609+0000","last_change":"2024-03-06T07:22:34.043149+0000","last_active":"2024-03-06T07:22:44.375609+0000","last_peered":"2024-03-06T07:22:44.375609+0000","last_clean":"2024-03-06T07:22:44.375609+0000","last_became_active":"2024-03-06T07:22:34.042888+0000","last_became_peered":"2024-03-06T07:22:34.042888+0000","last_unstale":"2024-03-06T07:22:44.375609+0000","last_undegraded":"2024-03-06T07:22:44.375609+0000","last_fullsized":"2024-03-06T07:22:44.375609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366248+0000","last_change":"2024-03-06T07:22:44.366248+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366248+0000","last_undegraded":"2024-03-06T07:22:44.366248+0000","last_fullsized":"2024-03-06T07:22:44.366248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384733+0000","last_change":"2024-03-06T07:22:44.379879+0000","last_active":"2024-03-06T07:22:44.384733+0000","last_peered":"2024-03-06T07:22:44.384733+0000","last_clean":"2024-03-06T07:22:44.384733+0000","last_became_active":"2024-03-06T07:22:44.379611+0000","last_became_peered":"2024-03-06T07:22:44.379611+0000","last_unstale":"2024-03-06T07:22:44.384733+0000","last_undegraded":"2024-03-06T07:22:44.384733+0000","last_fullsized":"2024-03-06T07:22:44.384733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:14.207036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363632+0000","last_change":"2024-03-06T07:22:35.034984+0000","last_active":"2024-03-06T07:22:44.363632+0000","last_peered":"2024-03-06T07:22:44.363632+0000","last_clean":"2024-03-06T07:22:44.363632+0000","last_became_active":"2024-03-06T07:22:35.031303+0000","last_became_peered":"2024-03-06T07:22:35.031303+0000","last_unstale":"2024-03-06T07:22:44.363632+0000","last_undegraded":"2024-03-06T07:22:44.363632+0000","last_fullsized":"2024-03-06T07:22:44.363632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365213+0000","last_change":"2024-03-06T07:22:34.044320+0000","last_active":"2024-03-06T07:22:44.365213+0000","last_peered":"2024-03-06T07:22:44.365213+0000","last_clean":"2024-03-06T07:22:44.365213+0000","last_became_active":"2024-03-06T07:22:34.043832+0000","last_became_peered":"2024-03-06T07:22:34.043832+0000","last_unstale":"2024-03-06T07:22:44.365213+0000","last_undegraded":"2024-03-06T07:22:44.365213+0000","last_fullsized":"2024-03-06T07:22:44.365213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366779+0000","last_change":"2024-03-06T07:22:44.366779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366779+0000","last_undegraded":"2024-03-06T07:22:44.366779+0000","last_fullsized":"2024-03-06T07:22:44.366779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377398+0000","last_change":"2024-03-06T07:22:44.373965+0000","last_active":"2024-03-06T07:22:44.377398+0000","last_peered":"2024-03-06T07:22:44.377398+0000","last_clean":"2024-03-06T07:22:44.377398+0000","last_became_active":"2024-03-06T07:22:44.373617+0000","last_became_peered":"2024-03-06T07:22:44.373617+0000","last_unstale":"2024-03-06T07:22:44.377398+0000","last_undegraded":"2024-03-06T07:22:44.377398+0000","last_fullsized":"2024-03-06T07:22:44.377398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:01.683892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363411+0000","last_change":"2024-03-06T07:22:35.034203+0000","last_active":"2024-03-06T07:22:44.363411+0000","last_peered":"2024-03-06T07:22:44.363411+0000","last_clean":"2024-03-06T07:22:44.363411+0000","last_became_active":"2024-03-06T07:22:35.033903+0000","last_became_peered":"2024-03-06T07:22:35.033903+0000","last_unstale":"2024-03-06T07:22:44.363411+0000","last_undegraded":"2024-03-06T07:22:44.363411+0000","last_fullsized":"2024-03-06T07:22:44.363411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:02:14.829352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361765+0000","last_change":"2024-03-06T07:22:34.036176+0000","last_active":"2024-03-06T07:22:44.361765+0000","last_peered":"2024-03-06T07:22:44.361765+0000","last_clean":"2024-03-06T07:22:44.361765+0000","last_became_active":"2024-03-06T07:22:34.035867+0000","last_became_peered":"2024-03-06T07:22:34.035867+0000","last_unstale":"2024-03-06T07:22:44.361765+0000","last_undegraded":"2024-03-06T07:22:44.361765+0000","last_fullsized":"2024-03-06T07:22:44.361765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:28.336495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.374791+0000","last_change":"2024-03-06T07:22:44.374791+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.374791+0000","last_undegraded":"2024-03-06T07:22:44.374791+0000","last_fullsized":"2024-03-06T07:22:44.374791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376378+0000","last_change":"2024-03-06T07:22:44.369401+0000","last_active":"2024-03-06T07:22:44.376378+0000","last_peered":"2024-03-06T07:22:44.376378+0000","last_clean":"2024-03-06T07:22:44.376378+0000","last_became_active":"2024-03-06T07:22:44.369149+0000","last_became_peered":"2024-03-06T07:22:44.369149+0000","last_unstale":"2024-03-06T07:22:44.376378+0000","last_undegraded":"2024-03-06T07:22:44.376378+0000","last_fullsized":"2024-03-06T07:22:44.376378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:29.324559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363089+0000","last_change":"2024-03-06T07:22:35.038497+0000","last_active":"2024-03-06T07:22:44.363089+0000","last_peered":"2024-03-06T07:22:44.363089+0000","last_clean":"2024-03-06T07:22:44.363089+0000","last_became_active":"2024-03-06T07:22:35.038189+0000","last_became_peered":"2024-03-06T07:22:35.038189+0000","last_unstale":"2024-03-06T07:22:44.363089+0000","last_undegraded":"2024-03-06T07:22:44.363089+0000","last_fullsized":"2024-03-06T07:22:44.363089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:34.698852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369442+0000","last_change":"2024-03-06T07:22:34.038423+0000","last_active":"2024-03-06T07:22:44.369442+0000","last_peered":"2024-03-06T07:22:44.369442+0000","last_clean":"2024-03-06T07:22:44.369442+0000","last_became_active":"2024-03-06T07:22:34.038126+0000","last_became_peered":"2024-03-06T07:22:34.038126+0000","last_unstale":"2024-03-06T07:22:44.369442+0000","last_undegraded":"2024-03-06T07:22:44.369442+0000","last_fullsized":"2024-03-06T07:22:44.369442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:43.441029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364377+0000","last_change":"2024-03-06T07:22:44.364377+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364377+0000","last_undegraded":"2024-03-06T07:22:44.364377+0000","last_fullsized":"2024-03-06T07:22:44.364377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377112+0000","last_change":"2024-03-06T07:22:44.373676+0000","last_active":"2024-03-06T07:22:44.377112+0000","last_peered":"2024-03-06T07:22:44.377112+0000","last_clean":"2024-03-06T07:22:44.377112+0000","last_became_active":"2024-03-06T07:22:44.373209+0000","last_became_peered":"2024-03-06T07:22:44.373209+0000","last_unstale":"2024-03-06T07:22:44.377112+0000","last_undegraded":"2024-03-06T07:22:44.377112+0000","last_fullsized":"2024-03-06T07:22:44.377112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:05.175843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364057+0000","last_change":"2024-03-06T07:22:35.042660+0000","last_active":"2024-03-06T07:22:44.364057+0000","last_peered":"2024-03-06T07:22:44.364057+0000","last_clean":"2024-03-06T07:22:44.364057+0000","last_became_active":"2024-03-06T07:22:35.042327+0000","last_became_peered":"2024-03-06T07:22:35.042327+0000","last_unstale":"2024-03-06T07:22:44.364057+0000","last_undegraded":"2024-03-06T07:22:44.364057+0000","last_fullsized":"2024-03-06T07:22:44.364057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358735+0000","last_change":"2024-03-06T07:22:34.035505+0000","last_active":"2024-03-06T07:22:44.358735+0000","last_peered":"2024-03-06T07:22:44.358735+0000","last_clean":"2024-03-06T07:22:44.358735+0000","last_became_active":"2024-03-06T07:22:34.035257+0000","last_became_peered":"2024-03-06T07:22:34.035257+0000","last_unstale":"2024-03-06T07:22:44.358735+0000","last_undegraded":"2024-03-06T07:22:44.358735+0000","last_fullsized":"2024-03-06T07:22:44.358735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359116+0000","last_change":"2024-03-06T07:22:44.359116+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359116+0000","last_undegraded":"2024-03-06T07:22:44.359116+0000","last_fullsized":"2024-03-06T07:22:44.359116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371251+0000","last_change":"2024-03-06T07:22:44.367628+0000","last_active":"2024-03-06T07:22:44.371251+0000","last_peered":"2024-03-06T07:22:44.371251+0000","last_clean":"2024-03-06T07:22:44.371251+0000","last_became_active":"2024-03-06T07:22:44.367384+0000","last_became_peered":"2024-03-06T07:22:44.367384+0000","last_unstale":"2024-03-06T07:22:44.371251+0000","last_undegraded":"2024-03-06T07:22:44.371251+0000","last_fullsized":"2024-03-06T07:22:44.371251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:37.145482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375993+0000","last_change":"2024-03-06T07:22:35.032174+0000","last_active":"2024-03-06T07:22:44.375993+0000","last_peered":"2024-03-06T07:22:44.375993+0000","last_clean":"2024-03-06T07:22:44.375993+0000","last_became_active":"2024-03-06T07:22:35.031869+0000","last_became_peered":"2024-03-06T07:22:35.031869+0000","last_unstale":"2024-03-06T07:22:44.375993+0000","last_undegraded":"2024-03-06T07:22:44.375993+0000","last_fullsized":"2024-03-06T07:22:44.375993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:58.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364160+0000","last_change":"2024-03-06T07:22:34.037475+0000","last_active":"2024-03-06T07:22:44.364160+0000","last_peered":"2024-03-06T07:22:44.364160+0000","last_clean":"2024-03-06T07:22:44.364160+0000","last_became_active":"2024-03-06T07:22:34.037206+0000","last_became_peered":"2024-03-06T07:22:34.037206+0000","last_unstale":"2024-03-06T07:22:44.364160+0000","last_undegraded":"2024-03-06T07:22:44.364160+0000","last_fullsized":"2024-03-06T07:22:44.364160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367154+0000","last_change":"2024-03-06T07:22:44.367154+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367154+0000","last_undegraded":"2024-03-06T07:22:44.367154+0000","last_fullsized":"2024-03-06T07:22:44.367154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376583+0000","last_change":"2024-03-06T07:22:44.372573+0000","last_active":"2024-03-06T07:22:44.376583+0000","last_peered":"2024-03-06T07:22:44.376583+0000","last_clean":"2024-03-06T07:22:44.376583+0000","last_became_active":"2024-03-06T07:22:44.372321+0000","last_became_peered":"2024-03-06T07:22:44.372321+0000","last_unstale":"2024-03-06T07:22:44.376583+0000","last_undegraded":"2024-03-06T07:22:44.376583+0000","last_fullsized":"2024-03-06T07:22:44.376583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:01.508839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373616+0000","last_change":"2024-03-06T07:22:35.042271+0000","last_active":"2024-03-06T07:22:44.373616+0000","last_peered":"2024-03-06T07:22:44.373616+0000","last_clean":"2024-03-06T07:22:44.373616+0000","last_became_active":"2024-03-06T07:22:35.036189+0000","last_became_peered":"2024-03-06T07:22:35.036189+0000","last_unstale":"2024-03-06T07:22:44.373616+0000","last_undegraded":"2024-03-06T07:22:44.373616+0000","last_fullsized":"2024-03-06T07:22:44.373616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:47.064005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364659+0000","last_change":"2024-03-06T07:22:34.036505+0000","last_active":"2024-03-06T07:22:44.364659+0000","last_peered":"2024-03-06T07:22:44.364659+0000","last_clean":"2024-03-06T07:22:44.364659+0000","last_became_active":"2024-03-06T07:22:34.036198+0000","last_became_peered":"2024-03-06T07:22:34.036198+0000","last_unstale":"2024-03-06T07:22:44.364659+0000","last_undegraded":"2024-03-06T07:22:44.364659+0000","last_fullsized":"2024-03-06T07:22:44.364659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362951+0000","last_change":"2024-03-06T07:22:44.362951+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362951+0000","last_undegraded":"2024-03-06T07:22:44.362951+0000","last_fullsized":"2024-03-06T07:22:44.362951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373381+0000","last_change":"2024-03-06T07:22:44.372300+0000","last_active":"2024-03-06T07:22:44.373381+0000","last_peered":"2024-03-06T07:22:44.373381+0000","last_clean":"2024-03-06T07:22:44.373381+0000","last_became_active":"2024-03-06T07:22:44.372036+0000","last_became_peered":"2024-03-06T07:22:44.372036+0000","last_unstale":"2024-03-06T07:22:44.373381+0000","last_undegraded":"2024-03-06T07:22:44.373381+0000","last_fullsized":"2024-03-06T07:22:44.373381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364758+0000","last_change":"2024-03-06T07:22:35.031196+0000","last_active":"2024-03-06T07:22:44.364758+0000","last_peered":"2024-03-06T07:22:44.364758+0000","last_clean":"2024-03-06T07:22:44.364758+0000","last_became_active":"2024-03-06T07:22:35.030767+0000","last_became_peered":"2024-03-06T07:22:35.030767+0000","last_unstale":"2024-03-06T07:22:44.364758+0000","last_undegraded":"2024-03-06T07:22:44.364758+0000","last_fullsized":"2024-03-06T07:22:44.364758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:43.288135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363735+0000","last_change":"2024-03-06T07:22:34.034863+0000","last_active":"2024-03-06T07:22:44.363735+0000","last_peered":"2024-03-06T07:22:44.363735+0000","last_clean":"2024-03-06T07:22:44.363735+0000","last_became_active":"2024-03-06T07:22:34.034577+0000","last_became_peered":"2024-03-06T07:22:34.034577+0000","last_unstale":"2024-03-06T07:22:44.363735+0000","last_undegraded":"2024-03-06T07:22:44.363735+0000","last_fullsized":"2024-03-06T07:22:44.363735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363562+0000","last_change":"2024-03-06T07:22:44.363562+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363562+0000","last_undegraded":"2024-03-06T07:22:44.363562+0000","last_fullsized":"2024-03-06T07:22:44.363562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379268+0000","last_change":"2024-03-06T07:22:44.374457+0000","last_active":"2024-03-06T07:22:44.379268+0000","last_peered":"2024-03-06T07:22:44.379268+0000","last_clean":"2024-03-06T07:22:44.379268+0000","last_became_active":"2024-03-06T07:22:44.374059+0000","last_became_peered":"2024-03-06T07:22:44.374059+0000","last_unstale":"2024-03-06T07:22:44.379268+0000","last_undegraded":"2024-03-06T07:22:44.379268+0000","last_fullsized":"2024-03-06T07:22:44.379268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:27.741841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.360947+0000","last_change":"2024-03-06T07:22:35.036664+0000","last_active":"2024-03-06T07:22:44.360947+0000","last_peered":"2024-03-06T07:22:44.360947+0000","last_clean":"2024-03-06T07:22:44.360947+0000","last_became_active":"2024-03-06T07:22:35.036379+0000","last_became_peered":"2024-03-06T07:22:35.036379+0000","last_unstale":"2024-03-06T07:22:44.360947+0000","last_undegraded":"2024-03-06T07:22:44.360947+0000","last_fullsized":"2024-03-06T07:22:44.360947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:56.002343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366830+0000","last_change":"2024-03-06T07:22:34.044993+0000","last_active":"2024-03-06T07:22:44.366830+0000","last_peered":"2024-03-06T07:22:44.366830+0000","last_clean":"2024-03-06T07:22:44.366830+0000","last_became_active":"2024-03-06T07:22:34.044795+0000","last_became_peered":"2024-03-06T07:22:34.044795+0000","last_unstale":"2024-03-06T07:22:44.366830+0000","last_undegraded":"2024-03-06T07:22:44.366830+0000","last_fullsized":"2024-03-06T07:22:44.366830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.256142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367883+0000","last_change":"2024-03-06T07:22:44.367883+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367883+0000","last_undegraded":"2024-03-06T07:22:44.367883+0000","last_fullsized":"2024-03-06T07:22:44.367883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371628+0000","last_change":"2024-03-06T07:22:44.369924+0000","last_active":"2024-03-06T07:22:44.371628+0000","last_peered":"2024-03-06T07:22:44.371628+0000","last_clean":"2024-03-06T07:22:44.371628+0000","last_became_active":"2024-03-06T07:22:44.369676+0000","last_became_peered":"2024-03-06T07:22:44.369676+0000","last_unstale":"2024-03-06T07:22:44.371628+0000","last_undegraded":"2024-03-06T07:22:44.371628+0000","last_fullsized":"2024-03-06T07:22:44.371628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:25.600800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365427+0000","last_change":"2024-03-06T07:22:35.031505+0000","last_active":"2024-03-06T07:22:44.365427+0000","last_peered":"2024-03-06T07:22:44.365427+0000","last_clean":"2024-03-06T07:22:44.365427+0000","last_became_active":"2024-03-06T07:22:35.031241+0000","last_became_peered":"2024-03-06T07:22:35.031241+0000","last_unstale":"2024-03-06T07:22:44.365427+0000","last_undegraded":"2024-03-06T07:22:44.365427+0000","last_fullsized":"2024-03-06T07:22:44.365427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373724+0000","last_change":"2024-03-06T07:22:34.044453+0000","last_active":"2024-03-06T07:22:44.373724+0000","last_peered":"2024-03-06T07:22:44.373724+0000","last_clean":"2024-03-06T07:22:44.373724+0000","last_became_active":"2024-03-06T07:22:34.044190+0000","last_became_peered":"2024-03-06T07:22:34.044190+0000","last_unstale":"2024-03-06T07:22:44.373724+0000","last_undegraded":"2024-03-06T07:22:44.373724+0000","last_fullsized":"2024-03-06T07:22:44.373724+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367845+0000","last_change":"2024-03-06T07:22:44.367845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367845+0000","last_undegraded":"2024-03-06T07:22:44.367845+0000","last_fullsized":"2024-03-06T07:22:44.367845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382847+0000","last_change":"2024-03-06T07:22:44.382519+0000","last_active":"2024-03-06T07:22:44.382847+0000","last_peered":"2024-03-06T07:22:44.382847+0000","last_clean":"2024-03-06T07:22:44.382847+0000","last_became_active":"2024-03-06T07:22:44.382295+0000","last_became_peered":"2024-03-06T07:22:44.382295+0000","last_unstale":"2024-03-06T07:22:44.382847+0000","last_undegraded":"2024-03-06T07:22:44.382847+0000","last_fullsized":"2024-03-06T07:22:44.382847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:00.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361555+0000","last_change":"2024-03-06T07:22:35.034792+0000","last_active":"2024-03-06T07:22:44.361555+0000","last_peered":"2024-03-06T07:22:44.361555+0000","last_clean":"2024-03-06T07:22:44.361555+0000","last_became_active":"2024-03-06T07:22:35.034503+0000","last_became_peered":"2024-03-06T07:22:35.034503+0000","last_unstale":"2024-03-06T07:22:44.361555+0000","last_undegraded":"2024-03-06T07:22:44.361555+0000","last_fullsized":"2024-03-06T07:22:44.361555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368734+0000","last_change":"2024-03-06T07:22:34.037188+0000","last_active":"2024-03-06T07:22:44.368734+0000","last_peered":"2024-03-06T07:22:44.368734+0000","last_clean":"2024-03-06T07:22:44.368734+0000","last_became_active":"2024-03-06T07:22:34.036918+0000","last_became_peered":"2024-03-06T07:22:34.036918+0000","last_unstale":"2024-03-06T07:22:44.368734+0000","last_undegraded":"2024-03-06T07:22:44.368734+0000","last_fullsized":"2024-03-06T07:22:44.368734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361861+0000","last_change":"2024-03-06T07:22:44.361861+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361861+0000","last_undegraded":"2024-03-06T07:22:44.361861+0000","last_fullsized":"2024-03-06T07:22:44.361861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379180+0000","last_change":"2024-03-06T07:22:44.371466+0000","last_active":"2024-03-06T07:22:44.379180+0000","last_peered":"2024-03-06T07:22:44.379180+0000","last_clean":"2024-03-06T07:22:44.379180+0000","last_became_active":"2024-03-06T07:22:44.371190+0000","last_became_peered":"2024-03-06T07:22:44.371190+0000","last_unstale":"2024-03-06T07:22:44.379180+0000","last_undegraded":"2024-03-06T07:22:44.379180+0000","last_fullsized":"2024-03-06T07:22:44.379180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:32.601916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370567+0000","last_change":"2024-03-06T07:22:35.033263+0000","last_active":"2024-03-06T07:22:44.370567+0000","last_peered":"2024-03-06T07:22:44.370567+0000","last_clean":"2024-03-06T07:22:44.370567+0000","last_became_active":"2024-03-06T07:22:35.033005+0000","last_became_peered":"2024-03-06T07:22:35.033005+0000","last_unstale":"2024-03-06T07:22:44.370567+0000","last_undegraded":"2024-03-06T07:22:44.370567+0000","last_fullsized":"2024-03-06T07:22:44.370567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:00.137617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364936+0000","last_change":"2024-03-06T07:22:34.035735+0000","last_active":"2024-03-06T07:22:44.364936+0000","last_peered":"2024-03-06T07:22:44.364936+0000","last_clean":"2024-03-06T07:22:44.364936+0000","last_became_active":"2024-03-06T07:22:34.035459+0000","last_became_peered":"2024-03-06T07:22:34.035459+0000","last_unstale":"2024-03-06T07:22:44.364936+0000","last_undegraded":"2024-03-06T07:22:44.364936+0000","last_fullsized":"2024-03-06T07:22:44.364936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366886+0000","last_change":"2024-03-06T07:22:44.366886+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366886+0000","last_undegraded":"2024-03-06T07:22:44.366886+0000","last_fullsized":"2024-03-06T07:22:44.366886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382022+0000","last_change":"2024-03-06T07:22:44.377358+0000","last_active":"2024-03-06T07:22:44.382022+0000","last_peered":"2024-03-06T07:22:44.382022+0000","last_clean":"2024-03-06T07:22:44.382022+0000","last_became_active":"2024-03-06T07:22:44.377098+0000","last_became_peered":"2024-03-06T07:22:44.377098+0000","last_unstale":"2024-03-06T07:22:44.382022+0000","last_undegraded":"2024-03-06T07:22:44.382022+0000","last_fullsized":"2024-03-06T07:22:44.382022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:43.869151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363978+0000","last_change":"2024-03-06T07:22:35.038111+0000","last_active":"2024-03-06T07:22:44.363978+0000","last_peered":"2024-03-06T07:22:44.363978+0000","last_clean":"2024-03-06T07:22:44.363978+0000","last_became_active":"2024-03-06T07:22:35.037779+0000","last_became_peered":"2024-03-06T07:22:35.037779+0000","last_unstale":"2024-03-06T07:22:44.363978+0000","last_undegraded":"2024-03-06T07:22:44.363978+0000","last_fullsized":"2024-03-06T07:22:44.363978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.362629+0000","last_change":"2024-03-06T07:22:34.039910+0000","last_active":"2024-03-06T07:22:44.362629+0000","last_peered":"2024-03-06T07:22:44.362629+0000","last_clean":"2024-03-06T07:22:44.362629+0000","last_became_active":"2024-03-06T07:22:34.039246+0000","last_became_peered":"2024-03-06T07:22:34.039246+0000","last_unstale":"2024-03-06T07:22:44.362629+0000","last_undegraded":"2024-03-06T07:22:44.362629+0000","last_fullsized":"2024-03-06T07:22:44.362629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:41.961694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362779+0000","last_change":"2024-03-06T07:22:44.362779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362779+0000","last_undegraded":"2024-03-06T07:22:44.362779+0000","last_fullsized":"2024-03-06T07:22:44.362779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379240+0000","last_change":"2024-03-06T07:22:44.374437+0000","last_active":"2024-03-06T07:22:44.379240+0000","last_peered":"2024-03-06T07:22:44.379240+0000","last_clean":"2024-03-06T07:22:44.379240+0000","last_became_active":"2024-03-06T07:22:44.374055+0000","last_became_peered":"2024-03-06T07:22:44.374055+0000","last_unstale":"2024-03-06T07:22:44.379240+0000","last_undegraded":"2024-03-06T07:22:44.379240+0000","last_fullsized":"2024-03-06T07:22:44.379240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:53:05.282920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363282+0000","last_change":"2024-03-06T07:22:35.027657+0000","last_active":"2024-03-06T07:22:44.363282+0000","last_peered":"2024-03-06T07:22:44.363282+0000","last_clean":"2024-03-06T07:22:44.363282+0000","last_became_active":"2024-03-06T07:22:35.027404+0000","last_became_peered":"2024-03-06T07:22:35.027404+0000","last_unstale":"2024-03-06T07:22:44.363282+0000","last_undegraded":"2024-03-06T07:22:44.363282+0000","last_fullsized":"2024-03-06T07:22:44.363282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:34.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364859+0000","last_change":"2024-03-06T07:22:34.039209+0000","last_active":"2024-03-06T07:22:44.364859+0000","last_peered":"2024-03-06T07:22:44.364859+0000","last_clean":"2024-03-06T07:22:44.364859+0000","last_became_active":"2024-03-06T07:22:34.038921+0000","last_became_peered":"2024-03-06T07:22:34.038921+0000","last_unstale":"2024-03-06T07:22:44.364859+0000","last_undegraded":"2024-03-06T07:22:44.364859+0000","last_fullsized":"2024-03-06T07:22:44.364859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:12.359435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363182+0000","last_change":"2024-03-06T07:22:44.363182+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363182+0000","last_undegraded":"2024-03-06T07:22:44.363182+0000","last_fullsized":"2024-03-06T07:22:44.363182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376720+0000","last_change":"2024-03-06T07:22:44.374940+0000","last_active":"2024-03-06T07:22:44.376720+0000","last_peered":"2024-03-06T07:22:44.376720+0000","last_clean":"2024-03-06T07:22:44.376720+0000","last_became_active":"2024-03-06T07:22:44.374653+0000","last_became_peered":"2024-03-06T07:22:44.374653+0000","last_unstale":"2024-03-06T07:22:44.376720+0000","last_undegraded":"2024-03-06T07:22:44.376720+0000","last_fullsized":"2024-03-06T07:22:44.376720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359439+0000","last_change":"2024-03-06T07:22:35.038010+0000","last_active":"2024-03-06T07:22:44.359439+0000","last_peered":"2024-03-06T07:22:44.359439+0000","last_clean":"2024-03-06T07:22:44.359439+0000","last_became_active":"2024-03-06T07:22:35.037717+0000","last_became_peered":"2024-03-06T07:22:35.037717+0000","last_unstale":"2024-03-06T07:22:44.359439+0000","last_undegraded":"2024-03-06T07:22:44.359439+0000","last_fullsized":"2024-03-06T07:22:44.359439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365513+0000","last_change":"2024-03-06T07:22:34.039679+0000","last_active":"2024-03-06T07:22:44.365513+0000","last_peered":"2024-03-06T07:22:44.365513+0000","last_clean":"2024-03-06T07:22:44.365513+0000","last_became_active":"2024-03-06T07:22:34.039092+0000","last_became_peered":"2024-03-06T07:22:34.039092+0000","last_unstale":"2024-03-06T07:22:44.365513+0000","last_undegraded":"2024-03-06T07:22:44.365513+0000","last_fullsized":"2024-03-06T07:22:44.365513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.370845+0000","last_change":"2024-03-06T07:22:44.370845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.370845+0000","last_undegraded":"2024-03-06T07:22:44.370845+0000","last_fullsized":"2024-03-06T07:22:44.370845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373582+0000","last_change":"2024-03-06T07:22:44.369327+0000","last_active":"2024-03-06T07:22:44.373582+0000","last_peered":"2024-03-06T07:22:44.373582+0000","last_clean":"2024-03-06T07:22:44.373582+0000","last_became_active":"2024-03-06T07:22:44.369076+0000","last_became_peered":"2024-03-06T07:22:44.369076+0000","last_unstale":"2024-03-06T07:22:44.373582+0000","last_undegraded":"2024-03-06T07:22:44.373582+0000","last_fullsized":"2024-03-06T07:22:44.373582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:52.087902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359257+0000","last_change":"2024-03-06T07:22:35.026900+0000","last_active":"2024-03-06T07:22:44.359257+0000","last_peered":"2024-03-06T07:22:44.359257+0000","last_clean":"2024-03-06T07:22:44.359257+0000","last_became_active":"2024-03-06T07:22:35.026626+0000","last_became_peered":"2024-03-06T07:22:35.026626+0000","last_unstale":"2024-03-06T07:22:44.359257+0000","last_undegraded":"2024-03-06T07:22:44.359257+0000","last_fullsized":"2024-03-06T07:22:44.359257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358578+0000","last_change":"2024-03-06T07:22:34.039382+0000","last_active":"2024-03-06T07:22:44.358578+0000","last_peered":"2024-03-06T07:22:44.358578+0000","last_clean":"2024-03-06T07:22:44.358578+0000","last_became_active":"2024-03-06T07:22:34.039088+0000","last_became_peered":"2024-03-06T07:22:34.039088+0000","last_unstale":"2024-03-06T07:22:44.358578+0000","last_undegraded":"2024-03-06T07:22:44.358578+0000","last_fullsized":"2024-03-06T07:22:44.358578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363505+0000","last_change":"2024-03-06T07:22:44.363505+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363505+0000","last_undegraded":"2024-03-06T07:22:44.363505+0000","last_fullsized":"2024-03-06T07:22:44.363505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375571+0000","last_change":"2024-03-06T07:22:44.374409+0000","last_active":"2024-03-06T07:22:44.375571+0000","last_peered":"2024-03-06T07:22:44.375571+0000","last_clean":"2024-03-06T07:22:44.375571+0000","last_became_active":"2024-03-06T07:22:44.374130+0000","last_became_peered":"2024-03-06T07:22:44.374130+0000","last_unstale":"2024-03-06T07:22:44.375571+0000","last_undegraded":"2024-03-06T07:22:44.375571+0000","last_fullsized":"2024-03-06T07:22:44.375571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:30.597343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368651+0000","last_change":"2024-03-06T07:22:35.033671+0000","last_active":"2024-03-06T07:22:44.368651+0000","last_peered":"2024-03-06T07:22:44.368651+0000","last_clean":"2024-03-06T07:22:44.368651+0000","last_became_active":"2024-03-06T07:22:35.033167+0000","last_became_peered":"2024-03-06T07:22:35.033167+0000","last_unstale":"2024-03-06T07:22:44.368651+0000","last_undegraded":"2024-03-06T07:22:44.368651+0000","last_fullsized":"2024-03-06T07:22:44.368651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365069+0000","last_change":"2024-03-06T07:22:34.037026+0000","last_active":"2024-03-06T07:22:44.365069+0000","last_peered":"2024-03-06T07:22:44.365069+0000","last_clean":"2024-03-06T07:22:44.365069+0000","last_became_active":"2024-03-06T07:22:34.036742+0000","last_became_peered":"2024-03-06T07:22:34.036742+0000","last_unstale":"2024-03-06T07:22:44.365069+0000","last_undegraded":"2024-03-06T07:22:44.365069+0000","last_fullsized":"2024-03-06T07:22:44.365069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363945+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:22:44.363945+0000","last_peered":"2024-03-06T07:22:44.363945+0000","last_clean":"2024-03-06T07:22:44.363945+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:22:44.363945+0000","last_undegraded":"2024-03-06T07:22:44.363945+0000","last_fullsized":"2024-03-06T07:22:44.363945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":1032479,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":443701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":442123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":443805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":435149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":442831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":442201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":1032457,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:48.431 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:48.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.563+0000 7fc7db1e9640 1 Processor -- start 2024-03-06T07:22:48.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.563+0000 7fc7db1e9640 1 -- start start 2024-03-06T07:22:48.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7db1e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7d4155ba0 0x7fc7d40c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7db1e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d41586f0 0x7fc7d4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7db1e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d41591c0 con 0x7fc7d40c5480 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7db1e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d406df40 con 0x7fc7d4155ba0 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7db1e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d406fdb0 con 0x7fc7d41586f0 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d8f85640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc7d40c5480 0x7fc7d406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57514/0 (socket says 172.21.15.26:57514) 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d8f85640 1 -- 172.21.15.26:0/3407414111 learned_addr learned my addr 172.21.15.26:0/3407414111 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d9786640 1 --2- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d41586f0 0x7fc7d4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d3fff640 1 --2- 172.21.15.26:0/3407414111 >> v2:172.21.15.117:3300/0 conn(0x7fc7d4155ba0 0x7fc7d40c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d37fe640 1 -- 172.21.15.26:0/3407414111 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281906171 0 0) 0x7fc7d41591c0 con 0x7fc7d40c5480 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d37fe640 1 -- 172.21.15.26:0/3407414111 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c4003520 con 0x7fc7d40c5480 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d3fff640 1 -- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d41586f0 msgr2=0x7fc7d4158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d3fff640 1 --2- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d41586f0 0x7fc7d4158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d3fff640 1 -- 172.21.15.26:0/3407414111 >> v1:172.21.15.26:6789/0 conn(0x7fc7d40c5480 legacy=0x7fc7d406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d3fff640 1 -- 172.21.15.26:0/3407414111 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d41594a0 con 0x7fc7d4155ba0 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.564+0000 7fc7d9786640 1 --2- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d41586f0 0x7fc7d4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.565+0000 7fc7d3fff640 1 --2- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d4155ba0 0x7fc7d40c5860 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00097a0 tx=0x7fc7c002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d0effb3532f8651 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.565+0000 7fc7d37fe640 1 -- 172.21.15.26:0/3407414111 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00315a0 con 0x7fc7d4155ba0 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.565+0000 7fc7d37fe640 1 -- 172.21.15.26:0/3407414111 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0037070 con 0x7fc7d4155ba0 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.565+0000 7fc7d37fe640 1 -- 172.21.15.26:0/3407414111 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0031960 con 0x7fc7d4155ba0 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d4155ba0 msgr2=0x7fc7d40c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d4155ba0 0x7fc7d40c5860 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00097a0 tx=0x7fc7c002b840 comp rx=0 tx=0).stop 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3407414111 shutdown_connections 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d4155ba0 0x7fc7d40c5860 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3407414111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d41586f0 0x7fc7d4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3407414111 >> 172.21.15.26:0/3407414111 conn(0x7fc7d406b1a0 msgr2=0x7fc7d406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3407414111 shutdown_connections 2024-03-06T07:22:48.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.566+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3407414111 wait complete. 2024-03-06T07:22:48.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.567+0000 7fc7db1e9640 1 Processor -- start 2024-03-06T07:22:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.567+0000 7fc7db1e9640 1 -- start start 2024-03-06T07:22:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7db1e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40c5480 0x7fc7d41ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7db1e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d4155ba0 0x7fc7d40ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7db1e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d41586f0 0x7fc7d41ded70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7db1e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4155680 con 0x7fc7d41586f0 2024-03-06T07:22:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7db1e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4156000 con 0x7fc7d40c5480 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7db1e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4152a20 con 0x7fc7d4155ba0 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d41586f0 0x7fc7d41ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d8f85640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40c5480 0x7fc7d41ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d41586f0 0x7fc7d41ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49178/0 (socket says 172.21.15.26:49178) 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 -- 172.21.15.26:0/3613886570 learned_addr learned my addr 172.21.15.26:0/3613886570 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d4155ba0 0x7fc7d40ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 -- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d4155ba0 msgr2=0x7fc7d40ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d4155ba0 0x7fc7d40ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 -- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40c5480 msgr2=0x7fc7d41ebbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40c5480 0x7fc7d41ebbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 -- 172.21.15.26:0/3613886570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d41dfc90 con 0x7fc7d41586f0 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d8f85640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40c5480 0x7fc7d41ebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.568+0000 7fc7d9786640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d41586f0 0x7fc7d41ded70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009db0 tx=0x7fc7cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.569+0000 7fc7d17fa640 1 -- 172.21.15.26:0/3613886570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc018030 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.569+0000 7fc7d17fa640 1 -- 172.21.15.26:0/3613886570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc00cd30 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.569+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d41dff70 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.569+0000 7fc7d17fa640 1 -- 172.21.15.26:0/3613886570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc003bb0 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.569+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d40acd60 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.570+0000 7fc7d17fa640 1 -- 172.21.15.26:0/3613886570 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7cc01b070 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.570+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d404f330 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.571+0000 7fc7d17fa640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc7b806b4d0 0x7fc7b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.571+0000 7fc7d17fa640 1 -- 172.21.15.26:0/3613886570 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fc7cc08adf0 con 0x7fc7d41586f0 2024-03-06T07:22:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.571+0000 7fc7d8f85640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc7b806b4d0 0x7fc7b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.571+0000 7fc7d8f85640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc7b806b4d0 0x7fc7b806d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005880 tx=0x7fc7c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.575+0000 7fc7d17fa640 1 -- 172.21.15.26:0/3613886570 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc7cc010040 con 0x7fc7d41586f0 2024-03-06T07:22:48.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.752+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7d41dfa40 con 0x7fc7b806b4d0 2024-03-06T07:22:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.763+0000 7fc7d17fa640 1 -- 172.21.15.26:0/3613886570 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485419 (secure 0 0 0) 0x7fc7d41dfa40 con 0x7fc7b806b4d0 2024-03-06T07:22:48.770 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:48.773 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc7b806b4d0 msgr2=0x7fc7b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc7b806b4d0 0x7fc7b806d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005880 tx=0x7fc7c8009040 comp rx=0 tx=0).stop 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d41586f0 msgr2=0x7fc7d41ded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d41586f0 0x7fc7d41ded70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009db0 tx=0x7fc7cc00c490 comp rx=0 tx=0).stop 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 shutdown_connections 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc7b806b4d0 0x7fc7b806d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d41586f0 0x7fc7d41ded70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d4155ba0 0x7fc7d40ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 --2- 172.21.15.26:0/3613886570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40c5480 0x7fc7d41ebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 >> 172.21.15.26:0/3613886570 conn(0x7fc7d406b1a0 msgr2=0x7fc7d40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 shutdown_connections 2024-03-06T07:22:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.767+0000 7fc7db1e9640 1 -- 172.21.15.26:0/3613886570 wait complete. 2024-03-06T07:22:48.780 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-03-06T07:22:47.063892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244432,"kb_used_data":7444,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730288,"statfs":{"total":773094113280,"available":772843814912,"internally_reserved":0,"allocated":7622656,"data_stored":4714746,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000767"},"pg_stats":[{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361902+0000","last_change":"2024-03-06T07:22:44.361902+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361902+0000","last_undegraded":"2024-03-06T07:22:44.361902+0000","last_fullsized":"2024-03-06T07:22:44.361902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365039+0000","last_change":"2024-03-06T07:22:35.025691+0000","last_active":"2024-03-06T07:22:44.365039+0000","last_peered":"2024-03-06T07:22:44.365039+0000","last_clean":"2024-03-06T07:22:44.365039+0000","last_became_active":"2024-03-06T07:22:35.025394+0000","last_became_peered":"2024-03-06T07:22:35.025394+0000","last_unstale":"2024-03-06T07:22:44.365039+0000","last_undegraded":"2024-03-06T07:22:44.365039+0000","last_fullsized":"2024-03-06T07:22:44.365039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366504+0000","last_change":"2024-03-06T07:22:44.366504+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366504+0000","last_undegraded":"2024-03-06T07:22:44.366504+0000","last_fullsized":"2024-03-06T07:22:44.366504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373511+0000","last_change":"2024-03-06T07:22:35.048775+0000","last_active":"2024-03-06T07:22:44.373511+0000","last_peered":"2024-03-06T07:22:44.373511+0000","last_clean":"2024-03-06T07:22:44.373511+0000","last_became_active":"2024-03-06T07:22:35.048476+0000","last_became_peered":"2024-03-06T07:22:35.048476+0000","last_unstale":"2024-03-06T07:22:44.373511+0000","last_undegraded":"2024-03-06T07:22:44.373511+0000","last_fullsized":"2024-03-06T07:22:44.373511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:41.704388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362319+0000","last_change":"2024-03-06T07:22:44.362319+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362319+0000","last_undegraded":"2024-03-06T07:22:44.362319+0000","last_fullsized":"2024-03-06T07:22:44.362319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359053+0000","last_change":"2024-03-06T07:22:35.037727+0000","last_active":"2024-03-06T07:22:44.359053+0000","last_peered":"2024-03-06T07:22:44.359053+0000","last_clean":"2024-03-06T07:22:44.359053+0000","last_became_active":"2024-03-06T07:22:35.037421+0000","last_became_peered":"2024-03-06T07:22:35.037421+0000","last_unstale":"2024-03-06T07:22:44.359053+0000","last_undegraded":"2024-03-06T07:22:44.359053+0000","last_fullsized":"2024-03-06T07:22:44.359053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362908+0000","last_change":"2024-03-06T07:22:44.362908+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362908+0000","last_undegraded":"2024-03-06T07:22:44.362908+0000","last_fullsized":"2024-03-06T07:22:44.362908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365569+0000","last_change":"2024-03-06T07:22:35.035949+0000","last_active":"2024-03-06T07:22:44.365569+0000","last_peered":"2024-03-06T07:22:44.365569+0000","last_clean":"2024-03-06T07:22:44.365569+0000","last_became_active":"2024-03-06T07:22:35.035687+0000","last_became_peered":"2024-03-06T07:22:35.035687+0000","last_unstale":"2024-03-06T07:22:44.365569+0000","last_undegraded":"2024-03-06T07:22:44.365569+0000","last_fullsized":"2024-03-06T07:22:44.365569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362767+0000","last_change":"2024-03-06T07:22:44.362767+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362767+0000","last_undegraded":"2024-03-06T07:22:44.362767+0000","last_fullsized":"2024-03-06T07:22:44.362767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365712+0000","last_change":"2024-03-06T07:22:35.033081+0000","last_active":"2024-03-06T07:22:44.365712+0000","last_peered":"2024-03-06T07:22:44.365712+0000","last_clean":"2024-03-06T07:22:44.365712+0000","last_became_active":"2024-03-06T07:22:35.032799+0000","last_became_peered":"2024-03-06T07:22:35.032799+0000","last_unstale":"2024-03-06T07:22:44.365712+0000","last_undegraded":"2024-03-06T07:22:44.365712+0000","last_fullsized":"2024-03-06T07:22:44.365712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.187775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365929+0000","last_change":"2024-03-06T07:22:44.365929+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365929+0000","last_undegraded":"2024-03-06T07:22:44.365929+0000","last_fullsized":"2024-03-06T07:22:44.365929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364635+0000","last_change":"2024-03-06T07:22:35.041976+0000","last_active":"2024-03-06T07:22:44.364635+0000","last_peered":"2024-03-06T07:22:44.364635+0000","last_clean":"2024-03-06T07:22:44.364635+0000","last_became_active":"2024-03-06T07:22:35.041546+0000","last_became_peered":"2024-03-06T07:22:35.041546+0000","last_unstale":"2024-03-06T07:22:44.364635+0000","last_undegraded":"2024-03-06T07:22:44.364635+0000","last_fullsized":"2024-03-06T07:22:44.364635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360301+0000","last_change":"2024-03-06T07:22:44.360301+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360301+0000","last_undegraded":"2024-03-06T07:22:44.360301+0000","last_fullsized":"2024-03-06T07:22:44.360301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365335+0000","last_change":"2024-03-06T07:22:35.038320+0000","last_active":"2024-03-06T07:22:44.365335+0000","last_peered":"2024-03-06T07:22:44.365335+0000","last_clean":"2024-03-06T07:22:44.365335+0000","last_became_active":"2024-03-06T07:22:35.038043+0000","last_became_peered":"2024-03-06T07:22:35.038043+0000","last_unstale":"2024-03-06T07:22:44.365335+0000","last_undegraded":"2024-03-06T07:22:44.365335+0000","last_fullsized":"2024-03-06T07:22:44.365335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366763+0000","last_change":"2024-03-06T07:22:44.366763+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366763+0000","last_undegraded":"2024-03-06T07:22:44.366763+0000","last_fullsized":"2024-03-06T07:22:44.366763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368165+0000","last_change":"2024-03-06T07:22:35.040651+0000","last_active":"2024-03-06T07:22:44.368165+0000","last_peered":"2024-03-06T07:22:44.368165+0000","last_clean":"2024-03-06T07:22:44.368165+0000","last_became_active":"2024-03-06T07:22:35.040382+0000","last_became_peered":"2024-03-06T07:22:35.040382+0000","last_unstale":"2024-03-06T07:22:44.368165+0000","last_undegraded":"2024-03-06T07:22:44.368165+0000","last_fullsized":"2024-03-06T07:22:44.368165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:54.755446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360342+0000","last_change":"2024-03-06T07:22:44.360342+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360342+0000","last_undegraded":"2024-03-06T07:22:44.360342+0000","last_fullsized":"2024-03-06T07:22:44.360342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361779+0000","last_change":"2024-03-06T07:22:35.035524+0000","last_active":"2024-03-06T07:22:44.361779+0000","last_peered":"2024-03-06T07:22:44.361779+0000","last_clean":"2024-03-06T07:22:44.361779+0000","last_became_active":"2024-03-06T07:22:35.035252+0000","last_became_peered":"2024-03-06T07:22:35.035252+0000","last_unstale":"2024-03-06T07:22:44.361779+0000","last_undegraded":"2024-03-06T07:22:44.361779+0000","last_fullsized":"2024-03-06T07:22:44.361779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364257+0000","last_change":"2024-03-06T07:22:44.364257+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364257+0000","last_undegraded":"2024-03-06T07:22:44.364257+0000","last_fullsized":"2024-03-06T07:22:44.364257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365193+0000","last_change":"2024-03-06T07:22:35.040188+0000","last_active":"2024-03-06T07:22:44.365193+0000","last_peered":"2024-03-06T07:22:44.365193+0000","last_clean":"2024-03-06T07:22:44.365193+0000","last_became_active":"2024-03-06T07:22:35.039868+0000","last_became_peered":"2024-03-06T07:22:35.039868+0000","last_unstale":"2024-03-06T07:22:44.365193+0000","last_undegraded":"2024-03-06T07:22:44.365193+0000","last_fullsized":"2024-03-06T07:22:44.365193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359274+0000","last_change":"2024-03-06T07:22:44.359274+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359274+0000","last_undegraded":"2024-03-06T07:22:44.359274+0000","last_fullsized":"2024-03-06T07:22:44.359274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375229+0000","last_change":"2024-03-06T07:22:35.049739+0000","last_active":"2024-03-06T07:22:44.375229+0000","last_peered":"2024-03-06T07:22:44.375229+0000","last_clean":"2024-03-06T07:22:44.375229+0000","last_became_active":"2024-03-06T07:22:35.049118+0000","last_became_peered":"2024-03-06T07:22:35.049118+0000","last_unstale":"2024-03-06T07:22:44.375229+0000","last_undegraded":"2024-03-06T07:22:44.375229+0000","last_fullsized":"2024-03-06T07:22:44.375229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:49.784740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362088+0000","last_change":"2024-03-06T07:22:44.362088+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362088+0000","last_undegraded":"2024-03-06T07:22:44.362088+0000","last_fullsized":"2024-03-06T07:22:44.362088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365663+0000","last_change":"2024-03-06T07:22:35.038831+0000","last_active":"2024-03-06T07:22:44.365663+0000","last_peered":"2024-03-06T07:22:44.365663+0000","last_clean":"2024-03-06T07:22:44.365663+0000","last_became_active":"2024-03-06T07:22:35.038550+0000","last_became_peered":"2024-03-06T07:22:35.038550+0000","last_unstale":"2024-03-06T07:22:44.365663+0000","last_undegraded":"2024-03-06T07:22:44.365663+0000","last_fullsized":"2024-03-06T07:22:44.365663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362839+0000","last_change":"2024-03-06T07:22:44.362839+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362839+0000","last_undegraded":"2024-03-06T07:22:44.362839+0000","last_fullsized":"2024-03-06T07:22:44.362839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365819+0000","last_change":"2024-03-06T07:22:35.037819+0000","last_active":"2024-03-06T07:22:44.365819+0000","last_peered":"2024-03-06T07:22:44.365819+0000","last_clean":"2024-03-06T07:22:44.365819+0000","last_became_active":"2024-03-06T07:22:35.037519+0000","last_became_peered":"2024-03-06T07:22:35.037519+0000","last_unstale":"2024-03-06T07:22:44.365819+0000","last_undegraded":"2024-03-06T07:22:44.365819+0000","last_fullsized":"2024-03-06T07:22:44.365819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:49.281137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.369795+0000","last_change":"2024-03-06T07:22:44.369795+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.369795+0000","last_undegraded":"2024-03-06T07:22:44.369795+0000","last_fullsized":"2024-03-06T07:22:44.369795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364770+0000","last_change":"2024-03-06T07:22:35.040052+0000","last_active":"2024-03-06T07:22:44.364770+0000","last_peered":"2024-03-06T07:22:44.364770+0000","last_clean":"2024-03-06T07:22:44.364770+0000","last_became_active":"2024-03-06T07:22:35.039794+0000","last_became_peered":"2024-03-06T07:22:35.039794+0000","last_unstale":"2024-03-06T07:22:44.364770+0000","last_undegraded":"2024-03-06T07:22:44.364770+0000","last_fullsized":"2024-03-06T07:22:44.364770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364008+0000","last_change":"2024-03-06T07:22:44.364008+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364008+0000","last_undegraded":"2024-03-06T07:22:44.364008+0000","last_fullsized":"2024-03-06T07:22:44.364008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366659+0000","last_change":"2024-03-06T07:22:35.034108+0000","last_active":"2024-03-06T07:22:44.366659+0000","last_peered":"2024-03-06T07:22:44.366659+0000","last_clean":"2024-03-06T07:22:44.366659+0000","last_became_active":"2024-03-06T07:22:35.033823+0000","last_became_peered":"2024-03-06T07:22:35.033823+0000","last_unstale":"2024-03-06T07:22:44.366659+0000","last_undegraded":"2024-03-06T07:22:44.366659+0000","last_fullsized":"2024-03-06T07:22:44.366659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:04.872840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359980+0000","last_change":"2024-03-06T07:22:44.359980+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359980+0000","last_undegraded":"2024-03-06T07:22:44.359980+0000","last_fullsized":"2024-03-06T07:22:44.359980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363762+0000","last_change":"2024-03-06T07:22:35.037317+0000","last_active":"2024-03-06T07:22:44.363762+0000","last_peered":"2024-03-06T07:22:44.363762+0000","last_clean":"2024-03-06T07:22:44.363762+0000","last_became_active":"2024-03-06T07:22:35.033185+0000","last_became_peered":"2024-03-06T07:22:35.033185+0000","last_unstale":"2024-03-06T07:22:44.363762+0000","last_undegraded":"2024-03-06T07:22:44.363762+0000","last_fullsized":"2024-03-06T07:22:44.363762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359992+0000","last_change":"2024-03-06T07:22:44.359992+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359992+0000","last_undegraded":"2024-03-06T07:22:44.359992+0000","last_fullsized":"2024-03-06T07:22:44.359992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365447+0000","last_change":"2024-03-06T07:22:35.035073+0000","last_active":"2024-03-06T07:22:44.365447+0000","last_peered":"2024-03-06T07:22:44.365447+0000","last_clean":"2024-03-06T07:22:44.365447+0000","last_became_active":"2024-03-06T07:22:35.034739+0000","last_became_peered":"2024-03-06T07:22:35.034739+0000","last_unstale":"2024-03-06T07:22:44.365447+0000","last_undegraded":"2024-03-06T07:22:44.365447+0000","last_fullsized":"2024-03-06T07:22:44.365447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:12.718069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363938+0000","last_change":"2024-03-06T07:22:44.363938+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363938+0000","last_undegraded":"2024-03-06T07:22:44.363938+0000","last_fullsized":"2024-03-06T07:22:44.363938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359224+0000","last_change":"2024-03-06T07:22:35.032202+0000","last_active":"2024-03-06T07:22:44.359224+0000","last_peered":"2024-03-06T07:22:44.359224+0000","last_clean":"2024-03-06T07:22:44.359224+0000","last_became_active":"2024-03-06T07:22:35.031858+0000","last_became_peered":"2024-03-06T07:22:35.031858+0000","last_unstale":"2024-03-06T07:22:44.359224+0000","last_undegraded":"2024-03-06T07:22:44.359224+0000","last_fullsized":"2024-03-06T07:22:44.359224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364868+0000","last_change":"2024-03-06T07:22:44.364868+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364868+0000","last_undegraded":"2024-03-06T07:22:44.364868+0000","last_fullsized":"2024-03-06T07:22:44.364868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364576+0000","last_change":"2024-03-06T07:22:35.041928+0000","last_active":"2024-03-06T07:22:44.364576+0000","last_peered":"2024-03-06T07:22:44.364576+0000","last_clean":"2024-03-06T07:22:44.364576+0000","last_became_active":"2024-03-06T07:22:35.041672+0000","last_became_peered":"2024-03-06T07:22:35.041672+0000","last_unstale":"2024-03-06T07:22:44.364576+0000","last_undegraded":"2024-03-06T07:22:44.364576+0000","last_fullsized":"2024-03-06T07:22:44.364576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362840+0000","last_change":"2024-03-06T07:22:44.362840+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362840+0000","last_undegraded":"2024-03-06T07:22:44.362840+0000","last_fullsized":"2024-03-06T07:22:44.362840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368275+0000","last_change":"2024-03-06T07:22:35.036178+0000","last_active":"2024-03-06T07:22:44.368275+0000","last_peered":"2024-03-06T07:22:44.368275+0000","last_clean":"2024-03-06T07:22:44.368275+0000","last_became_active":"2024-03-06T07:22:35.035874+0000","last_became_peered":"2024-03-06T07:22:35.035874+0000","last_unstale":"2024-03-06T07:22:44.368275+0000","last_undegraded":"2024-03-06T07:22:44.368275+0000","last_fullsized":"2024-03-06T07:22:44.368275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.764602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364726+0000","last_change":"2024-03-06T07:22:44.364726+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364726+0000","last_undegraded":"2024-03-06T07:22:44.364726+0000","last_fullsized":"2024-03-06T07:22:44.364726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361583+0000","last_change":"2024-03-06T07:22:35.031787+0000","last_active":"2024-03-06T07:22:44.361583+0000","last_peered":"2024-03-06T07:22:44.361583+0000","last_clean":"2024-03-06T07:22:44.361583+0000","last_became_active":"2024-03-06T07:22:35.031524+0000","last_became_peered":"2024-03-06T07:22:35.031524+0000","last_unstale":"2024-03-06T07:22:44.361583+0000","last_undegraded":"2024-03-06T07:22:44.361583+0000","last_fullsized":"2024-03-06T07:22:44.361583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361747+0000","last_change":"2024-03-06T07:22:44.361747+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361747+0000","last_undegraded":"2024-03-06T07:22:44.361747+0000","last_fullsized":"2024-03-06T07:22:44.361747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363506+0000","last_change":"2024-03-06T07:22:35.039311+0000","last_active":"2024-03-06T07:22:44.363506+0000","last_peered":"2024-03-06T07:22:44.363506+0000","last_clean":"2024-03-06T07:22:44.363506+0000","last_became_active":"2024-03-06T07:22:35.038971+0000","last_became_peered":"2024-03-06T07:22:35.038971+0000","last_unstale":"2024-03-06T07:22:44.363506+0000","last_undegraded":"2024-03-06T07:22:44.363506+0000","last_fullsized":"2024-03-06T07:22:44.363506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:42.465427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359016+0000","last_change":"2024-03-06T07:22:44.359016+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359016+0000","last_undegraded":"2024-03-06T07:22:44.359016+0000","last_fullsized":"2024-03-06T07:22:44.359016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361143+0000","last_change":"2024-03-06T07:22:35.031978+0000","last_active":"2024-03-06T07:22:44.361143+0000","last_peered":"2024-03-06T07:22:44.361143+0000","last_clean":"2024-03-06T07:22:44.361143+0000","last_became_active":"2024-03-06T07:22:35.031711+0000","last_became_peered":"2024-03-06T07:22:35.031711+0000","last_unstale":"2024-03-06T07:22:44.361143+0000","last_undegraded":"2024-03-06T07:22:44.361143+0000","last_fullsized":"2024-03-06T07:22:44.361143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:17.004593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359118+0000","last_change":"2024-03-06T07:22:44.359118+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359118+0000","last_undegraded":"2024-03-06T07:22:44.359118+0000","last_fullsized":"2024-03-06T07:22:44.359118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375412+0000","last_change":"2024-03-06T07:22:35.045711+0000","last_active":"2024-03-06T07:22:44.375412+0000","last_peered":"2024-03-06T07:22:44.375412+0000","last_clean":"2024-03-06T07:22:44.375412+0000","last_became_active":"2024-03-06T07:22:35.041242+0000","last_became_peered":"2024-03-06T07:22:35.041242+0000","last_unstale":"2024-03-06T07:22:44.375412+0000","last_undegraded":"2024-03-06T07:22:44.375412+0000","last_fullsized":"2024-03-06T07:22:44.375412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:16.266604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364653+0000","last_change":"2024-03-06T07:22:44.364653+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364653+0000","last_undegraded":"2024-03-06T07:22:44.364653+0000","last_fullsized":"2024-03-06T07:22:44.364653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374041+0000","last_change":"2024-03-06T07:22:35.042301+0000","last_active":"2024-03-06T07:22:44.374041+0000","last_peered":"2024-03-06T07:22:44.374041+0000","last_clean":"2024-03-06T07:22:44.374041+0000","last_became_active":"2024-03-06T07:22:35.041172+0000","last_became_peered":"2024-03-06T07:22:35.041172+0000","last_unstale":"2024-03-06T07:22:44.374041+0000","last_undegraded":"2024-03-06T07:22:44.374041+0000","last_fullsized":"2024-03-06T07:22:44.374041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:58.974636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363518+0000","last_change":"2024-03-06T07:22:44.363518+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363518+0000","last_undegraded":"2024-03-06T07:22:44.363518+0000","last_fullsized":"2024-03-06T07:22:44.363518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365279+0000","last_change":"2024-03-06T07:22:35.038117+0000","last_active":"2024-03-06T07:22:44.365279+0000","last_peered":"2024-03-06T07:22:44.365279+0000","last_clean":"2024-03-06T07:22:44.365279+0000","last_became_active":"2024-03-06T07:22:35.037812+0000","last_became_peered":"2024-03-06T07:22:35.037812+0000","last_unstale":"2024-03-06T07:22:44.365279+0000","last_undegraded":"2024-03-06T07:22:44.365279+0000","last_fullsized":"2024-03-06T07:22:44.365279+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363196+0000","last_change":"2024-03-06T07:22:44.363196+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363196+0000","last_undegraded":"2024-03-06T07:22:44.363196+0000","last_fullsized":"2024-03-06T07:22:44.363196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368368+0000","last_change":"2024-03-06T07:22:35.033425+0000","last_active":"2024-03-06T07:22:44.368368+0000","last_peered":"2024-03-06T07:22:44.368368+0000","last_clean":"2024-03-06T07:22:44.368368+0000","last_became_active":"2024-03-06T07:22:35.032784+0000","last_became_peered":"2024-03-06T07:22:35.032784+0000","last_unstale":"2024-03-06T07:22:44.368368+0000","last_undegraded":"2024-03-06T07:22:44.368368+0000","last_fullsized":"2024-03-06T07:22:44.368368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361026+0000","last_change":"2024-03-06T07:22:44.361026+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361026+0000","last_undegraded":"2024-03-06T07:22:44.361026+0000","last_fullsized":"2024-03-06T07:22:44.361026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368367+0000","last_change":"2024-03-06T07:22:35.033473+0000","last_active":"2024-03-06T07:22:44.368367+0000","last_peered":"2024-03-06T07:22:44.368367+0000","last_clean":"2024-03-06T07:22:44.368367+0000","last_became_active":"2024-03-06T07:22:35.032407+0000","last_became_peered":"2024-03-06T07:22:35.032407+0000","last_unstale":"2024-03-06T07:22:44.368367+0000","last_undegraded":"2024-03-06T07:22:44.368367+0000","last_fullsized":"2024-03-06T07:22:44.368367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:39.792272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363072+0000","last_change":"2024-03-06T07:22:44.363072+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363072+0000","last_undegraded":"2024-03-06T07:22:44.363072+0000","last_fullsized":"2024-03-06T07:22:44.363072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370470+0000","last_change":"2024-03-06T07:22:35.048127+0000","last_active":"2024-03-06T07:22:44.370470+0000","last_peered":"2024-03-06T07:22:44.370470+0000","last_clean":"2024-03-06T07:22:44.370470+0000","last_became_active":"2024-03-06T07:22:35.047869+0000","last_became_peered":"2024-03-06T07:22:35.047869+0000","last_unstale":"2024-03-06T07:22:44.370470+0000","last_undegraded":"2024-03-06T07:22:44.370470+0000","last_fullsized":"2024-03-06T07:22:44.370470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.375194+0000","last_change":"2024-03-06T07:22:44.375194+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.375194+0000","last_undegraded":"2024-03-06T07:22:44.375194+0000","last_fullsized":"2024-03-06T07:22:44.375194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363056+0000","last_change":"2024-03-06T07:22:35.038257+0000","last_active":"2024-03-06T07:22:44.363056+0000","last_peered":"2024-03-06T07:22:44.363056+0000","last_clean":"2024-03-06T07:22:44.363056+0000","last_became_active":"2024-03-06T07:22:35.037969+0000","last_became_peered":"2024-03-06T07:22:35.037969+0000","last_unstale":"2024-03-06T07:22:44.363056+0000","last_undegraded":"2024-03-06T07:22:44.363056+0000","last_fullsized":"2024-03-06T07:22:44.363056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362412+0000","last_change":"2024-03-06T07:22:44.362412+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362412+0000","last_undegraded":"2024-03-06T07:22:44.362412+0000","last_fullsized":"2024-03-06T07:22:44.362412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358712+0000","last_change":"2024-03-06T07:22:35.032129+0000","last_active":"2024-03-06T07:22:44.358712+0000","last_peered":"2024-03-06T07:22:44.358712+0000","last_clean":"2024-03-06T07:22:44.358712+0000","last_became_active":"2024-03-06T07:22:35.027787+0000","last_became_peered":"2024-03-06T07:22:35.027787+0000","last_unstale":"2024-03-06T07:22:44.358712+0000","last_undegraded":"2024-03-06T07:22:44.358712+0000","last_fullsized":"2024-03-06T07:22:44.358712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:27.011065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359560+0000","last_change":"2024-03-06T07:22:44.359560+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359560+0000","last_undegraded":"2024-03-06T07:22:44.359560+0000","last_fullsized":"2024-03-06T07:22:44.359560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363854+0000","last_change":"2024-03-06T07:22:35.036989+0000","last_active":"2024-03-06T07:22:44.363854+0000","last_peered":"2024-03-06T07:22:44.363854+0000","last_clean":"2024-03-06T07:22:44.363854+0000","last_became_active":"2024-03-06T07:22:35.032803+0000","last_became_peered":"2024-03-06T07:22:35.032803+0000","last_unstale":"2024-03-06T07:22:44.363854+0000","last_undegraded":"2024-03-06T07:22:44.363854+0000","last_fullsized":"2024-03-06T07:22:44.363854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365367+0000","last_change":"2024-03-06T07:22:44.365367+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365367+0000","last_undegraded":"2024-03-06T07:22:44.365367+0000","last_fullsized":"2024-03-06T07:22:44.365367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373250+0000","last_change":"2024-03-06T07:22:44.372693+0000","last_active":"2024-03-06T07:22:44.373250+0000","last_peered":"2024-03-06T07:22:44.373250+0000","last_clean":"2024-03-06T07:22:44.373250+0000","last_became_active":"2024-03-06T07:22:44.372375+0000","last_became_peered":"2024-03-06T07:22:44.372375+0000","last_unstale":"2024-03-06T07:22:44.373250+0000","last_undegraded":"2024-03-06T07:22:44.373250+0000","last_fullsized":"2024-03-06T07:22:44.373250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358464+0000","last_change":"2024-03-06T07:22:35.025981+0000","last_active":"2024-03-06T07:22:44.358464+0000","last_peered":"2024-03-06T07:22:44.358464+0000","last_clean":"2024-03-06T07:22:44.358464+0000","last_became_active":"2024-03-06T07:22:35.025721+0000","last_became_peered":"2024-03-06T07:22:35.025721+0000","last_unstale":"2024-03-06T07:22:44.358464+0000","last_undegraded":"2024-03-06T07:22:44.358464+0000","last_fullsized":"2024-03-06T07:22:44.358464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:18.894659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365555+0000","last_change":"2024-03-06T07:22:34.043747+0000","last_active":"2024-03-06T07:22:44.365555+0000","last_peered":"2024-03-06T07:22:44.365555+0000","last_clean":"2024-03-06T07:22:44.365555+0000","last_became_active":"2024-03-06T07:22:34.042367+0000","last_became_peered":"2024-03-06T07:22:34.042367+0000","last_unstale":"2024-03-06T07:22:44.365555+0000","last_undegraded":"2024-03-06T07:22:44.365555+0000","last_fullsized":"2024-03-06T07:22:44.365555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:33.138544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363123+0000","last_change":"2024-03-06T07:22:44.363123+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363123+0000","last_undegraded":"2024-03-06T07:22:44.363123+0000","last_fullsized":"2024-03-06T07:22:44.363123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384852+0000","last_change":"2024-03-06T07:22:44.380429+0000","last_active":"2024-03-06T07:22:44.384852+0000","last_peered":"2024-03-06T07:22:44.384852+0000","last_clean":"2024-03-06T07:22:44.384852+0000","last_became_active":"2024-03-06T07:22:44.380170+0000","last_became_peered":"2024-03-06T07:22:44.380170+0000","last_unstale":"2024-03-06T07:22:44.384852+0000","last_undegraded":"2024-03-06T07:22:44.384852+0000","last_fullsized":"2024-03-06T07:22:44.384852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:53.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370159+0000","last_change":"2024-03-06T07:22:35.032593+0000","last_active":"2024-03-06T07:22:44.370159+0000","last_peered":"2024-03-06T07:22:44.370159+0000","last_clean":"2024-03-06T07:22:44.370159+0000","last_became_active":"2024-03-06T07:22:35.031651+0000","last_became_peered":"2024-03-06T07:22:35.031651+0000","last_unstale":"2024-03-06T07:22:44.370159+0000","last_undegraded":"2024-03-06T07:22:44.370159+0000","last_fullsized":"2024-03-06T07:22:44.370159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364514+0000","last_change":"2024-03-06T07:22:34.038702+0000","last_active":"2024-03-06T07:22:44.364514+0000","last_peered":"2024-03-06T07:22:44.364514+0000","last_clean":"2024-03-06T07:22:44.364514+0000","last_became_active":"2024-03-06T07:22:34.038165+0000","last_became_peered":"2024-03-06T07:22:34.038165+0000","last_unstale":"2024-03-06T07:22:44.364514+0000","last_undegraded":"2024-03-06T07:22:44.364514+0000","last_fullsized":"2024-03-06T07:22:44.364514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362338+0000","last_change":"2024-03-06T07:22:44.362338+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362338+0000","last_undegraded":"2024-03-06T07:22:44.362338+0000","last_fullsized":"2024-03-06T07:22:44.362338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.381941+0000","last_change":"2024-03-06T07:22:44.375256+0000","last_active":"2024-03-06T07:22:44.381941+0000","last_peered":"2024-03-06T07:22:44.381941+0000","last_clean":"2024-03-06T07:22:44.381941+0000","last_became_active":"2024-03-06T07:22:44.374987+0000","last_became_peered":"2024-03-06T07:22:44.374987+0000","last_unstale":"2024-03-06T07:22:44.381941+0000","last_undegraded":"2024-03-06T07:22:44.381941+0000","last_fullsized":"2024-03-06T07:22:44.381941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:32.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361466+0000","last_change":"2024-03-06T07:22:35.035774+0000","last_active":"2024-03-06T07:22:44.361466+0000","last_peered":"2024-03-06T07:22:44.361466+0000","last_clean":"2024-03-06T07:22:44.361466+0000","last_became_active":"2024-03-06T07:22:35.035468+0000","last_became_peered":"2024-03-06T07:22:35.035468+0000","last_unstale":"2024-03-06T07:22:44.361466+0000","last_undegraded":"2024-03-06T07:22:44.361466+0000","last_fullsized":"2024-03-06T07:22:44.361466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:25.144870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368550+0000","last_change":"2024-03-06T07:22:34.042434+0000","last_active":"2024-03-06T07:22:44.368550+0000","last_peered":"2024-03-06T07:22:44.368550+0000","last_clean":"2024-03-06T07:22:44.368550+0000","last_became_active":"2024-03-06T07:22:34.042093+0000","last_became_peered":"2024-03-06T07:22:34.042093+0000","last_unstale":"2024-03-06T07:22:44.368550+0000","last_undegraded":"2024-03-06T07:22:44.368550+0000","last_fullsized":"2024-03-06T07:22:44.368550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364352+0000","last_change":"2024-03-06T07:22:44.364352+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364352+0000","last_undegraded":"2024-03-06T07:22:44.364352+0000","last_fullsized":"2024-03-06T07:22:44.364352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375700+0000","last_change":"2024-03-06T07:22:44.373849+0000","last_active":"2024-03-06T07:22:44.375700+0000","last_peered":"2024-03-06T07:22:44.375700+0000","last_clean":"2024-03-06T07:22:44.375700+0000","last_became_active":"2024-03-06T07:22:44.373135+0000","last_became_peered":"2024-03-06T07:22:44.373135+0000","last_unstale":"2024-03-06T07:22:44.375700+0000","last_undegraded":"2024-03-06T07:22:44.375700+0000","last_fullsized":"2024-03-06T07:22:44.375700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:52.622331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361651+0000","last_change":"2024-03-06T07:22:35.032012+0000","last_active":"2024-03-06T07:22:44.361651+0000","last_peered":"2024-03-06T07:22:44.361651+0000","last_clean":"2024-03-06T07:22:44.361651+0000","last_became_active":"2024-03-06T07:22:35.031728+0000","last_became_peered":"2024-03-06T07:22:35.031728+0000","last_unstale":"2024-03-06T07:22:44.361651+0000","last_undegraded":"2024-03-06T07:22:44.361651+0000","last_fullsized":"2024-03-06T07:22:44.361651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364137+0000","last_change":"2024-03-06T07:22:34.038862+0000","last_active":"2024-03-06T07:22:44.364137+0000","last_peered":"2024-03-06T07:22:44.364137+0000","last_clean":"2024-03-06T07:22:44.364137+0000","last_became_active":"2024-03-06T07:22:34.038590+0000","last_became_peered":"2024-03-06T07:22:34.038590+0000","last_unstale":"2024-03-06T07:22:44.364137+0000","last_undegraded":"2024-03-06T07:22:44.364137+0000","last_fullsized":"2024-03-06T07:22:44.364137+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:15.997587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362381+0000","last_change":"2024-03-06T07:22:44.362381+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362381+0000","last_undegraded":"2024-03-06T07:22:44.362381+0000","last_fullsized":"2024-03-06T07:22:44.362381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376768+0000","last_change":"2024-03-06T07:22:44.375296+0000","last_active":"2024-03-06T07:22:44.376768+0000","last_peered":"2024-03-06T07:22:44.376768+0000","last_clean":"2024-03-06T07:22:44.376768+0000","last_became_active":"2024-03-06T07:22:44.375028+0000","last_became_peered":"2024-03-06T07:22:44.375028+0000","last_unstale":"2024-03-06T07:22:44.376768+0000","last_undegraded":"2024-03-06T07:22:44.376768+0000","last_fullsized":"2024-03-06T07:22:44.376768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.786058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376180+0000","last_change":"2024-03-06T07:22:35.036832+0000","last_active":"2024-03-06T07:22:44.376180+0000","last_peered":"2024-03-06T07:22:44.376180+0000","last_clean":"2024-03-06T07:22:44.376180+0000","last_became_active":"2024-03-06T07:22:35.036392+0000","last_became_peered":"2024-03-06T07:22:35.036392+0000","last_unstale":"2024-03-06T07:22:44.376180+0000","last_undegraded":"2024-03-06T07:22:44.376180+0000","last_fullsized":"2024-03-06T07:22:44.376180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:13.614365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361685+0000","last_change":"2024-03-06T07:22:34.037848+0000","last_active":"2024-03-06T07:22:44.361685+0000","last_peered":"2024-03-06T07:22:44.361685+0000","last_clean":"2024-03-06T07:22:44.361685+0000","last_became_active":"2024-03-06T07:22:34.037572+0000","last_became_peered":"2024-03-06T07:22:34.037572+0000","last_unstale":"2024-03-06T07:22:44.361685+0000","last_undegraded":"2024-03-06T07:22:44.361685+0000","last_fullsized":"2024-03-06T07:22:44.361685+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:15.748729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364294+0000","last_change":"2024-03-06T07:22:44.364294+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364294+0000","last_undegraded":"2024-03-06T07:22:44.364294+0000","last_fullsized":"2024-03-06T07:22:44.364294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379327+0000","last_change":"2024-03-06T07:22:44.375707+0000","last_active":"2024-03-06T07:22:44.379327+0000","last_peered":"2024-03-06T07:22:44.379327+0000","last_clean":"2024-03-06T07:22:44.379327+0000","last_became_active":"2024-03-06T07:22:44.375510+0000","last_became_peered":"2024-03-06T07:22:44.375510+0000","last_unstale":"2024-03-06T07:22:44.379327+0000","last_undegraded":"2024-03-06T07:22:44.379327+0000","last_fullsized":"2024-03-06T07:22:44.379327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:57.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375276+0000","last_change":"2024-03-06T07:22:35.040723+0000","last_active":"2024-03-06T07:22:44.375276+0000","last_peered":"2024-03-06T07:22:44.375276+0000","last_clean":"2024-03-06T07:22:44.375276+0000","last_became_active":"2024-03-06T07:22:35.036544+0000","last_became_peered":"2024-03-06T07:22:35.036544+0000","last_unstale":"2024-03-06T07:22:44.375276+0000","last_undegraded":"2024-03-06T07:22:44.375276+0000","last_fullsized":"2024-03-06T07:22:44.375276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:44.185335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366731+0000","last_change":"2024-03-06T07:22:34.040746+0000","last_active":"2024-03-06T07:22:44.366731+0000","last_peered":"2024-03-06T07:22:44.366731+0000","last_clean":"2024-03-06T07:22:44.366731+0000","last_became_active":"2024-03-06T07:22:34.040466+0000","last_became_peered":"2024-03-06T07:22:34.040466+0000","last_unstale":"2024-03-06T07:22:44.366731+0000","last_undegraded":"2024-03-06T07:22:44.366731+0000","last_fullsized":"2024-03-06T07:22:44.366731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:35.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367578+0000","last_change":"2024-03-06T07:22:44.367578+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367578+0000","last_undegraded":"2024-03-06T07:22:44.367578+0000","last_fullsized":"2024-03-06T07:22:44.367578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384804+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384804+0000","last_peered":"2024-03-06T07:22:44.384804+0000","last_clean":"2024-03-06T07:22:44.384804+0000","last_became_active":"2024-03-06T07:22:44.380134+0000","last_became_peered":"2024-03-06T07:22:44.380134+0000","last_unstale":"2024-03-06T07:22:44.384804+0000","last_undegraded":"2024-03-06T07:22:44.384804+0000","last_fullsized":"2024-03-06T07:22:44.384804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:13.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361680+0000","last_change":"2024-03-06T07:22:35.035207+0000","last_active":"2024-03-06T07:22:44.361680+0000","last_peered":"2024-03-06T07:22:44.361680+0000","last_clean":"2024-03-06T07:22:44.361680+0000","last_became_active":"2024-03-06T07:22:35.034887+0000","last_became_peered":"2024-03-06T07:22:35.034887+0000","last_unstale":"2024-03-06T07:22:44.361680+0000","last_undegraded":"2024-03-06T07:22:44.361680+0000","last_fullsized":"2024-03-06T07:22:44.361680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370135+0000","last_change":"2024-03-06T07:22:34.042833+0000","last_active":"2024-03-06T07:22:44.370135+0000","last_peered":"2024-03-06T07:22:44.370135+0000","last_clean":"2024-03-06T07:22:44.370135+0000","last_became_active":"2024-03-06T07:22:34.042557+0000","last_became_peered":"2024-03-06T07:22:34.042557+0000","last_unstale":"2024-03-06T07:22:44.370135+0000","last_undegraded":"2024-03-06T07:22:44.370135+0000","last_fullsized":"2024-03-06T07:22:44.370135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.372269+0000","last_change":"2024-03-06T07:22:44.372269+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.372269+0000","last_undegraded":"2024-03-06T07:22:44.372269+0000","last_fullsized":"2024-03-06T07:22:44.372269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379060+0000","last_change":"2024-03-06T07:22:44.371322+0000","last_active":"2024-03-06T07:22:44.379060+0000","last_peered":"2024-03-06T07:22:44.379060+0000","last_clean":"2024-03-06T07:22:44.379060+0000","last_became_active":"2024-03-06T07:22:44.371058+0000","last_became_peered":"2024-03-06T07:22:44.371058+0000","last_unstale":"2024-03-06T07:22:44.379060+0000","last_undegraded":"2024-03-06T07:22:44.379060+0000","last_fullsized":"2024-03-06T07:22:44.379060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:25.531656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364553+0000","last_change":"2024-03-06T07:22:35.028293+0000","last_active":"2024-03-06T07:22:44.364553+0000","last_peered":"2024-03-06T07:22:44.364553+0000","last_clean":"2024-03-06T07:22:44.364553+0000","last_became_active":"2024-03-06T07:22:35.028017+0000","last_became_peered":"2024-03-06T07:22:35.028017+0000","last_unstale":"2024-03-06T07:22:44.364553+0000","last_undegraded":"2024-03-06T07:22:44.364553+0000","last_fullsized":"2024-03-06T07:22:44.364553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:36.890851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369543+0000","last_change":"2024-03-06T07:22:34.037660+0000","last_active":"2024-03-06T07:22:44.369543+0000","last_peered":"2024-03-06T07:22:44.369543+0000","last_clean":"2024-03-06T07:22:44.369543+0000","last_became_active":"2024-03-06T07:22:34.037347+0000","last_became_peered":"2024-03-06T07:22:34.037347+0000","last_unstale":"2024-03-06T07:22:44.369543+0000","last_undegraded":"2024-03-06T07:22:44.369543+0000","last_fullsized":"2024-03-06T07:22:44.369543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:43.483410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362846+0000","last_change":"2024-03-06T07:22:44.362846+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362846+0000","last_undegraded":"2024-03-06T07:22:44.362846+0000","last_fullsized":"2024-03-06T07:22:44.362846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374719+0000","last_change":"2024-03-06T07:22:44.370568+0000","last_active":"2024-03-06T07:22:44.374719+0000","last_peered":"2024-03-06T07:22:44.374719+0000","last_clean":"2024-03-06T07:22:44.374719+0000","last_became_active":"2024-03-06T07:22:44.370050+0000","last_became_peered":"2024-03-06T07:22:44.370050+0000","last_unstale":"2024-03-06T07:22:44.374719+0000","last_undegraded":"2024-03-06T07:22:44.374719+0000","last_fullsized":"2024-03-06T07:22:44.374719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:46.225047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375512+0000","last_change":"2024-03-06T07:22:35.045241+0000","last_active":"2024-03-06T07:22:44.375512+0000","last_peered":"2024-03-06T07:22:44.375512+0000","last_clean":"2024-03-06T07:22:44.375512+0000","last_became_active":"2024-03-06T07:22:35.044953+0000","last_became_peered":"2024-03-06T07:22:35.044953+0000","last_unstale":"2024-03-06T07:22:44.375512+0000","last_undegraded":"2024-03-06T07:22:44.375512+0000","last_fullsized":"2024-03-06T07:22:44.375512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365128+0000","last_change":"2024-03-06T07:22:34.038501+0000","last_active":"2024-03-06T07:22:44.365128+0000","last_peered":"2024-03-06T07:22:44.365128+0000","last_clean":"2024-03-06T07:22:44.365128+0000","last_became_active":"2024-03-06T07:22:34.038238+0000","last_became_peered":"2024-03-06T07:22:34.038238+0000","last_unstale":"2024-03-06T07:22:44.365128+0000","last_undegraded":"2024-03-06T07:22:44.365128+0000","last_fullsized":"2024-03-06T07:22:44.365128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363829+0000","last_change":"2024-03-06T07:22:44.363829+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363829+0000","last_undegraded":"2024-03-06T07:22:44.363829+0000","last_fullsized":"2024-03-06T07:22:44.363829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379031+0000","last_change":"2024-03-06T07:22:44.377341+0000","last_active":"2024-03-06T07:22:44.379031+0000","last_peered":"2024-03-06T07:22:44.379031+0000","last_clean":"2024-03-06T07:22:44.379031+0000","last_became_active":"2024-03-06T07:22:44.377070+0000","last_became_peered":"2024-03-06T07:22:44.377070+0000","last_unstale":"2024-03-06T07:22:44.379031+0000","last_undegraded":"2024-03-06T07:22:44.379031+0000","last_fullsized":"2024-03-06T07:22:44.379031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:28.968579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365969+0000","last_change":"2024-03-06T07:22:35.035913+0000","last_active":"2024-03-06T07:22:44.365969+0000","last_peered":"2024-03-06T07:22:44.365969+0000","last_clean":"2024-03-06T07:22:44.365969+0000","last_became_active":"2024-03-06T07:22:35.035658+0000","last_became_peered":"2024-03-06T07:22:35.035658+0000","last_unstale":"2024-03-06T07:22:44.365969+0000","last_undegraded":"2024-03-06T07:22:44.365969+0000","last_fullsized":"2024-03-06T07:22:44.365969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:55.082141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365373+0000","last_change":"2024-03-06T07:22:34.040689+0000","last_active":"2024-03-06T07:22:44.365373+0000","last_peered":"2024-03-06T07:22:44.365373+0000","last_clean":"2024-03-06T07:22:44.365373+0000","last_became_active":"2024-03-06T07:22:34.040410+0000","last_became_peered":"2024-03-06T07:22:34.040410+0000","last_unstale":"2024-03-06T07:22:44.365373+0000","last_undegraded":"2024-03-06T07:22:44.365373+0000","last_fullsized":"2024-03-06T07:22:44.365373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.281130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361564+0000","last_change":"2024-03-06T07:22:44.361564+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361564+0000","last_undegraded":"2024-03-06T07:22:44.361564+0000","last_fullsized":"2024-03-06T07:22:44.361564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384844+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384844+0000","last_peered":"2024-03-06T07:22:44.384844+0000","last_clean":"2024-03-06T07:22:44.384844+0000","last_became_active":"2024-03-06T07:22:44.380123+0000","last_became_peered":"2024-03-06T07:22:44.380123+0000","last_unstale":"2024-03-06T07:22:44.384844+0000","last_undegraded":"2024-03-06T07:22:44.384844+0000","last_fullsized":"2024-03-06T07:22:44.384844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:59.118936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364927+0000","last_change":"2024-03-06T07:22:35.032615+0000","last_active":"2024-03-06T07:22:44.364927+0000","last_peered":"2024-03-06T07:22:44.364927+0000","last_clean":"2024-03-06T07:22:44.364927+0000","last_became_active":"2024-03-06T07:22:35.031167+0000","last_became_peered":"2024-03-06T07:22:35.031167+0000","last_unstale":"2024-03-06T07:22:44.364927+0000","last_undegraded":"2024-03-06T07:22:44.364927+0000","last_fullsized":"2024-03-06T07:22:44.364927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369335+0000","last_change":"2024-03-06T07:22:34.038406+0000","last_active":"2024-03-06T07:22:44.369335+0000","last_peered":"2024-03-06T07:22:44.369335+0000","last_clean":"2024-03-06T07:22:44.369335+0000","last_became_active":"2024-03-06T07:22:34.038040+0000","last_became_peered":"2024-03-06T07:22:34.038040+0000","last_unstale":"2024-03-06T07:22:44.369335+0000","last_undegraded":"2024-03-06T07:22:44.369335+0000","last_fullsized":"2024-03-06T07:22:44.369335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:26.063950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359017+0000","last_change":"2024-03-06T07:22:44.359017+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359017+0000","last_undegraded":"2024-03-06T07:22:44.359017+0000","last_fullsized":"2024-03-06T07:22:44.359017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379693+0000","last_change":"2024-03-06T07:22:44.377976+0000","last_active":"2024-03-06T07:22:44.379693+0000","last_peered":"2024-03-06T07:22:44.379693+0000","last_clean":"2024-03-06T07:22:44.379693+0000","last_became_active":"2024-03-06T07:22:44.377659+0000","last_became_peered":"2024-03-06T07:22:44.377659+0000","last_unstale":"2024-03-06T07:22:44.379693+0000","last_undegraded":"2024-03-06T07:22:44.379693+0000","last_fullsized":"2024-03-06T07:22:44.379693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:42.020403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359010+0000","last_change":"2024-03-06T07:22:35.038459+0000","last_active":"2024-03-06T07:22:44.359010+0000","last_peered":"2024-03-06T07:22:44.359010+0000","last_clean":"2024-03-06T07:22:44.359010+0000","last_became_active":"2024-03-06T07:22:35.038185+0000","last_became_peered":"2024-03-06T07:22:35.038185+0000","last_unstale":"2024-03-06T07:22:44.359010+0000","last_undegraded":"2024-03-06T07:22:44.359010+0000","last_fullsized":"2024-03-06T07:22:44.359010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364863+0000","last_change":"2024-03-06T07:22:34.037962+0000","last_active":"2024-03-06T07:22:44.364863+0000","last_peered":"2024-03-06T07:22:44.364863+0000","last_clean":"2024-03-06T07:22:44.364863+0000","last_became_active":"2024-03-06T07:22:34.037635+0000","last_became_peered":"2024-03-06T07:22:34.037635+0000","last_unstale":"2024-03-06T07:22:44.364863+0000","last_undegraded":"2024-03-06T07:22:44.364863+0000","last_fullsized":"2024-03-06T07:22:44.364863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362391+0000","last_change":"2024-03-06T07:22:44.362391+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362391+0000","last_undegraded":"2024-03-06T07:22:44.362391+0000","last_fullsized":"2024-03-06T07:22:44.362391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382003+0000","last_change":"2024-03-06T07:22:44.377361+0000","last_active":"2024-03-06T07:22:44.382003+0000","last_peered":"2024-03-06T07:22:44.382003+0000","last_clean":"2024-03-06T07:22:44.382003+0000","last_became_active":"2024-03-06T07:22:44.377101+0000","last_became_peered":"2024-03-06T07:22:44.377101+0000","last_unstale":"2024-03-06T07:22:44.382003+0000","last_undegraded":"2024-03-06T07:22:44.382003+0000","last_fullsized":"2024-03-06T07:22:44.382003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:13.917348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365752+0000","last_change":"2024-03-06T07:22:35.036674+0000","last_active":"2024-03-06T07:22:44.365752+0000","last_peered":"2024-03-06T07:22:44.365752+0000","last_clean":"2024-03-06T07:22:44.365752+0000","last_became_active":"2024-03-06T07:22:35.036400+0000","last_became_peered":"2024-03-06T07:22:35.036400+0000","last_unstale":"2024-03-06T07:22:44.365752+0000","last_undegraded":"2024-03-06T07:22:44.365752+0000","last_fullsized":"2024-03-06T07:22:44.365752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365049+0000","last_change":"2024-03-06T07:22:34.041311+0000","last_active":"2024-03-06T07:22:44.365049+0000","last_peered":"2024-03-06T07:22:44.365049+0000","last_clean":"2024-03-06T07:22:44.365049+0000","last_became_active":"2024-03-06T07:22:34.040940+0000","last_became_peered":"2024-03-06T07:22:34.040940+0000","last_unstale":"2024-03-06T07:22:44.365049+0000","last_undegraded":"2024-03-06T07:22:44.365049+0000","last_fullsized":"2024-03-06T07:22:44.365049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:24.924844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362009+0000","last_change":"2024-03-06T07:22:44.362009+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362009+0000","last_undegraded":"2024-03-06T07:22:44.362009+0000","last_fullsized":"2024-03-06T07:22:44.362009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379734+0000","last_change":"2024-03-06T07:22:44.378302+0000","last_active":"2024-03-06T07:22:44.379734+0000","last_peered":"2024-03-06T07:22:44.379734+0000","last_clean":"2024-03-06T07:22:44.379734+0000","last_became_active":"2024-03-06T07:22:44.377994+0000","last_became_peered":"2024-03-06T07:22:44.377994+0000","last_unstale":"2024-03-06T07:22:44.379734+0000","last_undegraded":"2024-03-06T07:22:44.379734+0000","last_fullsized":"2024-03-06T07:22:44.379734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:35.747227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363188+0000","last_change":"2024-03-06T07:22:35.022953+0000","last_active":"2024-03-06T07:22:44.363188+0000","last_peered":"2024-03-06T07:22:44.363188+0000","last_clean":"2024-03-06T07:22:44.363188+0000","last_became_active":"2024-03-06T07:22:35.022681+0000","last_became_peered":"2024-03-06T07:22:35.022681+0000","last_unstale":"2024-03-06T07:22:44.363188+0000","last_undegraded":"2024-03-06T07:22:44.363188+0000","last_fullsized":"2024-03-06T07:22:44.363188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363145+0000","last_change":"2024-03-06T07:22:34.037840+0000","last_active":"2024-03-06T07:22:44.363145+0000","last_peered":"2024-03-06T07:22:44.363145+0000","last_clean":"2024-03-06T07:22:44.363145+0000","last_became_active":"2024-03-06T07:22:34.037556+0000","last_became_peered":"2024-03-06T07:22:34.037556+0000","last_unstale":"2024-03-06T07:22:44.363145+0000","last_undegraded":"2024-03-06T07:22:44.363145+0000","last_fullsized":"2024-03-06T07:22:44.363145+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359660+0000","last_change":"2024-03-06T07:22:44.359660+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359660+0000","last_undegraded":"2024-03-06T07:22:44.359660+0000","last_fullsized":"2024-03-06T07:22:44.359660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379220+0000","last_change":"2024-03-06T07:22:44.372722+0000","last_active":"2024-03-06T07:22:44.379220+0000","last_peered":"2024-03-06T07:22:44.379220+0000","last_clean":"2024-03-06T07:22:44.379220+0000","last_became_active":"2024-03-06T07:22:44.372451+0000","last_became_peered":"2024-03-06T07:22:44.372451+0000","last_unstale":"2024-03-06T07:22:44.379220+0000","last_undegraded":"2024-03-06T07:22:44.379220+0000","last_fullsized":"2024-03-06T07:22:44.379220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:15.672728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366071+0000","last_change":"2024-03-06T07:22:35.034503+0000","last_active":"2024-03-06T07:22:44.366071+0000","last_peered":"2024-03-06T07:22:44.366071+0000","last_clean":"2024-03-06T07:22:44.366071+0000","last_became_active":"2024-03-06T07:22:35.034230+0000","last_became_peered":"2024-03-06T07:22:35.034230+0000","last_unstale":"2024-03-06T07:22:44.366071+0000","last_undegraded":"2024-03-06T07:22:44.366071+0000","last_fullsized":"2024-03-06T07:22:44.366071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365219+0000","last_change":"2024-03-06T07:22:34.039959+0000","last_active":"2024-03-06T07:22:44.365219+0000","last_peered":"2024-03-06T07:22:44.365219+0000","last_clean":"2024-03-06T07:22:44.365219+0000","last_became_active":"2024-03-06T07:22:34.039689+0000","last_became_peered":"2024-03-06T07:22:34.039689+0000","last_unstale":"2024-03-06T07:22:44.365219+0000","last_undegraded":"2024-03-06T07:22:44.365219+0000","last_fullsized":"2024-03-06T07:22:44.365219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:23.420842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364356+0000","last_change":"2024-03-06T07:22:44.364356+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364356+0000","last_undegraded":"2024-03-06T07:22:44.364356+0000","last_fullsized":"2024-03-06T07:22:44.364356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.372909+0000","last_change":"2024-03-06T07:22:44.371938+0000","last_active":"2024-03-06T07:22:44.372909+0000","last_peered":"2024-03-06T07:22:44.372909+0000","last_clean":"2024-03-06T07:22:44.372909+0000","last_became_active":"2024-03-06T07:22:44.371634+0000","last_became_peered":"2024-03-06T07:22:44.371634+0000","last_unstale":"2024-03-06T07:22:44.372909+0000","last_undegraded":"2024-03-06T07:22:44.372909+0000","last_fullsized":"2024-03-06T07:22:44.372909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364645+0000","last_change":"2024-03-06T07:22:35.032290+0000","last_active":"2024-03-06T07:22:44.364645+0000","last_peered":"2024-03-06T07:22:44.364645+0000","last_clean":"2024-03-06T07:22:44.364645+0000","last_became_active":"2024-03-06T07:22:35.032048+0000","last_became_peered":"2024-03-06T07:22:35.032048+0000","last_unstale":"2024-03-06T07:22:44.364645+0000","last_undegraded":"2024-03-06T07:22:44.364645+0000","last_fullsized":"2024-03-06T07:22:44.364645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:25.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375609+0000","last_change":"2024-03-06T07:22:34.043149+0000","last_active":"2024-03-06T07:22:44.375609+0000","last_peered":"2024-03-06T07:22:44.375609+0000","last_clean":"2024-03-06T07:22:44.375609+0000","last_became_active":"2024-03-06T07:22:34.042888+0000","last_became_peered":"2024-03-06T07:22:34.042888+0000","last_unstale":"2024-03-06T07:22:44.375609+0000","last_undegraded":"2024-03-06T07:22:44.375609+0000","last_fullsized":"2024-03-06T07:22:44.375609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366248+0000","last_change":"2024-03-06T07:22:44.366248+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366248+0000","last_undegraded":"2024-03-06T07:22:44.366248+0000","last_fullsized":"2024-03-06T07:22:44.366248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384733+0000","last_change":"2024-03-06T07:22:44.379879+0000","last_active":"2024-03-06T07:22:44.384733+0000","last_peered":"2024-03-06T07:22:44.384733+0000","last_clean":"2024-03-06T07:22:44.384733+0000","last_became_active":"2024-03-06T07:22:44.379611+0000","last_became_peered":"2024-03-06T07:22:44.379611+0000","last_unstale":"2024-03-06T07:22:44.384733+0000","last_undegraded":"2024-03-06T07:22:44.384733+0000","last_fullsized":"2024-03-06T07:22:44.384733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:14.207036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363632+0000","last_change":"2024-03-06T07:22:35.034984+0000","last_active":"2024-03-06T07:22:44.363632+0000","last_peered":"2024-03-06T07:22:44.363632+0000","last_clean":"2024-03-06T07:22:44.363632+0000","last_became_active":"2024-03-06T07:22:35.031303+0000","last_became_peered":"2024-03-06T07:22:35.031303+0000","last_unstale":"2024-03-06T07:22:44.363632+0000","last_undegraded":"2024-03-06T07:22:44.363632+0000","last_fullsized":"2024-03-06T07:22:44.363632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365213+0000","last_change":"2024-03-06T07:22:34.044320+0000","last_active":"2024-03-06T07:22:44.365213+0000","last_peered":"2024-03-06T07:22:44.365213+0000","last_clean":"2024-03-06T07:22:44.365213+0000","last_became_active":"2024-03-06T07:22:34.043832+0000","last_became_peered":"2024-03-06T07:22:34.043832+0000","last_unstale":"2024-03-06T07:22:44.365213+0000","last_undegraded":"2024-03-06T07:22:44.365213+0000","last_fullsized":"2024-03-06T07:22:44.365213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366779+0000","last_change":"2024-03-06T07:22:44.366779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366779+0000","last_undegraded":"2024-03-06T07:22:44.366779+0000","last_fullsized":"2024-03-06T07:22:44.366779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377398+0000","last_change":"2024-03-06T07:22:44.373965+0000","last_active":"2024-03-06T07:22:44.377398+0000","last_peered":"2024-03-06T07:22:44.377398+0000","last_clean":"2024-03-06T07:22:44.377398+0000","last_became_active":"2024-03-06T07:22:44.373617+0000","last_became_peered":"2024-03-06T07:22:44.373617+0000","last_unstale":"2024-03-06T07:22:44.377398+0000","last_undegraded":"2024-03-06T07:22:44.377398+0000","last_fullsized":"2024-03-06T07:22:44.377398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:01.683892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363411+0000","last_change":"2024-03-06T07:22:35.034203+0000","last_active":"2024-03-06T07:22:44.363411+0000","last_peered":"2024-03-06T07:22:44.363411+0000","last_clean":"2024-03-06T07:22:44.363411+0000","last_became_active":"2024-03-06T07:22:35.033903+0000","last_became_peered":"2024-03-06T07:22:35.033903+0000","last_unstale":"2024-03-06T07:22:44.363411+0000","last_undegraded":"2024-03-06T07:22:44.363411+0000","last_fullsized":"2024-03-06T07:22:44.363411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:02:14.829352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361765+0000","last_change":"2024-03-06T07:22:34.036176+0000","last_active":"2024-03-06T07:22:44.361765+0000","last_peered":"2024-03-06T07:22:44.361765+0000","last_clean":"2024-03-06T07:22:44.361765+0000","last_became_active":"2024-03-06T07:22:34.035867+0000","last_became_peered":"2024-03-06T07:22:34.035867+0000","last_unstale":"2024-03-06T07:22:44.361765+0000","last_undegraded":"2024-03-06T07:22:44.361765+0000","last_fullsized":"2024-03-06T07:22:44.361765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:28.336495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.374791+0000","last_change":"2024-03-06T07:22:44.374791+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.374791+0000","last_undegraded":"2024-03-06T07:22:44.374791+0000","last_fullsized":"2024-03-06T07:22:44.374791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376378+0000","last_change":"2024-03-06T07:22:44.369401+0000","last_active":"2024-03-06T07:22:44.376378+0000","last_peered":"2024-03-06T07:22:44.376378+0000","last_clean":"2024-03-06T07:22:44.376378+0000","last_became_active":"2024-03-06T07:22:44.369149+0000","last_became_peered":"2024-03-06T07:22:44.369149+0000","last_unstale":"2024-03-06T07:22:44.376378+0000","last_undegraded":"2024-03-06T07:22:44.376378+0000","last_fullsized":"2024-03-06T07:22:44.376378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:29.324559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363089+0000","last_change":"2024-03-06T07:22:35.038497+0000","last_active":"2024-03-06T07:22:44.363089+0000","last_peered":"2024-03-06T07:22:44.363089+0000","last_clean":"2024-03-06T07:22:44.363089+0000","last_became_active":"2024-03-06T07:22:35.038189+0000","last_became_peered":"2024-03-06T07:22:35.038189+0000","last_unstale":"2024-03-06T07:22:44.363089+0000","last_undegraded":"2024-03-06T07:22:44.363089+0000","last_fullsized":"2024-03-06T07:22:44.363089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:34.698852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369442+0000","last_change":"2024-03-06T07:22:34.038423+0000","last_active":"2024-03-06T07:22:44.369442+0000","last_peered":"2024-03-06T07:22:44.369442+0000","last_clean":"2024-03-06T07:22:44.369442+0000","last_became_active":"2024-03-06T07:22:34.038126+0000","last_became_peered":"2024-03-06T07:22:34.038126+0000","last_unstale":"2024-03-06T07:22:44.369442+0000","last_undegraded":"2024-03-06T07:22:44.369442+0000","last_fullsized":"2024-03-06T07:22:44.369442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:43.441029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364377+0000","last_change":"2024-03-06T07:22:44.364377+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364377+0000","last_undegraded":"2024-03-06T07:22:44.364377+0000","last_fullsized":"2024-03-06T07:22:44.364377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377112+0000","last_change":"2024-03-06T07:22:44.373676+0000","last_active":"2024-03-06T07:22:44.377112+0000","last_peered":"2024-03-06T07:22:44.377112+0000","last_clean":"2024-03-06T07:22:44.377112+0000","last_became_active":"2024-03-06T07:22:44.373209+0000","last_became_peered":"2024-03-06T07:22:44.373209+0000","last_unstale":"2024-03-06T07:22:44.377112+0000","last_undegraded":"2024-03-06T07:22:44.377112+0000","last_fullsized":"2024-03-06T07:22:44.377112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:05.175843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364057+0000","last_change":"2024-03-06T07:22:35.042660+0000","last_active":"2024-03-06T07:22:44.364057+0000","last_peered":"2024-03-06T07:22:44.364057+0000","last_clean":"2024-03-06T07:22:44.364057+0000","last_became_active":"2024-03-06T07:22:35.042327+0000","last_became_peered":"2024-03-06T07:22:35.042327+0000","last_unstale":"2024-03-06T07:22:44.364057+0000","last_undegraded":"2024-03-06T07:22:44.364057+0000","last_fullsized":"2024-03-06T07:22:44.364057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358735+0000","last_change":"2024-03-06T07:22:34.035505+0000","last_active":"2024-03-06T07:22:44.358735+0000","last_peered":"2024-03-06T07:22:44.358735+0000","last_clean":"2024-03-06T07:22:44.358735+0000","last_became_active":"2024-03-06T07:22:34.035257+0000","last_became_peered":"2024-03-06T07:22:34.035257+0000","last_unstale":"2024-03-06T07:22:44.358735+0000","last_undegraded":"2024-03-06T07:22:44.358735+0000","last_fullsized":"2024-03-06T07:22:44.358735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359116+0000","last_change":"2024-03-06T07:22:44.359116+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359116+0000","last_undegraded":"2024-03-06T07:22:44.359116+0000","last_fullsized":"2024-03-06T07:22:44.359116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371251+0000","last_change":"2024-03-06T07:22:44.367628+0000","last_active":"2024-03-06T07:22:44.371251+0000","last_peered":"2024-03-06T07:22:44.371251+0000","last_clean":"2024-03-06T07:22:44.371251+0000","last_became_active":"2024-03-06T07:22:44.367384+0000","last_became_peered":"2024-03-06T07:22:44.367384+0000","last_unstale":"2024-03-06T07:22:44.371251+0000","last_undegraded":"2024-03-06T07:22:44.371251+0000","last_fullsized":"2024-03-06T07:22:44.371251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:37.145482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375993+0000","last_change":"2024-03-06T07:22:35.032174+0000","last_active":"2024-03-06T07:22:44.375993+0000","last_peered":"2024-03-06T07:22:44.375993+0000","last_clean":"2024-03-06T07:22:44.375993+0000","last_became_active":"2024-03-06T07:22:35.031869+0000","last_became_peered":"2024-03-06T07:22:35.031869+0000","last_unstale":"2024-03-06T07:22:44.375993+0000","last_undegraded":"2024-03-06T07:22:44.375993+0000","last_fullsized":"2024-03-06T07:22:44.375993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:58.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364160+0000","last_change":"2024-03-06T07:22:34.037475+0000","last_active":"2024-03-06T07:22:44.364160+0000","last_peered":"2024-03-06T07:22:44.364160+0000","last_clean":"2024-03-06T07:22:44.364160+0000","last_became_active":"2024-03-06T07:22:34.037206+0000","last_became_peered":"2024-03-06T07:22:34.037206+0000","last_unstale":"2024-03-06T07:22:44.364160+0000","last_undegraded":"2024-03-06T07:22:44.364160+0000","last_fullsized":"2024-03-06T07:22:44.364160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367154+0000","last_change":"2024-03-06T07:22:44.367154+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367154+0000","last_undegraded":"2024-03-06T07:22:44.367154+0000","last_fullsized":"2024-03-06T07:22:44.367154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376583+0000","last_change":"2024-03-06T07:22:44.372573+0000","last_active":"2024-03-06T07:22:44.376583+0000","last_peered":"2024-03-06T07:22:44.376583+0000","last_clean":"2024-03-06T07:22:44.376583+0000","last_became_active":"2024-03-06T07:22:44.372321+0000","last_became_peered":"2024-03-06T07:22:44.372321+0000","last_unstale":"2024-03-06T07:22:44.376583+0000","last_undegraded":"2024-03-06T07:22:44.376583+0000","last_fullsized":"2024-03-06T07:22:44.376583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:01.508839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373616+0000","last_change":"2024-03-06T07:22:35.042271+0000","last_active":"2024-03-06T07:22:44.373616+0000","last_peered":"2024-03-06T07:22:44.373616+0000","last_clean":"2024-03-06T07:22:44.373616+0000","last_became_active":"2024-03-06T07:22:35.036189+0000","last_became_peered":"2024-03-06T07:22:35.036189+0000","last_unstale":"2024-03-06T07:22:44.373616+0000","last_undegraded":"2024-03-06T07:22:44.373616+0000","last_fullsized":"2024-03-06T07:22:44.373616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:47.064005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364659+0000","last_change":"2024-03-06T07:22:34.036505+0000","last_active":"2024-03-06T07:22:44.364659+0000","last_peered":"2024-03-06T07:22:44.364659+0000","last_clean":"2024-03-06T07:22:44.364659+0000","last_became_active":"2024-03-06T07:22:34.036198+0000","last_became_peered":"2024-03-06T07:22:34.036198+0000","last_unstale":"2024-03-06T07:22:44.364659+0000","last_undegraded":"2024-03-06T07:22:44.364659+0000","last_fullsized":"2024-03-06T07:22:44.364659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362951+0000","last_change":"2024-03-06T07:22:44.362951+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362951+0000","last_undegraded":"2024-03-06T07:22:44.362951+0000","last_fullsized":"2024-03-06T07:22:44.362951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373381+0000","last_change":"2024-03-06T07:22:44.372300+0000","last_active":"2024-03-06T07:22:44.373381+0000","last_peered":"2024-03-06T07:22:44.373381+0000","last_clean":"2024-03-06T07:22:44.373381+0000","last_became_active":"2024-03-06T07:22:44.372036+0000","last_became_peered":"2024-03-06T07:22:44.372036+0000","last_unstale":"2024-03-06T07:22:44.373381+0000","last_undegraded":"2024-03-06T07:22:44.373381+0000","last_fullsized":"2024-03-06T07:22:44.373381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364758+0000","last_change":"2024-03-06T07:22:35.031196+0000","last_active":"2024-03-06T07:22:44.364758+0000","last_peered":"2024-03-06T07:22:44.364758+0000","last_clean":"2024-03-06T07:22:44.364758+0000","last_became_active":"2024-03-06T07:22:35.030767+0000","last_became_peered":"2024-03-06T07:22:35.030767+0000","last_unstale":"2024-03-06T07:22:44.364758+0000","last_undegraded":"2024-03-06T07:22:44.364758+0000","last_fullsized":"2024-03-06T07:22:44.364758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:43.288135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363735+0000","last_change":"2024-03-06T07:22:34.034863+0000","last_active":"2024-03-06T07:22:44.363735+0000","last_peered":"2024-03-06T07:22:44.363735+0000","last_clean":"2024-03-06T07:22:44.363735+0000","last_became_active":"2024-03-06T07:22:34.034577+0000","last_became_peered":"2024-03-06T07:22:34.034577+0000","last_unstale":"2024-03-06T07:22:44.363735+0000","last_undegraded":"2024-03-06T07:22:44.363735+0000","last_fullsized":"2024-03-06T07:22:44.363735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363562+0000","last_change":"2024-03-06T07:22:44.363562+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363562+0000","last_undegraded":"2024-03-06T07:22:44.363562+0000","last_fullsized":"2024-03-06T07:22:44.363562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379268+0000","last_change":"2024-03-06T07:22:44.374457+0000","last_active":"2024-03-06T07:22:44.379268+0000","last_peered":"2024-03-06T07:22:44.379268+0000","last_clean":"2024-03-06T07:22:44.379268+0000","last_became_active":"2024-03-06T07:22:44.374059+0000","last_became_peered":"2024-03-06T07:22:44.374059+0000","last_unstale":"2024-03-06T07:22:44.379268+0000","last_undegraded":"2024-03-06T07:22:44.379268+0000","last_fullsized":"2024-03-06T07:22:44.379268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:27.741841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.360947+0000","last_change":"2024-03-06T07:22:35.036664+0000","last_active":"2024-03-06T07:22:44.360947+0000","last_peered":"2024-03-06T07:22:44.360947+0000","last_clean":"2024-03-06T07:22:44.360947+0000","last_became_active":"2024-03-06T07:22:35.036379+0000","last_became_peered":"2024-03-06T07:22:35.036379+0000","last_unstale":"2024-03-06T07:22:44.360947+0000","last_undegraded":"2024-03-06T07:22:44.360947+0000","last_fullsized":"2024-03-06T07:22:44.360947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:56.002343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366830+0000","last_change":"2024-03-06T07:22:34.044993+0000","last_active":"2024-03-06T07:22:44.366830+0000","last_peered":"2024-03-06T07:22:44.366830+0000","last_clean":"2024-03-06T07:22:44.366830+0000","last_became_active":"2024-03-06T07:22:34.044795+0000","last_became_peered":"2024-03-06T07:22:34.044795+0000","last_unstale":"2024-03-06T07:22:44.366830+0000","last_undegraded":"2024-03-06T07:22:44.366830+0000","last_fullsized":"2024-03-06T07:22:44.366830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.256142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367883+0000","last_change":"2024-03-06T07:22:44.367883+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367883+0000","last_undegraded":"2024-03-06T07:22:44.367883+0000","last_fullsized":"2024-03-06T07:22:44.367883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371628+0000","last_change":"2024-03-06T07:22:44.369924+0000","last_active":"2024-03-06T07:22:44.371628+0000","last_peered":"2024-03-06T07:22:44.371628+0000","last_clean":"2024-03-06T07:22:44.371628+0000","last_became_active":"2024-03-06T07:22:44.369676+0000","last_became_peered":"2024-03-06T07:22:44.369676+0000","last_unstale":"2024-03-06T07:22:44.371628+0000","last_undegraded":"2024-03-06T07:22:44.371628+0000","last_fullsized":"2024-03-06T07:22:44.371628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:25.600800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365427+0000","last_change":"2024-03-06T07:22:35.031505+0000","last_active":"2024-03-06T07:22:44.365427+0000","last_peered":"2024-03-06T07:22:44.365427+0000","last_clean":"2024-03-06T07:22:44.365427+0000","last_became_active":"2024-03-06T07:22:35.031241+0000","last_became_peered":"2024-03-06T07:22:35.031241+0000","last_unstale":"2024-03-06T07:22:44.365427+0000","last_undegraded":"2024-03-06T07:22:44.365427+0000","last_fullsized":"2024-03-06T07:22:44.365427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373724+0000","last_change":"2024-03-06T07:22:34.044453+0000","last_active":"2024-03-06T07:22:44.373724+0000","last_peered":"2024-03-06T07:22:44.373724+0000","last_clean":"2024-03-06T07:22:44.373724+0000","last_became_active":"2024-03-06T07:22:34.044190+0000","last_became_peered":"2024-03-06T07:22:34.044190+0000","last_unstale":"2024-03-06T07:22:44.373724+0000","last_undegraded":"2024-03-06T07:22:44.373724+0000","last_fullsized":"2024-03-06T07:22:44.373724+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367845+0000","last_change":"2024-03-06T07:22:44.367845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367845+0000","last_undegraded":"2024-03-06T07:22:44.367845+0000","last_fullsized":"2024-03-06T07:22:44.367845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382847+0000","last_change":"2024-03-06T07:22:44.382519+0000","last_active":"2024-03-06T07:22:44.382847+0000","last_peered":"2024-03-06T07:22:44.382847+0000","last_clean":"2024-03-06T07:22:44.382847+0000","last_became_active":"2024-03-06T07:22:44.382295+0000","last_became_peered":"2024-03-06T07:22:44.382295+0000","last_unstale":"2024-03-06T07:22:44.382847+0000","last_undegraded":"2024-03-06T07:22:44.382847+0000","last_fullsized":"2024-03-06T07:22:44.382847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:00.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361555+0000","last_change":"2024-03-06T07:22:35.034792+0000","last_active":"2024-03-06T07:22:44.361555+0000","last_peered":"2024-03-06T07:22:44.361555+0000","last_clean":"2024-03-06T07:22:44.361555+0000","last_became_active":"2024-03-06T07:22:35.034503+0000","last_became_peered":"2024-03-06T07:22:35.034503+0000","last_unstale":"2024-03-06T07:22:44.361555+0000","last_undegraded":"2024-03-06T07:22:44.361555+0000","last_fullsized":"2024-03-06T07:22:44.361555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368734+0000","last_change":"2024-03-06T07:22:34.037188+0000","last_active":"2024-03-06T07:22:44.368734+0000","last_peered":"2024-03-06T07:22:44.368734+0000","last_clean":"2024-03-06T07:22:44.368734+0000","last_became_active":"2024-03-06T07:22:34.036918+0000","last_became_peered":"2024-03-06T07:22:34.036918+0000","last_unstale":"2024-03-06T07:22:44.368734+0000","last_undegraded":"2024-03-06T07:22:44.368734+0000","last_fullsized":"2024-03-06T07:22:44.368734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361861+0000","last_change":"2024-03-06T07:22:44.361861+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361861+0000","last_undegraded":"2024-03-06T07:22:44.361861+0000","last_fullsized":"2024-03-06T07:22:44.361861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379180+0000","last_change":"2024-03-06T07:22:44.371466+0000","last_active":"2024-03-06T07:22:44.379180+0000","last_peered":"2024-03-06T07:22:44.379180+0000","last_clean":"2024-03-06T07:22:44.379180+0000","last_became_active":"2024-03-06T07:22:44.371190+0000","last_became_peered":"2024-03-06T07:22:44.371190+0000","last_unstale":"2024-03-06T07:22:44.379180+0000","last_undegraded":"2024-03-06T07:22:44.379180+0000","last_fullsized":"2024-03-06T07:22:44.379180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:32.601916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370567+0000","last_change":"2024-03-06T07:22:35.033263+0000","last_active":"2024-03-06T07:22:44.370567+0000","last_peered":"2024-03-06T07:22:44.370567+0000","last_clean":"2024-03-06T07:22:44.370567+0000","last_became_active":"2024-03-06T07:22:35.033005+0000","last_became_peered":"2024-03-06T07:22:35.033005+0000","last_unstale":"2024-03-06T07:22:44.370567+0000","last_undegraded":"2024-03-06T07:22:44.370567+0000","last_fullsized":"2024-03-06T07:22:44.370567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:00.137617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364936+0000","last_change":"2024-03-06T07:22:34.035735+0000","last_active":"2024-03-06T07:22:44.364936+0000","last_peered":"2024-03-06T07:22:44.364936+0000","last_clean":"2024-03-06T07:22:44.364936+0000","last_became_active":"2024-03-06T07:22:34.035459+0000","last_became_peered":"2024-03-06T07:22:34.035459+0000","last_unstale":"2024-03-06T07:22:44.364936+0000","last_undegraded":"2024-03-06T07:22:44.364936+0000","last_fullsized":"2024-03-06T07:22:44.364936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366886+0000","last_change":"2024-03-06T07:22:44.366886+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366886+0000","last_undegraded":"2024-03-06T07:22:44.366886+0000","last_fullsized":"2024-03-06T07:22:44.366886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382022+0000","last_change":"2024-03-06T07:22:44.377358+0000","last_active":"2024-03-06T07:22:44.382022+0000","last_peered":"2024-03-06T07:22:44.382022+0000","last_clean":"2024-03-06T07:22:44.382022+0000","last_became_active":"2024-03-06T07:22:44.377098+0000","last_became_peered":"2024-03-06T07:22:44.377098+0000","last_unstale":"2024-03-06T07:22:44.382022+0000","last_undegraded":"2024-03-06T07:22:44.382022+0000","last_fullsized":"2024-03-06T07:22:44.382022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:43.869151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363978+0000","last_change":"2024-03-06T07:22:35.038111+0000","last_active":"2024-03-06T07:22:44.363978+0000","last_peered":"2024-03-06T07:22:44.363978+0000","last_clean":"2024-03-06T07:22:44.363978+0000","last_became_active":"2024-03-06T07:22:35.037779+0000","last_became_peered":"2024-03-06T07:22:35.037779+0000","last_unstale":"2024-03-06T07:22:44.363978+0000","last_undegraded":"2024-03-06T07:22:44.363978+0000","last_fullsized":"2024-03-06T07:22:44.363978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.362629+0000","last_change":"2024-03-06T07:22:34.039910+0000","last_active":"2024-03-06T07:22:44.362629+0000","last_peered":"2024-03-06T07:22:44.362629+0000","last_clean":"2024-03-06T07:22:44.362629+0000","last_became_active":"2024-03-06T07:22:34.039246+0000","last_became_peered":"2024-03-06T07:22:34.039246+0000","last_unstale":"2024-03-06T07:22:44.362629+0000","last_undegraded":"2024-03-06T07:22:44.362629+0000","last_fullsized":"2024-03-06T07:22:44.362629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:41.961694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362779+0000","last_change":"2024-03-06T07:22:44.362779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362779+0000","last_undegraded":"2024-03-06T07:22:44.362779+0000","last_fullsized":"2024-03-06T07:22:44.362779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379240+0000","last_change":"2024-03-06T07:22:44.374437+0000","last_active":"2024-03-06T07:22:44.379240+0000","last_peered":"2024-03-06T07:22:44.379240+0000","last_clean":"2024-03-06T07:22:44.379240+0000","last_became_active":"2024-03-06T07:22:44.374055+0000","last_became_peered":"2024-03-06T07:22:44.374055+0000","last_unstale":"2024-03-06T07:22:44.379240+0000","last_undegraded":"2024-03-06T07:22:44.379240+0000","last_fullsized":"2024-03-06T07:22:44.379240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:53:05.282920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363282+0000","last_change":"2024-03-06T07:22:35.027657+0000","last_active":"2024-03-06T07:22:44.363282+0000","last_peered":"2024-03-06T07:22:44.363282+0000","last_clean":"2024-03-06T07:22:44.363282+0000","last_became_active":"2024-03-06T07:22:35.027404+0000","last_became_peered":"2024-03-06T07:22:35.027404+0000","last_unstale":"2024-03-06T07:22:44.363282+0000","last_undegraded":"2024-03-06T07:22:44.363282+0000","last_fullsized":"2024-03-06T07:22:44.363282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:34.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364859+0000","last_change":"2024-03-06T07:22:34.039209+0000","last_active":"2024-03-06T07:22:44.364859+0000","last_peered":"2024-03-06T07:22:44.364859+0000","last_clean":"2024-03-06T07:22:44.364859+0000","last_became_active":"2024-03-06T07:22:34.038921+0000","last_became_peered":"2024-03-06T07:22:34.038921+0000","last_unstale":"2024-03-06T07:22:44.364859+0000","last_undegraded":"2024-03-06T07:22:44.364859+0000","last_fullsized":"2024-03-06T07:22:44.364859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:12.359435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363182+0000","last_change":"2024-03-06T07:22:44.363182+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363182+0000","last_undegraded":"2024-03-06T07:22:44.363182+0000","last_fullsized":"2024-03-06T07:22:44.363182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376720+0000","last_change":"2024-03-06T07:22:44.374940+0000","last_active":"2024-03-06T07:22:44.376720+0000","last_peered":"2024-03-06T07:22:44.376720+0000","last_clean":"2024-03-06T07:22:44.376720+0000","last_became_active":"2024-03-06T07:22:44.374653+0000","last_became_peered":"2024-03-06T07:22:44.374653+0000","last_unstale":"2024-03-06T07:22:44.376720+0000","last_undegraded":"2024-03-06T07:22:44.376720+0000","last_fullsized":"2024-03-06T07:22:44.376720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359439+0000","last_change":"2024-03-06T07:22:35.038010+0000","last_active":"2024-03-06T07:22:44.359439+0000","last_peered":"2024-03-06T07:22:44.359439+0000","last_clean":"2024-03-06T07:22:44.359439+0000","last_became_active":"2024-03-06T07:22:35.037717+0000","last_became_peered":"2024-03-06T07:22:35.037717+0000","last_unstale":"2024-03-06T07:22:44.359439+0000","last_undegraded":"2024-03-06T07:22:44.359439+0000","last_fullsized":"2024-03-06T07:22:44.359439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365513+0000","last_change":"2024-03-06T07:22:34.039679+0000","last_active":"2024-03-06T07:22:44.365513+0000","last_peered":"2024-03-06T07:22:44.365513+0000","last_clean":"2024-03-06T07:22:44.365513+0000","last_became_active":"2024-03-06T07:22:34.039092+0000","last_became_peered":"2024-03-06T07:22:34.039092+0000","last_unstale":"2024-03-06T07:22:44.365513+0000","last_undegraded":"2024-03-06T07:22:44.365513+0000","last_fullsized":"2024-03-06T07:22:44.365513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.370845+0000","last_change":"2024-03-06T07:22:44.370845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.370845+0000","last_undegraded":"2024-03-06T07:22:44.370845+0000","last_fullsized":"2024-03-06T07:22:44.370845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373582+0000","last_change":"2024-03-06T07:22:44.369327+0000","last_active":"2024-03-06T07:22:44.373582+0000","last_peered":"2024-03-06T07:22:44.373582+0000","last_clean":"2024-03-06T07:22:44.373582+0000","last_became_active":"2024-03-06T07:22:44.369076+0000","last_became_peered":"2024-03-06T07:22:44.369076+0000","last_unstale":"2024-03-06T07:22:44.373582+0000","last_undegraded":"2024-03-06T07:22:44.373582+0000","last_fullsized":"2024-03-06T07:22:44.373582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:52.087902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359257+0000","last_change":"2024-03-06T07:22:35.026900+0000","last_active":"2024-03-06T07:22:44.359257+0000","last_peered":"2024-03-06T07:22:44.359257+0000","last_clean":"2024-03-06T07:22:44.359257+0000","last_became_active":"2024-03-06T07:22:35.026626+0000","last_became_peered":"2024-03-06T07:22:35.026626+0000","last_unstale":"2024-03-06T07:22:44.359257+0000","last_undegraded":"2024-03-06T07:22:44.359257+0000","last_fullsized":"2024-03-06T07:22:44.359257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358578+0000","last_change":"2024-03-06T07:22:34.039382+0000","last_active":"2024-03-06T07:22:44.358578+0000","last_peered":"2024-03-06T07:22:44.358578+0000","last_clean":"2024-03-06T07:22:44.358578+0000","last_became_active":"2024-03-06T07:22:34.039088+0000","last_became_peered":"2024-03-06T07:22:34.039088+0000","last_unstale":"2024-03-06T07:22:44.358578+0000","last_undegraded":"2024-03-06T07:22:44.358578+0000","last_fullsized":"2024-03-06T07:22:44.358578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363505+0000","last_change":"2024-03-06T07:22:44.363505+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363505+0000","last_undegraded":"2024-03-06T07:22:44.363505+0000","last_fullsized":"2024-03-06T07:22:44.363505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375571+0000","last_change":"2024-03-06T07:22:44.374409+0000","last_active":"2024-03-06T07:22:44.375571+0000","last_peered":"2024-03-06T07:22:44.375571+0000","last_clean":"2024-03-06T07:22:44.375571+0000","last_became_active":"2024-03-06T07:22:44.374130+0000","last_became_peered":"2024-03-06T07:22:44.374130+0000","last_unstale":"2024-03-06T07:22:44.375571+0000","last_undegraded":"2024-03-06T07:22:44.375571+0000","last_fullsized":"2024-03-06T07:22:44.375571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:30.597343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368651+0000","last_change":"2024-03-06T07:22:35.033671+0000","last_active":"2024-03-06T07:22:44.368651+0000","last_peered":"2024-03-06T07:22:44.368651+0000","last_clean":"2024-03-06T07:22:44.368651+0000","last_became_active":"2024-03-06T07:22:35.033167+0000","last_became_peered":"2024-03-06T07:22:35.033167+0000","last_unstale":"2024-03-06T07:22:44.368651+0000","last_undegraded":"2024-03-06T07:22:44.368651+0000","last_fullsized":"2024-03-06T07:22:44.368651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365069+0000","last_change":"2024-03-06T07:22:34.037026+0000","last_active":"2024-03-06T07:22:44.365069+0000","last_peered":"2024-03-06T07:22:44.365069+0000","last_clean":"2024-03-06T07:22:44.365069+0000","last_became_active":"2024-03-06T07:22:34.036742+0000","last_became_peered":"2024-03-06T07:22:34.036742+0000","last_unstale":"2024-03-06T07:22:44.365069+0000","last_undegraded":"2024-03-06T07:22:44.365069+0000","last_fullsized":"2024-03-06T07:22:44.365069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363945+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:22:44.363945+0000","last_peered":"2024-03-06T07:22:44.363945+0000","last_clean":"2024-03-06T07:22:44.363945+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:22:44.363945+0000","last_undegraded":"2024-03-06T07:22:44.363945+0000","last_fullsized":"2024-03-06T07:22:44.363945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":1032479,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":443701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":442123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":443805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":435149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":442831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":442201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":1032457,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:48.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.930+0000 7f4482f5c640 1 Processor -- start 2024-03-06T07:22:48.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.931+0000 7f4482f5c640 1 -- start start 2024-03-06T07:22:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.931+0000 7f4482f5c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f447c0724b0 0x7f447c159090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.931+0000 7f4482f5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c072890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.931+0000 7f4482f5c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0a7e50 con 0x7f447c0c67f0 2024-03-06T07:22:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.931+0000 7f4482f5c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c0a8410 con 0x7f447c0724b0 2024-03-06T07:22:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.931+0000 7f4482f5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0a8130 con 0x7f447c0c3940 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f447bfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f4480cf8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f447c0724b0 0x7f447c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f44814f9640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f447c0c67f0 0x7f447c155090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:47014/0 (socket says 172.21.15.26:47014) 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f44814f9640 1 -- 172.21.15.26:0/4149852498 learned_addr learned my addr 172.21.15.26:0/4149852498 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f447bfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c072890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:39956/0 (socket says 172.21.15.26:39956) 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f447b7fe640 1 -- 172.21.15.26:0/4149852498 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761855179 0 0) 0x7f447c0a7e50 con 0x7f447c0c67f0 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f447b7fe640 1 -- 172.21.15.26:0/4149852498 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f446c003520 con 0x7f447c0c67f0 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f4480cf8640 1 -- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 msgr2=0x7f447c072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f4480cf8640 1 --2- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c072890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f4480cf8640 1 -- 172.21.15.26:0/4149852498 >> v1:172.21.15.117:6789/0 conn(0x7f447c0c67f0 legacy=0x7f447c155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f4480cf8640 1 -- 172.21.15.26:0/4149852498 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0c3420 con 0x7f447c0724b0 2024-03-06T07:22:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.932+0000 7f447bfff640 1 --2- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4480cf8640 1 --2- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0724b0 0x7f447c159090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44700097a0 tx=0x7f447002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0da04c4af78b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f447b7fe640 1 -- 172.21.15.26:0/4149852498 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470038070 con 0x7f447c0724b0 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f447b7fe640 1 -- 172.21.15.26:0/4149852498 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470031700 con 0x7f447c0724b0 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f447b7fe640 1 -- 172.21.15.26:0/4149852498 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470031ab0 con 0x7f447c0724b0 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 -- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0724b0 msgr2=0x7f447c159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 --2- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0724b0 0x7f447c159090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44700097a0 tx=0x7f447002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 -- 172.21.15.26:0/4149852498 shutdown_connections 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 --2- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 --2- 172.21.15.26:0/4149852498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0724b0 0x7f447c159090 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 -- 172.21.15.26:0/4149852498 >> 172.21.15.26:0/4149852498 conn(0x7f447c0aa6a0 msgr2=0x7f447c0aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 -- 172.21.15.26:0/4149852498 shutdown_connections 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.933+0000 7f4482f5c640 1 -- 172.21.15.26:0/4149852498 wait complete. 2024-03-06T07:22:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.934+0000 7f4482f5c640 1 Processor -- start 2024-03-06T07:22:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.935+0000 7f4482f5c640 1 -- start start 2024-03-06T07:22:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.935+0000 7f4482f5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447c0724b0 0x7f447c14e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.935+0000 7f4482f5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c0abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.935+0000 7f4482f5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0c67f0 0x7f447c1ee110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.935+0000 7f4482f5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c152ae0 con 0x7f447c0724b0 2024-03-06T07:22:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.935+0000 7f4482f5c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c152960 con 0x7f447c0c67f0 2024-03-06T07:22:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.935+0000 7f4482f5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1557a0 con 0x7f447c0c3940 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447c0724b0 0x7f447c14e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447c0724b0 0x7f447c14e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49188/0 (socket says 172.21.15.26:49188) 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 -- 172.21.15.26:0/1028802835 learned_addr learned my addr 172.21.15.26:0/1028802835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f447bfff640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c0abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f44814f9640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0c67f0 0x7f447c1ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 -- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 msgr2=0x7f447c0abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c0abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 -- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0c67f0 msgr2=0x7f447c1ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0c67f0 0x7f447c1ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 -- 172.21.15.26:0/1028802835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c14f100 con 0x7f447c0724b0 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4480cf8640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447c0724b0 0x7f447c14e700 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f447002b7e0 tx=0x7f44700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f44797fa640 1 -- 172.21.15.26:0/1028802835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470042070 con 0x7f447c0724b0 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f44797fa640 1 -- 172.21.15.26:0/1028802835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447003d890 con 0x7f447c0724b0 2024-03-06T07:22:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c14ee20 con 0x7f447c0724b0 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f44797fa640 1 -- 172.21.15.26:0/1028802835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470038040 con 0x7f447c0724b0 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.936+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1eec50 con 0x7f447c0724b0 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.937+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c04f330 con 0x7f447c0724b0 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.938+0000 7f44797fa640 1 -- 172.21.15.26:0/1028802835 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4470045070 con 0x7f447c0724b0 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.939+0000 7f44797fa640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446006b4d0 0x7f446006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.939+0000 7f44797fa640 1 -- 172.21.15.26:0/1028802835 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f44700ac3f0 con 0x7f447c0724b0 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.939+0000 7f447bfff640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446006b4d0 0x7f446006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.939+0000 7f447bfff640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446006b4d0 0x7f446006d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4468009560 tx=0x7f4468009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:48.943+0000 7f44797fa640 1 -- 172.21.15.26:0/1028802835 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4470030070 con 0x7f447c0724b0 2024-03-06T07:22:49.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.120+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f447c14ec40 con 0x7f446006b4d0 2024-03-06T07:22:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.131+0000 7f44797fa640 1 -- 172.21.15.26:0/1028802835 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485419 (secure 0 0 0) 0x7f4468011a30 con 0x7f446006b4d0 2024-03-06T07:22:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:49.142 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.134+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446006b4d0 msgr2=0x7f446006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.134+0000 7f4482f5c640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446006b4d0 0x7f446006d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4468009560 tx=0x7f4468009190 comp rx=0 tx=0).stop 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.134+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447c0724b0 msgr2=0x7f447c14e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.134+0000 7f4482f5c640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447c0724b0 0x7f447c14e700 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f447002b7e0 tx=0x7f44700046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 shutdown_connections 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446006b4d0 0x7f446006d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f447c0c67f0 0x7f447c1ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f447c0c3940 0x7f447c0abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 --2- 172.21.15.26:0/1028802835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447c0724b0 0x7f447c14e700 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 >> 172.21.15.26:0/1028802835 conn(0x7f447c0aa6a0 msgr2=0x7f447c0c1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 shutdown_connections 2024-03-06T07:22:49.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.135+0000 7f4482f5c640 1 -- 172.21.15.26:0/1028802835 wait complete. 2024-03-06T07:22:49.147 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-03-06T07:22:49.064227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244432,"kb_used_data":7444,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730288,"statfs":{"total":773094113280,"available":772843814912,"internally_reserved":0,"allocated":7622656,"data_stored":4714746,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.712574"},"pg_stats":[{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361902+0000","last_change":"2024-03-06T07:22:44.361902+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361902+0000","last_undegraded":"2024-03-06T07:22:44.361902+0000","last_fullsized":"2024-03-06T07:22:44.361902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365039+0000","last_change":"2024-03-06T07:22:35.025691+0000","last_active":"2024-03-06T07:22:44.365039+0000","last_peered":"2024-03-06T07:22:44.365039+0000","last_clean":"2024-03-06T07:22:44.365039+0000","last_became_active":"2024-03-06T07:22:35.025394+0000","last_became_peered":"2024-03-06T07:22:35.025394+0000","last_unstale":"2024-03-06T07:22:44.365039+0000","last_undegraded":"2024-03-06T07:22:44.365039+0000","last_fullsized":"2024-03-06T07:22:44.365039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366504+0000","last_change":"2024-03-06T07:22:44.366504+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366504+0000","last_undegraded":"2024-03-06T07:22:44.366504+0000","last_fullsized":"2024-03-06T07:22:44.366504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373511+0000","last_change":"2024-03-06T07:22:35.048775+0000","last_active":"2024-03-06T07:22:44.373511+0000","last_peered":"2024-03-06T07:22:44.373511+0000","last_clean":"2024-03-06T07:22:44.373511+0000","last_became_active":"2024-03-06T07:22:35.048476+0000","last_became_peered":"2024-03-06T07:22:35.048476+0000","last_unstale":"2024-03-06T07:22:44.373511+0000","last_undegraded":"2024-03-06T07:22:44.373511+0000","last_fullsized":"2024-03-06T07:22:44.373511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:41.704388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362319+0000","last_change":"2024-03-06T07:22:44.362319+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362319+0000","last_undegraded":"2024-03-06T07:22:44.362319+0000","last_fullsized":"2024-03-06T07:22:44.362319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359053+0000","last_change":"2024-03-06T07:22:35.037727+0000","last_active":"2024-03-06T07:22:44.359053+0000","last_peered":"2024-03-06T07:22:44.359053+0000","last_clean":"2024-03-06T07:22:44.359053+0000","last_became_active":"2024-03-06T07:22:35.037421+0000","last_became_peered":"2024-03-06T07:22:35.037421+0000","last_unstale":"2024-03-06T07:22:44.359053+0000","last_undegraded":"2024-03-06T07:22:44.359053+0000","last_fullsized":"2024-03-06T07:22:44.359053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362908+0000","last_change":"2024-03-06T07:22:44.362908+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362908+0000","last_undegraded":"2024-03-06T07:22:44.362908+0000","last_fullsized":"2024-03-06T07:22:44.362908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365569+0000","last_change":"2024-03-06T07:22:35.035949+0000","last_active":"2024-03-06T07:22:44.365569+0000","last_peered":"2024-03-06T07:22:44.365569+0000","last_clean":"2024-03-06T07:22:44.365569+0000","last_became_active":"2024-03-06T07:22:35.035687+0000","last_became_peered":"2024-03-06T07:22:35.035687+0000","last_unstale":"2024-03-06T07:22:44.365569+0000","last_undegraded":"2024-03-06T07:22:44.365569+0000","last_fullsized":"2024-03-06T07:22:44.365569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362767+0000","last_change":"2024-03-06T07:22:44.362767+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362767+0000","last_undegraded":"2024-03-06T07:22:44.362767+0000","last_fullsized":"2024-03-06T07:22:44.362767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365712+0000","last_change":"2024-03-06T07:22:35.033081+0000","last_active":"2024-03-06T07:22:44.365712+0000","last_peered":"2024-03-06T07:22:44.365712+0000","last_clean":"2024-03-06T07:22:44.365712+0000","last_became_active":"2024-03-06T07:22:35.032799+0000","last_became_peered":"2024-03-06T07:22:35.032799+0000","last_unstale":"2024-03-06T07:22:44.365712+0000","last_undegraded":"2024-03-06T07:22:44.365712+0000","last_fullsized":"2024-03-06T07:22:44.365712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.187775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365929+0000","last_change":"2024-03-06T07:22:44.365929+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365929+0000","last_undegraded":"2024-03-06T07:22:44.365929+0000","last_fullsized":"2024-03-06T07:22:44.365929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364635+0000","last_change":"2024-03-06T07:22:35.041976+0000","last_active":"2024-03-06T07:22:44.364635+0000","last_peered":"2024-03-06T07:22:44.364635+0000","last_clean":"2024-03-06T07:22:44.364635+0000","last_became_active":"2024-03-06T07:22:35.041546+0000","last_became_peered":"2024-03-06T07:22:35.041546+0000","last_unstale":"2024-03-06T07:22:44.364635+0000","last_undegraded":"2024-03-06T07:22:44.364635+0000","last_fullsized":"2024-03-06T07:22:44.364635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360301+0000","last_change":"2024-03-06T07:22:44.360301+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360301+0000","last_undegraded":"2024-03-06T07:22:44.360301+0000","last_fullsized":"2024-03-06T07:22:44.360301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365335+0000","last_change":"2024-03-06T07:22:35.038320+0000","last_active":"2024-03-06T07:22:44.365335+0000","last_peered":"2024-03-06T07:22:44.365335+0000","last_clean":"2024-03-06T07:22:44.365335+0000","last_became_active":"2024-03-06T07:22:35.038043+0000","last_became_peered":"2024-03-06T07:22:35.038043+0000","last_unstale":"2024-03-06T07:22:44.365335+0000","last_undegraded":"2024-03-06T07:22:44.365335+0000","last_fullsized":"2024-03-06T07:22:44.365335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366763+0000","last_change":"2024-03-06T07:22:44.366763+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366763+0000","last_undegraded":"2024-03-06T07:22:44.366763+0000","last_fullsized":"2024-03-06T07:22:44.366763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368165+0000","last_change":"2024-03-06T07:22:35.040651+0000","last_active":"2024-03-06T07:22:44.368165+0000","last_peered":"2024-03-06T07:22:44.368165+0000","last_clean":"2024-03-06T07:22:44.368165+0000","last_became_active":"2024-03-06T07:22:35.040382+0000","last_became_peered":"2024-03-06T07:22:35.040382+0000","last_unstale":"2024-03-06T07:22:44.368165+0000","last_undegraded":"2024-03-06T07:22:44.368165+0000","last_fullsized":"2024-03-06T07:22:44.368165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:54.755446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360342+0000","last_change":"2024-03-06T07:22:44.360342+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360342+0000","last_undegraded":"2024-03-06T07:22:44.360342+0000","last_fullsized":"2024-03-06T07:22:44.360342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361779+0000","last_change":"2024-03-06T07:22:35.035524+0000","last_active":"2024-03-06T07:22:44.361779+0000","last_peered":"2024-03-06T07:22:44.361779+0000","last_clean":"2024-03-06T07:22:44.361779+0000","last_became_active":"2024-03-06T07:22:35.035252+0000","last_became_peered":"2024-03-06T07:22:35.035252+0000","last_unstale":"2024-03-06T07:22:44.361779+0000","last_undegraded":"2024-03-06T07:22:44.361779+0000","last_fullsized":"2024-03-06T07:22:44.361779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364257+0000","last_change":"2024-03-06T07:22:44.364257+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364257+0000","last_undegraded":"2024-03-06T07:22:44.364257+0000","last_fullsized":"2024-03-06T07:22:44.364257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365193+0000","last_change":"2024-03-06T07:22:35.040188+0000","last_active":"2024-03-06T07:22:44.365193+0000","last_peered":"2024-03-06T07:22:44.365193+0000","last_clean":"2024-03-06T07:22:44.365193+0000","last_became_active":"2024-03-06T07:22:35.039868+0000","last_became_peered":"2024-03-06T07:22:35.039868+0000","last_unstale":"2024-03-06T07:22:44.365193+0000","last_undegraded":"2024-03-06T07:22:44.365193+0000","last_fullsized":"2024-03-06T07:22:44.365193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359274+0000","last_change":"2024-03-06T07:22:44.359274+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359274+0000","last_undegraded":"2024-03-06T07:22:44.359274+0000","last_fullsized":"2024-03-06T07:22:44.359274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375229+0000","last_change":"2024-03-06T07:22:35.049739+0000","last_active":"2024-03-06T07:22:44.375229+0000","last_peered":"2024-03-06T07:22:44.375229+0000","last_clean":"2024-03-06T07:22:44.375229+0000","last_became_active":"2024-03-06T07:22:35.049118+0000","last_became_peered":"2024-03-06T07:22:35.049118+0000","last_unstale":"2024-03-06T07:22:44.375229+0000","last_undegraded":"2024-03-06T07:22:44.375229+0000","last_fullsized":"2024-03-06T07:22:44.375229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:49.784740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362088+0000","last_change":"2024-03-06T07:22:44.362088+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362088+0000","last_undegraded":"2024-03-06T07:22:44.362088+0000","last_fullsized":"2024-03-06T07:22:44.362088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365663+0000","last_change":"2024-03-06T07:22:35.038831+0000","last_active":"2024-03-06T07:22:44.365663+0000","last_peered":"2024-03-06T07:22:44.365663+0000","last_clean":"2024-03-06T07:22:44.365663+0000","last_became_active":"2024-03-06T07:22:35.038550+0000","last_became_peered":"2024-03-06T07:22:35.038550+0000","last_unstale":"2024-03-06T07:22:44.365663+0000","last_undegraded":"2024-03-06T07:22:44.365663+0000","last_fullsized":"2024-03-06T07:22:44.365663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362839+0000","last_change":"2024-03-06T07:22:44.362839+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362839+0000","last_undegraded":"2024-03-06T07:22:44.362839+0000","last_fullsized":"2024-03-06T07:22:44.362839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365819+0000","last_change":"2024-03-06T07:22:35.037819+0000","last_active":"2024-03-06T07:22:44.365819+0000","last_peered":"2024-03-06T07:22:44.365819+0000","last_clean":"2024-03-06T07:22:44.365819+0000","last_became_active":"2024-03-06T07:22:35.037519+0000","last_became_peered":"2024-03-06T07:22:35.037519+0000","last_unstale":"2024-03-06T07:22:44.365819+0000","last_undegraded":"2024-03-06T07:22:44.365819+0000","last_fullsized":"2024-03-06T07:22:44.365819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:49.281137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.369795+0000","last_change":"2024-03-06T07:22:44.369795+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.369795+0000","last_undegraded":"2024-03-06T07:22:44.369795+0000","last_fullsized":"2024-03-06T07:22:44.369795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364770+0000","last_change":"2024-03-06T07:22:35.040052+0000","last_active":"2024-03-06T07:22:44.364770+0000","last_peered":"2024-03-06T07:22:44.364770+0000","last_clean":"2024-03-06T07:22:44.364770+0000","last_became_active":"2024-03-06T07:22:35.039794+0000","last_became_peered":"2024-03-06T07:22:35.039794+0000","last_unstale":"2024-03-06T07:22:44.364770+0000","last_undegraded":"2024-03-06T07:22:44.364770+0000","last_fullsized":"2024-03-06T07:22:44.364770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364008+0000","last_change":"2024-03-06T07:22:44.364008+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364008+0000","last_undegraded":"2024-03-06T07:22:44.364008+0000","last_fullsized":"2024-03-06T07:22:44.364008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366659+0000","last_change":"2024-03-06T07:22:35.034108+0000","last_active":"2024-03-06T07:22:44.366659+0000","last_peered":"2024-03-06T07:22:44.366659+0000","last_clean":"2024-03-06T07:22:44.366659+0000","last_became_active":"2024-03-06T07:22:35.033823+0000","last_became_peered":"2024-03-06T07:22:35.033823+0000","last_unstale":"2024-03-06T07:22:44.366659+0000","last_undegraded":"2024-03-06T07:22:44.366659+0000","last_fullsized":"2024-03-06T07:22:44.366659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:04.872840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359980+0000","last_change":"2024-03-06T07:22:44.359980+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359980+0000","last_undegraded":"2024-03-06T07:22:44.359980+0000","last_fullsized":"2024-03-06T07:22:44.359980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363762+0000","last_change":"2024-03-06T07:22:35.037317+0000","last_active":"2024-03-06T07:22:44.363762+0000","last_peered":"2024-03-06T07:22:44.363762+0000","last_clean":"2024-03-06T07:22:44.363762+0000","last_became_active":"2024-03-06T07:22:35.033185+0000","last_became_peered":"2024-03-06T07:22:35.033185+0000","last_unstale":"2024-03-06T07:22:44.363762+0000","last_undegraded":"2024-03-06T07:22:44.363762+0000","last_fullsized":"2024-03-06T07:22:44.363762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359992+0000","last_change":"2024-03-06T07:22:44.359992+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359992+0000","last_undegraded":"2024-03-06T07:22:44.359992+0000","last_fullsized":"2024-03-06T07:22:44.359992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365447+0000","last_change":"2024-03-06T07:22:35.035073+0000","last_active":"2024-03-06T07:22:44.365447+0000","last_peered":"2024-03-06T07:22:44.365447+0000","last_clean":"2024-03-06T07:22:44.365447+0000","last_became_active":"2024-03-06T07:22:35.034739+0000","last_became_peered":"2024-03-06T07:22:35.034739+0000","last_unstale":"2024-03-06T07:22:44.365447+0000","last_undegraded":"2024-03-06T07:22:44.365447+0000","last_fullsized":"2024-03-06T07:22:44.365447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:12.718069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363938+0000","last_change":"2024-03-06T07:22:44.363938+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363938+0000","last_undegraded":"2024-03-06T07:22:44.363938+0000","last_fullsized":"2024-03-06T07:22:44.363938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359224+0000","last_change":"2024-03-06T07:22:35.032202+0000","last_active":"2024-03-06T07:22:44.359224+0000","last_peered":"2024-03-06T07:22:44.359224+0000","last_clean":"2024-03-06T07:22:44.359224+0000","last_became_active":"2024-03-06T07:22:35.031858+0000","last_became_peered":"2024-03-06T07:22:35.031858+0000","last_unstale":"2024-03-06T07:22:44.359224+0000","last_undegraded":"2024-03-06T07:22:44.359224+0000","last_fullsized":"2024-03-06T07:22:44.359224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364868+0000","last_change":"2024-03-06T07:22:44.364868+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364868+0000","last_undegraded":"2024-03-06T07:22:44.364868+0000","last_fullsized":"2024-03-06T07:22:44.364868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364576+0000","last_change":"2024-03-06T07:22:35.041928+0000","last_active":"2024-03-06T07:22:44.364576+0000","last_peered":"2024-03-06T07:22:44.364576+0000","last_clean":"2024-03-06T07:22:44.364576+0000","last_became_active":"2024-03-06T07:22:35.041672+0000","last_became_peered":"2024-03-06T07:22:35.041672+0000","last_unstale":"2024-03-06T07:22:44.364576+0000","last_undegraded":"2024-03-06T07:22:44.364576+0000","last_fullsized":"2024-03-06T07:22:44.364576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362840+0000","last_change":"2024-03-06T07:22:44.362840+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362840+0000","last_undegraded":"2024-03-06T07:22:44.362840+0000","last_fullsized":"2024-03-06T07:22:44.362840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368275+0000","last_change":"2024-03-06T07:22:35.036178+0000","last_active":"2024-03-06T07:22:44.368275+0000","last_peered":"2024-03-06T07:22:44.368275+0000","last_clean":"2024-03-06T07:22:44.368275+0000","last_became_active":"2024-03-06T07:22:35.035874+0000","last_became_peered":"2024-03-06T07:22:35.035874+0000","last_unstale":"2024-03-06T07:22:44.368275+0000","last_undegraded":"2024-03-06T07:22:44.368275+0000","last_fullsized":"2024-03-06T07:22:44.368275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.764602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364726+0000","last_change":"2024-03-06T07:22:44.364726+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364726+0000","last_undegraded":"2024-03-06T07:22:44.364726+0000","last_fullsized":"2024-03-06T07:22:44.364726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361583+0000","last_change":"2024-03-06T07:22:35.031787+0000","last_active":"2024-03-06T07:22:44.361583+0000","last_peered":"2024-03-06T07:22:44.361583+0000","last_clean":"2024-03-06T07:22:44.361583+0000","last_became_active":"2024-03-06T07:22:35.031524+0000","last_became_peered":"2024-03-06T07:22:35.031524+0000","last_unstale":"2024-03-06T07:22:44.361583+0000","last_undegraded":"2024-03-06T07:22:44.361583+0000","last_fullsized":"2024-03-06T07:22:44.361583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361747+0000","last_change":"2024-03-06T07:22:44.361747+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361747+0000","last_undegraded":"2024-03-06T07:22:44.361747+0000","last_fullsized":"2024-03-06T07:22:44.361747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363506+0000","last_change":"2024-03-06T07:22:35.039311+0000","last_active":"2024-03-06T07:22:44.363506+0000","last_peered":"2024-03-06T07:22:44.363506+0000","last_clean":"2024-03-06T07:22:44.363506+0000","last_became_active":"2024-03-06T07:22:35.038971+0000","last_became_peered":"2024-03-06T07:22:35.038971+0000","last_unstale":"2024-03-06T07:22:44.363506+0000","last_undegraded":"2024-03-06T07:22:44.363506+0000","last_fullsized":"2024-03-06T07:22:44.363506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:42.465427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359016+0000","last_change":"2024-03-06T07:22:44.359016+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359016+0000","last_undegraded":"2024-03-06T07:22:44.359016+0000","last_fullsized":"2024-03-06T07:22:44.359016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361143+0000","last_change":"2024-03-06T07:22:35.031978+0000","last_active":"2024-03-06T07:22:44.361143+0000","last_peered":"2024-03-06T07:22:44.361143+0000","last_clean":"2024-03-06T07:22:44.361143+0000","last_became_active":"2024-03-06T07:22:35.031711+0000","last_became_peered":"2024-03-06T07:22:35.031711+0000","last_unstale":"2024-03-06T07:22:44.361143+0000","last_undegraded":"2024-03-06T07:22:44.361143+0000","last_fullsized":"2024-03-06T07:22:44.361143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:17.004593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359118+0000","last_change":"2024-03-06T07:22:44.359118+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359118+0000","last_undegraded":"2024-03-06T07:22:44.359118+0000","last_fullsized":"2024-03-06T07:22:44.359118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375412+0000","last_change":"2024-03-06T07:22:35.045711+0000","last_active":"2024-03-06T07:22:44.375412+0000","last_peered":"2024-03-06T07:22:44.375412+0000","last_clean":"2024-03-06T07:22:44.375412+0000","last_became_active":"2024-03-06T07:22:35.041242+0000","last_became_peered":"2024-03-06T07:22:35.041242+0000","last_unstale":"2024-03-06T07:22:44.375412+0000","last_undegraded":"2024-03-06T07:22:44.375412+0000","last_fullsized":"2024-03-06T07:22:44.375412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:16.266604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364653+0000","last_change":"2024-03-06T07:22:44.364653+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364653+0000","last_undegraded":"2024-03-06T07:22:44.364653+0000","last_fullsized":"2024-03-06T07:22:44.364653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374041+0000","last_change":"2024-03-06T07:22:35.042301+0000","last_active":"2024-03-06T07:22:44.374041+0000","last_peered":"2024-03-06T07:22:44.374041+0000","last_clean":"2024-03-06T07:22:44.374041+0000","last_became_active":"2024-03-06T07:22:35.041172+0000","last_became_peered":"2024-03-06T07:22:35.041172+0000","last_unstale":"2024-03-06T07:22:44.374041+0000","last_undegraded":"2024-03-06T07:22:44.374041+0000","last_fullsized":"2024-03-06T07:22:44.374041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:58.974636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363518+0000","last_change":"2024-03-06T07:22:44.363518+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363518+0000","last_undegraded":"2024-03-06T07:22:44.363518+0000","last_fullsized":"2024-03-06T07:22:44.363518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365279+0000","last_change":"2024-03-06T07:22:35.038117+0000","last_active":"2024-03-06T07:22:44.365279+0000","last_peered":"2024-03-06T07:22:44.365279+0000","last_clean":"2024-03-06T07:22:44.365279+0000","last_became_active":"2024-03-06T07:22:35.037812+0000","last_became_peered":"2024-03-06T07:22:35.037812+0000","last_unstale":"2024-03-06T07:22:44.365279+0000","last_undegraded":"2024-03-06T07:22:44.365279+0000","last_fullsized":"2024-03-06T07:22:44.365279+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363196+0000","last_change":"2024-03-06T07:22:44.363196+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363196+0000","last_undegraded":"2024-03-06T07:22:44.363196+0000","last_fullsized":"2024-03-06T07:22:44.363196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368368+0000","last_change":"2024-03-06T07:22:35.033425+0000","last_active":"2024-03-06T07:22:44.368368+0000","last_peered":"2024-03-06T07:22:44.368368+0000","last_clean":"2024-03-06T07:22:44.368368+0000","last_became_active":"2024-03-06T07:22:35.032784+0000","last_became_peered":"2024-03-06T07:22:35.032784+0000","last_unstale":"2024-03-06T07:22:44.368368+0000","last_undegraded":"2024-03-06T07:22:44.368368+0000","last_fullsized":"2024-03-06T07:22:44.368368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361026+0000","last_change":"2024-03-06T07:22:44.361026+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361026+0000","last_undegraded":"2024-03-06T07:22:44.361026+0000","last_fullsized":"2024-03-06T07:22:44.361026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368367+0000","last_change":"2024-03-06T07:22:35.033473+0000","last_active":"2024-03-06T07:22:44.368367+0000","last_peered":"2024-03-06T07:22:44.368367+0000","last_clean":"2024-03-06T07:22:44.368367+0000","last_became_active":"2024-03-06T07:22:35.032407+0000","last_became_peered":"2024-03-06T07:22:35.032407+0000","last_unstale":"2024-03-06T07:22:44.368367+0000","last_undegraded":"2024-03-06T07:22:44.368367+0000","last_fullsized":"2024-03-06T07:22:44.368367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:39.792272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363072+0000","last_change":"2024-03-06T07:22:44.363072+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363072+0000","last_undegraded":"2024-03-06T07:22:44.363072+0000","last_fullsized":"2024-03-06T07:22:44.363072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370470+0000","last_change":"2024-03-06T07:22:35.048127+0000","last_active":"2024-03-06T07:22:44.370470+0000","last_peered":"2024-03-06T07:22:44.370470+0000","last_clean":"2024-03-06T07:22:44.370470+0000","last_became_active":"2024-03-06T07:22:35.047869+0000","last_became_peered":"2024-03-06T07:22:35.047869+0000","last_unstale":"2024-03-06T07:22:44.370470+0000","last_undegraded":"2024-03-06T07:22:44.370470+0000","last_fullsized":"2024-03-06T07:22:44.370470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.375194+0000","last_change":"2024-03-06T07:22:44.375194+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.375194+0000","last_undegraded":"2024-03-06T07:22:44.375194+0000","last_fullsized":"2024-03-06T07:22:44.375194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363056+0000","last_change":"2024-03-06T07:22:35.038257+0000","last_active":"2024-03-06T07:22:44.363056+0000","last_peered":"2024-03-06T07:22:44.363056+0000","last_clean":"2024-03-06T07:22:44.363056+0000","last_became_active":"2024-03-06T07:22:35.037969+0000","last_became_peered":"2024-03-06T07:22:35.037969+0000","last_unstale":"2024-03-06T07:22:44.363056+0000","last_undegraded":"2024-03-06T07:22:44.363056+0000","last_fullsized":"2024-03-06T07:22:44.363056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362412+0000","last_change":"2024-03-06T07:22:44.362412+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362412+0000","last_undegraded":"2024-03-06T07:22:44.362412+0000","last_fullsized":"2024-03-06T07:22:44.362412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358712+0000","last_change":"2024-03-06T07:22:35.032129+0000","last_active":"2024-03-06T07:22:44.358712+0000","last_peered":"2024-03-06T07:22:44.358712+0000","last_clean":"2024-03-06T07:22:44.358712+0000","last_became_active":"2024-03-06T07:22:35.027787+0000","last_became_peered":"2024-03-06T07:22:35.027787+0000","last_unstale":"2024-03-06T07:22:44.358712+0000","last_undegraded":"2024-03-06T07:22:44.358712+0000","last_fullsized":"2024-03-06T07:22:44.358712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:27.011065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359560+0000","last_change":"2024-03-06T07:22:44.359560+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359560+0000","last_undegraded":"2024-03-06T07:22:44.359560+0000","last_fullsized":"2024-03-06T07:22:44.359560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363854+0000","last_change":"2024-03-06T07:22:35.036989+0000","last_active":"2024-03-06T07:22:44.363854+0000","last_peered":"2024-03-06T07:22:44.363854+0000","last_clean":"2024-03-06T07:22:44.363854+0000","last_became_active":"2024-03-06T07:22:35.032803+0000","last_became_peered":"2024-03-06T07:22:35.032803+0000","last_unstale":"2024-03-06T07:22:44.363854+0000","last_undegraded":"2024-03-06T07:22:44.363854+0000","last_fullsized":"2024-03-06T07:22:44.363854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365367+0000","last_change":"2024-03-06T07:22:44.365367+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365367+0000","last_undegraded":"2024-03-06T07:22:44.365367+0000","last_fullsized":"2024-03-06T07:22:44.365367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373250+0000","last_change":"2024-03-06T07:22:44.372693+0000","last_active":"2024-03-06T07:22:44.373250+0000","last_peered":"2024-03-06T07:22:44.373250+0000","last_clean":"2024-03-06T07:22:44.373250+0000","last_became_active":"2024-03-06T07:22:44.372375+0000","last_became_peered":"2024-03-06T07:22:44.372375+0000","last_unstale":"2024-03-06T07:22:44.373250+0000","last_undegraded":"2024-03-06T07:22:44.373250+0000","last_fullsized":"2024-03-06T07:22:44.373250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358464+0000","last_change":"2024-03-06T07:22:35.025981+0000","last_active":"2024-03-06T07:22:44.358464+0000","last_peered":"2024-03-06T07:22:44.358464+0000","last_clean":"2024-03-06T07:22:44.358464+0000","last_became_active":"2024-03-06T07:22:35.025721+0000","last_became_peered":"2024-03-06T07:22:35.025721+0000","last_unstale":"2024-03-06T07:22:44.358464+0000","last_undegraded":"2024-03-06T07:22:44.358464+0000","last_fullsized":"2024-03-06T07:22:44.358464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:18.894659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365555+0000","last_change":"2024-03-06T07:22:34.043747+0000","last_active":"2024-03-06T07:22:44.365555+0000","last_peered":"2024-03-06T07:22:44.365555+0000","last_clean":"2024-03-06T07:22:44.365555+0000","last_became_active":"2024-03-06T07:22:34.042367+0000","last_became_peered":"2024-03-06T07:22:34.042367+0000","last_unstale":"2024-03-06T07:22:44.365555+0000","last_undegraded":"2024-03-06T07:22:44.365555+0000","last_fullsized":"2024-03-06T07:22:44.365555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:33.138544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363123+0000","last_change":"2024-03-06T07:22:44.363123+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363123+0000","last_undegraded":"2024-03-06T07:22:44.363123+0000","last_fullsized":"2024-03-06T07:22:44.363123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384852+0000","last_change":"2024-03-06T07:22:44.380429+0000","last_active":"2024-03-06T07:22:44.384852+0000","last_peered":"2024-03-06T07:22:44.384852+0000","last_clean":"2024-03-06T07:22:44.384852+0000","last_became_active":"2024-03-06T07:22:44.380170+0000","last_became_peered":"2024-03-06T07:22:44.380170+0000","last_unstale":"2024-03-06T07:22:44.384852+0000","last_undegraded":"2024-03-06T07:22:44.384852+0000","last_fullsized":"2024-03-06T07:22:44.384852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:53.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370159+0000","last_change":"2024-03-06T07:22:35.032593+0000","last_active":"2024-03-06T07:22:44.370159+0000","last_peered":"2024-03-06T07:22:44.370159+0000","last_clean":"2024-03-06T07:22:44.370159+0000","last_became_active":"2024-03-06T07:22:35.031651+0000","last_became_peered":"2024-03-06T07:22:35.031651+0000","last_unstale":"2024-03-06T07:22:44.370159+0000","last_undegraded":"2024-03-06T07:22:44.370159+0000","last_fullsized":"2024-03-06T07:22:44.370159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364514+0000","last_change":"2024-03-06T07:22:34.038702+0000","last_active":"2024-03-06T07:22:44.364514+0000","last_peered":"2024-03-06T07:22:44.364514+0000","last_clean":"2024-03-06T07:22:44.364514+0000","last_became_active":"2024-03-06T07:22:34.038165+0000","last_became_peered":"2024-03-06T07:22:34.038165+0000","last_unstale":"2024-03-06T07:22:44.364514+0000","last_undegraded":"2024-03-06T07:22:44.364514+0000","last_fullsized":"2024-03-06T07:22:44.364514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362338+0000","last_change":"2024-03-06T07:22:44.362338+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362338+0000","last_undegraded":"2024-03-06T07:22:44.362338+0000","last_fullsized":"2024-03-06T07:22:44.362338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.381941+0000","last_change":"2024-03-06T07:22:44.375256+0000","last_active":"2024-03-06T07:22:44.381941+0000","last_peered":"2024-03-06T07:22:44.381941+0000","last_clean":"2024-03-06T07:22:44.381941+0000","last_became_active":"2024-03-06T07:22:44.374987+0000","last_became_peered":"2024-03-06T07:22:44.374987+0000","last_unstale":"2024-03-06T07:22:44.381941+0000","last_undegraded":"2024-03-06T07:22:44.381941+0000","last_fullsized":"2024-03-06T07:22:44.381941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:32.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361466+0000","last_change":"2024-03-06T07:22:35.035774+0000","last_active":"2024-03-06T07:22:44.361466+0000","last_peered":"2024-03-06T07:22:44.361466+0000","last_clean":"2024-03-06T07:22:44.361466+0000","last_became_active":"2024-03-06T07:22:35.035468+0000","last_became_peered":"2024-03-06T07:22:35.035468+0000","last_unstale":"2024-03-06T07:22:44.361466+0000","last_undegraded":"2024-03-06T07:22:44.361466+0000","last_fullsized":"2024-03-06T07:22:44.361466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:25.144870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368550+0000","last_change":"2024-03-06T07:22:34.042434+0000","last_active":"2024-03-06T07:22:44.368550+0000","last_peered":"2024-03-06T07:22:44.368550+0000","last_clean":"2024-03-06T07:22:44.368550+0000","last_became_active":"2024-03-06T07:22:34.042093+0000","last_became_peered":"2024-03-06T07:22:34.042093+0000","last_unstale":"2024-03-06T07:22:44.368550+0000","last_undegraded":"2024-03-06T07:22:44.368550+0000","last_fullsized":"2024-03-06T07:22:44.368550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364352+0000","last_change":"2024-03-06T07:22:44.364352+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364352+0000","last_undegraded":"2024-03-06T07:22:44.364352+0000","last_fullsized":"2024-03-06T07:22:44.364352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375700+0000","last_change":"2024-03-06T07:22:44.373849+0000","last_active":"2024-03-06T07:22:44.375700+0000","last_peered":"2024-03-06T07:22:44.375700+0000","last_clean":"2024-03-06T07:22:44.375700+0000","last_became_active":"2024-03-06T07:22:44.373135+0000","last_became_peered":"2024-03-06T07:22:44.373135+0000","last_unstale":"2024-03-06T07:22:44.375700+0000","last_undegraded":"2024-03-06T07:22:44.375700+0000","last_fullsized":"2024-03-06T07:22:44.375700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:52.622331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361651+0000","last_change":"2024-03-06T07:22:35.032012+0000","last_active":"2024-03-06T07:22:44.361651+0000","last_peered":"2024-03-06T07:22:44.361651+0000","last_clean":"2024-03-06T07:22:44.361651+0000","last_became_active":"2024-03-06T07:22:35.031728+0000","last_became_peered":"2024-03-06T07:22:35.031728+0000","last_unstale":"2024-03-06T07:22:44.361651+0000","last_undegraded":"2024-03-06T07:22:44.361651+0000","last_fullsized":"2024-03-06T07:22:44.361651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364137+0000","last_change":"2024-03-06T07:22:34.038862+0000","last_active":"2024-03-06T07:22:44.364137+0000","last_peered":"2024-03-06T07:22:44.364137+0000","last_clean":"2024-03-06T07:22:44.364137+0000","last_became_active":"2024-03-06T07:22:34.038590+0000","last_became_peered":"2024-03-06T07:22:34.038590+0000","last_unstale":"2024-03-06T07:22:44.364137+0000","last_undegraded":"2024-03-06T07:22:44.364137+0000","last_fullsized":"2024-03-06T07:22:44.364137+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:15.997587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362381+0000","last_change":"2024-03-06T07:22:44.362381+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362381+0000","last_undegraded":"2024-03-06T07:22:44.362381+0000","last_fullsized":"2024-03-06T07:22:44.362381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376768+0000","last_change":"2024-03-06T07:22:44.375296+0000","last_active":"2024-03-06T07:22:44.376768+0000","last_peered":"2024-03-06T07:22:44.376768+0000","last_clean":"2024-03-06T07:22:44.376768+0000","last_became_active":"2024-03-06T07:22:44.375028+0000","last_became_peered":"2024-03-06T07:22:44.375028+0000","last_unstale":"2024-03-06T07:22:44.376768+0000","last_undegraded":"2024-03-06T07:22:44.376768+0000","last_fullsized":"2024-03-06T07:22:44.376768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.786058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376180+0000","last_change":"2024-03-06T07:22:35.036832+0000","last_active":"2024-03-06T07:22:44.376180+0000","last_peered":"2024-03-06T07:22:44.376180+0000","last_clean":"2024-03-06T07:22:44.376180+0000","last_became_active":"2024-03-06T07:22:35.036392+0000","last_became_peered":"2024-03-06T07:22:35.036392+0000","last_unstale":"2024-03-06T07:22:44.376180+0000","last_undegraded":"2024-03-06T07:22:44.376180+0000","last_fullsized":"2024-03-06T07:22:44.376180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:13.614365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361685+0000","last_change":"2024-03-06T07:22:34.037848+0000","last_active":"2024-03-06T07:22:44.361685+0000","last_peered":"2024-03-06T07:22:44.361685+0000","last_clean":"2024-03-06T07:22:44.361685+0000","last_became_active":"2024-03-06T07:22:34.037572+0000","last_became_peered":"2024-03-06T07:22:34.037572+0000","last_unstale":"2024-03-06T07:22:44.361685+0000","last_undegraded":"2024-03-06T07:22:44.361685+0000","last_fullsized":"2024-03-06T07:22:44.361685+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:15.748729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364294+0000","last_change":"2024-03-06T07:22:44.364294+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364294+0000","last_undegraded":"2024-03-06T07:22:44.364294+0000","last_fullsized":"2024-03-06T07:22:44.364294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379327+0000","last_change":"2024-03-06T07:22:44.375707+0000","last_active":"2024-03-06T07:22:44.379327+0000","last_peered":"2024-03-06T07:22:44.379327+0000","last_clean":"2024-03-06T07:22:44.379327+0000","last_became_active":"2024-03-06T07:22:44.375510+0000","last_became_peered":"2024-03-06T07:22:44.375510+0000","last_unstale":"2024-03-06T07:22:44.379327+0000","last_undegraded":"2024-03-06T07:22:44.379327+0000","last_fullsized":"2024-03-06T07:22:44.379327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:57.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375276+0000","last_change":"2024-03-06T07:22:35.040723+0000","last_active":"2024-03-06T07:22:44.375276+0000","last_peered":"2024-03-06T07:22:44.375276+0000","last_clean":"2024-03-06T07:22:44.375276+0000","last_became_active":"2024-03-06T07:22:35.036544+0000","last_became_peered":"2024-03-06T07:22:35.036544+0000","last_unstale":"2024-03-06T07:22:44.375276+0000","last_undegraded":"2024-03-06T07:22:44.375276+0000","last_fullsized":"2024-03-06T07:22:44.375276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:44.185335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366731+0000","last_change":"2024-03-06T07:22:34.040746+0000","last_active":"2024-03-06T07:22:44.366731+0000","last_peered":"2024-03-06T07:22:44.366731+0000","last_clean":"2024-03-06T07:22:44.366731+0000","last_became_active":"2024-03-06T07:22:34.040466+0000","last_became_peered":"2024-03-06T07:22:34.040466+0000","last_unstale":"2024-03-06T07:22:44.366731+0000","last_undegraded":"2024-03-06T07:22:44.366731+0000","last_fullsized":"2024-03-06T07:22:44.366731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:35.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367578+0000","last_change":"2024-03-06T07:22:44.367578+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367578+0000","last_undegraded":"2024-03-06T07:22:44.367578+0000","last_fullsized":"2024-03-06T07:22:44.367578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384804+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384804+0000","last_peered":"2024-03-06T07:22:44.384804+0000","last_clean":"2024-03-06T07:22:44.384804+0000","last_became_active":"2024-03-06T07:22:44.380134+0000","last_became_peered":"2024-03-06T07:22:44.380134+0000","last_unstale":"2024-03-06T07:22:44.384804+0000","last_undegraded":"2024-03-06T07:22:44.384804+0000","last_fullsized":"2024-03-06T07:22:44.384804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:13.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361680+0000","last_change":"2024-03-06T07:22:35.035207+0000","last_active":"2024-03-06T07:22:44.361680+0000","last_peered":"2024-03-06T07:22:44.361680+0000","last_clean":"2024-03-06T07:22:44.361680+0000","last_became_active":"2024-03-06T07:22:35.034887+0000","last_became_peered":"2024-03-06T07:22:35.034887+0000","last_unstale":"2024-03-06T07:22:44.361680+0000","last_undegraded":"2024-03-06T07:22:44.361680+0000","last_fullsized":"2024-03-06T07:22:44.361680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370135+0000","last_change":"2024-03-06T07:22:34.042833+0000","last_active":"2024-03-06T07:22:44.370135+0000","last_peered":"2024-03-06T07:22:44.370135+0000","last_clean":"2024-03-06T07:22:44.370135+0000","last_became_active":"2024-03-06T07:22:34.042557+0000","last_became_peered":"2024-03-06T07:22:34.042557+0000","last_unstale":"2024-03-06T07:22:44.370135+0000","last_undegraded":"2024-03-06T07:22:44.370135+0000","last_fullsized":"2024-03-06T07:22:44.370135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.372269+0000","last_change":"2024-03-06T07:22:44.372269+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.372269+0000","last_undegraded":"2024-03-06T07:22:44.372269+0000","last_fullsized":"2024-03-06T07:22:44.372269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379060+0000","last_change":"2024-03-06T07:22:44.371322+0000","last_active":"2024-03-06T07:22:44.379060+0000","last_peered":"2024-03-06T07:22:44.379060+0000","last_clean":"2024-03-06T07:22:44.379060+0000","last_became_active":"2024-03-06T07:22:44.371058+0000","last_became_peered":"2024-03-06T07:22:44.371058+0000","last_unstale":"2024-03-06T07:22:44.379060+0000","last_undegraded":"2024-03-06T07:22:44.379060+0000","last_fullsized":"2024-03-06T07:22:44.379060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:25.531656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364553+0000","last_change":"2024-03-06T07:22:35.028293+0000","last_active":"2024-03-06T07:22:44.364553+0000","last_peered":"2024-03-06T07:22:44.364553+0000","last_clean":"2024-03-06T07:22:44.364553+0000","last_became_active":"2024-03-06T07:22:35.028017+0000","last_became_peered":"2024-03-06T07:22:35.028017+0000","last_unstale":"2024-03-06T07:22:44.364553+0000","last_undegraded":"2024-03-06T07:22:44.364553+0000","last_fullsized":"2024-03-06T07:22:44.364553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:36.890851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369543+0000","last_change":"2024-03-06T07:22:34.037660+0000","last_active":"2024-03-06T07:22:44.369543+0000","last_peered":"2024-03-06T07:22:44.369543+0000","last_clean":"2024-03-06T07:22:44.369543+0000","last_became_active":"2024-03-06T07:22:34.037347+0000","last_became_peered":"2024-03-06T07:22:34.037347+0000","last_unstale":"2024-03-06T07:22:44.369543+0000","last_undegraded":"2024-03-06T07:22:44.369543+0000","last_fullsized":"2024-03-06T07:22:44.369543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:43.483410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362846+0000","last_change":"2024-03-06T07:22:44.362846+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362846+0000","last_undegraded":"2024-03-06T07:22:44.362846+0000","last_fullsized":"2024-03-06T07:22:44.362846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374719+0000","last_change":"2024-03-06T07:22:44.370568+0000","last_active":"2024-03-06T07:22:44.374719+0000","last_peered":"2024-03-06T07:22:44.374719+0000","last_clean":"2024-03-06T07:22:44.374719+0000","last_became_active":"2024-03-06T07:22:44.370050+0000","last_became_peered":"2024-03-06T07:22:44.370050+0000","last_unstale":"2024-03-06T07:22:44.374719+0000","last_undegraded":"2024-03-06T07:22:44.374719+0000","last_fullsized":"2024-03-06T07:22:44.374719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:46.225047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375512+0000","last_change":"2024-03-06T07:22:35.045241+0000","last_active":"2024-03-06T07:22:44.375512+0000","last_peered":"2024-03-06T07:22:44.375512+0000","last_clean":"2024-03-06T07:22:44.375512+0000","last_became_active":"2024-03-06T07:22:35.044953+0000","last_became_peered":"2024-03-06T07:22:35.044953+0000","last_unstale":"2024-03-06T07:22:44.375512+0000","last_undegraded":"2024-03-06T07:22:44.375512+0000","last_fullsized":"2024-03-06T07:22:44.375512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365128+0000","last_change":"2024-03-06T07:22:34.038501+0000","last_active":"2024-03-06T07:22:44.365128+0000","last_peered":"2024-03-06T07:22:44.365128+0000","last_clean":"2024-03-06T07:22:44.365128+0000","last_became_active":"2024-03-06T07:22:34.038238+0000","last_became_peered":"2024-03-06T07:22:34.038238+0000","last_unstale":"2024-03-06T07:22:44.365128+0000","last_undegraded":"2024-03-06T07:22:44.365128+0000","last_fullsized":"2024-03-06T07:22:44.365128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363829+0000","last_change":"2024-03-06T07:22:44.363829+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363829+0000","last_undegraded":"2024-03-06T07:22:44.363829+0000","last_fullsized":"2024-03-06T07:22:44.363829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379031+0000","last_change":"2024-03-06T07:22:44.377341+0000","last_active":"2024-03-06T07:22:44.379031+0000","last_peered":"2024-03-06T07:22:44.379031+0000","last_clean":"2024-03-06T07:22:44.379031+0000","last_became_active":"2024-03-06T07:22:44.377070+0000","last_became_peered":"2024-03-06T07:22:44.377070+0000","last_unstale":"2024-03-06T07:22:44.379031+0000","last_undegraded":"2024-03-06T07:22:44.379031+0000","last_fullsized":"2024-03-06T07:22:44.379031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:28.968579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365969+0000","last_change":"2024-03-06T07:22:35.035913+0000","last_active":"2024-03-06T07:22:44.365969+0000","last_peered":"2024-03-06T07:22:44.365969+0000","last_clean":"2024-03-06T07:22:44.365969+0000","last_became_active":"2024-03-06T07:22:35.035658+0000","last_became_peered":"2024-03-06T07:22:35.035658+0000","last_unstale":"2024-03-06T07:22:44.365969+0000","last_undegraded":"2024-03-06T07:22:44.365969+0000","last_fullsized":"2024-03-06T07:22:44.365969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:55.082141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365373+0000","last_change":"2024-03-06T07:22:34.040689+0000","last_active":"2024-03-06T07:22:44.365373+0000","last_peered":"2024-03-06T07:22:44.365373+0000","last_clean":"2024-03-06T07:22:44.365373+0000","last_became_active":"2024-03-06T07:22:34.040410+0000","last_became_peered":"2024-03-06T07:22:34.040410+0000","last_unstale":"2024-03-06T07:22:44.365373+0000","last_undegraded":"2024-03-06T07:22:44.365373+0000","last_fullsized":"2024-03-06T07:22:44.365373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.281130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361564+0000","last_change":"2024-03-06T07:22:44.361564+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361564+0000","last_undegraded":"2024-03-06T07:22:44.361564+0000","last_fullsized":"2024-03-06T07:22:44.361564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384844+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384844+0000","last_peered":"2024-03-06T07:22:44.384844+0000","last_clean":"2024-03-06T07:22:44.384844+0000","last_became_active":"2024-03-06T07:22:44.380123+0000","last_became_peered":"2024-03-06T07:22:44.380123+0000","last_unstale":"2024-03-06T07:22:44.384844+0000","last_undegraded":"2024-03-06T07:22:44.384844+0000","last_fullsized":"2024-03-06T07:22:44.384844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:59.118936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364927+0000","last_change":"2024-03-06T07:22:35.032615+0000","last_active":"2024-03-06T07:22:44.364927+0000","last_peered":"2024-03-06T07:22:44.364927+0000","last_clean":"2024-03-06T07:22:44.364927+0000","last_became_active":"2024-03-06T07:22:35.031167+0000","last_became_peered":"2024-03-06T07:22:35.031167+0000","last_unstale":"2024-03-06T07:22:44.364927+0000","last_undegraded":"2024-03-06T07:22:44.364927+0000","last_fullsized":"2024-03-06T07:22:44.364927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369335+0000","last_change":"2024-03-06T07:22:34.038406+0000","last_active":"2024-03-06T07:22:44.369335+0000","last_peered":"2024-03-06T07:22:44.369335+0000","last_clean":"2024-03-06T07:22:44.369335+0000","last_became_active":"2024-03-06T07:22:34.038040+0000","last_became_peered":"2024-03-06T07:22:34.038040+0000","last_unstale":"2024-03-06T07:22:44.369335+0000","last_undegraded":"2024-03-06T07:22:44.369335+0000","last_fullsized":"2024-03-06T07:22:44.369335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:26.063950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359017+0000","last_change":"2024-03-06T07:22:44.359017+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359017+0000","last_undegraded":"2024-03-06T07:22:44.359017+0000","last_fullsized":"2024-03-06T07:22:44.359017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379693+0000","last_change":"2024-03-06T07:22:44.377976+0000","last_active":"2024-03-06T07:22:44.379693+0000","last_peered":"2024-03-06T07:22:44.379693+0000","last_clean":"2024-03-06T07:22:44.379693+0000","last_became_active":"2024-03-06T07:22:44.377659+0000","last_became_peered":"2024-03-06T07:22:44.377659+0000","last_unstale":"2024-03-06T07:22:44.379693+0000","last_undegraded":"2024-03-06T07:22:44.379693+0000","last_fullsized":"2024-03-06T07:22:44.379693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:42.020403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359010+0000","last_change":"2024-03-06T07:22:35.038459+0000","last_active":"2024-03-06T07:22:44.359010+0000","last_peered":"2024-03-06T07:22:44.359010+0000","last_clean":"2024-03-06T07:22:44.359010+0000","last_became_active":"2024-03-06T07:22:35.038185+0000","last_became_peered":"2024-03-06T07:22:35.038185+0000","last_unstale":"2024-03-06T07:22:44.359010+0000","last_undegraded":"2024-03-06T07:22:44.359010+0000","last_fullsized":"2024-03-06T07:22:44.359010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364863+0000","last_change":"2024-03-06T07:22:34.037962+0000","last_active":"2024-03-06T07:22:44.364863+0000","last_peered":"2024-03-06T07:22:44.364863+0000","last_clean":"2024-03-06T07:22:44.364863+0000","last_became_active":"2024-03-06T07:22:34.037635+0000","last_became_peered":"2024-03-06T07:22:34.037635+0000","last_unstale":"2024-03-06T07:22:44.364863+0000","last_undegraded":"2024-03-06T07:22:44.364863+0000","last_fullsized":"2024-03-06T07:22:44.364863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362391+0000","last_change":"2024-03-06T07:22:44.362391+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362391+0000","last_undegraded":"2024-03-06T07:22:44.362391+0000","last_fullsized":"2024-03-06T07:22:44.362391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382003+0000","last_change":"2024-03-06T07:22:44.377361+0000","last_active":"2024-03-06T07:22:44.382003+0000","last_peered":"2024-03-06T07:22:44.382003+0000","last_clean":"2024-03-06T07:22:44.382003+0000","last_became_active":"2024-03-06T07:22:44.377101+0000","last_became_peered":"2024-03-06T07:22:44.377101+0000","last_unstale":"2024-03-06T07:22:44.382003+0000","last_undegraded":"2024-03-06T07:22:44.382003+0000","last_fullsized":"2024-03-06T07:22:44.382003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:13.917348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365752+0000","last_change":"2024-03-06T07:22:35.036674+0000","last_active":"2024-03-06T07:22:44.365752+0000","last_peered":"2024-03-06T07:22:44.365752+0000","last_clean":"2024-03-06T07:22:44.365752+0000","last_became_active":"2024-03-06T07:22:35.036400+0000","last_became_peered":"2024-03-06T07:22:35.036400+0000","last_unstale":"2024-03-06T07:22:44.365752+0000","last_undegraded":"2024-03-06T07:22:44.365752+0000","last_fullsized":"2024-03-06T07:22:44.365752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365049+0000","last_change":"2024-03-06T07:22:34.041311+0000","last_active":"2024-03-06T07:22:44.365049+0000","last_peered":"2024-03-06T07:22:44.365049+0000","last_clean":"2024-03-06T07:22:44.365049+0000","last_became_active":"2024-03-06T07:22:34.040940+0000","last_became_peered":"2024-03-06T07:22:34.040940+0000","last_unstale":"2024-03-06T07:22:44.365049+0000","last_undegraded":"2024-03-06T07:22:44.365049+0000","last_fullsized":"2024-03-06T07:22:44.365049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:24.924844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362009+0000","last_change":"2024-03-06T07:22:44.362009+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362009+0000","last_undegraded":"2024-03-06T07:22:44.362009+0000","last_fullsized":"2024-03-06T07:22:44.362009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379734+0000","last_change":"2024-03-06T07:22:44.378302+0000","last_active":"2024-03-06T07:22:44.379734+0000","last_peered":"2024-03-06T07:22:44.379734+0000","last_clean":"2024-03-06T07:22:44.379734+0000","last_became_active":"2024-03-06T07:22:44.377994+0000","last_became_peered":"2024-03-06T07:22:44.377994+0000","last_unstale":"2024-03-06T07:22:44.379734+0000","last_undegraded":"2024-03-06T07:22:44.379734+0000","last_fullsized":"2024-03-06T07:22:44.379734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:35.747227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363188+0000","last_change":"2024-03-06T07:22:35.022953+0000","last_active":"2024-03-06T07:22:44.363188+0000","last_peered":"2024-03-06T07:22:44.363188+0000","last_clean":"2024-03-06T07:22:44.363188+0000","last_became_active":"2024-03-06T07:22:35.022681+0000","last_became_peered":"2024-03-06T07:22:35.022681+0000","last_unstale":"2024-03-06T07:22:44.363188+0000","last_undegraded":"2024-03-06T07:22:44.363188+0000","last_fullsized":"2024-03-06T07:22:44.363188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363145+0000","last_change":"2024-03-06T07:22:34.037840+0000","last_active":"2024-03-06T07:22:44.363145+0000","last_peered":"2024-03-06T07:22:44.363145+0000","last_clean":"2024-03-06T07:22:44.363145+0000","last_became_active":"2024-03-06T07:22:34.037556+0000","last_became_peered":"2024-03-06T07:22:34.037556+0000","last_unstale":"2024-03-06T07:22:44.363145+0000","last_undegraded":"2024-03-06T07:22:44.363145+0000","last_fullsized":"2024-03-06T07:22:44.363145+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359660+0000","last_change":"2024-03-06T07:22:44.359660+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359660+0000","last_undegraded":"2024-03-06T07:22:44.359660+0000","last_fullsized":"2024-03-06T07:22:44.359660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379220+0000","last_change":"2024-03-06T07:22:44.372722+0000","last_active":"2024-03-06T07:22:44.379220+0000","last_peered":"2024-03-06T07:22:44.379220+0000","last_clean":"2024-03-06T07:22:44.379220+0000","last_became_active":"2024-03-06T07:22:44.372451+0000","last_became_peered":"2024-03-06T07:22:44.372451+0000","last_unstale":"2024-03-06T07:22:44.379220+0000","last_undegraded":"2024-03-06T07:22:44.379220+0000","last_fullsized":"2024-03-06T07:22:44.379220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:15.672728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366071+0000","last_change":"2024-03-06T07:22:35.034503+0000","last_active":"2024-03-06T07:22:44.366071+0000","last_peered":"2024-03-06T07:22:44.366071+0000","last_clean":"2024-03-06T07:22:44.366071+0000","last_became_active":"2024-03-06T07:22:35.034230+0000","last_became_peered":"2024-03-06T07:22:35.034230+0000","last_unstale":"2024-03-06T07:22:44.366071+0000","last_undegraded":"2024-03-06T07:22:44.366071+0000","last_fullsized":"2024-03-06T07:22:44.366071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365219+0000","last_change":"2024-03-06T07:22:34.039959+0000","last_active":"2024-03-06T07:22:44.365219+0000","last_peered":"2024-03-06T07:22:44.365219+0000","last_clean":"2024-03-06T07:22:44.365219+0000","last_became_active":"2024-03-06T07:22:34.039689+0000","last_became_peered":"2024-03-06T07:22:34.039689+0000","last_unstale":"2024-03-06T07:22:44.365219+0000","last_undegraded":"2024-03-06T07:22:44.365219+0000","last_fullsized":"2024-03-06T07:22:44.365219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:23.420842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364356+0000","last_change":"2024-03-06T07:22:44.364356+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364356+0000","last_undegraded":"2024-03-06T07:22:44.364356+0000","last_fullsized":"2024-03-06T07:22:44.364356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.372909+0000","last_change":"2024-03-06T07:22:44.371938+0000","last_active":"2024-03-06T07:22:44.372909+0000","last_peered":"2024-03-06T07:22:44.372909+0000","last_clean":"2024-03-06T07:22:44.372909+0000","last_became_active":"2024-03-06T07:22:44.371634+0000","last_became_peered":"2024-03-06T07:22:44.371634+0000","last_unstale":"2024-03-06T07:22:44.372909+0000","last_undegraded":"2024-03-06T07:22:44.372909+0000","last_fullsized":"2024-03-06T07:22:44.372909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364645+0000","last_change":"2024-03-06T07:22:35.032290+0000","last_active":"2024-03-06T07:22:44.364645+0000","last_peered":"2024-03-06T07:22:44.364645+0000","last_clean":"2024-03-06T07:22:44.364645+0000","last_became_active":"2024-03-06T07:22:35.032048+0000","last_became_peered":"2024-03-06T07:22:35.032048+0000","last_unstale":"2024-03-06T07:22:44.364645+0000","last_undegraded":"2024-03-06T07:22:44.364645+0000","last_fullsized":"2024-03-06T07:22:44.364645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:25.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375609+0000","last_change":"2024-03-06T07:22:34.043149+0000","last_active":"2024-03-06T07:22:44.375609+0000","last_peered":"2024-03-06T07:22:44.375609+0000","last_clean":"2024-03-06T07:22:44.375609+0000","last_became_active":"2024-03-06T07:22:34.042888+0000","last_became_peered":"2024-03-06T07:22:34.042888+0000","last_unstale":"2024-03-06T07:22:44.375609+0000","last_undegraded":"2024-03-06T07:22:44.375609+0000","last_fullsized":"2024-03-06T07:22:44.375609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366248+0000","last_change":"2024-03-06T07:22:44.366248+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366248+0000","last_undegraded":"2024-03-06T07:22:44.366248+0000","last_fullsized":"2024-03-06T07:22:44.366248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384733+0000","last_change":"2024-03-06T07:22:44.379879+0000","last_active":"2024-03-06T07:22:44.384733+0000","last_peered":"2024-03-06T07:22:44.384733+0000","last_clean":"2024-03-06T07:22:44.384733+0000","last_became_active":"2024-03-06T07:22:44.379611+0000","last_became_peered":"2024-03-06T07:22:44.379611+0000","last_unstale":"2024-03-06T07:22:44.384733+0000","last_undegraded":"2024-03-06T07:22:44.384733+0000","last_fullsized":"2024-03-06T07:22:44.384733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:14.207036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363632+0000","last_change":"2024-03-06T07:22:35.034984+0000","last_active":"2024-03-06T07:22:44.363632+0000","last_peered":"2024-03-06T07:22:44.363632+0000","last_clean":"2024-03-06T07:22:44.363632+0000","last_became_active":"2024-03-06T07:22:35.031303+0000","last_became_peered":"2024-03-06T07:22:35.031303+0000","last_unstale":"2024-03-06T07:22:44.363632+0000","last_undegraded":"2024-03-06T07:22:44.363632+0000","last_fullsized":"2024-03-06T07:22:44.363632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365213+0000","last_change":"2024-03-06T07:22:34.044320+0000","last_active":"2024-03-06T07:22:44.365213+0000","last_peered":"2024-03-06T07:22:44.365213+0000","last_clean":"2024-03-06T07:22:44.365213+0000","last_became_active":"2024-03-06T07:22:34.043832+0000","last_became_peered":"2024-03-06T07:22:34.043832+0000","last_unstale":"2024-03-06T07:22:44.365213+0000","last_undegraded":"2024-03-06T07:22:44.365213+0000","last_fullsized":"2024-03-06T07:22:44.365213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366779+0000","last_change":"2024-03-06T07:22:44.366779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366779+0000","last_undegraded":"2024-03-06T07:22:44.366779+0000","last_fullsized":"2024-03-06T07:22:44.366779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377398+0000","last_change":"2024-03-06T07:22:44.373965+0000","last_active":"2024-03-06T07:22:44.377398+0000","last_peered":"2024-03-06T07:22:44.377398+0000","last_clean":"2024-03-06T07:22:44.377398+0000","last_became_active":"2024-03-06T07:22:44.373617+0000","last_became_peered":"2024-03-06T07:22:44.373617+0000","last_unstale":"2024-03-06T07:22:44.377398+0000","last_undegraded":"2024-03-06T07:22:44.377398+0000","last_fullsized":"2024-03-06T07:22:44.377398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:01.683892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363411+0000","last_change":"2024-03-06T07:22:35.034203+0000","last_active":"2024-03-06T07:22:44.363411+0000","last_peered":"2024-03-06T07:22:44.363411+0000","last_clean":"2024-03-06T07:22:44.363411+0000","last_became_active":"2024-03-06T07:22:35.033903+0000","last_became_peered":"2024-03-06T07:22:35.033903+0000","last_unstale":"2024-03-06T07:22:44.363411+0000","last_undegraded":"2024-03-06T07:22:44.363411+0000","last_fullsized":"2024-03-06T07:22:44.363411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:02:14.829352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361765+0000","last_change":"2024-03-06T07:22:34.036176+0000","last_active":"2024-03-06T07:22:44.361765+0000","last_peered":"2024-03-06T07:22:44.361765+0000","last_clean":"2024-03-06T07:22:44.361765+0000","last_became_active":"2024-03-06T07:22:34.035867+0000","last_became_peered":"2024-03-06T07:22:34.035867+0000","last_unstale":"2024-03-06T07:22:44.361765+0000","last_undegraded":"2024-03-06T07:22:44.361765+0000","last_fullsized":"2024-03-06T07:22:44.361765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:28.336495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.374791+0000","last_change":"2024-03-06T07:22:44.374791+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.374791+0000","last_undegraded":"2024-03-06T07:22:44.374791+0000","last_fullsized":"2024-03-06T07:22:44.374791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376378+0000","last_change":"2024-03-06T07:22:44.369401+0000","last_active":"2024-03-06T07:22:44.376378+0000","last_peered":"2024-03-06T07:22:44.376378+0000","last_clean":"2024-03-06T07:22:44.376378+0000","last_became_active":"2024-03-06T07:22:44.369149+0000","last_became_peered":"2024-03-06T07:22:44.369149+0000","last_unstale":"2024-03-06T07:22:44.376378+0000","last_undegraded":"2024-03-06T07:22:44.376378+0000","last_fullsized":"2024-03-06T07:22:44.376378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:29.324559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363089+0000","last_change":"2024-03-06T07:22:35.038497+0000","last_active":"2024-03-06T07:22:44.363089+0000","last_peered":"2024-03-06T07:22:44.363089+0000","last_clean":"2024-03-06T07:22:44.363089+0000","last_became_active":"2024-03-06T07:22:35.038189+0000","last_became_peered":"2024-03-06T07:22:35.038189+0000","last_unstale":"2024-03-06T07:22:44.363089+0000","last_undegraded":"2024-03-06T07:22:44.363089+0000","last_fullsized":"2024-03-06T07:22:44.363089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:34.698852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369442+0000","last_change":"2024-03-06T07:22:34.038423+0000","last_active":"2024-03-06T07:22:44.369442+0000","last_peered":"2024-03-06T07:22:44.369442+0000","last_clean":"2024-03-06T07:22:44.369442+0000","last_became_active":"2024-03-06T07:22:34.038126+0000","last_became_peered":"2024-03-06T07:22:34.038126+0000","last_unstale":"2024-03-06T07:22:44.369442+0000","last_undegraded":"2024-03-06T07:22:44.369442+0000","last_fullsized":"2024-03-06T07:22:44.369442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:43.441029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364377+0000","last_change":"2024-03-06T07:22:44.364377+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364377+0000","last_undegraded":"2024-03-06T07:22:44.364377+0000","last_fullsized":"2024-03-06T07:22:44.364377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377112+0000","last_change":"2024-03-06T07:22:44.373676+0000","last_active":"2024-03-06T07:22:44.377112+0000","last_peered":"2024-03-06T07:22:44.377112+0000","last_clean":"2024-03-06T07:22:44.377112+0000","last_became_active":"2024-03-06T07:22:44.373209+0000","last_became_peered":"2024-03-06T07:22:44.373209+0000","last_unstale":"2024-03-06T07:22:44.377112+0000","last_undegraded":"2024-03-06T07:22:44.377112+0000","last_fullsized":"2024-03-06T07:22:44.377112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:05.175843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364057+0000","last_change":"2024-03-06T07:22:35.042660+0000","last_active":"2024-03-06T07:22:44.364057+0000","last_peered":"2024-03-06T07:22:44.364057+0000","last_clean":"2024-03-06T07:22:44.364057+0000","last_became_active":"2024-03-06T07:22:35.042327+0000","last_became_peered":"2024-03-06T07:22:35.042327+0000","last_unstale":"2024-03-06T07:22:44.364057+0000","last_undegraded":"2024-03-06T07:22:44.364057+0000","last_fullsized":"2024-03-06T07:22:44.364057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358735+0000","last_change":"2024-03-06T07:22:34.035505+0000","last_active":"2024-03-06T07:22:44.358735+0000","last_peered":"2024-03-06T07:22:44.358735+0000","last_clean":"2024-03-06T07:22:44.358735+0000","last_became_active":"2024-03-06T07:22:34.035257+0000","last_became_peered":"2024-03-06T07:22:34.035257+0000","last_unstale":"2024-03-06T07:22:44.358735+0000","last_undegraded":"2024-03-06T07:22:44.358735+0000","last_fullsized":"2024-03-06T07:22:44.358735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359116+0000","last_change":"2024-03-06T07:22:44.359116+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359116+0000","last_undegraded":"2024-03-06T07:22:44.359116+0000","last_fullsized":"2024-03-06T07:22:44.359116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371251+0000","last_change":"2024-03-06T07:22:44.367628+0000","last_active":"2024-03-06T07:22:44.371251+0000","last_peered":"2024-03-06T07:22:44.371251+0000","last_clean":"2024-03-06T07:22:44.371251+0000","last_became_active":"2024-03-06T07:22:44.367384+0000","last_became_peered":"2024-03-06T07:22:44.367384+0000","last_unstale":"2024-03-06T07:22:44.371251+0000","last_undegraded":"2024-03-06T07:22:44.371251+0000","last_fullsized":"2024-03-06T07:22:44.371251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:37.145482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375993+0000","last_change":"2024-03-06T07:22:35.032174+0000","last_active":"2024-03-06T07:22:44.375993+0000","last_peered":"2024-03-06T07:22:44.375993+0000","last_clean":"2024-03-06T07:22:44.375993+0000","last_became_active":"2024-03-06T07:22:35.031869+0000","last_became_peered":"2024-03-06T07:22:35.031869+0000","last_unstale":"2024-03-06T07:22:44.375993+0000","last_undegraded":"2024-03-06T07:22:44.375993+0000","last_fullsized":"2024-03-06T07:22:44.375993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:58.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364160+0000","last_change":"2024-03-06T07:22:34.037475+0000","last_active":"2024-03-06T07:22:44.364160+0000","last_peered":"2024-03-06T07:22:44.364160+0000","last_clean":"2024-03-06T07:22:44.364160+0000","last_became_active":"2024-03-06T07:22:34.037206+0000","last_became_peered":"2024-03-06T07:22:34.037206+0000","last_unstale":"2024-03-06T07:22:44.364160+0000","last_undegraded":"2024-03-06T07:22:44.364160+0000","last_fullsized":"2024-03-06T07:22:44.364160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367154+0000","last_change":"2024-03-06T07:22:44.367154+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367154+0000","last_undegraded":"2024-03-06T07:22:44.367154+0000","last_fullsized":"2024-03-06T07:22:44.367154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376583+0000","last_change":"2024-03-06T07:22:44.372573+0000","last_active":"2024-03-06T07:22:44.376583+0000","last_peered":"2024-03-06T07:22:44.376583+0000","last_clean":"2024-03-06T07:22:44.376583+0000","last_became_active":"2024-03-06T07:22:44.372321+0000","last_became_peered":"2024-03-06T07:22:44.372321+0000","last_unstale":"2024-03-06T07:22:44.376583+0000","last_undegraded":"2024-03-06T07:22:44.376583+0000","last_fullsized":"2024-03-06T07:22:44.376583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:01.508839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373616+0000","last_change":"2024-03-06T07:22:35.042271+0000","last_active":"2024-03-06T07:22:44.373616+0000","last_peered":"2024-03-06T07:22:44.373616+0000","last_clean":"2024-03-06T07:22:44.373616+0000","last_became_active":"2024-03-06T07:22:35.036189+0000","last_became_peered":"2024-03-06T07:22:35.036189+0000","last_unstale":"2024-03-06T07:22:44.373616+0000","last_undegraded":"2024-03-06T07:22:44.373616+0000","last_fullsized":"2024-03-06T07:22:44.373616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:47.064005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364659+0000","last_change":"2024-03-06T07:22:34.036505+0000","last_active":"2024-03-06T07:22:44.364659+0000","last_peered":"2024-03-06T07:22:44.364659+0000","last_clean":"2024-03-06T07:22:44.364659+0000","last_became_active":"2024-03-06T07:22:34.036198+0000","last_became_peered":"2024-03-06T07:22:34.036198+0000","last_unstale":"2024-03-06T07:22:44.364659+0000","last_undegraded":"2024-03-06T07:22:44.364659+0000","last_fullsized":"2024-03-06T07:22:44.364659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362951+0000","last_change":"2024-03-06T07:22:44.362951+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362951+0000","last_undegraded":"2024-03-06T07:22:44.362951+0000","last_fullsized":"2024-03-06T07:22:44.362951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373381+0000","last_change":"2024-03-06T07:22:44.372300+0000","last_active":"2024-03-06T07:22:44.373381+0000","last_peered":"2024-03-06T07:22:44.373381+0000","last_clean":"2024-03-06T07:22:44.373381+0000","last_became_active":"2024-03-06T07:22:44.372036+0000","last_became_peered":"2024-03-06T07:22:44.372036+0000","last_unstale":"2024-03-06T07:22:44.373381+0000","last_undegraded":"2024-03-06T07:22:44.373381+0000","last_fullsized":"2024-03-06T07:22:44.373381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364758+0000","last_change":"2024-03-06T07:22:35.031196+0000","last_active":"2024-03-06T07:22:44.364758+0000","last_peered":"2024-03-06T07:22:44.364758+0000","last_clean":"2024-03-06T07:22:44.364758+0000","last_became_active":"2024-03-06T07:22:35.030767+0000","last_became_peered":"2024-03-06T07:22:35.030767+0000","last_unstale":"2024-03-06T07:22:44.364758+0000","last_undegraded":"2024-03-06T07:22:44.364758+0000","last_fullsized":"2024-03-06T07:22:44.364758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:43.288135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363735+0000","last_change":"2024-03-06T07:22:34.034863+0000","last_active":"2024-03-06T07:22:44.363735+0000","last_peered":"2024-03-06T07:22:44.363735+0000","last_clean":"2024-03-06T07:22:44.363735+0000","last_became_active":"2024-03-06T07:22:34.034577+0000","last_became_peered":"2024-03-06T07:22:34.034577+0000","last_unstale":"2024-03-06T07:22:44.363735+0000","last_undegraded":"2024-03-06T07:22:44.363735+0000","last_fullsized":"2024-03-06T07:22:44.363735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363562+0000","last_change":"2024-03-06T07:22:44.363562+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363562+0000","last_undegraded":"2024-03-06T07:22:44.363562+0000","last_fullsized":"2024-03-06T07:22:44.363562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379268+0000","last_change":"2024-03-06T07:22:44.374457+0000","last_active":"2024-03-06T07:22:44.379268+0000","last_peered":"2024-03-06T07:22:44.379268+0000","last_clean":"2024-03-06T07:22:44.379268+0000","last_became_active":"2024-03-06T07:22:44.374059+0000","last_became_peered":"2024-03-06T07:22:44.374059+0000","last_unstale":"2024-03-06T07:22:44.379268+0000","last_undegraded":"2024-03-06T07:22:44.379268+0000","last_fullsized":"2024-03-06T07:22:44.379268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:27.741841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.360947+0000","last_change":"2024-03-06T07:22:35.036664+0000","last_active":"2024-03-06T07:22:44.360947+0000","last_peered":"2024-03-06T07:22:44.360947+0000","last_clean":"2024-03-06T07:22:44.360947+0000","last_became_active":"2024-03-06T07:22:35.036379+0000","last_became_peered":"2024-03-06T07:22:35.036379+0000","last_unstale":"2024-03-06T07:22:44.360947+0000","last_undegraded":"2024-03-06T07:22:44.360947+0000","last_fullsized":"2024-03-06T07:22:44.360947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:56.002343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366830+0000","last_change":"2024-03-06T07:22:34.044993+0000","last_active":"2024-03-06T07:22:44.366830+0000","last_peered":"2024-03-06T07:22:44.366830+0000","last_clean":"2024-03-06T07:22:44.366830+0000","last_became_active":"2024-03-06T07:22:34.044795+0000","last_became_peered":"2024-03-06T07:22:34.044795+0000","last_unstale":"2024-03-06T07:22:44.366830+0000","last_undegraded":"2024-03-06T07:22:44.366830+0000","last_fullsized":"2024-03-06T07:22:44.366830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.256142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367883+0000","last_change":"2024-03-06T07:22:44.367883+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367883+0000","last_undegraded":"2024-03-06T07:22:44.367883+0000","last_fullsized":"2024-03-06T07:22:44.367883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371628+0000","last_change":"2024-03-06T07:22:44.369924+0000","last_active":"2024-03-06T07:22:44.371628+0000","last_peered":"2024-03-06T07:22:44.371628+0000","last_clean":"2024-03-06T07:22:44.371628+0000","last_became_active":"2024-03-06T07:22:44.369676+0000","last_became_peered":"2024-03-06T07:22:44.369676+0000","last_unstale":"2024-03-06T07:22:44.371628+0000","last_undegraded":"2024-03-06T07:22:44.371628+0000","last_fullsized":"2024-03-06T07:22:44.371628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:25.600800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365427+0000","last_change":"2024-03-06T07:22:35.031505+0000","last_active":"2024-03-06T07:22:44.365427+0000","last_peered":"2024-03-06T07:22:44.365427+0000","last_clean":"2024-03-06T07:22:44.365427+0000","last_became_active":"2024-03-06T07:22:35.031241+0000","last_became_peered":"2024-03-06T07:22:35.031241+0000","last_unstale":"2024-03-06T07:22:44.365427+0000","last_undegraded":"2024-03-06T07:22:44.365427+0000","last_fullsized":"2024-03-06T07:22:44.365427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373724+0000","last_change":"2024-03-06T07:22:34.044453+0000","last_active":"2024-03-06T07:22:44.373724+0000","last_peered":"2024-03-06T07:22:44.373724+0000","last_clean":"2024-03-06T07:22:44.373724+0000","last_became_active":"2024-03-06T07:22:34.044190+0000","last_became_peered":"2024-03-06T07:22:34.044190+0000","last_unstale":"2024-03-06T07:22:44.373724+0000","last_undegraded":"2024-03-06T07:22:44.373724+0000","last_fullsized":"2024-03-06T07:22:44.373724+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367845+0000","last_change":"2024-03-06T07:22:44.367845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367845+0000","last_undegraded":"2024-03-06T07:22:44.367845+0000","last_fullsized":"2024-03-06T07:22:44.367845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382847+0000","last_change":"2024-03-06T07:22:44.382519+0000","last_active":"2024-03-06T07:22:44.382847+0000","last_peered":"2024-03-06T07:22:44.382847+0000","last_clean":"2024-03-06T07:22:44.382847+0000","last_became_active":"2024-03-06T07:22:44.382295+0000","last_became_peered":"2024-03-06T07:22:44.382295+0000","last_unstale":"2024-03-06T07:22:44.382847+0000","last_undegraded":"2024-03-06T07:22:44.382847+0000","last_fullsized":"2024-03-06T07:22:44.382847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:00.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361555+0000","last_change":"2024-03-06T07:22:35.034792+0000","last_active":"2024-03-06T07:22:44.361555+0000","last_peered":"2024-03-06T07:22:44.361555+0000","last_clean":"2024-03-06T07:22:44.361555+0000","last_became_active":"2024-03-06T07:22:35.034503+0000","last_became_peered":"2024-03-06T07:22:35.034503+0000","last_unstale":"2024-03-06T07:22:44.361555+0000","last_undegraded":"2024-03-06T07:22:44.361555+0000","last_fullsized":"2024-03-06T07:22:44.361555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368734+0000","last_change":"2024-03-06T07:22:34.037188+0000","last_active":"2024-03-06T07:22:44.368734+0000","last_peered":"2024-03-06T07:22:44.368734+0000","last_clean":"2024-03-06T07:22:44.368734+0000","last_became_active":"2024-03-06T07:22:34.036918+0000","last_became_peered":"2024-03-06T07:22:34.036918+0000","last_unstale":"2024-03-06T07:22:44.368734+0000","last_undegraded":"2024-03-06T07:22:44.368734+0000","last_fullsized":"2024-03-06T07:22:44.368734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361861+0000","last_change":"2024-03-06T07:22:44.361861+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361861+0000","last_undegraded":"2024-03-06T07:22:44.361861+0000","last_fullsized":"2024-03-06T07:22:44.361861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379180+0000","last_change":"2024-03-06T07:22:44.371466+0000","last_active":"2024-03-06T07:22:44.379180+0000","last_peered":"2024-03-06T07:22:44.379180+0000","last_clean":"2024-03-06T07:22:44.379180+0000","last_became_active":"2024-03-06T07:22:44.371190+0000","last_became_peered":"2024-03-06T07:22:44.371190+0000","last_unstale":"2024-03-06T07:22:44.379180+0000","last_undegraded":"2024-03-06T07:22:44.379180+0000","last_fullsized":"2024-03-06T07:22:44.379180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:32.601916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370567+0000","last_change":"2024-03-06T07:22:35.033263+0000","last_active":"2024-03-06T07:22:44.370567+0000","last_peered":"2024-03-06T07:22:44.370567+0000","last_clean":"2024-03-06T07:22:44.370567+0000","last_became_active":"2024-03-06T07:22:35.033005+0000","last_became_peered":"2024-03-06T07:22:35.033005+0000","last_unstale":"2024-03-06T07:22:44.370567+0000","last_undegraded":"2024-03-06T07:22:44.370567+0000","last_fullsized":"2024-03-06T07:22:44.370567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:00.137617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364936+0000","last_change":"2024-03-06T07:22:34.035735+0000","last_active":"2024-03-06T07:22:44.364936+0000","last_peered":"2024-03-06T07:22:44.364936+0000","last_clean":"2024-03-06T07:22:44.364936+0000","last_became_active":"2024-03-06T07:22:34.035459+0000","last_became_peered":"2024-03-06T07:22:34.035459+0000","last_unstale":"2024-03-06T07:22:44.364936+0000","last_undegraded":"2024-03-06T07:22:44.364936+0000","last_fullsized":"2024-03-06T07:22:44.364936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366886+0000","last_change":"2024-03-06T07:22:44.366886+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366886+0000","last_undegraded":"2024-03-06T07:22:44.366886+0000","last_fullsized":"2024-03-06T07:22:44.366886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382022+0000","last_change":"2024-03-06T07:22:44.377358+0000","last_active":"2024-03-06T07:22:44.382022+0000","last_peered":"2024-03-06T07:22:44.382022+0000","last_clean":"2024-03-06T07:22:44.382022+0000","last_became_active":"2024-03-06T07:22:44.377098+0000","last_became_peered":"2024-03-06T07:22:44.377098+0000","last_unstale":"2024-03-06T07:22:44.382022+0000","last_undegraded":"2024-03-06T07:22:44.382022+0000","last_fullsized":"2024-03-06T07:22:44.382022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:43.869151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363978+0000","last_change":"2024-03-06T07:22:35.038111+0000","last_active":"2024-03-06T07:22:44.363978+0000","last_peered":"2024-03-06T07:22:44.363978+0000","last_clean":"2024-03-06T07:22:44.363978+0000","last_became_active":"2024-03-06T07:22:35.037779+0000","last_became_peered":"2024-03-06T07:22:35.037779+0000","last_unstale":"2024-03-06T07:22:44.363978+0000","last_undegraded":"2024-03-06T07:22:44.363978+0000","last_fullsized":"2024-03-06T07:22:44.363978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.362629+0000","last_change":"2024-03-06T07:22:34.039910+0000","last_active":"2024-03-06T07:22:44.362629+0000","last_peered":"2024-03-06T07:22:44.362629+0000","last_clean":"2024-03-06T07:22:44.362629+0000","last_became_active":"2024-03-06T07:22:34.039246+0000","last_became_peered":"2024-03-06T07:22:34.039246+0000","last_unstale":"2024-03-06T07:22:44.362629+0000","last_undegraded":"2024-03-06T07:22:44.362629+0000","last_fullsized":"2024-03-06T07:22:44.362629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:41.961694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362779+0000","last_change":"2024-03-06T07:22:44.362779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362779+0000","last_undegraded":"2024-03-06T07:22:44.362779+0000","last_fullsized":"2024-03-06T07:22:44.362779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379240+0000","last_change":"2024-03-06T07:22:44.374437+0000","last_active":"2024-03-06T07:22:44.379240+0000","last_peered":"2024-03-06T07:22:44.379240+0000","last_clean":"2024-03-06T07:22:44.379240+0000","last_became_active":"2024-03-06T07:22:44.374055+0000","last_became_peered":"2024-03-06T07:22:44.374055+0000","last_unstale":"2024-03-06T07:22:44.379240+0000","last_undegraded":"2024-03-06T07:22:44.379240+0000","last_fullsized":"2024-03-06T07:22:44.379240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:53:05.282920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363282+0000","last_change":"2024-03-06T07:22:35.027657+0000","last_active":"2024-03-06T07:22:44.363282+0000","last_peered":"2024-03-06T07:22:44.363282+0000","last_clean":"2024-03-06T07:22:44.363282+0000","last_became_active":"2024-03-06T07:22:35.027404+0000","last_became_peered":"2024-03-06T07:22:35.027404+0000","last_unstale":"2024-03-06T07:22:44.363282+0000","last_undegraded":"2024-03-06T07:22:44.363282+0000","last_fullsized":"2024-03-06T07:22:44.363282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:34.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364859+0000","last_change":"2024-03-06T07:22:34.039209+0000","last_active":"2024-03-06T07:22:44.364859+0000","last_peered":"2024-03-06T07:22:44.364859+0000","last_clean":"2024-03-06T07:22:44.364859+0000","last_became_active":"2024-03-06T07:22:34.038921+0000","last_became_peered":"2024-03-06T07:22:34.038921+0000","last_unstale":"2024-03-06T07:22:44.364859+0000","last_undegraded":"2024-03-06T07:22:44.364859+0000","last_fullsized":"2024-03-06T07:22:44.364859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:12.359435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363182+0000","last_change":"2024-03-06T07:22:44.363182+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363182+0000","last_undegraded":"2024-03-06T07:22:44.363182+0000","last_fullsized":"2024-03-06T07:22:44.363182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376720+0000","last_change":"2024-03-06T07:22:44.374940+0000","last_active":"2024-03-06T07:22:44.376720+0000","last_peered":"2024-03-06T07:22:44.376720+0000","last_clean":"2024-03-06T07:22:44.376720+0000","last_became_active":"2024-03-06T07:22:44.374653+0000","last_became_peered":"2024-03-06T07:22:44.374653+0000","last_unstale":"2024-03-06T07:22:44.376720+0000","last_undegraded":"2024-03-06T07:22:44.376720+0000","last_fullsized":"2024-03-06T07:22:44.376720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359439+0000","last_change":"2024-03-06T07:22:35.038010+0000","last_active":"2024-03-06T07:22:44.359439+0000","last_peered":"2024-03-06T07:22:44.359439+0000","last_clean":"2024-03-06T07:22:44.359439+0000","last_became_active":"2024-03-06T07:22:35.037717+0000","last_became_peered":"2024-03-06T07:22:35.037717+0000","last_unstale":"2024-03-06T07:22:44.359439+0000","last_undegraded":"2024-03-06T07:22:44.359439+0000","last_fullsized":"2024-03-06T07:22:44.359439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365513+0000","last_change":"2024-03-06T07:22:34.039679+0000","last_active":"2024-03-06T07:22:44.365513+0000","last_peered":"2024-03-06T07:22:44.365513+0000","last_clean":"2024-03-06T07:22:44.365513+0000","last_became_active":"2024-03-06T07:22:34.039092+0000","last_became_peered":"2024-03-06T07:22:34.039092+0000","last_unstale":"2024-03-06T07:22:44.365513+0000","last_undegraded":"2024-03-06T07:22:44.365513+0000","last_fullsized":"2024-03-06T07:22:44.365513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.370845+0000","last_change":"2024-03-06T07:22:44.370845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.370845+0000","last_undegraded":"2024-03-06T07:22:44.370845+0000","last_fullsized":"2024-03-06T07:22:44.370845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373582+0000","last_change":"2024-03-06T07:22:44.369327+0000","last_active":"2024-03-06T07:22:44.373582+0000","last_peered":"2024-03-06T07:22:44.373582+0000","last_clean":"2024-03-06T07:22:44.373582+0000","last_became_active":"2024-03-06T07:22:44.369076+0000","last_became_peered":"2024-03-06T07:22:44.369076+0000","last_unstale":"2024-03-06T07:22:44.373582+0000","last_undegraded":"2024-03-06T07:22:44.373582+0000","last_fullsized":"2024-03-06T07:22:44.373582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:52.087902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359257+0000","last_change":"2024-03-06T07:22:35.026900+0000","last_active":"2024-03-06T07:22:44.359257+0000","last_peered":"2024-03-06T07:22:44.359257+0000","last_clean":"2024-03-06T07:22:44.359257+0000","last_became_active":"2024-03-06T07:22:35.026626+0000","last_became_peered":"2024-03-06T07:22:35.026626+0000","last_unstale":"2024-03-06T07:22:44.359257+0000","last_undegraded":"2024-03-06T07:22:44.359257+0000","last_fullsized":"2024-03-06T07:22:44.359257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358578+0000","last_change":"2024-03-06T07:22:34.039382+0000","last_active":"2024-03-06T07:22:44.358578+0000","last_peered":"2024-03-06T07:22:44.358578+0000","last_clean":"2024-03-06T07:22:44.358578+0000","last_became_active":"2024-03-06T07:22:34.039088+0000","last_became_peered":"2024-03-06T07:22:34.039088+0000","last_unstale":"2024-03-06T07:22:44.358578+0000","last_undegraded":"2024-03-06T07:22:44.358578+0000","last_fullsized":"2024-03-06T07:22:44.358578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363505+0000","last_change":"2024-03-06T07:22:44.363505+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363505+0000","last_undegraded":"2024-03-06T07:22:44.363505+0000","last_fullsized":"2024-03-06T07:22:44.363505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375571+0000","last_change":"2024-03-06T07:22:44.374409+0000","last_active":"2024-03-06T07:22:44.375571+0000","last_peered":"2024-03-06T07:22:44.375571+0000","last_clean":"2024-03-06T07:22:44.375571+0000","last_became_active":"2024-03-06T07:22:44.374130+0000","last_became_peered":"2024-03-06T07:22:44.374130+0000","last_unstale":"2024-03-06T07:22:44.375571+0000","last_undegraded":"2024-03-06T07:22:44.375571+0000","last_fullsized":"2024-03-06T07:22:44.375571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:30.597343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368651+0000","last_change":"2024-03-06T07:22:35.033671+0000","last_active":"2024-03-06T07:22:44.368651+0000","last_peered":"2024-03-06T07:22:44.368651+0000","last_clean":"2024-03-06T07:22:44.368651+0000","last_became_active":"2024-03-06T07:22:35.033167+0000","last_became_peered":"2024-03-06T07:22:35.033167+0000","last_unstale":"2024-03-06T07:22:44.368651+0000","last_undegraded":"2024-03-06T07:22:44.368651+0000","last_fullsized":"2024-03-06T07:22:44.368651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365069+0000","last_change":"2024-03-06T07:22:34.037026+0000","last_active":"2024-03-06T07:22:44.365069+0000","last_peered":"2024-03-06T07:22:44.365069+0000","last_clean":"2024-03-06T07:22:44.365069+0000","last_became_active":"2024-03-06T07:22:34.036742+0000","last_became_peered":"2024-03-06T07:22:34.036742+0000","last_unstale":"2024-03-06T07:22:44.365069+0000","last_undegraded":"2024-03-06T07:22:44.365069+0000","last_fullsized":"2024-03-06T07:22:44.365069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363945+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:22:44.363945+0000","last_peered":"2024-03-06T07:22:44.363945+0000","last_clean":"2024-03-06T07:22:44.363945+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:22:44.363945+0000","last_undegraded":"2024-03-06T07:22:44.363945+0000","last_fullsized":"2024-03-06T07:22:44.363945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":1032479,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":443701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":442123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":443805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":435149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":442831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":442201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":1032457,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:49.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-03-06T07:22:49.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.361902+0000', 'last_change': '2024-03-06T07:22:44.361902+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.361902+0000', 'last_undegraded': '2024-03-06T07:22:44.361902+0000', 'last_fullsized': '2024-03-06T07:22:44.361902+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:49.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-03-06T07:22:49.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.366504+0000', 'last_change': '2024-03-06T07:22:44.366504+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.366504+0000', 'last_undegraded': '2024-03-06T07:22:44.366504+0000', 'last_fullsized': '2024-03-06T07:22:44.366504+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362319+0000', 'last_change': '2024-03-06T07:22:44.362319+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362319+0000', 'last_undegraded': '2024-03-06T07:22:44.362319+0000', 'last_fullsized': '2024-03-06T07:22:44.362319+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362908+0000', 'last_change': '2024-03-06T07:22:44.362908+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362908+0000', 'last_undegraded': '2024-03-06T07:22:44.362908+0000', 'last_fullsized': '2024-03-06T07:22:44.362908+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362767+0000', 'last_change': '2024-03-06T07:22:44.362767+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362767+0000', 'last_undegraded': '2024-03-06T07:22:44.362767+0000', 'last_fullsized': '2024-03-06T07:22:44.362767+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.365929+0000', 'last_change': '2024-03-06T07:22:44.365929+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.365929+0000', 'last_undegraded': '2024-03-06T07:22:44.365929+0000', 'last_fullsized': '2024-03-06T07:22:44.365929+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-03-06T07:22:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.360301+0000', 'last_change': '2024-03-06T07:22:44.360301+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.360301+0000', 'last_undegraded': '2024-03-06T07:22:44.360301+0000', 'last_fullsized': '2024-03-06T07:22:44.360301+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.366763+0000', 'last_change': '2024-03-06T07:22:44.366763+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.366763+0000', 'last_undegraded': '2024-03-06T07:22:44.366763+0000', 'last_fullsized': '2024-03-06T07:22:44.366763+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.360342+0000', 'last_change': '2024-03-06T07:22:44.360342+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.360342+0000', 'last_undegraded': '2024-03-06T07:22:44.360342+0000', 'last_fullsized': '2024-03-06T07:22:44.360342+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364257+0000', 'last_change': '2024-03-06T07:22:44.364257+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364257+0000', 'last_undegraded': '2024-03-06T07:22:44.364257+0000', 'last_fullsized': '2024-03-06T07:22:44.364257+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359274+0000', 'last_change': '2024-03-06T07:22:44.359274+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359274+0000', 'last_undegraded': '2024-03-06T07:22:44.359274+0000', 'last_fullsized': '2024-03-06T07:22:44.359274+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362088+0000', 'last_change': '2024-03-06T07:22:44.362088+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362088+0000', 'last_undegraded': '2024-03-06T07:22:44.362088+0000', 'last_fullsized': '2024-03-06T07:22:44.362088+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362839+0000', 'last_change': '2024-03-06T07:22:44.362839+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362839+0000', 'last_undegraded': '2024-03-06T07:22:44.362839+0000', 'last_fullsized': '2024-03-06T07:22:44.362839+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.369795+0000', 'last_change': '2024-03-06T07:22:44.369795+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.369795+0000', 'last_undegraded': '2024-03-06T07:22:44.369795+0000', 'last_fullsized': '2024-03-06T07:22:44.369795+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364008+0000', 'last_change': '2024-03-06T07:22:44.364008+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364008+0000', 'last_undegraded': '2024-03-06T07:22:44.364008+0000', 'last_fullsized': '2024-03-06T07:22:44.364008+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-03-06T07:22:49.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359980+0000', 'last_change': '2024-03-06T07:22:44.359980+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359980+0000', 'last_undegraded': '2024-03-06T07:22:44.359980+0000', 'last_fullsized': '2024-03-06T07:22:44.359980+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359992+0000', 'last_change': '2024-03-06T07:22:44.359992+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359992+0000', 'last_undegraded': '2024-03-06T07:22:44.359992+0000', 'last_fullsized': '2024-03-06T07:22:44.359992+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363938+0000', 'last_change': '2024-03-06T07:22:44.363938+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363938+0000', 'last_undegraded': '2024-03-06T07:22:44.363938+0000', 'last_fullsized': '2024-03-06T07:22:44.363938+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364868+0000', 'last_change': '2024-03-06T07:22:44.364868+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364868+0000', 'last_undegraded': '2024-03-06T07:22:44.364868+0000', 'last_fullsized': '2024-03-06T07:22:44.364868+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362840+0000', 'last_change': '2024-03-06T07:22:44.362840+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362840+0000', 'last_undegraded': '2024-03-06T07:22:44.362840+0000', 'last_fullsized': '2024-03-06T07:22:44.362840+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-03-06T07:22:49.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364726+0000', 'last_change': '2024-03-06T07:22:44.364726+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364726+0000', 'last_undegraded': '2024-03-06T07:22:44.364726+0000', 'last_fullsized': '2024-03-06T07:22:44.364726+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.361747+0000', 'last_change': '2024-03-06T07:22:44.361747+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.361747+0000', 'last_undegraded': '2024-03-06T07:22:44.361747+0000', 'last_fullsized': '2024-03-06T07:22:44.361747+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359016+0000', 'last_change': '2024-03-06T07:22:44.359016+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359016+0000', 'last_undegraded': '2024-03-06T07:22:44.359016+0000', 'last_fullsized': '2024-03-06T07:22:44.359016+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359118+0000', 'last_change': '2024-03-06T07:22:44.359118+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359118+0000', 'last_undegraded': '2024-03-06T07:22:44.359118+0000', 'last_fullsized': '2024-03-06T07:22:44.359118+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364653+0000', 'last_change': '2024-03-06T07:22:44.364653+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364653+0000', 'last_undegraded': '2024-03-06T07:22:44.364653+0000', 'last_fullsized': '2024-03-06T07:22:44.364653+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-03-06T07:22:49.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363518+0000', 'last_change': '2024-03-06T07:22:44.363518+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363518+0000', 'last_undegraded': '2024-03-06T07:22:44.363518+0000', 'last_fullsized': '2024-03-06T07:22:44.363518+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363196+0000', 'last_change': '2024-03-06T07:22:44.363196+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363196+0000', 'last_undegraded': '2024-03-06T07:22:44.363196+0000', 'last_fullsized': '2024-03-06T07:22:44.363196+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.361026+0000', 'last_change': '2024-03-06T07:22:44.361026+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.361026+0000', 'last_undegraded': '2024-03-06T07:22:44.361026+0000', 'last_fullsized': '2024-03-06T07:22:44.361026+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363072+0000', 'last_change': '2024-03-06T07:22:44.363072+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363072+0000', 'last_undegraded': '2024-03-06T07:22:44.363072+0000', 'last_fullsized': '2024-03-06T07:22:44.363072+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-03-06T07:22:49.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.375194+0000', 'last_change': '2024-03-06T07:22:44.375194+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.375194+0000', 'last_undegraded': '2024-03-06T07:22:44.375194+0000', 'last_fullsized': '2024-03-06T07:22:44.375194+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362412+0000', 'last_change': '2024-03-06T07:22:44.362412+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362412+0000', 'last_undegraded': '2024-03-06T07:22:44.362412+0000', 'last_fullsized': '2024-03-06T07:22:44.362412+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359560+0000', 'last_change': '2024-03-06T07:22:44.359560+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359560+0000', 'last_undegraded': '2024-03-06T07:22:44.359560+0000', 'last_fullsized': '2024-03-06T07:22:44.359560+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.365367+0000', 'last_change': '2024-03-06T07:22:44.365367+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.365367+0000', 'last_undegraded': '2024-03-06T07:22:44.365367+0000', 'last_fullsized': '2024-03-06T07:22:44.365367+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363123+0000', 'last_change': '2024-03-06T07:22:44.363123+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363123+0000', 'last_undegraded': '2024-03-06T07:22:44.363123+0000', 'last_fullsized': '2024-03-06T07:22:44.363123+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362338+0000', 'last_change': '2024-03-06T07:22:44.362338+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362338+0000', 'last_undegraded': '2024-03-06T07:22:44.362338+0000', 'last_fullsized': '2024-03-06T07:22:44.362338+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364352+0000', 'last_change': '2024-03-06T07:22:44.364352+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364352+0000', 'last_undegraded': '2024-03-06T07:22:44.364352+0000', 'last_fullsized': '2024-03-06T07:22:44.364352+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362381+0000', 'last_change': '2024-03-06T07:22:44.362381+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362381+0000', 'last_undegraded': '2024-03-06T07:22:44.362381+0000', 'last_fullsized': '2024-03-06T07:22:44.362381+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364294+0000', 'last_change': '2024-03-06T07:22:44.364294+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364294+0000', 'last_undegraded': '2024-03-06T07:22:44.364294+0000', 'last_fullsized': '2024-03-06T07:22:44.364294+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.367578+0000', 'last_change': '2024-03-06T07:22:44.367578+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.367578+0000', 'last_undegraded': '2024-03-06T07:22:44.367578+0000', 'last_fullsized': '2024-03-06T07:22:44.367578+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-03-06T07:22:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.372269+0000', 'last_change': '2024-03-06T07:22:44.372269+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.372269+0000', 'last_undegraded': '2024-03-06T07:22:44.372269+0000', 'last_fullsized': '2024-03-06T07:22:44.372269+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362846+0000', 'last_change': '2024-03-06T07:22:44.362846+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362846+0000', 'last_undegraded': '2024-03-06T07:22:44.362846+0000', 'last_fullsized': '2024-03-06T07:22:44.362846+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363829+0000', 'last_change': '2024-03-06T07:22:44.363829+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363829+0000', 'last_undegraded': '2024-03-06T07:22:44.363829+0000', 'last_fullsized': '2024-03-06T07:22:44.363829+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.361564+0000', 'last_change': '2024-03-06T07:22:44.361564+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.361564+0000', 'last_undegraded': '2024-03-06T07:22:44.361564+0000', 'last_fullsized': '2024-03-06T07:22:44.361564+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359017+0000', 'last_change': '2024-03-06T07:22:44.359017+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359017+0000', 'last_undegraded': '2024-03-06T07:22:44.359017+0000', 'last_fullsized': '2024-03-06T07:22:44.359017+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-03-06T07:22:49.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362391+0000', 'last_change': '2024-03-06T07:22:44.362391+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362391+0000', 'last_undegraded': '2024-03-06T07:22:44.362391+0000', 'last_fullsized': '2024-03-06T07:22:44.362391+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362009+0000', 'last_change': '2024-03-06T07:22:44.362009+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362009+0000', 'last_undegraded': '2024-03-06T07:22:44.362009+0000', 'last_fullsized': '2024-03-06T07:22:44.362009+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359660+0000', 'last_change': '2024-03-06T07:22:44.359660+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359660+0000', 'last_undegraded': '2024-03-06T07:22:44.359660+0000', 'last_fullsized': '2024-03-06T07:22:44.359660+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364356+0000', 'last_change': '2024-03-06T07:22:44.364356+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364356+0000', 'last_undegraded': '2024-03-06T07:22:44.364356+0000', 'last_fullsized': '2024-03-06T07:22:44.364356+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.366248+0000', 'last_change': '2024-03-06T07:22:44.366248+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.366248+0000', 'last_undegraded': '2024-03-06T07:22:44.366248+0000', 'last_fullsized': '2024-03-06T07:22:44.366248+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-03-06T07:22:49.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.366779+0000', 'last_change': '2024-03-06T07:22:44.366779+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.366779+0000', 'last_undegraded': '2024-03-06T07:22:44.366779+0000', 'last_fullsized': '2024-03-06T07:22:44.366779+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.374791+0000', 'last_change': '2024-03-06T07:22:44.374791+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.374791+0000', 'last_undegraded': '2024-03-06T07:22:44.374791+0000', 'last_fullsized': '2024-03-06T07:22:44.374791+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.364377+0000', 'last_change': '2024-03-06T07:22:44.364377+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.364377+0000', 'last_undegraded': '2024-03-06T07:22:44.364377+0000', 'last_fullsized': '2024-03-06T07:22:44.364377+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.359116+0000', 'last_change': '2024-03-06T07:22:44.359116+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.359116+0000', 'last_undegraded': '2024-03-06T07:22:44.359116+0000', 'last_fullsized': '2024-03-06T07:22:44.359116+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.367154+0000', 'last_change': '2024-03-06T07:22:44.367154+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.367154+0000', 'last_undegraded': '2024-03-06T07:22:44.367154+0000', 'last_fullsized': '2024-03-06T07:22:44.367154+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-03-06T07:22:49.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362951+0000', 'last_change': '2024-03-06T07:22:44.362951+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362951+0000', 'last_undegraded': '2024-03-06T07:22:44.362951+0000', 'last_fullsized': '2024-03-06T07:22:44.362951+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363562+0000', 'last_change': '2024-03-06T07:22:44.363562+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363562+0000', 'last_undegraded': '2024-03-06T07:22:44.363562+0000', 'last_fullsized': '2024-03-06T07:22:44.363562+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.367883+0000', 'last_change': '2024-03-06T07:22:44.367883+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.367883+0000', 'last_undegraded': '2024-03-06T07:22:44.367883+0000', 'last_fullsized': '2024-03-06T07:22:44.367883+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.367845+0000', 'last_change': '2024-03-06T07:22:44.367845+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.367845+0000', 'last_undegraded': '2024-03-06T07:22:44.367845+0000', 'last_fullsized': '2024-03-06T07:22:44.367845+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.361861+0000', 'last_change': '2024-03-06T07:22:44.361861+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.361861+0000', 'last_undegraded': '2024-03-06T07:22:44.361861+0000', 'last_fullsized': '2024-03-06T07:22:44.361861+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-03-06T07:22:49.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.366886+0000', 'last_change': '2024-03-06T07:22:44.366886+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.366886+0000', 'last_undegraded': '2024-03-06T07:22:44.366886+0000', 'last_fullsized': '2024-03-06T07:22:44.366886+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.362779+0000', 'last_change': '2024-03-06T07:22:44.362779+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.362779+0000', 'last_undegraded': '2024-03-06T07:22:44.362779+0000', 'last_fullsized': '2024-03-06T07:22:44.362779+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363182+0000', 'last_change': '2024-03-06T07:22:44.363182+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363182+0000', 'last_undegraded': '2024-03-06T07:22:44.363182+0000', 'last_fullsized': '2024-03-06T07:22:44.363182+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.370845+0000', 'last_change': '2024-03-06T07:22:44.370845+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.370845+0000', 'last_undegraded': '2024-03-06T07:22:44.370845+0000', 'last_fullsized': '2024-03-06T07:22:44.370845+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-03-06T07:22:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:22:44.363505+0000', 'last_change': '2024-03-06T07:22:44.363505+0000', 'last_active': '2024-03-06T07:22:44.351281+0000', 'last_peered': '2024-03-06T07:22:44.351281+0000', 'last_clean': '2024-03-06T07:22:44.351281+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:44.363505+0000', 'last_undegraded': '2024-03-06T07:22:44.363505+0000', 'last_fullsized': '2024-03-06T07:22:44.363505+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:44.351281+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:22:49.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.286+0000 7f33f337a640 1 Processor -- start 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.286+0000 7f33f337a640 1 -- start start 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f337a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33ec0c88c0 0x7f33ec155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f337a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33ec158b60 0x7f33ec158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f337a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ec0c73d0 con 0x7f33ec14f360 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f337a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ec0c8dd0 con 0x7f33ec0c88c0 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f337a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ec0be2e0 con 0x7f33ec158b60 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f0915640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33ec0c88c0 0x7f33ec155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f0915640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33ec0c88c0 0x7f33ec155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49192/0 (socket says 172.21.15.26:49192) 2024-03-06T07:22:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f1116640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f33ec14f360 0x7f33ec0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57516/0 (socket says 172.21.15.26:57516) 2024-03-06T07:22:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f0915640 1 -- 172.21.15.26:0/15277052 learned_addr learned my addr 172.21.15.26:0/15277052 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f1917640 1 --2- 172.21.15.26:0/15277052 >> v2:172.21.15.117:3300/0 conn(0x7f33ec158b60 0x7f33ec158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f0915640 1 -- 172.21.15.26:0/15277052 >> v2:172.21.15.117:3300/0 conn(0x7f33ec158b60 msgr2=0x7f33ec158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f0915640 1 --2- 172.21.15.26:0/15277052 >> v2:172.21.15.117:3300/0 conn(0x7f33ec158b60 0x7f33ec158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f0915640 1 -- 172.21.15.26:0/15277052 >> v1:172.21.15.26:6789/0 conn(0x7f33ec14f360 legacy=0x7f33ec0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.287+0000 7f33f0915640 1 -- 172.21.15.26:0/15277052 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0c7670 con 0x7f33ec0c88c0 2024-03-06T07:22:49.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.288+0000 7f33f0915640 1 --2- 172.21.15.26:0/15277052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec0c88c0 0x7f33ec155c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f33ec152d60 tx=0x7f33dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e23f7b37e55709 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.288+0000 7f33e3fff640 1 -- 172.21.15.26:0/15277052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc035020 con 0x7f33ec0c88c0 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33e3fff640 1 -- 172.21.15.26:0/15277052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc031870 con 0x7f33ec0c88c0 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33e3fff640 1 -- 172.21.15.26:0/15277052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc031c40 con 0x7f33ec0c88c0 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 -- 172.21.15.26:0/15277052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec0c88c0 msgr2=0x7f33ec155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 --2- 172.21.15.26:0/15277052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec0c88c0 0x7f33ec155c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f33ec152d60 tx=0x7f33dc02b910 comp rx=0 tx=0).stop 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 -- 172.21.15.26:0/15277052 shutdown_connections 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 --2- 172.21.15.26:0/15277052 >> v2:172.21.15.117:3300/0 conn(0x7f33ec158b60 0x7f33ec158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 --2- 172.21.15.26:0/15277052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec0c88c0 0x7f33ec155c10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 -- 172.21.15.26:0/15277052 >> 172.21.15.26:0/15277052 conn(0x7f33ec0ac570 msgr2=0x7f33ec0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 -- 172.21.15.26:0/15277052 shutdown_connections 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.289+0000 7f33f337a640 1 -- 172.21.15.26:0/15277052 wait complete. 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.290+0000 7f33f337a640 1 Processor -- start 2024-03-06T07:22:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.290+0000 7f33f337a640 1 -- start start 2024-03-06T07:22:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f337a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33ec0c88c0 0x7f33ec0c0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f337a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33ec14f360 0x7f33ec0bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f337a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec158b60 0x7f33ec0bf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f337a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c5a50 con 0x7f33ec158b60 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f337a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c7c30 con 0x7f33ec0c88c0 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f337a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c5bd0 con 0x7f33ec14f360 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec158b60 0x7f33ec0bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec158b60 0x7f33ec0bf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49196/0 (socket says 172.21.15.26:49196) 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 -- 172.21.15.26:0/1273970264 learned_addr learned my addr 172.21.15.26:0/1273970264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1116640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33ec0c88c0 0x7f33ec0c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f0915640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33ec14f360 0x7f33ec0bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 -- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33ec14f360 msgr2=0x7f33ec0bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33ec14f360 0x7f33ec0bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 -- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33ec0c88c0 msgr2=0x7f33ec0c0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33ec0c88c0 0x7f33ec0c0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.291+0000 7f33f1917640 1 -- 172.21.15.26:0/1273970264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0be620 con 0x7f33ec158b60 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.292+0000 7f33f1917640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec158b60 0x7f33ec0bf920 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f33e400efc0 tx=0x7f33e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.292+0000 7f33e1ffb640 1 -- 172.21.15.26:0/1273970264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4018020 con 0x7f33ec158b60 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.292+0000 7f33e1ffb640 1 -- 172.21.15.26:0/1273970264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e4014480 con 0x7f33ec158b60 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.292+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec0be900 con 0x7f33ec158b60 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.292+0000 7f33e1ffb640 1 -- 172.21.15.26:0/1273970264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4014870 con 0x7f33ec158b60 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.292+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec0beec0 con 0x7f33ec158b60 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.293+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ec0bf0c0 con 0x7f33ec158b60 2024-03-06T07:22:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.293+0000 7f33e1ffb640 1 -- 172.21.15.26:0/1273970264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33e4005d10 con 0x7f33ec158b60 2024-03-06T07:22:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.294+0000 7f33e1ffb640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33d006b4d0 0x7f33d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.294+0000 7f33e1ffb640 1 -- 172.21.15.26:0/1273970264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f33e408b020 con 0x7f33ec158b60 2024-03-06T07:22:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.294+0000 7f33f1116640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33d006b4d0 0x7f33d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.295+0000 7f33f1116640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33d006b4d0 0x7f33d006d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0c8280 tx=0x7f33e8007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.298+0000 7f33e1ffb640 1 -- 172.21.15.26:0/1273970264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33ec0bf0c0 con 0x7f33ec158b60 2024-03-06T07:22:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.652+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f33ec04f330 con 0x7f33ec158b60 2024-03-06T07:22:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.653+0000 7f33e1ffb640 1 -- 172.21.15.26:0/1273970264 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1557 (secure 0 0 0) 0x7f33e4022070 con 0x7f33ec158b60 2024-03-06T07:22:49.654 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:49.654 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":184,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":78,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709595,"num_in_osds":8,"osd_in_since":1709709586,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129},{"state_name":"creating+peering","count":64}],"num_pgs":193,"num_pools":5,"num_objects":24,"data_bytes":592778,"bytes_used":250298368,"bytes_avail":772843814912,"bytes_total":773094113280,"inactive_pgs_ratio":0.33160620927810669},"fsmap":{"epoch":58,"id":5,"up":1,"in":1,"max":1,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":6327},{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":6339}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-03-06T07:22:17.057794+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:22:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33d006b4d0 msgr2=0x7f33d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33d006b4d0 0x7f33d006d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0c8280 tx=0x7f33e8007e80 comp rx=0 tx=0).stop 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec158b60 msgr2=0x7f33ec0bf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec158b60 0x7f33ec0bf920 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f33e400efc0 tx=0x7f33e400c490 comp rx=0 tx=0).stop 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 shutdown_connections 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33d006b4d0 0x7f33d006d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33ec158b60 0x7f33ec0bf920 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33ec14f360 0x7f33ec0bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 --2- 172.21.15.26:0/1273970264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33ec0c88c0 0x7f33ec0c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 >> 172.21.15.26:0/1273970264 conn(0x7f33ec0ac570 msgr2=0x7f33ec14cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 shutdown_connections 2024-03-06T07:22:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.655+0000 7f33f337a640 1 -- 172.21.15.26:0/1273970264 wait complete. 2024-03-06T07:22:49.667 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:22:49.668 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.781+0000 7fc523156640 1 Processor -- start 2024-03-06T07:22:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.781+0000 7fc523156640 1 -- start start 2024-03-06T07:22:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc523156640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc51c06d9f0 0x7fc51c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc523156640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc51c0c3280 0x7fc51c0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc523156640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c158800 con 0x7fc51c1551e0 2024-03-06T07:22:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc523156640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c158ae0 con 0x7fc51c06d9f0 2024-03-06T07:22:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc523156640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c158dc0 con 0x7fc51c0c3280 2024-03-06T07:22:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc520ef2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc51c06d9f0 0x7fc51c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc5216f3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc51c1551e0 0x7fc51c1555f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57528/0 (socket says 172.21.15.26:57528) 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc520ef2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc51c06d9f0 0x7fc51c0a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49198/0 (socket says 172.21.15.26:49198) 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc5216f3640 1 -- 172.21.15.26:0/3037614815 learned_addr learned my addr 172.21.15.26:0/3037614815 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc51bfff640 1 --2- 172.21.15.26:0/3037614815 >> v2:172.21.15.117:3300/0 conn(0x7fc51c0c3280 0x7fc51c0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc520ef2640 1 -- 172.21.15.26:0/3037614815 >> v2:172.21.15.117:3300/0 conn(0x7fc51c0c3280 msgr2=0x7fc51c0c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc520ef2640 1 --2- 172.21.15.26:0/3037614815 >> v2:172.21.15.117:3300/0 conn(0x7fc51c0c3280 0x7fc51c0c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc520ef2640 1 -- 172.21.15.26:0/3037614815 >> v1:172.21.15.26:6789/0 conn(0x7fc51c1551e0 legacy=0x7fc51c1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc520ef2640 1 -- 172.21.15.26:0/3037614815 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c1590a0 con 0x7fc51c06d9f0 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.782+0000 7fc51b7fe640 1 -- 172.21.15.26:0/3037614815 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843520015 0 0) 0x7fc51c158800 con 0x7fc51c1551e0 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.783+0000 7fc520ef2640 1 --2- 172.21.15.26:0/3037614815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c06d9f0 0x7fc51c0a7cd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0721a0 tx=0x7fc51002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ad313e1a445360 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.783+0000 7fc51b7fe640 1 -- 172.21.15.26:0/3037614815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100315a0 con 0x7fc51c06d9f0 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.783+0000 7fc51b7fe640 1 -- 172.21.15.26:0/3037614815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc510037070 con 0x7fc51c06d9f0 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.783+0000 7fc51b7fe640 1 -- 172.21.15.26:0/3037614815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510031960 con 0x7fc51c06d9f0 2024-03-06T07:22:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 -- 172.21.15.26:0/3037614815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c06d9f0 msgr2=0x7fc51c0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 --2- 172.21.15.26:0/3037614815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c06d9f0 0x7fc51c0a7cd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0721a0 tx=0x7fc51002b840 comp rx=0 tx=0).stop 2024-03-06T07:22:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 -- 172.21.15.26:0/3037614815 shutdown_connections 2024-03-06T07:22:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 --2- 172.21.15.26:0/3037614815 >> v2:172.21.15.117:3300/0 conn(0x7fc51c0c3280 0x7fc51c0c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 --2- 172.21.15.26:0/3037614815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c06d9f0 0x7fc51c0a7cd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 -- 172.21.15.26:0/3037614815 >> 172.21.15.26:0/3037614815 conn(0x7fc51c0aa630 msgr2=0x7fc51c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 -- 172.21.15.26:0/3037614815 shutdown_connections 2024-03-06T07:22:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.784+0000 7fc523156640 1 -- 172.21.15.26:0/3037614815 wait complete. 2024-03-06T07:22:49.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.785+0000 7fc523156640 1 Processor -- start 2024-03-06T07:22:49.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.785+0000 7fc523156640 1 -- start start 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.785+0000 7fc523156640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc51c06d9f0 0x7fc51c0a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc523156640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc51c0c3280 0x7fc51c06e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc523156640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c1551e0 0x7fc51c06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc523156640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c06df50 con 0x7fc51c1551e0 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc523156640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c06ddd0 con 0x7fc51c06d9f0 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc523156640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0a8210 con 0x7fc51c0c3280 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c1551e0 0x7fc51c06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c1551e0 0x7fc51c06eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49214/0 (socket says 172.21.15.26:49214) 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 -- 172.21.15.26:0/3657688636 learned_addr learned my addr 172.21.15.26:0/3657688636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc520ef2640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc51c06d9f0 0x7fc51c0a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc51bfff640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc51c0c3280 0x7fc51c06e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 -- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc51c0c3280 msgr2=0x7fc51c06e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc51c0c3280 0x7fc51c06e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 -- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc51c06d9f0 msgr2=0x7fc51c0a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc51c06d9f0 0x7fc51c0a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 -- 172.21.15.26:0/3657688636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c1468f0 con 0x7fc51c1551e0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5216f3640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c1551e0 0x7fc51c06eaf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c780 tx=0x7fc51400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5197fa640 1 -- 172.21.15.26:0/3657688636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514018070 con 0x7fc51c1551e0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5197fa640 1 -- 172.21.15.26:0/3657688636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51400ce30 con 0x7fc51c1551e0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c146bd0 con 0x7fc51c1551e0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc5197fa640 1 -- 172.21.15.26:0/3657688636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51400a840 con 0x7fc51c1551e0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.786+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c0a9d40 con 0x7fc51c1551e0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.787+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c06e0d0 con 0x7fc51c1551e0 2024-03-06T07:22:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.788+0000 7fc5197fa640 1 -- 172.21.15.26:0/3657688636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc51401b070 con 0x7fc51c1551e0 2024-03-06T07:22:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.788+0000 7fc5197fa640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc50006b500 0x7fc50006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.789+0000 7fc5197fa640 1 -- 172.21.15.26:0/3657688636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fc514089b50 con 0x7fc51c1551e0 2024-03-06T07:22:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.789+0000 7fc520ef2640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc50006b500 0x7fc50006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.789+0000 7fc520ef2640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc50006b500 0x7fc50006d9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc510036000 tx=0x7fc51003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.793+0000 7fc5197fa640 1 -- 172.21.15.26:0/3657688636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc51c06e0d0 con 0x7fc51c1551e0 2024-03-06T07:22:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.971+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc51c04f330 con 0x7fc50006b500 2024-03-06T07:22:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.993+0000 7fc5197fa640 1 -- 172.21.15.26:0/3657688636 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+485419 (secure 0 0 0) 0x7fc51c04f330 con 0x7fc50006b500 2024-03-06T07:22:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc50006b500 msgr2=0x7fc50006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc50006b500 0x7fc50006d9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc510036000 tx=0x7fc51003d040 comp rx=0 tx=0).stop 2024-03-06T07:22:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c1551e0 msgr2=0x7fc51c06eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c1551e0 0x7fc51c06eaf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c780 tx=0x7fc51400cc40 comp rx=0 tx=0).stop 2024-03-06T07:22:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 shutdown_connections 2024-03-06T07:22:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc50006b500 0x7fc50006d9c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc51c1551e0 0x7fc51c06eaf0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc51c0c3280 0x7fc51c06e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 --2- 172.21.15.26:0/3657688636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc51c06d9f0 0x7fc51c0a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 >> 172.21.15.26:0/3657688636 conn(0x7fc51c0aa630 msgr2=0x7fc51c0c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 shutdown_connections 2024-03-06T07:22:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:49.997+0000 7fc523156640 1 -- 172.21.15.26:0/3657688636 wait complete. 2024-03-06T07:22:50.009 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-03-06T07:22:49.064227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244432,"kb_used_data":7444,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730288,"statfs":{"total":773094113280,"available":772843814912,"internally_reserved":0,"allocated":7622656,"data_stored":4714746,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.712574"},"pg_stats":[{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361902+0000","last_change":"2024-03-06T07:22:44.361902+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361902+0000","last_undegraded":"2024-03-06T07:22:44.361902+0000","last_fullsized":"2024-03-06T07:22:44.361902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365039+0000","last_change":"2024-03-06T07:22:35.025691+0000","last_active":"2024-03-06T07:22:44.365039+0000","last_peered":"2024-03-06T07:22:44.365039+0000","last_clean":"2024-03-06T07:22:44.365039+0000","last_became_active":"2024-03-06T07:22:35.025394+0000","last_became_peered":"2024-03-06T07:22:35.025394+0000","last_unstale":"2024-03-06T07:22:44.365039+0000","last_undegraded":"2024-03-06T07:22:44.365039+0000","last_fullsized":"2024-03-06T07:22:44.365039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366504+0000","last_change":"2024-03-06T07:22:44.366504+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366504+0000","last_undegraded":"2024-03-06T07:22:44.366504+0000","last_fullsized":"2024-03-06T07:22:44.366504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373511+0000","last_change":"2024-03-06T07:22:35.048775+0000","last_active":"2024-03-06T07:22:44.373511+0000","last_peered":"2024-03-06T07:22:44.373511+0000","last_clean":"2024-03-06T07:22:44.373511+0000","last_became_active":"2024-03-06T07:22:35.048476+0000","last_became_peered":"2024-03-06T07:22:35.048476+0000","last_unstale":"2024-03-06T07:22:44.373511+0000","last_undegraded":"2024-03-06T07:22:44.373511+0000","last_fullsized":"2024-03-06T07:22:44.373511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:41.704388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362319+0000","last_change":"2024-03-06T07:22:44.362319+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362319+0000","last_undegraded":"2024-03-06T07:22:44.362319+0000","last_fullsized":"2024-03-06T07:22:44.362319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359053+0000","last_change":"2024-03-06T07:22:35.037727+0000","last_active":"2024-03-06T07:22:44.359053+0000","last_peered":"2024-03-06T07:22:44.359053+0000","last_clean":"2024-03-06T07:22:44.359053+0000","last_became_active":"2024-03-06T07:22:35.037421+0000","last_became_peered":"2024-03-06T07:22:35.037421+0000","last_unstale":"2024-03-06T07:22:44.359053+0000","last_undegraded":"2024-03-06T07:22:44.359053+0000","last_fullsized":"2024-03-06T07:22:44.359053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362908+0000","last_change":"2024-03-06T07:22:44.362908+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362908+0000","last_undegraded":"2024-03-06T07:22:44.362908+0000","last_fullsized":"2024-03-06T07:22:44.362908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365569+0000","last_change":"2024-03-06T07:22:35.035949+0000","last_active":"2024-03-06T07:22:44.365569+0000","last_peered":"2024-03-06T07:22:44.365569+0000","last_clean":"2024-03-06T07:22:44.365569+0000","last_became_active":"2024-03-06T07:22:35.035687+0000","last_became_peered":"2024-03-06T07:22:35.035687+0000","last_unstale":"2024-03-06T07:22:44.365569+0000","last_undegraded":"2024-03-06T07:22:44.365569+0000","last_fullsized":"2024-03-06T07:22:44.365569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362767+0000","last_change":"2024-03-06T07:22:44.362767+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362767+0000","last_undegraded":"2024-03-06T07:22:44.362767+0000","last_fullsized":"2024-03-06T07:22:44.362767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365712+0000","last_change":"2024-03-06T07:22:35.033081+0000","last_active":"2024-03-06T07:22:44.365712+0000","last_peered":"2024-03-06T07:22:44.365712+0000","last_clean":"2024-03-06T07:22:44.365712+0000","last_became_active":"2024-03-06T07:22:35.032799+0000","last_became_peered":"2024-03-06T07:22:35.032799+0000","last_unstale":"2024-03-06T07:22:44.365712+0000","last_undegraded":"2024-03-06T07:22:44.365712+0000","last_fullsized":"2024-03-06T07:22:44.365712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.187775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365929+0000","last_change":"2024-03-06T07:22:44.365929+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365929+0000","last_undegraded":"2024-03-06T07:22:44.365929+0000","last_fullsized":"2024-03-06T07:22:44.365929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364635+0000","last_change":"2024-03-06T07:22:35.041976+0000","last_active":"2024-03-06T07:22:44.364635+0000","last_peered":"2024-03-06T07:22:44.364635+0000","last_clean":"2024-03-06T07:22:44.364635+0000","last_became_active":"2024-03-06T07:22:35.041546+0000","last_became_peered":"2024-03-06T07:22:35.041546+0000","last_unstale":"2024-03-06T07:22:44.364635+0000","last_undegraded":"2024-03-06T07:22:44.364635+0000","last_fullsized":"2024-03-06T07:22:44.364635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360301+0000","last_change":"2024-03-06T07:22:44.360301+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360301+0000","last_undegraded":"2024-03-06T07:22:44.360301+0000","last_fullsized":"2024-03-06T07:22:44.360301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365335+0000","last_change":"2024-03-06T07:22:35.038320+0000","last_active":"2024-03-06T07:22:44.365335+0000","last_peered":"2024-03-06T07:22:44.365335+0000","last_clean":"2024-03-06T07:22:44.365335+0000","last_became_active":"2024-03-06T07:22:35.038043+0000","last_became_peered":"2024-03-06T07:22:35.038043+0000","last_unstale":"2024-03-06T07:22:44.365335+0000","last_undegraded":"2024-03-06T07:22:44.365335+0000","last_fullsized":"2024-03-06T07:22:44.365335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366763+0000","last_change":"2024-03-06T07:22:44.366763+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366763+0000","last_undegraded":"2024-03-06T07:22:44.366763+0000","last_fullsized":"2024-03-06T07:22:44.366763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368165+0000","last_change":"2024-03-06T07:22:35.040651+0000","last_active":"2024-03-06T07:22:44.368165+0000","last_peered":"2024-03-06T07:22:44.368165+0000","last_clean":"2024-03-06T07:22:44.368165+0000","last_became_active":"2024-03-06T07:22:35.040382+0000","last_became_peered":"2024-03-06T07:22:35.040382+0000","last_unstale":"2024-03-06T07:22:44.368165+0000","last_undegraded":"2024-03-06T07:22:44.368165+0000","last_fullsized":"2024-03-06T07:22:44.368165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:54.755446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.360342+0000","last_change":"2024-03-06T07:22:44.360342+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.360342+0000","last_undegraded":"2024-03-06T07:22:44.360342+0000","last_fullsized":"2024-03-06T07:22:44.360342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361779+0000","last_change":"2024-03-06T07:22:35.035524+0000","last_active":"2024-03-06T07:22:44.361779+0000","last_peered":"2024-03-06T07:22:44.361779+0000","last_clean":"2024-03-06T07:22:44.361779+0000","last_became_active":"2024-03-06T07:22:35.035252+0000","last_became_peered":"2024-03-06T07:22:35.035252+0000","last_unstale":"2024-03-06T07:22:44.361779+0000","last_undegraded":"2024-03-06T07:22:44.361779+0000","last_fullsized":"2024-03-06T07:22:44.361779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364257+0000","last_change":"2024-03-06T07:22:44.364257+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364257+0000","last_undegraded":"2024-03-06T07:22:44.364257+0000","last_fullsized":"2024-03-06T07:22:44.364257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365193+0000","last_change":"2024-03-06T07:22:35.040188+0000","last_active":"2024-03-06T07:22:44.365193+0000","last_peered":"2024-03-06T07:22:44.365193+0000","last_clean":"2024-03-06T07:22:44.365193+0000","last_became_active":"2024-03-06T07:22:35.039868+0000","last_became_peered":"2024-03-06T07:22:35.039868+0000","last_unstale":"2024-03-06T07:22:44.365193+0000","last_undegraded":"2024-03-06T07:22:44.365193+0000","last_fullsized":"2024-03-06T07:22:44.365193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359274+0000","last_change":"2024-03-06T07:22:44.359274+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359274+0000","last_undegraded":"2024-03-06T07:22:44.359274+0000","last_fullsized":"2024-03-06T07:22:44.359274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375229+0000","last_change":"2024-03-06T07:22:35.049739+0000","last_active":"2024-03-06T07:22:44.375229+0000","last_peered":"2024-03-06T07:22:44.375229+0000","last_clean":"2024-03-06T07:22:44.375229+0000","last_became_active":"2024-03-06T07:22:35.049118+0000","last_became_peered":"2024-03-06T07:22:35.049118+0000","last_unstale":"2024-03-06T07:22:44.375229+0000","last_undegraded":"2024-03-06T07:22:44.375229+0000","last_fullsized":"2024-03-06T07:22:44.375229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:49.784740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362088+0000","last_change":"2024-03-06T07:22:44.362088+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362088+0000","last_undegraded":"2024-03-06T07:22:44.362088+0000","last_fullsized":"2024-03-06T07:22:44.362088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365663+0000","last_change":"2024-03-06T07:22:35.038831+0000","last_active":"2024-03-06T07:22:44.365663+0000","last_peered":"2024-03-06T07:22:44.365663+0000","last_clean":"2024-03-06T07:22:44.365663+0000","last_became_active":"2024-03-06T07:22:35.038550+0000","last_became_peered":"2024-03-06T07:22:35.038550+0000","last_unstale":"2024-03-06T07:22:44.365663+0000","last_undegraded":"2024-03-06T07:22:44.365663+0000","last_fullsized":"2024-03-06T07:22:44.365663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362839+0000","last_change":"2024-03-06T07:22:44.362839+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362839+0000","last_undegraded":"2024-03-06T07:22:44.362839+0000","last_fullsized":"2024-03-06T07:22:44.362839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365819+0000","last_change":"2024-03-06T07:22:35.037819+0000","last_active":"2024-03-06T07:22:44.365819+0000","last_peered":"2024-03-06T07:22:44.365819+0000","last_clean":"2024-03-06T07:22:44.365819+0000","last_became_active":"2024-03-06T07:22:35.037519+0000","last_became_peered":"2024-03-06T07:22:35.037519+0000","last_unstale":"2024-03-06T07:22:44.365819+0000","last_undegraded":"2024-03-06T07:22:44.365819+0000","last_fullsized":"2024-03-06T07:22:44.365819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:49.281137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.369795+0000","last_change":"2024-03-06T07:22:44.369795+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.369795+0000","last_undegraded":"2024-03-06T07:22:44.369795+0000","last_fullsized":"2024-03-06T07:22:44.369795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364770+0000","last_change":"2024-03-06T07:22:35.040052+0000","last_active":"2024-03-06T07:22:44.364770+0000","last_peered":"2024-03-06T07:22:44.364770+0000","last_clean":"2024-03-06T07:22:44.364770+0000","last_became_active":"2024-03-06T07:22:35.039794+0000","last_became_peered":"2024-03-06T07:22:35.039794+0000","last_unstale":"2024-03-06T07:22:44.364770+0000","last_undegraded":"2024-03-06T07:22:44.364770+0000","last_fullsized":"2024-03-06T07:22:44.364770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364008+0000","last_change":"2024-03-06T07:22:44.364008+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364008+0000","last_undegraded":"2024-03-06T07:22:44.364008+0000","last_fullsized":"2024-03-06T07:22:44.364008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366659+0000","last_change":"2024-03-06T07:22:35.034108+0000","last_active":"2024-03-06T07:22:44.366659+0000","last_peered":"2024-03-06T07:22:44.366659+0000","last_clean":"2024-03-06T07:22:44.366659+0000","last_became_active":"2024-03-06T07:22:35.033823+0000","last_became_peered":"2024-03-06T07:22:35.033823+0000","last_unstale":"2024-03-06T07:22:44.366659+0000","last_undegraded":"2024-03-06T07:22:44.366659+0000","last_fullsized":"2024-03-06T07:22:44.366659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:04.872840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359980+0000","last_change":"2024-03-06T07:22:44.359980+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359980+0000","last_undegraded":"2024-03-06T07:22:44.359980+0000","last_fullsized":"2024-03-06T07:22:44.359980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363762+0000","last_change":"2024-03-06T07:22:35.037317+0000","last_active":"2024-03-06T07:22:44.363762+0000","last_peered":"2024-03-06T07:22:44.363762+0000","last_clean":"2024-03-06T07:22:44.363762+0000","last_became_active":"2024-03-06T07:22:35.033185+0000","last_became_peered":"2024-03-06T07:22:35.033185+0000","last_unstale":"2024-03-06T07:22:44.363762+0000","last_undegraded":"2024-03-06T07:22:44.363762+0000","last_fullsized":"2024-03-06T07:22:44.363762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359992+0000","last_change":"2024-03-06T07:22:44.359992+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359992+0000","last_undegraded":"2024-03-06T07:22:44.359992+0000","last_fullsized":"2024-03-06T07:22:44.359992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365447+0000","last_change":"2024-03-06T07:22:35.035073+0000","last_active":"2024-03-06T07:22:44.365447+0000","last_peered":"2024-03-06T07:22:44.365447+0000","last_clean":"2024-03-06T07:22:44.365447+0000","last_became_active":"2024-03-06T07:22:35.034739+0000","last_became_peered":"2024-03-06T07:22:35.034739+0000","last_unstale":"2024-03-06T07:22:44.365447+0000","last_undegraded":"2024-03-06T07:22:44.365447+0000","last_fullsized":"2024-03-06T07:22:44.365447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:12.718069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363938+0000","last_change":"2024-03-06T07:22:44.363938+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363938+0000","last_undegraded":"2024-03-06T07:22:44.363938+0000","last_fullsized":"2024-03-06T07:22:44.363938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359224+0000","last_change":"2024-03-06T07:22:35.032202+0000","last_active":"2024-03-06T07:22:44.359224+0000","last_peered":"2024-03-06T07:22:44.359224+0000","last_clean":"2024-03-06T07:22:44.359224+0000","last_became_active":"2024-03-06T07:22:35.031858+0000","last_became_peered":"2024-03-06T07:22:35.031858+0000","last_unstale":"2024-03-06T07:22:44.359224+0000","last_undegraded":"2024-03-06T07:22:44.359224+0000","last_fullsized":"2024-03-06T07:22:44.359224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364868+0000","last_change":"2024-03-06T07:22:44.364868+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364868+0000","last_undegraded":"2024-03-06T07:22:44.364868+0000","last_fullsized":"2024-03-06T07:22:44.364868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364576+0000","last_change":"2024-03-06T07:22:35.041928+0000","last_active":"2024-03-06T07:22:44.364576+0000","last_peered":"2024-03-06T07:22:44.364576+0000","last_clean":"2024-03-06T07:22:44.364576+0000","last_became_active":"2024-03-06T07:22:35.041672+0000","last_became_peered":"2024-03-06T07:22:35.041672+0000","last_unstale":"2024-03-06T07:22:44.364576+0000","last_undegraded":"2024-03-06T07:22:44.364576+0000","last_fullsized":"2024-03-06T07:22:44.364576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362840+0000","last_change":"2024-03-06T07:22:44.362840+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362840+0000","last_undegraded":"2024-03-06T07:22:44.362840+0000","last_fullsized":"2024-03-06T07:22:44.362840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368275+0000","last_change":"2024-03-06T07:22:35.036178+0000","last_active":"2024-03-06T07:22:44.368275+0000","last_peered":"2024-03-06T07:22:44.368275+0000","last_clean":"2024-03-06T07:22:44.368275+0000","last_became_active":"2024-03-06T07:22:35.035874+0000","last_became_peered":"2024-03-06T07:22:35.035874+0000","last_unstale":"2024-03-06T07:22:44.368275+0000","last_undegraded":"2024-03-06T07:22:44.368275+0000","last_fullsized":"2024-03-06T07:22:44.368275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.764602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364726+0000","last_change":"2024-03-06T07:22:44.364726+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364726+0000","last_undegraded":"2024-03-06T07:22:44.364726+0000","last_fullsized":"2024-03-06T07:22:44.364726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361583+0000","last_change":"2024-03-06T07:22:35.031787+0000","last_active":"2024-03-06T07:22:44.361583+0000","last_peered":"2024-03-06T07:22:44.361583+0000","last_clean":"2024-03-06T07:22:44.361583+0000","last_became_active":"2024-03-06T07:22:35.031524+0000","last_became_peered":"2024-03-06T07:22:35.031524+0000","last_unstale":"2024-03-06T07:22:44.361583+0000","last_undegraded":"2024-03-06T07:22:44.361583+0000","last_fullsized":"2024-03-06T07:22:44.361583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361747+0000","last_change":"2024-03-06T07:22:44.361747+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361747+0000","last_undegraded":"2024-03-06T07:22:44.361747+0000","last_fullsized":"2024-03-06T07:22:44.361747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363506+0000","last_change":"2024-03-06T07:22:35.039311+0000","last_active":"2024-03-06T07:22:44.363506+0000","last_peered":"2024-03-06T07:22:44.363506+0000","last_clean":"2024-03-06T07:22:44.363506+0000","last_became_active":"2024-03-06T07:22:35.038971+0000","last_became_peered":"2024-03-06T07:22:35.038971+0000","last_unstale":"2024-03-06T07:22:44.363506+0000","last_undegraded":"2024-03-06T07:22:44.363506+0000","last_fullsized":"2024-03-06T07:22:44.363506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:42.465427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359016+0000","last_change":"2024-03-06T07:22:44.359016+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359016+0000","last_undegraded":"2024-03-06T07:22:44.359016+0000","last_fullsized":"2024-03-06T07:22:44.359016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361143+0000","last_change":"2024-03-06T07:22:35.031978+0000","last_active":"2024-03-06T07:22:44.361143+0000","last_peered":"2024-03-06T07:22:44.361143+0000","last_clean":"2024-03-06T07:22:44.361143+0000","last_became_active":"2024-03-06T07:22:35.031711+0000","last_became_peered":"2024-03-06T07:22:35.031711+0000","last_unstale":"2024-03-06T07:22:44.361143+0000","last_undegraded":"2024-03-06T07:22:44.361143+0000","last_fullsized":"2024-03-06T07:22:44.361143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:17.004593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359118+0000","last_change":"2024-03-06T07:22:44.359118+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359118+0000","last_undegraded":"2024-03-06T07:22:44.359118+0000","last_fullsized":"2024-03-06T07:22:44.359118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375412+0000","last_change":"2024-03-06T07:22:35.045711+0000","last_active":"2024-03-06T07:22:44.375412+0000","last_peered":"2024-03-06T07:22:44.375412+0000","last_clean":"2024-03-06T07:22:44.375412+0000","last_became_active":"2024-03-06T07:22:35.041242+0000","last_became_peered":"2024-03-06T07:22:35.041242+0000","last_unstale":"2024-03-06T07:22:44.375412+0000","last_undegraded":"2024-03-06T07:22:44.375412+0000","last_fullsized":"2024-03-06T07:22:44.375412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:16.266604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364653+0000","last_change":"2024-03-06T07:22:44.364653+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364653+0000","last_undegraded":"2024-03-06T07:22:44.364653+0000","last_fullsized":"2024-03-06T07:22:44.364653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374041+0000","last_change":"2024-03-06T07:22:35.042301+0000","last_active":"2024-03-06T07:22:44.374041+0000","last_peered":"2024-03-06T07:22:44.374041+0000","last_clean":"2024-03-06T07:22:44.374041+0000","last_became_active":"2024-03-06T07:22:35.041172+0000","last_became_peered":"2024-03-06T07:22:35.041172+0000","last_unstale":"2024-03-06T07:22:44.374041+0000","last_undegraded":"2024-03-06T07:22:44.374041+0000","last_fullsized":"2024-03-06T07:22:44.374041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:58.974636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363518+0000","last_change":"2024-03-06T07:22:44.363518+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363518+0000","last_undegraded":"2024-03-06T07:22:44.363518+0000","last_fullsized":"2024-03-06T07:22:44.363518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365279+0000","last_change":"2024-03-06T07:22:35.038117+0000","last_active":"2024-03-06T07:22:44.365279+0000","last_peered":"2024-03-06T07:22:44.365279+0000","last_clean":"2024-03-06T07:22:44.365279+0000","last_became_active":"2024-03-06T07:22:35.037812+0000","last_became_peered":"2024-03-06T07:22:35.037812+0000","last_unstale":"2024-03-06T07:22:44.365279+0000","last_undegraded":"2024-03-06T07:22:44.365279+0000","last_fullsized":"2024-03-06T07:22:44.365279+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363196+0000","last_change":"2024-03-06T07:22:44.363196+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363196+0000","last_undegraded":"2024-03-06T07:22:44.363196+0000","last_fullsized":"2024-03-06T07:22:44.363196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368368+0000","last_change":"2024-03-06T07:22:35.033425+0000","last_active":"2024-03-06T07:22:44.368368+0000","last_peered":"2024-03-06T07:22:44.368368+0000","last_clean":"2024-03-06T07:22:44.368368+0000","last_became_active":"2024-03-06T07:22:35.032784+0000","last_became_peered":"2024-03-06T07:22:35.032784+0000","last_unstale":"2024-03-06T07:22:44.368368+0000","last_undegraded":"2024-03-06T07:22:44.368368+0000","last_fullsized":"2024-03-06T07:22:44.368368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361026+0000","last_change":"2024-03-06T07:22:44.361026+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361026+0000","last_undegraded":"2024-03-06T07:22:44.361026+0000","last_fullsized":"2024-03-06T07:22:44.361026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368367+0000","last_change":"2024-03-06T07:22:35.033473+0000","last_active":"2024-03-06T07:22:44.368367+0000","last_peered":"2024-03-06T07:22:44.368367+0000","last_clean":"2024-03-06T07:22:44.368367+0000","last_became_active":"2024-03-06T07:22:35.032407+0000","last_became_peered":"2024-03-06T07:22:35.032407+0000","last_unstale":"2024-03-06T07:22:44.368367+0000","last_undegraded":"2024-03-06T07:22:44.368367+0000","last_fullsized":"2024-03-06T07:22:44.368367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:39.792272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363072+0000","last_change":"2024-03-06T07:22:44.363072+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363072+0000","last_undegraded":"2024-03-06T07:22:44.363072+0000","last_fullsized":"2024-03-06T07:22:44.363072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370470+0000","last_change":"2024-03-06T07:22:35.048127+0000","last_active":"2024-03-06T07:22:44.370470+0000","last_peered":"2024-03-06T07:22:44.370470+0000","last_clean":"2024-03-06T07:22:44.370470+0000","last_became_active":"2024-03-06T07:22:35.047869+0000","last_became_peered":"2024-03-06T07:22:35.047869+0000","last_unstale":"2024-03-06T07:22:44.370470+0000","last_undegraded":"2024-03-06T07:22:44.370470+0000","last_fullsized":"2024-03-06T07:22:44.370470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.375194+0000","last_change":"2024-03-06T07:22:44.375194+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.375194+0000","last_undegraded":"2024-03-06T07:22:44.375194+0000","last_fullsized":"2024-03-06T07:22:44.375194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363056+0000","last_change":"2024-03-06T07:22:35.038257+0000","last_active":"2024-03-06T07:22:44.363056+0000","last_peered":"2024-03-06T07:22:44.363056+0000","last_clean":"2024-03-06T07:22:44.363056+0000","last_became_active":"2024-03-06T07:22:35.037969+0000","last_became_peered":"2024-03-06T07:22:35.037969+0000","last_unstale":"2024-03-06T07:22:44.363056+0000","last_undegraded":"2024-03-06T07:22:44.363056+0000","last_fullsized":"2024-03-06T07:22:44.363056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362412+0000","last_change":"2024-03-06T07:22:44.362412+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362412+0000","last_undegraded":"2024-03-06T07:22:44.362412+0000","last_fullsized":"2024-03-06T07:22:44.362412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358712+0000","last_change":"2024-03-06T07:22:35.032129+0000","last_active":"2024-03-06T07:22:44.358712+0000","last_peered":"2024-03-06T07:22:44.358712+0000","last_clean":"2024-03-06T07:22:44.358712+0000","last_became_active":"2024-03-06T07:22:35.027787+0000","last_became_peered":"2024-03-06T07:22:35.027787+0000","last_unstale":"2024-03-06T07:22:44.358712+0000","last_undegraded":"2024-03-06T07:22:44.358712+0000","last_fullsized":"2024-03-06T07:22:44.358712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:27.011065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359560+0000","last_change":"2024-03-06T07:22:44.359560+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359560+0000","last_undegraded":"2024-03-06T07:22:44.359560+0000","last_fullsized":"2024-03-06T07:22:44.359560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363854+0000","last_change":"2024-03-06T07:22:35.036989+0000","last_active":"2024-03-06T07:22:44.363854+0000","last_peered":"2024-03-06T07:22:44.363854+0000","last_clean":"2024-03-06T07:22:44.363854+0000","last_became_active":"2024-03-06T07:22:35.032803+0000","last_became_peered":"2024-03-06T07:22:35.032803+0000","last_unstale":"2024-03-06T07:22:44.363854+0000","last_undegraded":"2024-03-06T07:22:44.363854+0000","last_fullsized":"2024-03-06T07:22:44.363854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.365367+0000","last_change":"2024-03-06T07:22:44.365367+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.365367+0000","last_undegraded":"2024-03-06T07:22:44.365367+0000","last_fullsized":"2024-03-06T07:22:44.365367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373250+0000","last_change":"2024-03-06T07:22:44.372693+0000","last_active":"2024-03-06T07:22:44.373250+0000","last_peered":"2024-03-06T07:22:44.373250+0000","last_clean":"2024-03-06T07:22:44.373250+0000","last_became_active":"2024-03-06T07:22:44.372375+0000","last_became_peered":"2024-03-06T07:22:44.372375+0000","last_unstale":"2024-03-06T07:22:44.373250+0000","last_undegraded":"2024-03-06T07:22:44.373250+0000","last_fullsized":"2024-03-06T07:22:44.373250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358464+0000","last_change":"2024-03-06T07:22:35.025981+0000","last_active":"2024-03-06T07:22:44.358464+0000","last_peered":"2024-03-06T07:22:44.358464+0000","last_clean":"2024-03-06T07:22:44.358464+0000","last_became_active":"2024-03-06T07:22:35.025721+0000","last_became_peered":"2024-03-06T07:22:35.025721+0000","last_unstale":"2024-03-06T07:22:44.358464+0000","last_undegraded":"2024-03-06T07:22:44.358464+0000","last_fullsized":"2024-03-06T07:22:44.358464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:18.894659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365555+0000","last_change":"2024-03-06T07:22:34.043747+0000","last_active":"2024-03-06T07:22:44.365555+0000","last_peered":"2024-03-06T07:22:44.365555+0000","last_clean":"2024-03-06T07:22:44.365555+0000","last_became_active":"2024-03-06T07:22:34.042367+0000","last_became_peered":"2024-03-06T07:22:34.042367+0000","last_unstale":"2024-03-06T07:22:44.365555+0000","last_undegraded":"2024-03-06T07:22:44.365555+0000","last_fullsized":"2024-03-06T07:22:44.365555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:33.138544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363123+0000","last_change":"2024-03-06T07:22:44.363123+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363123+0000","last_undegraded":"2024-03-06T07:22:44.363123+0000","last_fullsized":"2024-03-06T07:22:44.363123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384852+0000","last_change":"2024-03-06T07:22:44.380429+0000","last_active":"2024-03-06T07:22:44.384852+0000","last_peered":"2024-03-06T07:22:44.384852+0000","last_clean":"2024-03-06T07:22:44.384852+0000","last_became_active":"2024-03-06T07:22:44.380170+0000","last_became_peered":"2024-03-06T07:22:44.380170+0000","last_unstale":"2024-03-06T07:22:44.384852+0000","last_undegraded":"2024-03-06T07:22:44.384852+0000","last_fullsized":"2024-03-06T07:22:44.384852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:53.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370159+0000","last_change":"2024-03-06T07:22:35.032593+0000","last_active":"2024-03-06T07:22:44.370159+0000","last_peered":"2024-03-06T07:22:44.370159+0000","last_clean":"2024-03-06T07:22:44.370159+0000","last_became_active":"2024-03-06T07:22:35.031651+0000","last_became_peered":"2024-03-06T07:22:35.031651+0000","last_unstale":"2024-03-06T07:22:44.370159+0000","last_undegraded":"2024-03-06T07:22:44.370159+0000","last_fullsized":"2024-03-06T07:22:44.370159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364514+0000","last_change":"2024-03-06T07:22:34.038702+0000","last_active":"2024-03-06T07:22:44.364514+0000","last_peered":"2024-03-06T07:22:44.364514+0000","last_clean":"2024-03-06T07:22:44.364514+0000","last_became_active":"2024-03-06T07:22:34.038165+0000","last_became_peered":"2024-03-06T07:22:34.038165+0000","last_unstale":"2024-03-06T07:22:44.364514+0000","last_undegraded":"2024-03-06T07:22:44.364514+0000","last_fullsized":"2024-03-06T07:22:44.364514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362338+0000","last_change":"2024-03-06T07:22:44.362338+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362338+0000","last_undegraded":"2024-03-06T07:22:44.362338+0000","last_fullsized":"2024-03-06T07:22:44.362338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.381941+0000","last_change":"2024-03-06T07:22:44.375256+0000","last_active":"2024-03-06T07:22:44.381941+0000","last_peered":"2024-03-06T07:22:44.381941+0000","last_clean":"2024-03-06T07:22:44.381941+0000","last_became_active":"2024-03-06T07:22:44.374987+0000","last_became_peered":"2024-03-06T07:22:44.374987+0000","last_unstale":"2024-03-06T07:22:44.381941+0000","last_undegraded":"2024-03-06T07:22:44.381941+0000","last_fullsized":"2024-03-06T07:22:44.381941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:32.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361466+0000","last_change":"2024-03-06T07:22:35.035774+0000","last_active":"2024-03-06T07:22:44.361466+0000","last_peered":"2024-03-06T07:22:44.361466+0000","last_clean":"2024-03-06T07:22:44.361466+0000","last_became_active":"2024-03-06T07:22:35.035468+0000","last_became_peered":"2024-03-06T07:22:35.035468+0000","last_unstale":"2024-03-06T07:22:44.361466+0000","last_undegraded":"2024-03-06T07:22:44.361466+0000","last_fullsized":"2024-03-06T07:22:44.361466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:25.144870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368550+0000","last_change":"2024-03-06T07:22:34.042434+0000","last_active":"2024-03-06T07:22:44.368550+0000","last_peered":"2024-03-06T07:22:44.368550+0000","last_clean":"2024-03-06T07:22:44.368550+0000","last_became_active":"2024-03-06T07:22:34.042093+0000","last_became_peered":"2024-03-06T07:22:34.042093+0000","last_unstale":"2024-03-06T07:22:44.368550+0000","last_undegraded":"2024-03-06T07:22:44.368550+0000","last_fullsized":"2024-03-06T07:22:44.368550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364352+0000","last_change":"2024-03-06T07:22:44.364352+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364352+0000","last_undegraded":"2024-03-06T07:22:44.364352+0000","last_fullsized":"2024-03-06T07:22:44.364352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375700+0000","last_change":"2024-03-06T07:22:44.373849+0000","last_active":"2024-03-06T07:22:44.375700+0000","last_peered":"2024-03-06T07:22:44.375700+0000","last_clean":"2024-03-06T07:22:44.375700+0000","last_became_active":"2024-03-06T07:22:44.373135+0000","last_became_peered":"2024-03-06T07:22:44.373135+0000","last_unstale":"2024-03-06T07:22:44.375700+0000","last_undegraded":"2024-03-06T07:22:44.375700+0000","last_fullsized":"2024-03-06T07:22:44.375700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:52.622331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361651+0000","last_change":"2024-03-06T07:22:35.032012+0000","last_active":"2024-03-06T07:22:44.361651+0000","last_peered":"2024-03-06T07:22:44.361651+0000","last_clean":"2024-03-06T07:22:44.361651+0000","last_became_active":"2024-03-06T07:22:35.031728+0000","last_became_peered":"2024-03-06T07:22:35.031728+0000","last_unstale":"2024-03-06T07:22:44.361651+0000","last_undegraded":"2024-03-06T07:22:44.361651+0000","last_fullsized":"2024-03-06T07:22:44.361651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364137+0000","last_change":"2024-03-06T07:22:34.038862+0000","last_active":"2024-03-06T07:22:44.364137+0000","last_peered":"2024-03-06T07:22:44.364137+0000","last_clean":"2024-03-06T07:22:44.364137+0000","last_became_active":"2024-03-06T07:22:34.038590+0000","last_became_peered":"2024-03-06T07:22:34.038590+0000","last_unstale":"2024-03-06T07:22:44.364137+0000","last_undegraded":"2024-03-06T07:22:44.364137+0000","last_fullsized":"2024-03-06T07:22:44.364137+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:15.997587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362381+0000","last_change":"2024-03-06T07:22:44.362381+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362381+0000","last_undegraded":"2024-03-06T07:22:44.362381+0000","last_fullsized":"2024-03-06T07:22:44.362381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376768+0000","last_change":"2024-03-06T07:22:44.375296+0000","last_active":"2024-03-06T07:22:44.376768+0000","last_peered":"2024-03-06T07:22:44.376768+0000","last_clean":"2024-03-06T07:22:44.376768+0000","last_became_active":"2024-03-06T07:22:44.375028+0000","last_became_peered":"2024-03-06T07:22:44.375028+0000","last_unstale":"2024-03-06T07:22:44.376768+0000","last_undegraded":"2024-03-06T07:22:44.376768+0000","last_fullsized":"2024-03-06T07:22:44.376768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.786058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376180+0000","last_change":"2024-03-06T07:22:35.036832+0000","last_active":"2024-03-06T07:22:44.376180+0000","last_peered":"2024-03-06T07:22:44.376180+0000","last_clean":"2024-03-06T07:22:44.376180+0000","last_became_active":"2024-03-06T07:22:35.036392+0000","last_became_peered":"2024-03-06T07:22:35.036392+0000","last_unstale":"2024-03-06T07:22:44.376180+0000","last_undegraded":"2024-03-06T07:22:44.376180+0000","last_fullsized":"2024-03-06T07:22:44.376180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:13.614365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361685+0000","last_change":"2024-03-06T07:22:34.037848+0000","last_active":"2024-03-06T07:22:44.361685+0000","last_peered":"2024-03-06T07:22:44.361685+0000","last_clean":"2024-03-06T07:22:44.361685+0000","last_became_active":"2024-03-06T07:22:34.037572+0000","last_became_peered":"2024-03-06T07:22:34.037572+0000","last_unstale":"2024-03-06T07:22:44.361685+0000","last_undegraded":"2024-03-06T07:22:44.361685+0000","last_fullsized":"2024-03-06T07:22:44.361685+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:15.748729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364294+0000","last_change":"2024-03-06T07:22:44.364294+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364294+0000","last_undegraded":"2024-03-06T07:22:44.364294+0000","last_fullsized":"2024-03-06T07:22:44.364294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379327+0000","last_change":"2024-03-06T07:22:44.375707+0000","last_active":"2024-03-06T07:22:44.379327+0000","last_peered":"2024-03-06T07:22:44.379327+0000","last_clean":"2024-03-06T07:22:44.379327+0000","last_became_active":"2024-03-06T07:22:44.375510+0000","last_became_peered":"2024-03-06T07:22:44.375510+0000","last_unstale":"2024-03-06T07:22:44.379327+0000","last_undegraded":"2024-03-06T07:22:44.379327+0000","last_fullsized":"2024-03-06T07:22:44.379327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:57.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375276+0000","last_change":"2024-03-06T07:22:35.040723+0000","last_active":"2024-03-06T07:22:44.375276+0000","last_peered":"2024-03-06T07:22:44.375276+0000","last_clean":"2024-03-06T07:22:44.375276+0000","last_became_active":"2024-03-06T07:22:35.036544+0000","last_became_peered":"2024-03-06T07:22:35.036544+0000","last_unstale":"2024-03-06T07:22:44.375276+0000","last_undegraded":"2024-03-06T07:22:44.375276+0000","last_fullsized":"2024-03-06T07:22:44.375276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:44.185335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366731+0000","last_change":"2024-03-06T07:22:34.040746+0000","last_active":"2024-03-06T07:22:44.366731+0000","last_peered":"2024-03-06T07:22:44.366731+0000","last_clean":"2024-03-06T07:22:44.366731+0000","last_became_active":"2024-03-06T07:22:34.040466+0000","last_became_peered":"2024-03-06T07:22:34.040466+0000","last_unstale":"2024-03-06T07:22:44.366731+0000","last_undegraded":"2024-03-06T07:22:44.366731+0000","last_fullsized":"2024-03-06T07:22:44.366731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:35.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367578+0000","last_change":"2024-03-06T07:22:44.367578+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367578+0000","last_undegraded":"2024-03-06T07:22:44.367578+0000","last_fullsized":"2024-03-06T07:22:44.367578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384804+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384804+0000","last_peered":"2024-03-06T07:22:44.384804+0000","last_clean":"2024-03-06T07:22:44.384804+0000","last_became_active":"2024-03-06T07:22:44.380134+0000","last_became_peered":"2024-03-06T07:22:44.380134+0000","last_unstale":"2024-03-06T07:22:44.384804+0000","last_undegraded":"2024-03-06T07:22:44.384804+0000","last_fullsized":"2024-03-06T07:22:44.384804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:13.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361680+0000","last_change":"2024-03-06T07:22:35.035207+0000","last_active":"2024-03-06T07:22:44.361680+0000","last_peered":"2024-03-06T07:22:44.361680+0000","last_clean":"2024-03-06T07:22:44.361680+0000","last_became_active":"2024-03-06T07:22:35.034887+0000","last_became_peered":"2024-03-06T07:22:35.034887+0000","last_unstale":"2024-03-06T07:22:44.361680+0000","last_undegraded":"2024-03-06T07:22:44.361680+0000","last_fullsized":"2024-03-06T07:22:44.361680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370135+0000","last_change":"2024-03-06T07:22:34.042833+0000","last_active":"2024-03-06T07:22:44.370135+0000","last_peered":"2024-03-06T07:22:44.370135+0000","last_clean":"2024-03-06T07:22:44.370135+0000","last_became_active":"2024-03-06T07:22:34.042557+0000","last_became_peered":"2024-03-06T07:22:34.042557+0000","last_unstale":"2024-03-06T07:22:44.370135+0000","last_undegraded":"2024-03-06T07:22:44.370135+0000","last_fullsized":"2024-03-06T07:22:44.370135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.372269+0000","last_change":"2024-03-06T07:22:44.372269+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.372269+0000","last_undegraded":"2024-03-06T07:22:44.372269+0000","last_fullsized":"2024-03-06T07:22:44.372269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379060+0000","last_change":"2024-03-06T07:22:44.371322+0000","last_active":"2024-03-06T07:22:44.379060+0000","last_peered":"2024-03-06T07:22:44.379060+0000","last_clean":"2024-03-06T07:22:44.379060+0000","last_became_active":"2024-03-06T07:22:44.371058+0000","last_became_peered":"2024-03-06T07:22:44.371058+0000","last_unstale":"2024-03-06T07:22:44.379060+0000","last_undegraded":"2024-03-06T07:22:44.379060+0000","last_fullsized":"2024-03-06T07:22:44.379060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:25.531656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364553+0000","last_change":"2024-03-06T07:22:35.028293+0000","last_active":"2024-03-06T07:22:44.364553+0000","last_peered":"2024-03-06T07:22:44.364553+0000","last_clean":"2024-03-06T07:22:44.364553+0000","last_became_active":"2024-03-06T07:22:35.028017+0000","last_became_peered":"2024-03-06T07:22:35.028017+0000","last_unstale":"2024-03-06T07:22:44.364553+0000","last_undegraded":"2024-03-06T07:22:44.364553+0000","last_fullsized":"2024-03-06T07:22:44.364553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:36.890851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369543+0000","last_change":"2024-03-06T07:22:34.037660+0000","last_active":"2024-03-06T07:22:44.369543+0000","last_peered":"2024-03-06T07:22:44.369543+0000","last_clean":"2024-03-06T07:22:44.369543+0000","last_became_active":"2024-03-06T07:22:34.037347+0000","last_became_peered":"2024-03-06T07:22:34.037347+0000","last_unstale":"2024-03-06T07:22:44.369543+0000","last_undegraded":"2024-03-06T07:22:44.369543+0000","last_fullsized":"2024-03-06T07:22:44.369543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:43.483410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362846+0000","last_change":"2024-03-06T07:22:44.362846+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362846+0000","last_undegraded":"2024-03-06T07:22:44.362846+0000","last_fullsized":"2024-03-06T07:22:44.362846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.374719+0000","last_change":"2024-03-06T07:22:44.370568+0000","last_active":"2024-03-06T07:22:44.374719+0000","last_peered":"2024-03-06T07:22:44.374719+0000","last_clean":"2024-03-06T07:22:44.374719+0000","last_became_active":"2024-03-06T07:22:44.370050+0000","last_became_peered":"2024-03-06T07:22:44.370050+0000","last_unstale":"2024-03-06T07:22:44.374719+0000","last_undegraded":"2024-03-06T07:22:44.374719+0000","last_fullsized":"2024-03-06T07:22:44.374719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:46.225047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375512+0000","last_change":"2024-03-06T07:22:35.045241+0000","last_active":"2024-03-06T07:22:44.375512+0000","last_peered":"2024-03-06T07:22:44.375512+0000","last_clean":"2024-03-06T07:22:44.375512+0000","last_became_active":"2024-03-06T07:22:35.044953+0000","last_became_peered":"2024-03-06T07:22:35.044953+0000","last_unstale":"2024-03-06T07:22:44.375512+0000","last_undegraded":"2024-03-06T07:22:44.375512+0000","last_fullsized":"2024-03-06T07:22:44.375512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365128+0000","last_change":"2024-03-06T07:22:34.038501+0000","last_active":"2024-03-06T07:22:44.365128+0000","last_peered":"2024-03-06T07:22:44.365128+0000","last_clean":"2024-03-06T07:22:44.365128+0000","last_became_active":"2024-03-06T07:22:34.038238+0000","last_became_peered":"2024-03-06T07:22:34.038238+0000","last_unstale":"2024-03-06T07:22:44.365128+0000","last_undegraded":"2024-03-06T07:22:44.365128+0000","last_fullsized":"2024-03-06T07:22:44.365128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363829+0000","last_change":"2024-03-06T07:22:44.363829+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363829+0000","last_undegraded":"2024-03-06T07:22:44.363829+0000","last_fullsized":"2024-03-06T07:22:44.363829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379031+0000","last_change":"2024-03-06T07:22:44.377341+0000","last_active":"2024-03-06T07:22:44.379031+0000","last_peered":"2024-03-06T07:22:44.379031+0000","last_clean":"2024-03-06T07:22:44.379031+0000","last_became_active":"2024-03-06T07:22:44.377070+0000","last_became_peered":"2024-03-06T07:22:44.377070+0000","last_unstale":"2024-03-06T07:22:44.379031+0000","last_undegraded":"2024-03-06T07:22:44.379031+0000","last_fullsized":"2024-03-06T07:22:44.379031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:28.968579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365969+0000","last_change":"2024-03-06T07:22:35.035913+0000","last_active":"2024-03-06T07:22:44.365969+0000","last_peered":"2024-03-06T07:22:44.365969+0000","last_clean":"2024-03-06T07:22:44.365969+0000","last_became_active":"2024-03-06T07:22:35.035658+0000","last_became_peered":"2024-03-06T07:22:35.035658+0000","last_unstale":"2024-03-06T07:22:44.365969+0000","last_undegraded":"2024-03-06T07:22:44.365969+0000","last_fullsized":"2024-03-06T07:22:44.365969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:55.082141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365373+0000","last_change":"2024-03-06T07:22:34.040689+0000","last_active":"2024-03-06T07:22:44.365373+0000","last_peered":"2024-03-06T07:22:44.365373+0000","last_clean":"2024-03-06T07:22:44.365373+0000","last_became_active":"2024-03-06T07:22:34.040410+0000","last_became_peered":"2024-03-06T07:22:34.040410+0000","last_unstale":"2024-03-06T07:22:44.365373+0000","last_undegraded":"2024-03-06T07:22:44.365373+0000","last_fullsized":"2024-03-06T07:22:44.365373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.281130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361564+0000","last_change":"2024-03-06T07:22:44.361564+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361564+0000","last_undegraded":"2024-03-06T07:22:44.361564+0000","last_fullsized":"2024-03-06T07:22:44.361564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384844+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:44.384844+0000","last_peered":"2024-03-06T07:22:44.384844+0000","last_clean":"2024-03-06T07:22:44.384844+0000","last_became_active":"2024-03-06T07:22:44.380123+0000","last_became_peered":"2024-03-06T07:22:44.380123+0000","last_unstale":"2024-03-06T07:22:44.384844+0000","last_undegraded":"2024-03-06T07:22:44.384844+0000","last_fullsized":"2024-03-06T07:22:44.384844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:59.118936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364927+0000","last_change":"2024-03-06T07:22:35.032615+0000","last_active":"2024-03-06T07:22:44.364927+0000","last_peered":"2024-03-06T07:22:44.364927+0000","last_clean":"2024-03-06T07:22:44.364927+0000","last_became_active":"2024-03-06T07:22:35.031167+0000","last_became_peered":"2024-03-06T07:22:35.031167+0000","last_unstale":"2024-03-06T07:22:44.364927+0000","last_undegraded":"2024-03-06T07:22:44.364927+0000","last_fullsized":"2024-03-06T07:22:44.364927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369335+0000","last_change":"2024-03-06T07:22:34.038406+0000","last_active":"2024-03-06T07:22:44.369335+0000","last_peered":"2024-03-06T07:22:44.369335+0000","last_clean":"2024-03-06T07:22:44.369335+0000","last_became_active":"2024-03-06T07:22:34.038040+0000","last_became_peered":"2024-03-06T07:22:34.038040+0000","last_unstale":"2024-03-06T07:22:44.369335+0000","last_undegraded":"2024-03-06T07:22:44.369335+0000","last_fullsized":"2024-03-06T07:22:44.369335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:26.063950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359017+0000","last_change":"2024-03-06T07:22:44.359017+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359017+0000","last_undegraded":"2024-03-06T07:22:44.359017+0000","last_fullsized":"2024-03-06T07:22:44.359017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379693+0000","last_change":"2024-03-06T07:22:44.377976+0000","last_active":"2024-03-06T07:22:44.379693+0000","last_peered":"2024-03-06T07:22:44.379693+0000","last_clean":"2024-03-06T07:22:44.379693+0000","last_became_active":"2024-03-06T07:22:44.377659+0000","last_became_peered":"2024-03-06T07:22:44.377659+0000","last_unstale":"2024-03-06T07:22:44.379693+0000","last_undegraded":"2024-03-06T07:22:44.379693+0000","last_fullsized":"2024-03-06T07:22:44.379693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:42.020403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359010+0000","last_change":"2024-03-06T07:22:35.038459+0000","last_active":"2024-03-06T07:22:44.359010+0000","last_peered":"2024-03-06T07:22:44.359010+0000","last_clean":"2024-03-06T07:22:44.359010+0000","last_became_active":"2024-03-06T07:22:35.038185+0000","last_became_peered":"2024-03-06T07:22:35.038185+0000","last_unstale":"2024-03-06T07:22:44.359010+0000","last_undegraded":"2024-03-06T07:22:44.359010+0000","last_fullsized":"2024-03-06T07:22:44.359010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364863+0000","last_change":"2024-03-06T07:22:34.037962+0000","last_active":"2024-03-06T07:22:44.364863+0000","last_peered":"2024-03-06T07:22:44.364863+0000","last_clean":"2024-03-06T07:22:44.364863+0000","last_became_active":"2024-03-06T07:22:34.037635+0000","last_became_peered":"2024-03-06T07:22:34.037635+0000","last_unstale":"2024-03-06T07:22:44.364863+0000","last_undegraded":"2024-03-06T07:22:44.364863+0000","last_fullsized":"2024-03-06T07:22:44.364863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362391+0000","last_change":"2024-03-06T07:22:44.362391+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362391+0000","last_undegraded":"2024-03-06T07:22:44.362391+0000","last_fullsized":"2024-03-06T07:22:44.362391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382003+0000","last_change":"2024-03-06T07:22:44.377361+0000","last_active":"2024-03-06T07:22:44.382003+0000","last_peered":"2024-03-06T07:22:44.382003+0000","last_clean":"2024-03-06T07:22:44.382003+0000","last_became_active":"2024-03-06T07:22:44.377101+0000","last_became_peered":"2024-03-06T07:22:44.377101+0000","last_unstale":"2024-03-06T07:22:44.382003+0000","last_undegraded":"2024-03-06T07:22:44.382003+0000","last_fullsized":"2024-03-06T07:22:44.382003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:13.917348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365752+0000","last_change":"2024-03-06T07:22:35.036674+0000","last_active":"2024-03-06T07:22:44.365752+0000","last_peered":"2024-03-06T07:22:44.365752+0000","last_clean":"2024-03-06T07:22:44.365752+0000","last_became_active":"2024-03-06T07:22:35.036400+0000","last_became_peered":"2024-03-06T07:22:35.036400+0000","last_unstale":"2024-03-06T07:22:44.365752+0000","last_undegraded":"2024-03-06T07:22:44.365752+0000","last_fullsized":"2024-03-06T07:22:44.365752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365049+0000","last_change":"2024-03-06T07:22:34.041311+0000","last_active":"2024-03-06T07:22:44.365049+0000","last_peered":"2024-03-06T07:22:44.365049+0000","last_clean":"2024-03-06T07:22:44.365049+0000","last_became_active":"2024-03-06T07:22:34.040940+0000","last_became_peered":"2024-03-06T07:22:34.040940+0000","last_unstale":"2024-03-06T07:22:44.365049+0000","last_undegraded":"2024-03-06T07:22:44.365049+0000","last_fullsized":"2024-03-06T07:22:44.365049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:24.924844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362009+0000","last_change":"2024-03-06T07:22:44.362009+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362009+0000","last_undegraded":"2024-03-06T07:22:44.362009+0000","last_fullsized":"2024-03-06T07:22:44.362009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379734+0000","last_change":"2024-03-06T07:22:44.378302+0000","last_active":"2024-03-06T07:22:44.379734+0000","last_peered":"2024-03-06T07:22:44.379734+0000","last_clean":"2024-03-06T07:22:44.379734+0000","last_became_active":"2024-03-06T07:22:44.377994+0000","last_became_peered":"2024-03-06T07:22:44.377994+0000","last_unstale":"2024-03-06T07:22:44.379734+0000","last_undegraded":"2024-03-06T07:22:44.379734+0000","last_fullsized":"2024-03-06T07:22:44.379734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:35.747227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363188+0000","last_change":"2024-03-06T07:22:35.022953+0000","last_active":"2024-03-06T07:22:44.363188+0000","last_peered":"2024-03-06T07:22:44.363188+0000","last_clean":"2024-03-06T07:22:44.363188+0000","last_became_active":"2024-03-06T07:22:35.022681+0000","last_became_peered":"2024-03-06T07:22:35.022681+0000","last_unstale":"2024-03-06T07:22:44.363188+0000","last_undegraded":"2024-03-06T07:22:44.363188+0000","last_fullsized":"2024-03-06T07:22:44.363188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363145+0000","last_change":"2024-03-06T07:22:34.037840+0000","last_active":"2024-03-06T07:22:44.363145+0000","last_peered":"2024-03-06T07:22:44.363145+0000","last_clean":"2024-03-06T07:22:44.363145+0000","last_became_active":"2024-03-06T07:22:34.037556+0000","last_became_peered":"2024-03-06T07:22:34.037556+0000","last_unstale":"2024-03-06T07:22:44.363145+0000","last_undegraded":"2024-03-06T07:22:44.363145+0000","last_fullsized":"2024-03-06T07:22:44.363145+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359660+0000","last_change":"2024-03-06T07:22:44.359660+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359660+0000","last_undegraded":"2024-03-06T07:22:44.359660+0000","last_fullsized":"2024-03-06T07:22:44.359660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379220+0000","last_change":"2024-03-06T07:22:44.372722+0000","last_active":"2024-03-06T07:22:44.379220+0000","last_peered":"2024-03-06T07:22:44.379220+0000","last_clean":"2024-03-06T07:22:44.379220+0000","last_became_active":"2024-03-06T07:22:44.372451+0000","last_became_peered":"2024-03-06T07:22:44.372451+0000","last_unstale":"2024-03-06T07:22:44.379220+0000","last_undegraded":"2024-03-06T07:22:44.379220+0000","last_fullsized":"2024-03-06T07:22:44.379220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:15.672728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366071+0000","last_change":"2024-03-06T07:22:35.034503+0000","last_active":"2024-03-06T07:22:44.366071+0000","last_peered":"2024-03-06T07:22:44.366071+0000","last_clean":"2024-03-06T07:22:44.366071+0000","last_became_active":"2024-03-06T07:22:35.034230+0000","last_became_peered":"2024-03-06T07:22:35.034230+0000","last_unstale":"2024-03-06T07:22:44.366071+0000","last_undegraded":"2024-03-06T07:22:44.366071+0000","last_fullsized":"2024-03-06T07:22:44.366071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365219+0000","last_change":"2024-03-06T07:22:34.039959+0000","last_active":"2024-03-06T07:22:44.365219+0000","last_peered":"2024-03-06T07:22:44.365219+0000","last_clean":"2024-03-06T07:22:44.365219+0000","last_became_active":"2024-03-06T07:22:34.039689+0000","last_became_peered":"2024-03-06T07:22:34.039689+0000","last_unstale":"2024-03-06T07:22:44.365219+0000","last_undegraded":"2024-03-06T07:22:44.365219+0000","last_fullsized":"2024-03-06T07:22:44.365219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:23.420842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364356+0000","last_change":"2024-03-06T07:22:44.364356+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364356+0000","last_undegraded":"2024-03-06T07:22:44.364356+0000","last_fullsized":"2024-03-06T07:22:44.364356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.372909+0000","last_change":"2024-03-06T07:22:44.371938+0000","last_active":"2024-03-06T07:22:44.372909+0000","last_peered":"2024-03-06T07:22:44.372909+0000","last_clean":"2024-03-06T07:22:44.372909+0000","last_became_active":"2024-03-06T07:22:44.371634+0000","last_became_peered":"2024-03-06T07:22:44.371634+0000","last_unstale":"2024-03-06T07:22:44.372909+0000","last_undegraded":"2024-03-06T07:22:44.372909+0000","last_fullsized":"2024-03-06T07:22:44.372909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364645+0000","last_change":"2024-03-06T07:22:35.032290+0000","last_active":"2024-03-06T07:22:44.364645+0000","last_peered":"2024-03-06T07:22:44.364645+0000","last_clean":"2024-03-06T07:22:44.364645+0000","last_became_active":"2024-03-06T07:22:35.032048+0000","last_became_peered":"2024-03-06T07:22:35.032048+0000","last_unstale":"2024-03-06T07:22:44.364645+0000","last_undegraded":"2024-03-06T07:22:44.364645+0000","last_fullsized":"2024-03-06T07:22:44.364645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:25.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375609+0000","last_change":"2024-03-06T07:22:34.043149+0000","last_active":"2024-03-06T07:22:44.375609+0000","last_peered":"2024-03-06T07:22:44.375609+0000","last_clean":"2024-03-06T07:22:44.375609+0000","last_became_active":"2024-03-06T07:22:34.042888+0000","last_became_peered":"2024-03-06T07:22:34.042888+0000","last_unstale":"2024-03-06T07:22:44.375609+0000","last_undegraded":"2024-03-06T07:22:44.375609+0000","last_fullsized":"2024-03-06T07:22:44.375609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366248+0000","last_change":"2024-03-06T07:22:44.366248+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366248+0000","last_undegraded":"2024-03-06T07:22:44.366248+0000","last_fullsized":"2024-03-06T07:22:44.366248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.384733+0000","last_change":"2024-03-06T07:22:44.379879+0000","last_active":"2024-03-06T07:22:44.384733+0000","last_peered":"2024-03-06T07:22:44.384733+0000","last_clean":"2024-03-06T07:22:44.384733+0000","last_became_active":"2024-03-06T07:22:44.379611+0000","last_became_peered":"2024-03-06T07:22:44.379611+0000","last_unstale":"2024-03-06T07:22:44.384733+0000","last_undegraded":"2024-03-06T07:22:44.384733+0000","last_fullsized":"2024-03-06T07:22:44.384733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:14.207036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363632+0000","last_change":"2024-03-06T07:22:35.034984+0000","last_active":"2024-03-06T07:22:44.363632+0000","last_peered":"2024-03-06T07:22:44.363632+0000","last_clean":"2024-03-06T07:22:44.363632+0000","last_became_active":"2024-03-06T07:22:35.031303+0000","last_became_peered":"2024-03-06T07:22:35.031303+0000","last_unstale":"2024-03-06T07:22:44.363632+0000","last_undegraded":"2024-03-06T07:22:44.363632+0000","last_fullsized":"2024-03-06T07:22:44.363632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365213+0000","last_change":"2024-03-06T07:22:34.044320+0000","last_active":"2024-03-06T07:22:44.365213+0000","last_peered":"2024-03-06T07:22:44.365213+0000","last_clean":"2024-03-06T07:22:44.365213+0000","last_became_active":"2024-03-06T07:22:34.043832+0000","last_became_peered":"2024-03-06T07:22:34.043832+0000","last_unstale":"2024-03-06T07:22:44.365213+0000","last_undegraded":"2024-03-06T07:22:44.365213+0000","last_fullsized":"2024-03-06T07:22:44.365213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366779+0000","last_change":"2024-03-06T07:22:44.366779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366779+0000","last_undegraded":"2024-03-06T07:22:44.366779+0000","last_fullsized":"2024-03-06T07:22:44.366779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377398+0000","last_change":"2024-03-06T07:22:44.373965+0000","last_active":"2024-03-06T07:22:44.377398+0000","last_peered":"2024-03-06T07:22:44.377398+0000","last_clean":"2024-03-06T07:22:44.377398+0000","last_became_active":"2024-03-06T07:22:44.373617+0000","last_became_peered":"2024-03-06T07:22:44.373617+0000","last_unstale":"2024-03-06T07:22:44.377398+0000","last_undegraded":"2024-03-06T07:22:44.377398+0000","last_fullsized":"2024-03-06T07:22:44.377398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:01.683892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363411+0000","last_change":"2024-03-06T07:22:35.034203+0000","last_active":"2024-03-06T07:22:44.363411+0000","last_peered":"2024-03-06T07:22:44.363411+0000","last_clean":"2024-03-06T07:22:44.363411+0000","last_became_active":"2024-03-06T07:22:35.033903+0000","last_became_peered":"2024-03-06T07:22:35.033903+0000","last_unstale":"2024-03-06T07:22:44.363411+0000","last_undegraded":"2024-03-06T07:22:44.363411+0000","last_fullsized":"2024-03-06T07:22:44.363411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:02:14.829352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361765+0000","last_change":"2024-03-06T07:22:34.036176+0000","last_active":"2024-03-06T07:22:44.361765+0000","last_peered":"2024-03-06T07:22:44.361765+0000","last_clean":"2024-03-06T07:22:44.361765+0000","last_became_active":"2024-03-06T07:22:34.035867+0000","last_became_peered":"2024-03-06T07:22:34.035867+0000","last_unstale":"2024-03-06T07:22:44.361765+0000","last_undegraded":"2024-03-06T07:22:44.361765+0000","last_fullsized":"2024-03-06T07:22:44.361765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:28.336495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.374791+0000","last_change":"2024-03-06T07:22:44.374791+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.374791+0000","last_undegraded":"2024-03-06T07:22:44.374791+0000","last_fullsized":"2024-03-06T07:22:44.374791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376378+0000","last_change":"2024-03-06T07:22:44.369401+0000","last_active":"2024-03-06T07:22:44.376378+0000","last_peered":"2024-03-06T07:22:44.376378+0000","last_clean":"2024-03-06T07:22:44.376378+0000","last_became_active":"2024-03-06T07:22:44.369149+0000","last_became_peered":"2024-03-06T07:22:44.369149+0000","last_unstale":"2024-03-06T07:22:44.376378+0000","last_undegraded":"2024-03-06T07:22:44.376378+0000","last_fullsized":"2024-03-06T07:22:44.376378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:29.324559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363089+0000","last_change":"2024-03-06T07:22:35.038497+0000","last_active":"2024-03-06T07:22:44.363089+0000","last_peered":"2024-03-06T07:22:44.363089+0000","last_clean":"2024-03-06T07:22:44.363089+0000","last_became_active":"2024-03-06T07:22:35.038189+0000","last_became_peered":"2024-03-06T07:22:35.038189+0000","last_unstale":"2024-03-06T07:22:44.363089+0000","last_undegraded":"2024-03-06T07:22:44.363089+0000","last_fullsized":"2024-03-06T07:22:44.363089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:34.698852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.369442+0000","last_change":"2024-03-06T07:22:34.038423+0000","last_active":"2024-03-06T07:22:44.369442+0000","last_peered":"2024-03-06T07:22:44.369442+0000","last_clean":"2024-03-06T07:22:44.369442+0000","last_became_active":"2024-03-06T07:22:34.038126+0000","last_became_peered":"2024-03-06T07:22:34.038126+0000","last_unstale":"2024-03-06T07:22:44.369442+0000","last_undegraded":"2024-03-06T07:22:44.369442+0000","last_fullsized":"2024-03-06T07:22:44.369442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:43.441029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.364377+0000","last_change":"2024-03-06T07:22:44.364377+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.364377+0000","last_undegraded":"2024-03-06T07:22:44.364377+0000","last_fullsized":"2024-03-06T07:22:44.364377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.377112+0000","last_change":"2024-03-06T07:22:44.373676+0000","last_active":"2024-03-06T07:22:44.377112+0000","last_peered":"2024-03-06T07:22:44.377112+0000","last_clean":"2024-03-06T07:22:44.377112+0000","last_became_active":"2024-03-06T07:22:44.373209+0000","last_became_peered":"2024-03-06T07:22:44.373209+0000","last_unstale":"2024-03-06T07:22:44.377112+0000","last_undegraded":"2024-03-06T07:22:44.377112+0000","last_fullsized":"2024-03-06T07:22:44.377112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:05.175843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364057+0000","last_change":"2024-03-06T07:22:35.042660+0000","last_active":"2024-03-06T07:22:44.364057+0000","last_peered":"2024-03-06T07:22:44.364057+0000","last_clean":"2024-03-06T07:22:44.364057+0000","last_became_active":"2024-03-06T07:22:35.042327+0000","last_became_peered":"2024-03-06T07:22:35.042327+0000","last_unstale":"2024-03-06T07:22:44.364057+0000","last_undegraded":"2024-03-06T07:22:44.364057+0000","last_fullsized":"2024-03-06T07:22:44.364057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358735+0000","last_change":"2024-03-06T07:22:34.035505+0000","last_active":"2024-03-06T07:22:44.358735+0000","last_peered":"2024-03-06T07:22:44.358735+0000","last_clean":"2024-03-06T07:22:44.358735+0000","last_became_active":"2024-03-06T07:22:34.035257+0000","last_became_peered":"2024-03-06T07:22:34.035257+0000","last_unstale":"2024-03-06T07:22:44.358735+0000","last_undegraded":"2024-03-06T07:22:44.358735+0000","last_fullsized":"2024-03-06T07:22:44.358735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.359116+0000","last_change":"2024-03-06T07:22:44.359116+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.359116+0000","last_undegraded":"2024-03-06T07:22:44.359116+0000","last_fullsized":"2024-03-06T07:22:44.359116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371251+0000","last_change":"2024-03-06T07:22:44.367628+0000","last_active":"2024-03-06T07:22:44.371251+0000","last_peered":"2024-03-06T07:22:44.371251+0000","last_clean":"2024-03-06T07:22:44.371251+0000","last_became_active":"2024-03-06T07:22:44.367384+0000","last_became_peered":"2024-03-06T07:22:44.367384+0000","last_unstale":"2024-03-06T07:22:44.371251+0000","last_undegraded":"2024-03-06T07:22:44.371251+0000","last_fullsized":"2024-03-06T07:22:44.371251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:37.145482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375993+0000","last_change":"2024-03-06T07:22:35.032174+0000","last_active":"2024-03-06T07:22:44.375993+0000","last_peered":"2024-03-06T07:22:44.375993+0000","last_clean":"2024-03-06T07:22:44.375993+0000","last_became_active":"2024-03-06T07:22:35.031869+0000","last_became_peered":"2024-03-06T07:22:35.031869+0000","last_unstale":"2024-03-06T07:22:44.375993+0000","last_undegraded":"2024-03-06T07:22:44.375993+0000","last_fullsized":"2024-03-06T07:22:44.375993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:58.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364160+0000","last_change":"2024-03-06T07:22:34.037475+0000","last_active":"2024-03-06T07:22:44.364160+0000","last_peered":"2024-03-06T07:22:44.364160+0000","last_clean":"2024-03-06T07:22:44.364160+0000","last_became_active":"2024-03-06T07:22:34.037206+0000","last_became_peered":"2024-03-06T07:22:34.037206+0000","last_unstale":"2024-03-06T07:22:44.364160+0000","last_undegraded":"2024-03-06T07:22:44.364160+0000","last_fullsized":"2024-03-06T07:22:44.364160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367154+0000","last_change":"2024-03-06T07:22:44.367154+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367154+0000","last_undegraded":"2024-03-06T07:22:44.367154+0000","last_fullsized":"2024-03-06T07:22:44.367154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376583+0000","last_change":"2024-03-06T07:22:44.372573+0000","last_active":"2024-03-06T07:22:44.376583+0000","last_peered":"2024-03-06T07:22:44.376583+0000","last_clean":"2024-03-06T07:22:44.376583+0000","last_became_active":"2024-03-06T07:22:44.372321+0000","last_became_peered":"2024-03-06T07:22:44.372321+0000","last_unstale":"2024-03-06T07:22:44.376583+0000","last_undegraded":"2024-03-06T07:22:44.376583+0000","last_fullsized":"2024-03-06T07:22:44.376583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:01.508839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373616+0000","last_change":"2024-03-06T07:22:35.042271+0000","last_active":"2024-03-06T07:22:44.373616+0000","last_peered":"2024-03-06T07:22:44.373616+0000","last_clean":"2024-03-06T07:22:44.373616+0000","last_became_active":"2024-03-06T07:22:35.036189+0000","last_became_peered":"2024-03-06T07:22:35.036189+0000","last_unstale":"2024-03-06T07:22:44.373616+0000","last_undegraded":"2024-03-06T07:22:44.373616+0000","last_fullsized":"2024-03-06T07:22:44.373616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:47.064005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364659+0000","last_change":"2024-03-06T07:22:34.036505+0000","last_active":"2024-03-06T07:22:44.364659+0000","last_peered":"2024-03-06T07:22:44.364659+0000","last_clean":"2024-03-06T07:22:44.364659+0000","last_became_active":"2024-03-06T07:22:34.036198+0000","last_became_peered":"2024-03-06T07:22:34.036198+0000","last_unstale":"2024-03-06T07:22:44.364659+0000","last_undegraded":"2024-03-06T07:22:44.364659+0000","last_fullsized":"2024-03-06T07:22:44.364659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362951+0000","last_change":"2024-03-06T07:22:44.362951+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362951+0000","last_undegraded":"2024-03-06T07:22:44.362951+0000","last_fullsized":"2024-03-06T07:22:44.362951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373381+0000","last_change":"2024-03-06T07:22:44.372300+0000","last_active":"2024-03-06T07:22:44.373381+0000","last_peered":"2024-03-06T07:22:44.373381+0000","last_clean":"2024-03-06T07:22:44.373381+0000","last_became_active":"2024-03-06T07:22:44.372036+0000","last_became_peered":"2024-03-06T07:22:44.372036+0000","last_unstale":"2024-03-06T07:22:44.373381+0000","last_undegraded":"2024-03-06T07:22:44.373381+0000","last_fullsized":"2024-03-06T07:22:44.373381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364758+0000","last_change":"2024-03-06T07:22:35.031196+0000","last_active":"2024-03-06T07:22:44.364758+0000","last_peered":"2024-03-06T07:22:44.364758+0000","last_clean":"2024-03-06T07:22:44.364758+0000","last_became_active":"2024-03-06T07:22:35.030767+0000","last_became_peered":"2024-03-06T07:22:35.030767+0000","last_unstale":"2024-03-06T07:22:44.364758+0000","last_undegraded":"2024-03-06T07:22:44.364758+0000","last_fullsized":"2024-03-06T07:22:44.364758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:43.288135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363735+0000","last_change":"2024-03-06T07:22:34.034863+0000","last_active":"2024-03-06T07:22:44.363735+0000","last_peered":"2024-03-06T07:22:44.363735+0000","last_clean":"2024-03-06T07:22:44.363735+0000","last_became_active":"2024-03-06T07:22:34.034577+0000","last_became_peered":"2024-03-06T07:22:34.034577+0000","last_unstale":"2024-03-06T07:22:44.363735+0000","last_undegraded":"2024-03-06T07:22:44.363735+0000","last_fullsized":"2024-03-06T07:22:44.363735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363562+0000","last_change":"2024-03-06T07:22:44.363562+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363562+0000","last_undegraded":"2024-03-06T07:22:44.363562+0000","last_fullsized":"2024-03-06T07:22:44.363562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379268+0000","last_change":"2024-03-06T07:22:44.374457+0000","last_active":"2024-03-06T07:22:44.379268+0000","last_peered":"2024-03-06T07:22:44.379268+0000","last_clean":"2024-03-06T07:22:44.379268+0000","last_became_active":"2024-03-06T07:22:44.374059+0000","last_became_peered":"2024-03-06T07:22:44.374059+0000","last_unstale":"2024-03-06T07:22:44.379268+0000","last_undegraded":"2024-03-06T07:22:44.379268+0000","last_fullsized":"2024-03-06T07:22:44.379268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:27.741841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.360947+0000","last_change":"2024-03-06T07:22:35.036664+0000","last_active":"2024-03-06T07:22:44.360947+0000","last_peered":"2024-03-06T07:22:44.360947+0000","last_clean":"2024-03-06T07:22:44.360947+0000","last_became_active":"2024-03-06T07:22:35.036379+0000","last_became_peered":"2024-03-06T07:22:35.036379+0000","last_unstale":"2024-03-06T07:22:44.360947+0000","last_undegraded":"2024-03-06T07:22:44.360947+0000","last_fullsized":"2024-03-06T07:22:44.360947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:56.002343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.366830+0000","last_change":"2024-03-06T07:22:34.044993+0000","last_active":"2024-03-06T07:22:44.366830+0000","last_peered":"2024-03-06T07:22:44.366830+0000","last_clean":"2024-03-06T07:22:44.366830+0000","last_became_active":"2024-03-06T07:22:34.044795+0000","last_became_peered":"2024-03-06T07:22:34.044795+0000","last_unstale":"2024-03-06T07:22:44.366830+0000","last_undegraded":"2024-03-06T07:22:44.366830+0000","last_fullsized":"2024-03-06T07:22:44.366830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.256142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367883+0000","last_change":"2024-03-06T07:22:44.367883+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367883+0000","last_undegraded":"2024-03-06T07:22:44.367883+0000","last_fullsized":"2024-03-06T07:22:44.367883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.371628+0000","last_change":"2024-03-06T07:22:44.369924+0000","last_active":"2024-03-06T07:22:44.371628+0000","last_peered":"2024-03-06T07:22:44.371628+0000","last_clean":"2024-03-06T07:22:44.371628+0000","last_became_active":"2024-03-06T07:22:44.369676+0000","last_became_peered":"2024-03-06T07:22:44.369676+0000","last_unstale":"2024-03-06T07:22:44.371628+0000","last_undegraded":"2024-03-06T07:22:44.371628+0000","last_fullsized":"2024-03-06T07:22:44.371628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:25.600800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365427+0000","last_change":"2024-03-06T07:22:35.031505+0000","last_active":"2024-03-06T07:22:44.365427+0000","last_peered":"2024-03-06T07:22:44.365427+0000","last_clean":"2024-03-06T07:22:44.365427+0000","last_became_active":"2024-03-06T07:22:35.031241+0000","last_became_peered":"2024-03-06T07:22:35.031241+0000","last_unstale":"2024-03-06T07:22:44.365427+0000","last_undegraded":"2024-03-06T07:22:44.365427+0000","last_fullsized":"2024-03-06T07:22:44.365427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373724+0000","last_change":"2024-03-06T07:22:34.044453+0000","last_active":"2024-03-06T07:22:44.373724+0000","last_peered":"2024-03-06T07:22:44.373724+0000","last_clean":"2024-03-06T07:22:44.373724+0000","last_became_active":"2024-03-06T07:22:34.044190+0000","last_became_peered":"2024-03-06T07:22:34.044190+0000","last_unstale":"2024-03-06T07:22:44.373724+0000","last_undegraded":"2024-03-06T07:22:44.373724+0000","last_fullsized":"2024-03-06T07:22:44.373724+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.367845+0000","last_change":"2024-03-06T07:22:44.367845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.367845+0000","last_undegraded":"2024-03-06T07:22:44.367845+0000","last_fullsized":"2024-03-06T07:22:44.367845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382847+0000","last_change":"2024-03-06T07:22:44.382519+0000","last_active":"2024-03-06T07:22:44.382847+0000","last_peered":"2024-03-06T07:22:44.382847+0000","last_clean":"2024-03-06T07:22:44.382847+0000","last_became_active":"2024-03-06T07:22:44.382295+0000","last_became_peered":"2024-03-06T07:22:44.382295+0000","last_unstale":"2024-03-06T07:22:44.382847+0000","last_undegraded":"2024-03-06T07:22:44.382847+0000","last_fullsized":"2024-03-06T07:22:44.382847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:00.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.361555+0000","last_change":"2024-03-06T07:22:35.034792+0000","last_active":"2024-03-06T07:22:44.361555+0000","last_peered":"2024-03-06T07:22:44.361555+0000","last_clean":"2024-03-06T07:22:44.361555+0000","last_became_active":"2024-03-06T07:22:35.034503+0000","last_became_peered":"2024-03-06T07:22:35.034503+0000","last_unstale":"2024-03-06T07:22:44.361555+0000","last_undegraded":"2024-03-06T07:22:44.361555+0000","last_fullsized":"2024-03-06T07:22:44.361555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368734+0000","last_change":"2024-03-06T07:22:34.037188+0000","last_active":"2024-03-06T07:22:44.368734+0000","last_peered":"2024-03-06T07:22:44.368734+0000","last_clean":"2024-03-06T07:22:44.368734+0000","last_became_active":"2024-03-06T07:22:34.036918+0000","last_became_peered":"2024-03-06T07:22:34.036918+0000","last_unstale":"2024-03-06T07:22:44.368734+0000","last_undegraded":"2024-03-06T07:22:44.368734+0000","last_fullsized":"2024-03-06T07:22:44.368734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.361861+0000","last_change":"2024-03-06T07:22:44.361861+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.361861+0000","last_undegraded":"2024-03-06T07:22:44.361861+0000","last_fullsized":"2024-03-06T07:22:44.361861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379180+0000","last_change":"2024-03-06T07:22:44.371466+0000","last_active":"2024-03-06T07:22:44.379180+0000","last_peered":"2024-03-06T07:22:44.379180+0000","last_clean":"2024-03-06T07:22:44.379180+0000","last_became_active":"2024-03-06T07:22:44.371190+0000","last_became_peered":"2024-03-06T07:22:44.371190+0000","last_unstale":"2024-03-06T07:22:44.379180+0000","last_undegraded":"2024-03-06T07:22:44.379180+0000","last_fullsized":"2024-03-06T07:22:44.379180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:32.601916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.370567+0000","last_change":"2024-03-06T07:22:35.033263+0000","last_active":"2024-03-06T07:22:44.370567+0000","last_peered":"2024-03-06T07:22:44.370567+0000","last_clean":"2024-03-06T07:22:44.370567+0000","last_became_active":"2024-03-06T07:22:35.033005+0000","last_became_peered":"2024-03-06T07:22:35.033005+0000","last_unstale":"2024-03-06T07:22:44.370567+0000","last_undegraded":"2024-03-06T07:22:44.370567+0000","last_fullsized":"2024-03-06T07:22:44.370567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:00.137617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364936+0000","last_change":"2024-03-06T07:22:34.035735+0000","last_active":"2024-03-06T07:22:44.364936+0000","last_peered":"2024-03-06T07:22:44.364936+0000","last_clean":"2024-03-06T07:22:44.364936+0000","last_became_active":"2024-03-06T07:22:34.035459+0000","last_became_peered":"2024-03-06T07:22:34.035459+0000","last_unstale":"2024-03-06T07:22:44.364936+0000","last_undegraded":"2024-03-06T07:22:44.364936+0000","last_fullsized":"2024-03-06T07:22:44.364936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.366886+0000","last_change":"2024-03-06T07:22:44.366886+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.366886+0000","last_undegraded":"2024-03-06T07:22:44.366886+0000","last_fullsized":"2024-03-06T07:22:44.366886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.382022+0000","last_change":"2024-03-06T07:22:44.377358+0000","last_active":"2024-03-06T07:22:44.382022+0000","last_peered":"2024-03-06T07:22:44.382022+0000","last_clean":"2024-03-06T07:22:44.382022+0000","last_became_active":"2024-03-06T07:22:44.377098+0000","last_became_peered":"2024-03-06T07:22:44.377098+0000","last_unstale":"2024-03-06T07:22:44.382022+0000","last_undegraded":"2024-03-06T07:22:44.382022+0000","last_fullsized":"2024-03-06T07:22:44.382022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:43.869151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363978+0000","last_change":"2024-03-06T07:22:35.038111+0000","last_active":"2024-03-06T07:22:44.363978+0000","last_peered":"2024-03-06T07:22:44.363978+0000","last_clean":"2024-03-06T07:22:44.363978+0000","last_became_active":"2024-03-06T07:22:35.037779+0000","last_became_peered":"2024-03-06T07:22:35.037779+0000","last_unstale":"2024-03-06T07:22:44.363978+0000","last_undegraded":"2024-03-06T07:22:44.363978+0000","last_fullsized":"2024-03-06T07:22:44.363978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.362629+0000","last_change":"2024-03-06T07:22:34.039910+0000","last_active":"2024-03-06T07:22:44.362629+0000","last_peered":"2024-03-06T07:22:44.362629+0000","last_clean":"2024-03-06T07:22:44.362629+0000","last_became_active":"2024-03-06T07:22:34.039246+0000","last_became_peered":"2024-03-06T07:22:34.039246+0000","last_unstale":"2024-03-06T07:22:44.362629+0000","last_undegraded":"2024-03-06T07:22:44.362629+0000","last_fullsized":"2024-03-06T07:22:44.362629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:41.961694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.362779+0000","last_change":"2024-03-06T07:22:44.362779+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.362779+0000","last_undegraded":"2024-03-06T07:22:44.362779+0000","last_fullsized":"2024-03-06T07:22:44.362779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.379240+0000","last_change":"2024-03-06T07:22:44.374437+0000","last_active":"2024-03-06T07:22:44.379240+0000","last_peered":"2024-03-06T07:22:44.379240+0000","last_clean":"2024-03-06T07:22:44.379240+0000","last_became_active":"2024-03-06T07:22:44.374055+0000","last_became_peered":"2024-03-06T07:22:44.374055+0000","last_unstale":"2024-03-06T07:22:44.379240+0000","last_undegraded":"2024-03-06T07:22:44.379240+0000","last_fullsized":"2024-03-06T07:22:44.379240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:53:05.282920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363282+0000","last_change":"2024-03-06T07:22:35.027657+0000","last_active":"2024-03-06T07:22:44.363282+0000","last_peered":"2024-03-06T07:22:44.363282+0000","last_clean":"2024-03-06T07:22:44.363282+0000","last_became_active":"2024-03-06T07:22:35.027404+0000","last_became_peered":"2024-03-06T07:22:35.027404+0000","last_unstale":"2024-03-06T07:22:44.363282+0000","last_undegraded":"2024-03-06T07:22:44.363282+0000","last_fullsized":"2024-03-06T07:22:44.363282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:34.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.364859+0000","last_change":"2024-03-06T07:22:34.039209+0000","last_active":"2024-03-06T07:22:44.364859+0000","last_peered":"2024-03-06T07:22:44.364859+0000","last_clean":"2024-03-06T07:22:44.364859+0000","last_became_active":"2024-03-06T07:22:34.038921+0000","last_became_peered":"2024-03-06T07:22:34.038921+0000","last_unstale":"2024-03-06T07:22:44.364859+0000","last_undegraded":"2024-03-06T07:22:44.364859+0000","last_fullsized":"2024-03-06T07:22:44.364859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:12.359435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363182+0000","last_change":"2024-03-06T07:22:44.363182+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363182+0000","last_undegraded":"2024-03-06T07:22:44.363182+0000","last_fullsized":"2024-03-06T07:22:44.363182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.376720+0000","last_change":"2024-03-06T07:22:44.374940+0000","last_active":"2024-03-06T07:22:44.376720+0000","last_peered":"2024-03-06T07:22:44.376720+0000","last_clean":"2024-03-06T07:22:44.376720+0000","last_became_active":"2024-03-06T07:22:44.374653+0000","last_became_peered":"2024-03-06T07:22:44.374653+0000","last_unstale":"2024-03-06T07:22:44.376720+0000","last_undegraded":"2024-03-06T07:22:44.376720+0000","last_fullsized":"2024-03-06T07:22:44.376720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359439+0000","last_change":"2024-03-06T07:22:35.038010+0000","last_active":"2024-03-06T07:22:44.359439+0000","last_peered":"2024-03-06T07:22:44.359439+0000","last_clean":"2024-03-06T07:22:44.359439+0000","last_became_active":"2024-03-06T07:22:35.037717+0000","last_became_peered":"2024-03-06T07:22:35.037717+0000","last_unstale":"2024-03-06T07:22:44.359439+0000","last_undegraded":"2024-03-06T07:22:44.359439+0000","last_fullsized":"2024-03-06T07:22:44.359439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365513+0000","last_change":"2024-03-06T07:22:34.039679+0000","last_active":"2024-03-06T07:22:44.365513+0000","last_peered":"2024-03-06T07:22:44.365513+0000","last_clean":"2024-03-06T07:22:44.365513+0000","last_became_active":"2024-03-06T07:22:34.039092+0000","last_became_peered":"2024-03-06T07:22:34.039092+0000","last_unstale":"2024-03-06T07:22:44.365513+0000","last_undegraded":"2024-03-06T07:22:44.365513+0000","last_fullsized":"2024-03-06T07:22:44.365513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.370845+0000","last_change":"2024-03-06T07:22:44.370845+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.370845+0000","last_undegraded":"2024-03-06T07:22:44.370845+0000","last_fullsized":"2024-03-06T07:22:44.370845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.373582+0000","last_change":"2024-03-06T07:22:44.369327+0000","last_active":"2024-03-06T07:22:44.373582+0000","last_peered":"2024-03-06T07:22:44.373582+0000","last_clean":"2024-03-06T07:22:44.373582+0000","last_became_active":"2024-03-06T07:22:44.369076+0000","last_became_peered":"2024-03-06T07:22:44.369076+0000","last_unstale":"2024-03-06T07:22:44.373582+0000","last_undegraded":"2024-03-06T07:22:44.373582+0000","last_fullsized":"2024-03-06T07:22:44.373582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:52.087902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.359257+0000","last_change":"2024-03-06T07:22:35.026900+0000","last_active":"2024-03-06T07:22:44.359257+0000","last_peered":"2024-03-06T07:22:44.359257+0000","last_clean":"2024-03-06T07:22:44.359257+0000","last_became_active":"2024-03-06T07:22:35.026626+0000","last_became_peered":"2024-03-06T07:22:35.026626+0000","last_unstale":"2024-03-06T07:22:44.359257+0000","last_undegraded":"2024-03-06T07:22:44.359257+0000","last_fullsized":"2024-03-06T07:22:44.359257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.358578+0000","last_change":"2024-03-06T07:22:34.039382+0000","last_active":"2024-03-06T07:22:44.358578+0000","last_peered":"2024-03-06T07:22:44.358578+0000","last_clean":"2024-03-06T07:22:44.358578+0000","last_became_active":"2024-03-06T07:22:34.039088+0000","last_became_peered":"2024-03-06T07:22:34.039088+0000","last_unstale":"2024-03-06T07:22:44.358578+0000","last_undegraded":"2024-03-06T07:22:44.358578+0000","last_fullsized":"2024-03-06T07:22:44.358578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-03-06T07:22:44.363505+0000","last_change":"2024-03-06T07:22:44.363505+0000","last_active":"2024-03-06T07:22:44.351281+0000","last_peered":"2024-03-06T07:22:44.351281+0000","last_clean":"2024-03-06T07:22:44.351281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:44.363505+0000","last_undegraded":"2024-03-06T07:22:44.363505+0000","last_fullsized":"2024-03-06T07:22:44.363505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.375571+0000","last_change":"2024-03-06T07:22:44.374409+0000","last_active":"2024-03-06T07:22:44.375571+0000","last_peered":"2024-03-06T07:22:44.375571+0000","last_clean":"2024-03-06T07:22:44.375571+0000","last_became_active":"2024-03-06T07:22:44.374130+0000","last_became_peered":"2024-03-06T07:22:44.374130+0000","last_unstale":"2024-03-06T07:22:44.375571+0000","last_undegraded":"2024-03-06T07:22:44.375571+0000","last_fullsized":"2024-03-06T07:22:44.375571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:30.597343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.368651+0000","last_change":"2024-03-06T07:22:35.033671+0000","last_active":"2024-03-06T07:22:44.368651+0000","last_peered":"2024-03-06T07:22:44.368651+0000","last_clean":"2024-03-06T07:22:44.368651+0000","last_became_active":"2024-03-06T07:22:35.033167+0000","last_became_peered":"2024-03-06T07:22:35.033167+0000","last_unstale":"2024-03-06T07:22:44.368651+0000","last_undegraded":"2024-03-06T07:22:44.368651+0000","last_fullsized":"2024-03-06T07:22:44.368651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.365069+0000","last_change":"2024-03-06T07:22:34.037026+0000","last_active":"2024-03-06T07:22:44.365069+0000","last_peered":"2024-03-06T07:22:44.365069+0000","last_clean":"2024-03-06T07:22:44.365069+0000","last_became_active":"2024-03-06T07:22:34.036742+0000","last_became_peered":"2024-03-06T07:22:34.036742+0000","last_unstale":"2024-03-06T07:22:44.365069+0000","last_undegraded":"2024-03-06T07:22:44.365069+0000","last_fullsized":"2024-03-06T07:22:44.365069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-06T07:22:44.363945+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:22:44.363945+0000","last_peered":"2024-03-06T07:22:44.363945+0000","last_clean":"2024-03-06T07:22:44.363945+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:22:44.363945+0000","last_undegraded":"2024-03-06T07:22:44.363945+0000","last_fullsized":"2024-03-06T07:22:44.363945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":1032479,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":443701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":442123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":443805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":435149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":442831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":442201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":1032457,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:50.309 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-03-06T07:22:50.308+0000 7fbf748fb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:53.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0022237640 1 Processor -- start 2024-03-06T07:22:53.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0022237640 1 -- start start 2024-03-06T07:22:53.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0022237640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f001c155ba0 0x7f001c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0022237640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f001c1586f0 0x7f001c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0022237640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c1591c0 con 0x7f001c0c5480 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0022237640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c06df40 con 0x7f001c155ba0 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0022237640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c06fdb0 con 0x7f001c1586f0 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.134+0000 7f0012ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f001c155ba0 0x7f001c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f0012ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f001c155ba0 0x7f001c0c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49222/0 (socket says 172.21.15.26:49222) 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f0012ffd640 1 -- 172.21.15.26:0/267332556 learned_addr learned my addr 172.21.15.26:0/267332556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:53.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f001bfff640 1 --2- 172.21.15.26:0/267332556 >> v2:172.21.15.117:3300/0 conn(0x7f001c1586f0 0x7f001c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f0012ffd640 1 -- 172.21.15.26:0/267332556 >> v2:172.21.15.117:3300/0 conn(0x7f001c1586f0 msgr2=0x7f001c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f0012ffd640 1 --2- 172.21.15.26:0/267332556 >> v2:172.21.15.117:3300/0 conn(0x7f001c1586f0 0x7f001c158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f0012ffd640 1 -- 172.21.15.26:0/267332556 >> v1:172.21.15.26:6789/0 conn(0x7f001c0c5480 legacy=0x7f001c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f0012ffd640 1 -- 172.21.15.26:0/267332556 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c1594a0 con 0x7f001c155ba0 2024-03-06T07:22:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.135+0000 7f001affd640 1 -- 172.21.15.26:0/267332556 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504951427 0 0) 0x7f001c1591c0 con 0x7f001c0c5480 2024-03-06T07:22:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0012ffd640 1 --2- 172.21.15.26:0/267332556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c0c5860 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f001c0ab500 tx=0x7f000402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b820bb5b7958c502 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f001affd640 1 -- 172.21.15.26:0/267332556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004035020 con 0x7f001c155ba0 2024-03-06T07:22:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f001affd640 1 -- 172.21.15.26:0/267332556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0004031870 con 0x7f001c155ba0 2024-03-06T07:22:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f001affd640 1 -- 172.21.15.26:0/267332556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004031c40 con 0x7f001c155ba0 2024-03-06T07:22:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 -- 172.21.15.26:0/267332556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 msgr2=0x7f001c0c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 --2- 172.21.15.26:0/267332556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c0c5860 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f001c0ab500 tx=0x7f000402b910 comp rx=0 tx=0).stop 2024-03-06T07:22:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 -- 172.21.15.26:0/267332556 shutdown_connections 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 --2- 172.21.15.26:0/267332556 >> v2:172.21.15.117:3300/0 conn(0x7f001c1586f0 0x7f001c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 --2- 172.21.15.26:0/267332556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c0c5860 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 -- 172.21.15.26:0/267332556 >> 172.21.15.26:0/267332556 conn(0x7f001c06b1a0 msgr2=0x7f001c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 -- 172.21.15.26:0/267332556 shutdown_connections 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.136+0000 7f0022237640 1 -- 172.21.15.26:0/267332556 wait complete. 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.137+0000 7f0022237640 1 Processor -- start 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0022237640 1 -- start start 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0022237640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f001c0c5480 0x7f001c14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0022237640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c14d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0022237640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f001c1586f0 0x7f001c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0022237640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c155680 con 0x7f001c155ba0 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0022237640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c156000 con 0x7f001c0c5480 2024-03-06T07:22:53.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0022237640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c152a20 con 0x7f001c1586f0 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0012ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c14d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0012ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c14d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49238/0 (socket says 172.21.15.26:49238) 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f0012ffd640 1 -- 172.21.15.26:0/2907685230 learned_addr learned my addr 172.21.15.26:0/2907685230 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.138+0000 7f001b7fe640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f001c0c5480 0x7f001c14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f001bfff640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f001c1586f0 0x7f001c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0012ffd640 1 -- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f001c1586f0 msgr2=0x7f001c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0012ffd640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f001c1586f0 0x7f001c156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0012ffd640 1 -- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f001c0c5480 msgr2=0x7f001c14aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0012ffd640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f001c0c5480 0x7f001c14aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0012ffd640 1 -- 172.21.15.26:0/2907685230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c153a30 con 0x7f001c155ba0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0012ffd640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c14d160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f000402b950 tx=0x7f00040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0018ff9640 1 -- 172.21.15.26:0/2907685230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004040070 con 0x7f001c155ba0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0018ff9640 1 -- 172.21.15.26:0/2907685230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0004030d00 con 0x7f001c155ba0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c153d10 con 0x7f001c155ba0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0018ff9640 1 -- 172.21.15.26:0/2907685230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004035040 con 0x7f001c155ba0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.139+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1542d0 con 0x7f001c155ba0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.140+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c1544d0 con 0x7f001c155ba0 2024-03-06T07:22:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.141+0000 7f0018ff9640 1 -- 172.21.15.26:0/2907685230 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f000404b060 con 0x7f001c155ba0 2024-03-06T07:22:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.141+0000 7f0018ff9640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.141+0000 7f0018ff9640 1 -- 172.21.15.26:0/2907685230 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f00040ac1c0 con 0x7f001c155ba0 2024-03-06T07:22:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.141+0000 7f001b7fe640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.142+0000 7f001b7fe640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efffc06b4d0 0x7efffc06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f000c000f60 tx=0x7f000c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.146+0000 7f0018ff9640 1 -- 172.21.15.26:0/2907685230 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f001c1544d0 con 0x7f001c155ba0 2024-03-06T07:22:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.322+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f001c04f330 con 0x7efffc06b4d0 2024-03-06T07:22:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.334+0000 7f0018ff9640 1 -- 172.21.15.26:0/2907685230 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490719 (secure 0 0 0) 0x7f001c04f330 con 0x7efffc06b4d0 2024-03-06T07:22:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:53.344 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:22:53.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efffc06b4d0 msgr2=0x7efffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efffc06b4d0 0x7efffc06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f000c000f60 tx=0x7f000c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 msgr2=0x7f001c14d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c14d160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f000402b950 tx=0x7f00040046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 shutdown_connections 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f001c1586f0 0x7f001c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f001c155ba0 0x7f001c14d160 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 --2- 172.21.15.26:0/2907685230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f001c0c5480 0x7f001c14aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 >> 172.21.15.26:0/2907685230 conn(0x7f001c06b1a0 msgr2=0x7f001c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 shutdown_connections 2024-03-06T07:22:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.338+0000 7f0022237640 1 -- 172.21.15.26:0/2907685230 wait complete. 2024-03-06T07:22:53.351 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-03-06T07:22:53.065147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":381,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244676,"kb_used_data":7712,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730044,"statfs":{"total":773094113280,"available":772843565056,"internally_reserved":0,"allocated":7897088,"data_stored":4851680,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001753"},"pg_stats":[{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099175+0000","last_change":"2024-03-06T07:22:45.387294+0000","last_active":"2024-03-06T07:22:48.099175+0000","last_peered":"2024-03-06T07:22:48.099175+0000","last_clean":"2024-03-06T07:22:48.099175+0000","last_became_active":"2024-03-06T07:22:45.386937+0000","last_became_peered":"2024-03-06T07:22:45.386937+0000","last_unstale":"2024-03-06T07:22:48.099175+0000","last_undegraded":"2024-03-06T07:22:48.099175+0000","last_fullsized":"2024-03-06T07:22:48.099175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:14:27.420771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379482+0000","last_change":"2024-03-06T07:22:35.025691+0000","last_active":"2024-03-06T07:22:46.379482+0000","last_peered":"2024-03-06T07:22:46.379482+0000","last_clean":"2024-03-06T07:22:46.379482+0000","last_became_active":"2024-03-06T07:22:35.025394+0000","last_became_peered":"2024-03-06T07:22:35.025394+0000","last_unstale":"2024-03-06T07:22:46.379482+0000","last_undegraded":"2024-03-06T07:22:46.379482+0000","last_fullsized":"2024-03-06T07:22:46.379482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931420+0000","last_change":"2024-03-06T07:22:45.373367+0000","last_active":"2024-03-06T07:22:46.931420+0000","last_peered":"2024-03-06T07:22:46.931420+0000","last_clean":"2024-03-06T07:22:46.931420+0000","last_became_active":"2024-03-06T07:22:45.373099+0000","last_became_peered":"2024-03-06T07:22:45.373099+0000","last_unstale":"2024-03-06T07:22:46.931420+0000","last_undegraded":"2024-03-06T07:22:46.931420+0000","last_fullsized":"2024-03-06T07:22:46.931420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:09.313736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098457+0000","last_change":"2024-03-06T07:22:35.048775+0000","last_active":"2024-03-06T07:22:48.098457+0000","last_peered":"2024-03-06T07:22:48.098457+0000","last_clean":"2024-03-06T07:22:48.098457+0000","last_became_active":"2024-03-06T07:22:35.048476+0000","last_became_peered":"2024-03-06T07:22:35.048476+0000","last_unstale":"2024-03-06T07:22:48.098457+0000","last_undegraded":"2024-03-06T07:22:48.098457+0000","last_fullsized":"2024-03-06T07:22:48.098457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:41.704388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099169+0000","last_change":"2024-03-06T07:22:45.393480+0000","last_active":"2024-03-06T07:22:48.099169+0000","last_peered":"2024-03-06T07:22:48.099169+0000","last_clean":"2024-03-06T07:22:48.099169+0000","last_became_active":"2024-03-06T07:22:45.393199+0000","last_became_peered":"2024-03-06T07:22:45.393199+0000","last_unstale":"2024-03-06T07:22:48.099169+0000","last_undegraded":"2024-03-06T07:22:48.099169+0000","last_fullsized":"2024-03-06T07:22:48.099169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:01.230997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931702+0000","last_change":"2024-03-06T07:22:35.037727+0000","last_active":"2024-03-06T07:22:46.931702+0000","last_peered":"2024-03-06T07:22:46.931702+0000","last_clean":"2024-03-06T07:22:46.931702+0000","last_became_active":"2024-03-06T07:22:35.037421+0000","last_became_peered":"2024-03-06T07:22:35.037421+0000","last_unstale":"2024-03-06T07:22:46.931702+0000","last_undegraded":"2024-03-06T07:22:46.931702+0000","last_fullsized":"2024-03-06T07:22:46.931702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378489+0000","last_change":"2024-03-06T07:22:45.378622+0000","last_active":"2024-03-06T07:22:46.378489+0000","last_peered":"2024-03-06T07:22:46.378489+0000","last_clean":"2024-03-06T07:22:46.378489+0000","last_became_active":"2024-03-06T07:22:45.378001+0000","last_became_peered":"2024-03-06T07:22:45.378001+0000","last_unstale":"2024-03-06T07:22:46.378489+0000","last_undegraded":"2024-03-06T07:22:46.378489+0000","last_fullsized":"2024-03-06T07:22:46.378489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:31.228719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379032+0000","last_change":"2024-03-06T07:22:35.035949+0000","last_active":"2024-03-06T07:22:46.379032+0000","last_peered":"2024-03-06T07:22:46.379032+0000","last_clean":"2024-03-06T07:22:46.379032+0000","last_became_active":"2024-03-06T07:22:35.035687+0000","last_became_peered":"2024-03-06T07:22:35.035687+0000","last_unstale":"2024-03-06T07:22:46.379032+0000","last_undegraded":"2024-03-06T07:22:46.379032+0000","last_fullsized":"2024-03-06T07:22:46.379032+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:38.074532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378043+0000","last_change":"2024-03-06T07:22:45.377456+0000","last_active":"2024-03-06T07:22:46.378043+0000","last_peered":"2024-03-06T07:22:46.378043+0000","last_clean":"2024-03-06T07:22:46.378043+0000","last_became_active":"2024-03-06T07:22:45.377136+0000","last_became_peered":"2024-03-06T07:22:45.377136+0000","last_unstale":"2024-03-06T07:22:46.378043+0000","last_undegraded":"2024-03-06T07:22:46.378043+0000","last_fullsized":"2024-03-06T07:22:46.378043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:10.182979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078630+0000","last_change":"2024-03-06T07:22:35.033081+0000","last_active":"2024-03-06T07:22:48.078630+0000","last_peered":"2024-03-06T07:22:48.078630+0000","last_clean":"2024-03-06T07:22:48.078630+0000","last_became_active":"2024-03-06T07:22:35.032799+0000","last_became_peered":"2024-03-06T07:22:35.032799+0000","last_unstale":"2024-03-06T07:22:48.078630+0000","last_undegraded":"2024-03-06T07:22:48.078630+0000","last_fullsized":"2024-03-06T07:22:48.078630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:23.187775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.930902+0000","last_change":"2024-03-06T07:22:45.371490+0000","last_active":"2024-03-06T07:22:46.930902+0000","last_peered":"2024-03-06T07:22:46.930902+0000","last_clean":"2024-03-06T07:22:46.930902+0000","last_became_active":"2024-03-06T07:22:45.370989+0000","last_became_peered":"2024-03-06T07:22:45.370989+0000","last_unstale":"2024-03-06T07:22:46.930902+0000","last_undegraded":"2024-03-06T07:22:46.930902+0000","last_fullsized":"2024-03-06T07:22:46.930902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:33.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098792+0000","last_change":"2024-03-06T07:22:35.041976+0000","last_active":"2024-03-06T07:22:48.098792+0000","last_peered":"2024-03-06T07:22:48.098792+0000","last_clean":"2024-03-06T07:22:48.098792+0000","last_became_active":"2024-03-06T07:22:35.041546+0000","last_became_peered":"2024-03-06T07:22:35.041546+0000","last_unstale":"2024-03-06T07:22:48.098792+0000","last_undegraded":"2024-03-06T07:22:48.098792+0000","last_fullsized":"2024-03-06T07:22:48.098792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380437+0000","last_change":"2024-03-06T07:22:45.374895+0000","last_active":"2024-03-06T07:22:46.380437+0000","last_peered":"2024-03-06T07:22:46.380437+0000","last_clean":"2024-03-06T07:22:46.380437+0000","last_became_active":"2024-03-06T07:22:45.374631+0000","last_became_peered":"2024-03-06T07:22:45.374631+0000","last_unstale":"2024-03-06T07:22:46.380437+0000","last_undegraded":"2024-03-06T07:22:46.380437+0000","last_fullsized":"2024-03-06T07:22:46.380437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:33.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958041+0000","last_change":"2024-03-06T07:22:35.038320+0000","last_active":"2024-03-06T07:22:46.958041+0000","last_peered":"2024-03-06T07:22:46.958041+0000","last_clean":"2024-03-06T07:22:46.958041+0000","last_became_active":"2024-03-06T07:22:35.038043+0000","last_became_peered":"2024-03-06T07:22:35.038043+0000","last_unstale":"2024-03-06T07:22:46.958041+0000","last_undegraded":"2024-03-06T07:22:46.958041+0000","last_fullsized":"2024-03-06T07:22:46.958041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097497+0000","last_change":"2024-03-06T07:22:45.378427+0000","last_active":"2024-03-06T07:22:48.097497+0000","last_peered":"2024-03-06T07:22:48.097497+0000","last_clean":"2024-03-06T07:22:48.097497+0000","last_became_active":"2024-03-06T07:22:45.378116+0000","last_became_peered":"2024-03-06T07:22:45.378116+0000","last_unstale":"2024-03-06T07:22:48.097497+0000","last_undegraded":"2024-03-06T07:22:48.097497+0000","last_fullsized":"2024-03-06T07:22:48.097497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:40:38.256295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078437+0000","last_change":"2024-03-06T07:22:35.040651+0000","last_active":"2024-03-06T07:22:48.078437+0000","last_peered":"2024-03-06T07:22:48.078437+0000","last_clean":"2024-03-06T07:22:48.078437+0000","last_became_active":"2024-03-06T07:22:35.040382+0000","last_became_peered":"2024-03-06T07:22:35.040382+0000","last_unstale":"2024-03-06T07:22:48.078437+0000","last_undegraded":"2024-03-06T07:22:48.078437+0000","last_fullsized":"2024-03-06T07:22:48.078437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:54.755446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380834+0000","last_change":"2024-03-06T07:22:45.392277+0000","last_active":"2024-03-06T07:22:46.380834+0000","last_peered":"2024-03-06T07:22:46.380834+0000","last_clean":"2024-03-06T07:22:46.380834+0000","last_became_active":"2024-03-06T07:22:45.392109+0000","last_became_peered":"2024-03-06T07:22:45.392109+0000","last_unstale":"2024-03-06T07:22:46.380834+0000","last_undegraded":"2024-03-06T07:22:46.380834+0000","last_fullsized":"2024-03-06T07:22:46.380834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:26.605704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381043+0000","last_change":"2024-03-06T07:22:35.035524+0000","last_active":"2024-03-06T07:22:46.381043+0000","last_peered":"2024-03-06T07:22:46.381043+0000","last_clean":"2024-03-06T07:22:46.381043+0000","last_became_active":"2024-03-06T07:22:35.035252+0000","last_became_peered":"2024-03-06T07:22:35.035252+0000","last_unstale":"2024-03-06T07:22:46.381043+0000","last_undegraded":"2024-03-06T07:22:46.381043+0000","last_fullsized":"2024-03-06T07:22:46.381043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:26.923431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078865+0000","last_change":"2024-03-06T07:22:45.386887+0000","last_active":"2024-03-06T07:22:48.078865+0000","last_peered":"2024-03-06T07:22:48.078865+0000","last_clean":"2024-03-06T07:22:48.078865+0000","last_became_active":"2024-03-06T07:22:45.385539+0000","last_became_peered":"2024-03-06T07:22:45.385539+0000","last_unstale":"2024-03-06T07:22:48.078865+0000","last_undegraded":"2024-03-06T07:22:48.078865+0000","last_fullsized":"2024-03-06T07:22:48.078865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:27.343692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378591+0000","last_change":"2024-03-06T07:22:35.040188+0000","last_active":"2024-03-06T07:22:46.378591+0000","last_peered":"2024-03-06T07:22:46.378591+0000","last_clean":"2024-03-06T07:22:46.378591+0000","last_became_active":"2024-03-06T07:22:35.039868+0000","last_became_peered":"2024-03-06T07:22:35.039868+0000","last_unstale":"2024-03-06T07:22:46.378591+0000","last_undegraded":"2024-03-06T07:22:46.378591+0000","last_fullsized":"2024-03-06T07:22:46.378591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:04.193333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381024+0000","last_change":"2024-03-06T07:22:45.381531+0000","last_active":"2024-03-06T07:22:46.381024+0000","last_peered":"2024-03-06T07:22:46.381024+0000","last_clean":"2024-03-06T07:22:46.381024+0000","last_became_active":"2024-03-06T07:22:45.381274+0000","last_became_peered":"2024-03-06T07:22:45.381274+0000","last_unstale":"2024-03-06T07:22:46.381024+0000","last_undegraded":"2024-03-06T07:22:46.381024+0000","last_fullsized":"2024-03-06T07:22:46.381024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:16:00.123840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098782+0000","last_change":"2024-03-06T07:22:35.049739+0000","last_active":"2024-03-06T07:22:48.098782+0000","last_peered":"2024-03-06T07:22:48.098782+0000","last_clean":"2024-03-06T07:22:48.098782+0000","last_became_active":"2024-03-06T07:22:35.049118+0000","last_became_peered":"2024-03-06T07:22:35.049118+0000","last_unstale":"2024-03-06T07:22:48.098782+0000","last_undegraded":"2024-03-06T07:22:48.098782+0000","last_fullsized":"2024-03-06T07:22:48.098782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:49.784740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.079055+0000","last_change":"2024-03-06T07:22:45.375970+0000","last_active":"2024-03-06T07:22:48.079055+0000","last_peered":"2024-03-06T07:22:48.079055+0000","last_clean":"2024-03-06T07:22:48.079055+0000","last_became_active":"2024-03-06T07:22:45.375713+0000","last_became_peered":"2024-03-06T07:22:45.375713+0000","last_unstale":"2024-03-06T07:22:48.079055+0000","last_undegraded":"2024-03-06T07:22:48.079055+0000","last_fullsized":"2024-03-06T07:22:48.079055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:06.341443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379061+0000","last_change":"2024-03-06T07:22:35.038831+0000","last_active":"2024-03-06T07:22:46.379061+0000","last_peered":"2024-03-06T07:22:46.379061+0000","last_clean":"2024-03-06T07:22:46.379061+0000","last_became_active":"2024-03-06T07:22:35.038550+0000","last_became_peered":"2024-03-06T07:22:35.038550+0000","last_unstale":"2024-03-06T07:22:46.379061+0000","last_undegraded":"2024-03-06T07:22:46.379061+0000","last_fullsized":"2024-03-06T07:22:46.379061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:02:04.099109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931045+0000","last_change":"2024-03-06T07:22:45.380082+0000","last_active":"2024-03-06T07:22:46.931045+0000","last_peered":"2024-03-06T07:22:46.931045+0000","last_clean":"2024-03-06T07:22:46.931045+0000","last_became_active":"2024-03-06T07:22:45.379820+0000","last_became_peered":"2024-03-06T07:22:45.379820+0000","last_unstale":"2024-03-06T07:22:46.931045+0000","last_undegraded":"2024-03-06T07:22:46.931045+0000","last_fullsized":"2024-03-06T07:22:46.931045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:15.211940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078748+0000","last_change":"2024-03-06T07:22:35.037819+0000","last_active":"2024-03-06T07:22:48.078748+0000","last_peered":"2024-03-06T07:22:48.078748+0000","last_clean":"2024-03-06T07:22:48.078748+0000","last_became_active":"2024-03-06T07:22:35.037519+0000","last_became_peered":"2024-03-06T07:22:35.037519+0000","last_unstale":"2024-03-06T07:22:48.078748+0000","last_undegraded":"2024-03-06T07:22:48.078748+0000","last_fullsized":"2024-03-06T07:22:48.078748+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:49.281137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097574+0000","last_change":"2024-03-06T07:22:45.379889+0000","last_active":"2024-03-06T07:22:48.097574+0000","last_peered":"2024-03-06T07:22:48.097574+0000","last_clean":"2024-03-06T07:22:48.097574+0000","last_became_active":"2024-03-06T07:22:45.378220+0000","last_became_peered":"2024-03-06T07:22:45.378220+0000","last_unstale":"2024-03-06T07:22:48.097574+0000","last_undegraded":"2024-03-06T07:22:48.097574+0000","last_fullsized":"2024-03-06T07:22:48.097574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:15.284604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098879+0000","last_change":"2024-03-06T07:22:35.040052+0000","last_active":"2024-03-06T07:22:48.098879+0000","last_peered":"2024-03-06T07:22:48.098879+0000","last_clean":"2024-03-06T07:22:48.098879+0000","last_became_active":"2024-03-06T07:22:35.039794+0000","last_became_peered":"2024-03-06T07:22:35.039794+0000","last_unstale":"2024-03-06T07:22:48.098879+0000","last_undegraded":"2024-03-06T07:22:48.098879+0000","last_fullsized":"2024-03-06T07:22:48.098879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931117+0000","last_change":"2024-03-06T07:22:45.370155+0000","last_active":"2024-03-06T07:22:46.931117+0000","last_peered":"2024-03-06T07:22:46.931117+0000","last_clean":"2024-03-06T07:22:46.931117+0000","last_became_active":"2024-03-06T07:22:45.369849+0000","last_became_peered":"2024-03-06T07:22:45.369849+0000","last_unstale":"2024-03-06T07:22:46.931117+0000","last_undegraded":"2024-03-06T07:22:46.931117+0000","last_fullsized":"2024-03-06T07:22:46.931117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:10.476716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097524+0000","last_change":"2024-03-06T07:22:35.034108+0000","last_active":"2024-03-06T07:22:48.097524+0000","last_peered":"2024-03-06T07:22:48.097524+0000","last_clean":"2024-03-06T07:22:48.097524+0000","last_became_active":"2024-03-06T07:22:35.033823+0000","last_became_peered":"2024-03-06T07:22:35.033823+0000","last_unstale":"2024-03-06T07:22:48.097524+0000","last_undegraded":"2024-03-06T07:22:48.097524+0000","last_fullsized":"2024-03-06T07:22:48.097524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:04.872840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380355+0000","last_change":"2024-03-06T07:22:45.377080+0000","last_active":"2024-03-06T07:22:46.380355+0000","last_peered":"2024-03-06T07:22:46.380355+0000","last_clean":"2024-03-06T07:22:46.380355+0000","last_became_active":"2024-03-06T07:22:45.376606+0000","last_became_peered":"2024-03-06T07:22:45.376606+0000","last_unstale":"2024-03-06T07:22:46.380355+0000","last_undegraded":"2024-03-06T07:22:46.380355+0000","last_fullsized":"2024-03-06T07:22:46.380355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:45.060496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377851+0000","last_change":"2024-03-06T07:22:35.037317+0000","last_active":"2024-03-06T07:22:46.377851+0000","last_peered":"2024-03-06T07:22:46.377851+0000","last_clean":"2024-03-06T07:22:46.377851+0000","last_became_active":"2024-03-06T07:22:35.033185+0000","last_became_peered":"2024-03-06T07:22:35.033185+0000","last_unstale":"2024-03-06T07:22:46.377851+0000","last_undegraded":"2024-03-06T07:22:46.377851+0000","last_fullsized":"2024-03-06T07:22:46.377851+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381014+0000","last_change":"2024-03-06T07:22:45.387391+0000","last_active":"2024-03-06T07:22:46.381014+0000","last_peered":"2024-03-06T07:22:46.381014+0000","last_clean":"2024-03-06T07:22:46.381014+0000","last_became_active":"2024-03-06T07:22:45.387145+0000","last_became_peered":"2024-03-06T07:22:45.387145+0000","last_unstale":"2024-03-06T07:22:46.381014+0000","last_undegraded":"2024-03-06T07:22:46.381014+0000","last_fullsized":"2024-03-06T07:22:46.381014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:08.790962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.079112+0000","last_change":"2024-03-06T07:22:35.035073+0000","last_active":"2024-03-06T07:22:48.079112+0000","last_peered":"2024-03-06T07:22:48.079112+0000","last_clean":"2024-03-06T07:22:48.079112+0000","last_became_active":"2024-03-06T07:22:35.034739+0000","last_became_peered":"2024-03-06T07:22:35.034739+0000","last_unstale":"2024-03-06T07:22:48.079112+0000","last_undegraded":"2024-03-06T07:22:48.079112+0000","last_fullsized":"2024-03-06T07:22:48.079112+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:12.718069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931442+0000","last_change":"2024-03-06T07:22:45.379973+0000","last_active":"2024-03-06T07:22:46.931442+0000","last_peered":"2024-03-06T07:22:46.931442+0000","last_clean":"2024-03-06T07:22:46.931442+0000","last_became_active":"2024-03-06T07:22:45.379676+0000","last_became_peered":"2024-03-06T07:22:45.379676+0000","last_unstale":"2024-03-06T07:22:46.931442+0000","last_undegraded":"2024-03-06T07:22:46.931442+0000","last_fullsized":"2024-03-06T07:22:46.931442+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:08.790962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931727+0000","last_change":"2024-03-06T07:22:35.032202+0000","last_active":"2024-03-06T07:22:46.931727+0000","last_peered":"2024-03-06T07:22:46.931727+0000","last_clean":"2024-03-06T07:22:46.931727+0000","last_became_active":"2024-03-06T07:22:35.031858+0000","last_became_peered":"2024-03-06T07:22:35.031858+0000","last_unstale":"2024-03-06T07:22:46.931727+0000","last_undegraded":"2024-03-06T07:22:46.931727+0000","last_fullsized":"2024-03-06T07:22:46.931727+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098490+0000","last_change":"2024-03-06T07:22:45.387754+0000","last_active":"2024-03-06T07:22:48.098490+0000","last_peered":"2024-03-06T07:22:48.098490+0000","last_clean":"2024-03-06T07:22:48.098490+0000","last_became_active":"2024-03-06T07:22:45.387478+0000","last_became_peered":"2024-03-06T07:22:45.387478+0000","last_unstale":"2024-03-06T07:22:48.098490+0000","last_undegraded":"2024-03-06T07:22:48.098490+0000","last_fullsized":"2024-03-06T07:22:48.098490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:25:17.919983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099284+0000","last_change":"2024-03-06T07:22:35.041928+0000","last_active":"2024-03-06T07:22:48.099284+0000","last_peered":"2024-03-06T07:22:48.099284+0000","last_clean":"2024-03-06T07:22:48.099284+0000","last_became_active":"2024-03-06T07:22:35.041672+0000","last_became_peered":"2024-03-06T07:22:35.041672+0000","last_unstale":"2024-03-06T07:22:48.099284+0000","last_undegraded":"2024-03-06T07:22:48.099284+0000","last_fullsized":"2024-03-06T07:22:48.099284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099291+0000","last_change":"2024-03-06T07:22:45.375439+0000","last_active":"2024-03-06T07:22:48.099291+0000","last_peered":"2024-03-06T07:22:48.099291+0000","last_clean":"2024-03-06T07:22:48.099291+0000","last_became_active":"2024-03-06T07:22:45.375050+0000","last_became_peered":"2024-03-06T07:22:45.375050+0000","last_unstale":"2024-03-06T07:22:48.099291+0000","last_undegraded":"2024-03-06T07:22:48.099291+0000","last_fullsized":"2024-03-06T07:22:48.099291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:31.228719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078729+0000","last_change":"2024-03-06T07:22:35.036178+0000","last_active":"2024-03-06T07:22:48.078729+0000","last_peered":"2024-03-06T07:22:48.078729+0000","last_clean":"2024-03-06T07:22:48.078729+0000","last_became_active":"2024-03-06T07:22:35.035874+0000","last_became_peered":"2024-03-06T07:22:35.035874+0000","last_unstale":"2024-03-06T07:22:48.078729+0000","last_undegraded":"2024-03-06T07:22:48.078729+0000","last_fullsized":"2024-03-06T07:22:48.078729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.764602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958178+0000","last_change":"2024-03-06T07:22:45.387019+0000","last_active":"2024-03-06T07:22:46.958178+0000","last_peered":"2024-03-06T07:22:46.958178+0000","last_clean":"2024-03-06T07:22:46.958178+0000","last_became_active":"2024-03-06T07:22:45.386761+0000","last_became_peered":"2024-03-06T07:22:45.386761+0000","last_unstale":"2024-03-06T07:22:46.958178+0000","last_undegraded":"2024-03-06T07:22:46.958178+0000","last_fullsized":"2024-03-06T07:22:46.958178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:16.677632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380621+0000","last_change":"2024-03-06T07:22:35.031787+0000","last_active":"2024-03-06T07:22:46.380621+0000","last_peered":"2024-03-06T07:22:46.380621+0000","last_clean":"2024-03-06T07:22:46.380621+0000","last_became_active":"2024-03-06T07:22:35.031524+0000","last_became_peered":"2024-03-06T07:22:35.031524+0000","last_unstale":"2024-03-06T07:22:46.380621+0000","last_undegraded":"2024-03-06T07:22:46.380621+0000","last_fullsized":"2024-03-06T07:22:46.380621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:56.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097562+0000","last_change":"2024-03-06T07:22:45.383173+0000","last_active":"2024-03-06T07:22:48.097562+0000","last_peered":"2024-03-06T07:22:48.097562+0000","last_clean":"2024-03-06T07:22:48.097562+0000","last_became_active":"2024-03-06T07:22:45.382867+0000","last_became_peered":"2024-03-06T07:22:45.382867+0000","last_unstale":"2024-03-06T07:22:48.097562+0000","last_undegraded":"2024-03-06T07:22:48.097562+0000","last_fullsized":"2024-03-06T07:22:48.097562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:43.025307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078752+0000","last_change":"2024-03-06T07:22:35.039311+0000","last_active":"2024-03-06T07:22:48.078752+0000","last_peered":"2024-03-06T07:22:48.078752+0000","last_clean":"2024-03-06T07:22:48.078752+0000","last_became_active":"2024-03-06T07:22:35.038971+0000","last_became_peered":"2024-03-06T07:22:35.038971+0000","last_unstale":"2024-03-06T07:22:48.078752+0000","last_undegraded":"2024-03-06T07:22:48.078752+0000","last_fullsized":"2024-03-06T07:22:48.078752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:56:42.465427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380588+0000","last_change":"2024-03-06T07:22:45.374778+0000","last_active":"2024-03-06T07:22:46.380588+0000","last_peered":"2024-03-06T07:22:46.380588+0000","last_clean":"2024-03-06T07:22:46.380588+0000","last_became_active":"2024-03-06T07:22:45.374539+0000","last_became_peered":"2024-03-06T07:22:45.374539+0000","last_unstale":"2024-03-06T07:22:46.380588+0000","last_undegraded":"2024-03-06T07:22:46.380588+0000","last_fullsized":"2024-03-06T07:22:46.380588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:16.677632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380253+0000","last_change":"2024-03-06T07:22:35.031978+0000","last_active":"2024-03-06T07:22:46.380253+0000","last_peered":"2024-03-06T07:22:46.380253+0000","last_clean":"2024-03-06T07:22:46.380253+0000","last_became_active":"2024-03-06T07:22:35.031711+0000","last_became_peered":"2024-03-06T07:22:35.031711+0000","last_unstale":"2024-03-06T07:22:46.380253+0000","last_undegraded":"2024-03-06T07:22:46.380253+0000","last_fullsized":"2024-03-06T07:22:46.380253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:17.004593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380882+0000","last_change":"2024-03-06T07:22:45.376851+0000","last_active":"2024-03-06T07:22:46.380882+0000","last_peered":"2024-03-06T07:22:46.380882+0000","last_clean":"2024-03-06T07:22:46.380882+0000","last_became_active":"2024-03-06T07:22:45.376305+0000","last_became_peered":"2024-03-06T07:22:45.376305+0000","last_unstale":"2024-03-06T07:22:46.380882+0000","last_undegraded":"2024-03-06T07:22:46.380882+0000","last_fullsized":"2024-03-06T07:22:46.380882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:23.953464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097566+0000","last_change":"2024-03-06T07:22:35.045711+0000","last_active":"2024-03-06T07:22:48.097566+0000","last_peered":"2024-03-06T07:22:48.097566+0000","last_clean":"2024-03-06T07:22:48.097566+0000","last_became_active":"2024-03-06T07:22:35.041242+0000","last_became_peered":"2024-03-06T07:22:35.041242+0000","last_unstale":"2024-03-06T07:22:48.097566+0000","last_undegraded":"2024-03-06T07:22:48.097566+0000","last_fullsized":"2024-03-06T07:22:48.097566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:16.266604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078877+0000","last_change":"2024-03-06T07:22:45.372659+0000","last_active":"2024-03-06T07:22:48.078877+0000","last_peered":"2024-03-06T07:22:48.078877+0000","last_clean":"2024-03-06T07:22:48.078877+0000","last_became_active":"2024-03-06T07:22:45.372355+0000","last_became_peered":"2024-03-06T07:22:45.372355+0000","last_unstale":"2024-03-06T07:22:48.078877+0000","last_undegraded":"2024-03-06T07:22:48.078877+0000","last_fullsized":"2024-03-06T07:22:48.078877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:53.307799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098576+0000","last_change":"2024-03-06T07:22:35.042301+0000","last_active":"2024-03-06T07:22:48.098576+0000","last_peered":"2024-03-06T07:22:48.098576+0000","last_clean":"2024-03-06T07:22:48.098576+0000","last_became_active":"2024-03-06T07:22:35.041172+0000","last_became_peered":"2024-03-06T07:22:35.041172+0000","last_unstale":"2024-03-06T07:22:48.098576+0000","last_undegraded":"2024-03-06T07:22:48.098576+0000","last_fullsized":"2024-03-06T07:22:48.098576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:58.974636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931609+0000","last_change":"2024-03-06T07:22:45.387429+0000","last_active":"2024-03-06T07:22:46.931609+0000","last_peered":"2024-03-06T07:22:46.931609+0000","last_clean":"2024-03-06T07:22:46.931609+0000","last_became_active":"2024-03-06T07:22:45.387155+0000","last_became_peered":"2024-03-06T07:22:45.387155+0000","last_unstale":"2024-03-06T07:22:46.931609+0000","last_undegraded":"2024-03-06T07:22:46.931609+0000","last_fullsized":"2024-03-06T07:22:46.931609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:54.524435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378662+0000","last_change":"2024-03-06T07:22:35.038117+0000","last_active":"2024-03-06T07:22:46.378662+0000","last_peered":"2024-03-06T07:22:46.378662+0000","last_clean":"2024-03-06T07:22:46.378662+0000","last_became_active":"2024-03-06T07:22:35.037812+0000","last_became_peered":"2024-03-06T07:22:35.037812+0000","last_unstale":"2024-03-06T07:22:46.378662+0000","last_undegraded":"2024-03-06T07:22:46.378662+0000","last_fullsized":"2024-03-06T07:22:46.378662+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:12.966441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377674+0000","last_change":"2024-03-06T07:22:45.371608+0000","last_active":"2024-03-06T07:22:46.377674+0000","last_peered":"2024-03-06T07:22:46.377674+0000","last_clean":"2024-03-06T07:22:46.377674+0000","last_became_active":"2024-03-06T07:22:45.371309+0000","last_became_peered":"2024-03-06T07:22:45.371309+0000","last_unstale":"2024-03-06T07:22:46.377674+0000","last_undegraded":"2024-03-06T07:22:46.377674+0000","last_fullsized":"2024-03-06T07:22:46.377674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:43.025307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378181+0000","last_change":"2024-03-06T07:22:35.033425+0000","last_active":"2024-03-06T07:22:46.378181+0000","last_peered":"2024-03-06T07:22:46.378181+0000","last_clean":"2024-03-06T07:22:46.378181+0000","last_became_active":"2024-03-06T07:22:35.032784+0000","last_became_peered":"2024-03-06T07:22:35.032784+0000","last_unstale":"2024-03-06T07:22:46.378181+0000","last_undegraded":"2024-03-06T07:22:46.378181+0000","last_fullsized":"2024-03-06T07:22:46.378181+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380650+0000","last_change":"2024-03-06T07:22:45.376444+0000","last_active":"2024-03-06T07:22:46.380650+0000","last_peered":"2024-03-06T07:22:46.380650+0000","last_clean":"2024-03-06T07:22:46.380650+0000","last_became_active":"2024-03-06T07:22:45.376182+0000","last_became_peered":"2024-03-06T07:22:45.376182+0000","last_unstale":"2024-03-06T07:22:46.380650+0000","last_undegraded":"2024-03-06T07:22:46.380650+0000","last_fullsized":"2024-03-06T07:22:46.380650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:15.211940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078825+0000","last_change":"2024-03-06T07:22:35.033473+0000","last_active":"2024-03-06T07:22:48.078825+0000","last_peered":"2024-03-06T07:22:48.078825+0000","last_clean":"2024-03-06T07:22:48.078825+0000","last_became_active":"2024-03-06T07:22:35.032407+0000","last_became_peered":"2024-03-06T07:22:35.032407+0000","last_unstale":"2024-03-06T07:22:48.078825+0000","last_undegraded":"2024-03-06T07:22:48.078825+0000","last_fullsized":"2024-03-06T07:22:48.078825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:39.792272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377916+0000","last_change":"2024-03-06T07:22:45.371697+0000","last_active":"2024-03-06T07:22:46.377916+0000","last_peered":"2024-03-06T07:22:46.377916+0000","last_clean":"2024-03-06T07:22:46.377916+0000","last_became_active":"2024-03-06T07:22:45.371440+0000","last_became_peered":"2024-03-06T07:22:45.371440+0000","last_unstale":"2024-03-06T07:22:46.377916+0000","last_undegraded":"2024-03-06T07:22:46.377916+0000","last_fullsized":"2024-03-06T07:22:46.377916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:14:27.420771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098253+0000","last_change":"2024-03-06T07:22:35.048127+0000","last_active":"2024-03-06T07:22:48.098253+0000","last_peered":"2024-03-06T07:22:48.098253+0000","last_clean":"2024-03-06T07:22:48.098253+0000","last_became_active":"2024-03-06T07:22:35.047869+0000","last_became_peered":"2024-03-06T07:22:35.047869+0000","last_unstale":"2024-03-06T07:22:48.098253+0000","last_undegraded":"2024-03-06T07:22:48.098253+0000","last_fullsized":"2024-03-06T07:22:48.098253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097417+0000","last_change":"2024-03-06T07:22:45.383731+0000","last_active":"2024-03-06T07:22:48.097417+0000","last_peered":"2024-03-06T07:22:48.097417+0000","last_clean":"2024-03-06T07:22:48.097417+0000","last_became_active":"2024-03-06T07:22:45.382907+0000","last_became_peered":"2024-03-06T07:22:45.382907+0000","last_unstale":"2024-03-06T07:22:48.097417+0000","last_undegraded":"2024-03-06T07:22:48.097417+0000","last_fullsized":"2024-03-06T07:22:48.097417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:59.512045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378193+0000","last_change":"2024-03-06T07:22:35.038257+0000","last_active":"2024-03-06T07:22:46.378193+0000","last_peered":"2024-03-06T07:22:46.378193+0000","last_clean":"2024-03-06T07:22:46.378193+0000","last_became_active":"2024-03-06T07:22:35.037969+0000","last_became_peered":"2024-03-06T07:22:35.037969+0000","last_unstale":"2024-03-06T07:22:46.378193+0000","last_undegraded":"2024-03-06T07:22:46.378193+0000","last_fullsized":"2024-03-06T07:22:46.378193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378944+0000","last_change":"2024-03-06T07:22:45.373601+0000","last_active":"2024-03-06T07:22:46.378944+0000","last_peered":"2024-03-06T07:22:46.378944+0000","last_clean":"2024-03-06T07:22:46.378944+0000","last_became_active":"2024-03-06T07:22:45.373221+0000","last_became_peered":"2024-03-06T07:22:45.373221+0000","last_unstale":"2024-03-06T07:22:46.378944+0000","last_undegraded":"2024-03-06T07:22:46.378944+0000","last_fullsized":"2024-03-06T07:22:46.378944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:40.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931272+0000","last_change":"2024-03-06T07:22:35.032129+0000","last_active":"2024-03-06T07:22:46.931272+0000","last_peered":"2024-03-06T07:22:46.931272+0000","last_clean":"2024-03-06T07:22:46.931272+0000","last_became_active":"2024-03-06T07:22:35.027787+0000","last_became_peered":"2024-03-06T07:22:35.027787+0000","last_unstale":"2024-03-06T07:22:46.931272+0000","last_undegraded":"2024-03-06T07:22:46.931272+0000","last_fullsized":"2024-03-06T07:22:46.931272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:27.011065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380380+0000","last_change":"2024-03-06T07:22:45.381145+0000","last_active":"2024-03-06T07:22:46.380380+0000","last_peered":"2024-03-06T07:22:46.380380+0000","last_clean":"2024-03-06T07:22:46.380380+0000","last_became_active":"2024-03-06T07:22:45.380876+0000","last_became_peered":"2024-03-06T07:22:45.380876+0000","last_unstale":"2024-03-06T07:22:46.380380+0000","last_undegraded":"2024-03-06T07:22:46.380380+0000","last_fullsized":"2024-03-06T07:22:46.380380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:30.329463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377951+0000","last_change":"2024-03-06T07:22:35.036989+0000","last_active":"2024-03-06T07:22:46.377951+0000","last_peered":"2024-03-06T07:22:46.377951+0000","last_clean":"2024-03-06T07:22:46.377951+0000","last_became_active":"2024-03-06T07:22:35.032803+0000","last_became_peered":"2024-03-06T07:22:35.032803+0000","last_unstale":"2024-03-06T07:22:46.377951+0000","last_undegraded":"2024-03-06T07:22:46.377951+0000","last_fullsized":"2024-03-06T07:22:46.377951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:57:34.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097935+0000","last_change":"2024-03-06T07:22:45.379702+0000","last_active":"2024-03-06T07:22:48.097935+0000","last_peered":"2024-03-06T07:22:48.097935+0000","last_clean":"2024-03-06T07:22:48.097935+0000","last_became_active":"2024-03-06T07:22:45.379093+0000","last_became_peered":"2024-03-06T07:22:45.379093+0000","last_unstale":"2024-03-06T07:22:48.097935+0000","last_undegraded":"2024-03-06T07:22:48.097935+0000","last_fullsized":"2024-03-06T07:22:48.097935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:21:34.847722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098044+0000","last_change":"2024-03-06T07:22:44.372693+0000","last_active":"2024-03-06T07:22:48.098044+0000","last_peered":"2024-03-06T07:22:48.098044+0000","last_clean":"2024-03-06T07:22:48.098044+0000","last_became_active":"2024-03-06T07:22:44.372375+0000","last_became_peered":"2024-03-06T07:22:44.372375+0000","last_unstale":"2024-03-06T07:22:48.098044+0000","last_undegraded":"2024-03-06T07:22:48.098044+0000","last_fullsized":"2024-03-06T07:22:48.098044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:59.118936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931161+0000","last_change":"2024-03-06T07:22:35.025981+0000","last_active":"2024-03-06T07:22:46.931161+0000","last_peered":"2024-03-06T07:22:46.931161+0000","last_clean":"2024-03-06T07:22:46.931161+0000","last_became_active":"2024-03-06T07:22:35.025721+0000","last_became_peered":"2024-03-06T07:22:35.025721+0000","last_unstale":"2024-03-06T07:22:46.931161+0000","last_undegraded":"2024-03-06T07:22:46.931161+0000","last_fullsized":"2024-03-06T07:22:46.931161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:18.894659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.079121+0000","last_change":"2024-03-06T07:22:34.043747+0000","last_active":"2024-03-06T07:22:48.079121+0000","last_peered":"2024-03-06T07:22:48.079121+0000","last_clean":"2024-03-06T07:22:48.079121+0000","last_became_active":"2024-03-06T07:22:34.042367+0000","last_became_peered":"2024-03-06T07:22:34.042367+0000","last_unstale":"2024-03-06T07:22:48.079121+0000","last_undegraded":"2024-03-06T07:22:48.079121+0000","last_fullsized":"2024-03-06T07:22:48.079121+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:55:33.138544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931185+0000","last_change":"2024-03-06T07:22:45.372055+0000","last_active":"2024-03-06T07:22:46.931185+0000","last_peered":"2024-03-06T07:22:46.931185+0000","last_clean":"2024-03-06T07:22:46.931185+0000","last_became_active":"2024-03-06T07:22:45.371737+0000","last_became_peered":"2024-03-06T07:22:45.371737+0000","last_unstale":"2024-03-06T07:22:46.931185+0000","last_undegraded":"2024-03-06T07:22:46.931185+0000","last_fullsized":"2024-03-06T07:22:46.931185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:23.953464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377753+0000","last_change":"2024-03-06T07:22:44.380429+0000","last_active":"2024-03-06T07:22:46.377753+0000","last_peered":"2024-03-06T07:22:46.377753+0000","last_clean":"2024-03-06T07:22:46.377753+0000","last_became_active":"2024-03-06T07:22:44.380170+0000","last_became_peered":"2024-03-06T07:22:44.380170+0000","last_unstale":"2024-03-06T07:22:46.377753+0000","last_undegraded":"2024-03-06T07:22:46.377753+0000","last_fullsized":"2024-03-06T07:22:46.377753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:31:38.289550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097609+0000","last_change":"2024-03-06T07:22:35.032593+0000","last_active":"2024-03-06T07:22:48.097609+0000","last_peered":"2024-03-06T07:22:48.097609+0000","last_clean":"2024-03-06T07:22:48.097609+0000","last_became_active":"2024-03-06T07:22:35.031651+0000","last_became_peered":"2024-03-06T07:22:35.031651+0000","last_unstale":"2024-03-06T07:22:48.097609+0000","last_undegraded":"2024-03-06T07:22:48.097609+0000","last_fullsized":"2024-03-06T07:22:48.097609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378659+0000","last_change":"2024-03-06T07:22:34.038702+0000","last_active":"2024-03-06T07:22:46.378659+0000","last_peered":"2024-03-06T07:22:46.378659+0000","last_clean":"2024-03-06T07:22:46.378659+0000","last_became_active":"2024-03-06T07:22:34.038165+0000","last_became_peered":"2024-03-06T07:22:34.038165+0000","last_unstale":"2024-03-06T07:22:46.378659+0000","last_undegraded":"2024-03-06T07:22:46.378659+0000","last_fullsized":"2024-03-06T07:22:46.378659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931023+0000","last_change":"2024-03-06T07:22:45.379280+0000","last_active":"2024-03-06T07:22:46.931023+0000","last_peered":"2024-03-06T07:22:46.931023+0000","last_clean":"2024-03-06T07:22:46.931023+0000","last_became_active":"2024-03-06T07:22:45.378982+0000","last_became_peered":"2024-03-06T07:22:45.378982+0000","last_unstale":"2024-03-06T07:22:46.931023+0000","last_undegraded":"2024-03-06T07:22:46.931023+0000","last_fullsized":"2024-03-06T07:22:46.931023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:16.677632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380446+0000","last_change":"2024-03-06T07:22:44.375256+0000","last_active":"2024-03-06T07:22:46.380446+0000","last_peered":"2024-03-06T07:22:46.380446+0000","last_clean":"2024-03-06T07:22:46.380446+0000","last_became_active":"2024-03-06T07:22:44.374987+0000","last_became_peered":"2024-03-06T07:22:44.374987+0000","last_unstale":"2024-03-06T07:22:46.380446+0000","last_undegraded":"2024-03-06T07:22:46.380446+0000","last_fullsized":"2024-03-06T07:22:46.380446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:35.266045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380863+0000","last_change":"2024-03-06T07:22:35.035774+0000","last_active":"2024-03-06T07:22:46.380863+0000","last_peered":"2024-03-06T07:22:46.380863+0000","last_clean":"2024-03-06T07:22:46.380863+0000","last_became_active":"2024-03-06T07:22:35.035468+0000","last_became_peered":"2024-03-06T07:22:35.035468+0000","last_unstale":"2024-03-06T07:22:46.380863+0000","last_undegraded":"2024-03-06T07:22:46.380863+0000","last_fullsized":"2024-03-06T07:22:46.380863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:25.144870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378272+0000","last_change":"2024-03-06T07:22:34.042434+0000","last_active":"2024-03-06T07:22:46.378272+0000","last_peered":"2024-03-06T07:22:46.378272+0000","last_clean":"2024-03-06T07:22:46.378272+0000","last_became_active":"2024-03-06T07:22:34.042093+0000","last_became_peered":"2024-03-06T07:22:34.042093+0000","last_unstale":"2024-03-06T07:22:46.378272+0000","last_undegraded":"2024-03-06T07:22:46.378272+0000","last_fullsized":"2024-03-06T07:22:46.378272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958278+0000","last_change":"2024-03-06T07:22:45.378684+0000","last_active":"2024-03-06T07:22:46.958278+0000","last_peered":"2024-03-06T07:22:46.958278+0000","last_clean":"2024-03-06T07:22:46.958278+0000","last_became_active":"2024-03-06T07:22:45.377915+0000","last_became_peered":"2024-03-06T07:22:45.377915+0000","last_unstale":"2024-03-06T07:22:46.958278+0000","last_undegraded":"2024-03-06T07:22:46.958278+0000","last_fullsized":"2024-03-06T07:22:46.958278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:15.211940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931369+0000","last_change":"2024-03-06T07:22:44.373849+0000","last_active":"2024-03-06T07:22:46.931369+0000","last_peered":"2024-03-06T07:22:46.931369+0000","last_clean":"2024-03-06T07:22:46.931369+0000","last_became_active":"2024-03-06T07:22:44.373135+0000","last_became_peered":"2024-03-06T07:22:44.373135+0000","last_unstale":"2024-03-06T07:22:46.931369+0000","last_undegraded":"2024-03-06T07:22:46.931369+0000","last_fullsized":"2024-03-06T07:22:46.931369+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:27.741841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380670+0000","last_change":"2024-03-06T07:22:35.032012+0000","last_active":"2024-03-06T07:22:46.380670+0000","last_peered":"2024-03-06T07:22:46.380670+0000","last_clean":"2024-03-06T07:22:46.380670+0000","last_became_active":"2024-03-06T07:22:35.031728+0000","last_became_peered":"2024-03-06T07:22:35.031728+0000","last_unstale":"2024-03-06T07:22:46.380670+0000","last_undegraded":"2024-03-06T07:22:46.380670+0000","last_fullsized":"2024-03-06T07:22:46.380670+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:05.962020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958462+0000","last_change":"2024-03-06T07:22:34.038862+0000","last_active":"2024-03-06T07:22:46.958462+0000","last_peered":"2024-03-06T07:22:46.958462+0000","last_clean":"2024-03-06T07:22:46.958462+0000","last_became_active":"2024-03-06T07:22:34.038590+0000","last_became_peered":"2024-03-06T07:22:34.038590+0000","last_unstale":"2024-03-06T07:22:46.958462+0000","last_undegraded":"2024-03-06T07:22:46.958462+0000","last_fullsized":"2024-03-06T07:22:46.958462+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:15.997587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378433+0000","last_change":"2024-03-06T07:22:45.370890+0000","last_active":"2024-03-06T07:22:46.378433+0000","last_peered":"2024-03-06T07:22:46.378433+0000","last_clean":"2024-03-06T07:22:46.378433+0000","last_became_active":"2024-03-06T07:22:45.370624+0000","last_became_peered":"2024-03-06T07:22:45.370624+0000","last_unstale":"2024-03-06T07:22:46.378433+0000","last_undegraded":"2024-03-06T07:22:46.378433+0000","last_fullsized":"2024-03-06T07:22:46.378433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:13.385421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378087+0000","last_change":"2024-03-06T07:22:44.375296+0000","last_active":"2024-03-06T07:22:46.378087+0000","last_peered":"2024-03-06T07:22:46.378087+0000","last_clean":"2024-03-06T07:22:46.378087+0000","last_became_active":"2024-03-06T07:22:44.375028+0000","last_became_peered":"2024-03-06T07:22:44.375028+0000","last_unstale":"2024-03-06T07:22:46.378087+0000","last_undegraded":"2024-03-06T07:22:46.378087+0000","last_fullsized":"2024-03-06T07:22:46.378087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:42.020403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098850+0000","last_change":"2024-03-06T07:22:35.036832+0000","last_active":"2024-03-06T07:22:48.098850+0000","last_peered":"2024-03-06T07:22:48.098850+0000","last_clean":"2024-03-06T07:22:48.098850+0000","last_became_active":"2024-03-06T07:22:35.036392+0000","last_became_peered":"2024-03-06T07:22:35.036392+0000","last_unstale":"2024-03-06T07:22:48.098850+0000","last_undegraded":"2024-03-06T07:22:48.098850+0000","last_fullsized":"2024-03-06T07:22:48.098850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:13.614365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380761+0000","last_change":"2024-03-06T07:22:34.037848+0000","last_active":"2024-03-06T07:22:46.380761+0000","last_peered":"2024-03-06T07:22:46.380761+0000","last_clean":"2024-03-06T07:22:46.380761+0000","last_became_active":"2024-03-06T07:22:34.037572+0000","last_became_peered":"2024-03-06T07:22:34.037572+0000","last_unstale":"2024-03-06T07:22:46.380761+0000","last_undegraded":"2024-03-06T07:22:46.380761+0000","last_fullsized":"2024-03-06T07:22:46.380761+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:15.748729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098520+0000","last_change":"2024-03-06T07:22:45.376883+0000","last_active":"2024-03-06T07:22:48.098520+0000","last_peered":"2024-03-06T07:22:48.098520+0000","last_clean":"2024-03-06T07:22:48.098520+0000","last_became_active":"2024-03-06T07:22:45.376567+0000","last_became_peered":"2024-03-06T07:22:45.376567+0000","last_unstale":"2024-03-06T07:22:48.098520+0000","last_undegraded":"2024-03-06T07:22:48.098520+0000","last_fullsized":"2024-03-06T07:22:48.098520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:46.573551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958240+0000","last_change":"2024-03-06T07:22:44.375707+0000","last_active":"2024-03-06T07:22:46.958240+0000","last_peered":"2024-03-06T07:22:46.958240+0000","last_clean":"2024-03-06T07:22:46.958240+0000","last_became_active":"2024-03-06T07:22:44.375510+0000","last_became_peered":"2024-03-06T07:22:44.375510+0000","last_unstale":"2024-03-06T07:22:46.958240+0000","last_undegraded":"2024-03-06T07:22:46.958240+0000","last_fullsized":"2024-03-06T07:22:46.958240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:47.408727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098837+0000","last_change":"2024-03-06T07:22:35.040723+0000","last_active":"2024-03-06T07:22:48.098837+0000","last_peered":"2024-03-06T07:22:48.098837+0000","last_clean":"2024-03-06T07:22:48.098837+0000","last_became_active":"2024-03-06T07:22:35.036544+0000","last_became_peered":"2024-03-06T07:22:35.036544+0000","last_unstale":"2024-03-06T07:22:48.098837+0000","last_undegraded":"2024-03-06T07:22:48.098837+0000","last_fullsized":"2024-03-06T07:22:48.098837+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:44.185335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378404+0000","last_change":"2024-03-06T07:22:34.040746+0000","last_active":"2024-03-06T07:22:46.378404+0000","last_peered":"2024-03-06T07:22:46.378404+0000","last_clean":"2024-03-06T07:22:46.378404+0000","last_became_active":"2024-03-06T07:22:34.040466+0000","last_became_peered":"2024-03-06T07:22:34.040466+0000","last_unstale":"2024-03-06T07:22:46.378404+0000","last_undegraded":"2024-03-06T07:22:46.378404+0000","last_fullsized":"2024-03-06T07:22:46.378404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:35.883846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098431+0000","last_change":"2024-03-06T07:22:45.379532+0000","last_active":"2024-03-06T07:22:48.098431+0000","last_peered":"2024-03-06T07:22:48.098431+0000","last_clean":"2024-03-06T07:22:48.098431+0000","last_became_active":"2024-03-06T07:22:45.378096+0000","last_became_peered":"2024-03-06T07:22:45.378096+0000","last_unstale":"2024-03-06T07:22:48.098431+0000","last_undegraded":"2024-03-06T07:22:48.098431+0000","last_fullsized":"2024-03-06T07:22:48.098431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:37.457137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"77'4","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958522+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:46.958522+0000","last_peered":"2024-03-06T07:22:46.958522+0000","last_clean":"2024-03-06T07:22:46.958522+0000","last_became_active":"2024-03-06T07:22:44.380134+0000","last_became_peered":"2024-03-06T07:22:44.380134+0000","last_unstale":"2024-03-06T07:22:46.958522+0000","last_undegraded":"2024-03-06T07:22:46.958522+0000","last_fullsized":"2024-03-06T07:22:46.958522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:13.527528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380952+0000","last_change":"2024-03-06T07:22:35.035207+0000","last_active":"2024-03-06T07:22:46.380952+0000","last_peered":"2024-03-06T07:22:46.380952+0000","last_clean":"2024-03-06T07:22:46.380952+0000","last_became_active":"2024-03-06T07:22:35.034887+0000","last_became_peered":"2024-03-06T07:22:35.034887+0000","last_unstale":"2024-03-06T07:22:46.380952+0000","last_undegraded":"2024-03-06T07:22:46.380952+0000","last_fullsized":"2024-03-06T07:22:46.380952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:05.142259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097702+0000","last_change":"2024-03-06T07:22:34.042833+0000","last_active":"2024-03-06T07:22:48.097702+0000","last_peered":"2024-03-06T07:22:48.097702+0000","last_clean":"2024-03-06T07:22:48.097702+0000","last_became_active":"2024-03-06T07:22:34.042557+0000","last_became_peered":"2024-03-06T07:22:34.042557+0000","last_unstale":"2024-03-06T07:22:48.097702+0000","last_undegraded":"2024-03-06T07:22:48.097702+0000","last_fullsized":"2024-03-06T07:22:48.097702+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:18.627378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098167+0000","last_change":"2024-03-06T07:22:45.386371+0000","last_active":"2024-03-06T07:22:48.098167+0000","last_peered":"2024-03-06T07:22:48.098167+0000","last_clean":"2024-03-06T07:22:48.098167+0000","last_became_active":"2024-03-06T07:22:45.386098+0000","last_became_peered":"2024-03-06T07:22:45.386098+0000","last_unstale":"2024-03-06T07:22:48.098167+0000","last_undegraded":"2024-03-06T07:22:48.098167+0000","last_fullsized":"2024-03-06T07:22:48.098167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:22.976791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380610+0000","last_change":"2024-03-06T07:22:44.371322+0000","last_active":"2024-03-06T07:22:46.380610+0000","last_peered":"2024-03-06T07:22:46.380610+0000","last_clean":"2024-03-06T07:22:46.380610+0000","last_became_active":"2024-03-06T07:22:44.371058+0000","last_became_peered":"2024-03-06T07:22:44.371058+0000","last_unstale":"2024-03-06T07:22:46.380610+0000","last_undegraded":"2024-03-06T07:22:46.380610+0000","last_fullsized":"2024-03-06T07:22:46.380610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:46.225047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958724+0000","last_change":"2024-03-06T07:22:35.028293+0000","last_active":"2024-03-06T07:22:46.958724+0000","last_peered":"2024-03-06T07:22:46.958724+0000","last_clean":"2024-03-06T07:22:46.958724+0000","last_became_active":"2024-03-06T07:22:35.028017+0000","last_became_peered":"2024-03-06T07:22:35.028017+0000","last_unstale":"2024-03-06T07:22:46.958724+0000","last_undegraded":"2024-03-06T07:22:46.958724+0000","last_fullsized":"2024-03-06T07:22:46.958724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:36.890851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078533+0000","last_change":"2024-03-06T07:22:34.037660+0000","last_active":"2024-03-06T07:22:48.078533+0000","last_peered":"2024-03-06T07:22:48.078533+0000","last_clean":"2024-03-06T07:22:48.078533+0000","last_became_active":"2024-03-06T07:22:34.037347+0000","last_became_peered":"2024-03-06T07:22:34.037347+0000","last_unstale":"2024-03-06T07:22:48.078533+0000","last_undegraded":"2024-03-06T07:22:48.078533+0000","last_fullsized":"2024-03-06T07:22:48.078533+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:43.483410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078466+0000","last_change":"2024-03-06T07:22:45.373926+0000","last_active":"2024-03-06T07:22:48.078466+0000","last_peered":"2024-03-06T07:22:48.078466+0000","last_clean":"2024-03-06T07:22:48.078466+0000","last_became_active":"2024-03-06T07:22:45.373622+0000","last_became_peered":"2024-03-06T07:22:45.373622+0000","last_unstale":"2024-03-06T07:22:48.078466+0000","last_undegraded":"2024-03-06T07:22:48.078466+0000","last_fullsized":"2024-03-06T07:22:48.078466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:21.025777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931108+0000","last_change":"2024-03-06T07:22:44.370568+0000","last_active":"2024-03-06T07:22:46.931108+0000","last_peered":"2024-03-06T07:22:46.931108+0000","last_clean":"2024-03-06T07:22:46.931108+0000","last_became_active":"2024-03-06T07:22:44.370050+0000","last_became_peered":"2024-03-06T07:22:44.370050+0000","last_unstale":"2024-03-06T07:22:46.931108+0000","last_undegraded":"2024-03-06T07:22:46.931108+0000","last_fullsized":"2024-03-06T07:22:46.931108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:53:05.282920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097664+0000","last_change":"2024-03-06T07:22:35.045241+0000","last_active":"2024-03-06T07:22:48.097664+0000","last_peered":"2024-03-06T07:22:48.097664+0000","last_clean":"2024-03-06T07:22:48.097664+0000","last_became_active":"2024-03-06T07:22:35.044953+0000","last_became_peered":"2024-03-06T07:22:35.044953+0000","last_unstale":"2024-03-06T07:22:48.097664+0000","last_undegraded":"2024-03-06T07:22:48.097664+0000","last_fullsized":"2024-03-06T07:22:48.097664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:19.990363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379556+0000","last_change":"2024-03-06T07:22:34.038501+0000","last_active":"2024-03-06T07:22:46.379556+0000","last_peered":"2024-03-06T07:22:46.379556+0000","last_clean":"2024-03-06T07:22:46.379556+0000","last_became_active":"2024-03-06T07:22:34.038238+0000","last_became_peered":"2024-03-06T07:22:34.038238+0000","last_unstale":"2024-03-06T07:22:46.379556+0000","last_undegraded":"2024-03-06T07:22:46.379556+0000","last_fullsized":"2024-03-06T07:22:46.379556+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078634+0000","last_change":"2024-03-06T07:22:45.378628+0000","last_active":"2024-03-06T07:22:48.078634+0000","last_peered":"2024-03-06T07:22:48.078634+0000","last_clean":"2024-03-06T07:22:48.078634+0000","last_became_active":"2024-03-06T07:22:45.378308+0000","last_became_peered":"2024-03-06T07:22:45.378308+0000","last_unstale":"2024-03-06T07:22:48.078634+0000","last_undegraded":"2024-03-06T07:22:48.078634+0000","last_fullsized":"2024-03-06T07:22:48.078634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:19:00.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"77'8","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931129+0000","last_change":"2024-03-06T07:22:44.377341+0000","last_active":"2024-03-06T07:22:46.931129+0000","last_peered":"2024-03-06T07:22:46.931129+0000","last_clean":"2024-03-06T07:22:46.931129+0000","last_became_active":"2024-03-06T07:22:44.377070+0000","last_became_peered":"2024-03-06T07:22:44.377070+0000","last_unstale":"2024-03-06T07:22:46.931129+0000","last_undegraded":"2024-03-06T07:22:46.931129+0000","last_fullsized":"2024-03-06T07:22:46.931129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:13.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378116+0000","last_change":"2024-03-06T07:22:35.035913+0000","last_active":"2024-03-06T07:22:46.378116+0000","last_peered":"2024-03-06T07:22:46.378116+0000","last_clean":"2024-03-06T07:22:46.378116+0000","last_became_active":"2024-03-06T07:22:35.035658+0000","last_became_peered":"2024-03-06T07:22:35.035658+0000","last_unstale":"2024-03-06T07:22:46.378116+0000","last_undegraded":"2024-03-06T07:22:46.378116+0000","last_fullsized":"2024-03-06T07:22:46.378116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:55.082141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378703+0000","last_change":"2024-03-06T07:22:34.040689+0000","last_active":"2024-03-06T07:22:46.378703+0000","last_peered":"2024-03-06T07:22:46.378703+0000","last_clean":"2024-03-06T07:22:46.378703+0000","last_became_active":"2024-03-06T07:22:34.040410+0000","last_became_peered":"2024-03-06T07:22:34.040410+0000","last_unstale":"2024-03-06T07:22:46.378703+0000","last_undegraded":"2024-03-06T07:22:46.378703+0000","last_fullsized":"2024-03-06T07:22:46.378703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.281130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931063+0000","last_change":"2024-03-06T07:22:45.371270+0000","last_active":"2024-03-06T07:22:46.931063+0000","last_peered":"2024-03-06T07:22:46.931063+0000","last_clean":"2024-03-06T07:22:46.931063+0000","last_became_active":"2024-03-06T07:22:45.370584+0000","last_became_peered":"2024-03-06T07:22:45.370584+0000","last_unstale":"2024-03-06T07:22:46.931063+0000","last_undegraded":"2024-03-06T07:22:46.931063+0000","last_fullsized":"2024-03-06T07:22:46.931063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:26.605704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379033+0000","last_change":"2024-03-06T07:22:44.380354+0000","last_active":"2024-03-06T07:22:46.379033+0000","last_peered":"2024-03-06T07:22:46.379033+0000","last_clean":"2024-03-06T07:22:46.379033+0000","last_became_active":"2024-03-06T07:22:44.380123+0000","last_became_peered":"2024-03-06T07:22:44.380123+0000","last_unstale":"2024-03-06T07:22:46.379033+0000","last_undegraded":"2024-03-06T07:22:46.379033+0000","last_fullsized":"2024-03-06T07:22:46.379033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:56.398201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098982+0000","last_change":"2024-03-06T07:22:35.032615+0000","last_active":"2024-03-06T07:22:48.098982+0000","last_peered":"2024-03-06T07:22:48.098982+0000","last_clean":"2024-03-06T07:22:48.098982+0000","last_became_active":"2024-03-06T07:22:35.031167+0000","last_became_peered":"2024-03-06T07:22:35.031167+0000","last_unstale":"2024-03-06T07:22:48.098982+0000","last_undegraded":"2024-03-06T07:22:48.098982+0000","last_fullsized":"2024-03-06T07:22:48.098982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:06.338532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078914+0000","last_change":"2024-03-06T07:22:34.038406+0000","last_active":"2024-03-06T07:22:48.078914+0000","last_peered":"2024-03-06T07:22:48.078914+0000","last_clean":"2024-03-06T07:22:48.078914+0000","last_became_active":"2024-03-06T07:22:34.038040+0000","last_became_peered":"2024-03-06T07:22:34.038040+0000","last_unstale":"2024-03-06T07:22:48.078914+0000","last_undegraded":"2024-03-06T07:22:48.078914+0000","last_fullsized":"2024-03-06T07:22:48.078914+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:26.063950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380228+0000","last_change":"2024-03-06T07:22:45.374209+0000","last_active":"2024-03-06T07:22:46.380228+0000","last_peered":"2024-03-06T07:22:46.380228+0000","last_clean":"2024-03-06T07:22:46.380228+0000","last_became_active":"2024-03-06T07:22:45.373873+0000","last_became_peered":"2024-03-06T07:22:45.373873+0000","last_unstale":"2024-03-06T07:22:46.380228+0000","last_undegraded":"2024-03-06T07:22:46.380228+0000","last_fullsized":"2024-03-06T07:22:46.380228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:46.388027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097469+0000","last_change":"2024-03-06T07:22:44.377976+0000","last_active":"2024-03-06T07:22:48.097469+0000","last_peered":"2024-03-06T07:22:48.097469+0000","last_clean":"2024-03-06T07:22:48.097469+0000","last_became_active":"2024-03-06T07:22:44.377659+0000","last_became_peered":"2024-03-06T07:22:44.377659+0000","last_unstale":"2024-03-06T07:22:48.097469+0000","last_undegraded":"2024-03-06T07:22:48.097469+0000","last_fullsized":"2024-03-06T07:22:48.097469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:50.028073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931549+0000","last_change":"2024-03-06T07:22:35.038459+0000","last_active":"2024-03-06T07:22:46.931549+0000","last_peered":"2024-03-06T07:22:46.931549+0000","last_clean":"2024-03-06T07:22:46.931549+0000","last_became_active":"2024-03-06T07:22:35.038185+0000","last_became_peered":"2024-03-06T07:22:35.038185+0000","last_unstale":"2024-03-06T07:22:46.931549+0000","last_undegraded":"2024-03-06T07:22:46.931549+0000","last_fullsized":"2024-03-06T07:22:46.931549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:09:42.753706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958538+0000","last_change":"2024-03-06T07:22:34.037962+0000","last_active":"2024-03-06T07:22:46.958538+0000","last_peered":"2024-03-06T07:22:46.958538+0000","last_clean":"2024-03-06T07:22:46.958538+0000","last_became_active":"2024-03-06T07:22:34.037635+0000","last_became_peered":"2024-03-06T07:22:34.037635+0000","last_unstale":"2024-03-06T07:22:46.958538+0000","last_undegraded":"2024-03-06T07:22:46.958538+0000","last_fullsized":"2024-03-06T07:22:46.958538+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099292+0000","last_change":"2024-03-06T07:22:45.392270+0000","last_active":"2024-03-06T07:22:48.099292+0000","last_peered":"2024-03-06T07:22:48.099292+0000","last_clean":"2024-03-06T07:22:48.099292+0000","last_became_active":"2024-03-06T07:22:45.391985+0000","last_became_peered":"2024-03-06T07:22:45.391985+0000","last_unstale":"2024-03-06T07:22:48.099292+0000","last_undegraded":"2024-03-06T07:22:48.099292+0000","last_fullsized":"2024-03-06T07:22:48.099292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:51.032977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"77'3","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381039+0000","last_change":"2024-03-06T07:22:44.377361+0000","last_active":"2024-03-06T07:22:46.381039+0000","last_peered":"2024-03-06T07:22:46.381039+0000","last_clean":"2024-03-06T07:22:46.381039+0000","last_became_active":"2024-03-06T07:22:44.377101+0000","last_became_peered":"2024-03-06T07:22:44.377101+0000","last_unstale":"2024-03-06T07:22:46.381039+0000","last_undegraded":"2024-03-06T07:22:46.381039+0000","last_fullsized":"2024-03-06T07:22:46.381039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:30.597343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379125+0000","last_change":"2024-03-06T07:22:35.036674+0000","last_active":"2024-03-06T07:22:46.379125+0000","last_peered":"2024-03-06T07:22:46.379125+0000","last_clean":"2024-03-06T07:22:46.379125+0000","last_became_active":"2024-03-06T07:22:35.036400+0000","last_became_peered":"2024-03-06T07:22:35.036400+0000","last_unstale":"2024-03-06T07:22:46.379125+0000","last_undegraded":"2024-03-06T07:22:46.379125+0000","last_fullsized":"2024-03-06T07:22:46.379125+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:26.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377655+0000","last_change":"2024-03-06T07:22:34.041311+0000","last_active":"2024-03-06T07:22:46.377655+0000","last_peered":"2024-03-06T07:22:46.377655+0000","last_clean":"2024-03-06T07:22:46.377655+0000","last_became_active":"2024-03-06T07:22:34.040940+0000","last_became_peered":"2024-03-06T07:22:34.040940+0000","last_unstale":"2024-03-06T07:22:46.377655+0000","last_undegraded":"2024-03-06T07:22:46.377655+0000","last_fullsized":"2024-03-06T07:22:46.377655+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:24.924844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931549+0000","last_change":"2024-03-06T07:22:45.367411+0000","last_active":"2024-03-06T07:22:46.931549+0000","last_peered":"2024-03-06T07:22:46.931549+0000","last_clean":"2024-03-06T07:22:46.931549+0000","last_became_active":"2024-03-06T07:22:45.367125+0000","last_became_peered":"2024-03-06T07:22:45.367125+0000","last_unstale":"2024-03-06T07:22:46.931549+0000","last_undegraded":"2024-03-06T07:22:46.931549+0000","last_fullsized":"2024-03-06T07:22:46.931549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:16:00.123840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"77'6","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098694+0000","last_change":"2024-03-06T07:22:44.378302+0000","last_active":"2024-03-06T07:22:48.098694+0000","last_peered":"2024-03-06T07:22:48.098694+0000","last_clean":"2024-03-06T07:22:48.098694+0000","last_became_active":"2024-03-06T07:22:44.377994+0000","last_became_peered":"2024-03-06T07:22:44.377994+0000","last_unstale":"2024-03-06T07:22:48.098694+0000","last_undegraded":"2024-03-06T07:22:48.098694+0000","last_fullsized":"2024-03-06T07:22:48.098694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:30.223815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958262+0000","last_change":"2024-03-06T07:22:35.022953+0000","last_active":"2024-03-06T07:22:46.958262+0000","last_peered":"2024-03-06T07:22:46.958262+0000","last_clean":"2024-03-06T07:22:46.958262+0000","last_became_active":"2024-03-06T07:22:35.022681+0000","last_became_peered":"2024-03-06T07:22:35.022681+0000","last_unstale":"2024-03-06T07:22:46.958262+0000","last_undegraded":"2024-03-06T07:22:46.958262+0000","last_fullsized":"2024-03-06T07:22:46.958262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:04.583152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378276+0000","last_change":"2024-03-06T07:22:34.037840+0000","last_active":"2024-03-06T07:22:46.378276+0000","last_peered":"2024-03-06T07:22:46.378276+0000","last_clean":"2024-03-06T07:22:46.378276+0000","last_became_active":"2024-03-06T07:22:34.037556+0000","last_became_peered":"2024-03-06T07:22:34.037556+0000","last_unstale":"2024-03-06T07:22:46.378276+0000","last_undegraded":"2024-03-06T07:22:46.378276+0000","last_fullsized":"2024-03-06T07:22:46.378276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380882+0000","last_change":"2024-03-06T07:22:45.386319+0000","last_active":"2024-03-06T07:22:46.380882+0000","last_peered":"2024-03-06T07:22:46.380882+0000","last_clean":"2024-03-06T07:22:46.380882+0000","last_became_active":"2024-03-06T07:22:45.386062+0000","last_became_peered":"2024-03-06T07:22:45.386062+0000","last_unstale":"2024-03-06T07:22:46.380882+0000","last_undegraded":"2024-03-06T07:22:46.380882+0000","last_fullsized":"2024-03-06T07:22:46.380882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:10.476716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377766+0000","last_change":"2024-03-06T07:22:44.372722+0000","last_active":"2024-03-06T07:22:46.377766+0000","last_peered":"2024-03-06T07:22:46.377766+0000","last_clean":"2024-03-06T07:22:46.377766+0000","last_became_active":"2024-03-06T07:22:44.372451+0000","last_became_peered":"2024-03-06T07:22:44.372451+0000","last_unstale":"2024-03-06T07:22:46.377766+0000","last_undegraded":"2024-03-06T07:22:46.377766+0000","last_fullsized":"2024-03-06T07:22:46.377766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.786058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958366+0000","last_change":"2024-03-06T07:22:35.034503+0000","last_active":"2024-03-06T07:22:46.958366+0000","last_peered":"2024-03-06T07:22:46.958366+0000","last_clean":"2024-03-06T07:22:46.958366+0000","last_became_active":"2024-03-06T07:22:35.034230+0000","last_became_peered":"2024-03-06T07:22:35.034230+0000","last_unstale":"2024-03-06T07:22:46.958366+0000","last_undegraded":"2024-03-06T07:22:46.958366+0000","last_fullsized":"2024-03-06T07:22:46.958366+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:16.197460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078360+0000","last_change":"2024-03-06T07:22:34.039959+0000","last_active":"2024-03-06T07:22:48.078360+0000","last_peered":"2024-03-06T07:22:48.078360+0000","last_clean":"2024-03-06T07:22:48.078360+0000","last_became_active":"2024-03-06T07:22:34.039689+0000","last_became_peered":"2024-03-06T07:22:34.039689+0000","last_unstale":"2024-03-06T07:22:48.078360+0000","last_undegraded":"2024-03-06T07:22:48.078360+0000","last_fullsized":"2024-03-06T07:22:48.078360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:23.420842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378420+0000","last_change":"2024-03-06T07:22:45.378174+0000","last_active":"2024-03-06T07:22:46.378420+0000","last_peered":"2024-03-06T07:22:46.378420+0000","last_clean":"2024-03-06T07:22:46.378420+0000","last_became_active":"2024-03-06T07:22:45.377757+0000","last_became_peered":"2024-03-06T07:22:45.377757+0000","last_unstale":"2024-03-06T07:22:46.378420+0000","last_undegraded":"2024-03-06T07:22:46.378420+0000","last_fullsized":"2024-03-06T07:22:46.378420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:52.553060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099396+0000","last_change":"2024-03-06T07:22:44.371938+0000","last_active":"2024-03-06T07:22:48.099396+0000","last_peered":"2024-03-06T07:22:48.099396+0000","last_clean":"2024-03-06T07:22:48.099396+0000","last_became_active":"2024-03-06T07:22:44.371634+0000","last_became_peered":"2024-03-06T07:22:44.371634+0000","last_unstale":"2024-03-06T07:22:48.099396+0000","last_undegraded":"2024-03-06T07:22:48.099396+0000","last_fullsized":"2024-03-06T07:22:48.099396+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.786058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958729+0000","last_change":"2024-03-06T07:22:35.032290+0000","last_active":"2024-03-06T07:22:46.958729+0000","last_peered":"2024-03-06T07:22:46.958729+0000","last_clean":"2024-03-06T07:22:46.958729+0000","last_became_active":"2024-03-06T07:22:35.032048+0000","last_became_peered":"2024-03-06T07:22:35.032048+0000","last_unstale":"2024-03-06T07:22:46.958729+0000","last_undegraded":"2024-03-06T07:22:46.958729+0000","last_fullsized":"2024-03-06T07:22:46.958729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:25.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097766+0000","last_change":"2024-03-06T07:22:34.043149+0000","last_active":"2024-03-06T07:22:48.097766+0000","last_peered":"2024-03-06T07:22:48.097766+0000","last_clean":"2024-03-06T07:22:48.097766+0000","last_became_active":"2024-03-06T07:22:34.042888+0000","last_became_peered":"2024-03-06T07:22:34.042888+0000","last_unstale":"2024-03-06T07:22:48.097766+0000","last_undegraded":"2024-03-06T07:22:48.097766+0000","last_fullsized":"2024-03-06T07:22:48.097766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378357+0000","last_change":"2024-03-06T07:22:45.372356+0000","last_active":"2024-03-06T07:22:46.378357+0000","last_peered":"2024-03-06T07:22:46.378357+0000","last_clean":"2024-03-06T07:22:46.378357+0000","last_became_active":"2024-03-06T07:22:45.372081+0000","last_became_peered":"2024-03-06T07:22:45.372081+0000","last_unstale":"2024-03-06T07:22:46.378357+0000","last_undegraded":"2024-03-06T07:22:46.378357+0000","last_fullsized":"2024-03-06T07:22:46.378357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:51.032977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377775+0000","last_change":"2024-03-06T07:22:44.379879+0000","last_active":"2024-03-06T07:22:46.377775+0000","last_peered":"2024-03-06T07:22:46.377775+0000","last_clean":"2024-03-06T07:22:46.377775+0000","last_became_active":"2024-03-06T07:22:44.379611+0000","last_became_peered":"2024-03-06T07:22:44.379611+0000","last_unstale":"2024-03-06T07:22:46.377775+0000","last_undegraded":"2024-03-06T07:22:46.377775+0000","last_fullsized":"2024-03-06T07:22:46.377775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:09.471812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097826+0000","last_change":"2024-03-06T07:22:35.034984+0000","last_active":"2024-03-06T07:22:48.097826+0000","last_peered":"2024-03-06T07:22:48.097826+0000","last_clean":"2024-03-06T07:22:48.097826+0000","last_became_active":"2024-03-06T07:22:35.031303+0000","last_became_peered":"2024-03-06T07:22:35.031303+0000","last_unstale":"2024-03-06T07:22:48.097826+0000","last_undegraded":"2024-03-06T07:22:48.097826+0000","last_fullsized":"2024-03-06T07:22:48.097826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:48.860306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379600+0000","last_change":"2024-03-06T07:22:34.044320+0000","last_active":"2024-03-06T07:22:46.379600+0000","last_peered":"2024-03-06T07:22:46.379600+0000","last_clean":"2024-03-06T07:22:46.379600+0000","last_became_active":"2024-03-06T07:22:34.043832+0000","last_became_peered":"2024-03-06T07:22:34.043832+0000","last_unstale":"2024-03-06T07:22:46.379600+0000","last_undegraded":"2024-03-06T07:22:46.379600+0000","last_fullsized":"2024-03-06T07:22:46.379600+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.941719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098198+0000","last_change":"2024-03-06T07:22:45.380069+0000","last_active":"2024-03-06T07:22:48.098198+0000","last_peered":"2024-03-06T07:22:48.098198+0000","last_clean":"2024-03-06T07:22:48.098198+0000","last_became_active":"2024-03-06T07:22:45.379798+0000","last_became_peered":"2024-03-06T07:22:45.379798+0000","last_unstale":"2024-03-06T07:22:48.098198+0000","last_undegraded":"2024-03-06T07:22:48.098198+0000","last_fullsized":"2024-03-06T07:22:48.098198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:47.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078303+0000","last_change":"2024-03-06T07:22:44.373965+0000","last_active":"2024-03-06T07:22:48.078303+0000","last_peered":"2024-03-06T07:22:48.078303+0000","last_clean":"2024-03-06T07:22:48.078303+0000","last_became_active":"2024-03-06T07:22:44.373617+0000","last_became_peered":"2024-03-06T07:22:44.373617+0000","last_unstale":"2024-03-06T07:22:48.078303+0000","last_undegraded":"2024-03-06T07:22:48.078303+0000","last_fullsized":"2024-03-06T07:22:48.078303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:45.618922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078353+0000","last_change":"2024-03-06T07:22:35.034203+0000","last_active":"2024-03-06T07:22:48.078353+0000","last_peered":"2024-03-06T07:22:48.078353+0000","last_clean":"2024-03-06T07:22:48.078353+0000","last_became_active":"2024-03-06T07:22:35.033903+0000","last_became_peered":"2024-03-06T07:22:35.033903+0000","last_unstale":"2024-03-06T07:22:48.078353+0000","last_undegraded":"2024-03-06T07:22:48.078353+0000","last_fullsized":"2024-03-06T07:22:48.078353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:02:14.829352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381059+0000","last_change":"2024-03-06T07:22:34.036176+0000","last_active":"2024-03-06T07:22:46.381059+0000","last_peered":"2024-03-06T07:22:46.381059+0000","last_clean":"2024-03-06T07:22:46.381059+0000","last_became_active":"2024-03-06T07:22:34.035867+0000","last_became_peered":"2024-03-06T07:22:34.035867+0000","last_unstale":"2024-03-06T07:22:46.381059+0000","last_undegraded":"2024-03-06T07:22:46.381059+0000","last_fullsized":"2024-03-06T07:22:46.381059+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:28.336495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097475+0000","last_change":"2024-03-06T07:22:45.382398+0000","last_active":"2024-03-06T07:22:48.097475+0000","last_peered":"2024-03-06T07:22:48.097475+0000","last_clean":"2024-03-06T07:22:48.097475+0000","last_became_active":"2024-03-06T07:22:45.382118+0000","last_became_peered":"2024-03-06T07:22:45.382118+0000","last_unstale":"2024-03-06T07:22:48.097475+0000","last_undegraded":"2024-03-06T07:22:48.097475+0000","last_fullsized":"2024-03-06T07:22:48.097475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:51:24.674548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377526+0000","last_change":"2024-03-06T07:22:44.369401+0000","last_active":"2024-03-06T07:22:46.377526+0000","last_peered":"2024-03-06T07:22:46.377526+0000","last_clean":"2024-03-06T07:22:46.377526+0000","last_became_active":"2024-03-06T07:22:44.369149+0000","last_became_peered":"2024-03-06T07:22:44.369149+0000","last_unstale":"2024-03-06T07:22:46.377526+0000","last_undegraded":"2024-03-06T07:22:46.377526+0000","last_fullsized":"2024-03-06T07:22:46.377526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:25.600800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078567+0000","last_change":"2024-03-06T07:22:35.038497+0000","last_active":"2024-03-06T07:22:48.078567+0000","last_peered":"2024-03-06T07:22:48.078567+0000","last_clean":"2024-03-06T07:22:48.078567+0000","last_became_active":"2024-03-06T07:22:35.038189+0000","last_became_peered":"2024-03-06T07:22:35.038189+0000","last_unstale":"2024-03-06T07:22:48.078567+0000","last_undegraded":"2024-03-06T07:22:48.078567+0000","last_fullsized":"2024-03-06T07:22:48.078567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:34.698852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.079042+0000","last_change":"2024-03-06T07:22:34.038423+0000","last_active":"2024-03-06T07:22:48.079042+0000","last_peered":"2024-03-06T07:22:48.079042+0000","last_clean":"2024-03-06T07:22:48.079042+0000","last_became_active":"2024-03-06T07:22:34.038126+0000","last_became_peered":"2024-03-06T07:22:34.038126+0000","last_unstale":"2024-03-06T07:22:48.079042+0000","last_undegraded":"2024-03-06T07:22:48.079042+0000","last_fullsized":"2024-03-06T07:22:48.079042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:43.441029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377668+0000","last_change":"2024-03-06T07:22:45.378986+0000","last_active":"2024-03-06T07:22:46.377668+0000","last_peered":"2024-03-06T07:22:46.377668+0000","last_clean":"2024-03-06T07:22:46.377668+0000","last_became_active":"2024-03-06T07:22:45.378720+0000","last_became_peered":"2024-03-06T07:22:45.378720+0000","last_unstale":"2024-03-06T07:22:46.377668+0000","last_undegraded":"2024-03-06T07:22:46.377668+0000","last_fullsized":"2024-03-06T07:22:46.377668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:36.752131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078469+0000","last_change":"2024-03-06T07:22:44.373676+0000","last_active":"2024-03-06T07:22:48.078469+0000","last_peered":"2024-03-06T07:22:48.078469+0000","last_clean":"2024-03-06T07:22:48.078469+0000","last_became_active":"2024-03-06T07:22:44.373209+0000","last_became_peered":"2024-03-06T07:22:44.373209+0000","last_unstale":"2024-03-06T07:22:48.078469+0000","last_undegraded":"2024-03-06T07:22:48.078469+0000","last_fullsized":"2024-03-06T07:22:48.078469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:15.654077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099026+0000","last_change":"2024-03-06T07:22:35.042660+0000","last_active":"2024-03-06T07:22:48.099026+0000","last_peered":"2024-03-06T07:22:48.099026+0000","last_clean":"2024-03-06T07:22:48.099026+0000","last_became_active":"2024-03-06T07:22:35.042327+0000","last_became_peered":"2024-03-06T07:22:35.042327+0000","last_unstale":"2024-03-06T07:22:48.099026+0000","last_undegraded":"2024-03-06T07:22:48.099026+0000","last_fullsized":"2024-03-06T07:22:48.099026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:36.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931807+0000","last_change":"2024-03-06T07:22:34.035505+0000","last_active":"2024-03-06T07:22:46.931807+0000","last_peered":"2024-03-06T07:22:46.931807+0000","last_clean":"2024-03-06T07:22:46.931807+0000","last_became_active":"2024-03-06T07:22:34.035257+0000","last_became_peered":"2024-03-06T07:22:34.035257+0000","last_unstale":"2024-03-06T07:22:46.931807+0000","last_undegraded":"2024-03-06T07:22:46.931807+0000","last_fullsized":"2024-03-06T07:22:46.931807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:54.995338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381031+0000","last_change":"2024-03-06T07:22:45.383858+0000","last_active":"2024-03-06T07:22:46.381031+0000","last_peered":"2024-03-06T07:22:46.381031+0000","last_clean":"2024-03-06T07:22:46.381031+0000","last_became_active":"2024-03-06T07:22:45.383416+0000","last_became_peered":"2024-03-06T07:22:45.383416+0000","last_unstale":"2024-03-06T07:22:46.381031+0000","last_undegraded":"2024-03-06T07:22:46.381031+0000","last_fullsized":"2024-03-06T07:22:46.381031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:54.524435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381104+0000","last_change":"2024-03-06T07:22:44.367628+0000","last_active":"2024-03-06T07:22:46.381104+0000","last_peered":"2024-03-06T07:22:46.381104+0000","last_clean":"2024-03-06T07:22:46.381104+0000","last_became_active":"2024-03-06T07:22:44.367384+0000","last_became_peered":"2024-03-06T07:22:44.367384+0000","last_unstale":"2024-03-06T07:22:46.381104+0000","last_undegraded":"2024-03-06T07:22:46.381104+0000","last_fullsized":"2024-03-06T07:22:46.381104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:36:28.968579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098942+0000","last_change":"2024-03-06T07:22:35.032174+0000","last_active":"2024-03-06T07:22:48.098942+0000","last_peered":"2024-03-06T07:22:48.098942+0000","last_clean":"2024-03-06T07:22:48.098942+0000","last_became_active":"2024-03-06T07:22:35.031869+0000","last_became_peered":"2024-03-06T07:22:35.031869+0000","last_unstale":"2024-03-06T07:22:48.098942+0000","last_undegraded":"2024-03-06T07:22:48.098942+0000","last_fullsized":"2024-03-06T07:22:48.098942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:58.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099204+0000","last_change":"2024-03-06T07:22:34.037475+0000","last_active":"2024-03-06T07:22:48.099204+0000","last_peered":"2024-03-06T07:22:48.099204+0000","last_clean":"2024-03-06T07:22:48.099204+0000","last_became_active":"2024-03-06T07:22:34.037206+0000","last_became_peered":"2024-03-06T07:22:34.037206+0000","last_unstale":"2024-03-06T07:22:48.099204+0000","last_undegraded":"2024-03-06T07:22:48.099204+0000","last_fullsized":"2024-03-06T07:22:48.099204+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098587+0000","last_change":"2024-03-06T07:22:45.380008+0000","last_active":"2024-03-06T07:22:48.098587+0000","last_peered":"2024-03-06T07:22:48.098587+0000","last_clean":"2024-03-06T07:22:48.098587+0000","last_became_active":"2024-03-06T07:22:45.378392+0000","last_became_peered":"2024-03-06T07:22:45.378392+0000","last_unstale":"2024-03-06T07:22:48.098587+0000","last_undegraded":"2024-03-06T07:22:48.098587+0000","last_fullsized":"2024-03-06T07:22:48.098587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:55:29.857967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958623+0000","last_change":"2024-03-06T07:22:44.372573+0000","last_active":"2024-03-06T07:22:46.958623+0000","last_peered":"2024-03-06T07:22:46.958623+0000","last_clean":"2024-03-06T07:22:46.958623+0000","last_became_active":"2024-03-06T07:22:44.372321+0000","last_became_peered":"2024-03-06T07:22:44.372321+0000","last_unstale":"2024-03-06T07:22:46.958623+0000","last_undegraded":"2024-03-06T07:22:46.958623+0000","last_fullsized":"2024-03-06T07:22:46.958623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:22.300637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098609+0000","last_change":"2024-03-06T07:22:35.042271+0000","last_active":"2024-03-06T07:22:48.098609+0000","last_peered":"2024-03-06T07:22:48.098609+0000","last_clean":"2024-03-06T07:22:48.098609+0000","last_became_active":"2024-03-06T07:22:35.036189+0000","last_became_peered":"2024-03-06T07:22:35.036189+0000","last_unstale":"2024-03-06T07:22:48.098609+0000","last_undegraded":"2024-03-06T07:22:48.098609+0000","last_fullsized":"2024-03-06T07:22:48.098609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:47.064005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098447+0000","last_change":"2024-03-06T07:22:34.036505+0000","last_active":"2024-03-06T07:22:48.098447+0000","last_peered":"2024-03-06T07:22:48.098447+0000","last_clean":"2024-03-06T07:22:48.098447+0000","last_became_active":"2024-03-06T07:22:34.036198+0000","last_became_peered":"2024-03-06T07:22:34.036198+0000","last_unstale":"2024-03-06T07:22:48.098447+0000","last_undegraded":"2024-03-06T07:22:48.098447+0000","last_fullsized":"2024-03-06T07:22:48.098447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958295+0000","last_change":"2024-03-06T07:22:45.376092+0000","last_active":"2024-03-06T07:22:46.958295+0000","last_peered":"2024-03-06T07:22:46.958295+0000","last_clean":"2024-03-06T07:22:46.958295+0000","last_became_active":"2024-03-06T07:22:45.375812+0000","last_became_peered":"2024-03-06T07:22:45.375812+0000","last_unstale":"2024-03-06T07:22:46.958295+0000","last_undegraded":"2024-03-06T07:22:46.958295+0000","last_fullsized":"2024-03-06T07:22:46.958295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:26.605704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378515+0000","last_change":"2024-03-06T07:22:44.372300+0000","last_active":"2024-03-06T07:22:46.378515+0000","last_peered":"2024-03-06T07:22:46.378515+0000","last_clean":"2024-03-06T07:22:46.378515+0000","last_became_active":"2024-03-06T07:22:44.372036+0000","last_became_peered":"2024-03-06T07:22:44.372036+0000","last_unstale":"2024-03-06T07:22:46.378515+0000","last_undegraded":"2024-03-06T07:22:46.378515+0000","last_fullsized":"2024-03-06T07:22:46.378515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:35.747227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958808+0000","last_change":"2024-03-06T07:22:35.031196+0000","last_active":"2024-03-06T07:22:46.958808+0000","last_peered":"2024-03-06T07:22:46.958808+0000","last_clean":"2024-03-06T07:22:46.958808+0000","last_became_active":"2024-03-06T07:22:35.030767+0000","last_became_peered":"2024-03-06T07:22:35.030767+0000","last_unstale":"2024-03-06T07:22:46.958808+0000","last_undegraded":"2024-03-06T07:22:46.958808+0000","last_fullsized":"2024-03-06T07:22:46.958808+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:43.288135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.097971+0000","last_change":"2024-03-06T07:22:34.034863+0000","last_active":"2024-03-06T07:22:48.097971+0000","last_peered":"2024-03-06T07:22:48.097971+0000","last_clean":"2024-03-06T07:22:48.097971+0000","last_became_active":"2024-03-06T07:22:34.034577+0000","last_became_peered":"2024-03-06T07:22:34.034577+0000","last_unstale":"2024-03-06T07:22:48.097971+0000","last_undegraded":"2024-03-06T07:22:48.097971+0000","last_fullsized":"2024-03-06T07:22:48.097971+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:17.400640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958085+0000","last_change":"2024-03-06T07:22:45.386396+0000","last_active":"2024-03-06T07:22:46.958085+0000","last_peered":"2024-03-06T07:22:46.958085+0000","last_clean":"2024-03-06T07:22:46.958085+0000","last_became_active":"2024-03-06T07:22:45.386149+0000","last_became_peered":"2024-03-06T07:22:45.386149+0000","last_unstale":"2024-03-06T07:22:46.958085+0000","last_undegraded":"2024-03-06T07:22:46.958085+0000","last_fullsized":"2024-03-06T07:22:46.958085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:33.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.957911+0000","last_change":"2024-03-06T07:22:44.374457+0000","last_active":"2024-03-06T07:22:46.957911+0000","last_peered":"2024-03-06T07:22:46.957911+0000","last_clean":"2024-03-06T07:22:46.957911+0000","last_became_active":"2024-03-06T07:22:44.374059+0000","last_became_peered":"2024-03-06T07:22:44.374059+0000","last_unstale":"2024-03-06T07:22:46.957911+0000","last_undegraded":"2024-03-06T07:22:46.957911+0000","last_fullsized":"2024-03-06T07:22:46.957911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:47:04.416336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380258+0000","last_change":"2024-03-06T07:22:35.036664+0000","last_active":"2024-03-06T07:22:46.380258+0000","last_peered":"2024-03-06T07:22:46.380258+0000","last_clean":"2024-03-06T07:22:46.380258+0000","last_became_active":"2024-03-06T07:22:35.036379+0000","last_became_peered":"2024-03-06T07:22:35.036379+0000","last_unstale":"2024-03-06T07:22:46.380258+0000","last_undegraded":"2024-03-06T07:22:46.380258+0000","last_fullsized":"2024-03-06T07:22:46.380258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:20:56.002343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377928+0000","last_change":"2024-03-06T07:22:34.044993+0000","last_active":"2024-03-06T07:22:46.377928+0000","last_peered":"2024-03-06T07:22:46.377928+0000","last_clean":"2024-03-06T07:22:46.377928+0000","last_became_active":"2024-03-06T07:22:34.044795+0000","last_became_peered":"2024-03-06T07:22:34.044795+0000","last_unstale":"2024-03-06T07:22:46.377928+0000","last_undegraded":"2024-03-06T07:22:46.377928+0000","last_fullsized":"2024-03-06T07:22:46.377928+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.256142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098065+0000","last_change":"2024-03-06T07:22:45.392078+0000","last_active":"2024-03-06T07:22:48.098065+0000","last_peered":"2024-03-06T07:22:48.098065+0000","last_clean":"2024-03-06T07:22:48.098065+0000","last_became_active":"2024-03-06T07:22:45.391804+0000","last_became_peered":"2024-03-06T07:22:45.391804+0000","last_unstale":"2024-03-06T07:22:48.098065+0000","last_undegraded":"2024-03-06T07:22:48.098065+0000","last_fullsized":"2024-03-06T07:22:48.098065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:03.218729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377577+0000","last_change":"2024-03-06T07:22:44.369924+0000","last_active":"2024-03-06T07:22:46.377577+0000","last_peered":"2024-03-06T07:22:46.377577+0000","last_clean":"2024-03-06T07:22:46.377577+0000","last_became_active":"2024-03-06T07:22:44.369676+0000","last_became_peered":"2024-03-06T07:22:44.369676+0000","last_unstale":"2024-03-06T07:22:46.377577+0000","last_undegraded":"2024-03-06T07:22:46.377577+0000","last_fullsized":"2024-03-06T07:22:46.377577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:32:17.584765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958130+0000","last_change":"2024-03-06T07:22:35.031505+0000","last_active":"2024-03-06T07:22:46.958130+0000","last_peered":"2024-03-06T07:22:46.958130+0000","last_clean":"2024-03-06T07:22:46.958130+0000","last_became_active":"2024-03-06T07:22:35.031241+0000","last_became_peered":"2024-03-06T07:22:35.031241+0000","last_unstale":"2024-03-06T07:22:46.958130+0000","last_undegraded":"2024-03-06T07:22:46.958130+0000","last_fullsized":"2024-03-06T07:22:46.958130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098719+0000","last_change":"2024-03-06T07:22:34.044453+0000","last_active":"2024-03-06T07:22:48.098719+0000","last_peered":"2024-03-06T07:22:48.098719+0000","last_clean":"2024-03-06T07:22:48.098719+0000","last_became_active":"2024-03-06T07:22:34.044190+0000","last_became_peered":"2024-03-06T07:22:34.044190+0000","last_unstale":"2024-03-06T07:22:48.098719+0000","last_undegraded":"2024-03-06T07:22:48.098719+0000","last_fullsized":"2024-03-06T07:22:48.098719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.167638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078303+0000","last_change":"2024-03-06T07:22:45.392860+0000","last_active":"2024-03-06T07:22:48.078303+0000","last_peered":"2024-03-06T07:22:48.078303+0000","last_clean":"2024-03-06T07:22:48.078303+0000","last_became_active":"2024-03-06T07:22:45.392256+0000","last_became_peered":"2024-03-06T07:22:45.392256+0000","last_unstale":"2024-03-06T07:22:48.078303+0000","last_undegraded":"2024-03-06T07:22:48.078303+0000","last_fullsized":"2024-03-06T07:22:48.078303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:52.412501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098092+0000","last_change":"2024-03-06T07:22:44.382519+0000","last_active":"2024-03-06T07:22:48.098092+0000","last_peered":"2024-03-06T07:22:48.098092+0000","last_clean":"2024-03-06T07:22:48.098092+0000","last_became_active":"2024-03-06T07:22:44.382295+0000","last_became_peered":"2024-03-06T07:22:44.382295+0000","last_unstale":"2024-03-06T07:22:48.098092+0000","last_undegraded":"2024-03-06T07:22:48.098092+0000","last_fullsized":"2024-03-06T07:22:48.098092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:09.178075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381009+0000","last_change":"2024-03-06T07:22:35.034792+0000","last_active":"2024-03-06T07:22:46.381009+0000","last_peered":"2024-03-06T07:22:46.381009+0000","last_clean":"2024-03-06T07:22:46.381009+0000","last_became_active":"2024-03-06T07:22:35.034503+0000","last_became_peered":"2024-03-06T07:22:35.034503+0000","last_unstale":"2024-03-06T07:22:46.381009+0000","last_undegraded":"2024-03-06T07:22:46.381009+0000","last_fullsized":"2024-03-06T07:22:46.381009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:37.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378591+0000","last_change":"2024-03-06T07:22:34.037188+0000","last_active":"2024-03-06T07:22:46.378591+0000","last_peered":"2024-03-06T07:22:46.378591+0000","last_clean":"2024-03-06T07:22:46.378591+0000","last_became_active":"2024-03-06T07:22:34.036918+0000","last_became_peered":"2024-03-06T07:22:34.036918+0000","last_unstale":"2024-03-06T07:22:46.378591+0000","last_undegraded":"2024-03-06T07:22:46.378591+0000","last_fullsized":"2024-03-06T07:22:46.378591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:54.075136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098354+0000","last_change":"2024-03-06T07:22:45.389372+0000","last_active":"2024-03-06T07:22:48.098354+0000","last_peered":"2024-03-06T07:22:48.098354+0000","last_clean":"2024-03-06T07:22:48.098354+0000","last_became_active":"2024-03-06T07:22:45.389104+0000","last_became_peered":"2024-03-06T07:22:45.389104+0000","last_unstale":"2024-03-06T07:22:48.098354+0000","last_undegraded":"2024-03-06T07:22:48.098354+0000","last_fullsized":"2024-03-06T07:22:48.098354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:36.752131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378275+0000","last_change":"2024-03-06T07:22:44.371466+0000","last_active":"2024-03-06T07:22:46.378275+0000","last_peered":"2024-03-06T07:22:46.378275+0000","last_clean":"2024-03-06T07:22:46.378275+0000","last_became_active":"2024-03-06T07:22:44.371190+0000","last_became_peered":"2024-03-06T07:22:44.371190+0000","last_unstale":"2024-03-06T07:22:46.378275+0000","last_undegraded":"2024-03-06T07:22:46.378275+0000","last_fullsized":"2024-03-06T07:22:46.378275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:22.948560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098285+0000","last_change":"2024-03-06T07:22:35.033263+0000","last_active":"2024-03-06T07:22:48.098285+0000","last_peered":"2024-03-06T07:22:48.098285+0000","last_clean":"2024-03-06T07:22:48.098285+0000","last_became_active":"2024-03-06T07:22:35.033005+0000","last_became_peered":"2024-03-06T07:22:35.033005+0000","last_unstale":"2024-03-06T07:22:48.098285+0000","last_undegraded":"2024-03-06T07:22:48.098285+0000","last_fullsized":"2024-03-06T07:22:48.098285+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:00.137617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378069+0000","last_change":"2024-03-06T07:22:34.035735+0000","last_active":"2024-03-06T07:22:46.378069+0000","last_peered":"2024-03-06T07:22:46.378069+0000","last_clean":"2024-03-06T07:22:46.378069+0000","last_became_active":"2024-03-06T07:22:34.035459+0000","last_became_peered":"2024-03-06T07:22:34.035459+0000","last_unstale":"2024-03-06T07:22:46.378069+0000","last_undegraded":"2024-03-06T07:22:46.378069+0000","last_fullsized":"2024-03-06T07:22:46.378069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:47.286043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098522+0000","last_change":"2024-03-06T07:22:45.381082+0000","last_active":"2024-03-06T07:22:48.098522+0000","last_peered":"2024-03-06T07:22:48.098522+0000","last_clean":"2024-03-06T07:22:48.098522+0000","last_became_active":"2024-03-06T07:22:45.380817+0000","last_became_peered":"2024-03-06T07:22:45.380817+0000","last_unstale":"2024-03-06T07:22:48.098522+0000","last_undegraded":"2024-03-06T07:22:48.098522+0000","last_fullsized":"2024-03-06T07:22:48.098522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:29.643040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.381178+0000","last_change":"2024-03-06T07:22:44.377358+0000","last_active":"2024-03-06T07:22:46.381178+0000","last_peered":"2024-03-06T07:22:46.381178+0000","last_clean":"2024-03-06T07:22:46.381178+0000","last_became_active":"2024-03-06T07:22:44.377098+0000","last_became_peered":"2024-03-06T07:22:44.377098+0000","last_unstale":"2024-03-06T07:22:46.381178+0000","last_undegraded":"2024-03-06T07:22:46.381178+0000","last_fullsized":"2024-03-06T07:22:46.381178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:01.508839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958343+0000","last_change":"2024-03-06T07:22:35.038111+0000","last_active":"2024-03-06T07:22:46.958343+0000","last_peered":"2024-03-06T07:22:46.958343+0000","last_clean":"2024-03-06T07:22:46.958343+0000","last_became_active":"2024-03-06T07:22:35.037779+0000","last_became_peered":"2024-03-06T07:22:35.037779+0000","last_unstale":"2024-03-06T07:22:46.958343+0000","last_undegraded":"2024-03-06T07:22:46.958343+0000","last_fullsized":"2024-03-06T07:22:46.958343+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958162+0000","last_change":"2024-03-06T07:22:34.039910+0000","last_active":"2024-03-06T07:22:46.958162+0000","last_peered":"2024-03-06T07:22:46.958162+0000","last_clean":"2024-03-06T07:22:46.958162+0000","last_became_active":"2024-03-06T07:22:34.039246+0000","last_became_peered":"2024-03-06T07:22:34.039246+0000","last_unstale":"2024-03-06T07:22:46.958162+0000","last_undegraded":"2024-03-06T07:22:46.958162+0000","last_fullsized":"2024-03-06T07:22:46.958162+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:41.961694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.377732+0000","last_change":"2024-03-06T07:22:45.379802+0000","last_active":"2024-03-06T07:22:46.377732+0000","last_peered":"2024-03-06T07:22:46.377732+0000","last_clean":"2024-03-06T07:22:46.377732+0000","last_became_active":"2024-03-06T07:22:45.379486+0000","last_became_peered":"2024-03-06T07:22:45.379486+0000","last_unstale":"2024-03-06T07:22:46.377732+0000","last_undegraded":"2024-03-06T07:22:46.377732+0000","last_fullsized":"2024-03-06T07:22:46.377732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:01.230997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958269+0000","last_change":"2024-03-06T07:22:44.374437+0000","last_active":"2024-03-06T07:22:46.958269+0000","last_peered":"2024-03-06T07:22:46.958269+0000","last_clean":"2024-03-06T07:22:46.958269+0000","last_became_active":"2024-03-06T07:22:44.374055+0000","last_became_peered":"2024-03-06T07:22:44.374055+0000","last_unstale":"2024-03-06T07:22:46.958269+0000","last_undegraded":"2024-03-06T07:22:46.958269+0000","last_fullsized":"2024-03-06T07:22:46.958269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:35.517690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958405+0000","last_change":"2024-03-06T07:22:35.027657+0000","last_active":"2024-03-06T07:22:46.958405+0000","last_peered":"2024-03-06T07:22:46.958405+0000","last_clean":"2024-03-06T07:22:46.958405+0000","last_became_active":"2024-03-06T07:22:35.027404+0000","last_became_peered":"2024-03-06T07:22:35.027404+0000","last_unstale":"2024-03-06T07:22:46.958405+0000","last_undegraded":"2024-03-06T07:22:46.958405+0000","last_fullsized":"2024-03-06T07:22:46.958405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:34.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.078741+0000","last_change":"2024-03-06T07:22:34.039209+0000","last_active":"2024-03-06T07:22:48.078741+0000","last_peered":"2024-03-06T07:22:48.078741+0000","last_clean":"2024-03-06T07:22:48.078741+0000","last_became_active":"2024-03-06T07:22:34.038921+0000","last_became_peered":"2024-03-06T07:22:34.038921+0000","last_unstale":"2024-03-06T07:22:48.078741+0000","last_undegraded":"2024-03-06T07:22:48.078741+0000","last_fullsized":"2024-03-06T07:22:48.078741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:12.359435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958481+0000","last_change":"2024-03-06T07:22:45.370714+0000","last_active":"2024-03-06T07:22:46.958481+0000","last_peered":"2024-03-06T07:22:46.958481+0000","last_clean":"2024-03-06T07:22:46.958481+0000","last_became_active":"2024-03-06T07:22:45.370435+0000","last_became_peered":"2024-03-06T07:22:45.370435+0000","last_unstale":"2024-03-06T07:22:46.958481+0000","last_undegraded":"2024-03-06T07:22:46.958481+0000","last_fullsized":"2024-03-06T07:22:46.958481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:10.476716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379408+0000","last_change":"2024-03-06T07:22:44.374940+0000","last_active":"2024-03-06T07:22:46.379408+0000","last_peered":"2024-03-06T07:22:46.379408+0000","last_clean":"2024-03-06T07:22:46.379408+0000","last_became_active":"2024-03-06T07:22:44.374653+0000","last_became_peered":"2024-03-06T07:22:44.374653+0000","last_unstale":"2024-03-06T07:22:46.379408+0000","last_undegraded":"2024-03-06T07:22:46.379408+0000","last_fullsized":"2024-03-06T07:22:46.379408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:00.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931817+0000","last_change":"2024-03-06T07:22:35.038010+0000","last_active":"2024-03-06T07:22:46.931817+0000","last_peered":"2024-03-06T07:22:46.931817+0000","last_clean":"2024-03-06T07:22:46.931817+0000","last_became_active":"2024-03-06T07:22:35.037717+0000","last_became_peered":"2024-03-06T07:22:35.037717+0000","last_unstale":"2024-03-06T07:22:46.931817+0000","last_undegraded":"2024-03-06T07:22:46.931817+0000","last_fullsized":"2024-03-06T07:22:46.931817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:59:23.659305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.958285+0000","last_change":"2024-03-06T07:22:34.039679+0000","last_active":"2024-03-06T07:22:46.958285+0000","last_peered":"2024-03-06T07:22:46.958285+0000","last_clean":"2024-03-06T07:22:46.958285+0000","last_became_active":"2024-03-06T07:22:34.039092+0000","last_became_peered":"2024-03-06T07:22:34.039092+0000","last_unstale":"2024-03-06T07:22:46.958285+0000","last_undegraded":"2024-03-06T07:22:46.958285+0000","last_fullsized":"2024-03-06T07:22:46.958285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.098306+0000","last_change":"2024-03-06T07:22:45.386532+0000","last_active":"2024-03-06T07:22:48.098306+0000","last_peered":"2024-03-06T07:22:48.098306+0000","last_clean":"2024-03-06T07:22:48.098306+0000","last_became_active":"2024-03-06T07:22:45.386237+0000","last_became_peered":"2024-03-06T07:22:45.386237+0000","last_unstale":"2024-03-06T07:22:48.098306+0000","last_undegraded":"2024-03-06T07:22:48.098306+0000","last_fullsized":"2024-03-06T07:22:48.098306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:04:34.799679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.380757+0000","last_change":"2024-03-06T07:22:44.369327+0000","last_active":"2024-03-06T07:22:46.380757+0000","last_peered":"2024-03-06T07:22:46.380757+0000","last_clean":"2024-03-06T07:22:46.380757+0000","last_became_active":"2024-03-06T07:22:44.369076+0000","last_became_peered":"2024-03-06T07:22:44.369076+0000","last_unstale":"2024-03-06T07:22:46.380757+0000","last_undegraded":"2024-03-06T07:22:46.380757+0000","last_fullsized":"2024-03-06T07:22:46.380757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:52.622331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931674+0000","last_change":"2024-03-06T07:22:35.026900+0000","last_active":"2024-03-06T07:22:46.931674+0000","last_peered":"2024-03-06T07:22:46.931674+0000","last_clean":"2024-03-06T07:22:46.931674+0000","last_became_active":"2024-03-06T07:22:35.026626+0000","last_became_peered":"2024-03-06T07:22:35.026626+0000","last_unstale":"2024-03-06T07:22:46.931674+0000","last_undegraded":"2024-03-06T07:22:46.931674+0000","last_fullsized":"2024-03-06T07:22:46.931674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:27.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931355+0000","last_change":"2024-03-06T07:22:34.039382+0000","last_active":"2024-03-06T07:22:46.931355+0000","last_peered":"2024-03-06T07:22:46.931355+0000","last_clean":"2024-03-06T07:22:46.931355+0000","last_became_active":"2024-03-06T07:22:34.039088+0000","last_became_peered":"2024-03-06T07:22:34.039088+0000","last_unstale":"2024-03-06T07:22:46.931355+0000","last_undegraded":"2024-03-06T07:22:46.931355+0000","last_fullsized":"2024-03-06T07:22:46.931355+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:24.137864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.379361+0000","last_change":"2024-03-06T07:22:45.376865+0000","last_active":"2024-03-06T07:22:46.379361+0000","last_peered":"2024-03-06T07:22:46.379361+0000","last_clean":"2024-03-06T07:22:46.379361+0000","last_became_active":"2024-03-06T07:22:45.376601+0000","last_became_peered":"2024-03-06T07:22:45.376601+0000","last_unstale":"2024-03-06T07:22:46.379361+0000","last_undegraded":"2024-03-06T07:22:46.379361+0000","last_fullsized":"2024-03-06T07:22:46.379361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:44.351281+0000","last_clean_scrub_stamp":"2024-03-06T07:22:44.351281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:00.351065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.931706+0000","last_change":"2024-03-06T07:22:44.374409+0000","last_active":"2024-03-06T07:22:46.931706+0000","last_peered":"2024-03-06T07:22:46.931706+0000","last_clean":"2024-03-06T07:22:46.931706+0000","last_became_active":"2024-03-06T07:22:44.374130+0000","last_became_peered":"2024-03-06T07:22:44.374130+0000","last_unstale":"2024-03-06T07:22:46.931706+0000","last_undegraded":"2024-03-06T07:22:46.931706+0000","last_fullsized":"2024-03-06T07:22:46.931706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:43.346377+0000","last_clean_scrub_stamp":"2024-03-06T07:22:43.346377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:57.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378507+0000","last_change":"2024-03-06T07:22:35.033671+0000","last_active":"2024-03-06T07:22:46.378507+0000","last_peered":"2024-03-06T07:22:46.378507+0000","last_clean":"2024-03-06T07:22:46.378507+0000","last_became_active":"2024-03-06T07:22:35.033167+0000","last_became_peered":"2024-03-06T07:22:35.033167+0000","last_unstale":"2024-03-06T07:22:46.378507+0000","last_undegraded":"2024-03-06T07:22:46.378507+0000","last_fullsized":"2024-03-06T07:22:46.378507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.012182+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.012182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:23.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:48.099040+0000","last_change":"2024-03-06T07:22:34.037026+0000","last_active":"2024-03-06T07:22:48.099040+0000","last_peered":"2024-03-06T07:22:48.099040+0000","last_clean":"2024-03-06T07:22:48.099040+0000","last_became_active":"2024-03-06T07:22:34.036742+0000","last_became_peered":"2024-03-06T07:22:34.036742+0000","last_unstale":"2024-03-06T07:22:48.099040+0000","last_undegraded":"2024-03-06T07:22:48.099040+0000","last_fullsized":"2024-03-06T07:22:48.099040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.005176+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.005176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:03.002919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":280,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-06T07:22:46.378054+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:22:46.378054+0000","last_peered":"2024-03-06T07:22:46.378054+0000","last_clean":"2024-03-06T07:22:46.378054+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:22:46.378054+0000","last_undegraded":"2024-03-06T07:22:46.378054+0000","last_fullsized":"2024-03-06T07:22:46.378054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1032192,"data_stored":1048109,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":461491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":457697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":459539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":958464,"data_stored":458245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":966656,"data_stored":458405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":966656,"data_stored":457845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":1050349,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:53.358 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:22:53.359 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs refuse_standby_for_another_fs true 2024-03-06T07:22:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.498+0000 7efdbf13a640 1 Processor -- start 2024-03-06T07:22:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.498+0000 7efdbf13a640 1 -- start start 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdbf13a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efdb80c5370 0x7efdb80c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdbf13a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb8152dd0 con 0x7efdb81529f0 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdbf13a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb8158b00 con 0x7efdb8155530 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdbf13a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb8159710 con 0x7efdb80c5370 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdbced6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efdb81529f0 0x7efdb80c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57534/0 (socket says 172.21.15.26:57534) 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdbced6640 1 -- 172.21.15.26:0/2342636248 learned_addr learned my addr 172.21.15.26:0/2342636248 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdb7fff640 1 --2- 172.21.15.26:0/2342636248 >> v2:172.21.15.117:3300/0 conn(0x7efdb80c5370 0x7efdb80c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdb77fe640 1 -- 172.21.15.26:0/2342636248 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745065845 0 0) 0x7efdb8152dd0 con 0x7efdb81529f0 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdb77fe640 1 -- 172.21.15.26:0/2342636248 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efda8003520 con 0x7efdb81529f0 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdb7fff640 1 -- 172.21.15.26:0/2342636248 >> v1:172.21.15.117:6789/0 conn(0x7efdb8155530 legacy=0x7efdb81559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdb7fff640 1 -- 172.21.15.26:0/2342636248 >> v1:172.21.15.26:6789/0 conn(0x7efdb81529f0 legacy=0x7efdb80c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdb77fe640 1 -- 172.21.15.26:0/2342636248 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330679569 0 0) 0x7efdb8158b00 con 0x7efdb8155530 2024-03-06T07:22:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.499+0000 7efdb7fff640 1 -- 172.21.15.26:0/2342636248 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb8158de0 con 0x7efdb80c5370 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.500+0000 7efdb7fff640 1 --2- 172.21.15.26:0/2342636248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb80c5370 0x7efdb80c57d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84a343bde30e9a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.500+0000 7efdb77fe640 1 -- 172.21.15.26:0/2342636248 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4035020 con 0x7efdb80c5370 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.500+0000 7efdb77fe640 1 -- 172.21.15.26:0/2342636248 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda4031830 con 0x7efdb80c5370 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.500+0000 7efdb77fe640 1 -- 172.21.15.26:0/2342636248 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4031c00 con 0x7efdb80c5370 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.501+0000 7efdbf13a640 1 -- 172.21.15.26:0/2342636248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb80c5370 msgr2=0x7efdb80c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.501+0000 7efdbf13a640 1 --2- 172.21.15.26:0/2342636248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb80c5370 0x7efdb80c57d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.501+0000 7efdbf13a640 1 -- 172.21.15.26:0/2342636248 shutdown_connections 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.501+0000 7efdbf13a640 1 --2- 172.21.15.26:0/2342636248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb80c5370 0x7efdb80c57d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.501+0000 7efdbf13a640 1 -- 172.21.15.26:0/2342636248 >> 172.21.15.26:0/2342636248 conn(0x7efdb80ac580 msgr2=0x7efdb80ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.501+0000 7efdbf13a640 1 -- 172.21.15.26:0/2342636248 shutdown_connections 2024-03-06T07:22:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.501+0000 7efdbf13a640 1 -- 172.21.15.26:0/2342636248 wait complete. 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.502+0000 7efdbf13a640 1 Processor -- start 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.502+0000 7efdbf13a640 1 -- start start 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbf13a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdb80c5370 0x7efdb81ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbf13a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdb81529f0 0x7efdb80ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbf13a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb8155530 0x7efdb814b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbf13a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80c9000 con 0x7efdb81529f0 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbf13a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80c68b0 con 0x7efdb8155530 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbf13a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb81593a0 con 0x7efdb80c5370 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdb81529f0 0x7efdb80ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdb81529f0 0x7efdb80ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49250/0 (socket says 172.21.15.26:49250) 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 -- 172.21.15.26:0/906607115 learned_addr learned my addr 172.21.15.26:0/906607115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbd6d7640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb8155530 0x7efdb814b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbced6640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdb80c5370 0x7efdb81ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 -- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdb80c5370 msgr2=0x7efdb81ee700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdb80c5370 0x7efdb81ee700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 -- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb8155530 msgr2=0x7efdb814b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb8155530 0x7efdb814b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 -- 172.21.15.26:0/906607115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb81ea810 con 0x7efdb81529f0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbd6d7640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb8155530 0x7efdb814b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb7fff640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdb81529f0 0x7efdb80ab9f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efda402bee0 tx=0x7efda40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb57fa640 1 -- 172.21.15.26:0/906607115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4035040 con 0x7efdb81529f0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb57fa640 1 -- 172.21.15.26:0/906607115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda4004220 con 0x7efdb81529f0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb81eec40 con 0x7efdb81529f0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.503+0000 7efdb57fa640 1 -- 172.21.15.26:0/906607115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda403a700 con 0x7efdb81529f0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.504+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb81ead90 con 0x7efdb81529f0 2024-03-06T07:22:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.504+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb804f330 con 0x7efdb81529f0 2024-03-06T07:22:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.505+0000 7efdb57fa640 1 -- 172.21.15.26:0/906607115 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efda4038020 con 0x7efdb81529f0 2024-03-06T07:22:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.505+0000 7efdb57fa640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9c06b4d0 0x7efd9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.506+0000 7efdb57fa640 1 -- 172.21.15.26:0/906607115 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7efda40ac3e0 con 0x7efdb81529f0 2024-03-06T07:22:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.506+0000 7efdbced6640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9c06b4d0 0x7efd9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.506+0000 7efdbced6640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9c06b4d0 0x7efd9c06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efdac005880 tx=0x7efdac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.514+0000 7efdb57fa640 1 -- 172.21.15.26:0/906607115 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efda407a8a0 con 0x7efdb81529f0 2024-03-06T07:22:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:53.735+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "refuse_standby_for_another_fs", "val": "true"} v 0) v1 -- 0x7efdb806c340 con 0x7efdb81529f0 2024-03-06T07:22:54.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.392+0000 7efdb57fa640 1 -- 172.21.15.26:0/906607115 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "refuse_standby_for_another_fs", "val": "true"}]=0 set to refuse standby for another fs v59) v1 ==== 166+0+0 (secure 0 0 0) 0x7efda407e940 con 0x7efdb81529f0 2024-03-06T07:22:54.394 INFO:teuthology.orchestra.run.smithi026.stderr:set to refuse standby for another fs 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9c06b4d0 msgr2=0x7efd9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9c06b4d0 0x7efd9c06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efdac005880 tx=0x7efdac009040 comp rx=0 tx=0).stop 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdb81529f0 msgr2=0x7efdb80ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdb81529f0 0x7efdb80ab9f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efda402bee0 tx=0x7efda40046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 shutdown_connections 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9c06b4d0 0x7efd9c06d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdb8155530 0x7efdb814b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdb81529f0 0x7efdb80ab9f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 --2- 172.21.15.26:0/906607115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdb80c5370 0x7efdb81ee700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 >> 172.21.15.26:0/906607115 conn(0x7efdb80ac580 msgr2=0x7efdb814fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:54.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 shutdown_connections 2024-03-06T07:22:54.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.394+0000 7efdbf13a640 1 -- 172.21.15.26:0/906607115 wait complete. 2024-03-06T07:22:54.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 5:0 2024-03-06T07:22:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.508+0000 7fd298ca3640 1 Processor -- start 2024-03-06T07:22:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.508+0000 7fd298ca3640 1 -- start start 2024-03-06T07:22:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd298ca3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2940c7170 0x7fd2940c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd298ca3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2940c6c40 0x7fd2940c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd298ca3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940a8390 con 0x7fd29406ff10 2024-03-06T07:22:54.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd298ca3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940c7620 con 0x7fd2940c6c40 2024-03-06T07:22:54.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd298ca3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940730f0 con 0x7fd2940c7170 2024-03-06T07:22:54.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd292d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2940c6c40 0x7fd2940c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd29259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd29406ff10 0x7fd294153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57540/0 (socket says 172.21.15.26:57540) 2024-03-06T07:22:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd29259c640 1 -- 172.21.15.26:0/4253117659 learned_addr learned my addr 172.21.15.26:0/4253117659 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.509+0000 7fd291d9b640 1 --2- 172.21.15.26:0/4253117659 >> v2:172.21.15.117:3300/0 conn(0x7fd2940c7170 0x7fd2940c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.510+0000 7fd292d9d640 1 -- 172.21.15.26:0/4253117659 >> v2:172.21.15.117:3300/0 conn(0x7fd2940c7170 msgr2=0x7fd2940c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.510+0000 7fd292d9d640 1 --2- 172.21.15.26:0/4253117659 >> v2:172.21.15.117:3300/0 conn(0x7fd2940c7170 0x7fd2940c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.510+0000 7fd292d9d640 1 -- 172.21.15.26:0/4253117659 >> v1:172.21.15.26:6789/0 conn(0x7fd29406ff10 legacy=0x7fd294153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.510+0000 7fd292d9d640 1 -- 172.21.15.26:0/4253117659 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940733d0 con 0x7fd2940c6c40 2024-03-06T07:22:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.510+0000 7fd29159a640 1 -- 172.21.15.26:0/4253117659 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68814230 0 0) 0x7fd2940a8390 con 0x7fd29406ff10 2024-03-06T07:22:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.510+0000 7fd291d9b640 1 --2- 172.21.15.26:0/4253117659 >> v2:172.21.15.117:3300/0 conn(0x7fd2940c7170 0x7fd2940c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.510+0000 7fd292d9d640 1 --2- 172.21.15.26:0/4253117659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2940c8d50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd29406f420 tx=0x7fd28402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b421f571726e159 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd29159a640 1 -- 172.21.15.26:0/4253117659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284039070 con 0x7fd2940c6c40 2024-03-06T07:22:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd29159a640 1 -- 172.21.15.26:0/4253117659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28402cb60 con 0x7fd2940c6c40 2024-03-06T07:22:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd29159a640 1 -- 172.21.15.26:0/4253117659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284034440 con 0x7fd2940c6c40 2024-03-06T07:22:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 -- 172.21.15.26:0/4253117659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 msgr2=0x7fd2940c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 --2- 172.21.15.26:0/4253117659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2940c8d50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd29406f420 tx=0x7fd28402bdb0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 -- 172.21.15.26:0/4253117659 shutdown_connections 2024-03-06T07:22:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 --2- 172.21.15.26:0/4253117659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2940c8d50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 --2- 172.21.15.26:0/4253117659 >> v2:172.21.15.117:3300/0 conn(0x7fd2940c7170 0x7fd2940c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 -- 172.21.15.26:0/4253117659 >> 172.21.15.26:0/4253117659 conn(0x7fd29406b970 msgr2=0x7fd2940aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 -- 172.21.15.26:0/4253117659 shutdown_connections 2024-03-06T07:22:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.511+0000 7fd298ca3640 1 -- 172.21.15.26:0/4253117659 wait complete. 2024-03-06T07:22:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.513+0000 7fd298ca3640 1 Processor -- start 2024-03-06T07:22:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.513+0000 7fd298ca3640 1 -- start start 2024-03-06T07:22:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.513+0000 7fd298ca3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd29406ff10 0x7fd2940c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.513+0000 7fd298ca3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2941eb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd298ca3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c7170 0x7fd29414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd298ca3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2941544b0 con 0x7fd2940c6c40 2024-03-06T07:22:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd298ca3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294154330 con 0x7fd29406ff10 2024-03-06T07:22:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd298ca3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940702f0 con 0x7fd2940c7170 2024-03-06T07:22:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2941eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2941eb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49268/0 (socket says 172.21.15.26:49268) 2024-03-06T07:22:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 -- 172.21.15.26:0/3807151799 learned_addr learned my addr 172.21.15.26:0/3807151799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd29259c640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd29406ff10 0x7fd2940c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd292d9d640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c7170 0x7fd29414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 -- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c7170 msgr2=0x7fd29414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c7170 0x7fd29414cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 -- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd29406ff10 msgr2=0x7fd2940c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd29406ff10 0x7fd2940c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 -- 172.21.15.26:0/3807151799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd294154b50 con 0x7fd2940c6c40 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd291d9b640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2941eb320 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e990 tx=0x7fd28800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd282ffd640 1 -- 172.21.15.26:0/3807151799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288017070 con 0x7fd2940c6c40 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd282ffd640 1 -- 172.21.15.26:0/3807151799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28800c9e0 con 0x7fd2940c6c40 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd294155000 con 0x7fd2940c6c40 2024-03-06T07:22:54.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd282ffd640 1 -- 172.21.15.26:0/3807151799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28800cdb0 con 0x7fd2940c6c40 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.514+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941eae00 con 0x7fd2940c6c40 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.516+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29404f330 con 0x7fd2940c6c40 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.516+0000 7fd282ffd640 1 -- 172.21.15.26:0/3807151799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd28801a070 con 0x7fd2940c6c40 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.516+0000 7fd282ffd640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27406b4d0 0x7fd27406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.517+0000 7fd282ffd640 1 -- 172.21.15.26:0/3807151799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fd28808a520 con 0x7fd2940c6c40 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.517+0000 7fd29259c640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27406b4d0 0x7fd27406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.517+0000 7fd29259c640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27406b4d0 0x7fd27406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd28c000f60 tx=0x7fd28c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.520+0000 7fd282ffd640 1 -- 172.21.15.26:0/3807151799 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd288058860 con 0x7fd2940c6c40 2024-03-06T07:22:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.777+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "5:0"} v 0) v1 -- 0x7fd29406fa80 con 0x7fd2940c6c40 2024-03-06T07:22:54.778 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:22:54.777+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.778+0000 7fd282ffd640 1 -- 172.21.15.26:0/3807151799 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "5:0"}]=0 failed mds gid 6339 v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd28805c900 con 0x7fd2940c6c40 2024-03-06T07:22:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:failed mds gid 6339 2024-03-06T07:22:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.781+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27406b4d0 msgr2=0x7fd27406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.781+0000 7fd298ca3640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27406b4d0 0x7fd27406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd28c000f60 tx=0x7fd28c007d50 comp rx=0 tx=0).stop 2024-03-06T07:22:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.781+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 msgr2=0x7fd2941eb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.781+0000 7fd298ca3640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2941eb320 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e990 tx=0x7fd28800ee50 comp rx=0 tx=0).stop 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.781+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 shutdown_connections 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.781+0000 7fd298ca3640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27406b4d0 0x7fd27406d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.781+0000 7fd298ca3640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c7170 0x7fd29414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.782+0000 7fd298ca3640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c6c40 0x7fd2941eb320 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.782+0000 7fd298ca3640 1 --2- 172.21.15.26:0/3807151799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd29406ff10 0x7fd2940c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.782+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 >> 172.21.15.26:0/3807151799 conn(0x7fd29406b970 msgr2=0x7fd2941521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.782+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 shutdown_connections 2024-03-06T07:22:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.782+0000 7fd298ca3640 1 -- 172.21.15.26:0/3807151799 wait complete. 2024-03-06T07:22:54.785 INFO:tasks.ceph.mds.b.smithi117.stderr: -470> 2024-03-06T07:22:40.308+0000 7f7f9c337640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:54.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:54.806 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:22:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.895+0000 7fbba7bbd640 1 Processor -- start 2024-03-06T07:22:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.896+0000 7fbba7bbd640 1 -- start start 2024-03-06T07:22:54.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba7bbd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac4b0 0x7fbba0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba7bbd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbba0151d70 0x7fbba015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba7bbd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba0072f10 con 0x7fbba00ac890 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba7bbd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba0154c50 con 0x7fbba0151d70 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba7bbd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba01554f0 con 0x7fbba00ac4b0 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba5158640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbba00ac890 0x7fbba0151660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57542/0 (socket says 172.21.15.26:57542) 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba615a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbba0151d70 0x7fbba015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba5158640 1 -- 172.21.15.26:0/3098215225 learned_addr learned my addr 172.21.15.26:0/3098215225 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba5959640 1 --2- 172.21.15.26:0/3098215225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac4b0 0x7fbba0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba4957640 1 -- 172.21.15.26:0/3098215225 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383291836 0 0) 0x7fbba0072f10 con 0x7fbba00ac890 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba4957640 1 -- 172.21.15.26:0/3098215225 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb8c003520 con 0x7fbba00ac890 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba615a640 1 -- 172.21.15.26:0/3098215225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac4b0 msgr2=0x7fbba0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba615a640 1 --2- 172.21.15.26:0/3098215225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac4b0 0x7fbba0070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba615a640 1 -- 172.21.15.26:0/3098215225 >> v1:172.21.15.26:6789/0 conn(0x7fbba00ac890 legacy=0x7fbba0151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba615a640 1 -- 172.21.15.26:0/3098215225 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0154ea0 con 0x7fbba0151d70 2024-03-06T07:22:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.897+0000 7fbba5959640 1 --2- 172.21.15.26:0/3098215225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac4b0 0x7fbba0070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.898+0000 7fbba615a640 1 --2- 172.21.15.26:0/3098215225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba015bfe0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009d90 tx=0x7fbb9802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df0b76b3573e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.898+0000 7fbba4957640 1 -- 172.21.15.26:0/3098215225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98039070 con 0x7fbba0151d70 2024-03-06T07:22:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.898+0000 7fbba4957640 1 -- 172.21.15.26:0/3098215225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9802cbc0 con 0x7fbba0151d70 2024-03-06T07:22:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.898+0000 7fbba4957640 1 -- 172.21.15.26:0/3098215225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98034440 con 0x7fbba0151d70 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3098215225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 msgr2=0x7fbba015bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3098215225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba015bfe0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009d90 tx=0x7fbb9802be10 comp rx=0 tx=0).stop 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3098215225 shutdown_connections 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3098215225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba015bfe0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3098215225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac4b0 0x7fbba0070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3098215225 >> 172.21.15.26:0/3098215225 conn(0x7fbba00c3ad0 msgr2=0x7fbba00c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3098215225 shutdown_connections 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.899+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3098215225 wait complete. 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.900+0000 7fbba7bbd640 1 Processor -- start 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.900+0000 7fbba7bbd640 1 -- start start 2024-03-06T07:22:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba7bbd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba00ac4b0 0x7fbba0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba7bbd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac890 0x7fbba0147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba7bbd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba01475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba7bbd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba00c3680 con 0x7fbba0151d70 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba7bbd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba00c3500 con 0x7fbba00ac4b0 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba7bbd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba0155980 con 0x7fbba00ac890 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba01475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba01475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49286/0 (socket says 172.21.15.26:49286) 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 -- 172.21.15.26:0/3881139649 learned_addr learned my addr 172.21.15.26:0/3881139649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba5158640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac890 0x7fbba0147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba5959640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba00ac4b0 0x7fbba0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 -- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac890 msgr2=0x7fbba0147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac890 0x7fbba0147070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 -- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba00ac4b0 msgr2=0x7fbba0150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba00ac4b0 0x7fbba0150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 -- 172.21.15.26:0/3881139649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba00c16b0 con 0x7fbba0151d70 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba5959640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba00ac4b0 0x7fbba0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbba615a640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba01475b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009d90 tx=0x7fbb98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbb967fc640 1 -- 172.21.15.26:0/3881139649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98009450 con 0x7fbba0151d70 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.901+0000 7fbb967fc640 1 -- 172.21.15.26:0/3881139649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9803b610 con 0x7fbba0151d70 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.902+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba00be000 con 0x7fbba0151d70 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.902+0000 7fbb967fc640 1 -- 172.21.15.26:0/3881139649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9803ba00 con 0x7fbba0151d70 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.902+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba01e7d50 con 0x7fbba0151d70 2024-03-06T07:22:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.902+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba01e7860 con 0x7fbba0151d70 2024-03-06T07:22:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.903+0000 7fbb967fc640 1 -- 172.21.15.26:0/3881139649 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb9803bba0 con 0x7fbba0151d70 2024-03-06T07:22:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.903+0000 7fbb967fc640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8406b4d0 0x7fbb8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.904+0000 7fbb967fc640 1 -- 172.21.15.26:0/3881139649 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fbb980af220 con 0x7fbba0151d70 2024-03-06T07:22:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.904+0000 7fbba5959640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8406b4d0 0x7fbb8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.904+0000 7fbba5959640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8406b4d0 0x7fbb8406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006fd0 tx=0x7fbb9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:54.908+0000 7fbb967fc640 1 -- 172.21.15.26:0/3881139649 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb9807dbf0 con 0x7fbba0151d70 2024-03-06T07:22:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.149+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbba004f330 con 0x7fbba0151d70 2024-03-06T07:22:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.150+0000 7fbb967fc640 1 -- 172.21.15.26:0/3881139649 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5453 (secure 0 0 0) 0x7fbb9807d4f0 con 0x7fbba0151d70 2024-03-06T07:22:55.152 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:55.152 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":60,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:54.778251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:22:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 60 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8406b4d0 msgr2=0x7fbb8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8406b4d0 0x7fbb8406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006fd0 tx=0x7fbb9c008040 comp rx=0 tx=0).stop 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 msgr2=0x7fbba01475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba01475b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009d90 tx=0x7fbb98004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 shutdown_connections 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb8406b4d0 0x7fbb8406d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbba0151d70 0x7fbba01475b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbba00ac890 0x7fbba0147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 --2- 172.21.15.26:0/3881139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba00ac4b0 0x7fbba0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 >> 172.21.15.26:0/3881139649 conn(0x7fbba00c3ad0 msgr2=0x7fbba0070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:55.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 shutdown_connections 2024-03-06T07:22:55.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.152+0000 7fbba7bbd640 1 -- 172.21.15.26:0/3881139649 wait complete. 2024-03-06T07:22:55.167 INFO:tasks.cephfs.test_failover:Waiting for former active to reclaim its spot 2024-03-06T07:22:55.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-03-06T07:22:55.275 INFO:teuthology.orchestra.run.smithi026.stdout:{"mds_beacon_grace":"15.000000"} 2024-03-06T07:22:55.276 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-03-06T07:22:55.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:55.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.377+0000 7fe8303d7640 1 Processor -- start 2024-03-06T07:22:55.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.377+0000 7fe8303d7640 1 -- start start 2024-03-06T07:22:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.378+0000 7fe8303d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe828155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.378+0000 7fe8303d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe828158b60 0x7fe828158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.378+0000 7fe8303d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8280c73d0 con 0x7fe82814f360 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.378+0000 7fe8303d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8280c8dd0 con 0x7fe828158b60 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.378+0000 7fe8303d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280be2e0 con 0x7fe8280c88c0 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82e173640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe82814f360 0x7fe8280c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57544/0 (socket says 172.21.15.26:57544) 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82e974640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe828158b60 0x7fe828158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82e173640 1 -- 172.21.15.26:0/4262007709 learned_addr learned my addr 172.21.15.26:0/4262007709 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82d972640 1 --2- 172.21.15.26:0/4262007709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe828155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82e974640 1 -- 172.21.15.26:0/4262007709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 msgr2=0x7fe828155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82e974640 1 --2- 172.21.15.26:0/4262007709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe828155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82e974640 1 -- 172.21.15.26:0/4262007709 >> v1:172.21.15.26:6789/0 conn(0x7fe82814f360 legacy=0x7fe8280c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82d171640 1 -- 172.21.15.26:0/4262007709 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018889925 0 0) 0x7fe8280c73d0 con 0x7fe82814f360 2024-03-06T07:22:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.379+0000 7fe82e974640 1 -- 172.21.15.26:0/4262007709 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280c7670 con 0x7fe828158b60 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe82e974640 1 --2- 172.21.15.26:0/4262007709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe828158ff0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe820009d90 tx=0x7fe82002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bb806832ec0fbe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe82d171640 1 -- 172.21.15.26:0/4262007709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820039070 con 0x7fe828158b60 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe82d171640 1 -- 172.21.15.26:0/4262007709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82002cbc0 con 0x7fe828158b60 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe82d171640 1 -- 172.21.15.26:0/4262007709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820034440 con 0x7fe828158b60 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 -- 172.21.15.26:0/4262007709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 msgr2=0x7fe828158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 --2- 172.21.15.26:0/4262007709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe828158ff0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe820009d90 tx=0x7fe82002be10 comp rx=0 tx=0).stop 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 -- 172.21.15.26:0/4262007709 shutdown_connections 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 --2- 172.21.15.26:0/4262007709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe828158ff0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 --2- 172.21.15.26:0/4262007709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe828155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 -- 172.21.15.26:0/4262007709 >> 172.21.15.26:0/4262007709 conn(0x7fe8280ac570 msgr2=0x7fe8280ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:55.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 -- 172.21.15.26:0/4262007709 shutdown_connections 2024-03-06T07:22:55.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.380+0000 7fe8303d7640 1 -- 172.21.15.26:0/4262007709 wait complete. 2024-03-06T07:22:55.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.381+0000 7fe8303d7640 1 Processor -- start 2024-03-06T07:22:55.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe8303d7640 1 -- start start 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe8303d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe8281ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe8303d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe82814f360 0x7fe8280bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe8303d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe82814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe8303d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280c5a50 con 0x7fe828158b60 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe8303d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280c7c30 con 0x7fe82814f360 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe8303d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280c5bd0 con 0x7fe8280c88c0 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe82e974640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe82814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe82e974640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe82814b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49302/0 (socket says 172.21.15.26:49302) 2024-03-06T07:22:55.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe82e974640 1 -- 172.21.15.26:0/978383866 learned_addr learned my addr 172.21.15.26:0/978383866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe82d972640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe82814f360 0x7fe8280bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.382+0000 7fe82e173640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe8281ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe82e974640 1 -- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 msgr2=0x7fe8281ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe82e974640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe8281ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe82e974640 1 -- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe82814f360 msgr2=0x7fe8280bf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe82e974640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe82814f360 0x7fe8280bf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe82e974640 1 -- 172.21.15.26:0/978383866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8281ec610 con 0x7fe828158b60 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe82e974640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe82814b790 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe820037040 tx=0x7fe820004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe81affd640 1 -- 172.21.15.26:0/978383866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820039050 con 0x7fe828158b60 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe81affd640 1 -- 172.21.15.26:0/978383866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe820045070 con 0x7fe828158b60 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe81affd640 1 -- 172.21.15.26:0/978383866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820009450 con 0x7fe828158b60 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8280bfce0 con 0x7fe828158b60 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.383+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8280c0ca0 con 0x7fe828158b60 2024-03-06T07:22:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.384+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82804f330 con 0x7fe828158b60 2024-03-06T07:22:55.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.384+0000 7fe81affd640 1 -- 172.21.15.26:0/978383866 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe82003e850 con 0x7fe828158b60 2024-03-06T07:22:55.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.385+0000 7fe81affd640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80c06b500 0x7fe80c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.385+0000 7fe81affd640 1 -- 172.21.15.26:0/978383866 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fe8200accd0 con 0x7fe828158b60 2024-03-06T07:22:55.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.385+0000 7fe82e173640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80c06b500 0x7fe80c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.386+0000 7fe82e173640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80c06b500 0x7fe80c06d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe824007fd0 tx=0x7fe824007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.390+0000 7fe81affd640 1 -- 172.21.15.26:0/978383866 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe82004e020 con 0x7fe828158b60 2024-03-06T07:22:55.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.610+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe828155f20 con 0x7fe828158b60 2024-03-06T07:22:55.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.611+0000 7fe81affd640 1 -- 172.21.15.26:0/978383866 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5453 (secure 0 0 0) 0x7fe82007e560 con 0x7fe828158b60 2024-03-06T07:22:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:22:55.612 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":60,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:54.778251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:22:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 60 2024-03-06T07:22:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80c06b500 msgr2=0x7fe80c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80c06b500 0x7fe80c06d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe824007fd0 tx=0x7fe824007de0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 msgr2=0x7fe82814b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe82814b790 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe820037040 tx=0x7fe820004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 shutdown_connections 2024-03-06T07:22:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80c06b500 0x7fe80c06d9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe828158b60 0x7fe82814b790 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe82814f360 0x7fe8280bf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 --2- 172.21.15.26:0/978383866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8280c88c0 0x7fe8281ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 >> 172.21.15.26:0/978383866 conn(0x7fe8280ac570 msgr2=0x7fe8280ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.612+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 shutdown_connections 2024-03-06T07:22:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:22:55.613+0000 7fe8303d7640 1 -- 172.21.15.26:0/978383866 wait complete. 2024-03-06T07:22:55.625 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-03-06T07:23:00.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.752+0000 7fa62210c640 1 Processor -- start 2024-03-06T07:23:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.752+0000 7fa62210c640 1 -- start start 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.752+0000 7fa62210c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c14f360 0x7fa61c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.752+0000 7fa62210c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c0c73d0 con 0x7fa61c158b60 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa62210c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c0be2e0 con 0x7fa61c0c88c0 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa62210c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c159700 con 0x7fa61c14f360 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa61c158b60 0x7fa61c158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51940/0 (socket says 172.21.15.26:51940) 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61bfff640 1 -- 172.21.15.26:0/200708135 learned_addr learned my addr 172.21.15.26:0/200708135 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61b7fe640 1 --2- 172.21.15.26:0/200708135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c14f360 0x7fa61c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61a7fc640 1 -- 172.21.15.26:0/200708135 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788458060 0 0) 0x7fa61c0c73d0 con 0x7fa61c158b60 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61a7fc640 1 -- 172.21.15.26:0/200708135 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa604003520 con 0x7fa61c158b60 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61a7fc640 1 -- 172.21.15.26:0/200708135 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897348797 0 0) 0x7fa61c0be2e0 con 0x7fa61c0c88c0 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61a7fc640 1 -- 172.21.15.26:0/200708135 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa61c0c73d0 con 0x7fa61c0c88c0 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61b7fe640 1 -- 172.21.15.26:0/200708135 >> v1:172.21.15.117:6789/0 conn(0x7fa61c0c88c0 legacy=0x7fa61c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61b7fe640 1 -- 172.21.15.26:0/200708135 >> v1:172.21.15.26:6789/0 conn(0x7fa61c158b60 legacy=0x7fa61c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.753+0000 7fa61b7fe640 1 -- 172.21.15.26:0/200708135 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0c7670 con 0x7fa61c14f360 2024-03-06T07:23:00.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.754+0000 7fa61b7fe640 1 --2- 172.21.15.26:0/200708135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c14f360 0x7fa61c155c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0097d0 tx=0x7fa60c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7539d0427849fb36 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:00.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.754+0000 7fa61a7fc640 1 -- 172.21.15.26:0/200708135 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c035020 con 0x7fa61c14f360 2024-03-06T07:23:00.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.754+0000 7fa61a7fc640 1 -- 172.21.15.26:0/200708135 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c031800 con 0x7fa61c14f360 2024-03-06T07:23:00.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.754+0000 7fa61a7fc640 1 -- 172.21.15.26:0/200708135 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c031bb0 con 0x7fa61c14f360 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.755+0000 7fa62210c640 1 -- 172.21.15.26:0/200708135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c14f360 msgr2=0x7fa61c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.755+0000 7fa62210c640 1 --2- 172.21.15.26:0/200708135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c14f360 0x7fa61c155c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0097d0 tx=0x7fa60c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.755+0000 7fa62210c640 1 -- 172.21.15.26:0/200708135 shutdown_connections 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.755+0000 7fa62210c640 1 --2- 172.21.15.26:0/200708135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c14f360 0x7fa61c155c10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.755+0000 7fa62210c640 1 -- 172.21.15.26:0/200708135 >> 172.21.15.26:0/200708135 conn(0x7fa61c0ac570 msgr2=0x7fa61c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.755+0000 7fa62210c640 1 -- 172.21.15.26:0/200708135 shutdown_connections 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.755+0000 7fa62210c640 1 -- 172.21.15.26:0/200708135 wait complete. 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.756+0000 7fa62210c640 1 Processor -- start 2024-03-06T07:23:00.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.756+0000 7fa62210c640 1 -- start start 2024-03-06T07:23:00.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa62210c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa61c0c88c0 0x7fa61c0c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa62210c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c14f360 0x7fa61c150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa62210c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c158b60 0x7fa61c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa62210c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0c7c30 con 0x7fa61c14f360 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa62210c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0c5a50 con 0x7fa61c0c88c0 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa62210c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0c5bd0 con 0x7fa61c158b60 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c14f360 0x7fa61c150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c14f360 0x7fa61c150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39172/0 (socket says 172.21.15.26:39172) 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61affd640 1 -- 172.21.15.26:0/1079386669 learned_addr learned my addr 172.21.15.26:0/1079386669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61b7fe640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa61c0c88c0 0x7fa61c0c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61bfff640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c158b60 0x7fa61c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61b7fe640 1 -- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c158b60 msgr2=0x7fa61c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61b7fe640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c158b60 0x7fa61c157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61b7fe640 1 -- 172.21.15.26:0/1079386669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c14f360 msgr2=0x7fa61c150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61b7fe640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c14f360 0x7fa61c150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61b7fe640 1 -- 172.21.15.26:0/1079386669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c1eed80 con 0x7fa61c0c88c0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.757+0000 7fa61affd640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c14f360 0x7fa61c150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.758+0000 7fa61b7fe640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa61c0c88c0 0x7fa61c0c1b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa60c033000 tx=0x7fa60c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.758+0000 7fa603fff640 1 -- 172.21.15.26:0/1079386669 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c041070 con 0x7fa61c0c88c0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.758+0000 7fa603fff640 1 -- 172.21.15.26:0/1079386669 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c03ad00 con 0x7fa61c0c88c0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.758+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c1eeaa0 con 0x7fa61c0c88c0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.758+0000 7fa603fff640 1 -- 172.21.15.26:0/1079386669 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c035040 con 0x7fa61c0c88c0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.758+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c157d80 con 0x7fa61c0c88c0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.759+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c04f330 con 0x7fa61c0c88c0 2024-03-06T07:23:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.759+0000 7fa603fff640 1 -- 172.21.15.26:0/1079386669 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa60c038020 con 0x7fa61c0c88c0 2024-03-06T07:23:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.760+0000 7fa603fff640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa60806b4d0 0x7fa60806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.760+0000 7fa603fff640 1 -- 172.21.15.26:0/1079386669 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa60c0ac0c0 con 0x7fa61c0c88c0 2024-03-06T07:23:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.760+0000 7fa61affd640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa60806b4d0 0x7fa60806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.760+0000 7fa61affd640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa60806b4d0 0x7fa60806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa61c150020 tx=0x7fa610009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.764+0000 7fa603fff640 1 -- 172.21.15.26:0/1079386669 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa60c07a3b0 con 0x7fa61c0c88c0 2024-03-06T07:23:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.991+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa61c0c8d50 con 0x7fa61c0c88c0 2024-03-06T07:23:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.992+0000 7fa603fff640 1 -- 172.21.15.26:0/1079386669 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6260 (secure 0 0 0) 0x7fa60c07e450 con 0x7fa61c0c88c0 2024-03-06T07:23:00.993 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:00.993 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:58.795121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6457":{"gid":6457,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6836/252560219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":252560219},{"type":"v1","addr":"172.21.15.117:6836","nonce":252560219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6457,"qdb_cluster":[6457]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:23:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2024-03-06T07:23:00.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa60806b4d0 msgr2=0x7fa60806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa60806b4d0 0x7fa60806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa61c150020 tx=0x7fa610009040 comp rx=0 tx=0).stop 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa61c0c88c0 msgr2=0x7fa61c0c1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa61c0c88c0 0x7fa61c0c1b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa60c033000 tx=0x7fa60c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 shutdown_connections 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa60806b4d0 0x7fa60806d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61c158b60 0x7fa61c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c14f360 0x7fa61c150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 --2- 172.21.15.26:0/1079386669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa61c0c88c0 0x7fa61c0c1b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 >> 172.21.15.26:0/1079386669 conn(0x7fa61c0ac570 msgr2=0x7fa61c154340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 shutdown_connections 2024-03-06T07:23:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:00.994+0000 7fa62210c640 1 -- 172.21.15.26:0/1079386669 wait complete. 2024-03-06T07:23:01.006 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:23:01.007 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:01.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:01.042 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:01.094 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:01.094 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:23:01.094 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:01.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:01.113 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:23:01.113 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:01.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:01.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.283+0000 7f9941cdf640 1 Processor -- start 2024-03-06T07:23:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.283+0000 7f9941cdf640 1 -- start start 2024-03-06T07:23:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f9941cdf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f993c0c88c0 0x7f993c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f9941cdf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f993c158b60 0x7f993c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f9941cdf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0c73d0 con 0x7f993c14f360 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f9941cdf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c0c8dd0 con 0x7f993c0c88c0 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f9941cdf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c0be2e0 con 0x7f993c158b60 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f993c0c88c0 0x7f993c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f993c0c88c0 0x7f993c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39174/0 (socket says 172.21.15.26:39174) 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993affd640 1 -- 172.21.15.26:0/1501048373 learned_addr learned my addr 172.21.15.26:0/1501048373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993bfff640 1 --2- 172.21.15.26:0/1501048373 >> v2:172.21.15.117:3300/0 conn(0x7f993c158b60 0x7f993c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993affd640 1 -- 172.21.15.26:0/1501048373 >> v2:172.21.15.117:3300/0 conn(0x7f993c158b60 msgr2=0x7f993c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993affd640 1 --2- 172.21.15.26:0/1501048373 >> v2:172.21.15.117:3300/0 conn(0x7f993c158b60 0x7f993c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993affd640 1 -- 172.21.15.26:0/1501048373 >> v1:172.21.15.117:6789/0 conn(0x7f993c14f360 legacy=0x7f993c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.284+0000 7f993affd640 1 -- 172.21.15.26:0/1501048373 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0c7670 con 0x7f993c0c88c0 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.285+0000 7f993affd640 1 --2- 172.21.15.26:0/1501048373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c0c88c0 0x7f993c155c10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f993c152d60 tx=0x7f992402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74519255dd780e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.285+0000 7f993a7fc640 1 -- 172.21.15.26:0/1501048373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992402ca80 con 0x7f993c0c88c0 2024-03-06T07:23:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.285+0000 7f993a7fc640 1 -- 172.21.15.26:0/1501048373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992402cc20 con 0x7f993c0c88c0 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.285+0000 7f993a7fc640 1 -- 172.21.15.26:0/1501048373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240343f0 con 0x7f993c0c88c0 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 -- 172.21.15.26:0/1501048373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c0c88c0 msgr2=0x7f993c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 --2- 172.21.15.26:0/1501048373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c0c88c0 0x7f993c155c10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f993c152d60 tx=0x7f992402bee0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 -- 172.21.15.26:0/1501048373 shutdown_connections 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 --2- 172.21.15.26:0/1501048373 >> v2:172.21.15.117:3300/0 conn(0x7f993c158b60 0x7f993c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 --2- 172.21.15.26:0/1501048373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c0c88c0 0x7f993c155c10 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 -- 172.21.15.26:0/1501048373 >> 172.21.15.26:0/1501048373 conn(0x7f993c0ac570 msgr2=0x7f993c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 -- 172.21.15.26:0/1501048373 shutdown_connections 2024-03-06T07:23:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.286+0000 7f9941cdf640 1 -- 172.21.15.26:0/1501048373 wait complete. 2024-03-06T07:23:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.288+0000 7f9941cdf640 1 Processor -- start 2024-03-06T07:23:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.288+0000 7f9941cdf640 1 -- start start 2024-03-06T07:23:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f9941cdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993c0c88c0 0x7f993c0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f9941cdf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c14f360 0x7f993c14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f9941cdf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f993c158b60 0x7f993c14ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f9941cdf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0c5a50 con 0x7f993c14f360 2024-03-06T07:23:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f9941cdf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0c7c30 con 0x7f993c0c88c0 2024-03-06T07:23:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f9941cdf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0c5bd0 con 0x7f993c158b60 2024-03-06T07:23:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c14f360 0x7f993c14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c14f360 0x7f993c14ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39186/0 (socket says 172.21.15.26:39186) 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 -- 172.21.15.26:0/2098872103 learned_addr learned my addr 172.21.15.26:0/2098872103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993b7fe640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993c0c88c0 0x7f993c0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993bfff640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f993c158b60 0x7f993c14ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 -- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f993c158b60 msgr2=0x7f993c14ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f993c158b60 0x7f993c14ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 -- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993c0c88c0 msgr2=0x7f993c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993c0c88c0 0x7f993c0af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.289+0000 7f993affd640 1 -- 172.21.15.26:0/2098872103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c1eb1b0 con 0x7f993c14f360 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.290+0000 7f993affd640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c14f360 0x7f993c14ba40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9924036040 tx=0x7f9924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.290+0000 7f9933fff640 1 -- 172.21.15.26:0/2098872103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924043020 con 0x7f993c14f360 2024-03-06T07:23:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.290+0000 7f9933fff640 1 -- 172.21.15.26:0/2098872103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992403e450 con 0x7f993c14f360 2024-03-06T07:23:01.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.290+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c1eb490 con 0x7f993c14f360 2024-03-06T07:23:01.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.290+0000 7f9933fff640 1 -- 172.21.15.26:0/2098872103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924009510 con 0x7f993c14f360 2024-03-06T07:23:01.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.290+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c1eba50 con 0x7f993c14f360 2024-03-06T07:23:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.291+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c04f330 con 0x7f993c14f360 2024-03-06T07:23:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.291+0000 7f9933fff640 1 -- 172.21.15.26:0/2098872103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9924046020 con 0x7f993c14f360 2024-03-06T07:23:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.292+0000 7f9933fff640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f991c06b4d0 0x7f991c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.292+0000 7f9933fff640 1 -- 172.21.15.26:0/2098872103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f99240acd70 con 0x7f993c14f360 2024-03-06T07:23:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.292+0000 7f993b7fe640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f991c06b4d0 0x7f991c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.292+0000 7f993b7fe640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f991c06b4d0 0x7f991c06d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f992c0016e0 tx=0x7f992c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.296+0000 7f9933fff640 1 -- 172.21.15.26:0/2098872103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f992404d030 con 0x7f993c14f360 2024-03-06T07:23:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.531+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f993c0cb400 con 0x7f993c14f360 2024-03-06T07:23:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.532+0000 7f9933fff640 1 -- 172.21.15.26:0/2098872103 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6260 (secure 0 0 0) 0x7f992407b070 con 0x7f993c14f360 2024-03-06T07:23:01.534 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:01.534 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:58.795121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6457":{"gid":6457,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6836/252560219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":252560219},{"type":"v1","addr":"172.21.15.117:6836","nonce":252560219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6457,"qdb_cluster":[6457]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:23:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2024-03-06T07:23:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f991c06b4d0 msgr2=0x7f991c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f991c06b4d0 0x7f991c06d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f992c0016e0 tx=0x7f992c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c14f360 msgr2=0x7f993c14ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c14f360 0x7f993c14ba40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9924036040 tx=0x7f9924004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 shutdown_connections 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f991c06b4d0 0x7f991c06d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f993c158b60 0x7f993c14ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993c14f360 0x7f993c14ba40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 --2- 172.21.15.26:0/2098872103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993c0c88c0 0x7f993c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.534+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 >> 172.21.15.26:0/2098872103 conn(0x7f993c0ac570 msgr2=0x7f993c0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.535+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 shutdown_connections 2024-03-06T07:23:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.535+0000 7f9941cdf640 1 -- 172.21.15.26:0/2098872103 wait complete. 2024-03-06T07:23:01.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.647+0000 7f454ddac640 1 Processor -- start 2024-03-06T07:23:01.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.647+0000 7f454ddac640 1 -- start start 2024-03-06T07:23:01.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f454ddac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45480c5480 0x7f4548155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f454ddac640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45480c5860 0x7f454806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f454ddac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45481586f0 0x7f4548158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f454ddac640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454806df40 con 0x7f45480c5860 2024-03-06T07:23:01.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f454ddac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480c5cc0 con 0x7f45480c5480 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f454ddac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45481591c0 con 0x7f45481586f0 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45480c5860 0x7f454806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45480c5860 0x7f454806f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39190/0 (socket says 172.21.15.26:39190) 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 -- 172.21.15.26:0/76326242 learned_addr learned my addr 172.21.15.26:0/76326242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f45477fe640 1 --2- 172.21.15.26:0/76326242 >> v2:172.21.15.117:3300/0 conn(0x7f45480c5480 0x7f4548155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4547fff640 1 --2- 172.21.15.26:0/76326242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45481586f0 0x7f4548158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 -- 172.21.15.26:0/76326242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45481586f0 msgr2=0x7f4548158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 --2- 172.21.15.26:0/76326242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45481586f0 0x7f4548158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 -- 172.21.15.26:0/76326242 >> v2:172.21.15.117:3300/0 conn(0x7f45480c5480 msgr2=0x7f4548155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 --2- 172.21.15.26:0/76326242 >> v2:172.21.15.117:3300/0 conn(0x7f45480c5480 0x7f4548155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.648+0000 7f4546ffd640 1 -- 172.21.15.26:0/76326242 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45481594a0 con 0x7f45480c5860 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.649+0000 7f4546ffd640 1 --2- 172.21.15.26:0/76326242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5860 0x7f454806f6a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f45480c1cf0 tx=0x7f453c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bbd51f7a5d20ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.649+0000 7f45467fc640 1 -- 172.21.15.26:0/76326242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c02e070 con 0x7f45480c5860 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.649+0000 7f45467fc640 1 -- 172.21.15.26:0/76326242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c037070 con 0x7f45480c5860 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.649+0000 7f45467fc640 1 -- 172.21.15.26:0/76326242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c031a00 con 0x7f45480c5860 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.649+0000 7f454ddac640 1 -- 172.21.15.26:0/76326242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5860 msgr2=0x7f454806f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.649+0000 7f454ddac640 1 --2- 172.21.15.26:0/76326242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5860 0x7f454806f6a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f45480c1cf0 tx=0x7f453c02b880 comp rx=0 tx=0).stop 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.650+0000 7f454ddac640 1 -- 172.21.15.26:0/76326242 shutdown_connections 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.650+0000 7f454ddac640 1 --2- 172.21.15.26:0/76326242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45481586f0 0x7f4548158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.650+0000 7f454ddac640 1 --2- 172.21.15.26:0/76326242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5860 0x7f454806f6a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.650+0000 7f454ddac640 1 --2- 172.21.15.26:0/76326242 >> v2:172.21.15.117:3300/0 conn(0x7f45480c5480 0x7f4548155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.650+0000 7f454ddac640 1 -- 172.21.15.26:0/76326242 >> 172.21.15.26:0/76326242 conn(0x7f454806b1a0 msgr2=0x7f454806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.650+0000 7f454ddac640 1 -- 172.21.15.26:0/76326242 shutdown_connections 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.650+0000 7f454ddac640 1 -- 172.21.15.26:0/76326242 wait complete. 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 Processor -- start 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 -- start start 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5480 0x7f4548143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480c5860 0x7f45481ee680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45481586f0 0x7f45481eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548152a20 con 0x7f45480c5480 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548155680 con 0x7f45481586f0 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.651+0000 7f454ddac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4548155050 con 0x7f45480c5860 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5480 0x7f4548143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5480 0x7f4548143d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39200/0 (socket says 172.21.15.26:39200) 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 -- 172.21.15.26:0/835604843 learned_addr learned my addr 172.21.15.26:0/835604843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f4546ffd640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480c5860 0x7f45481ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f4547fff640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45481586f0 0x7f45481eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 -- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480c5860 msgr2=0x7f45481ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480c5860 0x7f45481ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 -- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45481586f0 msgr2=0x7f45481eebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45481586f0 0x7f45481eebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 -- 172.21.15.26:0/835604843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45481ef5c0 con 0x7f45480c5480 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f45477fe640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5480 0x7f4548143d80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f453800e8a0 tx=0x7f453800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f452ffff640 1 -- 172.21.15.26:0/835604843 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538017070 con 0x7f45480c5480 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f452ffff640 1 -- 172.21.15.26:0/835604843 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453800c940 con 0x7f45480c5480 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45481442c0 con 0x7f45480c5480 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f452ffff640 1 -- 172.21.15.26:0/835604843 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453800cd30 con 0x7f45480c5480 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.652+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45481ef2e0 con 0x7f45480c5480 2024-03-06T07:23:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.653+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4548156e50 con 0x7f45480c5480 2024-03-06T07:23:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.654+0000 7f452ffff640 1 -- 172.21.15.26:0/835604843 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f453801a070 con 0x7f45480c5480 2024-03-06T07:23:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.654+0000 7f452ffff640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f453406b500 0x7f453406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.654+0000 7f452ffff640 1 -- 172.21.15.26:0/835604843 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f453808a5c0 con 0x7f45480c5480 2024-03-06T07:23:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.655+0000 7f4546ffd640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f453406b500 0x7f453406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.655+0000 7f4546ffd640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f453406b500 0x7f453406d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f453c036000 tx=0x7f453c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.659+0000 7f452ffff640 1 -- 172.21.15.26:0/835604843 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f45481442c0 con 0x7f45480c5480 2024-03-06T07:23:01.789 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:23:01.787+0000 7fc38e93e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.841+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f454804f330 con 0x7f45480c5480 2024-03-06T07:23:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.842+0000 7f452ffff640 1 -- 172.21.15.26:0/835604843 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+20810 (secure 0 0 0) 0x7f4538058820 con 0x7f45480c5480 2024-03-06T07:23:01.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:01.844 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:54.778055+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/218606650":"2024-03-07T07:22:54.778025+0000","172.21.15.117:6834/218606650":"2024-03-07T07:22:54.778025+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f453406b500 msgr2=0x7f453406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f453406b500 0x7f453406d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f453c036000 tx=0x7f453c03d040 comp rx=0 tx=0).stop 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5480 msgr2=0x7f4548143d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5480 0x7f4548143d80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f453800e8a0 tx=0x7f453800ed60 comp rx=0 tx=0).stop 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 shutdown_connections 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f453406b500 0x7f453406d9c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45481586f0 0x7f45481eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480c5860 0x7f45481ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 --2- 172.21.15.26:0/835604843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480c5480 0x7f4548143d80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 >> 172.21.15.26:0/835604843 conn(0x7f454806b1a0 msgr2=0x7f454806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 shutdown_connections 2024-03-06T07:23:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.844+0000 7f454ddac640 1 -- 172.21.15.26:0/835604843 wait complete. 2024-03-06T07:23:01.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:22:32.646079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:22:33.350567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:22:42.855764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:22:43.690413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:01.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.964+0000 7fdd01f13640 1 Processor -- start 2024-03-06T07:23:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.964+0000 7fdd01f13640 1 -- start start 2024-03-06T07:23:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdd01f13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc0c67f0 0x7fdcfc155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdd01f13640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcfc0c4ef0 0x7fdcfc155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdd01f13640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc152cb0 con 0x7fdcfc1528d0 2024-03-06T07:23:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdd01f13640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc155ca0 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdd01f13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0c38b0 con 0x7fdcfc0c67f0 2024-03-06T07:23:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcfc0c4ef0 0x7fdcfc155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcfc0c4ef0 0x7fdcfc155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39208/0 (socket says 172.21.15.26:39208) 2024-03-06T07:23:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfbfff640 1 -- 172.21.15.26:0/2210647318 learned_addr learned my addr 172.21.15.26:0/2210647318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfaffd640 1 --2- 172.21.15.26:0/2210647318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc0c67f0 0x7fdcfc155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfbfff640 1 -- 172.21.15.26:0/2210647318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc0c67f0 msgr2=0x7fdcfc155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfbfff640 1 --2- 172.21.15.26:0/2210647318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc0c67f0 0x7fdcfc155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfbfff640 1 -- 172.21.15.26:0/2210647318 >> v1:172.21.15.117:6789/0 conn(0x7fdcfc1528d0 legacy=0x7fdcfc0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.965+0000 7fdcfbfff640 1 -- 172.21.15.26:0/2210647318 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc1594f0 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.966+0000 7fdcfbfff640 1 --2- 172.21.15.26:0/2210647318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc155760 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009e50 tx=0x7fdcf402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3a1f6090dcc935 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.966+0000 7fdcfa7fc640 1 -- 172.21.15.26:0/2210647318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf402cb90 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.966+0000 7fdcfa7fc640 1 -- 172.21.15.26:0/2210647318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf402cd30 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.966+0000 7fdcfa7fc640 1 -- 172.21.15.26:0/2210647318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4034610 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 -- 172.21.15.26:0/2210647318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 msgr2=0x7fdcfc155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 --2- 172.21.15.26:0/2210647318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc155760 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009e50 tx=0x7fdcf402bff0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 -- 172.21.15.26:0/2210647318 shutdown_connections 2024-03-06T07:23:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 --2- 172.21.15.26:0/2210647318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc155760 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 --2- 172.21.15.26:0/2210647318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc0c67f0 0x7fdcfc155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 -- 172.21.15.26:0/2210647318 >> 172.21.15.26:0/2210647318 conn(0x7fdcfc0aa6a0 msgr2=0x7fdcfc0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 -- 172.21.15.26:0/2210647318 shutdown_connections 2024-03-06T07:23:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.967+0000 7fdd01f13640 1 -- 172.21.15.26:0/2210647318 wait complete. 2024-03-06T07:23:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.968+0000 7fdd01f13640 1 Processor -- start 2024-03-06T07:23:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.968+0000 7fdd01f13640 1 -- start start 2024-03-06T07:23:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc0c67f0 0x7fdcfc1edd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc1528d0 0x7fdcfc1ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0a8250 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0c3390 con 0x7fdcfc0c67f0 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc1591a0 con 0x7fdcfc1528d0 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc1547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39222/0 (socket says 172.21.15.26:39222) 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 -- 172.21.15.26:0/1525057233 learned_addr learned my addr 172.21.15.26:0/1525057233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfbfff640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc1528d0 0x7fdcfc1ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfaffd640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc0c67f0 0x7fdcfc1edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 -- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc1528d0 msgr2=0x7fdcfc1ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc1528d0 0x7fdcfc1ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 -- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc0c67f0 msgr2=0x7fdcfc1edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc0c67f0 0x7fdcfc1edd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 -- 172.21.15.26:0/1525057233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc1ee7a0 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcfb7fe640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc1547a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c430 tx=0x7fdce400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcf3fff640 1 -- 172.21.15.26:0/1525057233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce40052d0 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcf3fff640 1 -- 172.21.15.26:0/1525057233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce4005490 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdcf3fff640 1 -- 172.21.15.26:0/1525057233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4007260 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc155f20 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.969+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc14cdc0 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.971+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc04f330 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.971+0000 7fdcf3fff640 1 -- 172.21.15.26:0/1525057233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdce4007400 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.971+0000 7fdcf3fff640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b500 0x7fdcdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.971+0000 7fdcf3fff640 1 -- 172.21.15.26:0/1525057233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fdce4089af0 con 0x7fdcfc0c4ef0 2024-03-06T07:23:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.971+0000 7fdcfaffd640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b500 0x7fdcdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.972+0000 7fdcfaffd640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b500 0x7fdcdc06d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0a7f40 tx=0x7fdcec009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:01.976+0000 7fdcf3fff640 1 -- 172.21.15.26:0/1525057233 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdce4057e70 con 0x7fdcfc0c4ef0 2024-03-06T07:23:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.155+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcfc0c4a80 con 0x7fdcfc0c4ef0 2024-03-06T07:23:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.155+0000 7fdcf3fff640 1 -- 172.21.15.26:0/1525057233 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdce405bf10 con 0x7fdcfc0c4ef0 2024-03-06T07:23:02.157 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b500 msgr2=0x7fdcdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b500 0x7fdcdc06d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0a7f40 tx=0x7fdcec009190 comp rx=0 tx=0).stop 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 msgr2=0x7fdcfc1547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc1547a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c430 tx=0x7fdce400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 shutdown_connections 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b500 0x7fdcdc06d9c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcfc1528d0 0x7fdcfc1ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc0c67f0 0x7fdcfc1edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 --2- 172.21.15.26:0/1525057233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc0c4ef0 0x7fdcfc1547a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 >> 172.21.15.26:0/1525057233 conn(0x7fdcfc0aa6a0 msgr2=0x7fdcfc0c5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 shutdown_connections 2024-03-06T07:23:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.157+0000 7fdd01f13640 1 -- 172.21.15.26:0/1525057233 wait complete. 2024-03-06T07:23:02.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:23:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.273+0000 7f6f47850640 1 Processor -- start 2024-03-06T07:23:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.273+0000 7f6f47850640 1 -- start start 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f47850640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f401528d0 0x7f6f400724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f47850640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f40072a10 0x7f6f40155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f47850640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f40152cb0 con 0x7f6f400c67f0 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f47850640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f400c38b0 con 0x7f6f40072a10 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f47850640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401594f0 con 0x7f6f401528d0 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f45ded640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f400c67f0 0x7f6f40155760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51952/0 (socket says 172.21.15.26:51952) 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f45ded640 1 -- 172.21.15.26:0/2896857753 learned_addr learned my addr 172.21.15.26:0/2896857753 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f455ec640 1 --2- 172.21.15.26:0/2896857753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f401528d0 0x7f6f400724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f44deb640 1 --2- 172.21.15.26:0/2896857753 >> v2:172.21.15.117:3300/0 conn(0x7f6f40072a10 0x7f6f40155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f33fff640 1 -- 172.21.15.26:0/2896857753 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070773133 0 0) 0x7f6f40152cb0 con 0x7f6f400c67f0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.274+0000 7f6f33fff640 1 -- 172.21.15.26:0/2896857753 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f2c003520 con 0x7f6f400c67f0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f455ec640 1 -- 172.21.15.26:0/2896857753 >> v2:172.21.15.117:3300/0 conn(0x7f6f40072a10 msgr2=0x7f6f40155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f455ec640 1 --2- 172.21.15.26:0/2896857753 >> v2:172.21.15.117:3300/0 conn(0x7f6f40072a10 0x7f6f40155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f455ec640 1 -- 172.21.15.26:0/2896857753 >> v1:172.21.15.26:6789/0 conn(0x7f6f400c67f0 legacy=0x7f6f40155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f455ec640 1 -- 172.21.15.26:0/2896857753 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f400a7f70 con 0x7f6f401528d0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f44deb640 1 --2- 172.21.15.26:0/2896857753 >> v2:172.21.15.117:3300/0 conn(0x7f6f40072a10 0x7f6f40155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f455ec640 1 --2- 172.21.15.26:0/2896857753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f401528d0 0x7f6f400724d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f340097a0 tx=0x7f6f3402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd0fdd1064f26e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f33fff640 1 -- 172.21.15.26:0/2896857753 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34034070 con 0x7f6f401528d0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.275+0000 7f6f33fff640 1 -- 172.21.15.26:0/2896857753 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f34030800 con 0x7f6f401528d0 2024-03-06T07:23:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f33fff640 1 -- 172.21.15.26:0/2896857753 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34030bb0 con 0x7f6f401528d0 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 -- 172.21.15.26:0/2896857753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f401528d0 msgr2=0x7f6f400724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 --2- 172.21.15.26:0/2896857753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f401528d0 0x7f6f400724d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f340097a0 tx=0x7f6f3402b940 comp rx=0 tx=0).stop 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 -- 172.21.15.26:0/2896857753 shutdown_connections 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 --2- 172.21.15.26:0/2896857753 >> v2:172.21.15.117:3300/0 conn(0x7f6f40072a10 0x7f6f40155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 --2- 172.21.15.26:0/2896857753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f401528d0 0x7f6f400724d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 -- 172.21.15.26:0/2896857753 >> 172.21.15.26:0/2896857753 conn(0x7f6f400aa6a0 msgr2=0x7f6f400aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 -- 172.21.15.26:0/2896857753 shutdown_connections 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.276+0000 7f6f47850640 1 -- 172.21.15.26:0/2896857753 wait complete. 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.277+0000 7f6f47850640 1 Processor -- start 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.277+0000 7f6f47850640 1 -- start start 2024-03-06T07:23:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f47850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f40072a10 0x7f6f401ee220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f47850640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f400c67f0 0x7f6f40158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f47850640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f401528d0 0x7f6f40148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f47850640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401591a0 con 0x7f6f401528d0 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f47850640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400a8250 con 0x7f6f40072a10 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f47850640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400c4ef0 con 0x7f6f400c67f0 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f401528d0 0x7f6f40148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f401528d0 0x7f6f40148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39236/0 (socket says 172.21.15.26:39236) 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 -- 172.21.15.26:0/1688617276 learned_addr learned my addr 172.21.15.26:0/1688617276 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f455ec640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f40072a10 0x7f6f401ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f44deb640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f400c67f0 0x7f6f40158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 -- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f400c67f0 msgr2=0x7f6f40158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f400c67f0 0x7f6f40158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 -- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f40072a10 msgr2=0x7f6f401ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f40072a10 0x7f6f401ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.278+0000 7f6f45ded640 1 -- 172.21.15.26:0/1688617276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f401eeeb0 con 0x7f6f401528d0 2024-03-06T07:23:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.279+0000 7f6f45ded640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f401528d0 0x7f6f40148c60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c400 tx=0x7f6f3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.279+0000 7f6f31ffb640 1 -- 172.21.15.26:0/1688617276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38017070 con 0x7f6f401528d0 2024-03-06T07:23:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.279+0000 7f6f31ffb640 1 -- 172.21.15.26:0/1688617276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38005ce0 con 0x7f6f401528d0 2024-03-06T07:23:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.279+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f401566c0 con 0x7f6f401528d0 2024-03-06T07:23:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.279+0000 7f6f31ffb640 1 -- 172.21.15.26:0/1688617276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f380072d0 con 0x7f6f401528d0 2024-03-06T07:23:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.279+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f400ae280 con 0x7f6f401528d0 2024-03-06T07:23:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.280+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4004f330 con 0x7f6f401528d0 2024-03-06T07:23:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.280+0000 7f6f31ffb640 1 -- 172.21.15.26:0/1688617276 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f3801a070 con 0x7f6f401528d0 2024-03-06T07:23:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.281+0000 7f6f31ffb640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f2406b4d0 0x7f6f2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.281+0000 7f6f31ffb640 1 -- 172.21.15.26:0/1688617276 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f6f38089a60 con 0x7f6f401528d0 2024-03-06T07:23:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.281+0000 7f6f455ec640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f2406b4d0 0x7f6f2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.281+0000 7f6f455ec640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f2406b4d0 0x7f6f2406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6f34036000 tx=0x7f6f34008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.285+0000 7f6f31ffb640 1 -- 172.21.15.26:0/1688617276 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f38057d60 con 0x7f6f401528d0 2024-03-06T07:23:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.468+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f4006dda0 con 0x7f6f401528d0 2024-03-06T07:23:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.469+0000 7f6f31ffb640 1 -- 172.21.15.26:0/1688617276 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f3805be00 con 0x7f6f401528d0 2024-03-06T07:23:02.470 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:23:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f2406b4d0 msgr2=0x7f6f2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f2406b4d0 0x7f6f2406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6f34036000 tx=0x7f6f34008030 comp rx=0 tx=0).stop 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f401528d0 msgr2=0x7f6f40148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f401528d0 0x7f6f40148c60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c400 tx=0x7f6f3800c8c0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 shutdown_connections 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f2406b4d0 0x7f6f2406d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f401528d0 0x7f6f40148c60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f400c67f0 0x7f6f40158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 --2- 172.21.15.26:0/1688617276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f40072a10 0x7f6f401ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 >> 172.21.15.26:0/1688617276 conn(0x7f6f400aa6a0 msgr2=0x7f6f400c5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 shutdown_connections 2024-03-06T07:23:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.471+0000 7f6f47850640 1 -- 172.21.15.26:0/1688617276 wait complete. 2024-03-06T07:23:02.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.589+0000 7f0f19efb640 1 Processor -- start 2024-03-06T07:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.589+0000 7f0f19efb640 1 -- start start 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f19efb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f14155a00 0x7f0f14159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f19efb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f14155e70 con 0x7f0f1414bcc0 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f19efb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f140c75e0 con 0x7f0f140be150 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f19efb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f140c7f10 con 0x7f0f14155a00 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f137fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0f1414bcc0 0x7f0f140c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51964/0 (socket says 172.21.15.26:51964) 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f137fe640 1 -- 172.21.15.26:0/20822432 learned_addr learned my addr 172.21.15.26:0/20822432 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f13fff640 1 --2- 172.21.15.26:0/20822432 >> v2:172.21.15.117:3300/0 conn(0x7f0f14155a00 0x7f0f14159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f127fc640 1 -- 172.21.15.26:0/20822432 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508371621 0 0) 0x7f0f14155e70 con 0x7f0f1414bcc0 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f127fc640 1 -- 172.21.15.26:0/20822432 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f00003520 con 0x7f0f1414bcc0 2024-03-06T07:23:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f127fc640 1 -- 172.21.15.26:0/20822432 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664514543 0 0) 0x7f0f140c75e0 con 0x7f0f140be150 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f127fc640 1 -- 172.21.15.26:0/20822432 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f14155e70 con 0x7f0f140be150 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f13fff640 1 -- 172.21.15.26:0/20822432 >> v1:172.21.15.117:6789/0 conn(0x7f0f140be150 legacy=0x7f0f14158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f13fff640 1 -- 172.21.15.26:0/20822432 >> v1:172.21.15.26:6789/0 conn(0x7f0f1414bcc0 legacy=0x7f0f140c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.590+0000 7f0f13fff640 1 -- 172.21.15.26:0/20822432 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f140c78c0 con 0x7f0f14155a00 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.591+0000 7f0f13fff640 1 --2- 172.21.15.26:0/20822432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14155a00 0x7f0f14159120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009d90 tx=0x7f0f0c02bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=879d5fea3b94ce4a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.591+0000 7f0f127fc640 1 -- 172.21.15.26:0/20822432 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c039070 con 0x7f0f14155a00 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.591+0000 7f0f127fc640 1 -- 172.21.15.26:0/20822432 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c02cd80 con 0x7f0f14155a00 2024-03-06T07:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.591+0000 7f0f127fc640 1 -- 172.21.15.26:0/20822432 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c034660 con 0x7f0f14155a00 2024-03-06T07:23:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.592+0000 7f0f19efb640 1 -- 172.21.15.26:0/20822432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14155a00 msgr2=0x7f0f14159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.592+0000 7f0f19efb640 1 --2- 172.21.15.26:0/20822432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14155a00 0x7f0f14159120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009d90 tx=0x7f0f0c02bfd0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.592+0000 7f0f19efb640 1 -- 172.21.15.26:0/20822432 shutdown_connections 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.592+0000 7f0f19efb640 1 --2- 172.21.15.26:0/20822432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14155a00 0x7f0f14159120 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.592+0000 7f0f19efb640 1 -- 172.21.15.26:0/20822432 >> 172.21.15.26:0/20822432 conn(0x7f0f140ac600 msgr2=0x7f0f140aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.592+0000 7f0f19efb640 1 -- 172.21.15.26:0/20822432 shutdown_connections 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.592+0000 7f0f19efb640 1 -- 172.21.15.26:0/20822432 wait complete. 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.593+0000 7f0f19efb640 1 Processor -- start 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.593+0000 7f0f19efb640 1 -- start start 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f19efb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f140be150 0x7f0f14145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f19efb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1414bcc0 0x7f0f1406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f19efb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f14155a00 0x7f0f14151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f19efb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1406d1d0 con 0x7f0f14155a00 2024-03-06T07:23:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f19efb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f140c5ae0 con 0x7f0f140be150 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f19efb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f140c5c60 con 0x7f0f1414bcc0 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f14155a00 0x7f0f14151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f14155a00 0x7f0f14151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39244/0 (socket says 172.21.15.26:39244) 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 -- 172.21.15.26:0/3473286339 learned_addr learned my addr 172.21.15.26:0/3473286339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f137fe640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f140be150 0x7f0f14145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f12ffd640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1414bcc0 0x7f0f1406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 -- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1414bcc0 msgr2=0x7f0f1406b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1414bcc0 0x7f0f1406b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 -- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f140be150 msgr2=0x7f0f14145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f140be150 0x7f0f14145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 -- 172.21.15.26:0/3473286339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1414f960 con 0x7f0f14155a00 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.594+0000 7f0f13fff640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f14155a00 0x7f0f14151850 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009eb0 tx=0x7f0f0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.595+0000 7f0f0bfff640 1 -- 172.21.15.26:0/3473286339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c039050 con 0x7f0f14155a00 2024-03-06T07:23:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.595+0000 7f0f0bfff640 1 -- 172.21.15.26:0/3473286339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c045070 con 0x7f0f14155a00 2024-03-06T07:23:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.595+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f141eb930 con 0x7f0f14155a00 2024-03-06T07:23:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.595+0000 7f0f0bfff640 1 -- 172.21.15.26:0/3473286339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c009450 con 0x7f0f14155a00 2024-03-06T07:23:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.595+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1414fee0 con 0x7f0f14155a00 2024-03-06T07:23:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.596+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1404f330 con 0x7f0f14155a00 2024-03-06T07:23:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.596+0000 7f0f0bfff640 1 -- 172.21.15.26:0/3473286339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f0c03e850 con 0x7f0f14155a00 2024-03-06T07:23:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.596+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ef406b4d0 0x7f0ef406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.597+0000 7f0f0bfff640 1 -- 172.21.15.26:0/3473286339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f0f0c0ad4f0 con 0x7f0f14155a00 2024-03-06T07:23:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.597+0000 7f0f137fe640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ef406b4d0 0x7f0ef406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.597+0000 7f0f137fe640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ef406b4d0 0x7f0ef406d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0efc000f60 tx=0x7f0efc007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.601+0000 7f0f0bfff640 1 -- 172.21.15.26:0/3473286339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f0c07b8f0 con 0x7f0f14155a00 2024-03-06T07:23:02.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.814+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f140c58c0 con 0x7f0f14155a00 2024-03-06T07:23:02.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.814+0000 7f0f0bfff640 1 -- 172.21.15.26:0/3473286339 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f0c07f990 con 0x7f0f14155a00 2024-03-06T07:23:02.816 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:23:02.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ef406b4d0 msgr2=0x7f0ef406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ef406b4d0 0x7f0ef406d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0efc000f60 tx=0x7f0efc007d50 comp rx=0 tx=0).stop 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f14155a00 msgr2=0x7f0f14151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f14155a00 0x7f0f14151850 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009eb0 tx=0x7f0f0c004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 shutdown_connections 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ef406b4d0 0x7f0ef406d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f14155a00 0x7f0f14151850 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1414bcc0 0x7f0f1406b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 --2- 172.21.15.26:0/3473286339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f140be150 0x7f0f14145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 >> 172.21.15.26:0/3473286339 conn(0x7f0f140ac600 msgr2=0x7f0f140ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 shutdown_connections 2024-03-06T07:23:02.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.817+0000 7f0f19efb640 1 -- 172.21.15.26:0/3473286339 wait complete. 2024-03-06T07:23:02.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.933+0000 7f09e69d6640 1 Processor -- start 2024-03-06T07:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.933+0000 7f09e69d6640 1 -- start start 2024-03-06T07:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09e69d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00b0070 0x7f09e0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09e69d6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f09e006ca70 0x7f09e00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09e69d6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e00b0450 con 0x7f09e00c5380 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09e69d6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e006c020 con 0x7f09e006ca70 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09e69d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00c77f0 con 0x7f09e00b0070 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09e4f73640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f09e00c5380 0x7f09e014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51978/0 (socket says 172.21.15.26:51978) 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09e4f73640 1 -- 172.21.15.26:0/3807344358 learned_addr learned my addr 172.21.15.26:0/3807344358 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09df7fe640 1 --2- 172.21.15.26:0/3807344358 >> v2:172.21.15.117:3300/0 conn(0x7f09e006ca70 0x7f09e00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.934+0000 7f09dffff640 1 --2- 172.21.15.26:0/3807344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00b0070 0x7f09e0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09deffd640 1 -- 172.21.15.26:0/3807344358 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457193294 0 0) 0x7f09d8001490 con 0x7f09e00c5380 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09deffd640 1 -- 172.21.15.26:0/3807344358 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c8003520 con 0x7f09e00c5380 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09dffff640 1 -- 172.21.15.26:0/3807344358 >> v2:172.21.15.117:3300/0 conn(0x7f09e006ca70 msgr2=0x7f09e00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09dffff640 1 --2- 172.21.15.26:0/3807344358 >> v2:172.21.15.117:3300/0 conn(0x7f09e006ca70 0x7f09e00c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09dffff640 1 -- 172.21.15.26:0/3807344358 >> v1:172.21.15.26:6789/0 conn(0x7f09e00c5380 legacy=0x7f09e014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09dffff640 1 -- 172.21.15.26:0/3807344358 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e00c7ad0 con 0x7f09e00b0070 2024-03-06T07:23:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09df7fe640 1 --2- 172.21.15.26:0/3807344358 >> v2:172.21.15.117:3300/0 conn(0x7f09e006ca70 0x7f09e00c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.935+0000 7f09dffff640 1 --2- 172.21.15.26:0/3807344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00b0070 0x7f09e0155c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09d00097a0 tx=0x7f09d002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5edc34351a5fdfde server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09deffd640 1 -- 172.21.15.26:0/3807344358 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d00315a0 con 0x7f09e00b0070 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09deffd640 1 -- 172.21.15.26:0/3807344358 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0037070 con 0x7f09e00b0070 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09deffd640 1 -- 172.21.15.26:0/3807344358 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0031960 con 0x7f09e00b0070 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 -- 172.21.15.26:0/3807344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00b0070 msgr2=0x7f09e0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 --2- 172.21.15.26:0/3807344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00b0070 0x7f09e0155c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09d00097a0 tx=0x7f09d002b840 comp rx=0 tx=0).stop 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 -- 172.21.15.26:0/3807344358 shutdown_connections 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 --2- 172.21.15.26:0/3807344358 >> v2:172.21.15.117:3300/0 conn(0x7f09e006ca70 0x7f09e00c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 --2- 172.21.15.26:0/3807344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00b0070 0x7f09e0155c30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 -- 172.21.15.26:0/3807344358 >> 172.21.15.26:0/3807344358 conn(0x7f09e00ac590 msgr2=0x7f09e00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 -- 172.21.15.26:0/3807344358 shutdown_connections 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.936+0000 7f09e69d6640 1 -- 172.21.15.26:0/3807344358 wait complete. 2024-03-06T07:23:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.937+0000 7f09e69d6640 1 Processor -- start 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.937+0000 7f09e69d6640 1 -- start start 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09e69d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09e006ca70 0x7f09e01ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09e69d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e00b0070 0x7f09e0158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09e69d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00c5380 0x7f09e0150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09e69d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e006cef0 con 0x7f09e006ca70 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09e69d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e006c550 con 0x7f09e00b0070 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09e69d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e006da80 con 0x7f09e00c5380 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09dffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09e006ca70 0x7f09e01ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09dffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09e006ca70 0x7f09e01ee050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39254/0 (socket says 172.21.15.26:39254) 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09dffff640 1 -- 172.21.15.26:0/1622547219 learned_addr learned my addr 172.21.15.26:0/1622547219 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09df7fe640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e00b0070 0x7f09e0158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09e4f73640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00c5380 0x7f09e0150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09dffff640 1 -- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00c5380 msgr2=0x7f09e0150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.938+0000 7f09dffff640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00c5380 0x7f09e0150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09dffff640 1 -- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e00b0070 msgr2=0x7f09e0158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09dffff640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e00b0070 0x7f09e0158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09dffff640 1 -- 172.21.15.26:0/1622547219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0156600 con 0x7f09e006ca70 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09dffff640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09e006ca70 0x7f09e01ee050 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f09d0034040 tx=0x7f09d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09dcff9640 1 -- 172.21.15.26:0/1622547219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0032630 con 0x7f09e006ca70 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09dcff9640 1 -- 172.21.15.26:0/1622547219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d00327f0 con 0x7f09e006ca70 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e01568e0 con 0x7f09e006ca70 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09dcff9640 1 -- 172.21.15.26:0/1622547219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0032be0 con 0x7f09e006ca70 2024-03-06T07:23:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.939+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e0158320 con 0x7f09e006ca70 2024-03-06T07:23:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.940+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e00c5f30 con 0x7f09e006ca70 2024-03-06T07:23:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.941+0000 7f09dcff9640 1 -- 172.21.15.26:0/1622547219 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09d004e430 con 0x7f09e006ca70 2024-03-06T07:23:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.941+0000 7f09dcff9640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f09cc06b4d0 0x7f09cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.941+0000 7f09dcff9640 1 -- 172.21.15.26:0/1622547219 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f09d002f030 con 0x7f09e006ca70 2024-03-06T07:23:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.941+0000 7f09df7fe640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f09cc06b4d0 0x7f09cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.942+0000 7f09df7fe640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f09cc06b4d0 0x7f09cc06d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009560 tx=0x7f09d4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:02.946+0000 7f09dcff9640 1 -- 172.21.15.26:0/1622547219 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f09e00c5f30 con 0x7f09e006ca70 2024-03-06T07:23:03.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.172+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09e004f330 con 0x7f09e006ca70 2024-03-06T07:23:03.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.173+0000 7f09dcff9640 1 -- 172.21.15.26:0/1622547219 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v79) v1 ==== 105+0+11 (secure 0 0 0) 0x7f09d0040070 con 0x7f09e006ca70 2024-03-06T07:23:03.175 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:23:03.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f09cc06b4d0 msgr2=0x7f09cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f09cc06b4d0 0x7f09cc06d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009560 tx=0x7f09d4009190 comp rx=0 tx=0).stop 2024-03-06T07:23:03.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09e006ca70 msgr2=0x7f09e01ee050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09e006ca70 0x7f09e01ee050 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f09d0034040 tx=0x7f09d00046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 shutdown_connections 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f09cc06b4d0 0x7f09cc06d990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e00c5380 0x7f09e0150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e00b0070 0x7f09e0158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 --2- 172.21.15.26:0/1622547219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09e006ca70 0x7f09e01ee050 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 >> 172.21.15.26:0/1622547219 conn(0x7f09e00ac590 msgr2=0x7f09e00ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.175+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 shutdown_connections 2024-03-06T07:23:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.176+0000 7f09e69d6640 1 -- 172.21.15.26:0/1622547219 wait complete. 2024-03-06T07:23:03.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:23:03.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.307+0000 7f999e0c2640 1 Processor -- start 2024-03-06T07:23:03.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.307+0000 7f999e0c2640 1 -- start start 2024-03-06T07:23:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.307+0000 7f999e0c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f999814bcc0 0x7f9998155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.307+0000 7f999e0c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99980be150 0x7f99980c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.307+0000 7f999e0c2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99980c75e0 con 0x7f99980c5910 2024-03-06T07:23:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.307+0000 7f999e0c2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99980c7f10 con 0x7f99980be150 2024-03-06T07:23:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.307+0000 7f999e0c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99980c78c0 con 0x7f999814bcc0 2024-03-06T07:23:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f9996ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99980be150 0x7f99980c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f9996ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99980be150 0x7f99980c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39264/0 (socket says 172.21.15.26:39264) 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f9996ffd640 1 -- 172.21.15.26:0/3725761300 learned_addr learned my addr 172.21.15.26:0/3725761300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f99977fe640 1 --2- 172.21.15.26:0/3725761300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f999814bcc0 0x7f9998155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f9996ffd640 1 -- 172.21.15.26:0/3725761300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f999814bcc0 msgr2=0x7f9998155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f9996ffd640 1 --2- 172.21.15.26:0/3725761300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f999814bcc0 0x7f9998155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f9996ffd640 1 -- 172.21.15.26:0/3725761300 >> v1:172.21.15.117:6789/0 conn(0x7f99980c5910 legacy=0x7f9998158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f99967fc640 1 -- 172.21.15.26:0/3725761300 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841325373 0 0) 0x7f99980c75e0 con 0x7f99980c5910 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.308+0000 7f9996ffd640 1 -- 172.21.15.26:0/3725761300 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980c7ba0 con 0x7f99980be150 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f9996ffd640 1 --2- 172.21.15.26:0/3725761300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99980be150 0x7f99980c53d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f998c009870 tx=0x7f998c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761a5b7c2070a73b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f99967fc640 1 -- 172.21.15.26:0/3725761300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c02f840 con 0x7f99980be150 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f99967fc640 1 -- 172.21.15.26:0/3725761300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c02f9e0 con 0x7f99980be150 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f99967fc640 1 -- 172.21.15.26:0/3725761300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c02fdb0 con 0x7f99980be150 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 -- 172.21.15.26:0/3725761300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99980be150 msgr2=0x7f99980c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 --2- 172.21.15.26:0/3725761300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99980be150 0x7f99980c53d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f998c009870 tx=0x7f998c02ba20 comp rx=0 tx=0).stop 2024-03-06T07:23:03.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 -- 172.21.15.26:0/3725761300 shutdown_connections 2024-03-06T07:23:03.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 --2- 172.21.15.26:0/3725761300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99980be150 0x7f99980c53d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 --2- 172.21.15.26:0/3725761300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f999814bcc0 0x7f9998155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 -- 172.21.15.26:0/3725761300 >> 172.21.15.26:0/3725761300 conn(0x7f99980ac600 msgr2=0x7f99980aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 -- 172.21.15.26:0/3725761300 shutdown_connections 2024-03-06T07:23:03.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.309+0000 7f999e0c2640 1 -- 172.21.15.26:0/3725761300 wait complete. 2024-03-06T07:23:03.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.311+0000 7f999e0c2640 1 Processor -- start 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.311+0000 7f999e0c2640 1 -- start start 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f999e0c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99980be150 0x7f99980ab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f999e0c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99980c5910 0x7f99981ee040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f999e0c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f999814bcc0 0x7f99981ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f999e0c2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99981592a0 con 0x7f999814bcc0 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f999e0c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998159120 con 0x7f99980be150 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f999e0c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9998159420 con 0x7f99980c5910 2024-03-06T07:23:03.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f9997fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f999814bcc0 0x7f99981ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f9997fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f999814bcc0 0x7f99981ee580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39274/0 (socket says 172.21.15.26:39274) 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f9997fff640 1 -- 172.21.15.26:0/424627582 learned_addr learned my addr 172.21.15.26:0/424627582 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f9996ffd640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99980c5910 0x7f99981ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.312+0000 7f99977fe640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99980be150 0x7f99980ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f9997fff640 1 -- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99980c5910 msgr2=0x7f99981ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f9997fff640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99980c5910 0x7f99981ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f9997fff640 1 -- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99980be150 msgr2=0x7f99980ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f9997fff640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99980be150 0x7f99980ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f9997fff640 1 -- 172.21.15.26:0/424627582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99981501b0 con 0x7f999814bcc0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f9996ffd640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99980c5910 0x7f99981ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f99977fe640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99980be150 0x7f99980ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f9997fff640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f999814bcc0 0x7f99981ee580 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f999000c760 tx=0x7f999000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f997ffff640 1 -- 172.21.15.26:0/424627582 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990003da0 con 0x7f999814bcc0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f997ffff640 1 -- 172.21.15.26:0/424627582 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990005070 con 0x7f999814bcc0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99980c2a10 con 0x7f999814bcc0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f997ffff640 1 -- 172.21.15.26:0/424627582 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990005460 con 0x7f999814bcc0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.313+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99980ab6c0 con 0x7f999814bcc0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.314+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999804f330 con 0x7f999814bcc0 2024-03-06T07:23:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.315+0000 7f997ffff640 1 -- 172.21.15.26:0/424627582 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9990005600 con 0x7f999814bcc0 2024-03-06T07:23:03.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.315+0000 7f997ffff640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f998406b4d0 0x7f998406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.315+0000 7f997ffff640 1 -- 172.21.15.26:0/424627582 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f999008a500 con 0x7f999814bcc0 2024-03-06T07:23:03.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.315+0000 7f99977fe640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f998406b4d0 0x7f998406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.316+0000 7f99977fe640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f998406b4d0 0x7f998406d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f99980c2900 tx=0x7f9988009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.319+0000 7f997ffff640 1 -- 172.21.15.26:0/424627582 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9990058880 con 0x7f999814bcc0 2024-03-06T07:23:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.502+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f99981456a0 con 0x7f999814bcc0 2024-03-06T07:23:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.502+0000 7f997ffff640 1 -- 172.21.15.26:0/424627582 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v79) v1 ==== 101+0+11 (secure 0 0 0) 0x7f999005c920 con 0x7f999814bcc0 2024-03-06T07:23:03.504 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f998406b4d0 msgr2=0x7f998406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f998406b4d0 0x7f998406d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f99980c2900 tx=0x7f9988009190 comp rx=0 tx=0).stop 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f999814bcc0 msgr2=0x7f99981ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f999814bcc0 0x7f99981ee580 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f999000c760 tx=0x7f999000cc20 comp rx=0 tx=0).stop 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 shutdown_connections 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f998406b4d0 0x7f998406d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f999814bcc0 0x7f99981ee580 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99980c5910 0x7f99981ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 --2- 172.21.15.26:0/424627582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99980be150 0x7f99980ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 >> 172.21.15.26:0/424627582 conn(0x7f99980ac600 msgr2=0x7f99980ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 shutdown_connections 2024-03-06T07:23:03.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.504+0000 7f999e0c2640 1 -- 172.21.15.26:0/424627582 wait complete. 2024-03-06T07:23:03.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.616+0000 7fde93fb6640 1 Processor -- start 2024-03-06T07:23:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.617+0000 7fde93fb6640 1 -- start start 2024-03-06T07:23:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.617+0000 7fde93fb6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde8c153990 0x7fde8c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.617+0000 7fde93fb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde8c0acc00 0x7fde8c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.617+0000 7fde93fb6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c070aa0 con 0x7fde8c1506b0 2024-03-06T07:23:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.617+0000 7fde93fb6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c156450 con 0x7fde8c0acc00 2024-03-06T07:23:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.617+0000 7fde93fb6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c156a10 con 0x7fde8c153990 2024-03-06T07:23:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde91d52640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde8c1506b0 0x7fde8c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51986/0 (socket says 172.21.15.26:51986) 2024-03-06T07:23:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde92553640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde8c0acc00 0x7fde8c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde91d52640 1 -- 172.21.15.26:0/1523692493 learned_addr learned my addr 172.21.15.26:0/1523692493 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde91551640 1 --2- 172.21.15.26:0/1523692493 >> v2:172.21.15.117:3300/0 conn(0x7fde8c153990 0x7fde8c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde92553640 1 -- 172.21.15.26:0/1523692493 >> v2:172.21.15.117:3300/0 conn(0x7fde8c153990 msgr2=0x7fde8c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde92553640 1 --2- 172.21.15.26:0/1523692493 >> v2:172.21.15.117:3300/0 conn(0x7fde8c153990 0x7fde8c156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde92553640 1 -- 172.21.15.26:0/1523692493 >> v1:172.21.15.26:6789/0 conn(0x7fde8c1506b0 legacy=0x7fde8c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde90d50640 1 -- 172.21.15.26:0/1523692493 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767204188 0 0) 0x7fde8c070aa0 con 0x7fde8c1506b0 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde92553640 1 -- 172.21.15.26:0/1523692493 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c070d80 con 0x7fde8c0acc00 2024-03-06T07:23:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.618+0000 7fde91551640 1 --2- 172.21.15.26:0/1523692493 >> v2:172.21.15.117:3300/0 conn(0x7fde8c153990 0x7fde8c156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.619+0000 7fde92553640 1 --2- 172.21.15.26:0/1523692493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c0acc00 0x7fde8c0c54a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fde8c150a90 tx=0x7fde8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b563fe529a25d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.619+0000 7fde90d50640 1 -- 172.21.15.26:0/1523692493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84039070 con 0x7fde8c0acc00 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.619+0000 7fde90d50640 1 -- 172.21.15.26:0/1523692493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8402cc80 con 0x7fde8c0acc00 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.619+0000 7fde90d50640 1 -- 172.21.15.26:0/1523692493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84034660 con 0x7fde8c0acc00 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 -- 172.21.15.26:0/1523692493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c0acc00 msgr2=0x7fde8c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1523692493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c0acc00 0x7fde8c0c54a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fde8c150a90 tx=0x7fde8402bed0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 -- 172.21.15.26:0/1523692493 shutdown_connections 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1523692493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c0acc00 0x7fde8c0c54a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1523692493 >> v2:172.21.15.117:3300/0 conn(0x7fde8c153990 0x7fde8c156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 -- 172.21.15.26:0/1523692493 >> 172.21.15.26:0/1523692493 conn(0x7fde8c06b1a0 msgr2=0x7fde8c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 -- 172.21.15.26:0/1523692493 shutdown_connections 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.620+0000 7fde93fb6640 1 -- 172.21.15.26:0/1523692493 wait complete. 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.622+0000 7fde93fb6640 1 Processor -- start 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.622+0000 7fde93fb6640 1 -- start start 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde93fb6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c0acc00 0x7fde8c1ece20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde93fb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1506b0 0x7fde8c0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde93fb6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c153990 0x7fde8c1e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde93fb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0c5b60 con 0x7fde8c1506b0 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde93fb6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0c59e0 con 0x7fde8c0acc00 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde93fb6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0c5ce0 con 0x7fde8c153990 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1506b0 0x7fde8c0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1506b0 0x7fde8c0c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39296/0 (socket says 172.21.15.26:39296) 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 -- 172.21.15.26:0/1215017212 learned_addr learned my addr 172.21.15.26:0/1215017212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91d52640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c0acc00 0x7fde8c1ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde92553640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c153990 0x7fde8c1e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 -- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c153990 msgr2=0x7fde8c1e8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c153990 0x7fde8c1e8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 -- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c0acc00 msgr2=0x7fde8c1ece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c0acc00 0x7fde8c1ece20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 -- 172.21.15.26:0/1215017212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c155e30 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde91551640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1506b0 0x7fde8c0c7c20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fde8000e960 tx=0x7fde8000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde8a7fc640 1 -- 172.21.15.26:0/1215017212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80017070 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde8a7fc640 1 -- 172.21.15.26:0/1215017212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000ca00 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.623+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c156110 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.624+0000 7fde8a7fc640 1 -- 172.21.15.26:0/1215017212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000cdd0 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.624+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c1ec620 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.624+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c153e40 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.625+0000 7fde8a7fc640 1 -- 172.21.15.26:0/1215017212 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde8001a070 con 0x7fde8c1506b0 2024-03-06T07:23:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.625+0000 7fde8a7fc640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde7806b4d0 0x7fde7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.626+0000 7fde8a7fc640 1 -- 172.21.15.26:0/1215017212 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fde8008a860 con 0x7fde8c1506b0 2024-03-06T07:23:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.626+0000 7fde91d52640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde7806b4d0 0x7fde7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.626+0000 7fde91d52640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde7806b4d0 0x7fde7806d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000f60 tx=0x7fde7c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.630+0000 7fde8a7fc640 1 -- 172.21.15.26:0/1215017212 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fde8c153e40 con 0x7fde8c1506b0 2024-03-06T07:23:03.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.850+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde8c04f330 con 0x7fde8c1506b0 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.850+0000 7fde8a7fc640 1 -- 172.21.15.26:0/1215017212 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6260 (secure 0 0 0) 0x7fde80058be0 con 0x7fde8c1506b0 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:58.795121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6457":{"gid":6457,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6836/252560219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":252560219},{"type":"v1","addr":"172.21.15.117:6836","nonce":252560219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6457,"qdb_cluster":[6457]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.852+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde7806b4d0 msgr2=0x7fde7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde7806b4d0 0x7fde7806d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000f60 tx=0x7fde7c007d50 comp rx=0 tx=0).stop 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1506b0 msgr2=0x7fde8c0c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1506b0 0x7fde8c0c7c20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fde8000e960 tx=0x7fde8000ee20 comp rx=0 tx=0).stop 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 shutdown_connections 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde7806b4d0 0x7fde7806d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c153990 0x7fde8c1e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1506b0 0x7fde8c0c7c20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 --2- 172.21.15.26:0/1215017212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c0acc00 0x7fde8c1ece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 >> 172.21.15.26:0/1215017212 conn(0x7fde8c06b1a0 msgr2=0x7fde8c14b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 shutdown_connections 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.853+0000 7fde93fb6640 1 -- 172.21.15.26:0/1215017212 wait complete. 2024-03-06T07:23:03.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.964+0000 7fa7511d0640 1 Processor -- start 2024-03-06T07:23:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa7511d0640 1 -- start start 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa7511d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c0c88c0 0x7fa74c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa7511d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa74c158b60 0x7fa74c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa7511d0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74c0c73d0 con 0x7fa74c14f360 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa7511d0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74c0c8dd0 con 0x7fa74c158b60 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa7511d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0be2e0 con 0x7fa74c0c88c0 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa74b59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa74c158b60 0x7fa74c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa74ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa74c14f360 0x7fa74c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51992/0 (socket says 172.21.15.26:51992) 2024-03-06T07:23:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa74ad9d640 1 -- 172.21.15.26:0/2899518855 learned_addr learned my addr 172.21.15.26:0/2899518855 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.965+0000 7fa74a59c640 1 --2- 172.21.15.26:0/2899518855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c0c88c0 0x7fa74c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.966+0000 7fa74b59e640 1 -- 172.21.15.26:0/2899518855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c0c88c0 msgr2=0x7fa74c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.966+0000 7fa74b59e640 1 --2- 172.21.15.26:0/2899518855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c0c88c0 0x7fa74c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.966+0000 7fa74b59e640 1 -- 172.21.15.26:0/2899518855 >> v1:172.21.15.26:6789/0 conn(0x7fa74c14f360 legacy=0x7fa74c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.966+0000 7fa74b59e640 1 -- 172.21.15.26:0/2899518855 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c0c7670 con 0x7fa74c158b60 2024-03-06T07:23:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.966+0000 7fa749d9b640 1 -- 172.21.15.26:0/2899518855 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114478518 0 0) 0x7fa74c0c73d0 con 0x7fa74c14f360 2024-03-06T07:23:03.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.966+0000 7fa74b59e640 1 --2- 172.21.15.26:0/2899518855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c158ff0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa744009d90 tx=0x7fa74402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f993c694c3597836 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa749d9b640 1 -- 172.21.15.26:0/2899518855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744039070 con 0x7fa74c158b60 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa749d9b640 1 -- 172.21.15.26:0/2899518855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74402cbc0 con 0x7fa74c158b60 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa749d9b640 1 -- 172.21.15.26:0/2899518855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744034440 con 0x7fa74c158b60 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 -- 172.21.15.26:0/2899518855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 msgr2=0x7fa74c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 --2- 172.21.15.26:0/2899518855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c158ff0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa744009d90 tx=0x7fa74402be10 comp rx=0 tx=0).stop 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 -- 172.21.15.26:0/2899518855 shutdown_connections 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 --2- 172.21.15.26:0/2899518855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c158ff0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 --2- 172.21.15.26:0/2899518855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c0c88c0 0x7fa74c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 -- 172.21.15.26:0/2899518855 >> 172.21.15.26:0/2899518855 conn(0x7fa74c0ac570 msgr2=0x7fa74c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 -- 172.21.15.26:0/2899518855 shutdown_connections 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.967+0000 7fa7511d0640 1 -- 172.21.15.26:0/2899518855 wait complete. 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 Processor -- start 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 -- start start 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa74c0c88c0 0x7fa74c1ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c14f360 0x7fa74c0bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0c5a50 con 0x7fa74c158b60 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0c7c30 con 0x7fa74c0c88c0 2024-03-06T07:23:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa7511d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0c5bd0 con 0x7fa74c14f360 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa74b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa74b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39316/0 (socket says 172.21.15.26:39316) 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa74b59e640 1 -- 172.21.15.26:0/234541500 learned_addr learned my addr 172.21.15.26:0/234541500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa74a59c640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c14f360 0x7fa74c0bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.969+0000 7fa74ad9d640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa74c0c88c0 0x7fa74c1ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa74b59e640 1 -- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c14f360 msgr2=0x7fa74c0bf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa74b59e640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c14f360 0x7fa74c0bf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa74b59e640 1 -- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa74c0c88c0 msgr2=0x7fa74c1ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa74b59e640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa74c0c88c0 0x7fa74c1ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa74b59e640 1 -- 172.21.15.26:0/234541500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c1ec610 con 0x7fa74c158b60 2024-03-06T07:23:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa74b59e640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c14b790 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa744037040 tx=0x7fa744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa7337fe640 1 -- 172.21.15.26:0/234541500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744039050 con 0x7fa74c158b60 2024-03-06T07:23:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa7337fe640 1 -- 172.21.15.26:0/234541500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa744045070 con 0x7fa74c158b60 2024-03-06T07:23:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c0bfce0 con 0x7fa74c158b60 2024-03-06T07:23:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa7337fe640 1 -- 172.21.15.26:0/234541500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744009450 con 0x7fa74c158b60 2024-03-06T07:23:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.970+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c0c0ca0 con 0x7fa74c158b60 2024-03-06T07:23:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.971+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c04f330 con 0x7fa74c158b60 2024-03-06T07:23:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.972+0000 7fa7337fe640 1 -- 172.21.15.26:0/234541500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa74403e850 con 0x7fa74c158b60 2024-03-06T07:23:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.972+0000 7fa7337fe640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa73806b500 0x7fa73806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.972+0000 7fa7337fe640 1 -- 172.21.15.26:0/234541500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa7440acbb0 con 0x7fa74c158b60 2024-03-06T07:23:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.972+0000 7fa74ad9d640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa73806b500 0x7fa73806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.973+0000 7fa74ad9d640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa73806b500 0x7fa73806d9c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa73c007fd0 tx=0x7fa73c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:03.976+0000 7fa7337fe640 1 -- 172.21.15.26:0/234541500 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa74404e020 con 0x7fa74c158b60 2024-03-06T07:23:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.157+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa74c155f20 con 0x7fa74c158b60 2024-03-06T07:23:04.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.158+0000 7fa7337fe640 1 -- 172.21.15.26:0/234541500 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+20810 (secure 0 0 0) 0x7fa74407af30 con 0x7fa74c158b60 2024-03-06T07:23:04.160 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:04.160 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:54.778055+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/218606650":"2024-03-07T07:22:54.778025+0000","172.21.15.117:6834/218606650":"2024-03-07T07:22:54.778025+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa73806b500 msgr2=0x7fa73806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa73806b500 0x7fa73806d9c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa73c007fd0 tx=0x7fa73c007de0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 msgr2=0x7fa74c14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c14b790 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa744037040 tx=0x7fa744004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 shutdown_connections 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa73806b500 0x7fa73806d9c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa74c158b60 0x7fa74c14b790 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa74c14f360 0x7fa74c0bf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 --2- 172.21.15.26:0/234541500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa74c0c88c0 0x7fa74c1ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 >> 172.21.15.26:0/234541500 conn(0x7fa74c0ac570 msgr2=0x7fa74c0ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 shutdown_connections 2024-03-06T07:23:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.160+0000 7fa7511d0640 1 -- 172.21.15.26:0/234541500 wait complete. 2024-03-06T07:23:04.173 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:23:04.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.273+0000 7faa975b6640 1 Processor -- start 2024-03-06T07:23:04.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.273+0000 7faa975b6640 1 -- start start 2024-03-06T07:23:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa975b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa900c88c0 0x7faa90155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa975b6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa90158b60 0x7faa90158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa975b6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa900c73d0 con 0x7faa9014f360 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa975b6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa900c8dd0 con 0x7faa90158b60 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa975b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa900be2e0 con 0x7faa900c88c0 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa94b51640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa900c88c0 0x7faa90155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa94b51640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa900c88c0 0x7faa90155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:43228/0 (socket says 172.21.15.26:43228) 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa95b53640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa90158b60 0x7faa90158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.274+0000 7faa94b51640 1 -- 172.21.15.26:0/2949282481 learned_addr learned my addr 172.21.15.26:0/2949282481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.275+0000 7faa94b51640 1 -- 172.21.15.26:0/2949282481 >> v2:172.21.15.117:3300/0 conn(0x7faa90158b60 msgr2=0x7faa90158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.275+0000 7faa94b51640 1 --2- 172.21.15.26:0/2949282481 >> v2:172.21.15.117:3300/0 conn(0x7faa90158b60 0x7faa90158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.275+0000 7faa94b51640 1 -- 172.21.15.26:0/2949282481 >> v1:172.21.15.117:6789/0 conn(0x7faa9014f360 legacy=0x7faa900c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.275+0000 7faa94b51640 1 -- 172.21.15.26:0/2949282481 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa900c7670 con 0x7faa900c88c0 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.275+0000 7faa94b51640 1 --2- 172.21.15.26:0/2949282481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa900c88c0 0x7faa90155c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa80009870 tx=0x7faa8002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d34b82e4b8965a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa87fff640 1 -- 172.21.15.26:0/2949282481 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80035020 con 0x7faa900c88c0 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa87fff640 1 -- 172.21.15.26:0/2949282481 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80031640 con 0x7faa900c88c0 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa87fff640 1 -- 172.21.15.26:0/2949282481 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8002d040 con 0x7faa900c88c0 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 -- 172.21.15.26:0/2949282481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa900c88c0 msgr2=0x7faa90155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 --2- 172.21.15.26:0/2949282481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa900c88c0 0x7faa90155c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa80009870 tx=0x7faa8002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 -- 172.21.15.26:0/2949282481 shutdown_connections 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 --2- 172.21.15.26:0/2949282481 >> v2:172.21.15.117:3300/0 conn(0x7faa90158b60 0x7faa90158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 --2- 172.21.15.26:0/2949282481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa900c88c0 0x7faa90155c10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 -- 172.21.15.26:0/2949282481 >> 172.21.15.26:0/2949282481 conn(0x7faa900ac570 msgr2=0x7faa900ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 -- 172.21.15.26:0/2949282481 shutdown_connections 2024-03-06T07:23:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.276+0000 7faa975b6640 1 -- 172.21.15.26:0/2949282481 wait complete. 2024-03-06T07:23:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.277+0000 7faa975b6640 1 Processor -- start 2024-03-06T07:23:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.277+0000 7faa975b6640 1 -- start start 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa975b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa900c88c0 0x7faa900af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa975b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9014f360 0x7faa9014dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa975b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa90158b60 0x7faa90156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa975b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900c5a50 con 0x7faa9014f360 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa975b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900c7c30 con 0x7faa900c88c0 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa975b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa900c5bd0 con 0x7faa90158b60 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa94b51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9014f360 0x7faa9014dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa94b51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9014f360 0x7faa9014dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39332/0 (socket says 172.21.15.26:39332) 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa94b51640 1 -- 172.21.15.26:0/4189183007 learned_addr learned my addr 172.21.15.26:0/4189183007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa95b53640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa90158b60 0x7faa90156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.278+0000 7faa95352640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa900c88c0 0x7faa900af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa94b51640 1 -- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa90158b60 msgr2=0x7faa90156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa94b51640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa90158b60 0x7faa90156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa94b51640 1 -- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa900c88c0 msgr2=0x7faa900af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa94b51640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa900c88c0 0x7faa900af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa94b51640 1 -- 172.21.15.26:0/4189183007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa901ebe20 con 0x7faa9014f360 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa94b51640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9014f360 0x7faa9014dfe0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faa80000c00 tx=0x7faa800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa85ffb640 1 -- 172.21.15.26:0/4189183007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80040070 con 0x7faa9014f360 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa85ffb640 1 -- 172.21.15.26:0/4189183007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa800399f0 con 0x7faa9014f360 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa90158100 con 0x7faa9014f360 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa85ffb640 1 -- 172.21.15.26:0/4189183007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80035040 con 0x7faa9014f360 2024-03-06T07:23:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.279+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa901ebb40 con 0x7faa9014f360 2024-03-06T07:23:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.280+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa901eb930 con 0x7faa9014f360 2024-03-06T07:23:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.281+0000 7faa85ffb640 1 -- 172.21.15.26:0/4189183007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa8004b060 con 0x7faa9014f360 2024-03-06T07:23:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.281+0000 7faa85ffb640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa7406b4d0 0x7faa7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.282+0000 7faa85ffb640 1 -- 172.21.15.26:0/4189183007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7faa800abe70 con 0x7faa9014f360 2024-03-06T07:23:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.282+0000 7faa95352640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa7406b4d0 0x7faa7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.282+0000 7faa95352640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa7406b4d0 0x7faa7406d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa900add40 tx=0x7faa8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.287+0000 7faa85ffb640 1 -- 172.21.15.26:0/4189183007 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa8007a0d0 con 0x7faa9014f360 2024-03-06T07:23:04.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.595+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa9004f330 con 0x7faa9014f360 2024-03-06T07:23:04.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.596+0000 7faa85ffb640 1 -- 172.21.15.26:0/4189183007 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6260 (secure 0 0 0) 0x7faa8007e170 con 0x7faa9014f360 2024-03-06T07:23:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:04.598 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:58.795121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6457":{"gid":6457,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6836/252560219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":252560219},{"type":"v1","addr":"172.21.15.117:6836","nonce":252560219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6457,"qdb_cluster":[6457]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:23:04.598 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa7406b4d0 msgr2=0x7faa7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa7406b4d0 0x7faa7406d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa900add40 tx=0x7faa8c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9014f360 msgr2=0x7faa9014dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9014f360 0x7faa9014dfe0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faa80000c00 tx=0x7faa800046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 shutdown_connections 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faa7406b4d0 0x7faa7406d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa90158b60 0x7faa90156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9014f360 0x7faa9014dfe0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 --2- 172.21.15.26:0/4189183007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa900c88c0 0x7faa900af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 >> 172.21.15.26:0/4189183007 conn(0x7faa900ac570 msgr2=0x7faa900ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 shutdown_connections 2024-03-06T07:23:04.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.598+0000 7faa975b6640 1 -- 172.21.15.26:0/4189183007 wait complete. 2024-03-06T07:23:04.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.710+0000 7f25eba6d640 1 Processor -- start 2024-03-06T07:23:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.710+0000 7f25eba6d640 1 -- start start 2024-03-06T07:23:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25eba6d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25e414f360 0x7f25e4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25eba6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e40c88c0 0x7f25e40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25eba6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e40c73d0 con 0x7f25e40c5880 2024-03-06T07:23:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25eba6d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e40c8dd0 con 0x7f25e414f360 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25eba6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40be2e0 con 0x7f25e40c88c0 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25ea00a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f25e40c5880 0x7f25e4158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52000/0 (socket says 172.21.15.26:52000) 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25ea00a640 1 -- 172.21.15.26:0/667993084 learned_addr learned my addr 172.21.15.26:0/667993084 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25e9008640 1 --2- 172.21.15.26:0/667993084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e40c88c0 0x7f25e40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25e9809640 1 --2- 172.21.15.26:0/667993084 >> v2:172.21.15.117:3300/0 conn(0x7f25e414f360 0x7f25e4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.711+0000 7f25e8807640 1 -- 172.21.15.26:0/667993084 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351365558 0 0) 0x7f25e40c73d0 con 0x7f25e40c5880 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.712+0000 7f25e8807640 1 -- 172.21.15.26:0/667993084 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d0003520 con 0x7f25e40c5880 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.712+0000 7f25e9008640 1 -- 172.21.15.26:0/667993084 >> v2:172.21.15.117:3300/0 conn(0x7f25e414f360 msgr2=0x7f25e4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.712+0000 7f25e9008640 1 --2- 172.21.15.26:0/667993084 >> v2:172.21.15.117:3300/0 conn(0x7f25e414f360 0x7f25e4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.712+0000 7f25e9008640 1 -- 172.21.15.26:0/667993084 >> v1:172.21.15.26:6789/0 conn(0x7f25e40c5880 legacy=0x7f25e4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.712+0000 7f25e9008640 1 -- 172.21.15.26:0/667993084 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40c7670 con 0x7f25e40c88c0 2024-03-06T07:23:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.712+0000 7f25e9008640 1 --2- 172.21.15.26:0/667993084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e40c88c0 0x7f25e40c5340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25d80097a0 tx=0x7f25d802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=972c7a8775cd2b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25e8807640 1 -- 172.21.15.26:0/667993084 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8031780 con 0x7f25e40c88c0 2024-03-06T07:23:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25e8807640 1 -- 172.21.15.26:0/667993084 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8037070 con 0x7f25e40c88c0 2024-03-06T07:23:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25e8807640 1 -- 172.21.15.26:0/667993084 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8031b40 con 0x7f25e40c88c0 2024-03-06T07:23:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 -- 172.21.15.26:0/667993084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e40c88c0 msgr2=0x7f25e40c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 --2- 172.21.15.26:0/667993084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e40c88c0 0x7f25e40c5340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25d80097a0 tx=0x7f25d802b820 comp rx=0 tx=0).stop 2024-03-06T07:23:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 -- 172.21.15.26:0/667993084 shutdown_connections 2024-03-06T07:23:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 --2- 172.21.15.26:0/667993084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e40c88c0 0x7f25e40c5340 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 --2- 172.21.15.26:0/667993084 >> v2:172.21.15.117:3300/0 conn(0x7f25e414f360 0x7f25e4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 -- 172.21.15.26:0/667993084 >> 172.21.15.26:0/667993084 conn(0x7f25e40ac570 msgr2=0x7f25e40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 -- 172.21.15.26:0/667993084 shutdown_connections 2024-03-06T07:23:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.713+0000 7f25eba6d640 1 -- 172.21.15.26:0/667993084 wait complete. 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.715+0000 7f25eba6d640 1 Processor -- start 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.715+0000 7f25eba6d640 1 -- start start 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25eba6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25e40c5880 0x7f25e40ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25eba6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e40c88c0 0x7f25e41ee650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25eba6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e414f360 0x7f25e41eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25eba6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e4159270 con 0x7f25e40c5880 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25eba6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40c7c30 con 0x7f25e40c88c0 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25eba6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e41593f0 con 0x7f25e414f360 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25e40c5880 0x7f25e40ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25e40c5880 0x7f25e40ab920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39342/0 (socket says 172.21.15.26:39342) 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 -- 172.21.15.26:0/4184722475 learned_addr learned my addr 172.21.15.26:0/4184722475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9008640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e40c88c0 0x7f25e41ee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25ea00a640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e414f360 0x7f25e41eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 -- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e414f360 msgr2=0x7f25e41eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e414f360 0x7f25e41eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 -- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e40c88c0 msgr2=0x7f25e41ee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e40c88c0 0x7f25e41ee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 -- 172.21.15.26:0/4184722475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40c2be0 con 0x7f25e40c5880 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25e9809640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25e40c5880 0x7f25e40ab920 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e8a0 tx=0x7f25e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.716+0000 7f25d67fc640 1 -- 172.21.15.26:0/4184722475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0017070 con 0x7f25e40c5880 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.717+0000 7f25d67fc640 1 -- 172.21.15.26:0/4184722475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e000c940 con 0x7f25e40c5880 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.717+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e40c38f0 con 0x7f25e40c5880 2024-03-06T07:23:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.717+0000 7f25d67fc640 1 -- 172.21.15.26:0/4184722475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000cd30 con 0x7f25e40c5880 2024-03-06T07:23:04.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.717+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e40c3eb0 con 0x7f25e40c5880 2024-03-06T07:23:04.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.718+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e4068f00 con 0x7f25e40c5880 2024-03-06T07:23:04.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.718+0000 7f25d67fc640 1 -- 172.21.15.26:0/4184722475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25e001a070 con 0x7f25e40c5880 2024-03-06T07:23:04.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.718+0000 7f25d67fc640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c806b4d0 0x7f25c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.719+0000 7f25d67fc640 1 -- 172.21.15.26:0/4184722475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f25e008a8c0 con 0x7f25e40c5880 2024-03-06T07:23:04.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.719+0000 7f25e9008640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c806b4d0 0x7f25c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.719+0000 7f25e9008640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c806b4d0 0x7f25c806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25d8036000 tx=0x7f25d803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:04.727+0000 7f25d67fc640 1 -- 172.21.15.26:0/4184722475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f25e4068f00 con 0x7f25e40c5880 2024-03-06T07:23:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.047+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25e4069460 con 0x7f25e40c5880 2024-03-06T07:23:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.048+0000 7f25d67fc640 1 -- 172.21.15.26:0/4184722475 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6260 (secure 0 0 0) 0x7f25e4069460 con 0x7f25e40c5880 2024-03-06T07:23:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:05.049 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.395648+0000","modified":"2024-03-06T07:22:58.795121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6457":{"gid":6457,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6836/252560219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":252560219},{"type":"v1","addr":"172.21.15.117:6836","nonce":252560219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6457,"qdb_cluster":[6457]},"id":5},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:22:46.725776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6327":{"gid":6327,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3035454765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3035454765},{"type":"v1","addr":"172.21.15.26:6835","nonce":3035454765}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6327,"qdb_cluster":[6327]},"id":6}]} 2024-03-06T07:23:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2024-03-06T07:23:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c806b4d0 msgr2=0x7f25c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c806b4d0 0x7f25c806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25d8036000 tx=0x7f25d803d040 comp rx=0 tx=0).stop 2024-03-06T07:23:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25e40c5880 msgr2=0x7f25e40ab920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25e40c5880 0x7f25e40ab920 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e8a0 tx=0x7f25e000ed60 comp rx=0 tx=0).stop 2024-03-06T07:23:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 shutdown_connections 2024-03-06T07:23:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25c806b4d0 0x7f25c806d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25e414f360 0x7f25e41eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e40c88c0 0x7f25e41ee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 --2- 172.21.15.26:0/4184722475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25e40c5880 0x7f25e40ab920 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 >> 172.21.15.26:0/4184722475 conn(0x7f25e40ac570 msgr2=0x7f25e414dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 shutdown_connections 2024-03-06T07:23:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.050+0000 7f25eba6d640 1 -- 172.21.15.26:0/4184722475 wait complete. 2024-03-06T07:23:05.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.163+0000 7f41c4c1c640 1 Processor -- start 2024-03-06T07:23:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.163+0000 7f41c4c1c640 1 -- start start 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41c4c1c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f41c0072e70 0x7f41c00bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41c4c1c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41c0073250 0x7f41c00c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41c4c1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c01589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41c4c1c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c006c8e0 con 0x7f41c0073250 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41c4c1c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c006ce80 con 0x7f41c0072e70 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41c4c1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00c1b00 con 0x7f41c00c3950 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41c0073250 0x7f41c00c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41c0073250 0x7f41c00c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39350/0 (socket says 172.21.15.26:39350) 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 -- 172.21.15.26:0/3363743787 learned_addr learned my addr 172.21.15.26:0/3363743787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41be59c640 1 --2- 172.21.15.26:0/3363743787 >> v2:172.21.15.117:3300/0 conn(0x7f41c0072e70 0x7f41c00bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bed9d640 1 --2- 172.21.15.26:0/3363743787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c01589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 -- 172.21.15.26:0/3363743787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 msgr2=0x7f41c01589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 --2- 172.21.15.26:0/3363743787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c01589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 -- 172.21.15.26:0/3363743787 >> v2:172.21.15.117:3300/0 conn(0x7f41c0072e70 msgr2=0x7f41c00bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 --2- 172.21.15.26:0/3363743787 >> v2:172.21.15.117:3300/0 conn(0x7f41c0072e70 0x7f41c00bd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.164+0000 7f41bdd9b640 1 -- 172.21.15.26:0/3363743787 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c006cba0 con 0x7f41c0073250 2024-03-06T07:23:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.165+0000 7f41bdd9b640 1 --2- 172.21.15.26:0/3363743787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0073250 0x7f41c00c3410 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f41c00c4000 tx=0x7f41b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfed2e31d94d5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.165+0000 7f41bd59a640 1 -- 172.21.15.26:0/3363743787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b002e070 con 0x7f41c0073250 2024-03-06T07:23:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.165+0000 7f41bd59a640 1 -- 172.21.15.26:0/3363743787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0037070 con 0x7f41c0073250 2024-03-06T07:23:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.165+0000 7f41bd59a640 1 -- 172.21.15.26:0/3363743787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0031a00 con 0x7f41c0073250 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 -- 172.21.15.26:0/3363743787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0073250 msgr2=0x7f41c00c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/3363743787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0073250 0x7f41c00c3410 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f41c00c4000 tx=0x7f41b002b880 comp rx=0 tx=0).stop 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 -- 172.21.15.26:0/3363743787 shutdown_connections 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/3363743787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c01589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/3363743787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0073250 0x7f41c00c3410 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/3363743787 >> v2:172.21.15.117:3300/0 conn(0x7f41c0072e70 0x7f41c00bd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 -- 172.21.15.26:0/3363743787 >> 172.21.15.26:0/3363743787 conn(0x7f41c006b1a0 msgr2=0x7f41c006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 -- 172.21.15.26:0/3363743787 shutdown_connections 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.166+0000 7f41c4c1c640 1 -- 172.21.15.26:0/3363743787 wait complete. 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.167+0000 7f41c4c1c640 1 Processor -- start 2024-03-06T07:23:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.167+0000 7f41c4c1c640 1 -- start start 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41c4c1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0072e70 0x7f41c00c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41c4c1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41c0073250 0x7f41c01e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41c4c1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c00a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41c4c1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00c4b80 con 0x7f41c0072e70 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41c4c1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00c4a00 con 0x7f41c0073250 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41c4c1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00bda50 con 0x7f41c00c3950 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0072e70 0x7f41c00c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0072e70 0x7f41c00c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39360/0 (socket says 172.21.15.26:39360) 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 -- 172.21.15.26:0/4011480269 learned_addr learned my addr 172.21.15.26:0/4011480269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41bed9d640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c00a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41bdd9b640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41c0073250 0x7f41c01e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 -- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 msgr2=0x7f41c00a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c00a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 -- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41c0073250 msgr2=0x7f41c01e0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41c0073250 0x7f41c01e0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 -- 172.21.15.26:0/4011480269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c01eb1a0 con 0x7f41c0072e70 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41bdd9b640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41c0073250 0x7f41c01e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.168+0000 7f41be59c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0072e70 0x7f41c00c21e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f41a800e870 tx=0x7f41a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.169+0000 7f41b6ffd640 1 -- 172.21.15.26:0/4011480269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8017070 con 0x7f41c0072e70 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.169+0000 7f41b6ffd640 1 -- 172.21.15.26:0/4011480269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a800ee60 con 0x7f41c0072e70 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.169+0000 7f41b6ffd640 1 -- 172.21.15.26:0/4011480269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a800cb20 con 0x7f41c0072e70 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.169+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c01eb480 con 0x7f41c0072e70 2024-03-06T07:23:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.169+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c01eba40 con 0x7f41c0072e70 2024-03-06T07:23:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.170+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c0070af0 con 0x7f41c0072e70 2024-03-06T07:23:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.170+0000 7f41b6ffd640 1 -- 172.21.15.26:0/4011480269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41a801a070 con 0x7f41c0072e70 2024-03-06T07:23:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.170+0000 7f41b6ffd640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41a006b4d0 0x7f41a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.171+0000 7f41b6ffd640 1 -- 172.21.15.26:0/4011480269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f41a808a7a0 con 0x7f41c0072e70 2024-03-06T07:23:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.171+0000 7f41bdd9b640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41a006b4d0 0x7f41a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.171+0000 7f41bdd9b640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41a006b4d0 0x7f41a006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41b0036000 tx=0x7f41b003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.179+0000 7f41b6ffd640 1 -- 172.21.15.26:0/4011480269 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f41c0070af0 con 0x7f41c0072e70 2024-03-06T07:23:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.366+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41c0072570 con 0x7f41c0072e70 2024-03-06T07:23:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.369+0000 7f41b6ffd640 1 -- 172.21.15.26:0/4011480269 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+20810 (secure 0 0 0) 0x7f41c0072570 con 0x7f41c0072e70 2024-03-06T07:23:05.370 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:05.371 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:22:54.778055+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.646079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.350567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/218606650":"2024-03-07T07:22:54.778025+0000","172.21.15.117:6834/218606650":"2024-03-07T07:22:54.778025+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41a006b4d0 msgr2=0x7f41a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41a006b4d0 0x7f41a006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41b0036000 tx=0x7f41b003d040 comp rx=0 tx=0).stop 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0072e70 msgr2=0x7f41c00c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0072e70 0x7f41c00c21e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f41a800e870 tx=0x7f41a800ed30 comp rx=0 tx=0).stop 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 shutdown_connections 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41a006b4d0 0x7f41a006d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41c00c3950 0x7f41c00a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41c0073250 0x7f41c01e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 --2- 172.21.15.26:0/4011480269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41c0072e70 0x7f41c00c21e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 >> 172.21.15.26:0/4011480269 conn(0x7f41c006b1a0 msgr2=0x7f41c006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 shutdown_connections 2024-03-06T07:23:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.371+0000 7f41c4c1c640 1 -- 172.21.15.26:0/4011480269 wait complete. 2024-03-06T07:23:05.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:23:05.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.485+0000 7f6978cb1640 1 Processor -- start 2024-03-06T07:23:05.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.485+0000 7f6978cb1640 1 -- start start 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f6978cb1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974158770 0x7f6974158b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f6978cb1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6974072d20 0x7f697415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f6978cb1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6974159090 con 0x7f6974073270 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f6978cb1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69740a7fd0 con 0x7f6974072d20 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f6978cb1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69740c26c0 con 0x7f6974158770 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f6972d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6974072d20 0x7f697415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f697259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974158770 0x7f6974158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f696bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6974073270 0x7f69740724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52002/0 (socket says 172.21.15.26:52002) 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.486+0000 7f696bfff640 1 -- 172.21.15.26:0/3950105997 learned_addr learned my addr 172.21.15.26:0/3950105997 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:05.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.487+0000 7f6972d9d640 1 -- 172.21.15.26:0/3950105997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974158770 msgr2=0x7f6974158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.487+0000 7f6972d9d640 1 --2- 172.21.15.26:0/3950105997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974158770 0x7f6974158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.487+0000 7f6972d9d640 1 -- 172.21.15.26:0/3950105997 >> v1:172.21.15.26:6789/0 conn(0x7f6974073270 legacy=0x7f69740724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.487+0000 7f6972d9d640 1 -- 172.21.15.26:0/3950105997 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697406f230 con 0x7f6974072d20 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.487+0000 7f6971d9b640 1 -- 172.21.15.26:0/3950105997 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700809363 0 0) 0x7f6974159090 con 0x7f6974073270 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.487+0000 7f697259c640 1 --2- 172.21.15.26:0/3950105997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974158770 0x7f6974158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.487+0000 7f6972d9d640 1 --2- 172.21.15.26:0/3950105997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f697415fae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f696c009d90 tx=0x7f696c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9aed16561b6c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6971d9b640 1 -- 172.21.15.26:0/3950105997 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c039070 con 0x7f6974072d20 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6971d9b640 1 -- 172.21.15.26:0/3950105997 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c02cb30 con 0x7f6974072d20 2024-03-06T07:23:05.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6971d9b640 1 -- 172.21.15.26:0/3950105997 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c034440 con 0x7f6974072d20 2024-03-06T07:23:05.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 -- 172.21.15.26:0/3950105997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 msgr2=0x7f697415fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 --2- 172.21.15.26:0/3950105997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f697415fae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f696c009d90 tx=0x7f696c02bd80 comp rx=0 tx=0).stop 2024-03-06T07:23:05.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 -- 172.21.15.26:0/3950105997 shutdown_connections 2024-03-06T07:23:05.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 --2- 172.21.15.26:0/3950105997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f697415fae0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 --2- 172.21.15.26:0/3950105997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974158770 0x7f6974158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 -- 172.21.15.26:0/3950105997 >> 172.21.15.26:0/3950105997 conn(0x7f697406b970 msgr2=0x7f69740aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 -- 172.21.15.26:0/3950105997 shutdown_connections 2024-03-06T07:23:05.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.488+0000 7f6978cb1640 1 -- 172.21.15.26:0/3950105997 wait complete. 2024-03-06T07:23:05.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.490+0000 7f6978cb1640 1 Processor -- start 2024-03-06T07:23:05.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.490+0000 7f6978cb1640 1 -- start start 2024-03-06T07:23:05.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6978cb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f6974151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6978cb1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974073270 0x7f69741ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6978cb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6974158770 0x7f6974156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6978cb1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69740c2910 con 0x7f6974072d20 2024-03-06T07:23:05.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6978cb1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69740a8220 con 0x7f6974158770 2024-03-06T07:23:05.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6978cb1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697406e2e0 con 0x7f6974073270 2024-03-06T07:23:05.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f6974151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f6974151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39366/0 (socket says 172.21.15.26:39366) 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 -- 172.21.15.26:0/2649217698 learned_addr learned my addr 172.21.15.26:0/2649217698 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f696bfff640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974073270 0x7f69741ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6972d9d640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6974158770 0x7f6974156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 -- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974073270 msgr2=0x7f69741ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974073270 0x7f69741ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 -- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6974158770 msgr2=0x7f6974156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6974158770 0x7f6974156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f697259c640 1 -- 172.21.15.26:0/2649217698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69741eec20 con 0x7f6974072d20 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.491+0000 7f6972d9d640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6974158770 0x7f6974156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.492+0000 7f697259c640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f6974151f50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f696400e960 tx=0x7f696400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.492+0000 7f696affd640 1 -- 172.21.15.26:0/2649217698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964017070 con 0x7f6974072d20 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.492+0000 7f696affd640 1 -- 172.21.15.26:0/2649217698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696400c9a0 con 0x7f6974072d20 2024-03-06T07:23:05.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.492+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6974156a90 con 0x7f6974072d20 2024-03-06T07:23:05.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.492+0000 7f696affd640 1 -- 172.21.15.26:0/2649217698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696400cd70 con 0x7f6974072d20 2024-03-06T07:23:05.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.492+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69740a95f0 con 0x7f6974072d20 2024-03-06T07:23:05.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.493+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697404f330 con 0x7f6974072d20 2024-03-06T07:23:05.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.493+0000 7f696affd640 1 -- 172.21.15.26:0/2649217698 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f696401a070 con 0x7f6974072d20 2024-03-06T07:23:05.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.494+0000 7f696affd640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f695406b4d0 0x7f695406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.494+0000 7f696affd640 1 -- 172.21.15.26:0/2649217698 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f696408a4b0 con 0x7f6974072d20 2024-03-06T07:23:05.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.494+0000 7f696bfff640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f695406b4d0 0x7f695406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.494+0000 7f696bfff640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f695406b4d0 0x7f695406d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f697406d580 tx=0x7f695c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.498+0000 7f696affd640 1 -- 172.21.15.26:0/2649217698 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6964058830 con 0x7f6974072d20 2024-03-06T07:23:05.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.797+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f69740a97f0 con 0x7f6974072d20 2024-03-06T07:23:05.799 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:23:05.798+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:23:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.799+0000 7f696affd640 1 -- 172.21.15.26:0/2649217698 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f696405c8d0 con 0x7f6974072d20 2024-03-06T07:23:05.801 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f695406b4d0 msgr2=0x7f695406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f695406b4d0 0x7f695406d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f697406d580 tx=0x7f695c007de0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 msgr2=0x7f6974151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f6974151f50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f696400e960 tx=0x7f696400ee20 comp rx=0 tx=0).stop 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 shutdown_connections 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f695406b4d0 0x7f695406d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6974158770 0x7f6974156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6974073270 0x7f69741ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 --2- 172.21.15.26:0/2649217698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974072d20 0x7f6974151f50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 >> 172.21.15.26:0/2649217698 conn(0x7f697406b970 msgr2=0x7f6974158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 shutdown_connections 2024-03-06T07:23:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.802+0000 7f6978cb1640 1 -- 172.21.15.26:0/2649217698 wait complete. 2024-03-06T07:23:05.808 INFO:tasks.ceph.mds.b.smithi117.stderr: -158> 2024-03-06T07:23:01.787+0000 7fc38e93e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:05.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:23:05.832 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:23:05.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.919+0000 7f121ae66640 1 Processor -- start 2024-03-06T07:23:05.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.919+0000 7f121ae66640 1 -- start start 2024-03-06T07:23:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f121ae66640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12140b0070 0x7f1214155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f121ae66640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12140c5380 0x7f12140c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f121ae66640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12140b0450 con 0x7f121406ca70 2024-03-06T07:23:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f121ae66640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121406c020 con 0x7f12140c5380 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f121ae66640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140c77f0 con 0x7f12140b0070 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f1213fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f121406ca70 0x7f12140c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:57272/0 (socket says 172.21.15.26:57272) 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f1213fff640 1 -- 172.21.15.26:0/3253851501 learned_addr learned my addr 172.21.15.26:0/3253851501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f1219403640 1 --2- 172.21.15.26:0/3253851501 >> v2:172.21.15.117:3300/0 conn(0x7f12140c5380 0x7f12140c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f1218c02640 1 --2- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12140b0070 0x7f1214155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.920+0000 7f12137fe640 1 -- 172.21.15.26:0/3253851501 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637228346 0 0) 0x7f11fc001110 con 0x7f121406ca70 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f12137fe640 1 -- 172.21.15.26:0/3253851501 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1200003520 con 0x7f121406ca70 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f1219403640 1 -- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12140b0070 msgr2=0x7f1214155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f1219403640 1 --2- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12140b0070 0x7f1214155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f1219403640 1 -- 172.21.15.26:0/3253851501 >> v1:172.21.15.117:6789/0 conn(0x7f121406ca70 legacy=0x7f12140c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f1219403640 1 -- 172.21.15.26:0/3253851501 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140c7ad0 con 0x7f12140c5380 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f1219403640 1 --2- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 0x7f12140c57d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f12040097a0 tx=0x7f120402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e26aa7860694080 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f12137fe640 1 -- 172.21.15.26:0/3253851501 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040315a0 con 0x7f12140c5380 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f12137fe640 1 -- 172.21.15.26:0/3253851501 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1204037070 con 0x7f12140c5380 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.921+0000 7f12137fe640 1 -- 172.21.15.26:0/3253851501 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204031960 con 0x7f12140c5380 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 -- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 msgr2=0x7f12140c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 --2- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 0x7f12140c57d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f12040097a0 tx=0x7f120402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 -- 172.21.15.26:0/3253851501 shutdown_connections 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 --2- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 0x7f12140c57d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 --2- 172.21.15.26:0/3253851501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12140b0070 0x7f1214155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 -- 172.21.15.26:0/3253851501 >> 172.21.15.26:0/3253851501 conn(0x7f12140ac590 msgr2=0x7f12140ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 -- 172.21.15.26:0/3253851501 shutdown_connections 2024-03-06T07:23:05.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.922+0000 7f121ae66640 1 -- 172.21.15.26:0/3253851501 wait complete. 2024-03-06T07:23:05.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.923+0000 7f121ae66640 1 Processor -- start 2024-03-06T07:23:05.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.923+0000 7f121ae66640 1 -- start start 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f121ae66640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f121406ca70 0x7f12140c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f121ae66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140b0070 0x7f12141541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f121ae66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 0x7f12141ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f121ae66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121406cef0 con 0x7f12140b0070 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f121ae66640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121406c550 con 0x7f12140c5380 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f121ae66640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121406da80 con 0x7f121406ca70 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1213fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140b0070 0x7f12141541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1218c02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f121406ca70 0x7f12140c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1213fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140b0070 0x7f12141541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39372/0 (socket says 172.21.15.26:39372) 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1213fff640 1 -- 172.21.15.26:0/546031116 learned_addr learned my addr 172.21.15.26:0/546031116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1219403640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 0x7f12141ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1218c02640 1 -- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 msgr2=0x7f12141ebee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1218c02640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 0x7f12141ebee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1218c02640 1 -- 172.21.15.26:0/546031116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140b0070 msgr2=0x7f12141541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1218c02640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140b0070 0x7f12141541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.924+0000 7f1218c02640 1 -- 172.21.15.26:0/546031116 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214154d40 con 0x7f121406ca70 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.925+0000 7f1218c02640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f121406ca70 0x7f12140c2750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f120c009db0 tx=0x7f120c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.925+0000 7f12117fa640 1 -- 172.21.15.26:0/546031116 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c018020 con 0x7f121406ca70 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.925+0000 7f12117fa640 1 -- 172.21.15.26:0/546031116 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c014480 con 0x7f121406ca70 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.925+0000 7f12117fa640 1 -- 172.21.15.26:0/546031116 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c014850 con 0x7f121406ca70 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.925+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1214157210 con 0x7f121406ca70 2024-03-06T07:23:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.925+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12140c1f50 con 0x7f121406ca70 2024-03-06T07:23:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.926+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121404f330 con 0x7f121406ca70 2024-03-06T07:23:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.926+0000 7f12117fa640 1 -- 172.21.15.26:0/546031116 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f120c005d10 con 0x7f121406ca70 2024-03-06T07:23:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.927+0000 7f12117fa640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11f406b4d0 0x7f11f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.927+0000 7f12117fa640 1 -- 172.21.15.26:0/546031116 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f120c08b3e0 con 0x7f121406ca70 2024-03-06T07:23:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.927+0000 7f1213fff640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11f406b4d0 0x7f11f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.927+0000 7f1213fff640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11f406b4d0 0x7f11f406d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1214068e20 tx=0x7f11fc007bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:05.931+0000 7f12117fa640 1 -- 172.21.15.26:0/546031116 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f120c022030 con 0x7f121406ca70 2024-03-06T07:23:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.155+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f121406d860 con 0x7f121406ca70 2024-03-06T07:23:06.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.831+0000 7f12117fa640 1 -- 172.21.15.26:0/546031116 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f120c010040 con 0x7f121406ca70 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11f406b4d0 msgr2=0x7f11f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11f406b4d0 0x7f11f406d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1214068e20 tx=0x7f11fc007bd0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f121406ca70 msgr2=0x7f12140c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f121406ca70 0x7f12140c2750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f120c009db0 tx=0x7f120c00c490 comp rx=0 tx=0).stop 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 shutdown_connections 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11f406b4d0 0x7f11f406d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12140c5380 0x7f12141ebee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140b0070 0x7f12141541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 --2- 172.21.15.26:0/546031116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f121406ca70 0x7f12140c2750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 >> 172.21.15.26:0/546031116 conn(0x7f12140ac590 msgr2=0x7f12140ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 shutdown_connections 2024-03-06T07:23:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.834+0000 7f121ae66640 1 -- 172.21.15.26:0/546031116 wait complete. 2024-03-06T07:23:06.841 INFO:tasks.ceph.mds.c.smithi026.stderr: -553> 2024-03-06T07:22:50.308+0000 7fbf748fb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:06.849 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:23:06.868 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:23:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.957+0000 7f6e2e9d7640 1 Processor -- start 2024-03-06T07:23:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.957+0000 7f6e2e9d7640 1 -- start start 2024-03-06T07:23:06.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.957+0000 7f6e2e9d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e280abb60 0x7f6e281502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.957+0000 7f6e2e9d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e280c2920 0x7f6e28156b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.957+0000 7f6e2e9d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e28070430 con 0x7f6e280c1d10 2024-03-06T07:23:06.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.957+0000 7f6e2e9d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e280711f0 con 0x7f6e280abb60 2024-03-06T07:23:06.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.957+0000 7f6e2e9d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28070d60 con 0x7f6e280c2920 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e27fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e280abb60 0x7f6e281502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e277fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e280c1d10 0x7f6e28156440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52018/0 (socket says 172.21.15.26:52018) 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e277fe640 1 -- 172.21.15.26:0/1284698583 learned_addr learned my addr 172.21.15.26:0/1284698583 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e2cf74640 1 --2- 172.21.15.26:0/1284698583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e280c2920 0x7f6e28156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e27fff640 1 -- 172.21.15.26:0/1284698583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e280c2920 msgr2=0x7f6e28156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e27fff640 1 --2- 172.21.15.26:0/1284698583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e280c2920 0x7f6e28156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e27fff640 1 -- 172.21.15.26:0/1284698583 >> v1:172.21.15.26:6789/0 conn(0x7f6e280c1d10 legacy=0x7f6e28156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e27fff640 1 -- 172.21.15.26:0/1284698583 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e28070710 con 0x7f6e280abb60 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.958+0000 7f6e26ffd640 1 -- 172.21.15.26:0/1284698583 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195588372 0 0) 0x7f6e28070430 con 0x7f6e280c1d10 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e27fff640 1 --2- 172.21.15.26:0/1284698583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281502f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6e180097a0 tx=0x7f6e1802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e1f33eb2cb4e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e26ffd640 1 -- 172.21.15.26:0/1284698583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e180315a0 con 0x7f6e280abb60 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e26ffd640 1 -- 172.21.15.26:0/1284698583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18037070 con 0x7f6e280abb60 2024-03-06T07:23:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e26ffd640 1 -- 172.21.15.26:0/1284698583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18031960 con 0x7f6e280abb60 2024-03-06T07:23:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1284698583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 msgr2=0x7f6e281502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1284698583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281502f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6e180097a0 tx=0x7f6e1802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1284698583 shutdown_connections 2024-03-06T07:23:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1284698583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e280c2920 0x7f6e28156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1284698583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281502f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1284698583 >> 172.21.15.26:0/1284698583 conn(0x7f6e280c65b0 msgr2=0x7f6e280c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:06.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1284698583 shutdown_connections 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.959+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1284698583 wait complete. 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.960+0000 7f6e2e9d7640 1 Processor -- start 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e2e9d7640 1 -- start start 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e2e9d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e2e9d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e280c2920 0x7f6e281511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e2e9d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e281536c0 0x7f6e281546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e2e9d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e280abf40 con 0x7f6e280abb60 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e2e9d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28157090 con 0x7f6e280c2920 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e2e9d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e280ac0c0 con 0x7f6e281536c0 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281df8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39406/0 (socket says 172.21.15.26:39406) 2024-03-06T07:23:06.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.961+0000 7f6e27fff640 1 -- 172.21.15.26:0/1950663440 learned_addr learned my addr 172.21.15.26:0/1950663440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e27fff640 1 -- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e281536c0 msgr2=0x7f6e281546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e27fff640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e281536c0 0x7f6e281546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e27fff640 1 -- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e280c2920 msgr2=0x7f6e281511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e27fff640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e280c2920 0x7f6e281511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e27fff640 1 -- 172.21.15.26:0/1950663440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e28152fa0 con 0x7f6e280abb60 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e27fff640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281df8d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6e180098c0 tx=0x7f6e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e24ff9640 1 -- 172.21.15.26:0/1950663440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18032840 con 0x7f6e280abb60 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e24ff9640 1 -- 172.21.15.26:0/1950663440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18032a00 con 0x7f6e280abb60 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e24ff9640 1 -- 172.21.15.26:0/1950663440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18032df0 con 0x7f6e280abb60 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e28151730 con 0x7f6e280abb60 2024-03-06T07:23:06.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.962+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e281519f0 con 0x7f6e280abb60 2024-03-06T07:23:06.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.963+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e28067ad0 con 0x7f6e280abb60 2024-03-06T07:23:06.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.964+0000 7f6e24ff9640 1 -- 172.21.15.26:0/1950663440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e1804e430 con 0x7f6e280abb60 2024-03-06T07:23:06.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.964+0000 7f6e24ff9640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e1406b4d0 0x7f6e1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.964+0000 7f6e24ff9640 1 -- 172.21.15.26:0/1950663440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7215+0+0 (secure 0 0 0) 0x7f6e1802f030 con 0x7f6e280abb60 2024-03-06T07:23:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.964+0000 7f6e277fe640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e1406b4d0 0x7f6e1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.965+0000 7f6e277fe640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e1406b4d0 0x7f6e1406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c007fd0 tx=0x7f6e1c007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:06.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:06.968+0000 7f6e24ff9640 1 -- 172.21.15.26:0/1950663440 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6e28067ad0 con 0x7f6e280abb60 2024-03-06T07:23:07.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.150+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e2804f330 con 0x7f6e280abb60 2024-03-06T07:23:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.839+0000 7f6e24ff9640 1 -- 172.21.15.26:0/1950663440 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6e18040070 con 0x7f6e280abb60 2024-03-06T07:23:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:23:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e1406b4d0 msgr2=0x7f6e1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e1406b4d0 0x7f6e1406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c007fd0 tx=0x7f6e1c007920 comp rx=0 tx=0).stop 2024-03-06T07:23:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 msgr2=0x7f6e281df8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281df8d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6e180098c0 tx=0x7f6e180046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 shutdown_connections 2024-03-06T07:23:07.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e1406b4d0 0x7f6e1406d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:07.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e281536c0 0x7f6e281546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:07.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e280c2920 0x7f6e281511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:07.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 --2- 172.21.15.26:0/1950663440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e280abb60 0x7f6e281df8d0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:07.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.842+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 >> 172.21.15.26:0/1950663440 conn(0x7f6e280c65b0 msgr2=0x7f6e280c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:07.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.843+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 shutdown_connections 2024-03-06T07:23:07.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:07.843+0000 7f6e2e9d7640 1 -- 172.21.15.26:0/1950663440 wait complete. 2024-03-06T07:23:07.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:23:08.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 Processor -- start 2024-03-06T07:23:08.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 -- start start 2024-03-06T07:23:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70152920 0x7f9a70155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9a70155eb0 0x7f9a700c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a700c80d0 0x7f9a700724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a70159740 con 0x7f9a700c80d0 2024-03-06T07:23:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a701551d0 con 0x7f9a70155eb0 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.008+0000 7f9a765ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700c8ab0 con 0x7f9a70152920 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a700c80d0 0x7f9a700724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a700c80d0 0x7f9a700724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60718/0 (socket says 172.21.15.26:60718) 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 -- 172.21.15.26:0/2538301262 learned_addr learned my addr 172.21.15.26:0/2538301262 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 -- 172.21.15.26:0/2538301262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70152920 msgr2=0x7f9a70155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a6ffff640 1 --2- 172.21.15.26:0/2538301262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70152920 0x7f9a70155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a6f7fe640 1 --2- 172.21.15.26:0/2538301262 >> v2:172.21.15.117:3300/0 conn(0x7f9a70155eb0 0x7f9a700c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 --2- 172.21.15.26:0/2538301262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70152920 0x7f9a70155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 -- 172.21.15.26:0/2538301262 >> v2:172.21.15.117:3300/0 conn(0x7f9a70155eb0 msgr2=0x7f9a700c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 --2- 172.21.15.26:0/2538301262 >> v2:172.21.15.117:3300/0 conn(0x7f9a70155eb0 0x7f9a700c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.009+0000 7f9a74b49640 1 -- 172.21.15.26:0/2538301262 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700c8d90 con 0x7f9a700c80d0 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a74b49640 1 --2- 172.21.15.26:0/2538301262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a700724f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9a7006bf30 tx=0x7f9a6802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98349a166a1075ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a6effd640 1 -- 172.21.15.26:0/2538301262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6802c9e0 con 0x7f9a700c80d0 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a6effd640 1 -- 172.21.15.26:0/2538301262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6802cb80 con 0x7f9a700c80d0 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a6effd640 1 -- 172.21.15.26:0/2538301262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a680343f0 con 0x7f9a700c80d0 2024-03-06T07:23:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a765ac640 1 -- 172.21.15.26:0/2538301262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 msgr2=0x7f9a700724f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2538301262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a700724f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9a7006bf30 tx=0x7f9a6802be40 comp rx=0 tx=0).stop 2024-03-06T07:23:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a765ac640 1 -- 172.21.15.26:0/2538301262 shutdown_connections 2024-03-06T07:23:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2538301262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a700724f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2538301262 >> v2:172.21.15.117:3300/0 conn(0x7f9a70155eb0 0x7f9a700c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2538301262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70152920 0x7f9a70155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.010+0000 7f9a765ac640 1 -- 172.21.15.26:0/2538301262 >> 172.21.15.26:0/2538301262 conn(0x7f9a700aa5c0 msgr2=0x7f9a700aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.011+0000 7f9a765ac640 1 -- 172.21.15.26:0/2538301262 shutdown_connections 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.011+0000 7f9a765ac640 1 -- 172.21.15.26:0/2538301262 wait complete. 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 Processor -- start 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 -- start start 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a701eb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a70152920 0x7f9a701ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70155eb0 0x7f9a700acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700c86c0 con 0x7f9a700c80d0 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70155450 con 0x7f9a70152920 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.012+0000 7f9a765ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700c8840 con 0x7f9a70155eb0 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a701eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a701eb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60730/0 (socket says 172.21.15.26:60730) 2024-03-06T07:23:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 -- 172.21.15.26:0/2327988642 learned_addr learned my addr 172.21.15.26:0/2327988642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 -- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70155eb0 msgr2=0x7f9a700acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70155eb0 0x7f9a700acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 -- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a70152920 msgr2=0x7f9a701ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a70152920 0x7f9a701ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 -- 172.21.15.26:0/2327988642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a701eb850 con 0x7f9a700c80d0 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6ffff640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a701eb310 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c7f0 tx=0x7f9a6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6cff9640 1 -- 172.21.15.26:0/2327988642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60015070 con 0x7f9a700c80d0 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6cff9640 1 -- 172.21.15.26:0/2327988642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a60004220 con 0x7f9a700c80d0 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a701eed30 con 0x7f9a700c80d0 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a6cff9640 1 -- 172.21.15.26:0/2327988642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60007620 con 0x7f9a700c80d0 2024-03-06T07:23:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.013+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a70157d50 con 0x7f9a700c80d0 2024-03-06T07:23:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.014+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7004f330 con 0x7f9a700c80d0 2024-03-06T07:23:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.015+0000 7f9a6cff9640 1 -- 172.21.15.26:0/2327988642 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a600079d0 con 0x7f9a700c80d0 2024-03-06T07:23:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.015+0000 7f9a6cff9640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.015+0000 7f9a6cff9640 1 -- 172.21.15.26:0/2327988642 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f9a6008a6f0 con 0x7f9a700c80d0 2024-03-06T07:23:08.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.015+0000 7f9a6f7fe640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.016+0000 7f9a6f7fe640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9a700aaaf0 tx=0x7f9a64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.019+0000 7f9a6cff9640 1 -- 172.21.15.26:0/2327988642 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9a600590b0 con 0x7f9a700c80d0 2024-03-06T07:23:08.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.204+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a700c1fd0 con 0x7f9a700c80d0 2024-03-06T07:23:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.844+0000 7f9a6cff9640 1 -- 172.21.15.26:0/2327988642 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9a600589b0 con 0x7f9a700c80d0 2024-03-06T07:23:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:23:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.846+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a5c06b4d0 msgr2=0x7f9a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9a700aaaf0 tx=0x7f9a64008040 comp rx=0 tx=0).stop 2024-03-06T07:23:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 msgr2=0x7f9a701eb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a701eb310 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c7f0 tx=0x7f9a6000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 shutdown_connections 2024-03-06T07:23:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a70155eb0 0x7f9a700acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a70152920 0x7f9a701ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 --2- 172.21.15.26:0/2327988642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a700c80d0 0x7f9a701eb310 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 >> 172.21.15.26:0/2327988642 conn(0x7f9a700aa5c0 msgr2=0x7f9a70155690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 shutdown_connections 2024-03-06T07:23:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.847+0000 7f9a765ac640 1 -- 172.21.15.26:0/2327988642 wait complete. 2024-03-06T07:23:08.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.967+0000 7f375046d640 1 Processor -- start 2024-03-06T07:23:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.967+0000 7f375046d640 1 -- start start 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.968+0000 7f375046d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.968+0000 7f375046d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3748155a40 0x7f3748155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.968+0000 7f375046d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3748156410 con 0x7f37480c1aa0 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.968+0000 7f375046d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37480c2680 con 0x7f3748155a40 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.968+0000 7f375046d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c28d0 con 0x7f374814f790 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374e209640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374ea0a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3748155a40 0x7f3748155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374ea0a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3748155a40 0x7f3748155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57156/0 (socket says 172.21.15.26:57156) 2024-03-06T07:23:08.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374e209640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:60036/0 (socket says 172.21.15.26:60036) 2024-03-06T07:23:08.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374da08640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f37480c1aa0 0x7f37480c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38384/0 (socket says 172.21.15.26:38384) 2024-03-06T07:23:08.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374ea0a640 1 -- 172.21.15.26:0/2768789902 learned_addr learned my addr 172.21.15.26:0/2768789902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374e209640 1 -- 172.21.15.26:0/2768789902 >> v2:172.21.15.117:3300/0 conn(0x7f3748155a40 msgr2=0x7f3748155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374e209640 1 --2- 172.21.15.26:0/2768789902 >> v2:172.21.15.117:3300/0 conn(0x7f3748155a40 0x7f3748155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374e209640 1 -- 172.21.15.26:0/2768789902 >> v1:172.21.15.117:6789/0 conn(0x7f37480c1aa0 legacy=0x7f37480c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374e209640 1 -- 172.21.15.26:0/2768789902 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480c2bb0 con 0x7f374814f790 2024-03-06T07:23:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.969+0000 7f374ea0a640 1 --2- 172.21.15.26:0/2768789902 >> v2:172.21.15.117:3300/0 conn(0x7f3748155a40 0x7f3748155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.970+0000 7f374e209640 1 --2- 172.21.15.26:0/2768789902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481529d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f37400097a0 tx=0x7f374002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9a7514fa936a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.970+0000 7f374d207640 1 -- 172.21.15.26:0/2768789902 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740035020 con 0x7f374814f790 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.970+0000 7f374d207640 1 -- 172.21.15.26:0/2768789902 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3740031810 con 0x7f374814f790 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.970+0000 7f374d207640 1 -- 172.21.15.26:0/2768789902 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740031be0 con 0x7f374814f790 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.970+0000 7f375046d640 1 -- 172.21.15.26:0/2768789902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 msgr2=0x7f37481529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.970+0000 7f375046d640 1 --2- 172.21.15.26:0/2768789902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481529d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f37400097a0 tx=0x7f374002b8b0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.971+0000 7f375046d640 1 -- 172.21.15.26:0/2768789902 shutdown_connections 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.971+0000 7f375046d640 1 --2- 172.21.15.26:0/2768789902 >> v2:172.21.15.117:3300/0 conn(0x7f3748155a40 0x7f3748155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.971+0000 7f375046d640 1 --2- 172.21.15.26:0/2768789902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481529d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.971+0000 7f375046d640 1 -- 172.21.15.26:0/2768789902 >> 172.21.15.26:0/2768789902 conn(0x7f37480c6710 msgr2=0x7f37480c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.971+0000 7f375046d640 1 -- 172.21.15.26:0/2768789902 shutdown_connections 2024-03-06T07:23:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.971+0000 7f375046d640 1 -- 172.21.15.26:0/2768789902 wait complete. 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 Processor -- start 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 -- start start 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37480c1aa0 0x7f37481513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481dee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748155a40 0x7f37481dc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c0d60 con 0x7f3748155a40 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748155520 con 0x7f37480c1aa0 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.972+0000 7f375046d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c5e30 con 0x7f374814f790 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374ea0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748155a40 0x7f37481dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374ea0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748155a40 0x7f37481dc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60738/0 (socket says 172.21.15.26:60738) 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374ea0a640 1 -- 172.21.15.26:0/3702795484 learned_addr learned my addr 172.21.15.26:0/3702795484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374da08640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374e209640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37480c1aa0 0x7f37481513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374da08640 1 -- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37480c1aa0 msgr2=0x7f37481513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374da08640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37480c1aa0 0x7f37481513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374da08640 1 -- 172.21.15.26:0/3702795484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748155a40 msgr2=0x7f37481dc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374da08640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748155a40 0x7f37481dc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374da08640 1 -- 172.21.15.26:0/3702795484 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37481519b0 con 0x7f374814f790 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374ea0a640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748155a40 0x7f37481dc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374e209640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37480c1aa0 0x7f37481513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f374da08640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481dee50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f373c00c370 tx=0x7f373c00c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f3746ffd640 1 -- 172.21.15.26:0/3702795484 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c0074e0 con 0x7f374814f790 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f3746ffd640 1 -- 172.21.15.26:0/3702795484 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c0047e0 con 0x7f374814f790 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f3746ffd640 1 -- 172.21.15.26:0/3702795484 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c004bd0 con 0x7f374814f790 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3748151c90 con 0x7f374814f790 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.973+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37480733d0 con 0x7f374814f790 2024-03-06T07:23:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.974+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374804f330 con 0x7f374814f790 2024-03-06T07:23:08.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.975+0000 7f3746ffd640 1 -- 172.21.15.26:0/3702795484 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f373c01a020 con 0x7f374814f790 2024-03-06T07:23:08.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.975+0000 7f3746ffd640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f373406b4d0 0x7f373406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.975+0000 7f3746ffd640 1 -- 172.21.15.26:0/3702795484 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f373c089a70 con 0x7f374814f790 2024-03-06T07:23:08.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.976+0000 7f374e209640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f373406b4d0 0x7f373406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.976+0000 7f374e209640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f373406b4d0 0x7f373406d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3740036000 tx=0x7f3740008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:08.980+0000 7f3746ffd640 1 -- 172.21.15.26:0/3702795484 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f373c058650 con 0x7f374814f790 2024-03-06T07:23:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.169+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37480c1f40 con 0x7f374814f790 2024-03-06T07:23:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.170+0000 7f3746ffd640 1 -- 172.21.15.26:0/3702795484 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17290 (secure 0 0 0) 0x7f373c057f50 con 0x7f374814f790 2024-03-06T07:23:09.172 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:09.172 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:08.840150+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.26:6835/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.117:6836/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6834/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6836/218606650":"2024-03-07T07:22:54.778025+0000","172.21.15.117:6834/218606650":"2024-03-07T07:22:54.778025+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f373406b4d0 msgr2=0x7f373406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f373406b4d0 0x7f373406d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3740036000 tx=0x7f3740008030 comp rx=0 tx=0).stop 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 msgr2=0x7f37481dee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481dee50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f373c00c370 tx=0x7f373c00c770 comp rx=0 tx=0).stop 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 shutdown_connections 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f373406b4d0 0x7f373406d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748155a40 0x7f37481dc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f374814f790 0x7f37481dee50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 --2- 172.21.15.26:0/3702795484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37480c1aa0 0x7f37481513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 >> 172.21.15.26:0/3702795484 conn(0x7f37480c6710 msgr2=0x7f37480c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 shutdown_connections 2024-03-06T07:23:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.172+0000 7f375046d640 1 -- 172.21.15.26:0/3702795484 wait complete. 2024-03-06T07:23:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:22:42.855764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:22:43.690413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:09.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.287+0000 7f25a4bac640 1 Processor -- start 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.287+0000 7f25a4bac640 1 -- start start 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.288+0000 7f25a4bac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.288+0000 7f25a4bac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25a00c5380 0x7f25a00c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.288+0000 7f25a4bac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a00b0450 con 0x7f25a00b0070 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.288+0000 7f25a4bac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a006cfb0 con 0x7f25a00c5380 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.288+0000 7f25a4bac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a006c020 con 0x7f25a0155c30 2024-03-06T07:23:09.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.288+0000 7f259dd9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.288+0000 7f259ed9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25a00c5380 0x7f25a00c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.289+0000 7f259e59c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f25a00b0070 0x7f25a00c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38386/0 (socket says 172.21.15.26:38386) 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.289+0000 7f259e59c640 1 -- 172.21.15.26:0/2978258154 learned_addr learned my addr 172.21.15.26:0/2978258154 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.289+0000 7f259ed9d640 1 -- 172.21.15.26:0/2978258154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 msgr2=0x7f25a006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.289+0000 7f259ed9d640 1 --2- 172.21.15.26:0/2978258154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a006ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.289+0000 7f259ed9d640 1 -- 172.21.15.26:0/2978258154 >> v1:172.21.15.117:6789/0 conn(0x7f25a00b0070 legacy=0x7f25a00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.289+0000 7f259ed9d640 1 -- 172.21.15.26:0/2978258154 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a01560e0 con 0x7f25a00c5380 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.290+0000 7f259ed9d640 1 --2- 172.21.15.26:0/2978258154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00c5380 0x7f25a00c57f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f25980097a0 tx=0x7f259802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b4404b7903fedf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.290+0000 7f259d59a640 1 -- 172.21.15.26:0/2978258154 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980315a0 con 0x7f25a00c5380 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.290+0000 7f259d59a640 1 -- 172.21.15.26:0/2978258154 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598037070 con 0x7f25a00c5380 2024-03-06T07:23:09.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.290+0000 7f259d59a640 1 -- 172.21.15.26:0/2978258154 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598031960 con 0x7f25a00c5380 2024-03-06T07:23:09.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.290+0000 7f25a4bac640 1 -- 172.21.15.26:0/2978258154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00c5380 msgr2=0x7f25a00c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.290+0000 7f25a4bac640 1 --2- 172.21.15.26:0/2978258154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00c5380 0x7f25a00c57f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f25980097a0 tx=0x7f259802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:09.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.291+0000 7f25a4bac640 1 -- 172.21.15.26:0/2978258154 shutdown_connections 2024-03-06T07:23:09.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.291+0000 7f25a4bac640 1 --2- 172.21.15.26:0/2978258154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00c5380 0x7f25a00c57f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.291+0000 7f25a4bac640 1 --2- 172.21.15.26:0/2978258154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.291+0000 7f25a4bac640 1 -- 172.21.15.26:0/2978258154 >> 172.21.15.26:0/2978258154 conn(0x7f25a00ac590 msgr2=0x7f25a00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.291+0000 7f25a4bac640 1 -- 172.21.15.26:0/2978258154 shutdown_connections 2024-03-06T07:23:09.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.291+0000 7f25a4bac640 1 -- 172.21.15.26:0/2978258154 wait complete. 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 Processor -- start 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 -- start start 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00b0070 0x7f25a01541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a00c5380 0x7f25a01e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a01e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a006c270 con 0x7f25a00c5380 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00c7ad0 con 0x7f25a00b0070 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.292+0000 7f25a4bac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a006c550 con 0x7f25a0155c30 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a00c5380 0x7f25a01e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a00c5380 0x7f25a01e00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60748/0 (socket says 172.21.15.26:60748) 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 -- 172.21.15.26:0/3745034604 learned_addr learned my addr 172.21.15.26:0/3745034604 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259ed9d640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a01e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259e59c640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00b0070 0x7f25a01541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 -- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 msgr2=0x7f25a01e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a01e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 -- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00b0070 msgr2=0x7f25a01541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00b0070 0x7f25a01541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 -- 172.21.15.26:0/3745034604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a01eea30 con 0x7f25a00c5380 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259ed9d640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a01e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f259dd9b640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a00c5380 0x7f25a01e00a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f259400e8a0 tx=0x7f259400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f258effd640 1 -- 172.21.15.26:0/3745034604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594017070 con 0x7f25a00c5380 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f258effd640 1 -- 172.21.15.26:0/3745034604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259400c940 con 0x7f25a00c5380 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f258effd640 1 -- 172.21.15.26:0/3745034604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400cd30 con 0x7f25a00c5380 2024-03-06T07:23:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a01591a0 con 0x7f25a00c5380 2024-03-06T07:23:09.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.293+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a01546f0 con 0x7f25a00c5380 2024-03-06T07:23:09.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.294+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a004f330 con 0x7f25a00c5380 2024-03-06T07:23:09.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.295+0000 7f258effd640 1 -- 172.21.15.26:0/3745034604 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f259401a070 con 0x7f25a00c5380 2024-03-06T07:23:09.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.295+0000 7f258effd640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f258006b500 0x7f258006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.295+0000 7f258effd640 1 -- 172.21.15.26:0/3745034604 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f259408a740 con 0x7f25a00c5380 2024-03-06T07:23:09.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.295+0000 7f259e59c640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f258006b500 0x7f258006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.296+0000 7f259e59c640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f258006b500 0x7f258006d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f25a01598c0 tx=0x7f2590009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.300+0000 7f258effd640 1 -- 172.21.15.26:0/3745034604 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2594059290 con 0x7f25a00c5380 2024-03-06T07:23:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.484+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25a006d860 con 0x7f25a00c5380 2024-03-06T07:23:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.485+0000 7f258effd640 1 -- 172.21.15.26:0/3745034604 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2594058b90 con 0x7f25a00c5380 2024-03-06T07:23:09.486 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f258006b500 msgr2=0x7f258006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f258006b500 0x7f258006d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f25a01598c0 tx=0x7f2590009040 comp rx=0 tx=0).stop 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a00c5380 msgr2=0x7f25a01e00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a00c5380 0x7f25a01e00a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f259400e8a0 tx=0x7f259400ed60 comp rx=0 tx=0).stop 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 shutdown_connections 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f258006b500 0x7f258006d9c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25a0155c30 0x7f25a01e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a00c5380 0x7f25a01e00a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 --2- 172.21.15.26:0/3745034604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25a00b0070 0x7f25a01541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 >> 172.21.15.26:0/3745034604 conn(0x7f25a00ac590 msgr2=0x7f25a00ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 shutdown_connections 2024-03-06T07:23:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.487+0000 7f25a4bac640 1 -- 172.21.15.26:0/3745034604 wait complete. 2024-03-06T07:23:09.500 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:23:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.605+0000 7f6cd0488640 1 Processor -- start 2024-03-06T07:23:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.605+0000 7f6cd0488640 1 -- start start 2024-03-06T07:23:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6cd0488640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6cc80c4ef0 0x7f6cc8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6cd0488640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc8152cb0 con 0x7f6cc81528d0 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6cd0488640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80c38b0 con 0x7f6cc80c67f0 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6cd0488640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc81594f0 con 0x7f6cc80c4ef0 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6cce224640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6cc81528d0 0x7f6cc80724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43760/0 (socket says 172.21.15.26:43760) 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6cce224640 1 -- 172.21.15.26:0/2031676179 learned_addr learned my addr 172.21.15.26:0/2031676179 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6ccea25640 1 --2- 172.21.15.26:0/2031676179 >> v2:172.21.15.117:3300/0 conn(0x7f6cc80c4ef0 0x7f6cc8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108329618 0 0) 0x7f6cc8152cb0 con 0x7f6cc81528d0 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.606+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb0003520 con 0x7f6cc81528d0 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3499343827 0 0) 0x7f6cb0003520 con 0x7f6cc81528d0 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 >> v2:172.21.15.117:3300/0 conn(0x7f6cc80c4ef0 msgr2=0x7f6cc8155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 --2- 172.21.15.26:0/2031676179 >> v2:172.21.15.117:3300/0 conn(0x7f6cc80c4ef0 0x7f6cc8155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 >> v1:172.21.15.117:6789/0 conn(0x7f6cc80c67f0 legacy=0x7f6cc8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80a7f70 con 0x7f6cc81528d0 2024-03-06T07:23:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6cc0001670 con 0x7f6cc81528d0 2024-03-06T07:23:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 >> v1:172.21.15.26:6789/0 conn(0x7f6cc81528d0 legacy=0x7f6cc80724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cb00039b0 0x7f6cb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb00043b0 0x7f6cb0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0005080 0x7f6cb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81594f0 con 0x7f6cb0005080 2024-03-06T07:23:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb000bf30 con 0x7f6cb00043b0 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb000c130 con 0x7f6cb00039b0 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccea25640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0005080 0x7f6cb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccda23640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb00043b0 0x7f6cb0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6cce224640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cb00039b0 0x7f6cb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccea25640 1 -- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cb00039b0 msgr2=0x7f6cb0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccea25640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cb00039b0 0x7f6cb0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccea25640 1 -- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb00043b0 msgr2=0x7f6cb0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccea25640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb00043b0 0x7f6cb0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6ccea25640 1 -- 172.21.15.26:0/2031676179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb0003520 con 0x7f6cb0005080 2024-03-06T07:23:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.607+0000 7f6cce224640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cb00039b0 0x7f6cb0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.608+0000 7f6ccea25640 1 -- 172.21.15.26:0/2031676179 reap_dead start 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.608+0000 7f6ccea25640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0005080 0x7f6cb000b9c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009f90 tx=0x7f6cb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca33b53de8b47dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.608+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8039070 con 0x7f6cb0005080 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.608+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb802cc20 con 0x7f6cb0005080 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.608+0000 7f6ccd222640 1 -- 172.21.15.26:0/2031676179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8009c10 con 0x7f6cb0005080 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.608+0000 7f6cd0488640 1 -- 172.21.15.26:0/2031676179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0005080 msgr2=0x7f6cb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.608+0000 7f6cd0488640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0005080 0x7f6cb000b9c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009f90 tx=0x7f6cb0005830 comp rx=0 tx=0).stop 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.609+0000 7f6cd0488640 1 -- 172.21.15.26:0/2031676179 shutdown_connections 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.609+0000 7f6cd0488640 1 --2- 172.21.15.26:0/2031676179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0005080 0x7f6cb000b9c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.609+0000 7f6cd0488640 1 -- 172.21.15.26:0/2031676179 >> 172.21.15.26:0/2031676179 conn(0x7f6cc80aa6a0 msgr2=0x7f6cc80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.609+0000 7f6cd0488640 1 -- 172.21.15.26:0/2031676179 shutdown_connections 2024-03-06T07:23:09.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.609+0000 7f6cd0488640 1 -- 172.21.15.26:0/2031676179 wait complete. 2024-03-06T07:23:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 Processor -- start 2024-03-06T07:23:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 -- start start 2024-03-06T07:23:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0005080 0x7f6cc8070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cc81513f0 0x7f6cc81eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cc8151840 0x7f6cc81eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81591a0 con 0x7f6cc8151840 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80a8250 con 0x7f6cb0005080 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6cd0488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80724d0 con 0x7f6cc81513f0 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6ccea25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cc8151840 0x7f6cc81eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6ccea25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cc8151840 0x7f6cc81eb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60762/0 (socket says 172.21.15.26:60762) 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.610+0000 7f6ccea25640 1 -- 172.21.15.26:0/586726430 learned_addr learned my addr 172.21.15.26:0/586726430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6cce224640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0005080 0x7f6cc8070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccda23640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cc81513f0 0x7f6cc81eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccea25640 1 -- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cc81513f0 msgr2=0x7f6cc81eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccea25640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cc81513f0 0x7f6cc81eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccea25640 1 -- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0005080 msgr2=0x7f6cc8070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccea25640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0005080 0x7f6cc8070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccea25640 1 -- 172.21.15.26:0/586726430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80c5bd0 con 0x7f6cc8151840 2024-03-06T07:23:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccda23640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cc81513f0 0x7f6cc81eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6ccea25640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cc8151840 0x7f6cc81eb1c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e230 tx=0x7f6cb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6cc6ffd640 1 -- 172.21.15.26:0/586726430 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8042070 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6cc6ffd640 1 -- 172.21.15.26:0/586726430 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb803e480 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc81ea730 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6cc6ffd640 1 -- 172.21.15.26:0/586726430 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8039040 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.611+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc8154c50 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.612+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc804f330 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.613+0000 7f6cc6ffd640 1 -- 172.21.15.26:0/586726430 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cb8045070 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.613+0000 7f6cc6ffd640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cb406b500 0x7f6cb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.613+0000 7f6cc6ffd640 1 -- 172.21.15.26:0/586726430 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f6cb80ac6f0 con 0x7f6cc8151840 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.613+0000 7f6cce224640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cb406b500 0x7f6cb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.614+0000 7f6cce224640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cb406b500 0x7f6cb406d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0000c00 tx=0x7f6cc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.617+0000 7f6cc6ffd640 1 -- 172.21.15.26:0/586726430 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6cc80c5bd0 con 0x7f6cc8151840 2024-03-06T07:23:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.802+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6cc8073120 con 0x7f6cc8151840 2024-03-06T07:23:09.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.802+0000 7f6cc6ffd640 1 -- 172.21.15.26:0/586726430 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v83) v1 ==== 105+0+11 (secure 0 0 0) 0x7f6cb807ae40 con 0x7f6cc8151840 2024-03-06T07:23:09.804 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cb406b500 msgr2=0x7f6cb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cb406b500 0x7f6cb406d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0000c00 tx=0x7f6cc0009040 comp rx=0 tx=0).stop 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cc8151840 msgr2=0x7f6cc81eb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cc8151840 0x7f6cc81eb1c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e230 tx=0x7f6cb8009750 comp rx=0 tx=0).stop 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 shutdown_connections 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cb406b500 0x7f6cb406d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cc8151840 0x7f6cc81eb1c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cc81513f0 0x7f6cc81eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 --2- 172.21.15.26:0/586726430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0005080 0x7f6cc8070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 >> 172.21.15.26:0/586726430 conn(0x7f6cc80aa6a0 msgr2=0x7f6cc80c5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 shutdown_connections 2024-03-06T07:23:09.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.804+0000 7f6cd0488640 1 -- 172.21.15.26:0/586726430 wait complete. 2024-03-06T07:23:09.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:23:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.923+0000 7f94443e9640 1 Processor -- start 2024-03-06T07:23:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.923+0000 7f94443e9640 1 -- start start 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f94443e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f943c155c30 0x7f943c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f94443e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f94443e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c0b0450 con 0x7f943c0b0070 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f94443e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c06cfb0 con 0x7f943c155c30 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f94443e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c06c020 con 0x7f943c0c5380 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f9441984640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f943c155c30 0x7f943c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f9442986640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f9442185640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f943c0b0070 0x7f943c0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38408/0 (socket says 172.21.15.26:38408) 2024-03-06T07:23:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.924+0000 7f9442185640 1 -- 172.21.15.26:0/1200168776 learned_addr learned my addr 172.21.15.26:0/1200168776 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.925+0000 7f9441984640 1 -- 172.21.15.26:0/1200168776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 msgr2=0x7f943c0c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.925+0000 7f9441984640 1 --2- 172.21.15.26:0/1200168776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c0c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.925+0000 7f9441984640 1 -- 172.21.15.26:0/1200168776 >> v1:172.21.15.117:6789/0 conn(0x7f943c0b0070 legacy=0x7f943c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.925+0000 7f9441984640 1 -- 172.21.15.26:0/1200168776 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c1560e0 con 0x7f943c155c30 2024-03-06T07:23:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.925+0000 7f9441984640 1 --2- 172.21.15.26:0/1200168776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 0x7f943c06ca70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94300097a0 tx=0x7f943002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88091be3fc1b913 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f9441183640 1 -- 172.21.15.26:0/1200168776 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94300315a0 con 0x7f943c155c30 2024-03-06T07:23:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f9441183640 1 -- 172.21.15.26:0/1200168776 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9430037070 con 0x7f943c155c30 2024-03-06T07:23:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f9441183640 1 -- 172.21.15.26:0/1200168776 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430031960 con 0x7f943c155c30 2024-03-06T07:23:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 -- 172.21.15.26:0/1200168776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 msgr2=0x7f943c06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 --2- 172.21.15.26:0/1200168776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 0x7f943c06ca70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94300097a0 tx=0x7f943002b840 comp rx=0 tx=0).stop 2024-03-06T07:23:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 -- 172.21.15.26:0/1200168776 shutdown_connections 2024-03-06T07:23:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 --2- 172.21.15.26:0/1200168776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c0c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 --2- 172.21.15.26:0/1200168776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 0x7f943c06ca70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 -- 172.21.15.26:0/1200168776 >> 172.21.15.26:0/1200168776 conn(0x7f943c0ac590 msgr2=0x7f943c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 -- 172.21.15.26:0/1200168776 shutdown_connections 2024-03-06T07:23:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.926+0000 7f94443e9640 1 -- 172.21.15.26:0/1200168776 wait complete. 2024-03-06T07:23:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.927+0000 7f94443e9640 1 Processor -- start 2024-03-06T07:23:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.927+0000 7f94443e9640 1 -- start start 2024-03-06T07:23:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f94443e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c0b0070 0x7f943c0c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f94443e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f94443e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 0x7f943c1ee680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f94443e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c06c270 con 0x7f943c0b0070 2024-03-06T07:23:09.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f94443e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0c7ad0 con 0x7f943c155c30 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f94443e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c06c550 con 0x7f943c0c5380 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c0b0070 0x7f943c0c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c0b0070 0x7f943c0c29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60766/0 (socket says 172.21.15.26:60766) 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 -- 172.21.15.26:0/4079944164 learned_addr learned my addr 172.21.15.26:0/4079944164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442986640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 0x7f943c1ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9441984640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 -- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 msgr2=0x7f943c157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 -- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 msgr2=0x7f943c1ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 0x7f943c1ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.928+0000 7f9442185640 1 -- 172.21.15.26:0/4079944164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c0c2f10 con 0x7f943c0b0070 2024-03-06T07:23:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.929+0000 7f9442185640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c0b0070 0x7f943c0c29d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f943400c3c0 tx=0x7f943400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.929+0000 7f943affd640 1 -- 172.21.15.26:0/4079944164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434017070 con 0x7f943c0b0070 2024-03-06T07:23:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.929+0000 7f943affd640 1 -- 172.21.15.26:0/4079944164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434005ce0 con 0x7f943c0b0070 2024-03-06T07:23:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.929+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c1596f0 con 0x7f943c0b0070 2024-03-06T07:23:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.929+0000 7f943affd640 1 -- 172.21.15.26:0/4079944164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94340072f0 con 0x7f943c0b0070 2024-03-06T07:23:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.929+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c1573a0 con 0x7f943c0b0070 2024-03-06T07:23:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.930+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c1571d0 con 0x7f943c0b0070 2024-03-06T07:23:09.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.930+0000 7f943affd640 1 -- 172.21.15.26:0/4079944164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f943401a070 con 0x7f943c0b0070 2024-03-06T07:23:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.931+0000 7f943affd640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942806b4d0 0x7f942806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.931+0000 7f943affd640 1 -- 172.21.15.26:0/4079944164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f94340897c0 con 0x7f943c0b0070 2024-03-06T07:23:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.931+0000 7f9441984640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942806b4d0 0x7f942806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.931+0000 7f9441984640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942806b4d0 0x7f942806d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9430036000 tx=0x7f943003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:09.935+0000 7f943affd640 1 -- 172.21.15.26:0/4079944164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f943c0b0450 con 0x7f943c0b0070 2024-03-06T07:23:10.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.115+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f943c04f330 con 0x7f943c0b0070 2024-03-06T07:23:10.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.115+0000 7f943affd640 1 -- 172.21.15.26:0/4079944164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v83) v1 ==== 101+0+11 (secure 0 0 0) 0x7f9434057d70 con 0x7f943c0b0070 2024-03-06T07:23:10.117 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942806b4d0 msgr2=0x7f942806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942806b4d0 0x7f942806d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9430036000 tx=0x7f943003d040 comp rx=0 tx=0).stop 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c0b0070 msgr2=0x7f943c0c29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c0b0070 0x7f943c0c29d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f943400c3c0 tx=0x7f943400c880 comp rx=0 tx=0).stop 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 shutdown_connections 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f942806b4d0 0x7f942806d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943c155c30 0x7f943c1ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f943c0c5380 0x7f943c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 --2- 172.21.15.26:0/4079944164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c0b0070 0x7f943c0c29d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 >> 172.21.15.26:0/4079944164 conn(0x7f943c0ac590 msgr2=0x7f943c0aa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 shutdown_connections 2024-03-06T07:23:10.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.117+0000 7f94443e9640 1 -- 172.21.15.26:0/4079944164 wait complete. 2024-03-06T07:23:10.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.230+0000 7f7dc6fbf640 1 Processor -- start 2024-03-06T07:23:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.230+0000 7f7dc6fbf640 1 -- start start 2024-03-06T07:23:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc6fbf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7dc0072720 0x7f7dc0156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc6fbf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7dc006f7b0 0x7f7dc01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc6fbf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc01506f0 0x7f7dc00c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc6fbf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc00c3740 con 0x7f7dc0072720 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc6fbf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc0157340 con 0x7f7dc006f7b0 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc6fbf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0156d80 con 0x7f7dc01506f0 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7dc0072720 0x7f7dc0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7dc0072720 0x7f7dc0156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60770/0 (socket says 172.21.15.26:60770) 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 -- 172.21.15.26:0/2809107456 learned_addr learned my addr 172.21.15.26:0/2809107456 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc555c640 1 --2- 172.21.15.26:0/2809107456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc01506f0 0x7f7dc00c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dbffff640 1 --2- 172.21.15.26:0/2809107456 >> v2:172.21.15.117:3300/0 conn(0x7f7dc006f7b0 0x7f7dc01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 -- 172.21.15.26:0/2809107456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc01506f0 msgr2=0x7f7dc00c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 --2- 172.21.15.26:0/2809107456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc01506f0 0x7f7dc00c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 -- 172.21.15.26:0/2809107456 >> v2:172.21.15.117:3300/0 conn(0x7f7dc006f7b0 msgr2=0x7f7dc01501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 --2- 172.21.15.26:0/2809107456 >> v2:172.21.15.117:3300/0 conn(0x7f7dc006f7b0 0x7f7dc01501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.231+0000 7f7dc4d5b640 1 -- 172.21.15.26:0/2809107456 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0157060 con 0x7f7dc0072720 2024-03-06T07:23:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.232+0000 7f7dc4d5b640 1 --2- 172.21.15.26:0/2809107456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc0072720 0x7f7dc0156430 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0072290 tx=0x7f7db402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3922cb2c52073e45 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.232+0000 7f7dbf7fe640 1 -- 172.21.15.26:0/2809107456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db402e070 con 0x7f7dc0072720 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.232+0000 7f7dbf7fe640 1 -- 172.21.15.26:0/2809107456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4037070 con 0x7f7dc0072720 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.232+0000 7f7dbf7fe640 1 -- 172.21.15.26:0/2809107456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4031a00 con 0x7f7dc0072720 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.232+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/2809107456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc0072720 msgr2=0x7f7dc0156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.232+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/2809107456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc0072720 0x7f7dc0156430 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0072290 tx=0x7f7db402b880 comp rx=0 tx=0).stop 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.233+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/2809107456 shutdown_connections 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.233+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/2809107456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc01506f0 0x7f7dc00c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.233+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/2809107456 >> v2:172.21.15.117:3300/0 conn(0x7f7dc006f7b0 0x7f7dc01501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.233+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/2809107456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc0072720 0x7f7dc0156430 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.233+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/2809107456 >> 172.21.15.26:0/2809107456 conn(0x7f7dc00aa630 msgr2=0x7f7dc00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.233+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/2809107456 shutdown_connections 2024-03-06T07:23:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.233+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/2809107456 wait complete. 2024-03-06T07:23:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 Processor -- start 2024-03-06T07:23:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 -- start start 2024-03-06T07:23:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc006f7b0 0x7f7dc0149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc0072720 0x7f7dc0155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dc01506f0 0x7f7dc00adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0156af0 con 0x7f7dc006f7b0 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0156970 con 0x7f7dc01506f0 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.234+0000 7f7dc6fbf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0072b00 con 0x7f7dc0072720 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dc4d5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc006f7b0 0x7f7dc0149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dc4d5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc006f7b0 0x7f7dc0149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60774/0 (socket says 172.21.15.26:60774) 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dc4d5b640 1 -- 172.21.15.26:0/1798986132 learned_addr learned my addr 172.21.15.26:0/1798986132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbffff640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc0072720 0x7f7dc0155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dc555c640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dc01506f0 0x7f7dc00adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbffff640 1 -- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dc01506f0 msgr2=0x7f7dc00adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbffff640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dc01506f0 0x7f7dc00adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbffff640 1 -- 172.21.15.26:0/1798986132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc006f7b0 msgr2=0x7f7dc0149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbffff640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc006f7b0 0x7f7dc0149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbffff640 1 -- 172.21.15.26:0/1798986132 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc00ad8f0 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dc4d5b640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc006f7b0 0x7f7dc0149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbffff640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc0072720 0x7f7dc0155af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e850 tx=0x7f7dac00ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbd7fa640 1 -- 172.21.15.26:0/1798986132 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac017070 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbd7fa640 1 -- 172.21.15.26:0/1798986132 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac00c940 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc0149ae0 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dbd7fa640 1 -- 172.21.15.26:0/1798986132 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00cd10 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.235+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc01eb620 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.236+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc004f330 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.237+0000 7f7dbd7fa640 1 -- 172.21.15.26:0/1798986132 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dac01a070 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.237+0000 7f7dbd7fa640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7da406b4d0 0x7f7da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.237+0000 7f7dbd7fa640 1 -- 172.21.15.26:0/1798986132 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f7dac08a3d0 con 0x7f7dc0072720 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.237+0000 7f7dc4d5b640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7da406b4d0 0x7f7da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.238+0000 7f7dc4d5b640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7da406b4d0 0x7f7da406d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7db4034040 tx=0x7f7db403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.242+0000 7f7dbd7fa640 1 -- 172.21.15.26:0/1798986132 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7dac0588e0 con 0x7f7dc0072720 2024-03-06T07:23:10.310 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:23:10.308+0000 7f199b337640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.461+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dc0074700 con 0x7f7dc0072720 2024-03-06T07:23:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.461+0000 7f7dbd7fa640 1 -- 172.21.15.26:0/1798986132 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f7dac05c980 con 0x7f7dc0072720 2024-03-06T07:23:10.463 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:10.463 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1880409927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1880409927},{"type":"v1","addr":"172.21.15.117:6836","nonce":1880409927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/174449231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":174449231},{"type":"v1","addr":"172.21.15.26:6835","nonce":174449231}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:23:09.844837+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5618},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5618":{"gid":5618,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5618,"qdb_cluster":[5618]},"id":6}]} 2024-03-06T07:23:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 71 2024-03-06T07:23:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7da406b4d0 msgr2=0x7f7da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7da406b4d0 0x7f7da406d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7db4034040 tx=0x7f7db403d040 comp rx=0 tx=0).stop 2024-03-06T07:23:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc0072720 msgr2=0x7f7dc0155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc0072720 0x7f7dc0155af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e850 tx=0x7f7dac00ed10 comp rx=0 tx=0).stop 2024-03-06T07:23:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 shutdown_connections 2024-03-06T07:23:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7da406b4d0 0x7f7da406d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dc01506f0 0x7f7dc00adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc0072720 0x7f7dc0155af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 --2- 172.21.15.26:0/1798986132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dc006f7b0 0x7f7dc0149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 >> 172.21.15.26:0/1798986132 conn(0x7f7dc00aa630 msgr2=0x7f7dc00c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.463+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 shutdown_connections 2024-03-06T07:23:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.464+0000 7f7dc6fbf640 1 -- 172.21.15.26:0/1798986132 wait complete. 2024-03-06T07:23:10.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.575+0000 7f2ade3d6640 1 Processor -- start 2024-03-06T07:23:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ade3d6640 1 -- start start 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ade3d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ad814f360 0x7f2ad8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ade3d6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ad8158b60 0x7f2ad8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ade3d6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad80c73d0 con 0x7f2ad80c88c0 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ade3d6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad80c8dd0 con 0x7f2ad814f360 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ade3d6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad80be2e0 con 0x7f2ad8158b60 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ad7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ad814f360 0x7f2ad8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ad77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2ad80c88c0 0x7f2ad80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43770/0 (socket says 172.21.15.26:43770) 2024-03-06T07:23:10.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.576+0000 7f2ad77fe640 1 -- 172.21.15.26:0/263193233 learned_addr learned my addr 172.21.15.26:0/263193233 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.577+0000 7f2adc973640 1 --2- 172.21.15.26:0/263193233 >> v2:172.21.15.117:3300/0 conn(0x7f2ad8158b60 0x7f2ad8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.577+0000 7f2ad7fff640 1 -- 172.21.15.26:0/263193233 >> v2:172.21.15.117:3300/0 conn(0x7f2ad8158b60 msgr2=0x7f2ad8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.577+0000 7f2ad7fff640 1 --2- 172.21.15.26:0/263193233 >> v2:172.21.15.117:3300/0 conn(0x7f2ad8158b60 0x7f2ad8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.577+0000 7f2ad7fff640 1 -- 172.21.15.26:0/263193233 >> v1:172.21.15.26:6789/0 conn(0x7f2ad80c88c0 legacy=0x7f2ad80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.577+0000 7f2ad7fff640 1 -- 172.21.15.26:0/263193233 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80c7670 con 0x7f2ad814f360 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.577+0000 7f2ad6ffd640 1 -- 172.21.15.26:0/263193233 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993008323 0 0) 0x7f2ad80c73d0 con 0x7f2ad80c88c0 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ad7fff640 1 --2- 172.21.15.26:0/263193233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad814f360 0x7f2ad8155c10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8152d60 tx=0x7f2ac802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b185f76bde588c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ad6ffd640 1 -- 172.21.15.26:0/263193233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80315a0 con 0x7f2ad814f360 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ad6ffd640 1 -- 172.21.15.26:0/263193233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac8037070 con 0x7f2ad814f360 2024-03-06T07:23:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ad6ffd640 1 -- 172.21.15.26:0/263193233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8031960 con 0x7f2ad814f360 2024-03-06T07:23:10.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 -- 172.21.15.26:0/263193233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad814f360 msgr2=0x7f2ad8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/263193233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad814f360 0x7f2ad8155c10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8152d60 tx=0x7f2ac802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:10.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 -- 172.21.15.26:0/263193233 shutdown_connections 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/263193233 >> v2:172.21.15.117:3300/0 conn(0x7f2ad8158b60 0x7f2ad8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/263193233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad814f360 0x7f2ad8155c10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 -- 172.21.15.26:0/263193233 >> 172.21.15.26:0/263193233 conn(0x7f2ad80ac570 msgr2=0x7f2ad80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 -- 172.21.15.26:0/263193233 shutdown_connections 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.578+0000 7f2ade3d6640 1 -- 172.21.15.26:0/263193233 wait complete. 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.579+0000 7f2ade3d6640 1 Processor -- start 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ade3d6640 1 -- start start 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ade3d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad80c88c0 0x7f2ad8151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ade3d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ad814f360 0x7f2ad81eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ade3d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ad8158b60 0x7f2ad81eb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ade3d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80c5a50 con 0x7f2ad80c88c0 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ade3d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80c7c30 con 0x7f2ad814f360 2024-03-06T07:23:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ade3d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80c5bd0 con 0x7f2ad8158b60 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ad7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad80c88c0 0x7f2ad8151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ad7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad80c88c0 0x7f2ad8151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60780/0 (socket says 172.21.15.26:60780) 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ad7fff640 1 -- 172.21.15.26:0/1761085967 learned_addr learned my addr 172.21.15.26:0/1761085967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2ad77fe640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ad814f360 0x7f2ad81eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.580+0000 7f2adc973640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ad8158b60 0x7f2ad81eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad7fff640 1 -- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ad8158b60 msgr2=0x7f2ad81eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad7fff640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ad8158b60 0x7f2ad81eb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad7fff640 1 -- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ad814f360 msgr2=0x7f2ad81eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad7fff640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ad814f360 0x7f2ad81eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad7fff640 1 -- 172.21.15.26:0/1761085967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad8152600 con 0x7f2ad80c88c0 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad7fff640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad80c88c0 0x7f2ad8151a10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802be50 tx=0x7f2ac80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad4ff9640 1 -- 172.21.15.26:0/1761085967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8032630 con 0x7f2ad80c88c0 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad4ff9640 1 -- 172.21.15.26:0/1761085967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac80327d0 con 0x7f2ad80c88c0 2024-03-06T07:23:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ad4ff9640 1 -- 172.21.15.26:0/1761085967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8032bc0 con 0x7f2ad80c88c0 2024-03-06T07:23:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad81eef20 con 0x7f2ad80c88c0 2024-03-06T07:23:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.581+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad81ee440 con 0x7f2ad80c88c0 2024-03-06T07:23:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.582+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad804f330 con 0x7f2ad80c88c0 2024-03-06T07:23:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.582+0000 7f2ad4ff9640 1 -- 172.21.15.26:0/1761085967 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ac804c020 con 0x7f2ad80c88c0 2024-03-06T07:23:10.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.583+0000 7f2ad4ff9640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ac406b4d0 0x7f2ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.583+0000 7f2ad4ff9640 1 -- 172.21.15.26:0/1761085967 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f2ac802f020 con 0x7f2ad80c88c0 2024-03-06T07:23:10.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.583+0000 7f2ad77fe640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ac406b4d0 0x7f2ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.583+0000 7f2ad77fe640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ac406b4d0 0x7f2ac406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80c1610 tx=0x7f2acc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.587+0000 7f2ad4ff9640 1 -- 172.21.15.26:0/1761085967 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ac80b2080 con 0x7f2ad80c88c0 2024-03-06T07:23:10.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.769+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ad80c1640 con 0x7f2ad80c88c0 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.771+0000 7f2ad4ff9640 1 -- 172.21.15.26:0/1761085967 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17290 (secure 0 0 0) 0x7f2ad8152600 con 0x7f2ad80c88c0 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:10.773 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:08.840150+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.26:6835/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.117:6836/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6834/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6836/218606650":"2024-03-07T07:22:54.778025+0000","172.21.15.117:6834/218606650":"2024-03-07T07:22:54.778025+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ac406b4d0 msgr2=0x7f2ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ac406b4d0 0x7f2ac406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80c1610 tx=0x7f2acc007de0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad80c88c0 msgr2=0x7f2ad8151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad80c88c0 0x7f2ad8151a10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802be50 tx=0x7f2ac80046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 shutdown_connections 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ac406b4d0 0x7f2ac406d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ad8158b60 0x7f2ad81eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ad814f360 0x7f2ad81eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 --2- 172.21.15.26:0/1761085967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ad80c88c0 0x7f2ad8151a10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 >> 172.21.15.26:0/1761085967 conn(0x7f2ad80ac570 msgr2=0x7f2ad814cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 shutdown_connections 2024-03-06T07:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.773+0000 7f2ade3d6640 1 -- 172.21.15.26:0/1761085967 wait complete. 2024-03-06T07:23:10.786 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-06T07:23:10.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.886+0000 7f838cb1e640 1 Processor -- start 2024-03-06T07:23:10.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.886+0000 7f838cb1e640 1 -- start start 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.886+0000 7f838cb1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8388156150 0x7f83881529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.886+0000 7f838cb1e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880c28d0 con 0x7f838814f790 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.886+0000 7f838cb1e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880c2bb0 con 0x7f83880c1aa0 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.886+0000 7f838cb1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c2680 con 0x7f8388156150 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f838659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f838814f790 0x7f83880c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43786/0 (socket says 172.21.15.26:43786) 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f838659c640 1 -- 172.21.15.26:0/2032248105 learned_addr learned my addr 172.21.15.26:0/2032248105 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f8386d9d640 1 --2- 172.21.15.26:0/2032248105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8388156150 0x7f83881529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f838559a640 1 -- 172.21.15.26:0/2032248105 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513856990 0 0) 0x7f83880c28d0 con 0x7f838814f790 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f838559a640 1 -- 172.21.15.26:0/2032248105 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8370003520 con 0x7f838814f790 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f838559a640 1 -- 172.21.15.26:0/2032248105 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723252460 0 0) 0x7f83880c2bb0 con 0x7f83880c1aa0 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f838559a640 1 -- 172.21.15.26:0/2032248105 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83880c28d0 con 0x7f83880c1aa0 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f8386d9d640 1 -- 172.21.15.26:0/2032248105 >> v1:172.21.15.117:6789/0 conn(0x7f83880c1aa0 legacy=0x7f8388155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f8386d9d640 1 -- 172.21.15.26:0/2032248105 >> v1:172.21.15.26:6789/0 conn(0x7f838814f790 legacy=0x7f83880c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.887+0000 7f8386d9d640 1 -- 172.21.15.26:0/2032248105 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880c2e90 con 0x7f8388156150 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.888+0000 7f8386d9d640 1 --2- 172.21.15.26:0/2032248105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8388156150 0x7f83881529d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8380009dc0 tx=0x7f838002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d120f915530167b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.888+0000 7f838559a640 1 -- 172.21.15.26:0/2032248105 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380039070 con 0x7f8388156150 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.888+0000 7f838559a640 1 -- 172.21.15.26:0/2032248105 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838002ccd0 con 0x7f8388156150 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.888+0000 7f838559a640 1 -- 172.21.15.26:0/2032248105 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380034660 con 0x7f8388156150 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.888+0000 7f838cb1e640 1 -- 172.21.15.26:0/2032248105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8388156150 msgr2=0x7f83881529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.888+0000 7f838cb1e640 1 --2- 172.21.15.26:0/2032248105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8388156150 0x7f83881529d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8380009dc0 tx=0x7f838002bf20 comp rx=0 tx=0).stop 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.889+0000 7f838cb1e640 1 -- 172.21.15.26:0/2032248105 shutdown_connections 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.889+0000 7f838cb1e640 1 --2- 172.21.15.26:0/2032248105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8388156150 0x7f83881529d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.889+0000 7f838cb1e640 1 -- 172.21.15.26:0/2032248105 >> 172.21.15.26:0/2032248105 conn(0x7f83880c6710 msgr2=0x7f83880c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.889+0000 7f838cb1e640 1 -- 172.21.15.26:0/2032248105 shutdown_connections 2024-03-06T07:23:10.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.889+0000 7f838cb1e640 1 -- 172.21.15.26:0/2032248105 wait complete. 2024-03-06T07:23:10.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.890+0000 7f838cb1e640 1 Processor -- start 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.890+0000 7f838cb1e640 1 -- start start 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.890+0000 7f838cb1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83880c1aa0 0x7f8388148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.890+0000 7f838cb1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838814f790 0x7f838814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.890+0000 7f838cb1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8388156150 0x7f83881ea070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f838cb1e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c5e30 con 0x7f838814f790 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f838cb1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c0d60 con 0x7f8388156150 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f838cb1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c5fb0 con 0x7f83880c1aa0 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838814f790 0x7f838814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838814f790 0x7f838814e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60794/0 (socket says 172.21.15.26:60794) 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 -- 172.21.15.26:0/3538108328 learned_addr learned my addr 172.21.15.26:0/3538108328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f838659c640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83880c1aa0 0x7f8388148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8386d9d640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8388156150 0x7f83881ea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 -- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83880c1aa0 msgr2=0x7f8388148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83880c1aa0 0x7f8388148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 -- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8388156150 msgr2=0x7f83881ea070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8388156150 0x7f83881ea070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 -- 172.21.15.26:0/3538108328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388148650 con 0x7f838814f790 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8385d9b640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838814f790 0x7f838814e460 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f837c00c400 tx=0x7f837c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8376ffd640 1 -- 172.21.15.26:0/3538108328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c017070 con 0x7f838814f790 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8376ffd640 1 -- 172.21.15.26:0/3538108328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c0052d0 con 0x7f838814f790 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83880731a0 con 0x7f838814f790 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f8376ffd640 1 -- 172.21.15.26:0/3538108328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c003810 con 0x7f838814f790 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.891+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838814e9a0 con 0x7f838814f790 2024-03-06T07:23:10.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.892+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83880c2bb0 con 0x7f838814f790 2024-03-06T07:23:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.893+0000 7f8376ffd640 1 -- 172.21.15.26:0/3538108328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f837c01a070 con 0x7f838814f790 2024-03-06T07:23:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.893+0000 7f8376ffd640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f836806b500 0x7f836806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.893+0000 7f8376ffd640 1 -- 172.21.15.26:0/3538108328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f837c089880 con 0x7f838814f790 2024-03-06T07:23:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.893+0000 7f838659c640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f836806b500 0x7f836806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.894+0000 7f838659c640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f836806b500 0x7f836806d9c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f83780011e0 tx=0x7f8378007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:10.897+0000 7f8376ffd640 1 -- 172.21.15.26:0/3538108328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83880c2bb0 con 0x7f838814f790 2024-03-06T07:23:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.120+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f838804f330 con 0x7f838814f790 2024-03-06T07:23:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.121+0000 7f8376ffd640 1 -- 172.21.15.26:0/3538108328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f837c057d80 con 0x7f838814f790 2024-03-06T07:23:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:11.123 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1880409927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1880409927},{"type":"v1","addr":"172.21.15.117:6836","nonce":1880409927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/174449231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":174449231},{"type":"v1","addr":"172.21.15.26:6835","nonce":174449231}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:23:09.844837+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5618},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5618":{"gid":5618,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5618,"qdb_cluster":[5618]},"id":6}]} 2024-03-06T07:23:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 71 2024-03-06T07:23:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f836806b500 msgr2=0x7f836806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f836806b500 0x7f836806d9c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f83780011e0 tx=0x7f8378007d30 comp rx=0 tx=0).stop 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838814f790 msgr2=0x7f838814e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838814f790 0x7f838814e460 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f837c00c400 tx=0x7f837c00c8c0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 shutdown_connections 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f836806b500 0x7f836806d9c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8388156150 0x7f83881ea070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f838814f790 0x7f838814e460 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 --2- 172.21.15.26:0/3538108328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83880c1aa0 0x7f8388148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 >> 172.21.15.26:0/3538108328 conn(0x7f83880c6710 msgr2=0x7f83880c45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 shutdown_connections 2024-03-06T07:23:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.123+0000 7f838cb1e640 1 -- 172.21.15.26:0/3538108328 wait complete. 2024-03-06T07:23:11.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:11.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.235+0000 7fd574f3e640 1 Processor -- start 2024-03-06T07:23:11.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.235+0000 7fd574f3e640 1 -- start start 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd574f3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5701564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd574f3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd570156f90 con 0x7fd5700c3d60 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd574f3e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd570157270 con 0x7fd570156a00 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd574f3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c38d0 con 0x7fd570072720 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5700c3d60 0x7fd5700c41f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43792/0 (socket says 172.21.15.26:43792) 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56ed9d640 1 -- 172.21.15.26:0/792794773 learned_addr learned my addr 172.21.15.26:0/792794773 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56e59c640 1 --2- 172.21.15.26:0/792794773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5701564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56dd9b640 1 -- 172.21.15.26:0/792794773 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324115434 0 0) 0x7fd570156f90 con 0x7fd5700c3d60 2024-03-06T07:23:11.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56dd9b640 1 -- 172.21.15.26:0/792794773 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd55c003520 con 0x7fd5700c3d60 2024-03-06T07:23:11.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56e59c640 1 -- 172.21.15.26:0/792794773 >> v1:172.21.15.117:6789/0 conn(0x7fd570156a00 legacy=0x7fd570150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56e59c640 1 -- 172.21.15.26:0/792794773 >> v1:172.21.15.26:6789/0 conn(0x7fd5700c3d60 legacy=0x7fd5700c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56dd9b640 1 -- 172.21.15.26:0/792794773 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274569146 0 0) 0x7fd570157270 con 0x7fd570156a00 2024-03-06T07:23:11.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.236+0000 7fd56e59c640 1 -- 172.21.15.26:0/792794773 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700c3280 con 0x7fd570072720 2024-03-06T07:23:11.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.237+0000 7fd56e59c640 1 --2- 172.21.15.26:0/792794773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5701564c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd560009800 tx=0x7fd56002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44b1abd4c681d87 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.237+0000 7fd56dd9b640 1 -- 172.21.15.26:0/792794773 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560035020 con 0x7fd570072720 2024-03-06T07:23:11.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.237+0000 7fd56dd9b640 1 -- 172.21.15.26:0/792794773 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560031800 con 0x7fd570072720 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.237+0000 7fd56dd9b640 1 -- 172.21.15.26:0/792794773 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560031bd0 con 0x7fd570072720 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.238+0000 7fd574f3e640 1 -- 172.21.15.26:0/792794773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 msgr2=0x7fd5701564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.238+0000 7fd574f3e640 1 --2- 172.21.15.26:0/792794773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5701564c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd560009800 tx=0x7fd56002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.238+0000 7fd574f3e640 1 -- 172.21.15.26:0/792794773 shutdown_connections 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.238+0000 7fd574f3e640 1 --2- 172.21.15.26:0/792794773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5701564c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.238+0000 7fd574f3e640 1 -- 172.21.15.26:0/792794773 >> 172.21.15.26:0/792794773 conn(0x7fd5700aa630 msgr2=0x7fd5700aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.238+0000 7fd574f3e640 1 -- 172.21.15.26:0/792794773 shutdown_connections 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.238+0000 7fd574f3e640 1 -- 172.21.15.26:0/792794773 wait complete. 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.239+0000 7fd574f3e640 1 Processor -- start 2024-03-06T07:23:11.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.239+0000 7fd574f3e640 1 -- start start 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5700c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700c3d60 0x7fd570149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570156a00 0x7fd570149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c4900 con 0x7fd570156a00 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57006f8f0 con 0x7fd5700c3d60 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701ee370 con 0x7fd570072720 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570156a00 0x7fd570149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570156a00 0x7fd570149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60804/0 (socket says 172.21.15.26:60804) 2024-03-06T07:23:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 -- 172.21.15.26:0/2858622879 learned_addr learned my addr 172.21.15.26:0/2858622879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd567fff640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700c3d60 0x7fd570149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56e59c640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5700c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 -- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 msgr2=0x7fd5700c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5700c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 -- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700c3d60 msgr2=0x7fd570149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700c3d60 0x7fd570149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 -- 172.21.15.26:0/2858622879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5701ee650 con 0x7fd570156a00 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd56ed9d640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570156a00 0x7fd570149ff0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd56800cad0 tx=0x7fd56800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd566ffd640 1 -- 172.21.15.26:0/2858622879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568017070 con 0x7fd570156a00 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd566ffd640 1 -- 172.21.15.26:0/2858622879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56800ac30 con 0x7fd570156a00 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd566ffd640 1 -- 172.21.15.26:0/2858622879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568012400 con 0x7fd570156a00 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5700abd60 con 0x7fd570156a00 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.240+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5700c2460 con 0x7fd570156a00 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.241+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57004f330 con 0x7fd570156a00 2024-03-06T07:23:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.242+0000 7fd566ffd640 1 -- 172.21.15.26:0/2858622879 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd56801a070 con 0x7fd570156a00 2024-03-06T07:23:11.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.242+0000 7fd566ffd640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd55006b4d0 0x7fd55006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.243+0000 7fd566ffd640 1 -- 172.21.15.26:0/2858622879 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fd56808a200 con 0x7fd570156a00 2024-03-06T07:23:11.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.243+0000 7fd56e59c640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd55006b4d0 0x7fd55006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.243+0000 7fd56e59c640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd55006b4d0 0x7fd55006d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd560036000 tx=0x7fd560008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.247+0000 7fd566ffd640 1 -- 172.21.15.26:0/2858622879 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5680587b0 con 0x7fd570156a00 2024-03-06T07:23:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.465+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5700a7df0 con 0x7fd570156a00 2024-03-06T07:23:11.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.465+0000 7fd566ffd640 1 -- 172.21.15.26:0/2858622879 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4980 (secure 0 0 0) 0x7fd56805c850 con 0x7fd570156a00 2024-03-06T07:23:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1880409927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1880409927},{"type":"v1","addr":"172.21.15.117:6836","nonce":1880409927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/174449231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":174449231},{"type":"v1","addr":"172.21.15.26:6835","nonce":174449231}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:44.728798+0000","modified":"2024-03-06T07:23:09.844837+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5618},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5618":{"gid":5618,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/378552977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":378552977},{"type":"v1","addr":"172.21.15.117:6837","nonce":378552977}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5618,"qdb_cluster":[5618]},"id":6}]} 2024-03-06T07:23:11.467 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 71 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd55006b4d0 msgr2=0x7fd55006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd55006b4d0 0x7fd55006d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd560036000 tx=0x7fd560008030 comp rx=0 tx=0).stop 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570156a00 msgr2=0x7fd570149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570156a00 0x7fd570149ff0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd56800cad0 tx=0x7fd56800cf90 comp rx=0 tx=0).stop 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 shutdown_connections 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd55006b4d0 0x7fd55006d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570156a00 0x7fd570149ff0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700c3d60 0x7fd570149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 --2- 172.21.15.26:0/2858622879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd570072720 0x7fd5700c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 >> 172.21.15.26:0/2858622879 conn(0x7fd5700aa630 msgr2=0x7fd57014cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 shutdown_connections 2024-03-06T07:23:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.467+0000 7fd574f3e640 1 -- 172.21.15.26:0/2858622879 wait complete. 2024-03-06T07:23:11.480 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.579+0000 7fe7a34a4640 1 Processor -- start 2024-03-06T07:23:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.579+0000 7fe7a34a4640 1 -- start start 2024-03-06T07:23:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a34a4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe79c0c88c0 0x7fe79c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a34a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe79c158b60 0x7fe79c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a34a4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79c0c73d0 con 0x7fe79c14f360 2024-03-06T07:23:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a34a4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79c0c8dd0 con 0x7fe79c158b60 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a34a4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79c0be2e0 con 0x7fe79c0c88c0 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a1a41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe79c158b60 0x7fe79c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a1a41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe79c158b60 0x7fe79c158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60806/0 (socket says 172.21.15.26:60806) 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a1a41640 1 -- 172.21.15.26:0/884691265 learned_addr learned my addr 172.21.15.26:0/884691265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.580+0000 7fe7a0a3f640 1 --2- 172.21.15.26:0/884691265 >> v2:172.21.15.117:3300/0 conn(0x7fe79c0c88c0 0x7fe79c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.581+0000 7fe7a1a41640 1 -- 172.21.15.26:0/884691265 >> v2:172.21.15.117:3300/0 conn(0x7fe79c0c88c0 msgr2=0x7fe79c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.581+0000 7fe7a1a41640 1 --2- 172.21.15.26:0/884691265 >> v2:172.21.15.117:3300/0 conn(0x7fe79c0c88c0 0x7fe79c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.581+0000 7fe7a1a41640 1 -- 172.21.15.26:0/884691265 >> v1:172.21.15.117:6789/0 conn(0x7fe79c14f360 legacy=0x7fe79c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.581+0000 7fe79bfff640 1 -- 172.21.15.26:0/884691265 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840783015 0 0) 0x7fe79c0c73d0 con 0x7fe79c14f360 2024-03-06T07:23:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.581+0000 7fe7a1a41640 1 -- 172.21.15.26:0/884691265 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c0c7670 con 0x7fe79c158b60 2024-03-06T07:23:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.581+0000 7fe7a1a41640 1 --2- 172.21.15.26:0/884691265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c158b60 0x7fe79c158ff0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0c84c0 tx=0x7fe78c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1399742130c8ca4b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.581+0000 7fe79bfff640 1 -- 172.21.15.26:0/884691265 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c02ca80 con 0x7fe79c158b60 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe79bfff640 1 -- 172.21.15.26:0/884691265 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c02cc20 con 0x7fe79c158b60 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe79bfff640 1 -- 172.21.15.26:0/884691265 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0343f0 con 0x7fe79c158b60 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 -- 172.21.15.26:0/884691265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c158b60 msgr2=0x7fe79c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/884691265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c158b60 0x7fe79c158ff0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0c84c0 tx=0x7fe78c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 -- 172.21.15.26:0/884691265 shutdown_connections 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/884691265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c158b60 0x7fe79c158ff0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/884691265 >> v2:172.21.15.117:3300/0 conn(0x7fe79c0c88c0 0x7fe79c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 -- 172.21.15.26:0/884691265 >> 172.21.15.26:0/884691265 conn(0x7fe79c0ac570 msgr2=0x7fe79c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 -- 172.21.15.26:0/884691265 shutdown_connections 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.582+0000 7fe7a34a4640 1 -- 172.21.15.26:0/884691265 wait complete. 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.583+0000 7fe7a34a4640 1 Processor -- start 2024-03-06T07:23:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.583+0000 7fe7a34a4640 1 -- start start 2024-03-06T07:23:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a34a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe79c0c88c0 0x7fe79c0c2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a34a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c14f360 0x7fe79c0bf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a34a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe79c158b60 0x7fe79c1ee420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a34a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0c5a50 con 0x7fe79c14f360 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a34a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0c7c30 con 0x7fe79c0c88c0 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a34a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0c5bd0 con 0x7fe79c158b60 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c14f360 0x7fe79c0bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c14f360 0x7fe79c0bf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60818/0 (socket says 172.21.15.26:60818) 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 -- 172.21.15.26:0/3913243227 learned_addr learned my addr 172.21.15.26:0/3913243227 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a1240640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe79c0c88c0 0x7fe79c0c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a1a41640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe79c158b60 0x7fe79c1ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 -- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe79c158b60 msgr2=0x7fe79c1ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe79c158b60 0x7fe79c1ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 -- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe79c0c88c0 msgr2=0x7fe79c0c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe79c0c88c0 0x7fe79c0c2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 -- 172.21.15.26:0/3913243227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c0c2950 con 0x7fe79c14f360 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.584+0000 7fe7a0a3f640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c14f360 0x7fe79c0bf760 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e840 tx=0x7fe79000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.585+0000 7fe799ffb640 1 -- 172.21.15.26:0/3913243227 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790017070 con 0x7fe79c14f360 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.585+0000 7fe799ffb640 1 -- 172.21.15.26:0/3913243227 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79000c940 con 0x7fe79c14f360 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.585+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c0befa0 con 0x7fe79c14f360 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.585+0000 7fe799ffb640 1 -- 172.21.15.26:0/3913243227 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79000cd10 con 0x7fe79c14f360 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.585+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c0c1210 con 0x7fe79c14f360 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.586+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c04f330 con 0x7fe79c14f360 2024-03-06T07:23:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.586+0000 7fe799ffb640 1 -- 172.21.15.26:0/3913243227 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe79001a070 con 0x7fe79c14f360 2024-03-06T07:23:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.587+0000 7fe799ffb640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe78806b4d0 0x7fe78806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.587+0000 7fe799ffb640 1 -- 172.21.15.26:0/3913243227 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fe79008a250 con 0x7fe79c14f360 2024-03-06T07:23:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.587+0000 7fe7a1240640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe78806b4d0 0x7fe78806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.587+0000 7fe7a1240640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe78806b4d0 0x7fe78806d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe794000f60 tx=0x7fe794009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.591+0000 7fe799ffb640 1 -- 172.21.15.26:0/3913243227 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe7900587f0 con 0x7fe79c14f360 2024-03-06T07:23:11.760 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:11.762 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:11.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.772+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe79c06cda0 con 0x7fe79c14f360 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.774+0000 7fe799ffb640 1 -- 172.21.15.26:0/3913243227 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17290 (secure 0 0 0) 0x7fe79005c890 con 0x7fe79c14f360 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:08.840150+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:42.855764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:43.690413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.26:6835/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.117:6836/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6834/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6836/218606650":"2024-03-07T07:22:54.778025+0000","172.21.15.117:6834/218606650":"2024-03-07T07:22:54.778025+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:11.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe78806b4d0 msgr2=0x7fe78806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe78806b4d0 0x7fe78806d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe794000f60 tx=0x7fe794009040 comp rx=0 tx=0).stop 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c14f360 msgr2=0x7fe79c0bf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c14f360 0x7fe79c0bf760 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e840 tx=0x7fe79000ed00 comp rx=0 tx=0).stop 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 shutdown_connections 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe78806b4d0 0x7fe78806d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe79c158b60 0x7fe79c1ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe79c14f360 0x7fe79c0bf760 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 --2- 172.21.15.26:0/3913243227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe79c0c88c0 0x7fe79c0c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 >> 172.21.15.26:0/3913243227 conn(0x7fe79c0ac570 msgr2=0x7fe79c0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 shutdown_connections 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.776+0000 7fe7a34a4640 1 -- 172.21.15.26:0/3913243227 wait complete. 2024-03-06T07:23:11.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-06T07:23:11.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.957+0000 7f49a468f640 1 Processor -- start 2024-03-06T07:23:11.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.957+0000 7f49a468f640 1 -- start start 2024-03-06T07:23:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.957+0000 7f49a468f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f499c0c6ab0 0x7f499c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.957+0000 7f49a468f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c0a9990 con 0x7f499c1561a0 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.957+0000 7f49a468f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c1556c0 con 0x7f499c14f110 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.957+0000 7f49a468f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c155d10 con 0x7f499c0c6ab0 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.958+0000 7f49a242b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f499c0c6ab0 0x7f499c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.958+0000 7f49a1c2a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f499c1561a0 0x7f499c0a9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43802/0 (socket says 172.21.15.26:43802) 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.958+0000 7f49a1c2a640 1 -- 172.21.15.26:0/1078070060 learned_addr learned my addr 172.21.15.26:0/1078070060 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.958+0000 7f49a242b640 1 -- 172.21.15.26:0/1078070060 >> v1:172.21.15.117:6789/0 conn(0x7f499c14f110 legacy=0x7f499c15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.958+0000 7f49a242b640 1 -- 172.21.15.26:0/1078070060 >> v1:172.21.15.26:6789/0 conn(0x7f499c1561a0 legacy=0x7f499c0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.958+0000 7f49a242b640 1 -- 172.21.15.26:0/1078070060 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c1559a0 con 0x7f499c0c6ab0 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.958+0000 7f49a1429640 1 -- 172.21.15.26:0/1078070060 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624563619 0 0) 0x7f499c0a9990 con 0x7f499c1561a0 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a242b640 1 --2- 172.21.15.26:0/1078070060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c0c6ab0 0x7f499c1529e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f49980097a0 tx=0x7f499802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28358aa0056a9ecb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a1429640 1 -- 172.21.15.26:0/1078070060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998035020 con 0x7f499c0c6ab0 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a1429640 1 -- 172.21.15.26:0/1078070060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4998031830 con 0x7f499c0c6ab0 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a1429640 1 -- 172.21.15.26:0/1078070060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998031c00 con 0x7f499c0c6ab0 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a468f640 1 -- 172.21.15.26:0/1078070060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c0c6ab0 msgr2=0x7f499c1529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a468f640 1 --2- 172.21.15.26:0/1078070060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c0c6ab0 0x7f499c1529e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f49980097a0 tx=0x7f499802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a468f640 1 -- 172.21.15.26:0/1078070060 shutdown_connections 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a468f640 1 --2- 172.21.15.26:0/1078070060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c0c6ab0 0x7f499c1529e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a468f640 1 -- 172.21.15.26:0/1078070060 >> 172.21.15.26:0/1078070060 conn(0x7f499c06b1a0 msgr2=0x7f499c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a468f640 1 -- 172.21.15.26:0/1078070060 shutdown_connections 2024-03-06T07:23:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.959+0000 7f49a468f640 1 -- 172.21.15.26:0/1078070060 wait complete. 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 Processor -- start 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 -- start start 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c6ab0 0x7f499c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c14f110 0x7f499c149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c1561a0 0x7f499c0bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0aa600 con 0x7f499c1561a0 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0c2a90 con 0x7f499c14f110 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a468f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1ee110 con 0x7f499c0c6ab0 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a2c2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c1561a0 0x7f499c0bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a2c2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c1561a0 0x7f499c0bf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60838/0 (socket says 172.21.15.26:60838) 2024-03-06T07:23:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a2c2c640 1 -- 172.21.15.26:0/1076934857 learned_addr learned my addr 172.21.15.26:0/1076934857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a242b640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c6ab0 0x7f499c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.961+0000 7f49a1c2a640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c14f110 0x7f499c149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a2c2c640 1 -- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c6ab0 msgr2=0x7f499c14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a2c2c640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c6ab0 0x7f499c14c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a2c2c640 1 -- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c14f110 msgr2=0x7f499c149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a2c2c640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c14f110 0x7f499c149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a2c2c640 1 -- 172.21.15.26:0/1076934857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c1eba90 con 0x7f499c1561a0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a2c2c640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c1561a0 0x7f499c0bf9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f499400c7a0 tx=0x7f499400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f4992ffd640 1 -- 172.21.15.26:0/1076934857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994003b70 con 0x7f499c1561a0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f4992ffd640 1 -- 172.21.15.26:0/1076934857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994003d30 con 0x7f499c1561a0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c154540 con 0x7f499c1561a0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f4992ffd640 1 -- 172.21.15.26:0/1076934857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400aa30 con 0x7f499c1561a0 2024-03-06T07:23:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.962+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c154b00 con 0x7f499c1561a0 2024-03-06T07:23:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.963+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c04f330 con 0x7f499c1561a0 2024-03-06T07:23:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.964+0000 7f4992ffd640 1 -- 172.21.15.26:0/1076934857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f499400abd0 con 0x7f499c1561a0 2024-03-06T07:23:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.964+0000 7f4992ffd640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f498006b500 0x7f498006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.964+0000 7f4992ffd640 1 -- 172.21.15.26:0/1076934857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f499408a190 con 0x7f499c1561a0 2024-03-06T07:23:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.964+0000 7f49a242b640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f498006b500 0x7f498006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.965+0000 7f49a242b640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f498006b500 0x7f498006d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4998036000 tx=0x7f499803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:11.968+0000 7f4992ffd640 1 -- 172.21.15.26:0/1076934857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49940587c0 con 0x7f499c1561a0 2024-03-06T07:23:12.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.185+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7f499c0c26b0 con 0x7f499c1561a0 2024-03-06T07:23:12.187 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:23:12.186+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:23:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.187+0000 7f4992ffd640 1 -- 172.21.15.26:0/1076934857 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 163+0+0 (secure 0 0 0) 0x7f499405c860 con 0x7f499c1561a0 2024-03-06T07:23:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:23:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f498006b500 msgr2=0x7f498006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f498006b500 0x7f498006d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4998036000 tx=0x7f499803c040 comp rx=0 tx=0).stop 2024-03-06T07:23:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c1561a0 msgr2=0x7f499c0bf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c1561a0 0x7f499c0bf9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f499400c7a0 tx=0x7f499400cc60 comp rx=0 tx=0).stop 2024-03-06T07:23:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 shutdown_connections 2024-03-06T07:23:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f498006b500 0x7f498006d9c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499c1561a0 0x7f499c0bf9d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c14f110 0x7f499c149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 --2- 172.21.15.26:0/1076934857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c6ab0 0x7f499c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 >> 172.21.15.26:0/1076934857 conn(0x7f499c06b1a0 msgr2=0x7f499c0ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 shutdown_connections 2024-03-06T07:23:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.189+0000 7f49a468f640 1 -- 172.21.15.26:0/1076934857 wait complete. 2024-03-06T07:23:12.195 INFO:tasks.ceph.mds.d.smithi117.stderr: -67> 2024-03-06T07:23:10.308+0000 7f199b337640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:12.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-06T07:23:12.218 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:23:12.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.305+0000 7fe2d76b8640 1 Processor -- start 2024-03-06T07:23:12.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.305+0000 7fe2d76b8640 1 -- start start 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2d76b8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe2d00c5990 0x7fe2d00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2d76b8640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d00c3ef0 con 0x7fe2d006c900 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2d76b8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d01556d0 con 0x7fe2d00ab170 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2d76b8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d01559b0 con 0x7fe2d00c5990 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2d5c55640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe2d006c900 0x7fe2d00a9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43806/0 (socket says 172.21.15.26:43806) 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2d5c55640 1 -- 172.21.15.26:0/2259693971 learned_addr learned my addr 172.21.15.26:0/2259693971 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2d5454640 1 --2- 172.21.15.26:0/2259693971 >> v2:172.21.15.117:3300/0 conn(0x7fe2d00c5990 0x7fe2d00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053701872 0 0) 0x7fe2d00c3ef0 con 0x7fe2d006c900 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2bc003520 con 0x7fe2d006c900 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 228462241 0 0) 0x7fe2bc003520 con 0x7fe2d006c900 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 >> v2:172.21.15.117:3300/0 conn(0x7fe2d00c5990 msgr2=0x7fe2d00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 --2- 172.21.15.26:0/2259693971 >> v2:172.21.15.117:3300/0 conn(0x7fe2d00c5990 0x7fe2d00c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 >> v1:172.21.15.117:6789/0 conn(0x7fe2d00ab170 legacy=0x7fe2d006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d0155c90 con 0x7fe2d006c900 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe2c80019f0 con 0x7fe2d006c900 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 >> v1:172.21.15.26:6789/0 conn(0x7fe2d006c900 legacy=0x7fe2d00a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0039b0 0x7fe2bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2bc0043b0 0x7fe2bc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2bc005080 0x7fe2bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01559b0 con 0x7fe2bc0043b0 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc00bfe0 con 0x7fe2bc0039b0 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.306+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc00c1e0 con 0x7fe2bc005080 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d4c53640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2bc0043b0 0x7fe2bc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d5454640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0039b0 0x7fe2bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d5c55640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2bc005080 0x7fe2bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d4c53640 1 -- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2bc005080 msgr2=0x7fe2bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d4c53640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2bc005080 0x7fe2bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d4c53640 1 -- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0039b0 msgr2=0x7fe2bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d4c53640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0039b0 0x7fe2bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d4c53640 1 -- 172.21.15.26:0/2259693971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc003520 con 0x7fe2bc0043b0 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.307+0000 7fe2d5c55640 1 -- 172.21.15.26:0/2259693971 reap_dead start 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d4c53640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2bc0043b0 0x7fe2bc004880 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0001910 tx=0x7fe2bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9112094c29faf451 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00014e0 con 0x7fe2bc0043b0 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c0037070 con 0x7fe2bc0043b0 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2c7fff640 1 -- 172.21.15.26:0/2259693971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0032860 con 0x7fe2bc0043b0 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d76b8640 1 -- 172.21.15.26:0/2259693971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2bc0043b0 msgr2=0x7fe2bc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2bc0043b0 0x7fe2bc004880 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0001910 tx=0x7fe2bc005830 comp rx=0 tx=0).stop 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d76b8640 1 -- 172.21.15.26:0/2259693971 shutdown_connections 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/2259693971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2bc0043b0 0x7fe2bc004880 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d76b8640 1 -- 172.21.15.26:0/2259693971 >> 172.21.15.26:0/2259693971 conn(0x7fe2d00c6710 msgr2=0x7fe2d00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d76b8640 1 -- 172.21.15.26:0/2259693971 shutdown_connections 2024-03-06T07:23:12.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.308+0000 7fe2d76b8640 1 -- 172.21.15.26:0/2259693971 wait complete. 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.309+0000 7fe2d76b8640 1 Processor -- start 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.309+0000 7fe2d76b8640 1 -- start start 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d76b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0043b0 0x7fe2d00c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d76b8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2d014b100 0x7fe2d0151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d76b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d0151c90 0x7fe2d01dd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d76b8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d0156610 con 0x7fe2d0151c90 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d76b8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d0156490 con 0x7fe2bc0043b0 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d76b8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c3af0 con 0x7fe2d014b100 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5c55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d0151c90 0x7fe2d01dd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5c55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d0151c90 0x7fe2d01dd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60856/0 (socket says 172.21.15.26:60856) 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5c55640 1 -- 172.21.15.26:0/1980439268 learned_addr learned my addr 172.21.15.26:0/1980439268 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5454640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0043b0 0x7fe2d00c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d4c53640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2d014b100 0x7fe2d0151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5c55640 1 -- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2d014b100 msgr2=0x7fe2d0151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5c55640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2d014b100 0x7fe2d0151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5c55640 1 -- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0043b0 msgr2=0x7fe2d00c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.310+0000 7fe2d5c55640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0043b0 0x7fe2d00c3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.311+0000 7fe2d5c55640 1 -- 172.21.15.26:0/1980439268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d01ebb10 con 0x7fe2d0151c90 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.311+0000 7fe2d5c55640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d0151c90 0x7fe2d01dd010 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8000f80 tx=0x7fe2c8007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.311+0000 7fe2c5ffb640 1 -- 172.21.15.26:0/1980439268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8017020 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.311+0000 7fe2c5ffb640 1 -- 172.21.15.26:0/1980439268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c800ab30 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.311+0000 7fe2c5ffb640 1 -- 172.21.15.26:0/1980439268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80133f0 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.311+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d01ebdb0 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.311+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d01ec370 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.312+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d01ec6a0 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.312+0000 7fe2c5ffb640 1 -- 172.21.15.26:0/1980439268 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2c801a020 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.313+0000 7fe2c5ffb640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2b406b500 0x7fe2b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.313+0000 7fe2c5ffb640 1 -- 172.21.15.26:0/1980439268 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fe2c808ab30 con 0x7fe2d0151c90 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.313+0000 7fe2d5454640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2b406b500 0x7fe2b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.313+0000 7fe2d5454640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2b406b500 0x7fe2b406d9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00a9550 tx=0x7fe2cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.317+0000 7fe2c5ffb640 1 -- 172.21.15.26:0/1980439268 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe2c8021020 con 0x7fe2d0151c90 2024-03-06T07:23:12.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:12.535+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe2d004f330 con 0x7fe2d0151c90 2024-03-06T07:23:13.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.189+0000 7fe2c5ffb640 1 -- 172.21.15.26:0/1980439268 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe2c8059750 con 0x7fe2d0151c90 2024-03-06T07:23:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.192+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2b406b500 msgr2=0x7fe2b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.192+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2b406b500 0x7fe2b406d9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00a9550 tx=0x7fe2cc008040 comp rx=0 tx=0).stop 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d0151c90 msgr2=0x7fe2d01dd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d0151c90 0x7fe2d01dd010 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8000f80 tx=0x7fe2c8007b20 comp rx=0 tx=0).stop 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 shutdown_connections 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2b406b500 0x7fe2b406d9c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2d0151c90 0x7fe2d01dd010 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2d014b100 0x7fe2d0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 --2- 172.21.15.26:0/1980439268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc0043b0 0x7fe2d00c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 >> 172.21.15.26:0/1980439268 conn(0x7fe2d00c6710 msgr2=0x7fe2d006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 shutdown_connections 2024-03-06T07:23:13.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.193+0000 7fe2d76b8640 1 -- 172.21.15.26:0/1980439268 wait complete. 2024-03-06T07:23:13.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-06T07:23:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.318+0000 7f69a3ac9640 1 Processor -- start 2024-03-06T07:23:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.318+0000 7f69a3ac9640 1 -- start start 2024-03-06T07:23:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a3ac9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f699c06fe20 0x7f699c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a3ac9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f699c0c6780 0x7f699c072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a3ac9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c156150 con 0x7f699c072a50 2024-03-06T07:23:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a3ac9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699c0a7d70 con 0x7f699c06fe20 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a3ac9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699c0a8050 con 0x7f699c0c6780 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1865640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f699c06fe20 0x7f699c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1865640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f699c06fe20 0x7f699c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60864/0 (socket says 172.21.15.26:60864) 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1865640 1 -- 172.21.15.26:0/4091474419 learned_addr learned my addr 172.21.15.26:0/4091474419 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1064640 1 --2- 172.21.15.26:0/4091474419 >> v2:172.21.15.117:3300/0 conn(0x7f699c0c6780 0x7f699c072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1865640 1 -- 172.21.15.26:0/4091474419 >> v2:172.21.15.117:3300/0 conn(0x7f699c0c6780 msgr2=0x7f699c072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1865640 1 --2- 172.21.15.26:0/4091474419 >> v2:172.21.15.117:3300/0 conn(0x7f699c0c6780 0x7f699c072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1865640 1 -- 172.21.15.26:0/4091474419 >> v1:172.21.15.117:6789/0 conn(0x7f699c072a50 legacy=0x7f699c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1865640 1 -- 172.21.15.26:0/4091474419 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0a8330 con 0x7f699c06fe20 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.319+0000 7f69a1064640 1 --2- 172.21.15.26:0/4091474419 >> v2:172.21.15.117:3300/0 conn(0x7f699c0c6780 0x7f699c072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.320+0000 7f69a1865640 1 --2- 172.21.15.26:0/4091474419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c155c10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f699c0722d0 tx=0x7f699402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f071b0ebb23f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.320+0000 7f69a0863640 1 -- 172.21.15.26:0/4091474419 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699402e070 con 0x7f699c06fe20 2024-03-06T07:23:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.320+0000 7f69a0863640 1 -- 172.21.15.26:0/4091474419 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6994037070 con 0x7f699c06fe20 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.320+0000 7f69a0863640 1 -- 172.21.15.26:0/4091474419 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994031a00 con 0x7f699c06fe20 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 -- 172.21.15.26:0/4091474419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 msgr2=0x7f699c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/4091474419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c155c10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f699c0722d0 tx=0x7f699402b880 comp rx=0 tx=0).stop 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 -- 172.21.15.26:0/4091474419 shutdown_connections 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/4091474419 >> v2:172.21.15.117:3300/0 conn(0x7f699c0c6780 0x7f699c072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/4091474419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c155c10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 -- 172.21.15.26:0/4091474419 >> 172.21.15.26:0/4091474419 conn(0x7f699c0aa630 msgr2=0x7f699c0aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 -- 172.21.15.26:0/4091474419 shutdown_connections 2024-03-06T07:23:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.321+0000 7f69a3ac9640 1 -- 172.21.15.26:0/4091474419 wait complete. 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.322+0000 7f69a3ac9640 1 Processor -- start 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.322+0000 7f69a3ac9640 1 -- start start 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a3ac9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c1ee560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a3ac9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c072a50 0x7f699c0acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a3ac9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c0c6780 0x7f699c0ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a3ac9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c070200 con 0x7f699c06fe20 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a3ac9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0c3600 con 0x7f699c0c6780 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a3ac9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c158e90 con 0x7f699c072a50 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c1ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c1ee560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60874/0 (socket says 172.21.15.26:60874) 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 -- 172.21.15.26:0/1285934887 learned_addr learned my addr 172.21.15.26:0/1285934887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a2066640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c0c6780 0x7f699c0ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1064640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c072a50 0x7f699c0acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 -- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c072a50 msgr2=0x7f699c0acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c072a50 0x7f699c0acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 -- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c0c6780 msgr2=0x7f699c0ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c0c6780 0x7f699c0ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 -- 172.21.15.26:0/1285934887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c1ef210 con 0x7f699c06fe20 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1064640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c072a50 0x7f699c0acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a1865640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c1ee560 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6994000c00 tx=0x7f69940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f699a7fc640 1 -- 172.21.15.26:0/1285934887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994032880 con 0x7f699c06fe20 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f699a7fc640 1 -- 172.21.15.26:0/1285934887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6994032a20 con 0x7f699c06fe20 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f699a7fc640 1 -- 172.21.15.26:0/1285934887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994032e10 con 0x7f699c06fe20 2024-03-06T07:23:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.323+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c1eef30 con 0x7f699c06fe20 2024-03-06T07:23:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.324+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c1debe0 con 0x7f699c06fe20 2024-03-06T07:23:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.324+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c04f330 con 0x7f699c06fe20 2024-03-06T07:23:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.325+0000 7f699a7fc640 1 -- 172.21.15.26:0/1285934887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69940453f0 con 0x7f699c06fe20 2024-03-06T07:23:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.325+0000 7f699a7fc640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.325+0000 7f699a7fc640 1 -- 172.21.15.26:0/1285934887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f69940ac590 con 0x7f699c06fe20 2024-03-06T07:23:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.325+0000 7f69a1064640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.326+0000 7f69a1064640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.329+0000 7f699a7fc640 1 -- 172.21.15.26:0/1285934887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6994040070 con 0x7f699c06fe20 2024-03-06T07:23:13.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:13.513+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f699c0730d0 con 0x7f699c06fe20 2024-03-06T07:23:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.190+0000 7f699a7fc640 1 -- 172.21.15.26:0/1285934887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v85) v1 ==== 184+0+0 (secure 0 0 0) 0x7f699c156150 con 0x7f699c06fe20 2024-03-06T07:23:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_metadata' removed 2024-03-06T07:23:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 msgr2=0x7f698806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).stop 2024-03-06T07:23:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 msgr2=0x7f699c1ee560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c1ee560 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6994000c00 tx=0x7f69940046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 shutdown_connections 2024-03-06T07:23:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f698806b4d0 0x7f698806d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c0c6780 0x7f699c0ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f699c072a50 0x7f699c0acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 --2- 172.21.15.26:0/1285934887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c06fe20 0x7f699c1ee560 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 >> 172.21.15.26:0/1285934887 conn(0x7f699c0aa630 msgr2=0x7f699c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 shutdown_connections 2024-03-06T07:23:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.194+0000 7f69a3ac9640 1 -- 172.21.15.26:0/1285934887 wait complete. 2024-03-06T07:23:14.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-06T07:23:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 Processor -- start 2024-03-06T07:23:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 -- start start 2024-03-06T07:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feaec152870 0x7feaec155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feaec0c88e0 0x7feaec0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec06bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaec06d320 con 0x7feaec152870 2024-03-06T07:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaec06b8d0 con 0x7feaec0c88e0 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaf2925640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec156100 con 0x7feaec0c7930 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.319+0000 7feaebfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feaec152870 0x7feaec155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaebfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feaec152870 0x7feaec155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60878/0 (socket says 172.21.15.26:60878) 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaebfff640 1 -- 172.21.15.26:0/2479686177 learned_addr learned my addr 172.21.15.26:0/2479686177 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaeb7fe640 1 --2- 172.21.15.26:0/2479686177 >> v2:172.21.15.117:3300/0 conn(0x7feaec0c88e0 0x7feaec0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaf0ec2640 1 --2- 172.21.15.26:0/2479686177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec06bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaebfff640 1 -- 172.21.15.26:0/2479686177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 msgr2=0x7feaec06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaebfff640 1 --2- 172.21.15.26:0/2479686177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec06bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaebfff640 1 -- 172.21.15.26:0/2479686177 >> v2:172.21.15.117:3300/0 conn(0x7feaec0c88e0 msgr2=0x7feaec0c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaebfff640 1 --2- 172.21.15.26:0/2479686177 >> v2:172.21.15.117:3300/0 conn(0x7feaec0c88e0 0x7feaec0c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaebfff640 1 -- 172.21.15.26:0/2479686177 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec0c5400 con 0x7feaec152870 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaeb7fe640 1 --2- 172.21.15.26:0/2479686177 >> v2:172.21.15.117:3300/0 conn(0x7feaec0c88e0 0x7feaec0c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.320+0000 7feaf0ec2640 1 --2- 172.21.15.26:0/2479686177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec06bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaebfff640 1 --2- 172.21.15.26:0/2479686177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec155bc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feaec0cba40 tx=0x7feadc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a5e250a2c556f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaeaffd640 1 -- 172.21.15.26:0/2479686177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc02e070 con 0x7feaec152870 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaeaffd640 1 -- 172.21.15.26:0/2479686177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc037070 con 0x7feaec152870 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaeaffd640 1 -- 172.21.15.26:0/2479686177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc031a00 con 0x7feaec152870 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 -- 172.21.15.26:0/2479686177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 msgr2=0x7feaec155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 --2- 172.21.15.26:0/2479686177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec155bc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feaec0cba40 tx=0x7feadc02b880 comp rx=0 tx=0).stop 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 -- 172.21.15.26:0/2479686177 shutdown_connections 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 --2- 172.21.15.26:0/2479686177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec06bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 --2- 172.21.15.26:0/2479686177 >> v2:172.21.15.117:3300/0 conn(0x7feaec0c88e0 0x7feaec0c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 --2- 172.21.15.26:0/2479686177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec155bc0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 -- 172.21.15.26:0/2479686177 >> 172.21.15.26:0/2479686177 conn(0x7feaec0ac5f0 msgr2=0x7feaec0aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 -- 172.21.15.26:0/2479686177 shutdown_connections 2024-03-06T07:23:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.321+0000 7feaf2925640 1 -- 172.21.15.26:0/2479686177 wait complete. 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.322+0000 7feaf2925640 1 Processor -- start 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf2925640 1 -- start start 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf2925640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec1eea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf2925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaec0c88e0 0x7feaec0c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf2925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf2925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec0c5ca0 con 0x7feaec152870 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf2925640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec0c59c0 con 0x7feaec0c88e0 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf2925640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec0c9470 con 0x7feaec0c7930 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf0ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf0ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60892/0 (socket says 172.21.15.26:60892) 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.323+0000 7feaf0ec2640 1 -- 172.21.15.26:0/2488078790 learned_addr learned my addr 172.21.15.26:0/2488078790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaebfff640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec1eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaeb7fe640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaec0c88e0 0x7feaec0c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf0ec2640 1 -- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 msgr2=0x7feaec1eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf0ec2640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec1eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf0ec2640 1 -- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaec0c88e0 msgr2=0x7feaec0c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf0ec2640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaec0c88e0 0x7feaec0c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf0ec2640 1 -- 172.21.15.26:0/2488078790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec0c38f0 con 0x7feaec152870 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaebfff640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec1eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaeb7fe640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaec0c88e0 0x7feaec0c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf0ec2640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec144080 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7feae400cd10 tx=0x7feae400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feae8ff9640 1 -- 172.21.15.26:0/2488078790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400f680 con 0x7feaec152870 2024-03-06T07:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feae8ff9640 1 -- 172.21.15.26:0/2488078790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae400f840 con 0x7feaec152870 2024-03-06T07:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec0c3bb0 con 0x7feaec152870 2024-03-06T07:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feae8ff9640 1 -- 172.21.15.26:0/2488078790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400fc30 con 0x7feaec152870 2024-03-06T07:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.324+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec0aef10 con 0x7feaec152870 2024-03-06T07:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.325+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec0af110 con 0x7feaec152870 2024-03-06T07:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.325+0000 7feae8ff9640 1 -- 172.21.15.26:0/2488078790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feae40213f0 con 0x7feaec152870 2024-03-06T07:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.326+0000 7feae8ff9640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fead806b4d0 0x7fead806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.326+0000 7feae8ff9640 1 -- 172.21.15.26:0/2488078790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6009+0+0 (secure 0 0 0) 0x7feae408afc0 con 0x7feaec152870 2024-03-06T07:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.326+0000 7feaebfff640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fead806b4d0 0x7fead806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.326+0000 7feaebfff640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fead806b4d0 0x7fead806d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feadc036000 tx=0x7feadc03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.330+0000 7feae8ff9640 1 -- 172.21.15.26:0/2488078790 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feae4014030 con 0x7feaec152870 2024-03-06T07:23:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:14.512+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feaec04f330 con 0x7feaec152870 2024-03-06T07:23:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.192+0000 7feae8ff9640 1 -- 172.21.15.26:0/2488078790 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v86) v1 ==== 172+0+0 (secure 0 0 0) 0x7feae4059750 con 0x7feaec152870 2024-03-06T07:23:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_data' removed 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fead806b4d0 msgr2=0x7fead806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fead806b4d0 0x7fead806d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feadc036000 tx=0x7feadc03d040 comp rx=0 tx=0).stop 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 msgr2=0x7feaec144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec144080 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7feae400cd10 tx=0x7feae400a650 comp rx=0 tx=0).stop 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 shutdown_connections 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fead806b4d0 0x7fead806d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaec152870 0x7feaec144080 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaec0c88e0 0x7feaec0c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 --2- 172.21.15.26:0/2488078790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feaec0c7930 0x7feaec1eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 >> 172.21.15.26:0/2488078790 conn(0x7feaec0ac5f0 msgr2=0x7feaec0aba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 shutdown_connections 2024-03-06T07:23:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.196+0000 7feaf2925640 1 -- 172.21.15.26:0/2488078790 wait complete. 2024-03-06T07:23:15.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.c mds_join_fs 2024-03-06T07:23:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.320+0000 7fbbd9dc7640 1 Processor -- start 2024-03-06T07:23:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd9dc7640 1 -- start start 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd9dc7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbd406afc0 0x7fbbd406b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd9dc7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbbd40c2120 0x7fbbd4158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd9dc7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd40c5840 con 0x7fbbd40c2580 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd9dc7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd4158790 con 0x7fbbd406afc0 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd9dc7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd40c5b20 con 0x7fbbd40c2120 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbd406afc0 0x7fbbd406b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbd406afc0 0x7fbbd406b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60906/0 (socket says 172.21.15.26:60906) 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.321+0000 7fbbd37fe640 1 -- 172.21.15.26:0/3903170400 learned_addr learned my addr 172.21.15.26:0/3903170400 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.322+0000 7fbbd2ffd640 1 --2- 172.21.15.26:0/3903170400 >> v2:172.21.15.117:3300/0 conn(0x7fbbd40c2120 0x7fbbd4158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.322+0000 7fbbd37fe640 1 -- 172.21.15.26:0/3903170400 >> v2:172.21.15.117:3300/0 conn(0x7fbbd40c2120 msgr2=0x7fbbd4158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.322+0000 7fbbd37fe640 1 --2- 172.21.15.26:0/3903170400 >> v2:172.21.15.117:3300/0 conn(0x7fbbd40c2120 0x7fbbd4158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.322+0000 7fbbd37fe640 1 -- 172.21.15.26:0/3903170400 >> v1:172.21.15.117:6789/0 conn(0x7fbbd40c2580 legacy=0x7fbbd4159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.322+0000 7fbbd37fe640 1 -- 172.21.15.26:0/3903170400 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd40c5e00 con 0x7fbbd406afc0 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.322+0000 7fbbd37fe640 1 --2- 172.21.15.26:0/3903170400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd406afc0 0x7fbbd406b3a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40710b0 tx=0x7fbbc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b119a8515400a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd27fc640 1 -- 172.21.15.26:0/3903170400 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4035020 con 0x7fbbd406afc0 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd27fc640 1 -- 172.21.15.26:0/3903170400 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc4031870 con 0x7fbbd406afc0 2024-03-06T07:23:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd27fc640 1 -- 172.21.15.26:0/3903170400 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4031c40 con 0x7fbbd406afc0 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/3903170400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd406afc0 msgr2=0x7fbbd406b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/3903170400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd406afc0 0x7fbbd406b3a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40710b0 tx=0x7fbbc402b910 comp rx=0 tx=0).stop 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/3903170400 shutdown_connections 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/3903170400 >> v2:172.21.15.117:3300/0 conn(0x7fbbd40c2120 0x7fbbd4158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/3903170400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd406afc0 0x7fbbd406b3a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/3903170400 >> 172.21.15.26:0/3903170400 conn(0x7fbbd40c4220 msgr2=0x7fbbd40c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/3903170400 shutdown_connections 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.323+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/3903170400 wait complete. 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.324+0000 7fbbd9dc7640 1 Processor -- start 2024-03-06T07:23:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.324+0000 7fbbd9dc7640 1 -- start start 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd9dc7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbd406afc0 0x7fbbd4072210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd9dc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd40c2120 0x7fbbd4157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd9dc7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbd40c2580 0x7fbbd41eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd9dc7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40c2b30 con 0x7fbbd40c2120 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd9dc7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd4155220 con 0x7fbbd40c2580 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd9dc7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40c2cb0 con 0x7fbbd406afc0 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd40c2120 0x7fbbd4157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd40c2120 0x7fbbd4157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60910/0 (socket says 172.21.15.26:60910) 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd2ffd640 1 -- 172.21.15.26:0/1358660729 learned_addr learned my addr 172.21.15.26:0/1358660729 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd3fff640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbd40c2580 0x7fbbd41eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.325+0000 7fbbd37fe640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbd406afc0 0x7fbbd4072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd2ffd640 1 -- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbd406afc0 msgr2=0x7fbbd4072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd2ffd640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbd406afc0 0x7fbbd4072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd2ffd640 1 -- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbd40c2580 msgr2=0x7fbbd41eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd2ffd640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbd40c2580 0x7fbbd41eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd2ffd640 1 -- 172.21.15.26:0/1358660729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd4072f30 con 0x7fbbd40c2120 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd2ffd640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd40c2120 0x7fbbd4157980 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800d800 tx=0x7fbbc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbbbfff640 1 -- 172.21.15.26:0/1358660729 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800b6c0 con 0x7fbbd40c2120 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbbbfff640 1 -- 172.21.15.26:0/1358660729 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc8019070 con 0x7fbbd40c2120 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd4073210 con 0x7fbbd40c2120 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbbbfff640 1 -- 172.21.15.26:0/1358660729 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800bac0 con 0x7fbbd40c2120 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.326+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd4157ec0 con 0x7fbbd40c2120 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.327+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd40733e0 con 0x7fbbd40c2120 2024-03-06T07:23:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.327+0000 7fbbbbfff640 1 -- 172.21.15.26:0/1358660729 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbc800bc60 con 0x7fbbd40c2120 2024-03-06T07:23:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.328+0000 7fbbbbfff640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.328+0000 7fbbbbfff640 1 -- 172.21.15.26:0/1358660729 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fbbc808a490 con 0x7fbbd40c2120 2024-03-06T07:23:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.328+0000 7fbbd37fe640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.329+0000 7fbbd37fe640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbc006b4d0 0x7fbbc006d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4036000 tx=0x7fbbc403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.332+0000 7fbbbbfff640 1 -- 172.21.15.26:0/1358660729 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbbc8058dc0 con 0x7fbbd40c2120 2024-03-06T07:23:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.527+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"} v 0) v1 -- 0x7fbbd404f330 con 0x7fbbd40c2120 2024-03-06T07:23:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.531+0000 7fbbbbfff640 1 -- 172.21.15.26:0/1358660729 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"}]=0 v27) v1 ==== 96+0+0 (secure 0 0 0) 0x7fbbc805ce60 con 0x7fbbd40c2120 2024-03-06T07:23:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbc006b4d0 msgr2=0x7fbbc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbc006b4d0 0x7fbbc006d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4036000 tx=0x7fbbc403c040 comp rx=0 tx=0).stop 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd40c2120 msgr2=0x7fbbd4157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd40c2120 0x7fbbd4157980 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800d800 tx=0x7fbbc800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 shutdown_connections 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbd40c2580 0x7fbbd41eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbd40c2120 0x7fbbd4157980 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 --2- 172.21.15.26:0/1358660729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbd406afc0 0x7fbbd4072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 >> 172.21.15.26:0/1358660729 conn(0x7fbbd40c4220 msgr2=0x7fbbd41546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 shutdown_connections 2024-03-06T07:23:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.533+0000 7fbbd9dc7640 1 -- 172.21.15.26:0/1358660729 wait complete. 2024-03-06T07:23:15.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:23:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.646+0000 7f2a2bab5640 1 Processor -- start 2024-03-06T07:23:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.646+0000 7f2a2bab5640 1 -- start start 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2bab5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a24158c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2bab5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a24155300 con 0x7f2a240c4d10 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2bab5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a241555e0 con 0x7f2a240c4930 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2bab5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241558c0 con 0x7f2a241587e0 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a29050640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a240c4d10 0x7f2a240c1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43816/0 (socket says 172.21.15.26:43816) 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a29050640 1 -- 172.21.15.26:0/2587614519 learned_addr learned my addr 172.21.15.26:0/2587614519 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2a052640 1 --2- 172.21.15.26:0/2587614519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a24158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2884f640 1 -- 172.21.15.26:0/2587614519 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561187799 0 0) 0x7f2a24155300 con 0x7f2a240c4d10 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2884f640 1 -- 172.21.15.26:0/2587614519 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a10003520 con 0x7f2a240c4d10 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2884f640 1 -- 172.21.15.26:0/2587614519 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696072803 0 0) 0x7f2a241555e0 con 0x7f2a240c4930 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2a052640 1 -- 172.21.15.26:0/2587614519 >> v1:172.21.15.117:6789/0 conn(0x7f2a240c4930 legacy=0x7f2a240c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2a052640 1 -- 172.21.15.26:0/2587614519 >> v1:172.21.15.26:6789/0 conn(0x7f2a240c4d10 legacy=0x7f2a240c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.647+0000 7f2a2a052640 1 -- 172.21.15.26:0/2587614519 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24155ba0 con 0x7f2a241587e0 2024-03-06T07:23:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.648+0000 7f2a2a052640 1 --2- 172.21.15.26:0/2587614519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a24158c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097d0 tx=0x7f2a1c02b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9210c4f84da67ab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.648+0000 7f2a2884f640 1 -- 172.21.15.26:0/2587614519 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c037070 con 0x7f2a241587e0 2024-03-06T07:23:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.648+0000 7f2a2884f640 1 -- 172.21.15.26:0/2587614519 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c030510 con 0x7f2a241587e0 2024-03-06T07:23:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.648+0000 7f2a2884f640 1 -- 172.21.15.26:0/2587614519 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14001670 con 0x7f2a241587e0 2024-03-06T07:23:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.649+0000 7f2a2bab5640 1 -- 172.21.15.26:0/2587614519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 msgr2=0x7f2a24158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.649+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/2587614519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a24158c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097d0 tx=0x7f2a1c02b690 comp rx=0 tx=0).stop 2024-03-06T07:23:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.649+0000 7f2a2bab5640 1 -- 172.21.15.26:0/2587614519 shutdown_connections 2024-03-06T07:23:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.649+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/2587614519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a24158c70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.649+0000 7f2a2bab5640 1 -- 172.21.15.26:0/2587614519 >> 172.21.15.26:0/2587614519 conn(0x7f2a240aa5c0 msgr2=0x7f2a240aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.649+0000 7f2a2bab5640 1 -- 172.21.15.26:0/2587614519 shutdown_connections 2024-03-06T07:23:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.649+0000 7f2a2bab5640 1 -- 172.21.15.26:0/2587614519 wait complete. 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.650+0000 7f2a2bab5640 1 Processor -- start 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.650+0000 7f2a2bab5640 1 -- start start 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a2bab5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a240c4930 0x7f2a241543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a2bab5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a240c4d10 0x7f2a241eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a2bab5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a241eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a2bab5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241560c0 con 0x7f2a240c4930 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a2bab5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240c8560 con 0x7f2a240c4d10 2024-03-06T07:23:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a2bab5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240c81c0 con 0x7f2a241587e0 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a240c4930 0x7f2a241543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a240c4930 0x7f2a241543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60914/0 (socket says 172.21.15.26:60914) 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 -- 172.21.15.26:0/3842895598 learned_addr learned my addr 172.21.15.26:0/3842895598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a2a052640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a241eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29050640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a240c4d10 0x7f2a241eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 -- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 msgr2=0x7f2a241eaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a241eaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 -- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a240c4d10 msgr2=0x7f2a241eaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a240c4d10 0x7f2a241eaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 -- 172.21.15.26:0/3842895598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240abe30 con 0x7f2a240c4930 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a29851640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a240c4930 0x7f2a241543a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000c890 tx=0x7f2a2000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.651+0000 7f2a1a7fc640 1 -- 172.21.15.26:0/3842895598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20017070 con 0x7f2a240c4930 2024-03-06T07:23:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.652+0000 7f2a1a7fc640 1 -- 172.21.15.26:0/3842895598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20003dc0 con 0x7f2a240c4930 2024-03-06T07:23:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.652+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a241ee100 con 0x7f2a240c4930 2024-03-06T07:23:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.652+0000 7f2a1a7fc640 1 -- 172.21.15.26:0/3842895598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a200052b0 con 0x7f2a240c4930 2024-03-06T07:23:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.652+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a241ee6c0 con 0x7f2a240c4930 2024-03-06T07:23:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.653+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a24069460 con 0x7f2a240c4930 2024-03-06T07:23:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.653+0000 7f2a1a7fc640 1 -- 172.21.15.26:0/3842895598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a2001a070 con 0x7f2a240c4930 2024-03-06T07:23:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.653+0000 7f2a1a7fc640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a0806b4d0 0x7f2a0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.654+0000 7f2a1a7fc640 1 -- 172.21.15.26:0/3842895598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f2a20089870 con 0x7f2a240c4930 2024-03-06T07:23:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.654+0000 7f2a29050640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a0806b4d0 0x7f2a0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.654+0000 7f2a29050640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a0806b4d0 0x7f2a0806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2a240adfb0 tx=0x7f2a14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.658+0000 7f2a1a7fc640 1 -- 172.21.15.26:0/3842895598 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2a24069460 con 0x7f2a240c4930 2024-03-06T07:23:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.838+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2a2404f330 con 0x7f2a240c4930 2024-03-06T07:23:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.841+0000 7f2a1a7fc640 1 -- 172.21.15.26:0/3842895598 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2a200587f0 con 0x7f2a240c4930 2024-03-06T07:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a0806b4d0 msgr2=0x7f2a0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a0806b4d0 0x7f2a0806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2a240adfb0 tx=0x7f2a14009040 comp rx=0 tx=0).stop 2024-03-06T07:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a240c4930 msgr2=0x7f2a241543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a240c4930 0x7f2a241543a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000c890 tx=0x7f2a2000cd50 comp rx=0 tx=0).stop 2024-03-06T07:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 shutdown_connections 2024-03-06T07:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a0806b4d0 0x7f2a0806d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a241587e0 0x7f2a241eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a240c4d10 0x7f2a241eaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 --2- 172.21.15.26:0/3842895598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a240c4930 0x7f2a241543a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 >> 172.21.15.26:0/3842895598 conn(0x7f2a240aa5c0 msgr2=0x7f2a240c3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 shutdown_connections 2024-03-06T07:23:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.843+0000 7f2a2bab5640 1 -- 172.21.15.26:0/3842895598 wait complete. 2024-03-06T07:23:15.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:23:15.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.957+0000 7f0060bc7640 1 Processor -- start 2024-03-06T07:23:15.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.958+0000 7f0060bc7640 1 -- start start 2024-03-06T07:23:15.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.958+0000 7f0060bc7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0c5990 0x7f005c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.958+0000 7f0060bc7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f005c0ab170 0x7f005c06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.958+0000 7f0060bc7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c0c3ef0 con 0x7f005c06c730 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.958+0000 7f0060bc7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005c06cbe0 con 0x7f005c0ab170 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.958+0000 7f0060bc7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c1556d0 con 0x7f005c0c5990 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f0059d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f005c0ab170 0x7f005c06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f005a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0c5990 0x7f005c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f005ad9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f005c06c730 0x7f005c1523f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:38482/0 (socket says 172.21.15.26:38482) 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f005ad9d640 1 -- 172.21.15.26:0/2269112240 learned_addr learned my addr 172.21.15.26:0/2269112240 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f0059d9b640 1 -- 172.21.15.26:0/2269112240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0c5990 msgr2=0x7f005c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f0059d9b640 1 --2- 172.21.15.26:0/2269112240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0c5990 0x7f005c0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f0059d9b640 1 -- 172.21.15.26:0/2269112240 >> v1:172.21.15.117:6789/0 conn(0x7f005c06c730 legacy=0x7f005c1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f005959a640 1 -- 172.21.15.26:0/2269112240 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888575333 0 0) 0x7f005c0c3ef0 con 0x7f005c06c730 2024-03-06T07:23:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.959+0000 7f0059d9b640 1 -- 172.21.15.26:0/2269112240 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c1559b0 con 0x7f005c0ab170 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.960+0000 7f0059d9b640 1 --2- 172.21.15.26:0/2269112240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c0ab170 0x7f005c06c1f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f00440097a0 tx=0x7f004402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb36085d6424e14 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.960+0000 7f005959a640 1 -- 172.21.15.26:0/2269112240 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00440315a0 con 0x7f005c0ab170 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.960+0000 7f005959a640 1 -- 172.21.15.26:0/2269112240 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044037070 con 0x7f005c0ab170 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.960+0000 7f005959a640 1 -- 172.21.15.26:0/2269112240 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044031960 con 0x7f005c0ab170 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 -- 172.21.15.26:0/2269112240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c0ab170 msgr2=0x7f005c06c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 --2- 172.21.15.26:0/2269112240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c0ab170 0x7f005c06c1f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f00440097a0 tx=0x7f004402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 -- 172.21.15.26:0/2269112240 shutdown_connections 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 --2- 172.21.15.26:0/2269112240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c0ab170 0x7f005c06c1f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 --2- 172.21.15.26:0/2269112240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0c5990 0x7f005c0c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 -- 172.21.15.26:0/2269112240 >> 172.21.15.26:0/2269112240 conn(0x7f005c0c6710 msgr2=0x7f005c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 -- 172.21.15.26:0/2269112240 shutdown_connections 2024-03-06T07:23:15.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.961+0000 7f0060bc7640 1 -- 172.21.15.26:0/2269112240 wait complete. 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.962+0000 7f0060bc7640 1 Processor -- start 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.962+0000 7f0060bc7640 1 -- start start 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0060bc7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c06c730 0x7f005c0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0060bc7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0ab170 0x7f005c1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0060bc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005c0c5990 0x7f005c1e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0060bc7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c156490 con 0x7f005c0c5990 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0060bc7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c155f70 con 0x7f005c06c730 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0060bc7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c156610 con 0x7f005c0ab170 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005c0c5990 0x7f005c1e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005c0c5990 0x7f005c1e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60918/0 (socket says 172.21.15.26:60918) 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 -- 172.21.15.26:0/3534415220 learned_addr learned my addr 172.21.15.26:0/3534415220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005a59c640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c06c730 0x7f005c0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0059d9b640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0ab170 0x7f005c1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 -- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0ab170 msgr2=0x7f005c1466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0ab170 0x7f005c1466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 -- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c06c730 msgr2=0x7f005c0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c06c730 0x7f005c0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 -- 172.21.15.26:0/3534415220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c1ebcc0 con 0x7f005c0c5990 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005a59c640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c06c730 0x7f005c0c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f005ad9d640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005c0c5990 0x7f005c1e87b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f004c00c810 tx=0x7f004c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0052ffd640 1 -- 172.21.15.26:0/3534415220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c018070 con 0x7f005c0c5990 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0052ffd640 1 -- 172.21.15.26:0/3534415220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c00a5e0 con 0x7f005c0c5990 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c1ebfa0 con 0x7f005c0c5990 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.963+0000 7f0052ffd640 1 -- 172.21.15.26:0/3534415220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00a9d0 con 0x7f005c0c5990 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.964+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c1ec560 con 0x7f005c0c5990 2024-03-06T07:23:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.964+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005c04f330 con 0x7f005c0c5990 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.965+0000 7f0052ffd640 1 -- 172.21.15.26:0/3534415220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f004c01b070 con 0x7f005c0c5990 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.965+0000 7f0052ffd640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f003c06b4d0 0x7f003c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.965+0000 7f0052ffd640 1 -- 172.21.15.26:0/3534415220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f004c089710 con 0x7f005c0c5990 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.965+0000 7f005a59c640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f003c06b4d0 0x7f003c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.966+0000 7f005a59c640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f003c06b4d0 0x7f003c06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0054009af0 tx=0x7f0054008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:15.969+0000 7f0052ffd640 1 -- 172.21.15.26:0/3534415220 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f004c058690 con 0x7f005c0c5990 2024-03-06T07:23:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.150+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7f005c0c28c0 con 0x7f005c0c5990 2024-03-06T07:23:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.153+0000 7f0052ffd640 1 -- 172.21.15.26:0/3534415220 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v29) v1 ==== 96+0+0 (secure 0 0 0) 0x7f004c057f90 con 0x7f005c0c5990 2024-03-06T07:23:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f003c06b4d0 msgr2=0x7f003c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f003c06b4d0 0x7f003c06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0054009af0 tx=0x7f0054008040 comp rx=0 tx=0).stop 2024-03-06T07:23:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005c0c5990 msgr2=0x7f005c1e87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005c0c5990 0x7f005c1e87b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f004c00c810 tx=0x7f004c00ccd0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 shutdown_connections 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f003c06b4d0 0x7f003c06d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005c0c5990 0x7f005c1e87b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f005c0ab170 0x7f005c1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 --2- 172.21.15.26:0/3534415220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c06c730 0x7f005c0c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.155+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 >> 172.21.15.26:0/3534415220 conn(0x7f005c0c6710 msgr2=0x7f005c0a8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.156+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 shutdown_connections 2024-03-06T07:23:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.156+0000 7f0060bc7640 1 -- 172.21.15.26:0/3534415220 wait complete. 2024-03-06T07:23:16.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.a mds_join_fs 2024-03-06T07:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.270+0000 7fdb3eda1640 1 Processor -- start 2024-03-06T07:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.270+0000 7fdb3eda1640 1 -- start start 2024-03-06T07:23:16.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3eda1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb380c4930 0x7fdb380c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3eda1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c7ff0 0x7fdb381587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3eda1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb38155300 con 0x7fdb380c4d10 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3eda1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb381555e0 con 0x7fdb380c4930 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3eda1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb381558c0 con 0x7fdb380c7ff0 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3cb3d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb380c4930 0x7fdb380c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3cb3d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb380c4930 0x7fdb380c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60920/0 (socket says 172.21.15.26:60920) 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/2749212572 learned_addr learned my addr 172.21.15.26:0/2749212572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3d33e640 1 --2- 172.21.15.26:0/2749212572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c7ff0 0x7fdb381587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/2749212572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c7ff0 msgr2=0x7fdb381587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3cb3d640 1 --2- 172.21.15.26:0/2749212572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c7ff0 0x7fdb381587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/2749212572 >> v1:172.21.15.117:6789/0 conn(0x7fdb380c4d10 legacy=0x7fdb380c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/2749212572 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38155ba0 con 0x7fdb380c4930 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.271+0000 7fdb3d33e640 1 --2- 172.21.15.26:0/2749212572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c7ff0 0x7fdb381587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.272+0000 7fdb3cb3d640 1 --2- 172.21.15.26:0/2749212572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb380c7ab0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb3002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f492f975e83a038 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.272+0000 7fdb377fe640 1 -- 172.21.15.26:0/2749212572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3002c9e0 con 0x7fdb380c4930 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.272+0000 7fdb377fe640 1 -- 172.21.15.26:0/2749212572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3002cb80 con 0x7fdb380c4930 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.272+0000 7fdb377fe640 1 -- 172.21.15.26:0/2749212572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300343f0 con 0x7fdb380c4930 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 -- 172.21.15.26:0/2749212572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 msgr2=0x7fdb380c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/2749212572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb380c7ab0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb3002be40 comp rx=0 tx=0).stop 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 -- 172.21.15.26:0/2749212572 shutdown_connections 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/2749212572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c7ff0 0x7fdb381587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/2749212572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb380c7ab0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 -- 172.21.15.26:0/2749212572 >> 172.21.15.26:0/2749212572 conn(0x7fdb380aa5c0 msgr2=0x7fdb380aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 -- 172.21.15.26:0/2749212572 shutdown_connections 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.273+0000 7fdb3eda1640 1 -- 172.21.15.26:0/2749212572 wait complete. 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.274+0000 7fdb3eda1640 1 Processor -- start 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.274+0000 7fdb3eda1640 1 -- start start 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.274+0000 7fdb3eda1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb38150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3eda1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c4d10 0x7fdb381ee080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3eda1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb380c7ff0 0x7fdb3814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3eda1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c2390 con 0x7fdb380c4930 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3eda1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb381560c0 con 0x7fdb380c7ff0 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3eda1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c2510 con 0x7fdb380c4d10 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb38150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb38150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60926/0 (socket says 172.21.15.26:60926) 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/1487439011 learned_addr learned my addr 172.21.15.26:0/1487439011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb37fff640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c4d10 0x7fdb381ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3d33e640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb380c7ff0 0x7fdb3814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c4d10 msgr2=0x7fdb381ee080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c4d10 0x7fdb381ee080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb380c7ff0 msgr2=0x7fdb3814fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb380c7ff0 0x7fdb3814fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 -- 172.21.15.26:0/1487439011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38151180 con 0x7fdb380c4930 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3cb3d640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb38150c40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb357fa640 1 -- 172.21.15.26:0/1487439011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30043020 con 0x7fdb380c4930 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb357fa640 1 -- 172.21.15.26:0/1487439011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3003e450 con 0x7fdb380c4930 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb38151400 con 0x7fdb380c4930 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb357fa640 1 -- 172.21.15.26:0/1487439011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3003e840 con 0x7fdb380c4930 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.275+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb381eecc0 con 0x7fdb380c4930 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.276+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38075300 con 0x7fdb380c4930 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.277+0000 7fdb357fa640 1 -- 172.21.15.26:0/1487439011 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb30046020 con 0x7fdb380c4930 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.277+0000 7fdb357fa640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb2406b500 0x7fdb2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.277+0000 7fdb357fa640 1 -- 172.21.15.26:0/1487439011 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fdb300ac830 con 0x7fdb380c4930 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.277+0000 7fdb37fff640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb2406b500 0x7fdb2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.278+0000 7fdb37fff640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb2406b500 0x7fdb2406d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806cbb0 tx=0x7fdb2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.282+0000 7fdb357fa640 1 -- 172.21.15.26:0/1487439011 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb38075300 con 0x7fdb380c4930 2024-03-06T07:23:16.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.461+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"} v 0) v1 -- 0x7fdb3804f330 con 0x7fdb380c4930 2024-03-06T07:23:16.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.464+0000 7fdb357fa640 1 -- 172.21.15.26:0/1487439011 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"}]=0 v30) v1 ==== 96+0+0 (secure 0 0 0) 0x7fdb3004d070 con 0x7fdb380c4930 2024-03-06T07:23:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb2406b500 msgr2=0x7fdb2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb2406b500 0x7fdb2406d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806cbb0 tx=0x7fdb2c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 msgr2=0x7fdb38150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb38150c40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb30004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 shutdown_connections 2024-03-06T07:23:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb2406b500 0x7fdb2406d9c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb380c7ff0 0x7fdb3814fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb380c4d10 0x7fdb381ee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 --2- 172.21.15.26:0/1487439011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb380c4930 0x7fdb38150c40 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.466+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 >> 172.21.15.26:0/1487439011 conn(0x7fdb380aa5c0 msgr2=0x7fdb380aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.467+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 shutdown_connections 2024-03-06T07:23:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.467+0000 7fdb3eda1640 1 -- 172.21.15.26:0/1487439011 wait complete. 2024-03-06T07:23:16.479 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused' 2024-03-06T07:23:16.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.578+0000 7f73f9b25640 1 Processor -- start 2024-03-06T07:23:16.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.578+0000 7f73f9b25640 1 -- start start 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f9b25640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73f40c6780 0x7f73f4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f9b25640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f73f41592d0 0x7f73f40c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f9b25640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f4072630 con 0x7f73f406da40 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f9b25640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f406de20 con 0x7f73f41592d0 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f9b25640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f4072910 con 0x7f73f40c6780 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f73f41592d0 0x7f73f40c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f73f406da40 0x7f73f4155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43824/0 (socket says 172.21.15.26:43824) 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f37fe640 1 -- 172.21.15.26:0/793697130 learned_addr learned my addr 172.21.15.26:0/793697130 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f2ffd640 1 --2- 172.21.15.26:0/793697130 >> v2:172.21.15.117:3300/0 conn(0x7f73f40c6780 0x7f73f4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f3fff640 1 -- 172.21.15.26:0/793697130 >> v2:172.21.15.117:3300/0 conn(0x7f73f40c6780 msgr2=0x7f73f4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f3fff640 1 --2- 172.21.15.26:0/793697130 >> v2:172.21.15.117:3300/0 conn(0x7f73f40c6780 0x7f73f4158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f3fff640 1 -- 172.21.15.26:0/793697130 >> v1:172.21.15.26:6789/0 conn(0x7f73f406da40 legacy=0x7f73f4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f27fc640 1 -- 172.21.15.26:0/793697130 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266247352 0 0) 0x7f73f4072630 con 0x7f73f406da40 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.579+0000 7f73f3fff640 1 -- 172.21.15.26:0/793697130 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f4072bf0 con 0x7f73f41592d0 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.580+0000 7f73f3fff640 1 --2- 172.21.15.26:0/793697130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f40c3280 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f73f40c7120 tx=0x7f73ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c580b6f9f4da2d58 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.580+0000 7f73f27fc640 1 -- 172.21.15.26:0/793697130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec039070 con 0x7f73f41592d0 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.580+0000 7f73f27fc640 1 -- 172.21.15.26:0/793697130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec02cbf0 con 0x7f73f41592d0 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.580+0000 7f73f27fc640 1 -- 172.21.15.26:0/793697130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec034440 con 0x7f73f41592d0 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 -- 172.21.15.26:0/793697130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 msgr2=0x7f73f40c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 --2- 172.21.15.26:0/793697130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f40c3280 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f73f40c7120 tx=0x7f73ec02be40 comp rx=0 tx=0).stop 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 -- 172.21.15.26:0/793697130 shutdown_connections 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 --2- 172.21.15.26:0/793697130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f40c3280 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 --2- 172.21.15.26:0/793697130 >> v2:172.21.15.117:3300/0 conn(0x7f73f40c6780 0x7f73f4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 -- 172.21.15.26:0/793697130 >> 172.21.15.26:0/793697130 conn(0x7f73f40aa630 msgr2=0x7f73f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 -- 172.21.15.26:0/793697130 shutdown_connections 2024-03-06T07:23:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.581+0000 7f73f9b25640 1 -- 172.21.15.26:0/793697130 wait complete. 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.582+0000 7f73f9b25640 1 Processor -- start 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.582+0000 7f73f9b25640 1 -- start start 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f9b25640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f406da40 0x7f73f406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f9b25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f40c6780 0x7f73f41eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f9b25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f41ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f9b25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40c3940 con 0x7f73f41592d0 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f9b25640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40c37c0 con 0x7f73f40c6780 2024-03-06T07:23:16.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f9b25640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40c3ac0 con 0x7f73f406da40 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f41ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f41ecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60948/0 (socket says 172.21.15.26:60948) 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 -- 172.21.15.26:0/226111224 learned_addr learned my addr 172.21.15.26:0/226111224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f2ffd640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f40c6780 0x7f73f41eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 -- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f406da40 msgr2=0x7f73f406f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f406da40 0x7f73f406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 -- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f40c6780 msgr2=0x7f73f41eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f40c6780 0x7f73f41eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 -- 172.21.15.26:0/226111224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f41570e0 con 0x7f73f41592d0 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.583+0000 7f73f3fff640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f41ecf60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009e80 tx=0x7f73ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.584+0000 7f73dbfff640 1 -- 172.21.15.26:0/226111224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec039050 con 0x7f73f41592d0 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.584+0000 7f73dbfff640 1 -- 172.21.15.26:0/226111224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec045070 con 0x7f73f41592d0 2024-03-06T07:23:16.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.584+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f41ebf40 con 0x7f73f41592d0 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.584+0000 7f73dbfff640 1 -- 172.21.15.26:0/226111224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec009450 con 0x7f73f41592d0 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.584+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f41573c0 con 0x7f73f41592d0 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.585+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f40c4bf0 con 0x7f73f41592d0 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.585+0000 7f73dbfff640 1 -- 172.21.15.26:0/226111224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73ec03e850 con 0x7f73f41592d0 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.586+0000 7f73dbfff640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f73e006b500 0x7f73e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.586+0000 7f73dbfff640 1 -- 172.21.15.26:0/226111224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f73ec0acea0 con 0x7f73f41592d0 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.586+0000 7f73f37fe640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f73e006b500 0x7f73e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.586+0000 7f73f37fe640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f73e006b500 0x7f73e006d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007fd0 tx=0x7f73e4007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.590+0000 7f73dbfff640 1 -- 172.21.15.26:0/226111224 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73f40c4bf0 con 0x7f73f41592d0 2024-03-06T07:23:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:16.839+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]} v 0) v1 -- 0x7f73f4073240 con 0x7f73f41592d0 2024-03-06T07:23:17.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.195+0000 7f73dbfff640 1 -- 172.21.15.26:0/226111224 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]}]=0 v200) v1 ==== 158+0+0 (secure 0 0 0) 0x7f73ec07be20 con 0x7f73f41592d0 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f73e006b500 msgr2=0x7f73e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f73e006b500 0x7f73e006d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007fd0 tx=0x7f73e4007960 comp rx=0 tx=0).stop 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 msgr2=0x7f73f41ecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f41ecf60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009e80 tx=0x7f73ec004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 shutdown_connections 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f73e006b500 0x7f73e006d9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f41592d0 0x7f73f41ecf60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f40c6780 0x7f73f41eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 --2- 172.21.15.26:0/226111224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f406da40 0x7f73f406f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 >> 172.21.15.26:0/226111224 conn(0x7f73f40aa630 msgr2=0x7f73f414e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 shutdown_connections 2024-03-06T07:23:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.197+0000 7f73f9b25640 1 -- 172.21.15.26:0/226111224 wait complete. 2024-03-06T07:23:17.211 INFO:tasks.cephfs_test_runner:That a standby with mds_join_fs set to another fs is not used if refuse_standby_for_another_fs is set. ... ok 2024-03-06T07:23:17.211 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_runtime (tasks.cephfs.test_failover.TestClusterAffinity) That setting mds_join_fs at runtime affects the cluster layout. 2024-03-06T07:23:17.211 INFO:tasks.cephfs_test_runner:test_join_fs_runtime (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:23:17.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.313+0000 7f954e1c7640 1 Processor -- start 2024-03-06T07:23:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.313+0000 7f954e1c7640 1 -- start start 2024-03-06T07:23:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f954e1c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9548072720 0x7f9548156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f954e1c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95481508c0 0x7f95480c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f954e1c7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9548157340 con 0x7f954806f7b0 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f954e1c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480c3740 con 0x7f95481508c0 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f954e1c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9548156d80 con 0x7f9548072720 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f9547fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95481508c0 0x7f95480c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f9547fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95481508c0 0x7f95480c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43064/0 (socket says 172.21.15.26:43064) 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f9547fff640 1 -- 172.21.15.26:0/138695639 learned_addr learned my addr 172.21.15.26:0/138695639 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f9547fff640 1 -- 172.21.15.26:0/138695639 >> v2:172.21.15.117:3300/0 conn(0x7f9548072720 msgr2=0x7f9548156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f9547fff640 1 --2- 172.21.15.26:0/138695639 >> v2:172.21.15.117:3300/0 conn(0x7f9548072720 0x7f9548156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f9547fff640 1 -- 172.21.15.26:0/138695639 >> v1:172.21.15.117:6789/0 conn(0x7f954806f7b0 legacy=0x7f95481501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.314+0000 7f9547fff640 1 -- 172.21.15.26:0/138695639 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548157060 con 0x7f95481508c0 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.315+0000 7f9547fff640 1 --2- 172.21.15.26:0/138695639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95481508c0 0x7f95480c3bd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9548071ec0 tx=0x7f954002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0709969df48e45 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.315+0000 7f95467fc640 1 -- 172.21.15.26:0/138695639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954002c9f0 con 0x7f95481508c0 2024-03-06T07:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.315+0000 7f95467fc640 1 -- 172.21.15.26:0/138695639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954002cb90 con 0x7f95481508c0 2024-03-06T07:23:17.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.315+0000 7f95467fc640 1 -- 172.21.15.26:0/138695639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95400343f0 con 0x7f95481508c0 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 -- 172.21.15.26:0/138695639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95481508c0 msgr2=0x7f95480c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 --2- 172.21.15.26:0/138695639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95481508c0 0x7f95480c3bd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9548071ec0 tx=0x7f954002be50 comp rx=0 tx=0).stop 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 -- 172.21.15.26:0/138695639 shutdown_connections 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 --2- 172.21.15.26:0/138695639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95481508c0 0x7f95480c3bd0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 --2- 172.21.15.26:0/138695639 >> v2:172.21.15.117:3300/0 conn(0x7f9548072720 0x7f9548156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 -- 172.21.15.26:0/138695639 >> 172.21.15.26:0/138695639 conn(0x7f95480aa630 msgr2=0x7f95480aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 -- 172.21.15.26:0/138695639 shutdown_connections 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.316+0000 7f954e1c7640 1 -- 172.21.15.26:0/138695639 wait complete. 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.317+0000 7f954e1c7640 1 Processor -- start 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.317+0000 7f954e1c7640 1 -- start start 2024-03-06T07:23:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f954806f7b0 0x7f95481dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9548072720 0x7f9548146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95481508c0 0x7f95481474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548156af0 con 0x7f954806f7b0 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548156970 con 0x7f9548072720 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9548072b00 con 0x7f95481508c0 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f954806f7b0 0x7f95481dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f954806f7b0 0x7f95481dfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43080/0 (socket says 172.21.15.26:43080) 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 -- 172.21.15.26:0/3135345636 learned_addr learned my addr 172.21.15.26:0/3135345636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f9547fff640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95481508c0 0x7f95481474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f9546ffd640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9548072720 0x7f9548146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 -- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95481508c0 msgr2=0x7f95481474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95481508c0 0x7f95481474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 -- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9548072720 msgr2=0x7f9548146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9548072720 0x7f9548146f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 -- 172.21.15.26:0/3135345636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548149e50 con 0x7f954806f7b0 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f95477fe640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f954806f7b0 0x7f95481dfa40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f953800b4a0 tx=0x7f953800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f952ffff640 1 -- 172.21.15.26:0/3135345636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538014070 con 0x7f954806f7b0 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f952ffff640 1 -- 172.21.15.26:0/3135345636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538010610 con 0x7f954806f7b0 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f952ffff640 1 -- 172.21.15.26:0/3135345636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538010a00 con 0x7f954806f7b0 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9548147e00 con 0x7f954806f7b0 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.318+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95480ae7a0 con 0x7f954806f7b0 2024-03-06T07:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.319+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954804f330 con 0x7f954806f7b0 2024-03-06T07:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.320+0000 7f952ffff640 1 -- 172.21.15.26:0/3135345636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9538017070 con 0x7f954806f7b0 2024-03-06T07:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.320+0000 7f952ffff640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f953406b500 0x7f953406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.320+0000 7f952ffff640 1 -- 172.21.15.26:0/3135345636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f9538089ae0 con 0x7f954806f7b0 2024-03-06T07:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.320+0000 7f9546ffd640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f953406b500 0x7f953406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.321+0000 7f9546ffd640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f953406b500 0x7f953406d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f953c006fd0 tx=0x7f953c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.324+0000 7f952ffff640 1 -- 172.21.15.26:0/3135345636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9538058a60 con 0x7f954806f7b0 2024-03-06T07:23:17.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.508+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95480c2d30 con 0x7f954806f7b0 2024-03-06T07:23:17.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.508+0000 7f952ffff640 1 -- 172.21.15.26:0/3135345636 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13696 (secure 0 0 0) 0x7f9538058360 con 0x7f954806f7b0 2024-03-06T07:23:17.510 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:17.510 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:15.191143+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6835/378552977":"2024-03-07T07:23:12.186843+0000","172.21.15.26:6834/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.26:6835/3035454765":"2024-03-07T07:23:06.830723+0000","172.21.15.117:6836/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6834/252560219":"2024-03-07T07:23:05.798750+0000","172.21.15.117:6837/378552977":"2024-03-07T07:23:12.186843+0000","172.21.15.117:6836/218606650":"2024-03-07T07:22:54.778025+0000","172.21.15.117:6834/218606650":"2024-03-07T07:22:54.778025+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.510+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f953406b500 msgr2=0x7f953406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f953406b500 0x7f953406d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f953c006fd0 tx=0x7f953c008040 comp rx=0 tx=0).stop 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f954806f7b0 msgr2=0x7f95481dfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f954806f7b0 0x7f95481dfa40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f953800b4a0 tx=0x7f953800b960 comp rx=0 tx=0).stop 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 shutdown_connections 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f953406b500 0x7f953406d9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95481508c0 0x7f95481474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9548072720 0x7f9548146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 --2- 172.21.15.26:0/3135345636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f954806f7b0 0x7f95481dfa40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 >> 172.21.15.26:0/3135345636 conn(0x7f95480aa630 msgr2=0x7f954814d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 shutdown_connections 2024-03-06T07:23:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.511+0000 7f954e1c7640 1 -- 172.21.15.26:0/3135345636 wait complete. 2024-03-06T07:23:17.524 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:17.524 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.627+0000 7fbe06ac3640 1 Processor -- start 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.627+0000 7fbe06ac3640 1 -- start start 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.627+0000 7fbe06ac3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 0x7fbe0006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.627+0000 7fbe06ac3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe000acbc0 0x7fbe000ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.627+0000 7fbe06ac3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe000c1c90 con 0x7fbe000abe90 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.627+0000 7fbe06ac3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe000ac750 con 0x7fbe000acbc0 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.627+0000 7fbe06ac3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000c1a40 con 0x7fbe0014d350 2024-03-06T07:23:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbe05060640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbe000abe90 0x7fbe00156540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34734/0 (socket says 172.21.15.26:34734) 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbe05060640 1 -- 172.21.15.26:0/1048717937 learned_addr learned my addr 172.21.15.26:0/1048717937 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdfffff640 1 --2- 172.21.15.26:0/1048717937 >> v2:172.21.15.117:3300/0 conn(0x7fbe000acbc0 0x7fbe000ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742166734 0 0) 0x7fbe000c1c90 con 0x7fbe000abe90 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbe0485f640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 0x7fbe0006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdec003520 con 0x7fbe000abe90 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252702543 0 0) 0x7fbdec003520 con 0x7fbe000abe90 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 msgr2=0x7fbe0006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 0x7fbe0006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 >> v2:172.21.15.117:3300/0 conn(0x7fbe000acbc0 msgr2=0x7fbe000ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 --2- 172.21.15.26:0/1048717937 >> v2:172.21.15.117:3300/0 conn(0x7fbe000acbc0 0x7fbe000ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000ac470 con 0x7fbe000abe90 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fbde8001670 con 0x7fbe000abe90 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 >> v1:172.21.15.26:6789/0 conn(0x7fbe000abe90 legacy=0x7fbe00156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0039b0 0x7fbdec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdec004800 0x7fbdec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 0x7fbdec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000c1a40 con 0x7fbdec004800 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00b9c0 con 0x7fbdec0039b0 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00bb40 con 0x7fbe0014d350 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbdfffff640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdec004800 0x7fbdec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbe0485f640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0039b0 0x7fbdec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.628+0000 7fbe05060640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 0x7fbdec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbdfffff640 1 -- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 msgr2=0x7fbdec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbdfffff640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 0x7fbdec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbdfffff640 1 -- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0039b0 msgr2=0x7fbdec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbdfffff640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0039b0 0x7fbdec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbdfffff640 1 -- 172.21.15.26:0/1048717937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec003520 con 0x7fbdec004800 2024-03-06T07:23:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbe05060640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014d350 0x7fbdec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbdfffff640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdec004800 0x7fbdec004c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009af0 tx=0x7fbdec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759c7d10933d5927 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.629+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0039070 con 0x7fbdec004800 2024-03-06T07:23:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf002c710 con 0x7fbdec004800 2024-03-06T07:23:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbdff7fe640 1 -- 172.21.15.26:0/1048717937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0009690 con 0x7fbdec004800 2024-03-06T07:23:17.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbe06ac3640 1 -- 172.21.15.26:0/1048717937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdec004800 msgr2=0x7fbdec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbe06ac3640 1 --2- 172.21.15.26:0/1048717937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdec004800 0x7fbdec004c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009af0 tx=0x7fbdec005830 comp rx=0 tx=0).stop 2024-03-06T07:23:17.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbe05060640 1 -- 172.21.15.26:0/1048717937 reap_dead start 2024-03-06T07:23:17.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbe06ac3640 1 -- 172.21.15.26:0/1048717937 shutdown_connections 2024-03-06T07:23:17.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbe06ac3640 1 -- 172.21.15.26:0/1048717937 >> 172.21.15.26:0/1048717937 conn(0x7fbe000c3b40 msgr2=0x7fbe000c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbe06ac3640 1 -- 172.21.15.26:0/1048717937 shutdown_connections 2024-03-06T07:23:17.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.630+0000 7fbe06ac3640 1 -- 172.21.15.26:0/1048717937 wait complete. 2024-03-06T07:23:17.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.631+0000 7fbe06ac3640 1 Processor -- start 2024-03-06T07:23:17.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.631+0000 7fbe06ac3640 1 -- start start 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe06ac3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec004800 0x7fbe000c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe06ac3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014c810 0x7fbe000c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe06ac3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe000c5d20 0x7fbe00151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe06ac3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000aa160 con 0x7fbe000c5d20 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe06ac3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00153c00 con 0x7fbdec004800 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe06ac3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000aa2e0 con 0x7fbe0014c810 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe000c5d20 0x7fbe00151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe000c5d20 0x7fbe00151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43096/0 (socket says 172.21.15.26:43096) 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 -- 172.21.15.26:0/4129590100 learned_addr learned my addr 172.21.15.26:0/4129590100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe0485f640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec004800 0x7fbe000c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbdfffff640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014c810 0x7fbe000c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 -- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014c810 msgr2=0x7fbe000c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014c810 0x7fbe000c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 -- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec004800 msgr2=0x7fbe000c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec004800 0x7fbe000c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.632+0000 7fbe05060640 1 -- 172.21.15.26:0/4129590100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe001ef8c0 con 0x7fbe000c5d20 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.633+0000 7fbe05060640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe000c5d20 0x7fbe00151e10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbe000c6f70 tx=0x7fbde800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.633+0000 7fbdfd7fa640 1 -- 172.21.15.26:0/4129590100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8019070 con 0x7fbe000c5d20 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.633+0000 7fbdfd7fa640 1 -- 172.21.15.26:0/4129590100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde800c610 con 0x7fbe000c5d20 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.633+0000 7fbdfd7fa640 1 -- 172.21.15.26:0/4129590100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800c9c0 con 0x7fbe000c5d20 2024-03-06T07:23:17.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.633+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe000c5300 con 0x7fbe000c5d20 2024-03-06T07:23:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.633+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe001ef5e0 con 0x7fbe000c5d20 2024-03-06T07:23:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.634+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0014be30 con 0x7fbe000c5d20 2024-03-06T07:23:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.634+0000 7fbdfd7fa640 1 -- 172.21.15.26:0/4129590100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbde801c070 con 0x7fbe000c5d20 2024-03-06T07:23:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.635+0000 7fbdfd7fa640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbde006b4d0 0x7fbde006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.635+0000 7fbdfd7fa640 1 -- 172.21.15.26:0/4129590100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fbde8089660 con 0x7fbe000c5d20 2024-03-06T07:23:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.635+0000 7fbe0485f640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbde006b4d0 0x7fbde006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.636+0000 7fbe0485f640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbde006b4d0 0x7fbde006d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8014430 tx=0x7fbdf800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.639+0000 7fbdfd7fa640 1 -- 172.21.15.26:0/4129590100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbde8057e60 con 0x7fbe000c5d20 2024-03-06T07:23:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.817+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe0004f330 con 0x7fbe000c5d20 2024-03-06T07:23:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.817+0000 7fbdfd7fa640 1 -- 172.21.15.26:0/4129590100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbde805ce20 con 0x7fbe000c5d20 2024-03-06T07:23:17.819 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:23:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbde006b4d0 msgr2=0x7fbde006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbde006b4d0 0x7fbde006d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8014430 tx=0x7fbdf800b040 comp rx=0 tx=0).stop 2024-03-06T07:23:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe000c5d20 msgr2=0x7fbe00151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe000c5d20 0x7fbe00151e10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbe000c6f70 tx=0x7fbde800eb90 comp rx=0 tx=0).stop 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 shutdown_connections 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbde006b4d0 0x7fbde006d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe000c5d20 0x7fbe00151e10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe0014c810 0x7fbe000c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 --2- 172.21.15.26:0/4129590100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec004800 0x7fbe000c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 >> 172.21.15.26:0/4129590100 conn(0x7fbe000c3b40 msgr2=0x7fbe0006b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.819+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 shutdown_connections 2024-03-06T07:23:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.820+0000 7fbe06ac3640 1 -- 172.21.15.26:0/4129590100 wait complete. 2024-03-06T07:23:17.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime' 2024-03-06T07:23:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.933+0000 7f0bf1fac640 1 Processor -- start 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.933+0000 7f0bf1fac640 1 -- start start 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bf1fac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0bec159270 0x7f0bec155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bf1fac640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bec0c73d0 con 0x7f0bec0c88c0 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bf1fac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bec0be2e0 con 0x7f0bec14f360 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bf1fac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec159700 con 0x7f0bec159270 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0beaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0bec0c88c0 0x7f0bec158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34740/0 (socket says 172.21.15.26:34740) 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0beaffd640 1 -- 172.21.15.26:0/3642686611 learned_addr learned my addr 172.21.15.26:0/3642686611 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bebfff640 1 --2- 172.21.15.26:0/3642686611 >> v2:172.21.15.117:3300/0 conn(0x7f0bec159270 0x7f0bec155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bea7fc640 1 -- 172.21.15.26:0/3642686611 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280599636 0 0) 0x7f0bec0c73d0 con 0x7f0bec0c88c0 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bea7fc640 1 -- 172.21.15.26:0/3642686611 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd4003520 con 0x7f0bec0c88c0 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bebfff640 1 -- 172.21.15.26:0/3642686611 >> v1:172.21.15.117:6789/0 conn(0x7f0bec14f360 legacy=0x7f0bec0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bea7fc640 1 -- 172.21.15.26:0/3642686611 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4266359501 0 0) 0x7f0bd4003520 con 0x7f0bec0c88c0 2024-03-06T07:23:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bebfff640 1 -- 172.21.15.26:0/3642686611 >> v1:172.21.15.26:6789/0 conn(0x7f0bec0c88c0 legacy=0x7f0bec158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.934+0000 7f0bebfff640 1 -- 172.21.15.26:0/3642686611 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0c7670 con 0x7f0bec159270 2024-03-06T07:23:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.935+0000 7f0bebfff640 1 --2- 172.21.15.26:0/3642686611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec159270 0x7f0bec155c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009d90 tx=0x7f0be402bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3de8a775f31f2bc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.935+0000 7f0bea7fc640 1 -- 172.21.15.26:0/3642686611 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0001670 con 0x7f0bec159270 2024-03-06T07:23:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.935+0000 7f0bea7fc640 1 -- 172.21.15.26:0/3642686611 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be402d720 con 0x7f0bec159270 2024-03-06T07:23:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.935+0000 7f0bea7fc640 1 -- 172.21.15.26:0/3642686611 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be402daf0 con 0x7f0bec159270 2024-03-06T07:23:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.936+0000 7f0bf1fac640 1 -- 172.21.15.26:0/3642686611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec159270 msgr2=0x7f0bec155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.936+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/3642686611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec159270 0x7f0bec155c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009d90 tx=0x7f0be402bbe0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.936+0000 7f0bf1fac640 1 -- 172.21.15.26:0/3642686611 shutdown_connections 2024-03-06T07:23:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.936+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/3642686611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec159270 0x7f0bec155c10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.936+0000 7f0bf1fac640 1 -- 172.21.15.26:0/3642686611 >> 172.21.15.26:0/3642686611 conn(0x7f0bec0ac570 msgr2=0x7f0bec0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.936+0000 7f0bf1fac640 1 -- 172.21.15.26:0/3642686611 shutdown_connections 2024-03-06T07:23:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.936+0000 7f0bf1fac640 1 -- 172.21.15.26:0/3642686611 wait complete. 2024-03-06T07:23:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.937+0000 7f0bf1fac640 1 Processor -- start 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.937+0000 7f0bf1fac640 1 -- start start 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec0c88c0 0x7f0bec0bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec14f360 0x7f0bec153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bec159270 0x7f0bec1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0c7c30 con 0x7f0bec0c88c0 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0c5a50 con 0x7f0bec14f360 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0c5bd0 con 0x7f0bec159270 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec0c88c0 0x7f0bec0bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec0c88c0 0x7f0bec0bedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43112/0 (socket says 172.21.15.26:43112) 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 -- 172.21.15.26:0/580901874 learned_addr learned my addr 172.21.15.26:0/580901874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 -- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bec159270 msgr2=0x7f0bec1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bec159270 0x7f0bec1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 -- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec14f360 msgr2=0x7f0bec153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec14f360 0x7f0bec153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 -- 172.21.15.26:0/580901874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec1531a0 con 0x7f0bec0c88c0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0beb7fe640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec0c88c0 0x7f0bec0bedc0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00c7b0 tx=0x7f0bdc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bd3fff640 1 -- 172.21.15.26:0/580901874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc017070 con 0x7f0bec0c88c0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bd3fff640 1 -- 172.21.15.26:0/580901874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc005290 con 0x7f0bec0c88c0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bd3fff640 1 -- 172.21.15.26:0/580901874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00a880 con 0x7f0bec0c88c0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec153480 con 0x7f0bec0c88c0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.938+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec153a40 con 0x7f0bec0c88c0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.939+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec153c40 con 0x7f0bec0c88c0 2024-03-06T07:23:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.940+0000 7f0bd3fff640 1 -- 172.21.15.26:0/580901874 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bdc01a070 con 0x7f0bec0c88c0 2024-03-06T07:23:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.940+0000 7f0bd3fff640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bd806b4d0 0x7f0bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.940+0000 7f0bd3fff640 1 -- 172.21.15.26:0/580901874 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f0bdc0897d0 con 0x7f0bec0c88c0 2024-03-06T07:23:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.940+0000 7f0beaffd640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bd806b4d0 0x7f0bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.941+0000 7f0beaffd640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bd806b4d0 0x7f0bd806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0be0001eb0 tx=0x7f0be0001a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:17.945+0000 7f0bd3fff640 1 -- 172.21.15.26:0/580901874 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0bec153c40 con 0x7f0bec0c88c0 2024-03-06T07:23:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:18.196+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]} v 0) v1 -- 0x7f0bec0c8d50 con 0x7f0bec0c88c0 2024-03-06T07:23:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.201+0000 7f0bd3fff640 1 -- 172.21.15.26:0/580901874 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]}]=0 v202) v1 ==== 149+0+0 (secure 0 0 0) 0x7f0bdc058590 con 0x7f0bec0c88c0 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bd806b4d0 msgr2=0x7f0bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bd806b4d0 0x7f0bd806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0be0001eb0 tx=0x7f0be0001a40 comp rx=0 tx=0).stop 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec0c88c0 msgr2=0x7f0bec0bedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec0c88c0 0x7f0bec0bedc0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00c7b0 tx=0x7f0bdc00cc70 comp rx=0 tx=0).stop 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 shutdown_connections 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bd806b4d0 0x7f0bd806d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bec159270 0x7f0bec1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bec14f360 0x7f0bec153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 --2- 172.21.15.26:0/580901874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec0c88c0 0x7f0bec0bedc0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 >> 172.21.15.26:0/580901874 conn(0x7f0bec0ac570 msgr2=0x7f0bec0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 shutdown_connections 2024-03-06T07:23:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.204+0000 7f0bf1fac640 1 -- 172.21.15.26:0/580901874 wait complete. 2024-03-06T07:23:19.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:23:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a9393640 1 Processor -- start 2024-03-06T07:23:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a9393640 1 -- start start 2024-03-06T07:23:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a9393640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb3a40c5990 0x7fb3a40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a9393640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3a40ab170 0x7fb3a406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a9393640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a406cba0 con 0x7fb3a406c730 2024-03-06T07:23:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a9393640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a40c3ef0 con 0x7fb3a40ab170 2024-03-06T07:23:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a9393640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a41556d0 con 0x7fb3a40c5990 2024-03-06T07:23:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.329+0000 7fb3a27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3a40ab170 0x7fb3a406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3a40ab170 0x7fb3a406c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43124/0 (socket says 172.21.15.26:43124) 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a27fc640 1 -- 172.21.15.26:0/4160179937 learned_addr learned my addr 172.21.15.26:0/4160179937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a2ffd640 1 --2- 172.21.15.26:0/4160179937 >> v2:172.21.15.117:3300/0 conn(0x7fb3a40c5990 0x7fb3a40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a27fc640 1 -- 172.21.15.26:0/4160179937 >> v2:172.21.15.117:3300/0 conn(0x7fb3a40c5990 msgr2=0x7fb3a40c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a27fc640 1 --2- 172.21.15.26:0/4160179937 >> v2:172.21.15.117:3300/0 conn(0x7fb3a40c5990 0x7fb3a40c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a27fc640 1 -- 172.21.15.26:0/4160179937 >> v1:172.21.15.117:6789/0 conn(0x7fb3a406c730 legacy=0x7fb3a41523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a27fc640 1 -- 172.21.15.26:0/4160179937 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a41559b0 con 0x7fb3a40ab170 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.330+0000 7fb3a2ffd640 1 --2- 172.21.15.26:0/4160179937 >> v2:172.21.15.117:3300/0 conn(0x7fb3a40c5990 0x7fb3a40c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a27fc640 1 --2- 172.21.15.26:0/4160179937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a40ab170 0x7fb3a406c1f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40c12a0 tx=0x7fb39802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f7ccaa095de550 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a1ffb640 1 -- 172.21.15.26:0/4160179937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398035020 con 0x7fb3a40ab170 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a1ffb640 1 -- 172.21.15.26:0/4160179937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398031870 con 0x7fb3a40ab170 2024-03-06T07:23:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a1ffb640 1 -- 172.21.15.26:0/4160179937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398031c40 con 0x7fb3a40ab170 2024-03-06T07:23:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 -- 172.21.15.26:0/4160179937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a40ab170 msgr2=0x7fb3a406c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 --2- 172.21.15.26:0/4160179937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a40ab170 0x7fb3a406c1f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40c12a0 tx=0x7fb39802b910 comp rx=0 tx=0).stop 2024-03-06T07:23:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 -- 172.21.15.26:0/4160179937 shutdown_connections 2024-03-06T07:23:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 --2- 172.21.15.26:0/4160179937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a40ab170 0x7fb3a406c1f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 --2- 172.21.15.26:0/4160179937 >> v2:172.21.15.117:3300/0 conn(0x7fb3a40c5990 0x7fb3a40c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 -- 172.21.15.26:0/4160179937 >> 172.21.15.26:0/4160179937 conn(0x7fb3a40c6710 msgr2=0x7fb3a40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 -- 172.21.15.26:0/4160179937 shutdown_connections 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.331+0000 7fb3a9393640 1 -- 172.21.15.26:0/4160179937 wait complete. 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 Processor -- start 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 -- start start 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a406c730 0x7fb3a40a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3a40ab170 0x7fb3a4154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3a40c5990 0x7fb3a4071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4156490 con 0x7fb3a406c730 2024-03-06T07:23:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4155f70 con 0x7fb3a40ab170 2024-03-06T07:23:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a9393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4156610 con 0x7fb3a40c5990 2024-03-06T07:23:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.333+0000 7fb3a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a406c730 0x7fb3a40a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a406c730 0x7fb3a40a75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43132/0 (socket says 172.21.15.26:43132) 2024-03-06T07:23:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 -- 172.21.15.26:0/2237577561 learned_addr learned my addr 172.21.15.26:0/2237577561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a27fc640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3a40ab170 0x7fb3a4154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a37fe640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3a40c5990 0x7fb3a4071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 -- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3a40c5990 msgr2=0x7fb3a4071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3a40c5990 0x7fb3a4071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 -- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3a40ab170 msgr2=0x7fb3a4154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3a40ab170 0x7fb3a4154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 -- 172.21.15.26:0/2237577561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a414cf10 con 0x7fb3a406c730 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a37fe640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3a40c5990 0x7fb3a4071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a2ffd640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a406c730 0x7fb3a40a75f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb39400aed0 tx=0x7fb39400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb38b7fe640 1 -- 172.21.15.26:0/2237577561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394007820 con 0x7fb3a406c730 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb38b7fe640 1 -- 172.21.15.26:0/2237577561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3940079c0 con 0x7fb3a406c730 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a414a380 con 0x7fb3a406c730 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb38b7fe640 1 -- 172.21.15.26:0/2237577561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394007d70 con 0x7fb3a406c730 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.334+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a4070f00 con 0x7fb3a406c730 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.335+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a404f330 con 0x7fb3a406c730 2024-03-06T07:23:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.336+0000 7fb38b7fe640 1 -- 172.21.15.26:0/2237577561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb394010650 con 0x7fb3a406c730 2024-03-06T07:23:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.336+0000 7fb38b7fe640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b4d0 0x7fb39006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.336+0000 7fb38b7fe640 1 -- 172.21.15.26:0/2237577561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fb39408c320 con 0x7fb3a406c730 2024-03-06T07:23:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.336+0000 7fb3a27fc640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b4d0 0x7fb39006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.337+0000 7fb3a27fc640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b4d0 0x7fb39006d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb398036000 tx=0x7fb39803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.340+0000 7fb38b7fe640 1 -- 172.21.15.26:0/2237577561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb39405b2a0 con 0x7fb3a406c730 2024-03-06T07:23:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.522+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb3a41483e0 con 0x7fb3a406c730 2024-03-06T07:23:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.525+0000 7fb38b7fe640 1 -- 172.21.15.26:0/2237577561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb39405aba0 con 0x7fb3a406c730 2024-03-06T07:23:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.527+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b4d0 msgr2=0x7fb39006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.527+0000 7fb3a9393640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b4d0 0x7fb39006d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb398036000 tx=0x7fb39803c040 comp rx=0 tx=0).stop 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a406c730 msgr2=0x7fb3a40a75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a406c730 0x7fb3a40a75f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb39400aed0 tx=0x7fb39400cb90 comp rx=0 tx=0).stop 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 shutdown_connections 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b4d0 0x7fb39006d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3a40c5990 0x7fb3a4071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3a40ab170 0x7fb3a4154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 --2- 172.21.15.26:0/2237577561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3a406c730 0x7fb3a40a75f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 >> 172.21.15.26:0/2237577561 conn(0x7fb3a40c6710 msgr2=0x7fb3a40c31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 shutdown_connections 2024-03-06T07:23:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.528+0000 7fb3a9393640 1 -- 172.21.15.26:0/2237577561 wait complete. 2024-03-06T07:23:19.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:19.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:19.583 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:19.625 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:19.626 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:23:19.626 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:19.641 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:19.641 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:23:19.642 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:19.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:19.684 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:23:19.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.783+0000 7f125e1e9640 1 Processor -- start 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f125e1e9640 1 -- start start 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f125e1e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f125814f360 0x7f1258155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f125e1e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12580c88c0 0x7f12580c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f125e1e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12580c73d0 con 0x7f12580c5880 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f125e1e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12580c8dd0 con 0x7f12580c88c0 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f125e1e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12580be2e0 con 0x7f125814f360 2024-03-06T07:23:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f1256ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12580c88c0 0x7f12580c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f1257fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f12580c5880 0x7f1258158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34742/0 (socket says 172.21.15.26:34742) 2024-03-06T07:23:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.784+0000 7f1257fff640 1 -- 172.21.15.26:0/1883661407 learned_addr learned my addr 172.21.15.26:0/1883661407 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.785+0000 7f12577fe640 1 --2- 172.21.15.26:0/1883661407 >> v2:172.21.15.117:3300/0 conn(0x7f125814f360 0x7f1258155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.785+0000 7f1256ffd640 1 -- 172.21.15.26:0/1883661407 >> v2:172.21.15.117:3300/0 conn(0x7f125814f360 msgr2=0x7f1258155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.785+0000 7f1256ffd640 1 --2- 172.21.15.26:0/1883661407 >> v2:172.21.15.117:3300/0 conn(0x7f125814f360 0x7f1258155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.785+0000 7f1256ffd640 1 -- 172.21.15.26:0/1883661407 >> v1:172.21.15.26:6789/0 conn(0x7f12580c5880 legacy=0x7f1258158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.785+0000 7f1256ffd640 1 -- 172.21.15.26:0/1883661407 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580c7670 con 0x7f12580c88c0 2024-03-06T07:23:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.785+0000 7f1256ffd640 1 --2- 172.21.15.26:0/1883661407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c88c0 0x7f12580c5340 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f125808c840 tx=0x7f124c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7938d871a132ab8b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.785+0000 7f12567fc640 1 -- 172.21.15.26:0/1883661407 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c0315a0 con 0x7f12580c88c0 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f12567fc640 1 -- 172.21.15.26:0/1883661407 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c037070 con 0x7f12580c88c0 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f12567fc640 1 -- 172.21.15.26:0/1883661407 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c031960 con 0x7f12580c88c0 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 -- 172.21.15.26:0/1883661407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c88c0 msgr2=0x7f12580c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1883661407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c88c0 0x7f12580c5340 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f125808c840 tx=0x7f124c02b840 comp rx=0 tx=0).stop 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 -- 172.21.15.26:0/1883661407 shutdown_connections 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1883661407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c88c0 0x7f12580c5340 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1883661407 >> v2:172.21.15.117:3300/0 conn(0x7f125814f360 0x7f1258155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 -- 172.21.15.26:0/1883661407 >> 172.21.15.26:0/1883661407 conn(0x7f12580ac570 msgr2=0x7f12580ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 -- 172.21.15.26:0/1883661407 shutdown_connections 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.786+0000 7f125e1e9640 1 -- 172.21.15.26:0/1883661407 wait complete. 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.787+0000 7f125e1e9640 1 Processor -- start 2024-03-06T07:23:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.787+0000 7f125e1e9640 1 -- start start 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f125e1e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c5880 0x7f125814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f125e1e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12580c88c0 0x7f125814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f125e1e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125814f360 0x7f12581ee070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f125e1e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1258159270 con 0x7f12580c5880 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f125e1e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12580c7c30 con 0x7f125814f360 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f125e1e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12581593f0 con 0x7f12580c88c0 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c5880 0x7f125814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c5880 0x7f125814b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43144/0 (socket says 172.21.15.26:43144) 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 -- 172.21.15.26:0/1785362020 learned_addr learned my addr 172.21.15.26:0/1785362020 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f1257fff640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125814f360 0x7f12581ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f1256ffd640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12580c88c0 0x7f125814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 -- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12580c88c0 msgr2=0x7f125814bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12580c88c0 0x7f125814bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 -- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125814f360 msgr2=0x7f12581ee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125814f360 0x7f12581ee070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f12577fe640 1 -- 172.21.15.26:0/1785362020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580c1470 con 0x7f12580c5880 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.788+0000 7f1257fff640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125814f360 0x7f12581ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.789+0000 7f12577fe640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c5880 0x7f125814b790 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f124800d5f0 tx=0x7f124800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.789+0000 7f123ffff640 1 -- 172.21.15.26:0/1785362020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12480125f0 con 0x7f12580c5880 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.789+0000 7f123ffff640 1 -- 172.21.15.26:0/1785362020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1248019070 con 0x7f12580c5880 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.789+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12580af750 con 0x7f12580c5880 2024-03-06T07:23:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.789+0000 7f123ffff640 1 -- 172.21.15.26:0/1785362020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12480129d0 con 0x7f12580c5880 2024-03-06T07:23:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.789+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12581eecd0 con 0x7f12580c5880 2024-03-06T07:23:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.790+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125804f330 con 0x7f12580c5880 2024-03-06T07:23:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.790+0000 7f123ffff640 1 -- 172.21.15.26:0/1785362020 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1248012b70 con 0x7f12580c5880 2024-03-06T07:23:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.790+0000 7f123ffff640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f124406b4d0 0x7f124406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.791+0000 7f123ffff640 1 -- 172.21.15.26:0/1785362020 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f124808a510 con 0x7f12580c5880 2024-03-06T07:23:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.791+0000 7f1256ffd640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f124406b4d0 0x7f124406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.791+0000 7f1256ffd640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f124406b4d0 0x7f124406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f124c036000 tx=0x7f124c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:19.795+0000 7f123ffff640 1 -- 172.21.15.26:0/1785362020 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1248059490 con 0x7f12580c5880 2024-03-06T07:23:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.013+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f125806cda0 con 0x7f12580c5880 2024-03-06T07:23:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.014+0000 7f123ffff640 1 -- 172.21.15.26:0/1785362020 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f1248058d90 con 0x7f12580c5880 2024-03-06T07:23:20.016 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:20.016 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1861806886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1861806886},{"type":"v1","addr":"172.21.15.117:6837","nonce":1861806886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6324,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.26:6833/3180689366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3180689366},{"type":"v1","addr":"172.21.15.26:6833","nonce":3180689366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1880409927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1880409927},{"type":"v1","addr":"172.21.15.117:6836","nonce":1880409927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/174449231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":174449231},{"type":"v1","addr":"172.21.15.26:6835","nonce":174449231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[]} 2024-03-06T07:23:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2024-03-06T07:23:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f124406b4d0 msgr2=0x7f124406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f124406b4d0 0x7f124406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f124c036000 tx=0x7f124c008030 comp rx=0 tx=0).stop 2024-03-06T07:23:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c5880 msgr2=0x7f125814b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c5880 0x7f125814b790 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f124800d5f0 tx=0x7f124800dab0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 shutdown_connections 2024-03-06T07:23:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f124406b4d0 0x7f124406d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125814f360 0x7f12581ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12580c88c0 0x7f125814bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 --2- 172.21.15.26:0/1785362020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12580c5880 0x7f125814b790 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 >> 172.21.15.26:0/1785362020 conn(0x7f12580ac570 msgr2=0x7f12580c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 shutdown_connections 2024-03-06T07:23:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.016+0000 7f125e1e9640 1 -- 172.21.15.26:0/1785362020 wait complete. 2024-03-06T07:23:20.029 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:23:20.029 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:23:20.030 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:23:20.030 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.030 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:23:20.030 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:23:20.030 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:23:20.030 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.031 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:23:20.031 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:23:20.031 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:23:20.031 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.031 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:23:20.031 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:23:20.031 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:23:20.031 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.039 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:23:20.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:23:20.040 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:23:20.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:23:20.042 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:23:20.042 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:23:20.043 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:23:20.043 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:23:20.082 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:23:20.082 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.083 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:23:20.083 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.085 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:23:20.086 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.086 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:23:20.086 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:23:20.158 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:23:20.158 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:23:20.168 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:23:20.168 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:23:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.192+0000 7fa450c8d640 1 Processor -- start 2024-03-06T07:23:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.192+0000 7fa450c8d640 1 -- start start 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa450c8d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa44c06ff10 0x7fa44c153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa450c8d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa44c0c8f20 0x7fa44c154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa450c8d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0a8390 con 0x7fa44c0c7170 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa450c8d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c0c7620 con 0x7fa44c0c8f20 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa450c8d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c0730f0 con 0x7fa44c06ff10 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa44c0c8f20 0x7fa44c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa44c0c8f20 0x7fa44c154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43170/0 (socket says 172.21.15.26:43170) 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44ad9d640 1 -- 172.21.15.26:0/4099436028 learned_addr learned my addr 172.21.15.26:0/4099436028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44a59c640 1 --2- 172.21.15.26:0/4099436028 >> v2:172.21.15.117:3300/0 conn(0x7fa44c06ff10 0x7fa44c153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44ad9d640 1 -- 172.21.15.26:0/4099436028 >> v2:172.21.15.117:3300/0 conn(0x7fa44c06ff10 msgr2=0x7fa44c153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44ad9d640 1 --2- 172.21.15.26:0/4099436028 >> v2:172.21.15.117:3300/0 conn(0x7fa44c06ff10 0x7fa44c153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44ad9d640 1 -- 172.21.15.26:0/4099436028 >> v1:172.21.15.117:6789/0 conn(0x7fa44c0c7170 legacy=0x7fa44c0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:20.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.193+0000 7fa44ad9d640 1 -- 172.21.15.26:0/4099436028 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0733d0 con 0x7fa44c0c8f20 2024-03-06T07:23:20.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa44ad9d640 1 --2- 172.21.15.26:0/4099436028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c0c8f20 0x7fa44c154160 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0ab220 tx=0x7fa44402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89a45436e0cee45 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:20.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa44959a640 1 -- 172.21.15.26:0/4099436028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44402c960 con 0x7fa44c0c8f20 2024-03-06T07:23:20.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa44959a640 1 -- 172.21.15.26:0/4099436028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44402cb00 con 0x7fa44c0c8f20 2024-03-06T07:23:20.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa44959a640 1 -- 172.21.15.26:0/4099436028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440343f0 con 0x7fa44c0c8f20 2024-03-06T07:23:20.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa450c8d640 1 -- 172.21.15.26:0/4099436028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c0c8f20 msgr2=0x7fa44c154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa450c8d640 1 --2- 172.21.15.26:0/4099436028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c0c8f20 0x7fa44c154160 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0ab220 tx=0x7fa44402bdc0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa450c8d640 1 -- 172.21.15.26:0/4099436028 shutdown_connections 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa450c8d640 1 --2- 172.21.15.26:0/4099436028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c0c8f20 0x7fa44c154160 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa450c8d640 1 --2- 172.21.15.26:0/4099436028 >> v2:172.21.15.117:3300/0 conn(0x7fa44c06ff10 0x7fa44c153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.194+0000 7fa450c8d640 1 -- 172.21.15.26:0/4099436028 >> 172.21.15.26:0/4099436028 conn(0x7fa44c06b970 msgr2=0x7fa44c0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.195+0000 7fa450c8d640 1 -- 172.21.15.26:0/4099436028 shutdown_connections 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.195+0000 7fa450c8d640 1 -- 172.21.15.26:0/4099436028 wait complete. 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 Processor -- start 2024-03-06T07:23:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 -- start start 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c06ff10 0x7fa44c1e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0c7170 0x7fa44c1e8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0c8f20 0x7fa44c0a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c070470 con 0x7fa44c06ff10 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0702f0 con 0x7fa44c0c7170 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.196+0000 7fa450c8d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0c6860 con 0x7fa44c0c8f20 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c06ff10 0x7fa44c1e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c06ff10 0x7fa44c1e97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43176/0 (socket says 172.21.15.26:43176) 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 -- 172.21.15.26:0/999932814 learned_addr learned my addr 172.21.15.26:0/999932814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44ad9d640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0c8f20 0x7fa44c0a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa449d9b640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0c7170 0x7fa44c1e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 -- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0c8f20 msgr2=0x7fa44c0a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0c8f20 0x7fa44c0a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 -- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0c7170 msgr2=0x7fa44c1e8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0c7170 0x7fa44c1e8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 -- 172.21.15.26:0/999932814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0aa970 con 0x7fa44c06ff10 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa44a59c640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c06ff10 0x7fa44c1e97d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00d830 tx=0x7fa43c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa43affd640 1 -- 172.21.15.26:0/999932814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00b6c0 con 0x7fa44c06ff10 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa43affd640 1 -- 172.21.15.26:0/999932814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c019070 con 0x7fa44c06ff10 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c155330 con 0x7fa44c06ff10 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa43affd640 1 -- 172.21.15.26:0/999932814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00ba80 con 0x7fa44c06ff10 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.197+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c154d70 con 0x7fa44c06ff10 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.199+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c04f330 con 0x7fa44c06ff10 2024-03-06T07:23:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.199+0000 7fa43affd640 1 -- 172.21.15.26:0/999932814 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa43c01e460 con 0x7fa44c06ff10 2024-03-06T07:23:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.201+0000 7fa43affd640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa42c06b4d0 0x7fa42c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.201+0000 7fa43affd640 1 -- 172.21.15.26:0/999932814 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fa43c08a770 con 0x7fa44c06ff10 2024-03-06T07:23:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.201+0000 7fa449d9b640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa42c06b4d0 0x7fa42c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.202+0000 7fa449d9b640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa42c06b4d0 0x7fa42c06d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa44c1524d0 tx=0x7fa440009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.204+0000 7fa43affd640 1 -- 172.21.15.26:0/999932814 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa43c0596f0 con 0x7fa44c06ff10 2024-03-06T07:23:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:20.397+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa44c06fa80 con 0x7fa44c06ff10 2024-03-06T07:23:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.217+0000 7fa43affd640 1 -- 172.21.15.26:0/999932814 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa43c058ff0 con 0x7fa44c06ff10 2024-03-06T07:23:21.219 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:23:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa42c06b4d0 msgr2=0x7fa42c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa42c06b4d0 0x7fa42c06d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa44c1524d0 tx=0x7fa440009040 comp rx=0 tx=0).stop 2024-03-06T07:23:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c06ff10 msgr2=0x7fa44c1e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c06ff10 0x7fa44c1e97d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00d830 tx=0x7fa43c00dcf0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 shutdown_connections 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa42c06b4d0 0x7fa42c06d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0c8f20 0x7fa44c0a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0c7170 0x7fa44c1e8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 --2- 172.21.15.26:0/999932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa44c06ff10 0x7fa44c1e97d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.220+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 >> 172.21.15.26:0/999932814 conn(0x7fa44c06b970 msgr2=0x7fa44c152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.221+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 shutdown_connections 2024-03-06T07:23:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.221+0000 7fa450c8d640 1 -- 172.21.15.26:0/999932814 wait complete. 2024-03-06T07:23:21.236 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:23:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd095e9640 1 Processor -- start 2024-03-06T07:23:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd095e9640 1 -- start start 2024-03-06T07:23:21.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd095e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd041529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd095e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd04155a40 0x7efd04155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd095e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd04156410 con 0x7efd040c1aa0 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd095e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd040c2680 con 0x7efd04155a40 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd095e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c28d0 con 0x7efd0414f790 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd04155a40 0x7efd04155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.342+0000 7efd027fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd040c1aa0 0x7efd040c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34758/0 (socket says 172.21.15.26:34758) 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd04155a40 0x7efd04155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43190/0 (socket says 172.21.15.26:43190) 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd027fc640 1 -- 172.21.15.26:0/575690501 learned_addr learned my addr 172.21.15.26:0/575690501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd02ffd640 1 --2- 172.21.15.26:0/575690501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd041529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd037fe640 1 -- 172.21.15.26:0/575690501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 msgr2=0x7efd041529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd037fe640 1 --2- 172.21.15.26:0/575690501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd041529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd037fe640 1 -- 172.21.15.26:0/575690501 >> v1:172.21.15.26:6789/0 conn(0x7efd040c1aa0 legacy=0x7efd040c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd037fe640 1 -- 172.21.15.26:0/575690501 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040c2bb0 con 0x7efd04155a40 2024-03-06T07:23:21.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.343+0000 7efd01ffb640 1 -- 172.21.15.26:0/575690501 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882490593 0 0) 0x7efd04156410 con 0x7efd040c1aa0 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd037fe640 1 --2- 172.21.15.26:0/575690501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd04155ed0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009d90 tx=0x7efcfc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8198d44cad49ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd01ffb640 1 -- 172.21.15.26:0/575690501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc039070 con 0x7efd04155a40 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd01ffb640 1 -- 172.21.15.26:0/575690501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc02cc50 con 0x7efd04155a40 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd01ffb640 1 -- 172.21.15.26:0/575690501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc034440 con 0x7efd04155a40 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 -- 172.21.15.26:0/575690501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 msgr2=0x7efd04155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 --2- 172.21.15.26:0/575690501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd04155ed0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009d90 tx=0x7efcfc02bea0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 -- 172.21.15.26:0/575690501 shutdown_connections 2024-03-06T07:23:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 --2- 172.21.15.26:0/575690501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd04155ed0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 --2- 172.21.15.26:0/575690501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd041529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 -- 172.21.15.26:0/575690501 >> 172.21.15.26:0/575690501 conn(0x7efd040c6710 msgr2=0x7efd040c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 -- 172.21.15.26:0/575690501 shutdown_connections 2024-03-06T07:23:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.344+0000 7efd095e9640 1 -- 172.21.15.26:0/575690501 wait complete. 2024-03-06T07:23:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 Processor -- start 2024-03-06T07:23:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 -- start start 2024-03-06T07:23:21.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd040c1aa0 0x7efd04072da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd04151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd041e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c0d60 con 0x7efd04155a40 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd04155520 con 0x7efd040c1aa0 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd095e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c5e30 con 0x7efd0414f790 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd041e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd041e90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43204/0 (socket says 172.21.15.26:43204) 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.346+0000 7efd037fe640 1 -- 172.21.15.26:0/4255269901 learned_addr learned my addr 172.21.15.26:0/4255269901 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd02ffd640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd040c1aa0 0x7efd04072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd027fc640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd04151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd037fe640 1 -- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 msgr2=0x7efd04151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd037fe640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd04151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd037fe640 1 -- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd040c1aa0 msgr2=0x7efd04072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd037fe640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd040c1aa0 0x7efd04072da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd037fe640 1 -- 172.21.15.26:0/4255269901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04159180 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd037fe640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd041e90d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009eb0 tx=0x7efcfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efceb7fe640 1 -- 172.21.15.26:0/4255269901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc039050 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efceb7fe640 1 -- 172.21.15.26:0/4255269901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc045070 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd04072300 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efceb7fe640 1 -- 172.21.15.26:0/4255269901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc03e610 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.347+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd041575e0 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.348+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0404f330 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.348+0000 7efceb7fe640 1 -- 172.21.15.26:0/4255269901 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcfc03e7b0 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.349+0000 7efceb7fe640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.349+0000 7efceb7fe640 1 -- 172.21.15.26:0/4255269901 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efcfc0acc10 con 0x7efd04155a40 2024-03-06T07:23:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.349+0000 7efd02ffd640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.350+0000 7efd02ffd640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf006b4d0 0x7efcf006d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efcf4006fd0 tx=0x7efcf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.353+0000 7efceb7fe640 1 -- 172.21.15.26:0/4255269901 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efcfc07bd60 con 0x7efd04155a40 2024-03-06T07:23:21.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.604+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efd040c1f40 con 0x7efd04155a40 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.604+0000 7efceb7fe640 1 -- 172.21.15.26:0/4255269901 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+4724 (secure 0 0 0) 0x7efcfc07b660 con 0x7efd04155a40 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:23:21.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.608 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.609 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:23:21.610 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:23:21.612 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.613 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:23:21.614 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:23:21.615 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:23:21.616 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:23:21.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:21.618 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:23:21.618 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:23:21.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf006b4d0 msgr2=0x7efcf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf006b4d0 0x7efcf006d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efcf4006fd0 tx=0x7efcf4008040 comp rx=0 tx=0).stop 2024-03-06T07:23:21.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 msgr2=0x7efd041e90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd041e90d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009eb0 tx=0x7efcfc004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 shutdown_connections 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd04155a40 0x7efd041e90d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd0414f790 0x7efd04151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 --2- 172.21.15.26:0/4255269901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd040c1aa0 0x7efd04072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 >> 172.21.15.26:0/4255269901 conn(0x7efd040c6710 msgr2=0x7efd040c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 shutdown_connections 2024-03-06T07:23:21.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.606+0000 7efd095e9640 1 -- 172.21.15.26:0/4255269901 wait complete. 2024-03-06T07:23:21.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:21.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.720+0000 7f683e993640 1 Processor -- start 2024-03-06T07:23:21.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.720+0000 7f683e993640 1 -- start start 2024-03-06T07:23:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f683e993640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 0x7f6838155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f683e993640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6838158b60 0x7f6838158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f683e993640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380c73d0 con 0x7f68380c88c0 2024-03-06T07:23:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f683e993640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380c8dd0 con 0x7f6838158b60 2024-03-06T07:23:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f683e993640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380be2e0 con 0x7f683814f360 2024-03-06T07:23:21.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f68377fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f68380c88c0 0x7f68380c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34774/0 (socket says 172.21.15.26:34774) 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f68377fe640 1 -- 172.21.15.26:0/1907575659 learned_addr learned my addr 172.21.15.26:0/1907575659 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110040777 0 0) 0x7f68380c73d0 con 0x7f68380c88c0 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.721+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6820003520 con 0x7f68380c88c0 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f683cf30640 1 --2- 172.21.15.26:0/1907575659 >> v2:172.21.15.117:3300/0 conn(0x7f6838158b60 0x7f6838158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6837fff640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 0x7f6838155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758920208 0 0) 0x7f6820003520 con 0x7f68380c88c0 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 msgr2=0x7f6838155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 0x7f6838155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 >> v2:172.21.15.117:3300/0 conn(0x7f6838158b60 msgr2=0x7f6838158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 --2- 172.21.15.26:0/1907575659 >> v2:172.21.15.117:3300/0 conn(0x7f6838158b60 0x7f6838158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380c7670 con 0x7f68380c88c0 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f682c001670 con 0x7f68380c88c0 2024-03-06T07:23:21.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 >> v1:172.21.15.26:6789/0 conn(0x7f68380c88c0 legacy=0x7f68380c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68200039b0 0x7f6820003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 0x7f6820004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6820004d40 0x7f68200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380be2e0 con 0x7f68200039b0 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682000baa0 con 0x7f6820004d40 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682000bca0 con 0x7f683814f360 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6837fff640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68200039b0 0x7f6820003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f68377fe640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 0x7f6820004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f683cf30640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6820004d40 0x7f68200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6837fff640 1 -- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 msgr2=0x7f6820004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6837fff640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683814f360 0x7f6820004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6837fff640 1 -- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6820004d40 msgr2=0x7f68200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6837fff640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6820004d40 0x7f68200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f6837fff640 1 -- 172.21.15.26:0/1907575659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820003520 con 0x7f68200039b0 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.722+0000 7f683cf30640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6820004d40 0x7f68200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.723+0000 7f6837fff640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68200039b0 0x7f6820003dd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f682800ba30 tx=0x7f6820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1a3a9a8b925ea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.723+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828036020 con 0x7f68200039b0 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.723+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828032640 con 0x7f68200039b0 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.723+0000 7f6836ffd640 1 -- 172.21.15.26:0/1907575659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800b600 con 0x7f68200039b0 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.724+0000 7f683e993640 1 -- 172.21.15.26:0/1907575659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68200039b0 msgr2=0x7f6820003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.724+0000 7f683e993640 1 --2- 172.21.15.26:0/1907575659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68200039b0 0x7f6820003dd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f682800ba30 tx=0x7f6820005830 comp rx=0 tx=0).stop 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.724+0000 7f683cf30640 1 -- 172.21.15.26:0/1907575659 reap_dead start 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.724+0000 7f683e993640 1 -- 172.21.15.26:0/1907575659 shutdown_connections 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.724+0000 7f683e993640 1 -- 172.21.15.26:0/1907575659 >> 172.21.15.26:0/1907575659 conn(0x7f68380ac570 msgr2=0x7f68380ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.724+0000 7f683e993640 1 -- 172.21.15.26:0/1907575659 shutdown_connections 2024-03-06T07:23:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.724+0000 7f683e993640 1 -- 172.21.15.26:0/1907575659 wait complete. 2024-03-06T07:23:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.725+0000 7f683e993640 1 Processor -- start 2024-03-06T07:23:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.725+0000 7f683e993640 1 -- start start 2024-03-06T07:23:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68200039b0 0x7f68380ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68380bedc0 0x7f683814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68380c47a0 0x7f683814c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c5a50 con 0x7f68380bedc0 2024-03-06T07:23:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c7c30 con 0x7f68200039b0 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c5bd0 con 0x7f68380c47a0 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68380bedc0 0x7f683814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68380bedc0 0x7f683814bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43224/0 (socket says 172.21.15.26:43224) 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 -- 172.21.15.26:0/302495797 learned_addr learned my addr 172.21.15.26:0/302495797 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 -- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68380c47a0 msgr2=0x7f683814c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68380c47a0 0x7f683814c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 -- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68200039b0 msgr2=0x7f68380ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68200039b0 0x7f68380ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 -- 172.21.15.26:0/302495797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68381eb5b0 con 0x7f68380bedc0 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f68377fe640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68380bedc0 0x7f683814bc50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f682c007340 tx=0x7f682c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f6834ff9640 1 -- 172.21.15.26:0/302495797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c01a070 con 0x7f68380bedc0 2024-03-06T07:23:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f6834ff9640 1 -- 172.21.15.26:0/302495797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c00f860 con 0x7f68380bedc0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380be620 con 0x7f68380bedc0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f6834ff9640 1 -- 172.21.15.26:0/302495797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00fc50 con 0x7f68380bedc0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.726+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68380be900 con 0x7f68380bedc0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.727+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683804f330 con 0x7f68380bedc0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.728+0000 7f6834ff9640 1 -- 172.21.15.26:0/302495797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f682c01e3f0 con 0x7f68380bedc0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.728+0000 7f6834ff9640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682406b500 0x7f682406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.728+0000 7f6834ff9640 1 -- 172.21.15.26:0/302495797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f682c089760 con 0x7f68380bedc0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.728+0000 7f6837fff640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682406b500 0x7f682406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.729+0000 7f6837fff640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682406b500 0x7f682406d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f682800ba00 tx=0x7f6828032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.732+0000 7f6834ff9640 1 -- 172.21.15.26:0/302495797 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f682c058810 con 0x7f68380bedc0 2024-03-06T07:23:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.912+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f683806cda0 con 0x7f68380bedc0 2024-03-06T07:23:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.913+0000 7f6834ff9640 1 -- 172.21.15.26:0/302495797 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13177 (secure 0 0 0) 0x7f682c058110 con 0x7f68380bedc0 2024-03-06T07:23:21.914 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:21.915 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":91,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:21.216909+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682406b500 msgr2=0x7f682406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682406b500 0x7f682406d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f682800ba00 tx=0x7f6828032000 comp rx=0 tx=0).stop 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68380bedc0 msgr2=0x7f683814bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68380bedc0 0x7f683814bc50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f682c007340 tx=0x7f682c007800 comp rx=0 tx=0).stop 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 shutdown_connections 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f682406b500 0x7f682406d9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68380c47a0 0x7f683814c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68380bedc0 0x7f683814bc50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 --2- 172.21.15.26:0/302495797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68200039b0 0x7f68380ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 >> 172.21.15.26:0/302495797 conn(0x7f68380ac570 msgr2=0x7f683814cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 shutdown_connections 2024-03-06T07:23:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:21.915+0000 7f683e993640 1 -- 172.21.15.26:0/302495797 wait complete. 2024-03-06T07:23:21.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:21.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.031+0000 7fdea5e15640 1 Processor -- start 2024-03-06T07:23:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.031+0000 7fdea5e15640 1 -- start start 2024-03-06T07:23:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fdea5e15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fdea5e15640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdea00c57e0 0x7fdea0155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fdea5e15640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdea0072cd0 0x7fdea0155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fdea5e15640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea00c6dd0 con 0x7fdea00c57e0 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fdea5e15640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea00c6940 con 0x7fdea0072cd0 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fdea5e15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0152db0 con 0x7fdea01529d0 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdea00c57e0 0x7fdea0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdea00c57e0 0x7fdea0155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43240/0 (socket says 172.21.15.26:43240) 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 -- 172.21.15.26:0/3490636514 learned_addr learned my addr 172.21.15.26:0/3490636514 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9f7fe640 1 --2- 172.21.15.26:0/3490636514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9ffff640 1 --2- 172.21.15.26:0/3490636514 >> v2:172.21.15.117:3300/0 conn(0x7fdea0072cd0 0x7fdea0155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 -- 172.21.15.26:0/3490636514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 msgr2=0x7fdea00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 --2- 172.21.15.26:0/3490636514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 -- 172.21.15.26:0/3490636514 >> v2:172.21.15.117:3300/0 conn(0x7fdea0072cd0 msgr2=0x7fdea0155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 --2- 172.21.15.26:0/3490636514 >> v2:172.21.15.117:3300/0 conn(0x7fdea0072cd0 0x7fdea0155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9effd640 1 -- 172.21.15.26:0/3490636514 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea01590f0 con 0x7fdea00c57e0 2024-03-06T07:23:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.032+0000 7fde9ffff640 1 --2- 172.21.15.26:0/3490636514 >> v2:172.21.15.117:3300/0 conn(0x7fdea0072cd0 0x7fdea0155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fde9effd640 1 --2- 172.21.15.26:0/3490636514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea00c57e0 0x7fdea0155120 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdea00c73f0 tx=0x7fde9402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7784fbf3195ff015 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fde9e7fc640 1 -- 172.21.15.26:0/3490636514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94038070 con 0x7fdea00c57e0 2024-03-06T07:23:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fde9e7fc640 1 -- 172.21.15.26:0/3490636514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9402f050 con 0x7fdea00c57e0 2024-03-06T07:23:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fde9e7fc640 1 -- 172.21.15.26:0/3490636514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94031970 con 0x7fdea00c57e0 2024-03-06T07:23:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fdea5e15640 1 -- 172.21.15.26:0/3490636514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea00c57e0 msgr2=0x7fdea0155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3490636514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea00c57e0 0x7fdea0155120 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdea00c73f0 tx=0x7fde9402b7f0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fdea5e15640 1 -- 172.21.15.26:0/3490636514 shutdown_connections 2024-03-06T07:23:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3490636514 >> v2:172.21.15.117:3300/0 conn(0x7fdea0072cd0 0x7fdea0155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3490636514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea00c57e0 0x7fdea0155120 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3490636514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.033+0000 7fdea5e15640 1 -- 172.21.15.26:0/3490636514 >> 172.21.15.26:0/3490636514 conn(0x7fdea00aa630 msgr2=0x7fdea00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.034+0000 7fdea5e15640 1 -- 172.21.15.26:0/3490636514 shutdown_connections 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.034+0000 7fdea5e15640 1 -- 172.21.15.26:0/3490636514 wait complete. 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 Processor -- start 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 -- start start 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea0072cd0 0x7fdea00c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdea00c57e0 0x7fdea006e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0159370 con 0x7fdea0072cd0 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0072720 con 0x7fdea00c57e0 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fdea5e15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00723b0 con 0x7fdea01529d0 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.035+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea0072cd0 0x7fdea00c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea0072cd0 0x7fdea00c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43250/0 (socket says 172.21.15.26:43250) 2024-03-06T07:23:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 -- 172.21.15.26:0/3349863289 learned_addr learned my addr 172.21.15.26:0/3349863289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9ffff640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9effd640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdea00c57e0 0x7fdea006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 -- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 msgr2=0x7fdea00ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 -- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdea00c57e0 msgr2=0x7fdea006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdea00c57e0 0x7fdea006e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 -- 172.21.15.26:0/3349863289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea01eb680 con 0x7fdea0072cd0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde9f7fe640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea0072cd0 0x7fdea00c30a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fde9000e9b0 tx=0x7fde9000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde87fff640 1 -- 172.21.15.26:0/3349863289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90017070 con 0x7fdea0072cd0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde87fff640 1 -- 172.21.15.26:0/3349863289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9000c9a0 con 0x7fdea0072cd0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea01eb960 con 0x7fdea0072cd0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fde87fff640 1 -- 172.21.15.26:0/3349863289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9000cd90 con 0x7fdea0072cd0 2024-03-06T07:23:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.036+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea01ee5d0 con 0x7fdea0072cd0 2024-03-06T07:23:22.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.037+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea0073540 con 0x7fdea0072cd0 2024-03-06T07:23:22.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.038+0000 7fde87fff640 1 -- 172.21.15.26:0/3349863289 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde9001a070 con 0x7fdea0072cd0 2024-03-06T07:23:22.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.038+0000 7fde87fff640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde8c06b500 0x7fde8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.038+0000 7fde87fff640 1 -- 172.21.15.26:0/3349863289 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fde9008a2e0 con 0x7fdea0072cd0 2024-03-06T07:23:22.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.038+0000 7fde9effd640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde8c06b500 0x7fde8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.039+0000 7fde9effd640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde8c06b500 0x7fde8c06d9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde94035000 tx=0x7fde94008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.043+0000 7fde87fff640 1 -- 172.21.15.26:0/3349863289 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdea0073540 con 0x7fdea0072cd0 2024-03-06T07:23:22.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.222+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdea00a9310 con 0x7fdea0072cd0 2024-03-06T07:23:22.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.223+0000 7fde87fff640 1 -- 172.21.15.26:0/3349863289 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdea00a9310 con 0x7fdea0072cd0 2024-03-06T07:23:22.224 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:23:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde8c06b500 msgr2=0x7fde8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde8c06b500 0x7fde8c06d9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde94035000 tx=0x7fde94008030 comp rx=0 tx=0).stop 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea0072cd0 msgr2=0x7fdea00c30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea0072cd0 0x7fdea00c30a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fde9000e9b0 tx=0x7fde9000ee70 comp rx=0 tx=0).stop 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 shutdown_connections 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde8c06b500 0x7fde8c06d9c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdea01529d0 0x7fdea00ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdea00c57e0 0x7fdea006e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 --2- 172.21.15.26:0/3349863289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdea0072cd0 0x7fdea00c30a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 >> 172.21.15.26:0/3349863289 conn(0x7fdea00aa630 msgr2=0x7fdea00c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 shutdown_connections 2024-03-06T07:23:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.225+0000 7fdea5e15640 1 -- 172.21.15.26:0/3349863289 wait complete. 2024-03-06T07:23:22.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:23:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.342+0000 7f150d50f640 1 Processor -- start 2024-03-06T07:23:22.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.342+0000 7f150d50f640 1 -- start start 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f150d50f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15080c7380 0x7f15080c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f150d50f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1508072630 con 0x7f15080c6780 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f150d50f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1508072910 con 0x7f150806da40 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f150d50f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150806de20 con 0x7f15080c7380 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f15077fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15080c7380 0x7f15080c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f15067fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15080c6780 0x7f1508158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34784/0 (socket says 172.21.15.26:34784) 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f15067fc640 1 -- 172.21.15.26:0/3659469068 learned_addr learned my addr 172.21.15.26:0/3659469068 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f15077fe640 1 -- 172.21.15.26:0/3659469068 >> v1:172.21.15.117:6789/0 conn(0x7f150806da40 legacy=0x7f1508155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f15077fe640 1 -- 172.21.15.26:0/3659469068 >> v1:172.21.15.26:6789/0 conn(0x7f15080c6780 legacy=0x7f1508158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f1505ffb640 1 -- 172.21.15.26:0/3659469068 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626197933 0 0) 0x7f1508072630 con 0x7f15080c6780 2024-03-06T07:23:22.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.343+0000 7f15077fe640 1 -- 172.21.15.26:0/3659469068 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508072bf0 con 0x7f15080c7380 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f15077fe640 1 --2- 172.21.15.26:0/3659469068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f15080c3280 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45083f7b2d081c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f1505ffb640 1 -- 172.21.15.26:0/3659469068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f80315a0 con 0x7f15080c7380 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f1505ffb640 1 -- 172.21.15.26:0/3659469068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f8037070 con 0x7f15080c7380 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f1505ffb640 1 -- 172.21.15.26:0/3659469068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8031960 con 0x7f15080c7380 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f150d50f640 1 -- 172.21.15.26:0/3659469068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 msgr2=0x7f15080c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f150d50f640 1 --2- 172.21.15.26:0/3659469068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f15080c3280 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f150d50f640 1 -- 172.21.15.26:0/3659469068 shutdown_connections 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f150d50f640 1 --2- 172.21.15.26:0/3659469068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f15080c3280 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.344+0000 7f150d50f640 1 -- 172.21.15.26:0/3659469068 >> 172.21.15.26:0/3659469068 conn(0x7f15080aa630 msgr2=0x7f15080aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.345+0000 7f150d50f640 1 -- 172.21.15.26:0/3659469068 shutdown_connections 2024-03-06T07:23:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.345+0000 7f150d50f640 1 -- 172.21.15.26:0/3659469068 wait complete. 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 Processor -- start 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 -- start start 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150806da40 0x7f15080a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15080c6780 0x7f15080c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f1508157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15080c37c0 con 0x7f15080c7380 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081594a0 con 0x7f15080c6780 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.346+0000 7f150d50f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15080c3940 con 0x7f150806da40 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f1508157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f1508157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43268/0 (socket says 172.21.15.26:43268) 2024-03-06T07:23:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 -- 172.21.15.26:0/633302189 learned_addr learned my addr 172.21.15.26:0/633302189 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15067fc640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15080c6780 0x7f15080c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f1506ffd640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150806da40 0x7f15080a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 -- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150806da40 msgr2=0x7f15080a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150806da40 0x7f15080a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 -- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15080c6780 msgr2=0x7f15080c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15080c6780 0x7f15080c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 -- 172.21.15.26:0/633302189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15080a9d30 con 0x7f15080c7380 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15067fc640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15080c6780 0x7f15080c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f1506ffd640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150806da40 0x7f15080a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f15077fe640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f1508157760 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f14ef7fe640 1 -- 172.21.15.26:0/633302189 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8032630 con 0x7f15080c7380 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f14ef7fe640 1 -- 172.21.15.26:0/633302189 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f80327d0 con 0x7f15080c7380 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15080ae4f0 con 0x7f15080c7380 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f14ef7fe640 1 -- 172.21.15.26:0/633302189 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8032bc0 con 0x7f15080c7380 2024-03-06T07:23:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.347+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150806fac0 con 0x7f15080c7380 2024-03-06T07:23:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.348+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150804f330 con 0x7f15080c7380 2024-03-06T07:23:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.349+0000 7f14ef7fe640 1 -- 172.21.15.26:0/633302189 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14f804c020 con 0x7f15080c7380 2024-03-06T07:23:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.349+0000 7f14ef7fe640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14f406b4d0 0x7f14f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.349+0000 7f14ef7fe640 1 -- 172.21.15.26:0/633302189 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f14f802f020 con 0x7f15080c7380 2024-03-06T07:23:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.349+0000 7f1506ffd640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14f406b4d0 0x7f14f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.350+0000 7f1506ffd640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14f406b4d0 0x7f14f406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1500007eb0 tx=0x7f15000073c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.353+0000 7f14ef7fe640 1 -- 172.21.15.26:0/633302189 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14f8040070 con 0x7f15080c7380 2024-03-06T07:23:22.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.541+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1508155fd0 con 0x7f15080c7380 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.541+0000 7f14ef7fe640 1 -- 172.21.15.26:0/633302189 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) v1 ==== 84+0+65 (secure 0 0 0) 0x7f14f807ac80 con 0x7f15080c7380 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:23:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14f406b4d0 msgr2=0x7f14f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14f406b4d0 0x7f14f406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1500007eb0 tx=0x7f15000073c0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 msgr2=0x7f1508157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f1508157760 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f80046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 shutdown_connections 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14f406b4d0 0x7f14f406d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15080c7380 0x7f1508157760 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15080c6780 0x7f15080c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 --2- 172.21.15.26:0/633302189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150806da40 0x7f15080a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 >> 172.21.15.26:0/633302189 conn(0x7f15080aa630 msgr2=0x7f150814d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 shutdown_connections 2024-03-06T07:23:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.543+0000 7f150d50f640 1 -- 172.21.15.26:0/633302189 wait complete. 2024-03-06T07:23:22.556 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:23:22.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:23:22.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.660+0000 7f2593502640 1 Processor -- start 2024-03-06T07:23:22.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.660+0000 7f2593502640 1 -- start start 2024-03-06T07:23:22.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.661+0000 7f2593502640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 0x7f258c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.661+0000 7f2593502640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c0c6220 con 0x7f258c0c5e40 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.661+0000 7f2593502640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c070210 con 0x7f258c072a10 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.661+0000 7f2593502640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0704d0 con 0x7f258c072fc0 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.661+0000 7f259129e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f258c0c5e40 0x7f258c153940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34798/0 (socket says 172.21.15.26:34798) 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.661+0000 7f259129e640 1 -- 172.21.15.26:0/2005955266 learned_addr learned my addr 172.21.15.26:0/2005955266 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734014827 0 0) 0x7f258c0c6220 con 0x7f258c0c5e40 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2574003520 con 0x7f258c0c5e40 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f2590a9d640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 0x7f258c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1548922537 0 0) 0x7f2574003520 con 0x7f258c0c5e40 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 msgr2=0x7f258c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 0x7f258c0724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 >> v1:172.21.15.117:6789/0 conn(0x7f258c072a10 legacy=0x7f258c0c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c06f610 con 0x7f258c0c5e40 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f2584001670 con 0x7f258c0c5e40 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 >> v1:172.21.15.26:6789/0 conn(0x7f258c0c5e40 legacy=0x7f258c153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25740039e0 0x7f2574003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2574004800 0x7f2574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 0x7f2574005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0704d0 con 0x7f25740039e0 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257400baf0 con 0x7f2574004800 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257400bcf0 con 0x7f258c072fc0 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f259129e640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25740039e0 0x7f2574003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f2590a9d640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2574004800 0x7f2574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f2591a9f640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 0x7f2574005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f259129e640 1 -- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 msgr2=0x7f2574005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f259129e640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 0x7f2574005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f259129e640 1 -- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2574004800 msgr2=0x7f2574004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f259129e640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2574004800 0x7f2574004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f259129e640 1 -- 172.21.15.26:0/2005955266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574003520 con 0x7f25740039e0 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.662+0000 7f2591a9f640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c072fc0 0x7f2574005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.663+0000 7f259129e640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25740039e0 0x7f2574003e00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f258400c210 tx=0x7f2574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2f33d1172a953e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.663+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584007d30 con 0x7f25740039e0 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.663+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584030070 con 0x7f25740039e0 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.663+0000 7f258bfff640 1 -- 172.21.15.26:0/2005955266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258402cd60 con 0x7f25740039e0 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.664+0000 7f2593502640 1 -- 172.21.15.26:0/2005955266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25740039e0 msgr2=0x7f2574003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.664+0000 7f2593502640 1 --2- 172.21.15.26:0/2005955266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25740039e0 0x7f2574003e00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f258400c210 tx=0x7f2574005830 comp rx=0 tx=0).stop 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.664+0000 7f2591a9f640 1 -- 172.21.15.26:0/2005955266 reap_dead start 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.664+0000 7f2593502640 1 -- 172.21.15.26:0/2005955266 shutdown_connections 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.664+0000 7f2593502640 1 -- 172.21.15.26:0/2005955266 >> 172.21.15.26:0/2005955266 conn(0x7f258c0c88f0 msgr2=0x7f258c0c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.664+0000 7f2593502640 1 -- 172.21.15.26:0/2005955266 shutdown_connections 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.664+0000 7f2593502640 1 -- 172.21.15.26:0/2005955266 wait complete. 2024-03-06T07:23:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.665+0000 7f2593502640 1 Processor -- start 2024-03-06T07:23:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.665+0000 7f2593502640 1 -- start start 2024-03-06T07:23:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2593502640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25740039e0 0x7f258c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2593502640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c0c24a0 0x7f258c1ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2593502640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c1ec2a0 0x7f258c0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2593502640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0724d0 con 0x7f258c1ec2a0 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2593502640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c06fbd0 con 0x7f25740039e0 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2593502640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c072650 con 0x7f258c0c24a0 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c1ec2a0 0x7f258c0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c1ec2a0 0x7f258c0ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43290/0 (socket says 172.21.15.26:43290) 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 -- 172.21.15.26:0/1735032822 learned_addr learned my addr 172.21.15.26:0/1735032822 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f259129e640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25740039e0 0x7f258c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2590a9d640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c0c24a0 0x7f258c1ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 -- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c0c24a0 msgr2=0x7f258c1ebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c0c24a0 0x7f258c1ebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 -- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25740039e0 msgr2=0x7f258c14d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25740039e0 0x7f258c14d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 -- 172.21.15.26:0/1735032822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c1ec810 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2590a9d640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c0c24a0 0x7f258c1ebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f259129e640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25740039e0 0x7f258c14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2591a9f640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c1ec2a0 0x7f258c0ad380 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f258c0707b0 tx=0x7f257c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2589ffb640 1 -- 172.21.15.26:0/1735032822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c006b50 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2589ffb640 1 -- 172.21.15.26:0/1735032822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c006d10 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c0ae630 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.666+0000 7f2589ffb640 1 -- 172.21.15.26:0/1735032822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c003970 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.667+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c14d750 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.668+0000 7f2589ffb640 1 -- 172.21.15.26:0/1735032822 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f257c01a020 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.668+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c04f330 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.668+0000 7f2589ffb640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f257806b4d0 0x7f257806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.668+0000 7f2589ffb640 1 -- 172.21.15.26:0/1735032822 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f257c089630 con 0x7f258c1ec2a0 2024-03-06T07:23:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.668+0000 7f259129e640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f257806b4d0 0x7f257806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.669+0000 7f259129e640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f257806b4d0 0x7f257806d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2584034000 tx=0x7f2584009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.672+0000 7f2589ffb640 1 -- 172.21.15.26:0/1735032822 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f257c058700 con 0x7f258c1ec2a0 2024-03-06T07:23:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:22.853+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f258c06f3f0 con 0x7f258c1ec2a0 2024-03-06T07:23:23.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.222+0000 7f2589ffb640 1 -- 172.21.15.26:0/1735032822 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) v1 ==== 138+0+0 (secure 0 0 0) 0x7f257c058000 con 0x7f258c1ec2a0 2024-03-06T07:23:23.224 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f257806b4d0 msgr2=0x7f257806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f257806b4d0 0x7f257806d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2584034000 tx=0x7f2584009030 comp rx=0 tx=0).stop 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c1ec2a0 msgr2=0x7f258c0ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c1ec2a0 0x7f258c0ad380 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f258c0707b0 tx=0x7f257c00eb90 comp rx=0 tx=0).stop 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 shutdown_connections 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f257806b4d0 0x7f257806d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c1ec2a0 0x7f258c0ad380 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f258c0c24a0 0x7f258c1ebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 --2- 172.21.15.26:0/1735032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25740039e0 0x7f258c14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.225+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 >> 172.21.15.26:0/1735032822 conn(0x7f258c0c88f0 msgr2=0x7f258c0a87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:23.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.226+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 shutdown_connections 2024-03-06T07:23:23.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.226+0000 7f2593502640 1 -- 172.21.15.26:0/1735032822 wait complete. 2024-03-06T07:23:23.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:23:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.347+0000 7fd6f0029640 1 Processor -- start 2024-03-06T07:23:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.348+0000 7fd6f0029640 1 -- start start 2024-03-06T07:23:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.348+0000 7fd6f0029640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6e81528d0 0x7fd6e80724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.348+0000 7fd6f0029640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e8072a10 0x7fd6e8155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.348+0000 7fd6f0029640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e8152cb0 con 0x7fd6e80c67f0 2024-03-06T07:23:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.348+0000 7fd6f0029640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e80c38b0 con 0x7fd6e81528d0 2024-03-06T07:23:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.348+0000 7fd6f0029640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e81594f0 con 0x7fd6e8072a10 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6eddc5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6e81528d0 0x7fd6e80724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6eddc5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6e81528d0 0x7fd6e80724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43300/0 (socket says 172.21.15.26:43300) 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6eddc5640 1 -- 172.21.15.26:0/2609513684 learned_addr learned my addr 172.21.15.26:0/2609513684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6eddc5640 1 -- 172.21.15.26:0/2609513684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e8072a10 msgr2=0x7fd6e8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6ed5c4640 1 --2- 172.21.15.26:0/2609513684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e8072a10 0x7fd6e8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6eddc5640 1 --2- 172.21.15.26:0/2609513684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e8072a10 0x7fd6e8155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6eddc5640 1 -- 172.21.15.26:0/2609513684 >> v1:172.21.15.117:6789/0 conn(0x7fd6e80c67f0 legacy=0x7fd6e8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6eddc5640 1 -- 172.21.15.26:0/2609513684 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80a7f70 con 0x7fd6e81528d0 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.349+0000 7fd6ed5c4640 1 --2- 172.21.15.26:0/2609513684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e8072a10 0x7fd6e8155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6eddc5640 1 --2- 172.21.15.26:0/2609513684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e81528d0 0x7fd6e80724d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009870 tx=0x7fd6e402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cd21f30fac88e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6ecdc3640 1 -- 172.21.15.26:0/2609513684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4034070 con 0x7fd6e81528d0 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6ecdc3640 1 -- 172.21.15.26:0/2609513684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4030840 con 0x7fd6e81528d0 2024-03-06T07:23:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6ecdc3640 1 -- 172.21.15.26:0/2609513684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4030c10 con 0x7fd6e81528d0 2024-03-06T07:23:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6f0029640 1 -- 172.21.15.26:0/2609513684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e81528d0 msgr2=0x7fd6e80724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6f0029640 1 --2- 172.21.15.26:0/2609513684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e81528d0 0x7fd6e80724d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009870 tx=0x7fd6e402b980 comp rx=0 tx=0).stop 2024-03-06T07:23:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6f0029640 1 -- 172.21.15.26:0/2609513684 shutdown_connections 2024-03-06T07:23:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6f0029640 1 --2- 172.21.15.26:0/2609513684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e8072a10 0x7fd6e8155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6f0029640 1 --2- 172.21.15.26:0/2609513684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e81528d0 0x7fd6e80724d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.350+0000 7fd6f0029640 1 -- 172.21.15.26:0/2609513684 >> 172.21.15.26:0/2609513684 conn(0x7fd6e80aa6a0 msgr2=0x7fd6e80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.351+0000 7fd6f0029640 1 -- 172.21.15.26:0/2609513684 shutdown_connections 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.351+0000 7fd6f0029640 1 -- 172.21.15.26:0/2609513684 wait complete. 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 Processor -- start 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 -- start start 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e8072a10 0x7fd6e8158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e80c67f0 0x7fd6e80a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6e81528d0 0x7fd6e81ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e81591a0 con 0x7fd6e8072a10 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80a8250 con 0x7fd6e81528d0 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6f0029640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80c4ef0 con 0x7fd6e80c67f0 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6eddc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e8072a10 0x7fd6e8158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6eddc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e8072a10 0x7fd6e8158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43304/0 (socket says 172.21.15.26:43304) 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.352+0000 7fd6eddc5640 1 -- 172.21.15.26:0/3660068392 learned_addr learned my addr 172.21.15.26:0/3660068392 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6ed5c4640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e80c67f0 0x7fd6e80a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6ee5c6640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6e81528d0 0x7fd6e81ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6eddc5640 1 -- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e80c67f0 msgr2=0x7fd6e80a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6eddc5640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e80c67f0 0x7fd6e80a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6eddc5640 1 -- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6e81528d0 msgr2=0x7fd6e81ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6eddc5640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6e81528d0 0x7fd6e81ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6eddc5640 1 -- 172.21.15.26:0/3660068392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e814bb80 con 0x7fd6e8072a10 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6eddc5640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e8072a10 0x7fd6e8158830 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6de7fc640 1 -- 172.21.15.26:0/3660068392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4034040 con 0x7fd6e8072a10 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6de7fc640 1 -- 172.21.15.26:0/3660068392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4039d90 con 0x7fd6e8072a10 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e80ae420 con 0x7fd6e8072a10 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6de7fc640 1 -- 172.21.15.26:0/3660068392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4043610 con 0x7fd6e8072a10 2024-03-06T07:23:23.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.353+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e80aba40 con 0x7fd6e8072a10 2024-03-06T07:23:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.354+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e804f330 con 0x7fd6e8072a10 2024-03-06T07:23:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.354+0000 7fd6de7fc640 1 -- 172.21.15.26:0/3660068392 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6e404a060 con 0x7fd6e8072a10 2024-03-06T07:23:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.355+0000 7fd6de7fc640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6cc06b500 0x7fd6cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.355+0000 7fd6de7fc640 1 -- 172.21.15.26:0/3660068392 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fd6e40ac260 con 0x7fd6e8072a10 2024-03-06T07:23:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.355+0000 7fd6ed5c4640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6cc06b500 0x7fd6cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.356+0000 7fd6ed5c4640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6cc06b500 0x7fd6cc06d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd6e814ca20 tx=0x7fd6d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.360+0000 7fd6de7fc640 1 -- 172.21.15.26:0/3660068392 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd6e403e070 con 0x7fd6e8072a10 2024-03-06T07:23:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:23.540+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd6e8074700 con 0x7fd6e8072a10 2024-03-06T07:23:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.225+0000 7fd6de7fc640 1 -- 172.21.15.26:0/3660068392 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd6e402d330 con 0x7fd6e8072a10 2024-03-06T07:23:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6cc06b500 msgr2=0x7fd6cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6cc06b500 0x7fd6cc06d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd6e814ca20 tx=0x7fd6d8008040 comp rx=0 tx=0).stop 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e8072a10 msgr2=0x7fd6e8158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e8072a10 0x7fd6e8158830 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e40046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 shutdown_connections 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6cc06b500 0x7fd6cc06d9c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6e81528d0 0x7fd6e81ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6e80c67f0 0x7fd6e80a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 --2- 172.21.15.26:0/3660068392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e8072a10 0x7fd6e8158830 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 >> 172.21.15.26:0/3660068392 conn(0x7fd6e80aa6a0 msgr2=0x7fd6e806d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 shutdown_connections 2024-03-06T07:23:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.228+0000 7fd6f0029640 1 -- 172.21.15.26:0/3660068392 wait complete. 2024-03-06T07:23:24.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:23:24.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.366+0000 7f91bb4e0640 1 Processor -- start 2024-03-06T07:23:24.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.366+0000 7f91bb4e0640 1 -- start start 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91bb4e0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91b40c6780 0x7f91b4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91bb4e0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b41592d0 0x7f91b40c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91bb4e0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b4072630 con 0x7f91b406da40 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91bb4e0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b406de20 con 0x7f91b40c6780 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91bb4e0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b4072910 con 0x7f91b41592d0 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b8a7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91b40c6780 0x7f91b4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b8a7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91b40c6780 0x7f91b4158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43306/0 (socket says 172.21.15.26:43306) 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b8a7b640 1 -- 172.21.15.26:0/1407970243 learned_addr learned my addr 172.21.15.26:0/1407970243 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b9a7d640 1 --2- 172.21.15.26:0/1407970243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b41592d0 0x7f91b40c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b8a7b640 1 -- 172.21.15.26:0/1407970243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b41592d0 msgr2=0x7f91b40c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b8a7b640 1 --2- 172.21.15.26:0/1407970243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b41592d0 0x7f91b40c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b8a7b640 1 -- 172.21.15.26:0/1407970243 >> v1:172.21.15.117:6789/0 conn(0x7f91b406da40 legacy=0x7f91b4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.367+0000 7f91b8a7b640 1 -- 172.21.15.26:0/1407970243 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4072bf0 con 0x7f91b40c6780 2024-03-06T07:23:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.368+0000 7f91b8a7b640 1 --2- 172.21.15.26:0/1407970243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4158d90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009870 tx=0x7f91a402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80745c1c2cc74366 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.368+0000 7f91abfff640 1 -- 172.21.15.26:0/1407970243 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4035020 con 0x7f91b40c6780 2024-03-06T07:23:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.368+0000 7f91abfff640 1 -- 172.21.15.26:0/1407970243 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a4031640 con 0x7f91b40c6780 2024-03-06T07:23:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.368+0000 7f91abfff640 1 -- 172.21.15.26:0/1407970243 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a402d040 con 0x7f91b40c6780 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 -- 172.21.15.26:0/1407970243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 msgr2=0x7f91b4158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/1407970243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4158d90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009870 tx=0x7f91a402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 -- 172.21.15.26:0/1407970243 shutdown_connections 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/1407970243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b41592d0 0x7f91b40c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/1407970243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4158d90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 -- 172.21.15.26:0/1407970243 >> 172.21.15.26:0/1407970243 conn(0x7f91b40aa630 msgr2=0x7f91b40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 -- 172.21.15.26:0/1407970243 shutdown_connections 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.369+0000 7f91bb4e0640 1 -- 172.21.15.26:0/1407970243 wait complete. 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 Processor -- start 2024-03-06T07:23:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 -- start start 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b406da40 0x7f91b41eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b41592d0 0x7f91b41ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b40c3940 con 0x7f91b40c6780 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b40c37c0 con 0x7f91b41592d0 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.370+0000 7f91bb4e0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b40c3ac0 con 0x7f91b406da40 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43308/0 (socket says 172.21.15.26:43308) 2024-03-06T07:23:24.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 -- 172.21.15.26:0/3556482276 learned_addr learned my addr 172.21.15.26:0/3556482276 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b9a7d640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b41592d0 0x7f91b41ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b927c640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b406da40 0x7f91b41eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 -- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b406da40 msgr2=0x7f91b41eec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b406da40 0x7f91b41eec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 -- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b41592d0 msgr2=0x7f91b41ead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b41592d0 0x7f91b41ead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 -- 172.21.15.26:0/3556482276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b41ef1b0 con 0x7f91b40c6780 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b9a7d640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b41592d0 0x7f91b41ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b927c640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b406da40 0x7f91b41eec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91b8a7b640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4157740 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f91a402b930 tx=0x7f91a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91a9ffb640 1 -- 172.21.15.26:0/3556482276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4040070 con 0x7f91b40c6780 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91a9ffb640 1 -- 172.21.15.26:0/3556482276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a403e070 con 0x7f91b40c6780 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b40acf40 con 0x7f91b40c6780 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91a9ffb640 1 -- 172.21.15.26:0/3556482276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4035040 con 0x7f91b40c6780 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.371+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b40ae040 con 0x7f91b40c6780 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.372+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b404f330 con 0x7f91b40c6780 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.373+0000 7f91a9ffb640 1 -- 172.21.15.26:0/3556482276 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91a4038020 con 0x7f91b40c6780 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.373+0000 7f91a9ffb640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f919806b4d0 0x7f919806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.373+0000 7f91a9ffb640 1 -- 172.21.15.26:0/3556482276 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f91a40abd40 con 0x7f91b40c6780 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.374+0000 7f91b927c640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f919806b4d0 0x7f919806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.374+0000 7f91b927c640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f919806b4d0 0x7f919806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f91b00015e0 tx=0x7f91b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.378+0000 7f91a9ffb640 1 -- 172.21.15.26:0/3556482276 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f91a407aa80 con 0x7f91b40c6780 2024-03-06T07:23:24.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:24.598+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91b40ae280 con 0x7f91b40c6780 2024-03-06T07:23:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.251+0000 7f91a9ffb640 1 -- 172.21.15.26:0/3556482276 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7f91a4043e40 con 0x7f91b40c6780 2024-03-06T07:23:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 15 and data pool 16 2024-03-06T07:23:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.257+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f919806b4d0 msgr2=0x7f919806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.257+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f919806b4d0 0x7f919806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f91b00015e0 tx=0x7f91b0009040 comp rx=0 tx=0).stop 2024-03-06T07:23:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.257+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 msgr2=0x7f91b4157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.257+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4157740 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f91a402b930 tx=0x7f91a40046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.259+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 shutdown_connections 2024-03-06T07:23:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.259+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f919806b4d0 0x7f919806d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.259+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b41592d0 0x7f91b41ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.259+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b40c6780 0x7f91b4157740 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.259+0000 7f91bb4e0640 1 --2- 172.21.15.26:0/3556482276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91b406da40 0x7f91b41eec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.259+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 >> 172.21.15.26:0/3556482276 conn(0x7f91b40aa630 msgr2=0x7f91b40c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.261+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 shutdown_connections 2024-03-06T07:23:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.261+0000 7f91bb4e0640 1 -- 172.21.15.26:0/3556482276 wait complete. 2024-03-06T07:23:25.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.389+0000 7fc8b881c640 1 Processor -- start 2024-03-06T07:23:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.389+0000 7fc8b881c640 1 -- start start 2024-03-06T07:23:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b881c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8b414f360 0x7fc8b4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b881c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc8b40c88c0 0x7fc8b40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b881c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b40c73d0 con 0x7fc8b40c5880 2024-03-06T07:23:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b881c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40c8dd0 con 0x7fc8b414f360 2024-03-06T07:23:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b881c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40be2e0 con 0x7fc8b40c88c0 2024-03-06T07:23:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8b414f360 0x7fc8b4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8b414f360 0x7fc8b4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43318/0 (socket says 172.21.15.26:43318) 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b259c640 1 -- 172.21.15.26:0/1211981100 learned_addr learned my addr 172.21.15.26:0/1211981100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.390+0000 7fc8b2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc8b40c5880 0x7fc8b4158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34800/0 (socket says 172.21.15.26:34800) 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.391+0000 7fc8b1d9b640 1 --2- 172.21.15.26:0/1211981100 >> v2:172.21.15.117:3300/0 conn(0x7fc8b40c88c0 0x7fc8b40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.391+0000 7fc8b259c640 1 -- 172.21.15.26:0/1211981100 >> v2:172.21.15.117:3300/0 conn(0x7fc8b40c88c0 msgr2=0x7fc8b40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.391+0000 7fc8b259c640 1 --2- 172.21.15.26:0/1211981100 >> v2:172.21.15.117:3300/0 conn(0x7fc8b40c88c0 0x7fc8b40c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.391+0000 7fc8b259c640 1 -- 172.21.15.26:0/1211981100 >> v1:172.21.15.26:6789/0 conn(0x7fc8b40c5880 legacy=0x7fc8b4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.391+0000 7fc8b259c640 1 -- 172.21.15.26:0/1211981100 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40c7670 con 0x7fc8b414f360 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.391+0000 7fc8b1d9b640 1 --2- 172.21.15.26:0/1211981100 >> v2:172.21.15.117:3300/0 conn(0x7fc8b40c88c0 0x7fc8b40c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.391+0000 7fc8b259c640 1 --2- 172.21.15.26:0/1211981100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b414f360 0x7fc8b4155c10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40c84c0 tx=0x7fc8ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a653afbfe722c54d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b159a640 1 -- 172.21.15.26:0/1211981100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac035020 con 0x7fc8b414f360 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b159a640 1 -- 172.21.15.26:0/1211981100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac031870 con 0x7fc8b414f360 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b159a640 1 -- 172.21.15.26:0/1211981100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac031c40 con 0x7fc8b414f360 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 -- 172.21.15.26:0/1211981100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b414f360 msgr2=0x7fc8b4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1211981100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b414f360 0x7fc8b4155c10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40c84c0 tx=0x7fc8ac02b910 comp rx=0 tx=0).stop 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 -- 172.21.15.26:0/1211981100 shutdown_connections 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1211981100 >> v2:172.21.15.117:3300/0 conn(0x7fc8b40c88c0 0x7fc8b40c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1211981100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b414f360 0x7fc8b4155c10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 -- 172.21.15.26:0/1211981100 >> 172.21.15.26:0/1211981100 conn(0x7fc8b40ac570 msgr2=0x7fc8b40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 -- 172.21.15.26:0/1211981100 shutdown_connections 2024-03-06T07:23:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.392+0000 7fc8b881c640 1 -- 172.21.15.26:0/1211981100 wait complete. 2024-03-06T07:23:25.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.393+0000 7fc8b881c640 1 Processor -- start 2024-03-06T07:23:25.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.393+0000 7fc8b881c640 1 -- start start 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b881c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8b40c5880 0x7fc8b41dead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b881c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b40c88c0 0x7fc8b40bf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b881c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b414f360 0x7fc8b4157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b881c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4159270 con 0x7fc8b40c88c0 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b881c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40c7c30 con 0x7fc8b414f360 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b881c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b41593f0 con 0x7fc8b40c5880 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b40c88c0 0x7fc8b40bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b40c88c0 0x7fc8b40bf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43324/0 (socket says 172.21.15.26:43324) 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b1d9b640 1 -- 172.21.15.26:0/1582527093 learned_addr learned my addr 172.21.15.26:0/1582527093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b2d9d640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b414f360 0x7fc8b4157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.394+0000 7fc8b259c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8b40c5880 0x7fc8b41dead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b1d9b640 1 -- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8b40c5880 msgr2=0x7fc8b41dead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b1d9b640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8b40c5880 0x7fc8b41dead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b1d9b640 1 -- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b414f360 msgr2=0x7fc8b4157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b1d9b640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b414f360 0x7fc8b4157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b1d9b640 1 -- 172.21.15.26:0/1582527093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b41e13a0 con 0x7fc8b40c88c0 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b2d9d640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b414f360 0x7fc8b4157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b259c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8b40c5880 0x7fc8b41dead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b1d9b640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b40c88c0 0x7fc8b40bf960 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c7f0 tx=0x7fc89c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8aaffd640 1 -- 172.21.15.26:0/1582527093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c015070 con 0x7fc8b40c88c0 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8aaffd640 1 -- 172.21.15.26:0/1582527093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c0079d0 con 0x7fc8b40c88c0 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b40b0420 con 0x7fc8b40c88c0 2024-03-06T07:23:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8aaffd640 1 -- 172.21.15.26:0/1582527093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c007da0 con 0x7fc8b40c88c0 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.395+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b40c0910 con 0x7fc8b40c88c0 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.396+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b404f330 con 0x7fc8b40c88c0 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.396+0000 7fc8aaffd640 1 -- 172.21.15.26:0/1582527093 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc89c018070 con 0x7fc8b40c88c0 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.397+0000 7fc8aaffd640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc89406b4d0 0x7fc89406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.397+0000 7fc8aaffd640 1 -- 172.21.15.26:0/1582527093 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc89c08a300 con 0x7fc8b40c88c0 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.397+0000 7fc8b259c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc89406b4d0 0x7fc89406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.398+0000 7fc8b259c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc89406b4d0 0x7fc89406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac036000 tx=0x7fc8ac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.401+0000 7fc8aaffd640 1 -- 172.21.15.26:0/1582527093 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc89c058ff0 con 0x7fc8b40c88c0 2024-03-06T07:23:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.583+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8b40b0680 con 0x7fc8b40c88c0 2024-03-06T07:23:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.584+0000 7fc8aaffd640 1 -- 172.21.15.26:0/1582527093 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16898 (secure 0 0 0) 0x7fc89c0588f0 con 0x7fc8b40c88c0 2024-03-06T07:23:25.586 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:25.586 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:24.599559+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:22.873048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.559370+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc89406b4d0 msgr2=0x7fc89406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc89406b4d0 0x7fc89406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac036000 tx=0x7fc8ac008030 comp rx=0 tx=0).stop 2024-03-06T07:23:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b40c88c0 msgr2=0x7fc8b40bf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b40c88c0 0x7fc8b40bf960 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c7f0 tx=0x7fc89c00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 shutdown_connections 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc89406b4d0 0x7fc89406d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b414f360 0x7fc8b4157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b40c88c0 0x7fc8b40bf960 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.586+0000 7fc8b881c640 1 --2- 172.21.15.26:0/1582527093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8b40c5880 0x7fc8b41dead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.587+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 >> 172.21.15.26:0/1582527093 conn(0x7fc8b40ac570 msgr2=0x7fc8b40c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.587+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 shutdown_connections 2024-03-06T07:23:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.587+0000 7fc8b881c640 1 -- 172.21.15.26:0/1582527093 wait complete. 2024-03-06T07:23:25.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.699+0000 7f7b29172640 1 Processor -- start 2024-03-06T07:23:25.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.699+0000 7f7b29172640 1 -- start start 2024-03-06T07:23:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b29172640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 0x7f7b241529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b29172640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7b240c1aa0 0x7f7b240c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b29172640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b240c28d0 con 0x7f7b240c5c60 2024-03-06T07:23:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b29172640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b240c2680 con 0x7f7b240c1aa0 2024-03-06T07:23:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b29172640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240c2bb0 con 0x7f7b2414f790 2024-03-06T07:23:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b2359e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7b240c5c60 0x7f7b24155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34812/0 (socket says 172.21.15.26:34812) 2024-03-06T07:23:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b2359e640 1 -- 172.21.15.26:0/2893546761 learned_addr learned my addr 172.21.15.26:0/2893546761 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2893546761 >> v2:172.21.15.117:3300/0 conn(0x7f7b240c1aa0 0x7f7b240c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844900122 0 0) 0x7f7b240c28d0 con 0x7f7b240c5c60 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b10003520 con 0x7f7b240c5c60 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b22d9d640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 0x7f7b241529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4159309809 0 0) 0x7f7b10003520 con 0x7f7b240c5c60 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 msgr2=0x7f7b241529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 0x7f7b241529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 >> v2:172.21.15.117:3300/0 conn(0x7f7b240c1aa0 msgr2=0x7f7b240c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 --2- 172.21.15.26:0/2893546761 >> v2:172.21.15.117:3300/0 conn(0x7f7b240c1aa0 0x7f7b240c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b240c2e90 con 0x7f7b240c5c60 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f7b140019f0 con 0x7f7b240c5c60 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 >> v1:172.21.15.26:6789/0 conn(0x7f7b240c5c60 legacy=0x7f7b24155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 0x7f7b100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b10003f30 0x7f7b10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b10004d40 0x7f7b100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240c2680 con 0x7f7b10003f30 2024-03-06T07:23:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240c2bb0 con 0x7f7b10004d40 2024-03-06T07:23:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1000dbd0 con 0x7f7b2414f790 2024-03-06T07:23:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.700+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b10003f30 0x7f7b10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b22d9d640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 0x7f7b100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b2359e640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b10004d40 0x7f7b100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b2259c640 1 -- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 msgr2=0x7f7b100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 0x7f7b100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b2259c640 1 -- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b10004d40 msgr2=0x7f7b100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b10004d40 0x7f7b100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b2259c640 1 -- 172.21.15.26:0/2893546761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b10003520 con 0x7f7b10003f30 2024-03-06T07:23:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b22d9d640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b2414f790 0x7f7b100039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.701+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b10003f30 0x7f7b10004800 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009ac0 tx=0x7f7b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0956123fc0c786 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.702+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c039070 con 0x7f7b10003f30 2024-03-06T07:23:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.702+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c02c6d0 con 0x7f7b10003f30 2024-03-06T07:23:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.702+0000 7f7b21d9b640 1 -- 172.21.15.26:0/2893546761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c009690 con 0x7f7b10003f30 2024-03-06T07:23:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.703+0000 7f7b29172640 1 -- 172.21.15.26:0/2893546761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b10003f30 msgr2=0x7f7b10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.703+0000 7f7b29172640 1 --2- 172.21.15.26:0/2893546761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b10003f30 0x7f7b10004800 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009ac0 tx=0x7f7b10005830 comp rx=0 tx=0).stop 2024-03-06T07:23:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.703+0000 7f7b2359e640 1 -- 172.21.15.26:0/2893546761 reap_dead start 2024-03-06T07:23:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.703+0000 7f7b29172640 1 -- 172.21.15.26:0/2893546761 shutdown_connections 2024-03-06T07:23:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.703+0000 7f7b29172640 1 -- 172.21.15.26:0/2893546761 >> 172.21.15.26:0/2893546761 conn(0x7f7b240c6710 msgr2=0x7f7b240c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.703+0000 7f7b29172640 1 -- 172.21.15.26:0/2893546761 shutdown_connections 2024-03-06T07:23:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.703+0000 7f7b29172640 1 -- 172.21.15.26:0/2893546761 wait complete. 2024-03-06T07:23:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.704+0000 7f7b29172640 1 Processor -- start 2024-03-06T07:23:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.704+0000 7f7b29172640 1 -- start start 2024-03-06T07:23:25.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b29172640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b10003f30 0x7f7b24142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b29172640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24154c50 0x7f7b2406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b29172640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b2414fdc0 0x7f7b241dda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b29172640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2406cc00 con 0x7f7b24154c50 2024-03-06T07:23:25.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b29172640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240c0d60 con 0x7f7b2414fdc0 2024-03-06T07:23:25.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b29172640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2406cd80 con 0x7f7b10003f30 2024-03-06T07:23:25.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24154c50 0x7f7b2406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24154c50 0x7f7b2406dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43340/0 (socket says 172.21.15.26:43340) 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 -- 172.21.15.26:0/2834981772 learned_addr learned my addr 172.21.15.26:0/2834981772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b22d9d640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b10003f30 0x7f7b24142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2359e640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b2414fdc0 0x7f7b241dda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 -- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b10003f30 msgr2=0x7f7b24142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b10003f30 0x7f7b24142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 -- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b2414fdc0 msgr2=0x7f7b241dda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b2414fdc0 0x7f7b241dda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.705+0000 7f7b2259c640 1 -- 172.21.15.26:0/2834981772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b241ea580 con 0x7f7b24154c50 2024-03-06T07:23:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.706+0000 7f7b2259c640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24154c50 0x7f7b2406dd10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009be0 tx=0x7f7b0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.706+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2834981772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c042020 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.706+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2834981772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c03a040 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.706+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b241ea860 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.706+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2834981772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c039070 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.706+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b24154450 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.707+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2404f330 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.707+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2834981772 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b0c045020 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.707+0000 7f7b1b7fe640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0406b500 0x7f7b0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.707+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2834981772 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b0c0ab6a0 con 0x7f7b24154c50 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.708+0000 7f7b22d9d640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0406b500 0x7f7b0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.708+0000 7f7b22d9d640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0406b500 0x7f7b0406d9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7b240abb10 tx=0x7f7b1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.712+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2834981772 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b0c04c070 con 0x7f7b24154c50 2024-03-06T07:23:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.892+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b240c1f40 con 0x7f7b24154c50 2024-03-06T07:23:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.893+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2834981772 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16898 (secure 0 0 0) 0x7f7b0c079d50 con 0x7f7b24154c50 2024-03-06T07:23:25.895 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:25.895 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:24.599559+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:22.873048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.559370+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0406b500 msgr2=0x7f7b0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0406b500 0x7f7b0406d9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7b240abb10 tx=0x7f7b1c008040 comp rx=0 tx=0).stop 2024-03-06T07:23:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24154c50 msgr2=0x7f7b2406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24154c50 0x7f7b2406dd10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009be0 tx=0x7f7b0c004730 comp rx=0 tx=0).stop 2024-03-06T07:23:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 shutdown_connections 2024-03-06T07:23:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0406b500 0x7f7b0406d9c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b2414fdc0 0x7f7b241dda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24154c50 0x7f7b2406dd10 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 --2- 172.21.15.26:0/2834981772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b10003f30 0x7f7b24142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 >> 172.21.15.26:0/2834981772 conn(0x7f7b240c6710 msgr2=0x7f7b240c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 shutdown_connections 2024-03-06T07:23:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:25.895+0000 7f7b29172640 1 -- 172.21.15.26:0/2834981772 wait complete. 2024-03-06T07:23:25.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:23:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.011+0000 7f16cb0b1640 1 Processor -- start 2024-03-06T07:23:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.011+0000 7f16cb0b1640 1 -- start start 2024-03-06T07:23:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.012+0000 7f16cb0b1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16c4155c30 0x7f16c406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.012+0000 7f16cb0b1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c40b0450 con 0x7f16c40c5380 2024-03-06T07:23:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.012+0000 7f16cb0b1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c41560e0 con 0x7f16c40b0070 2024-03-06T07:23:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.012+0000 7f16cb0b1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c406cfb0 con 0x7f16c4155c30 2024-03-06T07:23:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.012+0000 7f16c3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16c4155c30 0x7f16c406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.012+0000 7f16c964e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f16c40c5380 0x7f16c414e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34826/0 (socket says 172.21.15.26:34826) 2024-03-06T07:23:26.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.012+0000 7f16c964e640 1 -- 172.21.15.26:0/329942941 learned_addr learned my addr 172.21.15.26:0/329942941 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:26.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.013+0000 7f16c3fff640 1 -- 172.21.15.26:0/329942941 >> v1:172.21.15.117:6789/0 conn(0x7f16c40b0070 legacy=0x7f16c40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.013+0000 7f16c3fff640 1 -- 172.21.15.26:0/329942941 >> v1:172.21.15.26:6789/0 conn(0x7f16c40c5380 legacy=0x7f16c414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.013+0000 7f16c3fff640 1 -- 172.21.15.26:0/329942941 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c40c77f0 con 0x7f16c4155c30 2024-03-06T07:23:26.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.013+0000 7f16c3fff640 1 --2- 172.21.15.26:0/329942941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c4155c30 0x7f16c406ca70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d72157b03f8d878 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16c37fe640 1 -- 172.21.15.26:0/329942941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8035020 con 0x7f16c4155c30 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16c37fe640 1 -- 172.21.15.26:0/329942941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8031830 con 0x7f16c4155c30 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16c37fe640 1 -- 172.21.15.26:0/329942941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8031c00 con 0x7f16c4155c30 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16cb0b1640 1 -- 172.21.15.26:0/329942941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c4155c30 msgr2=0x7f16c406ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/329942941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c4155c30 0x7f16c406ca70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16cb0b1640 1 -- 172.21.15.26:0/329942941 shutdown_connections 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/329942941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c4155c30 0x7f16c406ca70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16cb0b1640 1 -- 172.21.15.26:0/329942941 >> 172.21.15.26:0/329942941 conn(0x7f16c40ac590 msgr2=0x7f16c40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16cb0b1640 1 -- 172.21.15.26:0/329942941 shutdown_connections 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.014+0000 7f16cb0b1640 1 -- 172.21.15.26:0/329942941 wait complete. 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.015+0000 7f16cb0b1640 1 Processor -- start 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.015+0000 7f16cb0b1640 1 -- start start 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16cb0b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c40b0070 0x7f16c4154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16cb0b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c40c5380 0x7f16c414ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16cb0b1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c4155c30 0x7f16c414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16cb0b1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c406c270 con 0x7f16c40c5380 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16cb0b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c406c020 con 0x7f16c40b0070 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16cb0b1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c406c550 con 0x7f16c4155c30 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c40c5380 0x7f16c414ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c40c5380 0x7f16c414ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43360/0 (socket says 172.21.15.26:43360) 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 -- 172.21.15.26:0/3188318551 learned_addr learned my addr 172.21.15.26:0/3188318551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c964e640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c4155c30 0x7f16c414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c8e4d640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c40b0070 0x7f16c4154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 -- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c4155c30 msgr2=0x7f16c414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c4155c30 0x7f16c414ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 -- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c40b0070 msgr2=0x7f16c4154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c40b0070 0x7f16c4154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c3fff640 1 -- 172.21.15.26:0/3188318551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c41eeb90 con 0x7f16c40c5380 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.016+0000 7f16c964e640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c4155c30 0x7f16c414ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.017+0000 7f16c3fff640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c40c5380 0x7f16c414ea30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009890 tx=0x7f16b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.017+0000 7f16c17fa640 1 -- 172.21.15.26:0/3188318551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8041070 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.017+0000 7f16c17fa640 1 -- 172.21.15.26:0/3188318551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b803aa30 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.017+0000 7f16c17fa640 1 -- 172.21.15.26:0/3188318551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8035040 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.017+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c414f700 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.017+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c4155150 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.018+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c404f330 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.018+0000 7f16c17fa640 1 -- 172.21.15.26:0/3188318551 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16b8038020 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.019+0000 7f16c17fa640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16b006b4d0 0x7f16b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.019+0000 7f16c17fa640 1 -- 172.21.15.26:0/3188318551 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16b80abd00 con 0x7f16c40c5380 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.019+0000 7f16c8e4d640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16b006b4d0 0x7f16b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.019+0000 7f16c8e4d640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16b006b4d0 0x7f16b006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f16bc001ca0 tx=0x7f16bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.023+0000 7f16c17fa640 1 -- 172.21.15.26:0/3188318551 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f16b807aa80 con 0x7f16c40c5380 2024-03-06T07:23:26.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.249+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f16c406d860 con 0x7f16c40c5380 2024-03-06T07:23:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.256+0000 7f16c17fa640 1 -- 172.21.15.26:0/3188318551 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f16b807a380 con 0x7f16c40c5380 2024-03-06T07:23:26.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16b006b4d0 msgr2=0x7f16b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16b006b4d0 0x7f16b006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f16bc001ca0 tx=0x7f16bc009040 comp rx=0 tx=0).stop 2024-03-06T07:23:26.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c40c5380 msgr2=0x7f16c414ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c40c5380 0x7f16c414ea30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009890 tx=0x7f16b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 shutdown_connections 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16b006b4d0 0x7f16b006d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c4155c30 0x7f16c414ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c40c5380 0x7f16c414ea30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.258+0000 7f16cb0b1640 1 --2- 172.21.15.26:0/3188318551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c40b0070 0x7f16c4154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.259+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 >> 172.21.15.26:0/3188318551 conn(0x7f16c40ac590 msgr2=0x7f16c40ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.259+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 shutdown_connections 2024-03-06T07:23:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.259+0000 7f16cb0b1640 1 -- 172.21.15.26:0/3188318551 wait complete. 2024-03-06T07:23:26.272 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:23:26.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:23:26.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.375+0000 7fe7d81f7640 1 Processor -- start 2024-03-06T07:23:26.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d81f7640 1 -- start start 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d81f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7d00b0070 0x7fe7d0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d81f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d006ca70 0x7fe7d00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d81f7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d00b0450 con 0x7fe7d00c5380 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d81f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d006c020 con 0x7fe7d00b0070 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d81f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d006cf20 con 0x7fe7d006ca70 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d6794640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe7d00c5380 0x7fe7d014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34832/0 (socket says 172.21.15.26:34832) 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d6794640 1 -- 172.21.15.26:0/533317848 learned_addr learned my addr 172.21.15.26:0/533317848 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d5792640 1 --2- 172.21.15.26:0/533317848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d006ca70 0x7fe7d00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.376+0000 7fe7d5f93640 1 --2- 172.21.15.26:0/533317848 >> v2:172.21.15.117:3300/0 conn(0x7fe7d00b0070 0x7fe7d0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.377+0000 7fe7d4f91640 1 -- 172.21.15.26:0/533317848 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097874463 0 0) 0x7fe7c8001490 con 0x7fe7d00c5380 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.377+0000 7fe7d4f91640 1 -- 172.21.15.26:0/533317848 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc003520 con 0x7fe7d00c5380 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.377+0000 7fe7d5792640 1 -- 172.21.15.26:0/533317848 >> v2:172.21.15.117:3300/0 conn(0x7fe7d00b0070 msgr2=0x7fe7d0155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.377+0000 7fe7d5792640 1 --2- 172.21.15.26:0/533317848 >> v2:172.21.15.117:3300/0 conn(0x7fe7d00b0070 0x7fe7d0155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.377+0000 7fe7d5792640 1 -- 172.21.15.26:0/533317848 >> v1:172.21.15.26:6789/0 conn(0x7fe7d00c5380 legacy=0x7fe7d014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.377+0000 7fe7d5792640 1 -- 172.21.15.26:0/533317848 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d00c77f0 con 0x7fe7d006ca70 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d5792640 1 --2- 172.21.15.26:0/533317848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d006ca70 0x7fe7d00c8900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0097a0 tx=0x7fe7cc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01800b62f50186d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d4f91640 1 -- 172.21.15.26:0/533317848 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0315a0 con 0x7fe7d006ca70 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d4f91640 1 -- 172.21.15.26:0/533317848 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc037070 con 0x7fe7d006ca70 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d4f91640 1 -- 172.21.15.26:0/533317848 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc031940 con 0x7fe7d006ca70 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 -- 172.21.15.26:0/533317848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d006ca70 msgr2=0x7fe7d00c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/533317848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d006ca70 0x7fe7d00c8900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0097a0 tx=0x7fe7cc02b840 comp rx=0 tx=0).stop 2024-03-06T07:23:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 -- 172.21.15.26:0/533317848 shutdown_connections 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/533317848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d006ca70 0x7fe7d00c8900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/533317848 >> v2:172.21.15.117:3300/0 conn(0x7fe7d00b0070 0x7fe7d0155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 -- 172.21.15.26:0/533317848 >> 172.21.15.26:0/533317848 conn(0x7fe7d00ac590 msgr2=0x7fe7d00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 -- 172.21.15.26:0/533317848 shutdown_connections 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.378+0000 7fe7d81f7640 1 -- 172.21.15.26:0/533317848 wait complete. 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.379+0000 7fe7d81f7640 1 Processor -- start 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.379+0000 7fe7d81f7640 1 -- start start 2024-03-06T07:23:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d81f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7d006ca70 0x7fe7d00c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d81f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d00b0070 0x7fe7d014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d81f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d00c5380 0x7fe7d0157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d81f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d006c550 con 0x7fe7d006ca70 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d81f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d006c270 con 0x7fe7d00b0070 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d81f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d006da80 con 0x7fe7d00c5380 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d6794640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d00c5380 0x7fe7d0157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d6794640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d00c5380 0x7fe7d0157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:59758/0 (socket says 172.21.15.26:59758) 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d6794640 1 -- 172.21.15.26:0/1863834650 learned_addr learned my addr 172.21.15.26:0/1863834650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.380+0000 7fe7d5f93640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7d006ca70 0x7fe7d00c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.381+0000 7fe7d5792640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d00b0070 0x7fe7d014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.381+0000 7fe7d6794640 1 -- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d00b0070 msgr2=0x7fe7d014e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.381+0000 7fe7d6794640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d00b0070 0x7fe7d014e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.381+0000 7fe7d6794640 1 -- 172.21.15.26:0/1863834650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7d006ca70 msgr2=0x7fe7d00c14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.381+0000 7fe7d6794640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7d006ca70 0x7fe7d00c14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.381+0000 7fe7d6794640 1 -- 172.21.15.26:0/1863834650 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d01eed70 con 0x7fe7d00c5380 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.381+0000 7fe7d6794640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d00c5380 0x7fe7d0157f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800cb50 tx=0x7fe7c800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.382+0000 7fe7c27fc640 1 -- 172.21.15.26:0/1863834650 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8017070 con 0x7fe7d00c5380 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.382+0000 7fe7c27fc640 1 -- 172.21.15.26:0/1863834650 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c800ad40 con 0x7fe7d00c5380 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.382+0000 7fe7c27fc640 1 -- 172.21.15.26:0/1863834650 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8012620 con 0x7fe7d00c5380 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.382+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d01ef050 con 0x7fe7d00c5380 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.382+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d00c4bd0 con 0x7fe7d00c5380 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.383+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d00c85d0 con 0x7fe7d00c5380 2024-03-06T07:23:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.384+0000 7fe7c27fc640 1 -- 172.21.15.26:0/1863834650 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7c801a070 con 0x7fe7d00c5380 2024-03-06T07:23:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.385+0000 7fe7c27fc640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7b406b520 0x7fe7b406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.385+0000 7fe7c27fc640 1 -- 172.21.15.26:0/1863834650 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7c808a570 con 0x7fe7d00c5380 2024-03-06T07:23:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.385+0000 7fe7d5f93640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7b406b520 0x7fe7b406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.385+0000 7fe7d5f93640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7b406b520 0x7fe7b406d9e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0157e60 tx=0x7fe7c4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.393+0000 7fe7c27fc640 1 -- 172.21.15.26:0/1863834650 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe7d00c85d0 con 0x7fe7d00c5380 2024-03-06T07:23:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:26.616+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe7d00c46e0 con 0x7fe7d00c5380 2024-03-06T07:23:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.264+0000 7fe7c27fc640 1 -- 172.21.15.26:0/1863834650 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7c8021020 con 0x7fe7d00c5380 2024-03-06T07:23:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7b406b520 msgr2=0x7fe7b406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7b406b520 0x7fe7b406d9e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0157e60 tx=0x7fe7c4009190 comp rx=0 tx=0).stop 2024-03-06T07:23:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d00c5380 msgr2=0x7fe7d0157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d00c5380 0x7fe7d0157f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800cb50 tx=0x7fe7c800a430 comp rx=0 tx=0).stop 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 shutdown_connections 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7b406b520 0x7fe7b406d9e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7d00c5380 0x7fe7d0157f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d00b0070 0x7fe7d014e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 --2- 172.21.15.26:0/1863834650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7d006ca70 0x7fe7d00c14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 >> 172.21.15.26:0/1863834650 conn(0x7fe7d00ac590 msgr2=0x7fe7d0155890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.266+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 shutdown_connections 2024-03-06T07:23:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.267+0000 7fe7d81f7640 1 -- 172.21.15.26:0/1863834650 wait complete. 2024-03-06T07:23:27.281 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:23:27.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.386+0000 7f1f69e9d640 1 Processor -- start 2024-03-06T07:23:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.386+0000 7f1f69e9d640 1 -- start start 2024-03-06T07:23:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f69e9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f640c3280 0x7f1f640c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f69e9d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f64158800 con 0x7f1f6406d9f0 2024-03-06T07:23:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f69e9d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f64158ae0 con 0x7f1f641551e0 2024-03-06T07:23:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f69e9d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f64158dc0 con 0x7f1f640c3280 2024-03-06T07:23:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f637fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f6406d9f0 0x7f1f640a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34818/0 (socket says 172.21.15.26:34818) 2024-03-06T07:23:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f637fe640 1 -- 172.21.15.26:0/3956704284 learned_addr learned my addr 172.21.15.26:0/3956704284 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f62ffd640 1 --2- 172.21.15.26:0/3956704284 >> v2:172.21.15.117:3300/0 conn(0x7f1f640c3280 0x7f1f640c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f627fc640 1 -- 172.21.15.26:0/3956704284 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676368663 0 0) 0x7f1f64158800 con 0x7f1f6406d9f0 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f627fc640 1 -- 172.21.15.26:0/3956704284 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f4c003520 con 0x7f1f6406d9f0 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f627fc640 1 -- 172.21.15.26:0/3956704284 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187453279 0 0) 0x7f1f64158ae0 con 0x7f1f641551e0 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f627fc640 1 -- 172.21.15.26:0/3956704284 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f64158800 con 0x7f1f641551e0 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f62ffd640 1 -- 172.21.15.26:0/3956704284 >> v1:172.21.15.117:6789/0 conn(0x7f1f641551e0 legacy=0x7f1f641555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f62ffd640 1 -- 172.21.15.26:0/3956704284 >> v1:172.21.15.26:6789/0 conn(0x7f1f6406d9f0 legacy=0x7f1f640a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.387+0000 7f1f62ffd640 1 -- 172.21.15.26:0/3956704284 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f641590a0 con 0x7f1f640c3280 2024-03-06T07:23:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.388+0000 7f1f62ffd640 1 --2- 172.21.15.26:0/3956704284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f640c3710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1f540097a0 tx=0x7f1f5402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a3a6414965837 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.388+0000 7f1f627fc640 1 -- 172.21.15.26:0/3956704284 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54035020 con 0x7f1f640c3280 2024-03-06T07:23:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.388+0000 7f1f627fc640 1 -- 172.21.15.26:0/3956704284 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f54031830 con 0x7f1f640c3280 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.388+0000 7f1f627fc640 1 -- 172.21.15.26:0/3956704284 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54031c00 con 0x7f1f640c3280 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.389+0000 7f1f69e9d640 1 -- 172.21.15.26:0/3956704284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 msgr2=0x7f1f640c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.389+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/3956704284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f640c3710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1f540097a0 tx=0x7f1f5402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.389+0000 7f1f69e9d640 1 -- 172.21.15.26:0/3956704284 shutdown_connections 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.389+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/3956704284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f640c3710 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.389+0000 7f1f69e9d640 1 -- 172.21.15.26:0/3956704284 >> 172.21.15.26:0/3956704284 conn(0x7f1f640aa630 msgr2=0x7f1f640aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.389+0000 7f1f69e9d640 1 -- 172.21.15.26:0/3956704284 shutdown_connections 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.389+0000 7f1f69e9d640 1 -- 172.21.15.26:0/3956704284 wait complete. 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.390+0000 7f1f69e9d640 1 Processor -- start 2024-03-06T07:23:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.390+0000 7f1f69e9d640 1 -- start start 2024-03-06T07:23:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f69e9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f6406d9f0 0x7f1f641ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f69e9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f6406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f69e9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f641551e0 0x7f1f6406f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f69e9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6406df50 con 0x7f1f6406d9f0 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f69e9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6406ddd0 con 0x7f1f640c3280 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f69e9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c6910 con 0x7f1f641551e0 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f6406d9f0 0x7f1f641ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f6406d9f0 0x7f1f641ee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49418/0 (socket says 172.21.15.26:49418) 2024-03-06T07:23:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 -- 172.21.15.26:0/2786243383 learned_addr learned my addr 172.21.15.26:0/2786243383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f62ffd640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f6406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f63fff640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f641551e0 0x7f1f6406f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 -- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f641551e0 msgr2=0x7f1f6406f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f641551e0 0x7f1f6406f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 -- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 msgr2=0x7f1f6406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f6406eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 -- 172.21.15.26:0/2786243383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6406d100 con 0x7f1f6406d9f0 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f63fff640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f641551e0 0x7f1f6406f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f62ffd640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f6406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f637fe640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f6406d9f0 0x7f1f641ee2f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c500 tx=0x7f1f5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f4bfff640 1 -- 172.21.15.26:0/2786243383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58017070 con 0x7f1f6406d9f0 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.391+0000 7f1f4bfff640 1 -- 172.21.15.26:0/2786243383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58005ce0 con 0x7f1f6406d9f0 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.392+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f641eeb30 con 0x7f1f6406d9f0 2024-03-06T07:23:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.392+0000 7f1f4bfff640 1 -- 172.21.15.26:0/2786243383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580072f0 con 0x7f1f6406d9f0 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.392+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f641ee830 con 0x7f1f6406d9f0 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.392+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6404f330 con 0x7f1f6406d9f0 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.393+0000 7f1f4bfff640 1 -- 172.21.15.26:0/2786243383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f5801a070 con 0x7f1f6406d9f0 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.393+0000 7f1f4bfff640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f5006b4d0 0x7f1f5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.393+0000 7f1f4bfff640 1 -- 172.21.15.26:0/2786243383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f58089680 con 0x7f1f6406d9f0 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.393+0000 7f1f62ffd640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f5006b4d0 0x7f1f5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.394+0000 7f1f62ffd640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f5006b4d0 0x7f1f5006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1f54036000 tx=0x7f1f54008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.398+0000 7f1f4bfff640 1 -- 172.21.15.26:0/2786243383 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f58057e70 con 0x7f1f6406d9f0 2024-03-06T07:23:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.615+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f640aec10 con 0x7f1f6406d9f0 2024-03-06T07:23:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.616+0000 7f1f4bfff640 1 -- 172.21.15.26:0/2786243383 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f1f5805bf10 con 0x7f1f6406d9f0 2024-03-06T07:23:27.618 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:26.618139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1940022472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1940022472},{"type":"v1","addr":"172.21.15.117:6837","nonce":1940022472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6966,"qdb_cluster":[6966]},"id":7}]} 2024-03-06T07:23:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 77 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f5006b4d0 msgr2=0x7f1f5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f5006b4d0 0x7f1f5006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1f54036000 tx=0x7f1f54008030 comp rx=0 tx=0).stop 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f6406d9f0 msgr2=0x7f1f641ee2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f6406d9f0 0x7f1f641ee2f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c500 tx=0x7f1f5800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 shutdown_connections 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f5006b4d0 0x7f1f5006d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f641551e0 0x7f1f6406f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f640c3280 0x7f1f6406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 --2- 172.21.15.26:0/2786243383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f6406d9f0 0x7f1f641ee2f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 >> 172.21.15.26:0/2786243383 conn(0x7f1f640aa630 msgr2=0x7f1f64154880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 shutdown_connections 2024-03-06T07:23:27.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.618+0000 7f1f69e9d640 1 -- 172.21.15.26:0/2786243383 wait complete. 2024-03-06T07:23:27.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.731+0000 7f875e97a640 1 Processor -- start 2024-03-06T07:23:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.731+0000 7f875e97a640 1 -- start start 2024-03-06T07:23:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.731+0000 7f875e97a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87581508c0 0x7f87580c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.731+0000 7f875e97a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8758157340 con 0x7f8758072720 2024-03-06T07:23:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f875e97a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8758156d80 con 0x7f875806f7b0 2024-03-06T07:23:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f875e97a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8758156b40 con 0x7f87581508c0 2024-03-06T07:23:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f8757fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8758072720 0x7f8758156430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34822/0 (socket says 172.21.15.26:34822) 2024-03-06T07:23:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f8757fff640 1 -- 172.21.15.26:0/3049852685 learned_addr learned my addr 172.21.15.26:0/3049852685 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f875cf17640 1 --2- 172.21.15.26:0/3049852685 >> v2:172.21.15.117:3300/0 conn(0x7f87581508c0 0x7f87580c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f8756ffd640 1 -- 172.21.15.26:0/3049852685 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154753465 0 0) 0x7f8758157340 con 0x7f8758072720 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f8756ffd640 1 -- 172.21.15.26:0/3049852685 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8744003520 con 0x7f8758072720 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f875cf17640 1 -- 172.21.15.26:0/3049852685 >> v1:172.21.15.117:6789/0 conn(0x7f875806f7b0 legacy=0x7f87581501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f875cf17640 1 -- 172.21.15.26:0/3049852685 >> v1:172.21.15.26:6789/0 conn(0x7f8758072720 legacy=0x7f8758156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f8756ffd640 1 -- 172.21.15.26:0/3049852685 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516573257 0 0) 0x7f8758156d80 con 0x7f875806f7b0 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.732+0000 7f875cf17640 1 -- 172.21.15.26:0/3049852685 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758157060 con 0x7f87581508c0 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f875cf17640 1 --2- 172.21.15.26:0/3049852685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87581508c0 0x7f87580c3bd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8750009d90 tx=0x7f875002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdccc67826c4dece server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f8756ffd640 1 -- 172.21.15.26:0/3049852685 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750039070 con 0x7f87581508c0 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f8756ffd640 1 -- 172.21.15.26:0/3049852685 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875002cc60 con 0x7f87581508c0 2024-03-06T07:23:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f8756ffd640 1 -- 172.21.15.26:0/3049852685 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750034660 con 0x7f87581508c0 2024-03-06T07:23:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f875e97a640 1 -- 172.21.15.26:0/3049852685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87581508c0 msgr2=0x7f87580c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f875e97a640 1 --2- 172.21.15.26:0/3049852685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87581508c0 0x7f87580c3bd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8750009d90 tx=0x7f875002beb0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f875e97a640 1 -- 172.21.15.26:0/3049852685 shutdown_connections 2024-03-06T07:23:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f875e97a640 1 --2- 172.21.15.26:0/3049852685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87581508c0 0x7f87580c3bd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.733+0000 7f875e97a640 1 -- 172.21.15.26:0/3049852685 >> 172.21.15.26:0/3049852685 conn(0x7f87580aa630 msgr2=0x7f87580aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.734+0000 7f875e97a640 1 -- 172.21.15.26:0/3049852685 shutdown_connections 2024-03-06T07:23:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.734+0000 7f875e97a640 1 -- 172.21.15.26:0/3049852685 wait complete. 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 Processor -- start 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 -- start start 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f875806f7b0 0x7f87580adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8758072720 0x7f87580c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87581508c0 0x7f87580bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758072b00 con 0x7f875806f7b0 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87580c3740 con 0x7f8758072720 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.735+0000 7f875e97a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8758072c80 con 0x7f87581508c0 2024-03-06T07:23:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f875806f7b0 0x7f87580adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f875806f7b0 0x7f87580adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49430/0 (socket says 172.21.15.26:49430) 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 -- 172.21.15.26:0/2926445442 learned_addr learned my addr 172.21.15.26:0/2926445442 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f875cf17640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87581508c0 0x7f87580bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f87577fe640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8758072720 0x7f87580c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 -- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87581508c0 msgr2=0x7f87580bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87581508c0 0x7f87580bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 -- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8758072720 msgr2=0x7f87580c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8758072720 0x7f87580c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 -- 172.21.15.26:0/2926445442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87580c19a0 con 0x7f875806f7b0 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8757fff640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f875806f7b0 0x7f87580adf10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f874000c460 tx=0x7f874000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8754ff9640 1 -- 172.21.15.26:0/2926445442 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740017070 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8754ff9640 1 -- 172.21.15.26:0/2926445442 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740005ce0 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87580c16c0 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f8754ff9640 1 -- 172.21.15.26:0/2926445442 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740007290 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.736+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87580c26e0 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.737+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87580c4ff0 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.738+0000 7f8754ff9640 1 -- 172.21.15.26:0/2926445442 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f874001a070 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.738+0000 7f8754ff9640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f873806b4d0 0x7f873806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.738+0000 7f8754ff9640 1 -- 172.21.15.26:0/2926445442 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87400896b0 con 0x7f875806f7b0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.738+0000 7f87577fe640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f873806b4d0 0x7f873806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.739+0000 7f87577fe640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f873806b4d0 0x7f873806d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87580c87a0 tx=0x7f8748009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.742+0000 7f8754ff9640 1 -- 172.21.15.26:0/2926445442 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f87580c4ff0 con 0x7f875806f7b0 2024-03-06T07:23:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.924+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f875804f330 con 0x7f875806f7b0 2024-03-06T07:23:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.925+0000 7f8754ff9640 1 -- 172.21.15.26:0/2926445442 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f8740057cc0 con 0x7f875806f7b0 2024-03-06T07:23:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:27.927 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":95,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:26.240830+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:22.873048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.559370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f873806b4d0 msgr2=0x7f873806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f873806b4d0 0x7f873806d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87580c87a0 tx=0x7f8748009040 comp rx=0 tx=0).stop 2024-03-06T07:23:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f875806f7b0 msgr2=0x7f87580adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f875806f7b0 0x7f87580adf10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f874000c460 tx=0x7f874000c920 comp rx=0 tx=0).stop 2024-03-06T07:23:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 shutdown_connections 2024-03-06T07:23:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f873806b4d0 0x7f873806d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87581508c0 0x7f87580bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8758072720 0x7f87580c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 --2- 172.21.15.26:0/2926445442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f875806f7b0 0x7f87580adf10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 >> 172.21.15.26:0/2926445442 conn(0x7f87580aa630 msgr2=0x7f875814f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 shutdown_connections 2024-03-06T07:23:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:27.927+0000 7f875e97a640 1 -- 172.21.15.26:0/2926445442 wait complete. 2024-03-06T07:23:27.940 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:23:27.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:28.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.042+0000 7fae476df640 1 Processor -- start 2024-03-06T07:23:28.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.042+0000 7fae476df640 1 -- start start 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae476df640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae400c3280 0x7fae400c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae476df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae401551e0 0x7fae40155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae476df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae40158800 con 0x7fae4006d9f0 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae476df640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae40158ae0 con 0x7fae400c3280 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae476df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae40158dc0 con 0x7fae401551e0 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae4547b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae4006d9f0 0x7fae400a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34828/0 (socket says 172.21.15.26:34828) 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae4547b640 1 -- 172.21.15.26:0/2652281630 learned_addr learned my addr 172.21.15.26:0/2652281630 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae45c7c640 1 --2- 172.21.15.26:0/2652281630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae401551e0 0x7fae40155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae44c7a640 1 --2- 172.21.15.26:0/2652281630 >> v2:172.21.15.117:3300/0 conn(0x7fae400c3280 0x7fae400c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae37fff640 1 -- 172.21.15.26:0/2652281630 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270714172 0 0) 0x7fae40158800 con 0x7fae4006d9f0 2024-03-06T07:23:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae37fff640 1 -- 172.21.15.26:0/2652281630 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae2c003520 con 0x7fae4006d9f0 2024-03-06T07:23:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae45c7c640 1 -- 172.21.15.26:0/2652281630 >> v2:172.21.15.117:3300/0 conn(0x7fae400c3280 msgr2=0x7fae400c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae45c7c640 1 --2- 172.21.15.26:0/2652281630 >> v2:172.21.15.117:3300/0 conn(0x7fae400c3280 0x7fae400c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae45c7c640 1 -- 172.21.15.26:0/2652281630 >> v1:172.21.15.26:6789/0 conn(0x7fae4006d9f0 legacy=0x7fae400a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.043+0000 7fae45c7c640 1 -- 172.21.15.26:0/2652281630 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae401590a0 con 0x7fae401551e0 2024-03-06T07:23:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.044+0000 7fae45c7c640 1 --2- 172.21.15.26:0/2652281630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae401551e0 0x7fae40155630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d90 tx=0x7fae3802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa8f7860b847f1a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.044+0000 7fae37fff640 1 -- 172.21.15.26:0/2652281630 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38039070 con 0x7fae401551e0 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.044+0000 7fae37fff640 1 -- 172.21.15.26:0/2652281630 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3802cbc0 con 0x7fae401551e0 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.044+0000 7fae37fff640 1 -- 172.21.15.26:0/2652281630 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38034440 con 0x7fae401551e0 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 -- 172.21.15.26:0/2652281630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae401551e0 msgr2=0x7fae40155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 --2- 172.21.15.26:0/2652281630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae401551e0 0x7fae40155630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d90 tx=0x7fae3802be10 comp rx=0 tx=0).stop 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 -- 172.21.15.26:0/2652281630 shutdown_connections 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 --2- 172.21.15.26:0/2652281630 >> v2:172.21.15.117:3300/0 conn(0x7fae400c3280 0x7fae400c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 --2- 172.21.15.26:0/2652281630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae401551e0 0x7fae40155630 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 -- 172.21.15.26:0/2652281630 >> 172.21.15.26:0/2652281630 conn(0x7fae400aa630 msgr2=0x7fae400aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 -- 172.21.15.26:0/2652281630 shutdown_connections 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.045+0000 7fae476df640 1 -- 172.21.15.26:0/2652281630 wait complete. 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.046+0000 7fae476df640 1 Processor -- start 2024-03-06T07:23:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.046+0000 7fae476df640 1 -- start start 2024-03-06T07:23:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae476df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4006d9f0 0x7fae4014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae476df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae400c3280 0x7fae401eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae476df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae401551e0 0x7fae401ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae476df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4006df50 con 0x7fae401551e0 2024-03-06T07:23:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae476df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4006ddd0 con 0x7fae4006d9f0 2024-03-06T07:23:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae476df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c6910 con 0x7fae400c3280 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae401551e0 0x7fae401ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae401551e0 0x7fae401ee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49436/0 (socket says 172.21.15.26:49436) 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 -- 172.21.15.26:0/2837474310 learned_addr learned my addr 172.21.15.26:0/2837474310 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae44c7a640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae400c3280 0x7fae401eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae4547b640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4006d9f0 0x7fae4014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 -- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae400c3280 msgr2=0x7fae401eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae400c3280 0x7fae401eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 -- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4006d9f0 msgr2=0x7fae4014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4006d9f0 0x7fae4014e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 -- 172.21.15.26:0/2837474310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4014e900 con 0x7fae401551e0 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae45c7c640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae401551e0 0x7fae401ee300 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d90 tx=0x7fae38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae35ffb640 1 -- 172.21.15.26:0/2837474310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38039050 con 0x7fae401551e0 2024-03-06T07:23:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae35ffb640 1 -- 172.21.15.26:0/2837474310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38045070 con 0x7fae401551e0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae401ee900 con 0x7fae401551e0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.047+0000 7fae35ffb640 1 -- 172.21.15.26:0/2837474310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38009450 con 0x7fae401551e0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.048+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae400ab6c0 con 0x7fae401551e0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.049+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae400c43c0 con 0x7fae401551e0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.049+0000 7fae35ffb640 1 -- 172.21.15.26:0/2837474310 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae3803e850 con 0x7fae401551e0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.049+0000 7fae35ffb640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae2406b500 0x7fae2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.049+0000 7fae35ffb640 1 -- 172.21.15.26:0/2837474310 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae380ac780 con 0x7fae401551e0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.049+0000 7fae4547b640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae2406b500 0x7fae2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.050+0000 7fae4547b640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae2406b500 0x7fae2406d9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae3c005880 tx=0x7fae3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.053+0000 7fae35ffb640 1 -- 172.21.15.26:0/2837474310 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae400c43c0 con 0x7fae401551e0 2024-03-06T07:23:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.231+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae4004f330 con 0x7fae2406b500 2024-03-06T07:23:28.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.237+0000 7fae35ffb640 1 -- 172.21.15.26:0/2837474310 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246843 (secure 0 0 0) 0x7fae4004f330 con 0x7fae2406b500 2024-03-06T07:23:28.241 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:28.243 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:23:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae2406b500 msgr2=0x7fae2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae2406b500 0x7fae2406d9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae3c005880 tx=0x7fae3c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae401551e0 msgr2=0x7fae401ee300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae401551e0 0x7fae401ee300 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d90 tx=0x7fae38004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 shutdown_connections 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae2406b500 0x7fae2406d9c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae401551e0 0x7fae401ee300 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.239+0000 7fae476df640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae400c3280 0x7fae401eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.240+0000 7fae476df640 1 --2- 172.21.15.26:0/2837474310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4006d9f0 0x7fae4014e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.240+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 >> 172.21.15.26:0/2837474310 conn(0x7fae400aa630 msgr2=0x7fae40154860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.240+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 shutdown_connections 2024-03-06T07:23:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.240+0000 7fae476df640 1 -- 172.21.15.26:0/2837474310 wait complete. 2024-03-06T07:23:28.252 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-06T07:23:27.072796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250296,"kb_used_data":9140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724424,"statfs":{"total":773094113280,"available":772837810176,"internally_reserved":0,"allocated":9359360,"data_stored":5619685,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000943"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236216+0000","last_change":"2024-03-06T07:23:24.236216+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236216+0000","last_undegraded":"2024-03-06T07:23:24.236216+0000","last_fullsized":"2024-03-06T07:23:24.236216+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237324+0000","last_change":"2024-03-06T07:23:24.237324+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237324+0000","last_undegraded":"2024-03-06T07:23:24.237324+0000","last_fullsized":"2024-03-06T07:23:24.237324+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235853+0000","last_change":"2024-03-06T07:23:24.235853+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235853+0000","last_undegraded":"2024-03-06T07:23:24.235853+0000","last_fullsized":"2024-03-06T07:23:24.235853+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234332+0000","last_change":"2024-03-06T07:23:24.234332+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234332+0000","last_undegraded":"2024-03-06T07:23:24.234332+0000","last_fullsized":"2024-03-06T07:23:24.234332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233017+0000","last_change":"2024-03-06T07:23:24.233017+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233017+0000","last_undegraded":"2024-03-06T07:23:24.233017+0000","last_fullsized":"2024-03-06T07:23:24.233017+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236616+0000","last_change":"2024-03-06T07:23:24.236616+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236616+0000","last_undegraded":"2024-03-06T07:23:24.236616+0000","last_fullsized":"2024-03-06T07:23:24.236616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238220+0000","last_change":"2024-03-06T07:23:24.238220+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238220+0000","last_undegraded":"2024-03-06T07:23:24.238220+0000","last_fullsized":"2024-03-06T07:23:24.238220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235822+0000","last_change":"2024-03-06T07:23:24.235822+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235822+0000","last_undegraded":"2024-03-06T07:23:24.235822+0000","last_fullsized":"2024-03-06T07:23:24.235822+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236313+0000","last_change":"2024-03-06T07:23:24.236313+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236313+0000","last_undegraded":"2024-03-06T07:23:24.236313+0000","last_fullsized":"2024-03-06T07:23:24.236313+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232154+0000","last_change":"2024-03-06T07:23:24.232154+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232154+0000","last_undegraded":"2024-03-06T07:23:24.232154+0000","last_fullsized":"2024-03-06T07:23:24.232154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242162+0000","last_change":"2024-03-06T07:23:24.242162+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242162+0000","last_undegraded":"2024-03-06T07:23:24.242162+0000","last_fullsized":"2024-03-06T07:23:24.242162+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234780+0000","last_change":"2024-03-06T07:23:24.234780+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234780+0000","last_undegraded":"2024-03-06T07:23:24.234780+0000","last_fullsized":"2024-03-06T07:23:24.234780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236601+0000","last_change":"2024-03-06T07:23:24.236601+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236601+0000","last_undegraded":"2024-03-06T07:23:24.236601+0000","last_fullsized":"2024-03-06T07:23:24.236601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232054+0000","last_change":"2024-03-06T07:23:24.232054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232054+0000","last_undegraded":"2024-03-06T07:23:24.232054+0000","last_fullsized":"2024-03-06T07:23:24.232054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231907+0000","last_change":"2024-03-06T07:23:24.231907+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231907+0000","last_undegraded":"2024-03-06T07:23:24.231907+0000","last_fullsized":"2024-03-06T07:23:24.231907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231901+0000","last_change":"2024-03-06T07:23:24.231901+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231901+0000","last_undegraded":"2024-03-06T07:23:24.231901+0000","last_fullsized":"2024-03-06T07:23:24.231901+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235823+0000","last_change":"2024-03-06T07:23:24.235823+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235823+0000","last_undegraded":"2024-03-06T07:23:24.235823+0000","last_fullsized":"2024-03-06T07:23:24.235823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236857+0000","last_change":"2024-03-06T07:23:24.236857+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236857+0000","last_undegraded":"2024-03-06T07:23:24.236857+0000","last_fullsized":"2024-03-06T07:23:24.236857+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235442+0000","last_change":"2024-03-06T07:23:24.235442+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235442+0000","last_undegraded":"2024-03-06T07:23:24.235442+0000","last_fullsized":"2024-03-06T07:23:24.235442+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234283+0000","last_change":"2024-03-06T07:23:24.234283+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234283+0000","last_undegraded":"2024-03-06T07:23:24.234283+0000","last_fullsized":"2024-03-06T07:23:24.234283+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234978+0000","last_change":"2024-03-06T07:23:24.234978+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234978+0000","last_undegraded":"2024-03-06T07:23:24.234978+0000","last_fullsized":"2024-03-06T07:23:24.234978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232196+0000","last_change":"2024-03-06T07:23:24.232196+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232196+0000","last_undegraded":"2024-03-06T07:23:24.232196+0000","last_fullsized":"2024-03-06T07:23:24.232196+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236484+0000","last_change":"2024-03-06T07:23:24.236484+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236484+0000","last_undegraded":"2024-03-06T07:23:24.236484+0000","last_fullsized":"2024-03-06T07:23:24.236484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231241+0000","last_change":"2024-03-06T07:23:24.231241+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231241+0000","last_undegraded":"2024-03-06T07:23:24.231241+0000","last_fullsized":"2024-03-06T07:23:24.231241+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233724+0000","last_change":"2024-03-06T07:23:24.233724+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233724+0000","last_undegraded":"2024-03-06T07:23:24.233724+0000","last_fullsized":"2024-03-06T07:23:24.233724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233286+0000","last_change":"2024-03-06T07:23:24.233286+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233286+0000","last_undegraded":"2024-03-06T07:23:24.233286+0000","last_fullsized":"2024-03-06T07:23:24.233286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.241335+0000","last_change":"2024-03-06T07:23:24.241335+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.241335+0000","last_undegraded":"2024-03-06T07:23:24.241335+0000","last_fullsized":"2024-03-06T07:23:24.241335+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238590+0000","last_change":"2024-03-06T07:23:24.238590+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238590+0000","last_undegraded":"2024-03-06T07:23:24.238590+0000","last_fullsized":"2024-03-06T07:23:24.238590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234554+0000","last_change":"2024-03-06T07:23:24.234554+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234554+0000","last_undegraded":"2024-03-06T07:23:24.234554+0000","last_fullsized":"2024-03-06T07:23:24.234554+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232453+0000","last_change":"2024-03-06T07:23:24.232453+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232453+0000","last_undegraded":"2024-03-06T07:23:24.232453+0000","last_fullsized":"2024-03-06T07:23:24.232453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237717+0000","last_change":"2024-03-06T07:23:24.237717+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237717+0000","last_undegraded":"2024-03-06T07:23:24.237717+0000","last_fullsized":"2024-03-06T07:23:24.237717+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235413+0000","last_change":"2024-03-06T07:23:24.235413+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235413+0000","last_undegraded":"2024-03-06T07:23:24.235413+0000","last_fullsized":"2024-03-06T07:23:24.235413+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233922+0000","last_change":"2024-03-06T07:23:24.233922+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233922+0000","last_undegraded":"2024-03-06T07:23:24.233922+0000","last_fullsized":"2024-03-06T07:23:24.233922+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250629+0000","last_change":"2024-03-06T07:23:24.250096+0000","last_active":"2024-03-06T07:23:24.250629+0000","last_peered":"2024-03-06T07:23:24.250629+0000","last_clean":"2024-03-06T07:23:24.250629+0000","last_became_active":"2024-03-06T07:23:24.249807+0000","last_became_peered":"2024-03-06T07:23:24.249807+0000","last_unstale":"2024-03-06T07:23:24.250629+0000","last_undegraded":"2024-03-06T07:23:24.250629+0000","last_fullsized":"2024-03-06T07:23:24.250629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:23.940701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232129+0000","last_change":"2024-03-06T07:23:24.232129+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232129+0000","last_undegraded":"2024-03-06T07:23:24.232129+0000","last_fullsized":"2024-03-06T07:23:24.232129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249418+0000","last_change":"2024-03-06T07:23:24.247409+0000","last_active":"2024-03-06T07:23:24.249418+0000","last_peered":"2024-03-06T07:23:24.249418+0000","last_clean":"2024-03-06T07:23:24.249418+0000","last_became_active":"2024-03-06T07:23:24.247083+0000","last_became_peered":"2024-03-06T07:23:24.247083+0000","last_unstale":"2024-03-06T07:23:24.249418+0000","last_undegraded":"2024-03-06T07:23:24.249418+0000","last_fullsized":"2024-03-06T07:23:24.249418+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:32.495551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231919+0000","last_change":"2024-03-06T07:23:24.231919+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231919+0000","last_undegraded":"2024-03-06T07:23:24.231919+0000","last_fullsized":"2024-03-06T07:23:24.231919+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252842+0000","last_change":"2024-03-06T07:23:24.247601+0000","last_active":"2024-03-06T07:23:24.252842+0000","last_peered":"2024-03-06T07:23:24.252842+0000","last_clean":"2024-03-06T07:23:24.252842+0000","last_became_active":"2024-03-06T07:23:24.246790+0000","last_became_peered":"2024-03-06T07:23:24.246790+0000","last_unstale":"2024-03-06T07:23:24.252842+0000","last_undegraded":"2024-03-06T07:23:24.252842+0000","last_fullsized":"2024-03-06T07:23:24.252842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239556+0000","last_change":"2024-03-06T07:23:24.239556+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239556+0000","last_undegraded":"2024-03-06T07:23:24.239556+0000","last_fullsized":"2024-03-06T07:23:24.239556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244586+0000","last_change":"2024-03-06T07:23:24.240691+0000","last_active":"2024-03-06T07:23:24.244586+0000","last_peered":"2024-03-06T07:23:24.244586+0000","last_clean":"2024-03-06T07:23:24.244586+0000","last_became_active":"2024-03-06T07:23:24.240338+0000","last_became_peered":"2024-03-06T07:23:24.240338+0000","last_unstale":"2024-03-06T07:23:24.244586+0000","last_undegraded":"2024-03-06T07:23:24.244586+0000","last_fullsized":"2024-03-06T07:23:24.244586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233786+0000","last_change":"2024-03-06T07:23:24.233786+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233786+0000","last_undegraded":"2024-03-06T07:23:24.233786+0000","last_fullsized":"2024-03-06T07:23:24.233786+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247263+0000","last_change":"2024-03-06T07:23:24.245937+0000","last_active":"2024-03-06T07:23:24.247263+0000","last_peered":"2024-03-06T07:23:24.247263+0000","last_clean":"2024-03-06T07:23:24.247263+0000","last_became_active":"2024-03-06T07:23:24.245614+0000","last_became_peered":"2024-03-06T07:23:24.245614+0000","last_unstale":"2024-03-06T07:23:24.247263+0000","last_undegraded":"2024-03-06T07:23:24.247263+0000","last_fullsized":"2024-03-06T07:23:24.247263+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:26.767861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234462+0000","last_change":"2024-03-06T07:23:24.234462+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234462+0000","last_undegraded":"2024-03-06T07:23:24.234462+0000","last_fullsized":"2024-03-06T07:23:24.234462+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.253182+0000","last_change":"2024-03-06T07:23:24.249153+0000","last_active":"2024-03-06T07:23:24.253182+0000","last_peered":"2024-03-06T07:23:24.253182+0000","last_clean":"2024-03-06T07:23:24.253182+0000","last_became_active":"2024-03-06T07:23:24.248846+0000","last_became_peered":"2024-03-06T07:23:24.248846+0000","last_unstale":"2024-03-06T07:23:24.253182+0000","last_undegraded":"2024-03-06T07:23:24.253182+0000","last_fullsized":"2024-03-06T07:23:24.253182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236089+0000","last_change":"2024-03-06T07:23:24.236089+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236089+0000","last_undegraded":"2024-03-06T07:23:24.236089+0000","last_fullsized":"2024-03-06T07:23:24.236089+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250267+0000","last_change":"2024-03-06T07:23:24.245204+0000","last_active":"2024-03-06T07:23:24.250267+0000","last_peered":"2024-03-06T07:23:24.250267+0000","last_clean":"2024-03-06T07:23:24.250267+0000","last_became_active":"2024-03-06T07:23:24.244893+0000","last_became_peered":"2024-03-06T07:23:24.244893+0000","last_unstale":"2024-03-06T07:23:24.250267+0000","last_undegraded":"2024-03-06T07:23:24.250267+0000","last_fullsized":"2024-03-06T07:23:24.250267+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236223+0000","last_change":"2024-03-06T07:23:24.236223+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236223+0000","last_undegraded":"2024-03-06T07:23:24.236223+0000","last_fullsized":"2024-03-06T07:23:24.236223+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247752+0000","last_change":"2024-03-06T07:23:24.246502+0000","last_active":"2024-03-06T07:23:24.247752+0000","last_peered":"2024-03-06T07:23:24.247752+0000","last_clean":"2024-03-06T07:23:24.247752+0000","last_became_active":"2024-03-06T07:23:24.246230+0000","last_became_peered":"2024-03-06T07:23:24.246230+0000","last_unstale":"2024-03-06T07:23:24.247752+0000","last_undegraded":"2024-03-06T07:23:24.247752+0000","last_fullsized":"2024-03-06T07:23:24.247752+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234549+0000","last_change":"2024-03-06T07:23:24.234549+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234549+0000","last_undegraded":"2024-03-06T07:23:24.234549+0000","last_fullsized":"2024-03-06T07:23:24.234549+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252877+0000","last_change":"2024-03-06T07:23:24.248199+0000","last_active":"2024-03-06T07:23:24.252877+0000","last_peered":"2024-03-06T07:23:24.252877+0000","last_clean":"2024-03-06T07:23:24.252877+0000","last_became_active":"2024-03-06T07:23:24.247705+0000","last_became_peered":"2024-03-06T07:23:24.247705+0000","last_unstale":"2024-03-06T07:23:24.252877+0000","last_undegraded":"2024-03-06T07:23:24.252877+0000","last_fullsized":"2024-03-06T07:23:24.252877+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.421375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231156+0000","last_change":"2024-03-06T07:23:24.231156+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231156+0000","last_undegraded":"2024-03-06T07:23:24.231156+0000","last_fullsized":"2024-03-06T07:23:24.231156+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247760+0000","last_change":"2024-03-06T07:23:24.245504+0000","last_active":"2024-03-06T07:23:24.247760+0000","last_peered":"2024-03-06T07:23:24.247760+0000","last_clean":"2024-03-06T07:23:24.247760+0000","last_became_active":"2024-03-06T07:23:24.245228+0000","last_became_peered":"2024-03-06T07:23:24.245228+0000","last_unstale":"2024-03-06T07:23:24.247760+0000","last_undegraded":"2024-03-06T07:23:24.247760+0000","last_fullsized":"2024-03-06T07:23:24.247760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:42.596673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232421+0000","last_change":"2024-03-06T07:23:24.232421+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232421+0000","last_undegraded":"2024-03-06T07:23:24.232421+0000","last_fullsized":"2024-03-06T07:23:24.232421+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244983+0000","last_change":"2024-03-06T07:23:24.242127+0000","last_active":"2024-03-06T07:23:24.244983+0000","last_peered":"2024-03-06T07:23:24.244983+0000","last_clean":"2024-03-06T07:23:24.244983+0000","last_became_active":"2024-03-06T07:23:24.241524+0000","last_became_peered":"2024-03-06T07:23:24.241524+0000","last_unstale":"2024-03-06T07:23:24.244983+0000","last_undegraded":"2024-03-06T07:23:24.244983+0000","last_fullsized":"2024-03-06T07:23:24.244983+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:42.087045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231299+0000","last_change":"2024-03-06T07:23:24.231299+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231299+0000","last_undegraded":"2024-03-06T07:23:24.231299+0000","last_fullsized":"2024-03-06T07:23:24.231299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252214+0000","last_change":"2024-03-06T07:23:24.246676+0000","last_active":"2024-03-06T07:23:24.252214+0000","last_peered":"2024-03-06T07:23:24.252214+0000","last_clean":"2024-03-06T07:23:24.252214+0000","last_became_active":"2024-03-06T07:23:24.246408+0000","last_became_peered":"2024-03-06T07:23:24.246408+0000","last_unstale":"2024-03-06T07:23:24.252214+0000","last_undegraded":"2024-03-06T07:23:24.252214+0000","last_fullsized":"2024-03-06T07:23:24.252214+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:15.620447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239944+0000","last_change":"2024-03-06T07:23:24.239944+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239944+0000","last_undegraded":"2024-03-06T07:23:24.239944+0000","last_fullsized":"2024-03-06T07:23:24.239944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245809+0000","last_change":"2024-03-06T07:23:24.242187+0000","last_active":"2024-03-06T07:23:24.245809+0000","last_peered":"2024-03-06T07:23:24.245809+0000","last_clean":"2024-03-06T07:23:24.245809+0000","last_became_active":"2024-03-06T07:23:24.241930+0000","last_became_peered":"2024-03-06T07:23:24.241930+0000","last_unstale":"2024-03-06T07:23:24.245809+0000","last_undegraded":"2024-03-06T07:23:24.245809+0000","last_fullsized":"2024-03-06T07:23:24.245809+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236307+0000","last_change":"2024-03-06T07:23:24.236307+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236307+0000","last_undegraded":"2024-03-06T07:23:24.236307+0000","last_fullsized":"2024-03-06T07:23:24.236307+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247677+0000","last_change":"2024-03-06T07:23:24.245524+0000","last_active":"2024-03-06T07:23:24.247677+0000","last_peered":"2024-03-06T07:23:24.247677+0000","last_clean":"2024-03-06T07:23:24.247677+0000","last_became_active":"2024-03-06T07:23:24.245245+0000","last_became_peered":"2024-03-06T07:23:24.245245+0000","last_unstale":"2024-03-06T07:23:24.247677+0000","last_undegraded":"2024-03-06T07:23:24.247677+0000","last_fullsized":"2024-03-06T07:23:24.247677+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234244+0000","last_change":"2024-03-06T07:23:24.234244+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234244+0000","last_undegraded":"2024-03-06T07:23:24.234244+0000","last_fullsized":"2024-03-06T07:23:24.234244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246012+0000","last_change":"2024-03-06T07:23:24.243043+0000","last_active":"2024-03-06T07:23:24.246012+0000","last_peered":"2024-03-06T07:23:24.246012+0000","last_clean":"2024-03-06T07:23:24.246012+0000","last_became_active":"2024-03-06T07:23:24.242793+0000","last_became_peered":"2024-03-06T07:23:24.242793+0000","last_unstale":"2024-03-06T07:23:24.246012+0000","last_undegraded":"2024-03-06T07:23:24.246012+0000","last_fullsized":"2024-03-06T07:23:24.246012+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:12.866720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235408+0000","last_change":"2024-03-06T07:23:24.235408+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235408+0000","last_undegraded":"2024-03-06T07:23:24.235408+0000","last_fullsized":"2024-03-06T07:23:24.235408+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250324+0000","last_change":"2024-03-06T07:23:24.244518+0000","last_active":"2024-03-06T07:23:24.250324+0000","last_peered":"2024-03-06T07:23:24.250324+0000","last_clean":"2024-03-06T07:23:24.250324+0000","last_became_active":"2024-03-06T07:23:24.244154+0000","last_became_peered":"2024-03-06T07:23:24.244154+0000","last_unstale":"2024-03-06T07:23:24.250324+0000","last_undegraded":"2024-03-06T07:23:24.250324+0000","last_fullsized":"2024-03-06T07:23:24.250324+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233094+0000","last_change":"2024-03-06T07:23:24.233094+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233094+0000","last_undegraded":"2024-03-06T07:23:24.233094+0000","last_fullsized":"2024-03-06T07:23:24.233094+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252274+0000","last_change":"2024-03-06T07:23:24.247115+0000","last_active":"2024-03-06T07:23:24.252274+0000","last_peered":"2024-03-06T07:23:24.252274+0000","last_clean":"2024-03-06T07:23:24.252274+0000","last_became_active":"2024-03-06T07:23:24.246835+0000","last_became_peered":"2024-03-06T07:23:24.246835+0000","last_unstale":"2024-03-06T07:23:24.252274+0000","last_undegraded":"2024-03-06T07:23:24.252274+0000","last_fullsized":"2024-03-06T07:23:24.252274+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:40.099312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236125+0000","last_change":"2024-03-06T07:23:24.236125+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236125+0000","last_undegraded":"2024-03-06T07:23:24.236125+0000","last_fullsized":"2024-03-06T07:23:24.236125+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245014+0000","last_change":"2024-03-06T07:23:24.242459+0000","last_active":"2024-03-06T07:23:24.245014+0000","last_peered":"2024-03-06T07:23:24.245014+0000","last_clean":"2024-03-06T07:23:24.245014+0000","last_became_active":"2024-03-06T07:23:24.241848+0000","last_became_peered":"2024-03-06T07:23:24.241848+0000","last_unstale":"2024-03-06T07:23:24.245014+0000","last_undegraded":"2024-03-06T07:23:24.245014+0000","last_fullsized":"2024-03-06T07:23:24.245014+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:01.845106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231767+0000","last_change":"2024-03-06T07:23:24.231767+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231767+0000","last_undegraded":"2024-03-06T07:23:24.231767+0000","last_fullsized":"2024-03-06T07:23:24.231767+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.255498+0000","last_change":"2024-03-06T07:23:24.252717+0000","last_active":"2024-03-06T07:23:24.255498+0000","last_peered":"2024-03-06T07:23:24.255498+0000","last_clean":"2024-03-06T07:23:24.255498+0000","last_became_active":"2024-03-06T07:23:24.252429+0000","last_became_peered":"2024-03-06T07:23:24.252429+0000","last_unstale":"2024-03-06T07:23:24.255498+0000","last_undegraded":"2024-03-06T07:23:24.255498+0000","last_fullsized":"2024-03-06T07:23:24.255498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234015+0000","last_change":"2024-03-06T07:23:24.234015+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234015+0000","last_undegraded":"2024-03-06T07:23:24.234015+0000","last_fullsized":"2024-03-06T07:23:24.234015+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249758+0000","last_change":"2024-03-06T07:23:24.248709+0000","last_active":"2024-03-06T07:23:24.249758+0000","last_peered":"2024-03-06T07:23:24.249758+0000","last_clean":"2024-03-06T07:23:24.249758+0000","last_became_active":"2024-03-06T07:23:24.247184+0000","last_became_peered":"2024-03-06T07:23:24.247184+0000","last_unstale":"2024-03-06T07:23:24.249758+0000","last_undegraded":"2024-03-06T07:23:24.249758+0000","last_fullsized":"2024-03-06T07:23:24.249758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:10.470563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234900+0000","last_change":"2024-03-06T07:23:24.234900+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234900+0000","last_undegraded":"2024-03-06T07:23:24.234900+0000","last_fullsized":"2024-03-06T07:23:24.234900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249143+0000","last_change":"2024-03-06T07:23:24.246510+0000","last_active":"2024-03-06T07:23:24.249143+0000","last_peered":"2024-03-06T07:23:24.249143+0000","last_clean":"2024-03-06T07:23:24.249143+0000","last_became_active":"2024-03-06T07:23:24.246222+0000","last_became_peered":"2024-03-06T07:23:24.246222+0000","last_unstale":"2024-03-06T07:23:24.249143+0000","last_undegraded":"2024-03-06T07:23:24.249143+0000","last_fullsized":"2024-03-06T07:23:24.249143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:26.098267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235380+0000","last_change":"2024-03-06T07:23:24.235380+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235380+0000","last_undegraded":"2024-03-06T07:23:24.235380+0000","last_fullsized":"2024-03-06T07:23:24.235380+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249498+0000","last_change":"2024-03-06T07:23:24.244174+0000","last_active":"2024-03-06T07:23:24.249498+0000","last_peered":"2024-03-06T07:23:24.249498+0000","last_clean":"2024-03-06T07:23:24.249498+0000","last_became_active":"2024-03-06T07:23:24.243046+0000","last_became_peered":"2024-03-06T07:23:24.243046+0000","last_unstale":"2024-03-06T07:23:24.249498+0000","last_undegraded":"2024-03-06T07:23:24.249498+0000","last_fullsized":"2024-03-06T07:23:24.249498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:18.162769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233848+0000","last_change":"2024-03-06T07:23:24.233848+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233848+0000","last_undegraded":"2024-03-06T07:23:24.233848+0000","last_fullsized":"2024-03-06T07:23:24.233848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246844+0000","last_change":"2024-03-06T07:23:24.245809+0000","last_active":"2024-03-06T07:23:24.246844+0000","last_peered":"2024-03-06T07:23:24.246844+0000","last_clean":"2024-03-06T07:23:24.246844+0000","last_became_active":"2024-03-06T07:23:24.245527+0000","last_became_peered":"2024-03-06T07:23:24.245527+0000","last_unstale":"2024-03-06T07:23:24.246844+0000","last_undegraded":"2024-03-06T07:23:24.246844+0000","last_fullsized":"2024-03-06T07:23:24.246844+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:15.139265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234987+0000","last_change":"2024-03-06T07:23:24.234987+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234987+0000","last_undegraded":"2024-03-06T07:23:24.234987+0000","last_fullsized":"2024-03-06T07:23:24.234987+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244456+0000","last_change":"2024-03-06T07:23:24.242259+0000","last_active":"2024-03-06T07:23:24.244456+0000","last_peered":"2024-03-06T07:23:24.244456+0000","last_clean":"2024-03-06T07:23:24.244456+0000","last_became_active":"2024-03-06T07:23:24.241958+0000","last_became_peered":"2024-03-06T07:23:24.241958+0000","last_unstale":"2024-03-06T07:23:24.244456+0000","last_undegraded":"2024-03-06T07:23:24.244456+0000","last_fullsized":"2024-03-06T07:23:24.244456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232370+0000","last_change":"2024-03-06T07:23:24.232370+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232370+0000","last_undegraded":"2024-03-06T07:23:24.232370+0000","last_fullsized":"2024-03-06T07:23:24.232370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247783+0000","last_change":"2024-03-06T07:23:24.246996+0000","last_active":"2024-03-06T07:23:24.247783+0000","last_peered":"2024-03-06T07:23:24.247783+0000","last_clean":"2024-03-06T07:23:24.247783+0000","last_became_active":"2024-03-06T07:23:24.246685+0000","last_became_peered":"2024-03-06T07:23:24.246685+0000","last_unstale":"2024-03-06T07:23:24.247783+0000","last_undegraded":"2024-03-06T07:23:24.247783+0000","last_fullsized":"2024-03-06T07:23:24.247783+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:48:35.768778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235450+0000","last_change":"2024-03-06T07:23:24.235450+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235450+0000","last_undegraded":"2024-03-06T07:23:24.235450+0000","last_fullsized":"2024-03-06T07:23:24.235450+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252103+0000","last_change":"2024-03-06T07:23:24.244748+0000","last_active":"2024-03-06T07:23:24.252103+0000","last_peered":"2024-03-06T07:23:24.252103+0000","last_clean":"2024-03-06T07:23:24.252103+0000","last_became_active":"2024-03-06T07:23:24.244211+0000","last_became_peered":"2024-03-06T07:23:24.244211+0000","last_unstale":"2024-03-06T07:23:24.252103+0000","last_undegraded":"2024-03-06T07:23:24.252103+0000","last_fullsized":"2024-03-06T07:23:24.252103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:21.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240054+0000","last_change":"2024-03-06T07:23:24.240054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240054+0000","last_undegraded":"2024-03-06T07:23:24.240054+0000","last_fullsized":"2024-03-06T07:23:24.240054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250382+0000","last_change":"2024-03-06T07:23:24.248695+0000","last_active":"2024-03-06T07:23:24.250382+0000","last_peered":"2024-03-06T07:23:24.250382+0000","last_clean":"2024-03-06T07:23:24.250382+0000","last_became_active":"2024-03-06T07:23:24.248417+0000","last_became_peered":"2024-03-06T07:23:24.248417+0000","last_unstale":"2024-03-06T07:23:24.250382+0000","last_undegraded":"2024-03-06T07:23:24.250382+0000","last_fullsized":"2024-03-06T07:23:24.250382+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233262+0000","last_change":"2024-03-06T07:23:24.233262+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233262+0000","last_undegraded":"2024-03-06T07:23:24.233262+0000","last_fullsized":"2024-03-06T07:23:24.233262+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250297+0000","last_change":"2024-03-06T07:23:24.246199+0000","last_active":"2024-03-06T07:23:24.250297+0000","last_peered":"2024-03-06T07:23:24.250297+0000","last_clean":"2024-03-06T07:23:24.250297+0000","last_became_active":"2024-03-06T07:23:24.245854+0000","last_became_peered":"2024-03-06T07:23:24.245854+0000","last_unstale":"2024-03-06T07:23:24.250297+0000","last_undegraded":"2024-03-06T07:23:24.250297+0000","last_fullsized":"2024-03-06T07:23:24.250297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232832+0000","last_change":"2024-03-06T07:23:24.232832+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232832+0000","last_undegraded":"2024-03-06T07:23:24.232832+0000","last_fullsized":"2024-03-06T07:23:24.232832+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246395+0000","last_change":"2024-03-06T07:23:24.244313+0000","last_active":"2024-03-06T07:23:24.246395+0000","last_peered":"2024-03-06T07:23:24.246395+0000","last_clean":"2024-03-06T07:23:24.246395+0000","last_became_active":"2024-03-06T07:23:24.244059+0000","last_became_peered":"2024-03-06T07:23:24.244059+0000","last_unstale":"2024-03-06T07:23:24.246395+0000","last_undegraded":"2024-03-06T07:23:24.246395+0000","last_fullsized":"2024-03-06T07:23:24.246395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:53.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232253+0000","last_change":"2024-03-06T07:23:24.232253+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232253+0000","last_undegraded":"2024-03-06T07:23:24.232253+0000","last_fullsized":"2024-03-06T07:23:24.232253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252266+0000","last_change":"2024-03-06T07:23:24.247137+0000","last_active":"2024-03-06T07:23:24.252266+0000","last_peered":"2024-03-06T07:23:24.252266+0000","last_clean":"2024-03-06T07:23:24.252266+0000","last_became_active":"2024-03-06T07:23:24.246871+0000","last_became_peered":"2024-03-06T07:23:24.246871+0000","last_unstale":"2024-03-06T07:23:24.252266+0000","last_undegraded":"2024-03-06T07:23:24.252266+0000","last_fullsized":"2024-03-06T07:23:24.252266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:15:06.289086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242099+0000","last_change":"2024-03-06T07:23:24.242099+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242099+0000","last_undegraded":"2024-03-06T07:23:24.242099+0000","last_fullsized":"2024-03-06T07:23:24.242099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245965+0000","last_change":"2024-03-06T07:23:24.243574+0000","last_active":"2024-03-06T07:23:24.245965+0000","last_peered":"2024-03-06T07:23:24.245965+0000","last_clean":"2024-03-06T07:23:24.245965+0000","last_became_active":"2024-03-06T07:23:24.243297+0000","last_became_peered":"2024-03-06T07:23:24.243297+0000","last_unstale":"2024-03-06T07:23:24.245965+0000","last_undegraded":"2024-03-06T07:23:24.245965+0000","last_fullsized":"2024-03-06T07:23:24.245965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:05.404876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240960+0000","last_change":"2024-03-06T07:23:24.240960+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240960+0000","last_undegraded":"2024-03-06T07:23:24.240960+0000","last_fullsized":"2024-03-06T07:23:24.240960+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247260+0000","last_change":"2024-03-06T07:23:24.244996+0000","last_active":"2024-03-06T07:23:24.247260+0000","last_peered":"2024-03-06T07:23:24.247260+0000","last_clean":"2024-03-06T07:23:24.247260+0000","last_became_active":"2024-03-06T07:23:24.244721+0000","last_became_peered":"2024-03-06T07:23:24.244721+0000","last_unstale":"2024-03-06T07:23:24.247260+0000","last_undegraded":"2024-03-06T07:23:24.247260+0000","last_fullsized":"2024-03-06T07:23:24.247260+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:23.742370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.233201+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:23:24.233201+0000","last_peered":"2024-03-06T07:23:24.233201+0000","last_clean":"2024-03-06T07:23:24.233201+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:23:24.233201+0000","last_undegraded":"2024-03-06T07:23:24.233201+0000","last_fullsized":"2024-03-06T07:23:24.233201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1142784,"data_stored":549035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:28.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:28.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.392+0000 7f9b64956640 1 Processor -- start 2024-03-06T07:23:28.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b64956640 1 -- start start 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b64956640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b600c88c0 0x7f9b60155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b64956640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b60158b60 0x7f9b60158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b64956640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b600c73d0 con 0x7f9b6014f360 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b64956640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b600c8dd0 con 0x7f9b600c88c0 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b64956640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600be2e0 con 0x7f9b60158b60 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b57fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b600c88c0 0x7f9b60155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b57fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b600c88c0 0x7f9b60155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49450/0 (socket says 172.21.15.26:49450) 2024-03-06T07:23:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b57fff640 1 -- 172.21.15.26:0/4035014835 learned_addr learned my addr 172.21.15.26:0/4035014835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.393+0000 7f9b5e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9b6014f360 0x7f9b600c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34834/0 (socket says 172.21.15.26:34834) 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.394+0000 7f9b5ed9d640 1 --2- 172.21.15.26:0/4035014835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b60158b60 0x7f9b60158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.394+0000 7f9b57fff640 1 -- 172.21.15.26:0/4035014835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b60158b60 msgr2=0x7f9b60158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.394+0000 7f9b57fff640 1 --2- 172.21.15.26:0/4035014835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b60158b60 0x7f9b60158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.394+0000 7f9b57fff640 1 -- 172.21.15.26:0/4035014835 >> v1:172.21.15.26:6789/0 conn(0x7f9b6014f360 legacy=0x7f9b600c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.394+0000 7f9b57fff640 1 -- 172.21.15.26:0/4035014835 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600c7670 con 0x7f9b600c88c0 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.394+0000 7f9b57fff640 1 --2- 172.21.15.26:0/4035014835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b600c88c0 0x7f9b60155c10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009870 tx=0x7f9b4802b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2667874d942a6f98 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b5dd9b640 1 -- 172.21.15.26:0/4035014835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48037070 con 0x7f9b600c88c0 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b5dd9b640 1 -- 172.21.15.26:0/4035014835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48030710 con 0x7f9b600c88c0 2024-03-06T07:23:28.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b5dd9b640 1 -- 172.21.15.26:0/4035014835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48030ae0 con 0x7f9b600c88c0 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 -- 172.21.15.26:0/4035014835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b600c88c0 msgr2=0x7f9b60155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 --2- 172.21.15.26:0/4035014835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b600c88c0 0x7f9b60155c10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009870 tx=0x7f9b4802b740 comp rx=0 tx=0).stop 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 -- 172.21.15.26:0/4035014835 shutdown_connections 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 --2- 172.21.15.26:0/4035014835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b600c88c0 0x7f9b60155c10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 --2- 172.21.15.26:0/4035014835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b60158b60 0x7f9b60158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 -- 172.21.15.26:0/4035014835 >> 172.21.15.26:0/4035014835 conn(0x7f9b600ac570 msgr2=0x7f9b600ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 -- 172.21.15.26:0/4035014835 shutdown_connections 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.395+0000 7f9b64956640 1 -- 172.21.15.26:0/4035014835 wait complete. 2024-03-06T07:23:28.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.396+0000 7f9b64956640 1 Processor -- start 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b64956640 1 -- start start 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b64956640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b600c88c0 0x7f9b60157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b64956640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b6014f360 0x7f9b601ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b64956640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60158b60 0x7f9b601ed320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b64956640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600c5a50 con 0x7f9b60158b60 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b64956640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600c7c30 con 0x7f9b6014f360 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b64956640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600c5bd0 con 0x7f9b600c88c0 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b5ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60158b60 0x7f9b601ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b5ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60158b60 0x7f9b601ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49464/0 (socket says 172.21.15.26:49464) 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b5e59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b600c88c0 0x7f9b60157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b5ed9d640 1 -- 172.21.15.26:0/895653720 learned_addr learned my addr 172.21.15.26:0/895653720 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.397+0000 7f9b57fff640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b6014f360 0x7f9b601ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b5ed9d640 1 -- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b600c88c0 msgr2=0x7f9b60157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b5ed9d640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b600c88c0 0x7f9b60157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b5ed9d640 1 -- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b6014f360 msgr2=0x7f9b601ecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b5ed9d640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b6014f360 0x7f9b601ecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b5ed9d640 1 -- 172.21.15.26:0/895653720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600b05c0 con 0x7f9b60158b60 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b5ed9d640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60158b60 0x7f9b601ed320 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800efc0 tx=0x7f9b5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b56ffd640 1 -- 172.21.15.26:0/895653720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58018020 con 0x7f9b60158b60 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b56ffd640 1 -- 172.21.15.26:0/895653720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b58014480 con 0x7f9b60158b60 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b600c0080 con 0x7f9b60158b60 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b56ffd640 1 -- 172.21.15.26:0/895653720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58014870 con 0x7f9b60158b60 2024-03-06T07:23:28.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.398+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b600ad390 con 0x7f9b60158b60 2024-03-06T07:23:28.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.399+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b600c05a0 con 0x7f9b60158b60 2024-03-06T07:23:28.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.399+0000 7f9b56ffd640 1 -- 172.21.15.26:0/895653720 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b58005d10 con 0x7f9b60158b60 2024-03-06T07:23:28.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.400+0000 7f9b56ffd640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b4006b4d0 0x7f9b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.400+0000 7f9b56ffd640 1 -- 172.21.15.26:0/895653720 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b5808b050 con 0x7f9b60158b60 2024-03-06T07:23:28.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.400+0000 7f9b5e59c640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b4006b4d0 0x7f9b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.400+0000 7f9b5e59c640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b4006b4d0 0x7f9b4006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9b50000f60 tx=0x7f9b50007e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.404+0000 7f9b56ffd640 1 -- 172.21.15.26:0/895653720 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b600c05a0 con 0x7f9b60158b60 2024-03-06T07:23:28.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.580+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b600ad6c0 con 0x7f9b4006b4d0 2024-03-06T07:23:28.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.586+0000 7f9b56ffd640 1 -- 172.21.15.26:0/895653720 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246843 (secure 0 0 0) 0x7f9b600ad6c0 con 0x7f9b4006b4d0 2024-03-06T07:23:28.590 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:28.592 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:23:28.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b4006b4d0 msgr2=0x7f9b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b4006b4d0 0x7f9b4006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9b50000f60 tx=0x7f9b50007e20 comp rx=0 tx=0).stop 2024-03-06T07:23:28.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60158b60 msgr2=0x7f9b601ed320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60158b60 0x7f9b601ed320 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800efc0 tx=0x7f9b5800c490 comp rx=0 tx=0).stop 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 shutdown_connections 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b4006b4d0 0x7f9b4006d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60158b60 0x7f9b601ed320 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b6014f360 0x7f9b601ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 --2- 172.21.15.26:0/895653720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b600c88c0 0x7f9b60157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 >> 172.21.15.26:0/895653720 conn(0x7f9b600ac570 msgr2=0x7f9b6014e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 shutdown_connections 2024-03-06T07:23:28.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.589+0000 7f9b64956640 1 -- 172.21.15.26:0/895653720 wait complete. 2024-03-06T07:23:28.601 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-06T07:23:27.072796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250296,"kb_used_data":9140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724424,"statfs":{"total":773094113280,"available":772837810176,"internally_reserved":0,"allocated":9359360,"data_stored":5619685,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000943"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236216+0000","last_change":"2024-03-06T07:23:24.236216+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236216+0000","last_undegraded":"2024-03-06T07:23:24.236216+0000","last_fullsized":"2024-03-06T07:23:24.236216+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237324+0000","last_change":"2024-03-06T07:23:24.237324+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237324+0000","last_undegraded":"2024-03-06T07:23:24.237324+0000","last_fullsized":"2024-03-06T07:23:24.237324+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235853+0000","last_change":"2024-03-06T07:23:24.235853+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235853+0000","last_undegraded":"2024-03-06T07:23:24.235853+0000","last_fullsized":"2024-03-06T07:23:24.235853+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234332+0000","last_change":"2024-03-06T07:23:24.234332+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234332+0000","last_undegraded":"2024-03-06T07:23:24.234332+0000","last_fullsized":"2024-03-06T07:23:24.234332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233017+0000","last_change":"2024-03-06T07:23:24.233017+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233017+0000","last_undegraded":"2024-03-06T07:23:24.233017+0000","last_fullsized":"2024-03-06T07:23:24.233017+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236616+0000","last_change":"2024-03-06T07:23:24.236616+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236616+0000","last_undegraded":"2024-03-06T07:23:24.236616+0000","last_fullsized":"2024-03-06T07:23:24.236616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238220+0000","last_change":"2024-03-06T07:23:24.238220+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238220+0000","last_undegraded":"2024-03-06T07:23:24.238220+0000","last_fullsized":"2024-03-06T07:23:24.238220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235822+0000","last_change":"2024-03-06T07:23:24.235822+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235822+0000","last_undegraded":"2024-03-06T07:23:24.235822+0000","last_fullsized":"2024-03-06T07:23:24.235822+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236313+0000","last_change":"2024-03-06T07:23:24.236313+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236313+0000","last_undegraded":"2024-03-06T07:23:24.236313+0000","last_fullsized":"2024-03-06T07:23:24.236313+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232154+0000","last_change":"2024-03-06T07:23:24.232154+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232154+0000","last_undegraded":"2024-03-06T07:23:24.232154+0000","last_fullsized":"2024-03-06T07:23:24.232154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242162+0000","last_change":"2024-03-06T07:23:24.242162+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242162+0000","last_undegraded":"2024-03-06T07:23:24.242162+0000","last_fullsized":"2024-03-06T07:23:24.242162+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234780+0000","last_change":"2024-03-06T07:23:24.234780+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234780+0000","last_undegraded":"2024-03-06T07:23:24.234780+0000","last_fullsized":"2024-03-06T07:23:24.234780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236601+0000","last_change":"2024-03-06T07:23:24.236601+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236601+0000","last_undegraded":"2024-03-06T07:23:24.236601+0000","last_fullsized":"2024-03-06T07:23:24.236601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232054+0000","last_change":"2024-03-06T07:23:24.232054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232054+0000","last_undegraded":"2024-03-06T07:23:24.232054+0000","last_fullsized":"2024-03-06T07:23:24.232054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231907+0000","last_change":"2024-03-06T07:23:24.231907+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231907+0000","last_undegraded":"2024-03-06T07:23:24.231907+0000","last_fullsized":"2024-03-06T07:23:24.231907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231901+0000","last_change":"2024-03-06T07:23:24.231901+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231901+0000","last_undegraded":"2024-03-06T07:23:24.231901+0000","last_fullsized":"2024-03-06T07:23:24.231901+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235823+0000","last_change":"2024-03-06T07:23:24.235823+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235823+0000","last_undegraded":"2024-03-06T07:23:24.235823+0000","last_fullsized":"2024-03-06T07:23:24.235823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236857+0000","last_change":"2024-03-06T07:23:24.236857+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236857+0000","last_undegraded":"2024-03-06T07:23:24.236857+0000","last_fullsized":"2024-03-06T07:23:24.236857+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235442+0000","last_change":"2024-03-06T07:23:24.235442+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235442+0000","last_undegraded":"2024-03-06T07:23:24.235442+0000","last_fullsized":"2024-03-06T07:23:24.235442+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234283+0000","last_change":"2024-03-06T07:23:24.234283+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234283+0000","last_undegraded":"2024-03-06T07:23:24.234283+0000","last_fullsized":"2024-03-06T07:23:24.234283+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234978+0000","last_change":"2024-03-06T07:23:24.234978+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234978+0000","last_undegraded":"2024-03-06T07:23:24.234978+0000","last_fullsized":"2024-03-06T07:23:24.234978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232196+0000","last_change":"2024-03-06T07:23:24.232196+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232196+0000","last_undegraded":"2024-03-06T07:23:24.232196+0000","last_fullsized":"2024-03-06T07:23:24.232196+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236484+0000","last_change":"2024-03-06T07:23:24.236484+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236484+0000","last_undegraded":"2024-03-06T07:23:24.236484+0000","last_fullsized":"2024-03-06T07:23:24.236484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231241+0000","last_change":"2024-03-06T07:23:24.231241+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231241+0000","last_undegraded":"2024-03-06T07:23:24.231241+0000","last_fullsized":"2024-03-06T07:23:24.231241+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233724+0000","last_change":"2024-03-06T07:23:24.233724+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233724+0000","last_undegraded":"2024-03-06T07:23:24.233724+0000","last_fullsized":"2024-03-06T07:23:24.233724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233286+0000","last_change":"2024-03-06T07:23:24.233286+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233286+0000","last_undegraded":"2024-03-06T07:23:24.233286+0000","last_fullsized":"2024-03-06T07:23:24.233286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.241335+0000","last_change":"2024-03-06T07:23:24.241335+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.241335+0000","last_undegraded":"2024-03-06T07:23:24.241335+0000","last_fullsized":"2024-03-06T07:23:24.241335+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238590+0000","last_change":"2024-03-06T07:23:24.238590+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238590+0000","last_undegraded":"2024-03-06T07:23:24.238590+0000","last_fullsized":"2024-03-06T07:23:24.238590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234554+0000","last_change":"2024-03-06T07:23:24.234554+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234554+0000","last_undegraded":"2024-03-06T07:23:24.234554+0000","last_fullsized":"2024-03-06T07:23:24.234554+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232453+0000","last_change":"2024-03-06T07:23:24.232453+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232453+0000","last_undegraded":"2024-03-06T07:23:24.232453+0000","last_fullsized":"2024-03-06T07:23:24.232453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237717+0000","last_change":"2024-03-06T07:23:24.237717+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237717+0000","last_undegraded":"2024-03-06T07:23:24.237717+0000","last_fullsized":"2024-03-06T07:23:24.237717+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235413+0000","last_change":"2024-03-06T07:23:24.235413+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235413+0000","last_undegraded":"2024-03-06T07:23:24.235413+0000","last_fullsized":"2024-03-06T07:23:24.235413+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233922+0000","last_change":"2024-03-06T07:23:24.233922+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233922+0000","last_undegraded":"2024-03-06T07:23:24.233922+0000","last_fullsized":"2024-03-06T07:23:24.233922+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250629+0000","last_change":"2024-03-06T07:23:24.250096+0000","last_active":"2024-03-06T07:23:24.250629+0000","last_peered":"2024-03-06T07:23:24.250629+0000","last_clean":"2024-03-06T07:23:24.250629+0000","last_became_active":"2024-03-06T07:23:24.249807+0000","last_became_peered":"2024-03-06T07:23:24.249807+0000","last_unstale":"2024-03-06T07:23:24.250629+0000","last_undegraded":"2024-03-06T07:23:24.250629+0000","last_fullsized":"2024-03-06T07:23:24.250629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:23.940701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232129+0000","last_change":"2024-03-06T07:23:24.232129+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232129+0000","last_undegraded":"2024-03-06T07:23:24.232129+0000","last_fullsized":"2024-03-06T07:23:24.232129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249418+0000","last_change":"2024-03-06T07:23:24.247409+0000","last_active":"2024-03-06T07:23:24.249418+0000","last_peered":"2024-03-06T07:23:24.249418+0000","last_clean":"2024-03-06T07:23:24.249418+0000","last_became_active":"2024-03-06T07:23:24.247083+0000","last_became_peered":"2024-03-06T07:23:24.247083+0000","last_unstale":"2024-03-06T07:23:24.249418+0000","last_undegraded":"2024-03-06T07:23:24.249418+0000","last_fullsized":"2024-03-06T07:23:24.249418+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:32.495551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231919+0000","last_change":"2024-03-06T07:23:24.231919+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231919+0000","last_undegraded":"2024-03-06T07:23:24.231919+0000","last_fullsized":"2024-03-06T07:23:24.231919+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252842+0000","last_change":"2024-03-06T07:23:24.247601+0000","last_active":"2024-03-06T07:23:24.252842+0000","last_peered":"2024-03-06T07:23:24.252842+0000","last_clean":"2024-03-06T07:23:24.252842+0000","last_became_active":"2024-03-06T07:23:24.246790+0000","last_became_peered":"2024-03-06T07:23:24.246790+0000","last_unstale":"2024-03-06T07:23:24.252842+0000","last_undegraded":"2024-03-06T07:23:24.252842+0000","last_fullsized":"2024-03-06T07:23:24.252842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239556+0000","last_change":"2024-03-06T07:23:24.239556+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239556+0000","last_undegraded":"2024-03-06T07:23:24.239556+0000","last_fullsized":"2024-03-06T07:23:24.239556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244586+0000","last_change":"2024-03-06T07:23:24.240691+0000","last_active":"2024-03-06T07:23:24.244586+0000","last_peered":"2024-03-06T07:23:24.244586+0000","last_clean":"2024-03-06T07:23:24.244586+0000","last_became_active":"2024-03-06T07:23:24.240338+0000","last_became_peered":"2024-03-06T07:23:24.240338+0000","last_unstale":"2024-03-06T07:23:24.244586+0000","last_undegraded":"2024-03-06T07:23:24.244586+0000","last_fullsized":"2024-03-06T07:23:24.244586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233786+0000","last_change":"2024-03-06T07:23:24.233786+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233786+0000","last_undegraded":"2024-03-06T07:23:24.233786+0000","last_fullsized":"2024-03-06T07:23:24.233786+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247263+0000","last_change":"2024-03-06T07:23:24.245937+0000","last_active":"2024-03-06T07:23:24.247263+0000","last_peered":"2024-03-06T07:23:24.247263+0000","last_clean":"2024-03-06T07:23:24.247263+0000","last_became_active":"2024-03-06T07:23:24.245614+0000","last_became_peered":"2024-03-06T07:23:24.245614+0000","last_unstale":"2024-03-06T07:23:24.247263+0000","last_undegraded":"2024-03-06T07:23:24.247263+0000","last_fullsized":"2024-03-06T07:23:24.247263+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:26.767861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234462+0000","last_change":"2024-03-06T07:23:24.234462+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234462+0000","last_undegraded":"2024-03-06T07:23:24.234462+0000","last_fullsized":"2024-03-06T07:23:24.234462+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.253182+0000","last_change":"2024-03-06T07:23:24.249153+0000","last_active":"2024-03-06T07:23:24.253182+0000","last_peered":"2024-03-06T07:23:24.253182+0000","last_clean":"2024-03-06T07:23:24.253182+0000","last_became_active":"2024-03-06T07:23:24.248846+0000","last_became_peered":"2024-03-06T07:23:24.248846+0000","last_unstale":"2024-03-06T07:23:24.253182+0000","last_undegraded":"2024-03-06T07:23:24.253182+0000","last_fullsized":"2024-03-06T07:23:24.253182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236089+0000","last_change":"2024-03-06T07:23:24.236089+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236089+0000","last_undegraded":"2024-03-06T07:23:24.236089+0000","last_fullsized":"2024-03-06T07:23:24.236089+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250267+0000","last_change":"2024-03-06T07:23:24.245204+0000","last_active":"2024-03-06T07:23:24.250267+0000","last_peered":"2024-03-06T07:23:24.250267+0000","last_clean":"2024-03-06T07:23:24.250267+0000","last_became_active":"2024-03-06T07:23:24.244893+0000","last_became_peered":"2024-03-06T07:23:24.244893+0000","last_unstale":"2024-03-06T07:23:24.250267+0000","last_undegraded":"2024-03-06T07:23:24.250267+0000","last_fullsized":"2024-03-06T07:23:24.250267+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236223+0000","last_change":"2024-03-06T07:23:24.236223+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236223+0000","last_undegraded":"2024-03-06T07:23:24.236223+0000","last_fullsized":"2024-03-06T07:23:24.236223+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247752+0000","last_change":"2024-03-06T07:23:24.246502+0000","last_active":"2024-03-06T07:23:24.247752+0000","last_peered":"2024-03-06T07:23:24.247752+0000","last_clean":"2024-03-06T07:23:24.247752+0000","last_became_active":"2024-03-06T07:23:24.246230+0000","last_became_peered":"2024-03-06T07:23:24.246230+0000","last_unstale":"2024-03-06T07:23:24.247752+0000","last_undegraded":"2024-03-06T07:23:24.247752+0000","last_fullsized":"2024-03-06T07:23:24.247752+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234549+0000","last_change":"2024-03-06T07:23:24.234549+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234549+0000","last_undegraded":"2024-03-06T07:23:24.234549+0000","last_fullsized":"2024-03-06T07:23:24.234549+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252877+0000","last_change":"2024-03-06T07:23:24.248199+0000","last_active":"2024-03-06T07:23:24.252877+0000","last_peered":"2024-03-06T07:23:24.252877+0000","last_clean":"2024-03-06T07:23:24.252877+0000","last_became_active":"2024-03-06T07:23:24.247705+0000","last_became_peered":"2024-03-06T07:23:24.247705+0000","last_unstale":"2024-03-06T07:23:24.252877+0000","last_undegraded":"2024-03-06T07:23:24.252877+0000","last_fullsized":"2024-03-06T07:23:24.252877+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.421375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231156+0000","last_change":"2024-03-06T07:23:24.231156+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231156+0000","last_undegraded":"2024-03-06T07:23:24.231156+0000","last_fullsized":"2024-03-06T07:23:24.231156+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247760+0000","last_change":"2024-03-06T07:23:24.245504+0000","last_active":"2024-03-06T07:23:24.247760+0000","last_peered":"2024-03-06T07:23:24.247760+0000","last_clean":"2024-03-06T07:23:24.247760+0000","last_became_active":"2024-03-06T07:23:24.245228+0000","last_became_peered":"2024-03-06T07:23:24.245228+0000","last_unstale":"2024-03-06T07:23:24.247760+0000","last_undegraded":"2024-03-06T07:23:24.247760+0000","last_fullsized":"2024-03-06T07:23:24.247760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:42.596673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232421+0000","last_change":"2024-03-06T07:23:24.232421+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232421+0000","last_undegraded":"2024-03-06T07:23:24.232421+0000","last_fullsized":"2024-03-06T07:23:24.232421+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244983+0000","last_change":"2024-03-06T07:23:24.242127+0000","last_active":"2024-03-06T07:23:24.244983+0000","last_peered":"2024-03-06T07:23:24.244983+0000","last_clean":"2024-03-06T07:23:24.244983+0000","last_became_active":"2024-03-06T07:23:24.241524+0000","last_became_peered":"2024-03-06T07:23:24.241524+0000","last_unstale":"2024-03-06T07:23:24.244983+0000","last_undegraded":"2024-03-06T07:23:24.244983+0000","last_fullsized":"2024-03-06T07:23:24.244983+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:42.087045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231299+0000","last_change":"2024-03-06T07:23:24.231299+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231299+0000","last_undegraded":"2024-03-06T07:23:24.231299+0000","last_fullsized":"2024-03-06T07:23:24.231299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252214+0000","last_change":"2024-03-06T07:23:24.246676+0000","last_active":"2024-03-06T07:23:24.252214+0000","last_peered":"2024-03-06T07:23:24.252214+0000","last_clean":"2024-03-06T07:23:24.252214+0000","last_became_active":"2024-03-06T07:23:24.246408+0000","last_became_peered":"2024-03-06T07:23:24.246408+0000","last_unstale":"2024-03-06T07:23:24.252214+0000","last_undegraded":"2024-03-06T07:23:24.252214+0000","last_fullsized":"2024-03-06T07:23:24.252214+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:15.620447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239944+0000","last_change":"2024-03-06T07:23:24.239944+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239944+0000","last_undegraded":"2024-03-06T07:23:24.239944+0000","last_fullsized":"2024-03-06T07:23:24.239944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245809+0000","last_change":"2024-03-06T07:23:24.242187+0000","last_active":"2024-03-06T07:23:24.245809+0000","last_peered":"2024-03-06T07:23:24.245809+0000","last_clean":"2024-03-06T07:23:24.245809+0000","last_became_active":"2024-03-06T07:23:24.241930+0000","last_became_peered":"2024-03-06T07:23:24.241930+0000","last_unstale":"2024-03-06T07:23:24.245809+0000","last_undegraded":"2024-03-06T07:23:24.245809+0000","last_fullsized":"2024-03-06T07:23:24.245809+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236307+0000","last_change":"2024-03-06T07:23:24.236307+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236307+0000","last_undegraded":"2024-03-06T07:23:24.236307+0000","last_fullsized":"2024-03-06T07:23:24.236307+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247677+0000","last_change":"2024-03-06T07:23:24.245524+0000","last_active":"2024-03-06T07:23:24.247677+0000","last_peered":"2024-03-06T07:23:24.247677+0000","last_clean":"2024-03-06T07:23:24.247677+0000","last_became_active":"2024-03-06T07:23:24.245245+0000","last_became_peered":"2024-03-06T07:23:24.245245+0000","last_unstale":"2024-03-06T07:23:24.247677+0000","last_undegraded":"2024-03-06T07:23:24.247677+0000","last_fullsized":"2024-03-06T07:23:24.247677+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234244+0000","last_change":"2024-03-06T07:23:24.234244+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234244+0000","last_undegraded":"2024-03-06T07:23:24.234244+0000","last_fullsized":"2024-03-06T07:23:24.234244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246012+0000","last_change":"2024-03-06T07:23:24.243043+0000","last_active":"2024-03-06T07:23:24.246012+0000","last_peered":"2024-03-06T07:23:24.246012+0000","last_clean":"2024-03-06T07:23:24.246012+0000","last_became_active":"2024-03-06T07:23:24.242793+0000","last_became_peered":"2024-03-06T07:23:24.242793+0000","last_unstale":"2024-03-06T07:23:24.246012+0000","last_undegraded":"2024-03-06T07:23:24.246012+0000","last_fullsized":"2024-03-06T07:23:24.246012+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:12.866720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235408+0000","last_change":"2024-03-06T07:23:24.235408+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235408+0000","last_undegraded":"2024-03-06T07:23:24.235408+0000","last_fullsized":"2024-03-06T07:23:24.235408+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250324+0000","last_change":"2024-03-06T07:23:24.244518+0000","last_active":"2024-03-06T07:23:24.250324+0000","last_peered":"2024-03-06T07:23:24.250324+0000","last_clean":"2024-03-06T07:23:24.250324+0000","last_became_active":"2024-03-06T07:23:24.244154+0000","last_became_peered":"2024-03-06T07:23:24.244154+0000","last_unstale":"2024-03-06T07:23:24.250324+0000","last_undegraded":"2024-03-06T07:23:24.250324+0000","last_fullsized":"2024-03-06T07:23:24.250324+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233094+0000","last_change":"2024-03-06T07:23:24.233094+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233094+0000","last_undegraded":"2024-03-06T07:23:24.233094+0000","last_fullsized":"2024-03-06T07:23:24.233094+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252274+0000","last_change":"2024-03-06T07:23:24.247115+0000","last_active":"2024-03-06T07:23:24.252274+0000","last_peered":"2024-03-06T07:23:24.252274+0000","last_clean":"2024-03-06T07:23:24.252274+0000","last_became_active":"2024-03-06T07:23:24.246835+0000","last_became_peered":"2024-03-06T07:23:24.246835+0000","last_unstale":"2024-03-06T07:23:24.252274+0000","last_undegraded":"2024-03-06T07:23:24.252274+0000","last_fullsized":"2024-03-06T07:23:24.252274+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:40.099312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236125+0000","last_change":"2024-03-06T07:23:24.236125+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236125+0000","last_undegraded":"2024-03-06T07:23:24.236125+0000","last_fullsized":"2024-03-06T07:23:24.236125+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245014+0000","last_change":"2024-03-06T07:23:24.242459+0000","last_active":"2024-03-06T07:23:24.245014+0000","last_peered":"2024-03-06T07:23:24.245014+0000","last_clean":"2024-03-06T07:23:24.245014+0000","last_became_active":"2024-03-06T07:23:24.241848+0000","last_became_peered":"2024-03-06T07:23:24.241848+0000","last_unstale":"2024-03-06T07:23:24.245014+0000","last_undegraded":"2024-03-06T07:23:24.245014+0000","last_fullsized":"2024-03-06T07:23:24.245014+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:01.845106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231767+0000","last_change":"2024-03-06T07:23:24.231767+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231767+0000","last_undegraded":"2024-03-06T07:23:24.231767+0000","last_fullsized":"2024-03-06T07:23:24.231767+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.255498+0000","last_change":"2024-03-06T07:23:24.252717+0000","last_active":"2024-03-06T07:23:24.255498+0000","last_peered":"2024-03-06T07:23:24.255498+0000","last_clean":"2024-03-06T07:23:24.255498+0000","last_became_active":"2024-03-06T07:23:24.252429+0000","last_became_peered":"2024-03-06T07:23:24.252429+0000","last_unstale":"2024-03-06T07:23:24.255498+0000","last_undegraded":"2024-03-06T07:23:24.255498+0000","last_fullsized":"2024-03-06T07:23:24.255498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234015+0000","last_change":"2024-03-06T07:23:24.234015+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234015+0000","last_undegraded":"2024-03-06T07:23:24.234015+0000","last_fullsized":"2024-03-06T07:23:24.234015+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249758+0000","last_change":"2024-03-06T07:23:24.248709+0000","last_active":"2024-03-06T07:23:24.249758+0000","last_peered":"2024-03-06T07:23:24.249758+0000","last_clean":"2024-03-06T07:23:24.249758+0000","last_became_active":"2024-03-06T07:23:24.247184+0000","last_became_peered":"2024-03-06T07:23:24.247184+0000","last_unstale":"2024-03-06T07:23:24.249758+0000","last_undegraded":"2024-03-06T07:23:24.249758+0000","last_fullsized":"2024-03-06T07:23:24.249758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:10.470563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234900+0000","last_change":"2024-03-06T07:23:24.234900+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234900+0000","last_undegraded":"2024-03-06T07:23:24.234900+0000","last_fullsized":"2024-03-06T07:23:24.234900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249143+0000","last_change":"2024-03-06T07:23:24.246510+0000","last_active":"2024-03-06T07:23:24.249143+0000","last_peered":"2024-03-06T07:23:24.249143+0000","last_clean":"2024-03-06T07:23:24.249143+0000","last_became_active":"2024-03-06T07:23:24.246222+0000","last_became_peered":"2024-03-06T07:23:24.246222+0000","last_unstale":"2024-03-06T07:23:24.249143+0000","last_undegraded":"2024-03-06T07:23:24.249143+0000","last_fullsized":"2024-03-06T07:23:24.249143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:26.098267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235380+0000","last_change":"2024-03-06T07:23:24.235380+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235380+0000","last_undegraded":"2024-03-06T07:23:24.235380+0000","last_fullsized":"2024-03-06T07:23:24.235380+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249498+0000","last_change":"2024-03-06T07:23:24.244174+0000","last_active":"2024-03-06T07:23:24.249498+0000","last_peered":"2024-03-06T07:23:24.249498+0000","last_clean":"2024-03-06T07:23:24.249498+0000","last_became_active":"2024-03-06T07:23:24.243046+0000","last_became_peered":"2024-03-06T07:23:24.243046+0000","last_unstale":"2024-03-06T07:23:24.249498+0000","last_undegraded":"2024-03-06T07:23:24.249498+0000","last_fullsized":"2024-03-06T07:23:24.249498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:18.162769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233848+0000","last_change":"2024-03-06T07:23:24.233848+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233848+0000","last_undegraded":"2024-03-06T07:23:24.233848+0000","last_fullsized":"2024-03-06T07:23:24.233848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246844+0000","last_change":"2024-03-06T07:23:24.245809+0000","last_active":"2024-03-06T07:23:24.246844+0000","last_peered":"2024-03-06T07:23:24.246844+0000","last_clean":"2024-03-06T07:23:24.246844+0000","last_became_active":"2024-03-06T07:23:24.245527+0000","last_became_peered":"2024-03-06T07:23:24.245527+0000","last_unstale":"2024-03-06T07:23:24.246844+0000","last_undegraded":"2024-03-06T07:23:24.246844+0000","last_fullsized":"2024-03-06T07:23:24.246844+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:15.139265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234987+0000","last_change":"2024-03-06T07:23:24.234987+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234987+0000","last_undegraded":"2024-03-06T07:23:24.234987+0000","last_fullsized":"2024-03-06T07:23:24.234987+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244456+0000","last_change":"2024-03-06T07:23:24.242259+0000","last_active":"2024-03-06T07:23:24.244456+0000","last_peered":"2024-03-06T07:23:24.244456+0000","last_clean":"2024-03-06T07:23:24.244456+0000","last_became_active":"2024-03-06T07:23:24.241958+0000","last_became_peered":"2024-03-06T07:23:24.241958+0000","last_unstale":"2024-03-06T07:23:24.244456+0000","last_undegraded":"2024-03-06T07:23:24.244456+0000","last_fullsized":"2024-03-06T07:23:24.244456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232370+0000","last_change":"2024-03-06T07:23:24.232370+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232370+0000","last_undegraded":"2024-03-06T07:23:24.232370+0000","last_fullsized":"2024-03-06T07:23:24.232370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247783+0000","last_change":"2024-03-06T07:23:24.246996+0000","last_active":"2024-03-06T07:23:24.247783+0000","last_peered":"2024-03-06T07:23:24.247783+0000","last_clean":"2024-03-06T07:23:24.247783+0000","last_became_active":"2024-03-06T07:23:24.246685+0000","last_became_peered":"2024-03-06T07:23:24.246685+0000","last_unstale":"2024-03-06T07:23:24.247783+0000","last_undegraded":"2024-03-06T07:23:24.247783+0000","last_fullsized":"2024-03-06T07:23:24.247783+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:48:35.768778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235450+0000","last_change":"2024-03-06T07:23:24.235450+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235450+0000","last_undegraded":"2024-03-06T07:23:24.235450+0000","last_fullsized":"2024-03-06T07:23:24.235450+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252103+0000","last_change":"2024-03-06T07:23:24.244748+0000","last_active":"2024-03-06T07:23:24.252103+0000","last_peered":"2024-03-06T07:23:24.252103+0000","last_clean":"2024-03-06T07:23:24.252103+0000","last_became_active":"2024-03-06T07:23:24.244211+0000","last_became_peered":"2024-03-06T07:23:24.244211+0000","last_unstale":"2024-03-06T07:23:24.252103+0000","last_undegraded":"2024-03-06T07:23:24.252103+0000","last_fullsized":"2024-03-06T07:23:24.252103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:21.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240054+0000","last_change":"2024-03-06T07:23:24.240054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240054+0000","last_undegraded":"2024-03-06T07:23:24.240054+0000","last_fullsized":"2024-03-06T07:23:24.240054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250382+0000","last_change":"2024-03-06T07:23:24.248695+0000","last_active":"2024-03-06T07:23:24.250382+0000","last_peered":"2024-03-06T07:23:24.250382+0000","last_clean":"2024-03-06T07:23:24.250382+0000","last_became_active":"2024-03-06T07:23:24.248417+0000","last_became_peered":"2024-03-06T07:23:24.248417+0000","last_unstale":"2024-03-06T07:23:24.250382+0000","last_undegraded":"2024-03-06T07:23:24.250382+0000","last_fullsized":"2024-03-06T07:23:24.250382+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233262+0000","last_change":"2024-03-06T07:23:24.233262+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233262+0000","last_undegraded":"2024-03-06T07:23:24.233262+0000","last_fullsized":"2024-03-06T07:23:24.233262+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250297+0000","last_change":"2024-03-06T07:23:24.246199+0000","last_active":"2024-03-06T07:23:24.250297+0000","last_peered":"2024-03-06T07:23:24.250297+0000","last_clean":"2024-03-06T07:23:24.250297+0000","last_became_active":"2024-03-06T07:23:24.245854+0000","last_became_peered":"2024-03-06T07:23:24.245854+0000","last_unstale":"2024-03-06T07:23:24.250297+0000","last_undegraded":"2024-03-06T07:23:24.250297+0000","last_fullsized":"2024-03-06T07:23:24.250297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232832+0000","last_change":"2024-03-06T07:23:24.232832+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232832+0000","last_undegraded":"2024-03-06T07:23:24.232832+0000","last_fullsized":"2024-03-06T07:23:24.232832+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246395+0000","last_change":"2024-03-06T07:23:24.244313+0000","last_active":"2024-03-06T07:23:24.246395+0000","last_peered":"2024-03-06T07:23:24.246395+0000","last_clean":"2024-03-06T07:23:24.246395+0000","last_became_active":"2024-03-06T07:23:24.244059+0000","last_became_peered":"2024-03-06T07:23:24.244059+0000","last_unstale":"2024-03-06T07:23:24.246395+0000","last_undegraded":"2024-03-06T07:23:24.246395+0000","last_fullsized":"2024-03-06T07:23:24.246395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:53.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232253+0000","last_change":"2024-03-06T07:23:24.232253+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232253+0000","last_undegraded":"2024-03-06T07:23:24.232253+0000","last_fullsized":"2024-03-06T07:23:24.232253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252266+0000","last_change":"2024-03-06T07:23:24.247137+0000","last_active":"2024-03-06T07:23:24.252266+0000","last_peered":"2024-03-06T07:23:24.252266+0000","last_clean":"2024-03-06T07:23:24.252266+0000","last_became_active":"2024-03-06T07:23:24.246871+0000","last_became_peered":"2024-03-06T07:23:24.246871+0000","last_unstale":"2024-03-06T07:23:24.252266+0000","last_undegraded":"2024-03-06T07:23:24.252266+0000","last_fullsized":"2024-03-06T07:23:24.252266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:15:06.289086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242099+0000","last_change":"2024-03-06T07:23:24.242099+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242099+0000","last_undegraded":"2024-03-06T07:23:24.242099+0000","last_fullsized":"2024-03-06T07:23:24.242099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245965+0000","last_change":"2024-03-06T07:23:24.243574+0000","last_active":"2024-03-06T07:23:24.245965+0000","last_peered":"2024-03-06T07:23:24.245965+0000","last_clean":"2024-03-06T07:23:24.245965+0000","last_became_active":"2024-03-06T07:23:24.243297+0000","last_became_peered":"2024-03-06T07:23:24.243297+0000","last_unstale":"2024-03-06T07:23:24.245965+0000","last_undegraded":"2024-03-06T07:23:24.245965+0000","last_fullsized":"2024-03-06T07:23:24.245965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:05.404876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240960+0000","last_change":"2024-03-06T07:23:24.240960+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240960+0000","last_undegraded":"2024-03-06T07:23:24.240960+0000","last_fullsized":"2024-03-06T07:23:24.240960+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247260+0000","last_change":"2024-03-06T07:23:24.244996+0000","last_active":"2024-03-06T07:23:24.247260+0000","last_peered":"2024-03-06T07:23:24.247260+0000","last_clean":"2024-03-06T07:23:24.247260+0000","last_became_active":"2024-03-06T07:23:24.244721+0000","last_became_peered":"2024-03-06T07:23:24.244721+0000","last_unstale":"2024-03-06T07:23:24.247260+0000","last_undegraded":"2024-03-06T07:23:24.247260+0000","last_fullsized":"2024-03-06T07:23:24.247260+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:23.742370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.233201+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:23:24.233201+0000","last_peered":"2024-03-06T07:23:24.233201+0000","last_clean":"2024-03-06T07:23:24.233201+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:23:24.233201+0000","last_undegraded":"2024-03-06T07:23:24.233201+0000","last_fullsized":"2024-03-06T07:23:24.233201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1142784,"data_stored":549035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:28.605 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.741+0000 7f2906f1e640 1 Processor -- start 2024-03-06T07:23:28.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.741+0000 7f2906f1e640 1 -- start start 2024-03-06T07:23:28.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.741+0000 7f2906f1e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29000c5330 0x7f29000c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2906f1e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29000c88b0 0x7f29001586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2906f1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c7dd0 0x7f2900158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2906f1e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29001559e0 con 0x7f29000c5330 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2906f1e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2900155110 con 0x7f29000c88b0 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2906f1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29000c8f40 con 0x7f29000c7dd0 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29000c5330 0x7f29000c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29000c5330 0x7f29000c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49466/0 (socket says 172.21.15.26:49466) 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 -- 172.21.15.26:0/704233215 learned_addr learned my addr 172.21.15.26:0/704233215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f28fffff640 1 --2- 172.21.15.26:0/704233215 >> v2:172.21.15.117:3300/0 conn(0x7f29000c88b0 0x7f29001586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f29054bb640 1 --2- 172.21.15.26:0/704233215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c7dd0 0x7f2900158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 -- 172.21.15.26:0/704233215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c7dd0 msgr2=0x7f2900158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 --2- 172.21.15.26:0/704233215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c7dd0 0x7f2900158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 -- 172.21.15.26:0/704233215 >> v2:172.21.15.117:3300/0 conn(0x7f29000c88b0 msgr2=0x7f29001586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 --2- 172.21.15.26:0/704233215 >> v2:172.21.15.117:3300/0 conn(0x7f29000c88b0 0x7f29001586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.742+0000 7f2904cba640 1 -- 172.21.15.26:0/704233215 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2900155390 con 0x7f29000c5330 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.743+0000 7f2904cba640 1 --2- 172.21.15.26:0/704233215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f29000c5710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2900156240 tx=0x7f28f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57489d3f63b10c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.743+0000 7f28ff7fe640 1 -- 172.21.15.26:0/704233215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f402e070 con 0x7f29000c5330 2024-03-06T07:23:28.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.743+0000 7f28ff7fe640 1 -- 172.21.15.26:0/704233215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4037070 con 0x7f29000c5330 2024-03-06T07:23:28.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.743+0000 7f28ff7fe640 1 -- 172.21.15.26:0/704233215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4031a00 con 0x7f29000c5330 2024-03-06T07:23:28.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.743+0000 7f2906f1e640 1 -- 172.21.15.26:0/704233215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 msgr2=0x7f29000c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.743+0000 7f2906f1e640 1 --2- 172.21.15.26:0/704233215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f29000c5710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2900156240 tx=0x7f28f402b880 comp rx=0 tx=0).stop 2024-03-06T07:23:28.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.744+0000 7f2906f1e640 1 -- 172.21.15.26:0/704233215 shutdown_connections 2024-03-06T07:23:28.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.744+0000 7f2906f1e640 1 --2- 172.21.15.26:0/704233215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c7dd0 0x7f2900158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.744+0000 7f2906f1e640 1 --2- 172.21.15.26:0/704233215 >> v2:172.21.15.117:3300/0 conn(0x7f29000c88b0 0x7f29001586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.744+0000 7f2906f1e640 1 --2- 172.21.15.26:0/704233215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f29000c5710 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.744+0000 7f2906f1e640 1 -- 172.21.15.26:0/704233215 >> 172.21.15.26:0/704233215 conn(0x7f29000ac560 msgr2=0x7f29000ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.744+0000 7f2906f1e640 1 -- 172.21.15.26:0/704233215 shutdown_connections 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.744+0000 7f2906f1e640 1 -- 172.21.15.26:0/704233215 wait complete. 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 Processor -- start 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 -- start start 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f290014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29000c7dd0 0x7f2900151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c88b0 0x7f29001eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29001529d0 con 0x7f29000c5330 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290006d090 con 0x7f29000c7dd0 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.745+0000 7f2906f1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2900152b50 con 0x7f29000c88b0 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f290014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f290014bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49472/0 (socket says 172.21.15.26:49472) 2024-03-06T07:23:28.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 -- 172.21.15.26:0/3741012289 learned_addr learned my addr 172.21.15.26:0/3741012289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f29054bb640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c88b0 0x7f29001eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f28fffff640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29000c7dd0 0x7f2900151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 -- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c88b0 msgr2=0x7f29001eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c88b0 0x7f29001eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 -- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29000c7dd0 msgr2=0x7f2900151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29000c7dd0 0x7f2900151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 -- 172.21.15.26:0/3741012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2900153690 con 0x7f29000c5330 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2904cba640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f290014bf80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097c0 tx=0x7f28f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f28fd7fa640 1 -- 172.21.15.26:0/3741012289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4032670 con 0x7f29000c5330 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f28fd7fa640 1 -- 172.21.15.26:0/3741012289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4032810 con 0x7f29000c5330 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29001eeeb0 con 0x7f29000c5330 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f28fd7fa640 1 -- 172.21.15.26:0/3741012289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4032c00 con 0x7f29000c5330 2024-03-06T07:23:28.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.746+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29001ee0f0 con 0x7f29000c5330 2024-03-06T07:23:28.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.747+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290004f330 con 0x7f29000c5330 2024-03-06T07:23:28.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.748+0000 7f28fd7fa640 1 -- 172.21.15.26:0/3741012289 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28f404c060 con 0x7f29000c5330 2024-03-06T07:23:28.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.748+0000 7f28fd7fa640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28e406b4d0 0x7f28e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.748+0000 7f28fd7fa640 1 -- 172.21.15.26:0/3741012289 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28f40ab650 con 0x7f29000c5330 2024-03-06T07:23:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.748+0000 7f28fffff640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28e406b4d0 0x7f28e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.749+0000 7f28fffff640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28e406b4d0 0x7f28e406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f29000ac9b0 tx=0x7f28ec009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.752+0000 7f28fd7fa640 1 -- 172.21.15.26:0/3741012289 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28f4040070 con 0x7f29000c5330 2024-03-06T07:23:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.931+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29000c99b0 con 0x7f28e406b4d0 2024-03-06T07:23:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.937+0000 7f28fd7fa640 1 -- 172.21.15.26:0/3741012289 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246843 (secure 0 0 0) 0x7f29000c99b0 con 0x7f28e406b4d0 2024-03-06T07:23:28.941 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:28.942 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28e406b4d0 msgr2=0x7f28e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28e406b4d0 0x7f28e406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f29000ac9b0 tx=0x7f28ec009190 comp rx=0 tx=0).stop 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 msgr2=0x7f290014bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f290014bf80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097c0 tx=0x7f28f40046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 shutdown_connections 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28e406b4d0 0x7f28e406d990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29000c88b0 0x7f29001eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29000c7dd0 0x7f2900151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 --2- 172.21.15.26:0/3741012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29000c5330 0x7f290014bf80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 >> 172.21.15.26:0/3741012289 conn(0x7f29000ac560 msgr2=0x7f29001589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 shutdown_connections 2024-03-06T07:23:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:28.940+0000 7f2906f1e640 1 -- 172.21.15.26:0/3741012289 wait complete. 2024-03-06T07:23:28.952 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-06T07:23:27.072796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250296,"kb_used_data":9140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724424,"statfs":{"total":773094113280,"available":772837810176,"internally_reserved":0,"allocated":9359360,"data_stored":5619685,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000943"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236216+0000","last_change":"2024-03-06T07:23:24.236216+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236216+0000","last_undegraded":"2024-03-06T07:23:24.236216+0000","last_fullsized":"2024-03-06T07:23:24.236216+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237324+0000","last_change":"2024-03-06T07:23:24.237324+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237324+0000","last_undegraded":"2024-03-06T07:23:24.237324+0000","last_fullsized":"2024-03-06T07:23:24.237324+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235853+0000","last_change":"2024-03-06T07:23:24.235853+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235853+0000","last_undegraded":"2024-03-06T07:23:24.235853+0000","last_fullsized":"2024-03-06T07:23:24.235853+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234332+0000","last_change":"2024-03-06T07:23:24.234332+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234332+0000","last_undegraded":"2024-03-06T07:23:24.234332+0000","last_fullsized":"2024-03-06T07:23:24.234332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233017+0000","last_change":"2024-03-06T07:23:24.233017+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233017+0000","last_undegraded":"2024-03-06T07:23:24.233017+0000","last_fullsized":"2024-03-06T07:23:24.233017+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236616+0000","last_change":"2024-03-06T07:23:24.236616+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236616+0000","last_undegraded":"2024-03-06T07:23:24.236616+0000","last_fullsized":"2024-03-06T07:23:24.236616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238220+0000","last_change":"2024-03-06T07:23:24.238220+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238220+0000","last_undegraded":"2024-03-06T07:23:24.238220+0000","last_fullsized":"2024-03-06T07:23:24.238220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235822+0000","last_change":"2024-03-06T07:23:24.235822+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235822+0000","last_undegraded":"2024-03-06T07:23:24.235822+0000","last_fullsized":"2024-03-06T07:23:24.235822+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236313+0000","last_change":"2024-03-06T07:23:24.236313+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236313+0000","last_undegraded":"2024-03-06T07:23:24.236313+0000","last_fullsized":"2024-03-06T07:23:24.236313+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232154+0000","last_change":"2024-03-06T07:23:24.232154+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232154+0000","last_undegraded":"2024-03-06T07:23:24.232154+0000","last_fullsized":"2024-03-06T07:23:24.232154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242162+0000","last_change":"2024-03-06T07:23:24.242162+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242162+0000","last_undegraded":"2024-03-06T07:23:24.242162+0000","last_fullsized":"2024-03-06T07:23:24.242162+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234780+0000","last_change":"2024-03-06T07:23:24.234780+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234780+0000","last_undegraded":"2024-03-06T07:23:24.234780+0000","last_fullsized":"2024-03-06T07:23:24.234780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236601+0000","last_change":"2024-03-06T07:23:24.236601+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236601+0000","last_undegraded":"2024-03-06T07:23:24.236601+0000","last_fullsized":"2024-03-06T07:23:24.236601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232054+0000","last_change":"2024-03-06T07:23:24.232054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232054+0000","last_undegraded":"2024-03-06T07:23:24.232054+0000","last_fullsized":"2024-03-06T07:23:24.232054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231907+0000","last_change":"2024-03-06T07:23:24.231907+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231907+0000","last_undegraded":"2024-03-06T07:23:24.231907+0000","last_fullsized":"2024-03-06T07:23:24.231907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231901+0000","last_change":"2024-03-06T07:23:24.231901+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231901+0000","last_undegraded":"2024-03-06T07:23:24.231901+0000","last_fullsized":"2024-03-06T07:23:24.231901+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235823+0000","last_change":"2024-03-06T07:23:24.235823+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235823+0000","last_undegraded":"2024-03-06T07:23:24.235823+0000","last_fullsized":"2024-03-06T07:23:24.235823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236857+0000","last_change":"2024-03-06T07:23:24.236857+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236857+0000","last_undegraded":"2024-03-06T07:23:24.236857+0000","last_fullsized":"2024-03-06T07:23:24.236857+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235442+0000","last_change":"2024-03-06T07:23:24.235442+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235442+0000","last_undegraded":"2024-03-06T07:23:24.235442+0000","last_fullsized":"2024-03-06T07:23:24.235442+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234283+0000","last_change":"2024-03-06T07:23:24.234283+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234283+0000","last_undegraded":"2024-03-06T07:23:24.234283+0000","last_fullsized":"2024-03-06T07:23:24.234283+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234978+0000","last_change":"2024-03-06T07:23:24.234978+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234978+0000","last_undegraded":"2024-03-06T07:23:24.234978+0000","last_fullsized":"2024-03-06T07:23:24.234978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232196+0000","last_change":"2024-03-06T07:23:24.232196+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232196+0000","last_undegraded":"2024-03-06T07:23:24.232196+0000","last_fullsized":"2024-03-06T07:23:24.232196+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236484+0000","last_change":"2024-03-06T07:23:24.236484+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236484+0000","last_undegraded":"2024-03-06T07:23:24.236484+0000","last_fullsized":"2024-03-06T07:23:24.236484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231241+0000","last_change":"2024-03-06T07:23:24.231241+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231241+0000","last_undegraded":"2024-03-06T07:23:24.231241+0000","last_fullsized":"2024-03-06T07:23:24.231241+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233724+0000","last_change":"2024-03-06T07:23:24.233724+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233724+0000","last_undegraded":"2024-03-06T07:23:24.233724+0000","last_fullsized":"2024-03-06T07:23:24.233724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233286+0000","last_change":"2024-03-06T07:23:24.233286+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233286+0000","last_undegraded":"2024-03-06T07:23:24.233286+0000","last_fullsized":"2024-03-06T07:23:24.233286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.241335+0000","last_change":"2024-03-06T07:23:24.241335+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.241335+0000","last_undegraded":"2024-03-06T07:23:24.241335+0000","last_fullsized":"2024-03-06T07:23:24.241335+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238590+0000","last_change":"2024-03-06T07:23:24.238590+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238590+0000","last_undegraded":"2024-03-06T07:23:24.238590+0000","last_fullsized":"2024-03-06T07:23:24.238590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234554+0000","last_change":"2024-03-06T07:23:24.234554+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234554+0000","last_undegraded":"2024-03-06T07:23:24.234554+0000","last_fullsized":"2024-03-06T07:23:24.234554+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232453+0000","last_change":"2024-03-06T07:23:24.232453+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232453+0000","last_undegraded":"2024-03-06T07:23:24.232453+0000","last_fullsized":"2024-03-06T07:23:24.232453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237717+0000","last_change":"2024-03-06T07:23:24.237717+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237717+0000","last_undegraded":"2024-03-06T07:23:24.237717+0000","last_fullsized":"2024-03-06T07:23:24.237717+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235413+0000","last_change":"2024-03-06T07:23:24.235413+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235413+0000","last_undegraded":"2024-03-06T07:23:24.235413+0000","last_fullsized":"2024-03-06T07:23:24.235413+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233922+0000","last_change":"2024-03-06T07:23:24.233922+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233922+0000","last_undegraded":"2024-03-06T07:23:24.233922+0000","last_fullsized":"2024-03-06T07:23:24.233922+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250629+0000","last_change":"2024-03-06T07:23:24.250096+0000","last_active":"2024-03-06T07:23:24.250629+0000","last_peered":"2024-03-06T07:23:24.250629+0000","last_clean":"2024-03-06T07:23:24.250629+0000","last_became_active":"2024-03-06T07:23:24.249807+0000","last_became_peered":"2024-03-06T07:23:24.249807+0000","last_unstale":"2024-03-06T07:23:24.250629+0000","last_undegraded":"2024-03-06T07:23:24.250629+0000","last_fullsized":"2024-03-06T07:23:24.250629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:23.940701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232129+0000","last_change":"2024-03-06T07:23:24.232129+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232129+0000","last_undegraded":"2024-03-06T07:23:24.232129+0000","last_fullsized":"2024-03-06T07:23:24.232129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249418+0000","last_change":"2024-03-06T07:23:24.247409+0000","last_active":"2024-03-06T07:23:24.249418+0000","last_peered":"2024-03-06T07:23:24.249418+0000","last_clean":"2024-03-06T07:23:24.249418+0000","last_became_active":"2024-03-06T07:23:24.247083+0000","last_became_peered":"2024-03-06T07:23:24.247083+0000","last_unstale":"2024-03-06T07:23:24.249418+0000","last_undegraded":"2024-03-06T07:23:24.249418+0000","last_fullsized":"2024-03-06T07:23:24.249418+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:32.495551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231919+0000","last_change":"2024-03-06T07:23:24.231919+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231919+0000","last_undegraded":"2024-03-06T07:23:24.231919+0000","last_fullsized":"2024-03-06T07:23:24.231919+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252842+0000","last_change":"2024-03-06T07:23:24.247601+0000","last_active":"2024-03-06T07:23:24.252842+0000","last_peered":"2024-03-06T07:23:24.252842+0000","last_clean":"2024-03-06T07:23:24.252842+0000","last_became_active":"2024-03-06T07:23:24.246790+0000","last_became_peered":"2024-03-06T07:23:24.246790+0000","last_unstale":"2024-03-06T07:23:24.252842+0000","last_undegraded":"2024-03-06T07:23:24.252842+0000","last_fullsized":"2024-03-06T07:23:24.252842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239556+0000","last_change":"2024-03-06T07:23:24.239556+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239556+0000","last_undegraded":"2024-03-06T07:23:24.239556+0000","last_fullsized":"2024-03-06T07:23:24.239556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244586+0000","last_change":"2024-03-06T07:23:24.240691+0000","last_active":"2024-03-06T07:23:24.244586+0000","last_peered":"2024-03-06T07:23:24.244586+0000","last_clean":"2024-03-06T07:23:24.244586+0000","last_became_active":"2024-03-06T07:23:24.240338+0000","last_became_peered":"2024-03-06T07:23:24.240338+0000","last_unstale":"2024-03-06T07:23:24.244586+0000","last_undegraded":"2024-03-06T07:23:24.244586+0000","last_fullsized":"2024-03-06T07:23:24.244586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233786+0000","last_change":"2024-03-06T07:23:24.233786+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233786+0000","last_undegraded":"2024-03-06T07:23:24.233786+0000","last_fullsized":"2024-03-06T07:23:24.233786+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247263+0000","last_change":"2024-03-06T07:23:24.245937+0000","last_active":"2024-03-06T07:23:24.247263+0000","last_peered":"2024-03-06T07:23:24.247263+0000","last_clean":"2024-03-06T07:23:24.247263+0000","last_became_active":"2024-03-06T07:23:24.245614+0000","last_became_peered":"2024-03-06T07:23:24.245614+0000","last_unstale":"2024-03-06T07:23:24.247263+0000","last_undegraded":"2024-03-06T07:23:24.247263+0000","last_fullsized":"2024-03-06T07:23:24.247263+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:26.767861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234462+0000","last_change":"2024-03-06T07:23:24.234462+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234462+0000","last_undegraded":"2024-03-06T07:23:24.234462+0000","last_fullsized":"2024-03-06T07:23:24.234462+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.253182+0000","last_change":"2024-03-06T07:23:24.249153+0000","last_active":"2024-03-06T07:23:24.253182+0000","last_peered":"2024-03-06T07:23:24.253182+0000","last_clean":"2024-03-06T07:23:24.253182+0000","last_became_active":"2024-03-06T07:23:24.248846+0000","last_became_peered":"2024-03-06T07:23:24.248846+0000","last_unstale":"2024-03-06T07:23:24.253182+0000","last_undegraded":"2024-03-06T07:23:24.253182+0000","last_fullsized":"2024-03-06T07:23:24.253182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236089+0000","last_change":"2024-03-06T07:23:24.236089+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236089+0000","last_undegraded":"2024-03-06T07:23:24.236089+0000","last_fullsized":"2024-03-06T07:23:24.236089+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250267+0000","last_change":"2024-03-06T07:23:24.245204+0000","last_active":"2024-03-06T07:23:24.250267+0000","last_peered":"2024-03-06T07:23:24.250267+0000","last_clean":"2024-03-06T07:23:24.250267+0000","last_became_active":"2024-03-06T07:23:24.244893+0000","last_became_peered":"2024-03-06T07:23:24.244893+0000","last_unstale":"2024-03-06T07:23:24.250267+0000","last_undegraded":"2024-03-06T07:23:24.250267+0000","last_fullsized":"2024-03-06T07:23:24.250267+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236223+0000","last_change":"2024-03-06T07:23:24.236223+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236223+0000","last_undegraded":"2024-03-06T07:23:24.236223+0000","last_fullsized":"2024-03-06T07:23:24.236223+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247752+0000","last_change":"2024-03-06T07:23:24.246502+0000","last_active":"2024-03-06T07:23:24.247752+0000","last_peered":"2024-03-06T07:23:24.247752+0000","last_clean":"2024-03-06T07:23:24.247752+0000","last_became_active":"2024-03-06T07:23:24.246230+0000","last_became_peered":"2024-03-06T07:23:24.246230+0000","last_unstale":"2024-03-06T07:23:24.247752+0000","last_undegraded":"2024-03-06T07:23:24.247752+0000","last_fullsized":"2024-03-06T07:23:24.247752+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234549+0000","last_change":"2024-03-06T07:23:24.234549+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234549+0000","last_undegraded":"2024-03-06T07:23:24.234549+0000","last_fullsized":"2024-03-06T07:23:24.234549+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252877+0000","last_change":"2024-03-06T07:23:24.248199+0000","last_active":"2024-03-06T07:23:24.252877+0000","last_peered":"2024-03-06T07:23:24.252877+0000","last_clean":"2024-03-06T07:23:24.252877+0000","last_became_active":"2024-03-06T07:23:24.247705+0000","last_became_peered":"2024-03-06T07:23:24.247705+0000","last_unstale":"2024-03-06T07:23:24.252877+0000","last_undegraded":"2024-03-06T07:23:24.252877+0000","last_fullsized":"2024-03-06T07:23:24.252877+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.421375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231156+0000","last_change":"2024-03-06T07:23:24.231156+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231156+0000","last_undegraded":"2024-03-06T07:23:24.231156+0000","last_fullsized":"2024-03-06T07:23:24.231156+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247760+0000","last_change":"2024-03-06T07:23:24.245504+0000","last_active":"2024-03-06T07:23:24.247760+0000","last_peered":"2024-03-06T07:23:24.247760+0000","last_clean":"2024-03-06T07:23:24.247760+0000","last_became_active":"2024-03-06T07:23:24.245228+0000","last_became_peered":"2024-03-06T07:23:24.245228+0000","last_unstale":"2024-03-06T07:23:24.247760+0000","last_undegraded":"2024-03-06T07:23:24.247760+0000","last_fullsized":"2024-03-06T07:23:24.247760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:42.596673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232421+0000","last_change":"2024-03-06T07:23:24.232421+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232421+0000","last_undegraded":"2024-03-06T07:23:24.232421+0000","last_fullsized":"2024-03-06T07:23:24.232421+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244983+0000","last_change":"2024-03-06T07:23:24.242127+0000","last_active":"2024-03-06T07:23:24.244983+0000","last_peered":"2024-03-06T07:23:24.244983+0000","last_clean":"2024-03-06T07:23:24.244983+0000","last_became_active":"2024-03-06T07:23:24.241524+0000","last_became_peered":"2024-03-06T07:23:24.241524+0000","last_unstale":"2024-03-06T07:23:24.244983+0000","last_undegraded":"2024-03-06T07:23:24.244983+0000","last_fullsized":"2024-03-06T07:23:24.244983+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:42.087045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231299+0000","last_change":"2024-03-06T07:23:24.231299+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231299+0000","last_undegraded":"2024-03-06T07:23:24.231299+0000","last_fullsized":"2024-03-06T07:23:24.231299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252214+0000","last_change":"2024-03-06T07:23:24.246676+0000","last_active":"2024-03-06T07:23:24.252214+0000","last_peered":"2024-03-06T07:23:24.252214+0000","last_clean":"2024-03-06T07:23:24.252214+0000","last_became_active":"2024-03-06T07:23:24.246408+0000","last_became_peered":"2024-03-06T07:23:24.246408+0000","last_unstale":"2024-03-06T07:23:24.252214+0000","last_undegraded":"2024-03-06T07:23:24.252214+0000","last_fullsized":"2024-03-06T07:23:24.252214+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:15.620447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239944+0000","last_change":"2024-03-06T07:23:24.239944+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239944+0000","last_undegraded":"2024-03-06T07:23:24.239944+0000","last_fullsized":"2024-03-06T07:23:24.239944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245809+0000","last_change":"2024-03-06T07:23:24.242187+0000","last_active":"2024-03-06T07:23:24.245809+0000","last_peered":"2024-03-06T07:23:24.245809+0000","last_clean":"2024-03-06T07:23:24.245809+0000","last_became_active":"2024-03-06T07:23:24.241930+0000","last_became_peered":"2024-03-06T07:23:24.241930+0000","last_unstale":"2024-03-06T07:23:24.245809+0000","last_undegraded":"2024-03-06T07:23:24.245809+0000","last_fullsized":"2024-03-06T07:23:24.245809+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236307+0000","last_change":"2024-03-06T07:23:24.236307+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236307+0000","last_undegraded":"2024-03-06T07:23:24.236307+0000","last_fullsized":"2024-03-06T07:23:24.236307+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247677+0000","last_change":"2024-03-06T07:23:24.245524+0000","last_active":"2024-03-06T07:23:24.247677+0000","last_peered":"2024-03-06T07:23:24.247677+0000","last_clean":"2024-03-06T07:23:24.247677+0000","last_became_active":"2024-03-06T07:23:24.245245+0000","last_became_peered":"2024-03-06T07:23:24.245245+0000","last_unstale":"2024-03-06T07:23:24.247677+0000","last_undegraded":"2024-03-06T07:23:24.247677+0000","last_fullsized":"2024-03-06T07:23:24.247677+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234244+0000","last_change":"2024-03-06T07:23:24.234244+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234244+0000","last_undegraded":"2024-03-06T07:23:24.234244+0000","last_fullsized":"2024-03-06T07:23:24.234244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246012+0000","last_change":"2024-03-06T07:23:24.243043+0000","last_active":"2024-03-06T07:23:24.246012+0000","last_peered":"2024-03-06T07:23:24.246012+0000","last_clean":"2024-03-06T07:23:24.246012+0000","last_became_active":"2024-03-06T07:23:24.242793+0000","last_became_peered":"2024-03-06T07:23:24.242793+0000","last_unstale":"2024-03-06T07:23:24.246012+0000","last_undegraded":"2024-03-06T07:23:24.246012+0000","last_fullsized":"2024-03-06T07:23:24.246012+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:12.866720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235408+0000","last_change":"2024-03-06T07:23:24.235408+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235408+0000","last_undegraded":"2024-03-06T07:23:24.235408+0000","last_fullsized":"2024-03-06T07:23:24.235408+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250324+0000","last_change":"2024-03-06T07:23:24.244518+0000","last_active":"2024-03-06T07:23:24.250324+0000","last_peered":"2024-03-06T07:23:24.250324+0000","last_clean":"2024-03-06T07:23:24.250324+0000","last_became_active":"2024-03-06T07:23:24.244154+0000","last_became_peered":"2024-03-06T07:23:24.244154+0000","last_unstale":"2024-03-06T07:23:24.250324+0000","last_undegraded":"2024-03-06T07:23:24.250324+0000","last_fullsized":"2024-03-06T07:23:24.250324+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233094+0000","last_change":"2024-03-06T07:23:24.233094+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233094+0000","last_undegraded":"2024-03-06T07:23:24.233094+0000","last_fullsized":"2024-03-06T07:23:24.233094+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252274+0000","last_change":"2024-03-06T07:23:24.247115+0000","last_active":"2024-03-06T07:23:24.252274+0000","last_peered":"2024-03-06T07:23:24.252274+0000","last_clean":"2024-03-06T07:23:24.252274+0000","last_became_active":"2024-03-06T07:23:24.246835+0000","last_became_peered":"2024-03-06T07:23:24.246835+0000","last_unstale":"2024-03-06T07:23:24.252274+0000","last_undegraded":"2024-03-06T07:23:24.252274+0000","last_fullsized":"2024-03-06T07:23:24.252274+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:40.099312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236125+0000","last_change":"2024-03-06T07:23:24.236125+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236125+0000","last_undegraded":"2024-03-06T07:23:24.236125+0000","last_fullsized":"2024-03-06T07:23:24.236125+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245014+0000","last_change":"2024-03-06T07:23:24.242459+0000","last_active":"2024-03-06T07:23:24.245014+0000","last_peered":"2024-03-06T07:23:24.245014+0000","last_clean":"2024-03-06T07:23:24.245014+0000","last_became_active":"2024-03-06T07:23:24.241848+0000","last_became_peered":"2024-03-06T07:23:24.241848+0000","last_unstale":"2024-03-06T07:23:24.245014+0000","last_undegraded":"2024-03-06T07:23:24.245014+0000","last_fullsized":"2024-03-06T07:23:24.245014+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:01.845106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231767+0000","last_change":"2024-03-06T07:23:24.231767+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231767+0000","last_undegraded":"2024-03-06T07:23:24.231767+0000","last_fullsized":"2024-03-06T07:23:24.231767+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.255498+0000","last_change":"2024-03-06T07:23:24.252717+0000","last_active":"2024-03-06T07:23:24.255498+0000","last_peered":"2024-03-06T07:23:24.255498+0000","last_clean":"2024-03-06T07:23:24.255498+0000","last_became_active":"2024-03-06T07:23:24.252429+0000","last_became_peered":"2024-03-06T07:23:24.252429+0000","last_unstale":"2024-03-06T07:23:24.255498+0000","last_undegraded":"2024-03-06T07:23:24.255498+0000","last_fullsized":"2024-03-06T07:23:24.255498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234015+0000","last_change":"2024-03-06T07:23:24.234015+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234015+0000","last_undegraded":"2024-03-06T07:23:24.234015+0000","last_fullsized":"2024-03-06T07:23:24.234015+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249758+0000","last_change":"2024-03-06T07:23:24.248709+0000","last_active":"2024-03-06T07:23:24.249758+0000","last_peered":"2024-03-06T07:23:24.249758+0000","last_clean":"2024-03-06T07:23:24.249758+0000","last_became_active":"2024-03-06T07:23:24.247184+0000","last_became_peered":"2024-03-06T07:23:24.247184+0000","last_unstale":"2024-03-06T07:23:24.249758+0000","last_undegraded":"2024-03-06T07:23:24.249758+0000","last_fullsized":"2024-03-06T07:23:24.249758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:10.470563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234900+0000","last_change":"2024-03-06T07:23:24.234900+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234900+0000","last_undegraded":"2024-03-06T07:23:24.234900+0000","last_fullsized":"2024-03-06T07:23:24.234900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249143+0000","last_change":"2024-03-06T07:23:24.246510+0000","last_active":"2024-03-06T07:23:24.249143+0000","last_peered":"2024-03-06T07:23:24.249143+0000","last_clean":"2024-03-06T07:23:24.249143+0000","last_became_active":"2024-03-06T07:23:24.246222+0000","last_became_peered":"2024-03-06T07:23:24.246222+0000","last_unstale":"2024-03-06T07:23:24.249143+0000","last_undegraded":"2024-03-06T07:23:24.249143+0000","last_fullsized":"2024-03-06T07:23:24.249143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:26.098267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235380+0000","last_change":"2024-03-06T07:23:24.235380+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235380+0000","last_undegraded":"2024-03-06T07:23:24.235380+0000","last_fullsized":"2024-03-06T07:23:24.235380+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249498+0000","last_change":"2024-03-06T07:23:24.244174+0000","last_active":"2024-03-06T07:23:24.249498+0000","last_peered":"2024-03-06T07:23:24.249498+0000","last_clean":"2024-03-06T07:23:24.249498+0000","last_became_active":"2024-03-06T07:23:24.243046+0000","last_became_peered":"2024-03-06T07:23:24.243046+0000","last_unstale":"2024-03-06T07:23:24.249498+0000","last_undegraded":"2024-03-06T07:23:24.249498+0000","last_fullsized":"2024-03-06T07:23:24.249498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:18.162769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233848+0000","last_change":"2024-03-06T07:23:24.233848+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233848+0000","last_undegraded":"2024-03-06T07:23:24.233848+0000","last_fullsized":"2024-03-06T07:23:24.233848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246844+0000","last_change":"2024-03-06T07:23:24.245809+0000","last_active":"2024-03-06T07:23:24.246844+0000","last_peered":"2024-03-06T07:23:24.246844+0000","last_clean":"2024-03-06T07:23:24.246844+0000","last_became_active":"2024-03-06T07:23:24.245527+0000","last_became_peered":"2024-03-06T07:23:24.245527+0000","last_unstale":"2024-03-06T07:23:24.246844+0000","last_undegraded":"2024-03-06T07:23:24.246844+0000","last_fullsized":"2024-03-06T07:23:24.246844+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:15.139265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234987+0000","last_change":"2024-03-06T07:23:24.234987+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234987+0000","last_undegraded":"2024-03-06T07:23:24.234987+0000","last_fullsized":"2024-03-06T07:23:24.234987+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244456+0000","last_change":"2024-03-06T07:23:24.242259+0000","last_active":"2024-03-06T07:23:24.244456+0000","last_peered":"2024-03-06T07:23:24.244456+0000","last_clean":"2024-03-06T07:23:24.244456+0000","last_became_active":"2024-03-06T07:23:24.241958+0000","last_became_peered":"2024-03-06T07:23:24.241958+0000","last_unstale":"2024-03-06T07:23:24.244456+0000","last_undegraded":"2024-03-06T07:23:24.244456+0000","last_fullsized":"2024-03-06T07:23:24.244456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232370+0000","last_change":"2024-03-06T07:23:24.232370+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232370+0000","last_undegraded":"2024-03-06T07:23:24.232370+0000","last_fullsized":"2024-03-06T07:23:24.232370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247783+0000","last_change":"2024-03-06T07:23:24.246996+0000","last_active":"2024-03-06T07:23:24.247783+0000","last_peered":"2024-03-06T07:23:24.247783+0000","last_clean":"2024-03-06T07:23:24.247783+0000","last_became_active":"2024-03-06T07:23:24.246685+0000","last_became_peered":"2024-03-06T07:23:24.246685+0000","last_unstale":"2024-03-06T07:23:24.247783+0000","last_undegraded":"2024-03-06T07:23:24.247783+0000","last_fullsized":"2024-03-06T07:23:24.247783+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:48:35.768778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235450+0000","last_change":"2024-03-06T07:23:24.235450+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235450+0000","last_undegraded":"2024-03-06T07:23:24.235450+0000","last_fullsized":"2024-03-06T07:23:24.235450+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252103+0000","last_change":"2024-03-06T07:23:24.244748+0000","last_active":"2024-03-06T07:23:24.252103+0000","last_peered":"2024-03-06T07:23:24.252103+0000","last_clean":"2024-03-06T07:23:24.252103+0000","last_became_active":"2024-03-06T07:23:24.244211+0000","last_became_peered":"2024-03-06T07:23:24.244211+0000","last_unstale":"2024-03-06T07:23:24.252103+0000","last_undegraded":"2024-03-06T07:23:24.252103+0000","last_fullsized":"2024-03-06T07:23:24.252103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:21.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240054+0000","last_change":"2024-03-06T07:23:24.240054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240054+0000","last_undegraded":"2024-03-06T07:23:24.240054+0000","last_fullsized":"2024-03-06T07:23:24.240054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250382+0000","last_change":"2024-03-06T07:23:24.248695+0000","last_active":"2024-03-06T07:23:24.250382+0000","last_peered":"2024-03-06T07:23:24.250382+0000","last_clean":"2024-03-06T07:23:24.250382+0000","last_became_active":"2024-03-06T07:23:24.248417+0000","last_became_peered":"2024-03-06T07:23:24.248417+0000","last_unstale":"2024-03-06T07:23:24.250382+0000","last_undegraded":"2024-03-06T07:23:24.250382+0000","last_fullsized":"2024-03-06T07:23:24.250382+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233262+0000","last_change":"2024-03-06T07:23:24.233262+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233262+0000","last_undegraded":"2024-03-06T07:23:24.233262+0000","last_fullsized":"2024-03-06T07:23:24.233262+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250297+0000","last_change":"2024-03-06T07:23:24.246199+0000","last_active":"2024-03-06T07:23:24.250297+0000","last_peered":"2024-03-06T07:23:24.250297+0000","last_clean":"2024-03-06T07:23:24.250297+0000","last_became_active":"2024-03-06T07:23:24.245854+0000","last_became_peered":"2024-03-06T07:23:24.245854+0000","last_unstale":"2024-03-06T07:23:24.250297+0000","last_undegraded":"2024-03-06T07:23:24.250297+0000","last_fullsized":"2024-03-06T07:23:24.250297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232832+0000","last_change":"2024-03-06T07:23:24.232832+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232832+0000","last_undegraded":"2024-03-06T07:23:24.232832+0000","last_fullsized":"2024-03-06T07:23:24.232832+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246395+0000","last_change":"2024-03-06T07:23:24.244313+0000","last_active":"2024-03-06T07:23:24.246395+0000","last_peered":"2024-03-06T07:23:24.246395+0000","last_clean":"2024-03-06T07:23:24.246395+0000","last_became_active":"2024-03-06T07:23:24.244059+0000","last_became_peered":"2024-03-06T07:23:24.244059+0000","last_unstale":"2024-03-06T07:23:24.246395+0000","last_undegraded":"2024-03-06T07:23:24.246395+0000","last_fullsized":"2024-03-06T07:23:24.246395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:53.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232253+0000","last_change":"2024-03-06T07:23:24.232253+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232253+0000","last_undegraded":"2024-03-06T07:23:24.232253+0000","last_fullsized":"2024-03-06T07:23:24.232253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252266+0000","last_change":"2024-03-06T07:23:24.247137+0000","last_active":"2024-03-06T07:23:24.252266+0000","last_peered":"2024-03-06T07:23:24.252266+0000","last_clean":"2024-03-06T07:23:24.252266+0000","last_became_active":"2024-03-06T07:23:24.246871+0000","last_became_peered":"2024-03-06T07:23:24.246871+0000","last_unstale":"2024-03-06T07:23:24.252266+0000","last_undegraded":"2024-03-06T07:23:24.252266+0000","last_fullsized":"2024-03-06T07:23:24.252266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:15:06.289086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242099+0000","last_change":"2024-03-06T07:23:24.242099+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242099+0000","last_undegraded":"2024-03-06T07:23:24.242099+0000","last_fullsized":"2024-03-06T07:23:24.242099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245965+0000","last_change":"2024-03-06T07:23:24.243574+0000","last_active":"2024-03-06T07:23:24.245965+0000","last_peered":"2024-03-06T07:23:24.245965+0000","last_clean":"2024-03-06T07:23:24.245965+0000","last_became_active":"2024-03-06T07:23:24.243297+0000","last_became_peered":"2024-03-06T07:23:24.243297+0000","last_unstale":"2024-03-06T07:23:24.245965+0000","last_undegraded":"2024-03-06T07:23:24.245965+0000","last_fullsized":"2024-03-06T07:23:24.245965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:05.404876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240960+0000","last_change":"2024-03-06T07:23:24.240960+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240960+0000","last_undegraded":"2024-03-06T07:23:24.240960+0000","last_fullsized":"2024-03-06T07:23:24.240960+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247260+0000","last_change":"2024-03-06T07:23:24.244996+0000","last_active":"2024-03-06T07:23:24.247260+0000","last_peered":"2024-03-06T07:23:24.247260+0000","last_clean":"2024-03-06T07:23:24.247260+0000","last_became_active":"2024-03-06T07:23:24.244721+0000","last_became_peered":"2024-03-06T07:23:24.244721+0000","last_unstale":"2024-03-06T07:23:24.247260+0000","last_undegraded":"2024-03-06T07:23:24.247260+0000","last_fullsized":"2024-03-06T07:23:24.247260+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:23.742370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.233201+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:23:24.233201+0000","last_peered":"2024-03-06T07:23:24.233201+0000","last_clean":"2024-03-06T07:23:24.233201+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:23:24.233201+0000","last_undegraded":"2024-03-06T07:23:24.233201+0000","last_fullsized":"2024-03-06T07:23:24.233201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1142784,"data_stored":549035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:28.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2024-03-06T07:23:28.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236216+0000', 'last_change': '2024-03-06T07:23:24.236216+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236216+0000', 'last_undegraded': '2024-03-06T07:23:24.236216+0000', 'last_fullsized': '2024-03-06T07:23:24.236216+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.237324+0000', 'last_change': '2024-03-06T07:23:24.237324+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.237324+0000', 'last_undegraded': '2024-03-06T07:23:24.237324+0000', 'last_fullsized': '2024-03-06T07:23:24.237324+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235853+0000', 'last_change': '2024-03-06T07:23:24.235853+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235853+0000', 'last_undegraded': '2024-03-06T07:23:24.235853+0000', 'last_fullsized': '2024-03-06T07:23:24.235853+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234332+0000', 'last_change': '2024-03-06T07:23:24.234332+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234332+0000', 'last_undegraded': '2024-03-06T07:23:24.234332+0000', 'last_fullsized': '2024-03-06T07:23:24.234332+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2024-03-06T07:23:28.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233017+0000', 'last_change': '2024-03-06T07:23:24.233017+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233017+0000', 'last_undegraded': '2024-03-06T07:23:24.233017+0000', 'last_fullsized': '2024-03-06T07:23:24.233017+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236616+0000', 'last_change': '2024-03-06T07:23:24.236616+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236616+0000', 'last_undegraded': '2024-03-06T07:23:24.236616+0000', 'last_fullsized': '2024-03-06T07:23:24.236616+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.238220+0000', 'last_change': '2024-03-06T07:23:24.238220+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.238220+0000', 'last_undegraded': '2024-03-06T07:23:24.238220+0000', 'last_fullsized': '2024-03-06T07:23:24.238220+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235822+0000', 'last_change': '2024-03-06T07:23:24.235822+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235822+0000', 'last_undegraded': '2024-03-06T07:23:24.235822+0000', 'last_fullsized': '2024-03-06T07:23:24.235822+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2024-03-06T07:23:28.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236313+0000', 'last_change': '2024-03-06T07:23:24.236313+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236313+0000', 'last_undegraded': '2024-03-06T07:23:24.236313+0000', 'last_fullsized': '2024-03-06T07:23:24.236313+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232154+0000', 'last_change': '2024-03-06T07:23:24.232154+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232154+0000', 'last_undegraded': '2024-03-06T07:23:24.232154+0000', 'last_fullsized': '2024-03-06T07:23:24.232154+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.242162+0000', 'last_change': '2024-03-06T07:23:24.242162+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.242162+0000', 'last_undegraded': '2024-03-06T07:23:24.242162+0000', 'last_fullsized': '2024-03-06T07:23:24.242162+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234780+0000', 'last_change': '2024-03-06T07:23:24.234780+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234780+0000', 'last_undegraded': '2024-03-06T07:23:24.234780+0000', 'last_fullsized': '2024-03-06T07:23:24.234780+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236601+0000', 'last_change': '2024-03-06T07:23:24.236601+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236601+0000', 'last_undegraded': '2024-03-06T07:23:24.236601+0000', 'last_fullsized': '2024-03-06T07:23:24.236601+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232054+0000', 'last_change': '2024-03-06T07:23:24.232054+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232054+0000', 'last_undegraded': '2024-03-06T07:23:24.232054+0000', 'last_fullsized': '2024-03-06T07:23:24.232054+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2024-03-06T07:23:28.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.231907+0000', 'last_change': '2024-03-06T07:23:24.231907+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.231907+0000', 'last_undegraded': '2024-03-06T07:23:24.231907+0000', 'last_fullsized': '2024-03-06T07:23:24.231907+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.231901+0000', 'last_change': '2024-03-06T07:23:24.231901+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.231901+0000', 'last_undegraded': '2024-03-06T07:23:24.231901+0000', 'last_fullsized': '2024-03-06T07:23:24.231901+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235823+0000', 'last_change': '2024-03-06T07:23:24.235823+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235823+0000', 'last_undegraded': '2024-03-06T07:23:24.235823+0000', 'last_fullsized': '2024-03-06T07:23:24.235823+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236857+0000', 'last_change': '2024-03-06T07:23:24.236857+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236857+0000', 'last_undegraded': '2024-03-06T07:23:24.236857+0000', 'last_fullsized': '2024-03-06T07:23:24.236857+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235442+0000', 'last_change': '2024-03-06T07:23:24.235442+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235442+0000', 'last_undegraded': '2024-03-06T07:23:24.235442+0000', 'last_fullsized': '2024-03-06T07:23:24.235442+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2024-03-06T07:23:28.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234283+0000', 'last_change': '2024-03-06T07:23:24.234283+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234283+0000', 'last_undegraded': '2024-03-06T07:23:24.234283+0000', 'last_fullsized': '2024-03-06T07:23:24.234283+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234978+0000', 'last_change': '2024-03-06T07:23:24.234978+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234978+0000', 'last_undegraded': '2024-03-06T07:23:24.234978+0000', 'last_fullsized': '2024-03-06T07:23:24.234978+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232196+0000', 'last_change': '2024-03-06T07:23:24.232196+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232196+0000', 'last_undegraded': '2024-03-06T07:23:24.232196+0000', 'last_fullsized': '2024-03-06T07:23:24.232196+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236484+0000', 'last_change': '2024-03-06T07:23:24.236484+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236484+0000', 'last_undegraded': '2024-03-06T07:23:24.236484+0000', 'last_fullsized': '2024-03-06T07:23:24.236484+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2024-03-06T07:23:28.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.231241+0000', 'last_change': '2024-03-06T07:23:24.231241+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.231241+0000', 'last_undegraded': '2024-03-06T07:23:24.231241+0000', 'last_fullsized': '2024-03-06T07:23:24.231241+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233724+0000', 'last_change': '2024-03-06T07:23:24.233724+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233724+0000', 'last_undegraded': '2024-03-06T07:23:24.233724+0000', 'last_fullsized': '2024-03-06T07:23:24.233724+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233286+0000', 'last_change': '2024-03-06T07:23:24.233286+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233286+0000', 'last_undegraded': '2024-03-06T07:23:24.233286+0000', 'last_fullsized': '2024-03-06T07:23:24.233286+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.241335+0000', 'last_change': '2024-03-06T07:23:24.241335+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.241335+0000', 'last_undegraded': '2024-03-06T07:23:24.241335+0000', 'last_fullsized': '2024-03-06T07:23:24.241335+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.238590+0000', 'last_change': '2024-03-06T07:23:24.238590+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.238590+0000', 'last_undegraded': '2024-03-06T07:23:24.238590+0000', 'last_fullsized': '2024-03-06T07:23:24.238590+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234554+0000', 'last_change': '2024-03-06T07:23:24.234554+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234554+0000', 'last_undegraded': '2024-03-06T07:23:24.234554+0000', 'last_fullsized': '2024-03-06T07:23:24.234554+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232453+0000', 'last_change': '2024-03-06T07:23:24.232453+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232453+0000', 'last_undegraded': '2024-03-06T07:23:24.232453+0000', 'last_fullsized': '2024-03-06T07:23:24.232453+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.237717+0000', 'last_change': '2024-03-06T07:23:24.237717+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.237717+0000', 'last_undegraded': '2024-03-06T07:23:24.237717+0000', 'last_fullsized': '2024-03-06T07:23:24.237717+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235413+0000', 'last_change': '2024-03-06T07:23:24.235413+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235413+0000', 'last_undegraded': '2024-03-06T07:23:24.235413+0000', 'last_fullsized': '2024-03-06T07:23:24.235413+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233922+0000', 'last_change': '2024-03-06T07:23:24.233922+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233922+0000', 'last_undegraded': '2024-03-06T07:23:24.233922+0000', 'last_fullsized': '2024-03-06T07:23:24.233922+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232129+0000', 'last_change': '2024-03-06T07:23:24.232129+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232129+0000', 'last_undegraded': '2024-03-06T07:23:24.232129+0000', 'last_fullsized': '2024-03-06T07:23:24.232129+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2024-03-06T07:23:28.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.231919+0000', 'last_change': '2024-03-06T07:23:24.231919+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.231919+0000', 'last_undegraded': '2024-03-06T07:23:24.231919+0000', 'last_fullsized': '2024-03-06T07:23:24.231919+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.239556+0000', 'last_change': '2024-03-06T07:23:24.239556+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.239556+0000', 'last_undegraded': '2024-03-06T07:23:24.239556+0000', 'last_fullsized': '2024-03-06T07:23:24.239556+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233786+0000', 'last_change': '2024-03-06T07:23:24.233786+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233786+0000', 'last_undegraded': '2024-03-06T07:23:24.233786+0000', 'last_fullsized': '2024-03-06T07:23:24.233786+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234462+0000', 'last_change': '2024-03-06T07:23:24.234462+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234462+0000', 'last_undegraded': '2024-03-06T07:23:24.234462+0000', 'last_fullsized': '2024-03-06T07:23:24.234462+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236089+0000', 'last_change': '2024-03-06T07:23:24.236089+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236089+0000', 'last_undegraded': '2024-03-06T07:23:24.236089+0000', 'last_fullsized': '2024-03-06T07:23:24.236089+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-03-06T07:23:28.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236223+0000', 'last_change': '2024-03-06T07:23:24.236223+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236223+0000', 'last_undegraded': '2024-03-06T07:23:24.236223+0000', 'last_fullsized': '2024-03-06T07:23:24.236223+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234549+0000', 'last_change': '2024-03-06T07:23:24.234549+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234549+0000', 'last_undegraded': '2024-03-06T07:23:24.234549+0000', 'last_fullsized': '2024-03-06T07:23:24.234549+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.231156+0000', 'last_change': '2024-03-06T07:23:24.231156+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.231156+0000', 'last_undegraded': '2024-03-06T07:23:24.231156+0000', 'last_fullsized': '2024-03-06T07:23:24.231156+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232421+0000', 'last_change': '2024-03-06T07:23:24.232421+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232421+0000', 'last_undegraded': '2024-03-06T07:23:24.232421+0000', 'last_fullsized': '2024-03-06T07:23:24.232421+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.231299+0000', 'last_change': '2024-03-06T07:23:24.231299+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.231299+0000', 'last_undegraded': '2024-03-06T07:23:24.231299+0000', 'last_fullsized': '2024-03-06T07:23:24.231299+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.239944+0000', 'last_change': '2024-03-06T07:23:24.239944+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.239944+0000', 'last_undegraded': '2024-03-06T07:23:24.239944+0000', 'last_fullsized': '2024-03-06T07:23:24.239944+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:23:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236307+0000', 'last_change': '2024-03-06T07:23:24.236307+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236307+0000', 'last_undegraded': '2024-03-06T07:23:24.236307+0000', 'last_fullsized': '2024-03-06T07:23:24.236307+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234244+0000', 'last_change': '2024-03-06T07:23:24.234244+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234244+0000', 'last_undegraded': '2024-03-06T07:23:24.234244+0000', 'last_fullsized': '2024-03-06T07:23:24.234244+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235408+0000', 'last_change': '2024-03-06T07:23:24.235408+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235408+0000', 'last_undegraded': '2024-03-06T07:23:24.235408+0000', 'last_fullsized': '2024-03-06T07:23:24.235408+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233094+0000', 'last_change': '2024-03-06T07:23:24.233094+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233094+0000', 'last_undegraded': '2024-03-06T07:23:24.233094+0000', 'last_fullsized': '2024-03-06T07:23:24.233094+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.236125+0000', 'last_change': '2024-03-06T07:23:24.236125+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.236125+0000', 'last_undegraded': '2024-03-06T07:23:24.236125+0000', 'last_fullsized': '2024-03-06T07:23:24.236125+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:23:28.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.231767+0000', 'last_change': '2024-03-06T07:23:24.231767+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.231767+0000', 'last_undegraded': '2024-03-06T07:23:24.231767+0000', 'last_fullsized': '2024-03-06T07:23:24.231767+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234015+0000', 'last_change': '2024-03-06T07:23:24.234015+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234015+0000', 'last_undegraded': '2024-03-06T07:23:24.234015+0000', 'last_fullsized': '2024-03-06T07:23:24.234015+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234900+0000', 'last_change': '2024-03-06T07:23:24.234900+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234900+0000', 'last_undegraded': '2024-03-06T07:23:24.234900+0000', 'last_fullsized': '2024-03-06T07:23:24.234900+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235380+0000', 'last_change': '2024-03-06T07:23:24.235380+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235380+0000', 'last_undegraded': '2024-03-06T07:23:24.235380+0000', 'last_fullsized': '2024-03-06T07:23:24.235380+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233848+0000', 'last_change': '2024-03-06T07:23:24.233848+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233848+0000', 'last_undegraded': '2024-03-06T07:23:24.233848+0000', 'last_fullsized': '2024-03-06T07:23:24.233848+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.234987+0000', 'last_change': '2024-03-06T07:23:24.234987+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.234987+0000', 'last_undegraded': '2024-03-06T07:23:24.234987+0000', 'last_fullsized': '2024-03-06T07:23:24.234987+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232370+0000', 'last_change': '2024-03-06T07:23:24.232370+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232370+0000', 'last_undegraded': '2024-03-06T07:23:24.232370+0000', 'last_fullsized': '2024-03-06T07:23:24.232370+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.235450+0000', 'last_change': '2024-03-06T07:23:24.235450+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.235450+0000', 'last_undegraded': '2024-03-06T07:23:24.235450+0000', 'last_fullsized': '2024-03-06T07:23:24.235450+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.240054+0000', 'last_change': '2024-03-06T07:23:24.240054+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.240054+0000', 'last_undegraded': '2024-03-06T07:23:24.240054+0000', 'last_fullsized': '2024-03-06T07:23:24.240054+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.233262+0000', 'last_change': '2024-03-06T07:23:24.233262+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.233262+0000', 'last_undegraded': '2024-03-06T07:23:24.233262+0000', 'last_fullsized': '2024-03-06T07:23:24.233262+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-03-06T07:23:28.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232832+0000', 'last_change': '2024-03-06T07:23:24.232832+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232832+0000', 'last_undegraded': '2024-03-06T07:23:24.232832+0000', 'last_fullsized': '2024-03-06T07:23:24.232832+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:23:28.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2024-03-06T07:23:28.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.232253+0000', 'last_change': '2024-03-06T07:23:24.232253+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.232253+0000', 'last_undegraded': '2024-03-06T07:23:24.232253+0000', 'last_fullsized': '2024-03-06T07:23:24.232253+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:23:28.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2024-03-06T07:23:28.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.242099+0000', 'last_change': '2024-03-06T07:23:24.242099+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.242099+0000', 'last_undegraded': '2024-03-06T07:23:24.242099+0000', 'last_fullsized': '2024-03-06T07:23:24.242099+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:23:28.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2024-03-06T07:23:28.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:23:24.240960+0000', 'last_change': '2024-03-06T07:23:24.240960+0000', 'last_active': '2024-03-06T07:23:24.223485+0000', 'last_peered': '2024-03-06T07:23:24.223485+0000', 'last_clean': '2024-03-06T07:23:24.223485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:23:24.240960+0000', 'last_undegraded': '2024-03-06T07:23:24.240960+0000', 'last_fullsized': '2024-03-06T07:23:24.240960+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'last_clean_scrub_stamp': '2024-03-06T07:23:24.223485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:23:28.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:23:29.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.094+0000 7fde263c9640 1 Processor -- start 2024-03-06T07:23:29.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.094+0000 7fde263c9640 1 -- start start 2024-03-06T07:23:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.094+0000 7fde263c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde2006d9f0 0x7fde200a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.094+0000 7fde263c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde20158800 con 0x7fde200c3280 2024-03-06T07:23:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.094+0000 7fde263c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde20158ae0 con 0x7fde200c36e0 2024-03-06T07:23:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.094+0000 7fde263c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde200c3b10 con 0x7fde2006d9f0 2024-03-06T07:23:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.095+0000 7fde1f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde200c3280 0x7fde201551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34846/0 (socket says 172.21.15.26:34846) 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.095+0000 7fde1f7fe640 1 -- 172.21.15.26:0/4050866501 learned_addr learned my addr 172.21.15.26:0/4050866501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.095+0000 7fde1ffff640 1 --2- 172.21.15.26:0/4050866501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde2006d9f0 0x7fde200a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.095+0000 7fde1effd640 1 -- 172.21.15.26:0/4050866501 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845760398 0 0) 0x7fde20158800 con 0x7fde200c3280 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.095+0000 7fde1ffff640 1 -- 172.21.15.26:0/4050866501 >> v1:172.21.15.117:6789/0 conn(0x7fde200c36e0 legacy=0x7fde201558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.095+0000 7fde1ffff640 1 -- 172.21.15.26:0/4050866501 >> v1:172.21.15.26:6789/0 conn(0x7fde200c3280 legacy=0x7fde201551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.095+0000 7fde1ffff640 1 -- 172.21.15.26:0/4050866501 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde20158dc0 con 0x7fde2006d9f0 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde1ffff640 1 --2- 172.21.15.26:0/4050866501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde2006d9f0 0x7fde200a7cd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fde10009750 tx=0x7fde1002b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e115d1773c4331 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde1effd640 1 -- 172.21.15.26:0/4050866501 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10037070 con 0x7fde2006d9f0 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde1effd640 1 -- 172.21.15.26:0/4050866501 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10030710 con 0x7fde2006d9f0 2024-03-06T07:23:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde1effd640 1 -- 172.21.15.26:0/4050866501 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10030ae0 con 0x7fde2006d9f0 2024-03-06T07:23:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde263c9640 1 -- 172.21.15.26:0/4050866501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde2006d9f0 msgr2=0x7fde200a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde263c9640 1 --2- 172.21.15.26:0/4050866501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde2006d9f0 0x7fde200a7cd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fde10009750 tx=0x7fde1002b600 comp rx=0 tx=0).stop 2024-03-06T07:23:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde263c9640 1 -- 172.21.15.26:0/4050866501 shutdown_connections 2024-03-06T07:23:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde263c9640 1 --2- 172.21.15.26:0/4050866501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde2006d9f0 0x7fde200a7cd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde263c9640 1 -- 172.21.15.26:0/4050866501 >> 172.21.15.26:0/4050866501 conn(0x7fde200aa630 msgr2=0x7fde200aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde263c9640 1 -- 172.21.15.26:0/4050866501 shutdown_connections 2024-03-06T07:23:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.096+0000 7fde263c9640 1 -- 172.21.15.26:0/4050866501 wait complete. 2024-03-06T07:23:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 Processor -- start 2024-03-06T07:23:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 -- start start 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde2006d9f0 0x7fde2009b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde200c3280 0x7fde20157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde200c36e0 0x7fde2014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2006ddd0 con 0x7fde200c36e0 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde20159380 con 0x7fde2006d9f0 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.098+0000 7fde263c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2006df50 con 0x7fde200c3280 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde200c36e0 0x7fde2014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde200c36e0 0x7fde2014ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49482/0 (socket says 172.21.15.26:49482) 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 -- 172.21.15.26:0/3462450841 learned_addr learned my addr 172.21.15.26:0/3462450841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde1f7fe640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde200c3280 0x7fde20157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde1ffff640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde2006d9f0 0x7fde2009b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 -- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde200c3280 msgr2=0x7fde20157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde200c3280 0x7fde20157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 -- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde2006d9f0 msgr2=0x7fde2009b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde2006d9f0 0x7fde2009b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 -- 172.21.15.26:0/3462450841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde201eb210 con 0x7fde200c36e0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde24966640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde200c36e0 0x7fde2014ed50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c7f0 tx=0x7fde1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde1cff9640 1 -- 172.21.15.26:0/3462450841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18017070 con 0x7fde200c36e0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde1cff9640 1 -- 172.21.15.26:0/3462450841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18003da0 con 0x7fde200c36e0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde201eb4b0 con 0x7fde200c36e0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde1cff9640 1 -- 172.21.15.26:0/3462450841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18005290 con 0x7fde200c36e0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.099+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde201582a0 con 0x7fde200c36e0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.100+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2004f330 con 0x7fde200c36e0 2024-03-06T07:23:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.101+0000 7fde1cff9640 1 -- 172.21.15.26:0/3462450841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde1801a070 con 0x7fde200c36e0 2024-03-06T07:23:29.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.101+0000 7fde1cff9640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde0c06b4d0 0x7fde0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.101+0000 7fde1cff9640 1 -- 172.21.15.26:0/3462450841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde18089960 con 0x7fde200c36e0 2024-03-06T07:23:29.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.101+0000 7fde1ffff640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde0c06b4d0 0x7fde0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.102+0000 7fde1ffff640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde0c06b4d0 0x7fde0c06d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fde10034000 tx=0x7fde10008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.105+0000 7fde1cff9640 1 -- 172.21.15.26:0/3462450841 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fde18058090 con 0x7fde200c36e0 2024-03-06T07:23:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.359+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fde201edca0 con 0x7fde200c36e0 2024-03-06T07:23:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.360+0000 7fde1cff9640 1 -- 172.21.15.26:0/3462450841 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1454 (secure 0 0 0) 0x7fde1805c130 con 0x7fde200c36e0 2024-03-06T07:23:29.362 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:29.362 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":224,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709595,"num_in_osds":8,"osd_in_since":1709709586,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":256303104,"bytes_avail":772837810176,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":77,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":6966}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-03-06T07:23:13.069747+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:23:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde0c06b4d0 msgr2=0x7fde0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde0c06b4d0 0x7fde0c06d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fde10034000 tx=0x7fde10008030 comp rx=0 tx=0).stop 2024-03-06T07:23:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde200c36e0 msgr2=0x7fde2014ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde200c36e0 0x7fde2014ed50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c7f0 tx=0x7fde1800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 shutdown_connections 2024-03-06T07:23:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde0c06b4d0 0x7fde0c06d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde200c36e0 0x7fde2014ed50 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde200c3280 0x7fde20157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 --2- 172.21.15.26:0/3462450841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde2006d9f0 0x7fde2009b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 >> 172.21.15.26:0/3462450841 conn(0x7fde200aa630 msgr2=0x7fde20154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 shutdown_connections 2024-03-06T07:23:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.362+0000 7fde263c9640 1 -- 172.21.15.26:0/3462450841 wait complete. 2024-03-06T07:23:29.375 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:23:29.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.475+0000 7feb54913640 1 Processor -- start 2024-03-06T07:23:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.475+0000 7feb54913640 1 -- start start 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.475+0000 7feb54913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb5006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.475+0000 7feb54913640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb500720a0 0x7feb5006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.475+0000 7feb54913640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb500c1ae0 con 0x7feb50155c80 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.475+0000 7feb54913640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb500c1dc0 con 0x7feb500720a0 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.475+0000 7feb54913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5006e450 con 0x7feb50152520 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb500720a0 0x7feb5006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb500720a0 0x7feb5006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49484/0 (socket says 172.21.15.26:49484) 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4ed9d640 1 -- 172.21.15.26:0/3506330527 learned_addr learned my addr 172.21.15.26:0/3506330527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4e59c640 1 --2- 172.21.15.26:0/3506330527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb5006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4ed9d640 1 -- 172.21.15.26:0/3506330527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 msgr2=0x7feb5006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4ed9d640 1 --2- 172.21.15.26:0/3506330527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb5006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4ed9d640 1 -- 172.21.15.26:0/3506330527 >> v1:172.21.15.117:6789/0 conn(0x7feb50155c80 legacy=0x7feb500c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4dd9b640 1 -- 172.21.15.26:0/3506330527 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992157493 0 0) 0x7feb500c1ae0 con 0x7feb50155c80 2024-03-06T07:23:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.476+0000 7feb4ed9d640 1 -- 172.21.15.26:0/3506330527 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5006e730 con 0x7feb500720a0 2024-03-06T07:23:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb4ed9d640 1 --2- 172.21.15.26:0/3506330527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb5006afc0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7feb48009e50 tx=0x7feb4802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613c929d1a8a645e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb4dd9b640 1 -- 172.21.15.26:0/3506330527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4802cb00 con 0x7feb500720a0 2024-03-06T07:23:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb4dd9b640 1 -- 172.21.15.26:0/3506330527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4802cca0 con 0x7feb500720a0 2024-03-06T07:23:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb4dd9b640 1 -- 172.21.15.26:0/3506330527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48034610 con 0x7feb500720a0 2024-03-06T07:23:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 -- 172.21.15.26:0/3506330527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 msgr2=0x7feb5006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 --2- 172.21.15.26:0/3506330527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb5006afc0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7feb48009e50 tx=0x7feb4802bf60 comp rx=0 tx=0).stop 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 -- 172.21.15.26:0/3506330527 shutdown_connections 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 --2- 172.21.15.26:0/3506330527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb5006afc0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 --2- 172.21.15.26:0/3506330527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb5006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 -- 172.21.15.26:0/3506330527 >> 172.21.15.26:0/3506330527 conn(0x7feb500c76b0 msgr2=0x7feb500c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 -- 172.21.15.26:0/3506330527 shutdown_connections 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.477+0000 7feb54913640 1 -- 172.21.15.26:0/3506330527 wait complete. 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 Processor -- start 2024-03-06T07:23:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 -- start start 2024-03-06T07:23:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb500a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb500a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb50155c80 0x7feb5014ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50152a80 con 0x7feb500720a0 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50152900 con 0x7feb50155c80 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb54913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5006b500 con 0x7feb50152520 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb500a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb500a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49490/0 (socket says 172.21.15.26:49490) 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb4e59c640 1 -- 172.21.15.26:0/616066680 learned_addr learned my addr 172.21.15.26:0/616066680 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb47fff640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb500a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.479+0000 7feb4ed9d640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb50155c80 0x7feb5014ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb4e59c640 1 -- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 msgr2=0x7feb500a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb4e59c640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb500a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb4e59c640 1 -- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb50155c80 msgr2=0x7feb5014ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb4e59c640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb50155c80 0x7feb5014ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb4e59c640 1 -- 172.21.15.26:0/616066680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb50158880 con 0x7feb500720a0 2024-03-06T07:23:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb4ed9d640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb50155c80 0x7feb5014ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb4e59c640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb500a87f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7feb4000e9e0 tx=0x7feb4000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb46ffd640 1 -- 172.21.15.26:0/616066680 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40017070 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb46ffd640 1 -- 172.21.15.26:0/616066680 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4000c9d0 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb50158b60 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb46ffd640 1 -- 172.21.15.26:0/616066680 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4000cdc0 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.480+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb500a8d30 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.481+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb501563d0 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.481+0000 7feb46ffd640 1 -- 172.21.15.26:0/616066680 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb4001a070 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.482+0000 7feb46ffd640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb3006b4d0 0x7feb3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.482+0000 7feb46ffd640 1 -- 172.21.15.26:0/616066680 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb4008a240 con 0x7feb500720a0 2024-03-06T07:23:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.482+0000 7feb47fff640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb3006b4d0 0x7feb3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.482+0000 7feb47fff640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb3006b4d0 0x7feb3006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feb501511a0 tx=0x7feb38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.486+0000 7feb46ffd640 1 -- 172.21.15.26:0/616066680 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb501563d0 con 0x7feb500720a0 2024-03-06T07:23:29.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.661+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb500c0b70 con 0x7feb3006b4d0 2024-03-06T07:23:29.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.668+0000 7feb46ffd640 1 -- 172.21.15.26:0/616066680 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246843 (secure 0 0 0) 0x7feb500c0b70 con 0x7feb3006b4d0 2024-03-06T07:23:29.672 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:29.673 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:23:29.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb3006b4d0 msgr2=0x7feb3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb3006b4d0 0x7feb3006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feb501511a0 tx=0x7feb38009040 comp rx=0 tx=0).stop 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 msgr2=0x7feb500a87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb500a87f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7feb4000e9e0 tx=0x7feb4000eea0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 shutdown_connections 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb3006b4d0 0x7feb3006d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb50155c80 0x7feb5014ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb50152520 0x7feb500a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 --2- 172.21.15.26:0/616066680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb500720a0 0x7feb500a87f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 >> 172.21.15.26:0/616066680 conn(0x7feb500c76b0 msgr2=0x7feb500c3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 shutdown_connections 2024-03-06T07:23:29.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:29.670+0000 7feb54913640 1 -- 172.21.15.26:0/616066680 wait complete. 2024-03-06T07:23:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-06T07:23:29.073056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250296,"kb_used_data":9140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724424,"statfs":{"total":773094113280,"available":772837810176,"internally_reserved":0,"allocated":9359360,"data_stored":5619685,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.848797"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236216+0000","last_change":"2024-03-06T07:23:24.236216+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236216+0000","last_undegraded":"2024-03-06T07:23:24.236216+0000","last_fullsized":"2024-03-06T07:23:24.236216+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237324+0000","last_change":"2024-03-06T07:23:24.237324+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237324+0000","last_undegraded":"2024-03-06T07:23:24.237324+0000","last_fullsized":"2024-03-06T07:23:24.237324+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235853+0000","last_change":"2024-03-06T07:23:24.235853+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235853+0000","last_undegraded":"2024-03-06T07:23:24.235853+0000","last_fullsized":"2024-03-06T07:23:24.235853+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234332+0000","last_change":"2024-03-06T07:23:24.234332+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234332+0000","last_undegraded":"2024-03-06T07:23:24.234332+0000","last_fullsized":"2024-03-06T07:23:24.234332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233017+0000","last_change":"2024-03-06T07:23:24.233017+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233017+0000","last_undegraded":"2024-03-06T07:23:24.233017+0000","last_fullsized":"2024-03-06T07:23:24.233017+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236616+0000","last_change":"2024-03-06T07:23:24.236616+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236616+0000","last_undegraded":"2024-03-06T07:23:24.236616+0000","last_fullsized":"2024-03-06T07:23:24.236616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238220+0000","last_change":"2024-03-06T07:23:24.238220+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238220+0000","last_undegraded":"2024-03-06T07:23:24.238220+0000","last_fullsized":"2024-03-06T07:23:24.238220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235822+0000","last_change":"2024-03-06T07:23:24.235822+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235822+0000","last_undegraded":"2024-03-06T07:23:24.235822+0000","last_fullsized":"2024-03-06T07:23:24.235822+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236313+0000","last_change":"2024-03-06T07:23:24.236313+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236313+0000","last_undegraded":"2024-03-06T07:23:24.236313+0000","last_fullsized":"2024-03-06T07:23:24.236313+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232154+0000","last_change":"2024-03-06T07:23:24.232154+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232154+0000","last_undegraded":"2024-03-06T07:23:24.232154+0000","last_fullsized":"2024-03-06T07:23:24.232154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242162+0000","last_change":"2024-03-06T07:23:24.242162+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242162+0000","last_undegraded":"2024-03-06T07:23:24.242162+0000","last_fullsized":"2024-03-06T07:23:24.242162+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234780+0000","last_change":"2024-03-06T07:23:24.234780+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234780+0000","last_undegraded":"2024-03-06T07:23:24.234780+0000","last_fullsized":"2024-03-06T07:23:24.234780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236601+0000","last_change":"2024-03-06T07:23:24.236601+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236601+0000","last_undegraded":"2024-03-06T07:23:24.236601+0000","last_fullsized":"2024-03-06T07:23:24.236601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232054+0000","last_change":"2024-03-06T07:23:24.232054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232054+0000","last_undegraded":"2024-03-06T07:23:24.232054+0000","last_fullsized":"2024-03-06T07:23:24.232054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231907+0000","last_change":"2024-03-06T07:23:24.231907+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231907+0000","last_undegraded":"2024-03-06T07:23:24.231907+0000","last_fullsized":"2024-03-06T07:23:24.231907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231901+0000","last_change":"2024-03-06T07:23:24.231901+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231901+0000","last_undegraded":"2024-03-06T07:23:24.231901+0000","last_fullsized":"2024-03-06T07:23:24.231901+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235823+0000","last_change":"2024-03-06T07:23:24.235823+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235823+0000","last_undegraded":"2024-03-06T07:23:24.235823+0000","last_fullsized":"2024-03-06T07:23:24.235823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236857+0000","last_change":"2024-03-06T07:23:24.236857+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236857+0000","last_undegraded":"2024-03-06T07:23:24.236857+0000","last_fullsized":"2024-03-06T07:23:24.236857+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235442+0000","last_change":"2024-03-06T07:23:24.235442+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235442+0000","last_undegraded":"2024-03-06T07:23:24.235442+0000","last_fullsized":"2024-03-06T07:23:24.235442+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234283+0000","last_change":"2024-03-06T07:23:24.234283+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234283+0000","last_undegraded":"2024-03-06T07:23:24.234283+0000","last_fullsized":"2024-03-06T07:23:24.234283+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234978+0000","last_change":"2024-03-06T07:23:24.234978+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234978+0000","last_undegraded":"2024-03-06T07:23:24.234978+0000","last_fullsized":"2024-03-06T07:23:24.234978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232196+0000","last_change":"2024-03-06T07:23:24.232196+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232196+0000","last_undegraded":"2024-03-06T07:23:24.232196+0000","last_fullsized":"2024-03-06T07:23:24.232196+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236484+0000","last_change":"2024-03-06T07:23:24.236484+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236484+0000","last_undegraded":"2024-03-06T07:23:24.236484+0000","last_fullsized":"2024-03-06T07:23:24.236484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231241+0000","last_change":"2024-03-06T07:23:24.231241+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231241+0000","last_undegraded":"2024-03-06T07:23:24.231241+0000","last_fullsized":"2024-03-06T07:23:24.231241+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233724+0000","last_change":"2024-03-06T07:23:24.233724+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233724+0000","last_undegraded":"2024-03-06T07:23:24.233724+0000","last_fullsized":"2024-03-06T07:23:24.233724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233286+0000","last_change":"2024-03-06T07:23:24.233286+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233286+0000","last_undegraded":"2024-03-06T07:23:24.233286+0000","last_fullsized":"2024-03-06T07:23:24.233286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.241335+0000","last_change":"2024-03-06T07:23:24.241335+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.241335+0000","last_undegraded":"2024-03-06T07:23:24.241335+0000","last_fullsized":"2024-03-06T07:23:24.241335+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.238590+0000","last_change":"2024-03-06T07:23:24.238590+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.238590+0000","last_undegraded":"2024-03-06T07:23:24.238590+0000","last_fullsized":"2024-03-06T07:23:24.238590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234554+0000","last_change":"2024-03-06T07:23:24.234554+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234554+0000","last_undegraded":"2024-03-06T07:23:24.234554+0000","last_fullsized":"2024-03-06T07:23:24.234554+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232453+0000","last_change":"2024-03-06T07:23:24.232453+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232453+0000","last_undegraded":"2024-03-06T07:23:24.232453+0000","last_fullsized":"2024-03-06T07:23:24.232453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.237717+0000","last_change":"2024-03-06T07:23:24.237717+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.237717+0000","last_undegraded":"2024-03-06T07:23:24.237717+0000","last_fullsized":"2024-03-06T07:23:24.237717+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235413+0000","last_change":"2024-03-06T07:23:24.235413+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235413+0000","last_undegraded":"2024-03-06T07:23:24.235413+0000","last_fullsized":"2024-03-06T07:23:24.235413+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233922+0000","last_change":"2024-03-06T07:23:24.233922+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233922+0000","last_undegraded":"2024-03-06T07:23:24.233922+0000","last_fullsized":"2024-03-06T07:23:24.233922+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250629+0000","last_change":"2024-03-06T07:23:24.250096+0000","last_active":"2024-03-06T07:23:24.250629+0000","last_peered":"2024-03-06T07:23:24.250629+0000","last_clean":"2024-03-06T07:23:24.250629+0000","last_became_active":"2024-03-06T07:23:24.249807+0000","last_became_peered":"2024-03-06T07:23:24.249807+0000","last_unstale":"2024-03-06T07:23:24.250629+0000","last_undegraded":"2024-03-06T07:23:24.250629+0000","last_fullsized":"2024-03-06T07:23:24.250629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:23.940701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232129+0000","last_change":"2024-03-06T07:23:24.232129+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232129+0000","last_undegraded":"2024-03-06T07:23:24.232129+0000","last_fullsized":"2024-03-06T07:23:24.232129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249418+0000","last_change":"2024-03-06T07:23:24.247409+0000","last_active":"2024-03-06T07:23:24.249418+0000","last_peered":"2024-03-06T07:23:24.249418+0000","last_clean":"2024-03-06T07:23:24.249418+0000","last_became_active":"2024-03-06T07:23:24.247083+0000","last_became_peered":"2024-03-06T07:23:24.247083+0000","last_unstale":"2024-03-06T07:23:24.249418+0000","last_undegraded":"2024-03-06T07:23:24.249418+0000","last_fullsized":"2024-03-06T07:23:24.249418+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:32.495551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231919+0000","last_change":"2024-03-06T07:23:24.231919+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231919+0000","last_undegraded":"2024-03-06T07:23:24.231919+0000","last_fullsized":"2024-03-06T07:23:24.231919+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252842+0000","last_change":"2024-03-06T07:23:24.247601+0000","last_active":"2024-03-06T07:23:24.252842+0000","last_peered":"2024-03-06T07:23:24.252842+0000","last_clean":"2024-03-06T07:23:24.252842+0000","last_became_active":"2024-03-06T07:23:24.246790+0000","last_became_peered":"2024-03-06T07:23:24.246790+0000","last_unstale":"2024-03-06T07:23:24.252842+0000","last_undegraded":"2024-03-06T07:23:24.252842+0000","last_fullsized":"2024-03-06T07:23:24.252842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239556+0000","last_change":"2024-03-06T07:23:24.239556+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239556+0000","last_undegraded":"2024-03-06T07:23:24.239556+0000","last_fullsized":"2024-03-06T07:23:24.239556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244586+0000","last_change":"2024-03-06T07:23:24.240691+0000","last_active":"2024-03-06T07:23:24.244586+0000","last_peered":"2024-03-06T07:23:24.244586+0000","last_clean":"2024-03-06T07:23:24.244586+0000","last_became_active":"2024-03-06T07:23:24.240338+0000","last_became_peered":"2024-03-06T07:23:24.240338+0000","last_unstale":"2024-03-06T07:23:24.244586+0000","last_undegraded":"2024-03-06T07:23:24.244586+0000","last_fullsized":"2024-03-06T07:23:24.244586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233786+0000","last_change":"2024-03-06T07:23:24.233786+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233786+0000","last_undegraded":"2024-03-06T07:23:24.233786+0000","last_fullsized":"2024-03-06T07:23:24.233786+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247263+0000","last_change":"2024-03-06T07:23:24.245937+0000","last_active":"2024-03-06T07:23:24.247263+0000","last_peered":"2024-03-06T07:23:24.247263+0000","last_clean":"2024-03-06T07:23:24.247263+0000","last_became_active":"2024-03-06T07:23:24.245614+0000","last_became_peered":"2024-03-06T07:23:24.245614+0000","last_unstale":"2024-03-06T07:23:24.247263+0000","last_undegraded":"2024-03-06T07:23:24.247263+0000","last_fullsized":"2024-03-06T07:23:24.247263+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:26.767861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234462+0000","last_change":"2024-03-06T07:23:24.234462+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234462+0000","last_undegraded":"2024-03-06T07:23:24.234462+0000","last_fullsized":"2024-03-06T07:23:24.234462+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.253182+0000","last_change":"2024-03-06T07:23:24.249153+0000","last_active":"2024-03-06T07:23:24.253182+0000","last_peered":"2024-03-06T07:23:24.253182+0000","last_clean":"2024-03-06T07:23:24.253182+0000","last_became_active":"2024-03-06T07:23:24.248846+0000","last_became_peered":"2024-03-06T07:23:24.248846+0000","last_unstale":"2024-03-06T07:23:24.253182+0000","last_undegraded":"2024-03-06T07:23:24.253182+0000","last_fullsized":"2024-03-06T07:23:24.253182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236089+0000","last_change":"2024-03-06T07:23:24.236089+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236089+0000","last_undegraded":"2024-03-06T07:23:24.236089+0000","last_fullsized":"2024-03-06T07:23:24.236089+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250267+0000","last_change":"2024-03-06T07:23:24.245204+0000","last_active":"2024-03-06T07:23:24.250267+0000","last_peered":"2024-03-06T07:23:24.250267+0000","last_clean":"2024-03-06T07:23:24.250267+0000","last_became_active":"2024-03-06T07:23:24.244893+0000","last_became_peered":"2024-03-06T07:23:24.244893+0000","last_unstale":"2024-03-06T07:23:24.250267+0000","last_undegraded":"2024-03-06T07:23:24.250267+0000","last_fullsized":"2024-03-06T07:23:24.250267+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236223+0000","last_change":"2024-03-06T07:23:24.236223+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236223+0000","last_undegraded":"2024-03-06T07:23:24.236223+0000","last_fullsized":"2024-03-06T07:23:24.236223+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247752+0000","last_change":"2024-03-06T07:23:24.246502+0000","last_active":"2024-03-06T07:23:24.247752+0000","last_peered":"2024-03-06T07:23:24.247752+0000","last_clean":"2024-03-06T07:23:24.247752+0000","last_became_active":"2024-03-06T07:23:24.246230+0000","last_became_peered":"2024-03-06T07:23:24.246230+0000","last_unstale":"2024-03-06T07:23:24.247752+0000","last_undegraded":"2024-03-06T07:23:24.247752+0000","last_fullsized":"2024-03-06T07:23:24.247752+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234549+0000","last_change":"2024-03-06T07:23:24.234549+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234549+0000","last_undegraded":"2024-03-06T07:23:24.234549+0000","last_fullsized":"2024-03-06T07:23:24.234549+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252877+0000","last_change":"2024-03-06T07:23:24.248199+0000","last_active":"2024-03-06T07:23:24.252877+0000","last_peered":"2024-03-06T07:23:24.252877+0000","last_clean":"2024-03-06T07:23:24.252877+0000","last_became_active":"2024-03-06T07:23:24.247705+0000","last_became_peered":"2024-03-06T07:23:24.247705+0000","last_unstale":"2024-03-06T07:23:24.252877+0000","last_undegraded":"2024-03-06T07:23:24.252877+0000","last_fullsized":"2024-03-06T07:23:24.252877+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.421375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231156+0000","last_change":"2024-03-06T07:23:24.231156+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231156+0000","last_undegraded":"2024-03-06T07:23:24.231156+0000","last_fullsized":"2024-03-06T07:23:24.231156+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247760+0000","last_change":"2024-03-06T07:23:24.245504+0000","last_active":"2024-03-06T07:23:24.247760+0000","last_peered":"2024-03-06T07:23:24.247760+0000","last_clean":"2024-03-06T07:23:24.247760+0000","last_became_active":"2024-03-06T07:23:24.245228+0000","last_became_peered":"2024-03-06T07:23:24.245228+0000","last_unstale":"2024-03-06T07:23:24.247760+0000","last_undegraded":"2024-03-06T07:23:24.247760+0000","last_fullsized":"2024-03-06T07:23:24.247760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:42.596673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232421+0000","last_change":"2024-03-06T07:23:24.232421+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232421+0000","last_undegraded":"2024-03-06T07:23:24.232421+0000","last_fullsized":"2024-03-06T07:23:24.232421+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244983+0000","last_change":"2024-03-06T07:23:24.242127+0000","last_active":"2024-03-06T07:23:24.244983+0000","last_peered":"2024-03-06T07:23:24.244983+0000","last_clean":"2024-03-06T07:23:24.244983+0000","last_became_active":"2024-03-06T07:23:24.241524+0000","last_became_peered":"2024-03-06T07:23:24.241524+0000","last_unstale":"2024-03-06T07:23:24.244983+0000","last_undegraded":"2024-03-06T07:23:24.244983+0000","last_fullsized":"2024-03-06T07:23:24.244983+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:42.087045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231299+0000","last_change":"2024-03-06T07:23:24.231299+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231299+0000","last_undegraded":"2024-03-06T07:23:24.231299+0000","last_fullsized":"2024-03-06T07:23:24.231299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252214+0000","last_change":"2024-03-06T07:23:24.246676+0000","last_active":"2024-03-06T07:23:24.252214+0000","last_peered":"2024-03-06T07:23:24.252214+0000","last_clean":"2024-03-06T07:23:24.252214+0000","last_became_active":"2024-03-06T07:23:24.246408+0000","last_became_peered":"2024-03-06T07:23:24.246408+0000","last_unstale":"2024-03-06T07:23:24.252214+0000","last_undegraded":"2024-03-06T07:23:24.252214+0000","last_fullsized":"2024-03-06T07:23:24.252214+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:37:15.620447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.239944+0000","last_change":"2024-03-06T07:23:24.239944+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.239944+0000","last_undegraded":"2024-03-06T07:23:24.239944+0000","last_fullsized":"2024-03-06T07:23:24.239944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245809+0000","last_change":"2024-03-06T07:23:24.242187+0000","last_active":"2024-03-06T07:23:24.245809+0000","last_peered":"2024-03-06T07:23:24.245809+0000","last_clean":"2024-03-06T07:23:24.245809+0000","last_became_active":"2024-03-06T07:23:24.241930+0000","last_became_peered":"2024-03-06T07:23:24.241930+0000","last_unstale":"2024-03-06T07:23:24.245809+0000","last_undegraded":"2024-03-06T07:23:24.245809+0000","last_fullsized":"2024-03-06T07:23:24.245809+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236307+0000","last_change":"2024-03-06T07:23:24.236307+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236307+0000","last_undegraded":"2024-03-06T07:23:24.236307+0000","last_fullsized":"2024-03-06T07:23:24.236307+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247677+0000","last_change":"2024-03-06T07:23:24.245524+0000","last_active":"2024-03-06T07:23:24.247677+0000","last_peered":"2024-03-06T07:23:24.247677+0000","last_clean":"2024-03-06T07:23:24.247677+0000","last_became_active":"2024-03-06T07:23:24.245245+0000","last_became_peered":"2024-03-06T07:23:24.245245+0000","last_unstale":"2024-03-06T07:23:24.247677+0000","last_undegraded":"2024-03-06T07:23:24.247677+0000","last_fullsized":"2024-03-06T07:23:24.247677+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:09:10.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234244+0000","last_change":"2024-03-06T07:23:24.234244+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234244+0000","last_undegraded":"2024-03-06T07:23:24.234244+0000","last_fullsized":"2024-03-06T07:23:24.234244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246012+0000","last_change":"2024-03-06T07:23:24.243043+0000","last_active":"2024-03-06T07:23:24.246012+0000","last_peered":"2024-03-06T07:23:24.246012+0000","last_clean":"2024-03-06T07:23:24.246012+0000","last_became_active":"2024-03-06T07:23:24.242793+0000","last_became_peered":"2024-03-06T07:23:24.242793+0000","last_unstale":"2024-03-06T07:23:24.246012+0000","last_undegraded":"2024-03-06T07:23:24.246012+0000","last_fullsized":"2024-03-06T07:23:24.246012+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:00:12.866720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235408+0000","last_change":"2024-03-06T07:23:24.235408+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235408+0000","last_undegraded":"2024-03-06T07:23:24.235408+0000","last_fullsized":"2024-03-06T07:23:24.235408+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250324+0000","last_change":"2024-03-06T07:23:24.244518+0000","last_active":"2024-03-06T07:23:24.250324+0000","last_peered":"2024-03-06T07:23:24.250324+0000","last_clean":"2024-03-06T07:23:24.250324+0000","last_became_active":"2024-03-06T07:23:24.244154+0000","last_became_peered":"2024-03-06T07:23:24.244154+0000","last_unstale":"2024-03-06T07:23:24.250324+0000","last_undegraded":"2024-03-06T07:23:24.250324+0000","last_fullsized":"2024-03-06T07:23:24.250324+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233094+0000","last_change":"2024-03-06T07:23:24.233094+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233094+0000","last_undegraded":"2024-03-06T07:23:24.233094+0000","last_fullsized":"2024-03-06T07:23:24.233094+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252274+0000","last_change":"2024-03-06T07:23:24.247115+0000","last_active":"2024-03-06T07:23:24.252274+0000","last_peered":"2024-03-06T07:23:24.252274+0000","last_clean":"2024-03-06T07:23:24.252274+0000","last_became_active":"2024-03-06T07:23:24.246835+0000","last_became_peered":"2024-03-06T07:23:24.246835+0000","last_unstale":"2024-03-06T07:23:24.252274+0000","last_undegraded":"2024-03-06T07:23:24.252274+0000","last_fullsized":"2024-03-06T07:23:24.252274+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:40.099312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.236125+0000","last_change":"2024-03-06T07:23:24.236125+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.236125+0000","last_undegraded":"2024-03-06T07:23:24.236125+0000","last_fullsized":"2024-03-06T07:23:24.236125+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245014+0000","last_change":"2024-03-06T07:23:24.242459+0000","last_active":"2024-03-06T07:23:24.245014+0000","last_peered":"2024-03-06T07:23:24.245014+0000","last_clean":"2024-03-06T07:23:24.245014+0000","last_became_active":"2024-03-06T07:23:24.241848+0000","last_became_peered":"2024-03-06T07:23:24.241848+0000","last_unstale":"2024-03-06T07:23:24.245014+0000","last_undegraded":"2024-03-06T07:23:24.245014+0000","last_fullsized":"2024-03-06T07:23:24.245014+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:01.845106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.231767+0000","last_change":"2024-03-06T07:23:24.231767+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.231767+0000","last_undegraded":"2024-03-06T07:23:24.231767+0000","last_fullsized":"2024-03-06T07:23:24.231767+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.255498+0000","last_change":"2024-03-06T07:23:24.252717+0000","last_active":"2024-03-06T07:23:24.255498+0000","last_peered":"2024-03-06T07:23:24.255498+0000","last_clean":"2024-03-06T07:23:24.255498+0000","last_became_active":"2024-03-06T07:23:24.252429+0000","last_became_peered":"2024-03-06T07:23:24.252429+0000","last_unstale":"2024-03-06T07:23:24.255498+0000","last_undegraded":"2024-03-06T07:23:24.255498+0000","last_fullsized":"2024-03-06T07:23:24.255498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:12.254926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234015+0000","last_change":"2024-03-06T07:23:24.234015+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234015+0000","last_undegraded":"2024-03-06T07:23:24.234015+0000","last_fullsized":"2024-03-06T07:23:24.234015+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249758+0000","last_change":"2024-03-06T07:23:24.248709+0000","last_active":"2024-03-06T07:23:24.249758+0000","last_peered":"2024-03-06T07:23:24.249758+0000","last_clean":"2024-03-06T07:23:24.249758+0000","last_became_active":"2024-03-06T07:23:24.247184+0000","last_became_peered":"2024-03-06T07:23:24.247184+0000","last_unstale":"2024-03-06T07:23:24.249758+0000","last_undegraded":"2024-03-06T07:23:24.249758+0000","last_fullsized":"2024-03-06T07:23:24.249758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:23:10.470563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234900+0000","last_change":"2024-03-06T07:23:24.234900+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234900+0000","last_undegraded":"2024-03-06T07:23:24.234900+0000","last_fullsized":"2024-03-06T07:23:24.234900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249143+0000","last_change":"2024-03-06T07:23:24.246510+0000","last_active":"2024-03-06T07:23:24.249143+0000","last_peered":"2024-03-06T07:23:24.249143+0000","last_clean":"2024-03-06T07:23:24.249143+0000","last_became_active":"2024-03-06T07:23:24.246222+0000","last_became_peered":"2024-03-06T07:23:24.246222+0000","last_unstale":"2024-03-06T07:23:24.249143+0000","last_undegraded":"2024-03-06T07:23:24.249143+0000","last_fullsized":"2024-03-06T07:23:24.249143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:26.098267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235380+0000","last_change":"2024-03-06T07:23:24.235380+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235380+0000","last_undegraded":"2024-03-06T07:23:24.235380+0000","last_fullsized":"2024-03-06T07:23:24.235380+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.249498+0000","last_change":"2024-03-06T07:23:24.244174+0000","last_active":"2024-03-06T07:23:24.249498+0000","last_peered":"2024-03-06T07:23:24.249498+0000","last_clean":"2024-03-06T07:23:24.249498+0000","last_became_active":"2024-03-06T07:23:24.243046+0000","last_became_peered":"2024-03-06T07:23:24.243046+0000","last_unstale":"2024-03-06T07:23:24.249498+0000","last_undegraded":"2024-03-06T07:23:24.249498+0000","last_fullsized":"2024-03-06T07:23:24.249498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:18.162769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233848+0000","last_change":"2024-03-06T07:23:24.233848+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233848+0000","last_undegraded":"2024-03-06T07:23:24.233848+0000","last_fullsized":"2024-03-06T07:23:24.233848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246844+0000","last_change":"2024-03-06T07:23:24.245809+0000","last_active":"2024-03-06T07:23:24.246844+0000","last_peered":"2024-03-06T07:23:24.246844+0000","last_clean":"2024-03-06T07:23:24.246844+0000","last_became_active":"2024-03-06T07:23:24.245527+0000","last_became_peered":"2024-03-06T07:23:24.245527+0000","last_unstale":"2024-03-06T07:23:24.246844+0000","last_undegraded":"2024-03-06T07:23:24.246844+0000","last_fullsized":"2024-03-06T07:23:24.246844+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:15.139265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.234987+0000","last_change":"2024-03-06T07:23:24.234987+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.234987+0000","last_undegraded":"2024-03-06T07:23:24.234987+0000","last_fullsized":"2024-03-06T07:23:24.234987+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.244456+0000","last_change":"2024-03-06T07:23:24.242259+0000","last_active":"2024-03-06T07:23:24.244456+0000","last_peered":"2024-03-06T07:23:24.244456+0000","last_clean":"2024-03-06T07:23:24.244456+0000","last_became_active":"2024-03-06T07:23:24.241958+0000","last_became_peered":"2024-03-06T07:23:24.241958+0000","last_unstale":"2024-03-06T07:23:24.244456+0000","last_undegraded":"2024-03-06T07:23:24.244456+0000","last_fullsized":"2024-03-06T07:23:24.244456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232370+0000","last_change":"2024-03-06T07:23:24.232370+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232370+0000","last_undegraded":"2024-03-06T07:23:24.232370+0000","last_fullsized":"2024-03-06T07:23:24.232370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247783+0000","last_change":"2024-03-06T07:23:24.246996+0000","last_active":"2024-03-06T07:23:24.247783+0000","last_peered":"2024-03-06T07:23:24.247783+0000","last_clean":"2024-03-06T07:23:24.247783+0000","last_became_active":"2024-03-06T07:23:24.246685+0000","last_became_peered":"2024-03-06T07:23:24.246685+0000","last_unstale":"2024-03-06T07:23:24.247783+0000","last_undegraded":"2024-03-06T07:23:24.247783+0000","last_fullsized":"2024-03-06T07:23:24.247783+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:48:35.768778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.235450+0000","last_change":"2024-03-06T07:23:24.235450+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.235450+0000","last_undegraded":"2024-03-06T07:23:24.235450+0000","last_fullsized":"2024-03-06T07:23:24.235450+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252103+0000","last_change":"2024-03-06T07:23:24.244748+0000","last_active":"2024-03-06T07:23:24.252103+0000","last_peered":"2024-03-06T07:23:24.252103+0000","last_clean":"2024-03-06T07:23:24.252103+0000","last_became_active":"2024-03-06T07:23:24.244211+0000","last_became_peered":"2024-03-06T07:23:24.244211+0000","last_unstale":"2024-03-06T07:23:24.252103+0000","last_undegraded":"2024-03-06T07:23:24.252103+0000","last_fullsized":"2024-03-06T07:23:24.252103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:21.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240054+0000","last_change":"2024-03-06T07:23:24.240054+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240054+0000","last_undegraded":"2024-03-06T07:23:24.240054+0000","last_fullsized":"2024-03-06T07:23:24.240054+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250382+0000","last_change":"2024-03-06T07:23:24.248695+0000","last_active":"2024-03-06T07:23:24.250382+0000","last_peered":"2024-03-06T07:23:24.250382+0000","last_clean":"2024-03-06T07:23:24.250382+0000","last_became_active":"2024-03-06T07:23:24.248417+0000","last_became_peered":"2024-03-06T07:23:24.248417+0000","last_unstale":"2024-03-06T07:23:24.250382+0000","last_undegraded":"2024-03-06T07:23:24.250382+0000","last_fullsized":"2024-03-06T07:23:24.250382+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.233262+0000","last_change":"2024-03-06T07:23:24.233262+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.233262+0000","last_undegraded":"2024-03-06T07:23:24.233262+0000","last_fullsized":"2024-03-06T07:23:24.233262+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.250297+0000","last_change":"2024-03-06T07:23:24.246199+0000","last_active":"2024-03-06T07:23:24.250297+0000","last_peered":"2024-03-06T07:23:24.250297+0000","last_clean":"2024-03-06T07:23:24.250297+0000","last_became_active":"2024-03-06T07:23:24.245854+0000","last_became_peered":"2024-03-06T07:23:24.245854+0000","last_unstale":"2024-03-06T07:23:24.250297+0000","last_undegraded":"2024-03-06T07:23:24.250297+0000","last_fullsized":"2024-03-06T07:23:24.250297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232832+0000","last_change":"2024-03-06T07:23:24.232832+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232832+0000","last_undegraded":"2024-03-06T07:23:24.232832+0000","last_fullsized":"2024-03-06T07:23:24.232832+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.246395+0000","last_change":"2024-03-06T07:23:24.244313+0000","last_active":"2024-03-06T07:23:24.246395+0000","last_peered":"2024-03-06T07:23:24.246395+0000","last_clean":"2024-03-06T07:23:24.246395+0000","last_became_active":"2024-03-06T07:23:24.244059+0000","last_became_peered":"2024-03-06T07:23:24.244059+0000","last_unstale":"2024-03-06T07:23:24.246395+0000","last_undegraded":"2024-03-06T07:23:24.246395+0000","last_fullsized":"2024-03-06T07:23:24.246395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:53.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.232253+0000","last_change":"2024-03-06T07:23:24.232253+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.232253+0000","last_undegraded":"2024-03-06T07:23:24.232253+0000","last_fullsized":"2024-03-06T07:23:24.232253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.252266+0000","last_change":"2024-03-06T07:23:24.247137+0000","last_active":"2024-03-06T07:23:24.252266+0000","last_peered":"2024-03-06T07:23:24.252266+0000","last_clean":"2024-03-06T07:23:24.252266+0000","last_became_active":"2024-03-06T07:23:24.246871+0000","last_became_peered":"2024-03-06T07:23:24.246871+0000","last_unstale":"2024-03-06T07:23:24.252266+0000","last_undegraded":"2024-03-06T07:23:24.252266+0000","last_fullsized":"2024-03-06T07:23:24.252266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:15:06.289086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.242099+0000","last_change":"2024-03-06T07:23:24.242099+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.242099+0000","last_undegraded":"2024-03-06T07:23:24.242099+0000","last_fullsized":"2024-03-06T07:23:24.242099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.245965+0000","last_change":"2024-03-06T07:23:24.243574+0000","last_active":"2024-03-06T07:23:24.245965+0000","last_peered":"2024-03-06T07:23:24.245965+0000","last_clean":"2024-03-06T07:23:24.245965+0000","last_became_active":"2024-03-06T07:23:24.243297+0000","last_became_peered":"2024-03-06T07:23:24.243297+0000","last_unstale":"2024-03-06T07:23:24.245965+0000","last_undegraded":"2024-03-06T07:23:24.245965+0000","last_fullsized":"2024-03-06T07:23:24.245965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:05.404876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-03-06T07:23:24.240960+0000","last_change":"2024-03-06T07:23:24.240960+0000","last_active":"2024-03-06T07:23:24.223485+0000","last_peered":"2024-03-06T07:23:24.223485+0000","last_clean":"2024-03-06T07:23:24.223485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:23:24.240960+0000","last_undegraded":"2024-03-06T07:23:24.240960+0000","last_fullsized":"2024-03-06T07:23:24.240960+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.247260+0000","last_change":"2024-03-06T07:23:24.244996+0000","last_active":"2024-03-06T07:23:24.247260+0000","last_peered":"2024-03-06T07:23:24.247260+0000","last_clean":"2024-03-06T07:23:24.247260+0000","last_became_active":"2024-03-06T07:23:24.244721+0000","last_became_peered":"2024-03-06T07:23:24.244721+0000","last_unstale":"2024-03-06T07:23:24.247260+0000","last_undegraded":"2024-03-06T07:23:24.247260+0000","last_fullsized":"2024-03-06T07:23:24.247260+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:23.742370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-06T07:23:24.233201+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:23:24.233201+0000","last_peered":"2024-03-06T07:23:24.233201+0000","last_clean":"2024-03-06T07:23:24.233201+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:23:24.233201+0000","last_undegraded":"2024-03-06T07:23:24.233201+0000","last_fullsized":"2024-03-06T07:23:24.233201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1142784,"data_stored":549035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":555702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":1146070,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:30.533 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:23:30.532+0000 7f301772d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:32.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:32.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.803+0000 7fd67e57a640 1 Processor -- start 2024-03-06T07:23:32.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.803+0000 7fd67e57a640 1 -- start start 2024-03-06T07:23:32.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd67e57a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6780c5330 0x7fd6780c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:32.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd67e57a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c88b0 0x7fd6781586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd67e57a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd678155110 con 0x7fd6780c7dd0 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd67e57a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6781559e0 con 0x7fd6780c5330 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd67e57a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780c8f40 con 0x7fd6780c88b0 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd677fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6780c5330 0x7fd6780c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd67cb17640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd6780c7dd0 0x7fd67806b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34862/0 (socket says 172.21.15.26:34862) 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd677fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6780c5330 0x7fd6780c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49492/0 (socket says 172.21.15.26:49492) 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd67cb17640 1 -- 172.21.15.26:0/73824988 learned_addr learned my addr 172.21.15.26:0/73824988 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.804+0000 7fd6777fe640 1 --2- 172.21.15.26:0/73824988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c88b0 0x7fd6781586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.805+0000 7fd677fff640 1 -- 172.21.15.26:0/73824988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c88b0 msgr2=0x7fd6781586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.805+0000 7fd677fff640 1 --2- 172.21.15.26:0/73824988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c88b0 0x7fd6781586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.805+0000 7fd677fff640 1 -- 172.21.15.26:0/73824988 >> v1:172.21.15.26:6789/0 conn(0x7fd6780c7dd0 legacy=0x7fd67806b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.805+0000 7fd677fff640 1 -- 172.21.15.26:0/73824988 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678155390 con 0x7fd6780c5330 2024-03-06T07:23:32.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.805+0000 7fd676ffd640 1 -- 172.21.15.26:0/73824988 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985314655 0 0) 0x7fd670001490 con 0x7fd6780c7dd0 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.805+0000 7fd677fff640 1 --2- 172.21.15.26:0/73824988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c5330 0x7fd6780c5710 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd6680097a0 tx=0x7fd66802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec01146245c1ce8b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd676ffd640 1 -- 172.21.15.26:0/73824988 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680315a0 con 0x7fd6780c5330 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd676ffd640 1 -- 172.21.15.26:0/73824988 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd668037070 con 0x7fd6780c5330 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd676ffd640 1 -- 172.21.15.26:0/73824988 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668031960 con 0x7fd6780c5330 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 -- 172.21.15.26:0/73824988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c5330 msgr2=0x7fd6780c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 --2- 172.21.15.26:0/73824988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c5330 0x7fd6780c5710 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd6680097a0 tx=0x7fd66802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 -- 172.21.15.26:0/73824988 shutdown_connections 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 --2- 172.21.15.26:0/73824988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c88b0 0x7fd6781586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:32.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 --2- 172.21.15.26:0/73824988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c5330 0x7fd6780c5710 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:32.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 -- 172.21.15.26:0/73824988 >> 172.21.15.26:0/73824988 conn(0x7fd6780ac560 msgr2=0x7fd6780ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:32.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 -- 172.21.15.26:0/73824988 shutdown_connections 2024-03-06T07:23:32.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.806+0000 7fd67e57a640 1 -- 172.21.15.26:0/73824988 wait complete. 2024-03-06T07:23:32.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.807+0000 7fd67e57a640 1 Processor -- start 2024-03-06T07:23:32.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67e57a640 1 -- start start 2024-03-06T07:23:32.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67e57a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c5330 0x7fd6781ec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:32.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67e57a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6780c7dd0 0x7fd6780c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:32.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67e57a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c88b0 0x7fd67814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:32.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67e57a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781529d0 con 0x7fd6780c88b0 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67e57a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67806d090 con 0x7fd6780c7dd0 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67e57a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd678152b50 con 0x7fd6780c5330 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67cb17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c88b0 0x7fd67814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67cb17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c88b0 0x7fd67814b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49502/0 (socket says 172.21.15.26:49502) 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.808+0000 7fd67cb17640 1 -- 172.21.15.26:0/93064568 learned_addr learned my addr 172.21.15.26:0/93064568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd6777fe640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6780c7dd0 0x7fd6780c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd677fff640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c5330 0x7fd6781ec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67cb17640 1 -- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c5330 msgr2=0x7fd6781ec2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67cb17640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c5330 0x7fd6781ec2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67cb17640 1 -- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6780c7dd0 msgr2=0x7fd6780c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67cb17640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6780c7dd0 0x7fd6780c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:32.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67cb17640 1 -- 172.21.15.26:0/93064568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6781ef250 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67cb17640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c88b0 0x7fd67814b780 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c7f0 tx=0x7fd67000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd674ff9640 1 -- 172.21.15.26:0/93064568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670018070 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd674ff9640 1 -- 172.21.15.26:0/93064568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67000a5e0 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6780c43b0 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd674ff9640 1 -- 172.21.15.26:0/93064568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000aa10 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.809+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6780c2630 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.810+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67804f330 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.811+0000 7fd674ff9640 1 -- 172.21.15.26:0/93064568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd67001b070 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.811+0000 7fd674ff9640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd66406b4d0 0x7fd66406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.811+0000 7fd674ff9640 1 -- 172.21.15.26:0/93064568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6700898b0 con 0x7fd6780c88b0 2024-03-06T07:23:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.811+0000 7fd677fff640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd66406b4d0 0x7fd66406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:32.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.812+0000 7fd677fff640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd66406b4d0 0x7fd66406d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd668036000 tx=0x7fd66803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:32.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.815+0000 7fd674ff9640 1 -- 172.21.15.26:0/93064568 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd678155110 con 0x7fd6780c88b0 2024-03-06T07:23:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.990+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6780c1850 con 0x7fd66406b4d0 2024-03-06T07:23:32.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:32.997+0000 7fd674ff9640 1 -- 172.21.15.26:0/93064568 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252160 (secure 0 0 0) 0x7fd668037040 con 0x7fd66406b4d0 2024-03-06T07:23:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd66406b4d0 msgr2=0x7fd66406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd66406b4d0 0x7fd66406d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd668036000 tx=0x7fd66803d040 comp rx=0 tx=0).stop 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c88b0 msgr2=0x7fd67814b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c88b0 0x7fd67814b780 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c7f0 tx=0x7fd67000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 shutdown_connections 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd66406b4d0 0x7fd66406d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6780c88b0 0x7fd67814b780 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6780c7dd0 0x7fd6780c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 --2- 172.21.15.26:0/93064568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6780c5330 0x7fd6781ec2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 >> 172.21.15.26:0/93064568 conn(0x7fd6780ac560 msgr2=0x7fd678152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 shutdown_connections 2024-03-06T07:23:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.000+0000 7fd67e57a640 1 -- 172.21.15.26:0/93064568 wait complete. 2024-03-06T07:23:33.013 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-03-06T07:23:31.073365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258896,"kb_used_data":9408,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715824,"statfs":{"total":773094113280,"available":772829003776,"internally_reserved":0,"allocated":9633792,"data_stored":5742044,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.843689"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244417+0000","last_change":"2024-03-06T07:23:25.264462+0000","last_active":"2024-03-06T07:23:26.244417+0000","last_peered":"2024-03-06T07:23:26.244417+0000","last_clean":"2024-03-06T07:23:26.244417+0000","last_became_active":"2024-03-06T07:23:25.264163+0000","last_became_peered":"2024-03-06T07:23:25.264163+0000","last_unstale":"2024-03-06T07:23:26.244417+0000","last_undegraded":"2024-03-06T07:23:26.244417+0000","last_fullsized":"2024-03-06T07:23:26.244417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:09.663052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385547+0000","last_change":"2024-03-06T07:23:25.264219+0000","last_active":"2024-03-06T07:23:26.385547+0000","last_peered":"2024-03-06T07:23:26.385547+0000","last_clean":"2024-03-06T07:23:26.385547+0000","last_became_active":"2024-03-06T07:23:25.263885+0000","last_became_peered":"2024-03-06T07:23:25.263885+0000","last_unstale":"2024-03-06T07:23:26.385547+0000","last_undegraded":"2024-03-06T07:23:26.385547+0000","last_fullsized":"2024-03-06T07:23:26.385547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:43.090933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244495+0000","last_change":"2024-03-06T07:23:25.272079+0000","last_active":"2024-03-06T07:23:26.244495+0000","last_peered":"2024-03-06T07:23:26.244495+0000","last_clean":"2024-03-06T07:23:26.244495+0000","last_became_active":"2024-03-06T07:23:25.271037+0000","last_became_peered":"2024-03-06T07:23:25.271037+0000","last_unstale":"2024-03-06T07:23:26.244495+0000","last_undegraded":"2024-03-06T07:23:26.244495+0000","last_fullsized":"2024-03-06T07:23:26.244495+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:58:48.085833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383211+0000","last_change":"2024-03-06T07:23:25.267479+0000","last_active":"2024-03-06T07:23:26.383211+0000","last_peered":"2024-03-06T07:23:26.383211+0000","last_clean":"2024-03-06T07:23:26.383211+0000","last_became_active":"2024-03-06T07:23:25.267069+0000","last_became_peered":"2024-03-06T07:23:25.267069+0000","last_unstale":"2024-03-06T07:23:26.383211+0000","last_undegraded":"2024-03-06T07:23:26.383211+0000","last_fullsized":"2024-03-06T07:23:26.383211+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383416+0000","last_change":"2024-03-06T07:23:25.268874+0000","last_active":"2024-03-06T07:23:26.383416+0000","last_peered":"2024-03-06T07:23:26.383416+0000","last_clean":"2024-03-06T07:23:26.383416+0000","last_became_active":"2024-03-06T07:23:25.268597+0000","last_became_peered":"2024-03-06T07:23:25.268597+0000","last_unstale":"2024-03-06T07:23:26.383416+0000","last_undegraded":"2024-03-06T07:23:26.383416+0000","last_fullsized":"2024-03-06T07:23:26.383416+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:39.384249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383781+0000","last_change":"2024-03-06T07:23:25.262220+0000","last_active":"2024-03-06T07:23:26.383781+0000","last_peered":"2024-03-06T07:23:26.383781+0000","last_clean":"2024-03-06T07:23:26.383781+0000","last_became_active":"2024-03-06T07:23:25.260767+0000","last_became_peered":"2024-03-06T07:23:25.260767+0000","last_unstale":"2024-03-06T07:23:26.383781+0000","last_undegraded":"2024-03-06T07:23:26.383781+0000","last_fullsized":"2024-03-06T07:23:26.383781+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:24.944589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384866+0000","last_change":"2024-03-06T07:23:25.270420+0000","last_active":"2024-03-06T07:23:26.384866+0000","last_peered":"2024-03-06T07:23:26.384866+0000","last_clean":"2024-03-06T07:23:26.384866+0000","last_became_active":"2024-03-06T07:23:25.270110+0000","last_became_peered":"2024-03-06T07:23:25.270110+0000","last_unstale":"2024-03-06T07:23:26.384866+0000","last_undegraded":"2024-03-06T07:23:26.384866+0000","last_fullsized":"2024-03-06T07:23:26.384866+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:18.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243375+0000","last_change":"2024-03-06T07:23:25.255615+0000","last_active":"2024-03-06T07:23:26.243375+0000","last_peered":"2024-03-06T07:23:26.243375+0000","last_clean":"2024-03-06T07:23:26.243375+0000","last_became_active":"2024-03-06T07:23:25.255350+0000","last_became_peered":"2024-03-06T07:23:25.255350+0000","last_unstale":"2024-03-06T07:23:26.243375+0000","last_undegraded":"2024-03-06T07:23:26.243375+0000","last_fullsized":"2024-03-06T07:23:26.243375+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:06.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384616+0000","last_change":"2024-03-06T07:23:25.261684+0000","last_active":"2024-03-06T07:23:26.384616+0000","last_peered":"2024-03-06T07:23:26.384616+0000","last_clean":"2024-03-06T07:23:26.384616+0000","last_became_active":"2024-03-06T07:23:25.261132+0000","last_became_peered":"2024-03-06T07:23:25.261132+0000","last_unstale":"2024-03-06T07:23:26.384616+0000","last_undegraded":"2024-03-06T07:23:26.384616+0000","last_fullsized":"2024-03-06T07:23:26.384616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:52:04.546752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384047+0000","last_change":"2024-03-06T07:23:25.263825+0000","last_active":"2024-03-06T07:23:26.384047+0000","last_peered":"2024-03-06T07:23:26.384047+0000","last_clean":"2024-03-06T07:23:26.384047+0000","last_became_active":"2024-03-06T07:23:25.263487+0000","last_became_peered":"2024-03-06T07:23:25.263487+0000","last_unstale":"2024-03-06T07:23:26.384047+0000","last_undegraded":"2024-03-06T07:23:26.384047+0000","last_fullsized":"2024-03-06T07:23:26.384047+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:43.090933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384341+0000","last_change":"2024-03-06T07:23:25.263514+0000","last_active":"2024-03-06T07:23:26.384341+0000","last_peered":"2024-03-06T07:23:26.384341+0000","last_clean":"2024-03-06T07:23:26.384341+0000","last_became_active":"2024-03-06T07:23:25.263214+0000","last_became_peered":"2024-03-06T07:23:25.263214+0000","last_unstale":"2024-03-06T07:23:26.384341+0000","last_undegraded":"2024-03-06T07:23:26.384341+0000","last_fullsized":"2024-03-06T07:23:26.384341+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:46:26.445755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384395+0000","last_change":"2024-03-06T07:23:25.268623+0000","last_active":"2024-03-06T07:23:26.384395+0000","last_peered":"2024-03-06T07:23:26.384395+0000","last_clean":"2024-03-06T07:23:26.384395+0000","last_became_active":"2024-03-06T07:23:25.266517+0000","last_became_peered":"2024-03-06T07:23:25.266517+0000","last_unstale":"2024-03-06T07:23:26.384395+0000","last_undegraded":"2024-03-06T07:23:26.384395+0000","last_fullsized":"2024-03-06T07:23:26.384395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:43.090933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.382972+0000","last_change":"2024-03-06T07:23:25.257062+0000","last_active":"2024-03-06T07:23:26.382972+0000","last_peered":"2024-03-06T07:23:26.382972+0000","last_clean":"2024-03-06T07:23:26.382972+0000","last_became_active":"2024-03-06T07:23:25.256766+0000","last_became_peered":"2024-03-06T07:23:25.256766+0000","last_unstale":"2024-03-06T07:23:26.382972+0000","last_undegraded":"2024-03-06T07:23:26.382972+0000","last_fullsized":"2024-03-06T07:23:26.382972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:25:57.792187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383579+0000","last_change":"2024-03-06T07:23:25.252234+0000","last_active":"2024-03-06T07:23:26.383579+0000","last_peered":"2024-03-06T07:23:26.383579+0000","last_clean":"2024-03-06T07:23:26.383579+0000","last_became_active":"2024-03-06T07:23:25.251957+0000","last_became_peered":"2024-03-06T07:23:25.251957+0000","last_unstale":"2024-03-06T07:23:26.383579+0000","last_undegraded":"2024-03-06T07:23:26.383579+0000","last_fullsized":"2024-03-06T07:23:26.383579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:27.381018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383338+0000","last_change":"2024-03-06T07:23:25.255856+0000","last_active":"2024-03-06T07:23:26.383338+0000","last_peered":"2024-03-06T07:23:26.383338+0000","last_clean":"2024-03-06T07:23:26.383338+0000","last_became_active":"2024-03-06T07:23:25.255574+0000","last_became_peered":"2024-03-06T07:23:25.255574+0000","last_unstale":"2024-03-06T07:23:26.383338+0000","last_undegraded":"2024-03-06T07:23:26.383338+0000","last_fullsized":"2024-03-06T07:23:26.383338+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:53.274693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383273+0000","last_change":"2024-03-06T07:23:25.252257+0000","last_active":"2024-03-06T07:23:26.383273+0000","last_peered":"2024-03-06T07:23:26.383273+0000","last_clean":"2024-03-06T07:23:26.383273+0000","last_became_active":"2024-03-06T07:23:25.251937+0000","last_became_peered":"2024-03-06T07:23:25.251937+0000","last_unstale":"2024-03-06T07:23:26.383273+0000","last_undegraded":"2024-03-06T07:23:26.383273+0000","last_fullsized":"2024-03-06T07:23:26.383273+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:13.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244918+0000","last_change":"2024-03-06T07:23:25.274873+0000","last_active":"2024-03-06T07:23:26.244918+0000","last_peered":"2024-03-06T07:23:26.244918+0000","last_clean":"2024-03-06T07:23:26.244918+0000","last_became_active":"2024-03-06T07:23:25.274276+0000","last_became_peered":"2024-03-06T07:23:25.274276+0000","last_unstale":"2024-03-06T07:23:26.244918+0000","last_undegraded":"2024-03-06T07:23:26.244918+0000","last_fullsized":"2024-03-06T07:23:26.244918+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:59.351547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385650+0000","last_change":"2024-03-06T07:23:25.269702+0000","last_active":"2024-03-06T07:23:26.385650+0000","last_peered":"2024-03-06T07:23:26.385650+0000","last_clean":"2024-03-06T07:23:26.385650+0000","last_became_active":"2024-03-06T07:23:25.269398+0000","last_became_peered":"2024-03-06T07:23:25.269398+0000","last_unstale":"2024-03-06T07:23:26.385650+0000","last_undegraded":"2024-03-06T07:23:26.385650+0000","last_fullsized":"2024-03-06T07:23:26.385650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:02.848995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244665+0000","last_change":"2024-03-06T07:23:25.272301+0000","last_active":"2024-03-06T07:23:26.244665+0000","last_peered":"2024-03-06T07:23:26.244665+0000","last_clean":"2024-03-06T07:23:26.244665+0000","last_became_active":"2024-03-06T07:23:25.271074+0000","last_became_peered":"2024-03-06T07:23:25.271074+0000","last_unstale":"2024-03-06T07:23:26.244665+0000","last_undegraded":"2024-03-06T07:23:26.244665+0000","last_fullsized":"2024-03-06T07:23:26.244665+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:18:45.718798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384534+0000","last_change":"2024-03-06T07:23:25.266786+0000","last_active":"2024-03-06T07:23:26.384534+0000","last_peered":"2024-03-06T07:23:26.384534+0000","last_clean":"2024-03-06T07:23:26.384534+0000","last_became_active":"2024-03-06T07:23:25.266150+0000","last_became_peered":"2024-03-06T07:23:25.266150+0000","last_unstale":"2024-03-06T07:23:26.384534+0000","last_undegraded":"2024-03-06T07:23:26.384534+0000","last_fullsized":"2024-03-06T07:23:26.384534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:02.848995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243625+0000","last_change":"2024-03-06T07:23:25.256962+0000","last_active":"2024-03-06T07:23:26.243625+0000","last_peered":"2024-03-06T07:23:26.243625+0000","last_clean":"2024-03-06T07:23:26.243625+0000","last_became_active":"2024-03-06T07:23:25.256674+0000","last_became_peered":"2024-03-06T07:23:25.256674+0000","last_unstale":"2024-03-06T07:23:26.243625+0000","last_undegraded":"2024-03-06T07:23:26.243625+0000","last_fullsized":"2024-03-06T07:23:26.243625+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:48.663166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383494+0000","last_change":"2024-03-06T07:23:25.255364+0000","last_active":"2024-03-06T07:23:26.383494+0000","last_peered":"2024-03-06T07:23:26.383494+0000","last_clean":"2024-03-06T07:23:26.383494+0000","last_became_active":"2024-03-06T07:23:25.255100+0000","last_became_peered":"2024-03-06T07:23:25.255100+0000","last_unstale":"2024-03-06T07:23:26.383494+0000","last_undegraded":"2024-03-06T07:23:26.383494+0000","last_fullsized":"2024-03-06T07:23:26.383494+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:52:04.546752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244602+0000","last_change":"2024-03-06T07:23:25.274594+0000","last_active":"2024-03-06T07:23:26.244602+0000","last_peered":"2024-03-06T07:23:26.244602+0000","last_clean":"2024-03-06T07:23:26.244602+0000","last_became_active":"2024-03-06T07:23:25.274253+0000","last_became_peered":"2024-03-06T07:23:25.274253+0000","last_unstale":"2024-03-06T07:23:26.244602+0000","last_undegraded":"2024-03-06T07:23:26.244602+0000","last_fullsized":"2024-03-06T07:23:26.244602+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:12.820605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383020+0000","last_change":"2024-03-06T07:23:25.273411+0000","last_active":"2024-03-06T07:23:26.383020+0000","last_peered":"2024-03-06T07:23:26.383020+0000","last_clean":"2024-03-06T07:23:26.383020+0000","last_became_active":"2024-03-06T07:23:25.273149+0000","last_became_peered":"2024-03-06T07:23:25.273149+0000","last_unstale":"2024-03-06T07:23:26.383020+0000","last_undegraded":"2024-03-06T07:23:26.383020+0000","last_fullsized":"2024-03-06T07:23:26.383020+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:53.688818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385006+0000","last_change":"2024-03-06T07:23:25.275388+0000","last_active":"2024-03-06T07:23:26.385006+0000","last_peered":"2024-03-06T07:23:26.385006+0000","last_clean":"2024-03-06T07:23:26.385006+0000","last_became_active":"2024-03-06T07:23:25.274862+0000","last_became_peered":"2024-03-06T07:23:25.274862+0000","last_unstale":"2024-03-06T07:23:26.385006+0000","last_undegraded":"2024-03-06T07:23:26.385006+0000","last_fullsized":"2024-03-06T07:23:26.385006+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:27.771749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385017+0000","last_change":"2024-03-06T07:23:25.266611+0000","last_active":"2024-03-06T07:23:26.385017+0000","last_peered":"2024-03-06T07:23:26.385017+0000","last_clean":"2024-03-06T07:23:26.385017+0000","last_became_active":"2024-03-06T07:23:25.266278+0000","last_became_peered":"2024-03-06T07:23:25.266278+0000","last_unstale":"2024-03-06T07:23:26.385017+0000","last_undegraded":"2024-03-06T07:23:26.385017+0000","last_fullsized":"2024-03-06T07:23:26.385017+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:42:17.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384491+0000","last_change":"2024-03-06T07:23:25.253902+0000","last_active":"2024-03-06T07:23:26.384491+0000","last_peered":"2024-03-06T07:23:26.384491+0000","last_clean":"2024-03-06T07:23:26.384491+0000","last_became_active":"2024-03-06T07:23:25.252988+0000","last_became_peered":"2024-03-06T07:23:25.252988+0000","last_unstale":"2024-03-06T07:23:26.384491+0000","last_undegraded":"2024-03-06T07:23:26.384491+0000","last_fullsized":"2024-03-06T07:23:26.384491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:09.515244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385279+0000","last_change":"2024-03-06T07:23:25.263530+0000","last_active":"2024-03-06T07:23:26.385279+0000","last_peered":"2024-03-06T07:23:26.385279+0000","last_clean":"2024-03-06T07:23:26.385279+0000","last_became_active":"2024-03-06T07:23:25.263161+0000","last_became_peered":"2024-03-06T07:23:25.263161+0000","last_unstale":"2024-03-06T07:23:26.385279+0000","last_undegraded":"2024-03-06T07:23:26.385279+0000","last_fullsized":"2024-03-06T07:23:26.385279+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:22:14.719926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243793+0000","last_change":"2024-03-06T07:23:25.254338+0000","last_active":"2024-03-06T07:23:26.243793+0000","last_peered":"2024-03-06T07:23:26.243793+0000","last_clean":"2024-03-06T07:23:26.243793+0000","last_became_active":"2024-03-06T07:23:25.253627+0000","last_became_peered":"2024-03-06T07:23:25.253627+0000","last_unstale":"2024-03-06T07:23:26.243793+0000","last_undegraded":"2024-03-06T07:23:26.243793+0000","last_fullsized":"2024-03-06T07:23:26.243793+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:34.396639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383672+0000","last_change":"2024-03-06T07:23:25.275241+0000","last_active":"2024-03-06T07:23:26.383672+0000","last_peered":"2024-03-06T07:23:26.383672+0000","last_clean":"2024-03-06T07:23:26.383672+0000","last_became_active":"2024-03-06T07:23:25.274936+0000","last_became_peered":"2024-03-06T07:23:25.274936+0000","last_unstale":"2024-03-06T07:23:26.383672+0000","last_undegraded":"2024-03-06T07:23:26.383672+0000","last_fullsized":"2024-03-06T07:23:26.383672+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:55.156808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385119+0000","last_change":"2024-03-06T07:23:25.272953+0000","last_active":"2024-03-06T07:23:26.385119+0000","last_peered":"2024-03-06T07:23:26.385119+0000","last_clean":"2024-03-06T07:23:26.385119+0000","last_became_active":"2024-03-06T07:23:25.272644+0000","last_became_peered":"2024-03-06T07:23:25.272644+0000","last_unstale":"2024-03-06T07:23:26.385119+0000","last_undegraded":"2024-03-06T07:23:26.385119+0000","last_fullsized":"2024-03-06T07:23:26.385119+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:55.156808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243452+0000","last_change":"2024-03-06T07:23:25.268878+0000","last_active":"2024-03-06T07:23:26.243452+0000","last_peered":"2024-03-06T07:23:26.243452+0000","last_clean":"2024-03-06T07:23:26.243452+0000","last_became_active":"2024-03-06T07:23:25.268570+0000","last_became_peered":"2024-03-06T07:23:25.268570+0000","last_unstale":"2024-03-06T07:23:26.243452+0000","last_undegraded":"2024-03-06T07:23:26.243452+0000","last_fullsized":"2024-03-06T07:23:26.243452+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:50.348920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244545+0000","last_change":"2024-03-06T07:23:25.273172+0000","last_active":"2024-03-06T07:23:26.244545+0000","last_peered":"2024-03-06T07:23:26.244545+0000","last_clean":"2024-03-06T07:23:26.244545+0000","last_became_active":"2024-03-06T07:23:25.272652+0000","last_became_peered":"2024-03-06T07:23:25.272652+0000","last_unstale":"2024-03-06T07:23:26.244545+0000","last_undegraded":"2024-03-06T07:23:26.244545+0000","last_fullsized":"2024-03-06T07:23:26.244545+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:11:50.555788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383408+0000","last_change":"2024-03-06T07:23:24.250096+0000","last_active":"2024-03-06T07:23:26.383408+0000","last_peered":"2024-03-06T07:23:26.383408+0000","last_clean":"2024-03-06T07:23:26.383408+0000","last_became_active":"2024-03-06T07:23:24.249807+0000","last_became_peered":"2024-03-06T07:23:24.249807+0000","last_unstale":"2024-03-06T07:23:26.383408+0000","last_undegraded":"2024-03-06T07:23:26.383408+0000","last_fullsized":"2024-03-06T07:23:26.383408+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:17.124610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243452+0000","last_change":"2024-03-06T07:23:25.262106+0000","last_active":"2024-03-06T07:23:26.243452+0000","last_peered":"2024-03-06T07:23:26.243452+0000","last_clean":"2024-03-06T07:23:26.243452+0000","last_became_active":"2024-03-06T07:23:25.261082+0000","last_became_peered":"2024-03-06T07:23:25.261082+0000","last_unstale":"2024-03-06T07:23:26.243452+0000","last_undegraded":"2024-03-06T07:23:26.243452+0000","last_fullsized":"2024-03-06T07:23:26.243452+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:55.084144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244144+0000","last_change":"2024-03-06T07:23:24.247409+0000","last_active":"2024-03-06T07:23:26.244144+0000","last_peered":"2024-03-06T07:23:26.244144+0000","last_clean":"2024-03-06T07:23:26.244144+0000","last_became_active":"2024-03-06T07:23:24.247083+0000","last_became_peered":"2024-03-06T07:23:24.247083+0000","last_unstale":"2024-03-06T07:23:26.244144+0000","last_undegraded":"2024-03-06T07:23:26.244144+0000","last_fullsized":"2024-03-06T07:23:26.244144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:53.217339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383815+0000","last_change":"2024-03-06T07:23:25.269708+0000","last_active":"2024-03-06T07:23:26.383815+0000","last_peered":"2024-03-06T07:23:26.383815+0000","last_clean":"2024-03-06T07:23:26.383815+0000","last_became_active":"2024-03-06T07:23:25.269431+0000","last_became_peered":"2024-03-06T07:23:25.269431+0000","last_unstale":"2024-03-06T07:23:26.383815+0000","last_undegraded":"2024-03-06T07:23:26.383815+0000","last_fullsized":"2024-03-06T07:23:26.383815+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:22:14.719926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385032+0000","last_change":"2024-03-06T07:23:24.247601+0000","last_active":"2024-03-06T07:23:26.385032+0000","last_peered":"2024-03-06T07:23:26.385032+0000","last_clean":"2024-03-06T07:23:26.385032+0000","last_became_active":"2024-03-06T07:23:24.246790+0000","last_became_peered":"2024-03-06T07:23:24.246790+0000","last_unstale":"2024-03-06T07:23:26.385032+0000","last_undegraded":"2024-03-06T07:23:26.385032+0000","last_fullsized":"2024-03-06T07:23:26.385032+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:07.101881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384480+0000","last_change":"2024-03-06T07:23:25.268848+0000","last_active":"2024-03-06T07:23:26.384480+0000","last_peered":"2024-03-06T07:23:26.384480+0000","last_clean":"2024-03-06T07:23:26.384480+0000","last_became_active":"2024-03-06T07:23:25.268574+0000","last_became_peered":"2024-03-06T07:23:25.268574+0000","last_unstale":"2024-03-06T07:23:26.384480+0000","last_undegraded":"2024-03-06T07:23:26.384480+0000","last_fullsized":"2024-03-06T07:23:26.384480+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:14.671883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383913+0000","last_change":"2024-03-06T07:23:24.240691+0000","last_active":"2024-03-06T07:23:26.383913+0000","last_peered":"2024-03-06T07:23:26.383913+0000","last_clean":"2024-03-06T07:23:26.383913+0000","last_became_active":"2024-03-06T07:23:24.240338+0000","last_became_peered":"2024-03-06T07:23:24.240338+0000","last_unstale":"2024-03-06T07:23:26.383913+0000","last_undegraded":"2024-03-06T07:23:26.383913+0000","last_fullsized":"2024-03-06T07:23:26.383913+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:19.138222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383314+0000","last_change":"2024-03-06T07:23:25.251061+0000","last_active":"2024-03-06T07:23:26.383314+0000","last_peered":"2024-03-06T07:23:26.383314+0000","last_clean":"2024-03-06T07:23:26.383314+0000","last_became_active":"2024-03-06T07:23:25.250719+0000","last_became_peered":"2024-03-06T07:23:25.250719+0000","last_unstale":"2024-03-06T07:23:26.383314+0000","last_undegraded":"2024-03-06T07:23:26.383314+0000","last_fullsized":"2024-03-06T07:23:26.383314+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:09.515244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383839+0000","last_change":"2024-03-06T07:23:24.245937+0000","last_active":"2024-03-06T07:23:26.383839+0000","last_peered":"2024-03-06T07:23:26.383839+0000","last_clean":"2024-03-06T07:23:26.383839+0000","last_became_active":"2024-03-06T07:23:24.245614+0000","last_became_peered":"2024-03-06T07:23:24.245614+0000","last_unstale":"2024-03-06T07:23:26.383839+0000","last_undegraded":"2024-03-06T07:23:26.383839+0000","last_fullsized":"2024-03-06T07:23:26.383839+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:46:25.441866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244978+0000","last_change":"2024-03-06T07:23:25.264849+0000","last_active":"2024-03-06T07:23:26.244978+0000","last_peered":"2024-03-06T07:23:26.244978+0000","last_clean":"2024-03-06T07:23:26.244978+0000","last_became_active":"2024-03-06T07:23:25.264162+0000","last_became_peered":"2024-03-06T07:23:25.264162+0000","last_unstale":"2024-03-06T07:23:26.244978+0000","last_undegraded":"2024-03-06T07:23:26.244978+0000","last_fullsized":"2024-03-06T07:23:26.244978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:57:30.608747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384550+0000","last_change":"2024-03-06T07:23:24.249153+0000","last_active":"2024-03-06T07:23:26.384550+0000","last_peered":"2024-03-06T07:23:26.384550+0000","last_clean":"2024-03-06T07:23:26.384550+0000","last_became_active":"2024-03-06T07:23:24.248846+0000","last_became_peered":"2024-03-06T07:23:24.248846+0000","last_unstale":"2024-03-06T07:23:26.384550+0000","last_undegraded":"2024-03-06T07:23:26.384550+0000","last_fullsized":"2024-03-06T07:23:26.384550+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:52.684929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385684+0000","last_change":"2024-03-06T07:23:25.260740+0000","last_active":"2024-03-06T07:23:26.385684+0000","last_peered":"2024-03-06T07:23:26.385684+0000","last_clean":"2024-03-06T07:23:26.385684+0000","last_became_active":"2024-03-06T07:23:25.260406+0000","last_became_peered":"2024-03-06T07:23:25.260406+0000","last_unstale":"2024-03-06T07:23:26.385684+0000","last_undegraded":"2024-03-06T07:23:26.385684+0000","last_fullsized":"2024-03-06T07:23:26.385684+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:27.771749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383631+0000","last_change":"2024-03-06T07:23:24.245204+0000","last_active":"2024-03-06T07:23:26.383631+0000","last_peered":"2024-03-06T07:23:26.383631+0000","last_clean":"2024-03-06T07:23:26.383631+0000","last_became_active":"2024-03-06T07:23:24.244893+0000","last_became_peered":"2024-03-06T07:23:24.244893+0000","last_unstale":"2024-03-06T07:23:26.383631+0000","last_undegraded":"2024-03-06T07:23:26.383631+0000","last_fullsized":"2024-03-06T07:23:26.383631+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:19.138222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383421+0000","last_change":"2024-03-06T07:23:25.267135+0000","last_active":"2024-03-06T07:23:26.383421+0000","last_peered":"2024-03-06T07:23:26.383421+0000","last_clean":"2024-03-06T07:23:26.383421+0000","last_became_active":"2024-03-06T07:23:25.266625+0000","last_became_peered":"2024-03-06T07:23:25.266625+0000","last_unstale":"2024-03-06T07:23:26.383421+0000","last_undegraded":"2024-03-06T07:23:26.383421+0000","last_fullsized":"2024-03-06T07:23:26.383421+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:49:16.411867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.382775+0000","last_change":"2024-03-06T07:23:24.246502+0000","last_active":"2024-03-06T07:23:26.382775+0000","last_peered":"2024-03-06T07:23:26.382775+0000","last_clean":"2024-03-06T07:23:26.382775+0000","last_became_active":"2024-03-06T07:23:24.246230+0000","last_became_peered":"2024-03-06T07:23:24.246230+0000","last_unstale":"2024-03-06T07:23:26.382775+0000","last_undegraded":"2024-03-06T07:23:26.382775+0000","last_fullsized":"2024-03-06T07:23:26.382775+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:52:03.542864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244404+0000","last_change":"2024-03-06T07:23:25.272172+0000","last_active":"2024-03-06T07:23:26.244404+0000","last_peered":"2024-03-06T07:23:26.244404+0000","last_clean":"2024-03-06T07:23:26.244404+0000","last_became_active":"2024-03-06T07:23:25.271271+0000","last_became_peered":"2024-03-06T07:23:25.271271+0000","last_unstale":"2024-03-06T07:23:26.244404+0000","last_undegraded":"2024-03-06T07:23:26.244404+0000","last_fullsized":"2024-03-06T07:23:26.244404+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:17:21.504560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385109+0000","last_change":"2024-03-06T07:23:24.248199+0000","last_active":"2024-03-06T07:23:26.385109+0000","last_peered":"2024-03-06T07:23:26.385109+0000","last_clean":"2024-03-06T07:23:26.385109+0000","last_became_active":"2024-03-06T07:23:24.247705+0000","last_became_peered":"2024-03-06T07:23:24.247705+0000","last_unstale":"2024-03-06T07:23:26.385109+0000","last_undegraded":"2024-03-06T07:23:26.385109+0000","last_fullsized":"2024-03-06T07:23:26.385109+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:39.219381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383666+0000","last_change":"2024-03-06T07:23:25.264214+0000","last_active":"2024-03-06T07:23:26.383666+0000","last_peered":"2024-03-06T07:23:26.383666+0000","last_clean":"2024-03-06T07:23:26.383666+0000","last_became_active":"2024-03-06T07:23:25.263700+0000","last_became_peered":"2024-03-06T07:23:25.263700+0000","last_unstale":"2024-03-06T07:23:26.383666+0000","last_undegraded":"2024-03-06T07:23:26.383666+0000","last_fullsized":"2024-03-06T07:23:26.383666+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:22:14.719926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244269+0000","last_change":"2024-03-06T07:23:24.245504+0000","last_active":"2024-03-06T07:23:26.244269+0000","last_peered":"2024-03-06T07:23:26.244269+0000","last_clean":"2024-03-06T07:23:26.244269+0000","last_became_active":"2024-03-06T07:23:24.245228+0000","last_became_peered":"2024-03-06T07:23:24.245228+0000","last_unstale":"2024-03-06T07:23:26.244269+0000","last_undegraded":"2024-03-06T07:23:26.244269+0000","last_fullsized":"2024-03-06T07:23:26.244269+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:32:36.927368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243458+0000","last_change":"2024-03-06T07:23:25.252134+0000","last_active":"2024-03-06T07:23:26.243458+0000","last_peered":"2024-03-06T07:23:26.243458+0000","last_clean":"2024-03-06T07:23:26.243458+0000","last_became_active":"2024-03-06T07:23:25.251802+0000","last_became_peered":"2024-03-06T07:23:25.251802+0000","last_unstale":"2024-03-06T07:23:26.243458+0000","last_undegraded":"2024-03-06T07:23:26.243458+0000","last_fullsized":"2024-03-06T07:23:26.243458+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:46:03.825668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385596+0000","last_change":"2024-03-06T07:23:24.242127+0000","last_active":"2024-03-06T07:23:26.385596+0000","last_peered":"2024-03-06T07:23:26.385596+0000","last_clean":"2024-03-06T07:23:26.385596+0000","last_became_active":"2024-03-06T07:23:24.241524+0000","last_became_peered":"2024-03-06T07:23:24.241524+0000","last_unstale":"2024-03-06T07:23:26.385596+0000","last_undegraded":"2024-03-06T07:23:26.385596+0000","last_fullsized":"2024-03-06T07:23:26.385596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:42:16.325453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383600+0000","last_change":"2024-03-06T07:23:25.260904+0000","last_active":"2024-03-06T07:23:26.383600+0000","last_peered":"2024-03-06T07:23:26.383600+0000","last_clean":"2024-03-06T07:23:26.383600+0000","last_became_active":"2024-03-06T07:23:25.260073+0000","last_became_peered":"2024-03-06T07:23:25.260073+0000","last_unstale":"2024-03-06T07:23:26.383600+0000","last_undegraded":"2024-03-06T07:23:26.383600+0000","last_fullsized":"2024-03-06T07:23:26.383600+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:55.156808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385360+0000","last_change":"2024-03-06T07:23:24.246676+0000","last_active":"2024-03-06T07:23:26.385360+0000","last_peered":"2024-03-06T07:23:26.385360+0000","last_clean":"2024-03-06T07:23:26.385360+0000","last_became_active":"2024-03-06T07:23:24.246408+0000","last_became_peered":"2024-03-06T07:23:24.246408+0000","last_unstale":"2024-03-06T07:23:26.385360+0000","last_undegraded":"2024-03-06T07:23:26.385360+0000","last_fullsized":"2024-03-06T07:23:26.385360+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:10.097035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383658+0000","last_change":"2024-03-06T07:23:25.272460+0000","last_active":"2024-03-06T07:23:26.383658+0000","last_peered":"2024-03-06T07:23:26.383658+0000","last_clean":"2024-03-06T07:23:26.383658+0000","last_became_active":"2024-03-06T07:23:25.272210+0000","last_became_peered":"2024-03-06T07:23:25.272210+0000","last_unstale":"2024-03-06T07:23:26.383658+0000","last_undegraded":"2024-03-06T07:23:26.383658+0000","last_fullsized":"2024-03-06T07:23:26.383658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:42:17.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383651+0000","last_change":"2024-03-06T07:23:24.242187+0000","last_active":"2024-03-06T07:23:26.383651+0000","last_peered":"2024-03-06T07:23:26.383651+0000","last_clean":"2024-03-06T07:23:26.383651+0000","last_became_active":"2024-03-06T07:23:24.241930+0000","last_became_peered":"2024-03-06T07:23:24.241930+0000","last_unstale":"2024-03-06T07:23:26.383651+0000","last_undegraded":"2024-03-06T07:23:26.383651+0000","last_fullsized":"2024-03-06T07:23:26.383651+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:52.253736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384217+0000","last_change":"2024-03-06T07:23:25.263138+0000","last_active":"2024-03-06T07:23:26.384217+0000","last_peered":"2024-03-06T07:23:26.384217+0000","last_clean":"2024-03-06T07:23:26.384217+0000","last_became_active":"2024-03-06T07:23:25.261482+0000","last_became_peered":"2024-03-06T07:23:25.261482+0000","last_unstale":"2024-03-06T07:23:26.384217+0000","last_undegraded":"2024-03-06T07:23:26.384217+0000","last_fullsized":"2024-03-06T07:23:26.384217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:39.384249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383095+0000","last_change":"2024-03-06T07:23:24.245524+0000","last_active":"2024-03-06T07:23:26.383095+0000","last_peered":"2024-03-06T07:23:26.383095+0000","last_clean":"2024-03-06T07:23:26.383095+0000","last_became_active":"2024-03-06T07:23:24.245245+0000","last_became_peered":"2024-03-06T07:23:24.245245+0000","last_unstale":"2024-03-06T07:23:26.383095+0000","last_undegraded":"2024-03-06T07:23:26.383095+0000","last_fullsized":"2024-03-06T07:23:26.383095+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:38.380361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244057+0000","last_change":"2024-03-06T07:23:25.268307+0000","last_active":"2024-03-06T07:23:26.244057+0000","last_peered":"2024-03-06T07:23:26.244057+0000","last_clean":"2024-03-06T07:23:26.244057+0000","last_became_active":"2024-03-06T07:23:25.268032+0000","last_became_peered":"2024-03-06T07:23:25.268032+0000","last_unstale":"2024-03-06T07:23:26.244057+0000","last_undegraded":"2024-03-06T07:23:26.244057+0000","last_fullsized":"2024-03-06T07:23:26.244057+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:32:58.461873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243500+0000","last_change":"2024-03-06T07:23:24.243043+0000","last_active":"2024-03-06T07:23:26.243500+0000","last_peered":"2024-03-06T07:23:26.243500+0000","last_clean":"2024-03-06T07:23:26.243500+0000","last_became_active":"2024-03-06T07:23:24.242793+0000","last_became_peered":"2024-03-06T07:23:24.242793+0000","last_unstale":"2024-03-06T07:23:26.243500+0000","last_undegraded":"2024-03-06T07:23:26.243500+0000","last_fullsized":"2024-03-06T07:23:26.243500+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:27.281947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.245063+0000","last_change":"2024-03-06T07:23:25.271882+0000","last_active":"2024-03-06T07:23:26.245063+0000","last_peered":"2024-03-06T07:23:26.245063+0000","last_clean":"2024-03-06T07:23:26.245063+0000","last_became_active":"2024-03-06T07:23:25.271617+0000","last_became_peered":"2024-03-06T07:23:25.271617+0000","last_unstale":"2024-03-06T07:23:26.245063+0000","last_undegraded":"2024-03-06T07:23:26.245063+0000","last_fullsized":"2024-03-06T07:23:26.245063+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:08.928271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384346+0000","last_change":"2024-03-06T07:23:24.244518+0000","last_active":"2024-03-06T07:23:26.384346+0000","last_peered":"2024-03-06T07:23:26.384346+0000","last_clean":"2024-03-06T07:23:26.384346+0000","last_became_active":"2024-03-06T07:23:24.244154+0000","last_became_peered":"2024-03-06T07:23:24.244154+0000","last_unstale":"2024-03-06T07:23:26.384346+0000","last_undegraded":"2024-03-06T07:23:26.384346+0000","last_fullsized":"2024-03-06T07:23:26.384346+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:52.253736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243710+0000","last_change":"2024-03-06T07:23:25.274660+0000","last_active":"2024-03-06T07:23:26.243710+0000","last_peered":"2024-03-06T07:23:26.243710+0000","last_clean":"2024-03-06T07:23:26.243710+0000","last_became_active":"2024-03-06T07:23:25.273889+0000","last_became_peered":"2024-03-06T07:23:25.273889+0000","last_unstale":"2024-03-06T07:23:26.243710+0000","last_undegraded":"2024-03-06T07:23:26.243710+0000","last_fullsized":"2024-03-06T07:23:26.243710+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:16:39.996044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384701+0000","last_change":"2024-03-06T07:23:24.247115+0000","last_active":"2024-03-06T07:23:26.384701+0000","last_peered":"2024-03-06T07:23:26.384701+0000","last_clean":"2024-03-06T07:23:26.384701+0000","last_became_active":"2024-03-06T07:23:24.246835+0000","last_became_peered":"2024-03-06T07:23:24.246835+0000","last_unstale":"2024-03-06T07:23:26.384701+0000","last_undegraded":"2024-03-06T07:23:26.384701+0000","last_fullsized":"2024-03-06T07:23:26.384701+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:16:29.901293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383927+0000","last_change":"2024-03-06T07:23:25.272097+0000","last_active":"2024-03-06T07:23:26.383927+0000","last_peered":"2024-03-06T07:23:26.383927+0000","last_clean":"2024-03-06T07:23:26.383927+0000","last_became_active":"2024-03-06T07:23:25.271835+0000","last_became_peered":"2024-03-06T07:23:25.271835+0000","last_unstale":"2024-03-06T07:23:26.383927+0000","last_undegraded":"2024-03-06T07:23:26.383927+0000","last_fullsized":"2024-03-06T07:23:26.383927+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:53.688818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384060+0000","last_change":"2024-03-06T07:23:24.242459+0000","last_active":"2024-03-06T07:23:26.384060+0000","last_peered":"2024-03-06T07:23:26.384060+0000","last_clean":"2024-03-06T07:23:26.384060+0000","last_became_active":"2024-03-06T07:23:24.241848+0000","last_became_peered":"2024-03-06T07:23:24.241848+0000","last_unstale":"2024-03-06T07:23:26.384060+0000","last_undegraded":"2024-03-06T07:23:26.384060+0000","last_fullsized":"2024-03-06T07:23:26.384060+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:25:56.788299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383556+0000","last_change":"2024-03-06T07:23:25.254489+0000","last_active":"2024-03-06T07:23:26.383556+0000","last_peered":"2024-03-06T07:23:26.383556+0000","last_clean":"2024-03-06T07:23:26.383556+0000","last_became_active":"2024-03-06T07:23:25.254188+0000","last_became_peered":"2024-03-06T07:23:25.254188+0000","last_unstale":"2024-03-06T07:23:26.383556+0000","last_undegraded":"2024-03-06T07:23:26.383556+0000","last_fullsized":"2024-03-06T07:23:26.383556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:18.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383757+0000","last_change":"2024-03-06T07:23:24.252717+0000","last_active":"2024-03-06T07:23:26.383757+0000","last_peered":"2024-03-06T07:23:26.383757+0000","last_clean":"2024-03-06T07:23:26.383757+0000","last_became_active":"2024-03-06T07:23:24.252429+0000","last_became_peered":"2024-03-06T07:23:24.252429+0000","last_unstale":"2024-03-06T07:23:26.383757+0000","last_undegraded":"2024-03-06T07:23:26.383757+0000","last_fullsized":"2024-03-06T07:23:26.383757+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:38.380361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244898+0000","last_change":"2024-03-06T07:23:25.271040+0000","last_active":"2024-03-06T07:23:26.244898+0000","last_peered":"2024-03-06T07:23:26.244898+0000","last_clean":"2024-03-06T07:23:26.244898+0000","last_became_active":"2024-03-06T07:23:25.269742+0000","last_became_peered":"2024-03-06T07:23:25.269742+0000","last_unstale":"2024-03-06T07:23:26.244898+0000","last_undegraded":"2024-03-06T07:23:26.244898+0000","last_fullsized":"2024-03-06T07:23:26.244898+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:11.896743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243878+0000","last_change":"2024-03-06T07:23:24.248709+0000","last_active":"2024-03-06T07:23:26.243878+0000","last_peered":"2024-03-06T07:23:26.243878+0000","last_clean":"2024-03-06T07:23:26.243878+0000","last_became_active":"2024-03-06T07:23:24.247184+0000","last_became_peered":"2024-03-06T07:23:24.247184+0000","last_unstale":"2024-03-06T07:23:26.243878+0000","last_undegraded":"2024-03-06T07:23:26.243878+0000","last_fullsized":"2024-03-06T07:23:26.243878+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:02:37.500463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244838+0000","last_change":"2024-03-06T07:23:25.272547+0000","last_active":"2024-03-06T07:23:26.244838+0000","last_peered":"2024-03-06T07:23:26.244838+0000","last_clean":"2024-03-06T07:23:26.244838+0000","last_became_active":"2024-03-06T07:23:25.271492+0000","last_became_peered":"2024-03-06T07:23:25.271492+0000","last_unstale":"2024-03-06T07:23:26.244838+0000","last_undegraded":"2024-03-06T07:23:26.244838+0000","last_fullsized":"2024-03-06T07:23:26.244838+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:48.641136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243368+0000","last_change":"2024-03-06T07:23:24.246510+0000","last_active":"2024-03-06T07:23:26.243368+0000","last_peered":"2024-03-06T07:23:26.243368+0000","last_clean":"2024-03-06T07:23:26.243368+0000","last_became_active":"2024-03-06T07:23:24.246222+0000","last_became_peered":"2024-03-06T07:23:24.246222+0000","last_unstale":"2024-03-06T07:23:26.243368+0000","last_undegraded":"2024-03-06T07:23:26.243368+0000","last_fullsized":"2024-03-06T07:23:26.243368+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:08.841799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383282+0000","last_change":"2024-03-06T07:23:25.262342+0000","last_active":"2024-03-06T07:23:26.383282+0000","last_peered":"2024-03-06T07:23:26.383282+0000","last_clean":"2024-03-06T07:23:26.383282+0000","last_became_active":"2024-03-06T07:23:25.262040+0000","last_became_peered":"2024-03-06T07:23:25.262040+0000","last_unstale":"2024-03-06T07:23:26.383282+0000","last_undegraded":"2024-03-06T07:23:26.383282+0000","last_fullsized":"2024-03-06T07:23:26.383282+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:14.671883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385201+0000","last_change":"2024-03-06T07:23:24.244174+0000","last_active":"2024-03-06T07:23:26.385201+0000","last_peered":"2024-03-06T07:23:26.385201+0000","last_clean":"2024-03-06T07:23:26.385201+0000","last_became_active":"2024-03-06T07:23:24.243046+0000","last_became_peered":"2024-03-06T07:23:24.243046+0000","last_unstale":"2024-03-06T07:23:26.385201+0000","last_undegraded":"2024-03-06T07:23:26.385201+0000","last_fullsized":"2024-03-06T07:23:26.385201+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:27.359045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243987+0000","last_change":"2024-03-06T07:23:25.254072+0000","last_active":"2024-03-06T07:23:26.243987+0000","last_peered":"2024-03-06T07:23:26.243987+0000","last_clean":"2024-03-06T07:23:26.243987+0000","last_became_active":"2024-03-06T07:23:25.253558+0000","last_became_peered":"2024-03-06T07:23:25.253558+0000","last_unstale":"2024-03-06T07:23:26.243987+0000","last_undegraded":"2024-03-06T07:23:26.243987+0000","last_fullsized":"2024-03-06T07:23:26.243987+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:31.915692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243681+0000","last_change":"2024-03-06T07:23:24.245809+0000","last_active":"2024-03-06T07:23:26.243681+0000","last_peered":"2024-03-06T07:23:26.243681+0000","last_clean":"2024-03-06T07:23:26.243681+0000","last_became_active":"2024-03-06T07:23:24.245527+0000","last_became_peered":"2024-03-06T07:23:24.245527+0000","last_unstale":"2024-03-06T07:23:26.243681+0000","last_undegraded":"2024-03-06T07:23:26.243681+0000","last_fullsized":"2024-03-06T07:23:26.243681+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:41.382059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244255+0000","last_change":"2024-03-06T07:23:25.270547+0000","last_active":"2024-03-06T07:23:26.244255+0000","last_peered":"2024-03-06T07:23:26.244255+0000","last_clean":"2024-03-06T07:23:26.244255+0000","last_became_active":"2024-03-06T07:23:25.270181+0000","last_became_peered":"2024-03-06T07:23:25.270181+0000","last_unstale":"2024-03-06T07:23:26.244255+0000","last_undegraded":"2024-03-06T07:23:26.244255+0000","last_fullsized":"2024-03-06T07:23:26.244255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:58.654626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384104+0000","last_change":"2024-03-06T07:23:24.242259+0000","last_active":"2024-03-06T07:23:26.384104+0000","last_peered":"2024-03-06T07:23:26.384104+0000","last_clean":"2024-03-06T07:23:26.384104+0000","last_became_active":"2024-03-06T07:23:24.241958+0000","last_became_peered":"2024-03-06T07:23:24.241958+0000","last_unstale":"2024-03-06T07:23:26.384104+0000","last_undegraded":"2024-03-06T07:23:26.384104+0000","last_fullsized":"2024-03-06T07:23:26.384104+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.421375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383540+0000","last_change":"2024-03-06T07:23:25.267432+0000","last_active":"2024-03-06T07:23:26.383540+0000","last_peered":"2024-03-06T07:23:26.383540+0000","last_clean":"2024-03-06T07:23:26.383540+0000","last_became_active":"2024-03-06T07:23:25.267175+0000","last_became_peered":"2024-03-06T07:23:25.267175+0000","last_unstale":"2024-03-06T07:23:26.383540+0000","last_undegraded":"2024-03-06T07:23:26.383540+0000","last_fullsized":"2024-03-06T07:23:26.383540+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:18.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244522+0000","last_change":"2024-03-06T07:23:24.246996+0000","last_active":"2024-03-06T07:23:26.244522+0000","last_peered":"2024-03-06T07:23:26.244522+0000","last_clean":"2024-03-06T07:23:26.244522+0000","last_became_active":"2024-03-06T07:23:24.246685+0000","last_became_peered":"2024-03-06T07:23:24.246685+0000","last_unstale":"2024-03-06T07:23:26.244522+0000","last_undegraded":"2024-03-06T07:23:26.244522+0000","last_fullsized":"2024-03-06T07:23:26.244522+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:57.757457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383028+0000","last_change":"2024-03-06T07:23:25.261458+0000","last_active":"2024-03-06T07:23:26.383028+0000","last_peered":"2024-03-06T07:23:26.383028+0000","last_clean":"2024-03-06T07:23:26.383028+0000","last_became_active":"2024-03-06T07:23:25.261178+0000","last_became_peered":"2024-03-06T07:23:25.261178+0000","last_unstale":"2024-03-06T07:23:26.383028+0000","last_undegraded":"2024-03-06T07:23:26.383028+0000","last_fullsized":"2024-03-06T07:23:26.383028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:46:26.445755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384344+0000","last_change":"2024-03-06T07:23:24.244748+0000","last_active":"2024-03-06T07:23:26.384344+0000","last_peered":"2024-03-06T07:23:26.384344+0000","last_clean":"2024-03-06T07:23:26.384344+0000","last_became_active":"2024-03-06T07:23:24.244211+0000","last_became_peered":"2024-03-06T07:23:24.244211+0000","last_unstale":"2024-03-06T07:23:26.384344+0000","last_undegraded":"2024-03-06T07:23:26.384344+0000","last_fullsized":"2024-03-06T07:23:26.384344+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:49.051294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384343+0000","last_change":"2024-03-06T07:23:25.251849+0000","last_active":"2024-03-06T07:23:26.384343+0000","last_peered":"2024-03-06T07:23:26.384343+0000","last_clean":"2024-03-06T07:23:26.384343+0000","last_became_active":"2024-03-06T07:23:25.251563+0000","last_became_peered":"2024-03-06T07:23:25.251563+0000","last_unstale":"2024-03-06T07:23:26.384343+0000","last_undegraded":"2024-03-06T07:23:26.384343+0000","last_fullsized":"2024-03-06T07:23:26.384343+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383999+0000","last_change":"2024-03-06T07:23:24.248695+0000","last_active":"2024-03-06T07:23:26.383999+0000","last_peered":"2024-03-06T07:23:26.383999+0000","last_clean":"2024-03-06T07:23:26.383999+0000","last_became_active":"2024-03-06T07:23:24.248417+0000","last_became_peered":"2024-03-06T07:23:24.248417+0000","last_unstale":"2024-03-06T07:23:26.383999+0000","last_undegraded":"2024-03-06T07:23:26.383999+0000","last_fullsized":"2024-03-06T07:23:26.383999+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:39.219381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243992+0000","last_change":"2024-03-06T07:23:25.253100+0000","last_active":"2024-03-06T07:23:26.243992+0000","last_peered":"2024-03-06T07:23:26.243992+0000","last_clean":"2024-03-06T07:23:26.243992+0000","last_became_active":"2024-03-06T07:23:25.252817+0000","last_became_peered":"2024-03-06T07:23:25.252817+0000","last_unstale":"2024-03-06T07:23:26.243992+0000","last_undegraded":"2024-03-06T07:23:26.243992+0000","last_fullsized":"2024-03-06T07:23:26.243992+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:37.275309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384289+0000","last_change":"2024-03-06T07:23:24.246199+0000","last_active":"2024-03-06T07:23:26.384289+0000","last_peered":"2024-03-06T07:23:26.384289+0000","last_clean":"2024-03-06T07:23:26.384289+0000","last_became_active":"2024-03-06T07:23:24.245854+0000","last_became_peered":"2024-03-06T07:23:24.245854+0000","last_unstale":"2024-03-06T07:23:26.384289+0000","last_undegraded":"2024-03-06T07:23:26.384289+0000","last_fullsized":"2024-03-06T07:23:26.384289+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:21.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243943+0000","last_change":"2024-03-06T07:23:25.251996+0000","last_active":"2024-03-06T07:23:26.243943+0000","last_peered":"2024-03-06T07:23:26.243943+0000","last_clean":"2024-03-06T07:23:26.243943+0000","last_became_active":"2024-03-06T07:23:25.251687+0000","last_became_peered":"2024-03-06T07:23:25.251687+0000","last_unstale":"2024-03-06T07:23:26.243943+0000","last_undegraded":"2024-03-06T07:23:26.243943+0000","last_fullsized":"2024-03-06T07:23:26.243943+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:49.185940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243499+0000","last_change":"2024-03-06T07:23:24.244313+0000","last_active":"2024-03-06T07:23:26.243499+0000","last_peered":"2024-03-06T07:23:26.243499+0000","last_clean":"2024-03-06T07:23:26.243499+0000","last_became_active":"2024-03-06T07:23:24.244059+0000","last_became_peered":"2024-03-06T07:23:24.244059+0000","last_unstale":"2024-03-06T07:23:26.243499+0000","last_undegraded":"2024-03-06T07:23:26.243499+0000","last_fullsized":"2024-03-06T07:23:26.243499+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:07.615061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383720+0000","last_change":"2024-03-06T07:23:25.265934+0000","last_active":"2024-03-06T07:23:26.383720+0000","last_peered":"2024-03-06T07:23:26.383720+0000","last_clean":"2024-03-06T07:23:26.383720+0000","last_became_active":"2024-03-06T07:23:25.265661+0000","last_became_peered":"2024-03-06T07:23:25.265661+0000","last_unstale":"2024-03-06T07:23:26.383720+0000","last_undegraded":"2024-03-06T07:23:26.383720+0000","last_fullsized":"2024-03-06T07:23:26.383720+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:21.856470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.385289+0000","last_change":"2024-03-06T07:23:24.247137+0000","last_active":"2024-03-06T07:23:26.385289+0000","last_peered":"2024-03-06T07:23:26.385289+0000","last_clean":"2024-03-06T07:23:26.385289+0000","last_became_active":"2024-03-06T07:23:24.246871+0000","last_became_peered":"2024-03-06T07:23:24.246871+0000","last_unstale":"2024-03-06T07:23:26.385289+0000","last_undegraded":"2024-03-06T07:23:26.385289+0000","last_fullsized":"2024-03-06T07:23:26.385289+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:00.082424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.384764+0000","last_change":"2024-03-06T07:23:25.255986+0000","last_active":"2024-03-06T07:23:26.384764+0000","last_peered":"2024-03-06T07:23:26.384764+0000","last_clean":"2024-03-06T07:23:26.384764+0000","last_became_active":"2024-03-06T07:23:25.255705+0000","last_became_peered":"2024-03-06T07:23:25.255705+0000","last_unstale":"2024-03-06T07:23:26.384764+0000","last_undegraded":"2024-03-06T07:23:26.384764+0000","last_fullsized":"2024-03-06T07:23:26.384764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:49:16.411867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.243767+0000","last_change":"2024-03-06T07:23:24.243574+0000","last_active":"2024-03-06T07:23:26.243767+0000","last_peered":"2024-03-06T07:23:26.243767+0000","last_clean":"2024-03-06T07:23:26.243767+0000","last_became_active":"2024-03-06T07:23:24.243297+0000","last_became_peered":"2024-03-06T07:23:24.243297+0000","last_unstale":"2024-03-06T07:23:26.243767+0000","last_undegraded":"2024-03-06T07:23:26.243767+0000","last_fullsized":"2024-03-06T07:23:26.243767+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:53:45.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383998+0000","last_change":"2024-03-06T07:23:25.271817+0000","last_active":"2024-03-06T07:23:26.383998+0000","last_peered":"2024-03-06T07:23:26.383998+0000","last_clean":"2024-03-06T07:23:26.383998+0000","last_became_active":"2024-03-06T07:23:25.271564+0000","last_became_peered":"2024-03-06T07:23:25.271564+0000","last_unstale":"2024-03-06T07:23:26.383998+0000","last_undegraded":"2024-03-06T07:23:26.383998+0000","last_fullsized":"2024-03-06T07:23:26.383998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.223485+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.223485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:25:57.792187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.244195+0000","last_change":"2024-03-06T07:23:24.244996+0000","last_active":"2024-03-06T07:23:26.244195+0000","last_peered":"2024-03-06T07:23:26.244195+0000","last_clean":"2024-03-06T07:23:26.244195+0000","last_became_active":"2024-03-06T07:23:24.244721+0000","last_became_peered":"2024-03-06T07:23:24.244721+0000","last_unstale":"2024-03-06T07:23:26.244195+0000","last_undegraded":"2024-03-06T07:23:26.244195+0000","last_fullsized":"2024-03-06T07:23:26.244195+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.219597+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.219597+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:47:44.289556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.383374+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:23:26.383374+0000","last_peered":"2024-03-06T07:23:26.383374+0000","last_clean":"2024-03-06T07:23:26.383374+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:23:26.383374+0000","last_undegraded":"2024-03-06T07:23:26.383374+0000","last_fullsized":"2024-03-06T07:23:26.383374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1253376,"data_stored":1160089,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1200128,"data_stored":571777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":569561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1187840,"data_stored":569653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":570143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":569595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":569595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":1161631,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:33.017 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:23:33.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:23:33.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.159+0000 7f271ff5e640 1 Processor -- start 2024-03-06T07:23:33.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271ff5e640 1 -- start start 2024-03-06T07:23:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271ff5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180b0070 0x7f2718155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271ff5e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f27180c5380 0x7f27180c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271ff5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27180b0450 con 0x7f271806ca70 2024-03-06T07:23:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271ff5e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271806c020 con 0x7f27180c5380 2024-03-06T07:23:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271ff5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27180c77f0 con 0x7f27180b0070 2024-03-06T07:23:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271d4f9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f271806ca70 0x7f27180c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34866/0 (socket says 172.21.15.26:34866) 2024-03-06T07:23:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.160+0000 7f271d4f9640 1 -- 172.21.15.26:0/3971931661 learned_addr learned my addr 172.21.15.26:0/3971931661 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271e4fb640 1 --2- 172.21.15.26:0/3971931661 >> v2:172.21.15.117:3300/0 conn(0x7f27180c5380 0x7f27180c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271dcfa640 1 --2- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180b0070 0x7f2718155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271ccf8640 1 -- 172.21.15.26:0/3971931661 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208114041 0 0) 0x7f2708001110 con 0x7f271806ca70 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271ccf8640 1 -- 172.21.15.26:0/3971931661 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2704003520 con 0x7f271806ca70 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271e4fb640 1 -- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180b0070 msgr2=0x7f2718155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271e4fb640 1 --2- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180b0070 0x7f2718155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271e4fb640 1 -- 172.21.15.26:0/3971931661 >> v1:172.21.15.26:6789/0 conn(0x7f271806ca70 legacy=0x7f27180c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.161+0000 7f271e4fb640 1 -- 172.21.15.26:0/3971931661 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27180c7ad0 con 0x7f27180c5380 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271e4fb640 1 --2- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27180c5380 0x7f27180c57d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f27100097a0 tx=0x7f271002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252e70f30339a8ae server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ccf8640 1 -- 172.21.15.26:0/3971931661 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100315a0 con 0x7f27180c5380 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ccf8640 1 -- 172.21.15.26:0/3971931661 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2710037070 con 0x7f27180c5380 2024-03-06T07:23:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ccf8640 1 -- 172.21.15.26:0/3971931661 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710031960 con 0x7f27180c5380 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 -- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27180c5380 msgr2=0x7f27180c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 --2- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27180c5380 0x7f27180c57d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f27100097a0 tx=0x7f271002b840 comp rx=0 tx=0).stop 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 -- 172.21.15.26:0/3971931661 shutdown_connections 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 --2- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27180c5380 0x7f27180c57d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 --2- 172.21.15.26:0/3971931661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180b0070 0x7f2718155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 -- 172.21.15.26:0/3971931661 >> 172.21.15.26:0/3971931661 conn(0x7f27180ac590 msgr2=0x7f27180ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 -- 172.21.15.26:0/3971931661 shutdown_connections 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.162+0000 7f271ff5e640 1 -- 172.21.15.26:0/3971931661 wait complete. 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.163+0000 7f271ff5e640 1 Processor -- start 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.163+0000 7f271ff5e640 1 -- start start 2024-03-06T07:23:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271ff5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271806ca70 0x7f27181de810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271ff5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27180b0070 0x7f27181df770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271ff5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180c5380 0x7f271814bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271ff5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271806cef0 con 0x7f27180b0070 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271ff5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271806c550 con 0x7f271806ca70 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271ff5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271806da80 con 0x7f27180c5380 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271d4f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27180b0070 0x7f27181df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271d4f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27180b0070 0x7f27181df770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49512/0 (socket says 172.21.15.26:49512) 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271d4f9640 1 -- 172.21.15.26:0/2389029731 learned_addr learned my addr 172.21.15.26:0/2389029731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271dcfa640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271806ca70 0x7f27181de810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.164+0000 7f271e4fb640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180c5380 0x7f271814bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271d4f9640 1 -- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180c5380 msgr2=0x7f271814bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271d4f9640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180c5380 0x7f271814bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271d4f9640 1 -- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271806ca70 msgr2=0x7f27181de810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271d4f9640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271806ca70 0x7f27181de810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271d4f9640 1 -- 172.21.15.26:0/2389029731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27181e15a0 con 0x7f27180b0070 2024-03-06T07:23:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271d4f9640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27180b0070 0x7f27181df770 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f270800c760 tx=0x7f270800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f270e7fc640 1 -- 172.21.15.26:0/2389029731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708017070 con 0x7f27180b0070 2024-03-06T07:23:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f270e7fc640 1 -- 172.21.15.26:0/2389029731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2708005ad0 con 0x7f27180b0070 2024-03-06T07:23:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27181deed0 con 0x7f27180b0070 2024-03-06T07:23:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f270e7fc640 1 -- 172.21.15.26:0/2389029731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708007e20 con 0x7f27180b0070 2024-03-06T07:23:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.165+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27181dfcb0 con 0x7f27180b0070 2024-03-06T07:23:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.166+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271804f330 con 0x7f27180b0070 2024-03-06T07:23:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.166+0000 7f270e7fc640 1 -- 172.21.15.26:0/2389029731 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f270801a070 con 0x7f27180b0070 2024-03-06T07:23:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.167+0000 7f270e7fc640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26fc06b500 0x7f26fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.167+0000 7f270e7fc640 1 -- 172.21.15.26:0/2389029731 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27080895b0 con 0x7f27180b0070 2024-03-06T07:23:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.167+0000 7f271dcfa640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26fc06b500 0x7f26fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.167+0000 7f271dcfa640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26fc06b500 0x7f26fc06d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f27180c1660 tx=0x7f271400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:33.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.171+0000 7f270e7fc640 1 -- 172.21.15.26:0/2389029731 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f27180b0450 con 0x7f27180b0070 2024-03-06T07:23:33.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.427+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f271806d860 con 0x7f27180b0070 2024-03-06T07:23:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.429+0000 7f270e7fc640 1 -- 172.21.15.26:0/2389029731 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2708057ce0 con 0x7f27180b0070 2024-03-06T07:23:33.431 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:23:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:23:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:23:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:23:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:23:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:23:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26fc06b500 msgr2=0x7f26fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26fc06b500 0x7f26fc06d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f27180c1660 tx=0x7f271400b040 comp rx=0 tx=0).stop 2024-03-06T07:23:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27180b0070 msgr2=0x7f27181df770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27180b0070 0x7f27181df770 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f270800c760 tx=0x7f270800cc20 comp rx=0 tx=0).stop 2024-03-06T07:23:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 shutdown_connections 2024-03-06T07:23:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f26fc06b500 0x7f26fc06d9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27180c5380 0x7f271814bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27180b0070 0x7f27181df770 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 --2- 172.21.15.26:0/2389029731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271806ca70 0x7f27181de810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 >> 172.21.15.26:0/2389029731 conn(0x7f27180ac590 msgr2=0x7f2718151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 shutdown_connections 2024-03-06T07:23:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.431+0000 7f271ff5e640 1 -- 172.21.15.26:0/2389029731 wait complete. 2024-03-06T07:23:33.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:33.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.543+0000 7f95d6af4640 1 Processor -- start 2024-03-06T07:23:33.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.543+0000 7f95d6af4640 1 -- start start 2024-03-06T07:23:33.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d6af4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00c7350 0x7f95d00c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d6af4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d006c150 con 0x7f95d00c8840 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d6af4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d014f630 con 0x7f95d014f250 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d6af4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d006c480 con 0x7f95d00c7350 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95cffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95d00c8840 0x7f95d006c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34872/0 (socket says 172.21.15.26:34872) 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95cffff640 1 -- 172.21.15.26:0/2713200328 learned_addr learned my addr 172.21.15.26:0/2713200328 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d5091640 1 --2- 172.21.15.26:0/2713200328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00c7350 0x7f95d00c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95cf7fe640 1 -- 172.21.15.26:0/2713200328 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766531861 0 0) 0x7f95d006c150 con 0x7f95d00c8840 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95cf7fe640 1 -- 172.21.15.26:0/2713200328 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c0003520 con 0x7f95d00c8840 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95cf7fe640 1 -- 172.21.15.26:0/2713200328 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019476456 0 0) 0x7f95d014f630 con 0x7f95d014f250 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95cf7fe640 1 -- 172.21.15.26:0/2713200328 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d006c150 con 0x7f95d014f250 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d5091640 1 -- 172.21.15.26:0/2713200328 >> v1:172.21.15.117:6789/0 conn(0x7f95d014f250 legacy=0x7f95d0155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d5091640 1 -- 172.21.15.26:0/2713200328 >> v1:172.21.15.26:6789/0 conn(0x7f95d00c8840 legacy=0x7f95d006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.544+0000 7f95d5091640 1 -- 172.21.15.26:0/2713200328 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00c5660 con 0x7f95d00c7350 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.545+0000 7f95d5091640 1 --2- 172.21.15.26:0/2713200328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00c7350 0x7f95d00c77e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95d006c600 tx=0x7f95c802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc54c6a1d22a25cb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:33.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.545+0000 7f95cf7fe640 1 -- 172.21.15.26:0/2713200328 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8039070 con 0x7f95d00c7350 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.545+0000 7f95cf7fe640 1 -- 172.21.15.26:0/2713200328 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c802cbb0 con 0x7f95d00c7350 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.545+0000 7f95cf7fe640 1 -- 172.21.15.26:0/2713200328 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8034440 con 0x7f95d00c7350 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.546+0000 7f95d6af4640 1 -- 172.21.15.26:0/2713200328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00c7350 msgr2=0x7f95d00c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.546+0000 7f95d6af4640 1 --2- 172.21.15.26:0/2713200328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00c7350 0x7f95d00c77e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95d006c600 tx=0x7f95c802be00 comp rx=0 tx=0).stop 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.546+0000 7f95d6af4640 1 -- 172.21.15.26:0/2713200328 shutdown_connections 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.546+0000 7f95d6af4640 1 --2- 172.21.15.26:0/2713200328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00c7350 0x7f95d00c77e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.546+0000 7f95d6af4640 1 -- 172.21.15.26:0/2713200328 >> 172.21.15.26:0/2713200328 conn(0x7f95d00ac4d0 msgr2=0x7f95d00ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.546+0000 7f95d6af4640 1 -- 172.21.15.26:0/2713200328 shutdown_connections 2024-03-06T07:23:33.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.546+0000 7f95d6af4640 1 -- 172.21.15.26:0/2713200328 wait complete. 2024-03-06T07:23:33.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.547+0000 7f95d6af4640 1 Processor -- start 2024-03-06T07:23:33.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.547+0000 7f95d6af4640 1 -- start start 2024-03-06T07:23:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d00c7350 0x7f95d00c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d00c8840 0x7f95d0158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d014f250 0x7f95d0157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d006daf0 con 0x7f95d00c7350 2024-03-06T07:23:33.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c8cc0 con 0x7f95d00c8840 2024-03-06T07:23:33.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c5c20 con 0x7f95d014f250 2024-03-06T07:23:33.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d00c7350 0x7f95d00c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d00c7350 0x7f95d00c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49520/0 (socket says 172.21.15.26:49520) 2024-03-06T07:23:33.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 -- 172.21.15.26:0/3664576251 learned_addr learned my addr 172.21.15.26:0/3664576251 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:33.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95cffff640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d00c8840 0x7f95d0158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d5091640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d014f250 0x7f95d0157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 -- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d014f250 msgr2=0x7f95d0157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d014f250 0x7f95d0157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 -- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d00c8840 msgr2=0x7f95d0158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d00c8840 0x7f95d0158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 -- 172.21.15.26:0/3664576251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d01ee680 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d4890640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d00c7350 0x7f95d00c4740 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c430 tx=0x7f95c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95cd7fa640 1 -- 172.21.15.26:0/3664576251 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4017070 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95cd7fa640 1 -- 172.21.15.26:0/3664576251 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c40052f0 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d0156590 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95cd7fa640 1 -- 172.21.15.26:0/3664576251 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4003830 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.548+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d0157e30 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.549+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d0156380 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.550+0000 7f95cd7fa640 1 -- 172.21.15.26:0/3664576251 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95c401a070 con 0x7f95d00c7350 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.550+0000 7f95cd7fa640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95b406b4d0 0x7f95b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:33.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.550+0000 7f95cd7fa640 1 -- 172.21.15.26:0/3664576251 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95c4089610 con 0x7f95d00c7350 2024-03-06T07:23:33.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.550+0000 7f95cffff640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95b406b4d0 0x7f95b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:33.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.551+0000 7f95cffff640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95b406b4d0 0x7f95b406d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f95d01554a0 tx=0x7f95bc007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:33.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.555+0000 7f95cd7fa640 1 -- 172.21.15.26:0/3664576251 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95c4057d40 con 0x7f95d00c7350 2024-03-06T07:23:33.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.772+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95d004f330 con 0x7f95d00c7350 2024-03-06T07:23:33.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.773+0000 7f95cd7fa640 1 -- 172.21.15.26:0/3664576251 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f95c405bde0 con 0x7f95d00c7350 2024-03-06T07:23:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:33.776 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:26.618139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1940022472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1940022472},{"type":"v1","addr":"172.21.15.117:6837","nonce":1940022472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6966,"qdb_cluster":[6966]},"id":7}]} 2024-03-06T07:23:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 77 2024-03-06T07:23:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95b406b4d0 msgr2=0x7f95b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95b406b4d0 0x7f95b406d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f95d01554a0 tx=0x7f95bc007d30 comp rx=0 tx=0).stop 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d00c7350 msgr2=0x7f95d00c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d00c7350 0x7f95d00c4740 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c430 tx=0x7f95c400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 shutdown_connections 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95b406b4d0 0x7f95b406d990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d014f250 0x7f95d0157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d00c8840 0x7f95d0158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 --2- 172.21.15.26:0/3664576251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d00c7350 0x7f95d00c4740 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 >> 172.21.15.26:0/3664576251 conn(0x7f95d00ac4d0 msgr2=0x7f95d014dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 shutdown_connections 2024-03-06T07:23:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:33.775+0000 7f95d6af4640 1 -- 172.21.15.26:0/3664576251 wait complete. 2024-03-06T07:23:34.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:24.599873+0000', 'modified': '2024-03-06T07:23:26.618139+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6966}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6966': {'gid': 6966, 'name': 'b', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/1940022472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 1940022472}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1940022472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6966, 'qdb_cluster': [6966]} 2024-03-06T07:23:34.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:23:34.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:23:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.296+0000 7fa5cfbf7640 1 Processor -- start 2024-03-06T07:23:34.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.296+0000 7fa5cfbf7640 1 -- start start 2024-03-06T07:23:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cfbf7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa5c8155120 0x7fa5c80c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cfbf7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c8159590 con 0x7fa5c80bfe90 2024-03-06T07:23:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cfbf7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c80c59d0 con 0x7fa5c81522d0 2024-03-06T07:23:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cfbf7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c8158710 con 0x7fa5c8155120 2024-03-06T07:23:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cd993640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa5c80bfe90 0x7fa5c8158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34876/0 (socket says 172.21.15.26:34876) 2024-03-06T07:23:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cd993640 1 -- 172.21.15.26:0/3750768164 learned_addr learned my addr 172.21.15.26:0/3750768164 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:34.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3750768164 >> v2:172.21.15.117:3300/0 conn(0x7fa5c8155120 0x7fa5c80c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cc991640 1 -- 172.21.15.26:0/3750768164 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831894008 0 0) 0x7fa5c8159590 con 0x7fa5c80bfe90 2024-03-06T07:23:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cc991640 1 -- 172.21.15.26:0/3750768164 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b4003520 con 0x7fa5c80bfe90 2024-03-06T07:23:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cd192640 1 -- 172.21.15.26:0/3750768164 >> v1:172.21.15.117:6789/0 conn(0x7fa5c81522d0 legacy=0x7fa5c815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cc991640 1 -- 172.21.15.26:0/3750768164 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598549330 0 0) 0x7fa5c80c59d0 con 0x7fa5c81522d0 2024-03-06T07:23:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cd192640 1 -- 172.21.15.26:0/3750768164 >> v1:172.21.15.26:6789/0 conn(0x7fa5c80bfe90 legacy=0x7fa5c8158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.297+0000 7fa5cd192640 1 -- 172.21.15.26:0/3750768164 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80c5cb0 con 0x7fa5c8155120 2024-03-06T07:23:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.298+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3750768164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c80c6d70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8158890 tx=0x7fa5b802b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7b380e0f4d92da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.298+0000 7fa5cc991640 1 -- 172.21.15.26:0/3750768164 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8035020 con 0x7fa5c8155120 2024-03-06T07:23:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.298+0000 7fa5cc991640 1 -- 172.21.15.26:0/3750768164 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b8031800 con 0x7fa5c8155120 2024-03-06T07:23:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.298+0000 7fa5cc991640 1 -- 172.21.15.26:0/3750768164 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8031bb0 con 0x7fa5c8155120 2024-03-06T07:23:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.299+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/3750768164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 msgr2=0x7fa5c80c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.299+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/3750768164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c80c6d70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8158890 tx=0x7fa5b802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.299+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/3750768164 shutdown_connections 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.299+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/3750768164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c80c6d70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.299+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/3750768164 >> 172.21.15.26:0/3750768164 conn(0x7fa5c80acaa0 msgr2=0x7fa5c80aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.299+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/3750768164 shutdown_connections 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.299+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/3750768164 wait complete. 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.300+0000 7fa5cfbf7640 1 Processor -- start 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.300+0000 7fa5cfbf7640 1 -- start start 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.300+0000 7fa5cfbf7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5c80bfe90 0x7fa5c81ee620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cfbf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5c81522d0 0x7fa5c814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cfbf7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c8150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cfbf7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80ca700 con 0x7fa5c81522d0 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cfbf7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8158960 con 0x7fa5c8155120 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cfbf7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80ca880 con 0x7fa5c80bfe90 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5c81522d0 0x7fa5c814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5c81522d0 0x7fa5c814fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49534/0 (socket says 172.21.15.26:49534) 2024-03-06T07:23:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 -- 172.21.15.26:0/2036690828 learned_addr learned my addr 172.21.15.26:0/2036690828 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd993640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5c80bfe90 0x7fa5c81ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5ce194640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c8150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 -- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5c80bfe90 msgr2=0x7fa5c81ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5c80bfe90 0x7fa5c81ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 -- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 msgr2=0x7fa5c8150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c8150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 -- 172.21.15.26:0/2036690828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80c1b20 con 0x7fa5c81522d0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd993640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5c80bfe90 0x7fa5c81ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5ce194640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c8150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cd192640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5c81522d0 0x7fa5c814fef0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa5b802b8e0 tx=0x7fa5b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5be7fc640 1 -- 172.21.15.26:0/2036690828 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8040070 con 0x7fa5c81522d0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5be7fc640 1 -- 172.21.15.26:0/2036690828 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b8039cf0 con 0x7fa5c81522d0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c80af6a0 con 0x7fa5c81522d0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5be7fc640 1 -- 172.21.15.26:0/2036690828 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8035040 con 0x7fa5c81522d0 2024-03-06T07:23:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.301+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c80afc60 con 0x7fa5c81522d0 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.303+0000 7fa5be7fc640 1 -- 172.21.15.26:0/2036690828 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5b8038020 con 0x7fa5c81522d0 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.303+0000 7fa5be7fc640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5ac06b500 0x7fa5ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.303+0000 7fa5be7fc640 1 -- 172.21.15.26:0/2036690828 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5b80abec0 con 0x7fa5c81522d0 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.303+0000 7fa5cd993640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5ac06b500 0x7fa5ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.304+0000 7fa5cd993640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5ac06b500 0x7fa5ac06d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007fd0 tx=0x7fa5c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.304+0000 7fa5d27dd740 1 Processor -- start 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.304+0000 7fa5d27dd740 1 -- start start 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fb1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb4330 0x55aa05fb1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb24b0 0x55aa05fae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa05f186b0 con 0x55aa05fb24b0 2024-03-06T07:23:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa05ec1a60 con 0x55aa05fb4330 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa05f14750 con 0x55aa05fb3f50 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb24b0 0x55aa05fae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5cd192640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb4330 0x55aa05fb1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5cd993640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fb1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb24b0 0x55aa05fae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49544/0 (socket says 172.21.15.26:49544) 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5cd192640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb4330 0x55aa05fb1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38632/0 (socket says 172.21.15.26:38632) 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 -- 172.21.15.26:0/1614014784 learned_addr learned my addr 172.21.15.26:0/1614014784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 -- 172.21.15.26:0/1614014784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 msgr2=0x55aa05fb1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fb1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 -- 172.21.15.26:0/1614014784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb4330 msgr2=0x55aa05fb1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb4330 0x55aa05fb1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 -- 172.21.15.26:0/1614014784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa05edff30 con 0x55aa05fb24b0 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5cd192640 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb4330 0x55aa05fb1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.305+0000 7fa5ce194640 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb24b0 0x55aa05fae680 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0001d50 tx=0x7fa5c000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a45bc67479d119 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5ab7fe640 1 -- 172.21.15.26:0/1614014784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0019070 con 0x55aa05fb24b0 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5ab7fe640 1 -- 172.21.15.26:0/1614014784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000eba0 con 0x55aa05fb24b0 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5ab7fe640 1 -- 172.21.15.26:0/1614014784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0003970 con 0x55aa05fb24b0 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 -- 172.21.15.26:0/1614014784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb24b0 msgr2=0x55aa05fae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb24b0 0x55aa05fae680 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0001d50 tx=0x7fa5c000e160 comp rx=0 tx=0).stop 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 -- 172.21.15.26:0/1614014784 shutdown_connections 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb24b0 0x55aa05fae680 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb4330 0x55aa05fb1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/1614014784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fb1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 -- 172.21.15.26:0/1614014784 >> 172.21.15.26:0/1614014784 conn(0x55aa05fb07d0 msgr2=0x55aa05fd53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 -- 172.21.15.26:0/1614014784 shutdown_connections 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.306+0000 7fa5d27dd740 1 -- 172.21.15.26:0/1614014784 wait complete. 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.307+0000 7fa5d27dd740 1 -- start start 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.307+0000 7fa5d27dd740 1 Processor -- start 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fd47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb4330 0x55aa06003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fd0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa05fd6230 con 0x55aa05fb4330 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa05ed4ea0 con 0x55aa06003fc0 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa05fd5c70 con 0x55aa05fb3f50 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 10 client.0 _ll_register_callbacks cb 0x55aa05ed4870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb4330 0x55aa06003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb4330 0x55aa06003a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49552/0 (socket says 172.21.15.26:49552) 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 -- 172.21.15.26:0/3489770437 learned_addr learned my addr 172.21.15.26:0/3489770437 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd993640 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fd47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5ce194640 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fd0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 -- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 msgr2=0x55aa05fd47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fd47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 -- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 msgr2=0x55aa05fd0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fd0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 -- 172.21.15.26:0/3489770437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa05ed5270 con 0x55aa05fb4330 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb4330 0x55aa06003a80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80097a0 tx=0x7fa5b8031800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b296ccc6ca665b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3489770437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b807e360 con 0x55aa05fb4330 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3489770437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5b8079860 con 0x55aa05fb4330 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.308+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5c000eba0 con 0x55aa05fb4330 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3489770437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b807a410 con 0x55aa05fb4330 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 10 client.7128 fetch_fsmap learned FSMap version 77 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55aa05c2d2d0 con 0x55aa05fb4330 2024-03-06T07:23:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3489770437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000eba0 con 0x55aa05fb4330 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3489770437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 77) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa5b80a3b20 con 0x55aa05fb4330 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 10 client.7128 fetch_fsmap finished waiting for FSMap version 77 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 10 client.7128 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 20 client.7128 populate_metadata read hostname 'smithi026' 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fd6da0 0x55aa0600f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 4 client.7128 mds_command: new command op to 6966 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 --> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa05f25bf0 con 0x55aa05fd6da0 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5cd993640 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fd6da0 0x55aa0600f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5a8ff9640 10 client.7128 ms_handle_connect on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.309+0000 7fa5cd993640 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fd6da0 0x55aa0600f2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28605682f8592f7 server_cookie=c0d9028dcb0c7bb4 in_seq=0 out_seq=0 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3489770437 <== mds.0 v2:172.21.15.117:6835/1940022472 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aa05f25bf0 con 0x55aa05fd6da0 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5a8ff9640 10 client.7128 handle_command_reply: tid=0 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 client.7128 shutdown 2024-03-06T07:23:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb4330 msgr2=0x55aa06003a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb4330 0x55aa06003a80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80097a0 tx=0x7fa5b8031800 comp rx=0 tx=0).stop 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 shutdown_connections 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fd6da0 0x55aa0600f2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fd0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb4330 0x55aa06003a80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5a8ff9640 0 client.7128 ms_handle_reset on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3489770437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb3f50 0x55aa05fd47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 >> 172.21.15.26:0/3489770437 conn(0x55aa05fb24b0 msgr2=0x55aa05fb2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 shutdown_connections 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3489770437 wait complete. 2024-03-06T07:23:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.310+0000 7fa5d27dd740 20 client.7128 trim_cache size 0 max 16384 2024-03-06T07:23:34.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.327+0000 7fa5d27dd740 1 Processor -- start 2024-03-06T07:23:34.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.327+0000 7fa5d27dd740 1 -- start start 2024-03-06T07:23:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb3f50 0x55aa05fed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa05ff2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa06003fc0 0x55aa060142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c81f6940 con 0x55aa06003fc0 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c81ebb60 con 0x55aa05fb3f50 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80aeb80 con 0x55aa05fb4330 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa06003fc0 0x55aa060142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa06003fc0 0x55aa060142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49556/0 (socket says 172.21.15.26:49556) 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 -- 172.21.15.26:0/2845086888 learned_addr learned my addr 172.21.15.26:0/2845086888 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5cd993640 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb3f50 0x55aa05fed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5cd192640 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa05ff2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 -- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 msgr2=0x55aa05ff2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa05ff2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 -- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb3f50 msgr2=0x55aa05fed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb3f50 0x55aa05fed710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.328+0000 7fa5ce194640 1 -- 172.21.15.26:0/2845086888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa05cf76b0 con 0x55aa06003fc0 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5ce194640 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa06003fc0 0x55aa060142d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0007df0 tx=0x7fa5c000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff4e3ba2d52b641 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5aaffd640 1 -- 172.21.15.26:0/2845086888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0024070 con 0x55aa06003fc0 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5aaffd640 1 -- 172.21.15.26:0/2845086888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000eba0 con 0x55aa06003fc0 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5aaffd640 1 -- 172.21.15.26:0/2845086888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0019040 con 0x55aa06003fc0 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 -- 172.21.15.26:0/2845086888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa06003fc0 msgr2=0x55aa060142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa06003fc0 0x55aa060142d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0007df0 tx=0x7fa5c000a000 comp rx=0 tx=0).stop 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 -- 172.21.15.26:0/2845086888 shutdown_connections 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa06003fc0 0x55aa060142d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa05ff2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/2845086888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa05fb3f50 0x55aa05fed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 -- 172.21.15.26:0/2845086888 >> 172.21.15.26:0/2845086888 conn(0x55aa05fb24b0 msgr2=0x55aa05fd45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 -- 172.21.15.26:0/2845086888 shutdown_connections 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.329+0000 7fa5d27dd740 1 -- 172.21.15.26:0/2845086888 wait complete. 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.330+0000 7fa5d27dd740 1 -- start start 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.330+0000 7fa5d27dd740 1 Processor -- start 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb3f50 0x55aa05fb18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa0605fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa05fd9a30 con 0x55aa05fb3f50 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa05fd50d0 con 0x55aa06003fc0 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa05fd1670 con 0x55aa05fb4330 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 10 client.0 _ll_register_callbacks cb 0x7fa5b8039ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb3f50 0x55aa05fb18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb3f50 0x55aa05fb18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49568/0 (socket says 172.21.15.26:49568) 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 -- 172.21.15.26:0/3463548704 learned_addr learned my addr 172.21.15.26:0/3463548704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5ce194640 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa0605fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 -- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 msgr2=0x55aa0605fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa0605fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 -- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 msgr2=0x55aa05fecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 -- 172.21.15.26:0/3463548704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa05ffe0c0 con 0x55aa05fb3f50 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5cd993640 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb3f50 0x55aa05fb18b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x55aa05fd5250 tx=0x7fa5c4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c1d5c4786e1298 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3463548704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4028b50 con 0x55aa05fb3f50 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3463548704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5c401e040 con 0x55aa05fb3f50 2024-03-06T07:23:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.331+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5c000eba0 con 0x55aa05fb3f50 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3463548704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5c4036050 con 0x55aa05fb3f50 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 10 client.7134 fetch_fsmap learned FSMap version 77 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55aa06063920 con 0x55aa05fb3f50 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3463548704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000eba0 con 0x55aa05fb3f50 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3463548704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 77) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa5c4039020 con 0x55aa05fb3f50 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 10 client.7134 fetch_fsmap finished waiting for FSMap version 77 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 10 client.7134 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 20 client.7134 populate_metadata read hostname 'smithi026' 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fae690 0x55aa05faea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 4 client.7134 mds_command: new command op to 6966 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 --> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa05f1ff40 con 0x55aa05fae690 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fae690 0x55aa05faea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5a8ff9640 10 client.7134 ms_handle_connect on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.332+0000 7fa5cd192640 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fae690 0x55aa05faea70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fd2682c7e2e278 server_cookie=9e1e98c4dd0765ae in_seq=0 out_seq=0 2024-03-06T07:23:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5a8ff9640 1 -- 172.21.15.26:0/3463548704 <== mds.0 v2:172.21.15.117:6835/1940022472 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55aa05f1ff40 con 0x55aa05fae690 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5a8ff9640 10 client.7134 handle_command_reply: tid=0 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 client.7134 shutdown 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb3f50 msgr2=0x55aa05fb18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb3f50 0x55aa05fb18b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x55aa05fd5250 tx=0x7fa5c4011000 comp rx=0 tx=0).stop 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 shutdown_connections 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55aa05fae690 0x55aa05faea70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa06003fc0 0x55aa05fecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5a8ff9640 0 client.7134 ms_handle_reset on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aa05fb4330 0x55aa0605fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 --2- 172.21.15.26:0/3463548704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa05fb3f50 0x55aa05fb18b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 >> 172.21.15.26:0/3463548704 conn(0x55aa05fb24b0 msgr2=0x55aa05fed610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 shutdown_connections 2024-03-06T07:23:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 1 -- 172.21.15.26:0/3463548704 wait complete. 2024-03-06T07:23:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5d27dd740 20 client.7134 trim_cache size 0 max 16384 2024-03-06T07:23:34.347 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:23:34.347 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:23:34.347 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:23:34.347 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6966, 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 9.0816403549999993, 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 77, 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 95, 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 95, 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 14.16592395 2024-03-06T07:23:34.348 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5ac06b500 msgr2=0x7fa5ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5ac06b500 0x7fa5ac06d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007fd0 tx=0x7fa5c4009040 comp rx=0 tx=0).stop 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5c81522d0 msgr2=0x7fa5c814fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5c81522d0 0x7fa5c814fef0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa5b802b8e0 tx=0x7fa5b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 shutdown_connections 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5ac06b500 0x7fa5ac06d9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c8155120 0x7fa5c8150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5c81522d0 0x7fa5c814fef0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 --2- 172.21.15.26:0/2036690828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5c80bfe90 0x7fa5c81ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.333+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 >> 172.21.15.26:0/2036690828 conn(0x7fa5c80acaa0 msgr2=0x7fa5c8157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.334+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 shutdown_connections 2024-03-06T07:23:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.334+0000 7fa5cfbf7640 1 -- 172.21.15.26:0/2036690828 wait complete. 2024-03-06T07:23:34.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.450+0000 7f1db898c640 1 Processor -- start 2024-03-06T07:23:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.450+0000 7f1db898c640 1 -- start start 2024-03-06T07:23:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.450+0000 7f1db898c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db40c6780 0x7f1db4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db898c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db4072630 con 0x7f1db41592d0 2024-03-06T07:23:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db898c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db4072910 con 0x7f1db406da40 2024-03-06T07:23:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db898c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db406de20 con 0x7f1db40c6780 2024-03-06T07:23:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1db41592d0 0x7f1db40c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34878/0 (socket says 172.21.15.26:34878) 2024-03-06T07:23:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db2d9d640 1 -- 172.21.15.26:0/1501606676 learned_addr learned my addr 172.21.15.26:0/1501606676 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1dabfff640 1 --2- 172.21.15.26:0/1501606676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db40c6780 0x7f1db4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db1d9b640 1 -- 172.21.15.26:0/1501606676 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212405766 0 0) 0x7f1db4072630 con 0x7f1db41592d0 2024-03-06T07:23:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db1d9b640 1 -- 172.21.15.26:0/1501606676 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da0003520 con 0x7f1db41592d0 2024-03-06T07:23:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db1d9b640 1 -- 172.21.15.26:0/1501606676 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750756846 0 0) 0x7f1db4072910 con 0x7f1db406da40 2024-03-06T07:23:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1db1d9b640 1 -- 172.21.15.26:0/1501606676 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1db4072630 con 0x7f1db406da40 2024-03-06T07:23:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1dabfff640 1 -- 172.21.15.26:0/1501606676 >> v1:172.21.15.117:6789/0 conn(0x7f1db406da40 legacy=0x7f1db4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1dabfff640 1 -- 172.21.15.26:0/1501606676 >> v1:172.21.15.26:6789/0 conn(0x7f1db41592d0 legacy=0x7f1db40c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.451+0000 7f1dabfff640 1 -- 172.21.15.26:0/1501606676 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4072bf0 con 0x7f1db40c6780 2024-03-06T07:23:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.452+0000 7f1dabfff640 1 --2- 172.21.15.26:0/1501606676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db40c6780 0x7f1db4158d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0097d0 tx=0x7f1d9c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483cb6e473251c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.452+0000 7f1db1d9b640 1 -- 172.21.15.26:0/1501606676 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c035020 con 0x7f1db40c6780 2024-03-06T07:23:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.452+0000 7f1db1d9b640 1 -- 172.21.15.26:0/1501606676 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c031800 con 0x7f1db40c6780 2024-03-06T07:23:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.452+0000 7f1db1d9b640 1 -- 172.21.15.26:0/1501606676 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c031bb0 con 0x7f1db40c6780 2024-03-06T07:23:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.453+0000 7f1db898c640 1 -- 172.21.15.26:0/1501606676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db40c6780 msgr2=0x7f1db4158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.453+0000 7f1db898c640 1 --2- 172.21.15.26:0/1501606676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db40c6780 0x7f1db4158d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0097d0 tx=0x7f1d9c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.453+0000 7f1db898c640 1 -- 172.21.15.26:0/1501606676 shutdown_connections 2024-03-06T07:23:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.453+0000 7f1db898c640 1 --2- 172.21.15.26:0/1501606676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db40c6780 0x7f1db4158d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.453+0000 7f1db898c640 1 -- 172.21.15.26:0/1501606676 >> 172.21.15.26:0/1501606676 conn(0x7f1db40aa630 msgr2=0x7f1db40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.453+0000 7f1db898c640 1 -- 172.21.15.26:0/1501606676 shutdown_connections 2024-03-06T07:23:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.453+0000 7f1db898c640 1 -- 172.21.15.26:0/1501606676 wait complete. 2024-03-06T07:23:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.454+0000 7f1db898c640 1 Processor -- start 2024-03-06T07:23:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.454+0000 7f1db898c640 1 -- start start 2024-03-06T07:23:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db898c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1db406da40 0x7f1db40ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db898c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40c6780 0x7f1db41eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db898c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db41592d0 0x7f1db40a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db898c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40c3b10 con 0x7f1db40c6780 2024-03-06T07:23:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db898c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40c3990 con 0x7f1db406da40 2024-03-06T07:23:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db898c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40c7380 con 0x7f1db41592d0 2024-03-06T07:23:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40c6780 0x7f1db41eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40c6780 0x7f1db41eb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49584/0 (socket says 172.21.15.26:49584) 2024-03-06T07:23:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 -- 172.21.15.26:0/1017021523 learned_addr learned my addr 172.21.15.26:0/1017021523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db259c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1db406da40 0x7f1db40ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db2d9d640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db41592d0 0x7f1db40a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 -- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db41592d0 msgr2=0x7f1db40a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db41592d0 0x7f1db40a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 -- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1db406da40 msgr2=0x7f1db40ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1db406da40 0x7f1db40ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 -- 172.21.15.26:0/1017021523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db41ea9f0 con 0x7f1db40c6780 2024-03-06T07:23:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db259c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1db406da40 0x7f1db40ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1dabfff640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40c6780 0x7f1db41eb4b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c00 tx=0x7f1d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1daaffd640 1 -- 172.21.15.26:0/1017021523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c041070 con 0x7f1db40c6780 2024-03-06T07:23:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1daaffd640 1 -- 172.21.15.26:0/1017021523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c031e30 con 0x7f1db40c6780 2024-03-06T07:23:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.455+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db406fff0 con 0x7f1db40c6780 2024-03-06T07:23:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.456+0000 7f1daaffd640 1 -- 172.21.15.26:0/1017021523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c035040 con 0x7f1db40c6780 2024-03-06T07:23:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.456+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db41eacb0 con 0x7f1db40c6780 2024-03-06T07:23:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.457+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db404f330 con 0x7f1db40c6780 2024-03-06T07:23:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.457+0000 7f1daaffd640 1 -- 172.21.15.26:0/1017021523 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d9c04c060 con 0x7f1db40c6780 2024-03-06T07:23:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.457+0000 7f1daaffd640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d9406b4d0 0x7f1d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.458+0000 7f1daaffd640 1 -- 172.21.15.26:0/1017021523 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d9c0abbf0 con 0x7f1db40c6780 2024-03-06T07:23:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.458+0000 7f1db259c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d9406b4d0 0x7f1d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.458+0000 7f1db259c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d9406b4d0 0x7f1d9406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007fd0 tx=0x7f1da4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.462+0000 7f1daaffd640 1 -- 172.21.15.26:0/1017021523 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1d9c07a320 con 0x7f1db40c6780 2024-03-06T07:23:34.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.679+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1db4073240 con 0x7f1db40c6780 2024-03-06T07:23:34.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.679+0000 7f1daaffd640 1 -- 172.21.15.26:0/1017021523 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f1d9c07e3c0 con 0x7f1db40c6780 2024-03-06T07:23:34.683 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:34.683 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:26.618139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1940022472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1940022472},{"type":"v1","addr":"172.21.15.117:6837","nonce":1940022472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6966,"qdb_cluster":[6966]},"id":7}]} 2024-03-06T07:23:34.685 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 77 2024-03-06T07:23:34.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d9406b4d0 msgr2=0x7f1d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d9406b4d0 0x7f1d9406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007fd0 tx=0x7f1da4009040 comp rx=0 tx=0).stop 2024-03-06T07:23:34.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40c6780 msgr2=0x7f1db41eb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40c6780 0x7f1db41eb4b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c00 tx=0x7f1d9c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 shutdown_connections 2024-03-06T07:23:34.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d9406b4d0 0x7f1d9406d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1db41592d0 0x7f1db40a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40c6780 0x7f1db41eb4b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 --2- 172.21.15.26:0/1017021523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1db406da40 0x7f1db40ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 >> 172.21.15.26:0/1017021523 conn(0x7f1db40aa630 msgr2=0x7f1db40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 shutdown_connections 2024-03-06T07:23:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.681+0000 7f1db898c640 1 -- 172.21.15.26:0/1017021523 wait complete. 2024-03-06T07:23:34.696 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 7, "epoch": 77, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 7, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:24.599873+0000", "damaged": [], "data_pools": [ 16 ], "enabled": true, "epoch": 77, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6966": { "addr": "172.21.15.117:6837/1940022472", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 1940022472, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1940022472, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6966, "incarnation": 75, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 15, "modified": "2024-03-06T07:23:26.618139+0000", "qdb_cluster": [ 6966 ], "qdb_leader": 6966, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6966 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/593719862", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 593719862, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 593719862, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6736, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/1479693385", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 1479693385, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 1479693385, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6948, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/3691780502", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 3691780502, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 3691780502, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6951, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:23:34.696 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:23:34.697 INFO:tasks.cephfs.test_failover:current = [{'gid': 6948, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/1479693385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1479693385}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1479693385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}, {'gid': 6966, 'name': 'b', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/1940022472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 1940022472}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1940022472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6951, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/3691780502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 3691780502}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3691780502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}, {'gid': 6736, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/593719862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 593719862}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 593719862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}] 2024-03-06T07:23:34.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs 2024-03-06T07:23:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.799+0000 7f405b09d640 1 Processor -- start 2024-03-06T07:23:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.799+0000 7f405b09d640 1 -- start start 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405b09d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4054072720 0x7f4054156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405b09d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40541508c0 0x7f40540c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405b09d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4054157340 con 0x7f405406f7b0 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405b09d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40540c3740 con 0x7f4054072720 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405b09d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4054156d80 con 0x7f40541508c0 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f4053fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f405406f7b0 0x7f40541501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34888/0 (socket says 172.21.15.26:34888) 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f4053fff640 1 -- 172.21.15.26:0/2876103588 learned_addr learned my addr 172.21.15.26:0/2876103588 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405963a640 1 --2- 172.21.15.26:0/2876103588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40541508c0 0x7f40540c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f4058e39640 1 --2- 172.21.15.26:0/2876103588 >> v2:172.21.15.117:3300/0 conn(0x7f4054072720 0x7f4054156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f40537fe640 1 -- 172.21.15.26:0/2876103588 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57235763 0 0) 0x7f4054157340 con 0x7f405406f7b0 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f40537fe640 1 -- 172.21.15.26:0/2876103588 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4040003520 con 0x7f405406f7b0 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405963a640 1 -- 172.21.15.26:0/2876103588 >> v2:172.21.15.117:3300/0 conn(0x7f4054072720 msgr2=0x7f4054156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405963a640 1 --2- 172.21.15.26:0/2876103588 >> v2:172.21.15.117:3300/0 conn(0x7f4054072720 0x7f4054156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405963a640 1 -- 172.21.15.26:0/2876103588 >> v1:172.21.15.26:6789/0 conn(0x7f405406f7b0 legacy=0x7f40541501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.800+0000 7f405963a640 1 -- 172.21.15.26:0/2876103588 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4054157060 con 0x7f40541508c0 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.801+0000 7f4058e39640 1 --2- 172.21.15.26:0/2876103588 >> v2:172.21.15.117:3300/0 conn(0x7f4054072720 0x7f4054156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.801+0000 7f405963a640 1 --2- 172.21.15.26:0/2876103588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40541508c0 0x7f40540c3bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f945d5beca94580a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.801+0000 7f40537fe640 1 -- 172.21.15.26:0/2876103588 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0315a0 con 0x7f40541508c0 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.801+0000 7f40537fe640 1 -- 172.21.15.26:0/2876103588 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c037070 con 0x7f40541508c0 2024-03-06T07:23:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.801+0000 7f40537fe640 1 -- 172.21.15.26:0/2876103588 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c031960 con 0x7f40541508c0 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 -- 172.21.15.26:0/2876103588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40541508c0 msgr2=0x7f40540c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 --2- 172.21.15.26:0/2876103588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40541508c0 0x7f40540c3bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c02b840 comp rx=0 tx=0).stop 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 -- 172.21.15.26:0/2876103588 shutdown_connections 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 --2- 172.21.15.26:0/2876103588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40541508c0 0x7f40540c3bd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 --2- 172.21.15.26:0/2876103588 >> v2:172.21.15.117:3300/0 conn(0x7f4054072720 0x7f4054156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 -- 172.21.15.26:0/2876103588 >> 172.21.15.26:0/2876103588 conn(0x7f40540aa630 msgr2=0x7f40540aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 -- 172.21.15.26:0/2876103588 shutdown_connections 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.802+0000 7f405b09d640 1 -- 172.21.15.26:0/2876103588 wait complete. 2024-03-06T07:23:34.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.803+0000 7f405b09d640 1 Processor -- start 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.803+0000 7f405b09d640 1 -- start start 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405b09d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f405406f7b0 0x7f40541455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405b09d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4054072720 0x7f40541545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405b09d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40541508c0 0x7f4054152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405b09d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054156af0 con 0x7f40541508c0 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405b09d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054156970 con 0x7f405406f7b0 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405b09d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4054072b00 con 0x7f4054072720 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405963a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40541508c0 0x7f4054152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f405406f7b0 0x7f40541455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4053fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4054072720 0x7f40541545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f405406f7b0 0x7f40541455b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38688/0 (socket says 172.21.15.26:38688) 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f405963a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40541508c0 0x7f4054152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49586/0 (socket says 172.21.15.26:49586) 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 -- 172.21.15.26:0/4268754783 learned_addr learned my addr 172.21.15.26:0/4268754783 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:34.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 -- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4054072720 msgr2=0x7f40541545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4054072720 0x7f40541545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 -- 172.21.15.26:0/4268754783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40541508c0 msgr2=0x7f4054152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40541508c0 0x7f4054152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.804+0000 7f4058e39640 1 -- 172.21.15.26:0/4268754783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540c3d50 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.805+0000 7f4058e39640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f405406f7b0 0x7f40541455b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f404c00ebe0 tx=0x7f404c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.805+0000 7f40517fa640 1 -- 172.21.15.26:0/4268754783 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c017070 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.805+0000 7f40517fa640 1 -- 172.21.15.26:0/4268754783 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c00cc30 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.805+0000 7f40517fa640 1 -- 172.21.15.26:0/4268754783 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c013470 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.805+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40540c4010 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.805+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40541e5e80 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.806+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405404f330 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.806+0000 7f40517fa640 1 -- 172.21.15.26:0/4268754783 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f404c01a070 con 0x7f405406f7b0 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.806+0000 7f40517fa640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f403406b4d0 0x7f403406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.807+0000 7f40517fa640 1 -- 172.21.15.26:0/4268754783 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f404c08a5f0 con 0x7f405406f7b0 2024-03-06T07:23:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.807+0000 7f4053fff640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f403406b4d0 0x7f403406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.807+0000 7f4053fff640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f403406b4d0 0x7f403406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4054070700 tx=0x7f4044009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:34.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.811+0000 7f40517fa640 1 -- 172.21.15.26:0/4268754783 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f404c058c80 con 0x7f405406f7b0 2024-03-06T07:23:34.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.992+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f405414fb80 con 0x7f405406f7b0 2024-03-06T07:23:34.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.996+0000 7f40517fa640 1 -- 172.21.15.26:0/4268754783 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v32)=0 v32) v1 ==== 116+0+0 (secure 0 0 0) 0x7f404c05cd20 con 0x7f405406f7b0 2024-03-06T07:23:35.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f403406b4d0 msgr2=0x7f403406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f403406b4d0 0x7f403406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4054070700 tx=0x7f4044009040 comp rx=0 tx=0).stop 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f405406f7b0 msgr2=0x7f40541455b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f405406f7b0 0x7f40541455b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f404c00ebe0 tx=0x7f404c00c520 comp rx=0 tx=0).stop 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 shutdown_connections 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f403406b4d0 0x7f403406d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40541508c0 0x7f4054152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4054072720 0x7f40541545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 --2- 172.21.15.26:0/4268754783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f405406f7b0 0x7f40541455b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 >> 172.21.15.26:0/4268754783 conn(0x7f40540aa630 msgr2=0x7f405414e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 shutdown_connections 2024-03-06T07:23:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:34.998+0000 7f405b09d640 1 -- 172.21.15.26:0/4268754783 wait complete. 2024-03-06T07:23:35.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.110+0000 7f16e24bb640 1 Processor -- start 2024-03-06T07:23:35.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.110+0000 7f16e24bb640 1 -- start start 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e24bb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc14f360 0x7f16dc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e24bb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16dc158b60 0x7f16dc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e24bb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16dc0c73d0 con 0x7f16dc0c88c0 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e24bb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16dc0c8dd0 con 0x7f16dc158b60 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e24bb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0be2e0 con 0x7f16dc14f360 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e0a58640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16dc158b60 0x7f16dc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16db7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f16dc0c88c0 0x7f16dc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34896/0 (socket says 172.21.15.26:34896) 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16db7fe640 1 -- 172.21.15.26:0/1204885146 learned_addr learned my addr 172.21.15.26:0/1204885146 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16dbfff640 1 --2- 172.21.15.26:0/1204885146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc14f360 0x7f16dc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e0a58640 1 -- 172.21.15.26:0/1204885146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc14f360 msgr2=0x7f16dc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e0a58640 1 --2- 172.21.15.26:0/1204885146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc14f360 0x7f16dc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e0a58640 1 -- 172.21.15.26:0/1204885146 >> v1:172.21.15.26:6789/0 conn(0x7f16dc0c88c0 legacy=0x7f16dc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16e0a58640 1 -- 172.21.15.26:0/1204885146 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0c7670 con 0x7f16dc158b60 2024-03-06T07:23:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.111+0000 7f16daffd640 1 -- 172.21.15.26:0/1204885146 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106351513 0 0) 0x7f16dc0c73d0 con 0x7f16dc0c88c0 2024-03-06T07:23:35.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.112+0000 7f16e0a58640 1 --2- 172.21.15.26:0/1204885146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc158b60 0x7f16dc158ff0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009d90 tx=0x7f16d402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7494e2a1e305f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.112+0000 7f16daffd640 1 -- 172.21.15.26:0/1204885146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4039070 con 0x7f16dc158b60 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.112+0000 7f16daffd640 1 -- 172.21.15.26:0/1204885146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d402cbc0 con 0x7f16dc158b60 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.112+0000 7f16daffd640 1 -- 172.21.15.26:0/1204885146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4034440 con 0x7f16dc158b60 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 -- 172.21.15.26:0/1204885146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc158b60 msgr2=0x7f16dc158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1204885146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc158b60 0x7f16dc158ff0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009d90 tx=0x7f16d402be10 comp rx=0 tx=0).stop 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 -- 172.21.15.26:0/1204885146 shutdown_connections 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1204885146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc158b60 0x7f16dc158ff0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1204885146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc14f360 0x7f16dc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 -- 172.21.15.26:0/1204885146 >> 172.21.15.26:0/1204885146 conn(0x7f16dc0ac570 msgr2=0x7f16dc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 -- 172.21.15.26:0/1204885146 shutdown_connections 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.113+0000 7f16e24bb640 1 -- 172.21.15.26:0/1204885146 wait complete. 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.114+0000 7f16e24bb640 1 Processor -- start 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.114+0000 7f16e24bb640 1 -- start start 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16e24bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16dc0c88c0 0x7f16dc157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16e24bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc14f360 0x7f16dc14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16e24bb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc158b60 0x7f16dc14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16e24bb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0c5a50 con 0x7f16dc14f360 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16e24bb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0c7c30 con 0x7f16dc0c88c0 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16e24bb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0c5bd0 con 0x7f16dc158b60 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc14f360 0x7f16dc14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc14f360 0x7f16dc14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49610/0 (socket says 172.21.15.26:49610) 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 -- 172.21.15.26:0/1470003430 learned_addr learned my addr 172.21.15.26:0/1470003430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16dbfff640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16dc0c88c0 0x7f16dc157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16e0a58640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc158b60 0x7f16dc14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 -- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc158b60 msgr2=0x7f16dc14bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc158b60 0x7f16dc14bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 -- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16dc0c88c0 msgr2=0x7f16dc157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16dc0c88c0 0x7f16dc157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 -- 172.21.15.26:0/1470003430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc1eb4d0 con 0x7f16dc14f360 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16dbfff640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16dc0c88c0 0x7f16dc157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.115+0000 7f16db7fe640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc14f360 0x7f16dc14b790 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c3c0 tx=0x7f16d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.116+0000 7f16d8ff9640 1 -- 172.21.15.26:0/1470003430 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00074e0 con 0x7f16dc14f360 2024-03-06T07:23:35.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.116+0000 7f16d8ff9640 1 -- 172.21.15.26:0/1470003430 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d00047e0 con 0x7f16dc14f360 2024-03-06T07:23:35.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.116+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc1eb770 con 0x7f16dc14f360 2024-03-06T07:23:35.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.116+0000 7f16d8ff9640 1 -- 172.21.15.26:0/1470003430 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004bd0 con 0x7f16dc14f360 2024-03-06T07:23:35.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.116+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc151d60 con 0x7f16dc14f360 2024-03-06T07:23:35.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.117+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16dc069460 con 0x7f16dc14f360 2024-03-06T07:23:35.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.117+0000 7f16d8ff9640 1 -- 172.21.15.26:0/1470003430 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16d0011070 con 0x7f16dc14f360 2024-03-06T07:23:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.117+0000 7f16d8ff9640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16c806b4d0 0x7f16c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.117+0000 7f16d8ff9640 1 -- 172.21.15.26:0/1470003430 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16d0089cc0 con 0x7f16dc14f360 2024-03-06T07:23:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.118+0000 7f16dbfff640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16c806b4d0 0x7f16c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.118+0000 7f16dbfff640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16c806b4d0 0x7f16c806d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f16dc156050 tx=0x7f16cc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.122+0000 7f16d8ff9640 1 -- 172.21.15.26:0/1470003430 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f16dc069460 con 0x7f16dc14f360 2024-03-06T07:23:35.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.340+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16dc04f330 con 0x7f16dc14f360 2024-03-06T07:23:35.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.340+0000 7f16d8ff9640 1 -- 172.21.15.26:0/1470003430 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f16d00583f0 con 0x7f16dc14f360 2024-03-06T07:23:35.343 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:35.343 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:26.618139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1940022472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1940022472},{"type":"v1","addr":"172.21.15.117:6837","nonce":1940022472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6966,"qdb_cluster":[6966]},"id":7}]} 2024-03-06T07:23:35.343 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 77 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16c806b4d0 msgr2=0x7f16c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16c806b4d0 0x7f16c806d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f16dc156050 tx=0x7f16cc009190 comp rx=0 tx=0).stop 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc14f360 msgr2=0x7f16dc14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc14f360 0x7f16dc14b790 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c3c0 tx=0x7f16d000c880 comp rx=0 tx=0).stop 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 shutdown_connections 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16c806b4d0 0x7f16c806d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16dc158b60 0x7f16dc14bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16dc14f360 0x7f16dc14b790 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 --2- 172.21.15.26:0/1470003430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16dc0c88c0 0x7f16dc157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 >> 172.21.15.26:0/1470003430 conn(0x7f16dc0ac570 msgr2=0x7f16dc06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 shutdown_connections 2024-03-06T07:23:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.342+0000 7f16e24bb640 1 -- 172.21.15.26:0/1470003430 wait complete. 2024-03-06T07:23:35.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:24.599873+0000', 'modified': '2024-03-06T07:23:26.618139+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6966}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6966': {'gid': 6966, 'name': 'b', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/1940022472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 1940022472}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1940022472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6966, 'qdb_cluster': [6966]} 2024-03-06T07:23:35.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:23:35.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:23:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.861+0000 7f8658c4e640 1 Processor -- start 2024-03-06T07:23:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.861+0000 7f8658c4e640 1 -- start start 2024-03-06T07:23:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f8658c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f8658c4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86540c8e30 con 0x7f86540c64f0 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f8658c4e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86540c9110 con 0x7f86540c8950 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f8658c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8654152ce0 con 0x7f8654152900 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f8652d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86540c64f0 0x7f86540c69a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34906/0 (socket says 172.21.15.26:34906) 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f8652d9d640 1 -- 172.21.15.26:0/129551930 learned_addr learned my addr 172.21.15.26:0/129551930 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865259c640 1 --2- 172.21.15.26:0/129551930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865159a640 1 -- 172.21.15.26:0/129551930 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383276630 0 0) 0x7f86540c8e30 con 0x7f86540c64f0 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865159a640 1 -- 172.21.15.26:0/129551930 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f863c003520 con 0x7f86540c64f0 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865159a640 1 -- 172.21.15.26:0/129551930 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454724015 0 0) 0x7f86540c9110 con 0x7f86540c8950 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865159a640 1 -- 172.21.15.26:0/129551930 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86540c8e30 con 0x7f86540c8950 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865259c640 1 -- 172.21.15.26:0/129551930 >> v1:172.21.15.117:6789/0 conn(0x7f86540c8950 legacy=0x7f86540bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865259c640 1 -- 172.21.15.26:0/129551930 >> v1:172.21.15.26:6789/0 conn(0x7f86540c64f0 legacy=0x7f86540c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.862+0000 7f865259c640 1 -- 172.21.15.26:0/129551930 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540c93f0 con 0x7f8654152900 2024-03-06T07:23:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.863+0000 7f865259c640 1 --2- 172.21.15.26:0/129551930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654155240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86440097d0 tx=0x7f864402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a75812e8f1256b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.863+0000 7f865159a640 1 -- 172.21.15.26:0/129551930 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644031770 con 0x7f8654152900 2024-03-06T07:23:35.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.863+0000 7f865159a640 1 -- 172.21.15.26:0/129551930 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644037070 con 0x7f8654152900 2024-03-06T07:23:35.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.863+0000 7f865159a640 1 -- 172.21.15.26:0/129551930 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644031b10 con 0x7f8654152900 2024-03-06T07:23:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.864+0000 7f8658c4e640 1 -- 172.21.15.26:0/129551930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 msgr2=0x7f8654155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.864+0000 7f8658c4e640 1 --2- 172.21.15.26:0/129551930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654155240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86440097d0 tx=0x7f864402b810 comp rx=0 tx=0).stop 2024-03-06T07:23:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.864+0000 7f8658c4e640 1 -- 172.21.15.26:0/129551930 shutdown_connections 2024-03-06T07:23:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.864+0000 7f8658c4e640 1 --2- 172.21.15.26:0/129551930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654155240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.864+0000 7f8658c4e640 1 -- 172.21.15.26:0/129551930 >> 172.21.15.26:0/129551930 conn(0x7f86540ac860 msgr2=0x7f86540acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.864+0000 7f8658c4e640 1 -- 172.21.15.26:0/129551930 shutdown_connections 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.864+0000 7f8658c4e640 1 -- 172.21.15.26:0/129551930 wait complete. 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.865+0000 7f8658c4e640 1 Processor -- start 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.865+0000 7f8658c4e640 1 -- start start 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8658c4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86540c64f0 0x7f86541ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8658c4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86540c8950 0x7f8654145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8658c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8658c4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c0000 con 0x7f86540c8950 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8658c4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c70b0 con 0x7f86540c64f0 2024-03-06T07:23:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8658c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c0180 con 0x7f8654152900 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86540c8950 0x7f8654145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86540c8950 0x7f8654145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49614/0 (socket says 172.21.15.26:49614) 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 -- 172.21.15.26:0/2641756827 learned_addr learned my addr 172.21.15.26:0/2641756827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f865259c640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86540c64f0 0x7f86541ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8652d9d640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 -- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 msgr2=0x7f8654145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 -- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86540c64f0 msgr2=0x7f86541ee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86540c64f0 0x7f86541ee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 -- 172.21.15.26:0/2641756827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540c1bb0 con 0x7f86540c8950 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8651d9b640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86540c8950 0x7f8654145430 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f864800c490 tx=0x7f864800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.866+0000 7f8642ffd640 1 -- 172.21.15.26:0/2641756827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480052d0 con 0x7f86540c8950 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.867+0000 7f8642ffd640 1 -- 172.21.15.26:0/2641756827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648005470 con 0x7f86540c8950 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.867+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86541eeb90 con 0x7f86540c8950 2024-03-06T07:23:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.867+0000 7f8642ffd640 1 -- 172.21.15.26:0/2641756827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648012850 con 0x7f86540c8950 2024-03-06T07:23:35.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.867+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86540c43b0 con 0x7f86540c8950 2024-03-06T07:23:35.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.868+0000 7f8642ffd640 1 -- 172.21.15.26:0/2641756827 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86480129f0 con 0x7f86540c8950 2024-03-06T07:23:35.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.868+0000 7f8642ffd640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f863406b500 0x7f863406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.868+0000 7f8642ffd640 1 -- 172.21.15.26:0/2641756827 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8648089f20 con 0x7f86540c8950 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.869+0000 7f865259c640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f863406b500 0x7f863406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.869+0000 7f865259c640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f863406b500 0x7f863406d9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8644036000 tx=0x7f864403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.869+0000 7f865b83a740 1 Processor -- start 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865b83a740 1 -- start start 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca783a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca786330 0x55c5ca783f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865b83a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca7844b0 0x55c5ca780680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865b83a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca6ea6b0 con 0x55c5ca7844b0 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca693a60 con 0x55c5ca785f50 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca6e6750 con 0x55c5ca786330 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f8652d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca7844b0 0x55c5ca780680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f865259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca783a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f8652d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca7844b0 0x55c5ca780680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49628/0 (socket says 172.21.15.26:49628) 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca786330 0x55c5ca783f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.870+0000 7f8652d9d640 1 -- 172.21.15.26:0/3787961809 learned_addr learned my addr 172.21.15.26:0/3787961809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f865259c640 1 -- 172.21.15.26:0/3787961809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca786330 msgr2=0x55c5ca783f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f865259c640 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca786330 0x55c5ca783f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f865259c640 1 -- 172.21.15.26:0/3787961809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca7844b0 msgr2=0x55c5ca780680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f865259c640 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca7844b0 0x55c5ca780680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f865259c640 1 -- 172.21.15.26:0/3787961809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ca6b1f30 con 0x55c5ca785f50 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f8652d9d640 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca7844b0 0x55c5ca780680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f865259c640 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca783a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8644033000 tx=0x7f8644030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3ecc9c80dbed79 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f8633fff640 1 -- 172.21.15.26:0/3787961809 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440478e0 con 0x55c5ca785f50 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f8633fff640 1 -- 172.21.15.26:0/3787961809 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644037040 con 0x55c5ca785f50 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.871+0000 7f8633fff640 1 -- 172.21.15.26:0/3787961809 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644047c80 con 0x55c5ca785f50 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 -- 172.21.15.26:0/3787961809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 msgr2=0x55c5ca783a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca783a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8644033000 tx=0x7f8644030000 comp rx=0 tx=0).stop 2024-03-06T07:23:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 -- 172.21.15.26:0/3787961809 shutdown_connections 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca7844b0 0x55c5ca780680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca786330 0x55c5ca783f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 --2- 172.21.15.26:0/3787961809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca783a30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 -- 172.21.15.26:0/3787961809 >> 172.21.15.26:0/3787961809 conn(0x55c5ca7827d0 msgr2=0x55c5ca7a73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 -- 172.21.15.26:0/3787961809 shutdown_connections 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 -- 172.21.15.26:0/3787961809 wait complete. 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 -- start start 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.872+0000 7f865b83a740 1 Processor -- start 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca7a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865b83a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7a9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865b83a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca7a8230 con 0x55c5ca786330 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca6a6ea0 con 0x55c5ca7d6570 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca7a7c70 con 0x55c5ca785f50 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865b83a740 10 client.0 _ll_register_callbacks cb 0x55c5ca6a6870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49638/0 (socket says 172.21.15.26:49638) 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f8652d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f8651d9b640 1 -- 172.21.15.26:0/1804460451 learned_addr learned my addr 172.21.15.26:0/1804460451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f865259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca7a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.873+0000 7f8651d9b640 1 -- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 msgr2=0x55c5ca7a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f8651d9b640 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca7a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f8651d9b640 1 -- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 msgr2=0x55c5ca7a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f8651d9b640 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f8651d9b640 1 -- 172.21.15.26:0/1804460451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ca6a7270 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f8651d9b640 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7ac840 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f86480584b0 tx=0x7f864805c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cbbad2357e1476 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f86317fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f86317fa640 1 -- 172.21.15.26:0/1804460451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648086060 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f86317fa640 1 -- 172.21.15.26:0/1804460451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8648081b80 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8644037040 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f86317fa640 1 -- 172.21.15.26:0/1804460451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8648081d80 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 10 client.7167 fetch_fsmap learned FSMap version 77 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55c5ca3ff2d0 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f86317fa640 1 -- 172.21.15.26:0/1804460451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644037040 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f86317fa640 1 -- 172.21.15.26:0/1804460451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 77) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f864805c0b0 con 0x55c5ca786330 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 10 client.7167 fetch_fsmap finished waiting for FSMap version 77 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 10 client.7167 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 20 client.7167 populate_metadata read hostname 'smithi026' 2024-03-06T07:23:35.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca7d3260 0x55c5ca7e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 4 client.7167 mds_command: new command op to 6966 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 --> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5ca6f7bf0 con 0x55c5ca7d3260 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.874+0000 7f865259c640 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca7d3260 0x55c5ca7e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.875+0000 7f865259c640 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca7d3260 0x55c5ca7e1250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c8ff3424b20e2a server_cookie=b5638fc9e15eb5a4 in_seq=0 out_seq=0 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.875+0000 7f86317fa640 10 client.7167 ms_handle_connect on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.875+0000 7f86317fa640 1 -- 172.21.15.26:0/1804460451 <== mds.0 v2:172.21.15.117:6835/1940022472 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c5ca6f7bf0 con 0x55c5ca7d3260 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.875+0000 7f86317fa640 10 client.7167 handle_command_reply: tid=0 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 client.7167 shutdown 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 msgr2=0x55c5ca7ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7ac840 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f86480584b0 tx=0x7f864805c730 comp rx=0 tx=0).stop 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 shutdown_connections 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca7d3260 0x55c5ca7e1250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7ac840 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f86317fa640 0 client.7167 ms_handle_reset on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:35.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 --2- 172.21.15.26:0/1804460451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca7a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 >> 172.21.15.26:0/1804460451 conn(0x55c5ca7844b0 msgr2=0x55c5ca77f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 shutdown_connections 2024-03-06T07:23:35.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 1 -- 172.21.15.26:0/1804460451 wait complete. 2024-03-06T07:23:35.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.876+0000 7f865b83a740 20 client.7167 trim_cache size 0 max 16384 2024-03-06T07:23:35.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 Processor -- start 2024-03-06T07:23:35.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 -- start start 2024-03-06T07:23:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca783980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541f7a00 con 0x55c5ca786330 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541f6b80 con 0x55c5ca7d6570 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.893+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c3cf0 con 0x55c5ca785f50 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49648/0 (socket says 172.21.15.26:49648) 2024-03-06T07:23:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 -- 172.21.15.26:0/2430887363 learned_addr learned my addr 172.21.15.26:0/2430887363 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f865259c640 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca783980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8652d9d640 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 -- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 msgr2=0x55c5ca783980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca783980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 -- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 msgr2=0x55c5ca7cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 -- 172.21.15.26:0/2430887363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ca4c96b0 con 0x55c5ca786330 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f8651d9b640 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7cfa30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8648012fd0 tx=0x7f864805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed99557425259af server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f86337fe640 1 -- 172.21.15.26:0/2430887363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648086070 con 0x55c5ca786330 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f86337fe640 1 -- 172.21.15.26:0/2430887363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864805c0b0 con 0x55c5ca786330 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.894+0000 7f86337fe640 1 -- 172.21.15.26:0/2430887363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648057970 con 0x55c5ca786330 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 -- 172.21.15.26:0/2430887363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 msgr2=0x55c5ca7cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7cfa30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8648012fd0 tx=0x7f864805c040 comp rx=0 tx=0).stop 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 -- 172.21.15.26:0/2430887363 shutdown_connections 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca7d6570 0x55c5ca7cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7cfa30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 --2- 172.21.15.26:0/2430887363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca785f50 0x55c5ca783980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 -- 172.21.15.26:0/2430887363 >> 172.21.15.26:0/2430887363 conn(0x55c5ca7844b0 msgr2=0x55c5ca77bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 -- 172.21.15.26:0/2430887363 shutdown_connections 2024-03-06T07:23:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 -- 172.21.15.26:0/2430887363 wait complete. 2024-03-06T07:23:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 -- start start 2024-03-06T07:23:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.895+0000 7f865b83a740 1 Processor -- start 2024-03-06T07:23:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca7a4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865b83a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865b83a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca7d6570 0x55c5ca82fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865b83a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca7a8810 con 0x55c5ca786330 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541f6620 con 0x55c5ca785f50 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865b83a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5ca7c2990 con 0x55c5ca7d6570 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865b83a740 10 client.0 _ll_register_callbacks cb 0x7f86480585d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49656/0 (socket says 172.21.15.26:49656) 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 -- 172.21.15.26:0/4263234725 learned_addr learned my addr 172.21.15.26:0/4263234725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8652d9d640 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca7d6570 0x55c5ca82fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f865259c640 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca7a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 -- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca7d6570 msgr2=0x55c5ca82fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca7d6570 0x55c5ca82fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 -- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 msgr2=0x55c5ca7a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca7a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.896+0000 7f8651d9b640 1 -- 172.21.15.26:0/4263234725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ca77f640 con 0x55c5ca786330 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f8651d9b640 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7c5760 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8648012fd0 tx=0x7f86480880f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20891e8f924ac17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f86317fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f86317fa640 1 -- 172.21.15.26:0/4263234725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648086070 con 0x55c5ca786330 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f86317fa640 1 -- 172.21.15.26:0/4263234725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8648057b80 con 0x55c5ca786330 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f864805c0b0 con 0x55c5ca786330 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f86317fa640 1 -- 172.21.15.26:0/4263234725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8648054270 con 0x55c5ca786330 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 10 client.7173 fetch_fsmap learned FSMap version 77 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55c5ca7ab330 con 0x55c5ca786330 2024-03-06T07:23:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f86317fa640 1 -- 172.21.15.26:0/4263234725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864805c0b0 con 0x55c5ca786330 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f86317fa640 1 -- 172.21.15.26:0/4263234725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 77) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f8648054410 con 0x55c5ca786330 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 10 client.7173 fetch_fsmap finished waiting for FSMap version 77 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 10 client.7173 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 20 client.7173 populate_metadata read hostname 'smithi026' 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca8302f0 0x55c5ca780750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 4 client.7173 mds_command: new command op to 6966 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 --> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c5ca6f1f40 con 0x55c5ca8302f0 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.897+0000 7f865259c640 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca8302f0 0x55c5ca780750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865259c640 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca8302f0 0x55c5ca780750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f211244dbe8c32fc server_cookie=676ca1287be0018a in_seq=0 out_seq=0 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f86317fa640 10 client.7173 ms_handle_connect on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f86317fa640 1 -- 172.21.15.26:0/4263234725 <== mds.0 v2:172.21.15.117:6835/1940022472 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55c5ca6f1f40 con 0x55c5ca8302f0 2024-03-06T07:23:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f86317fa640 10 client.7173 handle_command_reply: tid=0 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 client.7173 shutdown 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 msgr2=0x55c5ca7c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7c5760 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8648012fd0 tx=0x7f86480880f0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 shutdown_connections 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:6835/1940022472,v1:172.21.15.117:6837/1940022472] conn(0x55c5ca8302f0 0x55c5ca780750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5ca7d6570 0x55c5ca82fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5ca786330 0x55c5ca7c5760 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 --2- 172.21.15.26:0/4263234725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5ca785f50 0x55c5ca7a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f86317fa640 0 client.7173 ms_handle_reset on v2:172.21.15.117:6835/1940022472 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 >> 172.21.15.26:0/4263234725 conn(0x55c5ca7844b0 msgr2=0x55c5ca7c3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 shutdown_connections 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 1 -- 172.21.15.26:0/4263234725 wait complete. 2024-03-06T07:23:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.898+0000 7f865b83a740 20 client.7173 trim_cache size 0 max 16384 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6966, 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.646875895999999, 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 77, 2024-03-06T07:23:35.913 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 95, 2024-03-06T07:23:35.914 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 95, 2024-03-06T07:23:35.914 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.731159346 2024-03-06T07:23:35.914 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:23:35.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f863406b500 msgr2=0x7f863406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f863406b500 0x7f863406d9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8644036000 tx=0x7f864403d040 comp rx=0 tx=0).stop 2024-03-06T07:23:35.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86540c8950 msgr2=0x7f8654145430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:35.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86540c8950 0x7f8654145430 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f864800c490 tx=0x7f864800c950 comp rx=0 tx=0).stop 2024-03-06T07:23:35.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 shutdown_connections 2024-03-06T07:23:35.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f863406b500 0x7f863406d9c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8654152900 0x7f8654145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86540c8950 0x7f8654145430 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 --2- 172.21.15.26:0/2641756827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86540c64f0 0x7f86541ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:35.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 >> 172.21.15.26:0/2641756827 conn(0x7f86540ac860 msgr2=0x7f8654155850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:35.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 shutdown_connections 2024-03-06T07:23:35.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:35.899+0000 7f8658c4e640 1 -- 172.21.15.26:0/2641756827 wait complete. 2024-03-06T07:23:35.916 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 7, "epoch": 77, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 7, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:24.599873+0000", "damaged": [], "data_pools": [ 16 ], "enabled": true, "epoch": 77, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6966": { "addr": "172.21.15.117:6837/1940022472", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 1940022472, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1940022472, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6966, "incarnation": 75, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 15, "modified": "2024-03-06T07:23:26.618139+0000", "qdb_cluster": [ 6966 ], "qdb_leader": 6966, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6966 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/593719862", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 593719862, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 593719862, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6736, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/1479693385", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 1479693385, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 1479693385, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6948, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/3691780502", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 3691780502, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 3691780502, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6951, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:23:35.917 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 7, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:23:35.917 INFO:tasks.cephfs.test_failover:current = [{'gid': 6948, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/1479693385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1479693385}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1479693385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}, {'gid': 6966, 'name': 'b', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/1940022472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 1940022472}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1940022472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6951, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/3691780502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 3691780502}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3691780502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}, {'gid': 6736, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/593719862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 593719862}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 593719862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}] 2024-03-06T07:23:35.918 DEBUG:tasks.cephfs.test_failover:7 != -1 2024-03-06T07:23:35.918 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:23:37.005 INFO:tasks.ceph.mds.b.smithi117.stderr: -264> 2024-03-06T07:23:30.532+0000 7f301772d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:37.032 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:23:40.533 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:23:40.531+0000 7f0a9ebaf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:40.918 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:41.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.029+0000 7f75642eb640 1 Processor -- start 2024-03-06T07:23:41.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.029+0000 7f75642eb640 1 -- start start 2024-03-06T07:23:41.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f75642eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f75642eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f755c0c3280 0x7f755c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f75642eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c158800 con 0x7f755c1551e0 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f75642eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c158ae0 con 0x7f755c0c3280 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f75642eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c158dc0 con 0x7f755c06d9f0 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f7562888640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f755c1551e0 0x7f755c155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38310/0 (socket says 172.21.15.26:38310) 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f7562888640 1 -- 172.21.15.26:0/1018318375 learned_addr learned my addr 172.21.15.26:0/1018318375 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f7561886640 1 --2- 172.21.15.26:0/1018318375 >> v2:172.21.15.117:3300/0 conn(0x7f755c0c3280 0x7f755c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f7562087640 1 --2- 172.21.15.26:0/1018318375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.030+0000 7f7561085640 1 -- 172.21.15.26:0/1018318375 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855003206 0 0) 0x7f755c158800 con 0x7f755c1551e0 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.031+0000 7f7561085640 1 -- 172.21.15.26:0/1018318375 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7544003520 con 0x7f755c1551e0 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.031+0000 7f7562087640 1 -- 172.21.15.26:0/1018318375 >> v2:172.21.15.117:3300/0 conn(0x7f755c0c3280 msgr2=0x7f755c0c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.031+0000 7f7562087640 1 --2- 172.21.15.26:0/1018318375 >> v2:172.21.15.117:3300/0 conn(0x7f755c0c3280 0x7f755c0c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.031+0000 7f7562087640 1 -- 172.21.15.26:0/1018318375 >> v1:172.21.15.26:6789/0 conn(0x7f755c1551e0 legacy=0x7f755c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.031+0000 7f7562087640 1 -- 172.21.15.26:0/1018318375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c1590a0 con 0x7f755c06d9f0 2024-03-06T07:23:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.031+0000 7f7561886640 1 --2- 172.21.15.26:0/1018318375 >> v2:172.21.15.117:3300/0 conn(0x7f755c0c3280 0x7f755c0c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.031+0000 7f7562087640 1 --2- 172.21.15.26:0/1018318375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c0a7cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f75540097a0 tx=0x7f755402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d07c6821ed4eb20 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f7561085640 1 -- 172.21.15.26:0/1018318375 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540315a0 con 0x7f755c06d9f0 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f7561085640 1 -- 172.21.15.26:0/1018318375 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554037070 con 0x7f755c06d9f0 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f7561085640 1 -- 172.21.15.26:0/1018318375 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554031960 con 0x7f755c06d9f0 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 -- 172.21.15.26:0/1018318375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 msgr2=0x7f755c0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 --2- 172.21.15.26:0/1018318375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c0a7cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f75540097a0 tx=0x7f755402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 -- 172.21.15.26:0/1018318375 shutdown_connections 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 --2- 172.21.15.26:0/1018318375 >> v2:172.21.15.117:3300/0 conn(0x7f755c0c3280 0x7f755c0c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 --2- 172.21.15.26:0/1018318375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c0a7cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 -- 172.21.15.26:0/1018318375 >> 172.21.15.26:0/1018318375 conn(0x7f755c0aa630 msgr2=0x7f755c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 -- 172.21.15.26:0/1018318375 shutdown_connections 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.032+0000 7f75642eb640 1 -- 172.21.15.26:0/1018318375 wait complete. 2024-03-06T07:23:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.033+0000 7f75642eb640 1 Processor -- start 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.033+0000 7f75642eb640 1 -- start start 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f75642eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c1ede90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f75642eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f755c0c3280 0x7f755c1ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f75642eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f755c1551e0 0x7f755c0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f75642eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c06df50 con 0x7f755c0c3280 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f75642eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c06ddd0 con 0x7f755c1551e0 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f75642eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0a8210 con 0x7f755c06d9f0 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f7561886640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f755c0c3280 0x7f755c1ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f7562087640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c1ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f7561886640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f755c0c3280 0x7f755c1ee3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57360/0 (socket says 172.21.15.26:57360) 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f7562888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f755c1551e0 0x7f755c0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.034+0000 7f7561886640 1 -- 172.21.15.26:0/2803251575 learned_addr learned my addr 172.21.15.26:0/2803251575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f7561886640 1 -- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 msgr2=0x7f755c1ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f7561886640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c1ede90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f7561886640 1 -- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f755c1551e0 msgr2=0x7f755c0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f7561886640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f755c1551e0 0x7f755c0ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f7561886640 1 -- 172.21.15.26:0/2803251575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c06ec30 con 0x7f755c0c3280 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f7562087640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c1ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f7561886640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f755c0c3280 0x7f755c1ee3d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f755000ebe0 tx=0x7f755000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f755affd640 1 -- 172.21.15.26:0/2803251575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550017070 con 0x7f755c0c3280 2024-03-06T07:23:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f755affd640 1 -- 172.21.15.26:0/2803251575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755000cc30 con 0x7f755c0c3280 2024-03-06T07:23:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c06ef10 con 0x7f755c0c3280 2024-03-06T07:23:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f755affd640 1 -- 172.21.15.26:0/2803251575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550013470 con 0x7f755c0c3280 2024-03-06T07:23:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.035+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c158080 con 0x7f755c0c3280 2024-03-06T07:23:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.036+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c04f330 con 0x7f755c0c3280 2024-03-06T07:23:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.037+0000 7f755affd640 1 -- 172.21.15.26:0/2803251575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f755001a070 con 0x7f755c0c3280 2024-03-06T07:23:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.037+0000 7f755affd640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b4d0 0x7f754806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.037+0000 7f755affd640 1 -- 172.21.15.26:0/2803251575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f755008a600 con 0x7f755c0c3280 2024-03-06T07:23:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.037+0000 7f7562087640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b4d0 0x7f754806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.038+0000 7f7562087640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b4d0 0x7f754806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7554036000 tx=0x7f755403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.041+0000 7f755affd640 1 -- 172.21.15.26:0/2803251575 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7550058c50 con 0x7f755c0c3280 2024-03-06T07:23:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.261+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f755c072490 con 0x7f755c0c3280 2024-03-06T07:23:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.262+0000 7f755affd640 1 -- 172.21.15.26:0/2803251575 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f755005ccf0 con 0x7f755c0c3280 2024-03-06T07:23:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:41.265 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1702158641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1702158641},{"type":"v1","addr":"172.21.15.117:6837","nonce":1702158641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:40.012005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6736":{"gid":6736,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6736,"qdb_cluster":[6736]},"id":7}]} 2024-03-06T07:23:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 82 2024-03-06T07:23:41.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b4d0 msgr2=0x7f754806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b4d0 0x7f754806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7554036000 tx=0x7f755403d040 comp rx=0 tx=0).stop 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f755c0c3280 msgr2=0x7f755c1ee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f755c0c3280 0x7f755c1ee3d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f755000ebe0 tx=0x7f755000c520 comp rx=0 tx=0).stop 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 shutdown_connections 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f754806b4d0 0x7f754806d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f755c1551e0 0x7f755c0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f755c0c3280 0x7f755c1ee3d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 --2- 172.21.15.26:0/2803251575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f755c06d9f0 0x7f755c1ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 >> 172.21.15.26:0/2803251575 conn(0x7f755c0aa630 msgr2=0x7f755c072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 shutdown_connections 2024-03-06T07:23:41.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.264+0000 7f75642eb640 1 -- 172.21.15.26:0/2803251575 wait complete. 2024-03-06T07:23:41.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:24.599873+0000', 'modified': '2024-03-06T07:23:40.012005+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6736}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6736': {'gid': 6736, 'name': 'd', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.117:6836/593719862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 593719862}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 593719862}]}, 'join_fscid': 7, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6736, 'qdb_cluster': [6736]} 2024-03-06T07:23:41.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:23:41.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:23:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.785+0000 7fb56f240640 1 Processor -- start 2024-03-06T07:23:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.785+0000 7fb56f240640 1 -- start start 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56f240640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb568152720 0x7fb5681593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56f240640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5680733f0 con 0x7fb568072cd0 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56f240640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb568155ef0 con 0x7fb56806f550 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56f240640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb568155ca0 con 0x7fb568152720 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56cfdc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb568152720 0x7fb5681593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56d7dd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb568072cd0 0x7fb56806fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38312/0 (socket says 172.21.15.26:38312) 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56d7dd640 1 -- 172.21.15.26:0/1394243698 learned_addr learned my addr 172.21.15.26:0/1394243698 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56cfdc640 1 -- 172.21.15.26:0/1394243698 >> v1:172.21.15.117:6789/0 conn(0x7fb56806f550 legacy=0x7fb5680c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56cfdc640 1 -- 172.21.15.26:0/1394243698 >> v1:172.21.15.26:6789/0 conn(0x7fb568072cd0 legacy=0x7fb56806fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.786+0000 7fb56cfdc640 1 -- 172.21.15.26:0/1394243698 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5681588e0 con 0x7fb568152720 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.787+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/1394243698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb568152720 0x7fb5681593c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0097a0 tx=0x7fb55c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa790e83d96f9137 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.787+0000 7fb5677fe640 1 -- 172.21.15.26:0/1394243698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c035020 con 0x7fb568152720 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.787+0000 7fb5677fe640 1 -- 172.21.15.26:0/1394243698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c031830 con 0x7fb568152720 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.787+0000 7fb5677fe640 1 -- 172.21.15.26:0/1394243698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c031c00 con 0x7fb568152720 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.788+0000 7fb56f240640 1 -- 172.21.15.26:0/1394243698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb568152720 msgr2=0x7fb5681593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.788+0000 7fb56f240640 1 --2- 172.21.15.26:0/1394243698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb568152720 0x7fb5681593c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0097a0 tx=0x7fb55c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.788+0000 7fb56f240640 1 -- 172.21.15.26:0/1394243698 shutdown_connections 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.788+0000 7fb56f240640 1 --2- 172.21.15.26:0/1394243698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb568152720 0x7fb5681593c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.788+0000 7fb56f240640 1 -- 172.21.15.26:0/1394243698 >> 172.21.15.26:0/1394243698 conn(0x7fb56806ea40 msgr2=0x7fb56806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.788+0000 7fb56f240640 1 -- 172.21.15.26:0/1394243698 shutdown_connections 2024-03-06T07:23:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.788+0000 7fb56f240640 1 -- 172.21.15.26:0/1394243698 wait complete. 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.789+0000 7fb56f240640 1 Processor -- start 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.789+0000 7fb56f240640 1 -- start start 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.789+0000 7fb56f240640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56806f550 0x7fb5681ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.789+0000 7fb56f240640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb568072cd0 0x7fb5680c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56f240640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb568152720 0x7fb568149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56f240640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb568158ea0 con 0x7fb56806f550 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56f240640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5681561d0 con 0x7fb568152720 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56f240640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5680c7ef0 con 0x7fb568072cd0 2024-03-06T07:23:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56806f550 0x7fb5681ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56806f550 0x7fb5681ebab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57382/0 (socket says 172.21.15.26:57382) 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 -- 172.21.15.26:0/2816649800 learned_addr learned my addr 172.21.15.26:0/2816649800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb568152720 0x7fb568149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb567fff640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb568072cd0 0x7fb5680c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 -- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb568072cd0 msgr2=0x7fb5680c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb568072cd0 0x7fb5680c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 -- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb568152720 msgr2=0x7fb568149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb568152720 0x7fb568149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 -- 172.21.15.26:0/2816649800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5680c3cc0 con 0x7fb56806f550 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb567fff640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb568072cd0 0x7fb5680c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56806f550 0x7fb5681ebab0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034040 tx=0x7fb55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb5657fa640 1 -- 172.21.15.26:0/2816649800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c040070 con 0x7fb56806f550 2024-03-06T07:23:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb5657fa640 1 -- 172.21.15.26:0/2816649800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c039a20 con 0x7fb56806f550 2024-03-06T07:23:41.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5680c3fa0 con 0x7fb56806f550 2024-03-06T07:23:41.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb5657fa640 1 -- 172.21.15.26:0/2816649800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c035040 con 0x7fb56806f550 2024-03-06T07:23:41.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.790+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb568156600 con 0x7fb56806f550 2024-03-06T07:23:41.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.792+0000 7fb5657fa640 1 -- 172.21.15.26:0/2816649800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb55c038020 con 0x7fb56806f550 2024-03-06T07:23:41.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.792+0000 7fb5657fa640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb54c06b4d0 0x7fb54c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.792+0000 7fb5657fa640 1 -- 172.21.15.26:0/2816649800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb55c0abcf0 con 0x7fb56806f550 2024-03-06T07:23:41.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.792+0000 7fb567fff640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb54c06b4d0 0x7fb54c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.793+0000 7fb567fff640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb54c06b4d0 0x7fb54c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb5680c6440 tx=0x7fb554009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.793+0000 7fb571e22740 1 Processor -- start 2024-03-06T07:23:41.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.793+0000 7fb571e22740 1 -- start start 2024-03-06T07:23:41.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb571e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb759f50 0x561fbb757a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb7584b0 0x561fbb754680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb571e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbb6be6b0 con 0x561fbb759f50 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbb667a60 con 0x561fbb7584b0 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fbb6ba750 con 0x561fbb75a330 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb759f50 0x561fbb757a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb759f50 0x561fbb757a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57396/0 (socket says 172.21.15.26:57396) 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 -- 172.21.15.26:0/936330690 learned_addr learned my addr 172.21.15.26:0/936330690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb7584b0 0x561fbb754680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb567fff640 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 -- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 msgr2=0x561fbb757f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 -- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb7584b0 msgr2=0x561fbb754680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb7584b0 0x561fbb754680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 -- 172.21.15.26:0/936330690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fbb685f30 con 0x561fbb759f50 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.794+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb759f50 0x561fbb757a30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb55c082500 tx=0x7fb55c079d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af03c08a029f069a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb5527fc640 1 -- 172.21.15.26:0/936330690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c07e470 con 0x561fbb759f50 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb5527fc640 1 -- 172.21.15.26:0/936330690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c082a30 con 0x561fbb759f50 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb5527fc640 1 -- 172.21.15.26:0/936330690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c078620 con 0x561fbb759f50 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 -- 172.21.15.26:0/936330690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb759f50 msgr2=0x561fbb757a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb759f50 0x561fbb757a30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb55c082500 tx=0x7fb55c079d40 comp rx=0 tx=0).stop 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 -- 172.21.15.26:0/936330690 shutdown_connections 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb7584b0 0x561fbb754680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 --2- 172.21.15.26:0/936330690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb759f50 0x561fbb757a30 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 -- 172.21.15.26:0/936330690 >> 172.21.15.26:0/936330690 conn(0x561fbb7567d0 msgr2=0x561fbb77b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 -- 172.21.15.26:0/936330690 shutdown_connections 2024-03-06T07:23:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.795+0000 7fb571e22740 1 -- 172.21.15.26:0/936330690 wait complete. 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.796+0000 7fb571e22740 1 -- start start 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.796+0000 7fb571e22740 1 Processor -- start 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb77dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb75a330 0x561fbb795e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb780ac0 0x561fbb7a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbb6be6b0 con 0x561fbb75a330 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbb67aea0 con 0x561fbb759f50 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fbb77c230 con 0x561fbb780ac0 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 10 client.0 _ll_register_callbacks cb 0x561fbb67a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb75a330 0x561fbb795e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb75a330 0x561fbb795e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57402/0 (socket says 172.21.15.26:57402) 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb56cfdc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb77dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 -- 172.21.15.26:0/1976804151 learned_addr learned my addr 172.21.15.26:0/1976804151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb780ac0 0x561fbb7a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 -- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb780ac0 msgr2=0x561fbb7a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb780ac0 0x561fbb7a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 -- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 msgr2=0x561fbb77dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb77dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 -- 172.21.15.26:0/1976804151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fbb67b270 con 0x561fbb75a330 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb567fff640 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb75a330 0x561fbb795e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb56806f990 tx=0x7fb554000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b798162fe84091 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb5477fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb5477fe640 1 -- 172.21.15.26:0/1976804151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554027060 con 0x561fbb75a330 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb5477fe640 1 -- 172.21.15.26:0/1976804151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb554010040 con 0x561fbb75a330 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.797+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb55c082a30 con 0x561fbb75a330 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb5477fe640 1 -- 172.21.15.26:0/1976804151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb554003d80 con 0x561fbb75a330 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 10 client.7203 fetch_fsmap learned FSMap version 82 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x561fbb3d32d0 con 0x561fbb75a330 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb5477fe640 1 -- 172.21.15.26:0/1976804151 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c082a30 con 0x561fbb75a330 2024-03-06T07:23:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb5477fe640 1 -- 172.21.15.26:0/1976804151 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 82) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fb554013420 con 0x561fbb75a330 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 10 client.7203 fetch_fsmap finished waiting for FSMap version 82 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 10 client.7203 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 20 client.7203 populate_metadata read hostname 'smithi026' 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb7aa480 0x561fbb7b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 4 client.7203 mds_command: new command op to 6736 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 --> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fbb6cbbf0 con 0x561fbb7aa480 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb7aa480 0x561fbb7b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb5477fe640 10 client.7203 ms_handle_connect on v2:172.21.15.117:6834/593719862 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.798+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb7aa480 0x561fbb7b53c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b153ba9f1ab6120b server_cookie=9051553c5622f891 in_seq=0 out_seq=0 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb5477fe640 1 -- 172.21.15.26:0/1976804151 <== mds.0 v2:172.21.15.117:6834/593719862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561fbb6cbbf0 con 0x561fbb7aa480 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb5477fe640 10 client.7203 handle_command_reply: tid=0 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 client.7203 shutdown 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb75a330 msgr2=0x561fbb795e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb75a330 0x561fbb795e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb56806f990 tx=0x7fb554000ed0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 shutdown_connections 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb7aa480 0x561fbb7b53c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb780ac0 0x561fbb7a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb75a330 0x561fbb795e10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb5477fe640 0 client.7203 ms_handle_reset on v2:172.21.15.117:6834/593719862 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 --2- 172.21.15.26:0/1976804151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb77dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 >> 172.21.15.26:0/1976804151 conn(0x561fbb7584b0 msgr2=0x561fbb753510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 shutdown_connections 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 1 -- 172.21.15.26:0/1976804151 wait complete. 2024-03-06T07:23:41.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.799+0000 7fb571e22740 20 client.7203 trim_cache size 0 max 16384 2024-03-06T07:23:41.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.816+0000 7fb571e22740 1 Processor -- start 2024-03-06T07:23:41.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.816+0000 7fb571e22740 1 -- start start 2024-03-06T07:23:41.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb799730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb571e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb7a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb571e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5681f7180 con 0x561fbb780ac0 2024-03-06T07:23:41.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb568157770 con 0x561fbb759f50 2024-03-06T07:23:41.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5681f6bb0 con 0x561fbb75a330 2024-03-06T07:23:41.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb7a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb7a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57410/0 (socket says 172.21.15.26:57410) 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 -- 172.21.15.26:0/4126654983 learned_addr learned my addr 172.21.15.26:0/4126654983 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb799730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb567fff640 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 -- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 msgr2=0x561fbb757790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 -- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 msgr2=0x561fbb799730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb799730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 -- 172.21.15.26:0/4126654983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fbb49d6b0 con 0x561fbb780ac0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.817+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb7a3790 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb5600012e0 tx=0x7fb560001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf6501c8a347c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb551ffb640 1 -- 172.21.15.26:0/4126654983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5600166f0 con 0x561fbb780ac0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb551ffb640 1 -- 172.21.15.26:0/4126654983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56001a030 con 0x561fbb780ac0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb551ffb640 1 -- 172.21.15.26:0/4126654983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560016ab0 con 0x561fbb780ac0 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 -- 172.21.15.26:0/4126654983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 msgr2=0x561fbb7a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb7a3790 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb5600012e0 tx=0x7fb560001940 comp rx=0 tx=0).stop 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 -- 172.21.15.26:0/4126654983 shutdown_connections 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb7a3790 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb75a330 0x561fbb757790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 --2- 172.21.15.26:0/4126654983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb759f50 0x561fbb799730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 -- 172.21.15.26:0/4126654983 >> 172.21.15.26:0/4126654983 conn(0x561fbb7584b0 msgr2=0x561fbb799db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 -- 172.21.15.26:0/4126654983 shutdown_connections 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.818+0000 7fb571e22740 1 -- 172.21.15.26:0/4126654983 wait complete. 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 -- start start 2024-03-06T07:23:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 Processor -- start 2024-03-06T07:23:41.824 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb759f50 0x561fbb795290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb75a330 0x561fbb7a6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb753910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbb77d340 con 0x561fbb780ac0 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbb7aa300 con 0x561fbb75a330 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fbb77e4d0 con 0x561fbb759f50 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.819+0000 7fb571e22740 10 client.0 _ll_register_callbacks cb 0x7fb5540050f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb753910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56cfdc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb759f50 0x561fbb795290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb753910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57420/0 (socket says 172.21.15.26:57420) 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 -- 172.21.15.26:0/2136817269 learned_addr learned my addr 172.21.15.26:0/2136817269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb567fff640 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb75a330 0x561fbb7a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 -- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb759f50 msgr2=0x561fbb795290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb759f50 0x561fbb795290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 -- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb75a330 msgr2=0x561fbb7a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb75a330 0x561fbb7a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 -- 172.21.15.26:0/2136817269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fbb7934e0 con 0x561fbb780ac0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb56d7dd640 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb753910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb560001ba0 tx=0x7fb560001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73251a3c45284958 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb5477fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb5477fe640 1 -- 172.21.15.26:0/2136817269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560002090 con 0x561fbb780ac0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb5477fe640 1 -- 172.21.15.26:0/2136817269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb560016d10 con 0x561fbb780ac0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb56001a030 con 0x561fbb780ac0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.820+0000 7fb5477fe640 1 -- 172.21.15.26:0/2136817269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb560032050 con 0x561fbb780ac0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 10 client.7209 fetch_fsmap learned FSMap version 82 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x561fbb7957d0 con 0x561fbb780ac0 2024-03-06T07:23:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb5477fe640 1 -- 172.21.15.26:0/2136817269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56001a030 con 0x561fbb780ac0 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb5477fe640 1 -- 172.21.15.26:0/2136817269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 82) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fb560035020 con 0x561fbb780ac0 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 10 client.7209 fetch_fsmap finished waiting for FSMap version 82 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 10 client.7209 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 20 client.7209 populate_metadata read hostname 'smithi026' 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb809a10 0x561fbb809df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 4 client.7209 mds_command: new command op to 6736 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 --> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561fbb6c5f40 con 0x561fbb809a10 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb809a10 0x561fbb809df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb56cfdc640 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb809a10 0x561fbb809df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7ace3a09080b13 server_cookie=98045a55ab7e8656 in_seq=0 out_seq=0 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.821+0000 7fb5477fe640 10 client.7209 ms_handle_connect on v2:172.21.15.117:6834/593719862 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb5477fe640 1 -- 172.21.15.26:0/2136817269 <== mds.0 v2:172.21.15.117:6834/593719862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x561fbb6c5f40 con 0x561fbb809a10 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb5477fe640 10 client.7209 handle_command_reply: tid=0 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 client.7209 shutdown 2024-03-06T07:23:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 msgr2=0x561fbb753910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb753910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb560001ba0 tx=0x7fb560001250 comp rx=0 tx=0).stop 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 shutdown_connections 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:6834/593719862,v1:172.21.15.117:6836/593719862] conn(0x561fbb809a10 0x561fbb809df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fbb780ac0 0x561fbb753910 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fbb75a330 0x561fbb7a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb5477fe640 0 client.7209 ms_handle_reset on v2:172.21.15.117:6834/593719862 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 --2- 172.21.15.26:0/2136817269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561fbb759f50 0x561fbb795290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 >> 172.21.15.26:0/2136817269 conn(0x561fbb7584b0 msgr2=0x561fbb7ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 shutdown_connections 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 1 -- 172.21.15.26:0/2136817269 wait complete. 2024-03-06T07:23:41.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb571e22740 20 client.7209 trim_cache size 0 max 16384 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6736, 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.8206607469999998, 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 82, 2024-03-06T07:23:41.832 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 96, 2024-03-06T07:23:41.833 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 96, 2024-03-06T07:23:41.833 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 21.655186175000001 2024-03-06T07:23:41.833 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb54c06b4d0 msgr2=0x7fb54c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb56f240640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb54c06b4d0 0x7fb54c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb5680c6440 tx=0x7fb554009040 comp rx=0 tx=0).stop 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56806f550 msgr2=0x7fb5681ebab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb56f240640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56806f550 0x7fb5681ebab0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034040 tx=0x7fb55c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 shutdown_connections 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.822+0000 7fb56f240640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb54c06b4d0 0x7fb54c06d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.823+0000 7fb56f240640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb568152720 0x7fb568149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.823+0000 7fb56f240640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb568072cd0 0x7fb5680c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.823+0000 7fb56f240640 1 --2- 172.21.15.26:0/2816649800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56806f550 0x7fb5681ebab0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.823+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 >> 172.21.15.26:0/2816649800 conn(0x7fb56806ea40 msgr2=0x7fb56806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.823+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 shutdown_connections 2024-03-06T07:23:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:41.823+0000 7fb56f240640 1 -- 172.21.15.26:0/2816649800 wait complete. 2024-03-06T07:23:41.835 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:41.836 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 7, "epoch": 82, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 7, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:24.599873+0000", "damaged": [], "data_pools": [ 16 ], "enabled": true, "epoch": 82, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6736": { "addr": "172.21.15.117:6836/593719862", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 593719862, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 593719862, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6736, "incarnation": 79, "join_fscid": 7, "name": "d", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 96, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 15, "modified": "2024-03-06T07:23:40.012005+0000", "qdb_cluster": [ 6736 ], "qdb_leader": 6736, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6736 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/1702158641", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 1702158641, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1702158641, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 80, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6901, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/1479693385", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 1479693385, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 1479693385, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6948, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/3691780502", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 3691780502, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 3691780502, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6951, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:23:41.837 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 7, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:23:41.837 INFO:tasks.cephfs.test_failover:current = [{'gid': 6948, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/1479693385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1479693385}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1479693385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}, {'gid': 6901, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/1702158641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 1702158641}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1702158641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 80}, {'gid': 6951, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/3691780502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 3691780502}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3691780502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 74}, {'gid': 6736, 'name': 'd', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.117:6836/593719862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 593719862}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 593719862}]}, 'join_fscid': 7, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:23:41.838 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:23:41.838 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:41.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:41.927 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:41.970 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:41.970 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:23:41.971 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:41.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:41.985 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:23:41.985 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:42.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:42.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.125+0000 7fa846f02640 1 Processor -- start 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.125+0000 7fa846f02640 1 -- start start 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa846f02640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa84006da40 0x7fa840155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa846f02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8400c6780 0x7fa840158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa846f02640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa840156050 con 0x7fa8401592d0 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa846f02640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84006de20 con 0x7fa84006da40 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa846f02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa840072630 con 0x7fa8400c6780 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8400c6780 0x7fa840158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa844c9e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa84006da40 0x7fa840155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8400c6780 0x7fa840158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:38754/0 (socket says 172.21.15.26:38754) 2024-03-06T07:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa83ffff640 1 -- 172.21.15.26:0/615638509 learned_addr learned my addr 172.21.15.26:0/615638509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa83ffff640 1 -- 172.21.15.26:0/615638509 >> v2:172.21.15.117:3300/0 conn(0x7fa84006da40 msgr2=0x7fa840155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa83ffff640 1 --2- 172.21.15.26:0/615638509 >> v2:172.21.15.117:3300/0 conn(0x7fa84006da40 0x7fa840155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa83ffff640 1 -- 172.21.15.26:0/615638509 >> v1:172.21.15.117:6789/0 conn(0x7fa8401592d0 legacy=0x7fa8400c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa83ffff640 1 -- 172.21.15.26:0/615638509 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840072910 con 0x7fa8400c6780 2024-03-06T07:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.126+0000 7fa844c9e640 1 --2- 172.21.15.26:0/615638509 >> v2:172.21.15.117:3300/0 conn(0x7fa84006da40 0x7fa840155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.127+0000 7fa83ffff640 1 --2- 172.21.15.26:0/615638509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8400c6780 0x7fa840158d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009870 tx=0x7fa82c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7c0c8e224bfd4b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.129+0000 7fa83f7fe640 1 -- 172.21.15.26:0/615638509 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c035020 con 0x7fa8400c6780 2024-03-06T07:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.129+0000 7fa83f7fe640 1 -- 172.21.15.26:0/615638509 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c031640 con 0x7fa8400c6780 2024-03-06T07:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.129+0000 7fa83f7fe640 1 -- 172.21.15.26:0/615638509 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c02d040 con 0x7fa8400c6780 2024-03-06T07:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 -- 172.21.15.26:0/615638509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8400c6780 msgr2=0x7fa840158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 --2- 172.21.15.26:0/615638509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8400c6780 0x7fa840158d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009870 tx=0x7fa82c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 -- 172.21.15.26:0/615638509 shutdown_connections 2024-03-06T07:23:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 --2- 172.21.15.26:0/615638509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8400c6780 0x7fa840158d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 --2- 172.21.15.26:0/615638509 >> v2:172.21.15.117:3300/0 conn(0x7fa84006da40 0x7fa840155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 -- 172.21.15.26:0/615638509 >> 172.21.15.26:0/615638509 conn(0x7fa8400aa630 msgr2=0x7fa8400aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 -- 172.21.15.26:0/615638509 shutdown_connections 2024-03-06T07:23:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.130+0000 7fa846f02640 1 -- 172.21.15.26:0/615638509 wait complete. 2024-03-06T07:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.131+0000 7fa846f02640 1 Processor -- start 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.131+0000 7fa846f02640 1 -- start start 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa846f02640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa84006da40 0x7fa840156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa846f02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8400c6780 0x7fa8401eed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa846f02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8401592d0 0x7fa8400a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa846f02640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400c3990 con 0x7fa8400c6780 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa846f02640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa840158890 con 0x7fa84006da40 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa846f02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400c3b10 con 0x7fa8401592d0 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8400c6780 0x7fa8401eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8400c6780 0x7fa8401eed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57428/0 (socket says 172.21.15.26:57428) 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 -- 172.21.15.26:0/2868434394 learned_addr learned my addr 172.21.15.26:0/2868434394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa844c9e640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa84006da40 0x7fa840156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa84549f640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8401592d0 0x7fa8400a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 -- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8401592d0 msgr2=0x7fa8400a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8401592d0 0x7fa8400a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 -- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa84006da40 msgr2=0x7fa840156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa84006da40 0x7fa840156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa83ffff640 1 -- 172.21.15.26:0/2868434394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840157f90 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.132+0000 7fa844c9e640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa84006da40 0x7fa840156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.133+0000 7fa83ffff640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8400c6780 0x7fa8401eed70 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.133+0000 7fa83d7fa640 1 -- 172.21.15.26:0/2868434394 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c040070 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.133+0000 7fa83d7fa640 1 -- 172.21.15.26:0/2868434394 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c0399f0 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.133+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8400ae1a0 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.133+0000 7fa83d7fa640 1 -- 172.21.15.26:0/2868434394 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c035040 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.133+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8400aa070 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.134+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84004f330 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.134+0000 7fa83d7fa640 1 -- 172.21.15.26:0/2868434394 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa82c04b060 con 0x7fa8400c6780 2024-03-06T07:23:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.134+0000 7fa83d7fa640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa82406b4d0 0x7fa82406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.135+0000 7fa83d7fa640 1 -- 172.21.15.26:0/2868434394 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa82c0abb40 con 0x7fa8400c6780 2024-03-06T07:23:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.135+0000 7fa844c9e640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa82406b4d0 0x7fa82406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.135+0000 7fa844c9e640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa82406b4d0 0x7fa82406d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa834009560 tx=0x7fa834009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.139+0000 7fa83d7fa640 1 -- 172.21.15.26:0/2868434394 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa82c07a190 con 0x7fa8400c6780 2024-03-06T07:23:42.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.359+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8401535c0 con 0x7fa8400c6780 2024-03-06T07:23:42.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.360+0000 7fa83d7fa640 1 -- 172.21.15.26:0/2868434394 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4982 (secure 0 0 0) 0x7fa82c07e230 con 0x7fa8400c6780 2024-03-06T07:23:42.363 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:42.363 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1702158641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1702158641},{"type":"v1","addr":"172.21.15.117:6837","nonce":1702158641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:40.012005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6736":{"gid":6736,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6736,"qdb_cluster":[6736]},"id":7}]} 2024-03-06T07:23:42.363 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 82 2024-03-06T07:23:42.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa82406b4d0 msgr2=0x7fa82406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa82406b4d0 0x7fa82406d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa834009560 tx=0x7fa834009190 comp rx=0 tx=0).stop 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8400c6780 msgr2=0x7fa8401eed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8400c6780 0x7fa8401eed70 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 shutdown_connections 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa82406b4d0 0x7fa82406d990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8401592d0 0x7fa8400a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8400c6780 0x7fa8401eed70 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 --2- 172.21.15.26:0/2868434394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa84006da40 0x7fa840156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 >> 172.21.15.26:0/2868434394 conn(0x7fa8400aa630 msgr2=0x7fa8400c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 shutdown_connections 2024-03-06T07:23:42.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.362+0000 7fa846f02640 1 -- 172.21.15.26:0/2868434394 wait complete. 2024-03-06T07:23:42.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.475+0000 7fd7c69e9640 1 Processor -- start 2024-03-06T07:23:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.475+0000 7fd7c69e9640 1 -- start start 2024-03-06T07:23:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c69e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c01592d0 0x7fd7c01552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c69e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd7c00ac260 0x7fd7c0155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c69e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c0152d40 con 0x7fd7c0152960 2024-03-06T07:23:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c69e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c0066e30 con 0x7fd7c00ac260 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c69e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0155d50 con 0x7fd7c01592d0 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7bffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd7c0152960 0x7fd7c006dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38326/0 (socket says 172.21.15.26:38326) 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7bffff640 1 -- 172.21.15.26:0/3679250130 learned_addr learned my addr 172.21.15.26:0/3679250130 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c4f86640 1 --2- 172.21.15.26:0/3679250130 >> v2:172.21.15.117:3300/0 conn(0x7fd7c00ac260 0x7fd7c0155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7bf7fe640 1 --2- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c01592d0 0x7fd7c01552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7beffd640 1 -- 172.21.15.26:0/3679250130 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267064096 0 0) 0x7fd7c0152d40 con 0x7fd7c0152960 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7beffd640 1 -- 172.21.15.26:0/3679250130 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a8003520 con 0x7fd7c0152960 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c4f86640 1 -- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c01592d0 msgr2=0x7fd7c01552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c4f86640 1 --2- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c01592d0 0x7fd7c01552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c4f86640 1 -- 172.21.15.26:0/3679250130 >> v1:172.21.15.26:6789/0 conn(0x7fd7c0152960 legacy=0x7fd7c006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7c4f86640 1 -- 172.21.15.26:0/3679250130 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00c1b50 con 0x7fd7c00ac260 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.476+0000 7fd7bf7fe640 1 --2- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c01592d0 0x7fd7c01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.477+0000 7fd7c4f86640 1 --2- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 0x7fd7c0155810 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5644cad42684b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.477+0000 7fd7beffd640 1 -- 172.21.15.26:0/3679250130 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8039070 con 0x7fd7c00ac260 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.477+0000 7fd7beffd640 1 -- 172.21.15.26:0/3679250130 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b802cc50 con 0x7fd7c00ac260 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.477+0000 7fd7beffd640 1 -- 172.21.15.26:0/3679250130 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8034440 con 0x7fd7c00ac260 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.477+0000 7fd7c69e9640 1 -- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 msgr2=0x7fd7c0155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.477+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 0x7fd7c0155810 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b802bea0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.478+0000 7fd7c69e9640 1 -- 172.21.15.26:0/3679250130 shutdown_connections 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.478+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 0x7fd7c0155810 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.478+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/3679250130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c01592d0 0x7fd7c01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.478+0000 7fd7c69e9640 1 -- 172.21.15.26:0/3679250130 >> 172.21.15.26:0/3679250130 conn(0x7fd7c00c70b0 msgr2=0x7fd7c00c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.478+0000 7fd7c69e9640 1 -- 172.21.15.26:0/3679250130 shutdown_connections 2024-03-06T07:23:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.478+0000 7fd7c69e9640 1 -- 172.21.15.26:0/3679250130 wait complete. 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 Processor -- start 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 -- start start 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 0x7fd7c014f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c0152960 0x7fd7c0157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7c01df190 0x7fd7c014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0159760 con 0x7fd7c01df190 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00672c0 con 0x7fd7c00ac260 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.479+0000 7fd7c69e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c006e3a0 con 0x7fd7c0152960 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7c01df190 0x7fd7c014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7c01df190 0x7fd7c014e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57442/0 (socket says 172.21.15.26:57442) 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 -- 172.21.15.26:0/1295167453 learned_addr learned my addr 172.21.15.26:0/1295167453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7bffff640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 0x7fd7c014f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7bf7fe640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c0152960 0x7fd7c0157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 -- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c0152960 msgr2=0x7fd7c0157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c0152960 0x7fd7c0157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 -- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 msgr2=0x7fd7c014f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 0x7fd7c014f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 -- 172.21.15.26:0/1295167453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c014f050 con 0x7fd7c01df190 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c4f86640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7c01df190 0x7fd7c014e510 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7bcff9640 1 -- 172.21.15.26:0/1295167453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8039050 con 0x7fd7c01df190 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7bcff9640 1 -- 172.21.15.26:0/1295167453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8045070 con 0x7fd7c01df190 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c01eeec0 con 0x7fd7c01df190 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7bcff9640 1 -- 172.21.15.26:0/1295167453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8009450 con 0x7fd7c01df190 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.480+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c0158360 con 0x7fd7c01df190 2024-03-06T07:23:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.481+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c004f330 con 0x7fd7c01df190 2024-03-06T07:23:42.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.482+0000 7fd7bcff9640 1 -- 172.21.15.26:0/1295167453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7b803e610 con 0x7fd7c01df190 2024-03-06T07:23:42.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.482+0000 7fd7bcff9640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7ac06b4d0 0x7fd7ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.482+0000 7fd7bcff9640 1 -- 172.21.15.26:0/1295167453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd7b80acc90 con 0x7fd7c01df190 2024-03-06T07:23:42.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.482+0000 7fd7bffff640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7ac06b4d0 0x7fd7ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.483+0000 7fd7bffff640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7ac06b4d0 0x7fd7ac06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005880 tx=0x7fd7b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.486+0000 7fd7bcff9640 1 -- 172.21.15.26:0/1295167453 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd7b807b430 con 0x7fd7c01df190 2024-03-06T07:23:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.667+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7c0156270 con 0x7fd7c01df190 2024-03-06T07:23:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.669+0000 7fd7bcff9640 1 -- 172.21.15.26:0/1295167453 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17016 (secure 0 0 0) 0x7fd7b807f4d0 con 0x7fd7c01df190 2024-03-06T07:23:42.671 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:42.672 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:37.000731+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:22.873048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.559370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/1940022472":"2024-03-07T07:23:37.000697+0000","172.21.15.117:6835/1940022472":"2024-03-07T07:23:37.000697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7ac06b4d0 msgr2=0x7fd7ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7ac06b4d0 0x7fd7ac06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005880 tx=0x7fd7b4009040 comp rx=0 tx=0).stop 2024-03-06T07:23:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7c01df190 msgr2=0x7fd7c014e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7c01df190 0x7fd7c014e510 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b8004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 shutdown_connections 2024-03-06T07:23:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7ac06b4d0 0x7fd7ac06d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7c01df190 0x7fd7c014e510 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c0152960 0x7fd7c0157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 --2- 172.21.15.26:0/1295167453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c00ac260 0x7fd7c014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.670+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 >> 172.21.15.26:0/1295167453 conn(0x7fd7c00c70b0 msgr2=0x7fd7c006dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.671+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 shutdown_connections 2024-03-06T07:23:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.671+0000 7fd7c69e9640 1 -- 172.21.15.26:0/1295167453 wait complete. 2024-03-06T07:23:42.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:23:22.873048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:23:23.559370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:42.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:42.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e937a640 1 Processor -- start 2024-03-06T07:23:42.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e937a640 1 -- start start 2024-03-06T07:23:42.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e937a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31e40c26c0 0x7f31e4155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e937a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31e40c2aa0 0x7f31e406bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e937a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e406c110 con 0x7f31e40c0470 2024-03-06T07:23:42.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e937a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e406b740 con 0x7f31e40c26c0 2024-03-06T07:23:42.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e937a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e40670c0 con 0x7f31e40c2aa0 2024-03-06T07:23:42.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.786+0000 7f31e2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31e40c26c0 0x7f31e4155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.787+0000 7f31e2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31e40c26c0 0x7f31e4155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57452/0 (socket says 172.21.15.26:57452) 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.787+0000 7f31e2ffd640 1 -- 172.21.15.26:0/255554144 learned_addr learned my addr 172.21.15.26:0/255554144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.787+0000 7f31e27fc640 1 --2- 172.21.15.26:0/255554144 >> v2:172.21.15.117:3300/0 conn(0x7f31e40c2aa0 0x7f31e406bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.787+0000 7f31e2ffd640 1 -- 172.21.15.26:0/255554144 >> v2:172.21.15.117:3300/0 conn(0x7f31e40c2aa0 msgr2=0x7f31e406bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.787+0000 7f31e2ffd640 1 --2- 172.21.15.26:0/255554144 >> v2:172.21.15.117:3300/0 conn(0x7f31e40c2aa0 0x7f31e406bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.787+0000 7f31e2ffd640 1 -- 172.21.15.26:0/255554144 >> v1:172.21.15.117:6789/0 conn(0x7f31e40c0470 legacy=0x7f31e4152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.787+0000 7f31e2ffd640 1 -- 172.21.15.26:0/255554144 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e406b0f0 con 0x7f31e40c26c0 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e2ffd640 1 --2- 172.21.15.26:0/255554144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4155bd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f31e406f590 tx=0x7f31d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2301b269fd05730 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e1ffb640 1 -- 172.21.15.26:0/255554144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d402e070 con 0x7f31e40c26c0 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e1ffb640 1 -- 172.21.15.26:0/255554144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d4037070 con 0x7f31e40c26c0 2024-03-06T07:23:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e1ffb640 1 -- 172.21.15.26:0/255554144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d4031a00 con 0x7f31e40c26c0 2024-03-06T07:23:42.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 -- 172.21.15.26:0/255554144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 msgr2=0x7f31e4155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 --2- 172.21.15.26:0/255554144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4155bd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f31e406f590 tx=0x7f31d402b880 comp rx=0 tx=0).stop 2024-03-06T07:23:42.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 -- 172.21.15.26:0/255554144 shutdown_connections 2024-03-06T07:23:42.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 --2- 172.21.15.26:0/255554144 >> v2:172.21.15.117:3300/0 conn(0x7f31e40c2aa0 0x7f31e406bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 --2- 172.21.15.26:0/255554144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4155bd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 -- 172.21.15.26:0/255554144 >> 172.21.15.26:0/255554144 conn(0x7f31e40c3b40 msgr2=0x7f31e40c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 -- 172.21.15.26:0/255554144 shutdown_connections 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.788+0000 7f31e937a640 1 -- 172.21.15.26:0/255554144 wait complete. 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.789+0000 7f31e937a640 1 Processor -- start 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e937a640 1 -- start start 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e937a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e40c0470 0x7f31e41e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e937a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e937a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e40c2aa0 0x7f31e40ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e937a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c1f30 con 0x7f31e40c26c0 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e937a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c0900 con 0x7f31e40c0470 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e937a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c20b0 con 0x7f31e40c2aa0 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57460/0 (socket says 172.21.15.26:57460) 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e27fc640 1 -- 172.21.15.26:0/1057693102 learned_addr learned my addr 172.21.15.26:0/1057693102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e37fe640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e40c2aa0 0x7f31e40ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e2ffd640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e40c0470 0x7f31e41e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e27fc640 1 -- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e40c2aa0 msgr2=0x7f31e40ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.790+0000 7f31e27fc640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e40c2aa0 0x7f31e40ab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31e27fc640 1 -- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e40c0470 msgr2=0x7f31e41e1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31e27fc640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e40c0470 0x7f31e41e1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31e27fc640 1 -- 172.21.15.26:0/1057693102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40aa200 con 0x7f31e40c26c0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31e2ffd640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e40c0470 0x7f31e41e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31e27fc640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4152f10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f31d800e8a0 tx=0x7f31d800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31cb7fe640 1 -- 172.21.15.26:0/1057693102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8017070 con 0x7f31e40c26c0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31cb7fe640 1 -- 172.21.15.26:0/1057693102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d800c940 con 0x7f31e40c26c0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e40aa4e0 con 0x7f31e40c26c0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31cb7fe640 1 -- 172.21.15.26:0/1057693102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800cd30 con 0x7f31e40c26c0 2024-03-06T07:23:42.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.791+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e40aaaa0 con 0x7f31e40c26c0 2024-03-06T07:23:42.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.792+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e404f330 con 0x7f31e40c26c0 2024-03-06T07:23:42.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.792+0000 7f31cb7fe640 1 -- 172.21.15.26:0/1057693102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31d801a070 con 0x7f31e40c26c0 2024-03-06T07:23:42.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.793+0000 7f31cb7fe640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f31d006b500 0x7f31d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.793+0000 7f31cb7fe640 1 -- 172.21.15.26:0/1057693102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f31d808a280 con 0x7f31e40c26c0 2024-03-06T07:23:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.793+0000 7f31e2ffd640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f31d006b500 0x7f31d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.793+0000 7f31e2ffd640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f31d006b500 0x7f31d006d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f31d4036000 tx=0x7f31d403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.797+0000 7f31cb7fe640 1 -- 172.21.15.26:0/1057693102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f31d8058850 con 0x7f31e40c26c0 2024-03-06T07:23:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.978+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31e406bd70 con 0x7f31e40c26c0 2024-03-06T07:23:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.978+0000 7f31cb7fe640 1 -- 172.21.15.26:0/1057693102 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31d805c8f0 con 0x7f31e40c26c0 2024-03-06T07:23:42.981 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:23:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f31d006b500 msgr2=0x7f31d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f31d006b500 0x7f31d006d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f31d4036000 tx=0x7f31d403d040 comp rx=0 tx=0).stop 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 msgr2=0x7f31e4152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4152f10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f31d800e8a0 tx=0x7f31d800ed60 comp rx=0 tx=0).stop 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 shutdown_connections 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f31d006b500 0x7f31d006d9c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e40c2aa0 0x7f31e40ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31e40c26c0 0x7f31e4152f10 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 --2- 172.21.15.26:0/1057693102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e40c0470 0x7f31e41e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 >> 172.21.15.26:0/1057693102 conn(0x7f31e40c3b40 msgr2=0x7f31e406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 shutdown_connections 2024-03-06T07:23:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:42.981+0000 7f31e937a640 1 -- 172.21.15.26:0/1057693102 wait complete. 2024-03-06T07:23:42.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:23:43.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.096+0000 7f6205c2a640 1 Processor -- start 2024-03-06T07:23:43.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.096+0000 7f6205c2a640 1 -- start start 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f6205c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620006b3a0 0x7f6200158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f6205c2a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6200159160 0x7f620015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f6205c2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620006b800 con 0x7f620006afc0 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f6205c2a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6200158790 con 0x7f6200159160 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f6205c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62000c5840 con 0x7f620006b3a0 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61ff7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f620006afc0 0x7f62000c2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38334/0 (socket says 172.21.15.26:38334) 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61ff7fe640 1 -- 172.21.15.26:0/2361747411 learned_addr learned my addr 172.21.15.26:0/2361747411 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61feffd640 1 --2- 172.21.15.26:0/2361747411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620006b3a0 0x7f6200158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61fffff640 1 --2- 172.21.15.26:0/2361747411 >> v2:172.21.15.117:3300/0 conn(0x7f6200159160 0x7f620015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61fe7fc640 1 -- 172.21.15.26:0/2361747411 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818503188 0 0) 0x7f620006b800 con 0x7f620006afc0 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61fe7fc640 1 -- 172.21.15.26:0/2361747411 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e8003520 con 0x7f620006afc0 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61feffd640 1 -- 172.21.15.26:0/2361747411 >> v2:172.21.15.117:3300/0 conn(0x7f6200159160 msgr2=0x7f620015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61feffd640 1 --2- 172.21.15.26:0/2361747411 >> v2:172.21.15.117:3300/0 conn(0x7f6200159160 0x7f620015fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61feffd640 1 -- 172.21.15.26:0/2361747411 >> v1:172.21.15.26:6789/0 conn(0x7f620006afc0 legacy=0x7f62000c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61feffd640 1 -- 172.21.15.26:0/2361747411 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000c5b20 con 0x7f620006b3a0 2024-03-06T07:23:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.097+0000 7f61fffff640 1 --2- 172.21.15.26:0/2361747411 >> v2:172.21.15.117:3300/0 conn(0x7f6200159160 0x7f620015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.098+0000 7f61feffd640 1 --2- 172.21.15.26:0/2361747411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620006b3a0 0x7f6200158c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f61f40097a0 tx=0x7f61f402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8caf0be83e0d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.098+0000 7f61fe7fc640 1 -- 172.21.15.26:0/2361747411 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40315a0 con 0x7f620006b3a0 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.098+0000 7f61fe7fc640 1 -- 172.21.15.26:0/2361747411 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4037070 con 0x7f620006b3a0 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.098+0000 7f61fe7fc640 1 -- 172.21.15.26:0/2361747411 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4031940 con 0x7f620006b3a0 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 -- 172.21.15.26:0/2361747411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620006b3a0 msgr2=0x7f6200158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 --2- 172.21.15.26:0/2361747411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620006b3a0 0x7f6200158c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f61f40097a0 tx=0x7f61f402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 -- 172.21.15.26:0/2361747411 shutdown_connections 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 --2- 172.21.15.26:0/2361747411 >> v2:172.21.15.117:3300/0 conn(0x7f6200159160 0x7f620015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 --2- 172.21.15.26:0/2361747411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620006b3a0 0x7f6200158c20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 -- 172.21.15.26:0/2361747411 >> 172.21.15.26:0/2361747411 conn(0x7f62000c4220 msgr2=0x7f62000c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 -- 172.21.15.26:0/2361747411 shutdown_connections 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.099+0000 7f6205c2a640 1 -- 172.21.15.26:0/2361747411 wait complete. 2024-03-06T07:23:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.100+0000 7f6205c2a640 1 Processor -- start 2024-03-06T07:23:43.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.100+0000 7f6205c2a640 1 -- start start 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f6205c2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620006afc0 0x7f6200157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f6205c2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f620006b3a0 0x7f6200156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f6205c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6200159160 0x7f6200156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f6205c2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62001595f0 con 0x7f620006b3a0 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f6205c2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000c60e0 con 0x7f620006afc0 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f6205c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6200155220 con 0x7f6200159160 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f620006b3a0 0x7f6200156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f620006b3a0 0x7f6200156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57464/0 (socket says 172.21.15.26:57464) 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 -- 172.21.15.26:0/1880953740 learned_addr learned my addr 172.21.15.26:0/1880953740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61ff7fe640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620006afc0 0x7f6200157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61fffff640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6200159160 0x7f6200156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 -- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6200159160 msgr2=0x7f6200156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6200159160 0x7f6200156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 -- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620006afc0 msgr2=0x7f6200157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620006afc0 0x7f6200157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 -- 172.21.15.26:0/1880953740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000a8a50 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.101+0000 7f61feffd640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f620006b3a0 0x7f6200156a20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f61f402b880 tx=0x7f61f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.102+0000 7f61e7fff640 1 -- 172.21.15.26:0/1880953740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4032630 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.102+0000 7f61e7fff640 1 -- 172.21.15.26:0/1880953740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f40327f0 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.102+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62000a8d30 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.102+0000 7f61e7fff640 1 -- 172.21.15.26:0/1880953740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4032be0 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.102+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62001582c0 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.103+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620004f330 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.103+0000 7f61e7fff640 1 -- 172.21.15.26:0/1880953740 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61f403b3f0 con 0x7f620006b3a0 2024-03-06T07:23:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.104+0000 7f61e7fff640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61ec06b4d0 0x7f61ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.104+0000 7f61e7fff640 1 -- 172.21.15.26:0/1880953740 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f61f402f030 con 0x7f620006b3a0 2024-03-06T07:23:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.104+0000 7f61ff7fe640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61ec06b4d0 0x7f61ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.104+0000 7f61ff7fe640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61ec06b4d0 0x7f61ec06d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005880 tx=0x7f61f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.108+0000 7f61e7fff640 1 -- 172.21.15.26:0/1880953740 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f61f4040070 con 0x7f620006b3a0 2024-03-06T07:23:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.291+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62000c6730 con 0x7f620006b3a0 2024-03-06T07:23:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.292+0000 7f61e7fff640 1 -- 172.21.15.26:0/1880953740 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61f407ac70 con 0x7f620006b3a0 2024-03-06T07:23:43.294 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:23:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.293+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61ec06b4d0 msgr2=0x7f61ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.293+0000 7f6205c2a640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61ec06b4d0 0x7f61ec06d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005880 tx=0x7f61f0009040 comp rx=0 tx=0).stop 2024-03-06T07:23:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.293+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f620006b3a0 msgr2=0x7f6200156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.293+0000 7f6205c2a640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f620006b3a0 0x7f6200156a20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f61f402b880 tx=0x7f61f40046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 shutdown_connections 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f61ec06b4d0 0x7f61ec06d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6200159160 0x7f6200156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f620006b3a0 0x7f6200156a20 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 --2- 172.21.15.26:0/1880953740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620006afc0 0x7f6200157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 >> 172.21.15.26:0/1880953740 conn(0x7f62000c4220 msgr2=0x7f6200153090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 shutdown_connections 2024-03-06T07:23:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.294+0000 7f6205c2a640 1 -- 172.21.15.26:0/1880953740 wait complete. 2024-03-06T07:23:43.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:23:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.409+0000 7f5794f81640 1 Processor -- start 2024-03-06T07:23:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.409+0000 7f5794f81640 1 -- start start 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f5794f81640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f579006afc0 0x7f579006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f5794f81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c2120 0x7f5790158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f5794f81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57900c5840 con 0x7f57900c2580 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f5794f81640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5790158790 con 0x7f579006afc0 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f5794f81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900c5b20 con 0x7f57900c2120 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f57900c2580 0x7f5790159160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38342/0 (socket says 172.21.15.26:38342) 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578ed9d640 1 -- 172.21.15.26:0/2969190114 learned_addr learned my addr 172.21.15.26:0/2969190114 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578e59c640 1 --2- 172.21.15.26:0/2969190114 >> v2:172.21.15.117:3300/0 conn(0x7f579006afc0 0x7f579006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f5787fff640 1 --2- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c2120 0x7f5790158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578dd9b640 1 -- 172.21.15.26:0/2969190114 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217040594 0 0) 0x7f57900c5840 con 0x7f57900c2580 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578dd9b640 1 -- 172.21.15.26:0/2969190114 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f577c003520 con 0x7f57900c2580 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578e59c640 1 -- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c2120 msgr2=0x7f5790158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578e59c640 1 --2- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c2120 0x7f5790158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578e59c640 1 -- 172.21.15.26:0/2969190114 >> v1:172.21.15.26:6789/0 conn(0x7f57900c2580 legacy=0x7f5790159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.410+0000 7f578e59c640 1 -- 172.21.15.26:0/2969190114 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900c5e00 con 0x7f579006afc0 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.411+0000 7f578e59c640 1 --2- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579006afc0 0x7f579006b3a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f57800097a0 tx=0x7f578002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341dfa3e67b90a53 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.411+0000 7f578dd9b640 1 -- 172.21.15.26:0/2969190114 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800315a0 con 0x7f579006afc0 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.411+0000 7f578dd9b640 1 -- 172.21.15.26:0/2969190114 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780037070 con 0x7f579006afc0 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.411+0000 7f578dd9b640 1 -- 172.21.15.26:0/2969190114 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780031940 con 0x7f579006afc0 2024-03-06T07:23:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 -- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579006afc0 msgr2=0x7f579006b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 --2- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579006afc0 0x7f579006b3a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f57800097a0 tx=0x7f578002b840 comp rx=0 tx=0).stop 2024-03-06T07:23:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 -- 172.21.15.26:0/2969190114 shutdown_connections 2024-03-06T07:23:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 --2- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c2120 0x7f5790158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 --2- 172.21.15.26:0/2969190114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579006afc0 0x7f579006b3a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 -- 172.21.15.26:0/2969190114 >> 172.21.15.26:0/2969190114 conn(0x7f57900c4220 msgr2=0x7f57900c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 -- 172.21.15.26:0/2969190114 shutdown_connections 2024-03-06T07:23:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.412+0000 7f5794f81640 1 -- 172.21.15.26:0/2969190114 wait complete. 2024-03-06T07:23:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.413+0000 7f5794f81640 1 Processor -- start 2024-03-06T07:23:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.413+0000 7f5794f81640 1 -- start start 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5794f81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f579006afc0 0x7f57901501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5794f81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57900c2120 0x7f5790072730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5794f81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57900c2580 0x7f57900aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5794f81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900c2b30 con 0x7f57900c2580 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5794f81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790155220 con 0x7f57900c2120 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5794f81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900c2cb0 con 0x7f579006afc0 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57900c2580 0x7f57900aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57900c2580 0x7f57900aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57476/0 (socket says 172.21.15.26:57476) 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 -- 172.21.15.26:0/458800238 learned_addr learned my addr 172.21.15.26:0/458800238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5787fff640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57900c2120 0x7f5790072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578e59c640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f579006afc0 0x7f57901501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 -- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f579006afc0 msgr2=0x7f57901501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f579006afc0 0x7f57901501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 -- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57900c2120 msgr2=0x7f5790072730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57900c2120 0x7f5790072730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 -- 172.21.15.26:0/458800238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57901eecc0 con 0x7f57900c2580 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5787fff640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57900c2120 0x7f5790072730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f578ed9d640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57900c2580 0x7f57900aaa10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f578800caa0 tx=0x7f578800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5786ffd640 1 -- 172.21.15.26:0/458800238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788017070 con 0x7f57900c2580 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5786ffd640 1 -- 172.21.15.26:0/458800238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800ac00 con 0x7f57900c2580 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.414+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579006edd0 con 0x7f57900c2580 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.415+0000 7f5786ffd640 1 -- 172.21.15.26:0/458800238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788012400 con 0x7f57900c2580 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.415+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579014fe50 con 0x7f57900c2580 2024-03-06T07:23:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.415+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579004f330 con 0x7f57900c2580 2024-03-06T07:23:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.416+0000 7f5786ffd640 1 -- 172.21.15.26:0/458800238 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f578801a070 con 0x7f57900c2580 2024-03-06T07:23:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.416+0000 7f5786ffd640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f577006b4d0 0x7f577006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.416+0000 7f5786ffd640 1 -- 172.21.15.26:0/458800238 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f578808a1f0 con 0x7f57900c2580 2024-03-06T07:23:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.416+0000 7f578e59c640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f577006b4d0 0x7f577006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.417+0000 7f578e59c640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f577006b4d0 0x7f577006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5780036000 tx=0x7f578003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.420+0000 7f5786ffd640 1 -- 172.21.15.26:0/458800238 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57880587b0 con 0x7f57900c2580 2024-03-06T07:23:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.600+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57900c6730 con 0x7f57900c2580 2024-03-06T07:23:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.600+0000 7f5786ffd640 1 -- 172.21.15.26:0/458800238 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f578805c850 con 0x7f57900c2580 2024-03-06T07:23:43.603 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f577006b4d0 msgr2=0x7f577006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f577006b4d0 0x7f577006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5780036000 tx=0x7f578003d040 comp rx=0 tx=0).stop 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57900c2580 msgr2=0x7f57900aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57900c2580 0x7f57900aaa10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f578800caa0 tx=0x7f578800cf60 comp rx=0 tx=0).stop 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 shutdown_connections 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f577006b4d0 0x7f577006d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57900c2580 0x7f57900aaa10 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57900c2120 0x7f5790072730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 --2- 172.21.15.26:0/458800238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f579006afc0 0x7f57901501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 >> 172.21.15.26:0/458800238 conn(0x7f57900c4220 msgr2=0x7f5790153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 shutdown_connections 2024-03-06T07:23:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.602+0000 7f5794f81640 1 -- 172.21.15.26:0/458800238 wait complete. 2024-03-06T07:23:43.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.714+0000 7fa863bbc640 1 Processor -- start 2024-03-06T07:23:43.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.714+0000 7fa863bbc640 1 -- start start 2024-03-06T07:23:43.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa863bbc640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa85c152920 0x7fa85c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa863bbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa863bbc640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c156180 con 0x7fa85c159460 2024-03-06T07:23:43.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa863bbc640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85c1587b0 con 0x7fa85c152920 2024-03-06T07:23:43.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa863bbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0c7760 con 0x7fa85c0c8a00 2024-03-06T07:23:43.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861157640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861958640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa85c152920 0x7fa85c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861157640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:38802/0 (socket says 172.21.15.26:38802) 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa862159640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa85c159460 0x7fa85c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:41342/0 (socket says 172.21.15.26:41342) 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861157640 1 -- 172.21.15.26:0/2789772168 learned_addr learned my addr 172.21.15.26:0/2789772168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861157640 1 -- 172.21.15.26:0/2789772168 >> v2:172.21.15.117:3300/0 conn(0x7fa85c152920 msgr2=0x7fa85c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861157640 1 --2- 172.21.15.26:0/2789772168 >> v2:172.21.15.117:3300/0 conn(0x7fa85c152920 0x7fa85c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861157640 1 -- 172.21.15.26:0/2789772168 >> v1:172.21.15.117:6789/0 conn(0x7fa85c159460 legacy=0x7fa85c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa860956640 1 -- 172.21.15.26:0/2789772168 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046480880 0 0) 0x7fa854001110 con 0x7fa85c159460 2024-03-06T07:23:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.715+0000 7fa861157640 1 -- 172.21.15.26:0/2789772168 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0c7a40 con 0x7fa85c0c8a00 2024-03-06T07:23:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.716+0000 7fa861157640 1 --2- 172.21.15.26:0/2789772168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c158f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa850009870 tx=0x7fa85002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db380f670e3d80f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.716+0000 7fa860956640 1 -- 172.21.15.26:0/2789772168 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850035020 con 0x7fa85c0c8a00 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.716+0000 7fa860956640 1 -- 172.21.15.26:0/2789772168 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa850031640 con 0x7fa85c0c8a00 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.716+0000 7fa860956640 1 -- 172.21.15.26:0/2789772168 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85002d040 con 0x7fa85c0c8a00 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 -- 172.21.15.26:0/2789772168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 msgr2=0x7fa85c158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2789772168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c158f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa850009870 tx=0x7fa85002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 -- 172.21.15.26:0/2789772168 shutdown_connections 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2789772168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c158f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2789772168 >> v2:172.21.15.117:3300/0 conn(0x7fa85c152920 0x7fa85c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 -- 172.21.15.26:0/2789772168 >> 172.21.15.26:0/2789772168 conn(0x7fa85c0ac560 msgr2=0x7fa85c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 -- 172.21.15.26:0/2789772168 shutdown_connections 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.717+0000 7fa863bbc640 1 -- 172.21.15.26:0/2789772168 wait complete. 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.718+0000 7fa863bbc640 1 Processor -- start 2024-03-06T07:23:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.718+0000 7fa863bbc640 1 -- start start 2024-03-06T07:23:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa863bbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c14b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa863bbc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c152920 0x7fa85c0c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa863bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa85c1ee4c0 0x7fa85c0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa863bbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c06d030 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa863bbc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c158a00 con 0x7fa85c152920 2024-03-06T07:23:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa863bbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0c5a40 con 0x7fa85c0c8a00 2024-03-06T07:23:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa85c1ee4c0 0x7fa85c0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa85c1ee4c0 0x7fa85c0c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57482/0 (socket says 172.21.15.26:57482) 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa861958640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c14b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 -- 172.21.15.26:0/2880131465 learned_addr learned my addr 172.21.15.26:0/2880131465 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa861157640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c152920 0x7fa85c0c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 -- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 msgr2=0x7fa85c14b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c14b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 -- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c152920 msgr2=0x7fa85c0c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c152920 0x7fa85c0c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 -- 172.21.15.26:0/2880131465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0c23d0 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa861157640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c152920 0x7fa85c0c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa862159640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa85c1ee4c0 0x7fa85c0c4530 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa85400c410 tx=0x7fa85400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa85a7fc640 1 -- 172.21.15.26:0/2880131465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540074e0 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa85a7fc640 1 -- 172.21.15.26:0/2880131465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8540047e0 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.719+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c1ec610 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.720+0000 7fa85a7fc640 1 -- 172.21.15.26:0/2880131465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854004bd0 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.720+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c1ecbd0 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.721+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c0cb830 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.721+0000 7fa85a7fc640 1 -- 172.21.15.26:0/2880131465 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa854011070 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.721+0000 7fa85a7fc640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa84806b500 0x7fa84806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.721+0000 7fa85a7fc640 1 -- 172.21.15.26:0/2880131465 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa854089ca0 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.722+0000 7fa861958640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa84806b500 0x7fa84806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.722+0000 7fa861958640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa84806b500 0x7fa84806d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0113e0 tx=0x7fa84c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.726+0000 7fa85a7fc640 1 -- 172.21.15.26:0/2880131465 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa85c0cb830 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.945+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa85c075300 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.945+0000 7fa85a7fc640 1 -- 172.21.15.26:0/2880131465 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4982 (secure 0 0 0) 0x7fa85c075300 con 0x7fa85c1ee4c0 2024-03-06T07:23:43.948 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:43.948 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1702158641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1702158641},{"type":"v1","addr":"172.21.15.117:6837","nonce":1702158641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:40.012005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6736":{"gid":6736,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6736,"qdb_cluster":[6736]},"id":7}]} 2024-03-06T07:23:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 82 2024-03-06T07:23:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.947+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa84806b500 msgr2=0x7fa84806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.947+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa84806b500 0x7fa84806d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0113e0 tx=0x7fa84c00b040 comp rx=0 tx=0).stop 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa85c1ee4c0 msgr2=0x7fa85c0c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa85c1ee4c0 0x7fa85c0c4530 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa85400c410 tx=0x7fa85400c8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 shutdown_connections 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa84806b500 0x7fa84806d9c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa85c1ee4c0 0x7fa85c0c4530 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c152920 0x7fa85c0c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 --2- 172.21.15.26:0/2880131465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c0c8a00 0x7fa85c14b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 >> 172.21.15.26:0/2880131465 conn(0x7fa85c0ac560 msgr2=0x7fa85c14f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 shutdown_connections 2024-03-06T07:23:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:43.948+0000 7fa863bbc640 1 -- 172.21.15.26:0/2880131465 wait complete. 2024-03-06T07:23:43.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.060+0000 7f0a59aeb640 1 Processor -- start 2024-03-06T07:23:44.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.060+0000 7f0a59aeb640 1 -- start start 2024-03-06T07:23:44.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.060+0000 7f0a59aeb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0a540c6780 0x7f0a54158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.060+0000 7f0a59aeb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a541592d0 0x7f0a540c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.060+0000 7f0a59aeb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a54072630 con 0x7f0a5406da40 2024-03-06T07:23:44.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.060+0000 7f0a59aeb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5406de20 con 0x7f0a541592d0 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.060+0000 7f0a59aeb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a54072910 con 0x7f0a540c6780 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a53fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a541592d0 0x7f0a540c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a537fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0a5406da40 0x7f0a54155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38352/0 (socket says 172.21.15.26:38352) 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a537fe640 1 -- 172.21.15.26:0/1682015437 learned_addr learned my addr 172.21.15.26:0/1682015437 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a52ffd640 1 --2- 172.21.15.26:0/1682015437 >> v2:172.21.15.117:3300/0 conn(0x7f0a540c6780 0x7f0a54158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a53fff640 1 -- 172.21.15.26:0/1682015437 >> v2:172.21.15.117:3300/0 conn(0x7f0a540c6780 msgr2=0x7f0a54158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a53fff640 1 --2- 172.21.15.26:0/1682015437 >> v2:172.21.15.117:3300/0 conn(0x7f0a540c6780 0x7f0a54158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a53fff640 1 -- 172.21.15.26:0/1682015437 >> v1:172.21.15.26:6789/0 conn(0x7f0a5406da40 legacy=0x7f0a54155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a53fff640 1 -- 172.21.15.26:0/1682015437 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a54072bf0 con 0x7f0a541592d0 2024-03-06T07:23:44.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a527fc640 1 -- 172.21.15.26:0/1682015437 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452494113 0 0) 0x7f0a54072630 con 0x7f0a5406da40 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.061+0000 7f0a52ffd640 1 --2- 172.21.15.26:0/1682015437 >> v2:172.21.15.117:3300/0 conn(0x7f0a540c6780 0x7f0a54158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a53fff640 1 --2- 172.21.15.26:0/1682015437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a541592d0 0x7f0a540c3280 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0a540c7120 tx=0x7f0a4c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f80d793099d062 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a527fc640 1 -- 172.21.15.26:0/1682015437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c039070 con 0x7f0a541592d0 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a527fc640 1 -- 172.21.15.26:0/1682015437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c02cbf0 con 0x7f0a541592d0 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a527fc640 1 -- 172.21.15.26:0/1682015437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c034440 con 0x7f0a541592d0 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 -- 172.21.15.26:0/1682015437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a541592d0 msgr2=0x7f0a540c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/1682015437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a541592d0 0x7f0a540c3280 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0a540c7120 tx=0x7f0a4c02be40 comp rx=0 tx=0).stop 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 -- 172.21.15.26:0/1682015437 shutdown_connections 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/1682015437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a541592d0 0x7f0a540c3280 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/1682015437 >> v2:172.21.15.117:3300/0 conn(0x7f0a540c6780 0x7f0a54158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 -- 172.21.15.26:0/1682015437 >> 172.21.15.26:0/1682015437 conn(0x7f0a540aa630 msgr2=0x7f0a540aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 -- 172.21.15.26:0/1682015437 shutdown_connections 2024-03-06T07:23:44.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.062+0000 7f0a59aeb640 1 -- 172.21.15.26:0/1682015437 wait complete. 2024-03-06T07:23:44.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.063+0000 7f0a59aeb640 1 Processor -- start 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a59aeb640 1 -- start start 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a59aeb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a5406da40 0x7f0a54157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a59aeb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a540c6780 0x7f0a540ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a59aeb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a541592d0 0x7f0a5414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a59aeb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540c3940 con 0x7f0a540c6780 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a59aeb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540c37c0 con 0x7f0a541592d0 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a59aeb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540c3ac0 con 0x7f0a5406da40 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a540c6780 0x7f0a540ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a540c6780 0x7f0a540ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57494/0 (socket says 172.21.15.26:57494) 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.064+0000 7f0a52ffd640 1 -- 172.21.15.26:0/2028488677 learned_addr learned my addr 172.21.15.26:0/2028488677 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:44.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a537fe640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a5406da40 0x7f0a54157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a53fff640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a541592d0 0x7f0a5414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a52ffd640 1 -- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a5406da40 msgr2=0x7f0a54157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a52ffd640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a5406da40 0x7f0a54157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a52ffd640 1 -- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a541592d0 msgr2=0x7f0a5414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a52ffd640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a541592d0 0x7f0a5414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a52ffd640 1 -- 172.21.15.26:0/2028488677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a541ea860 con 0x7f0a540c6780 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a52ffd640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a540c6780 0x7f0a540ae410 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800ec10 tx=0x7f0a4800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a3bfff640 1 -- 172.21.15.26:0/2028488677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48017070 con 0x7f0a540c6780 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a3bfff640 1 -- 172.21.15.26:0/2028488677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4800cc30 con 0x7f0a540c6780 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a541eab40 con 0x7f0a540c6780 2024-03-06T07:23:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a3bfff640 1 -- 172.21.15.26:0/2028488677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48013470 con 0x7f0a540c6780 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.065+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a541eb100 con 0x7f0a540c6780 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.066+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5404f330 con 0x7f0a540c6780 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.067+0000 7f0a3bfff640 1 -- 172.21.15.26:0/2028488677 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a4801a070 con 0x7f0a540c6780 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.067+0000 7f0a3bfff640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a4006b4d0 0x7f0a4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.067+0000 7f0a3bfff640 1 -- 172.21.15.26:0/2028488677 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0a4808a550 con 0x7f0a540c6780 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.067+0000 7f0a537fe640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a4006b4d0 0x7f0a4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.067+0000 7f0a537fe640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a4006b4d0 0x7f0a4006d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a44007fd0 tx=0x7f0a44007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.071+0000 7f0a3bfff640 1 -- 172.21.15.26:0/2028488677 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0a48058c30 con 0x7f0a540c6780 2024-03-06T07:23:44.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.256+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a54073240 con 0x7f0a540c6780 2024-03-06T07:23:44.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.257+0000 7f0a3bfff640 1 -- 172.21.15.26:0/2028488677 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17016 (secure 0 0 0) 0x7f0a4805ccd0 con 0x7f0a540c6780 2024-03-06T07:23:44.260 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:44.260 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:37.000731+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:22.873048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.559370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/1940022472":"2024-03-07T07:23:37.000697+0000","172.21.15.117:6835/1940022472":"2024-03-07T07:23:37.000697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:44.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a4006b4d0 msgr2=0x7f0a4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a4006b4d0 0x7f0a4006d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a44007fd0 tx=0x7f0a44007de0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a540c6780 msgr2=0x7f0a540ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a540c6780 0x7f0a540ae410 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800ec10 tx=0x7f0a4800c520 comp rx=0 tx=0).stop 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 shutdown_connections 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a4006b4d0 0x7f0a4006d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a541592d0 0x7f0a5414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a540c6780 0x7f0a540ae410 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 --2- 172.21.15.26:0/2028488677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a5406da40 0x7f0a54157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 >> 172.21.15.26:0/2028488677 conn(0x7f0a540aa630 msgr2=0x7f0a540aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 shutdown_connections 2024-03-06T07:23:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.259+0000 7f0a59aeb640 1 -- 172.21.15.26:0/2028488677 wait complete. 2024-03-06T07:23:44.273 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:23:44.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:44.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f615ed7a640 1 Processor -- start 2024-03-06T07:23:44.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f615ed7a640 1 -- start start 2024-03-06T07:23:44.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f615ed7a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f615814f790 0x7f61581529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f615ed7a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61580c1aa0 0x7f61580c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f615ed7a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61580c28d0 con 0x7f61580c5c60 2024-03-06T07:23:44.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f615ed7a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61580c2680 con 0x7f61580c1aa0 2024-03-06T07:23:44.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f615ed7a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61580c2bb0 con 0x7f615814f790 2024-03-06T07:23:44.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f6157fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61580c1aa0 0x7f61580c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.372+0000 7f6157fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61580c1aa0 0x7f61580c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57510/0 (socket says 172.21.15.26:57510) 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.373+0000 7f6157fff640 1 -- 172.21.15.26:0/2065992172 learned_addr learned my addr 172.21.15.26:0/2065992172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.373+0000 7f615cb16640 1 --2- 172.21.15.26:0/2065992172 >> v2:172.21.15.117:3300/0 conn(0x7f615814f790 0x7f61581529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.373+0000 7f6157fff640 1 -- 172.21.15.26:0/2065992172 >> v2:172.21.15.117:3300/0 conn(0x7f615814f790 msgr2=0x7f61581529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.373+0000 7f6157fff640 1 --2- 172.21.15.26:0/2065992172 >> v2:172.21.15.117:3300/0 conn(0x7f615814f790 0x7f61581529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.373+0000 7f6157fff640 1 -- 172.21.15.26:0/2065992172 >> v1:172.21.15.117:6789/0 conn(0x7f61580c5c60 legacy=0x7f6158155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.373+0000 7f6157fff640 1 -- 172.21.15.26:0/2065992172 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580c2e90 con 0x7f61580c1aa0 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.373+0000 7f6157fff640 1 --2- 172.21.15.26:0/2065992172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61580c1aa0 0x7f61580c5720 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f61580c3170 tx=0x7f614c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98efe2d3766c340e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f61577fe640 1 -- 172.21.15.26:0/2065992172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c034070 con 0x7f61580c1aa0 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f61577fe640 1 -- 172.21.15.26:0/2065992172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c030860 con 0x7f61580c1aa0 2024-03-06T07:23:44.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f61577fe640 1 -- 172.21.15.26:0/2065992172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c030c30 con 0x7f61580c1aa0 2024-03-06T07:23:44.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 -- 172.21.15.26:0/2065992172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61580c1aa0 msgr2=0x7f61580c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 --2- 172.21.15.26:0/2065992172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61580c1aa0 0x7f61580c5720 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f61580c3170 tx=0x7f614c02b9a0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 -- 172.21.15.26:0/2065992172 shutdown_connections 2024-03-06T07:23:44.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 --2- 172.21.15.26:0/2065992172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61580c1aa0 0x7f61580c5720 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 --2- 172.21.15.26:0/2065992172 >> v2:172.21.15.117:3300/0 conn(0x7f615814f790 0x7f61581529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 -- 172.21.15.26:0/2065992172 >> 172.21.15.26:0/2065992172 conn(0x7f61580c6710 msgr2=0x7f61580c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 -- 172.21.15.26:0/2065992172 shutdown_connections 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.374+0000 7f615ed7a640 1 -- 172.21.15.26:0/2065992172 wait complete. 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.375+0000 7f615ed7a640 1 Processor -- start 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.375+0000 7f615ed7a640 1 -- start start 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615ed7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61580c1aa0 0x7f61581eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615ed7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61580c5c60 0x7f61580bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615ed7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f615814f790 0x7f61581eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615ed7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615806cc00 con 0x7f615814f790 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615ed7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61580c0d60 con 0x7f61580c5c60 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615ed7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615806cd80 con 0x7f61580c1aa0 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615d317640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f615814f790 0x7f61581eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615d317640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f615814f790 0x7f61581eaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57526/0 (socket says 172.21.15.26:57526) 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615d317640 1 -- 172.21.15.26:0/3291804304 learned_addr learned my addr 172.21.15.26:0/3291804304 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:44.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f6157fff640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61580c5c60 0x7f61580bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.376+0000 7f615cb16640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61580c1aa0 0x7f61581eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615d317640 1 -- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61580c1aa0 msgr2=0x7f61581eeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615d317640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61580c1aa0 0x7f61581eeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615d317640 1 -- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61580c5c60 msgr2=0x7f61580bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615d317640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61580c5c60 0x7f61580bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615d317640 1 -- 172.21.15.26:0/3291804304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158071fd0 con 0x7f615814f790 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615d317640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f615814f790 0x7f61581eaf40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6148001eb0 tx=0x7f614800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f61557fa640 1 -- 172.21.15.26:0/3291804304 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148017070 con 0x7f615814f790 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f61557fa640 1 -- 172.21.15.26:0/3291804304 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61480052d0 con 0x7f615814f790 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6158154e90 con 0x7f615814f790 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f61557fa640 1 -- 172.21.15.26:0/3291804304 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148003810 con 0x7f615814f790 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.377+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61581ee6c0 con 0x7f615814f790 2024-03-06T07:23:44.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.378+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6158067ad0 con 0x7f615814f790 2024-03-06T07:23:44.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.378+0000 7f61557fa640 1 -- 172.21.15.26:0/3291804304 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f614801a070 con 0x7f615814f790 2024-03-06T07:23:44.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.379+0000 7f61557fa640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f614406b4d0 0x7f614406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.379+0000 7f61557fa640 1 -- 172.21.15.26:0/3291804304 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6148089640 con 0x7f615814f790 2024-03-06T07:23:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.379+0000 7f615cb16640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f614406b4d0 0x7f614406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.379+0000 7f615cb16640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f614406b4d0 0x7f614406d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f61580c08f0 tx=0x7f615000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.383+0000 7f61557fa640 1 -- 172.21.15.26:0/3291804304 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6158067ad0 con 0x7f615814f790 2024-03-06T07:23:44.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.601+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f615804f330 con 0x7f615814f790 2024-03-06T07:23:44.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.602+0000 7f61557fa640 1 -- 172.21.15.26:0/3291804304 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f6148057d20 con 0x7f615814f790 2024-03-06T07:23:44.605 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:44.605 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1702158641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1702158641},{"type":"v1","addr":"172.21.15.117:6837","nonce":1702158641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:40.012005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6736":{"gid":6736,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6736,"qdb_cluster":[6736]},"id":7}]} 2024-03-06T07:23:44.605 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 82 2024-03-06T07:23:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f614406b4d0 msgr2=0x7f614406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f614406b4d0 0x7f614406d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f61580c08f0 tx=0x7f615000b040 comp rx=0 tx=0).stop 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f615814f790 msgr2=0x7f61581eaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f615814f790 0x7f61581eaf40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6148001eb0 tx=0x7f614800c890 comp rx=0 tx=0).stop 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 shutdown_connections 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f614406b4d0 0x7f614406d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f615814f790 0x7f61581eaf40 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61580c5c60 0x7f61580bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 --2- 172.21.15.26:0/3291804304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61580c1aa0 0x7f61581eeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 >> 172.21.15.26:0/3291804304 conn(0x7f61580c6710 msgr2=0x7f615814d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 shutdown_connections 2024-03-06T07:23:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.604+0000 7f615ed7a640 1 -- 172.21.15.26:0/3291804304 wait complete. 2024-03-06T07:23:44.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:44.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.716+0000 7fae860fb640 1 Processor -- start 2024-03-06T07:23:44.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.716+0000 7fae860fb640 1 -- start start 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae860fb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae8006da40 0x7fae80155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae860fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae800c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae860fb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae80156050 con 0x7fae800c6780 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae860fb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8006de20 con 0x7fae8006da40 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae860fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae80072630 con 0x7fae800c7380 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7f7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae8006da40 0x7fae80155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7effd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fae800c6780 0x7fae80158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:41366/0 (socket says 172.21.15.26:41366) 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7effd640 1 -- 172.21.15.26:0/59330511 learned_addr learned my addr 172.21.15.26:0/59330511 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae800c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7f7fe640 1 -- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 msgr2=0x7fae800c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7f7fe640 1 --2- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae800c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7f7fe640 1 -- 172.21.15.26:0/59330511 >> v1:172.21.15.117:6789/0 conn(0x7fae800c6780 legacy=0x7fae80158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7f7fe640 1 -- 172.21.15.26:0/59330511 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80072910 con 0x7fae8006da40 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7e7fc640 1 -- 172.21.15.26:0/59330511 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816851415 0 0) 0x7fae74001110 con 0x7fae800c6780 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.717+0000 7fae7ffff640 1 --2- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae800c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.718+0000 7fae7f7fe640 1 --2- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80155b10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fae700097a0 tx=0x7fae7002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e7b4c1ae7973851 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.718+0000 7fae7e7fc640 1 -- 172.21.15.26:0/59330511 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70031780 con 0x7fae8006da40 2024-03-06T07:23:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.718+0000 7fae7e7fc640 1 -- 172.21.15.26:0/59330511 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70037070 con 0x7fae8006da40 2024-03-06T07:23:44.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.718+0000 7fae7e7fc640 1 -- 172.21.15.26:0/59330511 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70031b40 con 0x7fae8006da40 2024-03-06T07:23:44.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 -- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 msgr2=0x7fae80155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 --2- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80155b10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fae700097a0 tx=0x7fae7002b820 comp rx=0 tx=0).stop 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 -- 172.21.15.26:0/59330511 shutdown_connections 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 --2- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae800c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 --2- 172.21.15.26:0/59330511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80155b10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 -- 172.21.15.26:0/59330511 >> 172.21.15.26:0/59330511 conn(0x7fae800aa630 msgr2=0x7fae800aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 -- 172.21.15.26:0/59330511 shutdown_connections 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.719+0000 7fae860fb640 1 -- 172.21.15.26:0/59330511 wait complete. 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.720+0000 7fae860fb640 1 Processor -- start 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.720+0000 7fae860fb640 1 -- start start 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae860fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae860fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae800c6780 0x7fae8006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae860fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae80156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae860fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae801594a0 con 0x7fae800c6780 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae860fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae80158890 con 0x7fae8006da40 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae860fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae800c37c0 con 0x7fae800c7380 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae800c6780 0x7fae8006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:33824/0 (socket says 172.21.15.26:33824) 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae800c6780 0x7fae8006f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57530/0 (socket says 172.21.15.26:57530) 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2795328911 learned_addr learned my addr 172.21.15.26:0/2795328911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7ffff640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae80156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7effd640 1 -- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 msgr2=0x7fae80156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7effd640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae80156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7effd640 1 -- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 msgr2=0x7fae80152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7effd640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.721+0000 7fae7effd640 1 -- 172.21.15.26:0/2795328911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae801526c0 con 0x7fae800c6780 2024-03-06T07:23:44.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.722+0000 7fae7effd640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae800c6780 0x7fae8006f5a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c780 tx=0x7fae7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.722+0000 7fae67fff640 1 -- 172.21.15.26:0/2795328911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74018070 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.722+0000 7fae67fff640 1 -- 172.21.15.26:0/2795328911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae740070d0 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.722+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8014c850 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.722+0000 7fae67fff640 1 -- 172.21.15.26:0/2795328911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740074c0 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.722+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae801eae20 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.723+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8004f330 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.723+0000 7fae67fff640 1 -- 172.21.15.26:0/2795328911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae7401b070 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.723+0000 7fae67fff640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae6c06b4d0 0x7fae6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.724+0000 7fae67fff640 1 -- 172.21.15.26:0/2795328911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fae74089560 con 0x7fae800c6780 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.724+0000 7fae7f7fe640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae6c06b4d0 0x7fae6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.724+0000 7fae7f7fe640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae6c06b4d0 0x7fae6c06d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fae700097a0 tx=0x7fae7003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.728+0000 7fae67fff640 1 -- 172.21.15.26:0/2795328911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae801526c0 con 0x7fae800c6780 2024-03-06T07:23:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.948+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae800741a0 con 0x7fae800c6780 2024-03-06T07:23:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.948+0000 7fae67fff640 1 -- 172.21.15.26:0/2795328911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4982 (secure 0 0 0) 0x7fae80156050 con 0x7fae800c6780 2024-03-06T07:23:44.951 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:44.951 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1702158641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1702158641},{"type":"v1","addr":"172.21.15.117:6837","nonce":1702158641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.599873+0000","modified":"2024-03-06T07:23:40.012005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6736":{"gid":6736,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/593719862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":593719862},{"type":"v1","addr":"172.21.15.117:6836","nonce":593719862}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6736,"qdb_cluster":[6736]},"id":7}]} 2024-03-06T07:23:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 82 2024-03-06T07:23:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae6c06b4d0 msgr2=0x7fae6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae6c06b4d0 0x7fae6c06d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fae700097a0 tx=0x7fae7003d040 comp rx=0 tx=0).stop 2024-03-06T07:23:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae800c6780 msgr2=0x7fae8006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae800c6780 0x7fae8006f5a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c780 tx=0x7fae7400cc40 comp rx=0 tx=0).stop 2024-03-06T07:23:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 shutdown_connections 2024-03-06T07:23:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae6c06b4d0 0x7fae6c06d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae800c7380 0x7fae80156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae800c6780 0x7fae8006f5a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 --2- 172.21.15.26:0/2795328911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8006da40 0x7fae80152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 >> 172.21.15.26:0/2795328911 conn(0x7fae800aa630 msgr2=0x7fae800c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 shutdown_connections 2024-03-06T07:23:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:44.950+0000 7fae860fb640 1 -- 172.21.15.26:0/2795328911 wait complete. 2024-03-06T07:23:44.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:45.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.063+0000 7ffa001a1640 1 Processor -- start 2024-03-06T07:23:45.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.063+0000 7ffa001a1640 1 -- start start 2024-03-06T07:23:45.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.064+0000 7ffa001a1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff9f80c88c0 0x7ff9f8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.064+0000 7ffa001a1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80c73d0 con 0x7ff9f814f360 2024-03-06T07:23:45.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.064+0000 7ffa001a1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80be2e0 con 0x7ff9f8158b60 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.064+0000 7ffa001a1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f8159700 con 0x7ff9f80c88c0 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.064+0000 7ff9fdf3d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff9f814f360 0x7ff9f80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38360/0 (socket says 172.21.15.26:38360) 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.064+0000 7ff9fdf3d640 1 -- 172.21.15.26:0/2693296947 learned_addr learned my addr 172.21.15.26:0/2693296947 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.064+0000 7ff9fd73c640 1 --2- 172.21.15.26:0/2693296947 >> v2:172.21.15.117:3300/0 conn(0x7ff9f80c88c0 0x7ff9f8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.065+0000 7ff9fcf3b640 1 -- 172.21.15.26:0/2693296947 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359136709 0 0) 0x7ff9f80c73d0 con 0x7ff9f814f360 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.065+0000 7ff9fcf3b640 1 -- 172.21.15.26:0/2693296947 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e0003520 con 0x7ff9f814f360 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.065+0000 7ff9fd73c640 1 -- 172.21.15.26:0/2693296947 >> v1:172.21.15.117:6789/0 conn(0x7ff9f8158b60 legacy=0x7ff9f8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.065+0000 7ff9fd73c640 1 -- 172.21.15.26:0/2693296947 >> v1:172.21.15.26:6789/0 conn(0x7ff9f814f360 legacy=0x7ff9f80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.065+0000 7ff9fd73c640 1 -- 172.21.15.26:0/2693296947 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80c7670 con 0x7ff9f80c88c0 2024-03-06T07:23:45.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ff9fd73c640 1 --2- 172.21.15.26:0/2693296947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f80c88c0 0x7ff9f8155c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0097a0 tx=0x7ff9ec02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44239409670ab217 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ff9fcf3b640 1 -- 172.21.15.26:0/2693296947 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec035020 con 0x7ff9f80c88c0 2024-03-06T07:23:45.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ff9fcf3b640 1 -- 172.21.15.26:0/2693296947 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec031830 con 0x7ff9f80c88c0 2024-03-06T07:23:45.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ff9fcf3b640 1 -- 172.21.15.26:0/2693296947 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec031be0 con 0x7ff9f80c88c0 2024-03-06T07:23:45.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ffa001a1640 1 -- 172.21.15.26:0/2693296947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f80c88c0 msgr2=0x7ff9f8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2693296947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f80c88c0 0x7ff9f8155c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0097a0 tx=0x7ff9ec02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ffa001a1640 1 -- 172.21.15.26:0/2693296947 shutdown_connections 2024-03-06T07:23:45.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2693296947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f80c88c0 0x7ff9f8155c10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ffa001a1640 1 -- 172.21.15.26:0/2693296947 >> 172.21.15.26:0/2693296947 conn(0x7ff9f80ac570 msgr2=0x7ff9f80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:45.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ffa001a1640 1 -- 172.21.15.26:0/2693296947 shutdown_connections 2024-03-06T07:23:45.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.066+0000 7ffa001a1640 1 -- 172.21.15.26:0/2693296947 wait complete. 2024-03-06T07:23:45.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.067+0000 7ffa001a1640 1 Processor -- start 2024-03-06T07:23:45.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ffa001a1640 1 -- start start 2024-03-06T07:23:45.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ffa001a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f80c88c0 0x7ff9f81eb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ffa001a1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9f814f360 0x7ff9f8157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ffa001a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f8158b60 0x7ff9f814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ffa001a1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c7c30 con 0x7ff9f80c88c0 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ffa001a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c5a50 con 0x7ff9f8158b60 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ffa001a1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c5bd0 con 0x7ff9f814f360 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ff9fdf3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f80c88c0 0x7ff9f81eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ff9fdf3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f80c88c0 0x7ff9f81eb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57540/0 (socket says 172.21.15.26:57540) 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ff9fdf3d640 1 -- 172.21.15.26:0/2395726475 learned_addr learned my addr 172.21.15.26:0/2395726475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ff9fe73e640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f8158b60 0x7ff9f814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.068+0000 7ff9fd73c640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9f814f360 0x7ff9f8157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9fdf3d640 1 -- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9f814f360 msgr2=0x7ff9f8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9fdf3d640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9f814f360 0x7ff9f8157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9fdf3d640 1 -- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f8158b60 msgr2=0x7ff9f814b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9fdf3d640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f8158b60 0x7ff9f814b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9fdf3d640 1 -- 172.21.15.26:0/2395726475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80c1560 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9fdf3d640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f80c88c0 0x7ff9f81eb370 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c530 tx=0x7ff9f000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9f67fc640 1 -- 172.21.15.26:0/2395726475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0017070 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9f67fc640 1 -- 172.21.15.26:0/2395726475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0005ce0 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f80c0fe0 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ff9f67fc640 1 -- 172.21.15.26:0/2395726475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00072b0 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.069+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f80c1280 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.070+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f804f330 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.070+0000 7ff9f67fc640 1 -- 172.21.15.26:0/2395726475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9f001a070 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.071+0000 7ff9f67fc640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9e406b4d0 0x7ff9e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.071+0000 7ff9f67fc640 1 -- 172.21.15.26:0/2395726475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff9f00896f0 con 0x7ff9f80c88c0 2024-03-06T07:23:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.071+0000 7ff9fd73c640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9e406b4d0 0x7ff9e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.071+0000 7ff9fd73c640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9e406b4d0 0x7ff9e406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.075+0000 7ff9f67fc640 1 -- 172.21.15.26:0/2395726475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff9f0057e90 con 0x7ff9f80c88c0 2024-03-06T07:23:45.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.258+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9f806cda0 con 0x7ff9f80c88c0 2024-03-06T07:23:45.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.259+0000 7ff9f67fc640 1 -- 172.21.15.26:0/2395726475 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17016 (secure 0 0 0) 0x7ff9f005bf30 con 0x7ff9f80c88c0 2024-03-06T07:23:45.262 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:45.262 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:37.000731+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:22.873048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.559370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/1940022472":"2024-03-07T07:23:37.000697+0000","172.21.15.117:6835/1940022472":"2024-03-07T07:23:37.000697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9e406b4d0 msgr2=0x7ff9e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9e406b4d0 0x7ff9e406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec03d040 comp rx=0 tx=0).stop 2024-03-06T07:23:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f80c88c0 msgr2=0x7ff9f81eb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f80c88c0 0x7ff9f81eb370 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c530 tx=0x7ff9f000c9f0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 shutdown_connections 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9e406b4d0 0x7ff9e406d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f8158b60 0x7ff9f814b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9f814f360 0x7ff9f8157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 --2- 172.21.15.26:0/2395726475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f80c88c0 0x7ff9f81eb370 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 >> 172.21.15.26:0/2395726475 conn(0x7ff9f80ac570 msgr2=0x7ff9f814e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.261+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 shutdown_connections 2024-03-06T07:23:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.262+0000 7ffa001a1640 1 -- 172.21.15.26:0/2395726475 wait complete. 2024-03-06T07:23:45.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:23:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.375+0000 7f6a7f13a640 1 Processor -- start 2024-03-06T07:23:45.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.375+0000 7f6a7f13a640 1 -- start start 2024-03-06T07:23:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.375+0000 7f6a7f13a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a78155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7f13a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a780c8950 0x7f6a780bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7f13a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a780c8e30 con 0x7f6a780bfe30 2024-03-06T07:23:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7f13a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a78152ce0 con 0x7f6a780c8950 2024-03-06T07:23:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7f13a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780c9110 con 0x7f6a78152900 2024-03-06T07:23:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7d6d7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6a780bfe30 0x7f6a780c64f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38364/0 (socket says 172.21.15.26:38364) 2024-03-06T07:23:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a780c8950 0x7f6a780bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7d6d7640 1 -- 172.21.15.26:0/1805463913 learned_addr learned my addr 172.21.15.26:0/1805463913 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7ced6640 1 --2- 172.21.15.26:0/1805463913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a78155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a777fe640 1 -- 172.21.15.26:0/1805463913 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794666386 0 0) 0x7f6a780c8e30 con 0x7f6a780bfe30 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a777fe640 1 -- 172.21.15.26:0/1805463913 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a68003520 con 0x7f6a780bfe30 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a77fff640 1 -- 172.21.15.26:0/1805463913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 msgr2=0x7f6a78155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a77fff640 1 --2- 172.21.15.26:0/1805463913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a78155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a77fff640 1 -- 172.21.15.26:0/1805463913 >> v1:172.21.15.26:6789/0 conn(0x7f6a780bfe30 legacy=0x7f6a780c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a77fff640 1 -- 172.21.15.26:0/1805463913 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a780c93f0 con 0x7f6a780c8950 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.376+0000 7f6a7ced6640 1 --2- 172.21.15.26:0/1805463913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a78155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a77fff640 1 --2- 172.21.15.26:0/1805463913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780c8950 0x7f6a780bf8f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6a640097a0 tx=0x7f6a6402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cfd1564e35366a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a777fe640 1 -- 172.21.15.26:0/1805463913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64038070 con 0x7f6a780c8950 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a777fe640 1 -- 172.21.15.26:0/1805463913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a64031700 con 0x7f6a780c8950 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a777fe640 1 -- 172.21.15.26:0/1805463913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64031ab0 con 0x7f6a780c8950 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1805463913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780c8950 msgr2=0x7f6a780bf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1805463913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780c8950 0x7f6a780bf8f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6a640097a0 tx=0x7f6a6402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1805463913 shutdown_connections 2024-03-06T07:23:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1805463913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780c8950 0x7f6a780bf8f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1805463913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a78155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.377+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1805463913 >> 172.21.15.26:0/1805463913 conn(0x7f6a780ac860 msgr2=0x7f6a780acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.378+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1805463913 shutdown_connections 2024-03-06T07:23:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.378+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1805463913 wait complete. 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 Processor -- start 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 -- start start 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780bfe30 0x7f6a781eb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a780c8950 0x7f6a781e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a781edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780c5770 con 0x7f6a780bfe30 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780c55f0 con 0x7f6a780c8950 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7f13a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780c6c00 con 0x7f6a78152900 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7ced6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780bfe30 0x7f6a781eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7ced6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780bfe30 0x7f6a781eb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57564/0 (socket says 172.21.15.26:57564) 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.379+0000 7f6a7ced6640 1 -- 172.21.15.26:0/1870460945 learned_addr learned my addr 172.21.15.26:0/1870460945 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a77fff640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a780c8950 0x7f6a781e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7d6d7640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a781edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7ced6640 1 -- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 msgr2=0x7f6a781edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7ced6640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a781edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7ced6640 1 -- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a780c8950 msgr2=0x7f6a781e7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7ced6640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a780c8950 0x7f6a781e7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7ced6640 1 -- 172.21.15.26:0/1870460945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a78155050 con 0x7f6a780bfe30 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7ced6640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780bfe30 0x7f6a781eb240 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e990 tx=0x7f6a6c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a757fa640 1 -- 172.21.15.26:0/1870460945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c017070 con 0x7f6a780bfe30 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a757fa640 1 -- 172.21.15.26:0/1870460945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c00c9e0 con 0x7f6a780bfe30 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a781e8310 con 0x7f6a780bfe30 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a757fa640 1 -- 172.21.15.26:0/1870460945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00cdb0 con 0x7f6a780bfe30 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.380+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a780c2c60 con 0x7f6a780bfe30 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.381+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7804f330 con 0x7f6a780bfe30 2024-03-06T07:23:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.382+0000 7f6a757fa640 1 -- 172.21.15.26:0/1870460945 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a6c01a070 con 0x7f6a780bfe30 2024-03-06T07:23:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.382+0000 7f6a757fa640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.382+0000 7f6a757fa640 1 -- 172.21.15.26:0/1870460945 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6a6c08a150 con 0x7f6a780bfe30 2024-03-06T07:23:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.382+0000 7f6a77fff640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.382+0000 7f6a77fff640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6a64035000 tx=0x7f6a64008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.386+0000 7f6a757fa640 1 -- 172.21.15.26:0/1870460945 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6a6c058830 con 0x7f6a780bfe30 2024-03-06T07:23:45.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.603+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6a780c53d0 con 0x7f6a780bfe30 2024-03-06T07:23:45.606 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:23:45.604+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:23:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.605+0000 7f6a757fa640 1 -- 172.21.15.26:0/1870460945 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6a6c05c8d0 con 0x7f6a780bfe30 2024-03-06T07:23:45.608 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:23:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.607+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6a5c06b500 msgr2=0x7f6a5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.607+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6a64035000 tx=0x7f6a64008030 comp rx=0 tx=0).stop 2024-03-06T07:23:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.607+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780bfe30 msgr2=0x7f6a781eb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.607+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780bfe30 0x7f6a781eb240 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e990 tx=0x7f6a6c00ee50 comp rx=0 tx=0).stop 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 shutdown_connections 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a78152900 0x7f6a781edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a780c8950 0x7f6a781e7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 --2- 172.21.15.26:0/1870460945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a780bfe30 0x7f6a781eb240 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 >> 172.21.15.26:0/1870460945 conn(0x7f6a780ac860 msgr2=0x7f6a7806ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 shutdown_connections 2024-03-06T07:23:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.608+0000 7f6a7f13a640 1 -- 172.21.15.26:0/1870460945 wait complete. 2024-03-06T07:23:45.612 INFO:tasks.ceph.mds.d.smithi117.stderr: -202> 2024-03-06T07:23:40.531+0000 7f0a9ebaf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:45.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:23:45.634 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:23:45.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.722+0000 7ff7d8e0e640 1 Processor -- start 2024-03-06T07:23:45.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.722+0000 7ff7d8e0e640 1 -- start start 2024-03-06T07:23:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.722+0000 7ff7d8e0e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7d40c5990 0x7ff7d40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.722+0000 7ff7d8e0e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7d40ab170 0x7ff7d406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.722+0000 7ff7d8e0e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d406c730 0x7ff7d41523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.722+0000 7ff7d8e0e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d406cba0 con 0x7ff7d40ab170 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7d8e0e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d40c3ef0 con 0x7ff7d40c5990 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7d8e0e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d41556d0 con 0x7ff7d406c730 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7d40ab170 0x7ff7d406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7d40ab170 0x7ff7d406c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57572/0 (socket says 172.21.15.26:57572) 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 -- 172.21.15.26:0/1468137191 learned_addr learned my addr 172.21.15.26:0/1468137191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7d259c640 1 --2- 172.21.15.26:0/1468137191 >> v2:172.21.15.117:3300/0 conn(0x7ff7d40c5990 0x7ff7d40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7d2d9d640 1 --2- 172.21.15.26:0/1468137191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d406c730 0x7ff7d41523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 -- 172.21.15.26:0/1468137191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d406c730 msgr2=0x7ff7d41523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 --2- 172.21.15.26:0/1468137191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d406c730 0x7ff7d41523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 -- 172.21.15.26:0/1468137191 >> v2:172.21.15.117:3300/0 conn(0x7ff7d40c5990 msgr2=0x7ff7d40c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 --2- 172.21.15.26:0/1468137191 >> v2:172.21.15.117:3300/0 conn(0x7ff7d40c5990 0x7ff7d40c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.723+0000 7ff7cbfff640 1 -- 172.21.15.26:0/1468137191 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d41559b0 con 0x7ff7d40ab170 2024-03-06T07:23:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7cbfff640 1 --2- 172.21.15.26:0/1468137191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d40ab170 0x7ff7d406c1f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40c12a0 tx=0x7ff7bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b528595803d15e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d1d9b640 1 -- 172.21.15.26:0/1468137191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc035020 con 0x7ff7d40ab170 2024-03-06T07:23:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d1d9b640 1 -- 172.21.15.26:0/1468137191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc031870 con 0x7ff7d40ab170 2024-03-06T07:23:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d1d9b640 1 -- 172.21.15.26:0/1468137191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc031c40 con 0x7ff7d40ab170 2024-03-06T07:23:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/1468137191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d40ab170 msgr2=0x7ff7d406c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/1468137191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d40ab170 0x7ff7d406c1f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40c12a0 tx=0x7ff7bc02b910 comp rx=0 tx=0).stop 2024-03-06T07:23:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/1468137191 shutdown_connections 2024-03-06T07:23:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/1468137191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d406c730 0x7ff7d41523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/1468137191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d40ab170 0x7ff7d406c1f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/1468137191 >> v2:172.21.15.117:3300/0 conn(0x7ff7d40c5990 0x7ff7d40c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/1468137191 >> 172.21.15.26:0/1468137191 conn(0x7ff7d40c6710 msgr2=0x7ff7d40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/1468137191 shutdown_connections 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.724+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/1468137191 wait complete. 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 Processor -- start 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 -- start start 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d406c730 0x7ff7d41eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d40ab170 0x7ff7d40a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d40c5990 0x7ff7d4070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4156490 con 0x7ff7d406c730 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4155f70 con 0x7ff7d40ab170 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d8e0e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4156610 con 0x7ff7d40c5990 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d406c730 0x7ff7d41eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d406c730 0x7ff7d41eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57588/0 (socket says 172.21.15.26:57588) 2024-03-06T07:23:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.726+0000 7ff7d259c640 1 -- 172.21.15.26:0/699215862 learned_addr learned my addr 172.21.15.26:0/699215862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7cbfff640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d40ab170 0x7ff7d40a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d2d9d640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d40c5990 0x7ff7d4070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d259c640 1 -- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d40c5990 msgr2=0x7ff7d4070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d259c640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d40c5990 0x7ff7d4070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d259c640 1 -- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d40ab170 msgr2=0x7ff7d40a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d259c640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d40ab170 0x7ff7d40a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d259c640 1 -- 172.21.15.26:0/699215862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d41eeb10 con 0x7ff7d406c730 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d259c640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d406c730 0x7ff7d41eabc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400e9b0 tx=0x7ff7c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7caffd640 1 -- 172.21.15.26:0/699215862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4017070 con 0x7ff7d406c730 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7caffd640 1 -- 172.21.15.26:0/699215862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c400c9a0 con 0x7ff7d406c730 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d41eed90 con 0x7ff7d406c730 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7caffd640 1 -- 172.21.15.26:0/699215862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400cd90 con 0x7ff7d406c730 2024-03-06T07:23:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.727+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d4153b20 con 0x7ff7d406c730 2024-03-06T07:23:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.728+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d404f330 con 0x7ff7d406c730 2024-03-06T07:23:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.729+0000 7ff7caffd640 1 -- 172.21.15.26:0/699215862 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7c401a070 con 0x7ff7d406c730 2024-03-06T07:23:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.729+0000 7ff7caffd640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7b406b4d0 0x7ff7b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.729+0000 7ff7caffd640 1 -- 172.21.15.26:0/699215862 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff7c408a330 con 0x7ff7d406c730 2024-03-06T07:23:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.729+0000 7ff7cbfff640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7b406b4d0 0x7ff7b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.730+0000 7ff7cbfff640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7b406b4d0 0x7ff7b406d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc036000 tx=0x7ff7bc03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.733+0000 7ff7caffd640 1 -- 172.21.15.26:0/699215862 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7d41eeb10 con 0x7ff7d406c730 2024-03-06T07:23:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:45.955+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff7d40c6bf0 con 0x7ff7d406c730 2024-03-06T07:23:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.607+0000 7ff7caffd640 1 -- 172.21.15.26:0/699215862 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7c4058930 con 0x7ff7d406c730 2024-03-06T07:23:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7b406b4d0 msgr2=0x7ff7b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7b406b4d0 0x7ff7b406d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc036000 tx=0x7ff7bc03c040 comp rx=0 tx=0).stop 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d406c730 msgr2=0x7ff7d41eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d406c730 0x7ff7d41eabc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400e9b0 tx=0x7ff7c400ee70 comp rx=0 tx=0).stop 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 shutdown_connections 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7b406b4d0 0x7ff7b406d990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7d40c5990 0x7ff7d4070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d40ab170 0x7ff7d40a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 --2- 172.21.15.26:0/699215862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d406c730 0x7ff7d41eabc0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 >> 172.21.15.26:0/699215862 conn(0x7ff7d40c6710 msgr2=0x7ff7d4152a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 shutdown_connections 2024-03-06T07:23:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.610+0000 7ff7d8e0e640 1 -- 172.21.15.26:0/699215862 wait complete. 2024-03-06T07:23:46.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:23:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.737+0000 7f246e639640 1 Processor -- start 2024-03-06T07:23:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.737+0000 7f246e639640 1 -- start start 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.737+0000 7f246e639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c7f10 0x7f24680c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246e639640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f246806f2d0 0x7f24681556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246e639640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2468072620 con 0x7f24680c8720 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246e639640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2468072900 con 0x7f246806f2d0 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246e639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2468072be0 con 0x7f24680c7f10 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246cbd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f246806f2d0 0x7f24681556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246cbd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f246806f2d0 0x7f24681556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57594/0 (socket says 172.21.15.26:57594) 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246cbd6640 1 -- 172.21.15.26:0/2620383135 learned_addr learned my addr 172.21.15.26:0/2620383135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246cbd6640 1 -- 172.21.15.26:0/2620383135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c7f10 msgr2=0x7f24680c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246cbd6640 1 --2- 172.21.15.26:0/2620383135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c7f10 0x7f24680c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246cbd6640 1 -- 172.21.15.26:0/2620383135 >> v1:172.21.15.117:6789/0 conn(0x7f24680c8720 legacy=0x7f246806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.738+0000 7f246cbd6640 1 -- 172.21.15.26:0/2620383135 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2468072ec0 con 0x7f246806f2d0 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246cbd6640 1 --2- 172.21.15.26:0/2620383135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24681556e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2460009e50 tx=0x7f246002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb04cd6b84aa664 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f2466ffd640 1 -- 172.21.15.26:0/2620383135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246002c950 con 0x7f246806f2d0 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f2466ffd640 1 -- 172.21.15.26:0/2620383135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246002caf0 con 0x7f246806f2d0 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f2466ffd640 1 -- 172.21.15.26:0/2620383135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600343f0 con 0x7f246806f2d0 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246e639640 1 -- 172.21.15.26:0/2620383135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 msgr2=0x7f24681556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246e639640 1 --2- 172.21.15.26:0/2620383135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24681556e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2460009e50 tx=0x7f246002bdb0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246e639640 1 -- 172.21.15.26:0/2620383135 shutdown_connections 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246e639640 1 --2- 172.21.15.26:0/2620383135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24681556e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246e639640 1 --2- 172.21.15.26:0/2620383135 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c7f10 0x7f24680c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246e639640 1 -- 172.21.15.26:0/2620383135 >> 172.21.15.26:0/2620383135 conn(0x7f24680aa5c0 msgr2=0x7f24680aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.739+0000 7f246e639640 1 -- 172.21.15.26:0/2620383135 shutdown_connections 2024-03-06T07:23:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.740+0000 7f246e639640 1 -- 172.21.15.26:0/2620383135 wait complete. 2024-03-06T07:23:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 Processor -- start 2024-03-06T07:23:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 -- start start 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24680c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24680c7f10 0x7f246806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c8720 0x7f24681edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246806f860 con 0x7f246806f2d0 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246806f6e0 con 0x7f24680c7f10 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.741+0000 7f246e639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24680c82f0 con 0x7f24680c8720 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24680c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24680c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57606/0 (socket says 172.21.15.26:57606) 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 -- 172.21.15.26:0/1469840256 learned_addr learned my addr 172.21.15.26:0/1469840256 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f246cbd6640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c8720 0x7f24681edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f24677fe640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24680c7f10 0x7f246806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 -- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c8720 msgr2=0x7f24681edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c8720 0x7f24681edbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 -- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24680c7f10 msgr2=0x7f246806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24680c7f10 0x7f246806d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 -- 172.21.15.26:0/1469840256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24681ea660 con 0x7f246806f2d0 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f24677fe640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24680c7f10 0x7f246806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2467fff640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24680c1eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f245800b4a0 tx=0x7f245800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2464ff9640 1 -- 172.21.15.26:0/1469840256 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458014070 con 0x7f246806f2d0 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2464ff9640 1 -- 172.21.15.26:0/1469840256 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2458010610 con 0x7f246806f2d0 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246806dae0 con 0x7f246806f2d0 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f2464ff9640 1 -- 172.21.15.26:0/1469840256 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458010a00 con 0x7f246806f2d0 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.742+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24681eac00 con 0x7f246806f2d0 2024-03-06T07:23:46.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.743+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246804f330 con 0x7f246806f2d0 2024-03-06T07:23:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.744+0000 7f2464ff9640 1 -- 172.21.15.26:0/1469840256 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2458017070 con 0x7f246806f2d0 2024-03-06T07:23:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.744+0000 7f2464ff9640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f245406b4d0 0x7f245406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.744+0000 7f2464ff9640 1 -- 172.21.15.26:0/1469840256 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f2458089e00 con 0x7f246806f2d0 2024-03-06T07:23:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.744+0000 7f24677fe640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f245406b4d0 0x7f245406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.745+0000 7f24677fe640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f245406b4d0 0x7f245406d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f24680ac390 tx=0x7f245c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.748+0000 7f2464ff9640 1 -- 172.21.15.26:0/1469840256 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2458058ad0 con 0x7f246806f2d0 2024-03-06T07:23:46.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:46.927+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24680add90 con 0x7f246806f2d0 2024-03-06T07:23:47.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.612+0000 7f2464ff9640 1 -- 172.21.15.26:0/1469840256 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7f24580583d0 con 0x7f246806f2d0 2024-03-06T07:23:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:23:47.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.616+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f245406b4d0 msgr2=0x7f245406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.616+0000 7f246e639640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f245406b4d0 0x7f245406d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f24680ac390 tx=0x7f245c008040 comp rx=0 tx=0).stop 2024-03-06T07:23:47.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.616+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 msgr2=0x7f24680c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.616+0000 7f246e639640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24680c1eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f245800b4a0 tx=0x7f245800b960 comp rx=0 tx=0).stop 2024-03-06T07:23:47.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.617+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 shutdown_connections 2024-03-06T07:23:47.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.617+0000 7f246e639640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f245406b4d0 0x7f245406d990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.617+0000 7f246e639640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24680c8720 0x7f24681edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.617+0000 7f246e639640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24680c7f10 0x7f246806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.617+0000 7f246e639640 1 --2- 172.21.15.26:0/1469840256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f246806f2d0 0x7f24680c1eb0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.617+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 >> 172.21.15.26:0/1469840256 conn(0x7f24680aa5c0 msgr2=0x7f24680aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:47.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.617+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 shutdown_connections 2024-03-06T07:23:47.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.618+0000 7f246e639640 1 -- 172.21.15.26:0/1469840256 wait complete. 2024-03-06T07:23:47.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:23:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.743+0000 7f3439615640 1 Processor -- start 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3439615640 1 -- start start 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3439615640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f343406fe20 0x7f3434155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3439615640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3434158780 0x7f3434158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3439615640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3434156150 con 0x7f34340c6780 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3439615640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34340a7d70 con 0x7f343406fe20 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3439615640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34340a8050 con 0x7f3434158780 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3432ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f343406fe20 0x7f3434155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3432ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f343406fe20 0x7f3434155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39344/0 (socket says 172.21.15.26:39344) 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.744+0000 7f3432ffd640 1 -- 172.21.15.26:0/382221841 learned_addr learned my addr 172.21.15.26:0/382221841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.745+0000 7f34337fe640 1 --2- 172.21.15.26:0/382221841 >> v2:172.21.15.117:3300/0 conn(0x7f3434158780 0x7f3434158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.745+0000 7f3432ffd640 1 -- 172.21.15.26:0/382221841 >> v2:172.21.15.117:3300/0 conn(0x7f3434158780 msgr2=0x7f3434158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.745+0000 7f3432ffd640 1 --2- 172.21.15.26:0/382221841 >> v2:172.21.15.117:3300/0 conn(0x7f3434158780 0x7f3434158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.745+0000 7f3432ffd640 1 -- 172.21.15.26:0/382221841 >> v1:172.21.15.117:6789/0 conn(0x7f34340c6780 legacy=0x7f3434072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.745+0000 7f3432ffd640 1 -- 172.21.15.26:0/382221841 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34340a8330 con 0x7f343406fe20 2024-03-06T07:23:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.745+0000 7f34337fe640 1 --2- 172.21.15.26:0/382221841 >> v2:172.21.15.117:3300/0 conn(0x7f3434158780 0x7f3434158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.745+0000 7f3432ffd640 1 --2- 172.21.15.26:0/382221841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343406fe20 0x7f3434155c10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f34340ab6c0 tx=0x7f342402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d74a899961ac71 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3431ffb640 1 -- 172.21.15.26:0/382221841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342402c9f0 con 0x7f343406fe20 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3431ffb640 1 -- 172.21.15.26:0/382221841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342402cb90 con 0x7f343406fe20 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3431ffb640 1 -- 172.21.15.26:0/382221841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34240343f0 con 0x7f343406fe20 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 -- 172.21.15.26:0/382221841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343406fe20 msgr2=0x7f3434155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 --2- 172.21.15.26:0/382221841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343406fe20 0x7f3434155c10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f34340ab6c0 tx=0x7f342402be50 comp rx=0 tx=0).stop 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 -- 172.21.15.26:0/382221841 shutdown_connections 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 --2- 172.21.15.26:0/382221841 >> v2:172.21.15.117:3300/0 conn(0x7f3434158780 0x7f3434158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 --2- 172.21.15.26:0/382221841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343406fe20 0x7f3434155c10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 -- 172.21.15.26:0/382221841 >> 172.21.15.26:0/382221841 conn(0x7f34340aa630 msgr2=0x7f34340aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 -- 172.21.15.26:0/382221841 shutdown_connections 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.746+0000 7f3439615640 1 -- 172.21.15.26:0/382221841 wait complete. 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 Processor -- start 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 -- start start 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f343406fe20 0x7f34341eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34340c6780 0x7f3434157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434158780 0x7f34341eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434070200 con 0x7f3434158780 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34340c3600 con 0x7f34340c6780 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3439615640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3434072c20 con 0x7f343406fe20 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f34337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434158780 0x7f34341eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f34337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434158780 0x7f34341eace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39352/0 (socket says 172.21.15.26:39352) 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f34337fe640 1 -- 172.21.15.26:0/3785435023 learned_addr learned my addr 172.21.15.26:0/3785435023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f34327fc640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34340c6780 0x7f3434157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.748+0000 7f3432ffd640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f343406fe20 0x7f34341eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f34337fe640 1 -- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f343406fe20 msgr2=0x7f34341eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f34337fe640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f343406fe20 0x7f34341eed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f34337fe640 1 -- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34340c6780 msgr2=0x7f3434157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f34337fe640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34340c6780 0x7f3434157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f34337fe640 1 -- 172.21.15.26:0/3785435023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34340ac4c0 con 0x7f3434158780 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f34337fe640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434158780 0x7f34341eace0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c590 tx=0x7f342c00ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f341b7fe640 1 -- 172.21.15.26:0/3785435023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c015020 con 0x7f3434158780 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f341b7fe640 1 -- 172.21.15.26:0/3785435023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c004440 con 0x7f3434158780 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34340adee0 con 0x7f3434158780 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f341b7fe640 1 -- 172.21.15.26:0/3785435023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c003850 con 0x7f3434158780 2024-03-06T07:23:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.749+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34341ee2b0 con 0x7f3434158780 2024-03-06T07:23:47.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.750+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343404f330 con 0x7f3434158780 2024-03-06T07:23:47.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.751+0000 7f341b7fe640 1 -- 172.21.15.26:0/3785435023 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f342c018020 con 0x7f3434158780 2024-03-06T07:23:47.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.751+0000 7f341b7fe640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f342006b500 0x7f342006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.751+0000 7f341b7fe640 1 -- 172.21.15.26:0/3785435023 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f342c089fa0 con 0x7f3434158780 2024-03-06T07:23:47.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.751+0000 7f3432ffd640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f342006b500 0x7f342006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.752+0000 7f3432ffd640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f342006b500 0x7f342006d9c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3424035000 tx=0x7f3424030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:47.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.755+0000 7f341b7fe640 1 -- 172.21.15.26:0/3785435023 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f342c01f020 con 0x7f3434158780 2024-03-06T07:23:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:47.938+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34340730d0 con 0x7f3434158780 2024-03-06T07:23:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.617+0000 7f341b7fe640 1 -- 172.21.15.26:0/3785435023 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7f342c0587e0 con 0x7f3434158780 2024-03-06T07:23:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f342006b500 msgr2=0x7f342006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f342006b500 0x7f342006d9c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3424035000 tx=0x7f3424030000 comp rx=0 tx=0).stop 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434158780 msgr2=0x7f34341eace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434158780 0x7f34341eace0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c590 tx=0x7f342c00ca50 comp rx=0 tx=0).stop 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 shutdown_connections 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f342006b500 0x7f342006d9c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434158780 0x7f34341eace0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34340c6780 0x7f3434157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 --2- 172.21.15.26:0/3785435023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f343406fe20 0x7f34341eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 >> 172.21.15.26:0/3785435023 conn(0x7f34340aa630 msgr2=0x7f343406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 shutdown_connections 2024-03-06T07:23:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.621+0000 7f3439615640 1 -- 172.21.15.26:0/3785435023 wait complete. 2024-03-06T07:23:48.638 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:23:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.745+0000 7ff2c6271640 1 Processor -- start 2024-03-06T07:23:48.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.745+0000 7ff2c6271640 1 -- start start 2024-03-06T07:23:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2c6271640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c01527f0 0x7ff2c00c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2c6271640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2c00c88f0 0x7ff2c0155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2c6271640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2c00c58b0 0x7ff2c0155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2c6271640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c00c9130 con 0x7ff2c00c88f0 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2c6271640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c0159260 con 0x7ff2c00c58b0 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2c6271640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c00c6b90 con 0x7ff2c01527f0 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2c00c88f0 0x7ff2c0155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2c00c88f0 0x7ff2c0155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39364/0 (socket says 172.21.15.26:39364) 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 -- 172.21.15.26:0/900769113 learned_addr learned my addr 172.21.15.26:0/900769113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 -- 172.21.15.26:0/900769113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c01527f0 msgr2=0x7ff2c00c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 --2- 172.21.15.26:0/900769113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c01527f0 0x7ff2c00c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 -- 172.21.15.26:0/900769113 >> v2:172.21.15.117:3300/0 conn(0x7ff2c00c58b0 msgr2=0x7ff2c0155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2c480e640 1 --2- 172.21.15.26:0/900769113 >> v2:172.21.15.117:3300/0 conn(0x7ff2c00c58b0 0x7ff2c0155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 --2- 172.21.15.26:0/900769113 >> v2:172.21.15.117:3300/0 conn(0x7ff2c00c58b0 0x7ff2c0155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.746+0000 7ff2bf7fe640 1 -- 172.21.15.26:0/900769113 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c01596f0 con 0x7ff2c00c88f0 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.747+0000 7ff2bf7fe640 1 --2- 172.21.15.26:0/900769113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c00c88f0 0x7ff2c0155210 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff2c008cb10 tx=0x7ff2b002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe311ac59ae8e61c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.747+0000 7ff2beffd640 1 -- 172.21.15.26:0/900769113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0038070 con 0x7ff2c00c88f0 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.747+0000 7ff2beffd640 1 -- 172.21.15.26:0/900769113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b002f050 con 0x7ff2c00c88f0 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.747+0000 7ff2beffd640 1 -- 172.21.15.26:0/900769113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0031970 con 0x7ff2c00c88f0 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 -- 172.21.15.26:0/900769113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c00c88f0 msgr2=0x7ff2c0155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 --2- 172.21.15.26:0/900769113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c00c88f0 0x7ff2c0155210 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff2c008cb10 tx=0x7ff2b002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 -- 172.21.15.26:0/900769113 shutdown_connections 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 --2- 172.21.15.26:0/900769113 >> v2:172.21.15.117:3300/0 conn(0x7ff2c00c58b0 0x7ff2c0155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 --2- 172.21.15.26:0/900769113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c00c88f0 0x7ff2c0155210 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 --2- 172.21.15.26:0/900769113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c01527f0 0x7ff2c00c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 -- 172.21.15.26:0/900769113 >> 172.21.15.26:0/900769113 conn(0x7ff2c00ac580 msgr2=0x7ff2c00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 -- 172.21.15.26:0/900769113 shutdown_connections 2024-03-06T07:23:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.748+0000 7ff2c6271640 1 -- 172.21.15.26:0/900769113 wait complete. 2024-03-06T07:23:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.749+0000 7ff2c6271640 1 Processor -- start 2024-03-06T07:23:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.749+0000 7ff2c6271640 1 -- start start 2024-03-06T07:23:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c6271640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c00c58b0 0x7ff2c01eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c6271640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c00c88f0 0x7ff2c006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c6271640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c01527f0 0x7ff2c01ead90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c6271640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c0158c10 con 0x7ff2c01527f0 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c6271640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c0158930 con 0x7ff2c00c88f0 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c6271640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c0158ef0 con 0x7ff2c00c58b0 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c01527f0 0x7ff2c01ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c01527f0 0x7ff2c01ead90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39368/0 (socket says 172.21.15.26:39368) 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 -- 172.21.15.26:0/1256027384 learned_addr learned my addr 172.21.15.26:0/1256027384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2bf7fe640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c00c88f0 0x7ff2c006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2bffff640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c00c58b0 0x7ff2c01eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 -- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c00c58b0 msgr2=0x7ff2c01eeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c00c58b0 0x7ff2c01eeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 -- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c00c88f0 msgr2=0x7ff2c006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c00c88f0 0x7ff2c006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.750+0000 7ff2c480e640 1 -- 172.21.15.26:0/1256027384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c01ef1e0 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.751+0000 7ff2c480e640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c01527f0 0x7ff2c01ead90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800cc60 tx=0x7ff2b800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.751+0000 7ff2bcff9640 1 -- 172.21.15.26:0/1256027384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800cc90 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.751+0000 7ff2bcff9640 1 -- 172.21.15.26:0/1256027384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b800ce30 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.751+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c0147f90 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.751+0000 7ff2bcff9640 1 -- 172.21.15.26:0/1256027384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8003b90 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.751+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c01ee4a0 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.752+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c004f330 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.752+0000 7ff2bcff9640 1 -- 172.21.15.26:0/1256027384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2b800f3f0 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.753+0000 7ff2bcff9640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.753+0000 7ff2bcff9640 1 -- 172.21.15.26:0/1256027384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff2b808a410 con 0x7ff2c01527f0 2024-03-06T07:23:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.753+0000 7ff2bffff640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.753+0000 7ff2bffff640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b4d0 0x7ff2a006d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0146100 tx=0x7ff2a8009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.757+0000 7ff2bcff9640 1 -- 172.21.15.26:0/1256027384 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2b8014030 con 0x7ff2c01527f0 2024-03-06T07:23:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.937+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff2c01563e0 con 0x7ff2c01527f0 2024-03-06T07:23:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.940+0000 7ff2bcff9640 1 -- 172.21.15.26:0/1256027384 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff2b8058d60 con 0x7ff2c01527f0 2024-03-06T07:23:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b4d0 msgr2=0x7ff2a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b4d0 0x7ff2a006d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0146100 tx=0x7ff2a8009110 comp rx=0 tx=0).stop 2024-03-06T07:23:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c01527f0 msgr2=0x7ff2c01ead90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c01527f0 0x7ff2c01ead90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800cc60 tx=0x7ff2b800a5c0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 shutdown_connections 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c01527f0 0x7ff2c01ead90 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c00c88f0 0x7ff2c006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 --2- 172.21.15.26:0/1256027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c00c58b0 0x7ff2c01eeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 >> 172.21.15.26:0/1256027384 conn(0x7ff2c00ac580 msgr2=0x7ff2c014aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 shutdown_connections 2024-03-06T07:23:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:48.942+0000 7ff2c6271640 1 -- 172.21.15.26:0/1256027384 wait complete. 2024-03-06T07:23:48.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:23:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.057+0000 7f5f4382a640 1 Processor -- start 2024-03-06T07:23:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.058+0000 7f5f4382a640 1 -- start start 2024-03-06T07:23:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.058+0000 7f5f4382a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 0x7f5f3c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.058+0000 7f5f4382a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c0c3ef0 con 0x7f5f3c06c900 2024-03-06T07:23:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.058+0000 7f5f4382a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c1556d0 con 0x7f5f3c0c5990 2024-03-06T07:23:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.058+0000 7f5f4382a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c1559b0 con 0x7f5f3c0ab170 2024-03-06T07:23:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.058+0000 7f5f40dc5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5f3c06c900 0x7f5f3c0a9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60464/0 (socket says 172.21.15.26:60464) 2024-03-06T07:23:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.058+0000 7f5f40dc5640 1 -- 172.21.15.26:0/3177140881 learned_addr learned my addr 172.21.15.26:0/3177140881 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:49.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 0x7f5f3c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448160627 0 0) 0x7f5f3c0c3ef0 con 0x7f5f3c06c900 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f28003520 con 0x7f5f3c06c900 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294151122 0 0) 0x7f5f3c1556d0 con 0x7f5f3c0c5990 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f3c0c3ef0 con 0x7f5f3c0c5990 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2285250524 0 0) 0x7f5f28003520 con 0x7f5f3c06c900 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 msgr2=0x7f5f3c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 0x7f5f3c0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 >> v1:172.21.15.117:6789/0 conn(0x7f5f3c0c5990 legacy=0x7f5f3c06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c155c90 con 0x7f5f3c06c900 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f5f2c001670 con 0x7f5f3c06c900 2024-03-06T07:23:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 >> v1:172.21.15.26:6789/0 conn(0x7f5f3c06c900 legacy=0x7f5f3c0a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 0x7f5f28003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f28004120 0x7f5f28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f28004d40 0x7f5f280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c1559b0 con 0x7f5f28004d40 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2800bb00 con 0x7f5f28004120 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2800bd00 con 0x7f5f3c0ab170 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f28004d40 0x7f5f280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f415c6640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 0x7f5f28003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f40dc5640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f28004120 0x7f5f28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f41dc7640 1 -- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 msgr2=0x7f5f28003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c0ab170 0x7f5f28003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f41dc7640 1 -- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f28004120 msgr2=0x7f5f28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f28004120 0x7f5f28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.059+0000 7f5f41dc7640 1 -- 172.21.15.26:0/3177140881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28003520 con 0x7f5f28004d40 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.060+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f28004d40 0x7f5f280051c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009fc0 tx=0x7f5f28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320e82a8650eb4da server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.060+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38039070 con 0x7f5f28004d40 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.060+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3802cc20 con 0x7f5f28004d40 2024-03-06T07:23:49.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.060+0000 7f5f33fff640 1 -- 172.21.15.26:0/3177140881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38009c10 con 0x7f5f28004d40 2024-03-06T07:23:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.061+0000 7f5f4382a640 1 -- 172.21.15.26:0/3177140881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f28004d40 msgr2=0x7f5f280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.061+0000 7f5f4382a640 1 --2- 172.21.15.26:0/3177140881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f28004d40 0x7f5f280051c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009fc0 tx=0x7f5f28005830 comp rx=0 tx=0).stop 2024-03-06T07:23:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.061+0000 7f5f41dc7640 1 -- 172.21.15.26:0/3177140881 reap_dead start 2024-03-06T07:23:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.061+0000 7f5f4382a640 1 -- 172.21.15.26:0/3177140881 shutdown_connections 2024-03-06T07:23:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.061+0000 7f5f4382a640 1 -- 172.21.15.26:0/3177140881 >> 172.21.15.26:0/3177140881 conn(0x7f5f3c0c6710 msgr2=0x7f5f3c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.061+0000 7f5f4382a640 1 -- 172.21.15.26:0/3177140881 shutdown_connections 2024-03-06T07:23:49.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.061+0000 7f5f4382a640 1 -- 172.21.15.26:0/3177140881 wait complete. 2024-03-06T07:23:49.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.062+0000 7f5f4382a640 1 Processor -- start 2024-03-06T07:23:49.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.062+0000 7f5f4382a640 1 -- start start 2024-03-06T07:23:49.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f4382a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c1537b0 0x7f5f3c145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f4382a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c144210 0x7f5f3c1450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f4382a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f3c145630 0x7f5f3c13e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f4382a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c156610 con 0x7f5f3c145630 2024-03-06T07:23:49.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f4382a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c156490 con 0x7f5f3c1537b0 2024-03-06T07:23:49.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f4382a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0c3af0 con 0x7f5f3c144210 2024-03-06T07:23:49.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f3c145630 0x7f5f3c13e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f3c145630 0x7f5f3c13e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39388/0 (socket says 172.21.15.26:39388) 2024-03-06T07:23:49.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 -- 172.21.15.26:0/1675468368 learned_addr learned my addr 172.21.15.26:0/1675468368 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:49.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f415c6640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c1537b0 0x7f5f3c145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f40dc5640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c144210 0x7f5f3c1450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 -- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c144210 msgr2=0x7f5f3c1450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c144210 0x7f5f3c1450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 -- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c1537b0 msgr2=0x7f5f3c145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c1537b0 0x7f5f3c145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 -- 172.21.15.26:0/1675468368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c155640 con 0x7f5f3c145630 2024-03-06T07:23:49.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.063+0000 7f5f41dc7640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f3c145630 0x7f5f3c13e610 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e230 tx=0x7f5f38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.064+0000 7f5f31ffb640 1 -- 172.21.15.26:0/1675468368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38039030 con 0x7f5f3c145630 2024-03-06T07:23:49.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.064+0000 7f5f31ffb640 1 -- 172.21.15.26:0/1675468368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38043070 con 0x7f5f3c145630 2024-03-06T07:23:49.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.064+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c154310 con 0x7f5f3c145630 2024-03-06T07:23:49.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.064+0000 7f5f31ffb640 1 -- 172.21.15.26:0/1675468368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38009c10 con 0x7f5f3c145630 2024-03-06T07:23:49.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.064+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c155360 con 0x7f5f3c145630 2024-03-06T07:23:49.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.065+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c04f330 con 0x7f5f3c145630 2024-03-06T07:23:49.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.065+0000 7f5f31ffb640 1 -- 172.21.15.26:0/1675468368 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f38005d10 con 0x7f5f3c145630 2024-03-06T07:23:49.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.066+0000 7f5f31ffb640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f2006b4d0 0x7f5f2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.066+0000 7f5f31ffb640 1 -- 172.21.15.26:0/1675468368 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5f380ac140 con 0x7f5f3c145630 2024-03-06T07:23:49.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.066+0000 7f5f415c6640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f2006b4d0 0x7f5f2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.067+0000 7f5f415c6640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f2006b4d0 0x7f5f2006d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5f34001910 tx=0x7f5f34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.070+0000 7f5f31ffb640 1 -- 172.21.15.26:0/1675468368 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5f3807ab40 con 0x7f5f3c145630 2024-03-06T07:23:49.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.250+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7f5f3c06dd40 con 0x7f5f3c145630 2024-03-06T07:23:49.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.253+0000 7f5f31ffb640 1 -- 172.21.15.26:0/1675468368 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v34) v1 ==== 96+0+0 (secure 0 0 0) 0x7f5f3807ebe0 con 0x7f5f3c145630 2024-03-06T07:23:49.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.255+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f2006b4d0 msgr2=0x7f5f2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.255+0000 7f5f4382a640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f2006b4d0 0x7f5f2006d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5f34001910 tx=0x7f5f34009040 comp rx=0 tx=0).stop 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.255+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f3c145630 msgr2=0x7f5f3c13e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.255+0000 7f5f4382a640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f3c145630 0x7f5f3c13e610 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e230 tx=0x7f5f38009750 comp rx=0 tx=0).stop 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.255+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 shutdown_connections 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.256+0000 7f5f4382a640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5f2006b4d0 0x7f5f2006d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.256+0000 7f5f4382a640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f3c145630 0x7f5f3c13e610 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.256+0000 7f5f4382a640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c144210 0x7f5f3c1450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.256+0000 7f5f4382a640 1 --2- 172.21.15.26:0/1675468368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c1537b0 0x7f5f3c145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.256+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 >> 172.21.15.26:0/1675468368 conn(0x7f5f3c0c6710 msgr2=0x7f5f3c14ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:49.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.256+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 shutdown_connections 2024-03-06T07:23:49.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.256+0000 7f5f4382a640 1 -- 172.21.15.26:0/1675468368 wait complete. 2024-03-06T07:23:49.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime' 2024-03-06T07:23:49.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.369+0000 7f38f8d65640 1 Processor -- start 2024-03-06T07:23:49.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.369+0000 7f38f8d65640 1 -- start start 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f8d65640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f38f40c3280 0x7f38f40c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f8d65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 0x7f38f4155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f8d65640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f4158800 con 0x7f38f406d9f0 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f8d65640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f4158ae0 con 0x7f38f40c3280 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f8d65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4158dc0 con 0x7f38f41551e0 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38f406d9f0 0x7f38f40a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60476/0 (socket says 172.21.15.26:60476) 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f259c640 1 -- 172.21.15.26:0/2170257050 learned_addr learned my addr 172.21.15.26:0/2170257050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f2d9d640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 0x7f38f4155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/2170257050 >> v2:172.21.15.117:3300/0 conn(0x7f38f40c3280 0x7f38f40c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509334555 0 0) 0x7f38f4158800 con 0x7f38f406d9f0 2024-03-06T07:23:49.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e0003520 con 0x7f38f406d9f0 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 90822213 0 0) 0x7f38e0003520 con 0x7f38f406d9f0 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 msgr2=0x7f38f4155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 0x7f38f4155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 >> v2:172.21.15.117:3300/0 conn(0x7f38f40c3280 msgr2=0x7f38f40c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 --2- 172.21.15.26:0/2170257050 >> v2:172.21.15.117:3300/0 conn(0x7f38f40c3280 0x7f38f40c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f41590a0 con 0x7f38f406d9f0 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f38e4001670 con 0x7f38f406d9f0 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 >> v1:172.21.15.26:6789/0 conn(0x7f38f406d9f0 legacy=0x7f38f40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e00039b0 0x7f38e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 0x7f38e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4158ae0 con 0x7f38e0004800 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4158dc0 con 0x7f38e00039b0 2024-03-06T07:23:49.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e000b9c0 con 0x7f38f41551e0 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.370+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38f259c640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e00039b0 0x7f38e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38f2d9d640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 0x7f38e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38e9d9b640 1 -- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 msgr2=0x7f38e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f41551e0 0x7f38e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38e9d9b640 1 -- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e00039b0 msgr2=0x7f38e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e00039b0 0x7f38e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38e9d9b640 1 -- 172.21.15.26:0/2170257050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0003520 con 0x7f38e0004800 2024-03-06T07:23:49.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.371+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38e0004c80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009af0 tx=0x7f38e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b412b80da4f9ce server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc039070 con 0x7f38e0004800 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc02c710 con 0x7f38e0004800 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f1d9b640 1 -- 172.21.15.26:0/2170257050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc009690 con 0x7f38e0004800 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f8d65640 1 -- 172.21.15.26:0/2170257050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 msgr2=0x7f38e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f8d65640 1 --2- 172.21.15.26:0/2170257050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38e0004c80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009af0 tx=0x7f38e0005830 comp rx=0 tx=0).stop 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f2d9d640 1 -- 172.21.15.26:0/2170257050 reap_dead start 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f8d65640 1 -- 172.21.15.26:0/2170257050 shutdown_connections 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f8d65640 1 -- 172.21.15.26:0/2170257050 >> 172.21.15.26:0/2170257050 conn(0x7f38f40aa630 msgr2=0x7f38f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f8d65640 1 -- 172.21.15.26:0/2170257050 shutdown_connections 2024-03-06T07:23:49.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.372+0000 7f38f8d65640 1 -- 172.21.15.26:0/2170257050 wait complete. 2024-03-06T07:23:49.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 Processor -- start 2024-03-06T07:23:49.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 -- start start 2024-03-06T07:23:49.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38f40ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f414f520 0x7f38f41e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38f41e2030 0x7f38f406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f406df50 con 0x7f38e0004800 2024-03-06T07:23:49.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f406ddd0 con 0x7f38f41e2030 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f8d65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40c6910 con 0x7f38f414f520 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38f40ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38f40ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39408/0 (socket says 172.21.15.26:39408) 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.374+0000 7f38f259c640 1 -- 172.21.15.26:0/175445 learned_addr learned my addr 172.21.15.26:0/175445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f414f520 0x7f38f41e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f2d9d640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38f41e2030 0x7f38f406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f259c640 1 -- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f414f520 msgr2=0x7f38f41e1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f259c640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f414f520 0x7f38f41e1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f259c640 1 -- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38f41e2030 msgr2=0x7f38f406e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f259c640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38f41e2030 0x7f38f406e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f259c640 1 -- 172.21.15.26:0/175445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f414f2b0 con 0x7f38e0004800 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f414f520 0x7f38f41e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f2d9d640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38f41e2030 0x7f38f406e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:49.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f259c640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38f40ab890 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f38e4007370 tx=0x7f38e4007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38eb7fe640 1 -- 172.21.15.26:0/175445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e400f9c0 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38eb7fe640 1 -- 172.21.15.26:0/175445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4018070 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f414efd0 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38eb7fe640 1 -- 172.21.15.26:0/175445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e400fda0 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.375+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f41eb8a0 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.376+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f404f330 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.376+0000 7f38eb7fe640 1 -- 172.21.15.26:0/175445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38e401d460 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.377+0000 7f38eb7fe640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f38d406b4d0 0x7f38d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.377+0000 7f38eb7fe640 1 -- 172.21.15.26:0/175445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f38e408a1a0 con 0x7f38e0004800 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.377+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f38d406b4d0 0x7f38d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.378+0000 7f38e9d9b640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f38d406b4d0 0x7f38d406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f38dc036000 tx=0x7f38dc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.381+0000 7f38eb7fe640 1 -- 172.21.15.26:0/175445 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f38e40591d0 con 0x7f38e0004800 2024-03-06T07:23:49.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:49.629+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]} v 0) v1 -- 0x7f38f40ab160 con 0x7f38e0004800 2024-03-06T07:23:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.620+0000 7f38eb7fe640 1 -- 172.21.15.26:0/175445 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]}]=0 v229) v1 ==== 146+0+0 (secure 0 0 0) 0x7f38e4058ad0 con 0x7f38e0004800 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f38d406b4d0 msgr2=0x7f38d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f38d406b4d0 0x7f38d406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f38dc036000 tx=0x7f38dc030000 comp rx=0 tx=0).stop 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 msgr2=0x7f38f40ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38f40ab890 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f38e4007370 tx=0x7f38e4007830 comp rx=0 tx=0).stop 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 shutdown_connections 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f38d406b4d0 0x7f38d406d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38f41e2030 0x7f38f406e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38f414f520 0x7f38f41e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 --2- 172.21.15.26:0/175445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38e0004800 0x7f38f40ab890 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 >> 172.21.15.26:0/175445 conn(0x7f38f40aa630 msgr2=0x7f38f40c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 shutdown_connections 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.623+0000 7f38f8d65640 1 -- 172.21.15.26:0/175445 wait complete. 2024-03-06T07:23:50.639 INFO:tasks.cephfs_test_runner:That setting mds_join_fs at runtime affects the cluster layout. ... ok 2024-03-06T07:23:50.639 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_standby_replay (tasks.cephfs.test_failover.TestClusterAffinity) That a standby-replay daemon with weak affinity is replaced by a stronger one. 2024-03-06T07:23:50.639 INFO:tasks.cephfs_test_runner:test_join_fs_standby_replay (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:23:50.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.744+0000 7effdcdf6640 1 Processor -- start 2024-03-06T07:23:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.744+0000 7effdcdf6640 1 -- start start 2024-03-06T07:23:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effdcdf6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effd814f360 0x7effd8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effdcdf6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7effd80c88c0 0x7effd80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effdcdf6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd80c73d0 con 0x7effd80c5880 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effdcdf6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd80c8dd0 con 0x7effd814f360 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effdcdf6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd80be2e0 con 0x7effd80c88c0 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effd659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effd814f360 0x7effd8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effd6d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7effd80c5880 0x7effd8158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60488/0 (socket says 172.21.15.26:60488) 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.745+0000 7effd6d9d640 1 -- 172.21.15.26:0/3570685373 learned_addr learned my addr 172.21.15.26:0/3570685373 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.746+0000 7effd5d9b640 1 --2- 172.21.15.26:0/3570685373 >> v2:172.21.15.117:3300/0 conn(0x7effd80c88c0 0x7effd80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.746+0000 7effd659c640 1 -- 172.21.15.26:0/3570685373 >> v2:172.21.15.117:3300/0 conn(0x7effd80c88c0 msgr2=0x7effd80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.746+0000 7effd659c640 1 --2- 172.21.15.26:0/3570685373 >> v2:172.21.15.117:3300/0 conn(0x7effd80c88c0 0x7effd80c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.746+0000 7effd659c640 1 -- 172.21.15.26:0/3570685373 >> v1:172.21.15.26:6789/0 conn(0x7effd80c5880 legacy=0x7effd8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.746+0000 7effd659c640 1 -- 172.21.15.26:0/3570685373 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80c7670 con 0x7effd814f360 2024-03-06T07:23:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effd659c640 1 --2- 172.21.15.26:0/3570685373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd814f360 0x7effd8155c10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7effd80c84c0 tx=0x7effd002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8021745ca8394736 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effd559a640 1 -- 172.21.15.26:0/3570685373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd00315a0 con 0x7effd814f360 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effd559a640 1 -- 172.21.15.26:0/3570685373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0037070 con 0x7effd814f360 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effd559a640 1 -- 172.21.15.26:0/3570685373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0031960 con 0x7effd814f360 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 -- 172.21.15.26:0/3570685373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd814f360 msgr2=0x7effd8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3570685373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd814f360 0x7effd8155c10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7effd80c84c0 tx=0x7effd002b840 comp rx=0 tx=0).stop 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 -- 172.21.15.26:0/3570685373 shutdown_connections 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3570685373 >> v2:172.21.15.117:3300/0 conn(0x7effd80c88c0 0x7effd80c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3570685373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd814f360 0x7effd8155c10 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 -- 172.21.15.26:0/3570685373 >> 172.21.15.26:0/3570685373 conn(0x7effd80ac570 msgr2=0x7effd80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 -- 172.21.15.26:0/3570685373 shutdown_connections 2024-03-06T07:23:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.747+0000 7effdcdf6640 1 -- 172.21.15.26:0/3570685373 wait complete. 2024-03-06T07:23:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 Processor -- start 2024-03-06T07:23:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 -- start start 2024-03-06T07:23:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd80c5880 0x7effd814bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd80c88c0 0x7effd80aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd814f360 0x7effd80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd8159270 con 0x7effd80c5880 2024-03-06T07:23:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd80c7c30 con 0x7effd814f360 2024-03-06T07:23:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effdcdf6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd81593f0 con 0x7effd80c88c0 2024-03-06T07:23:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effd659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd80c5880 0x7effd814bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effd659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd80c5880 0x7effd814bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39428/0 (socket says 172.21.15.26:39428) 2024-03-06T07:23:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.749+0000 7effd659c640 1 -- 172.21.15.26:0/3078461073 learned_addr learned my addr 172.21.15.26:0/3078461073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd6d9d640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd814f360 0x7effd80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd5d9b640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd80c88c0 0x7effd80aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd659c640 1 -- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd80c88c0 msgr2=0x7effd80aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd659c640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd80c88c0 0x7effd80aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd659c640 1 -- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd814f360 msgr2=0x7effd80ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd659c640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd814f360 0x7effd80ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd659c640 1 -- 172.21.15.26:0/3078461073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd81ee6c0 con 0x7effd80c5880 2024-03-06T07:23:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effd659c640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd80c5880 0x7effd814bc30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7effd002be50 tx=0x7effd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effc6ffd640 1 -- 172.21.15.26:0/3078461073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0032630 con 0x7effd80c5880 2024-03-06T07:23:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effc6ffd640 1 -- 172.21.15.26:0/3078461073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd00327d0 con 0x7effd80c5880 2024-03-06T07:23:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd81ee9a0 con 0x7effd80c5880 2024-03-06T07:23:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effc6ffd640 1 -- 172.21.15.26:0/3078461073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0032ba0 con 0x7effd80c5880 2024-03-06T07:23:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.750+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd80c0c10 con 0x7effd80c5880 2024-03-06T07:23:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.751+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd80c6530 con 0x7effd80c5880 2024-03-06T07:23:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.751+0000 7effc6ffd640 1 -- 172.21.15.26:0/3078461073 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effd004c020 con 0x7effd80c5880 2024-03-06T07:23:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.752+0000 7effc6ffd640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb806b4d0 0x7effb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.752+0000 7effc6ffd640 1 -- 172.21.15.26:0/3078461073 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7effd002f020 con 0x7effd80c5880 2024-03-06T07:23:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.752+0000 7effd5d9b640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb806b4d0 0x7effb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.753+0000 7effd5d9b640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb806b4d0 0x7effb806d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7effd8068fe0 tx=0x7effcc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.756+0000 7effc6ffd640 1 -- 172.21.15.26:0/3078461073 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7effd80c6530 con 0x7effd80c5880 2024-03-06T07:23:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.938+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effd80c0e50 con 0x7effd80c5880 2024-03-06T07:23:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.939+0000 7effc6ffd640 1 -- 172.21.15.26:0/3078461073 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13438 (secure 0 0 0) 0x7effd80c0e50 con 0x7effd80c5880 2024-03-06T07:23:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:50.942 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:23:48.614065+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/593719862":"2024-03-07T07:23:45.605853+0000","172.21.15.117:6834/593719862":"2024-03-07T07:23:45.605853+0000","172.21.15.117:6837/1940022472":"2024-03-07T07:23:37.000697+0000","172.21.15.117:6835/1940022472":"2024-03-07T07:23:37.000697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb806b4d0 msgr2=0x7effb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb806b4d0 0x7effb806d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7effd8068fe0 tx=0x7effcc008040 comp rx=0 tx=0).stop 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd80c5880 msgr2=0x7effd814bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd80c5880 0x7effd814bc30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7effd002be50 tx=0x7effd00046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 shutdown_connections 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb806b4d0 0x7effb806d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd814f360 0x7effd80ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd80c88c0 0x7effd80aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 --2- 172.21.15.26:0/3078461073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd80c5880 0x7effd814bc30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 >> 172.21.15.26:0/3078461073 conn(0x7effd80ac570 msgr2=0x7effd80ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 shutdown_connections 2024-03-06T07:23:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:50.941+0000 7effdcdf6640 1 -- 172.21.15.26:0/3078461073 wait complete. 2024-03-06T07:23:50.955 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:50.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.057+0000 7fa554ac2640 1 Processor -- start 2024-03-06T07:23:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.057+0000 7fa554ac2640 1 -- start start 2024-03-06T07:23:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa554ac2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa550155c30 0x7fa55006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa554ac2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5500c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa554ac2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5500b0450 con 0x7fa5500b0070 2024-03-06T07:23:51.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa554ac2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55006cfb0 con 0x7fa550155c30 2024-03-06T07:23:51.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa554ac2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55006c020 con 0x7fa5500c5380 2024-03-06T07:23:51.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa547fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa550155c30 0x7fa55006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa54e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa5500b0070 0x7fa5500c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60502/0 (socket says 172.21.15.26:60502) 2024-03-06T07:23:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa54e59c640 1 -- 172.21.15.26:0/3602562948 learned_addr learned my addr 172.21.15.26:0/3602562948 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.058+0000 7fa54ed9d640 1 --2- 172.21.15.26:0/3602562948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5500c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.059+0000 7fa547fff640 1 -- 172.21.15.26:0/3602562948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 msgr2=0x7fa5500c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.059+0000 7fa547fff640 1 --2- 172.21.15.26:0/3602562948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5500c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.059+0000 7fa547fff640 1 -- 172.21.15.26:0/3602562948 >> v1:172.21.15.26:6789/0 conn(0x7fa5500b0070 legacy=0x7fa5500c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.059+0000 7fa547fff640 1 -- 172.21.15.26:0/3602562948 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5501560e0 con 0x7fa550155c30 2024-03-06T07:23:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.059+0000 7fa54dd9b640 1 -- 172.21.15.26:0/3602562948 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558187735 0 0) 0x7fa540001110 con 0x7fa5500b0070 2024-03-06T07:23:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.059+0000 7fa547fff640 1 --2- 172.21.15.26:0/3602562948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa550155c30 0x7fa55006ca70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa5380097a0 tx=0x7fa53802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82159aa5a863b551 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa54dd9b640 1 -- 172.21.15.26:0/3602562948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380315a0 con 0x7fa550155c30 2024-03-06T07:23:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa54dd9b640 1 -- 172.21.15.26:0/3602562948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538037070 con 0x7fa550155c30 2024-03-06T07:23:51.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa54dd9b640 1 -- 172.21.15.26:0/3602562948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538031960 con 0x7fa550155c30 2024-03-06T07:23:51.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 -- 172.21.15.26:0/3602562948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa550155c30 msgr2=0x7fa55006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 --2- 172.21.15.26:0/3602562948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa550155c30 0x7fa55006ca70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa5380097a0 tx=0x7fa53802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:51.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 -- 172.21.15.26:0/3602562948 shutdown_connections 2024-03-06T07:23:51.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 --2- 172.21.15.26:0/3602562948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa550155c30 0x7fa55006ca70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 --2- 172.21.15.26:0/3602562948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5500c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 -- 172.21.15.26:0/3602562948 >> 172.21.15.26:0/3602562948 conn(0x7fa5500ac590 msgr2=0x7fa5500ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 -- 172.21.15.26:0/3602562948 shutdown_connections 2024-03-06T07:23:51.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.060+0000 7fa554ac2640 1 -- 172.21.15.26:0/3602562948 wait complete. 2024-03-06T07:23:51.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.061+0000 7fa554ac2640 1 Processor -- start 2024-03-06T07:23:51.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.061+0000 7fa554ac2640 1 -- start start 2024-03-06T07:23:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa554ac2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5500b0070 0x7fa5501df490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa554ac2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5501576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa554ac2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa550155c30 0x7fa5501dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa554ac2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55006c270 con 0x7fa5500b0070 2024-03-06T07:23:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa554ac2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500c7ad0 con 0x7fa550155c30 2024-03-06T07:23:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa554ac2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55006c550 con 0x7fa5500c5380 2024-03-06T07:23:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5500b0070 0x7fa5501df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5500b0070 0x7fa5501df490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39434/0 (socket says 172.21.15.26:39434) 2024-03-06T07:23:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 -- 172.21.15.26:0/1684943102 learned_addr learned my addr 172.21.15.26:0/1684943102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa547fff640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5501576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 -- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 msgr2=0x7fa5501576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5501576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54ed9d640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa550155c30 0x7fa5501dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 -- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa550155c30 msgr2=0x7fa5501dffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa550155c30 0x7fa5501dffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.062+0000 7fa54e59c640 1 -- 172.21.15.26:0/1684943102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5501594d0 con 0x7fa5500b0070 2024-03-06T07:23:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.063+0000 7fa54e59c640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5500b0070 0x7fa5501df490 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c460 tx=0x7fa54000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.063+0000 7fa546ffd640 1 -- 172.21.15.26:0/1684943102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540018070 con 0x7fa5500b0070 2024-03-06T07:23:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.063+0000 7fa546ffd640 1 -- 172.21.15.26:0/1684943102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa540007500 con 0x7fa5500b0070 2024-03-06T07:23:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.063+0000 7fa546ffd640 1 -- 172.21.15.26:0/1684943102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540004a20 con 0x7fa5500b0070 2024-03-06T07:23:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.063+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5501de9b0 con 0x7fa5500b0070 2024-03-06T07:23:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.063+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5501def70 con 0x7fa5500b0070 2024-03-06T07:23:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.064+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55004f330 con 0x7fa5500b0070 2024-03-06T07:23:51.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.064+0000 7fa546ffd640 1 -- 172.21.15.26:0/1684943102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa54001b070 con 0x7fa5500b0070 2024-03-06T07:23:51.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.065+0000 7fa546ffd640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa53006b4d0 0x7fa53006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.065+0000 7fa546ffd640 1 -- 172.21.15.26:0/1684943102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa540089320 con 0x7fa5500b0070 2024-03-06T07:23:51.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.065+0000 7fa547fff640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa53006b4d0 0x7fa53006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.066+0000 7fa547fff640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa53006b4d0 0x7fa53006d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa538036000 tx=0x7fa538008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.069+0000 7fa546ffd640 1 -- 172.21.15.26:0/1684943102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa5500b0450 con 0x7fa5500b0070 2024-03-06T07:23:51.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.246+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5501df1b0 con 0x7fa5500b0070 2024-03-06T07:23:51.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.247+0000 7fa546ffd640 1 -- 172.21.15.26:0/1684943102 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa540058350 con 0x7fa5500b0070 2024-03-06T07:23:51.250 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:23:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa53006b4d0 msgr2=0x7fa53006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa53006b4d0 0x7fa53006d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa538036000 tx=0x7fa538008030 comp rx=0 tx=0).stop 2024-03-06T07:23:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5500b0070 msgr2=0x7fa5501df490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5500b0070 0x7fa5501df490 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c460 tx=0x7fa54000c920 comp rx=0 tx=0).stop 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 shutdown_connections 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa53006b4d0 0x7fa53006d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa550155c30 0x7fa5501dffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5500c5380 0x7fa5501576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 --2- 172.21.15.26:0/1684943102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5500b0070 0x7fa5501df490 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 >> 172.21.15.26:0/1684943102 conn(0x7fa5500ac590 msgr2=0x7fa5500aa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 shutdown_connections 2024-03-06T07:23:51.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.249+0000 7fa554ac2640 1 -- 172.21.15.26:0/1684943102 wait complete. 2024-03-06T07:23:51.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay' 2024-03-06T07:23:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.361+0000 7fb2ccfb5640 1 Processor -- start 2024-03-06T07:23:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.361+0000 7fb2ccfb5640 1 -- start start 2024-03-06T07:23:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.362+0000 7fb2ccfb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 0x7fb2c8153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.362+0000 7fb2ccfb5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c8070aa0 con 0x7fb2c8156ea0 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.362+0000 7fb2ccfb5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c8070d80 con 0x7fb2c80acc00 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.362+0000 7fb2ccfb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8157320 con 0x7fb2c81506b0 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.362+0000 7fb2c5d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb2c8156ea0 0x7fb2c80702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60510/0 (socket says 172.21.15.26:60510) 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.362+0000 7fb2c5d9b640 1 -- 172.21.15.26:0/1720108898 learned_addr learned my addr 172.21.15.26:0/1720108898 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.362+0000 7fb2c659c640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 0x7fb2c8153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573056371 0 0) 0x7fb2c8070aa0 con 0x7fb2c8156ea0 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b4003520 con 0x7fb2c8156ea0 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28053168 0 0) 0x7fb2c8070d80 con 0x7fb2c80acc00 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c8070aa0 con 0x7fb2c80acc00 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747219556 0 0) 0x7fb2b4003520 con 0x7fb2c8156ea0 2024-03-06T07:23:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 msgr2=0x7fb2c8153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 0x7fb2c8153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 >> v1:172.21.15.117:6789/0 conn(0x7fb2c80acc00 legacy=0x7fb2c80c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c81566a0 con 0x7fb2c8156ea0 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fb2b0001670 con 0x7fb2c8156ea0 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 >> v1:172.21.15.26:6789/0 conn(0x7fb2c8156ea0 legacy=0x7fb2c80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 0x7fb2b4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b40040d0 0x7fb2b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8157320 con 0x7fb2b4004d40 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b400baa0 con 0x7fb2b40040d0 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b400bca0 con 0x7fb2c81506b0 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c6d9d640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c659c640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 0x7fb2b4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c5d9b640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b40040d0 0x7fb2b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c6d9d640 1 -- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 msgr2=0x7fb2b4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c6d9d640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81506b0 0x7fb2b4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c6d9d640 1 -- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b40040d0 msgr2=0x7fb2b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c6d9d640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b40040d0 0x7fb2b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.363+0000 7fb2c6d9d640 1 -- 172.21.15.26:0/1720108898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4003520 con 0x7fb2b4004d40 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.364+0000 7fb2c6d9d640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2b40051c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8001fd0 tx=0x7fb2b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59f9145c779e9ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.364+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8001bf0 con 0x7fb2b4004d40 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.364+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b8003b70 con 0x7fb2b4004d40 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.364+0000 7fb2c559a640 1 -- 172.21.15.26:0/1720108898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80050d0 con 0x7fb2b4004d40 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.365+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/1720108898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 msgr2=0x7fb2b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.365+0000 7fb2ccfb5640 1 --2- 172.21.15.26:0/1720108898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2b40051c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8001fd0 tx=0x7fb2b4005830 comp rx=0 tx=0).stop 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.365+0000 7fb2c6d9d640 1 -- 172.21.15.26:0/1720108898 reap_dead start 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.365+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/1720108898 shutdown_connections 2024-03-06T07:23:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.365+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/1720108898 >> 172.21.15.26:0/1720108898 conn(0x7fb2c806b1a0 msgr2=0x7fb2c806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.365+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/1720108898 shutdown_connections 2024-03-06T07:23:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.365+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/1720108898 wait complete. 2024-03-06T07:23:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.366+0000 7fb2ccfb5640 1 Processor -- start 2024-03-06T07:23:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.366+0000 7fb2ccfb5640 1 -- start start 2024-03-06T07:23:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2c80c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c80c6f30 0x7fb2c81e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81e94b0 0x7fb2c8146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8156a10 con 0x7fb2b4004d40 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8156450 con 0x7fb2c80c6f30 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80ad090 con 0x7fb2c81e94b0 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2c80c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2c80c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39450/0 (socket says 172.21.15.26:39450) 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 -- 172.21.15.26:0/3329032622 learned_addr learned my addr 172.21.15.26:0/3329032622 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c5d9b640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c80c6f30 0x7fb2c81e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c6d9d640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81e94b0 0x7fb2c8146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 -- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81e94b0 msgr2=0x7fb2c8146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81e94b0 0x7fb2c8146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 -- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c80c6f30 msgr2=0x7fb2c81e8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c80c6f30 0x7fb2c81e8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 -- 172.21.15.26:0/3329032622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8149a20 con 0x7fb2b4004d40 2024-03-06T07:23:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2c659c640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2c80c4280 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00119a0 tx=0x7fb2c0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2beffd640 1 -- 172.21.15.26:0/3329032622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000fa90 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2beffd640 1 -- 172.21.15.26:0/3329032622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c000fc30 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c80c8020 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2beffd640 1 -- 172.21.15.26:0/3329032622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0014440 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.367+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c80c47c0 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.368+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c8069460 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.369+0000 7fb2beffd640 1 -- 172.21.15.26:0/3329032622 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2c0012040 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.369+0000 7fb2beffd640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a806b4d0 0x7fb2a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.369+0000 7fb2beffd640 1 -- 172.21.15.26:0/3329032622 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb2c008a700 con 0x7fb2b4004d40 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.369+0000 7fb2c5d9b640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a806b4d0 0x7fb2a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.370+0000 7fb2c5d9b640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a806b4d0 0x7fb2a806d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80c7330 tx=0x7fb2b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.374+0000 7fb2beffd640 1 -- 172.21.15.26:0/3329032622 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb2c8069460 con 0x7fb2b4004d40 2024-03-06T07:23:51.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:51.631+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]} v 0) v1 -- 0x7fb2c804f330 con 0x7fb2b4004d40 2024-03-06T07:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.626+0000 7fb2beffd640 1 -- 172.21.15.26:0/3329032622 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]}]=0 v231) v1 ==== 156+0+0 (secure 0 0 0) 0x7fb2c0019030 con 0x7fb2b4004d40 2024-03-06T07:23:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a806b4d0 msgr2=0x7fb2a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a806b4d0 0x7fb2a806d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80c7330 tx=0x7fb2b0009040 comp rx=0 tx=0).stop 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 msgr2=0x7fb2c80c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2c80c4280 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00119a0 tx=0x7fb2c0011e60 comp rx=0 tx=0).stop 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 shutdown_connections 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2a806b4d0 0x7fb2a806d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2c81e94b0 0x7fb2c8146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c80c6f30 0x7fb2c81e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 --2- 172.21.15.26:0/3329032622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4004d40 0x7fb2c80c4280 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 >> 172.21.15.26:0/3329032622 conn(0x7fb2c806b1a0 msgr2=0x7fb2c80c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 shutdown_connections 2024-03-06T07:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.629+0000 7fb2ccfb5640 1 -- 172.21.15.26:0/3329032622 wait complete. 2024-03-06T07:23:52.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:23:52.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.753+0000 7f3422f55640 1 Processor -- start 2024-03-06T07:23:52.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.753+0000 7f3422f55640 1 -- start start 2024-03-06T07:23:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3422f55640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f341c1529d0 0x7f341c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3422f55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 0x7f341c155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3422f55640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c152db0 con 0x7f341c0c57e0 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3422f55640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0c6dd0 con 0x7f341c1529d0 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3422f55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c6940 con 0x7f341c072cd0 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3413fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f341c0c57e0 0x7f341c155120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60526/0 (socket says 172.21.15.26:60526) 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3413fff640 1 -- 172.21.15.26:0/1891572985 learned_addr learned my addr 172.21.15.26:0/1891572985 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34214f2640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 0x7f341c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3420cf1640 1 --2- 172.21.15.26:0/1891572985 >> v2:172.21.15.117:3300/0 conn(0x7f341c1529d0 0x7f341c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928833835 0 0) 0x7f340c001110 con 0x7f341c0c57e0 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3404003520 con 0x7f341c0c57e0 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608041 0 0) 0x7f3404003520 con 0x7f341c0c57e0 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 msgr2=0x7f341c155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 0x7f341c155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 >> v2:172.21.15.117:3300/0 conn(0x7f341c1529d0 msgr2=0x7f341c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 --2- 172.21.15.26:0/1891572985 >> v2:172.21.15.117:3300/0 conn(0x7f341c1529d0 0x7f341c0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c1590f0 con 0x7f341c0c57e0 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f340c001840 con 0x7f341c0c57e0 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 >> v1:172.21.15.26:6789/0 conn(0x7f341c0c57e0 legacy=0x7f341c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040039b0 0x7f3404003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f3404004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 0x7f34040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c6dd0 con 0x7f3404004800 2024-03-06T07:23:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c6940 con 0x7f34040039b0 2024-03-06T07:23:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340400ba70 con 0x7f341c072cd0 2024-03-06T07:23:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.754+0000 7f3413fff640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f3404004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f3420cf1640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040039b0 0x7f3404003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f34214f2640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 0x7f34040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f3413fff640 1 -- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 msgr2=0x7f34040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f3413fff640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 0x7f34040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f3413fff640 1 -- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040039b0 msgr2=0x7f3404003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f3413fff640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040039b0 0x7f3404003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f3413fff640 1 -- 172.21.15.26:0/1891572985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404003520 con 0x7f3404004800 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.755+0000 7f34214f2640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c072cd0 0x7f34040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f3413fff640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f3404004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c490 tx=0x7f3404005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3698b043ec3a20 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c00c0e0 con 0x7f3404004800 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c030070 con 0x7f3404004800 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f34137fe640 1 -- 172.21.15.26:0/1891572985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0058d0 con 0x7f3404004800 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f3422f55640 1 -- 172.21.15.26:0/1891572985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 msgr2=0x7f3404004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f3422f55640 1 --2- 172.21.15.26:0/1891572985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f3404004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c490 tx=0x7f3404005830 comp rx=0 tx=0).stop 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f34214f2640 1 -- 172.21.15.26:0/1891572985 reap_dead start 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f3422f55640 1 -- 172.21.15.26:0/1891572985 shutdown_connections 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f3422f55640 1 -- 172.21.15.26:0/1891572985 >> 172.21.15.26:0/1891572985 conn(0x7f341c0aa630 msgr2=0x7f341c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f3422f55640 1 -- 172.21.15.26:0/1891572985 shutdown_connections 2024-03-06T07:23:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.756+0000 7f3422f55640 1 -- 172.21.15.26:0/1891572985 wait complete. 2024-03-06T07:23:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.757+0000 7f3422f55640 1 Processor -- start 2024-03-06T07:23:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.757+0000 7f3422f55640 1 -- start start 2024-03-06T07:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3422f55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f341c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3422f55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c147e10 0x7f341c150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3422f55640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c152010 0x7f341c1eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3422f55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c159370 con 0x7f3404004800 2024-03-06T07:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3422f55640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c072720 con 0x7f341c152010 2024-03-06T07:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3422f55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0723b0 con 0x7f341c147e10 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3420cf1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f341c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3420cf1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f341c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39464/0 (socket says 172.21.15.26:39464) 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.758+0000 7f3420cf1640 1 -- 172.21.15.26:0/3215016021 learned_addr learned my addr 172.21.15.26:0/3215016021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3420cf1640 1 -- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c147e10 msgr2=0x7f341c150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3413fff640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c147e10 0x7f341c150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3420cf1640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c147e10 0x7f341c150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3420cf1640 1 -- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c152010 msgr2=0x7f341c1eaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3420cf1640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c152010 0x7f341c1eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3420cf1640 1 -- 172.21.15.26:0/3215016021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c150d50 con 0x7f3404004800 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3420cf1640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f341c157780 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f34180119c0 tx=0x7f3418011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f34117fa640 1 -- 172.21.15.26:0/3215016021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341800fa70 con 0x7f3404004800 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f34117fa640 1 -- 172.21.15.26:0/3215016021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341800fc10 con 0x7f3404004800 2024-03-06T07:23:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f34117fa640 1 -- 172.21.15.26:0/3215016021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418014440 con 0x7f3404004800 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c151030 con 0x7f3404004800 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.759+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c157cc0 con 0x7f3404004800 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.760+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c075300 con 0x7f3404004800 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.760+0000 7f34117fa640 1 -- 172.21.15.26:0/3215016021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3418012040 con 0x7f3404004800 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.761+0000 7f34117fa640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.761+0000 7f34117fa640 1 -- 172.21.15.26:0/3215016021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f341808a620 con 0x7f3404004800 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.761+0000 7f3413fff640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.762+0000 7f3413fff640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f340806b4d0 0x7f340806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f340c001050 tx=0x7f340c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.765+0000 7f34117fa640 1 -- 172.21.15.26:0/3215016021 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f341c075300 con 0x7f3404004800 2024-03-06T07:23:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.945+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f341c04f330 con 0x7f3404004800 2024-03-06T07:23:52.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.948+0000 7f34117fa640 1 -- 172.21.15.26:0/3215016021 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3418019030 con 0x7f3404004800 2024-03-06T07:23:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.950+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f340806b4d0 msgr2=0x7f340806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.950+0000 7f3422f55640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f340806b4d0 0x7f340806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f340c001050 tx=0x7f340c007000 comp rx=0 tx=0).stop 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.950+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 msgr2=0x7f341c157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.950+0000 7f3422f55640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f341c157780 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f34180119c0 tx=0x7f3418011e80 comp rx=0 tx=0).stop 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 shutdown_connections 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c152010 0x7f341c1eaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c147e10 0x7f341c150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 --2- 172.21.15.26:0/3215016021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404004800 0x7f341c157780 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 >> 172.21.15.26:0/3215016021 conn(0x7f341c0aa630 msgr2=0x7f341c0a85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 shutdown_connections 2024-03-06T07:23:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:52.951+0000 7f3422f55640 1 -- 172.21.15.26:0/3215016021 wait complete. 2024-03-06T07:23:52.964 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:52.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:53.001 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:53.049 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:53.049 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:23:53.050 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:53.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:53.065 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:23:53.065 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:53.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:53.083 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:23:53.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.204+0000 7f8639b8a640 1 Processor -- start 2024-03-06T07:23:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.204+0000 7f8639b8a640 1 -- start start 2024-03-06T07:23:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8639b8a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f863406d9f0 0x7f86340a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8639b8a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86340c3280 0x7f86340c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8639b8a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86341551e0 0x7f8634155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8639b8a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8634158800 con 0x7f86341551e0 2024-03-06T07:23:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8639b8a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8634158ae0 con 0x7f863406d9f0 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8639b8a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8634158dc0 con 0x7f86340c3280 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86341551e0 0x7f8634155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86341551e0 0x7f8634155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39480/0 (socket says 172.21.15.26:39480) 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 -- 172.21.15.26:0/1551895430 learned_addr learned my addr 172.21.15.26:0/1551895430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f86337fe640 1 --2- 172.21.15.26:0/1551895430 >> v2:172.21.15.117:3300/0 conn(0x7f863406d9f0 0x7f86340a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8632ffd640 1 --2- 172.21.15.26:0/1551895430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86340c3280 0x7f86340c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 -- 172.21.15.26:0/1551895430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86340c3280 msgr2=0x7f86340c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 --2- 172.21.15.26:0/1551895430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86340c3280 0x7f86340c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 -- 172.21.15.26:0/1551895430 >> v2:172.21.15.117:3300/0 conn(0x7f863406d9f0 msgr2=0x7f86340a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 --2- 172.21.15.26:0/1551895430 >> v2:172.21.15.117:3300/0 conn(0x7f863406d9f0 0x7f86340a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8633fff640 1 -- 172.21.15.26:0/1551895430 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86341590a0 con 0x7f86341551e0 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.205+0000 7f8632ffd640 1 --2- 172.21.15.26:0/1551895430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86340c3280 0x7f86340c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.206+0000 7f8633fff640 1 --2- 172.21.15.26:0/1551895430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86341551e0 0x7f8634155610 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f86340c9820 tx=0x7f862c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd926a2dd8ab9ea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.206+0000 7f86327fc640 1 -- 172.21.15.26:0/1551895430 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c02c9e0 con 0x7f86341551e0 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.206+0000 7f86327fc640 1 -- 172.21.15.26:0/1551895430 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c02cb80 con 0x7f86341551e0 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.206+0000 7f86327fc640 1 -- 172.21.15.26:0/1551895430 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0343f0 con 0x7f86341551e0 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 -- 172.21.15.26:0/1551895430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86341551e0 msgr2=0x7f8634155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 --2- 172.21.15.26:0/1551895430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86341551e0 0x7f8634155610 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f86340c9820 tx=0x7f862c02be40 comp rx=0 tx=0).stop 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 -- 172.21.15.26:0/1551895430 shutdown_connections 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 --2- 172.21.15.26:0/1551895430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86340c3280 0x7f86340c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 --2- 172.21.15.26:0/1551895430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86341551e0 0x7f8634155610 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 --2- 172.21.15.26:0/1551895430 >> v2:172.21.15.117:3300/0 conn(0x7f863406d9f0 0x7f86340a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 -- 172.21.15.26:0/1551895430 >> 172.21.15.26:0/1551895430 conn(0x7f86340aa630 msgr2=0x7f86340aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 -- 172.21.15.26:0/1551895430 shutdown_connections 2024-03-06T07:23:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.207+0000 7f8639b8a640 1 -- 172.21.15.26:0/1551895430 wait complete. 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.208+0000 7f8639b8a640 1 Processor -- start 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.208+0000 7f8639b8a640 1 -- start start 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863406d9f0 0x7f86340a9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86340c3280 0x7f86340aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86341551e0 0x7f86341ee810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863406df50 con 0x7f863406d9f0 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863406ddd0 con 0x7f86340c3280 2024-03-06T07:23:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340a8210 con 0x7f86341551e0 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863406d9f0 0x7f86340a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863406d9f0 0x7f86340a9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39492/0 (socket says 172.21.15.26:39492) 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 -- 172.21.15.26:0/2285072500 learned_addr learned my addr 172.21.15.26:0/2285072500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8632ffd640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86340c3280 0x7f86340aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8633fff640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86341551e0 0x7f86341ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 -- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86341551e0 msgr2=0x7f86341ee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86341551e0 0x7f86341ee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 -- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86340c3280 msgr2=0x7f86340aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86340c3280 0x7f86340aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 -- 172.21.15.26:0/2285072500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86341eed50 con 0x7f863406d9f0 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f86337fe640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863406d9f0 0x7f86340a9520 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f862800d5f0 tx=0x7f862800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f861bfff640 1 -- 172.21.15.26:0/2285072500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280125f0 con 0x7f863406d9f0 2024-03-06T07:23:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f861bfff640 1 -- 172.21.15.26:0/2285072500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8628019070 con 0x7f863406d9f0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f861bfff640 1 -- 172.21.15.26:0/2285072500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280129d0 con 0x7f863406d9f0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86340a9b00 con 0x7f863406d9f0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.209+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863406e250 con 0x7f863406d9f0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.210+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86340aa020 con 0x7f863406d9f0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.211+0000 7f861bfff640 1 -- 172.21.15.26:0/2285072500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8628012b70 con 0x7f863406d9f0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.211+0000 7f861bfff640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f862006b4d0 0x7f862006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.211+0000 7f861bfff640 1 -- 172.21.15.26:0/2285072500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f862808a410 con 0x7f863406d9f0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.211+0000 7f8632ffd640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f862006b4d0 0x7f862006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.212+0000 7f8632ffd640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f862006b4d0 0x7f862006d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8634152360 tx=0x7f8624008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.216+0000 7f861bfff640 1 -- 172.21.15.26:0/2285072500 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8628059440 con 0x7f863406d9f0 2024-03-06T07:23:53.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.432+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f863404f330 con 0x7f863406d9f0 2024-03-06T07:23:53.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.433+0000 7f861bfff640 1 -- 172.21.15.26:0/2285072500 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f8628058d40 con 0x7f863406d9f0 2024-03-06T07:23:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:23:53.436 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1702158641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1702158641},{"type":"v1","addr":"172.21.15.117:6837","nonce":1702158641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1479693385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1479693385},{"type":"v1","addr":"172.21.15.26:6834","nonce":1479693385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3691780502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3691780502},{"type":"v1","addr":"172.21.15.26:6835","nonce":3691780502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/834732227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":834732227},{"type":"v1","addr":"172.21.15.117:6836","nonce":834732227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[]} 2024-03-06T07:23:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 84 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f862006b4d0 msgr2=0x7f862006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f862006b4d0 0x7f862006d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8634152360 tx=0x7f8624008040 comp rx=0 tx=0).stop 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863406d9f0 msgr2=0x7f86340a9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863406d9f0 0x7f86340a9520 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f862800d5f0 tx=0x7f862800dab0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 shutdown_connections 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f862006b4d0 0x7f862006d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86341551e0 0x7f86341ee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86340c3280 0x7f86340aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 --2- 172.21.15.26:0/2285072500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863406d9f0 0x7f86340a9520 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 >> 172.21.15.26:0/2285072500 conn(0x7f86340aa630 msgr2=0x7f8634155860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 shutdown_connections 2024-03-06T07:23:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:53.435+0000 7f8639b8a640 1 -- 172.21.15.26:0/2285072500 wait complete. 2024-03-06T07:23:53.449 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:23:53.449 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:23:53.449 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:23:53.449 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:53.449 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:23:53.449 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:23:53.449 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:23:53.450 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:53.450 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:23:53.450 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:23:53.450 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:23:53.450 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:53.450 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:23:53.450 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:23:53.450 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:23:53.451 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:53.458 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:23:53.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:23:53.459 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:23:53.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:23:53.462 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:23:53.462 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:23:53.500 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:23:53.501 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:53.501 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:23:53.501 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:53.505 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:23:53.505 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:53.575 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:23:53.575 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:23:53.586 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:23:55.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-03-06T07:23:59.462 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:23:59.463 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:23:59.466 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:23:59.466 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:59.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:23:59.549 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:23:59.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cffbf640 1 Processor -- start 2024-03-06T07:23:59.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cffbf640 1 -- start start 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cffbf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f17c81561a0 0x7f17c81529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cffbf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f17c814f110 0x7f17c814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cffbf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c80a9990 con 0x7f17c80c6ab0 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cffbf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c8155d10 con 0x7f17c81561a0 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cffbf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c81556c0 con 0x7f17c814f110 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cd55a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f17c81561a0 0x7f17c81529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cd55a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f17c81561a0 0x7f17c81529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34998/0 (socket says 172.21.15.26:34998) 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cd55a640 1 -- 172.21.15.26:0/4287327044 learned_addr learned my addr 172.21.15.26:0/4287327044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17ce55c640 1 --2- 172.21.15.26:0/4287327044 >> v2:172.21.15.117:3300/0 conn(0x7f17c814f110 0x7f17c814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cd55a640 1 -- 172.21.15.26:0/4287327044 >> v2:172.21.15.117:3300/0 conn(0x7f17c814f110 msgr2=0x7f17c814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cd55a640 1 --2- 172.21.15.26:0/4287327044 >> v2:172.21.15.117:3300/0 conn(0x7f17c814f110 0x7f17c814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cd55a640 1 -- 172.21.15.26:0/4287327044 >> v1:172.21.15.117:6789/0 conn(0x7f17c80c6ab0 legacy=0x7f17c80a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.580+0000 7f17cd55a640 1 -- 172.21.15.26:0/4287327044 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c81559a0 con 0x7f17c81561a0 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17cd55a640 1 --2- 172.21.15.26:0/4287327044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81529e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f17c8155c80 tx=0x7f17b802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5236723ea0345c7e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17ccd59640 1 -- 172.21.15.26:0/4287327044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b802e070 con 0x7f17c81561a0 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17ccd59640 1 -- 172.21.15.26:0/4287327044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8037070 con 0x7f17c81561a0 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17ccd59640 1 -- 172.21.15.26:0/4287327044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8031a00 con 0x7f17c81561a0 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17cffbf640 1 -- 172.21.15.26:0/4287327044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 msgr2=0x7f17c81529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17cffbf640 1 --2- 172.21.15.26:0/4287327044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81529e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f17c8155c80 tx=0x7f17b802b880 comp rx=0 tx=0).stop 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17cffbf640 1 -- 172.21.15.26:0/4287327044 shutdown_connections 2024-03-06T07:23:59.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17cffbf640 1 --2- 172.21.15.26:0/4287327044 >> v2:172.21.15.117:3300/0 conn(0x7f17c814f110 0x7f17c814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17cffbf640 1 --2- 172.21.15.26:0/4287327044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81529e0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.581+0000 7f17cffbf640 1 -- 172.21.15.26:0/4287327044 >> 172.21.15.26:0/4287327044 conn(0x7f17c806b1a0 msgr2=0x7f17c806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.582+0000 7f17cffbf640 1 -- 172.21.15.26:0/4287327044 shutdown_connections 2024-03-06T07:23:59.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.582+0000 7f17cffbf640 1 -- 172.21.15.26:0/4287327044 wait complete. 2024-03-06T07:23:59.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 Processor -- start 2024-03-06T07:23:59.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 -- start start 2024-03-06T07:23:59.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17c80c6ab0 0x7f17c814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c814f110 0x7f17c814e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80aa600 con 0x7f17c81561a0 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80c2a90 con 0x7f17c814f110 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17cffbf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c814bfa0 con 0x7f17c80c6ab0 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.583+0000 7f17ce55c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35008/0 (socket says 172.21.15.26:35008) 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 -- 172.21.15.26:0/3037998116 learned_addr learned my addr 172.21.15.26:0/3037998116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17cd55a640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c814f110 0x7f17c814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17cdd5b640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17c80c6ab0 0x7f17c814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 -- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17c80c6ab0 msgr2=0x7f17c814e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17c80c6ab0 0x7f17c814e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 -- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c814f110 msgr2=0x7f17c814e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c814f110 0x7f17c814e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 -- 172.21.15.26:0/3037998116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8140ac0 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17cdd5b640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17c80c6ab0 0x7f17c814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17ce55c640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81486f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f17c0007e40 tx=0x7f17c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17be7fc640 1 -- 172.21.15.26:0/3037998116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0014620 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17be7fc640 1 -- 172.21.15.26:0/3037998116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0019070 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c81407e0 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17be7fc640 1 -- 172.21.15.26:0/3037998116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0014a20 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.584+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81e99d0 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.585+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c804f330 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.586+0000 7f17be7fc640 1 -- 172.21.15.26:0/3037998116 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17c0014bc0 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.586+0000 7f17be7fc640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f17ac06b4d0 0x7f17ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.586+0000 7f17be7fc640 1 -- 172.21.15.26:0/3037998116 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f17c008a8d0 con 0x7f17c81561a0 2024-03-06T07:23:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.586+0000 7f17cdd5b640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f17ac06b4d0 0x7f17ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.587+0000 7f17cdd5b640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f17ac06b4d0 0x7f17ac06d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f17c80c6510 tx=0x7f17c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.590+0000 7f17be7fc640 1 -- 172.21.15.26:0/3037998116 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f17c0059900 con 0x7f17c81561a0 2024-03-06T07:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:23:59.783+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f17c80c26b0 con 0x7f17c81561a0 2024-03-06T07:24:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.641+0000 7f17be7fc640 1 -- 172.21.15.26:0/3037998116 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7f17c0059200 con 0x7f17c81561a0 2024-03-06T07:24:00.643 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:24:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f17ac06b4d0 msgr2=0x7f17ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f17ac06b4d0 0x7f17ac06d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f17c80c6510 tx=0x7f17c4008040 comp rx=0 tx=0).stop 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 msgr2=0x7f17c81486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81486f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f17c0007e40 tx=0x7f17c000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 shutdown_connections 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f17ac06b4d0 0x7f17ac06d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c81561a0 0x7f17c81486f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c814f110 0x7f17c814e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 --2- 172.21.15.26:0/3037998116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17c80c6ab0 0x7f17c814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 >> 172.21.15.26:0/3037998116 conn(0x7f17c806b1a0 msgr2=0x7f17c80c0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 shutdown_connections 2024-03-06T07:24:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.644+0000 7f17cffbf640 1 -- 172.21.15.26:0/3037998116 wait complete. 2024-03-06T07:24:00.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:24:00.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.765+0000 7f8b3938a640 1 Processor -- start 2024-03-06T07:24:00.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.765+0000 7f8b3938a640 1 -- start start 2024-03-06T07:24:00.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b3938a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8b340c5330 0x7f8b340c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b3938a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b34159630 con 0x7f8b340c8a00 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b3938a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340c7760 con 0x7f8b34152920 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b3938a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b341587b0 con 0x7f8b340c5330 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b327fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b340c8a00 0x7f8b34158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41888/0 (socket says 172.21.15.26:41888) 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b327fc640 1 -- 172.21.15.26:0/1758617633 learned_addr learned my addr 172.21.15.26:0/1758617633 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b337fe640 1 --2- 172.21.15.26:0/1758617633 >> v2:172.21.15.117:3300/0 conn(0x7f8b340c5330 0x7f8b340c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724952352 0 0) 0x7f8b34159630 con 0x7f8b340c8a00 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1c003520 con 0x7f8b340c8a00 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812909398 0 0) 0x7f8b340c7760 con 0x7f8b34152920 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b34159630 con 0x7f8b34152920 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1637032691 0 0) 0x7f8b1c003520 con 0x7f8b340c8a00 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 >> v2:172.21.15.117:3300/0 conn(0x7f8b340c5330 msgr2=0x7f8b340c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 --2- 172.21.15.26:0/1758617633 >> v2:172.21.15.117:3300/0 conn(0x7f8b340c5330 0x7f8b340c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 >> v1:172.21.15.117:6789/0 conn(0x7f8b34152920 legacy=0x7f8b34155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340c7a40 con 0x7f8b340c8a00 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f8b28001670 con 0x7f8b340c8a00 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b337fe640 1 --2- 172.21.15.26:0/1758617633 >> v2:172.21.15.117:3300/0 conn(0x7f8b340c5330 0x7f8b340c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.766+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 >> v1:172.21.15.26:6789/0 conn(0x7f8b340c8a00 legacy=0x7f8b34158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b31ffb640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b1c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b31ffb640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b1c004800 0x7f8b1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b31ffb640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b1c005210 0x7f8b1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b341587b0 con 0x7f8b1c003b90 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c00c140 con 0x7f8b1c004800 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c00c340 con 0x7f8b1c005210 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b32ffd640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b1c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b327fc640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b1c004800 0x7f8b1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b337fe640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b1c005210 0x7f8b1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b32ffd640 1 -- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b1c005210 msgr2=0x7f8b1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b32ffd640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b1c005210 0x7f8b1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b32ffd640 1 -- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b1c004800 msgr2=0x7f8b1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b32ffd640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b1c004800 0x7f8b1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b32ffd640 1 -- 172.21.15.26:0/1758617633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c003520 con 0x7f8b1c003b90 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b337fe640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b1c005210 0x7f8b1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b327fc640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b1c004800 0x7f8b1c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.767+0000 7f8b337fe640 1 -- 172.21.15.26:0/1758617633 reap_dead start 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b32ffd640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b1c003f90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001fd0 tx=0x7f8b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b7b27d937d2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24001bf0 con 0x7f8b1c003b90 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b24003b70 con 0x7f8b1c003b90 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b31ffb640 1 -- 172.21.15.26:0/1758617633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240050d0 con 0x7f8b1c003b90 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b3938a640 1 -- 172.21.15.26:0/1758617633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 msgr2=0x7f8b1c003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b3938a640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b1c003f90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001fd0 tx=0x7f8b1c005830 comp rx=0 tx=0).stop 2024-03-06T07:24:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b3938a640 1 -- 172.21.15.26:0/1758617633 shutdown_connections 2024-03-06T07:24:00.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b3938a640 1 --2- 172.21.15.26:0/1758617633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b1c003f90 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b3938a640 1 -- 172.21.15.26:0/1758617633 >> 172.21.15.26:0/1758617633 conn(0x7f8b340ac560 msgr2=0x7f8b340ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:00.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.768+0000 7f8b3938a640 1 -- 172.21.15.26:0/1758617633 shutdown_connections 2024-03-06T07:24:00.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.769+0000 7f8b3938a640 1 -- 172.21.15.26:0/1758617633 wait complete. 2024-03-06T07:24:00.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 Processor -- start 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 -- start start 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b3414b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b34149060 0x7f8b34148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b341485b0 0x7f8b341ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406d030 con 0x7f8b1c003b90 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34158a00 con 0x7f8b341485b0 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b3938a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340c7390 con 0x7f8b34149060 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.770+0000 7f8b32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b3414b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b3414b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35036/0 (socket says 172.21.15.26:35036) 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 -- 172.21.15.26:0/3586438797 learned_addr learned my addr 172.21.15.26:0/3586438797 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b327fc640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b34149060 0x7f8b34148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b337fe640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b341485b0 0x7f8b341ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 -- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b34149060 msgr2=0x7f8b34148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b34149060 0x7f8b34148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 -- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b341485b0 msgr2=0x7f8b341ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b341485b0 0x7f8b341ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 -- 172.21.15.26:0/3586438797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340be240 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b32ffd640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b3414b110 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001fd0 tx=0x7f8b24001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b1b7fe640 1 -- 172.21.15.26:0/3586438797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24041030 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b1b7fe640 1 -- 172.21.15.26:0/3586438797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240052e0 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b34148b10 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b1b7fe640 1 -- 172.21.15.26:0/3586438797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24005490 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.771+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b341eba40 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.772+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b34067910 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.773+0000 7f8b1b7fe640 1 -- 172.21.15.26:0/3586438797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b24044070 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.773+0000 7f8b1b7fe640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b2006b4d0 0x7f8b2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.773+0000 7f8b1b7fe640 1 -- 172.21.15.26:0/3586438797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8b240ab650 con 0x7f8b1c003b90 2024-03-06T07:24:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.774+0000 7f8b327fc640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b2006b4d0 0x7f8b2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.774+0000 7f8b327fc640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b2006b4d0 0x7f8b2006d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b28001200 tx=0x7f8b28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:00.778+0000 7f8b1b7fe640 1 -- 172.21.15.26:0/3586438797 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8b34067910 con 0x7f8b1c003b90 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.027+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8b3404f330 con 0x7f8b1c003b90 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.027+0000 7f8b1b7fe640 1 -- 172.21.15.26:0/3586438797 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f8b2407a720 con 0x7f8b1c003b90 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:24:01.036 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:01.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:24:01.039 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:24:01.040 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b2006b4d0 msgr2=0x7f8b2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b2006b4d0 0x7f8b2006d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b28001200 tx=0x7f8b28009040 comp rx=0 tx=0).stop 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 msgr2=0x7f8b3414b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b3414b110 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001fd0 tx=0x7f8b24001650 comp rx=0 tx=0).stop 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 shutdown_connections 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b2006b4d0 0x7f8b2006d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b341485b0 0x7f8b341ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b34149060 0x7f8b34148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 --2- 172.21.15.26:0/3586438797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b1c003b90 0x7f8b3414b110 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 >> 172.21.15.26:0/3586438797 conn(0x7f8b340ac560 msgr2=0x7f8b340ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 shutdown_connections 2024-03-06T07:24:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.029+0000 7f8b3938a640 1 -- 172.21.15.26:0/3586438797 wait complete. 2024-03-06T07:24:01.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:01.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.144+0000 7fd7ffdd7640 1 Processor -- start 2024-03-06T07:24:01.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.144+0000 7fd7ffdd7640 1 -- start start 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7ffdd7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7f81592d0 0x7fd7f81552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7ffdd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f8155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7ffdd7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f8152d40 con 0x7fd7f8152960 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7ffdd7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f8066e30 con 0x7fd7f81592d0 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7ffdd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8155d50 con 0x7fd7f80ac260 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fd372640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7f81592d0 0x7fd7f81552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fdb73640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd7f8152960 0x7fd7f806dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41892/0 (socket says 172.21.15.26:41892) 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fdb73640 1 -- 172.21.15.26:0/3809457784 learned_addr learned my addr 172.21.15.26:0/3809457784 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fe374640 1 --2- 172.21.15.26:0/3809457784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f8155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fd372640 1 -- 172.21.15.26:0/3809457784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 msgr2=0x7fd7f8155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fd372640 1 --2- 172.21.15.26:0/3809457784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f8155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fd372640 1 -- 172.21.15.26:0/3809457784 >> v1:172.21.15.26:6789/0 conn(0x7fd7f8152960 legacy=0x7fd7f806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fcb71640 1 -- 172.21.15.26:0/3809457784 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288655091 0 0) 0x7fd7f8152d40 con 0x7fd7f8152960 2024-03-06T07:24:01.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fd372640 1 -- 172.21.15.26:0/3809457784 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80c1b50 con 0x7fd7f81592d0 2024-03-06T07:24:01.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.145+0000 7fd7fe374640 1 --2- 172.21.15.26:0/3809457784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f8155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7fd372640 1 --2- 172.21.15.26:0/3809457784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81592d0 0x7fd7f81552d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80097a0 tx=0x7fd7e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdca6d99f2d3f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7fcb71640 1 -- 172.21.15.26:0/3809457784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8035020 con 0x7fd7f81592d0 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7fcb71640 1 -- 172.21.15.26:0/3809457784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e8031830 con 0x7fd7f81592d0 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7fcb71640 1 -- 172.21.15.26:0/3809457784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8031c00 con 0x7fd7f81592d0 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/3809457784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81592d0 msgr2=0x7fd7f81552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/3809457784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81592d0 0x7fd7f81552d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80097a0 tx=0x7fd7e802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/3809457784 shutdown_connections 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/3809457784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81592d0 0x7fd7f81552d0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/3809457784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f8155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.146+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/3809457784 >> 172.21.15.26:0/3809457784 conn(0x7fd7f80c70b0 msgr2=0x7fd7f80c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.147+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/3809457784 shutdown_connections 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.147+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/3809457784 wait complete. 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 Processor -- start 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 -- start start 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7f8152960 0x7fd7f8157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81e24e0 0x7fd7f8156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8159760 con 0x7fd7f81e24e0 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80672c0 con 0x7fd7f8152960 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7ffdd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f806e3a0 con 0x7fd7f80ac260 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7fe374640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81e24e0 0x7fd7f8156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7fe374640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81e24e0 0x7fd7f8156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35046/0 (socket says 172.21.15.26:35046) 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.148+0000 7fd7fe374640 1 -- 172.21.15.26:0/382278157 learned_addr learned my addr 172.21.15.26:0/382278157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fd372640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7f8152960 0x7fd7f8157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fdb73640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fe374640 1 -- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 msgr2=0x7fd7f814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fe374640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fe374640 1 -- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7f8152960 msgr2=0x7fd7f8157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fe374640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7f8152960 0x7fd7f8157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fe374640 1 -- 172.21.15.26:0/382278157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8156bb0 con 0x7fd7f81e24e0 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fd372640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7f8152960 0x7fd7f8157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7fe374640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81e24e0 0x7fd7f8156670 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000cef0 tx=0x7fd7f000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7ee7fc640 1 -- 172.21.15.26:0/382278157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f001a070 con 0x7fd7f81e24e0 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7ee7fc640 1 -- 172.21.15.26:0/382278157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000d860 con 0x7fd7f81e24e0 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f81deb80 con 0x7fd7f81e24e0 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7ee7fc640 1 -- 172.21.15.26:0/382278157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000dc50 con 0x7fd7f81e24e0 2024-03-06T07:24:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.149+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f81eeba0 con 0x7fd7f81e24e0 2024-03-06T07:24:01.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.150+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f804f330 con 0x7fd7f81e24e0 2024-03-06T07:24:01.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.151+0000 7fd7ee7fc640 1 -- 172.21.15.26:0/382278157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7f001d070 con 0x7fd7f81e24e0 2024-03-06T07:24:01.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.151+0000 7fd7ee7fc640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.151+0000 7fd7ee7fc640 1 -- 172.21.15.26:0/382278157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd7f008a830 con 0x7fd7f81e24e0 2024-03-06T07:24:01.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.151+0000 7fd7fdb73640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.152+0000 7fd7fdb73640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000f60 tx=0x7fd7f4007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.155+0000 7fd7ee7fc640 1 -- 172.21.15.26:0/382278157 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd7f00599c0 con 0x7fd7f81e24e0 2024-03-06T07:24:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.337+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f806ebc0 con 0x7fd7f81e24e0 2024-03-06T07:24:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.338+0000 7fd7ee7fc640 1 -- 172.21.15.26:0/382278157 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+13178 (secure 0 0 0) 0x7fd7f00592c0 con 0x7fd7f81e24e0 2024-03-06T07:24:01.339 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:01.339 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":104,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:00.640474+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7dc06b4d0 msgr2=0x7fd7dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000f60 tx=0x7fd7f4007b50 comp rx=0 tx=0).stop 2024-03-06T07:24:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81e24e0 msgr2=0x7fd7f8156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81e24e0 0x7fd7f8156670 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000cef0 tx=0x7fd7f000a710 comp rx=0 tx=0).stop 2024-03-06T07:24:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 shutdown_connections 2024-03-06T07:24:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f81e24e0 0x7fd7f8156670 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7f8152960 0x7fd7f8157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 --2- 172.21.15.26:0/382278157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7f80ac260 0x7fd7f814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 >> 172.21.15.26:0/382278157 conn(0x7fd7f80c70b0 msgr2=0x7fd7f806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.339+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 shutdown_connections 2024-03-06T07:24:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.340+0000 7fd7ffdd7640 1 -- 172.21.15.26:0/382278157 wait complete. 2024-03-06T07:24:01.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:01.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.455+0000 7feb84925640 1 Processor -- start 2024-03-06T07:24:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.455+0000 7feb84925640 1 -- start start 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb84925640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb801529f0 0x7feb800c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb84925640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb800c5370 0x7feb800c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb84925640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb80152dd0 con 0x7feb80155530 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb84925640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb80159710 con 0x7feb800c5370 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb84925640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb800c68b0 con 0x7feb801529f0 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb800c5370 0x7feb800c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb800c5370 0x7feb800c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35060/0 (socket says 172.21.15.26:35060) 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb77fff640 1 -- 172.21.15.26:0/805265359 learned_addr learned my addr 172.21.15.26:0/805265359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb77fff640 1 -- 172.21.15.26:0/805265359 >> v2:172.21.15.117:3300/0 conn(0x7feb801529f0 msgr2=0x7feb800c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb77fff640 1 --2- 172.21.15.26:0/805265359 >> v2:172.21.15.117:3300/0 conn(0x7feb801529f0 0x7feb800c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb77fff640 1 -- 172.21.15.26:0/805265359 >> v1:172.21.15.117:6789/0 conn(0x7feb80155530 legacy=0x7feb801559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.456+0000 7feb77fff640 1 -- 172.21.15.26:0/805265359 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb80158b00 con 0x7feb800c5370 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.457+0000 7feb77fff640 1 --2- 172.21.15.26:0/805265359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800c5370 0x7feb800c57d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7feb800cba90 tx=0x7feb6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c4c30938af4cd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.457+0000 7feb7dd9b640 1 -- 172.21.15.26:0/805265359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6802e070 con 0x7feb800c5370 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.457+0000 7feb7dd9b640 1 -- 172.21.15.26:0/805265359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb68037070 con 0x7feb800c5370 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.457+0000 7feb7dd9b640 1 -- 172.21.15.26:0/805265359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68031a00 con 0x7feb800c5370 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.457+0000 7feb84925640 1 -- 172.21.15.26:0/805265359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800c5370 msgr2=0x7feb800c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.457+0000 7feb84925640 1 --2- 172.21.15.26:0/805265359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800c5370 0x7feb800c57d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7feb800cba90 tx=0x7feb6802b880 comp rx=0 tx=0).stop 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.458+0000 7feb84925640 1 -- 172.21.15.26:0/805265359 shutdown_connections 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.458+0000 7feb84925640 1 --2- 172.21.15.26:0/805265359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800c5370 0x7feb800c57d0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.458+0000 7feb84925640 1 --2- 172.21.15.26:0/805265359 >> v2:172.21.15.117:3300/0 conn(0x7feb801529f0 0x7feb800c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.458+0000 7feb84925640 1 -- 172.21.15.26:0/805265359 >> 172.21.15.26:0/805265359 conn(0x7feb800ac580 msgr2=0x7feb800ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.458+0000 7feb84925640 1 -- 172.21.15.26:0/805265359 shutdown_connections 2024-03-06T07:24:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.458+0000 7feb84925640 1 -- 172.21.15.26:0/805265359 wait complete. 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 Processor -- start 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 -- start start 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb800c5370 0x7feb801ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb801529f0 0x7feb8014c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb80155530 0x7feb8014c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb801593a0 con 0x7feb801529f0 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb801590c0 con 0x7feb800c5370 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.459+0000 7feb84925640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb801560d0 con 0x7feb80155530 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb801529f0 0x7feb8014c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb801529f0 0x7feb8014c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35064/0 (socket says 172.21.15.26:35064) 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 -- 172.21.15.26:0/1086760229 learned_addr learned my addr 172.21.15.26:0/1086760229 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb7e59c640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb800c5370 0x7feb801ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb7ed9d640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb80155530 0x7feb8014c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 -- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb80155530 msgr2=0x7feb8014c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb80155530 0x7feb8014c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 -- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb800c5370 msgr2=0x7feb801ee520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb800c5370 0x7feb801ee520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 -- 172.21.15.26:0/1086760229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800aefc0 con 0x7feb801529f0 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb7e59c640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb800c5370 0x7feb801ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb77fff640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb801529f0 0x7feb8014c060 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feb6802b8c0 tx=0x7feb680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb76ffd640 1 -- 172.21.15.26:0/1086760229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68032880 con 0x7feb801529f0 2024-03-06T07:24:01.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb76ffd640 1 -- 172.21.15.26:0/1086760229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb68032a20 con 0x7feb801529f0 2024-03-06T07:24:01.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb80158740 con 0x7feb801529f0 2024-03-06T07:24:01.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb76ffd640 1 -- 172.21.15.26:0/1086760229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68032e10 con 0x7feb801529f0 2024-03-06T07:24:01.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.460+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb800ae4e0 con 0x7feb801529f0 2024-03-06T07:24:01.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.461+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8004f330 con 0x7feb801529f0 2024-03-06T07:24:01.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.462+0000 7feb76ffd640 1 -- 172.21.15.26:0/1086760229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb680453f0 con 0x7feb801529f0 2024-03-06T07:24:01.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.462+0000 7feb76ffd640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6006b4d0 0x7feb6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.462+0000 7feb76ffd640 1 -- 172.21.15.26:0/1086760229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7feb680ac090 con 0x7feb801529f0 2024-03-06T07:24:01.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.462+0000 7feb7e59c640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6006b4d0 0x7feb6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.463+0000 7feb7e59c640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6006b4d0 0x7feb6006d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7feb70009930 tx=0x7feb70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.466+0000 7feb76ffd640 1 -- 172.21.15.26:0/1086760229 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb68040070 con 0x7feb801529f0 2024-03-06T07:24:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.647+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb800c7dd0 con 0x7feb801529f0 2024-03-06T07:24:01.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.649+0000 7feb76ffd640 1 -- 172.21.15.26:0/1086760229 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb6807b160 con 0x7feb801529f0 2024-03-06T07:24:01.650 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:24:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6006b4d0 msgr2=0x7feb6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6006b4d0 0x7feb6006d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7feb70009930 tx=0x7feb70008040 comp rx=0 tx=0).stop 2024-03-06T07:24:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb801529f0 msgr2=0x7feb8014c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb801529f0 0x7feb8014c060 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feb6802b8c0 tx=0x7feb680046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 shutdown_connections 2024-03-06T07:24:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6006b4d0 0x7feb6006d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb80155530 0x7feb8014c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb801529f0 0x7feb8014c060 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 --2- 172.21.15.26:0/1086760229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb800c5370 0x7feb801ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 >> 172.21.15.26:0/1086760229 conn(0x7feb800ac580 msgr2=0x7feb8006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 shutdown_connections 2024-03-06T07:24:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.651+0000 7feb84925640 1 -- 172.21.15.26:0/1086760229 wait complete. 2024-03-06T07:24:01.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:24:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.764+0000 7ff1fd75c640 1 Processor -- start 2024-03-06T07:24:01.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.764+0000 7ff1fd75c640 1 -- start start 2024-03-06T07:24:01.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.764+0000 7ff1fd75c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1f80c3280 0x7ff1f80c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.764+0000 7ff1fd75c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 0x7ff1f8155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.764+0000 7ff1fd75c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f8158800 con 0x7ff1f806d9f0 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.764+0000 7ff1fd75c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f8158ae0 con 0x7ff1f80c3280 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.764+0000 7ff1fd75c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f8158dc0 con 0x7ff1f81551e0 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1effff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1f80c3280 0x7ff1f80c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1f6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1f806d9f0 0x7ff1f80a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41904/0 (socket says 172.21.15.26:41904) 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1f6ffd640 1 -- 172.21.15.26:0/1852385361 learned_addr learned my addr 172.21.15.26:0/1852385361 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1effff640 1 -- 172.21.15.26:0/1852385361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 msgr2=0x7ff1f8155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1effff640 1 --2- 172.21.15.26:0/1852385361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 0x7ff1f8155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1effff640 1 -- 172.21.15.26:0/1852385361 >> v1:172.21.15.26:6789/0 conn(0x7ff1f806d9f0 legacy=0x7ff1f80a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1f67fc640 1 -- 172.21.15.26:0/1852385361 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458741049 0 0) 0x7ff1f8158800 con 0x7ff1f806d9f0 2024-03-06T07:24:01.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.765+0000 7ff1effff640 1 -- 172.21.15.26:0/1852385361 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f81590a0 con 0x7ff1f80c3280 2024-03-06T07:24:01.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1effff640 1 --2- 172.21.15.26:0/1852385361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80c3710 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00097a0 tx=0x7ff1e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e64cdfedea99ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1f67fc640 1 -- 172.21.15.26:0/1852385361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00315a0 con 0x7ff1f80c3280 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1f67fc640 1 -- 172.21.15.26:0/1852385361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e0037070 con 0x7ff1f80c3280 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1f67fc640 1 -- 172.21.15.26:0/1852385361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0031960 con 0x7ff1f80c3280 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1fd75c640 1 -- 172.21.15.26:0/1852385361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 msgr2=0x7ff1f80c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/1852385361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80c3710 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00097a0 tx=0x7ff1e002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1fd75c640 1 -- 172.21.15.26:0/1852385361 shutdown_connections 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/1852385361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80c3710 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/1852385361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 0x7ff1f8155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.766+0000 7ff1fd75c640 1 -- 172.21.15.26:0/1852385361 >> 172.21.15.26:0/1852385361 conn(0x7ff1f80aa630 msgr2=0x7ff1f80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.767+0000 7ff1fd75c640 1 -- 172.21.15.26:0/1852385361 shutdown_connections 2024-03-06T07:24:01.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.767+0000 7ff1fd75c640 1 -- 172.21.15.26:0/1852385361 wait complete. 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 Processor -- start 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 -- start start 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f806d9f0 0x7ff1f8153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80a8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 0x7ff1f80a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f806df50 con 0x7ff1f80c3280 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f806ddd0 con 0x7ff1f806d9f0 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1fd75c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f80c6910 con 0x7ff1f81551e0 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35074/0 (socket says 172.21.15.26:35074) 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.768+0000 7ff1effff640 1 -- 172.21.15.26:0/560390094 learned_addr learned my addr 172.21.15.26:0/560390094 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1f77fe640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 0x7ff1f80a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1effff640 1 -- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 msgr2=0x7ff1f80a8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1effff640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 0x7ff1f80a8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1f6ffd640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f806d9f0 0x7ff1f8153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1effff640 1 -- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f806d9f0 msgr2=0x7ff1f8153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1effff640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f806d9f0 0x7ff1f8153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1effff640 1 -- 172.21.15.26:0/560390094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f80c7550 con 0x7ff1f80c3280 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1effff640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80a8590 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0034040 tx=0x7ff1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1ef7fe640 1 -- 172.21.15.26:0/560390094 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0004030 con 0x7ff1f80c3280 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1ef7fe640 1 -- 172.21.15.26:0/560390094 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e00041f0 con 0x7ff1f80c3280 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f806f5f0 con 0x7ff1f80c3280 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1ef7fe640 1 -- 172.21.15.26:0/560390094 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0032640 con 0x7ff1f80c3280 2024-03-06T07:24:01.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.769+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f81ee8c0 con 0x7ff1f80c3280 2024-03-06T07:24:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.770+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f804f330 con 0x7ff1f80c3280 2024-03-06T07:24:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.771+0000 7ff1ef7fe640 1 -- 172.21.15.26:0/560390094 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1e00327e0 con 0x7ff1f80c3280 2024-03-06T07:24:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.771+0000 7ff1ef7fe640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1d806b4d0 0x7ff1d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.771+0000 7ff1ef7fe640 1 -- 172.21.15.26:0/560390094 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff1e002f030 con 0x7ff1f80c3280 2024-03-06T07:24:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.771+0000 7ff1f6ffd640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1d806b4d0 0x7ff1d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.772+0000 7ff1f6ffd640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1d806b4d0 0x7ff1d806d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8000f60 tx=0x7ff1e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.775+0000 7ff1ef7fe640 1 -- 172.21.15.26:0/560390094 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff1f80c7550 con 0x7ff1f80c3280 2024-03-06T07:24:01.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.956+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff1f8073240 con 0x7ff1f80c3280 2024-03-06T07:24:01.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.957+0000 7ff1ef7fe640 1 -- 172.21.15.26:0/560390094 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff1e0041070 con 0x7ff1f80c3280 2024-03-06T07:24:01.958 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:01.958 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:24:01.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:01.959 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:24:01.959 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:24:01.959 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:01.959 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:24:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1d806b4d0 msgr2=0x7ff1d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1d806b4d0 0x7ff1d806d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8000f60 tx=0x7ff1e8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 msgr2=0x7ff1f80a8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80a8590 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0034040 tx=0x7ff1e00046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 shutdown_connections 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1d806b4d0 0x7ff1d806d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f81551e0 0x7ff1f80a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f80c3280 0x7ff1f80a8590 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 --2- 172.21.15.26:0/560390094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f806d9f0 0x7ff1f8153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.958+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 >> 172.21.15.26:0/560390094 conn(0x7ff1f80aa630 msgr2=0x7ff1f80a9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.959+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 shutdown_connections 2024-03-06T07:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:01.959+0000 7ff1fd75c640 1 -- 172.21.15.26:0/560390094 wait complete. 2024-03-06T07:24:01.971 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:24:01.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:24:02.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.075+0000 7f1670e84640 1 Processor -- start 2024-03-06T07:24:02.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.075+0000 7f1670e84640 1 -- start start 2024-03-06T07:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f1670e84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f1670e84640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f166c155eb0 0x7f166c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f1670e84640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c159740 con 0x7f166c0c80d0 2024-03-06T07:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f1670e84640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c1551d0 con 0x7f166c155eb0 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f1670e84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0c8ab0 con 0x7f166c152920 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f166ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f166c0c80d0 0x7f166c0724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41910/0 (socket says 172.21.15.26:41910) 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f166ad9d640 1 -- 172.21.15.26:0/2605669334 learned_addr learned my addr 172.21.15.26:0/2605669334 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f166a59c640 1 --2- 172.21.15.26:0/2605669334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f1669d9b640 1 --2- 172.21.15.26:0/2605669334 >> v2:172.21.15.117:3300/0 conn(0x7f166c155eb0 0x7f166c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.076+0000 7f166959a640 1 -- 172.21.15.26:0/2605669334 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503523185 0 0) 0x7f166c159740 con 0x7f166c0c80d0 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.077+0000 7f166959a640 1 -- 172.21.15.26:0/2605669334 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1658003520 con 0x7f166c0c80d0 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.077+0000 7f166a59c640 1 -- 172.21.15.26:0/2605669334 >> v2:172.21.15.117:3300/0 conn(0x7f166c155eb0 msgr2=0x7f166c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.077+0000 7f166a59c640 1 --2- 172.21.15.26:0/2605669334 >> v2:172.21.15.117:3300/0 conn(0x7f166c155eb0 0x7f166c0c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.077+0000 7f166a59c640 1 -- 172.21.15.26:0/2605669334 >> v1:172.21.15.26:6789/0 conn(0x7f166c0c80d0 legacy=0x7f166c0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.077+0000 7f166a59c640 1 -- 172.21.15.26:0/2605669334 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0c8d90 con 0x7f166c152920 2024-03-06T07:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.077+0000 7f1669d9b640 1 --2- 172.21.15.26:0/2605669334 >> v2:172.21.15.117:3300/0 conn(0x7f166c155eb0 0x7f166c0c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.077+0000 7f166a59c640 1 --2- 172.21.15.26:0/2605669334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c155970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f16640097a0 tx=0x7f166402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26979c24bcba7aef server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f166959a640 1 -- 172.21.15.26:0/2605669334 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640315a0 con 0x7f166c152920 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f166959a640 1 -- 172.21.15.26:0/2605669334 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664037070 con 0x7f166c152920 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f166959a640 1 -- 172.21.15.26:0/2605669334 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664031960 con 0x7f166c152920 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 -- 172.21.15.26:0/2605669334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 msgr2=0x7f166c155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 --2- 172.21.15.26:0/2605669334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c155970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f16640097a0 tx=0x7f166402b840 comp rx=0 tx=0).stop 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 -- 172.21.15.26:0/2605669334 shutdown_connections 2024-03-06T07:24:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 --2- 172.21.15.26:0/2605669334 >> v2:172.21.15.117:3300/0 conn(0x7f166c155eb0 0x7f166c0c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 --2- 172.21.15.26:0/2605669334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c155970 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 -- 172.21.15.26:0/2605669334 >> 172.21.15.26:0/2605669334 conn(0x7f166c0aa5c0 msgr2=0x7f166c0aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 -- 172.21.15.26:0/2605669334 shutdown_connections 2024-03-06T07:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.078+0000 7f1670e84640 1 -- 172.21.15.26:0/2605669334 wait complete. 2024-03-06T07:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.079+0000 7f1670e84640 1 Processor -- start 2024-03-06T07:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1670e84640 1 -- start start 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1670e84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f166c0c80d0 0x7f166c14caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1670e84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1670e84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166c155eb0 0x7f166c1ea520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1670e84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0c86c0 con 0x7f166c155eb0 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1670e84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c155450 con 0x7f166c0c80d0 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1670e84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0c8840 con 0x7f166c152920 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f166ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166c155eb0 0x7f166c1ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f166ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166c155eb0 0x7f166c1ea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35078/0 (socket says 172.21.15.26:35078) 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f166ad9d640 1 -- 172.21.15.26:0/2447222989 learned_addr learned my addr 172.21.15.26:0/2447222989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f166a59c640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f166c0c80d0 0x7f166c14caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.080+0000 7f1669d9b640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f166ad9d640 1 -- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 msgr2=0x7f166c157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f166ad9d640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f166ad9d640 1 -- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f166c0c80d0 msgr2=0x7f166c14caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f166ad9d640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f166c0c80d0 0x7f166c14caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f166ad9d640 1 -- 172.21.15.26:0/2447222989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0a8830 con 0x7f166c155eb0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f166a59c640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f166c0c80d0 0x7f166c14caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f1669d9b640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f166ad9d640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166c155eb0 0x7f166c1ea520 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f165c00cb40 tx=0x7f165c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f1662ffd640 1 -- 172.21.15.26:0/2447222989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c017070 con 0x7f166c155eb0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f1662ffd640 1 -- 172.21.15.26:0/2447222989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c00aca0 con 0x7f166c155eb0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c1edbb0 con 0x7f166c155eb0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f1662ffd640 1 -- 172.21.15.26:0/2447222989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c012620 con 0x7f166c155eb0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.081+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c1ee170 con 0x7f166c155eb0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.082+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c04f330 con 0x7f166c155eb0 2024-03-06T07:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.082+0000 7f1662ffd640 1 -- 172.21.15.26:0/2447222989 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f165c01a070 con 0x7f166c155eb0 2024-03-06T07:24:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.083+0000 7f1662ffd640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f164c06b4d0 0x7f164c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.083+0000 7f1662ffd640 1 -- 172.21.15.26:0/2447222989 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f165c089d40 con 0x7f166c155eb0 2024-03-06T07:24:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.083+0000 7f166a59c640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f164c06b4d0 0x7f164c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.084+0000 7f166a59c640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f164c06b4d0 0x7f164c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1664036000 tx=0x7f166403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:02.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.087+0000 7f1662ffd640 1 -- 172.21.15.26:0/2447222989 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f165c058e10 con 0x7f166c155eb0 2024-03-06T07:24:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.267+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f166c0c1fd0 con 0x7f166c155eb0 2024-03-06T07:24:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.657+0000 7f1662ffd640 1 -- 172.21.15.26:0/2447222989 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v105) v1 ==== 138+0+0 (secure 0 0 0) 0x7f165c058710 con 0x7f166c155eb0 2024-03-06T07:24:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:24:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.661+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f164c06b4d0 msgr2=0x7f164c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.661+0000 7f1670e84640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f164c06b4d0 0x7f164c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1664036000 tx=0x7f166403d040 comp rx=0 tx=0).stop 2024-03-06T07:24:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.661+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166c155eb0 msgr2=0x7f166c1ea520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.661+0000 7f1670e84640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166c155eb0 0x7f166c1ea520 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f165c00cb40 tx=0x7f165c00a060 comp rx=0 tx=0).stop 2024-03-06T07:24:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.662+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 shutdown_connections 2024-03-06T07:24:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.662+0000 7f1670e84640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f164c06b4d0 0x7f164c06d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.662+0000 7f1670e84640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166c155eb0 0x7f166c1ea520 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.662+0000 7f1670e84640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f166c152920 0x7f166c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.662+0000 7f1670e84640 1 --2- 172.21.15.26:0/2447222989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f166c0c80d0 0x7f166c14caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.662+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 >> 172.21.15.26:0/2447222989 conn(0x7f166c0aa5c0 msgr2=0x7f166c151f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.663+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 shutdown_connections 2024-03-06T07:24:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.663+0000 7f1670e84640 1 -- 172.21.15.26:0/2447222989 wait complete. 2024-03-06T07:24:02.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:24:02.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-03-06T07:24:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.785+0000 7f4b93ac9640 1 Processor -- start 2024-03-06T07:24:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.785+0000 7f4b93ac9640 1 -- start start 2024-03-06T07:24:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b93ac9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 0x7f4b8c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b93ac9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c0b0450 con 0x7f4b8c0b0070 2024-03-06T07:24:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b93ac9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c1560e0 con 0x7f4b8c155c30 2024-03-06T07:24:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b93ac9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c06c020 con 0x7f4b8c06ca70 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b91865640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4b8c0b0070 0x7f4b8c0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41914/0 (socket says 172.21.15.26:41914) 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b91865640 1 -- 172.21.15.26:0/2604115713 learned_addr learned my addr 172.21.15.26:0/2604115713 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333419235 0 0) 0x7f4b84001110 con 0x7f4b8c0b0070 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b78003520 con 0x7f4b8c0b0070 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.786+0000 7f4b92066640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 0x7f4b8c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1097034226 0 0) 0x7f4b78003520 con 0x7f4b8c0b0070 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 msgr2=0x7f4b8c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 0x7f4b8c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 >> v1:172.21.15.117:6789/0 conn(0x7f4b8c155c30 legacy=0x7f4b8c0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0c77f0 con 0x7f4b8c0b0070 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4b84001840 con 0x7f4b8c0b0070 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 >> v1:172.21.15.26:6789/0 conn(0x7f4b8c0b0070 legacy=0x7f4b8c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b780039e0 0x7f4b78003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 0x7f4b78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78004d40 0x7f4b780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c06c020 con 0x7f4b780039e0 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7800bb50 con 0x7f4b78004d40 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7800bd20 con 0x7f4b8c06ca70 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b91865640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b780039e0 0x7f4b78003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b91064640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 0x7f4b78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b92066640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78004d40 0x7f4b780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b91865640 1 -- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 msgr2=0x7f4b78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b91865640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b8c06ca70 0x7f4b78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b91865640 1 -- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78004d40 msgr2=0x7f4b780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b91865640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78004d40 0x7f4b780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.787+0000 7f4b91865640 1 -- 172.21.15.26:0/2604115713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78003520 con 0x7f4b780039e0 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.788+0000 7f4b91865640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b780039e0 0x7f4b78003e20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400c4c0 tx=0x7f4b78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2021455a501bbe45 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.788+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400c0e0 con 0x7f4b780039e0 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.788+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b84030070 con 0x7f4b780039e0 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.788+0000 7f4b90863640 1 -- 172.21.15.26:0/2604115713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840058d0 con 0x7f4b780039e0 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.789+0000 7f4b93ac9640 1 -- 172.21.15.26:0/2604115713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b780039e0 msgr2=0x7f4b78003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.789+0000 7f4b93ac9640 1 --2- 172.21.15.26:0/2604115713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b780039e0 0x7f4b78003e20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400c4c0 tx=0x7f4b78005830 comp rx=0 tx=0).stop 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.789+0000 7f4b92066640 1 -- 172.21.15.26:0/2604115713 reap_dead start 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.789+0000 7f4b93ac9640 1 -- 172.21.15.26:0/2604115713 shutdown_connections 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.789+0000 7f4b93ac9640 1 -- 172.21.15.26:0/2604115713 >> 172.21.15.26:0/2604115713 conn(0x7f4b8c0ac590 msgr2=0x7f4b8c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.789+0000 7f4b93ac9640 1 -- 172.21.15.26:0/2604115713 shutdown_connections 2024-03-06T07:24:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.789+0000 7f4b93ac9640 1 -- 172.21.15.26:0/2604115713 wait complete. 2024-03-06T07:24:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.790+0000 7f4b93ac9640 1 Processor -- start 2024-03-06T07:24:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.790+0000 7f4b93ac9640 1 -- start start 2024-03-06T07:24:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.790+0000 7f4b93ac9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b780039e0 0x7f4b8c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.790+0000 7f4b93ac9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b8c14e4b0 0x7f4b8c1ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.790+0000 7f4b93ac9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b8c14df60 0x7f4b8c1ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.790+0000 7f4b93ac9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c06c550 con 0x7f4b8c14df60 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.790+0000 7f4b93ac9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c06c270 con 0x7f4b8c14e4b0 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b93ac9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c06da80 con 0x7f4b780039e0 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b8c14df60 0x7f4b8c1ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b8c14df60 0x7f4b8c1ee7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35084/0 (socket says 172.21.15.26:35084) 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 -- 172.21.15.26:0/140180059 learned_addr learned my addr 172.21.15.26:0/140180059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b91865640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b780039e0 0x7f4b8c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 -- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b780039e0 msgr2=0x7f4b8c1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b780039e0 0x7f4b8c1549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b91064640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b8c14e4b0 0x7f4b8c1ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 -- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b8c14e4b0 msgr2=0x7f4b8c1ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b8c14e4b0 0x7f4b8c1ee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 -- 172.21.15.26:0/140180059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c14dbb0 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b92066640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b8c14df60 0x7f4b8c1ee7e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0cba60 tx=0x7f4b8800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b827fc640 1 -- 172.21.15.26:0/140180059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8800fe10 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b827fc640 1 -- 172.21.15.26:0/140180059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b88015430 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c0c45e0 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b827fc640 1 -- 172.21.15.26:0/140180059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88015820 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.791+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c154f10 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.792+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c04f330 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.793+0000 7f4b827fc640 1 -- 172.21.15.26:0/140180059 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b880159c0 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.793+0000 7f4b827fc640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b7006b500 0x7f4b7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.793+0000 7f4b827fc640 1 -- 172.21.15.26:0/140180059 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f4b8808aef0 con 0x7f4b8c14df60 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.793+0000 7f4b91865640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b7006b500 0x7f4b7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.794+0000 7f4b91865640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b7006b500 0x7f4b7006d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b84034000 tx=0x7f4b84009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.798+0000 7f4b827fc640 1 -- 172.21.15.26:0/140180059 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b88019030 con 0x7f4b8c14df60 2024-03-06T07:24:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:02.981+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4b8c0ca360 con 0x7f4b8c14df60 2024-03-06T07:24:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.663+0000 7f4b827fc640 1 -- 172.21.15.26:0/140180059 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v106) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4b88010040 con 0x7f4b8c14df60 2024-03-06T07:24:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:24:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.667+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b7006b500 msgr2=0x7f4b7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.667+0000 7f4b93ac9640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b7006b500 0x7f4b7006d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b84034000 tx=0x7f4b84009030 comp rx=0 tx=0).stop 2024-03-06T07:24:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.667+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b8c14df60 msgr2=0x7f4b8c1ee7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.667+0000 7f4b93ac9640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b8c14df60 0x7f4b8c1ee7e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0cba60 tx=0x7f4b8800f580 comp rx=0 tx=0).stop 2024-03-06T07:24:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.668+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 shutdown_connections 2024-03-06T07:24:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.668+0000 7f4b93ac9640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b7006b500 0x7f4b7006d9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.668+0000 7f4b93ac9640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b8c14df60 0x7f4b8c1ee7e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.668+0000 7f4b93ac9640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b8c14e4b0 0x7f4b8c1ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.668+0000 7f4b93ac9640 1 --2- 172.21.15.26:0/140180059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b780039e0 0x7f4b8c1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.668+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 >> 172.21.15.26:0/140180059 conn(0x7f4b8c0ac590 msgr2=0x7f4b8c0aaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.673+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 shutdown_connections 2024-03-06T07:24:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.674+0000 7f4b93ac9640 1 -- 172.21.15.26:0/140180059 wait complete. 2024-03-06T07:24:03.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.804+0000 7ff810484640 1 Processor -- start 2024-03-06T07:24:03.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.804+0000 7ff810484640 1 -- start start 2024-03-06T07:24:03.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff810484640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8081527f0 0x7ff8080c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff810484640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8080c7370 0x7ff8080c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff810484640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff808155260 con 0x7ff80806c0b0 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff810484640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff808155540 con 0x7ff8080c7370 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff810484640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff808155820 con 0x7ff8081527f0 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff80da1f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8080c7370 0x7ff8080c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff80ea21640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff80806c0b0 0x7ff80806c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41924/0 (socket says 172.21.15.26:41924) 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.805+0000 7ff80ea21640 1 -- 172.21.15.26:0/3601049207 learned_addr learned my addr 172.21.15.26:0/3601049207 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80e220640 1 --2- 172.21.15.26:0/3601049207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8081527f0 0x7ff8080c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80da1f640 1 -- 172.21.15.26:0/3601049207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8081527f0 msgr2=0x7ff8080c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80da1f640 1 --2- 172.21.15.26:0/3601049207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8081527f0 0x7ff8080c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80da1f640 1 -- 172.21.15.26:0/3601049207 >> v1:172.21.15.26:6789/0 conn(0x7ff80806c0b0 legacy=0x7ff80806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80d21e640 1 -- 172.21.15.26:0/3601049207 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116490235 0 0) 0x7ff808155260 con 0x7ff80806c0b0 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80da1f640 1 -- 172.21.15.26:0/3601049207 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808155b00 con 0x7ff8080c7370 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80e220640 1 --2- 172.21.15.26:0/3601049207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8081527f0 0x7ff8080c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80da1f640 1 --2- 172.21.15.26:0/3601049207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff8080c77d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0097a0 tx=0x7ff7fc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a811222f5bbcd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.806+0000 7ff80d21e640 1 -- 172.21.15.26:0/3601049207 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc038070 con 0x7ff8080c7370 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff80d21e640 1 -- 172.21.15.26:0/3601049207 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc031700 con 0x7ff8080c7370 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff80d21e640 1 -- 172.21.15.26:0/3601049207 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc031ad0 con 0x7ff8080c7370 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 -- 172.21.15.26:0/3601049207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 msgr2=0x7ff8080c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 --2- 172.21.15.26:0/3601049207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff8080c77d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0097a0 tx=0x7ff7fc02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 -- 172.21.15.26:0/3601049207 shutdown_connections 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 --2- 172.21.15.26:0/3601049207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff8080c77d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 --2- 172.21.15.26:0/3601049207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8081527f0 0x7ff8080c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 -- 172.21.15.26:0/3601049207 >> 172.21.15.26:0/3601049207 conn(0x7ff8080ac4f0 msgr2=0x7ff8080ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 -- 172.21.15.26:0/3601049207 shutdown_connections 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.807+0000 7ff810484640 1 -- 172.21.15.26:0/3601049207 wait complete. 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.808+0000 7ff810484640 1 Processor -- start 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.808+0000 7ff810484640 1 -- start start 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff810484640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80806c0b0 0x7ff8080bec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff810484640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff808149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff810484640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8081527f0 0x7ff8080c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff810484640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80806cd70 con 0x7ff8080c7370 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff810484640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80806cbf0 con 0x7ff8081527f0 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff810484640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080c8da0 con 0x7ff80806c0b0 2024-03-06T07:24:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff808149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff808149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35116/0 (socket says 172.21.15.26:35116) 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 -- 172.21.15.26:0/2733709360 learned_addr learned my addr 172.21.15.26:0/2733709360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80e220640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80806c0b0 0x7ff8080bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80ea21640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8081527f0 0x7ff8080c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 -- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80806c0b0 msgr2=0x7ff8080bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80806c0b0 0x7ff8080bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 -- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8081527f0 msgr2=0x7ff8080c0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8081527f0 0x7ff8080c0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 -- 172.21.15.26:0/2733709360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808154fa0 con 0x7ff8080c7370 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80ea21640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8081527f0 0x7ff8080c0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80e220640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80806c0b0 0x7ff8080bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.809+0000 7ff80da1f640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff808149360 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0097a0 tx=0x7ff7fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.810+0000 7ff806ffd640 1 -- 172.21.15.26:0/2733709360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc038040 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.810+0000 7ff806ffd640 1 -- 172.21.15.26:0/2733709360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc004250 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.810+0000 7ff806ffd640 1 -- 172.21.15.26:0/2733709360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc033820 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.810+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8081549e0 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.810+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8080be450 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.811+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80804f330 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.811+0000 7ff806ffd640 1 -- 172.21.15.26:0/2733709360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7fc0315c0 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.812+0000 7ff806ffd640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7f406b500 0x7ff7f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.812+0000 7ff806ffd640 1 -- 172.21.15.26:0/2733709360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6001+0+0 (secure 0 0 0) 0x7ff7fc0ac700 con 0x7ff8080c7370 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.812+0000 7ff80e220640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7f406b500 0x7ff7f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.813+0000 7ff80e220640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7f406b500 0x7ff7f406d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff808149e80 tx=0x7ff7f8007640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:03.817+0000 7ff806ffd640 1 -- 172.21.15.26:0/2733709360 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7fc030070 con 0x7ff8080c7370 2024-03-06T07:24:04.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.036+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff80806d860 con 0x7ff8080c7370 2024-03-06T07:24:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.664+0000 7ff806ffd640 1 -- 172.21.15.26:0/2733709360 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff7fc07b440 con 0x7ff8080c7370 2024-03-06T07:24:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 17 and data pool 18 2024-03-06T07:24:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7f406b500 msgr2=0x7ff7f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7f406b500 0x7ff7f406d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff808149e80 tx=0x7ff7f8007640 comp rx=0 tx=0).stop 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 msgr2=0x7ff808149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff808149360 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0097a0 tx=0x7ff7fc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 shutdown_connections 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff7f406b500 0x7ff7f406d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8081527f0 0x7ff8080c0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080c7370 0x7ff808149360 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 --2- 172.21.15.26:0/2733709360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80806c0b0 0x7ff8080bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.667+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 >> 172.21.15.26:0/2733709360 conn(0x7ff8080ac4f0 msgr2=0x7ff8080ab600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.668+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 shutdown_connections 2024-03-06T07:24:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.668+0000 7ff810484640 1 -- 172.21.15.26:0/2733709360 wait complete. 2024-03-06T07:24:04.706 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.810+0000 7f28d2281640 1 Processor -- start 2024-03-06T07:24:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.810+0000 7f28d2281640 1 -- start start 2024-03-06T07:24:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28d2281640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28cc14f790 0x7f28cc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28d2281640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc0c28d0 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28d2281640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc0c2bb0 con 0x7f28cc155a40 2024-03-06T07:24:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28d2281640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc0c2680 con 0x7f28cc14f790 2024-03-06T07:24:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28cc14f790 0x7f28cc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28cb7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f28cc0c1aa0 0x7f28cc0c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41940/0 (socket says 172.21.15.26:41940) 2024-03-06T07:24:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28cc14f790 0x7f28cc1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35126/0 (socket says 172.21.15.26:35126) 2024-03-06T07:24:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28cb7fe640 1 -- 172.21.15.26:0/2365476609 learned_addr learned my addr 172.21.15.26:0/2365476609 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28cbfff640 1 -- 172.21.15.26:0/2365476609 >> v1:172.21.15.117:6789/0 conn(0x7f28cc155a40 legacy=0x7f28cc155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.811+0000 7f28caffd640 1 -- 172.21.15.26:0/2365476609 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104282830 0 0) 0x7f28cc0c28d0 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.812+0000 7f28cbfff640 1 -- 172.21.15.26:0/2365476609 >> v1:172.21.15.26:6789/0 conn(0x7f28cc0c1aa0 legacy=0x7f28cc0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.812+0000 7f28cbfff640 1 -- 172.21.15.26:0/2365476609 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0c2e90 con 0x7f28cc14f790 2024-03-06T07:24:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.812+0000 7f28cbfff640 1 --2- 172.21.15.26:0/2365476609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc14f790 0x7f28cc1529d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0c2800 tx=0x7f28c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513f4a04df2d4e62 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.812+0000 7f28caffd640 1 -- 172.21.15.26:0/2365476609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0034070 con 0x7f28cc14f790 2024-03-06T07:24:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.812+0000 7f28caffd640 1 -- 172.21.15.26:0/2365476609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0030610 con 0x7f28cc14f790 2024-03-06T07:24:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.812+0000 7f28caffd640 1 -- 172.21.15.26:0/2365476609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00309e0 con 0x7f28cc14f790 2024-03-06T07:24:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.813+0000 7f28d2281640 1 -- 172.21.15.26:0/2365476609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc14f790 msgr2=0x7f28cc1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.813+0000 7f28d2281640 1 --2- 172.21.15.26:0/2365476609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc14f790 0x7f28cc1529d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0c2800 tx=0x7f28c002b960 comp rx=0 tx=0).stop 2024-03-06T07:24:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.813+0000 7f28d2281640 1 -- 172.21.15.26:0/2365476609 shutdown_connections 2024-03-06T07:24:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.813+0000 7f28d2281640 1 --2- 172.21.15.26:0/2365476609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc14f790 0x7f28cc1529d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.813+0000 7f28d2281640 1 -- 172.21.15.26:0/2365476609 >> 172.21.15.26:0/2365476609 conn(0x7f28cc0c6710 msgr2=0x7f28cc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.813+0000 7f28d2281640 1 -- 172.21.15.26:0/2365476609 shutdown_connections 2024-03-06T07:24:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.813+0000 7f28d2281640 1 -- 172.21.15.26:0/2365476609 wait complete. 2024-03-06T07:24:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.814+0000 7f28d2281640 1 Processor -- start 2024-03-06T07:24:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.814+0000 7f28d2281640 1 -- start start 2024-03-06T07:24:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28d2281640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc0c1aa0 0x7f28cc14ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28d2281640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28cc14f790 0x7f28cc149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28d2281640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28cc155a40 0x7f28cc149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28d2281640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0c5e30 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28d2281640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0c0d60 con 0x7f28cc155a40 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28d2281640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0c5fb0 con 0x7f28cc14f790 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc0c1aa0 0x7f28cc14ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc0c1aa0 0x7f28cc14ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35134/0 (socket says 172.21.15.26:35134) 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 -- 172.21.15.26:0/1572401309 learned_addr learned my addr 172.21.15.26:0/1572401309 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cb7fe640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28cc14f790 0x7f28cc149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28d081e640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28cc155a40 0x7f28cc149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 -- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28cc14f790 msgr2=0x7f28cc149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28cc14f790 0x7f28cc149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 -- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28cc155a40 msgr2=0x7f28cc149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28cc155a40 0x7f28cc149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.815+0000 7f28cbfff640 1 -- 172.21.15.26:0/1572401309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc1541b0 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.816+0000 7f28cbfff640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc0c1aa0 0x7f28cc14ce10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f28c002bf70 tx=0x7f28c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.816+0000 7f28c8ff9640 1 -- 172.21.15.26:0/1572401309 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0034040 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.816+0000 7f28c8ff9640 1 -- 172.21.15.26:0/1572401309 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0039a10 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.816+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc154420 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.816+0000 7f28c8ff9640 1 -- 172.21.15.26:0/1572401309 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0039e00 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.816+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc153980 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.817+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc0c1010 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.817+0000 7f28c8ff9640 1 -- 172.21.15.26:0/1572401309 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28c0037070 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.818+0000 7f28c8ff9640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b806b4d0 0x7f28b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.818+0000 7f28c8ff9640 1 -- 172.21.15.26:0/1572401309 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28c00ac350 con 0x7f28cc0c1aa0 2024-03-06T07:24:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.818+0000 7f28cb7fe640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b806b4d0 0x7f28b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:04.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.818+0000 7f28cb7fe640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b806b4d0 0x7f28b806d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f28bc007f70 tx=0x7f28bc007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:04.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:04.822+0000 7f28c8ff9640 1 -- 172.21.15.26:0/1572401309 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28cc0c1010 con 0x7f28cc0c1aa0 2024-03-06T07:24:05.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.003+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28cc04f330 con 0x7f28cc0c1aa0 2024-03-06T07:24:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.004+0000 7f28c8ff9640 1 -- 172.21.15.26:0/1572401309 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16977 (secure 0 0 0) 0x7f28c007ab20 con 0x7f28cc0c1aa0 2024-03-06T07:24:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:05.006 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":107,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:04.037300+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:02.286561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:03.001111+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b806b4d0 msgr2=0x7f28b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b806b4d0 0x7f28b806d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f28bc007f70 tx=0x7f28bc007f00 comp rx=0 tx=0).stop 2024-03-06T07:24:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc0c1aa0 msgr2=0x7f28cc14ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc0c1aa0 0x7f28cc14ce10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f28c002bf70 tx=0x7f28c00046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 shutdown_connections 2024-03-06T07:24:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b806b4d0 0x7f28b806d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28cc155a40 0x7f28cc149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28cc14f790 0x7f28cc149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 --2- 172.21.15.26:0/1572401309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc0c1aa0 0x7f28cc14ce10 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 >> 172.21.15.26:0/1572401309 conn(0x7f28cc0c6710 msgr2=0x7f28cc0c3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 shutdown_connections 2024-03-06T07:24:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.006+0000 7f28d2281640 1 -- 172.21.15.26:0/1572401309 wait complete. 2024-03-06T07:24:05.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:05.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.119+0000 7fd50e793640 1 Processor -- start 2024-03-06T07:24:05.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.119+0000 7fd50e793640 1 -- start start 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.120+0000 7fd50e793640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 0x7fd508155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.120+0000 7fd50e793640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5080c73d0 con 0x7fd5080c88c0 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.120+0000 7fd50e793640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5080be2e0 con 0x7fd508158b60 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.120+0000 7fd50e793640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd508159700 con 0x7fd50814f360 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.120+0000 7fd5077fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5080c88c0 0x7fd5080c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41948/0 (socket says 172.21.15.26:41948) 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.120+0000 7fd5077fe640 1 -- 172.21.15.26:0/2069951566 learned_addr learned my addr 172.21.15.26:0/2069951566 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881377731 0 0) 0x7fd5080c73d0 con 0x7fd5080c88c0 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f0003520 con 0x7fd5080c88c0 2024-03-06T07:24:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd507fff640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 0x7fd508155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 971005189 0 0) 0x7fd4f0003520 con 0x7fd5080c88c0 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 msgr2=0x7fd508155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 0x7fd508155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 >> v1:172.21.15.117:6789/0 conn(0x7fd508158b60 legacy=0x7fd508158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5080c7670 con 0x7fd5080c88c0 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd4f8001670 con 0x7fd5080c88c0 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 >> v1:172.21.15.26:6789/0 conn(0x7fd5080c88c0 legacy=0x7fd5080c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 0x7fd4f00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd4f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f0004d40 0x7fd4f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd508159700 con 0x7fd4f0004010 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f000bc00 con 0x7fd4f0004d40 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f000be00 con 0x7fd50814f360 2024-03-06T07:24:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd5077fe640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd4f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd507fff640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 0x7fd4f00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd50cd30640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f0004d40 0x7fd4f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd5077fe640 1 -- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 msgr2=0x7fd4f00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd5077fe640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd50814f360 0x7fd4f00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd5077fe640 1 -- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f0004d40 msgr2=0x7fd4f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd5077fe640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f0004d40 0x7fd4f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd5077fe640 1 -- 172.21.15.26:0/2069951566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0003520 con 0x7fd4f0004010 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.121+0000 7fd50cd30640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f0004d40 0x7fd4f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.122+0000 7fd5077fe640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd4f0004800 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd5080c8d20 tx=0x7fd4f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fdc285f3e61e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.122+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8007d30 con 0x7fd4f0004010 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.122+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f8030070 con 0x7fd4f0004010 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.122+0000 7fd506ffd640 1 -- 172.21.15.26:0/2069951566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8005d70 con 0x7fd4f0004010 2024-03-06T07:24:05.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.123+0000 7fd50e793640 1 -- 172.21.15.26:0/2069951566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 msgr2=0x7fd4f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.123+0000 7fd50e793640 1 --2- 172.21.15.26:0/2069951566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd4f0004800 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd5080c8d20 tx=0x7fd4f0005830 comp rx=0 tx=0).stop 2024-03-06T07:24:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.123+0000 7fd50cd30640 1 -- 172.21.15.26:0/2069951566 reap_dead start 2024-03-06T07:24:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.123+0000 7fd50e793640 1 -- 172.21.15.26:0/2069951566 shutdown_connections 2024-03-06T07:24:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.123+0000 7fd50e793640 1 -- 172.21.15.26:0/2069951566 >> 172.21.15.26:0/2069951566 conn(0x7fd5080ac570 msgr2=0x7fd5080ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.123+0000 7fd50e793640 1 -- 172.21.15.26:0/2069951566 shutdown_connections 2024-03-06T07:24:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.123+0000 7fd50e793640 1 -- 172.21.15.26:0/2069951566 wait complete. 2024-03-06T07:24:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.124+0000 7fd50e793640 1 Processor -- start 2024-03-06T07:24:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.124+0000 7fd50e793640 1 -- start start 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd50e793640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd5081eb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd50e793640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5080c3820 0x7fd5080c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd50e793640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5080c2ee0 0x7fd50814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd50e793640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5080c7c30 con 0x7fd4f0004010 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd50e793640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5080c5a50 con 0x7fd5080c3820 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd50e793640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5080c5bd0 con 0x7fd5080c2ee0 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd5081eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd5081eb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35158/0 (socket says 172.21.15.26:35158) 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 -- 172.21.15.26:0/425652831 learned_addr learned my addr 172.21.15.26:0/425652831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 -- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5080c2ee0 msgr2=0x7fd50814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5080c2ee0 0x7fd50814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 -- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5080c3820 msgr2=0x7fd5080c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd5077fe640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5080c3820 0x7fd5080c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5080c3820 0x7fd5080c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 -- 172.21.15.26:0/425652831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5081eb880 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd507fff640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd5081eb340 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b7f0 tx=0x7fd4fc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd504ff9640 1 -- 172.21.15.26:0/425652831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc018070 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd504ff9640 1 -- 172.21.15.26:0/425652831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc010830 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5080aff20 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.125+0000 7fd504ff9640 1 -- 172.21.15.26:0/425652831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc010c00 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.126+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5080c21a0 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.126+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50804f330 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.127+0000 7fd504ff9640 1 -- 172.21.15.26:0/425652831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4fc01b070 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.127+0000 7fd504ff9640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4f406b500 0x7fd4f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.127+0000 7fd504ff9640 1 -- 172.21.15.26:0/425652831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4fc08a790 con 0x7fd4f0004010 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.128+0000 7fd5077fe640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4f406b500 0x7fd4f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.128+0000 7fd5077fe640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4f406b500 0x7fd4f406d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8034000 tx=0x7fd4f8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.132+0000 7fd504ff9640 1 -- 172.21.15.26:0/425652831 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5081eb880 con 0x7fd4f0004010 2024-03-06T07:24:05.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.312+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5080c8d50 con 0x7fd4f0004010 2024-03-06T07:24:05.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.313+0000 7fd504ff9640 1 -- 172.21.15.26:0/425652831 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16977 (secure 0 0 0) 0x7fd4fc059470 con 0x7fd4f0004010 2024-03-06T07:24:05.315 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:05.315 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":107,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:04.037300+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:02.286561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:03.001111+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:05.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4f406b500 msgr2=0x7fd4f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4f406b500 0x7fd4f406d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8034000 tx=0x7fd4f8009030 comp rx=0 tx=0).stop 2024-03-06T07:24:05.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 msgr2=0x7fd5081eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd5081eb340 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b7f0 tx=0x7fd4fc00bcb0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 shutdown_connections 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4f406b500 0x7fd4f406d9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5080c2ee0 0x7fd50814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5080c3820 0x7fd5080c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 --2- 172.21.15.26:0/425652831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4f0004010 0x7fd5081eb340 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.315+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 >> 172.21.15.26:0/425652831 conn(0x7fd5080ac570 msgr2=0x7fd5080ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.316+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 shutdown_connections 2024-03-06T07:24:05.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.316+0000 7fd50e793640 1 -- 172.21.15.26:0/425652831 wait complete. 2024-03-06T07:24:05.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:24:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.431+0000 7ffb3e9b6640 1 Processor -- start 2024-03-06T07:24:05.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.431+0000 7ffb3e9b6640 1 -- start start 2024-03-06T07:24:05.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb3e9b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb381536a0 0x7ffb380c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb3e9b6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffb3806b3a0 0x7ffb3806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb3e9b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb380c7430 con 0x7ffb3806afc0 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb3e9b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3806fde0 con 0x7ffb381536a0 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb3e9b6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb38156fc0 con 0x7ffb3806b3a0 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb381536a0 0x7ffb380c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb37fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffb3806afc0 0x7ffb38152f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41962/0 (socket says 172.21.15.26:41962) 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb37fff640 1 -- 172.21.15.26:0/3532638027 learned_addr learned my addr 172.21.15.26:0/3532638027 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb3cf53640 1 --2- 172.21.15.26:0/3532638027 >> v2:172.21.15.117:3300/0 conn(0x7ffb3806b3a0 0x7ffb3806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb36ffd640 1 -- 172.21.15.26:0/3532638027 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811484898 0 0) 0x7ffb380c7430 con 0x7ffb3806afc0 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb36ffd640 1 -- 172.21.15.26:0/3532638027 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb20003520 con 0x7ffb3806afc0 2024-03-06T07:24:05.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb377fe640 1 -- 172.21.15.26:0/3532638027 >> v2:172.21.15.117:3300/0 conn(0x7ffb3806b3a0 msgr2=0x7ffb3806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb377fe640 1 --2- 172.21.15.26:0/3532638027 >> v2:172.21.15.117:3300/0 conn(0x7ffb3806b3a0 0x7ffb3806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb377fe640 1 -- 172.21.15.26:0/3532638027 >> v1:172.21.15.26:6789/0 conn(0x7ffb3806afc0 legacy=0x7ffb38152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.432+0000 7ffb377fe640 1 -- 172.21.15.26:0/3532638027 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3806b800 con 0x7ffb381536a0 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.433+0000 7ffb377fe640 1 --2- 172.21.15.26:0/3532638027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380c6da0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ffb380c5200 tx=0x7ffb2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3e7ffd0f9027d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.433+0000 7ffb36ffd640 1 -- 172.21.15.26:0/3532638027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280315a0 con 0x7ffb381536a0 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.433+0000 7ffb36ffd640 1 -- 172.21.15.26:0/3532638027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb28037070 con 0x7ffb381536a0 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.433+0000 7ffb36ffd640 1 -- 172.21.15.26:0/3532638027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28031940 con 0x7ffb381536a0 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/3532638027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 msgr2=0x7ffb380c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/3532638027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380c6da0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ffb380c5200 tx=0x7ffb2802b840 comp rx=0 tx=0).stop 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/3532638027 shutdown_connections 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/3532638027 >> v2:172.21.15.117:3300/0 conn(0x7ffb3806b3a0 0x7ffb3806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/3532638027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380c6da0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/3532638027 >> 172.21.15.26:0/3532638027 conn(0x7ffb380c4220 msgr2=0x7ffb380c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:05.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/3532638027 shutdown_connections 2024-03-06T07:24:05.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.434+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/3532638027 wait complete. 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.435+0000 7ffb3e9b6640 1 Processor -- start 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.435+0000 7ffb3e9b6640 1 -- start start 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3e9b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb3806afc0 0x7ffb38149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3e9b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3806b3a0 0x7ffb380713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3e9b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3e9b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38156970 con 0x7ffb381536a0 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3e9b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38156690 con 0x7ffb3806b3a0 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3e9b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38156c50 con 0x7ffb3806afc0 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380a8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35166/0 (socket says 172.21.15.26:35166) 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 -- 172.21.15.26:0/2696555830 learned_addr learned my addr 172.21.15.26:0/2696555830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb377fe640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3806b3a0 0x7ffb380713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb37fff640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb3806afc0 0x7ffb38149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 -- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb3806afc0 msgr2=0x7ffb38149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb3806afc0 0x7ffb38149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 -- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3806b3a0 msgr2=0x7ffb380713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3806b3a0 0x7ffb380713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 -- 172.21.15.26:0/2696555830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38071900 con 0x7ffb381536a0 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb37fff640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb3806afc0 0x7ffb38149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.436+0000 7ffb3cf53640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380a8c10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ffb30007e70 tx=0x7ffb3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.437+0000 7ffb34ff9640 1 -- 172.21.15.26:0/2696555830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30014420 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.437+0000 7ffb34ff9640 1 -- 172.21.15.26:0/2696555830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30019070 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.437+0000 7ffb34ff9640 1 -- 172.21.15.26:0/2696555830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30014820 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.437+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3814ab80 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.437+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb380c12e0 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.438+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3804f330 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.438+0000 7ffb34ff9640 1 -- 172.21.15.26:0/2696555830 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb300149c0 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.438+0000 7ffb34ff9640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb2406b4d0 0x7ffb2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.439+0000 7ffb34ff9640 1 -- 172.21.15.26:0/2696555830 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb3008aba0 con 0x7ffb381536a0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.439+0000 7ffb37fff640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb2406b4d0 0x7ffb2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.439+0000 7ffb37fff640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb2406b4d0 0x7ffb2406d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c007fd0 tx=0x7ffb2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.443+0000 7ffb34ff9640 1 -- 172.21.15.26:0/2696555830 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffb30059890 con 0x7ffb381536a0 2024-03-06T07:24:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.663+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffb380c19e0 con 0x7ffb381536a0 2024-03-06T07:24:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.679+0000 7ffb34ff9640 1 -- 172.21.15.26:0/2696555830 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffb380c19e0 con 0x7ffb381536a0 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb2406b4d0 msgr2=0x7ffb2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb2406b4d0 0x7ffb2406d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c007fd0 tx=0x7ffb2c009040 comp rx=0 tx=0).stop 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 msgr2=0x7ffb380a8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380a8c10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ffb30007e70 tx=0x7ffb3000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 shutdown_connections 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb2406b4d0 0x7ffb2406d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb381536a0 0x7ffb380a8c10 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3806b3a0 0x7ffb380713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 --2- 172.21.15.26:0/2696555830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb3806afc0 0x7ffb38149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 >> 172.21.15.26:0/2696555830 conn(0x7ffb380c4220 msgr2=0x7ffb380c5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 shutdown_connections 2024-03-06T07:24:05.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.681+0000 7ffb3e9b6640 1 -- 172.21.15.26:0/2696555830 wait complete. 2024-03-06T07:24:05.693 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:24:05.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:24:05.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.796+0000 7fbfc1065640 1 Processor -- start 2024-03-06T07:24:05.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.796+0000 7fbfc1065640 1 -- start start 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.797+0000 7fbfc1065640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfbc0c4740 0x7fbfbc1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfc1065640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbfbc0c69a0 0x7fbfbc072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfc1065640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc0c62a0 con 0x7fbfbc0a8290 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfc1065640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc153f00 con 0x7fbfbc0c4740 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfc1065640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc0c4c00 con 0x7fbfbc0c69a0 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfba59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfbc0c4740 0x7fbfbc1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfba59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfbc0c4740 0x7fbfbc1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35172/0 (socket says 172.21.15.26:35172) 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfba59c640 1 -- 172.21.15.26:0/3889070710 learned_addr learned my addr 172.21.15.26:0/3889070710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfba59c640 1 -- 172.21.15.26:0/3889070710 >> v2:172.21.15.117:3300/0 conn(0x7fbfbc0c69a0 msgr2=0x7fbfbc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfba59c640 1 --2- 172.21.15.26:0/3889070710 >> v2:172.21.15.117:3300/0 conn(0x7fbfbc0c69a0 0x7fbfbc072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfba59c640 1 -- 172.21.15.26:0/3889070710 >> v1:172.21.15.117:6789/0 conn(0x7fbfbc0a8290 legacy=0x7fbfbc0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.798+0000 7fbfba59c640 1 -- 172.21.15.26:0/3889070710 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0c4ee0 con 0x7fbfbc0c4740 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfba59c640 1 --2- 172.21.15.26:0/3889070710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0c4740 0x7fbfbc1539c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0722a0 tx=0x7fbfa402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cfd5f5169180e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfb9d9b640 1 -- 172.21.15.26:0/3889070710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4038070 con 0x7fbfbc0c4740 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfb9d9b640 1 -- 172.21.15.26:0/3889070710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa402f050 con 0x7fbfbc0c4740 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfb9d9b640 1 -- 172.21.15.26:0/3889070710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4031970 con 0x7fbfbc0c4740 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 -- 172.21.15.26:0/3889070710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0c4740 msgr2=0x7fbfbc1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 --2- 172.21.15.26:0/3889070710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0c4740 0x7fbfbc1539c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0722a0 tx=0x7fbfa402b7f0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 -- 172.21.15.26:0/3889070710 shutdown_connections 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 --2- 172.21.15.26:0/3889070710 >> v2:172.21.15.117:3300/0 conn(0x7fbfbc0c69a0 0x7fbfbc072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 --2- 172.21.15.26:0/3889070710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0c4740 0x7fbfbc1539c0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 -- 172.21.15.26:0/3889070710 >> 172.21.15.26:0/3889070710 conn(0x7fbfbc0c5560 msgr2=0x7fbfbc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 -- 172.21.15.26:0/3889070710 shutdown_connections 2024-03-06T07:24:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.799+0000 7fbfc1065640 1 -- 172.21.15.26:0/3889070710 wait complete. 2024-03-06T07:24:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.801+0000 7fbfc1065640 1 Processor -- start 2024-03-06T07:24:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.802+0000 7fbfc1065640 1 -- start start 2024-03-06T07:24:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfc1065640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0a8290 0x7fbfbc1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfc1065640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc0c4740 0x7fbfbc1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfc1065640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfbc0c69a0 0x7fbfbc14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfc1065640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0c6df0 con 0x7fbfbc0a8290 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfc1065640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc06fc80 con 0x7fbfbc0c4740 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfc1065640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0c6f70 con 0x7fbfbc0c69a0 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0a8290 0x7fbfbc1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0a8290 0x7fbfbc1551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35184/0 (socket says 172.21.15.26:35184) 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 -- 172.21.15.26:0/1817620168 learned_addr learned my addr 172.21.15.26:0/1817620168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfba59c640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc0c4740 0x7fbfbc1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbb59e640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfbc0c69a0 0x7fbfbc14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 -- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfbc0c69a0 msgr2=0x7fbfbc14f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfbc0c69a0 0x7fbfbc14f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 -- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc0c4740 msgr2=0x7fbfbc1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc0c4740 0x7fbfbc1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 -- 172.21.15.26:0/1817620168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0ab940 con 0x7fbfbc0a8290 2024-03-06T07:24:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbb59e640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfbc0c69a0 0x7fbfbc14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfbad9d640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0a8290 0x7fbfbc1551a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c440 tx=0x7fbfac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfb37fe640 1 -- 172.21.15.26:0/1817620168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac017070 con 0x7fbfbc0a8290 2024-03-06T07:24:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfb37fe640 1 -- 172.21.15.26:0/1817620168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac0035f0 con 0x7fbfbc0a8290 2024-03-06T07:24:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc154cc0 con 0x7fbfbc0a8290 2024-03-06T07:24:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.803+0000 7fbfb37fe640 1 -- 172.21.15.26:0/1817620168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0039e0 con 0x7fbfbc0a8290 2024-03-06T07:24:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.804+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc0abec0 con 0x7fbfbc0a8290 2024-03-06T07:24:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.804+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc154e90 con 0x7fbfbc0a8290 2024-03-06T07:24:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.805+0000 7fbfb37fe640 1 -- 172.21.15.26:0/1817620168 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfac01a070 con 0x7fbfbc0a8290 2024-03-06T07:24:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.805+0000 7fbfb37fe640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf9c06b4d0 0x7fbf9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.806+0000 7fbfb37fe640 1 -- 172.21.15.26:0/1817620168 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbfac0895b0 con 0x7fbfbc0a8290 2024-03-06T07:24:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.806+0000 7fbfba59c640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf9c06b4d0 0x7fbf9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.806+0000 7fbfba59c640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf9c06b4d0 0x7fbf9c06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4035000 tx=0x7fbfa4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:05.814+0000 7fbfb37fe640 1 -- 172.21.15.26:0/1817620168 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbfac0010c0 con 0x7fbfbc0a8290 2024-03-06T07:24:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.032+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbfbc04f330 con 0x7fbfbc0a8290 2024-03-06T07:24:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.683+0000 7fbfb37fe640 1 -- 172.21.15.26:0/1817620168 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbfac057ce0 con 0x7fbfbc0a8290 2024-03-06T07:24:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf9c06b4d0 msgr2=0x7fbf9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf9c06b4d0 0x7fbf9c06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4035000 tx=0x7fbfa4008030 comp rx=0 tx=0).stop 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0a8290 msgr2=0x7fbfbc1551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0a8290 0x7fbfbc1551a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c440 tx=0x7fbfac00c900 comp rx=0 tx=0).stop 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 shutdown_connections 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf9c06b4d0 0x7fbf9c06d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfbc0c69a0 0x7fbfbc14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc0c4740 0x7fbfbc1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 --2- 172.21.15.26:0/1817620168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfbc0a8290 0x7fbfbc1551a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 >> 172.21.15.26:0/1817620168 conn(0x7fbfbc0c5560 msgr2=0x7fbfbc0a8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 shutdown_connections 2024-03-06T07:24:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.686+0000 7fbfc1065640 1 -- 172.21.15.26:0/1817620168 wait complete. 2024-03-06T07:24:06.699 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:24:06.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:06.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbc00a29640 1 Processor -- start 2024-03-06T07:24:06.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbc00a29640 1 -- start start 2024-03-06T07:24:06.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbc00a29640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc155ba0 0x7fbbfc0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbc00a29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbfc1586f0 0x7fbbfc158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbc00a29640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc1591c0 con 0x7fbbfc0c5480 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbc00a29640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc06df40 con 0x7fbbfc1586f0 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbc00a29640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc06fdb0 con 0x7fbbfc155ba0 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbbfad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbfc1586f0 0x7fbbfc158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.803+0000 7fbbfad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbfc1586f0 0x7fbbfc158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35190/0 (socket says 172.21.15.26:35190) 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.804+0000 7fbbfad9d640 1 -- 172.21.15.26:0/3729047200 learned_addr learned my addr 172.21.15.26:0/3729047200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.804+0000 7fbbf3fff640 1 --2- 172.21.15.26:0/3729047200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc155ba0 0x7fbbfc0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.804+0000 7fbbfad9d640 1 -- 172.21.15.26:0/3729047200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc155ba0 msgr2=0x7fbbfc0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.804+0000 7fbbfad9d640 1 --2- 172.21.15.26:0/3729047200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc155ba0 0x7fbbfc0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.804+0000 7fbbfad9d640 1 -- 172.21.15.26:0/3729047200 >> v1:172.21.15.117:6789/0 conn(0x7fbbfc0c5480 legacy=0x7fbbfc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.804+0000 7fbbfad9d640 1 -- 172.21.15.26:0/3729047200 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc1594a0 con 0x7fbbfc1586f0 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbbfad9d640 1 --2- 172.21.15.26:0/3729047200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc1586f0 0x7fbbfc158b80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009e50 tx=0x7fbbf402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6129f1ac8db730 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbbf9d9b640 1 -- 172.21.15.26:0/3729047200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf402ca70 con 0x7fbbfc1586f0 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbbf9d9b640 1 -- 172.21.15.26:0/3729047200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf402cc10 con 0x7fbbfc1586f0 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbbf9d9b640 1 -- 172.21.15.26:0/3729047200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40343f0 con 0x7fbbfc1586f0 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 -- 172.21.15.26:0/3729047200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc1586f0 msgr2=0x7fbbfc158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 --2- 172.21.15.26:0/3729047200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc1586f0 0x7fbbfc158b80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009e50 tx=0x7fbbf402bed0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 -- 172.21.15.26:0/3729047200 shutdown_connections 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 --2- 172.21.15.26:0/3729047200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc1586f0 0x7fbbfc158b80 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 --2- 172.21.15.26:0/3729047200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc155ba0 0x7fbbfc0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 -- 172.21.15.26:0/3729047200 >> 172.21.15.26:0/3729047200 conn(0x7fbbfc06b1a0 msgr2=0x7fbbfc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 -- 172.21.15.26:0/3729047200 shutdown_connections 2024-03-06T07:24:06.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.805+0000 7fbc00a29640 1 -- 172.21.15.26:0/3729047200 wait complete. 2024-03-06T07:24:06.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 Processor -- start 2024-03-06T07:24:06.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 -- start start 2024-03-06T07:24:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc0c5480 0x7fbbfc151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbfc155ba0 0x7fbbfc1ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc1586f0 0x7fbbfc1e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc155680 con 0x7fbbfc0c5480 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc156000 con 0x7fbbfc155ba0 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbc00a29640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc152a20 con 0x7fbbfc1586f0 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbbfa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc0c5480 0x7fbbfc151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbbfa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc0c5480 0x7fbbfc151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35202/0 (socket says 172.21.15.26:35202) 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.807+0000 7fbbfa59c640 1 -- 172.21.15.26:0/417071867 learned_addr learned my addr 172.21.15.26:0/417071867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbf3fff640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbfc155ba0 0x7fbbfc1ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfad9d640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc1586f0 0x7fbbfc1e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfa59c640 1 -- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc1586f0 msgr2=0x7fbbfc1e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfa59c640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc1586f0 0x7fbbfc1e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfa59c640 1 -- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbfc155ba0 msgr2=0x7fbbfc1ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfa59c640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbfc155ba0 0x7fbbfc1ee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfa59c640 1 -- 172.21.15.26:0/417071867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0ad2a0 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfad9d640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc1586f0 0x7fbbfc1e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbfa59c640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc0c5480 0x7fbbfc151170 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c3c0 tx=0x7fbbec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbf2ffd640 1 -- 172.21.15.26:0/417071867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0052d0 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbf2ffd640 1 -- 172.21.15.26:0/417071867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec005490 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc1eeb10 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbbf2ffd640 1 -- 172.21.15.26:0/417071867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec007260 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.808+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc0c4ed0 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.809+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc1eef00 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.810+0000 7fbbf2ffd640 1 -- 172.21.15.26:0/417071867 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbec007400 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.810+0000 7fbbf2ffd640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.810+0000 7fbbf2ffd640 1 -- 172.21.15.26:0/417071867 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbbec089780 con 0x7fbbfc0c5480 2024-03-06T07:24:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.810+0000 7fbbf3fff640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.811+0000 7fbbf3fff640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0c59d0 tx=0x7fbbe4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:06.815+0000 7fbbf2ffd640 1 -- 172.21.15.26:0/417071867 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbbec057e20 con 0x7fbbfc0c5480 2024-03-06T07:24:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.034+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbfc04f330 con 0x7fbbfc0c5480 2024-03-06T07:24:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.034+0000 7fbbf2ffd640 1 -- 172.21.15.26:0/417071867 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4979 (secure 0 0 0) 0x7fbbec05bec0 con 0x7fbbfc0c5480 2024-03-06T07:24:07.036 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:07.036 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:06.033301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1745266733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1745266733},{"type":"v1","addr":"172.21.15.26:6835","nonce":1745266733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-03-06T07:24:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbdc06b4d0 msgr2=0x7fbbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0c59d0 tx=0x7fbbe4008040 comp rx=0 tx=0).stop 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc0c5480 msgr2=0x7fbbfc151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc0c5480 0x7fbbfc151170 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c3c0 tx=0x7fbbec00c880 comp rx=0 tx=0).stop 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 shutdown_connections 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbfc1586f0 0x7fbbfc1e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbfc155ba0 0x7fbbfc1ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 --2- 172.21.15.26:0/417071867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbfc0c5480 0x7fbbfc151170 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 >> 172.21.15.26:0/417071867 conn(0x7fbbfc06b1a0 msgr2=0x7fbbfc06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 shutdown_connections 2024-03-06T07:24:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.036+0000 7fbc00a29640 1 -- 172.21.15.26:0/417071867 wait complete. 2024-03-06T07:24:07.049 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc33d3c9640 1 Processor -- start 2024-03-06T07:24:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc33d3c9640 1 -- start start 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc33d3c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 0x7fc338159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc33d3c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380c7c60 con 0x7fc3381527b0 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc33d3c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380c8060 con 0x7fc3380c88b0 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc33d3c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc338159640 con 0x7fc3380c7180 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc336ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc3381527b0 0x7fc338155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37128/0 (socket says 172.21.15.26:37128) 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.149+0000 7fc336ffd640 1 -- 172.21.15.26:0/1845383801 learned_addr learned my addr 172.21.15.26:0/1845383801 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc3367fc640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 0x7fc338159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759610486 0 0) 0x7fc328001490 con 0x7fc3381527b0 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc324003520 con 0x7fc3381527b0 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1403257088 0 0) 0x7fc324003520 con 0x7fc3381527b0 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 msgr2=0x7fc338159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 0x7fc338159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 >> v1:172.21.15.117:6789/0 conn(0x7fc3380c88b0 legacy=0x7fc3380c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380c76a0 con 0x7fc3381527b0 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fc328001bc0 con 0x7fc3381527b0 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 >> v1:172.21.15.26:6789/0 conn(0x7fc3381527b0 legacy=0x7fc338155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 0x7fc3240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324003f20 0x7fc324004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc3240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338159640 con 0x7fc324004d40 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32400bad0 con 0x7fc324003f20 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32400bcd0 con 0x7fc3380c7180 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc3377fe640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc3240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc336ffd640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 0x7fc3240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.150+0000 7fc3367fc640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324003f20 0x7fc324004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.151+0000 7fc3377fe640 1 -- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 msgr2=0x7fc3240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.151+0000 7fc3377fe640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3380c7180 0x7fc3240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.151+0000 7fc3377fe640 1 -- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324003f20 msgr2=0x7fc324004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.151+0000 7fc3377fe640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324003f20 0x7fc324004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.151+0000 7fc3377fe640 1 -- 172.21.15.26:0/1845383801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc324003520 con 0x7fc324004d40 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.151+0000 7fc3377fe640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc3240051c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc330001860 tx=0x7fc324005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9301f19c7d354f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330038070 con 0x7fc324004d40 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc330033410 con 0x7fc324004d40 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc335ffb640 1 -- 172.21.15.26:0/1845383801 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330001430 con 0x7fc324004d40 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc33d3c9640 1 -- 172.21.15.26:0/1845383801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 msgr2=0x7fc3240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc33d3c9640 1 --2- 172.21.15.26:0/1845383801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc3240051c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc330001860 tx=0x7fc324005830 comp rx=0 tx=0).stop 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc3377fe640 1 -- 172.21.15.26:0/1845383801 reap_dead start 2024-03-06T07:24:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc33d3c9640 1 -- 172.21.15.26:0/1845383801 shutdown_connections 2024-03-06T07:24:07.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc33d3c9640 1 -- 172.21.15.26:0/1845383801 >> 172.21.15.26:0/1845383801 conn(0x7fc3380ac560 msgr2=0x7fc3380ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc33d3c9640 1 -- 172.21.15.26:0/1845383801 shutdown_connections 2024-03-06T07:24:07.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.152+0000 7fc33d3c9640 1 -- 172.21.15.26:0/1845383801 wait complete. 2024-03-06T07:24:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.154+0000 7fc33d3c9640 1 Processor -- start 2024-03-06T07:24:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.154+0000 7fc33d3c9640 1 -- start start 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc33d3c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc338147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc33d3c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc338154c20 0x7fc3381574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc33d3c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc338157a30 0x7fc3381edea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc33d3c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338155600 con 0x7fc324004d40 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc33d3c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338155320 con 0x7fc338154c20 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc33d3c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380c8d60 con 0x7fc338157a30 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc336ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc338147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc336ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc338147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45174/0 (socket says 172.21.15.26:45174) 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.155+0000 7fc336ffd640 1 -- 172.21.15.26:0/3661955466 learned_addr learned my addr 172.21.15.26:0/3661955466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc3367fc640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc338154c20 0x7fc3381574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc3377fe640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc338157a30 0x7fc3381edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc336ffd640 1 -- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc338157a30 msgr2=0x7fc3381edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc336ffd640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc338157a30 0x7fc3381edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc336ffd640 1 -- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc338154c20 msgr2=0x7fc3381574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc336ffd640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc338154c20 0x7fc3381574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc336ffd640 1 -- 172.21.15.26:0/3661955466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338155110 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc3367fc640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc338154c20 0x7fc3381574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc336ffd640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc338147340 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc32800ac60 tx=0x7fc328007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc32f7fe640 1 -- 172.21.15.26:0/3661955466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328017020 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc32f7fe640 1 -- 172.21.15.26:0/3661955466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3280133f0 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3381eaa50 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc32f7fe640 1 -- 172.21.15.26:0/3661955466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280137e0 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.156+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3381eb010 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.157+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33804f330 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.158+0000 7fc32f7fe640 1 -- 172.21.15.26:0/3661955466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc32801a020 con 0x7fc324004d40 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.158+0000 7fc32f7fe640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b500 0x7fc31806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.158+0000 7fc32f7fe640 1 -- 172.21.15.26:0/3661955466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc32808ae90 con 0x7fc324004d40 2024-03-06T07:24:07.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.158+0000 7fc3367fc640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b500 0x7fc31806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.159+0000 7fc3367fc640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b500 0x7fc31806d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc338155040 tx=0x7fc320008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.162+0000 7fc32f7fe640 1 -- 172.21.15.26:0/3661955466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc328021020 con 0x7fc324004d40 2024-03-06T07:24:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.344+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3380ca110 con 0x7fc324004d40 2024-03-06T07:24:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.345+0000 7fc32f7fe640 1 -- 172.21.15.26:0/3661955466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16964 (secure 0 0 0) 0x7fc3380c7c60 con 0x7fc324004d40 2024-03-06T07:24:07.346 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:07.347 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":108,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:05.665361+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:02.286561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:03.001111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b500 msgr2=0x7fc31806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b500 0x7fc31806d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc338155040 tx=0x7fc320008040 comp rx=0 tx=0).stop 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 msgr2=0x7fc338147340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc338147340 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc32800ac60 tx=0x7fc328007430 comp rx=0 tx=0).stop 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 shutdown_connections 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc31806b500 0x7fc31806d9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc338157a30 0x7fc3381edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc338154c20 0x7fc3381574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 --2- 172.21.15.26:0/3661955466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc324004d40 0x7fc338147340 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 >> 172.21.15.26:0/3661955466 conn(0x7fc3380ac560 msgr2=0x7fc3380c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 shutdown_connections 2024-03-06T07:24:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.347+0000 7fc33d3c9640 1 -- 172.21.15.26:0/3661955466 wait complete. 2024-03-06T07:24:07.359 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:24:07.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:24:07.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.460+0000 7fcdd0593640 1 Processor -- start 2024-03-06T07:24:07.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.460+0000 7fcdd0593640 1 -- start start 2024-03-06T07:24:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.461+0000 7fcdd0593640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdc80c88c0 0x7fcdc8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.461+0000 7fcdd0593640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcdc8158b60 0x7fcdc8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.461+0000 7fcdd0593640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc80c73d0 con 0x7fcdc814f360 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.461+0000 7fcdd0593640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80c8dd0 con 0x7fcdc80c88c0 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.461+0000 7fcdd0593640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80be2e0 con 0x7fcdc8158b60 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdcdb2e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdc80c88c0 0x7fcdc8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdce32f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcdc814f360 0x7fcdc80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37130/0 (socket says 172.21.15.26:37130) 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdce32f640 1 -- 172.21.15.26:0/2697567285 learned_addr learned my addr 172.21.15.26:0/2697567285 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdcdb2e640 1 -- 172.21.15.26:0/2697567285 >> v2:172.21.15.117:3300/0 conn(0x7fcdc8158b60 msgr2=0x7fcdc8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdcdb2e640 1 --2- 172.21.15.26:0/2697567285 >> v2:172.21.15.117:3300/0 conn(0x7fcdc8158b60 0x7fcdc8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdcdb2e640 1 -- 172.21.15.26:0/2697567285 >> v1:172.21.15.26:6789/0 conn(0x7fcdc814f360 legacy=0x7fcdc80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdcdb2e640 1 -- 172.21.15.26:0/2697567285 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc80c7670 con 0x7fcdc80c88c0 2024-03-06T07:24:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.462+0000 7fcdcd32d640 1 -- 172.21.15.26:0/2697567285 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205800882 0 0) 0x7fcdc80c73d0 con 0x7fcdc814f360 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.463+0000 7fcdcdb2e640 1 --2- 172.21.15.26:0/2697567285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc8155c10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8152d60 tx=0x7fcdb802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17a7cc8db4664e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.463+0000 7fcdcd32d640 1 -- 172.21.15.26:0/2697567285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80315a0 con 0x7fcdc80c88c0 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.463+0000 7fcdcd32d640 1 -- 172.21.15.26:0/2697567285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8037070 con 0x7fcdc80c88c0 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.463+0000 7fcdcd32d640 1 -- 172.21.15.26:0/2697567285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8031960 con 0x7fcdc80c88c0 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.463+0000 7fcdd0593640 1 -- 172.21.15.26:0/2697567285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 msgr2=0x7fcdc8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.463+0000 7fcdd0593640 1 --2- 172.21.15.26:0/2697567285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc8155c10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8152d60 tx=0x7fcdb802b840 comp rx=0 tx=0).stop 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.464+0000 7fcdd0593640 1 -- 172.21.15.26:0/2697567285 shutdown_connections 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.464+0000 7fcdd0593640 1 --2- 172.21.15.26:0/2697567285 >> v2:172.21.15.117:3300/0 conn(0x7fcdc8158b60 0x7fcdc8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.464+0000 7fcdd0593640 1 --2- 172.21.15.26:0/2697567285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc8155c10 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.464+0000 7fcdd0593640 1 -- 172.21.15.26:0/2697567285 >> 172.21.15.26:0/2697567285 conn(0x7fcdc80ac570 msgr2=0x7fcdc80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.464+0000 7fcdd0593640 1 -- 172.21.15.26:0/2697567285 shutdown_connections 2024-03-06T07:24:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.464+0000 7fcdd0593640 1 -- 172.21.15.26:0/2697567285 wait complete. 2024-03-06T07:24:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.465+0000 7fcdd0593640 1 Processor -- start 2024-03-06T07:24:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.465+0000 7fcdd0593640 1 -- start start 2024-03-06T07:24:07.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.466+0000 7fcdd0593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc81ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.466+0000 7fcdd0593640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdc814f360 0x7fcdc814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.466+0000 7fcdd0593640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc8158b60 0x7fcdc814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.466+0000 7fcdd0593640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80c5a50 con 0x7fcdc80c88c0 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.466+0000 7fcdd0593640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80c7c30 con 0x7fcdc8158b60 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.466+0000 7fcdd0593640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80c5bd0 con 0x7fcdc814f360 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.466+0000 7fcdce32f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc81ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc81ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45192/0 (socket says 172.21.15.26:45192) 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 -- 172.21.15.26:0/30054848 learned_addr learned my addr 172.21.15.26:0/30054848 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdcdb2e640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdc814f360 0x7fcdc814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdceb30640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc8158b60 0x7fcdc814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 -- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdc814f360 msgr2=0x7fcdc814b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdc814f360 0x7fcdc814b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 -- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc8158b60 msgr2=0x7fcdc814bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc8158b60 0x7fcdc814bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 -- 172.21.15.26:0/30054848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc81ef050 con 0x7fcdc80c88c0 2024-03-06T07:24:07.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdcdb2e640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdc814f360 0x7fcdc814b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdceb30640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc8158b60 0x7fcdc814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdce32f640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc81ee5d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c4d0 tx=0x7fcdc000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdbeffd640 1 -- 172.21.15.26:0/30054848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0018070 con 0x7fcdc80c88c0 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdbeffd640 1 -- 172.21.15.26:0/30054848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc0007500 con 0x7fcdc80c88c0 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdbeffd640 1 -- 172.21.15.26:0/30054848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0004a20 con 0x7fcdc80c88c0 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc81ec400 con 0x7fcdc80c88c0 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.467+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc814c440 con 0x7fcdc80c88c0 2024-03-06T07:24:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.468+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc804f330 con 0x7fcdc80c88c0 2024-03-06T07:24:07.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.469+0000 7fcdbeffd640 1 -- 172.21.15.26:0/30054848 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdc001b070 con 0x7fcdc80c88c0 2024-03-06T07:24:07.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.469+0000 7fcdbeffd640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcdac06b4d0 0x7fcdac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.469+0000 7fcdbeffd640 1 -- 172.21.15.26:0/30054848 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdc0089640 con 0x7fcdc80c88c0 2024-03-06T07:24:07.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.469+0000 7fcdcdb2e640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcdac06b4d0 0x7fcdac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.470+0000 7fcdcdb2e640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcdac06b4d0 0x7fcdac06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8036000 tx=0x7fcdb8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.473+0000 7fcdbeffd640 1 -- 172.21.15.26:0/30054848 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcdc00583c0 con 0x7fcdc80c88c0 2024-03-06T07:24:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.651+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdc80c81c0 con 0x7fcdac06b4d0 2024-03-06T07:24:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.657+0000 7fcdbeffd640 1 -- 172.21.15.26:0/30054848 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252494 (secure 0 0 0) 0x7fcdc80c81c0 con 0x7fcdac06b4d0 2024-03-06T07:24:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:24:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.660+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcdac06b4d0 msgr2=0x7fcdac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.660+0000 7fcdd0593640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcdac06b4d0 0x7fcdac06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8036000 tx=0x7fcdb8008030 comp rx=0 tx=0).stop 2024-03-06T07:24:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.660+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 msgr2=0x7fcdc81ee5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.660+0000 7fcdd0593640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc81ee5d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c4d0 tx=0x7fcdc000c990 comp rx=0 tx=0).stop 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 shutdown_connections 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcdac06b4d0 0x7fcdac06d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc8158b60 0x7fcdc814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdc814f360 0x7fcdc814b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 --2- 172.21.15.26:0/30054848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc80c88c0 0x7fcdc81ee5d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 >> 172.21.15.26:0/30054848 conn(0x7fcdc80ac570 msgr2=0x7fcdc80c9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 shutdown_connections 2024-03-06T07:24:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.661+0000 7fcdd0593640 1 -- 172.21.15.26:0/30054848 wait complete. 2024-03-06T07:24:07.673 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-03-06T07:24:07.080374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259924,"kb_used_data":10488,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714796,"statfs":{"total":773094113280,"available":772827951104,"internally_reserved":0,"allocated":10739712,"data_stored":6278043,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.420933"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679432+0000","last_change":"2024-03-06T07:24:04.678785+0000","last_active":"2024-03-06T07:24:04.679432+0000","last_peered":"2024-03-06T07:24:04.679432+0000","last_clean":"2024-03-06T07:24:04.679432+0000","last_became_active":"2024-03-06T07:24:04.678526+0000","last_became_peered":"2024-03-06T07:24:04.678526+0000","last_unstale":"2024-03-06T07:24:04.679432+0000","last_undegraded":"2024-03-06T07:24:04.679432+0000","last_fullsized":"2024-03-06T07:24:04.679432+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:15.486735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.688817+0000","last_change":"2024-03-06T07:24:04.682976+0000","last_active":"2024-03-06T07:24:04.688817+0000","last_peered":"2024-03-06T07:24:04.688817+0000","last_clean":"2024-03-06T07:24:04.688817+0000","last_became_active":"2024-03-06T07:24:04.681993+0000","last_became_peered":"2024-03-06T07:24:04.681993+0000","last_unstale":"2024-03-06T07:24:04.688817+0000","last_undegraded":"2024-03-06T07:24:04.688817+0000","last_fullsized":"2024-03-06T07:24:04.688817+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:28:11.772430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696199+0000","last_change":"2024-03-06T07:24:04.685766+0000","last_active":"2024-03-06T07:24:04.696199+0000","last_peered":"2024-03-06T07:24:04.696199+0000","last_clean":"2024-03-06T07:24:04.696199+0000","last_became_active":"2024-03-06T07:24:04.685499+0000","last_became_peered":"2024-03-06T07:24:04.685499+0000","last_unstale":"2024-03-06T07:24:04.696199+0000","last_undegraded":"2024-03-06T07:24:04.696199+0000","last_fullsized":"2024-03-06T07:24:04.696199+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:18:00.939752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679211+0000","last_change":"2024-03-06T07:24:04.673507+0000","last_active":"2024-03-06T07:24:04.679211+0000","last_peered":"2024-03-06T07:24:04.679211+0000","last_clean":"2024-03-06T07:24:04.679211+0000","last_became_active":"2024-03-06T07:24:04.673185+0000","last_became_peered":"2024-03-06T07:24:04.673185+0000","last_unstale":"2024-03-06T07:24:04.679211+0000","last_undegraded":"2024-03-06T07:24:04.679211+0000","last_fullsized":"2024-03-06T07:24:04.679211+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:38.089819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.683774+0000","last_change":"2024-03-06T07:24:04.681818+0000","last_active":"2024-03-06T07:24:04.683774+0000","last_peered":"2024-03-06T07:24:04.683774+0000","last_clean":"2024-03-06T07:24:04.683774+0000","last_became_active":"2024-03-06T07:24:04.681461+0000","last_became_peered":"2024-03-06T07:24:04.681461+0000","last_unstale":"2024-03-06T07:24:04.683774+0000","last_undegraded":"2024-03-06T07:24:04.683774+0000","last_fullsized":"2024-03-06T07:24:04.683774+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:47.901117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693784+0000","last_change":"2024-03-06T07:24:04.692244+0000","last_active":"2024-03-06T07:24:04.693784+0000","last_peered":"2024-03-06T07:24:04.693784+0000","last_clean":"2024-03-06T07:24:04.693784+0000","last_became_active":"2024-03-06T07:24:04.691975+0000","last_became_peered":"2024-03-06T07:24:04.691975+0000","last_unstale":"2024-03-06T07:24:04.693784+0000","last_undegraded":"2024-03-06T07:24:04.693784+0000","last_fullsized":"2024-03-06T07:24:04.693784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:47.105582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692404+0000","last_change":"2024-03-06T07:24:04.687610+0000","last_active":"2024-03-06T07:24:04.692404+0000","last_peered":"2024-03-06T07:24:04.692404+0000","last_clean":"2024-03-06T07:24:04.692404+0000","last_became_active":"2024-03-06T07:24:04.687338+0000","last_became_peered":"2024-03-06T07:24:04.687338+0000","last_unstale":"2024-03-06T07:24:04.692404+0000","last_undegraded":"2024-03-06T07:24:04.692404+0000","last_fullsized":"2024-03-06T07:24:04.692404+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:06.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696898+0000","last_change":"2024-03-06T07:24:04.690717+0000","last_active":"2024-03-06T07:24:04.696898+0000","last_peered":"2024-03-06T07:24:04.696898+0000","last_clean":"2024-03-06T07:24:04.696898+0000","last_became_active":"2024-03-06T07:24:04.689237+0000","last_became_peered":"2024-03-06T07:24:04.689237+0000","last_unstale":"2024-03-06T07:24:04.696898+0000","last_undegraded":"2024-03-06T07:24:04.696898+0000","last_fullsized":"2024-03-06T07:24:04.696898+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.153991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693666+0000","last_change":"2024-03-06T07:24:04.691303+0000","last_active":"2024-03-06T07:24:04.693666+0000","last_peered":"2024-03-06T07:24:04.693666+0000","last_clean":"2024-03-06T07:24:04.693666+0000","last_became_active":"2024-03-06T07:24:04.691018+0000","last_became_peered":"2024-03-06T07:24:04.691018+0000","last_unstale":"2024-03-06T07:24:04.693666+0000","last_undegraded":"2024-03-06T07:24:04.693666+0000","last_fullsized":"2024-03-06T07:24:04.693666+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:28.076328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696379+0000","last_change":"2024-03-06T07:24:04.692966+0000","last_active":"2024-03-06T07:24:04.696379+0000","last_peered":"2024-03-06T07:24:04.696379+0000","last_clean":"2024-03-06T07:24:04.696379+0000","last_became_active":"2024-03-06T07:24:04.692652+0000","last_became_peered":"2024-03-06T07:24:04.692652+0000","last_unstale":"2024-03-06T07:24:04.696379+0000","last_undegraded":"2024-03-06T07:24:04.696379+0000","last_fullsized":"2024-03-06T07:24:04.696379+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:17.366449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.686836+0000","last_change":"2024-03-06T07:24:04.682802+0000","last_active":"2024-03-06T07:24:04.686836+0000","last_peered":"2024-03-06T07:24:04.686836+0000","last_clean":"2024-03-06T07:24:04.686836+0000","last_became_active":"2024-03-06T07:24:04.682532+0000","last_became_peered":"2024-03-06T07:24:04.682532+0000","last_unstale":"2024-03-06T07:24:04.686836+0000","last_undegraded":"2024-03-06T07:24:04.686836+0000","last_fullsized":"2024-03-06T07:24:04.686836+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:16.207859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692506+0000","last_change":"2024-03-06T07:24:04.688276+0000","last_active":"2024-03-06T07:24:04.692506+0000","last_peered":"2024-03-06T07:24:04.692506+0000","last_clean":"2024-03-06T07:24:04.692506+0000","last_became_active":"2024-03-06T07:24:04.687952+0000","last_became_peered":"2024-03-06T07:24:04.687952+0000","last_unstale":"2024-03-06T07:24:04.692506+0000","last_undegraded":"2024-03-06T07:24:04.692506+0000","last_fullsized":"2024-03-06T07:24:04.692506+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:41.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.684907+0000","last_change":"2024-03-06T07:24:04.682372+0000","last_active":"2024-03-06T07:24:04.684907+0000","last_peered":"2024-03-06T07:24:04.684907+0000","last_clean":"2024-03-06T07:24:04.684907+0000","last_became_active":"2024-03-06T07:24:04.682088+0000","last_became_peered":"2024-03-06T07:24:04.682088+0000","last_unstale":"2024-03-06T07:24:04.684907+0000","last_undegraded":"2024-03-06T07:24:04.684907+0000","last_fullsized":"2024-03-06T07:24:04.684907+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:15.486735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697382+0000","last_change":"2024-03-06T07:24:04.686629+0000","last_active":"2024-03-06T07:24:04.697382+0000","last_peered":"2024-03-06T07:24:04.697382+0000","last_clean":"2024-03-06T07:24:04.697382+0000","last_became_active":"2024-03-06T07:24:04.685296+0000","last_became_peered":"2024-03-06T07:24:04.685296+0000","last_unstale":"2024-03-06T07:24:04.697382+0000","last_undegraded":"2024-03-06T07:24:04.697382+0000","last_fullsized":"2024-03-06T07:24:04.697382+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:49.098245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692631+0000","last_change":"2024-03-06T07:24:04.682748+0000","last_active":"2024-03-06T07:24:04.692631+0000","last_peered":"2024-03-06T07:24:04.692631+0000","last_clean":"2024-03-06T07:24:04.692631+0000","last_became_active":"2024-03-06T07:24:04.682388+0000","last_became_peered":"2024-03-06T07:24:04.682388+0000","last_unstale":"2024-03-06T07:24:04.692631+0000","last_undegraded":"2024-03-06T07:24:04.692631+0000","last_fullsized":"2024-03-06T07:24:04.692631+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:23.035754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692278+0000","last_change":"2024-03-06T07:24:04.688219+0000","last_active":"2024-03-06T07:24:04.692278+0000","last_peered":"2024-03-06T07:24:04.692278+0000","last_clean":"2024-03-06T07:24:04.692278+0000","last_became_active":"2024-03-06T07:24:04.687940+0000","last_became_peered":"2024-03-06T07:24:04.687940+0000","last_unstale":"2024-03-06T07:24:04.692278+0000","last_undegraded":"2024-03-06T07:24:04.692278+0000","last_fullsized":"2024-03-06T07:24:04.692278+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:40.100249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.684984+0000","last_change":"2024-03-06T07:24:04.681730+0000","last_active":"2024-03-06T07:24:04.684984+0000","last_peered":"2024-03-06T07:24:04.684984+0000","last_clean":"2024-03-06T07:24:04.684984+0000","last_became_active":"2024-03-06T07:24:04.681445+0000","last_became_peered":"2024-03-06T07:24:04.681445+0000","last_unstale":"2024-03-06T07:24:04.684984+0000","last_undegraded":"2024-03-06T07:24:04.684984+0000","last_fullsized":"2024-03-06T07:24:04.684984+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:34.000425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695395+0000","last_change":"2024-03-06T07:24:04.683306+0000","last_active":"2024-03-06T07:24:04.695395+0000","last_peered":"2024-03-06T07:24:04.695395+0000","last_clean":"2024-03-06T07:24:04.695395+0000","last_became_active":"2024-03-06T07:24:04.682772+0000","last_became_peered":"2024-03-06T07:24:04.682772+0000","last_unstale":"2024-03-06T07:24:04.695395+0000","last_undegraded":"2024-03-06T07:24:04.695395+0000","last_fullsized":"2024-03-06T07:24:04.695395+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:38.089819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692790+0000","last_change":"2024-03-06T07:24:04.685167+0000","last_active":"2024-03-06T07:24:04.692790+0000","last_peered":"2024-03-06T07:24:04.692790+0000","last_clean":"2024-03-06T07:24:04.692790+0000","last_became_active":"2024-03-06T07:24:04.684529+0000","last_became_peered":"2024-03-06T07:24:04.684529+0000","last_unstale":"2024-03-06T07:24:04.692790+0000","last_undegraded":"2024-03-06T07:24:04.692790+0000","last_fullsized":"2024-03-06T07:24:04.692790+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:23.035754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696164+0000","last_change":"2024-03-06T07:24:04.693998+0000","last_active":"2024-03-06T07:24:04.696164+0000","last_peered":"2024-03-06T07:24:04.696164+0000","last_clean":"2024-03-06T07:24:04.696164+0000","last_became_active":"2024-03-06T07:24:04.693707+0000","last_became_peered":"2024-03-06T07:24:04.693707+0000","last_unstale":"2024-03-06T07:24:04.696164+0000","last_undegraded":"2024-03-06T07:24:04.696164+0000","last_fullsized":"2024-03-06T07:24:04.696164+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:33.519123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.689911+0000","last_change":"2024-03-06T07:24:04.686080+0000","last_active":"2024-03-06T07:24:04.689911+0000","last_peered":"2024-03-06T07:24:04.689911+0000","last_clean":"2024-03-06T07:24:04.689911+0000","last_became_active":"2024-03-06T07:24:04.685674+0000","last_became_peered":"2024-03-06T07:24:04.685674+0000","last_unstale":"2024-03-06T07:24:04.689911+0000","last_undegraded":"2024-03-06T07:24:04.689911+0000","last_fullsized":"2024-03-06T07:24:04.689911+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:29.990981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693518+0000","last_change":"2024-03-06T07:24:04.689579+0000","last_active":"2024-03-06T07:24:04.693518+0000","last_peered":"2024-03-06T07:24:04.693518+0000","last_clean":"2024-03-06T07:24:04.693518+0000","last_became_active":"2024-03-06T07:24:04.688900+0000","last_became_peered":"2024-03-06T07:24:04.688900+0000","last_unstale":"2024-03-06T07:24:04.693518+0000","last_undegraded":"2024-03-06T07:24:04.693518+0000","last_fullsized":"2024-03-06T07:24:04.693518+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:38.196538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.684023+0000","last_change":"2024-03-06T07:24:04.681464+0000","last_active":"2024-03-06T07:24:04.684023+0000","last_peered":"2024-03-06T07:24:04.684023+0000","last_clean":"2024-03-06T07:24:04.684023+0000","last_became_active":"2024-03-06T07:24:04.681199+0000","last_became_peered":"2024-03-06T07:24:04.681199+0000","last_unstale":"2024-03-06T07:24:04.684023+0000","last_undegraded":"2024-03-06T07:24:04.684023+0000","last_fullsized":"2024-03-06T07:24:04.684023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:27.521026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692460+0000","last_change":"2024-03-06T07:24:04.688245+0000","last_active":"2024-03-06T07:24:04.692460+0000","last_peered":"2024-03-06T07:24:04.692460+0000","last_clean":"2024-03-06T07:24:04.692460+0000","last_became_active":"2024-03-06T07:24:04.687929+0000","last_became_peered":"2024-03-06T07:24:04.687929+0000","last_unstale":"2024-03-06T07:24:04.692460+0000","last_undegraded":"2024-03-06T07:24:04.692460+0000","last_fullsized":"2024-03-06T07:24:04.692460+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:23.244706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692479+0000","last_change":"2024-03-06T07:24:04.687412+0000","last_active":"2024-03-06T07:24:04.692479+0000","last_peered":"2024-03-06T07:24:04.692479+0000","last_clean":"2024-03-06T07:24:04.692479+0000","last_became_active":"2024-03-06T07:24:04.687140+0000","last_became_peered":"2024-03-06T07:24:04.687140+0000","last_unstale":"2024-03-06T07:24:04.692479+0000","last_undegraded":"2024-03-06T07:24:04.692479+0000","last_fullsized":"2024-03-06T07:24:04.692479+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:50.575791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.682499+0000","last_change":"2024-03-06T07:24:04.680986+0000","last_active":"2024-03-06T07:24:04.682499+0000","last_peered":"2024-03-06T07:24:04.682499+0000","last_clean":"2024-03-06T07:24:04.682499+0000","last_became_active":"2024-03-06T07:24:04.680719+0000","last_became_peered":"2024-03-06T07:24:04.680719+0000","last_unstale":"2024-03-06T07:24:04.682499+0000","last_undegraded":"2024-03-06T07:24:04.682499+0000","last_fullsized":"2024-03-06T07:24:04.682499+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:47.540962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.681805+0000","last_change":"2024-03-06T07:24:04.681044+0000","last_active":"2024-03-06T07:24:04.681805+0000","last_peered":"2024-03-06T07:24:04.681805+0000","last_clean":"2024-03-06T07:24:04.681805+0000","last_became_active":"2024-03-06T07:24:04.680754+0000","last_became_peered":"2024-03-06T07:24:04.680754+0000","last_unstale":"2024-03-06T07:24:04.681805+0000","last_undegraded":"2024-03-06T07:24:04.681805+0000","last_fullsized":"2024-03-06T07:24:04.681805+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:15.486735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.687718+0000","last_change":"2024-03-06T07:24:04.683726+0000","last_active":"2024-03-06T07:24:04.687718+0000","last_peered":"2024-03-06T07:24:04.687718+0000","last_clean":"2024-03-06T07:24:04.687718+0000","last_became_active":"2024-03-06T07:24:04.683455+0000","last_became_peered":"2024-03-06T07:24:04.683455+0000","last_unstale":"2024-03-06T07:24:04.687718+0000","last_undegraded":"2024-03-06T07:24:04.687718+0000","last_fullsized":"2024-03-06T07:24:04.687718+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:08.698708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695305+0000","last_change":"2024-03-06T07:24:04.692695+0000","last_active":"2024-03-06T07:24:04.695305+0000","last_peered":"2024-03-06T07:24:04.695305+0000","last_clean":"2024-03-06T07:24:04.695305+0000","last_became_active":"2024-03-06T07:24:04.692417+0000","last_became_peered":"2024-03-06T07:24:04.692417+0000","last_unstale":"2024-03-06T07:24:04.695305+0000","last_undegraded":"2024-03-06T07:24:04.695305+0000","last_fullsized":"2024-03-06T07:24:04.695305+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:20.367676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697409+0000","last_change":"2024-03-06T07:24:04.690731+0000","last_active":"2024-03-06T07:24:04.697409+0000","last_peered":"2024-03-06T07:24:04.697409+0000","last_clean":"2024-03-06T07:24:04.697409+0000","last_became_active":"2024-03-06T07:24:04.689308+0000","last_became_peered":"2024-03-06T07:24:04.689308+0000","last_unstale":"2024-03-06T07:24:04.697409+0000","last_undegraded":"2024-03-06T07:24:04.697409+0000","last_fullsized":"2024-03-06T07:24:04.697409+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:28.076328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701322+0000","last_change":"2024-03-06T07:24:04.693519+0000","last_active":"2024-03-06T07:24:04.701322+0000","last_peered":"2024-03-06T07:24:04.701322+0000","last_clean":"2024-03-06T07:24:04.701322+0000","last_became_active":"2024-03-06T07:24:04.693250+0000","last_became_peered":"2024-03-06T07:24:04.693250+0000","last_unstale":"2024-03-06T07:24:04.701322+0000","last_undegraded":"2024-03-06T07:24:04.701322+0000","last_fullsized":"2024-03-06T07:24:04.701322+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:52.798080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.690186+0000","last_change":"2024-03-06T07:24:04.685356+0000","last_active":"2024-03-06T07:24:04.690186+0000","last_peered":"2024-03-06T07:24:04.690186+0000","last_clean":"2024-03-06T07:24:04.690186+0000","last_became_active":"2024-03-06T07:24:04.685066+0000","last_became_peered":"2024-03-06T07:24:04.685066+0000","last_unstale":"2024-03-06T07:24:04.690186+0000","last_undegraded":"2024-03-06T07:24:04.690186+0000","last_fullsized":"2024-03-06T07:24:04.690186+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:30.138870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696417+0000","last_change":"2024-03-06T07:24:04.694303+0000","last_active":"2024-03-06T07:24:04.696417+0000","last_peered":"2024-03-06T07:24:04.696417+0000","last_clean":"2024-03-06T07:24:04.696417+0000","last_became_active":"2024-03-06T07:24:04.693789+0000","last_became_peered":"2024-03-06T07:24:04.693789+0000","last_unstale":"2024-03-06T07:24:04.696417+0000","last_undegraded":"2024-03-06T07:24:04.696417+0000","last_fullsized":"2024-03-06T07:24:04.696417+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:38.196538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702091+0000","last_change":"2024-03-06T07:24:03.682632+0000","last_active":"2024-03-06T07:24:04.702091+0000","last_peered":"2024-03-06T07:24:04.702091+0000","last_clean":"2024-03-06T07:24:04.702091+0000","last_became_active":"2024-03-06T07:24:03.682342+0000","last_became_peered":"2024-03-06T07:24:03.682342+0000","last_unstale":"2024-03-06T07:24:04.702091+0000","last_undegraded":"2024-03-06T07:24:04.702091+0000","last_fullsized":"2024-03-06T07:24:04.702091+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:16.284868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.699202+0000","last_change":"2024-03-06T07:24:04.698155+0000","last_active":"2024-03-06T07:24:04.699202+0000","last_peered":"2024-03-06T07:24:04.699202+0000","last_clean":"2024-03-06T07:24:04.699202+0000","last_became_active":"2024-03-06T07:24:04.697879+0000","last_became_peered":"2024-03-06T07:24:04.697879+0000","last_unstale":"2024-03-06T07:24:04.699202+0000","last_undegraded":"2024-03-06T07:24:04.699202+0000","last_fullsized":"2024-03-06T07:24:04.699202+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:57.714898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695489+0000","last_change":"2024-03-06T07:24:03.680197+0000","last_active":"2024-03-06T07:24:04.695489+0000","last_peered":"2024-03-06T07:24:04.695489+0000","last_clean":"2024-03-06T07:24:04.695489+0000","last_became_active":"2024-03-06T07:24:03.679925+0000","last_became_peered":"2024-03-06T07:24:03.679925+0000","last_unstale":"2024-03-06T07:24:04.695489+0000","last_undegraded":"2024-03-06T07:24:04.695489+0000","last_fullsized":"2024-03-06T07:24:04.695489+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:16.284868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.690228+0000","last_change":"2024-03-06T07:24:04.686651+0000","last_active":"2024-03-06T07:24:04.690228+0000","last_peered":"2024-03-06T07:24:04.690228+0000","last_clean":"2024-03-06T07:24:04.690228+0000","last_became_active":"2024-03-06T07:24:04.686361+0000","last_became_peered":"2024-03-06T07:24:04.686361+0000","last_unstale":"2024-03-06T07:24:04.690228+0000","last_undegraded":"2024-03-06T07:24:04.690228+0000","last_fullsized":"2024-03-06T07:24:04.690228+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:47.755350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701969+0000","last_change":"2024-03-06T07:24:03.679292+0000","last_active":"2024-03-06T07:24:04.701969+0000","last_peered":"2024-03-06T07:24:04.701969+0000","last_clean":"2024-03-06T07:24:04.701969+0000","last_became_active":"2024-03-06T07:24:03.678728+0000","last_became_peered":"2024-03-06T07:24:03.678728+0000","last_unstale":"2024-03-06T07:24:04.701969+0000","last_undegraded":"2024-03-06T07:24:04.701969+0000","last_fullsized":"2024-03-06T07:24:04.701969+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:51.791521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679307+0000","last_change":"2024-03-06T07:24:04.676037+0000","last_active":"2024-03-06T07:24:04.679307+0000","last_peered":"2024-03-06T07:24:04.679307+0000","last_clean":"2024-03-06T07:24:04.679307+0000","last_became_active":"2024-03-06T07:24:04.675629+0000","last_became_peered":"2024-03-06T07:24:04.675629+0000","last_unstale":"2024-03-06T07:24:04.679307+0000","last_undegraded":"2024-03-06T07:24:04.679307+0000","last_fullsized":"2024-03-06T07:24:04.679307+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:49.098245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697951+0000","last_change":"2024-03-06T07:24:03.682239+0000","last_active":"2024-03-06T07:24:04.697951+0000","last_peered":"2024-03-06T07:24:04.697951+0000","last_clean":"2024-03-06T07:24:04.697951+0000","last_became_active":"2024-03-06T07:24:03.681976+0000","last_became_peered":"2024-03-06T07:24:03.681976+0000","last_unstale":"2024-03-06T07:24:04.697951+0000","last_undegraded":"2024-03-06T07:24:04.697951+0000","last_fullsized":"2024-03-06T07:24:04.697951+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:00.737487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.685602+0000","last_change":"2024-03-06T07:24:04.683269+0000","last_active":"2024-03-06T07:24:04.685602+0000","last_peered":"2024-03-06T07:24:04.685602+0000","last_clean":"2024-03-06T07:24:04.685602+0000","last_became_active":"2024-03-06T07:24:04.682989+0000","last_became_peered":"2024-03-06T07:24:04.682989+0000","last_unstale":"2024-03-06T07:24:04.685602+0000","last_undegraded":"2024-03-06T07:24:04.685602+0000","last_fullsized":"2024-03-06T07:24:04.685602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:47.901117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696687+0000","last_change":"2024-03-06T07:24:03.686816+0000","last_active":"2024-03-06T07:24:04.696687+0000","last_peered":"2024-03-06T07:24:04.696687+0000","last_clean":"2024-03-06T07:24:04.696687+0000","last_became_active":"2024-03-06T07:24:03.686569+0000","last_became_peered":"2024-03-06T07:24:03.686569+0000","last_unstale":"2024-03-06T07:24:04.696687+0000","last_undegraded":"2024-03-06T07:24:04.696687+0000","last_fullsized":"2024-03-06T07:24:04.696687+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:32.993866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692655+0000","last_change":"2024-03-06T07:24:04.684168+0000","last_active":"2024-03-06T07:24:04.692655+0000","last_peered":"2024-03-06T07:24:04.692655+0000","last_clean":"2024-03-06T07:24:04.692655+0000","last_became_active":"2024-03-06T07:24:04.683857+0000","last_became_peered":"2024-03-06T07:24:04.683857+0000","last_unstale":"2024-03-06T07:24:04.692655+0000","last_undegraded":"2024-03-06T07:24:04.692655+0000","last_fullsized":"2024-03-06T07:24:04.692655+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:17.366449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.700688+0000","last_change":"2024-03-06T07:24:03.681122+0000","last_active":"2024-03-06T07:24:04.700688+0000","last_peered":"2024-03-06T07:24:04.700688+0000","last_clean":"2024-03-06T07:24:04.700688+0000","last_became_active":"2024-03-06T07:24:03.680825+0000","last_became_peered":"2024-03-06T07:24:03.680825+0000","last_unstale":"2024-03-06T07:24:04.700688+0000","last_undegraded":"2024-03-06T07:24:04.700688+0000","last_fullsized":"2024-03-06T07:24:04.700688+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:20.143531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.688324+0000","last_change":"2024-03-06T07:24:04.681126+0000","last_active":"2024-03-06T07:24:04.688324+0000","last_peered":"2024-03-06T07:24:04.688324+0000","last_clean":"2024-03-06T07:24:04.688324+0000","last_became_active":"2024-03-06T07:24:04.680864+0000","last_became_peered":"2024-03-06T07:24:04.680864+0000","last_unstale":"2024-03-06T07:24:04.688324+0000","last_undegraded":"2024-03-06T07:24:04.688324+0000","last_fullsized":"2024-03-06T07:24:04.688324+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:08.751391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.676515+0000","last_change":"2024-03-06T07:24:03.679003+0000","last_active":"2024-03-06T07:24:04.676515+0000","last_peered":"2024-03-06T07:24:04.676515+0000","last_clean":"2024-03-06T07:24:04.676515+0000","last_became_active":"2024-03-06T07:24:03.678718+0000","last_became_peered":"2024-03-06T07:24:03.678718+0000","last_unstale":"2024-03-06T07:24:04.676515+0000","last_undegraded":"2024-03-06T07:24:04.676515+0000","last_fullsized":"2024-03-06T07:24:04.676515+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:17:55.524547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692221+0000","last_change":"2024-03-06T07:24:04.689963+0000","last_active":"2024-03-06T07:24:04.692221+0000","last_peered":"2024-03-06T07:24:04.692221+0000","last_clean":"2024-03-06T07:24:04.692221+0000","last_became_active":"2024-03-06T07:24:04.689688+0000","last_became_peered":"2024-03-06T07:24:04.689688+0000","last_unstale":"2024-03-06T07:24:04.692221+0000","last_undegraded":"2024-03-06T07:24:04.692221+0000","last_fullsized":"2024-03-06T07:24:04.692221+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:16.207859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.678792+0000","last_change":"2024-03-06T07:24:03.680294+0000","last_active":"2024-03-06T07:24:04.678792+0000","last_peered":"2024-03-06T07:24:04.678792+0000","last_clean":"2024-03-06T07:24:04.678792+0000","last_became_active":"2024-03-06T07:24:03.680035+0000","last_became_peered":"2024-03-06T07:24:03.680035+0000","last_unstale":"2024-03-06T07:24:04.678792+0000","last_undegraded":"2024-03-06T07:24:04.678792+0000","last_fullsized":"2024-03-06T07:24:04.678792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:17.383767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.687414+0000","last_change":"2024-03-06T07:24:04.683624+0000","last_active":"2024-03-06T07:24:04.687414+0000","last_peered":"2024-03-06T07:24:04.687414+0000","last_clean":"2024-03-06T07:24:04.687414+0000","last_became_active":"2024-03-06T07:24:04.683361+0000","last_became_peered":"2024-03-06T07:24:04.683361+0000","last_unstale":"2024-03-06T07:24:04.687414+0000","last_undegraded":"2024-03-06T07:24:04.687414+0000","last_fullsized":"2024-03-06T07:24:04.687414+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:47.901117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.674579+0000","last_change":"2024-03-06T07:24:03.686247+0000","last_active":"2024-03-06T07:24:04.674579+0000","last_peered":"2024-03-06T07:24:04.674579+0000","last_clean":"2024-03-06T07:24:04.674579+0000","last_became_active":"2024-03-06T07:24:03.685959+0000","last_became_peered":"2024-03-06T07:24:03.685959+0000","last_unstale":"2024-03-06T07:24:04.674579+0000","last_undegraded":"2024-03-06T07:24:04.674579+0000","last_fullsized":"2024-03-06T07:24:04.674579+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:44.619751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.706908+0000","last_change":"2024-03-06T07:24:04.701767+0000","last_active":"2024-03-06T07:24:04.706908+0000","last_peered":"2024-03-06T07:24:04.706908+0000","last_clean":"2024-03-06T07:24:04.706908+0000","last_became_active":"2024-03-06T07:24:04.701472+0000","last_became_peered":"2024-03-06T07:24:04.701472+0000","last_unstale":"2024-03-06T07:24:04.706908+0000","last_undegraded":"2024-03-06T07:24:04.706908+0000","last_fullsized":"2024-03-06T07:24:04.706908+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:38.089819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.677209+0000","last_change":"2024-03-06T07:24:03.679962+0000","last_active":"2024-03-06T07:24:04.677209+0000","last_peered":"2024-03-06T07:24:04.677209+0000","last_clean":"2024-03-06T07:24:04.677209+0000","last_became_active":"2024-03-06T07:24:03.679693+0000","last_became_peered":"2024-03-06T07:24:03.679693+0000","last_unstale":"2024-03-06T07:24:04.677209+0000","last_undegraded":"2024-03-06T07:24:04.677209+0000","last_fullsized":"2024-03-06T07:24:04.677209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:10.264883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696840+0000","last_change":"2024-03-06T07:24:04.688141+0000","last_active":"2024-03-06T07:24:04.696840+0000","last_peered":"2024-03-06T07:24:04.696840+0000","last_clean":"2024-03-06T07:24:04.696840+0000","last_became_active":"2024-03-06T07:24:04.687886+0000","last_became_peered":"2024-03-06T07:24:04.687886+0000","last_unstale":"2024-03-06T07:24:04.696840+0000","last_undegraded":"2024-03-06T07:24:04.696840+0000","last_fullsized":"2024-03-06T07:24:04.696840+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:52.255797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"107'3","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696066+0000","last_change":"2024-03-06T07:24:03.682450+0000","last_active":"2024-03-06T07:24:04.696066+0000","last_peered":"2024-03-06T07:24:04.696066+0000","last_clean":"2024-03-06T07:24:04.696066+0000","last_became_active":"2024-03-06T07:24:03.682030+0000","last_became_peered":"2024-03-06T07:24:03.682030+0000","last_unstale":"2024-03-06T07:24:04.696066+0000","last_undegraded":"2024-03-06T07:24:04.696066+0000","last_fullsized":"2024-03-06T07:24:04.696066+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:50:08.123282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701378+0000","last_change":"2024-03-06T07:24:04.693710+0000","last_active":"2024-03-06T07:24:04.701378+0000","last_peered":"2024-03-06T07:24:04.701378+0000","last_clean":"2024-03-06T07:24:04.701378+0000","last_became_active":"2024-03-06T07:24:04.693434+0000","last_became_peered":"2024-03-06T07:24:04.693434+0000","last_unstale":"2024-03-06T07:24:04.701378+0000","last_undegraded":"2024-03-06T07:24:04.701378+0000","last_fullsized":"2024-03-06T07:24:04.701378+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:08.751391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"107'6","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.704445+0000","last_change":"2024-03-06T07:24:03.683633+0000","last_active":"2024-03-06T07:24:04.704445+0000","last_peered":"2024-03-06T07:24:04.704445+0000","last_clean":"2024-03-06T07:24:04.704445+0000","last_became_active":"2024-03-06T07:24:03.683135+0000","last_became_peered":"2024-03-06T07:24:03.683135+0000","last_unstale":"2024-03-06T07:24:04.704445+0000","last_undegraded":"2024-03-06T07:24:04.704445+0000","last_fullsized":"2024-03-06T07:24:04.704445+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:51.791521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701933+0000","last_change":"2024-03-06T07:24:04.696886+0000","last_active":"2024-03-06T07:24:04.701933+0000","last_peered":"2024-03-06T07:24:04.701933+0000","last_clean":"2024-03-06T07:24:04.701933+0000","last_became_active":"2024-03-06T07:24:04.696662+0000","last_became_peered":"2024-03-06T07:24:04.696662+0000","last_unstale":"2024-03-06T07:24:04.701933+0000","last_undegraded":"2024-03-06T07:24:04.701933+0000","last_fullsized":"2024-03-06T07:24:04.701933+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:16.207859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.704140+0000","last_change":"2024-03-06T07:24:03.678845+0000","last_active":"2024-03-06T07:24:04.704140+0000","last_peered":"2024-03-06T07:24:04.704140+0000","last_clean":"2024-03-06T07:24:04.704140+0000","last_became_active":"2024-03-06T07:24:03.678529+0000","last_became_peered":"2024-03-06T07:24:03.678529+0000","last_unstale":"2024-03-06T07:24:04.704140+0000","last_undegraded":"2024-03-06T07:24:04.704140+0000","last_fullsized":"2024-03-06T07:24:04.704140+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:29.132311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.685633+0000","last_change":"2024-03-06T07:24:04.684170+0000","last_active":"2024-03-06T07:24:04.685633+0000","last_peered":"2024-03-06T07:24:04.685633+0000","last_clean":"2024-03-06T07:24:04.685633+0000","last_became_active":"2024-03-06T07:24:04.683891+0000","last_became_peered":"2024-03-06T07:24:04.683891+0000","last_unstale":"2024-03-06T07:24:04.685633+0000","last_undegraded":"2024-03-06T07:24:04.685633+0000","last_fullsized":"2024-03-06T07:24:04.685633+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:23.035754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.673743+0000","last_change":"2024-03-06T07:24:03.677156+0000","last_active":"2024-03-06T07:24:04.673743+0000","last_peered":"2024-03-06T07:24:04.673743+0000","last_clean":"2024-03-06T07:24:04.673743+0000","last_became_active":"2024-03-06T07:24:03.676900+0000","last_became_peered":"2024-03-06T07:24:03.676900+0000","last_unstale":"2024-03-06T07:24:04.673743+0000","last_undegraded":"2024-03-06T07:24:04.673743+0000","last_fullsized":"2024-03-06T07:24:04.673743+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:20.317474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.682438+0000","last_change":"2024-03-06T07:24:04.677402+0000","last_active":"2024-03-06T07:24:04.682438+0000","last_peered":"2024-03-06T07:24:04.682438+0000","last_clean":"2024-03-06T07:24:04.682438+0000","last_became_active":"2024-03-06T07:24:04.677130+0000","last_became_peered":"2024-03-06T07:24:04.677130+0000","last_unstale":"2024-03-06T07:24:04.682438+0000","last_undegraded":"2024-03-06T07:24:04.682438+0000","last_fullsized":"2024-03-06T07:24:04.682438+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:19.658462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.673244+0000","last_change":"2024-03-06T07:24:03.684357+0000","last_active":"2024-03-06T07:24:04.673244+0000","last_peered":"2024-03-06T07:24:04.673244+0000","last_clean":"2024-03-06T07:24:04.673244+0000","last_became_active":"2024-03-06T07:24:03.684091+0000","last_became_peered":"2024-03-06T07:24:03.684091+0000","last_unstale":"2024-03-06T07:24:04.673244+0000","last_undegraded":"2024-03-06T07:24:04.673244+0000","last_fullsized":"2024-03-06T07:24:04.673244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:46.748792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695804+0000","last_change":"2024-03-06T07:24:04.692313+0000","last_active":"2024-03-06T07:24:04.695804+0000","last_peered":"2024-03-06T07:24:04.695804+0000","last_clean":"2024-03-06T07:24:04.695804+0000","last_became_active":"2024-03-06T07:24:04.692034+0000","last_became_peered":"2024-03-06T07:24:04.692034+0000","last_unstale":"2024-03-06T07:24:04.695804+0000","last_undegraded":"2024-03-06T07:24:04.695804+0000","last_fullsized":"2024-03-06T07:24:04.695804+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:21.324032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693491+0000","last_change":"2024-03-06T07:24:03.686441+0000","last_active":"2024-03-06T07:24:04.693491+0000","last_peered":"2024-03-06T07:24:04.693491+0000","last_clean":"2024-03-06T07:24:04.693491+0000","last_became_active":"2024-03-06T07:24:03.686116+0000","last_became_peered":"2024-03-06T07:24:03.686116+0000","last_unstale":"2024-03-06T07:24:04.693491+0000","last_undegraded":"2024-03-06T07:24:04.693491+0000","last_fullsized":"2024-03-06T07:24:04.693491+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:01.728277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.694108+0000","last_change":"2024-03-06T07:24:04.690422+0000","last_active":"2024-03-06T07:24:04.694108+0000","last_peered":"2024-03-06T07:24:04.694108+0000","last_clean":"2024-03-06T07:24:04.694108+0000","last_became_active":"2024-03-06T07:24:04.688975+0000","last_became_peered":"2024-03-06T07:24:04.688975+0000","last_unstale":"2024-03-06T07:24:04.694108+0000","last_undegraded":"2024-03-06T07:24:04.694108+0000","last_fullsized":"2024-03-06T07:24:04.694108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:17:56.531106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.699107+0000","last_change":"2024-03-06T07:24:03.684908+0000","last_active":"2024-03-06T07:24:04.699107+0000","last_peered":"2024-03-06T07:24:04.699107+0000","last_clean":"2024-03-06T07:24:04.699107+0000","last_became_active":"2024-03-06T07:24:03.684623+0000","last_became_peered":"2024-03-06T07:24:03.684623+0000","last_unstale":"2024-03-06T07:24:04.699107+0000","last_undegraded":"2024-03-06T07:24:04.699107+0000","last_fullsized":"2024-03-06T07:24:04.699107+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:58.570744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.687348+0000","last_change":"2024-03-06T07:24:04.684997+0000","last_active":"2024-03-06T07:24:04.687348+0000","last_peered":"2024-03-06T07:24:04.687348+0000","last_clean":"2024-03-06T07:24:04.687348+0000","last_became_active":"2024-03-06T07:24:04.684737+0000","last_became_peered":"2024-03-06T07:24:04.684737+0000","last_unstale":"2024-03-06T07:24:04.687348+0000","last_undegraded":"2024-03-06T07:24:04.687348+0000","last_fullsized":"2024-03-06T07:24:04.687348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:52.426992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702134+0000","last_change":"2024-03-06T07:24:03.682451+0000","last_active":"2024-03-06T07:24:04.702134+0000","last_peered":"2024-03-06T07:24:04.702134+0000","last_clean":"2024-03-06T07:24:04.702134+0000","last_became_active":"2024-03-06T07:24:03.682163+0000","last_became_peered":"2024-03-06T07:24:03.682163+0000","last_unstale":"2024-03-06T07:24:04.702134+0000","last_undegraded":"2024-03-06T07:24:04.702134+0000","last_fullsized":"2024-03-06T07:24:04.702134+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:07.744832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.690147+0000","last_change":"2024-03-06T07:24:04.684865+0000","last_active":"2024-03-06T07:24:04.690147+0000","last_peered":"2024-03-06T07:24:04.690147+0000","last_clean":"2024-03-06T07:24:04.690147+0000","last_became_active":"2024-03-06T07:24:04.684609+0000","last_became_peered":"2024-03-06T07:24:04.684609+0000","last_unstale":"2024-03-06T07:24:04.690147+0000","last_undegraded":"2024-03-06T07:24:04.690147+0000","last_fullsized":"2024-03-06T07:24:04.690147+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:47.027547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668984+0000","last_change":"2024-03-06T07:24:03.684490+0000","last_active":"2024-03-06T07:24:04.668984+0000","last_peered":"2024-03-06T07:24:04.668984+0000","last_clean":"2024-03-06T07:24:04.668984+0000","last_became_active":"2024-03-06T07:24:03.684199+0000","last_became_peered":"2024-03-06T07:24:03.684199+0000","last_unstale":"2024-03-06T07:24:04.668984+0000","last_undegraded":"2024-03-06T07:24:04.668984+0000","last_fullsized":"2024-03-06T07:24:04.668984+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:20.317474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693316+0000","last_change":"2024-03-06T07:24:04.690146+0000","last_active":"2024-03-06T07:24:04.693316+0000","last_peered":"2024-03-06T07:24:04.693316+0000","last_clean":"2024-03-06T07:24:04.693316+0000","last_became_active":"2024-03-06T07:24:04.689894+0000","last_became_peered":"2024-03-06T07:24:04.689894+0000","last_unstale":"2024-03-06T07:24:04.693316+0000","last_undegraded":"2024-03-06T07:24:04.693316+0000","last_fullsized":"2024-03-06T07:24:04.693316+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:39.501600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702020+0000","last_change":"2024-03-06T07:24:03.678126+0000","last_active":"2024-03-06T07:24:04.702020+0000","last_peered":"2024-03-06T07:24:04.702020+0000","last_clean":"2024-03-06T07:24:04.702020+0000","last_became_active":"2024-03-06T07:24:03.677848+0000","last_became_peered":"2024-03-06T07:24:03.677848+0000","last_unstale":"2024-03-06T07:24:04.702020+0000","last_undegraded":"2024-03-06T07:24:04.702020+0000","last_fullsized":"2024-03-06T07:24:04.702020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:07:44.078772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701083+0000","last_change":"2024-03-06T07:24:04.700361+0000","last_active":"2024-03-06T07:24:04.701083+0000","last_peered":"2024-03-06T07:24:04.701083+0000","last_clean":"2024-03-06T07:24:04.701083+0000","last_became_active":"2024-03-06T07:24:04.700055+0000","last_became_peered":"2024-03-06T07:24:04.700055+0000","last_unstale":"2024-03-06T07:24:04.701083+0000","last_undegraded":"2024-03-06T07:24:04.701083+0000","last_fullsized":"2024-03-06T07:24:04.701083+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:51.331935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.669478+0000","last_change":"2024-03-06T07:24:03.682134+0000","last_active":"2024-03-06T07:24:04.669478+0000","last_peered":"2024-03-06T07:24:04.669478+0000","last_clean":"2024-03-06T07:24:04.669478+0000","last_became_active":"2024-03-06T07:24:03.681827+0000","last_became_peered":"2024-03-06T07:24:03.681827+0000","last_unstale":"2024-03-06T07:24:04.669478+0000","last_undegraded":"2024-03-06T07:24:04.669478+0000","last_fullsized":"2024-03-06T07:24:04.669478+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:28:10.765871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697397+0000","last_change":"2024-03-06T07:24:04.686857+0000","last_active":"2024-03-06T07:24:04.697397+0000","last_peered":"2024-03-06T07:24:04.697397+0000","last_clean":"2024-03-06T07:24:04.697397+0000","last_became_active":"2024-03-06T07:24:04.686354+0000","last_became_peered":"2024-03-06T07:24:04.686354+0000","last_unstale":"2024-03-06T07:24:04.697397+0000","last_undegraded":"2024-03-06T07:24:04.697397+0000","last_fullsized":"2024-03-06T07:24:04.697397+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:27.521026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.673408+0000","last_change":"2024-03-06T07:24:03.674812+0000","last_active":"2024-03-06T07:24:04.673408+0000","last_peered":"2024-03-06T07:24:04.673408+0000","last_clean":"2024-03-06T07:24:04.673408+0000","last_became_active":"2024-03-06T07:24:03.674516+0000","last_became_peered":"2024-03-06T07:24:03.674516+0000","last_unstale":"2024-03-06T07:24:04.673408+0000","last_undegraded":"2024-03-06T07:24:04.673408+0000","last_fullsized":"2024-03-06T07:24:04.673408+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:49.569232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701417+0000","last_change":"2024-03-06T07:24:04.694424+0000","last_active":"2024-03-06T07:24:04.701417+0000","last_peered":"2024-03-06T07:24:04.701417+0000","last_clean":"2024-03-06T07:24:04.701417+0000","last_became_active":"2024-03-06T07:24:04.694158+0000","last_became_peered":"2024-03-06T07:24:04.694158+0000","last_unstale":"2024-03-06T07:24:04.701417+0000","last_undegraded":"2024-03-06T07:24:04.701417+0000","last_fullsized":"2024-03-06T07:24:04.701417+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:28:11.772430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668115+0000","last_change":"2024-03-06T07:24:03.679323+0000","last_active":"2024-03-06T07:24:04.668115+0000","last_peered":"2024-03-06T07:24:04.668115+0000","last_clean":"2024-03-06T07:24:04.668115+0000","last_became_active":"2024-03-06T07:24:03.679075+0000","last_became_peered":"2024-03-06T07:24:03.679075+0000","last_unstale":"2024-03-06T07:24:04.668115+0000","last_undegraded":"2024-03-06T07:24:04.668115+0000","last_fullsized":"2024-03-06T07:24:04.668115+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:49.569232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693348+0000","last_change":"2024-03-06T07:24:04.691239+0000","last_active":"2024-03-06T07:24:04.693348+0000","last_peered":"2024-03-06T07:24:04.693348+0000","last_clean":"2024-03-06T07:24:04.693348+0000","last_became_active":"2024-03-06T07:24:04.690994+0000","last_became_peered":"2024-03-06T07:24:04.690994+0000","last_unstale":"2024-03-06T07:24:04.693348+0000","last_undegraded":"2024-03-06T07:24:04.693348+0000","last_fullsized":"2024-03-06T07:24:04.693348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:24.805015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.699388+0000","last_change":"2024-03-06T07:24:03.685336+0000","last_active":"2024-03-06T07:24:04.699388+0000","last_peered":"2024-03-06T07:24:04.699388+0000","last_clean":"2024-03-06T07:24:04.699388+0000","last_became_active":"2024-03-06T07:24:03.685016+0000","last_became_peered":"2024-03-06T07:24:03.685016+0000","last_unstale":"2024-03-06T07:24:04.699388+0000","last_undegraded":"2024-03-06T07:24:04.699388+0000","last_fullsized":"2024-03-06T07:24:04.699388+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:25:44.005879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.682571+0000","last_change":"2024-03-06T07:24:04.680868+0000","last_active":"2024-03-06T07:24:04.682571+0000","last_peered":"2024-03-06T07:24:04.682571+0000","last_clean":"2024-03-06T07:24:04.682571+0000","last_became_active":"2024-03-06T07:24:04.680592+0000","last_became_peered":"2024-03-06T07:24:04.680592+0000","last_unstale":"2024-03-06T07:24:04.682571+0000","last_undegraded":"2024-03-06T07:24:04.682571+0000","last_fullsized":"2024-03-06T07:24:04.682571+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:00.690207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.675381+0000","last_change":"2024-03-06T07:24:03.684559+0000","last_active":"2024-03-06T07:24:04.675381+0000","last_peered":"2024-03-06T07:24:04.675381+0000","last_clean":"2024-03-06T07:24:04.675381+0000","last_became_active":"2024-03-06T07:24:03.684275+0000","last_became_peered":"2024-03-06T07:24:03.684275+0000","last_unstale":"2024-03-06T07:24:04.675381+0000","last_undegraded":"2024-03-06T07:24:04.675381+0000","last_fullsized":"2024-03-06T07:24:04.675381+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:59.683648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679244+0000","last_change":"2024-03-06T07:24:04.673515+0000","last_active":"2024-03-06T07:24:04.679244+0000","last_peered":"2024-03-06T07:24:04.679244+0000","last_clean":"2024-03-06T07:24:04.679244+0000","last_became_active":"2024-03-06T07:24:04.673200+0000","last_became_peered":"2024-03-06T07:24:04.673200+0000","last_unstale":"2024-03-06T07:24:04.679244+0000","last_undegraded":"2024-03-06T07:24:04.679244+0000","last_fullsized":"2024-03-06T07:24:04.679244+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:18:00.939752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.675487+0000","last_change":"2024-03-06T07:24:03.684961+0000","last_active":"2024-03-06T07:24:04.675487+0000","last_peered":"2024-03-06T07:24:04.675487+0000","last_clean":"2024-03-06T07:24:04.675487+0000","last_became_active":"2024-03-06T07:24:03.684660+0000","last_became_peered":"2024-03-06T07:24:03.684660+0000","last_unstale":"2024-03-06T07:24:04.675487+0000","last_undegraded":"2024-03-06T07:24:04.675487+0000","last_fullsized":"2024-03-06T07:24:04.675487+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:32.993866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692108+0000","last_change":"2024-03-06T07:24:04.684383+0000","last_active":"2024-03-06T07:24:04.692108+0000","last_peered":"2024-03-06T07:24:04.692108+0000","last_clean":"2024-03-06T07:24:04.692108+0000","last_became_active":"2024-03-06T07:24:04.684136+0000","last_became_peered":"2024-03-06T07:24:04.684136+0000","last_unstale":"2024-03-06T07:24:04.692108+0000","last_undegraded":"2024-03-06T07:24:04.692108+0000","last_fullsized":"2024-03-06T07:24:04.692108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:01.614069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.681692+0000","last_change":"2024-03-06T07:24:03.682563+0000","last_active":"2024-03-06T07:24:04.681692+0000","last_peered":"2024-03-06T07:24:04.681692+0000","last_clean":"2024-03-06T07:24:04.681692+0000","last_became_active":"2024-03-06T07:24:03.682279+0000","last_became_peered":"2024-03-06T07:24:03.682279+0000","last_unstale":"2024-03-06T07:24:04.681692+0000","last_undegraded":"2024-03-06T07:24:04.681692+0000","last_fullsized":"2024-03-06T07:24:04.681692+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:02.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692264+0000","last_change":"2024-03-06T07:24:04.688131+0000","last_active":"2024-03-06T07:24:04.692264+0000","last_peered":"2024-03-06T07:24:04.692264+0000","last_clean":"2024-03-06T07:24:04.692264+0000","last_became_active":"2024-03-06T07:24:04.687816+0000","last_became_peered":"2024-03-06T07:24:04.687816+0000","last_unstale":"2024-03-06T07:24:04.692264+0000","last_undegraded":"2024-03-06T07:24:04.692264+0000","last_fullsized":"2024-03-06T07:24:04.692264+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:33.116358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.670981+0000","last_change":"2024-03-06T07:24:03.685692+0000","last_active":"2024-03-06T07:24:04.670981+0000","last_peered":"2024-03-06T07:24:04.670981+0000","last_clean":"2024-03-06T07:24:04.670981+0000","last_became_active":"2024-03-06T07:24:03.685395+0000","last_became_peered":"2024-03-06T07:24:03.685395+0000","last_unstale":"2024-03-06T07:24:04.670981+0000","last_undegraded":"2024-03-06T07:24:04.670981+0000","last_fullsized":"2024-03-06T07:24:04.670981+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:59.683648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697095+0000","last_change":"2024-03-06T07:24:04.687555+0000","last_active":"2024-03-06T07:24:04.697095+0000","last_peered":"2024-03-06T07:24:04.697095+0000","last_clean":"2024-03-06T07:24:04.697095+0000","last_became_active":"2024-03-06T07:24:04.687176+0000","last_became_peered":"2024-03-06T07:24:04.687176+0000","last_unstale":"2024-03-06T07:24:04.697095+0000","last_undegraded":"2024-03-06T07:24:04.697095+0000","last_fullsized":"2024-03-06T07:24:04.697095+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:29.990981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.670429+0000","last_change":"2024-03-06T07:24:03.685423+0000","last_active":"2024-03-06T07:24:04.670429+0000","last_peered":"2024-03-06T07:24:04.670429+0000","last_clean":"2024-03-06T07:24:04.670429+0000","last_became_active":"2024-03-06T07:24:03.685122+0000","last_became_peered":"2024-03-06T07:24:03.685122+0000","last_unstale":"2024-03-06T07:24:04.670429+0000","last_undegraded":"2024-03-06T07:24:04.670429+0000","last_fullsized":"2024-03-06T07:24:04.670429+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:44.619751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692635+0000","last_change":"2024-03-06T07:24:04.688408+0000","last_active":"2024-03-06T07:24:04.692635+0000","last_peered":"2024-03-06T07:24:04.692635+0000","last_clean":"2024-03-06T07:24:04.692635+0000","last_became_active":"2024-03-06T07:24:04.688128+0000","last_became_peered":"2024-03-06T07:24:04.688128+0000","last_unstale":"2024-03-06T07:24:04.692635+0000","last_undegraded":"2024-03-06T07:24:04.692635+0000","last_fullsized":"2024-03-06T07:24:04.692635+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.153991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.681468+0000","last_change":"2024-03-06T07:24:03.682581+0000","last_active":"2024-03-06T07:24:04.681468+0000","last_peered":"2024-03-06T07:24:04.681468+0000","last_clean":"2024-03-06T07:24:04.681468+0000","last_became_active":"2024-03-06T07:24:03.682290+0000","last_became_peered":"2024-03-06T07:24:03.682290+0000","last_unstale":"2024-03-06T07:24:04.681468+0000","last_undegraded":"2024-03-06T07:24:04.681468+0000","last_fullsized":"2024-03-06T07:24:04.681468+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:41.359866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.680570+0000","last_change":"2024-03-06T07:24:04.680027+0000","last_active":"2024-03-06T07:24:04.680570+0000","last_peered":"2024-03-06T07:24:04.680570+0000","last_clean":"2024-03-06T07:24:04.680570+0000","last_became_active":"2024-03-06T07:24:04.679762+0000","last_became_peered":"2024-03-06T07:24:04.679762+0000","last_unstale":"2024-03-06T07:24:04.680570+0000","last_undegraded":"2024-03-06T07:24:04.680570+0000","last_fullsized":"2024-03-06T07:24:04.680570+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:45.626310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668284+0000","last_change":"2024-03-06T07:24:03.679899+0000","last_active":"2024-03-06T07:24:04.668284+0000","last_peered":"2024-03-06T07:24:04.668284+0000","last_clean":"2024-03-06T07:24:04.668284+0000","last_became_active":"2024-03-06T07:24:03.679646+0000","last_became_peered":"2024-03-06T07:24:03.679646+0000","last_unstale":"2024-03-06T07:24:04.668284+0000","last_undegraded":"2024-03-06T07:24:04.668284+0000","last_fullsized":"2024-03-06T07:24:04.668284+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:22.238147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.689933+0000","last_change":"2024-03-06T07:24:04.686167+0000","last_active":"2024-03-06T07:24:04.689933+0000","last_peered":"2024-03-06T07:24:04.689933+0000","last_clean":"2024-03-06T07:24:04.689933+0000","last_became_active":"2024-03-06T07:24:04.685535+0000","last_became_peered":"2024-03-06T07:24:04.685535+0000","last_unstale":"2024-03-06T07:24:04.689933+0000","last_undegraded":"2024-03-06T07:24:04.689933+0000","last_fullsized":"2024-03-06T07:24:04.689933+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:52.255797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"107'8","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702017+0000","last_change":"2024-03-06T07:24:03.680038+0000","last_active":"2024-03-06T07:24:04.702017+0000","last_peered":"2024-03-06T07:24:04.702017+0000","last_clean":"2024-03-06T07:24:04.702017+0000","last_became_active":"2024-03-06T07:24:03.679755+0000","last_became_peered":"2024-03-06T07:24:03.679755+0000","last_unstale":"2024-03-06T07:24:04.702017+0000","last_undegraded":"2024-03-06T07:24:04.702017+0000","last_fullsized":"2024-03-06T07:24:04.702017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:22.238147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668389+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:24:04.668389+0000","last_peered":"2024-03-06T07:24:04.668389+0000","last_clean":"2024-03-06T07:24:04.668389+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:24:04.668389+0000","last_undegraded":"2024-03-06T07:24:04.668389+0000","last_fullsized":"2024-03-06T07:24:04.668389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1388544,"data_stored":1226434,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":644845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1388544,"data_stored":1226434,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:24:07.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:24:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.813+0000 7fd145c81640 1 Processor -- start 2024-03-06T07:24:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.813+0000 7fd145c81640 1 -- start start 2024-03-06T07:24:07.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd145c81640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd1401506b0 0x7fd140153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd145c81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 0x7fd1400702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd145c81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd140070800 con 0x7fd1400acc00 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd145c81640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd140156450 con 0x7fd1401506b0 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd145c81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd140156a10 con 0x7fd140156ea0 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd13ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd1400acc00 0x7fd1400c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37138/0 (socket says 172.21.15.26:37138) 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd13ffff640 1 -- 172.21.15.26:0/1352604541 learned_addr learned my addr 172.21.15.26:0/1352604541 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812369708 0 0) 0x7fd140070800 con 0x7fd1400acc00 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd12c003520 con 0x7fd1400acc00 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd13effd640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 0x7fd1400702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.814+0000 7fd13f7fe640 1 --2- 172.21.15.26:0/1352604541 >> v2:172.21.15.117:3300/0 conn(0x7fd1401506b0 0x7fd140153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3420792799 0 0) 0x7fd12c003520 con 0x7fd1400acc00 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 msgr2=0x7fd1400702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 0x7fd1400702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 >> v2:172.21.15.117:3300/0 conn(0x7fd1401506b0 msgr2=0x7fd140153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 --2- 172.21.15.26:0/1352604541 >> v2:172.21.15.117:3300/0 conn(0x7fd1401506b0 0x7fd140153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd140070aa0 con 0x7fd1400acc00 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd1380019f0 con 0x7fd1400acc00 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 >> v1:172.21.15.26:6789/0 conn(0x7fd1400acc00 legacy=0x7fd1400c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 0x7fd12c0039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c003f50 0x7fd12c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c004d40 0x7fd12c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140156450 con 0x7fd12c003f50 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140156a10 con 0x7fd12c004d40 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c00b9c0 con 0x7fd140156ea0 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13effd640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c003f50 0x7fd12c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13f7fe640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 0x7fd12c0039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13ffff640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c004d40 0x7fd12c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13effd640 1 -- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 msgr2=0x7fd12c0039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13effd640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd140156ea0 0x7fd12c0039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13effd640 1 -- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c004d40 msgr2=0x7fd12c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13effd640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c004d40 0x7fd12c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.815+0000 7fd13effd640 1 -- 172.21.15.26:0/1352604541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c003520 con 0x7fd12c003f50 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.816+0000 7fd13effd640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c003f50 0x7fd12c004800 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd130009af0 tx=0x7fd12c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94491594432d9e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.816+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130039070 con 0x7fd12c003f50 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.816+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13002c6b0 con 0x7fd12c003f50 2024-03-06T07:24:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.816+0000 7fd13e7fc640 1 -- 172.21.15.26:0/1352604541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130009690 con 0x7fd12c003f50 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.817+0000 7fd145c81640 1 -- 172.21.15.26:0/1352604541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c003f50 msgr2=0x7fd12c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.817+0000 7fd145c81640 1 --2- 172.21.15.26:0/1352604541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c003f50 0x7fd12c004800 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd130009af0 tx=0x7fd12c005830 comp rx=0 tx=0).stop 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.817+0000 7fd13ffff640 1 -- 172.21.15.26:0/1352604541 reap_dead start 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.817+0000 7fd145c81640 1 -- 172.21.15.26:0/1352604541 shutdown_connections 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.817+0000 7fd145c81640 1 -- 172.21.15.26:0/1352604541 >> 172.21.15.26:0/1352604541 conn(0x7fd14006b1a0 msgr2=0x7fd14006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.817+0000 7fd145c81640 1 -- 172.21.15.26:0/1352604541 shutdown_connections 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.817+0000 7fd145c81640 1 -- 172.21.15.26:0/1352604541 wait complete. 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.818+0000 7fd145c81640 1 Processor -- start 2024-03-06T07:24:07.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.818+0000 7fd145c81640 1 -- start start 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd145c81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c003f50 0x7fd1401e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd145c81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd140155f80 0x7fd140147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd145c81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd14014cbf0 0x7fd140147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd145c81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400c5bb0 con 0x7fd140155f80 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd145c81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1401566a0 con 0x7fd12c003f50 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd145c81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400c5d30 con 0x7fd14014cbf0 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd140155f80 0x7fd140147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd140155f80 0x7fd140147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45218/0 (socket says 172.21.15.26:45218) 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 -- 172.21.15.26:0/2586262093 learned_addr learned my addr 172.21.15.26:0/2586262093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13f7fe640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c003f50 0x7fd1401e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13ffff640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd14014cbf0 0x7fd140147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 -- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd14014cbf0 msgr2=0x7fd140147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd14014cbf0 0x7fd140147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 -- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c003f50 msgr2=0x7fd1401e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c003f50 0x7fd1401e8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 -- 172.21.15.26:0/2586262093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400c4f10 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13f7fe640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c003f50 0x7fd1401e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd13effd640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd140155f80 0x7fd140147020 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd130000c00 tx=0x7fd130004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd137fff640 1 -- 172.21.15.26:0/2586262093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130042020 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd137fff640 1 -- 172.21.15.26:0/2586262093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13003a040 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1401e97a0 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.819+0000 7fd137fff640 1 -- 172.21.15.26:0/2586262093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130039070 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.820+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1401559c0 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.821+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd140069460 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.821+0000 7fd137fff640 1 -- 172.21.15.26:0/2586262093 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd130045020 con 0x7fd140155f80 2024-03-06T07:24:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.821+0000 7fd137fff640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd12006b500 0x7fd12006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.821+0000 7fd137fff640 1 -- 172.21.15.26:0/2586262093 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1300ab640 con 0x7fd140155f80 2024-03-06T07:24:07.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.822+0000 7fd13f7fe640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd12006b500 0x7fd12006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.822+0000 7fd13f7fe640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd12006b500 0x7fd12006d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd128009c40 tx=0x7fd128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:07.826+0000 7fd137fff640 1 -- 172.21.15.26:0/2586262093 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd140069460 con 0x7fd140155f80 2024-03-06T07:24:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.004+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd140155530 con 0x7fd12006b500 2024-03-06T07:24:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.010+0000 7fd137fff640 1 -- 172.21.15.26:0/2586262093 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252494 (secure 0 0 0) 0x7fd140155530 con 0x7fd12006b500 2024-03-06T07:24:08.015 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:24:08.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd12006b500 msgr2=0x7fd12006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd12006b500 0x7fd12006d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd128009c40 tx=0x7fd128008040 comp rx=0 tx=0).stop 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd140155f80 msgr2=0x7fd140147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd140155f80 0x7fd140147020 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd130000c00 tx=0x7fd130004730 comp rx=0 tx=0).stop 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 shutdown_connections 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd12006b500 0x7fd12006d9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd14014cbf0 0x7fd140147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd140155f80 0x7fd140147020 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 --2- 172.21.15.26:0/2586262093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd12c003f50 0x7fd1401e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 >> 172.21.15.26:0/2586262093 conn(0x7fd14006b1a0 msgr2=0x7fd140152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 shutdown_connections 2024-03-06T07:24:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.013+0000 7fd145c81640 1 -- 172.21.15.26:0/2586262093 wait complete. 2024-03-06T07:24:08.026 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-03-06T07:24:07.080374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259924,"kb_used_data":10488,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714796,"statfs":{"total":773094113280,"available":772827951104,"internally_reserved":0,"allocated":10739712,"data_stored":6278043,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.420933"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679432+0000","last_change":"2024-03-06T07:24:04.678785+0000","last_active":"2024-03-06T07:24:04.679432+0000","last_peered":"2024-03-06T07:24:04.679432+0000","last_clean":"2024-03-06T07:24:04.679432+0000","last_became_active":"2024-03-06T07:24:04.678526+0000","last_became_peered":"2024-03-06T07:24:04.678526+0000","last_unstale":"2024-03-06T07:24:04.679432+0000","last_undegraded":"2024-03-06T07:24:04.679432+0000","last_fullsized":"2024-03-06T07:24:04.679432+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:15.486735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.688817+0000","last_change":"2024-03-06T07:24:04.682976+0000","last_active":"2024-03-06T07:24:04.688817+0000","last_peered":"2024-03-06T07:24:04.688817+0000","last_clean":"2024-03-06T07:24:04.688817+0000","last_became_active":"2024-03-06T07:24:04.681993+0000","last_became_peered":"2024-03-06T07:24:04.681993+0000","last_unstale":"2024-03-06T07:24:04.688817+0000","last_undegraded":"2024-03-06T07:24:04.688817+0000","last_fullsized":"2024-03-06T07:24:04.688817+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:28:11.772430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696199+0000","last_change":"2024-03-06T07:24:04.685766+0000","last_active":"2024-03-06T07:24:04.696199+0000","last_peered":"2024-03-06T07:24:04.696199+0000","last_clean":"2024-03-06T07:24:04.696199+0000","last_became_active":"2024-03-06T07:24:04.685499+0000","last_became_peered":"2024-03-06T07:24:04.685499+0000","last_unstale":"2024-03-06T07:24:04.696199+0000","last_undegraded":"2024-03-06T07:24:04.696199+0000","last_fullsized":"2024-03-06T07:24:04.696199+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:18:00.939752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679211+0000","last_change":"2024-03-06T07:24:04.673507+0000","last_active":"2024-03-06T07:24:04.679211+0000","last_peered":"2024-03-06T07:24:04.679211+0000","last_clean":"2024-03-06T07:24:04.679211+0000","last_became_active":"2024-03-06T07:24:04.673185+0000","last_became_peered":"2024-03-06T07:24:04.673185+0000","last_unstale":"2024-03-06T07:24:04.679211+0000","last_undegraded":"2024-03-06T07:24:04.679211+0000","last_fullsized":"2024-03-06T07:24:04.679211+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:38.089819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.683774+0000","last_change":"2024-03-06T07:24:04.681818+0000","last_active":"2024-03-06T07:24:04.683774+0000","last_peered":"2024-03-06T07:24:04.683774+0000","last_clean":"2024-03-06T07:24:04.683774+0000","last_became_active":"2024-03-06T07:24:04.681461+0000","last_became_peered":"2024-03-06T07:24:04.681461+0000","last_unstale":"2024-03-06T07:24:04.683774+0000","last_undegraded":"2024-03-06T07:24:04.683774+0000","last_fullsized":"2024-03-06T07:24:04.683774+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:47.901117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693784+0000","last_change":"2024-03-06T07:24:04.692244+0000","last_active":"2024-03-06T07:24:04.693784+0000","last_peered":"2024-03-06T07:24:04.693784+0000","last_clean":"2024-03-06T07:24:04.693784+0000","last_became_active":"2024-03-06T07:24:04.691975+0000","last_became_peered":"2024-03-06T07:24:04.691975+0000","last_unstale":"2024-03-06T07:24:04.693784+0000","last_undegraded":"2024-03-06T07:24:04.693784+0000","last_fullsized":"2024-03-06T07:24:04.693784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:47.105582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692404+0000","last_change":"2024-03-06T07:24:04.687610+0000","last_active":"2024-03-06T07:24:04.692404+0000","last_peered":"2024-03-06T07:24:04.692404+0000","last_clean":"2024-03-06T07:24:04.692404+0000","last_became_active":"2024-03-06T07:24:04.687338+0000","last_became_peered":"2024-03-06T07:24:04.687338+0000","last_unstale":"2024-03-06T07:24:04.692404+0000","last_undegraded":"2024-03-06T07:24:04.692404+0000","last_fullsized":"2024-03-06T07:24:04.692404+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:06.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696898+0000","last_change":"2024-03-06T07:24:04.690717+0000","last_active":"2024-03-06T07:24:04.696898+0000","last_peered":"2024-03-06T07:24:04.696898+0000","last_clean":"2024-03-06T07:24:04.696898+0000","last_became_active":"2024-03-06T07:24:04.689237+0000","last_became_peered":"2024-03-06T07:24:04.689237+0000","last_unstale":"2024-03-06T07:24:04.696898+0000","last_undegraded":"2024-03-06T07:24:04.696898+0000","last_fullsized":"2024-03-06T07:24:04.696898+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.153991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693666+0000","last_change":"2024-03-06T07:24:04.691303+0000","last_active":"2024-03-06T07:24:04.693666+0000","last_peered":"2024-03-06T07:24:04.693666+0000","last_clean":"2024-03-06T07:24:04.693666+0000","last_became_active":"2024-03-06T07:24:04.691018+0000","last_became_peered":"2024-03-06T07:24:04.691018+0000","last_unstale":"2024-03-06T07:24:04.693666+0000","last_undegraded":"2024-03-06T07:24:04.693666+0000","last_fullsized":"2024-03-06T07:24:04.693666+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:28.076328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696379+0000","last_change":"2024-03-06T07:24:04.692966+0000","last_active":"2024-03-06T07:24:04.696379+0000","last_peered":"2024-03-06T07:24:04.696379+0000","last_clean":"2024-03-06T07:24:04.696379+0000","last_became_active":"2024-03-06T07:24:04.692652+0000","last_became_peered":"2024-03-06T07:24:04.692652+0000","last_unstale":"2024-03-06T07:24:04.696379+0000","last_undegraded":"2024-03-06T07:24:04.696379+0000","last_fullsized":"2024-03-06T07:24:04.696379+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:17.366449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.686836+0000","last_change":"2024-03-06T07:24:04.682802+0000","last_active":"2024-03-06T07:24:04.686836+0000","last_peered":"2024-03-06T07:24:04.686836+0000","last_clean":"2024-03-06T07:24:04.686836+0000","last_became_active":"2024-03-06T07:24:04.682532+0000","last_became_peered":"2024-03-06T07:24:04.682532+0000","last_unstale":"2024-03-06T07:24:04.686836+0000","last_undegraded":"2024-03-06T07:24:04.686836+0000","last_fullsized":"2024-03-06T07:24:04.686836+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:16.207859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692506+0000","last_change":"2024-03-06T07:24:04.688276+0000","last_active":"2024-03-06T07:24:04.692506+0000","last_peered":"2024-03-06T07:24:04.692506+0000","last_clean":"2024-03-06T07:24:04.692506+0000","last_became_active":"2024-03-06T07:24:04.687952+0000","last_became_peered":"2024-03-06T07:24:04.687952+0000","last_unstale":"2024-03-06T07:24:04.692506+0000","last_undegraded":"2024-03-06T07:24:04.692506+0000","last_fullsized":"2024-03-06T07:24:04.692506+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:41.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.684907+0000","last_change":"2024-03-06T07:24:04.682372+0000","last_active":"2024-03-06T07:24:04.684907+0000","last_peered":"2024-03-06T07:24:04.684907+0000","last_clean":"2024-03-06T07:24:04.684907+0000","last_became_active":"2024-03-06T07:24:04.682088+0000","last_became_peered":"2024-03-06T07:24:04.682088+0000","last_unstale":"2024-03-06T07:24:04.684907+0000","last_undegraded":"2024-03-06T07:24:04.684907+0000","last_fullsized":"2024-03-06T07:24:04.684907+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:15.486735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697382+0000","last_change":"2024-03-06T07:24:04.686629+0000","last_active":"2024-03-06T07:24:04.697382+0000","last_peered":"2024-03-06T07:24:04.697382+0000","last_clean":"2024-03-06T07:24:04.697382+0000","last_became_active":"2024-03-06T07:24:04.685296+0000","last_became_peered":"2024-03-06T07:24:04.685296+0000","last_unstale":"2024-03-06T07:24:04.697382+0000","last_undegraded":"2024-03-06T07:24:04.697382+0000","last_fullsized":"2024-03-06T07:24:04.697382+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:49.098245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692631+0000","last_change":"2024-03-06T07:24:04.682748+0000","last_active":"2024-03-06T07:24:04.692631+0000","last_peered":"2024-03-06T07:24:04.692631+0000","last_clean":"2024-03-06T07:24:04.692631+0000","last_became_active":"2024-03-06T07:24:04.682388+0000","last_became_peered":"2024-03-06T07:24:04.682388+0000","last_unstale":"2024-03-06T07:24:04.692631+0000","last_undegraded":"2024-03-06T07:24:04.692631+0000","last_fullsized":"2024-03-06T07:24:04.692631+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:23.035754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692278+0000","last_change":"2024-03-06T07:24:04.688219+0000","last_active":"2024-03-06T07:24:04.692278+0000","last_peered":"2024-03-06T07:24:04.692278+0000","last_clean":"2024-03-06T07:24:04.692278+0000","last_became_active":"2024-03-06T07:24:04.687940+0000","last_became_peered":"2024-03-06T07:24:04.687940+0000","last_unstale":"2024-03-06T07:24:04.692278+0000","last_undegraded":"2024-03-06T07:24:04.692278+0000","last_fullsized":"2024-03-06T07:24:04.692278+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:40.100249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.684984+0000","last_change":"2024-03-06T07:24:04.681730+0000","last_active":"2024-03-06T07:24:04.684984+0000","last_peered":"2024-03-06T07:24:04.684984+0000","last_clean":"2024-03-06T07:24:04.684984+0000","last_became_active":"2024-03-06T07:24:04.681445+0000","last_became_peered":"2024-03-06T07:24:04.681445+0000","last_unstale":"2024-03-06T07:24:04.684984+0000","last_undegraded":"2024-03-06T07:24:04.684984+0000","last_fullsized":"2024-03-06T07:24:04.684984+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:34.000425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695395+0000","last_change":"2024-03-06T07:24:04.683306+0000","last_active":"2024-03-06T07:24:04.695395+0000","last_peered":"2024-03-06T07:24:04.695395+0000","last_clean":"2024-03-06T07:24:04.695395+0000","last_became_active":"2024-03-06T07:24:04.682772+0000","last_became_peered":"2024-03-06T07:24:04.682772+0000","last_unstale":"2024-03-06T07:24:04.695395+0000","last_undegraded":"2024-03-06T07:24:04.695395+0000","last_fullsized":"2024-03-06T07:24:04.695395+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:38.089819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692790+0000","last_change":"2024-03-06T07:24:04.685167+0000","last_active":"2024-03-06T07:24:04.692790+0000","last_peered":"2024-03-06T07:24:04.692790+0000","last_clean":"2024-03-06T07:24:04.692790+0000","last_became_active":"2024-03-06T07:24:04.684529+0000","last_became_peered":"2024-03-06T07:24:04.684529+0000","last_unstale":"2024-03-06T07:24:04.692790+0000","last_undegraded":"2024-03-06T07:24:04.692790+0000","last_fullsized":"2024-03-06T07:24:04.692790+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:23.035754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696164+0000","last_change":"2024-03-06T07:24:04.693998+0000","last_active":"2024-03-06T07:24:04.696164+0000","last_peered":"2024-03-06T07:24:04.696164+0000","last_clean":"2024-03-06T07:24:04.696164+0000","last_became_active":"2024-03-06T07:24:04.693707+0000","last_became_peered":"2024-03-06T07:24:04.693707+0000","last_unstale":"2024-03-06T07:24:04.696164+0000","last_undegraded":"2024-03-06T07:24:04.696164+0000","last_fullsized":"2024-03-06T07:24:04.696164+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:33.519123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.689911+0000","last_change":"2024-03-06T07:24:04.686080+0000","last_active":"2024-03-06T07:24:04.689911+0000","last_peered":"2024-03-06T07:24:04.689911+0000","last_clean":"2024-03-06T07:24:04.689911+0000","last_became_active":"2024-03-06T07:24:04.685674+0000","last_became_peered":"2024-03-06T07:24:04.685674+0000","last_unstale":"2024-03-06T07:24:04.689911+0000","last_undegraded":"2024-03-06T07:24:04.689911+0000","last_fullsized":"2024-03-06T07:24:04.689911+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:29.990981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693518+0000","last_change":"2024-03-06T07:24:04.689579+0000","last_active":"2024-03-06T07:24:04.693518+0000","last_peered":"2024-03-06T07:24:04.693518+0000","last_clean":"2024-03-06T07:24:04.693518+0000","last_became_active":"2024-03-06T07:24:04.688900+0000","last_became_peered":"2024-03-06T07:24:04.688900+0000","last_unstale":"2024-03-06T07:24:04.693518+0000","last_undegraded":"2024-03-06T07:24:04.693518+0000","last_fullsized":"2024-03-06T07:24:04.693518+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:38.196538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.684023+0000","last_change":"2024-03-06T07:24:04.681464+0000","last_active":"2024-03-06T07:24:04.684023+0000","last_peered":"2024-03-06T07:24:04.684023+0000","last_clean":"2024-03-06T07:24:04.684023+0000","last_became_active":"2024-03-06T07:24:04.681199+0000","last_became_peered":"2024-03-06T07:24:04.681199+0000","last_unstale":"2024-03-06T07:24:04.684023+0000","last_undegraded":"2024-03-06T07:24:04.684023+0000","last_fullsized":"2024-03-06T07:24:04.684023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:27.521026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692460+0000","last_change":"2024-03-06T07:24:04.688245+0000","last_active":"2024-03-06T07:24:04.692460+0000","last_peered":"2024-03-06T07:24:04.692460+0000","last_clean":"2024-03-06T07:24:04.692460+0000","last_became_active":"2024-03-06T07:24:04.687929+0000","last_became_peered":"2024-03-06T07:24:04.687929+0000","last_unstale":"2024-03-06T07:24:04.692460+0000","last_undegraded":"2024-03-06T07:24:04.692460+0000","last_fullsized":"2024-03-06T07:24:04.692460+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:23.244706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692479+0000","last_change":"2024-03-06T07:24:04.687412+0000","last_active":"2024-03-06T07:24:04.692479+0000","last_peered":"2024-03-06T07:24:04.692479+0000","last_clean":"2024-03-06T07:24:04.692479+0000","last_became_active":"2024-03-06T07:24:04.687140+0000","last_became_peered":"2024-03-06T07:24:04.687140+0000","last_unstale":"2024-03-06T07:24:04.692479+0000","last_undegraded":"2024-03-06T07:24:04.692479+0000","last_fullsized":"2024-03-06T07:24:04.692479+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:50.575791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.682499+0000","last_change":"2024-03-06T07:24:04.680986+0000","last_active":"2024-03-06T07:24:04.682499+0000","last_peered":"2024-03-06T07:24:04.682499+0000","last_clean":"2024-03-06T07:24:04.682499+0000","last_became_active":"2024-03-06T07:24:04.680719+0000","last_became_peered":"2024-03-06T07:24:04.680719+0000","last_unstale":"2024-03-06T07:24:04.682499+0000","last_undegraded":"2024-03-06T07:24:04.682499+0000","last_fullsized":"2024-03-06T07:24:04.682499+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:47.540962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.681805+0000","last_change":"2024-03-06T07:24:04.681044+0000","last_active":"2024-03-06T07:24:04.681805+0000","last_peered":"2024-03-06T07:24:04.681805+0000","last_clean":"2024-03-06T07:24:04.681805+0000","last_became_active":"2024-03-06T07:24:04.680754+0000","last_became_peered":"2024-03-06T07:24:04.680754+0000","last_unstale":"2024-03-06T07:24:04.681805+0000","last_undegraded":"2024-03-06T07:24:04.681805+0000","last_fullsized":"2024-03-06T07:24:04.681805+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:15.486735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.687718+0000","last_change":"2024-03-06T07:24:04.683726+0000","last_active":"2024-03-06T07:24:04.687718+0000","last_peered":"2024-03-06T07:24:04.687718+0000","last_clean":"2024-03-06T07:24:04.687718+0000","last_became_active":"2024-03-06T07:24:04.683455+0000","last_became_peered":"2024-03-06T07:24:04.683455+0000","last_unstale":"2024-03-06T07:24:04.687718+0000","last_undegraded":"2024-03-06T07:24:04.687718+0000","last_fullsized":"2024-03-06T07:24:04.687718+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:08.698708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695305+0000","last_change":"2024-03-06T07:24:04.692695+0000","last_active":"2024-03-06T07:24:04.695305+0000","last_peered":"2024-03-06T07:24:04.695305+0000","last_clean":"2024-03-06T07:24:04.695305+0000","last_became_active":"2024-03-06T07:24:04.692417+0000","last_became_peered":"2024-03-06T07:24:04.692417+0000","last_unstale":"2024-03-06T07:24:04.695305+0000","last_undegraded":"2024-03-06T07:24:04.695305+0000","last_fullsized":"2024-03-06T07:24:04.695305+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:20.367676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697409+0000","last_change":"2024-03-06T07:24:04.690731+0000","last_active":"2024-03-06T07:24:04.697409+0000","last_peered":"2024-03-06T07:24:04.697409+0000","last_clean":"2024-03-06T07:24:04.697409+0000","last_became_active":"2024-03-06T07:24:04.689308+0000","last_became_peered":"2024-03-06T07:24:04.689308+0000","last_unstale":"2024-03-06T07:24:04.697409+0000","last_undegraded":"2024-03-06T07:24:04.697409+0000","last_fullsized":"2024-03-06T07:24:04.697409+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:28.076328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701322+0000","last_change":"2024-03-06T07:24:04.693519+0000","last_active":"2024-03-06T07:24:04.701322+0000","last_peered":"2024-03-06T07:24:04.701322+0000","last_clean":"2024-03-06T07:24:04.701322+0000","last_became_active":"2024-03-06T07:24:04.693250+0000","last_became_peered":"2024-03-06T07:24:04.693250+0000","last_unstale":"2024-03-06T07:24:04.701322+0000","last_undegraded":"2024-03-06T07:24:04.701322+0000","last_fullsized":"2024-03-06T07:24:04.701322+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:52.798080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.690186+0000","last_change":"2024-03-06T07:24:04.685356+0000","last_active":"2024-03-06T07:24:04.690186+0000","last_peered":"2024-03-06T07:24:04.690186+0000","last_clean":"2024-03-06T07:24:04.690186+0000","last_became_active":"2024-03-06T07:24:04.685066+0000","last_became_peered":"2024-03-06T07:24:04.685066+0000","last_unstale":"2024-03-06T07:24:04.690186+0000","last_undegraded":"2024-03-06T07:24:04.690186+0000","last_fullsized":"2024-03-06T07:24:04.690186+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:30.138870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696417+0000","last_change":"2024-03-06T07:24:04.694303+0000","last_active":"2024-03-06T07:24:04.696417+0000","last_peered":"2024-03-06T07:24:04.696417+0000","last_clean":"2024-03-06T07:24:04.696417+0000","last_became_active":"2024-03-06T07:24:04.693789+0000","last_became_peered":"2024-03-06T07:24:04.693789+0000","last_unstale":"2024-03-06T07:24:04.696417+0000","last_undegraded":"2024-03-06T07:24:04.696417+0000","last_fullsized":"2024-03-06T07:24:04.696417+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:38.196538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702091+0000","last_change":"2024-03-06T07:24:03.682632+0000","last_active":"2024-03-06T07:24:04.702091+0000","last_peered":"2024-03-06T07:24:04.702091+0000","last_clean":"2024-03-06T07:24:04.702091+0000","last_became_active":"2024-03-06T07:24:03.682342+0000","last_became_peered":"2024-03-06T07:24:03.682342+0000","last_unstale":"2024-03-06T07:24:04.702091+0000","last_undegraded":"2024-03-06T07:24:04.702091+0000","last_fullsized":"2024-03-06T07:24:04.702091+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:16.284868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.699202+0000","last_change":"2024-03-06T07:24:04.698155+0000","last_active":"2024-03-06T07:24:04.699202+0000","last_peered":"2024-03-06T07:24:04.699202+0000","last_clean":"2024-03-06T07:24:04.699202+0000","last_became_active":"2024-03-06T07:24:04.697879+0000","last_became_peered":"2024-03-06T07:24:04.697879+0000","last_unstale":"2024-03-06T07:24:04.699202+0000","last_undegraded":"2024-03-06T07:24:04.699202+0000","last_fullsized":"2024-03-06T07:24:04.699202+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:57.714898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695489+0000","last_change":"2024-03-06T07:24:03.680197+0000","last_active":"2024-03-06T07:24:04.695489+0000","last_peered":"2024-03-06T07:24:04.695489+0000","last_clean":"2024-03-06T07:24:04.695489+0000","last_became_active":"2024-03-06T07:24:03.679925+0000","last_became_peered":"2024-03-06T07:24:03.679925+0000","last_unstale":"2024-03-06T07:24:04.695489+0000","last_undegraded":"2024-03-06T07:24:04.695489+0000","last_fullsized":"2024-03-06T07:24:04.695489+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:16.284868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.690228+0000","last_change":"2024-03-06T07:24:04.686651+0000","last_active":"2024-03-06T07:24:04.690228+0000","last_peered":"2024-03-06T07:24:04.690228+0000","last_clean":"2024-03-06T07:24:04.690228+0000","last_became_active":"2024-03-06T07:24:04.686361+0000","last_became_peered":"2024-03-06T07:24:04.686361+0000","last_unstale":"2024-03-06T07:24:04.690228+0000","last_undegraded":"2024-03-06T07:24:04.690228+0000","last_fullsized":"2024-03-06T07:24:04.690228+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:47.755350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701969+0000","last_change":"2024-03-06T07:24:03.679292+0000","last_active":"2024-03-06T07:24:04.701969+0000","last_peered":"2024-03-06T07:24:04.701969+0000","last_clean":"2024-03-06T07:24:04.701969+0000","last_became_active":"2024-03-06T07:24:03.678728+0000","last_became_peered":"2024-03-06T07:24:03.678728+0000","last_unstale":"2024-03-06T07:24:04.701969+0000","last_undegraded":"2024-03-06T07:24:04.701969+0000","last_fullsized":"2024-03-06T07:24:04.701969+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:51.791521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679307+0000","last_change":"2024-03-06T07:24:04.676037+0000","last_active":"2024-03-06T07:24:04.679307+0000","last_peered":"2024-03-06T07:24:04.679307+0000","last_clean":"2024-03-06T07:24:04.679307+0000","last_became_active":"2024-03-06T07:24:04.675629+0000","last_became_peered":"2024-03-06T07:24:04.675629+0000","last_unstale":"2024-03-06T07:24:04.679307+0000","last_undegraded":"2024-03-06T07:24:04.679307+0000","last_fullsized":"2024-03-06T07:24:04.679307+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:49.098245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697951+0000","last_change":"2024-03-06T07:24:03.682239+0000","last_active":"2024-03-06T07:24:04.697951+0000","last_peered":"2024-03-06T07:24:04.697951+0000","last_clean":"2024-03-06T07:24:04.697951+0000","last_became_active":"2024-03-06T07:24:03.681976+0000","last_became_peered":"2024-03-06T07:24:03.681976+0000","last_unstale":"2024-03-06T07:24:04.697951+0000","last_undegraded":"2024-03-06T07:24:04.697951+0000","last_fullsized":"2024-03-06T07:24:04.697951+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:00.737487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.685602+0000","last_change":"2024-03-06T07:24:04.683269+0000","last_active":"2024-03-06T07:24:04.685602+0000","last_peered":"2024-03-06T07:24:04.685602+0000","last_clean":"2024-03-06T07:24:04.685602+0000","last_became_active":"2024-03-06T07:24:04.682989+0000","last_became_peered":"2024-03-06T07:24:04.682989+0000","last_unstale":"2024-03-06T07:24:04.685602+0000","last_undegraded":"2024-03-06T07:24:04.685602+0000","last_fullsized":"2024-03-06T07:24:04.685602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:47.901117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696687+0000","last_change":"2024-03-06T07:24:03.686816+0000","last_active":"2024-03-06T07:24:04.696687+0000","last_peered":"2024-03-06T07:24:04.696687+0000","last_clean":"2024-03-06T07:24:04.696687+0000","last_became_active":"2024-03-06T07:24:03.686569+0000","last_became_peered":"2024-03-06T07:24:03.686569+0000","last_unstale":"2024-03-06T07:24:04.696687+0000","last_undegraded":"2024-03-06T07:24:04.696687+0000","last_fullsized":"2024-03-06T07:24:04.696687+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:32.993866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692655+0000","last_change":"2024-03-06T07:24:04.684168+0000","last_active":"2024-03-06T07:24:04.692655+0000","last_peered":"2024-03-06T07:24:04.692655+0000","last_clean":"2024-03-06T07:24:04.692655+0000","last_became_active":"2024-03-06T07:24:04.683857+0000","last_became_peered":"2024-03-06T07:24:04.683857+0000","last_unstale":"2024-03-06T07:24:04.692655+0000","last_undegraded":"2024-03-06T07:24:04.692655+0000","last_fullsized":"2024-03-06T07:24:04.692655+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:17.366449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.700688+0000","last_change":"2024-03-06T07:24:03.681122+0000","last_active":"2024-03-06T07:24:04.700688+0000","last_peered":"2024-03-06T07:24:04.700688+0000","last_clean":"2024-03-06T07:24:04.700688+0000","last_became_active":"2024-03-06T07:24:03.680825+0000","last_became_peered":"2024-03-06T07:24:03.680825+0000","last_unstale":"2024-03-06T07:24:04.700688+0000","last_undegraded":"2024-03-06T07:24:04.700688+0000","last_fullsized":"2024-03-06T07:24:04.700688+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:20.143531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.688324+0000","last_change":"2024-03-06T07:24:04.681126+0000","last_active":"2024-03-06T07:24:04.688324+0000","last_peered":"2024-03-06T07:24:04.688324+0000","last_clean":"2024-03-06T07:24:04.688324+0000","last_became_active":"2024-03-06T07:24:04.680864+0000","last_became_peered":"2024-03-06T07:24:04.680864+0000","last_unstale":"2024-03-06T07:24:04.688324+0000","last_undegraded":"2024-03-06T07:24:04.688324+0000","last_fullsized":"2024-03-06T07:24:04.688324+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:08.751391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.676515+0000","last_change":"2024-03-06T07:24:03.679003+0000","last_active":"2024-03-06T07:24:04.676515+0000","last_peered":"2024-03-06T07:24:04.676515+0000","last_clean":"2024-03-06T07:24:04.676515+0000","last_became_active":"2024-03-06T07:24:03.678718+0000","last_became_peered":"2024-03-06T07:24:03.678718+0000","last_unstale":"2024-03-06T07:24:04.676515+0000","last_undegraded":"2024-03-06T07:24:04.676515+0000","last_fullsized":"2024-03-06T07:24:04.676515+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:17:55.524547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692221+0000","last_change":"2024-03-06T07:24:04.689963+0000","last_active":"2024-03-06T07:24:04.692221+0000","last_peered":"2024-03-06T07:24:04.692221+0000","last_clean":"2024-03-06T07:24:04.692221+0000","last_became_active":"2024-03-06T07:24:04.689688+0000","last_became_peered":"2024-03-06T07:24:04.689688+0000","last_unstale":"2024-03-06T07:24:04.692221+0000","last_undegraded":"2024-03-06T07:24:04.692221+0000","last_fullsized":"2024-03-06T07:24:04.692221+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:16.207859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.678792+0000","last_change":"2024-03-06T07:24:03.680294+0000","last_active":"2024-03-06T07:24:04.678792+0000","last_peered":"2024-03-06T07:24:04.678792+0000","last_clean":"2024-03-06T07:24:04.678792+0000","last_became_active":"2024-03-06T07:24:03.680035+0000","last_became_peered":"2024-03-06T07:24:03.680035+0000","last_unstale":"2024-03-06T07:24:04.678792+0000","last_undegraded":"2024-03-06T07:24:04.678792+0000","last_fullsized":"2024-03-06T07:24:04.678792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:17.383767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.687414+0000","last_change":"2024-03-06T07:24:04.683624+0000","last_active":"2024-03-06T07:24:04.687414+0000","last_peered":"2024-03-06T07:24:04.687414+0000","last_clean":"2024-03-06T07:24:04.687414+0000","last_became_active":"2024-03-06T07:24:04.683361+0000","last_became_peered":"2024-03-06T07:24:04.683361+0000","last_unstale":"2024-03-06T07:24:04.687414+0000","last_undegraded":"2024-03-06T07:24:04.687414+0000","last_fullsized":"2024-03-06T07:24:04.687414+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:47.901117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.674579+0000","last_change":"2024-03-06T07:24:03.686247+0000","last_active":"2024-03-06T07:24:04.674579+0000","last_peered":"2024-03-06T07:24:04.674579+0000","last_clean":"2024-03-06T07:24:04.674579+0000","last_became_active":"2024-03-06T07:24:03.685959+0000","last_became_peered":"2024-03-06T07:24:03.685959+0000","last_unstale":"2024-03-06T07:24:04.674579+0000","last_undegraded":"2024-03-06T07:24:04.674579+0000","last_fullsized":"2024-03-06T07:24:04.674579+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:44.619751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.706908+0000","last_change":"2024-03-06T07:24:04.701767+0000","last_active":"2024-03-06T07:24:04.706908+0000","last_peered":"2024-03-06T07:24:04.706908+0000","last_clean":"2024-03-06T07:24:04.706908+0000","last_became_active":"2024-03-06T07:24:04.701472+0000","last_became_peered":"2024-03-06T07:24:04.701472+0000","last_unstale":"2024-03-06T07:24:04.706908+0000","last_undegraded":"2024-03-06T07:24:04.706908+0000","last_fullsized":"2024-03-06T07:24:04.706908+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:38.089819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.677209+0000","last_change":"2024-03-06T07:24:03.679962+0000","last_active":"2024-03-06T07:24:04.677209+0000","last_peered":"2024-03-06T07:24:04.677209+0000","last_clean":"2024-03-06T07:24:04.677209+0000","last_became_active":"2024-03-06T07:24:03.679693+0000","last_became_peered":"2024-03-06T07:24:03.679693+0000","last_unstale":"2024-03-06T07:24:04.677209+0000","last_undegraded":"2024-03-06T07:24:04.677209+0000","last_fullsized":"2024-03-06T07:24:04.677209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:10.264883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696840+0000","last_change":"2024-03-06T07:24:04.688141+0000","last_active":"2024-03-06T07:24:04.696840+0000","last_peered":"2024-03-06T07:24:04.696840+0000","last_clean":"2024-03-06T07:24:04.696840+0000","last_became_active":"2024-03-06T07:24:04.687886+0000","last_became_peered":"2024-03-06T07:24:04.687886+0000","last_unstale":"2024-03-06T07:24:04.696840+0000","last_undegraded":"2024-03-06T07:24:04.696840+0000","last_fullsized":"2024-03-06T07:24:04.696840+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:52.255797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"107'3","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.696066+0000","last_change":"2024-03-06T07:24:03.682450+0000","last_active":"2024-03-06T07:24:04.696066+0000","last_peered":"2024-03-06T07:24:04.696066+0000","last_clean":"2024-03-06T07:24:04.696066+0000","last_became_active":"2024-03-06T07:24:03.682030+0000","last_became_peered":"2024-03-06T07:24:03.682030+0000","last_unstale":"2024-03-06T07:24:04.696066+0000","last_undegraded":"2024-03-06T07:24:04.696066+0000","last_fullsized":"2024-03-06T07:24:04.696066+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:50:08.123282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701378+0000","last_change":"2024-03-06T07:24:04.693710+0000","last_active":"2024-03-06T07:24:04.701378+0000","last_peered":"2024-03-06T07:24:04.701378+0000","last_clean":"2024-03-06T07:24:04.701378+0000","last_became_active":"2024-03-06T07:24:04.693434+0000","last_became_peered":"2024-03-06T07:24:04.693434+0000","last_unstale":"2024-03-06T07:24:04.701378+0000","last_undegraded":"2024-03-06T07:24:04.701378+0000","last_fullsized":"2024-03-06T07:24:04.701378+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:08.751391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"107'6","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.704445+0000","last_change":"2024-03-06T07:24:03.683633+0000","last_active":"2024-03-06T07:24:04.704445+0000","last_peered":"2024-03-06T07:24:04.704445+0000","last_clean":"2024-03-06T07:24:04.704445+0000","last_became_active":"2024-03-06T07:24:03.683135+0000","last_became_peered":"2024-03-06T07:24:03.683135+0000","last_unstale":"2024-03-06T07:24:04.704445+0000","last_undegraded":"2024-03-06T07:24:04.704445+0000","last_fullsized":"2024-03-06T07:24:04.704445+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:51.791521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701933+0000","last_change":"2024-03-06T07:24:04.696886+0000","last_active":"2024-03-06T07:24:04.701933+0000","last_peered":"2024-03-06T07:24:04.701933+0000","last_clean":"2024-03-06T07:24:04.701933+0000","last_became_active":"2024-03-06T07:24:04.696662+0000","last_became_peered":"2024-03-06T07:24:04.696662+0000","last_unstale":"2024-03-06T07:24:04.701933+0000","last_undegraded":"2024-03-06T07:24:04.701933+0000","last_fullsized":"2024-03-06T07:24:04.701933+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:16.207859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.704140+0000","last_change":"2024-03-06T07:24:03.678845+0000","last_active":"2024-03-06T07:24:04.704140+0000","last_peered":"2024-03-06T07:24:04.704140+0000","last_clean":"2024-03-06T07:24:04.704140+0000","last_became_active":"2024-03-06T07:24:03.678529+0000","last_became_peered":"2024-03-06T07:24:03.678529+0000","last_unstale":"2024-03-06T07:24:04.704140+0000","last_undegraded":"2024-03-06T07:24:04.704140+0000","last_fullsized":"2024-03-06T07:24:04.704140+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:29.132311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.685633+0000","last_change":"2024-03-06T07:24:04.684170+0000","last_active":"2024-03-06T07:24:04.685633+0000","last_peered":"2024-03-06T07:24:04.685633+0000","last_clean":"2024-03-06T07:24:04.685633+0000","last_became_active":"2024-03-06T07:24:04.683891+0000","last_became_peered":"2024-03-06T07:24:04.683891+0000","last_unstale":"2024-03-06T07:24:04.685633+0000","last_undegraded":"2024-03-06T07:24:04.685633+0000","last_fullsized":"2024-03-06T07:24:04.685633+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:23.035754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.673743+0000","last_change":"2024-03-06T07:24:03.677156+0000","last_active":"2024-03-06T07:24:04.673743+0000","last_peered":"2024-03-06T07:24:04.673743+0000","last_clean":"2024-03-06T07:24:04.673743+0000","last_became_active":"2024-03-06T07:24:03.676900+0000","last_became_peered":"2024-03-06T07:24:03.676900+0000","last_unstale":"2024-03-06T07:24:04.673743+0000","last_undegraded":"2024-03-06T07:24:04.673743+0000","last_fullsized":"2024-03-06T07:24:04.673743+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:20.317474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.682438+0000","last_change":"2024-03-06T07:24:04.677402+0000","last_active":"2024-03-06T07:24:04.682438+0000","last_peered":"2024-03-06T07:24:04.682438+0000","last_clean":"2024-03-06T07:24:04.682438+0000","last_became_active":"2024-03-06T07:24:04.677130+0000","last_became_peered":"2024-03-06T07:24:04.677130+0000","last_unstale":"2024-03-06T07:24:04.682438+0000","last_undegraded":"2024-03-06T07:24:04.682438+0000","last_fullsized":"2024-03-06T07:24:04.682438+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:19.658462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.673244+0000","last_change":"2024-03-06T07:24:03.684357+0000","last_active":"2024-03-06T07:24:04.673244+0000","last_peered":"2024-03-06T07:24:04.673244+0000","last_clean":"2024-03-06T07:24:04.673244+0000","last_became_active":"2024-03-06T07:24:03.684091+0000","last_became_peered":"2024-03-06T07:24:03.684091+0000","last_unstale":"2024-03-06T07:24:04.673244+0000","last_undegraded":"2024-03-06T07:24:04.673244+0000","last_fullsized":"2024-03-06T07:24:04.673244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:46.748792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.695804+0000","last_change":"2024-03-06T07:24:04.692313+0000","last_active":"2024-03-06T07:24:04.695804+0000","last_peered":"2024-03-06T07:24:04.695804+0000","last_clean":"2024-03-06T07:24:04.695804+0000","last_became_active":"2024-03-06T07:24:04.692034+0000","last_became_peered":"2024-03-06T07:24:04.692034+0000","last_unstale":"2024-03-06T07:24:04.695804+0000","last_undegraded":"2024-03-06T07:24:04.695804+0000","last_fullsized":"2024-03-06T07:24:04.695804+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:21.324032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693491+0000","last_change":"2024-03-06T07:24:03.686441+0000","last_active":"2024-03-06T07:24:04.693491+0000","last_peered":"2024-03-06T07:24:04.693491+0000","last_clean":"2024-03-06T07:24:04.693491+0000","last_became_active":"2024-03-06T07:24:03.686116+0000","last_became_peered":"2024-03-06T07:24:03.686116+0000","last_unstale":"2024-03-06T07:24:04.693491+0000","last_undegraded":"2024-03-06T07:24:04.693491+0000","last_fullsized":"2024-03-06T07:24:04.693491+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:01.728277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.694108+0000","last_change":"2024-03-06T07:24:04.690422+0000","last_active":"2024-03-06T07:24:04.694108+0000","last_peered":"2024-03-06T07:24:04.694108+0000","last_clean":"2024-03-06T07:24:04.694108+0000","last_became_active":"2024-03-06T07:24:04.688975+0000","last_became_peered":"2024-03-06T07:24:04.688975+0000","last_unstale":"2024-03-06T07:24:04.694108+0000","last_undegraded":"2024-03-06T07:24:04.694108+0000","last_fullsized":"2024-03-06T07:24:04.694108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:17:56.531106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.699107+0000","last_change":"2024-03-06T07:24:03.684908+0000","last_active":"2024-03-06T07:24:04.699107+0000","last_peered":"2024-03-06T07:24:04.699107+0000","last_clean":"2024-03-06T07:24:04.699107+0000","last_became_active":"2024-03-06T07:24:03.684623+0000","last_became_peered":"2024-03-06T07:24:03.684623+0000","last_unstale":"2024-03-06T07:24:04.699107+0000","last_undegraded":"2024-03-06T07:24:04.699107+0000","last_fullsized":"2024-03-06T07:24:04.699107+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:58.570744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.687348+0000","last_change":"2024-03-06T07:24:04.684997+0000","last_active":"2024-03-06T07:24:04.687348+0000","last_peered":"2024-03-06T07:24:04.687348+0000","last_clean":"2024-03-06T07:24:04.687348+0000","last_became_active":"2024-03-06T07:24:04.684737+0000","last_became_peered":"2024-03-06T07:24:04.684737+0000","last_unstale":"2024-03-06T07:24:04.687348+0000","last_undegraded":"2024-03-06T07:24:04.687348+0000","last_fullsized":"2024-03-06T07:24:04.687348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:52.426992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702134+0000","last_change":"2024-03-06T07:24:03.682451+0000","last_active":"2024-03-06T07:24:04.702134+0000","last_peered":"2024-03-06T07:24:04.702134+0000","last_clean":"2024-03-06T07:24:04.702134+0000","last_became_active":"2024-03-06T07:24:03.682163+0000","last_became_peered":"2024-03-06T07:24:03.682163+0000","last_unstale":"2024-03-06T07:24:04.702134+0000","last_undegraded":"2024-03-06T07:24:04.702134+0000","last_fullsized":"2024-03-06T07:24:04.702134+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:07.744832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.690147+0000","last_change":"2024-03-06T07:24:04.684865+0000","last_active":"2024-03-06T07:24:04.690147+0000","last_peered":"2024-03-06T07:24:04.690147+0000","last_clean":"2024-03-06T07:24:04.690147+0000","last_became_active":"2024-03-06T07:24:04.684609+0000","last_became_peered":"2024-03-06T07:24:04.684609+0000","last_unstale":"2024-03-06T07:24:04.690147+0000","last_undegraded":"2024-03-06T07:24:04.690147+0000","last_fullsized":"2024-03-06T07:24:04.690147+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:47.027547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668984+0000","last_change":"2024-03-06T07:24:03.684490+0000","last_active":"2024-03-06T07:24:04.668984+0000","last_peered":"2024-03-06T07:24:04.668984+0000","last_clean":"2024-03-06T07:24:04.668984+0000","last_became_active":"2024-03-06T07:24:03.684199+0000","last_became_peered":"2024-03-06T07:24:03.684199+0000","last_unstale":"2024-03-06T07:24:04.668984+0000","last_undegraded":"2024-03-06T07:24:04.668984+0000","last_fullsized":"2024-03-06T07:24:04.668984+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:20.317474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693316+0000","last_change":"2024-03-06T07:24:04.690146+0000","last_active":"2024-03-06T07:24:04.693316+0000","last_peered":"2024-03-06T07:24:04.693316+0000","last_clean":"2024-03-06T07:24:04.693316+0000","last_became_active":"2024-03-06T07:24:04.689894+0000","last_became_peered":"2024-03-06T07:24:04.689894+0000","last_unstale":"2024-03-06T07:24:04.693316+0000","last_undegraded":"2024-03-06T07:24:04.693316+0000","last_fullsized":"2024-03-06T07:24:04.693316+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:39.501600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702020+0000","last_change":"2024-03-06T07:24:03.678126+0000","last_active":"2024-03-06T07:24:04.702020+0000","last_peered":"2024-03-06T07:24:04.702020+0000","last_clean":"2024-03-06T07:24:04.702020+0000","last_became_active":"2024-03-06T07:24:03.677848+0000","last_became_peered":"2024-03-06T07:24:03.677848+0000","last_unstale":"2024-03-06T07:24:04.702020+0000","last_undegraded":"2024-03-06T07:24:04.702020+0000","last_fullsized":"2024-03-06T07:24:04.702020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:07:44.078772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701083+0000","last_change":"2024-03-06T07:24:04.700361+0000","last_active":"2024-03-06T07:24:04.701083+0000","last_peered":"2024-03-06T07:24:04.701083+0000","last_clean":"2024-03-06T07:24:04.701083+0000","last_became_active":"2024-03-06T07:24:04.700055+0000","last_became_peered":"2024-03-06T07:24:04.700055+0000","last_unstale":"2024-03-06T07:24:04.701083+0000","last_undegraded":"2024-03-06T07:24:04.701083+0000","last_fullsized":"2024-03-06T07:24:04.701083+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:51.331935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.669478+0000","last_change":"2024-03-06T07:24:03.682134+0000","last_active":"2024-03-06T07:24:04.669478+0000","last_peered":"2024-03-06T07:24:04.669478+0000","last_clean":"2024-03-06T07:24:04.669478+0000","last_became_active":"2024-03-06T07:24:03.681827+0000","last_became_peered":"2024-03-06T07:24:03.681827+0000","last_unstale":"2024-03-06T07:24:04.669478+0000","last_undegraded":"2024-03-06T07:24:04.669478+0000","last_fullsized":"2024-03-06T07:24:04.669478+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:28:10.765871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697397+0000","last_change":"2024-03-06T07:24:04.686857+0000","last_active":"2024-03-06T07:24:04.697397+0000","last_peered":"2024-03-06T07:24:04.697397+0000","last_clean":"2024-03-06T07:24:04.697397+0000","last_became_active":"2024-03-06T07:24:04.686354+0000","last_became_peered":"2024-03-06T07:24:04.686354+0000","last_unstale":"2024-03-06T07:24:04.697397+0000","last_undegraded":"2024-03-06T07:24:04.697397+0000","last_fullsized":"2024-03-06T07:24:04.697397+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:27.521026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.673408+0000","last_change":"2024-03-06T07:24:03.674812+0000","last_active":"2024-03-06T07:24:04.673408+0000","last_peered":"2024-03-06T07:24:04.673408+0000","last_clean":"2024-03-06T07:24:04.673408+0000","last_became_active":"2024-03-06T07:24:03.674516+0000","last_became_peered":"2024-03-06T07:24:03.674516+0000","last_unstale":"2024-03-06T07:24:04.673408+0000","last_undegraded":"2024-03-06T07:24:04.673408+0000","last_fullsized":"2024-03-06T07:24:04.673408+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:49.569232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.701417+0000","last_change":"2024-03-06T07:24:04.694424+0000","last_active":"2024-03-06T07:24:04.701417+0000","last_peered":"2024-03-06T07:24:04.701417+0000","last_clean":"2024-03-06T07:24:04.701417+0000","last_became_active":"2024-03-06T07:24:04.694158+0000","last_became_peered":"2024-03-06T07:24:04.694158+0000","last_unstale":"2024-03-06T07:24:04.701417+0000","last_undegraded":"2024-03-06T07:24:04.701417+0000","last_fullsized":"2024-03-06T07:24:04.701417+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:28:11.772430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668115+0000","last_change":"2024-03-06T07:24:03.679323+0000","last_active":"2024-03-06T07:24:04.668115+0000","last_peered":"2024-03-06T07:24:04.668115+0000","last_clean":"2024-03-06T07:24:04.668115+0000","last_became_active":"2024-03-06T07:24:03.679075+0000","last_became_peered":"2024-03-06T07:24:03.679075+0000","last_unstale":"2024-03-06T07:24:04.668115+0000","last_undegraded":"2024-03-06T07:24:04.668115+0000","last_fullsized":"2024-03-06T07:24:04.668115+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:49.569232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.693348+0000","last_change":"2024-03-06T07:24:04.691239+0000","last_active":"2024-03-06T07:24:04.693348+0000","last_peered":"2024-03-06T07:24:04.693348+0000","last_clean":"2024-03-06T07:24:04.693348+0000","last_became_active":"2024-03-06T07:24:04.690994+0000","last_became_peered":"2024-03-06T07:24:04.690994+0000","last_unstale":"2024-03-06T07:24:04.693348+0000","last_undegraded":"2024-03-06T07:24:04.693348+0000","last_fullsized":"2024-03-06T07:24:04.693348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:24.805015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.699388+0000","last_change":"2024-03-06T07:24:03.685336+0000","last_active":"2024-03-06T07:24:04.699388+0000","last_peered":"2024-03-06T07:24:04.699388+0000","last_clean":"2024-03-06T07:24:04.699388+0000","last_became_active":"2024-03-06T07:24:03.685016+0000","last_became_peered":"2024-03-06T07:24:03.685016+0000","last_unstale":"2024-03-06T07:24:04.699388+0000","last_undegraded":"2024-03-06T07:24:04.699388+0000","last_fullsized":"2024-03-06T07:24:04.699388+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:25:44.005879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.682571+0000","last_change":"2024-03-06T07:24:04.680868+0000","last_active":"2024-03-06T07:24:04.682571+0000","last_peered":"2024-03-06T07:24:04.682571+0000","last_clean":"2024-03-06T07:24:04.682571+0000","last_became_active":"2024-03-06T07:24:04.680592+0000","last_became_peered":"2024-03-06T07:24:04.680592+0000","last_unstale":"2024-03-06T07:24:04.682571+0000","last_undegraded":"2024-03-06T07:24:04.682571+0000","last_fullsized":"2024-03-06T07:24:04.682571+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:00.690207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.675381+0000","last_change":"2024-03-06T07:24:03.684559+0000","last_active":"2024-03-06T07:24:04.675381+0000","last_peered":"2024-03-06T07:24:04.675381+0000","last_clean":"2024-03-06T07:24:04.675381+0000","last_became_active":"2024-03-06T07:24:03.684275+0000","last_became_peered":"2024-03-06T07:24:03.684275+0000","last_unstale":"2024-03-06T07:24:04.675381+0000","last_undegraded":"2024-03-06T07:24:04.675381+0000","last_fullsized":"2024-03-06T07:24:04.675381+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:59.683648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.679244+0000","last_change":"2024-03-06T07:24:04.673515+0000","last_active":"2024-03-06T07:24:04.679244+0000","last_peered":"2024-03-06T07:24:04.679244+0000","last_clean":"2024-03-06T07:24:04.679244+0000","last_became_active":"2024-03-06T07:24:04.673200+0000","last_became_peered":"2024-03-06T07:24:04.673200+0000","last_unstale":"2024-03-06T07:24:04.679244+0000","last_undegraded":"2024-03-06T07:24:04.679244+0000","last_fullsized":"2024-03-06T07:24:04.679244+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:18:00.939752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.675487+0000","last_change":"2024-03-06T07:24:03.684961+0000","last_active":"2024-03-06T07:24:04.675487+0000","last_peered":"2024-03-06T07:24:04.675487+0000","last_clean":"2024-03-06T07:24:04.675487+0000","last_became_active":"2024-03-06T07:24:03.684660+0000","last_became_peered":"2024-03-06T07:24:03.684660+0000","last_unstale":"2024-03-06T07:24:04.675487+0000","last_undegraded":"2024-03-06T07:24:04.675487+0000","last_fullsized":"2024-03-06T07:24:04.675487+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:32.993866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692108+0000","last_change":"2024-03-06T07:24:04.684383+0000","last_active":"2024-03-06T07:24:04.692108+0000","last_peered":"2024-03-06T07:24:04.692108+0000","last_clean":"2024-03-06T07:24:04.692108+0000","last_became_active":"2024-03-06T07:24:04.684136+0000","last_became_peered":"2024-03-06T07:24:04.684136+0000","last_unstale":"2024-03-06T07:24:04.692108+0000","last_undegraded":"2024-03-06T07:24:04.692108+0000","last_fullsized":"2024-03-06T07:24:04.692108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:01.614069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.681692+0000","last_change":"2024-03-06T07:24:03.682563+0000","last_active":"2024-03-06T07:24:04.681692+0000","last_peered":"2024-03-06T07:24:04.681692+0000","last_clean":"2024-03-06T07:24:04.681692+0000","last_became_active":"2024-03-06T07:24:03.682279+0000","last_became_peered":"2024-03-06T07:24:03.682279+0000","last_unstale":"2024-03-06T07:24:04.681692+0000","last_undegraded":"2024-03-06T07:24:04.681692+0000","last_fullsized":"2024-03-06T07:24:04.681692+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:02.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692264+0000","last_change":"2024-03-06T07:24:04.688131+0000","last_active":"2024-03-06T07:24:04.692264+0000","last_peered":"2024-03-06T07:24:04.692264+0000","last_clean":"2024-03-06T07:24:04.692264+0000","last_became_active":"2024-03-06T07:24:04.687816+0000","last_became_peered":"2024-03-06T07:24:04.687816+0000","last_unstale":"2024-03-06T07:24:04.692264+0000","last_undegraded":"2024-03-06T07:24:04.692264+0000","last_fullsized":"2024-03-06T07:24:04.692264+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:33.116358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.670981+0000","last_change":"2024-03-06T07:24:03.685692+0000","last_active":"2024-03-06T07:24:04.670981+0000","last_peered":"2024-03-06T07:24:04.670981+0000","last_clean":"2024-03-06T07:24:04.670981+0000","last_became_active":"2024-03-06T07:24:03.685395+0000","last_became_peered":"2024-03-06T07:24:03.685395+0000","last_unstale":"2024-03-06T07:24:04.670981+0000","last_undegraded":"2024-03-06T07:24:04.670981+0000","last_fullsized":"2024-03-06T07:24:04.670981+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:59.683648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.697095+0000","last_change":"2024-03-06T07:24:04.687555+0000","last_active":"2024-03-06T07:24:04.697095+0000","last_peered":"2024-03-06T07:24:04.697095+0000","last_clean":"2024-03-06T07:24:04.697095+0000","last_became_active":"2024-03-06T07:24:04.687176+0000","last_became_peered":"2024-03-06T07:24:04.687176+0000","last_unstale":"2024-03-06T07:24:04.697095+0000","last_undegraded":"2024-03-06T07:24:04.697095+0000","last_fullsized":"2024-03-06T07:24:04.697095+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:29.990981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.670429+0000","last_change":"2024-03-06T07:24:03.685423+0000","last_active":"2024-03-06T07:24:04.670429+0000","last_peered":"2024-03-06T07:24:04.670429+0000","last_clean":"2024-03-06T07:24:04.670429+0000","last_became_active":"2024-03-06T07:24:03.685122+0000","last_became_peered":"2024-03-06T07:24:03.685122+0000","last_unstale":"2024-03-06T07:24:04.670429+0000","last_undegraded":"2024-03-06T07:24:04.670429+0000","last_fullsized":"2024-03-06T07:24:04.670429+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:44.619751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.692635+0000","last_change":"2024-03-06T07:24:04.688408+0000","last_active":"2024-03-06T07:24:04.692635+0000","last_peered":"2024-03-06T07:24:04.692635+0000","last_clean":"2024-03-06T07:24:04.692635+0000","last_became_active":"2024-03-06T07:24:04.688128+0000","last_became_peered":"2024-03-06T07:24:04.688128+0000","last_unstale":"2024-03-06T07:24:04.692635+0000","last_undegraded":"2024-03-06T07:24:04.692635+0000","last_fullsized":"2024-03-06T07:24:04.692635+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:25.153991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.681468+0000","last_change":"2024-03-06T07:24:03.682581+0000","last_active":"2024-03-06T07:24:04.681468+0000","last_peered":"2024-03-06T07:24:04.681468+0000","last_clean":"2024-03-06T07:24:04.681468+0000","last_became_active":"2024-03-06T07:24:03.682290+0000","last_became_peered":"2024-03-06T07:24:03.682290+0000","last_unstale":"2024-03-06T07:24:04.681468+0000","last_undegraded":"2024-03-06T07:24:04.681468+0000","last_fullsized":"2024-03-06T07:24:04.681468+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:41.359866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.680570+0000","last_change":"2024-03-06T07:24:04.680027+0000","last_active":"2024-03-06T07:24:04.680570+0000","last_peered":"2024-03-06T07:24:04.680570+0000","last_clean":"2024-03-06T07:24:04.680570+0000","last_became_active":"2024-03-06T07:24:04.679762+0000","last_became_peered":"2024-03-06T07:24:04.679762+0000","last_unstale":"2024-03-06T07:24:04.680570+0000","last_undegraded":"2024-03-06T07:24:04.680570+0000","last_fullsized":"2024-03-06T07:24:04.680570+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:45.626310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668284+0000","last_change":"2024-03-06T07:24:03.679899+0000","last_active":"2024-03-06T07:24:04.668284+0000","last_peered":"2024-03-06T07:24:04.668284+0000","last_clean":"2024-03-06T07:24:04.668284+0000","last_became_active":"2024-03-06T07:24:03.679646+0000","last_became_peered":"2024-03-06T07:24:03.679646+0000","last_unstale":"2024-03-06T07:24:04.668284+0000","last_undegraded":"2024-03-06T07:24:04.668284+0000","last_fullsized":"2024-03-06T07:24:04.668284+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:22.238147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.689933+0000","last_change":"2024-03-06T07:24:04.686167+0000","last_active":"2024-03-06T07:24:04.689933+0000","last_peered":"2024-03-06T07:24:04.689933+0000","last_clean":"2024-03-06T07:24:04.689933+0000","last_became_active":"2024-03-06T07:24:04.685535+0000","last_became_peered":"2024-03-06T07:24:04.685535+0000","last_unstale":"2024-03-06T07:24:04.689933+0000","last_undegraded":"2024-03-06T07:24:04.689933+0000","last_fullsized":"2024-03-06T07:24:04.689933+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:03.658678+0000","last_clean_scrub_stamp":"2024-03-06T07:24:03.658678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:52.255797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"107'8","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.702017+0000","last_change":"2024-03-06T07:24:03.680038+0000","last_active":"2024-03-06T07:24:04.702017+0000","last_peered":"2024-03-06T07:24:04.702017+0000","last_clean":"2024-03-06T07:24:04.702017+0000","last_became_active":"2024-03-06T07:24:03.679755+0000","last_became_peered":"2024-03-06T07:24:03.679755+0000","last_unstale":"2024-03-06T07:24:04.702017+0000","last_undegraded":"2024-03-06T07:24:04.702017+0000","last_fullsized":"2024-03-06T07:24:04.702017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:02.652119+0000","last_clean_scrub_stamp":"2024-03-06T07:24:02.652119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:22.238147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-03-06T07:24:04.668389+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:24:04.668389+0000","last_peered":"2024-03-06T07:24:04.668389+0000","last_clean":"2024-03-06T07:24:04.668389+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:24:04.668389+0000","last_undegraded":"2024-03-06T07:24:04.668389+0000","last_fullsized":"2024-03-06T07:24:04.668389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1388544,"data_stored":1226434,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":644845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":636066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1388544,"data_stored":1226434,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:24:08.029 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:24:08.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:24:08.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.173+0000 7f3aca684640 1 Processor -- start 2024-03-06T07:24:08.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.173+0000 7f3aca684640 1 -- start start 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3aca684640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3ac406bff0 0x7f3ac40c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3aca684640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ac40c7210 0x7f3ac40c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3aca684640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac40b04b0 con 0x7f3ac40b00d0 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3aca684640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac4156160 con 0x7f3ac40c7210 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3aca684640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40c5710 con 0x7f3ac406bff0 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac8c21640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ac40c7210 0x7f3ac40c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac8c21640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ac40c7210 0x7f3ac40c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45228/0 (socket says 172.21.15.26:45228) 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac8c21640 1 -- 172.21.15.26:0/2870037815 learned_addr learned my addr 172.21.15.26:0/2870037815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac37fe640 1 --2- 172.21.15.26:0/2870037815 >> v2:172.21.15.117:3300/0 conn(0x7f3ac406bff0 0x7f3ac40c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac8c21640 1 -- 172.21.15.26:0/2870037815 >> v2:172.21.15.117:3300/0 conn(0x7f3ac406bff0 msgr2=0x7f3ac40c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac8c21640 1 --2- 172.21.15.26:0/2870037815 >> v2:172.21.15.117:3300/0 conn(0x7f3ac406bff0 0x7f3ac40c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac8c21640 1 -- 172.21.15.26:0/2870037815 >> v1:172.21.15.117:6789/0 conn(0x7f3ac40b00d0 legacy=0x7f3ac406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:08.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.174+0000 7f3ac8c21640 1 -- 172.21.15.26:0/2870037815 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac406d210 con 0x7f3ac40c7210 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3ac8c21640 1 --2- 172.21.15.26:0/2870037815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac40c7680 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3ac406ce20 tx=0x7f3abc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a4b27a48f207d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3ac2ffd640 1 -- 172.21.15.26:0/2870037815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc02ca80 con 0x7f3ac40c7210 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3ac2ffd640 1 -- 172.21.15.26:0/2870037815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc02cc20 con 0x7f3ac40c7210 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3ac2ffd640 1 -- 172.21.15.26:0/2870037815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0343f0 con 0x7f3ac40c7210 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 -- 172.21.15.26:0/2870037815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 msgr2=0x7f3ac40c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 --2- 172.21.15.26:0/2870037815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac40c7680 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3ac406ce20 tx=0x7f3abc02bee0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 -- 172.21.15.26:0/2870037815 shutdown_connections 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 --2- 172.21.15.26:0/2870037815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac40c7680 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 --2- 172.21.15.26:0/2870037815 >> v2:172.21.15.117:3300/0 conn(0x7f3ac406bff0 0x7f3ac40c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 -- 172.21.15.26:0/2870037815 >> 172.21.15.26:0/2870037815 conn(0x7f3ac40ac810 msgr2=0x7f3ac40acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 -- 172.21.15.26:0/2870037815 shutdown_connections 2024-03-06T07:24:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.175+0000 7f3aca684640 1 -- 172.21.15.26:0/2870037815 wait complete. 2024-03-06T07:24:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.176+0000 7f3aca684640 1 Processor -- start 2024-03-06T07:24:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3aca684640 1 -- start start 2024-03-06T07:24:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3aca684640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ac406bff0 0x7f3ac41504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3aca684640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ac40b00d0 0x7f3ac414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3aca684640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3aca684640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40c5ba0 con 0x7f3ac40c7210 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3aca684640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4152c50 con 0x7f3ac406bff0 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3aca684640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40c5d20 con 0x7f3ac40b00d0 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3ac8c21640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3ac8c21640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac414e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45236/0 (socket says 172.21.15.26:45236) 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.177+0000 7f3ac8c21640 1 -- 172.21.15.26:0/1343824224 learned_addr learned my addr 172.21.15.26:0/1343824224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac37fe640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ac40b00d0 0x7f3ac414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac8c21640 1 -- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ac40b00d0 msgr2=0x7f3ac414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac8c21640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ac40b00d0 0x7f3ac414e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac8c21640 1 -- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ac406bff0 msgr2=0x7f3ac41504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac3fff640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ac406bff0 0x7f3ac41504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac8c21640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ac406bff0 0x7f3ac41504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac8c21640 1 -- 172.21.15.26:0/1343824224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4159110 con 0x7f3ac40c7210 2024-03-06T07:24:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac8c21640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac414e5a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3abc02c4f0 tx=0x7f3abc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac0ff9640 1 -- 172.21.15.26:0/1343824224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc043020 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac0ff9640 1 -- 172.21.15.26:0/1343824224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc03e450 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac4157c80 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3ac0ff9640 1 -- 172.21.15.26:0/1343824224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc009510 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.178+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac4158240 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.179+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac40caa20 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.180+0000 7f3ac0ff9640 1 -- 172.21.15.26:0/1343824224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3abc046020 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.180+0000 7f3ac0ff9640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.180+0000 7f3ac0ff9640 1 -- 172.21.15.26:0/1343824224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3abc0acbd0 con 0x7f3ac40c7210 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.180+0000 7f3ac3fff640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.181+0000 7f3ac3fff640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ab006b4d0 0x7f3ab006d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40015d0 tx=0x7f3ab4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.184+0000 7f3ac0ff9640 1 -- 172.21.15.26:0/1343824224 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3abc04d030 con 0x7f3ac40c7210 2024-03-06T07:24:08.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.562+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3ac404f330 con 0x7f3ac40c7210 2024-03-06T07:24:08.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.564+0000 7f3ac0ff9640 1 -- 172.21.15.26:0/1343824224 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3abc07b950 con 0x7f3ac40c7210 2024-03-06T07:24:08.566 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:24:08.566 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:24:08.566 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:24:08.566 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:24:08.566 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:24:08.567 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:24:08.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ab006b4d0 msgr2=0x7f3ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ab006b4d0 0x7f3ab006d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40015d0 tx=0x7f3ab4009040 comp rx=0 tx=0).stop 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 msgr2=0x7f3ac414e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac414e5a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3abc02c4f0 tx=0x7f3abc004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 shutdown_connections 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ac40c7210 0x7f3ac414e5a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ac40b00d0 0x7f3ac414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 --2- 172.21.15.26:0/1343824224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ac406bff0 0x7f3ac41504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 >> 172.21.15.26:0/1343824224 conn(0x7f3ac40ac810 msgr2=0x7f3ac40acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 shutdown_connections 2024-03-06T07:24:08.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.567+0000 7f3aca684640 1 -- 172.21.15.26:0/1343824224 wait complete. 2024-03-06T07:24:08.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:08.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.684+0000 7f287778c640 1 Processor -- start 2024-03-06T07:24:08.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.685+0000 7f287778c640 1 -- start start 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.685+0000 7f287778c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28700c88c0 0x7f2870155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.685+0000 7f287778c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f2870158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.685+0000 7f287778c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700c73d0 con 0x7f287014f360 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.685+0000 7f287778c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28700c8dd0 con 0x7f28700c88c0 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.685+0000 7f287778c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28700be2e0 con 0x7f2870158b60 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.685+0000 7f2874d27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28700c88c0 0x7f2870155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.686+0000 7f2874d27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28700c88c0 0x7f2870155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45244/0 (socket says 172.21.15.26:45244) 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.686+0000 7f2874d27640 1 -- 172.21.15.26:0/2959229685 learned_addr learned my addr 172.21.15.26:0/2959229685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.686+0000 7f2875d29640 1 --2- 172.21.15.26:0/2959229685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f2870158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.686+0000 7f2874d27640 1 -- 172.21.15.26:0/2959229685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 msgr2=0x7f2870158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.686+0000 7f2874d27640 1 --2- 172.21.15.26:0/2959229685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f2870158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.686+0000 7f2874d27640 1 -- 172.21.15.26:0/2959229685 >> v1:172.21.15.117:6789/0 conn(0x7f287014f360 legacy=0x7f28700c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.686+0000 7f2874d27640 1 -- 172.21.15.26:0/2959229685 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28700c7670 con 0x7f28700c88c0 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f2874d27640 1 --2- 172.21.15.26:0/2959229685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28700c88c0 0x7f2870155c10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2864009870 tx=0x7f286402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8846b96c1556348c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f286ffff640 1 -- 172.21.15.26:0/2959229685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28640317c0 con 0x7f28700c88c0 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f286ffff640 1 -- 172.21.15.26:0/2959229685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864037070 con 0x7f28700c88c0 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f286ffff640 1 -- 172.21.15.26:0/2959229685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864031b80 con 0x7f28700c88c0 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 -- 172.21.15.26:0/2959229685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28700c88c0 msgr2=0x7f2870155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 --2- 172.21.15.26:0/2959229685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28700c88c0 0x7f2870155c10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2864009870 tx=0x7f286402b860 comp rx=0 tx=0).stop 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 -- 172.21.15.26:0/2959229685 shutdown_connections 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 --2- 172.21.15.26:0/2959229685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f2870158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 --2- 172.21.15.26:0/2959229685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28700c88c0 0x7f2870155c10 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 -- 172.21.15.26:0/2959229685 >> 172.21.15.26:0/2959229685 conn(0x7f28700ac570 msgr2=0x7f28700ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 -- 172.21.15.26:0/2959229685 shutdown_connections 2024-03-06T07:24:08.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.687+0000 7f287778c640 1 -- 172.21.15.26:0/2959229685 wait complete. 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.689+0000 7f287778c640 1 Processor -- start 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.689+0000 7f287778c640 1 -- start start 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f287778c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700c88c0 0x7f28701eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f287778c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287014f360 0x7f287014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f287778c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f28700abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f287778c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28700c5a50 con 0x7f287014f360 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f287778c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28700c7c30 con 0x7f28700c88c0 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f287778c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28700c5bd0 con 0x7f2870158b60 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287014f360 0x7f287014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287014f360 0x7f287014fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45252/0 (socket says 172.21.15.26:45252) 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 -- 172.21.15.26:0/2254642555 learned_addr learned my addr 172.21.15.26:0/2254642555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2875d29640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f28700abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2875528640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700c88c0 0x7f28701eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 -- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 msgr2=0x7f28700abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f28700abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 -- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700c88c0 msgr2=0x7f28701eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700c88c0 0x7f28701eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 -- 172.21.15.26:0/2254642555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287014c3a0 con 0x7f287014f360 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.690+0000 7f2874d27640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287014f360 0x7f287014fad0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2864000c00 tx=0x7f28640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.691+0000 7f286dffb640 1 -- 172.21.15.26:0/2254642555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864032650 con 0x7f287014f360 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.691+0000 7f286dffb640 1 -- 172.21.15.26:0/2254642555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28640327f0 con 0x7f287014f360 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.691+0000 7f286dffb640 1 -- 172.21.15.26:0/2254642555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864032be0 con 0x7f287014f360 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.691+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28700c3ed0 con 0x7f287014f360 2024-03-06T07:24:08.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.691+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28701eb000 con 0x7f287014f360 2024-03-06T07:24:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.692+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28700c81d0 con 0x7f287014f360 2024-03-06T07:24:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.692+0000 7f286dffb640 1 -- 172.21.15.26:0/2254642555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f286404c060 con 0x7f287014f360 2024-03-06T07:24:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.693+0000 7f286dffb640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f285c06b500 0x7f285c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.693+0000 7f286dffb640 1 -- 172.21.15.26:0/2254642555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28640abd00 con 0x7f287014f360 2024-03-06T07:24:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.693+0000 7f2875528640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f285c06b500 0x7f285c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.693+0000 7f2875528640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f285c06b500 0x7f285c06d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f28700ac470 tx=0x7f2868009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.697+0000 7f286dffb640 1 -- 172.21.15.26:0/2254642555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28700c81d0 con 0x7f287014f360 2024-03-06T07:24:08.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.915+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f287004f330 con 0x7f287014f360 2024-03-06T07:24:08.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.915+0000 7f286dffb640 1 -- 172.21.15.26:0/2254642555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4979 (secure 0 0 0) 0x7f2864040070 con 0x7f287014f360 2024-03-06T07:24:08.917 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:08.917 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:06.033301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1745266733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1745266733},{"type":"v1","addr":"172.21.15.26:6835","nonce":1745266733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-03-06T07:24:08.917 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2024-03-06T07:24:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f285c06b500 msgr2=0x7f285c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f285c06b500 0x7f285c06d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f28700ac470 tx=0x7f2868009060 comp rx=0 tx=0).stop 2024-03-06T07:24:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287014f360 msgr2=0x7f287014fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287014f360 0x7f287014fad0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2864000c00 tx=0x7f28640046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 shutdown_connections 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f285c06b500 0x7f285c06d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2870158b60 0x7f28700abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287014f360 0x7f287014fad0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 --2- 172.21.15.26:0/2254642555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700c88c0 0x7f28701eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 >> 172.21.15.26:0/2254642555 conn(0x7f28700ac570 msgr2=0x7f28700ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.917+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 shutdown_connections 2024-03-06T07:24:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:08.918+0000 7f287778c640 1 -- 172.21.15.26:0/2254642555 wait complete. 2024-03-06T07:24:08.961 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-03-06T07:24:08.960+0000 7efc4c48a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:09.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:04.037603+0000', 'modified': '2024-03-06T07:24:06.033301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7368}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7368': {'gid': 7368, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.26:6835/1745266733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1745266733}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1745266733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7368, 'qdb_cluster': [7368]} 2024-03-06T07:24:09.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:09.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:24:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.437+0000 7fa4167a2640 1 Processor -- start 2024-03-06T07:24:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4167a2640 1 -- start start 2024-03-06T07:24:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4167a2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa4100721a0 0x7fa410155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4167a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4100c25f0 0x7fa4100c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4167a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4100c3af0 con 0x7fa41014f3e0 2024-03-06T07:24:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4167a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4100713e0 con 0x7fa4100c25f0 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4167a2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4100716c0 con 0x7fa4100721a0 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa415fa1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4100c25f0 0x7fa4100c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4157a0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa41014f3e0 0x7fa4100c5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37150/0 (socket says 172.21.15.26:37150) 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.438+0000 7fa4157a0640 1 -- 172.21.15.26:0/1397015429 learned_addr learned my addr 172.21.15.26:0/1397015429 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.439+0000 7fa414f9f640 1 --2- 172.21.15.26:0/1397015429 >> v2:172.21.15.117:3300/0 conn(0x7fa4100721a0 0x7fa410155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.439+0000 7fa415fa1640 1 -- 172.21.15.26:0/1397015429 >> v2:172.21.15.117:3300/0 conn(0x7fa4100721a0 msgr2=0x7fa410155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.439+0000 7fa415fa1640 1 --2- 172.21.15.26:0/1397015429 >> v2:172.21.15.117:3300/0 conn(0x7fa4100721a0 0x7fa410155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.439+0000 7fa415fa1640 1 -- 172.21.15.26:0/1397015429 >> v1:172.21.15.26:6789/0 conn(0x7fa41014f3e0 legacy=0x7fa4100c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.439+0000 7fa407fff640 1 -- 172.21.15.26:0/1397015429 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240018796 0 0) 0x7fa4100c3af0 con 0x7fa41014f3e0 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.439+0000 7fa415fa1640 1 -- 172.21.15.26:0/1397015429 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4100719a0 con 0x7fa4100c25f0 2024-03-06T07:24:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.439+0000 7fa414f9f640 1 --2- 172.21.15.26:0/1397015429 >> v2:172.21.15.117:3300/0 conn(0x7fa4100721a0 0x7fa410155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.440+0000 7fa415fa1640 1 --2- 172.21.15.26:0/1397015429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa4100c2a60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa4100c8620 tx=0x7fa40802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c9ea813f5f7cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.440+0000 7fa407fff640 1 -- 172.21.15.26:0/1397015429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408038070 con 0x7fa4100c25f0 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.440+0000 7fa407fff640 1 -- 172.21.15.26:0/1397015429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa408031710 con 0x7fa4100c25f0 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.440+0000 7fa407fff640 1 -- 172.21.15.26:0/1397015429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408031ae0 con 0x7fa4100c25f0 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 -- 172.21.15.26:0/1397015429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 msgr2=0x7fa4100c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 --2- 172.21.15.26:0/1397015429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa4100c2a60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa4100c8620 tx=0x7fa40802b7b0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 -- 172.21.15.26:0/1397015429 shutdown_connections 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 --2- 172.21.15.26:0/1397015429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa4100c2a60 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 --2- 172.21.15.26:0/1397015429 >> v2:172.21.15.117:3300/0 conn(0x7fa4100721a0 0x7fa410155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 -- 172.21.15.26:0/1397015429 >> 172.21.15.26:0/1397015429 conn(0x7fa4100c4010 msgr2=0x7fa4100c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 -- 172.21.15.26:0/1397015429 shutdown_connections 2024-03-06T07:24:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.441+0000 7fa4167a2640 1 -- 172.21.15.26:0/1397015429 wait complete. 2024-03-06T07:24:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 Processor -- start 2024-03-06T07:24:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 -- start start 2024-03-06T07:24:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4100721a0 0x7fa41014a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa41015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41014f3e0 0x7fa410147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100c2fa0 con 0x7fa4100c25f0 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100c5b60 con 0x7fa41014f3e0 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.442+0000 7fa4167a2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100c3120 con 0x7fa4100721a0 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa41015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa41015cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45278/0 (socket says 172.21.15.26:45278) 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 -- 172.21.15.26:0/3929394545 learned_addr learned my addr 172.21.15.26:0/3929394545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 -- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4100721a0 msgr2=0x7fa41014a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa415fa1640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41014f3e0 0x7fa410147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4100721a0 0x7fa41014a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 -- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41014f3e0 msgr2=0x7fa410147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41014f3e0 0x7fa410147310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 -- 172.21.15.26:0/3929394545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4101e97b0 con 0x7fa4100c25f0 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa414f9f640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa41015cac0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e7e0 tx=0x7fa40000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa405ffb640 1 -- 172.21.15.26:0/3929394545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40000f040 con 0x7fa4100c25f0 2024-03-06T07:24:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa405ffb640 1 -- 172.21.15.26:0/3929394545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400004030 con 0x7fa4100c25f0 2024-03-06T07:24:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41014b210 con 0x7fa4100c25f0 2024-03-06T07:24:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa405ffb640 1 -- 172.21.15.26:0/3929394545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400004400 con 0x7fa4100c25f0 2024-03-06T07:24:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.443+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101e8cd0 con 0x7fa4100c25f0 2024-03-06T07:24:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.445+0000 7fa405ffb640 1 -- 172.21.15.26:0/3929394545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa40000c6d0 con 0x7fa4100c25f0 2024-03-06T07:24:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.445+0000 7fa405ffb640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3f406b4d0 0x7fa3f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.445+0000 7fa405ffb640 1 -- 172.21.15.26:0/3929394545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa40008a160 con 0x7fa4100c25f0 2024-03-06T07:24:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.445+0000 7fa4157a0640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3f406b4d0 0x7fa3f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.446+0000 7fa4157a0640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3f406b4d0 0x7fa3f406d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa40c007fd0 tx=0x7fa40c007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.447+0000 7fa41a5ea740 1 Processor -- start 2024-03-06T07:24:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.447+0000 7fa41a5ea740 1 -- start start 2024-03-06T07:24:09.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.448+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166449b0 0x5613165fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.448+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613165fb700 0x561316644d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.448+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166355d0 0x5613165f5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.448+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131655f6b0 con 0x5613166449b0 2024-03-06T07:24:09.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.448+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561316508a60 con 0x5613165fb700 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.448+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56131655b750 con 0x5613166355d0 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.448+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166449b0 0x5613165fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166449b0 0x5613165fb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45280/0 (socket says 172.21.15.26:45280) 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 -- 172.21.15.26:0/1090081102 learned_addr learned my addr 172.21.15.26:0/1090081102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa414f9f640 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613165fb700 0x561316644d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa415fa1640 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166355d0 0x5613165f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 -- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166355d0 msgr2=0x5613165f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166355d0 0x5613165f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 -- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613165fb700 msgr2=0x561316644d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613165fb700 0x561316644d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 -- 172.21.15.26:0/1090081102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561316526f30 con 0x5613166449b0 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa4157a0640 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166449b0 0x5613165fb1c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c790 tx=0x7fa40c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62fbee958821830 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa3faffd640 1 -- 172.21.15.26:0/1090081102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c004e30 con 0x5613166449b0 2024-03-06T07:24:09.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa3faffd640 1 -- 172.21.15.26:0/1090081102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c0072c0 con 0x5613166449b0 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.449+0000 7fa3faffd640 1 -- 172.21.15.26:0/1090081102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c007670 con 0x5613166449b0 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1090081102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166449b0 msgr2=0x5613165fb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166449b0 0x5613165fb1c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c790 tx=0x7fa40c00f000 comp rx=0 tx=0).stop 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1090081102 shutdown_connections 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166355d0 0x5613165f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613165fb700 0x561316644d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1090081102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166449b0 0x5613165fb1c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1090081102 >> 172.21.15.26:0/1090081102 conn(0x5613165f7980 msgr2=0x56131661c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1090081102 shutdown_connections 2024-03-06T07:24:09.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.450+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1090081102 wait complete. 2024-03-06T07:24:09.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.451+0000 7fa41a5ea740 1 -- start start 2024-03-06T07:24:09.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.451+0000 7fa41a5ea740 1 Processor -- start 2024-03-06T07:24:09.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x56131661b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x5613166369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x561316619180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561316639740 con 0x5613166355d0 2024-03-06T07:24:09.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131651bea0 con 0x56131664c980 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56131661cf60 con 0x5613166449b0 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa41a5ea740 10 client.0 _ll_register_callbacks cb 0x56131651b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x56131661b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x56131661b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45286/0 (socket says 172.21.15.26:45286) 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 -- 172.21.15.26:0/2112626426 learned_addr learned my addr 172.21.15.26:0/2112626426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa414f9f640 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x5613166369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa415fa1640 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x561316619180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 -- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 msgr2=0x5613166369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x5613166369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 -- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 msgr2=0x561316619180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x561316619180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.452+0000 7fa4157a0640 1 -- 172.21.15.26:0/2112626426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56131651c270 con 0x5613166355d0 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa4157a0640 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x56131661b160 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa40c015000 tx=0x7fa40c001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd5a98d4b87e1a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa3effff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa3effff640 1 -- 172.21.15.26:0/2112626426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0137a0 con 0x5613166355d0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa3effff640 1 -- 172.21.15.26:0/2112626426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa40c01c070 con 0x5613166355d0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa40c0072c0 con 0x5613166355d0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa3effff640 1 -- 172.21.15.26:0/2112626426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40c031050 con 0x5613166355d0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 10 client.7521 fetch_fsmap learned FSMap version 89 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5613162742d0 con 0x5613166355d0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa3effff640 1 -- 172.21.15.26:0/2112626426 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c0072c0 con 0x5613166355d0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa3effff640 1 -- 172.21.15.26:0/2112626426 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa40c034020 con 0x5613166355d0 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 10 client.7521 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 10 client.7521 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 20 client.7521 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x56131664fc10 0x56131665aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 4 client.7521 mds_command: new command op to 7368 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:09.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.453+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 --> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56131656cbf0 con 0x56131664fc10 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.454+0000 7fa414f9f640 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x56131664fc10 0x56131665aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.454+0000 7fa3effff640 10 client.7521 ms_handle_connect on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.454+0000 7fa414f9f640 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x56131664fc10 0x56131665aab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=debd53394453f016 server_cookie=c918d088d3e57fb8 in_seq=0 out_seq=0 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.454+0000 7fa3effff640 1 -- 172.21.15.26:0/2112626426 <== mds.0 v2:172.21.15.26:6833/1745266733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fa400081e50 con 0x56131664fc10 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.454+0000 7fa3effff640 10 client.7521 handle_command_reply: tid=0 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.454+0000 7fa41a5ea740 1 client.7521 shutdown 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 msgr2=0x56131661b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x56131661b160 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa40c015000 tx=0x7fa40c001470 comp rx=0 tx=0).stop 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 shutdown_connections 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x56131664fc10 0x56131665aab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x561316619180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x5613166369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/2112626426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x56131661b160 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 >> 172.21.15.26:0/2112626426 conn(0x5613165fb700 msgr2=0x5613165f5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa3effff640 0 client.7521 ms_handle_reset on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:09.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 shutdown_connections 2024-03-06T07:24:09.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 1 -- 172.21.15.26:0/2112626426 wait complete. 2024-03-06T07:24:09.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.455+0000 7fa41a5ea740 20 client.7521 trim_cache size 0 max 16384 2024-03-06T07:24:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 Processor -- start 2024-03-06T07:24:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 -- start start 2024-03-06T07:24:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x561316638d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613166449b0 0x56131664bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56131664c980 0x56131665ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41015eec0 con 0x5613166355d0 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410151550 con 0x5613166449b0 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41015e8c0 con 0x56131664c980 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x561316638d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x561316638d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45294/0 (socket says 172.21.15.26:45294) 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.488+0000 7fa4157a0640 1 -- 172.21.15.26:0/1539739366 learned_addr learned my addr 172.21.15.26:0/1539739366 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa4157a0640 1 -- 172.21.15.26:0/1539739366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56131664c980 msgr2=0x56131665ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa4157a0640 1 --2- 172.21.15.26:0/1539739366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56131664c980 0x56131665ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa4157a0640 1 -- 172.21.15.26:0/1539739366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613166449b0 msgr2=0x56131664bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa4157a0640 1 --2- 172.21.15.26:0/1539739366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613166449b0 0x56131664bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa4157a0640 1 -- 172.21.15.26:0/1539739366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56131633e6b0 con 0x5613166355d0 2024-03-06T07:24:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa4157a0640 1 --2- 172.21.15.26:0/1539739366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x561316638d90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c6d0 tx=0x7fa40c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c823c39a278deba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa3fa7fc640 1 -- 172.21.15.26:0/1539739366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0072c0 con 0x5613166355d0 2024-03-06T07:24:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa3fa7fc640 1 -- 172.21.15.26:0/1539739366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c03a070 con 0x5613166355d0 2024-03-06T07:24:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.489+0000 7fa3fa7fc640 1 -- 172.21.15.26:0/1539739366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c013040 con 0x5613166355d0 2024-03-06T07:24:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1539739366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 msgr2=0x561316638d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1539739366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x561316638d90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c6d0 tx=0x7fa40c00a000 comp rx=0 tx=0).stop 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1539739366 shutdown_connections 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1539739366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56131664c980 0x56131665ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1539739366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613166449b0 0x56131664bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/1539739366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x561316638d90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1539739366 >> 172.21.15.26:0/1539739366 conn(0x5613165fb700 msgr2=0x5613166367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1539739366 shutdown_connections 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 -- 172.21.15.26:0/1539739366 wait complete. 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 -- start start 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.490+0000 7fa41a5ea740 1 Processor -- start 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x5613165f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x561316630980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa41a5ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x5613166a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613166196c0 con 0x5613166355d0 2024-03-06T07:24:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561316636350 con 0x56131664c980 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa41a5ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561316619c00 con 0x5613166449b0 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa41a5ea740 10 client.0 _ll_register_callbacks cb 0x7fa40c007850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x5613165f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x5613165f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45300/0 (socket says 172.21.15.26:45300) 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 -- 172.21.15.26:0/401048945 learned_addr learned my addr 172.21.15.26:0/401048945 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa415fa1640 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x5613166a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa414f9f640 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x561316630980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 -- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 msgr2=0x561316630980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x561316630980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 -- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 msgr2=0x5613166a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x5613166a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.491+0000 7fa4157a0640 1 -- 172.21.15.26:0/401048945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561316634880 con 0x5613166355d0 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa4157a0640 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x5613165f6560 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c4b0 tx=0x7fa40c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb8bc5613c9d86b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa3effff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa3effff640 1 -- 172.21.15.26:0/401048945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c013ab0 con 0x5613166355d0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa3effff640 1 -- 172.21.15.26:0/401048945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa40c013c50 con 0x5613166355d0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa40c03a070 con 0x5613166355d0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa3effff640 1 -- 172.21.15.26:0/401048945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40c047050 con 0x5613166355d0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 10 client.7527 fetch_fsmap learned FSMap version 89 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x561316631240 con 0x5613166355d0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa3effff640 1 -- 172.21.15.26:0/401048945 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c03a070 con 0x5613166355d0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa3effff640 1 -- 172.21.15.26:0/401048945 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa40c04a020 con 0x5613166355d0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 10 client.7527 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 10 client.7527 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 20 client.7527 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5613166a9190 0x561316640ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 4 client.7527 mds_command: new command op to 7368 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 --> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561316566f40 con 0x5613166a9190 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.492+0000 7fa414f9f640 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5613166a9190 0x561316640ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa3effff640 10 client.7527 ms_handle_connect on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa414f9f640 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5613166a9190 0x561316640ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc64abd8b90d2a49 server_cookie=7e7d8f3ed09dc4b in_seq=0 out_seq=0 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa3effff640 1 -- 172.21.15.26:0/401048945 <== mds.0 v2:172.21.15.26:6833/1745266733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561316566f40 con 0x5613166a9190 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa3effff640 10 client.7527 handle_command_reply: tid=0 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 client.7527 shutdown 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 msgr2=0x5613165f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x5613165f6560 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c4b0 tx=0x7fa40c010000 comp rx=0 tx=0).stop 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 shutdown_connections 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5613166a9190 0x561316640ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56131664c980 0x5613166a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5613166449b0 0x561316630980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa3effff640 0 client.7527 ms_handle_reset on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 --2- 172.21.15.26:0/401048945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613166355d0 0x5613165f6560 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 >> 172.21.15.26:0/401048945 conn(0x5613165fb700 msgr2=0x5613165f4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 shutdown_connections 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 1 -- 172.21.15.26:0/401048945 wait complete. 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.493+0000 7fa41a5ea740 20 client.7527 trim_cache size 0 max 16384 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7368, 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:09.501 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:09.502 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.8277729620000001, 2024-03-06T07:24:09.502 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 89, 2024-03-06T07:24:09.502 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 108, 2024-03-06T07:24:09.502 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 108, 2024-03-06T07:24:09.502 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.9175488 2024-03-06T07:24:09.502 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3f406b4d0 msgr2=0x7fa3f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3f406b4d0 0x7fa3f406d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa40c007fd0 tx=0x7fa40c007ce0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 msgr2=0x7fa41015cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa41015cac0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e7e0 tx=0x7fa40000eca0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 shutdown_connections 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3f406b4d0 0x7fa3f406d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41014f3e0 0x7fa410147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4100c25f0 0x7fa41015cac0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 --2- 172.21.15.26:0/3929394545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4100721a0 0x7fa41014a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 >> 172.21.15.26:0/3929394545 conn(0x7fa4100c4010 msgr2=0x7fa41006f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 shutdown_connections 2024-03-06T07:24:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.494+0000 7fa4167a2640 1 -- 172.21.15.26:0/3929394545 wait complete. 2024-03-06T07:24:09.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.607+0000 7f14f6fd6640 1 Processor -- start 2024-03-06T07:24:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.607+0000 7f14f6fd6640 1 -- start start 2024-03-06T07:24:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f6fd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14f00c5480 0x7f14f0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f6fd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5860 0x7f14f006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f6fd6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14f01586f0 0x7f14f0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f6fd6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f006df40 con 0x7f14f00c5480 2024-03-06T07:24:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f6fd6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00c5cc0 con 0x7f14f01586f0 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f6fd6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f01591c0 con 0x7f14f00c5860 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f4d72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14f00c5480 0x7f14f0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f4d72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14f00c5480 0x7f14f0155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45308/0 (socket says 172.21.15.26:45308) 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.608+0000 7f14f4d72640 1 -- 172.21.15.26:0/2055671438 learned_addr learned my addr 172.21.15.26:0/2055671438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14effff640 1 --2- 172.21.15.26:0/2055671438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5860 0x7f14f006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14f4d72640 1 -- 172.21.15.26:0/2055671438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5860 msgr2=0x7f14f006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14f4d72640 1 --2- 172.21.15.26:0/2055671438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5860 0x7f14f006f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14f4d72640 1 -- 172.21.15.26:0/2055671438 >> v2:172.21.15.117:3300/0 conn(0x7f14f01586f0 msgr2=0x7f14f0158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14f4d72640 1 --2- 172.21.15.26:0/2055671438 >> v2:172.21.15.117:3300/0 conn(0x7f14f01586f0 0x7f14f0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14f4d72640 1 -- 172.21.15.26:0/2055671438 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f01594a0 con 0x7f14f00c5480 2024-03-06T07:24:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14f4d72640 1 --2- 172.21.15.26:0/2055671438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5480 0x7f14f0155ba0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f14f0071060 tx=0x7f14e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e7fadf746f8e50 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.609+0000 7f14ef7fe640 1 -- 172.21.15.26:0/2055671438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e402e070 con 0x7f14f00c5480 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14ef7fe640 1 -- 172.21.15.26:0/2055671438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e4037070 con 0x7f14f00c5480 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14ef7fe640 1 -- 172.21.15.26:0/2055671438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4031a00 con 0x7f14f00c5480 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 -- 172.21.15.26:0/2055671438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5480 msgr2=0x7f14f0155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/2055671438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5480 0x7f14f0155ba0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f14f0071060 tx=0x7f14e402b880 comp rx=0 tx=0).stop 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 -- 172.21.15.26:0/2055671438 shutdown_connections 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/2055671438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5860 0x7f14f006f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/2055671438 >> v2:172.21.15.117:3300/0 conn(0x7f14f01586f0 0x7f14f0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/2055671438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5480 0x7f14f0155ba0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 -- 172.21.15.26:0/2055671438 >> 172.21.15.26:0/2055671438 conn(0x7f14f006b1a0 msgr2=0x7f14f006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 -- 172.21.15.26:0/2055671438 shutdown_connections 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.610+0000 7f14f6fd6640 1 -- 172.21.15.26:0/2055671438 wait complete. 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.611+0000 7f14f6fd6640 1 Processor -- start 2024-03-06T07:24:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.611+0000 7f14f6fd6640 1 -- start start 2024-03-06T07:24:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f6fd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5480 0x7f14f01ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f6fd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5860 0x7f14f0156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f6fd6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f01586f0 0x7f14f0156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f6fd6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f0152a20 con 0x7f14f00c5860 2024-03-06T07:24:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f6fd6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f0155680 con 0x7f14f01586f0 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f6fd6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f0155050 con 0x7f14f00c5480 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5860 0x7f14f0156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5860 0x7f14f0156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45324/0 (socket says 172.21.15.26:45324) 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 -- 172.21.15.26:0/3561908211 learned_addr learned my addr 172.21.15.26:0/3561908211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f4d72640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5480 0x7f14f01ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14f5573640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f01586f0 0x7f14f0156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 -- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5480 msgr2=0x7f14f01ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5480 0x7f14f01ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 -- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f01586f0 msgr2=0x7f14f0156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f01586f0 0x7f14f0156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.612+0000 7f14effff640 1 -- 172.21.15.26:0/3561908211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f00ab6a0 con 0x7f14f00c5860 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.613+0000 7f14effff640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5860 0x7f14f0156440 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c820 tx=0x7f14dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.613+0000 7f14ed7fa640 1 -- 172.21.15.26:0/3561908211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc015070 con 0x7f14f00c5860 2024-03-06T07:24:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.613+0000 7f14ed7fa640 1 -- 172.21.15.26:0/3561908211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc004240 con 0x7f14f00c5860 2024-03-06T07:24:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.613+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f01eeb40 con 0x7f14f00c5860 2024-03-06T07:24:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.613+0000 7f14ed7fa640 1 -- 172.21.15.26:0/3561908211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00ce10 con 0x7f14f00c5860 2024-03-06T07:24:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.613+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f00ad940 con 0x7f14f00c5860 2024-03-06T07:24:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.614+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f004f330 con 0x7f14f00c5860 2024-03-06T07:24:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.614+0000 7f14ed7fa640 1 -- 172.21.15.26:0/3561908211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14dc007830 con 0x7f14f00c5860 2024-03-06T07:24:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.615+0000 7f14ed7fa640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14d406b4d0 0x7f14d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.615+0000 7f14ed7fa640 1 -- 172.21.15.26:0/3561908211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14dc08a250 con 0x7f14f00c5860 2024-03-06T07:24:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.615+0000 7f14f4d72640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14d406b4d0 0x7f14d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.616+0000 7f14f4d72640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14d406b4d0 0x7f14d406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f14e4036000 tx=0x7f14e403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.619+0000 7f14ed7fa640 1 -- 172.21.15.26:0/3561908211 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14dc058860 con 0x7f14f00c5860 2024-03-06T07:24:09.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.843+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14f006e0c0 con 0x7f14f00c5860 2024-03-06T07:24:09.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.843+0000 7f14ed7fa640 1 -- 172.21.15.26:0/3561908211 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4979 (secure 0 0 0) 0x7f14dc08a070 con 0x7f14f00c5860 2024-03-06T07:24:09.845 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:09.845 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:06.033301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1745266733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1745266733},{"type":"v1","addr":"172.21.15.26:6835","nonce":1745266733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-03-06T07:24:09.845 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14d406b4d0 msgr2=0x7f14d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14d406b4d0 0x7f14d406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f14e4036000 tx=0x7f14e403d040 comp rx=0 tx=0).stop 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5860 msgr2=0x7f14f0156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5860 0x7f14f0156440 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c820 tx=0x7f14dc00cce0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 shutdown_connections 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14d406b4d0 0x7f14d406d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f01586f0 0x7f14f0156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f00c5860 0x7f14f0156440 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 --2- 172.21.15.26:0/3561908211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14f00c5480 0x7f14f01ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 >> 172.21.15.26:0/3561908211 conn(0x7f14f006b1a0 msgr2=0x7f14f006fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 shutdown_connections 2024-03-06T07:24:09.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.845+0000 7f14f6fd6640 1 -- 172.21.15.26:0/3561908211 wait complete. 2024-03-06T07:24:09.858 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:04.037603+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7368": { "addr": "172.21.15.26:6835/1745266733", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 1745266733, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 1745266733, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7368, "incarnation": 87, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:06.033301+0000", "qdb_cluster": [ 7368 ], "qdb_leader": 7368, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7368 } } } ], "standbys": [ { "addr": "172.21.15.117:6835/548706796", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 548706796, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 548706796, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 85, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6305, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3362094815", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3362094815, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3362094815, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7084, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/256924791", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 256924791, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 256924791, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 85, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7365, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:09.858 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:09.858 INFO:tasks.cephfs.test_failover:current = [{'gid': 7368, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.26:6835/1745266733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1745266733}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1745266733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6305, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 85}, {'gid': 7365, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/256924791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 256924791}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 256924791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 85}, {'gid': 7084, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3362094815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3362094815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3362094815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}] 2024-03-06T07:24:09.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.b mds_join_fs cephfs 2024-03-06T07:24:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.961+0000 7fa78d713640 1 Processor -- start 2024-03-06T07:24:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.961+0000 7fa78d713640 1 -- start start 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.962+0000 7fa78d713640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7880ca1a0 0x7fa7880c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.962+0000 7fa78d713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880c5310 0x7fa7880c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.962+0000 7fa78d713640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa788155d20 con 0x7fa7880af720 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.962+0000 7fa78d713640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880ad900 con 0x7fa7880ca1a0 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.962+0000 7fa78d713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa788155fc0 con 0x7fa7880c5310 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.962+0000 7fa786ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa7880af720 0x7fa78814ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37152/0 (socket says 172.21.15.26:37152) 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.962+0000 7fa786ffd640 1 -- 172.21.15.26:0/3036881025 learned_addr learned my addr 172.21.15.26:0/3036881025 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa7877fe640 1 --2- 172.21.15.26:0/3036881025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880c5310 0x7fa7880c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa7867fc640 1 --2- 172.21.15.26:0/3036881025 >> v2:172.21.15.117:3300/0 conn(0x7fa7880ca1a0 0x7fa7880c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa785ffb640 1 -- 172.21.15.26:0/3036881025 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454876460 0 0) 0x7fa788155d20 con 0x7fa7880af720 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa785ffb640 1 -- 172.21.15.26:0/3036881025 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa770003520 con 0x7fa7880af720 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa785ffb640 1 -- 172.21.15.26:0/3036881025 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 604264306 0 0) 0x7fa770003520 con 0x7fa7880af720 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa7877fe640 1 -- 172.21.15.26:0/3036881025 >> v2:172.21.15.117:3300/0 conn(0x7fa7880ca1a0 msgr2=0x7fa7880c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa7877fe640 1 --2- 172.21.15.26:0/3036881025 >> v2:172.21.15.117:3300/0 conn(0x7fa7880ca1a0 0x7fa7880c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa7877fe640 1 -- 172.21.15.26:0/3036881025 >> v1:172.21.15.26:6789/0 conn(0x7fa7880af720 legacy=0x7fa78814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.963+0000 7fa7877fe640 1 -- 172.21.15.26:0/3036881025 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880adb50 con 0x7fa7880c5310 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa7877fe640 1 --2- 172.21.15.26:0/3036881025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880c5310 0x7fa7880c9070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7880ada80 tx=0x7fa77802bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90ab38a76fb9720 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa785ffb640 1 -- 172.21.15.26:0/3036881025 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778039070 con 0x7fa7880c5310 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa785ffb640 1 -- 172.21.15.26:0/3036881025 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77802c780 con 0x7fa7880c5310 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa785ffb640 1 -- 172.21.15.26:0/3036881025 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780001690 con 0x7fa7880c5310 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa78d713640 1 -- 172.21.15.26:0/3036881025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880c5310 msgr2=0x7fa7880c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa78d713640 1 --2- 172.21.15.26:0/3036881025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880c5310 0x7fa7880c9070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7880ada80 tx=0x7fa77802bc50 comp rx=0 tx=0).stop 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa78d713640 1 -- 172.21.15.26:0/3036881025 shutdown_connections 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa78d713640 1 --2- 172.21.15.26:0/3036881025 >> v2:172.21.15.117:3300/0 conn(0x7fa7880ca1a0 0x7fa7880c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa78d713640 1 --2- 172.21.15.26:0/3036881025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880c5310 0x7fa7880c9070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.964+0000 7fa78d713640 1 -- 172.21.15.26:0/3036881025 >> 172.21.15.26:0/3036881025 conn(0x7fa7880ac7e0 msgr2=0x7fa7880acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.965+0000 7fa78d713640 1 -- 172.21.15.26:0/3036881025 shutdown_connections 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.965+0000 7fa78d713640 1 -- 172.21.15.26:0/3036881025 wait complete. 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 Processor -- start 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 -- start start 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880af720 0x7fa788146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7880c5310 0x7fa78814b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7880ca1a0 0x7fa788154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880adfb0 con 0x7fa7880c5310 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880ade30 con 0x7fa7880ca1a0 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.966+0000 7fa78d713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78814f4a0 con 0x7fa7880af720 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7880c5310 0x7fa78814b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7880c5310 0x7fa78814b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45338/0 (socket says 172.21.15.26:45338) 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 -- 172.21.15.26:0/158635707 learned_addr learned my addr 172.21.15.26:0/158635707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa786ffd640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880af720 0x7fa788146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7877fe640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7880ca1a0 0x7fa788154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 -- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880af720 msgr2=0x7fa788146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880af720 0x7fa788146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 -- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7880ca1a0 msgr2=0x7fa788154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7880ca1a0 0x7fa788154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 -- 172.21.15.26:0/158635707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7881eb890 con 0x7fa7880c5310 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7877fe640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7880ca1a0 0x7fa788154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa7867fc640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7880c5310 0x7fa78814b220 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e980 tx=0x7fa77c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa76f7fe640 1 -- 172.21.15.26:0/158635707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c017070 con 0x7fa7880c5310 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa76f7fe640 1 -- 172.21.15.26:0/158635707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00ca00 con 0x7fa7880c5310 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa76f7fe640 1 -- 172.21.15.26:0/158635707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00cdd0 con 0x7fa7880c5310 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7881eab10 con 0x7fa7880c5310 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.967+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7881eb090 con 0x7fa7880c5310 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.968+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78804f330 con 0x7fa7880c5310 2024-03-06T07:24:09.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.969+0000 7fa76f7fe640 1 -- 172.21.15.26:0/158635707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa77c01a070 con 0x7fa7880c5310 2024-03-06T07:24:09.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.969+0000 7fa76f7fe640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa77406b4d0 0x7fa77406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.969+0000 7fa76f7fe640 1 -- 172.21.15.26:0/158635707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa77c08a250 con 0x7fa7880c5310 2024-03-06T07:24:09.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.969+0000 7fa786ffd640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa77406b4d0 0x7fa77406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.970+0000 7fa786ffd640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa77406b4d0 0x7fa77406d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa7881485e0 tx=0x7fa780001e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:09.973+0000 7fa76f7fe640 1 -- 172.21.15.26:0/158635707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa77c058980 con 0x7fa7880c5310 2024-03-06T07:24:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.245+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fa7880c9f80 con 0x7fa7880c5310 2024-03-06T07:24:10.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.248+0000 7fa76f7fe640 1 -- 172.21.15.26:0/158635707 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v36)=0 v36) v1 ==== 116+0+0 (secure 0 0 0) 0x7fa77c05ca20 con 0x7fa7880c5310 2024-03-06T07:24:10.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa77406b4d0 msgr2=0x7fa77406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa77406b4d0 0x7fa77406d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa7881485e0 tx=0x7fa780001e40 comp rx=0 tx=0).stop 2024-03-06T07:24:10.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7880c5310 msgr2=0x7fa78814b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7880c5310 0x7fa78814b220 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e980 tx=0x7fa77c00ee40 comp rx=0 tx=0).stop 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 shutdown_connections 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa77406b4d0 0x7fa77406d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7880ca1a0 0x7fa788154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7880c5310 0x7fa78814b220 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 --2- 172.21.15.26:0/158635707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7880af720 0x7fa788146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 >> 172.21.15.26:0/158635707 conn(0x7fa7880ac7e0 msgr2=0x7fa7880c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 shutdown_connections 2024-03-06T07:24:10.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.251+0000 7fa78d713640 1 -- 172.21.15.26:0/158635707 wait complete. 2024-03-06T07:24:10.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.364+0000 7fa420dd7640 1 Processor -- start 2024-03-06T07:24:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.364+0000 7fa420dd7640 1 -- start start 2024-03-06T07:24:10.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.364+0000 7fa420dd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5480 0x7fa41c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:10.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.364+0000 7fa420dd7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa41c1586f0 0x7fa41c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:10.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.364+0000 7fa420dd7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c0c5cc0 con 0x7fa41c0c5860 2024-03-06T07:24:10.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.364+0000 7fa420dd7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41c06df40 con 0x7fa41c1586f0 2024-03-06T07:24:10.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.364+0000 7fa420dd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c06fdb0 con 0x7fa41c0c5480 2024-03-06T07:24:10.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa41ad9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa41c1586f0 0x7fa41c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa41a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5480 0x7fa41c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa413fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa41c0c5860 0x7fa41c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:42310/0 (socket says 172.21.15.26:42310) 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa413fff640 1 -- 172.21.15.26:0/1377663108 learned_addr learned my addr 172.21.15.26:0/1377663108 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa41ad9d640 1 -- 172.21.15.26:0/1377663108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5480 msgr2=0x7fa41c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa41ad9d640 1 --2- 172.21.15.26:0/1377663108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5480 0x7fa41c155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa41ad9d640 1 -- 172.21.15.26:0/1377663108 >> v1:172.21.15.117:6789/0 conn(0x7fa41c0c5860 legacy=0x7fa41c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa41ad9d640 1 -- 172.21.15.26:0/1377663108 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c1591c0 con 0x7fa41c1586f0 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.365+0000 7fa419d9b640 1 -- 172.21.15.26:0/1377663108 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176384681 0 0) 0x7fa41c0c5cc0 con 0x7fa41c0c5860 2024-03-06T07:24:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.366+0000 7fa41ad9d640 1 --2- 172.21.15.26:0/1377663108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c1586f0 0x7fa41c158b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa41402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=979634fcae8b2f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:10.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.366+0000 7fa419d9b640 1 -- 172.21.15.26:0/1377663108 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414039070 con 0x7fa41c1586f0 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.366+0000 7fa419d9b640 1 -- 172.21.15.26:0/1377663108 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41402cbc0 con 0x7fa41c1586f0 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.366+0000 7fa419d9b640 1 -- 172.21.15.26:0/1377663108 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414034440 con 0x7fa41c1586f0 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 -- 172.21.15.26:0/1377663108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c1586f0 msgr2=0x7fa41c158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 --2- 172.21.15.26:0/1377663108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c1586f0 0x7fa41c158b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa41402be10 comp rx=0 tx=0).stop 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 -- 172.21.15.26:0/1377663108 shutdown_connections 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 --2- 172.21.15.26:0/1377663108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c1586f0 0x7fa41c158b80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 --2- 172.21.15.26:0/1377663108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5480 0x7fa41c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 -- 172.21.15.26:0/1377663108 >> 172.21.15.26:0/1377663108 conn(0x7fa41c06b1a0 msgr2=0x7fa41c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 -- 172.21.15.26:0/1377663108 shutdown_connections 2024-03-06T07:24:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.367+0000 7fa420dd7640 1 -- 172.21.15.26:0/1377663108 wait complete. 2024-03-06T07:24:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.369+0000 7fa420dd7640 1 Processor -- start 2024-03-06T07:24:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.369+0000 7fa420dd7640 1 -- start start 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa420dd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c0c5480 0x7fa41c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa420dd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5860 0x7fa41c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa420dd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c1586f0 0x7fa41c14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa420dd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c155680 con 0x7fa41c1586f0 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa420dd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c1553a0 con 0x7fa41c0c5480 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa420dd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c152a20 con 0x7fa41c0c5860 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c1586f0 0x7fa41c14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c1586f0 0x7fa41c14c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45352/0 (socket says 172.21.15.26:45352) 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 -- 172.21.15.26:0/2759285235 learned_addr learned my addr 172.21.15.26:0/2759285235 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41a59c640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c0c5480 0x7fa41c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa413fff640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5860 0x7fa41c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 -- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5860 msgr2=0x7fa41c157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5860 0x7fa41c157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 -- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c0c5480 msgr2=0x7fa41c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c0c5480 0x7fa41c14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.370+0000 7fa41ad9d640 1 -- 172.21.15.26:0/2759285235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0ad970 con 0x7fa41c1586f0 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.371+0000 7fa41ad9d640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c1586f0 0x7fa41c14c7f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa414004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.371+0000 7fa412ffd640 1 -- 172.21.15.26:0/2759285235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414044020 con 0x7fa41c1586f0 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.371+0000 7fa412ffd640 1 -- 172.21.15.26:0/2759285235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41403a040 con 0x7fa41c1586f0 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.371+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c0adc50 con 0x7fa41c1586f0 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.371+0000 7fa412ffd640 1 -- 172.21.15.26:0/2759285235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414039050 con 0x7fa41c1586f0 2024-03-06T07:24:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.371+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c14d3c0 con 0x7fa41c1586f0 2024-03-06T07:24:10.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.372+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c04f330 con 0x7fa41c1586f0 2024-03-06T07:24:10.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.372+0000 7fa412ffd640 1 -- 172.21.15.26:0/2759285235 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa414003bb0 con 0x7fa41c1586f0 2024-03-06T07:24:10.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.373+0000 7fa412ffd640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3fc06b4d0 0x7fa3fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:10.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.373+0000 7fa412ffd640 1 -- 172.21.15.26:0/2759285235 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4140ac740 con 0x7fa41c1586f0 2024-03-06T07:24:10.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.373+0000 7fa41a59c640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3fc06b4d0 0x7fa3fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:10.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.373+0000 7fa41a59c640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3fc06b4d0 0x7fa3fc06d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0c4ee0 tx=0x7fa40c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.377+0000 7fa412ffd640 1 -- 172.21.15.26:0/2759285235 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa41404e060 con 0x7fa41c1586f0 2024-03-06T07:24:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.601+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa41c06c6c0 con 0x7fa41c1586f0 2024-03-06T07:24:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.601+0000 7fa412ffd640 1 -- 172.21.15.26:0/2759285235 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4979 (secure 0 0 0) 0x7fa41407e390 con 0x7fa41c1586f0 2024-03-06T07:24:10.603 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:10.603 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:06.033301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1745266733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1745266733},{"type":"v1","addr":"172.21.15.26:6835","nonce":1745266733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-03-06T07:24:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2024-03-06T07:24:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3fc06b4d0 msgr2=0x7fa3fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3fc06b4d0 0x7fa3fc06d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0c4ee0 tx=0x7fa40c009190 comp rx=0 tx=0).stop 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c1586f0 msgr2=0x7fa41c14c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c1586f0 0x7fa41c14c7f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa414004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 shutdown_connections 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3fc06b4d0 0x7fa3fc06d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c1586f0 0x7fa41c14c7f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa41c0c5860 0x7fa41c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 --2- 172.21.15.26:0/2759285235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa41c0c5480 0x7fa41c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 >> 172.21.15.26:0/2759285235 conn(0x7fa41c06b1a0 msgr2=0x7fa41c154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 shutdown_connections 2024-03-06T07:24:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:10.603+0000 7fa420dd7640 1 -- 172.21.15.26:0/2759285235 wait complete. 2024-03-06T07:24:11.018 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:04.037603+0000', 'modified': '2024-03-06T07:24:06.033301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7368}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7368': {'gid': 7368, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.26:6835/1745266733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1745266733}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1745266733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7368, 'qdb_cluster': [7368]} 2024-03-06T07:24:11.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:11.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:24:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.120+0000 7f6f827c7640 1 Processor -- start 2024-03-06T07:24:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.120+0000 7f6f827c7640 1 -- start start 2024-03-06T07:24:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f827c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f7c0c69f0 0x7f6f7c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f827c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f7c1558d0 0x7f6f7c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f827c7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c155130 con 0x7f6f7c0c8950 2024-03-06T07:24:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f827c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c155440 con 0x7f6f7c1558d0 2024-03-06T07:24:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f827c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c0ad480 con 0x7f6f7c0c69f0 2024-03-06T07:24:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f80d64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f7c1558d0 0x7f6f7c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f80d64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f7c1558d0 0x7f6f7c155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45368/0 (socket says 172.21.15.26:45368) 2024-03-06T07:24:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f80d64640 1 -- 172.21.15.26:0/3964649558 learned_addr learned my addr 172.21.15.26:0/3964649558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3964649558 >> v2:172.21.15.117:3300/0 conn(0x7f6f7c0c69f0 0x7f6f7c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f80d64640 1 -- 172.21.15.26:0/3964649558 >> v2:172.21.15.117:3300/0 conn(0x7f6f7c0c69f0 msgr2=0x7f6f7c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f80d64640 1 --2- 172.21.15.26:0/3964649558 >> v2:172.21.15.117:3300/0 conn(0x7f6f7c0c69f0 0x7f6f7c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f80d64640 1 -- 172.21.15.26:0/3964649558 >> v1:172.21.15.117:6789/0 conn(0x7f6f7c0c8950 legacy=0x7f6f7c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.121+0000 7f6f80d64640 1 -- 172.21.15.26:0/3964649558 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0c5d50 con 0x7f6f7c1558d0 2024-03-06T07:24:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f80d64640 1 --2- 172.21.15.26:0/3964649558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c1558d0 0x7f6f7c155d00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0aa430 tx=0x7f6f7402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b7577e8cc8636a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f7affd640 1 -- 172.21.15.26:0/3964649558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7402c9f0 con 0x7f6f7c1558d0 2024-03-06T07:24:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f7affd640 1 -- 172.21.15.26:0/3964649558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7402cb90 con 0x7f6f7c1558d0 2024-03-06T07:24:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f7affd640 1 -- 172.21.15.26:0/3964649558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f740343f0 con 0x7f6f7c1558d0 2024-03-06T07:24:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 -- 172.21.15.26:0/3964649558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c1558d0 msgr2=0x7f6f7c155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3964649558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c1558d0 0x7f6f7c155d00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0aa430 tx=0x7f6f7402be50 comp rx=0 tx=0).stop 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 -- 172.21.15.26:0/3964649558 shutdown_connections 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3964649558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c1558d0 0x7f6f7c155d00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3964649558 >> v2:172.21.15.117:3300/0 conn(0x7f6f7c0c69f0 0x7f6f7c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 -- 172.21.15.26:0/3964649558 >> 172.21.15.26:0/3964649558 conn(0x7f6f7c0cae70 msgr2=0x7f6f7c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 -- 172.21.15.26:0/3964649558 shutdown_connections 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.123+0000 7f6f827c7640 1 -- 172.21.15.26:0/3964649558 wait complete. 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.124+0000 7f6f827c7640 1 Processor -- start 2024-03-06T07:24:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.125+0000 7f6f827c7640 1 -- start start 2024-03-06T07:24:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c0c69f0 0x7f6f7c14ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c0c8950 0x7f6f7c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f7c1558d0 0x7f6f7c14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c152a20 con 0x7f6f7c0c8950 2024-03-06T07:24:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0c6310 con 0x7f6f7c0c69f0 2024-03-06T07:24:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c152ba0 con 0x7f6f7c1558d0 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c0c8950 0x7f6f7c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c0c8950 0x7f6f7c158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45378/0 (socket says 172.21.15.26:45378) 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3378638001 learned_addr learned my addr 172.21.15.26:0/3378638001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c0c69f0 0x7f6f7c14ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f7c1558d0 msgr2=0x7f6f7c14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f7c1558d0 0x7f6f7c14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c0c69f0 msgr2=0x7f6f7c14ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c0c69f0 0x7f6f7c14ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3378638001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c14cee0 con 0x7f6f7c0c8950 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c0c8950 0x7f6f7c158250 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00b3f0 tx=0x7f6f6c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f78ff9640 1 -- 172.21.15.26:0/3378638001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c015020 con 0x7f6f7c0c8950 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f78ff9640 1 -- 172.21.15.26:0/3378638001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c011830 con 0x7f6f7c0c8950 2024-03-06T07:24:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c14b270 con 0x7f6f7c0c8950 2024-03-06T07:24:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f78ff9640 1 -- 172.21.15.26:0/3378638001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c011c20 con 0x7f6f7c0c8950 2024-03-06T07:24:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.126+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c1ebbc0 con 0x7f6f7c0c8950 2024-03-06T07:24:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.128+0000 7f6f78ff9640 1 -- 172.21.15.26:0/3378638001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f6c018020 con 0x7f6f7c0c8950 2024-03-06T07:24:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.129+0000 7f6f78ff9640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.129+0000 7f6f78ff9640 1 -- 172.21.15.26:0/3378638001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f6c089f90 con 0x7f6f7c0c8950 2024-03-06T07:24:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.129+0000 7f6f853af740 1 Processor -- start 2024-03-06T07:24:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.129+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.129+0000 7f6f853af740 1 -- start start 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6f64006fd0 tx=0x7f6f64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf349f50 0x5565bf347a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f853af740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf34a330 0x5565bf347f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf3484b0 0x5565bf344680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f853af740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bf2ae6b0 con 0x5565bf34a330 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bf257a60 con 0x5565bf3484b0 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bf2aa750 con 0x5565bf349f50 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf34a330 0x5565bf347f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf34a330 0x5565bf347f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45394/0 (socket says 172.21.15.26:45394) 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3089591142 learned_addr learned my addr 172.21.15.26:0/3089591142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf349f50 0x5565bf347a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f80d64640 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf3484b0 0x5565bf344680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf349f50 msgr2=0x5565bf347a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf349f50 0x5565bf347a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf3484b0 msgr2=0x5565bf344680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf3484b0 0x5565bf344680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 -- 172.21.15.26:0/3089591142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565bf275f30 con 0x5565bf34a330 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.130+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf34a330 0x5565bf347f70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c060760 tx=0x7f6f6c0586c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce834d0fe266daf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f71ffb640 1 -- 172.21.15.26:0/3089591142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c086070 con 0x5565bf34a330 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f71ffb640 1 -- 172.21.15.26:0/3089591142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c056880 con 0x5565bf34a330 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f71ffb640 1 -- 172.21.15.26:0/3089591142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c070070 con 0x5565bf34a330 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 -- 172.21.15.26:0/3089591142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf34a330 msgr2=0x5565bf347f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf34a330 0x5565bf347f70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c060760 tx=0x7f6f6c0586c0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 -- 172.21.15.26:0/3089591142 shutdown_connections 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf3484b0 0x5565bf344680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf34a330 0x5565bf347f70 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 --2- 172.21.15.26:0/3089591142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf349f50 0x5565bf347a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 -- 172.21.15.26:0/3089591142 >> 172.21.15.26:0/3089591142 conn(0x5565bf3467d0 msgr2=0x5565bf36b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 -- 172.21.15.26:0/3089591142 shutdown_connections 2024-03-06T07:24:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.131+0000 7f6f853af740 1 -- 172.21.15.26:0/3089591142 wait complete. 2024-03-06T07:24:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.132+0000 7f6f853af740 1 -- start start 2024-03-06T07:24:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.132+0000 7f6f853af740 1 Processor -- start 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f853af740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf36c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 0x5565bf36a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 0x5565bf399c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f853af740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bf36c230 con 0x5565bf349f50 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bf26aea0 con 0x5565bf397320 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bf36bc70 con 0x5565bf34a330 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f853af740 10 client.0 _ll_register_callbacks cb 0x5565bf26a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f7bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf36c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f7bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf36c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45404/0 (socket says 172.21.15.26:45404) 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.133+0000 7f6f7bfff640 1 -- 172.21.15.26:0/4273862070 learned_addr learned my addr 172.21.15.26:0/4273862070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f7bfff640 1 -- 172.21.15.26:0/4273862070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 msgr2=0x5565bf36a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f80d64640 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 0x5565bf399c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 0x5565bf36a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f7bfff640 1 -- 172.21.15.26:0/4273862070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 msgr2=0x5565bf399c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 0x5565bf399c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f7bfff640 1 -- 172.21.15.26:0/4273862070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565bf26b270 con 0x5565bf349f50 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf36c600 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400ee60 tx=0x7f6f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafd3a29e1f9bf42 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f5affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f5affd640 1 -- 172.21.15.26:0/4273862070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64025020 con 0x5565bf349f50 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f5affd640 1 -- 172.21.15.26:0/4273862070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f64015c90 con 0x5565bf349f50 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f6c056880 con 0x5565bf349f50 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f5affd640 1 -- 172.21.15.26:0/4273862070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f6402f050 con 0x5565bf349f50 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f853af740 10 client.7554 fetch_fsmap learned FSMap version 89 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5565befc32d0 con 0x5565bf349f50 2024-03-06T07:24:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f5affd640 1 -- 172.21.15.26:0/4273862070 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c056880 con 0x5565bf349f50 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f5affd640 1 -- 172.21.15.26:0/4273862070 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6f64032020 con 0x5565bf349f50 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.134+0000 7f6f853af740 10 client.7554 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f853af740 10 client.7554 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f853af740 20 client.7554 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f853af740 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3a3320 0x5565bf3a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f853af740 4 client.7554 mds_command: new command op to 7368 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 --> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565bf2bbbf0 con 0x5565bf3a3320 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3a3320 0x5565bf3a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f5affd640 10 client.7554 ms_handle_connect on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.135+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3a3320 0x5565bf3a5710 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b244333e6df6686 server_cookie=860be818d737f59f in_seq=0 out_seq=0 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f5affd640 1 -- 172.21.15.26:0/4273862070 <== mds.0 v2:172.21.15.26:6833/1745266733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5565bf2bbbf0 con 0x5565bf3a3320 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f5affd640 10 client.7554 handle_command_reply: tid=0 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 client.7554 shutdown 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 msgr2=0x5565bf36c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf36c600 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400ee60 tx=0x7f6f640046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 shutdown_connections 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3a3320 0x5565bf3a5710 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 0x5565bf399c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 0x5565bf36a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f5affd640 0 client.7554 ms_handle_reset on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 --2- 172.21.15.26:0/4273862070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf36c600 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 >> 172.21.15.26:0/4273862070 conn(0x5565bf3484b0 msgr2=0x5565bf3488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 shutdown_connections 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 1 -- 172.21.15.26:0/4273862070 wait complete. 2024-03-06T07:24:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.136+0000 7f6f853af740 20 client.7554 trim_cache size 0 max 16384 2024-03-06T07:24:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 Processor -- start 2024-03-06T07:24:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 -- start start 2024-03-06T07:24:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf343510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 0x5565bf364070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 0x5565bf3645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1f6f80 con 0x5565bf349f50 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1eee70 con 0x5565bf397320 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1f6a50 con 0x5565bf34a330 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f7bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf343510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f7bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf343510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45414/0 (socket says 172.21.15.26:45414) 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.153+0000 7f6f7bfff640 1 -- 172.21.15.26:0/2427181375 learned_addr learned my addr 172.21.15.26:0/2427181375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f7bfff640 1 -- 172.21.15.26:0/2427181375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 msgr2=0x5565bf364070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/2427181375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 0x5565bf364070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f7bfff640 1 -- 172.21.15.26:0/2427181375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 msgr2=0x5565bf3645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/2427181375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 0x5565bf3645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f7bfff640 1 -- 172.21.15.26:0/2427181375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565bf08d6b0 con 0x5565bf349f50 2024-03-06T07:24:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/2427181375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf343510 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400e190 tx=0x7f6f64021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedf4402ec5b0970 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f717fa640 1 -- 172.21.15.26:0/2427181375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64038070 con 0x5565bf349f50 2024-03-06T07:24:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f717fa640 1 -- 172.21.15.26:0/2427181375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64021780 con 0x5565bf349f50 2024-03-06T07:24:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.154+0000 7f6f717fa640 1 -- 172.21.15.26:0/2427181375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64025070 con 0x5565bf349f50 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 -- 172.21.15.26:0/2427181375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 msgr2=0x5565bf343510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 --2- 172.21.15.26:0/2427181375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf343510 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400e190 tx=0x7f6f64021000 comp rx=0 tx=0).stop 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 -- 172.21.15.26:0/2427181375 shutdown_connections 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 --2- 172.21.15.26:0/2427181375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf397320 0x5565bf3645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 --2- 172.21.15.26:0/2427181375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf34a330 0x5565bf364070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 --2- 172.21.15.26:0/2427181375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf343510 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 -- 172.21.15.26:0/2427181375 >> 172.21.15.26:0/2427181375 conn(0x5565bf3484b0 msgr2=0x5565bf3710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 -- 172.21.15.26:0/2427181375 shutdown_connections 2024-03-06T07:24:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 -- 172.21.15.26:0/2427181375 wait complete. 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.155+0000 7f6f853af740 1 -- start start 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 1 Processor -- start 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf368570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf34a330 0x5565bf3f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf397320 0x5565bf344260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bf394250 con 0x5565bf349f50 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bf396b20 con 0x5565bf34a330 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bf3676c0 con 0x5565bf397320 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f853af740 10 client.0 _ll_register_callbacks cb 0x7f6f6400eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f7bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf368570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f7bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf368570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45422/0 (socket says 172.21.15.26:45422) 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.156+0000 7f6f7bfff640 1 -- 172.21.15.26:0/3850381518 learned_addr learned my addr 172.21.15.26:0/3850381518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f80d64640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf397320 0x5565bf344260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf34a330 0x5565bf3f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7bfff640 1 -- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf397320 msgr2=0x5565bf344260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf397320 0x5565bf344260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7bfff640 1 -- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf34a330 msgr2=0x5565bf3f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf34a330 0x5565bf3f9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7bfff640 1 -- 172.21.15.26:0/3850381518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565bf382ca0 con 0x5565bf349f50 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f80d64640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf397320 0x5565bf344260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf34a330 0x5565bf3f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f7bfff640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf368570 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400e310 tx=0x7f6f64015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5de09c7d14ca8d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f5affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f5affd640 1 -- 172.21.15.26:0/3850381518 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64043060 con 0x5565bf349f50 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f5affd640 1 -- 172.21.15.26:0/3850381518 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f64021cb0 con 0x5565bf349f50 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f64021780 con 0x5565bf349f50 2024-03-06T07:24:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f5affd640 1 -- 172.21.15.26:0/3850381518 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f64049050 con 0x5565bf349f50 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f853af740 10 client.7560 fetch_fsmap learned FSMap version 89 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5565bf349140 con 0x5565bf349f50 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f5affd640 1 -- 172.21.15.26:0/3850381518 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64021780 con 0x5565bf349f50 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f5affd640 1 -- 172.21.15.26:0/3850381518 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6f6404c020 con 0x5565bf349f50 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f853af740 10 client.7560 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f853af740 10 client.7560 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.157+0000 7f6f853af740 20 client.7560 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3f2cf0 0x5565bf3f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 4 client.7560 mds_command: new command op to 7368 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 --> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565bf388b00 con 0x5565bf3f2cf0 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3f2cf0 0x5565bf3f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f5affd640 10 client.7560 ms_handle_connect on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f7b7fe640 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3f2cf0 0x5565bf3f30d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f6f6a3919b5707 server_cookie=5a5d8f5dff69ee76 in_seq=0 out_seq=0 2024-03-06T07:24:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f5affd640 1 -- 172.21.15.26:0/3850381518 <== mds.0 v2:172.21.15.26:6833/1745266733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5565bf388b00 con 0x5565bf3f2cf0 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f5affd640 10 client.7560 handle_command_reply: tid=0 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 client.7560 shutdown 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 msgr2=0x5565bf368570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf368570 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400e310 tx=0x7f6f64015000 comp rx=0 tx=0).stop 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 shutdown_connections 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:6833/1745266733,v1:172.21.15.26:6835/1745266733] conn(0x5565bf3f2cf0 0x5565bf3f30d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565bf397320 0x5565bf344260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565bf34a330 0x5565bf3f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f5affd640 0 client.7560 ms_handle_reset on v2:172.21.15.26:6833/1745266733 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 --2- 172.21.15.26:0/3850381518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565bf349f50 0x5565bf368570 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 >> 172.21.15.26:0/3850381518 conn(0x5565bf3484b0 msgr2=0x5565bf36aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 shutdown_connections 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.158+0000 7f6f853af740 1 -- 172.21.15.26:0/3850381518 wait complete. 2024-03-06T07:24:11.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f853af740 20 client.7560 trim_cache size 0 max 16384 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7368, 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 6.4930794690000004, 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 89, 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 108, 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 108, 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 17.582857392000001 2024-03-06T07:24:11.167 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f5c06b500 msgr2=0x7f6f5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6f64006fd0 tx=0x7f6f64008040 comp rx=0 tx=0).stop 2024-03-06T07:24:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c0c8950 msgr2=0x7f6f7c158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c0c8950 0x7f6f7c158250 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00b3f0 tx=0x7f6f6c00b8b0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 shutdown_connections 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f7c1558d0 0x7f6f7c14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f7c0c8950 0x7f6f7c158250 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 --2- 172.21.15.26:0/3378638001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c0c69f0 0x7f6f7c14ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 >> 172.21.15.26:0/3378638001 conn(0x7f6f7c0cae70 msgr2=0x7f6f7c154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 shutdown_connections 2024-03-06T07:24:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:11.159+0000 7f6f827c7640 1 -- 172.21.15.26:0/3378638001 wait complete. 2024-03-06T07:24:11.171 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:04.037603+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7368": { "addr": "172.21.15.26:6835/1745266733", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 1745266733, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 1745266733, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7368, "incarnation": 87, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:06.033301+0000", "qdb_cluster": [ 7368 ], "qdb_leader": 7368, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7368 } } } ], "standbys": [ { "addr": "172.21.15.117:6835/548706796", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 548706796, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 548706796, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 85, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6305, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3362094815", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3362094815, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3362094815, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7084, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/256924791", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 256924791, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 256924791, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 85, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7365, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:11.172 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 8, 'name': 'b', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:11.172 INFO:tasks.cephfs.test_failover:current = [{'gid': 7368, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.26:6835/1745266733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1745266733}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1745266733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6305, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 85}, {'gid': 7365, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/256924791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 256924791}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 256924791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 85}, {'gid': 7084, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3362094815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3362094815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3362094815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}] 2024-03-06T07:24:11.173 DEBUG:tasks.cephfs.test_failover:8 != -1 2024-03-06T07:24:11.173 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:24:11.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:11.952 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:13.711 INFO:tasks.ceph.mds.a.smithi026.stderr: -234> 2024-03-06T07:24:08.960+0000 7efc4c48a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:13.735 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:24:16.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:16.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.286+0000 7f7ad3f65640 1 Processor -- start 2024-03-06T07:24:16.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.286+0000 7f7ad3f65640 1 -- start start 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.287+0000 7f7ad3f65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 0x7f7acc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.287+0000 7f7ad3f65640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7acc0c73d0 con 0x7f7acc158b60 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.287+0000 7f7ad3f65640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7acc0be2e0 con 0x7f7acc0c88c0 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.287+0000 7f7ad3f65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc159700 con 0x7f7acc14f360 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.287+0000 7f7ad2502640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7acc158b60 0x7f7acc158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37168/0 (socket says 172.21.15.26:37168) 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.287+0000 7f7ad2502640 1 -- 172.21.15.26:0/2234230182 learned_addr learned my addr 172.21.15.26:0/2234230182 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 0x7f7acc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660510587 0 0) 0x7f7acc0c73d0 con 0x7f7acc158b60 2024-03-06T07:24:16.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab4003520 con 0x7f7acc158b60 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016984182 0 0) 0x7f7acc0be2e0 con 0x7f7acc0c88c0 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7acc0c73d0 con 0x7f7acc0c88c0 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3986358486 0 0) 0x7f7ab4003520 con 0x7f7acc158b60 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 msgr2=0x7f7acc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 0x7f7acc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 >> v1:172.21.15.117:6789/0 conn(0x7f7acc0c88c0 legacy=0x7f7acc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc0c7670 con 0x7f7acc158b60 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f7ac4001670 con 0x7f7acc158b60 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 0x7f7acc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 >> v1:172.21.15.26:6789/0 conn(0x7f7acc158b60 legacy=0x7f7acc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7ab4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 0x7f7ab4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ab4004d40 0x7f7ab40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc159700 con 0x7f7ab4003b70 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab400baa0 con 0x7f7ab4004d40 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab400bca0 con 0x7f7acc14f360 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7ab4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1500640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 0x7f7ab4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad2502640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ab4004d40 0x7f7ab40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 -- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 msgr2=0x7f7ab4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc14f360 0x7f7ab4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 -- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ab4004d40 msgr2=0x7f7ab40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ab4004d40 0x7f7ab40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.288+0000 7f7ad1d01640 1 -- 172.21.15.26:0/2234230182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4003520 con 0x7f7ab4003b70 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.289+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7ab4003f70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e350 tx=0x7f7ab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621e3f76d739a8a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.289+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc039070 con 0x7f7ab4003b70 2024-03-06T07:24:16.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.289+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc02cd70 con 0x7f7ab4003b70 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.289+0000 7f7ad0cff640 1 -- 172.21.15.26:0/2234230182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc009de0 con 0x7f7ab4003b70 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.290+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2234230182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 msgr2=0x7f7ab4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.290+0000 7f7ad3f65640 1 --2- 172.21.15.26:0/2234230182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7ab4003f70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e350 tx=0x7f7ab4005830 comp rx=0 tx=0).stop 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.290+0000 7f7ad2502640 1 -- 172.21.15.26:0/2234230182 reap_dead start 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.290+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2234230182 shutdown_connections 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.290+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2234230182 >> 172.21.15.26:0/2234230182 conn(0x7f7acc0ac570 msgr2=0x7f7acc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.290+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2234230182 shutdown_connections 2024-03-06T07:24:16.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.290+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2234230182 wait complete. 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.291+0000 7f7ad3f65640 1 Processor -- start 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.291+0000 7f7ad3f65640 1 -- start start 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad3f65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7acc0af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad3f65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7acc0affd0 0x7f7acc0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad3f65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc1e8ba0 0x7f7acc1ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad3f65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0c7c30 con 0x7f7ab4003b70 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad3f65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0c5a50 con 0x7f7acc0affd0 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad3f65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0c5bd0 con 0x7f7acc1e8ba0 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7acc0af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7acc0af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45472/0 (socket says 172.21.15.26:45472) 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 -- 172.21.15.26:0/2183490684 learned_addr learned my addr 172.21.15.26:0/2183490684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1500640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7acc0affd0 0x7f7acc0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad2502640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc1e8ba0 0x7f7acc1ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 -- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc1e8ba0 msgr2=0x7f7acc1ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc1e8ba0 0x7f7acc1ee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 -- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7acc0affd0 msgr2=0x7f7acc0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7acc0affd0 0x7f7acc0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.292+0000 7f7ad1d01640 1 -- 172.21.15.26:0/2183490684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc0ade40 con 0x7f7ab4003b70 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.293+0000 7f7ad1d01640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7acc0af8f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e2f0 tx=0x7f7abc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.293+0000 7f7aca7fc640 1 -- 172.21.15.26:0/2183490684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc042020 con 0x7f7ab4003b70 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.293+0000 7f7aca7fc640 1 -- 172.21.15.26:0/2183490684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc03a040 con 0x7f7ab4003b70 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.293+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc1eece0 con 0x7f7ab4003b70 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.293+0000 7f7aca7fc640 1 -- 172.21.15.26:0/2183490684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc039070 con 0x7f7ab4003b70 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.293+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc0ae3c0 con 0x7f7ab4003b70 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.294+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc04f330 con 0x7f7ab4003b70 2024-03-06T07:24:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.294+0000 7f7aca7fc640 1 -- 172.21.15.26:0/2183490684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7abc045020 con 0x7f7ab4003b70 2024-03-06T07:24:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.295+0000 7f7aca7fc640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ab806b500 0x7f7ab806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.295+0000 7f7aca7fc640 1 -- 172.21.15.26:0/2183490684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7abc0ac350 con 0x7f7ab4003b70 2024-03-06T07:24:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.295+0000 7f7ad1500640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ab806b500 0x7f7ab806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.295+0000 7f7ad1500640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ab806b500 0x7f7ab806d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007ae0 tx=0x7f7ac0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.299+0000 7f7aca7fc640 1 -- 172.21.15.26:0/2183490684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7abc04c070 con 0x7f7ab4003b70 2024-03-06T07:24:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.517+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7acc0c9510 con 0x7f7ab4003b70 2024-03-06T07:24:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.517+0000 7f7aca7fc640 1 -- 172.21.15.26:0/2183490684 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4970 (secure 0 0 0) 0x7f7abc07aff0 con 0x7f7ab4003b70 2024-03-06T07:24:16.520 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:16.520 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/131071104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":131071104},{"type":"v1","addr":"172.21.15.26:6835","nonce":131071104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:14.711223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-06T07:24:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 93 2024-03-06T07:24:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ab806b500 msgr2=0x7f7ab806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ab806b500 0x7f7ab806d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007ae0 tx=0x7f7ac0009040 comp rx=0 tx=0).stop 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 msgr2=0x7f7acc0af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7acc0af8f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e2f0 tx=0x7f7abc009750 comp rx=0 tx=0).stop 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 shutdown_connections 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ab806b500 0x7f7ab806d9c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7acc1e8ba0 0x7f7acc1ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7acc0affd0 0x7f7acc0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 --2- 172.21.15.26:0/2183490684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003b70 0x7f7acc0af8f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.519+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 >> 172.21.15.26:0/2183490684 conn(0x7f7acc0ac570 msgr2=0x7f7acc0cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.520+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 shutdown_connections 2024-03-06T07:24:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:16.520+0000 7f7ad3f65640 1 -- 172.21.15.26:0/2183490684 wait complete. 2024-03-06T07:24:16.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:04.037603+0000', 'modified': '2024-03-06T07:24:14.711223+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6305}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6305': {'gid': 6305, 'name': 'b', 'rank': 0, 'incarnation': 91, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:24:16.934 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6305:up:rejoin 2024-03-06T07:24:17.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:18.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.050+0000 7f0934684640 1 Processor -- start 2024-03-06T07:24:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.050+0000 7f0934684640 1 -- start start 2024-03-06T07:24:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.051+0000 7f0934684640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c1550c0 0x7f092c1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.051+0000 7f0934684640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0aa660 con 0x7f092c1559e0 2024-03-06T07:24:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.051+0000 7f0934684640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0aa940 con 0x7f092c0aa180 2024-03-06T07:24:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.051+0000 7f0934684640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c4610 con 0x7f092c1550c0 2024-03-06T07:24:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.051+0000 7f0931c1f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f092c1559e0 0x7f092c14ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59836/0 (socket says 172.21.15.26:59836) 2024-03-06T07:24:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.051+0000 7f0931c1f640 1 -- 172.21.15.26:0/4046003797 learned_addr learned my addr 172.21.15.26:0/4046003797 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.051+0000 7f0932420640 1 --2- 172.21.15.26:0/4046003797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c1550c0 0x7f092c1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.052+0000 7f093141e640 1 -- 172.21.15.26:0/4046003797 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848060992 0 0) 0x7f092c0aa660 con 0x7f092c1559e0 2024-03-06T07:24:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.052+0000 7f093141e640 1 -- 172.21.15.26:0/4046003797 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0914004630 con 0x7f092c1559e0 2024-03-06T07:24:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.052+0000 7f0932420640 1 -- 172.21.15.26:0/4046003797 >> v1:172.21.15.117:6789/0 conn(0x7f092c0aa180 legacy=0x7f092c0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.052+0000 7f0932420640 1 -- 172.21.15.26:0/4046003797 >> v1:172.21.15.26:6789/0 conn(0x7f092c1559e0 legacy=0x7f092c14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.052+0000 7f093141e640 1 -- 172.21.15.26:0/4046003797 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901408048 0 0) 0x7f092c0aa940 con 0x7f092c0aa180 2024-03-06T07:24:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.052+0000 7f0932420640 1 -- 172.21.15.26:0/4046003797 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0c4860 con 0x7f092c1550c0 2024-03-06T07:24:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.052+0000 7f0932420640 1 --2- 172.21.15.26:0/4046003797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c1550c0 0x7f092c1554a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0924009dc0 tx=0x7f092402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402d6221465688f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f093141e640 1 -- 172.21.15.26:0/4046003797 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924039070 con 0x7f092c1550c0 2024-03-06T07:24:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f093141e640 1 -- 172.21.15.26:0/4046003797 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092402cbc0 con 0x7f092c1550c0 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f093141e640 1 -- 172.21.15.26:0/4046003797 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924034440 con 0x7f092c1550c0 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f0934684640 1 -- 172.21.15.26:0/4046003797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c1550c0 msgr2=0x7f092c1554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f0934684640 1 --2- 172.21.15.26:0/4046003797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c1550c0 0x7f092c1554a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0924009dc0 tx=0x7f092402be10 comp rx=0 tx=0).stop 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f0934684640 1 -- 172.21.15.26:0/4046003797 shutdown_connections 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f0934684640 1 --2- 172.21.15.26:0/4046003797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c1550c0 0x7f092c1554a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f0934684640 1 -- 172.21.15.26:0/4046003797 >> 172.21.15.26:0/4046003797 conn(0x7f092c0c6710 msgr2=0x7f092c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f0934684640 1 -- 172.21.15.26:0/4046003797 shutdown_connections 2024-03-06T07:24:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.053+0000 7f0934684640 1 -- 172.21.15.26:0/4046003797 wait complete. 2024-03-06T07:24:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.054+0000 7f0934684640 1 Processor -- start 2024-03-06T07:24:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0934684640 1 -- start start 2024-03-06T07:24:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0934684640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c0aa180 0x7f092c15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0934684640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092c1550c0 0x7f092c14c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0934684640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c1559e0 0x7f092c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0934684640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c14f310 con 0x7f092c1559e0 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0934684640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c1520 con 0x7f092c1550c0 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0934684640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c14f490 con 0x7f092c0aa180 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0932c21640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c1559e0 0x7f092c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0932c21640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c1559e0 0x7f092c06ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53662/0 (socket says 172.21.15.26:53662) 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0932c21640 1 -- 172.21.15.26:0/2911489979 learned_addr learned my addr 172.21.15.26:0/2911489979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0932420640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c0aa180 0x7f092c15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.055+0000 7f0931c1f640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092c1550c0 0x7f092c14c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0932c21640 1 -- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c0aa180 msgr2=0x7f092c15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0932c21640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c0aa180 0x7f092c15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0932c21640 1 -- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092c1550c0 msgr2=0x7f092c14c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0932c21640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092c1550c0 0x7f092c14c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0932c21640 1 -- 172.21.15.26:0/2911489979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c14e590 con 0x7f092c1559e0 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0932420640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c0aa180 0x7f092c15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0932c21640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c1559e0 0x7f092c06ca70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c440 tx=0x7f091c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f092affd640 1 -- 172.21.15.26:0/2911489979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c017070 con 0x7f092c1559e0 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f092affd640 1 -- 172.21.15.26:0/2911489979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c0035f0 con 0x7f092c1559e0 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f092affd640 1 -- 172.21.15.26:0/2911489979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c0039e0 con 0x7f092c1559e0 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c0bfbe0 con 0x7f092c1559e0 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.056+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c0c01a0 con 0x7f092c1559e0 2024-03-06T07:24:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.057+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c04f330 con 0x7f092c1559e0 2024-03-06T07:24:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.057+0000 7f092affd640 1 -- 172.21.15.26:0/2911489979 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f091c01a070 con 0x7f092c1559e0 2024-03-06T07:24:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.058+0000 7f092affd640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.058+0000 7f092affd640 1 -- 172.21.15.26:0/2911489979 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f091c089530 con 0x7f092c1559e0 2024-03-06T07:24:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.058+0000 7f0932420640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.059+0000 7f0932420640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0924036000 tx=0x7f0924030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.062+0000 7f092affd640 1 -- 172.21.15.26:0/2911489979 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f091c057cd0 con 0x7f092c1559e0 2024-03-06T07:24:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.282+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f092c068450 con 0x7f092c1559e0 2024-03-06T07:24:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.282+0000 7f092affd640 1 -- 172.21.15.26:0/2911489979 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4977 (secure 0 0 0) 0x7f091c05bd70 con 0x7f092c1559e0 2024-03-06T07:24:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:18.284 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/131071104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":131071104},{"type":"v1","addr":"172.21.15.26:6835","nonce":131071104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:16.712967+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 msgr2=0x7f091806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0924036000 tx=0x7f0924030000 comp rx=0 tx=0).stop 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c1559e0 msgr2=0x7f092c06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c1559e0 0x7f092c06ca70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c440 tx=0x7f091c00c900 comp rx=0 tx=0).stop 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 shutdown_connections 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c1559e0 0x7f092c06ca70 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092c1550c0 0x7f092c14c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 --2- 172.21.15.26:0/2911489979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f092c0aa180 0x7f092c15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 >> 172.21.15.26:0/2911489979 conn(0x7f092c0c6710 msgr2=0x7f092c0c3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 shutdown_connections 2024-03-06T07:24:18.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.284+0000 7f0934684640 1 -- 172.21.15.26:0/2911489979 wait complete. 2024-03-06T07:24:18.698 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:04.037603+0000', 'modified': '2024-03-06T07:24:16.712967+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6305}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6305': {'gid': 6305, 'name': 'b', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6305, 'qdb_cluster': [6305]} 2024-03-06T07:24:18.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:18.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.805+0000 7f0650116640 1 Processor -- start 2024-03-06T07:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.805+0000 7f0650116640 1 -- start start 2024-03-06T07:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f0650116640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480c2b20 0x7f06480c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f0650116640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06481587e0 0x7f0648158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f0650116640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06480c5140 con 0x7f06481526c0 2024-03-06T07:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f0650116640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06480c3790 con 0x7f06481587e0 2024-03-06T07:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f0650116640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0648155bb0 con 0x7f06480c2b20 2024-03-06T07:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064e6b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06481587e0 0x7f0648158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064e6b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06481587e0 0x7f0648158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53668/0 (socket says 172.21.15.26:53668) 2024-03-06T07:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064e6b3640 1 -- 172.21.15.26:0/339728832 learned_addr learned my addr 172.21.15.26:0/339728832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064d6b1640 1 --2- 172.21.15.26:0/339728832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480c2b20 0x7f06480c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064e6b3640 1 -- 172.21.15.26:0/339728832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480c2b20 msgr2=0x7f06480c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064e6b3640 1 --2- 172.21.15.26:0/339728832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480c2b20 0x7f06480c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064e6b3640 1 -- 172.21.15.26:0/339728832 >> v1:172.21.15.117:6789/0 conn(0x7f06481526c0 legacy=0x7f06480c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.806+0000 7f064e6b3640 1 -- 172.21.15.26:0/339728832 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648155e90 con 0x7f06481587e0 2024-03-06T07:24:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.807+0000 7f064e6b3640 1 --2- 172.21.15.26:0/339728832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648158c90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0638009e50 tx=0x7f063802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0b2bfa48bfb98e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.807+0000 7f064ceb0640 1 -- 172.21.15.26:0/339728832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063802ca70 con 0x7f06481587e0 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.807+0000 7f064ceb0640 1 -- 172.21.15.26:0/339728832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063802cc10 con 0x7f06481587e0 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.807+0000 7f064ceb0640 1 -- 172.21.15.26:0/339728832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380343f0 con 0x7f06481587e0 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 -- 172.21.15.26:0/339728832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 msgr2=0x7f0648158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 --2- 172.21.15.26:0/339728832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648158c90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0638009e50 tx=0x7f063802bed0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 -- 172.21.15.26:0/339728832 shutdown_connections 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 --2- 172.21.15.26:0/339728832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648158c90 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 --2- 172.21.15.26:0/339728832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480c2b20 0x7f06480c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 -- 172.21.15.26:0/339728832 >> 172.21.15.26:0/339728832 conn(0x7f06480aa810 msgr2=0x7f06480aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 -- 172.21.15.26:0/339728832 shutdown_connections 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.808+0000 7f0650116640 1 -- 172.21.15.26:0/339728832 wait complete. 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.809+0000 7f0650116640 1 Processor -- start 2024-03-06T07:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.809+0000 7f0650116640 1 -- start start 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f0650116640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480c2b20 0x7f064806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f0650116640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06481526c0 0x7f0648153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f0650116640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f0650116640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480c9040 con 0x7f06481587e0 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f0650116640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648152aa0 con 0x7f06480c2b20 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f0650116640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06480c91c0 con 0x7f06481526c0 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53682/0 (socket says 172.21.15.26:53682) 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 -- 172.21.15.26:0/1684219254 learned_addr learned my addr 172.21.15.26:0/1684219254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064deb2640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480c2b20 0x7f064806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064d6b1640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06481526c0 0x7f0648153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 -- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06481526c0 msgr2=0x7f0648153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06481526c0 0x7f0648153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 -- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480c2b20 msgr2=0x7f064806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480c2b20 0x7f064806f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.810+0000 7f064e6b3640 1 -- 172.21.15.26:0/1684219254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06481ee580 con 0x7f06481587e0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.811+0000 7f064e6b3640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648148c50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0638009e50 tx=0x7f0638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.811+0000 7f06467fc640 1 -- 172.21.15.26:0/1684219254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638043020 con 0x7f06481587e0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.811+0000 7f06467fc640 1 -- 172.21.15.26:0/1684219254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063803e450 con 0x7f06481587e0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.811+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06481ee820 con 0x7f06481587e0 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.811+0000 7f06467fc640 1 -- 172.21.15.26:0/1684219254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638009510 con 0x7f06481587e0 2024-03-06T07:24:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.811+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064814d5e0 con 0x7f06481587e0 2024-03-06T07:24:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.812+0000 7f06467fc640 1 -- 172.21.15.26:0/1684219254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0638046020 con 0x7f06481587e0 2024-03-06T07:24:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.813+0000 7f06467fc640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f063406b4d0 0x7f063406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.813+0000 7f06467fc640 1 -- 172.21.15.26:0/1684219254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f06380ac910 con 0x7f06481587e0 2024-03-06T07:24:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.813+0000 7f064deb2640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f063406b4d0 0x7f063406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.813+0000 7f064deb2640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f063406b4d0 0x7f063406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0648155690 tx=0x7f0640009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.814+0000 7f0652dce740 1 Processor -- start 2024-03-06T07:24:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.814+0000 7f0652dce740 1 -- start start 2024-03-06T07:24:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f0652dce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fd330 0x5611a50faf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fb4b0 0x5611a50f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f0652dce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611a50616b0 con 0x5611a50fd330 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611a500aa60 con 0x5611a50fcf50 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611a505d750 con 0x5611a50fb4b0 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fd330 0x5611a50faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fd330 0x5611a50faf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53694/0 (socket says 172.21.15.26:53694) 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 -- 172.21.15.26:0/3410528713 learned_addr learned my addr 172.21.15.26:0/3410528713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064deb2640 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064e6b3640 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fb4b0 0x5611a50f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 -- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fb4b0 msgr2=0x5611a50f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fb4b0 0x5611a50f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 -- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 msgr2=0x5611a50faa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50faa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 -- 172.21.15.26:0/3410528713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611a5028f30 con 0x5611a50fd330 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f064d6b1640 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fd330 0x5611a50faf70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f063c009b90 tx=0x7f063c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a23782b14f6b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f062f7fe640 1 -- 172.21.15.26:0/3410528713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c019070 con 0x5611a50fd330 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f062f7fe640 1 -- 172.21.15.26:0/3410528713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c004030 con 0x5611a50fd330 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.815+0000 7f062f7fe640 1 -- 172.21.15.26:0/3410528713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0043e0 con 0x5611a50fd330 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 -- 172.21.15.26:0/3410528713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fd330 msgr2=0x5611a50faf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fd330 0x5611a50faf70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f063c009b90 tx=0x7f063c011690 comp rx=0 tx=0).stop 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 -- 172.21.15.26:0/3410528713 shutdown_connections 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fb4b0 0x5611a50f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fd330 0x5611a50faf70 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 --2- 172.21.15.26:0/3410528713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 -- 172.21.15.26:0/3410528713 >> 172.21.15.26:0/3410528713 conn(0x5611a50f97d0 msgr2=0x5611a511e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 -- 172.21.15.26:0/3410528713 shutdown_connections 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.816+0000 7f0652dce740 1 -- 172.21.15.26:0/3410528713 wait complete. 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.817+0000 7f0652dce740 1 -- start start 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.817+0000 7f0652dce740 1 Processor -- start 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a5121490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a5149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a5122be0 0x5611a511d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611a511f230 con 0x5611a5122be0 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611a501dea0 con 0x5611a50fcf50 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611a511ec70 con 0x5611a50fd330 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 10 client.0 _ll_register_callbacks cb 0x5611a501d870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a5122be0 0x5611a511d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a5122be0 0x5611a511d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53706/0 (socket says 172.21.15.26:53706) 2024-03-06T07:24:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 -- 172.21.15.26:0/3811699812 learned_addr learned my addr 172.21.15.26:0/3811699812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064deb2640 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a5121490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064d6b1640 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a5149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 -- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 msgr2=0x5611a5149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a5149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 -- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 msgr2=0x5611a5121490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a5121490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 -- 172.21.15.26:0/3811699812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611a501e270 con 0x5611a5122be0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f064e6b3640 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a5122be0 0x5611a511d8b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f063802bf10 tx=0x7f063807aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6b07d511f39687 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f062cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f062cff9640 1 -- 172.21.15.26:0/3811699812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380831c0 con 0x5611a5122be0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f062cff9640 1 -- 172.21.15.26:0/3811699812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0638083360 con 0x5611a5122be0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f063c004030 con 0x5611a5122be0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f062cff9640 1 -- 172.21.15.26:0/3811699812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06380810c0 con 0x5611a5122be0 2024-03-06T07:24:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 10 client.7590 fetch_fsmap learned FSMap version 94 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.818+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5611a4d762d0 con 0x5611a5122be0 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f062cff9640 1 -- 172.21.15.26:0/3811699812 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c004030 con 0x5611a5122be0 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f062cff9640 1 -- 172.21.15.26:0/3811699812 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f063807a440 con 0x5611a5122be0 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f0652dce740 10 client.7590 fetch_fsmap finished waiting for FSMap version 94 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f0652dce740 10 client.7590 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f0652dce740 20 client.7590 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f0652dce740 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a511fd80 0x5611a5158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f0652dce740 4 client.7590 mds_command: new command op to 6305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 --> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611a506ebf0 con 0x5611a511fd80 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f064deb2640 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a511fd80 0x5611a5158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f064deb2640 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a511fd80 0x5611a5158840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b9a62f07b5d796 server_cookie=fa1dd1de0f0716dd in_seq=0 out_seq=0 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.819+0000 7f062cff9640 10 client.7590 ms_handle_connect on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f062cff9640 1 -- 172.21.15.26:0/3811699812 <== mds.0 v2:172.21.15.117:6834/548706796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5611a506ebf0 con 0x5611a511fd80 2024-03-06T07:24:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f062cff9640 10 client.7590 handle_command_reply: tid=0 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 client.7590 shutdown 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a5122be0 msgr2=0x5611a511d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a5122be0 0x5611a511d8b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f063802bf10 tx=0x7f063807aa10 comp rx=0 tx=0).stop 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 shutdown_connections 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a511fd80 0x5611a5158840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a5122be0 0x5611a511d8b0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a5149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f062cff9640 0 client.7590 ms_handle_reset on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 --2- 172.21.15.26:0/3811699812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a5121490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 >> 172.21.15.26:0/3811699812 conn(0x5611a50fb4b0 msgr2=0x5611a50fb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 shutdown_connections 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 1 -- 172.21.15.26:0/3811699812 wait complete. 2024-03-06T07:24:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.820+0000 7f0652dce740 20 client.7590 trim_cache size 0 max 16384 2024-03-06T07:24:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.837+0000 7f0652dce740 1 Processor -- start 2024-03-06T07:24:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.837+0000 7f0652dce740 1 -- start start 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a50fe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f0652dce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a511fd80 0x5611a514a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f0652dce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481f7050 con 0x5611a511fd80 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481f6450 con 0x5611a50fcf50 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06481f6a50 con 0x5611a50fd330 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a511fd80 0x5611a514a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a511fd80 0x5611a514a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53710/0 (socket says 172.21.15.26:53710) 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 -- 172.21.15.26:0/623181757 learned_addr learned my addr 172.21.15.26:0/623181757 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064deb2640 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064d6b1640 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a50fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 -- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 msgr2=0x5611a50fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a50fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 -- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 msgr2=0x5611a50fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 -- 172.21.15.26:0/623181757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611a4e406b0 con 0x5611a511fd80 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f064e6b3640 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a511fd80 0x5611a514a400 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f0638077970 tx=0x7f0638081050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d8a9991b6badd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f062effd640 1 -- 172.21.15.26:0/623181757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638099020 con 0x5611a511fd80 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f062effd640 1 -- 172.21.15.26:0/623181757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063807a440 con 0x5611a511fd80 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.838+0000 7f062effd640 1 -- 172.21.15.26:0/623181757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380831c0 con 0x5611a511fd80 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 -- 172.21.15.26:0/623181757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a511fd80 msgr2=0x5611a514a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a511fd80 0x5611a514a400 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f0638077970 tx=0x7f0638081050 comp rx=0 tx=0).stop 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 -- 172.21.15.26:0/623181757 shutdown_connections 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a511fd80 0x5611a514a400 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a50fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 --2- 172.21.15.26:0/623181757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a50fcf50 0x5611a50fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 -- 172.21.15.26:0/623181757 >> 172.21.15.26:0/623181757 conn(0x5611a50fb4b0 msgr2=0x5611a5146c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 -- 172.21.15.26:0/623181757 shutdown_connections 2024-03-06T07:24:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 -- 172.21.15.26:0/623181757 wait complete. 2024-03-06T07:24:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 -- start start 2024-03-06T07:24:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.839+0000 7f0652dce740 1 Processor -- start 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f0652dce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fcf50 0x5611a515d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a51a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f0652dce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a511fd80 0x5611a51a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f0652dce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611a511f230 con 0x5611a50fcf50 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611a511fc00 con 0x5611a511fd80 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f0652dce740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611a5125240 con 0x5611a50fd330 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f0652dce740 10 client.0 _ll_register_callbacks cb 0x7f063807ae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f064deb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fcf50 0x5611a515d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f064deb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fcf50 0x5611a515d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53714/0 (socket says 172.21.15.26:53714) 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.840+0000 7f064deb2640 1 -- 172.21.15.26:0/1999971980 learned_addr learned my addr 172.21.15.26:0/1999971980 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064d6b1640 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a51a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064e6b3640 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a511fd80 0x5611a51a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064deb2640 1 -- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 msgr2=0x5611a51a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064deb2640 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a51a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064deb2640 1 -- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a511fd80 msgr2=0x5611a51a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064deb2640 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a511fd80 0x5611a51a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064deb2640 1 -- 172.21.15.26:0/1999971980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611a50fe210 con 0x5611a50fcf50 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f064deb2640 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fcf50 0x5611a515d040 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f064000fef0 tx=0x7f0640012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00535a00bf521fe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f062cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f062cff9640 1 -- 172.21.15.26:0/1999971980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640023450 con 0x5611a50fcf50 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f062cff9640 1 -- 172.21.15.26:0/1999971980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f06400235f0 con 0x5611a50fcf50 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f063807a440 con 0x5611a50fcf50 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f062cff9640 1 -- 172.21.15.26:0/1999971980 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0640039050 con 0x5611a50fcf50 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f0652dce740 10 client.7596 fetch_fsmap learned FSMap version 94 2024-03-06T07:24:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5611a514d2c0 con 0x5611a50fcf50 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f062cff9640 1 -- 172.21.15.26:0/1999971980 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063807a440 con 0x5611a50fcf50 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f062cff9640 1 -- 172.21.15.26:0/1999971980 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f064003c020 con 0x5611a50fcf50 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.841+0000 7f0652dce740 10 client.7596 fetch_fsmap finished waiting for FSMap version 94 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 10 client.7596 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 20 client.7596 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a5122be0 0x5611a51ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 4 client.7596 mds_command: new command op to 6305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 --> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5611a5068f40 con 0x5611a5122be0 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f064d6b1640 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a5122be0 0x5611a51ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f062cff9640 10 client.7596 ms_handle_connect on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f064d6b1640 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a5122be0 0x5611a51ad4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254ceecaa66f4f98 server_cookie=22e989a498f19e29 in_seq=0 out_seq=0 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f062cff9640 1 -- 172.21.15.26:0/1999971980 <== mds.0 v2:172.21.15.117:6834/548706796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5611a5068f40 con 0x5611a5122be0 2024-03-06T07:24:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f062cff9640 10 client.7596 handle_command_reply: tid=0 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 client.7596 shutdown 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fcf50 msgr2=0x5611a515d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fcf50 0x5611a515d040 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f064000fef0 tx=0x7f0640012000 comp rx=0 tx=0).stop 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 shutdown_connections 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5611a5122be0 0x5611a51ad4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611a511fd80 0x5611a51a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f062cff9640 0 client.7596 ms_handle_reset on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611a50fd330 0x5611a51a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 --2- 172.21.15.26:0/1999971980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611a50fcf50 0x5611a515d040 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.842+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 >> 172.21.15.26:0/1999971980 conn(0x5611a50fb4b0 msgr2=0x5611a514a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 shutdown_connections 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0652dce740 1 -- 172.21.15.26:0/1999971980 wait complete. 2024-03-06T07:24:18.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0652dce740 20 client.7596 trim_cache size 0 max 16384 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6305, 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.1353055620000001, 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 94, 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 109, 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 109, 2024-03-06T07:24:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 25.256164390999999 2024-03-06T07:24:18.852 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:18.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f063406b4d0 msgr2=0x7f063406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f063406b4d0 0x7f063406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0648155690 tx=0x7f0640009040 comp rx=0 tx=0).stop 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 msgr2=0x7f0648148c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648148c50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0638009e50 tx=0x7f0638004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 shutdown_connections 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f063406b4d0 0x7f063406d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06481587e0 0x7f0648148c50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06481526c0 0x7f0648153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 --2- 172.21.15.26:0/1684219254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480c2b20 0x7f064806f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 >> 172.21.15.26:0/1684219254 conn(0x7f06480aa810 msgr2=0x7f06480a9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 shutdown_connections 2024-03-06T07:24:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.843+0000 7f0650116640 1 -- 172.21.15.26:0/1684219254 wait complete. 2024-03-06T07:24:18.855 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 94, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:04.037603+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 94, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6305": { "addr": "172.21.15.117:6835/548706796", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 548706796, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 548706796, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6305, "incarnation": 91, "join_fscid": 8, "name": "b", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 109, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:16.712967+0000", "qdb_cluster": [ 6305 ], "qdb_leader": 6305, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6305 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/3362094815", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3362094815, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3362094815, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7084, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/256924791", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 256924791, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 256924791, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 85, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7365, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/131071104", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 131071104, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 131071104, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 92, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7566, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:18.856 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 8, 'name': 'b', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:18.856 INFO:tasks.cephfs.test_failover:current = [{'gid': 7566, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/131071104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 131071104}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 131071104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 92}, {'gid': 6305, 'name': 'b', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7365, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/256924791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 256924791}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 256924791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 85}, {'gid': 7084, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3362094815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3362094815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3362094815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}] 2024-03-06T07:24:18.856 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:24:18.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-06T07:24:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.960+0000 7f381359e640 1 Processor -- start 2024-03-06T07:24:18.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.960+0000 7f381359e640 1 -- start start 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f381359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f381406c270 0x7f3814155d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f381359e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38140c5910 0x7f381406c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f381359e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140c5130 con 0x7f381406bbf0 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f381359e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381406ce90 con 0x7f381406c270 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f381359e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140c53d0 con 0x7f38140c5910 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f381259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f381406c270 0x7f3814155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f3812d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38140c5910 0x7f381406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f380bfff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f381406bbf0 0x7f38140c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60830/0 (socket says 172.21.15.26:60830) 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f380bfff640 1 -- 172.21.15.26:0/1864528497 learned_addr learned my addr 172.21.15.26:0/1864528497 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f381259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f381406c270 0x7f3814155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53720/0 (socket says 172.21.15.26:53720) 2024-03-06T07:24:18.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f3812d9d640 1 -- 172.21.15.26:0/1864528497 >> v2:172.21.15.26:3300/0 conn(0x7f381406c270 msgr2=0x7f3814155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f3812d9d640 1 --2- 172.21.15.26:0/1864528497 >> v2:172.21.15.26:3300/0 conn(0x7f381406c270 0x7f3814155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f3812d9d640 1 -- 172.21.15.26:0/1864528497 >> v1:172.21.15.117:6789/0 conn(0x7f381406bbf0 legacy=0x7f38140c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f3811d9b640 1 -- 172.21.15.26:0/1864528497 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886984033 0 0) 0x7f38140c5130 con 0x7f381406bbf0 2024-03-06T07:24:18.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.961+0000 7f3812d9d640 1 -- 172.21.15.26:0/1864528497 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140c56b0 con 0x7f38140c5910 2024-03-06T07:24:18.964 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:24:18.961+0000 7f38f6dbd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:18.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f3812d9d640 1 --2- 172.21.15.26:0/1864528497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38140c5910 0x7f381406c650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f380c009d90 tx=0x7f380c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9905deaebca85418 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f3811d9b640 1 -- 172.21.15.26:0/1864528497 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c039070 con 0x7f38140c5910 2024-03-06T07:24:18.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f3811d9b640 1 -- 172.21.15.26:0/1864528497 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c02cc50 con 0x7f38140c5910 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f3811d9b640 1 -- 172.21.15.26:0/1864528497 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c034440 con 0x7f38140c5910 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f381359e640 1 -- 172.21.15.26:0/1864528497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38140c5910 msgr2=0x7f381406c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f381359e640 1 --2- 172.21.15.26:0/1864528497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38140c5910 0x7f381406c650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f380c009d90 tx=0x7f380c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f381359e640 1 -- 172.21.15.26:0/1864528497 shutdown_connections 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f381359e640 1 --2- 172.21.15.26:0/1864528497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38140c5910 0x7f381406c650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f381359e640 1 --2- 172.21.15.26:0/1864528497 >> v2:172.21.15.26:3300/0 conn(0x7f381406c270 0x7f3814155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.962+0000 7f381359e640 1 -- 172.21.15.26:0/1864528497 >> 172.21.15.26:0/1864528497 conn(0x7f38140ca7b0 msgr2=0x7f38140cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.963+0000 7f381359e640 1 -- 172.21.15.26:0/1864528497 shutdown_connections 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.963+0000 7f381359e640 1 -- 172.21.15.26:0/1864528497 wait complete. 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 Processor -- start 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 -- start start 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381406bbf0 0x7f381414c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f381406c270 0x7f381414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38140c5910 0x7f381414df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141557f0 con 0x7f381406c270 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814155510 con 0x7f38140c5910 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f381359e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140c8dc0 con 0x7f381406bbf0 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.964+0000 7f380bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f381406c270 0x7f381414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f381406c270 0x7f381414b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53726/0 (socket says 172.21.15.26:53726) 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 -- 172.21.15.26:0/3115333233 learned_addr learned my addr 172.21.15.26:0/3115333233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f381259c640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381406bbf0 0x7f381414c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f3812d9d640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38140c5910 0x7f381414df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 -- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381406bbf0 msgr2=0x7f381414c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381406bbf0 0x7f381414c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 -- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38140c5910 msgr2=0x7f381414df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38140c5910 0x7f381414df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 -- 172.21.15.26:0/3115333233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38141eba30 con 0x7f381406c270 2024-03-06T07:24:18.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380bfff640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f381406c270 0x7f381414b540 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00c410 tx=0x7f37fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380affd640 1 -- 172.21.15.26:0/3115333233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0074e0 con 0x7f381406c270 2024-03-06T07:24:18.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380affd640 1 -- 172.21.15.26:0/3115333233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc0047e0 con 0x7f381406c270 2024-03-06T07:24:18.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38140c3ba0 con 0x7f381406c270 2024-03-06T07:24:18.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f380affd640 1 -- 172.21.15.26:0/3115333233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc004bd0 con 0x7f381406c270 2024-03-06T07:24:18.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.965+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38140c4160 con 0x7f381406c270 2024-03-06T07:24:18.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.966+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38140ca1b0 con 0x7f381406c270 2024-03-06T07:24:18.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.967+0000 7f380affd640 1 -- 172.21.15.26:0/3115333233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37fc011070 con 0x7f381406c270 2024-03-06T07:24:18.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.967+0000 7f380affd640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37f406b4d0 0x7f37f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.967+0000 7f380affd640 1 -- 172.21.15.26:0/3115333233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f37fc089e70 con 0x7f381406c270 2024-03-06T07:24:18.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.967+0000 7f381259c640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37f406b4d0 0x7f37f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.968+0000 7f381259c640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37f406b4d0 0x7f37f406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3804009aa0 tx=0x7f3804009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:18.971+0000 7f380affd640 1 -- 172.21.15.26:0/3115333233 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f38140ca1b0 con 0x7f381406c270 2024-03-06T07:24:19.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.189+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f38140693c0 con 0x7f381406c270 2024-03-06T07:24:19.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.622+0000 7f380affd640 1 -- 172.21.15.26:0/3115333233 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v95) v1 ==== 121+0+0 (secure 0 0 0) 0x7f37fc0584b0 con 0x7f381406c270 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37f406b4d0 msgr2=0x7f37f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37f406b4d0 0x7f37f406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3804009aa0 tx=0x7f3804009190 comp rx=0 tx=0).stop 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f381406c270 msgr2=0x7f381414b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f381406c270 0x7f381414b540 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00c410 tx=0x7f37fc00c8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 shutdown_connections 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f37f406b4d0 0x7f37f406d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38140c5910 0x7f381414df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f381406c270 0x7f381414b540 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 --2- 172.21.15.26:0/3115333233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381406bbf0 0x7f381414c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 >> 172.21.15.26:0/3115333233 conn(0x7f38140ca7b0 msgr2=0x7f38140c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:19.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 shutdown_connections 2024-03-06T07:24:19.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.624+0000 7f381359e640 1 -- 172.21.15.26:0/3115333233 wait complete. 2024-03-06T07:24:19.640 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.745+0000 7fade8f0c640 1 Processor -- start 2024-03-06T07:24:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.745+0000 7fade8f0c640 1 -- start start 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.745+0000 7fade8f0c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fade4150710 0x7fade40c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade8f0c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade4156f90 con 0x7fade4150290 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade8f0c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade4157270 con 0x7fade4072720 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade8f0c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade4150b60 con 0x7fade4150710 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade1d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fade4150290 0x7fade40c3d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59840/0 (socket says 172.21.15.26:59840) 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade1d9b640 1 -- 172.21.15.26:0/3685247100 learned_addr learned my addr 172.21.15.26:0/3685247100 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade2d9d640 1 --2- 172.21.15.26:0/3685247100 >> v2:172.21.15.117:3300/0 conn(0x7fade4150710 0x7fade40c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade159a640 1 -- 172.21.15.26:0/3685247100 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495538165 0 0) 0x7fade4156f90 con 0x7fade4150290 2024-03-06T07:24:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade159a640 1 -- 172.21.15.26:0/3685247100 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc003520 con 0x7fade4150290 2024-03-06T07:24:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade159a640 1 -- 172.21.15.26:0/3685247100 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346500474 0 0) 0x7fade4157270 con 0x7fade4072720 2024-03-06T07:24:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade159a640 1 -- 172.21.15.26:0/3685247100 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade4156f90 con 0x7fade4072720 2024-03-06T07:24:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade2d9d640 1 -- 172.21.15.26:0/3685247100 >> v1:172.21.15.117:6789/0 conn(0x7fade4072720 legacy=0x7fade41564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade2d9d640 1 -- 172.21.15.26:0/3685247100 >> v1:172.21.15.26:6789/0 conn(0x7fade4150290 legacy=0x7fade40c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.746+0000 7fade2d9d640 1 -- 172.21.15.26:0/3685247100 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40c3280 con 0x7fade4150710 2024-03-06T07:24:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.747+0000 7fade2d9d640 1 --2- 172.21.15.26:0/3685247100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150710 0x7fade40c4470 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faddc009d90 tx=0x7faddc02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d680cd396a432ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.747+0000 7fade159a640 1 -- 172.21.15.26:0/3685247100 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc039070 con 0x7fade4150710 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.747+0000 7fade159a640 1 -- 172.21.15.26:0/3685247100 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc02cc60 con 0x7fade4150710 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.747+0000 7fade159a640 1 -- 172.21.15.26:0/3685247100 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc034660 con 0x7fade4150710 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.748+0000 7fade8f0c640 1 -- 172.21.15.26:0/3685247100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150710 msgr2=0x7fade40c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.748+0000 7fade8f0c640 1 --2- 172.21.15.26:0/3685247100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150710 0x7fade40c4470 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faddc009d90 tx=0x7faddc02beb0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.748+0000 7fade8f0c640 1 -- 172.21.15.26:0/3685247100 shutdown_connections 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.748+0000 7fade8f0c640 1 --2- 172.21.15.26:0/3685247100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150710 0x7fade40c4470 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.748+0000 7fade8f0c640 1 -- 172.21.15.26:0/3685247100 >> 172.21.15.26:0/3685247100 conn(0x7fade40aa630 msgr2=0x7fade40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.748+0000 7fade8f0c640 1 -- 172.21.15.26:0/3685247100 shutdown_connections 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.748+0000 7fade8f0c640 1 -- 172.21.15.26:0/3685247100 wait complete. 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.749+0000 7fade8f0c640 1 Processor -- start 2024-03-06T07:24:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.749+0000 7fade8f0c640 1 -- start start 2024-03-06T07:24:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade8f0c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fade4072720 0x7fade40ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade8f0c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150290 0x7fade40ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade8f0c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4150710 0x7fade41ec130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade8f0c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c49b0 con 0x7fade4150710 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade8f0c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c38d0 con 0x7fade4150290 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade8f0c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade406f8f0 con 0x7fade4072720 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4150710 0x7fade41ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4150710 0x7fade41ec130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53738/0 (socket says 172.21.15.26:53738) 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 -- 172.21.15.26:0/2211332638 learned_addr learned my addr 172.21.15.26:0/2211332638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade259c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fade4072720 0x7fade40ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade1d9b640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150290 0x7fade40ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 -- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fade4072720 msgr2=0x7fade40ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fade4072720 0x7fade40ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 -- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150290 msgr2=0x7fade40ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150290 0x7fade40ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 -- 172.21.15.26:0/2211332638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4145ca0 con 0x7fade4150710 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade2d9d640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4150710 0x7fade41ec130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7faddc009d90 tx=0x7faddc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fadd2ffd640 1 -- 172.21.15.26:0/2211332638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc039050 con 0x7fade4150710 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fadd2ffd640 1 -- 172.21.15.26:0/2211332638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc045070 con 0x7fade4150710 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.750+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade4145800 con 0x7fade4150710 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.751+0000 7fadd2ffd640 1 -- 172.21.15.26:0/2211332638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc009450 con 0x7fade4150710 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.751+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40ac8a0 con 0x7fade4150710 2024-03-06T07:24:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.752+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade404f330 con 0x7fade4150710 2024-03-06T07:24:19.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.752+0000 7fadd2ffd640 1 -- 172.21.15.26:0/2211332638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faddc03e850 con 0x7fade4150710 2024-03-06T07:24:19.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.752+0000 7fadd2ffd640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadc406b4d0 0x7fadc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.753+0000 7fadd2ffd640 1 -- 172.21.15.26:0/2211332638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faddc0ad170 con 0x7fade4150710 2024-03-06T07:24:19.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.753+0000 7fade259c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadc406b4d0 0x7fadc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.753+0000 7fade259c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadc406b4d0 0x7fadc406d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fadd4007fd0 tx=0x7fadd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.757+0000 7fadd2ffd640 1 -- 172.21.15.26:0/2211332638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faddc07b970 con 0x7fade4150710 2024-03-06T07:24:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.977+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fade41459d0 con 0x7fade4150710 2024-03-06T07:24:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.978+0000 7fadd2ffd640 1 -- 172.21.15.26:0/2211332638 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4984 (secure 0 0 0) 0x7faddc07fa10 con 0x7fade4150710 2024-03-06T07:24:19.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:19.980 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":95,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:19.621541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7566":{"gid":7566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/131071104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":131071104},{"type":"v1","addr":"172.21.15.26:6835","nonce":131071104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:19.980 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 95 2024-03-06T07:24:19.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadc406b4d0 msgr2=0x7fadc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadc406b4d0 0x7fadc406d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fadd4007fd0 tx=0x7fadd4009040 comp rx=0 tx=0).stop 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4150710 msgr2=0x7fade41ec130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4150710 0x7fade41ec130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7faddc009d90 tx=0x7faddc004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 shutdown_connections 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadc406b4d0 0x7fadc406d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4150710 0x7fade41ec130 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4150290 0x7fade40ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 --2- 172.21.15.26:0/2211332638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fade4072720 0x7fade40ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 >> 172.21.15.26:0/2211332638 conn(0x7fade40aa630 msgr2=0x7fade40a9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 shutdown_connections 2024-03-06T07:24:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:19.980+0000 7fade8f0c640 1 -- 172.21.15.26:0/2211332638 wait complete. 2024-03-06T07:24:19.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs 2024-03-06T07:24:20.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.095+0000 7f747c565640 1 Processor -- start 2024-03-06T07:24:20.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.095+0000 7f747c565640 1 -- start start 2024-03-06T07:24:20.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747c565640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740af720 0x7f747414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747c565640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74740ca1a0 0x7f747414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747c565640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7474155d20 con 0x7f74740c5310 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747c565640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74740ad900 con 0x7f74740ca1a0 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747c565640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7474155fc0 con 0x7f74740af720 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747ab02640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f74740c5310 0x7f74740c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59848/0 (socket says 172.21.15.26:59848) 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747ab02640 1 -- 172.21.15.26:0/2016410532 learned_addr learned my addr 172.21.15.26:0/2016410532 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f747a301640 1 --2- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740af720 0x7f747414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.096+0000 7f7479b00640 1 --2- 172.21.15.26:0/2016410532 >> v2:172.21.15.117:3300/0 conn(0x7f74740ca1a0 0x7f747414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.097+0000 7f74792ff640 1 -- 172.21.15.26:0/2016410532 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823080477 0 0) 0x7f7474155d20 con 0x7f74740c5310 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.097+0000 7f74792ff640 1 -- 172.21.15.26:0/2016410532 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7460003520 con 0x7f74740c5310 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.097+0000 7f7479b00640 1 -- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740af720 msgr2=0x7f747414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.097+0000 7f7479b00640 1 --2- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740af720 0x7f747414ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.097+0000 7f7479b00640 1 -- 172.21.15.26:0/2016410532 >> v1:172.21.15.26:6789/0 conn(0x7f74740c5310 legacy=0x7f74740c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.097+0000 7f7479b00640 1 -- 172.21.15.26:0/2016410532 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740adb50 con 0x7f74740ca1a0 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.097+0000 7f7479b00640 1 --2- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f747414f2d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f74740ada80 tx=0x7f746402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=346956310d8eb639 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f74792ff640 1 -- 172.21.15.26:0/2016410532 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464039070 con 0x7f74740ca1a0 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f74792ff640 1 -- 172.21.15.26:0/2016410532 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74640316f0 con 0x7f74740ca1a0 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f74792ff640 1 -- 172.21.15.26:0/2016410532 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464031ac0 con 0x7f74740ca1a0 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 -- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 msgr2=0x7f747414f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 --2- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f747414f2d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f74740ada80 tx=0x7f746402b790 comp rx=0 tx=0).stop 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 -- 172.21.15.26:0/2016410532 shutdown_connections 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 --2- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f747414f2d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 --2- 172.21.15.26:0/2016410532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740af720 0x7f747414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 -- 172.21.15.26:0/2016410532 >> 172.21.15.26:0/2016410532 conn(0x7f74740ac7e0 msgr2=0x7f74740acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 -- 172.21.15.26:0/2016410532 shutdown_connections 2024-03-06T07:24:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.098+0000 7f747c565640 1 -- 172.21.15.26:0/2016410532 wait complete. 2024-03-06T07:24:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.099+0000 7f747c565640 1 Processor -- start 2024-03-06T07:24:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.099+0000 7f747c565640 1 -- start start 2024-03-06T07:24:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747c565640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74740af720 0x7f74740c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747c565640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740c5310 0x7f74741ee750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747c565640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f7474156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747c565640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740adfb0 con 0x7f74740af720 2024-03-06T07:24:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747c565640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740ade30 con 0x7f74740ca1a0 2024-03-06T07:24:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747c565640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74740afb00 con 0x7f74740c5310 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74740af720 0x7f74740c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74740af720 0x7f74740c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53754/0 (socket says 172.21.15.26:53754) 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 -- 172.21.15.26:0/3097750307 learned_addr learned my addr 172.21.15.26:0/3097750307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f7479b00640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740c5310 0x7f74741ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747ab02640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f7474156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 -- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740c5310 msgr2=0x7f74741ee750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740c5310 0x7f74741ee750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 -- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 msgr2=0x7f7474156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f7474156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747a301640 1 -- 172.21.15.26:0/3097750307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74741ef020 con 0x7f74740af720 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.100+0000 7f747ab02640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f7474156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.101+0000 7f747a301640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74740af720 0x7f74740c59b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f747000e870 tx=0x7f747000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.101+0000 7f746affd640 1 -- 172.21.15.26:0/3097750307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470017070 con 0x7f74740af720 2024-03-06T07:24:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.101+0000 7f746affd640 1 -- 172.21.15.26:0/3097750307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747000c940 con 0x7f74740af720 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.101+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74741ef300 con 0x7f74740af720 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.101+0000 7f746affd640 1 -- 172.21.15.26:0/3097750307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747000cd30 con 0x7f74740af720 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.101+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74741eec90 con 0x7f74740af720 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.102+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747404f330 con 0x7f74740af720 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.102+0000 7f746affd640 1 -- 172.21.15.26:0/3097750307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f747001a070 con 0x7f74740af720 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.103+0000 7f746affd640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f745806b4d0 0x7f745806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.103+0000 7f746affd640 1 -- 172.21.15.26:0/3097750307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f747008a4c0 con 0x7f74740af720 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.103+0000 7f7479b00640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f745806b4d0 0x7f745806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.103+0000 7f7479b00640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f745806b4d0 0x7f745806d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7464036000 tx=0x7f7464008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.107+0000 7f746affd640 1 -- 172.21.15.26:0/3097750307 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7470058ba0 con 0x7f74740af720 2024-03-06T07:24:20.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.287+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f7474074bc0 con 0x7f74740af720 2024-03-06T07:24:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.290+0000 7f746affd640 1 -- 172.21.15.26:0/3097750307 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v37)=0 v37) v1 ==== 116+0+0 (secure 0 0 0) 0x7f747005cc40 con 0x7f74740af720 2024-03-06T07:24:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f745806b4d0 msgr2=0x7f745806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f745806b4d0 0x7f745806d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7464036000 tx=0x7f7464008030 comp rx=0 tx=0).stop 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74740af720 msgr2=0x7f74740c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74740af720 0x7f74740c59b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f747000e870 tx=0x7f747000ed30 comp rx=0 tx=0).stop 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 shutdown_connections 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f745806b4d0 0x7f745806d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74740ca1a0 0x7f7474156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74740c5310 0x7f74741ee750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 --2- 172.21.15.26:0/3097750307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74740af720 0x7f74740c59b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 >> 172.21.15.26:0/3097750307 conn(0x7f74740ac7e0 msgr2=0x7f74740c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 shutdown_connections 2024-03-06T07:24:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.292+0000 7f747c565640 1 -- 172.21.15.26:0/3097750307 wait complete. 2024-03-06T07:24:20.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.403+0000 7f6b55ce1640 1 Processor -- start 2024-03-06T07:24:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.403+0000 7f6b55ce1640 1 -- start start 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.404+0000 7f6b55ce1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b500c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.404+0000 7f6b55ce1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b501591c0 con 0x7f6b501586f0 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.404+0000 7f6b55ce1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b501594a0 con 0x7f6b500c5480 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.404+0000 7f6b55ce1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5006df40 con 0x7f6b50155ba0 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.404+0000 7f6b4ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b501586f0 0x7f6b5015f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59852/0 (socket says 172.21.15.26:59852) 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.404+0000 7f6b4ffff640 1 -- 172.21.15.26:0/594564041 learned_addr learned my addr 172.21.15.26:0/594564041 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.404+0000 7f6b4effd640 1 --2- 172.21.15.26:0/594564041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b500c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4e7fc640 1 -- 172.21.15.26:0/594564041 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299377286 0 0) 0x7f6b501591c0 con 0x7f6b501586f0 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4e7fc640 1 -- 172.21.15.26:0/594564041 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b38003520 con 0x7f6b501586f0 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4e7fc640 1 -- 172.21.15.26:0/594564041 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207723954 0 0) 0x7f6b501594a0 con 0x7f6b500c5480 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4e7fc640 1 -- 172.21.15.26:0/594564041 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b501591c0 con 0x7f6b500c5480 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4effd640 1 -- 172.21.15.26:0/594564041 >> v1:172.21.15.117:6789/0 conn(0x7f6b500c5480 legacy=0x7f6b5006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4effd640 1 -- 172.21.15.26:0/594564041 >> v1:172.21.15.26:6789/0 conn(0x7f6b501586f0 legacy=0x7f6b5015f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4effd640 1 -- 172.21.15.26:0/594564041 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b501553a0 con 0x7f6b50155ba0 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.405+0000 7f6b4effd640 1 --2- 172.21.15.26:0/594564041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b500c5860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b440097d0 tx=0x7f6b4402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9db04d68ccc9975 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b4e7fc640 1 -- 172.21.15.26:0/594564041 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44035020 con 0x7f6b50155ba0 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b4e7fc640 1 -- 172.21.15.26:0/594564041 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b44031800 con 0x7f6b50155ba0 2024-03-06T07:24:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b4e7fc640 1 -- 172.21.15.26:0/594564041 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44031bd0 con 0x7f6b50155ba0 2024-03-06T07:24:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b55ce1640 1 -- 172.21.15.26:0/594564041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 msgr2=0x7f6b500c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/594564041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b500c5860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b440097d0 tx=0x7f6b4402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b55ce1640 1 -- 172.21.15.26:0/594564041 shutdown_connections 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/594564041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b500c5860 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b55ce1640 1 -- 172.21.15.26:0/594564041 >> 172.21.15.26:0/594564041 conn(0x7f6b5006b1a0 msgr2=0x7f6b5006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b55ce1640 1 -- 172.21.15.26:0/594564041 shutdown_connections 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.406+0000 7f6b55ce1640 1 -- 172.21.15.26:0/594564041 wait complete. 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.407+0000 7f6b55ce1640 1 Processor -- start 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.407+0000 7f6b55ce1640 1 -- start start 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b55ce1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b500c5480 0x7f6b50070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b55ce1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b50156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b55ce1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b501586f0 0x7f6b501ee100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b55ce1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50156000 con 0x7f6b501586f0 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b55ce1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5006fdb0 con 0x7f6b500c5480 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b55ce1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50152a20 con 0x7f6b50155ba0 2024-03-06T07:24:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b501586f0 0x7f6b501ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b501586f0 0x7f6b501ee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53766/0 (socket says 172.21.15.26:53766) 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4ffff640 1 -- 172.21.15.26:0/4267821683 learned_addr learned my addr 172.21.15.26:0/4267821683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4f7fe640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b500c5480 0x7f6b50070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4effd640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b50156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4ffff640 1 -- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 msgr2=0x7f6b50156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4ffff640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b50156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.408+0000 7f6b4ffff640 1 -- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b500c5480 msgr2=0x7f6b50070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b4ffff640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b500c5480 0x7f6b50070200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b4ffff640 1 -- 172.21.15.26:0/4267821683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50070740 con 0x7f6b501586f0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b4ffff640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b501586f0 0x7f6b501ee100 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c9f0 tx=0x7f6b4800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b37fff640 1 -- 172.21.15.26:0/4267821683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48017070 con 0x7f6b501586f0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b37fff640 1 -- 172.21.15.26:0/4267821683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4800ab70 con 0x7f6b501586f0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b500ab2e0 con 0x7f6b501586f0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b37fff640 1 -- 172.21.15.26:0/4267821683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48012400 con 0x7f6b501586f0 2024-03-06T07:24:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.409+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b500ab000 con 0x7f6b501586f0 2024-03-06T07:24:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.410+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5004f330 con 0x7f6b501586f0 2024-03-06T07:24:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.410+0000 7f6b37fff640 1 -- 172.21.15.26:0/4267821683 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b4801a070 con 0x7f6b501586f0 2024-03-06T07:24:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.411+0000 7f6b37fff640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b3c06b4d0 0x7f6b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.411+0000 7f6b37fff640 1 -- 172.21.15.26:0/4267821683 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6b48089ec0 con 0x7f6b501586f0 2024-03-06T07:24:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.411+0000 7f6b4f7fe640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b3c06b4d0 0x7f6b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.411+0000 7f6b4f7fe640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b3c06b4d0 0x7f6b3c06d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6b40007fd0 tx=0x7f6b40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.415+0000 7f6b37fff640 1 -- 172.21.15.26:0/4267821683 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6b480585a0 con 0x7f6b501586f0 2024-03-06T07:24:20.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.633+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b500c9380 con 0x7f6b501586f0 2024-03-06T07:24:20.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.633+0000 7f6b37fff640 1 -- 172.21.15.26:0/4267821683 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4984 (secure 0 0 0) 0x7f6b4805c640 con 0x7f6b501586f0 2024-03-06T07:24:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:20.635 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":95,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:19.621541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7566":{"gid":7566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/131071104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":131071104},{"type":"v1","addr":"172.21.15.26:6835","nonce":131071104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:20.636 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 95 2024-03-06T07:24:20.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b3c06b4d0 msgr2=0x7f6b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b3c06b4d0 0x7f6b3c06d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6b40007fd0 tx=0x7f6b40009040 comp rx=0 tx=0).stop 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b501586f0 msgr2=0x7f6b501ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b501586f0 0x7f6b501ee100 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c9f0 tx=0x7f6b4800ceb0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 shutdown_connections 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6b3c06b4d0 0x7f6b3c06d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b501586f0 0x7f6b501ee100 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b50155ba0 0x7f6b50156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 --2- 172.21.15.26:0/4267821683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b500c5480 0x7f6b50070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 >> 172.21.15.26:0/4267821683 conn(0x7f6b5006b1a0 msgr2=0x7f6b5006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 shutdown_connections 2024-03-06T07:24:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:20.636+0000 7f6b55ce1640 1 -- 172.21.15.26:0/4267821683 wait complete. 2024-03-06T07:24:21.051 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-06T07:24:04.037603+0000', 'modified': '2024-03-06T07:24:19.621541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6305}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6305': {'gid': 6305, 'name': 'b', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7566': {'gid': 7566, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/131071104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 131071104}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 131071104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6305, 'qdb_cluster': [6305]} 2024-03-06T07:24:21.051 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:21.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:24:21.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.156+0000 7f696b69d640 1 Processor -- start 2024-03-06T07:24:21.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.156+0000 7f696b69d640 1 -- start start 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f696b69d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f69640ab860 0x7f69640c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f696b69d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69641553c0 con 0x7f696406c240 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f696b69d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69641556a0 con 0x7f69640ac2b0 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f696b69d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6964155980 con 0x7f69640ab860 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f6969439640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f696406c240 0x7f69640ab150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59860/0 (socket says 172.21.15.26:59860) 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f6969439640 1 -- 172.21.15.26:0/3928552375 learned_addr learned my addr 172.21.15.26:0/3928552375 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f6968c38640 1 --2- 172.21.15.26:0/3928552375 >> v2:172.21.15.117:3300/0 conn(0x7f69640ab860 0x7f69640c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f695bfff640 1 -- 172.21.15.26:0/3928552375 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996394548 0 0) 0x7f69641553c0 con 0x7f696406c240 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f695bfff640 1 -- 172.21.15.26:0/3928552375 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6950003520 con 0x7f696406c240 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f6968c38640 1 -- 172.21.15.26:0/3928552375 >> v1:172.21.15.117:6789/0 conn(0x7f69640ac2b0 legacy=0x7f69641588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f6968c38640 1 -- 172.21.15.26:0/3928552375 >> v1:172.21.15.26:6789/0 conn(0x7f696406c240 legacy=0x7f69640ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.157+0000 7f6968c38640 1 -- 172.21.15.26:0/3928552375 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964155c60 con 0x7f69640ab860 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.158+0000 7f6968c38640 1 --2- 172.21.15.26:0/3928552375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ab860 0x7f69640c4250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f69540097a0 tx=0x7f695402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2d4a26bc0ce80c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.158+0000 7f695bfff640 1 -- 172.21.15.26:0/3928552375 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954035020 con 0x7f69640ab860 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.158+0000 7f695bfff640 1 -- 172.21.15.26:0/3928552375 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954031830 con 0x7f69640ab860 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.158+0000 7f695bfff640 1 -- 172.21.15.26:0/3928552375 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954031be0 con 0x7f69640ab860 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.159+0000 7f696b69d640 1 -- 172.21.15.26:0/3928552375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ab860 msgr2=0x7f69640c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.159+0000 7f696b69d640 1 --2- 172.21.15.26:0/3928552375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ab860 0x7f69640c4250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f69540097a0 tx=0x7f695402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.159+0000 7f696b69d640 1 -- 172.21.15.26:0/3928552375 shutdown_connections 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.159+0000 7f696b69d640 1 --2- 172.21.15.26:0/3928552375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ab860 0x7f69640c4250 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.159+0000 7f696b69d640 1 -- 172.21.15.26:0/3928552375 >> 172.21.15.26:0/3928552375 conn(0x7f69640c7670 msgr2=0x7f69640c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.159+0000 7f696b69d640 1 -- 172.21.15.26:0/3928552375 shutdown_connections 2024-03-06T07:24:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.159+0000 7f696b69d640 1 -- 172.21.15.26:0/3928552375 wait complete. 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.160+0000 7f696b69d640 1 Processor -- start 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.160+0000 7f696b69d640 1 -- start start 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f696b69d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f696406c240 0x7f69641ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f696b69d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69640ab860 0x7f69641e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f696b69d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ac2b0 0x7f69641e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f696b69d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69640c4830 con 0x7f69640ab860 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f696b69d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964156180 con 0x7f69640ac2b0 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f696b69d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696406c620 con 0x7f696406c240 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69640ab860 0x7f69641e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69640ab860 0x7f69641e7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53782/0 (socket says 172.21.15.26:53782) 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 -- 172.21.15.26:0/3732022001 learned_addr learned my addr 172.21.15.26:0/3732022001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6969439640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f696406c240 0x7f69641ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6969c3a640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ac2b0 0x7f69641e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 -- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f696406c240 msgr2=0x7f69641ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f696406c240 0x7f69641ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 -- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ac2b0 msgr2=0x7f69641e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ac2b0 0x7f69641e7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 -- 172.21.15.26:0/3732022001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69641ee640 con 0x7f69640ab860 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6968c38640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69640ab860 0x7f69641e7920 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6954000c00 tx=0x7f69540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6959ffb640 1 -- 172.21.15.26:0/3732022001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954035070 con 0x7f69640ab860 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6959ffb640 1 -- 172.21.15.26:0/3732022001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954004230 con 0x7f69640ab860 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69641ee920 con 0x7f69640ab860 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.161+0000 7f6959ffb640 1 -- 172.21.15.26:0/3732022001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695403a630 con 0x7f69640ab860 2024-03-06T07:24:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.162+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69641ee340 con 0x7f69640ab860 2024-03-06T07:24:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.163+0000 7f6959ffb640 1 -- 172.21.15.26:0/3732022001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6954038020 con 0x7f69640ab860 2024-03-06T07:24:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.163+0000 7f6959ffb640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f694806b4d0 0x7f694806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.164+0000 7f6959ffb640 1 -- 172.21.15.26:0/3732022001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f69540abf20 con 0x7f69640ab860 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.164+0000 7f6969439640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f694806b4d0 0x7f694806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.164+0000 7f6969439640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f694806b4d0 0x7f694806d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f69600011e0 tx=0x7f6960007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.164+0000 7f696e363740 1 Processor -- start 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.164+0000 7f696e363740 1 -- start start 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee7c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f696e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c64b0 0x5559ee7c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f696e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ee72c6b0 con 0x5559ee7c64b0 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ee6d5a60 con 0x5559ee7c7f50 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559ee728750 con 0x5559ee7c8330 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c64b0 0x5559ee7c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c64b0 0x5559ee7c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53784/0 (socket says 172.21.15.26:53784) 2024-03-06T07:24:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969c3a640 1 -- 172.21.15.26:0/2118535603 learned_addr learned my addr 172.21.15.26:0/2118535603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969439640 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6968c38640 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee7c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969c3a640 1 -- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 msgr2=0x5559ee7c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969c3a640 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee7c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969c3a640 1 -- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 msgr2=0x5559ee7c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.165+0000 7f6969c3a640 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f6969c3a640 1 -- 172.21.15.26:0/2118535603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559ee6f3f30 con 0x5559ee7c64b0 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f6969c3a640 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c64b0 0x5559ee7c2680 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f695c001e60 tx=0x7f695c0015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337cfc46d739f9a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f694effd640 1 -- 172.21.15.26:0/2118535603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c002070 con 0x5559ee7c64b0 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f694effd640 1 -- 172.21.15.26:0/2118535603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c018070 con 0x5559ee7c64b0 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f694effd640 1 -- 172.21.15.26:0/2118535603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c003b70 con 0x5559ee7c64b0 2024-03-06T07:24:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 -- 172.21.15.26:0/2118535603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c64b0 msgr2=0x5559ee7c2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c64b0 0x5559ee7c2680 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f695c001e60 tx=0x7f695c0015d0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 -- 172.21.15.26:0/2118535603 shutdown_connections 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c64b0 0x5559ee7c2680 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee7c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 --2- 172.21.15.26:0/2118535603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 -- 172.21.15.26:0/2118535603 >> 172.21.15.26:0/2118535603 conn(0x5559ee7c47d0 msgr2=0x5559ee7e93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.166+0000 7f696e363740 1 -- 172.21.15.26:0/2118535603 shutdown_connections 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.167+0000 7f696e363740 1 -- 172.21.15.26:0/2118535603 wait complete. 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.167+0000 7f696e363740 1 -- start start 2024-03-06T07:24:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.167+0000 7f696e363740 1 Processor -- start 2024-03-06T07:24:21.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee7e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f696e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7ea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f696e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ee7ea230 con 0x5559ee7c8330 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ee6e8ea0 con 0x5559ee7eec00 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559ee7e9c70 con 0x5559ee7c7f50 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f696e363740 10 client.0 _ll_register_callbacks cb 0x5559ee6e8870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7ea600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53790/0 (socket says 172.21.15.26:53790) 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 -- 172.21.15.26:0/3430018333 learned_addr learned my addr 172.21.15.26:0/3430018333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6969439640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee7e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6969c3a640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 -- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 msgr2=0x5559ee7e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee7e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 -- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 msgr2=0x5559ee7e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7e4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6968c38640 1 -- 172.21.15.26:0/3430018333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559ee6e9270 con 0x5559ee7c8330 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.168+0000 7f6969c3a640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6968c38640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7ea600 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f69540aa260 tx=0x7f695407a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ded7e812e654379 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6943fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6943fff640 1 -- 172.21.15.26:0/3430018333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540a7030 con 0x5559ee7c8330 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6943fff640 1 -- 172.21.15.26:0/3430018333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6954082730 con 0x5559ee7c8330 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f695c018070 con 0x5559ee7c8330 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6943fff640 1 -- 172.21.15.26:0/3430018333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6954078920 con 0x5559ee7c8330 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 10 client.7632 fetch_fsmap learned FSMap version 95 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x5559ee4412d0 con 0x5559ee7c8330 2024-03-06T07:24:21.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6943fff640 1 -- 172.21.15.26:0/3430018333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c018070 con 0x5559ee7c8330 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6943fff640 1 -- 172.21.15.26:0/3430018333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 95) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f6954042070 con 0x5559ee7c8330 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 10 client.7632 fetch_fsmap finished waiting for FSMap version 95 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 10 client.7632 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 20 client.7632 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee818300 0x5559ee8186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 4 client.7632 mds_command: new command op to 6305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 --> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559ee739bf0 con 0x5559ee818300 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.169+0000 7f6969439640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee818300 0x5559ee8186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f6969439640 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee818300 0x5559ee8186e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d783dfdfc7b1c44e server_cookie=4d44551c4d4f8598 in_seq=0 out_seq=0 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f6943fff640 10 client.7632 ms_handle_connect on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f6943fff640 1 -- 172.21.15.26:0/3430018333 <== mds.0 v2:172.21.15.117:6834/548706796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5559ee739bf0 con 0x5559ee818300 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f6943fff640 10 client.7632 handle_command_reply: tid=0 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f696e363740 1 client.7632 shutdown 2024-03-06T07:24:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 msgr2=0x5559ee7ea600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f696e363740 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7ea600 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f69540aa260 tx=0x7f695407a6a0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 shutdown_connections 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f696e363740 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee818300 0x5559ee8186e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.170+0000 7f696e363740 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.171+0000 7f696e363740 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7ea600 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.171+0000 7f6943fff640 0 client.7632 ms_handle_reset on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.171+0000 7f696e363740 1 --2- 172.21.15.26:0/3430018333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee7e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.171+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 >> 172.21.15.26:0/3430018333 conn(0x5559ee7c64b0 msgr2=0x5559ee7c3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.171+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 shutdown_connections 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.171+0000 7f696e363740 1 -- 172.21.15.26:0/3430018333 wait complete. 2024-03-06T07:24:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.171+0000 7f696e363740 20 client.7632 trim_cache size 0 max 16384 2024-03-06T07:24:21.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.187+0000 7f696e363740 1 Processor -- start 2024-03-06T07:24:21.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.187+0000 7f696e363740 1 -- start start 2024-03-06T07:24:21.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee805820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f696e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7c2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f696e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69641f7c60 con 0x5559ee7c8330 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69641f6220 con 0x5559ee7eec00 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69641f6820 con 0x5559ee7c7f50 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7c2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53794/0 (socket says 172.21.15.26:53794) 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 -- 172.21.15.26:0/2807310760 learned_addr learned my addr 172.21.15.26:0/2807310760 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6969c3a640 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6969439640 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee805820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 -- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 msgr2=0x5559ee805820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee805820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 -- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 msgr2=0x5559ee7c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6968c38640 1 -- 172.21.15.26:0/2807310760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559ee50b6b0 con 0x5559ee7c8330 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.188+0000 7f6969c3a640 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f6968c38640 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7c2300 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f695407a030 tx=0x7f695408f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fa82a42144fcc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f694e7fc640 1 -- 172.21.15.26:0/2807310760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540a7020 con 0x5559ee7c8330 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f694e7fc640 1 -- 172.21.15.26:0/2807310760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69540a3bc0 con 0x5559ee7c8330 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f694e7fc640 1 -- 172.21.15.26:0/2807310760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540799b0 con 0x5559ee7c8330 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 -- 172.21.15.26:0/2807310760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 msgr2=0x5559ee7c2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7c2300 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f695407a030 tx=0x7f695408f000 comp rx=0 tx=0).stop 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 -- 172.21.15.26:0/2807310760 shutdown_connections 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7eec00 0x5559ee7c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7c8330 0x5559ee7c2300 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 --2- 172.21.15.26:0/2807310760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c7f50 0x5559ee805820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 -- 172.21.15.26:0/2807310760 >> 172.21.15.26:0/2807310760 conn(0x5559ee7c64b0 msgr2=0x5559ee7c3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.189+0000 7f696e363740 1 -- 172.21.15.26:0/2807310760 shutdown_connections 2024-03-06T07:24:21.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.190+0000 7f696e363740 1 -- 172.21.15.26:0/2807310760 wait complete. 2024-03-06T07:24:21.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.190+0000 7f696e363740 1 -- start start 2024-03-06T07:24:21.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.190+0000 7f696e363740 1 Processor -- start 2024-03-06T07:24:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f696e363740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee8777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f696e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7eec00 0x5559ee8736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f696e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ee7ea230 con 0x5559ee7eec00 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ee7ead50 con 0x5559ee7c7f50 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f696e363740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559ee7ecdf0 con 0x5559ee7c8330 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f696e363740 10 client.0 _ll_register_callbacks cb 0x7f6954079ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7eec00 0x5559ee8736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7eec00 0x5559ee8736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53804/0 (socket says 172.21.15.26:53804) 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969439640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969c3a640 1 -- 172.21.15.26:0/2146893209 learned_addr learned my addr 172.21.15.26:0/2146893209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6968c38640 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee8777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969439640 1 -- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 msgr2=0x5559ee8777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969439640 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee8777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969439640 1 -- 172.21.15.26:0/2146893209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7eec00 msgr2=0x5559ee8736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969439640 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7eec00 0x5559ee8736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969439640 1 -- 172.21.15.26:0/2146893209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559ee7e1cc0 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.191+0000 7f6969c3a640 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7eec00 0x5559ee8736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6969439640 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c2700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5559ee7ec470 tx=0x7f6960001150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68916f4225faa295 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6943fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6943fff640 1 -- 172.21.15.26:0/2146893209 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960005850 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6943fff640 1 -- 172.21.15.26:0/2146893209 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6960030ce0 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69540a3bc0 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6943fff640 1 -- 172.21.15.26:0/2146893209 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69600059f0 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 10 client.7261 fetch_fsmap learned FSMap version 95 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x5559ee805dd0 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6943fff640 1 -- 172.21.15.26:0/2146893209 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69540a3bc0 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6943fff640 1 -- 172.21.15.26:0/2146893209 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 95) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f696002c020 con 0x5559ee7c7f50 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 10 client.7261 fetch_fsmap finished waiting for FSMap version 95 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 10 client.7261 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:24:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 20 client.7261 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee7c68e0 0x5559ee7c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 4 client.7261 mds_command: new command op to 6305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 --> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5559ee733f40 con 0x5559ee7c68e0 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.192+0000 7f6968c38640 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee7c68e0 0x5559ee7c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f6943fff640 10 client.7261 ms_handle_connect on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f6968c38640 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee7c68e0 0x5559ee7c6cc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df5ebe540ce6a4a server_cookie=6b4c44b70b373834 in_seq=0 out_seq=0 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f6943fff640 1 -- 172.21.15.26:0/2146893209 <== mds.0 v2:172.21.15.117:6834/548706796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5559ee733f40 con 0x5559ee7c68e0 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f6943fff640 10 client.7261 handle_command_reply: tid=0 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 client.7261 shutdown 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 msgr2=0x5559ee7c2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c2700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5559ee7ec470 tx=0x7f6960001150 comp rx=0 tx=0).stop 2024-03-06T07:24:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 shutdown_connections 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x5559ee7c68e0 0x5559ee7c6cc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559ee7eec00 0x5559ee8736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5559ee7c8330 0x5559ee8777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 --2- 172.21.15.26:0/2146893209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559ee7c7f50 0x5559ee7c2700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f6943fff640 0 client.7261 ms_handle_reset on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 >> 172.21.15.26:0/2146893209 conn(0x5559ee7c64b0 msgr2=0x5559ee7e8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 shutdown_connections 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 1 -- 172.21.15.26:0/2146893209 wait complete. 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.193+0000 7f696e363740 20 client.7261 trim_cache size 0 max 16384 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:21.201 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6305, 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 7.4860050109999996, 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 95, 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 109, 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 109, 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 27.606863973999999 2024-03-06T07:24:21.202 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f694806b4d0 msgr2=0x7f694806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f694806b4d0 0x7f694806d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f69600011e0 tx=0x7f6960007d30 comp rx=0 tx=0).stop 2024-03-06T07:24:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69640ab860 msgr2=0x7f69641e7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69640ab860 0x7f69641e7920 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6954000c00 tx=0x7f69540046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 shutdown_connections 2024-03-06T07:24:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f694806b4d0 0x7f694806d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69640ac2b0 0x7f69641e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69640ab860 0x7f69641e7920 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 --2- 172.21.15.26:0/3732022001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f696406c240 0x7f69641ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 >> 172.21.15.26:0/3732022001 conn(0x7f69640c7670 msgr2=0x7f6964153300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 shutdown_connections 2024-03-06T07:24:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:21.194+0000 7f696b69d640 1 -- 172.21.15.26:0/3732022001 wait complete. 2024-03-06T07:24:21.206 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 95, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:04.037603+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 95, "ever_allowed_features": 32, "explicitly_allowed_features": 32, "failed": [], "flags": 50, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": true, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6305": { "addr": "172.21.15.117:6835/548706796", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 548706796, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 548706796, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6305, "incarnation": 91, "join_fscid": 8, "name": "b", "rank": 0, "state": "up:active", "state_seq": 9 }, "gid_7566": { "addr": "172.21.15.26:6835/131071104", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 131071104, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 131071104, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7566, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:standby-replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 109, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:19.621541+0000", "qdb_cluster": [ 6305 ], "qdb_leader": 6305, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6305 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/3362094815", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3362094815, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3362094815, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7084, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/256924791", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 256924791, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 256924791, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 85, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7365, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:21.207 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 8, 'name': 'b', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 8, 'name': 'd', 'state': 'up:standby-replay'}] 2024-03-06T07:24:21.207 INFO:tasks.cephfs.test_failover:current = [{'gid': 7566, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/131071104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 131071104}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 131071104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6305, 'name': 'b', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7365, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/256924791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 256924791}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 256924791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 85}, {'gid': 7084, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3362094815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3362094815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3362094815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}] 2024-03-06T07:24:21.207 DEBUG:tasks.cephfs.test_failover:8 != -1 2024-03-06T07:24:21.207 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:24:23.776 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:24:26.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:26.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.321+0000 7f3c79aeb640 1 Processor -- start 2024-03-06T07:24:26.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.321+0000 7f3c79aeb640 1 -- start start 2024-03-06T07:24:26.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c79aeb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c7406d9f0 0x7f3c740a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c79aeb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c740c3280 0x7f3c740c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c79aeb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c74158800 con 0x7f3c741551e0 2024-03-06T07:24:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c79aeb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c74158ae0 con 0x7f3c7406d9f0 2024-03-06T07:24:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c79aeb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74158dc0 con 0x7f3c740c3280 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c737fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c7406d9f0 0x7f3c740a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c72ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c740c3280 0x7f3c740c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c73fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3c741551e0 0x7f3c74155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60860/0 (socket says 172.21.15.26:60860) 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c72ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c740c3280 0x7f3c740c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:37542/0 (socket says 172.21.15.26:37542) 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c73fff640 1 -- 172.21.15.26:0/1062164276 learned_addr learned my addr 172.21.15.26:0/1062164276 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.322+0000 7f3c737fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c7406d9f0 0x7f3c740a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36210/0 (socket says 172.21.15.26:36210) 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.323+0000 7f3c72ffd640 1 -- 172.21.15.26:0/1062164276 >> v2:172.21.15.117:3300/0 conn(0x7f3c7406d9f0 msgr2=0x7f3c740a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.323+0000 7f3c72ffd640 1 --2- 172.21.15.26:0/1062164276 >> v2:172.21.15.117:3300/0 conn(0x7f3c7406d9f0 0x7f3c740a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.323+0000 7f3c72ffd640 1 -- 172.21.15.26:0/1062164276 >> v1:172.21.15.117:6789/0 conn(0x7f3c741551e0 legacy=0x7f3c74155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.323+0000 7f3c72ffd640 1 -- 172.21.15.26:0/1062164276 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c741590a0 con 0x7f3c740c3280 2024-03-06T07:24:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.323+0000 7f3c727fc640 1 -- 172.21.15.26:0/1062164276 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536698026 0 0) 0x7f3c74158800 con 0x7f3c741551e0 2024-03-06T07:24:26.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.323+0000 7f3c737fe640 1 --2- 172.21.15.26:0/1062164276 >> v2:172.21.15.117:3300/0 conn(0x7f3c7406d9f0 0x7f3c740a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.323+0000 7f3c72ffd640 1 --2- 172.21.15.26:0/1062164276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c740c3280 0x7f3c740c36e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c680097a0 tx=0x7f3c6802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78332326e9c6e1d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c727fc640 1 -- 172.21.15.26:0/1062164276 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680315a0 con 0x7f3c740c3280 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c727fc640 1 -- 172.21.15.26:0/1062164276 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c68037070 con 0x7f3c740c3280 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c727fc640 1 -- 172.21.15.26:0/1062164276 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68031960 con 0x7f3c740c3280 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 -- 172.21.15.26:0/1062164276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c740c3280 msgr2=0x7f3c740c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/1062164276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c740c3280 0x7f3c740c36e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c680097a0 tx=0x7f3c6802b840 comp rx=0 tx=0).stop 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 -- 172.21.15.26:0/1062164276 shutdown_connections 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/1062164276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c740c3280 0x7f3c740c36e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/1062164276 >> v2:172.21.15.117:3300/0 conn(0x7f3c7406d9f0 0x7f3c740a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 -- 172.21.15.26:0/1062164276 >> 172.21.15.26:0/1062164276 conn(0x7f3c740aa630 msgr2=0x7f3c740aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 -- 172.21.15.26:0/1062164276 shutdown_connections 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.324+0000 7f3c79aeb640 1 -- 172.21.15.26:0/1062164276 wait complete. 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.325+0000 7f3c79aeb640 1 Processor -- start 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c79aeb640 1 -- start start 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c79aeb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c7406d9f0 0x7f3c74156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c79aeb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c740c3280 0x7f3c741eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c79aeb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c741551e0 0x7f3c7406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c79aeb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7406df50 con 0x7f3c741551e0 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c79aeb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7406ddd0 con 0x7f3c740c3280 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c79aeb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740a8210 con 0x7f3c7406d9f0 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c741551e0 0x7f3c7406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c741551e0 0x7f3c7406f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53834/0 (socket says 172.21.15.26:53834) 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c73fff640 1 -- 172.21.15.26:0/2653063317 learned_addr learned my addr 172.21.15.26:0/2653063317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c737fe640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c7406d9f0 0x7f3c74156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.326+0000 7f3c72ffd640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c740c3280 0x7f3c741eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c73fff640 1 -- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c7406d9f0 msgr2=0x7f3c74156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c73fff640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c7406d9f0 0x7f3c74156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c73fff640 1 -- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c740c3280 msgr2=0x7f3c741eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c73fff640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c740c3280 0x7f3c741eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c73fff640 1 -- 172.21.15.26:0/2653063317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c740a99d0 con 0x7f3c741551e0 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c737fe640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c7406d9f0 0x7f3c74156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c73fff640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c741551e0 0x7f3c7406f980 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c830 tx=0x7f3c6c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c5bfff640 1 -- 172.21.15.26:0/2653063317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c018070 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c5bfff640 1 -- 172.21.15.26:0/2653063317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c00ce20 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c740a9cb0 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c5bfff640 1 -- 172.21.15.26:0/2653063317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00a880 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.327+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c741578b0 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.328+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7404f330 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.329+0000 7f3c5bfff640 1 -- 172.21.15.26:0/2653063317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c6c01b070 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.329+0000 7f3c5bfff640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3c6006b4d0 0x7f3c6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.329+0000 7f3c5bfff640 1 -- 172.21.15.26:0/2653063317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3c6c089890 con 0x7f3c741551e0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.329+0000 7f3c737fe640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3c6006b4d0 0x7f3c6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.330+0000 7f3c737fe640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3c6006b4d0 0x7f3c6006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009560 tx=0x7f3c64009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.334+0000 7f3c5bfff640 1 -- 172.21.15.26:0/2653063317 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3c6c057fa0 con 0x7f3c741551e0 2024-03-06T07:24:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.552+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c74073240 con 0x7f3c741551e0 2024-03-06T07:24:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.553+0000 7f3c5bfff640 1 -- 172.21.15.26:0/2653063317 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4981 (secure 0 0 0) 0x7f3c6c05c040 con 0x7f3c741551e0 2024-03-06T07:24:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:26.555 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7647,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/423809105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":423809105},{"type":"v1","addr":"172.21.15.26:6835","nonce":423809105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":97,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:23.745787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7084":{"gid":7084,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":7,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:26.555 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 98 2024-03-06T07:24:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.554+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3c6006b4d0 msgr2=0x7f3c6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.554+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3c6006b4d0 0x7f3c6006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009560 tx=0x7f3c64009190 comp rx=0 tx=0).stop 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.554+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c741551e0 msgr2=0x7f3c7406f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.554+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c741551e0 0x7f3c7406f980 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c830 tx=0x7f3c6c00ccf0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.554+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 shutdown_connections 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.555+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3c6006b4d0 0x7f3c6006d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.555+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c741551e0 0x7f3c7406f980 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.555+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c740c3280 0x7f3c741eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.555+0000 7f3c79aeb640 1 --2- 172.21.15.26:0/2653063317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c7406d9f0 0x7f3c74156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.555+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 >> 172.21.15.26:0/2653063317 conn(0x7f3c740aa630 msgr2=0x7f3c7406c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.555+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 shutdown_connections 2024-03-06T07:24:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:26.555+0000 7f3c79aeb640 1 -- 172.21.15.26:0/2653063317 wait complete. 2024-03-06T07:24:26.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-06T07:24:04.037603+0000', 'modified': '2024-03-06T07:24:23.745787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6305}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6305': {'gid': 6305, 'name': 'b', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7084': {'gid': 7084, 'name': 'd', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 7, 'addr': '172.21.15.117:6837/3362094815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3362094815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3362094815}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6305, 'qdb_cluster': [6305]} 2024-03-06T07:24:26.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:26.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:24:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.075+0000 7f8eb4029640 1 Processor -- start 2024-03-06T07:24:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.075+0000 7f8eb4029640 1 -- start start 2024-03-06T07:24:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb4029640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb4029640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8eac0c7170 0x7f8eac154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb4029640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac0a8390 con 0x7f8eac0c6c40 2024-03-06T07:24:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb4029640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac0c7620 con 0x7f8eac0c7170 2024-03-06T07:24:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb4029640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0730f0 con 0x7f8eac06ff10 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb1dc5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb15c4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8eac0c7170 0x7f8eac154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb25c6640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f8eac0c6c40 0x7f8eac0c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35916/0 (socket says 172.21.15.26:35916) 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.076+0000 7f8eb25c6640 1 -- 172.21.15.26:0/119745428 learned_addr learned my addr 172.21.15.26:0/119745428 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.077+0000 7f8eb0dc3640 1 -- 172.21.15.26:0/119745428 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001999214 0 0) 0x7f8eac0a8390 con 0x7f8eac0c6c40 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.077+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/119745428 >> v2:172.21.15.117:3300/0 conn(0x7f8eac0c7170 msgr2=0x7f8eac154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.077+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/119745428 >> v2:172.21.15.117:3300/0 conn(0x7f8eac0c7170 0x7f8eac154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.077+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/119745428 >> v1:172.21.15.117:6789/0 conn(0x7f8eac0c6c40 legacy=0x7f8eac0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.077+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/119745428 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac0733d0 con 0x7f8eac06ff10 2024-03-06T07:24:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.077+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/119745428 >> v2:172.21.15.117:3300/0 conn(0x7f8eac0c7170 0x7f8eac154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.077+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/119745428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac153c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40097a0 tx=0x7f8ea402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d00ce1b2bb602b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb0dc3640 1 -- 172.21.15.26:0/119745428 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4039070 con 0x7f8eac06ff10 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb0dc3640 1 -- 172.21.15.26:0/119745428 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea40316f0 con 0x7f8eac06ff10 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb0dc3640 1 -- 172.21.15.26:0/119745428 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4031ac0 con 0x7f8eac06ff10 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 -- 172.21.15.26:0/119745428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 msgr2=0x7f8eac153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 --2- 172.21.15.26:0/119745428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac153c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40097a0 tx=0x7f8ea402b790 comp rx=0 tx=0).stop 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 -- 172.21.15.26:0/119745428 shutdown_connections 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 --2- 172.21.15.26:0/119745428 >> v2:172.21.15.117:3300/0 conn(0x7f8eac0c7170 0x7f8eac154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 --2- 172.21.15.26:0/119745428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac153c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 -- 172.21.15.26:0/119745428 >> 172.21.15.26:0/119745428 conn(0x7f8eac06b970 msgr2=0x7f8eac0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 -- 172.21.15.26:0/119745428 shutdown_connections 2024-03-06T07:24:27.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.078+0000 7f8eb4029640 1 -- 172.21.15.26:0/119745428 wait complete. 2024-03-06T07:24:27.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.079+0000 7f8eb4029640 1 Processor -- start 2024-03-06T07:24:27.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.079+0000 7f8eb4029640 1 -- start start 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb4029640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb4029640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eac0c6c40 0x7f8eac0c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb4029640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eac0c7170 0x7f8eac1ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb4029640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac070470 con 0x7f8eac0c6c40 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb4029640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0702f0 con 0x7f8eac0c7170 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb4029640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0c6860 con 0x7f8eac06ff10 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eac0c6c40 0x7f8eac0c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eac0c6c40 0x7f8eac0c1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48396/0 (socket says 172.21.15.26:48396) 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 -- 172.21.15.26:0/2468036242 learned_addr learned my addr 172.21.15.26:0/2468036242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb25c6640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eac0c7170 0x7f8eac1ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 -- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 msgr2=0x7f8eac156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 -- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eac0c7170 msgr2=0x7f8eac1ea7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eac0c7170 0x7f8eac1ea7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.080+0000 7f8eb15c4640 1 -- 172.21.15.26:0/2468036242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac1ed9d0 con 0x7f8eac0c6c40 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.081+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eac0c6c40 0x7f8eac0c1cb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e900 tx=0x7f8ea000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.081+0000 7f8eaa7fc640 1 -- 172.21.15.26:0/2468036242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0017070 con 0x7f8eac0c6c40 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.081+0000 7f8eaa7fc640 1 -- 172.21.15.26:0/2468036242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea000ca00 con 0x7f8eac0c6c40 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.081+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac1edcb0 con 0x7f8eac0c6c40 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.081+0000 7f8eaa7fc640 1 -- 172.21.15.26:0/2468036242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000cdf0 con 0x7f8eac0c6c40 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.081+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac1ee270 con 0x7f8eac0c6c40 2024-03-06T07:24:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.082+0000 7f8eaa7fc640 1 -- 172.21.15.26:0/2468036242 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ea001a070 con 0x7f8eac0c6c40 2024-03-06T07:24:27.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.083+0000 7f8eaa7fc640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e9806b4d0 0x7f8e9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.083+0000 7f8eaa7fc640 1 -- 172.21.15.26:0/2468036242 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8ea008a1d0 con 0x7f8eac0c6c40 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.083+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e9806b4d0 0x7f8e9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.084+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e9806b4d0 0x7f8e9806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4036000 tx=0x7f8ea4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.084+0000 7f8eb6c0c740 1 Processor -- start 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.084+0000 7f8eb6c0c740 1 -- start start 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450bf50 0x557af4509a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450c330 0x557af4509f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450a4b0 0x557af4506680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af44706b0 con 0x557af450c330 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4419a60 con 0x557af450bf50 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557af446c750 con 0x557af450a4b0 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450c330 0x557af4509f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450c330 0x557af4509f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48398/0 (socket says 172.21.15.26:48398) 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 -- 172.21.15.26:0/1767816812 learned_addr learned my addr 172.21.15.26:0/1767816812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450bf50 0x557af4509a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb25c6640 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450a4b0 0x557af4506680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 -- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450a4b0 msgr2=0x557af4506680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450a4b0 0x557af4506680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 -- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450bf50 msgr2=0x557af4509a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450bf50 0x557af4509a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 -- 172.21.15.26:0/1767816812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af4437f30 con 0x557af450c330 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450c330 0x557af4509f70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0054d10 tx=0x7f8ea0058340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787e332968aaca56 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8e937fe640 1 -- 172.21.15.26:0/1767816812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0021040 con 0x557af450c330 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8e937fe640 1 -- 172.21.15.26:0/1767816812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea0056950 con 0x557af450c330 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.085+0000 7f8e937fe640 1 -- 172.21.15.26:0/1767816812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0054f20 con 0x557af450c330 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/1767816812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450c330 msgr2=0x557af4509f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450c330 0x557af4509f70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0054d10 tx=0x7f8ea0058340 comp rx=0 tx=0).stop 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/1767816812 shutdown_connections 2024-03-06T07:24:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450a4b0 0x557af4506680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450c330 0x557af4509f70 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/1767816812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450bf50 0x557af4509a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/1767816812 >> 172.21.15.26:0/1767816812 conn(0x557af45087d0 msgr2=0x557af452d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/1767816812 shutdown_connections 2024-03-06T07:24:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.086+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/1767816812 wait complete. 2024-03-06T07:24:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 -- start start 2024-03-06T07:24:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 Processor -- start 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450bf50 0x557af452c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af4558f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af452e7b0 0x557af452ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af452e230 con 0x557af452e7b0 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af442cea0 con 0x557af450c330 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.087+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557af452dc70 con 0x557af450bf50 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb6c0c740 10 client.0 _ll_register_callbacks cb 0x557af442c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb25c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af452e7b0 0x557af452ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb1dc5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450bf50 0x557af452c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb1dc5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450bf50 0x557af452c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:43454/0 (socket says 172.21.15.26:43454) 2024-03-06T07:24:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/3977903692 learned_addr learned my addr 172.21.15.26:0/3977903692 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb15c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af4558f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb25c6640 1 -- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450bf50 msgr2=0x557af452c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb25c6640 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450bf50 0x557af452c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb25c6640 1 -- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 msgr2=0x557af4558f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb25c6640 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af4558f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb25c6640 1 -- 172.21.15.26:0/3977903692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557af442d270 con 0x557af452e7b0 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af4558f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb25c6640 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af452e7b0 0x557af452ec30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c000f80 tx=0x7f8e9c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e20cc6ade4fdb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8e90ff9640 1 -- 172.21.15.26:0/3977903692 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c012820 con 0x557af452e7b0 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8e90ff9640 1 -- 172.21.15.26:0/3977903692 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8e9c0129c0 con 0x557af452e7b0 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ea0056950 con 0x557af452e7b0 2024-03-06T07:24:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8e90ff9640 1 -- 172.21.15.26:0/3977903692 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e9c022050 con 0x557af452e7b0 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb6c0c740 10 client.7659 fetch_fsmap learned FSMap version 98 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.088+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x557af41852d0 con 0x557af452e7b0 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8e90ff9640 1 -- 172.21.15.26:0/3977903692 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea0056950 con 0x557af452e7b0 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8e90ff9640 1 -- 172.21.15.26:0/3977903692 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 98) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8e9c025020 con 0x557af452e7b0 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb6c0c740 10 client.7659 fetch_fsmap finished waiting for FSMap version 98 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb6c0c740 10 client.7659 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb6c0c740 20 client.7659 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4528e80 0x557af4529260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb6c0c740 4 client.7659 mds_command: new command op to 6305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 --> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557af447dbf0 con 0x557af4528e80 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4528e80 0x557af4529260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8e90ff9640 10 client.7659 ms_handle_connect on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.089+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4528e80 0x557af4529260 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89fd4f9882f664ab server_cookie=b3808eb2ac315c69 in_seq=0 out_seq=0 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8e90ff9640 1 -- 172.21.15.26:0/3977903692 <== mds.0 v2:172.21.15.117:6834/548706796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557af447dbf0 con 0x557af4528e80 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8e90ff9640 10 client.7659 handle_command_reply: tid=0 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 client.7659 shutdown 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af452e7b0 msgr2=0x557af452ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af452e7b0 0x557af452ec30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c000f80 tx=0x7f8e9c007750 comp rx=0 tx=0).stop 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 shutdown_connections 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4528e80 0x557af4529260 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af452e7b0 0x557af452ec30 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8e90ff9640 0 client.7659 ms_handle_reset on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af4558f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3977903692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450bf50 0x557af452c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 >> 172.21.15.26:0/3977903692 conn(0x557af450a4b0 msgr2=0x557af450b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 shutdown_connections 2024-03-06T07:24:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3977903692 wait complete. 2024-03-06T07:24:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.090+0000 7f8eb6c0c740 20 client.7659 trim_cache size 0 max 16384 2024-03-06T07:24:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.107+0000 7f8eb6c0c740 1 Processor -- start 2024-03-06T07:24:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.107+0000 7f8eb6c0c740 1 -- start start 2024-03-06T07:24:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af4506800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450c330 0x557af45095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af452e7b0 0x557af4509ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac1f7ce0 con 0x557af450bf50 2024-03-06T07:24:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac06cc20 con 0x557af452e7b0 2024-03-06T07:24:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac1f6910 con 0x557af450c330 2024-03-06T07:24:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af4506800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af4506800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48414/0 (socket says 172.21.15.26:48414) 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/3789851362 learned_addr learned my addr 172.21.15.26:0/3789851362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb25c6640 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af452e7b0 0x557af4509ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450c330 0x557af45095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450c330 msgr2=0x557af45095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450c330 0x557af45095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af452e7b0 msgr2=0x557af4509ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af452e7b0 0x557af4509ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/3789851362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af424f6b0 con 0x557af450bf50 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af4506800 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009890 tx=0x7f8ea4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8fd7009a9bc672 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8e92ffd640 1 -- 172.21.15.26:0/3789851362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4039040 con 0x557af450bf50 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8e92ffd640 1 -- 172.21.15.26:0/3789851362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4034b00 con 0x557af450bf50 2024-03-06T07:24:27.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.108+0000 7f8e92ffd640 1 -- 172.21.15.26:0/3789851362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4054590 con 0x557af450bf50 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3789851362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 msgr2=0x557af4506800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af4506800 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009890 tx=0x7f8ea4030000 comp rx=0 tx=0).stop 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3789851362 shutdown_connections 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af452e7b0 0x557af4509ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af450c330 0x557af45095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/3789851362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af4506800 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3789851362 >> 172.21.15.26:0/3789851362 conn(0x557af450a4b0 msgr2=0x557af4505510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3789851362 shutdown_connections 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/3789851362 wait complete. 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 -- start start 2024-03-06T07:24:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.109+0000 7f8eb6c0c740 1 Processor -- start 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af45b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af454ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb6c0c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af452e7b0 0x557af4554a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4528d00 con 0x557af450bf50 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af455c6f0 con 0x557af450c330 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb6c0c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557af452f210 con 0x557af452e7b0 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb6c0c740 10 client.0 _ll_register_callbacks cb 0x7f8e9c002000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb1dc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af45b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb15c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af454ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb1dc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af45b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48422/0 (socket says 172.21.15.26:48422) 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/404756734 learned_addr learned my addr 172.21.15.26:0/404756734 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.110+0000 7f8eb25c6640 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af452e7b0 0x557af4554a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af452e7b0 msgr2=0x557af4554a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af452e7b0 0x557af4554a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 msgr2=0x557af454ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af454ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb1dc5640 1 -- 172.21.15.26:0/404756734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557af4527ec0 con 0x557af450bf50 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb1dc5640 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af45b7da0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009560 tx=0x7f8ea4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5f6b367b73383e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8e90ff9640 1 -- 172.21.15.26:0/404756734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4055650 con 0x557af450bf50 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8e90ff9640 1 -- 172.21.15.26:0/404756734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8ea40557f0 con 0x557af450bf50 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ea4034b00 con 0x557af450bf50 2024-03-06T07:24:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8e90ff9640 1 -- 172.21.15.26:0/404756734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea40559f0 con 0x557af450bf50 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 10 client.7665 fetch_fsmap learned FSMap version 98 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x557af454bf90 con 0x557af450bf50 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8e90ff9640 1 -- 172.21.15.26:0/404756734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4034b00 con 0x557af450bf50 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8e90ff9640 1 -- 172.21.15.26:0/404756734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 98) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8ea4055b90 con 0x557af450bf50 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 10 client.7665 fetch_fsmap finished waiting for FSMap version 98 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 10 client.7665 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 20 client.7665 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4555560 0x557af45b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 4 client.7665 mds_command: new command op to 6305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.111+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 --> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557af4477f40 con 0x557af4555560 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4555560 0x557af45b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8e90ff9640 10 client.7665 ms_handle_connect on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb15c4640 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4555560 0x557af45b1700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a10a2ef3c2fadf43 server_cookie=c3f315e7087ce342 in_seq=0 out_seq=0 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8e90ff9640 1 -- 172.21.15.26:0/404756734 <== mds.0 v2:172.21.15.117:6834/548706796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557af4477f40 con 0x557af4555560 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8e90ff9640 10 client.7665 handle_command_reply: tid=0 2024-03-06T07:24:27.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 client.7665 shutdown 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 msgr2=0x557af45b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af45b7da0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009560 tx=0x7f8ea4004730 comp rx=0 tx=0).stop 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 shutdown_connections 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:6834/548706796,v1:172.21.15.117:6835/548706796] conn(0x557af4555560 0x557af45b1700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557af452e7b0 0x557af4554a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557af450c330 0x557af454ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 --2- 172.21.15.26:0/404756734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557af450bf50 0x557af45b7da0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8e90ff9640 0 client.7665 ms_handle_reset on v2:172.21.15.117:6834/548706796 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 >> 172.21.15.26:0/404756734 conn(0x557af450a4b0 msgr2=0x557af4555c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 shutdown_connections 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 1 -- 172.21.15.26:0/404756734 wait complete. 2024-03-06T07:24:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.112+0000 7f8eb6c0c740 20 client.7665 trim_cache size 0 max 16384 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6305, 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 13.405107710999999, 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 97, 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 109, 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 109, 2024-03-06T07:24:27.121 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 33.525967166999997 2024-03-06T07:24:27.122 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e9806b4d0 msgr2=0x7f8e9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e9806b4d0 0x7f8e9806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4036000 tx=0x7f8ea4008030 comp rx=0 tx=0).stop 2024-03-06T07:24:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eac0c6c40 msgr2=0x7f8eac0c1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eac0c6c40 0x7f8eac0c1cb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e900 tx=0x7f8ea000edc0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 shutdown_connections 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8e9806b4d0 0x7f8e9806d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eac0c7170 0x7f8eac1ea7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eac0c6c40 0x7f8eac0c1cb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 --2- 172.21.15.26:0/2468036242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eac06ff10 0x7f8eac156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 >> 172.21.15.26:0/2468036242 conn(0x7f8eac06b970 msgr2=0x7f8eac152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 shutdown_connections 2024-03-06T07:24:27.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.113+0000 7f8eb4029640 1 -- 172.21.15.26:0/2468036242 wait complete. 2024-03-06T07:24:27.125 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 98, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:04.037603+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 97, "ever_allowed_features": 32, "explicitly_allowed_features": 32, "failed": [], "flags": 50, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": true, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6305": { "addr": "172.21.15.117:6835/548706796", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 548706796, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 548706796, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6305, "incarnation": 91, "join_fscid": 8, "name": "b", "rank": 0, "state": "up:active", "state_seq": 9 }, "gid_7084": { "addr": "172.21.15.117:6837/3362094815", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3362094815, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3362094815, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7084, "incarnation": 0, "join_fscid": 8, "name": "d", "rank": 0, "state": "up:standby-replay", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:23.745787+0000", "qdb_cluster": [ 6305 ], "qdb_leader": 6305, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6305 } } } ], "standbys": [ { "addr": "172.21.15.26:6834/256924791", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 256924791, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 256924791, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 85, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7365, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/423809105", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 423809105, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 423809105, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 98, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7647, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:27.126 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 8, 'name': 'b', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 8, 'name': 'd', 'state': 'up:standby-replay'}] 2024-03-06T07:24:27.126 INFO:tasks.cephfs.test_failover:current = [{'gid': 7647, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/423809105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 423809105}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 423809105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 98}, {'gid': 6305, 'name': 'b', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6835/548706796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 548706796}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 548706796}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7365, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/256924791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 256924791}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 256924791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 85}, {'gid': 7084, 'name': 'd', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 7, 'addr': '172.21.15.117:6837/3362094815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3362094815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3362094815}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:24:27.126 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:24:27.127 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:24:27.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:27.181 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:27.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:24:27.223 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:24:27.223 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:27.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:24:27.238 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:24:27.238 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:27.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:24:27.278 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.378+0000 7f03be647640 1 Processor -- start 2024-03-06T07:24:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.378+0000 7f03be647640 1 -- start start 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.378+0000 7f03be647640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03b80c7380 0x7f03b80c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.378+0000 7f03be647640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b8072630 con 0x7f03b80c6780 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.378+0000 7f03be647640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b8072910 con 0x7f03b806da40 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.378+0000 7f03be647640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b806de20 con 0x7f03b80c7380 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.379+0000 7f03bcbe4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03b80c7380 0x7f03b80c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.379+0000 7f03b77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03b80c6780 0x7f03b8158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35682/0 (socket says 172.21.15.26:35682) 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.379+0000 7f03b77fe640 1 -- 172.21.15.26:0/252863212 learned_addr learned my addr 172.21.15.26:0/252863212 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.379+0000 7f03bcbe4640 1 -- 172.21.15.26:0/252863212 >> v1:172.21.15.117:6789/0 conn(0x7f03b806da40 legacy=0x7f03b8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.379+0000 7f03b6ffd640 1 -- 172.21.15.26:0/252863212 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059301596 0 0) 0x7f03b8072630 con 0x7f03b80c6780 2024-03-06T07:24:27.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.379+0000 7f03bcbe4640 1 -- 172.21.15.26:0/252863212 >> v1:172.21.15.26:6789/0 conn(0x7f03b80c6780 legacy=0x7f03b8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.379+0000 7f03bcbe4640 1 -- 172.21.15.26:0/252863212 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8072bf0 con 0x7f03b80c7380 2024-03-06T07:24:27.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.380+0000 7f03bcbe4640 1 --2- 172.21.15.26:0/252863212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b80c3280 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009d90 tx=0x7f03b002bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0cd58dd6ffc146 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.380+0000 7f03b6ffd640 1 -- 172.21.15.26:0/252863212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0039070 con 0x7f03b80c7380 2024-03-06T07:24:27.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.380+0000 7f03b6ffd640 1 -- 172.21.15.26:0/252863212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b002ca50 con 0x7f03b80c7380 2024-03-06T07:24:27.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.380+0000 7f03b6ffd640 1 -- 172.21.15.26:0/252863212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b002ce20 con 0x7f03b80c7380 2024-03-06T07:24:27.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.380+0000 7f03be647640 1 -- 172.21.15.26:0/252863212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 msgr2=0x7f03b80c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.380+0000 7f03be647640 1 --2- 172.21.15.26:0/252863212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b80c3280 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009d90 tx=0x7f03b002bd20 comp rx=0 tx=0).stop 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.381+0000 7f03be647640 1 -- 172.21.15.26:0/252863212 shutdown_connections 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.381+0000 7f03be647640 1 --2- 172.21.15.26:0/252863212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c7380 0x7f03b80c3280 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.381+0000 7f03be647640 1 -- 172.21.15.26:0/252863212 >> 172.21.15.26:0/252863212 conn(0x7f03b80aa630 msgr2=0x7f03b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.381+0000 7f03be647640 1 -- 172.21.15.26:0/252863212 shutdown_connections 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.381+0000 7f03be647640 1 -- 172.21.15.26:0/252863212 wait complete. 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 Processor -- start 2024-03-06T07:24:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 -- start start 2024-03-06T07:24:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b806da40 0x7f03b81eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c6780 0x7f03b81574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c37c0 con 0x7f03b80c6780 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b81594a0 con 0x7f03b80c7380 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.382+0000 7f03be647640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c3940 con 0x7f03b806da40 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c6780 0x7f03b81574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c6780 0x7f03b81574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48440/0 (socket says 172.21.15.26:48440) 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 -- 172.21.15.26:0/2568511738 learned_addr learned my addr 172.21.15.26:0/2568511738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b7fff640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b806da40 0x7f03b81eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03bcbe4640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 -- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b806da40 msgr2=0x7f03b81eeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b806da40 0x7f03b81eeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 -- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 msgr2=0x7f03b814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b814ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 -- 172.21.15.26:0/2568511738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8157c40 con 0x7f03b80c6780 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03bcbe4640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b77fe640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c6780 0x7f03b81574d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c470 tx=0x7f03ac00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b4ff9640 1 -- 172.21.15.26:0/2568511738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac018070 con 0x7f03b80c6780 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b4ff9640 1 -- 172.21.15.26:0/2568511738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac0074e0 con 0x7f03b80c6780 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b81ee040 con 0x7f03b80c6780 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03b4ff9640 1 -- 172.21.15.26:0/2568511738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac004a20 con 0x7f03b80c6780 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.383+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b81ee600 con 0x7f03b80c6780 2024-03-06T07:24:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.384+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b81ee800 con 0x7f03b80c6780 2024-03-06T07:24:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.385+0000 7f03b4ff9640 1 -- 172.21.15.26:0/2568511738 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03ac01b070 con 0x7f03b80c6780 2024-03-06T07:24:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.385+0000 7f03b4ff9640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b500 0x7f03a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.385+0000 7f03b4ff9640 1 -- 172.21.15.26:0/2568511738 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f03ac0895c0 con 0x7f03b80c6780 2024-03-06T07:24:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.385+0000 7f03b7fff640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b500 0x7f03a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.386+0000 7f03b7fff640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b500 0x7f03a406d9c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000f60 tx=0x7f03a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.389+0000 7f03b4ff9640 1 -- 172.21.15.26:0/2568511738 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f03b81ee800 con 0x7f03b80c6780 2024-03-06T07:24:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.608+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03b804f330 con 0x7f03b80c6780 2024-03-06T07:24:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.608+0000 7f03b4ff9640 1 -- 172.21.15.26:0/2568511738 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4981 (secure 0 0 0) 0x7f03ac057c10 con 0x7f03b80c6780 2024-03-06T07:24:27.610 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:27.610 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7647,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/423809105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":423809105},{"type":"v1","addr":"172.21.15.26:6835","nonce":423809105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":97,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:23.745787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7084":{"gid":7084,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":7,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 98 2024-03-06T07:24:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.610+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b500 msgr2=0x7f03a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.610+0000 7f03be647640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b500 0x7f03a406d9c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000f60 tx=0x7f03a8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.610+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c6780 msgr2=0x7f03b81574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.610+0000 7f03be647640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c6780 0x7f03b81574d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c470 tx=0x7f03ac00c930 comp rx=0 tx=0).stop 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 shutdown_connections 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f03a406b500 0x7f03a406d9c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03b80c7380 0x7f03b814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80c6780 0x7f03b81574d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 --2- 172.21.15.26:0/2568511738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03b806da40 0x7f03b81eeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 >> 172.21.15.26:0/2568511738 conn(0x7f03b80aa630 msgr2=0x7f03b80c5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 shutdown_connections 2024-03-06T07:24:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.611+0000 7f03be647640 1 -- 172.21.15.26:0/2568511738 wait complete. 2024-03-06T07:24:27.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.724+0000 7fbbb978a640 1 Processor -- start 2024-03-06T07:24:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.724+0000 7fbbb978a640 1 -- start start 2024-03-06T07:24:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb978a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbbb41527b0 0x7fbbb4159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb978a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb40c7c60 con 0x7fbbb40c88b0 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb978a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb40c8060 con 0x7fbbb40c7180 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb978a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb4159640 con 0x7fbbb41527b0 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbbb40c88b0 0x7fbbb40c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35698/0 (socket says 172.21.15.26:35698) 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb37fe640 1 -- 172.21.15.26:0/4007777408 learned_addr learned my addr 172.21.15.26:0/4007777408 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb2ffd640 1 --2- 172.21.15.26:0/4007777408 >> v2:172.21.15.117:3300/0 conn(0x7fbbb41527b0 0x7fbbb4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb1ffb640 1 -- 172.21.15.26:0/4007777408 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887127020 0 0) 0x7fbbac001110 con 0x7fbbb40c88b0 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb1ffb640 1 -- 172.21.15.26:0/4007777408 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba0003520 con 0x7fbbb40c88b0 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb1ffb640 1 -- 172.21.15.26:0/4007777408 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092802326 0 0) 0x7fbbb40c8060 con 0x7fbbb40c7180 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb1ffb640 1 -- 172.21.15.26:0/4007777408 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbac001110 con 0x7fbbb40c7180 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb2ffd640 1 -- 172.21.15.26:0/4007777408 >> v1:172.21.15.117:6789/0 conn(0x7fbbb40c7180 legacy=0x7fbbb4155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb2ffd640 1 -- 172.21.15.26:0/4007777408 >> v1:172.21.15.26:6789/0 conn(0x7fbbb40c88b0 legacy=0x7fbbb40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.725+0000 7fbbb2ffd640 1 -- 172.21.15.26:0/4007777408 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb40c76a0 con 0x7fbbb41527b0 2024-03-06T07:24:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.726+0000 7fbbb2ffd640 1 --2- 172.21.15.26:0/4007777408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 0x7fbbb4159100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009d90 tx=0x7fbba402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf391862fdfe86f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.726+0000 7fbbb1ffb640 1 -- 172.21.15.26:0/4007777408 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4039070 con 0x7fbbb41527b0 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.726+0000 7fbbb1ffb640 1 -- 172.21.15.26:0/4007777408 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba402cc60 con 0x7fbbb41527b0 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.726+0000 7fbbb1ffb640 1 -- 172.21.15.26:0/4007777408 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4034640 con 0x7fbbb41527b0 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.727+0000 7fbbb978a640 1 -- 172.21.15.26:0/4007777408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 msgr2=0x7fbbb4159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.727+0000 7fbbb978a640 1 --2- 172.21.15.26:0/4007777408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 0x7fbbb4159100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009d90 tx=0x7fbba402beb0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.727+0000 7fbbb978a640 1 -- 172.21.15.26:0/4007777408 shutdown_connections 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.727+0000 7fbbb978a640 1 --2- 172.21.15.26:0/4007777408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 0x7fbbb4159100 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.727+0000 7fbbb978a640 1 -- 172.21.15.26:0/4007777408 >> 172.21.15.26:0/4007777408 conn(0x7fbbb40ac560 msgr2=0x7fbbb40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.727+0000 7fbbb978a640 1 -- 172.21.15.26:0/4007777408 shutdown_connections 2024-03-06T07:24:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.727+0000 7fbbb978a640 1 -- 172.21.15.26:0/4007777408 wait complete. 2024-03-06T07:24:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.728+0000 7fbbb978a640 1 Processor -- start 2024-03-06T07:24:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.728+0000 7fbbb978a640 1 -- start start 2024-03-06T07:24:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb978a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb40c7180 0x7fbbb4154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb978a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbb40c88b0 0x7fbbb4145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb978a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 0x7fbbb4145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb978a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb4155600 con 0x7fbbb40c7180 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb978a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb4155320 con 0x7fbbb41527b0 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb978a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb4152b90 con 0x7fbbb40c88b0 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb40c7180 0x7fbbb4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb40c7180 0x7fbbb4154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48452/0 (socket says 172.21.15.26:48452) 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 -- 172.21.15.26:0/2462314402 learned_addr learned my addr 172.21.15.26:0/2462314402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb37fe640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 0x7fbbb4145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb27fc640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbb40c88b0 0x7fbbb4145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 -- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbb40c88b0 msgr2=0x7fbbb4145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbb40c88b0 0x7fbbb4145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 -- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 msgr2=0x7fbbb4145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 0x7fbbb4145c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 -- 172.21.15.26:0/2462314402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb41461a0 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.729+0000 7fbbb2ffd640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb40c7180 0x7fbbb4154070 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009eb0 tx=0x7fbba4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.730+0000 7fbbab7fe640 1 -- 172.21.15.26:0/2462314402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4042020 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.730+0000 7fbbab7fe640 1 -- 172.21.15.26:0/2462314402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba403a040 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.730+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb41ebe20 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.730+0000 7fbbab7fe640 1 -- 172.21.15.26:0/2462314402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4039070 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.730+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41ec3e0 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.731+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb404f330 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.731+0000 7fbbab7fe640 1 -- 172.21.15.26:0/2462314402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbba4045020 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.731+0000 7fbbab7fe640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb9406b4d0 0x7fbb9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.732+0000 7fbbab7fe640 1 -- 172.21.15.26:0/2462314402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbba40ac530 con 0x7fbbb40c7180 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.732+0000 7fbbb27fc640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb9406b4d0 0x7fbb9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.732+0000 7fbbb27fc640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb9406b4d0 0x7fbb9406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c007fd0 tx=0x7fbb9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.736+0000 7fbbab7fe640 1 -- 172.21.15.26:0/2462314402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbba404c030 con 0x7fbbb40c7180 2024-03-06T07:24:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.921+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbb406ce00 con 0x7fbbb40c7180 2024-03-06T07:24:27.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.922+0000 7fbbab7fe640 1 -- 172.21.15.26:0/2462314402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17093 (secure 0 0 0) 0x7fbbb41461a0 con 0x7fbbb40c7180 2024-03-06T07:24:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:27.925 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":109,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:13.704122+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:02.286561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:03.001111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1745266733":"2024-03-07T07:24:13.704095+0000","172.21.15.26:6833/1745266733":"2024-03-07T07:24:13.704095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.924+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb9406b4d0 msgr2=0x7fbb9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.924+0000 7fbbb978a640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb9406b4d0 0x7fbb9406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c007fd0 tx=0x7fbb9c009040 comp rx=0 tx=0).stop 2024-03-06T07:24:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.924+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb40c7180 msgr2=0x7fbbb4154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.924+0000 7fbbb978a640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb40c7180 0x7fbbb4154070 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009eb0 tx=0x7fbba4004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 shutdown_connections 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbb9406b4d0 0x7fbb9406d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb41527b0 0x7fbbb4145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbb40c88b0 0x7fbbb4145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 --2- 172.21.15.26:0/2462314402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb40c7180 0x7fbbb4154070 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 >> 172.21.15.26:0/2462314402 conn(0x7fbbb40ac560 msgr2=0x7fbbb406b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 shutdown_connections 2024-03-06T07:24:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:27.925+0000 7fbbb978a640 1 -- 172.21.15.26:0/2462314402 wait complete. 2024-03-06T07:24:27.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:24:02.286561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:24:03.001111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:27.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.041+0000 7f5885281640 1 Processor -- start 2024-03-06T07:24:28.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.041+0000 7f5885281640 1 -- start start 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f5885281640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f588014bcc0 0x7f5880155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f5885281640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58800be150 0x7f58800c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f5885281640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58800c5910 0x7f5880158a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f5885281640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58800c7f10 con 0x7f58800be150 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f5885281640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58800c75e0 con 0x7f588014bcc0 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f5885281640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c78c0 con 0x7f58800c5910 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58800be150 0x7f58800c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58800be150 0x7f58800c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48468/0 (socket says 172.21.15.26:48468) 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 -- 172.21.15.26:0/2560771544 learned_addr learned my addr 172.21.15.26:0/2560771544 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:28.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587f7fe640 1 --2- 172.21.15.26:0/2560771544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58800c5910 0x7f5880158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587effd640 1 --2- 172.21.15.26:0/2560771544 >> v2:172.21.15.117:3300/0 conn(0x7f588014bcc0 0x7f5880155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 -- 172.21.15.26:0/2560771544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58800c5910 msgr2=0x7f5880158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 --2- 172.21.15.26:0/2560771544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58800c5910 0x7f5880158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 -- 172.21.15.26:0/2560771544 >> v2:172.21.15.117:3300/0 conn(0x7f588014bcc0 msgr2=0x7f5880155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 --2- 172.21.15.26:0/2560771544 >> v2:172.21.15.117:3300/0 conn(0x7f588014bcc0 0x7f5880155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587e7fc640 1 -- 172.21.15.26:0/2560771544 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800c7ba0 con 0x7f58800be150 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.042+0000 7f587effd640 1 --2- 172.21.15.26:0/2560771544 >> v2:172.21.15.117:3300/0 conn(0x7f588014bcc0 0x7f5880155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.043+0000 7f587e7fc640 1 --2- 172.21.15.26:0/2560771544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58800c53d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f588008c180 tx=0x7f586802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939d6bdfb9b731b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.043+0000 7f587dffb640 1 -- 172.21.15.26:0/2560771544 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868034070 con 0x7f58800be150 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.043+0000 7f587dffb640 1 -- 172.21.15.26:0/2560771544 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5868030850 con 0x7f58800be150 2024-03-06T07:24:28.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.043+0000 7f587dffb640 1 -- 172.21.15.26:0/2560771544 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868030c20 con 0x7f58800be150 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 -- 172.21.15.26:0/2560771544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 msgr2=0x7f58800c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 --2- 172.21.15.26:0/2560771544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58800c53d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f588008c180 tx=0x7f586802b990 comp rx=0 tx=0).stop 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 -- 172.21.15.26:0/2560771544 shutdown_connections 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 --2- 172.21.15.26:0/2560771544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58800c5910 0x7f5880158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 --2- 172.21.15.26:0/2560771544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58800c53d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 --2- 172.21.15.26:0/2560771544 >> v2:172.21.15.117:3300/0 conn(0x7f588014bcc0 0x7f5880155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 -- 172.21.15.26:0/2560771544 >> 172.21.15.26:0/2560771544 conn(0x7f58800ac600 msgr2=0x7f58800aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 -- 172.21.15.26:0/2560771544 shutdown_connections 2024-03-06T07:24:28.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.044+0000 7f5885281640 1 -- 172.21.15.26:0/2560771544 wait complete. 2024-03-06T07:24:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.045+0000 7f5885281640 1 Processor -- start 2024-03-06T07:24:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.045+0000 7f5885281640 1 -- start start 2024-03-06T07:24:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f5885281640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58801e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f5885281640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58800c5910 0x7f5880153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f5885281640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f588014bcc0 0x7f5880154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f5885281640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58801590d0 con 0x7f58800be150 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f5885281640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5880158f50 con 0x7f58800c5910 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f5885281640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5880159250 con 0x7f588014bcc0 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58801e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58801e0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48474/0 (socket says 172.21.15.26:48474) 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 -- 172.21.15.26:0/655362072 learned_addr learned my addr 172.21.15.26:0/655362072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587f7fe640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f588014bcc0 0x7f5880154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587e7fc640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58800c5910 0x7f5880153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 -- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f588014bcc0 msgr2=0x7f5880154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f588014bcc0 0x7f5880154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 -- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58800c5910 msgr2=0x7f5880153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58800c5910 0x7f5880153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 -- 172.21.15.26:0/655362072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58801452f0 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f587effd640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58801e0f30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f587000efc0 tx=0x7f587000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f58777fe640 1 -- 172.21.15.26:0/655362072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870018020 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f58777fe640 1 -- 172.21.15.26:0/655362072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5870014680 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f58777fe640 1 -- 172.21.15.26:0/655362072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870014a50 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.046+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5880154bd0 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.047+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58801ebfc0 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.048+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5880154810 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.048+0000 7f58777fe640 1 -- 172.21.15.26:0/655362072 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5870005d10 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.048+0000 7f58777fe640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f586006b500 0x7f586006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.049+0000 7f58777fe640 1 -- 172.21.15.26:0/655362072 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f587008af20 con 0x7f58800be150 2024-03-06T07:24:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.049+0000 7f587e7fc640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f586006b500 0x7f586006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.049+0000 7f587e7fc640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f586006b500 0x7f586006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5868036000 tx=0x7f5868008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.053+0000 7f58777fe640 1 -- 172.21.15.26:0/655362072 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5870022030 con 0x7f58800be150 2024-03-06T07:24:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.232+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f588004f330 con 0x7f58800be150 2024-03-06T07:24:28.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.232+0000 7f58777fe640 1 -- 172.21.15.26:0/655362072 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5880154bd0 con 0x7f58800be150 2024-03-06T07:24:28.234 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f586006b500 msgr2=0x7f586006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f586006b500 0x7f586006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5868036000 tx=0x7f5868008030 comp rx=0 tx=0).stop 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 msgr2=0x7f58801e0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58801e0f30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f587000efc0 tx=0x7f587000c490 comp rx=0 tx=0).stop 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 shutdown_connections 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f586006b500 0x7f586006d9c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f588014bcc0 0x7f5880154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58800c5910 0x7f5880153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 --2- 172.21.15.26:0/655362072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58800be150 0x7f58801e0f30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 >> 172.21.15.26:0/655362072 conn(0x7f58800ac600 msgr2=0x7f588006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 shutdown_connections 2024-03-06T07:24:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.234+0000 7f5885281640 1 -- 172.21.15.26:0/655362072 wait complete. 2024-03-06T07:24:28.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:24:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.349+0000 7fb68da0e640 1 Processor -- start 2024-03-06T07:24:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb68da0e640 1 -- start start 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb68da0e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6881529f0 0x7fb6880c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb68da0e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6880c5370 0x7fb6880c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb68da0e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb688155530 0x7fb6881559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb68da0e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb688159710 con 0x7fb688155530 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb68da0e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6880c68b0 con 0x7fb6881529f0 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb68da0e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb688152dd0 con 0x7fb6880c5370 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb6877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb688155530 0x7fb6881559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb6877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb688155530 0x7fb6881559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48480/0 (socket says 172.21.15.26:48480) 2024-03-06T07:24:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb6877fe640 1 -- 172.21.15.26:0/2268247158 learned_addr learned my addr 172.21.15.26:0/2268247158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb6867fc640 1 --2- 172.21.15.26:0/2268247158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6880c5370 0x7fb6880c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.350+0000 7fb686ffd640 1 --2- 172.21.15.26:0/2268247158 >> v2:172.21.15.117:3300/0 conn(0x7fb6881529f0 0x7fb6880c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.351+0000 7fb6877fe640 1 -- 172.21.15.26:0/2268247158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6880c5370 msgr2=0x7fb6880c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.351+0000 7fb6877fe640 1 --2- 172.21.15.26:0/2268247158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6880c5370 0x7fb6880c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.351+0000 7fb6877fe640 1 -- 172.21.15.26:0/2268247158 >> v2:172.21.15.117:3300/0 conn(0x7fb6881529f0 msgr2=0x7fb6880c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.351+0000 7fb6877fe640 1 --2- 172.21.15.26:0/2268247158 >> v2:172.21.15.117:3300/0 conn(0x7fb6881529f0 0x7fb6880c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.351+0000 7fb6877fe640 1 -- 172.21.15.26:0/2268247158 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688158b00 con 0x7fb688155530 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.351+0000 7fb6867fc640 1 --2- 172.21.15.26:0/2268247158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6880c5370 0x7fb6880c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.351+0000 7fb6877fe640 1 --2- 172.21.15.26:0/2268247158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb688155530 0x7fb6881559c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb6880cba90 tx=0x7fb68002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af45d45b65dd37a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb685ffb640 1 -- 172.21.15.26:0/2268247158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68002c9e0 con 0x7fb688155530 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb685ffb640 1 -- 172.21.15.26:0/2268247158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68002cb80 con 0x7fb688155530 2024-03-06T07:24:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb685ffb640 1 -- 172.21.15.26:0/2268247158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6800343f0 con 0x7fb688155530 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 -- 172.21.15.26:0/2268247158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb688155530 msgr2=0x7fb6881559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 --2- 172.21.15.26:0/2268247158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb688155530 0x7fb6881559c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb6880cba90 tx=0x7fb68002be40 comp rx=0 tx=0).stop 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 -- 172.21.15.26:0/2268247158 shutdown_connections 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 --2- 172.21.15.26:0/2268247158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6880c5370 0x7fb6880c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 --2- 172.21.15.26:0/2268247158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb688155530 0x7fb6881559c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 --2- 172.21.15.26:0/2268247158 >> v2:172.21.15.117:3300/0 conn(0x7fb6881529f0 0x7fb6880c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 -- 172.21.15.26:0/2268247158 >> 172.21.15.26:0/2268247158 conn(0x7fb6880ac580 msgr2=0x7fb6880ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 -- 172.21.15.26:0/2268247158 shutdown_connections 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.352+0000 7fb68da0e640 1 -- 172.21.15.26:0/2268247158 wait complete. 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.353+0000 7fb68da0e640 1 Processor -- start 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.353+0000 7fb68da0e640 1 -- start start 2024-03-06T07:24:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb68da0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6880c5370 0x7fb6881ed630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb68da0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6881529f0 0x7fb688154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb68da0e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb688155530 0x7fb6881eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb68da0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881593a0 con 0x7fb6881529f0 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb68da0e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881590c0 con 0x7fb6880c5370 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb68da0e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb688158740 con 0x7fb688155530 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb6867fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6881529f0 0x7fb688154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb6867fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6881529f0 0x7fb688154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48488/0 (socket says 172.21.15.26:48488) 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb6867fc640 1 -- 172.21.15.26:0/3882632701 learned_addr learned my addr 172.21.15.26:0/3882632701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb686ffd640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6880c5370 0x7fb6881ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.354+0000 7fb6877fe640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb688155530 0x7fb6881eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb6867fc640 1 -- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb688155530 msgr2=0x7fb6881eafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb6867fc640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb688155530 0x7fb6881eafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb6867fc640 1 -- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6880c5370 msgr2=0x7fb6881ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb6867fc640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6880c5370 0x7fb6881ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb6867fc640 1 -- 172.21.15.26:0/3882632701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688148fa0 con 0x7fb6881529f0 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb6877fe640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb688155530 0x7fb6881eafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb6867fc640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6881529f0 0x7fb688154450 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00e9b0 tx=0x7fb67c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb66ffff640 1 -- 172.21.15.26:0/3882632701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c017070 con 0x7fb6881529f0 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb66ffff640 1 -- 172.21.15.26:0/3882632701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67c00ca00 con 0x7fb6881529f0 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6881eeae0 con 0x7fb6881529f0 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb66ffff640 1 -- 172.21.15.26:0/3882632701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00cdd0 con 0x7fb6881529f0 2024-03-06T07:24:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.355+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6880c4d00 con 0x7fb6881529f0 2024-03-06T07:24:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.356+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68804f330 con 0x7fb6881529f0 2024-03-06T07:24:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.356+0000 7fb66ffff640 1 -- 172.21.15.26:0/3882632701 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb67c01a070 con 0x7fb6881529f0 2024-03-06T07:24:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.357+0000 7fb66ffff640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb67406b500 0x7fb67406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.357+0000 7fb66ffff640 1 -- 172.21.15.26:0/3882632701 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb67c08a4b0 con 0x7fb6881529f0 2024-03-06T07:24:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.357+0000 7fb686ffd640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb67406b500 0x7fb67406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.357+0000 7fb686ffd640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb67406b500 0x7fb67406d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb6881520c0 tx=0x7fb678009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.361+0000 7fb66ffff640 1 -- 172.21.15.26:0/3882632701 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb67c058b90 con 0x7fb6881529f0 2024-03-06T07:24:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.539+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb68814db70 con 0x7fb6881529f0 2024-03-06T07:24:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.540+0000 7fb66ffff640 1 -- 172.21.15.26:0/3882632701 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb67c05cc30 con 0x7fb6881529f0 2024-03-06T07:24:28.542 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb67406b500 msgr2=0x7fb67406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb67406b500 0x7fb67406d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb6881520c0 tx=0x7fb678009190 comp rx=0 tx=0).stop 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6881529f0 msgr2=0x7fb688154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6881529f0 0x7fb688154450 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00e9b0 tx=0x7fb67c00ee70 comp rx=0 tx=0).stop 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 shutdown_connections 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb67406b500 0x7fb67406d9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb688155530 0x7fb6881eafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6881529f0 0x7fb688154450 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 --2- 172.21.15.26:0/3882632701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6880c5370 0x7fb6881ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 >> 172.21.15.26:0/3882632701 conn(0x7fb6880ac580 msgr2=0x7fb68814e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 shutdown_connections 2024-03-06T07:24:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.542+0000 7fb68da0e640 1 -- 172.21.15.26:0/3882632701 wait complete. 2024-03-06T07:24:28.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:24:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.656+0000 7f8c7fac9640 1 Processor -- start 2024-03-06T07:24:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7fac9640 1 -- start start 2024-03-06T07:24:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7fac9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c780b0070 0x7f8c78155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7fac9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780b0450 con 0x7f8c780c5380 2024-03-06T07:24:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7fac9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7806cf20 con 0x7f8c7806ca70 2024-03-06T07:24:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7fac9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7806c020 con 0x7f8c780b0070 2024-03-06T07:24:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7d865640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c780b0070 0x7f8c78155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7e066640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8c780c5380 0x7f8c7814e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35710/0 (socket says 172.21.15.26:35710) 2024-03-06T07:24:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.657+0000 7f8c7e066640 1 -- 172.21.15.26:0/2079099251 learned_addr learned my addr 172.21.15.26:0/2079099251 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.658+0000 7f8c7d865640 1 -- 172.21.15.26:0/2079099251 >> v1:172.21.15.117:6789/0 conn(0x7f8c7806ca70 legacy=0x7f8c780c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.658+0000 7f8c7d865640 1 -- 172.21.15.26:0/2079099251 >> v1:172.21.15.26:6789/0 conn(0x7f8c780c5380 legacy=0x7f8c7814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.658+0000 7f8c7c863640 1 -- 172.21.15.26:0/2079099251 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425385615 0 0) 0x7f8c68001490 con 0x7f8c780c5380 2024-03-06T07:24:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.658+0000 7f8c7d865640 1 -- 172.21.15.26:0/2079099251 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780c77f0 con 0x7f8c780b0070 2024-03-06T07:24:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7d865640 1 --2- 172.21.15.26:0/2079099251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780b0070 0x7f8c78155c30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8c700097a0 tx=0x7f8c7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d374197f889f06be server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7c863640 1 -- 172.21.15.26:0/2079099251 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70035020 con 0x7f8c780b0070 2024-03-06T07:24:28.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7c863640 1 -- 172.21.15.26:0/2079099251 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c70031830 con 0x7f8c780b0070 2024-03-06T07:24:28.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7c863640 1 -- 172.21.15.26:0/2079099251 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70031c00 con 0x7f8c780b0070 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2079099251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780b0070 msgr2=0x7f8c78155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2079099251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780b0070 0x7f8c78155c30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8c700097a0 tx=0x7f8c7002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2079099251 shutdown_connections 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2079099251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780b0070 0x7f8c78155c30 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2079099251 >> 172.21.15.26:0/2079099251 conn(0x7f8c780ac590 msgr2=0x7f8c780ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2079099251 shutdown_connections 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.659+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2079099251 wait complete. 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.660+0000 7f8c7fac9640 1 Processor -- start 2024-03-06T07:24:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.660+0000 7f8c7fac9640 1 -- start start 2024-03-06T07:24:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7fac9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7806ca70 0x7f8c781542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7fac9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c780b0070 0x7f8c78158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7fac9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780c5380 0x7f8c781eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7fac9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7806c550 con 0x7f8c780c5380 2024-03-06T07:24:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7fac9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7806c270 con 0x7f8c7806ca70 2024-03-06T07:24:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7fac9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7806da80 con 0x7f8c780b0070 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780c5380 0x7f8c781eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780c5380 0x7f8c781eb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48514/0 (socket says 172.21.15.26:48514) 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 -- 172.21.15.26:0/2443986723 learned_addr learned my addr 172.21.15.26:0/2443986723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7d865640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7806ca70 0x7f8c781542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7d064640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c780b0070 0x7f8c78158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 -- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c780b0070 msgr2=0x7f8c78158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c780b0070 0x7f8c78158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 -- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7806ca70 msgr2=0x7f8c781542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7806ca70 0x7f8c781542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.661+0000 7f8c7e066640 1 -- 172.21.15.26:0/2443986723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c78157d40 con 0x7f8c780c5380 2024-03-06T07:24:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.662+0000 7f8c7e066640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780c5380 0x7f8c781eb1b0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c820 tx=0x7f8c6800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.662+0000 7f8c767fc640 1 -- 172.21.15.26:0/2443986723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68018070 con 0x7f8c780c5380 2024-03-06T07:24:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.662+0000 7f8c767fc640 1 -- 172.21.15.26:0/2443986723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800a5e0 con 0x7f8c780c5380 2024-03-06T07:24:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.662+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c78158020 con 0x7f8c780c5380 2024-03-06T07:24:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.662+0000 7f8c767fc640 1 -- 172.21.15.26:0/2443986723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800a9d0 con 0x7f8c780c5380 2024-03-06T07:24:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.662+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c781547f0 con 0x7f8c780c5380 2024-03-06T07:24:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.663+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c78158540 con 0x7f8c780c5380 2024-03-06T07:24:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.663+0000 7f8c767fc640 1 -- 172.21.15.26:0/2443986723 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c6801b070 con 0x7f8c780c5380 2024-03-06T07:24:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.664+0000 7f8c767fc640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c6406b4d0 0x7f8c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.664+0000 7f8c767fc640 1 -- 172.21.15.26:0/2443986723 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8c680898d0 con 0x7f8c780c5380 2024-03-06T07:24:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.664+0000 7f8c7d865640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c6406b4d0 0x7f8c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.664+0000 7f8c7d865640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c6406b4d0 0x7f8c6406d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8c70036000 tx=0x7f8c70008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.668+0000 7f8c767fc640 1 -- 172.21.15.26:0/2443986723 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c78158540 con 0x7f8c780c5380 2024-03-06T07:24:28.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.852+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c7804f330 con 0x7f8c780c5380 2024-03-06T07:24:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.852+0000 7f8c767fc640 1 -- 172.21.15.26:0/2443986723 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c780b0450 con 0x7f8c780c5380 2024-03-06T07:24:28.854 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c6406b4d0 msgr2=0x7f8c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c6406b4d0 0x7f8c6406d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8c70036000 tx=0x7f8c70008030 comp rx=0 tx=0).stop 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780c5380 msgr2=0x7f8c781eb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780c5380 0x7f8c781eb1b0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c820 tx=0x7f8c6800cce0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 shutdown_connections 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c6406b4d0 0x7f8c6406d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c780c5380 0x7f8c781eb1b0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c780b0070 0x7f8c78158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 --2- 172.21.15.26:0/2443986723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7806ca70 0x7f8c781542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 >> 172.21.15.26:0/2443986723 conn(0x7f8c780ac590 msgr2=0x7f8c780ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 shutdown_connections 2024-03-06T07:24:28.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.854+0000 7f8c7fac9640 1 -- 172.21.15.26:0/2443986723 wait complete. 2024-03-06T07:24:28.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:28.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.966+0000 7ff6e8e8d640 1 Processor -- start 2024-03-06T07:24:28.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.966+0000 7ff6e8e8d640 1 -- start start 2024-03-06T07:24:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e8e8d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e4152520 0x7ff6e406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e8e8d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40c1ae0 con 0x7ff6e4155c80 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e8e8d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e406e450 con 0x7ff6e40720a0 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e8e8d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40c1dc0 con 0x7ff6e4152520 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e1d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff6e4155c80 0x7ff6e40c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35714/0 (socket says 172.21.15.26:35714) 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e1d9b640 1 -- 172.21.15.26:0/2483964519 learned_addr learned my addr 172.21.15.26:0/2483964519 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e259c640 1 --2- 172.21.15.26:0/2483964519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e4152520 0x7ff6e406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e159a640 1 -- 172.21.15.26:0/2483964519 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403156193 0 0) 0x7ff6e40c1ae0 con 0x7ff6e4155c80 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e159a640 1 -- 172.21.15.26:0/2483964519 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6cc003520 con 0x7ff6e4155c80 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e159a640 1 -- 172.21.15.26:0/2483964519 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153608235 0 0) 0x7ff6e406e450 con 0x7ff6e40720a0 2024-03-06T07:24:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e259c640 1 -- 172.21.15.26:0/2483964519 >> v1:172.21.15.117:6789/0 conn(0x7ff6e40720a0 legacy=0x7ff6e406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e259c640 1 -- 172.21.15.26:0/2483964519 >> v1:172.21.15.26:6789/0 conn(0x7ff6e4155c80 legacy=0x7ff6e40c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.967+0000 7ff6e259c640 1 -- 172.21.15.26:0/2483964519 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e406e730 con 0x7ff6e4152520 2024-03-06T07:24:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.968+0000 7ff6e259c640 1 --2- 172.21.15.26:0/2483964519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e4152520 0x7ff6e406ef30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009dc0 tx=0x7ff6dc02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81136bac3fb873a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.968+0000 7ff6e159a640 1 -- 172.21.15.26:0/2483964519 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc039070 con 0x7ff6e4152520 2024-03-06T07:24:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.968+0000 7ff6e159a640 1 -- 172.21.15.26:0/2483964519 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc02ccd0 con 0x7ff6e4152520 2024-03-06T07:24:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.968+0000 7ff6e159a640 1 -- 172.21.15.26:0/2483964519 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc034640 con 0x7ff6e4152520 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.969+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/2483964519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e4152520 msgr2=0x7ff6e406ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.969+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/2483964519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e4152520 0x7ff6e406ef30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009dc0 tx=0x7ff6dc02bf20 comp rx=0 tx=0).stop 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.969+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/2483964519 shutdown_connections 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.969+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/2483964519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e4152520 0x7ff6e406ef30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.969+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/2483964519 >> 172.21.15.26:0/2483964519 conn(0x7ff6e40c76b0 msgr2=0x7ff6e40c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.969+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/2483964519 shutdown_connections 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.969+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/2483964519 wait complete. 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.970+0000 7ff6e8e8d640 1 Processor -- start 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.970+0000 7ff6e8e8d640 1 -- start start 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e8e8d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40720a0 0x7ff6e414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e8e8d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e4152520 0x7ff6e40a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e8e8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4155c80 0x7ff6e4158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e8e8d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406b850 con 0x7ff6e4155c80 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e8e8d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406b6d0 con 0x7ff6e4152520 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e8e8d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4152900 con 0x7ff6e40720a0 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4155c80 0x7ff6e4158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40720a0 0x7ff6e414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4155c80 0x7ff6e4158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48522/0 (socket says 172.21.15.26:48522) 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e2d9d640 1 -- 172.21.15.26:0/3554478776 learned_addr learned my addr 172.21.15.26:0/3554478776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40720a0 0x7ff6e414e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:43548/0 (socket says 172.21.15.26:43548) 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e1d9b640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e4152520 0x7ff6e40a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 -- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e4152520 msgr2=0x7ff6e40a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e4152520 0x7ff6e40a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 -- 172.21.15.26:0/3554478776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4155c80 msgr2=0x7ff6e4158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4155c80 0x7ff6e4158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 -- 172.21.15.26:0/3554478776 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4152300 con 0x7ff6e40720a0 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e2d9d640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4155c80 0x7ff6e4158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.971+0000 7ff6e259c640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40720a0 0x7ff6e414e2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc033000 tx=0x7ff6dc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.972+0000 7ff6d2ffd640 1 -- 172.21.15.26:0/3554478776 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc039050 con 0x7ff6e40720a0 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.972+0000 7ff6d2ffd640 1 -- 172.21.15.26:0/3554478776 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc045070 con 0x7ff6e40720a0 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.972+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e41eb990 con 0x7ff6e40720a0 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.972+0000 7ff6d2ffd640 1 -- 172.21.15.26:0/3554478776 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc009450 con 0x7ff6e40720a0 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.972+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e4157a60 con 0x7ff6e40720a0 2024-03-06T07:24:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.973+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e404f330 con 0x7ff6e40720a0 2024-03-06T07:24:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.973+0000 7ff6d2ffd640 1 -- 172.21.15.26:0/3554478776 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6dc03e810 con 0x7ff6e40720a0 2024-03-06T07:24:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.973+0000 7ff6d2ffd640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff6c406b4d0 0x7ff6c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.974+0000 7ff6d2ffd640 1 -- 172.21.15.26:0/3554478776 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff6dc0acf50 con 0x7ff6e40720a0 2024-03-06T07:24:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.974+0000 7ff6e1d9b640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff6c406b4d0 0x7ff6c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.974+0000 7ff6e1d9b640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff6c406b4d0 0x7ff6c406d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4153bc0 tx=0x7ff6d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:28.982+0000 7ff6d2ffd640 1 -- 172.21.15.26:0/3554478776 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff6dc07b6b0 con 0x7ff6e40720a0 2024-03-06T07:24:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.201+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e4151380 con 0x7ff6e40720a0 2024-03-06T07:24:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.202+0000 7ff6d2ffd640 1 -- 172.21.15.26:0/3554478776 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4981 (secure 0 0 0) 0x7ff6dc07f750 con 0x7ff6e40720a0 2024-03-06T07:24:29.204 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:29.204 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7647,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/423809105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":423809105},{"type":"v1","addr":"172.21.15.26:6835","nonce":423809105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":97,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:23.745787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7084":{"gid":7084,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":7,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 98 2024-03-06T07:24:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff6c406b4d0 msgr2=0x7ff6c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff6c406b4d0 0x7ff6c406d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4153bc0 tx=0x7ff6d8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40720a0 msgr2=0x7ff6e414e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40720a0 0x7ff6e414e2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc033000 tx=0x7ff6dc004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 shutdown_connections 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff6c406b4d0 0x7ff6c406d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4155c80 0x7ff6e4158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e4152520 0x7ff6e40a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 --2- 172.21.15.26:0/3554478776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40720a0 0x7ff6e414e2a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 >> 172.21.15.26:0/3554478776 conn(0x7ff6e40c76b0 msgr2=0x7ff6e40c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 shutdown_connections 2024-03-06T07:24:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.204+0000 7ff6e8e8d640 1 -- 172.21.15.26:0/3554478776 wait complete. 2024-03-06T07:24:29.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.316+0000 7fa0d3215640 1 Processor -- start 2024-03-06T07:24:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.316+0000 7fa0d3215640 1 -- start start 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d3215640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0cc150580 0x7fa0cc06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d3215640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc153a60 con 0x7fa0cc0c1320 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d3215640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc153d40 con 0x7fa0cc0c6000 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d3215640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc06c2c0 con 0x7fa0cc150580 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d0fb1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0cc150580 0x7fa0cc06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0cbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa0cc0c1320 0x7fa0cc1566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35730/0 (socket says 172.21.15.26:35730) 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0cbfff640 1 -- 172.21.15.26:0/3032869545 learned_addr learned my addr 172.21.15.26:0/3032869545 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d0fb1640 1 -- 172.21.15.26:0/3032869545 >> v1:172.21.15.117:6789/0 conn(0x7fa0cc0c6000 legacy=0x7fa0cc156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d0fb1640 1 -- 172.21.15.26:0/3032869545 >> v1:172.21.15.26:6789/0 conn(0x7fa0cc0c1320 legacy=0x7fa0cc1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.317+0000 7fa0d0fb1640 1 -- 172.21.15.26:0/3032869545 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc06c5a0 con 0x7fa0cc150580 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.318+0000 7fa0d0fb1640 1 --2- 172.21.15.26:0/3032869545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc06dd10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40097a0 tx=0x7fa0c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b769011c3feeb110 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.318+0000 7fa0cb7fe640 1 -- 172.21.15.26:0/3032869545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4035020 con 0x7fa0cc150580 2024-03-06T07:24:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.318+0000 7fa0cb7fe640 1 -- 172.21.15.26:0/3032869545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c4031830 con 0x7fa0cc150580 2024-03-06T07:24:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.318+0000 7fa0cb7fe640 1 -- 172.21.15.26:0/3032869545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4031c00 con 0x7fa0cc150580 2024-03-06T07:24:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.319+0000 7fa0d3215640 1 -- 172.21.15.26:0/3032869545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 msgr2=0x7fa0cc06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.319+0000 7fa0d3215640 1 --2- 172.21.15.26:0/3032869545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc06dd10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40097a0 tx=0x7fa0c402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.319+0000 7fa0d3215640 1 -- 172.21.15.26:0/3032869545 shutdown_connections 2024-03-06T07:24:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.319+0000 7fa0d3215640 1 --2- 172.21.15.26:0/3032869545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc06dd10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.319+0000 7fa0d3215640 1 -- 172.21.15.26:0/3032869545 >> 172.21.15.26:0/3032869545 conn(0x7fa0cc0c6710 msgr2=0x7fa0cc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.319+0000 7fa0d3215640 1 -- 172.21.15.26:0/3032869545 shutdown_connections 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.319+0000 7fa0d3215640 1 -- 172.21.15.26:0/3032869545 wait complete. 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.320+0000 7fa0d3215640 1 Processor -- start 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.320+0000 7fa0d3215640 1 -- start start 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d3215640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0cc0c1320 0x7fa0cc14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d3215640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0cc0c6000 0x7fa0cc0aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d3215640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc0aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d3215640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0c0dc0 con 0x7fa0cc150580 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d3215640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0c0c40 con 0x7fa0cc0c6000 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d3215640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc06e250 con 0x7fa0cc0c1320 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc0aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc0aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48532/0 (socket says 172.21.15.26:48532) 2024-03-06T07:24:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 -- 172.21.15.26:0/1996395466 learned_addr learned my addr 172.21.15.26:0/1996395466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0cbfff640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0cc0c6000 0x7fa0cc0aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d0fb1640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0cc0c1320 0x7fa0cc14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 -- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0cc0c1320 msgr2=0x7fa0cc14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0cc0c1320 0x7fa0cc14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 -- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0cc0c6000 msgr2=0x7fa0cc0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0cc0c6000 0x7fa0cc0aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 -- 172.21.15.26:0/1996395466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0c0550 con 0x7fa0cc150580 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0d17b2640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc0aa590 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001eb0 tx=0x7fa0b400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.321+0000 7fa0c97fa640 1 -- 172.21.15.26:0/1996395466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4017070 con 0x7fa0cc150580 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.322+0000 7fa0c97fa640 1 -- 172.21.15.26:0/1996395466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b40052d0 con 0x7fa0cc150580 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.322+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc14b520 con 0x7fa0cc150580 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.322+0000 7fa0c97fa640 1 -- 172.21.15.26:0/1996395466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4003810 con 0x7fa0cc150580 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.322+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc14b7e0 con 0x7fa0cc150580 2024-03-06T07:24:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.322+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc14b330 con 0x7fa0cc150580 2024-03-06T07:24:29.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.323+0000 7fa0c97fa640 1 -- 172.21.15.26:0/1996395466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0b401a070 con 0x7fa0cc150580 2024-03-06T07:24:29.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.323+0000 7fa0c97fa640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0ac06b500 0x7fa0ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.323+0000 7fa0c97fa640 1 -- 172.21.15.26:0/1996395466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa0b4089610 con 0x7fa0cc150580 2024-03-06T07:24:29.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.324+0000 7fa0d0fb1640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0ac06b500 0x7fa0ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.324+0000 7fa0d0fb1640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0ac06b500 0x7fa0ac06d9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036000 tx=0x7fa0c403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.328+0000 7fa0c97fa640 1 -- 172.21.15.26:0/1996395466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa0b4057cf0 con 0x7fa0cc150580 2024-03-06T07:24:29.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.508+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0cc04f330 con 0x7fa0cc150580 2024-03-06T07:24:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.509+0000 7fa0c97fa640 1 -- 172.21.15.26:0/1996395466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17093 (secure 0 0 0) 0x7fa0b405bd90 con 0x7fa0cc150580 2024-03-06T07:24:29.511 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:29.511 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":109,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:13.704122+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:02.286561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:03.001111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1745266733":"2024-03-07T07:24:13.704095+0000","172.21.15.26:6833/1745266733":"2024-03-07T07:24:13.704095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:29.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0ac06b500 msgr2=0x7fa0ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0ac06b500 0x7fa0ac06d9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036000 tx=0x7fa0c403c040 comp rx=0 tx=0).stop 2024-03-06T07:24:29.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 msgr2=0x7fa0cc0aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc0aa590 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001eb0 tx=0x7fa0b400c890 comp rx=0 tx=0).stop 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 shutdown_connections 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0ac06b500 0x7fa0ac06d9c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0cc150580 0x7fa0cc0aa590 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0cc0c6000 0x7fa0cc0aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 --2- 172.21.15.26:0/1996395466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0cc0c1320 0x7fa0cc14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 >> 172.21.15.26:0/1996395466 conn(0x7fa0cc0c6710 msgr2=0x7fa0cc0a7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 shutdown_connections 2024-03-06T07:24:29.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.511+0000 7fa0d3215640 1 -- 172.21.15.26:0/1996395466 wait complete. 2024-03-06T07:24:29.524 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:24:29.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.626+0000 7fec83246640 1 Processor -- start 2024-03-06T07:24:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.626+0000 7fec83246640 1 -- start start 2024-03-06T07:24:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.626+0000 7fec83246640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec7c06d9f0 0x7fec7c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec83246640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec7c0c3280 0x7fec7c0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec83246640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec83246640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c158800 con 0x7fec7c0c3280 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec83246640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c158ae0 con 0x7fec7c06d9f0 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec83246640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c158dc0 con 0x7fec7c1551e0 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec7c0c3280 0x7fec7c0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec7c0c3280 0x7fec7c0c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48540/0 (socket says 172.21.15.26:48540) 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 -- 172.21.15.26:0/2809861915 learned_addr learned my addr 172.21.15.26:0/2809861915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec817e3640 1 --2- 172.21.15.26:0/2809861915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec80fe2640 1 --2- 172.21.15.26:0/2809861915 >> v2:172.21.15.117:3300/0 conn(0x7fec7c06d9f0 0x7fec7c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 -- 172.21.15.26:0/2809861915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 msgr2=0x7fec7c155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 --2- 172.21.15.26:0/2809861915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 -- 172.21.15.26:0/2809861915 >> v2:172.21.15.117:3300/0 conn(0x7fec7c06d9f0 msgr2=0x7fec7c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 --2- 172.21.15.26:0/2809861915 >> v2:172.21.15.117:3300/0 conn(0x7fec7c06d9f0 0x7fec7c0a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.627+0000 7fec7bfff640 1 -- 172.21.15.26:0/2809861915 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c1590a0 con 0x7fec7c0c3280 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.628+0000 7fec7bfff640 1 --2- 172.21.15.26:0/2809861915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0c3280 0x7fec7c0c3710 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c9820 tx=0x7fec6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb3787ff68b189d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.628+0000 7fec7b7fe640 1 -- 172.21.15.26:0/2809861915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6802e070 con 0x7fec7c0c3280 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.628+0000 7fec7b7fe640 1 -- 172.21.15.26:0/2809861915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68037070 con 0x7fec7c0c3280 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.628+0000 7fec7b7fe640 1 -- 172.21.15.26:0/2809861915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68031a00 con 0x7fec7c0c3280 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.628+0000 7fec83246640 1 -- 172.21.15.26:0/2809861915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0c3280 msgr2=0x7fec7c0c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.628+0000 7fec83246640 1 --2- 172.21.15.26:0/2809861915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0c3280 0x7fec7c0c3710 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c9820 tx=0x7fec6802b880 comp rx=0 tx=0).stop 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.629+0000 7fec83246640 1 -- 172.21.15.26:0/2809861915 shutdown_connections 2024-03-06T07:24:29.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.629+0000 7fec83246640 1 --2- 172.21.15.26:0/2809861915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.629+0000 7fec83246640 1 --2- 172.21.15.26:0/2809861915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0c3280 0x7fec7c0c3710 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.629+0000 7fec83246640 1 --2- 172.21.15.26:0/2809861915 >> v2:172.21.15.117:3300/0 conn(0x7fec7c06d9f0 0x7fec7c0a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.629+0000 7fec83246640 1 -- 172.21.15.26:0/2809861915 >> 172.21.15.26:0/2809861915 conn(0x7fec7c0aa630 msgr2=0x7fec7c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.629+0000 7fec83246640 1 -- 172.21.15.26:0/2809861915 shutdown_connections 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.629+0000 7fec83246640 1 -- 172.21.15.26:0/2809861915 wait complete. 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 Processor -- start 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 -- start start 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c06d9f0 0x7fec7c0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7c0c3280 0x7fec7c1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c06df50 con 0x7fec7c06d9f0 2024-03-06T07:24:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c06ddd0 con 0x7fec7c0c3280 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.630+0000 7fec83246640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0a8210 con 0x7fec7c1551e0 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c06d9f0 0x7fec7c0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c06d9f0 0x7fec7c0c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48550/0 (socket says 172.21.15.26:48550) 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 -- 172.21.15.26:0/634515982 learned_addr learned my addr 172.21.15.26:0/634515982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec7bfff640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7c0c3280 0x7fec7c1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec817e3640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 -- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 msgr2=0x7fec7c142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 -- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7c0c3280 msgr2=0x7fec7c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7c0c3280 0x7fec7c1417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 -- 172.21.15.26:0/634515982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c06f460 con 0x7fec7c06d9f0 2024-03-06T07:24:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec80fe2640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c06d9f0 0x7fec7c0c7bd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fec7000e870 tx=0x7fec7000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec797fa640 1 -- 172.21.15.26:0/634515982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70017070 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec797fa640 1 -- 172.21.15.26:0/634515982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7000c940 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c1e6cb0 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec797fa640 1 -- 172.21.15.26:0/634515982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000cd30 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.631+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c06f6e0 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.632+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c04f330 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.633+0000 7fec797fa640 1 -- 172.21.15.26:0/634515982 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec7001a070 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.633+0000 7fec797fa640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fec6006b4d0 0x7fec6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.633+0000 7fec797fa640 1 -- 172.21.15.26:0/634515982 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fec7008a5b0 con 0x7fec7c06d9f0 2024-03-06T07:24:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.633+0000 7fec7bfff640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fec6006b4d0 0x7fec6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.634+0000 7fec7bfff640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fec6006b4d0 0x7fec6006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fec68036000 tx=0x7fec6803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.638+0000 7fec797fa640 1 -- 172.21.15.26:0/634515982 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fec7c06f460 con 0x7fec7c06d9f0 2024-03-06T07:24:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.857+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7c073240 con 0x7fec7c06d9f0 2024-03-06T07:24:29.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.857+0000 7fec797fa640 1 -- 172.21.15.26:0/634515982 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4981 (secure 0 0 0) 0x7fec70058c00 con 0x7fec7c06d9f0 2024-03-06T07:24:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:29.859 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7647,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/423809105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":423809105},{"type":"v1","addr":"172.21.15.26:6835","nonce":423809105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":97,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:23.745787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7084":{"gid":7084,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":7,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:29.859 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 98 2024-03-06T07:24:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fec6006b4d0 msgr2=0x7fec6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fec6006b4d0 0x7fec6006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fec68036000 tx=0x7fec6803d040 comp rx=0 tx=0).stop 2024-03-06T07:24:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c06d9f0 msgr2=0x7fec7c0c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c06d9f0 0x7fec7c0c7bd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fec7000e870 tx=0x7fec7000ed30 comp rx=0 tx=0).stop 2024-03-06T07:24:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 shutdown_connections 2024-03-06T07:24:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fec6006b4d0 0x7fec6006d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec7c1551e0 0x7fec7c142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7c0c3280 0x7fec7c1417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 --2- 172.21.15.26:0/634515982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c06d9f0 0x7fec7c0c7bd0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 >> 172.21.15.26:0/634515982 conn(0x7fec7c0aa630 msgr2=0x7fec7c14fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 shutdown_connections 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.859+0000 7fec83246640 1 -- 172.21.15.26:0/634515982 wait complete. 2024-03-06T07:24:29.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:29.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00b22d4640 1 Processor -- start 2024-03-06T07:24:29.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00b22d4640 1 -- start start 2024-03-06T07:24:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00b22d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00ac1550c0 0x7f00ac1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00b22d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00b22d4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac0aa660 con 0x7f00ac0aa180 2024-03-06T07:24:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00b22d4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0c4610 con 0x7f00ac1550c0 2024-03-06T07:24:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00b22d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0aa940 con 0x7f00ac1559e0 2024-03-06T07:24:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.972+0000 7f00abfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00ac1550c0 0x7f00ac1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00abfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00ac1550c0 0x7f00ac1554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48554/0 (socket says 172.21.15.26:48554) 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00abfff640 1 -- 172.21.15.26:0/3472564735 learned_addr learned my addr 172.21.15.26:0/3472564735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00ab7fe640 1 --2- 172.21.15.26:0/3472564735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00abfff640 1 -- 172.21.15.26:0/3472564735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 msgr2=0x7f00ac14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00abfff640 1 --2- 172.21.15.26:0/3472564735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00abfff640 1 -- 172.21.15.26:0/3472564735 >> v1:172.21.15.117:6789/0 conn(0x7f00ac0aa180 legacy=0x7f00ac0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00abfff640 1 -- 172.21.15.26:0/3472564735 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0c4860 con 0x7f00ac1550c0 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.973+0000 7f00aaffd640 1 -- 172.21.15.26:0/3472564735 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899121916 0 0) 0x7f00ac0aa660 con 0x7f00ac0aa180 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00abfff640 1 --2- 172.21.15.26:0/3472564735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1554a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f009c00bb00 tx=0x7f009c02de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86183f45ab652fae server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00aaffd640 1 -- 172.21.15.26:0/3472564735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c034070 con 0x7f00ac1550c0 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00aaffd640 1 -- 172.21.15.26:0/3472564735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c02ebe0 con 0x7f00ac1550c0 2024-03-06T07:24:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00aaffd640 1 -- 172.21.15.26:0/3472564735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c038400 con 0x7f00ac1550c0 2024-03-06T07:24:29.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 -- 172.21.15.26:0/3472564735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 msgr2=0x7f00ac1554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 --2- 172.21.15.26:0/3472564735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1554a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f009c00bb00 tx=0x7f009c02de30 comp rx=0 tx=0).stop 2024-03-06T07:24:29.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 -- 172.21.15.26:0/3472564735 shutdown_connections 2024-03-06T07:24:29.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 --2- 172.21.15.26:0/3472564735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 --2- 172.21.15.26:0/3472564735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1554a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 -- 172.21.15.26:0/3472564735 >> 172.21.15.26:0/3472564735 conn(0x7f00ac0c6710 msgr2=0x7f00ac0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 -- 172.21.15.26:0/3472564735 shutdown_connections 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.974+0000 7f00b22d4640 1 -- 172.21.15.26:0/3472564735 wait complete. 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.975+0000 7f00b22d4640 1 Processor -- start 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b22d4640 1 -- start start 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b22d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ac0aa180 0x7f00ac0a7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b22d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1ea150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b22d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b22d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac14f140 con 0x7f00ac1550c0 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b22d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0c1520 con 0x7f00ac0aa180 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b22d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac14f2c0 con 0x7f00ac1559e0 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1ea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48560/0 (socket says 172.21.15.26:48560) 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00ab7fe640 1 -- 172.21.15.26:0/305222229 learned_addr learned my addr 172.21.15.26:0/305222229 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00abfff640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ac0aa180 0x7f00ac0a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.976+0000 7f00b0871640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00ab7fe640 1 -- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 msgr2=0x7f00ac14b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00ab7fe640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00ab7fe640 1 -- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ac0aa180 msgr2=0x7f00ac0a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00ab7fe640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ac0aa180 0x7f00ac0a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00ab7fe640 1 -- 172.21.15.26:0/305222229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0a8c80 con 0x7f00ac1550c0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00ab7fe640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1ea150 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e870 tx=0x7f00a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00a8ff9640 1 -- 172.21.15.26:0/305222229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0017070 con 0x7f00ac1550c0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00a8ff9640 1 -- 172.21.15.26:0/305222229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000c940 con 0x7f00ac1550c0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac0a8f60 con 0x7f00ac1550c0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00a8ff9640 1 -- 172.21.15.26:0/305222229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000cd30 con 0x7f00ac1550c0 2024-03-06T07:24:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.977+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac0a6ee0 con 0x7f00ac1550c0 2024-03-06T07:24:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.978+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac04f330 con 0x7f00ac1550c0 2024-03-06T07:24:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.978+0000 7f00a8ff9640 1 -- 172.21.15.26:0/305222229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00a001a070 con 0x7f00ac1550c0 2024-03-06T07:24:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.979+0000 7f00a8ff9640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f009806b4d0 0x7f009806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.979+0000 7f00a8ff9640 1 -- 172.21.15.26:0/305222229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f00a008a180 con 0x7f00ac1550c0 2024-03-06T07:24:29.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.979+0000 7f00abfff640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f009806b4d0 0x7f009806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.979+0000 7f00abfff640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f009806b4d0 0x7f009806d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f009c039000 tx=0x7f009c034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:29.983+0000 7f00a8ff9640 1 -- 172.21.15.26:0/305222229 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f00a0058860 con 0x7f00ac1550c0 2024-03-06T07:24:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.201+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00ac152a80 con 0x7f00ac1550c0 2024-03-06T07:24:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.202+0000 7f00a8ff9640 1 -- 172.21.15.26:0/305222229 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4981 (secure 0 0 0) 0x7f00a005c900 con 0x7f00ac1550c0 2024-03-06T07:24:30.203 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:30.203 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7647,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/423809105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":423809105},{"type":"v1","addr":"172.21.15.26:6835","nonce":423809105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":97,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:24:04.037603+0000","modified":"2024-03-06T07:24:23.745787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6305":{"gid":6305,"name":"b","rank":0,"incarnation":91,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6835/548706796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":548706796},{"type":"v1","addr":"172.21.15.117:6835","nonce":548706796}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7084":{"gid":7084,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":7,"addr":"172.21.15.117:6837/3362094815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3362094815},{"type":"v1","addr":"172.21.15.117:6837","nonce":3362094815}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6305,"qdb_cluster":[6305]},"id":8}]} 2024-03-06T07:24:30.204 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 98 2024-03-06T07:24:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f009806b4d0 msgr2=0x7f009806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f009806b4d0 0x7f009806d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f009c039000 tx=0x7f009c034000 comp rx=0 tx=0).stop 2024-03-06T07:24:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 msgr2=0x7f00ac1ea150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1ea150 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e870 tx=0x7f00a000ed30 comp rx=0 tx=0).stop 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 shutdown_connections 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f009806b4d0 0x7f009806d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00ac1559e0 0x7f00ac14b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1550c0 0x7f00ac1ea150 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 --2- 172.21.15.26:0/305222229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ac0aa180 0x7f00ac0a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.203+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 >> 172.21.15.26:0/305222229 conn(0x7f00ac0c6710 msgr2=0x7f00ac1539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.204+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 shutdown_connections 2024-03-06T07:24:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.204+0000 7f00b22d4640 1 -- 172.21.15.26:0/305222229 wait complete. 2024-03-06T07:24:30.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.316+0000 7f5bc19c9640 1 Processor -- start 2024-03-06T07:24:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.316+0000 7f5bc19c9640 1 -- start start 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bc19c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5bbc158db0 0x7f5bbc0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bc19c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc155110 con 0x7f5bbc0c5330 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bc19c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc0c8f40 con 0x7f5bbc0c88b0 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bc19c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc1559e0 con 0x7f5bbc158db0 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bba7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5bbc158db0 0x7f5bbc0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bbaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5bbc0c5330 0x7f5bbc1586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35742/0 (socket says 172.21.15.26:35742) 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bbaffd640 1 -- 172.21.15.26:0/3300150760 learned_addr learned my addr 172.21.15.26:0/3300150760 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bba7fc640 1 -- 172.21.15.26:0/3300150760 >> v1:172.21.15.117:6789/0 conn(0x7f5bbc0c88b0 legacy=0x7f5bbc15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bba7fc640 1 -- 172.21.15.26:0/3300150760 >> v1:172.21.15.26:6789/0 conn(0x7f5bbc0c5330 legacy=0x7f5bbc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bb9ffb640 1 -- 172.21.15.26:0/3300150760 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815712304 0 0) 0x7f5bbc0c8f40 con 0x7f5bbc0c88b0 2024-03-06T07:24:30.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.317+0000 7f5bba7fc640 1 -- 172.21.15.26:0/3300150760 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc155390 con 0x7f5bbc158db0 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.318+0000 7f5bba7fc640 1 --2- 172.21.15.26:0/3300150760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc158db0 0x7f5bbc0c5710 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40097a0 tx=0x7f5ba402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1dff6efa65155c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.318+0000 7f5bb9ffb640 1 -- 172.21.15.26:0/3300150760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4037070 con 0x7f5bbc158db0 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.318+0000 7f5bb9ffb640 1 -- 172.21.15.26:0/3300150760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4030710 con 0x7f5bbc158db0 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.318+0000 7f5bb9ffb640 1 -- 172.21.15.26:0/3300150760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4030ae0 con 0x7f5bbc158db0 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.319+0000 7f5bc19c9640 1 -- 172.21.15.26:0/3300150760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc158db0 msgr2=0x7f5bbc0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.319+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/3300150760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc158db0 0x7f5bbc0c5710 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40097a0 tx=0x7f5ba402b710 comp rx=0 tx=0).stop 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.319+0000 7f5bc19c9640 1 -- 172.21.15.26:0/3300150760 shutdown_connections 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.319+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/3300150760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc158db0 0x7f5bbc0c5710 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.319+0000 7f5bc19c9640 1 -- 172.21.15.26:0/3300150760 >> 172.21.15.26:0/3300150760 conn(0x7f5bbc0ac560 msgr2=0x7f5bbc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.319+0000 7f5bc19c9640 1 -- 172.21.15.26:0/3300150760 shutdown_connections 2024-03-06T07:24:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.319+0000 7f5bc19c9640 1 -- 172.21.15.26:0/3300150760 wait complete. 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.320+0000 7f5bc19c9640 1 Processor -- start 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.320+0000 7f5bc19c9640 1 -- start start 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc0c5330 0x7f5bbc14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bbc0c88b0 0x7f5bbc0c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bbc1577c0 0x7f5bbc14bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc1529d0 con 0x7f5bbc0c5330 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc06d090 con 0x7f5bbc0c88b0 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc152b50 con 0x7f5bbc1577c0 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc0c5330 0x7f5bbc14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc0c5330 0x7f5bbc14e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48574/0 (socket says 172.21.15.26:48574) 2024-03-06T07:24:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 -- 172.21.15.26:0/75522298 learned_addr learned my addr 172.21.15.26:0/75522298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bba7fc640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bbc0c88b0 0x7f5bbc0c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbb7fe640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bbc1577c0 0x7f5bbc14bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 -- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bbc1577c0 msgr2=0x7f5bbc14bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bbc1577c0 0x7f5bbc14bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 -- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bbc0c88b0 msgr2=0x7f5bbc0c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bbc0c88b0 0x7f5bbc0c1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 -- 172.21.15.26:0/75522298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc1ee160 con 0x7f5bbc0c5330 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbb7fe640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bbc1577c0 0x7f5bbc14bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bbaffd640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc0c5330 0x7f5bbc14e030 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00cc40 tx=0x7f5bac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bb3fff640 1 -- 172.21.15.26:0/75522298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac018070 con 0x7f5bbc0c5330 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bb3fff640 1 -- 172.21.15.26:0/75522298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac004d60 con 0x7f5bbc0c5330 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc1eacf0 con 0x7f5bbc0c5330 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bb3fff640 1 -- 172.21.15.26:0/75522298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00ad20 con 0x7f5bbc0c5330 2024-03-06T07:24:30.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.321+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc14e570 con 0x7f5bbc0c5330 2024-03-06T07:24:30.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.323+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc04f330 con 0x7f5bbc0c5330 2024-03-06T07:24:30.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.323+0000 7f5bb3fff640 1 -- 172.21.15.26:0/75522298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bac01b070 con 0x7f5bbc0c5330 2024-03-06T07:24:30.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.323+0000 7f5bb3fff640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.323+0000 7f5bb3fff640 1 -- 172.21.15.26:0/75522298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5bac089ca0 con 0x7f5bbc0c5330 2024-03-06T07:24:30.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.323+0000 7f5bba7fc640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.324+0000 7f5bba7fc640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4034000 tx=0x7f5ba4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.327+0000 7f5bb3fff640 1 -- 172.21.15.26:0/75522298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5bbc155110 con 0x7f5bbc0c5330 2024-03-06T07:24:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.509+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bbc06cdc0 con 0x7f5bbc0c5330 2024-03-06T07:24:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.511+0000 7f5bb3fff640 1 -- 172.21.15.26:0/75522298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17093 (secure 0 0 0) 0x7f5bac058380 con 0x7f5bbc0c5330 2024-03-06T07:24:30.512 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:30.512 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":109,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:13.704122+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:02.286561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:03.001111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1745266733":"2024-03-07T07:24:13.704095+0000","172.21.15.26:6833/1745266733":"2024-03-07T07:24:13.704095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.512+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b9c06b4d0 msgr2=0x7f5b9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.512+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4034000 tx=0x7f5ba4008030 comp rx=0 tx=0).stop 2024-03-06T07:24:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.512+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc0c5330 msgr2=0x7f5bbc14e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.512+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc0c5330 0x7f5bbc14e030 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00cc40 tx=0x7f5bac00a430 comp rx=0 tx=0).stop 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 shutdown_connections 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bbc1577c0 0x7f5bbc14bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bbc0c88b0 0x7f5bbc0c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 --2- 172.21.15.26:0/75522298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc0c5330 0x7f5bbc14e030 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 >> 172.21.15.26:0/75522298 conn(0x7f5bbc0ac560 msgr2=0x7f5bbc0ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 shutdown_connections 2024-03-06T07:24:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.513+0000 7f5bc19c9640 1 -- 172.21.15.26:0/75522298 wait complete. 2024-03-06T07:24:30.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:24:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.626+0000 7faf4dd5e640 1 Processor -- start 2024-03-06T07:24:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.626+0000 7faf4dd5e640 1 -- start start 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf4dd5e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf480c69f0 0x7faf480c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf4dd5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48155130 con 0x7faf480c8950 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf4dd5e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf480ad480 con 0x7faf481558d0 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf4dd5e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf48155440 con 0x7faf480c69f0 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf477fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf480c69f0 0x7faf480c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf46ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf480c8950 0x7faf480acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35754/0 (socket says 172.21.15.26:35754) 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf47fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7faf481558d0 0x7faf4815fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:35996/0 (socket says 172.21.15.26:35996) 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf46ffd640 1 -- 172.21.15.26:0/1475109681 learned_addr learned my addr 172.21.15.26:0/1475109681 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf467fc640 1 -- 172.21.15.26:0/1475109681 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813504328 0 0) 0x7faf480ad480 con 0x7faf481558d0 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf477fe640 1 -- 172.21.15.26:0/1475109681 >> v1:172.21.15.117:6789/0 conn(0x7faf481558d0 legacy=0x7faf4815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf477fe640 1 -- 172.21.15.26:0/1475109681 >> v1:172.21.15.26:6789/0 conn(0x7faf480c8950 legacy=0x7faf480acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.627+0000 7faf477fe640 1 -- 172.21.15.26:0/1475109681 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf480c5d50 con 0x7faf480c69f0 2024-03-06T07:24:30.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.628+0000 7faf477fe640 1 --2- 172.21.15.26:0/1475109681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf480c6dd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7faf380097a0 tx=0x7faf3802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f9105cb95fae9e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.628+0000 7faf467fc640 1 -- 172.21.15.26:0/1475109681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38037070 con 0x7faf480c69f0 2024-03-06T07:24:30.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.628+0000 7faf467fc640 1 -- 172.21.15.26:0/1475109681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf38030710 con 0x7faf480c69f0 2024-03-06T07:24:30.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.628+0000 7faf467fc640 1 -- 172.21.15.26:0/1475109681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38030ae0 con 0x7faf480c69f0 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.629+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1475109681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 msgr2=0x7faf480c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.629+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1475109681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf480c6dd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7faf380097a0 tx=0x7faf3802b710 comp rx=0 tx=0).stop 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.629+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1475109681 shutdown_connections 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.629+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1475109681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf480c6dd0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.629+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1475109681 >> 172.21.15.26:0/1475109681 conn(0x7faf480cae70 msgr2=0x7faf480cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.629+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1475109681 shutdown_connections 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.629+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1475109681 wait complete. 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.630+0000 7faf4dd5e640 1 Processor -- start 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.630+0000 7faf4dd5e640 1 -- start start 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf4dd5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf48149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf4dd5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf480c8950 0x7faf4806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf4dd5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf481558d0 0x7faf48158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf4dd5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48152a20 con 0x7faf480c69f0 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf4dd5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf480c6310 con 0x7faf480c8950 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf4dd5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf48152ba0 con 0x7faf481558d0 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf48149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf48149490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48600/0 (socket says 172.21.15.26:48600) 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 -- 172.21.15.26:0/1204269595 learned_addr learned my addr 172.21.15.26:0/1204269595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf47fff640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf481558d0 0x7faf48158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf46ffd640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf480c8950 0x7faf4806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 -- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf481558d0 msgr2=0x7faf48158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf481558d0 0x7faf48158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 -- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf480c8950 msgr2=0x7faf4806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf480c8950 0x7faf4806c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.631+0000 7faf477fe640 1 -- 172.21.15.26:0/1204269595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf481491b0 con 0x7faf480c69f0 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.632+0000 7faf477fe640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf48149490 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faf380098c0 tx=0x7faf380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.632+0000 7faf2ffff640 1 -- 172.21.15.26:0/1204269595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38042070 con 0x7faf480c69f0 2024-03-06T07:24:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.632+0000 7faf2ffff640 1 -- 172.21.15.26:0/1204269595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf38032a40 con 0x7faf480c69f0 2024-03-06T07:24:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.632+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf480c2550 con 0x7faf480c69f0 2024-03-06T07:24:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.632+0000 7faf2ffff640 1 -- 172.21.15.26:0/1204269595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38037040 con 0x7faf480c69f0 2024-03-06T07:24:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.632+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4806ce40 con 0x7faf480c69f0 2024-03-06T07:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.633+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf480c1500 con 0x7faf480c69f0 2024-03-06T07:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.633+0000 7faf2ffff640 1 -- 172.21.15.26:0/1204269595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf38045070 con 0x7faf480c69f0 2024-03-06T07:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.634+0000 7faf2ffff640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf3406b4d0 0x7faf3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.634+0000 7faf2ffff640 1 -- 172.21.15.26:0/1204269595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faf380ac3a0 con 0x7faf480c69f0 2024-03-06T07:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.634+0000 7faf46ffd640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf3406b4d0 0x7faf3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.634+0000 7faf46ffd640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf3406b4d0 0x7faf3406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faf3c000f60 tx=0x7faf3c007e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.638+0000 7faf2ffff640 1 -- 172.21.15.26:0/1204269595 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf480c1500 con 0x7faf480c69f0 2024-03-06T07:24:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.859+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf4804f330 con 0x7faf480c69f0 2024-03-06T07:24:30.861 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:24:30.860+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:24:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.861+0000 7faf2ffff640 1 -- 172.21.15.26:0/1204269595 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf3807ab00 con 0x7faf480c69f0 2024-03-06T07:24:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf3406b4d0 msgr2=0x7faf3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf3406b4d0 0x7faf3406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faf3c000f60 tx=0x7faf3c007e20 comp rx=0 tx=0).stop 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 msgr2=0x7faf48149490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf48149490 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faf380098c0 tx=0x7faf380046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 shutdown_connections 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf3406b4d0 0x7faf3406d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf481558d0 0x7faf48158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf480c8950 0x7faf4806c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 --2- 172.21.15.26:0/1204269595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf480c69f0 0x7faf48149490 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 >> 172.21.15.26:0/1204269595 conn(0x7faf480cae70 msgr2=0x7faf480cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 shutdown_connections 2024-03-06T07:24:30.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.863+0000 7faf4dd5e640 1 -- 172.21.15.26:0/1204269595 wait complete. 2024-03-06T07:24:30.869 INFO:tasks.ceph.mds.b.smithi117.stderr: -477> 2024-03-06T07:24:18.961+0000 7f38f6dbd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:30.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:24:30.888 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:24:30.893 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:24:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.977+0000 7f632a4a4640 1 Processor -- start 2024-03-06T07:24:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.977+0000 7f632a4a4640 1 -- start start 2024-03-06T07:24:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f632a4a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63240726b0 0x7f63240c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f632a4a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f632406b6a0 0x7f632406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f632a4a4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63240c3bc0 con 0x7f6324153fe0 2024-03-06T07:24:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f632a4a4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63240c3990 con 0x7f632406b6a0 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f632a4a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63241543c0 con 0x7f63240726b0 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f6323fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6324153fe0 0x7f6324071fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35756/0 (socket says 172.21.15.26:35756) 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f6323fff640 1 -- 172.21.15.26:0/967974447 learned_addr learned my addr 172.21.15.26:0/967974447 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f6328a41640 1 --2- 172.21.15.26:0/967974447 >> v2:172.21.15.26:3300/0 conn(0x7f632406b6a0 0x7f632406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.978+0000 7f63237fe640 1 --2- 172.21.15.26:0/967974447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63240726b0 0x7f63240c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.979+0000 7f6328a41640 1 -- 172.21.15.26:0/967974447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63240726b0 msgr2=0x7f63240c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.979+0000 7f6328a41640 1 --2- 172.21.15.26:0/967974447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63240726b0 0x7f63240c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.979+0000 7f6328a41640 1 -- 172.21.15.26:0/967974447 >> v1:172.21.15.26:6789/0 conn(0x7f6324153fe0 legacy=0x7f6324071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.979+0000 7f6322ffd640 1 -- 172.21.15.26:0/967974447 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795337437 0 0) 0x7f63240c3bc0 con 0x7f6324153fe0 2024-03-06T07:24:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.979+0000 7f6328a41640 1 -- 172.21.15.26:0/967974447 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240714c0 con 0x7f632406b6a0 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.979+0000 7f6328a41640 1 --2- 172.21.15.26:0/967974447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f632406b6a0 0x7f632406bb10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f631c009d90 tx=0x7f631c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48b2612e4b88778 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f6322ffd640 1 -- 172.21.15.26:0/967974447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c039070 con 0x7f632406b6a0 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f6322ffd640 1 -- 172.21.15.26:0/967974447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c02cbc0 con 0x7f632406b6a0 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f6322ffd640 1 -- 172.21.15.26:0/967974447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c034440 con 0x7f632406b6a0 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 -- 172.21.15.26:0/967974447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f632406b6a0 msgr2=0x7f632406bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 --2- 172.21.15.26:0/967974447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f632406b6a0 0x7f632406bb10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f631c009d90 tx=0x7f631c02be10 comp rx=0 tx=0).stop 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 -- 172.21.15.26:0/967974447 shutdown_connections 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 --2- 172.21.15.26:0/967974447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f632406b6a0 0x7f632406bb10 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 --2- 172.21.15.26:0/967974447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63240726b0 0x7f63240c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 -- 172.21.15.26:0/967974447 >> 172.21.15.26:0/967974447 conn(0x7f63240a94d0 msgr2=0x7f63240a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 -- 172.21.15.26:0/967974447 shutdown_connections 2024-03-06T07:24:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.980+0000 7f632a4a4640 1 -- 172.21.15.26:0/967974447 wait complete. 2024-03-06T07:24:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.981+0000 7f632a4a4640 1 Processor -- start 2024-03-06T07:24:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.981+0000 7f632a4a4640 1 -- start start 2024-03-06T07:24:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f632a4a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f632406b6a0 0x7f632414fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f632a4a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63240726b0 0x7f6324140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f632a4a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6324153fe0 0x7f6324140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f632a4a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63240c3ea0 con 0x7f63240726b0 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f632a4a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6324071b10 con 0x7f6324153fe0 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f632a4a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63240c5620 con 0x7f632406b6a0 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63240726b0 0x7f6324140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63240726b0 0x7f6324140960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48608/0 (socket says 172.21.15.26:48608) 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 -- 172.21.15.26:0/169919493 learned_addr learned my addr 172.21.15.26:0/169919493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f6328a41640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6324153fe0 0x7f6324140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f6323fff640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f632406b6a0 0x7f632414fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 -- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f632406b6a0 msgr2=0x7f632414fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f632406b6a0 0x7f632414fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 -- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6324153fe0 msgr2=0x7f6324140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6324153fe0 0x7f6324140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.982+0000 7f63237fe640 1 -- 172.21.15.26:0/169919493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63241e9a40 con 0x7f63240726b0 2024-03-06T07:24:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.983+0000 7f63237fe640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63240726b0 0x7f6324140960 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f631400e9e0 tx=0x7f631400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.983+0000 7f6320ff9640 1 -- 172.21.15.26:0/169919493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314017070 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.983+0000 7f6320ff9640 1 -- 172.21.15.26:0/169919493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631400c9d0 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.983+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63241e9d20 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.983+0000 7f6320ff9640 1 -- 172.21.15.26:0/169919493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631400cdc0 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.983+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632414ff70 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.984+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63240a7eb0 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.984+0000 7f6320ff9640 1 -- 172.21.15.26:0/169919493 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f631401a070 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.985+0000 7f6320ff9640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f630406b4d0 0x7f630406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.985+0000 7f6320ff9640 1 -- 172.21.15.26:0/169919493 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f631408a740 con 0x7f63240726b0 2024-03-06T07:24:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.985+0000 7f6323fff640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f630406b4d0 0x7f630406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.985+0000 7f6323fff640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f630406b4d0 0x7f630406d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f630c007fd0 tx=0x7f630c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:30.989+0000 7f6320ff9640 1 -- 172.21.15.26:0/169919493 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f63240a7eb0 con 0x7f63240726b0 2024-03-06T07:24:31.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.207+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6324071f80 con 0x7f63240726b0 2024-03-06T07:24:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.865+0000 7f6320ff9640 1 -- 172.21.15.26:0/169919493 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6314058dc0 con 0x7f63240726b0 2024-03-06T07:24:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f630406b4d0 msgr2=0x7f630406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f630406b4d0 0x7f630406d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f630c007fd0 tx=0x7f630c007de0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63240726b0 msgr2=0x7f6324140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63240726b0 0x7f6324140960 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f631400e9e0 tx=0x7f631400eea0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 shutdown_connections 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f630406b4d0 0x7f630406d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6324153fe0 0x7f6324140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63240726b0 0x7f6324140960 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 --2- 172.21.15.26:0/169919493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f632406b6a0 0x7f632414fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 >> 172.21.15.26:0/169919493 conn(0x7f63240a94d0 msgr2=0x7f63240a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 shutdown_connections 2024-03-06T07:24:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.868+0000 7f632a4a4640 1 -- 172.21.15.26:0/169919493 wait complete. 2024-03-06T07:24:31.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:24:31.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.993+0000 7fd5b3f79640 1 Processor -- start 2024-03-06T07:24:31.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.994+0000 7fd5b3f79640 1 -- start start 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.994+0000 7fd5b3f79640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5ac0c88e0 0x7fd5ac0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.994+0000 7fd5b3f79640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac0c5400 con 0x7fd5ac0c7930 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.994+0000 7fd5b3f79640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac0c56e0 con 0x7fd5ac152870 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.994+0000 7fd5b3f79640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ac06d320 con 0x7fd5ac0c88e0 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.994+0000 7fd5b2516640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5ac0c7930 0x7fd5ac06bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35764/0 (socket says 172.21.15.26:35764) 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.994+0000 7fd5b2516640 1 -- 172.21.15.26:0/3560368936 learned_addr learned my addr 172.21.15.26:0/3560368936 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b1514640 1 --2- 172.21.15.26:0/3560368936 >> v2:172.21.15.117:3300/0 conn(0x7fd5ac0c88e0 0x7fd5ac0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118684041 0 0) 0x7fd5ac0c5400 con 0x7fd5ac0c7930 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd594003520 con 0x7fd5ac0c7930 2024-03-06T07:24:31.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530643652 0 0) 0x7fd594003520 con 0x7fd5ac0c7930 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 >> v2:172.21.15.117:3300/0 conn(0x7fd5ac0c88e0 msgr2=0x7fd5ac0c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 --2- 172.21.15.26:0/3560368936 >> v2:172.21.15.117:3300/0 conn(0x7fd5ac0c88e0 0x7fd5ac0c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 >> v1:172.21.15.117:6789/0 conn(0x7fd5ac152870 legacy=0x7fd5ac155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac0c59c0 con 0x7fd5ac0c7930 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd59c0019f0 con 0x7fd5ac0c7930 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 >> v1:172.21.15.26:6789/0 conn(0x7fd5ac0c7930 legacy=0x7fd5ac06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd594003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd594004800 0x7fd594004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5940051c0 0x7fd59400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac06d320 con 0x7fd5940039b0 2024-03-06T07:24:32.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59400c160 con 0x7fd5940051c0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59400c360 con 0x7fd594004800 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b1d15640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd594003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b2516640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5940051c0 0x7fd59400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.995+0000 7fd5b1514640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd594004800 0x7fd594004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b1d15640 1 -- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd594004800 msgr2=0x7fd594004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b1d15640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd594004800 0x7fd594004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b1d15640 1 -- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5940051c0 msgr2=0x7fd59400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b1d15640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5940051c0 0x7fd59400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b2516640 1 -- 172.21.15.26:0/3560368936 reap_dead start 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b1d15640 1 -- 172.21.15.26:0/3560368936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd594003520 con 0x7fd5940039b0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b1d15640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd594003dd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4001750 tx=0x7fd594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4927bde89e61 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4001370 con 0x7fd5940039b0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a40052d0 con 0x7fd5940039b0 2024-03-06T07:24:32.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.996+0000 7fd5b0d13640 1 -- 172.21.15.26:0/3560368936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4003830 con 0x7fd5940039b0 2024-03-06T07:24:32.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.997+0000 7fd5b3f79640 1 -- 172.21.15.26:0/3560368936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 msgr2=0x7fd594003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.997+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd594003dd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4001750 tx=0x7fd594005830 comp rx=0 tx=0).stop 2024-03-06T07:24:32.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.997+0000 7fd5b3f79640 1 -- 172.21.15.26:0/3560368936 shutdown_connections 2024-03-06T07:24:32.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.997+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/3560368936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd594003dd0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.997+0000 7fd5b3f79640 1 -- 172.21.15.26:0/3560368936 >> 172.21.15.26:0/3560368936 conn(0x7fd5ac0ac5f0 msgr2=0x7fd5ac0aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.997+0000 7fd5b3f79640 1 -- 172.21.15.26:0/3560368936 shutdown_connections 2024-03-06T07:24:32.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.997+0000 7fd5b3f79640 1 -- 172.21.15.26:0/3560368936 wait complete. 2024-03-06T07:24:32.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.998+0000 7fd5b3f79640 1 Processor -- start 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.998+0000 7fd5b3f79640 1 -- start start 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b3f79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd5ac1ea650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b3f79640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac141da0 0x7fd5ac0c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b3f79640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac0c1360 0x7fd5ac0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b3f79640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0c9470 con 0x7fd5940039b0 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b3f79640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac06b8d0 con 0x7fd5ac141da0 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b3f79640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0c95f0 con 0x7fd5ac0c1360 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd5ac1ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd5ac1ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48614/0 (socket says 172.21.15.26:48614) 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 -- 172.21.15.26:0/1568157814 learned_addr learned my addr 172.21.15.26:0/1568157814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1514640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac141da0 0x7fd5ac0c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b2516640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac0c1360 0x7fd5ac0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 -- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac0c1360 msgr2=0x7fd5ac0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac0c1360 0x7fd5ac0aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 -- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac141da0 msgr2=0x7fd5ac0c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac141da0 0x7fd5ac0c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1d15640 1 -- 172.21.15.26:0/1568157814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac1edf30 con 0x7fd5940039b0 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b2516640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac0c1360 0x7fd5ac0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:31.999+0000 7fd5b1514640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac141da0 0x7fd5ac0c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.000+0000 7fd5b1d15640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd5ac1ea650 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4000c00 tx=0x7fd5a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.000+0000 7fd5aa7fc640 1 -- 172.21.15.26:0/1568157814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4041020 con 0x7fd5940039b0 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.000+0000 7fd5aa7fc640 1 -- 172.21.15.26:0/1568157814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a4005480 con 0x7fd5940039b0 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.000+0000 7fd5aa7fc640 1 -- 172.21.15.26:0/1568157814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4007a30 con 0x7fd5940039b0 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.000+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac1ee210 con 0x7fd5940039b0 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.000+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac1ee7d0 con 0x7fd5940039b0 2024-03-06T07:24:32.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.001+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac04f330 con 0x7fd5940039b0 2024-03-06T07:24:32.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.001+0000 7fd5aa7fc640 1 -- 172.21.15.26:0/1568157814 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5a4044020 con 0x7fd5940039b0 2024-03-06T07:24:32.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.001+0000 7fd5aa7fc640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59806b500 0x7fd59806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.002+0000 7fd5aa7fc640 1 -- 172.21.15.26:0/1568157814 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd5a40abd90 con 0x7fd5940039b0 2024-03-06T07:24:32.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.002+0000 7fd5b1514640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59806b500 0x7fd59806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.002+0000 7fd5b1514640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59806b500 0x7fd59806d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac14ba60 tx=0x7fd5a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.006+0000 7fd5aa7fc640 1 -- 172.21.15.26:0/1568157814 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5a404b030 con 0x7fd5940039b0 2024-03-06T07:24:32.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.186+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd5ac06cf70 con 0x7fd5940039b0 2024-03-06T07:24:32.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.867+0000 7fd5aa7fc640 1 -- 172.21.15.26:0/1568157814 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd5a40303c0 con 0x7fd5940039b0 2024-03-06T07:24:32.869 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59806b500 msgr2=0x7fd59806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59806b500 0x7fd59806d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac14ba60 tx=0x7fd5a0008040 comp rx=0 tx=0).stop 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 msgr2=0x7fd5ac1ea650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd5ac1ea650 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4000c00 tx=0x7fd5a4001120 comp rx=0 tx=0).stop 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 shutdown_connections 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59806b500 0x7fd59806d9c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac0c1360 0x7fd5ac0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac141da0 0x7fd5ac0c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 --2- 172.21.15.26:0/1568157814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5940039b0 0x7fd5ac1ea650 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.871+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 >> 172.21.15.26:0/1568157814 conn(0x7fd5ac0ac5f0 msgr2=0x7fd5ac0aadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.872+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 shutdown_connections 2024-03-06T07:24:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.872+0000 7fd5b3f79640 1 -- 172.21.15.26:0/1568157814 wait complete. 2024-03-06T07:24:32.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:24:33.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.999+0000 7fb3b3ec2640 1 Processor -- start 2024-03-06T07:24:33.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:32.999+0000 7fb3b3ec2640 1 -- start start 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b3ec2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 0x7fb3ac14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b3ec2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac06e280 con 0x7fb3ac06c240 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b3ec2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac0ac170 con 0x7fb3ac0a9450 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b3ec2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0ac450 con 0x7fb3ac0ab700 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b1c5e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb3ac06c240 0x7fb3ac06db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35772/0 (socket says 172.21.15.26:35772) 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b1c5e640 1 -- 172.21.15.26:0/1175179810 learned_addr learned my addr 172.21.15.26:0/1175179810 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b145d640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 0x7fb3ac14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357213548 0 0) 0x7fb3ac06e280 con 0x7fb3ac06c240 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb398003520 con 0x7fb3ac06c240 2024-03-06T07:24:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240068311 0 0) 0x7fb3ac0ac170 con 0x7fb3ac0a9450 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3ac06e280 con 0x7fb3ac0a9450 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885729306 0 0) 0x7fb398003520 con 0x7fb3ac06c240 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 msgr2=0x7fb3ac14ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 0x7fb3ac14ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 >> v1:172.21.15.117:6789/0 conn(0x7fb3ac0a9450 legacy=0x7fb3ac15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac152f90 con 0x7fb3ac06c240 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fb3a8001670 con 0x7fb3ac06c240 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 >> v1:172.21.15.26:6789/0 conn(0x7fb3ac06c240 legacy=0x7fb3ac06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb398003b70 0x7fb398003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb398004800 0x7fb398004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 0x7fb398005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0ac450 con 0x7fb398004800 2024-03-06T07:24:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39800bb50 con 0x7fb398003b70 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39800bd20 con 0x7fb3ac0ab700 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.000+0000 7fb3b145d640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb398004800 0x7fb398004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.001+0000 7fb3b1c5e640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb398003b70 0x7fb398003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.001+0000 7fb3b245f640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 0x7fb398005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.001+0000 7fb3b145d640 1 -- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 msgr2=0x7fb398005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.001+0000 7fb3b145d640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ac0ab700 0x7fb398005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.001+0000 7fb3b145d640 1 -- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb398003b70 msgr2=0x7fb398003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.001+0000 7fb3b145d640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb398003b70 0x7fb398003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.001+0000 7fb3b145d640 1 -- 172.21.15.26:0/1175179810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb398003520 con 0x7fb398004800 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b145d640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb398004800 0x7fb398004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009ac0 tx=0x7fb398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e312fc0e5bc33b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c039070 con 0x7fb398004800 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c02c680 con 0x7fb398004800 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b0c5c640 1 -- 172.21.15.26:0/1175179810 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c009690 con 0x7fb398004800 2024-03-06T07:24:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/1175179810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb398004800 msgr2=0x7fb398004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b3ec2640 1 --2- 172.21.15.26:0/1175179810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb398004800 0x7fb398004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009ac0 tx=0x7fb398005830 comp rx=0 tx=0).stop 2024-03-06T07:24:33.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b245f640 1 -- 172.21.15.26:0/1175179810 reap_dead start 2024-03-06T07:24:33.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/1175179810 shutdown_connections 2024-03-06T07:24:33.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/1175179810 >> 172.21.15.26:0/1175179810 conn(0x7fb3ac0c3960 msgr2=0x7fb3ac0c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:33.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/1175179810 shutdown_connections 2024-03-06T07:24:33.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.002+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/1175179810 wait complete. 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.003+0000 7fb3b3ec2640 1 Processor -- start 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.003+0000 7fb3b3ec2640 1 -- start start 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b3ec2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb398004800 0x7fb3ac1428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b3ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac071120 0x7fb3ac0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b3ec2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ac0c64f0 0x7fb3ac1eab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b3ec2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0abdb0 con 0x7fb3ac071120 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b3ec2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac153550 con 0x7fb3ac0c64f0 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b3ec2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0abf30 con 0x7fb398004800 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac071120 0x7fb3ac0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac071120 0x7fb3ac0c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48630/0 (socket says 172.21.15.26:48630) 2024-03-06T07:24:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 -- 172.21.15.26:0/2658490710 learned_addr learned my addr 172.21.15.26:0/2658490710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b1c5e640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb398004800 0x7fb3ac1428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b245f640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ac0c64f0 0x7fb3ac1eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 -- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb398004800 msgr2=0x7fb3ac1428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb398004800 0x7fb3ac1428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 -- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ac0c64f0 msgr2=0x7fb3ac1eab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ac0c64f0 0x7fb3ac1eab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.004+0000 7fb3b145d640 1 -- 172.21.15.26:0/2658490710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac156ce0 con 0x7fb3ac071120 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.005+0000 7fb3b145d640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac071120 0x7fb3ac0c5fb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb39c000c00 tx=0x7fb39c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.005+0000 7fb3a27fc640 1 -- 172.21.15.26:0/2658490710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c042020 con 0x7fb3ac071120 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.005+0000 7fb3a27fc640 1 -- 172.21.15.26:0/2658490710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c03a040 con 0x7fb3ac071120 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.005+0000 7fb3a27fc640 1 -- 172.21.15.26:0/2658490710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c039070 con 0x7fb3ac071120 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.005+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac1eec70 con 0x7fb3ac071120 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.005+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac142e00 con 0x7fb3ac071120 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.006+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac075300 con 0x7fb3ac071120 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.006+0000 7fb3a27fc640 1 -- 172.21.15.26:0/2658490710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb39c045020 con 0x7fb3ac071120 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.007+0000 7fb3a27fc640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b500 0x7fb39006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.007+0000 7fb3a27fc640 1 -- 172.21.15.26:0/2658490710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fb39c0ab7e0 con 0x7fb3ac071120 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.007+0000 7fb3b1c5e640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b500 0x7fb39006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.007+0000 7fb3b1c5e640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b500 0x7fb39006d9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8001200 tx=0x7fb3a8001d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.011+0000 7fb3a27fc640 1 -- 172.21.15.26:0/2658490710 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb3ac075300 con 0x7fb3ac071120 2024-03-06T07:24:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.192+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb3ac04f330 con 0x7fb3ac071120 2024-03-06T07:24:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.871+0000 7fb3a27fc640 1 -- 172.21.15.26:0/2658490710 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb39c04c030 con 0x7fb3ac071120 2024-03-06T07:24:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:24:33.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b500 msgr2=0x7fb39006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b500 0x7fb39006d9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8001200 tx=0x7fb3a8001d90 comp rx=0 tx=0).stop 2024-03-06T07:24:33.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac071120 msgr2=0x7fb3ac0c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac071120 0x7fb3ac0c5fb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb39c000c00 tx=0x7fb39c004730 comp rx=0 tx=0).stop 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 shutdown_connections 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb39006b500 0x7fb39006d9c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ac0c64f0 0x7fb3ac1eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac071120 0x7fb3ac0c5fb0 secure :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb39c000c00 tx=0x7fb39c004730 comp rx=0 tx=0).stop 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.875+0000 7fb3b3ec2640 1 --2- 172.21.15.26:0/2658490710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb398004800 0x7fb3ac1428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.876+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 >> 172.21.15.26:0/2658490710 conn(0x7fb3ac0c3960 msgr2=0x7fb3ac14b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.876+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 shutdown_connections 2024-03-06T07:24:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.876+0000 7fb3b3ec2640 1 -- 172.21.15.26:0/2658490710 wait complete. 2024-03-06T07:24:33.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:24:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.999+0000 7f814c919640 1 Processor -- start 2024-03-06T07:24:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:33.999+0000 7f814c919640 1 -- start start 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f814c919640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8148155c30 0x7f814806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f814c919640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f81480c5380 0x7f81480c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f814c919640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81480b0450 con 0x7f81480b0070 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f814c919640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814806cfb0 con 0x7f8148155c30 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f814c919640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814806c020 con 0x7f81480c5380 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f8145d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8148155c30 0x7f814806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f814659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f81480b0070 0x7f81480c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35784/0 (socket says 172.21.15.26:35784) 2024-03-06T07:24:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.000+0000 7f814659c640 1 -- 172.21.15.26:0/3905919721 learned_addr learned my addr 172.21.15.26:0/3905919721 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f8146d9d640 1 --2- 172.21.15.26:0/3905919721 >> v2:172.21.15.117:3300/0 conn(0x7f81480c5380 0x7f81480c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f8145d9b640 1 -- 172.21.15.26:0/3905919721 >> v2:172.21.15.117:3300/0 conn(0x7f81480c5380 msgr2=0x7f81480c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f8145d9b640 1 --2- 172.21.15.26:0/3905919721 >> v2:172.21.15.117:3300/0 conn(0x7f81480c5380 0x7f81480c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f8145d9b640 1 -- 172.21.15.26:0/3905919721 >> v1:172.21.15.26:6789/0 conn(0x7f81480b0070 legacy=0x7f81480c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f814559a640 1 -- 172.21.15.26:0/3905919721 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418940149 0 0) 0x7f8140001490 con 0x7f81480b0070 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f8145d9b640 1 -- 172.21.15.26:0/3905919721 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81481560e0 con 0x7f8148155c30 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f8146d9d640 1 --2- 172.21.15.26:0/3905919721 >> v2:172.21.15.117:3300/0 conn(0x7f81480c5380 0x7f81480c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.001+0000 7f8145d9b640 1 --2- 172.21.15.26:0/3905919721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148155c30 0x7f814806ca70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f81480be030 tx=0x7f813c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ba0f8471f7a25 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814559a640 1 -- 172.21.15.26:0/3905919721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c0315a0 con 0x7f8148155c30 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814559a640 1 -- 172.21.15.26:0/3905919721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c037070 con 0x7f8148155c30 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814559a640 1 -- 172.21.15.26:0/3905919721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c031960 con 0x7f8148155c30 2024-03-06T07:24:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 -- 172.21.15.26:0/3905919721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148155c30 msgr2=0x7f814806ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 --2- 172.21.15.26:0/3905919721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148155c30 0x7f814806ca70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f81480be030 tx=0x7f813c02b840 comp rx=0 tx=0).stop 2024-03-06T07:24:34.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 -- 172.21.15.26:0/3905919721 shutdown_connections 2024-03-06T07:24:34.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 --2- 172.21.15.26:0/3905919721 >> v2:172.21.15.117:3300/0 conn(0x7f81480c5380 0x7f81480c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 --2- 172.21.15.26:0/3905919721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148155c30 0x7f814806ca70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 -- 172.21.15.26:0/3905919721 >> 172.21.15.26:0/3905919721 conn(0x7f81480ac590 msgr2=0x7f81480ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 -- 172.21.15.26:0/3905919721 shutdown_connections 2024-03-06T07:24:34.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.002+0000 7f814c919640 1 -- 172.21.15.26:0/3905919721 wait complete. 2024-03-06T07:24:34.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 Processor -- start 2024-03-06T07:24:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 -- start start 2024-03-06T07:24:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81480b0070 0x7f814814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81480c5380 0x7f8148154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8148155c30 0x7f81481ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814806c270 con 0x7f81480b0070 2024-03-06T07:24:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81480c7ad0 con 0x7f8148155c30 2024-03-06T07:24:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814c919640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814806c550 con 0x7f81480c5380 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81480b0070 0x7f814814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81480b0070 0x7f814814ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48658/0 (socket says 172.21.15.26:48658) 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.004+0000 7f814659c640 1 -- 172.21.15.26:0/4123296290 learned_addr learned my addr 172.21.15.26:0/4123296290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814659c640 1 -- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81480c5380 msgr2=0x7f8148154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814659c640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81480c5380 0x7f8148154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814659c640 1 -- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8148155c30 msgr2=0x7f81481ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f8146d9d640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8148155c30 0x7f81481ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814659c640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8148155c30 0x7f81481ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814659c640 1 -- 172.21.15.26:0/4123296290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81481591a0 con 0x7f81480b0070 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814659c640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81480b0070 0x7f814814ede0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f814000c7f0 tx=0x7f814000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f8136ffd640 1 -- 172.21.15.26:0/4123296290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140018070 con 0x7f81480b0070 2024-03-06T07:24:34.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f8136ffd640 1 -- 172.21.15.26:0/4123296290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814000a640 con 0x7f81480b0070 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814814f320 con 0x7f81480b0070 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f8136ffd640 1 -- 172.21.15.26:0/4123296290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814000aa30 con 0x7f81480b0070 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.005+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814814eb00 con 0x7f81480b0070 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.006+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814804f330 con 0x7f81480b0070 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.007+0000 7f8136ffd640 1 -- 172.21.15.26:0/4123296290 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f814001b070 con 0x7f81480b0070 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.007+0000 7f8136ffd640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f812806b500 0x7f812806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.007+0000 7f8136ffd640 1 -- 172.21.15.26:0/4123296290 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8140089730 con 0x7f81480b0070 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.007+0000 7f8145d9b640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f812806b500 0x7f812806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.008+0000 7f8145d9b640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f812806b500 0x7f812806d9c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f813c036000 tx=0x7f813c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.011+0000 7f8136ffd640 1 -- 172.21.15.26:0/4123296290 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f81480b0450 con 0x7f81480b0070 2024-03-06T07:24:34.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.194+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8148067b10 con 0x7f81480b0070 2024-03-06T07:24:34.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.197+0000 7f8136ffd640 1 -- 172.21.15.26:0/4123296290 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81400586d0 con 0x7f81480b0070 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f812806b500 msgr2=0x7f812806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f812806b500 0x7f812806d9c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f813c036000 tx=0x7f813c030000 comp rx=0 tx=0).stop 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81480b0070 msgr2=0x7f814814ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81480b0070 0x7f814814ede0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f814000c7f0 tx=0x7f814000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 shutdown_connections 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f812806b500 0x7f812806d9c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8148155c30 0x7f81481ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81480c5380 0x7f8148154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 --2- 172.21.15.26:0/4123296290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81480b0070 0x7f814814ede0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 >> 172.21.15.26:0/4123296290 conn(0x7f81480ac590 msgr2=0x7f81480af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 shutdown_connections 2024-03-06T07:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.200+0000 7f814c919640 1 -- 172.21.15.26:0/4123296290 wait complete. 2024-03-06T07:24:34.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:24:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.313+0000 7fae696ed640 1 Processor -- start 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.313+0000 7fae696ed640 1 -- start start 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae696ed640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae640b0070 0x7fae64155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae696ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae640b0450 con 0x7fae640c5380 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae696ed640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6406cf20 con 0x7fae6406ca70 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae696ed640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6406c020 con 0x7fae640b0070 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae637fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae640c5380 0x7fae6414e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35786/0 (socket says 172.21.15.26:35786) 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae637fe640 1 -- 172.21.15.26:0/769610059 learned_addr learned my addr 172.21.15.26:0/769610059 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae62ffd640 1 --2- 172.21.15.26:0/769610059 >> v2:172.21.15.117:3300/0 conn(0x7fae640b0070 0x7fae64155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657948482 0 0) 0x7fae54001490 con 0x7fae640c5380 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.314+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae4c003520 con 0x7fae640c5380 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3237582709 0 0) 0x7fae4c003520 con 0x7fae640c5380 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 >> v2:172.21.15.117:3300/0 conn(0x7fae640b0070 msgr2=0x7fae64155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 --2- 172.21.15.26:0/769610059 >> v2:172.21.15.117:3300/0 conn(0x7fae640b0070 0x7fae64155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 >> v1:172.21.15.117:6789/0 conn(0x7fae6406ca70 legacy=0x7fae640c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae640c77f0 con 0x7fae640c5380 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fae54001bc0 con 0x7fae640c5380 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 >> v1:172.21.15.26:6789/0 conn(0x7fae640c5380 legacy=0x7fae6414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae4c0039b0 0x7fae4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c004800 0x7fae4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c0051c0 0x7fae4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406c020 con 0x7fae4c0051c0 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c00c160 con 0x7fae4c004800 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c00c360 con 0x7fae4c0039b0 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae62ffd640 1 --2- 172.21.15.26:0/769610059 >> v2:172.21.15.117:3300/0 conn(0x7fae640b0070 0x7fae64155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae637fe640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c0051c0 0x7fae4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae627fc640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c004800 0x7fae4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae62ffd640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae4c0039b0 0x7fae4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae637fe640 1 -- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae4c0039b0 msgr2=0x7fae4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae637fe640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae4c0039b0 0x7fae4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae637fe640 1 -- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c004800 msgr2=0x7fae4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae637fe640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c004800 0x7fae4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae637fe640 1 -- 172.21.15.26:0/769610059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c003520 con 0x7fae4c0051c0 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.315+0000 7fae627fc640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c004800 0x7fae4c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:34.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae637fe640 1 -- 172.21.15.26:0/769610059 reap_dead start 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae637fe640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c0051c0 0x7fae4c00b9c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fae5400aa30 tx=0x7fae4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1464d6f4ff25d864 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54038070 con 0x7fae4c0051c0 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae54033620 con 0x7fae4c0051c0 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae61ffb640 1 -- 172.21.15.26:0/769610059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400a650 con 0x7fae4c0051c0 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae696ed640 1 -- 172.21.15.26:0/769610059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c0051c0 msgr2=0x7fae4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae696ed640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c0051c0 0x7fae4c00b9c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fae5400aa30 tx=0x7fae4c005830 comp rx=0 tx=0).stop 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae696ed640 1 -- 172.21.15.26:0/769610059 shutdown_connections 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae696ed640 1 --2- 172.21.15.26:0/769610059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c0051c0 0x7fae4c00b9c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae696ed640 1 -- 172.21.15.26:0/769610059 >> 172.21.15.26:0/769610059 conn(0x7fae640ac590 msgr2=0x7fae640ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.316+0000 7fae696ed640 1 -- 172.21.15.26:0/769610059 shutdown_connections 2024-03-06T07:24:34.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.317+0000 7fae696ed640 1 -- 172.21.15.26:0/769610059 wait complete. 2024-03-06T07:24:34.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 Processor -- start 2024-03-06T07:24:34.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 -- start start 2024-03-06T07:24:34.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0051c0 0x7fae641ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640c4a90 0x7fae641ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae64157ac0 0x7fae64157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406c550 con 0x7fae64157ac0 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406c270 con 0x7fae4c0051c0 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.318+0000 7fae696ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406da80 con 0x7fae640c4a90 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae64157ac0 0x7fae64157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae64157ac0 0x7fae64157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48680/0 (socket says 172.21.15.26:48680) 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 -- 172.21.15.26:0/35989733 learned_addr learned my addr 172.21.15.26:0/35989733 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae62ffd640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0051c0 0x7fae641ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae627fc640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640c4a90 0x7fae641ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 -- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640c4a90 msgr2=0x7fae641ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640c4a90 0x7fae641ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 -- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0051c0 msgr2=0x7fae641ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0051c0 0x7fae641ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 -- 172.21.15.26:0/35989733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae641eedb0 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae637fe640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae64157ac0 0x7fae64157f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fae5400a0b0 tx=0x7fae641551b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae4b7fe640 1 -- 172.21.15.26:0/35989733 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54038040 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae4b7fe640 1 -- 172.21.15.26:0/35989733 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae54044070 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae641575c0 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae4b7fe640 1 -- 172.21.15.26:0/35989733 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400a650 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.319+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6414f890 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.320+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae64157790 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.321+0000 7fae4b7fe640 1 -- 172.21.15.26:0/35989733 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae54033620 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.321+0000 7fae4b7fe640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.321+0000 7fae4b7fe640 1 -- 172.21.15.26:0/35989733 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fae540ab9f0 con 0x7fae64157ac0 2024-03-06T07:24:34.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.321+0000 7fae62ffd640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.322+0000 7fae62ffd640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0093d0 tx=0x7fae5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.325+0000 7fae4b7fe640 1 -- 172.21.15.26:0/35989733 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae640b0450 con 0x7fae64157ac0 2024-03-06T07:24:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.508+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7fae640c5bb0 con 0x7fae64157ac0 2024-03-06T07:24:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.511+0000 7fae4b7fe640 1 -- 172.21.15.26:0/35989733 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v39) v1 ==== 96+0+0 (secure 0 0 0) 0x7fae5407a340 con 0x7fae64157ac0 2024-03-06T07:24:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.512+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 msgr2=0x7fae5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.512+0000 7fae696ed640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0093d0 tx=0x7fae5c008040 comp rx=0 tx=0).stop 2024-03-06T07:24:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae64157ac0 msgr2=0x7fae64157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae64157ac0 0x7fae64157f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fae5400a0b0 tx=0x7fae641551b0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 shutdown_connections 2024-03-06T07:24:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae64157ac0 0x7fae64157f40 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640c4a90 0x7fae641ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 --2- 172.21.15.26:0/35989733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0051c0 0x7fae641ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 >> 172.21.15.26:0/35989733 conn(0x7fae640ac590 msgr2=0x7fae64154010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 shutdown_connections 2024-03-06T07:24:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.513+0000 7fae696ed640 1 -- 172.21.15.26:0/35989733 wait complete. 2024-03-06T07:24:34.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:24:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.995+0000 7f465081c640 1 Processor -- start 2024-03-06T07:24:34.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.995+0000 7f465081c640 1 -- start start 2024-03-06T07:24:34.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f465081c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f464c0b0070 0x7f464c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f465081c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c0c5380 0x7f464c0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f465081c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c0b0450 con 0x7f464c06ca70 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f465081c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c06c020 con 0x7f464c0b0070 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f465081c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c06cf20 con 0x7f464c0c5380 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f464a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f464c0b0070 0x7f464c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f464a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f464c0b0070 0x7f464c155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48682/0 (socket says 172.21.15.26:48682) 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f464a59c640 1 -- 172.21.15.26:0/1696337853 learned_addr learned my addr 172.21.15.26:0/1696337853 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f464ad9d640 1 --2- 172.21.15.26:0/1696337853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c0c5380 0x7f464c0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f464a59c640 1 -- 172.21.15.26:0/1696337853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c0c5380 msgr2=0x7f464c0c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f464a59c640 1 --2- 172.21.15.26:0/1696337853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c0c5380 0x7f464c0c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.996+0000 7f464a59c640 1 -- 172.21.15.26:0/1696337853 >> v1:172.21.15.117:6789/0 conn(0x7f464c06ca70 legacy=0x7f464c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.997+0000 7f464a59c640 1 -- 172.21.15.26:0/1696337853 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c0c77f0 con 0x7f464c0b0070 2024-03-06T07:24:34.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.997+0000 7f464a59c640 1 --2- 172.21.15.26:0/1696337853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0b0070 0x7f464c155c30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4644009870 tx=0x7f464402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a96bcb55245e53 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.997+0000 7f464959a640 1 -- 172.21.15.26:0/1696337853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46440317c0 con 0x7f464c0b0070 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.997+0000 7f464959a640 1 -- 172.21.15.26:0/1696337853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644037070 con 0x7f464c0b0070 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f464959a640 1 -- 172.21.15.26:0/1696337853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644031b80 con 0x7f464c0b0070 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 -- 172.21.15.26:0/1696337853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0b0070 msgr2=0x7f464c155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 --2- 172.21.15.26:0/1696337853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0b0070 0x7f464c155c30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4644009870 tx=0x7f464402b860 comp rx=0 tx=0).stop 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 -- 172.21.15.26:0/1696337853 shutdown_connections 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 --2- 172.21.15.26:0/1696337853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c0c5380 0x7f464c0c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 --2- 172.21.15.26:0/1696337853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0b0070 0x7f464c155c30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 -- 172.21.15.26:0/1696337853 >> 172.21.15.26:0/1696337853 conn(0x7f464c0ac590 msgr2=0x7f464c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 -- 172.21.15.26:0/1696337853 shutdown_connections 2024-03-06T07:24:35.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.998+0000 7f465081c640 1 -- 172.21.15.26:0/1696337853 wait complete. 2024-03-06T07:24:35.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.999+0000 7f465081c640 1 Processor -- start 2024-03-06T07:24:35.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:34.999+0000 7f465081c640 1 -- start start 2024-03-06T07:24:35.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f465081c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c06ca70 0x7f464c0c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f465081c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c0b0070 0x7f464c0c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f465081c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0c5380 0x7f464c0c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f465081c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c06c550 con 0x7f464c0c5380 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f465081c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c06c270 con 0x7f464c0b0070 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f465081c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c06da80 con 0x7f464c06ca70 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0c5380 0x7f464c0c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0c5380 0x7f464c0c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48694/0 (socket says 172.21.15.26:48694) 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 -- 172.21.15.26:0/980294660 learned_addr learned my addr 172.21.15.26:0/980294660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464a59c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c06ca70 0x7f464c0c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f4649d9b640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c0b0070 0x7f464c0c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 -- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c06ca70 msgr2=0x7f464c0c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c06ca70 0x7f464c0c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 -- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c0b0070 msgr2=0x7f464c0c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c0b0070 0x7f464c0c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.000+0000 7f464ad9d640 1 -- 172.21.15.26:0/980294660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c158060 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.001+0000 7f464ad9d640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0c5380 0x7f464c0c4780 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f463c00cda0 tx=0x7f463c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.001+0000 7f4642ffd640 1 -- 172.21.15.26:0/980294660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c003b90 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.001+0000 7f4642ffd640 1 -- 172.21.15.26:0/980294660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c003d30 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.001+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c158340 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.001+0000 7f4642ffd640 1 -- 172.21.15.26:0/980294660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c004240 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.001+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c0c1ad0 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.002+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c06bda0 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.002+0000 7f4642ffd640 1 -- 172.21.15.26:0/980294660 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f463c0043e0 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.003+0000 7f4642ffd640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462c06b500 0x7f462c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.003+0000 7f4642ffd640 1 -- 172.21.15.26:0/980294660 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f463c08ab50 con 0x7f464c0c5380 2024-03-06T07:24:35.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.003+0000 7f464a59c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462c06b500 0x7f462c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.003+0000 7f464a59c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462c06b500 0x7f462c06d9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4644036000 tx=0x7f464403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.007+0000 7f4642ffd640 1 -- 172.21.15.26:0/980294660 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f464c06bda0 con 0x7f464c0c5380 2024-03-06T07:24:35.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.191+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7f464c04f330 con 0x7f464c0c5380 2024-03-06T07:24:35.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.194+0000 7f4642ffd640 1 -- 172.21.15.26:0/980294660 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v40) v1 ==== 96+0+0 (secure 0 0 0) 0x7f463c059b80 con 0x7f464c0c5380 2024-03-06T07:24:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462c06b500 msgr2=0x7f462c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462c06b500 0x7f462c06d9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4644036000 tx=0x7f464403d040 comp rx=0 tx=0).stop 2024-03-06T07:24:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0c5380 msgr2=0x7f464c0c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0c5380 0x7f464c0c4780 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f463c00cda0 tx=0x7f463c00a660 comp rx=0 tx=0).stop 2024-03-06T07:24:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 shutdown_connections 2024-03-06T07:24:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462c06b500 0x7f462c06d9c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f464c0c5380 0x7f464c0c4780 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c0b0070 0x7f464c0c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 --2- 172.21.15.26:0/980294660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464c06ca70 0x7f464c0c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 >> 172.21.15.26:0/980294660 conn(0x7f464c0ac590 msgr2=0x7f464c1511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 shutdown_connections 2024-03-06T07:24:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.196+0000 7f465081c640 1 -- 172.21.15.26:0/980294660 wait complete. 2024-03-06T07:24:35.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay' 2024-03-06T07:24:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.309+0000 7fb613313640 1 Processor -- start 2024-03-06T07:24:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.309+0000 7fb613313640 1 -- start start 2024-03-06T07:24:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb613313640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb60c150580 0x7fb60c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb613313640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb60c0c6000 0x7fb60c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb613313640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c153a60 con 0x7fb60c0c1320 2024-03-06T07:24:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb613313640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c153d40 con 0x7fb60c150580 2024-03-06T07:24:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb613313640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c06c2c0 con 0x7fb60c0c6000 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6110af640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb60c150580 0x7fb60c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6110af640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb60c150580 0x7fb60c06dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48706/0 (socket says 172.21.15.26:48706) 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6110af640 1 -- 172.21.15.26:0/2834732387 learned_addr learned my addr 172.21.15.26:0/2834732387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6118b0640 1 --2- 172.21.15.26:0/2834732387 >> v2:172.21.15.117:3300/0 conn(0x7fb60c0c6000 0x7fb60c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6110af640 1 -- 172.21.15.26:0/2834732387 >> v2:172.21.15.117:3300/0 conn(0x7fb60c0c6000 msgr2=0x7fb60c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6110af640 1 --2- 172.21.15.26:0/2834732387 >> v2:172.21.15.117:3300/0 conn(0x7fb60c0c6000 0x7fb60c06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6110af640 1 -- 172.21.15.26:0/2834732387 >> v1:172.21.15.117:6789/0 conn(0x7fb60c0c1320 legacy=0x7fb60c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6110af640 1 -- 172.21.15.26:0/2834732387 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c06c5a0 con 0x7fb60c150580 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.310+0000 7fb6118b0640 1 --2- 172.21.15.26:0/2834732387 >> v2:172.21.15.117:3300/0 conn(0x7fb60c0c6000 0x7fb60c06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.311+0000 7fb6110af640 1 --2- 172.21.15.26:0/2834732387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c150580 0x7fb60c06dd10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0c5960 tx=0x7fb60402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c5250dc0f89cc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.311+0000 7fb60bfff640 1 -- 172.21.15.26:0/2834732387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60402c9f0 con 0x7fb60c150580 2024-03-06T07:24:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.311+0000 7fb60bfff640 1 -- 172.21.15.26:0/2834732387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60402cb90 con 0x7fb60c150580 2024-03-06T07:24:35.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.311+0000 7fb60bfff640 1 -- 172.21.15.26:0/2834732387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040343f0 con 0x7fb60c150580 2024-03-06T07:24:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 -- 172.21.15.26:0/2834732387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c150580 msgr2=0x7fb60c06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 --2- 172.21.15.26:0/2834732387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c150580 0x7fb60c06dd10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0c5960 tx=0x7fb60402be50 comp rx=0 tx=0).stop 2024-03-06T07:24:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 -- 172.21.15.26:0/2834732387 shutdown_connections 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 --2- 172.21.15.26:0/2834732387 >> v2:172.21.15.117:3300/0 conn(0x7fb60c0c6000 0x7fb60c06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 --2- 172.21.15.26:0/2834732387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c150580 0x7fb60c06dd10 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 -- 172.21.15.26:0/2834732387 >> 172.21.15.26:0/2834732387 conn(0x7fb60c0c6710 msgr2=0x7fb60c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 -- 172.21.15.26:0/2834732387 shutdown_connections 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.312+0000 7fb613313640 1 -- 172.21.15.26:0/2834732387 wait complete. 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.313+0000 7fb613313640 1 Processor -- start 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.313+0000 7fb613313640 1 -- start start 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb613313640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c0c1320 0x7fb60c1ebec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb613313640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c0c6000 0x7fb60c0a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb613313640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c150580 0x7fb60c154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb613313640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c0dc0 con 0x7fb60c0c6000 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb613313640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c0c40 con 0x7fb60c0c1320 2024-03-06T07:24:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb613313640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c156dc0 con 0x7fb60c150580 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c0c6000 0x7fb60c0a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c0c6000 0x7fb60c0a75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48712/0 (socket says 172.21.15.26:48712) 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 -- 172.21.15.26:0/2848837559 learned_addr learned my addr 172.21.15.26:0/2848837559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 -- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c150580 msgr2=0x7fb60c154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c150580 0x7fb60c154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 -- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c0c1320 msgr2=0x7fb60c1ebec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c0c1320 0x7fb60c1ebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 -- 172.21.15.26:0/2848837559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c0a80e0 con 0x7fb60c0c6000 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb6108ae640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c0c6000 0x7fb60c0a75f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c430 tx=0x7fb60000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb609ffb640 1 -- 172.21.15.26:0/2848837559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600011790 con 0x7fb60c0c6000 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb609ffb640 1 -- 172.21.15.26:0/2848837559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb600017070 con 0x7fb60c0c6000 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb609ffb640 1 -- 172.21.15.26:0/2848837559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600011b70 con 0x7fb60c0c6000 2024-03-06T07:24:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.314+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c1ecdf0 con 0x7fb60c0c6000 2024-03-06T07:24:35.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.315+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c1ec830 con 0x7fb60c0c6000 2024-03-06T07:24:35.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.316+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c04f330 con 0x7fb60c0c6000 2024-03-06T07:24:35.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.316+0000 7fb609ffb640 1 -- 172.21.15.26:0/2848837559 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb600011d10 con 0x7fb60c0c6000 2024-03-06T07:24:35.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.316+0000 7fb609ffb640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f806b4d0 0x7fb5f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.317+0000 7fb609ffb640 1 -- 172.21.15.26:0/2848837559 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb600089fa0 con 0x7fb60c0c6000 2024-03-06T07:24:35.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.317+0000 7fb6110af640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f806b4d0 0x7fb5f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.317+0000 7fb6110af640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f806b4d0 0x7fb5f806d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb604035000 tx=0x7fb60402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.321+0000 7fb609ffb640 1 -- 172.21.15.26:0/2848837559 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb600058f40 con 0x7fb60c0c6000 2024-03-06T07:24:35.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.571+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]} v 0) v1 -- 0x7fb60c0c3a40 con 0x7fb60c0c6000 2024-03-06T07:24:35.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.877+0000 7fb609ffb640 1 -- 172.21.15.26:0/2848837559 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]}]=0 v268) v1 ==== 153+0+0 (secure 0 0 0) 0x7fb600058840 con 0x7fb60c0c6000 2024-03-06T07:24:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f806b4d0 msgr2=0x7fb5f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f806b4d0 0x7fb5f806d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb604035000 tx=0x7fb60402e000 comp rx=0 tx=0).stop 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c0c6000 msgr2=0x7fb60c0a75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c0c6000 0x7fb60c0a75f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c430 tx=0x7fb60000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 shutdown_connections 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb5f806b4d0 0x7fb5f806d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c150580 0x7fb60c154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb60c0c6000 0x7fb60c0a75f0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 --2- 172.21.15.26:0/2848837559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c0c1320 0x7fb60c1ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 >> 172.21.15.26:0/2848837559 conn(0x7fb60c0c6710 msgr2=0x7fb60c0c4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 shutdown_connections 2024-03-06T07:24:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.879+0000 7fb613313640 1 -- 172.21.15.26:0/2848837559 wait complete. 2024-03-06T07:24:35.892 INFO:tasks.cephfs_test_runner:That a standby-replay daemon with weak affinity is replaced by a stronger one. ... ok 2024-03-06T07:24:35.892 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_steady (tasks.cephfs.test_failover.TestClusterAffinity) That a sole MDS with mds_join_fs set will come back as active eventually even after failover. 2024-03-06T07:24:35.892 INFO:tasks.cephfs_test_runner:test_join_fs_steady (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:24:35.893 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:35.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.993+0000 7f1919a02640 1 Processor -- start 2024-03-06T07:24:35.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.993+0000 7f1919a02640 1 -- start start 2024-03-06T07:24:35.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f1919a02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191414bad0 0x7f1914155990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f1919a02640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f19140c9240 0x7f19140cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f1919a02640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f191406cc00 0x7f19140c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f1919a02640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19140c52f0 con 0x7f191406cc00 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f1919a02640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19140c55d0 con 0x7f19140c9240 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f1919a02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19140c58b0 con 0x7f191414bad0 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f191406cc00 0x7f19140c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f191406cc00 0x7f19140c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48722/0 (socket says 172.21.15.26:48722) 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 -- 172.21.15.26:0/1059028118 learned_addr learned my addr 172.21.15.26:0/1059028118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 -- 172.21.15.26:0/1059028118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191414bad0 msgr2=0x7f1914155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 --2- 172.21.15.26:0/1059028118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191414bad0 0x7f1914155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 -- 172.21.15.26:0/1059028118 >> v2:172.21.15.117:3300/0 conn(0x7f19140c9240 msgr2=0x7f19140cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19127fc640 1 --2- 172.21.15.26:0/1059028118 >> v2:172.21.15.117:3300/0 conn(0x7f19140c9240 0x7f19140cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 --2- 172.21.15.26:0/1059028118 >> v2:172.21.15.117:3300/0 conn(0x7f19140c9240 0x7f19140cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.994+0000 7f19137fe640 1 -- 172.21.15.26:0/1059028118 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19140c7310 con 0x7f191406cc00 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.995+0000 7f19137fe640 1 --2- 172.21.15.26:0/1059028118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191406cc00 0x7f19140c7830 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f19140c8350 tx=0x7f190402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58152fc4cd92fee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.995+0000 7f1911ffb640 1 -- 172.21.15.26:0/1059028118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190402c9e0 con 0x7f191406cc00 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.995+0000 7f1911ffb640 1 -- 172.21.15.26:0/1059028118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190402cb80 con 0x7f191406cc00 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.995+0000 7f1911ffb640 1 -- 172.21.15.26:0/1059028118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040343f0 con 0x7f191406cc00 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 -- 172.21.15.26:0/1059028118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191406cc00 msgr2=0x7f19140c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 --2- 172.21.15.26:0/1059028118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191406cc00 0x7f19140c7830 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f19140c8350 tx=0x7f190402be40 comp rx=0 tx=0).stop 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 -- 172.21.15.26:0/1059028118 shutdown_connections 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 --2- 172.21.15.26:0/1059028118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191406cc00 0x7f19140c7830 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 --2- 172.21.15.26:0/1059028118 >> v2:172.21.15.117:3300/0 conn(0x7f19140c9240 0x7f19140cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 --2- 172.21.15.26:0/1059028118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191414bad0 0x7f1914155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 -- 172.21.15.26:0/1059028118 >> 172.21.15.26:0/1059028118 conn(0x7f1914072b10 msgr2=0x7f1914072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 -- 172.21.15.26:0/1059028118 shutdown_connections 2024-03-06T07:24:35.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.996+0000 7f1919a02640 1 -- 172.21.15.26:0/1059028118 wait complete. 2024-03-06T07:24:36.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.997+0000 7f1919a02640 1 Processor -- start 2024-03-06T07:24:36.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.997+0000 7f1919a02640 1 -- start start 2024-03-06T07:24:36.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f1919a02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191406cc00 0x7f19141e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f1919a02640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19140c9240 0x7f1914144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f1919a02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191414bad0 0x7f1914145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f1919a02640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19140cb430 con 0x7f191414bad0 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f1919a02640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19140cb2b0 con 0x7f19140c9240 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f1919a02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19140cb5b0 con 0x7f191406cc00 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191414bad0 0x7f1914145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191414bad0 0x7f1914145100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48730/0 (socket says 172.21.15.26:48730) 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 -- 172.21.15.26:0/2154637186 learned_addr learned my addr 172.21.15.26:0/2154637186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19127fc640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19140c9240 0x7f1914144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f1912ffd640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191406cc00 0x7f19141e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 -- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191406cc00 msgr2=0x7f19141e1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191406cc00 0x7f19141e1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 -- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19140c9240 msgr2=0x7f1914144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19140c9240 0x7f1914144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 -- 172.21.15.26:0/2154637186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19141ee8f0 con 0x7f191414bad0 2024-03-06T07:24:36.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f19137fe640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191414bad0 0x7f1914145100 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1904036040 tx=0x7f1904004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f190bfff640 1 -- 172.21.15.26:0/2154637186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904043020 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.998+0000 7f190bfff640 1 -- 172.21.15.26:0/2154637186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190403e450 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.999+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19140c1ce0 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.999+0000 7f190bfff640 1 -- 172.21.15.26:0/2154637186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904009510 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.999+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19140c22a0 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:35.999+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19140684b0 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.000+0000 7f190bfff640 1 -- 172.21.15.26:0/2154637186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1904046020 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.000+0000 7f190bfff640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f18f406b4d0 0x7f18f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.000+0000 7f190bfff640 1 -- 172.21.15.26:0/2154637186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f19040acd00 con 0x7f191414bad0 2024-03-06T07:24:36.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.001+0000 7f1912ffd640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f18f406b4d0 0x7f18f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.001+0000 7f1912ffd640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f18f406b4d0 0x7f18f406d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f190c009890 tx=0x7f190c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.005+0000 7f190bfff640 1 -- 172.21.15.26:0/2154637186 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f190404d020 con 0x7f191414bad0 2024-03-06T07:24:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.185+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19140693c0 con 0x7f191414bad0 2024-03-06T07:24:36.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.186+0000 7f190bfff640 1 -- 172.21.15.26:0/2154637186 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+13445 (secure 0 0 0) 0x7f190407bcb0 con 0x7f191414bad0 2024-03-06T07:24:36.188 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:36.188 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":112,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:33.867998+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6834/548706796":"2024-03-07T07:24:30.860590+0000","172.21.15.26:6835/1745266733":"2024-03-07T07:24:13.704095+0000","172.21.15.117:6835/548706796":"2024-03-07T07:24:30.860590+0000","172.21.15.26:6833/1745266733":"2024-03-07T07:24:13.704095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f18f406b4d0 msgr2=0x7f18f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f18f406b4d0 0x7f18f406d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f190c009890 tx=0x7f190c008040 comp rx=0 tx=0).stop 2024-03-06T07:24:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191414bad0 msgr2=0x7f1914145100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191414bad0 0x7f1914145100 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1904036040 tx=0x7f1904004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 shutdown_connections 2024-03-06T07:24:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f18f406b4d0 0x7f18f406d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191414bad0 0x7f1914145100 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19140c9240 0x7f1914144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 --2- 172.21.15.26:0/2154637186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191406cc00 0x7f19141e1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 >> 172.21.15.26:0/2154637186 conn(0x7f1914072b10 msgr2=0x7f19140c0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 shutdown_connections 2024-03-06T07:24:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.188+0000 7f1919a02640 1 -- 172.21.15.26:0/2154637186 wait complete. 2024-03-06T07:24:36.201 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:36.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.304+0000 7f2c3fc39640 1 Processor -- start 2024-03-06T07:24:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.305+0000 7f2c3fc39640 1 -- start start 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.305+0000 7f2c3fc39640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c3806ca70 0x7f2c3814e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.305+0000 7f2c3fc39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c380b0450 con 0x7f2c38155c30 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.305+0000 7f2c3fc39640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c381560e0 con 0x7f2c380b0070 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.305+0000 7f2c3fc39640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3806c020 con 0x7f2c3806ca70 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.305+0000 7f2c3d1d4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2c38155c30 0x7f2c380c5380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35788/0 (socket says 172.21.15.26:35788) 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.305+0000 7f2c3d1d4640 1 -- 172.21.15.26:0/710046261 learned_addr learned my addr 172.21.15.26:0/710046261 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3e1d6640 1 --2- 172.21.15.26:0/710046261 >> v2:172.21.15.117:3300/0 conn(0x7f2c3806ca70 0x7f2c3814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913755719 0 0) 0x7f2c28001110 con 0x7f2c38155c30 2024-03-06T07:24:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c24003520 con 0x7f2c38155c30 2024-03-06T07:24:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4207166612 0 0) 0x7f2c24003520 con 0x7f2c38155c30 2024-03-06T07:24:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 >> v2:172.21.15.117:3300/0 conn(0x7f2c3806ca70 msgr2=0x7f2c3814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 --2- 172.21.15.26:0/710046261 >> v2:172.21.15.117:3300/0 conn(0x7f2c3806ca70 0x7f2c3814e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 >> v1:172.21.15.117:6789/0 conn(0x7f2c380b0070 legacy=0x7f2c380c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380c77f0 con 0x7f2c38155c30 2024-03-06T07:24:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f2c28001840 con 0x7f2c38155c30 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 >> v1:172.21.15.26:6789/0 conn(0x7f2c38155c30 legacy=0x7f2c380c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c240039b0 0x7f2c24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c24004800 0x7f2c24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3806c020 con 0x7f2c240051c0 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2400c160 con 0x7f2c24004800 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2400c360 con 0x7f2c240039b0 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3e1d6640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3d1d4640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c24004800 0x7f2c24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3d9d5640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c240039b0 0x7f2c24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3e1d6640 1 -- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c240039b0 msgr2=0x7f2c24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3e1d6640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c240039b0 0x7f2c24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3e1d6640 1 -- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c24004800 msgr2=0x7f2c24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3e1d6640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c24004800 0x7f2c24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.306+0000 7f2c3e1d6640 1 -- 172.21.15.26:0/710046261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24003520 con 0x7f2c240051c0 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.307+0000 7f2c3e1d6640 1 -- 172.21.15.26:0/710046261 reap_dead start 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.307+0000 7f2c3e1d6640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c2400b9c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009fc0 tx=0x7f2c24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a1b8927a360b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.307+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30009c10 con 0x7f2c240051c0 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.307+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3002cca0 con 0x7f2c240051c0 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.307+0000 7f2c3c9d3640 1 -- 172.21.15.26:0/710046261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30003bd0 con 0x7f2c240051c0 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.308+0000 7f2c3fc39640 1 -- 172.21.15.26:0/710046261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 msgr2=0x7f2c2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.308+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c2400b9c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009fc0 tx=0x7f2c24005830 comp rx=0 tx=0).stop 2024-03-06T07:24:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.308+0000 7f2c3fc39640 1 -- 172.21.15.26:0/710046261 shutdown_connections 2024-03-06T07:24:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.308+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/710046261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c2400b9c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.308+0000 7f2c3fc39640 1 -- 172.21.15.26:0/710046261 >> 172.21.15.26:0/710046261 conn(0x7f2c380ac590 msgr2=0x7f2c380ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.308+0000 7f2c3fc39640 1 -- 172.21.15.26:0/710046261 shutdown_connections 2024-03-06T07:24:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.308+0000 7f2c3fc39640 1 -- 172.21.15.26:0/710046261 wait complete. 2024-03-06T07:24:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.309+0000 7f2c3fc39640 1 Processor -- start 2024-03-06T07:24:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.309+0000 7f2c3fc39640 1 -- start start 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3fc39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c380c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3fc39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c380c4500 0x7f2c380c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3fc39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c380c2c20 0x7f2c38151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3fc39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3806c550 con 0x7f2c240051c0 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3fc39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3806c270 con 0x7f2c380c2c20 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3fc39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3806da80 con 0x7f2c380c4500 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c380c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c380c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48748/0 (socket says 172.21.15.26:48748) 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 -- 172.21.15.26:0/1527304658 learned_addr learned my addr 172.21.15.26:0/1527304658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3e1d6640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c380c2c20 0x7f2c38151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d1d4640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c380c4500 0x7f2c380c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 -- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c380c4500 msgr2=0x7f2c380c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c380c4500 0x7f2c380c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 -- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c380c2c20 msgr2=0x7f2c38151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c380c2c20 0x7f2c38151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 -- 172.21.15.26:0/1527304658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c381ebba0 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3d9d5640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c380c3fc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400f620 tx=0x7f2c3400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c2e7fc640 1 -- 172.21.15.26:0/1527304658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340052e0 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c2e7fc640 1 -- 172.21.15.26:0/1527304658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c34018070 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c380c1c00 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.310+0000 7f2c2e7fc640 1 -- 172.21.15.26:0/1527304658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34003810 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.311+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c380c21c0 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.312+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c38075300 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.312+0000 7f2c2e7fc640 1 -- 172.21.15.26:0/1527304658 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c340039b0 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.312+0000 7f2c2e7fc640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.312+0000 7f2c2e7fc640 1 -- 172.21.15.26:0/1527304658 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2c34089db0 con 0x7f2c240051c0 2024-03-06T07:24:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.313+0000 7f2c3d1d4640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.313+0000 7f2c3d1d4640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2c280012b0 tx=0x7f2c28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.317+0000 7f2c2e7fc640 1 -- 172.21.15.26:0/1527304658 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2c38075300 con 0x7f2c240051c0 2024-03-06T07:24:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.497+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c3806ce80 con 0x7f2c240051c0 2024-03-06T07:24:36.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.497+0000 7f2c2e7fc640 1 -- 172.21.15.26:0/1527304658 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c3806ce80 con 0x7f2c240051c0 2024-03-06T07:24:36.499 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c1c06b500 msgr2=0x7f2c1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2c280012b0 tx=0x7f2c28009040 comp rx=0 tx=0).stop 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 msgr2=0x7f2c380c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c380c3fc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400f620 tx=0x7f2c3400fae0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 shutdown_connections 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c380c2c20 0x7f2c38151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c380c4500 0x7f2c380c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 --2- 172.21.15.26:0/1527304658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240051c0 0x7f2c380c3fc0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 >> 172.21.15.26:0/1527304658 conn(0x7f2c380ac590 msgr2=0x7f2c380c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 shutdown_connections 2024-03-06T07:24:36.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.499+0000 7f2c3fc39640 1 -- 172.21.15.26:0/1527304658 wait complete. 2024-03-06T07:24:36.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady' 2024-03-06T07:24:36.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.611+0000 7f21db9eb640 1 Processor -- start 2024-03-06T07:24:36.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21db9eb640 1 -- start start 2024-03-06T07:24:36.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21db9eb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21d40c1aa0 0x7f21d41529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21db9eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f21d4155a40 0x7f21d4155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21db9eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d40c28d0 con 0x7f21d414f790 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21db9eb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d40c2680 con 0x7f21d40c1aa0 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21db9eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d40c2bb0 con 0x7f21d4155a40 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21d8f86640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21d40c1aa0 0x7f21d41529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21d9787640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21d414f790 0x7f21d40c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35802/0 (socket says 172.21.15.26:35802) 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.612+0000 7f21d9787640 1 -- 172.21.15.26:0/3581248047 learned_addr learned my addr 172.21.15.26:0/3581248047 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.613+0000 7f21d9f88640 1 --2- 172.21.15.26:0/3581248047 >> v2:172.21.15.117:3300/0 conn(0x7f21d4155a40 0x7f21d4155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.613+0000 7f21d8f86640 1 -- 172.21.15.26:0/3581248047 >> v2:172.21.15.117:3300/0 conn(0x7f21d4155a40 msgr2=0x7f21d4155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.613+0000 7f21d8f86640 1 --2- 172.21.15.26:0/3581248047 >> v2:172.21.15.117:3300/0 conn(0x7f21d4155a40 0x7f21d4155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.613+0000 7f21d8f86640 1 -- 172.21.15.26:0/3581248047 >> v1:172.21.15.26:6789/0 conn(0x7f21d414f790 legacy=0x7f21d40c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.613+0000 7f21d8f86640 1 -- 172.21.15.26:0/3581248047 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40c2e90 con 0x7f21d40c1aa0 2024-03-06T07:24:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.613+0000 7f21cbfff640 1 -- 172.21.15.26:0/3581248047 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38573756 0 0) 0x7f21d40c28d0 con 0x7f21d414f790 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21d8f86640 1 --2- 172.21.15.26:0/3581248047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d40c1aa0 0x7f21d41529d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f21d406c960 tx=0x7f21c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf27ce089ff2fadf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21cbfff640 1 -- 172.21.15.26:0/3581248047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4035020 con 0x7f21d40c1aa0 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21cbfff640 1 -- 172.21.15.26:0/3581248047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4031830 con 0x7f21d40c1aa0 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21cbfff640 1 -- 172.21.15.26:0/3581248047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4031c00 con 0x7f21d40c1aa0 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 -- 172.21.15.26:0/3581248047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d40c1aa0 msgr2=0x7f21d41529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 --2- 172.21.15.26:0/3581248047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d40c1aa0 0x7f21d41529d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f21d406c960 tx=0x7f21c402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 -- 172.21.15.26:0/3581248047 shutdown_connections 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 --2- 172.21.15.26:0/3581248047 >> v2:172.21.15.117:3300/0 conn(0x7f21d4155a40 0x7f21d4155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 --2- 172.21.15.26:0/3581248047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d40c1aa0 0x7f21d41529d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 -- 172.21.15.26:0/3581248047 >> 172.21.15.26:0/3581248047 conn(0x7f21d40c6710 msgr2=0x7f21d40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 -- 172.21.15.26:0/3581248047 shutdown_connections 2024-03-06T07:24:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.614+0000 7f21db9eb640 1 -- 172.21.15.26:0/3581248047 wait complete. 2024-03-06T07:24:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.615+0000 7f21db9eb640 1 Processor -- start 2024-03-06T07:24:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.615+0000 7f21db9eb640 1 -- start start 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21db9eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d40c1aa0 0x7f21d414c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21db9eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d414f790 0x7f21d41eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21db9eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d4155a40 0x7f21d41eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21db9eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40c0d60 con 0x7f21d414f790 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21db9eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d4156390 con 0x7f21d40c1aa0 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21db9eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40c5e30 con 0x7f21d4155a40 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d414f790 0x7f21d41eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d414f790 0x7f21d41eb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48772/0 (socket says 172.21.15.26:48772) 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 -- 172.21.15.26:0/840411739 learned_addr learned my addr 172.21.15.26:0/840411739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d9f88640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d4155a40 0x7f21d41eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d9787640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d40c1aa0 0x7f21d414c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 -- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d4155a40 msgr2=0x7f21d41eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d4155a40 0x7f21d41eb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 -- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d40c1aa0 msgr2=0x7f21d414c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d40c1aa0 0x7f21d414c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.616+0000 7f21d8f86640 1 -- 172.21.15.26:0/840411739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40702f0 con 0x7f21d414f790 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.617+0000 7f21d8f86640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d414f790 0x7f21d41eb2b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009890 tx=0x7f21c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.617+0000 7f21c9ffb640 1 -- 172.21.15.26:0/840411739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4041070 con 0x7f21d414f790 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.617+0000 7f21c9ffb640 1 -- 172.21.15.26:0/840411739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c403aa30 con 0x7f21d414f790 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.617+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d40705d0 con 0x7f21d414f790 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.617+0000 7f21c9ffb640 1 -- 172.21.15.26:0/840411739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4035040 con 0x7f21d414f790 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.617+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d4070b90 con 0x7f21d414f790 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.618+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d404f330 con 0x7f21d414f790 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.619+0000 7f21c9ffb640 1 -- 172.21.15.26:0/840411739 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21c4038020 con 0x7f21d414f790 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.619+0000 7f21c9ffb640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f21b806b4d0 0x7f21b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.619+0000 7f21c9ffb640 1 -- 172.21.15.26:0/840411739 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f21c40abb50 con 0x7f21d414f790 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.619+0000 7f21d9787640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f21b806b4d0 0x7f21b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.620+0000 7f21d9787640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f21b806b4d0 0x7f21b806d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21d0000f60 tx=0x7f21d0007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.623+0000 7f21c9ffb640 1 -- 172.21.15.26:0/840411739 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f21c407ab80 con 0x7f21d414f790 2024-03-06T07:24:36.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:36.878+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]} v 0) v1 -- 0x7f21d40c1f40 con 0x7f21d414f790 2024-03-06T07:24:37.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.882+0000 7f21c9ffb640 1 -- 172.21.15.26:0/840411739 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]}]=0 v270) v1 ==== 148+0+0 (secure 0 0 0) 0x7f21c407a480 con 0x7f21d414f790 2024-03-06T07:24:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f21b806b4d0 msgr2=0x7f21b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f21b806b4d0 0x7f21b806d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21d0000f60 tx=0x7f21d0007b50 comp rx=0 tx=0).stop 2024-03-06T07:24:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d414f790 msgr2=0x7f21d41eb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d414f790 0x7f21d41eb2b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009890 tx=0x7f21c40046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 shutdown_connections 2024-03-06T07:24:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f21b806b4d0 0x7f21b806d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d4155a40 0x7f21d41eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21d414f790 0x7f21d41eb2b0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 --2- 172.21.15.26:0/840411739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d40c1aa0 0x7f21d414c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 >> 172.21.15.26:0/840411739 conn(0x7f21d40c6710 msgr2=0x7f21d40c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 shutdown_connections 2024-03-06T07:24:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:37.885+0000 7f21db9eb640 1 -- 172.21.15.26:0/840411739 wait complete. 2024-03-06T07:24:37.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:24:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.009+0000 7f3884271640 1 Processor -- start 2024-03-06T07:24:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.009+0000 7f3884271640 1 -- start start 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.010+0000 7f3884271640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f387c06afc0 0x7f387c0ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.010+0000 7f3884271640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c06b440 con 0x7f387c0abe90 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.010+0000 7f3884271640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c0c1c90 con 0x7f387c14d350 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.010+0000 7f3884271640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c0ac750 con 0x7f387c06afc0 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.010+0000 7f388280e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f387c0abe90 0x7f387c15d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41318/0 (socket says 172.21.15.26:41318) 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.010+0000 7f388280e640 1 -- 172.21.15.26:0/206186241 learned_addr learned my addr 172.21.15.26:0/206186241 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388180c640 1 --2- 172.21.15.26:0/206186241 >> v2:172.21.15.117:3300/0 conn(0x7f387c06afc0 0x7f387c0ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507934250 0 0) 0x7f3874001490 con 0x7f387c0abe90 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3868003520 con 0x7f387c0abe90 2024-03-06T07:24:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757732519 0 0) 0x7f387c0c1c90 con 0x7f387c14d350 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3874001490 con 0x7f387c14d350 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006518925 0 0) 0x7f3868003520 con 0x7f387c0abe90 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 >> v2:172.21.15.117:3300/0 conn(0x7f387c06afc0 msgr2=0x7f387c0ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 --2- 172.21.15.26:0/206186241 >> v2:172.21.15.117:3300/0 conn(0x7f387c06afc0 0x7f387c0ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 >> v1:172.21.15.117:6789/0 conn(0x7f387c14d350 legacy=0x7f387c0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0ac470 con 0x7f387c0abe90 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f3874001bc0 con 0x7f387c0abe90 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388180c640 1 --2- 172.21.15.26:0/206186241 >> v2:172.21.15.117:3300/0 conn(0x7f387c06afc0 0x7f387c0ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 >> v1:172.21.15.26:6789/0 conn(0x7f387c0abe90 legacy=0x7f387c15d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3868003b40 0x7f3868003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868004800 0x7f3868004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38680051c0 0x7f386800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386800c0c0 con 0x7f38680051c0 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386800c270 con 0x7f3868003b40 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386800c470 con 0x7f3868004800 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388280e640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38680051c0 0x7f386800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388200d640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3868003b40 0x7f3868003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.011+0000 7f388180c640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868004800 0x7f3868004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388280e640 1 -- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868004800 msgr2=0x7f3868004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388280e640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868004800 0x7f3868004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388280e640 1 -- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3868003b40 msgr2=0x7f3868003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388280e640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3868003b40 0x7f3868003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388280e640 1 -- 172.21.15.26:0/206186241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3868003520 con 0x7f38680051c0 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388280e640 1 -- 172.21.15.26:0/206186241 reap_dead start 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388280e640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38680051c0 0x7f386800b9c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f387400a9c0 tx=0x7f3868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6ef4af1349f0b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400a5e0 con 0x7f38680051c0 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3874035420 con 0x7f38680051c0 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.012+0000 7f388100b640 1 -- 172.21.15.26:0/206186241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874035830 con 0x7f38680051c0 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.013+0000 7f3884271640 1 -- 172.21.15.26:0/206186241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38680051c0 msgr2=0x7f386800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.013+0000 7f3884271640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38680051c0 0x7f386800b9c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f387400a9c0 tx=0x7f3868005830 comp rx=0 tx=0).stop 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.013+0000 7f3884271640 1 -- 172.21.15.26:0/206186241 shutdown_connections 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.013+0000 7f3884271640 1 --2- 172.21.15.26:0/206186241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38680051c0 0x7f386800b9c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.013+0000 7f3884271640 1 -- 172.21.15.26:0/206186241 >> 172.21.15.26:0/206186241 conn(0x7f387c0c3b40 msgr2=0x7f387c0c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.013+0000 7f3884271640 1 -- 172.21.15.26:0/206186241 shutdown_connections 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.013+0000 7f3884271640 1 -- 172.21.15.26:0/206186241 wait complete. 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.014+0000 7f3884271640 1 Processor -- start 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.014+0000 7f3884271640 1 -- start start 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f3884271640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680051c0 0x7f387c0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f3884271640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f387c0c1500 0x7f387c1428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f3884271640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f387c1dcc80 0x7f387c142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f3884271640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c153c00 con 0x7f387c0c1500 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f3884271640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c1a40 con 0x7f38680051c0 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f3884271640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0aa160 con 0x7f387c1dcc80 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f387c0c1500 0x7f387c1428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f387c0c1500 0x7f387c1428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57212/0 (socket says 172.21.15.26:57212) 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 -- 172.21.15.26:0/1088633662 learned_addr learned my addr 172.21.15.26:0/1088633662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388200d640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680051c0 0x7f387c0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388280e640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f387c1dcc80 0x7f387c142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 -- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f387c1dcc80 msgr2=0x7f387c142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f387c1dcc80 0x7f387c142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 -- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680051c0 msgr2=0x7f387c0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680051c0 0x7f387c0c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.015+0000 7f388180c640 1 -- 172.21.15.26:0/1088633662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c1ed020 con 0x7f387c0c1500 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.016+0000 7f388180c640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f387c0c1500 0x7f387c1428a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f386c011af0 tx=0x7f386c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.016+0000 7f3872ffd640 1 -- 172.21.15.26:0/1088633662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c00fb50 con 0x7f387c0c1500 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.016+0000 7f3872ffd640 1 -- 172.21.15.26:0/1088633662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386c00fcf0 con 0x7f387c0c1500 2024-03-06T07:24:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.016+0000 7f3872ffd640 1 -- 172.21.15.26:0/1088633662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c00fea0 con 0x7f387c0c1500 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.016+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c1521f0 con 0x7f387c0c1500 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.016+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c1eccd0 con 0x7f387c0c1500 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.017+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c151fe0 con 0x7f387c0c1500 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.017+0000 7f3872ffd640 1 -- 172.21.15.26:0/1088633662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f386c012040 con 0x7f387c0c1500 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.018+0000 7f3872ffd640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f386006b500 0x7f386006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.018+0000 7f3872ffd640 1 -- 172.21.15.26:0/1088633662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f386c08a2f0 con 0x7f387c0c1500 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.018+0000 7f388200d640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f386006b500 0x7f386006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.018+0000 7f388200d640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f386006b500 0x7f386006d9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f3878009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.022+0000 7f3872ffd640 1 -- 172.21.15.26:0/1088633662 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f386c019030 con 0x7f387c0c1500 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.204+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f387c0734a0 con 0x7f387c0c1500 2024-03-06T07:24:38.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.208+0000 7f3872ffd640 1 -- 172.21.15.26:0/1088633662 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f386c08a110 con 0x7f387c0c1500 2024-03-06T07:24:38.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f386006b500 msgr2=0x7f386006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f386006b500 0x7f386006d9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f3878009040 comp rx=0 tx=0).stop 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f387c0c1500 msgr2=0x7f387c1428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f387c0c1500 0x7f387c1428a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f386c011af0 tx=0x7f386c011ef0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 shutdown_connections 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f386006b500 0x7f386006d9c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f387c1dcc80 0x7f387c142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f387c0c1500 0x7f387c1428a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 --2- 172.21.15.26:0/1088633662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680051c0 0x7f387c0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 >> 172.21.15.26:0/1088633662 conn(0x7f387c0c3b40 msgr2=0x7f387c06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 shutdown_connections 2024-03-06T07:24:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.210+0000 7f3884271640 1 -- 172.21.15.26:0/1088633662 wait complete. 2024-03-06T07:24:38.222 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:24:38.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:38.274 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:38.319 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:24:38.319 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:24:38.319 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:38.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:24:38.334 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:24:38.334 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:38.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:24:38.376 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:24:38.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.475+0000 7f3cdc713640 1 Processor -- start 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.475+0000 7f3cdc713640 1 -- start start 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.476+0000 7f3cdc713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 0x7f3cd4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.476+0000 7f3cdc713640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd4156180 con 0x7f3cd40c5330 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.476+0000 7f3cdc713640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd4159630 con 0x7f3cd40c8a00 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.476+0000 7f3cdc713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41587b0 con 0x7f3cd4152920 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.476+0000 7f3cdacb0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3cd40c5330 0x7f3cd415fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41322/0 (socket says 172.21.15.26:41322) 2024-03-06T07:24:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.476+0000 7f3cdacb0640 1 -- 172.21.15.26:0/474122058 learned_addr learned my addr 172.21.15.26:0/474122058 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cda4af640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 0x7f3cd4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620207545 0 0) 0x7f3cd0001490 con 0x7f3cd40c5330 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc0003520 con 0x7f3cd40c5330 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578107736 0 0) 0x7f3cd4159630 con 0x7f3cd40c8a00 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd0001490 con 0x7f3cd40c8a00 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172270469 0 0) 0x7f3cc0003520 con 0x7f3cd40c5330 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 msgr2=0x7f3cd4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 0x7f3cd4155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 >> v1:172.21.15.117:6789/0 conn(0x7f3cd40c8a00 legacy=0x7f3cd4158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40c7760 con 0x7f3cd40c5330 2024-03-06T07:24:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f3cd0001bc0 con 0x7f3cd40c5330 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 >> v1:172.21.15.26:6789/0 conn(0x7f3cd40c5330 legacy=0x7f3cd415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 0x7f3cc0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc00040d0 0x7f3cc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41587b0 con 0x7f3cc0004d40 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc000bb50 con 0x7f3cc00040d0 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc000bd20 con 0x7f3cd4152920 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cdacb0640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cda4af640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 0x7f3cc0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.477+0000 7f3cd9cae640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc00040d0 0x7f3cc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cdacb0640 1 -- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 msgr2=0x7f3cc0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cdacb0640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd4152920 0x7f3cc0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cdacb0640 1 -- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc00040d0 msgr2=0x7f3cc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cdacb0640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc00040d0 0x7f3cc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cdacb0640 1 -- 172.21.15.26:0/474122058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0003520 con 0x7f3cc0004d40 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cdacb0640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000a9d0 tx=0x7f3cc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dcc7c9aa1086bc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000a620 con 0x7f3cc0004d40 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd0007290 con 0x7f3cc0004d40 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.478+0000 7f3cd94ad640 1 -- 172.21.15.26:0/474122058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00076c0 con 0x7f3cc0004d40 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.479+0000 7f3cdc713640 1 -- 172.21.15.26:0/474122058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cc0004d40 msgr2=0x7f3cc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.479+0000 7f3cdc713640 1 --2- 172.21.15.26:0/474122058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000a9d0 tx=0x7f3cc0005830 comp rx=0 tx=0).stop 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.479+0000 7f3cdacb0640 1 -- 172.21.15.26:0/474122058 reap_dead start 2024-03-06T07:24:38.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.479+0000 7f3cdc713640 1 -- 172.21.15.26:0/474122058 shutdown_connections 2024-03-06T07:24:38.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.479+0000 7f3cdc713640 1 -- 172.21.15.26:0/474122058 >> 172.21.15.26:0/474122058 conn(0x7f3cd40ac560 msgr2=0x7f3cd40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.479+0000 7f3cdc713640 1 -- 172.21.15.26:0/474122058 shutdown_connections 2024-03-06T07:24:38.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.479+0000 7f3cdc713640 1 -- 172.21.15.26:0/474122058 wait complete. 2024-03-06T07:24:38.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.480+0000 7f3cdc713640 1 Processor -- start 2024-03-06T07:24:38.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.480+0000 7f3cdc713640 1 -- start start 2024-03-06T07:24:38.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cdc713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cc0004d40 0x7f3cd41e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cdc713640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd41ee080 0x7f3cd40bf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cdc713640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40bf5d0 0x7f3cd40c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cdc713640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4158a00 con 0x7f3cd41ee080 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cdc713640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c7d20 con 0x7f3cd40bf5d0 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cdc713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd406d030 con 0x7f3cc0004d40 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd41ee080 0x7f3cd40bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd41ee080 0x7f3cd40bf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57236/0 (socket says 172.21.15.26:57236) 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 -- 172.21.15.26:0/3520277407 learned_addr learned my addr 172.21.15.26:0/3520277407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cda4af640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cc0004d40 0x7f3cd41e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cdacb0640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40bf5d0 0x7f3cd40c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 -- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cc0004d40 msgr2=0x7f3cd41e22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cc0004d40 0x7f3cd41e22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 -- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40bf5d0 msgr2=0x7f3cd40c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40bf5d0 0x7f3cd40c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 -- 172.21.15.26:0/3520277407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40c47c0 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.481+0000 7f3cd9cae640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd41ee080 0x7f3cd40bf090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400f7a0 tx=0x7f3cc400fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.482+0000 7f3ccaffd640 1 -- 172.21.15.26:0/3520277407 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40050d0 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.482+0000 7f3ccaffd640 1 -- 172.21.15.26:0/3520277407 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4018070 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.482+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd40c44e0 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.482+0000 7f3ccaffd640 1 -- 172.21.15.26:0/3520277407 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400d730 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.482+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd40be4e0 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.483+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd41eeb00 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.483+0000 7f3ccaffd640 1 -- 172.21.15.26:0/3520277407 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cc400d8d0 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.484+0000 7f3ccaffd640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3cb806b4d0 0x7f3cb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.484+0000 7f3ccaffd640 1 -- 172.21.15.26:0/3520277407 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3cc4089ee0 con 0x7f3cd41ee080 2024-03-06T07:24:38.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.484+0000 7f3cda4af640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3cb806b4d0 0x7f3cb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.484+0000 7f3cda4af640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3cb806b4d0 0x7f3cb806d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40c4c10 tx=0x7f3ccc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.488+0000 7f3ccaffd640 1 -- 172.21.15.26:0/3520277407 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3cd41eeb00 con 0x7f3cd41ee080 2024-03-06T07:24:38.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.704+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cd404f330 con 0x7f3cd41ee080 2024-03-06T07:24:38.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.705+0000 7f3ccaffd640 1 -- 172.21.15.26:0/3520277407 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3699 (secure 0 0 0) 0x7f3cc4058f10 con 0x7f3cd41ee080 2024-03-06T07:24:38.707 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:38.707 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":100,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/256924791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":256924791},{"type":"v1","addr":"172.21.15.26:6834","nonce":256924791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3603807075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3603807075},{"type":"v1","addr":"172.21.15.117:6835","nonce":3603807075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":7378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/944337169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":944337169},{"type":"v1","addr":"172.21.15.117:6837","nonce":944337169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":7647,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/423809105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":423809105},{"type":"v1","addr":"172.21.15.26:6835","nonce":423809105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[]} 2024-03-06T07:24:38.707 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 100 2024-03-06T07:24:38.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3cb806b4d0 msgr2=0x7f3cb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3cb806b4d0 0x7f3cb806d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40c4c10 tx=0x7f3ccc009190 comp rx=0 tx=0).stop 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd41ee080 msgr2=0x7f3cd40bf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd41ee080 0x7f3cd40bf090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400f7a0 tx=0x7f3cc400fba0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 shutdown_connections 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3cb806b4d0 0x7f3cb806d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40bf5d0 0x7f3cd40c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd41ee080 0x7f3cd40bf090 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 --2- 172.21.15.26:0/3520277407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cc0004d40 0x7f3cd41e22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 >> 172.21.15.26:0/3520277407 conn(0x7f3cd40ac560 msgr2=0x7f3cd40aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 shutdown_connections 2024-03-06T07:24:38.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.707+0000 7f3cdc713640 1 -- 172.21.15.26:0/3520277407 wait complete. 2024-03-06T07:24:38.719 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:24:38.719 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:24:38.719 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:24:38.720 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:38.720 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:24:38.720 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:24:38.720 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:24:38.720 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:38.720 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:24:38.720 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:24:38.721 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:24:38.721 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:38.721 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:24:38.721 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:24:38.721 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:24:38.721 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:38.728 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:24:38.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:24:38.729 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:24:38.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:24:38.732 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:24:38.732 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:24:38.733 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:24:38.733 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:24:38.771 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:24:38.771 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:38.771 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:24:38.772 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:38.775 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:24:38.775 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:38.775 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:24:38.776 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:38.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:24:38.847 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:24:38.848 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:24:38.857 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:24:38.858 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.882+0000 7f1f262af640 1 Processor -- start 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.882+0000 7f1f262af640 1 -- start start 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.882+0000 7f1f262af640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f200c69f0 0x7f1f200c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.882+0000 7f1f262af640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f201558d0 0x7f1f20155d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.882+0000 7f1f2484c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f201558d0 0x7f1f20155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.882+0000 7f1f1f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f200c8950 0x7f1f200acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41338/0 (socket says 172.21.15.26:41338) 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f1f7fe640 1 -- 172.21.15.26:0/2363942221 learned_addr learned my addr 172.21.15.26:0/2363942221 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f2484c640 1 --2- 172.21.15.26:0/2363942221 >> v2:172.21.15.26:3300/0 conn(0x7f1f201558d0 0x7f1f20155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57270/0 (socket says 172.21.15.26:57270) 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.882+0000 7f1f262af640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f20155130 con 0x7f1f200c8950 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f262af640 1 -- 172.21.15.26:0/2363942221 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f20155440 con 0x7f1f201558d0 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f262af640 1 -- 172.21.15.26:0/2363942221 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200ad480 con 0x7f1f200c69f0 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/2363942221 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c69f0 0x7f1f200c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f1effd640 1 -- 172.21.15.26:0/2363942221 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746775719 0 0) 0x7f1f14001110 con 0x7f1f200c8950 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f1effd640 1 -- 172.21.15.26:0/2363942221 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f08003520 con 0x7f1f200c8950 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f2484c640 1 -- 172.21.15.26:0/2363942221 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c69f0 msgr2=0x7f1f200c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f2484c640 1 --2- 172.21.15.26:0/2363942221 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c69f0 0x7f1f200c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f2484c640 1 -- 172.21.15.26:0/2363942221 >> v1:172.21.15.26:6789/0 conn(0x7f1f200c8950 legacy=0x7f1f200acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.883+0000 7f1f2484c640 1 -- 172.21.15.26:0/2363942221 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200c5d50 con 0x7f1f201558d0 2024-03-06T07:24:38.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.884+0000 7f1f2484c640 1 --2- 172.21.15.26:0/2363942221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f20155d00 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1f200aa430 tx=0x7f1f1802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de1ad4ebeecfd25 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.887+0000 7f1f1effd640 1 -- 172.21.15.26:0/2363942221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18039070 con 0x7f1f201558d0 2024-03-06T07:24:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.887+0000 7f1f1effd640 1 -- 172.21.15.26:0/2363942221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1802cbf0 con 0x7f1f201558d0 2024-03-06T07:24:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.887+0000 7f1f1effd640 1 -- 172.21.15.26:0/2363942221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18034440 con 0x7f1f201558d0 2024-03-06T07:24:38.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 -- 172.21.15.26:0/2363942221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 msgr2=0x7f1f20155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 --2- 172.21.15.26:0/2363942221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f20155d00 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1f200aa430 tx=0x7f1f1802be40 comp rx=0 tx=0).stop 2024-03-06T07:24:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 -- 172.21.15.26:0/2363942221 shutdown_connections 2024-03-06T07:24:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 --2- 172.21.15.26:0/2363942221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f20155d00 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 --2- 172.21.15.26:0/2363942221 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c69f0 0x7f1f200c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 -- 172.21.15.26:0/2363942221 >> 172.21.15.26:0/2363942221 conn(0x7f1f200cae70 msgr2=0x7f1f200cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 -- 172.21.15.26:0/2363942221 shutdown_connections 2024-03-06T07:24:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.888+0000 7f1f262af640 1 -- 172.21.15.26:0/2363942221 wait complete. 2024-03-06T07:24:38.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.889+0000 7f1f262af640 1 Processor -- start 2024-03-06T07:24:38.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.890+0000 7f1f262af640 1 -- start start 2024-03-06T07:24:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.890+0000 7f1f262af640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c69f0 0x7f1f200c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.890+0000 7f1f262af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f200c8950 0x7f1f201ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.890+0000 7f1f262af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f201ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.890+0000 7f1f262af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20152a20 con 0x7f1f201558d0 2024-03-06T07:24:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.890+0000 7f1f262af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200c6310 con 0x7f1f200c8950 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.890+0000 7f1f262af640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20152ba0 con 0x7f1f200c69f0 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f201ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f201ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57272/0 (socket says 172.21.15.26:57272) 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 -- 172.21.15.26:0/81468666 learned_addr learned my addr 172.21.15.26:0/81468666 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f1f7fe640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f200c8950 0x7f1f201ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c69f0 0x7f1f200c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 -- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c69f0 msgr2=0x7f1f200c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c69f0 0x7f1f200c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 -- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f200c8950 msgr2=0x7f1f201ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f200c8950 0x7f1f201ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 -- 172.21.15.26:0/81468666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2014c900 con 0x7f1f201558d0 2024-03-06T07:24:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c69f0 0x7f1f200c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f2484c640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f201ee350 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009e80 tx=0x7f1f18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f1cff9640 1 -- 172.21.15.26:0/81468666 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18039050 con 0x7f1f201558d0 2024-03-06T07:24:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f1cff9640 1 -- 172.21.15.26:0/81468666 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f18045070 con 0x7f1f201558d0 2024-03-06T07:24:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f1cff9640 1 -- 172.21.15.26:0/81468666 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18009450 con 0x7f1f201558d0 2024-03-06T07:24:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f20158ff0 con 0x7f1f201558d0 2024-03-06T07:24:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.891+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f20158a30 con 0x7f1f201558d0 2024-03-06T07:24:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.892+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2004f330 con 0x7f1f201558d0 2024-03-06T07:24:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.893+0000 7f1f1cff9640 1 -- 172.21.15.26:0/81468666 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f1803e850 con 0x7f1f201558d0 2024-03-06T07:24:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.893+0000 7f1f1cff9640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0c06b4d0 0x7f1f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.893+0000 7f1f1cff9640 1 -- 172.21.15.26:0/81468666 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1f180acc60 con 0x7f1f201558d0 2024-03-06T07:24:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.893+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0c06b4d0 0x7f1f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.894+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0c06b4d0 0x7f1f0c06d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009a20 tx=0x7f1f10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:38.898+0000 7f1f1cff9640 1 -- 172.21.15.26:0/81468666 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f18032e20 con 0x7f1f201558d0 2024-03-06T07:24:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.090+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1f200ca650 con 0x7f1f201558d0 2024-03-06T07:24:39.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.907+0000 7f1f1cff9640 1 -- 172.21.15.26:0/81468666 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f1807bdb0 con 0x7f1f201558d0 2024-03-06T07:24:39.909 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:24:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0c06b4d0 msgr2=0x7f1f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0c06b4d0 0x7f1f0c06d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009a20 tx=0x7f1f10008040 comp rx=0 tx=0).stop 2024-03-06T07:24:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 msgr2=0x7f1f201ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f201ee350 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009e80 tx=0x7f1f18004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 shutdown_connections 2024-03-06T07:24:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0c06b4d0 0x7f1f0c06d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f201558d0 0x7f1f201ee350 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f200c8950 0x7f1f201ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 --2- 172.21.15.26:0/81468666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c69f0 0x7f1f200c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 >> 172.21.15.26:0/81468666 conn(0x7f1f200cae70 msgr2=0x7f1f20150790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 shutdown_connections 2024-03-06T07:24:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:39.910+0000 7f1f262af640 1 -- 172.21.15.26:0/81468666 wait complete. 2024-03-06T07:24:39.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:24:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.032+0000 7f17051e9640 1 Processor -- start 2024-03-06T07:24:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.032+0000 7f17051e9640 1 -- start start 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f17051e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f170006afc0 0x7f170006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f17051e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f17000c20b0 0x7f1700153180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f17051e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1700153dd0 con 0x7f17000c25c0 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f17051e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17000a9530 con 0x7f17000c20b0 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f17051e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17000a9810 con 0x7f170006afc0 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f16fed9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f170006afc0 0x7f170006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f16fe59c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f17000c20b0 0x7f1700153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f16ff59e640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f17000c25c0 0x7f17001536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:40986/0 (socket says 172.21.15.26:40986) 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f16ff59e640 1 -- 172.21.15.26:0/744402491 learned_addr learned my addr 172.21.15.26:0/744402491 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:40.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f16fdd9b640 1 -- 172.21.15.26:0/744402491 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676640807 0 0) 0x7f1700153dd0 con 0x7f17000c25c0 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.033+0000 7f16fdd9b640 1 -- 172.21.15.26:0/744402491 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e8003520 con 0x7f17000c25c0 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.034+0000 7f16fed9d640 1 -- 172.21.15.26:0/744402491 >> v2:172.21.15.117:3300/0 conn(0x7f17000c20b0 msgr2=0x7f1700153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.034+0000 7f16fed9d640 1 --2- 172.21.15.26:0/744402491 >> v2:172.21.15.117:3300/0 conn(0x7f17000c20b0 0x7f1700153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.034+0000 7f16fed9d640 1 -- 172.21.15.26:0/744402491 >> v1:172.21.15.117:6789/0 conn(0x7f17000c25c0 legacy=0x7f17001536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.034+0000 7f16fed9d640 1 -- 172.21.15.26:0/744402491 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17000a9af0 con 0x7f170006afc0 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.034+0000 7f16fe59c640 1 --2- 172.21.15.26:0/744402491 >> v2:172.21.15.117:3300/0 conn(0x7f17000c20b0 0x7f1700153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.034+0000 7f16fed9d640 1 --2- 172.21.15.26:0/744402491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f170006afc0 0x7f170006b3a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16f00097a0 tx=0x7f16f002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7660bb6337dbd6ef server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.034+0000 7f16fdd9b640 1 -- 172.21.15.26:0/744402491 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0039070 con 0x7f170006afc0 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f16fdd9b640 1 -- 172.21.15.26:0/744402491 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f00316f0 con 0x7f170006afc0 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f16fdd9b640 1 -- 172.21.15.26:0/744402491 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0031ac0 con 0x7f170006afc0 2024-03-06T07:24:40.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 -- 172.21.15.26:0/744402491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f170006afc0 msgr2=0x7f170006b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 --2- 172.21.15.26:0/744402491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f170006afc0 0x7f170006b3a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16f00097a0 tx=0x7f16f002b790 comp rx=0 tx=0).stop 2024-03-06T07:24:40.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 -- 172.21.15.26:0/744402491 shutdown_connections 2024-03-06T07:24:40.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 --2- 172.21.15.26:0/744402491 >> v2:172.21.15.117:3300/0 conn(0x7f17000c20b0 0x7f1700153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 --2- 172.21.15.26:0/744402491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f170006afc0 0x7f170006b3a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 -- 172.21.15.26:0/744402491 >> 172.21.15.26:0/744402491 conn(0x7f17000c76b0 msgr2=0x7f17000c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:40.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 -- 172.21.15.26:0/744402491 shutdown_connections 2024-03-06T07:24:40.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.035+0000 7f17051e9640 1 -- 172.21.15.26:0/744402491 wait complete. 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.036+0000 7f17051e9640 1 Processor -- start 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f17051e9640 1 -- start start 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f17051e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170006afc0 0x7f17001ec480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f17051e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17000c20b0 0x7f1700148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f17051e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000c25c0 0x7f17001492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f17051e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700150980 con 0x7f170006afc0 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f17051e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17000aa0b0 con 0x7f17000c25c0 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f17051e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1700150b00 con 0x7f17000c20b0 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f16fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170006afc0 0x7f17001ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f16fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170006afc0 0x7f17001ec480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57284/0 (socket says 172.21.15.26:57284) 2024-03-06T07:24:40.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f16fed9d640 1 -- 172.21.15.26:0/1071137152 learned_addr learned my addr 172.21.15.26:0/1071137152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f16fe59c640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17000c20b0 0x7f1700148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.037+0000 7f16ff59e640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000c25c0 0x7f17001492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16fed9d640 1 -- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17000c20b0 msgr2=0x7f1700148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16fed9d640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17000c20b0 0x7f1700148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16fed9d640 1 -- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000c25c0 msgr2=0x7f17001492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16fed9d640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000c25c0 0x7f17001492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16fed9d640 1 -- 172.21.15.26:0/1071137152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17001dc530 con 0x7f170006afc0 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16ff59e640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000c25c0 0x7f17001492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16fe59c640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17000c20b0 0x7f1700148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16fed9d640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170006afc0 0x7f17001ec480 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f16f0037040 tx=0x7f16f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16e77fe640 1 -- 172.21.15.26:0/1071137152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0039040 con 0x7f170006afc0 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16e77fe640 1 -- 172.21.15.26:0/1071137152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f0004030 con 0x7f170006afc0 2024-03-06T07:24:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17001dc7d0 con 0x7f170006afc0 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f16e77fe640 1 -- 172.21.15.26:0/1071137152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0004400 con 0x7f170006afc0 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.038+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1700149810 con 0x7f170006afc0 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.039+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17000ac860 con 0x7f170006afc0 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.040+0000 7f16e77fe640 1 -- 172.21.15.26:0/1071137152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16f003c070 con 0x7f170006afc0 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.040+0000 7f16e77fe640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16ec06b4d0 0x7f16ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.040+0000 7f16e77fe640 1 -- 172.21.15.26:0/1071137152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f16f00ab900 con 0x7f170006afc0 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.040+0000 7f16fe59c640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16ec06b4d0 0x7f16ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.041+0000 7f16fe59c640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16ec06b4d0 0x7f16ec06d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1700151ee0 tx=0x7f16f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.044+0000 7f16e77fe640 1 -- 172.21.15.26:0/1071137152 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f16f004c030 con 0x7f170006afc0 2024-03-06T07:24:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.298+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f170004f330 con 0x7f170006afc0 2024-03-06T07:24:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.298+0000 7f16e77fe640 1 -- 172.21.15.26:0/1071137152 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f16f0043030 con 0x7f170006afc0 2024-03-06T07:24:40.300 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:40.300 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:40.300 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:24:40.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.300 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:24:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:40.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:24:40.304 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.305 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.306 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.307 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.309 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:24:40.310 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.311 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:24:40.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:24:40.313 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:24:40.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:40.313 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:24:40.313 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16ec06b4d0 msgr2=0x7f16ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16ec06b4d0 0x7f16ec06d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1700151ee0 tx=0x7f16f4008040 comp rx=0 tx=0).stop 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170006afc0 msgr2=0x7f17001ec480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170006afc0 0x7f17001ec480 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f16f0037040 tx=0x7f16f00046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 shutdown_connections 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f16ec06b4d0 0x7f16ec06d990 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000c25c0 0x7f17001492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17000c20b0 0x7f1700148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 --2- 172.21.15.26:0/1071137152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170006afc0 0x7f17001ec480 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 >> 172.21.15.26:0/1071137152 conn(0x7f17000c76b0 msgr2=0x7f170014d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 shutdown_connections 2024-03-06T07:24:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.300+0000 7f17051e9640 1 -- 172.21.15.26:0/1071137152 wait complete. 2024-03-06T07:24:40.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:40.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.416+0000 7f2ac4372640 1 Processor -- start 2024-03-06T07:24:40.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.416+0000 7f2ac4372640 1 -- start start 2024-03-06T07:24:40.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac4372640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2abc1527b0 0x7f2abc159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac4372640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2abc0c7180 0x7f2abc0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac4372640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc0c7c60 con 0x7f2abc155b20 2024-03-06T07:24:40.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac4372640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc159640 con 0x7f2abc1527b0 2024-03-06T07:24:40.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac4372640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc0c8df0 con 0x7f2abc0c7180 2024-03-06T07:24:40.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac210e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2abc1527b0 0x7f2abc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac210e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2abc1527b0 0x7f2abc159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57296/0 (socket says 172.21.15.26:57296) 2024-03-06T07:24:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac210e640 1 -- 172.21.15.26:0/2237542275 learned_addr learned my addr 172.21.15.26:0/2237542275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac190d640 1 --2- 172.21.15.26:0/2237542275 >> v2:172.21.15.117:3300/0 conn(0x7f2abc0c7180 0x7f2abc0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.417+0000 7f2ac210e640 1 -- 172.21.15.26:0/2237542275 >> v2:172.21.15.117:3300/0 conn(0x7f2abc0c7180 msgr2=0x7f2abc0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.418+0000 7f2ac210e640 1 --2- 172.21.15.26:0/2237542275 >> v2:172.21.15.117:3300/0 conn(0x7f2abc0c7180 0x7f2abc0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.418+0000 7f2ac210e640 1 -- 172.21.15.26:0/2237542275 >> v1:172.21.15.117:6789/0 conn(0x7f2abc155b20 legacy=0x7f2abc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:40.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.418+0000 7f2ac210e640 1 -- 172.21.15.26:0/2237542275 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0c8060 con 0x7f2abc1527b0 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.418+0000 7f2ac210e640 1 --2- 172.21.15.26:0/2237542275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc1527b0 0x7f2abc159100 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2abc08cac0 tx=0x7f2ab802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56c45be39002063 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.418+0000 7f2ac110c640 1 -- 172.21.15.26:0/2237542275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab802e070 con 0x7f2abc1527b0 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.418+0000 7f2ac110c640 1 -- 172.21.15.26:0/2237542275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab8037070 con 0x7f2abc1527b0 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.418+0000 7f2ac110c640 1 -- 172.21.15.26:0/2237542275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8031a00 con 0x7f2abc1527b0 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 -- 172.21.15.26:0/2237542275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc1527b0 msgr2=0x7f2abc159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2237542275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc1527b0 0x7f2abc159100 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2abc08cac0 tx=0x7f2ab802b880 comp rx=0 tx=0).stop 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 -- 172.21.15.26:0/2237542275 shutdown_connections 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2237542275 >> v2:172.21.15.117:3300/0 conn(0x7f2abc0c7180 0x7f2abc0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2237542275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc1527b0 0x7f2abc159100 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 -- 172.21.15.26:0/2237542275 >> 172.21.15.26:0/2237542275 conn(0x7f2abc0ac560 msgr2=0x7f2abc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 -- 172.21.15.26:0/2237542275 shutdown_connections 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.419+0000 7f2ac4372640 1 -- 172.21.15.26:0/2237542275 wait complete. 2024-03-06T07:24:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.420+0000 7f2ac4372640 1 Processor -- start 2024-03-06T07:24:40.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.420+0000 7f2ac4372640 1 -- start start 2024-03-06T07:24:40.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac4372640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2abc0c7180 0x7f2abc14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac4372640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2abc1527b0 0x7f2abc0aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac4372640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc155b20 0x7f2abc154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac4372640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc155320 con 0x7f2abc155b20 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac4372640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0c7980 con 0x7f2abc1527b0 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac4372640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc155600 con 0x7f2abc0c7180 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc155b20 0x7f2abc154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc155b20 0x7f2abc154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57308/0 (socket says 172.21.15.26:57308) 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 -- 172.21.15.26:0/2775199163 learned_addr learned my addr 172.21.15.26:0/2775199163 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac190d640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2abc1527b0 0x7f2abc0aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac210e640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2abc0c7180 0x7f2abc14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 -- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2abc0c7180 msgr2=0x7f2abc14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2abc0c7180 0x7f2abc14bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 -- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2abc1527b0 msgr2=0x7f2abc0aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2abc1527b0 0x7f2abc0aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 -- 172.21.15.26:0/2775199163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc1ee320 con 0x7f2abc155b20 2024-03-06T07:24:40.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac210e640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2abc0c7180 0x7f2abc14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ac290f640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc155b20 0x7f2abc154040 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c780 tx=0x7f2ab400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.421+0000 7f2ab2ffd640 1 -- 172.21.15.26:0/2775199163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4004b60 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.422+0000 7f2ab2ffd640 1 -- 172.21.15.26:0/2775199163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab4004d00 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.422+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc1ee600 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.422+0000 7f2ab2ffd640 1 -- 172.21.15.26:0/2775199163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400aa30 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.422+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc1eebc0 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.422+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc04f330 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.423+0000 7f2ab2ffd640 1 -- 172.21.15.26:0/2775199163 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ab400abd0 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.423+0000 7f2ab2ffd640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2aa006b4d0 0x7f2aa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.423+0000 7f2ab2ffd640 1 -- 172.21.15.26:0/2775199163 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2ab408a1c0 con 0x7f2abc155b20 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.424+0000 7f2ac210e640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2aa006b4d0 0x7f2aa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.424+0000 7f2ac210e640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2aa006b4d0 0x7f2aa006d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8036000 tx=0x7f2ab8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.428+0000 7f2ab2ffd640 1 -- 172.21.15.26:0/2775199163 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ab4020070 con 0x7f2abc155b20 2024-03-06T07:24:40.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.609+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2abc06ce00 con 0x7f2abc155b20 2024-03-06T07:24:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.609+0000 7f2ab2ffd640 1 -- 172.21.15.26:0/2775199163 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+13186 (secure 0 0 0) 0x7f2ab4059290 con 0x7f2abc155b20 2024-03-06T07:24:40.611 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:40.611 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:39.906721+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2aa006b4d0 msgr2=0x7f2aa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2aa006b4d0 0x7f2aa006d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8036000 tx=0x7f2ab8008030 comp rx=0 tx=0).stop 2024-03-06T07:24:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc155b20 msgr2=0x7f2abc154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc155b20 0x7f2abc154040 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c780 tx=0x7f2ab400cc40 comp rx=0 tx=0).stop 2024-03-06T07:24:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 shutdown_connections 2024-03-06T07:24:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2aa006b4d0 0x7f2aa006d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2abc155b20 0x7f2abc154040 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2abc1527b0 0x7f2abc0aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 --2- 172.21.15.26:0/2775199163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2abc0c7180 0x7f2abc14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 >> 172.21.15.26:0/2775199163 conn(0x7f2abc0ac560 msgr2=0x7f2abc0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 shutdown_connections 2024-03-06T07:24:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.611+0000 7f2ac4372640 1 -- 172.21.15.26:0/2775199163 wait complete. 2024-03-06T07:24:40.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:40.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.727+0000 7fc23a13a640 1 Processor -- start 2024-03-06T07:24:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.727+0000 7fc23a13a640 1 -- start start 2024-03-06T07:24:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc23a13a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2341529f0 0x7fc2340c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc23a13a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2340c5370 0x7fc2340c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc23a13a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc234152dd0 con 0x7fc234155530 2024-03-06T07:24:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc23a13a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc234159710 con 0x7fc2341529f0 2024-03-06T07:24:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc23a13a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340c68b0 con 0x7fc2340c5370 2024-03-06T07:24:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc2337fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2341529f0 0x7fc2340c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc2337fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2341529f0 0x7fc2340c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57322/0 (socket says 172.21.15.26:57322) 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc2337fe640 1 -- 172.21.15.26:0/3136518025 learned_addr learned my addr 172.21.15.26:0/3136518025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc232ffd640 1 --2- 172.21.15.26:0/3136518025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2340c5370 0x7fc2340c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc2337fe640 1 -- 172.21.15.26:0/3136518025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2340c5370 msgr2=0x7fc2340c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc2337fe640 1 --2- 172.21.15.26:0/3136518025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2340c5370 0x7fc2340c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc2337fe640 1 -- 172.21.15.26:0/3136518025 >> v1:172.21.15.117:6789/0 conn(0x7fc234155530 legacy=0x7fc2341559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.728+0000 7fc2337fe640 1 -- 172.21.15.26:0/3136518025 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234158b00 con 0x7fc2341529f0 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.729+0000 7fc2337fe640 1 --2- 172.21.15.26:0/3136518025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2341529f0 0x7fc2340c88f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc224009870 tx=0x7fc22402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d23169da0b92c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.729+0000 7fc2327fc640 1 -- 172.21.15.26:0/3136518025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224035020 con 0x7fc2341529f0 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.729+0000 7fc2327fc640 1 -- 172.21.15.26:0/3136518025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224031640 con 0x7fc2341529f0 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.729+0000 7fc2327fc640 1 -- 172.21.15.26:0/3136518025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22402d040 con 0x7fc2341529f0 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.729+0000 7fc23a13a640 1 -- 172.21.15.26:0/3136518025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2341529f0 msgr2=0x7fc2340c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.729+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3136518025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2341529f0 0x7fc2340c88f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc224009870 tx=0x7fc22402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.730+0000 7fc23a13a640 1 -- 172.21.15.26:0/3136518025 shutdown_connections 2024-03-06T07:24:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.730+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3136518025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2340c5370 0x7fc2340c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.730+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3136518025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2341529f0 0x7fc2340c88f0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.730+0000 7fc23a13a640 1 -- 172.21.15.26:0/3136518025 >> 172.21.15.26:0/3136518025 conn(0x7fc2340ac580 msgr2=0x7fc2340ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.730+0000 7fc23a13a640 1 -- 172.21.15.26:0/3136518025 shutdown_connections 2024-03-06T07:24:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.730+0000 7fc23a13a640 1 -- 172.21.15.26:0/3136518025 wait complete. 2024-03-06T07:24:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.731+0000 7fc23a13a640 1 Processor -- start 2024-03-06T07:24:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.731+0000 7fc23a13a640 1 -- start start 2024-03-06T07:24:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.731+0000 7fc23a13a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2340c5370 0x7fc2340b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.731+0000 7fc23a13a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2341529f0 0x7fc234147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc23a13a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc234155530 0x7fc234148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc23a13a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341593a0 con 0x7fc234155530 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc23a13a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341590c0 con 0x7fc2340c5370 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc23a13a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341560d0 con 0x7fc2341529f0 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc234155530 0x7fc234148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc234155530 0x7fc234148210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57338/0 (socket says 172.21.15.26:57338) 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 -- 172.21.15.26:0/3304968189 learned_addr learned my addr 172.21.15.26:0/3304968189 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc2337fe640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2340c5370 0x7fc2340b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc232ffd640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2341529f0 0x7fc234147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 -- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2341529f0 msgr2=0x7fc234147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2341529f0 0x7fc234147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 -- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2340c5370 msgr2=0x7fc2340b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2340c5370 0x7fc2340b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 -- 172.21.15.26:0/3304968189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2341eb400 con 0x7fc234155530 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc2337fe640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2340c5370 0x7fc2340b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc232ffd640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2341529f0 0x7fc234147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc233fff640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc234155530 0x7fc234148210 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c6b0 tx=0x7fc22c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc21bfff640 1 -- 172.21.15.26:0/3304968189 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c003b70 con 0x7fc234155530 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc21bfff640 1 -- 172.21.15.26:0/3304968189 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22c003d30 con 0x7fc234155530 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2341eb6e0 con 0x7fc234155530 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc21bfff640 1 -- 172.21.15.26:0/3304968189 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c005310 con 0x7fc234155530 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.732+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc234148750 con 0x7fc234155530 2024-03-06T07:24:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.733+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23404f330 con 0x7fc234155530 2024-03-06T07:24:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.734+0000 7fc21bfff640 1 -- 172.21.15.26:0/3304968189 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc22c0054b0 con 0x7fc234155530 2024-03-06T07:24:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.734+0000 7fc21bfff640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc22006b4d0 0x7fc22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.734+0000 7fc21bfff640 1 -- 172.21.15.26:0/3304968189 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc22c089ec0 con 0x7fc234155530 2024-03-06T07:24:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.734+0000 7fc2337fe640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc22006b4d0 0x7fc22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.735+0000 7fc2337fe640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc22006b4d0 0x7fc22006d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc224036000 tx=0x7fc22403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.738+0000 7fc21bfff640 1 -- 172.21.15.26:0/3304968189 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc22c058f00 con 0x7fc234155530 2024-03-06T07:24:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.918+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2340ac200 con 0x7fc234155530 2024-03-06T07:24:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.919+0000 7fc21bfff640 1 -- 172.21.15.26:0/3304968189 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc22c058800 con 0x7fc234155530 2024-03-06T07:24:40.921 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:24:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.920+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc22006b4d0 msgr2=0x7fc22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc22006b4d0 0x7fc22006d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc224036000 tx=0x7fc22403c040 comp rx=0 tx=0).stop 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc234155530 msgr2=0x7fc234148210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc234155530 0x7fc234148210 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c6b0 tx=0x7fc22c00cb70 comp rx=0 tx=0).stop 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 shutdown_connections 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc22006b4d0 0x7fc22006d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc234155530 0x7fc234148210 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2341529f0 0x7fc234147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 --2- 172.21.15.26:0/3304968189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2340c5370 0x7fc2340b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 >> 172.21.15.26:0/3304968189 conn(0x7fc2340ac580 msgr2=0x7fc2340c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 shutdown_connections 2024-03-06T07:24:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:40.921+0000 7fc23a13a640 1 -- 172.21.15.26:0/3304968189 wait complete. 2024-03-06T07:24:40.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:24:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.033+0000 7f3f03888640 1 Processor -- start 2024-03-06T07:24:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.033+0000 7f3f03888640 1 -- start start 2024-03-06T07:24:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f03888640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3efc06da40 0x7f3efc155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f03888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc0c6780 0x7f3efc158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f03888640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc156050 con 0x7f3efc1592d0 2024-03-06T07:24:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f03888640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3efc06de20 con 0x7f3efc06da40 2024-03-06T07:24:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f03888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc072630 con 0x7f3efc0c6780 2024-03-06T07:24:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f01624640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3efc06da40 0x7f3efc155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f01624640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3efc06da40 0x7f3efc155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57344/0 (socket says 172.21.15.26:57344) 2024-03-06T07:24:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f01624640 1 -- 172.21.15.26:0/2424094253 learned_addr learned my addr 172.21.15.26:0/2424094253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f00e23640 1 --2- 172.21.15.26:0/2424094253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc0c6780 0x7f3efc158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f01624640 1 -- 172.21.15.26:0/2424094253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc0c6780 msgr2=0x7f3efc158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f01624640 1 --2- 172.21.15.26:0/2424094253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc0c6780 0x7f3efc158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f01624640 1 -- 172.21.15.26:0/2424094253 >> v1:172.21.15.117:6789/0 conn(0x7f3efc1592d0 legacy=0x7f3efc0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.034+0000 7f3f01624640 1 -- 172.21.15.26:0/2424094253 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc072910 con 0x7f3efc06da40 2024-03-06T07:24:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.035+0000 7f3f01624640 1 --2- 172.21.15.26:0/2424094253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc155b10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e78f586549b7f59 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.035+0000 7f3efbfff640 1 -- 172.21.15.26:0/2424094253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40317c0 con 0x7f3efc06da40 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.035+0000 7f3efbfff640 1 -- 172.21.15.26:0/2424094253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef4037070 con 0x7f3efc06da40 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.035+0000 7f3efbfff640 1 -- 172.21.15.26:0/2424094253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4031b80 con 0x7f3efc06da40 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 -- 172.21.15.26:0/2424094253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 msgr2=0x7f3efc155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 --2- 172.21.15.26:0/2424094253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc155b10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef402b860 comp rx=0 tx=0).stop 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 -- 172.21.15.26:0/2424094253 shutdown_connections 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 --2- 172.21.15.26:0/2424094253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc0c6780 0x7f3efc158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 --2- 172.21.15.26:0/2424094253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc155b10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 -- 172.21.15.26:0/2424094253 >> 172.21.15.26:0/2424094253 conn(0x7f3efc0aa630 msgr2=0x7f3efc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 -- 172.21.15.26:0/2424094253 shutdown_connections 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.036+0000 7f3f03888640 1 -- 172.21.15.26:0/2424094253 wait complete. 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.037+0000 7f3f03888640 1 Processor -- start 2024-03-06T07:24:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.037+0000 7f3f03888640 1 -- start start 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f03888640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc06e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f03888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3efc0c6780 0x7f3efc157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f03888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc1592d0 0x7f3efc0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f03888640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0c3990 con 0x7f3efc06da40 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f03888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc158890 con 0x7f3efc0c6780 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f03888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0c3b10 con 0x7f3efc1592d0 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc06e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc06e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57352/0 (socket says 172.21.15.26:57352) 2024-03-06T07:24:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 -- 172.21.15.26:0/1101960128 learned_addr learned my addr 172.21.15.26:0/1101960128 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 -- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc1592d0 msgr2=0x7f3efc0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f00e23640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3efc0c6780 0x7f3efc157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc1592d0 0x7f3efc0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 -- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3efc0c6780 msgr2=0x7f3efc157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3efc0c6780 0x7f3efc157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 -- 172.21.15.26:0/1101960128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc14f960 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f01624640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc06e2e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3ef9ffb640 1 -- 172.21.15.26:0/1101960128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4032860 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3ef9ffb640 1 -- 172.21.15.26:0/1101960128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef4032a20 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3ef9ffb640 1 -- 172.21.15.26:0/1101960128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4032e10 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.038+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc0702c0 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.039+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc0aa1b0 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.040+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc0ad1c0 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.040+0000 7f3ef9ffb640 1 -- 172.21.15.26:0/1101960128 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ef404c060 con 0x7f3efc06da40 2024-03-06T07:24:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.040+0000 7f3ef9ffb640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ee806b4d0 0x7f3ee806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.040+0000 7f3ef9ffb640 1 -- 172.21.15.26:0/1101960128 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3ef40ab8e0 con 0x7f3efc06da40 2024-03-06T07:24:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.041+0000 7f3f00e23640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ee806b4d0 0x7f3ee806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.041+0000 7f3f00e23640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ee806b4d0 0x7f3ee806d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.045+0000 7f3ef9ffb640 1 -- 172.21.15.26:0/1101960128 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ef4040070 con 0x7f3efc06da40 2024-03-06T07:24:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.226+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3efc0700b0 con 0x7f3efc06da40 2024-03-06T07:24:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.227+0000 7f3ef9ffb640 1 -- 172.21.15.26:0/1101960128 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3efc156050 con 0x7f3efc06da40 2024-03-06T07:24:41.229 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:41.229 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:24:41.229 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:24:41.229 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:24:41.229 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:24:41.229 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:24:41.229 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:24:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ee806b4d0 msgr2=0x7f3ee806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ee806b4d0 0x7f3ee806d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0008040 comp rx=0 tx=0).stop 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 msgr2=0x7f3efc06e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc06e2e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef40046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 shutdown_connections 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ee806b4d0 0x7f3ee806d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3efc1592d0 0x7f3efc0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3efc0c6780 0x7f3efc157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 --2- 172.21.15.26:0/1101960128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc06da40 0x7f3efc06e2e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 >> 172.21.15.26:0/1101960128 conn(0x7f3efc0aa630 msgr2=0x7f3efc154140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 shutdown_connections 2024-03-06T07:24:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.229+0000 7f3f03888640 1 -- 172.21.15.26:0/1101960128 wait complete. 2024-03-06T07:24:41.241 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:24:41.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:24:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.345+0000 7f751e325640 1 Processor -- start 2024-03-06T07:24:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.345+0000 7f751e325640 1 -- start start 2024-03-06T07:24:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f751e325640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 0x7f75180c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f751e325640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75180c8720 0x7f751806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f751e325640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7518072620 con 0x7f751806f100 2024-03-06T07:24:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f751e325640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7518072900 con 0x7f75180c8720 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f751e325640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7518072be0 con 0x7f75180c7f10 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f751c8c2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f751806f100 0x7f75181567d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41340/0 (socket says 172.21.15.26:41340) 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f751c8c2640 1 -- 172.21.15.26:0/1038618156 learned_addr learned my addr 172.21.15.26:0/1038618156 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7517fff640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 0x7f75180c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f75177fe640 1 --2- 172.21.15.26:0/1038618156 >> v2:172.21.15.117:3300/0 conn(0x7f75180c8720 0x7f751806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778314170 0 0) 0x7f7518072620 con 0x7f751806f100 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7500003520 con 0x7f751806f100 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3302561000 0 0) 0x7f7500003520 con 0x7f751806f100 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 msgr2=0x7f75180c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 0x7f75180c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 >> v2:172.21.15.117:3300/0 conn(0x7f75180c8720 msgr2=0x7f751806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 --2- 172.21.15.26:0/1038618156 >> v2:172.21.15.117:3300/0 conn(0x7f75180c8720 0x7f751806ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7518072ec0 con 0x7f751806f100 2024-03-06T07:24:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f75100019f0 con 0x7f751806f100 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 >> v1:172.21.15.26:6789/0 conn(0x7f751806f100 legacy=0x7f75181567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75000039b0 0x7f7500003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.346+0000 7f7516ffd640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500004800 0x7f7500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f7516ffd640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 0x7f75000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518072900 con 0x7f7500004800 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518072be0 con 0x7f75000039b0 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750000ba40 con 0x7f75180c7f10 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f75177fe640 1 --2- 172.21.15.26:0/1038618156 >> v2:172.21.15.117:3300/0 conn(0x7f75180c8720 0x7f751806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f7517fff640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75000039b0 0x7f7500003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f751c8c2640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 0x7f75000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f75177fe640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500004800 0x7f7500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f75177fe640 1 -- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 msgr2=0x7f75000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f75177fe640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75180c7f10 0x7f75000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f75177fe640 1 -- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75000039b0 msgr2=0x7f7500003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f75177fe640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75000039b0 0x7f7500003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.347+0000 7f75177fe640 1 -- 172.21.15.26:0/1038618156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500003520 con 0x7f7500004800 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f75177fe640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500004800 0x7f7500004c80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7508009c70 tx=0x7f7500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71efc362a074239 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508009840 con 0x7f7500004800 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7508004020 con 0x7f7500004800 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f7516ffd640 1 -- 172.21.15.26:0/1038618156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508004410 con 0x7f7500004800 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f751e325640 1 -- 172.21.15.26:0/1038618156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500004800 msgr2=0x7f7500004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f751e325640 1 --2- 172.21.15.26:0/1038618156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500004800 0x7f7500004c80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7508009c70 tx=0x7f7500005830 comp rx=0 tx=0).stop 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f751c8c2640 1 -- 172.21.15.26:0/1038618156 reap_dead start 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f751e325640 1 -- 172.21.15.26:0/1038618156 shutdown_connections 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f751e325640 1 -- 172.21.15.26:0/1038618156 >> 172.21.15.26:0/1038618156 conn(0x7f75180aa5c0 msgr2=0x7f75180aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f751e325640 1 -- 172.21.15.26:0/1038618156 shutdown_connections 2024-03-06T07:24:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.348+0000 7f751e325640 1 -- 172.21.15.26:0/1038618156 wait complete. 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 Processor -- start 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 -- start start 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500004800 0x7f75181ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751806d380 0x7f7518152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7518152f40 0x7f75181edf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751806f690 con 0x7f751806d380 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751806f510 con 0x7f7500004800 2024-03-06T07:24:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.350+0000 7f751e325640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751806f810 con 0x7f7518152f40 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751806d380 0x7f7518152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751806d380 0x7f7518152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57372/0 (socket says 172.21.15.26:57372) 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 -- 172.21.15.26:0/4012271158 learned_addr learned my addr 172.21.15.26:0/4012271158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 -- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7518152f40 msgr2=0x7f75181edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7518152f40 0x7f75181edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 -- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500004800 msgr2=0x7f75181ebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500004800 0x7f75181ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 -- 172.21.15.26:0/4012271158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180bf460 con 0x7f751806d380 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f75177fe640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751806d380 0x7f7518152a00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7508009d60 tx=0x7f7508004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f7514ff9640 1 -- 172.21.15.26:0/4012271158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508043020 con 0x7f751806d380 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f7514ff9640 1 -- 172.21.15.26:0/4012271158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750802cb70 con 0x7f751806d380 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75181ec130 con 0x7f751806d380 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f7514ff9640 1 -- 172.21.15.26:0/4012271158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508003610 con 0x7f751806d380 2024-03-06T07:24:41.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.351+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75180bfa30 con 0x7f751806d380 2024-03-06T07:24:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.352+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751804f330 con 0x7f751806d380 2024-03-06T07:24:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.353+0000 7f7514ff9640 1 -- 172.21.15.26:0/4012271158 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7508046020 con 0x7f751806d380 2024-03-06T07:24:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.353+0000 7f7514ff9640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f750406b4d0 0x7f750406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.353+0000 7f7514ff9640 1 -- 172.21.15.26:0/4012271158 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f75080ac080 con 0x7f751806d380 2024-03-06T07:24:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.353+0000 7f7517fff640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f750406b4d0 0x7f750406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.354+0000 7f7517fff640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f750406b4d0 0x7f750406d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f750c009960 tx=0x7f750c006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.357+0000 7f7514ff9640 1 -- 172.21.15.26:0/4012271158 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f750804d030 con 0x7f751806d380 2024-03-06T07:24:41.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.539+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f75180abcd0 con 0x7f751806d380 2024-03-06T07:24:41.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.912+0000 7f7514ff9640 1 -- 172.21.15.26:0/4012271158 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) v1 ==== 138+0+0 (secure 0 0 0) 0x7f750807b270 con 0x7f751806d380 2024-03-06T07:24:41.914 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:24:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f750406b4d0 msgr2=0x7f750406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f750406b4d0 0x7f750406d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f750c009960 tx=0x7f750c006bd0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751806d380 msgr2=0x7f7518152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751806d380 0x7f7518152a00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7508009d60 tx=0x7f7508004730 comp rx=0 tx=0).stop 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 shutdown_connections 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f750406b4d0 0x7f750406d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7518152f40 0x7f75181edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751806d380 0x7f7518152a00 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 --2- 172.21.15.26:0/4012271158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500004800 0x7f75181ebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 >> 172.21.15.26:0/4012271158 conn(0x7f75180aa5c0 msgr2=0x7f75180a7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 shutdown_connections 2024-03-06T07:24:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:41.915+0000 7f751e325640 1 -- 172.21.15.26:0/4012271158 wait complete. 2024-03-06T07:24:41.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:24:42.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:42.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:42.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.040+0000 7f6d9fcdf640 1 Processor -- start 2024-03-06T07:24:42.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9fcdf640 1 -- start start 2024-03-06T07:24:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9fcdf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980c0bc0 0x7f6d980c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9fcdf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d980a6cc0 0x7f6d9806e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9fcdf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d98155270 con 0x7f6d980c5690 2024-03-06T07:24:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9fcdf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d98155e80 con 0x7f6d980a6cc0 2024-03-06T07:24:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9fcdf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d98155550 con 0x7f6d980c0bc0 2024-03-06T07:24:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9e27c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d980a6cc0 0x7f6d9806e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9e27c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d980a6cc0 0x7f6d9806e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57384/0 (socket says 172.21.15.26:57384) 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.041+0000 7f6d9e27c640 1 -- 172.21.15.26:0/1301997542 learned_addr learned my addr 172.21.15.26:0/1301997542 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.042+0000 7f6d9da7b640 1 --2- 172.21.15.26:0/1301997542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980c0bc0 0x7f6d980c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.042+0000 7f6d9e27c640 1 -- 172.21.15.26:0/1301997542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980c0bc0 msgr2=0x7f6d980c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.042+0000 7f6d9e27c640 1 --2- 172.21.15.26:0/1301997542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980c0bc0 0x7f6d980c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.042+0000 7f6d9e27c640 1 -- 172.21.15.26:0/1301997542 >> v1:172.21.15.117:6789/0 conn(0x7f6d980c5690 legacy=0x7f6d9806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.042+0000 7f6d9e27c640 1 -- 172.21.15.26:0/1301997542 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98155830 con 0x7f6d980a6cc0 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.042+0000 7f6d9da7b640 1 --2- 172.21.15.26:0/1301997542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980c0bc0 0x7f6d980c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.042+0000 7f6d9e27c640 1 --2- 172.21.15.26:0/1301997542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980a6cc0 0x7f6d9806e280 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009e50 tx=0x7f6d9002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcf78409c83d123 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9ca79640 1 -- 172.21.15.26:0/1301997542 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9002c9e0 con 0x7f6d980a6cc0 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9ca79640 1 -- 172.21.15.26:0/1301997542 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9002cb80 con 0x7f6d980a6cc0 2024-03-06T07:24:42.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9ca79640 1 -- 172.21.15.26:0/1301997542 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900343f0 con 0x7f6d980a6cc0 2024-03-06T07:24:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/1301997542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980a6cc0 msgr2=0x7f6d9806e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/1301997542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980a6cc0 0x7f6d9806e280 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009e50 tx=0x7f6d9002be40 comp rx=0 tx=0).stop 2024-03-06T07:24:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/1301997542 shutdown_connections 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/1301997542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980a6cc0 0x7f6d9806e280 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/1301997542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980c0bc0 0x7f6d980c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/1301997542 >> 172.21.15.26:0/1301997542 conn(0x7f6d980a9540 msgr2=0x7f6d980a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/1301997542 shutdown_connections 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.043+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/1301997542 wait complete. 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.044+0000 7f6d9fcdf640 1 Processor -- start 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.044+0000 7f6d9fcdf640 1 -- start start 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9fcdf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980a6cc0 0x7f6d98153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9fcdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d980c0bc0 0x7f6d98072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9fcdf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980c5690 0x7f6d98072650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9fcdf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9806f410 con 0x7f6d980c5690 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9fcdf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9806e7c0 con 0x7f6d980c0bc0 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9fcdf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d98152ab0 con 0x7f6d980a6cc0 2024-03-06T07:24:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9e27c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980c5690 0x7f6d98072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9e27c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980c5690 0x7f6d98072650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57398/0 (socket says 172.21.15.26:57398) 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9e27c640 1 -- 172.21.15.26:0/997390162 learned_addr learned my addr 172.21.15.26:0/997390162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9da7b640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980a6cc0 0x7f6d98153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.045+0000 7f6d9d27a640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d980c0bc0 0x7f6d98072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9e27c640 1 -- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980a6cc0 msgr2=0x7f6d98153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9e27c640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980a6cc0 0x7f6d98153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9e27c640 1 -- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d980c0bc0 msgr2=0x7f6d98072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9e27c640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d980c0bc0 0x7f6d98072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9e27c640 1 -- 172.21.15.26:0/997390162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98153540 con 0x7f6d980c5690 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9e27c640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980c5690 0x7f6d98072650 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6d90036040 tx=0x7f6d90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d8e7fc640 1 -- 172.21.15.26:0/997390162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90043020 con 0x7f6d980c5690 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d8e7fc640 1 -- 172.21.15.26:0/997390162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9003e450 con 0x7f6d980c5690 2024-03-06T07:24:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d980a84a0 con 0x7f6d980c5690 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d8e7fc640 1 -- 172.21.15.26:0/997390162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90009510 con 0x7f6d980c5690 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.046+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d980a8a60 con 0x7f6d980c5690 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.047+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9804f330 con 0x7f6d980c5690 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.048+0000 7f6d8e7fc640 1 -- 172.21.15.26:0/997390162 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d90046020 con 0x7f6d980c5690 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.048+0000 7f6d8e7fc640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d7c06b500 0x7f6d7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.048+0000 7f6d8e7fc640 1 -- 172.21.15.26:0/997390162 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f6d900acb70 con 0x7f6d980c5690 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.048+0000 7f6d9da7b640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d7c06b500 0x7f6d7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.049+0000 7f6d9da7b640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d7c06b500 0x7f6d7c06d9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6d98150920 tx=0x7f6d88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.052+0000 7f6d8e7fc640 1 -- 172.21.15.26:0/997390162 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6d9004d030 con 0x7f6d980c5690 2024-03-06T07:24:42.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.232+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6d9806f180 con 0x7f6d980c5690 2024-03-06T07:24:42.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.915+0000 7f6d8e7fc640 1 -- 172.21.15.26:0/997390162 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6d9007b4d0 con 0x7f6d980c5690 2024-03-06T07:24:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:24:42.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d7c06b500 msgr2=0x7f6d7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d7c06b500 0x7f6d7c06d9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6d98150920 tx=0x7f6d88008040 comp rx=0 tx=0).stop 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980c5690 msgr2=0x7f6d98072650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980c5690 0x7f6d98072650 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6d90036040 tx=0x7f6d90004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 shutdown_connections 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d7c06b500 0x7f6d7c06d9c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d980c5690 0x7f6d98072650 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d980c0bc0 0x7f6d98072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 --2- 172.21.15.26:0/997390162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d980a6cc0 0x7f6d98153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 >> 172.21.15.26:0/997390162 conn(0x7f6d980a9540 msgr2=0x7f6d9806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 shutdown_connections 2024-03-06T07:24:42.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:42.918+0000 7f6d9fcdf640 1 -- 172.21.15.26:0/997390162 wait complete. 2024-03-06T07:24:42.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:24:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.056+0000 7f351c84e640 1 Processor -- start 2024-03-06T07:24:43.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.056+0000 7f351c84e640 1 -- start start 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.056+0000 7f351c84e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f351806d9f0 0x7f35180a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.056+0000 7f351c84e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35180c3280 0x7f35180c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.056+0000 7f351c84e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3518158800 con 0x7f35181551e0 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.056+0000 7f351c84e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3518158ae0 con 0x7f35180c3280 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.056+0000 7f351c84e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3518158dc0 con 0x7f351806d9f0 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f350ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35180c3280 0x7f35180c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f3516d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f35181551e0 0x7f35181555f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41356/0 (socket says 172.21.15.26:41356) 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f3516d9d640 1 -- 172.21.15.26:0/2928695423 learned_addr learned my addr 172.21.15.26:0/2928695423 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f351659c640 1 --2- 172.21.15.26:0/2928695423 >> v2:172.21.15.117:3300/0 conn(0x7f351806d9f0 0x7f35180a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f350ffff640 1 -- 172.21.15.26:0/2928695423 >> v2:172.21.15.117:3300/0 conn(0x7f351806d9f0 msgr2=0x7f35180a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:43.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f350ffff640 1 --2- 172.21.15.26:0/2928695423 >> v2:172.21.15.117:3300/0 conn(0x7f351806d9f0 0x7f35180a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f350ffff640 1 -- 172.21.15.26:0/2928695423 >> v1:172.21.15.26:6789/0 conn(0x7f35181551e0 legacy=0x7f35181555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.057+0000 7f350ffff640 1 -- 172.21.15.26:0/2928695423 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35181590a0 con 0x7f35180c3280 2024-03-06T07:24:43.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f350ffff640 1 --2- 172.21.15.26:0/2928695423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35180c3280 0x7f35180c3710 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f35180c9820 tx=0x7f350002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8920a1a156b2ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:43.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f3515d9b640 1 -- 172.21.15.26:0/2928695423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000315a0 con 0x7f35180c3280 2024-03-06T07:24:43.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f3515d9b640 1 -- 172.21.15.26:0/2928695423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500037070 con 0x7f35180c3280 2024-03-06T07:24:43.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f3515d9b640 1 -- 172.21.15.26:0/2928695423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500031960 con 0x7f35180c3280 2024-03-06T07:24:43.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 -- 172.21.15.26:0/2928695423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35180c3280 msgr2=0x7f35180c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 --2- 172.21.15.26:0/2928695423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35180c3280 0x7f35180c3710 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f35180c9820 tx=0x7f350002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 -- 172.21.15.26:0/2928695423 shutdown_connections 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 --2- 172.21.15.26:0/2928695423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35180c3280 0x7f35180c3710 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 --2- 172.21.15.26:0/2928695423 >> v2:172.21.15.117:3300/0 conn(0x7f351806d9f0 0x7f35180a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 -- 172.21.15.26:0/2928695423 >> 172.21.15.26:0/2928695423 conn(0x7f35180aa630 msgr2=0x7f35180aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 -- 172.21.15.26:0/2928695423 shutdown_connections 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.058+0000 7f351c84e640 1 -- 172.21.15.26:0/2928695423 wait complete. 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 Processor -- start 2024-03-06T07:24:43.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 -- start start 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f351806d9f0 0x7f35180a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35180c3280 0x7f351814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35181551e0 0x7f351814db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351806df50 con 0x7f351806d9f0 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351806ddd0 con 0x7f35180c3280 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351c84e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35180a8210 con 0x7f35181551e0 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f351806d9f0 0x7f35180a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.060+0000 7f351659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f351806d9f0 0x7f35180a9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57418/0 (socket says 172.21.15.26:57418) 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351659c640 1 -- 172.21.15.26:0/324229032 learned_addr learned my addr 172.21.15.26:0/324229032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f3516d9d640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35181551e0 0x7f351814db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f350ffff640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35180c3280 0x7f351814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351659c640 1 -- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35181551e0 msgr2=0x7f351814db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351659c640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35181551e0 0x7f351814db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351659c640 1 -- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35180c3280 msgr2=0x7f351814d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351659c640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35180c3280 0x7f351814d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351659c640 1 -- 172.21.15.26:0/324229032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180ae5a0 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351659c640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f351806d9f0 0x7f35180a9590 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f350800c7f0 tx=0x7f350800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f350effd640 1 -- 172.21.15.26:0/324229032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508015070 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f350effd640 1 -- 172.21.15.26:0/324229032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508004240 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35181ebcf0 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f350effd640 1 -- 172.21.15.26:0/324229032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508007620 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.061+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3518154340 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.062+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35180c73f0 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.063+0000 7f350effd640 1 -- 172.21.15.26:0/324229032 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35080079d0 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.063+0000 7f350effd640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f34f806b4d0 0x7f34f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.063+0000 7f350effd640 1 -- 172.21.15.26:0/324229032 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f350808a3a0 con 0x7f351806d9f0 2024-03-06T07:24:43.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.063+0000 7f350ffff640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f34f806b4d0 0x7f34f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.064+0000 7f350ffff640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f34f806b4d0 0x7f34f806d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3500036000 tx=0x7f350003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:43.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.067+0000 7f350effd640 1 -- 172.21.15.26:0/324229032 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35180c73f0 con 0x7f351806d9f0 2024-03-06T07:24:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.286+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3518073240 con 0x7f351806d9f0 2024-03-06T07:24:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 19 and data pool 20 2024-03-06T07:24:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.932+0000 7f350effd640 1 -- 172.21.15.26:0/324229032 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v102) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3508059180 con 0x7f351806d9f0 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.940+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f34f806b4d0 msgr2=0x7f34f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.940+0000 7f351c84e640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f34f806b4d0 0x7f34f806d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3500036000 tx=0x7f350003d040 comp rx=0 tx=0).stop 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.940+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f351806d9f0 msgr2=0x7f35180a9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.940+0000 7f351c84e640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f351806d9f0 0x7f35180a9590 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f350800c7f0 tx=0x7f350800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.943+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 shutdown_connections 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.943+0000 7f351c84e640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f34f806b4d0 0x7f34f806d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.943+0000 7f351c84e640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35181551e0 0x7f351814db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.943+0000 7f351c84e640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35180c3280 0x7f351814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.943+0000 7f351c84e640 1 --2- 172.21.15.26:0/324229032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f351806d9f0 0x7f35180a9590 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.943+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 >> 172.21.15.26:0/324229032 conn(0x7f35180aa630 msgr2=0x7f351806ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:43.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.944+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 shutdown_connections 2024-03-06T07:24:43.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:43.945+0000 7f351c84e640 1 -- 172.21.15.26:0/324229032 wait complete. 2024-03-06T07:24:43.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:44.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.073+0000 7f668ddf7640 1 Processor -- start 2024-03-06T07:24:44.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.073+0000 7f668ddf7640 1 -- start start 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f668ddf7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6688150580 0x7f668806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f668ddf7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6688153a60 con 0x7f66880c6000 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f668ddf7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6688153d40 con 0x7f66880c1320 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f668ddf7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668806c2c0 con 0x7f6688150580 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f6687fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f66880c6000 0x7f6688156dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41360/0 (socket says 172.21.15.26:41360) 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f6687fff640 1 -- 172.21.15.26:0/3986998196 learned_addr learned my addr 172.21.15.26:0/3986998196 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f66877fe640 1 --2- 172.21.15.26:0/3986998196 >> v2:172.21.15.117:3300/0 conn(0x7f6688150580 0x7f668806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f66867fc640 1 -- 172.21.15.26:0/3986998196 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615994111 0 0) 0x7f6688153a60 con 0x7f66880c6000 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f66867fc640 1 -- 172.21.15.26:0/3986998196 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6670003520 con 0x7f66880c6000 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.074+0000 7f66867fc640 1 -- 172.21.15.26:0/3986998196 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628014140 0 0) 0x7f6688153d40 con 0x7f66880c1320 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.075+0000 7f66867fc640 1 -- 172.21.15.26:0/3986998196 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6688153a60 con 0x7f66880c1320 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.075+0000 7f66877fe640 1 -- 172.21.15.26:0/3986998196 >> v1:172.21.15.117:6789/0 conn(0x7f66880c1320 legacy=0x7f66881566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.075+0000 7f66877fe640 1 -- 172.21.15.26:0/3986998196 >> v1:172.21.15.26:6789/0 conn(0x7f66880c6000 legacy=0x7f6688156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.075+0000 7f66877fe640 1 -- 172.21.15.26:0/3986998196 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668806c5a0 con 0x7f6688150580 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.075+0000 7f66877fe640 1 --2- 172.21.15.26:0/3986998196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688150580 0x7f668806dd10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f66780097a0 tx=0x7f667802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae85888647790a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f66867fc640 1 -- 172.21.15.26:0/3986998196 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678035020 con 0x7f6688150580 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f66867fc640 1 -- 172.21.15.26:0/3986998196 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6678031830 con 0x7f6688150580 2024-03-06T07:24:44.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f66867fc640 1 -- 172.21.15.26:0/3986998196 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678031be0 con 0x7f6688150580 2024-03-06T07:24:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f668ddf7640 1 -- 172.21.15.26:0/3986998196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688150580 msgr2=0x7f668806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3986998196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688150580 0x7f668806dd10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f66780097a0 tx=0x7f667802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f668ddf7640 1 -- 172.21.15.26:0/3986998196 shutdown_connections 2024-03-06T07:24:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3986998196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688150580 0x7f668806dd10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f668ddf7640 1 -- 172.21.15.26:0/3986998196 >> 172.21.15.26:0/3986998196 conn(0x7f66880c6710 msgr2=0x7f66880c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f668ddf7640 1 -- 172.21.15.26:0/3986998196 shutdown_connections 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.076+0000 7f668ddf7640 1 -- 172.21.15.26:0/3986998196 wait complete. 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.077+0000 7f668ddf7640 1 Processor -- start 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f668ddf7640 1 -- start start 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f668ddf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880c1320 0x7f6688148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f668ddf7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c6000 0x7f66881ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f668ddf7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6688150580 0x7f66881ece00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f668ddf7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c0dc0 con 0x7f66880c1320 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f668ddf7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c0c40 con 0x7f66880c6000 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f668ddf7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668806e250 con 0x7f6688150580 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f66877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880c1320 0x7f6688148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f66877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880c1320 0x7f6688148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57430/0 (socket says 172.21.15.26:57430) 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.078+0000 7f66877fe640 1 -- 172.21.15.26:0/3695500963 learned_addr learned my addr 172.21.15.26:0/3695500963 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f6687fff640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6688150580 0x7f66881ece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f6686ffd640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c6000 0x7f66881ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f66877fe640 1 -- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6688150580 msgr2=0x7f66881ece00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f66877fe640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6688150580 0x7f66881ece00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f66877fe640 1 -- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c6000 msgr2=0x7f66881ec8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f66877fe640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c6000 0x7f66881ec8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f66877fe640 1 -- 172.21.15.26:0/3695500963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668814d2d0 con 0x7f66880c1320 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f6686ffd640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c6000 0x7f66881ec8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f6687fff640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6688150580 0x7f66881ece00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f66877fe640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880c1320 0x7f6688148fa0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f66780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f666ffff640 1 -- 172.21.15.26:0/3695500963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678035050 con 0x7f66880c1320 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f666ffff640 1 -- 172.21.15.26:0/3695500963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667803e070 con 0x7f66880c1320 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668814d5b0 con 0x7f66880c1320 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f666ffff640 1 -- 172.21.15.26:0/3695500963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678003610 con 0x7f66880c1320 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.079+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6688072ab0 con 0x7f66880c1320 2024-03-06T07:24:44.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.080+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668804f330 con 0x7f66880c1320 2024-03-06T07:24:44.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.081+0000 7f666ffff640 1 -- 172.21.15.26:0/3695500963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6678038020 con 0x7f66880c1320 2024-03-06T07:24:44.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.081+0000 7f666ffff640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b500 0x7f667406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.081+0000 7f666ffff640 1 -- 172.21.15.26:0/3695500963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f66780aba10 con 0x7f66880c1320 2024-03-06T07:24:44.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.081+0000 7f6686ffd640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b500 0x7f667406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.082+0000 7f6686ffd640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b500 0x7f667406d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6688072a60 tx=0x7f667c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.085+0000 7f666ffff640 1 -- 172.21.15.26:0/3695500963 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f667807a710 con 0x7f66880c1320 2024-03-06T07:24:44.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.266+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66880c5de0 con 0x7f66880c1320 2024-03-06T07:24:44.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.267+0000 7f666ffff640 1 -- 172.21.15.26:0/3695500963 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16925 (secure 0 0 0) 0x7f667807a010 con 0x7f66880c1320 2024-03-06T07:24:44.269 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:44.269 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":120,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:43.287395+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:41.558889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:42.251998+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b500 msgr2=0x7f667406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b500 0x7f667406d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6688072a60 tx=0x7f667c009040 comp rx=0 tx=0).stop 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880c1320 msgr2=0x7f6688148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880c1320 0x7f6688148fa0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f66780046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 shutdown_connections 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b500 0x7f667406d9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6688150580 0x7f66881ece00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c6000 0x7f66881ec8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 --2- 172.21.15.26:0/3695500963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880c1320 0x7f6688148fa0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 >> 172.21.15.26:0/3695500963 conn(0x7f66880c6710 msgr2=0x7f66880c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 shutdown_connections 2024-03-06T07:24:44.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.269+0000 7f668ddf7640 1 -- 172.21.15.26:0/3695500963 wait complete. 2024-03-06T07:24:44.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:44.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.381+0000 7fed23eaf640 1 Processor -- start 2024-03-06T07:24:44.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.381+0000 7fed23eaf640 1 -- start start 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed23eaf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed1c155ba0 0x7fed1c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed23eaf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c1586f0 0x7fed1c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed23eaf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1c1591c0 con 0x7fed1c0c5480 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed23eaf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1c06df40 con 0x7fed1c155ba0 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed23eaf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c06fdb0 con 0x7fed1c1586f0 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed2144a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed1c155ba0 0x7fed1c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed2144a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed1c155ba0 0x7fed1c0c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57446/0 (socket says 172.21.15.26:57446) 2024-03-06T07:24:44.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed2144a640 1 -- 172.21.15.26:0/500695974 learned_addr learned my addr 172.21.15.26:0/500695974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:44.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.382+0000 7fed2244c640 1 --2- 172.21.15.26:0/500695974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c1586f0 0x7fed1c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.383+0000 7fed2144a640 1 -- 172.21.15.26:0/500695974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c1586f0 msgr2=0x7fed1c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.383+0000 7fed2144a640 1 --2- 172.21.15.26:0/500695974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c1586f0 0x7fed1c158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.383+0000 7fed2144a640 1 -- 172.21.15.26:0/500695974 >> v1:172.21.15.117:6789/0 conn(0x7fed1c0c5480 legacy=0x7fed1c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.383+0000 7fed2144a640 1 -- 172.21.15.26:0/500695974 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c1594a0 con 0x7fed1c155ba0 2024-03-06T07:24:44.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.383+0000 7fed2144a640 1 --2- 172.21.15.26:0/500695974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c155ba0 0x7fed1c0c5860 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fed10009870 tx=0x7fed1002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72836fa35c20ad78 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.383+0000 7fed20c49640 1 -- 172.21.15.26:0/500695974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10035020 con 0x7fed1c155ba0 2024-03-06T07:24:44.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed20c49640 1 -- 172.21.15.26:0/500695974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10031640 con 0x7fed1c155ba0 2024-03-06T07:24:44.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed20c49640 1 -- 172.21.15.26:0/500695974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1002d040 con 0x7fed1c155ba0 2024-03-06T07:24:44.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 -- 172.21.15.26:0/500695974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c155ba0 msgr2=0x7fed1c0c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 --2- 172.21.15.26:0/500695974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c155ba0 0x7fed1c0c5860 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fed10009870 tx=0x7fed1002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 -- 172.21.15.26:0/500695974 shutdown_connections 2024-03-06T07:24:44.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 --2- 172.21.15.26:0/500695974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c1586f0 0x7fed1c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 --2- 172.21.15.26:0/500695974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c155ba0 0x7fed1c0c5860 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 -- 172.21.15.26:0/500695974 >> 172.21.15.26:0/500695974 conn(0x7fed1c06b1a0 msgr2=0x7fed1c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 -- 172.21.15.26:0/500695974 shutdown_connections 2024-03-06T07:24:44.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.384+0000 7fed23eaf640 1 -- 172.21.15.26:0/500695974 wait complete. 2024-03-06T07:24:44.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.385+0000 7fed23eaf640 1 Processor -- start 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.385+0000 7fed23eaf640 1 -- start start 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed23eaf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c0c5480 0x7fed1c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed23eaf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed1c155ba0 0x7fed1c148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed23eaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c1586f0 0x7fed1c149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed23eaf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c155680 con 0x7fed1c1586f0 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed23eaf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c156000 con 0x7fed1c155ba0 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed23eaf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c152a20 con 0x7fed1c0c5480 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c1586f0 0x7fed1c149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c1586f0 0x7fed1c149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57448/0 (socket says 172.21.15.26:57448) 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 -- 172.21.15.26:0/2625071240 learned_addr learned my addr 172.21.15.26:0/2625071240 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2144a640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed1c155ba0 0x7fed1c148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed21c4b640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c0c5480 0x7fed1c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 -- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c0c5480 msgr2=0x7fed1c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c0c5480 0x7fed1c156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 -- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed1c155ba0 msgr2=0x7fed1c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed1c155ba0 0x7fed1c148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 -- 172.21.15.26:0/2625071240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c14e2e0 con 0x7fed1c1586f0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2144a640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed1c155ba0 0x7fed1c148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed21c4b640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c0c5480 0x7fed1c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.386+0000 7fed2244c640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c1586f0 0x7fed1c149030 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00ec40 tx=0x7fed0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.387+0000 7fed1a7fc640 1 -- 172.21.15.26:0/2625071240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c018020 con 0x7fed1c1586f0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.387+0000 7fed1a7fc640 1 -- 172.21.15.26:0/2625071240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c00ee50 con 0x7fed1c1586f0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.387+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1c14e5c0 con 0x7fed1c1586f0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.387+0000 7fed1a7fc640 1 -- 172.21.15.26:0/2625071240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0146c0 con 0x7fed1c1586f0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.387+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1c06cbf0 con 0x7fed1c1586f0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.388+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1c04f330 con 0x7fed1c1586f0 2024-03-06T07:24:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.388+0000 7fed1a7fc640 1 -- 172.21.15.26:0/2625071240 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed0c005d10 con 0x7fed1c1586f0 2024-03-06T07:24:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.389+0000 7fed1a7fc640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.389+0000 7fed1a7fc640 1 -- 172.21.15.26:0/2625071240 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed0c08ad20 con 0x7fed1c1586f0 2024-03-06T07:24:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.389+0000 7fed21c4b640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.389+0000 7fed21c4b640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0c92c0 tx=0x7fed14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.393+0000 7fed1a7fc640 1 -- 172.21.15.26:0/2625071240 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed0c022070 con 0x7fed1c1586f0 2024-03-06T07:24:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.574+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed1c14e790 con 0x7fed1c1586f0 2024-03-06T07:24:44.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.575+0000 7fed1a7fc640 1 -- 172.21.15.26:0/2625071240 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16925 (secure 0 0 0) 0x7fed0c010040 con 0x7fed1c1586f0 2024-03-06T07:24:44.577 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:44.577 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":120,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:43.287395+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:41.558889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:42.251998+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:44.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 msgr2=0x7fed0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0c92c0 tx=0x7fed14009040 comp rx=0 tx=0).stop 2024-03-06T07:24:44.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c1586f0 msgr2=0x7fed1c149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c1586f0 0x7fed1c149030 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00ec40 tx=0x7fed0c00c490 comp rx=0 tx=0).stop 2024-03-06T07:24:44.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 shutdown_connections 2024-03-06T07:24:44.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed0806b4d0 0x7fed0806d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1c1586f0 0x7fed1c149030 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed1c155ba0 0x7fed1c148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 --2- 172.21.15.26:0/2625071240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1c0c5480 0x7fed1c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.577+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 >> 172.21.15.26:0/2625071240 conn(0x7fed1c06b1a0 msgr2=0x7fed1c0707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.578+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 shutdown_connections 2024-03-06T07:24:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.578+0000 7fed23eaf640 1 -- 172.21.15.26:0/2625071240 wait complete. 2024-03-06T07:24:44.590 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:24:44.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.693+0000 7f3ae37e2640 1 Processor -- start 2024-03-06T07:24:44.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.693+0000 7f3ae37e2640 1 -- start start 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae37e2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3adc06dca0 0x7f3adc153420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae37e2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3adc155520 0x7f3adc155950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae37e2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc0c2d80 con 0x7f3adc14f580 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae37e2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc0c2b30 con 0x7f3adc155520 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae37e2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc0c3d70 con 0x7f3adc06dca0 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae1d7f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3adc155520 0x7f3adc155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae1d7f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3adc155520 0x7f3adc155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57458/0 (socket says 172.21.15.26:57458) 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae1d7f640 1 -- 172.21.15.26:0/1815954082 learned_addr learned my addr 172.21.15.26:0/1815954082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae0d7d640 1 --2- 172.21.15.26:0/1815954082 >> v2:172.21.15.117:3300/0 conn(0x7f3adc06dca0 0x7f3adc153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae1d7f640 1 -- 172.21.15.26:0/1815954082 >> v2:172.21.15.117:3300/0 conn(0x7f3adc06dca0 msgr2=0x7f3adc153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae1d7f640 1 --2- 172.21.15.26:0/1815954082 >> v2:172.21.15.117:3300/0 conn(0x7f3adc06dca0 0x7f3adc153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae1d7f640 1 -- 172.21.15.26:0/1815954082 >> v1:172.21.15.117:6789/0 conn(0x7f3adc14f580 legacy=0x7f3adc06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae1d7f640 1 -- 172.21.15.26:0/1815954082 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc0c3a00 con 0x7f3adc155520 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.694+0000 7f3ae0d7d640 1 --2- 172.21.15.26:0/1815954082 >> v2:172.21.15.117:3300/0 conn(0x7f3adc06dca0 0x7f3adc153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.695+0000 7f3ae1d7f640 1 --2- 172.21.15.26:0/1815954082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc155520 0x7f3adc155950 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3adc06ed20 tx=0x7f3ad402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b913e7a3af4cac71 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.695+0000 7f3ad3fff640 1 -- 172.21.15.26:0/1815954082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad402c960 con 0x7f3adc155520 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.695+0000 7f3ad3fff640 1 -- 172.21.15.26:0/1815954082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad402cb00 con 0x7f3adc155520 2024-03-06T07:24:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.695+0000 7f3ad3fff640 1 -- 172.21.15.26:0/1815954082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40343f0 con 0x7f3adc155520 2024-03-06T07:24:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 -- 172.21.15.26:0/1815954082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc155520 msgr2=0x7f3adc155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/1815954082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc155520 0x7f3adc155950 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3adc06ed20 tx=0x7f3ad402bdc0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 -- 172.21.15.26:0/1815954082 shutdown_connections 2024-03-06T07:24:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/1815954082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc155520 0x7f3adc155950 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/1815954082 >> v2:172.21.15.117:3300/0 conn(0x7f3adc06dca0 0x7f3adc153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 -- 172.21.15.26:0/1815954082 >> 172.21.15.26:0/1815954082 conn(0x7f3adc0a94d0 msgr2=0x7f3adc0a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 -- 172.21.15.26:0/1815954082 shutdown_connections 2024-03-06T07:24:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.696+0000 7f3ae37e2640 1 -- 172.21.15.26:0/1815954082 wait complete. 2024-03-06T07:24:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.698+0000 7f3ae37e2640 1 Processor -- start 2024-03-06T07:24:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.698+0000 7f3ae37e2640 1 -- start start 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae37e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc06dca0 0x7f3adc1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae37e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3adc14f580 0x7f3adc06b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae37e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3adc155520 0x7f3adc14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae37e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc0c3060 con 0x7f3adc06dca0 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae37e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc14f960 con 0x7f3adc155520 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae37e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc155e90 con 0x7f3adc14f580 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc06dca0 0x7f3adc1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc06dca0 0x7f3adc1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57470/0 (socket says 172.21.15.26:57470) 2024-03-06T07:24:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 -- 172.21.15.26:0/3623586298 learned_addr learned my addr 172.21.15.26:0/3623586298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae1d7f640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3adc155520 0x7f3adc14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae0d7d640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3adc14f580 0x7f3adc06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 -- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3adc14f580 msgr2=0x7f3adc06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3adc14f580 0x7f3adc06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 -- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3adc155520 msgr2=0x7f3adc14feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3adc155520 0x7f3adc14feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae157e640 1 -- 172.21.15.26:0/3623586298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc1e8880 con 0x7f3adc06dca0 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae0d7d640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3adc14f580 0x7f3adc06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.699+0000 7f3ae1d7f640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3adc155520 0x7f3adc14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.700+0000 7f3ae157e640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc06dca0 0x7f3adc1525c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c430 tx=0x7f3acc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.700+0000 7f3ad1ffb640 1 -- 172.21.15.26:0/3623586298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc011790 con 0x7f3adc06dca0 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.700+0000 7f3ad1ffb640 1 -- 172.21.15.26:0/3623586298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc017070 con 0x7f3adc06dca0 2024-03-06T07:24:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.700+0000 7f3ad1ffb640 1 -- 172.21.15.26:0/3623586298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc011bb0 con 0x7f3adc06dca0 2024-03-06T07:24:44.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.700+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc1e8b60 con 0x7f3adc06dca0 2024-03-06T07:24:44.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.700+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc1e9120 con 0x7f3adc06dca0 2024-03-06T07:24:44.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.701+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc04f330 con 0x7f3adc06dca0 2024-03-06T07:24:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.701+0000 7f3ad1ffb640 1 -- 172.21.15.26:0/3623586298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3acc01c460 con 0x7f3adc06dca0 2024-03-06T07:24:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.702+0000 7f3ad1ffb640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ac006b4d0 0x7f3ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.702+0000 7f3ad1ffb640 1 -- 172.21.15.26:0/3623586298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3acc08a1e0 con 0x7f3adc06dca0 2024-03-06T07:24:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.702+0000 7f3ae0d7d640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ac006b4d0 0x7f3ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.703+0000 7f3ae0d7d640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ac006b4d0 0x7f3ac006d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3adc06bc10 tx=0x7f3ad8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.706+0000 7f3ad1ffb640 1 -- 172.21.15.26:0/3623586298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3acc058f60 con 0x7f3adc06dca0 2024-03-06T07:24:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:44.928+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3adc0c3740 con 0x7f3adc06dca0 2024-03-06T07:24:45.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.626+0000 7f3ad1ffb640 1 -- 172.21.15.26:0/3623586298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3acc058860 con 0x7f3adc06dca0 2024-03-06T07:24:45.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.628+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ac006b4d0 msgr2=0x7f3ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.628+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ac006b4d0 0x7f3ac006d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3adc06bc10 tx=0x7f3ad8008040 comp rx=0 tx=0).stop 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc06dca0 msgr2=0x7f3adc1525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc06dca0 0x7f3adc1525c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c430 tx=0x7f3acc00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 shutdown_connections 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3ac006b4d0 0x7f3ac006d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3adc155520 0x7f3adc14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3adc14f580 0x7f3adc06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 --2- 172.21.15.26:0/3623586298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3adc06dca0 0x7f3adc1525c0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 >> 172.21.15.26:0/3623586298 conn(0x7f3adc0a94d0 msgr2=0x7f3adc0a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 shutdown_connections 2024-03-06T07:24:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.629+0000 7f3ae37e2640 1 -- 172.21.15.26:0/3623586298 wait complete. 2024-03-06T07:24:45.641 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:24:45.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:24:45.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.743+0000 7fad5d639640 1 Processor -- start 2024-03-06T07:24:45.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad5d639640 1 -- start start 2024-03-06T07:24:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad5d639640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad580ab170 0x7fad580c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad5d639640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad580c3ef0 con 0x7fad580c5990 2024-03-06T07:24:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad5d639640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad581556d0 con 0x7fad5806c900 2024-03-06T07:24:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad5d639640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad581559b0 con 0x7fad580ab170 2024-03-06T07:24:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad577fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad580ab170 0x7fad580c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad56ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fad580c5990 0x7fad5806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41374/0 (socket says 172.21.15.26:41374) 2024-03-06T07:24:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.744+0000 7fad56ffd640 1 -- 172.21.15.26:0/2000270848 learned_addr learned my addr 172.21.15.26:0/2000270848 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.745+0000 7fad577fe640 1 -- 172.21.15.26:0/2000270848 >> v1:172.21.15.117:6789/0 conn(0x7fad5806c900 legacy=0x7fad580a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.745+0000 7fad577fe640 1 -- 172.21.15.26:0/2000270848 >> v1:172.21.15.26:6789/0 conn(0x7fad580c5990 legacy=0x7fad5806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.745+0000 7fad55ffb640 1 -- 172.21.15.26:0/2000270848 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065420102 0 0) 0x7fad580c3ef0 con 0x7fad580c5990 2024-03-06T07:24:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.745+0000 7fad577fe640 1 -- 172.21.15.26:0/2000270848 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58155c90 con 0x7fad580ab170 2024-03-06T07:24:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.745+0000 7fad577fe640 1 --2- 172.21.15.26:0/2000270848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad580c0ab0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fad50009dc0 tx=0x7fad5002bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdaee4b7d3d5ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad55ffb640 1 -- 172.21.15.26:0/2000270848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50039070 con 0x7fad580ab170 2024-03-06T07:24:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad55ffb640 1 -- 172.21.15.26:0/2000270848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5002ccd0 con 0x7fad580ab170 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad55ffb640 1 -- 172.21.15.26:0/2000270848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50034660 con 0x7fad580ab170 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad5d639640 1 -- 172.21.15.26:0/2000270848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 msgr2=0x7fad580c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad5d639640 1 --2- 172.21.15.26:0/2000270848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad580c0ab0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fad50009dc0 tx=0x7fad5002bf20 comp rx=0 tx=0).stop 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad5d639640 1 -- 172.21.15.26:0/2000270848 shutdown_connections 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad5d639640 1 --2- 172.21.15.26:0/2000270848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad580c0ab0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad5d639640 1 -- 172.21.15.26:0/2000270848 >> 172.21.15.26:0/2000270848 conn(0x7fad580c6710 msgr2=0x7fad580c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad5d639640 1 -- 172.21.15.26:0/2000270848 shutdown_connections 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.746+0000 7fad5d639640 1 -- 172.21.15.26:0/2000270848 wait complete. 2024-03-06T07:24:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.748+0000 7fad5d639640 1 Processor -- start 2024-03-06T07:24:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.748+0000 7fad5d639640 1 -- start start 2024-03-06T07:24:45.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad5d639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5806c900 0x7fad581eb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:45.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad5d639640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad581dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:45.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad5d639640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad580c5990 0x7fad58145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:45.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad5d639640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58156610 con 0x7fad580ab170 2024-03-06T07:24:45.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad5d639640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58156490 con 0x7fad580c5990 2024-03-06T07:24:45.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad5d639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580c3af0 con 0x7fad5806c900 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad567fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad581dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad567fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad581dc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57478/0 (socket says 172.21.15.26:57478) 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.749+0000 7fad567fc640 1 -- 172.21.15.26:0/625207833 learned_addr learned my addr 172.21.15.26:0/625207833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad577fe640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad580c5990 0x7fad58145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad56ffd640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5806c900 0x7fad581eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad567fc640 1 -- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5806c900 msgr2=0x7fad581eb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad567fc640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5806c900 0x7fad581eb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad567fc640 1 -- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad580c5990 msgr2=0x7fad58145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad567fc640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad580c5990 0x7fad58145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad567fc640 1 -- 172.21.15.26:0/625207833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad581dcdd0 con 0x7fad580ab170 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad567fc640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad581dc1e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e730 tx=0x7fad4800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad4f7fe640 1 -- 172.21.15.26:0/625207833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480052d0 con 0x7fad580ab170 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad4f7fe640 1 -- 172.21.15.26:0/625207833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad48005470 con 0x7fad580ab170 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad4f7fe640 1 -- 172.21.15.26:0/625207833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800cca0 con 0x7fad580ab170 2024-03-06T07:24:45.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad581549b0 con 0x7fad580ab170 2024-03-06T07:24:45.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.750+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad581dc720 con 0x7fad580ab170 2024-03-06T07:24:45.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.751+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad58067ad0 con 0x7fad580ab170 2024-03-06T07:24:45.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.752+0000 7fad4f7fe640 1 -- 172.21.15.26:0/625207833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad480035f0 con 0x7fad580ab170 2024-03-06T07:24:45.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.752+0000 7fad4f7fe640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b500 0x7fad3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:45.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.752+0000 7fad4f7fe640 1 -- 172.21.15.26:0/625207833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad4808c630 con 0x7fad580ab170 2024-03-06T07:24:45.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.752+0000 7fad56ffd640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b500 0x7fad3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:45.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.753+0000 7fad56ffd640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b500 0x7fad3806d9c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad40000f60 tx=0x7fad40007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:45.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.756+0000 7fad4f7fe640 1 -- 172.21.15.26:0/625207833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fad58067ad0 con 0x7fad580ab170 2024-03-06T07:24:45.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:45.976+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fad580abcb0 con 0x7fad580ab170 2024-03-06T07:24:46.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.955+0000 7fad4f7fe640 1 -- 172.21.15.26:0/625207833 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) v1 ==== 122+0+0 (secure 0 0 0) 0x7fad580abcb0 con 0x7fad580ab170 2024-03-06T07:24:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b500 msgr2=0x7fad3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b500 0x7fad3806d9c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad40000f60 tx=0x7fad40007d00 comp rx=0 tx=0).stop 2024-03-06T07:24:46.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 msgr2=0x7fad581dc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad581dc1e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e730 tx=0x7fad4800ebf0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 shutdown_connections 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad3806b500 0x7fad3806d9c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad580c5990 0x7fad58145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad580ab170 0x7fad581dc1e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 --2- 172.21.15.26:0/625207833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad5806c900 0x7fad581eb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 >> 172.21.15.26:0/625207833 conn(0x7fad580c6710 msgr2=0x7fad580c1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 shutdown_connections 2024-03-06T07:24:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:46.958+0000 7fad5d639640 1 -- 172.21.15.26:0/625207833 wait complete. 2024-03-06T07:24:46.970 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:24:46.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.070+0000 7f9387c39640 1 Processor -- start 2024-03-06T07:24:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.070+0000 7f9387c39640 1 -- start start 2024-03-06T07:24:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f9387c39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93800c88c0 0x7f9380155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f9387c39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9380158b60 0x7f9380158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f9387c39640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93800c73d0 con 0x7f938014f360 2024-03-06T07:24:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f9387c39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93800c8dd0 con 0x7f93800c88c0 2024-03-06T07:24:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f9387c39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93800be2e0 con 0x7f9380158b60 2024-03-06T07:24:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93851d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93800c88c0 0x7f9380155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93851d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93800c88c0 0x7f9380155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35438/0 (socket says 172.21.15.26:35438) 2024-03-06T07:24:47.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93851d4640 1 -- 172.21.15.26:0/2773079558 learned_addr learned my addr 172.21.15.26:0/2773079558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:47.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93861d6640 1 --2- 172.21.15.26:0/2773079558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9380158b60 0x7f9380158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93851d4640 1 -- 172.21.15.26:0/2773079558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9380158b60 msgr2=0x7f9380158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:47.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93851d4640 1 --2- 172.21.15.26:0/2773079558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9380158b60 0x7f9380158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93851d4640 1 -- 172.21.15.26:0/2773079558 >> v1:172.21.15.117:6789/0 conn(0x7f938014f360 legacy=0x7f93800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.071+0000 7f93851d4640 1 -- 172.21.15.26:0/2773079558 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93800c7670 con 0x7f93800c88c0 2024-03-06T07:24:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.072+0000 7f93851d4640 1 --2- 172.21.15.26:0/2773079558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380155c10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9374009e50 tx=0x7f937402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480ac6b256cde33 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.072+0000 7f93849d3640 1 -- 172.21.15.26:0/2773079558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937402ca70 con 0x7f93800c88c0 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.072+0000 7f93849d3640 1 -- 172.21.15.26:0/2773079558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937402cc10 con 0x7f93800c88c0 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.072+0000 7f93849d3640 1 -- 172.21.15.26:0/2773079558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93740343f0 con 0x7f93800c88c0 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 -- 172.21.15.26:0/2773079558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 msgr2=0x7f9380155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 --2- 172.21.15.26:0/2773079558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380155c10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9374009e50 tx=0x7f937402bed0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 -- 172.21.15.26:0/2773079558 shutdown_connections 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 --2- 172.21.15.26:0/2773079558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9380158b60 0x7f9380158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 --2- 172.21.15.26:0/2773079558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380155c10 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 -- 172.21.15.26:0/2773079558 >> 172.21.15.26:0/2773079558 conn(0x7f93800ac570 msgr2=0x7f93800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 -- 172.21.15.26:0/2773079558 shutdown_connections 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.073+0000 7f9387c39640 1 -- 172.21.15.26:0/2773079558 wait complete. 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.074+0000 7f9387c39640 1 Processor -- start 2024-03-06T07:24:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.074+0000 7f9387c39640 1 -- start start 2024-03-06T07:24:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f9387c39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f9387c39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938014f360 0x7f93800c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f9387c39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9380158b60 0x7f93800bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f9387c39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93800c5a50 con 0x7f93800c88c0 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f9387c39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93800c7c30 con 0x7f9380158b60 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f9387c39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93800c5bd0 con 0x7f938014f360 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f93859d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f93859d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35454/0 (socket says 172.21.15.26:35454) 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f93859d5640 1 -- 172.21.15.26:0/3352158630 learned_addr learned my addr 172.21.15.26:0/3352158630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f93861d6640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9380158b60 0x7f93800bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.075+0000 7f93851d4640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938014f360 0x7f93800c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93859d5640 1 -- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938014f360 msgr2=0x7f93800c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93859d5640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938014f360 0x7f93800c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93859d5640 1 -- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9380158b60 msgr2=0x7f93800bf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93859d5640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9380158b60 0x7f93800bf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93859d5640 1 -- 172.21.15.26:0/3352158630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93801ebea0 con 0x7f93800c88c0 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93861d6640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9380158b60 0x7f93800bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93851d4640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938014f360 0x7f93800c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f93859d5640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380151400 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f937800c3f0 tx=0x7f937800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f937e7fc640 1 -- 172.21.15.26:0/3352158630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93780052d0 con 0x7f93800c88c0 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f937e7fc640 1 -- 172.21.15.26:0/3352158630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9378005490 con 0x7f93800c88c0 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93800be530 con 0x7f93800c88c0 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f937e7fc640 1 -- 172.21.15.26:0/3352158630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378007260 con 0x7f93800c88c0 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.076+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93800beaf0 con 0x7f93800c88c0 2024-03-06T07:24:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.077+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9380155670 con 0x7f93800c88c0 2024-03-06T07:24:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.078+0000 7f937e7fc640 1 -- 172.21.15.26:0/3352158630 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9378007400 con 0x7f93800c88c0 2024-03-06T07:24:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.078+0000 7f937e7fc640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f936c06b4d0 0x7f936c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.079+0000 7f937e7fc640 1 -- 172.21.15.26:0/3352158630 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93780897a0 con 0x7f93800c88c0 2024-03-06T07:24:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.079+0000 7f93851d4640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f936c06b4d0 0x7f936c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.079+0000 7f93851d4640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f936c06b4d0 0x7f936c06d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9374035000 tx=0x7f9374030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.083+0000 7f937e7fc640 1 -- 172.21.15.26:0/3352158630 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9380155670 con 0x7f93800c88c0 2024-03-06T07:24:47.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.302+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9380155320 con 0x7f93800c88c0 2024-03-06T07:24:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.303+0000 7f937e7fc640 1 -- 172.21.15.26:0/3352158630 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f9378058520 con 0x7f93800c88c0 2024-03-06T07:24:47.304 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:47.304 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1856740515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1856740515},{"type":"v1","addr":"172.21.15.26:6834","nonce":1856740515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:24:45.977307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/577277314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":577277314},{"type":"v1","addr":"172.21.15.26:6835","nonce":577277314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-03-06T07:24:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 105 2024-03-06T07:24:47.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.304+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f936c06b4d0 msgr2=0x7f936c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.304+0000 7f9387c39640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f936c06b4d0 0x7f936c06d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9374035000 tx=0x7f9374030000 comp rx=0 tx=0).stop 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 msgr2=0x7f9380151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380151400 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f937800c3f0 tx=0x7f937800c8b0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 shutdown_connections 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f936c06b4d0 0x7f936c06d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9380158b60 0x7f93800bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938014f360 0x7f93800c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 --2- 172.21.15.26:0/3352158630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93800c88c0 0x7f9380151400 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 >> 172.21.15.26:0/3352158630 conn(0x7f93800ac570 msgr2=0x7f938006b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 shutdown_connections 2024-03-06T07:24:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.305+0000 7f9387c39640 1 -- 172.21.15.26:0/3352158630 wait complete. 2024-03-06T07:24:47.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:47.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.421+0000 7f25ca6af640 1 Processor -- start 2024-03-06T07:24:47.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.421+0000 7f25ca6af640 1 -- start start 2024-03-06T07:24:47.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25ca6af640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25c414f360 0x7f25c4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25ca6af640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25c40c88c0 0x7f25c40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25ca6af640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c5880 0x7f25c4158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25ca6af640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c40c8dd0 con 0x7f25c40c88c0 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25ca6af640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c40c73d0 con 0x7f25c414f360 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25ca6af640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c40be2e0 con 0x7f25c40c5880 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25c40c88c0 0x7f25c40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25c40c88c0 0x7f25c40c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35462/0 (socket says 172.21.15.26:35462) 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 -- 172.21.15.26:0/1946042716 learned_addr learned my addr 172.21.15.26:0/1946042716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c3fff640 1 --2- 172.21.15.26:0/1946042716 >> v2:172.21.15.117:3300/0 conn(0x7f25c414f360 0x7f25c4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c8c4c640 1 --2- 172.21.15.26:0/1946042716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c5880 0x7f25c4158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 -- 172.21.15.26:0/1946042716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c5880 msgr2=0x7f25c4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 --2- 172.21.15.26:0/1946042716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c5880 0x7f25c4158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 -- 172.21.15.26:0/1946042716 >> v2:172.21.15.117:3300/0 conn(0x7f25c414f360 msgr2=0x7f25c4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:47.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 --2- 172.21.15.26:0/1946042716 >> v2:172.21.15.117:3300/0 conn(0x7f25c414f360 0x7f25c4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c37fe640 1 -- 172.21.15.26:0/1946042716 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40c7670 con 0x7f25c40c88c0 2024-03-06T07:24:47.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.422+0000 7f25c3fff640 1 --2- 172.21.15.26:0/1946042716 >> v2:172.21.15.117:3300/0 conn(0x7f25c414f360 0x7f25c4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:47.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.423+0000 7f25c37fe640 1 --2- 172.21.15.26:0/1946042716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c40c88c0 0x7f25c40c5340 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f25c408c840 tx=0x7f25b802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cbcab7d3a6d767 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.423+0000 7f25c2ffd640 1 -- 172.21.15.26:0/1946042716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b802e070 con 0x7f25c40c88c0 2024-03-06T07:24:47.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.423+0000 7f25c2ffd640 1 -- 172.21.15.26:0/1946042716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8037070 con 0x7f25c40c88c0 2024-03-06T07:24:47.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.423+0000 7f25c2ffd640 1 -- 172.21.15.26:0/1946042716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8031a00 con 0x7f25c40c88c0 2024-03-06T07:24:47.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 -- 172.21.15.26:0/1946042716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c40c88c0 msgr2=0x7f25c40c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1946042716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c40c88c0 0x7f25c40c5340 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f25c408c840 tx=0x7f25b802b880 comp rx=0 tx=0).stop 2024-03-06T07:24:47.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 -- 172.21.15.26:0/1946042716 shutdown_connections 2024-03-06T07:24:47.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1946042716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c5880 0x7f25c4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1946042716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c40c88c0 0x7f25c40c5340 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1946042716 >> v2:172.21.15.117:3300/0 conn(0x7f25c414f360 0x7f25c4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 -- 172.21.15.26:0/1946042716 >> 172.21.15.26:0/1946042716 conn(0x7f25c40ac570 msgr2=0x7f25c40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:47.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 -- 172.21.15.26:0/1946042716 shutdown_connections 2024-03-06T07:24:47.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.424+0000 7f25ca6af640 1 -- 172.21.15.26:0/1946042716 wait complete. 2024-03-06T07:24:47.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.426+0000 7f25ca6af640 1 Processor -- start 2024-03-06T07:24:47.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.426+0000 7f25ca6af640 1 -- start start 2024-03-06T07:24:47.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25ca6af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c40c5880 0x7f25c40c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25ca6af640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c88c0 0x7f25c40c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25ca6af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c414f360 0x7f25c40c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25ca6af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c41590a0 con 0x7f25c414f360 2024-03-06T07:24:47.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25ca6af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c40c7c30 con 0x7f25c40c5880 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25ca6af640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4159220 con 0x7f25c40c88c0 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c414f360 0x7f25c40c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c414f360 0x7f25c40c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35474/0 (socket says 172.21.15.26:35474) 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 -- 172.21.15.26:0/1921666317 learned_addr learned my addr 172.21.15.26:0/1921666317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c37fe640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c88c0 0x7f25c40c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c3fff640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c40c5880 0x7f25c40c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 -- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c88c0 msgr2=0x7f25c40c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c88c0 0x7f25c40c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 -- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c40c5880 msgr2=0x7f25c40c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c40c5880 0x7f25c40c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.427+0000 7f25c8c4c640 1 -- 172.21.15.26:0/1921666317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40be740 con 0x7f25c414f360 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.428+0000 7f25c8c4c640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c414f360 0x7f25c40c9ea0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f25bc007e60 tx=0x7f25bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.428+0000 7f25c0ff9640 1 -- 172.21.15.26:0/1921666317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc014620 con 0x7f25c414f360 2024-03-06T07:24:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.428+0000 7f25c0ff9640 1 -- 172.21.15.26:0/1921666317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc019070 con 0x7f25c414f360 2024-03-06T07:24:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.428+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c41ee880 con 0x7f25c414f360 2024-03-06T07:24:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.428+0000 7f25c0ff9640 1 -- 172.21.15.26:0/1921666317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc014a20 con 0x7f25c414f360 2024-03-06T07:24:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.428+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c41eeb60 con 0x7f25c414f360 2024-03-06T07:24:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.429+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c41ee6b0 con 0x7f25c414f360 2024-03-06T07:24:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.430+0000 7f25c0ff9640 1 -- 172.21.15.26:0/1921666317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25bc014bc0 con 0x7f25c414f360 2024-03-06T07:24:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.430+0000 7f25c0ff9640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25b006b500 0x7f25b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.430+0000 7f25c0ff9640 1 -- 172.21.15.26:0/1921666317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f25bc08aa70 con 0x7f25c414f360 2024-03-06T07:24:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.430+0000 7f25c3fff640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25b006b500 0x7f25b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.431+0000 7f25c3fff640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25b006b500 0x7f25b006d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006fd0 tx=0x7f25b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.434+0000 7f25c0ff9640 1 -- 172.21.15.26:0/1921666317 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f25bc059100 con 0x7f25c414f360 2024-03-06T07:24:47.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.618+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25c404f330 con 0x7f25c414f360 2024-03-06T07:24:47.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.619+0000 7f25c0ff9640 1 -- 172.21.15.26:0/1921666317 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16912 (secure 0 0 0) 0x7f25bc05d1a0 con 0x7f25c414f360 2024-03-06T07:24:47.622 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:47.622 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":121,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:24:44.932679+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:41.558889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:42.251998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:47.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25b006b500 msgr2=0x7f25b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25b006b500 0x7f25b006d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006fd0 tx=0x7f25b4008040 comp rx=0 tx=0).stop 2024-03-06T07:24:47.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c414f360 msgr2=0x7f25c40c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c414f360 0x7f25c40c9ea0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f25bc007e60 tx=0x7f25bc00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 shutdown_connections 2024-03-06T07:24:47.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f25b006b500 0x7f25b006d9c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c414f360 0x7f25c40c9ea0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25c40c88c0 0x7f25c40c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 --2- 172.21.15.26:0/1921666317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c40c5880 0x7f25c40c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 >> 172.21.15.26:0/1921666317 conn(0x7f25c40ac570 msgr2=0x7f25c40cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:47.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 shutdown_connections 2024-03-06T07:24:47.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.621+0000 7f25ca6af640 1 -- 172.21.15.26:0/1921666317 wait complete. 2024-03-06T07:24:47.635 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:24:47.636 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:24:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.734+0000 7f50cdb48640 1 Processor -- start 2024-03-06T07:24:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.734+0000 7f50cdb48640 1 -- start start 2024-03-06T07:24:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50cdb48640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50c80c88c0 0x7f50c8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50cdb48640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50c8158b60 0x7f50c8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50cdb48640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c80c73d0 con 0x7f50c814f360 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50cdb48640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80c8dd0 con 0x7f50c80c88c0 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50cdb48640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80be2e0 con 0x7f50c8158b60 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50c6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50c80c88c0 0x7f50c8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50c77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f50c814f360 0x7f50c80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55812/0 (socket says 172.21.15.26:55812) 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.735+0000 7f50c77fe640 1 -- 172.21.15.26:0/2312589671 learned_addr learned my addr 172.21.15.26:0/2312589671 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.736+0000 7f50c7fff640 1 --2- 172.21.15.26:0/2312589671 >> v2:172.21.15.117:3300/0 conn(0x7f50c8158b60 0x7f50c8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.736+0000 7f50c67fc640 1 -- 172.21.15.26:0/2312589671 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791664644 0 0) 0x7f50c80c73d0 con 0x7f50c814f360 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.736+0000 7f50c67fc640 1 -- 172.21.15.26:0/2312589671 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b4003520 con 0x7f50c814f360 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.736+0000 7f50c6ffd640 1 -- 172.21.15.26:0/2312589671 >> v2:172.21.15.117:3300/0 conn(0x7f50c8158b60 msgr2=0x7f50c8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.736+0000 7f50c6ffd640 1 --2- 172.21.15.26:0/2312589671 >> v2:172.21.15.117:3300/0 conn(0x7f50c8158b60 0x7f50c8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.736+0000 7f50c6ffd640 1 -- 172.21.15.26:0/2312589671 >> v1:172.21.15.26:6789/0 conn(0x7f50c814f360 legacy=0x7f50c80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.736+0000 7f50c6ffd640 1 -- 172.21.15.26:0/2312589671 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80c7670 con 0x7f50c80c88c0 2024-03-06T07:24:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50c6ffd640 1 --2- 172.21.15.26:0/2312589671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c80c88c0 0x7f50c8155c10 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f50c8152d60 tx=0x7f50b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef473290998ee7f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50c67fc640 1 -- 172.21.15.26:0/2312589671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00315a0 con 0x7f50c80c88c0 2024-03-06T07:24:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50c67fc640 1 -- 172.21.15.26:0/2312589671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b0037070 con 0x7f50c80c88c0 2024-03-06T07:24:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50c67fc640 1 -- 172.21.15.26:0/2312589671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0031940 con 0x7f50c80c88c0 2024-03-06T07:24:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 -- 172.21.15.26:0/2312589671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c80c88c0 msgr2=0x7f50c8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 --2- 172.21.15.26:0/2312589671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c80c88c0 0x7f50c8155c10 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f50c8152d60 tx=0x7f50b002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 -- 172.21.15.26:0/2312589671 shutdown_connections 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 --2- 172.21.15.26:0/2312589671 >> v2:172.21.15.117:3300/0 conn(0x7f50c8158b60 0x7f50c8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 --2- 172.21.15.26:0/2312589671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c80c88c0 0x7f50c8155c10 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 -- 172.21.15.26:0/2312589671 >> 172.21.15.26:0/2312589671 conn(0x7f50c80ac570 msgr2=0x7f50c80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 -- 172.21.15.26:0/2312589671 shutdown_connections 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.737+0000 7f50cdb48640 1 -- 172.21.15.26:0/2312589671 wait complete. 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.738+0000 7f50cdb48640 1 Processor -- start 2024-03-06T07:24:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50cdb48640 1 -- start start 2024-03-06T07:24:47.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50cdb48640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c80c88c0 0x7f50c81ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50cdb48640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c814f360 0x7f50c814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50cdb48640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c8158b60 0x7f50c814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50cdb48640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80c5a50 con 0x7f50c814f360 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50cdb48640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80c7c30 con 0x7f50c80c88c0 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50cdb48640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80c5bd0 con 0x7f50c8158b60 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c814f360 0x7f50c814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c814f360 0x7f50c814bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35486/0 (socket says 172.21.15.26:35486) 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.739+0000 7f50c6ffd640 1 -- 172.21.15.26:0/416708307 learned_addr learned my addr 172.21.15.26:0/416708307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c7fff640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c8158b60 0x7f50c814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c77fe640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c80c88c0 0x7f50c81ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c6ffd640 1 -- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c8158b60 msgr2=0x7f50c814c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c6ffd640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c8158b60 0x7f50c814c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c6ffd640 1 -- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c80c88c0 msgr2=0x7f50c81ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c6ffd640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c80c88c0 0x7f50c81ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c6ffd640 1 -- 172.21.15.26:0/416708307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c81eedc0 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50c6ffd640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c814f360 0x7f50c814bbe0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009890 tx=0x7f50b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50bffff640 1 -- 172.21.15.26:0/416708307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0032630 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50bffff640 1 -- 172.21.15.26:0/416708307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b00327d0 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c81ef0a0 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50bffff640 1 -- 172.21.15.26:0/416708307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0032b80 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.740+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c81ec500 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.741+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c80c8590 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.742+0000 7f50bffff640 1 -- 172.21.15.26:0/416708307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50b004c020 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.742+0000 7f50bffff640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50a806b500 0x7f50a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.742+0000 7f50bffff640 1 -- 172.21.15.26:0/416708307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f50b002f020 con 0x7f50c814f360 2024-03-06T07:24:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.742+0000 7f50c77fe640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50a806b500 0x7f50a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.743+0000 7f50c77fe640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50a806b500 0x7f50a806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005880 tx=0x7f50b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.751+0000 7f50bffff640 1 -- 172.21.15.26:0/416708307 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f50c80c8590 con 0x7f50c814f360 2024-03-06T07:24:47.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.927+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50c814c660 con 0x7f50a806b500 2024-03-06T07:24:47.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.934+0000 7f50bffff640 1 -- 172.21.15.26:0/416708307 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252531 (secure 0 0 0) 0x7f50c814c660 con 0x7f50a806b500 2024-03-06T07:24:47.939 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:24:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.936+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50a806b500 msgr2=0x7f50a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.936+0000 7f50cdb48640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50a806b500 0x7f50a806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005880 tx=0x7f50b8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.936+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c814f360 msgr2=0x7f50c814bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.936+0000 7f50cdb48640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c814f360 0x7f50c814bbe0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009890 tx=0x7f50b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.936+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 shutdown_connections 2024-03-06T07:24:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.937+0000 7f50cdb48640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50a806b500 0x7f50a806d9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.937+0000 7f50cdb48640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c8158b60 0x7f50c814c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.937+0000 7f50cdb48640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c814f360 0x7f50c814bbe0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.937+0000 7f50cdb48640 1 --2- 172.21.15.26:0/416708307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c80c88c0 0x7f50c81ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.937+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 >> 172.21.15.26:0/416708307 conn(0x7f50c80ac570 msgr2=0x7f50c80ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.937+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 shutdown_connections 2024-03-06T07:24:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:47.937+0000 7f50cdb48640 1 -- 172.21.15.26:0/416708307 wait complete. 2024-03-06T07:24:47.950 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-03-06T07:24:47.088916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261440,"kb_used_data":11948,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713280,"statfs":{"total":773094113280,"available":772826398720,"internally_reserved":0,"allocated":12234752,"data_stored":6980960,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.170171"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.945152+0000","last_change":"2024-03-06T07:24:43.939874+0000","last_active":"2024-03-06T07:24:43.945152+0000","last_peered":"2024-03-06T07:24:43.945152+0000","last_clean":"2024-03-06T07:24:43.945152+0000","last_became_active":"2024-03-06T07:24:43.939600+0000","last_became_peered":"2024-03-06T07:24:43.939600+0000","last_unstale":"2024-03-06T07:24:43.945152+0000","last_undegraded":"2024-03-06T07:24:43.945152+0000","last_fullsized":"2024-03-06T07:24:43.945152+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:56.546228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949866+0000","last_change":"2024-03-06T07:24:43.944313+0000","last_active":"2024-03-06T07:24:44.949866+0000","last_peered":"2024-03-06T07:24:44.949866+0000","last_clean":"2024-03-06T07:24:44.949866+0000","last_became_active":"2024-03-06T07:24:43.943952+0000","last_became_peered":"2024-03-06T07:24:43.943952+0000","last_unstale":"2024-03-06T07:24:44.949866+0000","last_undegraded":"2024-03-06T07:24:44.949866+0000","last_fullsized":"2024-03-06T07:24:44.949866+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:18.756401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.954227+0000","last_change":"2024-03-06T07:24:43.948264+0000","last_active":"2024-03-06T07:24:43.954227+0000","last_peered":"2024-03-06T07:24:43.954227+0000","last_clean":"2024-03-06T07:24:43.954227+0000","last_became_active":"2024-03-06T07:24:43.948006+0000","last_became_peered":"2024-03-06T07:24:43.948006+0000","last_unstale":"2024-03-06T07:24:43.954227+0000","last_undegraded":"2024-03-06T07:24:43.954227+0000","last_fullsized":"2024-03-06T07:24:43.954227+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:13.675959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949744+0000","last_change":"2024-03-06T07:24:43.942587+0000","last_active":"2024-03-06T07:24:44.949744+0000","last_peered":"2024-03-06T07:24:44.949744+0000","last_clean":"2024-03-06T07:24:44.949744+0000","last_became_active":"2024-03-06T07:24:43.942291+0000","last_became_peered":"2024-03-06T07:24:43.942291+0000","last_unstale":"2024-03-06T07:24:44.949744+0000","last_undegraded":"2024-03-06T07:24:44.949744+0000","last_fullsized":"2024-03-06T07:24:44.949744+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:12.371160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995028+0000","last_change":"2024-03-06T07:24:43.939928+0000","last_active":"2024-03-06T07:24:44.995028+0000","last_peered":"2024-03-06T07:24:44.995028+0000","last_clean":"2024-03-06T07:24:44.995028+0000","last_became_active":"2024-03-06T07:24:43.939622+0000","last_became_peered":"2024-03-06T07:24:43.939622+0000","last_unstale":"2024-03-06T07:24:44.995028+0000","last_undegraded":"2024-03-06T07:24:44.995028+0000","last_fullsized":"2024-03-06T07:24:44.995028+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:05.929697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.956096+0000","last_change":"2024-03-06T07:24:43.954743+0000","last_active":"2024-03-06T07:24:43.956096+0000","last_peered":"2024-03-06T07:24:43.956096+0000","last_clean":"2024-03-06T07:24:43.956096+0000","last_became_active":"2024-03-06T07:24:43.954348+0000","last_became_peered":"2024-03-06T07:24:43.954348+0000","last_unstale":"2024-03-06T07:24:43.956096+0000","last_undegraded":"2024-03-06T07:24:43.956096+0000","last_fullsized":"2024-03-06T07:24:43.956096+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:57.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940570+0000","last_change":"2024-03-06T07:24:43.941641+0000","last_active":"2024-03-06T07:24:44.940570+0000","last_peered":"2024-03-06T07:24:44.940570+0000","last_clean":"2024-03-06T07:24:44.940570+0000","last_became_active":"2024-03-06T07:24:43.941379+0000","last_became_peered":"2024-03-06T07:24:43.941379+0000","last_unstale":"2024-03-06T07:24:44.940570+0000","last_undegraded":"2024-03-06T07:24:44.940570+0000","last_fullsized":"2024-03-06T07:24:44.940570+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:59.011549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946648+0000","last_change":"2024-03-06T07:24:43.943066+0000","last_active":"2024-03-06T07:24:44.946648+0000","last_peered":"2024-03-06T07:24:44.946648+0000","last_clean":"2024-03-06T07:24:44.946648+0000","last_became_active":"2024-03-06T07:24:43.942796+0000","last_became_peered":"2024-03-06T07:24:43.942796+0000","last_unstale":"2024-03-06T07:24:44.946648+0000","last_undegraded":"2024-03-06T07:24:44.946648+0000","last_fullsized":"2024-03-06T07:24:44.946648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:47.769348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.953522+0000","last_change":"2024-03-06T07:24:43.950547+0000","last_active":"2024-03-06T07:24:43.953522+0000","last_peered":"2024-03-06T07:24:43.953522+0000","last_clean":"2024-03-06T07:24:43.953522+0000","last_became_active":"2024-03-06T07:24:43.949947+0000","last_became_peered":"2024-03-06T07:24:43.949947+0000","last_unstale":"2024-03-06T07:24:43.953522+0000","last_undegraded":"2024-03-06T07:24:43.953522+0000","last_fullsized":"2024-03-06T07:24:43.953522+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:25.973751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941276+0000","last_change":"2024-03-06T07:24:43.946279+0000","last_active":"2024-03-06T07:24:44.941276+0000","last_peered":"2024-03-06T07:24:44.941276+0000","last_clean":"2024-03-06T07:24:44.941276+0000","last_became_active":"2024-03-06T07:24:43.945997+0000","last_became_peered":"2024-03-06T07:24:43.945997+0000","last_unstale":"2024-03-06T07:24:44.941276+0000","last_undegraded":"2024-03-06T07:24:44.941276+0000","last_fullsized":"2024-03-06T07:24:44.941276+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:35:36.815110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949780+0000","last_change":"2024-03-06T07:24:43.944418+0000","last_active":"2024-03-06T07:24:44.949780+0000","last_peered":"2024-03-06T07:24:44.949780+0000","last_clean":"2024-03-06T07:24:44.949780+0000","last_became_active":"2024-03-06T07:24:43.944094+0000","last_became_peered":"2024-03-06T07:24:43.944094+0000","last_unstale":"2024-03-06T07:24:44.949780+0000","last_undegraded":"2024-03-06T07:24:44.949780+0000","last_fullsized":"2024-03-06T07:24:44.949780+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:19.355051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944495+0000","last_change":"2024-03-06T07:24:43.940910+0000","last_active":"2024-03-06T07:24:43.944495+0000","last_peered":"2024-03-06T07:24:43.944495+0000","last_clean":"2024-03-06T07:24:43.944495+0000","last_became_active":"2024-03-06T07:24:43.940629+0000","last_became_peered":"2024-03-06T07:24:43.940629+0000","last_unstale":"2024-03-06T07:24:43.944495+0000","last_undegraded":"2024-03-06T07:24:43.944495+0000","last_fullsized":"2024-03-06T07:24:43.944495+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:39.945008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950761+0000","last_change":"2024-03-06T07:24:43.949291+0000","last_active":"2024-03-06T07:24:43.950761+0000","last_peered":"2024-03-06T07:24:43.950761+0000","last_clean":"2024-03-06T07:24:43.950761+0000","last_became_active":"2024-03-06T07:24:43.949021+0000","last_became_peered":"2024-03-06T07:24:43.949021+0000","last_unstale":"2024-03-06T07:24:43.950761+0000","last_undegraded":"2024-03-06T07:24:43.950761+0000","last_fullsized":"2024-03-06T07:24:43.950761+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:31.866021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951893+0000","last_change":"2024-03-06T07:24:43.944892+0000","last_active":"2024-03-06T07:24:43.951893+0000","last_peered":"2024-03-06T07:24:43.951893+0000","last_clean":"2024-03-06T07:24:43.951893+0000","last_became_active":"2024-03-06T07:24:43.944580+0000","last_became_peered":"2024-03-06T07:24:43.944580+0000","last_unstale":"2024-03-06T07:24:43.951893+0000","last_undegraded":"2024-03-06T07:24:43.951893+0000","last_fullsized":"2024-03-06T07:24:43.951893+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:27.155918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.952012+0000","last_change":"2024-03-06T07:24:43.944735+0000","last_active":"2024-03-06T07:24:43.952012+0000","last_peered":"2024-03-06T07:24:43.952012+0000","last_clean":"2024-03-06T07:24:43.952012+0000","last_became_active":"2024-03-06T07:24:43.944396+0000","last_became_peered":"2024-03-06T07:24:43.944396+0000","last_unstale":"2024-03-06T07:24:43.952012+0000","last_undegraded":"2024-03-06T07:24:43.952012+0000","last_fullsized":"2024-03-06T07:24:43.952012+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:54.741536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951775+0000","last_change":"2024-03-06T07:24:43.950149+0000","last_active":"2024-03-06T07:24:43.951775+0000","last_peered":"2024-03-06T07:24:43.951775+0000","last_clean":"2024-03-06T07:24:43.951775+0000","last_became_active":"2024-03-06T07:24:43.949549+0000","last_became_peered":"2024-03-06T07:24:43.949549+0000","last_unstale":"2024-03-06T07:24:43.951775+0000","last_undegraded":"2024-03-06T07:24:43.951775+0000","last_fullsized":"2024-03-06T07:24:43.951775+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:19.247439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951806+0000","last_change":"2024-03-06T07:24:43.950209+0000","last_active":"2024-03-06T07:24:43.951806+0000","last_peered":"2024-03-06T07:24:43.951806+0000","last_clean":"2024-03-06T07:24:43.951806+0000","last_became_active":"2024-03-06T07:24:43.949663+0000","last_became_peered":"2024-03-06T07:24:43.949663+0000","last_unstale":"2024-03-06T07:24:43.951806+0000","last_undegraded":"2024-03-06T07:24:43.951806+0000","last_fullsized":"2024-03-06T07:24:43.951806+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:49.531375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995223+0000","last_change":"2024-03-06T07:24:43.935701+0000","last_active":"2024-03-06T07:24:44.995223+0000","last_peered":"2024-03-06T07:24:44.995223+0000","last_clean":"2024-03-06T07:24:44.995223+0000","last_became_active":"2024-03-06T07:24:43.935380+0000","last_became_peered":"2024-03-06T07:24:43.935380+0000","last_unstale":"2024-03-06T07:24:44.995223+0000","last_undegraded":"2024-03-06T07:24:44.995223+0000","last_fullsized":"2024-03-06T07:24:44.995223+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:44.400364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940297+0000","last_change":"2024-03-06T07:24:43.934962+0000","last_active":"2024-03-06T07:24:44.940297+0000","last_peered":"2024-03-06T07:24:44.940297+0000","last_clean":"2024-03-06T07:24:44.940297+0000","last_became_active":"2024-03-06T07:24:43.934683+0000","last_became_peered":"2024-03-06T07:24:43.934683+0000","last_unstale":"2024-03-06T07:24:44.940297+0000","last_undegraded":"2024-03-06T07:24:44.940297+0000","last_fullsized":"2024-03-06T07:24:44.940297+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:22.677609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995061+0000","last_change":"2024-03-06T07:24:43.937922+0000","last_active":"2024-03-06T07:24:44.995061+0000","last_peered":"2024-03-06T07:24:44.995061+0000","last_clean":"2024-03-06T07:24:44.995061+0000","last_became_active":"2024-03-06T07:24:43.937648+0000","last_became_peered":"2024-03-06T07:24:43.937648+0000","last_unstale":"2024-03-06T07:24:44.995061+0000","last_undegraded":"2024-03-06T07:24:44.995061+0000","last_fullsized":"2024-03-06T07:24:44.995061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:55.447843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949641+0000","last_change":"2024-03-06T07:24:43.952274+0000","last_active":"2024-03-06T07:24:44.949641+0000","last_peered":"2024-03-06T07:24:44.949641+0000","last_clean":"2024-03-06T07:24:44.949641+0000","last_became_active":"2024-03-06T07:24:43.951976+0000","last_became_peered":"2024-03-06T07:24:43.951976+0000","last_unstale":"2024-03-06T07:24:44.949641+0000","last_undegraded":"2024-03-06T07:24:44.949641+0000","last_fullsized":"2024-03-06T07:24:44.949641+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:31.681793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944771+0000","last_change":"2024-03-06T07:24:43.940519+0000","last_active":"2024-03-06T07:24:43.944771+0000","last_peered":"2024-03-06T07:24:43.944771+0000","last_clean":"2024-03-06T07:24:43.944771+0000","last_became_active":"2024-03-06T07:24:43.940164+0000","last_became_peered":"2024-03-06T07:24:43.940164+0000","last_unstale":"2024-03-06T07:24:43.944771+0000","last_undegraded":"2024-03-06T07:24:43.944771+0000","last_fullsized":"2024-03-06T07:24:43.944771+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:24.881111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949596+0000","last_change":"2024-03-06T07:24:43.941775+0000","last_active":"2024-03-06T07:24:44.949596+0000","last_peered":"2024-03-06T07:24:44.949596+0000","last_clean":"2024-03-06T07:24:44.949596+0000","last_became_active":"2024-03-06T07:24:43.941487+0000","last_became_peered":"2024-03-06T07:24:43.941487+0000","last_unstale":"2024-03-06T07:24:44.949596+0000","last_undegraded":"2024-03-06T07:24:44.949596+0000","last_fullsized":"2024-03-06T07:24:44.949596+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:47.953509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951754+0000","last_change":"2024-03-06T07:24:43.949877+0000","last_active":"2024-03-06T07:24:43.951754+0000","last_peered":"2024-03-06T07:24:43.951754+0000","last_clean":"2024-03-06T07:24:43.951754+0000","last_became_active":"2024-03-06T07:24:43.949031+0000","last_became_peered":"2024-03-06T07:24:43.949031+0000","last_unstale":"2024-03-06T07:24:43.951754+0000","last_undegraded":"2024-03-06T07:24:43.951754+0000","last_fullsized":"2024-03-06T07:24:43.951754+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:36.834369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948986+0000","last_change":"2024-03-06T07:24:43.946356+0000","last_active":"2024-03-06T07:24:43.948986+0000","last_peered":"2024-03-06T07:24:43.948986+0000","last_clean":"2024-03-06T07:24:43.948986+0000","last_became_active":"2024-03-06T07:24:43.944601+0000","last_became_peered":"2024-03-06T07:24:43.944601+0000","last_unstale":"2024-03-06T07:24:43.948986+0000","last_undegraded":"2024-03-06T07:24:43.948986+0000","last_fullsized":"2024-03-06T07:24:43.948986+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:07.760345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948742+0000","last_change":"2024-03-06T07:24:43.944501+0000","last_active":"2024-03-06T07:24:43.948742+0000","last_peered":"2024-03-06T07:24:43.948742+0000","last_clean":"2024-03-06T07:24:43.948742+0000","last_became_active":"2024-03-06T07:24:43.944229+0000","last_became_peered":"2024-03-06T07:24:43.944229+0000","last_unstale":"2024-03-06T07:24:43.948742+0000","last_undegraded":"2024-03-06T07:24:43.948742+0000","last_fullsized":"2024-03-06T07:24:43.948742+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:48.383289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940459+0000","last_change":"2024-03-06T07:24:43.953312+0000","last_active":"2024-03-06T07:24:44.940459+0000","last_peered":"2024-03-06T07:24:44.940459+0000","last_clean":"2024-03-06T07:24:44.940459+0000","last_became_active":"2024-03-06T07:24:43.952387+0000","last_became_peered":"2024-03-06T07:24:43.952387+0000","last_unstale":"2024-03-06T07:24:44.940459+0000","last_undegraded":"2024-03-06T07:24:44.940459+0000","last_fullsized":"2024-03-06T07:24:44.940459+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:44.400364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941208+0000","last_change":"2024-03-06T07:24:43.947437+0000","last_active":"2024-03-06T07:24:44.941208+0000","last_peered":"2024-03-06T07:24:44.941208+0000","last_clean":"2024-03-06T07:24:44.941208+0000","last_became_active":"2024-03-06T07:24:43.946787+0000","last_became_peered":"2024-03-06T07:24:43.946787+0000","last_unstale":"2024-03-06T07:24:44.941208+0000","last_undegraded":"2024-03-06T07:24:44.941208+0000","last_fullsized":"2024-03-06T07:24:44.941208+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:15.119383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.957166+0000","last_change":"2024-03-06T07:24:43.953413+0000","last_active":"2024-03-06T07:24:43.957166+0000","last_peered":"2024-03-06T07:24:43.957166+0000","last_clean":"2024-03-06T07:24:43.957166+0000","last_became_active":"2024-03-06T07:24:43.953163+0000","last_became_peered":"2024-03-06T07:24:43.953163+0000","last_unstale":"2024-03-06T07:24:43.957166+0000","last_undegraded":"2024-03-06T07:24:43.957166+0000","last_fullsized":"2024-03-06T07:24:43.957166+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:42.298185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948859+0000","last_change":"2024-03-06T07:24:43.944557+0000","last_active":"2024-03-06T07:24:43.948859+0000","last_peered":"2024-03-06T07:24:43.948859+0000","last_clean":"2024-03-06T07:24:43.948859+0000","last_became_active":"2024-03-06T07:24:43.944255+0000","last_became_peered":"2024-03-06T07:24:43.944255+0000","last_unstale":"2024-03-06T07:24:43.948859+0000","last_undegraded":"2024-03-06T07:24:43.948859+0000","last_fullsized":"2024-03-06T07:24:43.948859+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:32:11.860704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946805+0000","last_change":"2024-03-06T07:24:43.954117+0000","last_active":"2024-03-06T07:24:44.946805+0000","last_peered":"2024-03-06T07:24:44.946805+0000","last_clean":"2024-03-06T07:24:44.946805+0000","last_became_active":"2024-03-06T07:24:43.953742+0000","last_became_peered":"2024-03-06T07:24:43.953742+0000","last_unstale":"2024-03-06T07:24:44.946805+0000","last_undegraded":"2024-03-06T07:24:44.946805+0000","last_fullsized":"2024-03-06T07:24:44.946805+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:59.661391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941049+0000","last_change":"2024-03-06T07:24:43.953105+0000","last_active":"2024-03-06T07:24:44.941049+0000","last_peered":"2024-03-06T07:24:44.941049+0000","last_clean":"2024-03-06T07:24:44.941049+0000","last_became_active":"2024-03-06T07:24:43.952816+0000","last_became_peered":"2024-03-06T07:24:43.952816+0000","last_unstale":"2024-03-06T07:24:44.941049+0000","last_undegraded":"2024-03-06T07:24:44.941049+0000","last_fullsized":"2024-03-06T07:24:44.941049+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.370886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946713+0000","last_change":"2024-03-06T07:24:43.931794+0000","last_active":"2024-03-06T07:24:44.946713+0000","last_peered":"2024-03-06T07:24:44.946713+0000","last_clean":"2024-03-06T07:24:44.946713+0000","last_became_active":"2024-03-06T07:24:43.930959+0000","last_became_peered":"2024-03-06T07:24:43.930959+0000","last_unstale":"2024-03-06T07:24:44.946713+0000","last_undegraded":"2024-03-06T07:24:44.946713+0000","last_fullsized":"2024-03-06T07:24:44.946713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:49.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"120'4","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.959962+0000","last_change":"2024-03-06T07:24:42.930288+0000","last_active":"2024-03-06T07:24:43.959962+0000","last_peered":"2024-03-06T07:24:43.959962+0000","last_clean":"2024-03-06T07:24:43.959962+0000","last_became_active":"2024-03-06T07:24:42.930016+0000","last_became_peered":"2024-03-06T07:24:42.930016+0000","last_unstale":"2024-03-06T07:24:43.959962+0000","last_undegraded":"2024-03-06T07:24:43.959962+0000","last_fullsized":"2024-03-06T07:24:43.959962+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:06.327089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944195+0000","last_change":"2024-03-06T07:24:43.938001+0000","last_active":"2024-03-06T07:24:43.944195+0000","last_peered":"2024-03-06T07:24:43.944195+0000","last_clean":"2024-03-06T07:24:43.944195+0000","last_became_active":"2024-03-06T07:24:43.937712+0000","last_became_peered":"2024-03-06T07:24:43.937712+0000","last_unstale":"2024-03-06T07:24:43.944195+0000","last_undegraded":"2024-03-06T07:24:43.944195+0000","last_fullsized":"2024-03-06T07:24:43.944195+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:41.989637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995322+0000","last_change":"2024-03-06T07:24:42.933650+0000","last_active":"2024-03-06T07:24:44.995322+0000","last_peered":"2024-03-06T07:24:44.995322+0000","last_clean":"2024-03-06T07:24:44.995322+0000","last_became_active":"2024-03-06T07:24:42.932924+0000","last_became_peered":"2024-03-06T07:24:42.932924+0000","last_unstale":"2024-03-06T07:24:44.995322+0000","last_undegraded":"2024-03-06T07:24:44.995322+0000","last_fullsized":"2024-03-06T07:24:44.995322+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:19.886553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949715+0000","last_change":"2024-03-06T07:24:43.941880+0000","last_active":"2024-03-06T07:24:44.949715+0000","last_peered":"2024-03-06T07:24:44.949715+0000","last_clean":"2024-03-06T07:24:44.949715+0000","last_became_active":"2024-03-06T07:24:43.941463+0000","last_became_peered":"2024-03-06T07:24:43.941463+0000","last_unstale":"2024-03-06T07:24:44.949715+0000","last_undegraded":"2024-03-06T07:24:44.949715+0000","last_fullsized":"2024-03-06T07:24:44.949715+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:45.762532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940572+0000","last_change":"2024-03-06T07:24:42.935712+0000","last_active":"2024-03-06T07:24:44.940572+0000","last_peered":"2024-03-06T07:24:44.940572+0000","last_clean":"2024-03-06T07:24:44.940572+0000","last_became_active":"2024-03-06T07:24:42.935432+0000","last_became_peered":"2024-03-06T07:24:42.935432+0000","last_unstale":"2024-03-06T07:24:44.940572+0000","last_undegraded":"2024-03-06T07:24:44.940572+0000","last_fullsized":"2024-03-06T07:24:44.940572+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:11.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944161+0000","last_change":"2024-03-06T07:24:43.941772+0000","last_active":"2024-03-06T07:24:43.944161+0000","last_peered":"2024-03-06T07:24:43.944161+0000","last_clean":"2024-03-06T07:24:43.944161+0000","last_became_active":"2024-03-06T07:24:43.941459+0000","last_became_peered":"2024-03-06T07:24:43.941459+0000","last_unstale":"2024-03-06T07:24:43.944161+0000","last_undegraded":"2024-03-06T07:24:43.944161+0000","last_fullsized":"2024-03-06T07:24:43.944161+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:13.255227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940760+0000","last_change":"2024-03-06T07:24:42.928151+0000","last_active":"2024-03-06T07:24:44.940760+0000","last_peered":"2024-03-06T07:24:44.940760+0000","last_clean":"2024-03-06T07:24:44.940760+0000","last_became_active":"2024-03-06T07:24:42.927844+0000","last_became_peered":"2024-03-06T07:24:42.927844+0000","last_unstale":"2024-03-06T07:24:44.940760+0000","last_undegraded":"2024-03-06T07:24:44.940760+0000","last_fullsized":"2024-03-06T07:24:44.940760+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:17.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946952+0000","last_change":"2024-03-06T07:24:43.954506+0000","last_active":"2024-03-06T07:24:44.946952+0000","last_peered":"2024-03-06T07:24:44.946952+0000","last_clean":"2024-03-06T07:24:44.946952+0000","last_became_active":"2024-03-06T07:24:43.953630+0000","last_became_peered":"2024-03-06T07:24:43.953630+0000","last_unstale":"2024-03-06T07:24:44.946952+0000","last_undegraded":"2024-03-06T07:24:44.946952+0000","last_fullsized":"2024-03-06T07:24:44.946952+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:31.866021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946987+0000","last_change":"2024-03-06T07:24:42.935678+0000","last_active":"2024-03-06T07:24:44.946987+0000","last_peered":"2024-03-06T07:24:44.946987+0000","last_clean":"2024-03-06T07:24:44.946987+0000","last_became_active":"2024-03-06T07:24:42.935426+0000","last_became_peered":"2024-03-06T07:24:42.935426+0000","last_unstale":"2024-03-06T07:24:44.946987+0000","last_undegraded":"2024-03-06T07:24:44.946987+0000","last_fullsized":"2024-03-06T07:24:44.946987+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:47.379247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.947081+0000","last_change":"2024-03-06T07:24:43.940354+0000","last_active":"2024-03-06T07:24:44.947081+0000","last_peered":"2024-03-06T07:24:44.947081+0000","last_clean":"2024-03-06T07:24:44.947081+0000","last_became_active":"2024-03-06T07:24:43.940073+0000","last_became_peered":"2024-03-06T07:24:43.940073+0000","last_unstale":"2024-03-06T07:24:44.947081+0000","last_undegraded":"2024-03-06T07:24:44.947081+0000","last_fullsized":"2024-03-06T07:24:44.947081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:36.834369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.994927+0000","last_change":"2024-03-06T07:24:42.934439+0000","last_active":"2024-03-06T07:24:44.994927+0000","last_peered":"2024-03-06T07:24:44.994927+0000","last_clean":"2024-03-06T07:24:44.994927+0000","last_became_active":"2024-03-06T07:24:42.934180+0000","last_became_peered":"2024-03-06T07:24:42.934180+0000","last_unstale":"2024-03-06T07:24:44.994927+0000","last_undegraded":"2024-03-06T07:24:44.994927+0000","last_fullsized":"2024-03-06T07:24:44.994927+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:44.758491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995214+0000","last_change":"2024-03-06T07:24:43.940027+0000","last_active":"2024-03-06T07:24:44.995214+0000","last_peered":"2024-03-06T07:24:44.995214+0000","last_clean":"2024-03-06T07:24:44.995214+0000","last_became_active":"2024-03-06T07:24:43.939721+0000","last_became_peered":"2024-03-06T07:24:43.939721+0000","last_unstale":"2024-03-06T07:24:44.995214+0000","last_undegraded":"2024-03-06T07:24:44.995214+0000","last_fullsized":"2024-03-06T07:24:44.995214+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:22.677609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.933397+0000","last_change":"2024-03-06T07:24:42.939720+0000","last_active":"2024-03-06T07:24:43.933397+0000","last_peered":"2024-03-06T07:24:43.933397+0000","last_clean":"2024-03-06T07:24:43.933397+0000","last_became_active":"2024-03-06T07:24:42.937175+0000","last_became_peered":"2024-03-06T07:24:42.937175+0000","last_unstale":"2024-03-06T07:24:43.933397+0000","last_undegraded":"2024-03-06T07:24:43.933397+0000","last_fullsized":"2024-03-06T07:24:43.933397+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:12.671918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940548+0000","last_change":"2024-03-06T07:24:43.952850+0000","last_active":"2024-03-06T07:24:44.940548+0000","last_peered":"2024-03-06T07:24:44.940548+0000","last_clean":"2024-03-06T07:24:44.940548+0000","last_became_active":"2024-03-06T07:24:43.952590+0000","last_became_peered":"2024-03-06T07:24:43.952590+0000","last_unstale":"2024-03-06T07:24:44.940548+0000","last_undegraded":"2024-03-06T07:24:44.940548+0000","last_fullsized":"2024-03-06T07:24:44.940548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:32.291753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.932470+0000","last_change":"2024-03-06T07:24:42.937330+0000","last_active":"2024-03-06T07:24:43.932470+0000","last_peered":"2024-03-06T07:24:43.932470+0000","last_clean":"2024-03-06T07:24:43.932470+0000","last_became_active":"2024-03-06T07:24:42.937053+0000","last_became_peered":"2024-03-06T07:24:42.937053+0000","last_unstale":"2024-03-06T07:24:43.932470+0000","last_undegraded":"2024-03-06T07:24:43.932470+0000","last_fullsized":"2024-03-06T07:24:43.932470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:03.404751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951973+0000","last_change":"2024-03-06T07:24:43.945377+0000","last_active":"2024-03-06T07:24:43.951973+0000","last_peered":"2024-03-06T07:24:43.951973+0000","last_clean":"2024-03-06T07:24:43.951973+0000","last_became_active":"2024-03-06T07:24:43.943817+0000","last_became_peered":"2024-03-06T07:24:43.943817+0000","last_unstale":"2024-03-06T07:24:43.951973+0000","last_undegraded":"2024-03-06T07:24:43.951973+0000","last_fullsized":"2024-03-06T07:24:43.951973+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:55.462660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949848+0000","last_change":"2024-03-06T07:24:42.938610+0000","last_active":"2024-03-06T07:24:44.949848+0000","last_peered":"2024-03-06T07:24:44.949848+0000","last_clean":"2024-03-06T07:24:44.949848+0000","last_became_active":"2024-03-06T07:24:42.938357+0000","last_became_peered":"2024-03-06T07:24:42.938357+0000","last_unstale":"2024-03-06T07:24:44.949848+0000","last_undegraded":"2024-03-06T07:24:44.949848+0000","last_fullsized":"2024-03-06T07:24:44.949848+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:48.527334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.953487+0000","last_change":"2024-03-06T07:24:43.950966+0000","last_active":"2024-03-06T07:24:43.953487+0000","last_peered":"2024-03-06T07:24:43.953487+0000","last_clean":"2024-03-06T07:24:43.953487+0000","last_became_active":"2024-03-06T07:24:43.950482+0000","last_became_peered":"2024-03-06T07:24:43.950482+0000","last_unstale":"2024-03-06T07:24:43.953487+0000","last_undegraded":"2024-03-06T07:24:43.953487+0000","last_fullsized":"2024-03-06T07:24:43.953487+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:24.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.929674+0000","last_change":"2024-03-06T07:24:42.936304+0000","last_active":"2024-03-06T07:24:43.929674+0000","last_peered":"2024-03-06T07:24:43.929674+0000","last_clean":"2024-03-06T07:24:43.929674+0000","last_became_active":"2024-03-06T07:24:42.935985+0000","last_became_peered":"2024-03-06T07:24:42.935985+0000","last_unstale":"2024-03-06T07:24:43.929674+0000","last_undegraded":"2024-03-06T07:24:43.929674+0000","last_fullsized":"2024-03-06T07:24:43.929674+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:10.960646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948879+0000","last_change":"2024-03-06T07:24:43.944575+0000","last_active":"2024-03-06T07:24:43.948879+0000","last_peered":"2024-03-06T07:24:43.948879+0000","last_clean":"2024-03-06T07:24:43.948879+0000","last_became_active":"2024-03-06T07:24:43.942026+0000","last_became_peered":"2024-03-06T07:24:43.942026+0000","last_unstale":"2024-03-06T07:24:43.948879+0000","last_undegraded":"2024-03-06T07:24:43.948879+0000","last_fullsized":"2024-03-06T07:24:43.948879+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:01.126798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940684+0000","last_change":"2024-03-06T07:24:42.926149+0000","last_active":"2024-03-06T07:24:44.940684+0000","last_peered":"2024-03-06T07:24:44.940684+0000","last_clean":"2024-03-06T07:24:44.940684+0000","last_became_active":"2024-03-06T07:24:42.925859+0000","last_became_peered":"2024-03-06T07:24:42.925859+0000","last_unstale":"2024-03-06T07:24:44.940684+0000","last_undegraded":"2024-03-06T07:24:44.940684+0000","last_fullsized":"2024-03-06T07:24:44.940684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:18.351009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.947874+0000","last_change":"2024-03-06T07:24:43.941527+0000","last_active":"2024-03-06T07:24:43.947874+0000","last_peered":"2024-03-06T07:24:43.947874+0000","last_clean":"2024-03-06T07:24:43.947874+0000","last_became_active":"2024-03-06T07:24:43.940197+0000","last_became_peered":"2024-03-06T07:24:43.940197+0000","last_unstale":"2024-03-06T07:24:43.947874+0000","last_undegraded":"2024-03-06T07:24:43.947874+0000","last_fullsized":"2024-03-06T07:24:43.947874+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:18.041541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940726+0000","last_change":"2024-03-06T07:24:42.931361+0000","last_active":"2024-03-06T07:24:44.940726+0000","last_peered":"2024-03-06T07:24:44.940726+0000","last_clean":"2024-03-06T07:24:44.940726+0000","last_became_active":"2024-03-06T07:24:42.931054+0000","last_became_peered":"2024-03-06T07:24:42.931054+0000","last_unstale":"2024-03-06T07:24:44.940726+0000","last_undegraded":"2024-03-06T07:24:44.940726+0000","last_fullsized":"2024-03-06T07:24:44.940726+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:44.758491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941238+0000","last_change":"2024-03-06T07:24:43.947680+0000","last_active":"2024-03-06T07:24:44.941238+0000","last_peered":"2024-03-06T07:24:44.941238+0000","last_clean":"2024-03-06T07:24:44.941238+0000","last_became_active":"2024-03-06T07:24:43.947149+0000","last_became_peered":"2024-03-06T07:24:43.947149+0000","last_unstale":"2024-03-06T07:24:44.941238+0000","last_undegraded":"2024-03-06T07:24:44.941238+0000","last_fullsized":"2024-03-06T07:24:44.941238+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:54.442533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"120'2","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.958926+0000","last_change":"2024-03-06T07:24:42.935511+0000","last_active":"2024-03-06T07:24:43.958926+0000","last_peered":"2024-03-06T07:24:43.958926+0000","last_clean":"2024-03-06T07:24:43.958926+0000","last_became_active":"2024-03-06T07:24:42.935241+0000","last_became_peered":"2024-03-06T07:24:42.935241+0000","last_unstale":"2024-03-06T07:24:43.958926+0000","last_undegraded":"2024-03-06T07:24:43.958926+0000","last_fullsized":"2024-03-06T07:24:43.958926+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:44:39.542423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940598+0000","last_change":"2024-03-06T07:24:43.942292+0000","last_active":"2024-03-06T07:24:44.940598+0000","last_peered":"2024-03-06T07:24:44.940598+0000","last_clean":"2024-03-06T07:24:44.940598+0000","last_became_active":"2024-03-06T07:24:43.942036+0000","last_became_peered":"2024-03-06T07:24:43.942036+0000","last_unstale":"2024-03-06T07:24:44.940598+0000","last_undegraded":"2024-03-06T07:24:44.940598+0000","last_fullsized":"2024-03-06T07:24:44.940598+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:51.018259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.932344+0000","last_change":"2024-03-06T07:24:42.931721+0000","last_active":"2024-03-06T07:24:43.932344+0000","last_peered":"2024-03-06T07:24:43.932344+0000","last_clean":"2024-03-06T07:24:43.932344+0000","last_became_active":"2024-03-06T07:24:42.931437+0000","last_became_peered":"2024-03-06T07:24:42.931437+0000","last_unstale":"2024-03-06T07:24:43.932344+0000","last_undegraded":"2024-03-06T07:24:43.932344+0000","last_fullsized":"2024-03-06T07:24:43.932344+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:30.506558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995322+0000","last_change":"2024-03-06T07:24:43.957348+0000","last_active":"2024-03-06T07:24:44.995322+0000","last_peered":"2024-03-06T07:24:44.995322+0000","last_clean":"2024-03-06T07:24:44.995322+0000","last_became_active":"2024-03-06T07:24:43.957049+0000","last_became_peered":"2024-03-06T07:24:43.957049+0000","last_unstale":"2024-03-06T07:24:44.995322+0000","last_undegraded":"2024-03-06T07:24:44.995322+0000","last_fullsized":"2024-03-06T07:24:44.995322+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:32.291753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"120'8","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.960563+0000","last_change":"2024-03-06T07:24:42.932357+0000","last_active":"2024-03-06T07:24:43.960563+0000","last_peered":"2024-03-06T07:24:43.960563+0000","last_clean":"2024-03-06T07:24:43.960563+0000","last_became_active":"2024-03-06T07:24:42.931427+0000","last_became_peered":"2024-03-06T07:24:42.931427+0000","last_unstale":"2024-03-06T07:24:43.960563+0000","last_undegraded":"2024-03-06T07:24:43.960563+0000","last_fullsized":"2024-03-06T07:24:43.960563+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:41.294144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948769+0000","last_change":"2024-03-06T07:24:43.943185+0000","last_active":"2024-03-06T07:24:43.948769+0000","last_peered":"2024-03-06T07:24:43.948769+0000","last_clean":"2024-03-06T07:24:43.948769+0000","last_became_active":"2024-03-06T07:24:43.942874+0000","last_became_peered":"2024-03-06T07:24:43.942874+0000","last_unstale":"2024-03-06T07:24:43.948769+0000","last_undegraded":"2024-03-06T07:24:43.948769+0000","last_fullsized":"2024-03-06T07:24:43.948769+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:27.618265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949956+0000","last_change":"2024-03-06T07:24:42.934663+0000","last_active":"2024-03-06T07:24:44.949956+0000","last_peered":"2024-03-06T07:24:44.949956+0000","last_clean":"2024-03-06T07:24:44.949956+0000","last_became_active":"2024-03-06T07:24:42.934390+0000","last_became_peered":"2024-03-06T07:24:42.934390+0000","last_unstale":"2024-03-06T07:24:44.949956+0000","last_undegraded":"2024-03-06T07:24:44.949956+0000","last_fullsized":"2024-03-06T07:24:44.949956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:18.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940663+0000","last_change":"2024-03-06T07:24:43.935478+0000","last_active":"2024-03-06T07:24:44.940663+0000","last_peered":"2024-03-06T07:24:44.940663+0000","last_clean":"2024-03-06T07:24:44.940663+0000","last_became_active":"2024-03-06T07:24:43.935211+0000","last_became_peered":"2024-03-06T07:24:43.935211+0000","last_unstale":"2024-03-06T07:24:44.940663+0000","last_undegraded":"2024-03-06T07:24:44.940663+0000","last_fullsized":"2024-03-06T07:24:44.940663+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:15.352025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"120'2","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.960053+0000","last_change":"2024-03-06T07:24:42.937923+0000","last_active":"2024-03-06T07:24:43.960053+0000","last_peered":"2024-03-06T07:24:43.960053+0000","last_clean":"2024-03-06T07:24:43.960053+0000","last_became_active":"2024-03-06T07:24:42.937634+0000","last_became_peered":"2024-03-06T07:24:42.937634+0000","last_unstale":"2024-03-06T07:24:43.960053+0000","last_undegraded":"2024-03-06T07:24:43.960053+0000","last_fullsized":"2024-03-06T07:24:43.960053+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:29.582696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940295+0000","last_change":"2024-03-06T07:24:43.952266+0000","last_active":"2024-03-06T07:24:44.940295+0000","last_peered":"2024-03-06T07:24:44.940295+0000","last_clean":"2024-03-06T07:24:44.940295+0000","last_became_active":"2024-03-06T07:24:43.951943+0000","last_became_peered":"2024-03-06T07:24:43.951943+0000","last_unstale":"2024-03-06T07:24:44.940295+0000","last_undegraded":"2024-03-06T07:24:44.940295+0000","last_fullsized":"2024-03-06T07:24:44.940295+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:38.370864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"120'1","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.958165+0000","last_change":"2024-03-06T07:24:42.940039+0000","last_active":"2024-03-06T07:24:43.958165+0000","last_peered":"2024-03-06T07:24:43.958165+0000","last_clean":"2024-03-06T07:24:43.958165+0000","last_became_active":"2024-03-06T07:24:42.939533+0000","last_became_peered":"2024-03-06T07:24:42.939533+0000","last_unstale":"2024-03-06T07:24:43.958165+0000","last_undegraded":"2024-03-06T07:24:43.958165+0000","last_fullsized":"2024-03-06T07:24:43.958165+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:06.756303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941166+0000","last_change":"2024-03-06T07:24:43.944803+0000","last_active":"2024-03-06T07:24:44.941166+0000","last_peered":"2024-03-06T07:24:44.941166+0000","last_clean":"2024-03-06T07:24:44.941166+0000","last_became_active":"2024-03-06T07:24:43.944536+0000","last_became_peered":"2024-03-06T07:24:43.944536+0000","last_unstale":"2024-03-06T07:24:44.941166+0000","last_undegraded":"2024-03-06T07:24:44.941166+0000","last_fullsized":"2024-03-06T07:24:44.941166+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:59:46.764059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995197+0000","last_change":"2024-03-06T07:24:42.929535+0000","last_active":"2024-03-06T07:24:44.995197+0000","last_peered":"2024-03-06T07:24:44.995197+0000","last_clean":"2024-03-06T07:24:44.995197+0000","last_became_active":"2024-03-06T07:24:42.929133+0000","last_became_peered":"2024-03-06T07:24:42.929133+0000","last_unstale":"2024-03-06T07:24:44.995197+0000","last_undegraded":"2024-03-06T07:24:44.995197+0000","last_fullsized":"2024-03-06T07:24:44.995197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:11.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995086+0000","last_change":"2024-03-06T07:24:43.945219+0000","last_active":"2024-03-06T07:24:44.995086+0000","last_peered":"2024-03-06T07:24:44.995086+0000","last_clean":"2024-03-06T07:24:44.995086+0000","last_became_active":"2024-03-06T07:24:43.944840+0000","last_became_peered":"2024-03-06T07:24:43.944840+0000","last_unstale":"2024-03-06T07:24:44.995086+0000","last_undegraded":"2024-03-06T07:24:44.995086+0000","last_fullsized":"2024-03-06T07:24:44.995086+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:38.370864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946987+0000","last_change":"2024-03-06T07:24:42.932533+0000","last_active":"2024-03-06T07:24:44.946987+0000","last_peered":"2024-03-06T07:24:44.946987+0000","last_clean":"2024-03-06T07:24:44.946987+0000","last_became_active":"2024-03-06T07:24:42.932245+0000","last_became_peered":"2024-03-06T07:24:42.932245+0000","last_unstale":"2024-03-06T07:24:44.946987+0000","last_undegraded":"2024-03-06T07:24:44.946987+0000","last_fullsized":"2024-03-06T07:24:44.946987+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:00.122757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951937+0000","last_change":"2024-03-06T07:24:43.944929+0000","last_active":"2024-03-06T07:24:43.951937+0000","last_peered":"2024-03-06T07:24:43.951937+0000","last_clean":"2024-03-06T07:24:43.951937+0000","last_became_active":"2024-03-06T07:24:43.944613+0000","last_became_peered":"2024-03-06T07:24:43.944613+0000","last_unstale":"2024-03-06T07:24:43.951937+0000","last_undegraded":"2024-03-06T07:24:43.951937+0000","last_fullsized":"2024-03-06T07:24:43.951937+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:02.290556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.930358+0000","last_change":"2024-03-06T07:24:42.932737+0000","last_active":"2024-03-06T07:24:43.930358+0000","last_peered":"2024-03-06T07:24:43.930358+0000","last_clean":"2024-03-06T07:24:43.930358+0000","last_became_active":"2024-03-06T07:24:42.932456+0000","last_became_peered":"2024-03-06T07:24:42.932456+0000","last_unstale":"2024-03-06T07:24:43.930358+0000","last_undegraded":"2024-03-06T07:24:43.930358+0000","last_fullsized":"2024-03-06T07:24:43.930358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:45.066971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995360+0000","last_change":"2024-03-06T07:24:43.945878+0000","last_active":"2024-03-06T07:24:44.995360+0000","last_peered":"2024-03-06T07:24:44.995360+0000","last_clean":"2024-03-06T07:24:44.995360+0000","last_became_active":"2024-03-06T07:24:43.945537+0000","last_became_peered":"2024-03-06T07:24:43.945537+0000","last_unstale":"2024-03-06T07:24:44.995360+0000","last_undegraded":"2024-03-06T07:24:44.995360+0000","last_fullsized":"2024-03-06T07:24:44.995360+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:59.011549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940677+0000","last_change":"2024-03-06T07:24:42.929247+0000","last_active":"2024-03-06T07:24:44.940677+0000","last_peered":"2024-03-06T07:24:44.940677+0000","last_clean":"2024-03-06T07:24:44.940677+0000","last_became_active":"2024-03-06T07:24:42.928944+0000","last_became_peered":"2024-03-06T07:24:42.928944+0000","last_unstale":"2024-03-06T07:24:44.940677+0000","last_undegraded":"2024-03-06T07:24:44.940677+0000","last_fullsized":"2024-03-06T07:24:44.940677+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:30.677752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949851+0000","last_change":"2024-03-06T07:24:43.944324+0000","last_active":"2024-03-06T07:24:44.949851+0000","last_peered":"2024-03-06T07:24:44.949851+0000","last_clean":"2024-03-06T07:24:44.949851+0000","last_became_active":"2024-03-06T07:24:43.944029+0000","last_became_peered":"2024-03-06T07:24:43.944029+0000","last_unstale":"2024-03-06T07:24:44.949851+0000","last_undegraded":"2024-03-06T07:24:44.949851+0000","last_fullsized":"2024-03-06T07:24:44.949851+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:20.890594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.934368+0000","last_change":"2024-03-06T07:24:42.940607+0000","last_active":"2024-03-06T07:24:43.934368+0000","last_peered":"2024-03-06T07:24:43.934368+0000","last_clean":"2024-03-06T07:24:43.934368+0000","last_became_active":"2024-03-06T07:24:42.940329+0000","last_became_peered":"2024-03-06T07:24:42.940329+0000","last_unstale":"2024-03-06T07:24:43.934368+0000","last_undegraded":"2024-03-06T07:24:43.934368+0000","last_fullsized":"2024-03-06T07:24:43.934368+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:37:05.544366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.994979+0000","last_change":"2024-03-06T07:24:43.938750+0000","last_active":"2024-03-06T07:24:44.994979+0000","last_peered":"2024-03-06T07:24:44.994979+0000","last_clean":"2024-03-06T07:24:44.994979+0000","last_became_active":"2024-03-06T07:24:43.938457+0000","last_became_peered":"2024-03-06T07:24:43.938457+0000","last_unstale":"2024-03-06T07:24:44.994979+0000","last_undegraded":"2024-03-06T07:24:44.994979+0000","last_fullsized":"2024-03-06T07:24:44.994979+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:34:31.367107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"120'2","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.958710+0000","last_change":"2024-03-06T07:24:42.931744+0000","last_active":"2024-03-06T07:24:43.958710+0000","last_peered":"2024-03-06T07:24:43.958710+0000","last_clean":"2024-03-06T07:24:43.958710+0000","last_became_active":"2024-03-06T07:24:42.931427+0000","last_became_peered":"2024-03-06T07:24:42.931427+0000","last_unstale":"2024-03-06T07:24:43.958710+0000","last_undegraded":"2024-03-06T07:24:43.958710+0000","last_fullsized":"2024-03-06T07:24:43.958710+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:39.994806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.947900+0000","last_change":"2024-03-06T07:24:43.942131+0000","last_active":"2024-03-06T07:24:43.947900+0000","last_peered":"2024-03-06T07:24:43.947900+0000","last_clean":"2024-03-06T07:24:43.947900+0000","last_became_active":"2024-03-06T07:24:43.940065+0000","last_became_peered":"2024-03-06T07:24:43.940065+0000","last_unstale":"2024-03-06T07:24:43.947900+0000","last_undegraded":"2024-03-06T07:24:43.947900+0000","last_fullsized":"2024-03-06T07:24:43.947900+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:49.298741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949009+0000","last_change":"2024-03-06T07:24:42.934734+0000","last_active":"2024-03-06T07:24:44.949009+0000","last_peered":"2024-03-06T07:24:44.949009+0000","last_clean":"2024-03-06T07:24:44.949009+0000","last_became_active":"2024-03-06T07:24:42.934472+0000","last_became_peered":"2024-03-06T07:24:42.934472+0000","last_unstale":"2024-03-06T07:24:44.949009+0000","last_undegraded":"2024-03-06T07:24:44.949009+0000","last_fullsized":"2024-03-06T07:24:44.949009+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:06.756303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940133+0000","last_change":"2024-03-06T07:24:43.947576+0000","last_active":"2024-03-06T07:24:44.940133+0000","last_peered":"2024-03-06T07:24:44.940133+0000","last_clean":"2024-03-06T07:24:44.940133+0000","last_became_active":"2024-03-06T07:24:43.947301+0000","last_became_peered":"2024-03-06T07:24:43.947301+0000","last_unstale":"2024-03-06T07:24:44.940133+0000","last_undegraded":"2024-03-06T07:24:44.940133+0000","last_fullsized":"2024-03-06T07:24:44.940133+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:55.447843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.947108+0000","last_change":"2024-03-06T07:24:42.936487+0000","last_active":"2024-03-06T07:24:44.947108+0000","last_peered":"2024-03-06T07:24:44.947108+0000","last_clean":"2024-03-06T07:24:44.947108+0000","last_became_active":"2024-03-06T07:24:42.936207+0000","last_became_peered":"2024-03-06T07:24:42.936207+0000","last_unstale":"2024-03-06T07:24:44.947108+0000","last_undegraded":"2024-03-06T07:24:44.947108+0000","last_fullsized":"2024-03-06T07:24:44.947108+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:32:10.856663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941211+0000","last_change":"2024-03-06T07:24:43.944204+0000","last_active":"2024-03-06T07:24:44.941211+0000","last_peered":"2024-03-06T07:24:44.941211+0000","last_clean":"2024-03-06T07:24:44.941211+0000","last_became_active":"2024-03-06T07:24:43.943939+0000","last_became_peered":"2024-03-06T07:24:43.943939+0000","last_unstale":"2024-03-06T07:24:44.941211+0000","last_undegraded":"2024-03-06T07:24:44.941211+0000","last_fullsized":"2024-03-06T07:24:44.941211+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:48.361482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.935393+0000","last_change":"2024-03-06T07:24:42.936217+0000","last_active":"2024-03-06T07:24:43.935393+0000","last_peered":"2024-03-06T07:24:43.935393+0000","last_clean":"2024-03-06T07:24:43.935393+0000","last_became_active":"2024-03-06T07:24:42.935902+0000","last_became_peered":"2024-03-06T07:24:42.935902+0000","last_unstale":"2024-03-06T07:24:43.935393+0000","last_undegraded":"2024-03-06T07:24:43.935393+0000","last_fullsized":"2024-03-06T07:24:43.935393+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:32.177158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950758+0000","last_change":"2024-03-06T07:24:43.948449+0000","last_active":"2024-03-06T07:24:43.950758+0000","last_peered":"2024-03-06T07:24:43.950758+0000","last_clean":"2024-03-06T07:24:43.950758+0000","last_became_active":"2024-03-06T07:24:43.948156+0000","last_became_peered":"2024-03-06T07:24:43.948156+0000","last_unstale":"2024-03-06T07:24:43.950758+0000","last_undegraded":"2024-03-06T07:24:43.950758+0000","last_fullsized":"2024-03-06T07:24:43.950758+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:49.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949676+0000","last_change":"2024-03-06T07:24:42.935809+0000","last_active":"2024-03-06T07:24:44.949676+0000","last_peered":"2024-03-06T07:24:44.949676+0000","last_clean":"2024-03-06T07:24:44.949676+0000","last_became_active":"2024-03-06T07:24:42.935540+0000","last_became_peered":"2024-03-06T07:24:42.935540+0000","last_unstale":"2024-03-06T07:24:44.949676+0000","last_undegraded":"2024-03-06T07:24:44.949676+0000","last_fullsized":"2024-03-06T07:24:44.949676+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:35.830328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950786+0000","last_change":"2024-03-06T07:24:43.948331+0000","last_active":"2024-03-06T07:24:43.950786+0000","last_peered":"2024-03-06T07:24:43.950786+0000","last_clean":"2024-03-06T07:24:43.950786+0000","last_became_active":"2024-03-06T07:24:43.948035+0000","last_became_peered":"2024-03-06T07:24:43.948035+0000","last_unstale":"2024-03-06T07:24:43.950786+0000","last_undegraded":"2024-03-06T07:24:43.950786+0000","last_fullsized":"2024-03-06T07:24:43.950786+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:47.769348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940727+0000","last_change":"2024-03-06T07:24:42.926540+0000","last_active":"2024-03-06T07:24:44.940727+0000","last_peered":"2024-03-06T07:24:44.940727+0000","last_clean":"2024-03-06T07:24:44.940727+0000","last_became_active":"2024-03-06T07:24:42.926223+0000","last_became_peered":"2024-03-06T07:24:42.926223+0000","last_unstale":"2024-03-06T07:24:44.940727+0000","last_undegraded":"2024-03-06T07:24:44.940727+0000","last_fullsized":"2024-03-06T07:24:44.940727+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:03.055775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950721+0000","last_change":"2024-03-06T07:24:43.948827+0000","last_active":"2024-03-06T07:24:43.950721+0000","last_peered":"2024-03-06T07:24:43.950721+0000","last_clean":"2024-03-06T07:24:43.950721+0000","last_became_active":"2024-03-06T07:24:43.948558+0000","last_became_peered":"2024-03-06T07:24:43.948558+0000","last_unstale":"2024-03-06T07:24:43.950721+0000","last_undegraded":"2024-03-06T07:24:43.950721+0000","last_fullsized":"2024-03-06T07:24:43.950721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:59.661391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.933586+0000","last_change":"2024-03-06T07:24:42.936906+0000","last_active":"2024-03-06T07:24:43.933586+0000","last_peered":"2024-03-06T07:24:43.933586+0000","last_clean":"2024-03-06T07:24:43.933586+0000","last_became_active":"2024-03-06T07:24:42.936619+0000","last_became_peered":"2024-03-06T07:24:42.936619+0000","last_unstale":"2024-03-06T07:24:43.933586+0000","last_undegraded":"2024-03-06T07:24:43.933586+0000","last_fullsized":"2024-03-06T07:24:43.933586+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:41:24.564414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940750+0000","last_change":"2024-03-06T07:24:43.945502+0000","last_active":"2024-03-06T07:24:44.940750+0000","last_peered":"2024-03-06T07:24:44.940750+0000","last_clean":"2024-03-06T07:24:44.940750+0000","last_became_active":"2024-03-06T07:24:43.945230+0000","last_became_peered":"2024-03-06T07:24:43.945230+0000","last_unstale":"2024-03-06T07:24:44.940750+0000","last_undegraded":"2024-03-06T07:24:44.940750+0000","last_fullsized":"2024-03-06T07:24:44.940750+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:40.438201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995033+0000","last_change":"2024-03-06T07:24:42.936142+0000","last_active":"2024-03-06T07:24:44.995033+0000","last_peered":"2024-03-06T07:24:44.995033+0000","last_clean":"2024-03-06T07:24:44.995033+0000","last_became_active":"2024-03-06T07:24:42.935849+0000","last_became_peered":"2024-03-06T07:24:42.935849+0000","last_unstale":"2024-03-06T07:24:44.995033+0000","last_undegraded":"2024-03-06T07:24:44.995033+0000","last_fullsized":"2024-03-06T07:24:44.995033+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:18.351009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.956831+0000","last_change":"2024-03-06T07:24:43.948822+0000","last_active":"2024-03-06T07:24:43.956831+0000","last_peered":"2024-03-06T07:24:43.956831+0000","last_clean":"2024-03-06T07:24:43.956831+0000","last_became_active":"2024-03-06T07:24:43.948568+0000","last_became_peered":"2024-03-06T07:24:43.948568+0000","last_unstale":"2024-03-06T07:24:43.956831+0000","last_undegraded":"2024-03-06T07:24:43.956831+0000","last_fullsized":"2024-03-06T07:24:43.956831+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:41:25.568455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940276+0000","last_change":"2024-03-06T07:24:42.934742+0000","last_active":"2024-03-06T07:24:44.940276+0000","last_peered":"2024-03-06T07:24:44.940276+0000","last_clean":"2024-03-06T07:24:44.940276+0000","last_became_active":"2024-03-06T07:24:42.934458+0000","last_became_peered":"2024-03-06T07:24:42.934458+0000","last_unstale":"2024-03-06T07:24:44.940276+0000","last_undegraded":"2024-03-06T07:24:44.940276+0000","last_fullsized":"2024-03-06T07:24:44.940276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:19.886553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":361,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.939903+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:24:44.939903+0000","last_peered":"2024-03-06T07:24:44.939903+0000","last_clean":"2024-03-06T07:24:44.939903+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:24:44.939903+0000","last_undegraded":"2024-03-06T07:24:44.939903+0000","last_fullsized":"2024-03-06T07:24:44.939903+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":1315659,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":724261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":723611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1515520,"data_stored":730281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":725271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":724205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":723659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1576960,"data_stored":1314013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:24:47.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:24:48.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9315113640 1 Processor -- start 2024-03-06T07:24:48.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9315113640 1 -- start start 2024-03-06T07:24:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9315113640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93100c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9315113640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9310158db0 0x7f931015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9315113640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9310155110 con 0x7f93100c88b0 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9315113640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93101559e0 con 0x7f9310158db0 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9315113640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93100c8f40 con 0x7f93100c5330 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.090+0000 7f9307fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f93100c88b0 0x7f93101586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55820/0 (socket says 172.21.15.26:55820) 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f9307fff640 1 -- 172.21.15.26:0/24333979 learned_addr learned my addr 172.21.15.26:0/24333979 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930f59e640 1 --2- 172.21.15.26:0/24333979 >> v2:172.21.15.117:3300/0 conn(0x7f9310158db0 0x7f931015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930ed9d640 1 --2- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93100c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930e59c640 1 -- 172.21.15.26:0/24333979 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246139790 0 0) 0x7f92f8001110 con 0x7f93100c88b0 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930e59c640 1 -- 172.21.15.26:0/24333979 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92fc003520 con 0x7f93100c88b0 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930f59e640 1 -- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 msgr2=0x7f93100c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930f59e640 1 --2- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93100c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930f59e640 1 -- 172.21.15.26:0/24333979 >> v1:172.21.15.26:6789/0 conn(0x7f93100c88b0 legacy=0x7f93101586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930f59e640 1 -- 172.21.15.26:0/24333979 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310155390 con 0x7f9310158db0 2024-03-06T07:24:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.091+0000 7f930ed9d640 1 --2- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93100c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f930f59e640 1 --2- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9310158db0 0x7f931015fa30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9308009d90 tx=0x7f930802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8890e7b12074e3ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f930e59c640 1 -- 172.21.15.26:0/24333979 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308039070 con 0x7f9310158db0 2024-03-06T07:24:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f930e59c640 1 -- 172.21.15.26:0/24333979 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930802cbc0 con 0x7f9310158db0 2024-03-06T07:24:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f930e59c640 1 -- 172.21.15.26:0/24333979 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308034440 con 0x7f9310158db0 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f9315113640 1 -- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9310158db0 msgr2=0x7f931015fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f9315113640 1 --2- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9310158db0 0x7f931015fa30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9308009d90 tx=0x7f930802be10 comp rx=0 tx=0).stop 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f9315113640 1 -- 172.21.15.26:0/24333979 shutdown_connections 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f9315113640 1 --2- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9310158db0 0x7f931015fa30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.092+0000 7f9315113640 1 --2- 172.21.15.26:0/24333979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93100c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.093+0000 7f9315113640 1 -- 172.21.15.26:0/24333979 >> 172.21.15.26:0/24333979 conn(0x7f93100ac560 msgr2=0x7f93100ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.093+0000 7f9315113640 1 -- 172.21.15.26:0/24333979 shutdown_connections 2024-03-06T07:24:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.093+0000 7f9315113640 1 -- 172.21.15.26:0/24333979 wait complete. 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 Processor -- start 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 -- start start 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93101ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93100c88b0 0x7f93101490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f931014e120 0x7f9310150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93101529d0 con 0x7f93100c88b0 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931006d090 con 0x7f931014e120 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.094+0000 7f9315113640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9310152b50 con 0x7f93100c5330 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93100c88b0 0x7f93101490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93100c88b0 0x7f93101490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35502/0 (socket says 172.21.15.26:35502) 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 -- 172.21.15.26:0/221705254 learned_addr learned my addr 172.21.15.26:0/221705254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f930f59e640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f931014e120 0x7f9310150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f930ed9d640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93101ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 -- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 msgr2=0x7f93101ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93101ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 -- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f931014e120 msgr2=0x7f9310150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f931014e120 0x7f9310150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 -- 172.21.15.26:0/221705254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93101495f0 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f930ed9d640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93101ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9307fff640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93100c88b0 0x7f93101490b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f92f800c730 tx=0x7f92f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f93077fe640 1 -- 172.21.15.26:0/221705254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8017070 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f93077fe640 1 -- 172.21.15.26:0/221705254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f80058b0 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931014ba90 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f93077fe640 1 -- 172.21.15.26:0/221705254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8005ce0 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.095+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9310150650 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.096+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931004f330 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.097+0000 7f93077fe640 1 -- 172.21.15.26:0/221705254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92f801a070 con 0x7f93100c88b0 2024-03-06T07:24:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.097+0000 7f93077fe640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f92f006b500 0x7f92f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.097+0000 7f93077fe640 1 -- 172.21.15.26:0/221705254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92f8089890 con 0x7f93100c88b0 2024-03-06T07:24:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.097+0000 7f930ed9d640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f92f006b500 0x7f92f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.098+0000 7f930ed9d640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f92f006b500 0x7f92f006d9c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f93100bdc60 tx=0x7f9300008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.106+0000 7f93077fe640 1 -- 172.21.15.26:0/221705254 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f93101495f0 con 0x7f93100c88b0 2024-03-06T07:24:48.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.376+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f931006cdc0 con 0x7f92f006b500 2024-03-06T07:24:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.382+0000 7f93077fe640 1 -- 172.21.15.26:0/221705254 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252531 (secure 0 0 0) 0x7f931006cdc0 con 0x7f92f006b500 2024-03-06T07:24:48.387 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:24:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f92f006b500 msgr2=0x7f92f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f92f006b500 0x7f92f006d9c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f93100bdc60 tx=0x7f9300008040 comp rx=0 tx=0).stop 2024-03-06T07:24:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93100c88b0 msgr2=0x7f93101490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93100c88b0 0x7f93101490b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f92f800c730 tx=0x7f92f800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 shutdown_connections 2024-03-06T07:24:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f92f006b500 0x7f92f006d9c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f931014e120 0x7f9310150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93100c88b0 0x7f93101490b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 --2- 172.21.15.26:0/221705254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93100c5330 0x7f93101ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 >> 172.21.15.26:0/221705254 conn(0x7f93100ac560 msgr2=0x7f931014f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 shutdown_connections 2024-03-06T07:24:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.385+0000 7f9315113640 1 -- 172.21.15.26:0/221705254 wait complete. 2024-03-06T07:24:48.399 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-03-06T07:24:47.088916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261440,"kb_used_data":11948,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713280,"statfs":{"total":773094113280,"available":772826398720,"internally_reserved":0,"allocated":12234752,"data_stored":6980960,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.170171"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.945152+0000","last_change":"2024-03-06T07:24:43.939874+0000","last_active":"2024-03-06T07:24:43.945152+0000","last_peered":"2024-03-06T07:24:43.945152+0000","last_clean":"2024-03-06T07:24:43.945152+0000","last_became_active":"2024-03-06T07:24:43.939600+0000","last_became_peered":"2024-03-06T07:24:43.939600+0000","last_unstale":"2024-03-06T07:24:43.945152+0000","last_undegraded":"2024-03-06T07:24:43.945152+0000","last_fullsized":"2024-03-06T07:24:43.945152+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:56.546228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949866+0000","last_change":"2024-03-06T07:24:43.944313+0000","last_active":"2024-03-06T07:24:44.949866+0000","last_peered":"2024-03-06T07:24:44.949866+0000","last_clean":"2024-03-06T07:24:44.949866+0000","last_became_active":"2024-03-06T07:24:43.943952+0000","last_became_peered":"2024-03-06T07:24:43.943952+0000","last_unstale":"2024-03-06T07:24:44.949866+0000","last_undegraded":"2024-03-06T07:24:44.949866+0000","last_fullsized":"2024-03-06T07:24:44.949866+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:18.756401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.954227+0000","last_change":"2024-03-06T07:24:43.948264+0000","last_active":"2024-03-06T07:24:43.954227+0000","last_peered":"2024-03-06T07:24:43.954227+0000","last_clean":"2024-03-06T07:24:43.954227+0000","last_became_active":"2024-03-06T07:24:43.948006+0000","last_became_peered":"2024-03-06T07:24:43.948006+0000","last_unstale":"2024-03-06T07:24:43.954227+0000","last_undegraded":"2024-03-06T07:24:43.954227+0000","last_fullsized":"2024-03-06T07:24:43.954227+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:13.675959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949744+0000","last_change":"2024-03-06T07:24:43.942587+0000","last_active":"2024-03-06T07:24:44.949744+0000","last_peered":"2024-03-06T07:24:44.949744+0000","last_clean":"2024-03-06T07:24:44.949744+0000","last_became_active":"2024-03-06T07:24:43.942291+0000","last_became_peered":"2024-03-06T07:24:43.942291+0000","last_unstale":"2024-03-06T07:24:44.949744+0000","last_undegraded":"2024-03-06T07:24:44.949744+0000","last_fullsized":"2024-03-06T07:24:44.949744+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:12.371160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995028+0000","last_change":"2024-03-06T07:24:43.939928+0000","last_active":"2024-03-06T07:24:44.995028+0000","last_peered":"2024-03-06T07:24:44.995028+0000","last_clean":"2024-03-06T07:24:44.995028+0000","last_became_active":"2024-03-06T07:24:43.939622+0000","last_became_peered":"2024-03-06T07:24:43.939622+0000","last_unstale":"2024-03-06T07:24:44.995028+0000","last_undegraded":"2024-03-06T07:24:44.995028+0000","last_fullsized":"2024-03-06T07:24:44.995028+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:05.929697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.956096+0000","last_change":"2024-03-06T07:24:43.954743+0000","last_active":"2024-03-06T07:24:43.956096+0000","last_peered":"2024-03-06T07:24:43.956096+0000","last_clean":"2024-03-06T07:24:43.956096+0000","last_became_active":"2024-03-06T07:24:43.954348+0000","last_became_peered":"2024-03-06T07:24:43.954348+0000","last_unstale":"2024-03-06T07:24:43.956096+0000","last_undegraded":"2024-03-06T07:24:43.956096+0000","last_fullsized":"2024-03-06T07:24:43.956096+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:57.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940570+0000","last_change":"2024-03-06T07:24:43.941641+0000","last_active":"2024-03-06T07:24:44.940570+0000","last_peered":"2024-03-06T07:24:44.940570+0000","last_clean":"2024-03-06T07:24:44.940570+0000","last_became_active":"2024-03-06T07:24:43.941379+0000","last_became_peered":"2024-03-06T07:24:43.941379+0000","last_unstale":"2024-03-06T07:24:44.940570+0000","last_undegraded":"2024-03-06T07:24:44.940570+0000","last_fullsized":"2024-03-06T07:24:44.940570+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:59.011549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946648+0000","last_change":"2024-03-06T07:24:43.943066+0000","last_active":"2024-03-06T07:24:44.946648+0000","last_peered":"2024-03-06T07:24:44.946648+0000","last_clean":"2024-03-06T07:24:44.946648+0000","last_became_active":"2024-03-06T07:24:43.942796+0000","last_became_peered":"2024-03-06T07:24:43.942796+0000","last_unstale":"2024-03-06T07:24:44.946648+0000","last_undegraded":"2024-03-06T07:24:44.946648+0000","last_fullsized":"2024-03-06T07:24:44.946648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:47.769348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.953522+0000","last_change":"2024-03-06T07:24:43.950547+0000","last_active":"2024-03-06T07:24:43.953522+0000","last_peered":"2024-03-06T07:24:43.953522+0000","last_clean":"2024-03-06T07:24:43.953522+0000","last_became_active":"2024-03-06T07:24:43.949947+0000","last_became_peered":"2024-03-06T07:24:43.949947+0000","last_unstale":"2024-03-06T07:24:43.953522+0000","last_undegraded":"2024-03-06T07:24:43.953522+0000","last_fullsized":"2024-03-06T07:24:43.953522+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:25.973751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941276+0000","last_change":"2024-03-06T07:24:43.946279+0000","last_active":"2024-03-06T07:24:44.941276+0000","last_peered":"2024-03-06T07:24:44.941276+0000","last_clean":"2024-03-06T07:24:44.941276+0000","last_became_active":"2024-03-06T07:24:43.945997+0000","last_became_peered":"2024-03-06T07:24:43.945997+0000","last_unstale":"2024-03-06T07:24:44.941276+0000","last_undegraded":"2024-03-06T07:24:44.941276+0000","last_fullsized":"2024-03-06T07:24:44.941276+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:35:36.815110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949780+0000","last_change":"2024-03-06T07:24:43.944418+0000","last_active":"2024-03-06T07:24:44.949780+0000","last_peered":"2024-03-06T07:24:44.949780+0000","last_clean":"2024-03-06T07:24:44.949780+0000","last_became_active":"2024-03-06T07:24:43.944094+0000","last_became_peered":"2024-03-06T07:24:43.944094+0000","last_unstale":"2024-03-06T07:24:44.949780+0000","last_undegraded":"2024-03-06T07:24:44.949780+0000","last_fullsized":"2024-03-06T07:24:44.949780+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:19.355051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944495+0000","last_change":"2024-03-06T07:24:43.940910+0000","last_active":"2024-03-06T07:24:43.944495+0000","last_peered":"2024-03-06T07:24:43.944495+0000","last_clean":"2024-03-06T07:24:43.944495+0000","last_became_active":"2024-03-06T07:24:43.940629+0000","last_became_peered":"2024-03-06T07:24:43.940629+0000","last_unstale":"2024-03-06T07:24:43.944495+0000","last_undegraded":"2024-03-06T07:24:43.944495+0000","last_fullsized":"2024-03-06T07:24:43.944495+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:39.945008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950761+0000","last_change":"2024-03-06T07:24:43.949291+0000","last_active":"2024-03-06T07:24:43.950761+0000","last_peered":"2024-03-06T07:24:43.950761+0000","last_clean":"2024-03-06T07:24:43.950761+0000","last_became_active":"2024-03-06T07:24:43.949021+0000","last_became_peered":"2024-03-06T07:24:43.949021+0000","last_unstale":"2024-03-06T07:24:43.950761+0000","last_undegraded":"2024-03-06T07:24:43.950761+0000","last_fullsized":"2024-03-06T07:24:43.950761+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:31.866021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951893+0000","last_change":"2024-03-06T07:24:43.944892+0000","last_active":"2024-03-06T07:24:43.951893+0000","last_peered":"2024-03-06T07:24:43.951893+0000","last_clean":"2024-03-06T07:24:43.951893+0000","last_became_active":"2024-03-06T07:24:43.944580+0000","last_became_peered":"2024-03-06T07:24:43.944580+0000","last_unstale":"2024-03-06T07:24:43.951893+0000","last_undegraded":"2024-03-06T07:24:43.951893+0000","last_fullsized":"2024-03-06T07:24:43.951893+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:27.155918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.952012+0000","last_change":"2024-03-06T07:24:43.944735+0000","last_active":"2024-03-06T07:24:43.952012+0000","last_peered":"2024-03-06T07:24:43.952012+0000","last_clean":"2024-03-06T07:24:43.952012+0000","last_became_active":"2024-03-06T07:24:43.944396+0000","last_became_peered":"2024-03-06T07:24:43.944396+0000","last_unstale":"2024-03-06T07:24:43.952012+0000","last_undegraded":"2024-03-06T07:24:43.952012+0000","last_fullsized":"2024-03-06T07:24:43.952012+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:54.741536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951775+0000","last_change":"2024-03-06T07:24:43.950149+0000","last_active":"2024-03-06T07:24:43.951775+0000","last_peered":"2024-03-06T07:24:43.951775+0000","last_clean":"2024-03-06T07:24:43.951775+0000","last_became_active":"2024-03-06T07:24:43.949549+0000","last_became_peered":"2024-03-06T07:24:43.949549+0000","last_unstale":"2024-03-06T07:24:43.951775+0000","last_undegraded":"2024-03-06T07:24:43.951775+0000","last_fullsized":"2024-03-06T07:24:43.951775+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:19.247439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951806+0000","last_change":"2024-03-06T07:24:43.950209+0000","last_active":"2024-03-06T07:24:43.951806+0000","last_peered":"2024-03-06T07:24:43.951806+0000","last_clean":"2024-03-06T07:24:43.951806+0000","last_became_active":"2024-03-06T07:24:43.949663+0000","last_became_peered":"2024-03-06T07:24:43.949663+0000","last_unstale":"2024-03-06T07:24:43.951806+0000","last_undegraded":"2024-03-06T07:24:43.951806+0000","last_fullsized":"2024-03-06T07:24:43.951806+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:49.531375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995223+0000","last_change":"2024-03-06T07:24:43.935701+0000","last_active":"2024-03-06T07:24:44.995223+0000","last_peered":"2024-03-06T07:24:44.995223+0000","last_clean":"2024-03-06T07:24:44.995223+0000","last_became_active":"2024-03-06T07:24:43.935380+0000","last_became_peered":"2024-03-06T07:24:43.935380+0000","last_unstale":"2024-03-06T07:24:44.995223+0000","last_undegraded":"2024-03-06T07:24:44.995223+0000","last_fullsized":"2024-03-06T07:24:44.995223+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:44.400364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940297+0000","last_change":"2024-03-06T07:24:43.934962+0000","last_active":"2024-03-06T07:24:44.940297+0000","last_peered":"2024-03-06T07:24:44.940297+0000","last_clean":"2024-03-06T07:24:44.940297+0000","last_became_active":"2024-03-06T07:24:43.934683+0000","last_became_peered":"2024-03-06T07:24:43.934683+0000","last_unstale":"2024-03-06T07:24:44.940297+0000","last_undegraded":"2024-03-06T07:24:44.940297+0000","last_fullsized":"2024-03-06T07:24:44.940297+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:22.677609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995061+0000","last_change":"2024-03-06T07:24:43.937922+0000","last_active":"2024-03-06T07:24:44.995061+0000","last_peered":"2024-03-06T07:24:44.995061+0000","last_clean":"2024-03-06T07:24:44.995061+0000","last_became_active":"2024-03-06T07:24:43.937648+0000","last_became_peered":"2024-03-06T07:24:43.937648+0000","last_unstale":"2024-03-06T07:24:44.995061+0000","last_undegraded":"2024-03-06T07:24:44.995061+0000","last_fullsized":"2024-03-06T07:24:44.995061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:55.447843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949641+0000","last_change":"2024-03-06T07:24:43.952274+0000","last_active":"2024-03-06T07:24:44.949641+0000","last_peered":"2024-03-06T07:24:44.949641+0000","last_clean":"2024-03-06T07:24:44.949641+0000","last_became_active":"2024-03-06T07:24:43.951976+0000","last_became_peered":"2024-03-06T07:24:43.951976+0000","last_unstale":"2024-03-06T07:24:44.949641+0000","last_undegraded":"2024-03-06T07:24:44.949641+0000","last_fullsized":"2024-03-06T07:24:44.949641+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:31.681793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944771+0000","last_change":"2024-03-06T07:24:43.940519+0000","last_active":"2024-03-06T07:24:43.944771+0000","last_peered":"2024-03-06T07:24:43.944771+0000","last_clean":"2024-03-06T07:24:43.944771+0000","last_became_active":"2024-03-06T07:24:43.940164+0000","last_became_peered":"2024-03-06T07:24:43.940164+0000","last_unstale":"2024-03-06T07:24:43.944771+0000","last_undegraded":"2024-03-06T07:24:43.944771+0000","last_fullsized":"2024-03-06T07:24:43.944771+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:24.881111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949596+0000","last_change":"2024-03-06T07:24:43.941775+0000","last_active":"2024-03-06T07:24:44.949596+0000","last_peered":"2024-03-06T07:24:44.949596+0000","last_clean":"2024-03-06T07:24:44.949596+0000","last_became_active":"2024-03-06T07:24:43.941487+0000","last_became_peered":"2024-03-06T07:24:43.941487+0000","last_unstale":"2024-03-06T07:24:44.949596+0000","last_undegraded":"2024-03-06T07:24:44.949596+0000","last_fullsized":"2024-03-06T07:24:44.949596+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:47.953509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951754+0000","last_change":"2024-03-06T07:24:43.949877+0000","last_active":"2024-03-06T07:24:43.951754+0000","last_peered":"2024-03-06T07:24:43.951754+0000","last_clean":"2024-03-06T07:24:43.951754+0000","last_became_active":"2024-03-06T07:24:43.949031+0000","last_became_peered":"2024-03-06T07:24:43.949031+0000","last_unstale":"2024-03-06T07:24:43.951754+0000","last_undegraded":"2024-03-06T07:24:43.951754+0000","last_fullsized":"2024-03-06T07:24:43.951754+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:36.834369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948986+0000","last_change":"2024-03-06T07:24:43.946356+0000","last_active":"2024-03-06T07:24:43.948986+0000","last_peered":"2024-03-06T07:24:43.948986+0000","last_clean":"2024-03-06T07:24:43.948986+0000","last_became_active":"2024-03-06T07:24:43.944601+0000","last_became_peered":"2024-03-06T07:24:43.944601+0000","last_unstale":"2024-03-06T07:24:43.948986+0000","last_undegraded":"2024-03-06T07:24:43.948986+0000","last_fullsized":"2024-03-06T07:24:43.948986+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:07.760345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948742+0000","last_change":"2024-03-06T07:24:43.944501+0000","last_active":"2024-03-06T07:24:43.948742+0000","last_peered":"2024-03-06T07:24:43.948742+0000","last_clean":"2024-03-06T07:24:43.948742+0000","last_became_active":"2024-03-06T07:24:43.944229+0000","last_became_peered":"2024-03-06T07:24:43.944229+0000","last_unstale":"2024-03-06T07:24:43.948742+0000","last_undegraded":"2024-03-06T07:24:43.948742+0000","last_fullsized":"2024-03-06T07:24:43.948742+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:48.383289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940459+0000","last_change":"2024-03-06T07:24:43.953312+0000","last_active":"2024-03-06T07:24:44.940459+0000","last_peered":"2024-03-06T07:24:44.940459+0000","last_clean":"2024-03-06T07:24:44.940459+0000","last_became_active":"2024-03-06T07:24:43.952387+0000","last_became_peered":"2024-03-06T07:24:43.952387+0000","last_unstale":"2024-03-06T07:24:44.940459+0000","last_undegraded":"2024-03-06T07:24:44.940459+0000","last_fullsized":"2024-03-06T07:24:44.940459+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:44.400364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941208+0000","last_change":"2024-03-06T07:24:43.947437+0000","last_active":"2024-03-06T07:24:44.941208+0000","last_peered":"2024-03-06T07:24:44.941208+0000","last_clean":"2024-03-06T07:24:44.941208+0000","last_became_active":"2024-03-06T07:24:43.946787+0000","last_became_peered":"2024-03-06T07:24:43.946787+0000","last_unstale":"2024-03-06T07:24:44.941208+0000","last_undegraded":"2024-03-06T07:24:44.941208+0000","last_fullsized":"2024-03-06T07:24:44.941208+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:15.119383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.957166+0000","last_change":"2024-03-06T07:24:43.953413+0000","last_active":"2024-03-06T07:24:43.957166+0000","last_peered":"2024-03-06T07:24:43.957166+0000","last_clean":"2024-03-06T07:24:43.957166+0000","last_became_active":"2024-03-06T07:24:43.953163+0000","last_became_peered":"2024-03-06T07:24:43.953163+0000","last_unstale":"2024-03-06T07:24:43.957166+0000","last_undegraded":"2024-03-06T07:24:43.957166+0000","last_fullsized":"2024-03-06T07:24:43.957166+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:42.298185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948859+0000","last_change":"2024-03-06T07:24:43.944557+0000","last_active":"2024-03-06T07:24:43.948859+0000","last_peered":"2024-03-06T07:24:43.948859+0000","last_clean":"2024-03-06T07:24:43.948859+0000","last_became_active":"2024-03-06T07:24:43.944255+0000","last_became_peered":"2024-03-06T07:24:43.944255+0000","last_unstale":"2024-03-06T07:24:43.948859+0000","last_undegraded":"2024-03-06T07:24:43.948859+0000","last_fullsized":"2024-03-06T07:24:43.948859+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:32:11.860704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946805+0000","last_change":"2024-03-06T07:24:43.954117+0000","last_active":"2024-03-06T07:24:44.946805+0000","last_peered":"2024-03-06T07:24:44.946805+0000","last_clean":"2024-03-06T07:24:44.946805+0000","last_became_active":"2024-03-06T07:24:43.953742+0000","last_became_peered":"2024-03-06T07:24:43.953742+0000","last_unstale":"2024-03-06T07:24:44.946805+0000","last_undegraded":"2024-03-06T07:24:44.946805+0000","last_fullsized":"2024-03-06T07:24:44.946805+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:59.661391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941049+0000","last_change":"2024-03-06T07:24:43.953105+0000","last_active":"2024-03-06T07:24:44.941049+0000","last_peered":"2024-03-06T07:24:44.941049+0000","last_clean":"2024-03-06T07:24:44.941049+0000","last_became_active":"2024-03-06T07:24:43.952816+0000","last_became_peered":"2024-03-06T07:24:43.952816+0000","last_unstale":"2024-03-06T07:24:44.941049+0000","last_undegraded":"2024-03-06T07:24:44.941049+0000","last_fullsized":"2024-03-06T07:24:44.941049+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:42.370886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946713+0000","last_change":"2024-03-06T07:24:43.931794+0000","last_active":"2024-03-06T07:24:44.946713+0000","last_peered":"2024-03-06T07:24:44.946713+0000","last_clean":"2024-03-06T07:24:44.946713+0000","last_became_active":"2024-03-06T07:24:43.930959+0000","last_became_peered":"2024-03-06T07:24:43.930959+0000","last_unstale":"2024-03-06T07:24:44.946713+0000","last_undegraded":"2024-03-06T07:24:44.946713+0000","last_fullsized":"2024-03-06T07:24:44.946713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:49.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"120'4","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.959962+0000","last_change":"2024-03-06T07:24:42.930288+0000","last_active":"2024-03-06T07:24:43.959962+0000","last_peered":"2024-03-06T07:24:43.959962+0000","last_clean":"2024-03-06T07:24:43.959962+0000","last_became_active":"2024-03-06T07:24:42.930016+0000","last_became_peered":"2024-03-06T07:24:42.930016+0000","last_unstale":"2024-03-06T07:24:43.959962+0000","last_undegraded":"2024-03-06T07:24:43.959962+0000","last_fullsized":"2024-03-06T07:24:43.959962+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:06.327089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944195+0000","last_change":"2024-03-06T07:24:43.938001+0000","last_active":"2024-03-06T07:24:43.944195+0000","last_peered":"2024-03-06T07:24:43.944195+0000","last_clean":"2024-03-06T07:24:43.944195+0000","last_became_active":"2024-03-06T07:24:43.937712+0000","last_became_peered":"2024-03-06T07:24:43.937712+0000","last_unstale":"2024-03-06T07:24:43.944195+0000","last_undegraded":"2024-03-06T07:24:43.944195+0000","last_fullsized":"2024-03-06T07:24:43.944195+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:41.989637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995322+0000","last_change":"2024-03-06T07:24:42.933650+0000","last_active":"2024-03-06T07:24:44.995322+0000","last_peered":"2024-03-06T07:24:44.995322+0000","last_clean":"2024-03-06T07:24:44.995322+0000","last_became_active":"2024-03-06T07:24:42.932924+0000","last_became_peered":"2024-03-06T07:24:42.932924+0000","last_unstale":"2024-03-06T07:24:44.995322+0000","last_undegraded":"2024-03-06T07:24:44.995322+0000","last_fullsized":"2024-03-06T07:24:44.995322+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:19.886553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949715+0000","last_change":"2024-03-06T07:24:43.941880+0000","last_active":"2024-03-06T07:24:44.949715+0000","last_peered":"2024-03-06T07:24:44.949715+0000","last_clean":"2024-03-06T07:24:44.949715+0000","last_became_active":"2024-03-06T07:24:43.941463+0000","last_became_peered":"2024-03-06T07:24:43.941463+0000","last_unstale":"2024-03-06T07:24:44.949715+0000","last_undegraded":"2024-03-06T07:24:44.949715+0000","last_fullsized":"2024-03-06T07:24:44.949715+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:45.762532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940572+0000","last_change":"2024-03-06T07:24:42.935712+0000","last_active":"2024-03-06T07:24:44.940572+0000","last_peered":"2024-03-06T07:24:44.940572+0000","last_clean":"2024-03-06T07:24:44.940572+0000","last_became_active":"2024-03-06T07:24:42.935432+0000","last_became_peered":"2024-03-06T07:24:42.935432+0000","last_unstale":"2024-03-06T07:24:44.940572+0000","last_undegraded":"2024-03-06T07:24:44.940572+0000","last_fullsized":"2024-03-06T07:24:44.940572+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:11.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.944161+0000","last_change":"2024-03-06T07:24:43.941772+0000","last_active":"2024-03-06T07:24:43.944161+0000","last_peered":"2024-03-06T07:24:43.944161+0000","last_clean":"2024-03-06T07:24:43.944161+0000","last_became_active":"2024-03-06T07:24:43.941459+0000","last_became_peered":"2024-03-06T07:24:43.941459+0000","last_unstale":"2024-03-06T07:24:43.944161+0000","last_undegraded":"2024-03-06T07:24:43.944161+0000","last_fullsized":"2024-03-06T07:24:43.944161+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:13.255227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940760+0000","last_change":"2024-03-06T07:24:42.928151+0000","last_active":"2024-03-06T07:24:44.940760+0000","last_peered":"2024-03-06T07:24:44.940760+0000","last_clean":"2024-03-06T07:24:44.940760+0000","last_became_active":"2024-03-06T07:24:42.927844+0000","last_became_peered":"2024-03-06T07:24:42.927844+0000","last_unstale":"2024-03-06T07:24:44.940760+0000","last_undegraded":"2024-03-06T07:24:44.940760+0000","last_fullsized":"2024-03-06T07:24:44.940760+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:17.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946952+0000","last_change":"2024-03-06T07:24:43.954506+0000","last_active":"2024-03-06T07:24:44.946952+0000","last_peered":"2024-03-06T07:24:44.946952+0000","last_clean":"2024-03-06T07:24:44.946952+0000","last_became_active":"2024-03-06T07:24:43.953630+0000","last_became_peered":"2024-03-06T07:24:43.953630+0000","last_unstale":"2024-03-06T07:24:44.946952+0000","last_undegraded":"2024-03-06T07:24:44.946952+0000","last_fullsized":"2024-03-06T07:24:44.946952+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:23:31.866021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946987+0000","last_change":"2024-03-06T07:24:42.935678+0000","last_active":"2024-03-06T07:24:44.946987+0000","last_peered":"2024-03-06T07:24:44.946987+0000","last_clean":"2024-03-06T07:24:44.946987+0000","last_became_active":"2024-03-06T07:24:42.935426+0000","last_became_peered":"2024-03-06T07:24:42.935426+0000","last_unstale":"2024-03-06T07:24:44.946987+0000","last_undegraded":"2024-03-06T07:24:44.946987+0000","last_fullsized":"2024-03-06T07:24:44.946987+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:47.379247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.947081+0000","last_change":"2024-03-06T07:24:43.940354+0000","last_active":"2024-03-06T07:24:44.947081+0000","last_peered":"2024-03-06T07:24:44.947081+0000","last_clean":"2024-03-06T07:24:44.947081+0000","last_became_active":"2024-03-06T07:24:43.940073+0000","last_became_peered":"2024-03-06T07:24:43.940073+0000","last_unstale":"2024-03-06T07:24:44.947081+0000","last_undegraded":"2024-03-06T07:24:44.947081+0000","last_fullsized":"2024-03-06T07:24:44.947081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:36.834369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.994927+0000","last_change":"2024-03-06T07:24:42.934439+0000","last_active":"2024-03-06T07:24:44.994927+0000","last_peered":"2024-03-06T07:24:44.994927+0000","last_clean":"2024-03-06T07:24:44.994927+0000","last_became_active":"2024-03-06T07:24:42.934180+0000","last_became_peered":"2024-03-06T07:24:42.934180+0000","last_unstale":"2024-03-06T07:24:44.994927+0000","last_undegraded":"2024-03-06T07:24:44.994927+0000","last_fullsized":"2024-03-06T07:24:44.994927+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:44.758491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995214+0000","last_change":"2024-03-06T07:24:43.940027+0000","last_active":"2024-03-06T07:24:44.995214+0000","last_peered":"2024-03-06T07:24:44.995214+0000","last_clean":"2024-03-06T07:24:44.995214+0000","last_became_active":"2024-03-06T07:24:43.939721+0000","last_became_peered":"2024-03-06T07:24:43.939721+0000","last_unstale":"2024-03-06T07:24:44.995214+0000","last_undegraded":"2024-03-06T07:24:44.995214+0000","last_fullsized":"2024-03-06T07:24:44.995214+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:22.677609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.933397+0000","last_change":"2024-03-06T07:24:42.939720+0000","last_active":"2024-03-06T07:24:43.933397+0000","last_peered":"2024-03-06T07:24:43.933397+0000","last_clean":"2024-03-06T07:24:43.933397+0000","last_became_active":"2024-03-06T07:24:42.937175+0000","last_became_peered":"2024-03-06T07:24:42.937175+0000","last_unstale":"2024-03-06T07:24:43.933397+0000","last_undegraded":"2024-03-06T07:24:43.933397+0000","last_fullsized":"2024-03-06T07:24:43.933397+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:12.671918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940548+0000","last_change":"2024-03-06T07:24:43.952850+0000","last_active":"2024-03-06T07:24:44.940548+0000","last_peered":"2024-03-06T07:24:44.940548+0000","last_clean":"2024-03-06T07:24:44.940548+0000","last_became_active":"2024-03-06T07:24:43.952590+0000","last_became_peered":"2024-03-06T07:24:43.952590+0000","last_unstale":"2024-03-06T07:24:44.940548+0000","last_undegraded":"2024-03-06T07:24:44.940548+0000","last_fullsized":"2024-03-06T07:24:44.940548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:32.291753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.932470+0000","last_change":"2024-03-06T07:24:42.937330+0000","last_active":"2024-03-06T07:24:43.932470+0000","last_peered":"2024-03-06T07:24:43.932470+0000","last_clean":"2024-03-06T07:24:43.932470+0000","last_became_active":"2024-03-06T07:24:42.937053+0000","last_became_peered":"2024-03-06T07:24:42.937053+0000","last_unstale":"2024-03-06T07:24:43.932470+0000","last_undegraded":"2024-03-06T07:24:43.932470+0000","last_fullsized":"2024-03-06T07:24:43.932470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:03.404751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951973+0000","last_change":"2024-03-06T07:24:43.945377+0000","last_active":"2024-03-06T07:24:43.951973+0000","last_peered":"2024-03-06T07:24:43.951973+0000","last_clean":"2024-03-06T07:24:43.951973+0000","last_became_active":"2024-03-06T07:24:43.943817+0000","last_became_peered":"2024-03-06T07:24:43.943817+0000","last_unstale":"2024-03-06T07:24:43.951973+0000","last_undegraded":"2024-03-06T07:24:43.951973+0000","last_fullsized":"2024-03-06T07:24:43.951973+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:49:55.462660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949848+0000","last_change":"2024-03-06T07:24:42.938610+0000","last_active":"2024-03-06T07:24:44.949848+0000","last_peered":"2024-03-06T07:24:44.949848+0000","last_clean":"2024-03-06T07:24:44.949848+0000","last_became_active":"2024-03-06T07:24:42.938357+0000","last_became_peered":"2024-03-06T07:24:42.938357+0000","last_unstale":"2024-03-06T07:24:44.949848+0000","last_undegraded":"2024-03-06T07:24:44.949848+0000","last_fullsized":"2024-03-06T07:24:44.949848+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:48.527334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.953487+0000","last_change":"2024-03-06T07:24:43.950966+0000","last_active":"2024-03-06T07:24:43.953487+0000","last_peered":"2024-03-06T07:24:43.953487+0000","last_clean":"2024-03-06T07:24:43.953487+0000","last_became_active":"2024-03-06T07:24:43.950482+0000","last_became_peered":"2024-03-06T07:24:43.950482+0000","last_unstale":"2024-03-06T07:24:43.953487+0000","last_undegraded":"2024-03-06T07:24:43.953487+0000","last_fullsized":"2024-03-06T07:24:43.953487+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:24.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.929674+0000","last_change":"2024-03-06T07:24:42.936304+0000","last_active":"2024-03-06T07:24:43.929674+0000","last_peered":"2024-03-06T07:24:43.929674+0000","last_clean":"2024-03-06T07:24:43.929674+0000","last_became_active":"2024-03-06T07:24:42.935985+0000","last_became_peered":"2024-03-06T07:24:42.935985+0000","last_unstale":"2024-03-06T07:24:43.929674+0000","last_undegraded":"2024-03-06T07:24:43.929674+0000","last_fullsized":"2024-03-06T07:24:43.929674+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:10.960646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948879+0000","last_change":"2024-03-06T07:24:43.944575+0000","last_active":"2024-03-06T07:24:43.948879+0000","last_peered":"2024-03-06T07:24:43.948879+0000","last_clean":"2024-03-06T07:24:43.948879+0000","last_became_active":"2024-03-06T07:24:43.942026+0000","last_became_peered":"2024-03-06T07:24:43.942026+0000","last_unstale":"2024-03-06T07:24:43.948879+0000","last_undegraded":"2024-03-06T07:24:43.948879+0000","last_fullsized":"2024-03-06T07:24:43.948879+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:01.126798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940684+0000","last_change":"2024-03-06T07:24:42.926149+0000","last_active":"2024-03-06T07:24:44.940684+0000","last_peered":"2024-03-06T07:24:44.940684+0000","last_clean":"2024-03-06T07:24:44.940684+0000","last_became_active":"2024-03-06T07:24:42.925859+0000","last_became_peered":"2024-03-06T07:24:42.925859+0000","last_unstale":"2024-03-06T07:24:44.940684+0000","last_undegraded":"2024-03-06T07:24:44.940684+0000","last_fullsized":"2024-03-06T07:24:44.940684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:18.351009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.947874+0000","last_change":"2024-03-06T07:24:43.941527+0000","last_active":"2024-03-06T07:24:43.947874+0000","last_peered":"2024-03-06T07:24:43.947874+0000","last_clean":"2024-03-06T07:24:43.947874+0000","last_became_active":"2024-03-06T07:24:43.940197+0000","last_became_peered":"2024-03-06T07:24:43.940197+0000","last_unstale":"2024-03-06T07:24:43.947874+0000","last_undegraded":"2024-03-06T07:24:43.947874+0000","last_fullsized":"2024-03-06T07:24:43.947874+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:18.041541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940726+0000","last_change":"2024-03-06T07:24:42.931361+0000","last_active":"2024-03-06T07:24:44.940726+0000","last_peered":"2024-03-06T07:24:44.940726+0000","last_clean":"2024-03-06T07:24:44.940726+0000","last_became_active":"2024-03-06T07:24:42.931054+0000","last_became_peered":"2024-03-06T07:24:42.931054+0000","last_unstale":"2024-03-06T07:24:44.940726+0000","last_undegraded":"2024-03-06T07:24:44.940726+0000","last_fullsized":"2024-03-06T07:24:44.940726+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:21:44.758491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941238+0000","last_change":"2024-03-06T07:24:43.947680+0000","last_active":"2024-03-06T07:24:44.941238+0000","last_peered":"2024-03-06T07:24:44.941238+0000","last_clean":"2024-03-06T07:24:44.941238+0000","last_became_active":"2024-03-06T07:24:43.947149+0000","last_became_peered":"2024-03-06T07:24:43.947149+0000","last_unstale":"2024-03-06T07:24:44.941238+0000","last_undegraded":"2024-03-06T07:24:44.941238+0000","last_fullsized":"2024-03-06T07:24:44.941238+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:54.442533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"120'2","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.958926+0000","last_change":"2024-03-06T07:24:42.935511+0000","last_active":"2024-03-06T07:24:43.958926+0000","last_peered":"2024-03-06T07:24:43.958926+0000","last_clean":"2024-03-06T07:24:43.958926+0000","last_became_active":"2024-03-06T07:24:42.935241+0000","last_became_peered":"2024-03-06T07:24:42.935241+0000","last_unstale":"2024-03-06T07:24:43.958926+0000","last_undegraded":"2024-03-06T07:24:43.958926+0000","last_fullsized":"2024-03-06T07:24:43.958926+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:44:39.542423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940598+0000","last_change":"2024-03-06T07:24:43.942292+0000","last_active":"2024-03-06T07:24:44.940598+0000","last_peered":"2024-03-06T07:24:44.940598+0000","last_clean":"2024-03-06T07:24:44.940598+0000","last_became_active":"2024-03-06T07:24:43.942036+0000","last_became_peered":"2024-03-06T07:24:43.942036+0000","last_unstale":"2024-03-06T07:24:44.940598+0000","last_undegraded":"2024-03-06T07:24:44.940598+0000","last_fullsized":"2024-03-06T07:24:44.940598+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:11:51.018259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.932344+0000","last_change":"2024-03-06T07:24:42.931721+0000","last_active":"2024-03-06T07:24:43.932344+0000","last_peered":"2024-03-06T07:24:43.932344+0000","last_clean":"2024-03-06T07:24:43.932344+0000","last_became_active":"2024-03-06T07:24:42.931437+0000","last_became_peered":"2024-03-06T07:24:42.931437+0000","last_unstale":"2024-03-06T07:24:43.932344+0000","last_undegraded":"2024-03-06T07:24:43.932344+0000","last_fullsized":"2024-03-06T07:24:43.932344+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:30.506558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995322+0000","last_change":"2024-03-06T07:24:43.957348+0000","last_active":"2024-03-06T07:24:44.995322+0000","last_peered":"2024-03-06T07:24:44.995322+0000","last_clean":"2024-03-06T07:24:44.995322+0000","last_became_active":"2024-03-06T07:24:43.957049+0000","last_became_peered":"2024-03-06T07:24:43.957049+0000","last_unstale":"2024-03-06T07:24:44.995322+0000","last_undegraded":"2024-03-06T07:24:44.995322+0000","last_fullsized":"2024-03-06T07:24:44.995322+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:32.291753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"120'8","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.960563+0000","last_change":"2024-03-06T07:24:42.932357+0000","last_active":"2024-03-06T07:24:43.960563+0000","last_peered":"2024-03-06T07:24:43.960563+0000","last_clean":"2024-03-06T07:24:43.960563+0000","last_became_active":"2024-03-06T07:24:42.931427+0000","last_became_peered":"2024-03-06T07:24:42.931427+0000","last_unstale":"2024-03-06T07:24:43.960563+0000","last_undegraded":"2024-03-06T07:24:43.960563+0000","last_fullsized":"2024-03-06T07:24:43.960563+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:41.294144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.948769+0000","last_change":"2024-03-06T07:24:43.943185+0000","last_active":"2024-03-06T07:24:43.948769+0000","last_peered":"2024-03-06T07:24:43.948769+0000","last_clean":"2024-03-06T07:24:43.948769+0000","last_became_active":"2024-03-06T07:24:43.942874+0000","last_became_peered":"2024-03-06T07:24:43.942874+0000","last_unstale":"2024-03-06T07:24:43.948769+0000","last_undegraded":"2024-03-06T07:24:43.948769+0000","last_fullsized":"2024-03-06T07:24:43.948769+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:27.618265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949956+0000","last_change":"2024-03-06T07:24:42.934663+0000","last_active":"2024-03-06T07:24:44.949956+0000","last_peered":"2024-03-06T07:24:44.949956+0000","last_clean":"2024-03-06T07:24:44.949956+0000","last_became_active":"2024-03-06T07:24:42.934390+0000","last_became_peered":"2024-03-06T07:24:42.934390+0000","last_unstale":"2024-03-06T07:24:44.949956+0000","last_undegraded":"2024-03-06T07:24:44.949956+0000","last_fullsized":"2024-03-06T07:24:44.949956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:18.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940663+0000","last_change":"2024-03-06T07:24:43.935478+0000","last_active":"2024-03-06T07:24:44.940663+0000","last_peered":"2024-03-06T07:24:44.940663+0000","last_clean":"2024-03-06T07:24:44.940663+0000","last_became_active":"2024-03-06T07:24:43.935211+0000","last_became_peered":"2024-03-06T07:24:43.935211+0000","last_unstale":"2024-03-06T07:24:44.940663+0000","last_undegraded":"2024-03-06T07:24:44.940663+0000","last_fullsized":"2024-03-06T07:24:44.940663+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:15.352025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"120'2","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.960053+0000","last_change":"2024-03-06T07:24:42.937923+0000","last_active":"2024-03-06T07:24:43.960053+0000","last_peered":"2024-03-06T07:24:43.960053+0000","last_clean":"2024-03-06T07:24:43.960053+0000","last_became_active":"2024-03-06T07:24:42.937634+0000","last_became_peered":"2024-03-06T07:24:42.937634+0000","last_unstale":"2024-03-06T07:24:43.960053+0000","last_undegraded":"2024-03-06T07:24:43.960053+0000","last_fullsized":"2024-03-06T07:24:43.960053+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:29.582696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940295+0000","last_change":"2024-03-06T07:24:43.952266+0000","last_active":"2024-03-06T07:24:44.940295+0000","last_peered":"2024-03-06T07:24:44.940295+0000","last_clean":"2024-03-06T07:24:44.940295+0000","last_became_active":"2024-03-06T07:24:43.951943+0000","last_became_peered":"2024-03-06T07:24:43.951943+0000","last_unstale":"2024-03-06T07:24:44.940295+0000","last_undegraded":"2024-03-06T07:24:44.940295+0000","last_fullsized":"2024-03-06T07:24:44.940295+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:38.370864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"120'1","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.958165+0000","last_change":"2024-03-06T07:24:42.940039+0000","last_active":"2024-03-06T07:24:43.958165+0000","last_peered":"2024-03-06T07:24:43.958165+0000","last_clean":"2024-03-06T07:24:43.958165+0000","last_became_active":"2024-03-06T07:24:42.939533+0000","last_became_peered":"2024-03-06T07:24:42.939533+0000","last_unstale":"2024-03-06T07:24:43.958165+0000","last_undegraded":"2024-03-06T07:24:43.958165+0000","last_fullsized":"2024-03-06T07:24:43.958165+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:06.756303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941166+0000","last_change":"2024-03-06T07:24:43.944803+0000","last_active":"2024-03-06T07:24:44.941166+0000","last_peered":"2024-03-06T07:24:44.941166+0000","last_clean":"2024-03-06T07:24:44.941166+0000","last_became_active":"2024-03-06T07:24:43.944536+0000","last_became_peered":"2024-03-06T07:24:43.944536+0000","last_unstale":"2024-03-06T07:24:44.941166+0000","last_undegraded":"2024-03-06T07:24:44.941166+0000","last_fullsized":"2024-03-06T07:24:44.941166+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:59:46.764059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995197+0000","last_change":"2024-03-06T07:24:42.929535+0000","last_active":"2024-03-06T07:24:44.995197+0000","last_peered":"2024-03-06T07:24:44.995197+0000","last_clean":"2024-03-06T07:24:44.995197+0000","last_became_active":"2024-03-06T07:24:42.929133+0000","last_became_peered":"2024-03-06T07:24:42.929133+0000","last_unstale":"2024-03-06T07:24:44.995197+0000","last_undegraded":"2024-03-06T07:24:44.995197+0000","last_fullsized":"2024-03-06T07:24:44.995197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:11.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995086+0000","last_change":"2024-03-06T07:24:43.945219+0000","last_active":"2024-03-06T07:24:44.995086+0000","last_peered":"2024-03-06T07:24:44.995086+0000","last_clean":"2024-03-06T07:24:44.995086+0000","last_became_active":"2024-03-06T07:24:43.944840+0000","last_became_peered":"2024-03-06T07:24:43.944840+0000","last_unstale":"2024-03-06T07:24:44.995086+0000","last_undegraded":"2024-03-06T07:24:44.995086+0000","last_fullsized":"2024-03-06T07:24:44.995086+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:38.370864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.946987+0000","last_change":"2024-03-06T07:24:42.932533+0000","last_active":"2024-03-06T07:24:44.946987+0000","last_peered":"2024-03-06T07:24:44.946987+0000","last_clean":"2024-03-06T07:24:44.946987+0000","last_became_active":"2024-03-06T07:24:42.932245+0000","last_became_peered":"2024-03-06T07:24:42.932245+0000","last_unstale":"2024-03-06T07:24:44.946987+0000","last_undegraded":"2024-03-06T07:24:44.946987+0000","last_fullsized":"2024-03-06T07:24:44.946987+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:00.122757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.951937+0000","last_change":"2024-03-06T07:24:43.944929+0000","last_active":"2024-03-06T07:24:43.951937+0000","last_peered":"2024-03-06T07:24:43.951937+0000","last_clean":"2024-03-06T07:24:43.951937+0000","last_became_active":"2024-03-06T07:24:43.944613+0000","last_became_peered":"2024-03-06T07:24:43.944613+0000","last_unstale":"2024-03-06T07:24:43.951937+0000","last_undegraded":"2024-03-06T07:24:43.951937+0000","last_fullsized":"2024-03-06T07:24:43.951937+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:02.290556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.930358+0000","last_change":"2024-03-06T07:24:42.932737+0000","last_active":"2024-03-06T07:24:43.930358+0000","last_peered":"2024-03-06T07:24:43.930358+0000","last_clean":"2024-03-06T07:24:43.930358+0000","last_became_active":"2024-03-06T07:24:42.932456+0000","last_became_peered":"2024-03-06T07:24:42.932456+0000","last_unstale":"2024-03-06T07:24:43.930358+0000","last_undegraded":"2024-03-06T07:24:43.930358+0000","last_fullsized":"2024-03-06T07:24:43.930358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:45.066971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995360+0000","last_change":"2024-03-06T07:24:43.945878+0000","last_active":"2024-03-06T07:24:44.995360+0000","last_peered":"2024-03-06T07:24:44.995360+0000","last_clean":"2024-03-06T07:24:44.995360+0000","last_became_active":"2024-03-06T07:24:43.945537+0000","last_became_peered":"2024-03-06T07:24:43.945537+0000","last_unstale":"2024-03-06T07:24:44.995360+0000","last_undegraded":"2024-03-06T07:24:44.995360+0000","last_fullsized":"2024-03-06T07:24:44.995360+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:59.011549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940677+0000","last_change":"2024-03-06T07:24:42.929247+0000","last_active":"2024-03-06T07:24:44.940677+0000","last_peered":"2024-03-06T07:24:44.940677+0000","last_clean":"2024-03-06T07:24:44.940677+0000","last_became_active":"2024-03-06T07:24:42.928944+0000","last_became_peered":"2024-03-06T07:24:42.928944+0000","last_unstale":"2024-03-06T07:24:44.940677+0000","last_undegraded":"2024-03-06T07:24:44.940677+0000","last_fullsized":"2024-03-06T07:24:44.940677+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:30.677752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949851+0000","last_change":"2024-03-06T07:24:43.944324+0000","last_active":"2024-03-06T07:24:44.949851+0000","last_peered":"2024-03-06T07:24:44.949851+0000","last_clean":"2024-03-06T07:24:44.949851+0000","last_became_active":"2024-03-06T07:24:43.944029+0000","last_became_peered":"2024-03-06T07:24:43.944029+0000","last_unstale":"2024-03-06T07:24:44.949851+0000","last_undegraded":"2024-03-06T07:24:44.949851+0000","last_fullsized":"2024-03-06T07:24:44.949851+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:20.890594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.934368+0000","last_change":"2024-03-06T07:24:42.940607+0000","last_active":"2024-03-06T07:24:43.934368+0000","last_peered":"2024-03-06T07:24:43.934368+0000","last_clean":"2024-03-06T07:24:43.934368+0000","last_became_active":"2024-03-06T07:24:42.940329+0000","last_became_peered":"2024-03-06T07:24:42.940329+0000","last_unstale":"2024-03-06T07:24:43.934368+0000","last_undegraded":"2024-03-06T07:24:43.934368+0000","last_fullsized":"2024-03-06T07:24:43.934368+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:37:05.544366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.994979+0000","last_change":"2024-03-06T07:24:43.938750+0000","last_active":"2024-03-06T07:24:44.994979+0000","last_peered":"2024-03-06T07:24:44.994979+0000","last_clean":"2024-03-06T07:24:44.994979+0000","last_became_active":"2024-03-06T07:24:43.938457+0000","last_became_peered":"2024-03-06T07:24:43.938457+0000","last_unstale":"2024-03-06T07:24:44.994979+0000","last_undegraded":"2024-03-06T07:24:44.994979+0000","last_fullsized":"2024-03-06T07:24:44.994979+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:34:31.367107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"120'2","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.958710+0000","last_change":"2024-03-06T07:24:42.931744+0000","last_active":"2024-03-06T07:24:43.958710+0000","last_peered":"2024-03-06T07:24:43.958710+0000","last_clean":"2024-03-06T07:24:43.958710+0000","last_became_active":"2024-03-06T07:24:42.931427+0000","last_became_peered":"2024-03-06T07:24:42.931427+0000","last_unstale":"2024-03-06T07:24:43.958710+0000","last_undegraded":"2024-03-06T07:24:43.958710+0000","last_fullsized":"2024-03-06T07:24:43.958710+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:39.994806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.947900+0000","last_change":"2024-03-06T07:24:43.942131+0000","last_active":"2024-03-06T07:24:43.947900+0000","last_peered":"2024-03-06T07:24:43.947900+0000","last_clean":"2024-03-06T07:24:43.947900+0000","last_became_active":"2024-03-06T07:24:43.940065+0000","last_became_peered":"2024-03-06T07:24:43.940065+0000","last_unstale":"2024-03-06T07:24:43.947900+0000","last_undegraded":"2024-03-06T07:24:43.947900+0000","last_fullsized":"2024-03-06T07:24:43.947900+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:49.298741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949009+0000","last_change":"2024-03-06T07:24:42.934734+0000","last_active":"2024-03-06T07:24:44.949009+0000","last_peered":"2024-03-06T07:24:44.949009+0000","last_clean":"2024-03-06T07:24:44.949009+0000","last_became_active":"2024-03-06T07:24:42.934472+0000","last_became_peered":"2024-03-06T07:24:42.934472+0000","last_unstale":"2024-03-06T07:24:44.949009+0000","last_undegraded":"2024-03-06T07:24:44.949009+0000","last_fullsized":"2024-03-06T07:24:44.949009+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:56:06.756303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940133+0000","last_change":"2024-03-06T07:24:43.947576+0000","last_active":"2024-03-06T07:24:44.940133+0000","last_peered":"2024-03-06T07:24:44.940133+0000","last_clean":"2024-03-06T07:24:44.940133+0000","last_became_active":"2024-03-06T07:24:43.947301+0000","last_became_peered":"2024-03-06T07:24:43.947301+0000","last_unstale":"2024-03-06T07:24:44.940133+0000","last_undegraded":"2024-03-06T07:24:44.940133+0000","last_fullsized":"2024-03-06T07:24:44.940133+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:55.447843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.947108+0000","last_change":"2024-03-06T07:24:42.936487+0000","last_active":"2024-03-06T07:24:44.947108+0000","last_peered":"2024-03-06T07:24:44.947108+0000","last_clean":"2024-03-06T07:24:44.947108+0000","last_became_active":"2024-03-06T07:24:42.936207+0000","last_became_peered":"2024-03-06T07:24:42.936207+0000","last_unstale":"2024-03-06T07:24:44.947108+0000","last_undegraded":"2024-03-06T07:24:44.947108+0000","last_fullsized":"2024-03-06T07:24:44.947108+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:32:10.856663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.941211+0000","last_change":"2024-03-06T07:24:43.944204+0000","last_active":"2024-03-06T07:24:44.941211+0000","last_peered":"2024-03-06T07:24:44.941211+0000","last_clean":"2024-03-06T07:24:44.941211+0000","last_became_active":"2024-03-06T07:24:43.943939+0000","last_became_peered":"2024-03-06T07:24:43.943939+0000","last_unstale":"2024-03-06T07:24:44.941211+0000","last_undegraded":"2024-03-06T07:24:44.941211+0000","last_fullsized":"2024-03-06T07:24:44.941211+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:48.361482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.935393+0000","last_change":"2024-03-06T07:24:42.936217+0000","last_active":"2024-03-06T07:24:43.935393+0000","last_peered":"2024-03-06T07:24:43.935393+0000","last_clean":"2024-03-06T07:24:43.935393+0000","last_became_active":"2024-03-06T07:24:42.935902+0000","last_became_peered":"2024-03-06T07:24:42.935902+0000","last_unstale":"2024-03-06T07:24:43.935393+0000","last_undegraded":"2024-03-06T07:24:43.935393+0000","last_fullsized":"2024-03-06T07:24:43.935393+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:32.177158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950758+0000","last_change":"2024-03-06T07:24:43.948449+0000","last_active":"2024-03-06T07:24:43.950758+0000","last_peered":"2024-03-06T07:24:43.950758+0000","last_clean":"2024-03-06T07:24:43.950758+0000","last_became_active":"2024-03-06T07:24:43.948156+0000","last_became_peered":"2024-03-06T07:24:43.948156+0000","last_unstale":"2024-03-06T07:24:43.950758+0000","last_undegraded":"2024-03-06T07:24:43.950758+0000","last_fullsized":"2024-03-06T07:24:43.950758+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:49.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.949676+0000","last_change":"2024-03-06T07:24:42.935809+0000","last_active":"2024-03-06T07:24:44.949676+0000","last_peered":"2024-03-06T07:24:44.949676+0000","last_clean":"2024-03-06T07:24:44.949676+0000","last_became_active":"2024-03-06T07:24:42.935540+0000","last_became_peered":"2024-03-06T07:24:42.935540+0000","last_unstale":"2024-03-06T07:24:44.949676+0000","last_undegraded":"2024-03-06T07:24:44.949676+0000","last_fullsized":"2024-03-06T07:24:44.949676+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:35.830328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950786+0000","last_change":"2024-03-06T07:24:43.948331+0000","last_active":"2024-03-06T07:24:43.950786+0000","last_peered":"2024-03-06T07:24:43.950786+0000","last_clean":"2024-03-06T07:24:43.950786+0000","last_became_active":"2024-03-06T07:24:43.948035+0000","last_became_peered":"2024-03-06T07:24:43.948035+0000","last_unstale":"2024-03-06T07:24:43.950786+0000","last_undegraded":"2024-03-06T07:24:43.950786+0000","last_fullsized":"2024-03-06T07:24:43.950786+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:47.769348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940727+0000","last_change":"2024-03-06T07:24:42.926540+0000","last_active":"2024-03-06T07:24:44.940727+0000","last_peered":"2024-03-06T07:24:44.940727+0000","last_clean":"2024-03-06T07:24:44.940727+0000","last_became_active":"2024-03-06T07:24:42.926223+0000","last_became_peered":"2024-03-06T07:24:42.926223+0000","last_unstale":"2024-03-06T07:24:44.940727+0000","last_undegraded":"2024-03-06T07:24:44.940727+0000","last_fullsized":"2024-03-06T07:24:44.940727+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:03.055775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.950721+0000","last_change":"2024-03-06T07:24:43.948827+0000","last_active":"2024-03-06T07:24:43.950721+0000","last_peered":"2024-03-06T07:24:43.950721+0000","last_clean":"2024-03-06T07:24:43.950721+0000","last_became_active":"2024-03-06T07:24:43.948558+0000","last_became_peered":"2024-03-06T07:24:43.948558+0000","last_unstale":"2024-03-06T07:24:43.950721+0000","last_undegraded":"2024-03-06T07:24:43.950721+0000","last_fullsized":"2024-03-06T07:24:43.950721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:59.661391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.933586+0000","last_change":"2024-03-06T07:24:42.936906+0000","last_active":"2024-03-06T07:24:43.933586+0000","last_peered":"2024-03-06T07:24:43.933586+0000","last_clean":"2024-03-06T07:24:43.933586+0000","last_became_active":"2024-03-06T07:24:42.936619+0000","last_became_peered":"2024-03-06T07:24:42.936619+0000","last_unstale":"2024-03-06T07:24:43.933586+0000","last_undegraded":"2024-03-06T07:24:43.933586+0000","last_fullsized":"2024-03-06T07:24:43.933586+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:41:24.564414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940750+0000","last_change":"2024-03-06T07:24:43.945502+0000","last_active":"2024-03-06T07:24:44.940750+0000","last_peered":"2024-03-06T07:24:44.940750+0000","last_clean":"2024-03-06T07:24:44.940750+0000","last_became_active":"2024-03-06T07:24:43.945230+0000","last_became_peered":"2024-03-06T07:24:43.945230+0000","last_unstale":"2024-03-06T07:24:44.940750+0000","last_undegraded":"2024-03-06T07:24:44.940750+0000","last_fullsized":"2024-03-06T07:24:44.940750+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:40.438201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.995033+0000","last_change":"2024-03-06T07:24:42.936142+0000","last_active":"2024-03-06T07:24:44.995033+0000","last_peered":"2024-03-06T07:24:44.995033+0000","last_clean":"2024-03-06T07:24:44.995033+0000","last_became_active":"2024-03-06T07:24:42.935849+0000","last_became_peered":"2024-03-06T07:24:42.935849+0000","last_unstale":"2024-03-06T07:24:44.995033+0000","last_undegraded":"2024-03-06T07:24:44.995033+0000","last_fullsized":"2024-03-06T07:24:44.995033+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:18.351009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-06T07:24:43.956831+0000","last_change":"2024-03-06T07:24:43.948822+0000","last_active":"2024-03-06T07:24:43.956831+0000","last_peered":"2024-03-06T07:24:43.956831+0000","last_clean":"2024-03-06T07:24:43.956831+0000","last_became_active":"2024-03-06T07:24:43.948568+0000","last_became_peered":"2024-03-06T07:24:43.948568+0000","last_unstale":"2024-03-06T07:24:43.956831+0000","last_undegraded":"2024-03-06T07:24:43.956831+0000","last_fullsized":"2024-03-06T07:24:43.956831+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:42.913479+0000","last_clean_scrub_stamp":"2024-03-06T07:24:42.913479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:41:25.568455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.940276+0000","last_change":"2024-03-06T07:24:42.934742+0000","last_active":"2024-03-06T07:24:44.940276+0000","last_peered":"2024-03-06T07:24:44.940276+0000","last_clean":"2024-03-06T07:24:44.940276+0000","last_became_active":"2024-03-06T07:24:42.934458+0000","last_became_peered":"2024-03-06T07:24:42.934458+0000","last_unstale":"2024-03-06T07:24:44.940276+0000","last_undegraded":"2024-03-06T07:24:44.940276+0000","last_fullsized":"2024-03-06T07:24:44.940276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:41.909438+0000","last_clean_scrub_stamp":"2024-03-06T07:24:41.909438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:19.886553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":361,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:44.939903+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:24:44.939903+0000","last_peered":"2024-03-06T07:24:44.939903+0000","last_clean":"2024-03-06T07:24:44.939903+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:24:44.939903+0000","last_undegraded":"2024-03-06T07:24:44.939903+0000","last_fullsized":"2024-03-06T07:24:44.939903+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":1315659,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":724261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":723611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1515520,"data_stored":730281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":725271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":724205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":723659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1576960,"data_stored":1314013,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:24:48.403 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:24:48.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:24:48.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62eb5e640 1 Processor -- start 2024-03-06T07:24:48.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62eb5e640 1 -- start start 2024-03-06T07:24:48.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62eb5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 0x7fd6280c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62eb5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628152dd0 con 0x7fd628155530 2024-03-06T07:24:48.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62eb5e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628158b00 con 0x7fd6281529f0 2024-03-06T07:24:48.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62eb5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd628159710 con 0x7fd6280c5370 2024-03-06T07:24:48.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62d0fb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd628155530 0x7fd6281559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55828/0 (socket says 172.21.15.26:55828) 2024-03-06T07:24:48.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.546+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4205970498 learned_addr learned my addr 172.21.15.26:0/4205970498 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd627fff640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 0x7fd6280c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564482452 0 0) 0x7fd628152dd0 con 0x7fd628155530 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd614003520 con 0x7fd628155530 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792716002 0 0) 0x7fd628158b00 con 0x7fd6281529f0 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd628152dd0 con 0x7fd6281529f0 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1020162876 0 0) 0x7fd614003520 con 0x7fd628155530 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 msgr2=0x7fd6280c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 0x7fd6280c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 >> v1:172.21.15.117:6789/0 conn(0x7fd6281529f0 legacy=0x7fd6280c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd628158de0 con 0x7fd628155530 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd620001670 con 0x7fd628155530 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 >> v1:172.21.15.26:6789/0 conn(0x7fd628155530 legacy=0x7fd6281559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614003b70 0x7fd614003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 0x7fd614004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd614004d40 0x7fd6140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628159710 con 0x7fd614004d40 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61400bb50 con 0x7fd614003b70 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61400bd20 con 0x7fd6280c5370 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd62d0fb640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd614004d40 0x7fd6140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd62c8fa640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614003b70 0x7fd614003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.547+0000 7fd627fff640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 0x7fd614004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.548+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 msgr2=0x7fd614004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.548+0000 7fd62d0fb640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6280c5370 0x7fd614004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.548+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614003b70 msgr2=0x7fd614003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.548+0000 7fd62d0fb640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614003b70 0x7fd614003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.548+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4205970498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614003520 con 0x7fd614004d40 2024-03-06T07:24:48.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62d0fb640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd614004d40 0x7fd6140051c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c0e0 tx=0x7fd614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559bfa7b1044b6f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620007c00 con 0x7fd614004d40 2024-03-06T07:24:48.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd620030070 con 0x7fd614004d40 2024-03-06T07:24:48.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd6277fe640 1 -- 172.21.15.26:0/4205970498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620005d30 con 0x7fd614004d40 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4205970498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd614004d40 msgr2=0x7fd6140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62eb5e640 1 --2- 172.21.15.26:0/4205970498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd614004d40 0x7fd6140051c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c0e0 tx=0x7fd614005830 comp rx=0 tx=0).stop 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4205970498 reap_dead start 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4205970498 shutdown_connections 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4205970498 >> 172.21.15.26:0/4205970498 conn(0x7fd6280ac580 msgr2=0x7fd6280ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4205970498 shutdown_connections 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.549+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4205970498 wait complete. 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 Processor -- start 2024-03-06T07:24:48.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 -- start start 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614004d40 0x7fd628154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd628156b80 0x7fd62806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6281eaac0 0x7fd6281ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c9000 con 0x7fd6281eaac0 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c68b0 con 0x7fd614004d40 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62eb5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6281593a0 con 0x7fd628156b80 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.551+0000 7fd62d0fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6281eaac0 0x7fd6281ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6281eaac0 0x7fd6281ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35520/0 (socket says 172.21.15.26:35520) 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4283923012 learned_addr learned my addr 172.21.15.26:0/4283923012 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd627fff640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd628156b80 0x7fd62806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62c8fa640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614004d40 0x7fd628154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd628156b80 msgr2=0x7fd62806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd628156b80 0x7fd62806c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614004d40 msgr2=0x7fd628154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614004d40 0x7fd628154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 -- 172.21.15.26:0/4283923012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62814cee0 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62d0fb640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6281eaac0 0x7fd6281ee030 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd620000c00 tx=0x7fd62002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd6257fa640 1 -- 172.21.15.26:0/4283923012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620000e40 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd6257fa640 1 -- 172.21.15.26:0/4283923012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62002cba0 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd628143170 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd6257fa640 1 -- 172.21.15.26:0/4283923012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620007be0 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.552+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62814ef80 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.553+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62804f330 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.554+0000 7fd6257fa640 1 -- 172.21.15.26:0/4283923012 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd620007070 con 0x7fd6281eaac0 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.555+0000 7fd6257fa640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd60806b4d0 0x7fd60806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.555+0000 7fd6257fa640 1 -- 172.21.15.26:0/4283923012 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6200ab0a0 con 0x7fd6281eaac0 2024-03-06T07:24:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.555+0000 7fd62c8fa640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd60806b4d0 0x7fd60806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.555+0000 7fd62c8fa640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd60806b4d0 0x7fd60806d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd610001940 tx=0x7fd610009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.559+0000 7fd6257fa640 1 -- 172.21.15.26:0/4283923012 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd6200797d0 con 0x7fd6281eaac0 2024-03-06T07:24:48.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.807+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6280acae0 con 0x7fd6281eaac0 2024-03-06T07:24:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.809+0000 7fd6257fa640 1 -- 172.21.15.26:0/4283923012 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd62007d870 con 0x7fd6281eaac0 2024-03-06T07:24:48.812 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:24:48.812 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:24:48.812 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:24:48.812 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:24:48.812 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:24:48.813 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:24:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd60806b4d0 msgr2=0x7fd60806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd60806b4d0 0x7fd60806d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd610001940 tx=0x7fd610009040 comp rx=0 tx=0).stop 2024-03-06T07:24:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6281eaac0 msgr2=0x7fd6281ee030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6281eaac0 0x7fd6281ee030 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd620000c00 tx=0x7fd62002e070 comp rx=0 tx=0).stop 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 shutdown_connections 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd60806b4d0 0x7fd60806d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6281eaac0 0x7fd6281ee030 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd628156b80 0x7fd62806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 --2- 172.21.15.26:0/4283923012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd614004d40 0x7fd628154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 >> 172.21.15.26:0/4283923012 conn(0x7fd6280ac580 msgr2=0x7fd62814fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 shutdown_connections 2024-03-06T07:24:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.812+0000 7fd62eb5e640 1 -- 172.21.15.26:0/4283923012 wait complete. 2024-03-06T07:24:48.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:48.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.932+0000 7f9b43b35640 1 Processor -- start 2024-03-06T07:24:48.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.932+0000 7f9b43b35640 1 -- start start 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b43b35640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9b3c14f360 0x7f9b3c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b43b35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b3c0c88c0 0x7f9b3c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b43b35640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0c73d0 con 0x7f9b3c0c5880 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b43b35640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c0c8dd0 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b43b35640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c0be2e0 con 0x7f9b3c14f360 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b410d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b3c0c88c0 0x7f9b3c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b410d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b3c0c88c0 0x7f9b3c0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35534/0 (socket says 172.21.15.26:35534) 2024-03-06T07:24:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b410d0640 1 -- 172.21.15.26:0/3604136115 learned_addr learned my addr 172.21.15.26:0/3604136115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b418d1640 1 --2- 172.21.15.26:0/3604136115 >> v2:172.21.15.117:3300/0 conn(0x7f9b3c14f360 0x7f9b3c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b410d0640 1 -- 172.21.15.26:0/3604136115 >> v2:172.21.15.117:3300/0 conn(0x7f9b3c14f360 msgr2=0x7f9b3c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b410d0640 1 --2- 172.21.15.26:0/3604136115 >> v2:172.21.15.117:3300/0 conn(0x7f9b3c14f360 0x7f9b3c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b410d0640 1 -- 172.21.15.26:0/3604136115 >> v1:172.21.15.117:6789/0 conn(0x7f9b3c0c5880 legacy=0x7f9b3c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.933+0000 7f9b410d0640 1 -- 172.21.15.26:0/3604136115 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0c7670 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.934+0000 7f9b410d0640 1 --2- 172.21.15.26:0/3604136115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c5340 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c08c840 tx=0x7f9b2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b883457a02824db8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.934+0000 7f9b408cf640 1 -- 172.21.15.26:0/3604136115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c035020 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.934+0000 7f9b408cf640 1 -- 172.21.15.26:0/3604136115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c031870 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.934+0000 7f9b408cf640 1 -- 172.21.15.26:0/3604136115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c031c40 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 -- 172.21.15.26:0/3604136115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 msgr2=0x7f9b3c0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 --2- 172.21.15.26:0/3604136115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c5340 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c08c840 tx=0x7f9b2c02b910 comp rx=0 tx=0).stop 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 -- 172.21.15.26:0/3604136115 shutdown_connections 2024-03-06T07:24:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 --2- 172.21.15.26:0/3604136115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c5340 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 --2- 172.21.15.26:0/3604136115 >> v2:172.21.15.117:3300/0 conn(0x7f9b3c14f360 0x7f9b3c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 -- 172.21.15.26:0/3604136115 >> 172.21.15.26:0/3604136115 conn(0x7f9b3c0ac570 msgr2=0x7f9b3c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 -- 172.21.15.26:0/3604136115 shutdown_connections 2024-03-06T07:24:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.935+0000 7f9b43b35640 1 -- 172.21.15.26:0/3604136115 wait complete. 2024-03-06T07:24:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.936+0000 7f9b43b35640 1 Processor -- start 2024-03-06T07:24:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.936+0000 7f9b43b35640 1 -- start start 2024-03-06T07:24:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b43b35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b3c0c5880 0x7f9b3c0ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b43b35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b43b35640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b3c14f360 0x7f9b3c14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b43b35640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c159270 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b43b35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0c7c30 con 0x7f9b3c0c5880 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b43b35640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c1593f0 con 0x7f9b3c14f360 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35536/0 (socket says 172.21.15.26:35536) 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 -- 172.21.15.26:0/782731485 learned_addr learned my addr 172.21.15.26:0/782731485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b418d1640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b3c0c5880 0x7f9b3c0ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b420d2640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b3c14f360 0x7f9b3c14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 -- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b3c14f360 msgr2=0x7f9b3c14bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b3c14f360 0x7f9b3c14bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 -- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b3c0c5880 msgr2=0x7f9b3c0ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b3c0c5880 0x7f9b3c0ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 -- 172.21.15.26:0/782731485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0c1d30 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b420d2640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b3c14f360 0x7f9b3c14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.937+0000 7f9b410d0640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c0cd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c02b950 tx=0x7f9b2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.938+0000 7f9b327fc640 1 -- 172.21.15.26:0/782731485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c040070 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.938+0000 7f9b327fc640 1 -- 172.21.15.26:0/782731485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c030d00 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.938+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c1edf40 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.938+0000 7f9b327fc640 1 -- 172.21.15.26:0/782731485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c035040 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.938+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c1ee4c0 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.939+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c04f330 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.939+0000 7f9b327fc640 1 -- 172.21.15.26:0/782731485 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b2c04b060 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.940+0000 7f9b327fc640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b2006b4d0 0x7f9b2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.940+0000 7f9b327fc640 1 -- 172.21.15.26:0/782731485 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b2c0ac110 con 0x7f9b3c0c88c0 2024-03-06T07:24:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.940+0000 7f9b418d1640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b2006b4d0 0x7f9b2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.941+0000 7f9b418d1640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b2006b4d0 0x7f9b2006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9b38006fd0 tx=0x7f9b38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:48.944+0000 7f9b327fc640 1 -- 172.21.15.26:0/782731485 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b3c0c1d30 con 0x7f9b3c0c88c0 2024-03-06T07:24:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.165+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b3c06cda0 con 0x7f9b3c0c88c0 2024-03-06T07:24:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.165+0000 7f9b327fc640 1 -- 172.21.15.26:0/782731485 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f9b2c07ae90 con 0x7f9b3c0c88c0 2024-03-06T07:24:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:49.168 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1856740515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1856740515},{"type":"v1","addr":"172.21.15.26:6834","nonce":1856740515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:24:45.977307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/577277314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":577277314},{"type":"v1","addr":"172.21.15.26:6835","nonce":577277314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-03-06T07:24:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 105 2024-03-06T07:24:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b2006b4d0 msgr2=0x7f9b2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b2006b4d0 0x7f9b2006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9b38006fd0 tx=0x7f9b38008040 comp rx=0 tx=0).stop 2024-03-06T07:24:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 msgr2=0x7f9b3c0c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c0cd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c02b950 tx=0x7f9b2c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 shutdown_connections 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9b2006b4d0 0x7f9b2006d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b3c14f360 0x7f9b3c14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b3c0c88c0 0x7f9b3c0c0cd0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 --2- 172.21.15.26:0/782731485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b3c0c5880 0x7f9b3c0ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 >> 172.21.15.26:0/782731485 conn(0x7f9b3c0ac570 msgr2=0x7f9b3c1542c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 shutdown_connections 2024-03-06T07:24:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.167+0000 7f9b43b35640 1 -- 172.21.15.26:0/782731485 wait complete. 2024-03-06T07:24:49.215 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-03-06T07:24:49.212+0000 7fa0ed3f2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:49.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:24:45.977307+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7854}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7854': {'gid': 7854, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/577277314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 577277314}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 577277314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7854, 'qdb_cluster': [7854]} 2024-03-06T07:24:49.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:49.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:24:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.683+0000 7f584715c640 1 Processor -- start 2024-03-06T07:24:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.683+0000 7f584715c640 1 -- start start 2024-03-06T07:24:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f584715c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58400c7170 0x7f58400c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f584715c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f58400c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f584715c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58400a8390 con 0x7f584006ff10 2024-03-06T07:24:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f584715c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58400c7620 con 0x7f58400c7170 2024-03-06T07:24:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f584715c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58400730f0 con 0x7f58400c6c40 2024-03-06T07:24:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f583ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58400c7170 0x7f58400c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f58456f9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f58400c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f5844ef8640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f584006ff10 0x7f5840153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:59358/0 (socket says 172.21.15.26:59358) 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f5844ef8640 1 -- 172.21.15.26:0/235278589 learned_addr learned my addr 172.21.15.26:0/235278589 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.684+0000 7f583ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58400c7170 0x7f58400c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36004/0 (socket says 172.21.15.26:36004) 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.685+0000 7f583ffff640 1 -- 172.21.15.26:0/235278589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 msgr2=0x7f58400c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.685+0000 7f583ffff640 1 --2- 172.21.15.26:0/235278589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f58400c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.685+0000 7f583ffff640 1 -- 172.21.15.26:0/235278589 >> v1:172.21.15.117:6789/0 conn(0x7f584006ff10 legacy=0x7f5840153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.685+0000 7f583ffff640 1 -- 172.21.15.26:0/235278589 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58400733d0 con 0x7f58400c7170 2024-03-06T07:24:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.685+0000 7f583f7fe640 1 -- 172.21.15.26:0/235278589 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223926871 0 0) 0x7f58400a8390 con 0x7f584006ff10 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f583ffff640 1 --2- 172.21.15.26:0/235278589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58400c7170 0x7f58400c8810 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f582c0097a0 tx=0x7f582c02b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ef6e7a26c20bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f583f7fe640 1 -- 172.21.15.26:0/235278589 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c038070 con 0x7f58400c7170 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f583f7fe640 1 -- 172.21.15.26:0/235278589 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c031700 con 0x7f58400c7170 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f583f7fe640 1 -- 172.21.15.26:0/235278589 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c031ad0 con 0x7f58400c7170 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 -- 172.21.15.26:0/235278589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58400c7170 msgr2=0x7f58400c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 --2- 172.21.15.26:0/235278589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58400c7170 0x7f58400c8810 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f582c0097a0 tx=0x7f582c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 -- 172.21.15.26:0/235278589 shutdown_connections 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 --2- 172.21.15.26:0/235278589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f58400c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 --2- 172.21.15.26:0/235278589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58400c7170 0x7f58400c8810 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 -- 172.21.15.26:0/235278589 >> 172.21.15.26:0/235278589 conn(0x7f584006b970 msgr2=0x7f58400aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 -- 172.21.15.26:0/235278589 shutdown_connections 2024-03-06T07:24:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.686+0000 7f584715c640 1 -- 172.21.15.26:0/235278589 wait complete. 2024-03-06T07:24:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.687+0000 7f584715c640 1 Processor -- start 2024-03-06T07:24:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.687+0000 7f584715c640 1 -- start start 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f584715c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584006ff10 0x7f58401eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f584715c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f5840151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f584715c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58400c7170 0x7f584014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f584715c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58401544b0 con 0x7f58400c7170 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f584715c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5840154330 con 0x7f584006ff10 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f584715c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58400702f0 con 0x7f58400c6c40 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58400c7170 0x7f584014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58400c7170 0x7f584014eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35550/0 (socket says 172.21.15.26:35550) 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 -- 172.21.15.26:0/1102315970 learned_addr learned my addr 172.21.15.26:0/1102315970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:49.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584006ff10 0x7f58401eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f583ffff640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f5840151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 -- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 msgr2=0x7f5840151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f5840151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 -- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584006ff10 msgr2=0x7f58401eeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584006ff10 0x7f58401eeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 -- 172.21.15.26:0/1102315970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58401de700 con 0x7f58400c7170 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f583ffff640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f5840151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584006ff10 0x7f58401eeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f58456f9640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58400c7170 0x7f584014eb40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f583800efc0 tx=0x7f583800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f583d7fa640 1 -- 172.21.15.26:0/1102315970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838018020 con 0x7f58400c7170 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f583d7fa640 1 -- 172.21.15.26:0/1102315970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838014480 con 0x7f58400c7170 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58401de9e0 con 0x7f58400c7170 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.689+0000 7f583d7fa640 1 -- 172.21.15.26:0/1102315970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838014870 con 0x7f58400c7170 2024-03-06T07:24:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.690+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58401e03a0 con 0x7f58400c7170 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.691+0000 7f583d7fa640 1 -- 172.21.15.26:0/1102315970 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5838005d10 con 0x7f58400c7170 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.691+0000 7f583d7fa640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f582406b4d0 0x7f582406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.691+0000 7f583d7fa640 1 -- 172.21.15.26:0/1102315970 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f583808ae20 con 0x7f58400c7170 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.692+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f582406b4d0 0x7f582406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.692+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f582406b4d0 0x7f582406d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5834000f60 tx=0x7f5834007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.692+0000 7f5849e14740 1 Processor -- start 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.692+0000 7f5849e14740 1 -- start start 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb1ff50 0x5653fdb1da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb20330 0x5653fdb1df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f5849e14740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1e4b0 0x5653fdb1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f5849e14740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653fda846b0 con 0x5653fdb1e4b0 2024-03-06T07:24:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653fda2da60 con 0x5653fdb1ff50 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653fda80750 con 0x5653fdb20330 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1e4b0 0x5653fdb1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1e4b0 0x5653fdb1a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35562/0 (socket says 172.21.15.26:35562) 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 -- 172.21.15.26:0/1245677753 learned_addr learned my addr 172.21.15.26:0/1245677753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f583ffff640 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb20330 0x5653fdb1df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb1ff50 0x5653fdb1da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 -- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb20330 msgr2=0x5653fdb1df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb20330 0x5653fdb1df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 -- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb1ff50 msgr2=0x5653fdb1da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb1ff50 0x5653fdb1da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.693+0000 7f58456f9640 1 -- 172.21.15.26:0/1245677753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653fda4bf30 con 0x5653fdb1e4b0 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f58456f9640 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1e4b0 0x5653fdb1a680 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f583800c990 tx=0x7f5838058e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72df909a9ba09a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f582a7fc640 1 -- 172.21.15.26:0/1245677753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838061b10 con 0x5653fdb1e4b0 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f582a7fc640 1 -- 172.21.15.26:0/1245677753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838057700 con 0x5653fdb1e4b0 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f582a7fc640 1 -- 172.21.15.26:0/1245677753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583805d160 con 0x5653fdb1e4b0 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 -- 172.21.15.26:0/1245677753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1e4b0 msgr2=0x5653fdb1a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1e4b0 0x5653fdb1a680 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f583800c990 tx=0x7f5838058e90 comp rx=0 tx=0).stop 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 -- 172.21.15.26:0/1245677753 shutdown_connections 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1e4b0 0x5653fdb1a680 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb20330 0x5653fdb1df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 --2- 172.21.15.26:0/1245677753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb1ff50 0x5653fdb1da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 -- 172.21.15.26:0/1245677753 >> 172.21.15.26:0/1245677753 conn(0x5653fdb1c7d0 msgr2=0x5653fdb413c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 -- 172.21.15.26:0/1245677753 shutdown_connections 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.694+0000 7f5849e14740 1 -- 172.21.15.26:0/1245677753 wait complete. 2024-03-06T07:24:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.695+0000 7f5849e14740 1 -- start start 2024-03-06T07:24:49.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.695+0000 7f5849e14740 1 Processor -- start 2024-03-06T07:24:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb20330 0x5653fdb70630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb70b70 0x5653fdb40ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653fdb42230 con 0x5653fdb20330 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653fda40ea0 con 0x5653fdb70b70 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653fdb41c70 con 0x5653fdb1ff50 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 10 client.0 _ll_register_callbacks cb 0x5653fda40870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb20330 0x5653fdb70630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb20330 0x5653fdb70630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35574/0 (socket says 172.21.15.26:35574) 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 -- 172.21.15.26:0/849414338 learned_addr learned my addr 172.21.15.26:0/849414338 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f58456f9640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb70b70 0x5653fdb40ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5844ef8640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 -- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 msgr2=0x5653fdb5fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb5fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 -- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb70b70 msgr2=0x5653fdb40ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb70b70 0x5653fdb40ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 -- 172.21.15.26:0/849414338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653fda41270 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f58456f9640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb70b70 0x5653fdb40ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5844ef8640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f583ffff640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb20330 0x5653fdb70630 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f582c0097a0 tx=0x7f582c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffb0bfdf5b84a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f581f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f581f7fe640 1 -- 172.21.15.26:0/849414338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c044070 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f581f7fe640 1 -- 172.21.15.26:0/849414338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f582c003810 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.697+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5838057700 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f581f7fe640 1 -- 172.21.15.26:0/849414338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f582c038040 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 10 client.7998 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5653fd7992d0 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f581f7fe640 1 -- 172.21.15.26:0/849414338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838057700 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f581f7fe640 1 -- 172.21.15.26:0/849414338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f582c032070 con 0x5653fdb20330 2024-03-06T07:24:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 10 client.7998 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 10 client.7998 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 20 client.7998 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb3fad0 0x5653fdb7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 4 client.7998 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 --> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653fda91bf0 con 0x5653fdb3fad0 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5844ef8640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb3fad0 0x5653fdb7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f581f7fe640 10 client.7998 ms_handle_connect on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.698+0000 7f5844ef8640 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb3fad0 0x5653fdb7b560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3893da72043c4b server_cookie=8cee5101b0598b09 in_seq=0 out_seq=0 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f581f7fe640 1 -- 172.21.15.26:0/849414338 <== mds.0 v2:172.21.15.26:6833/577277314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5653fda91bf0 con 0x5653fdb3fad0 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f581f7fe640 10 client.7998 handle_command_reply: tid=0 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 client.7998 shutdown 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb20330 msgr2=0x5653fdb70630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb20330 0x5653fdb70630 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f582c0097a0 tx=0x7f582c004730 comp rx=0 tx=0).stop 2024-03-06T07:24:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 shutdown_connections 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb3fad0 0x5653fdb7b560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb70b70 0x5653fdb40ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb20330 0x5653fdb70630 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f581f7fe640 0 client.7998 ms_handle_reset on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 --2- 172.21.15.26:0/849414338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 >> 172.21.15.26:0/849414338 conn(0x5653fdb1e4b0 msgr2=0x5653fdb1e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 shutdown_connections 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.699+0000 7f5849e14740 1 -- 172.21.15.26:0/849414338 wait complete. 2024-03-06T07:24:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.700+0000 7f5849e14740 20 client.7998 trim_cache size 0 max 16384 2024-03-06T07:24:49.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.732+0000 7f5849e14740 1 Processor -- start 2024-03-06T07:24:49.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.732+0000 7f5849e14740 1 -- start start 2024-03-06T07:24:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5849e14740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1ff50 0x5653fdb1a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 0x5653fdb1d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1a8f0 0x5653fdb1dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5849e14740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58401f6d10 con 0x5653fdb1ff50 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58400aa270 con 0x5653fdb20330 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58401f6710 con 0x5653fdb1a8f0 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5844ef8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1ff50 0x5653fdb1a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5844ef8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1ff50 0x5653fdb1a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35580/0 (socket says 172.21.15.26:35580) 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.733+0000 7f5844ef8640 1 -- 172.21.15.26:0/1687848132 learned_addr learned my addr 172.21.15.26:0/1687848132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5844ef8640 1 -- 172.21.15.26:0/1687848132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1a8f0 msgr2=0x5653fdb1dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1687848132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1a8f0 0x5653fdb1dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5844ef8640 1 -- 172.21.15.26:0/1687848132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 msgr2=0x5653fdb1d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1687848132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 0x5653fdb1d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5844ef8640 1 -- 172.21.15.26:0/1687848132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653fd8636b0 con 0x5653fdb1ff50 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5844ef8640 1 --2- 172.21.15.26:0/1687848132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1ff50 0x5653fdb1a3b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5834007eb0 tx=0x7f5834012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b81f57754495c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5829ffb640 1 -- 172.21.15.26:0/1687848132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340261b0 con 0x5653fdb1ff50 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5829ffb640 1 -- 172.21.15.26:0/1687848132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58340139b0 con 0x5653fdb1ff50 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.734+0000 7f5829ffb640 1 -- 172.21.15.26:0/1687848132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834013b80 con 0x5653fdb1ff50 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 -- 172.21.15.26:0/1687848132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1ff50 msgr2=0x5653fdb1a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 --2- 172.21.15.26:0/1687848132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1ff50 0x5653fdb1a3b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5834007eb0 tx=0x7f5834012000 comp rx=0 tx=0).stop 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 -- 172.21.15.26:0/1687848132 shutdown_connections 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 --2- 172.21.15.26:0/1687848132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1a8f0 0x5653fdb1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 --2- 172.21.15.26:0/1687848132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 0x5653fdb1d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 --2- 172.21.15.26:0/1687848132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1ff50 0x5653fdb1a3b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 -- 172.21.15.26:0/1687848132 >> 172.21.15.26:0/1687848132 conn(0x5653fdb1e4b0 msgr2=0x5653fdb5dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 -- 172.21.15.26:0/1687848132 shutdown_connections 2024-03-06T07:24:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 -- 172.21.15.26:0/1687848132 wait complete. 2024-03-06T07:24:49.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 -- start start 2024-03-06T07:24:49.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.735+0000 7f5849e14740 1 Processor -- start 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5849e14740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1a8f0 0x5653fdb3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb19910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5849e14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 0x5653fdb80100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5849e14740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653fdb70160 con 0x5653fdb1a8f0 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653fdb6d3f0 con 0x5653fdb20330 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5849e14740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653fdb3e450 con 0x5653fdb1ff50 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5849e14740 10 client.0 _ll_register_callbacks cb 0x7f58380095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1a8f0 0x5653fdb3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1a8f0 0x5653fdb3de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35594/0 (socket says 172.21.15.26:35594) 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 -- 172.21.15.26:0/3230219325 learned_addr learned my addr 172.21.15.26:0/3230219325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f583ffff640 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb19910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f58456f9640 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 0x5653fdb80100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 -- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 msgr2=0x5653fdb19910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb19910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 -- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 msgr2=0x5653fdb80100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 0x5653fdb80100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.736+0000 7f5844ef8640 1 -- 172.21.15.26:0/3230219325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653fdb69d90 con 0x5653fdb1a8f0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5844ef8640 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1a8f0 0x5653fdb3de40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5834015000 tx=0x7f5834000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e87d4dcd5330d0a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f581f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f581f7fe640 1 -- 172.21.15.26:0/3230219325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834013d70 con 0x5653fdb1a8f0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f581f7fe640 1 -- 172.21.15.26:0/3230219325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58340267a0 con 0x5653fdb1a8f0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58340139b0 con 0x5653fdb1a8f0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f581f7fe640 1 -- 172.21.15.26:0/3230219325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f583403b050 con 0x5653fdb1a8f0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 10 client.8004 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5653fdb1da00 con 0x5653fdb1a8f0 2024-03-06T07:24:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f581f7fe640 1 -- 172.21.15.26:0/3230219325 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58340139b0 con 0x5653fdb1a8f0 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f581f7fe640 1 -- 172.21.15.26:0/3230219325 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f583403e020 con 0x5653fdb1a8f0 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 10 client.8004 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 10 client.8004 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 20 client.8004 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb57ac0 0x5653fdb57ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 4 client.8004 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 --> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5653fda8bf40 con 0x5653fdb57ac0 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.737+0000 7f583ffff640 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb57ac0 0x5653fdb57ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f581f7fe640 10 client.8004 ms_handle_connect on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f583ffff640 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb57ac0 0x5653fdb57ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcba4914614586a server_cookie=9820108461bafb7 in_seq=0 out_seq=0 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f581f7fe640 1 -- 172.21.15.26:0/3230219325 <== mds.0 v2:172.21.15.26:6833/577277314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5653fda8bf40 con 0x5653fdb57ac0 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f581f7fe640 10 client.8004 handle_command_reply: tid=0 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 client.8004 shutdown 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1a8f0 msgr2=0x5653fdb3de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1a8f0 0x5653fdb3de40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5834015000 tx=0x7f5834000ed0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 shutdown_connections 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x5653fdb57ac0 0x5653fdb57ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653fdb20330 0x5653fdb80100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5653fdb1ff50 0x5653fdb19910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f581f7fe640 0 client.8004 ms_handle_reset on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 --2- 172.21.15.26:0/3230219325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653fdb1a8f0 0x5653fdb3de40 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 >> 172.21.15.26:0/3230219325 conn(0x5653fdb1e4b0 msgr2=0x5653fdb43820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 shutdown_connections 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 1 -- 172.21.15.26:0/3230219325 wait complete. 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.738+0000 7f5849e14740 20 client.8004 trim_cache size 0 max 16384 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7854, 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.8023260800000003, 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 105, 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 121, 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 121, 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.891174640999999 2024-03-06T07:24:49.748 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f582406b4d0 msgr2=0x7f582406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f582406b4d0 0x7f582406d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5834000f60 tx=0x7f5834007ca0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58400c7170 msgr2=0x7f584014eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58400c7170 0x7f584014eb40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f583800efc0 tx=0x7f583800c490 comp rx=0 tx=0).stop 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 shutdown_connections 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f582406b4d0 0x7f582406d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58400c7170 0x7f584014eb40 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58400c6c40 0x7f5840151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 --2- 172.21.15.26:0/1102315970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584006ff10 0x7f58401eeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 >> 172.21.15.26:0/1102315970 conn(0x7f584006b970 msgr2=0x7f58400c5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 shutdown_connections 2024-03-06T07:24:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.739+0000 7f584715c640 1 -- 172.21.15.26:0/1102315970 wait complete. 2024-03-06T07:24:49.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.851+0000 7f5753e66640 1 Processor -- start 2024-03-06T07:24:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.852+0000 7f5753e66640 1 -- start start 2024-03-06T07:24:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.852+0000 7f5753e66640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f574c0c88c0 0x7f574c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.852+0000 7f5753e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0c73d0 con 0x7f574c14f360 2024-03-06T07:24:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.852+0000 7f5753e66640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0be2e0 con 0x7f574c158b60 2024-03-06T07:24:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.852+0000 7f5753e66640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c159700 con 0x7f574c0c88c0 2024-03-06T07:24:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5751c02640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f574c14f360 0x7f574c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55842/0 (socket says 172.21.15.26:55842) 2024-03-06T07:24:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5751c02640 1 -- 172.21.15.26:0/1407434894 learned_addr learned my addr 172.21.15.26:0/1407434894 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5751401640 1 --2- 172.21.15.26:0/1407434894 >> v2:172.21.15.117:3300/0 conn(0x7f574c0c88c0 0x7f574c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5750c00640 1 -- 172.21.15.26:0/1407434894 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413714351 0 0) 0x7f574c0c73d0 con 0x7f574c14f360 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5750c00640 1 -- 172.21.15.26:0/1407434894 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5734003520 con 0x7f574c14f360 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5750c00640 1 -- 172.21.15.26:0/1407434894 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526260877 0 0) 0x7f574c0be2e0 con 0x7f574c158b60 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5750c00640 1 -- 172.21.15.26:0/1407434894 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574c0c73d0 con 0x7f574c158b60 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5751401640 1 -- 172.21.15.26:0/1407434894 >> v1:172.21.15.117:6789/0 conn(0x7f574c158b60 legacy=0x7f574c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5751401640 1 -- 172.21.15.26:0/1407434894 >> v1:172.21.15.26:6789/0 conn(0x7f574c14f360 legacy=0x7f574c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.853+0000 7f5751401640 1 -- 172.21.15.26:0/1407434894 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0c7670 con 0x7f574c0c88c0 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.854+0000 7f5751401640 1 --2- 172.21.15.26:0/1407434894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c0c88c0 0x7f574c155c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f57400097a0 tx=0x7f574002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0a9bd292ce00f60 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.854+0000 7f5750c00640 1 -- 172.21.15.26:0/1407434894 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740035020 con 0x7f574c0c88c0 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.854+0000 7f5750c00640 1 -- 172.21.15.26:0/1407434894 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5740031830 con 0x7f574c0c88c0 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.854+0000 7f5750c00640 1 -- 172.21.15.26:0/1407434894 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740031be0 con 0x7f574c0c88c0 2024-03-06T07:24:49.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.855+0000 7f5753e66640 1 -- 172.21.15.26:0/1407434894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c0c88c0 msgr2=0x7f574c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.855+0000 7f5753e66640 1 --2- 172.21.15.26:0/1407434894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c0c88c0 0x7f574c155c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f57400097a0 tx=0x7f574002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.855+0000 7f5753e66640 1 -- 172.21.15.26:0/1407434894 shutdown_connections 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.855+0000 7f5753e66640 1 --2- 172.21.15.26:0/1407434894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c0c88c0 0x7f574c155c10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.855+0000 7f5753e66640 1 -- 172.21.15.26:0/1407434894 >> 172.21.15.26:0/1407434894 conn(0x7f574c0ac570 msgr2=0x7f574c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.855+0000 7f5753e66640 1 -- 172.21.15.26:0/1407434894 shutdown_connections 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.855+0000 7f5753e66640 1 -- 172.21.15.26:0/1407434894 wait complete. 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.856+0000 7f5753e66640 1 Processor -- start 2024-03-06T07:24:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.856+0000 7f5753e66640 1 -- start start 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5753e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0c88c0 0x7f574c152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5753e66640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c14f360 0x7f574c0c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5753e66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c158b60 0x7f574c0ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5753e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c7c30 con 0x7f574c0c88c0 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5753e66640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c5a50 con 0x7f574c158b60 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5753e66640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c5bd0 con 0x7f574c14f360 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0c88c0 0x7f574c152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0c88c0 0x7f574c152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35600/0 (socket says 172.21.15.26:35600) 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 -- 172.21.15.26:0/1836373182 learned_addr learned my addr 172.21.15.26:0/1836373182 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751401640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c14f360 0x7f574c0c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5752403640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c158b60 0x7f574c0ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 -- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c14f360 msgr2=0x7f574c0c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c14f360 0x7f574c0c1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 -- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c158b60 msgr2=0x7f574c0ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c158b60 0x7f574c0ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.857+0000 7f5751c02640 1 -- 172.21.15.26:0/1836373182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c1eed50 con 0x7f574c0c88c0 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.858+0000 7f5751c02640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0c88c0 0x7f574c152410 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f574400c500 tx=0x7f574400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.858+0000 7f574a7fc640 1 -- 172.21.15.26:0/1836373182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744017070 con 0x7f574c0c88c0 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.858+0000 7f574a7fc640 1 -- 172.21.15.26:0/1836373182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744005ce0 con 0x7f574c0c88c0 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.858+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c1eadb0 con 0x7f574c0c88c0 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.858+0000 7f574a7fc640 1 -- 172.21.15.26:0/1836373182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440072f0 con 0x7f574c0c88c0 2024-03-06T07:24:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.858+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c158530 con 0x7f574c0c88c0 2024-03-06T07:24:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.859+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c04f330 con 0x7f574c0c88c0 2024-03-06T07:24:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.859+0000 7f574a7fc640 1 -- 172.21.15.26:0/1836373182 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f574401a070 con 0x7f574c0c88c0 2024-03-06T07:24:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.860+0000 7f574a7fc640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573806b500 0x7f573806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.860+0000 7f574a7fc640 1 -- 172.21.15.26:0/1836373182 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5744089650 con 0x7f574c0c88c0 2024-03-06T07:24:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.860+0000 7f5751401640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573806b500 0x7f573806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.860+0000 7f5751401640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573806b500 0x7f573806d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5740036000 tx=0x7f5740008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:49.868+0000 7f574a7fc640 1 -- 172.21.15.26:0/1836373182 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5744057d80 con 0x7f574c0c88c0 2024-03-06T07:24:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.144+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f574c0c8d50 con 0x7f574c0c88c0 2024-03-06T07:24:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.144+0000 7f574a7fc640 1 -- 172.21.15.26:0/1836373182 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f574405be20 con 0x7f574c0c88c0 2024-03-06T07:24:50.147 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:50.147 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1856740515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1856740515},{"type":"v1","addr":"172.21.15.26:6834","nonce":1856740515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:24:45.977307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/577277314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":577277314},{"type":"v1","addr":"172.21.15.26:6835","nonce":577277314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-03-06T07:24:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 105 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573806b500 msgr2=0x7f573806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573806b500 0x7f573806d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5740036000 tx=0x7f5740008030 comp rx=0 tx=0).stop 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0c88c0 msgr2=0x7f574c152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0c88c0 0x7f574c152410 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f574400c500 tx=0x7f574400c9c0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 shutdown_connections 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573806b500 0x7f573806d9c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c158b60 0x7f574c0ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c14f360 0x7f574c0c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 --2- 172.21.15.26:0/1836373182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0c88c0 0x7f574c152410 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 >> 172.21.15.26:0/1836373182 conn(0x7f574c0ac570 msgr2=0x7f574c0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 shutdown_connections 2024-03-06T07:24:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.146+0000 7f5753e66640 1 -- 172.21.15.26:0/1836373182 wait complete. 2024-03-06T07:24:50.161 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 105, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:43.287707+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 105, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7854": { "addr": "172.21.15.26:6835/577277314", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 577277314, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 577277314, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7854, "incarnation": 102, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:24:45.977307+0000", "qdb_cluster": [ 7854 ], "qdb_leader": 7854, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7854 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/3943022187", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3943022187, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3943022187, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7465, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3949772965", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3949772965, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3949772965, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7468, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/1856740515", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 1856740515, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 1856740515, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7851, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:50.161 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:50.161 INFO:tasks.cephfs.test_failover:current = [{'gid': 7851, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/1856740515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1856740515}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1856740515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}, {'gid': 7465, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3943022187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3943022187}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3943022187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}, {'gid': 7854, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/577277314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 577277314}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 577277314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7468, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3949772965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3949772965}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3949772965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}] 2024-03-06T07:24:50.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.c mds_join_fs cephfs 2024-03-06T07:24:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.266+0000 7f83ab155640 1 Processor -- start 2024-03-06T07:24:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.266+0000 7f83ab155640 1 -- start start 2024-03-06T07:24:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.266+0000 7f83ab155640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83a4152880 0x7f83a40c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.266+0000 7f83ab155640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83a40a7cd0 0x7f83a40c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.266+0000 7f83ab155640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a4155f40 con 0x7f83a40c6710 2024-03-06T07:24:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.266+0000 7f83ab155640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a4072510 con 0x7f83a40a7cd0 2024-03-06T07:24:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.266+0000 7f83ab155640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a4072db0 con 0x7f83a4152880 2024-03-06T07:24:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a96f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83a40a7cd0 0x7f83a40c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a96f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83a40a7cd0 0x7f83a40c2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35614/0 (socket says 172.21.15.26:35614) 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a96f2640 1 -- 172.21.15.26:0/2519281092 learned_addr learned my addr 172.21.15.26:0/2519281092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a96f2640 1 -- 172.21.15.26:0/2519281092 >> v2:172.21.15.117:3300/0 conn(0x7f83a4152880 msgr2=0x7f83a40c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a8ef1640 1 --2- 172.21.15.26:0/2519281092 >> v2:172.21.15.117:3300/0 conn(0x7f83a4152880 0x7f83a40c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a96f2640 1 --2- 172.21.15.26:0/2519281092 >> v2:172.21.15.117:3300/0 conn(0x7f83a4152880 0x7f83a40c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a96f2640 1 -- 172.21.15.26:0/2519281092 >> v1:172.21.15.117:6789/0 conn(0x7f83a40c6710 legacy=0x7f83a40c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.267+0000 7f83a96f2640 1 -- 172.21.15.26:0/2519281092 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4072760 con 0x7f83a40a7cd0 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83a96f2640 1 --2- 172.21.15.26:0/2519281092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40a7cd0 0x7f83a40c2590 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f83a4072690 tx=0x7f839c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e916abf9faecec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f839b7fe640 1 -- 172.21.15.26:0/2519281092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c02c9e0 con 0x7f83a40a7cd0 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f839b7fe640 1 -- 172.21.15.26:0/2519281092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c02cb80 con 0x7f83a40a7cd0 2024-03-06T07:24:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f839b7fe640 1 -- 172.21.15.26:0/2519281092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c0343f0 con 0x7f83a40a7cd0 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 -- 172.21.15.26:0/2519281092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40a7cd0 msgr2=0x7f83a40c2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 --2- 172.21.15.26:0/2519281092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40a7cd0 0x7f83a40c2590 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f83a4072690 tx=0x7f839c02be40 comp rx=0 tx=0).stop 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 -- 172.21.15.26:0/2519281092 shutdown_connections 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 --2- 172.21.15.26:0/2519281092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a40a7cd0 0x7f83a40c2590 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 --2- 172.21.15.26:0/2519281092 >> v2:172.21.15.117:3300/0 conn(0x7f83a4152880 0x7f83a40c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 -- 172.21.15.26:0/2519281092 >> 172.21.15.26:0/2519281092 conn(0x7f83a40aa5c0 msgr2=0x7f83a40aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 -- 172.21.15.26:0/2519281092 shutdown_connections 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.268+0000 7f83ab155640 1 -- 172.21.15.26:0/2519281092 wait complete. 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.269+0000 7f83ab155640 1 Processor -- start 2024-03-06T07:24:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.269+0000 7f83ab155640 1 -- start start 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83ab155640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40a7cd0 0x7f83a41e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83ab155640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c6710 0x7f83a414f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83ab155640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a4152880 0x7f83a414fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83ab155640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4155b80 con 0x7f83a4152880 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83ab155640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a40c2ad0 con 0x7f83a40a7cd0 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83ab155640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4155d00 con 0x7f83a40c6710 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83a96f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a4152880 0x7f83a414fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83a96f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a4152880 0x7f83a414fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35624/0 (socket says 172.21.15.26:35624) 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.270+0000 7f83a96f2640 1 -- 172.21.15.26:0/2048053402 learned_addr learned my addr 172.21.15.26:0/2048053402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a8ef1640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40a7cd0 0x7f83a41e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f839bfff640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c6710 0x7f83a414f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a96f2640 1 -- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c6710 msgr2=0x7f83a414f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a96f2640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c6710 0x7f83a414f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a96f2640 1 -- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40a7cd0 msgr2=0x7f83a41e1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a96f2640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40a7cd0 0x7f83a41e1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a96f2640 1 -- 172.21.15.26:0/2048053402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a40ab750 con 0x7f83a4152880 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a8ef1640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40a7cd0 0x7f83a41e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f839bfff640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c6710 0x7f83a414f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83a96f2640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a4152880 0x7f83a414fd50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f839c036040 tx=0x7f839c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83997fa640 1 -- 172.21.15.26:0/2048053402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c043020 con 0x7f83a4152880 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83997fa640 1 -- 172.21.15.26:0/2048053402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c03e450 con 0x7f83a4152880 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a40ac180 con 0x7f83a4152880 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83997fa640 1 -- 172.21.15.26:0/2048053402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c03e860 con 0x7f83a4152880 2024-03-06T07:24:50.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.271+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a4154870 con 0x7f83a4152880 2024-03-06T07:24:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.272+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a404f330 con 0x7f83a4152880 2024-03-06T07:24:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.273+0000 7f83997fa640 1 -- 172.21.15.26:0/2048053402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f839c046020 con 0x7f83a4152880 2024-03-06T07:24:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.273+0000 7f83997fa640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f839006b4d0 0x7f839006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.273+0000 7f83997fa640 1 -- 172.21.15.26:0/2048053402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f839c0acd00 con 0x7f83a4152880 2024-03-06T07:24:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.274+0000 7f83a8ef1640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f839006b4d0 0x7f839006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.274+0000 7f83a8ef1640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f839006b4d0 0x7f839006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f83a000b820 tx=0x7f83a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.277+0000 7f83997fa640 1 -- 172.21.15.26:0/2048053402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f839c04d030 con 0x7f83a4152880 2024-03-06T07:24:50.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.456+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f83a4070ea0 con 0x7f83a4152880 2024-03-06T07:24:50.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.459+0000 7f83997fa640 1 -- 172.21.15.26:0/2048053402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v42)=0 v42) v1 ==== 116+0+0 (secure 0 0 0) 0x7f839c07ba00 con 0x7f83a4152880 2024-03-06T07:24:50.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.461+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f839006b4d0 msgr2=0x7f839006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.461+0000 7f83ab155640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f839006b4d0 0x7f839006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f83a000b820 tx=0x7f83a0008040 comp rx=0 tx=0).stop 2024-03-06T07:24:50.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.461+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a4152880 msgr2=0x7f83a414fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.461+0000 7f83ab155640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a4152880 0x7f83a414fd50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f839c036040 tx=0x7f839c004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.461+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 shutdown_connections 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.461+0000 7f83ab155640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f839006b4d0 0x7f839006d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.462+0000 7f83ab155640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83a4152880 0x7f83a414fd50 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.462+0000 7f83ab155640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83a40c6710 0x7f83a414f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.462+0000 7f83ab155640 1 --2- 172.21.15.26:0/2048053402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83a40a7cd0 0x7f83a41e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.462+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 >> 172.21.15.26:0/2048053402 conn(0x7f83a40aa5c0 msgr2=0x7f83a40c3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.462+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 shutdown_connections 2024-03-06T07:24:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.462+0000 7f83ab155640 1 -- 172.21.15.26:0/2048053402 wait complete. 2024-03-06T07:24:50.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.577+0000 7f820178a640 1 Processor -- start 2024-03-06T07:24:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.577+0000 7f820178a640 1 -- start start 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f820178a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81fc152520 0x7f81fc06ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f820178a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc155c80 0x7f81fc0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f820178a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc0c1ae0 con 0x7f81fc0720a0 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f820178a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0c1dc0 con 0x7f81fc152520 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f820178a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc06e450 con 0x7f81fc155c80 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f81faffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81fc152520 0x7f81fc06ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f81faffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81fc152520 0x7f81fc06ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35628/0 (socket says 172.21.15.26:35628) 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f81faffd640 1 -- 172.21.15.26:0/1128426127 learned_addr learned my addr 172.21.15.26:0/1128426127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.578+0000 7f81fa7fc640 1 --2- 172.21.15.26:0/1128426127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc155c80 0x7f81fc0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.579+0000 7f81faffd640 1 -- 172.21.15.26:0/1128426127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc155c80 msgr2=0x7f81fc0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.579+0000 7f81faffd640 1 --2- 172.21.15.26:0/1128426127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc155c80 0x7f81fc0c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.579+0000 7f81faffd640 1 -- 172.21.15.26:0/1128426127 >> v1:172.21.15.117:6789/0 conn(0x7f81fc0720a0 legacy=0x7f81fc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.579+0000 7f81faffd640 1 -- 172.21.15.26:0/1128426127 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc06e730 con 0x7f81fc152520 2024-03-06T07:24:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.579+0000 7f81faffd640 1 --2- 172.21.15.26:0/1128426127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc152520 0x7f81fc06ef30 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009870 tx=0x7f81e402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ad8c3b30edb923 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f81f9ffb640 1 -- 172.21.15.26:0/1128426127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4034070 con 0x7f81fc152520 2024-03-06T07:24:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f81f9ffb640 1 -- 172.21.15.26:0/1128426127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4030840 con 0x7f81fc152520 2024-03-06T07:24:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f81f9ffb640 1 -- 172.21.15.26:0/1128426127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4030c10 con 0x7f81fc152520 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 -- 172.21.15.26:0/1128426127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc152520 msgr2=0x7f81fc06ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 --2- 172.21.15.26:0/1128426127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc152520 0x7f81fc06ef30 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009870 tx=0x7f81e402b980 comp rx=0 tx=0).stop 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 -- 172.21.15.26:0/1128426127 shutdown_connections 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 --2- 172.21.15.26:0/1128426127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc155c80 0x7f81fc0c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 --2- 172.21.15.26:0/1128426127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc152520 0x7f81fc06ef30 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 -- 172.21.15.26:0/1128426127 >> 172.21.15.26:0/1128426127 conn(0x7f81fc0c76b0 msgr2=0x7f81fc0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 -- 172.21.15.26:0/1128426127 shutdown_connections 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.580+0000 7f820178a640 1 -- 172.21.15.26:0/1128426127 wait complete. 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.581+0000 7f820178a640 1 Processor -- start 2024-03-06T07:24:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.581+0000 7f820178a640 1 -- start start 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f820178a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc0720a0 0x7f81fc0c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f820178a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81fc152520 0x7f81fc1ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f820178a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc155c80 0x7f81fc1ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f820178a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc06b6d0 con 0x7f81fc155c80 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f820178a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0c17f0 con 0x7f81fc152520 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f820178a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc06b850 con 0x7f81fc0720a0 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f81fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc155c80 0x7f81fc1ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f81fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc155c80 0x7f81fc1ee9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35642/0 (socket says 172.21.15.26:35642) 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f81fb7fe640 1 -- 172.21.15.26:0/3622061832 learned_addr learned my addr 172.21.15.26:0/3622061832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f81faffd640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc0720a0 0x7f81fc0c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.582+0000 7f81fa7fc640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81fc152520 0x7f81fc1ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81fb7fe640 1 -- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc0720a0 msgr2=0x7f81fc0c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81fb7fe640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc0720a0 0x7f81fc0c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81fb7fe640 1 -- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81fc152520 msgr2=0x7f81fc1ee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81fb7fe640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81fc152520 0x7f81fc1ee4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81fb7fe640 1 -- 172.21.15.26:0/3622061832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc1e1490 con 0x7f81fc155c80 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81fb7fe640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc155c80 0x7f81fc1ee9f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c6b0 tx=0x7f81f400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81f37fe640 1 -- 172.21.15.26:0/3622061832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003b70 con 0x7f81fc155c80 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81f37fe640 1 -- 172.21.15.26:0/3622061832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4003d30 con 0x7f81fc155c80 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc1e0650 con 0x7f81fc155c80 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f81f37fe640 1 -- 172.21.15.26:0/3622061832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4005310 con 0x7f81fc155c80 2024-03-06T07:24:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.583+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc1e0c10 con 0x7f81fc155c80 2024-03-06T07:24:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.584+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc04f330 con 0x7f81fc155c80 2024-03-06T07:24:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.584+0000 7f81f37fe640 1 -- 172.21.15.26:0/3622061832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81f40054b0 con 0x7f81fc155c80 2024-03-06T07:24:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.585+0000 7f81f37fe640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f81dc06b4d0 0x7f81dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.585+0000 7f81f37fe640 1 -- 172.21.15.26:0/3622061832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81f408a160 con 0x7f81fc155c80 2024-03-06T07:24:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.585+0000 7f81faffd640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f81dc06b4d0 0x7f81dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.586+0000 7f81faffd640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f81dc06b4d0 0x7f81dc06d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f81e4036000 tx=0x7f81e403b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.589+0000 7f81f37fe640 1 -- 172.21.15.26:0/3622061832 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f81f4058ee0 con 0x7f81fc155c80 2024-03-06T07:24:50.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.807+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81fc0c0b70 con 0x7f81fc155c80 2024-03-06T07:24:50.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.807+0000 7f81f37fe640 1 -- 172.21.15.26:0/3622061832 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f81f40587e0 con 0x7f81fc155c80 2024-03-06T07:24:50.810 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:50.810 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1856740515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1856740515},{"type":"v1","addr":"172.21.15.26:6834","nonce":1856740515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:24:45.977307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/577277314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":577277314},{"type":"v1","addr":"172.21.15.26:6835","nonce":577277314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-03-06T07:24:50.810 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 105 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f81dc06b4d0 msgr2=0x7f81dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f81dc06b4d0 0x7f81dc06d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f81e4036000 tx=0x7f81e403b040 comp rx=0 tx=0).stop 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc155c80 msgr2=0x7f81fc1ee9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc155c80 0x7f81fc1ee9f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c6b0 tx=0x7f81f400cb70 comp rx=0 tx=0).stop 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 shutdown_connections 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f81dc06b4d0 0x7f81dc06d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81fc155c80 0x7f81fc1ee9f0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81fc152520 0x7f81fc1ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 --2- 172.21.15.26:0/3622061832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81fc0720a0 0x7f81fc0c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 >> 172.21.15.26:0/3622061832 conn(0x7f81fc0c76b0 msgr2=0x7f81fc0c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 shutdown_connections 2024-03-06T07:24:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:50.809+0000 7f820178a640 1 -- 172.21.15.26:0/3622061832 wait complete. 2024-03-06T07:24:51.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:24:45.977307+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7854}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7854': {'gid': 7854, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/577277314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 577277314}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 577277314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7854, 'qdb_cluster': [7854]} 2024-03-06T07:24:51.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:51.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:24:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.325+0000 7f2a866fb640 1 Processor -- start 2024-03-06T07:24:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.325+0000 7f2a866fb640 1 -- start start 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.326+0000 7f2a866fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 0x7f2a80153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.326+0000 7f2a866fb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a80070550 con 0x7f2a8006d990 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.326+0000 7f2a866fb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a80070830 con 0x7f2a800c0fa0 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.326+0000 7f2a866fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80070b10 con 0x7f2a80072f00 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.326+0000 7f2a84c98640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a8006d990 0x7f2a8015d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55856/0 (socket says 172.21.15.26:55856) 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.326+0000 7f2a84c98640 1 -- 172.21.15.26:0/3876803913 learned_addr learned my addr 172.21.15.26:0/3876803913 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 0x7f2a80153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322559391 0 0) 0x7f2a80070550 con 0x7f2a8006d990 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a68003520 con 0x7f2a8006d990 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 893615012 0 0) 0x7f2a68003520 con 0x7f2a8006d990 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 msgr2=0x7f2a80153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 0x7f2a80153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 >> v1:172.21.15.117:6789/0 conn(0x7f2a800c0fa0 legacy=0x7f2a800c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800732e0 con 0x7f2a8006d990 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f2a780019f0 con 0x7f2a8006d990 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 >> v1:172.21.15.26:6789/0 conn(0x7f2a8006d990 legacy=0x7f2a8015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a68003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 0x7f2a68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68004d40 0x7f2a680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80070b10 con 0x7f2a680039e0 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6800ba70 con 0x7f2a68004d40 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6800bc70 con 0x7f2a80072f00 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a68003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a84c98640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68004d40 0x7f2a680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 0x7f2a68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7ffff640 1 -- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 msgr2=0x7f2a68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a80072f00 0x7f2a68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7ffff640 1 -- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68004d40 msgr2=0x7f2a680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68004d40 0x7f2a680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a7ffff640 1 -- 172.21.15.26:0/3876803913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68003520 con 0x7f2a680039e0 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.327+0000 7f2a84c98640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68004d40 0x7f2a680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.328+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a68003e00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009a90 tx=0x7f2a68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dec1964574b289 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.328+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70039070 con 0x7f2a680039e0 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.328+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7002c680 con 0x7f2a680039e0 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.328+0000 7f2a7effd640 1 -- 172.21.15.26:0/3876803913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70009690 con 0x7f2a680039e0 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.329+0000 7f2a866fb640 1 -- 172.21.15.26:0/3876803913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 msgr2=0x7f2a68003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.329+0000 7f2a866fb640 1 --2- 172.21.15.26:0/3876803913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a68003e00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009a90 tx=0x7f2a68005830 comp rx=0 tx=0).stop 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.329+0000 7f2a84c98640 1 -- 172.21.15.26:0/3876803913 reap_dead start 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.329+0000 7f2a866fb640 1 -- 172.21.15.26:0/3876803913 shutdown_connections 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.329+0000 7f2a866fb640 1 -- 172.21.15.26:0/3876803913 >> 172.21.15.26:0/3876803913 conn(0x7f2a8006b1a0 msgr2=0x7f2a8006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.329+0000 7f2a866fb640 1 -- 172.21.15.26:0/3876803913 shutdown_connections 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.329+0000 7f2a866fb640 1 -- 172.21.15.26:0/3876803913 wait complete. 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.331+0000 7f2a866fb640 1 Processor -- start 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.331+0000 7f2a866fb640 1 -- start start 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a866fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a801ebfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a866fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a801ea160 0x7f2a801ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a866fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a801eab00 0x7f2a8014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a866fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80070f70 con 0x7f2a680039e0 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a866fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80070df0 con 0x7f2a801eab00 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a866fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80072400 con 0x7f2a801ea160 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a801ebfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a801ebfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35670/0 (socket says 172.21.15.26:35670) 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1459527413 learned_addr learned my addr 172.21.15.26:0/1459527413 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a801ea160 0x7f2a801ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a84c98640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a801eab00 0x7f2a8014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a801ea160 msgr2=0x7f2a801ea5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a801ea160 0x7f2a801ea5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a801eab00 msgr2=0x7f2a8014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a801eab00 0x7f2a8014f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1459527413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a801dd5a0 con 0x7f2a680039e0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.332+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a801ebfb0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2a70000c00 tx=0x7f2a70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.333+0000 7f2a7cff9640 1 -- 172.21.15.26:0/1459527413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70042020 con 0x7f2a680039e0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.333+0000 7f2a7cff9640 1 -- 172.21.15.26:0/1459527413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7003a040 con 0x7f2a680039e0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.333+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a801dd2c0 con 0x7f2a680039e0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.333+0000 7f2a7cff9640 1 -- 172.21.15.26:0/1459527413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70039070 con 0x7f2a680039e0 2024-03-06T07:24:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.333+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a801ecb40 con 0x7f2a680039e0 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.334+0000 7f2a7cff9640 1 -- 172.21.15.26:0/1459527413 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a70045020 con 0x7f2a680039e0 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.335+0000 7f2a7cff9640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a6c06b500 0x7f2a6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.335+0000 7f2a7cff9640 1 -- 172.21.15.26:0/1459527413 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a700ab780 con 0x7f2a680039e0 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.335+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a6c06b500 0x7f2a6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.335+0000 7f2a893c1740 1 Processor -- start 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a893c1740 1 -- start start 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a6c06b500 0x7f2a6c06d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2a80151170 tx=0x7f2a74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f506c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506c4550 0x555f5070dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506be680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a893c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f506286b0 con 0x555f5070d800 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f505d1a60 con 0x555f506fe420 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f50624750 con 0x555f506c4550 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f506c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f506c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35676/0 (socket says 172.21.15.26:35676) 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1006156874 learned_addr learned my addr 172.21.15.26:0/1006156874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a84c98640 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.336+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506c4550 0x555f5070dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506c4550 msgr2=0x555f5070dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506c4550 0x555f5070dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 msgr2=0x555f506be680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506be680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a7ffff640 1 -- 172.21.15.26:0/1006156874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f505eff30 con 0x555f5070d800 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506c4550 0x555f5070dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f506c4010 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2a70081f50 tx=0x7f2a700798e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e00815fee59e62d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a65ffb640 1 -- 172.21.15.26:0/1006156874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70079ce0 con 0x555f5070d800 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a65ffb640 1 -- 172.21.15.26:0/1006156874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7007de30 con 0x555f5070d800 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a65ffb640 1 -- 172.21.15.26:0/1006156874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70082480 con 0x555f5070d800 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a893c1740 1 -- 172.21.15.26:0/1006156874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 msgr2=0x555f506c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a893c1740 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f506c4010 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2a70081f50 tx=0x7f2a700798e0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a893c1740 1 -- 172.21.15.26:0/1006156874 shutdown_connections 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a893c1740 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506be680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a893c1740 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506c4550 0x555f5070dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a893c1740 1 --2- 172.21.15.26:0/1006156874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f506c4010 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.337+0000 7f2a893c1740 1 -- 172.21.15.26:0/1006156874 >> 172.21.15.26:0/1006156874 conn(0x555f506c07d0 msgr2=0x555f506e53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.338+0000 7f2a893c1740 1 -- 172.21.15.26:0/1006156874 shutdown_connections 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.338+0000 7f2a893c1740 1 -- 172.21.15.26:0/1006156874 wait complete. 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.338+0000 7f2a893c1740 1 -- start start 2024-03-06T07:24:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.339+0000 7f2a893c1740 1 Processor -- start 2024-03-06T07:24:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f5070b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f50715d30 0x555f506e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a893c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f50624750 con 0x555f5070d800 2024-03-06T07:24:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f505e4ea0 con 0x555f506fe420 2024-03-06T07:24:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f506286b0 con 0x555f50715d30 2024-03-06T07:24:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a893c1740 10 client.0 _ll_register_callbacks cb 0x555f505e4870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f5070b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f5070b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35692/0 (socket says 172.21.15.26:35692) 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/3630753803 learned_addr learned my addr 172.21.15.26:0/3630753803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a84c98640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f50715d30 0x555f506e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f50715d30 msgr2=0x555f506e45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f50715d30 0x555f506e45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 msgr2=0x555f506e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/3630753803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f505e5270 con 0x555f5070d800 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f5070b3a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2a74001e30 tx=0x7f2a74001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315f224074079e16 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.340+0000 7f2a5effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a5effd640 1 -- 172.21.15.26:0/3630753803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74029060 con 0x555f5070d800 2024-03-06T07:24:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a5effd640 1 -- 172.21.15.26:0/3630753803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a74024c40 con 0x555f5070d800 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a7007de30 con 0x555f5070d800 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a5effd640 1 -- 172.21.15.26:0/3630753803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a74024e40 con 0x555f5070d800 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 10 client.8034 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x555f5033d2d0 con 0x555f5070d800 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a5effd640 1 -- 172.21.15.26:0/3630753803 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7007de30 con 0x555f5070d800 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a5effd640 1 -- 172.21.15.26:0/3630753803 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f2a740143d0 con 0x555f5070d800 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 10 client.8034 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 10 client.8034 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 20 client.8034 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50718ce0 0x555f50723bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 4 client.8034 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 --> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f50635bf0 con 0x555f50718ce0 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50718ce0 0x555f50723bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a5effd640 10 client.8034 ms_handle_connect on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.341+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50718ce0 0x555f50723bf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48016f0f0c4017cb server_cookie=5c273bac36e1ea7d in_seq=0 out_seq=0 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a5effd640 1 -- 172.21.15.26:0/3630753803 <== mds.0 v2:172.21.15.26:6833/577277314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f50635bf0 con 0x555f50718ce0 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a5effd640 10 client.8034 handle_command_reply: tid=0 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 client.8034 shutdown 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 msgr2=0x555f5070b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f5070b3a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2a74001e30 tx=0x7f2a74001120 comp rx=0 tx=0).stop 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 shutdown_connections 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50718ce0 0x555f50723bf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f50715d30 0x555f506e45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a5effd640 0 client.8034 ms_handle_reset on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f5070b3a0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3630753803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f506fe420 0x555f506e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 >> 172.21.15.26:0/3630753803 conn(0x555f506c4550 msgr2=0x555f506bf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 shutdown_connections 2024-03-06T07:24:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.342+0000 7f2a893c1740 1 -- 172.21.15.26:0/3630753803 wait complete. 2024-03-06T07:24:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.343+0000 7f2a893c1740 20 client.8034 trim_cache size 0 max 16384 2024-03-06T07:24:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.375+0000 7f2a893c1740 1 Processor -- start 2024-03-06T07:24:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.375+0000 7f2a893c1740 1 -- start start 2024-03-06T07:24:51.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f506fe420 0x555f507001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f5070d800 0x555f506bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f506bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a893c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801f3300 con 0x555f506fe420 2024-03-06T07:24:51.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8014e4b0 con 0x555f50715d30 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800ab5f0 con 0x555f5070d800 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f506fe420 0x555f507001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f506fe420 0x555f507001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35706/0 (socket says 172.21.15.26:35706) 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 -- 172.21.15.26:0/3849530587 learned_addr learned my addr 172.21.15.26:0/3849530587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a84c98640 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f506bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f5070d800 0x555f506bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 -- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f5070d800 msgr2=0x555f506bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f5070d800 0x555f506bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 -- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 msgr2=0x555f506bfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f506bfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.376+0000 7f2a7ffff640 1 -- 172.21.15.26:0/3849530587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f504076b0 con 0x555f506fe420 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f506fe420 0x555f507001e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2a700799e0 tx=0x7f2a700ab000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdd6ab6439d0110 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a657fa640 1 -- 172.21.15.26:0/3849530587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7007de30 con 0x555f506fe420 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a657fa640 1 -- 172.21.15.26:0/3849530587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a70077fe0 con 0x555f506fe420 2024-03-06T07:24:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a657fa640 1 -- 172.21.15.26:0/3849530587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70081ff0 con 0x555f506fe420 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 -- 172.21.15.26:0/3849530587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f506fe420 msgr2=0x555f507001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f506fe420 0x555f507001e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2a700799e0 tx=0x7f2a700ab000 comp rx=0 tx=0).stop 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 -- 172.21.15.26:0/3849530587 shutdown_connections 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f506bfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f5070d800 0x555f506bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 --2- 172.21.15.26:0/3849530587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f506fe420 0x555f507001e0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 -- 172.21.15.26:0/3849530587 >> 172.21.15.26:0/3849530587 conn(0x555f506c4550 msgr2=0x555f506e5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 -- 172.21.15.26:0/3849530587 shutdown_connections 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.377+0000 7f2a893c1740 1 -- 172.21.15.26:0/3849530587 wait complete. 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.378+0000 7f2a893c1740 1 -- start start 2024-03-06T07:24:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.378+0000 7f2a893c1740 1 Processor -- start 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506fe420 0x555f50728910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f50703490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f50706f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5070b9e0 con 0x555f5070d800 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f505d1a60 con 0x555f50715d30 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f506e27c0 con 0x555f506fe420 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 10 client.0 _ll_register_callbacks cb 0x7f2a740017a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f50703490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f50703490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35718/0 (socket says 172.21.15.26:35718) 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/2529975925 learned_addr learned my addr 172.21.15.26:0/2529975925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506fe420 0x555f50728910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a84c98640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f50706f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506fe420 msgr2=0x555f50728910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506fe420 0x555f50728910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 msgr2=0x555f50706f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f50706f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 -- 172.21.15.26:0/2529975925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f50703c60 con 0x555f5070d800 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a84c98640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f50706f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a7f7fe640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f50703490 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x555f506e4130 tx=0x7f2a74015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d6fe7e6862b35c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a5effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a5effd640 1 -- 172.21.15.26:0/2529975925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7403c060 con 0x555f5070d800 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a5effd640 1 -- 172.21.15.26:0/2529975925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a7402f430 con 0x555f5070d800 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.379+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a70077fe0 con 0x555f5070d800 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a5effd640 1 -- 172.21.15.26:0/2529975925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a74042050 con 0x555f5070d800 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 10 client.8040 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x555f507074b0 con 0x555f5070d800 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a5effd640 1 -- 172.21.15.26:0/2529975925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a70077fe0 con 0x555f5070d800 2024-03-06T07:24:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a5effd640 1 -- 172.21.15.26:0/2529975925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f2a74045020 con 0x555f5070d800 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 10 client.8040 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 10 client.8040 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 20 client.8040 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50771a10 0x555f507774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 4 client.8040 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 --> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555f5062ff40 con 0x555f50771a10 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50771a10 0x555f507774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a5effd640 10 client.8040 ms_handle_connect on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.380+0000 7f2a7ffff640 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50771a10 0x555f507774d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a95988c7c4ebd81 server_cookie=8f152ef03326bfaf in_seq=0 out_seq=0 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a5effd640 1 -- 172.21.15.26:0/2529975925 <== mds.0 v2:172.21.15.26:6833/577277314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555f5062ff40 con 0x555f50771a10 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a5effd640 10 client.8040 handle_command_reply: tid=0 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 client.8040 shutdown 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 msgr2=0x555f50703490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f50703490 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x555f506e4130 tx=0x7f2a74015000 comp rx=0 tx=0).stop 2024-03-06T07:24:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 shutdown_connections 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555f50771a10 0x555f507774d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f50715d30 0x555f50706f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f5070d800 0x555f50703490 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 --2- 172.21.15.26:0/2529975925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555f506fe420 0x555f50728910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a5effd640 0 client.8040 ms_handle_reset on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 >> 172.21.15.26:0/2529975925 conn(0x555f506c4550 msgr2=0x555f506ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 shutdown_connections 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 1 -- 172.21.15.26:0/2529975925 wait complete. 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.381+0000 7f2a893c1740 20 client.8040 trim_cache size 0 max 16384 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7854, 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 7.444999911, 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 105, 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 121, 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 121, 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 12.533844031999999 2024-03-06T07:24:51.390 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a6c06b500 msgr2=0x7f2a6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a6c06b500 0x7f2a6c06d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2a80151170 tx=0x7f2a74009040 comp rx=0 tx=0).stop 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 msgr2=0x7f2a801ebfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a801ebfb0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2a70000c00 tx=0x7f2a70004730 comp rx=0 tx=0).stop 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 shutdown_connections 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a6c06b500 0x7f2a6c06d9c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a801eab00 0x7f2a8014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a801ea160 0x7f2a801ea5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 --2- 172.21.15.26:0/1459527413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a680039e0 0x7f2a801ebfb0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 >> 172.21.15.26:0/1459527413 conn(0x7f2a8006b1a0 msgr2=0x7f2a80153890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 shutdown_connections 2024-03-06T07:24:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:51.382+0000 7f2a866fb640 1 -- 172.21.15.26:0/1459527413 wait complete. 2024-03-06T07:24:51.395 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 105, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:43.287707+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 105, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7854": { "addr": "172.21.15.26:6835/577277314", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 577277314, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 577277314, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7854, "incarnation": 102, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:24:45.977307+0000", "qdb_cluster": [ 7854 ], "qdb_leader": 7854, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7854 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/3943022187", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3943022187, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3943022187, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7465, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3949772965", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3949772965, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3949772965, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7468, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/1856740515", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 1856740515, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 1856740515, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7851, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:51.396 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 9, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:51.396 INFO:tasks.cephfs.test_failover:current = [{'gid': 7851, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/1856740515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1856740515}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1856740515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}, {'gid': 7465, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3943022187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3943022187}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3943022187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}, {'gid': 7854, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/577277314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 577277314}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 577277314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7468, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3949772965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3949772965}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3949772965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}] 2024-03-06T07:24:51.396 DEBUG:tasks.cephfs.test_failover:9 != -1 2024-03-06T07:24:51.396 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:24:56.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:56.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.497+0000 7fb46af93640 1 Processor -- start 2024-03-06T07:24:56.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.497+0000 7fb46af93640 1 -- start start 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.497+0000 7fb46af93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb4640c7180 0x7fb464159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.497+0000 7fb46af93640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4640c88b0 0x7fb4640c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.497+0000 7fb46af93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640c7c60 con 0x7fb4641527b0 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.497+0000 7fb46af93640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb464159640 con 0x7fb4640c88b0 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.497+0000 7fb46af93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640c8060 con 0x7fb4640c7180 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb469530640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4640c88b0 0x7fb4640c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb468d2f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb4641527b0 0x7fb464155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55870/0 (socket says 172.21.15.26:55870) 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb468d2f640 1 -- 172.21.15.26:0/2218440949 learned_addr learned my addr 172.21.15.26:0/2218440949 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb463fff640 1 --2- 172.21.15.26:0/2218440949 >> v2:172.21.15.117:3300/0 conn(0x7fb4640c7180 0x7fb464159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb469530640 1 -- 172.21.15.26:0/2218440949 >> v2:172.21.15.117:3300/0 conn(0x7fb4640c7180 msgr2=0x7fb464159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb469530640 1 --2- 172.21.15.26:0/2218440949 >> v2:172.21.15.117:3300/0 conn(0x7fb4640c7180 0x7fb464159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb469530640 1 -- 172.21.15.26:0/2218440949 >> v1:172.21.15.26:6789/0 conn(0x7fb4641527b0 legacy=0x7fb464155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb469530640 1 -- 172.21.15.26:0/2218440949 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640c76a0 con 0x7fb4640c88b0 2024-03-06T07:24:56.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.498+0000 7fb4637fe640 1 -- 172.21.15.26:0/2218440949 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180334947 0 0) 0x7fb458001110 con 0x7fb4641527b0 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb469530640 1 --2- 172.21.15.26:0/2218440949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4640c88b0 0x7fb4640c5330 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb46408cac0 tx=0x7fb45c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18935a8b01eda80a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb4637fe640 1 -- 172.21.15.26:0/2218440949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c039070 con 0x7fb4640c88b0 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb4637fe640 1 -- 172.21.15.26:0/2218440949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c02cbf0 con 0x7fb4640c88b0 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb4637fe640 1 -- 172.21.15.26:0/2218440949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c034440 con 0x7fb4640c88b0 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 -- 172.21.15.26:0/2218440949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4640c88b0 msgr2=0x7fb4640c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 --2- 172.21.15.26:0/2218440949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4640c88b0 0x7fb4640c5330 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb46408cac0 tx=0x7fb45c02be40 comp rx=0 tx=0).stop 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 -- 172.21.15.26:0/2218440949 shutdown_connections 2024-03-06T07:24:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 --2- 172.21.15.26:0/2218440949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4640c88b0 0x7fb4640c5330 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 --2- 172.21.15.26:0/2218440949 >> v2:172.21.15.117:3300/0 conn(0x7fb4640c7180 0x7fb464159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 -- 172.21.15.26:0/2218440949 >> 172.21.15.26:0/2218440949 conn(0x7fb4640ac560 msgr2=0x7fb4640ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 -- 172.21.15.26:0/2218440949 shutdown_connections 2024-03-06T07:24:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.499+0000 7fb46af93640 1 -- 172.21.15.26:0/2218440949 wait complete. 2024-03-06T07:24:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.500+0000 7fb46af93640 1 Processor -- start 2024-03-06T07:24:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb46af93640 1 -- start start 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb46af93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640c7180 0x7fb4640bf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb46af93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4640c88b0 0x7fb4641ec330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb46af93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4641527b0 0x7fb4641ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb46af93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464155600 con 0x7fb4641527b0 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb46af93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464155320 con 0x7fb4640c7180 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb46af93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb464152b90 con 0x7fb4640c88b0 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb469530640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4641527b0 0x7fb4641ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb469530640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4641527b0 0x7fb4641ee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35732/0 (socket says 172.21.15.26:35732) 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.501+0000 7fb469530640 1 -- 172.21.15.26:0/1563672467 learned_addr learned my addr 172.21.15.26:0/1563672467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb468d2f640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640c7180 0x7fb4640bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb463fff640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4640c88b0 0x7fb4641ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb469530640 1 -- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4640c88b0 msgr2=0x7fb4641ec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb469530640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4640c88b0 0x7fb4641ec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb469530640 1 -- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640c7180 msgr2=0x7fb4640bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb469530640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640c7180 0x7fb4640bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb469530640 1 -- 172.21.15.26:0/1563672467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640c0e50 con 0x7fb4641527b0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb469530640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4641527b0 0x7fb4641ee6a0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e80 tx=0x7fb45c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb4617fa640 1 -- 172.21.15.26:0/1563672467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c039050 con 0x7fb4641527b0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb4617fa640 1 -- 172.21.15.26:0/1563672467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c045070 con 0x7fb4641527b0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb4617fa640 1 -- 172.21.15.26:0/1563672467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c03e830 con 0x7fb4641527b0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4640c1130 con 0x7fb4641527b0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.502+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4640c0100 con 0x7fb4641527b0 2024-03-06T07:24:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.503+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46404f330 con 0x7fb4641527b0 2024-03-06T07:24:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.504+0000 7fb4617fa640 1 -- 172.21.15.26:0/1563672467 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb45c03e9d0 con 0x7fb4641527b0 2024-03-06T07:24:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.504+0000 7fb4617fa640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb44806b4d0 0x7fb44806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.504+0000 7fb4617fa640 1 -- 172.21.15.26:0/1563672467 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb45c0ad090 con 0x7fb4641527b0 2024-03-06T07:24:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.504+0000 7fb468d2f640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb44806b4d0 0x7fb44806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.505+0000 7fb468d2f640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb44806b4d0 0x7fb44806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb458007fd0 tx=0x7fb458007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.513+0000 7fb4617fa640 1 -- 172.21.15.26:0/1563672467 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb45c07b8e0 con 0x7fb4641527b0 2024-03-06T07:24:56.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.736+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb46406ce00 con 0x7fb4641527b0 2024-03-06T07:24:56.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.737+0000 7fb4617fa640 1 -- 172.21.15.26:0/1563672467 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4987 (secure 0 0 0) 0x7fb45c07f980 con 0x7fb4641527b0 2024-03-06T07:24:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:56.740 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":106,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1856740515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1856740515},{"type":"v1","addr":"172.21.15.26:6834","nonce":1856740515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:24:51.962771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/577277314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":577277314},{"type":"v1","addr":"172.21.15.26:6835","nonce":577277314}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-03-06T07:24:56.740 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 106 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb44806b4d0 msgr2=0x7fb44806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb44806b4d0 0x7fb44806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb458007fd0 tx=0x7fb458007de0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4641527b0 msgr2=0x7fb4641ee6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4641527b0 0x7fb4641ee6a0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e80 tx=0x7fb45c004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 shutdown_connections 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb44806b4d0 0x7fb44806d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4641527b0 0x7fb4641ee6a0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4640c88b0 0x7fb4641ec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 --2- 172.21.15.26:0/1563672467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640c7180 0x7fb4640bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 >> 172.21.15.26:0/1563672467 conn(0x7fb4640ac560 msgr2=0x7fb4640c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 shutdown_connections 2024-03-06T07:24:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:56.739+0000 7fb46af93640 1 -- 172.21.15.26:0/1563672467 wait complete. 2024-03-06T07:24:57.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:24:51.962771+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7854}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7854': {'gid': 7854, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6835/577277314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 577277314}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 577277314}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7854, 'qdb_cluster': [7854]} 2024-03-06T07:24:57.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:57.159 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:24:57.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.258+0000 7f63b99a2640 1 Processor -- start 2024-03-06T07:24:57.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.259+0000 7f63b99a2640 1 -- start start 2024-03-06T07:24:57.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.259+0000 7f63b99a2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f63b413e490 0x7f63b40723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.259+0000 7f63b99a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b40729c0 con 0x7f63b4155d70 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.259+0000 7f63b99a2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b4155290 con 0x7f63b40c3b20 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.259+0000 7f63b99a2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b41558e0 con 0x7f63b413e490 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.259+0000 7f63b3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f63b4155d70 0x7f63b4159180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33632/0 (socket says 172.21.15.26:33632) 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.259+0000 7f63b3fff640 1 -- 172.21.15.26:0/1236218230 learned_addr learned my addr 172.21.15.26:0/1236218230 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000263829 0 0) 0x7f63b40729c0 con 0x7f63b4155d70 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1236218230 >> v2:172.21.15.117:3300/0 conn(0x7f63b413e490 0x7f63b40723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a0003520 con 0x7f63b4155d70 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590819241 0 0) 0x7f63a0003520 con 0x7f63b4155d70 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 >> v2:172.21.15.117:3300/0 conn(0x7f63b413e490 msgr2=0x7f63b40723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 --2- 172.21.15.26:0/1236218230 >> v2:172.21.15.117:3300/0 conn(0x7f63b413e490 0x7f63b40723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 >> v1:172.21.15.117:6789/0 conn(0x7f63b40c3b20 legacy=0x7f63b40c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4155570 con 0x7f63b4155d70 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f63a40019f0 con 0x7f63b4155d70 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 >> v1:172.21.15.26:6789/0 conn(0x7f63b4155d70 legacy=0x7f63b4159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a0004800 0x7f63a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a00051c0 0x7f63a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41558e0 con 0x7f63a00039b0 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a000c060 con 0x7f63a00051c0 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a000c260 con 0x7f63a0004800 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b91a1640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a00051c0 0x7f63a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a0004800 0x7f63a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b89a0640 1 -- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a0004800 msgr2=0x7f63a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a0004800 0x7f63a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b89a0640 1 -- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a00051c0 msgr2=0x7f63a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.260+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a00051c0 0x7f63a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.261+0000 7f63b89a0640 1 -- 172.21.15.26:0/1236218230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0003520 con 0x7f63a00039b0 2024-03-06T07:24:57.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.261+0000 7f63b91a1640 1 -- 172.21.15.26:0/1236218230 reap_dead start 2024-03-06T07:24:57.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.261+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a0004800 0x7f63a0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:57.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63a0003dd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f639c009ac0 tx=0x7f63a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c546ecf25480ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c039070 con 0x7f63a00039b0 2024-03-06T07:24:57.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c02c6b0 con 0x7f63a00039b0 2024-03-06T07:24:57.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b37fe640 1 -- 172.21.15.26:0/1236218230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c009690 con 0x7f63a00039b0 2024-03-06T07:24:57.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b99a2640 1 -- 172.21.15.26:0/1236218230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 msgr2=0x7f63a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63a0003dd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f639c009ac0 tx=0x7f63a0005830 comp rx=0 tx=0).stop 2024-03-06T07:24:57.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b99a2640 1 -- 172.21.15.26:0/1236218230 shutdown_connections 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1236218230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63a0003dd0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b99a2640 1 -- 172.21.15.26:0/1236218230 >> 172.21.15.26:0/1236218230 conn(0x7f63b406afc0 msgr2=0x7f63b406b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b99a2640 1 -- 172.21.15.26:0/1236218230 shutdown_connections 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.262+0000 7f63b99a2640 1 -- 172.21.15.26:0/1236218230 wait complete. 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.263+0000 7f63b99a2640 1 Processor -- start 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b99a2640 1 -- start start 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b99a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63b40c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b99a2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63b4157800 0x7f63b4156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b99a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b40c82c0 0x7f63b4151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b99a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40c4130 con 0x7f63a00039b0 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b99a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40c3fb0 con 0x7f63b40c82c0 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b99a2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4152d70 con 0x7f63b4157800 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b89a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63b40c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b89a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63b40c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52618/0 (socket says 172.21.15.26:52618) 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.264+0000 7f63b89a0640 1 -- 172.21.15.26:0/1012137070 learned_addr learned my addr 172.21.15.26:0/1012137070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63b4157800 0x7f63b4156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b91a1640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b40c82c0 0x7f63b4151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b89a0640 1 -- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63b4157800 msgr2=0x7f63b4156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63b4157800 0x7f63b4156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b89a0640 1 -- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b40c82c0 msgr2=0x7f63b4151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b40c82c0 0x7f63b4151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b89a0640 1 -- 172.21.15.26:0/1012137070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4158d00 con 0x7f63a00039b0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63b40c7c60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f639c037040 tx=0x7f639c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b17fa640 1 -- 172.21.15.26:0/1012137070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c042020 con 0x7f63a00039b0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b17fa640 1 -- 172.21.15.26:0/1012137070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c03a040 con 0x7f63a00039b0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b17fa640 1 -- 172.21.15.26:0/1012137070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c039070 con 0x7f63a00039b0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b40c7460 con 0x7f63a00039b0 2024-03-06T07:24:57.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.265+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b40c7010 con 0x7f63a00039b0 2024-03-06T07:24:57.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.267+0000 7f63b17fa640 1 -- 172.21.15.26:0/1012137070 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f639c045020 con 0x7f63a00039b0 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.267+0000 7f63b17fa640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f639406b500 0x7f639406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.267+0000 7f63b17fa640 1 -- 172.21.15.26:0/1012137070 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f639c0ab710 con 0x7f63a00039b0 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.268+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f639406b500 0x7f639406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.268+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f639406b500 0x7f639406d9c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f63b4136f60 tx=0x7f63a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.268+0000 7f63bd7e0740 1 Processor -- start 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.268+0000 7f63bd7e0740 1 -- start start 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac3100 0x555a0eac0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac34e0 0x555a0eac1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac1660 0x555a0eabd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0ea276b0 con 0x555a0eac1660 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0e9d0a60 con 0x555a0eac34e0 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a0ea23750 con 0x555a0eac3100 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac1660 0x555a0eabd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac1660 0x555a0eabd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52622/0 (socket says 172.21.15.26:52622) 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 -- 172.21.15.26:0/3921825062 learned_addr learned my addr 172.21.15.26:0/3921825062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b89a0640 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac3100 0x555a0eac0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b3fff640 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac34e0 0x555a0eac1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 -- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac3100 msgr2=0x555a0eac0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac3100 0x555a0eac0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 -- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac34e0 msgr2=0x555a0eac1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac34e0 0x555a0eac1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 -- 172.21.15.26:0/3921825062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0e9eef30 con 0x555a0eac1660 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b3fff640 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac34e0 0x555a0eac1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.269+0000 7f63b91a1640 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac1660 0x555a0eabd830 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f63b4074b90 tx=0x7f63ac007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9d36c3d87be580 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63aa7fc640 1 -- 172.21.15.26:0/3921825062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac01b070 con 0x555a0eac1660 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63aa7fc640 1 -- 172.21.15.26:0/3921825062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac001560 con 0x555a0eac1660 2024-03-06T07:24:57.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63aa7fc640 1 -- 172.21.15.26:0/3921825062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac001930 con 0x555a0eac1660 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3921825062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac1660 msgr2=0x555a0eabd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac1660 0x555a0eabd830 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f63b4074b90 tx=0x7f63ac007c80 comp rx=0 tx=0).stop 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3921825062 shutdown_connections 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac1660 0x555a0eabd830 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac34e0 0x555a0eac1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3921825062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac3100 0x555a0eac0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3921825062 >> 172.21.15.26:0/3921825062 conn(0x555a0eabf980 msgr2=0x555a0eae4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3921825062 shutdown_connections 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.270+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3921825062 wait complete. 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.271+0000 7f63bd7e0740 1 -- start start 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.271+0000 7f63bd7e0740 1 Processor -- start 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac3100 0x555a0eb13d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eadf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eb0fd20 0x555a0eabda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0eae53e0 con 0x555a0eac3100 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0e9e3ea0 con 0x555a0eb0fd20 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a0eae4e20 con 0x555a0eac34e0 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 10 client.0 _ll_register_callbacks cb 0x555a0e9e3870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac3100 0x555a0eb13d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac3100 0x555a0eb13d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52634/0 (socket says 172.21.15.26:52634) 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 -- 172.21.15.26:0/1069022292 learned_addr learned my addr 172.21.15.26:0/1069022292 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eadf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b91a1640 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eb0fd20 0x555a0eabda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 -- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 msgr2=0x555a0eadf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eadf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 -- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eb0fd20 msgr2=0x555a0eabda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eb0fd20 0x555a0eabda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 -- 172.21.15.26:0/1069022292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a0e9e4270 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63b89a0640 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac3100 0x555a0eb13d20 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c0044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cfb15a6bce0eda server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63937fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63937fe640 1 -- 172.21.15.26:0/1069022292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c07de50 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63937fe640 1 -- 172.21.15.26:0/1069022292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f639c081f80 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63ac001560 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.272+0000 7f63937fe640 1 -- 172.21.15.26:0/1069022292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f639c082410 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 10 client.8061 fetch_fsmap learned FSMap version 106 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x555a0e73c2d0 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63937fe640 1 -- 172.21.15.26:0/1069022292 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac001560 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63937fe640 1 -- 172.21.15.26:0/1069022292 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 106) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f639c079150 con 0x555a0eac3100 2024-03-06T07:24:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 10 client.8061 fetch_fsmap finished waiting for FSMap version 106 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 10 client.8061 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 20 client.8061 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eae3b60 0x555a0eae3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 4 client.8061 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 --> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a0ea34bf0 con 0x555a0eae3b60 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.273+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eae3b60 0x555a0eae3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63b3fff640 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eae3b60 0x555a0eae3f60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f300e96fb7116c server_cookie=10ae1b1e6a34c514 in_seq=0 out_seq=0 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63937fe640 10 client.8061 ms_handle_connect on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63937fe640 1 -- 172.21.15.26:0/1069022292 <== mds.0 v2:172.21.15.26:6833/577277314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555a0ea34bf0 con 0x555a0eae3b60 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63937fe640 10 client.8061 handle_command_reply: tid=0 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 client.8061 shutdown 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac3100 msgr2=0x555a0eb13d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac3100 0x555a0eb13d20 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c0044e0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 shutdown_connections 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eae3b60 0x555a0eae3f60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eb0fd20 0x555a0eabda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eadf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63937fe640 0 client.8061 ms_handle_reset on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/1069022292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac3100 0x555a0eb13d20 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 >> 172.21.15.26:0/1069022292 conn(0x555a0eac1660 msgr2=0x555a0eabc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.274+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 shutdown_connections 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.275+0000 7f63bd7e0740 1 -- 172.21.15.26:0/1069022292 wait complete. 2024-03-06T07:24:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.275+0000 7f63bd7e0740 20 client.8061 trim_cache size 0 max 16384 2024-03-06T07:24:57.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.291+0000 7f63bd7e0740 1 Processor -- start 2024-03-06T07:24:57.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.291+0000 7f63bd7e0740 1 -- start start 2024-03-06T07:24:57.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eac0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac34e0 0x555a0eb027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eb0fd20 0x555a0eb02cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41f7450 con 0x555a0eac34e0 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41eea00 con 0x555a0eac3100 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41f6e50 con 0x555a0eb0fd20 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac34e0 0x555a0eb027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac34e0 0x555a0eb027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52638/0 (socket says 172.21.15.26:52638) 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 -- 172.21.15.26:0/4013909463 learned_addr learned my addr 172.21.15.26:0/4013909463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b89a0640 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eac0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b91a1640 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eb0fd20 0x555a0eb02cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 -- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eb0fd20 msgr2=0x555a0eb02cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eb0fd20 0x555a0eb02cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 -- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 msgr2=0x555a0eac0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eac0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b3fff640 1 -- 172.21.15.26:0/4013909463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0e8066b0 con 0x555a0eac34e0 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.292+0000 7f63b91a1640 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eb0fd20 0x555a0eb02cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63b3fff640 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac34e0 0x555a0eb027b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f63b40696e0 tx=0x7f63a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841f533f38a9a047 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63a9ffb640 1 -- 172.21.15.26:0/4013909463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40280b0 con 0x555a0eac34e0 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63a9ffb640 1 -- 172.21.15.26:0/4013909463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a402c020 con 0x555a0eac34e0 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63a9ffb640 1 -- 172.21.15.26:0/4013909463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4016b10 con 0x555a0eac34e0 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 -- 172.21.15.26:0/4013909463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac34e0 msgr2=0x555a0eb027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac34e0 0x555a0eb027b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f63b40696e0 tx=0x7f63a4013000 comp rx=0 tx=0).stop 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 -- 172.21.15.26:0/4013909463 shutdown_connections 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eb0fd20 0x555a0eb02cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eac34e0 0x555a0eb027b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/4013909463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eac0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 -- 172.21.15.26:0/4013909463 >> 172.21.15.26:0/4013909463 conn(0x555a0eac1660 msgr2=0x555a0eb13af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 -- 172.21.15.26:0/4013909463 shutdown_connections 2024-03-06T07:24:57.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.293+0000 7f63bd7e0740 1 -- 172.21.15.26:0/4013909463 wait complete. 2024-03-06T07:24:57.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.294+0000 7f63bd7e0740 1 -- start start 2024-03-06T07:24:57.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.294+0000 7f63bd7e0740 1 Processor -- start 2024-03-06T07:24:57.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.295+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eae0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.295+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eb725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.295+0000 7f63bd7e0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eb0fd20 0x555a0eb72ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.295+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0eae53e0 con 0x555a0eb0fd20 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.295+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0e9e3ea0 con 0x555a0eac3100 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.295+0000 7f63bd7e0740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a0eb106b0 con 0x555a0eac34e0 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63bd7e0740 10 client.0 _ll_register_callbacks cb 0x7f639c079930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eb0fd20 0x555a0eb72ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eb0fd20 0x555a0eb72ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52640/0 (socket says 172.21.15.26:52640) 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 -- 172.21.15.26:0/3833240785 learned_addr learned my addr 172.21.15.26:0/3833240785 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b89a0640 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eae0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b3fff640 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eb725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 -- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 msgr2=0x555a0eb725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eb725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 -- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 msgr2=0x555a0eae0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eae0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 -- 172.21.15.26:0/3833240785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a0eb101c0 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63b91a1640 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eb0fd20 0x555a0eb72ae0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f63ac019040 tx=0x7f63ac001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2027434562c76be server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63937fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63937fe640 1 -- 172.21.15.26:0/3833240785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac027060 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63937fe640 1 -- 172.21.15.26:0/3833240785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63ac015670 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63a402c020 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63937fe640 1 -- 172.21.15.26:0/3833240785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63ac02d050 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63bd7e0740 10 client.8067 fetch_fsmap learned FSMap version 106 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x555a0eb0ce70 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.296+0000 7f63937fe640 1 -- 172.21.15.26:0/3833240785 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a402c020 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63937fe640 1 -- 172.21.15.26:0/3833240785 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 106) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f63ac030020 con 0x555a0eb0fd20 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 10 client.8067 fetch_fsmap finished waiting for FSMap version 106 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 10 client.8067 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 20 client.8067 populate_metadata read hostname 'smithi026' 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eb22aa0 0x555a0eb22e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 4 client.8067 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 --> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a0ea2ef40 con 0x555a0eb22aa0 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63b89a0640 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eb22aa0 0x555a0eb22e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63937fe640 10 client.8067 ms_handle_connect on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63b89a0640 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eb22aa0 0x555a0eb22e80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47621ee4d82d2b5e server_cookie=2d91550e0a897216 in_seq=0 out_seq=0 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63937fe640 1 -- 172.21.15.26:0/3833240785 <== mds.0 v2:172.21.15.26:6833/577277314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x555a0ea2ef40 con 0x555a0eb22aa0 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63937fe640 10 client.8067 handle_command_reply: tid=0 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 client.8067 shutdown 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eb0fd20 msgr2=0x555a0eb72ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eb0fd20 0x555a0eb72ae0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f63ac019040 tx=0x7f63ac001190 comp rx=0 tx=0).stop 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 shutdown_connections 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:6833/577277314,v1:172.21.15.26:6835/577277314] conn(0x555a0eb22aa0 0x555a0eb22e80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a0eb0fd20 0x555a0eb72ae0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555a0eac34e0 0x555a0eb725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 --2- 172.21.15.26:0/3833240785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a0eac3100 0x555a0eae0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 >> 172.21.15.26:0/3833240785 conn(0x555a0eac1660 msgr2=0x555a0eac0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.297+0000 7f63937fe640 0 client.8067 ms_handle_reset on v2:172.21.15.26:6833/577277314 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 shutdown_connections 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63bd7e0740 1 -- 172.21.15.26:0/3833240785 wait complete. 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63bd7e0740 20 client.8067 trim_cache size 0 max 16384 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:24:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7854, 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 13.361595934, 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 106, 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 121, 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 121, 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 18.45044102 2024-03-06T07:24:57.307 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f639406b500 msgr2=0x7f639406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f639406b500 0x7f639406d9c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f63b4136f60 tx=0x7f63a4009040 comp rx=0 tx=0).stop 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 msgr2=0x7f63b40c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63b40c7c60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f639c037040 tx=0x7f639c004730 comp rx=0 tx=0).stop 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 shutdown_connections 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f639406b500 0x7f639406d9c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b40c82c0 0x7f63b4151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63b4157800 0x7f63b4156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 --2- 172.21.15.26:0/1012137070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a00039b0 0x7f63b40c7c60 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 >> 172.21.15.26:0/1012137070 conn(0x7f63b406afc0 msgr2=0x7f63b4154390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 shutdown_connections 2024-03-06T07:24:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.298+0000 7f63b99a2640 1 -- 172.21.15.26:0/1012137070 wait complete. 2024-03-06T07:24:57.312 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 106, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:43.287707+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 106, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7854": { "addr": "172.21.15.26:6835/577277314", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 577277314, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 577277314, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7854, "incarnation": 102, "join_fscid": 9, "name": "c", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:24:51.962771+0000", "qdb_cluster": [ 7854 ], "qdb_leader": 7854, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7854 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/3943022187", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3943022187, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3943022187, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7465, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3949772965", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3949772965, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3949772965, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7468, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/1856740515", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 1856740515, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 1856740515, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7851, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:57.313 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 9, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:57.313 INFO:tasks.cephfs.test_failover:current = [{'gid': 7851, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/1856740515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1856740515}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1856740515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}, {'gid': 7465, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3943022187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3943022187}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3943022187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}, {'gid': 7854, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6835/577277314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 577277314}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 577277314}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7468, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3949772965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3949772965}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3949772965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}] 2024-03-06T07:24:57.313 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:24:57.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 9:0 2024-03-06T07:24:57.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.413+0000 7f69ba650640 1 Processor -- start 2024-03-06T07:24:57.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.413+0000 7f69ba650640 1 -- start start 2024-03-06T07:24:57.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.413+0000 7f69ba650640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f69b40c7170 0x7f69b40c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.413+0000 7f69ba650640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 0x7f69b40c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69ba650640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b40a8390 con 0x7f69b406ff10 2024-03-06T07:24:57.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69ba650640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b40c7620 con 0x7f69b40c7170 2024-03-06T07:24:57.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69ba650640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40730f0 con 0x7f69b40c6c40 2024-03-06T07:24:57.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f69b406ff10 0x7f69b4153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33636/0 (socket says 172.21.15.26:33636) 2024-03-06T07:24:57.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b3fff640 1 -- 172.21.15.26:0/3100229850 learned_addr learned my addr 172.21.15.26:0/3100229850 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b8bed640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 0x7f69b40c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b37fe640 1 --2- 172.21.15.26:0/3100229850 >> v2:172.21.15.117:3300/0 conn(0x7f69b40c7170 0x7f69b40c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136189253 0 0) 0x7f69b40a8390 con 0x7f69b406ff10 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699c003520 con 0x7f69b406ff10 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974816591 0 0) 0x7f699c003520 con 0x7f69b406ff10 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 msgr2=0x7f69b40c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 0x7f69b40c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 >> v2:172.21.15.117:3300/0 conn(0x7f69b40c7170 msgr2=0x7f69b40c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 --2- 172.21.15.26:0/3100229850 >> v2:172.21.15.117:3300/0 conn(0x7f69b40c7170 0x7f69b40c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40733d0 con 0x7f69b406ff10 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f69a4001670 con 0x7f69b406ff10 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 >> v1:172.21.15.26:6789/0 conn(0x7f69b406ff10 legacy=0x7f69b4153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 0x7f699c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c003f30 0x7f699c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c004d40 0x7f699c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40c7620 con 0x7f699c003f30 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40730f0 con 0x7f699c004d40 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c00baa0 con 0x7f69b40c6c40 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.414+0000 7f69b37fe640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c003f30 0x7f699c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.415+0000 7f69b3fff640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 0x7f699c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.415+0000 7f69b8bed640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c004d40 0x7f699c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.415+0000 7f69b37fe640 1 -- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 msgr2=0x7f699c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.415+0000 7f69b37fe640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40c6c40 0x7f699c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.415+0000 7f69b37fe640 1 -- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c004d40 msgr2=0x7f699c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.415+0000 7f69b37fe640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c004d40 0x7f699c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.415+0000 7f69b37fe640 1 -- 172.21.15.26:0/3100229850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c003520 con 0x7f699c003f30 2024-03-06T07:24:57.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69b37fe640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c003f30 0x7f699c004800 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009ac0 tx=0x7f699c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942bdb83c60463b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8039070 con 0x7f699c003f30 2024-03-06T07:24:57.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a802c710 con 0x7f699c003f30 2024-03-06T07:24:57.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69b2ffd640 1 -- 172.21.15.26:0/3100229850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8009690 con 0x7f699c003f30 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69ba650640 1 -- 172.21.15.26:0/3100229850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c003f30 msgr2=0x7f699c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69ba650640 1 --2- 172.21.15.26:0/3100229850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f699c003f30 0x7f699c004800 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009ac0 tx=0x7f699c005830 comp rx=0 tx=0).stop 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69b8bed640 1 -- 172.21.15.26:0/3100229850 reap_dead start 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69ba650640 1 -- 172.21.15.26:0/3100229850 shutdown_connections 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69ba650640 1 -- 172.21.15.26:0/3100229850 >> 172.21.15.26:0/3100229850 conn(0x7f69b406b970 msgr2=0x7f69b40aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69ba650640 1 -- 172.21.15.26:0/3100229850 shutdown_connections 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.416+0000 7f69ba650640 1 -- 172.21.15.26:0/3100229850 wait complete. 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.417+0000 7f69ba650640 1 Processor -- start 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69ba650640 1 -- start start 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69ba650640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c003f30 0x7f69b41ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69ba650640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40bd8a0 0x7f69b41ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69ba650640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b40a9640 0x7f69b40a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69ba650640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b41544b0 con 0x7f69b40a9640 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69ba650640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4154330 con 0x7f699c003f30 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69ba650640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40702f0 con 0x7f69b40bd8a0 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69b8bed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b40a9640 0x7f69b40a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69b8bed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b40a9640 0x7f69b40a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52662/0 (socket says 172.21.15.26:52662) 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.418+0000 7f69b8bed640 1 -- 172.21.15.26:0/2430642930 learned_addr learned my addr 172.21.15.26:0/2430642930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b37fe640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40bd8a0 0x7f69b41ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b3fff640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c003f30 0x7f69b41ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b8bed640 1 -- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40bd8a0 msgr2=0x7f69b41ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b8bed640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40bd8a0 0x7f69b41ec000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b8bed640 1 -- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c003f30 msgr2=0x7f69b41ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b8bed640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c003f30 0x7f69b41ebac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b8bed640 1 -- 172.21.15.26:0/2430642930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40c2530 con 0x7f69b40a9640 2024-03-06T07:24:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b8bed640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b40a9640 0x7f69b40a9aa0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f69b406bf80 tx=0x7f69ac00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b0ff9640 1 -- 172.21.15.26:0/2430642930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00fde0 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b0ff9640 1 -- 172.21.15.26:0/2430642930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac01d070 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b40c2810 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69b0ff9640 1 -- 172.21.15.26:0/2430642930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac015670 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.419+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b41ec540 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.420+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b41550d0 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.420+0000 7f69b0ff9640 1 -- 172.21.15.26:0/2430642930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69ac015810 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.421+0000 7f69b0ff9640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69a006b4d0 0x7f69a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.421+0000 7f69b0ff9640 1 -- 172.21.15.26:0/2430642930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69ac08abc0 con 0x7f69b40a9640 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.421+0000 7f69b3fff640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69a006b4d0 0x7f69a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.421+0000 7f69b3fff640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69a006b4d0 0x7f69a006d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f69a4001120 tx=0x7f69a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.425+0000 7f69b0ff9640 1 -- 172.21.15.26:0/2430642930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f69ac010040 con 0x7f69b40a9640 2024-03-06T07:24:57.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.675+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "9:0"} v 0) v1 -- 0x7f69b404f330 con 0x7f69b40a9640 2024-03-06T07:24:57.678 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:24:57.676+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:24:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.677+0000 7f69b0ff9640 1 -- 172.21.15.26:0/2430642930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "9:0"}]=0 failed mds gid 7854 v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7f69ac0598b0 con 0x7f69b40a9640 2024-03-06T07:24:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:failed mds gid 7854 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69a006b4d0 msgr2=0x7f69a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69a006b4d0 0x7f69a006d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f69a4001120 tx=0x7f69a4009040 comp rx=0 tx=0).stop 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b40a9640 msgr2=0x7f69b40a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b40a9640 0x7f69b40a9aa0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f69b406bf80 tx=0x7f69ac00f580 comp rx=0 tx=0).stop 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 shutdown_connections 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69a006b4d0 0x7f69a006d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b40a9640 0x7f69b40a9aa0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69b40bd8a0 0x7f69b41ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 --2- 172.21.15.26:0/2430642930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f699c003f30 0x7f69b41ebac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 >> 172.21.15.26:0/2430642930 conn(0x7f69b406b970 msgr2=0x7f69b40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 shutdown_connections 2024-03-06T07:24:57.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.679+0000 7f69ba650640 1 -- 172.21.15.26:0/2430642930 wait complete. 2024-03-06T07:24:57.683 INFO:tasks.ceph.mds.c.smithi026.stderr: -380> 2024-03-06T07:24:49.212+0000 7fa0ed3f2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:57.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:57.710 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:24:57.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.796+0000 7fcb96df6640 1 Processor -- start 2024-03-06T07:24:57.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb96df6640 1 -- start start 2024-03-06T07:24:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb96df6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb90153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb96df6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb90156ea0 0x7fcb900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb96df6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb90070800 con 0x7fcb900acc00 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb96df6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb90157320 con 0x7fcb90156ea0 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb96df6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90156450 con 0x7fcb901506b0 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb95393640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcb900acc00 0x7fcb900c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33666/0 (socket says 172.21.15.26:33666) 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb95393640 1 -- 172.21.15.26:0/8197325 learned_addr learned my addr 172.21.15.26:0/8197325 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb94b92640 1 --2- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb90153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.797+0000 7fcb8ffff640 1 --2- 172.21.15.26:0/8197325 >> v2:172.21.15.117:3300/0 conn(0x7fcb90156ea0 0x7fcb900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb8f7fe640 1 -- 172.21.15.26:0/8197325 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637669515 0 0) 0x7fcb90070800 con 0x7fcb900acc00 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb8f7fe640 1 -- 172.21.15.26:0/8197325 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb80003520 con 0x7fcb900acc00 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb8ffff640 1 -- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 msgr2=0x7fcb90153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb8ffff640 1 --2- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb90153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb8ffff640 1 -- 172.21.15.26:0/8197325 >> v1:172.21.15.26:6789/0 conn(0x7fcb900acc00 legacy=0x7fcb900c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb8f7fe640 1 -- 172.21.15.26:0/8197325 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3728199225 0 0) 0x7fcb80003520 con 0x7fcb900acc00 2024-03-06T07:24:57.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb8ffff640 1 -- 172.21.15.26:0/8197325 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90070aa0 con 0x7fcb90156ea0 2024-03-06T07:24:57.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.798+0000 7fcb94b92640 1 --2- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb90153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb8ffff640 1 --2- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 0x7fcb900702c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097a0 tx=0x7fcb7c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61dd6b5997f18100 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb8f7fe640 1 -- 172.21.15.26:0/8197325 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c035020 con 0x7fcb90156ea0 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb8f7fe640 1 -- 172.21.15.26:0/8197325 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c031830 con 0x7fcb90156ea0 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb8f7fe640 1 -- 172.21.15.26:0/8197325 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c031c00 con 0x7fcb90156ea0 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 -- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 msgr2=0x7fcb900702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 --2- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 0x7fcb900702c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097a0 tx=0x7fcb7c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 -- 172.21.15.26:0/8197325 shutdown_connections 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 --2- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 0x7fcb900702c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 --2- 172.21.15.26:0/8197325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb90153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 -- 172.21.15.26:0/8197325 >> 172.21.15.26:0/8197325 conn(0x7fcb9006b1a0 msgr2=0x7fcb9006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 -- 172.21.15.26:0/8197325 shutdown_connections 2024-03-06T07:24:57.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.799+0000 7fcb96df6640 1 -- 172.21.15.26:0/8197325 wait complete. 2024-03-06T07:24:57.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.800+0000 7fcb96df6640 1 Processor -- start 2024-03-06T07:24:57.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb96df6640 1 -- start start 2024-03-06T07:24:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb96df6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb900acc00 0x7fcb901e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb96df6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb900a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb96df6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 0x7fcb90146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb96df6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901566a0 con 0x7fcb900acc00 2024-03-06T07:24:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb96df6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90156a10 con 0x7fcb90156ea0 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb96df6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900c5bb0 con 0x7fcb901506b0 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb94b92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb900acc00 0x7fcb901e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb94b92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb900acc00 0x7fcb901e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52682/0 (socket says 172.21.15.26:52682) 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.801+0000 7fcb94b92640 1 -- 172.21.15.26:0/857951022 learned_addr learned my addr 172.21.15.26:0/857951022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb8ffff640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb900a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb95393640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 0x7fcb90146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb94b92640 1 -- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 msgr2=0x7fcb900a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb94b92640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb900a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb94b92640 1 -- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 msgr2=0x7fcb90146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb94b92640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 0x7fcb90146bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb94b92640 1 -- 172.21.15.26:0/857951022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900aab00 con 0x7fcb900acc00 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb94b92640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb900acc00 0x7fcb901e9490 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ec10 tx=0x7fcb8400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb8d7fa640 1 -- 172.21.15.26:0/857951022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84017070 con 0x7fcb900acc00 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb8d7fa640 1 -- 172.21.15.26:0/857951022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8400cc30 con 0x7fcb900acc00 2024-03-06T07:24:57.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb900aade0 con 0x7fcb900acc00 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb8d7fa640 1 -- 172.21.15.26:0/857951022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84013470 con 0x7fcb900acc00 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.802+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb900a91f0 con 0x7fcb900acc00 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.803+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9004f330 con 0x7fcb900acc00 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.804+0000 7fcb8d7fa640 1 -- 172.21.15.26:0/857951022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb8401a070 con 0x7fcb900acc00 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.804+0000 7fcb8d7fa640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.804+0000 7fcb8d7fa640 1 -- 172.21.15.26:0/857951022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb8408a680 con 0x7fcb900acc00 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.804+0000 7fcb8ffff640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.805+0000 7fcb8ffff640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7406b4d0 0x7fcb7406d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c036000 tx=0x7fcb7c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:57.808+0000 7fcb8d7fa640 1 -- 172.21.15.26:0/857951022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb840593b0 con 0x7fcb900acc00 2024-03-06T07:24:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.028+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb900700a0 con 0x7fcb900acc00 2024-03-06T07:24:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.029+0000 7fcb8d7fa640 1 -- 172.21.15.26:0/857951022 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4194 (secure 0 0 0) 0x7fcb84058cb0 con 0x7fcb900acc00 2024-03-06T07:24:58.032 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:58.032 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":108,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:24:57.681901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.26:6834/1856740515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1856740515},{"type":"v1","addr":"172.21.15.26:6834","nonce":1856740515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":9}]} 2024-03-06T07:24:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 108 2024-03-06T07:24:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7406b4d0 msgr2=0x7fcb7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7406b4d0 0x7fcb7406d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c036000 tx=0x7fcb7c03c040 comp rx=0 tx=0).stop 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb900acc00 msgr2=0x7fcb901e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb900acc00 0x7fcb901e9490 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ec10 tx=0x7fcb8400c520 comp rx=0 tx=0).stop 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 shutdown_connections 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb90156ea0 0x7fcb90146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb901506b0 0x7fcb900a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 --2- 172.21.15.26:0/857951022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb900acc00 0x7fcb901e9490 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 >> 172.21.15.26:0/857951022 conn(0x7fcb9006b1a0 msgr2=0x7fcb900724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 shutdown_connections 2024-03-06T07:24:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:58.031+0000 7fcb96df6640 1 -- 172.21.15.26:0/857951022 wait complete. 2024-03-06T07:24:58.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:24:57.681901+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7851': {'gid': 7851, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.26:6834/1856740515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1856740515}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1856740515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:24:58.446 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7851:up:replay 2024-03-06T07:24:59.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:59.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f86288640 1 Processor -- start 2024-03-06T07:24:59.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f86288640 1 -- start start 2024-03-06T07:24:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f86288640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f80155c80 0x7f6f8006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f86288640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f800720a0 0x7f6f8006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f86288640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f800c1ae0 con 0x7f6f80152520 2024-03-06T07:24:59.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f86288640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f800c1dc0 con 0x7f6f80155c80 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f86288640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8006e450 con 0x7f6f800720a0 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f7f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f80155c80 0x7f6f8006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.560+0000 7f6f7ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f80152520 0x7f6f800c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33670/0 (socket says 172.21.15.26:33670) 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.561+0000 7f6f7ffff640 1 -- 172.21.15.26:0/3349354895 learned_addr learned my addr 172.21.15.26:0/3349354895 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.561+0000 7f6f84825640 1 --2- 172.21.15.26:0/3349354895 >> v2:172.21.15.117:3300/0 conn(0x7f6f800720a0 0x7f6f8006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.561+0000 7f6f7f7fe640 1 -- 172.21.15.26:0/3349354895 >> v2:172.21.15.117:3300/0 conn(0x7f6f800720a0 msgr2=0x7f6f8006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.561+0000 7f6f7f7fe640 1 --2- 172.21.15.26:0/3349354895 >> v2:172.21.15.117:3300/0 conn(0x7f6f800720a0 0x7f6f8006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.561+0000 7f6f7f7fe640 1 -- 172.21.15.26:0/3349354895 >> v1:172.21.15.26:6789/0 conn(0x7f6f80152520 legacy=0x7f6f800c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.561+0000 7f6f7f7fe640 1 -- 172.21.15.26:0/3349354895 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8006e730 con 0x7f6f80155c80 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f7f7fe640 1 --2- 172.21.15.26:0/3349354895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f8006ef30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6f80070f30 tx=0x7f6f7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7267b610256f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f7effd640 1 -- 172.21.15.26:0/3349354895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74035020 con 0x7f6f80155c80 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f7effd640 1 -- 172.21.15.26:0/3349354895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f74031830 con 0x7f6f80155c80 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f7effd640 1 -- 172.21.15.26:0/3349354895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74031c00 con 0x7f6f80155c80 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 -- 172.21.15.26:0/3349354895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 msgr2=0x7f6f8006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 --2- 172.21.15.26:0/3349354895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f8006ef30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6f80070f30 tx=0x7f6f7402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 -- 172.21.15.26:0/3349354895 shutdown_connections 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 --2- 172.21.15.26:0/3349354895 >> v2:172.21.15.117:3300/0 conn(0x7f6f800720a0 0x7f6f8006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 --2- 172.21.15.26:0/3349354895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f8006ef30 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 -- 172.21.15.26:0/3349354895 >> 172.21.15.26:0/3349354895 conn(0x7f6f800c76b0 msgr2=0x7f6f800c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 -- 172.21.15.26:0/3349354895 shutdown_connections 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.562+0000 7f6f86288640 1 -- 172.21.15.26:0/3349354895 wait complete. 2024-03-06T07:24:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.563+0000 7f6f86288640 1 Processor -- start 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f86288640 1 -- start start 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f86288640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f800720a0 0x7f6f801eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f86288640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80152520 0x7f6f801e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f86288640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f801ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f86288640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80152a80 con 0x7f6f80155c80 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f86288640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80152900 con 0x7f6f80152520 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f86288640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8006b500 con 0x7f6f800720a0 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f84825640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f801ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f84825640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f801ee8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52708/0 (socket says 172.21.15.26:52708) 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.564+0000 7f6f84825640 1 -- 172.21.15.26:0/2824307611 learned_addr learned my addr 172.21.15.26:0/2824307611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f7f7fe640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80152520 0x7f6f801e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f7ffff640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f800720a0 0x7f6f801eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f84825640 1 -- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f800720a0 msgr2=0x7f6f801eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f84825640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f800720a0 0x7f6f801eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f84825640 1 -- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80152520 msgr2=0x7f6f801e0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f84825640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80152520 0x7f6f801e0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f84825640 1 -- 172.21.15.26:0/2824307611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8014db20 con 0x7f6f80155c80 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f7f7fe640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80152520 0x7f6f801e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f84825640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f801ee8a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6f78007e60 tx=0x7f6f7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f7cff9640 1 -- 172.21.15.26:0/2824307611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78014420 con 0x7f6f80155c80 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f7cff9640 1 -- 172.21.15.26:0/2824307611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f78019070 con 0x7f6f80155c80 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f801de640 con 0x7f6f80155c80 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f7cff9640 1 -- 172.21.15.26:0/2824307611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78014820 con 0x7f6f80155c80 2024-03-06T07:24:59.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.565+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f801e1290 con 0x7f6f80155c80 2024-03-06T07:24:59.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.566+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f801ee170 con 0x7f6f80155c80 2024-03-06T07:24:59.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.567+0000 7f6f7cff9640 1 -- 172.21.15.26:0/2824307611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f780149c0 con 0x7f6f80155c80 2024-03-06T07:24:59.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.567+0000 7f6f7cff9640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f6c06b4d0 0x7f6f6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:59.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.567+0000 7f6f7cff9640 1 -- 172.21.15.26:0/2824307611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6f7808abc0 con 0x7f6f80155c80 2024-03-06T07:24:59.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.567+0000 7f6f7ffff640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f6c06b4d0 0x7f6f6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:59.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.568+0000 7f6f7ffff640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f6c06b4d0 0x7f6f6c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f70000f60 tx=0x7f6f70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.571+0000 7f6f7cff9640 1 -- 172.21.15.26:0/2824307611 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f780598f0 con 0x7f6f80155c80 2024-03-06T07:24:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.792+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f8004f330 con 0x7f6f80155c80 2024-03-06T07:24:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.793+0000 7f6f7cff9640 1 -- 172.21.15.26:0/2824307611 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f6f780591f0 con 0x7f6f80155c80 2024-03-06T07:24:59.797 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:24:59.797 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":110,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8082,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:24:58.690253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":108,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.26:6834/1856740515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1856740515},{"type":"v1","addr":"172.21.15.26:6834","nonce":1856740515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":9}]} 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 110 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.794+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f6c06b4d0 msgr2=0x7f6f6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f6c06b4d0 0x7f6f6c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f70000f60 tx=0x7f6f70009040 comp rx=0 tx=0).stop 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 msgr2=0x7f6f801ee8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f801ee8a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6f78007e60 tx=0x7f6f7800e4b0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 shutdown_connections 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6f6c06b4d0 0x7f6f6c06d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f80155c80 0x7f6f801ee8a0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80152520 0x7f6f801e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 --2- 172.21.15.26:0/2824307611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f800720a0 0x7f6f801eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 >> 172.21.15.26:0/2824307611 conn(0x7f6f800c76b0 msgr2=0x7f6f800c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 shutdown_connections 2024-03-06T07:24:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:24:59.795+0000 7f6f86288640 1 -- 172.21.15.26:0/2824307611 wait complete. 2024-03-06T07:25:00.211 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:24:58.690253+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7851': {'gid': 7851, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.26:6834/1856740515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1856740515}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 1856740515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:25:00.211 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7851:up:rejoin 2024-03-06T07:25:00.733 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:25:01.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.324+0000 7fd40b229640 1 Processor -- start 2024-03-06T07:25:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.324+0000 7fd40b229640 1 -- start start 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd40b229640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4040c3280 0x7fd4040c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd40b229640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4041551e0 0x7fd4041555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd40b229640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd404158800 con 0x7fd40406d9f0 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd40b229640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd404158ae0 con 0x7fd4040c3280 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd40b229640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd404158dc0 con 0x7fd4041551e0 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd403fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4040c3280 0x7fd4040c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd403fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4040c3280 0x7fd4040c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52726/0 (socket says 172.21.15.26:52726) 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd403fff640 1 -- 172.21.15.26:0/3416475906 learned_addr learned my addr 172.21.15.26:0/3416475906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd4097c6640 1 --2- 172.21.15.26:0/3416475906 >> v2:172.21.15.117:3300/0 conn(0x7fd4041551e0 0x7fd4041555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd403fff640 1 -- 172.21.15.26:0/3416475906 >> v2:172.21.15.117:3300/0 conn(0x7fd4041551e0 msgr2=0x7fd4041555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd403fff640 1 --2- 172.21.15.26:0/3416475906 >> v2:172.21.15.117:3300/0 conn(0x7fd4041551e0 0x7fd4041555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd403fff640 1 -- 172.21.15.26:0/3416475906 >> v1:172.21.15.117:6789/0 conn(0x7fd40406d9f0 legacy=0x7fd4040a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.325+0000 7fd403fff640 1 -- 172.21.15.26:0/3416475906 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4041590a0 con 0x7fd4040c3280 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.326+0000 7fd403fff640 1 --2- 172.21.15.26:0/3416475906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040c3710 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd404072f50 tx=0x7fd3f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0117531142f5a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.326+0000 7fd4037fe640 1 -- 172.21.15.26:0/3416475906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0035020 con 0x7fd4040c3280 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.326+0000 7fd4037fe640 1 -- 172.21.15.26:0/3416475906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f0031870 con 0x7fd4040c3280 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.326+0000 7fd4037fe640 1 -- 172.21.15.26:0/3416475906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0031c40 con 0x7fd4040c3280 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 -- 172.21.15.26:0/3416475906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 msgr2=0x7fd4040c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 --2- 172.21.15.26:0/3416475906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040c3710 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd404072f50 tx=0x7fd3f002b910 comp rx=0 tx=0).stop 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 -- 172.21.15.26:0/3416475906 shutdown_connections 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 --2- 172.21.15.26:0/3416475906 >> v2:172.21.15.117:3300/0 conn(0x7fd4041551e0 0x7fd4041555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 --2- 172.21.15.26:0/3416475906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040c3710 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 -- 172.21.15.26:0/3416475906 >> 172.21.15.26:0/3416475906 conn(0x7fd4040aa630 msgr2=0x7fd4040aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 -- 172.21.15.26:0/3416475906 shutdown_connections 2024-03-06T07:25:01.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.327+0000 7fd40b229640 1 -- 172.21.15.26:0/3416475906 wait complete. 2024-03-06T07:25:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.328+0000 7fd40b229640 1 Processor -- start 2024-03-06T07:25:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.328+0000 7fd40b229640 1 -- start start 2024-03-06T07:25:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd40b229640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40406d9f0 0x7fd40406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd40b229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd40b229640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4041551e0 0x7fd4040acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd40b229640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40406df50 con 0x7fd4040c3280 2024-03-06T07:25:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd40b229640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40406ddd0 con 0x7fd4041551e0 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd40b229640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040c6910 con 0x7fd40406d9f0 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52738/0 (socket says 172.21.15.26:52738) 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 -- 172.21.15.26:0/2986329895 learned_addr learned my addr 172.21.15.26:0/2986329895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd4097c6640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4041551e0 0x7fd4040acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 -- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40406d9f0 msgr2=0x7fd40406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd408fc5640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40406d9f0 0x7fd40406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40406d9f0 0x7fd40406f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 -- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4041551e0 msgr2=0x7fd4040acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4041551e0 0x7fd4040acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd403fff640 1 -- 172.21.15.26:0/2986329895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4041ee770 con 0x7fd4040c3280 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.329+0000 7fd4097c6640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4041551e0 0x7fd4040acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.330+0000 7fd403fff640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040ac590 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd3f002b950 tx=0x7fd3f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.330+0000 7fd4017fa640 1 -- 172.21.15.26:0/2986329895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0040070 con 0x7fd4040c3280 2024-03-06T07:25:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.330+0000 7fd4017fa640 1 -- 172.21.15.26:0/2986329895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f0030d00 con 0x7fd4040c3280 2024-03-06T07:25:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.330+0000 7fd4017fa640 1 -- 172.21.15.26:0/2986329895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0035040 con 0x7fd4040c3280 2024-03-06T07:25:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.330+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4041eea50 con 0x7fd4040c3280 2024-03-06T07:25:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.330+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4041560c0 con 0x7fd4040c3280 2024-03-06T07:25:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.331+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40404f330 con 0x7fd4040c3280 2024-03-06T07:25:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.331+0000 7fd4017fa640 1 -- 172.21.15.26:0/2986329895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3f004b060 con 0x7fd4040c3280 2024-03-06T07:25:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.332+0000 7fd4017fa640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3e806b4d0 0x7fd3e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.332+0000 7fd4017fa640 1 -- 172.21.15.26:0/2986329895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd3f00ac250 con 0x7fd4040c3280 2024-03-06T07:25:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.332+0000 7fd408fc5640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3e806b4d0 0x7fd3e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.332+0000 7fd408fc5640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3e806b4d0 0x7fd3e806d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd4041df0f0 tx=0x7fd3f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.336+0000 7fd4017fa640 1 -- 172.21.15.26:0/2986329895 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd3f007ae80 con 0x7fd4040c3280 2024-03-06T07:25:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.555+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd404073240 con 0x7fd4040c3280 2024-03-06T07:25:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.556+0000 7fd4017fa640 1 -- 172.21.15.26:0/2986329895 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4193 (secure 0 0 0) 0x7fd3f007a780 con 0x7fd4040c3280 2024-03-06T07:25:01.559 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:01.559 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":112,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:25:00.699719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8082":{"gid":8082,"name":"c","rank":0,"incarnation":112,"state":"up:replay","state_seq":1,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":9}]} 2024-03-06T07:25:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 112 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3e806b4d0 msgr2=0x7fd3e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3e806b4d0 0x7fd3e806d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd4041df0f0 tx=0x7fd3f8009040 comp rx=0 tx=0).stop 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 msgr2=0x7fd4040ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040ac590 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd3f002b950 tx=0x7fd3f00046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 shutdown_connections 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd3e806b4d0 0x7fd3e806d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4041551e0 0x7fd4040acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4040c3280 0x7fd4040ac590 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 --2- 172.21.15.26:0/2986329895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40406d9f0 0x7fd40406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 >> 172.21.15.26:0/2986329895 conn(0x7fd4040aa630 msgr2=0x7fd4040c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 shutdown_connections 2024-03-06T07:25:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:01.558+0000 7fd40b229640 1 -- 172.21.15.26:0/2986329895 wait complete. 2024-03-06T07:25:01.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:25:00.699719+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8082}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8082': {'gid': 8082, 'name': 'c', 'rank': 0, 'incarnation': 112, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/1367162338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1367162338}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1367162338}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:25:01.974 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8082:up:replay 2024-03-06T07:25:02.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.087+0000 7fa3cfceb640 1 Processor -- start 2024-03-06T07:25:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.087+0000 7fa3cfceb640 1 -- start start 2024-03-06T07:25:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cfceb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cfceb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3c80c1320 0x7fa3c806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cfceb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c8153a60 con 0x7fa3c80c6000 2024-03-06T07:25:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cfceb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c8153d40 con 0x7fa3c80c1320 2024-03-06T07:25:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cfceb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c806c2c0 con 0x7fa3c8150580 2024-03-06T07:25:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cd286640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3c80c1320 0x7fa3c806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cd286640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3c80c1320 0x7fa3c806e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52750/0 (socket says 172.21.15.26:52750) 2024-03-06T07:25:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cd286640 1 -- 172.21.15.26:0/1863266827 learned_addr learned my addr 172.21.15.26:0/1863266827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.088+0000 7fa3cda87640 1 --2- 172.21.15.26:0/1863266827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.089+0000 7fa3cd286640 1 -- 172.21.15.26:0/1863266827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 msgr2=0x7fa3c806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.089+0000 7fa3cd286640 1 --2- 172.21.15.26:0/1863266827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.089+0000 7fa3cd286640 1 -- 172.21.15.26:0/1863266827 >> v1:172.21.15.117:6789/0 conn(0x7fa3c80c6000 legacy=0x7fa3c81566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.089+0000 7fa3cd286640 1 -- 172.21.15.26:0/1863266827 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c806c5a0 con 0x7fa3c80c1320 2024-03-06T07:25:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.089+0000 7fa3cd286640 1 --2- 172.21.15.26:0/1863266827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c1320 0x7fa3c806e250 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9698b49aac91ebd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cca85640 1 -- 172.21.15.26:0/1863266827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc035020 con 0x7fa3c80c1320 2024-03-06T07:25:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cca85640 1 -- 172.21.15.26:0/1863266827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc031640 con 0x7fa3c80c1320 2024-03-06T07:25:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cca85640 1 -- 172.21.15.26:0/1863266827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc02d040 con 0x7fa3c80c1320 2024-03-06T07:25:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1863266827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c1320 msgr2=0x7fa3c806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1863266827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c1320 0x7fa3c806e250 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1863266827 shutdown_connections 2024-03-06T07:25:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1863266827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c1320 0x7fa3c806e250 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1863266827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1863266827 >> 172.21.15.26:0/1863266827 conn(0x7fa3c80c6710 msgr2=0x7fa3c80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1863266827 shutdown_connections 2024-03-06T07:25:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.090+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1863266827 wait complete. 2024-03-06T07:25:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.091+0000 7fa3cfceb640 1 Processor -- start 2024-03-06T07:25:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.091+0000 7fa3cfceb640 1 -- start start 2024-03-06T07:25:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cfceb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c80c1320 0x7fa3c80a7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cfceb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c6000 0x7fa3c80c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cfceb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c814dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cfceb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80c0dc0 con 0x7fa3c80c6000 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cfceb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80c0c40 con 0x7fa3c80c1320 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cfceb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8156dc0 con 0x7fa3c8150580 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cd286640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c6000 0x7fa3c80c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cd286640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c6000 0x7fa3c80c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52754/0 (socket says 172.21.15.26:52754) 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cd286640 1 -- 172.21.15.26:0/1228742318 learned_addr learned my addr 172.21.15.26:0/1228742318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3cda87640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c80c1320 0x7fa3c80a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.092+0000 7fa3ce288640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c814dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cd286640 1 -- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 msgr2=0x7fa3c814dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cd286640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c814dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cd286640 1 -- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c80c1320 msgr2=0x7fa3c80a7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cd286640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c80c1320 0x7fa3c80a7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cd286640 1 -- 172.21.15.26:0/1228742318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80c2830 con 0x7fa3c80c6000 2024-03-06T07:25:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cd286640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c6000 0x7fa3c80c3200 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02b930 tx=0x7fa3bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3c67fc640 1 -- 172.21.15.26:0/1228742318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc040070 con 0x7fa3c80c6000 2024-03-06T07:25:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3c67fc640 1 -- 172.21.15.26:0/1228742318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc03e070 con 0x7fa3c80c6000 2024-03-06T07:25:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c814fb00 con 0x7fa3c80c6000 2024-03-06T07:25:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3c67fc640 1 -- 172.21.15.26:0/1228742318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc035040 con 0x7fa3c80c6000 2024-03-06T07:25:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.093+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c80a7dc0 con 0x7fa3c80c6000 2024-03-06T07:25:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.094+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c804f330 con 0x7fa3c80c6000 2024-03-06T07:25:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.094+0000 7fa3c67fc640 1 -- 172.21.15.26:0/1228742318 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3bc038020 con 0x7fa3c80c6000 2024-03-06T07:25:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.095+0000 7fa3c67fc640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3b406b500 0x7fa3b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.095+0000 7fa3c67fc640 1 -- 172.21.15.26:0/1228742318 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa3bc0abd20 con 0x7fa3c80c6000 2024-03-06T07:25:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.095+0000 7fa3cda87640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3b406b500 0x7fa3b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.096+0000 7fa3cda87640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3b406b500 0x7fa3b406d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80073f0 tx=0x7fa3b800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.099+0000 7fa3c67fc640 1 -- 172.21.15.26:0/1228742318 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa3bc07a9f0 con 0x7fa3c80c6000 2024-03-06T07:25:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.318+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3c80a5fd0 con 0x7fa3c80c6000 2024-03-06T07:25:03.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.318+0000 7fa3c67fc640 1 -- 172.21.15.26:0/1228742318 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4982 (secure 0 0 0) 0x7fa3bc07a2f0 con 0x7fa3c80c6000 2024-03-06T07:25:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:03.321 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":114,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/317629496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":317629496},{"type":"v1","addr":"172.21.15.26:6834","nonce":317629496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:25:01.706069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8082":{"gid":8082,"name":"c","rank":0,"incarnation":112,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":9}]} 2024-03-06T07:25:03.321 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 114 2024-03-06T07:25:03.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3b406b500 msgr2=0x7fa3b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3b406b500 0x7fa3b406d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80073f0 tx=0x7fa3b800b040 comp rx=0 tx=0).stop 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c6000 msgr2=0x7fa3c80c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c6000 0x7fa3c80c3200 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02b930 tx=0x7fa3bc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 shutdown_connections 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3b406b500 0x7fa3b406d9c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3c8150580 0x7fa3c814dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c80c6000 0x7fa3c80c3200 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 --2- 172.21.15.26:0/1228742318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c80c1320 0x7fa3c80a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 >> 172.21.15.26:0/1228742318 conn(0x7fa3c80c6710 msgr2=0x7fa3c80c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 shutdown_connections 2024-03-06T07:25:03.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:03.321+0000 7fa3cfceb640 1 -- 172.21.15.26:0/1228742318 wait complete. 2024-03-06T07:25:03.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:25:01.706069+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8082}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8082': {'gid': 8082, 'name': 'c', 'rank': 0, 'incarnation': 112, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.26:6835/1367162338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1367162338}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1367162338}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:25:03.736 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8082:up:rejoin 2024-03-06T07:25:04.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.840+0000 7f6d665c7640 1 Processor -- start 2024-03-06T07:25:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.840+0000 7f6d665c7640 1 -- start start 2024-03-06T07:25:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d665c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6d6014f360 0x7f6d60155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d665c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d60158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d665c7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d600c73d0 con 0x7f6d600c88c0 2024-03-06T07:25:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d665c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d600c8dd0 con 0x7f6d6014f360 2024-03-06T07:25:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d665c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600be2e0 con 0x7f6d60158b60 2024-03-06T07:25:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6d6014f360 0x7f6d60155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d64b64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d60158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5f7fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6d600c88c0 0x7f6d600c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:36028/0 (socket says 172.21.15.26:36028) 2024-03-06T07:25:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5f7fe640 1 -- 172.21.15.26:0/3022018563 learned_addr learned my addr 172.21.15.26:0/3022018563 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5ffff640 1 -- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 msgr2=0x7f6d60158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5ffff640 1 --2- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d60158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5ffff640 1 -- 172.21.15.26:0/3022018563 >> v1:172.21.15.117:6789/0 conn(0x7f6d600c88c0 legacy=0x7f6d600c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5effd640 1 -- 172.21.15.26:0/3022018563 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545351110 0 0) 0x7f6d600c73d0 con 0x7f6d600c88c0 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d5ffff640 1 -- 172.21.15.26:0/3022018563 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600c7670 con 0x7f6d6014f360 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.842+0000 7f6d64b64640 1 --2- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d60158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.843+0000 7f6d5ffff640 1 --2- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 0x7f6d60155c10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6d500097a0 tx=0x7f6d5002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404e7f53c1d9d132 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.843+0000 7f6d5effd640 1 -- 172.21.15.26:0/3022018563 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50031780 con 0x7f6d6014f360 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.843+0000 7f6d5effd640 1 -- 172.21.15.26:0/3022018563 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d50037070 con 0x7f6d6014f360 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.843+0000 7f6d5effd640 1 -- 172.21.15.26:0/3022018563 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50031b40 con 0x7f6d6014f360 2024-03-06T07:25:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 -- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 msgr2=0x7f6d60155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 --2- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 0x7f6d60155c10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6d500097a0 tx=0x7f6d5002b820 comp rx=0 tx=0).stop 2024-03-06T07:25:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 -- 172.21.15.26:0/3022018563 shutdown_connections 2024-03-06T07:25:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 --2- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d60158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 --2- 172.21.15.26:0/3022018563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 0x7f6d60155c10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 -- 172.21.15.26:0/3022018563 >> 172.21.15.26:0/3022018563 conn(0x7f6d600ac570 msgr2=0x7f6d600ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 -- 172.21.15.26:0/3022018563 shutdown_connections 2024-03-06T07:25:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.844+0000 7f6d665c7640 1 -- 172.21.15.26:0/3022018563 wait complete. 2024-03-06T07:25:04.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.845+0000 7f6d665c7640 1 Processor -- start 2024-03-06T07:25:04.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.845+0000 7f6d665c7640 1 -- start start 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d665c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d600c88c0 0x7f6d601e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d665c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 0x7f6d601eb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d665c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d6014ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d665c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c5a50 con 0x7f6d600c88c0 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d665c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c7c30 con 0x7f6d6014f360 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d665c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c5bd0 con 0x7f6d60158b60 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d600c88c0 0x7f6d601e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d600c88c0 0x7f6d601e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52762/0 (socket says 172.21.15.26:52762) 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 -- 172.21.15.26:0/1949319137 learned_addr learned my addr 172.21.15.26:0/1949319137 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5f7fe640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 0x7f6d601eb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d64b64640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d6014ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 -- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 msgr2=0x7f6d6014ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d6014ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 -- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 msgr2=0x7f6d601eb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 0x7f6d601eb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.846+0000 7f6d5ffff640 1 -- 172.21.15.26:0/1949319137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600c4e10 con 0x7f6d600c88c0 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.847+0000 7f6d5ffff640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d600c88c0 0x7f6d601e7d20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6d50034040 tx=0x7f6d500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.847+0000 7f6d5cff9640 1 -- 172.21.15.26:0/1949319137 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50004230 con 0x7f6d600c88c0 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.847+0000 7f6d5cff9640 1 -- 172.21.15.26:0/1949319137 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d50031ec0 con 0x7f6d600c88c0 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.847+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d60157cb0 con 0x7f6d600c88c0 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.847+0000 7f6d5cff9640 1 -- 172.21.15.26:0/1949319137 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003610 con 0x7f6d600c88c0 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.847+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d60158270 con 0x7f6d600c88c0 2024-03-06T07:25:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.848+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6004f330 con 0x7f6d600c88c0 2024-03-06T07:25:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.848+0000 7f6d5cff9640 1 -- 172.21.15.26:0/1949319137 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d500037b0 con 0x7f6d600c88c0 2024-03-06T07:25:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.849+0000 7f6d5cff9640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d4c06b4d0 0x7f6d4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.849+0000 7f6d5cff9640 1 -- 172.21.15.26:0/1949319137 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f6d500ac0f0 con 0x7f6d600c88c0 2024-03-06T07:25:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.849+0000 7f6d5f7fe640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d4c06b4d0 0x7f6d4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.850+0000 7f6d5f7fe640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d4c06b4d0 0x7f6d4c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6d600bdc50 tx=0x7f6d54007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:04.853+0000 7f6d5cff9640 1 -- 172.21.15.26:0/1949319137 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6d50041070 con 0x7f6d600c88c0 2024-03-06T07:25:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.074+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d60149770 con 0x7f6d600c88c0 2024-03-06T07:25:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.074+0000 7f6d5cff9640 1 -- 172.21.15.26:0/1949319137 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4989 (secure 0 0 0) 0x7f6d5002f050 con 0x7f6d600c88c0 2024-03-06T07:25:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:05.077 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/317629496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":317629496},{"type":"v1","addr":"172.21.15.26:6834","nonce":317629496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:25:03.710057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8082":{"gid":8082,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8082,"qdb_cluster":[8082]},"id":9}]} 2024-03-06T07:25:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 115 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d4c06b4d0 msgr2=0x7f6d4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d4c06b4d0 0x7f6d4c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6d600bdc50 tx=0x7f6d54007de0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d600c88c0 msgr2=0x7f6d601e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d600c88c0 0x7f6d601e7d20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6d50034040 tx=0x7f6d500046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 shutdown_connections 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6d4c06b4d0 0x7f6d4c06d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d60158b60 0x7f6d6014ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6014f360 0x7f6d601eb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 --2- 172.21.15.26:0/1949319137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d600c88c0 0x7f6d601e7d20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 >> 172.21.15.26:0/1949319137 conn(0x7f6d600ac570 msgr2=0x7f6d600ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 shutdown_connections 2024-03-06T07:25:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.077+0000 7f6d665c7640 1 -- 172.21.15.26:0/1949319137 wait complete. 2024-03-06T07:25:05.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 115, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:43.287707+0000', 'modified': '2024-03-06T07:25:03.710057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8082}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8082': {'gid': 8082, 'name': 'c', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.26:6835/1367162338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1367162338}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1367162338}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8082, 'qdb_cluster': [8082]} 2024-03-06T07:25:05.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:05.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:25:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.601+0000 7f470a1f7640 1 Processor -- start 2024-03-06T07:25:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.601+0000 7f470a1f7640 1 -- start start 2024-03-06T07:25:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f470a1f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f47040c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f470a1f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47040c8950 0x7f47040acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f470a1f7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4704155130 con 0x7f47040ad2b0 2024-03-06T07:25:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f470a1f7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4704155440 con 0x7f47040c8950 2024-03-06T07:25:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f470a1f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47040c5d50 con 0x7f47040c69f0 2024-03-06T07:25:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f4702ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47040c8950 0x7f47040acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f4702ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47040c8950 0x7f47040acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52774/0 (socket says 172.21.15.26:52774) 2024-03-06T07:25:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.602+0000 7f4702ffd640 1 -- 172.21.15.26:0/1614624474 learned_addr learned my addr 172.21.15.26:0/1614624474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.603+0000 7f47037fe640 1 --2- 172.21.15.26:0/1614624474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f47040c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.603+0000 7f4702ffd640 1 -- 172.21.15.26:0/1614624474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 msgr2=0x7f47040c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.603+0000 7f4702ffd640 1 --2- 172.21.15.26:0/1614624474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f47040c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.603+0000 7f4702ffd640 1 -- 172.21.15.26:0/1614624474 >> v1:172.21.15.117:6789/0 conn(0x7f47040ad2b0 legacy=0x7f47041558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.603+0000 7f4702ffd640 1 -- 172.21.15.26:0/1614624474 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040c6030 con 0x7f47040c8950 2024-03-06T07:25:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.603+0000 7f4702ffd640 1 --2- 172.21.15.26:0/1614624474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47040acd70 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28da97a3aafc351 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.603+0000 7f47027fc640 1 -- 172.21.15.26:0/1614624474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80317c0 con 0x7f47040c8950 2024-03-06T07:25:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f47027fc640 1 -- 172.21.15.26:0/1614624474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8037070 con 0x7f47040c8950 2024-03-06T07:25:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f47027fc640 1 -- 172.21.15.26:0/1614624474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8031b80 con 0x7f47040c8950 2024-03-06T07:25:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 -- 172.21.15.26:0/1614624474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 msgr2=0x7f47040acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 --2- 172.21.15.26:0/1614624474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47040acd70 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802b860 comp rx=0 tx=0).stop 2024-03-06T07:25:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 -- 172.21.15.26:0/1614624474 shutdown_connections 2024-03-06T07:25:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 --2- 172.21.15.26:0/1614624474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47040acd70 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 --2- 172.21.15.26:0/1614624474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f47040c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 -- 172.21.15.26:0/1614624474 >> 172.21.15.26:0/1614624474 conn(0x7f47040cae70 msgr2=0x7f47040cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 -- 172.21.15.26:0/1614624474 shutdown_connections 2024-03-06T07:25:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.604+0000 7f470a1f7640 1 -- 172.21.15.26:0/1614624474 wait complete. 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.605+0000 7f470a1f7640 1 Processor -- start 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f470a1f7640 1 -- start start 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f470a1f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47040ad2b0 0x7f470414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f470a1f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f470414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f470a1f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47041ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f470a1f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4704152a20 con 0x7f47040c8950 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f470a1f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4704155fe0 con 0x7f47040ad2b0 2024-03-06T07:25:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f470a1f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4704152ba0 con 0x7f47040c69f0 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f4703fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47041ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f4703fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47041ee3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52788/0 (socket says 172.21.15.26:52788) 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.606+0000 7f4703fff640 1 -- 172.21.15.26:0/788393051 learned_addr learned my addr 172.21.15.26:0/788393051 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f47037fe640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47040ad2b0 0x7f470414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f4702ffd640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f470414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f4703fff640 1 -- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 msgr2=0x7f470414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f4703fff640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f470414dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f4703fff640 1 -- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47040ad2b0 msgr2=0x7f470414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f4703fff640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47040ad2b0 0x7f470414b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f4703fff640 1 -- 172.21.15.26:0/788393051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040c0890 con 0x7f47040c8950 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f4703fff640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47041ee3c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b410 tx=0x7f46fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f46ebfff640 1 -- 172.21.15.26:0/788393051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc015030 con 0x7f47040c8950 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f46ebfff640 1 -- 172.21.15.26:0/788393051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc004020 con 0x7f47040c8950 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f46ebfff640 1 -- 172.21.15.26:0/788393051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc004410 con 0x7f47040c8950 2024-03-06T07:25:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4704156800 con 0x7f47040c8950 2024-03-06T07:25:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.607+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4704158920 con 0x7f47040c8950 2024-03-06T07:25:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.608+0000 7f46ebfff640 1 -- 172.21.15.26:0/788393051 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46fc018070 con 0x7f47040c8950 2024-03-06T07:25:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.609+0000 7f46ebfff640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46f006b4d0 0x7f46f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.609+0000 7f46ebfff640 1 -- 172.21.15.26:0/788393051 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f46fc089e60 con 0x7f47040c8950 2024-03-06T07:25:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.609+0000 7f47037fe640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46f006b4d0 0x7f46f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.610+0000 7f47037fe640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46f006b4d0 0x7f46f006d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f470414e5a0 tx=0x7f46f40079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.611+0000 7f470cde6740 1 Processor -- start 2024-03-06T07:25:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.611+0000 7f470cde6740 1 -- start start 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2051f50 0x563da204fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2052330 0x563da204ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f470cde6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da20504b0 0x563da204c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f470cde6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da1fb66b0 con 0x563da20504b0 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da1f5fa60 con 0x563da2052330 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563da1fb2750 con 0x563da2051f50 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f4703fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da20504b0 0x563da204c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f4703fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da20504b0 0x563da204c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52798/0 (socket says 172.21.15.26:52798) 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.612+0000 7f4703fff640 1 -- 172.21.15.26:0/1408537092 learned_addr learned my addr 172.21.15.26:0/1408537092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f47037fe640 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2051f50 0x563da204fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4702ffd640 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2052330 0x563da204ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4703fff640 1 -- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2051f50 msgr2=0x563da204fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4703fff640 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2051f50 0x563da204fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4703fff640 1 -- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2052330 msgr2=0x563da204ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4703fff640 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2052330 0x563da204ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4703fff640 1 -- 172.21.15.26:0/1408537092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563da1f7df30 con 0x563da20504b0 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4702ffd640 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2052330 0x563da204ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f4703fff640 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da20504b0 0x563da204c680 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009210 tx=0x7f46fc058570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52f0094ece637b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f46e97fa640 1 -- 172.21.15.26:0/1408537092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc056730 con 0x563da20504b0 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f46e97fa640 1 -- 172.21.15.26:0/1408537092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc054aa0 con 0x563da20504b0 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.613+0000 7f46e97fa640 1 -- 172.21.15.26:0/1408537092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc081b40 con 0x563da20504b0 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 -- 172.21.15.26:0/1408537092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da20504b0 msgr2=0x563da204c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da20504b0 0x563da204c680 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009210 tx=0x7f46fc058570 comp rx=0 tx=0).stop 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 -- 172.21.15.26:0/1408537092 shutdown_connections 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da20504b0 0x563da204c680 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2052330 0x563da204ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 --2- 172.21.15.26:0/1408537092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2051f50 0x563da204fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 -- 172.21.15.26:0/1408537092 >> 172.21.15.26:0/1408537092 conn(0x563da204e7d0 msgr2=0x563da20733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 -- 172.21.15.26:0/1408537092 shutdown_connections 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 -- 172.21.15.26:0/1408537092 wait complete. 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 -- start start 2024-03-06T07:25:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.614+0000 7f470cde6740 1 Processor -- start 2024-03-06T07:25:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f470cde6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2051f50 0x563da20727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2052330 0x563da20a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2076120 0x563da20765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f470cde6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da1fb66b0 con 0x563da2051f50 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da1f72ea0 con 0x563da2076120 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563da2074230 con 0x563da2052330 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f470cde6740 10 client.0 _ll_register_callbacks cb 0x563da1f72870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f47037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2051f50 0x563da20727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f47037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2051f50 0x563da20727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52810/0 (socket says 172.21.15.26:52810) 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f47037fe640 1 -- 172.21.15.26:0/3154030364 learned_addr learned my addr 172.21.15.26:0/3154030364 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f4702ffd640 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2052330 0x563da20a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.615+0000 7f4703fff640 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2076120 0x563da20765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f47037fe640 1 -- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2052330 msgr2=0x563da20a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f47037fe640 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2052330 0x563da20a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f47037fe640 1 -- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2076120 msgr2=0x563da20765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f47037fe640 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2076120 0x563da20765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f47037fe640 1 -- 172.21.15.26:0/3154030364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563da1f73270 con 0x563da2051f50 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f47037fe640 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2051f50 0x563da20727b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f46f4004610 tx=0x7f46f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817678f58c26e7ce server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f46e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f46e27fc640 1 -- 172.21.15.26:0/3154030364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f401e610 con 0x563da2051f50 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f46e27fc640 1 -- 172.21.15.26:0/3154030364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f46f401e7b0 con 0x563da2051f50 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46fc054aa0 con 0x563da2051f50 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f46e27fc640 1 -- 172.21.15.26:0/3154030364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f402c050 con 0x563da2051f50 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 10 client.8130 fetch_fsmap learned FSMap version 115 2024-03-06T07:25:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x563da1ccb2d0 con 0x563da2051f50 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f46e27fc640 1 -- 172.21.15.26:0/3154030364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc054aa0 con 0x563da2051f50 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f46e27fc640 1 -- 172.21.15.26:0/3154030364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 115) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f46f402f020 con 0x563da2051f50 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 10 client.8130 fetch_fsmap finished waiting for FSMap version 115 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 10 client.8130 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 20 client.8130 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da2075040 0x563da20ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 4 client.8130 mds_command: new command op to 8082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.616+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 --> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563da1fc3bf0 con 0x563da2075040 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.617+0000 7f4702ffd640 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da2075040 0x563da20ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.617+0000 7f4702ffd640 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da2075040 0x563da20ad230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5031f1833d0731b9 server_cookie=d575f1430e9071d0 in_seq=0 out_seq=0 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.617+0000 7f46e27fc640 10 client.8130 ms_handle_connect on v2:172.21.15.26:6833/1367162338 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.617+0000 7f46e27fc640 1 -- 172.21.15.26:0/3154030364 <== mds.0 v2:172.21.15.26:6833/1367162338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563da1fc3bf0 con 0x563da2075040 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.617+0000 7f46e27fc640 10 client.8130 handle_command_reply: tid=0 2024-03-06T07:25:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.617+0000 7f470cde6740 1 client.8130 shutdown 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2051f50 msgr2=0x563da20727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2051f50 0x563da20727b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f46f4004610 tx=0x7f46f4004ab0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 shutdown_connections 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da2075040 0x563da20ad230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2076120 0x563da20765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2052330 0x563da20a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f46e27fc640 0 client.8130 ms_handle_reset on v2:172.21.15.26:6833/1367162338 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 --2- 172.21.15.26:0/3154030364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2051f50 0x563da20727b0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 >> 172.21.15.26:0/3154030364 conn(0x563da20504b0 msgr2=0x563da208fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 shutdown_connections 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 1 -- 172.21.15.26:0/3154030364 wait complete. 2024-03-06T07:25:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.618+0000 7f470cde6740 20 client.8130 trim_cache size 0 max 16384 2024-03-06T07:25:05.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.634+0000 7f470cde6740 1 Processor -- start 2024-03-06T07:25:05.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.634+0000 7f470cde6740 1 -- start start 2024-03-06T07:25:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2090dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f470cde6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da20718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da208eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f470cde6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47041f7fa0 con 0x563da2052330 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47041f70e0 con 0x563da2051f50 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4704145cf0 con 0x563da2076120 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f4702ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da20718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f4702ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da20718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52822/0 (socket says 172.21.15.26:52822) 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f4702ffd640 1 -- 172.21.15.26:0/769433409 learned_addr learned my addr 172.21.15.26:0/769433409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f47037fe640 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2090dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.635+0000 7f4703fff640 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da208eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f4702ffd640 1 -- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 msgr2=0x563da208eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f4702ffd640 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da208eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f4702ffd640 1 -- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 msgr2=0x563da2090dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f4702ffd640 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2090dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f4702ffd640 1 -- 172.21.15.26:0/769433409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563da1d956b0 con 0x563da2052330 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f4702ffd640 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da20718b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55d2a2199801968 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f46e8ff9640 1 -- 172.21.15.26:0/769433409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f804f070 con 0x563da2052330 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f46e8ff9640 1 -- 172.21.15.26:0/769433409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8037040 con 0x563da2052330 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f46e8ff9640 1 -- 172.21.15.26:0/769433409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004360 con 0x563da2052330 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f470cde6740 1 -- 172.21.15.26:0/769433409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 msgr2=0x563da20718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.636+0000 7f470cde6740 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da20718b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f8030000 comp rx=0 tx=0).stop 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 -- 172.21.15.26:0/769433409 shutdown_connections 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da208eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da20718b0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 --2- 172.21.15.26:0/769433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2090dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 -- 172.21.15.26:0/769433409 >> 172.21.15.26:0/769433409 conn(0x563da20504b0 msgr2=0x563da20725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 -- 172.21.15.26:0/769433409 shutdown_connections 2024-03-06T07:25:05.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 -- 172.21.15.26:0/769433409 wait complete. 2024-03-06T07:25:05.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 -- start start 2024-03-06T07:25:05.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.637+0000 7f470cde6740 1 Processor -- start 2024-03-06T07:25:05.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f470cde6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da2100a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f470cde6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da20fa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f470cde6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da2078430 con 0x563da2052330 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47041f6b10 con 0x563da2051f50 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f470cde6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563da2074b60 con 0x563da2076120 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f470cde6740 10 client.0 _ll_register_callbacks cb 0x7f46f4004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da2100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da2100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52830/0 (socket says 172.21.15.26:52830) 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 -- 172.21.15.26:0/3478973403 learned_addr learned my addr 172.21.15.26:0/3478973403 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f47037fe640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4703fff640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da20fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 -- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 msgr2=0x563da20fa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da20fa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 -- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 msgr2=0x563da2071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2071cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 -- 172.21.15.26:0/3478973403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563da204d190 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4703fff640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da20fa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.638+0000 7f4702ffd640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da2100a40 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f46f802be70 tx=0x7f46f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8543f3dc4b7065d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f46e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f46e27fc640 1 -- 172.21.15.26:0/3478973403 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8059060 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f46e27fc640 1 -- 172.21.15.26:0/3478973403 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f46f8004340 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46f8037040 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f46e27fc640 1 -- 172.21.15.26:0/3478973403 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f805f050 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 10 client.8136 fetch_fsmap learned FSMap version 115 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x563da2100f80 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f46e27fc640 1 -- 172.21.15.26:0/3478973403 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8037040 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f46e27fc640 1 -- 172.21.15.26:0/3478973403 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 115) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f46f8062020 con 0x563da2052330 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 10 client.8136 fetch_fsmap finished waiting for FSMap version 115 2024-03-06T07:25:05.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 10 client.8136 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 20 client.8136 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da209e5c0 0x563da209e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 4 client.8136 mds_command: new command op to 8082 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 --> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563da1fbdf40 con 0x563da209e5c0 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.639+0000 7f47037fe640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da209e5c0 0x563da209e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f46e27fc640 10 client.8136 ms_handle_connect on v2:172.21.15.26:6833/1367162338 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f47037fe640 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da209e5c0 0x563da209e9a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55999cf259b8d400 server_cookie=75e08fe97c8fd727 in_seq=0 out_seq=0 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f46e27fc640 1 -- 172.21.15.26:0/3478973403 <== mds.0 v2:172.21.15.26:6833/1367162338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x563da1fbdf40 con 0x563da209e5c0 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f46e27fc640 10 client.8136 handle_command_reply: tid=0 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 client.8136 shutdown 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 msgr2=0x563da2100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da2100a40 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f46f802be70 tx=0x7f46f8004730 comp rx=0 tx=0).stop 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 shutdown_connections 2024-03-06T07:25:05.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:6833/1367162338,v1:172.21.15.26:6835/1367162338] conn(0x563da209e5c0 0x563da209e9a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563da2076120 0x563da20fa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563da2052330 0x563da2100a40 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 --2- 172.21.15.26:0/3478973403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563da2051f50 0x563da2071cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 >> 172.21.15.26:0/3478973403 conn(0x563da20504b0 msgr2=0x563da2051150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f46e27fc640 0 client.8136 ms_handle_reset on v2:172.21.15.26:6833/1367162338 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 shutdown_connections 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 1 -- 172.21.15.26:0/3478973403 wait complete. 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.640+0000 7f470cde6740 20 client.8136 trim_cache size 0 max 16384 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:25:05.649 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8082, 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.94031033, 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 115, 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 123, 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 123, 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 7.9296473880000002 2024-03-06T07:25:05.650 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46f006b4d0 msgr2=0x7f46f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46f006b4d0 0x7f46f006d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f470414e5a0 tx=0x7f46f40079a0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 msgr2=0x7f47041ee3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47041ee3c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b410 tx=0x7f46fc00b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 shutdown_connections 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46f006b4d0 0x7f46f006d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47040c8950 0x7f47041ee3c0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47040c69f0 0x7f470414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 --2- 172.21.15.26:0/788393051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47040ad2b0 0x7f470414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 >> 172.21.15.26:0/788393051 conn(0x7f47040cae70 msgr2=0x7f4704151540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 shutdown_connections 2024-03-06T07:25:05.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.641+0000 7f470a1f7640 1 -- 172.21.15.26:0/788393051 wait complete. 2024-03-06T07:25:05.657 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 115, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:43.287707+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 115, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8082": { "addr": "172.21.15.26:6835/1367162338", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 1367162338, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 1367162338, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8082, "incarnation": 112, "join_fscid": 9, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 123, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:25:03.710057+0000", "qdb_cluster": [ 8082 ], "qdb_leader": 8082, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8082 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/3943022187", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3943022187, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3943022187, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7465, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3949772965", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3949772965, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3949772965, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 101, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7468, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6834/317629496", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 317629496, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 317629496, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 113, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8103, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:05.658 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 9, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:25:05.658 INFO:tasks.cephfs.test_failover:current = [{'gid': 8103, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6834/317629496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 317629496}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 317629496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 113}, {'gid': 7465, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3943022187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3943022187}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3943022187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}, {'gid': 8082, 'name': 'c', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.26:6835/1367162338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1367162338}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1367162338}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7468, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3949772965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3949772965}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3949772965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 101}] 2024-03-06T07:25:05.658 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-06T07:25:05.658 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:25:05.658 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:05.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:05.745 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:25:05.745 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:25:05.745 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:05.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:25:05.761 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:25:05.761 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:05.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:25:05.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:05.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.899+0000 7f729b6eb640 1 Processor -- start 2024-03-06T07:25:05.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.900+0000 7f729b6eb640 1 -- start start 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f729b6eb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72940c5480 0x7f7294155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f729b6eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f72941586f0 0x7f7294158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f729b6eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72940c5cc0 con 0x7f72940c5860 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f729b6eb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729406df40 con 0x7f72940c5480 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f729b6eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729406fdb0 con 0x7f72941586f0 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7299487640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72940c5480 0x7f7294155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7298c86640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f72940c5860 0x7f729406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33686/0 (socket says 172.21.15.26:33686) 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7298c86640 1 -- 172.21.15.26:0/1638669108 learned_addr learned my addr 172.21.15.26:0/1638669108 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7299c88640 1 --2- 172.21.15.26:0/1638669108 >> v2:172.21.15.117:3300/0 conn(0x7f72941586f0 0x7f7294158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7299487640 1 -- 172.21.15.26:0/1638669108 >> v2:172.21.15.117:3300/0 conn(0x7f72941586f0 msgr2=0x7f7294158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7299487640 1 --2- 172.21.15.26:0/1638669108 >> v2:172.21.15.117:3300/0 conn(0x7f72941586f0 0x7f7294158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7299487640 1 -- 172.21.15.26:0/1638669108 >> v1:172.21.15.26:6789/0 conn(0x7f72940c5860 legacy=0x7f729406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.901+0000 7f7299487640 1 -- 172.21.15.26:0/1638669108 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72941591c0 con 0x7f72940c5480 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.902+0000 7f7299487640 1 --2- 172.21.15.26:0/1638669108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72940c5480 0x7f7294155ba0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f72940ab500 tx=0x7f729002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f637299b665b96e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.902+0000 7f728bfff640 1 -- 172.21.15.26:0/1638669108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900315a0 con 0x7f72940c5480 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.902+0000 7f728bfff640 1 -- 172.21.15.26:0/1638669108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7290037070 con 0x7f72940c5480 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.902+0000 7f728bfff640 1 -- 172.21.15.26:0/1638669108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290031960 con 0x7f72940c5480 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 -- 172.21.15.26:0/1638669108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72940c5480 msgr2=0x7f7294155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 --2- 172.21.15.26:0/1638669108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72940c5480 0x7f7294155ba0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f72940ab500 tx=0x7f729002b840 comp rx=0 tx=0).stop 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 -- 172.21.15.26:0/1638669108 shutdown_connections 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 --2- 172.21.15.26:0/1638669108 >> v2:172.21.15.117:3300/0 conn(0x7f72941586f0 0x7f7294158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 --2- 172.21.15.26:0/1638669108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72940c5480 0x7f7294155ba0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 -- 172.21.15.26:0/1638669108 >> 172.21.15.26:0/1638669108 conn(0x7f729406b1a0 msgr2=0x7f729406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 -- 172.21.15.26:0/1638669108 shutdown_connections 2024-03-06T07:25:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.903+0000 7f729b6eb640 1 -- 172.21.15.26:0/1638669108 wait complete. 2024-03-06T07:25:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.905+0000 7f729b6eb640 1 Processor -- start 2024-03-06T07:25:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.905+0000 7f729b6eb640 1 -- start start 2024-03-06T07:25:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f729b6eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72940c5480 0x7f7294153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f729b6eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72940c5860 0x7f7294156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f729b6eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72941586f0 0x7f72941572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f729b6eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7294155680 con 0x7f72941586f0 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f729b6eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72941553a0 con 0x7f72940c5860 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f729b6eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7294152a20 con 0x7f72940c5480 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72941586f0 0x7f72941572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72941586f0 0x7f72941572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52854/0 (socket says 172.21.15.26:52854) 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 -- 172.21.15.26:0/3839176278 learned_addr learned my addr 172.21.15.26:0/3839176278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7298c86640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72940c5860 0x7f7294156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299487640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72940c5480 0x7f7294153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 -- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72940c5480 msgr2=0x7f7294153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72940c5480 0x7f7294153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 -- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72940c5860 msgr2=0x7f7294156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72940c5860 0x7f7294156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 -- 172.21.15.26:0/3839176278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294157800 con 0x7f72941586f0 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299487640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72940c5480 0x7f7294153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.906+0000 7f7299c88640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72941586f0 0x7f72941572c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f728c007cf0 tx=0x7f728c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.907+0000 7f7289ffb640 1 -- 172.21.15.26:0/3839176278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c014420 con 0x7f72941586f0 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.907+0000 7f7289ffb640 1 -- 172.21.15.26:0/3839176278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c019070 con 0x7f72941586f0 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.907+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72941eba90 con 0x7f72941586f0 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.907+0000 7f7289ffb640 1 -- 172.21.15.26:0/3839176278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c014820 con 0x7f72941586f0 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.907+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7294153ae0 con 0x7f72941586f0 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.908+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729404f330 con 0x7f72941586f0 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.908+0000 7f7289ffb640 1 -- 172.21.15.26:0/3839176278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f728c0149c0 con 0x7f72941586f0 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.909+0000 7f7289ffb640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f727806b4d0 0x7f727806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.909+0000 7f7289ffb640 1 -- 172.21.15.26:0/3839176278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f728c08aba0 con 0x7f72941586f0 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.909+0000 7f7299487640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f727806b4d0 0x7f727806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.909+0000 7f7299487640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f727806b4d0 0x7f727806d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7290036000 tx=0x7f7290008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:05.913+0000 7f7289ffb640 1 -- 172.21.15.26:0/3839176278 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f728c059870 con 0x7f72941586f0 2024-03-06T07:25:06.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.130+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f729406d4b0 con 0x7f72941586f0 2024-03-06T07:25:06.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.131+0000 7f7289ffb640 1 -- 172.21.15.26:0/3839176278 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4989 (secure 0 0 0) 0x7f728c059170 con 0x7f72941586f0 2024-03-06T07:25:06.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:06.133 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/317629496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":317629496},{"type":"v1","addr":"172.21.15.26:6834","nonce":317629496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:25:03.710057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8082":{"gid":8082,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8082,"qdb_cluster":[8082]},"id":9}]} 2024-03-06T07:25:06.134 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 115 2024-03-06T07:25:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f727806b4d0 msgr2=0x7f727806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f727806b4d0 0x7f727806d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7290036000 tx=0x7f7290008030 comp rx=0 tx=0).stop 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72941586f0 msgr2=0x7f72941572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72941586f0 0x7f72941572c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f728c007cf0 tx=0x7f728c00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 shutdown_connections 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f727806b4d0 0x7f727806d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72941586f0 0x7f72941572c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72940c5860 0x7f7294156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 --2- 172.21.15.26:0/3839176278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72940c5480 0x7f7294153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 >> 172.21.15.26:0/3839176278 conn(0x7f729406b1a0 msgr2=0x7f729406ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 shutdown_connections 2024-03-06T07:25:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.133+0000 7f729b6eb640 1 -- 172.21.15.26:0/3839176278 wait complete. 2024-03-06T07:25:06.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:06.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.246+0000 7fd29afac640 1 Processor -- start 2024-03-06T07:25:06.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.246+0000 7fd29afac640 1 -- start start 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd29afac640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2940c5480 0x7fd294155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd29afac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5860 0x7fd29406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd29afac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940c5cc0 con 0x7fd2941586f0 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd29afac640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29406df40 con 0x7fd2940c5480 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd29afac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2941591c0 con 0x7fd2940c5860 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd298d48640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2940c5480 0x7fd294155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd298d48640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2940c5480 0x7fd294155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52862/0 (socket says 172.21.15.26:52862) 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd298d48640 1 -- 172.21.15.26:0/1852596476 learned_addr learned my addr 172.21.15.26:0/1852596476 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:06.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd293fff640 1 --2- 172.21.15.26:0/1852596476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5860 0x7fd29406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd298d48640 1 -- 172.21.15.26:0/1852596476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5860 msgr2=0x7fd29406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd298d48640 1 --2- 172.21.15.26:0/1852596476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5860 0x7fd29406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd298d48640 1 -- 172.21.15.26:0/1852596476 >> v1:172.21.15.117:6789/0 conn(0x7fd2941586f0 legacy=0x7fd2940724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.247+0000 7fd298d48640 1 -- 172.21.15.26:0/1852596476 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2941594a0 con 0x7fd2940c5480 2024-03-06T07:25:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.248+0000 7fd298d48640 1 --2- 172.21.15.26:0/1852596476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c5480 0x7fd294155ba0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd288009870 tx=0x7fd28802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec25cbe59a5b25d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.248+0000 7fd2937fe640 1 -- 172.21.15.26:0/1852596476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288035020 con 0x7fd2940c5480 2024-03-06T07:25:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.248+0000 7fd2937fe640 1 -- 172.21.15.26:0/1852596476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd288031640 con 0x7fd2940c5480 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.248+0000 7fd2937fe640 1 -- 172.21.15.26:0/1852596476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28802d040 con 0x7fd2940c5480 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 -- 172.21.15.26:0/1852596476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c5480 msgr2=0x7fd294155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 --2- 172.21.15.26:0/1852596476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c5480 0x7fd294155ba0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd288009870 tx=0x7fd28802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 -- 172.21.15.26:0/1852596476 shutdown_connections 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 --2- 172.21.15.26:0/1852596476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5860 0x7fd29406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 --2- 172.21.15.26:0/1852596476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2940c5480 0x7fd294155ba0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 -- 172.21.15.26:0/1852596476 >> 172.21.15.26:0/1852596476 conn(0x7fd29406b1a0 msgr2=0x7fd29406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 -- 172.21.15.26:0/1852596476 shutdown_connections 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.249+0000 7fd29afac640 1 -- 172.21.15.26:0/1852596476 wait complete. 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.250+0000 7fd29afac640 1 Processor -- start 2024-03-06T07:25:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.250+0000 7fd29afac640 1 -- start start 2024-03-06T07:25:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd29afac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5480 0x7fd2941e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd29afac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2940c5860 0x7fd2940abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd29afac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2941586f0 0x7fd2940ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd29afac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294152a20 con 0x7fd2941586f0 2024-03-06T07:25:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd29afac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294155680 con 0x7fd2940c5860 2024-03-06T07:25:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd29afac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd294155050 con 0x7fd2940c5480 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2941586f0 0x7fd2940ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2941586f0 0x7fd2940ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52876/0 (socket says 172.21.15.26:52876) 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 -- 172.21.15.26:0/1886665701 learned_addr learned my addr 172.21.15.26:0/1886665701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd298d48640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5480 0x7fd2941e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd293fff640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2940c5860 0x7fd2940abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 -- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5480 msgr2=0x7fd2941e0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5480 0x7fd2941e0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 -- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2940c5860 msgr2=0x7fd2940abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2940c5860 0x7fd2940abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 -- 172.21.15.26:0/1886665701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940aad80 con 0x7fd2941586f0 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd293fff640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2940c5860 0x7fd2940abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.251+0000 7fd299549640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2941586f0 0x7fd2940ac000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c730 tx=0x7fd28c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.252+0000 7fd2917fa640 1 -- 172.21.15.26:0/1886665701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c003b70 con 0x7fd2941586f0 2024-03-06T07:25:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.252+0000 7fd2917fa640 1 -- 172.21.15.26:0/1886665701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c003d30 con 0x7fd2941586f0 2024-03-06T07:25:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.252+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2941eed40 con 0x7fd2941586f0 2024-03-06T07:25:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.252+0000 7fd2917fa640 1 -- 172.21.15.26:0/1886665701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c005310 con 0x7fd2941586f0 2024-03-06T07:25:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.252+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2940c4870 con 0x7fd2941586f0 2024-03-06T07:25:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.253+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2941ef260 con 0x7fd2941586f0 2024-03-06T07:25:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.253+0000 7fd2917fa640 1 -- 172.21.15.26:0/1886665701 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd28c0054b0 con 0x7fd2941586f0 2024-03-06T07:25:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.254+0000 7fd2917fa640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27806b4d0 0x7fd27806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.254+0000 7fd2917fa640 1 -- 172.21.15.26:0/1886665701 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd28c08a270 con 0x7fd2941586f0 2024-03-06T07:25:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.254+0000 7fd298d48640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27806b4d0 0x7fd27806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.254+0000 7fd298d48640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27806b4d0 0x7fd27806d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd288036000 tx=0x7fd28803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.258+0000 7fd2917fa640 1 -- 172.21.15.26:0/1886665701 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd2941ef260 con 0x7fd2941586f0 2024-03-06T07:25:06.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.441+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2940c4a70 con 0x7fd2941586f0 2024-03-06T07:25:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.442+0000 7fd2917fa640 1 -- 172.21.15.26:0/1886665701 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17169 (secure 0 0 0) 0x7fd28c058f40 con 0x7fd2941586f0 2024-03-06T07:25:06.445 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:06.445 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":123,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:00.699519+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:41.558889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:42.251998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6832/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6835/577277314":"2024-03-07T07:24:57.677470+0000","172.21.15.26:6833/577277314":"2024-03-07T07:24:57.677470+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27806b4d0 msgr2=0x7fd27806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27806b4d0 0x7fd27806d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd288036000 tx=0x7fd28803c040 comp rx=0 tx=0).stop 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2941586f0 msgr2=0x7fd2940ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2941586f0 0x7fd2940ac000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c730 tx=0x7fd28c00cbf0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 shutdown_connections 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd27806b4d0 0x7fd27806d990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2941586f0 0x7fd2940ac000 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2940c5860 0x7fd2940abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 --2- 172.21.15.26:0/1886665701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2940c5480 0x7fd2941e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 >> 172.21.15.26:0/1886665701 conn(0x7fd29406b1a0 msgr2=0x7fd29406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 shutdown_connections 2024-03-06T07:25:06.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.444+0000 7fd29afac640 1 -- 172.21.15.26:0/1886665701 wait complete. 2024-03-06T07:25:06.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:24:41.558889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:24:42.251998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:06.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:06.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.559+0000 7f11db5e9640 1 Processor -- start 2024-03-06T07:25:06.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.559+0000 7f11db5e9640 1 -- start start 2024-03-06T07:25:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.560+0000 7f11db5e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f11d40b00d0 0x7f11d40c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.560+0000 7f11db5e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d40b04b0 con 0x7f11d406b8e0 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.560+0000 7f11db5e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d406d210 con 0x7f11d40c7210 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.560+0000 7f11db5e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d4156160 con 0x7f11d40b00d0 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.560+0000 7f11d8b84640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f11d406b8e0 0x7f11d406bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33696/0 (socket says 172.21.15.26:33696) 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.560+0000 7f11d8b84640 1 -- 172.21.15.26:0/1780617412 learned_addr learned my addr 172.21.15.26:0/1780617412 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9385640 1 --2- 172.21.15.26:0/1780617412 >> v2:172.21.15.117:3300/0 conn(0x7f11d40b00d0 0x7f11d40c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908057988 0 0) 0x7f11d40b04b0 con 0x7f11d406b8e0 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c0003520 con 0x7f11d406b8e0 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208875770 0 0) 0x7f11c0003520 con 0x7f11d406b8e0 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 >> v2:172.21.15.117:3300/0 conn(0x7f11d40b00d0 msgr2=0x7f11d40c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 --2- 172.21.15.26:0/1780617412 >> v2:172.21.15.117:3300/0 conn(0x7f11d40b00d0 0x7f11d40c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 >> v1:172.21.15.117:6789/0 conn(0x7f11d40c7210 legacy=0x7f11d40c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4155e80 con 0x7f11d406b8e0 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f11c4001670 con 0x7f11d406b8e0 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 >> v1:172.21.15.26:6789/0 conn(0x7f11d406b8e0 legacy=0x7f11d406bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 0x7f11c0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c00051c0 0x7f11c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d4156160 con 0x7f11c00051c0 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c000c090 con 0x7f11c0004800 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c000c290 con 0x7f11c00039b0 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9b86640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c00051c0 0x7f11c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d8b84640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9385640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 0x7f11c0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9b86640 1 -- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 msgr2=0x7f11c0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9b86640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 0x7f11c0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9b86640 1 -- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c0004800 msgr2=0x7f11c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9b86640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.561+0000 7f11d9b86640 1 -- 172.21.15.26:0/1780617412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0003520 con 0x7f11c00051c0 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.562+0000 7f11d9b86640 1 -- 172.21.15.26:0/1780617412 reap_dead start 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.562+0000 7f11d9b86640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c00051c0 0x7f11c000b9c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f11cc001e50 tx=0x7f11c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a24d52ee7f90ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.562+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc001a70 con 0x7f11c00051c0 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.562+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc003b90 con 0x7f11c00051c0 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.562+0000 7f11cbfff640 1 -- 172.21.15.26:0/1780617412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0050d0 con 0x7f11c00051c0 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.563+0000 7f11db5e9640 1 -- 172.21.15.26:0/1780617412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c00051c0 msgr2=0x7f11c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.563+0000 7f11db5e9640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c00051c0 0x7f11c000b9c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f11cc001e50 tx=0x7f11c0005830 comp rx=0 tx=0).stop 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.563+0000 7f11db5e9640 1 -- 172.21.15.26:0/1780617412 shutdown_connections 2024-03-06T07:25:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.563+0000 7f11db5e9640 1 --2- 172.21.15.26:0/1780617412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c00051c0 0x7f11c000b9c0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.563+0000 7f11db5e9640 1 -- 172.21.15.26:0/1780617412 >> 172.21.15.26:0/1780617412 conn(0x7f11d40ac810 msgr2=0x7f11d40acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.563+0000 7f11db5e9640 1 -- 172.21.15.26:0/1780617412 shutdown_connections 2024-03-06T07:25:06.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.563+0000 7f11db5e9640 1 -- 172.21.15.26:0/1780617412 wait complete. 2024-03-06T07:25:06.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.564+0000 7f11db5e9640 1 Processor -- start 2024-03-06T07:25:06.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.564+0000 7f11db5e9640 1 -- start start 2024-03-06T07:25:06.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11db5e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 0x7f11d40c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11db5e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d40c1d30 0x7f11d41e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11db5e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d41515f0 0x7f11d4151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11db5e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d4152c50 con 0x7f11d41515f0 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11db5e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40c5710 con 0x7f11c00051c0 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11db5e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40c8ff0 con 0x7f11d40c1d30 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d41515f0 0x7f11d4151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d41515f0 0x7f11d4151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52906/0 (socket says 172.21.15.26:52906) 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 -- 172.21.15.26:0/3793990052 learned_addr learned my addr 172.21.15.26:0/3793990052 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d8b84640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d40c1d30 0x7f11d41e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9385640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 0x7f11d40c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 -- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d40c1d30 msgr2=0x7f11d41e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d40c1d30 0x7f11d41e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 -- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 msgr2=0x7f11d40c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 0x7f11d40c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 -- 172.21.15.26:0/3793990052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d41eec20 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.565+0000 7f11d9b86640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d41515f0 0x7f11d4151a70 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f11cc02c060 tx=0x7f11cc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.566+0000 7f11c9ffb640 1 -- 172.21.15.26:0/3793990052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc007db0 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.566+0000 7f11c9ffb640 1 -- 172.21.15.26:0/3793990052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc003d40 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.566+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d41e05a0 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.566+0000 7f11c9ffb640 1 -- 172.21.15.26:0/3793990052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc006d70 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.566+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d41df8e0 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.567+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d4152f60 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.568+0000 7f11c9ffb640 1 -- 172.21.15.26:0/3793990052 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11cc007030 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.568+0000 7f11c9ffb640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11b806b4d0 0x7f11b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.568+0000 7f11c9ffb640 1 -- 172.21.15.26:0/3793990052 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f11cc0abf00 con 0x7f11d41515f0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.569+0000 7f11d9385640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11b806b4d0 0x7f11b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.569+0000 7f11d9385640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11b806b4d0 0x7f11b806d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009b40 tx=0x7f11d0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.573+0000 7f11c9ffb640 1 -- 172.21.15.26:0/3793990052 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f11d4152f60 con 0x7f11d41515f0 2024-03-06T07:25:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.779+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11d40c2110 con 0x7f11d41515f0 2024-03-06T07:25:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.779+0000 7f11c9ffb640 1 -- 172.21.15.26:0/3793990052 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11cc049060 con 0x7f11d41515f0 2024-03-06T07:25:06.782 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:25:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11b806b4d0 msgr2=0x7f11b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11b806b4d0 0x7f11b806d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009b40 tx=0x7f11d0009190 comp rx=0 tx=0).stop 2024-03-06T07:25:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d41515f0 msgr2=0x7f11d4151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d41515f0 0x7f11d4151a70 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f11cc02c060 tx=0x7f11cc0014a0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 shutdown_connections 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f11b806b4d0 0x7f11b806d990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d41515f0 0x7f11d4151a70 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d40c1d30 0x7f11d41e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 --2- 172.21.15.26:0/3793990052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 0x7f11d40c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 >> 172.21.15.26:0/3793990052 conn(0x7f11d40ac810 msgr2=0x7f11d4154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 shutdown_connections 2024-03-06T07:25:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.781+0000 7f11db5e9640 1 -- 172.21.15.26:0/3793990052 wait complete. 2024-03-06T07:25:06.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:25:06.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.898+0000 7f7d99d56640 1 Processor -- start 2024-03-06T07:25:06.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.898+0000 7f7d99d56640 1 -- start start 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d99d56640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d940c8960 0x7f7d94155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d99d56640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7d94158aa0 0x7f7d94158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d99d56640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9414f630 con 0x7f7d9414f250 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d99d56640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d940c7470 con 0x7f7d940c8960 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d99d56640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d940c7f60 con 0x7f7d94158aa0 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d92ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d940c8960 0x7f7d94155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7d9414f250 0x7f7d940c53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33706/0 (socket says 172.21.15.26:33706) 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d937fe640 1 -- 172.21.15.26:0/1245042551 learned_addr learned my addr 172.21.15.26:0/1245042551 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d93fff640 1 --2- 172.21.15.26:0/1245042551 >> v2:172.21.15.117:3300/0 conn(0x7f7d94158aa0 0x7f7d94158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d92ffd640 1 -- 172.21.15.26:0/1245042551 >> v2:172.21.15.117:3300/0 conn(0x7f7d94158aa0 msgr2=0x7f7d94158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d92ffd640 1 --2- 172.21.15.26:0/1245042551 >> v2:172.21.15.117:3300/0 conn(0x7f7d94158aa0 0x7f7d94158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.899+0000 7f7d92ffd640 1 -- 172.21.15.26:0/1245042551 >> v1:172.21.15.26:6789/0 conn(0x7f7d9414f250 legacy=0x7f7d940c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.900+0000 7f7d92ffd640 1 -- 172.21.15.26:0/1245042551 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d940c76c0 con 0x7f7d940c8960 2024-03-06T07:25:06.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.900+0000 7f7d927fc640 1 -- 172.21.15.26:0/1245042551 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892946100 0 0) 0x7f7d9414f630 con 0x7f7d9414f250 2024-03-06T07:25:06.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d92ffd640 1 --2- 172.21.15.26:0/1245042551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d940c8960 0x7f7d94155bc0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7d9408c750 tx=0x7f7d8402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316c3165a68dedd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d927fc640 1 -- 172.21.15.26:0/1245042551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84038070 con 0x7f7d940c8960 2024-03-06T07:25:06.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d927fc640 1 -- 172.21.15.26:0/1245042551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d84031710 con 0x7f7d940c8960 2024-03-06T07:25:06.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d927fc640 1 -- 172.21.15.26:0/1245042551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84031ae0 con 0x7f7d940c8960 2024-03-06T07:25:06.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 -- 172.21.15.26:0/1245042551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d940c8960 msgr2=0x7f7d94155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 --2- 172.21.15.26:0/1245042551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d940c8960 0x7f7d94155bc0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7d9408c750 tx=0x7f7d8402b7b0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 -- 172.21.15.26:0/1245042551 shutdown_connections 2024-03-06T07:25:06.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 --2- 172.21.15.26:0/1245042551 >> v2:172.21.15.117:3300/0 conn(0x7f7d94158aa0 0x7f7d94158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 --2- 172.21.15.26:0/1245042551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d940c8960 0x7f7d94155bc0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 -- 172.21.15.26:0/1245042551 >> 172.21.15.26:0/1245042551 conn(0x7f7d940ac610 msgr2=0x7f7d940aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 -- 172.21.15.26:0/1245042551 shutdown_connections 2024-03-06T07:25:06.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.901+0000 7f7d99d56640 1 -- 172.21.15.26:0/1245042551 wait complete. 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.902+0000 7f7d99d56640 1 Processor -- start 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d99d56640 1 -- start start 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d99d56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d940c8960 0x7f7d940ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d99d56640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9414f250 0x7f7d941ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d99d56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d94158aa0 0x7f7d94150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d99d56640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940be310 con 0x7f7d94158aa0 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d99d56640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940c7c80 con 0x7f7d9414f250 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d99d56640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940c5af0 con 0x7f7d940c8960 2024-03-06T07:25:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d94158aa0 0x7f7d94150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d94158aa0 0x7f7d94150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52922/0 (socket says 172.21.15.26:52922) 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.903+0000 7f7d93fff640 1 -- 172.21.15.26:0/679477957 learned_addr learned my addr 172.21.15.26:0/679477957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d937fe640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d940c8960 0x7f7d940ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d92ffd640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9414f250 0x7f7d941ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d93fff640 1 -- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d940c8960 msgr2=0x7f7d940ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d93fff640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d940c8960 0x7f7d940ab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d93fff640 1 -- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9414f250 msgr2=0x7f7d941ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d93fff640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9414f250 0x7f7d941ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d93fff640 1 -- 172.21.15.26:0/679477957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d940b0670 con 0x7f7d94158aa0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d93fff640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d94158aa0 0x7f7d94150da0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c007e70 tx=0x7f7d8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d8bfff640 1 -- 172.21.15.26:0/679477957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c014620 con 0x7f7d94158aa0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d8bfff640 1 -- 172.21.15.26:0/679477957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c019070 con 0x7f7d94158aa0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d8bfff640 1 -- 172.21.15.26:0/679477957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c014a20 con 0x7f7d94158aa0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d940b0390 con 0x7f7d94158aa0 2024-03-06T07:25:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.904+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d941eace0 con 0x7f7d94158aa0 2024-03-06T07:25:06.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.905+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9404f330 con 0x7f7d94158aa0 2024-03-06T07:25:06.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.906+0000 7f7d8bfff640 1 -- 172.21.15.26:0/679477957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d8c014bc0 con 0x7f7d94158aa0 2024-03-06T07:25:06.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.906+0000 7f7d8bfff640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d7406b4d0 0x7f7d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.906+0000 7f7d8bfff640 1 -- 172.21.15.26:0/679477957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f7d8c08ad50 con 0x7f7d94158aa0 2024-03-06T07:25:06.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.906+0000 7f7d937fe640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d7406b4d0 0x7f7d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.907+0000 7f7d937fe640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d7406b4d0 0x7f7d7406d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c000f60 tx=0x7f7d7c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:06.913+0000 7f7d8bfff640 1 -- 172.21.15.26:0/679477957 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d8c08e030 con 0x7f7d94158aa0 2024-03-06T07:25:07.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.096+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d940ac350 con 0x7f7d94158aa0 2024-03-06T07:25:07.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.096+0000 7f7d8bfff640 1 -- 172.21.15.26:0/679477957 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d8c0599a0 con 0x7f7d94158aa0 2024-03-06T07:25:07.099 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:25:07.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d7406b4d0 msgr2=0x7f7d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d7406b4d0 0x7f7d7406d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c000f60 tx=0x7f7d7c007c00 comp rx=0 tx=0).stop 2024-03-06T07:25:07.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d94158aa0 msgr2=0x7f7d94150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d94158aa0 0x7f7d94150da0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c007e70 tx=0x7f7d8c00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 shutdown_connections 2024-03-06T07:25:07.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d7406b4d0 0x7f7d7406d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d94158aa0 0x7f7d94150da0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9414f250 0x7f7d941ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 --2- 172.21.15.26:0/679477957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d940c8960 0x7f7d940ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 >> 172.21.15.26:0/679477957 conn(0x7f7d940ac610 msgr2=0x7f7d940c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 shutdown_connections 2024-03-06T07:25:07.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.098+0000 7f7d99d56640 1 -- 172.21.15.26:0/679477957 wait complete. 2024-03-06T07:25:07.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:25:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.213+0000 7f6eec993640 1 Processor -- start 2024-03-06T07:25:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.214+0000 7f6eec993640 1 -- start start 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.214+0000 7f6eec993640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 0x7f6ee80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.214+0000 7f6eec993640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee8156060 con 0x7f6ee80c3a70 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.214+0000 7f6eec993640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80a8430 con 0x7f6ee806db30 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.214+0000 7f6eec993640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8155b30 con 0x7f6ee806d750 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.214+0000 7f6edffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6ee80c3a70 0x7f6ee8072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47568/0 (socket says 172.21.15.26:47568) 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.214+0000 7f6edffff640 1 -- 172.21.15.26:0/4010136155 learned_addr learned my addr 172.21.15.26:0/4010136155 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee659c640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 0x7f6ee80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973341585 0 0) 0x7f6ee8156060 con 0x7f6ee80c3a70 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed4003520 con 0x7f6ee80c3a70 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084247250 0 0) 0x7f6ee80a8430 con 0x7f6ee806db30 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee8156060 con 0x7f6ee806db30 2024-03-06T07:25:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 960612987 0 0) 0x7f6ed4003520 con 0x7f6ee80c3a70 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 msgr2=0x7f6ee80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 0x7f6ee80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 >> v1:172.21.15.117:6789/0 conn(0x7f6ee806db30 legacy=0x7f6ee80c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee8155d80 con 0x7f6ee80c3a70 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6ed0001670 con 0x7f6ee80c3a70 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 >> v1:172.21.15.26:6789/0 conn(0x7f6ee80c3a70 legacy=0x7f6ee8072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4003b70 0x7f6ed4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 0x7f6ed4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed4004d40 0x7f6ed40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8155b30 con 0x7f6ed4004d40 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed400bb80 con 0x7f6ed4003b70 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed400bd80 con 0x7f6ee806d750 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee6d9d640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed4004d40 0x7f6ed40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee659c640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4003b70 0x7f6ed4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6edffff640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 0x7f6ed4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee6d9d640 1 -- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 msgr2=0x7f6ed4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee6d9d640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee806d750 0x7f6ed4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee6d9d640 1 -- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4003b70 msgr2=0x7f6ed4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee6d9d640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4003b70 0x7f6ed4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.215+0000 7f6ee6d9d640 1 -- 172.21.15.26:0/4010136155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed4003520 con 0x7f6ed4004d40 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.216+0000 7f6ee6d9d640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed4004d40 0x7f6ed40051c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0001fd0 tx=0x7f6ed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710ad942ba5190eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.216+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0001bf0 con 0x7f6ed4004d40 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.216+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee0003b70 con 0x7f6ed4004d40 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.216+0000 7f6ee5d9b640 1 -- 172.21.15.26:0/4010136155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00050d0 con 0x7f6ed4004d40 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.217+0000 7f6eec993640 1 -- 172.21.15.26:0/4010136155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed4004d40 msgr2=0x7f6ed40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.217+0000 7f6eec993640 1 --2- 172.21.15.26:0/4010136155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ed4004d40 0x7f6ed40051c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0001fd0 tx=0x7f6ed4005830 comp rx=0 tx=0).stop 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.217+0000 7f6ee6d9d640 1 -- 172.21.15.26:0/4010136155 reap_dead start 2024-03-06T07:25:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.217+0000 7f6eec993640 1 -- 172.21.15.26:0/4010136155 shutdown_connections 2024-03-06T07:25:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.217+0000 7f6eec993640 1 -- 172.21.15.26:0/4010136155 >> 172.21.15.26:0/4010136155 conn(0x7f6ee80c5560 msgr2=0x7f6ee80c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.217+0000 7f6eec993640 1 -- 172.21.15.26:0/4010136155 shutdown_connections 2024-03-06T07:25:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.217+0000 7f6eec993640 1 -- 172.21.15.26:0/4010136155 wait complete. 2024-03-06T07:25:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.218+0000 7f6eec993640 1 Processor -- start 2024-03-06T07:25:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.218+0000 7f6eec993640 1 -- start start 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6eec993640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee8157f80 0x7f6ee8158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6eec993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee8156430 0x7f6ee80c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6eec993640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee80abeb0 0x7f6ee814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6eec993640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80c2720 con 0x7f6ee8156430 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6eec993640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80c25a0 con 0x7f6ee8157f80 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6eec993640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80a7cd0 con 0x7f6ee80abeb0 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee8156430 0x7f6ee80c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee8156430 0x7f6ee80c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52752/0 (socket says 172.21.15.26:52752) 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 -- 172.21.15.26:0/2612709711 learned_addr learned my addr 172.21.15.26:0/2612709711 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6ee659c640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee8157f80 0x7f6ee8158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6ee6d9d640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee80abeb0 0x7f6ee814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 -- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee80abeb0 msgr2=0x7f6ee814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee80abeb0 0x7f6ee814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 -- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee8157f80 msgr2=0x7f6ee8158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee8157f80 0x7f6ee8158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 -- 172.21.15.26:0/2612709711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee80ae550 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6ee6d9d640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee80abeb0 0x7f6ee814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.219+0000 7f6edffff640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee8156430 0x7f6ee80c7870 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00011e0 tx=0x7f6ed00074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.220+0000 7f6edeffd640 1 -- 172.21.15.26:0/2612709711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000f750 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.220+0000 7f6edeffd640 1 -- 172.21.15.26:0/2612709711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed0018070 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.220+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee80ae830 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.220+0000 7f6edeffd640 1 -- 172.21.15.26:0/2612709711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000fb50 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.220+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee80c8bc0 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.221+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee8158bb0 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.222+0000 7f6edeffd640 1 -- 172.21.15.26:0/2612709711 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ed000fcf0 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.222+0000 7f6edeffd640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ec806b4d0 0x7f6ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.222+0000 7f6edeffd640 1 -- 172.21.15.26:0/2612709711 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f6ed008a3d0 con 0x7f6ee8156430 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.222+0000 7f6ee659c640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ec806b4d0 0x7f6ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.223+0000 7f6ee659c640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ec806b4d0 0x7f6ec806d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009b00 tx=0x7f6ed8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.227+0000 7f6edeffd640 1 -- 172.21.15.26:0/2612709711 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ee8158bb0 con 0x7f6ee8156430 2024-03-06T07:25:07.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.405+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ee804f330 con 0x7f6ee8156430 2024-03-06T07:25:07.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.405+0000 7f6edeffd640 1 -- 172.21.15.26:0/2612709711 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ed0059010 con 0x7f6ee8156430 2024-03-06T07:25:07.408 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ec806b4d0 msgr2=0x7f6ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ec806b4d0 0x7f6ec806d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009b00 tx=0x7f6ed8008040 comp rx=0 tx=0).stop 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee8156430 msgr2=0x7f6ee80c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee8156430 0x7f6ee80c7870 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00011e0 tx=0x7f6ed00074f0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 shutdown_connections 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6ec806b4d0 0x7f6ec806d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ee80abeb0 0x7f6ee814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee8156430 0x7f6ee80c7870 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 --2- 172.21.15.26:0/2612709711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee8157f80 0x7f6ee8158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 >> 172.21.15.26:0/2612709711 conn(0x7f6ee80c5560 msgr2=0x7f6ee8155120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 shutdown_connections 2024-03-06T07:25:07.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.408+0000 7f6eec993640 1 -- 172.21.15.26:0/2612709711 wait complete. 2024-03-06T07:25:07.421 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.530+0000 7ffb7733a640 1 Processor -- start 2024-03-06T07:25:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.530+0000 7ffb7733a640 1 -- start start 2024-03-06T07:25:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb7733a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 0x7ffb70155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb7733a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffb70158b60 0x7ffb70158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb7733a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb700c73d0 con 0x7ffb7014f360 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb7733a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb700c8dd0 con 0x7ffb70158b60 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb7733a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700be2e0 con 0x7ffb700c88c0 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb750d6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffb7014f360 0x7ffb700c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47578/0 (socket says 172.21.15.26:47578) 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb750d6640 1 -- 172.21.15.26:0/2229179445 learned_addr learned my addr 172.21.15.26:0/2229179445 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574814305 0 0) 0x7ffb700c73d0 con 0x7ffb7014f360 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb758d7640 1 --2- 172.21.15.26:0/2229179445 >> v2:172.21.15.117:3300/0 conn(0x7ffb70158b60 0x7ffb70158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb748d5640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 0x7ffb70155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.531+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb58003520 con 0x7ffb7014f360 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 81405424 0 0) 0x7ffb58003520 con 0x7ffb7014f360 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 msgr2=0x7ffb70155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 0x7ffb70155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 >> v2:172.21.15.117:3300/0 conn(0x7ffb70158b60 msgr2=0x7ffb70158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 --2- 172.21.15.26:0/2229179445 >> v2:172.21.15.117:3300/0 conn(0x7ffb70158b60 0x7ffb70158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700c7670 con 0x7ffb7014f360 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7ffb600019f0 con 0x7ffb7014f360 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 >> v1:172.21.15.26:6789/0 conn(0x7ffb7014f360 legacy=0x7ffb700c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 0x7ffb580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb58004d40 0x7ffb580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700be2e0 con 0x7ffb58003f30 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5800b9c0 con 0x7ffb58004d40 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5800bb90 con 0x7ffb700c88c0 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb748d5640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb750d6640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 0x7ffb580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb758d7640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb58004d40 0x7ffb580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb748d5640 1 -- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 msgr2=0x7ffb580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb748d5640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 0x7ffb580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb748d5640 1 -- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb58004d40 msgr2=0x7ffb580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb748d5640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb58004d40 0x7ffb580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb748d5640 1 -- 172.21.15.26:0/2229179445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58003520 con 0x7ffb58003f30 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.532+0000 7ffb750d6640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb700c88c0 0x7ffb580039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.533+0000 7ffb748d5640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb58004800 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009ac0 tx=0x7ffb58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5efdde055e59e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.533+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64039070 con 0x7ffb58003f30 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.533+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6402c680 con 0x7ffb58003f30 2024-03-06T07:25:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.533+0000 7ffb6ffff640 1 -- 172.21.15.26:0/2229179445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64009690 con 0x7ffb58003f30 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.534+0000 7ffb7733a640 1 -- 172.21.15.26:0/2229179445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 msgr2=0x7ffb58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.534+0000 7ffb7733a640 1 --2- 172.21.15.26:0/2229179445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb58004800 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009ac0 tx=0x7ffb58005830 comp rx=0 tx=0).stop 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.534+0000 7ffb758d7640 1 -- 172.21.15.26:0/2229179445 reap_dead start 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.534+0000 7ffb7733a640 1 -- 172.21.15.26:0/2229179445 shutdown_connections 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.534+0000 7ffb7733a640 1 -- 172.21.15.26:0/2229179445 >> 172.21.15.26:0/2229179445 conn(0x7ffb700ac570 msgr2=0x7ffb700ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.534+0000 7ffb7733a640 1 -- 172.21.15.26:0/2229179445 shutdown_connections 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.534+0000 7ffb7733a640 1 -- 172.21.15.26:0/2229179445 wait complete. 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.536+0000 7ffb7733a640 1 Processor -- start 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.536+0000 7ffb7733a640 1 -- start start 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb7733a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb701ec320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb7733a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb701ee420 0x7ffb701eb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb7733a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb701ebbd0 0x7ffb700c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb7733a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c5a50 con 0x7ffb58003f30 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb7733a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c7c30 con 0x7ffb701ee420 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb7733a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c5bd0 con 0x7ffb701ebbd0 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb701ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb701ec320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52764/0 (socket says 172.21.15.26:52764) 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 -- 172.21.15.26:0/555252744 learned_addr learned my addr 172.21.15.26:0/555252744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 -- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb701ebbd0 msgr2=0x7ffb700c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb701ebbd0 0x7ffb700c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 -- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb701ee420 msgr2=0x7ffb701eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb701ee420 0x7ffb701eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 -- 172.21.15.26:0/555252744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb701eaeb0 con 0x7ffb58003f30 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb750d6640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb701ec320 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ffb60007ab0 tx=0x7ffb6000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb6dffb640 1 -- 172.21.15.26:0/555252744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60013420 con 0x7ffb58003f30 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb6dffb640 1 -- 172.21.15.26:0/555252744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60018070 con 0x7ffb58003f30 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.537+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb701eec00 con 0x7ffb58003f30 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.538+0000 7ffb6dffb640 1 -- 172.21.15.26:0/555252744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600137e0 con 0x7ffb58003f30 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.538+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb701ef1c0 con 0x7ffb58003f30 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.539+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb700c8590 con 0x7ffb58003f30 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.539+0000 7ffb6dffb640 1 -- 172.21.15.26:0/555252744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb60013980 con 0x7ffb58003f30 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.539+0000 7ffb6dffb640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b500 0x7ffb5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.539+0000 7ffb6dffb640 1 -- 172.21.15.26:0/555252744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ffb6008aae0 con 0x7ffb58003f30 2024-03-06T07:25:07.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.540+0000 7ffb748d5640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b500 0x7ffb5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.540+0000 7ffb748d5640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b500 0x7ffb5c06d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffb64036000 tx=0x7ffb6402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.544+0000 7ffb6dffb640 1 -- 172.21.15.26:0/555252744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffb700c8590 con 0x7ffb58003f30 2024-03-06T07:25:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.765+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb7004f330 con 0x7ffb58003f30 2024-03-06T07:25:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.765+0000 7ffb6dffb640 1 -- 172.21.15.26:0/555252744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4989 (secure 0 0 0) 0x7ffb600597b0 con 0x7ffb58003f30 2024-03-06T07:25:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:07.768 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/317629496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":317629496},{"type":"v1","addr":"172.21.15.26:6834","nonce":317629496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:25:03.710057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8082":{"gid":8082,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8082,"qdb_cluster":[8082]},"id":9}]} 2024-03-06T07:25:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 115 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b500 msgr2=0x7ffb5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b500 0x7ffb5c06d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffb64036000 tx=0x7ffb6402e000 comp rx=0 tx=0).stop 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 msgr2=0x7ffb701ec320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb701ec320 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ffb60007ab0 tx=0x7ffb6000e530 comp rx=0 tx=0).stop 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 shutdown_connections 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffb5c06b500 0x7ffb5c06d9c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb701ebbd0 0x7ffb700c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb701ee420 0x7ffb701eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 --2- 172.21.15.26:0/555252744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb58003f30 0x7ffb701ec320 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 >> 172.21.15.26:0/555252744 conn(0x7ffb700ac570 msgr2=0x7ffb7014d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 shutdown_connections 2024-03-06T07:25:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.767+0000 7ffb7733a640 1 -- 172.21.15.26:0/555252744 wait complete. 2024-03-06T07:25:07.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.913+0000 7fbf5b913640 1 Processor -- start 2024-03-06T07:25:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf5b913640 1 -- start start 2024-03-06T07:25:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf5b913640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf54152920 0x7fbf54155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf5b913640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbf540c8a00 0x7fbf54158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf5b913640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf54156180 con 0x7fbf54159460 2024-03-06T07:25:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf5b913640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf541587b0 con 0x7fbf54152920 2024-03-06T07:25:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf5b913640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf540c7760 con 0x7fbf540c8a00 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf596af640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf54152920 0x7fbf54155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf59eb0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbf54159460 0x7fbf540c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47586/0 (socket says 172.21.15.26:47586) 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.914+0000 7fbf59eb0640 1 -- 172.21.15.26:0/2956450848 learned_addr learned my addr 172.21.15.26:0/2956450848 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.915+0000 7fbf58eae640 1 --2- 172.21.15.26:0/2956450848 >> v2:172.21.15.117:3300/0 conn(0x7fbf540c8a00 0x7fbf54158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.915+0000 7fbf596af640 1 -- 172.21.15.26:0/2956450848 >> v2:172.21.15.117:3300/0 conn(0x7fbf540c8a00 msgr2=0x7fbf54158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.915+0000 7fbf596af640 1 --2- 172.21.15.26:0/2956450848 >> v2:172.21.15.117:3300/0 conn(0x7fbf540c8a00 0x7fbf54158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.915+0000 7fbf596af640 1 -- 172.21.15.26:0/2956450848 >> v1:172.21.15.26:6789/0 conn(0x7fbf54159460 legacy=0x7fbf540c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.915+0000 7fbf4bfff640 1 -- 172.21.15.26:0/2956450848 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514958774 0 0) 0x7fbf4c001490 con 0x7fbf54159460 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.915+0000 7fbf596af640 1 -- 172.21.15.26:0/2956450848 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540c7a40 con 0x7fbf54152920 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.915+0000 7fbf58eae640 1 --2- 172.21.15.26:0/2956450848 >> v2:172.21.15.117:3300/0 conn(0x7fbf540c8a00 0x7fbf54158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.916+0000 7fbf596af640 1 --2- 172.21.15.26:0/2956450848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf54152920 0x7fbf54155c40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbf54067c30 tx=0x7fbf4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f58f392b48ee0c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.916+0000 7fbf4bfff640 1 -- 172.21.15.26:0/2956450848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf440315a0 con 0x7fbf54152920 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.916+0000 7fbf4bfff640 1 -- 172.21.15.26:0/2956450848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf44037070 con 0x7fbf54152920 2024-03-06T07:25:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.916+0000 7fbf4bfff640 1 -- 172.21.15.26:0/2956450848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44031960 con 0x7fbf54152920 2024-03-06T07:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.916+0000 7fbf5b913640 1 -- 172.21.15.26:0/2956450848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf54152920 msgr2=0x7fbf54155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.916+0000 7fbf5b913640 1 --2- 172.21.15.26:0/2956450848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf54152920 0x7fbf54155c40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbf54067c30 tx=0x7fbf4402b840 comp rx=0 tx=0).stop 2024-03-06T07:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.917+0000 7fbf5b913640 1 -- 172.21.15.26:0/2956450848 shutdown_connections 2024-03-06T07:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.917+0000 7fbf5b913640 1 --2- 172.21.15.26:0/2956450848 >> v2:172.21.15.117:3300/0 conn(0x7fbf540c8a00 0x7fbf54158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.917+0000 7fbf5b913640 1 --2- 172.21.15.26:0/2956450848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf54152920 0x7fbf54155c40 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.917+0000 7fbf5b913640 1 -- 172.21.15.26:0/2956450848 >> 172.21.15.26:0/2956450848 conn(0x7fbf540ac560 msgr2=0x7fbf540ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.917+0000 7fbf5b913640 1 -- 172.21.15.26:0/2956450848 shutdown_connections 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.917+0000 7fbf5b913640 1 -- 172.21.15.26:0/2956450848 wait complete. 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 Processor -- start 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 -- start start 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf540c8a00 0x7fbf540aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54152920 0x7fbf540c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541464f0 0x7fbf5414d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5406d030 con 0x7fbf541464f0 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54158a00 con 0x7fbf54152920 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf5b913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540c5a40 con 0x7fbf540c8a00 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.918+0000 7fbf59eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541464f0 0x7fbf5414d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541464f0 0x7fbf5414d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52786/0 (socket says 172.21.15.26:52786) 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 -- 172.21.15.26:0/4248462893 learned_addr learned my addr 172.21.15.26:0/4248462893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf58eae640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54152920 0x7fbf540c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf596af640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf540c8a00 0x7fbf540aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 -- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf540c8a00 msgr2=0x7fbf540aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf540c8a00 0x7fbf540aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 -- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54152920 msgr2=0x7fbf540c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54152920 0x7fbf540c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 -- 172.21.15.26:0/4248462893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf541469b0 con 0x7fbf541464f0 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf59eb0640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541464f0 0x7fbf5414d490 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00c6d0 tx=0x7fbf4c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf49ffb640 1 -- 172.21.15.26:0/4248462893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c018070 con 0x7fbf541464f0 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf49ffb640 1 -- 172.21.15.26:0/4248462893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c004d60 con 0x7fbf541464f0 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf540abe10 con 0x7fbf541464f0 2024-03-06T07:25:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf49ffb640 1 -- 172.21.15.26:0/4248462893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00a880 con 0x7fbf541464f0 2024-03-06T07:25:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.919+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf540c9670 con 0x7fbf541464f0 2024-03-06T07:25:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.920+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf54067910 con 0x7fbf541464f0 2024-03-06T07:25:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.921+0000 7fbf49ffb640 1 -- 172.21.15.26:0/4248462893 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf4c01b070 con 0x7fbf541464f0 2024-03-06T07:25:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.921+0000 7fbf49ffb640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf3806b4d0 0x7fbf3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.921+0000 7fbf49ffb640 1 -- 172.21.15.26:0/4248462893 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbf4c089b40 con 0x7fbf541464f0 2024-03-06T07:25:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.922+0000 7fbf596af640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf3806b4d0 0x7fbf3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.922+0000 7fbf596af640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf3806b4d0 0x7fbf3806d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036000 tx=0x7fbf44008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:07.926+0000 7fbf49ffb640 1 -- 172.21.15.26:0/4248462893 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbf54067910 con 0x7fbf541464f0 2024-03-06T07:25:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.119+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf540c7390 con 0x7fbf541464f0 2024-03-06T07:25:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.120+0000 7fbf49ffb640 1 -- 172.21.15.26:0/4248462893 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17169 (secure 0 0 0) 0x7fbf540c7390 con 0x7fbf541464f0 2024-03-06T07:25:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:08.123 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":123,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:00.699519+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:41.558889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:42.251998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6832/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6835/577277314":"2024-03-07T07:24:57.677470+0000","172.21.15.26:6833/577277314":"2024-03-07T07:24:57.677470+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf3806b4d0 msgr2=0x7fbf3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf3806b4d0 0x7fbf3806d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036000 tx=0x7fbf44008030 comp rx=0 tx=0).stop 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541464f0 msgr2=0x7fbf5414d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541464f0 0x7fbf5414d490 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00c6d0 tx=0x7fbf4c00cb90 comp rx=0 tx=0).stop 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 shutdown_connections 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbf3806b4d0 0x7fbf3806d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf541464f0 0x7fbf5414d490 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54152920 0x7fbf540c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 --2- 172.21.15.26:0/4248462893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf540c8a00 0x7fbf540aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 >> 172.21.15.26:0/4248462893 conn(0x7fbf540ac560 msgr2=0x7fbf540ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 shutdown_connections 2024-03-06T07:25:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.122+0000 7fbf5b913640 1 -- 172.21.15.26:0/4248462893 wait complete. 2024-03-06T07:25:08.135 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:25:08.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.234+0000 7fd2e720f640 1 Processor -- start 2024-03-06T07:25:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.234+0000 7fd2e720f640 1 -- start start 2024-03-06T07:25:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.234+0000 7fd2e720f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e014f360 0x7fd2e0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.234+0000 7fd2e720f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2e00c88c0 0x7fd2e00c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e720f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e00c73d0 con 0x7fd2e00c5880 2024-03-06T07:25:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e720f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e00c8dd0 con 0x7fd2e00c88c0 2024-03-06T07:25:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e720f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00be2e0 con 0x7fd2e014f360 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e4fab640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e014f360 0x7fd2e0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2dffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2e00c88c0 0x7fd2e00c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e57ac640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd2e00c5880 0x7fd2e0158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44050/0 (socket says 172.21.15.26:44050) 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e57ac640 1 -- 172.21.15.26:0/3908265033 learned_addr learned my addr 172.21.15.26:0/3908265033 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e4fab640 1 -- 172.21.15.26:0/3908265033 >> v2:172.21.15.117:3300/0 conn(0x7fd2e00c88c0 msgr2=0x7fd2e00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e4fab640 1 --2- 172.21.15.26:0/3908265033 >> v2:172.21.15.117:3300/0 conn(0x7fd2e00c88c0 0x7fd2e00c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e4fab640 1 -- 172.21.15.26:0/3908265033 >> v1:172.21.15.117:6789/0 conn(0x7fd2e00c5880 legacy=0x7fd2e0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2e4fab640 1 -- 172.21.15.26:0/3908265033 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00c7670 con 0x7fd2e014f360 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2df7fe640 1 -- 172.21.15.26:0/3908265033 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977184929 0 0) 0x7fd2e00c73d0 con 0x7fd2e00c5880 2024-03-06T07:25:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.235+0000 7fd2dffff640 1 --2- 172.21.15.26:0/3908265033 >> v2:172.21.15.117:3300/0 conn(0x7fd2e00c88c0 0x7fd2e00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.236+0000 7fd2e4fab640 1 --2- 172.21.15.26:0/3908265033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e014f360 0x7fd2e0155c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40097a0 tx=0x7fd2d402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14fc841c48991ceb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.236+0000 7fd2df7fe640 1 -- 172.21.15.26:0/3908265033 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40315a0 con 0x7fd2e014f360 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.236+0000 7fd2df7fe640 1 -- 172.21.15.26:0/3908265033 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d4037070 con 0x7fd2e014f360 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.236+0000 7fd2df7fe640 1 -- 172.21.15.26:0/3908265033 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4031960 con 0x7fd2e014f360 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 -- 172.21.15.26:0/3908265033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e014f360 msgr2=0x7fd2e0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 --2- 172.21.15.26:0/3908265033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e014f360 0x7fd2e0155c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40097a0 tx=0x7fd2d402b840 comp rx=0 tx=0).stop 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 -- 172.21.15.26:0/3908265033 shutdown_connections 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 --2- 172.21.15.26:0/3908265033 >> v2:172.21.15.117:3300/0 conn(0x7fd2e00c88c0 0x7fd2e00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 --2- 172.21.15.26:0/3908265033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e014f360 0x7fd2e0155c10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 -- 172.21.15.26:0/3908265033 >> 172.21.15.26:0/3908265033 conn(0x7fd2e00ac570 msgr2=0x7fd2e00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 -- 172.21.15.26:0/3908265033 shutdown_connections 2024-03-06T07:25:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.237+0000 7fd2e720f640 1 -- 172.21.15.26:0/3908265033 wait complete. 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.238+0000 7fd2e720f640 1 Processor -- start 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.238+0000 7fd2e720f640 1 -- start start 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e720f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2e00c5880 0x7fd2e014b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e720f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e00c88c0 0x7fd2e014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e720f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e014f360 0x7fd2e01ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e720f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0159270 con 0x7fd2e014f360 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e720f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00c7c30 con 0x7fd2e00c5880 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e720f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e01593f0 con 0x7fd2e00c88c0 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e014f360 0x7fd2e01ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e014f360 0x7fd2e01ee1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52796/0 (socket says 172.21.15.26:52796) 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 -- 172.21.15.26:0/417277374 learned_addr learned my addr 172.21.15.26:0/417277374 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2dffff640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e00c88c0 0x7fd2e014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e4fab640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2e00c5880 0x7fd2e014b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 -- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e00c88c0 msgr2=0x7fd2e014bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e00c88c0 0x7fd2e014bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 -- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2e00c5880 msgr2=0x7fd2e014b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2e00c5880 0x7fd2e014b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 -- 172.21.15.26:0/417277374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00c1550 con 0x7fd2e014f360 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2e57ac640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e014f360 0x7fd2e01ee1a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c7d0 tx=0x7fd2d800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.239+0000 7fd2dd7fa640 1 -- 172.21.15.26:0/417277374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8018070 con 0x7fd2e014f360 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.240+0000 7fd2dd7fa640 1 -- 172.21.15.26:0/417277374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d800a640 con 0x7fd2e014f360 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.240+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e01ebff0 con 0x7fd2e014f360 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.240+0000 7fd2dd7fa640 1 -- 172.21.15.26:0/417277374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800aa30 con 0x7fd2e014f360 2024-03-06T07:25:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.240+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e01ec5b0 con 0x7fd2e014f360 2024-03-06T07:25:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.241+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e01ec7b0 con 0x7fd2e014f360 2024-03-06T07:25:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.241+0000 7fd2dd7fa640 1 -- 172.21.15.26:0/417277374 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2d801b070 con 0x7fd2e014f360 2024-03-06T07:25:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.241+0000 7fd2dd7fa640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd2c406b4d0 0x7fd2c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.242+0000 7fd2dd7fa640 1 -- 172.21.15.26:0/417277374 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd2d8089a30 con 0x7fd2e014f360 2024-03-06T07:25:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.242+0000 7fd2e4fab640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd2c406b4d0 0x7fd2c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.242+0000 7fd2e4fab640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd2c406b4d0 0x7fd2c406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4036000 tx=0x7fd2d4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.246+0000 7fd2dd7fa640 1 -- 172.21.15.26:0/417277374 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd2e01ec7b0 con 0x7fd2e014f360 2024-03-06T07:25:08.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.473+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2e004f330 con 0x7fd2e014f360 2024-03-06T07:25:08.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.473+0000 7fd2dd7fa640 1 -- 172.21.15.26:0/417277374 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4989 (secure 0 0 0) 0x7fd2d8058670 con 0x7fd2e014f360 2024-03-06T07:25:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:08.476 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/317629496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":317629496},{"type":"v1","addr":"172.21.15.26:6834","nonce":317629496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:25:03.710057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8082":{"gid":8082,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8082,"qdb_cluster":[8082]},"id":9}]} 2024-03-06T07:25:08.477 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 115 2024-03-06T07:25:08.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd2c406b4d0 msgr2=0x7fd2c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd2c406b4d0 0x7fd2c406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4036000 tx=0x7fd2d4008030 comp rx=0 tx=0).stop 2024-03-06T07:25:08.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e014f360 msgr2=0x7fd2e01ee1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e014f360 0x7fd2e01ee1a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c7d0 tx=0x7fd2d800cc90 comp rx=0 tx=0).stop 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 shutdown_connections 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd2c406b4d0 0x7fd2c406d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e014f360 0x7fd2e01ee1a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2e00c88c0 0x7fd2e014bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 --2- 172.21.15.26:0/417277374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2e00c5880 0x7fd2e014b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 >> 172.21.15.26:0/417277374 conn(0x7fd2e00ac570 msgr2=0x7fd2e00c9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.475+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 shutdown_connections 2024-03-06T07:25:08.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.476+0000 7fd2e720f640 1 -- 172.21.15.26:0/417277374 wait complete. 2024-03-06T07:25:08.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.592+0000 7fab9e042640 1 Processor -- start 2024-03-06T07:25:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.592+0000 7fab9e042640 1 -- start start 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab9e042640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab9806d9f0 0x7fab980a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab9e042640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab980c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab9e042640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab981551e0 0x7fab98155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab9e042640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab98158800 con 0x7fab9806d9f0 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab9e042640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab98158ae0 con 0x7fab981551e0 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab9e042640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab98158dc0 con 0x7fab980c3280 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab9806d9f0 0x7fab980a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab9806d9f0 0x7fab980a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52812/0 (socket says 172.21.15.26:52812) 2024-03-06T07:25:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 -- 172.21.15.26:0/2318583722 learned_addr learned my addr 172.21.15.26:0/2318583722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab96ffd640 1 --2- 172.21.15.26:0/2318583722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab980c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab97fff640 1 --2- 172.21.15.26:0/2318583722 >> v2:172.21.15.117:3300/0 conn(0x7fab981551e0 0x7fab98155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 -- 172.21.15.26:0/2318583722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 msgr2=0x7fab980c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 --2- 172.21.15.26:0/2318583722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab980c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 -- 172.21.15.26:0/2318583722 >> v2:172.21.15.117:3300/0 conn(0x7fab981551e0 msgr2=0x7fab98155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 --2- 172.21.15.26:0/2318583722 >> v2:172.21.15.117:3300/0 conn(0x7fab981551e0 0x7fab98155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab977fe640 1 -- 172.21.15.26:0/2318583722 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab981590a0 con 0x7fab9806d9f0 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.593+0000 7fab97fff640 1 --2- 172.21.15.26:0/2318583722 >> v2:172.21.15.117:3300/0 conn(0x7fab981551e0 0x7fab98155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.594+0000 7fab977fe640 1 --2- 172.21.15.26:0/2318583722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab980a7cd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fab980c3130 tx=0x7fab8802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faef6f050f0979fa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.594+0000 7fab967fc640 1 -- 172.21.15.26:0/2318583722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8802e070 con 0x7fab9806d9f0 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.594+0000 7fab967fc640 1 -- 172.21.15.26:0/2318583722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab88037070 con 0x7fab9806d9f0 2024-03-06T07:25:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.594+0000 7fab967fc640 1 -- 172.21.15.26:0/2318583722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88031a00 con 0x7fab9806d9f0 2024-03-06T07:25:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 -- 172.21.15.26:0/2318583722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 msgr2=0x7fab980a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 --2- 172.21.15.26:0/2318583722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab980a7cd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fab980c3130 tx=0x7fab8802b880 comp rx=0 tx=0).stop 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 -- 172.21.15.26:0/2318583722 shutdown_connections 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 --2- 172.21.15.26:0/2318583722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab980c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 --2- 172.21.15.26:0/2318583722 >> v2:172.21.15.117:3300/0 conn(0x7fab981551e0 0x7fab98155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 --2- 172.21.15.26:0/2318583722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab980a7cd0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 -- 172.21.15.26:0/2318583722 >> 172.21.15.26:0/2318583722 conn(0x7fab980aa630 msgr2=0x7fab980aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 -- 172.21.15.26:0/2318583722 shutdown_connections 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.595+0000 7fab9e042640 1 -- 172.21.15.26:0/2318583722 wait complete. 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.596+0000 7fab9e042640 1 Processor -- start 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.596+0000 7fab9e042640 1 -- start start 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab9e042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab98157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab9e042640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab9806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab9e042640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab981551e0 0x7fab9806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab9e042640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9806df50 con 0x7fab9806d9f0 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab9e042640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9806ddd0 con 0x7fab981551e0 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab9e042640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab980a8210 con 0x7fab980c3280 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab98157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab98157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52820/0 (socket says 172.21.15.26:52820) 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.597+0000 7fab977fe640 1 -- 172.21.15.26:0/1760987358 learned_addr learned my addr 172.21.15.26:0/1760987358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab977fe640 1 -- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 msgr2=0x7fab9806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab96ffd640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab9806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab97fff640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab981551e0 0x7fab9806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab977fe640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab9806e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab977fe640 1 -- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab981551e0 msgr2=0x7fab9806eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab977fe640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab981551e0 0x7fab9806eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab977fe640 1 -- 172.21.15.26:0/1760987358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab981ebac0 con 0x7fab9806d9f0 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab97fff640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab981551e0 0x7fab9806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab977fe640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab98157220 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fab8802b8c0 tx=0x7fab880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab8ffff640 1 -- 172.21.15.26:0/1760987358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88032670 con 0x7fab9806d9f0 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab8ffff640 1 -- 172.21.15.26:0/1760987358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab88032830 con 0x7fab9806d9f0 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9806f580 con 0x7fab9806d9f0 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab8ffff640 1 -- 172.21.15.26:0/1760987358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88032c20 con 0x7fab9806d9f0 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.598+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9806fb40 con 0x7fab9806d9f0 2024-03-06T07:25:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.599+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9804f330 con 0x7fab9806d9f0 2024-03-06T07:25:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.600+0000 7fab8ffff640 1 -- 172.21.15.26:0/1760987358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab8804c060 con 0x7fab9806d9f0 2024-03-06T07:25:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.600+0000 7fab8ffff640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b4d0 0x7fab7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.600+0000 7fab8ffff640 1 -- 172.21.15.26:0/1760987358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fab880abe00 con 0x7fab9806d9f0 2024-03-06T07:25:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.600+0000 7fab96ffd640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b4d0 0x7fab7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.601+0000 7fab96ffd640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b4d0 0x7fab7806d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fab98154dc0 tx=0x7fab80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.604+0000 7fab8ffff640 1 -- 172.21.15.26:0/1760987358 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab88040070 con 0x7fab9806d9f0 2024-03-06T07:25:08.694 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-03-06T07:25:08.692+0000 7f870b92f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:08.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.878+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab98073240 con 0x7fab9806d9f0 2024-03-06T07:25:08.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.878+0000 7fab8ffff640 1 -- 172.21.15.26:0/1760987358 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4989 (secure 0 0 0) 0x7fab8807aad0 con 0x7fab9806d9f0 2024-03-06T07:25:08.881 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:08.881 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/317629496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":317629496},{"type":"v1","addr":"172.21.15.26:6834","nonce":317629496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:43.287707+0000","modified":"2024-03-06T07:25:03.710057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8082":{"gid":8082,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6835/1367162338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1367162338},{"type":"v1","addr":"172.21.15.26:6835","nonce":1367162338}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8082,"qdb_cluster":[8082]},"id":9}]} 2024-03-06T07:25:08.881 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 115 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b4d0 msgr2=0x7fab7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b4d0 0x7fab7806d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fab98154dc0 tx=0x7fab80008040 comp rx=0 tx=0).stop 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 msgr2=0x7fab98157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab98157220 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fab8802b8c0 tx=0x7fab880046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 shutdown_connections 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b4d0 0x7fab7806d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab981551e0 0x7fab9806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab980c3280 0x7fab9806e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 --2- 172.21.15.26:0/1760987358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab9806d9f0 0x7fab98157220 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 >> 172.21.15.26:0/1760987358 conn(0x7fab980aa630 msgr2=0x7fab9806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 shutdown_connections 2024-03-06T07:25:08.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:08.881+0000 7fab9e042640 1 -- 172.21.15.26:0/1760987358 wait complete. 2024-03-06T07:25:08.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.011+0000 7f56ecb72640 1 Processor -- start 2024-03-06T07:25:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.011+0000 7f56ecb72640 1 -- start start 2024-03-06T07:25:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.012+0000 7f56ecb72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56e8155c80 0x7f56e806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.012+0000 7f56ecb72640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e80720a0 0x7f56e806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56ecb72640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80c1ae0 con 0x7f56e8152520 2024-03-06T07:25:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56ecb72640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e80c1dc0 con 0x7f56e8155c80 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56ecb72640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e806e450 con 0x7f56e80720a0 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e5d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56e8155c80 0x7f56e806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e5d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56e8155c80 0x7f56e806ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52828/0 (socket says 172.21.15.26:52828) 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e5d9b640 1 -- 172.21.15.26:0/4293519411 learned_addr learned my addr 172.21.15.26:0/4293519411 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e6d9d640 1 --2- 172.21.15.26:0/4293519411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e80720a0 0x7f56e806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e5d9b640 1 -- 172.21.15.26:0/4293519411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e80720a0 msgr2=0x7f56e806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e5d9b640 1 --2- 172.21.15.26:0/4293519411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e80720a0 0x7f56e806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e5d9b640 1 -- 172.21.15.26:0/4293519411 >> v1:172.21.15.117:6789/0 conn(0x7f56e8152520 legacy=0x7f56e80c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.013+0000 7f56e5d9b640 1 -- 172.21.15.26:0/4293519411 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e806e730 con 0x7f56e8155c80 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.014+0000 7f56e5d9b640 1 --2- 172.21.15.26:0/4293519411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8155c80 0x7f56e806ef30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009870 tx=0x7f56d002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43148d95e865b91e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.014+0000 7f56e559a640 1 -- 172.21.15.26:0/4293519411 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0034070 con 0x7f56e8155c80 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.014+0000 7f56e559a640 1 -- 172.21.15.26:0/4293519411 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d0030840 con 0x7f56e8155c80 2024-03-06T07:25:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.014+0000 7f56e559a640 1 -- 172.21.15.26:0/4293519411 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0030c10 con 0x7f56e8155c80 2024-03-06T07:25:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.014+0000 7f56ecb72640 1 -- 172.21.15.26:0/4293519411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8155c80 msgr2=0x7f56e806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.014+0000 7f56ecb72640 1 --2- 172.21.15.26:0/4293519411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8155c80 0x7f56e806ef30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009870 tx=0x7f56d002b980 comp rx=0 tx=0).stop 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.015+0000 7f56ecb72640 1 -- 172.21.15.26:0/4293519411 shutdown_connections 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.015+0000 7f56ecb72640 1 --2- 172.21.15.26:0/4293519411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e80720a0 0x7f56e806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.015+0000 7f56ecb72640 1 --2- 172.21.15.26:0/4293519411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8155c80 0x7f56e806ef30 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.015+0000 7f56ecb72640 1 -- 172.21.15.26:0/4293519411 >> 172.21.15.26:0/4293519411 conn(0x7f56e80c76b0 msgr2=0x7f56e80c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.015+0000 7f56ecb72640 1 -- 172.21.15.26:0/4293519411 shutdown_connections 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.015+0000 7f56ecb72640 1 -- 172.21.15.26:0/4293519411 wait complete. 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.016+0000 7f56ecb72640 1 Processor -- start 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.016+0000 7f56ecb72640 1 -- start start 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.016+0000 7f56ecb72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56e80720a0 0x7f56e81e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.016+0000 7f56ecb72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8152520 0x7f56e8157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56ecb72640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e8155c80 0x7f56e8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56ecb72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e8152a80 con 0x7f56e8152520 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56ecb72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e8152900 con 0x7f56e80720a0 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56ecb72640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e806b500 con 0x7f56e8155c80 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8152520 0x7f56e8157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8152520 0x7f56e8157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52840/0 (socket says 172.21.15.26:52840) 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 -- 172.21.15.26:0/3972239502 learned_addr learned my addr 172.21.15.26:0/3972239502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e659c640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56e80720a0 0x7f56e81e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e6d9d640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e8155c80 0x7f56e8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 -- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e8155c80 msgr2=0x7f56e8157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e8155c80 0x7f56e8157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 -- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56e80720a0 msgr2=0x7f56e81e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56e80720a0 0x7f56e81e1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 -- 172.21.15.26:0/3972239502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e8155290 con 0x7f56e8152520 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e6d9d640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e8155c80 0x7f56e8157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56e5d9b640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8152520 0x7f56e8157780 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56deffd640 1 -- 172.21.15.26:0/3972239502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0048070 con 0x7f56e8152520 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56deffd640 1 -- 172.21.15.26:0/3972239502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d0030640 con 0x7f56e8152520 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e81eeb30 con 0x7f56e8152520 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56deffd640 1 -- 172.21.15.26:0/3972239502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0034040 con 0x7f56e8152520 2024-03-06T07:25:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.017+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e8156840 con 0x7f56e8152520 2024-03-06T07:25:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.019+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e8156fa0 con 0x7f56e8152520 2024-03-06T07:25:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.019+0000 7f56deffd640 1 -- 172.21.15.26:0/3972239502 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56d004b070 con 0x7f56e8152520 2024-03-06T07:25:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.019+0000 7f56deffd640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56c806b4d0 0x7f56c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.020+0000 7f56deffd640 1 -- 172.21.15.26:0/3972239502 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f56d00ac2b0 con 0x7f56e8152520 2024-03-06T07:25:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.020+0000 7f56e659c640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56c806b4d0 0x7f56c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.020+0000 7f56e659c640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56c806b4d0 0x7f56c806d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56e0001780 tx=0x7f56e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.024+0000 7f56deffd640 1 -- 172.21.15.26:0/3972239502 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f56e8155290 con 0x7f56e8152520 2024-03-06T07:25:09.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.206+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56e8152c00 con 0x7f56e8152520 2024-03-06T07:25:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.207+0000 7f56deffd640 1 -- 172.21.15.26:0/3972239502 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17169 (secure 0 0 0) 0x7f56d007a800 con 0x7f56e8152520 2024-03-06T07:25:09.210 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:09.210 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":123,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:00.699519+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:41.558889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:42.251998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6832/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6835/577277314":"2024-03-07T07:24:57.677470+0000","172.21.15.26:6833/577277314":"2024-03-07T07:24:57.677470+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56c806b4d0 msgr2=0x7f56c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56c806b4d0 0x7f56c806d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56e0001780 tx=0x7f56e0009040 comp rx=0 tx=0).stop 2024-03-06T07:25:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8152520 msgr2=0x7f56e8157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8152520 0x7f56e8157780 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d0004730 comp rx=0 tx=0).stop 2024-03-06T07:25:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 shutdown_connections 2024-03-06T07:25:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56c806b4d0 0x7f56c806d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56e8155c80 0x7f56e8157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e8152520 0x7f56e8157780 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 --2- 172.21.15.26:0/3972239502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56e80720a0 0x7f56e81e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 >> 172.21.15.26:0/3972239502 conn(0x7f56e80c76b0 msgr2=0x7f56e81de640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 shutdown_connections 2024-03-06T07:25:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.209+0000 7f56ecb72640 1 -- 172.21.15.26:0/3972239502 wait complete. 2024-03-06T07:25:09.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.324+0000 7f583dc78640 1 Processor -- start 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.324+0000 7f583dc78640 1 -- start start 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f583dc78640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58380c69f0 0x7f58380c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f583dc78640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f583dc78640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5838155130 con 0x7f58380c8950 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f583dc78640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5838155440 con 0x7f58380c69f0 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f583dc78640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58380ad480 con 0x7f58381558d0 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f58377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58380c69f0 0x7f58380c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f58377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58380c69f0 0x7f58380c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52846/0 (socket says 172.21.15.26:52846) 2024-03-06T07:25:09.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f58377fe640 1 -- 172.21.15.26:0/3660764584 learned_addr learned my addr 172.21.15.26:0/3660764584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:09.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f5837fff640 1 --2- 172.21.15.26:0/3660764584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f58377fe640 1 -- 172.21.15.26:0/3660764584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 msgr2=0x7f5838155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:09.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f58377fe640 1 --2- 172.21.15.26:0/3660764584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f58377fe640 1 -- 172.21.15.26:0/3660764584 >> v1:172.21.15.117:6789/0 conn(0x7f58380c8950 legacy=0x7f58380acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.325+0000 7f58377fe640 1 -- 172.21.15.26:0/3660764584 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380c5d50 con 0x7f58380c69f0 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.326+0000 7f58377fe640 1 --2- 172.21.15.26:0/3660764584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c69f0 0x7f58380c6dd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5828009870 tx=0x7f582802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59caa75b283995a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.326+0000 7f5836ffd640 1 -- 172.21.15.26:0/3660764584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828035020 con 0x7f58380c69f0 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.326+0000 7f5836ffd640 1 -- 172.21.15.26:0/3660764584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828031640 con 0x7f58380c69f0 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.326+0000 7f5836ffd640 1 -- 172.21.15.26:0/3660764584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582802d040 con 0x7f58380c69f0 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.326+0000 7f583dc78640 1 -- 172.21.15.26:0/3660764584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c69f0 msgr2=0x7f58380c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.326+0000 7f583dc78640 1 --2- 172.21.15.26:0/3660764584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c69f0 0x7f58380c6dd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5828009870 tx=0x7f582802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.327+0000 7f583dc78640 1 -- 172.21.15.26:0/3660764584 shutdown_connections 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.327+0000 7f583dc78640 1 --2- 172.21.15.26:0/3660764584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.327+0000 7f583dc78640 1 --2- 172.21.15.26:0/3660764584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c69f0 0x7f58380c6dd0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.327+0000 7f583dc78640 1 -- 172.21.15.26:0/3660764584 >> 172.21.15.26:0/3660764584 conn(0x7f58380cae70 msgr2=0x7f58380cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.327+0000 7f583dc78640 1 -- 172.21.15.26:0/3660764584 shutdown_connections 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.327+0000 7f583dc78640 1 -- 172.21.15.26:0/3660764584 wait complete. 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 Processor -- start 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 -- start start 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58380c69f0 0x7f58380c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c8950 0x7f58380c2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838152a20 con 0x7f58380c8950 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c6310 con 0x7f58380c69f0 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f583dc78640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5838152ba0 con 0x7f58381558d0 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.328+0000 7f582ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c8950 0x7f58380c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c8950 0x7f58380c2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52850/0 (socket says 172.21.15.26:52850) 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 -- 172.21.15.26:0/1373115967 learned_addr learned my addr 172.21.15.26:0/1373115967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f58377fe640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58380c69f0 0x7f58380c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f5837fff640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 -- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 msgr2=0x7f5838157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 -- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58380c69f0 msgr2=0x7f58380c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58380c69f0 0x7f58380c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 -- 172.21.15.26:0/1373115967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380c1b30 con 0x7f58380c8950 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f582ffff640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c8950 0x7f58380c2010 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f582000c430 tx=0x7f582000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f5834ff9640 1 -- 172.21.15.26:0/1373115967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200052d0 con 0x7f58380c8950 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f5834ff9640 1 -- 172.21.15.26:0/1373115967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820005490 con 0x7f58380c8950 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58380c2940 con 0x7f58380c8950 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f5834ff9640 1 -- 172.21.15.26:0/1373115967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820007260 con 0x7f58380c8950 2024-03-06T07:25:09.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.329+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58381581f0 con 0x7f58380c8950 2024-03-06T07:25:09.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.330+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58380c1d00 con 0x7f58380c8950 2024-03-06T07:25:09.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.331+0000 7f5834ff9640 1 -- 172.21.15.26:0/1373115967 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5820007400 con 0x7f58380c8950 2024-03-06T07:25:09.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.331+0000 7f5834ff9640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f581806b500 0x7f581806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.331+0000 7f5834ff9640 1 -- 172.21.15.26:0/1373115967 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f58200897f0 con 0x7f58380c8950 2024-03-06T07:25:09.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.331+0000 7f58377fe640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f581806b500 0x7f581806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.332+0000 7f58377fe640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f581806b500 0x7f581806d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5828036000 tx=0x7f582803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.335+0000 7f5834ff9640 1 -- 172.21.15.26:0/1373115967 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5838155130 con 0x7f58380c8950 2024-03-06T07:25:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.556+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f583804f330 con 0x7f58380c8950 2024-03-06T07:25:09.558 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:25:09.556+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:25:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.558+0000 7f5834ff9640 1 -- 172.21.15.26:0/1373115967 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v116) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5820057e70 con 0x7f58380c8950 2024-03-06T07:25:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f581806b500 msgr2=0x7f581806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f581806b500 0x7f581806d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5828036000 tx=0x7f582803c040 comp rx=0 tx=0).stop 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c8950 msgr2=0x7f58380c2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c8950 0x7f58380c2010 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f582000c430 tx=0x7f582000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 shutdown_connections 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f581806b500 0x7f581806d9c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58381558d0 0x7f5838157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58380c8950 0x7f58380c2010 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 --2- 172.21.15.26:0/1373115967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58380c69f0 0x7f58380c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 >> 172.21.15.26:0/1373115967 conn(0x7f58380cae70 msgr2=0x7f58380cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 shutdown_connections 2024-03-06T07:25:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.560+0000 7f583dc78640 1 -- 172.21.15.26:0/1373115967 wait complete. 2024-03-06T07:25:09.564 INFO:tasks.ceph.mds.c.smithi026.stderr: -40> 2024-03-06T07:25:08.692+0000 7f870b92f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:09.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:25:09.587 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:25:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 Processor -- start 2024-03-06T07:25:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 -- start start 2024-03-06T07:25:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff9d4152730 0x7ff9d40c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d40c6710 0x7ff9d41554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9d40c3750 0x7ff9d4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d4152b10 con 0x7ff9d40c3750 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d4158ad0 con 0x7ff9d4152730 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.676+0000 7ff9d8e84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d4158db0 con 0x7ff9d40c6710 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9d40c3750 0x7ff9d4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9d40c3750 0x7ff9d4155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52880/0 (socket says 172.21.15.26:52880) 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 -- 172.21.15.26:0/783745015 learned_addr learned my addr 172.21.15.26:0/783745015 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d259c640 1 --2- 172.21.15.26:0/783745015 >> v2:172.21.15.117:3300/0 conn(0x7ff9d4152730 0x7ff9d40c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d1d9b640 1 --2- 172.21.15.26:0/783745015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d40c6710 0x7ff9d41554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 -- 172.21.15.26:0/783745015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d40c6710 msgr2=0x7ff9d41554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 --2- 172.21.15.26:0/783745015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d40c6710 0x7ff9d41554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 -- 172.21.15.26:0/783745015 >> v2:172.21.15.117:3300/0 conn(0x7ff9d4152730 msgr2=0x7ff9d40c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 --2- 172.21.15.26:0/783745015 >> v2:172.21.15.117:3300/0 conn(0x7ff9d4152730 0x7ff9d40c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d2d9d640 1 -- 172.21.15.26:0/783745015 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d4159090 con 0x7ff9d40c3750 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.677+0000 7ff9d259c640 1 --2- 172.21.15.26:0/783745015 >> v2:172.21.15.117:3300/0 conn(0x7ff9d4152730 0x7ff9d40c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d2d9d640 1 --2- 172.21.15.26:0/783745015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d4155a00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4073290 tx=0x7ff9c402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d21d33f5ae61fd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d159a640 1 -- 172.21.15.26:0/783745015 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c402c9e0 con 0x7ff9d40c3750 2024-03-06T07:25:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d159a640 1 -- 172.21.15.26:0/783745015 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c402cb80 con 0x7ff9d40c3750 2024-03-06T07:25:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d159a640 1 -- 172.21.15.26:0/783745015 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40343f0 con 0x7ff9d40c3750 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 -- 172.21.15.26:0/783745015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 msgr2=0x7ff9d4155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/783745015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d4155a00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4073290 tx=0x7ff9c402be40 comp rx=0 tx=0).stop 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 -- 172.21.15.26:0/783745015 shutdown_connections 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/783745015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d40c6710 0x7ff9d41554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/783745015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d4155a00 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/783745015 >> v2:172.21.15.117:3300/0 conn(0x7ff9d4152730 0x7ff9d40c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 -- 172.21.15.26:0/783745015 >> 172.21.15.26:0/783745015 conn(0x7ff9d40aa5c0 msgr2=0x7ff9d40aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.678+0000 7ff9d8e84640 1 -- 172.21.15.26:0/783745015 shutdown_connections 2024-03-06T07:25:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.679+0000 7ff9d8e84640 1 -- 172.21.15.26:0/783745015 wait complete. 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 Processor -- start 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 -- start start 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d41577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d40c6710 0x7ff9d41ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d4152730 0x7ff9d41ee730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d4158870 con 0x7ff9d40c3750 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d41586f0 con 0x7ff9d40c6710 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.680+0000 7ff9d8e84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d4155f40 con 0x7ff9d4152730 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d41577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d41577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52892/0 (socket says 172.21.15.26:52892) 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 -- 172.21.15.26:0/667889292 learned_addr learned my addr 172.21.15.26:0/667889292 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d1d9b640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d40c6710 0x7ff9d41ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d2d9d640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d4152730 0x7ff9d41ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 -- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d4152730 msgr2=0x7ff9d41ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d4152730 0x7ff9d41ee730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 -- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d40c6710 msgr2=0x7ff9d41ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d40c6710 0x7ff9d41ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 -- 172.21.15.26:0/667889292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d4154fb0 con 0x7ff9d40c3750 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d259c640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d41577b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00d7d0 tx=0x7ff9cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9caffd640 1 -- 172.21.15.26:0/667889292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc004030 con 0x7ff9d40c3750 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9caffd640 1 -- 172.21.15.26:0/667889292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0041d0 con 0x7ff9d40c3750 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d4154530 con 0x7ff9d40c3750 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9caffd640 1 -- 172.21.15.26:0/667889292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00b6c0 con 0x7ff9d40c3750 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.681+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d4070480 con 0x7ff9d40c3750 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.682+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d404f330 con 0x7ff9d40c3750 2024-03-06T07:25:09.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.683+0000 7ff9caffd640 1 -- 172.21.15.26:0/667889292 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9cc00b860 con 0x7ff9d40c3750 2024-03-06T07:25:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.683+0000 7ff9caffd640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9b406b4d0 0x7ff9b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.683+0000 7ff9caffd640 1 -- 172.21.15.26:0/667889292 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6359+0+0 (secure 0 0 0) 0x7ff9cc08a580 con 0x7ff9d40c3750 2024-03-06T07:25:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.683+0000 7ff9d1d9b640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9b406b4d0 0x7ff9b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.684+0000 7ff9d1d9b640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9b406b4d0 0x7ff9b406d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40c8430 tx=0x7ff9bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.687+0000 7ff9caffd640 1 -- 172.21.15.26:0/667889292 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff9cc059200 con 0x7ff9d40c3750 2024-03-06T07:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:09.914+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff9d414c460 con 0x7ff9d40c3750 2024-03-06T07:25:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.559+0000 7ff9caffd640 1 -- 172.21.15.26:0/667889292 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v117) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff9cc058b00 con 0x7ff9d40c3750 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9b406b4d0 msgr2=0x7ff9b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9b406b4d0 0x7ff9b406d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40c8430 tx=0x7ff9bc008040 comp rx=0 tx=0).stop 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 msgr2=0x7ff9d41577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d41577b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00d7d0 tx=0x7ff9cc00dc90 comp rx=0 tx=0).stop 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 shutdown_connections 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9b406b4d0 0x7ff9b406d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9d4152730 0x7ff9d41ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d40c6710 0x7ff9d41ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 --2- 172.21.15.26:0/667889292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d40c3750 0x7ff9d41577b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 >> 172.21.15.26:0/667889292 conn(0x7ff9d40aa5c0 msgr2=0x7ff9d41554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 shutdown_connections 2024-03-06T07:25:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.562+0000 7ff9d8e84640 1 -- 172.21.15.26:0/667889292 wait complete. 2024-03-06T07:25:10.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:25:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.729+0000 7f16b9b9a640 1 Processor -- start 2024-03-06T07:25:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.729+0000 7f16b9b9a640 1 -- start start 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b9b9a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f16b40c5e40 0x7f16b4153940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b9b9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16b40c46d0 0x7f16b40c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b9b9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b4153e80 con 0x7f16b4072fc0 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b9b9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b40c6220 con 0x7f16b40c46d0 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b9b9a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b4070210 con 0x7f16b40c5e40 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16b40c46d0 0x7f16b40c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b2ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f16b4072fc0 0x7f16b40724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47588/0 (socket says 172.21.15.26:47588) 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b2ffd640 1 -- 172.21.15.26:0/2988243504 learned_addr learned my addr 172.21.15.26:0/2988243504 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b37fe640 1 --2- 172.21.15.26:0/2988243504 >> v2:172.21.15.117:3300/0 conn(0x7f16b40c5e40 0x7f16b4153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b3fff640 1 -- 172.21.15.26:0/2988243504 >> v2:172.21.15.117:3300/0 conn(0x7f16b40c5e40 msgr2=0x7f16b4153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b3fff640 1 --2- 172.21.15.26:0/2988243504 >> v2:172.21.15.117:3300/0 conn(0x7f16b40c5e40 0x7f16b4153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b3fff640 1 -- 172.21.15.26:0/2988243504 >> v1:172.21.15.26:6789/0 conn(0x7f16b4072fc0 legacy=0x7f16b40724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b3fff640 1 -- 172.21.15.26:0/2988243504 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40704d0 con 0x7f16b40c46d0 2024-03-06T07:25:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.730+0000 7f16b27fc640 1 -- 172.21.15.26:0/2988243504 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474775241 0 0) 0x7f16b4153e80 con 0x7f16b4072fc0 2024-03-06T07:25:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.731+0000 7f16b3fff640 1 --2- 172.21.15.26:0/2988243504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b40c4b20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f16b4071220 tx=0x7f16ac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71ab3d3f7c02fe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.731+0000 7f16b27fc640 1 -- 172.21.15.26:0/2988243504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac039070 con 0x7f16b40c46d0 2024-03-06T07:25:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.731+0000 7f16b27fc640 1 -- 172.21.15.26:0/2988243504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac02cbf0 con 0x7f16b40c46d0 2024-03-06T07:25:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.731+0000 7f16b27fc640 1 -- 172.21.15.26:0/2988243504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac034440 con 0x7f16b40c46d0 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2988243504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 msgr2=0x7f16b40c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2988243504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b40c4b20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f16b4071220 tx=0x7f16ac02be40 comp rx=0 tx=0).stop 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2988243504 shutdown_connections 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2988243504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b40c4b20 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2988243504 >> v2:172.21.15.117:3300/0 conn(0x7f16b40c5e40 0x7f16b4153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2988243504 >> 172.21.15.26:0/2988243504 conn(0x7f16b40c88f0 msgr2=0x7f16b40c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2988243504 shutdown_connections 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.732+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2988243504 wait complete. 2024-03-06T07:25:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.733+0000 7f16b9b9a640 1 Processor -- start 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.733+0000 7f16b9b9a640 1 -- start start 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b9b9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b4072fc0 0x7f16b414ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b9b9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b4148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b9b9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b40c5e40 0x7f16b4149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b9b9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b406fbd0 con 0x7f16b40c46d0 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b9b9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b406f8f0 con 0x7f16b40c5e40 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b9b9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b4072be0 con 0x7f16b4072fc0 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b4148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b4148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52910/0 (socket says 172.21.15.26:52910) 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 -- 172.21.15.26:0/2369754926 learned_addr learned my addr 172.21.15.26:0/2369754926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b37fe640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b4072fc0 0x7f16b414ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b3fff640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b40c5e40 0x7f16b4149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 -- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b4072fc0 msgr2=0x7f16b414ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b4072fc0 0x7f16b414ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 -- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b40c5e40 msgr2=0x7f16b4149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b40c5e40 0x7f16b4149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b2ffd640 1 -- 172.21.15.26:0/2369754926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40ae760 con 0x7f16b40c46d0 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.734+0000 7f16b3fff640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b40c5e40 0x7f16b4149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.735+0000 7f16b2ffd640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b4148d40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f16a400c3e0 tx=0x7f16a400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.735+0000 7f16abfff640 1 -- 172.21.15.26:0/2369754926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4017070 con 0x7f16b40c46d0 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.735+0000 7f16abfff640 1 -- 172.21.15.26:0/2369754926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a4005ce0 con 0x7f16b40c46d0 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.735+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b40ab6d0 con 0x7f16b40c46d0 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.735+0000 7f16abfff640 1 -- 172.21.15.26:0/2369754926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40072f0 con 0x7f16b40c46d0 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.735+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b40ab3f0 con 0x7f16b40c46d0 2024-03-06T07:25:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.736+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b40c86a0 con 0x7f16b40c46d0 2024-03-06T07:25:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.736+0000 7f16abfff640 1 -- 172.21.15.26:0/2369754926 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16a401a070 con 0x7f16b40c46d0 2024-03-06T07:25:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.737+0000 7f16abfff640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f169406b4d0 0x7f169406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.737+0000 7f16abfff640 1 -- 172.21.15.26:0/2369754926 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f16a4089730 con 0x7f16b40c46d0 2024-03-06T07:25:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.737+0000 7f16b37fe640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f169406b4d0 0x7f169406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.737+0000 7f16b37fe640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f169406b4d0 0x7f169406d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f16b40c81e0 tx=0x7f169c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.741+0000 7f16abfff640 1 -- 172.21.15.26:0/2369754926 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f16b40c86a0 con 0x7f16b40c46d0 2024-03-06T07:25:10.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:10.920+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16b404f330 con 0x7f16b40c46d0 2024-03-06T07:25:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.731+0000 7f16abfff640 1 -- 172.21.15.26:0/2369754926 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7f16b40ae760 con 0x7f16b40c46d0 2024-03-06T07:25:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:25:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f169406b4d0 msgr2=0x7f169406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f169406b4d0 0x7f169406d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f16b40c81e0 tx=0x7f169c009190 comp rx=0 tx=0).stop 2024-03-06T07:25:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 msgr2=0x7f16b4148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b4148d40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f16a400c3e0 tx=0x7f16a400c8a0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 shutdown_connections 2024-03-06T07:25:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f169406b4d0 0x7f169406d990 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b40c5e40 0x7f16b4149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16b40c46d0 0x7f16b4148d40 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 --2- 172.21.15.26:0/2369754926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b4072fc0 0x7f16b414ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.735+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 >> 172.21.15.26:0/2369754926 conn(0x7f16b40c88f0 msgr2=0x7f16b40c4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.736+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 shutdown_connections 2024-03-06T07:25:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.736+0000 7f16b9b9a640 1 -- 172.21.15.26:0/2369754926 wait complete. 2024-03-06T07:25:11.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:25:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88e513640 1 Processor -- start 2024-03-06T07:25:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88e513640 1 -- start start 2024-03-06T07:25:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88e513640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8880c3c20 0x7fa8880c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88e513640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa88806f8d0 0x7fa888153910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88e513640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa888152e90 con 0x7fa888156480 2024-03-06T07:25:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88e513640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa888153480 con 0x7fa88806f8d0 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88e513640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880c6f10 con 0x7fa8880c3c20 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa8877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa88806f8d0 0x7fa888153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.861+0000 7fa88cab0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa888156480 0x7fa88806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47604/0 (socket says 172.21.15.26:47604) 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.862+0000 7fa88cab0640 1 -- 172.21.15.26:0/2242005556 learned_addr learned my addr 172.21.15.26:0/2242005556 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.862+0000 7fa887fff640 1 --2- 172.21.15.26:0/2242005556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8880c3c20 0x7fa8880c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.862+0000 7fa8877fe640 1 -- 172.21.15.26:0/2242005556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8880c3c20 msgr2=0x7fa8880c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.862+0000 7fa8877fe640 1 --2- 172.21.15.26:0/2242005556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8880c3c20 0x7fa8880c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.862+0000 7fa8877fe640 1 -- 172.21.15.26:0/2242005556 >> v1:172.21.15.26:6789/0 conn(0x7fa888156480 legacy=0x7fa88806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.862+0000 7fa886ffd640 1 -- 172.21.15.26:0/2242005556 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641142215 0 0) 0x7fa880001490 con 0x7fa888156480 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.862+0000 7fa8877fe640 1 -- 172.21.15.26:0/2242005556 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880c71f0 con 0x7fa88806f8d0 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa8877fe640 1 --2- 172.21.15.26:0/2242005556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa88806f8d0 0x7fa888153910 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa888153600 tx=0x7fa87c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614c8a42cae1aac5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa886ffd640 1 -- 172.21.15.26:0/2242005556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c035020 con 0x7fa88806f8d0 2024-03-06T07:25:11.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa886ffd640 1 -- 172.21.15.26:0/2242005556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c031800 con 0x7fa88806f8d0 2024-03-06T07:25:11.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa886ffd640 1 -- 172.21.15.26:0/2242005556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c031bd0 con 0x7fa88806f8d0 2024-03-06T07:25:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 -- 172.21.15.26:0/2242005556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa88806f8d0 msgr2=0x7fa888153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 --2- 172.21.15.26:0/2242005556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa88806f8d0 0x7fa888153910 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa888153600 tx=0x7fa87c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 -- 172.21.15.26:0/2242005556 shutdown_connections 2024-03-06T07:25:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 --2- 172.21.15.26:0/2242005556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa88806f8d0 0x7fa888153910 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 --2- 172.21.15.26:0/2242005556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8880c3c20 0x7fa8880c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 -- 172.21.15.26:0/2242005556 >> 172.21.15.26:0/2242005556 conn(0x7fa8880c4300 msgr2=0x7fa8880c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 -- 172.21.15.26:0/2242005556 shutdown_connections 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.863+0000 7fa88e513640 1 -- 172.21.15.26:0/2242005556 wait complete. 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.864+0000 7fa88e513640 1 Processor -- start 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa88e513640 1 -- start start 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa88e513640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa88806f8d0 0x7fa8880a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa88e513640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8880c3c20 0x7fa8881eb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa88e513640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa888156480 0x7fa888154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa88e513640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88806b6d0 con 0x7fa8880c3c20 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa88e513640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88806fd10 con 0x7fa888156480 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa88e513640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88806b850 con 0x7fa88806f8d0 2024-03-06T07:25:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8880c3c20 0x7fa8881eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8880c3c20 0x7fa8881eb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52926/0 (socket says 172.21.15.26:52926) 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.865+0000 7fa8877fe640 1 -- 172.21.15.26:0/2036633539 learned_addr learned my addr 172.21.15.26:0/2036633539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa88cab0640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa888156480 0x7fa888154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa887fff640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa88806f8d0 0x7fa8880a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa8877fe640 1 -- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa88806f8d0 msgr2=0x7fa8880a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa8877fe640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa88806f8d0 0x7fa8880a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa8877fe640 1 -- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa888156480 msgr2=0x7fa888154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa8877fe640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa888156480 0x7fa888154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa8877fe640 1 -- 172.21.15.26:0/2036633539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880a8480 con 0x7fa8880c3c20 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa88cab0640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa888156480 0x7fa888154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa8877fe640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8880c3c20 0x7fa8881eb110 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009890 tx=0x7fa87c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa884ff9640 1 -- 172.21.15.26:0/2036633539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c041070 con 0x7fa8880c3c20 2024-03-06T07:25:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa884ff9640 1 -- 172.21.15.26:0/2036633539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c03ad00 con 0x7fa8880c3c20 2024-03-06T07:25:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8880a8760 con 0x7fa8880c3c20 2024-03-06T07:25:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa884ff9640 1 -- 172.21.15.26:0/2036633539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c035040 con 0x7fa8880c3c20 2024-03-06T07:25:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.866+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8880721d0 con 0x7fa8880c3c20 2024-03-06T07:25:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.867+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88804f330 con 0x7fa8880c3c20 2024-03-06T07:25:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.868+0000 7fa884ff9640 1 -- 172.21.15.26:0/2036633539 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa87c038020 con 0x7fa8880c3c20 2024-03-06T07:25:11.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.868+0000 7fa884ff9640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa87406b4d0 0x7fa87406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.868+0000 7fa884ff9640 1 -- 172.21.15.26:0/2036633539 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5921+0+0 (secure 0 0 0) 0x7fa87c0abef0 con 0x7fa8880c3c20 2024-03-06T07:25:11.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.868+0000 7fa887fff640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa87406b4d0 0x7fa87406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.869+0000 7fa887fff640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa87406b4d0 0x7fa87406d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa8880c1cd0 tx=0x7fa878008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:11.873+0000 7fa884ff9640 1 -- 172.21.15.26:0/2036633539 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa87c07ac90 con 0x7fa8880c3c20 2024-03-06T07:25:12.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.059+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa888150440 con 0x7fa8880c3c20 2024-03-06T07:25:12.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:12.169 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.732+0000 7fa884ff9640 1 -- 172.21.15.26:0/2036633539 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa87c07a590 con 0x7fa8880c3c20 2024-03-06T07:25:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa87406b4d0 msgr2=0x7fa87406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa87406b4d0 0x7fa87406d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa8880c1cd0 tx=0x7fa878008040 comp rx=0 tx=0).stop 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8880c3c20 msgr2=0x7fa8881eb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8880c3c20 0x7fa8881eb110 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009890 tx=0x7fa87c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 shutdown_connections 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa87406b4d0 0x7fa87406d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa888156480 0x7fa888154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8880c3c20 0x7fa8881eb110 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 --2- 172.21.15.26:0/2036633539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa88806f8d0 0x7fa8880a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 >> 172.21.15.26:0/2036633539 conn(0x7fa8880c4300 msgr2=0x7fa888152750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 shutdown_connections 2024-03-06T07:25:12.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.735+0000 7fa88e513640 1 -- 172.21.15.26:0/2036633539 wait complete. 2024-03-06T07:25:12.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.c mds_join_fs 2024-03-06T07:25:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.853+0000 7f770b37a640 1 Processor -- start 2024-03-06T07:25:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.853+0000 7f770b37a640 1 -- start start 2024-03-06T07:25:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f770b37a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77040c6080 0x7f770406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f770b37a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7704156650 con 0x7f77040c4e60 2024-03-06T07:25:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f770b37a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77041568f0 con 0x7f7704153030 2024-03-06T07:25:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f770b37a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770406ec50 con 0x7f77040c6080 2024-03-06T07:25:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f7708915640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77040c6080 0x7f770406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f7709116640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f77040c4e60 0x7f7704072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47614/0 (socket says 172.21.15.26:47614) 2024-03-06T07:25:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f7709116640 1 -- 172.21.15.26:0/574088500 learned_addr learned my addr 172.21.15.26:0/574088500 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f7708915640 1 -- 172.21.15.26:0/574088500 >> v1:172.21.15.117:6789/0 conn(0x7f7704153030 legacy=0x7f77041534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f7708915640 1 -- 172.21.15.26:0/574088500 >> v1:172.21.15.26:6789/0 conn(0x7f77040c4e60 legacy=0x7f7704072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f7703fff640 1 -- 172.21.15.26:0/574088500 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815996497 0 0) 0x7f7704156650 con 0x7f77040c4e60 2024-03-06T07:25:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.854+0000 7f7708915640 1 -- 172.21.15.26:0/574088500 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704156bd0 con 0x7f77040c6080 2024-03-06T07:25:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.855+0000 7f7708915640 1 --2- 172.21.15.26:0/574088500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77040c6080 0x7f770406f7d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097a0 tx=0x7f76f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b651e35c9e82fe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.855+0000 7f7703fff640 1 -- 172.21.15.26:0/574088500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8035020 con 0x7f77040c6080 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.855+0000 7f7703fff640 1 -- 172.21.15.26:0/574088500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8031830 con 0x7f77040c6080 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.855+0000 7f7703fff640 1 -- 172.21.15.26:0/574088500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8031c00 con 0x7f77040c6080 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.856+0000 7f770b37a640 1 -- 172.21.15.26:0/574088500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77040c6080 msgr2=0x7f770406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.856+0000 7f770b37a640 1 --2- 172.21.15.26:0/574088500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77040c6080 0x7f770406f7d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097a0 tx=0x7f76f802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.856+0000 7f770b37a640 1 -- 172.21.15.26:0/574088500 shutdown_connections 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.856+0000 7f770b37a640 1 --2- 172.21.15.26:0/574088500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77040c6080 0x7f770406f7d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.856+0000 7f770b37a640 1 -- 172.21.15.26:0/574088500 >> 172.21.15.26:0/574088500 conn(0x7f77040c5560 msgr2=0x7f77040c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.856+0000 7f770b37a640 1 -- 172.21.15.26:0/574088500 shutdown_connections 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.856+0000 7f770b37a640 1 -- 172.21.15.26:0/574088500 wait complete. 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.857+0000 7f770b37a640 1 Processor -- start 2024-03-06T07:25:12.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.857+0000 7f770b37a640 1 -- start start 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f770b37a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77040c4e60 0x7f7704150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f770b37a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77040c6080 0x7f77040ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f770b37a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7704153030 0x7f77040c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f770b37a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77040c87d0 con 0x7f7704153030 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f770b37a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704157190 con 0x7f77040c6080 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f770b37a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7704072d90 con 0x7f77040c4e60 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7704153030 0x7f77040c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7704153030 0x7f77040c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52942/0 (socket says 172.21.15.26:52942) 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 -- 172.21.15.26:0/3076246336 learned_addr learned my addr 172.21.15.26:0/3076246336 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7708915640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77040c6080 0x7f77040ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709116640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77040c4e60 0x7f7704150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 -- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77040c4e60 msgr2=0x7f7704150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77040c4e60 0x7f7704150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 -- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77040c6080 msgr2=0x7f77040ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77040c6080 0x7f77040ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 -- 172.21.15.26:0/3076246336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77041dd740 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.858+0000 7f7709917640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7704153030 0x7f77040c2520 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f76f400b9a0 tx=0x7f76f400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.859+0000 7f7701ffb640 1 -- 172.21.15.26:0/3076246336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4019070 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.859+0000 7f7701ffb640 1 -- 172.21.15.26:0/3076246336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f400cba0 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.859+0000 7f7701ffb640 1 -- 172.21.15.26:0/3076246336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4014440 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.859+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770414fe90 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.859+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77040ac550 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.860+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77041dd910 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.860+0000 7f7701ffb640 1 -- 172.21.15.26:0/3076246336 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76f401c070 con 0x7f7704153030 2024-03-06T07:25:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.861+0000 7f7701ffb640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76f006b4d0 0x7f76f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.861+0000 7f7701ffb640 1 -- 172.21.15.26:0/3076246336 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f76f4089cb0 con 0x7f7704153030 2024-03-06T07:25:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.861+0000 7f7709116640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76f006b4d0 0x7f76f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.861+0000 7f7709116640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76f006b4d0 0x7f76f006d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000f60 tx=0x7f76fc007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:12.865+0000 7f7701ffb640 1 -- 172.21.15.26:0/3076246336 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f76f4058630 con 0x7f7704153030 2024-03-06T07:25:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.050+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"} v 0) v1 -- 0x7f770404f330 con 0x7f7704153030 2024-03-06T07:25:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.053+0000 7f7701ffb640 1 -- 172.21.15.26:0/3076246336 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"}]=0 v43) v1 ==== 96+0+0 (secure 0 0 0) 0x7f76f405c6d0 con 0x7f7704153030 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76f006b4d0 msgr2=0x7f76f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76f006b4d0 0x7f76f006d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000f60 tx=0x7f76fc007d50 comp rx=0 tx=0).stop 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7704153030 msgr2=0x7f77040c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7704153030 0x7f77040c2520 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f76f400b9a0 tx=0x7f76f400be60 comp rx=0 tx=0).stop 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 shutdown_connections 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76f006b4d0 0x7f76f006d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7704153030 0x7f77040c2520 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77040c6080 0x7f77040ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 --2- 172.21.15.26:0/3076246336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77040c4e60 0x7f7704150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 >> 172.21.15.26:0/3076246336 conn(0x7f77040c5560 msgr2=0x7f77040c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 shutdown_connections 2024-03-06T07:25:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.055+0000 7f770b37a640 1 -- 172.21.15.26:0/3076246336 wait complete. 2024-03-06T07:25:13.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.172+0000 7fe24b593640 1 Processor -- start 2024-03-06T07:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.172+0000 7fe24b593640 1 -- start start 2024-03-06T07:25:13.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe24b593640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 0x7fe244155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe24b593640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe244152b10 con 0x7fe2440c6710 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe24b593640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe244158ad0 con 0x7fe244152730 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe24b593640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe244156110 con 0x7fe2440a7cd0 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe248b2e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe2440c6710 0x7fe2441554c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47620/0 (socket says 172.21.15.26:47620) 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe248b2e640 1 -- 172.21.15.26:0/2010413831 learned_addr learned my addr 172.21.15.26:0/2010413831 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe249b30640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 0x7fe244155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693058003 0 0) 0x7fe244152b10 con 0x7fe2440c6710 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.173+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c003520 con 0x7fe2440c6710 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1890594652 0 0) 0x7fe22c003520 con 0x7fe2440c6710 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 msgr2=0x7fe244155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 0x7fe244155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 >> v1:172.21.15.117:6789/0 conn(0x7fe244152730 legacy=0x7fe2440c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244158db0 con 0x7fe2440c6710 2024-03-06T07:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe238001670 con 0x7fe2440c6710 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 >> v1:172.21.15.26:6789/0 conn(0x7fe2440c6710 legacy=0x7fe2441554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe22c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22c004390 0x7fe22c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 0x7fe22c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00ba70 con 0x7fe22c0039e0 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00bc20 con 0x7fe22c004390 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00be20 con 0x7fe2440a7cd0 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe24932f640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe22c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe248b2e640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22c004390 0x7fe22c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe249b30640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 0x7fe22c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe24932f640 1 -- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 msgr2=0x7fe22c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe24932f640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2440a7cd0 0x7fe22c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe24932f640 1 -- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22c004390 msgr2=0x7fe22c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe24932f640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22c004390 0x7fe22c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.174+0000 7fe24932f640 1 -- 172.21.15.26:0/2010413831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c003520 con 0x7fe22c0039e0 2024-03-06T07:25:13.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.175+0000 7fe24932f640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe22c003dc0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe23c001860 tx=0x7fe22c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd88f8bbe114268 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.175+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c02f050 con 0x7fe22c0039e0 2024-03-06T07:25:13.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.175+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c033650 con 0x7fe22c0039e0 2024-03-06T07:25:13.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.175+0000 7fe243fff640 1 -- 172.21.15.26:0/2010413831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c001430 con 0x7fe22c0039e0 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.176+0000 7fe24b593640 1 -- 172.21.15.26:0/2010413831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 msgr2=0x7fe22c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.176+0000 7fe24b593640 1 --2- 172.21.15.26:0/2010413831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe22c003dc0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe23c001860 tx=0x7fe22c005830 comp rx=0 tx=0).stop 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.176+0000 7fe249b30640 1 -- 172.21.15.26:0/2010413831 reap_dead start 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.176+0000 7fe24b593640 1 -- 172.21.15.26:0/2010413831 shutdown_connections 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.176+0000 7fe24b593640 1 -- 172.21.15.26:0/2010413831 >> 172.21.15.26:0/2010413831 conn(0x7fe2440aa5c0 msgr2=0x7fe2440aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.176+0000 7fe24b593640 1 -- 172.21.15.26:0/2010413831 shutdown_connections 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.176+0000 7fe24b593640 1 -- 172.21.15.26:0/2010413831 wait complete. 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 Processor -- start 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 -- start start 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe24406cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe244147df0 0x7fe244149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2441eaba0 0x7fe244146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440c3920 con 0x7fe22c0039e0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244159370 con 0x7fe2441eaba0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24b593640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440c3aa0 con 0x7fe244147df0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.177+0000 7fe24932f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe24406cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe24406cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52954/0 (socket says 172.21.15.26:52954) 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 -- 172.21.15.26:0/2017843434 learned_addr learned my addr 172.21.15.26:0/2017843434 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe249b30640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2441eaba0 0x7fe244146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe248b2e640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe244147df0 0x7fe244149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 -- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe244147df0 msgr2=0x7fe244149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe244147df0 0x7fe244149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 -- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2441eaba0 msgr2=0x7fe244146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2441eaba0 0x7fe244146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 -- 172.21.15.26:0/2017843434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24406c480 con 0x7fe22c0039e0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24932f640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe24406cd80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe23c001980 tx=0x7fe23c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe241ffb640 1 -- 172.21.15.26:0/2017843434 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0035f0 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe241ffb640 1 -- 172.21.15.26:0/2017843434 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c003790 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe241ffb640 1 -- 172.21.15.26:0/2017843434 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c003b40 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2441eecc0 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.178+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe244149fe0 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.179+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24404f330 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.180+0000 7fe241ffb640 1 -- 172.21.15.26:0/2017843434 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe23c003ce0 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.180+0000 7fe241ffb640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.180+0000 7fe241ffb640 1 -- 172.21.15.26:0/2017843434 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe23c0ac1f0 con 0x7fe22c0039e0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.180+0000 7fe248b2e640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.181+0000 7fe248b2e640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe23006b4d0 0x7fe23006d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe244154050 tx=0x7fe238009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.184+0000 7fe241ffb640 1 -- 172.21.15.26:0/2017843434 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe23c04a060 con 0x7fe22c0039e0 2024-03-06T07:25:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.363+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe244154ee0 con 0x7fe22c0039e0 2024-03-06T07:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.366+0000 7fe241ffb640 1 -- 172.21.15.26:0/2017843434 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe23c041030 con 0x7fe22c0039e0 2024-03-06T07:25:13.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.368+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe23006b4d0 msgr2=0x7fe23006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.368+0000 7fe24b593640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe23006b4d0 0x7fe23006d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe244154050 tx=0x7fe238009040 comp rx=0 tx=0).stop 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.368+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 msgr2=0x7fe24406cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.368+0000 7fe24b593640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe24406cd80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe23c001980 tx=0x7fe23c001230 comp rx=0 tx=0).stop 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 shutdown_connections 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2441eaba0 0x7fe244146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe244147df0 0x7fe244149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 --2- 172.21.15.26:0/2017843434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c0039e0 0x7fe24406cd80 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 >> 172.21.15.26:0/2017843434 conn(0x7fe2440aa5c0 msgr2=0x7fe244072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 shutdown_connections 2024-03-06T07:25:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.369+0000 7fe24b593640 1 -- 172.21.15.26:0/2017843434 wait complete. 2024-03-06T07:25:13.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady' 2024-03-06T07:25:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 Processor -- start 2024-03-06T07:25:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 -- start start 2024-03-06T07:25:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e98152920 0x7f3e98155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e980c8a00 0x7f3e98158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e98159460 0x7f3e980c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e981587b0 con 0x7f3e98152920 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e98156180 con 0x7f3e98159460 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9f4a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c7760 con 0x7f3e980c8a00 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.481+0000 7f3e9d240640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e98152920 0x7f3e98155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e98152920 0x7f3e98155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52966/0 (socket says 172.21.15.26:52966) 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 -- 172.21.15.26:0/2285214603 learned_addr learned my addr 172.21.15.26:0/2285214603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 -- 172.21.15.26:0/2285214603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e980c8a00 msgr2=0x7f3e98158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 --2- 172.21.15.26:0/2285214603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e980c8a00 0x7f3e98158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 -- 172.21.15.26:0/2285214603 >> v2:172.21.15.117:3300/0 conn(0x7f3e98159460 msgr2=0x7f3e980c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 --2- 172.21.15.26:0/2285214603 >> v2:172.21.15.117:3300/0 conn(0x7f3e98159460 0x7f3e980c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 -- 172.21.15.26:0/2285214603 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980c7a40 con 0x7f3e98152920 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.482+0000 7f3e9d240640 1 --2- 172.21.15.26:0/2285214603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98152920 0x7f3e98155c40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3e9806cab0 tx=0x7f3e9402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49060c6ab41b67c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e8ffff640 1 -- 172.21.15.26:0/2285214603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94035020 con 0x7f3e98152920 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e8ffff640 1 -- 172.21.15.26:0/2285214603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e94031800 con 0x7f3e98152920 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e8ffff640 1 -- 172.21.15.26:0/2285214603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94031bd0 con 0x7f3e98152920 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2285214603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98152920 msgr2=0x7f3e98155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2285214603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98152920 0x7f3e98155c40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3e9806cab0 tx=0x7f3e9402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2285214603 shutdown_connections 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2285214603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e980c8a00 0x7f3e98158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2285214603 >> v2:172.21.15.117:3300/0 conn(0x7f3e98159460 0x7f3e980c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2285214603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98152920 0x7f3e98155c40 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2285214603 >> 172.21.15.26:0/2285214603 conn(0x7f3e980ac560 msgr2=0x7f3e980ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2285214603 shutdown_connections 2024-03-06T07:25:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.483+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2285214603 wait complete. 2024-03-06T07:25:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.484+0000 7f3e9f4a4640 1 Processor -- start 2024-03-06T07:25:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.484+0000 7f3e9f4a4640 1 -- start start 2024-03-06T07:25:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9f4a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e980c8a00 0x7f3e981de820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9f4a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e98152920 0x7f3e98147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9f4a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e98148150 0x7f3e98149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9f4a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9806d030 con 0x7f3e980c8a00 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9f4a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e98158a00 con 0x7f3e98152920 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9f4a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c7390 con 0x7f3e98148150 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e980c8a00 0x7f3e981de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e980c8a00 0x7f3e981de820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52980/0 (socket says 172.21.15.26:52980) 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 -- 172.21.15.26:0/2304145601 learned_addr learned my addr 172.21.15.26:0/2304145601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9da41640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e98148150 0x7f3e98149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9ca3f640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e98152920 0x7f3e98147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 -- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e98148150 msgr2=0x7f3e98149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e98148150 0x7f3e98149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 -- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e98152920 msgr2=0x7f3e98147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e98152920 0x7f3e98147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.485+0000 7f3e9d240640 1 -- 172.21.15.26:0/2304145601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98156650 con 0x7f3e980c8a00 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e9ca3f640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e98152920 0x7f3e98147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e9da41640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e98148150 0x7f3e98149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e9d240640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e980c8a00 0x7f3e981de820 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3e9402b8e0 tx=0x7f3e940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e8dffb640 1 -- 172.21.15.26:0/2304145601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94047070 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e8dffb640 1 -- 172.21.15.26:0/2304145601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9403ad60 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e98147f00 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e8dffb640 1 -- 172.21.15.26:0/2304145601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94042610 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.486+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e98158220 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.487+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9804f330 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.488+0000 7f3e8dffb640 1 -- 172.21.15.26:0/2304145601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e9404a070 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.488+0000 7f3e8dffb640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e7c06b500 0x7f3e7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.488+0000 7f3e8dffb640 1 -- 172.21.15.26:0/2304145601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f3e940abea0 con 0x7f3e980c8a00 2024-03-06T07:25:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.488+0000 7f3e9ca3f640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e7c06b500 0x7f3e7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.489+0000 7f3e9ca3f640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e7c06b500 0x7f3e7c06d9c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3e9814f190 tx=0x7f3e88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.493+0000 7f3e8dffb640 1 -- 172.21.15.26:0/2304145601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3e9403f070 con 0x7f3e980c8a00 2024-03-06T07:25:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.741+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]} v 0) v1 -- 0x7f3e98156c70 con 0x7f3e980c8a00 2024-03-06T07:25:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.744+0000 7f3e8dffb640 1 -- 172.21.15.26:0/2304145601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]}]=0 v304) v1 ==== 145+0+0 (secure 0 0 0) 0x7f3e9407af90 con 0x7f3e980c8a00 2024-03-06T07:25:13.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.746+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e7c06b500 msgr2=0x7f3e7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.746+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e7c06b500 0x7f3e7c06d9c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3e9814f190 tx=0x7f3e88008040 comp rx=0 tx=0).stop 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.746+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e980c8a00 msgr2=0x7f3e981de820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.746+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e980c8a00 0x7f3e981de820 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3e9402b8e0 tx=0x7f3e940046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 shutdown_connections 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e7c06b500 0x7f3e7c06d9c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e98148150 0x7f3e98149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e98152920 0x7f3e98147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 --2- 172.21.15.26:0/2304145601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e980c8a00 0x7f3e981de820 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 >> 172.21.15.26:0/2304145601 conn(0x7f3e980ac560 msgr2=0x7f3e980c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 shutdown_connections 2024-03-06T07:25:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.747+0000 7f3e9f4a4640 1 -- 172.21.15.26:0/2304145601 wait complete. 2024-03-06T07:25:13.760 INFO:tasks.cephfs_test_runner:That a sole MDS with mds_join_fs set will come back as active eventually even after failover. ... ok 2024-03-06T07:25:13.760 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_unset (tasks.cephfs.test_failover.TestClusterAffinity) That unsetting mds_join_fs will cause failover if another high-affinity standby exists. 2024-03-06T07:25:13.760 INFO:tasks.cephfs_test_runner:test_join_fs_unset (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:25:13.760 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.863+0000 7fcfe217a640 1 Processor -- start 2024-03-06T07:25:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.863+0000 7fcfe217a640 1 -- start start 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfe217a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 0x7fcfdc0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfe217a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc155110 con 0x7fcfdc158db0 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfe217a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0c8f40 con 0x7fcfdc0c5330 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfe217a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1559e0 con 0x7fcfdc0c88b0 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfdaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcfdc158db0 0x7fcfdc15da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47634/0 (socket says 172.21.15.26:47634) 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfdaffd640 1 -- 172.21.15.26:0/4031842796 learned_addr learned my addr 172.21.15.26:0/4031842796 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfdbfff640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 0x7fcfdc0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543383387 0 0) 0x7fcfd0001110 con 0x7fcfdc158db0 2024-03-06T07:25:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.864+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc4003520 con 0x7fcfdc158db0 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091176205 0 0) 0x7fcfdc0c8f40 con 0x7fcfdc0c5330 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd0001110 con 0x7fcfdc0c5330 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119886721 0 0) 0x7fcfc4003520 con 0x7fcfdc158db0 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 msgr2=0x7fcfdc0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 0x7fcfdc0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 >> v1:172.21.15.117:6789/0 conn(0x7fcfdc0c5330 legacy=0x7fcfdc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc155390 con 0x7fcfdc158db0 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fcfd00017c0 con 0x7fcfdc158db0 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 >> v1:172.21.15.26:6789/0 conn(0x7fcfdc158db0 legacy=0x7fcfdc15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 0x7fcfc4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc4004140 0x7fcfc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfc4004d40 0x7fcfc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1559e0 con 0x7fcfc4004140 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc400bbb0 con 0x7fcfc4004d40 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc400bdb0 con 0x7fcfdc0c88b0 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdaffd640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc4004140 0x7fcfc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdb7fe640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 0x7fcfc4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdbfff640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfc4004d40 0x7fcfc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdaffd640 1 -- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 msgr2=0x7fcfc4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdaffd640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c88b0 0x7fcfc4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdaffd640 1 -- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfc4004d40 msgr2=0x7fcfc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdaffd640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfc4004d40 0x7fcfc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.865+0000 7fcfdaffd640 1 -- 172.21.15.26:0/4031842796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4003520 con 0x7fcfc4004140 2024-03-06T07:25:13.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.866+0000 7fcfdaffd640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc4004140 0x7fcfc4004800 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c240 tx=0x7fcfc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad5653f5480db29 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.866+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0007d30 con 0x7fcfc4004140 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.866+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0030070 con 0x7fcfc4004140 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.866+0000 7fcfda7fc640 1 -- 172.21.15.26:0/4031842796 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0005d30 con 0x7fcfc4004140 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.866+0000 7fcfe217a640 1 -- 172.21.15.26:0/4031842796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc4004140 msgr2=0x7fcfc4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.867+0000 7fcfe217a640 1 --2- 172.21.15.26:0/4031842796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc4004140 0x7fcfc4004800 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c240 tx=0x7fcfc4005830 comp rx=0 tx=0).stop 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.867+0000 7fcfdbfff640 1 -- 172.21.15.26:0/4031842796 reap_dead start 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.867+0000 7fcfe217a640 1 -- 172.21.15.26:0/4031842796 shutdown_connections 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.867+0000 7fcfe217a640 1 -- 172.21.15.26:0/4031842796 >> 172.21.15.26:0/4031842796 conn(0x7fcfdc0ac560 msgr2=0x7fcfdc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.867+0000 7fcfe217a640 1 -- 172.21.15.26:0/4031842796 shutdown_connections 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.867+0000 7fcfe217a640 1 -- 172.21.15.26:0/4031842796 wait complete. 2024-03-06T07:25:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.868+0000 7fcfe217a640 1 Processor -- start 2024-03-06T07:25:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.868+0000 7fcfe217a640 1 -- start start 2024-03-06T07:25:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.868+0000 7fcfe217a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfc4004140 0x7fcfdc1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.868+0000 7fcfe217a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc1ee540 0x7fcfdc14cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.868+0000 7fcfe217a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc0c4160 0x7fcfdc14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfe217a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1529d0 con 0x7fcfdc0c4160 2024-03-06T07:25:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfe217a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc06d090 con 0x7fcfdc1ee540 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfe217a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc152b50 con 0x7fcfc4004140 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc0c4160 0x7fcfdc14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc0c4160 0x7fcfdc14b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53010/0 (socket says 172.21.15.26:53010) 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 -- 172.21.15.26:0/1335877364 learned_addr learned my addr 172.21.15.26:0/1335877364 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdb7fe640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfc4004140 0x7fcfdc1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdaffd640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc1ee540 0x7fcfdc14cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 -- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfc4004140 msgr2=0x7fcfdc1514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfc4004140 0x7fcfdc1514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 -- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc1ee540 msgr2=0x7fcfdc14cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc1ee540 0x7fcfdc14cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 -- 172.21.15.26:0/1335877364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc151a30 con 0x7fcfdc0c4160 2024-03-06T07:25:13.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdaffd640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc1ee540 0x7fcfdc14cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:13.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdb7fe640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfc4004140 0x7fcfdc1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:13.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfdbfff640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc0c4160 0x7fcfdc14b1a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc06cd90 tx=0x7fcfd400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfc3fff640 1 -- 172.21.15.26:0/1335877364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400fe10 con 0x7fcfdc0c4160 2024-03-06T07:25:13.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfc3fff640 1 -- 172.21.15.26:0/1335877364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd4015430 con 0x7fcfdc0c4160 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc1eea20 con 0x7fcfdc0c4160 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfc3fff640 1 -- 172.21.15.26:0/1335877364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4015820 con 0x7fcfdc0c4160 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.869+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc150730 con 0x7fcfdc0c4160 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.870+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc04f330 con 0x7fcfdc0c4160 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.871+0000 7fcfc3fff640 1 -- 172.21.15.26:0/1335877364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfd40159c0 con 0x7fcfdc0c4160 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.871+0000 7fcfc3fff640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfc806b500 0x7fcfc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.871+0000 7fcfc3fff640 1 -- 172.21.15.26:0/1335877364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fcfd408ae70 con 0x7fcfdc0c4160 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.871+0000 7fcfdb7fe640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfc806b500 0x7fcfc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.872+0000 7fcfdb7fe640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfc806b500 0x7fcfc806d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc159280 tx=0x7fcfcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:13.875+0000 7fcfc3fff640 1 -- 172.21.15.26:0/1335877364 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcfd4019030 con 0x7fcfdc0c4160 2024-03-06T07:25:14.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.057+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfdc06cdc0 con 0x7fcfdc0c4160 2024-03-06T07:25:14.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.058+0000 7fcfc3fff640 1 -- 172.21.15.26:0/1335877364 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+13573 (secure 0 0 0) 0x7fcfd4010040 con 0x7fcfdc0c4160 2024-03-06T07:25:14.061 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:14.061 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":126,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:12.732888+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6832/1856740515":"2024-03-07T07:25:00.699492+0000","172.21.15.26:6833/1367162338":"2024-03-07T07:25:09.558060+0000","172.21.15.26:6835/577277314":"2024-03-07T07:24:57.677470+0000","172.21.15.26:6835/1367162338":"2024-03-07T07:25:09.558060+0000","172.21.15.26:6833/577277314":"2024-03-07T07:24:57.677470+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfc806b500 msgr2=0x7fcfc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfc806b500 0x7fcfc806d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc159280 tx=0x7fcfcc009040 comp rx=0 tx=0).stop 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc0c4160 msgr2=0x7fcfdc14b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc0c4160 0x7fcfdc14b1a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc06cd90 tx=0x7fcfd400f580 comp rx=0 tx=0).stop 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 shutdown_connections 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfc806b500 0x7fcfc806d9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfdc0c4160 0x7fcfdc14b1a0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc1ee540 0x7fcfdc14cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 --2- 172.21.15.26:0/1335877364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfc4004140 0x7fcfdc1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 >> 172.21.15.26:0/1335877364 conn(0x7fcfdc0ac560 msgr2=0x7fcfdc0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 shutdown_connections 2024-03-06T07:25:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.060+0000 7fcfe217a640 1 -- 172.21.15.26:0/1335877364 wait complete. 2024-03-06T07:25:14.073 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:14.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:14.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.176+0000 7f3436888640 1 Processor -- start 2024-03-06T07:25:14.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.176+0000 7f3436888640 1 -- start start 2024-03-06T07:25:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3436888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f3430155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3436888640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3430152db0 con 0x7f34301529d0 2024-03-06T07:25:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3436888640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34301590f0 con 0x7f3430072cd0 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3436888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34300c6dd0 con 0x7f3430073290 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f342ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f34301529d0 0x7f34300c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47642/0 (socket says 172.21.15.26:47642) 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f342ffff640 1 -- 172.21.15.26:0/637930974 learned_addr learned my addr 172.21.15.26:0/637930974 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3434e25640 1 --2- 172.21.15.26:0/637930974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f3430155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722810033 0 0) 0x7f3424001110 con 0x7f34301529d0 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3418003520 con 0x7f34301529d0 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139703930 0 0) 0x7f3420001110 con 0x7f3430072cd0 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3424001110 con 0x7f3430072cd0 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927675864 0 0) 0x7f3418003520 con 0x7f34301529d0 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3434e25640 1 -- 172.21.15.26:0/637930974 >> v1:172.21.15.117:6789/0 conn(0x7f3430072cd0 legacy=0x7f3430155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3434e25640 1 -- 172.21.15.26:0/637930974 >> v1:172.21.15.26:6789/0 conn(0x7f34301529d0 legacy=0x7f34300c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.177+0000 7f3434e25640 1 -- 172.21.15.26:0/637930974 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430159650 con 0x7f3430073290 2024-03-06T07:25:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.178+0000 7f3434e25640 1 --2- 172.21.15.26:0/637930974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f3430155830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3428009dc0 tx=0x7f342802bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b471021bfd087e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.178+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428039070 con 0x7f3430073290 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.178+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342802c7f0 con 0x7f3430073290 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.178+0000 7f342effd640 1 -- 172.21.15.26:0/637930974 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424001840 con 0x7f3430073290 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.179+0000 7f3436888640 1 -- 172.21.15.26:0/637930974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 msgr2=0x7f3430155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.179+0000 7f3436888640 1 --2- 172.21.15.26:0/637930974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f3430155830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3428009dc0 tx=0x7f342802bcc0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.179+0000 7f3436888640 1 -- 172.21.15.26:0/637930974 shutdown_connections 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.179+0000 7f3436888640 1 --2- 172.21.15.26:0/637930974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f3430155830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.179+0000 7f3436888640 1 -- 172.21.15.26:0/637930974 >> 172.21.15.26:0/637930974 conn(0x7f34300aa630 msgr2=0x7f34300aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.179+0000 7f3436888640 1 -- 172.21.15.26:0/637930974 shutdown_connections 2024-03-06T07:25:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.179+0000 7f3436888640 1 -- 172.21.15.26:0/637930974 wait complete. 2024-03-06T07:25:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.180+0000 7f3436888640 1 Processor -- start 2024-03-06T07:25:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.180+0000 7f3436888640 1 -- start start 2024-03-06T07:25:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3430072cd0 0x7f3430151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f34301eae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34301529d0 0x7f34301eb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3430155d70 con 0x7f3430072cd0 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34300c6940 con 0x7f34301529d0 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3430159370 con 0x7f3430073290 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3430072cd0 0x7f3430151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3430072cd0 0x7f3430151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53022/0 (socket says 172.21.15.26:53022) 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 -- 172.21.15.26:0/2940838755 learned_addr learned my addr 172.21.15.26:0/2940838755 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3434e25640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34301529d0 0x7f34301eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342f7fe640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f34301eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 -- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 msgr2=0x7f34301eae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f34301eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 -- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34301529d0 msgr2=0x7f34301eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34301529d0 0x7f34301eb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 -- 172.21.15.26:0/2940838755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343006e040 con 0x7f3430072cd0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342ffff640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3430072cd0 0x7f3430151240 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f342400c710 tx=0x7f342400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342cff9640 1 -- 172.21.15.26:0/2940838755 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424017070 con 0x7f3430072cd0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342cff9640 1 -- 172.21.15.26:0/2940838755 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3424005d00 con 0x7f3430072cd0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f342cff9640 1 -- 172.21.15.26:0/2940838755 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34240072b0 con 0x7f3430072cd0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34301ee320 con 0x7f3430072cd0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.182+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34301eebc0 con 0x7f3430072cd0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.184+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343004f330 con 0x7f3430072cd0 2024-03-06T07:25:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.184+0000 7f342cff9640 1 -- 172.21.15.26:0/2940838755 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f342401a070 con 0x7f3430072cd0 2024-03-06T07:25:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.184+0000 7f342cff9640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.184+0000 7f342cff9640 1 -- 172.21.15.26:0/2940838755 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f34240897c0 con 0x7f3430072cd0 2024-03-06T07:25:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.184+0000 7f342f7fe640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.184+0000 7f342f7fe640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f341c06b4d0 0x7f341c06d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34200012b0 tx=0x7f3420009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.188+0000 7f342cff9640 1 -- 172.21.15.26:0/2940838755 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3430152db0 con 0x7f3430072cd0 2024-03-06T07:25:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.367+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34300a9e90 con 0x7f3430072cd0 2024-03-06T07:25:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.368+0000 7f342cff9640 1 -- 172.21.15.26:0/2940838755 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f342400ce20 con 0x7f3430072cd0 2024-03-06T07:25:14.370 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f341c06b4d0 msgr2=0x7f341c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f341c06b4d0 0x7f341c06d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34200012b0 tx=0x7f3420009040 comp rx=0 tx=0).stop 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3430072cd0 msgr2=0x7f3430151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3430072cd0 0x7f3430151240 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f342400c710 tx=0x7f342400cbd0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 shutdown_connections 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34301529d0 0x7f34301eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3430073290 0x7f34301eae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.369+0000 7f3436888640 1 --2- 172.21.15.26:0/2940838755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3430072cd0 0x7f3430151240 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.370+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 >> 172.21.15.26:0/2940838755 conn(0x7f34300aa630 msgr2=0x7f34300aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.370+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 shutdown_connections 2024-03-06T07:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.370+0000 7f3436888640 1 -- 172.21.15.26:0/2940838755 wait complete. 2024-03-06T07:25:14.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset' 2024-03-06T07:25:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.482+0000 7fc18f8eb640 1 Processor -- start 2024-03-06T07:25:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.482+0000 7fc18f8eb640 1 -- start start 2024-03-06T07:25:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18f8eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc18814f790 0x7fc1881529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18f8eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1880c1aa0 0x7fc1880c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18f8eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880c28d0 con 0x7fc1880c5c60 2024-03-06T07:25:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18f8eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1880c2680 con 0x7fc1880c1aa0 2024-03-06T07:25:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18f8eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880c2bb0 con 0x7fc18814f790 2024-03-06T07:25:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18de88640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1880c5c60 0x7fc188155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47650/0 (socket says 172.21.15.26:47650) 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18de88640 1 -- 172.21.15.26:0/1307126111 learned_addr learned my addr 172.21.15.26:0/1307126111 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18d687640 1 --2- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc18814f790 0x7fc1881529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc18ce86640 1 --2- 172.21.15.26:0/1307126111 >> v2:172.21.15.117:3300/0 conn(0x7fc1880c1aa0 0x7fc1880c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc17ffff640 1 -- 172.21.15.26:0/1307126111 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118605505 0 0) 0x7fc1880c28d0 con 0x7fc1880c5c60 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.483+0000 7fc17ffff640 1 -- 172.21.15.26:0/1307126111 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc174003520 con 0x7fc1880c5c60 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.484+0000 7fc18ce86640 1 -- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc18814f790 msgr2=0x7fc1881529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.484+0000 7fc18ce86640 1 --2- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc18814f790 0x7fc1881529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.484+0000 7fc18ce86640 1 -- 172.21.15.26:0/1307126111 >> v1:172.21.15.26:6789/0 conn(0x7fc1880c5c60 legacy=0x7fc188155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.484+0000 7fc18ce86640 1 -- 172.21.15.26:0/1307126111 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880c2e90 con 0x7fc1880c1aa0 2024-03-06T07:25:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.484+0000 7fc18d687640 1 --2- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc18814f790 0x7fc1881529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.484+0000 7fc18ce86640 1 --2- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1880c1aa0 0x7fc1880c5720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc1780097a0 tx=0x7fc17802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a0ca57b006e7f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc17ffff640 1 -- 172.21.15.26:0/1307126111 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178035020 con 0x7fc1880c1aa0 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc17ffff640 1 -- 172.21.15.26:0/1307126111 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178031830 con 0x7fc1880c1aa0 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc17ffff640 1 -- 172.21.15.26:0/1307126111 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178031c00 con 0x7fc1880c1aa0 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1880c1aa0 msgr2=0x7fc1880c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1880c1aa0 0x7fc1880c5720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc1780097a0 tx=0x7fc17802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1307126111 shutdown_connections 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1880c1aa0 0x7fc1880c5720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1307126111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc18814f790 0x7fc1881529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1307126111 >> 172.21.15.26:0/1307126111 conn(0x7fc1880c6710 msgr2=0x7fc1880c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1307126111 shutdown_connections 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.485+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1307126111 wait complete. 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.486+0000 7fc18f8eb640 1 Processor -- start 2024-03-06T07:25:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.486+0000 7fc18f8eb640 1 -- start start 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18f8eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1880c1aa0 0x7fc1880bce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18f8eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1880c5c60 0x7fc1880bc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18f8eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc18814f790 0x7fc18815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18f8eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18806cc00 con 0x7fc1880c1aa0 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18f8eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880c0d60 con 0x7fc18814f790 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18f8eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18806cd80 con 0x7fc1880c5c60 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1880c1aa0 0x7fc1880bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1880c1aa0 0x7fc1880bce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53032/0 (socket says 172.21.15.26:53032) 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 -- 172.21.15.26:0/1970565216 learned_addr learned my addr 172.21.15.26:0/1970565216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18de88640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc18814f790 0x7fc18815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18ce86640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1880c5c60 0x7fc1880bc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 -- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1880c5c60 msgr2=0x7fc1880bc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1880c5c60 0x7fc1880bc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 -- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc18814f790 msgr2=0x7fc18815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc18814f790 0x7fc18815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18d687640 1 -- 172.21.15.26:0/1970565216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880722a0 con 0x7fc1880c1aa0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.487+0000 7fc18ce86640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1880c5c60 0x7fc1880bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.488+0000 7fc18d687640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1880c1aa0 0x7fc1880bce40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e990 tx=0x7fc18400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.488+0000 7fc17dffb640 1 -- 172.21.15.26:0/1970565216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184017070 con 0x7fc1880c1aa0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.488+0000 7fc17dffb640 1 -- 172.21.15.26:0/1970565216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18400c9e0 con 0x7fc1880c1aa0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.488+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc188072580 con 0x7fc1880c1aa0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.488+0000 7fc17dffb640 1 -- 172.21.15.26:0/1970565216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400cdb0 con 0x7fc1880c1aa0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.488+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1880bd380 con 0x7fc1880c1aa0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.489+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc188075300 con 0x7fc1880c1aa0 2024-03-06T07:25:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.489+0000 7fc17dffb640 1 -- 172.21.15.26:0/1970565216 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc18401a070 con 0x7fc1880c1aa0 2024-03-06T07:25:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.490+0000 7fc17dffb640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc16c06b500 0x7fc16c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.490+0000 7fc17dffb640 1 -- 172.21.15.26:0/1970565216 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc184089e80 con 0x7fc1880c1aa0 2024-03-06T07:25:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.490+0000 7fc18ce86640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc16c06b500 0x7fc16c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.491+0000 7fc18ce86640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc16c06b500 0x7fc16c06d9c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc178036000 tx=0x7fc17803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.495+0000 7fc17dffb640 1 -- 172.21.15.26:0/1970565216 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc188075300 con 0x7fc1880c1aa0 2024-03-06T07:25:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.745+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]} v 0) v1 -- 0x7fc18804f330 con 0x7fc1880c1aa0 2024-03-06T07:25:14.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.747+0000 7fc17dffb640 1 -- 172.21.15.26:0/1970565216 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]}]=0 v305) v1 ==== 147+0+0 (secure 0 0 0) 0x7fc184058800 con 0x7fc1880c1aa0 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc16c06b500 msgr2=0x7fc16c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc16c06b500 0x7fc16c06d9c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc178036000 tx=0x7fc17803c040 comp rx=0 tx=0).stop 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1880c1aa0 msgr2=0x7fc1880bce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1880c1aa0 0x7fc1880bce40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e990 tx=0x7fc18400ee50 comp rx=0 tx=0).stop 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 shutdown_connections 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc16c06b500 0x7fc16c06d9c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc18814f790 0x7fc18815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1880c5c60 0x7fc1880bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 --2- 172.21.15.26:0/1970565216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1880c1aa0 0x7fc1880bce40 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 >> 172.21.15.26:0/1970565216 conn(0x7fc1880c6710 msgr2=0x7fc1880c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 shutdown_connections 2024-03-06T07:25:14.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.749+0000 7fc18f8eb640 1 -- 172.21.15.26:0/1970565216 wait complete. 2024-03-06T07:25:14.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:25:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.864+0000 7fc5b7713640 1 Processor -- start 2024-03-06T07:25:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.864+0000 7fc5b7713640 1 -- start start 2024-03-06T07:25:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b7713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b0155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b7713640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5b0073290 0x7fc5b0155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b7713640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b0152db0 con 0x7fc5b01529d0 2024-03-06T07:25:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b7713640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b00c6dd0 con 0x7fc5b0073290 2024-03-06T07:25:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b7713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b00c6940 con 0x7fc5b0072cd0 2024-03-06T07:25:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b4cae640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b5cb0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5b0073290 0x7fc5b0155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b54af640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc5b01529d0 0x7fc5b00c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44108/0 (socket says 172.21.15.26:44108) 2024-03-06T07:25:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b54af640 1 -- 172.21.15.26:0/3051704097 learned_addr learned my addr 172.21.15.26:0/3051704097 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5a7fff640 1 -- 172.21.15.26:0/3051704097 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982213876 0 0) 0x7fc5a8001110 con 0x7fc5b01529d0 2024-03-06T07:25:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b4cae640 1 -- 172.21.15.26:0/3051704097 >> v2:172.21.15.117:3300/0 conn(0x7fc5b0073290 msgr2=0x7fc5b0155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.865+0000 7fc5b4cae640 1 --2- 172.21.15.26:0/3051704097 >> v2:172.21.15.117:3300/0 conn(0x7fc5b0073290 0x7fc5b0155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.866+0000 7fc5b4cae640 1 -- 172.21.15.26:0/3051704097 >> v1:172.21.15.117:6789/0 conn(0x7fc5b01529d0 legacy=0x7fc5b00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.866+0000 7fc5b4cae640 1 -- 172.21.15.26:0/3051704097 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b01590f0 con 0x7fc5b0072cd0 2024-03-06T07:25:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.866+0000 7fc5b4cae640 1 --2- 172.21.15.26:0/3051704097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b0155120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00097a0 tx=0x7fc5a002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c5e36e5696bc89 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5a7fff640 1 -- 172.21.15.26:0/3051704097 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0038070 con 0x7fc5b0072cd0 2024-03-06T07:25:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5a7fff640 1 -- 172.21.15.26:0/3051704097 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0031700 con 0x7fc5b0072cd0 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5a7fff640 1 -- 172.21.15.26:0/3051704097 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0031ad0 con 0x7fc5b0072cd0 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 -- 172.21.15.26:0/3051704097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 msgr2=0x7fc5b0155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 --2- 172.21.15.26:0/3051704097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b0155120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00097a0 tx=0x7fc5a002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 -- 172.21.15.26:0/3051704097 shutdown_connections 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 --2- 172.21.15.26:0/3051704097 >> v2:172.21.15.117:3300/0 conn(0x7fc5b0073290 0x7fc5b0155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 --2- 172.21.15.26:0/3051704097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b0155120 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 -- 172.21.15.26:0/3051704097 >> 172.21.15.26:0/3051704097 conn(0x7fc5b00aa630 msgr2=0x7fc5b00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 -- 172.21.15.26:0/3051704097 shutdown_connections 2024-03-06T07:25:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.867+0000 7fc5b7713640 1 -- 172.21.15.26:0/3051704097 wait complete. 2024-03-06T07:25:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.868+0000 7fc5b7713640 1 Processor -- start 2024-03-06T07:25:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.868+0000 7fc5b7713640 1 -- start start 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b7713640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b7713640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5b0073290 0x7fc5b0147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b7713640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5b01529d0 0x7fc5b0147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b7713640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b0159370 con 0x7fc5b01529d0 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b7713640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b0072720 con 0x7fc5b0073290 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b7713640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b00723b0 con 0x7fc5b0072cd0 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b5cb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5b01529d0 0x7fc5b0147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b5cb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5b01529d0 0x7fc5b0147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53034/0 (socket says 172.21.15.26:53034) 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b5cb0640 1 -- 172.21.15.26:0/361530339 learned_addr learned my addr 172.21.15.26:0/361530339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b4cae640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5b0073290 0x7fc5b0147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b54af640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b5cb0640 1 -- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 msgr2=0x7fc5b014e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.869+0000 7fc5b5cb0640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b014e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5b5cb0640 1 -- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5b0073290 msgr2=0x7fc5b0147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5b5cb0640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5b0073290 0x7fc5b0147820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5b5cb0640 1 -- 172.21.15.26:0/361530339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b0156f70 con 0x7fc5b01529d0 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5b5cb0640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5b01529d0 0x7fc5b0147d60 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00ec10 tx=0x7fc5ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5a5ffb640 1 -- 172.21.15.26:0/361530339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac018020 con 0x7fc5b01529d0 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5a5ffb640 1 -- 172.21.15.26:0/361530339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac00cd20 con 0x7fc5b01529d0 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b0157250 con 0x7fc5b01529d0 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5a5ffb640 1 -- 172.21.15.26:0/361530339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0146c0 con 0x7fc5b01529d0 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.870+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b0157810 con 0x7fc5b01529d0 2024-03-06T07:25:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.871+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b004f330 con 0x7fc5b01529d0 2024-03-06T07:25:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.872+0000 7fc5a5ffb640 1 -- 172.21.15.26:0/361530339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5ac005d10 con 0x7fc5b01529d0 2024-03-06T07:25:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.872+0000 7fc5a5ffb640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59406b4d0 0x7fc59406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.872+0000 7fc5a5ffb640 1 -- 172.21.15.26:0/361530339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc5ac08ac00 con 0x7fc5b01529d0 2024-03-06T07:25:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.872+0000 7fc5b54af640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59406b4d0 0x7fc59406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.873+0000 7fc5b54af640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59406b4d0 0x7fc59406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8007fd0 tx=0x7fc5a8007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:14.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:14.876+0000 7fc5a5ffb640 1 -- 172.21.15.26:0/361530339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5ac022030 con 0x7fc5b01529d0 2024-03-06T07:25:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.058+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc5b0068e60 con 0x7fc5b01529d0 2024-03-06T07:25:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.061+0000 7fc5a5ffb640 1 -- 172.21.15.26:0/361530339 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc5ac010040 con 0x7fc5b01529d0 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59406b4d0 msgr2=0x7fc59406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59406b4d0 0x7fc59406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8007fd0 tx=0x7fc5a8007de0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5b01529d0 msgr2=0x7fc5b0147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5b01529d0 0x7fc5b0147d60 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00ec10 tx=0x7fc5ac00c490 comp rx=0 tx=0).stop 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 shutdown_connections 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59406b4d0 0x7fc59406d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5b01529d0 0x7fc5b0147d60 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5b0073290 0x7fc5b0147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 --2- 172.21.15.26:0/361530339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5b0072cd0 0x7fc5b014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 >> 172.21.15.26:0/361530339 conn(0x7fc5b00aa630 msgr2=0x7fc5b00c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 shutdown_connections 2024-03-06T07:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.063+0000 7fc5b7713640 1 -- 172.21.15.26:0/361530339 wait complete. 2024-03-06T07:25:15.075 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:25:15.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:15.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:15.157 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:25:15.157 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:25:15.158 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:15.172 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:25:15.172 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:25:15.173 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:15.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:25:15.214 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:25:15.215 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:15.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.314+0000 7f445a62a640 1 Processor -- start 2024-03-06T07:25:15.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.314+0000 7f445a62a640 1 -- start start 2024-03-06T07:25:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f445a62a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44540c88c0 0x7f4454155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f445a62a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540c73d0 con 0x7f445414f360 2024-03-06T07:25:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f445a62a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540be2e0 con 0x7f4454158b60 2024-03-06T07:25:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f445a62a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4454159700 con 0x7f44540c88c0 2024-03-06T07:25:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f4453fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f445414f360 0x7f44540c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47654/0 (socket says 172.21.15.26:47654) 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f4453fff640 1 -- 172.21.15.26:0/1695848921 learned_addr learned my addr 172.21.15.26:0/1695848921 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854842617 0 0) 0x7f44540c73d0 con 0x7f445414f360 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f44537fe640 1 --2- 172.21.15.26:0/1695848921 >> v2:172.21.15.117:3300/0 conn(0x7f44540c88c0 0x7f4454155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.315+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f443c003520 con 0x7f445414f360 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109665124 0 0) 0x7f443c003520 con 0x7f445414f360 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 >> v2:172.21.15.117:3300/0 conn(0x7f44540c88c0 msgr2=0x7f4454155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 --2- 172.21.15.26:0/1695848921 >> v2:172.21.15.117:3300/0 conn(0x7f44540c88c0 0x7f4454155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 >> v1:172.21.15.117:6789/0 conn(0x7f4454158b60 legacy=0x7f4454158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540c7670 con 0x7f445414f360 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4448001670 con 0x7f445414f360 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 >> v1:172.21.15.26:6789/0 conn(0x7f445414f360 legacy=0x7f44540c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0039b0 0x7f443c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c004800 0x7f443c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0051c0 0x7f443c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454159700 con 0x7f443c0051c0 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c00c160 con 0x7f443c004800 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c00c360 con 0x7f443c0039b0 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4458bc7640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0051c0 0x7f443c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f44537fe640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c004800 0x7f443c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4453fff640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0039b0 0x7f443c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4458bc7640 1 -- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0039b0 msgr2=0x7f443c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4458bc7640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0039b0 0x7f443c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4458bc7640 1 -- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c004800 msgr2=0x7f443c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4458bc7640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c004800 0x7f443c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.316+0000 7f4458bc7640 1 -- 172.21.15.26:0/1695848921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c003520 con 0x7f443c0051c0 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.317+0000 7f4458bc7640 1 -- 172.21.15.26:0/1695848921 reap_dead start 2024-03-06T07:25:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.317+0000 7f4458bc7640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0051c0 0x7f443c00b9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f444c001e50 tx=0x7f443c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd87ee8b58803b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.317+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c001a70 con 0x7f443c0051c0 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.317+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c003b90 con 0x7f443c0051c0 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.317+0000 7f4452ffd640 1 -- 172.21.15.26:0/1695848921 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0050d0 con 0x7f443c0051c0 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.318+0000 7f445a62a640 1 -- 172.21.15.26:0/1695848921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0051c0 msgr2=0x7f443c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.318+0000 7f445a62a640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0051c0 0x7f443c00b9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f444c001e50 tx=0x7f443c005830 comp rx=0 tx=0).stop 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.318+0000 7f445a62a640 1 -- 172.21.15.26:0/1695848921 shutdown_connections 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.318+0000 7f445a62a640 1 --2- 172.21.15.26:0/1695848921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f443c0051c0 0x7f443c00b9c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.318+0000 7f445a62a640 1 -- 172.21.15.26:0/1695848921 >> 172.21.15.26:0/1695848921 conn(0x7f44540ac570 msgr2=0x7f44540ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.318+0000 7f445a62a640 1 -- 172.21.15.26:0/1695848921 shutdown_connections 2024-03-06T07:25:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.318+0000 7f445a62a640 1 -- 172.21.15.26:0/1695848921 wait complete. 2024-03-06T07:25:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.319+0000 7f445a62a640 1 Processor -- start 2024-03-06T07:25:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.319+0000 7f445a62a640 1 -- start start 2024-03-06T07:25:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.319+0000 7f445a62a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0051c0 0x7f44540b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f445a62a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44540c1200 0x7f4454157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f445a62a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44541ee8f0 0x7f44541eed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f445a62a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c7c30 con 0x7f44541ee8f0 2024-03-06T07:25:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f445a62a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454155c10 con 0x7f44540c1200 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f445a62a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4454155d90 con 0x7f443c0051c0 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44541ee8f0 0x7f44541eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44541ee8f0 0x7f44541eed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53054/0 (socket says 172.21.15.26:53054) 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 -- 172.21.15.26:0/1172218016 learned_addr learned my addr 172.21.15.26:0/1172218016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f44537fe640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44540c1200 0x7f4454157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4453fff640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0051c0 0x7f44540b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 -- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0051c0 msgr2=0x7f44540b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0051c0 0x7f44540b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 -- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44540c1200 msgr2=0x7f4454157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44540c1200 0x7f4454157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 -- 172.21.15.26:0/1172218016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44541586c0 con 0x7f44541ee8f0 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4458bc7640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44541ee8f0 0x7f44541eed70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f444c005040 tx=0x7f444c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4450ff9640 1 -- 172.21.15.26:0/1172218016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c006d50 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4450ff9640 1 -- 172.21.15.26:0/1172218016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c007030 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f4450ff9640 1 -- 172.21.15.26:0/1172218016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c007400 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44541ef2b0 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.320+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44540ae630 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.321+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445404f330 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.322+0000 7f4450ff9640 1 -- 172.21.15.26:0/1172218016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f444c0075a0 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.322+0000 7f4450ff9640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f444006b500 0x7f444006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.322+0000 7f4450ff9640 1 -- 172.21.15.26:0/1172218016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f444c0abc10 con 0x7f44541ee8f0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.322+0000 7f4453fff640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f444006b500 0x7f444006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.323+0000 7f4453fff640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f444006b500 0x7f444006d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f44541e8cc0 tx=0x7f4448001c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.326+0000 7f4450ff9640 1 -- 172.21.15.26:0/1172218016 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f444c033030 con 0x7f44541ee8f0 2024-03-06T07:25:15.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.549+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44540c8d50 con 0x7f44541ee8f0 2024-03-06T07:25:15.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.549+0000 7f4450ff9640 1 -- 172.21.15.26:0/1172218016 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+3707 (secure 0 0 0) 0x7f444c07ab40 con 0x7f44541ee8f0 2024-03-06T07:25:15.552 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:15.552 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3943022187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3943022187},{"type":"v1","addr":"172.21.15.117:6836","nonce":3943022187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3949772965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3949772965},{"type":"v1","addr":"172.21.15.117:6837","nonce":3949772965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/317629496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":317629496},{"type":"v1","addr":"172.21.15.26:6834","nonce":317629496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3690514899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3690514899},{"type":"v1","addr":"172.21.15.26:6835","nonce":3690514899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[]} 2024-03-06T07:25:15.552 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 117 2024-03-06T07:25:15.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.551+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f444006b500 msgr2=0x7f444006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.551+0000 7f445a62a640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f444006b500 0x7f444006d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f44541e8cc0 tx=0x7f4448001c60 comp rx=0 tx=0).stop 2024-03-06T07:25:15.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44541ee8f0 msgr2=0x7f44541eed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44541ee8f0 0x7f44541eed70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f444c005040 tx=0x7f444c0014a0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 shutdown_connections 2024-03-06T07:25:15.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f444006b500 0x7f444006d9c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44541ee8f0 0x7f44541eed70 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44540c1200 0x7f4454157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 --2- 172.21.15.26:0/1172218016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443c0051c0 0x7f44540b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 >> 172.21.15.26:0/1172218016 conn(0x7f44540ac570 msgr2=0x7f44540c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:15.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 shutdown_connections 2024-03-06T07:25:15.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.552+0000 7f445a62a640 1 -- 172.21.15.26:0/1172218016 wait complete. 2024-03-06T07:25:15.565 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:25:15.565 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:25:15.565 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:25:15.565 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:15.566 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:25:15.566 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:25:15.566 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:25:15.566 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:15.566 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:25:15.566 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:25:15.567 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:25:15.567 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:15.567 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:25:15.567 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:25:15.567 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:25:15.568 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:15.576 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:25:15.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:25:15.578 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:25:15.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:25:15.579 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:25:15.579 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:25:15.580 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:25:15.580 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:25:15.619 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:25:15.619 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:15.620 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:25:15.620 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:15.623 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:25:15.623 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:15.625 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:25:15.625 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:15.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:25:15.695 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:25:15.696 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:25:15.696 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-03-06T07:25:15.695+0000 7f041f904640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-03-06T07:25:15.707 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:25:15.707 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:25:15.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.730+0000 7ff341a4e640 1 Processor -- start 2024-03-06T07:25:15.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff341a4e640 1 -- start start 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff341a4e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c69f0 0x7ff33c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff341a4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff33c0c8950 0x7ff33c0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff341a4e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c155130 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff341a4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c155440 con 0x7ff33c0c8950 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff341a4e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c0c5d50 con 0x7ff33c0c69f0 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff33a7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff33c0c8950 0x7ff33c0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff33a7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff33c0c8950 0x7ff33c0acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53076/0 (socket says 172.21.15.26:53076) 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.731+0000 7ff33a7fc640 1 -- 172.21.15.26:0/1727734047 learned_addr learned my addr 172.21.15.26:0/1727734047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.732+0000 7ff33affd640 1 --2- 172.21.15.26:0/1727734047 >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c69f0 0x7ff33c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.732+0000 7ff33a7fc640 1 -- 172.21.15.26:0/1727734047 >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c69f0 msgr2=0x7ff33c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.732+0000 7ff33a7fc640 1 --2- 172.21.15.26:0/1727734047 >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c69f0 0x7ff33c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.732+0000 7ff33a7fc640 1 -- 172.21.15.26:0/1727734047 >> v1:172.21.15.117:6789/0 conn(0x7ff33c0ad2b0 legacy=0x7ff33c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.732+0000 7ff33a7fc640 1 -- 172.21.15.26:0/1727734047 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c0c6030 con 0x7ff33c0c8950 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.732+0000 7ff33affd640 1 --2- 172.21.15.26:0/1727734047 >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c69f0 0x7ff33c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.732+0000 7ff33a7fc640 1 --2- 172.21.15.26:0/1727734047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0c8950 0x7ff33c0acd70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0aa430 tx=0x7ff33002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6c920f5021efe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff339ffb640 1 -- 172.21.15.26:0/1727734047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330035020 con 0x7ff33c0c8950 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff339ffb640 1 -- 172.21.15.26:0/1727734047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330031870 con 0x7ff33c0c8950 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff339ffb640 1 -- 172.21.15.26:0/1727734047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330031c40 con 0x7ff33c0c8950 2024-03-06T07:25:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 -- 172.21.15.26:0/1727734047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0c8950 msgr2=0x7ff33c0acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 --2- 172.21.15.26:0/1727734047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0c8950 0x7ff33c0acd70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0aa430 tx=0x7ff33002b910 comp rx=0 tx=0).stop 2024-03-06T07:25:15.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 -- 172.21.15.26:0/1727734047 shutdown_connections 2024-03-06T07:25:15.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 --2- 172.21.15.26:0/1727734047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0c8950 0x7ff33c0acd70 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 --2- 172.21.15.26:0/1727734047 >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c69f0 0x7ff33c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 -- 172.21.15.26:0/1727734047 >> 172.21.15.26:0/1727734047 conn(0x7ff33c0cae70 msgr2=0x7ff33c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:15.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 -- 172.21.15.26:0/1727734047 shutdown_connections 2024-03-06T07:25:15.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.733+0000 7ff341a4e640 1 -- 172.21.15.26:0/1727734047 wait complete. 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.734+0000 7ff341a4e640 1 Processor -- start 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff341a4e640 1 -- start start 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff341a4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0ad2b0 0x7ff33c154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff341a4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c69f0 0x7ff33c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff341a4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c8950 0x7ff33c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff341a4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c152a20 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff341a4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c155fe0 con 0x7ff33c0c8950 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff341a4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c152ba0 con 0x7ff33c0c69f0 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff33affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0ad2b0 0x7ff33c154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff33affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0ad2b0 0x7ff33c154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53088/0 (socket says 172.21.15.26:53088) 2024-03-06T07:25:15.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff33affd640 1 -- 172.21.15.26:0/4212345930 learned_addr learned my addr 172.21.15.26:0/4212345930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff33b7fe640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c8950 0x7ff33c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.735+0000 7ff33a7fc640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c69f0 0x7ff33c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff33affd640 1 -- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c69f0 msgr2=0x7ff33c0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff33affd640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c69f0 0x7ff33c0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff33affd640 1 -- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c8950 msgr2=0x7ff33c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff33affd640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c8950 0x7ff33c0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff33affd640 1 -- 172.21.15.26:0/4212345930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c159100 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff33a7fc640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c69f0 0x7ff33c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff33affd640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0ad2b0 0x7ff33c154260 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00e810 tx=0x7ff32c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff323fff640 1 -- 172.21.15.26:0/4212345930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00f040 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff323fff640 1 -- 172.21.15.26:0/4212345930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c004030 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c1eb760 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff323fff640 1 -- 172.21.15.26:0/4212345930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004400 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.736+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c1547a0 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.737+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c067910 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.737+0000 7ff323fff640 1 -- 172.21.15.26:0/4212345930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff32c00c6d0 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.738+0000 7ff323fff640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff32806b4d0 0x7ff32806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.738+0000 7ff323fff640 1 -- 172.21.15.26:0/4212345930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff32c089f70 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.738+0000 7ff33a7fc640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff32806b4d0 0x7ff32806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.738+0000 7ff33a7fc640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff32806b4d0 0x7ff32806d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff330036000 tx=0x7ff33003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.742+0000 7ff323fff640 1 -- 172.21.15.26:0/4212345930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff33c067910 con 0x7ff33c0ad2b0 2024-03-06T07:25:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:15.937+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff33c152d80 con 0x7ff33c0ad2b0 2024-03-06T07:25:16.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.760+0000 7ff323fff640 1 -- 172.21.15.26:0/4212345930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v131) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff33c152d80 con 0x7ff33c0ad2b0 2024-03-06T07:25:16.762 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff32806b4d0 msgr2=0x7ff32806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff32806b4d0 0x7ff32806d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff330036000 tx=0x7ff33003c040 comp rx=0 tx=0).stop 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0ad2b0 msgr2=0x7ff33c154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0ad2b0 0x7ff33c154260 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00e810 tx=0x7ff32c00ecd0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 shutdown_connections 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff32806b4d0 0x7ff32806d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c8950 0x7ff33c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c69f0 0x7ff33c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 --2- 172.21.15.26:0/4212345930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff33c0ad2b0 0x7ff33c154260 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 >> 172.21.15.26:0/4212345930 conn(0x7ff33c0cae70 msgr2=0x7ff33c0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 shutdown_connections 2024-03-06T07:25:16.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.763+0000 7ff341a4e640 1 -- 172.21.15.26:0/4212345930 wait complete. 2024-03-06T07:25:16.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:25:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.884+0000 7f46dda4e640 1 Processor -- start 2024-03-06T07:25:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46dda4e640 1 -- start start 2024-03-06T07:25:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46dda4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46d814f790 0x7f46d81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46dda4e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f46d8155a40 0x7f46d8155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46dda4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d80c28d0 con 0x7f46d80c1aa0 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46dda4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d80c2680 con 0x7f46d814f790 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46dda4e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d80c2bb0 con 0x7f46d8155a40 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46d6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46d814f790 0x7f46d81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46d67fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f46d80c1aa0 0x7f46d80c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47670/0 (socket says 172.21.15.26:47670) 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.885+0000 7f46d67fc640 1 -- 172.21.15.26:0/385300880 learned_addr learned my addr 172.21.15.26:0/385300880 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.886+0000 7f46d77fe640 1 --2- 172.21.15.26:0/385300880 >> v2:172.21.15.117:3300/0 conn(0x7f46d8155a40 0x7f46d8155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.886+0000 7f46d6ffd640 1 -- 172.21.15.26:0/385300880 >> v2:172.21.15.117:3300/0 conn(0x7f46d8155a40 msgr2=0x7f46d8155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.886+0000 7f46d6ffd640 1 --2- 172.21.15.26:0/385300880 >> v2:172.21.15.117:3300/0 conn(0x7f46d8155a40 0x7f46d8155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.886+0000 7f46d6ffd640 1 -- 172.21.15.26:0/385300880 >> v1:172.21.15.26:6789/0 conn(0x7f46d80c1aa0 legacy=0x7f46d80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.886+0000 7f46d6ffd640 1 -- 172.21.15.26:0/385300880 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80c2e90 con 0x7f46d814f790 2024-03-06T07:25:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.886+0000 7f46d5ffb640 1 -- 172.21.15.26:0/385300880 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747147910 0 0) 0x7f46d80c28d0 con 0x7f46d80c1aa0 2024-03-06T07:25:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.886+0000 7f46d6ffd640 1 --2- 172.21.15.26:0/385300880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d81529d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f46d806c960 tx=0x7f46c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8806383480c7d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46d5ffb640 1 -- 172.21.15.26:0/385300880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8035020 con 0x7f46d814f790 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46d5ffb640 1 -- 172.21.15.26:0/385300880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c8031830 con 0x7f46d814f790 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46d5ffb640 1 -- 172.21.15.26:0/385300880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8031c00 con 0x7f46d814f790 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 -- 172.21.15.26:0/385300880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 msgr2=0x7f46d81529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 --2- 172.21.15.26:0/385300880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d81529d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f46d806c960 tx=0x7f46c802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 -- 172.21.15.26:0/385300880 shutdown_connections 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 --2- 172.21.15.26:0/385300880 >> v2:172.21.15.117:3300/0 conn(0x7f46d8155a40 0x7f46d8155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 --2- 172.21.15.26:0/385300880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d81529d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 -- 172.21.15.26:0/385300880 >> 172.21.15.26:0/385300880 conn(0x7f46d80c6710 msgr2=0x7f46d80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 -- 172.21.15.26:0/385300880 shutdown_connections 2024-03-06T07:25:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.887+0000 7f46dda4e640 1 -- 172.21.15.26:0/385300880 wait complete. 2024-03-06T07:25:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.888+0000 7f46dda4e640 1 Processor -- start 2024-03-06T07:25:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46dda4e640 1 -- start start 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46dda4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d80c1aa0 0x7f46d8071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46dda4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d80c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46dda4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46d8155a40 0x7f46d80c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46dda4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80c0d60 con 0x7f46d814f790 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46dda4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8156390 con 0x7f46d80c1aa0 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46dda4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80c5e30 con 0x7f46d8155a40 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46d67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d80c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46d67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d80c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53106/0 (socket says 172.21.15.26:53106) 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.889+0000 7f46d67fc640 1 -- 172.21.15.26:0/2324447794 learned_addr learned my addr 172.21.15.26:0/2324447794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d77fe640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46d8155a40 0x7f46d80c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d6ffd640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d80c1aa0 0x7f46d8071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d67fc640 1 -- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46d8155a40 msgr2=0x7f46d80c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d67fc640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46d8155a40 0x7f46d80c3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d67fc640 1 -- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d80c1aa0 msgr2=0x7f46d8071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d67fc640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d80c1aa0 0x7f46d8071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d67fc640 1 -- 172.21.15.26:0/2324447794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80aadb0 con 0x7f46d814f790 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d6ffd640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d80c1aa0 0x7f46d8071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46d67fc640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d80c0730 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c4e0 tx=0x7f46cc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46bffff640 1 -- 172.21.15.26:0/2324447794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc018070 con 0x7f46d814f790 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46bffff640 1 -- 172.21.15.26:0/2324447794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc007500 con 0x7f46d814f790 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46bffff640 1 -- 172.21.15.26:0/2324447794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc004a20 con 0x7f46d814f790 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d81eb870 con 0x7f46d814f790 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.890+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d80702f0 con 0x7f46d814f790 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.891+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d804f330 con 0x7f46d814f790 2024-03-06T07:25:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.891+0000 7f46bffff640 1 -- 172.21.15.26:0/2324447794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46cc01b070 con 0x7f46d814f790 2024-03-06T07:25:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.892+0000 7f46bffff640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46c406b500 0x7f46c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.892+0000 7f46bffff640 1 -- 172.21.15.26:0/2324447794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f46cc089300 con 0x7f46d814f790 2024-03-06T07:25:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.892+0000 7f46d6ffd640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46c406b500 0x7f46c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.893+0000 7f46d6ffd640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46c406b500 0x7f46c406d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f46c8036000 tx=0x7f46c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:16.896+0000 7f46bffff640 1 -- 172.21.15.26:0/2324447794 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46cc058400 con 0x7f46d814f790 2024-03-06T07:25:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.142+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f46d80c1f40 con 0x7f46d814f790 2024-03-06T07:25:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.143+0000 7f46bffff640 1 -- 172.21.15.26:0/2324447794 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f46cc057d00 con 0x7f46d814f790 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.145 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:25:17.146 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:25:17.147 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.148 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.149 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.151 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:25:17.155 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:17.156 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:25:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46c406b500 msgr2=0x7f46c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46c406b500 0x7f46c406d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f46c8036000 tx=0x7f46c8008030 comp rx=0 tx=0).stop 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 msgr2=0x7f46d80c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d80c0730 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c4e0 tx=0x7f46cc00c9a0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 shutdown_connections 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46c406b500 0x7f46c406d9c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46d8155a40 0x7f46d80c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46d814f790 0x7f46d80c0730 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 --2- 172.21.15.26:0/2324447794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d80c1aa0 0x7f46d8071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 >> 172.21.15.26:0/2324447794 conn(0x7f46d80c6710 msgr2=0x7f46d814ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 shutdown_connections 2024-03-06T07:25:17.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.145+0000 7f46dda4e640 1 -- 172.21.15.26:0/2324447794 wait complete. 2024-03-06T07:25:17.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:17.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.260+0000 7f7b20666640 1 Processor -- start 2024-03-06T07:25:17.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.260+0000 7f7b20666640 1 -- start start 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b20666640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7b181550c0 0x7f7b181554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b20666640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 0x7f7b1814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b20666640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b180aa660 con 0x7f7b180aa180 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b20666640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b180c4610 con 0x7f7b181550c0 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b20666640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180aa940 con 0x7f7b181559e0 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b1ec03640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7b180aa180 0x7f7b180c5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37192/0 (socket says 172.21.15.26:37192) 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b1ec03640 1 -- 172.21.15.26:0/1214490219 learned_addr learned my addr 172.21.15.26:0/1214490219 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:17.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 0x7f7b1814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.261+0000 7f7b1e402640 1 --2- 172.21.15.26:0/1214490219 >> v2:172.21.15.117:3300/0 conn(0x7f7b181550c0 0x7f7b181554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931232695 0 0) 0x7f7b180aa660 con 0x7f7b180aa180 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b04003520 con 0x7f7b180aa180 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2805837986 0 0) 0x7f7b04003520 con 0x7f7b180aa180 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 msgr2=0x7f7b1814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 0x7f7b1814ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 >> v2:172.21.15.117:3300/0 conn(0x7f7b181550c0 msgr2=0x7f7b181554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 --2- 172.21.15.26:0/1214490219 >> v2:172.21.15.117:3300/0 conn(0x7f7b181550c0 0x7f7b181554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b180c4860 con 0x7f7b180aa180 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f7b100019f0 con 0x7f7b180aa180 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 >> v1:172.21.15.26:6789/0 conn(0x7f7b180aa180 legacy=0x7f7b180c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b040039b0 0x7f7b04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 0x7f7b040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180c4610 con 0x7f7b04004800 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180aa940 con 0x7f7b040039b0 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0400baa0 con 0x7f7b181559e0 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1e402640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b040039b0 0x7f7b04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1ec03640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 0x7f7b040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1dc01640 1 -- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 msgr2=0x7f7b040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 0x7f7b040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1dc01640 1 -- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b040039b0 msgr2=0x7f7b04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b040039b0 0x7f7b04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1dc01640 1 -- 172.21.15.26:0/1214490219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04003520 con 0x7f7b04004800 2024-03-06T07:25:17.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.262+0000 7f7b1ec03640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181559e0 0x7f7b040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:17.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.263+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b04004c80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800aa90 tx=0x7f7b04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04de1b7d1072939 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.263+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0803a070 con 0x7f7b04004800 2024-03-06T07:25:17.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.263+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08035440 con 0x7f7b04004800 2024-03-06T07:25:17.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.263+0000 7f7b1d400640 1 -- 172.21.15.26:0/1214490219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800b040 con 0x7f7b04004800 2024-03-06T07:25:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.264+0000 7f7b20666640 1 -- 172.21.15.26:0/1214490219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 msgr2=0x7f7b04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.264+0000 7f7b20666640 1 --2- 172.21.15.26:0/1214490219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b04004c80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800aa90 tx=0x7f7b04005830 comp rx=0 tx=0).stop 2024-03-06T07:25:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.264+0000 7f7b1ec03640 1 -- 172.21.15.26:0/1214490219 reap_dead start 2024-03-06T07:25:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.264+0000 7f7b20666640 1 -- 172.21.15.26:0/1214490219 shutdown_connections 2024-03-06T07:25:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.264+0000 7f7b20666640 1 -- 172.21.15.26:0/1214490219 >> 172.21.15.26:0/1214490219 conn(0x7f7b180c6710 msgr2=0x7f7b180c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.264+0000 7f7b20666640 1 -- 172.21.15.26:0/1214490219 shutdown_connections 2024-03-06T07:25:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.264+0000 7f7b20666640 1 -- 172.21.15.26:0/1214490219 wait complete. 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.265+0000 7f7b20666640 1 Processor -- start 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.265+0000 7f7b20666640 1 -- start start 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b20666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b18150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b20666640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181469e0 0x7f7b1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b20666640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b181ea7f0 0x7f7b18147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b20666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180c5770 con 0x7f7b04004800 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b20666640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180c1520 con 0x7f7b181ea7f0 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b20666640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180c58f0 con 0x7f7b181469e0 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b18150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b18150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37290/0 (socket says 172.21.15.26:37290) 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 -- 172.21.15.26:0/3643376816 learned_addr learned my addr 172.21.15.26:0/3643376816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181469e0 0x7f7b1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1ec03640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b181ea7f0 0x7f7b18147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 -- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181469e0 msgr2=0x7f7b1806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181469e0 0x7f7b1806e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 -- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b181ea7f0 msgr2=0x7f7b18147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b181ea7f0 0x7f7b18147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 -- 172.21.15.26:0/3643376816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1806e860 con 0x7f7b04004800 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1ec03640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b181ea7f0 0x7f7b18147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.266+0000 7f7b1e402640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b18150bc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7b14011ad0 tx=0x7f7b14011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.267+0000 7f7b0effd640 1 -- 172.21.15.26:0/3643376816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400fb50 con 0x7f7b04004800 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.267+0000 7f7b0effd640 1 -- 172.21.15.26:0/3643376816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1400fcf0 con 0x7f7b04004800 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.267+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b181eb720 con 0x7f7b04004800 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.267+0000 7f7b0effd640 1 -- 172.21.15.26:0/3643376816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400fea0 con 0x7f7b04004800 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.267+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b181eb440 con 0x7f7b04004800 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.268+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1804f330 con 0x7f7b04004800 2024-03-06T07:25:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.268+0000 7f7b0effd640 1 -- 172.21.15.26:0/3643376816 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b14012040 con 0x7f7b04004800 2024-03-06T07:25:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.269+0000 7f7b0effd640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.269+0000 7f7b0effd640 1 -- 172.21.15.26:0/3643376816 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7b1408a630 con 0x7f7b04004800 2024-03-06T07:25:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.269+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.269+0000 7f7b1dc01640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7afc06b4d0 0x7f7afc06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7b08037000 tx=0x7f7b08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.273+0000 7f7b0effd640 1 -- 172.21.15.26:0/3643376816 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b14019030 con 0x7f7b04004800 2024-03-06T07:25:17.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.457+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b18068450 con 0x7f7b04004800 2024-03-06T07:25:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.458+0000 7f7b0effd640 1 -- 172.21.15.26:0/3643376816 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+13186 (secure 0 0 0) 0x7f7b14059670 con 0x7f7b04004800 2024-03-06T07:25:17.460 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:17.460 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":131,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:16.759745+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7afc06b4d0 msgr2=0x7f7afc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7afc06b4d0 0x7f7afc06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7b08037000 tx=0x7f7b08008040 comp rx=0 tx=0).stop 2024-03-06T07:25:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 msgr2=0x7f7b18150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b18150bc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7b14011ad0 tx=0x7f7b14011f90 comp rx=0 tx=0).stop 2024-03-06T07:25:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 shutdown_connections 2024-03-06T07:25:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b181ea7f0 0x7f7b18147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b181469e0 0x7f7b1806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 --2- 172.21.15.26:0/3643376816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b04004800 0x7f7b18150bc0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 >> 172.21.15.26:0/3643376816 conn(0x7f7b180c6710 msgr2=0x7f7b1814e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 shutdown_connections 2024-03-06T07:25:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.460+0000 7f7b20666640 1 -- 172.21.15.26:0/3643376816 wait complete. 2024-03-06T07:25:17.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:17.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.576+0000 7f86c87e9640 1 Processor -- start 2024-03-06T07:25:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.576+0000 7f86c87e9640 1 -- start start 2024-03-06T07:25:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c87e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c00c5370 0x7f86c00c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c87e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f86c0155530 0x7f86c01559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c87e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c0152dd0 con 0x7f86c01529f0 2024-03-06T07:25:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c87e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c0159710 con 0x7f86c0155530 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c87e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00c68b0 con 0x7f86c00c5370 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c5d84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c00c5370 0x7f86c00c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c6d86640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f86c0155530 0x7f86c01559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c5d84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c00c5370 0x7f86c00c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:34470/0 (socket says 172.21.15.26:34470) 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c6d86640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f86c0155530 0x7f86c01559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55042/0 (socket says 172.21.15.26:55042) 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.577+0000 7f86c5d84640 1 -- 172.21.15.26:0/566313401 learned_addr learned my addr 172.21.15.26:0/566313401 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.578+0000 7f86c5d84640 1 -- 172.21.15.26:0/566313401 >> v2:172.21.15.117:3300/0 conn(0x7f86c0155530 msgr2=0x7f86c01559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.578+0000 7f86c5d84640 1 --2- 172.21.15.26:0/566313401 >> v2:172.21.15.117:3300/0 conn(0x7f86c0155530 0x7f86c01559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.578+0000 7f86c5d84640 1 -- 172.21.15.26:0/566313401 >> v1:172.21.15.117:6789/0 conn(0x7f86c01529f0 legacy=0x7f86c00c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.578+0000 7f86c5d84640 1 -- 172.21.15.26:0/566313401 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0158b00 con 0x7f86c00c5370 2024-03-06T07:25:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.578+0000 7f86c6d86640 1 --2- 172.21.15.26:0/566313401 >> v2:172.21.15.117:3300/0 conn(0x7f86c0155530 0x7f86c01559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:17.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c5d84640 1 --2- 172.21.15.26:0/566313401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c00c5370 0x7f86c00c57d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009870 tx=0x7f86b402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434febc35ac03b02 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c5583640 1 -- 172.21.15.26:0/566313401 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4035020 con 0x7f86c00c5370 2024-03-06T07:25:17.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c5583640 1 -- 172.21.15.26:0/566313401 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b4031640 con 0x7f86c00c5370 2024-03-06T07:25:17.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c5583640 1 -- 172.21.15.26:0/566313401 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b402d040 con 0x7f86c00c5370 2024-03-06T07:25:17.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 -- 172.21.15.26:0/566313401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c00c5370 msgr2=0x7f86c00c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 --2- 172.21.15.26:0/566313401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c00c5370 0x7f86c00c57d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009870 tx=0x7f86b402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 -- 172.21.15.26:0/566313401 shutdown_connections 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 --2- 172.21.15.26:0/566313401 >> v2:172.21.15.117:3300/0 conn(0x7f86c0155530 0x7f86c01559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 --2- 172.21.15.26:0/566313401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c00c5370 0x7f86c00c57d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 -- 172.21.15.26:0/566313401 >> 172.21.15.26:0/566313401 conn(0x7f86c00ac580 msgr2=0x7f86c00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 -- 172.21.15.26:0/566313401 shutdown_connections 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.579+0000 7f86c87e9640 1 -- 172.21.15.26:0/566313401 wait complete. 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.580+0000 7f86c87e9640 1 Processor -- start 2024-03-06T07:25:17.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c87e9640 1 -- start start 2024-03-06T07:25:17.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c87e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c00c5370 0x7f86c01eb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c87e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c01529f0 0x7f86c0145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c87e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86c0155530 0x7f86c0152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c87e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c01590c0 con 0x7f86c0155530 2024-03-06T07:25:17.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c87e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00c9000 con 0x7f86c00c5370 2024-03-06T07:25:17.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c87e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c01593a0 con 0x7f86c01529f0 2024-03-06T07:25:17.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c6d86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86c0155530 0x7f86c0152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c6d86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86c0155530 0x7f86c0152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37306/0 (socket says 172.21.15.26:37306) 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c6d86640 1 -- 172.21.15.26:0/579641985 learned_addr learned my addr 172.21.15.26:0/579641985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c6585640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c00c5370 0x7f86c01eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.581+0000 7f86c5d84640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c01529f0 0x7f86c0145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c6d86640 1 -- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c01529f0 msgr2=0x7f86c0145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c6d86640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c01529f0 0x7f86c0145db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c6d86640 1 -- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c00c5370 msgr2=0x7f86c01eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c6d86640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c00c5370 0x7f86c01eb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c6d86640 1 -- 172.21.15.26:0/579641985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c01eb8c0 con 0x7f86c0155530 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c6d86640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86c0155530 0x7f86c0152fc0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f86b000e880 tx=0x7f86b000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86beffd640 1 -- 172.21.15.26:0/579641985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0017070 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86beffd640 1 -- 172.21.15.26:0/579641985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b0004020 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86beffd640 1 -- 172.21.15.26:0/579641985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00043f0 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c01462f0 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.582+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c01543d0 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.583+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c004f330 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.583+0000 7f86beffd640 1 -- 172.21.15.26:0/579641985 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86b000c940 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.584+0000 7f86beffd640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86ac06b500 0x7f86ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.584+0000 7f86beffd640 1 -- 172.21.15.26:0/579641985 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f86b0089d40 con 0x7f86c0155530 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.584+0000 7f86c6585640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86ac06b500 0x7f86ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.585+0000 7f86c6585640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86ac06b500 0x7f86ac06d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86b8001f10 tx=0x7f86b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.588+0000 7f86beffd640 1 -- 172.21.15.26:0/579641985 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f86b0058730 con 0x7f86c0155530 2024-03-06T07:25:17.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.766+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86c014f140 con 0x7f86c0155530 2024-03-06T07:25:17.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.766+0000 7f86beffd640 1 -- 172.21.15.26:0/579641985 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86b005c7d0 con 0x7f86c0155530 2024-03-06T07:25:17.768 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:25:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86ac06b500 msgr2=0x7f86ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86ac06b500 0x7f86ac06d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86b8001f10 tx=0x7f86b8009040 comp rx=0 tx=0).stop 2024-03-06T07:25:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86c0155530 msgr2=0x7f86c0152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86c0155530 0x7f86c0152fc0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f86b000e880 tx=0x7f86b000ed40 comp rx=0 tx=0).stop 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 shutdown_connections 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86ac06b500 0x7f86ac06d9c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86c0155530 0x7f86c0152fc0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86c01529f0 0x7f86c0145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 --2- 172.21.15.26:0/579641985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c00c5370 0x7f86c01eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 >> 172.21.15.26:0/579641985 conn(0x7f86c00ac580 msgr2=0x7f86c006cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 shutdown_connections 2024-03-06T07:25:17.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.768+0000 7f86c87e9640 1 -- 172.21.15.26:0/579641985 wait complete. 2024-03-06T07:25:17.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:25:17.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.881+0000 7f8c919c9640 1 Processor -- start 2024-03-06T07:25:17.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.881+0000 7f8c919c9640 1 -- start start 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c919c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c8c1506b0 0x7f8c8c153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c919c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c070aa0 con 0x7f8c8c0acc00 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c919c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c070d80 con 0x7f8c8c156ea0 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c919c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c156450 con 0x7f8c8c1506b0 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c8affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c8c1506b0 0x7f8c8c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c8b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8c8c0acc00 0x7f8c8c0c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37206/0 (socket says 172.21.15.26:37206) 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c8b7fe640 1 -- 172.21.15.26:0/266598495 learned_addr learned my addr 172.21.15.26:0/266598495 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c8affd640 1 -- 172.21.15.26:0/266598495 >> v1:172.21.15.117:6789/0 conn(0x7f8c8c156ea0 legacy=0x7f8c8c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c8affd640 1 -- 172.21.15.26:0/266598495 >> v1:172.21.15.26:6789/0 conn(0x7f8c8c0acc00 legacy=0x7f8c8c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.882+0000 7f8c8affd640 1 -- 172.21.15.26:0/266598495 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c1566a0 con 0x7f8c8c1506b0 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.883+0000 7f8c8affd640 1 --2- 172.21.15.26:0/266598495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c1506b0 0x7f8c8c153990 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8c840097a0 tx=0x7f8c8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91188f0e024fe46c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.883+0000 7f8c89ffb640 1 -- 172.21.15.26:0/266598495 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84035020 con 0x7f8c8c1506b0 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.883+0000 7f8c89ffb640 1 -- 172.21.15.26:0/266598495 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c84031830 con 0x7f8c8c1506b0 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.883+0000 7f8c89ffb640 1 -- 172.21.15.26:0/266598495 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84031c00 con 0x7f8c8c1506b0 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.884+0000 7f8c919c9640 1 -- 172.21.15.26:0/266598495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c1506b0 msgr2=0x7f8c8c153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.884+0000 7f8c919c9640 1 --2- 172.21.15.26:0/266598495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c1506b0 0x7f8c8c153990 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8c840097a0 tx=0x7f8c8402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.884+0000 7f8c919c9640 1 -- 172.21.15.26:0/266598495 shutdown_connections 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.884+0000 7f8c919c9640 1 --2- 172.21.15.26:0/266598495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c1506b0 0x7f8c8c153990 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.884+0000 7f8c919c9640 1 -- 172.21.15.26:0/266598495 >> 172.21.15.26:0/266598495 conn(0x7f8c8c06b1a0 msgr2=0x7f8c8c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.884+0000 7f8c919c9640 1 -- 172.21.15.26:0/266598495 shutdown_connections 2024-03-06T07:25:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.884+0000 7f8c919c9640 1 -- 172.21.15.26:0/266598495 wait complete. 2024-03-06T07:25:17.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.885+0000 7f8c919c9640 1 Processor -- start 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.885+0000 7f8c919c9640 1 -- start start 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c919c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c0acc00 0x7f8c8c154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c919c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c8c1506b0 0x7f8c8c149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c919c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c8c156ea0 0x7f8c8c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c919c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0ad050 con 0x7f8c8c0acc00 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c919c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c156a10 con 0x7f8c8c1506b0 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c919c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0ad1d0 con 0x7f8c8c156ea0 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c0acc00 0x7f8c8c154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c0acc00 0x7f8c8c154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37330/0 (socket says 172.21.15.26:37330) 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 -- 172.21.15.26:0/2576927366 learned_addr learned my addr 172.21.15.26:0/2576927366 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8b7fe640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c8c156ea0 0x7f8c8c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8a7fc640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c8c1506b0 0x7f8c8c149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 -- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c8c156ea0 msgr2=0x7f8c8c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c8c156ea0 0x7f8c8c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 -- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c8c1506b0 msgr2=0x7f8c8c149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c8c1506b0 0x7f8c8c149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 -- 172.21.15.26:0/2576927366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c152c00 con 0x7f8c8c0acc00 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8b7fe640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c8c156ea0 0x7f8c8c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c8affd640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c0acc00 0x7f8c8c154c20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8c840098c0 tx=0x7f8c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c73fff640 1 -- 172.21.15.26:0/2576927366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8403ba80 con 0x7f8c8c0acc00 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c73fff640 1 -- 172.21.15.26:0/2576927366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8403bc40 con 0x7f8c8c0acc00 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c1514c0 con 0x7f8c8c0acc00 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.886+0000 7f8c73fff640 1 -- 172.21.15.26:0/2576927366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840453f0 con 0x7f8c8c0acc00 2024-03-06T07:25:17.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.887+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c155900 con 0x7f8c8c0acc00 2024-03-06T07:25:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.888+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c04f330 con 0x7f8c8c0acc00 2024-03-06T07:25:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.888+0000 7f8c73fff640 1 -- 172.21.15.26:0/2576927366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c8404c060 con 0x7f8c8c0acc00 2024-03-06T07:25:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.888+0000 7f8c73fff640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c7806b4d0 0x7f8c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.888+0000 7f8c73fff640 1 -- 172.21.15.26:0/2576927366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8c840abde0 con 0x7f8c8c0acc00 2024-03-06T07:25:17.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.888+0000 7f8c8a7fc640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c7806b4d0 0x7f8c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.889+0000 7f8c8a7fc640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c7806b4d0 0x7f8c7806d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006fd0 tx=0x7f8c80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:17.893+0000 7f8c73fff640 1 -- 172.21.15.26:0/2576927366 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c84040070 con 0x7f8c8c0acc00 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.075+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8c8c0700a0 con 0x7f8c8c0acc00 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.075+0000 7f8c73fff640 1 -- 172.21.15.26:0/2576927366 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8c8407aeb0 con 0x7f8c8c0acc00 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:25:18.077 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c7806b4d0 msgr2=0x7f8c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c7806b4d0 0x7f8c7806d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006fd0 tx=0x7f8c80008040 comp rx=0 tx=0).stop 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c0acc00 msgr2=0x7f8c8c154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c0acc00 0x7f8c8c154c20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8c840098c0 tx=0x7f8c840046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 shutdown_connections 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8c7806b4d0 0x7f8c7806d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c8c156ea0 0x7f8c8c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c8c1506b0 0x7f8c8c149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 --2- 172.21.15.26:0/2576927366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c8c0acc00 0x7f8c8c154c20 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 >> 172.21.15.26:0/2576927366 conn(0x7f8c8c06b1a0 msgr2=0x7f8c8c0ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 shutdown_connections 2024-03-06T07:25:18.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.077+0000 7f8c919c9640 1 -- 172.21.15.26:0/2576927366 wait complete. 2024-03-06T07:25:18.089 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:25:18.090 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:25:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.193+0000 7f093cd65640 1 Processor -- start 2024-03-06T07:25:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.193+0000 7f093cd65640 1 -- start start 2024-03-06T07:25:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f093cd65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09380c5e40 0x7f0938153940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f093cd65640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0938072fc0 0x7f09380724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f093cd65640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0938153e80 con 0x7f0938072a10 2024-03-06T07:25:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f093cd65640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09380c6220 con 0x7f0938072fc0 2024-03-06T07:25:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f093cd65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0938070210 con 0x7f09380c5e40 2024-03-06T07:25:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f0935d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0938072fc0 0x7f09380724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f0935d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0938072fc0 0x7f09380724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37338/0 (socket says 172.21.15.26:37338) 2024-03-06T07:25:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f0935d9b640 1 -- 172.21.15.26:0/3888148866 learned_addr learned my addr 172.21.15.26:0/3888148866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f093659c640 1 --2- 172.21.15.26:0/3888148866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09380c5e40 0x7f0938153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f0935d9b640 1 -- 172.21.15.26:0/3888148866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09380c5e40 msgr2=0x7f0938153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f0935d9b640 1 --2- 172.21.15.26:0/3888148866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09380c5e40 0x7f0938153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f0935d9b640 1 -- 172.21.15.26:0/3888148866 >> v1:172.21.15.117:6789/0 conn(0x7f0938072a10 legacy=0x7f09380c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f0935d9b640 1 -- 172.21.15.26:0/3888148866 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380704d0 con 0x7f0938072fc0 2024-03-06T07:25:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.194+0000 7f093659c640 1 --2- 172.21.15.26:0/3888148866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09380c5e40 0x7f0938153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.195+0000 7f0935d9b640 1 --2- 172.21.15.26:0/3888148866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0938072fc0 0x7f09380724d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0920009870 tx=0x7f092002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622b4de78963a85c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.195+0000 7f093559a640 1 -- 172.21.15.26:0/3888148866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920035020 con 0x7f0938072fc0 2024-03-06T07:25:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.195+0000 7f093559a640 1 -- 172.21.15.26:0/3888148866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0920031640 con 0x7f0938072fc0 2024-03-06T07:25:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.195+0000 7f093559a640 1 -- 172.21.15.26:0/3888148866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092002d040 con 0x7f0938072fc0 2024-03-06T07:25:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 -- 172.21.15.26:0/3888148866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0938072fc0 msgr2=0x7f09380724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 --2- 172.21.15.26:0/3888148866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0938072fc0 0x7f09380724d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0920009870 tx=0x7f092002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 -- 172.21.15.26:0/3888148866 shutdown_connections 2024-03-06T07:25:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 --2- 172.21.15.26:0/3888148866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0938072fc0 0x7f09380724d0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 --2- 172.21.15.26:0/3888148866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09380c5e40 0x7f0938153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 -- 172.21.15.26:0/3888148866 >> 172.21.15.26:0/3888148866 conn(0x7f09380c88f0 msgr2=0x7f09380c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 -- 172.21.15.26:0/3888148866 shutdown_connections 2024-03-06T07:25:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.196+0000 7f093cd65640 1 -- 172.21.15.26:0/3888148866 wait complete. 2024-03-06T07:25:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.197+0000 7f093cd65640 1 Processor -- start 2024-03-06T07:25:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.197+0000 7f093cd65640 1 -- start start 2024-03-06T07:25:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093cd65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0938072a10 0x7f09380adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093cd65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0938072fc0 0x7f09380c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093cd65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09380c5e40 0x7f09380c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093cd65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093806fbd0 con 0x7f09380c5e40 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093cd65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093806f8f0 con 0x7f0938072fc0 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093cd65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09380c4de0 con 0x7f0938072a10 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09380c5e40 0x7f09380c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09380c5e40 0x7f09380c18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37346/0 (socket says 172.21.15.26:37346) 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 -- 172.21.15.26:0/3321078991 learned_addr learned my addr 172.21.15.26:0/3321078991 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093659c640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0938072a10 0x7f09380adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0935d9b640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0938072fc0 0x7f09380c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 -- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0938072a10 msgr2=0x7f09380adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0938072a10 0x7f09380adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 -- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0938072fc0 msgr2=0x7f09380c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0938072fc0 0x7f09380c1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f0936d9d640 1 -- 172.21.15.26:0/3321078991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09381e91a0 con 0x7f09380c5e40 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.198+0000 7f093659c640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0938072a10 0x7f09380adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.199+0000 7f0936d9d640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09380c5e40 0x7f09380c18b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f092800c760 tx=0x7f092800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.199+0000 7f092effd640 1 -- 172.21.15.26:0/3321078991 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928003b90 con 0x7f09380c5e40 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.199+0000 7f092effd640 1 -- 172.21.15.26:0/3321078991 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928003d30 con 0x7f09380c5e40 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.199+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09380bee00 con 0x7f09380c5e40 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.199+0000 7f092effd640 1 -- 172.21.15.26:0/3321078991 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800aa90 con 0x7f09380c5e40 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.199+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09380ae070 con 0x7f09380c5e40 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.200+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09380bebf0 con 0x7f09380c5e40 2024-03-06T07:25:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.200+0000 7f092effd640 1 -- 172.21.15.26:0/3321078991 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f092800ac30 con 0x7f09380c5e40 2024-03-06T07:25:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.200+0000 7f092effd640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.201+0000 7f092effd640 1 -- 172.21.15.26:0/3321078991 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0928089f60 con 0x7f09380c5e40 2024-03-06T07:25:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.201+0000 7f093659c640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.201+0000 7f093659c640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f09380ae350 tx=0x7f0930008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.205+0000 7f092effd640 1 -- 172.21.15.26:0/3321078991 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0928058f90 con 0x7f09380c5e40 2024-03-06T07:25:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.387+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f093804f330 con 0x7f09380c5e40 2024-03-06T07:25:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.772+0000 7f092effd640 1 -- 172.21.15.26:0/3321078991 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0928058890 con 0x7f09380c5e40 2024-03-06T07:25:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:25:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 msgr2=0x7f091806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f09380ae350 tx=0x7f0930008040 comp rx=0 tx=0).stop 2024-03-06T07:25:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09380c5e40 msgr2=0x7f09380c18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09380c5e40 0x7f09380c18b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f092800c760 tx=0x7f092800cc20 comp rx=0 tx=0).stop 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 shutdown_connections 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f091806b4d0 0x7f091806d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09380c5e40 0x7f09380c18b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0938072fc0 0x7f09380c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 --2- 172.21.15.26:0/3321078991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0938072a10 0x7f09380adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 >> 172.21.15.26:0/3321078991 conn(0x7f09380c88f0 msgr2=0x7f09381562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 shutdown_connections 2024-03-06T07:25:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.774+0000 7f093cd65640 1 -- 172.21.15.26:0/3321078991 wait complete. 2024-03-06T07:25:18.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:25:18.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.898+0000 7f862efc9640 1 Processor -- start 2024-03-06T07:25:18.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.898+0000 7f862efc9640 1 -- start start 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.899+0000 7f862efc9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86280c8ff0 0x7f86280c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.899+0000 7f862efc9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280b04b0 con 0x7f86280b00d0 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.899+0000 7f862efc9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862806d210 con 0x7f862806bff0 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.899+0000 7f862efc9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8628156160 con 0x7f86280c8ff0 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.899+0000 7f862d566640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86280c8ff0 0x7f86280c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.899+0000 7f862cd65640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86280b00d0 0x7f862806b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37208/0 (socket says 172.21.15.26:37208) 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.899+0000 7f862cd65640 1 -- 172.21.15.26:0/2299563488 learned_addr learned my addr 172.21.15.26:0/2299563488 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.900+0000 7f862d566640 1 -- 172.21.15.26:0/2299563488 >> v1:172.21.15.117:6789/0 conn(0x7f862806bff0 legacy=0x7f86280c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.900+0000 7f862d566640 1 -- 172.21.15.26:0/2299563488 >> v1:172.21.15.26:6789/0 conn(0x7f86280b00d0 legacy=0x7f862806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.900+0000 7f862d566640 1 -- 172.21.15.26:0/2299563488 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628155e80 con 0x7f86280c8ff0 2024-03-06T07:25:18.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.900+0000 7f862d566640 1 --2- 172.21.15.26:0/2299563488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280c8ff0 0x7f86280c7920 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f86281562e0 tx=0x7f861802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7697a90f635b8edf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.900+0000 7f86277fe640 1 -- 172.21.15.26:0/2299563488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618035020 con 0x7f86280c8ff0 2024-03-06T07:25:18.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.900+0000 7f86277fe640 1 -- 172.21.15.26:0/2299563488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618031830 con 0x7f86280c8ff0 2024-03-06T07:25:18.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.900+0000 7f86277fe640 1 -- 172.21.15.26:0/2299563488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618031c00 con 0x7f86280c8ff0 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.901+0000 7f862efc9640 1 -- 172.21.15.26:0/2299563488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280c8ff0 msgr2=0x7f86280c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.901+0000 7f862efc9640 1 --2- 172.21.15.26:0/2299563488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280c8ff0 0x7f86280c7920 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f86281562e0 tx=0x7f861802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.901+0000 7f862efc9640 1 -- 172.21.15.26:0/2299563488 shutdown_connections 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.901+0000 7f862efc9640 1 --2- 172.21.15.26:0/2299563488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280c8ff0 0x7f86280c7920 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.901+0000 7f862efc9640 1 -- 172.21.15.26:0/2299563488 >> 172.21.15.26:0/2299563488 conn(0x7f86280ac810 msgr2=0x7f86280acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.901+0000 7f862efc9640 1 -- 172.21.15.26:0/2299563488 shutdown_connections 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.901+0000 7f862efc9640 1 -- 172.21.15.26:0/2299563488 wait complete. 2024-03-06T07:25:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.902+0000 7f862efc9640 1 Processor -- start 2024-03-06T07:25:18.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.902+0000 7f862efc9640 1 -- start start 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862efc9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f862806bff0 0x7f8628159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862efc9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b00d0 0x7f86281ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862efc9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86280c8ff0 0x7f86280c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862efc9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8628152c50 con 0x7f86280b00d0 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862efc9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c5710 con 0x7f862806bff0 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862efc9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c5ba0 con 0x7f86280c8ff0 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b00d0 0x7f86281ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b00d0 0x7f86281ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37368/0 (socket says 172.21.15.26:37368) 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 -- 172.21.15.26:0/2032627389 learned_addr learned my addr 172.21.15.26:0/2032627389 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:18.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 -- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86280c8ff0 msgr2=0x7f86280c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862d566640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86280c8ff0 0x7f86280c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f862cd65640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f862806bff0 0x7f8628159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86280c8ff0 0x7f86280c2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 -- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f862806bff0 msgr2=0x7f8628159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f862806bff0 0x7f8628159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.903+0000 7f8627fff640 1 -- 172.21.15.26:0/2032627389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280c1d30 con 0x7f86280b00d0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.904+0000 7f8627fff640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b00d0 0x7f86281ee3e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b410 tx=0x7f861c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.904+0000 7f86257fa640 1 -- 172.21.15.26:0/2032627389 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c015020 con 0x7f86280b00d0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.904+0000 7f86257fa640 1 -- 172.21.15.26:0/2032627389 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c011830 con 0x7f86280b00d0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.904+0000 7f86257fa640 1 -- 172.21.15.26:0/2032627389 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c011c20 con 0x7f86280b00d0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.904+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86280c2010 con 0x7f86280b00d0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.904+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862814ff80 con 0x7f86280b00d0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.905+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862804f330 con 0x7f86280b00d0 2024-03-06T07:25:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.906+0000 7f86257fa640 1 -- 172.21.15.26:0/2032627389 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f861c018020 con 0x7f86280b00d0 2024-03-06T07:25:18.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.906+0000 7f86257fa640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f861406b500 0x7f861406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.906+0000 7f86257fa640 1 -- 172.21.15.26:0/2032627389 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f861c08a1d0 con 0x7f86280b00d0 2024-03-06T07:25:18.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.906+0000 7f862cd65640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f861406b500 0x7f861406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.907+0000 7f862cd65640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f861406b500 0x7f861406d9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f86200012e0 tx=0x7f8620009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:18.910+0000 7f86257fa640 1 -- 172.21.15.26:0/2032627389 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f861c01f070 con 0x7f86280b00d0 2024-03-06T07:25:19.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.090+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f862806ce50 con 0x7f86280b00d0 2024-03-06T07:25:19.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.777+0000 7f86257fa640 1 -- 172.21.15.26:0/2032627389 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) v1 ==== 170+0+0 (secure 0 0 0) 0x7f861c059210 con 0x7f86280b00d0 2024-03-06T07:25:19.779 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:25:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f861406b500 msgr2=0x7f861406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f861406b500 0x7f861406d9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f86200012e0 tx=0x7f8620009040 comp rx=0 tx=0).stop 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b00d0 msgr2=0x7f86281ee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b00d0 0x7f86281ee3e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b410 tx=0x7f861c00b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 shutdown_connections 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f861406b500 0x7f861406d9c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86280c8ff0 0x7f86280c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b00d0 0x7f86281ee3e0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 --2- 172.21.15.26:0/2032627389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f862806bff0 0x7f8628159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 >> 172.21.15.26:0/2032627389 conn(0x7f86280ac810 msgr2=0x7f86280ab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 shutdown_connections 2024-03-06T07:25:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.781+0000 7f862efc9640 1 -- 172.21.15.26:0/2032627389 wait complete. 2024-03-06T07:25:19.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:25:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.914+0000 7fada3113640 1 Processor -- start 2024-03-06T07:25:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fada3113640 1 -- start start 2024-03-06T07:25:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fada3113640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad9c152880 0x7fad9c0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fada3113640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fad9c0a7cd0 0x7fad9c0c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fada3113640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c155f40 con 0x7fad9c0c6710 2024-03-06T07:25:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fada3113640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c072510 con 0x7fad9c152880 2024-03-06T07:25:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fada3113640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c072db0 con 0x7fad9c0a7cd0 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fada0eaf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad9c152880 0x7fad9c0c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fad9bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fad9c0c6710 0x7fad9c0c1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37220/0 (socket says 172.21.15.26:37220) 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.915+0000 7fad9bfff640 1 -- 172.21.15.26:0/1067089242 learned_addr learned my addr 172.21.15.26:0/1067089242 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fada16b0640 1 --2- 172.21.15.26:0/1067089242 >> v2:172.21.15.117:3300/0 conn(0x7fad9c0a7cd0 0x7fad9c0c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fada0eaf640 1 -- 172.21.15.26:0/1067089242 >> v2:172.21.15.117:3300/0 conn(0x7fad9c0a7cd0 msgr2=0x7fad9c0c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fada0eaf640 1 --2- 172.21.15.26:0/1067089242 >> v2:172.21.15.117:3300/0 conn(0x7fad9c0a7cd0 0x7fad9c0c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fada0eaf640 1 -- 172.21.15.26:0/1067089242 >> v1:172.21.15.26:6789/0 conn(0x7fad9c0c6710 legacy=0x7fad9c0c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fada0eaf640 1 -- 172.21.15.26:0/1067089242 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c072760 con 0x7fad9c152880 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fad9b7fe640 1 -- 172.21.15.26:0/1067089242 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288226326 0 0) 0x7fad9c155f40 con 0x7fad9c0c6710 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fada16b0640 1 --2- 172.21.15.26:0/1067089242 >> v2:172.21.15.117:3300/0 conn(0x7fad9c0a7cd0 0x7fad9c0c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.916+0000 7fada0eaf640 1 --2- 172.21.15.26:0/1067089242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c152880 0x7fad9c0c3210 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fad9c072690 tx=0x7fad9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27498ab27d3c1ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fad9b7fe640 1 -- 172.21.15.26:0/1067089242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad900315a0 con 0x7fad9c152880 2024-03-06T07:25:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fad9b7fe640 1 -- 172.21.15.26:0/1067089242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90037070 con 0x7fad9c152880 2024-03-06T07:25:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fad9b7fe640 1 -- 172.21.15.26:0/1067089242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90031960 con 0x7fad9c152880 2024-03-06T07:25:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 -- 172.21.15.26:0/1067089242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c152880 msgr2=0x7fad9c0c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 --2- 172.21.15.26:0/1067089242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c152880 0x7fad9c0c3210 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fad9c072690 tx=0x7fad9002b840 comp rx=0 tx=0).stop 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 -- 172.21.15.26:0/1067089242 shutdown_connections 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 --2- 172.21.15.26:0/1067089242 >> v2:172.21.15.117:3300/0 conn(0x7fad9c0a7cd0 0x7fad9c0c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 --2- 172.21.15.26:0/1067089242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c152880 0x7fad9c0c3210 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 -- 172.21.15.26:0/1067089242 >> 172.21.15.26:0/1067089242 conn(0x7fad9c0aa5c0 msgr2=0x7fad9c0aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 -- 172.21.15.26:0/1067089242 shutdown_connections 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.917+0000 7fada3113640 1 -- 172.21.15.26:0/1067089242 wait complete. 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.918+0000 7fada3113640 1 Processor -- start 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.918+0000 7fada3113640 1 -- start start 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada3113640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c0a7cd0 0x7fad9c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada3113640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c0c6710 0x7fad9c14afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada3113640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9c152880 0x7fad9c14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada3113640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c155b80 con 0x7fad9c0c6710 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada3113640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0c2ad0 con 0x7fad9c0a7cd0 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada3113640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c155d00 con 0x7fad9c152880 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fad9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c0c6710 0x7fad9c14afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fad9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c0c6710 0x7fad9c14afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37396/0 (socket says 172.21.15.26:37396) 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fad9bfff640 1 -- 172.21.15.26:0/3676096622 learned_addr learned my addr 172.21.15.26:0/3676096622 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada0eaf640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c0a7cd0 0x7fad9c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.919+0000 7fada16b0640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9c152880 0x7fad9c14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad9bfff640 1 -- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9c152880 msgr2=0x7fad9c14cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad9bfff640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9c152880 0x7fad9c14cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad9bfff640 1 -- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c0a7cd0 msgr2=0x7fad9c1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad9bfff640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c0a7cd0 0x7fad9c1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad9bfff640 1 -- 172.21.15.26:0/3676096622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c0700f0 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad9bfff640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c0c6710 0x7fad9c14afa0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c410 tx=0x7fad8800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad997fa640 1 -- 172.21.15.26:0/3676096622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad880074e0 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad997fa640 1 -- 172.21.15.26:0/3676096622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad880047e0 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c14b4e0 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fad997fa640 1 -- 172.21.15.26:0/3676096622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88004bd0 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.920+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c1ef160 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.921+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c04f330 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.922+0000 7fad997fa640 1 -- 172.21.15.26:0/3676096622 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad88011070 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.922+0000 7fad997fa640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad8006b4d0 0x7fad8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.922+0000 7fad997fa640 1 -- 172.21.15.26:0/3676096622 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fad88089dd0 con 0x7fad9c0c6710 2024-03-06T07:25:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.922+0000 7fada0eaf640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad8006b4d0 0x7fad8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.923+0000 7fada0eaf640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad8006b4d0 0x7fad8006d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fad90036000 tx=0x7fad90008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:19.927+0000 7fad997fa640 1 -- 172.21.15.26:0/3676096622 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fad88058bb0 con 0x7fad9c0c6710 2024-03-06T07:25:20.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.148+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fad9c0a8820 con 0x7fad9c0c6710 2024-03-06T07:25:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.779+0000 7fad997fa640 1 -- 172.21.15.26:0/3676096622 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v119) v1 ==== 174+0+0 (secure 0 0 0) 0x7fad880584b0 con 0x7fad9c0c6710 2024-03-06T07:25:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 21 and data pool 22 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad8006b4d0 msgr2=0x7fad8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad8006b4d0 0x7fad8006d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fad90036000 tx=0x7fad90008030 comp rx=0 tx=0).stop 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c0c6710 msgr2=0x7fad9c14afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c0c6710 0x7fad9c14afa0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c410 tx=0x7fad8800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 shutdown_connections 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad8006b4d0 0x7fad8006d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9c152880 0x7fad9c14cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9c0c6710 0x7fad9c14afa0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 --2- 172.21.15.26:0/3676096622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c0a7cd0 0x7fad9c1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 >> 172.21.15.26:0/3676096622 conn(0x7fad9c0aa5c0 msgr2=0x7fad9c0c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:20.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 shutdown_connections 2024-03-06T07:25:20.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.782+0000 7fada3113640 1 -- 172.21.15.26:0/3676096622 wait complete. 2024-03-06T07:25:20.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:20.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.915+0000 7f42416a4640 1 Processor -- start 2024-03-06T07:25:20.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.915+0000 7f42416a4640 1 -- start start 2024-03-06T07:25:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f42416a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f423c1527b0 0x7f423c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f42416a4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f423c0c88b0 0x7f423c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f42416a4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c0c7c60 con 0x7f423c0c7180 2024-03-06T07:25:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f42416a4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c159640 con 0x7f423c1527b0 2024-03-06T07:25:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f42416a4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c0c8060 con 0x7f423c0c88b0 2024-03-06T07:25:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f423c1527b0 0x7f423c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f423c1527b0 0x7f423c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37400/0 (socket says 172.21.15.26:37400) 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423affd640 1 -- 172.21.15.26:0/2775723528 learned_addr learned my addr 172.21.15.26:0/2775723528 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423b7fe640 1 --2- 172.21.15.26:0/2775723528 >> v2:172.21.15.117:3300/0 conn(0x7f423c0c88b0 0x7f423c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423affd640 1 -- 172.21.15.26:0/2775723528 >> v2:172.21.15.117:3300/0 conn(0x7f423c0c88b0 msgr2=0x7f423c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423affd640 1 --2- 172.21.15.26:0/2775723528 >> v2:172.21.15.117:3300/0 conn(0x7f423c0c88b0 0x7f423c0c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423affd640 1 -- 172.21.15.26:0/2775723528 >> v1:172.21.15.117:6789/0 conn(0x7f423c0c7180 legacy=0x7f423c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423affd640 1 -- 172.21.15.26:0/2775723528 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0c76a0 con 0x7f423c1527b0 2024-03-06T07:25:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.916+0000 7f423b7fe640 1 --2- 172.21.15.26:0/2775723528 >> v2:172.21.15.117:3300/0 conn(0x7f423c0c88b0 0x7f423c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.917+0000 7f423affd640 1 --2- 172.21.15.26:0/2775723528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c1527b0 0x7f423c159100 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f423c0ad600 tx=0x7f423402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575d0d55deba3d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.917+0000 7f4239ffb640 1 -- 172.21.15.26:0/2775723528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234035020 con 0x7f423c1527b0 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.917+0000 7f4239ffb640 1 -- 172.21.15.26:0/2775723528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234031870 con 0x7f423c1527b0 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.917+0000 7f4239ffb640 1 -- 172.21.15.26:0/2775723528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234031c40 con 0x7f423c1527b0 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 -- 172.21.15.26:0/2775723528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c1527b0 msgr2=0x7f423c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 --2- 172.21.15.26:0/2775723528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c1527b0 0x7f423c159100 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f423c0ad600 tx=0x7f423402b910 comp rx=0 tx=0).stop 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 -- 172.21.15.26:0/2775723528 shutdown_connections 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 --2- 172.21.15.26:0/2775723528 >> v2:172.21.15.117:3300/0 conn(0x7f423c0c88b0 0x7f423c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 --2- 172.21.15.26:0/2775723528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c1527b0 0x7f423c159100 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 -- 172.21.15.26:0/2775723528 >> 172.21.15.26:0/2775723528 conn(0x7f423c0ac560 msgr2=0x7f423c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 -- 172.21.15.26:0/2775723528 shutdown_connections 2024-03-06T07:25:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.918+0000 7f42416a4640 1 -- 172.21.15.26:0/2775723528 wait complete. 2024-03-06T07:25:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.919+0000 7f42416a4640 1 Processor -- start 2024-03-06T07:25:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.919+0000 7f42416a4640 1 -- start start 2024-03-06T07:25:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f42416a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0c7180 0x7f423c14be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f42416a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f423c0c88b0 0x7f423c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f42416a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c1527b0 0x7f423c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f42416a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c155600 con 0x7f423c0c7180 2024-03-06T07:25:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f42416a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c155320 con 0x7f423c1527b0 2024-03-06T07:25:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f42416a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c152b90 con 0x7f423c0c88b0 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0c7180 0x7f423c14be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0c7180 0x7f423c14be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37406/0 (socket says 172.21.15.26:37406) 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 -- 172.21.15.26:0/783175478 learned_addr learned my addr 172.21.15.26:0/783175478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423a7fc640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f423c0c88b0 0x7f423c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423b7fe640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c1527b0 0x7f423c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 -- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f423c0c88b0 msgr2=0x7f423c151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f423c0c88b0 0x7f423c151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 -- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c1527b0 msgr2=0x7f423c1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c1527b0 0x7f423c1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 -- 172.21.15.26:0/783175478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c14c560 con 0x7f423c0c7180 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423b7fe640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c1527b0 0x7f423c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.920+0000 7f423affd640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0c7180 0x7f423c14be40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f423402b950 tx=0x7f42340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.921+0000 7f42337fe640 1 -- 172.21.15.26:0/783175478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234040070 con 0x7f423c0c7180 2024-03-06T07:25:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.921+0000 7f42337fe640 1 -- 172.21.15.26:0/783175478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234030d00 con 0x7f423c0c7180 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.921+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c14bb60 con 0x7f423c0c7180 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.921+0000 7f42337fe640 1 -- 172.21.15.26:0/783175478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234035040 con 0x7f423c0c7180 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.921+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c1eafc0 con 0x7f423c0c7180 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.922+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c1eaaf0 con 0x7f423c0c7180 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.922+0000 7f42337fe640 1 -- 172.21.15.26:0/783175478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4234038020 con 0x7f423c0c7180 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.923+0000 7f42337fe640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f421c06b4d0 0x7f421c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.923+0000 7f42337fe640 1 -- 172.21.15.26:0/783175478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42340ac050 con 0x7f423c0c7180 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.923+0000 7f423a7fc640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f421c06b4d0 0x7f421c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.923+0000 7f423a7fc640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f421c06b4d0 0x7f421c06d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f423c0c1730 tx=0x7f4224009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:20.927+0000 7f42337fe640 1 -- 172.21.15.26:0/783175478 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f423407a780 con 0x7f423c0c7180 2024-03-06T07:25:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.111+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f423c04f330 con 0x7f423c0c7180 2024-03-06T07:25:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.112+0000 7f42337fe640 1 -- 172.21.15.26:0/783175478 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16917 (secure 0 0 0) 0x7f423407e820 con 0x7f423c0c7180 2024-03-06T07:25:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:21.114 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":134,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:20.148993+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:18.406892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:19.110341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f421c06b4d0 msgr2=0x7f421c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f421c06b4d0 0x7f421c06d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f423c0c1730 tx=0x7f4224009040 comp rx=0 tx=0).stop 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0c7180 msgr2=0x7f423c14be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0c7180 0x7f423c14be40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f423402b950 tx=0x7f42340046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 shutdown_connections 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f421c06b4d0 0x7f421c06d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c1527b0 0x7f423c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f423c0c88b0 0x7f423c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 --2- 172.21.15.26:0/783175478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0c7180 0x7f423c14be40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 >> 172.21.15.26:0/783175478 conn(0x7f423c0ac560 msgr2=0x7f423c0c9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:21.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 shutdown_connections 2024-03-06T07:25:21.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.114+0000 7f42416a4640 1 -- 172.21.15.26:0/783175478 wait complete. 2024-03-06T07:25:21.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:21.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.227+0000 7f7dd3b8c640 1 Processor -- start 2024-03-06T07:25:21.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.227+0000 7f7dd3b8c640 1 -- start start 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.227+0000 7f7dd3b8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7dcc0c5330 0x7f7dcc0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd3b8c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7dcc158db0 0x7f7dcc15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd3b8c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc155110 con 0x7f7dcc0c88b0 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd3b8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc1559e0 con 0x7f7dcc0c5330 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd3b8c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc0c8f40 con 0x7f7dcc158db0 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd1928640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7dcc0c5330 0x7f7dcc0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd1928640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7dcc0c5330 0x7f7dcc0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37412/0 (socket says 172.21.15.26:37412) 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd1928640 1 -- 172.21.15.26:0/2618554212 learned_addr learned my addr 172.21.15.26:0/2618554212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd2129640 1 --2- 172.21.15.26:0/2618554212 >> v2:172.21.15.117:3300/0 conn(0x7f7dcc158db0 0x7f7dcc15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd1928640 1 -- 172.21.15.26:0/2618554212 >> v2:172.21.15.117:3300/0 conn(0x7f7dcc158db0 msgr2=0x7f7dcc15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd1928640 1 --2- 172.21.15.26:0/2618554212 >> v2:172.21.15.117:3300/0 conn(0x7f7dcc158db0 0x7f7dcc15fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd1928640 1 -- 172.21.15.26:0/2618554212 >> v1:172.21.15.117:6789/0 conn(0x7f7dcc0c88b0 legacy=0x7f7dcc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:21.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.228+0000 7f7dd1928640 1 -- 172.21.15.26:0/2618554212 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc155390 con 0x7f7dcc0c5330 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd1928640 1 --2- 172.21.15.26:0/2618554212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc0c5710 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0ca870 tx=0x7f7dc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cc30f06012428a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd0926640 1 -- 172.21.15.26:0/2618554212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8035020 con 0x7f7dcc0c5330 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd0926640 1 -- 172.21.15.26:0/2618554212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc8031870 con 0x7f7dcc0c5330 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd0926640 1 -- 172.21.15.26:0/2618554212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8031c40 con 0x7f7dcc0c5330 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/2618554212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 msgr2=0x7f7dcc0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/2618554212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc0c5710 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0ca870 tx=0x7f7dc802b910 comp rx=0 tx=0).stop 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/2618554212 shutdown_connections 2024-03-06T07:25:21.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/2618554212 >> v2:172.21.15.117:3300/0 conn(0x7f7dcc158db0 0x7f7dcc15fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/2618554212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc0c5710 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.229+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/2618554212 >> 172.21.15.26:0/2618554212 conn(0x7f7dcc0ac560 msgr2=0x7f7dcc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.230+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/2618554212 shutdown_connections 2024-03-06T07:25:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.230+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/2618554212 wait complete. 2024-03-06T07:25:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 Processor -- start 2024-03-06T07:25:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 -- start start 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dcc0c88b0 0x7f7dcc0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dcc0c2520 0x7f7dcc0c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc1529d0 con 0x7f7dcc0c5330 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06d090 con 0x7f7dcc0c88b0 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd3b8c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc152b50 con 0x7f7dcc0c2520 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd1928640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd1928640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc14e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37426/0 (socket says 172.21.15.26:37426) 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd1928640 1 -- 172.21.15.26:0/998465725 learned_addr learned my addr 172.21.15.26:0/998465725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd2129640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dcc0c2520 0x7f7dcc0c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.231+0000 7f7dd1127640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dcc0c88b0 0x7f7dcc0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd1928640 1 -- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dcc0c2520 msgr2=0x7f7dcc0c0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd1928640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dcc0c2520 0x7f7dcc0c0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd1928640 1 -- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dcc0c88b0 msgr2=0x7f7dcc0c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd1928640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dcc0c88b0 0x7f7dcc0c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd1928640 1 -- 172.21.15.26:0/998465725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc1eaae0 con 0x7f7dcc0c5330 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd1928640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc14e2b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7dc802b950 tx=0x7f7dc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dc27fc640 1 -- 172.21.15.26:0/998465725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8040070 con 0x7f7dcc0c5330 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dc27fc640 1 -- 172.21.15.26:0/998465725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc8030d00 con 0x7f7dcc0c5330 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc1ee8d0 con 0x7f7dcc0c5330 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dc27fc640 1 -- 172.21.15.26:0/998465725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8035040 con 0x7f7dcc0c5330 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.232+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc1ead80 con 0x7f7dcc0c5330 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.233+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc067910 con 0x7f7dcc0c5330 2024-03-06T07:25:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.234+0000 7f7dc27fc640 1 -- 172.21.15.26:0/998465725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dc804b060 con 0x7f7dcc0c5330 2024-03-06T07:25:21.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.234+0000 7f7dc27fc640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7db006b4d0 0x7f7db006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.234+0000 7f7dc27fc640 1 -- 172.21.15.26:0/998465725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7dc80ac040 con 0x7f7dcc0c5330 2024-03-06T07:25:21.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.234+0000 7f7dd1127640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7db006b4d0 0x7f7db006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.235+0000 7f7dd1127640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7db006b4d0 0x7f7db006d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc14ecf0 tx=0x7f7dbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.238+0000 7f7dc27fc640 1 -- 172.21.15.26:0/998465725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7dcc067910 con 0x7f7dcc0c5330 2024-03-06T07:25:21.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.420+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dcc04f330 con 0x7f7dcc0c5330 2024-03-06T07:25:21.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.421+0000 7f7dc27fc640 1 -- 172.21.15.26:0/998465725 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16917 (secure 0 0 0) 0x7f7dc807adc0 con 0x7f7dcc0c5330 2024-03-06T07:25:21.423 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:21.423 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":134,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:20.148993+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:18.406892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:19.110341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7db006b4d0 msgr2=0x7f7db006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7db006b4d0 0x7f7db006d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc14ecf0 tx=0x7f7dbc009040 comp rx=0 tx=0).stop 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 msgr2=0x7f7dcc14e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc14e2b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7dc802b950 tx=0x7f7dc80046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 shutdown_connections 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7db006b4d0 0x7f7db006d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dcc0c2520 0x7f7dcc0c0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7dcc0c88b0 0x7f7dcc0c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 --2- 172.21.15.26:0/998465725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dcc0c5330 0x7f7dcc14e2b0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 >> 172.21.15.26:0/998465725 conn(0x7f7dcc0ac560 msgr2=0x7f7dcc154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 shutdown_connections 2024-03-06T07:25:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.423+0000 7f7dd3b8c640 1 -- 172.21.15.26:0/998465725 wait complete. 2024-03-06T07:25:21.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:25:21.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.539+0000 7f155d781640 1 Processor -- start 2024-03-06T07:25:21.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.539+0000 7f155d781640 1 -- start start 2024-03-06T07:25:21.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.539+0000 7f155d781640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1558155e10 0x7f155806c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.539+0000 7f155d781640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1558152ba0 con 0x7f15581527c0 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.539+0000 7f155d781640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15580c8dd0 con 0x7f15580c88c0 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.539+0000 7f155d781640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15580c9390 con 0x7f1558155e10 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.540+0000 7f15567fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1558155e10 0x7f155806c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.540+0000 7f1556ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15581527c0 0x7f15580c73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37224/0 (socket says 172.21.15.26:37224) 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.540+0000 7f1556ffd640 1 -- 172.21.15.26:0/1850276124 learned_addr learned my addr 172.21.15.26:0/1850276124 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.540+0000 7f15567fc640 1 -- 172.21.15.26:0/1850276124 >> v1:172.21.15.117:6789/0 conn(0x7f15580c88c0 legacy=0x7f155815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.540+0000 7f15567fc640 1 -- 172.21.15.26:0/1850276124 >> v1:172.21.15.26:6789/0 conn(0x7f15581527c0 legacy=0x7f15580c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.540+0000 7f1555ffb640 1 -- 172.21.15.26:0/1850276124 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008561071 0 0) 0x7f1548001110 con 0x7f15581527c0 2024-03-06T07:25:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.540+0000 7f15567fc640 1 -- 172.21.15.26:0/1850276124 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580c90b0 con 0x7f1558155e10 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f15567fc640 1 --2- 172.21.15.26:0/1850276124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f155806c020 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097a0 tx=0x7f154c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8f0a6cc7af8dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f1555ffb640 1 -- 172.21.15.26:0/1850276124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c0315a0 con 0x7f1558155e10 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f1555ffb640 1 -- 172.21.15.26:0/1850276124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c037070 con 0x7f1558155e10 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f1555ffb640 1 -- 172.21.15.26:0/1850276124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c031960 con 0x7f1558155e10 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f155d781640 1 -- 172.21.15.26:0/1850276124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 msgr2=0x7f155806c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f155d781640 1 --2- 172.21.15.26:0/1850276124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f155806c020 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097a0 tx=0x7f154c02b840 comp rx=0 tx=0).stop 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f155d781640 1 -- 172.21.15.26:0/1850276124 shutdown_connections 2024-03-06T07:25:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f155d781640 1 --2- 172.21.15.26:0/1850276124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f155806c020 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f155d781640 1 -- 172.21.15.26:0/1850276124 >> 172.21.15.26:0/1850276124 conn(0x7f15580ac550 msgr2=0x7f15580ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:21.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f155d781640 1 -- 172.21.15.26:0/1850276124 shutdown_connections 2024-03-06T07:25:21.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.541+0000 7f155d781640 1 -- 172.21.15.26:0/1850276124 wait complete. 2024-03-06T07:25:21.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 Processor -- start 2024-03-06T07:25:21.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 -- start start 2024-03-06T07:25:21.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15580c88c0 0x7f15580c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15581527c0 0x7f15581507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f15580c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155806d590 con 0x7f1558155e10 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1558155610 con 0x7f15581527c0 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f155d781640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15580c7ae0 con 0x7f15580c88c0 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f15577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f15580c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f15577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f15580c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37442/0 (socket says 172.21.15.26:37442) 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.543+0000 7f15577fe640 1 -- 172.21.15.26:0/108056464 learned_addr learned my addr 172.21.15.26:0/108056464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f1556ffd640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15580c88c0 0x7f15580c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15567fc640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15581527c0 0x7f15581507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15577fe640 1 -- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15580c88c0 msgr2=0x7f15580c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15577fe640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15580c88c0 0x7f15580c2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15577fe640 1 -- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15581527c0 msgr2=0x7f15581507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15577fe640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15581527c0 0x7f15581507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15577fe640 1 -- 172.21.15.26:0/108056464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580c3020 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15567fc640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15581527c0 0x7f15581507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f15577fe640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f15580c4280 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f155000c780 tx=0x7f155000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f153f7fe640 1 -- 172.21.15.26:0/108056464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550004b60 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f153f7fe640 1 -- 172.21.15.26:0/108056464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1550004d00 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155814bc70 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f153f7fe640 1 -- 172.21.15.26:0/108056464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000aa90 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.544+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15581eb8b0 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.545+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15581ebab0 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.546+0000 7f153f7fe640 1 -- 172.21.15.26:0/108056464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f155000ac30 con 0x7f1558155e10 2024-03-06T07:25:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.546+0000 7f153f7fe640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f154406b4d0 0x7f154406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.546+0000 7f153f7fe640 1 -- 172.21.15.26:0/108056464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f155008a7d0 con 0x7f1558155e10 2024-03-06T07:25:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.546+0000 7f1556ffd640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f154406b4d0 0x7f154406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.547+0000 7f1556ffd640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f154406b4d0 0x7f154406d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1548000f60 tx=0x7f1548007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.550+0000 7f153f7fe640 1 -- 172.21.15.26:0/108056464 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1550059550 con 0x7f1558155e10 2024-03-06T07:25:21.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.767+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f155804f330 con 0x7f1558155e10 2024-03-06T07:25:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.796+0000 7f153f7fe640 1 -- 172.21.15.26:0/108056464 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v120) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1550058e50 con 0x7f1558155e10 2024-03-06T07:25:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f154406b4d0 msgr2=0x7f154406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f154406b4d0 0x7f154406d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1548000f60 tx=0x7f1548007b50 comp rx=0 tx=0).stop 2024-03-06T07:25:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 msgr2=0x7f15580c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f15580c4280 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f155000c780 tx=0x7f155000cc40 comp rx=0 tx=0).stop 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 shutdown_connections 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f154406b4d0 0x7f154406d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1558155e10 0x7f15580c4280 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15581527c0 0x7f15581507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 --2- 172.21.15.26:0/108056464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15580c88c0 0x7f15580c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.798+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 >> 172.21.15.26:0/108056464 conn(0x7f15580ac550 msgr2=0x7f15580c73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.799+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 shutdown_connections 2024-03-06T07:25:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.799+0000 7f155d781640 1 -- 172.21.15.26:0/108056464 wait complete. 2024-03-06T07:25:21.811 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:25:21.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:25:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.915+0000 7f9c1adb6640 1 Processor -- start 2024-03-06T07:25:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.915+0000 7f9c1adb6640 1 -- start start 2024-03-06T07:25:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c1adb6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9c14155c30 0x7f9c1406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c1adb6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c140b0450 con 0x7f9c140c5380 2024-03-06T07:25:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c1adb6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c141560e0 con 0x7f9c140b0070 2024-03-06T07:25:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c1adb6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1406cfb0 con 0x7f9c14155c30 2024-03-06T07:25:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c19353640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9c140c5380 0x7f9c1414e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37234/0 (socket says 172.21.15.26:37234) 2024-03-06T07:25:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c19353640 1 -- 172.21.15.26:0/1832554876 learned_addr learned my addr 172.21.15.26:0/1832554876 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c13fff640 1 --2- 172.21.15.26:0/1832554876 >> v2:172.21.15.117:3300/0 conn(0x7f9c14155c30 0x7f9c1406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c137fe640 1 -- 172.21.15.26:0/1832554876 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399234608 0 0) 0x7f9c04001110 con 0x7f9c140c5380 2024-03-06T07:25:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c137fe640 1 -- 172.21.15.26:0/1832554876 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c00003520 con 0x7f9c140c5380 2024-03-06T07:25:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c13fff640 1 -- 172.21.15.26:0/1832554876 >> v1:172.21.15.117:6789/0 conn(0x7f9c140b0070 legacy=0x7f9c140c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.916+0000 7f9c13fff640 1 -- 172.21.15.26:0/1832554876 >> v1:172.21.15.26:6789/0 conn(0x7f9c140c5380 legacy=0x7f9c1414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.917+0000 7f9c13fff640 1 -- 172.21.15.26:0/1832554876 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c140c77f0 con 0x7f9c14155c30 2024-03-06T07:25:21.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c13fff640 1 --2- 172.21.15.26:0/1832554876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c14155c30 0x7f9c1406ca70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0097a0 tx=0x7f9bfc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b87a896587bd7242 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c137fe640 1 -- 172.21.15.26:0/1832554876 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc035020 con 0x7f9c14155c30 2024-03-06T07:25:21.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c137fe640 1 -- 172.21.15.26:0/1832554876 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc031830 con 0x7f9c14155c30 2024-03-06T07:25:21.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c137fe640 1 -- 172.21.15.26:0/1832554876 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc031be0 con 0x7f9c14155c30 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c1adb6640 1 -- 172.21.15.26:0/1832554876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c14155c30 msgr2=0x7f9c1406ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/1832554876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c14155c30 0x7f9c1406ca70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0097a0 tx=0x7f9bfc02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c1adb6640 1 -- 172.21.15.26:0/1832554876 shutdown_connections 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/1832554876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c14155c30 0x7f9c1406ca70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c1adb6640 1 -- 172.21.15.26:0/1832554876 >> 172.21.15.26:0/1832554876 conn(0x7f9c140ac590 msgr2=0x7f9c140ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c1adb6640 1 -- 172.21.15.26:0/1832554876 shutdown_connections 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.918+0000 7f9c1adb6640 1 -- 172.21.15.26:0/1832554876 wait complete. 2024-03-06T07:25:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.920+0000 7f9c1adb6640 1 Processor -- start 2024-03-06T07:25:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.920+0000 7f9c1adb6640 1 -- start start 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c1adb6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c140b0070 0x7f9c141ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c1adb6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c140c5380 0x7f9c141ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c1adb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c14155c30 0x7f9c1414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c1adb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1406c270 con 0x7f9c14155c30 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c1adb6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1406c020 con 0x7f9c140b0070 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c1adb6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1406c550 con 0x7f9c140c5380 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c19353640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c14155c30 0x7f9c1414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c19353640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c14155c30 0x7f9c1414e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37450/0 (socket says 172.21.15.26:37450) 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c19353640 1 -- 172.21.15.26:0/2166485708 learned_addr learned my addr 172.21.15.26:0/2166485708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c13fff640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c140c5380 0x7f9c141ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.921+0000 7f9c18b52640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c140b0070 0x7f9c141ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c19353640 1 -- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c140c5380 msgr2=0x7f9c141ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c19353640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c140c5380 0x7f9c141ee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c19353640 1 -- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c140b0070 msgr2=0x7f9c141ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c19353640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c140b0070 0x7f9c141ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c19353640 1 -- 172.21.15.26:0/2166485708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c141eed40 con 0x7f9c14155c30 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c13fff640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c140c5380 0x7f9c141ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c18b52640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c140b0070 0x7f9c141ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c19353640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c14155c30 0x7f9c1414e8a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c760 tx=0x7f9c0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c117fa640 1 -- 172.21.15.26:0/2166485708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04017070 con 0x7f9c14155c30 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c117fa640 1 -- 172.21.15.26:0/2166485708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c040058b0 con 0x7f9c14155c30 2024-03-06T07:25:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1414f660 con 0x7f9c14155c30 2024-03-06T07:25:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c117fa640 1 -- 172.21.15.26:0/2166485708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04005ca0 con 0x7f9c14155c30 2024-03-06T07:25:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.922+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c140c4850 con 0x7f9c14155c30 2024-03-06T07:25:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.923+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1404f330 con 0x7f9c14155c30 2024-03-06T07:25:21.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.924+0000 7f9c117fa640 1 -- 172.21.15.26:0/2166485708 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c0401a070 con 0x7f9c14155c30 2024-03-06T07:25:21.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.924+0000 7f9c117fa640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9bf406b4d0 0x7f9bf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:21.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.924+0000 7f9c117fa640 1 -- 172.21.15.26:0/2166485708 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c04089660 con 0x7f9c14155c30 2024-03-06T07:25:21.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.925+0000 7f9c18b52640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9bf406b4d0 0x7f9bf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:21.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.925+0000 7f9c18b52640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9bf406b4d0 0x7f9bf406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c001d60 tx=0x7f9c0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:21.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:21.929+0000 7f9c117fa640 1 -- 172.21.15.26:0/2166485708 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c140b0450 con 0x7f9c14155c30 2024-03-06T07:25:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.148+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9c1406d510 con 0x7f9c14155c30 2024-03-06T07:25:22.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.801+0000 7f9c117fa640 1 -- 172.21.15.26:0/2166485708 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v121) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c04087e20 con 0x7f9c14155c30 2024-03-06T07:25:22.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9bf406b4d0 msgr2=0x7f9bf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:22.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9bf406b4d0 0x7f9bf406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c001d60 tx=0x7f9c0c009040 comp rx=0 tx=0).stop 2024-03-06T07:25:22.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c14155c30 msgr2=0x7f9c1414e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:22.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c14155c30 0x7f9c1414e8a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c760 tx=0x7f9c0400cc20 comp rx=0 tx=0).stop 2024-03-06T07:25:22.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 shutdown_connections 2024-03-06T07:25:22.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9bf406b4d0 0x7f9bf406d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:22.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c14155c30 0x7f9c1414e8a0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:22.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c140c5380 0x7f9c141ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:22.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 --2- 172.21.15.26:0/2166485708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c140b0070 0x7f9c141ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:22.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 >> 172.21.15.26:0/2166485708 conn(0x7f9c140ac590 msgr2=0x7f9c140c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:22.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 shutdown_connections 2024-03-06T07:25:22.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.803+0000 7f9c1adb6640 1 -- 172.21.15.26:0/2166485708 wait complete. 2024-03-06T07:25:22.815 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:25:22.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.915+0000 7f1fa4125640 1 Processor -- start 2024-03-06T07:25:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.915+0000 7f1fa4125640 1 -- start start 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa4125640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f9c156ea0 0x7f1f9c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa4125640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c0ad310 con 0x7f1f9c153990 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa4125640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c1572a0 con 0x7f1f9c1506b0 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa4125640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c156450 con 0x7f1f9c156ea0 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa26c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f9c156ea0 0x7f1f9c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa26c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f9c156ea0 0x7f1f9c0c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37454/0 (socket says 172.21.15.26:37454) 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa26c2640 1 -- 172.21.15.26:0/3837928298 learned_addr learned my addr 172.21.15.26:0/3837928298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa16c0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f9c153990 0x7f1f9c0acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37248/0 (socket says 172.21.15.26:37248) 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa26c2640 1 -- 172.21.15.26:0/3837928298 >> v1:172.21.15.117:6789/0 conn(0x7f1f9c1506b0 legacy=0x7f1f9c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa26c2640 1 -- 172.21.15.26:0/3837928298 >> v1:172.21.15.26:6789/0 conn(0x7f1f9c153990 legacy=0x7f1f9c0acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa0ebf640 1 -- 172.21.15.26:0/3837928298 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057958111 0 0) 0x7f1f9c0ad310 con 0x7f1f9c153990 2024-03-06T07:25:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.916+0000 7f1fa26c2640 1 -- 172.21.15.26:0/3837928298 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c070aa0 con 0x7f1f9c156ea0 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.917+0000 7f1fa26c2640 1 --2- 172.21.15.26:0/3837928298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c0c54a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009e50 tx=0x7f1f9402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae470dc73c99907a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.917+0000 7f1fa0ebf640 1 -- 172.21.15.26:0/3837928298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9402cb20 con 0x7f1f9c156ea0 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.917+0000 7f1fa0ebf640 1 -- 172.21.15.26:0/3837928298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9402ccc0 con 0x7f1f9c156ea0 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.917+0000 7f1fa0ebf640 1 -- 172.21.15.26:0/3837928298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94034610 con 0x7f1f9c156ea0 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.918+0000 7f1fa4125640 1 -- 172.21.15.26:0/3837928298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 msgr2=0x7f1f9c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.918+0000 7f1fa4125640 1 --2- 172.21.15.26:0/3837928298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c0c54a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009e50 tx=0x7f1f9402bf80 comp rx=0 tx=0).stop 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.918+0000 7f1fa4125640 1 -- 172.21.15.26:0/3837928298 shutdown_connections 2024-03-06T07:25:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.918+0000 7f1fa4125640 1 --2- 172.21.15.26:0/3837928298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c0c54a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.918+0000 7f1fa4125640 1 -- 172.21.15.26:0/3837928298 >> 172.21.15.26:0/3837928298 conn(0x7f1f9c06b1a0 msgr2=0x7f1f9c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.918+0000 7f1fa4125640 1 -- 172.21.15.26:0/3837928298 shutdown_connections 2024-03-06T07:25:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.918+0000 7f1fa4125640 1 -- 172.21.15.26:0/3837928298 wait complete. 2024-03-06T07:25:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.919+0000 7f1fa4125640 1 Processor -- start 2024-03-06T07:25:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.919+0000 7f1fa4125640 1 -- start start 2024-03-06T07:25:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa4125640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c1506b0 0x7f1f9c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa4125640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c153990 0x7f1f9c1e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa4125640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c1e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa4125640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1566a0 con 0x7f1f9c156ea0 2024-03-06T07:25:22.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa4125640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c156a10 con 0x7f1f9c1506b0 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa4125640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0c59e0 con 0x7f1f9c153990 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c1e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c1e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37466/0 (socket says 172.21.15.26:37466) 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 -- 172.21.15.26:0/2468112005 learned_addr learned my addr 172.21.15.26:0/2468112005 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa1ec1640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c1506b0 0x7f1f9c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa16c0640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c153990 0x7f1f9c1e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 -- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c153990 msgr2=0x7f1f9c1e8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c153990 0x7f1f9c1e8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 -- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c1506b0 msgr2=0x7f1f9c154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c1506b0 0x7f1f9c154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:22.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 -- 172.21.15.26:0/2468112005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0c2cb0 con 0x7f1f9c156ea0 2024-03-06T07:25:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1fa26c2640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c1e5880 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f1f94036040 tx=0x7f1f94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.920+0000 7f1f927fc640 1 -- 172.21.15.26:0/2468112005 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94043020 con 0x7f1f9c156ea0 2024-03-06T07:25:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.921+0000 7f1f927fc640 1 -- 172.21.15.26:0/2468112005 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9403e450 con 0x7f1f9c156ea0 2024-03-06T07:25:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.921+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c149500 con 0x7f1f9c156ea0 2024-03-06T07:25:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.921+0000 7f1f927fc640 1 -- 172.21.15.26:0/2468112005 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94009510 con 0x7f1f9c156ea0 2024-03-06T07:25:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.921+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c149ac0 con 0x7f1f9c156ea0 2024-03-06T07:25:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.922+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c04f330 con 0x7f1f9c156ea0 2024-03-06T07:25:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.922+0000 7f1f927fc640 1 -- 172.21.15.26:0/2468112005 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f94046020 con 0x7f1f9c156ea0 2024-03-06T07:25:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.922+0000 7f1f927fc640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f8006b4d0 0x7f1f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.923+0000 7f1f927fc640 1 -- 172.21.15.26:0/2468112005 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f940acbc0 con 0x7f1f9c156ea0 2024-03-06T07:25:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.923+0000 7f1fa1ec1640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f8006b4d0 0x7f1f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.923+0000 7f1fa1ec1640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f8006b4d0 0x7f1f8006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1f98004ea0 tx=0x7f1f98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:22.927+0000 7f1f927fc640 1 -- 172.21.15.26:0/2468112005 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f9404d030 con 0x7f1f9c156ea0 2024-03-06T07:25:23.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.147+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f9c151240 con 0x7f1f9c156ea0 2024-03-06T07:25:23.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.148+0000 7f1f927fc640 1 -- 172.21.15.26:0/2468112005 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f1f9407ba00 con 0x7f1f9c156ea0 2024-03-06T07:25:23.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:23.149 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":121,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1667664264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1667664264},{"type":"v1","addr":"172.21.15.117:6836","nonce":1667664264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":7840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:22.149645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8295":{"gid":8295,"name":"a","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1099020571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1099020571},{"type":"v1","addr":"172.21.15.26:6835","nonce":1099020571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8295,"qdb_cluster":[8295]},"id":10}]} 2024-03-06T07:25:23.150 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 121 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f8006b4d0 msgr2=0x7f1f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f8006b4d0 0x7f1f8006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1f98004ea0 tx=0x7f1f98009040 comp rx=0 tx=0).stop 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 msgr2=0x7f1f9c1e5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c1e5880 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f1f94036040 tx=0x7f1f94004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 shutdown_connections 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f8006b4d0 0x7f1f8006d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9c156ea0 0x7f1f9c1e5880 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c153990 0x7f1f9c1e8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 --2- 172.21.15.26:0/2468112005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c1506b0 0x7f1f9c154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 >> 172.21.15.26:0/2468112005 conn(0x7f1f9c06b1a0 msgr2=0x7f1f9c14e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 shutdown_connections 2024-03-06T07:25:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.150+0000 7f1fa4125640 1 -- 172.21.15.26:0/2468112005 wait complete. 2024-03-06T07:25:23.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.264+0000 7f7ed1d01640 1 Processor -- start 2024-03-06T07:25:23.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.264+0000 7f7ed1d01640 1 -- start start 2024-03-06T07:25:23.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.264+0000 7f7ed1d01640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 0x7f7ecc1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.264+0000 7f7ed1d01640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc1591c0 con 0x7f7ecc155ba0 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.264+0000 7f7ed1d01640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc1594a0 con 0x7f7ecc0c5480 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.264+0000 7f7ed1d01640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc06df40 con 0x7f7ecc0c5860 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7ecaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7ecc155ba0 0x7f7ecc1586f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37252/0 (socket says 172.21.15.26:37252) 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7ecaffd640 1 -- 172.21.15.26:0/2268809458 learned_addr learned my addr 172.21.15.26:0/2268809458 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7ecbfff640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 0x7f7ecc1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877263333 0 0) 0x7f7ecc1591c0 con 0x7f7ecc155ba0 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb4003520 con 0x7f7ecc155ba0 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1041929489 0 0) 0x7f7eb4003520 con 0x7f7ecc155ba0 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 msgr2=0x7f7ecc1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 0x7f7ecc1609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 >> v1:172.21.15.117:6789/0 conn(0x7f7ecc0c5480 legacy=0x7f7ecc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc1553a0 con 0x7f7ecc155ba0 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f7ec0001670 con 0x7f7ecc155ba0 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7ecbfff640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 0x7f7ecc1609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 >> v1:172.21.15.26:6789/0 conn(0x7f7ecc155ba0 legacy=0x7f7ecc1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb40039e0 0x7f7eb4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7eb4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 0x7f7eb4005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc06df40 con 0x7f7eb4004390 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb400ba40 con 0x7f7eb40039e0 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb400bc40 con 0x7f7ecc0c5860 2024-03-06T07:25:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7ecaffd640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7eb4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7ecb7fe640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb40039e0 0x7f7eb4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.265+0000 7f7ecbfff640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 0x7f7eb4005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.266+0000 7f7ecaffd640 1 -- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 msgr2=0x7f7eb4005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.266+0000 7f7ecaffd640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc0c5860 0x7f7eb4005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.266+0000 7f7ecaffd640 1 -- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb40039e0 msgr2=0x7f7eb4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.266+0000 7f7ecaffd640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb40039e0 0x7f7eb4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.266+0000 7f7ecaffd640 1 -- 172.21.15.26:0/2268809458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4003520 con 0x7f7eb4004390 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.266+0000 7f7ecaffd640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7eb4004860 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c190 tx=0x7f7eb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b3cbd56749a1ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0007cc0 con 0x7f7eb4004390 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec0030070 con 0x7f7eb4004390 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7eca7fc640 1 -- 172.21.15.26:0/2268809458 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0005d10 con 0x7f7eb4004390 2024-03-06T07:25:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2268809458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 msgr2=0x7f7eb4004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7ed1d01640 1 --2- 172.21.15.26:0/2268809458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7eb4004860 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c190 tx=0x7f7eb4005830 comp rx=0 tx=0).stop 2024-03-06T07:25:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7ecbfff640 1 -- 172.21.15.26:0/2268809458 reap_dead start 2024-03-06T07:25:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2268809458 shutdown_connections 2024-03-06T07:25:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2268809458 >> 172.21.15.26:0/2268809458 conn(0x7f7ecc06b1a0 msgr2=0x7f7ecc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2268809458 shutdown_connections 2024-03-06T07:25:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.267+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2268809458 wait complete. 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.269+0000 7f7ed1d01640 1 Processor -- start 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.269+0000 7f7ed1d01640 1 -- start start 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ed1d01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7ecc1ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ed1d01640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ecc1ec9e0 0x7f7ecc1ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ed1d01640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc14f550 0x7f7ecc157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ed1d01640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0c5cf0 con 0x7f7eb4004390 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ed1d01640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc06fdb0 con 0x7f7ecc1ec9e0 2024-03-06T07:25:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ed1d01640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc156000 con 0x7f7ecc14f550 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ecb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7ecc1ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ecb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7ecc1ef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37488/0 (socket says 172.21.15.26:37488) 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ecb7fe640 1 -- 172.21.15.26:0/2527907963 learned_addr learned my addr 172.21.15.26:0/2527907963 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ecbfff640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc14f550 0x7f7ecc157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.270+0000 7f7ecaffd640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ecc1ec9e0 0x7f7ecc1ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ecb7fe640 1 -- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc14f550 msgr2=0x7f7ecc157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ecb7fe640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc14f550 0x7f7ecc157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ecb7fe640 1 -- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ecc1ec9e0 msgr2=0x7f7ecc1ebf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ecb7fe640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ecc1ec9e0 0x7f7ecc1ebf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ecb7fe640 1 -- 172.21.15.26:0/2527907963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0aad70 con 0x7f7eb4004390 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ecbfff640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc14f550 0x7f7ecc157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ecb7fe640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7ecc1ef4f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00f7f0 tx=0x7f7ebc00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7eb3fff640 1 -- 172.21.15.26:0/2527907963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00fea0 con 0x7f7eb4004390 2024-03-06T07:25:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7eb3fff640 1 -- 172.21.15.26:0/2527907963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc018070 con 0x7f7eb4004390 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc0ad150 con 0x7f7eb4004390 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7eb3fff640 1 -- 172.21.15.26:0/2527907963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00d730 con 0x7f7eb4004390 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.271+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc1eb710 con 0x7f7eb4004390 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.272+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc075300 con 0x7f7eb4004390 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.272+0000 7f7eb3fff640 1 -- 172.21.15.26:0/2527907963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ebc00d8d0 con 0x7f7eb4004390 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.273+0000 7f7eb3fff640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7eb806b500 0x7f7eb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.273+0000 7f7eb3fff640 1 -- 172.21.15.26:0/2527907963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ebc08a2b0 con 0x7f7eb4004390 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.273+0000 7f7ecaffd640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7eb806b500 0x7f7eb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.273+0000 7f7ecaffd640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7eb806b500 0x7f7eb806d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0034000 tx=0x7f7ec0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.281+0000 7f7eb3fff640 1 -- 172.21.15.26:0/2527907963 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ecc075300 con 0x7f7eb4004390 2024-03-06T07:25:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.463+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ecc04f330 con 0x7f7eb4004390 2024-03-06T07:25:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.464+0000 7f7eb3fff640 1 -- 172.21.15.26:0/2527907963 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16904 (secure 0 0 0) 0x7f7ebc058fa0 con 0x7f7eb4004390 2024-03-06T07:25:23.468 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:23.468 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":135,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:21.780235+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:18.406892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:19.110341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7eb806b500 msgr2=0x7f7eb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7eb806b500 0x7f7eb806d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0034000 tx=0x7f7ec0009030 comp rx=0 tx=0).stop 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 msgr2=0x7f7ecc1ef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7ecc1ef4f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00f7f0 tx=0x7f7ebc00fcb0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 shutdown_connections 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7eb806b500 0x7f7eb806d9c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ecc14f550 0x7f7ecc157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ecc1ec9e0 0x7f7ecc1ebf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 --2- 172.21.15.26:0/2527907963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb4004390 0x7f7ecc1ef4f0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 >> 172.21.15.26:0/2527907963 conn(0x7f7ecc06b1a0 msgr2=0x7f7ecc153eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 shutdown_connections 2024-03-06T07:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.466+0000 7f7ed1d01640 1 -- 172.21.15.26:0/2527907963 wait complete. 2024-03-06T07:25:23.482 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:25:23.482 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.582+0000 7f406f532640 1 Processor -- start 2024-03-06T07:25:23.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.582+0000 7f406f532640 1 -- start start 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.583+0000 7f406f532640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4068152c20 0x7f406815d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.583+0000 7f406f532640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4068072440 con 0x7f40680c3150 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.583+0000 7f406f532640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40680c2b10 con 0x7f406814f650 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.583+0000 7f406f532640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4068152140 con 0x7f4068152c20 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f406cacd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f40680c3150 0x7f40680c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37258/0 (socket says 172.21.15.26:37258) 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f406cacd640 1 -- 172.21.15.26:0/2890037288 learned_addr learned my addr 172.21.15.26:0/2890037288 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f406dacf640 1 --2- 172.21.15.26:0/2890037288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4068152c20 0x7f406815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f405ffff640 1 -- 172.21.15.26:0/2890037288 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055083990 0 0) 0x7f4058001110 con 0x7f40680c3150 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f405ffff640 1 -- 172.21.15.26:0/2890037288 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4054003520 con 0x7f40680c3150 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f406dacf640 1 -- 172.21.15.26:0/2890037288 >> v1:172.21.15.117:6789/0 conn(0x7f406814f650 legacy=0x7f4068071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f405ffff640 1 -- 172.21.15.26:0/2890037288 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179769970 0 0) 0x7f40680c2b10 con 0x7f406814f650 2024-03-06T07:25:23.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f406dacf640 1 -- 172.21.15.26:0/2890037288 >> v1:172.21.15.26:6789/0 conn(0x7f40680c3150 legacy=0x7f40680c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.584+0000 7f406dacf640 1 -- 172.21.15.26:0/2890037288 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068152420 con 0x7f4068152c20 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.585+0000 7f406dacf640 1 --2- 172.21.15.26:0/2890037288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4068152c20 0x7f406815d820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4060009dc0 tx=0x7f406002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd3c9b395718462 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.585+0000 7f405ffff640 1 -- 172.21.15.26:0/2890037288 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060039070 con 0x7f4068152c20 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.585+0000 7f405ffff640 1 -- 172.21.15.26:0/2890037288 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406002ccd0 con 0x7f4068152c20 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.585+0000 7f405ffff640 1 -- 172.21.15.26:0/2890037288 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060034640 con 0x7f4068152c20 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.586+0000 7f406f532640 1 -- 172.21.15.26:0/2890037288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4068152c20 msgr2=0x7f406815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.586+0000 7f406f532640 1 --2- 172.21.15.26:0/2890037288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4068152c20 0x7f406815d820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4060009dc0 tx=0x7f406002bf20 comp rx=0 tx=0).stop 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.586+0000 7f406f532640 1 -- 172.21.15.26:0/2890037288 shutdown_connections 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.586+0000 7f406f532640 1 --2- 172.21.15.26:0/2890037288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4068152c20 0x7f406815d820 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.586+0000 7f406f532640 1 -- 172.21.15.26:0/2890037288 >> 172.21.15.26:0/2890037288 conn(0x7f40680c4290 msgr2=0x7f40680c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.586+0000 7f406f532640 1 -- 172.21.15.26:0/2890037288 shutdown_connections 2024-03-06T07:25:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.586+0000 7f406f532640 1 -- 172.21.15.26:0/2890037288 wait complete. 2024-03-06T07:25:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.587+0000 7f406f532640 1 Processor -- start 2024-03-06T07:25:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.587+0000 7f406f532640 1 -- start start 2024-03-06T07:25:23.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.588+0000 7f406f532640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40680c3150 0x7f40680acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.588+0000 7f406f532640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f406814f650 0x7f4068148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.588+0000 7f406f532640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4068152c20 0x7f4068148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.588+0000 7f406f532640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068072110 con 0x7f40680c3150 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.588+0000 7f406f532640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068071f90 con 0x7f4068152c20 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.588+0000 7f406f532640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680c3f40 con 0x7f406814f650 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40680c3150 0x7f40680acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40680c3150 0x7f40680acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37496/0 (socket says 172.21.15.26:37496) 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 -- 172.21.15.26:0/3958330974 learned_addr learned my addr 172.21.15.26:0/3958330974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406dacf640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4068152c20 0x7f4068148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406cacd640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f406814f650 0x7f4068148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 -- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f406814f650 msgr2=0x7f4068148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f406814f650 0x7f4068148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 -- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4068152c20 msgr2=0x7f4068148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4068152c20 0x7f4068148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 -- 172.21.15.26:0/3958330974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068155cf0 con 0x7f40680c3150 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406d2ce640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40680c3150 0x7f40680acab0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f406400c460 tx=0x7f406400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f405dffb640 1 -- 172.21.15.26:0/3958330974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064017070 con 0x7f40680c3150 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f405dffb640 1 -- 172.21.15.26:0/3958330974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40640035f0 con 0x7f40680c3150 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40681530a0 con 0x7f40680c3150 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.589+0000 7f405dffb640 1 -- 172.21.15.26:0/3958330974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640039e0 con 0x7f40680c3150 2024-03-06T07:25:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.590+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4068153340 con 0x7f40680c3150 2024-03-06T07:25:23.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.591+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406814e1c0 con 0x7f40680c3150 2024-03-06T07:25:23.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.591+0000 7f405dffb640 1 -- 172.21.15.26:0/3958330974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f406401a070 con 0x7f40680c3150 2024-03-06T07:25:23.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.591+0000 7f405dffb640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f404c06b4d0 0x7f404c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.591+0000 7f405dffb640 1 -- 172.21.15.26:0/3958330974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4064089600 con 0x7f40680c3150 2024-03-06T07:25:23.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.591+0000 7f406cacd640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f404c06b4d0 0x7f404c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.592+0000 7f406cacd640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f404c06b4d0 0x7f404c06d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f40680c1fe0 tx=0x7f4058009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.595+0000 7f405dffb640 1 -- 172.21.15.26:0/3958330974 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f406814e1c0 con 0x7f40680c3150 2024-03-06T07:25:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.773+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40681493d0 con 0x7f404c06b4d0 2024-03-06T07:25:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.780+0000 7f405dffb640 1 -- 172.21.15.26:0/3958330974 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247164 (secure 0 0 0) 0x7f40681493d0 con 0x7f404c06b4d0 2024-03-06T07:25:23.784 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:25:23.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f404c06b4d0 msgr2=0x7f404c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f404c06b4d0 0x7f404c06d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f40680c1fe0 tx=0x7f4058009040 comp rx=0 tx=0).stop 2024-03-06T07:25:23.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40680c3150 msgr2=0x7f40680acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40680c3150 0x7f40680acab0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f406400c460 tx=0x7f406400c920 comp rx=0 tx=0).stop 2024-03-06T07:25:23.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 shutdown_connections 2024-03-06T07:25:23.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f404c06b4d0 0x7f404c06d990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4068152c20 0x7f4068148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f406814f650 0x7f4068148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 --2- 172.21.15.26:0/3958330974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40680c3150 0x7f40680acab0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.782+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 >> 172.21.15.26:0/3958330974 conn(0x7f40680c4290 msgr2=0x7f406814cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:23.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.783+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 shutdown_connections 2024-03-06T07:25:23.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.783+0000 7f406f532640 1 -- 172.21.15.26:0/3958330974 wait complete. 2024-03-06T07:25:23.795 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-03-06T07:25:23.097094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262564,"kb_used_data":13100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712156,"statfs":{"total":773094113280,"available":772825247744,"internally_reserved":0,"allocated":13414400,"data_stored":7566661,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319375"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790263+0000","last_change":"2024-03-06T07:25:19.790263+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790263+0000","last_undegraded":"2024-03-06T07:25:19.790263+0000","last_fullsized":"2024-03-06T07:25:19.790263+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788156+0000","last_change":"2024-03-06T07:25:19.788156+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788156+0000","last_undegraded":"2024-03-06T07:25:19.788156+0000","last_fullsized":"2024-03-06T07:25:19.788156+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787767+0000","last_change":"2024-03-06T07:25:19.787767+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787767+0000","last_undegraded":"2024-03-06T07:25:19.787767+0000","last_fullsized":"2024-03-06T07:25:19.787767+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788939+0000","last_change":"2024-03-06T07:25:19.788939+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788939+0000","last_undegraded":"2024-03-06T07:25:19.788939+0000","last_fullsized":"2024-03-06T07:25:19.788939+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.795996+0000","last_change":"2024-03-06T07:25:19.795996+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.795996+0000","last_undegraded":"2024-03-06T07:25:19.795996+0000","last_fullsized":"2024-03-06T07:25:19.795996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.795192+0000","last_change":"2024-03-06T07:25:19.795192+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.795192+0000","last_undegraded":"2024-03-06T07:25:19.795192+0000","last_fullsized":"2024-03-06T07:25:19.795192+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.791435+0000","last_change":"2024-03-06T07:25:19.791435+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.791435+0000","last_undegraded":"2024-03-06T07:25:19.791435+0000","last_fullsized":"2024-03-06T07:25:19.791435+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784134+0000","last_change":"2024-03-06T07:25:19.784134+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784134+0000","last_undegraded":"2024-03-06T07:25:19.784134+0000","last_fullsized":"2024-03-06T07:25:19.784134+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792512+0000","last_change":"2024-03-06T07:25:19.792512+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792512+0000","last_undegraded":"2024-03-06T07:25:19.792512+0000","last_fullsized":"2024-03-06T07:25:19.792512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792010+0000","last_change":"2024-03-06T07:25:19.792010+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792010+0000","last_undegraded":"2024-03-06T07:25:19.792010+0000","last_fullsized":"2024-03-06T07:25:19.792010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785099+0000","last_change":"2024-03-06T07:25:19.785099+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785099+0000","last_undegraded":"2024-03-06T07:25:19.785099+0000","last_fullsized":"2024-03-06T07:25:19.785099+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788546+0000","last_change":"2024-03-06T07:25:19.788546+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788546+0000","last_undegraded":"2024-03-06T07:25:19.788546+0000","last_fullsized":"2024-03-06T07:25:19.788546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.793411+0000","last_change":"2024-03-06T07:25:19.793411+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.793411+0000","last_undegraded":"2024-03-06T07:25:19.793411+0000","last_fullsized":"2024-03-06T07:25:19.793411+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789475+0000","last_change":"2024-03-06T07:25:19.789475+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789475+0000","last_undegraded":"2024-03-06T07:25:19.789475+0000","last_fullsized":"2024-03-06T07:25:19.789475+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789405+0000","last_change":"2024-03-06T07:25:19.789405+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789405+0000","last_undegraded":"2024-03-06T07:25:19.789405+0000","last_fullsized":"2024-03-06T07:25:19.789405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790829+0000","last_change":"2024-03-06T07:25:19.790829+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790829+0000","last_undegraded":"2024-03-06T07:25:19.790829+0000","last_fullsized":"2024-03-06T07:25:19.790829+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.793506+0000","last_change":"2024-03-06T07:25:19.793506+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.793506+0000","last_undegraded":"2024-03-06T07:25:19.793506+0000","last_fullsized":"2024-03-06T07:25:19.793506+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784444+0000","last_change":"2024-03-06T07:25:19.784444+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784444+0000","last_undegraded":"2024-03-06T07:25:19.784444+0000","last_fullsized":"2024-03-06T07:25:19.784444+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785275+0000","last_change":"2024-03-06T07:25:19.785275+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785275+0000","last_undegraded":"2024-03-06T07:25:19.785275+0000","last_fullsized":"2024-03-06T07:25:19.785275+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785315+0000","last_change":"2024-03-06T07:25:19.785315+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785315+0000","last_undegraded":"2024-03-06T07:25:19.785315+0000","last_fullsized":"2024-03-06T07:25:19.785315+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792999+0000","last_change":"2024-03-06T07:25:19.792999+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792999+0000","last_undegraded":"2024-03-06T07:25:19.792999+0000","last_fullsized":"2024-03-06T07:25:19.792999+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788950+0000","last_change":"2024-03-06T07:25:19.788950+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788950+0000","last_undegraded":"2024-03-06T07:25:19.788950+0000","last_fullsized":"2024-03-06T07:25:19.788950+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783383+0000","last_change":"2024-03-06T07:25:19.783383+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783383+0000","last_undegraded":"2024-03-06T07:25:19.783383+0000","last_fullsized":"2024-03-06T07:25:19.783383+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784142+0000","last_change":"2024-03-06T07:25:19.784142+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784142+0000","last_undegraded":"2024-03-06T07:25:19.784142+0000","last_fullsized":"2024-03-06T07:25:19.784142+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789851+0000","last_change":"2024-03-06T07:25:19.789851+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789851+0000","last_undegraded":"2024-03-06T07:25:19.789851+0000","last_fullsized":"2024-03-06T07:25:19.789851+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790728+0000","last_change":"2024-03-06T07:25:19.790728+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790728+0000","last_undegraded":"2024-03-06T07:25:19.790728+0000","last_fullsized":"2024-03-06T07:25:19.790728+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.791445+0000","last_change":"2024-03-06T07:25:19.791445+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.791445+0000","last_undegraded":"2024-03-06T07:25:19.791445+0000","last_fullsized":"2024-03-06T07:25:19.791445+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784190+0000","last_change":"2024-03-06T07:25:19.784190+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784190+0000","last_undegraded":"2024-03-06T07:25:19.784190+0000","last_fullsized":"2024-03-06T07:25:19.784190+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783073+0000","last_change":"2024-03-06T07:25:19.783073+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783073+0000","last_undegraded":"2024-03-06T07:25:19.783073+0000","last_fullsized":"2024-03-06T07:25:19.783073+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784550+0000","last_change":"2024-03-06T07:25:19.784550+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784550+0000","last_undegraded":"2024-03-06T07:25:19.784550+0000","last_fullsized":"2024-03-06T07:25:19.784550+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790473+0000","last_change":"2024-03-06T07:25:19.790473+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790473+0000","last_undegraded":"2024-03-06T07:25:19.790473+0000","last_fullsized":"2024-03-06T07:25:19.790473+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783060+0000","last_change":"2024-03-06T07:25:19.783060+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783060+0000","last_undegraded":"2024-03-06T07:25:19.783060+0000","last_fullsized":"2024-03-06T07:25:19.783060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788370+0000","last_change":"2024-03-06T07:25:19.788370+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788370+0000","last_undegraded":"2024-03-06T07:25:19.788370+0000","last_fullsized":"2024-03-06T07:25:19.788370+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.800985+0000","last_change":"2024-03-06T07:25:19.800026+0000","last_active":"2024-03-06T07:25:19.800985+0000","last_peered":"2024-03-06T07:25:19.800985+0000","last_clean":"2024-03-06T07:25:19.800985+0000","last_became_active":"2024-03-06T07:25:19.799694+0000","last_became_peered":"2024-03-06T07:25:19.799694+0000","last_unstale":"2024-03-06T07:25:19.800985+0000","last_undegraded":"2024-03-06T07:25:19.800985+0000","last_fullsized":"2024-03-06T07:25:19.800985+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:05.339137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787369+0000","last_change":"2024-03-06T07:25:19.787369+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787369+0000","last_undegraded":"2024-03-06T07:25:19.787369+0000","last_fullsized":"2024-03-06T07:25:19.787369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.796334+0000","last_change":"2024-03-06T07:25:19.794850+0000","last_active":"2024-03-06T07:25:19.796334+0000","last_peered":"2024-03-06T07:25:19.796334+0000","last_clean":"2024-03-06T07:25:19.796334+0000","last_became_active":"2024-03-06T07:25:19.794587+0000","last_became_peered":"2024-03-06T07:25:19.794587+0000","last_unstale":"2024-03-06T07:25:19.796334+0000","last_undegraded":"2024-03-06T07:25:19.796334+0000","last_fullsized":"2024-03-06T07:25:19.796334+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:28.129920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782596+0000","last_change":"2024-03-06T07:25:19.782596+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782596+0000","last_undegraded":"2024-03-06T07:25:19.782596+0000","last_fullsized":"2024-03-06T07:25:19.782596+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798990+0000","last_change":"2024-03-06T07:25:19.796329+0000","last_active":"2024-03-06T07:25:19.798990+0000","last_peered":"2024-03-06T07:25:19.798990+0000","last_clean":"2024-03-06T07:25:19.798990+0000","last_became_active":"2024-03-06T07:25:19.796081+0000","last_became_peered":"2024-03-06T07:25:19.796081+0000","last_unstale":"2024-03-06T07:25:19.798990+0000","last_undegraded":"2024-03-06T07:25:19.798990+0000","last_fullsized":"2024-03-06T07:25:19.798990+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:18.223482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784369+0000","last_change":"2024-03-06T07:25:19.784369+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784369+0000","last_undegraded":"2024-03-06T07:25:19.784369+0000","last_fullsized":"2024-03-06T07:25:19.784369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.793189+0000","last_change":"2024-03-06T07:25:19.789450+0000","last_active":"2024-03-06T07:25:19.793189+0000","last_peered":"2024-03-06T07:25:19.793189+0000","last_clean":"2024-03-06T07:25:19.793189+0000","last_became_active":"2024-03-06T07:25:19.789139+0000","last_became_peered":"2024-03-06T07:25:19.789139+0000","last_unstale":"2024-03-06T07:25:19.793189+0000","last_undegraded":"2024-03-06T07:25:19.793189+0000","last_fullsized":"2024-03-06T07:25:19.793189+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:58.530205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785243+0000","last_change":"2024-03-06T07:25:19.785243+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785243+0000","last_undegraded":"2024-03-06T07:25:19.785243+0000","last_fullsized":"2024-03-06T07:25:19.785243+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.800808+0000","last_change":"2024-03-06T07:25:19.797666+0000","last_active":"2024-03-06T07:25:19.800808+0000","last_peered":"2024-03-06T07:25:19.800808+0000","last_clean":"2024-03-06T07:25:19.800808+0000","last_became_active":"2024-03-06T07:25:19.797391+0000","last_became_peered":"2024-03-06T07:25:19.797391+0000","last_unstale":"2024-03-06T07:25:19.800808+0000","last_undegraded":"2024-03-06T07:25:19.800808+0000","last_fullsized":"2024-03-06T07:25:19.800808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:48.223756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790355+0000","last_change":"2024-03-06T07:25:19.790355+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790355+0000","last_undegraded":"2024-03-06T07:25:19.790355+0000","last_fullsized":"2024-03-06T07:25:19.790355+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801006+0000","last_change":"2024-03-06T07:25:19.799648+0000","last_active":"2024-03-06T07:25:19.801006+0000","last_peered":"2024-03-06T07:25:19.801006+0000","last_clean":"2024-03-06T07:25:19.801006+0000","last_became_active":"2024-03-06T07:25:19.799352+0000","last_became_peered":"2024-03-06T07:25:19.799352+0000","last_unstale":"2024-03-06T07:25:19.801006+0000","last_undegraded":"2024-03-06T07:25:19.801006+0000","last_fullsized":"2024-03-06T07:25:19.801006+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:30.875962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787231+0000","last_change":"2024-03-06T07:25:19.787231+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787231+0000","last_undegraded":"2024-03-06T07:25:19.787231+0000","last_fullsized":"2024-03-06T07:25:19.787231+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801537+0000","last_change":"2024-03-06T07:25:19.797676+0000","last_active":"2024-03-06T07:25:19.801537+0000","last_peered":"2024-03-06T07:25:19.801537+0000","last_clean":"2024-03-06T07:25:19.801537+0000","last_became_active":"2024-03-06T07:25:19.797399+0000","last_became_peered":"2024-03-06T07:25:19.797399+0000","last_unstale":"2024-03-06T07:25:19.801537+0000","last_undegraded":"2024-03-06T07:25:19.801537+0000","last_fullsized":"2024-03-06T07:25:19.801537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:34.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788917+0000","last_change":"2024-03-06T07:25:19.788917+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788917+0000","last_undegraded":"2024-03-06T07:25:19.788917+0000","last_fullsized":"2024-03-06T07:25:19.788917+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.802569+0000","last_change":"2024-03-06T07:25:19.802156+0000","last_active":"2024-03-06T07:25:19.802569+0000","last_peered":"2024-03-06T07:25:19.802569+0000","last_clean":"2024-03-06T07:25:19.802569+0000","last_became_active":"2024-03-06T07:25:19.801881+0000","last_became_peered":"2024-03-06T07:25:19.801881+0000","last_unstale":"2024-03-06T07:25:19.802569+0000","last_undegraded":"2024-03-06T07:25:19.802569+0000","last_fullsized":"2024-03-06T07:25:19.802569+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:14.322866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782407+0000","last_change":"2024-03-06T07:25:19.782407+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782407+0000","last_undegraded":"2024-03-06T07:25:19.782407+0000","last_fullsized":"2024-03-06T07:25:19.782407+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798410+0000","last_change":"2024-03-06T07:25:19.795560+0000","last_active":"2024-03-06T07:25:19.798410+0000","last_peered":"2024-03-06T07:25:19.798410+0000","last_clean":"2024-03-06T07:25:19.798410+0000","last_became_active":"2024-03-06T07:25:19.795293+0000","last_became_peered":"2024-03-06T07:25:19.795293+0000","last_unstale":"2024-03-06T07:25:19.798410+0000","last_undegraded":"2024-03-06T07:25:19.798410+0000","last_fullsized":"2024-03-06T07:25:19.798410+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:14.322866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784536+0000","last_change":"2024-03-06T07:25:19.784536+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784536+0000","last_undegraded":"2024-03-06T07:25:19.784536+0000","last_fullsized":"2024-03-06T07:25:19.784536+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804177+0000","last_change":"2024-03-06T07:25:19.798578+0000","last_active":"2024-03-06T07:25:19.804177+0000","last_peered":"2024-03-06T07:25:19.804177+0000","last_clean":"2024-03-06T07:25:19.804177+0000","last_became_active":"2024-03-06T07:25:19.798263+0000","last_became_peered":"2024-03-06T07:25:19.798263+0000","last_unstale":"2024-03-06T07:25:19.804177+0000","last_undegraded":"2024-03-06T07:25:19.804177+0000","last_fullsized":"2024-03-06T07:25:19.804177+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:51.204621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783803+0000","last_change":"2024-03-06T07:25:19.783803+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783803+0000","last_undegraded":"2024-03-06T07:25:19.783803+0000","last_fullsized":"2024-03-06T07:25:19.783803+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.793843+0000","last_change":"2024-03-06T07:25:19.792711+0000","last_active":"2024-03-06T07:25:19.793843+0000","last_peered":"2024-03-06T07:25:19.793843+0000","last_clean":"2024-03-06T07:25:19.793843+0000","last_became_active":"2024-03-06T07:25:19.792405+0000","last_became_peered":"2024-03-06T07:25:19.792405+0000","last_unstale":"2024-03-06T07:25:19.793843+0000","last_undegraded":"2024-03-06T07:25:19.793843+0000","last_fullsized":"2024-03-06T07:25:19.793843+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:40.261389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782423+0000","last_change":"2024-03-06T07:25:19.782423+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782423+0000","last_undegraded":"2024-03-06T07:25:19.782423+0000","last_fullsized":"2024-03-06T07:25:19.782423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801343+0000","last_change":"2024-03-06T07:25:19.800658+0000","last_active":"2024-03-06T07:25:19.801343+0000","last_peered":"2024-03-06T07:25:19.801343+0000","last_clean":"2024-03-06T07:25:19.801343+0000","last_became_active":"2024-03-06T07:25:19.800361+0000","last_became_peered":"2024-03-06T07:25:19.800361+0000","last_unstale":"2024-03-06T07:25:19.801343+0000","last_undegraded":"2024-03-06T07:25:19.801343+0000","last_fullsized":"2024-03-06T07:25:19.801343+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:28.129920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788966+0000","last_change":"2024-03-06T07:25:19.788966+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788966+0000","last_undegraded":"2024-03-06T07:25:19.788966+0000","last_fullsized":"2024-03-06T07:25:19.788966+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804522+0000","last_change":"2024-03-06T07:25:19.802236+0000","last_active":"2024-03-06T07:25:19.804522+0000","last_peered":"2024-03-06T07:25:19.804522+0000","last_clean":"2024-03-06T07:25:19.804522+0000","last_became_active":"2024-03-06T07:25:19.801866+0000","last_became_peered":"2024-03-06T07:25:19.801866+0000","last_unstale":"2024-03-06T07:25:19.804522+0000","last_undegraded":"2024-03-06T07:25:19.804522+0000","last_fullsized":"2024-03-06T07:25:19.804522+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:02.212979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790418+0000","last_change":"2024-03-06T07:25:19.790418+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790418+0000","last_undegraded":"2024-03-06T07:25:19.790418+0000","last_fullsized":"2024-03-06T07:25:19.790418+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804286+0000","last_change":"2024-03-06T07:25:19.800076+0000","last_active":"2024-03-06T07:25:19.804286+0000","last_peered":"2024-03-06T07:25:19.804286+0000","last_clean":"2024-03-06T07:25:19.804286+0000","last_became_active":"2024-03-06T07:25:19.799501+0000","last_became_peered":"2024-03-06T07:25:19.799501+0000","last_unstale":"2024-03-06T07:25:19.804286+0000","last_undegraded":"2024-03-06T07:25:19.804286+0000","last_fullsized":"2024-03-06T07:25:19.804286+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:56.743191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786604+0000","last_change":"2024-03-06T07:25:19.786604+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786604+0000","last_undegraded":"2024-03-06T07:25:19.786604+0000","last_fullsized":"2024-03-06T07:25:19.786604+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801485+0000","last_change":"2024-03-06T07:25:19.797067+0000","last_active":"2024-03-06T07:25:19.801485+0000","last_peered":"2024-03-06T07:25:19.801485+0000","last_clean":"2024-03-06T07:25:19.801485+0000","last_became_active":"2024-03-06T07:25:19.796773+0000","last_became_peered":"2024-03-06T07:25:19.796773+0000","last_unstale":"2024-03-06T07:25:19.801485+0000","last_undegraded":"2024-03-06T07:25:19.801485+0000","last_fullsized":"2024-03-06T07:25:19.801485+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:08.144349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784876+0000","last_change":"2024-03-06T07:25:19.784876+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784876+0000","last_undegraded":"2024-03-06T07:25:19.784876+0000","last_fullsized":"2024-03-06T07:25:19.784876+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.790405+0000","last_change":"2024-03-06T07:25:19.789597+0000","last_active":"2024-03-06T07:25:19.790405+0000","last_peered":"2024-03-06T07:25:19.790405+0000","last_clean":"2024-03-06T07:25:19.790405+0000","last_became_active":"2024-03-06T07:25:19.789334+0000","last_became_peered":"2024-03-06T07:25:19.789334+0000","last_unstale":"2024-03-06T07:25:19.790405+0000","last_undegraded":"2024-03-06T07:25:19.790405+0000","last_fullsized":"2024-03-06T07:25:19.790405+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:45.098378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784468+0000","last_change":"2024-03-06T07:25:19.784468+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784468+0000","last_undegraded":"2024-03-06T07:25:19.784468+0000","last_fullsized":"2024-03-06T07:25:19.784468+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799011+0000","last_change":"2024-03-06T07:25:19.796190+0000","last_active":"2024-03-06T07:25:19.799011+0000","last_peered":"2024-03-06T07:25:19.799011+0000","last_clean":"2024-03-06T07:25:19.799011+0000","last_became_active":"2024-03-06T07:25:19.795879+0000","last_became_peered":"2024-03-06T07:25:19.795879+0000","last_unstale":"2024-03-06T07:25:19.799011+0000","last_undegraded":"2024-03-06T07:25:19.799011+0000","last_fullsized":"2024-03-06T07:25:19.799011+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:59.395378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790299+0000","last_change":"2024-03-06T07:25:19.790299+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790299+0000","last_undegraded":"2024-03-06T07:25:19.790299+0000","last_fullsized":"2024-03-06T07:25:19.790299+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799742+0000","last_change":"2024-03-06T07:25:19.798347+0000","last_active":"2024-03-06T07:25:19.799742+0000","last_peered":"2024-03-06T07:25:19.799742+0000","last_clean":"2024-03-06T07:25:19.799742+0000","last_became_active":"2024-03-06T07:25:19.798021+0000","last_became_peered":"2024-03-06T07:25:19.798021+0000","last_unstale":"2024-03-06T07:25:19.799742+0000","last_undegraded":"2024-03-06T07:25:19.799742+0000","last_fullsized":"2024-03-06T07:25:19.799742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:08.144349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789405+0000","last_change":"2024-03-06T07:25:19.789405+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789405+0000","last_undegraded":"2024-03-06T07:25:19.789405+0000","last_fullsized":"2024-03-06T07:25:19.789405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804335+0000","last_change":"2024-03-06T07:25:19.801586+0000","last_active":"2024-03-06T07:25:19.804335+0000","last_peered":"2024-03-06T07:25:19.804335+0000","last_clean":"2024-03-06T07:25:19.804335+0000","last_became_active":"2024-03-06T07:25:19.801242+0000","last_became_peered":"2024-03-06T07:25:19.801242+0000","last_unstale":"2024-03-06T07:25:19.804335+0000","last_undegraded":"2024-03-06T07:25:19.804335+0000","last_fullsized":"2024-03-06T07:25:19.804335+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:39.912413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784346+0000","last_change":"2024-03-06T07:25:19.784346+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784346+0000","last_undegraded":"2024-03-06T07:25:19.784346+0000","last_fullsized":"2024-03-06T07:25:19.784346+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.797760+0000","last_change":"2024-03-06T07:25:19.796315+0000","last_active":"2024-03-06T07:25:19.797760+0000","last_peered":"2024-03-06T07:25:19.797760+0000","last_clean":"2024-03-06T07:25:19.797760+0000","last_became_active":"2024-03-06T07:25:19.796036+0000","last_became_peered":"2024-03-06T07:25:19.796036+0000","last_unstale":"2024-03-06T07:25:19.797760+0000","last_undegraded":"2024-03-06T07:25:19.797760+0000","last_fullsized":"2024-03-06T07:25:19.797760+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:15.257584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788465+0000","last_change":"2024-03-06T07:25:19.788465+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788465+0000","last_undegraded":"2024-03-06T07:25:19.788465+0000","last_fullsized":"2024-03-06T07:25:19.788465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792150+0000","last_change":"2024-03-06T07:25:19.789672+0000","last_active":"2024-03-06T07:25:19.792150+0000","last_peered":"2024-03-06T07:25:19.792150+0000","last_clean":"2024-03-06T07:25:19.792150+0000","last_became_active":"2024-03-06T07:25:19.789409+0000","last_became_peered":"2024-03-06T07:25:19.789409+0000","last_unstale":"2024-03-06T07:25:19.792150+0000","last_undegraded":"2024-03-06T07:25:19.792150+0000","last_fullsized":"2024-03-06T07:25:19.792150+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:07.534390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784743+0000","last_change":"2024-03-06T07:25:19.784743+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784743+0000","last_undegraded":"2024-03-06T07:25:19.784743+0000","last_fullsized":"2024-03-06T07:25:19.784743+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799242+0000","last_change":"2024-03-06T07:25:19.797588+0000","last_active":"2024-03-06T07:25:19.799242+0000","last_peered":"2024-03-06T07:25:19.799242+0000","last_clean":"2024-03-06T07:25:19.799242+0000","last_became_active":"2024-03-06T07:25:19.797322+0000","last_became_peered":"2024-03-06T07:25:19.797322+0000","last_unstale":"2024-03-06T07:25:19.799242+0000","last_undegraded":"2024-03-06T07:25:19.799242+0000","last_fullsized":"2024-03-06T07:25:19.799242+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:19.871887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786003+0000","last_change":"2024-03-06T07:25:19.786003+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786003+0000","last_undegraded":"2024-03-06T07:25:19.786003+0000","last_fullsized":"2024-03-06T07:25:19.786003+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795817+0000","last_change":"2024-03-06T07:25:19.794012+0000","last_active":"2024-03-06T07:25:19.795817+0000","last_peered":"2024-03-06T07:25:19.795817+0000","last_clean":"2024-03-06T07:25:19.795817+0000","last_became_active":"2024-03-06T07:25:19.793743+0000","last_became_peered":"2024-03-06T07:25:19.793743+0000","last_unstale":"2024-03-06T07:25:19.795817+0000","last_undegraded":"2024-03-06T07:25:19.795817+0000","last_fullsized":"2024-03-06T07:25:19.795817+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:22:21.615129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786591+0000","last_change":"2024-03-06T07:25:19.786591+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786591+0000","last_undegraded":"2024-03-06T07:25:19.786591+0000","last_fullsized":"2024-03-06T07:25:19.786591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804333+0000","last_change":"2024-03-06T07:25:19.801518+0000","last_active":"2024-03-06T07:25:19.804333+0000","last_peered":"2024-03-06T07:25:19.804333+0000","last_clean":"2024-03-06T07:25:19.804333+0000","last_became_active":"2024-03-06T07:25:19.801213+0000","last_became_peered":"2024-03-06T07:25:19.801213+0000","last_unstale":"2024-03-06T07:25:19.804333+0000","last_undegraded":"2024-03-06T07:25:19.804333+0000","last_fullsized":"2024-03-06T07:25:19.804333+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:54.608998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792586+0000","last_change":"2024-03-06T07:25:19.792586+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792586+0000","last_undegraded":"2024-03-06T07:25:19.792586+0000","last_fullsized":"2024-03-06T07:25:19.792586+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798951+0000","last_change":"2024-03-06T07:25:19.793750+0000","last_active":"2024-03-06T07:25:19.798951+0000","last_peered":"2024-03-06T07:25:19.798951+0000","last_clean":"2024-03-06T07:25:19.798951+0000","last_became_active":"2024-03-06T07:25:19.793481+0000","last_became_peered":"2024-03-06T07:25:19.793481+0000","last_unstale":"2024-03-06T07:25:19.798951+0000","last_undegraded":"2024-03-06T07:25:19.798951+0000","last_fullsized":"2024-03-06T07:25:19.798951+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:58.530205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789100+0000","last_change":"2024-03-06T07:25:19.789100+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789100+0000","last_undegraded":"2024-03-06T07:25:19.789100+0000","last_fullsized":"2024-03-06T07:25:19.789100+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.794368+0000","last_change":"2024-03-06T07:25:19.793729+0000","last_active":"2024-03-06T07:25:19.794368+0000","last_peered":"2024-03-06T07:25:19.794368+0000","last_clean":"2024-03-06T07:25:19.794368+0000","last_became_active":"2024-03-06T07:25:19.793432+0000","last_became_peered":"2024-03-06T07:25:19.793432+0000","last_unstale":"2024-03-06T07:25:19.794368+0000","last_undegraded":"2024-03-06T07:25:19.794368+0000","last_fullsized":"2024-03-06T07:25:19.794368+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:20.252960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788943+0000","last_change":"2024-03-06T07:25:19.788943+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788943+0000","last_undegraded":"2024-03-06T07:25:19.788943+0000","last_fullsized":"2024-03-06T07:25:19.788943+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795674+0000","last_change":"2024-03-06T07:25:19.793893+0000","last_active":"2024-03-06T07:25:19.795674+0000","last_peered":"2024-03-06T07:25:19.795674+0000","last_clean":"2024-03-06T07:25:19.795674+0000","last_became_active":"2024-03-06T07:25:19.793604+0000","last_became_peered":"2024-03-06T07:25:19.793604+0000","last_unstale":"2024-03-06T07:25:19.795674+0000","last_undegraded":"2024-03-06T07:25:19.795674+0000","last_fullsized":"2024-03-06T07:25:19.795674+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:06.983583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.781837+0000","last_change":"2024-03-06T07:25:19.781837+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.781837+0000","last_undegraded":"2024-03-06T07:25:19.781837+0000","last_fullsized":"2024-03-06T07:25:19.781837+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801503+0000","last_change":"2024-03-06T07:25:19.797615+0000","last_active":"2024-03-06T07:25:19.801503+0000","last_peered":"2024-03-06T07:25:19.801503+0000","last_clean":"2024-03-06T07:25:19.801503+0000","last_became_active":"2024-03-06T07:25:19.797345+0000","last_became_peered":"2024-03-06T07:25:19.797345+0000","last_unstale":"2024-03-06T07:25:19.801503+0000","last_undegraded":"2024-03-06T07:25:19.801503+0000","last_fullsized":"2024-03-06T07:25:19.801503+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:26.870856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788147+0000","last_change":"2024-03-06T07:25:19.788147+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788147+0000","last_undegraded":"2024-03-06T07:25:19.788147+0000","last_fullsized":"2024-03-06T07:25:19.788147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792242+0000","last_change":"2024-03-06T07:25:19.791673+0000","last_active":"2024-03-06T07:25:19.792242+0000","last_peered":"2024-03-06T07:25:19.792242+0000","last_clean":"2024-03-06T07:25:19.792242+0000","last_became_active":"2024-03-06T07:25:19.791395+0000","last_became_peered":"2024-03-06T07:25:19.791395+0000","last_unstale":"2024-03-06T07:25:19.792242+0000","last_undegraded":"2024-03-06T07:25:19.792242+0000","last_fullsized":"2024-03-06T07:25:19.792242+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:26:07.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785270+0000","last_change":"2024-03-06T07:25:19.785270+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785270+0000","last_undegraded":"2024-03-06T07:25:19.785270+0000","last_fullsized":"2024-03-06T07:25:19.785270+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798915+0000","last_change":"2024-03-06T07:25:19.794392+0000","last_active":"2024-03-06T07:25:19.798915+0000","last_peered":"2024-03-06T07:25:19.798915+0000","last_clean":"2024-03-06T07:25:19.798915+0000","last_became_active":"2024-03-06T07:25:19.794116+0000","last_became_peered":"2024-03-06T07:25:19.794116+0000","last_unstale":"2024-03-06T07:25:19.798915+0000","last_undegraded":"2024-03-06T07:25:19.798915+0000","last_fullsized":"2024-03-06T07:25:19.798915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:20.252960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784010+0000","last_change":"2024-03-06T07:25:19.784010+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784010+0000","last_undegraded":"2024-03-06T07:25:19.784010+0000","last_fullsized":"2024-03-06T07:25:19.784010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795008+0000","last_change":"2024-03-06T07:25:19.793614+0000","last_active":"2024-03-06T07:25:19.795008+0000","last_peered":"2024-03-06T07:25:19.795008+0000","last_clean":"2024-03-06T07:25:19.795008+0000","last_became_active":"2024-03-06T07:25:19.793338+0000","last_became_peered":"2024-03-06T07:25:19.793338+0000","last_unstale":"2024-03-06T07:25:19.795008+0000","last_undegraded":"2024-03-06T07:25:19.795008+0000","last_fullsized":"2024-03-06T07:25:19.795008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:05.339137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783262+0000","last_change":"2024-03-06T07:25:19.783262+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783262+0000","last_undegraded":"2024-03-06T07:25:19.783262+0000","last_fullsized":"2024-03-06T07:25:19.783262+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792135+0000","last_change":"2024-03-06T07:25:19.790987+0000","last_active":"2024-03-06T07:25:19.792135+0000","last_peered":"2024-03-06T07:25:19.792135+0000","last_clean":"2024-03-06T07:25:19.792135+0000","last_became_active":"2024-03-06T07:25:19.790677+0000","last_became_peered":"2024-03-06T07:25:19.790677+0000","last_unstale":"2024-03-06T07:25:19.792135+0000","last_undegraded":"2024-03-06T07:25:19.792135+0000","last_fullsized":"2024-03-06T07:25:19.792135+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:07.375118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":382,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.785971+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:25:19.785971+0000","last_peered":"2024-03-06T07:25:19.785971+0000","last_clean":"2024-03-06T07:25:19.785971+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:25:19.785971+0000","last_undegraded":"2024-03-06T07:25:19.785971+0000","last_fullsized":"2024-03-06T07:25:19.785971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1724416,"data_stored":1387718,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":804017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":804017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":791141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1387718,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:23.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:23.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.935+0000 7f0302bd0640 1 Processor -- start 2024-03-06T07:25:23.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.935+0000 7f0302bd0640 1 -- start start 2024-03-06T07:25:23.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f0302bd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f0302bd0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f02fc1592d0 0x7f02fc0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f0302bd0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc072630 con 0x7f02fc06da40 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f0302bd0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02fc06de20 con 0x7f02fc1592d0 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f0302bd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc072910 con 0x7f02fc0c6780 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f030116d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f02fc1592d0 0x7f02fc0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f02fbfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f030096c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f02fc06da40 0x7f02fc155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60850/0 (socket says 172.21.15.26:60850) 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f030096c640 1 -- 172.21.15.26:0/588299154 learned_addr learned my addr 172.21.15.26:0/588299154 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.936+0000 7f030116d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f02fc1592d0 0x7f02fc0c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55214/0 (socket says 172.21.15.26:55214) 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.937+0000 7f02fb7fe640 1 -- 172.21.15.26:0/588299154 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204887515 0 0) 0x7f02fc072630 con 0x7f02fc06da40 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.937+0000 7f02fb7fe640 1 -- 172.21.15.26:0/588299154 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e4003520 con 0x7f02fc06da40 2024-03-06T07:25:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.937+0000 7f030116d640 1 -- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 msgr2=0x7f02fc158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.937+0000 7f030116d640 1 --2- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.937+0000 7f030116d640 1 -- 172.21.15.26:0/588299154 >> v1:172.21.15.117:6789/0 conn(0x7f02fc06da40 legacy=0x7f02fc155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.937+0000 7f030116d640 1 -- 172.21.15.26:0/588299154 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc072bf0 con 0x7f02fc1592d0 2024-03-06T07:25:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.937+0000 7f02fbfff640 1 --2- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f030116d640 1 --2- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc1592d0 0x7f02fc0c3280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009d90 tx=0x7f02f402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=861fe79598bcc6d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f02fb7fe640 1 -- 172.21.15.26:0/588299154 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4039070 con 0x7f02fc1592d0 2024-03-06T07:25:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f02fb7fe640 1 -- 172.21.15.26:0/588299154 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f402cbc0 con 0x7f02fc1592d0 2024-03-06T07:25:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f02fb7fe640 1 -- 172.21.15.26:0/588299154 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4034440 con 0x7f02fc1592d0 2024-03-06T07:25:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 -- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc1592d0 msgr2=0x7f02fc0c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 --2- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc1592d0 0x7f02fc0c3280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009d90 tx=0x7f02f402be10 comp rx=0 tx=0).stop 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 -- 172.21.15.26:0/588299154 shutdown_connections 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 --2- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc1592d0 0x7f02fc0c3280 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 --2- 172.21.15.26:0/588299154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 -- 172.21.15.26:0/588299154 >> 172.21.15.26:0/588299154 conn(0x7f02fc0aa630 msgr2=0x7f02fc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 -- 172.21.15.26:0/588299154 shutdown_connections 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.938+0000 7f0302bd0640 1 -- 172.21.15.26:0/588299154 wait complete. 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.939+0000 7f0302bd0640 1 Processor -- start 2024-03-06T07:25:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.939+0000 7f0302bd0640 1 -- start start 2024-03-06T07:25:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f0302bd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc06da40 0x7f02fc14c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f0302bd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc0ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f0302bd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02fc1592d0 0x7f02fc1eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f0302bd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0c3940 con 0x7f02fc1592d0 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f0302bd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0c37c0 con 0x7f02fc06da40 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f0302bd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0c3ac0 con 0x7f02fc0c6780 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f030116d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02fc1592d0 0x7f02fc1eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f030116d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02fc1592d0 0x7f02fc1eaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37512/0 (socket says 172.21.15.26:37512) 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.940+0000 7f030116d640 1 -- 172.21.15.26:0/631413507 learned_addr learned my addr 172.21.15.26:0/631413507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030096c640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc06da40 0x7f02fc14c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f02fbfff640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc0ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030096c640 1 -- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 msgr2=0x7f02fc0ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030096c640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc0ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030096c640 1 -- 172.21.15.26:0/631413507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02fc1592d0 msgr2=0x7f02fc1eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030096c640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02fc1592d0 0x7f02fc1eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030096c640 1 -- 172.21.15.26:0/631413507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc070200 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030116d640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02fc1592d0 0x7f02fc1eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f030096c640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc06da40 0x7f02fc14c000 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c390 tx=0x7f02ec001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f02f97fa640 1 -- 172.21.15.26:0/631413507 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec017070 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f02f97fa640 1 -- 172.21.15.26:0/631413507 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec005ce0 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02fc0ae2c0 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f02f97fa640 1 -- 172.21.15.26:0/631413507 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec0072f0 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.941+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc0aa2e0 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.942+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc04f330 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.943+0000 7f02f97fa640 1 -- 172.21.15.26:0/631413507 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02ec01a070 con 0x7f02fc06da40 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.943+0000 7f02f97fa640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f02e806b4d0 0x7f02e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.943+0000 7f02f97fa640 1 -- 172.21.15.26:0/631413507 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02ec0895f0 con 0x7f02fc06da40 2024-03-06T07:25:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.943+0000 7f02fbfff640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f02e806b4d0 0x7f02e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.944+0000 7f02fbfff640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f02e806b4d0 0x7f02e806d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f02fc152990 tx=0x7f02f0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:23.948+0000 7f02f97fa640 1 -- 172.21.15.26:0/631413507 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f02ec057c80 con 0x7f02fc06da40 2024-03-06T07:25:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.125+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02fc0ac070 con 0x7f02e806b4d0 2024-03-06T07:25:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.131+0000 7f02f97fa640 1 -- 172.21.15.26:0/631413507 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247164 (secure 0 0 0) 0x7f02fc0ac070 con 0x7f02e806b4d0 2024-03-06T07:25:24.135 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f02e806b4d0 msgr2=0x7f02e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f02e806b4d0 0x7f02e806d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f02fc152990 tx=0x7f02f0009190 comp rx=0 tx=0).stop 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc06da40 msgr2=0x7f02fc14c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc06da40 0x7f02fc14c000 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c390 tx=0x7f02ec001e70 comp rx=0 tx=0).stop 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 shutdown_connections 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f02e806b4d0 0x7f02e806d990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02fc1592d0 0x7f02fc1eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02fc0c6780 0x7f02fc0ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 --2- 172.21.15.26:0/631413507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02fc06da40 0x7f02fc14c000 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 >> 172.21.15.26:0/631413507 conn(0x7f02fc0aa630 msgr2=0x7f02fc158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 shutdown_connections 2024-03-06T07:25:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.134+0000 7f0302bd0640 1 -- 172.21.15.26:0/631413507 wait complete. 2024-03-06T07:25:24.146 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-03-06T07:25:23.097094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262564,"kb_used_data":13100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712156,"statfs":{"total":773094113280,"available":772825247744,"internally_reserved":0,"allocated":13414400,"data_stored":7566661,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319375"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790263+0000","last_change":"2024-03-06T07:25:19.790263+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790263+0000","last_undegraded":"2024-03-06T07:25:19.790263+0000","last_fullsized":"2024-03-06T07:25:19.790263+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788156+0000","last_change":"2024-03-06T07:25:19.788156+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788156+0000","last_undegraded":"2024-03-06T07:25:19.788156+0000","last_fullsized":"2024-03-06T07:25:19.788156+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787767+0000","last_change":"2024-03-06T07:25:19.787767+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787767+0000","last_undegraded":"2024-03-06T07:25:19.787767+0000","last_fullsized":"2024-03-06T07:25:19.787767+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788939+0000","last_change":"2024-03-06T07:25:19.788939+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788939+0000","last_undegraded":"2024-03-06T07:25:19.788939+0000","last_fullsized":"2024-03-06T07:25:19.788939+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.795996+0000","last_change":"2024-03-06T07:25:19.795996+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.795996+0000","last_undegraded":"2024-03-06T07:25:19.795996+0000","last_fullsized":"2024-03-06T07:25:19.795996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.795192+0000","last_change":"2024-03-06T07:25:19.795192+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.795192+0000","last_undegraded":"2024-03-06T07:25:19.795192+0000","last_fullsized":"2024-03-06T07:25:19.795192+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.791435+0000","last_change":"2024-03-06T07:25:19.791435+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.791435+0000","last_undegraded":"2024-03-06T07:25:19.791435+0000","last_fullsized":"2024-03-06T07:25:19.791435+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784134+0000","last_change":"2024-03-06T07:25:19.784134+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784134+0000","last_undegraded":"2024-03-06T07:25:19.784134+0000","last_fullsized":"2024-03-06T07:25:19.784134+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792512+0000","last_change":"2024-03-06T07:25:19.792512+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792512+0000","last_undegraded":"2024-03-06T07:25:19.792512+0000","last_fullsized":"2024-03-06T07:25:19.792512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792010+0000","last_change":"2024-03-06T07:25:19.792010+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792010+0000","last_undegraded":"2024-03-06T07:25:19.792010+0000","last_fullsized":"2024-03-06T07:25:19.792010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785099+0000","last_change":"2024-03-06T07:25:19.785099+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785099+0000","last_undegraded":"2024-03-06T07:25:19.785099+0000","last_fullsized":"2024-03-06T07:25:19.785099+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788546+0000","last_change":"2024-03-06T07:25:19.788546+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788546+0000","last_undegraded":"2024-03-06T07:25:19.788546+0000","last_fullsized":"2024-03-06T07:25:19.788546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.793411+0000","last_change":"2024-03-06T07:25:19.793411+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.793411+0000","last_undegraded":"2024-03-06T07:25:19.793411+0000","last_fullsized":"2024-03-06T07:25:19.793411+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789475+0000","last_change":"2024-03-06T07:25:19.789475+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789475+0000","last_undegraded":"2024-03-06T07:25:19.789475+0000","last_fullsized":"2024-03-06T07:25:19.789475+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789405+0000","last_change":"2024-03-06T07:25:19.789405+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789405+0000","last_undegraded":"2024-03-06T07:25:19.789405+0000","last_fullsized":"2024-03-06T07:25:19.789405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790829+0000","last_change":"2024-03-06T07:25:19.790829+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790829+0000","last_undegraded":"2024-03-06T07:25:19.790829+0000","last_fullsized":"2024-03-06T07:25:19.790829+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.793506+0000","last_change":"2024-03-06T07:25:19.793506+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.793506+0000","last_undegraded":"2024-03-06T07:25:19.793506+0000","last_fullsized":"2024-03-06T07:25:19.793506+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784444+0000","last_change":"2024-03-06T07:25:19.784444+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784444+0000","last_undegraded":"2024-03-06T07:25:19.784444+0000","last_fullsized":"2024-03-06T07:25:19.784444+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785275+0000","last_change":"2024-03-06T07:25:19.785275+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785275+0000","last_undegraded":"2024-03-06T07:25:19.785275+0000","last_fullsized":"2024-03-06T07:25:19.785275+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785315+0000","last_change":"2024-03-06T07:25:19.785315+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785315+0000","last_undegraded":"2024-03-06T07:25:19.785315+0000","last_fullsized":"2024-03-06T07:25:19.785315+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792999+0000","last_change":"2024-03-06T07:25:19.792999+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792999+0000","last_undegraded":"2024-03-06T07:25:19.792999+0000","last_fullsized":"2024-03-06T07:25:19.792999+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788950+0000","last_change":"2024-03-06T07:25:19.788950+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788950+0000","last_undegraded":"2024-03-06T07:25:19.788950+0000","last_fullsized":"2024-03-06T07:25:19.788950+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783383+0000","last_change":"2024-03-06T07:25:19.783383+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783383+0000","last_undegraded":"2024-03-06T07:25:19.783383+0000","last_fullsized":"2024-03-06T07:25:19.783383+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784142+0000","last_change":"2024-03-06T07:25:19.784142+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784142+0000","last_undegraded":"2024-03-06T07:25:19.784142+0000","last_fullsized":"2024-03-06T07:25:19.784142+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789851+0000","last_change":"2024-03-06T07:25:19.789851+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789851+0000","last_undegraded":"2024-03-06T07:25:19.789851+0000","last_fullsized":"2024-03-06T07:25:19.789851+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790728+0000","last_change":"2024-03-06T07:25:19.790728+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790728+0000","last_undegraded":"2024-03-06T07:25:19.790728+0000","last_fullsized":"2024-03-06T07:25:19.790728+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.791445+0000","last_change":"2024-03-06T07:25:19.791445+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.791445+0000","last_undegraded":"2024-03-06T07:25:19.791445+0000","last_fullsized":"2024-03-06T07:25:19.791445+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784190+0000","last_change":"2024-03-06T07:25:19.784190+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784190+0000","last_undegraded":"2024-03-06T07:25:19.784190+0000","last_fullsized":"2024-03-06T07:25:19.784190+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783073+0000","last_change":"2024-03-06T07:25:19.783073+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783073+0000","last_undegraded":"2024-03-06T07:25:19.783073+0000","last_fullsized":"2024-03-06T07:25:19.783073+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784550+0000","last_change":"2024-03-06T07:25:19.784550+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784550+0000","last_undegraded":"2024-03-06T07:25:19.784550+0000","last_fullsized":"2024-03-06T07:25:19.784550+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790473+0000","last_change":"2024-03-06T07:25:19.790473+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790473+0000","last_undegraded":"2024-03-06T07:25:19.790473+0000","last_fullsized":"2024-03-06T07:25:19.790473+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783060+0000","last_change":"2024-03-06T07:25:19.783060+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783060+0000","last_undegraded":"2024-03-06T07:25:19.783060+0000","last_fullsized":"2024-03-06T07:25:19.783060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788370+0000","last_change":"2024-03-06T07:25:19.788370+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788370+0000","last_undegraded":"2024-03-06T07:25:19.788370+0000","last_fullsized":"2024-03-06T07:25:19.788370+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.800985+0000","last_change":"2024-03-06T07:25:19.800026+0000","last_active":"2024-03-06T07:25:19.800985+0000","last_peered":"2024-03-06T07:25:19.800985+0000","last_clean":"2024-03-06T07:25:19.800985+0000","last_became_active":"2024-03-06T07:25:19.799694+0000","last_became_peered":"2024-03-06T07:25:19.799694+0000","last_unstale":"2024-03-06T07:25:19.800985+0000","last_undegraded":"2024-03-06T07:25:19.800985+0000","last_fullsized":"2024-03-06T07:25:19.800985+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:05.339137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787369+0000","last_change":"2024-03-06T07:25:19.787369+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787369+0000","last_undegraded":"2024-03-06T07:25:19.787369+0000","last_fullsized":"2024-03-06T07:25:19.787369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.796334+0000","last_change":"2024-03-06T07:25:19.794850+0000","last_active":"2024-03-06T07:25:19.796334+0000","last_peered":"2024-03-06T07:25:19.796334+0000","last_clean":"2024-03-06T07:25:19.796334+0000","last_became_active":"2024-03-06T07:25:19.794587+0000","last_became_peered":"2024-03-06T07:25:19.794587+0000","last_unstale":"2024-03-06T07:25:19.796334+0000","last_undegraded":"2024-03-06T07:25:19.796334+0000","last_fullsized":"2024-03-06T07:25:19.796334+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:28.129920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782596+0000","last_change":"2024-03-06T07:25:19.782596+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782596+0000","last_undegraded":"2024-03-06T07:25:19.782596+0000","last_fullsized":"2024-03-06T07:25:19.782596+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798990+0000","last_change":"2024-03-06T07:25:19.796329+0000","last_active":"2024-03-06T07:25:19.798990+0000","last_peered":"2024-03-06T07:25:19.798990+0000","last_clean":"2024-03-06T07:25:19.798990+0000","last_became_active":"2024-03-06T07:25:19.796081+0000","last_became_peered":"2024-03-06T07:25:19.796081+0000","last_unstale":"2024-03-06T07:25:19.798990+0000","last_undegraded":"2024-03-06T07:25:19.798990+0000","last_fullsized":"2024-03-06T07:25:19.798990+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:18.223482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784369+0000","last_change":"2024-03-06T07:25:19.784369+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784369+0000","last_undegraded":"2024-03-06T07:25:19.784369+0000","last_fullsized":"2024-03-06T07:25:19.784369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.793189+0000","last_change":"2024-03-06T07:25:19.789450+0000","last_active":"2024-03-06T07:25:19.793189+0000","last_peered":"2024-03-06T07:25:19.793189+0000","last_clean":"2024-03-06T07:25:19.793189+0000","last_became_active":"2024-03-06T07:25:19.789139+0000","last_became_peered":"2024-03-06T07:25:19.789139+0000","last_unstale":"2024-03-06T07:25:19.793189+0000","last_undegraded":"2024-03-06T07:25:19.793189+0000","last_fullsized":"2024-03-06T07:25:19.793189+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:58.530205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785243+0000","last_change":"2024-03-06T07:25:19.785243+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785243+0000","last_undegraded":"2024-03-06T07:25:19.785243+0000","last_fullsized":"2024-03-06T07:25:19.785243+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.800808+0000","last_change":"2024-03-06T07:25:19.797666+0000","last_active":"2024-03-06T07:25:19.800808+0000","last_peered":"2024-03-06T07:25:19.800808+0000","last_clean":"2024-03-06T07:25:19.800808+0000","last_became_active":"2024-03-06T07:25:19.797391+0000","last_became_peered":"2024-03-06T07:25:19.797391+0000","last_unstale":"2024-03-06T07:25:19.800808+0000","last_undegraded":"2024-03-06T07:25:19.800808+0000","last_fullsized":"2024-03-06T07:25:19.800808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:48.223756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790355+0000","last_change":"2024-03-06T07:25:19.790355+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790355+0000","last_undegraded":"2024-03-06T07:25:19.790355+0000","last_fullsized":"2024-03-06T07:25:19.790355+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801006+0000","last_change":"2024-03-06T07:25:19.799648+0000","last_active":"2024-03-06T07:25:19.801006+0000","last_peered":"2024-03-06T07:25:19.801006+0000","last_clean":"2024-03-06T07:25:19.801006+0000","last_became_active":"2024-03-06T07:25:19.799352+0000","last_became_peered":"2024-03-06T07:25:19.799352+0000","last_unstale":"2024-03-06T07:25:19.801006+0000","last_undegraded":"2024-03-06T07:25:19.801006+0000","last_fullsized":"2024-03-06T07:25:19.801006+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:30.875962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787231+0000","last_change":"2024-03-06T07:25:19.787231+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787231+0000","last_undegraded":"2024-03-06T07:25:19.787231+0000","last_fullsized":"2024-03-06T07:25:19.787231+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801537+0000","last_change":"2024-03-06T07:25:19.797676+0000","last_active":"2024-03-06T07:25:19.801537+0000","last_peered":"2024-03-06T07:25:19.801537+0000","last_clean":"2024-03-06T07:25:19.801537+0000","last_became_active":"2024-03-06T07:25:19.797399+0000","last_became_peered":"2024-03-06T07:25:19.797399+0000","last_unstale":"2024-03-06T07:25:19.801537+0000","last_undegraded":"2024-03-06T07:25:19.801537+0000","last_fullsized":"2024-03-06T07:25:19.801537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:34.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788917+0000","last_change":"2024-03-06T07:25:19.788917+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788917+0000","last_undegraded":"2024-03-06T07:25:19.788917+0000","last_fullsized":"2024-03-06T07:25:19.788917+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.802569+0000","last_change":"2024-03-06T07:25:19.802156+0000","last_active":"2024-03-06T07:25:19.802569+0000","last_peered":"2024-03-06T07:25:19.802569+0000","last_clean":"2024-03-06T07:25:19.802569+0000","last_became_active":"2024-03-06T07:25:19.801881+0000","last_became_peered":"2024-03-06T07:25:19.801881+0000","last_unstale":"2024-03-06T07:25:19.802569+0000","last_undegraded":"2024-03-06T07:25:19.802569+0000","last_fullsized":"2024-03-06T07:25:19.802569+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:14.322866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782407+0000","last_change":"2024-03-06T07:25:19.782407+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782407+0000","last_undegraded":"2024-03-06T07:25:19.782407+0000","last_fullsized":"2024-03-06T07:25:19.782407+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798410+0000","last_change":"2024-03-06T07:25:19.795560+0000","last_active":"2024-03-06T07:25:19.798410+0000","last_peered":"2024-03-06T07:25:19.798410+0000","last_clean":"2024-03-06T07:25:19.798410+0000","last_became_active":"2024-03-06T07:25:19.795293+0000","last_became_peered":"2024-03-06T07:25:19.795293+0000","last_unstale":"2024-03-06T07:25:19.798410+0000","last_undegraded":"2024-03-06T07:25:19.798410+0000","last_fullsized":"2024-03-06T07:25:19.798410+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:14.322866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784536+0000","last_change":"2024-03-06T07:25:19.784536+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784536+0000","last_undegraded":"2024-03-06T07:25:19.784536+0000","last_fullsized":"2024-03-06T07:25:19.784536+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804177+0000","last_change":"2024-03-06T07:25:19.798578+0000","last_active":"2024-03-06T07:25:19.804177+0000","last_peered":"2024-03-06T07:25:19.804177+0000","last_clean":"2024-03-06T07:25:19.804177+0000","last_became_active":"2024-03-06T07:25:19.798263+0000","last_became_peered":"2024-03-06T07:25:19.798263+0000","last_unstale":"2024-03-06T07:25:19.804177+0000","last_undegraded":"2024-03-06T07:25:19.804177+0000","last_fullsized":"2024-03-06T07:25:19.804177+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:51.204621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783803+0000","last_change":"2024-03-06T07:25:19.783803+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783803+0000","last_undegraded":"2024-03-06T07:25:19.783803+0000","last_fullsized":"2024-03-06T07:25:19.783803+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.793843+0000","last_change":"2024-03-06T07:25:19.792711+0000","last_active":"2024-03-06T07:25:19.793843+0000","last_peered":"2024-03-06T07:25:19.793843+0000","last_clean":"2024-03-06T07:25:19.793843+0000","last_became_active":"2024-03-06T07:25:19.792405+0000","last_became_peered":"2024-03-06T07:25:19.792405+0000","last_unstale":"2024-03-06T07:25:19.793843+0000","last_undegraded":"2024-03-06T07:25:19.793843+0000","last_fullsized":"2024-03-06T07:25:19.793843+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:40.261389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782423+0000","last_change":"2024-03-06T07:25:19.782423+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782423+0000","last_undegraded":"2024-03-06T07:25:19.782423+0000","last_fullsized":"2024-03-06T07:25:19.782423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801343+0000","last_change":"2024-03-06T07:25:19.800658+0000","last_active":"2024-03-06T07:25:19.801343+0000","last_peered":"2024-03-06T07:25:19.801343+0000","last_clean":"2024-03-06T07:25:19.801343+0000","last_became_active":"2024-03-06T07:25:19.800361+0000","last_became_peered":"2024-03-06T07:25:19.800361+0000","last_unstale":"2024-03-06T07:25:19.801343+0000","last_undegraded":"2024-03-06T07:25:19.801343+0000","last_fullsized":"2024-03-06T07:25:19.801343+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:28.129920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788966+0000","last_change":"2024-03-06T07:25:19.788966+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788966+0000","last_undegraded":"2024-03-06T07:25:19.788966+0000","last_fullsized":"2024-03-06T07:25:19.788966+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804522+0000","last_change":"2024-03-06T07:25:19.802236+0000","last_active":"2024-03-06T07:25:19.804522+0000","last_peered":"2024-03-06T07:25:19.804522+0000","last_clean":"2024-03-06T07:25:19.804522+0000","last_became_active":"2024-03-06T07:25:19.801866+0000","last_became_peered":"2024-03-06T07:25:19.801866+0000","last_unstale":"2024-03-06T07:25:19.804522+0000","last_undegraded":"2024-03-06T07:25:19.804522+0000","last_fullsized":"2024-03-06T07:25:19.804522+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:02.212979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790418+0000","last_change":"2024-03-06T07:25:19.790418+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790418+0000","last_undegraded":"2024-03-06T07:25:19.790418+0000","last_fullsized":"2024-03-06T07:25:19.790418+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804286+0000","last_change":"2024-03-06T07:25:19.800076+0000","last_active":"2024-03-06T07:25:19.804286+0000","last_peered":"2024-03-06T07:25:19.804286+0000","last_clean":"2024-03-06T07:25:19.804286+0000","last_became_active":"2024-03-06T07:25:19.799501+0000","last_became_peered":"2024-03-06T07:25:19.799501+0000","last_unstale":"2024-03-06T07:25:19.804286+0000","last_undegraded":"2024-03-06T07:25:19.804286+0000","last_fullsized":"2024-03-06T07:25:19.804286+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:56.743191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786604+0000","last_change":"2024-03-06T07:25:19.786604+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786604+0000","last_undegraded":"2024-03-06T07:25:19.786604+0000","last_fullsized":"2024-03-06T07:25:19.786604+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801485+0000","last_change":"2024-03-06T07:25:19.797067+0000","last_active":"2024-03-06T07:25:19.801485+0000","last_peered":"2024-03-06T07:25:19.801485+0000","last_clean":"2024-03-06T07:25:19.801485+0000","last_became_active":"2024-03-06T07:25:19.796773+0000","last_became_peered":"2024-03-06T07:25:19.796773+0000","last_unstale":"2024-03-06T07:25:19.801485+0000","last_undegraded":"2024-03-06T07:25:19.801485+0000","last_fullsized":"2024-03-06T07:25:19.801485+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:08.144349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784876+0000","last_change":"2024-03-06T07:25:19.784876+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784876+0000","last_undegraded":"2024-03-06T07:25:19.784876+0000","last_fullsized":"2024-03-06T07:25:19.784876+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.790405+0000","last_change":"2024-03-06T07:25:19.789597+0000","last_active":"2024-03-06T07:25:19.790405+0000","last_peered":"2024-03-06T07:25:19.790405+0000","last_clean":"2024-03-06T07:25:19.790405+0000","last_became_active":"2024-03-06T07:25:19.789334+0000","last_became_peered":"2024-03-06T07:25:19.789334+0000","last_unstale":"2024-03-06T07:25:19.790405+0000","last_undegraded":"2024-03-06T07:25:19.790405+0000","last_fullsized":"2024-03-06T07:25:19.790405+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:45.098378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784468+0000","last_change":"2024-03-06T07:25:19.784468+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784468+0000","last_undegraded":"2024-03-06T07:25:19.784468+0000","last_fullsized":"2024-03-06T07:25:19.784468+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799011+0000","last_change":"2024-03-06T07:25:19.796190+0000","last_active":"2024-03-06T07:25:19.799011+0000","last_peered":"2024-03-06T07:25:19.799011+0000","last_clean":"2024-03-06T07:25:19.799011+0000","last_became_active":"2024-03-06T07:25:19.795879+0000","last_became_peered":"2024-03-06T07:25:19.795879+0000","last_unstale":"2024-03-06T07:25:19.799011+0000","last_undegraded":"2024-03-06T07:25:19.799011+0000","last_fullsized":"2024-03-06T07:25:19.799011+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:59.395378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790299+0000","last_change":"2024-03-06T07:25:19.790299+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790299+0000","last_undegraded":"2024-03-06T07:25:19.790299+0000","last_fullsized":"2024-03-06T07:25:19.790299+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799742+0000","last_change":"2024-03-06T07:25:19.798347+0000","last_active":"2024-03-06T07:25:19.799742+0000","last_peered":"2024-03-06T07:25:19.799742+0000","last_clean":"2024-03-06T07:25:19.799742+0000","last_became_active":"2024-03-06T07:25:19.798021+0000","last_became_peered":"2024-03-06T07:25:19.798021+0000","last_unstale":"2024-03-06T07:25:19.799742+0000","last_undegraded":"2024-03-06T07:25:19.799742+0000","last_fullsized":"2024-03-06T07:25:19.799742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:08.144349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789405+0000","last_change":"2024-03-06T07:25:19.789405+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789405+0000","last_undegraded":"2024-03-06T07:25:19.789405+0000","last_fullsized":"2024-03-06T07:25:19.789405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804335+0000","last_change":"2024-03-06T07:25:19.801586+0000","last_active":"2024-03-06T07:25:19.804335+0000","last_peered":"2024-03-06T07:25:19.804335+0000","last_clean":"2024-03-06T07:25:19.804335+0000","last_became_active":"2024-03-06T07:25:19.801242+0000","last_became_peered":"2024-03-06T07:25:19.801242+0000","last_unstale":"2024-03-06T07:25:19.804335+0000","last_undegraded":"2024-03-06T07:25:19.804335+0000","last_fullsized":"2024-03-06T07:25:19.804335+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:39.912413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784346+0000","last_change":"2024-03-06T07:25:19.784346+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784346+0000","last_undegraded":"2024-03-06T07:25:19.784346+0000","last_fullsized":"2024-03-06T07:25:19.784346+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.797760+0000","last_change":"2024-03-06T07:25:19.796315+0000","last_active":"2024-03-06T07:25:19.797760+0000","last_peered":"2024-03-06T07:25:19.797760+0000","last_clean":"2024-03-06T07:25:19.797760+0000","last_became_active":"2024-03-06T07:25:19.796036+0000","last_became_peered":"2024-03-06T07:25:19.796036+0000","last_unstale":"2024-03-06T07:25:19.797760+0000","last_undegraded":"2024-03-06T07:25:19.797760+0000","last_fullsized":"2024-03-06T07:25:19.797760+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:15.257584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788465+0000","last_change":"2024-03-06T07:25:19.788465+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788465+0000","last_undegraded":"2024-03-06T07:25:19.788465+0000","last_fullsized":"2024-03-06T07:25:19.788465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792150+0000","last_change":"2024-03-06T07:25:19.789672+0000","last_active":"2024-03-06T07:25:19.792150+0000","last_peered":"2024-03-06T07:25:19.792150+0000","last_clean":"2024-03-06T07:25:19.792150+0000","last_became_active":"2024-03-06T07:25:19.789409+0000","last_became_peered":"2024-03-06T07:25:19.789409+0000","last_unstale":"2024-03-06T07:25:19.792150+0000","last_undegraded":"2024-03-06T07:25:19.792150+0000","last_fullsized":"2024-03-06T07:25:19.792150+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:07.534390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784743+0000","last_change":"2024-03-06T07:25:19.784743+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784743+0000","last_undegraded":"2024-03-06T07:25:19.784743+0000","last_fullsized":"2024-03-06T07:25:19.784743+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799242+0000","last_change":"2024-03-06T07:25:19.797588+0000","last_active":"2024-03-06T07:25:19.799242+0000","last_peered":"2024-03-06T07:25:19.799242+0000","last_clean":"2024-03-06T07:25:19.799242+0000","last_became_active":"2024-03-06T07:25:19.797322+0000","last_became_peered":"2024-03-06T07:25:19.797322+0000","last_unstale":"2024-03-06T07:25:19.799242+0000","last_undegraded":"2024-03-06T07:25:19.799242+0000","last_fullsized":"2024-03-06T07:25:19.799242+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:19.871887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786003+0000","last_change":"2024-03-06T07:25:19.786003+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786003+0000","last_undegraded":"2024-03-06T07:25:19.786003+0000","last_fullsized":"2024-03-06T07:25:19.786003+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795817+0000","last_change":"2024-03-06T07:25:19.794012+0000","last_active":"2024-03-06T07:25:19.795817+0000","last_peered":"2024-03-06T07:25:19.795817+0000","last_clean":"2024-03-06T07:25:19.795817+0000","last_became_active":"2024-03-06T07:25:19.793743+0000","last_became_peered":"2024-03-06T07:25:19.793743+0000","last_unstale":"2024-03-06T07:25:19.795817+0000","last_undegraded":"2024-03-06T07:25:19.795817+0000","last_fullsized":"2024-03-06T07:25:19.795817+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:22:21.615129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786591+0000","last_change":"2024-03-06T07:25:19.786591+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786591+0000","last_undegraded":"2024-03-06T07:25:19.786591+0000","last_fullsized":"2024-03-06T07:25:19.786591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804333+0000","last_change":"2024-03-06T07:25:19.801518+0000","last_active":"2024-03-06T07:25:19.804333+0000","last_peered":"2024-03-06T07:25:19.804333+0000","last_clean":"2024-03-06T07:25:19.804333+0000","last_became_active":"2024-03-06T07:25:19.801213+0000","last_became_peered":"2024-03-06T07:25:19.801213+0000","last_unstale":"2024-03-06T07:25:19.804333+0000","last_undegraded":"2024-03-06T07:25:19.804333+0000","last_fullsized":"2024-03-06T07:25:19.804333+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:54.608998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792586+0000","last_change":"2024-03-06T07:25:19.792586+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792586+0000","last_undegraded":"2024-03-06T07:25:19.792586+0000","last_fullsized":"2024-03-06T07:25:19.792586+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798951+0000","last_change":"2024-03-06T07:25:19.793750+0000","last_active":"2024-03-06T07:25:19.798951+0000","last_peered":"2024-03-06T07:25:19.798951+0000","last_clean":"2024-03-06T07:25:19.798951+0000","last_became_active":"2024-03-06T07:25:19.793481+0000","last_became_peered":"2024-03-06T07:25:19.793481+0000","last_unstale":"2024-03-06T07:25:19.798951+0000","last_undegraded":"2024-03-06T07:25:19.798951+0000","last_fullsized":"2024-03-06T07:25:19.798951+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:58.530205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789100+0000","last_change":"2024-03-06T07:25:19.789100+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789100+0000","last_undegraded":"2024-03-06T07:25:19.789100+0000","last_fullsized":"2024-03-06T07:25:19.789100+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.794368+0000","last_change":"2024-03-06T07:25:19.793729+0000","last_active":"2024-03-06T07:25:19.794368+0000","last_peered":"2024-03-06T07:25:19.794368+0000","last_clean":"2024-03-06T07:25:19.794368+0000","last_became_active":"2024-03-06T07:25:19.793432+0000","last_became_peered":"2024-03-06T07:25:19.793432+0000","last_unstale":"2024-03-06T07:25:19.794368+0000","last_undegraded":"2024-03-06T07:25:19.794368+0000","last_fullsized":"2024-03-06T07:25:19.794368+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:20.252960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788943+0000","last_change":"2024-03-06T07:25:19.788943+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788943+0000","last_undegraded":"2024-03-06T07:25:19.788943+0000","last_fullsized":"2024-03-06T07:25:19.788943+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795674+0000","last_change":"2024-03-06T07:25:19.793893+0000","last_active":"2024-03-06T07:25:19.795674+0000","last_peered":"2024-03-06T07:25:19.795674+0000","last_clean":"2024-03-06T07:25:19.795674+0000","last_became_active":"2024-03-06T07:25:19.793604+0000","last_became_peered":"2024-03-06T07:25:19.793604+0000","last_unstale":"2024-03-06T07:25:19.795674+0000","last_undegraded":"2024-03-06T07:25:19.795674+0000","last_fullsized":"2024-03-06T07:25:19.795674+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:06.983583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.781837+0000","last_change":"2024-03-06T07:25:19.781837+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.781837+0000","last_undegraded":"2024-03-06T07:25:19.781837+0000","last_fullsized":"2024-03-06T07:25:19.781837+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801503+0000","last_change":"2024-03-06T07:25:19.797615+0000","last_active":"2024-03-06T07:25:19.801503+0000","last_peered":"2024-03-06T07:25:19.801503+0000","last_clean":"2024-03-06T07:25:19.801503+0000","last_became_active":"2024-03-06T07:25:19.797345+0000","last_became_peered":"2024-03-06T07:25:19.797345+0000","last_unstale":"2024-03-06T07:25:19.801503+0000","last_undegraded":"2024-03-06T07:25:19.801503+0000","last_fullsized":"2024-03-06T07:25:19.801503+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:26.870856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788147+0000","last_change":"2024-03-06T07:25:19.788147+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788147+0000","last_undegraded":"2024-03-06T07:25:19.788147+0000","last_fullsized":"2024-03-06T07:25:19.788147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792242+0000","last_change":"2024-03-06T07:25:19.791673+0000","last_active":"2024-03-06T07:25:19.792242+0000","last_peered":"2024-03-06T07:25:19.792242+0000","last_clean":"2024-03-06T07:25:19.792242+0000","last_became_active":"2024-03-06T07:25:19.791395+0000","last_became_peered":"2024-03-06T07:25:19.791395+0000","last_unstale":"2024-03-06T07:25:19.792242+0000","last_undegraded":"2024-03-06T07:25:19.792242+0000","last_fullsized":"2024-03-06T07:25:19.792242+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:26:07.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785270+0000","last_change":"2024-03-06T07:25:19.785270+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785270+0000","last_undegraded":"2024-03-06T07:25:19.785270+0000","last_fullsized":"2024-03-06T07:25:19.785270+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798915+0000","last_change":"2024-03-06T07:25:19.794392+0000","last_active":"2024-03-06T07:25:19.798915+0000","last_peered":"2024-03-06T07:25:19.798915+0000","last_clean":"2024-03-06T07:25:19.798915+0000","last_became_active":"2024-03-06T07:25:19.794116+0000","last_became_peered":"2024-03-06T07:25:19.794116+0000","last_unstale":"2024-03-06T07:25:19.798915+0000","last_undegraded":"2024-03-06T07:25:19.798915+0000","last_fullsized":"2024-03-06T07:25:19.798915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:20.252960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784010+0000","last_change":"2024-03-06T07:25:19.784010+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784010+0000","last_undegraded":"2024-03-06T07:25:19.784010+0000","last_fullsized":"2024-03-06T07:25:19.784010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795008+0000","last_change":"2024-03-06T07:25:19.793614+0000","last_active":"2024-03-06T07:25:19.795008+0000","last_peered":"2024-03-06T07:25:19.795008+0000","last_clean":"2024-03-06T07:25:19.795008+0000","last_became_active":"2024-03-06T07:25:19.793338+0000","last_became_peered":"2024-03-06T07:25:19.793338+0000","last_unstale":"2024-03-06T07:25:19.795008+0000","last_undegraded":"2024-03-06T07:25:19.795008+0000","last_fullsized":"2024-03-06T07:25:19.795008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:05.339137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783262+0000","last_change":"2024-03-06T07:25:19.783262+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783262+0000","last_undegraded":"2024-03-06T07:25:19.783262+0000","last_fullsized":"2024-03-06T07:25:19.783262+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792135+0000","last_change":"2024-03-06T07:25:19.790987+0000","last_active":"2024-03-06T07:25:19.792135+0000","last_peered":"2024-03-06T07:25:19.792135+0000","last_clean":"2024-03-06T07:25:19.792135+0000","last_became_active":"2024-03-06T07:25:19.790677+0000","last_became_peered":"2024-03-06T07:25:19.790677+0000","last_unstale":"2024-03-06T07:25:19.792135+0000","last_undegraded":"2024-03-06T07:25:19.792135+0000","last_fullsized":"2024-03-06T07:25:19.792135+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:07.375118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":382,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.785971+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:25:19.785971+0000","last_peered":"2024-03-06T07:25:19.785971+0000","last_clean":"2024-03-06T07:25:19.785971+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:25:19.785971+0000","last_undegraded":"2024-03-06T07:25:19.785971+0000","last_fullsized":"2024-03-06T07:25:19.785971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1724416,"data_stored":1387718,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":804017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":804017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":791141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1387718,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:24.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.287+0000 7f95b4895640 1 Processor -- start 2024-03-06T07:25:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.287+0000 7f95b4895640 1 -- start start 2024-03-06T07:25:24.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95b4895640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95b00c88c0 0x7f95b0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95b4895640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00c73d0 con 0x7f95b014f360 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95b4895640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00be2e0 con 0x7f95b0158b60 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95b4895640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b0159700 con 0x7f95b00c88c0 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95a7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95b00c88c0 0x7f95b0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95ae59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95b014f360 0x7f95b00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37264/0 (socket says 172.21.15.26:37264) 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95ae59c640 1 -- 172.21.15.26:0/2211591226 learned_addr learned my addr 172.21.15.26:0/2211591226 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95add9b640 1 -- 172.21.15.26:0/2211591226 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414096671 0 0) 0x7f95b00c73d0 con 0x7f95b014f360 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95add9b640 1 -- 172.21.15.26:0/2211591226 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f959c003520 con 0x7f95b014f360 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95a7fff640 1 -- 172.21.15.26:0/2211591226 >> v1:172.21.15.117:6789/0 conn(0x7f95b0158b60 legacy=0x7f95b0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95a7fff640 1 -- 172.21.15.26:0/2211591226 >> v1:172.21.15.26:6789/0 conn(0x7f95b014f360 legacy=0x7f95b00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.288+0000 7f95a7fff640 1 -- 172.21.15.26:0/2211591226 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00c7670 con 0x7f95b00c88c0 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.289+0000 7f95a7fff640 1 --2- 172.21.15.26:0/2211591226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b00c88c0 0x7f95b0155c10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f95980097a0 tx=0x7f959802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a13fbcd43a35b7e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.289+0000 7f95add9b640 1 -- 172.21.15.26:0/2211591226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95980315a0 con 0x7f95b00c88c0 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.289+0000 7f95add9b640 1 -- 172.21.15.26:0/2211591226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598037070 con 0x7f95b00c88c0 2024-03-06T07:25:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.289+0000 7f95add9b640 1 -- 172.21.15.26:0/2211591226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598031960 con 0x7f95b00c88c0 2024-03-06T07:25:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.290+0000 7f95b4895640 1 -- 172.21.15.26:0/2211591226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b00c88c0 msgr2=0x7f95b0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.290+0000 7f95b4895640 1 --2- 172.21.15.26:0/2211591226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b00c88c0 0x7f95b0155c10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f95980097a0 tx=0x7f959802b840 comp rx=0 tx=0).stop 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.290+0000 7f95b4895640 1 -- 172.21.15.26:0/2211591226 shutdown_connections 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.290+0000 7f95b4895640 1 --2- 172.21.15.26:0/2211591226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b00c88c0 0x7f95b0155c10 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.290+0000 7f95b4895640 1 -- 172.21.15.26:0/2211591226 >> 172.21.15.26:0/2211591226 conn(0x7f95b00ac570 msgr2=0x7f95b00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.290+0000 7f95b4895640 1 -- 172.21.15.26:0/2211591226 shutdown_connections 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.290+0000 7f95b4895640 1 -- 172.21.15.26:0/2211591226 wait complete. 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.291+0000 7f95b4895640 1 Processor -- start 2024-03-06T07:25:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.291+0000 7f95b4895640 1 -- start start 2024-03-06T07:25:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95b4895640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b00c88c0 0x7f95b00c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95b4895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b014f360 0x7f95b01ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95b4895640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95b0158b60 0x7f95b01ec620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95b4895640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00c7c30 con 0x7f95b014f360 2024-03-06T07:25:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95b4895640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00c5a50 con 0x7f95b00c88c0 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95b4895640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00c5bd0 con 0x7f95b0158b60 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b014f360 0x7f95b01ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b014f360 0x7f95b01ec0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37538/0 (socket says 172.21.15.26:37538) 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 -- 172.21.15.26:0/3427816180 learned_addr learned my addr 172.21.15.26:0/3427816180 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95ae59c640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b00c88c0 0x7f95b00c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95aed9d640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95b0158b60 0x7f95b01ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 -- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95b0158b60 msgr2=0x7f95b01ec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95b0158b60 0x7f95b01ec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 -- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b00c88c0 msgr2=0x7f95b00c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b00c88c0 0x7f95b00c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 -- 172.21.15.26:0/3427816180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00bf0a0 con 0x7f95b014f360 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a7fff640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b014f360 0x7f95b01ec0e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9598000c00 tx=0x7f95980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a6ffd640 1 -- 172.21.15.26:0/3427816180 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598032630 con 0x7f95b014f360 2024-03-06T07:25:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.292+0000 7f95a6ffd640 1 -- 172.21.15.26:0/3427816180 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95980327f0 con 0x7f95b014f360 2024-03-06T07:25:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.293+0000 7f95a6ffd640 1 -- 172.21.15.26:0/3427816180 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598032be0 con 0x7f95b014f360 2024-03-06T07:25:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.293+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00bf340 con 0x7f95b014f360 2024-03-06T07:25:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.293+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00bf900 con 0x7f95b014f360 2024-03-06T07:25:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.294+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b004f330 con 0x7f95b014f360 2024-03-06T07:25:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.295+0000 7f95a6ffd640 1 -- 172.21.15.26:0/3427816180 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f959804d3f0 con 0x7f95b014f360 2024-03-06T07:25:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.295+0000 7f95a6ffd640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959006b4d0 0x7f959006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.295+0000 7f95a6ffd640 1 -- 172.21.15.26:0/3427816180 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f959802f030 con 0x7f95b014f360 2024-03-06T07:25:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.295+0000 7f95ae59c640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959006b4d0 0x7f959006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.296+0000 7f95ae59c640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959006b4d0 0x7f959006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f95a0007fd0 tx=0x7f95a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.299+0000 7f95a6ffd640 1 -- 172.21.15.26:0/3427816180 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9598040070 con 0x7f95b014f360 2024-03-06T07:25:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.477+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95b01554a0 con 0x7f959006b4d0 2024-03-06T07:25:24.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.483+0000 7f95a6ffd640 1 -- 172.21.15.26:0/3427816180 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247164 (secure 0 0 0) 0x7f95b01554a0 con 0x7f959006b4d0 2024-03-06T07:25:24.487 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:24.491 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:25:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959006b4d0 msgr2=0x7f959006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959006b4d0 0x7f959006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f95a0007fd0 tx=0x7f95a0009040 comp rx=0 tx=0).stop 2024-03-06T07:25:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b014f360 msgr2=0x7f95b01ec0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b014f360 0x7f95b01ec0e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9598000c00 tx=0x7f95980046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 shutdown_connections 2024-03-06T07:25:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f959006b4d0 0x7f959006d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95b0158b60 0x7f95b01ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b014f360 0x7f95b01ec0e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 --2- 172.21.15.26:0/3427816180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b00c88c0 0x7f95b00c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 >> 172.21.15.26:0/3427816180 conn(0x7f95b00ac570 msgr2=0x7f95b00ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 shutdown_connections 2024-03-06T07:25:24.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.486+0000 7f95b4895640 1 -- 172.21.15.26:0/3427816180 wait complete. 2024-03-06T07:25:24.498 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-03-06T07:25:23.097094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262564,"kb_used_data":13100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712156,"statfs":{"total":773094113280,"available":772825247744,"internally_reserved":0,"allocated":13414400,"data_stored":7566661,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319375"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790263+0000","last_change":"2024-03-06T07:25:19.790263+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790263+0000","last_undegraded":"2024-03-06T07:25:19.790263+0000","last_fullsized":"2024-03-06T07:25:19.790263+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788156+0000","last_change":"2024-03-06T07:25:19.788156+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788156+0000","last_undegraded":"2024-03-06T07:25:19.788156+0000","last_fullsized":"2024-03-06T07:25:19.788156+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787767+0000","last_change":"2024-03-06T07:25:19.787767+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787767+0000","last_undegraded":"2024-03-06T07:25:19.787767+0000","last_fullsized":"2024-03-06T07:25:19.787767+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788939+0000","last_change":"2024-03-06T07:25:19.788939+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788939+0000","last_undegraded":"2024-03-06T07:25:19.788939+0000","last_fullsized":"2024-03-06T07:25:19.788939+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.795996+0000","last_change":"2024-03-06T07:25:19.795996+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.795996+0000","last_undegraded":"2024-03-06T07:25:19.795996+0000","last_fullsized":"2024-03-06T07:25:19.795996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.795192+0000","last_change":"2024-03-06T07:25:19.795192+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.795192+0000","last_undegraded":"2024-03-06T07:25:19.795192+0000","last_fullsized":"2024-03-06T07:25:19.795192+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.791435+0000","last_change":"2024-03-06T07:25:19.791435+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.791435+0000","last_undegraded":"2024-03-06T07:25:19.791435+0000","last_fullsized":"2024-03-06T07:25:19.791435+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784134+0000","last_change":"2024-03-06T07:25:19.784134+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784134+0000","last_undegraded":"2024-03-06T07:25:19.784134+0000","last_fullsized":"2024-03-06T07:25:19.784134+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792512+0000","last_change":"2024-03-06T07:25:19.792512+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792512+0000","last_undegraded":"2024-03-06T07:25:19.792512+0000","last_fullsized":"2024-03-06T07:25:19.792512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792010+0000","last_change":"2024-03-06T07:25:19.792010+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792010+0000","last_undegraded":"2024-03-06T07:25:19.792010+0000","last_fullsized":"2024-03-06T07:25:19.792010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785099+0000","last_change":"2024-03-06T07:25:19.785099+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785099+0000","last_undegraded":"2024-03-06T07:25:19.785099+0000","last_fullsized":"2024-03-06T07:25:19.785099+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788546+0000","last_change":"2024-03-06T07:25:19.788546+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788546+0000","last_undegraded":"2024-03-06T07:25:19.788546+0000","last_fullsized":"2024-03-06T07:25:19.788546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.793411+0000","last_change":"2024-03-06T07:25:19.793411+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.793411+0000","last_undegraded":"2024-03-06T07:25:19.793411+0000","last_fullsized":"2024-03-06T07:25:19.793411+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789475+0000","last_change":"2024-03-06T07:25:19.789475+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789475+0000","last_undegraded":"2024-03-06T07:25:19.789475+0000","last_fullsized":"2024-03-06T07:25:19.789475+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789405+0000","last_change":"2024-03-06T07:25:19.789405+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789405+0000","last_undegraded":"2024-03-06T07:25:19.789405+0000","last_fullsized":"2024-03-06T07:25:19.789405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790829+0000","last_change":"2024-03-06T07:25:19.790829+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790829+0000","last_undegraded":"2024-03-06T07:25:19.790829+0000","last_fullsized":"2024-03-06T07:25:19.790829+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.793506+0000","last_change":"2024-03-06T07:25:19.793506+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.793506+0000","last_undegraded":"2024-03-06T07:25:19.793506+0000","last_fullsized":"2024-03-06T07:25:19.793506+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784444+0000","last_change":"2024-03-06T07:25:19.784444+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784444+0000","last_undegraded":"2024-03-06T07:25:19.784444+0000","last_fullsized":"2024-03-06T07:25:19.784444+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785275+0000","last_change":"2024-03-06T07:25:19.785275+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785275+0000","last_undegraded":"2024-03-06T07:25:19.785275+0000","last_fullsized":"2024-03-06T07:25:19.785275+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785315+0000","last_change":"2024-03-06T07:25:19.785315+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785315+0000","last_undegraded":"2024-03-06T07:25:19.785315+0000","last_fullsized":"2024-03-06T07:25:19.785315+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792999+0000","last_change":"2024-03-06T07:25:19.792999+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792999+0000","last_undegraded":"2024-03-06T07:25:19.792999+0000","last_fullsized":"2024-03-06T07:25:19.792999+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788950+0000","last_change":"2024-03-06T07:25:19.788950+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788950+0000","last_undegraded":"2024-03-06T07:25:19.788950+0000","last_fullsized":"2024-03-06T07:25:19.788950+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783383+0000","last_change":"2024-03-06T07:25:19.783383+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783383+0000","last_undegraded":"2024-03-06T07:25:19.783383+0000","last_fullsized":"2024-03-06T07:25:19.783383+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784142+0000","last_change":"2024-03-06T07:25:19.784142+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784142+0000","last_undegraded":"2024-03-06T07:25:19.784142+0000","last_fullsized":"2024-03-06T07:25:19.784142+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789851+0000","last_change":"2024-03-06T07:25:19.789851+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789851+0000","last_undegraded":"2024-03-06T07:25:19.789851+0000","last_fullsized":"2024-03-06T07:25:19.789851+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790728+0000","last_change":"2024-03-06T07:25:19.790728+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790728+0000","last_undegraded":"2024-03-06T07:25:19.790728+0000","last_fullsized":"2024-03-06T07:25:19.790728+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.791445+0000","last_change":"2024-03-06T07:25:19.791445+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.791445+0000","last_undegraded":"2024-03-06T07:25:19.791445+0000","last_fullsized":"2024-03-06T07:25:19.791445+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784190+0000","last_change":"2024-03-06T07:25:19.784190+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784190+0000","last_undegraded":"2024-03-06T07:25:19.784190+0000","last_fullsized":"2024-03-06T07:25:19.784190+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783073+0000","last_change":"2024-03-06T07:25:19.783073+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783073+0000","last_undegraded":"2024-03-06T07:25:19.783073+0000","last_fullsized":"2024-03-06T07:25:19.783073+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784550+0000","last_change":"2024-03-06T07:25:19.784550+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784550+0000","last_undegraded":"2024-03-06T07:25:19.784550+0000","last_fullsized":"2024-03-06T07:25:19.784550+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790473+0000","last_change":"2024-03-06T07:25:19.790473+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790473+0000","last_undegraded":"2024-03-06T07:25:19.790473+0000","last_fullsized":"2024-03-06T07:25:19.790473+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783060+0000","last_change":"2024-03-06T07:25:19.783060+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783060+0000","last_undegraded":"2024-03-06T07:25:19.783060+0000","last_fullsized":"2024-03-06T07:25:19.783060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788370+0000","last_change":"2024-03-06T07:25:19.788370+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788370+0000","last_undegraded":"2024-03-06T07:25:19.788370+0000","last_fullsized":"2024-03-06T07:25:19.788370+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.800985+0000","last_change":"2024-03-06T07:25:19.800026+0000","last_active":"2024-03-06T07:25:19.800985+0000","last_peered":"2024-03-06T07:25:19.800985+0000","last_clean":"2024-03-06T07:25:19.800985+0000","last_became_active":"2024-03-06T07:25:19.799694+0000","last_became_peered":"2024-03-06T07:25:19.799694+0000","last_unstale":"2024-03-06T07:25:19.800985+0000","last_undegraded":"2024-03-06T07:25:19.800985+0000","last_fullsized":"2024-03-06T07:25:19.800985+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:05.339137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787369+0000","last_change":"2024-03-06T07:25:19.787369+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787369+0000","last_undegraded":"2024-03-06T07:25:19.787369+0000","last_fullsized":"2024-03-06T07:25:19.787369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.796334+0000","last_change":"2024-03-06T07:25:19.794850+0000","last_active":"2024-03-06T07:25:19.796334+0000","last_peered":"2024-03-06T07:25:19.796334+0000","last_clean":"2024-03-06T07:25:19.796334+0000","last_became_active":"2024-03-06T07:25:19.794587+0000","last_became_peered":"2024-03-06T07:25:19.794587+0000","last_unstale":"2024-03-06T07:25:19.796334+0000","last_undegraded":"2024-03-06T07:25:19.796334+0000","last_fullsized":"2024-03-06T07:25:19.796334+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:28.129920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782596+0000","last_change":"2024-03-06T07:25:19.782596+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782596+0000","last_undegraded":"2024-03-06T07:25:19.782596+0000","last_fullsized":"2024-03-06T07:25:19.782596+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798990+0000","last_change":"2024-03-06T07:25:19.796329+0000","last_active":"2024-03-06T07:25:19.798990+0000","last_peered":"2024-03-06T07:25:19.798990+0000","last_clean":"2024-03-06T07:25:19.798990+0000","last_became_active":"2024-03-06T07:25:19.796081+0000","last_became_peered":"2024-03-06T07:25:19.796081+0000","last_unstale":"2024-03-06T07:25:19.798990+0000","last_undegraded":"2024-03-06T07:25:19.798990+0000","last_fullsized":"2024-03-06T07:25:19.798990+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:18.223482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784369+0000","last_change":"2024-03-06T07:25:19.784369+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784369+0000","last_undegraded":"2024-03-06T07:25:19.784369+0000","last_fullsized":"2024-03-06T07:25:19.784369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.793189+0000","last_change":"2024-03-06T07:25:19.789450+0000","last_active":"2024-03-06T07:25:19.793189+0000","last_peered":"2024-03-06T07:25:19.793189+0000","last_clean":"2024-03-06T07:25:19.793189+0000","last_became_active":"2024-03-06T07:25:19.789139+0000","last_became_peered":"2024-03-06T07:25:19.789139+0000","last_unstale":"2024-03-06T07:25:19.793189+0000","last_undegraded":"2024-03-06T07:25:19.793189+0000","last_fullsized":"2024-03-06T07:25:19.793189+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:58.530205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785243+0000","last_change":"2024-03-06T07:25:19.785243+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785243+0000","last_undegraded":"2024-03-06T07:25:19.785243+0000","last_fullsized":"2024-03-06T07:25:19.785243+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.800808+0000","last_change":"2024-03-06T07:25:19.797666+0000","last_active":"2024-03-06T07:25:19.800808+0000","last_peered":"2024-03-06T07:25:19.800808+0000","last_clean":"2024-03-06T07:25:19.800808+0000","last_became_active":"2024-03-06T07:25:19.797391+0000","last_became_peered":"2024-03-06T07:25:19.797391+0000","last_unstale":"2024-03-06T07:25:19.800808+0000","last_undegraded":"2024-03-06T07:25:19.800808+0000","last_fullsized":"2024-03-06T07:25:19.800808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:48.223756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790355+0000","last_change":"2024-03-06T07:25:19.790355+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790355+0000","last_undegraded":"2024-03-06T07:25:19.790355+0000","last_fullsized":"2024-03-06T07:25:19.790355+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801006+0000","last_change":"2024-03-06T07:25:19.799648+0000","last_active":"2024-03-06T07:25:19.801006+0000","last_peered":"2024-03-06T07:25:19.801006+0000","last_clean":"2024-03-06T07:25:19.801006+0000","last_became_active":"2024-03-06T07:25:19.799352+0000","last_became_peered":"2024-03-06T07:25:19.799352+0000","last_unstale":"2024-03-06T07:25:19.801006+0000","last_undegraded":"2024-03-06T07:25:19.801006+0000","last_fullsized":"2024-03-06T07:25:19.801006+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:30.875962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.787231+0000","last_change":"2024-03-06T07:25:19.787231+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.787231+0000","last_undegraded":"2024-03-06T07:25:19.787231+0000","last_fullsized":"2024-03-06T07:25:19.787231+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801537+0000","last_change":"2024-03-06T07:25:19.797676+0000","last_active":"2024-03-06T07:25:19.801537+0000","last_peered":"2024-03-06T07:25:19.801537+0000","last_clean":"2024-03-06T07:25:19.801537+0000","last_became_active":"2024-03-06T07:25:19.797399+0000","last_became_peered":"2024-03-06T07:25:19.797399+0000","last_unstale":"2024-03-06T07:25:19.801537+0000","last_undegraded":"2024-03-06T07:25:19.801537+0000","last_fullsized":"2024-03-06T07:25:19.801537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:34.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788917+0000","last_change":"2024-03-06T07:25:19.788917+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788917+0000","last_undegraded":"2024-03-06T07:25:19.788917+0000","last_fullsized":"2024-03-06T07:25:19.788917+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.802569+0000","last_change":"2024-03-06T07:25:19.802156+0000","last_active":"2024-03-06T07:25:19.802569+0000","last_peered":"2024-03-06T07:25:19.802569+0000","last_clean":"2024-03-06T07:25:19.802569+0000","last_became_active":"2024-03-06T07:25:19.801881+0000","last_became_peered":"2024-03-06T07:25:19.801881+0000","last_unstale":"2024-03-06T07:25:19.802569+0000","last_undegraded":"2024-03-06T07:25:19.802569+0000","last_fullsized":"2024-03-06T07:25:19.802569+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:14.322866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782407+0000","last_change":"2024-03-06T07:25:19.782407+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782407+0000","last_undegraded":"2024-03-06T07:25:19.782407+0000","last_fullsized":"2024-03-06T07:25:19.782407+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798410+0000","last_change":"2024-03-06T07:25:19.795560+0000","last_active":"2024-03-06T07:25:19.798410+0000","last_peered":"2024-03-06T07:25:19.798410+0000","last_clean":"2024-03-06T07:25:19.798410+0000","last_became_active":"2024-03-06T07:25:19.795293+0000","last_became_peered":"2024-03-06T07:25:19.795293+0000","last_unstale":"2024-03-06T07:25:19.798410+0000","last_undegraded":"2024-03-06T07:25:19.798410+0000","last_fullsized":"2024-03-06T07:25:19.798410+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:14.322866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784536+0000","last_change":"2024-03-06T07:25:19.784536+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784536+0000","last_undegraded":"2024-03-06T07:25:19.784536+0000","last_fullsized":"2024-03-06T07:25:19.784536+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804177+0000","last_change":"2024-03-06T07:25:19.798578+0000","last_active":"2024-03-06T07:25:19.804177+0000","last_peered":"2024-03-06T07:25:19.804177+0000","last_clean":"2024-03-06T07:25:19.804177+0000","last_became_active":"2024-03-06T07:25:19.798263+0000","last_became_peered":"2024-03-06T07:25:19.798263+0000","last_unstale":"2024-03-06T07:25:19.804177+0000","last_undegraded":"2024-03-06T07:25:19.804177+0000","last_fullsized":"2024-03-06T07:25:19.804177+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:51.204621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783803+0000","last_change":"2024-03-06T07:25:19.783803+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783803+0000","last_undegraded":"2024-03-06T07:25:19.783803+0000","last_fullsized":"2024-03-06T07:25:19.783803+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.793843+0000","last_change":"2024-03-06T07:25:19.792711+0000","last_active":"2024-03-06T07:25:19.793843+0000","last_peered":"2024-03-06T07:25:19.793843+0000","last_clean":"2024-03-06T07:25:19.793843+0000","last_became_active":"2024-03-06T07:25:19.792405+0000","last_became_peered":"2024-03-06T07:25:19.792405+0000","last_unstale":"2024-03-06T07:25:19.793843+0000","last_undegraded":"2024-03-06T07:25:19.793843+0000","last_fullsized":"2024-03-06T07:25:19.793843+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:40.261389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.782423+0000","last_change":"2024-03-06T07:25:19.782423+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.782423+0000","last_undegraded":"2024-03-06T07:25:19.782423+0000","last_fullsized":"2024-03-06T07:25:19.782423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801343+0000","last_change":"2024-03-06T07:25:19.800658+0000","last_active":"2024-03-06T07:25:19.801343+0000","last_peered":"2024-03-06T07:25:19.801343+0000","last_clean":"2024-03-06T07:25:19.801343+0000","last_became_active":"2024-03-06T07:25:19.800361+0000","last_became_peered":"2024-03-06T07:25:19.800361+0000","last_unstale":"2024-03-06T07:25:19.801343+0000","last_undegraded":"2024-03-06T07:25:19.801343+0000","last_fullsized":"2024-03-06T07:25:19.801343+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:28.129920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788966+0000","last_change":"2024-03-06T07:25:19.788966+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788966+0000","last_undegraded":"2024-03-06T07:25:19.788966+0000","last_fullsized":"2024-03-06T07:25:19.788966+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804522+0000","last_change":"2024-03-06T07:25:19.802236+0000","last_active":"2024-03-06T07:25:19.804522+0000","last_peered":"2024-03-06T07:25:19.804522+0000","last_clean":"2024-03-06T07:25:19.804522+0000","last_became_active":"2024-03-06T07:25:19.801866+0000","last_became_peered":"2024-03-06T07:25:19.801866+0000","last_unstale":"2024-03-06T07:25:19.804522+0000","last_undegraded":"2024-03-06T07:25:19.804522+0000","last_fullsized":"2024-03-06T07:25:19.804522+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:02.212979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790418+0000","last_change":"2024-03-06T07:25:19.790418+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790418+0000","last_undegraded":"2024-03-06T07:25:19.790418+0000","last_fullsized":"2024-03-06T07:25:19.790418+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804286+0000","last_change":"2024-03-06T07:25:19.800076+0000","last_active":"2024-03-06T07:25:19.804286+0000","last_peered":"2024-03-06T07:25:19.804286+0000","last_clean":"2024-03-06T07:25:19.804286+0000","last_became_active":"2024-03-06T07:25:19.799501+0000","last_became_peered":"2024-03-06T07:25:19.799501+0000","last_unstale":"2024-03-06T07:25:19.804286+0000","last_undegraded":"2024-03-06T07:25:19.804286+0000","last_fullsized":"2024-03-06T07:25:19.804286+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:56.743191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786604+0000","last_change":"2024-03-06T07:25:19.786604+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786604+0000","last_undegraded":"2024-03-06T07:25:19.786604+0000","last_fullsized":"2024-03-06T07:25:19.786604+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801485+0000","last_change":"2024-03-06T07:25:19.797067+0000","last_active":"2024-03-06T07:25:19.801485+0000","last_peered":"2024-03-06T07:25:19.801485+0000","last_clean":"2024-03-06T07:25:19.801485+0000","last_became_active":"2024-03-06T07:25:19.796773+0000","last_became_peered":"2024-03-06T07:25:19.796773+0000","last_unstale":"2024-03-06T07:25:19.801485+0000","last_undegraded":"2024-03-06T07:25:19.801485+0000","last_fullsized":"2024-03-06T07:25:19.801485+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:08.144349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784876+0000","last_change":"2024-03-06T07:25:19.784876+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784876+0000","last_undegraded":"2024-03-06T07:25:19.784876+0000","last_fullsized":"2024-03-06T07:25:19.784876+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.790405+0000","last_change":"2024-03-06T07:25:19.789597+0000","last_active":"2024-03-06T07:25:19.790405+0000","last_peered":"2024-03-06T07:25:19.790405+0000","last_clean":"2024-03-06T07:25:19.790405+0000","last_became_active":"2024-03-06T07:25:19.789334+0000","last_became_peered":"2024-03-06T07:25:19.789334+0000","last_unstale":"2024-03-06T07:25:19.790405+0000","last_undegraded":"2024-03-06T07:25:19.790405+0000","last_fullsized":"2024-03-06T07:25:19.790405+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:45.098378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784468+0000","last_change":"2024-03-06T07:25:19.784468+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784468+0000","last_undegraded":"2024-03-06T07:25:19.784468+0000","last_fullsized":"2024-03-06T07:25:19.784468+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799011+0000","last_change":"2024-03-06T07:25:19.796190+0000","last_active":"2024-03-06T07:25:19.799011+0000","last_peered":"2024-03-06T07:25:19.799011+0000","last_clean":"2024-03-06T07:25:19.799011+0000","last_became_active":"2024-03-06T07:25:19.795879+0000","last_became_peered":"2024-03-06T07:25:19.795879+0000","last_unstale":"2024-03-06T07:25:19.799011+0000","last_undegraded":"2024-03-06T07:25:19.799011+0000","last_fullsized":"2024-03-06T07:25:19.799011+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:59.395378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.790299+0000","last_change":"2024-03-06T07:25:19.790299+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.790299+0000","last_undegraded":"2024-03-06T07:25:19.790299+0000","last_fullsized":"2024-03-06T07:25:19.790299+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799742+0000","last_change":"2024-03-06T07:25:19.798347+0000","last_active":"2024-03-06T07:25:19.799742+0000","last_peered":"2024-03-06T07:25:19.799742+0000","last_clean":"2024-03-06T07:25:19.799742+0000","last_became_active":"2024-03-06T07:25:19.798021+0000","last_became_peered":"2024-03-06T07:25:19.798021+0000","last_unstale":"2024-03-06T07:25:19.799742+0000","last_undegraded":"2024-03-06T07:25:19.799742+0000","last_fullsized":"2024-03-06T07:25:19.799742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:08.144349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789405+0000","last_change":"2024-03-06T07:25:19.789405+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789405+0000","last_undegraded":"2024-03-06T07:25:19.789405+0000","last_fullsized":"2024-03-06T07:25:19.789405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804335+0000","last_change":"2024-03-06T07:25:19.801586+0000","last_active":"2024-03-06T07:25:19.804335+0000","last_peered":"2024-03-06T07:25:19.804335+0000","last_clean":"2024-03-06T07:25:19.804335+0000","last_became_active":"2024-03-06T07:25:19.801242+0000","last_became_peered":"2024-03-06T07:25:19.801242+0000","last_unstale":"2024-03-06T07:25:19.804335+0000","last_undegraded":"2024-03-06T07:25:19.804335+0000","last_fullsized":"2024-03-06T07:25:19.804335+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:39.912413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784346+0000","last_change":"2024-03-06T07:25:19.784346+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784346+0000","last_undegraded":"2024-03-06T07:25:19.784346+0000","last_fullsized":"2024-03-06T07:25:19.784346+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.797760+0000","last_change":"2024-03-06T07:25:19.796315+0000","last_active":"2024-03-06T07:25:19.797760+0000","last_peered":"2024-03-06T07:25:19.797760+0000","last_clean":"2024-03-06T07:25:19.797760+0000","last_became_active":"2024-03-06T07:25:19.796036+0000","last_became_peered":"2024-03-06T07:25:19.796036+0000","last_unstale":"2024-03-06T07:25:19.797760+0000","last_undegraded":"2024-03-06T07:25:19.797760+0000","last_fullsized":"2024-03-06T07:25:19.797760+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:15.257584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788465+0000","last_change":"2024-03-06T07:25:19.788465+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788465+0000","last_undegraded":"2024-03-06T07:25:19.788465+0000","last_fullsized":"2024-03-06T07:25:19.788465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792150+0000","last_change":"2024-03-06T07:25:19.789672+0000","last_active":"2024-03-06T07:25:19.792150+0000","last_peered":"2024-03-06T07:25:19.792150+0000","last_clean":"2024-03-06T07:25:19.792150+0000","last_became_active":"2024-03-06T07:25:19.789409+0000","last_became_peered":"2024-03-06T07:25:19.789409+0000","last_unstale":"2024-03-06T07:25:19.792150+0000","last_undegraded":"2024-03-06T07:25:19.792150+0000","last_fullsized":"2024-03-06T07:25:19.792150+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:07.534390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784743+0000","last_change":"2024-03-06T07:25:19.784743+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784743+0000","last_undegraded":"2024-03-06T07:25:19.784743+0000","last_fullsized":"2024-03-06T07:25:19.784743+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.799242+0000","last_change":"2024-03-06T07:25:19.797588+0000","last_active":"2024-03-06T07:25:19.799242+0000","last_peered":"2024-03-06T07:25:19.799242+0000","last_clean":"2024-03-06T07:25:19.799242+0000","last_became_active":"2024-03-06T07:25:19.797322+0000","last_became_peered":"2024-03-06T07:25:19.797322+0000","last_unstale":"2024-03-06T07:25:19.799242+0000","last_undegraded":"2024-03-06T07:25:19.799242+0000","last_fullsized":"2024-03-06T07:25:19.799242+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:19.871887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786003+0000","last_change":"2024-03-06T07:25:19.786003+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786003+0000","last_undegraded":"2024-03-06T07:25:19.786003+0000","last_fullsized":"2024-03-06T07:25:19.786003+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795817+0000","last_change":"2024-03-06T07:25:19.794012+0000","last_active":"2024-03-06T07:25:19.795817+0000","last_peered":"2024-03-06T07:25:19.795817+0000","last_clean":"2024-03-06T07:25:19.795817+0000","last_became_active":"2024-03-06T07:25:19.793743+0000","last_became_peered":"2024-03-06T07:25:19.793743+0000","last_unstale":"2024-03-06T07:25:19.795817+0000","last_undegraded":"2024-03-06T07:25:19.795817+0000","last_fullsized":"2024-03-06T07:25:19.795817+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:22:21.615129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.786591+0000","last_change":"2024-03-06T07:25:19.786591+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.786591+0000","last_undegraded":"2024-03-06T07:25:19.786591+0000","last_fullsized":"2024-03-06T07:25:19.786591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.804333+0000","last_change":"2024-03-06T07:25:19.801518+0000","last_active":"2024-03-06T07:25:19.804333+0000","last_peered":"2024-03-06T07:25:19.804333+0000","last_clean":"2024-03-06T07:25:19.804333+0000","last_became_active":"2024-03-06T07:25:19.801213+0000","last_became_peered":"2024-03-06T07:25:19.801213+0000","last_unstale":"2024-03-06T07:25:19.804333+0000","last_undegraded":"2024-03-06T07:25:19.804333+0000","last_fullsized":"2024-03-06T07:25:19.804333+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:54.608998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.792586+0000","last_change":"2024-03-06T07:25:19.792586+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.792586+0000","last_undegraded":"2024-03-06T07:25:19.792586+0000","last_fullsized":"2024-03-06T07:25:19.792586+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798951+0000","last_change":"2024-03-06T07:25:19.793750+0000","last_active":"2024-03-06T07:25:19.798951+0000","last_peered":"2024-03-06T07:25:19.798951+0000","last_clean":"2024-03-06T07:25:19.798951+0000","last_became_active":"2024-03-06T07:25:19.793481+0000","last_became_peered":"2024-03-06T07:25:19.793481+0000","last_unstale":"2024-03-06T07:25:19.798951+0000","last_undegraded":"2024-03-06T07:25:19.798951+0000","last_fullsized":"2024-03-06T07:25:19.798951+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:58.530205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.789100+0000","last_change":"2024-03-06T07:25:19.789100+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.789100+0000","last_undegraded":"2024-03-06T07:25:19.789100+0000","last_fullsized":"2024-03-06T07:25:19.789100+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.794368+0000","last_change":"2024-03-06T07:25:19.793729+0000","last_active":"2024-03-06T07:25:19.794368+0000","last_peered":"2024-03-06T07:25:19.794368+0000","last_clean":"2024-03-06T07:25:19.794368+0000","last_became_active":"2024-03-06T07:25:19.793432+0000","last_became_peered":"2024-03-06T07:25:19.793432+0000","last_unstale":"2024-03-06T07:25:19.794368+0000","last_undegraded":"2024-03-06T07:25:19.794368+0000","last_fullsized":"2024-03-06T07:25:19.794368+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:20.252960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788943+0000","last_change":"2024-03-06T07:25:19.788943+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788943+0000","last_undegraded":"2024-03-06T07:25:19.788943+0000","last_fullsized":"2024-03-06T07:25:19.788943+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795674+0000","last_change":"2024-03-06T07:25:19.793893+0000","last_active":"2024-03-06T07:25:19.795674+0000","last_peered":"2024-03-06T07:25:19.795674+0000","last_clean":"2024-03-06T07:25:19.795674+0000","last_became_active":"2024-03-06T07:25:19.793604+0000","last_became_peered":"2024-03-06T07:25:19.793604+0000","last_unstale":"2024-03-06T07:25:19.795674+0000","last_undegraded":"2024-03-06T07:25:19.795674+0000","last_fullsized":"2024-03-06T07:25:19.795674+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:06.983583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.781837+0000","last_change":"2024-03-06T07:25:19.781837+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.781837+0000","last_undegraded":"2024-03-06T07:25:19.781837+0000","last_fullsized":"2024-03-06T07:25:19.781837+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.801503+0000","last_change":"2024-03-06T07:25:19.797615+0000","last_active":"2024-03-06T07:25:19.801503+0000","last_peered":"2024-03-06T07:25:19.801503+0000","last_clean":"2024-03-06T07:25:19.801503+0000","last_became_active":"2024-03-06T07:25:19.797345+0000","last_became_peered":"2024-03-06T07:25:19.797345+0000","last_unstale":"2024-03-06T07:25:19.801503+0000","last_undegraded":"2024-03-06T07:25:19.801503+0000","last_fullsized":"2024-03-06T07:25:19.801503+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:26.870856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.788147+0000","last_change":"2024-03-06T07:25:19.788147+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.788147+0000","last_undegraded":"2024-03-06T07:25:19.788147+0000","last_fullsized":"2024-03-06T07:25:19.788147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792242+0000","last_change":"2024-03-06T07:25:19.791673+0000","last_active":"2024-03-06T07:25:19.792242+0000","last_peered":"2024-03-06T07:25:19.792242+0000","last_clean":"2024-03-06T07:25:19.792242+0000","last_became_active":"2024-03-06T07:25:19.791395+0000","last_became_peered":"2024-03-06T07:25:19.791395+0000","last_unstale":"2024-03-06T07:25:19.792242+0000","last_undegraded":"2024-03-06T07:25:19.792242+0000","last_fullsized":"2024-03-06T07:25:19.792242+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:26:07.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.785270+0000","last_change":"2024-03-06T07:25:19.785270+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.785270+0000","last_undegraded":"2024-03-06T07:25:19.785270+0000","last_fullsized":"2024-03-06T07:25:19.785270+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.798915+0000","last_change":"2024-03-06T07:25:19.794392+0000","last_active":"2024-03-06T07:25:19.798915+0000","last_peered":"2024-03-06T07:25:19.798915+0000","last_clean":"2024-03-06T07:25:19.798915+0000","last_became_active":"2024-03-06T07:25:19.794116+0000","last_became_peered":"2024-03-06T07:25:19.794116+0000","last_unstale":"2024-03-06T07:25:19.798915+0000","last_undegraded":"2024-03-06T07:25:19.798915+0000","last_fullsized":"2024-03-06T07:25:19.798915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:20.252960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.784010+0000","last_change":"2024-03-06T07:25:19.784010+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.784010+0000","last_undegraded":"2024-03-06T07:25:19.784010+0000","last_fullsized":"2024-03-06T07:25:19.784010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.795008+0000","last_change":"2024-03-06T07:25:19.793614+0000","last_active":"2024-03-06T07:25:19.795008+0000","last_peered":"2024-03-06T07:25:19.795008+0000","last_clean":"2024-03-06T07:25:19.795008+0000","last_became_active":"2024-03-06T07:25:19.793338+0000","last_became_peered":"2024-03-06T07:25:19.793338+0000","last_unstale":"2024-03-06T07:25:19.795008+0000","last_undegraded":"2024-03-06T07:25:19.795008+0000","last_fullsized":"2024-03-06T07:25:19.795008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:05.339137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2024-03-06T07:25:19.783262+0000","last_change":"2024-03-06T07:25:19.783262+0000","last_active":"2024-03-06T07:25:19.773061+0000","last_peered":"2024-03-06T07:25:19.773061+0000","last_clean":"2024-03-06T07:25:19.773061+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:19.783262+0000","last_undegraded":"2024-03-06T07:25:19.783262+0000","last_fullsized":"2024-03-06T07:25:19.783262+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.792135+0000","last_change":"2024-03-06T07:25:19.790987+0000","last_active":"2024-03-06T07:25:19.792135+0000","last_peered":"2024-03-06T07:25:19.792135+0000","last_clean":"2024-03-06T07:25:19.792135+0000","last_became_active":"2024-03-06T07:25:19.790677+0000","last_became_peered":"2024-03-06T07:25:19.790677+0000","last_unstale":"2024-03-06T07:25:19.792135+0000","last_undegraded":"2024-03-06T07:25:19.792135+0000","last_fullsized":"2024-03-06T07:25:19.792135+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:07.375118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":382,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-03-06T07:25:19.785971+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:25:19.785971+0000","last_peered":"2024-03-06T07:25:19.785971+0000","last_clean":"2024-03-06T07:25:19.785971+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:25:19.785971+0000","last_undegraded":"2024-03-06T07:25:19.785971+0000","last_fullsized":"2024-03-06T07:25:19.785971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1724416,"data_stored":1387718,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":804017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":804017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":791141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":797350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1387718,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3f is not active+clean 2024-03-06T07:25:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.790263+0000', 'last_change': '2024-03-06T07:25:19.790263+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.790263+0000', 'last_undegraded': '2024-03-06T07:25:19.790263+0000', 'last_fullsized': '2024-03-06T07:25:19.790263+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3e is not active+clean 2024-03-06T07:25:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788156+0000', 'last_change': '2024-03-06T07:25:19.788156+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788156+0000', 'last_undegraded': '2024-03-06T07:25:19.788156+0000', 'last_fullsized': '2024-03-06T07:25:19.788156+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3d is not active+clean 2024-03-06T07:25:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.787767+0000', 'last_change': '2024-03-06T07:25:19.787767+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.787767+0000', 'last_undegraded': '2024-03-06T07:25:19.787767+0000', 'last_fullsized': '2024-03-06T07:25:19.787767+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3c is not active+clean 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788939+0000', 'last_change': '2024-03-06T07:25:19.788939+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788939+0000', 'last_undegraded': '2024-03-06T07:25:19.788939+0000', 'last_fullsized': '2024-03-06T07:25:19.788939+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3b is not active+clean 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.795996+0000', 'last_change': '2024-03-06T07:25:19.795996+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.795996+0000', 'last_undegraded': '2024-03-06T07:25:19.795996+0000', 'last_fullsized': '2024-03-06T07:25:19.795996+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3a is not active+clean 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.795192+0000', 'last_change': '2024-03-06T07:25:19.795192+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.795192+0000', 'last_undegraded': '2024-03-06T07:25:19.795192+0000', 'last_fullsized': '2024-03-06T07:25:19.795192+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.39 is not active+clean 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.791435+0000', 'last_change': '2024-03-06T07:25:19.791435+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.791435+0000', 'last_undegraded': '2024-03-06T07:25:19.791435+0000', 'last_fullsized': '2024-03-06T07:25:19.791435+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.38 is not active+clean 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784134+0000', 'last_change': '2024-03-06T07:25:19.784134+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784134+0000', 'last_undegraded': '2024-03-06T07:25:19.784134+0000', 'last_fullsized': '2024-03-06T07:25:19.784134+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.37 is not active+clean 2024-03-06T07:25:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.792512+0000', 'last_change': '2024-03-06T07:25:19.792512+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.792512+0000', 'last_undegraded': '2024-03-06T07:25:19.792512+0000', 'last_fullsized': '2024-03-06T07:25:19.792512+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.36 is not active+clean 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.792010+0000', 'last_change': '2024-03-06T07:25:19.792010+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.792010+0000', 'last_undegraded': '2024-03-06T07:25:19.792010+0000', 'last_fullsized': '2024-03-06T07:25:19.792010+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.35 is not active+clean 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.785099+0000', 'last_change': '2024-03-06T07:25:19.785099+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.785099+0000', 'last_undegraded': '2024-03-06T07:25:19.785099+0000', 'last_fullsized': '2024-03-06T07:25:19.785099+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.34 is not active+clean 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788546+0000', 'last_change': '2024-03-06T07:25:19.788546+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788546+0000', 'last_undegraded': '2024-03-06T07:25:19.788546+0000', 'last_fullsized': '2024-03-06T07:25:19.788546+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.33 is not active+clean 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.793411+0000', 'last_change': '2024-03-06T07:25:19.793411+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.793411+0000', 'last_undegraded': '2024-03-06T07:25:19.793411+0000', 'last_fullsized': '2024-03-06T07:25:19.793411+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.32 is not active+clean 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.789475+0000', 'last_change': '2024-03-06T07:25:19.789475+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.789475+0000', 'last_undegraded': '2024-03-06T07:25:19.789475+0000', 'last_fullsized': '2024-03-06T07:25:19.789475+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:24.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.31 is not active+clean 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.789405+0000', 'last_change': '2024-03-06T07:25:19.789405+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.789405+0000', 'last_undegraded': '2024-03-06T07:25:19.789405+0000', 'last_fullsized': '2024-03-06T07:25:19.789405+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.30 is not active+clean 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.790829+0000', 'last_change': '2024-03-06T07:25:19.790829+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.790829+0000', 'last_undegraded': '2024-03-06T07:25:19.790829+0000', 'last_fullsized': '2024-03-06T07:25:19.790829+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2f is not active+clean 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.793506+0000', 'last_change': '2024-03-06T07:25:19.793506+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.793506+0000', 'last_undegraded': '2024-03-06T07:25:19.793506+0000', 'last_fullsized': '2024-03-06T07:25:19.793506+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2e is not active+clean 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784444+0000', 'last_change': '2024-03-06T07:25:19.784444+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784444+0000', 'last_undegraded': '2024-03-06T07:25:19.784444+0000', 'last_fullsized': '2024-03-06T07:25:19.784444+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2d is not active+clean 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.785275+0000', 'last_change': '2024-03-06T07:25:19.785275+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.785275+0000', 'last_undegraded': '2024-03-06T07:25:19.785275+0000', 'last_fullsized': '2024-03-06T07:25:19.785275+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2c is not active+clean 2024-03-06T07:25:24.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.785315+0000', 'last_change': '2024-03-06T07:25:19.785315+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.785315+0000', 'last_undegraded': '2024-03-06T07:25:19.785315+0000', 'last_fullsized': '2024-03-06T07:25:19.785315+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2b is not active+clean 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.792999+0000', 'last_change': '2024-03-06T07:25:19.792999+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.792999+0000', 'last_undegraded': '2024-03-06T07:25:19.792999+0000', 'last_fullsized': '2024-03-06T07:25:19.792999+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2a is not active+clean 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788950+0000', 'last_change': '2024-03-06T07:25:19.788950+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788950+0000', 'last_undegraded': '2024-03-06T07:25:19.788950+0000', 'last_fullsized': '2024-03-06T07:25:19.788950+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.29 is not active+clean 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.783383+0000', 'last_change': '2024-03-06T07:25:19.783383+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.783383+0000', 'last_undegraded': '2024-03-06T07:25:19.783383+0000', 'last_fullsized': '2024-03-06T07:25:19.783383+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.28 is not active+clean 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784142+0000', 'last_change': '2024-03-06T07:25:19.784142+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784142+0000', 'last_undegraded': '2024-03-06T07:25:19.784142+0000', 'last_fullsized': '2024-03-06T07:25:19.784142+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.27 is not active+clean 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.789851+0000', 'last_change': '2024-03-06T07:25:19.789851+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.789851+0000', 'last_undegraded': '2024-03-06T07:25:19.789851+0000', 'last_fullsized': '2024-03-06T07:25:19.789851+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.26 is not active+clean 2024-03-06T07:25:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.790728+0000', 'last_change': '2024-03-06T07:25:19.790728+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.790728+0000', 'last_undegraded': '2024-03-06T07:25:19.790728+0000', 'last_fullsized': '2024-03-06T07:25:19.790728+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.25 is not active+clean 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.791445+0000', 'last_change': '2024-03-06T07:25:19.791445+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.791445+0000', 'last_undegraded': '2024-03-06T07:25:19.791445+0000', 'last_fullsized': '2024-03-06T07:25:19.791445+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.24 is not active+clean 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784190+0000', 'last_change': '2024-03-06T07:25:19.784190+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784190+0000', 'last_undegraded': '2024-03-06T07:25:19.784190+0000', 'last_fullsized': '2024-03-06T07:25:19.784190+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.23 is not active+clean 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.783073+0000', 'last_change': '2024-03-06T07:25:19.783073+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.783073+0000', 'last_undegraded': '2024-03-06T07:25:19.783073+0000', 'last_fullsized': '2024-03-06T07:25:19.783073+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.22 is not active+clean 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784550+0000', 'last_change': '2024-03-06T07:25:19.784550+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784550+0000', 'last_undegraded': '2024-03-06T07:25:19.784550+0000', 'last_fullsized': '2024-03-06T07:25:19.784550+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.21 is not active+clean 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.790473+0000', 'last_change': '2024-03-06T07:25:19.790473+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.790473+0000', 'last_undegraded': '2024-03-06T07:25:19.790473+0000', 'last_fullsized': '2024-03-06T07:25:19.790473+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.20 is not active+clean 2024-03-06T07:25:24.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.783060+0000', 'last_change': '2024-03-06T07:25:19.783060+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.783060+0000', 'last_undegraded': '2024-03-06T07:25:19.783060+0000', 'last_fullsized': '2024-03-06T07:25:19.783060+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788370+0000', 'last_change': '2024-03-06T07:25:19.788370+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788370+0000', 'last_undegraded': '2024-03-06T07:25:19.788370+0000', 'last_fullsized': '2024-03-06T07:25:19.788370+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.787369+0000', 'last_change': '2024-03-06T07:25:19.787369+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.787369+0000', 'last_undegraded': '2024-03-06T07:25:19.787369+0000', 'last_fullsized': '2024-03-06T07:25:19.787369+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.782596+0000', 'last_change': '2024-03-06T07:25:19.782596+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.782596+0000', 'last_undegraded': '2024-03-06T07:25:19.782596+0000', 'last_fullsized': '2024-03-06T07:25:19.782596+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784369+0000', 'last_change': '2024-03-06T07:25:19.784369+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784369+0000', 'last_undegraded': '2024-03-06T07:25:19.784369+0000', 'last_fullsized': '2024-03-06T07:25:19.784369+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2024-03-06T07:25:24.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.785243+0000', 'last_change': '2024-03-06T07:25:19.785243+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.785243+0000', 'last_undegraded': '2024-03-06T07:25:19.785243+0000', 'last_fullsized': '2024-03-06T07:25:19.785243+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.790355+0000', 'last_change': '2024-03-06T07:25:19.790355+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.790355+0000', 'last_undegraded': '2024-03-06T07:25:19.790355+0000', 'last_fullsized': '2024-03-06T07:25:19.790355+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.787231+0000', 'last_change': '2024-03-06T07:25:19.787231+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.787231+0000', 'last_undegraded': '2024-03-06T07:25:19.787231+0000', 'last_fullsized': '2024-03-06T07:25:19.787231+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788917+0000', 'last_change': '2024-03-06T07:25:19.788917+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788917+0000', 'last_undegraded': '2024-03-06T07:25:19.788917+0000', 'last_fullsized': '2024-03-06T07:25:19.788917+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.782407+0000', 'last_change': '2024-03-06T07:25:19.782407+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.782407+0000', 'last_undegraded': '2024-03-06T07:25:19.782407+0000', 'last_fullsized': '2024-03-06T07:25:19.782407+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784536+0000', 'last_change': '2024-03-06T07:25:19.784536+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784536+0000', 'last_undegraded': '2024-03-06T07:25:19.784536+0000', 'last_fullsized': '2024-03-06T07:25:19.784536+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2024-03-06T07:25:24.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.783803+0000', 'last_change': '2024-03-06T07:25:19.783803+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.783803+0000', 'last_undegraded': '2024-03-06T07:25:19.783803+0000', 'last_fullsized': '2024-03-06T07:25:19.783803+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.782423+0000', 'last_change': '2024-03-06T07:25:19.782423+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.782423+0000', 'last_undegraded': '2024-03-06T07:25:19.782423+0000', 'last_fullsized': '2024-03-06T07:25:19.782423+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788966+0000', 'last_change': '2024-03-06T07:25:19.788966+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788966+0000', 'last_undegraded': '2024-03-06T07:25:19.788966+0000', 'last_fullsized': '2024-03-06T07:25:19.788966+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.790418+0000', 'last_change': '2024-03-06T07:25:19.790418+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.790418+0000', 'last_undegraded': '2024-03-06T07:25:19.790418+0000', 'last_fullsized': '2024-03-06T07:25:19.790418+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.786604+0000', 'last_change': '2024-03-06T07:25:19.786604+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.786604+0000', 'last_undegraded': '2024-03-06T07:25:19.786604+0000', 'last_fullsized': '2024-03-06T07:25:19.786604+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784876+0000', 'last_change': '2024-03-06T07:25:19.784876+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784876+0000', 'last_undegraded': '2024-03-06T07:25:19.784876+0000', 'last_fullsized': '2024-03-06T07:25:19.784876+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2024-03-06T07:25:24.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784468+0000', 'last_change': '2024-03-06T07:25:19.784468+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784468+0000', 'last_undegraded': '2024-03-06T07:25:19.784468+0000', 'last_fullsized': '2024-03-06T07:25:19.784468+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.790299+0000', 'last_change': '2024-03-06T07:25:19.790299+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.790299+0000', 'last_undegraded': '2024-03-06T07:25:19.790299+0000', 'last_fullsized': '2024-03-06T07:25:19.790299+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.789405+0000', 'last_change': '2024-03-06T07:25:19.789405+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.789405+0000', 'last_undegraded': '2024-03-06T07:25:19.789405+0000', 'last_fullsized': '2024-03-06T07:25:19.789405+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784346+0000', 'last_change': '2024-03-06T07:25:19.784346+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784346+0000', 'last_undegraded': '2024-03-06T07:25:19.784346+0000', 'last_fullsized': '2024-03-06T07:25:19.784346+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788465+0000', 'last_change': '2024-03-06T07:25:19.788465+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788465+0000', 'last_undegraded': '2024-03-06T07:25:19.788465+0000', 'last_fullsized': '2024-03-06T07:25:19.788465+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784743+0000', 'last_change': '2024-03-06T07:25:19.784743+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784743+0000', 'last_undegraded': '2024-03-06T07:25:19.784743+0000', 'last_fullsized': '2024-03-06T07:25:19.784743+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:24.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.786003+0000', 'last_change': '2024-03-06T07:25:19.786003+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.786003+0000', 'last_undegraded': '2024-03-06T07:25:19.786003+0000', 'last_fullsized': '2024-03-06T07:25:19.786003+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.786591+0000', 'last_change': '2024-03-06T07:25:19.786591+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.786591+0000', 'last_undegraded': '2024-03-06T07:25:19.786591+0000', 'last_fullsized': '2024-03-06T07:25:19.786591+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.792586+0000', 'last_change': '2024-03-06T07:25:19.792586+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.792586+0000', 'last_undegraded': '2024-03-06T07:25:19.792586+0000', 'last_fullsized': '2024-03-06T07:25:19.792586+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.789100+0000', 'last_change': '2024-03-06T07:25:19.789100+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.789100+0000', 'last_undegraded': '2024-03-06T07:25:19.789100+0000', 'last_fullsized': '2024-03-06T07:25:19.789100+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788943+0000', 'last_change': '2024-03-06T07:25:19.788943+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788943+0000', 'last_undegraded': '2024-03-06T07:25:19.788943+0000', 'last_fullsized': '2024-03-06T07:25:19.788943+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2024-03-06T07:25:24.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.781837+0000', 'last_change': '2024-03-06T07:25:19.781837+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.781837+0000', 'last_undegraded': '2024-03-06T07:25:19.781837+0000', 'last_fullsized': '2024-03-06T07:25:19.781837+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.788147+0000', 'last_change': '2024-03-06T07:25:19.788147+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.788147+0000', 'last_undegraded': '2024-03-06T07:25:19.788147+0000', 'last_fullsized': '2024-03-06T07:25:19.788147+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.785270+0000', 'last_change': '2024-03-06T07:25:19.785270+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.785270+0000', 'last_undegraded': '2024-03-06T07:25:19.785270+0000', 'last_fullsized': '2024-03-06T07:25:19.785270+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.784010+0000', 'last_change': '2024-03-06T07:25:19.784010+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.784010+0000', 'last_undegraded': '2024-03-06T07:25:19.784010+0000', 'last_fullsized': '2024-03-06T07:25:19.784010+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2024-03-06T07:25:24.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 133, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:19.783262+0000', 'last_change': '2024-03-06T07:25:19.783262+0000', 'last_active': '2024-03-06T07:25:19.773061+0000', 'last_peered': '2024-03-06T07:25:19.773061+0000', 'last_clean': '2024-03-06T07:25:19.773061+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:19.783262+0000', 'last_undegraded': '2024-03-06T07:25:19.783262+0000', 'last_fullsized': '2024-03-06T07:25:19.783262+0000', 'mapping_epoch': 133, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 133, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:19.773061+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:24.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:25:24.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.637+0000 7f00718df640 1 Processor -- start 2024-03-06T07:25:24.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.637+0000 7f00718df640 1 -- start start 2024-03-06T07:25:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f00718df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f006c1527b0 0x7f006c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f00718df640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f006c0c7180 0x7f006c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f00718df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c0c7c60 con 0x7f006c155b20 2024-03-06T07:25:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f00718df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c159640 con 0x7f006c1527b0 2024-03-06T07:25:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f00718df640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c0c8df0 con 0x7f006c0c7180 2024-03-06T07:25:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f006affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f006c1527b0 0x7f006c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f006b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f006c155b20 0x7f006c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37276/0 (socket says 172.21.15.26:37276) 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f006affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f006c1527b0 0x7f006c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37546/0 (socket says 172.21.15.26:37546) 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.638+0000 7f006b7fe640 1 -- 172.21.15.26:0/384410623 learned_addr learned my addr 172.21.15.26:0/384410623 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f006a7fc640 1 --2- 172.21.15.26:0/384410623 >> v2:172.21.15.117:3300/0 conn(0x7f006c0c7180 0x7f006c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f006affd640 1 -- 172.21.15.26:0/384410623 >> v2:172.21.15.117:3300/0 conn(0x7f006c0c7180 msgr2=0x7f006c0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f006affd640 1 --2- 172.21.15.26:0/384410623 >> v2:172.21.15.117:3300/0 conn(0x7f006c0c7180 0x7f006c0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f006affd640 1 -- 172.21.15.26:0/384410623 >> v1:172.21.15.26:6789/0 conn(0x7f006c155b20 legacy=0x7f006c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f006affd640 1 -- 172.21.15.26:0/384410623 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0c8060 con 0x7f006c1527b0 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f0069ffb640 1 -- 172.21.15.26:0/384410623 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964107727 0 0) 0x7f0064001490 con 0x7f006c155b20 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f006a7fc640 1 --2- 172.21.15.26:0/384410623 >> v2:172.21.15.117:3300/0 conn(0x7f006c0c7180 0x7f006c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.639+0000 7f006affd640 1 --2- 172.21.15.26:0/384410623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c159100 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f006c08cac0 tx=0x7f005c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0da45c550f3685 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f0069ffb640 1 -- 172.21.15.26:0/384410623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c0315a0 con 0x7f006c1527b0 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f0069ffb640 1 -- 172.21.15.26:0/384410623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c037070 con 0x7f006c1527b0 2024-03-06T07:25:24.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f0069ffb640 1 -- 172.21.15.26:0/384410623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c031960 con 0x7f006c1527b0 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 -- 172.21.15.26:0/384410623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 msgr2=0x7f006c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 --2- 172.21.15.26:0/384410623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c159100 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f006c08cac0 tx=0x7f005c02b840 comp rx=0 tx=0).stop 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 -- 172.21.15.26:0/384410623 shutdown_connections 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 --2- 172.21.15.26:0/384410623 >> v2:172.21.15.117:3300/0 conn(0x7f006c0c7180 0x7f006c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 --2- 172.21.15.26:0/384410623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c159100 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 -- 172.21.15.26:0/384410623 >> 172.21.15.26:0/384410623 conn(0x7f006c0ac560 msgr2=0x7f006c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 -- 172.21.15.26:0/384410623 shutdown_connections 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.640+0000 7f00718df640 1 -- 172.21.15.26:0/384410623 wait complete. 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.642+0000 7f00718df640 1 Processor -- start 2024-03-06T07:25:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.642+0000 7f00718df640 1 -- start start 2024-03-06T07:25:24.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f00718df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c0c7180 0x7f006c0c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f00718df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c0c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f00718df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f006c155b20 0x7f006c0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f00718df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c155320 con 0x7f006c1527b0 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f00718df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0c7980 con 0x7f006c0c7180 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f00718df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c155600 con 0x7f006c155b20 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f006a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c0c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f006a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c0c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37556/0 (socket says 172.21.15.26:37556) 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.643+0000 7f006a7fc640 1 -- 172.21.15.26:0/2343023272 learned_addr learned my addr 172.21.15.26:0/2343023272 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006b7fe640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f006c155b20 0x7f006c0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006affd640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c0c7180 0x7f006c0c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006a7fc640 1 -- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f006c155b20 msgr2=0x7f006c0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006a7fc640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f006c155b20 0x7f006c0c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006a7fc640 1 -- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c0c7180 msgr2=0x7f006c0c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006a7fc640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c0c7180 0x7f006c0c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006a7fc640 1 -- 172.21.15.26:0/2343023272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c14bc70 con 0x7f006c1527b0 2024-03-06T07:25:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f006a7fc640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c0c3ce0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f005400c7f0 tx=0x7f005400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f0063fff640 1 -- 172.21.15.26:0/2343023272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054015070 con 0x7f006c1527b0 2024-03-06T07:25:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f0063fff640 1 -- 172.21.15.26:0/2343023272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0054007820 con 0x7f006c1527b0 2024-03-06T07:25:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c157040 con 0x7f006c1527b0 2024-03-06T07:25:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f0063fff640 1 -- 172.21.15.26:0/2343023272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054007bd0 con 0x7f006c1527b0 2024-03-06T07:25:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.644+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c0c2e60 con 0x7f006c1527b0 2024-03-06T07:25:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.645+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c04f330 con 0x7f006c1527b0 2024-03-06T07:25:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.646+0000 7f0063fff640 1 -- 172.21.15.26:0/2343023272 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0054018070 con 0x7f006c1527b0 2024-03-06T07:25:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.646+0000 7f0063fff640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f004c06b4d0 0x7f004c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.646+0000 7f0063fff640 1 -- 172.21.15.26:0/2343023272 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f005408a4c0 con 0x7f006c1527b0 2024-03-06T07:25:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.646+0000 7f006affd640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f004c06b4d0 0x7f004c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.647+0000 7f006affd640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f004c06b4d0 0x7f004c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f005c036000 tx=0x7f005c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.650+0000 7f0063fff640 1 -- 172.21.15.26:0/2343023272 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f00540591b0 con 0x7f006c1527b0 2024-03-06T07:25:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.902+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f006c06ce00 con 0x7f006c1527b0 2024-03-06T07:25:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.903+0000 7f0063fff640 1 -- 172.21.15.26:0/2343023272 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1458 (secure 0 0 0) 0x7f0054058ab0 con 0x7f006c1527b0 2024-03-06T07:25:24.904 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:24.905 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":340,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":135,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709595,"num_in_osds":8,"osd_in_since":1709709586,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":268865536,"bytes_avail":772825247744,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":121,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":8295}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-03-06T07:25:01.092176+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:25:24.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f004c06b4d0 msgr2=0x7f004c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f004c06b4d0 0x7f004c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f005c036000 tx=0x7f005c008030 comp rx=0 tx=0).stop 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 msgr2=0x7f006c0c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c0c3ce0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f005400c7f0 tx=0x7f005400ccb0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 shutdown_connections 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f004c06b4d0 0x7f004c06d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f006c155b20 0x7f006c0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1527b0 0x7f006c0c3ce0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 --2- 172.21.15.26:0/2343023272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c0c7180 0x7f006c0c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 >> 172.21.15.26:0/2343023272 conn(0x7f006c0ac560 msgr2=0x7f006c0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 shutdown_connections 2024-03-06T07:25:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:24.905+0000 7f00718df640 1 -- 172.21.15.26:0/2343023272 wait complete. 2024-03-06T07:25:24.917 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:25:24.917 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.017+0000 7f5900579640 1 Processor -- start 2024-03-06T07:25:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.018+0000 7f5900579640 1 -- start start 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f5900579640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58f806d9f0 0x7f58f80a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f5900579640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f8158800 con 0x7f58f80c3710 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f5900579640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f8158ae0 con 0x7f58f80c3280 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f5900579640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f8158dc0 con 0x7f58f806d9f0 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f58fe315640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58f806d9f0 0x7f58f80a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f58feb16640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f58f80c3710 0x7f58f81558f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37286/0 (socket says 172.21.15.26:37286) 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f58feb16640 1 -- 172.21.15.26:0/2071927264 learned_addr learned my addr 172.21.15.26:0/2071927264 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f58fe315640 1 -- 172.21.15.26:0/2071927264 >> v1:172.21.15.117:6789/0 conn(0x7f58f80c3280 legacy=0x7f58f81551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f58fe315640 1 -- 172.21.15.26:0/2071927264 >> v1:172.21.15.26:6789/0 conn(0x7f58f80c3710 legacy=0x7f58f81558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f58fe315640 1 -- 172.21.15.26:0/2071927264 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f81590a0 con 0x7f58f806d9f0 2024-03-06T07:25:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.019+0000 7f58fd313640 1 -- 172.21.15.26:0/2071927264 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862680002 0 0) 0x7f58f8158800 con 0x7f58f80c3710 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f58fe315640 1 --2- 172.21.15.26:0/2071927264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a7cd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097a0 tx=0x7f58f002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9117e7092c24c3f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f58fd313640 1 -- 172.21.15.26:0/2071927264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0038070 con 0x7f58f806d9f0 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f58fd313640 1 -- 172.21.15.26:0/2071927264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0031700 con 0x7f58f806d9f0 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f58fd313640 1 -- 172.21.15.26:0/2071927264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0031ad0 con 0x7f58f806d9f0 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f5900579640 1 -- 172.21.15.26:0/2071927264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 msgr2=0x7f58f80a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f5900579640 1 --2- 172.21.15.26:0/2071927264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a7cd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097a0 tx=0x7f58f002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f5900579640 1 -- 172.21.15.26:0/2071927264 shutdown_connections 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f5900579640 1 --2- 172.21.15.26:0/2071927264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a7cd0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f5900579640 1 -- 172.21.15.26:0/2071927264 >> 172.21.15.26:0/2071927264 conn(0x7f58f80aa630 msgr2=0x7f58f80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f5900579640 1 -- 172.21.15.26:0/2071927264 shutdown_connections 2024-03-06T07:25:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.021+0000 7f5900579640 1 -- 172.21.15.26:0/2071927264 wait complete. 2024-03-06T07:25:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.022+0000 7f5900579640 1 Processor -- start 2024-03-06T07:25:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f5900579640 1 -- start start 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f5900579640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f5900579640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58f80c3280 0x7f58f81e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f5900579640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58f80c3710 0x7f58f81e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f5900579640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f806df50 con 0x7f58f806d9f0 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f5900579640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f806ddd0 con 0x7f58f80c3710 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f5900579640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80a8210 con 0x7f58f80c3280 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f58fe315640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f58fe315640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37576/0 (socket says 172.21.15.26:37576) 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.023+0000 7f58fe315640 1 -- 172.21.15.26:0/2938150674 learned_addr learned my addr 172.21.15.26:0/2938150674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58fe315640 1 -- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58f80c3280 msgr2=0x7f58f81e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58fe315640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58f80c3280 0x7f58f81e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58fe315640 1 -- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58f80c3710 msgr2=0x7f58f81e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58fe315640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58f80c3710 0x7f58f81e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58fe315640 1 -- 172.21.15.26:0/2938150674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f81ea840 con 0x7f58f806d9f0 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58fe315640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a9b30 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097a0 tx=0x7f58f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58f6ffd640 1 -- 172.21.15.26:0/2938150674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0043070 con 0x7f58f806d9f0 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58f6ffd640 1 -- 172.21.15.26:0/2938150674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0033a30 con 0x7f58f806d9f0 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f8158210 con 0x7f58f806d9f0 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f58f6ffd640 1 -- 172.21.15.26:0/2938150674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0038040 con 0x7f58f806d9f0 2024-03-06T07:25:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.024+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81ee8e0 con 0x7f58f806d9f0 2024-03-06T07:25:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.025+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f804f330 con 0x7f58f806d9f0 2024-03-06T07:25:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.025+0000 7f58f6ffd640 1 -- 172.21.15.26:0/2938150674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58f0046070 con 0x7f58f806d9f0 2024-03-06T07:25:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.026+0000 7f58f6ffd640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58e406b4d0 0x7f58e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.026+0000 7f58f6ffd640 1 -- 172.21.15.26:0/2938150674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58f00ac1c0 con 0x7f58f806d9f0 2024-03-06T07:25:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.026+0000 7f58fdb14640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58e406b4d0 0x7f58e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.027+0000 7f58fdb14640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58e406b4d0 0x7f58e406d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f58f814bca0 tx=0x7f58ec0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:25.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.035+0000 7f58f6ffd640 1 -- 172.21.15.26:0/2938150674 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f58f0030070 con 0x7f58f806d9f0 2024-03-06T07:25:25.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.217+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58f81eec80 con 0x7f58e406b4d0 2024-03-06T07:25:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.223+0000 7f58f6ffd640 1 -- 172.21.15.26:0/2938150674 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252563 (secure 0 0 0) 0x7f58f81eec80 con 0x7f58e406b4d0 2024-03-06T07:25:25.227 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:25:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58e406b4d0 msgr2=0x7f58e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58e406b4d0 0x7f58e406d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f58f814bca0 tx=0x7f58ec0079b0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 msgr2=0x7f58f80a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a9b30 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097a0 tx=0x7f58f00046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 shutdown_connections 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58e406b4d0 0x7f58e406d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58f80c3710 0x7f58f81e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58f80c3280 0x7f58f81e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 --2- 172.21.15.26:0/2938150674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f806d9f0 0x7f58f80a9b30 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 >> 172.21.15.26:0/2938150674 conn(0x7f58f80aa630 msgr2=0x7f58f81551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 shutdown_connections 2024-03-06T07:25:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:25.226+0000 7f5900579640 1 -- 172.21.15.26:0/2938150674 wait complete. 2024-03-06T07:25:25.242 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-03-06T07:25:25.097404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262888,"kb_used_data":13440,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711832,"statfs":{"total":773094113280,"available":772824915968,"internally_reserved":0,"allocated":13762560,"data_stored":7728564,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001095"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786929+0000","last_change":"2024-03-06T07:25:20.804807+0000","last_active":"2024-03-06T07:25:22.786929+0000","last_peered":"2024-03-06T07:25:22.786929+0000","last_clean":"2024-03-06T07:25:22.786929+0000","last_became_active":"2024-03-06T07:25:20.804360+0000","last_became_peered":"2024-03-06T07:25:20.804360+0000","last_unstale":"2024-03-06T07:25:22.786929+0000","last_undegraded":"2024-03-06T07:25:22.786929+0000","last_fullsized":"2024-03-06T07:25:22.786929+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:31.302114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176009+0000","last_change":"2024-03-06T07:25:20.804907+0000","last_active":"2024-03-06T07:25:23.176009+0000","last_peered":"2024-03-06T07:25:23.176009+0000","last_clean":"2024-03-06T07:25:23.176009+0000","last_became_active":"2024-03-06T07:25:20.804651+0000","last_became_peered":"2024-03-06T07:25:20.804651+0000","last_unstale":"2024-03-06T07:25:23.176009+0000","last_undegraded":"2024-03-06T07:25:23.176009+0000","last_fullsized":"2024-03-06T07:25:23.176009+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:25.221063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176102+0000","last_change":"2024-03-06T07:25:20.798433+0000","last_active":"2024-03-06T07:25:23.176102+0000","last_peered":"2024-03-06T07:25:23.176102+0000","last_clean":"2024-03-06T07:25:23.176102+0000","last_became_active":"2024-03-06T07:25:20.798153+0000","last_became_peered":"2024-03-06T07:25:20.798153+0000","last_unstale":"2024-03-06T07:25:23.176102+0000","last_undegraded":"2024-03-06T07:25:23.176102+0000","last_fullsized":"2024-03-06T07:25:23.176102+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:00.402363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175931+0000","last_change":"2024-03-06T07:25:20.799036+0000","last_active":"2024-03-06T07:25:23.175931+0000","last_peered":"2024-03-06T07:25:23.175931+0000","last_clean":"2024-03-06T07:25:23.175931+0000","last_became_active":"2024-03-06T07:25:20.798703+0000","last_became_peered":"2024-03-06T07:25:20.798703+0000","last_unstale":"2024-03-06T07:25:23.175931+0000","last_undegraded":"2024-03-06T07:25:23.175931+0000","last_fullsized":"2024-03-06T07:25:23.175931+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:52.211606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176729+0000","last_change":"2024-03-06T07:25:20.799886+0000","last_active":"2024-03-06T07:25:23.176729+0000","last_peered":"2024-03-06T07:25:23.176729+0000","last_clean":"2024-03-06T07:25:23.176729+0000","last_became_active":"2024-03-06T07:25:20.799607+0000","last_became_peered":"2024-03-06T07:25:20.799607+0000","last_unstale":"2024-03-06T07:25:23.176729+0000","last_undegraded":"2024-03-06T07:25:23.176729+0000","last_fullsized":"2024-03-06T07:25:23.176729+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:23.190252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176729+0000","last_change":"2024-03-06T07:25:20.799883+0000","last_active":"2024-03-06T07:25:23.176729+0000","last_peered":"2024-03-06T07:25:23.176729+0000","last_clean":"2024-03-06T07:25:23.176729+0000","last_became_active":"2024-03-06T07:25:20.799440+0000","last_became_peered":"2024-03-06T07:25:20.799440+0000","last_unstale":"2024-03-06T07:25:23.176729+0000","last_undegraded":"2024-03-06T07:25:23.176729+0000","last_fullsized":"2024-03-06T07:25:23.176729+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:27:38.405208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786615+0000","last_change":"2024-03-06T07:25:20.795682+0000","last_active":"2024-03-06T07:25:22.786615+0000","last_peered":"2024-03-06T07:25:22.786615+0000","last_clean":"2024-03-06T07:25:22.786615+0000","last_became_active":"2024-03-06T07:25:20.795226+0000","last_became_peered":"2024-03-06T07:25:20.795226+0000","last_unstale":"2024-03-06T07:25:22.786615+0000","last_undegraded":"2024-03-06T07:25:22.786615+0000","last_fullsized":"2024-03-06T07:25:22.786615+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:19.230467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787253+0000","last_change":"2024-03-06T07:25:20.801418+0000","last_active":"2024-03-06T07:25:21.787253+0000","last_peered":"2024-03-06T07:25:21.787253+0000","last_clean":"2024-03-06T07:25:21.787253+0000","last_became_active":"2024-03-06T07:25:20.801143+0000","last_became_peered":"2024-03-06T07:25:20.801143+0000","last_unstale":"2024-03-06T07:25:21.787253+0000","last_undegraded":"2024-03-06T07:25:21.787253+0000","last_fullsized":"2024-03-06T07:25:21.787253+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:42.609468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176191+0000","last_change":"2024-03-06T07:25:20.807824+0000","last_active":"2024-03-06T07:25:23.176191+0000","last_peered":"2024-03-06T07:25:23.176191+0000","last_clean":"2024-03-06T07:25:23.176191+0000","last_became_active":"2024-03-06T07:25:20.807506+0000","last_became_peered":"2024-03-06T07:25:20.807506+0000","last_unstale":"2024-03-06T07:25:23.176191+0000","last_undegraded":"2024-03-06T07:25:23.176191+0000","last_fullsized":"2024-03-06T07:25:23.176191+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:52.162571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176341+0000","last_change":"2024-03-06T07:25:20.807654+0000","last_active":"2024-03-06T07:25:23.176341+0000","last_peered":"2024-03-06T07:25:23.176341+0000","last_clean":"2024-03-06T07:25:23.176341+0000","last_became_active":"2024-03-06T07:25:20.807327+0000","last_became_peered":"2024-03-06T07:25:20.807327+0000","last_unstale":"2024-03-06T07:25:23.176341+0000","last_undegraded":"2024-03-06T07:25:23.176341+0000","last_fullsized":"2024-03-06T07:25:23.176341+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:26.295322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177040+0000","last_change":"2024-03-06T07:25:20.795487+0000","last_active":"2024-03-06T07:25:23.177040+0000","last_peered":"2024-03-06T07:25:23.177040+0000","last_clean":"2024-03-06T07:25:23.177040+0000","last_became_active":"2024-03-06T07:25:20.794983+0000","last_became_peered":"2024-03-06T07:25:20.794983+0000","last_unstale":"2024-03-06T07:25:23.177040+0000","last_undegraded":"2024-03-06T07:25:23.177040+0000","last_fullsized":"2024-03-06T07:25:23.177040+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:24.628930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175933+0000","last_change":"2024-03-06T07:25:20.811371+0000","last_active":"2024-03-06T07:25:23.175933+0000","last_peered":"2024-03-06T07:25:23.175933+0000","last_clean":"2024-03-06T07:25:23.175933+0000","last_became_active":"2024-03-06T07:25:20.811040+0000","last_became_peered":"2024-03-06T07:25:20.811040+0000","last_unstale":"2024-03-06T07:25:23.175933+0000","last_undegraded":"2024-03-06T07:25:23.175933+0000","last_fullsized":"2024-03-06T07:25:23.175933+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:17.297783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176491+0000","last_change":"2024-03-06T07:25:20.807627+0000","last_active":"2024-03-06T07:25:23.176491+0000","last_peered":"2024-03-06T07:25:23.176491+0000","last_clean":"2024-03-06T07:25:23.176491+0000","last_became_active":"2024-03-06T07:25:20.807100+0000","last_became_peered":"2024-03-06T07:25:20.807100+0000","last_unstale":"2024-03-06T07:25:23.176491+0000","last_undegraded":"2024-03-06T07:25:23.176491+0000","last_fullsized":"2024-03-06T07:25:23.176491+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:43:36.617191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175941+0000","last_change":"2024-03-06T07:25:20.802427+0000","last_active":"2024-03-06T07:25:23.175941+0000","last_peered":"2024-03-06T07:25:23.175941+0000","last_clean":"2024-03-06T07:25:23.175941+0000","last_became_active":"2024-03-06T07:25:20.802032+0000","last_became_peered":"2024-03-06T07:25:20.802032+0000","last_unstale":"2024-03-06T07:25:23.175941+0000","last_undegraded":"2024-03-06T07:25:23.175941+0000","last_fullsized":"2024-03-06T07:25:23.175941+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:27.877841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175623+0000","last_change":"2024-03-06T07:25:20.800655+0000","last_active":"2024-03-06T07:25:23.175623+0000","last_peered":"2024-03-06T07:25:23.175623+0000","last_clean":"2024-03-06T07:25:23.175623+0000","last_became_active":"2024-03-06T07:25:20.800383+0000","last_became_peered":"2024-03-06T07:25:20.800383+0000","last_unstale":"2024-03-06T07:25:23.175623+0000","last_undegraded":"2024-03-06T07:25:23.175623+0000","last_fullsized":"2024-03-06T07:25:23.175623+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:31.882947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175895+0000","last_change":"2024-03-06T07:25:20.809926+0000","last_active":"2024-03-06T07:25:23.175895+0000","last_peered":"2024-03-06T07:25:23.175895+0000","last_clean":"2024-03-06T07:25:23.175895+0000","last_became_active":"2024-03-06T07:25:20.808230+0000","last_became_peered":"2024-03-06T07:25:20.808230+0000","last_unstale":"2024-03-06T07:25:23.175895+0000","last_undegraded":"2024-03-06T07:25:23.175895+0000","last_fullsized":"2024-03-06T07:25:23.175895+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:20.878873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176326+0000","last_change":"2024-03-06T07:25:20.801015+0000","last_active":"2024-03-06T07:25:23.176326+0000","last_peered":"2024-03-06T07:25:23.176326+0000","last_clean":"2024-03-06T07:25:23.176326+0000","last_became_active":"2024-03-06T07:25:20.800727+0000","last_became_peered":"2024-03-06T07:25:20.800727+0000","last_unstale":"2024-03-06T07:25:23.176326+0000","last_undegraded":"2024-03-06T07:25:23.176326+0000","last_fullsized":"2024-03-06T07:25:23.176326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:09.703019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177318+0000","last_change":"2024-03-06T07:25:20.797251+0000","last_active":"2024-03-06T07:25:23.177318+0000","last_peered":"2024-03-06T07:25:23.177318+0000","last_clean":"2024-03-06T07:25:23.177318+0000","last_became_active":"2024-03-06T07:25:20.796917+0000","last_became_peered":"2024-03-06T07:25:20.796917+0000","last_unstale":"2024-03-06T07:25:23.177318+0000","last_undegraded":"2024-03-06T07:25:23.177318+0000","last_fullsized":"2024-03-06T07:25:23.177318+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:36.520973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786678+0000","last_change":"2024-03-06T07:25:20.788309+0000","last_active":"2024-03-06T07:25:21.786678+0000","last_peered":"2024-03-06T07:25:21.786678+0000","last_clean":"2024-03-06T07:25:21.786678+0000","last_became_active":"2024-03-06T07:25:20.787962+0000","last_became_peered":"2024-03-06T07:25:20.787962+0000","last_unstale":"2024-03-06T07:25:21.786678+0000","last_undegraded":"2024-03-06T07:25:21.786678+0000","last_fullsized":"2024-03-06T07:25:21.786678+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:20.878873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134735+0000","last_change":"2024-03-06T07:25:20.803075+0000","last_active":"2024-03-06T07:25:22.134735+0000","last_peered":"2024-03-06T07:25:22.134735+0000","last_clean":"2024-03-06T07:25:22.134735+0000","last_became_active":"2024-03-06T07:25:20.802786+0000","last_became_peered":"2024-03-06T07:25:20.802786+0000","last_unstale":"2024-03-06T07:25:22.134735+0000","last_undegraded":"2024-03-06T07:25:22.134735+0000","last_fullsized":"2024-03-06T07:25:22.134735+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:27:38.405208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176549+0000","last_change":"2024-03-06T07:25:20.809764+0000","last_active":"2024-03-06T07:25:23.176549+0000","last_peered":"2024-03-06T07:25:23.176549+0000","last_clean":"2024-03-06T07:25:23.176549+0000","last_became_active":"2024-03-06T07:25:20.808007+0000","last_became_peered":"2024-03-06T07:25:20.808007+0000","last_unstale":"2024-03-06T07:25:23.176549+0000","last_undegraded":"2024-03-06T07:25:23.176549+0000","last_fullsized":"2024-03-06T07:25:23.176549+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:46:39.265614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786818+0000","last_change":"2024-03-06T07:25:20.795691+0000","last_active":"2024-03-06T07:25:22.786818+0000","last_peered":"2024-03-06T07:25:22.786818+0000","last_clean":"2024-03-06T07:25:22.786818+0000","last_became_active":"2024-03-06T07:25:20.795434+0000","last_became_peered":"2024-03-06T07:25:20.795434+0000","last_unstale":"2024-03-06T07:25:22.786818+0000","last_undegraded":"2024-03-06T07:25:22.786818+0000","last_fullsized":"2024-03-06T07:25:22.786818+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:23.623640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787581+0000","last_change":"2024-03-06T07:25:20.798729+0000","last_active":"2024-03-06T07:25:21.787581+0000","last_peered":"2024-03-06T07:25:21.787581+0000","last_clean":"2024-03-06T07:25:21.787581+0000","last_became_active":"2024-03-06T07:25:20.798013+0000","last_became_peered":"2024-03-06T07:25:20.798013+0000","last_unstale":"2024-03-06T07:25:21.787581+0000","last_undegraded":"2024-03-06T07:25:21.787581+0000","last_fullsized":"2024-03-06T07:25:21.787581+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:27.877841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176947+0000","last_change":"2024-03-06T07:25:20.808949+0000","last_active":"2024-03-06T07:25:23.176947+0000","last_peered":"2024-03-06T07:25:23.176947+0000","last_clean":"2024-03-06T07:25:23.176947+0000","last_became_active":"2024-03-06T07:25:20.808673+0000","last_became_peered":"2024-03-06T07:25:20.808673+0000","last_unstale":"2024-03-06T07:25:23.176947+0000","last_undegraded":"2024-03-06T07:25:23.176947+0000","last_fullsized":"2024-03-06T07:25:23.176947+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:37:43.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787025+0000","last_change":"2024-03-06T07:25:20.802065+0000","last_active":"2024-03-06T07:25:22.787025+0000","last_peered":"2024-03-06T07:25:22.787025+0000","last_clean":"2024-03-06T07:25:22.787025+0000","last_became_active":"2024-03-06T07:25:20.801761+0000","last_became_peered":"2024-03-06T07:25:20.801761+0000","last_unstale":"2024-03-06T07:25:22.787025+0000","last_undegraded":"2024-03-06T07:25:22.787025+0000","last_fullsized":"2024-03-06T07:25:22.787025+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:06:14.995202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786829+0000","last_change":"2024-03-06T07:25:20.802165+0000","last_active":"2024-03-06T07:25:22.786829+0000","last_peered":"2024-03-06T07:25:22.786829+0000","last_clean":"2024-03-06T07:25:22.786829+0000","last_became_active":"2024-03-06T07:25:20.801879+0000","last_became_peered":"2024-03-06T07:25:20.801879+0000","last_unstale":"2024-03-06T07:25:22.786829+0000","last_undegraded":"2024-03-06T07:25:22.786829+0000","last_fullsized":"2024-03-06T07:25:22.786829+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:51.978965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176042+0000","last_change":"2024-03-06T07:25:20.808182+0000","last_active":"2024-03-06T07:25:23.176042+0000","last_peered":"2024-03-06T07:25:23.176042+0000","last_clean":"2024-03-06T07:25:23.176042+0000","last_became_active":"2024-03-06T07:25:20.807868+0000","last_became_peered":"2024-03-06T07:25:20.807868+0000","last_unstale":"2024-03-06T07:25:23.176042+0000","last_undegraded":"2024-03-06T07:25:23.176042+0000","last_fullsized":"2024-03-06T07:25:23.176042+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:44.614572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177269+0000","last_change":"2024-03-06T07:25:20.800712+0000","last_active":"2024-03-06T07:25:23.177269+0000","last_peered":"2024-03-06T07:25:23.177269+0000","last_clean":"2024-03-06T07:25:23.177269+0000","last_became_active":"2024-03-06T07:25:20.800379+0000","last_became_peered":"2024-03-06T07:25:20.800379+0000","last_unstale":"2024-03-06T07:25:23.177269+0000","last_undegraded":"2024-03-06T07:25:23.177269+0000","last_fullsized":"2024-03-06T07:25:23.177269+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:30:26.776837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787531+0000","last_change":"2024-03-06T07:25:20.801436+0000","last_active":"2024-03-06T07:25:21.787531+0000","last_peered":"2024-03-06T07:25:21.787531+0000","last_clean":"2024-03-06T07:25:21.787531+0000","last_became_active":"2024-03-06T07:25:20.800603+0000","last_became_peered":"2024-03-06T07:25:20.800603+0000","last_unstale":"2024-03-06T07:25:21.787531+0000","last_undegraded":"2024-03-06T07:25:21.787531+0000","last_fullsized":"2024-03-06T07:25:21.787531+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:09.151335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177029+0000","last_change":"2024-03-06T07:25:20.795945+0000","last_active":"2024-03-06T07:25:23.177029+0000","last_peered":"2024-03-06T07:25:23.177029+0000","last_clean":"2024-03-06T07:25:23.177029+0000","last_became_active":"2024-03-06T07:25:20.795661+0000","last_became_peered":"2024-03-06T07:25:20.795661+0000","last_unstale":"2024-03-06T07:25:23.177029+0000","last_undegraded":"2024-03-06T07:25:23.177029+0000","last_fullsized":"2024-03-06T07:25:23.177029+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:09:10.040781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176238+0000","last_change":"2024-03-06T07:25:20.807674+0000","last_active":"2024-03-06T07:25:23.176238+0000","last_peered":"2024-03-06T07:25:23.176238+0000","last_clean":"2024-03-06T07:25:23.176238+0000","last_became_active":"2024-03-06T07:25:20.807348+0000","last_became_peered":"2024-03-06T07:25:20.807348+0000","last_unstale":"2024-03-06T07:25:23.176238+0000","last_undegraded":"2024-03-06T07:25:23.176238+0000","last_fullsized":"2024-03-06T07:25:23.176238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:25.887031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787112+0000","last_change":"2024-03-06T07:25:20.801409+0000","last_active":"2024-03-06T07:25:21.787112+0000","last_peered":"2024-03-06T07:25:21.787112+0000","last_clean":"2024-03-06T07:25:21.787112+0000","last_became_active":"2024-03-06T07:25:20.800983+0000","last_became_peered":"2024-03-06T07:25:20.800983+0000","last_unstale":"2024-03-06T07:25:21.787112+0000","last_undegraded":"2024-03-06T07:25:21.787112+0000","last_fullsized":"2024-03-06T07:25:21.787112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:40:15.230445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176200+0000","last_change":"2024-03-06T07:25:20.798349+0000","last_active":"2024-03-06T07:25:23.176200+0000","last_peered":"2024-03-06T07:25:23.176200+0000","last_clean":"2024-03-06T07:25:23.176200+0000","last_became_active":"2024-03-06T07:25:20.798048+0000","last_became_peered":"2024-03-06T07:25:20.798048+0000","last_unstale":"2024-03-06T07:25:23.176200+0000","last_undegraded":"2024-03-06T07:25:23.176200+0000","last_fullsized":"2024-03-06T07:25:23.176200+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:23.623640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"134'4","reported_seq":18,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176415+0000","last_change":"2024-03-06T07:25:19.800026+0000","last_active":"2024-03-06T07:25:23.176415+0000","last_peered":"2024-03-06T07:25:23.176415+0000","last_clean":"2024-03-06T07:25:23.176415+0000","last_became_active":"2024-03-06T07:25:19.799694+0000","last_became_peered":"2024-03-06T07:25:19.799694+0000","last_unstale":"2024-03-06T07:25:23.176415+0000","last_undegraded":"2024-03-06T07:25:23.176415+0000","last_fullsized":"2024-03-06T07:25:23.176415+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:24.214078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176241+0000","last_change":"2024-03-06T07:25:20.802835+0000","last_active":"2024-03-06T07:25:23.176241+0000","last_peered":"2024-03-06T07:25:23.176241+0000","last_clean":"2024-03-06T07:25:23.176241+0000","last_became_active":"2024-03-06T07:25:20.802543+0000","last_became_peered":"2024-03-06T07:25:20.802543+0000","last_unstale":"2024-03-06T07:25:23.176241+0000","last_undegraded":"2024-03-06T07:25:23.176241+0000","last_fullsized":"2024-03-06T07:25:23.176241+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:51.978965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786411+0000","last_change":"2024-03-06T07:25:19.794850+0000","last_active":"2024-03-06T07:25:21.786411+0000","last_peered":"2024-03-06T07:25:21.786411+0000","last_clean":"2024-03-06T07:25:21.786411+0000","last_became_active":"2024-03-06T07:25:19.794587+0000","last_became_peered":"2024-03-06T07:25:19.794587+0000","last_unstale":"2024-03-06T07:25:21.786411+0000","last_undegraded":"2024-03-06T07:25:21.786411+0000","last_fullsized":"2024-03-06T07:25:21.786411+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:22.616655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787494+0000","last_change":"2024-03-06T07:25:20.792527+0000","last_active":"2024-03-06T07:25:21.787494+0000","last_peered":"2024-03-06T07:25:21.787494+0000","last_clean":"2024-03-06T07:25:21.787494+0000","last_became_active":"2024-03-06T07:25:20.792261+0000","last_became_peered":"2024-03-06T07:25:20.792261+0000","last_unstale":"2024-03-06T07:25:21.787494+0000","last_undegraded":"2024-03-06T07:25:21.787494+0000","last_fullsized":"2024-03-06T07:25:21.787494+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:21.259945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786564+0000","last_change":"2024-03-06T07:25:19.796329+0000","last_active":"2024-03-06T07:25:21.786564+0000","last_peered":"2024-03-06T07:25:21.786564+0000","last_clean":"2024-03-06T07:25:21.786564+0000","last_became_active":"2024-03-06T07:25:19.796081+0000","last_became_peered":"2024-03-06T07:25:19.796081+0000","last_unstale":"2024-03-06T07:25:21.786564+0000","last_undegraded":"2024-03-06T07:25:21.786564+0000","last_fullsized":"2024-03-06T07:25:21.786564+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:30.295129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786502+0000","last_change":"2024-03-06T07:25:20.796806+0000","last_active":"2024-03-06T07:25:21.786502+0000","last_peered":"2024-03-06T07:25:21.786502+0000","last_clean":"2024-03-06T07:25:21.786502+0000","last_became_active":"2024-03-06T07:25:20.796533+0000","last_became_peered":"2024-03-06T07:25:20.796533+0000","last_unstale":"2024-03-06T07:25:21.786502+0000","last_undegraded":"2024-03-06T07:25:21.786502+0000","last_fullsized":"2024-03-06T07:25:21.786502+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:16.264569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176288+0000","last_change":"2024-03-06T07:25:19.789450+0000","last_active":"2024-03-06T07:25:23.176288+0000","last_peered":"2024-03-06T07:25:23.176288+0000","last_clean":"2024-03-06T07:25:23.176288+0000","last_became_active":"2024-03-06T07:25:19.789139+0000","last_became_peered":"2024-03-06T07:25:19.789139+0000","last_unstale":"2024-03-06T07:25:23.176288+0000","last_undegraded":"2024-03-06T07:25:23.176288+0000","last_fullsized":"2024-03-06T07:25:23.176288+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:34.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134587+0000","last_change":"2024-03-06T07:25:20.803756+0000","last_active":"2024-03-06T07:25:22.134587+0000","last_peered":"2024-03-06T07:25:22.134587+0000","last_clean":"2024-03-06T07:25:22.134587+0000","last_became_active":"2024-03-06T07:25:20.803514+0000","last_became_peered":"2024-03-06T07:25:20.803514+0000","last_unstale":"2024-03-06T07:25:22.134587+0000","last_undegraded":"2024-03-06T07:25:22.134587+0000","last_fullsized":"2024-03-06T07:25:22.134587+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:46:39.265614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787682+0000","last_change":"2024-03-06T07:25:19.797666+0000","last_active":"2024-03-06T07:25:21.787682+0000","last_peered":"2024-03-06T07:25:21.787682+0000","last_clean":"2024-03-06T07:25:21.787682+0000","last_became_active":"2024-03-06T07:25:19.797391+0000","last_became_peered":"2024-03-06T07:25:19.797391+0000","last_unstale":"2024-03-06T07:25:21.787682+0000","last_undegraded":"2024-03-06T07:25:21.787682+0000","last_fullsized":"2024-03-06T07:25:21.787682+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:54.608998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175914+0000","last_change":"2024-03-06T07:25:20.797676+0000","last_active":"2024-03-06T07:25:23.175914+0000","last_peered":"2024-03-06T07:25:23.175914+0000","last_clean":"2024-03-06T07:25:23.175914+0000","last_became_active":"2024-03-06T07:25:20.797374+0000","last_became_peered":"2024-03-06T07:25:20.797374+0000","last_unstale":"2024-03-06T07:25:23.175914+0000","last_undegraded":"2024-03-06T07:25:23.175914+0000","last_fullsized":"2024-03-06T07:25:23.175914+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:35.871130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176969+0000","last_change":"2024-03-06T07:25:19.799648+0000","last_active":"2024-03-06T07:25:23.176969+0000","last_peered":"2024-03-06T07:25:23.176969+0000","last_clean":"2024-03-06T07:25:23.176969+0000","last_became_active":"2024-03-06T07:25:19.799352+0000","last_became_peered":"2024-03-06T07:25:19.799352+0000","last_unstale":"2024-03-06T07:25:23.176969+0000","last_undegraded":"2024-03-06T07:25:23.176969+0000","last_fullsized":"2024-03-06T07:25:23.176969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:18.223482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175782+0000","last_change":"2024-03-06T07:25:20.801510+0000","last_active":"2024-03-06T07:25:23.175782+0000","last_peered":"2024-03-06T07:25:23.175782+0000","last_clean":"2024-03-06T07:25:23.175782+0000","last_became_active":"2024-03-06T07:25:20.801227+0000","last_became_peered":"2024-03-06T07:25:20.801227+0000","last_unstale":"2024-03-06T07:25:23.175782+0000","last_undegraded":"2024-03-06T07:25:23.175782+0000","last_fullsized":"2024-03-06T07:25:23.175782+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:29.136905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786545+0000","last_change":"2024-03-06T07:25:19.797676+0000","last_active":"2024-03-06T07:25:22.786545+0000","last_peered":"2024-03-06T07:25:22.786545+0000","last_clean":"2024-03-06T07:25:22.786545+0000","last_became_active":"2024-03-06T07:25:19.797399+0000","last_became_peered":"2024-03-06T07:25:19.797399+0000","last_unstale":"2024-03-06T07:25:22.786545+0000","last_undegraded":"2024-03-06T07:25:22.786545+0000","last_fullsized":"2024-03-06T07:25:22.786545+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:26:54.998525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786593+0000","last_change":"2024-03-06T07:25:20.804750+0000","last_active":"2024-03-06T07:25:22.786593+0000","last_peered":"2024-03-06T07:25:22.786593+0000","last_clean":"2024-03-06T07:25:22.786593+0000","last_became_active":"2024-03-06T07:25:20.804379+0000","last_became_peered":"2024-03-06T07:25:20.804379+0000","last_unstale":"2024-03-06T07:25:22.786593+0000","last_undegraded":"2024-03-06T07:25:22.786593+0000","last_fullsized":"2024-03-06T07:25:22.786593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:00.402363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175985+0000","last_change":"2024-03-06T07:25:19.802156+0000","last_active":"2024-03-06T07:25:23.175985+0000","last_peered":"2024-03-06T07:25:23.175985+0000","last_clean":"2024-03-06T07:25:23.175985+0000","last_became_active":"2024-03-06T07:25:19.801881+0000","last_became_peered":"2024-03-06T07:25:19.801881+0000","last_unstale":"2024-03-06T07:25:23.175985+0000","last_undegraded":"2024-03-06T07:25:23.175985+0000","last_fullsized":"2024-03-06T07:25:23.175985+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:59.395378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787625+0000","last_change":"2024-03-06T07:25:20.793249+0000","last_active":"2024-03-06T07:25:21.787625+0000","last_peered":"2024-03-06T07:25:21.787625+0000","last_clean":"2024-03-06T07:25:21.787625+0000","last_became_active":"2024-03-06T07:25:20.792954+0000","last_became_peered":"2024-03-06T07:25:20.792954+0000","last_unstale":"2024-03-06T07:25:21.787625+0000","last_undegraded":"2024-03-06T07:25:21.787625+0000","last_fullsized":"2024-03-06T07:25:21.787625+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:17.297783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786479+0000","last_change":"2024-03-06T07:25:19.795560+0000","last_active":"2024-03-06T07:25:21.786479+0000","last_peered":"2024-03-06T07:25:21.786479+0000","last_clean":"2024-03-06T07:25:21.786479+0000","last_became_active":"2024-03-06T07:25:19.795293+0000","last_became_peered":"2024-03-06T07:25:19.795293+0000","last_unstale":"2024-03-06T07:25:21.786479+0000","last_undegraded":"2024-03-06T07:25:21.786479+0000","last_fullsized":"2024-03-06T07:25:21.786479+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:12.667706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176908+0000","last_change":"2024-03-06T07:25:20.795250+0000","last_active":"2024-03-06T07:25:23.176908+0000","last_peered":"2024-03-06T07:25:23.176908+0000","last_clean":"2024-03-06T07:25:23.176908+0000","last_became_active":"2024-03-06T07:25:20.794551+0000","last_became_peered":"2024-03-06T07:25:20.794551+0000","last_unstale":"2024-03-06T07:25:23.176908+0000","last_undegraded":"2024-03-06T07:25:23.176908+0000","last_fullsized":"2024-03-06T07:25:23.176908+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:38:19.157766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786986+0000","last_change":"2024-03-06T07:25:19.798578+0000","last_active":"2024-03-06T07:25:22.786986+0000","last_peered":"2024-03-06T07:25:22.786986+0000","last_clean":"2024-03-06T07:25:22.786986+0000","last_became_active":"2024-03-06T07:25:19.798263+0000","last_became_peered":"2024-03-06T07:25:19.798263+0000","last_unstale":"2024-03-06T07:25:22.786986+0000","last_undegraded":"2024-03-06T07:25:22.786986+0000","last_fullsized":"2024-03-06T07:25:22.786986+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:41.773697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177379+0000","last_change":"2024-03-06T07:25:20.797282+0000","last_active":"2024-03-06T07:25:23.177379+0000","last_peered":"2024-03-06T07:25:23.177379+0000","last_clean":"2024-03-06T07:25:23.177379+0000","last_became_active":"2024-03-06T07:25:20.796941+0000","last_became_peered":"2024-03-06T07:25:20.796941+0000","last_unstale":"2024-03-06T07:25:23.177379+0000","last_undegraded":"2024-03-06T07:25:23.177379+0000","last_fullsized":"2024-03-06T07:25:23.177379+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:08.725602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"134'3","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177168+0000","last_change":"2024-03-06T07:25:19.792711+0000","last_active":"2024-03-06T07:25:23.177168+0000","last_peered":"2024-03-06T07:25:23.177168+0000","last_clean":"2024-03-06T07:25:23.177168+0000","last_became_active":"2024-03-06T07:25:19.792405+0000","last_became_peered":"2024-03-06T07:25:19.792405+0000","last_unstale":"2024-03-06T07:25:23.177168+0000","last_undegraded":"2024-03-06T07:25:23.177168+0000","last_fullsized":"2024-03-06T07:25:23.177168+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:03:06.439333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787297+0000","last_change":"2024-03-06T07:25:20.793655+0000","last_active":"2024-03-06T07:25:21.787297+0000","last_peered":"2024-03-06T07:25:21.787297+0000","last_clean":"2024-03-06T07:25:21.787297+0000","last_became_active":"2024-03-06T07:25:20.793109+0000","last_became_peered":"2024-03-06T07:25:20.793109+0000","last_unstale":"2024-03-06T07:25:21.787297+0000","last_undegraded":"2024-03-06T07:25:21.787297+0000","last_fullsized":"2024-03-06T07:25:21.787297+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:59.537190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176719+0000","last_change":"2024-03-06T07:25:19.800658+0000","last_active":"2024-03-06T07:25:23.176719+0000","last_peered":"2024-03-06T07:25:23.176719+0000","last_clean":"2024-03-06T07:25:23.176719+0000","last_became_active":"2024-03-06T07:25:19.800361+0000","last_became_peered":"2024-03-06T07:25:19.800361+0000","last_unstale":"2024-03-06T07:25:23.176719+0000","last_undegraded":"2024-03-06T07:25:23.176719+0000","last_fullsized":"2024-03-06T07:25:23.176719+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:22.616655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175681+0000","last_change":"2024-03-06T07:25:20.805290+0000","last_active":"2024-03-06T07:25:23.175681+0000","last_peered":"2024-03-06T07:25:23.175681+0000","last_clean":"2024-03-06T07:25:23.175681+0000","last_became_active":"2024-03-06T07:25:20.802094+0000","last_became_peered":"2024-03-06T07:25:20.802094+0000","last_unstale":"2024-03-06T07:25:23.175681+0000","last_undegraded":"2024-03-06T07:25:23.175681+0000","last_fullsized":"2024-03-06T07:25:23.175681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:06.346122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175860+0000","last_change":"2024-03-06T07:25:19.802236+0000","last_active":"2024-03-06T07:25:23.175860+0000","last_peered":"2024-03-06T07:25:23.175860+0000","last_clean":"2024-03-06T07:25:23.175860+0000","last_became_active":"2024-03-06T07:25:19.801866+0000","last_became_peered":"2024-03-06T07:25:19.801866+0000","last_unstale":"2024-03-06T07:25:23.175860+0000","last_undegraded":"2024-03-06T07:25:23.175860+0000","last_fullsized":"2024-03-06T07:25:23.175860+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:08.859617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176883+0000","last_change":"2024-03-06T07:25:20.798080+0000","last_active":"2024-03-06T07:25:23.176883+0000","last_peered":"2024-03-06T07:25:23.176883+0000","last_clean":"2024-03-06T07:25:23.176883+0000","last_became_active":"2024-03-06T07:25:20.797792+0000","last_became_peered":"2024-03-06T07:25:20.797792+0000","last_unstale":"2024-03-06T07:25:23.176883+0000","last_undegraded":"2024-03-06T07:25:23.176883+0000","last_fullsized":"2024-03-06T07:25:23.176883+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:48.629438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176048+0000","last_change":"2024-03-06T07:25:19.800076+0000","last_active":"2024-03-06T07:25:23.176048+0000","last_peered":"2024-03-06T07:25:23.176048+0000","last_clean":"2024-03-06T07:25:23.176048+0000","last_became_active":"2024-03-06T07:25:19.799501+0000","last_became_peered":"2024-03-06T07:25:19.799501+0000","last_unstale":"2024-03-06T07:25:23.176048+0000","last_undegraded":"2024-03-06T07:25:23.176048+0000","last_fullsized":"2024-03-06T07:25:23.176048+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:27.059865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175933+0000","last_change":"2024-03-06T07:25:20.811709+0000","last_active":"2024-03-06T07:25:23.175933+0000","last_peered":"2024-03-06T07:25:23.175933+0000","last_clean":"2024-03-06T07:25:23.175933+0000","last_became_active":"2024-03-06T07:25:20.811403+0000","last_became_peered":"2024-03-06T07:25:20.811403+0000","last_unstale":"2024-03-06T07:25:23.175933+0000","last_undegraded":"2024-03-06T07:25:23.175933+0000","last_fullsized":"2024-03-06T07:25:23.175933+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:15.329851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"134'8","reported_seq":22,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786402+0000","last_change":"2024-03-06T07:25:19.797067+0000","last_active":"2024-03-06T07:25:22.786402+0000","last_peered":"2024-03-06T07:25:22.786402+0000","last_clean":"2024-03-06T07:25:22.786402+0000","last_became_active":"2024-03-06T07:25:19.796773+0000","last_became_peered":"2024-03-06T07:25:19.796773+0000","last_unstale":"2024-03-06T07:25:22.786402+0000","last_undegraded":"2024-03-06T07:25:22.786402+0000","last_fullsized":"2024-03-06T07:25:22.786402+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:16.290798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786133+0000","last_change":"2024-03-06T07:25:20.808364+0000","last_active":"2024-03-06T07:25:21.786133+0000","last_peered":"2024-03-06T07:25:21.786133+0000","last_clean":"2024-03-06T07:25:21.786133+0000","last_became_active":"2024-03-06T07:25:20.808035+0000","last_became_peered":"2024-03-06T07:25:20.808035+0000","last_unstale":"2024-03-06T07:25:21.786133+0000","last_undegraded":"2024-03-06T07:25:21.786133+0000","last_fullsized":"2024-03-06T07:25:21.786133+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:30.948895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177243+0000","last_change":"2024-03-06T07:25:19.789597+0000","last_active":"2024-03-06T07:25:23.177243+0000","last_peered":"2024-03-06T07:25:23.177243+0000","last_clean":"2024-03-06T07:25:23.177243+0000","last_became_active":"2024-03-06T07:25:19.789334+0000","last_became_peered":"2024-03-06T07:25:19.789334+0000","last_unstale":"2024-03-06T07:25:23.177243+0000","last_undegraded":"2024-03-06T07:25:23.177243+0000","last_fullsized":"2024-03-06T07:25:23.177243+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:43.183726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786457+0000","last_change":"2024-03-06T07:25:20.797381+0000","last_active":"2024-03-06T07:25:21.786457+0000","last_peered":"2024-03-06T07:25:21.786457+0000","last_clean":"2024-03-06T07:25:21.786457+0000","last_became_active":"2024-03-06T07:25:20.797071+0000","last_became_peered":"2024-03-06T07:25:20.797071+0000","last_unstale":"2024-03-06T07:25:21.786457+0000","last_undegraded":"2024-03-06T07:25:21.786457+0000","last_fullsized":"2024-03-06T07:25:21.786457+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:08.382103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786533+0000","last_change":"2024-03-06T07:25:19.796190+0000","last_active":"2024-03-06T07:25:21.786533+0000","last_peered":"2024-03-06T07:25:21.786533+0000","last_clean":"2024-03-06T07:25:21.786533+0000","last_became_active":"2024-03-06T07:25:19.795879+0000","last_became_peered":"2024-03-06T07:25:19.795879+0000","last_unstale":"2024-03-06T07:25:21.786533+0000","last_undegraded":"2024-03-06T07:25:21.786533+0000","last_fullsized":"2024-03-06T07:25:21.786533+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:24.214078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786819+0000","last_change":"2024-03-06T07:25:20.807614+0000","last_active":"2024-03-06T07:25:22.786819+0000","last_peered":"2024-03-06T07:25:22.786819+0000","last_clean":"2024-03-06T07:25:22.786819+0000","last_became_active":"2024-03-06T07:25:20.807350+0000","last_became_peered":"2024-03-06T07:25:20.807350+0000","last_unstale":"2024-03-06T07:25:22.786819+0000","last_undegraded":"2024-03-06T07:25:22.786819+0000","last_fullsized":"2024-03-06T07:25:22.786819+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:25.221063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176322+0000","last_change":"2024-03-06T07:25:19.798347+0000","last_active":"2024-03-06T07:25:23.176322+0000","last_peered":"2024-03-06T07:25:23.176322+0000","last_clean":"2024-03-06T07:25:23.176322+0000","last_became_active":"2024-03-06T07:25:19.798021+0000","last_became_peered":"2024-03-06T07:25:19.798021+0000","last_unstale":"2024-03-06T07:25:23.176322+0000","last_undegraded":"2024-03-06T07:25:23.176322+0000","last_fullsized":"2024-03-06T07:25:23.176322+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:26.870856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787120+0000","last_change":"2024-03-06T07:25:20.795987+0000","last_active":"2024-03-06T07:25:22.787120+0000","last_peered":"2024-03-06T07:25:22.787120+0000","last_clean":"2024-03-06T07:25:22.787120+0000","last_became_active":"2024-03-06T07:25:20.795728+0000","last_became_peered":"2024-03-06T07:25:20.795728+0000","last_unstale":"2024-03-06T07:25:22.787120+0000","last_undegraded":"2024-03-06T07:25:22.787120+0000","last_fullsized":"2024-03-06T07:25:22.787120+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:15:43.836498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176067+0000","last_change":"2024-03-06T07:25:19.801586+0000","last_active":"2024-03-06T07:25:23.176067+0000","last_peered":"2024-03-06T07:25:23.176067+0000","last_clean":"2024-03-06T07:25:23.176067+0000","last_became_active":"2024-03-06T07:25:19.801242+0000","last_became_peered":"2024-03-06T07:25:19.801242+0000","last_unstale":"2024-03-06T07:25:23.176067+0000","last_undegraded":"2024-03-06T07:25:23.176067+0000","last_fullsized":"2024-03-06T07:25:23.176067+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:43.584796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.785920+0000","last_change":"2024-03-06T07:25:20.797641+0000","last_active":"2024-03-06T07:25:21.785920+0000","last_peered":"2024-03-06T07:25:21.785920+0000","last_clean":"2024-03-06T07:25:21.785920+0000","last_became_active":"2024-03-06T07:25:20.797376+0000","last_became_peered":"2024-03-06T07:25:20.797376+0000","last_unstale":"2024-03-06T07:25:21.785920+0000","last_undegraded":"2024-03-06T07:25:21.785920+0000","last_fullsized":"2024-03-06T07:25:21.785920+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:10.857381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134609+0000","last_change":"2024-03-06T07:25:19.796315+0000","last_active":"2024-03-06T07:25:22.134609+0000","last_peered":"2024-03-06T07:25:22.134609+0000","last_clean":"2024-03-06T07:25:22.134609+0000","last_became_active":"2024-03-06T07:25:19.796036+0000","last_became_peered":"2024-03-06T07:25:19.796036+0000","last_unstale":"2024-03-06T07:25:22.134609+0000","last_undegraded":"2024-03-06T07:25:22.134609+0000","last_fullsized":"2024-03-06T07:25:22.134609+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:51.155586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175823+0000","last_change":"2024-03-06T07:25:20.797245+0000","last_active":"2024-03-06T07:25:23.175823+0000","last_peered":"2024-03-06T07:25:23.175823+0000","last_clean":"2024-03-06T07:25:23.175823+0000","last_became_active":"2024-03-06T07:25:20.796952+0000","last_became_peered":"2024-03-06T07:25:20.796952+0000","last_unstale":"2024-03-06T07:25:23.175823+0000","last_undegraded":"2024-03-06T07:25:23.175823+0000","last_fullsized":"2024-03-06T07:25:23.175823+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:13.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787220+0000","last_change":"2024-03-06T07:25:19.789672+0000","last_active":"2024-03-06T07:25:21.787220+0000","last_peered":"2024-03-06T07:25:21.787220+0000","last_clean":"2024-03-06T07:25:21.787220+0000","last_became_active":"2024-03-06T07:25:19.789409+0000","last_became_peered":"2024-03-06T07:25:19.789409+0000","last_unstale":"2024-03-06T07:25:21.787220+0000","last_undegraded":"2024-03-06T07:25:21.787220+0000","last_fullsized":"2024-03-06T07:25:21.787220+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:56.743191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786068+0000","last_change":"2024-03-06T07:25:20.799585+0000","last_active":"2024-03-06T07:25:21.786068+0000","last_peered":"2024-03-06T07:25:21.786068+0000","last_clean":"2024-03-06T07:25:21.786068+0000","last_became_active":"2024-03-06T07:25:20.799321+0000","last_became_peered":"2024-03-06T07:25:20.799321+0000","last_unstale":"2024-03-06T07:25:21.786068+0000","last_undegraded":"2024-03-06T07:25:21.786068+0000","last_fullsized":"2024-03-06T07:25:21.786068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:28:53.428162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134476+0000","last_change":"2024-03-06T07:25:19.797588+0000","last_active":"2024-03-06T07:25:22.134476+0000","last_peered":"2024-03-06T07:25:22.134476+0000","last_clean":"2024-03-06T07:25:22.134476+0000","last_became_active":"2024-03-06T07:25:19.797322+0000","last_became_peered":"2024-03-06T07:25:19.797322+0000","last_unstale":"2024-03-06T07:25:22.134476+0000","last_undegraded":"2024-03-06T07:25:22.134476+0000","last_fullsized":"2024-03-06T07:25:22.134476+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:41:38.265186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176177+0000","last_change":"2024-03-06T07:25:20.801382+0000","last_active":"2024-03-06T07:25:23.176177+0000","last_peered":"2024-03-06T07:25:23.176177+0000","last_clean":"2024-03-06T07:25:23.176177+0000","last_became_active":"2024-03-06T07:25:20.801111+0000","last_became_peered":"2024-03-06T07:25:20.801111+0000","last_unstale":"2024-03-06T07:25:23.176177+0000","last_undegraded":"2024-03-06T07:25:23.176177+0000","last_fullsized":"2024-03-06T07:25:23.176177+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:19.230467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787764+0000","last_change":"2024-03-06T07:25:19.794012+0000","last_active":"2024-03-06T07:25:21.787764+0000","last_peered":"2024-03-06T07:25:21.787764+0000","last_clean":"2024-03-06T07:25:21.787764+0000","last_became_active":"2024-03-06T07:25:19.793743+0000","last_became_peered":"2024-03-06T07:25:19.793743+0000","last_unstale":"2024-03-06T07:25:21.787764+0000","last_undegraded":"2024-03-06T07:25:21.787764+0000","last_fullsized":"2024-03-06T07:25:21.787764+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:39.912413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176091+0000","last_change":"2024-03-06T07:25:20.804519+0000","last_active":"2024-03-06T07:25:23.176091+0000","last_peered":"2024-03-06T07:25:23.176091+0000","last_clean":"2024-03-06T07:25:23.176091+0000","last_became_active":"2024-03-06T07:25:20.804262+0000","last_became_peered":"2024-03-06T07:25:20.804262+0000","last_unstale":"2024-03-06T07:25:23.176091+0000","last_undegraded":"2024-03-06T07:25:23.176091+0000","last_fullsized":"2024-03-06T07:25:23.176091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:41:39.272171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176327+0000","last_change":"2024-03-06T07:25:19.801518+0000","last_active":"2024-03-06T07:25:23.176327+0000","last_peered":"2024-03-06T07:25:23.176327+0000","last_clean":"2024-03-06T07:25:23.176327+0000","last_became_active":"2024-03-06T07:25:19.801213+0000","last_became_peered":"2024-03-06T07:25:19.801213+0000","last_unstale":"2024-03-06T07:25:23.176327+0000","last_undegraded":"2024-03-06T07:25:23.176327+0000","last_fullsized":"2024-03-06T07:25:23.176327+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:51.514593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176026+0000","last_change":"2024-03-06T07:25:20.811157+0000","last_active":"2024-03-06T07:25:23.176026+0000","last_peered":"2024-03-06T07:25:23.176026+0000","last_clean":"2024-03-06T07:25:23.176026+0000","last_became_active":"2024-03-06T07:25:20.810825+0000","last_became_peered":"2024-03-06T07:25:20.810825+0000","last_unstale":"2024-03-06T07:25:23.176026+0000","last_undegraded":"2024-03-06T07:25:23.176026+0000","last_fullsized":"2024-03-06T07:25:23.176026+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:42.609468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"135'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787090+0000","last_change":"2024-03-06T07:25:19.793750+0000","last_active":"2024-03-06T07:25:22.787090+0000","last_peered":"2024-03-06T07:25:22.787090+0000","last_clean":"2024-03-06T07:25:22.787090+0000","last_became_active":"2024-03-06T07:25:19.793481+0000","last_became_peered":"2024-03-06T07:25:19.793481+0000","last_unstale":"2024-03-06T07:25:22.787090+0000","last_undegraded":"2024-03-06T07:25:22.787090+0000","last_fullsized":"2024-03-06T07:25:22.787090+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:50.049774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787026+0000","last_change":"2024-03-06T07:25:20.804879+0000","last_active":"2024-03-06T07:25:22.787026+0000","last_peered":"2024-03-06T07:25:22.787026+0000","last_clean":"2024-03-06T07:25:22.787026+0000","last_became_active":"2024-03-06T07:25:20.804423+0000","last_became_peered":"2024-03-06T07:25:20.804423+0000","last_unstale":"2024-03-06T07:25:22.787026+0000","last_undegraded":"2024-03-06T07:25:22.787026+0000","last_fullsized":"2024-03-06T07:25:22.787026+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:13.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176406+0000","last_change":"2024-03-06T07:25:19.793729+0000","last_active":"2024-03-06T07:25:23.176406+0000","last_peered":"2024-03-06T07:25:23.176406+0000","last_clean":"2024-03-06T07:25:23.176406+0000","last_became_active":"2024-03-06T07:25:19.793432+0000","last_became_peered":"2024-03-06T07:25:19.793432+0000","last_unstale":"2024-03-06T07:25:23.176406+0000","last_undegraded":"2024-03-06T07:25:23.176406+0000","last_fullsized":"2024-03-06T07:25:23.176406+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:51.204621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176328+0000","last_change":"2024-03-06T07:25:20.803635+0000","last_active":"2024-03-06T07:25:23.176328+0000","last_peered":"2024-03-06T07:25:23.176328+0000","last_clean":"2024-03-06T07:25:23.176328+0000","last_became_active":"2024-03-06T07:25:20.803382+0000","last_became_peered":"2024-03-06T07:25:20.803382+0000","last_unstale":"2024-03-06T07:25:23.176328+0000","last_undegraded":"2024-03-06T07:25:23.176328+0000","last_fullsized":"2024-03-06T07:25:23.176328+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:51.048107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176180+0000","last_change":"2024-03-06T07:25:19.793893+0000","last_active":"2024-03-06T07:25:23.176180+0000","last_peered":"2024-03-06T07:25:23.176180+0000","last_clean":"2024-03-06T07:25:23.176180+0000","last_became_active":"2024-03-06T07:25:19.793604+0000","last_became_peered":"2024-03-06T07:25:19.793604+0000","last_unstale":"2024-03-06T07:25:23.176180+0000","last_undegraded":"2024-03-06T07:25:23.176180+0000","last_fullsized":"2024-03-06T07:25:23.176180+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:50.971979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787703+0000","last_change":"2024-03-06T07:25:20.803289+0000","last_active":"2024-03-06T07:25:21.787703+0000","last_peered":"2024-03-06T07:25:21.787703+0000","last_clean":"2024-03-06T07:25:21.787703+0000","last_became_active":"2024-03-06T07:25:20.803019+0000","last_became_peered":"2024-03-06T07:25:20.803019+0000","last_unstale":"2024-03-06T07:25:21.787703+0000","last_undegraded":"2024-03-06T07:25:21.787703+0000","last_fullsized":"2024-03-06T07:25:21.787703+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:09.703019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786722+0000","last_change":"2024-03-06T07:25:19.797615+0000","last_active":"2024-03-06T07:25:22.786722+0000","last_peered":"2024-03-06T07:25:22.786722+0000","last_clean":"2024-03-06T07:25:22.786722+0000","last_became_active":"2024-03-06T07:25:19.797345+0000","last_became_peered":"2024-03-06T07:25:19.797345+0000","last_unstale":"2024-03-06T07:25:22.786722+0000","last_undegraded":"2024-03-06T07:25:22.786722+0000","last_fullsized":"2024-03-06T07:25:22.786722+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:08.696034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175873+0000","last_change":"2024-03-06T07:25:20.810575+0000","last_active":"2024-03-06T07:25:23.175873+0000","last_peered":"2024-03-06T07:25:23.175873+0000","last_clean":"2024-03-06T07:25:23.175873+0000","last_became_active":"2024-03-06T07:25:20.810317+0000","last_became_peered":"2024-03-06T07:25:20.810317+0000","last_unstale":"2024-03-06T07:25:23.175873+0000","last_undegraded":"2024-03-06T07:25:23.175873+0000","last_fullsized":"2024-03-06T07:25:23.175873+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:09.151335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177125+0000","last_change":"2024-03-06T07:25:19.791673+0000","last_active":"2024-03-06T07:25:23.177125+0000","last_peered":"2024-03-06T07:25:23.177125+0000","last_clean":"2024-03-06T07:25:23.177125+0000","last_became_active":"2024-03-06T07:25:19.791395+0000","last_became_peered":"2024-03-06T07:25:19.791395+0000","last_unstale":"2024-03-06T07:25:23.177125+0000","last_undegraded":"2024-03-06T07:25:23.177125+0000","last_fullsized":"2024-03-06T07:25:23.177125+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:53.894137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175932+0000","last_change":"2024-03-06T07:25:20.802114+0000","last_active":"2024-03-06T07:25:23.175932+0000","last_peered":"2024-03-06T07:25:23.175932+0000","last_clean":"2024-03-06T07:25:23.175932+0000","last_became_active":"2024-03-06T07:25:20.801842+0000","last_became_peered":"2024-03-06T07:25:20.801842+0000","last_unstale":"2024-03-06T07:25:23.175932+0000","last_undegraded":"2024-03-06T07:25:23.175932+0000","last_fullsized":"2024-03-06T07:25:23.175932+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:31.302114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786719+0000","last_change":"2024-03-06T07:25:19.794392+0000","last_active":"2024-03-06T07:25:22.786719+0000","last_peered":"2024-03-06T07:25:22.786719+0000","last_clean":"2024-03-06T07:25:22.786719+0000","last_became_active":"2024-03-06T07:25:19.794116+0000","last_became_peered":"2024-03-06T07:25:19.794116+0000","last_unstale":"2024-03-06T07:25:22.786719+0000","last_undegraded":"2024-03-06T07:25:22.786719+0000","last_fullsized":"2024-03-06T07:25:22.786719+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:41.602483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787331+0000","last_change":"2024-03-06T07:25:20.802319+0000","last_active":"2024-03-06T07:25:21.787331+0000","last_peered":"2024-03-06T07:25:21.787331+0000","last_clean":"2024-03-06T07:25:21.787331+0000","last_became_active":"2024-03-06T07:25:20.799478+0000","last_became_peered":"2024-03-06T07:25:20.799478+0000","last_unstale":"2024-03-06T07:25:21.787331+0000","last_undegraded":"2024-03-06T07:25:21.787331+0000","last_fullsized":"2024-03-06T07:25:21.787331+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:52.211606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.785980+0000","last_change":"2024-03-06T07:25:19.793614+0000","last_active":"2024-03-06T07:25:21.785980+0000","last_peered":"2024-03-06T07:25:21.785980+0000","last_clean":"2024-03-06T07:25:21.785980+0000","last_became_active":"2024-03-06T07:25:19.793338+0000","last_became_peered":"2024-03-06T07:25:19.793338+0000","last_unstale":"2024-03-06T07:25:21.785980+0000","last_undegraded":"2024-03-06T07:25:21.785980+0000","last_fullsized":"2024-03-06T07:25:21.785980+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:50.971979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787553+0000","last_change":"2024-03-06T07:25:20.790396+0000","last_active":"2024-03-06T07:25:21.787553+0000","last_peered":"2024-03-06T07:25:21.787553+0000","last_clean":"2024-03-06T07:25:21.787553+0000","last_became_active":"2024-03-06T07:25:20.790134+0000","last_became_peered":"2024-03-06T07:25:20.790134+0000","last_unstale":"2024-03-06T07:25:21.787553+0000","last_undegraded":"2024-03-06T07:25:21.787553+0000","last_fullsized":"2024-03-06T07:25:21.787553+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:35.871130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"134'6","reported_seq":20,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134715+0000","last_change":"2024-03-06T07:25:19.790987+0000","last_active":"2024-03-06T07:25:22.134715+0000","last_peered":"2024-03-06T07:25:22.134715+0000","last_clean":"2024-03-06T07:25:22.134715+0000","last_became_active":"2024-03-06T07:25:19.790677+0000","last_became_peered":"2024-03-06T07:25:19.790677+0000","last_unstale":"2024-03-06T07:25:22.134715+0000","last_undegraded":"2024-03-06T07:25:22.134715+0000","last_fullsized":"2024-03-06T07:25:22.134715+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:24.880046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":386,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786036+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:25:21.786036+0000","last_peered":"2024-03-06T07:25:21.786036+0000","last_clean":"2024-03-06T07:25:21.786036+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:25:21.786036+0000","last_undegraded":"2024-03-06T07:25:21.786036+0000","last_fullsized":"2024-03-06T07:25:21.786036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":1408266,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":817946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1695744,"data_stored":817876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":819488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":817988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":817946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574916,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32924,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338916,"statfs":{"total":96636764160,"available":96603049984,"internally_reserved":0,"allocated":1785856,"data_stored":1410562,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:26.070 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-03-06T07:25:26.069+0000 7f04198f8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:28.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.347+0000 7fd1b5ab8640 1 Processor -- start 2024-03-06T07:25:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.347+0000 7fd1b5ab8640 1 -- start start 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.348+0000 7fd1b5ab8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1b00c5480 0x7fd1b0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.348+0000 7fd1b5ab8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b01586f0 0x7fd1b0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1b5ab8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b00c5cc0 con 0x7fd1b00c5860 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1b5ab8640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b006df40 con 0x7fd1b00c5480 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1b5ab8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b006fdb0 con 0x7fd1b01586f0 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1af7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1b00c5480 0x7fd1b0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1af7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1b00c5480 0x7fd1b0155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50052/0 (socket says 172.21.15.26:50052) 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1af7fe640 1 -- 172.21.15.26:0/952343076 learned_addr learned my addr 172.21.15.26:0/952343076 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1affff640 1 --2- 172.21.15.26:0/952343076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b01586f0 0x7fd1b0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1af7fe640 1 -- 172.21.15.26:0/952343076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b01586f0 msgr2=0x7fd1b0158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1af7fe640 1 --2- 172.21.15.26:0/952343076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b01586f0 0x7fd1b0158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1af7fe640 1 -- 172.21.15.26:0/952343076 >> v1:172.21.15.117:6789/0 conn(0x7fd1b00c5860 legacy=0x7fd1b006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.349+0000 7fd1af7fe640 1 -- 172.21.15.26:0/952343076 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b01591c0 con 0x7fd1b00c5480 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.350+0000 7fd1af7fe640 1 --2- 172.21.15.26:0/952343076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b00c5480 0x7fd1b0155ba0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009e50 tx=0x7fd1a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cfba7d427d90b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.350+0000 7fd1ae7fc640 1 -- 172.21.15.26:0/952343076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a002ca70 con 0x7fd1b00c5480 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.350+0000 7fd1ae7fc640 1 -- 172.21.15.26:0/952343076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a002cc10 con 0x7fd1b00c5480 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.350+0000 7fd1ae7fc640 1 -- 172.21.15.26:0/952343076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00343f0 con 0x7fd1b00c5480 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.350+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/952343076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b00c5480 msgr2=0x7fd1b0155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.350+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/952343076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b00c5480 0x7fd1b0155ba0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009e50 tx=0x7fd1a002bed0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.350+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/952343076 shutdown_connections 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.351+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/952343076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b01586f0 0x7fd1b0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.351+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/952343076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b00c5480 0x7fd1b0155ba0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.351+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/952343076 >> 172.21.15.26:0/952343076 conn(0x7fd1b006b1a0 msgr2=0x7fd1b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.351+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/952343076 shutdown_connections 2024-03-06T07:25:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.351+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/952343076 wait complete. 2024-03-06T07:25:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 Processor -- start 2024-03-06T07:25:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 -- start start 2024-03-06T07:25:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b00c5480 0x7fd1b014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b00c5860 0x7fd1b01eb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b01586f0 0x7fd1b01eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0155680 con 0x7fd1b01586f0 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01553a0 con 0x7fd1b00c5480 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.352+0000 7fd1b5ab8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0152a20 con 0x7fd1b00c5860 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b01586f0 0x7fd1b01eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b01586f0 0x7fd1b01eefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50066/0 (socket says 172.21.15.26:50066) 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 -- 172.21.15.26:0/483475444 learned_addr learned my addr 172.21.15.26:0/483475444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1aeffd640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b00c5860 0x7fd1b01eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 -- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b00c5860 msgr2=0x7fd1b01eb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b00c5860 0x7fd1b01eb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1af7fe640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b00c5480 0x7fd1b014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 -- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b00c5480 msgr2=0x7fd1b014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b00c5480 0x7fd1b014be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 -- 172.21.15.26:0/483475444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b01eb650 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1affff640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b01586f0 0x7fd1b01eefb0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800e7f0 tx=0x7fd1a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd197fff640 1 -- 172.21.15.26:0/483475444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8017070 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd197fff640 1 -- 172.21.15.26:0/483475444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a800c730 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b01de830 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd197fff640 1 -- 172.21.15.26:0/483475444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800cb20 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.353+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b01ee7b0 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.354+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b004f330 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.355+0000 7fd197fff640 1 -- 172.21.15.26:0/483475444 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1a801a070 con 0x7fd1b01586f0 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.355+0000 7fd197fff640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.355+0000 7fd197fff640 1 -- 172.21.15.26:0/483475444 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1a808a800 con 0x7fd1b01586f0 2024-03-06T07:25:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.355+0000 7fd1af7fe640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.356+0000 7fd1af7fe640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd19c06b4d0 0x7fd19c06d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0035000 tx=0x7fd1a0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.360+0000 7fd197fff640 1 -- 172.21.15.26:0/483475444 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd1a8059580 con 0x7fd1b01586f0 2024-03-06T07:25:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.536+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1b006e0c0 con 0x7fd19c06b4d0 2024-03-06T07:25:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.542+0000 7fd197fff640 1 -- 172.21.15.26:0/483475444 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252563 (secure 0 0 0) 0x7fd1b006e0c0 con 0x7fd19c06b4d0 2024-03-06T07:25:28.546 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:25:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd19c06b4d0 msgr2=0x7fd19c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd19c06b4d0 0x7fd19c06d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0035000 tx=0x7fd1a0030000 comp rx=0 tx=0).stop 2024-03-06T07:25:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b01586f0 msgr2=0x7fd1b01eefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b01586f0 0x7fd1b01eefb0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800e7f0 tx=0x7fd1a800ecb0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 shutdown_connections 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1b01586f0 0x7fd1b01eefb0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1b00c5860 0x7fd1b01eb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 --2- 172.21.15.26:0/483475444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b00c5480 0x7fd1b014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 >> 172.21.15.26:0/483475444 conn(0x7fd1b006b1a0 msgr2=0x7fd1b0070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 shutdown_connections 2024-03-06T07:25:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.545+0000 7fd1b5ab8640 1 -- 172.21.15.26:0/483475444 wait complete. 2024-03-06T07:25:28.557 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-03-06T07:25:27.097979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262888,"kb_used_data":13440,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711832,"statfs":{"total":773094113280,"available":772824915968,"internally_reserved":0,"allocated":13762560,"data_stored":7728564,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.318538"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786929+0000","last_change":"2024-03-06T07:25:20.804807+0000","last_active":"2024-03-06T07:25:22.786929+0000","last_peered":"2024-03-06T07:25:22.786929+0000","last_clean":"2024-03-06T07:25:22.786929+0000","last_became_active":"2024-03-06T07:25:20.804360+0000","last_became_peered":"2024-03-06T07:25:20.804360+0000","last_unstale":"2024-03-06T07:25:22.786929+0000","last_undegraded":"2024-03-06T07:25:22.786929+0000","last_fullsized":"2024-03-06T07:25:22.786929+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:31.302114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176009+0000","last_change":"2024-03-06T07:25:20.804907+0000","last_active":"2024-03-06T07:25:23.176009+0000","last_peered":"2024-03-06T07:25:23.176009+0000","last_clean":"2024-03-06T07:25:23.176009+0000","last_became_active":"2024-03-06T07:25:20.804651+0000","last_became_peered":"2024-03-06T07:25:20.804651+0000","last_unstale":"2024-03-06T07:25:23.176009+0000","last_undegraded":"2024-03-06T07:25:23.176009+0000","last_fullsized":"2024-03-06T07:25:23.176009+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:25.221063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176102+0000","last_change":"2024-03-06T07:25:20.798433+0000","last_active":"2024-03-06T07:25:23.176102+0000","last_peered":"2024-03-06T07:25:23.176102+0000","last_clean":"2024-03-06T07:25:23.176102+0000","last_became_active":"2024-03-06T07:25:20.798153+0000","last_became_peered":"2024-03-06T07:25:20.798153+0000","last_unstale":"2024-03-06T07:25:23.176102+0000","last_undegraded":"2024-03-06T07:25:23.176102+0000","last_fullsized":"2024-03-06T07:25:23.176102+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:00.402363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175931+0000","last_change":"2024-03-06T07:25:20.799036+0000","last_active":"2024-03-06T07:25:23.175931+0000","last_peered":"2024-03-06T07:25:23.175931+0000","last_clean":"2024-03-06T07:25:23.175931+0000","last_became_active":"2024-03-06T07:25:20.798703+0000","last_became_peered":"2024-03-06T07:25:20.798703+0000","last_unstale":"2024-03-06T07:25:23.175931+0000","last_undegraded":"2024-03-06T07:25:23.175931+0000","last_fullsized":"2024-03-06T07:25:23.175931+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:52.211606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176729+0000","last_change":"2024-03-06T07:25:20.799886+0000","last_active":"2024-03-06T07:25:23.176729+0000","last_peered":"2024-03-06T07:25:23.176729+0000","last_clean":"2024-03-06T07:25:23.176729+0000","last_became_active":"2024-03-06T07:25:20.799607+0000","last_became_peered":"2024-03-06T07:25:20.799607+0000","last_unstale":"2024-03-06T07:25:23.176729+0000","last_undegraded":"2024-03-06T07:25:23.176729+0000","last_fullsized":"2024-03-06T07:25:23.176729+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:23.190252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176729+0000","last_change":"2024-03-06T07:25:20.799883+0000","last_active":"2024-03-06T07:25:23.176729+0000","last_peered":"2024-03-06T07:25:23.176729+0000","last_clean":"2024-03-06T07:25:23.176729+0000","last_became_active":"2024-03-06T07:25:20.799440+0000","last_became_peered":"2024-03-06T07:25:20.799440+0000","last_unstale":"2024-03-06T07:25:23.176729+0000","last_undegraded":"2024-03-06T07:25:23.176729+0000","last_fullsized":"2024-03-06T07:25:23.176729+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:27:38.405208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786615+0000","last_change":"2024-03-06T07:25:20.795682+0000","last_active":"2024-03-06T07:25:22.786615+0000","last_peered":"2024-03-06T07:25:22.786615+0000","last_clean":"2024-03-06T07:25:22.786615+0000","last_became_active":"2024-03-06T07:25:20.795226+0000","last_became_peered":"2024-03-06T07:25:20.795226+0000","last_unstale":"2024-03-06T07:25:22.786615+0000","last_undegraded":"2024-03-06T07:25:22.786615+0000","last_fullsized":"2024-03-06T07:25:22.786615+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:19.230467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787253+0000","last_change":"2024-03-06T07:25:20.801418+0000","last_active":"2024-03-06T07:25:21.787253+0000","last_peered":"2024-03-06T07:25:21.787253+0000","last_clean":"2024-03-06T07:25:21.787253+0000","last_became_active":"2024-03-06T07:25:20.801143+0000","last_became_peered":"2024-03-06T07:25:20.801143+0000","last_unstale":"2024-03-06T07:25:21.787253+0000","last_undegraded":"2024-03-06T07:25:21.787253+0000","last_fullsized":"2024-03-06T07:25:21.787253+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:42.609468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176191+0000","last_change":"2024-03-06T07:25:20.807824+0000","last_active":"2024-03-06T07:25:23.176191+0000","last_peered":"2024-03-06T07:25:23.176191+0000","last_clean":"2024-03-06T07:25:23.176191+0000","last_became_active":"2024-03-06T07:25:20.807506+0000","last_became_peered":"2024-03-06T07:25:20.807506+0000","last_unstale":"2024-03-06T07:25:23.176191+0000","last_undegraded":"2024-03-06T07:25:23.176191+0000","last_fullsized":"2024-03-06T07:25:23.176191+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:52.162571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176341+0000","last_change":"2024-03-06T07:25:20.807654+0000","last_active":"2024-03-06T07:25:23.176341+0000","last_peered":"2024-03-06T07:25:23.176341+0000","last_clean":"2024-03-06T07:25:23.176341+0000","last_became_active":"2024-03-06T07:25:20.807327+0000","last_became_peered":"2024-03-06T07:25:20.807327+0000","last_unstale":"2024-03-06T07:25:23.176341+0000","last_undegraded":"2024-03-06T07:25:23.176341+0000","last_fullsized":"2024-03-06T07:25:23.176341+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:26.295322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177040+0000","last_change":"2024-03-06T07:25:20.795487+0000","last_active":"2024-03-06T07:25:23.177040+0000","last_peered":"2024-03-06T07:25:23.177040+0000","last_clean":"2024-03-06T07:25:23.177040+0000","last_became_active":"2024-03-06T07:25:20.794983+0000","last_became_peered":"2024-03-06T07:25:20.794983+0000","last_unstale":"2024-03-06T07:25:23.177040+0000","last_undegraded":"2024-03-06T07:25:23.177040+0000","last_fullsized":"2024-03-06T07:25:23.177040+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:24.628930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175933+0000","last_change":"2024-03-06T07:25:20.811371+0000","last_active":"2024-03-06T07:25:23.175933+0000","last_peered":"2024-03-06T07:25:23.175933+0000","last_clean":"2024-03-06T07:25:23.175933+0000","last_became_active":"2024-03-06T07:25:20.811040+0000","last_became_peered":"2024-03-06T07:25:20.811040+0000","last_unstale":"2024-03-06T07:25:23.175933+0000","last_undegraded":"2024-03-06T07:25:23.175933+0000","last_fullsized":"2024-03-06T07:25:23.175933+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:17.297783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176491+0000","last_change":"2024-03-06T07:25:20.807627+0000","last_active":"2024-03-06T07:25:23.176491+0000","last_peered":"2024-03-06T07:25:23.176491+0000","last_clean":"2024-03-06T07:25:23.176491+0000","last_became_active":"2024-03-06T07:25:20.807100+0000","last_became_peered":"2024-03-06T07:25:20.807100+0000","last_unstale":"2024-03-06T07:25:23.176491+0000","last_undegraded":"2024-03-06T07:25:23.176491+0000","last_fullsized":"2024-03-06T07:25:23.176491+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:43:36.617191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175941+0000","last_change":"2024-03-06T07:25:20.802427+0000","last_active":"2024-03-06T07:25:23.175941+0000","last_peered":"2024-03-06T07:25:23.175941+0000","last_clean":"2024-03-06T07:25:23.175941+0000","last_became_active":"2024-03-06T07:25:20.802032+0000","last_became_peered":"2024-03-06T07:25:20.802032+0000","last_unstale":"2024-03-06T07:25:23.175941+0000","last_undegraded":"2024-03-06T07:25:23.175941+0000","last_fullsized":"2024-03-06T07:25:23.175941+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:27.877841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175623+0000","last_change":"2024-03-06T07:25:20.800655+0000","last_active":"2024-03-06T07:25:23.175623+0000","last_peered":"2024-03-06T07:25:23.175623+0000","last_clean":"2024-03-06T07:25:23.175623+0000","last_became_active":"2024-03-06T07:25:20.800383+0000","last_became_peered":"2024-03-06T07:25:20.800383+0000","last_unstale":"2024-03-06T07:25:23.175623+0000","last_undegraded":"2024-03-06T07:25:23.175623+0000","last_fullsized":"2024-03-06T07:25:23.175623+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:31.882947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175895+0000","last_change":"2024-03-06T07:25:20.809926+0000","last_active":"2024-03-06T07:25:23.175895+0000","last_peered":"2024-03-06T07:25:23.175895+0000","last_clean":"2024-03-06T07:25:23.175895+0000","last_became_active":"2024-03-06T07:25:20.808230+0000","last_became_peered":"2024-03-06T07:25:20.808230+0000","last_unstale":"2024-03-06T07:25:23.175895+0000","last_undegraded":"2024-03-06T07:25:23.175895+0000","last_fullsized":"2024-03-06T07:25:23.175895+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:20.878873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176326+0000","last_change":"2024-03-06T07:25:20.801015+0000","last_active":"2024-03-06T07:25:23.176326+0000","last_peered":"2024-03-06T07:25:23.176326+0000","last_clean":"2024-03-06T07:25:23.176326+0000","last_became_active":"2024-03-06T07:25:20.800727+0000","last_became_peered":"2024-03-06T07:25:20.800727+0000","last_unstale":"2024-03-06T07:25:23.176326+0000","last_undegraded":"2024-03-06T07:25:23.176326+0000","last_fullsized":"2024-03-06T07:25:23.176326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:09.703019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177318+0000","last_change":"2024-03-06T07:25:20.797251+0000","last_active":"2024-03-06T07:25:23.177318+0000","last_peered":"2024-03-06T07:25:23.177318+0000","last_clean":"2024-03-06T07:25:23.177318+0000","last_became_active":"2024-03-06T07:25:20.796917+0000","last_became_peered":"2024-03-06T07:25:20.796917+0000","last_unstale":"2024-03-06T07:25:23.177318+0000","last_undegraded":"2024-03-06T07:25:23.177318+0000","last_fullsized":"2024-03-06T07:25:23.177318+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:36.520973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786678+0000","last_change":"2024-03-06T07:25:20.788309+0000","last_active":"2024-03-06T07:25:21.786678+0000","last_peered":"2024-03-06T07:25:21.786678+0000","last_clean":"2024-03-06T07:25:21.786678+0000","last_became_active":"2024-03-06T07:25:20.787962+0000","last_became_peered":"2024-03-06T07:25:20.787962+0000","last_unstale":"2024-03-06T07:25:21.786678+0000","last_undegraded":"2024-03-06T07:25:21.786678+0000","last_fullsized":"2024-03-06T07:25:21.786678+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:21:20.878873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134735+0000","last_change":"2024-03-06T07:25:20.803075+0000","last_active":"2024-03-06T07:25:22.134735+0000","last_peered":"2024-03-06T07:25:22.134735+0000","last_clean":"2024-03-06T07:25:22.134735+0000","last_became_active":"2024-03-06T07:25:20.802786+0000","last_became_peered":"2024-03-06T07:25:20.802786+0000","last_unstale":"2024-03-06T07:25:22.134735+0000","last_undegraded":"2024-03-06T07:25:22.134735+0000","last_fullsized":"2024-03-06T07:25:22.134735+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:27:38.405208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176549+0000","last_change":"2024-03-06T07:25:20.809764+0000","last_active":"2024-03-06T07:25:23.176549+0000","last_peered":"2024-03-06T07:25:23.176549+0000","last_clean":"2024-03-06T07:25:23.176549+0000","last_became_active":"2024-03-06T07:25:20.808007+0000","last_became_peered":"2024-03-06T07:25:20.808007+0000","last_unstale":"2024-03-06T07:25:23.176549+0000","last_undegraded":"2024-03-06T07:25:23.176549+0000","last_fullsized":"2024-03-06T07:25:23.176549+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:46:39.265614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786818+0000","last_change":"2024-03-06T07:25:20.795691+0000","last_active":"2024-03-06T07:25:22.786818+0000","last_peered":"2024-03-06T07:25:22.786818+0000","last_clean":"2024-03-06T07:25:22.786818+0000","last_became_active":"2024-03-06T07:25:20.795434+0000","last_became_peered":"2024-03-06T07:25:20.795434+0000","last_unstale":"2024-03-06T07:25:22.786818+0000","last_undegraded":"2024-03-06T07:25:22.786818+0000","last_fullsized":"2024-03-06T07:25:22.786818+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:23.623640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787581+0000","last_change":"2024-03-06T07:25:20.798729+0000","last_active":"2024-03-06T07:25:21.787581+0000","last_peered":"2024-03-06T07:25:21.787581+0000","last_clean":"2024-03-06T07:25:21.787581+0000","last_became_active":"2024-03-06T07:25:20.798013+0000","last_became_peered":"2024-03-06T07:25:20.798013+0000","last_unstale":"2024-03-06T07:25:21.787581+0000","last_undegraded":"2024-03-06T07:25:21.787581+0000","last_fullsized":"2024-03-06T07:25:21.787581+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:27.877841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176947+0000","last_change":"2024-03-06T07:25:20.808949+0000","last_active":"2024-03-06T07:25:23.176947+0000","last_peered":"2024-03-06T07:25:23.176947+0000","last_clean":"2024-03-06T07:25:23.176947+0000","last_became_active":"2024-03-06T07:25:20.808673+0000","last_became_peered":"2024-03-06T07:25:20.808673+0000","last_unstale":"2024-03-06T07:25:23.176947+0000","last_undegraded":"2024-03-06T07:25:23.176947+0000","last_fullsized":"2024-03-06T07:25:23.176947+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:37:43.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787025+0000","last_change":"2024-03-06T07:25:20.802065+0000","last_active":"2024-03-06T07:25:22.787025+0000","last_peered":"2024-03-06T07:25:22.787025+0000","last_clean":"2024-03-06T07:25:22.787025+0000","last_became_active":"2024-03-06T07:25:20.801761+0000","last_became_peered":"2024-03-06T07:25:20.801761+0000","last_unstale":"2024-03-06T07:25:22.787025+0000","last_undegraded":"2024-03-06T07:25:22.787025+0000","last_fullsized":"2024-03-06T07:25:22.787025+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:06:14.995202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786829+0000","last_change":"2024-03-06T07:25:20.802165+0000","last_active":"2024-03-06T07:25:22.786829+0000","last_peered":"2024-03-06T07:25:22.786829+0000","last_clean":"2024-03-06T07:25:22.786829+0000","last_became_active":"2024-03-06T07:25:20.801879+0000","last_became_peered":"2024-03-06T07:25:20.801879+0000","last_unstale":"2024-03-06T07:25:22.786829+0000","last_undegraded":"2024-03-06T07:25:22.786829+0000","last_fullsized":"2024-03-06T07:25:22.786829+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:51.978965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176042+0000","last_change":"2024-03-06T07:25:20.808182+0000","last_active":"2024-03-06T07:25:23.176042+0000","last_peered":"2024-03-06T07:25:23.176042+0000","last_clean":"2024-03-06T07:25:23.176042+0000","last_became_active":"2024-03-06T07:25:20.807868+0000","last_became_peered":"2024-03-06T07:25:20.807868+0000","last_unstale":"2024-03-06T07:25:23.176042+0000","last_undegraded":"2024-03-06T07:25:23.176042+0000","last_fullsized":"2024-03-06T07:25:23.176042+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:44.614572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177269+0000","last_change":"2024-03-06T07:25:20.800712+0000","last_active":"2024-03-06T07:25:23.177269+0000","last_peered":"2024-03-06T07:25:23.177269+0000","last_clean":"2024-03-06T07:25:23.177269+0000","last_became_active":"2024-03-06T07:25:20.800379+0000","last_became_peered":"2024-03-06T07:25:20.800379+0000","last_unstale":"2024-03-06T07:25:23.177269+0000","last_undegraded":"2024-03-06T07:25:23.177269+0000","last_fullsized":"2024-03-06T07:25:23.177269+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:30:26.776837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787531+0000","last_change":"2024-03-06T07:25:20.801436+0000","last_active":"2024-03-06T07:25:21.787531+0000","last_peered":"2024-03-06T07:25:21.787531+0000","last_clean":"2024-03-06T07:25:21.787531+0000","last_became_active":"2024-03-06T07:25:20.800603+0000","last_became_peered":"2024-03-06T07:25:20.800603+0000","last_unstale":"2024-03-06T07:25:21.787531+0000","last_undegraded":"2024-03-06T07:25:21.787531+0000","last_fullsized":"2024-03-06T07:25:21.787531+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:09.151335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177029+0000","last_change":"2024-03-06T07:25:20.795945+0000","last_active":"2024-03-06T07:25:23.177029+0000","last_peered":"2024-03-06T07:25:23.177029+0000","last_clean":"2024-03-06T07:25:23.177029+0000","last_became_active":"2024-03-06T07:25:20.795661+0000","last_became_peered":"2024-03-06T07:25:20.795661+0000","last_unstale":"2024-03-06T07:25:23.177029+0000","last_undegraded":"2024-03-06T07:25:23.177029+0000","last_fullsized":"2024-03-06T07:25:23.177029+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:09:10.040781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176238+0000","last_change":"2024-03-06T07:25:20.807674+0000","last_active":"2024-03-06T07:25:23.176238+0000","last_peered":"2024-03-06T07:25:23.176238+0000","last_clean":"2024-03-06T07:25:23.176238+0000","last_became_active":"2024-03-06T07:25:20.807348+0000","last_became_peered":"2024-03-06T07:25:20.807348+0000","last_unstale":"2024-03-06T07:25:23.176238+0000","last_undegraded":"2024-03-06T07:25:23.176238+0000","last_fullsized":"2024-03-06T07:25:23.176238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:25.887031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787112+0000","last_change":"2024-03-06T07:25:20.801409+0000","last_active":"2024-03-06T07:25:21.787112+0000","last_peered":"2024-03-06T07:25:21.787112+0000","last_clean":"2024-03-06T07:25:21.787112+0000","last_became_active":"2024-03-06T07:25:20.800983+0000","last_became_peered":"2024-03-06T07:25:20.800983+0000","last_unstale":"2024-03-06T07:25:21.787112+0000","last_undegraded":"2024-03-06T07:25:21.787112+0000","last_fullsized":"2024-03-06T07:25:21.787112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:40:15.230445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176200+0000","last_change":"2024-03-06T07:25:20.798349+0000","last_active":"2024-03-06T07:25:23.176200+0000","last_peered":"2024-03-06T07:25:23.176200+0000","last_clean":"2024-03-06T07:25:23.176200+0000","last_became_active":"2024-03-06T07:25:20.798048+0000","last_became_peered":"2024-03-06T07:25:20.798048+0000","last_unstale":"2024-03-06T07:25:23.176200+0000","last_undegraded":"2024-03-06T07:25:23.176200+0000","last_fullsized":"2024-03-06T07:25:23.176200+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:23.623640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"134'4","reported_seq":18,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176415+0000","last_change":"2024-03-06T07:25:19.800026+0000","last_active":"2024-03-06T07:25:23.176415+0000","last_peered":"2024-03-06T07:25:23.176415+0000","last_clean":"2024-03-06T07:25:23.176415+0000","last_became_active":"2024-03-06T07:25:19.799694+0000","last_became_peered":"2024-03-06T07:25:19.799694+0000","last_unstale":"2024-03-06T07:25:23.176415+0000","last_undegraded":"2024-03-06T07:25:23.176415+0000","last_fullsized":"2024-03-06T07:25:23.176415+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:24.214078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176241+0000","last_change":"2024-03-06T07:25:20.802835+0000","last_active":"2024-03-06T07:25:23.176241+0000","last_peered":"2024-03-06T07:25:23.176241+0000","last_clean":"2024-03-06T07:25:23.176241+0000","last_became_active":"2024-03-06T07:25:20.802543+0000","last_became_peered":"2024-03-06T07:25:20.802543+0000","last_unstale":"2024-03-06T07:25:23.176241+0000","last_undegraded":"2024-03-06T07:25:23.176241+0000","last_fullsized":"2024-03-06T07:25:23.176241+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:51.978965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786411+0000","last_change":"2024-03-06T07:25:19.794850+0000","last_active":"2024-03-06T07:25:21.786411+0000","last_peered":"2024-03-06T07:25:21.786411+0000","last_clean":"2024-03-06T07:25:21.786411+0000","last_became_active":"2024-03-06T07:25:19.794587+0000","last_became_peered":"2024-03-06T07:25:19.794587+0000","last_unstale":"2024-03-06T07:25:21.786411+0000","last_undegraded":"2024-03-06T07:25:21.786411+0000","last_fullsized":"2024-03-06T07:25:21.786411+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:22.616655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787494+0000","last_change":"2024-03-06T07:25:20.792527+0000","last_active":"2024-03-06T07:25:21.787494+0000","last_peered":"2024-03-06T07:25:21.787494+0000","last_clean":"2024-03-06T07:25:21.787494+0000","last_became_active":"2024-03-06T07:25:20.792261+0000","last_became_peered":"2024-03-06T07:25:20.792261+0000","last_unstale":"2024-03-06T07:25:21.787494+0000","last_undegraded":"2024-03-06T07:25:21.787494+0000","last_fullsized":"2024-03-06T07:25:21.787494+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:21.259945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786564+0000","last_change":"2024-03-06T07:25:19.796329+0000","last_active":"2024-03-06T07:25:21.786564+0000","last_peered":"2024-03-06T07:25:21.786564+0000","last_clean":"2024-03-06T07:25:21.786564+0000","last_became_active":"2024-03-06T07:25:19.796081+0000","last_became_peered":"2024-03-06T07:25:19.796081+0000","last_unstale":"2024-03-06T07:25:21.786564+0000","last_undegraded":"2024-03-06T07:25:21.786564+0000","last_fullsized":"2024-03-06T07:25:21.786564+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:30.295129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786502+0000","last_change":"2024-03-06T07:25:20.796806+0000","last_active":"2024-03-06T07:25:21.786502+0000","last_peered":"2024-03-06T07:25:21.786502+0000","last_clean":"2024-03-06T07:25:21.786502+0000","last_became_active":"2024-03-06T07:25:20.796533+0000","last_became_peered":"2024-03-06T07:25:20.796533+0000","last_unstale":"2024-03-06T07:25:21.786502+0000","last_undegraded":"2024-03-06T07:25:21.786502+0000","last_fullsized":"2024-03-06T07:25:21.786502+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:16.264569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176288+0000","last_change":"2024-03-06T07:25:19.789450+0000","last_active":"2024-03-06T07:25:23.176288+0000","last_peered":"2024-03-06T07:25:23.176288+0000","last_clean":"2024-03-06T07:25:23.176288+0000","last_became_active":"2024-03-06T07:25:19.789139+0000","last_became_peered":"2024-03-06T07:25:19.789139+0000","last_unstale":"2024-03-06T07:25:23.176288+0000","last_undegraded":"2024-03-06T07:25:23.176288+0000","last_fullsized":"2024-03-06T07:25:23.176288+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:34.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134587+0000","last_change":"2024-03-06T07:25:20.803756+0000","last_active":"2024-03-06T07:25:22.134587+0000","last_peered":"2024-03-06T07:25:22.134587+0000","last_clean":"2024-03-06T07:25:22.134587+0000","last_became_active":"2024-03-06T07:25:20.803514+0000","last_became_peered":"2024-03-06T07:25:20.803514+0000","last_unstale":"2024-03-06T07:25:22.134587+0000","last_undegraded":"2024-03-06T07:25:22.134587+0000","last_fullsized":"2024-03-06T07:25:22.134587+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:46:39.265614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787682+0000","last_change":"2024-03-06T07:25:19.797666+0000","last_active":"2024-03-06T07:25:21.787682+0000","last_peered":"2024-03-06T07:25:21.787682+0000","last_clean":"2024-03-06T07:25:21.787682+0000","last_became_active":"2024-03-06T07:25:19.797391+0000","last_became_peered":"2024-03-06T07:25:19.797391+0000","last_unstale":"2024-03-06T07:25:21.787682+0000","last_undegraded":"2024-03-06T07:25:21.787682+0000","last_fullsized":"2024-03-06T07:25:21.787682+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:54.608998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175914+0000","last_change":"2024-03-06T07:25:20.797676+0000","last_active":"2024-03-06T07:25:23.175914+0000","last_peered":"2024-03-06T07:25:23.175914+0000","last_clean":"2024-03-06T07:25:23.175914+0000","last_became_active":"2024-03-06T07:25:20.797374+0000","last_became_peered":"2024-03-06T07:25:20.797374+0000","last_unstale":"2024-03-06T07:25:23.175914+0000","last_undegraded":"2024-03-06T07:25:23.175914+0000","last_fullsized":"2024-03-06T07:25:23.175914+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:35.871130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176969+0000","last_change":"2024-03-06T07:25:19.799648+0000","last_active":"2024-03-06T07:25:23.176969+0000","last_peered":"2024-03-06T07:25:23.176969+0000","last_clean":"2024-03-06T07:25:23.176969+0000","last_became_active":"2024-03-06T07:25:19.799352+0000","last_became_peered":"2024-03-06T07:25:19.799352+0000","last_unstale":"2024-03-06T07:25:23.176969+0000","last_undegraded":"2024-03-06T07:25:23.176969+0000","last_fullsized":"2024-03-06T07:25:23.176969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:18.223482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175782+0000","last_change":"2024-03-06T07:25:20.801510+0000","last_active":"2024-03-06T07:25:23.175782+0000","last_peered":"2024-03-06T07:25:23.175782+0000","last_clean":"2024-03-06T07:25:23.175782+0000","last_became_active":"2024-03-06T07:25:20.801227+0000","last_became_peered":"2024-03-06T07:25:20.801227+0000","last_unstale":"2024-03-06T07:25:23.175782+0000","last_undegraded":"2024-03-06T07:25:23.175782+0000","last_fullsized":"2024-03-06T07:25:23.175782+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:29.136905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786545+0000","last_change":"2024-03-06T07:25:19.797676+0000","last_active":"2024-03-06T07:25:22.786545+0000","last_peered":"2024-03-06T07:25:22.786545+0000","last_clean":"2024-03-06T07:25:22.786545+0000","last_became_active":"2024-03-06T07:25:19.797399+0000","last_became_peered":"2024-03-06T07:25:19.797399+0000","last_unstale":"2024-03-06T07:25:22.786545+0000","last_undegraded":"2024-03-06T07:25:22.786545+0000","last_fullsized":"2024-03-06T07:25:22.786545+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:26:54.998525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786593+0000","last_change":"2024-03-06T07:25:20.804750+0000","last_active":"2024-03-06T07:25:22.786593+0000","last_peered":"2024-03-06T07:25:22.786593+0000","last_clean":"2024-03-06T07:25:22.786593+0000","last_became_active":"2024-03-06T07:25:20.804379+0000","last_became_peered":"2024-03-06T07:25:20.804379+0000","last_unstale":"2024-03-06T07:25:22.786593+0000","last_undegraded":"2024-03-06T07:25:22.786593+0000","last_fullsized":"2024-03-06T07:25:22.786593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:00.402363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175985+0000","last_change":"2024-03-06T07:25:19.802156+0000","last_active":"2024-03-06T07:25:23.175985+0000","last_peered":"2024-03-06T07:25:23.175985+0000","last_clean":"2024-03-06T07:25:23.175985+0000","last_became_active":"2024-03-06T07:25:19.801881+0000","last_became_peered":"2024-03-06T07:25:19.801881+0000","last_unstale":"2024-03-06T07:25:23.175985+0000","last_undegraded":"2024-03-06T07:25:23.175985+0000","last_fullsized":"2024-03-06T07:25:23.175985+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:59.395378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787625+0000","last_change":"2024-03-06T07:25:20.793249+0000","last_active":"2024-03-06T07:25:21.787625+0000","last_peered":"2024-03-06T07:25:21.787625+0000","last_clean":"2024-03-06T07:25:21.787625+0000","last_became_active":"2024-03-06T07:25:20.792954+0000","last_became_peered":"2024-03-06T07:25:20.792954+0000","last_unstale":"2024-03-06T07:25:21.787625+0000","last_undegraded":"2024-03-06T07:25:21.787625+0000","last_fullsized":"2024-03-06T07:25:21.787625+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:17.297783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786479+0000","last_change":"2024-03-06T07:25:19.795560+0000","last_active":"2024-03-06T07:25:21.786479+0000","last_peered":"2024-03-06T07:25:21.786479+0000","last_clean":"2024-03-06T07:25:21.786479+0000","last_became_active":"2024-03-06T07:25:19.795293+0000","last_became_peered":"2024-03-06T07:25:19.795293+0000","last_unstale":"2024-03-06T07:25:21.786479+0000","last_undegraded":"2024-03-06T07:25:21.786479+0000","last_fullsized":"2024-03-06T07:25:21.786479+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:12.667706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176908+0000","last_change":"2024-03-06T07:25:20.795250+0000","last_active":"2024-03-06T07:25:23.176908+0000","last_peered":"2024-03-06T07:25:23.176908+0000","last_clean":"2024-03-06T07:25:23.176908+0000","last_became_active":"2024-03-06T07:25:20.794551+0000","last_became_peered":"2024-03-06T07:25:20.794551+0000","last_unstale":"2024-03-06T07:25:23.176908+0000","last_undegraded":"2024-03-06T07:25:23.176908+0000","last_fullsized":"2024-03-06T07:25:23.176908+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:38:19.157766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786986+0000","last_change":"2024-03-06T07:25:19.798578+0000","last_active":"2024-03-06T07:25:22.786986+0000","last_peered":"2024-03-06T07:25:22.786986+0000","last_clean":"2024-03-06T07:25:22.786986+0000","last_became_active":"2024-03-06T07:25:19.798263+0000","last_became_peered":"2024-03-06T07:25:19.798263+0000","last_unstale":"2024-03-06T07:25:22.786986+0000","last_undegraded":"2024-03-06T07:25:22.786986+0000","last_fullsized":"2024-03-06T07:25:22.786986+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:41.773697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177379+0000","last_change":"2024-03-06T07:25:20.797282+0000","last_active":"2024-03-06T07:25:23.177379+0000","last_peered":"2024-03-06T07:25:23.177379+0000","last_clean":"2024-03-06T07:25:23.177379+0000","last_became_active":"2024-03-06T07:25:20.796941+0000","last_became_peered":"2024-03-06T07:25:20.796941+0000","last_unstale":"2024-03-06T07:25:23.177379+0000","last_undegraded":"2024-03-06T07:25:23.177379+0000","last_fullsized":"2024-03-06T07:25:23.177379+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:08.725602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"134'3","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177168+0000","last_change":"2024-03-06T07:25:19.792711+0000","last_active":"2024-03-06T07:25:23.177168+0000","last_peered":"2024-03-06T07:25:23.177168+0000","last_clean":"2024-03-06T07:25:23.177168+0000","last_became_active":"2024-03-06T07:25:19.792405+0000","last_became_peered":"2024-03-06T07:25:19.792405+0000","last_unstale":"2024-03-06T07:25:23.177168+0000","last_undegraded":"2024-03-06T07:25:23.177168+0000","last_fullsized":"2024-03-06T07:25:23.177168+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:03:06.439333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787297+0000","last_change":"2024-03-06T07:25:20.793655+0000","last_active":"2024-03-06T07:25:21.787297+0000","last_peered":"2024-03-06T07:25:21.787297+0000","last_clean":"2024-03-06T07:25:21.787297+0000","last_became_active":"2024-03-06T07:25:20.793109+0000","last_became_peered":"2024-03-06T07:25:20.793109+0000","last_unstale":"2024-03-06T07:25:21.787297+0000","last_undegraded":"2024-03-06T07:25:21.787297+0000","last_fullsized":"2024-03-06T07:25:21.787297+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:59.537190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176719+0000","last_change":"2024-03-06T07:25:19.800658+0000","last_active":"2024-03-06T07:25:23.176719+0000","last_peered":"2024-03-06T07:25:23.176719+0000","last_clean":"2024-03-06T07:25:23.176719+0000","last_became_active":"2024-03-06T07:25:19.800361+0000","last_became_peered":"2024-03-06T07:25:19.800361+0000","last_unstale":"2024-03-06T07:25:23.176719+0000","last_undegraded":"2024-03-06T07:25:23.176719+0000","last_fullsized":"2024-03-06T07:25:23.176719+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:22.616655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175681+0000","last_change":"2024-03-06T07:25:20.805290+0000","last_active":"2024-03-06T07:25:23.175681+0000","last_peered":"2024-03-06T07:25:23.175681+0000","last_clean":"2024-03-06T07:25:23.175681+0000","last_became_active":"2024-03-06T07:25:20.802094+0000","last_became_peered":"2024-03-06T07:25:20.802094+0000","last_unstale":"2024-03-06T07:25:23.175681+0000","last_undegraded":"2024-03-06T07:25:23.175681+0000","last_fullsized":"2024-03-06T07:25:23.175681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:06.346122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175860+0000","last_change":"2024-03-06T07:25:19.802236+0000","last_active":"2024-03-06T07:25:23.175860+0000","last_peered":"2024-03-06T07:25:23.175860+0000","last_clean":"2024-03-06T07:25:23.175860+0000","last_became_active":"2024-03-06T07:25:19.801866+0000","last_became_peered":"2024-03-06T07:25:19.801866+0000","last_unstale":"2024-03-06T07:25:23.175860+0000","last_undegraded":"2024-03-06T07:25:23.175860+0000","last_fullsized":"2024-03-06T07:25:23.175860+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:08.859617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176883+0000","last_change":"2024-03-06T07:25:20.798080+0000","last_active":"2024-03-06T07:25:23.176883+0000","last_peered":"2024-03-06T07:25:23.176883+0000","last_clean":"2024-03-06T07:25:23.176883+0000","last_became_active":"2024-03-06T07:25:20.797792+0000","last_became_peered":"2024-03-06T07:25:20.797792+0000","last_unstale":"2024-03-06T07:25:23.176883+0000","last_undegraded":"2024-03-06T07:25:23.176883+0000","last_fullsized":"2024-03-06T07:25:23.176883+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:48.629438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176048+0000","last_change":"2024-03-06T07:25:19.800076+0000","last_active":"2024-03-06T07:25:23.176048+0000","last_peered":"2024-03-06T07:25:23.176048+0000","last_clean":"2024-03-06T07:25:23.176048+0000","last_became_active":"2024-03-06T07:25:19.799501+0000","last_became_peered":"2024-03-06T07:25:19.799501+0000","last_unstale":"2024-03-06T07:25:23.176048+0000","last_undegraded":"2024-03-06T07:25:23.176048+0000","last_fullsized":"2024-03-06T07:25:23.176048+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:27.059865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175933+0000","last_change":"2024-03-06T07:25:20.811709+0000","last_active":"2024-03-06T07:25:23.175933+0000","last_peered":"2024-03-06T07:25:23.175933+0000","last_clean":"2024-03-06T07:25:23.175933+0000","last_became_active":"2024-03-06T07:25:20.811403+0000","last_became_peered":"2024-03-06T07:25:20.811403+0000","last_unstale":"2024-03-06T07:25:23.175933+0000","last_undegraded":"2024-03-06T07:25:23.175933+0000","last_fullsized":"2024-03-06T07:25:23.175933+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:15.329851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"134'8","reported_seq":22,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786402+0000","last_change":"2024-03-06T07:25:19.797067+0000","last_active":"2024-03-06T07:25:22.786402+0000","last_peered":"2024-03-06T07:25:22.786402+0000","last_clean":"2024-03-06T07:25:22.786402+0000","last_became_active":"2024-03-06T07:25:19.796773+0000","last_became_peered":"2024-03-06T07:25:19.796773+0000","last_unstale":"2024-03-06T07:25:22.786402+0000","last_undegraded":"2024-03-06T07:25:22.786402+0000","last_fullsized":"2024-03-06T07:25:22.786402+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:16.290798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786133+0000","last_change":"2024-03-06T07:25:20.808364+0000","last_active":"2024-03-06T07:25:21.786133+0000","last_peered":"2024-03-06T07:25:21.786133+0000","last_clean":"2024-03-06T07:25:21.786133+0000","last_became_active":"2024-03-06T07:25:20.808035+0000","last_became_peered":"2024-03-06T07:25:20.808035+0000","last_unstale":"2024-03-06T07:25:21.786133+0000","last_undegraded":"2024-03-06T07:25:21.786133+0000","last_fullsized":"2024-03-06T07:25:21.786133+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:30.948895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177243+0000","last_change":"2024-03-06T07:25:19.789597+0000","last_active":"2024-03-06T07:25:23.177243+0000","last_peered":"2024-03-06T07:25:23.177243+0000","last_clean":"2024-03-06T07:25:23.177243+0000","last_became_active":"2024-03-06T07:25:19.789334+0000","last_became_peered":"2024-03-06T07:25:19.789334+0000","last_unstale":"2024-03-06T07:25:23.177243+0000","last_undegraded":"2024-03-06T07:25:23.177243+0000","last_fullsized":"2024-03-06T07:25:23.177243+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:43.183726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786457+0000","last_change":"2024-03-06T07:25:20.797381+0000","last_active":"2024-03-06T07:25:21.786457+0000","last_peered":"2024-03-06T07:25:21.786457+0000","last_clean":"2024-03-06T07:25:21.786457+0000","last_became_active":"2024-03-06T07:25:20.797071+0000","last_became_peered":"2024-03-06T07:25:20.797071+0000","last_unstale":"2024-03-06T07:25:21.786457+0000","last_undegraded":"2024-03-06T07:25:21.786457+0000","last_fullsized":"2024-03-06T07:25:21.786457+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:08.382103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786533+0000","last_change":"2024-03-06T07:25:19.796190+0000","last_active":"2024-03-06T07:25:21.786533+0000","last_peered":"2024-03-06T07:25:21.786533+0000","last_clean":"2024-03-06T07:25:21.786533+0000","last_became_active":"2024-03-06T07:25:19.795879+0000","last_became_peered":"2024-03-06T07:25:19.795879+0000","last_unstale":"2024-03-06T07:25:21.786533+0000","last_undegraded":"2024-03-06T07:25:21.786533+0000","last_fullsized":"2024-03-06T07:25:21.786533+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:24.214078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786819+0000","last_change":"2024-03-06T07:25:20.807614+0000","last_active":"2024-03-06T07:25:22.786819+0000","last_peered":"2024-03-06T07:25:22.786819+0000","last_clean":"2024-03-06T07:25:22.786819+0000","last_became_active":"2024-03-06T07:25:20.807350+0000","last_became_peered":"2024-03-06T07:25:20.807350+0000","last_unstale":"2024-03-06T07:25:22.786819+0000","last_undegraded":"2024-03-06T07:25:22.786819+0000","last_fullsized":"2024-03-06T07:25:22.786819+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:25.221063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"134'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176322+0000","last_change":"2024-03-06T07:25:19.798347+0000","last_active":"2024-03-06T07:25:23.176322+0000","last_peered":"2024-03-06T07:25:23.176322+0000","last_clean":"2024-03-06T07:25:23.176322+0000","last_became_active":"2024-03-06T07:25:19.798021+0000","last_became_peered":"2024-03-06T07:25:19.798021+0000","last_unstale":"2024-03-06T07:25:23.176322+0000","last_undegraded":"2024-03-06T07:25:23.176322+0000","last_fullsized":"2024-03-06T07:25:23.176322+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:12:26.870856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787120+0000","last_change":"2024-03-06T07:25:20.795987+0000","last_active":"2024-03-06T07:25:22.787120+0000","last_peered":"2024-03-06T07:25:22.787120+0000","last_clean":"2024-03-06T07:25:22.787120+0000","last_became_active":"2024-03-06T07:25:20.795728+0000","last_became_peered":"2024-03-06T07:25:20.795728+0000","last_unstale":"2024-03-06T07:25:22.787120+0000","last_undegraded":"2024-03-06T07:25:22.787120+0000","last_fullsized":"2024-03-06T07:25:22.787120+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:15:43.836498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176067+0000","last_change":"2024-03-06T07:25:19.801586+0000","last_active":"2024-03-06T07:25:23.176067+0000","last_peered":"2024-03-06T07:25:23.176067+0000","last_clean":"2024-03-06T07:25:23.176067+0000","last_became_active":"2024-03-06T07:25:19.801242+0000","last_became_peered":"2024-03-06T07:25:19.801242+0000","last_unstale":"2024-03-06T07:25:23.176067+0000","last_undegraded":"2024-03-06T07:25:23.176067+0000","last_fullsized":"2024-03-06T07:25:23.176067+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:43.584796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.785920+0000","last_change":"2024-03-06T07:25:20.797641+0000","last_active":"2024-03-06T07:25:21.785920+0000","last_peered":"2024-03-06T07:25:21.785920+0000","last_clean":"2024-03-06T07:25:21.785920+0000","last_became_active":"2024-03-06T07:25:20.797376+0000","last_became_peered":"2024-03-06T07:25:20.797376+0000","last_unstale":"2024-03-06T07:25:21.785920+0000","last_undegraded":"2024-03-06T07:25:21.785920+0000","last_fullsized":"2024-03-06T07:25:21.785920+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:10.857381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134609+0000","last_change":"2024-03-06T07:25:19.796315+0000","last_active":"2024-03-06T07:25:22.134609+0000","last_peered":"2024-03-06T07:25:22.134609+0000","last_clean":"2024-03-06T07:25:22.134609+0000","last_became_active":"2024-03-06T07:25:19.796036+0000","last_became_peered":"2024-03-06T07:25:19.796036+0000","last_unstale":"2024-03-06T07:25:22.134609+0000","last_undegraded":"2024-03-06T07:25:22.134609+0000","last_fullsized":"2024-03-06T07:25:22.134609+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:51.155586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175823+0000","last_change":"2024-03-06T07:25:20.797245+0000","last_active":"2024-03-06T07:25:23.175823+0000","last_peered":"2024-03-06T07:25:23.175823+0000","last_clean":"2024-03-06T07:25:23.175823+0000","last_became_active":"2024-03-06T07:25:20.796952+0000","last_became_peered":"2024-03-06T07:25:20.796952+0000","last_unstale":"2024-03-06T07:25:23.175823+0000","last_undegraded":"2024-03-06T07:25:23.175823+0000","last_fullsized":"2024-03-06T07:25:23.175823+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:13.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787220+0000","last_change":"2024-03-06T07:25:19.789672+0000","last_active":"2024-03-06T07:25:21.787220+0000","last_peered":"2024-03-06T07:25:21.787220+0000","last_clean":"2024-03-06T07:25:21.787220+0000","last_became_active":"2024-03-06T07:25:19.789409+0000","last_became_peered":"2024-03-06T07:25:19.789409+0000","last_unstale":"2024-03-06T07:25:21.787220+0000","last_undegraded":"2024-03-06T07:25:21.787220+0000","last_fullsized":"2024-03-06T07:25:21.787220+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:56.743191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786068+0000","last_change":"2024-03-06T07:25:20.799585+0000","last_active":"2024-03-06T07:25:21.786068+0000","last_peered":"2024-03-06T07:25:21.786068+0000","last_clean":"2024-03-06T07:25:21.786068+0000","last_became_active":"2024-03-06T07:25:20.799321+0000","last_became_peered":"2024-03-06T07:25:20.799321+0000","last_unstale":"2024-03-06T07:25:21.786068+0000","last_undegraded":"2024-03-06T07:25:21.786068+0000","last_fullsized":"2024-03-06T07:25:21.786068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:28:53.428162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134476+0000","last_change":"2024-03-06T07:25:19.797588+0000","last_active":"2024-03-06T07:25:22.134476+0000","last_peered":"2024-03-06T07:25:22.134476+0000","last_clean":"2024-03-06T07:25:22.134476+0000","last_became_active":"2024-03-06T07:25:19.797322+0000","last_became_peered":"2024-03-06T07:25:19.797322+0000","last_unstale":"2024-03-06T07:25:22.134476+0000","last_undegraded":"2024-03-06T07:25:22.134476+0000","last_fullsized":"2024-03-06T07:25:22.134476+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:41:38.265186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176177+0000","last_change":"2024-03-06T07:25:20.801382+0000","last_active":"2024-03-06T07:25:23.176177+0000","last_peered":"2024-03-06T07:25:23.176177+0000","last_clean":"2024-03-06T07:25:23.176177+0000","last_became_active":"2024-03-06T07:25:20.801111+0000","last_became_peered":"2024-03-06T07:25:20.801111+0000","last_unstale":"2024-03-06T07:25:23.176177+0000","last_undegraded":"2024-03-06T07:25:23.176177+0000","last_fullsized":"2024-03-06T07:25:23.176177+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:19.230467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787764+0000","last_change":"2024-03-06T07:25:19.794012+0000","last_active":"2024-03-06T07:25:21.787764+0000","last_peered":"2024-03-06T07:25:21.787764+0000","last_clean":"2024-03-06T07:25:21.787764+0000","last_became_active":"2024-03-06T07:25:19.793743+0000","last_became_peered":"2024-03-06T07:25:19.793743+0000","last_unstale":"2024-03-06T07:25:21.787764+0000","last_undegraded":"2024-03-06T07:25:21.787764+0000","last_fullsized":"2024-03-06T07:25:21.787764+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:39.912413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176091+0000","last_change":"2024-03-06T07:25:20.804519+0000","last_active":"2024-03-06T07:25:23.176091+0000","last_peered":"2024-03-06T07:25:23.176091+0000","last_clean":"2024-03-06T07:25:23.176091+0000","last_became_active":"2024-03-06T07:25:20.804262+0000","last_became_peered":"2024-03-06T07:25:20.804262+0000","last_unstale":"2024-03-06T07:25:23.176091+0000","last_undegraded":"2024-03-06T07:25:23.176091+0000","last_fullsized":"2024-03-06T07:25:23.176091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:41:39.272171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"134'2","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176327+0000","last_change":"2024-03-06T07:25:19.801518+0000","last_active":"2024-03-06T07:25:23.176327+0000","last_peered":"2024-03-06T07:25:23.176327+0000","last_clean":"2024-03-06T07:25:23.176327+0000","last_became_active":"2024-03-06T07:25:19.801213+0000","last_became_peered":"2024-03-06T07:25:19.801213+0000","last_unstale":"2024-03-06T07:25:23.176327+0000","last_undegraded":"2024-03-06T07:25:23.176327+0000","last_fullsized":"2024-03-06T07:25:23.176327+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:51.514593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176026+0000","last_change":"2024-03-06T07:25:20.811157+0000","last_active":"2024-03-06T07:25:23.176026+0000","last_peered":"2024-03-06T07:25:23.176026+0000","last_clean":"2024-03-06T07:25:23.176026+0000","last_became_active":"2024-03-06T07:25:20.810825+0000","last_became_peered":"2024-03-06T07:25:20.810825+0000","last_unstale":"2024-03-06T07:25:23.176026+0000","last_undegraded":"2024-03-06T07:25:23.176026+0000","last_fullsized":"2024-03-06T07:25:23.176026+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:42.609468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"135'1","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787090+0000","last_change":"2024-03-06T07:25:19.793750+0000","last_active":"2024-03-06T07:25:22.787090+0000","last_peered":"2024-03-06T07:25:22.787090+0000","last_clean":"2024-03-06T07:25:22.787090+0000","last_became_active":"2024-03-06T07:25:19.793481+0000","last_became_peered":"2024-03-06T07:25:19.793481+0000","last_unstale":"2024-03-06T07:25:22.787090+0000","last_undegraded":"2024-03-06T07:25:22.787090+0000","last_fullsized":"2024-03-06T07:25:22.787090+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:50.049774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.787026+0000","last_change":"2024-03-06T07:25:20.804879+0000","last_active":"2024-03-06T07:25:22.787026+0000","last_peered":"2024-03-06T07:25:22.787026+0000","last_clean":"2024-03-06T07:25:22.787026+0000","last_became_active":"2024-03-06T07:25:20.804423+0000","last_became_peered":"2024-03-06T07:25:20.804423+0000","last_unstale":"2024-03-06T07:25:22.787026+0000","last_undegraded":"2024-03-06T07:25:22.787026+0000","last_fullsized":"2024-03-06T07:25:22.787026+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:13.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176406+0000","last_change":"2024-03-06T07:25:19.793729+0000","last_active":"2024-03-06T07:25:23.176406+0000","last_peered":"2024-03-06T07:25:23.176406+0000","last_clean":"2024-03-06T07:25:23.176406+0000","last_became_active":"2024-03-06T07:25:19.793432+0000","last_became_peered":"2024-03-06T07:25:19.793432+0000","last_unstale":"2024-03-06T07:25:23.176406+0000","last_undegraded":"2024-03-06T07:25:23.176406+0000","last_fullsized":"2024-03-06T07:25:23.176406+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:51.204621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176328+0000","last_change":"2024-03-06T07:25:20.803635+0000","last_active":"2024-03-06T07:25:23.176328+0000","last_peered":"2024-03-06T07:25:23.176328+0000","last_clean":"2024-03-06T07:25:23.176328+0000","last_became_active":"2024-03-06T07:25:20.803382+0000","last_became_peered":"2024-03-06T07:25:20.803382+0000","last_unstale":"2024-03-06T07:25:23.176328+0000","last_undegraded":"2024-03-06T07:25:23.176328+0000","last_fullsized":"2024-03-06T07:25:23.176328+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:51.048107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.176180+0000","last_change":"2024-03-06T07:25:19.793893+0000","last_active":"2024-03-06T07:25:23.176180+0000","last_peered":"2024-03-06T07:25:23.176180+0000","last_clean":"2024-03-06T07:25:23.176180+0000","last_became_active":"2024-03-06T07:25:19.793604+0000","last_became_peered":"2024-03-06T07:25:19.793604+0000","last_unstale":"2024-03-06T07:25:23.176180+0000","last_undegraded":"2024-03-06T07:25:23.176180+0000","last_fullsized":"2024-03-06T07:25:23.176180+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:50.971979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787703+0000","last_change":"2024-03-06T07:25:20.803289+0000","last_active":"2024-03-06T07:25:21.787703+0000","last_peered":"2024-03-06T07:25:21.787703+0000","last_clean":"2024-03-06T07:25:21.787703+0000","last_became_active":"2024-03-06T07:25:20.803019+0000","last_became_peered":"2024-03-06T07:25:20.803019+0000","last_unstale":"2024-03-06T07:25:21.787703+0000","last_undegraded":"2024-03-06T07:25:21.787703+0000","last_fullsized":"2024-03-06T07:25:21.787703+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:09.703019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786722+0000","last_change":"2024-03-06T07:25:19.797615+0000","last_active":"2024-03-06T07:25:22.786722+0000","last_peered":"2024-03-06T07:25:22.786722+0000","last_clean":"2024-03-06T07:25:22.786722+0000","last_became_active":"2024-03-06T07:25:19.797345+0000","last_became_peered":"2024-03-06T07:25:19.797345+0000","last_unstale":"2024-03-06T07:25:22.786722+0000","last_undegraded":"2024-03-06T07:25:22.786722+0000","last_fullsized":"2024-03-06T07:25:22.786722+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:08.696034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175873+0000","last_change":"2024-03-06T07:25:20.810575+0000","last_active":"2024-03-06T07:25:23.175873+0000","last_peered":"2024-03-06T07:25:23.175873+0000","last_clean":"2024-03-06T07:25:23.175873+0000","last_became_active":"2024-03-06T07:25:20.810317+0000","last_became_peered":"2024-03-06T07:25:20.810317+0000","last_unstale":"2024-03-06T07:25:23.175873+0000","last_undegraded":"2024-03-06T07:25:23.175873+0000","last_fullsized":"2024-03-06T07:25:23.175873+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:09.151335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.177125+0000","last_change":"2024-03-06T07:25:19.791673+0000","last_active":"2024-03-06T07:25:23.177125+0000","last_peered":"2024-03-06T07:25:23.177125+0000","last_clean":"2024-03-06T07:25:23.177125+0000","last_became_active":"2024-03-06T07:25:19.791395+0000","last_became_peered":"2024-03-06T07:25:19.791395+0000","last_unstale":"2024-03-06T07:25:23.177125+0000","last_undegraded":"2024-03-06T07:25:23.177125+0000","last_fullsized":"2024-03-06T07:25:23.177125+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:53.894137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:23.175932+0000","last_change":"2024-03-06T07:25:20.802114+0000","last_active":"2024-03-06T07:25:23.175932+0000","last_peered":"2024-03-06T07:25:23.175932+0000","last_clean":"2024-03-06T07:25:23.175932+0000","last_became_active":"2024-03-06T07:25:20.801842+0000","last_became_peered":"2024-03-06T07:25:20.801842+0000","last_unstale":"2024-03-06T07:25:23.175932+0000","last_undegraded":"2024-03-06T07:25:23.175932+0000","last_fullsized":"2024-03-06T07:25:23.175932+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:31.302114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.786719+0000","last_change":"2024-03-06T07:25:19.794392+0000","last_active":"2024-03-06T07:25:22.786719+0000","last_peered":"2024-03-06T07:25:22.786719+0000","last_clean":"2024-03-06T07:25:22.786719+0000","last_became_active":"2024-03-06T07:25:19.794116+0000","last_became_peered":"2024-03-06T07:25:19.794116+0000","last_unstale":"2024-03-06T07:25:22.786719+0000","last_undegraded":"2024-03-06T07:25:22.786719+0000","last_fullsized":"2024-03-06T07:25:22.786719+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:41.602483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787331+0000","last_change":"2024-03-06T07:25:20.802319+0000","last_active":"2024-03-06T07:25:21.787331+0000","last_peered":"2024-03-06T07:25:21.787331+0000","last_clean":"2024-03-06T07:25:21.787331+0000","last_became_active":"2024-03-06T07:25:20.799478+0000","last_became_peered":"2024-03-06T07:25:20.799478+0000","last_unstale":"2024-03-06T07:25:21.787331+0000","last_undegraded":"2024-03-06T07:25:21.787331+0000","last_fullsized":"2024-03-06T07:25:21.787331+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:52.211606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.785980+0000","last_change":"2024-03-06T07:25:19.793614+0000","last_active":"2024-03-06T07:25:21.785980+0000","last_peered":"2024-03-06T07:25:21.785980+0000","last_clean":"2024-03-06T07:25:21.785980+0000","last_became_active":"2024-03-06T07:25:19.793338+0000","last_became_peered":"2024-03-06T07:25:19.793338+0000","last_unstale":"2024-03-06T07:25:21.785980+0000","last_undegraded":"2024-03-06T07:25:21.785980+0000","last_fullsized":"2024-03-06T07:25:21.785980+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:50.971979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.787553+0000","last_change":"2024-03-06T07:25:20.790396+0000","last_active":"2024-03-06T07:25:21.787553+0000","last_peered":"2024-03-06T07:25:21.787553+0000","last_clean":"2024-03-06T07:25:21.787553+0000","last_became_active":"2024-03-06T07:25:20.790134+0000","last_became_peered":"2024-03-06T07:25:20.790134+0000","last_unstale":"2024-03-06T07:25:21.787553+0000","last_undegraded":"2024-03-06T07:25:21.787553+0000","last_fullsized":"2024-03-06T07:25:21.787553+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:19.773061+0000","last_clean_scrub_stamp":"2024-03-06T07:25:19.773061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:35.871130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"134'6","reported_seq":20,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:22.134715+0000","last_change":"2024-03-06T07:25:19.790987+0000","last_active":"2024-03-06T07:25:22.134715+0000","last_peered":"2024-03-06T07:25:22.134715+0000","last_clean":"2024-03-06T07:25:22.134715+0000","last_became_active":"2024-03-06T07:25:19.790677+0000","last_became_peered":"2024-03-06T07:25:19.790677+0000","last_unstale":"2024-03-06T07:25:22.134715+0000","last_undegraded":"2024-03-06T07:25:22.134715+0000","last_fullsized":"2024-03-06T07:25:22.134715+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:18.766076+0000","last_clean_scrub_stamp":"2024-03-06T07:25:18.766076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:24.880046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":386,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-03-06T07:25:21.786036+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:25:21.786036+0000","last_peered":"2024-03-06T07:25:21.786036+0000","last_clean":"2024-03-06T07:25:21.786036+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:25:21.786036+0000","last_undegraded":"2024-03-06T07:25:21.786036+0000","last_fullsized":"2024-03-06T07:25:21.786036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":1408266,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":817946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1695744,"data_stored":817876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":819488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":817988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":817946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574916,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32924,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338916,"statfs":{"total":96636764160,"available":96603049984,"internally_reserved":0,"allocated":1785856,"data_stored":1410562,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:28.561 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:25:28.561 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:25:28.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.706+0000 7f7e49b5e640 1 Processor -- start 2024-03-06T07:25:28.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.706+0000 7f7e49b5e640 1 -- start start 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.706+0000 7f7e49b5e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e44152770 0x7f7e440c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e49b5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e44155ed0 con 0x7f7e440c7710 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e49b5e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e440c6c30 con 0x7f7e440c12a0 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e49b5e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e440c6a00 con 0x7f7e44152770 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e43fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7e440c7710 0x7f7e440c7b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56336/0 (socket says 172.21.15.26:56336) 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e43fff640 1 -- 172.21.15.26:0/3342996709 learned_addr learned my addr 172.21.15.26:0/3342996709 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e437fe640 1 --2- 172.21.15.26:0/3342996709 >> v2:172.21.15.117:3300/0 conn(0x7f7e44152770 0x7f7e440c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e427fc640 1 -- 172.21.15.26:0/3342996709 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20429260 0 0) 0x7f7e44155ed0 con 0x7f7e440c7710 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e427fc640 1 -- 172.21.15.26:0/3342996709 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e2c003520 con 0x7f7e440c7710 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e437fe640 1 -- 172.21.15.26:0/3342996709 >> v1:172.21.15.117:6789/0 conn(0x7f7e440c12a0 legacy=0x7f7e4406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e437fe640 1 -- 172.21.15.26:0/3342996709 >> v1:172.21.15.26:6789/0 conn(0x7f7e440c7710 legacy=0x7f7e440c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.707+0000 7f7e437fe640 1 -- 172.21.15.26:0/3342996709 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e440c6f10 con 0x7f7e44152770 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.708+0000 7f7e437fe640 1 --2- 172.21.15.26:0/3342996709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e44152770 0x7f7e440c1990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7e340097a0 tx=0x7f7e3402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be39d9bf1728c02c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.708+0000 7f7e427fc640 1 -- 172.21.15.26:0/3342996709 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34034070 con 0x7f7e44152770 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.708+0000 7f7e427fc640 1 -- 172.21.15.26:0/3342996709 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e34030610 con 0x7f7e44152770 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.708+0000 7f7e427fc640 1 -- 172.21.15.26:0/3342996709 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e340309c0 con 0x7f7e44152770 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.709+0000 7f7e49b5e640 1 -- 172.21.15.26:0/3342996709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e44152770 msgr2=0x7f7e440c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.709+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/3342996709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e44152770 0x7f7e440c1990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7e340097a0 tx=0x7f7e3402b960 comp rx=0 tx=0).stop 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.709+0000 7f7e49b5e640 1 -- 172.21.15.26:0/3342996709 shutdown_connections 2024-03-06T07:25:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.709+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/3342996709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e44152770 0x7f7e440c1990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.709+0000 7f7e49b5e640 1 -- 172.21.15.26:0/3342996709 >> 172.21.15.26:0/3342996709 conn(0x7f7e440a94d0 msgr2=0x7f7e440a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.709+0000 7f7e49b5e640 1 -- 172.21.15.26:0/3342996709 shutdown_connections 2024-03-06T07:25:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.709+0000 7f7e49b5e640 1 -- 172.21.15.26:0/3342996709 wait complete. 2024-03-06T07:25:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.710+0000 7f7e49b5e640 1 Processor -- start 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.710+0000 7f7e49b5e640 1 -- start start 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e49b5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e440c12a0 0x7f7e44158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e49b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e440c7710 0x7f7e44159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e49b5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e44152770 0x7f7e4414ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e49b5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44155ba0 con 0x7f7e44152770 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e49b5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44152b50 con 0x7f7e440c7710 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e49b5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4406b6d0 con 0x7f7e440c12a0 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e44152770 0x7f7e4414ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e44152770 0x7f7e4414ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50072/0 (socket says 172.21.15.26:50072) 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 -- 172.21.15.26:0/668287424 learned_addr learned my addr 172.21.15.26:0/668287424 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e42ffd640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e440c7710 0x7f7e44159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e437fe640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e440c12a0 0x7f7e44158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 -- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e440c12a0 msgr2=0x7f7e44158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e440c12a0 0x7f7e44158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 -- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e440c7710 msgr2=0x7f7e44159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e440c7710 0x7f7e44159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 -- 172.21.15.26:0/668287424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44154cb0 con 0x7f7e44152770 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e43fff640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e44152770 0x7f7e4414ac20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00cb40 tx=0x7f7e3c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e2bfff640 1 -- 172.21.15.26:0/668287424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c017070 con 0x7f7e44152770 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e2bfff640 1 -- 172.21.15.26:0/668287424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c00aca0 con 0x7f7e44152770 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4414b900 con 0x7f7e44152770 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.711+0000 7f7e2bfff640 1 -- 172.21.15.26:0/668287424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c012620 con 0x7f7e44152770 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.712+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4414c000 con 0x7f7e44152770 2024-03-06T07:25:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.712+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4404f330 con 0x7f7e44152770 2024-03-06T07:25:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.713+0000 7f7e2bfff640 1 -- 172.21.15.26:0/668287424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e3c01a070 con 0x7f7e44152770 2024-03-06T07:25:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.713+0000 7f7e2bfff640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e3006b500 0x7f7e3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.714+0000 7f7e2bfff640 1 -- 172.21.15.26:0/668287424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e3c08a120 con 0x7f7e44152770 2024-03-06T07:25:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.714+0000 7f7e437fe640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e3006b500 0x7f7e3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.714+0000 7f7e437fe640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e3006b500 0x7f7e3006d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7e34036000 tx=0x7f7e34008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:28.718+0000 7f7e2bfff640 1 -- 172.21.15.26:0/668287424 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7e3c0587d0 con 0x7f7e44152770 2024-03-06T07:25:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.006+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7e441eac80 con 0x7f7e44152770 2024-03-06T07:25:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.008+0000 7f7e2bfff640 1 -- 172.21.15.26:0/668287424 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7e3c05c870 con 0x7f7e44152770 2024-03-06T07:25:29.010 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:25:29.010 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:25:29.010 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:25:29.010 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:25:29.010 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:25:29.010 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:25:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e3006b500 msgr2=0x7f7e3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e3006b500 0x7f7e3006d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7e34036000 tx=0x7f7e34008030 comp rx=0 tx=0).stop 2024-03-06T07:25:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e44152770 msgr2=0x7f7e4414ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e44152770 0x7f7e4414ac20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00cb40 tx=0x7f7e3c00a060 comp rx=0 tx=0).stop 2024-03-06T07:25:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 shutdown_connections 2024-03-06T07:25:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e3006b500 0x7f7e3006d9c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e44152770 0x7f7e4414ac20 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e440c7710 0x7f7e44159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 --2- 172.21.15.26:0/668287424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e440c12a0 0x7f7e44158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 >> 172.21.15.26:0/668287424 conn(0x7f7e440a94d0 msgr2=0x7f7e440a76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 shutdown_connections 2024-03-06T07:25:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.010+0000 7f7e49b5e640 1 -- 172.21.15.26:0/668287424 wait complete. 2024-03-06T07:25:29.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:29.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.123+0000 7efca886f640 1 Processor -- start 2024-03-06T07:25:29.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.123+0000 7efca886f640 1 -- start start 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca886f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca40c5480 0x7efca4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca886f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efca41586f0 0x7efca4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca886f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca40c5cc0 con 0x7efca40c5860 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca886f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca406df40 con 0x7efca41586f0 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca886f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca406fdb0 con 0x7efca40c5480 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efca41586f0 0x7efca4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efca41586f0 0x7efca4158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50080/0 (socket says 172.21.15.26:50080) 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca2d9d640 1 -- 172.21.15.26:0/3952795071 learned_addr learned my addr 172.21.15.26:0/3952795071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca259c640 1 --2- 172.21.15.26:0/3952795071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca40c5480 0x7efca4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca2d9d640 1 -- 172.21.15.26:0/3952795071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca40c5480 msgr2=0x7efca4155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca2d9d640 1 --2- 172.21.15.26:0/3952795071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca40c5480 0x7efca4155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca2d9d640 1 -- 172.21.15.26:0/3952795071 >> v1:172.21.15.117:6789/0 conn(0x7efca40c5860 legacy=0x7efca406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca2d9d640 1 -- 172.21.15.26:0/3952795071 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca41591c0 con 0x7efca41586f0 2024-03-06T07:25:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.124+0000 7efca259c640 1 --2- 172.21.15.26:0/3952795071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca40c5480 0x7efca4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.125+0000 7efca2d9d640 1 --2- 172.21.15.26:0/3952795071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca41586f0 0x7efca4158b80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e50 tx=0x7efc9c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76515ba055695f81 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.125+0000 7efca1d9b640 1 -- 172.21.15.26:0/3952795071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c02c9e0 con 0x7efca41586f0 2024-03-06T07:25:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.125+0000 7efca1d9b640 1 -- 172.21.15.26:0/3952795071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c02cb80 con 0x7efca41586f0 2024-03-06T07:25:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.125+0000 7efca1d9b640 1 -- 172.21.15.26:0/3952795071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0343f0 con 0x7efca41586f0 2024-03-06T07:25:29.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 -- 172.21.15.26:0/3952795071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca41586f0 msgr2=0x7efca4158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 --2- 172.21.15.26:0/3952795071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca41586f0 0x7efca4158b80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e50 tx=0x7efc9c02be40 comp rx=0 tx=0).stop 2024-03-06T07:25:29.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 -- 172.21.15.26:0/3952795071 shutdown_connections 2024-03-06T07:25:29.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 --2- 172.21.15.26:0/3952795071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca41586f0 0x7efca4158b80 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 --2- 172.21.15.26:0/3952795071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca40c5480 0x7efca4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 -- 172.21.15.26:0/3952795071 >> 172.21.15.26:0/3952795071 conn(0x7efca406b1a0 msgr2=0x7efca406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 -- 172.21.15.26:0/3952795071 shutdown_connections 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.126+0000 7efca886f640 1 -- 172.21.15.26:0/3952795071 wait complete. 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.127+0000 7efca886f640 1 Processor -- start 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.127+0000 7efca886f640 1 -- start start 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca886f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca40c5480 0x7efca41ee940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca886f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca40c5860 0x7efca40c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca886f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca41586f0 0x7efca41ea980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca886f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4155680 con 0x7efca40c5480 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca886f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41553a0 con 0x7efca40c5860 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca886f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca4152a20 con 0x7efca41586f0 2024-03-06T07:25:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca40c5480 0x7efca41ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca40c5480 0x7efca41ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50084/0 (socket says 172.21.15.26:50084) 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 -- 172.21.15.26:0/4042070382 learned_addr learned my addr 172.21.15.26:0/4042070382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efc9bfff640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca40c5860 0x7efca40c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca2d9d640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca41586f0 0x7efca41ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 -- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca41586f0 msgr2=0x7efca41ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca41586f0 0x7efca41ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 -- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca40c5860 msgr2=0x7efca40c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca40c5860 0x7efca40c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 -- 172.21.15.26:0/4042070382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40c4910 con 0x7efca40c5480 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.128+0000 7efca259c640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca40c5480 0x7efca41ee940 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c820 tx=0x7efc9400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.129+0000 7efc9affd640 1 -- 172.21.15.26:0/4042070382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94015070 con 0x7efca40c5480 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.129+0000 7efc9affd640 1 -- 172.21.15.26:0/4042070382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc94007a30 con 0x7efca40c5480 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.129+0000 7efc9affd640 1 -- 172.21.15.26:0/4042070382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94007e00 con 0x7efca40c5480 2024-03-06T07:25:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.129+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca40c4bf0 con 0x7efca40c5480 2024-03-06T07:25:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.129+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca41ee420 con 0x7efca40c5480 2024-03-06T07:25:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.130+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40c4dc0 con 0x7efca40c5480 2024-03-06T07:25:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.130+0000 7efc9affd640 1 -- 172.21.15.26:0/4042070382 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc94018070 con 0x7efca40c5480 2024-03-06T07:25:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.131+0000 7efc9affd640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc8406b4d0 0x7efc8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.131+0000 7efc9affd640 1 -- 172.21.15.26:0/4042070382 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efc9408a2e0 con 0x7efca40c5480 2024-03-06T07:25:29.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.131+0000 7efc9bfff640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc8406b4d0 0x7efc8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.131+0000 7efc9bfff640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc8406b4d0 0x7efc8406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efca40c2380 tx=0x7efc8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.135+0000 7efc9affd640 1 -- 172.21.15.26:0/4042070382 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc94059060 con 0x7efca40c5480 2024-03-06T07:25:29.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.352+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efca404f330 con 0x7efca40c5480 2024-03-06T07:25:29.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.353+0000 7efc9affd640 1 -- 172.21.15.26:0/4042070382 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4993 (secure 0 0 0) 0x7efc94058960 con 0x7efca40c5480 2024-03-06T07:25:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:29.354 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":121,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1667664264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1667664264},{"type":"v1","addr":"172.21.15.117:6836","nonce":1667664264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":7840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:22.149645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8295":{"gid":8295,"name":"a","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1099020571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1099020571},{"type":"v1","addr":"172.21.15.26:6835","nonce":1099020571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8295,"qdb_cluster":[8295]},"id":10}]} 2024-03-06T07:25:29.355 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 121 2024-03-06T07:25:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc8406b4d0 msgr2=0x7efc8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc8406b4d0 0x7efc8406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efca40c2380 tx=0x7efc8c009040 comp rx=0 tx=0).stop 2024-03-06T07:25:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca40c5480 msgr2=0x7efca41ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca40c5480 0x7efca41ee940 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c820 tx=0x7efc9400cc20 comp rx=0 tx=0).stop 2024-03-06T07:25:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 shutdown_connections 2024-03-06T07:25:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc8406b4d0 0x7efc8406d990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca41586f0 0x7efca41ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca40c5860 0x7efca40c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 --2- 172.21.15.26:0/4042070382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efca40c5480 0x7efca41ee940 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 >> 172.21.15.26:0/4042070382 conn(0x7efca406b1a0 msgr2=0x7efca41515a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 shutdown_connections 2024-03-06T07:25:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.354+0000 7efca886f640 1 -- 172.21.15.26:0/4042070382 wait complete. 2024-03-06T07:25:29.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:20.149307+0000', 'modified': '2024-03-06T07:25:22.149645+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8295': {'gid': 8295, 'name': 'a', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1099020571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1099020571}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1099020571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8295, 'qdb_cluster': [8295]} 2024-03-06T07:25:29.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:29.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:25:29.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.873+0000 7f8bb8fe9640 1 Processor -- start 2024-03-06T07:25:29.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.873+0000 7f8bb8fe9640 1 -- start start 2024-03-06T07:25:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb8fe9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bb4152b40 0x7f8bb40c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb8fe9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8bb40724d0 0x7f8bb41551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb8fe9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb40733f0 con 0x7f8bb40a7ff0 2024-03-06T07:25:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb8fe9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb4072d70 con 0x7f8bb4152b40 2024-03-06T07:25:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb8fe9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40c82c0 con 0x7f8bb40724d0 2024-03-06T07:25:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bb4152b40 0x7f8bb40c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8bb40a7ff0 0x7f8bb4155730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56346/0 (socket says 172.21.15.26:56346) 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.874+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/4059081047 learned_addr learned my addr 172.21.15.26:0/4059081047 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.875+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/4059081047 >> v2:172.21.15.117:3300/0 conn(0x7f8bb40724d0 0x7f8bb41551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.875+0000 7f8bb259c640 1 -- 172.21.15.26:0/4059081047 >> v2:172.21.15.117:3300/0 conn(0x7f8bb40724d0 msgr2=0x7f8bb41551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.875+0000 7f8bb259c640 1 --2- 172.21.15.26:0/4059081047 >> v2:172.21.15.117:3300/0 conn(0x7f8bb40724d0 0x7f8bb41551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.875+0000 7f8bb259c640 1 -- 172.21.15.26:0/4059081047 >> v1:172.21.15.26:6789/0 conn(0x7f8bb40a7ff0 legacy=0x7f8bb4155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.875+0000 7f8bb259c640 1 -- 172.21.15.26:0/4059081047 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb406fd30 con 0x7f8bb4152b40 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.875+0000 7f8bb159a640 1 -- 172.21.15.26:0/4059081047 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690665068 0 0) 0x7f8bb40733f0 con 0x7f8bb40a7ff0 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb259c640 1 --2- 172.21.15.26:0/4059081047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c8810 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40c5330 tx=0x7f8b9c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44584cca70d31ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb159a640 1 -- 172.21.15.26:0/4059081047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0315a0 con 0x7f8bb4152b40 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb159a640 1 -- 172.21.15.26:0/4059081047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c037070 con 0x7f8bb4152b40 2024-03-06T07:25:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb159a640 1 -- 172.21.15.26:0/4059081047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c031960 con 0x7f8bb4152b40 2024-03-06T07:25:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/4059081047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 msgr2=0x7f8bb40c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/4059081047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c8810 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40c5330 tx=0x7f8b9c02b840 comp rx=0 tx=0).stop 2024-03-06T07:25:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/4059081047 shutdown_connections 2024-03-06T07:25:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/4059081047 >> v2:172.21.15.117:3300/0 conn(0x7f8bb40724d0 0x7f8bb41551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/4059081047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c8810 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/4059081047 >> 172.21.15.26:0/4059081047 conn(0x7f8bb406b970 msgr2=0x7f8bb40aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/4059081047 shutdown_connections 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.876+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/4059081047 wait complete. 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 Processor -- start 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 -- start start 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb40724d0 0x7f8bb4148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb40a7ff0 0x7f8bb406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40c8d50 con 0x7f8bb4152b40 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4070490 con 0x7f8bb40724d0 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb8fe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40c8ed0 con 0x7f8bb40a7ff0 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.878+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50102/0 (socket says 172.21.15.26:50102) 2024-03-06T07:25:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/1671031318 learned_addr learned my addr 172.21.15.26:0/1671031318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb259c640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb40724d0 0x7f8bb4148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb40a7ff0 msgr2=0x7f8bb406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb40a7ff0 0x7f8bb406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb40a7ff0 0x7f8bb406d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb40724d0 msgr2=0x7f8bb4148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb40724d0 0x7f8bb4148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/1671031318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40c8a20 con 0x7f8bb4152b40 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb259c640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb40724d0 0x7f8bb4148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c5b00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c700 tx=0x7f8bac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8baaffd640 1 -- 172.21.15.26:0/1671031318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac018070 con 0x7f8bb4152b40 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8baaffd640 1 -- 172.21.15.26:0/1671031318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00ce10 con 0x7f8bb4152b40 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb4148ff0 con 0x7f8bb4152b40 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8baaffd640 1 -- 172.21.15.26:0/1671031318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00a860 con 0x7f8bb4152b40 2024-03-06T07:25:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.879+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb4149290 con 0x7f8bb4152b40 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.881+0000 7f8baaffd640 1 -- 172.21.15.26:0/1671031318 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bac01b070 con 0x7f8bb4152b40 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.882+0000 7f8baaffd640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9406b4d0 0x7f8b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.882+0000 7f8baaffd640 1 -- 172.21.15.26:0/1671031318 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8bac089860 con 0x7f8bb4152b40 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.882+0000 7f8bb259c640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9406b4d0 0x7f8b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.882+0000 7f8bbbbd4740 1 Processor -- start 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.882+0000 7f8bbbbd4740 1 -- start start 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.882+0000 7f8bb259c640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9406b4d0 0x7f8b9406d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c036000 tx=0x7f8b9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484df50 0x56264484ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264484bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484c4b0 0x562644848680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626447b26b0 con 0x56264484df50 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264475ba60 con 0x56264484e330 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626447ae750 con 0x56264484c4b0 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484df50 0x56264484ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484df50 0x56264484ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50104/0 (socket says 172.21.15.26:50104) 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 -- 172.21.15.26:0/3406665091 learned_addr learned my addr 172.21.15.26:0/3406665091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264484bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484c4b0 0x562644848680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 -- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484c4b0 msgr2=0x562644848680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484c4b0 0x562644848680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 -- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 msgr2=0x56264484bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264484bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.883+0000 7f8bb259c640 1 -- 172.21.15.26:0/3406665091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562644779f30 con 0x56264484df50 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bb259c640 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484df50 0x56264484ba30 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c006ce0 tx=0x7f8b9c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b225f651ec07eda1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8b93fff640 1 -- 172.21.15.26:0/3406665091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c032980 con 0x56264484df50 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8b93fff640 1 -- 172.21.15.26:0/3406665091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c037040 con 0x56264484df50 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8b93fff640 1 -- 172.21.15.26:0/3406665091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c032d60 con 0x56264484df50 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3406665091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484df50 msgr2=0x56264484ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484df50 0x56264484ba30 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c006ce0 tx=0x7f8b9c030000 comp rx=0 tx=0).stop 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3406665091 shutdown_connections 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484c4b0 0x562644848680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264484bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3406665091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484df50 0x56264484ba30 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.884+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3406665091 >> 172.21.15.26:0/3406665091 conn(0x56264484a7d0 msgr2=0x56264486f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.885+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3406665091 shutdown_connections 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.885+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3406665091 wait complete. 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.885+0000 7f8bbbbd4740 1 -- start start 2024-03-06T07:25:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.885+0000 7f8bbbbd4740 1 Processor -- start 2024-03-06T07:25:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x56264488a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264489e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x562644886a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562644870230 con 0x5626448725e0 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264476eea0 con 0x56264484e330 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56264486fc70 con 0x56264484df50 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bbbbd4740 10 client.0 _ll_register_callbacks cb 0x56264476e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x562644886a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x562644886a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50118/0 (socket says 172.21.15.26:50118) 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/2399067122 learned_addr learned my addr 172.21.15.26:0/2399067122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bb259c640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x56264488a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.886+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264489e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 msgr2=0x56264488a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x56264488a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 msgr2=0x56264489e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264489e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/2399067122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56264476f270 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bb259c640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x56264488a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x562644886a00 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8bac057e20 tx=0x7f8bac05c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e34f5e655256f5c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8b917fa640 1 -- 172.21.15.26:0/2399067122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac057330 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8b917fa640 1 -- 172.21.15.26:0/2399067122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8bac0574d0 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b9c037040 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8b917fa640 1 -- 172.21.15.26:0/2399067122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bac053c30 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bbbbd4740 10 client.8472 fetch_fsmap learned FSMap version 121 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x5626444c72d0 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8b917fa640 1 -- 172.21.15.26:0/2399067122 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c037040 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8b917fa640 1 -- 172.21.15.26:0/2399067122 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 121) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f8bac053dd0 con 0x5626448725e0 2024-03-06T07:25:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bbbbd4740 10 client.8472 fetch_fsmap finished waiting for FSMap version 121 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bbbbd4740 10 client.8472 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.887+0000 7f8bbbbd4740 20 client.8472 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.888+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x56264489b340 0x5626448a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.888+0000 7f8bbbbd4740 4 client.8472 mds_command: new command op to 8295 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.888+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 --> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626447bfbf0 con 0x56264489b340 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.888+0000 7f8bb259c640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x56264489b340 0x5626448a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.888+0000 7f8b917fa640 10 client.8472 ms_handle_connect on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.888+0000 7f8bb259c640 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x56264489b340 0x5626448a94b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c5b277ed7c76c3 server_cookie=fe9a86032b425ecf in_seq=0 out_seq=0 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8b917fa640 1 -- 172.21.15.26:0/2399067122 <== mds.0 v2:172.21.15.26:6834/1099020571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5626447bfbf0 con 0x56264489b340 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8b917fa640 10 client.8472 handle_command_reply: tid=0 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 client.8472 shutdown 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 msgr2=0x562644886a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x562644886a00 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8bac057e20 tx=0x7f8bac05c140 comp rx=0 tx=0).stop 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 shutdown_connections 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x56264489b340 0x5626448a94b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x562644886a00 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x56264489e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/2399067122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x56264488a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8b917fa640 0 client.8472 ms_handle_reset on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 >> 172.21.15.26:0/2399067122 conn(0x56264484c4b0 msgr2=0x56264486f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 shutdown_connections 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/2399067122 wait complete. 2024-03-06T07:25:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.889+0000 7f8bbbbd4740 20 client.8472 trim_cache size 0 max 16384 2024-03-06T07:25:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 Processor -- start 2024-03-06T07:25:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 -- start start 2024-03-06T07:25:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484df50 0x562644874c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484e330 0x5626448865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5626448725e0 0x562644847510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41f6de0 con 0x56264484e330 2024-03-06T07:25:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40ad430 con 0x56264484df50 2024-03-06T07:25:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.906+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41f6810 con 0x5626448725e0 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484e330 0x5626448865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484e330 0x5626448865e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50130/0 (socket says 172.21.15.26:50130) 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 -- 172.21.15.26:0/756432339 learned_addr learned my addr 172.21.15.26:0/756432339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb259c640 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484df50 0x562644874c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5626448725e0 0x562644847510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 -- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5626448725e0 msgr2=0x562644847510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5626448725e0 0x562644847510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 -- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484df50 msgr2=0x562644874c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484df50 0x562644874c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 -- 172.21.15.26:0/756432339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626445916b0 con 0x56264484e330 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484e330 0x5626448865e0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009820 tx=0x7f8ba4009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8319d1be9bcd6e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8b937fe640 1 -- 172.21.15.26:0/756432339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba401a030 con 0x56264484e330 2024-03-06T07:25:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8b937fe640 1 -- 172.21.15.26:0/756432339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4013840 con 0x56264484e330 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.907+0000 7f8b937fe640 1 -- 172.21.15.26:0/756432339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400f040 con 0x56264484e330 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/756432339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484e330 msgr2=0x5626448865e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484e330 0x5626448865e0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009820 tx=0x7f8ba4009ce0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/756432339 shutdown_connections 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5626448725e0 0x562644847510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56264484e330 0x5626448865e0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/756432339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484df50 0x562644874c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/756432339 >> 172.21.15.26:0/756432339 conn(0x56264484c4b0 msgr2=0x56264484f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/756432339 shutdown_connections 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/756432339 wait complete. 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 -- start start 2024-03-06T07:25:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.908+0000 7f8bbbbd4740 1 Processor -- start 2024-03-06T07:25:29.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x562644847910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x5626448fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bbbbd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x5626448f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264489e050 con 0x5626448725e0 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562644872b50 con 0x56264484e330 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bbbbd4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562644870600 con 0x56264484df50 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bbbbd4740 10 client.0 _ll_register_callbacks cb 0x7f8bac057f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x5626448f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x5626448f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50146/0 (socket says 172.21.15.26:50146) 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/3640426451 learned_addr learned my addr 172.21.15.26:0/3640426451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bb1d9b640 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x5626448fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.909+0000 7f8bb259c640 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x562644847910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 msgr2=0x562644847910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x562644847910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 msgr2=0x5626448fe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x5626448fe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bb2d9d640 1 -- 172.21.15.26:0/3640426451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56264484ccc0 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bb2d9d640 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x5626448f26c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8bac089b80 tx=0x7f8bac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91fd07d2ae4f5cd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8b917fa640 1 -- 172.21.15.26:0/3640426451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac073070 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8b917fa640 1 -- 172.21.15.26:0/3640426451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8bac056e90 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ba4013840 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8b917fa640 1 -- 172.21.15.26:0/3640426451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bac057840 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 10 client.8478 fetch_fsmap learned FSMap version 121 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x5626448ae520 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8b917fa640 1 -- 172.21.15.26:0/3640426451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4013840 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8b917fa640 1 -- 172.21.15.26:0/3640426451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 121) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f8bac00aa00 con 0x5626448725e0 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 10 client.8478 fetch_fsmap finished waiting for FSMap version 121 2024-03-06T07:25:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 10 client.8478 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 20 client.8478 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x562644873a30 0x562644873e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 4 client.8478 mds_command: new command op to 8295 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.910+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 --> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5626447b9f40 con 0x562644873a30 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bb259c640 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x562644873a30 0x562644873e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bb259c640 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x562644873a30 0x562644873e10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55bbc66fdf085b4 server_cookie=d3d9e6dd3e237e53 in_seq=0 out_seq=0 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8b917fa640 10 client.8478 ms_handle_connect on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8b917fa640 1 -- 172.21.15.26:0/3640426451 <== mds.0 v2:172.21.15.26:6834/1099020571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5626447b9f40 con 0x562644873a30 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8b917fa640 10 client.8478 handle_command_reply: tid=0 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 client.8478 shutdown 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 msgr2=0x5626448f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x5626448f26c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8bac089b80 tx=0x7f8bac009040 comp rx=0 tx=0).stop 2024-03-06T07:25:29.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 shutdown_connections 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x562644873a30 0x562644873e10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626448725e0 0x5626448f26c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56264484e330 0x5626448fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 --2- 172.21.15.26:0/3640426451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56264484df50 0x562644847910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8b917fa640 0 client.8478 ms_handle_reset on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 >> 172.21.15.26:0/3640426451 conn(0x56264484c4b0 msgr2=0x56264484efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 shutdown_connections 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.911+0000 7f8bbbbd4740 1 -- 172.21.15.26:0/3640426451 wait complete. 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bbbbd4740 20 client.8478 trim_cache size 0 max 16384 2024-03-06T07:25:29.920 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8295, 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 9.131146459, 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 121, 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 135, 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 135, 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 14.214978972000001 2024-03-06T07:25:29.921 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:29.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9406b4d0 msgr2=0x7f8b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9406b4d0 0x7f8b9406d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c036000 tx=0x7f8b9c008030 comp rx=0 tx=0).stop 2024-03-06T07:25:29.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 msgr2=0x7f8bb40c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c5b00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c700 tx=0x7f8bac00cbc0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 shutdown_connections 2024-03-06T07:25:29.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9406b4d0 0x7f8b9406d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb4152b40 0x7f8bb40c5b00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb40a7ff0 0x7f8bb406d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 --2- 172.21.15.26:0/1671031318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb40724d0 0x7f8bb4148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 >> 172.21.15.26:0/1671031318 conn(0x7f8bb406b970 msgr2=0x7f8bb414f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 shutdown_connections 2024-03-06T07:25:29.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:29.912+0000 7f8bb8fe9640 1 -- 172.21.15.26:0/1671031318 wait complete. 2024-03-06T07:25:29.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.049+0000 7f963f3f6640 1 Processor -- start 2024-03-06T07:25:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.050+0000 7f963f3f6640 1 -- start start 2024-03-06T07:25:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.050+0000 7f963f3f6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f96381506b0 0x7f9638153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.050+0000 7f963f3f6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9638070aa0 con 0x7f96380acc00 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.050+0000 7f963f3f6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9638070d80 con 0x7f9638156ea0 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.050+0000 7f963f3f6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9638156450 con 0x7f96381506b0 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.050+0000 7f963d993640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f96380acc00 0x7f96380c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56356/0 (socket says 172.21.15.26:56356) 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.050+0000 7f963d993640 1 -- 172.21.15.26:0/3534214727 learned_addr learned my addr 172.21.15.26:0/3534214727 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d192640 1 --2- 172.21.15.26:0/3534214727 >> v2:172.21.15.117:3300/0 conn(0x7f96381506b0 0x7f9638153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602025557 0 0) 0x7f9638070aa0 con 0x7f96380acc00 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9624003520 con 0x7f96380acc00 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 657991539 0 0) 0x7f9624003520 con 0x7f96380acc00 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 >> v2:172.21.15.117:3300/0 conn(0x7f96381506b0 msgr2=0x7f9638153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 --2- 172.21.15.26:0/3534214727 >> v2:172.21.15.117:3300/0 conn(0x7f96381506b0 0x7f9638153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 >> v1:172.21.15.117:6789/0 conn(0x7f9638156ea0 legacy=0x7f96380702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96381566a0 con 0x7f96380acc00 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f9634001670 con 0x7f96380acc00 2024-03-06T07:25:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 >> v1:172.21.15.26:6789/0 conn(0x7f96380acc00 legacy=0x7f96380c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240039b0 0x7f9624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9624004800 0x7f9624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96240051c0 0x7f962400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9638156450 con 0x7f96240039b0 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962400c060 con 0x7f96240051c0 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962400c260 con 0x7f9624004800 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d192640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240039b0 0x7f9624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d993640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96240051c0 0x7f962400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963c991640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9624004800 0x7f9624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d192640 1 -- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9624004800 msgr2=0x7f9624004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d192640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9624004800 0x7f9624004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d192640 1 -- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96240051c0 msgr2=0x7f962400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d192640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96240051c0 0x7f962400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.051+0000 7f963d192640 1 -- 172.21.15.26:0/3534214727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9624003520 con 0x7f96240039b0 2024-03-06T07:25:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.052+0000 7f963d993640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96240051c0 0x7f962400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.052+0000 7f963d993640 1 -- 172.21.15.26:0/3534214727 reap_dead start 2024-03-06T07:25:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963d192640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240039b0 0x7f9624003dd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9630009fc0 tx=0x7f9624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6736ccd760aaed43 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630039070 con 0x7f96240039b0 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963002cc20 con 0x7f96240039b0 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f962ffff640 1 -- 172.21.15.26:0/3534214727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630009c10 con 0x7f96240039b0 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963f3f6640 1 -- 172.21.15.26:0/3534214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240039b0 msgr2=0x7f9624003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963f3f6640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240039b0 0x7f9624003dd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9630009fc0 tx=0x7f9624005830 comp rx=0 tx=0).stop 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963f3f6640 1 -- 172.21.15.26:0/3534214727 shutdown_connections 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963f3f6640 1 --2- 172.21.15.26:0/3534214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240039b0 0x7f9624003dd0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963f3f6640 1 -- 172.21.15.26:0/3534214727 >> 172.21.15.26:0/3534214727 conn(0x7f963806b1a0 msgr2=0x7f963806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963f3f6640 1 -- 172.21.15.26:0/3534214727 shutdown_connections 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.053+0000 7f963f3f6640 1 -- 172.21.15.26:0/3534214727 wait complete. 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 Processor -- start 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 -- start start 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96240039b0 0x7f96381eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9638149fe0 0x7f96380c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9638147920 0x7f9638147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9638156a10 con 0x7f9638147920 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380ad260 con 0x7f9638149fe0 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963f3f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9638153990 con 0x7f96240039b0 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963d993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9638147920 0x7f9638147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963d993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9638147920 0x7f9638147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50154/0 (socket says 172.21.15.26:50154) 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963d993640 1 -- 172.21.15.26:0/359030420 learned_addr learned my addr 172.21.15.26:0/359030420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963d192640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96240039b0 0x7f96381eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.055+0000 7f963c991640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9638149fe0 0x7f96380c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963d993640 1 -- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96240039b0 msgr2=0x7f96381eeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963d993640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96240039b0 0x7f96381eeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963d993640 1 -- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9638149fe0 msgr2=0x7f96380c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963d993640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9638149fe0 0x7f96380c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963d993640 1 -- 172.21.15.26:0/359030420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96381482e0 con 0x7f9638147920 2024-03-06T07:25:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963d993640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9638147920 0x7f9638147da0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f963400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f962dffb640 1 -- 172.21.15.26:0/359030420 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634019070 con 0x7f9638147920 2024-03-06T07:25:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f962dffb640 1 -- 172.21.15.26:0/359030420 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634014480 con 0x7f9638147920 2024-03-06T07:25:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963806db80 con 0x7f9638147920 2024-03-06T07:25:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f962dffb640 1 -- 172.21.15.26:0/359030420 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634014830 con 0x7f9638147920 2024-03-06T07:25:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.056+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96380c22a0 con 0x7f9638147920 2024-03-06T07:25:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.057+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9638156bc0 con 0x7f9638147920 2024-03-06T07:25:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.058+0000 7f962dffb640 1 -- 172.21.15.26:0/359030420 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f963401c070 con 0x7f9638147920 2024-03-06T07:25:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.058+0000 7f962dffb640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f961c06b500 0x7f961c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.058+0000 7f962dffb640 1 -- 172.21.15.26:0/359030420 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96340895c0 con 0x7f9638147920 2024-03-06T07:25:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.058+0000 7f963d192640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f961c06b500 0x7f961c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.059+0000 7f963d192640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f961c06b500 0x7f961c06d9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9630036000 tx=0x7f963002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.063+0000 7f962dffb640 1 -- 172.21.15.26:0/359030420 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9638156bc0 con 0x7f9638147920 2024-03-06T07:25:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.306+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f963804f330 con 0x7f9638147920 2024-03-06T07:25:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.307+0000 7f962dffb640 1 -- 172.21.15.26:0/359030420 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f9634057c50 con 0x7f9638147920 2024-03-06T07:25:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:30.309 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":121,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1667664264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1667664264},{"type":"v1","addr":"172.21.15.117:6836","nonce":1667664264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":7840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:22.149645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8295":{"gid":8295,"name":"a","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1099020571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1099020571},{"type":"v1","addr":"172.21.15.26:6835","nonce":1099020571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8295,"qdb_cluster":[8295]},"id":10}]} 2024-03-06T07:25:30.309 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 121 2024-03-06T07:25:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.309+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f961c06b500 msgr2=0x7f961c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.309+0000 7f963f3f6640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f961c06b500 0x7f961c06d9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9630036000 tx=0x7f963002e000 comp rx=0 tx=0).stop 2024-03-06T07:25:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.309+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9638147920 msgr2=0x7f9638147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.309+0000 7f963f3f6640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9638147920 0x7f9638147da0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f963400e820 comp rx=0 tx=0).stop 2024-03-06T07:25:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 shutdown_connections 2024-03-06T07:25:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f961c06b500 0x7f961c06d9c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9638147920 0x7f9638147da0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9638149fe0 0x7f96380c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 --2- 172.21.15.26:0/359030420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96240039b0 0x7f96381eeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 >> 172.21.15.26:0/359030420 conn(0x7f963806b1a0 msgr2=0x7f96380ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 shutdown_connections 2024-03-06T07:25:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.310+0000 7f963f3f6640 1 -- 172.21.15.26:0/359030420 wait complete. 2024-03-06T07:25:30.326 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 121, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:20.149307+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 121, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8295": { "addr": "172.21.15.26:6835/1099020571", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 1099020571, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 1099020571, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8295, "incarnation": 119, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:22.149645+0000", "qdb_cluster": [ 8295 ], "qdb_leader": 8295, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8295 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/1667664264", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1667664264, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 1667664264, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6848, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3380233594", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3380233594, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3380233594, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7840, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6833/2142785555", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 2142785555, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 2142785555, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8292, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:30.326 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:25:30.326 INFO:tasks.cephfs.test_failover:current = [{'gid': 8295, 'name': 'a', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1099020571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1099020571}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1099020571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6848, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/1667664264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1667664264}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1667664264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}, {'gid': 8292, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/2142785555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2142785555}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 2142785555}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}, {'gid': 7840, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3380233594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3380233594}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3380233594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}] 2024-03-06T07:25:30.326 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.b mds_join_fs cephfs 2024-03-06T07:25:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.429+0000 7f7654c42640 1 Processor -- start 2024-03-06T07:25:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.430+0000 7f7654c42640 1 -- start start 2024-03-06T07:25:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f7654c42640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f76501536c0 0x7f765006f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f7654c42640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 0x7f765015d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f7654c42640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500a70f0 con 0x7f765006afc0 2024-03-06T07:25:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f7654c42640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7650150a20 con 0x7f76501536c0 2024-03-06T07:25:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f7654c42640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76501565a0 con 0x7f765006b3a0 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f765006afc0 0x7f7650152fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56360/0 (socket says 172.21.15.26:56360) 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764e59c640 1 -- 172.21.15.26:0/4180729487 learned_addr learned my addr 172.21.15.26:0/4180729487 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147059034 0 0) 0x7f76500a70f0 con 0x7f765006afc0 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7638003520 con 0x7f765006afc0 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764dd9b640 1 --2- 172.21.15.26:0/4180729487 >> v2:172.21.15.117:3300/0 conn(0x7f76501536c0 0x7f765006f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764ed9d640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 0x7f765015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 637757859 0 0) 0x7f7638003520 con 0x7f765006afc0 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 msgr2=0x7f765015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.431+0000 7f764d59a640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 0x7f765015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 >> v2:172.21.15.117:3300/0 conn(0x7f76501536c0 msgr2=0x7f765006f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 --2- 172.21.15.26:0/4180729487 >> v2:172.21.15.117:3300/0 conn(0x7f76501536c0 0x7f765006f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76501567f0 con 0x7f765006afc0 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f7640001670 con 0x7f765006afc0 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 >> v1:172.21.15.26:6789/0 conn(0x7f765006afc0 legacy=0x7f7650152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 0x7f76380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7638003f30 0x7f7638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638004d40 0x7f76380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650150a20 con 0x7f7638003f30 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501565a0 con 0x7f7638004d40 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763800baa0 con 0x7f765006b3a0 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764dd9b640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7638003f30 0x7f7638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764e59c640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 0x7f76380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764ed9d640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638004d40 0x7f76380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764dd9b640 1 -- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 msgr2=0x7f76380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764dd9b640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f765006b3a0 0x7f76380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764dd9b640 1 -- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638004d40 msgr2=0x7f76380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764dd9b640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638004d40 0x7f76380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.432+0000 7f764dd9b640 1 -- 172.21.15.26:0/4180729487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638003520 con 0x7f7638003f30 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.433+0000 7f764dd9b640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7638003f30 0x7f7638004800 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7644009a90 tx=0x7f7638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d422325b51aacb09 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.433+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644039070 con 0x7f7638003f30 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.433+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764402c6b0 con 0x7f7638003f30 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.433+0000 7f764d59a640 1 -- 172.21.15.26:0/4180729487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644009690 con 0x7f7638003f30 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.433+0000 7f7654c42640 1 -- 172.21.15.26:0/4180729487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7638003f30 msgr2=0x7f7638004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.433+0000 7f7654c42640 1 --2- 172.21.15.26:0/4180729487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7638003f30 0x7f7638004800 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7644009a90 tx=0x7f7638005830 comp rx=0 tx=0).stop 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.433+0000 7f764ed9d640 1 -- 172.21.15.26:0/4180729487 reap_dead start 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.434+0000 7f7654c42640 1 -- 172.21.15.26:0/4180729487 shutdown_connections 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.434+0000 7f7654c42640 1 -- 172.21.15.26:0/4180729487 >> 172.21.15.26:0/4180729487 conn(0x7f76500c4290 msgr2=0x7f76500c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.434+0000 7f7654c42640 1 -- 172.21.15.26:0/4180729487 shutdown_connections 2024-03-06T07:25:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.434+0000 7f7654c42640 1 -- 172.21.15.26:0/4180729487 wait complete. 2024-03-06T07:25:30.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.435+0000 7f7654c42640 1 Processor -- start 2024-03-06T07:25:30.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.435+0000 7f7654c42640 1 -- start start 2024-03-06T07:25:30.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638003f30 0x7f76501ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76501ec850 0x7f7650151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7650152330 0x7f76501e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650156db0 con 0x7f7650152330 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765006b810 con 0x7f7638003f30 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500c7360 con 0x7f76501ec850 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7650152330 0x7f76501e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7650152330 0x7f76501e8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50170/0 (socket says 172.21.15.26:50170) 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 -- 172.21.15.26:0/26821416 learned_addr learned my addr 172.21.15.26:0/26821416 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764e59c640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638003f30 0x7f76501ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 -- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76501ec850 msgr2=0x7f7650151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764dd9b640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76501ec850 0x7f7650151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76501ec850 0x7f7650151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 -- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638003f30 msgr2=0x7f76501ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638003f30 0x7f76501ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 -- 172.21.15.26:0/26821416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76501ecf90 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f764ed9d640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7650152330 0x7f76501e8b30 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f76500a7480 tx=0x7f76480115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f763effd640 1 -- 172.21.15.26:0/26821416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648011e40 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f763effd640 1 -- 172.21.15.26:0/26821416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764801b070 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76501eccd0 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f763effd640 1 -- 172.21.15.26:0/26821416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648003990 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.436+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76500aab60 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.438+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765004f330 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.438+0000 7f763effd640 1 -- 172.21.15.26:0/26821416 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7648005d10 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.438+0000 7f763effd640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.439+0000 7f763effd640 1 -- 172.21.15.26:0/26821416 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f764808a660 con 0x7f7650152330 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.439+0000 7f764e59c640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.439+0000 7f764e59c640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f763006b4d0 0x7f763006d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f76500aa6e0 tx=0x7f7640009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.443+0000 7f763effd640 1 -- 172.21.15.26:0/26821416 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7648014350 con 0x7f7650152330 2024-03-06T07:25:30.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.681+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f7650068460 con 0x7f7650152330 2024-03-06T07:25:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.684+0000 7f763effd640 1 -- 172.21.15.26:0/26821416 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v46)=0 v46) v1 ==== 116+0+0 (secure 0 0 0) 0x7f76480593e0 con 0x7f7650152330 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f763006b4d0 msgr2=0x7f763006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f763006b4d0 0x7f763006d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f76500aa6e0 tx=0x7f7640009040 comp rx=0 tx=0).stop 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7650152330 msgr2=0x7f76501e8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7650152330 0x7f76501e8b30 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f76500a7480 tx=0x7f76480115a0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 shutdown_connections 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7650152330 0x7f76501e8b30 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76501ec850 0x7f7650151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 --2- 172.21.15.26:0/26821416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7638003f30 0x7f76501ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 >> 172.21.15.26:0/26821416 conn(0x7f76500c4290 msgr2=0x7f7650073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 shutdown_connections 2024-03-06T07:25:30.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.687+0000 7f7654c42640 1 -- 172.21.15.26:0/26821416 wait complete. 2024-03-06T07:25:30.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs 2024-03-06T07:25:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.802+0000 7fe007e39640 1 Processor -- start 2024-03-06T07:25:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.802+0000 7fe007e39640 1 -- start start 2024-03-06T07:25:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe007e39640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe0000c88b0 0x7fe000158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe007e39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0001560b0 con 0x7fe000155b60 2024-03-06T07:25:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe007e39640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00014f580 con 0x7fe00014f1a0 2024-03-06T07:25:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe007e39640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0000c8dc0 con 0x7fe0000c88b0 2024-03-06T07:25:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe0063d6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe000155b60 0x7fe000158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56376/0 (socket says 172.21.15.26:56376) 2024-03-06T07:25:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe0063d6640 1 -- 172.21.15.26:0/319944600 learned_addr learned my addr 172.21.15.26:0/319944600 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe0053d4640 1 --2- 172.21.15.26:0/319944600 >> v2:172.21.15.117:3300/0 conn(0x7fe0000c88b0 0x7fe000158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.803+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075683023 0 0) 0x7fe0001560b0 con 0x7fe000155b60 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfec003520 con 0x7fe000155b60 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2407409508 0 0) 0x7fdfec003520 con 0x7fe000155b60 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 >> v2:172.21.15.117:3300/0 conn(0x7fe0000c88b0 msgr2=0x7fe000158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 --2- 172.21.15.26:0/319944600 >> v2:172.21.15.117:3300/0 conn(0x7fe0000c88b0 0x7fe000158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 >> v1:172.21.15.117:6789/0 conn(0x7fe00014f1a0 legacy=0x7fe0000c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000c78d0 con 0x7fe000155b60 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fdff80019f0 con 0x7fe000155b60 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 >> v1:172.21.15.26:6789/0 conn(0x7fe000155b60 legacy=0x7fe000158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfec0039b0 0x7fdfec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec004800 0x7fdfec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec0051c0 0x7fdfec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec00c130 con 0x7fdfec0039b0 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec00c2e0 con 0x7fdfec0051c0 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec00c4e0 con 0x7fdfec004800 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe005bd5640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfec0039b0 0x7fdfec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe0053d4640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec004800 0x7fdfec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe0063d6640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec0051c0 0x7fdfec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe005bd5640 1 -- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec004800 msgr2=0x7fdfec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe005bd5640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec004800 0x7fdfec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe005bd5640 1 -- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec0051c0 msgr2=0x7fdfec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe005bd5640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec0051c0 0x7fdfec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe005bd5640 1 -- 172.21.15.26:0/319944600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec003520 con 0x7fdfec0039b0 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe0053d4640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec004800 0x7fdfec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe0063d6640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec0051c0 0x7fdfec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.804+0000 7fe0063d6640 1 -- 172.21.15.26:0/319944600 reap_dead start 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.805+0000 7fe005bd5640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfec0039b0 0x7fdfec003dd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdffc001750 tx=0x7fdfec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35baf39a32f51d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc001370 con 0x7fdfec0039b0 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc0052d0 con 0x7fdfec0039b0 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe004bd3640 1 -- 172.21.15.26:0/319944600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc034b50 con 0x7fdfec0039b0 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe007e39640 1 -- 172.21.15.26:0/319944600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfec0039b0 msgr2=0x7fdfec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe007e39640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfec0039b0 0x7fdfec003dd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdffc001750 tx=0x7fdfec005830 comp rx=0 tx=0).stop 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe007e39640 1 -- 172.21.15.26:0/319944600 shutdown_connections 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe007e39640 1 --2- 172.21.15.26:0/319944600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfec0039b0 0x7fdfec003dd0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe007e39640 1 -- 172.21.15.26:0/319944600 >> 172.21.15.26:0/319944600 conn(0x7fe0000728d0 msgr2=0x7fe000072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe007e39640 1 -- 172.21.15.26:0/319944600 shutdown_connections 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.806+0000 7fe007e39640 1 -- 172.21.15.26:0/319944600 wait complete. 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.807+0000 7fe007e39640 1 Processor -- start 2024-03-06T07:25:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.807+0000 7fe007e39640 1 -- start start 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe007e39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec0039b0 0x7fe000146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe007e39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0001474c0 0x7fe0000aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe007e39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0000aad00 0x7fe000144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe007e39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c7e90 con 0x7fe0001474c0 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe007e39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c8170 con 0x7fe0000aad00 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe007e39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00006d100 con 0x7fdfec0039b0 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0001474c0 0x7fe0000aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0001474c0 0x7fe0000aa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50178/0 (socket says 172.21.15.26:50178) 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 -- 172.21.15.26:0/2220345838 learned_addr learned my addr 172.21.15.26:0/2220345838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe005bd5640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec0039b0 0x7fe000146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0063d6640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0000aad00 0x7fe000144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 -- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec0039b0 msgr2=0x7fe000146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec0039b0 0x7fe000146f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 -- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0000aad00 msgr2=0x7fe000144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0000aad00 0x7fe000144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe0053d4640 1 -- 172.21.15.26:0/2220345838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000147d50 con 0x7fe0001474c0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.808+0000 7fe005bd5640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec0039b0 0x7fe000146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.809+0000 7fe0053d4640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0001474c0 0x7fe0000aa7c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdff0011510 tx=0x7fdff00119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.809+0000 7fdff67fc640 1 -- 172.21.15.26:0/2220345838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0015870 con 0x7fe0001474c0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.809+0000 7fdff67fc640 1 -- 172.21.15.26:0/2220345838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff0015a10 con 0x7fe0001474c0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.809+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0001ea940 con 0x7fe0001474c0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.809+0000 7fdff67fc640 1 -- 172.21.15.26:0/2220345838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0015e00 con 0x7fe0001474c0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.809+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0001ea0a0 con 0x7fe0001474c0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.810+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00004f330 con 0x7fe0001474c0 2024-03-06T07:25:30.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.810+0000 7fdff67fc640 1 -- 172.21.15.26:0/2220345838 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdff001d400 con 0x7fe0001474c0 2024-03-06T07:25:30.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.811+0000 7fdff67fc640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdfe406b4d0 0x7fdfe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.811+0000 7fdff67fc640 1 -- 172.21.15.26:0/2220345838 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdff008ab60 con 0x7fe0001474c0 2024-03-06T07:25:30.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.811+0000 7fe005bd5640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdfe406b4d0 0x7fdfe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.811+0000 7fe005bd5640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdfe406b4d0 0x7fdfe406d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdffc03c000 tx=0x7fdffc038000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.815+0000 7fdff67fc640 1 -- 172.21.15.26:0/2220345838 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdff0019030 con 0x7fe0001474c0 2024-03-06T07:25:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:30.996+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fe00006cee0 con 0x7fe0001474c0 2024-03-06T07:25:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.000+0000 7fdff67fc640 1 -- 172.21.15.26:0/2220345838 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v47)=0 v47) v1 ==== 116+0+0 (secure 0 0 0) 0x7fdff005f070 con 0x7fe0001474c0 2024-03-06T07:25:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdfe406b4d0 msgr2=0x7fdfe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdfe406b4d0 0x7fdfe406d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdffc03c000 tx=0x7fdffc038000 comp rx=0 tx=0).stop 2024-03-06T07:25:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0001474c0 msgr2=0x7fe0000aa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0001474c0 0x7fe0000aa7c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdff0011510 tx=0x7fdff00119d0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 shutdown_connections 2024-03-06T07:25:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdfe406b4d0 0x7fdfe406d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0000aad00 0x7fe000144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0001474c0 0x7fe0000aa7c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 --2- 172.21.15.26:0/2220345838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfec0039b0 0x7fe000146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 >> 172.21.15.26:0/2220345838 conn(0x7fe0000728d0 msgr2=0x7fe00014e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 shutdown_connections 2024-03-06T07:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.002+0000 7fe007e39640 1 -- 172.21.15.26:0/2220345838 wait complete. 2024-03-06T07:25:31.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.115+0000 7fba23078640 1 Processor -- start 2024-03-06T07:25:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.115+0000 7fba23078640 1 -- start start 2024-03-06T07:25:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba23078640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba1c1506b0 0x7fba1c153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba23078640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fba1c156ea0 0x7fba1c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba23078640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c0acc00 0x7fba1c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba23078640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c156450 con 0x7fba1c1506b0 2024-03-06T07:25:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba23078640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c156a10 con 0x7fba1c156ea0 2024-03-06T07:25:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba23078640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c070800 con 0x7fba1c0acc00 2024-03-06T07:25:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba1c1506b0 0x7fba1c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba1c1506b0 0x7fba1c153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50190/0 (socket says 172.21.15.26:50190) 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 -- 172.21.15.26:0/2728053061 learned_addr learned my addr 172.21.15.26:0/2728053061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba1bfff640 1 --2- 172.21.15.26:0/2728053061 >> v2:172.21.15.117:3300/0 conn(0x7fba1c156ea0 0x7fba1c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 -- 172.21.15.26:0/2728053061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c0acc00 msgr2=0x7fba1c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 --2- 172.21.15.26:0/2728053061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c0acc00 0x7fba1c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 -- 172.21.15.26:0/2728053061 >> v2:172.21.15.117:3300/0 conn(0x7fba1c156ea0 msgr2=0x7fba1c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 --2- 172.21.15.26:0/2728053061 >> v2:172.21.15.117:3300/0 conn(0x7fba1c156ea0 0x7fba1c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.116+0000 7fba20e14640 1 -- 172.21.15.26:0/2728053061 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c070aa0 con 0x7fba1c1506b0 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.117+0000 7fba20e14640 1 --2- 172.21.15.26:0/2728053061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c1506b0 0x7fba1c153990 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fba1c06bf40 tx=0x7fba0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a293cec8a18d1369 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.117+0000 7fba1b7fe640 1 -- 172.21.15.26:0/2728053061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c035020 con 0x7fba1c1506b0 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.117+0000 7fba1b7fe640 1 -- 172.21.15.26:0/2728053061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c031870 con 0x7fba1c1506b0 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.117+0000 7fba1b7fe640 1 -- 172.21.15.26:0/2728053061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c031c40 con 0x7fba1c1506b0 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 -- 172.21.15.26:0/2728053061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c1506b0 msgr2=0x7fba1c153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 --2- 172.21.15.26:0/2728053061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c1506b0 0x7fba1c153990 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fba1c06bf40 tx=0x7fba0c02b910 comp rx=0 tx=0).stop 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 -- 172.21.15.26:0/2728053061 shutdown_connections 2024-03-06T07:25:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 --2- 172.21.15.26:0/2728053061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c0acc00 0x7fba1c0c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 --2- 172.21.15.26:0/2728053061 >> v2:172.21.15.117:3300/0 conn(0x7fba1c156ea0 0x7fba1c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 --2- 172.21.15.26:0/2728053061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c1506b0 0x7fba1c153990 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 -- 172.21.15.26:0/2728053061 >> 172.21.15.26:0/2728053061 conn(0x7fba1c06b1a0 msgr2=0x7fba1c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 -- 172.21.15.26:0/2728053061 shutdown_connections 2024-03-06T07:25:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.118+0000 7fba23078640 1 -- 172.21.15.26:0/2728053061 wait complete. 2024-03-06T07:25:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.119+0000 7fba23078640 1 Processor -- start 2024-03-06T07:25:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.119+0000 7fba23078640 1 -- start start 2024-03-06T07:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba23078640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba1c0acc00 0x7fba1c0c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba23078640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c1506b0 0x7fba1c0c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba23078640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c156ea0 0x7fba1c152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba23078640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0c59e0 con 0x7fba1c156ea0 2024-03-06T07:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba23078640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c1566a0 con 0x7fba1c0acc00 2024-03-06T07:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba23078640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0c5b60 con 0x7fba1c1506b0 2024-03-06T07:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c156ea0 0x7fba1c152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c156ea0 0x7fba1c152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50196/0 (socket says 172.21.15.26:50196) 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 -- 172.21.15.26:0/1785146042 learned_addr learned my addr 172.21.15.26:0/1785146042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba20e14640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba1c0acc00 0x7fba1c0c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba1bfff640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c1506b0 0x7fba1c0c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 -- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c1506b0 msgr2=0x7fba1c0c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c1506b0 0x7fba1c0c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 -- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba1c0acc00 msgr2=0x7fba1c0c2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba1c0acc00 0x7fba1c0c2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.120+0000 7fba21615640 1 -- 172.21.15.26:0/1785146042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c155e80 con 0x7fba1c156ea0 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.121+0000 7fba21615640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c156ea0 0x7fba1c152430 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fba1400ba10 tx=0x7fba1400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.121+0000 7fba197fa640 1 -- 172.21.15.26:0/1785146042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14019070 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.121+0000 7fba197fa640 1 -- 172.21.15.26:0/1785146042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1400cae0 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.121+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c0c4040 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.121+0000 7fba197fa640 1 -- 172.21.15.26:0/1785146042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14014440 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.121+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c152970 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.122+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c0c8230 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.122+0000 7fba197fa640 1 -- 172.21.15.26:0/1785146042 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba1401c070 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.123+0000 7fba197fa640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9fc06b500 0x7fb9fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.123+0000 7fba197fa640 1 -- 172.21.15.26:0/1785146042 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba14089f20 con 0x7fba1c156ea0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.123+0000 7fba20e14640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9fc06b500 0x7fb9fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.123+0000 7fba20e14640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9fc06b500 0x7fb9fc06d9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fba0c036000 tx=0x7fba0c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.127+0000 7fba197fa640 1 -- 172.21.15.26:0/1785146042 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fba14058ca0 con 0x7fba1c156ea0 2024-03-06T07:25:31.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.344+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba1c04f330 con 0x7fba1c156ea0 2024-03-06T07:25:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.344+0000 7fba197fa640 1 -- 172.21.15.26:0/1785146042 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4993 (secure 0 0 0) 0x7fba140585a0 con 0x7fba1c156ea0 2024-03-06T07:25:31.346 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:31.346 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":121,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1667664264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1667664264},{"type":"v1","addr":"172.21.15.117:6836","nonce":1667664264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":7840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:22.149645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8295":{"gid":8295,"name":"a","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1099020571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1099020571},{"type":"v1","addr":"172.21.15.26:6835","nonce":1099020571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8295,"qdb_cluster":[8295]},"id":10}]} 2024-03-06T07:25:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 121 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9fc06b500 msgr2=0x7fb9fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9fc06b500 0x7fb9fc06d9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fba0c036000 tx=0x7fba0c03c040 comp rx=0 tx=0).stop 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c156ea0 msgr2=0x7fba1c152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c156ea0 0x7fba1c152430 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fba1400ba10 tx=0x7fba1400bed0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 shutdown_connections 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9fc06b500 0x7fb9fc06d9c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba1c156ea0 0x7fba1c152430 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba1c1506b0 0x7fba1c0c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 --2- 172.21.15.26:0/1785146042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba1c0acc00 0x7fba1c0c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 >> 172.21.15.26:0/1785146042 conn(0x7fba1c06b1a0 msgr2=0x7fba1c0acfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 shutdown_connections 2024-03-06T07:25:31.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.346+0000 7fba23078640 1 -- 172.21.15.26:0/1785146042 wait complete. 2024-03-06T07:25:31.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:20.149307+0000', 'modified': '2024-03-06T07:25:22.149645+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8295': {'gid': 8295, 'name': 'a', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1099020571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1099020571}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1099020571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8295, 'qdb_cluster': [8295]} 2024-03-06T07:25:31.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:31.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:25:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.862+0000 7fe7c844e640 1 Processor -- start 2024-03-06T07:25:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.862+0000 7fe7c844e640 1 -- start start 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c844e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c844e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7c01558d0 0x7fe7c0155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c844e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c0155130 con 0x7fe7c00c8950 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c844e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c0155440 con 0x7fe7c01558d0 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c844e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00ad480 con 0x7fe7c00c69f0 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c61ea640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c69eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7c01558d0 0x7fe7c0155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c59e9640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe7c00c8950 0x7fe7c00acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:58642/0 (socket says 172.21.15.26:58642) 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c59e9640 1 -- 172.21.15.26:0/439191472 learned_addr learned my addr 172.21.15.26:0/439191472 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c51e8640 1 -- 172.21.15.26:0/439191472 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815619015 0 0) 0x7fe7b0001110 con 0x7fe7c00c8950 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c51e8640 1 -- 172.21.15.26:0/439191472 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ac003520 con 0x7fe7c00c8950 2024-03-06T07:25:31.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c61ea640 1 -- 172.21.15.26:0/439191472 >> v2:172.21.15.117:3300/0 conn(0x7fe7c01558d0 msgr2=0x7fe7c0155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/439191472 >> v2:172.21.15.117:3300/0 conn(0x7fe7c01558d0 0x7fe7c0155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c61ea640 1 -- 172.21.15.26:0/439191472 >> v1:172.21.15.117:6789/0 conn(0x7fe7c00c8950 legacy=0x7fe7c00acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c61ea640 1 -- 172.21.15.26:0/439191472 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c00c5d50 con 0x7fe7c00c69f0 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.863+0000 7fe7c69eb640 1 --2- 172.21.15.26:0/439191472 >> v2:172.21.15.117:3300/0 conn(0x7fe7c01558d0 0x7fe7c0155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.864+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/439191472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c00c6dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0097a0 tx=0x7fe7bc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f8483fbcbf10ab server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.864+0000 7fe7c51e8640 1 -- 172.21.15.26:0/439191472 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0315a0 con 0x7fe7c00c69f0 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.864+0000 7fe7c51e8640 1 -- 172.21.15.26:0/439191472 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc037070 con 0x7fe7c00c69f0 2024-03-06T07:25:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.864+0000 7fe7c51e8640 1 -- 172.21.15.26:0/439191472 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc031960 con 0x7fe7c00c69f0 2024-03-06T07:25:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 -- 172.21.15.26:0/439191472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 msgr2=0x7fe7c00c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 --2- 172.21.15.26:0/439191472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c00c6dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0097a0 tx=0x7fe7bc02b840 comp rx=0 tx=0).stop 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 -- 172.21.15.26:0/439191472 shutdown_connections 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 --2- 172.21.15.26:0/439191472 >> v2:172.21.15.117:3300/0 conn(0x7fe7c01558d0 0x7fe7c0155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 --2- 172.21.15.26:0/439191472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c00c6dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 -- 172.21.15.26:0/439191472 >> 172.21.15.26:0/439191472 conn(0x7fe7c00cae70 msgr2=0x7fe7c00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 -- 172.21.15.26:0/439191472 shutdown_connections 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.865+0000 7fe7c844e640 1 -- 172.21.15.26:0/439191472 wait complete. 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.866+0000 7fe7c844e640 1 Processor -- start 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.866+0000 7fe7c844e640 1 -- start start 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c844e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c0158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c844e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00c8950 0x7fe7c014c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c844e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c01558d0 0x7fe7c00b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c844e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0152a20 con 0x7fe7c00c8950 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c844e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00c6310 con 0x7fe7c01558d0 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c844e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0152ba0 con 0x7fe7c00c69f0 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00c8950 0x7fe7c014c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00c8950 0x7fe7c014c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50210/0 (socket says 172.21.15.26:50210) 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 -- 172.21.15.26:0/273581707 learned_addr learned my addr 172.21.15.26:0/273581707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c69eb640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c01558d0 0x7fe7c00b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 -- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 msgr2=0x7fe7c0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c0158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 -- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c01558d0 msgr2=0x7fe7c00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c01558d0 0x7fe7c00b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 -- 172.21.15.26:0/273581707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c014ce10 con 0x7fe7c00c8950 2024-03-06T07:25:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.867+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00c8950 0x7fe7c014c8d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c3a0 tx=0x7fe7b000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.868+0000 7fe7b6ffd640 1 -- 172.21.15.26:0/273581707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0017070 con 0x7fe7c00c8950 2024-03-06T07:25:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.868+0000 7fe7b6ffd640 1 -- 172.21.15.26:0/273581707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0005ce0 con 0x7fe7c00c8950 2024-03-06T07:25:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.868+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c014c630 con 0x7fe7c00c8950 2024-03-06T07:25:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.868+0000 7fe7b6ffd640 1 -- 172.21.15.26:0/273581707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0007310 con 0x7fe7c00c8950 2024-03-06T07:25:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.868+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c00b09c0 con 0x7fe7c00c8950 2024-03-06T07:25:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.869+0000 7fe7b6ffd640 1 -- 172.21.15.26:0/273581707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7b001a070 con 0x7fe7c00c8950 2024-03-06T07:25:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.869+0000 7fe7b6ffd640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7a406b4d0 0x7fe7a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.870+0000 7fe7b6ffd640 1 -- 172.21.15.26:0/273581707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7b0089690 con 0x7fe7c00c8950 2024-03-06T07:25:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.870+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7a406b4d0 0x7fe7a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.870+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7a406b4d0 0x7fe7a406d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc036000 tx=0x7fe7bc03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 Processor -- start 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 -- start start 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f14f50 0x55f9c1f12a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f15330 0x55f9c1f12f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f134b0 0x55f9c1f0f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1e796b0 con 0x55f9c1f15330 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1e22a60 con 0x55f9c1f14f50 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.871+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1e75750 con 0x55f9c1f134b0 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f15330 0x55f9c1f12f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f15330 0x55f9c1f12f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50216/0 (socket says 172.21.15.26:50216) 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 -- 172.21.15.26:0/939705042 learned_addr learned my addr 172.21.15.26:0/939705042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f14f50 0x55f9c1f12a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c69eb640 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f134b0 0x55f9c1f0f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 -- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f134b0 msgr2=0x55f9c1f0f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f134b0 0x55f9c1f0f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 -- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f14f50 msgr2=0x55f9c1f12a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f14f50 0x55f9c1f12a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 -- 172.21.15.26:0/939705042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9c1e40f30 con 0x55f9c1f15330 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f15330 0x55f9c1f12f70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0000f30 tx=0x7fe7b0057e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199cc4cca006262f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7a3fff640 1 -- 172.21.15.26:0/939705042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b005e070 con 0x55f9c1f15330 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7a3fff640 1 -- 172.21.15.26:0/939705042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0055dc0 con 0x55f9c1f15330 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.872+0000 7fe7a3fff640 1 -- 172.21.15.26:0/939705042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0054390 con 0x55f9c1f15330 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 -- 172.21.15.26:0/939705042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f15330 msgr2=0x55f9c1f12f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f15330 0x55f9c1f12f70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0000f30 tx=0x7fe7b0057e60 comp rx=0 tx=0).stop 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 -- 172.21.15.26:0/939705042 shutdown_connections 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f134b0 0x55f9c1f0f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f15330 0x55f9c1f12f70 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 --2- 172.21.15.26:0/939705042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f14f50 0x55f9c1f12a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 -- 172.21.15.26:0/939705042 >> 172.21.15.26:0/939705042 conn(0x55f9c1f117d0 msgr2=0x55f9c1f363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 -- 172.21.15.26:0/939705042 shutdown_connections 2024-03-06T07:25:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.873+0000 7fe7cb036740 1 -- 172.21.15.26:0/939705042 wait complete. 2024-03-06T07:25:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.874+0000 7fe7cb036740 1 -- start start 2024-03-06T07:25:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.874+0000 7fe7cb036740 1 Processor -- start 2024-03-06T07:25:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f14f50 0x55f9c1f65720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f15330 0x55f9c1f35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f37600 0x55f9c1f37a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7cb036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1f37230 con 0x55f9c1f37600 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1e35ea0 con 0x55f9c1f15330 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1f36c70 con 0x55f9c1f14f50 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f37600 0x55f9c1f37a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7cb036740 10 client.0 _ll_register_callbacks cb 0x55f9c1e35870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f37600 0x55f9c1f37a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50218/0 (socket says 172.21.15.26:50218) 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 -- 172.21.15.26:0/3162500405 learned_addr learned my addr 172.21.15.26:0/3162500405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f14f50 0x55f9c1f65720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f15330 0x55f9c1f35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 -- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f14f50 msgr2=0x55f9c1f65720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f14f50 0x55f9c1f65720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 -- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f15330 msgr2=0x55f9c1f35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f15330 0x55f9c1f35c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.875+0000 7fe7c69eb640 1 -- 172.21.15.26:0/3162500405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9c1e36270 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7c69eb640 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f37600 0x55f9c1f37a60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8014da0 tx=0x7fe7b8012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf56d743df456ad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3162500405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b801e620 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7b0055dc0 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3162500405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7b801e7c0 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3162500405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b801ea10 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 10 client.8511 fetch_fsmap learned FSMap version 121 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55f9c1b8e2d0 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3162500405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0055dc0 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3162500405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 121) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fe7b802b080 con 0x55f9c1f37600 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 10 client.8511 fetch_fsmap finished waiting for FSMap version 121 2024-03-06T07:25:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 10 client.8511 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 20 client.8511 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f62210 0x55f9c1f703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 4 client.8511 mds_command: new command op to 8295 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.876+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 --> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9c1e86bf0 con 0x55f9c1f62210 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.877+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f62210 0x55f9c1f703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.877+0000 7fe7a17fa640 10 client.8511 ms_handle_connect on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.877+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f62210 0x55f9c1f703b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5055e173ab329ee3 server_cookie=56d1d0c2f67b8105 in_seq=0 out_seq=0 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.877+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3162500405 <== mds.0 v2:172.21.15.26:6834/1099020571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f9c1e86bf0 con 0x55f9c1f62210 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.877+0000 7fe7a17fa640 10 client.8511 handle_command_reply: tid=0 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.877+0000 7fe7cb036740 1 client.8511 shutdown 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f37600 msgr2=0x55f9c1f37a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f37600 0x55f9c1f37a60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8014da0 tx=0x7fe7b8012670 comp rx=0 tx=0).stop 2024-03-06T07:25:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 shutdown_connections 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f62210 0x55f9c1f703b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f37600 0x55f9c1f37a60 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f15330 0x55f9c1f35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7a17fa640 0 client.8511 ms_handle_reset on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3162500405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f14f50 0x55f9c1f65720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 >> 172.21.15.26:0/3162500405 conn(0x55f9c1f134b0 msgr2=0x55f9c1f138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 shutdown_connections 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 1 -- 172.21.15.26:0/3162500405 wait complete. 2024-03-06T07:25:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.878+0000 7fe7cb036740 20 client.8511 trim_cache size 0 max 16384 2024-03-06T07:25:31.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.895+0000 7fe7cb036740 1 Processor -- start 2024-03-06T07:25:31.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.895+0000 7fe7cb036740 1 -- start start 2024-03-06T07:25:31.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 0x55f9c1f34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 0x55f9c1f35430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7cb036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c01f6e30 con 0x55f9c1f14f50 2024-03-06T07:25:31.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c01eed00 con 0x55f9c1f37600 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c01f6830 con 0x55f9c1f15330 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f0e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50224/0 (socket says 172.21.15.26:50224) 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 -- 172.21.15.26:0/1048862371 learned_addr learned my addr 172.21.15.26:0/1048862371 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 -- 172.21.15.26:0/1048862371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 msgr2=0x55f9c1f34ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/1048862371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 0x55f9c1f34ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 -- 172.21.15.26:0/1048862371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 msgr2=0x55f9c1f35430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/1048862371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 0x55f9c1f35430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 -- 172.21.15.26:0/1048862371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9c1f35fe0 con 0x55f9c1f14f50 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.896+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/1048862371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f0e510 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc034040 tx=0x7fe7bc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c7c7789ad3b2f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7a37fe640 1 -- 172.21.15.26:0/1048862371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc05a070 con 0x55f9c1f14f50 2024-03-06T07:25:31.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7a37fe640 1 -- 172.21.15.26:0/1048862371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc037040 con 0x55f9c1f14f50 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7a37fe640 1 -- 172.21.15.26:0/1048862371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0552d0 con 0x55f9c1f14f50 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 -- 172.21.15.26:0/1048862371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 msgr2=0x55f9c1f0e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 --2- 172.21.15.26:0/1048862371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f0e510 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc034040 tx=0x7fe7bc030000 comp rx=0 tx=0).stop 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 -- 172.21.15.26:0/1048862371 shutdown_connections 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 --2- 172.21.15.26:0/1048862371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 0x55f9c1f35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 --2- 172.21.15.26:0/1048862371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 0x55f9c1f34ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 --2- 172.21.15.26:0/1048862371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f0e510 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 -- 172.21.15.26:0/1048862371 >> 172.21.15.26:0/1048862371 conn(0x55f9c1f134b0 msgr2=0x55f9c1f16070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 -- 172.21.15.26:0/1048862371 shutdown_connections 2024-03-06T07:25:31.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.897+0000 7fe7cb036740 1 -- 172.21.15.26:0/1048862371 wait complete. 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.898+0000 7fe7cb036740 1 -- start start 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.898+0000 7fe7cb036740 1 Processor -- start 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f6b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 0x55f9c1f5ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 0x55f9c1fbf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1f399a0 con 0x55f9c1f14f50 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1f39340 con 0x55f9c1f37600 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9c1f361b0 con 0x55f9c1f15330 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 10 client.0 _ll_register_callbacks cb 0x7fe7b80149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f6b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f6b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50230/0 (socket says 172.21.15.26:50230) 2024-03-06T07:25:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 -- 172.21.15.26:0/3157419184 learned_addr learned my addr 172.21.15.26:0/3157419184 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 -- 172.21.15.26:0/3157419184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 msgr2=0x55f9c1f5ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 0x55f9c1f5ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 -- 172.21.15.26:0/3157419184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 msgr2=0x55f9c1fbf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 0x55f9c1fbf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 -- 172.21.15.26:0/3157419184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9c1c586b0 con 0x55f9c1f14f50 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7c61ea640 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f6b6b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc058040 tx=0x7fe7bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a098fbceb519ffe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3157419184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc05a040 con 0x55f9c1f14f50 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3157419184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7bc061c60 con 0x55f9c1f14f50 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7bc037040 con 0x55f9c1f14f50 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3157419184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7bc061e60 con 0x55f9c1f14f50 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 10 client.8517 fetch_fsmap learned FSMap version 121 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.900+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55f9c1f4dff0 con 0x55f9c1f14f50 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3157419184 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc037040 con 0x55f9c1f14f50 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3157419184 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 121) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fe7bc031070 con 0x55f9c1f14f50 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 10 client.8517 fetch_fsmap finished waiting for FSMap version 121 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 10 client.8517 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 20 client.8517 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f743d0 0x55f9c1f747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 4 client.8517 mds_command: new command op to 8295 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 --> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f9c1f62960 con 0x55f9c1f743d0 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f743d0 0x55f9c1f747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7a17fa640 10 client.8517 ms_handle_connect on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7c59e9640 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f743d0 0x55f9c1f747b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7d69b47d5a59ca server_cookie=ae8b44c1b873a4a9 in_seq=0 out_seq=0 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7a17fa640 1 -- 172.21.15.26:0/3157419184 <== mds.0 v2:172.21.15.26:6834/1099020571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55f9c1f62960 con 0x55f9c1f743d0 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7a17fa640 10 client.8517 handle_command_reply: tid=0 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 1 client.8517 shutdown 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 msgr2=0x55f9c1f6b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.901+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f6b6b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc058040 tx=0x7fe7bc004730 comp rx=0 tx=0).stop 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 shutdown_connections 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:6834/1099020571,v1:172.21.15.26:6835/1099020571] conn(0x55f9c1f743d0 0x55f9c1f747b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9c1f37600 0x55f9c1fbf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9c1f15330 0x55f9c1f5ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7a17fa640 0 client.8517 ms_handle_reset on v2:172.21.15.26:6834/1099020571 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 --2- 172.21.15.26:0/3157419184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9c1f14f50 0x55f9c1f6b6b0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 >> 172.21.15.26:0/3157419184 conn(0x55f9c1f134b0 msgr2=0x55f9c1f53430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 shutdown_connections 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 1 -- 172.21.15.26:0/3157419184 wait complete. 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7cb036740 20 client.8517 trim_cache size 0 max 16384 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8295, 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.121408454999999, 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 121, 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 135, 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 135, 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 16.205238207000001 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7a406b4d0 msgr2=0x7fe7a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7a406b4d0 0x7fe7a406d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc036000 tx=0x7fe7bc03d040 comp rx=0 tx=0).stop 2024-03-06T07:25:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00c8950 msgr2=0x7fe7c014c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00c8950 0x7fe7c014c8d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c3a0 tx=0x7fe7b000c7a0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 shutdown_connections 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7a406b4d0 0x7fe7a406d990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c01558d0 0x7fe7c00b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00c8950 0x7fe7c014c8d0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 --2- 172.21.15.26:0/273581707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7c00c69f0 0x7fe7c0158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 >> 172.21.15.26:0/273581707 conn(0x7fe7c00cae70 msgr2=0x7fe7c0150240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 shutdown_connections 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:31.902+0000 7fe7c844e640 1 -- 172.21.15.26:0/273581707 wait complete. 2024-03-06T07:25:31.914 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 121, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:20.149307+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 121, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8295": { "addr": "172.21.15.26:6835/1099020571", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 1099020571, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 1099020571, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8295, "incarnation": 119, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:22.149645+0000", "qdb_cluster": [ 8295 ], "qdb_leader": 8295, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8295 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/1667664264", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1667664264, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 1667664264, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6848, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/3380233594", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3380233594, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3380233594, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7840, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6833/2142785555", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 2142785555, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 2142785555, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8292, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:31.915 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 10, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 10, 'name': 'd'}] 2024-03-06T07:25:31.915 INFO:tasks.cephfs.test_failover:current = [{'gid': 8295, 'name': 'a', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1099020571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1099020571}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1099020571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6848, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/1667664264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1667664264}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1667664264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}, {'gid': 8292, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/2142785555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2142785555}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 2142785555}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}, {'gid': 7840, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/3380233594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3380233594}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3380233594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}] 2024-03-06T07:25:31.916 DEBUG:tasks.cephfs.test_failover:10 != -1 2024-03-06T07:25:31.916 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:25:32.011 INFO:tasks.ceph.mds.a.smithi026.stderr: -1535> 2024-03-06T07:25:15.695+0000 7f041f904640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-03-06T07:25:32.013 INFO:tasks.ceph.mds.a.smithi026.stderr: -264> 2024-03-06T07:25:26.069+0000 7f04198f8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:32.039 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:25:36.071 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:25:36.069+0000 7fa5179e5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:36.917 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.029+0000 7fb30e513640 1 Processor -- start 2024-03-06T07:25:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.029+0000 7fb30e513640 1 -- start start 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb30e513640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb308152920 0x7fb308155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb30e513640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb3080c8a00 0x7fb308158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb30e513640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb308159460 0x7fb3080c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb30e513640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3081587b0 con 0x7fb308152920 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb30e513640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb308156180 con 0x7fb3080c8a00 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb30e513640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3080c7760 con 0x7fb308159460 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb307fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb308152920 0x7fb308155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb307fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb308152920 0x7fb308155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50262/0 (socket says 172.21.15.26:50262) 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb307fff640 1 -- 172.21.15.26:0/4007442723 learned_addr learned my addr 172.21.15.26:0/4007442723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb3077fe640 1 --2- 172.21.15.26:0/4007442723 >> v2:172.21.15.117:3300/0 conn(0x7fb3080c8a00 0x7fb308158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb30cab0640 1 --2- 172.21.15.26:0/4007442723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb308159460 0x7fb3080c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb307fff640 1 -- 172.21.15.26:0/4007442723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb308159460 msgr2=0x7fb3080c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb307fff640 1 --2- 172.21.15.26:0/4007442723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb308159460 0x7fb3080c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb307fff640 1 -- 172.21.15.26:0/4007442723 >> v2:172.21.15.117:3300/0 conn(0x7fb3080c8a00 msgr2=0x7fb308158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.030+0000 7fb307fff640 1 --2- 172.21.15.26:0/4007442723 >> v2:172.21.15.117:3300/0 conn(0x7fb3080c8a00 0x7fb308158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.031+0000 7fb307fff640 1 -- 172.21.15.26:0/4007442723 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3080c7a40 con 0x7fb308152920 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.031+0000 7fb307fff640 1 --2- 172.21.15.26:0/4007442723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb308155c40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb308067c30 tx=0x7fb2f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee977b547c1dfb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.031+0000 7fb306ffd640 1 -- 172.21.15.26:0/4007442723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f802e070 con 0x7fb308152920 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.031+0000 7fb306ffd640 1 -- 172.21.15.26:0/4007442723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f8037070 con 0x7fb308152920 2024-03-06T07:25:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.031+0000 7fb306ffd640 1 -- 172.21.15.26:0/4007442723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8031a00 con 0x7fb308152920 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 -- 172.21.15.26:0/4007442723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 msgr2=0x7fb308155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 --2- 172.21.15.26:0/4007442723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb308155c40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb308067c30 tx=0x7fb2f802b880 comp rx=0 tx=0).stop 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 -- 172.21.15.26:0/4007442723 shutdown_connections 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 --2- 172.21.15.26:0/4007442723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb308159460 0x7fb3080c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 --2- 172.21.15.26:0/4007442723 >> v2:172.21.15.117:3300/0 conn(0x7fb3080c8a00 0x7fb308158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 --2- 172.21.15.26:0/4007442723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb308155c40 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 -- 172.21.15.26:0/4007442723 >> 172.21.15.26:0/4007442723 conn(0x7fb3080ac560 msgr2=0x7fb3080ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 -- 172.21.15.26:0/4007442723 shutdown_connections 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.032+0000 7fb30e513640 1 -- 172.21.15.26:0/4007442723 wait complete. 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.033+0000 7fb30e513640 1 Processor -- start 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.033+0000 7fb30e513640 1 -- start start 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb30e513640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3080c8a00 0x7fb3080c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb30e513640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb3081eb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb30e513640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3080c4650 0x7fb3081eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb30e513640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30806d030 con 0x7fb308152920 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb30e513640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb308158a00 con 0x7fb3080c8a00 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb30e513640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3080c7390 con 0x7fb3080c4650 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb3077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb3081eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb307fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3080c8a00 0x7fb3080c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb3077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb3081eb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50264/0 (socket says 172.21.15.26:50264) 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb3077fe640 1 -- 172.21.15.26:0/3486955804 learned_addr learned my addr 172.21.15.26:0/3486955804 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.034+0000 7fb30cab0640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3080c4650 0x7fb3081eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb3077fe640 1 -- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3080c4650 msgr2=0x7fb3081eb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb3077fe640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3080c4650 0x7fb3081eb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb3077fe640 1 -- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3080c8a00 msgr2=0x7fb3080c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb3077fe640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3080c8a00 0x7fb3080c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb3077fe640 1 -- 172.21.15.26:0/3486955804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3081ef120 con 0x7fb308152920 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb307fff640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3080c8a00 0x7fb3080c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb3077fe640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb3081eb390 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000e870 tx=0x7fb2f000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb304ff9640 1 -- 172.21.15.26:0/3486955804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0017070 con 0x7fb308152920 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb304ff9640 1 -- 172.21.15.26:0/3486955804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f000ee60 con 0x7fb308152920 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb304ff9640 1 -- 172.21.15.26:0/3486955804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000cb20 con 0x7fb308152920 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb308155bb0 con 0x7fb308152920 2024-03-06T07:25:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.035+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30814cbc0 con 0x7fb308152920 2024-03-06T07:25:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.036+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30804f330 con 0x7fb308152920 2024-03-06T07:25:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.037+0000 7fb304ff9640 1 -- 172.21.15.26:0/3486955804 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2f001a070 con 0x7fb308152920 2024-03-06T07:25:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.037+0000 7fb304ff9640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2e806b500 0x7fb2e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.037+0000 7fb304ff9640 1 -- 172.21.15.26:0/3486955804 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb2f008a430 con 0x7fb308152920 2024-03-06T07:25:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.037+0000 7fb307fff640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2e806b500 0x7fb2e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.038+0000 7fb307fff640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2e806b500 0x7fb2e806d9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8036000 tx=0x7fb2f803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.041+0000 7fb304ff9640 1 -- 172.21.15.26:0/3486955804 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb2f0058a80 con 0x7fb308152920 2024-03-06T07:25:37.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.260+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3080be1f0 con 0x7fb308152920 2024-03-06T07:25:37.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.260+0000 7fb304ff9640 1 -- 172.21.15.26:0/3486955804 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4995 (secure 0 0 0) 0x7fb2f005cb20 con 0x7fb308152920 2024-03-06T07:25:37.262 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:37.262 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":126,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:35.012165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6848":{"gid":6848,"name":"b","rank":0,"incarnation":123,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/1667664264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1667664264},{"type":"v1","addr":"172.21.15.117:6836","nonce":1667664264}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6848,"qdb_cluster":[6848]},"id":10}]} 2024-03-06T07:25:37.262 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 126 2024-03-06T07:25:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2e806b500 msgr2=0x7fb2e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2e806b500 0x7fb2e806d9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8036000 tx=0x7fb2f803d040 comp rx=0 tx=0).stop 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 msgr2=0x7fb3081eb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb3081eb390 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000e870 tx=0x7fb2f000ed30 comp rx=0 tx=0).stop 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 shutdown_connections 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2e806b500 0x7fb2e806d9c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3080c4650 0x7fb3081eb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb308152920 0x7fb3081eb390 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 --2- 172.21.15.26:0/3486955804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3080c8a00 0x7fb3080c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 >> 172.21.15.26:0/3486955804 conn(0x7fb3080ac560 msgr2=0x7fb308151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 shutdown_connections 2024-03-06T07:25:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.262+0000 7fb30e513640 1 -- 172.21.15.26:0/3486955804 wait complete. 2024-03-06T07:25:37.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:20.149307+0000', 'modified': '2024-03-06T07:25:35.012165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 136, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6848}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6848': {'gid': 6848, 'name': 'b', 'rank': 0, 'incarnation': 123, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.117:6836/1667664264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1667664264}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1667664264}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6848, 'qdb_cluster': [6848]} 2024-03-06T07:25:37.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:37.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:25:37.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.782+0000 7f203c21c640 1 Processor -- start 2024-03-06T07:25:37.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f203c21c640 1 -- start start 2024-03-06T07:25:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f203c21c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2034152720 0x7f20341593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f203c21c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20340c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f203c21c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20340733f0 con 0x7f20340c7d20 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f203c21c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2034155ca0 con 0x7f2034152720 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f203c21c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20341561d0 con 0x7f203406f550 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f2039fb8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2034152720 0x7f20341593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.783+0000 7f203a7b9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f20340c7d20 0x7f203406fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44654/0 (socket says 172.21.15.26:44654) 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.784+0000 7f203a7b9640 1 -- 172.21.15.26:0/3084335938 learned_addr learned my addr 172.21.15.26:0/3084335938 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.784+0000 7f20397b7640 1 --2- 172.21.15.26:0/3084335938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20340c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.784+0000 7f2039fb8640 1 -- 172.21.15.26:0/3084335938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 msgr2=0x7f20340c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.784+0000 7f2039fb8640 1 --2- 172.21.15.26:0/3084335938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20340c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.784+0000 7f2039fb8640 1 -- 172.21.15.26:0/3084335938 >> v1:172.21.15.26:6789/0 conn(0x7f20340c7d20 legacy=0x7f203406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.784+0000 7f2039fb8640 1 -- 172.21.15.26:0/3084335938 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034155ef0 con 0x7f2034152720 2024-03-06T07:25:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f2039fb8640 1 --2- 172.21.15.26:0/3084335938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2034152720 0x7f20341593c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f202c0097a0 tx=0x7f202c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8742b7afbd4a3d84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f2038fb6640 1 -- 172.21.15.26:0/3084335938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0315a0 con 0x7f2034152720 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f2038fb6640 1 -- 172.21.15.26:0/3084335938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c037070 con 0x7f2034152720 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f2038fb6640 1 -- 172.21.15.26:0/3084335938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c031960 con 0x7f2034152720 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 -- 172.21.15.26:0/3084335938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2034152720 msgr2=0x7f20341593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 --2- 172.21.15.26:0/3084335938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2034152720 0x7f20341593c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f202c0097a0 tx=0x7f202c02b840 comp rx=0 tx=0).stop 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 -- 172.21.15.26:0/3084335938 shutdown_connections 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 --2- 172.21.15.26:0/3084335938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20340c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 --2- 172.21.15.26:0/3084335938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2034152720 0x7f20341593c0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 -- 172.21.15.26:0/3084335938 >> 172.21.15.26:0/3084335938 conn(0x7f203406ea40 msgr2=0x7f203406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 -- 172.21.15.26:0/3084335938 shutdown_connections 2024-03-06T07:25:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.785+0000 7f203c21c640 1 -- 172.21.15.26:0/3084335938 wait complete. 2024-03-06T07:25:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.786+0000 7f203c21c640 1 Processor -- start 2024-03-06T07:25:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.786+0000 7f203c21c640 1 -- start start 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f203c21c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20341ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f203c21c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20340c7d20 0x7f203414ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f203c21c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2034152720 0x7f203406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f203c21c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034158ea0 con 0x7f20340c7d20 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f203c21c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034158bc0 con 0x7f2034152720 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f203c21c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340704a0 con 0x7f203406f550 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f20397b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20340c7d20 0x7f203414ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f2039fb8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20341ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f20397b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20340c7d20 0x7f203414ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57860/0 (socket says 172.21.15.26:57860) 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f20397b7640 1 -- 172.21.15.26:0/1378093186 learned_addr learned my addr 172.21.15.26:0/1378093186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.787+0000 7f203a7b9640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2034152720 0x7f203406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20397b7640 1 -- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 msgr2=0x7f20341ee5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20397b7640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20341ee5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20397b7640 1 -- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2034152720 msgr2=0x7f203406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20397b7640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2034152720 0x7f203406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20397b7640 1 -- 172.21.15.26:0/1378093186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203406d5a0 con 0x7f20340c7d20 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f2039fb8640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20341ee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20397b7640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20340c7d20 0x7f203414ae70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f202800e980 tx=0x7f202800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20327fc640 1 -- 172.21.15.26:0/1378093186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028017070 con 0x7f20340c7d20 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20327fc640 1 -- 172.21.15.26:0/1378093186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202800ca00 con 0x7f20340c7d20 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203406d880 con 0x7f20340c7d20 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f20327fc640 1 -- 172.21.15.26:0/1378093186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800cdd0 con 0x7f20340c7d20 2024-03-06T07:25:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.788+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203406de40 con 0x7f20340c7d20 2024-03-06T07:25:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.790+0000 7f20327fc640 1 -- 172.21.15.26:0/1378093186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f202801a070 con 0x7f20340c7d20 2024-03-06T07:25:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.790+0000 7f20327fc640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f202006b4d0 0x7f202006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.790+0000 7f20327fc640 1 -- 172.21.15.26:0/1378093186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f202808a660 con 0x7f20340c7d20 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.790+0000 7f2039fb8640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f202006b4d0 0x7f202006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.791+0000 7f2039fb8640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f202006b4d0 0x7f202006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f202c036000 tx=0x7f202c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.791+0000 7f203ee0a740 1 Processor -- start 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.791+0000 7f203ee0a740 1 -- start start 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead363a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead363f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead3644b0 0x55aead360680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203ee0a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aead2ca6b0 con 0x55aead3644b0 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aead273a60 con 0x55aead365f50 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aead2c6750 con 0x55aead366330 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203a7b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead3644b0 0x55aead360680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203a7b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead3644b0 0x55aead360680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57864/0 (socket says 172.21.15.26:57864) 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203a7b9640 1 -- 172.21.15.26:0/2438392197 learned_addr learned my addr 172.21.15.26:0/2438392197 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead363a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f20397b7640 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead363f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f2039fb8640 1 -- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 msgr2=0x55aead363f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead363f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f2039fb8640 1 -- 172.21.15.26:0/2438392197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead3644b0 msgr2=0x55aead360680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead3644b0 0x55aead360680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f2039fb8640 1 -- 172.21.15.26:0/2438392197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aead291f30 con 0x55aead365f50 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead3644b0 0x55aead360680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead363a30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f202c033000 tx=0x7f202c030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82fabedbac27cfd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f201b7fe640 1 -- 172.21.15.26:0/2438392197 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c032a40 con 0x55aead365f50 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f201b7fe640 1 -- 172.21.15.26:0/2438392197 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c037040 con 0x55aead365f50 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.792+0000 7f201b7fe640 1 -- 172.21.15.26:0/2438392197 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c032de0 con 0x55aead365f50 2024-03-06T07:25:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 -- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 msgr2=0x55aead363a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead363a30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f202c033000 tx=0x7f202c030000 comp rx=0 tx=0).stop 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 -- 172.21.15.26:0/2438392197 shutdown_connections 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead3644b0 0x55aead360680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead363f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2438392197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead363a30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 -- 172.21.15.26:0/2438392197 >> 172.21.15.26:0/2438392197 conn(0x55aead3627d0 msgr2=0x55aead3873c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 -- 172.21.15.26:0/2438392197 shutdown_connections 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.793+0000 7f203ee0a740 1 -- 172.21.15.26:0/2438392197 wait complete. 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.794+0000 7f203ee0a740 1 -- start start 2024-03-06T07:25:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.794+0000 7f203ee0a740 1 Processor -- start 2024-03-06T07:25:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead365f50 0x55aead3b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead38ba30 0x55aead39ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aead388230 con 0x55aead365f50 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aead286ea0 con 0x55aead366330 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aead387c70 con 0x55aead38ba30 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 10 client.0 _ll_register_callbacks cb 0x55aead286870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead365f50 0x55aead3b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead365f50 0x55aead3b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57878/0 (socket says 172.21.15.26:57878) 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 -- 172.21.15.26:0/2449912252 learned_addr learned my addr 172.21.15.26:0/2449912252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f20397b7640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead38ba30 0x55aead39ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 -- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead38ba30 msgr2=0x55aead39ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead38ba30 0x55aead39ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 -- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 msgr2=0x55aead3893b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3893b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 -- 172.21.15.26:0/2449912252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aead287270 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f20397b7640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead365f50 0x55aead3b6460 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f202c02be50 tx=0x7f202c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1456e198d68ed7be server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2018ff9640 1 -- 172.21.15.26:0/2449912252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c004380 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2018ff9640 1 -- 172.21.15.26:0/2449912252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f202c05acb0 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f202c037040 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f2018ff9640 1 -- 172.21.15.26:0/2449912252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f202c003610 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 10 client.8544 fetch_fsmap learned FSMap version 126 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.795+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x55aeacfdf2d0 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f2018ff9640 1 -- 172.21.15.26:0/2449912252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c037040 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f2018ff9640 1 -- 172.21.15.26:0/2449912252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 126) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f202c064080 con 0x55aead365f50 2024-03-06T07:25:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f203ee0a740 10 client.8544 fetch_fsmap finished waiting for FSMap version 126 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f203ee0a740 10 client.8544 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f203ee0a740 20 client.8544 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3b2de0 0x55aead3b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f203ee0a740 4 client.8544 mds_command: new command op to 6848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 --> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aead2d7bf0 con 0x55aead3b2de0 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f20397b7640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3b2de0 0x55aead3b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f2018ff9640 10 client.8544 ms_handle_connect on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.796+0000 7f20397b7640 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3b2de0 0x55aead3b31c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec83ca054e7cfad5 server_cookie=b00fa065639b0fa0 in_seq=0 out_seq=0 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f2018ff9640 1 -- 172.21.15.26:0/2449912252 <== mds.0 v2:172.21.15.117:6834/1667664264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aead2d7bf0 con 0x55aead3b2de0 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f2018ff9640 10 client.8544 handle_command_reply: tid=0 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 client.8544 shutdown 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead365f50 msgr2=0x55aead3b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead365f50 0x55aead3b6460 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f202c02be50 tx=0x7f202c004730 comp rx=0 tx=0).stop 2024-03-06T07:25:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 shutdown_connections 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3b2de0 0x55aead3b31c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead38ba30 0x55aead39ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2449912252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead365f50 0x55aead3b6460 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f2018ff9640 0 client.8544 ms_handle_reset on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.797+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 >> 172.21.15.26:0/2449912252 conn(0x55aead3644b0 msgr2=0x55aead35f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.798+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 shutdown_connections 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.798+0000 7f203ee0a740 1 -- 172.21.15.26:0/2449912252 wait complete. 2024-03-06T07:25:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.798+0000 7f203ee0a740 20 client.8544 trim_cache size 0 max 16384 2024-03-06T07:25:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.814+0000 7f203ee0a740 1 Processor -- start 2024-03-06T07:25:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203ee0a740 1 -- start start 2024-03-06T07:25:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead388a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead3617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead35f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203ee0a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20341eb400 con 0x55aead38ba30 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20341ea800 con 0x55aead365f50 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20341eae00 con 0x55aead366330 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203a7b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead35f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203a7b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead35f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57884/0 (socket says 172.21.15.26:57884) 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f203a7b9640 1 -- 172.21.15.26:0/2074396657 learned_addr learned my addr 172.21.15.26:0/2074396657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f20397b7640 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead3617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.815+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead388a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203a7b9640 1 -- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 msgr2=0x55aead3617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead3617b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203a7b9640 1 -- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 msgr2=0x55aead388a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead388a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203a7b9640 1 -- 172.21.15.26:0/2074396657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aead0a96b0 con 0x55aead38ba30 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead35f510 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2024015580 tx=0x7f2024001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e28c522d4e72e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f201affd640 1 -- 172.21.15.26:0/2074396657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024005df0 con 0x55aead38ba30 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f201affd640 1 -- 172.21.15.26:0/2074396657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20240164e0 con 0x55aead38ba30 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f201affd640 1 -- 172.21.15.26:0/2074396657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024016890 con 0x55aead38ba30 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 -- 172.21.15.26:0/2074396657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 msgr2=0x55aead35f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead35f510 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2024015580 tx=0x7f2024001940 comp rx=0 tx=0).stop 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 -- 172.21.15.26:0/2074396657 shutdown_connections 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead35f510 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead366330 0x55aead3617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2074396657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead365f50 0x55aead388a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 -- 172.21.15.26:0/2074396657 >> 172.21.15.26:0/2074396657 conn(0x55aead3644b0 msgr2=0x55aead3a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 -- 172.21.15.26:0/2074396657 shutdown_connections 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.816+0000 7f203ee0a740 1 -- 172.21.15.26:0/2074396657 wait complete. 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.817+0000 7f203ee0a740 1 -- start start 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.817+0000 7f203ee0a740 1 Processor -- start 2024-03-06T07:25:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead365f50 0x55aead35f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3a5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203ee0a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead3c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203ee0a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aead388230 con 0x55aead38ba30 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aead3b2c60 con 0x55aead366330 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203ee0a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aead38a5b0 con 0x55aead365f50 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203ee0a740 10 client.0 _ll_register_callbacks cb 0x7f202c031a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead3c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead3c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57886/0 (socket says 172.21.15.26:57886) 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 -- 172.21.15.26:0/2326900704 learned_addr learned my addr 172.21.15.26:0/2326900704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead365f50 0x55aead35f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f20397b7640 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 -- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead365f50 msgr2=0x55aead35f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead365f50 0x55aead35f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 -- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 msgr2=0x55aead3a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 -- 172.21.15.26:0/2326900704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aead3637e0 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f203a7b9640 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead3c5720 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2024009010 tx=0x7f2024001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc45b14355f489e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.818+0000 7f2018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f2018ff9640 1 -- 172.21.15.26:0/2326900704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024002040 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f2018ff9640 1 -- 172.21.15.26:0/2326900704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f202402bcb0 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20240164e0 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f2018ff9640 1 -- 172.21.15.26:0/2326900704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2024016bd0 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 10 client.8550 fetch_fsmap learned FSMap version 126 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x55aead3894f0 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f2018ff9640 1 -- 172.21.15.26:0/2326900704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20240164e0 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f2018ff9640 1 -- 172.21.15.26:0/2326900704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 126) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f20240070e0 con 0x55aead38ba30 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 10 client.8550 fetch_fsmap finished waiting for FSMap version 126 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 10 client.8550 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 20 client.8550 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3c6320 0x55aead40e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 4 client.8550 mds_command: new command op to 6848 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 --> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aead2d1f40 con 0x55aead3c6320 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.819+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3c6320 0x55aead40e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f2018ff9640 10 client.8550 ms_handle_connect on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f2039fb8640 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3c6320 0x55aead40e940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9adbdac344b3714 server_cookie=7d78b7d08af6de5d in_seq=0 out_seq=0 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f2018ff9640 1 -- 172.21.15.26:0/2326900704 <== mds.0 v2:172.21.15.117:6834/1667664264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55aead2d1f40 con 0x55aead3c6320 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f2018ff9640 10 client.8550 handle_command_reply: tid=0 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 client.8550 shutdown 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 msgr2=0x55aead3c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead3c5720 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2024009010 tx=0x7f2024001250 comp rx=0 tx=0).stop 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 shutdown_connections 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55aead3c6320 0x55aead40e940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aead38ba30 0x55aead3c5720 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aead366330 0x55aead3a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f2018ff9640 0 client.8550 ms_handle_reset on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 --2- 172.21.15.26:0/2326900704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aead365f50 0x55aead35f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 >> 172.21.15.26:0/2326900704 conn(0x55aead3644b0 msgr2=0x55aead3a3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 shutdown_connections 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 1 -- 172.21.15.26:0/2326900704 wait complete. 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.820+0000 7f203ee0a740 20 client.8550 trim_cache size 0 max 16384 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:25:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6848, 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.8088869220000001, 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 126, 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 136, 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 136, 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 22.112653758 2024-03-06T07:25:37.829 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f202006b4d0 msgr2=0x7f202006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f202006b4d0 0x7f202006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f202c036000 tx=0x7f202c03d040 comp rx=0 tx=0).stop 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20340c7d20 msgr2=0x7f203414ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20340c7d20 0x7f203414ae70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f202800e980 tx=0x7f202800ee40 comp rx=0 tx=0).stop 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 shutdown_connections 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f202006b4d0 0x7f202006d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2034152720 0x7f203406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20340c7d20 0x7f203414ae70 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 --2- 172.21.15.26:0/1378093186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203406f550 0x7f20341ee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 >> 172.21.15.26:0/1378093186 conn(0x7f203406ea40 msgr2=0x7f203406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 shutdown_connections 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:37.821+0000 7f203c21c640 1 -- 172.21.15.26:0/1378093186 wait complete. 2024-03-06T07:25:37.833 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 126, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:20.149307+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 126, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6848": { "addr": "172.21.15.117:6836/1667664264", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1667664264, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 1667664264, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6848, "incarnation": 123, "join_fscid": 10, "name": "b", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 136, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:35.012165+0000", "qdb_cluster": [ 6848 ], "qdb_leader": 6848, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6848 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/3380233594", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3380233594, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3380233594, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 122, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7840, "incarnation": 0, "join_fscid": 10, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 5 }, { "addr": "172.21.15.26:6833/2142785555", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 2142785555, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 2142785555, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8292, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2316269046", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2316269046, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2316269046, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 124, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8523, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:37.834 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 10, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 10, 'name': 'd'}] 2024-03-06T07:25:37.834 INFO:tasks.cephfs.test_failover:current = [{'gid': 8523, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2316269046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2316269046}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2316269046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 124}, {'gid': 6848, 'name': 'b', 'rank': 0, 'incarnation': 123, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.117:6836/1667664264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1667664264}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1667664264}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8292, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/2142785555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2142785555}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 2142785555}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}, {'gid': 7840, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 5, 'addr': '172.21.15.117:6837/3380233594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3380233594}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3380233594}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 122}] 2024-03-06T07:25:37.834 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:25:42.278 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:42.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:42.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:42.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.947+0000 7f2458d8a640 1 Processor -- start 2024-03-06T07:25:42.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.947+0000 7f2458d8a640 1 -- start start 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f2458d8a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24540c3280 0x7f24540c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f2458d8a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24541551e0 0x7f24541555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f2458d8a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2454158800 con 0x7f245406d9f0 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f2458d8a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2454158ae0 con 0x7f24540c3280 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f2458d8a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2454158dc0 con 0x7f24541551e0 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f244bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24540c3280 0x7f24540c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f245259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f245406d9f0 0x7f24540a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44656/0 (socket says 172.21.15.26:44656) 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f245259c640 1 -- 172.21.15.26:0/2604198285 learned_addr learned my addr 172.21.15.26:0/2604198285 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f2452d9d640 1 --2- 172.21.15.26:0/2604198285 >> v2:172.21.15.117:3300/0 conn(0x7f24541551e0 0x7f24541555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:42.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f244bfff640 1 -- 172.21.15.26:0/2604198285 >> v2:172.21.15.117:3300/0 conn(0x7f24541551e0 msgr2=0x7f24541555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:42.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f244bfff640 1 --2- 172.21.15.26:0/2604198285 >> v2:172.21.15.117:3300/0 conn(0x7f24541551e0 0x7f24541555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f244bfff640 1 -- 172.21.15.26:0/2604198285 >> v1:172.21.15.26:6789/0 conn(0x7f245406d9f0 legacy=0x7f24540a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:42.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f244bfff640 1 -- 172.21.15.26:0/2604198285 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24541590a0 con 0x7f24540c3280 2024-03-06T07:25:42.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.948+0000 7f2451d9b640 1 -- 172.21.15.26:0/2604198285 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639399375 0 0) 0x7f2454158800 con 0x7f245406d9f0 2024-03-06T07:25:42.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.949+0000 7f244bfff640 1 --2- 172.21.15.26:0/2604198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24540c3710 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2454072f50 tx=0x7f243c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faed5aa3fe01f2ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:42.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.949+0000 7f2451d9b640 1 -- 172.21.15.26:0/2604198285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0315a0 con 0x7f24540c3280 2024-03-06T07:25:42.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.949+0000 7f2451d9b640 1 -- 172.21.15.26:0/2604198285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c037070 con 0x7f24540c3280 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.949+0000 7f2451d9b640 1 -- 172.21.15.26:0/2604198285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c031960 con 0x7f24540c3280 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 -- 172.21.15.26:0/2604198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 msgr2=0x7f24540c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 --2- 172.21.15.26:0/2604198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24540c3710 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2454072f50 tx=0x7f243c02b840 comp rx=0 tx=0).stop 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 -- 172.21.15.26:0/2604198285 shutdown_connections 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 --2- 172.21.15.26:0/2604198285 >> v2:172.21.15.117:3300/0 conn(0x7f24541551e0 0x7f24541555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 --2- 172.21.15.26:0/2604198285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24540c3710 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 -- 172.21.15.26:0/2604198285 >> 172.21.15.26:0/2604198285 conn(0x7f24540aa630 msgr2=0x7f24540aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 -- 172.21.15.26:0/2604198285 shutdown_connections 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.950+0000 7f2458d8a640 1 -- 172.21.15.26:0/2604198285 wait complete. 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.951+0000 7f2458d8a640 1 Processor -- start 2024-03-06T07:25:42.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.951+0000 7f2458d8a640 1 -- start start 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f2458d8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f245406d9f0 0x7f24541ea710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f2458d8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24541eac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f2458d8a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24541551e0 0x7f24540ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f2458d8a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245406df50 con 0x7f24540c3280 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f2458d8a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245406ddd0 con 0x7f245406d9f0 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f2458d8a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24540c6910 con 0x7f24541551e0 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24541eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24541eac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57904/0 (socket says 172.21.15.26:57904) 2024-03-06T07:25:42.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 -- 172.21.15.26:0/278988414 learned_addr learned my addr 172.21.15.26:0/278988414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f245259c640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f245406d9f0 0x7f24541ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f2452d9d640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24541551e0 0x7f24540ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 -- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24541551e0 msgr2=0x7f24540ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24541551e0 0x7f24540ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 -- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f245406d9f0 msgr2=0x7f24541ea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f245406d9f0 0x7f24541ea710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 -- 172.21.15.26:0/278988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24540ab750 con 0x7f24540c3280 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244bfff640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24541eac50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f243c000c00 tx=0x7f243c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.952+0000 7f244affd640 1 -- 172.21.15.26:0/278988414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c032630 con 0x7f24540c3280 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.953+0000 7f244affd640 1 -- 172.21.15.26:0/278988414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c0327f0 con 0x7f24540c3280 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.953+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24540aba30 con 0x7f24540c3280 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.953+0000 7f244affd640 1 -- 172.21.15.26:0/278988414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c032be0 con 0x7f24540c3280 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.953+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24540701a0 con 0x7f24540c3280 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.954+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2454070510 con 0x7f24540c3280 2024-03-06T07:25:42.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.954+0000 7f244affd640 1 -- 172.21.15.26:0/278988414 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f243c04d3f0 con 0x7f24540c3280 2024-03-06T07:25:42.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.954+0000 7f244affd640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243406b500 0x7f243406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:42.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.954+0000 7f244affd640 1 -- 172.21.15.26:0/278988414 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f243c02f030 con 0x7f24540c3280 2024-03-06T07:25:42.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.955+0000 7f245259c640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243406b500 0x7f243406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:42.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.955+0000 7f245259c640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243406b500 0x7f243406d9c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2444000f60 tx=0x7f2444007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:42.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:42.959+0000 7f244affd640 1 -- 172.21.15.26:0/278988414 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f243c040070 con 0x7f24540c3280 2024-03-06T07:25:43.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.179+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f245404f330 con 0x7f24540c3280 2024-03-06T07:25:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.179+0000 7f244affd640 1 -- 172.21.15.26:0/278988414 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f243c07a420 con 0x7f24540c3280 2024-03-06T07:25:43.181 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:43.181 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":126,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:35.012165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6848":{"gid":6848,"name":"b","rank":0,"incarnation":123,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/1667664264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1667664264},{"type":"v1","addr":"172.21.15.117:6836","nonce":1667664264}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6848,"qdb_cluster":[6848]},"id":10}]} 2024-03-06T07:25:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 126 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243406b500 msgr2=0x7f243406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243406b500 0x7f243406d9c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2444000f60 tx=0x7f2444007d50 comp rx=0 tx=0).stop 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 msgr2=0x7f24541eac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24541eac50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f243c000c00 tx=0x7f243c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 shutdown_connections 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f243406b500 0x7f243406d9c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24541551e0 0x7f24540ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24540c3280 0x7f24541eac50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 --2- 172.21.15.26:0/278988414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f245406d9f0 0x7f24541ea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 >> 172.21.15.26:0/278988414 conn(0x7f24540aa630 msgr2=0x7f2454154880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 shutdown_connections 2024-03-06T07:25:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.181+0000 7f2458d8a640 1 -- 172.21.15.26:0/278988414 wait complete. 2024-03-06T07:25:43.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:20.149307+0000', 'modified': '2024-03-06T07:25:35.012165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 136, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6848}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6848': {'gid': 6848, 'name': 'b', 'rank': 0, 'incarnation': 123, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.117:6836/1667664264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1667664264}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1667664264}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6848, 'qdb_cluster': [6848]} 2024-03-06T07:25:43.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:43.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:25:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.702+0000 7ff4b1878640 1 Processor -- start 2024-03-06T07:25:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.702+0000 7ff4b1878640 1 -- start start 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4b1878640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff4ac152b40 0x7ff4ac0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4b1878640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ac0733f0 con 0x7ff4ac0724d0 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4b1878640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ac0c82c0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4b1878640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ac072d70 con 0x7ff4ac152b40 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4aa7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff4ac0724d0 0x7ff4ac1551f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44670/0 (socket says 172.21.15.26:44670) 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/739699976 learned_addr learned my addr 172.21.15.26:0/739699976 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/739699976 >> v2:172.21.15.117:3300/0 conn(0x7ff4ac152b40 0x7ff4ac0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264276074 0 0) 0x7ff4ac0733f0 con 0x7ff4ac0724d0 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.703+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff498003520 con 0x7ff4ac0724d0 2024-03-06T07:25:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.704+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494537178 0 0) 0x7ff4ac0c82c0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.704+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ac0733f0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.704+0000 7ff4aaffd640 1 -- 172.21.15.26:0/739699976 >> v1:172.21.15.117:6789/0 conn(0x7ff4ac0a7ff0 legacy=0x7ff4ac15fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.704+0000 7ff4aaffd640 1 -- 172.21.15.26:0/739699976 >> v1:172.21.15.26:6789/0 conn(0x7ff4ac0724d0 legacy=0x7ff4ac1551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.704+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915306394 0 0) 0x7ff498003520 con 0x7ff4ac0724d0 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.704+0000 7ff4aaffd640 1 -- 172.21.15.26:0/739699976 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac06fd30 con 0x7ff4ac152b40 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.704+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/739699976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 0x7ff4ac0c8810 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40097a0 tx=0x7ff4a402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb65a69b2fd8071 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4037070 con 0x7ff4ac152b40 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4030510 con 0x7ff4ac152b40 2024-03-06T07:25:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4a9ffb640 1 -- 172.21.15.26:0/739699976 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494001670 con 0x7ff4ac152b40 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4b1878640 1 -- 172.21.15.26:0/739699976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 msgr2=0x7ff4ac0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4b1878640 1 --2- 172.21.15.26:0/739699976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 0x7ff4ac0c8810 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40097a0 tx=0x7ff4a402b790 comp rx=0 tx=0).stop 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4b1878640 1 -- 172.21.15.26:0/739699976 shutdown_connections 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4b1878640 1 --2- 172.21.15.26:0/739699976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 0x7ff4ac0c8810 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4b1878640 1 -- 172.21.15.26:0/739699976 >> 172.21.15.26:0/739699976 conn(0x7ff4ac06b970 msgr2=0x7ff4ac0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4b1878640 1 -- 172.21.15.26:0/739699976 shutdown_connections 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.705+0000 7ff4b1878640 1 -- 172.21.15.26:0/739699976 wait complete. 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.706+0000 7ff4b1878640 1 Processor -- start 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.706+0000 7ff4b1878640 1 -- start start 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4b1878640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ac0724d0 0x7ff4ac149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4b1878640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac0a7ff0 0x7ff4ac0ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4b1878640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 0x7ff4ac154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4b1878640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0c8d50 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4b1878640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0704d0 con 0x7ff4ac152b40 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4b1878640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0c8ed0 con 0x7ff4ac0724d0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac0a7ff0 0x7ff4ac0ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac0a7ff0 0x7ff4ac0ab930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57918/0 (socket says 172.21.15.26:57918) 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/246238715 learned_addr learned my addr 172.21.15.26:0/246238715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 0x7ff4ac154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ac0724d0 0x7ff4ac149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ac0724d0 msgr2=0x7ff4ac149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ac0724d0 0x7ff4ac149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 msgr2=0x7ff4ac154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 0x7ff4ac154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.707+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/246238715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac155cd0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.708+0000 7ff4aa7fc640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac0a7ff0 0x7ff4ac0ab930 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c750 tx=0x7ff49400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.708+0000 7ff4a3fff640 1 -- 172.21.15.26:0/246238715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494005ad0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.708+0000 7ff4a3fff640 1 -- 172.21.15.26:0/246238715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494005c70 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.708+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac1eb440 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.708+0000 7ff4a3fff640 1 -- 172.21.15.26:0/246238715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494005e60 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.708+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac0ad4f0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.709+0000 7ff4a3fff640 1 -- 172.21.15.26:0/246238715 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4940072b0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.710+0000 7ff4a3fff640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48c06b4d0 0x7ff48c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.710+0000 7ff4a3fff640 1 -- 172.21.15.26:0/246238715 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff494089fe0 con 0x7ff4ac0a7ff0 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.710+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48c06b4d0 0x7ff48c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.710+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48c06b4d0 0x7ff48c06d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4034000 tx=0x7ff4a402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 Processor -- start 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 -- start start 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c04b0 0x5569368bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569368266b0 con 0x5569368c04b0 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569367cfa60 con 0x5569368c1f50 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.711+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556936822750 con 0x5569368c2330 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c04b0 0x5569368bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aa7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59536/0 (socket says 172.21.15.26:59536) 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 -- 172.21.15.26:0/661611139 learned_addr learned my addr 172.21.15.26:0/661611139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 -- 172.21.15.26:0/661611139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 msgr2=0x5569368bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/661611139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 -- 172.21.15.26:0/661611139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c04b0 msgr2=0x5569368bc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/661611139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c04b0 0x5569368bc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 -- 172.21.15.26:0/661611139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569367edf30 con 0x5569368c1f50 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/661611139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bfa30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4006f80 tx=0x7ff4a402d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b990093a586f947 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4a17fa640 1 -- 172.21.15.26:0/661611139 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4037040 con 0x5569368c1f50 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4a17fa640 1 -- 172.21.15.26:0/661611139 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4003ae0 con 0x5569368c1f50 2024-03-06T07:25:43.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.712+0000 7ff4a17fa640 1 -- 172.21.15.26:0/661611139 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4030510 con 0x5569368c1f50 2024-03-06T07:25:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 -- 172.21.15.26:0/661611139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 msgr2=0x5569368bfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 --2- 172.21.15.26:0/661611139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bfa30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4006f80 tx=0x7ff4a402d000 comp rx=0 tx=0).stop 2024-03-06T07:25:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 -- 172.21.15.26:0/661611139 shutdown_connections 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 --2- 172.21.15.26:0/661611139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c04b0 0x5569368bc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 --2- 172.21.15.26:0/661611139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 --2- 172.21.15.26:0/661611139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bfa30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 -- 172.21.15.26:0/661611139 >> 172.21.15.26:0/661611139 conn(0x5569368be7d0 msgr2=0x5569368e33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 -- 172.21.15.26:0/661611139 shutdown_connections 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 -- 172.21.15.26:0/661611139 wait complete. 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.713+0000 7ff4b4459740 1 -- start start 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 1 Processor -- start 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368faa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569368e4230 con 0x556936911f20 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569367e2ea0 con 0x5569368c1f50 2024-03-06T07:25:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569368e3c70 con 0x5569368c2330 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4b4459740 10 client.0 _ll_register_callbacks cb 0x5569367e2870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4aa7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368faa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57932/0 (socket says 172.21.15.26:57932) 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/941620722 learned_addr learned my addr 172.21.15.26:0/941620722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.714+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 msgr2=0x5569368e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 msgr2=0x5569368fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/941620722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569367e3270 con 0x556936911f20 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368faa50 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff49c012660 tx=0x7ff49c012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fea0a2a9cddb32 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff48a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff48a7fc640 1 -- 172.21.15.26:0/941620722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c019020 con 0x556936911f20 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4a4003ae0 con 0x556936911f20 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff48a7fc640 1 -- 172.21.15.26:0/941620722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff49c003970 con 0x556936911f20 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff48a7fc640 1 -- 172.21.15.26:0/941620722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49c003b70 con 0x556936911f20 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4b4459740 10 client.8571 fetch_fsmap learned FSMap version 126 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x55693653b2d0 con 0x556936911f20 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff48a7fc640 1 -- 172.21.15.26:0/941620722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4003ae0 con 0x556936911f20 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff48a7fc640 1 -- 172.21.15.26:0/941620722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 126) v1 ==== 3314+0+0 (secure 0 0 0) 0x7ff49c026080 con 0x556936911f20 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4b4459740 10 client.8571 fetch_fsmap finished waiting for FSMap version 126 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4b4459740 10 client.8571 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.715+0000 7ff4b4459740 20 client.8571 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.716+0000 7ff4b4459740 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693690ef00 0x55693690f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.716+0000 7ff4b4459740 4 client.8571 mds_command: new command op to 6848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.716+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 --> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556936833bf0 con 0x55693690ef00 2024-03-06T07:25:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.716+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693690ef00 0x55693690f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.716+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693690ef00 0x55693690f2e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f6a0116d358e84 server_cookie=cdd50225ffaf749b in_seq=0 out_seq=0 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.716+0000 7ff48a7fc640 10 client.8571 ms_handle_connect on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff48a7fc640 1 -- 172.21.15.26:0/941620722 <== mds.0 v2:172.21.15.117:6834/1667664264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556936833bf0 con 0x55693690ef00 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff48a7fc640 10 client.8571 handle_command_reply: tid=0 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 client.8571 shutdown 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 msgr2=0x5569368faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368faa50 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff49c012660 tx=0x7ff49c012b20 comp rx=0 tx=0).stop 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 shutdown_connections 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693690ef00 0x55693690f2e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368faa50 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff48a7fc640 0 client.8571 ms_handle_reset on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 --2- 172.21.15.26:0/941620722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 >> 172.21.15.26:0/941620722 conn(0x5569368c04b0 msgr2=0x5569368bb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 shutdown_connections 2024-03-06T07:25:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 1 -- 172.21.15.26:0/941620722 wait complete. 2024-03-06T07:25:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.717+0000 7ff4b4459740 20 client.8571 trim_cache size 0 max 16384 2024-03-06T07:25:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 Processor -- start 2024-03-06T07:25:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 -- start start 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac1f6e20 con 0x556936911f20 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac1f6340 con 0x5569368c1f50 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.734+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac1f6820 con 0x5569368c2330 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368dfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57944/0 (socket says 172.21.15.26:57944) 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/325752841 learned_addr learned my addr 172.21.15.26:0/325752841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4aa7fc640 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 msgr2=0x5569368df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 msgr2=0x5569368bf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/325752841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569366056b0 con 0x556936911f20 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368dfee0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009010 tx=0x7ff49c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11516cb5ac2aa061 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4a0ff9640 1 -- 172.21.15.26:0/325752841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c019030 con 0x556936911f20 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4a0ff9640 1 -- 172.21.15.26:0/325752841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c029070 con 0x556936911f20 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.735+0000 7ff4a0ff9640 1 -- 172.21.15.26:0/325752841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c01e700 con 0x556936911f20 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 -- 172.21.15.26:0/325752841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 msgr2=0x5569368dfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368dfee0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009010 tx=0x7ff49c00c000 comp rx=0 tx=0).stop 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 -- 172.21.15.26:0/325752841 shutdown_connections 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556936911f20 0x5569368dfee0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5569368c2330 0x5569368df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 --2- 172.21.15.26:0/325752841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x5569368bf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 -- 172.21.15.26:0/325752841 >> 172.21.15.26:0/325752841 conn(0x5569368c04b0 msgr2=0x5569368fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 -- 172.21.15.26:0/325752841 shutdown_connections 2024-03-06T07:25:43.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 -- 172.21.15.26:0/325752841 wait complete. 2024-03-06T07:25:43.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 -- start start 2024-03-06T07:25:43.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.736+0000 7ff4b4459740 1 Processor -- start 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x556936971a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c2330 0x5569369707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4b4459740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556936911f20 0x556936956ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4b4459740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569368e4fa0 con 0x5569368c2330 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693690ed80 con 0x5569368c1f50 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4b4459740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569368fee60 con 0x556936911f20 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4b4459740 10 client.0 _ll_register_callbacks cb 0x7ff49c0122c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4aa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c2330 0x5569369707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4aa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c2330 0x5569369707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57948/0 (socket says 172.21.15.26:57948) 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/1720731238 learned_addr learned my addr 172.21.15.26:0/1720731238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556936911f20 0x556936956ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.737+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x556936971a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556936911f20 msgr2=0x556936956ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4aa7fc640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556936911f20 0x556936956ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 msgr2=0x556936971a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4aa7fc640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x556936971a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4aa7fc640 1 -- 172.21.15.26:0/1720731238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556936971f60 con 0x5569368c2330 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x556936971a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4aa7fc640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c2330 0x5569369707d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff494058480 tx=0x7ff494056b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f02204ced3701c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff48a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff48a7fc640 1 -- 172.21.15.26:0/1720731238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494057bd0 con 0x5569368c2330 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff48a7fc640 1 -- 172.21.15.26:0/1720731238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff494057d70 con 0x5569368c2330 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff49c029070 con 0x5569368c2330 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff48a7fc640 1 -- 172.21.15.26:0/1720731238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49407e050 con 0x5569368c2330 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 10 client.8577 fetch_fsmap learned FSMap version 126 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x556936971230 con 0x5569368c2330 2024-03-06T07:25:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff48a7fc640 1 -- 172.21.15.26:0/1720731238 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c029070 con 0x5569368c2330 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff48a7fc640 1 -- 172.21.15.26:0/1720731238 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 126) v1 ==== 3314+0+0 (secure 0 0 0) 0x7ff494081020 con 0x5569368c2330 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 10 client.8577 fetch_fsmap finished waiting for FSMap version 126 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 10 client.8577 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 20 client.8577 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693696dff0 0x55693696e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 4 client.8577 mds_command: new command op to 6848 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.738+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 --> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55693682df40 con 0x55693696dff0 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693696dff0 0x55693696e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff48a7fc640 10 client.8577 ms_handle_connect on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693696dff0 0x55693696e3d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3def7b11736f873 server_cookie=9f47d88bce96ab65 in_seq=0 out_seq=0 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff48a7fc640 1 -- 172.21.15.26:0/1720731238 <== mds.0 v2:172.21.15.117:6834/1667664264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55693682df40 con 0x55693696dff0 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff48a7fc640 10 client.8577 handle_command_reply: tid=0 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 client.8577 shutdown 2024-03-06T07:25:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c2330 msgr2=0x5569369707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c2330 0x5569369707d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff494058480 tx=0x7ff494056b40 comp rx=0 tx=0).stop 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 shutdown_connections 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55693696dff0 0x55693696e3d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556936911f20 0x556936956ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff48a7fc640 0 client.8577 ms_handle_reset on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569368c2330 0x5569369707d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 --2- 172.21.15.26:0/1720731238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569368c1f50 0x556936971a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 >> 172.21.15.26:0/1720731238 conn(0x5569368c04b0 msgr2=0x5569368bd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 shutdown_connections 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 1 -- 172.21.15.26:0/1720731238 wait complete. 2024-03-06T07:25:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.739+0000 7ff4b4459740 20 client.8577 trim_cache size 0 max 16384 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6848, 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.728234325000001, 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 126, 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 136, 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 136, 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 28.032000923999998 2024-03-06T07:25:43.748 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:43.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48c06b4d0 msgr2=0x7ff48c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48c06b4d0 0x7ff48c06d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4034000 tx=0x7ff4a402f000 comp rx=0 tx=0).stop 2024-03-06T07:25:43.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac0a7ff0 msgr2=0x7ff4ac0ab930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac0a7ff0 0x7ff4ac0ab930 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c750 tx=0x7ff49400cc10 comp rx=0 tx=0).stop 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 shutdown_connections 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff48c06b4d0 0x7ff48c06d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ac152b40 0x7ff4ac154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac0a7ff0 0x7ff4ac0ab930 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 --2- 172.21.15.26:0/246238715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ac0724d0 0x7ff4ac149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 >> 172.21.15.26:0/246238715 conn(0x7ff4ac06b970 msgr2=0x7ff4ac06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 shutdown_connections 2024-03-06T07:25:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.740+0000 7ff4b1878640 1 -- 172.21.15.26:0/246238715 wait complete. 2024-03-06T07:25:43.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:25:43.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.854+0000 7f4125664640 1 Processor -- start 2024-03-06T07:25:43.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.854+0000 7f4125664640 1 -- start start 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f4125664640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41200c6080 0x7f412006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f4125664640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f41201534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f4125664640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4120156650 con 0x7f41200c4e60 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f4125664640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412006ec50 con 0x7f41200c6080 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f4125664640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41201568f0 con 0x7f4120153030 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f411e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41200c6080 0x7f412006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f411e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41200c6080 0x7f412006f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57952/0 (socket says 172.21.15.26:57952) 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f411e7fc640 1 -- 172.21.15.26:0/212108301 learned_addr learned my addr 172.21.15.26:0/212108301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.855+0000 7f411f7fe640 1 --2- 172.21.15.26:0/212108301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f41201534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411e7fc640 1 -- 172.21.15.26:0/212108301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 msgr2=0x7f41201534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411e7fc640 1 --2- 172.21.15.26:0/212108301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f41201534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411e7fc640 1 -- 172.21.15.26:0/212108301 >> v1:172.21.15.117:6789/0 conn(0x7f41200c4e60 legacy=0x7f4120072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411dffb640 1 -- 172.21.15.26:0/212108301 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770154380 0 0) 0x7f4120156650 con 0x7f41200c4e60 2024-03-06T07:25:43.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411e7fc640 1 -- 172.21.15.26:0/212108301 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120156bd0 con 0x7f41200c6080 2024-03-06T07:25:43.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411e7fc640 1 --2- 172.21.15.26:0/212108301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412006f7d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4114009870 tx=0x7f411402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2214d81b049cef10 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411dffb640 1 -- 172.21.15.26:0/212108301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114034070 con 0x7f41200c6080 2024-03-06T07:25:43.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.856+0000 7f411dffb640 1 -- 172.21.15.26:0/212108301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114030840 con 0x7f41200c6080 2024-03-06T07:25:43.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f411dffb640 1 -- 172.21.15.26:0/212108301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114030c10 con 0x7f41200c6080 2024-03-06T07:25:43.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 -- 172.21.15.26:0/212108301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 msgr2=0x7f412006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 --2- 172.21.15.26:0/212108301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412006f7d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4114009870 tx=0x7f411402b980 comp rx=0 tx=0).stop 2024-03-06T07:25:43.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 -- 172.21.15.26:0/212108301 shutdown_connections 2024-03-06T07:25:43.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 --2- 172.21.15.26:0/212108301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f41201534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 --2- 172.21.15.26:0/212108301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412006f7d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 -- 172.21.15.26:0/212108301 >> 172.21.15.26:0/212108301 conn(0x7f41200c5560 msgr2=0x7f41200c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 -- 172.21.15.26:0/212108301 shutdown_connections 2024-03-06T07:25:43.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.857+0000 7f4125664640 1 -- 172.21.15.26:0/212108301 wait complete. 2024-03-06T07:25:43.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.858+0000 7f4125664640 1 Processor -- start 2024-03-06T07:25:43.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.858+0000 7f4125664640 1 -- start start 2024-03-06T07:25:43.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f4125664640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41200c4e60 0x7f41201dff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f4125664640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412014d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f4125664640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f412014d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f4125664640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200c87d0 con 0x7f41200c6080 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f4125664640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4120157190 con 0x7f41200c4e60 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f4125664640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4120072d90 con 0x7f4120153030 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412014d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412014d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57956/0 (socket says 172.21.15.26:57956) 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 -- 172.21.15.26:0/1352881980 learned_addr learned my addr 172.21.15.26:0/1352881980 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411f7fe640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f412014d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411effd640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41200c4e60 0x7f41201dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 -- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 msgr2=0x7f412014d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f412014d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 -- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41200c4e60 msgr2=0x7f41201dff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41200c4e60 0x7f41201dff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 -- 172.21.15.26:0/1352881980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120155d60 con 0x7f41200c6080 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.859+0000 7f411e7fc640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412014d190 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4114000c00 tx=0x7f41140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.860+0000 7f41077fe640 1 -- 172.21.15.26:0/1352881980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114034040 con 0x7f41200c6080 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.860+0000 7f41077fe640 1 -- 172.21.15.26:0/1352881980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114030640 con 0x7f41200c6080 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.860+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41200aa590 con 0x7f41200c6080 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.860+0000 7f41077fe640 1 -- 172.21.15.26:0/1352881980 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114039d40 con 0x7f41200c6080 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.860+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412014cc70 con 0x7f41200c6080 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.861+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412004f330 con 0x7f41200c6080 2024-03-06T07:25:43.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.861+0000 7f41077fe640 1 -- 172.21.15.26:0/1352881980 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f411404b060 con 0x7f41200c6080 2024-03-06T07:25:43.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.862+0000 7f41077fe640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410c06b4d0 0x7f410c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.862+0000 7f41077fe640 1 -- 172.21.15.26:0/1352881980 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f41140abe70 con 0x7f41200c6080 2024-03-06T07:25:43.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.862+0000 7f411effd640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410c06b4d0 0x7f410c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.862+0000 7f411effd640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410c06b4d0 0x7f410c06d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4118000f60 tx=0x7f4118009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:43.866+0000 7f41077fe640 1 -- 172.21.15.26:0/1352881980 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f411407a670 con 0x7f41200c6080 2024-03-06T07:25:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.047+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7f41200c8330 con 0x7f41200c6080 2024-03-06T07:25:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.050+0000 7f41077fe640 1 -- 172.21.15.26:0/1352881980 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v48) v1 ==== 96+0+0 (secure 0 0 0) 0x7f411407e710 con 0x7f41200c6080 2024-03-06T07:25:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410c06b4d0 msgr2=0x7f410c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410c06b4d0 0x7f410c06d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4118000f60 tx=0x7f4118009040 comp rx=0 tx=0).stop 2024-03-06T07:25:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 msgr2=0x7f412014d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412014d190 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4114000c00 tx=0x7f41140046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 shutdown_connections 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410c06b4d0 0x7f410c06d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4120153030 0x7f412014d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41200c6080 0x7f412014d190 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 --2- 172.21.15.26:0/1352881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41200c4e60 0x7f41201dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 >> 172.21.15.26:0/1352881980 conn(0x7f41200c5560 msgr2=0x7f41201534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 shutdown_connections 2024-03-06T07:25:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.052+0000 7f4125664640 1 -- 172.21.15.26:0/1352881980 wait complete. 2024-03-06T07:25:44.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.165+0000 7f447f920640 1 Processor -- start 2024-03-06T07:25:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.165+0000 7f447f920640 1 -- start start 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447f920640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f447814f360 0x7f4478155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447f920640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4478158b60 0x7f4478158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447f920640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44780c73d0 con 0x7f44780c88c0 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447f920640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780c8dd0 con 0x7f4478158b60 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447f920640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780be2e0 con 0x7f447814f360 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447debd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4478158b60 0x7f4478158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447debd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4478158b60 0x7f4478158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57972/0 (socket says 172.21.15.26:57972) 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447debd640 1 -- 172.21.15.26:0/747285453 learned_addr learned my addr 172.21.15.26:0/747285453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447d6bc640 1 --2- 172.21.15.26:0/747285453 >> v2:172.21.15.117:3300/0 conn(0x7f447814f360 0x7f4478155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447debd640 1 -- 172.21.15.26:0/747285453 >> v2:172.21.15.117:3300/0 conn(0x7f447814f360 msgr2=0x7f4478155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447debd640 1 --2- 172.21.15.26:0/747285453 >> v2:172.21.15.117:3300/0 conn(0x7f447814f360 0x7f4478155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447debd640 1 -- 172.21.15.26:0/747285453 >> v1:172.21.15.117:6789/0 conn(0x7f44780c88c0 legacy=0x7f44780c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.166+0000 7f447debd640 1 -- 172.21.15.26:0/747285453 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780c7670 con 0x7f4478158b60 2024-03-06T07:25:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.167+0000 7f447debd640 1 --2- 172.21.15.26:0/747285453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4478158b60 0x7f4478158ff0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f447808c840 tx=0x7f446802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8d0c88fa337d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.167+0000 7f4477fff640 1 -- 172.21.15.26:0/747285453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468035020 con 0x7f4478158b60 2024-03-06T07:25:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.167+0000 7f4477fff640 1 -- 172.21.15.26:0/747285453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468031870 con 0x7f4478158b60 2024-03-06T07:25:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.167+0000 7f4477fff640 1 -- 172.21.15.26:0/747285453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468031c40 con 0x7f4478158b60 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 -- 172.21.15.26:0/747285453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4478158b60 msgr2=0x7f4478158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 --2- 172.21.15.26:0/747285453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4478158b60 0x7f4478158ff0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f447808c840 tx=0x7f446802b910 comp rx=0 tx=0).stop 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 -- 172.21.15.26:0/747285453 shutdown_connections 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 --2- 172.21.15.26:0/747285453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4478158b60 0x7f4478158ff0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 --2- 172.21.15.26:0/747285453 >> v2:172.21.15.117:3300/0 conn(0x7f447814f360 0x7f4478155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 -- 172.21.15.26:0/747285453 >> 172.21.15.26:0/747285453 conn(0x7f44780ac570 msgr2=0x7f44780ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 -- 172.21.15.26:0/747285453 shutdown_connections 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.168+0000 7f447f920640 1 -- 172.21.15.26:0/747285453 wait complete. 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.169+0000 7f447f920640 1 Processor -- start 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.169+0000 7f447f920640 1 -- start start 2024-03-06T07:25:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447f920640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44780c88c0 0x7f44780be4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447f920640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447814f360 0x7f44781ea810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447f920640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4478158b60 0x7f44780bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447f920640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c5a50 con 0x7f447814f360 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447f920640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c7c30 con 0x7f4478158b60 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447f920640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c5bd0 con 0x7f44780c88c0 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447814f360 0x7f44781ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447814f360 0x7f44781ea810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57982/0 (socket says 172.21.15.26:57982) 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 -- 172.21.15.26:0/4192090873 learned_addr learned my addr 172.21.15.26:0/4192090873 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447debd640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4478158b60 0x7f44780bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447d6bc640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44780c88c0 0x7f44780be4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 -- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44780c88c0 msgr2=0x7f44780be4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44780c88c0 0x7f44780be4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 -- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4478158b60 msgr2=0x7f44780bfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4478158b60 0x7f44780bfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 -- 172.21.15.26:0/4192090873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780bead0 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447cebb640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447814f360 0x7f44781ea810 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c470 tx=0x7f446c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f4475ffb640 1 -- 172.21.15.26:0/4192090873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0052d0 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f4475ffb640 1 -- 172.21.15.26:0/4192090873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c005490 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f4475ffb640 1 -- 172.21.15.26:0/4192090873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c007260 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.170+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44780bedb0 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.171+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44780bf370 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.172+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447804f330 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.172+0000 7f4475ffb640 1 -- 172.21.15.26:0/4192090873 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f446c007400 con 0x7f447814f360 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.172+0000 7f4475ffb640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b4d0 0x7f446406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.172+0000 7f4475ffb640 1 -- 172.21.15.26:0/4192090873 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f446c089710 con 0x7f447814f360 2024-03-06T07:25:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.173+0000 7f447d6bc640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b4d0 0x7f446406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.173+0000 7f447d6bc640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b4d0 0x7f446406d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f44700113e0 tx=0x7f447000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.177+0000 7f4475ffb640 1 -- 172.21.15.26:0/4192090873 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f446c057df0 con 0x7f447814f360 2024-03-06T07:25:44.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.398+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f447806cda0 con 0x7f447814f360 2024-03-06T07:25:44.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.398+0000 7f4475ffb640 1 -- 172.21.15.26:0/4192090873 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f446c05be90 con 0x7f447814f360 2024-03-06T07:25:44.400 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:44.400 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":126,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:35.012165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6848":{"gid":6848,"name":"b","rank":0,"incarnation":123,"state":"up:active","state_seq":8,"addr":"172.21.15.117:6836/1667664264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1667664264},{"type":"v1","addr":"172.21.15.117:6836","nonce":1667664264}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6848,"qdb_cluster":[6848]},"id":10}]} 2024-03-06T07:25:44.400 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 126 2024-03-06T07:25:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b4d0 msgr2=0x7f446406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b4d0 0x7f446406d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f44700113e0 tx=0x7f447000b040 comp rx=0 tx=0).stop 2024-03-06T07:25:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447814f360 msgr2=0x7f44781ea810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447814f360 0x7f44781ea810 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c470 tx=0x7f446c00c930 comp rx=0 tx=0).stop 2024-03-06T07:25:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 shutdown_connections 2024-03-06T07:25:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b4d0 0x7f446406d990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4478158b60 0x7f44780bfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f447814f360 0x7f44781ea810 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 --2- 172.21.15.26:0/4192090873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44780c88c0 0x7f44780be4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 >> 172.21.15.26:0/4192090873 conn(0x7f44780ac570 msgr2=0x7f44780c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.400+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 shutdown_connections 2024-03-06T07:25:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.401+0000 7f447f920640 1 -- 172.21.15.26:0/4192090873 wait complete. 2024-03-06T07:25:44.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:20.149307+0000', 'modified': '2024-03-06T07:25:35.012165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 136, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6848}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6848': {'gid': 6848, 'name': 'b', 'rank': 0, 'incarnation': 123, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.117:6836/1667664264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1667664264}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1667664264}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6848, 'qdb_cluster': [6848]} 2024-03-06T07:25:44.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:44.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:25:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.919+0000 7fcc1bff6640 1 Processor -- start 2024-03-06T07:25:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.919+0000 7fcc1bff6640 1 -- start start 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc1bff6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc1406c240 0x7fcc140c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc1bff6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc141588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc1bff6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc141553c0 con 0x7fcc140ac2b0 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc1bff6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc141556a0 con 0x7fcc1406c240 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc1bff6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14155980 con 0x7fcc140ab860 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc19d92640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc1406c240 0x7fcc140c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc19d92640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc1406c240 0x7fcc140c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57988/0 (socket says 172.21.15.26:57988) 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc19d92640 1 -- 172.21.15.26:0/2943176555 learned_addr learned my addr 172.21.15.26:0/2943176555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc1a593640 1 --2- 172.21.15.26:0/2943176555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc141588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc19d92640 1 -- 172.21.15.26:0/2943176555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 msgr2=0x7fcc141588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc19d92640 1 --2- 172.21.15.26:0/2943176555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc141588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc19d92640 1 -- 172.21.15.26:0/2943176555 >> v1:172.21.15.117:6789/0 conn(0x7fcc140ac2b0 legacy=0x7fcc140ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.920+0000 7fcc19d92640 1 -- 172.21.15.26:0/2943176555 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc14155c60 con 0x7fcc1406c240 2024-03-06T07:25:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.921+0000 7fcc19d92640 1 --2- 172.21.15.26:0/2943176555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1406c240 0x7fcc140c4250 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009870 tx=0x7fcc1002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5149237e6c4f580b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.921+0000 7fcc18d90640 1 -- 172.21.15.26:0/2943176555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100317c0 con 0x7fcc1406c240 2024-03-06T07:25:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.921+0000 7fcc18d90640 1 -- 172.21.15.26:0/2943176555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10037070 con 0x7fcc1406c240 2024-03-06T07:25:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.921+0000 7fcc18d90640 1 -- 172.21.15.26:0/2943176555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10031b80 con 0x7fcc1406c240 2024-03-06T07:25:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.921+0000 7fcc1bff6640 1 -- 172.21.15.26:0/2943176555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1406c240 msgr2=0x7fcc140c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.922+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/2943176555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1406c240 0x7fcc140c4250 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009870 tx=0x7fcc1002b860 comp rx=0 tx=0).stop 2024-03-06T07:25:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.922+0000 7fcc1bff6640 1 -- 172.21.15.26:0/2943176555 shutdown_connections 2024-03-06T07:25:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.922+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/2943176555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc141588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.922+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/2943176555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1406c240 0x7fcc140c4250 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.922+0000 7fcc1bff6640 1 -- 172.21.15.26:0/2943176555 >> 172.21.15.26:0/2943176555 conn(0x7fcc140c7670 msgr2=0x7fcc140c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.922+0000 7fcc1bff6640 1 -- 172.21.15.26:0/2943176555 shutdown_connections 2024-03-06T07:25:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.922+0000 7fcc1bff6640 1 -- 172.21.15.26:0/2943176555 wait complete. 2024-03-06T07:25:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 Processor -- start 2024-03-06T07:25:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 -- start start 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1406c240 0x7fcc141ea850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc14156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc140ac2b0 0x7fcc141ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140c4830 con 0x7fcc140ac2b0 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14156180 con 0x7fcc1406c240 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.923+0000 7fcc1bff6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1406c620 con 0x7fcc140ab860 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc140ac2b0 0x7fcc141ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc140ac2b0 0x7fcc141ebd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58002/0 (socket says 172.21.15.26:58002) 2024-03-06T07:25:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 -- 172.21.15.26:0/3878321950 learned_addr learned my addr 172.21.15.26:0/3878321950 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1406c240 0x7fcc141ea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc19591640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc14156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 -- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 msgr2=0x7fcc14156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc14156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 -- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1406c240 msgr2=0x7fcc141ea850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1406c240 0x7fcc141ea850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 -- 172.21.15.26:0/3878321950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc140711c0 con 0x7fcc140ac2b0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1a593640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc140ac2b0 0x7fcc141ebd40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009e30 tx=0x7fcc0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc0a7fc640 1 -- 172.21.15.26:0/3878321950 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c018020 con 0x7fcc140ac2b0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc0a7fc640 1 -- 172.21.15.26:0/3878321950 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c014480 con 0x7fcc140ac2b0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc140714a0 con 0x7fcc140ac2b0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc0a7fc640 1 -- 172.21.15.26:0/3878321950 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c014870 con 0x7fcc140ac2b0 2024-03-06T07:25:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.924+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc14071a60 con 0x7fcc140ac2b0 2024-03-06T07:25:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.926+0000 7fcc0a7fc640 1 -- 172.21.15.26:0/3878321950 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc0c005d10 con 0x7fcc140ac2b0 2024-03-06T07:25:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.926+0000 7fcc0a7fc640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbf806b4d0 0x7fcbf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.926+0000 7fcc0a7fc640 1 -- 172.21.15.26:0/3878321950 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcc0c08b070 con 0x7fcc140ac2b0 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.927+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbf806b4d0 0x7fcbf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.927+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbf806b4d0 0x7fcbf806d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcc10036000 tx=0x7fcc1003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.927+0000 7fcc1ebdb740 1 Processor -- start 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.927+0000 7fcc1ebdb740 1 -- start start 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2a9f50 0x55ccac2a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2aa330 0x55ccac2a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2a84b0 0x55ccac2a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccac20e6b0 con 0x55ccac2a84b0 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccac1b7a60 con 0x55ccac2aa330 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccac20a750 con 0x55ccac2a9f50 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2a84b0 0x55ccac2a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc19d92640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2a9f50 0x55ccac2a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2a84b0 0x55ccac2a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58012/0 (socket says 172.21.15.26:58012) 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 -- 172.21.15.26:0/2488966024 learned_addr learned my addr 172.21.15.26:0/2488966024 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc19d92640 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2a9f50 0x55ccac2a7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:50232/0 (socket says 172.21.15.26:50232) 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc19591640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2aa330 0x55ccac2a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 -- 172.21.15.26:0/2488966024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2a9f50 msgr2=0x55ccac2a7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2a9f50 0x55ccac2a7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 -- 172.21.15.26:0/2488966024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2aa330 msgr2=0x55ccac2a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2aa330 0x55ccac2a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 -- 172.21.15.26:0/2488966024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccac1d5f30 con 0x55ccac2a84b0 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.928+0000 7fcc1a593640 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2a84b0 0x55ccac2a4680 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c055bb0 tx=0x7fcc0c0593c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b42c25d0ac32a17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcbf77fe640 1 -- 172.21.15.26:0/2488966024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c071200 con 0x55ccac2a84b0 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcbf77fe640 1 -- 172.21.15.26:0/2488966024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c055dc0 con 0x55ccac2a84b0 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcbf77fe640 1 -- 172.21.15.26:0/2488966024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c058da0 con 0x55ccac2a84b0 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/2488966024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2a84b0 msgr2=0x55ccac2a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2a84b0 0x55ccac2a4680 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c055bb0 tx=0x7fcc0c0593c0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/2488966024 shutdown_connections 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2a84b0 0x55ccac2a4680 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2aa330 0x55ccac2a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/2488966024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2a9f50 0x55ccac2a7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/2488966024 >> 172.21.15.26:0/2488966024 conn(0x55ccac2a67d0 msgr2=0x55ccac2cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/2488966024 shutdown_connections 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.929+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/2488966024 wait complete. 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.930+0000 7fcc1ebdb740 1 -- start start 2024-03-06T07:25:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.930+0000 7fcc1ebdb740 1 Processor -- start 2024-03-06T07:25:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2aa330 0x55ccac2cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2e38c0 0x55ccac2e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccac2cc230 con 0x55ccac2e38c0 2024-03-06T07:25:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccac1caea0 con 0x55ccac2a9f50 2024-03-06T07:25:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccac2cbc70 con 0x55ccac2aa330 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 10 client.0 _ll_register_callbacks cb 0x55ccac1ca870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2e38c0 0x55ccac2e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2e38c0 0x55ccac2e2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58018/0 (socket says 172.21.15.26:58018) 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 -- 172.21.15.26:0/3877153029 learned_addr learned my addr 172.21.15.26:0/3877153029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc19591640 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2aa330 0x55ccac2cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 -- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2aa330 msgr2=0x55ccac2cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2aa330 0x55ccac2cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 -- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 msgr2=0x55ccac2ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 -- 172.21.15.26:0/3877153029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccac1cb270 con 0x55ccac2e38c0 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1a593640 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2e38c0 0x55ccac2e2870 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c055bb0 tx=0x7fcc0c089320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe23dd2f50c9c16 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcbf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/3877153029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c076060 con 0x55ccac2e38c0 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/3877153029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcc0c058610 con 0x55ccac2e38c0 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.931+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc0c055dc0 con 0x55ccac2e38c0 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/3877153029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc0c07d050 con 0x55ccac2e38c0 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 10 client.8601 fetch_fsmap learned FSMap version 126 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x55ccabf232d0 con 0x55ccac2e38c0 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/3877153029 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c055dc0 con 0x55ccac2e38c0 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/3877153029 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 126) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fcc0c058810 con 0x55ccac2e38c0 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 10 client.8601 fetch_fsmap finished waiting for FSMap version 126 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 10 client.8601 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 20 client.8601 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2f6d90 0x55ccac2f7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 4 client.8601 mds_command: new command op to 6848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 --> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccac21bbf0 con 0x55ccac2f6d90 2024-03-06T07:25:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2f6d90 0x55ccac2f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcbf4ff9640 10 client.8601 ms_handle_connect on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.932+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2f6d90 0x55ccac2f7170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aafd2d52fb634f93 server_cookie=6055eddb518f7d5d in_seq=0 out_seq=0 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/3877153029 <== mds.0 v2:172.21.15.117:6834/1667664264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ccac21bbf0 con 0x55ccac2f6d90 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcbf4ff9640 10 client.8601 handle_command_reply: tid=0 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcc1ebdb740 1 client.8601 shutdown 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2e38c0 msgr2=0x55ccac2e2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2e38c0 0x55ccac2e2870 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c055bb0 tx=0x7fcc0c089320 comp rx=0 tx=0).stop 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 shutdown_connections 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2f6d90 0x55ccac2f7170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.933+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2e38c0 0x55ccac2e2870 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.934+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2aa330 0x55ccac2cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.934+0000 7fcbf4ff9640 0 client.8601 ms_handle_reset on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.934+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3877153029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.934+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 >> 172.21.15.26:0/3877153029 conn(0x55ccac2a84b0 msgr2=0x55ccac2a3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.934+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 shutdown_connections 2024-03-06T07:25:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.934+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3877153029 wait complete. 2024-03-06T07:25:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.934+0000 7fcc1ebdb740 20 client.8601 trim_cache size 0 max 16384 2024-03-06T07:25:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.950+0000 7fcc1ebdb740 1 Processor -- start 2024-03-06T07:25:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.950+0000 7fcc1ebdb740 1 -- start start 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac2a3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141f6e60 con 0x55ccac2aa330 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141f63e0 con 0x55ccac2a9f50 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141f6860 con 0x55ccac2e38c0 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac2a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac2a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58024/0 (socket says 172.21.15.26:58024) 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 -- 172.21.15.26:0/3553167860 learned_addr learned my addr 172.21.15.26:0/3553167860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc1a593640 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 -- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 msgr2=0x55ccac2a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 -- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 msgr2=0x55ccac2c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19591640 1 -- 172.21.15.26:0/3553167860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccabfed6b0 con 0x55ccac2aa330 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.951+0000 7fcc19d92640 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc19591640 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac2a3740 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fcc040018f0 tx=0x7fcc04007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c841255fb8f80a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcbf6ffd640 1 -- 172.21.15.26:0/3553167860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04005320 con 0x55ccac2aa330 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcbf6ffd640 1 -- 172.21.15.26:0/3553167860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc040159d0 con 0x55ccac2aa330 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcbf6ffd640 1 -- 172.21.15.26:0/3553167860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04015d80 con 0x55ccac2aa330 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3553167860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 msgr2=0x55ccac2a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac2a3740 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fcc040018f0 tx=0x7fcc04007e50 comp rx=0 tx=0).stop 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3553167860 shutdown_connections 2024-03-06T07:25:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac2a3740 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/3553167860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3553167860 >> 172.21.15.26:0/3553167860 conn(0x55ccac2a84b0 msgr2=0x55ccac2ab030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3553167860 shutdown_connections 2024-03-06T07:25:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.952+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/3553167860 wait complete. 2024-03-06T07:25:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.953+0000 7fcc1ebdb740 1 -- start start 2024-03-06T07:25:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.953+0000 7fcc1ebdb740 1 Processor -- start 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac356ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1ebdb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccac2cc230 con 0x55ccac2aa330 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccac2cd870 con 0x55ccac2a9f50 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1ebdb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccac2ccb30 con 0x55ccac2e38c0 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1ebdb740 10 client.0 _ll_register_callbacks cb 0x7fcc0c059750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac356ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac356ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58026/0 (socket says 172.21.15.26:58026) 2024-03-06T07:25:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 -- 172.21.15.26:0/1338943472 learned_addr learned my addr 172.21.15.26:0/1338943472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19d92640 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc1a593640 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 -- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 msgr2=0x55ccac2cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 -- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 msgr2=0x55ccac2c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 -- 172.21.15.26:0/1338943472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccac2ca2a0 con 0x55ccac2aa330 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcc19591640 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac356ab0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fcc04019000 tx=0x7fcc04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f0c506966c0308 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcbf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/1338943472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04007020 con 0x55ccac2aa330 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.954+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/1338943472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcc040071c0 con 0x55ccac2aa330 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc040159d0 con 0x55ccac2aa330 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/1338943472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc040073c0 con 0x55ccac2aa330 2024-03-06T07:25:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 10 client.8607 fetch_fsmap learned FSMap version 126 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x55ccac2a5500 con 0x55ccac2aa330 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/1338943472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc040159d0 con 0x55ccac2aa330 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/1338943472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 126) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fcc040067d0 con 0x55ccac2aa330 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 10 client.8607 fetch_fsmap finished waiting for FSMap version 126 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 10 client.8607 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 20 client.8607 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2d0640 0x55ccac3537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 4 client.8607 mds_command: new command op to 6848 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 --> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ccac215f40 con 0x55ccac2d0640 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc19d92640 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2d0640 0x55ccac3537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcc19d92640 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2d0640 0x55ccac3537f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab21c0f3bdb29bd server_cookie=86c8a214a0f1ad39 in_seq=0 out_seq=0 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.955+0000 7fcbf4ff9640 10 client.8607 ms_handle_connect on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcbf4ff9640 1 -- 172.21.15.26:0/1338943472 <== mds.0 v2:172.21.15.117:6834/1667664264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55ccac215f40 con 0x55ccac2d0640 2024-03-06T07:25:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcbf4ff9640 10 client.8607 handle_command_reply: tid=0 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 client.8607 shutdown 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 msgr2=0x55ccac356ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac356ab0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fcc04019000 tx=0x7fcc04000bc0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 shutdown_connections 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:6834/1667664264,v1:172.21.15.117:6836/1667664264] conn(0x55ccac2d0640 0x55ccac3537f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccac2e38c0 0x55ccac2cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcbf4ff9640 0 client.8607 ms_handle_reset on v2:172.21.15.117:6834/1667664264 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccac2aa330 0x55ccac356ab0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 --2- 172.21.15.26:0/1338943472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccac2a9f50 0x55ccac2c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 >> 172.21.15.26:0/1338943472 conn(0x55ccac2a84b0 msgr2=0x55ccac2e9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 shutdown_connections 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 1 -- 172.21.15.26:0/1338943472 wait complete. 2024-03-06T07:25:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1ebdb740 20 client.8607 trim_cache size 0 max 16384 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6848, 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 12.944798252, 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 126, 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 136, 2024-03-06T07:25:44.965 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 136, 2024-03-06T07:25:44.966 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 29.248568077000002 2024-03-06T07:25:44.966 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbf806b4d0 msgr2=0x7fcbf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbf806b4d0 0x7fcbf806d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcc10036000 tx=0x7fcc1003d040 comp rx=0 tx=0).stop 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc140ac2b0 msgr2=0x7fcc141ebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.956+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc140ac2b0 0x7fcc141ebd40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009e30 tx=0x7fcc0c00c490 comp rx=0 tx=0).stop 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 shutdown_connections 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbf806b4d0 0x7fcbf806d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc140ac2b0 0x7fcc141ebd40 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc140ab860 0x7fcc14156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 --2- 172.21.15.26:0/3878321950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1406c240 0x7fcc141ea850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 >> 172.21.15.26:0/3878321950 conn(0x7fcc140c7670 msgr2=0x7fcc140c5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 shutdown_connections 2024-03-06T07:25:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:44.957+0000 7fcc1bff6640 1 -- 172.21.15.26:0/3878321950 wait complete. 2024-03-06T07:25:44.968 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 126, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:20.149307+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 126, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6848": { "addr": "172.21.15.117:6836/1667664264", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1667664264, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 1667664264, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6848, "incarnation": 123, "join_fscid": 10, "name": "b", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 136, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:35.012165+0000", "qdb_cluster": [ 6848 ], "qdb_leader": 6848, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6848 } } } ], "standbys": [ { "addr": "172.21.15.117:6837/3380233594", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3380233594, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3380233594, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 122, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7840, "incarnation": 0, "join_fscid": 10, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 5 }, { "addr": "172.21.15.26:6833/2142785555", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 2142785555, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 2142785555, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8292, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2316269046", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2316269046, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2316269046, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 124, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8523, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:44.969 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 10, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:25:44.969 INFO:tasks.cephfs.test_failover:current = [{'gid': 8523, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2316269046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2316269046}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2316269046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 124}, {'gid': 6848, 'name': 'b', 'rank': 0, 'incarnation': 123, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.117:6836/1667664264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1667664264}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1667664264}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8292, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/2142785555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2142785555}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 2142785555}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}, {'gid': 7840, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 5, 'addr': '172.21.15.117:6837/3380233594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3380233594}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3380233594}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 122}] 2024-03-06T07:25:44.970 DEBUG:tasks.cephfs.test_failover:-1 != 10 2024-03-06T07:25:44.970 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:25:46.174 INFO:tasks.ceph.mds.b.smithi117.stderr: -453> 2024-03-06T07:25:36.069+0000 7fa5179e5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:46.200 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:25:49.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.083+0000 7ff57f446640 1 Processor -- start 2024-03-06T07:25:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.083+0000 7ff57f446640 1 -- start start 2024-03-06T07:25:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.084+0000 7ff57f446640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff578159270 0x7ff578155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.084+0000 7ff57f446640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5780c73d0 con 0x7ff57814f360 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.084+0000 7ff57f446640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5780be2e0 con 0x7ff5780c88c0 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.084+0000 7ff57f446640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff578159700 con 0x7ff578159270 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.084+0000 7ff57d1e2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff57814f360 0x7ff5780c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56684/0 (socket says 172.21.15.26:56684) 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.084+0000 7ff57d1e2640 1 -- 172.21.15.26:0/3900658738 learned_addr learned my addr 172.21.15.26:0/3900658738 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.084+0000 7ff57d9e3640 1 --2- 172.21.15.26:0/3900658738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff578159270 0x7ff578155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.085+0000 7ff577fff640 1 -- 172.21.15.26:0/3900658738 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968622601 0 0) 0x7ff5780be2e0 con 0x7ff5780c88c0 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.085+0000 7ff577fff640 1 -- 172.21.15.26:0/3900658738 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff560003520 con 0x7ff5780c88c0 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.085+0000 7ff577fff640 1 -- 172.21.15.26:0/3900658738 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151576275 0 0) 0x7ff5780c73d0 con 0x7ff57814f360 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.085+0000 7ff577fff640 1 -- 172.21.15.26:0/3900658738 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5780be2e0 con 0x7ff57814f360 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.085+0000 7ff57d9e3640 1 -- 172.21.15.26:0/3900658738 >> v1:172.21.15.117:6789/0 conn(0x7ff5780c88c0 legacy=0x7ff578158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.085+0000 7ff57d9e3640 1 -- 172.21.15.26:0/3900658738 >> v1:172.21.15.26:6789/0 conn(0x7ff57814f360 legacy=0x7ff5780c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.085+0000 7ff57d9e3640 1 -- 172.21.15.26:0/3900658738 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780c7670 con 0x7ff578159270 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff57d9e3640 1 --2- 172.21.15.26:0/3900658738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff578159270 0x7ff578155c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff568009dc0 tx=0x7ff56802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155a8fa003acbbce server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff577fff640 1 -- 172.21.15.26:0/3900658738 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568039070 con 0x7ff578159270 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff577fff640 1 -- 172.21.15.26:0/3900658738 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56802cc40 con 0x7ff578159270 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff577fff640 1 -- 172.21.15.26:0/3900658738 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568034440 con 0x7ff578159270 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff57f446640 1 -- 172.21.15.26:0/3900658738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff578159270 msgr2=0x7ff578155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff57f446640 1 --2- 172.21.15.26:0/3900658738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff578159270 0x7ff578155c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff568009dc0 tx=0x7ff56802be90 comp rx=0 tx=0).stop 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff57f446640 1 -- 172.21.15.26:0/3900658738 shutdown_connections 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff57f446640 1 --2- 172.21.15.26:0/3900658738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff578159270 0x7ff578155c10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.086+0000 7ff57f446640 1 -- 172.21.15.26:0/3900658738 >> 172.21.15.26:0/3900658738 conn(0x7ff5780ac570 msgr2=0x7ff5780ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.087+0000 7ff57f446640 1 -- 172.21.15.26:0/3900658738 shutdown_connections 2024-03-06T07:25:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.087+0000 7ff57f446640 1 -- 172.21.15.26:0/3900658738 wait complete. 2024-03-06T07:25:50.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 Processor -- start 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 -- start start 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780c88c0 0x7ff5781ec220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57814f360 0x7ff5780bf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff578159270 0x7ff57814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780c7c30 con 0x7ff578159270 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780c5a50 con 0x7ff57814f360 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.088+0000 7ff57f446640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780c5bd0 con 0x7ff5780c88c0 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff578159270 0x7ff57814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff578159270 0x7ff57814b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58508/0 (socket says 172.21.15.26:58508) 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57c9e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57814f360 0x7ff5780bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 -- 172.21.15.26:0/1297565464 learned_addr learned my addr 172.21.15.26:0/1297565464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d1e2640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780c88c0 0x7ff5781ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 -- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780c88c0 msgr2=0x7ff5781ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780c88c0 0x7ff5781ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 -- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57814f360 msgr2=0x7ff5780bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57814f360 0x7ff5780bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 -- 172.21.15.26:0/1297565464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780c1ef0 con 0x7ff578159270 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57c9e1640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57814f360 0x7ff5780bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57d9e3640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff578159270 0x7ff57814b790 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff568009ee0 tx=0x7ff568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff575ffb640 1 -- 172.21.15.26:0/1297565464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568039050 con 0x7ff578159270 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff575ffb640 1 -- 172.21.15.26:0/1297565464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff568045070 con 0x7ff578159270 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff575ffb640 1 -- 172.21.15.26:0/1297565464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568040690 con 0x7ff578159270 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.089+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5781eb960 con 0x7ff578159270 2024-03-06T07:25:50.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.090+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5780bfca0 con 0x7ff578159270 2024-03-06T07:25:50.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.091+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57804f330 con 0x7ff578159270 2024-03-06T07:25:50.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.091+0000 7ff575ffb640 1 -- 172.21.15.26:0/1297565464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff568040830 con 0x7ff578159270 2024-03-06T07:25:50.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.091+0000 7ff575ffb640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff56406b500 0x7ff56406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.091+0000 7ff575ffb640 1 -- 172.21.15.26:0/1297565464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff5680ac600 con 0x7ff578159270 2024-03-06T07:25:50.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.092+0000 7ff57d1e2640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff56406b500 0x7ff56406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.092+0000 7ff57d1e2640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff56406b500 0x7ff56406d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff5700011e0 tx=0x7ff570007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.096+0000 7ff575ffb640 1 -- 172.21.15.26:0/1297565464 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff56804e020 con 0x7ff578159270 2024-03-06T07:25:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.315+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5780c8d50 con 0x7ff578159270 2024-03-06T07:25:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.315+0000 7ff575ffb640 1 -- 172.21.15.26:0/1297565464 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4996 (secure 0 0 0) 0x7ff56807abe0 con 0x7ff578159270 2024-03-06T07:25:50.317 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:50.317 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":131,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3312878094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3312878094},{"type":"v1","addr":"172.21.15.117:6836","nonce":3312878094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:49.177870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7840":{"gid":7840,"name":"d","rank":0,"incarnation":128,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7840,"qdb_cluster":[7840]},"id":10}]} 2024-03-06T07:25:50.318 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 131 2024-03-06T07:25:50.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff56406b500 msgr2=0x7ff56406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff56406b500 0x7ff56406d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff5700011e0 tx=0x7ff570007d30 comp rx=0 tx=0).stop 2024-03-06T07:25:50.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff578159270 msgr2=0x7ff57814b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff578159270 0x7ff57814b790 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff568009ee0 tx=0x7ff568004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 shutdown_connections 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff56406b500 0x7ff56406d9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff578159270 0x7ff57814b790 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57814f360 0x7ff5780bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 --2- 172.21.15.26:0/1297565464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780c88c0 0x7ff5781ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 >> 172.21.15.26:0/1297565464 conn(0x7ff5780ac570 msgr2=0x7ff578153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.317+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 shutdown_connections 2024-03-06T07:25:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.318+0000 7ff57f446640 1 -- 172.21.15.26:0/1297565464 wait complete. 2024-03-06T07:25:50.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:20.149307+0000', 'modified': '2024-03-06T07:25:49.177870+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 137, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7840}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7840': {'gid': 7840, 'name': 'd', 'rank': 0, 'incarnation': 128, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.117:6837/3380233594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3380233594}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3380233594}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7840, 'qdb_cluster': [7840]} 2024-03-06T07:25:50.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:50.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:25:50.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.837+0000 7f0fdcfe9640 1 Processor -- start 2024-03-06T07:25:50.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.837+0000 7f0fdcfe9640 1 -- start start 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fdcfe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80c20b0 0x7f0fd8153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fdcfe9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd806fa80 con 0x7f0fd81571e0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fdcfe9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd80a75d0 con 0x7f0fd80a9ea0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fdcfe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd8153100 con 0x7f0fd80c20b0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fd659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0fd81571e0 0x7f0fd806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56696/0 (socket says 172.21.15.26:56696) 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fd659c640 1 -- 172.21.15.26:0/3921021206 learned_addr learned my addr 172.21.15.26:0/3921021206 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fcffff640 1 --2- 172.21.15.26:0/3921021206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80c20b0 0x7f0fd8153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fd5d9b640 1 -- 172.21.15.26:0/3921021206 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722085400 0 0) 0x7f0fd806fa80 con 0x7f0fd81571e0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fd5d9b640 1 -- 172.21.15.26:0/3921021206 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fc4003520 con 0x7f0fd81571e0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fd5d9b640 1 -- 172.21.15.26:0/3921021206 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635713069 0 0) 0x7f0fd80a75d0 con 0x7f0fd80a9ea0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fd5d9b640 1 -- 172.21.15.26:0/3921021206 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd806fa80 con 0x7f0fd80a9ea0 2024-03-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fcffff640 1 -- 172.21.15.26:0/3921021206 >> v1:172.21.15.117:6789/0 conn(0x7f0fd80a9ea0 legacy=0x7f0fd815d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fcffff640 1 -- 172.21.15.26:0/3921021206 >> v1:172.21.15.26:6789/0 conn(0x7f0fd81571e0 legacy=0x7f0fd806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.838+0000 7f0fcffff640 1 -- 172.21.15.26:0/3921021206 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd80a78b0 con 0x7f0fd80c20b0 2024-03-06T07:25:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.839+0000 7f0fcffff640 1 --2- 172.21.15.26:0/3921021206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80c20b0 0x7f0fd8153590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00097d0 tx=0x7f0fc002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ace5e55b9fa246 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.839+0000 7f0fd5d9b640 1 -- 172.21.15.26:0/3921021206 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0035020 con 0x7f0fd80c20b0 2024-03-06T07:25:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.839+0000 7f0fd5d9b640 1 -- 172.21.15.26:0/3921021206 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0031800 con 0x7f0fd80c20b0 2024-03-06T07:25:50.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.839+0000 7f0fd5d9b640 1 -- 172.21.15.26:0/3921021206 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0031bd0 con 0x7f0fd80c20b0 2024-03-06T07:25:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.840+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3921021206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80c20b0 msgr2=0x7f0fd8153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.840+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3921021206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80c20b0 0x7f0fd8153590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00097d0 tx=0x7f0fc002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.840+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3921021206 shutdown_connections 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.840+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3921021206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80c20b0 0x7f0fd8153590 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.840+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3921021206 >> 172.21.15.26:0/3921021206 conn(0x7f0fd80c41b0 msgr2=0x7f0fd80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.840+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3921021206 shutdown_connections 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.840+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3921021206 wait complete. 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.841+0000 7f0fdcfe9640 1 Processor -- start 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.841+0000 7f0fdcfe9640 1 -- start start 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fdcfe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80a9ea0 0x7f0fd80c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fdcfe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fd80c20b0 0x7f0fd81dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fdcfe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fd81571e0 0x7f0fd8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fdcfe9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80a7e70 con 0x7f0fd81571e0 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fdcfe9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80a72c0 con 0x7f0fd80c20b0 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fdcfe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd806b6d0 con 0x7f0fd80a9ea0 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fd81571e0 0x7f0fd8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80a9ea0 0x7f0fd80c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fd81571e0 0x7f0fd8150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58524/0 (socket says 172.21.15.26:58524) 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/3587943668 learned_addr learned my addr 172.21.15.26:0/3587943668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fcffff640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fd80c20b0 0x7f0fd81dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80a9ea0 msgr2=0x7f0fd80c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80a9ea0 0x7f0fd80c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fd80c20b0 msgr2=0x7f0fd81dfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fd80c20b0 0x7f0fd81dfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/3587943668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8152010 con 0x7f0fd81571e0 2024-03-06T07:25:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fd81571e0 0x7f0fd8150d40 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c790 tx=0x7f0fd000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fceffd640 1 -- 172.21.15.26:0/3587943668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0003b70 con 0x7f0fd81571e0 2024-03-06T07:25:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.842+0000 7f0fceffd640 1 -- 172.21.15.26:0/3587943668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0003d10 con 0x7f0fd81571e0 2024-03-06T07:25:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.843+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd81eeb10 con 0x7f0fd81571e0 2024-03-06T07:25:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.843+0000 7f0fceffd640 1 -- 172.21.15.26:0/3587943668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000aa50 con 0x7f0fd81571e0 2024-03-06T07:25:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.843+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd80c72a0 con 0x7f0fd81571e0 2024-03-06T07:25:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.844+0000 7f0fceffd640 1 -- 172.21.15.26:0/3587943668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fd000abf0 con 0x7f0fd81571e0 2024-03-06T07:25:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.844+0000 7f0fceffd640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.845+0000 7f0fceffd640 1 -- 172.21.15.26:0/3587943668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f0fd008a250 con 0x7f0fd81571e0 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.845+0000 7f0fd659c640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.845+0000 7f0fd659c640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8000f60 tx=0x7f0fc8007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.845+0000 7f0fdfbd1740 1 Processor -- start 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fdfbd1740 1 -- start start 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d352c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c54b0 0x559d352c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3522b6b0 con 0x559d352c54b0 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d351d4a60 con 0x559d352c7330 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d35227750 con 0x559d352c6f50 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fd6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c54b0 0x559d352c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fd659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d352c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60226/0 (socket says 172.21.15.26:60226) 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fd6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c54b0 0x559d352c1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58536/0 (socket says 172.21.15.26:58536) 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.846+0000 7f0fcffff640 1 -- 172.21.15.26:0/1758882167 learned_addr learned my addr 172.21.15.26:0/1758882167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/1758882167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 msgr2=0x559d352c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d352c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/1758882167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 msgr2=0x559d352c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/1758882167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d351f2f30 con 0x559d352c54b0 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fcffff640 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c54b0 0x559d352c1680 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00885f0 tx=0x7f0fd0058200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabb8788f0941bed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fb7fff640 1 -- 172.21.15.26:0/1758882167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd008a070 con 0x559d352c54b0 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fb7fff640 1 -- 172.21.15.26:0/1758882167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0060f00 con 0x559d352c54b0 2024-03-06T07:25:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.847+0000 7f0fb7fff640 1 -- 172.21.15.26:0/1758882167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0054f10 con 0x559d352c54b0 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1758882167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c54b0 msgr2=0x559d352c1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c54b0 0x559d352c1680 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00885f0 tx=0x7f0fd0058200 comp rx=0 tx=0).stop 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1758882167 shutdown_connections 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c54b0 0x559d352c1680 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1758882167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d352c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1758882167 >> 172.21.15.26:0/1758882167 conn(0x559d352c37d0 msgr2=0x559d352e83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1758882167 shutdown_connections 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1758882167 wait complete. 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 -- start start 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.848+0000 7f0fdfbd1740 1 Processor -- start 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d35313f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352ec380 0x559d35316cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d352e9230 con 0x559d352ec380 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d351e7ea0 con 0x559d352c7330 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d352e8c70 con 0x559d352c6f50 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fdfbd1740 10 client.0 _ll_register_callbacks cb 0x559d351e7870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fd6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352ec380 0x559d35316cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fd6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352ec380 0x559d35316cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58538/0 (socket says 172.21.15.26:58538) 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/2137879925 learned_addr learned my addr 172.21.15.26:0/2137879925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fcffff640 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.849+0000 7f0fd659c640 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d35313f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 msgr2=0x559d35313f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d35313f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 msgr2=0x559d352e77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352e77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fd6d9d640 1 -- 172.21.15.26:0/2137879925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d351e8270 con 0x559d352ec380 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352ec380 0x559d35316cd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0058830 tx=0x7f0fd0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2eb34e7f44642c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fb57fa640 1 -- 172.21.15.26:0/2137879925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0085020 con 0x559d352ec380 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fb57fa640 1 -- 172.21.15.26:0/2137879925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f0fd00701d0 con 0x559d352ec380 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fd0060f00 con 0x559d352ec380 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fb57fa640 1 -- 172.21.15.26:0/2137879925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fd0057cf0 con 0x559d352ec380 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 10 client.8628 fetch_fsmap learned FSMap version 131 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x559d34f402d0 con 0x559d352ec380 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fb57fa640 1 -- 172.21.15.26:0/2137879925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0060f00 con 0x559d352ec380 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fb57fa640 1 -- 172.21.15.26:0/2137879925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 131) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0fd0057e90 con 0x559d352ec380 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 10 client.8628 fetch_fsmap finished waiting for FSMap version 131 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 10 client.8628 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 20 client.8628 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352e9600 0x559d352e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 4 client.8628 mds_command: new command op to 7840 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.850+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 --> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d35238bf0 con 0x559d352e9600 2024-03-06T07:25:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.851+0000 7f0fd659c640 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352e9600 0x559d352e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.851+0000 7f0fb57fa640 10 client.8628 ms_handle_connect on v2:172.21.15.117:6835/3380233594 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.851+0000 7f0fd659c640 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352e9600 0x559d352e99e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be8bd622a329171 server_cookie=6d9219eaef807791 in_seq=0 out_seq=0 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fb57fa640 1 -- 172.21.15.26:0/2137879925 <== mds.0 v2:172.21.15.117:6835/3380233594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559d35238bf0 con 0x559d352e9600 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fb57fa640 10 client.8628 handle_command_reply: tid=0 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 client.8628 shutdown 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352ec380 msgr2=0x559d35316cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352ec380 0x559d35316cd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0058830 tx=0x7f0fd0001250 comp rx=0 tx=0).stop 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 shutdown_connections 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352e9600 0x559d352e99e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352ec380 0x559d35316cd0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/2137879925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352c6f50 0x559d35313f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fb57fa640 0 client.8628 ms_handle_reset on v2:172.21.15.117:6835/3380233594 2024-03-06T07:25:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 >> 172.21.15.26:0/2137879925 conn(0x559d352c54b0 msgr2=0x559d352c58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 shutdown_connections 2024-03-06T07:25:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/2137879925 wait complete. 2024-03-06T07:25:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.852+0000 7f0fdfbd1740 20 client.8628 trim_cache size 0 max 16384 2024-03-06T07:25:50.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 Processor -- start 2024-03-06T07:25:50.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 -- start start 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c6f50 0x559d352bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d353134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd81f7090 con 0x559d352c6f50 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd81ebb30 con 0x559d352c7330 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.869+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd81f6a90 con 0x559d352ec380 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c6f50 0x559d352bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c6f50 0x559d352bcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58550/0 (socket says 172.21.15.26:58550) 2024-03-06T07:25:50.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 -- 172.21.15.26:0/22090173 learned_addr learned my addr 172.21.15.26:0/22090173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fcffff640 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d353134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 -- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 msgr2=0x559d353134f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d353134f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 -- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 msgr2=0x559d352c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 -- 172.21.15.26:0/22090173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3500a6b0 con 0x559d352c6f50 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fd659c640 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c6f50 0x559d352bcc00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80011e0 tx=0x7f0fc800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a9db02962c54ab server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fb77fe640 1 -- 172.21.15.26:0/22090173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80291c0 con 0x559d352c6f50 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fb77fe640 1 -- 172.21.15.26:0/22090173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8001410 con 0x559d352c6f50 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.870+0000 7f0fb77fe640 1 -- 172.21.15.26:0/22090173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8001800 con 0x559d352c6f50 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/22090173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c6f50 msgr2=0x559d352bcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c6f50 0x559d352bcc00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80011e0 tx=0x7f0fc800a000 comp rx=0 tx=0).stop 2024-03-06T07:25:50.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/22090173 shutdown_connections 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d353134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c7330 0x559d352c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/22090173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c6f50 0x559d352bcc00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/22090173 >> 172.21.15.26:0/22090173 conn(0x559d352c54b0 msgr2=0x559d352c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/22090173 shutdown_connections 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/22090173 wait complete. 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 -- start start 2024-03-06T07:25:50.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.871+0000 7f0fdfbd1740 1 Processor -- start 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c6f50 0x559d35326a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c7330 0x559d3530f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fdfbd1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d3536fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d351e7ea0 con 0x559d352c7330 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d352e4810 con 0x559d352c6f50 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fdfbd1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d352e9230 con 0x559d352ec380 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fdfbd1740 10 client.0 _ll_register_callbacks cb 0x7f0fc8006b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c7330 0x559d3530f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c7330 0x559d3530f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58556/0 (socket says 172.21.15.26:58556) 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 -- 172.21.15.26:0/1141180774 learned_addr learned my addr 172.21.15.26:0/1141180774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fd659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c6f50 0x559d35326a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fd6d9d640 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d3536fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 -- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 msgr2=0x559d3536fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d3536fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 -- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c6f50 msgr2=0x559d35326a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c6f50 0x559d35326a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fcffff640 1 -- 172.21.15.26:0/1141180774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d352c0bd0 con 0x559d352c7330 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.872+0000 7f0fd659c640 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c6f50 0x559d35326a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fcffff640 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c7330 0x559d3530f950 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00098c0 tx=0x7f0fc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eef8c22c61e36c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fb57fa640 1 -- 172.21.15.26:0/1141180774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0035040 con 0x559d352c7330 2024-03-06T07:25:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fb57fa640 1 -- 172.21.15.26:0/1141180774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f0fc0047910 con 0x559d352c7330 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fc8001410 con 0x559d352c7330 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fb57fa640 1 -- 172.21.15.26:0/1141180774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc0047b10 con 0x559d352c7330 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 10 client.8634 fetch_fsmap learned FSMap version 131 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x559d352ff0e0 con 0x559d352c7330 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fb57fa640 1 -- 172.21.15.26:0/1141180774 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8001410 con 0x559d352c7330 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fb57fa640 1 -- 172.21.15.26:0/1141180774 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 131) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0fc0057080 con 0x559d352c7330 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 10 client.8634 fetch_fsmap finished waiting for FSMap version 131 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 10 client.8634 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 20 client.8634 populate_metadata read hostname 'smithi026' 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352c4440 0x559d352c4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 4 client.8634 mds_command: new command op to 7840 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 --> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d35232f40 con 0x559d352c4440 2024-03-06T07:25:50.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.873+0000 7f0fd659c640 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352c4440 0x559d352c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fb57fa640 10 client.8634 ms_handle_connect on v2:172.21.15.117:6835/3380233594 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fd659c640 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352c4440 0x559d352c4820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e95424738415cc server_cookie=75d4da97fb64b90c in_seq=0 out_seq=0 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fb57fa640 1 -- 172.21.15.26:0/1141180774 <== mds.0 v2:172.21.15.117:6835/3380233594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x559d35232f40 con 0x559d352c4440 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fb57fa640 10 client.8634 handle_command_reply: tid=0 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 client.8634 shutdown 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c7330 msgr2=0x559d3530f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c7330 0x559d3530f950 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00098c0 tx=0x7f0fc0004730 comp rx=0 tx=0).stop 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 shutdown_connections 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:6835/3380233594,v1:172.21.15.117:6837/3380233594] conn(0x559d352c4440 0x559d352c4820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d352ec380 0x559d3536fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d352c7330 0x559d3530f950 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 --2- 172.21.15.26:0/1141180774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d352c6f50 0x559d35326a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fb57fa640 0 client.8634 ms_handle_reset on v2:172.21.15.117:6835/3380233594 2024-03-06T07:25:50.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 >> 172.21.15.26:0/1141180774 conn(0x559d352c54b0 msgr2=0x559d352bcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 shutdown_connections 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 1 -- 172.21.15.26:0/1141180774 wait complete. 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.874+0000 7f0fdfbd1740 20 client.8634 trim_cache size 0 max 16384 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7840, 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:25:50.883 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.7011037370000004, 2024-03-06T07:25:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 131, 2024-03-06T07:25:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 138, 2024-03-06T07:25:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 137, 2024-03-06T07:25:50.884 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 35.166759024999998 2024-03-06T07:25:50.884 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fb806b4d0 msgr2=0x7f0fb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8000f60 tx=0x7f0fc8007d30 comp rx=0 tx=0).stop 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fd81571e0 msgr2=0x7f0fd8150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fd81571e0 0x7f0fd8150d40 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c790 tx=0x7f0fd000cc50 comp rx=0 tx=0).stop 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 shutdown_connections 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fd81571e0 0x7f0fd8150d40 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fd80c20b0 0x7f0fd81dfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 --2- 172.21.15.26:0/3587943668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fd80a9ea0 0x7f0fd80c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 >> 172.21.15.26:0/3587943668 conn(0x7f0fd80c41b0 msgr2=0x7f0fd806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 shutdown_connections 2024-03-06T07:25:50.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:50.875+0000 7f0fdcfe9640 1 -- 172.21.15.26:0/3587943668 wait complete. 2024-03-06T07:25:50.887 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 131, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:20.149307+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 131, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7840": { "addr": "172.21.15.117:6837/3380233594", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 3380233594, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3380233594, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7840, "incarnation": 128, "join_fscid": 10, "name": "d", "rank": 0, "state": "up:active", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 137, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:49.177870+0000", "qdb_cluster": [ 7840 ], "qdb_leader": 7840, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7840 } } } ], "standbys": [ { "addr": "172.21.15.117:6836/3312878094", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3312878094, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 3312878094, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 129, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8101, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6833/2142785555", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 2142785555, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 2142785555, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 118, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8292, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.26:6835/2316269046", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 2316269046, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 2316269046, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 124, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8523, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:50.888 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 10, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:25:50.888 INFO:tasks.cephfs.test_failover:current = [{'gid': 8523, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/2316269046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2316269046}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2316269046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 124}, {'gid': 8101, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6836/3312878094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3312878094}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3312878094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 129}, {'gid': 8292, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/2142785555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2142785555}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 2142785555}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 118}, {'gid': 7840, 'name': 'd', 'rank': 0, 'incarnation': 128, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.117:6837/3380233594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3380233594}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3380233594}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:25:50.888 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:25:50.889 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:25:50.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:50.936 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:50.979 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:25:50.980 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:25:50.980 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:50.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:25:50.994 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:25:50.995 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:51.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 none /run/credentials/systemd-tmpfiles-clean.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 2024-03-06T07:25:51.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:51.072 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:25:51.070+0000 7fc3deb48640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.134+0000 7fefcb593640 1 Processor -- start 2024-03-06T07:25:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.135+0000 7fefcb593640 1 -- start start 2024-03-06T07:25:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.135+0000 7fefcb593640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fefc406da40 0x7fefc4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.135+0000 7fefcb593640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc4156050 con 0x7fefc40c7380 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.135+0000 7fefcb593640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc4072630 con 0x7fefc40c6780 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.135+0000 7fefcb593640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc406de20 con 0x7fefc406da40 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.136+0000 7fefc932f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fefc406da40 0x7fefc4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.136+0000 7fefc9b30640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fefc40c7380 0x7fefc40c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56708/0 (socket says 172.21.15.26:56708) 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.136+0000 7fefc9b30640 1 -- 172.21.15.26:0/4292916135 learned_addr learned my addr 172.21.15.26:0/4292916135 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.136+0000 7fefc932f640 1 -- 172.21.15.26:0/4292916135 >> v1:172.21.15.117:6789/0 conn(0x7fefc40c6780 legacy=0x7fefc4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.136+0000 7fefc932f640 1 -- 172.21.15.26:0/4292916135 >> v1:172.21.15.26:6789/0 conn(0x7fefc40c7380 legacy=0x7fefc40c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.136+0000 7fefbbfff640 1 -- 172.21.15.26:0/4292916135 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400961425 0 0) 0x7fefbc001490 con 0x7fefc40c7380 2024-03-06T07:25:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.136+0000 7fefc932f640 1 -- 172.21.15.26:0/4292916135 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4072910 con 0x7fefc406da40 2024-03-06T07:25:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefc932f640 1 --2- 172.21.15.26:0/4292916135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc4155b10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fefc00097a0 tx=0x7fefc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d37ede9d0eb4c54 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefbbfff640 1 -- 172.21.15.26:0/4292916135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0035020 con 0x7fefc406da40 2024-03-06T07:25:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefbbfff640 1 -- 172.21.15.26:0/4292916135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc0031830 con 0x7fefc406da40 2024-03-06T07:25:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefbbfff640 1 -- 172.21.15.26:0/4292916135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0031c00 con 0x7fefc406da40 2024-03-06T07:25:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefcb593640 1 -- 172.21.15.26:0/4292916135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 msgr2=0x7fefc4155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefcb593640 1 --2- 172.21.15.26:0/4292916135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc4155b10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fefc00097a0 tx=0x7fefc002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefcb593640 1 -- 172.21.15.26:0/4292916135 shutdown_connections 2024-03-06T07:25:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefcb593640 1 --2- 172.21.15.26:0/4292916135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc4155b10 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefcb593640 1 -- 172.21.15.26:0/4292916135 >> 172.21.15.26:0/4292916135 conn(0x7fefc40aa630 msgr2=0x7fefc40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefcb593640 1 -- 172.21.15.26:0/4292916135 shutdown_connections 2024-03-06T07:25:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.137+0000 7fefcb593640 1 -- 172.21.15.26:0/4292916135 wait complete. 2024-03-06T07:25:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.138+0000 7fefcb593640 1 Processor -- start 2024-03-06T07:25:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.138+0000 7fefcb593640 1 -- start start 2024-03-06T07:25:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefcb593640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc40ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefcb593640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc40c6780 0x7fefc41ee310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefcb593640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefc40c7380 0x7fefc41ee850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefcb593640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc41594a0 con 0x7fefc406da40 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefcb593640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc4158890 con 0x7fefc40c6780 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefcb593640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc40c3990 con 0x7fefc40c7380 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc40ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc40ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58566/0 (socket says 172.21.15.26:58566) 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 -- 172.21.15.26:0/2739475947 learned_addr learned my addr 172.21.15.26:0/2739475947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc8b2e640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc40c6780 0x7fefc41ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc9b30640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefc40c7380 0x7fefc41ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 -- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefc40c7380 msgr2=0x7fefc41ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefc40c7380 0x7fefc41ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 -- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc40c6780 msgr2=0x7fefc41ee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc40c6780 0x7fefc41ee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.139+0000 7fefc932f640 1 -- 172.21.15.26:0/2739475947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc41df020 con 0x7fefc406da40 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.140+0000 7fefc932f640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc40ad660 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fefc00097a0 tx=0x7fefc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.140+0000 7fefb9ffb640 1 -- 172.21.15.26:0/2739475947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0041070 con 0x7fefc406da40 2024-03-06T07:25:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.140+0000 7fefb9ffb640 1 -- 172.21.15.26:0/2739475947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc0004230 con 0x7fefc406da40 2024-03-06T07:25:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.140+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc41dea60 con 0x7fefc406da40 2024-03-06T07:25:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.140+0000 7fefb9ffb640 1 -- 172.21.15.26:0/2739475947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0035040 con 0x7fefc406da40 2024-03-06T07:25:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.140+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc40ad140 con 0x7fefc406da40 2024-03-06T07:25:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.141+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc404f330 con 0x7fefc406da40 2024-03-06T07:25:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.142+0000 7fefb9ffb640 1 -- 172.21.15.26:0/2739475947 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefc0038020 con 0x7fefc406da40 2024-03-06T07:25:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.142+0000 7fefb9ffb640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefa806b500 0x7fefa806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.142+0000 7fefb9ffb640 1 -- 172.21.15.26:0/2739475947 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fefc00abec0 con 0x7fefc406da40 2024-03-06T07:25:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.142+0000 7fefc8b2e640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefa806b500 0x7fefa806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.143+0000 7fefc8b2e640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefa806b500 0x7fefa806d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fefc40aed70 tx=0x7fefb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.146+0000 7fefb9ffb640 1 -- 172.21.15.26:0/2739475947 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fefc007ab60 con 0x7fefc406da40 2024-03-06T07:25:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.369+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefc4073240 con 0x7fefc406da40 2024-03-06T07:25:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.370+0000 7fefb9ffb640 1 -- 172.21.15.26:0/2739475947 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4996 (secure 0 0 0) 0x7fefc007a460 con 0x7fefc406da40 2024-03-06T07:25:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:51.372 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":131,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3312878094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3312878094},{"type":"v1","addr":"172.21.15.117:6836","nonce":3312878094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:49.177870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7840":{"gid":7840,"name":"d","rank":0,"incarnation":128,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7840,"qdb_cluster":[7840]},"id":10}]} 2024-03-06T07:25:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 131 2024-03-06T07:25:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefa806b500 msgr2=0x7fefa806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefa806b500 0x7fefa806d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fefc40aed70 tx=0x7fefb4009040 comp rx=0 tx=0).stop 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 msgr2=0x7fefc40ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc40ad660 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fefc00097a0 tx=0x7fefc00046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 shutdown_connections 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fefa806b500 0x7fefa806d9c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefc40c7380 0x7fefc41ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc40c6780 0x7fefc41ee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 --2- 172.21.15.26:0/2739475947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc406da40 0x7fefc40ad660 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 >> 172.21.15.26:0/2739475947 conn(0x7fefc40aa630 msgr2=0x7fefc40c3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 shutdown_connections 2024-03-06T07:25:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.372+0000 7fefcb593640 1 -- 172.21.15.26:0/2739475947 wait complete. 2024-03-06T07:25:51.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.484+0000 7feb3280e640 1 Processor -- start 2024-03-06T07:25:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.484+0000 7feb3280e640 1 -- start start 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb3280e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb2c155c80 0x7feb2c06ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb3280e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c0c1ae0 con 0x7feb2c152520 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb3280e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c06e450 con 0x7feb2c0720a0 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb3280e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c0c1dc0 con 0x7feb2c155c80 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb2bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb2c152520 0x7feb2c0c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56724/0 (socket says 172.21.15.26:56724) 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb2bfff640 1 -- 172.21.15.26:0/3690131556 learned_addr learned my addr 172.21.15.26:0/3690131556 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb2b7fe640 1 --2- 172.21.15.26:0/3690131556 >> v2:172.21.15.117:3300/0 conn(0x7feb2c155c80 0x7feb2c06ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb2affd640 1 -- 172.21.15.26:0/3690131556 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225048385 0 0) 0x7feb2c0c1ae0 con 0x7feb2c152520 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb2affd640 1 -- 172.21.15.26:0/3690131556 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb18003520 con 0x7feb2c152520 2024-03-06T07:25:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb2affd640 1 -- 172.21.15.26:0/3690131556 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402733725 0 0) 0x7feb2c06e450 con 0x7feb2c0720a0 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.485+0000 7feb2affd640 1 -- 172.21.15.26:0/3690131556 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb2c0c1ae0 con 0x7feb2c0720a0 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.486+0000 7feb2b7fe640 1 -- 172.21.15.26:0/3690131556 >> v1:172.21.15.117:6789/0 conn(0x7feb2c0720a0 legacy=0x7feb2c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.486+0000 7feb2b7fe640 1 -- 172.21.15.26:0/3690131556 >> v1:172.21.15.26:6789/0 conn(0x7feb2c152520 legacy=0x7feb2c0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.486+0000 7feb2b7fe640 1 -- 172.21.15.26:0/3690131556 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c06e730 con 0x7feb2c155c80 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.486+0000 7feb2b7fe640 1 --2- 172.21.15.26:0/3690131556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c155c80 0x7feb2c06ef30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0097a0 tx=0x7feb1c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7dbb972156bbabc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.486+0000 7feb2affd640 1 -- 172.21.15.26:0/3690131556 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c034070 con 0x7feb2c155c80 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.486+0000 7feb2affd640 1 -- 172.21.15.26:0/3690131556 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c030610 con 0x7feb2c155c80 2024-03-06T07:25:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.486+0000 7feb2affd640 1 -- 172.21.15.26:0/3690131556 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0309e0 con 0x7feb2c155c80 2024-03-06T07:25:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.487+0000 7feb3280e640 1 -- 172.21.15.26:0/3690131556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c155c80 msgr2=0x7feb2c06ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.487+0000 7feb3280e640 1 --2- 172.21.15.26:0/3690131556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c155c80 0x7feb2c06ef30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0097a0 tx=0x7feb1c02b960 comp rx=0 tx=0).stop 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.487+0000 7feb3280e640 1 -- 172.21.15.26:0/3690131556 shutdown_connections 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.487+0000 7feb3280e640 1 --2- 172.21.15.26:0/3690131556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c155c80 0x7feb2c06ef30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.487+0000 7feb3280e640 1 -- 172.21.15.26:0/3690131556 >> 172.21.15.26:0/3690131556 conn(0x7feb2c0c76b0 msgr2=0x7feb2c0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.487+0000 7feb3280e640 1 -- 172.21.15.26:0/3690131556 shutdown_connections 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.487+0000 7feb3280e640 1 -- 172.21.15.26:0/3690131556 wait complete. 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.488+0000 7feb3280e640 1 Processor -- start 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.488+0000 7feb3280e640 1 -- start start 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb3280e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2c0720a0 0x7feb2c14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb3280e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c152520 0x7feb2c1eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb3280e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb2c155c80 0x7feb2c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb3280e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c06b850 con 0x7feb2c155c80 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb3280e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c06b6d0 con 0x7feb2c152520 2024-03-06T07:25:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb3280e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c152900 con 0x7feb2c0720a0 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb2c155c80 0x7feb2c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb2b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c152520 0x7feb2c1eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb2c155c80 0x7feb2c1584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58578/0 (socket says 172.21.15.26:58578) 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 -- 172.21.15.26:0/480508365 learned_addr learned my addr 172.21.15.26:0/480508365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb2bfff640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2c0720a0 0x7feb2c14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 -- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2c0720a0 msgr2=0x7feb2c14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2c0720a0 0x7feb2c14cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 -- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c152520 msgr2=0x7feb2c1eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c152520 0x7feb2c1eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb30dab640 1 -- 172.21.15.26:0/480508365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c147aa0 con 0x7feb2c155c80 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.489+0000 7feb2b7fe640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c152520 0x7feb2c1eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.490+0000 7feb30dab640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb2c155c80 0x7feb2c1584b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c6e0 tx=0x7feb2400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.490+0000 7feb28ff9640 1 -- 172.21.15.26:0/480508365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24003b70 con 0x7feb2c155c80 2024-03-06T07:25:51.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.490+0000 7feb28ff9640 1 -- 172.21.15.26:0/480508365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24003d10 con 0x7feb2c155c80 2024-03-06T07:25:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.490+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c147fc0 con 0x7feb2c155c80 2024-03-06T07:25:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.490+0000 7feb28ff9640 1 -- 172.21.15.26:0/480508365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb240052d0 con 0x7feb2c155c80 2024-03-06T07:25:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.490+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c147ce0 con 0x7feb2c155c80 2024-03-06T07:25:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.491+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c0c26c0 con 0x7feb2c155c80 2024-03-06T07:25:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.491+0000 7feb28ff9640 1 -- 172.21.15.26:0/480508365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb24005470 con 0x7feb2c155c80 2024-03-06T07:25:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.491+0000 7feb28ff9640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb0c06b500 0x7feb0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.492+0000 7feb28ff9640 1 -- 172.21.15.26:0/480508365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7feb2408a1c0 con 0x7feb2c155c80 2024-03-06T07:25:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.492+0000 7feb2bfff640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb0c06b500 0x7feb0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.492+0000 7feb2bfff640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb0c06b500 0x7feb0c06d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feb14000f60 tx=0x7feb14007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.496+0000 7feb28ff9640 1 -- 172.21.15.26:0/480508365 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb2c0c26c0 con 0x7feb2c155c80 2024-03-06T07:25:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.679+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2c1558c0 con 0x7feb2c155c80 2024-03-06T07:25:51.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.681+0000 7feb28ff9640 1 -- 172.21.15.26:0/480508365 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+17262 (secure 0 0 0) 0x7feb24058780 con 0x7feb2c155c80 2024-03-06T07:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":139,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:51.193380+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:18.406892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:19.110341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.5","mappings":[{"from":3,"to":0}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.117:6834/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.26:6835/1099020571":"2024-03-07T07:25:32.004862+0000","172.21.15.26:6834/1099020571":"2024-03-07T07:25:32.004862+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb0c06b500 msgr2=0x7feb0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb0c06b500 0x7feb0c06d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feb14000f60 tx=0x7feb14007d30 comp rx=0 tx=0).stop 2024-03-06T07:25:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb2c155c80 msgr2=0x7feb2c1584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb2c155c80 0x7feb2c1584b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c6e0 tx=0x7feb2400cba0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 shutdown_connections 2024-03-06T07:25:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb0c06b500 0x7feb0c06d9c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb2c155c80 0x7feb2c1584b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb2c152520 0x7feb2c1eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 --2- 172.21.15.26:0/480508365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2c0720a0 0x7feb2c14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.682+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 >> 172.21.15.26:0/480508365 conn(0x7feb2c0c76b0 msgr2=0x7feb2c0c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.683+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 shutdown_connections 2024-03-06T07:25:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.683+0000 7feb3280e640 1 -- 172.21.15.26:0/480508365 wait complete. 2024-03-06T07:25:51.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:25:18.406892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:25:19.110341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:51.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:51.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.799+0000 7f5d4460e640 1 Processor -- start 2024-03-06T07:25:51.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.799+0000 7f5d4460e640 1 -- start start 2024-03-06T07:25:51.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.800+0000 7f5d4460e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c155c30 0x7f5d3c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.800+0000 7f5d4460e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0b0450 con 0x7f5d3c0c5380 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.800+0000 7f5d4460e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c1560e0 con 0x7f5d3c0b0070 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.800+0000 7f5d4460e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06cfb0 con 0x7f5d3c155c30 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.800+0000 7f5d42bab640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5d3c0c5380 0x7f5d3c14e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56728/0 (socket says 172.21.15.26:56728) 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.800+0000 7f5d42bab640 1 -- 172.21.15.26:0/1318509247 learned_addr learned my addr 172.21.15.26:0/1318509247 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.800+0000 7f5d41ba9640 1 --2- 172.21.15.26:0/1318509247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c155c30 0x7f5d3c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.801+0000 7f5d413a8640 1 -- 172.21.15.26:0/1318509247 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621523448 0 0) 0x7f5d34001490 con 0x7f5d3c0c5380 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.801+0000 7f5d413a8640 1 -- 172.21.15.26:0/1318509247 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d28003520 con 0x7f5d3c0c5380 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.801+0000 7f5d413a8640 1 -- 172.21.15.26:0/1318509247 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444016675 0 0) 0x7f5d3c1560e0 con 0x7f5d3c0b0070 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.801+0000 7f5d413a8640 1 -- 172.21.15.26:0/1318509247 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d34001490 con 0x7f5d3c0b0070 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.801+0000 7f5d41ba9640 1 -- 172.21.15.26:0/1318509247 >> v1:172.21.15.117:6789/0 conn(0x7f5d3c0b0070 legacy=0x7f5d3c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.801+0000 7f5d41ba9640 1 -- 172.21.15.26:0/1318509247 >> v1:172.21.15.26:6789/0 conn(0x7f5d3c0c5380 legacy=0x7f5d3c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.801+0000 7f5d41ba9640 1 -- 172.21.15.26:0/1318509247 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0c77f0 con 0x7f5d3c155c30 2024-03-06T07:25:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d41ba9640 1 --2- 172.21.15.26:0/1318509247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c155c30 0x7f5d3c06ca70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097d0 tx=0x7f5d2c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6ba139689d37c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d413a8640 1 -- 172.21.15.26:0/1318509247 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c035020 con 0x7f5d3c155c30 2024-03-06T07:25:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d413a8640 1 -- 172.21.15.26:0/1318509247 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c031800 con 0x7f5d3c155c30 2024-03-06T07:25:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d413a8640 1 -- 172.21.15.26:0/1318509247 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c031bd0 con 0x7f5d3c155c30 2024-03-06T07:25:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d4460e640 1 -- 172.21.15.26:0/1318509247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c155c30 msgr2=0x7f5d3c06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d4460e640 1 --2- 172.21.15.26:0/1318509247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c155c30 0x7f5d3c06ca70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097d0 tx=0x7f5d2c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d4460e640 1 -- 172.21.15.26:0/1318509247 shutdown_connections 2024-03-06T07:25:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d4460e640 1 --2- 172.21.15.26:0/1318509247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c155c30 0x7f5d3c06ca70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d4460e640 1 -- 172.21.15.26:0/1318509247 >> 172.21.15.26:0/1318509247 conn(0x7f5d3c0ac590 msgr2=0x7f5d3c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d4460e640 1 -- 172.21.15.26:0/1318509247 shutdown_connections 2024-03-06T07:25:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.802+0000 7f5d4460e640 1 -- 172.21.15.26:0/1318509247 wait complete. 2024-03-06T07:25:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 Processor -- start 2024-03-06T07:25:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 -- start start 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c0b0070 0x7f5d3c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c0c5380 0x7f5d3c0c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d3c155c30 0x7f5d3c14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06c270 con 0x7f5d3c155c30 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06c020 con 0x7f5d3c0c5380 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d4460e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06c550 con 0x7f5d3c0b0070 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d42bab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d3c155c30 0x7f5d3c14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d42bab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d3c155c30 0x7f5d3c14b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58580/0 (socket says 172.21.15.26:58580) 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d42bab640 1 -- 172.21.15.26:0/2377969621 learned_addr learned my addr 172.21.15.26:0/2377969621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d41ba9640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c0c5380 0x7f5d3c0c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.804+0000 7f5d423aa640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c0b0070 0x7f5d3c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d42bab640 1 -- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c0b0070 msgr2=0x7f5d3c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d42bab640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c0b0070 0x7f5d3c156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d42bab640 1 -- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c0c5380 msgr2=0x7f5d3c0c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d42bab640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c0c5380 0x7f5d3c0c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d42bab640 1 -- 172.21.15.26:0/2377969621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0c3f40 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d41ba9640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c0c5380 0x7f5d3c0c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d42bab640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d3c155c30 0x7f5d3c14b7d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001fc0 tx=0x7f5d3400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d32ffd640 1 -- 172.21.15.26:0/2377969621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34017070 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d32ffd640 1 -- 172.21.15.26:0/2377969621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d34012400 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c0c4220 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d32ffd640 1 -- 172.21.15.26:0/2377969621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340127b0 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.805+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c0c4500 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.806+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c0ca7a0 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.807+0000 7f5d32ffd640 1 -- 172.21.15.26:0/2377969621 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d3401a070 con 0x7f5d3c155c30 2024-03-06T07:25:51.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.807+0000 7f5d32ffd640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d2006b4d0 0x7f5d2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.807+0000 7f5d32ffd640 1 -- 172.21.15.26:0/2377969621 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f5d3408a360 con 0x7f5d3c155c30 2024-03-06T07:25:51.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.807+0000 7f5d423aa640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d2006b4d0 0x7f5d2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.808+0000 7f5d423aa640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d2006b4d0 0x7f5d2006d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5d38007fd0 tx=0x7f5d38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.811+0000 7f5d32ffd640 1 -- 172.21.15.26:0/2377969621 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d3c0b0450 con 0x7f5d3c155c30 2024-03-06T07:25:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.993+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d3c06d860 con 0x7f5d3c155c30 2024-03-06T07:25:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.993+0000 7f5d32ffd640 1 -- 172.21.15.26:0/2377969621 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d34058890 con 0x7f5d3c155c30 2024-03-06T07:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:25:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d2006b4d0 msgr2=0x7f5d2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d2006b4d0 0x7f5d2006d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5d38007fd0 tx=0x7f5d38009040 comp rx=0 tx=0).stop 2024-03-06T07:25:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d3c155c30 msgr2=0x7f5d3c14b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d3c155c30 0x7f5d3c14b7d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001fc0 tx=0x7f5d3400a430 comp rx=0 tx=0).stop 2024-03-06T07:25:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 shutdown_connections 2024-03-06T07:25:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d2006b4d0 0x7f5d2006d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d3c155c30 0x7f5d3c14b7d0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c0c5380 0x7f5d3c0c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 --2- 172.21.15.26:0/2377969621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d3c0b0070 0x7f5d3c156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 >> 172.21.15.26:0/2377969621 conn(0x7f5d3c0ac590 msgr2=0x7f5d3c0aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 shutdown_connections 2024-03-06T07:25:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:51.995+0000 7f5d4460e640 1 -- 172.21.15.26:0/2377969621 wait complete. 2024-03-06T07:25:52.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:25:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.111+0000 7ff105913640 1 Processor -- start 2024-03-06T07:25:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.111+0000 7ff105913640 1 -- start start 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff105913640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1000c26c0 0x7ff100155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff105913640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff1000c2aa0 0x7ff10006bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff105913640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10006c110 con 0x7ff1000c0470 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff105913640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10006b740 con 0x7ff1000c26c0 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff105913640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1000670c0 con 0x7ff1000c2aa0 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0feffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1000c26c0 0x7ff100155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0ff7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1000c0470 0x7ff100152350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56738/0 (socket says 172.21.15.26:56738) 2024-03-06T07:25:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0ff7fe640 1 -- 172.21.15.26:0/2066461617 learned_addr learned my addr 172.21.15.26:0/2066461617 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0fe7fc640 1 --2- 172.21.15.26:0/2066461617 >> v2:172.21.15.117:3300/0 conn(0x7ff1000c2aa0 0x7ff10006bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0feffd640 1 -- 172.21.15.26:0/2066461617 >> v2:172.21.15.117:3300/0 conn(0x7ff1000c2aa0 msgr2=0x7ff10006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0feffd640 1 --2- 172.21.15.26:0/2066461617 >> v2:172.21.15.117:3300/0 conn(0x7ff1000c2aa0 0x7ff10006bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0feffd640 1 -- 172.21.15.26:0/2066461617 >> v1:172.21.15.26:6789/0 conn(0x7ff1000c0470 legacy=0x7ff100152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0feffd640 1 -- 172.21.15.26:0/2066461617 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10006b0f0 con 0x7ff1000c26c0 2024-03-06T07:25:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.112+0000 7ff0fdffb640 1 -- 172.21.15.26:0/2066461617 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550685094 0 0) 0x7ff10006c110 con 0x7ff1000c0470 2024-03-06T07:25:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.113+0000 7ff0feffd640 1 --2- 172.21.15.26:0/2066461617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c26c0 0x7ff100155bd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff10006f590 tx=0x7ff0f802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eb5689db52e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.113+0000 7ff0fdffb640 1 -- 172.21.15.26:0/2066461617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8038070 con 0x7ff1000c26c0 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.113+0000 7ff0fdffb640 1 -- 172.21.15.26:0/2066461617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8031710 con 0x7ff1000c26c0 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.113+0000 7ff0fdffb640 1 -- 172.21.15.26:0/2066461617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8031ae0 con 0x7ff1000c26c0 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 -- 172.21.15.26:0/2066461617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c26c0 msgr2=0x7ff100155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 --2- 172.21.15.26:0/2066461617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c26c0 0x7ff100155bd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff10006f590 tx=0x7ff0f802b7b0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 -- 172.21.15.26:0/2066461617 shutdown_connections 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 --2- 172.21.15.26:0/2066461617 >> v2:172.21.15.117:3300/0 conn(0x7ff1000c2aa0 0x7ff10006bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 --2- 172.21.15.26:0/2066461617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c26c0 0x7ff100155bd0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 -- 172.21.15.26:0/2066461617 >> 172.21.15.26:0/2066461617 conn(0x7ff1000c3b40 msgr2=0x7ff1000c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 -- 172.21.15.26:0/2066461617 shutdown_connections 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.114+0000 7ff105913640 1 -- 172.21.15.26:0/2066461617 wait complete. 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.115+0000 7ff105913640 1 Processor -- start 2024-03-06T07:25:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.115+0000 7ff105913640 1 -- start start 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1000c0470 0x7ff10014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1000c26c0 0x7ff100149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c2aa0 0x7ff1001ede80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000c1f30 con 0x7ff1000c2aa0 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000c0900 con 0x7ff1000c0470 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000c20b0 con 0x7ff1000c26c0 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c2aa0 0x7ff1001ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c2aa0 0x7ff1001ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58588/0 (socket says 172.21.15.26:58588) 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 -- 172.21.15.26:0/726879938 learned_addr learned my addr 172.21.15.26:0/726879938 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0fe7fc640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1000c26c0 0x7ff100149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0feffd640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1000c0470 0x7ff10014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 -- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1000c26c0 msgr2=0x7ff100149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1000c26c0 0x7ff100149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 -- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1000c0470 msgr2=0x7ff10014fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1000c0470 0x7ff10014fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 -- 172.21.15.26:0/726879938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000aa160 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0ff7fe640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c2aa0 0x7ff1001ede80 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000c7a0 tx=0x7ff0f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0e7fff640 1 -- 172.21.15.26:0/726879938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0018070 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0e7fff640 1 -- 172.21.15.26:0/726879938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f000a640 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff0e7fff640 1 -- 172.21.15.26:0/726879938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000aa70 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1001557c0 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.116+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1000abda0 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.117+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff100075300 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.118+0000 7ff0e7fff640 1 -- 172.21.15.26:0/726879938 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0f000ac10 con 0x7ff1000c2aa0 2024-03-06T07:25:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.118+0000 7ff0e7fff640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff0ec06b500 0x7ff0ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.118+0000 7ff0e7fff640 1 -- 172.21.15.26:0/726879938 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7ff0f0089860 con 0x7ff1000c2aa0 2024-03-06T07:25:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.118+0000 7ff0feffd640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff0ec06b500 0x7ff0ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.119+0000 7ff0feffd640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff0ec06b500 0x7ff0ec06d9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8035000 tx=0x7ff0f8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.122+0000 7ff0e7fff640 1 -- 172.21.15.26:0/726879938 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff100075300 con 0x7ff1000c2aa0 2024-03-06T07:25:52.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.304+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1001edb50 con 0x7ff1000c2aa0 2024-03-06T07:25:52.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.304+0000 7ff0e7fff640 1 -- 172.21.15.26:0/726879938 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0f0057eb0 con 0x7ff1000c2aa0 2024-03-06T07:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:25:52.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff0ec06b500 msgr2=0x7ff0ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff0ec06b500 0x7ff0ec06d9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8035000 tx=0x7ff0f8008030 comp rx=0 tx=0).stop 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c2aa0 msgr2=0x7ff1001ede80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c2aa0 0x7ff1001ede80 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000c7a0 tx=0x7ff0f000cc60 comp rx=0 tx=0).stop 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 shutdown_connections 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff0ec06b500 0x7ff0ec06d9c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1000c2aa0 0x7ff1001ede80 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1000c26c0 0x7ff100149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 --2- 172.21.15.26:0/726879938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1000c0470 0x7ff10014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 >> 172.21.15.26:0/726879938 conn(0x7ff1000c3b40 msgr2=0x7ff10006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 shutdown_connections 2024-03-06T07:25:52.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.306+0000 7ff105913640 1 -- 172.21.15.26:0/726879938 wait complete. 2024-03-06T07:25:52.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:25:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.421+0000 7f428b639640 1 Processor -- start 2024-03-06T07:25:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.421+0000 7f428b639640 1 -- start start 2024-03-06T07:25:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f428b639640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f428414bcc0 0x7f4284155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f428b639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4284158a10 0x7f4284158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f428b639640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42840c75e0 con 0x7f42840be150 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f428b639640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42840c7f10 con 0x7f428414bcc0 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f428b639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42840c78c0 con 0x7f4284158a10 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f4289bd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4284158a10 0x7f4284158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f42893d5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f428414bcc0 0x7f4284155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f4288bd4640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f42840be150 0x7f42840c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:36774/0 (socket says 172.21.15.26:36774) 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f42893d5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f428414bcc0 0x7f4284155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60304/0 (socket says 172.21.15.26:60304) 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f4289bd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4284158a10 0x7f4284158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:33276/0 (socket says 172.21.15.26:33276) 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.422+0000 7f42893d5640 1 -- 172.21.15.26:0/3950243174 learned_addr learned my addr 172.21.15.26:0/3950243174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.423+0000 7f42893d5640 1 -- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4284158a10 msgr2=0x7f4284158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.423+0000 7f42893d5640 1 --2- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4284158a10 0x7f4284158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.423+0000 7f42893d5640 1 -- 172.21.15.26:0/3950243174 >> v1:172.21.15.117:6789/0 conn(0x7f42840be150 legacy=0x7f42840c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.423+0000 7f42893d5640 1 -- 172.21.15.26:0/3950243174 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42840c7ba0 con 0x7f428414bcc0 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.423+0000 7f4283fff640 1 -- 172.21.15.26:0/3950243174 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765619260 0 0) 0x7f42840c75e0 con 0x7f42840be150 2024-03-06T07:25:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.423+0000 7f4289bd6640 1 --2- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4284158a10 0x7f4284158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.423+0000 7f42893d5640 1 --2- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f428414bcc0 0x7f4284155a00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f427c009870 tx=0x7f427c02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=722ce7e47ba6b2e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f4283fff640 1 -- 172.21.15.26:0/3950243174 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c02f830 con 0x7f428414bcc0 2024-03-06T07:25:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f4283fff640 1 -- 172.21.15.26:0/3950243174 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c02f9d0 con 0x7f428414bcc0 2024-03-06T07:25:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f4283fff640 1 -- 172.21.15.26:0/3950243174 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c02fda0 con 0x7f428414bcc0 2024-03-06T07:25:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 -- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f428414bcc0 msgr2=0x7f4284155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 --2- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f428414bcc0 0x7f4284155a00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f427c009870 tx=0x7f427c02ba10 comp rx=0 tx=0).stop 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 -- 172.21.15.26:0/3950243174 shutdown_connections 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 --2- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4284158a10 0x7f4284158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 --2- 172.21.15.26:0/3950243174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f428414bcc0 0x7f4284155a00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 -- 172.21.15.26:0/3950243174 >> 172.21.15.26:0/3950243174 conn(0x7f42840ac600 msgr2=0x7f42840aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 -- 172.21.15.26:0/3950243174 shutdown_connections 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.424+0000 7f428b639640 1 -- 172.21.15.26:0/3950243174 wait complete. 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.425+0000 7f428b639640 1 Processor -- start 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.425+0000 7f428b639640 1 -- start start 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f428b639640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42840be150 0x7f4284150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f428b639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f428414bcc0 0x7f42841df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f428b639640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4284158a10 0x7f42841dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f428b639640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42840c5c60 con 0x7f42840be150 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f428b639640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42840c5ae0 con 0x7f4284158a10 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f428b639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42841593c0 con 0x7f428414bcc0 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42840be150 0x7f4284150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42840be150 0x7f4284150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58600/0 (socket says 172.21.15.26:58600) 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 -- 172.21.15.26:0/2564435056 learned_addr learned my addr 172.21.15.26:0/2564435056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f4289bd6640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4284158a10 0x7f42841dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f4288bd4640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f428414bcc0 0x7f42841df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 -- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f428414bcc0 msgr2=0x7f42841df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f428414bcc0 0x7f42841df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 -- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4284158a10 msgr2=0x7f42841dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4284158a10 0x7f42841dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 -- 172.21.15.26:0/2564435056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42841526f0 con 0x7f42840be150 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.426+0000 7f42893d5640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42840be150 0x7f4284150870 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.427+0000 7f4281ffb640 1 -- 172.21.15.26:0/2564435056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c037c40 con 0x7f42840be150 2024-03-06T07:25:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.427+0000 7f4281ffb640 1 -- 172.21.15.26:0/2564435056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c046070 con 0x7f42840be150 2024-03-06T07:25:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.427+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42841ebbd0 con 0x7f42840be150 2024-03-06T07:25:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.427+0000 7f4281ffb640 1 -- 172.21.15.26:0/2564435056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c041400 con 0x7f42840be150 2024-03-06T07:25:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.427+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42841e1380 con 0x7f42840be150 2024-03-06T07:25:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.428+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428404f330 con 0x7f42840be150 2024-03-06T07:25:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.428+0000 7f4281ffb640 1 -- 172.21.15.26:0/2564435056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f427c040070 con 0x7f42840be150 2024-03-06T07:25:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.429+0000 7f4281ffb640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f427006b500 0x7f427006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.429+0000 7f4281ffb640 1 -- 172.21.15.26:0/2564435056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f427c0ac860 con 0x7f42840be150 2024-03-06T07:25:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.429+0000 7f4288bd4640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f427006b500 0x7f427006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.429+0000 7f4288bd4640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f427006b500 0x7f427006d9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f42841ef650 tx=0x7f4278007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.433+0000 7f4281ffb640 1 -- 172.21.15.26:0/2564435056 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f427c033030 con 0x7f42840be150 2024-03-06T07:25:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.611+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42840c83e0 con 0x7f42840be150 2024-03-06T07:25:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.612+0000 7f4281ffb640 1 -- 172.21.15.26:0/2564435056 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7f427c07b580 con 0x7f42840be150 2024-03-06T07:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:25:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f427006b500 msgr2=0x7f427006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f427006b500 0x7f427006d9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f42841ef650 tx=0x7f4278007d50 comp rx=0 tx=0).stop 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42840be150 msgr2=0x7f4284150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42840be150 0x7f4284150870 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c004730 comp rx=0 tx=0).stop 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 shutdown_connections 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f427006b500 0x7f427006d9c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4284158a10 0x7f42841dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f428414bcc0 0x7f42841df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 --2- 172.21.15.26:0/2564435056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42840be150 0x7f4284150870 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 >> 172.21.15.26:0/2564435056 conn(0x7f42840ac600 msgr2=0x7f42840c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 shutdown_connections 2024-03-06T07:25:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.614+0000 7f428b639640 1 -- 172.21.15.26:0/2564435056 wait complete. 2024-03-06T07:25:52.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.726+0000 7fb03ba88640 1 Processor -- start 2024-03-06T07:25:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.726+0000 7fb03ba88640 1 -- start start 2024-03-06T07:25:52.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.726+0000 7fb03ba88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03406d9f0 0x7fb0340a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.726+0000 7fb03ba88640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb034158800 con 0x7fb0340c3280 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.726+0000 7fb03ba88640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb034158ae0 con 0x7fb0340c36e0 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.726+0000 7fb03ba88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340c3b10 con 0x7fb03406d9f0 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb039023640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb0340c3280 0x7fb0341551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56744/0 (socket says 172.21.15.26:56744) 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb039023640 1 -- 172.21.15.26:0/4080304155 learned_addr learned my addr 172.21.15.26:0/4080304155 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb039824640 1 --2- 172.21.15.26:0/4080304155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03406d9f0 0x7fb0340a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb038822640 1 -- 172.21.15.26:0/4080304155 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738744965 0 0) 0x7fb034158800 con 0x7fb0340c3280 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb038822640 1 -- 172.21.15.26:0/4080304155 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb020003520 con 0x7fb0340c3280 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb038822640 1 -- 172.21.15.26:0/4080304155 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889116542 0 0) 0x7fb034158ae0 con 0x7fb0340c36e0 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb038822640 1 -- 172.21.15.26:0/4080304155 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb034158800 con 0x7fb0340c36e0 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb039824640 1 -- 172.21.15.26:0/4080304155 >> v1:172.21.15.117:6789/0 conn(0x7fb0340c36e0 legacy=0x7fb0341558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb039824640 1 -- 172.21.15.26:0/4080304155 >> v1:172.21.15.26:6789/0 conn(0x7fb0340c3280 legacy=0x7fb0341551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.727+0000 7fb039824640 1 -- 172.21.15.26:0/4080304155 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034158dc0 con 0x7fb03406d9f0 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb039824640 1 --2- 172.21.15.26:0/4080304155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03406d9f0 0x7fb0340a7cd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0300097d0 tx=0x7fb03002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0380b483dfc7078 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb038822640 1 -- 172.21.15.26:0/4080304155 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030031770 con 0x7fb03406d9f0 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb038822640 1 -- 172.21.15.26:0/4080304155 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb030037070 con 0x7fb03406d9f0 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb038822640 1 -- 172.21.15.26:0/4080304155 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030031b10 con 0x7fb03406d9f0 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb03ba88640 1 -- 172.21.15.26:0/4080304155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03406d9f0 msgr2=0x7fb0340a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb03ba88640 1 --2- 172.21.15.26:0/4080304155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03406d9f0 0x7fb0340a7cd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0300097d0 tx=0x7fb03002b810 comp rx=0 tx=0).stop 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb03ba88640 1 -- 172.21.15.26:0/4080304155 shutdown_connections 2024-03-06T07:25:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb03ba88640 1 --2- 172.21.15.26:0/4080304155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03406d9f0 0x7fb0340a7cd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.728+0000 7fb03ba88640 1 -- 172.21.15.26:0/4080304155 >> 172.21.15.26:0/4080304155 conn(0x7fb0340aa630 msgr2=0x7fb0340aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.729+0000 7fb03ba88640 1 -- 172.21.15.26:0/4080304155 shutdown_connections 2024-03-06T07:25:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.729+0000 7fb03ba88640 1 -- 172.21.15.26:0/4080304155 wait complete. 2024-03-06T07:25:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 Processor -- start 2024-03-06T07:25:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 -- start start 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03406d9f0 0x7fb03406f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0340c3280 0x7fb0341eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0340c36e0 0x7fb0341eb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03406ddd0 con 0x7fb03406d9f0 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034159380 con 0x7fb0340c36e0 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03ba88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03406df50 con 0x7fb0340c3280 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb039824640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03406d9f0 0x7fb03406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb039824640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03406d9f0 0x7fb03406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58610/0 (socket says 172.21.15.26:58610) 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb039824640 1 -- 172.21.15.26:0/3586483474 learned_addr learned my addr 172.21.15.26:0/3586483474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb039023640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0340c3280 0x7fb0341eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.730+0000 7fb03a025640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0340c36e0 0x7fb0341eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb039824640 1 -- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0340c3280 msgr2=0x7fb0341eadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb039824640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0340c3280 0x7fb0341eadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb039824640 1 -- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0340c36e0 msgr2=0x7fb0341eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb039824640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0340c36e0 0x7fb0341eb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb039824640 1 -- 172.21.15.26:0/3586483474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0341eec70 con 0x7fb03406d9f0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb039824640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03406d9f0 0x7fb03406f930 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb03002b850 tx=0x7fb0300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb02a7fc640 1 -- 172.21.15.26:0/3586483474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030032600 con 0x7fb03406d9f0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb02a7fc640 1 -- 172.21.15.26:0/3586483474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0300327a0 con 0x7fb03406d9f0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0340ab830 con 0x7fb03406d9f0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb02a7fc640 1 -- 172.21.15.26:0/3586483474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030032b70 con 0x7fb03406d9f0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.731+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03406fe70 con 0x7fb03406d9f0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.732+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03404f330 con 0x7fb03406d9f0 2024-03-06T07:25:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.733+0000 7fb02a7fc640 1 -- 172.21.15.26:0/3586483474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb03004c060 con 0x7fb03406d9f0 2024-03-06T07:25:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.733+0000 7fb02a7fc640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb01806b4d0 0x7fb01806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.733+0000 7fb02a7fc640 1 -- 172.21.15.26:0/3586483474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fb0300ab7d0 con 0x7fb03406d9f0 2024-03-06T07:25:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.733+0000 7fb039023640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb01806b4d0 0x7fb01806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.734+0000 7fb039023640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb01806b4d0 0x7fb01806d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb0340ae8e0 tx=0x7fb024007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.737+0000 7fb02a7fc640 1 -- 172.21.15.26:0/3586483474 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb030040070 con 0x7fb03406d9f0 2024-03-06T07:25:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.959+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb034073240 con 0x7fb03406d9f0 2024-03-06T07:25:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.959+0000 7fb02a7fc640 1 -- 172.21.15.26:0/3586483474 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4996 (secure 0 0 0) 0x7fb030079e20 con 0x7fb03406d9f0 2024-03-06T07:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":131,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3312878094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3312878094},{"type":"v1","addr":"172.21.15.117:6836","nonce":3312878094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:49.177870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7840":{"gid":7840,"name":"d","rank":0,"incarnation":128,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7840,"qdb_cluster":[7840]},"id":10}]} 2024-03-06T07:25:52.961 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 131 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb01806b4d0 msgr2=0x7fb01806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb01806b4d0 0x7fb01806d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb0340ae8e0 tx=0x7fb024007d50 comp rx=0 tx=0).stop 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03406d9f0 msgr2=0x7fb03406f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03406d9f0 0x7fb03406f930 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb03002b850 tx=0x7fb0300046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 shutdown_connections 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb01806b4d0 0x7fb01806d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0340c36e0 0x7fb0341eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0340c3280 0x7fb0341eadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 --2- 172.21.15.26:0/3586483474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03406d9f0 0x7fb03406f930 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 >> 172.21.15.26:0/3586483474 conn(0x7fb0340aa630 msgr2=0x7fb0340c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 shutdown_connections 2024-03-06T07:25:52.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:52.961+0000 7fb03ba88640 1 -- 172.21.15.26:0/3586483474 wait complete. 2024-03-06T07:25:52.973 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:53.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.074+0000 7f36bf743640 1 Processor -- start 2024-03-06T07:25:53.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.074+0000 7f36bf743640 1 -- start start 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.074+0000 7f36bf743640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f36b806da40 0x7f36b8155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.074+0000 7f36bf743640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36b80c6780 0x7f36b8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.074+0000 7f36bf743640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b8156050 con 0x7f36b81592d0 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.074+0000 7f36bf743640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b806de20 con 0x7f36b80c6780 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.074+0000 7f36bf743640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b8072630 con 0x7f36b806da40 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bccde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36b80c6780 0x7f36b8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bccde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36b80c6780 0x7f36b8158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58622/0 (socket says 172.21.15.26:58622) 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bccde640 1 -- 172.21.15.26:0/2410427943 learned_addr learned my addr 172.21.15.26:0/2410427943 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bd4df640 1 --2- 172.21.15.26:0/2410427943 >> v2:172.21.15.117:3300/0 conn(0x7f36b806da40 0x7f36b8155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bccde640 1 -- 172.21.15.26:0/2410427943 >> v2:172.21.15.117:3300/0 conn(0x7f36b806da40 msgr2=0x7f36b8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bccde640 1 --2- 172.21.15.26:0/2410427943 >> v2:172.21.15.117:3300/0 conn(0x7f36b806da40 0x7f36b8155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bccde640 1 -- 172.21.15.26:0/2410427943 >> v1:172.21.15.117:6789/0 conn(0x7f36b81592d0 legacy=0x7f36b80c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36affff640 1 -- 172.21.15.26:0/2410427943 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170361350 0 0) 0x7f36b0001490 con 0x7f36b81592d0 2024-03-06T07:25:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.075+0000 7f36bccde640 1 -- 172.21.15.26:0/2410427943 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b8072910 con 0x7f36b80c6780 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bccde640 1 --2- 172.21.15.26:0/2410427943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b8158d90 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f36b80c98a0 tx=0x7f36a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2af71b3edf448b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36affff640 1 -- 172.21.15.26:0/2410427943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8035020 con 0x7f36b80c6780 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36affff640 1 -- 172.21.15.26:0/2410427943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a8031870 con 0x7f36b80c6780 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36affff640 1 -- 172.21.15.26:0/2410427943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8031c40 con 0x7f36b80c6780 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bf743640 1 -- 172.21.15.26:0/2410427943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 msgr2=0x7f36b8158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bf743640 1 --2- 172.21.15.26:0/2410427943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b8158d90 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f36b80c98a0 tx=0x7f36a802b910 comp rx=0 tx=0).stop 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bf743640 1 -- 172.21.15.26:0/2410427943 shutdown_connections 2024-03-06T07:25:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bf743640 1 --2- 172.21.15.26:0/2410427943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b8158d90 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bf743640 1 --2- 172.21.15.26:0/2410427943 >> v2:172.21.15.117:3300/0 conn(0x7f36b806da40 0x7f36b8155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bf743640 1 -- 172.21.15.26:0/2410427943 >> 172.21.15.26:0/2410427943 conn(0x7f36b80aa630 msgr2=0x7f36b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.076+0000 7f36bf743640 1 -- 172.21.15.26:0/2410427943 shutdown_connections 2024-03-06T07:25:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.077+0000 7f36bf743640 1 -- 172.21.15.26:0/2410427943 wait complete. 2024-03-06T07:25:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 Processor -- start 2024-03-06T07:25:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 -- start start 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36b806da40 0x7f36b806e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36b81592d0 0x7f36b81eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b80c3990 con 0x7f36b80c6780 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8158890 con 0x7f36b81592d0 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bf743640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b80c3b10 con 0x7f36b806da40 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bccde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bccde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b814db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58632/0 (socket says 172.21.15.26:58632) 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bccde640 1 -- 172.21.15.26:0/2596713466 learned_addr learned my addr 172.21.15.26:0/2596713466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bd4df640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36b806da40 0x7f36b806e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.078+0000 7f36bdce0640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36b81592d0 0x7f36b81eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bccde640 1 -- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36b806da40 msgr2=0x7f36b806e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bccde640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36b806da40 0x7f36b806e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bccde640 1 -- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36b81592d0 msgr2=0x7f36b81eb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bccde640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36b81592d0 0x7f36b81eb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bccde640 1 -- 172.21.15.26:0/2596713466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b8155220 con 0x7f36b80c6780 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bd4df640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36b806da40 0x7f36b806e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bccde640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b814db10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f36a8000c00 tx=0x7f36a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36adffb640 1 -- 172.21.15.26:0/2596713466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8035040 con 0x7f36b80c6780 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36adffb640 1 -- 172.21.15.26:0/2596713466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a8030d00 con 0x7f36b80c6780 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b806fcd0 con 0x7f36b80c6780 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36adffb640 1 -- 172.21.15.26:0/2596713466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80456a0 con 0x7f36b80c6780 2024-03-06T07:25:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.079+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b814e050 con 0x7f36b80c6780 2024-03-06T07:25:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.080+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b804f330 con 0x7f36b80c6780 2024-03-06T07:25:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.081+0000 7f36adffb640 1 -- 172.21.15.26:0/2596713466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36a8038020 con 0x7f36b80c6780 2024-03-06T07:25:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.081+0000 7f36adffb640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369c06b4d0 0x7f369c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.081+0000 7f36adffb640 1 -- 172.21.15.26:0/2596713466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f36a80ac6d0 con 0x7f36b80c6780 2024-03-06T07:25:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.081+0000 7f36bd4df640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369c06b4d0 0x7f369c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.082+0000 7f36bd4df640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369c06b4d0 0x7f369c06d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009f30 tx=0x7f36b4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.086+0000 7f36adffb640 1 -- 172.21.15.26:0/2596713466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f36a807ac90 con 0x7f36b80c6780 2024-03-06T07:25:53.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.268+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36b8073240 con 0x7f36b80c6780 2024-03-06T07:25:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.269+0000 7f36adffb640 1 -- 172.21.15.26:0/2596713466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+17262 (secure 0 0 0) 0x7f36a807ed30 con 0x7f36b80c6780 2024-03-06T07:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":139,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:51.193380+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:18.406892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:19.110341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.5","mappings":[{"from":3,"to":0}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.117:6834/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.26:6835/1099020571":"2024-03-07T07:25:32.004862+0000","172.21.15.26:6834/1099020571":"2024-03-07T07:25:32.004862+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369c06b4d0 msgr2=0x7f369c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369c06b4d0 0x7f369c06d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009f30 tx=0x7f36b4009190 comp rx=0 tx=0).stop 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 msgr2=0x7f36b814db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b814db10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f36a8000c00 tx=0x7f36a80046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 shutdown_connections 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369c06b4d0 0x7f369c06d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36b81592d0 0x7f36b81eb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80c6780 0x7f36b814db10 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 --2- 172.21.15.26:0/2596713466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36b806da40 0x7f36b806e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 >> 172.21.15.26:0/2596713466 conn(0x7f36b80aa630 msgr2=0x7f36b80c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 shutdown_connections 2024-03-06T07:25:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.271+0000 7f36bf743640 1 -- 172.21.15.26:0/2596713466 wait complete. 2024-03-06T07:25:53.284 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:25:53.284 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.384+0000 7f4929963640 1 Processor -- start 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.384+0000 7f4929963640 1 -- start start 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.384+0000 7f4929963640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4924152920 0x7f4924155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.384+0000 7f4929963640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4924156180 con 0x7f49240c8a00 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.384+0000 7f4929963640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4924159630 con 0x7f49240c5330 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.384+0000 7f4929963640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49241587b0 con 0x7f4924152920 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f491bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f49240c8a00 0x7f4924158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56756/0 (socket says 172.21.15.26:56756) 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f491bfff640 1 -- 172.21.15.26:0/2883487897 learned_addr learned my addr 172.21.15.26:0/2883487897 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f4922ffd640 1 --2- 172.21.15.26:0/2883487897 >> v2:172.21.15.117:3300/0 conn(0x7f4924152920 0x7f4924155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887474891 0 0) 0x7f490c001110 con 0x7f49240c8a00 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4910003520 con 0x7f49240c8a00 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927318602 0 0) 0x7f4924159630 con 0x7f49240c5330 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490c001110 con 0x7f49240c5330 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314004135 0 0) 0x7f4910003520 con 0x7f49240c8a00 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 >> v2:172.21.15.117:3300/0 conn(0x7f4924152920 msgr2=0x7f4924155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 --2- 172.21.15.26:0/2883487897 >> v2:172.21.15.117:3300/0 conn(0x7f4924152920 0x7f4924155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 >> v1:172.21.15.117:6789/0 conn(0x7f49240c5330 legacy=0x7f492415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49240c7760 con 0x7f49240c8a00 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f490c001840 con 0x7f49240c8a00 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 >> v1:172.21.15.26:6789/0 conn(0x7f49240c8a00 legacy=0x7f4924158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910003b40 0x7f4910003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4910004800 0x7f4910004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49100051c0 0x7f491000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49241587b0 con 0x7f4910004800 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491000c0f0 con 0x7f49100051c0 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491000c2f0 con 0x7f4910003b40 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f4922ffd640 1 --2- 172.21.15.26:0/2883487897 >> v2:172.21.15.117:3300/0 conn(0x7f4924152920 0x7f4924155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f491bfff640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4910004800 0x7f4910004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f49237fe640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49100051c0 0x7f491000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.385+0000 7f4922ffd640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910003b40 0x7f4910003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f491bfff640 1 -- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910003b40 msgr2=0x7f4910003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f491bfff640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910003b40 0x7f4910003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f491bfff640 1 -- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49100051c0 msgr2=0x7f491000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f491bfff640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49100051c0 0x7f491000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f491bfff640 1 -- 172.21.15.26:0/2883487897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910003520 con 0x7f4910004800 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f49237fe640 1 -- 172.21.15.26:0/2883487897 reap_dead start 2024-03-06T07:25:53.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f491bfff640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4910004800 0x7f4910004c80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f490c00c490 tx=0x7f4910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd05d805e158188 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c00c0e0 con 0x7f4910004800 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c030070 con 0x7f4910004800 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.386+0000 7f49227fc640 1 -- 172.21.15.26:0/2883487897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0058d0 con 0x7f4910004800 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.387+0000 7f4929963640 1 -- 172.21.15.26:0/2883487897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4910004800 msgr2=0x7f4910004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.387+0000 7f4929963640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4910004800 0x7f4910004c80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f490c00c490 tx=0x7f4910005830 comp rx=0 tx=0).stop 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.387+0000 7f4929963640 1 -- 172.21.15.26:0/2883487897 shutdown_connections 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.387+0000 7f4929963640 1 --2- 172.21.15.26:0/2883487897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4910004800 0x7f4910004c80 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.387+0000 7f4929963640 1 -- 172.21.15.26:0/2883487897 >> 172.21.15.26:0/2883487897 conn(0x7f49240ac560 msgr2=0x7f49240ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.387+0000 7f4929963640 1 -- 172.21.15.26:0/2883487897 shutdown_connections 2024-03-06T07:25:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.387+0000 7f4929963640 1 -- 172.21.15.26:0/2883487897 wait complete. 2024-03-06T07:25:53.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.388+0000 7f4929963640 1 Processor -- start 2024-03-06T07:25:53.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.388+0000 7f4929963640 1 -- start start 2024-03-06T07:25:53.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f4929963640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910004800 0x7f4924157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f4929963640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4924151ee0 0x7f49241512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f4929963640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49241e7ae0 0x7f49241e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f4929963640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924158a00 con 0x7f49241e7ae0 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f4929963640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49240c7d20 con 0x7f4924151ee0 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f4929963640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492406d030 con 0x7f4910004800 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49241e7ae0 0x7f49241e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49241e7ae0 0x7f49241e7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58660/0 (socket says 172.21.15.26:58660) 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 -- 172.21.15.26:0/2760165546 learned_addr learned my addr 172.21.15.26:0/2760165546 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f4922ffd640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910004800 0x7f4924157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f491bfff640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4924151ee0 0x7f49241512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 -- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910004800 msgr2=0x7f4924157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910004800 0x7f4924157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 -- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4924151ee0 msgr2=0x7f49241512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4924151ee0 0x7f49241512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 -- 172.21.15.26:0/2760165546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49241eb6c0 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.389+0000 7f49237fe640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49241e7ae0 0x7f49241e7f60 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f49240c8960 tx=0x7f491c00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.390+0000 7f4920ff9640 1 -- 172.21.15.26:0/2760165546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c01a070 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.390+0000 7f4920ff9640 1 -- 172.21.15.26:0/2760165546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c003c40 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.390+0000 7f4920ff9640 1 -- 172.21.15.26:0/2760165546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c005070 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.390+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49241ea900 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.390+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49241eaec0 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.391+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492404f330 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.391+0000 7f4920ff9640 1 -- 172.21.15.26:0/2760165546 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f491c01d070 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.392+0000 7f4920ff9640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f490406b500 0x7f490406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.392+0000 7f4920ff9640 1 -- 172.21.15.26:0/2760165546 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f491c089b00 con 0x7f49241e7ae0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.392+0000 7f4922ffd640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f490406b500 0x7f490406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.392+0000 7f4922ffd640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f490406b500 0x7f490406d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f49140093d0 tx=0x7f4914009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.396+0000 7f4920ff9640 1 -- 172.21.15.26:0/2760165546 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f491c058150 con 0x7f49241e7ae0 2024-03-06T07:25:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.612+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f492406cda0 con 0x7f49241e7ae0 2024-03-06T07:25:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.612+0000 7f4920ff9640 1 -- 172.21.15.26:0/2760165546 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4996 (secure 0 0 0) 0x7f491c05c1f0 con 0x7f49241e7ae0 2024-03-06T07:25:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:53.614 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":131,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3312878094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3312878094},{"type":"v1","addr":"172.21.15.117:6836","nonce":3312878094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:49.177870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7840":{"gid":7840,"name":"d","rank":0,"incarnation":128,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7840,"qdb_cluster":[7840]},"id":10}]} 2024-03-06T07:25:53.615 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 131 2024-03-06T07:25:53.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f490406b500 msgr2=0x7f490406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f490406b500 0x7f490406d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f49140093d0 tx=0x7f4914009190 comp rx=0 tx=0).stop 2024-03-06T07:25:53.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49241e7ae0 msgr2=0x7f49241e7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49241e7ae0 0x7f49241e7f60 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f49240c8960 tx=0x7f491c00ff70 comp rx=0 tx=0).stop 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 shutdown_connections 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f490406b500 0x7f490406d9c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49241e7ae0 0x7f49241e7f60 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4924151ee0 0x7f49241512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 --2- 172.21.15.26:0/2760165546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4910004800 0x7f4924157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 >> 172.21.15.26:0/2760165546 conn(0x7f49240ac560 msgr2=0x7f49240aad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 shutdown_connections 2024-03-06T07:25:53.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.614+0000 7f4929963640 1 -- 172.21.15.26:0/2760165546 wait complete. 2024-03-06T07:25:53.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:53.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.726+0000 7f48bbcf4640 1 Processor -- start 2024-03-06T07:25:53.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.726+0000 7f48bbcf4640 1 -- start start 2024-03-06T07:25:53.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48bbcf4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48b40c5480 0x7f48b4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48bbcf4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f48b40c5860 0x7f48b406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48bbcf4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48bbcf4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b406df40 con 0x7f48b40c5480 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48bbcf4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40c5cc0 con 0x7f48b40c5860 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48bbcf4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41591c0 con 0x7f48b41586f0 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48b9a90640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48b40c5480 0x7f48b4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48b9a90640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48b40c5480 0x7f48b4155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58676/0 (socket says 172.21.15.26:58676) 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48b9a90640 1 -- 172.21.15.26:0/134797998 learned_addr learned my addr 172.21.15.26:0/134797998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48ba291640 1 --2- 172.21.15.26:0/134797998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.727+0000 7f48b928f640 1 --2- 172.21.15.26:0/134797998 >> v2:172.21.15.117:3300/0 conn(0x7f48b40c5860 0x7f48b406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48b9a90640 1 -- 172.21.15.26:0/134797998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 msgr2=0x7f48b4158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48b9a90640 1 --2- 172.21.15.26:0/134797998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b4158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48b9a90640 1 -- 172.21.15.26:0/134797998 >> v2:172.21.15.117:3300/0 conn(0x7f48b40c5860 msgr2=0x7f48b406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48b9a90640 1 --2- 172.21.15.26:0/134797998 >> v2:172.21.15.117:3300/0 conn(0x7f48b40c5860 0x7f48b406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48b9a90640 1 -- 172.21.15.26:0/134797998 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b41594a0 con 0x7f48b40c5480 2024-03-06T07:25:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48ba291640 1 --2- 172.21.15.26:0/134797998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48b928f640 1 --2- 172.21.15.26:0/134797998 >> v2:172.21.15.117:3300/0 conn(0x7f48b40c5860 0x7f48b406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.728+0000 7f48b9a90640 1 --2- 172.21.15.26:0/134797998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b4155ba0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c9770 tx=0x7f48a402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8f8ad0a9892f8b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48b8a8e640 1 -- 172.21.15.26:0/134797998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a402c9e0 con 0x7f48b40c5480 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48b8a8e640 1 -- 172.21.15.26:0/134797998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a402cb80 con 0x7f48b40c5480 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48b8a8e640 1 -- 172.21.15.26:0/134797998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40343f0 con 0x7f48b40c5480 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 -- 172.21.15.26:0/134797998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 msgr2=0x7f48b4155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/134797998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b4155ba0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c9770 tx=0x7f48a402be40 comp rx=0 tx=0).stop 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 -- 172.21.15.26:0/134797998 shutdown_connections 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/134797998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/134797998 >> v2:172.21.15.117:3300/0 conn(0x7f48b40c5860 0x7f48b406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/134797998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b4155ba0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 -- 172.21.15.26:0/134797998 >> 172.21.15.26:0/134797998 conn(0x7f48b406b1a0 msgr2=0x7f48b406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 -- 172.21.15.26:0/134797998 shutdown_connections 2024-03-06T07:25:53.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.729+0000 7f48bbcf4640 1 -- 172.21.15.26:0/134797998 wait complete. 2024-03-06T07:25:53.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.730+0000 7f48bbcf4640 1 Processor -- start 2024-03-06T07:25:53.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.730+0000 7f48bbcf4640 1 -- start start 2024-03-06T07:25:53.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48bbcf4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b40aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48bbcf4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40c5860 0x7f48b4156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48bbcf4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b41ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48bbcf4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4152a20 con 0x7f48b40c5480 2024-03-06T07:25:53.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48bbcf4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4155680 con 0x7f48b40c5860 2024-03-06T07:25:53.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48bbcf4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4155050 con 0x7f48b41586f0 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b40aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b40aca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58692/0 (socket says 172.21.15.26:58692) 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 -- 172.21.15.26:0/2526307315 learned_addr learned my addr 172.21.15.26:0/2526307315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b928f640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40c5860 0x7f48b4156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48ba291640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b41ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 -- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 msgr2=0x7f48b41ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b41ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 -- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40c5860 msgr2=0x7f48b4156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40c5860 0x7f48b4156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.731+0000 7f48b9a90640 1 -- 172.21.15.26:0/2526307315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b41eee30 con 0x7f48b40c5480 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.732+0000 7f48b9a90640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b40aca00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f48a4036040 tx=0x7f48a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.732+0000 7f48b27fc640 1 -- 172.21.15.26:0/2526307315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4043020 con 0x7f48b40c5480 2024-03-06T07:25:53.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.732+0000 7f48b27fc640 1 -- 172.21.15.26:0/2526307315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a403e450 con 0x7f48b40c5480 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.732+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b40ad2f0 con 0x7f48b40c5480 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.732+0000 7f48b27fc640 1 -- 172.21.15.26:0/2526307315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4009510 con 0x7f48b40c5480 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.732+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b4156980 con 0x7f48b40c5480 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.733+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b40ad100 con 0x7f48b40c5480 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.733+0000 7f48b27fc640 1 -- 172.21.15.26:0/2526307315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48a4046020 con 0x7f48b40c5480 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.734+0000 7f48b27fc640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48a006b500 0x7f48a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.734+0000 7f48b27fc640 1 -- 172.21.15.26:0/2526307315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f48a40ac9d0 con 0x7f48b40c5480 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.734+0000 7f48b928f640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48a006b500 0x7f48a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.734+0000 7f48b928f640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48a006b500 0x7f48a006d9c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009560 tx=0x7f48a8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.738+0000 7f48b27fc640 1 -- 172.21.15.26:0/2526307315 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f48a404d030 con 0x7f48b40c5480 2024-03-06T07:25:53.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.956+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48b404f330 con 0x7f48b40c5480 2024-03-06T07:25:53.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.957+0000 7f48b27fc640 1 -- 172.21.15.26:0/2526307315 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4996 (secure 0 0 0) 0x7f48a407af90 con 0x7f48b40c5480 2024-03-06T07:25:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:53.959 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":131,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8101,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3312878094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3312878094},{"type":"v1","addr":"172.21.15.117:6836","nonce":3312878094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:20.149307+0000","modified":"2024-03-06T07:25:49.177870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7840":{"gid":7840,"name":"d","rank":0,"incarnation":128,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6837/3380233594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3380233594},{"type":"v1","addr":"172.21.15.117:6837","nonce":3380233594}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7840,"qdb_cluster":[7840]},"id":10}]} 2024-03-06T07:25:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 131 2024-03-06T07:25:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48a006b500 msgr2=0x7f48a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48a006b500 0x7f48a006d9c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009560 tx=0x7f48a8009190 comp rx=0 tx=0).stop 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 msgr2=0x7f48b40aca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b40aca00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f48a4036040 tx=0x7f48a4004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 shutdown_connections 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48a006b500 0x7f48a006d9c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b41586f0 0x7f48b41ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40c5860 0x7f48b4156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 --2- 172.21.15.26:0/2526307315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b40c5480 0x7f48b40aca00 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 >> 172.21.15.26:0/2526307315 conn(0x7f48b406b1a0 msgr2=0x7f48b40c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 shutdown_connections 2024-03-06T07:25:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:53.959+0000 7f48bbcf4640 1 -- 172.21.15.26:0/2526307315 wait complete. 2024-03-06T07:25:53.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.071+0000 7f6cf5f25640 1 Processor -- start 2024-03-06T07:25:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.071+0000 7f6cf5f25640 1 -- start start 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cf5f25640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6cf0072720 0x7f6cf0156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cf5f25640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf0157340 con 0x7f6cf006f7b0 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cf5f25640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf0156d80 con 0x7f6cf01508c0 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cf5f25640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00c3740 con 0x7f6cf0072720 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6ceeffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6cf006f7b0 0x7f6cf01501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56768/0 (socket says 172.21.15.26:56768) 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6ceeffd640 1 -- 172.21.15.26:0/3159571108 learned_addr learned my addr 172.21.15.26:0/3159571108 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cef7fe640 1 --2- 172.21.15.26:0/3159571108 >> v2:172.21.15.117:3300/0 conn(0x7f6cf0072720 0x7f6cf0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cee7fc640 1 -- 172.21.15.26:0/3159571108 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189429494 0 0) 0x7f6cf0157340 con 0x7f6cf006f7b0 2024-03-06T07:25:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cee7fc640 1 -- 172.21.15.26:0/3159571108 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd8003520 con 0x7f6cf006f7b0 2024-03-06T07:25:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cee7fc640 1 -- 172.21.15.26:0/3159571108 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087406930 0 0) 0x7f6cf0156d80 con 0x7f6cf01508c0 2024-03-06T07:25:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cee7fc640 1 -- 172.21.15.26:0/3159571108 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf0157340 con 0x7f6cf01508c0 2024-03-06T07:25:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cef7fe640 1 -- 172.21.15.26:0/3159571108 >> v1:172.21.15.117:6789/0 conn(0x7f6cf01508c0 legacy=0x7f6cf00c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cef7fe640 1 -- 172.21.15.26:0/3159571108 >> v1:172.21.15.26:6789/0 conn(0x7f6cf006f7b0 legacy=0x7f6cf01501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.072+0000 7f6cef7fe640 1 -- 172.21.15.26:0/3159571108 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0157060 con 0x7f6cf0072720 2024-03-06T07:25:54.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.073+0000 7f6cef7fe640 1 --2- 172.21.15.26:0/3159571108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0072720 0x7f6cf0156430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00c38c0 tx=0x7f6ce002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d56bd282dde74e46 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.073+0000 7f6cee7fc640 1 -- 172.21.15.26:0/3159571108 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0035020 con 0x7f6cf0072720 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.073+0000 7f6cee7fc640 1 -- 172.21.15.26:0/3159571108 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0031830 con 0x7f6cf0072720 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.073+0000 7f6cee7fc640 1 -- 172.21.15.26:0/3159571108 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0031be0 con 0x7f6cf0072720 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.074+0000 7f6cf5f25640 1 -- 172.21.15.26:0/3159571108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0072720 msgr2=0x7f6cf0156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.074+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/3159571108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0072720 0x7f6cf0156430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00c38c0 tx=0x7f6ce002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.074+0000 7f6cf5f25640 1 -- 172.21.15.26:0/3159571108 shutdown_connections 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.074+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/3159571108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0072720 0x7f6cf0156430 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.074+0000 7f6cf5f25640 1 -- 172.21.15.26:0/3159571108 >> 172.21.15.26:0/3159571108 conn(0x7f6cf00aa630 msgr2=0x7f6cf00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.074+0000 7f6cf5f25640 1 -- 172.21.15.26:0/3159571108 shutdown_connections 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.074+0000 7f6cf5f25640 1 -- 172.21.15.26:0/3159571108 wait complete. 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.075+0000 7f6cf5f25640 1 Processor -- start 2024-03-06T07:25:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.075+0000 7f6cf5f25640 1 -- start start 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf006f7b0 0x7f6cf00c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cf0072720 0x7f6cf00ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf01508c0 0x7f6cf00ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0156af0 con 0x7f6cf01508c0 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0156970 con 0x7f6cf006f7b0 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0072b00 con 0x7f6cf0072720 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf01508c0 0x7f6cf00ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf01508c0 0x7f6cf00ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58700/0 (socket says 172.21.15.26:58700) 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 -- 172.21.15.26:0/1175205028 learned_addr learned my addr 172.21.15.26:0/1175205028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cef7fe640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf006f7b0 0x7f6cf00c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceeffd640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cf0072720 0x7f6cf00ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 -- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cf0072720 msgr2=0x7f6cf00ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cf0072720 0x7f6cf00ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 -- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf006f7b0 msgr2=0x7f6cf00c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf006f7b0 0x7f6cf00c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 -- 172.21.15.26:0/1175205028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00c7630 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6ceffff640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf01508c0 0x7f6cf00ad7c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c760 tx=0x7f6ce800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cd7fff640 1 -- 172.21.15.26:0/1175205028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8003b70 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cd7fff640 1 -- 172.21.15.26:0/1175205028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce8003d10 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cd7fff640 1 -- 172.21.15.26:0/1175205028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8005310 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf01ecea0 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.076+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf00aca80 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.077+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf004f330 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.078+0000 7f6cd7fff640 1 -- 172.21.15.26:0/1175205028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ce80054b0 con 0x7f6cf01508c0 2024-03-06T07:25:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.078+0000 7f6cd7fff640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.078+0000 7f6cd7fff640 1 -- 172.21.15.26:0/1175205028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f6ce808a1e0 con 0x7f6cf01508c0 2024-03-06T07:25:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.078+0000 7f6cef7fe640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.079+0000 7f6cef7fe640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0036000 tx=0x7f6ce0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.082+0000 7f6cd7fff640 1 -- 172.21.15.26:0/1175205028 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ce80587a0 con 0x7f6cf01508c0 2024-03-06T07:25:54.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.267+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cf0154490 con 0x7f6cf01508c0 2024-03-06T07:25:54.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.268+0000 7f6cd7fff640 1 -- 172.21.15.26:0/1175205028 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+17262 (secure 0 0 0) 0x7f6ce805c840 con 0x7f6cf01508c0 2024-03-06T07:25:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:54.270 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":139,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:51.193380+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:18.406892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:19.110341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.5","mappings":[{"from":3,"to":0}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.117:6834/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.26:6835/1099020571":"2024-03-07T07:25:32.004862+0000","172.21.15.26:6834/1099020571":"2024-03-07T07:25:32.004862+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cdc06b4d0 msgr2=0x7f6cdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0036000 tx=0x7f6ce0008030 comp rx=0 tx=0).stop 2024-03-06T07:25:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf01508c0 msgr2=0x7f6cf00ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf01508c0 0x7f6cf00ad7c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c760 tx=0x7f6ce800cc20 comp rx=0 tx=0).stop 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 shutdown_connections 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf01508c0 0x7f6cf00ad7c0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6cf0072720 0x7f6cf00ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 --2- 172.21.15.26:0/1175205028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf006f7b0 0x7f6cf00c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 >> 172.21.15.26:0/1175205028 conn(0x7f6cf00aa630 msgr2=0x7f6cf014c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 shutdown_connections 2024-03-06T07:25:54.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.270+0000 7f6cf5f25640 1 -- 172.21.15.26:0/1175205028 wait complete. 2024-03-06T07:25:54.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:25:54.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.384+0000 7fa4f85c7640 1 Processor -- start 2024-03-06T07:25:54.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.384+0000 7fa4f85c7640 1 -- start start 2024-03-06T07:25:54.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.384+0000 7fa4f85c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f01571e0 0x7fa4f006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.384+0000 7fa4f85c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa4f00c20b0 0x7fa4f0153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.384+0000 7fa4f85c7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f006fa80 con 0x7fa4f00a9ea0 2024-03-06T07:25:54.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.384+0000 7fa4f85c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f0153100 con 0x7fa4f00c20b0 2024-03-06T07:25:54.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.384+0000 7fa4f85c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00a75d0 con 0x7fa4f01571e0 2024-03-06T07:25:54.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6363640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f01571e0 0x7fa4f006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6363640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f01571e0 0x7fa4f006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:33382/0 (socket says 172.21.15.26:33382) 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6363640 1 -- 172.21.15.26:0/3545995279 learned_addr learned my addr 172.21.15.26:0/3545995279 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6b64640 1 --1- 172.21.15.26:0/3545995279 >> v1:172.21.15.117:6789/0 conn(0x7fa4f00a9ea0 0x7fa4f006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:36830/0 (socket says 172.21.15.26:36830) 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f5b62640 1 --2- 172.21.15.26:0/3545995279 >> v2:172.21.15.117:3300/0 conn(0x7fa4f00c20b0 0x7fa4f0153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6363640 1 -- 172.21.15.26:0/3545995279 >> v2:172.21.15.117:3300/0 conn(0x7fa4f00c20b0 msgr2=0x7fa4f0153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6363640 1 --2- 172.21.15.26:0/3545995279 >> v2:172.21.15.117:3300/0 conn(0x7fa4f00c20b0 0x7fa4f0153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6363640 1 -- 172.21.15.26:0/3545995279 >> v1:172.21.15.117:6789/0 conn(0x7fa4f00a9ea0 legacy=0x7fa4f006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.385+0000 7fa4f6363640 1 -- 172.21.15.26:0/3545995279 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00a78b0 con 0x7fa4f01571e0 2024-03-06T07:25:54.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f6363640 1 --2- 172.21.15.26:0/3545995279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f01571e0 0x7fa4f006afc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009870 tx=0x7fa4e802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2900c3f541416a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f5361640 1 -- 172.21.15.26:0/3545995279 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8035020 con 0x7fa4f01571e0 2024-03-06T07:25:54.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f5361640 1 -- 172.21.15.26:0/3545995279 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8031640 con 0x7fa4f01571e0 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f5361640 1 -- 172.21.15.26:0/3545995279 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e802d040 con 0x7fa4f01571e0 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f85c7640 1 -- 172.21.15.26:0/3545995279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f01571e0 msgr2=0x7fa4f006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/3545995279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f01571e0 0x7fa4f006afc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009870 tx=0x7fa4e802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f85c7640 1 -- 172.21.15.26:0/3545995279 shutdown_connections 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/3545995279 >> v2:172.21.15.117:3300/0 conn(0x7fa4f00c20b0 0x7fa4f0153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/3545995279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f01571e0 0x7fa4f006afc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.386+0000 7fa4f85c7640 1 -- 172.21.15.26:0/3545995279 >> 172.21.15.26:0/3545995279 conn(0x7fa4f00c41b0 msgr2=0x7fa4f00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.387+0000 7fa4f85c7640 1 -- 172.21.15.26:0/3545995279 shutdown_connections 2024-03-06T07:25:54.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.387+0000 7fa4f85c7640 1 -- 172.21.15.26:0/3545995279 wait complete. 2024-03-06T07:25:54.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 Processor -- start 2024-03-06T07:25:54.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 -- start start 2024-03-06T07:25:54.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f00a9ea0 0x7fa4f00c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4f00c20b0 0x7fa4f00c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f01571e0 0x7fa4f0149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00a7e70 con 0x7fa4f01571e0 2024-03-06T07:25:54.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00a72c0 con 0x7fa4f00c20b0 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f85c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f01564c0 con 0x7fa4f00a9ea0 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f6b64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f01571e0 0x7fa4f0149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f01571e0 0x7fa4f0149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58706/0 (socket says 172.21.15.26:58706) 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f5b62640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4f00c20b0 0x7fa4f00c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.388+0000 7fa4f6363640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f00a9ea0 0x7fa4f00c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 -- 172.21.15.26:0/2627656436 learned_addr learned my addr 172.21.15.26:0/2627656436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 -- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f00a9ea0 msgr2=0x7fa4f00c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f00a9ea0 0x7fa4f00c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 -- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4f00c20b0 msgr2=0x7fa4f00c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4f00c20b0 0x7fa4f00c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 -- 172.21.15.26:0/2627656436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00c5860 con 0x7fa4f01571e0 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6363640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f00a9ea0 0x7fa4f00c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f6b64640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f01571e0 0x7fa4f0149d90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00c780 tx=0x7fa4ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4e6ffd640 1 -- 172.21.15.26:0/2627656436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec017070 con 0x7fa4f01571e0 2024-03-06T07:25:54.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4e6ffd640 1 -- 172.21.15.26:0/2627656436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec00a640 con 0x7fa4f01571e0 2024-03-06T07:25:54.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4e6ffd640 1 -- 172.21.15.26:0/2627656436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00a9f0 con 0x7fa4f01571e0 2024-03-06T07:25:54.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f00c5b40 con 0x7fa4f01571e0 2024-03-06T07:25:54.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.389+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f0072050 con 0x7fa4f01571e0 2024-03-06T07:25:54.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.390+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f004f330 con 0x7fa4f01571e0 2024-03-06T07:25:54.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.391+0000 7fa4e6ffd640 1 -- 172.21.15.26:0/2627656436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4ec01a070 con 0x7fa4f01571e0 2024-03-06T07:25:54.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.391+0000 7fa4e6ffd640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4d406b4d0 0x7fa4d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.391+0000 7fa4e6ffd640 1 -- 172.21.15.26:0/2627656436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fa4ec0899a0 con 0x7fa4f01571e0 2024-03-06T07:25:54.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.391+0000 7fa4f6363640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4d406b4d0 0x7fa4d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.391+0000 7fa4f6363640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4d406b4d0 0x7fa4d406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8036000 tx=0x7fa4e803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.395+0000 7fa4e6ffd640 1 -- 172.21.15.26:0/2627656436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa4ec057f60 con 0x7fa4f01571e0 2024-03-06T07:25:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.614+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa4f006fc70 con 0x7fa4f01571e0 2024-03-06T07:25:54.616 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:25:54.615+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:25:54.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.616+0000 7fa4e6ffd640 1 -- 172.21.15.26:0/2627656436 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v132) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa4ec05c000 con 0x7fa4f01571e0 2024-03-06T07:25:54.617 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:25:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4d406b4d0 msgr2=0x7fa4d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4d406b4d0 0x7fa4d406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8036000 tx=0x7fa4e803c040 comp rx=0 tx=0).stop 2024-03-06T07:25:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f01571e0 msgr2=0x7fa4f0149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f01571e0 0x7fa4f0149d90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00c780 tx=0x7fa4ec00cc40 comp rx=0 tx=0).stop 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 shutdown_connections 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4d406b4d0 0x7fa4d406d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f01571e0 0x7fa4f0149d90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4f00c20b0 0x7fa4f00c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 --2- 172.21.15.26:0/2627656436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4f00a9ea0 0x7fa4f00c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 >> 172.21.15.26:0/2627656436 conn(0x7fa4f00c41b0 msgr2=0x7fa4f0155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 shutdown_connections 2024-03-06T07:25:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.618+0000 7fa4f85c7640 1 -- 172.21.15.26:0/2627656436 wait complete. 2024-03-06T07:25:54.625 INFO:tasks.ceph.mds.d.smithi117.stderr: -127> 2024-03-06T07:25:51.070+0000 7fc3deb48640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:54.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:25:54.648 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:25:54.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.732+0000 7faffc7e9640 1 Processor -- start 2024-03-06T07:25:54.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.732+0000 7faffc7e9640 1 -- start start 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faffc7e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff406b3a0 0x7faff4158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faffc7e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff406b800 con 0x7faff4159160 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faffc7e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff40c5840 con 0x7faff406afc0 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faffc7e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff4158790 con 0x7faff406b3a0 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faffad86640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faff4159160 0x7faff415fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56770/0 (socket says 172.21.15.26:56770) 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faffad86640 1 -- 172.21.15.26:0/4223521909 learned_addr learned my addr 172.21.15.26:0/4223521909 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9d84640 1 --2- 172.21.15.26:0/4223521909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff406b3a0 0x7faff4158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9583640 1 -- 172.21.15.26:0/4223521909 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057608319 0 0) 0x7faff406b800 con 0x7faff4159160 2024-03-06T07:25:54.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9583640 1 -- 172.21.15.26:0/4223521909 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe0003520 con 0x7faff4159160 2024-03-06T07:25:54.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9583640 1 -- 172.21.15.26:0/4223521909 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818456729 0 0) 0x7faff40c5840 con 0x7faff406afc0 2024-03-06T07:25:54.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9583640 1 -- 172.21.15.26:0/4223521909 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff406b800 con 0x7faff406afc0 2024-03-06T07:25:54.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9d84640 1 -- 172.21.15.26:0/4223521909 >> v1:172.21.15.117:6789/0 conn(0x7faff406afc0 legacy=0x7faff40c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9d84640 1 -- 172.21.15.26:0/4223521909 >> v1:172.21.15.26:6789/0 conn(0x7faff4159160 legacy=0x7faff415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.733+0000 7faff9d84640 1 -- 172.21.15.26:0/4223521909 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40c5b20 con 0x7faff406b3a0 2024-03-06T07:25:54.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.734+0000 7faff9d84640 1 --2- 172.21.15.26:0/4223521909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff406b3a0 0x7faff4158c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fafe40097d0 tx=0x7fafe402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcf6417aa2b951e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.734+0000 7faff9583640 1 -- 172.21.15.26:0/4223521909 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4035020 con 0x7faff406b3a0 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.734+0000 7faff9583640 1 -- 172.21.15.26:0/4223521909 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe4031800 con 0x7faff406b3a0 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.734+0000 7faff9583640 1 -- 172.21.15.26:0/4223521909 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4031bb0 con 0x7faff406b3a0 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.735+0000 7faffc7e9640 1 -- 172.21.15.26:0/4223521909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff406b3a0 msgr2=0x7faff4158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.735+0000 7faffc7e9640 1 --2- 172.21.15.26:0/4223521909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff406b3a0 0x7faff4158c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fafe40097d0 tx=0x7fafe402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.735+0000 7faffc7e9640 1 -- 172.21.15.26:0/4223521909 shutdown_connections 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.735+0000 7faffc7e9640 1 --2- 172.21.15.26:0/4223521909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff406b3a0 0x7faff4158c20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.735+0000 7faffc7e9640 1 -- 172.21.15.26:0/4223521909 >> 172.21.15.26:0/4223521909 conn(0x7faff40c4220 msgr2=0x7faff40c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.735+0000 7faffc7e9640 1 -- 172.21.15.26:0/4223521909 shutdown_connections 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.735+0000 7faffc7e9640 1 -- 172.21.15.26:0/4223521909 wait complete. 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 Processor -- start 2024-03-06T07:25:54.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 -- start start 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faff406afc0 0x7faff414e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faff406b3a0 0x7faff41eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff4159160 0x7faff4146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff41595f0 con 0x7faff406b3a0 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40c60e0 con 0x7faff406afc0 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.736+0000 7faffc7e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff4155220 con 0x7faff4159160 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faff406b3a0 0x7faff41eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faff406b3a0 0x7faff41eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58714/0 (socket says 172.21.15.26:58714) 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 -- 172.21.15.26:0/1164294110 learned_addr learned my addr 172.21.15.26:0/1164294110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faffad86640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff4159160 0x7faff4146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faffa585640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faff406afc0 0x7faff414e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 -- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff4159160 msgr2=0x7faff4146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff4159160 0x7faff4146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 -- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faff406afc0 msgr2=0x7faff414e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faff406afc0 0x7faff414e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 -- 172.21.15.26:0/1164294110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff41471f0 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faff9d84640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faff406b3a0 0x7faff41eb930 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000c00 tx=0x7fafe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7fafeaffd640 1 -- 172.21.15.26:0/1164294110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4041070 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7fafeaffd640 1 -- 172.21.15.26:0/1164294110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe4031e30 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff40a91d0 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7fafeaffd640 1 -- 172.21.15.26:0/1164294110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4035040 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.737+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41eb0e0 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.738+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff40a8fc0 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.739+0000 7fafeaffd640 1 -- 172.21.15.26:0/1164294110 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafe404c060 con 0x7faff406b3a0 2024-03-06T07:25:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.739+0000 7fafeaffd640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fafd806b4d0 0x7fafd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.739+0000 7fafeaffd640 1 -- 172.21.15.26:0/1164294110 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fafe40abd80 con 0x7faff406b3a0 2024-03-06T07:25:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.739+0000 7faffa585640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fafd806b4d0 0x7fafd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.740+0000 7faffa585640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fafd806b4d0 0x7fafd806d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7faff0007fd0 tx=0x7faff0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.743+0000 7fafeaffd640 1 -- 172.21.15.26:0/1164294110 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fafe407a370 con 0x7faff406b3a0 2024-03-06T07:25:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:54.966+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faff4073590 con 0x7faff406b3a0 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.625+0000 7fafeaffd640 1 -- 172.21.15.26:0/1164294110 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v133) v1 ==== 104+0+0 (secure 0 0 0) 0x7fafe407e410 con 0x7faff406b3a0 2024-03-06T07:25:55.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fafd806b4d0 msgr2=0x7fafd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fafd806b4d0 0x7fafd806d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7faff0007fd0 tx=0x7faff0009040 comp rx=0 tx=0).stop 2024-03-06T07:25:55.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faff406b3a0 msgr2=0x7faff41eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faff406b3a0 0x7faff41eb930 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000c00 tx=0x7fafe40046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 shutdown_connections 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fafd806b4d0 0x7fafd806d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faff4159160 0x7faff4146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faff406b3a0 0x7faff41eb930 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 --2- 172.21.15.26:0/1164294110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faff406afc0 0x7faff414e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 >> 172.21.15.26:0/1164294110 conn(0x7faff40c4220 msgr2=0x7faff40c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.628+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 shutdown_connections 2024-03-06T07:25:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.629+0000 7faffc7e9640 1 -- 172.21.15.26:0/1164294110 wait complete. 2024-03-06T07:25:55.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:25:55.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.753+0000 7f3406981640 1 Processor -- start 2024-03-06T07:25:55.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f3406981640 1 -- start start 2024-03-06T07:25:55.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f3406981640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f340014f180 0x7f34000c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f3406981640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34001551d0 0x7f34001566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f3406981640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34000c8f60 con 0x7f34000ca000 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f3406981640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3400156c00 con 0x7f34001551d0 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f3406981640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34000c9240 con 0x7f340014f180 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f33ff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34001551d0 0x7f34001566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f33ff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34001551d0 0x7f34001566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58720/0 (socket says 172.21.15.26:58720) 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.754+0000 7f33ff7fe640 1 -- 172.21.15.26:0/1775686818 learned_addr learned my addr 172.21.15.26:0/1775686818 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.755+0000 7f33ff7fe640 1 -- 172.21.15.26:0/1775686818 >> v2:172.21.15.117:3300/0 conn(0x7f340014f180 msgr2=0x7f34000c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.755+0000 7f33fffff640 1 --2- 172.21.15.26:0/1775686818 >> v2:172.21.15.117:3300/0 conn(0x7f340014f180 0x7f34000c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.755+0000 7f33ff7fe640 1 --2- 172.21.15.26:0/1775686818 >> v2:172.21.15.117:3300/0 conn(0x7f340014f180 0x7f34000c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.755+0000 7f33ff7fe640 1 -- 172.21.15.26:0/1775686818 >> v1:172.21.15.117:6789/0 conn(0x7f34000ca000 legacy=0x7f340006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.755+0000 7f33ff7fe640 1 -- 172.21.15.26:0/1775686818 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000c9520 con 0x7f34001551d0 2024-03-06T07:25:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.755+0000 7f33fffff640 1 --2- 172.21.15.26:0/1775686818 >> v2:172.21.15.117:3300/0 conn(0x7f340014f180 0x7f34000c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f33ff7fe640 1 --2- 172.21.15.26:0/1775686818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001566c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3400073630 tx=0x7f33f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7680505f6a39dbe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f33feffd640 1 -- 172.21.15.26:0/1775686818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4035020 con 0x7f34001551d0 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f33feffd640 1 -- 172.21.15.26:0/1775686818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4031870 con 0x7f34001551d0 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f33feffd640 1 -- 172.21.15.26:0/1775686818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4031c40 con 0x7f34001551d0 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 -- 172.21.15.26:0/1775686818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 msgr2=0x7f34001566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 --2- 172.21.15.26:0/1775686818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001566c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3400073630 tx=0x7f33f402b910 comp rx=0 tx=0).stop 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 -- 172.21.15.26:0/1775686818 shutdown_connections 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 --2- 172.21.15.26:0/1775686818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001566c0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 --2- 172.21.15.26:0/1775686818 >> v2:172.21.15.117:3300/0 conn(0x7f340014f180 0x7f34000c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 -- 172.21.15.26:0/1775686818 >> 172.21.15.26:0/1775686818 conn(0x7f3400072d20 msgr2=0x7f34000ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 -- 172.21.15.26:0/1775686818 shutdown_connections 2024-03-06T07:25:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.756+0000 7f3406981640 1 -- 172.21.15.26:0/1775686818 wait complete. 2024-03-06T07:25:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.757+0000 7f3406981640 1 Processor -- start 2024-03-06T07:25:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3406981640 1 -- start start 2024-03-06T07:25:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3406981640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34000ca000 0x7f34001521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3406981640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f340014f180 0x7f34001dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3406981640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001e0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3406981640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340014f560 con 0x7f34001551d0 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3406981640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000c9ae0 con 0x7f340014f180 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3406981640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340014f6e0 con 0x7f34000ca000 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3404f1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3404f1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001e0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58732/0 (socket says 172.21.15.26:58732) 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.758+0000 7f3404f1e640 1 -- 172.21.15.26:0/4124682119 learned_addr learned my addr 172.21.15.26:0/4124682119 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f33ff7fe640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f340014f180 0x7f34001dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f33fffff640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34000ca000 0x7f34001521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3404f1e640 1 -- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34000ca000 msgr2=0x7f34001521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3404f1e640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34000ca000 0x7f34001521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3404f1e640 1 -- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f340014f180 msgr2=0x7f34001dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3404f1e640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f340014f180 0x7f34001dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3404f1e640 1 -- 172.21.15.26:0/4124682119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34001eee30 con 0x7f34001551d0 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f33ff7fe640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f340014f180 0x7f34001dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f33fffff640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34000ca000 0x7f34001521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3404f1e640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001e0330 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c7b0 tx=0x7f33f800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f33fcff9640 1 -- 172.21.15.26:0/4124682119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8004b60 con 0x7f34001551d0 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f33fcff9640 1 -- 172.21.15.26:0/4124682119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f8004d00 con 0x7f34001551d0 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34001ef0d0 con 0x7f34001551d0 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f33fcff9640 1 -- 172.21.15.26:0/4124682119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f800aa90 con 0x7f34001551d0 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.759+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34001e2230 con 0x7f34001551d0 2024-03-06T07:25:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.760+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340004f330 con 0x7f34001551d0 2024-03-06T07:25:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.760+0000 7f33fcff9640 1 -- 172.21.15.26:0/4124682119 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33f800ac30 con 0x7f34001551d0 2024-03-06T07:25:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.761+0000 7f33fcff9640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33ec06b4d0 0x7f33ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.761+0000 7f33fcff9640 1 -- 172.21.15.26:0/4124682119 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f33f808aa50 con 0x7f34001551d0 2024-03-06T07:25:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.761+0000 7f33fffff640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33ec06b4d0 0x7f33ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.762+0000 7f33fffff640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33ec06b4d0 0x7f33ec06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006fd0 tx=0x7f33f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.765+0000 7f33fcff9640 1 -- 172.21.15.26:0/4124682119 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33f8059600 con 0x7f34001551d0 2024-03-06T07:25:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:55.946+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34001ef2a0 con 0x7f34001551d0 2024-03-06T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.627+0000 7f33fcff9640 1 -- 172.21.15.26:0/4124682119 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v141) v1 ==== 181+0+0 (secure 0 0 0) 0x7f33f8058f00 con 0x7f34001551d0 2024-03-06T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33ec06b4d0 msgr2=0x7f33ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33ec06b4d0 0x7f33ec06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006fd0 tx=0x7f33f0008040 comp rx=0 tx=0).stop 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 msgr2=0x7f34001e0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001e0330 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c7b0 tx=0x7f33f800cc70 comp rx=0 tx=0).stop 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 shutdown_connections 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33ec06b4d0 0x7f33ec06d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34001551d0 0x7f34001e0330 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f340014f180 0x7f34001dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 --2- 172.21.15.26:0/4124682119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34000ca000 0x7f34001521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 >> 172.21.15.26:0/4124682119 conn(0x7f3400072d20 msgr2=0x7f3400153940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 shutdown_connections 2024-03-06T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.631+0000 7f3406981640 1 -- 172.21.15.26:0/4124682119 wait complete. 2024-03-06T07:25:56.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.755+0000 7ff69dbf7640 1 Processor -- start 2024-03-06T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.755+0000 7ff69dbf7640 1 -- start start 2024-03-06T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.756+0000 7ff69dbf7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6980c69f0 0x7ff6980c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.756+0000 7ff69dbf7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69813e070 con 0x7ff6980c53f0 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.756+0000 7ff69dbf7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69813e420 con 0x7ff69813dc90 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.756+0000 7ff69dbf7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6980c7340 con 0x7ff6980c69f0 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.756+0000 7ff697fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6980c69f0 0x7ff6980c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.756+0000 7ff696ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff6980c53f0 0x7ff69806b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56786/0 (socket says 172.21.15.26:56786) 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.756+0000 7ff696ffd640 1 -- 172.21.15.26:0/2962179995 learned_addr learned my addr 172.21.15.26:0/2962179995 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.757+0000 7ff697fff640 1 -- 172.21.15.26:0/2962179995 >> v1:172.21.15.117:6789/0 conn(0x7ff69813dc90 legacy=0x7ff698159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.757+0000 7ff697fff640 1 -- 172.21.15.26:0/2962179995 >> v1:172.21.15.26:6789/0 conn(0x7ff6980c53f0 legacy=0x7ff69806b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.757+0000 7ff6967fc640 1 -- 172.21.15.26:0/2962179995 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481139000 0 0) 0x7ff69813e070 con 0x7ff6980c53f0 2024-03-06T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.757+0000 7ff697fff640 1 -- 172.21.15.26:0/2962179995 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69813e700 con 0x7ff6980c69f0 2024-03-06T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.757+0000 7ff697fff640 1 --2- 172.21.15.26:0/2962179995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c69f0 0x7ff6980c6e00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff690009d90 tx=0x7ff69002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1db17f066d1218 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff6967fc640 1 -- 172.21.15.26:0/2962179995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690039070 con 0x7ff6980c69f0 2024-03-06T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff6967fc640 1 -- 172.21.15.26:0/2962179995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69002cbd0 con 0x7ff6980c69f0 2024-03-06T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff6967fc640 1 -- 172.21.15.26:0/2962179995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690034440 con 0x7ff6980c69f0 2024-03-06T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff69dbf7640 1 -- 172.21.15.26:0/2962179995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c69f0 msgr2=0x7ff6980c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/2962179995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c69f0 0x7ff6980c6e00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff690009d90 tx=0x7ff69002be20 comp rx=0 tx=0).stop 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff69dbf7640 1 -- 172.21.15.26:0/2962179995 shutdown_connections 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/2962179995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c69f0 0x7ff6980c6e00 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff69dbf7640 1 -- 172.21.15.26:0/2962179995 >> 172.21.15.26:0/2962179995 conn(0x7ff6980caa70 msgr2=0x7ff6980cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff69dbf7640 1 -- 172.21.15.26:0/2962179995 shutdown_connections 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.758+0000 7ff69dbf7640 1 -- 172.21.15.26:0/2962179995 wait complete. 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.759+0000 7ff69dbf7640 1 Processor -- start 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.759+0000 7ff69dbf7640 1 -- start start 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff69dbf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c53f0 0x7ff6981579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff69dbf7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6980c69f0 0x7ff698156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff69dbf7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff69813dc90 0x7ff698156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff69dbf7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69806bde0 con 0x7ff6980c53f0 2024-03-06T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff69dbf7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980c61f0 con 0x7ff6980c69f0 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff69dbf7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69806b8d0 con 0x7ff69813dc90 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff6977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c53f0 0x7ff6981579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff6977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c53f0 0x7ff6981579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58742/0 (socket says 172.21.15.26:58742) 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff6977fe640 1 -- 172.21.15.26:0/1072100355 learned_addr learned my addr 172.21.15.26:0/1072100355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff696ffd640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6980c69f0 0x7ff698156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff697fff640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff69813dc90 0x7ff698156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff6977fe640 1 -- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff69813dc90 msgr2=0x7ff698156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff6977fe640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff69813dc90 0x7ff698156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff6977fe640 1 -- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6980c69f0 msgr2=0x7ff698156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.760+0000 7ff6977fe640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6980c69f0 0x7ff698156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff6977fe640 1 -- 172.21.15.26:0/1072100355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6981edf80 con 0x7ff6980c53f0 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff697fff640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff69813dc90 0x7ff698156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff6977fe640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c53f0 0x7ff6981579e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b3f0 tx=0x7ff68800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff67ffff640 1 -- 172.21.15.26:0/1072100355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688015020 con 0x7ff6980c53f0 2024-03-06T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff67ffff640 1 -- 172.21.15.26:0/1072100355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff688011830 con 0x7ff6980c53f0 2024-03-06T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6981ee260 con 0x7ff6980c53f0 2024-03-06T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff67ffff640 1 -- 172.21.15.26:0/1072100355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688011c20 con 0x7ff6980c53f0 2024-03-06T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.761+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6981ee820 con 0x7ff6980c53f0 2024-03-06T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.762+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69804f330 con 0x7ff6980c53f0 2024-03-06T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.762+0000 7ff67ffff640 1 -- 172.21.15.26:0/1072100355 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff688018020 con 0x7ff6980c53f0 2024-03-06T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.763+0000 7ff67ffff640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff68406b4d0 0x7ff68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.763+0000 7ff67ffff640 1 -- 172.21.15.26:0/1072100355 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5979+0+0 (secure 0 0 0) 0x7ff688089fe0 con 0x7ff6980c53f0 2024-03-06T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.763+0000 7ff696ffd640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff68406b4d0 0x7ff68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.764+0000 7ff696ffd640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff68406b4d0 0x7ff68406d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff69813cbf0 tx=0x7ff68c007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.767+0000 7ff67ffff640 1 -- 172.21.15.26:0/1072100355 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff68801f030 con 0x7ff6980c53f0 2024-03-06T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:56.950+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff6980c6f00 con 0x7ff6980c53f0 2024-03-06T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.631+0000 7ff67ffff640 1 -- 172.21.15.26:0/1072100355 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v142) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff6880172c0 con 0x7ff6980c53f0 2024-03-06T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff68406b4d0 msgr2=0x7ff68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff68406b4d0 0x7ff68406d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff69813cbf0 tx=0x7ff68c007b50 comp rx=0 tx=0).stop 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c53f0 msgr2=0x7ff6981579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c53f0 0x7ff6981579e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b3f0 tx=0x7ff68800b8b0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 shutdown_connections 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff68406b4d0 0x7ff68406d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff69813dc90 0x7ff698156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6980c69f0 0x7ff698156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 --2- 172.21.15.26:0/1072100355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6980c53f0 0x7ff6981579e0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 >> 172.21.15.26:0/1072100355 conn(0x7ff6980caa70 msgr2=0x7ff6980c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 shutdown_connections 2024-03-06T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.634+0000 7ff69dbf7640 1 -- 172.21.15.26:0/1072100355 wait complete. 2024-03-06T07:25:57.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.757+0000 7f1caef1e640 1 Processor -- start 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.757+0000 7f1caef1e640 1 -- start start 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1caef1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 0x7f1ca80c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1caef1e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ca80c58b0 0x7f1ca8155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1caef1e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca81596f0 con 0x7f1ca80c88f0 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1caef1e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca80c9130 con 0x7f1ca80c58b0 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1caef1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8159260 con 0x7f1ca81527f0 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ca80c88f0 0x7f1ca8155210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37974/0 (socket says 172.21.15.26:37974) 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca7fff640 1 -- 172.21.15.26:0/299954393 learned_addr learned my addr 172.21.15.26:0/299954393 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365786946 0 0) 0x7f1ca81596f0 con 0x7f1ca80c88f0 2024-03-06T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1cad4bb640 1 --2- 172.21.15.26:0/299954393 >> v2:172.21.15.117:3300/0 conn(0x7f1ca80c58b0 0x7f1ca8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1caccba640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 0x7f1ca80c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c90003520 con 0x7f1ca80c88f0 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2345583931 0 0) 0x7f1c90003520 con 0x7f1ca80c88f0 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 msgr2=0x7f1ca80c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 0x7f1ca80c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 >> v2:172.21.15.117:3300/0 conn(0x7f1ca80c58b0 msgr2=0x7f1ca8155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 --2- 172.21.15.26:0/299954393 >> v2:172.21.15.117:3300/0 conn(0x7f1ca80c58b0 0x7f1ca8155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.758+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca80c82a0 con 0x7f1ca80c88f0 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f1c9c001670 con 0x7f1ca80c88f0 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 >> v1:172.21.15.26:6789/0 conn(0x7f1ca80c88f0 legacy=0x7f1ca8155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c900039b0 0x7f1c90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 0x7f1c90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c90004d40 0x7f1c900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca80c9130 con 0x7f1c90004d40 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8159260 con 0x7f1c900039b0 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9000ba40 con 0x7f1ca81527f0 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1cad4bb640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c90004d40 0x7f1c900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1caccba640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c900039b0 0x7f1c90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1ca7fff640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 0x7f1c90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1caccba640 1 -- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 msgr2=0x7f1c90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1caccba640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca81527f0 0x7f1c90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1caccba640 1 -- 172.21.15.26:0/299954393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c90004d40 msgr2=0x7f1c900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1caccba640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c90004d40 0x7f1c900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1caccba640 1 -- 172.21.15.26:0/299954393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90003520 con 0x7f1c900039b0 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.759+0000 7f1cad4bb640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c90004d40 0x7f1c900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.760+0000 7f1caccba640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c900039b0 0x7f1c90003dd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009fc0 tx=0x7f1c90005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53268e95441d139b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.760+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0039070 con 0x7f1c900039b0 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.760+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca002cc20 con 0x7f1c900039b0 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.760+0000 7f1ca77fe640 1 -- 172.21.15.26:0/299954393 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0009c10 con 0x7f1c900039b0 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.761+0000 7f1caef1e640 1 -- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c900039b0 msgr2=0x7f1c90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.761+0000 7f1caef1e640 1 --2- 172.21.15.26:0/299954393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c900039b0 0x7f1c90003dd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009fc0 tx=0x7f1c90005830 comp rx=0 tx=0).stop 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.761+0000 7f1cad4bb640 1 -- 172.21.15.26:0/299954393 reap_dead start 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.761+0000 7f1caef1e640 1 -- 172.21.15.26:0/299954393 shutdown_connections 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.761+0000 7f1caef1e640 1 -- 172.21.15.26:0/299954393 >> 172.21.15.26:0/299954393 conn(0x7f1ca80ac580 msgr2=0x7f1ca80ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.761+0000 7f1caef1e640 1 -- 172.21.15.26:0/299954393 shutdown_connections 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.761+0000 7f1caef1e640 1 -- 172.21.15.26:0/299954393 wait complete. 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.762+0000 7f1caef1e640 1 Processor -- start 2024-03-06T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.762+0000 7f1caef1e640 1 -- start start 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c900039b0 0x7f1ca81eb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca81557a0 0x7f1ca80ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca8154d50 0x7f1ca81ec020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8158c10 con 0x7f1ca81557a0 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca80c6b90 con 0x7f1ca8154d50 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8158ef0 con 0x7f1c900039b0 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca81557a0 0x7f1ca80ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca81557a0 0x7f1ca80ab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48382/0 (socket says 172.21.15.26:48382) 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 -- 172.21.15.26:0/23569285 learned_addr learned my addr 172.21.15.26:0/23569285 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1cad4bb640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca8154d50 0x7f1ca81ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caccba640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c900039b0 0x7f1ca81eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 -- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c900039b0 msgr2=0x7f1ca81eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c900039b0 0x7f1ca81eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 -- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca8154d50 msgr2=0x7f1ca81ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca8154d50 0x7f1ca81ec020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 -- 172.21.15.26:0/23569285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8155c30 con 0x7f1ca81557a0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caccba640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c900039b0 0x7f1ca81eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca7fff640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca81557a0 0x7f1ca80ab790 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c007340 tx=0x7f1c9c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca57fa640 1 -- 172.21.15.26:0/23569285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00f860 con 0x7f1ca81557a0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca57fa640 1 -- 172.21.15.26:0/23569285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c018070 con 0x7f1ca81557a0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca8154270 con 0x7f1ca81557a0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1ca57fa640 1 -- 172.21.15.26:0/23569285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00fc20 con 0x7f1ca81557a0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.763+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca8155520 con 0x7f1ca81557a0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.764+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca804f330 con 0x7f1ca81557a0 2024-03-06T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.765+0000 7f1ca57fa640 1 -- 172.21.15.26:0/23569285 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c9c01d460 con 0x7f1ca81557a0 2024-03-06T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.765+0000 7f1ca57fa640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c9406b500 0x7f1c9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.766+0000 7f1ca57fa640 1 -- 172.21.15.26:0/23569285 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f1c9c08a090 con 0x7f1ca81557a0 2024-03-06T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.766+0000 7f1caccba640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c9406b500 0x7f1c9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.766+0000 7f1caccba640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c9406b500 0x7f1c9406d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0036000 tx=0x7f1ca002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.770+0000 7f1ca57fa640 1 -- 172.21.15.26:0/23569285 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1c9c007b70 con 0x7f1ca81557a0 2024-03-06T07:25:57.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.950+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1ca80c8080 con 0x7f1ca81557a0 2024-03-06T07:25:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.953+0000 7f1ca57fa640 1 -- 172.21.15.26:0/23569285 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1c9c058a10 con 0x7f1ca81557a0 2024-03-06T07:25:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c9406b500 msgr2=0x7f1c9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c9406b500 0x7f1c9406d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0036000 tx=0x7f1ca002e000 comp rx=0 tx=0).stop 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca81557a0 msgr2=0x7f1ca80ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca81557a0 0x7f1ca80ab790 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c007340 tx=0x7f1c9c007800 comp rx=0 tx=0).stop 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 shutdown_connections 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c9406b500 0x7f1c9406d9c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca8154d50 0x7f1ca81ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca81557a0 0x7f1ca80ab790 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 --2- 172.21.15.26:0/23569285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c900039b0 0x7f1ca81eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 >> 172.21.15.26:0/23569285 conn(0x7f1ca80ac580 msgr2=0x7f1ca8150880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 shutdown_connections 2024-03-06T07:25:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:57.955+0000 7f1caef1e640 1 -- 172.21.15.26:0/23569285 wait complete. 2024-03-06T07:25:57.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:25:58.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.069+0000 7f0aa926f640 1 Processor -- start 2024-03-06T07:25:58.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.069+0000 7f0aa926f640 1 -- start start 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa926f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa40b01e0 0x7f0aa4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa926f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0aa406c170 0x7f0aa406c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa926f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa40c7e00 con 0x7f0aa41565b0 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa926f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa40c7840 con 0x7f0aa406c170 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa926f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40c7b20 con 0x7f0aa40b01e0 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0aa406c170 0x7f0aa406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa40b01e0 0x7f0aa4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0a9bfff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0aa41565b0 0x7f0aa40c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52768/0 (socket says 172.21.15.26:52768) 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0aa406c170 0x7f0aa406c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42830/0 (socket says 172.21.15.26:42830) 2024-03-06T07:25:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0a9bfff640 1 -- 172.21.15.26:0/1634910062 learned_addr learned my addr 172.21.15.26:0/1634910062 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa37fe640 1 -- 172.21.15.26:0/1634910062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa40b01e0 msgr2=0x7f0aa4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa37fe640 1 --2- 172.21.15.26:0/1634910062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa40b01e0 0x7f0aa4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa37fe640 1 -- 172.21.15.26:0/1634910062 >> v1:172.21.15.117:6789/0 conn(0x7f0aa41565b0 legacy=0x7f0aa40c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.070+0000 7f0aa37fe640 1 -- 172.21.15.26:0/1634910062 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa406b970 con 0x7f0aa406c170 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.071+0000 7f0aa37fe640 1 --2- 172.21.15.26:0/1634910062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa406c170 0x7f0aa406c620 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009d90 tx=0x7f0a9c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e14a0466031fb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.071+0000 7f0aa27fc640 1 -- 172.21.15.26:0/1634910062 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c039070 con 0x7f0aa406c170 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.071+0000 7f0aa27fc640 1 -- 172.21.15.26:0/1634910062 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c02cc50 con 0x7f0aa406c170 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.071+0000 7f0aa27fc640 1 -- 172.21.15.26:0/1634910062 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c034440 con 0x7f0aa406c170 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 -- 172.21.15.26:0/1634910062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa406c170 msgr2=0x7f0aa406c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 --2- 172.21.15.26:0/1634910062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa406c170 0x7f0aa406c620 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009d90 tx=0x7f0a9c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 -- 172.21.15.26:0/1634910062 shutdown_connections 2024-03-06T07:25:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 --2- 172.21.15.26:0/1634910062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa406c170 0x7f0aa406c620 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 --2- 172.21.15.26:0/1634910062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa40b01e0 0x7f0aa4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 -- 172.21.15.26:0/1634910062 >> 172.21.15.26:0/1634910062 conn(0x7f0aa40ac600 msgr2=0x7f0aa40aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 -- 172.21.15.26:0/1634910062 shutdown_connections 2024-03-06T07:25:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.072+0000 7f0aa926f640 1 -- 172.21.15.26:0/1634910062 wait complete. 2024-03-06T07:25:58.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.073+0000 7f0aa926f640 1 Processor -- start 2024-03-06T07:25:58.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.073+0000 7f0aa926f640 1 -- start start 2024-03-06T07:25:58.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa926f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa406c170 0x7f0aa40c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa926f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa40b01e0 0x7f0aa41ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa926f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa41565b0 0x7f0aa4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa926f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40c5c60 con 0x7f0aa406c170 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa926f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40c5ae0 con 0x7f0aa40b01e0 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa926f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40be250 con 0x7f0aa41565b0 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa406c170 0x7f0aa40c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa406c170 0x7f0aa40c0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48388/0 (socket says 172.21.15.26:48388) 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/3963724835 learned_addr learned my addr 172.21.15.26:0/3963724835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa37fe640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa41565b0 0x7f0aa4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0a9bfff640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa40b01e0 0x7f0aa41ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa41565b0 msgr2=0x7f0aa4151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa41565b0 0x7f0aa4151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa40b01e0 msgr2=0x7f0aa41ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa40b01e0 0x7f0aa41ee4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/3963724835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40c0f90 con 0x7f0aa406c170 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0a9bfff640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa40b01e0 0x7f0aa41ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.074+0000 7f0aa2ffd640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa406c170 0x7f0aa40c0a50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400e9e0 tx=0x7f0a9400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.075+0000 7f0a9b7fe640 1 -- 172.21.15.26:0/3963724835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94017070 con 0x7f0aa406c170 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.075+0000 7f0a9b7fe640 1 -- 172.21.15.26:0/3963724835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9400c9d0 con 0x7f0aa406c170 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.075+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa41eede0 con 0x7f0aa406c170 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.075+0000 7f0a9b7fe640 1 -- 172.21.15.26:0/3963724835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400cdc0 con 0x7f0aa406c170 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.075+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa40c4170 con 0x7f0aa406c170 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.076+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa40c3c90 con 0x7f0aa406c170 2024-03-06T07:25:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.076+0000 7f0a9b7fe640 1 -- 172.21.15.26:0/3963724835 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a9401a070 con 0x7f0aa406c170 2024-03-06T07:25:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.077+0000 7f0a9b7fe640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a8406b4d0 0x7f0a8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.077+0000 7f0a9b7fe640 1 -- 172.21.15.26:0/3963724835 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f0a9408a4d0 con 0x7f0aa406c170 2024-03-06T07:25:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.077+0000 7f0a9bfff640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a8406b4d0 0x7f0a8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.077+0000 7f0a9bfff640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a8406b4d0 0x7f0a8406d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0aa41524a0 tx=0x7f0a8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.081+0000 7f0a9b7fe640 1 -- 172.21.15.26:0/3963724835 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0a94058d30 con 0x7f0aa406c170 2024-03-06T07:25:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.265+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7f0aa404f330 con 0x7f0aa406c170 2024-03-06T07:25:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.266+0000 7f0a9b7fe640 1 -- 172.21.15.26:0/3963724835 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v49) v1 ==== 96+0+0 (secure 0 0 0) 0x7f0a9405cdd0 con 0x7f0aa406c170 2024-03-06T07:25:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.267+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a8406b4d0 msgr2=0x7f0a8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.267+0000 7f0aa926f640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a8406b4d0 0x7f0a8406d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0aa41524a0 tx=0x7f0a8c009040 comp rx=0 tx=0).stop 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.267+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa406c170 msgr2=0x7f0aa40c0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.267+0000 7f0aa926f640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa406c170 0x7f0aa40c0a50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400e9e0 tx=0x7f0a9400eea0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 shutdown_connections 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0a8406b4d0 0x7f0a8406d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0aa41565b0 0x7f0aa4151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa40b01e0 0x7f0aa41ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 --2- 172.21.15.26:0/3963724835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa406c170 0x7f0aa40c0a50 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 >> 172.21.15.26:0/3963724835 conn(0x7f0aa40ac600 msgr2=0x7f0aa40ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 shutdown_connections 2024-03-06T07:25:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.268+0000 7f0aa926f640 1 -- 172.21.15.26:0/3963724835 wait complete. 2024-03-06T07:25:58.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:25:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.382+0000 7fe96bf9a640 1 Processor -- start 2024-03-06T07:25:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.382+0000 7fe96bf9a640 1 -- start start 2024-03-06T07:25:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.382+0000 7fe96bf9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe96414ef70 0x7fe96406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.382+0000 7fe96bf9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe964154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.382+0000 7fe96bf9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9640c2790 con 0x7fe9640c1820 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.382+0000 7fe96bf9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9640c3090 con 0x7fe96414ef70 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.382+0000 7fe96bf9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96414f350 con 0x7fe96406cad0 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe969d36640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe96414ef70 0x7fe96406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe969535640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe9640c1820 0x7fe9640c2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37984/0 (socket says 172.21.15.26:37984) 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe969535640 1 -- 172.21.15.26:0/476681486 learned_addr learned my addr 172.21.15.26:0/476681486 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe96a537640 1 --2- 172.21.15.26:0/476681486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe964154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe969d36640 1 -- 172.21.15.26:0/476681486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 msgr2=0x7fe964154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe969d36640 1 --2- 172.21.15.26:0/476681486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe964154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe969d36640 1 -- 172.21.15.26:0/476681486 >> v1:172.21.15.26:6789/0 conn(0x7fe9640c1820 legacy=0x7fe9640c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe969d36640 1 -- 172.21.15.26:0/476681486 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640c1540 con 0x7fe96414ef70 2024-03-06T07:25:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.383+0000 7fe968d34640 1 -- 172.21.15.26:0/476681486 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076942483 0 0) 0x7fe9640c2790 con 0x7fe9640c1820 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe969d36640 1 --2- 172.21.15.26:0/476681486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96414ef70 0x7fe96406c1f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097a0 tx=0x7fe96002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f13db50050d30c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe968d34640 1 -- 172.21.15.26:0/476681486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600315a0 con 0x7fe96414ef70 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe968d34640 1 -- 172.21.15.26:0/476681486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960037070 con 0x7fe96414ef70 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe968d34640 1 -- 172.21.15.26:0/476681486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960031960 con 0x7fe96414ef70 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 -- 172.21.15.26:0/476681486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96414ef70 msgr2=0x7fe96406c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/476681486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96414ef70 0x7fe96406c1f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097a0 tx=0x7fe96002b840 comp rx=0 tx=0).stop 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 -- 172.21.15.26:0/476681486 shutdown_connections 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/476681486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe964154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/476681486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96414ef70 0x7fe96406c1f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 -- 172.21.15.26:0/476681486 >> 172.21.15.26:0/476681486 conn(0x7fe9640c6710 msgr2=0x7fe9640c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 -- 172.21.15.26:0/476681486 shutdown_connections 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.384+0000 7fe96bf9a640 1 -- 172.21.15.26:0/476681486 wait complete. 2024-03-06T07:25:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 Processor -- start 2024-03-06T07:25:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 -- start start 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe9640a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640c1820 0x7fe96413f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe96414ef70 0x7fe96413b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe964155750 con 0x7fe9640c1820 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96406cf60 con 0x7fe96414ef70 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96bf9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe964155170 con 0x7fe96406cad0 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe969535640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640c1820 0x7fe96413f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe969535640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640c1820 0x7fe96413f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48404/0 (socket says 172.21.15.26:48404) 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe969535640 1 -- 172.21.15.26:0/2127228653 learned_addr learned my addr 172.21.15.26:0/2127228653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe96a537640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe96414ef70 0x7fe96413b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.386+0000 7fe969d36640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe9640a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe969535640 1 -- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 msgr2=0x7fe9640a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe969535640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe9640a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe969535640 1 -- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe96414ef70 msgr2=0x7fe96413b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe969535640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe96414ef70 0x7fe96413b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe969535640 1 -- 172.21.15.26:0/2127228653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640a8c50 con 0x7fe9640c1820 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe969535640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640c1820 0x7fe96413f0c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c4d0 tx=0x7fe95400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe95a7fc640 1 -- 172.21.15.26:0/2127228653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954018070 con 0x7fe9640c1820 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9641ecdc0 con 0x7fe9640c1820 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe95a7fc640 1 -- 172.21.15.26:0/2127228653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe954007500 con 0x7fe9640c1820 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe95a7fc640 1 -- 172.21.15.26:0/2127228653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954004a20 con 0x7fe9640c1820 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.387+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9640a91f0 con 0x7fe9640c1820 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.388+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964075300 con 0x7fe9640c1820 2024-03-06T07:25:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.388+0000 7fe95a7fc640 1 -- 172.21.15.26:0/2127228653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe954004bc0 con 0x7fe9640c1820 2024-03-06T07:25:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.389+0000 7fe95a7fc640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94806b4d0 0x7fe94806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.389+0000 7fe95a7fc640 1 -- 172.21.15.26:0/2127228653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe9540893e0 con 0x7fe9640c1820 2024-03-06T07:25:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.389+0000 7fe969d36640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94806b4d0 0x7fe94806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.389+0000 7fe969d36640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94806b4d0 0x7fe94806d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe960036000 tx=0x7fe960008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.393+0000 7fe95a7fc640 1 -- 172.21.15.26:0/2127228653 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe964075300 con 0x7fe9640c1820 2024-03-06T07:25:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.574+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7fe96404f330 con 0x7fe9640c1820 2024-03-06T07:25:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.577+0000 7fe95a7fc640 1 -- 172.21.15.26:0/2127228653 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v50) v1 ==== 96+0+0 (secure 0 0 0) 0x7fe954058330 con 0x7fe9640c1820 2024-03-06T07:25:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94806b4d0 msgr2=0x7fe94806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94806b4d0 0x7fe94806d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe960036000 tx=0x7fe960008030 comp rx=0 tx=0).stop 2024-03-06T07:25:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640c1820 msgr2=0x7fe96413f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640c1820 0x7fe96413f0c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c4d0 tx=0x7fe95400c990 comp rx=0 tx=0).stop 2024-03-06T07:25:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 shutdown_connections 2024-03-06T07:25:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94806b4d0 0x7fe94806d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe96414ef70 0x7fe96413b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640c1820 0x7fe96413f0c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 --2- 172.21.15.26:0/2127228653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96406cad0 0x7fe9640a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 >> 172.21.15.26:0/2127228653 conn(0x7fe9640c6710 msgr2=0x7fe9641546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 shutdown_connections 2024-03-06T07:25:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.579+0000 7fe96bf9a640 1 -- 172.21.15.26:0/2127228653 wait complete. 2024-03-06T07:25:58.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset' 2024-03-06T07:25:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.692+0000 7fac001eb640 1 Processor -- start 2024-03-06T07:25:58.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.692+0000 7fac001eb640 1 -- start start 2024-03-06T07:25:58.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.693+0000 7fac001eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 0x7fabf8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.693+0000 7fac001eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf8070aa0 con 0x7fabf81506b0 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.693+0000 7fac001eb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf8070d80 con 0x7fabf80acc00 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.693+0000 7fac001eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8153e10 con 0x7fabf8153990 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.693+0000 7fabfdf87640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fabf81506b0 0x7fabf80702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37996/0 (socket says 172.21.15.26:37996) 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.693+0000 7fabfdf87640 1 -- 172.21.15.26:0/1586396086 learned_addr learned my addr 172.21.15.26:0/1586396086 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfd786640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 0x7fabf8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77617466 0 0) 0x7fabf8070aa0 con 0x7fabf81506b0 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe4003520 con 0x7fabf81506b0 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089796270 0 0) 0x7fabf8070d80 con 0x7fabf80acc00 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf8070aa0 con 0x7fabf80acc00 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630049608 0 0) 0x7fabe4003520 con 0x7fabf81506b0 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 msgr2=0x7fabf8156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 0x7fabf8156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 >> v1:172.21.15.117:6789/0 conn(0x7fabf80acc00 legacy=0x7fabf80c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf81566a0 con 0x7fabf81506b0 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fabf4001670 con 0x7fabf81506b0 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 >> v1:172.21.15.26:6789/0 conn(0x7fabf81506b0 legacy=0x7fabf80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe4003b70 0x7fabe4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 0x7fabe4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe4004d40 0x7fabe40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8153e10 con 0x7fabe4003b70 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe400ba70 con 0x7fabe4004d40 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe400bc70 con 0x7fabf8153990 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfdf87640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe4003b70 0x7fabe4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfd786640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 0x7fabe4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfe788640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe4004d40 0x7fabe40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfdf87640 1 -- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 msgr2=0x7fabe4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfdf87640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 0x7fabe4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfdf87640 1 -- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe4004d40 msgr2=0x7fabe40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfdf87640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe4004d40 0x7fabe40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfdf87640 1 -- 172.21.15.26:0/1586396086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe4003520 con 0x7fabe4003b70 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfd786640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabf8153990 0x7fabe4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.694+0000 7fabfe788640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe4004d40 0x7fabe40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.695+0000 7fabfdf87640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe4003b70 0x7fabe4003f70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fabf400c210 tx=0x7fabe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c74b070f01f9c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.695+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4007ce0 con 0x7fabe4003b70 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.695+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf4030070 con 0x7fabe4003b70 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.695+0000 7fabfcf85640 1 -- 172.21.15.26:0/1586396086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4005d50 con 0x7fabe4003b70 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.696+0000 7fac001eb640 1 -- 172.21.15.26:0/1586396086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe4003b70 msgr2=0x7fabe4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.696+0000 7fac001eb640 1 --2- 172.21.15.26:0/1586396086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe4003b70 0x7fabe4003f70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fabf400c210 tx=0x7fabe4005830 comp rx=0 tx=0).stop 2024-03-06T07:25:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.696+0000 7fabfe788640 1 -- 172.21.15.26:0/1586396086 reap_dead start 2024-03-06T07:25:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.696+0000 7fac001eb640 1 -- 172.21.15.26:0/1586396086 shutdown_connections 2024-03-06T07:25:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.696+0000 7fac001eb640 1 -- 172.21.15.26:0/1586396086 >> 172.21.15.26:0/1586396086 conn(0x7fabf806b1a0 msgr2=0x7fabf806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.696+0000 7fac001eb640 1 -- 172.21.15.26:0/1586396086 shutdown_connections 2024-03-06T07:25:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.696+0000 7fac001eb640 1 -- 172.21.15.26:0/1586396086 wait complete. 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.697+0000 7fac001eb640 1 Processor -- start 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.697+0000 7fac001eb640 1 -- start start 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabe4003b70 0x7fabf806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8154480 0x7fabf8154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8155e70 0x7fabf80c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8156a10 con 0x7fabf8154480 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8156450 con 0x7fabf8155e70 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80ad070 con 0x7fabe4003b70 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8154480 0x7fabf8154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8154480 0x7fabf8154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48414/0 (socket says 172.21.15.26:48414) 2024-03-06T07:25:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 -- 172.21.15.26:0/289730803 learned_addr learned my addr 172.21.15.26:0/289730803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 -- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabe4003b70 msgr2=0x7fabf806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabe4003b70 0x7fabf806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 -- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8155e70 msgr2=0x7fabf80c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8155e70 0x7fabf80c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 -- 172.21.15.26:0/289730803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf806c610 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabfd786640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8154480 0x7fabf8154e90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fabe8011730 tx=0x7fabe8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabee7fc640 1 -- 172.21.15.26:0/289730803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800f940 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabee7fc640 1 -- 172.21.15.26:0/289730803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe800fae0 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf81553d0 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fabee7fc640 1 -- 172.21.15.26:0/289730803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8015460 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.698+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf814ca80 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.699+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf804f330 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.700+0000 7fabee7fc640 1 -- 172.21.15.26:0/289730803 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabe8015600 con 0x7fabf8154480 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.700+0000 7fabee7fc640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabdc06b500 0x7fabdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.700+0000 7fabee7fc640 1 -- 172.21.15.26:0/289730803 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fabe808aac0 con 0x7fabf8154480 2024-03-06T07:25:58.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.701+0000 7fabfdf87640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabdc06b500 0x7fabdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.701+0000 7fabfdf87640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabdc06b500 0x7fabdc06d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fabf4034000 tx=0x7fabf4039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.705+0000 7fabee7fc640 1 -- 172.21.15.26:0/289730803 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fabe8019030 con 0x7fabf8154480 2024-03-06T07:25:58.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:58.959+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]} v 0) v1 -- 0x7fabf80700a0 con 0x7fabf8154480 2024-03-06T07:25:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.640+0000 7fabee7fc640 1 -- 172.21.15.26:0/289730803 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]}]=0 v344) v1 ==== 144+0+0 (secure 0 0 0) 0x7fabe8059a00 con 0x7fabf8154480 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabdc06b500 msgr2=0x7fabdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabdc06b500 0x7fabdc06d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fabf4034000 tx=0x7fabf4039040 comp rx=0 tx=0).stop 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8154480 msgr2=0x7fabf8154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8154480 0x7fabf8154e90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fabe8011730 tx=0x7fabe8011bf0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 shutdown_connections 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabdc06b500 0x7fabdc06d9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8155e70 0x7fabf80c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8154480 0x7fabf8154e90 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 --2- 172.21.15.26:0/289730803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabe4003b70 0x7fabf806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 >> 172.21.15.26:0/289730803 conn(0x7fabf806b1a0 msgr2=0x7fabf8152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:59.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 shutdown_connections 2024-03-06T07:25:59.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.643+0000 7fac001eb640 1 -- 172.21.15.26:0/289730803 wait complete. 2024-03-06T07:25:59.658 INFO:tasks.cephfs_test_runner:That unsetting mds_join_fs will cause failover if another high-affinity standby exists. ... ok 2024-03-06T07:25:59.658 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_vanilla (tasks.cephfs.test_failover.TestClusterAffinity) That a vanilla standby is preferred over others with mds_join_fs set to another fs. 2024-03-06T07:25:59.658 INFO:tasks.cephfs_test_runner:test_join_fs_vanilla (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:25:59.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.764+0000 7fa19fe15640 1 Processor -- start 2024-03-06T07:25:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.764+0000 7fa19fe15640 1 -- start start 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.765+0000 7fa19fe15640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa198153990 0x7fa198156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.765+0000 7fa19fe15640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa198070aa0 con 0x7fa1981506b0 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.765+0000 7fa19fe15640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa198070d80 con 0x7fa1980acc00 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.765+0000 7fa19fe15640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa198156450 con 0x7fa198153990 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.765+0000 7fa19dbb1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa1981506b0 0x7fa1980702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38010/0 (socket says 172.21.15.26:38010) 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.765+0000 7fa19dbb1640 1 -- 172.21.15.26:0/2890077358 learned_addr learned my addr 172.21.15.26:0/2890077358 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.765+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2890077358 >> v2:172.21.15.117:3300/0 conn(0x7fa198153990 0x7fa198156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703351306 0 0) 0x7fa198070aa0 con 0x7fa1981506b0 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa180003520 con 0x7fa1981506b0 2024-03-06T07:25:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124041329 0 0) 0x7fa198070d80 con 0x7fa1980acc00 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa198070aa0 con 0x7fa1980acc00 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2184229042 0 0) 0x7fa180003520 con 0x7fa1981506b0 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 >> v2:172.21.15.117:3300/0 conn(0x7fa198153990 msgr2=0x7fa198156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 --2- 172.21.15.26:0/2890077358 >> v2:172.21.15.117:3300/0 conn(0x7fa198153990 0x7fa198156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 >> v1:172.21.15.117:6789/0 conn(0x7fa1980acc00 legacy=0x7fa1980c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1981566a0 con 0x7fa1981506b0 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fa190001670 con 0x7fa1981506b0 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 >> v1:172.21.15.26:6789/0 conn(0x7fa1981506b0 legacy=0x7fa1980702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180003b40 0x7fa180003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180004800 0x7fa180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800051c0 0x7fa18000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198156450 con 0x7fa180004800 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18000c040 con 0x7fa180003b40 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18000c240 con 0x7fa1800051c0 2024-03-06T07:25:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2890077358 >> v2:172.21.15.117:3300/0 conn(0x7fa198153990 0x7fa198156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19dbb1640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180003b40 0x7fa180003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180004800 0x7fa180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19e3b2640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800051c0 0x7fa18000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19d3b0640 1 -- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800051c0 msgr2=0x7fa18000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800051c0 0x7fa18000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19d3b0640 1 -- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180003b40 msgr2=0x7fa180003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180003b40 0x7fa180003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19d3b0640 1 -- 172.21.15.26:0/2890077358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180003520 con 0x7fa180004800 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.766+0000 7fa19e3b2640 1 -- 172.21.15.26:0/2890077358 reap_dead start 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.767+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180004800 0x7fa180004c80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009c70 tx=0x7fa180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438067aab82b829f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.767+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c009840 con 0x7fa180004800 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.767+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c004020 con 0x7fa180004800 2024-03-06T07:25:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.767+0000 7fa19cbaf640 1 -- 172.21.15.26:0/2890077358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c004450 con 0x7fa180004800 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.768+0000 7fa19fe15640 1 -- 172.21.15.26:0/2890077358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180004800 msgr2=0x7fa180004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.768+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180004800 0x7fa180004c80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009c70 tx=0x7fa180005830 comp rx=0 tx=0).stop 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.768+0000 7fa19fe15640 1 -- 172.21.15.26:0/2890077358 shutdown_connections 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.768+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2890077358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180004800 0x7fa180004c80 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.768+0000 7fa19fe15640 1 -- 172.21.15.26:0/2890077358 >> 172.21.15.26:0/2890077358 conn(0x7fa19806b1a0 msgr2=0x7fa19806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.768+0000 7fa19fe15640 1 -- 172.21.15.26:0/2890077358 shutdown_connections 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.768+0000 7fa19fe15640 1 -- 172.21.15.26:0/2890077358 wait complete. 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.769+0000 7fa19fe15640 1 Processor -- start 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.769+0000 7fa19fe15640 1 -- start start 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19fe15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa180004800 0x7fa1980c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19fe15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa198154c80 0x7fa1981e6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19fe15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1981e6ba0 0x7fa198149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19fe15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198156a10 con 0x7fa198154c80 2024-03-06T07:25:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19fe15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980ad210 con 0x7fa1981e6ba0 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19fe15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa198156ea0 con 0x7fa180004800 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa198154c80 0x7fa1981e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa198154c80 0x7fa1981e6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48426/0 (socket says 172.21.15.26:48426) 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 -- 172.21.15.26:0/2561725780 learned_addr learned my addr 172.21.15.26:0/2561725780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19dbb1640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa180004800 0x7fa1980c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19e3b2640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1981e6ba0 0x7fa198149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 -- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa180004800 msgr2=0x7fa1980c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa180004800 0x7fa1980c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 -- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1981e6ba0 msgr2=0x7fa198149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1981e6ba0 0x7fa198149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 -- 172.21.15.26:0/2561725780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1981ea5e0 con 0x7fa198154c80 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.770+0000 7fa19d3b0640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa198154c80 0x7fa1981e6660 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009d90 tx=0x7fa18c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.771+0000 7fa1967fc640 1 -- 172.21.15.26:0/2561725780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c043020 con 0x7fa198154c80 2024-03-06T07:25:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.771+0000 7fa1967fc640 1 -- 172.21.15.26:0/2561725780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c02c910 con 0x7fa198154c80 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.771+0000 7fa1967fc640 1 -- 172.21.15.26:0/2561725780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c02ccc0 con 0x7fa198154c80 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.771+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1981ea8c0 con 0x7fa198154c80 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.771+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa198154470 con 0x7fa198154c80 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.772+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198150e80 con 0x7fa198154c80 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.772+0000 7fa1967fc640 1 -- 172.21.15.26:0/2561725780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa18c046020 con 0x7fa198154c80 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.772+0000 7fa1967fc640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa18406b500 0x7fa18406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.772+0000 7fa1967fc640 1 -- 172.21.15.26:0/2561725780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa18c0ac460 con 0x7fa198154c80 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.773+0000 7fa19dbb1640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa18406b500 0x7fa18406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.773+0000 7fa19dbb1640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa18406b500 0x7fa18406d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa190001eb0 tx=0x7fa190009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.777+0000 7fa1967fc640 1 -- 172.21.15.26:0/2561725780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa198150e80 con 0x7fa198154c80 2024-03-06T07:25:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.962+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa19804f330 con 0x7fa198154c80 2024-03-06T07:25:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.962+0000 7fa1967fc640 1 -- 172.21.15.26:0/2561725780 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+13579 (secure 0 0 0) 0x7fa18c04d020 con 0x7fa198154c80 2024-03-06T07:25:59.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:25:59.964 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":142,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:25:57.628075+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3380233594":"2024-03-07T07:25:54.615739+0000","172.21.15.117:6835/3380233594":"2024-03-07T07:25:54.615739+0000","172.21.15.117:6836/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.117:6834/1667664264":"2024-03-07T07:25:46.167307+0000","172.21.15.26:6835/1099020571":"2024-03-07T07:25:32.004862+0000","172.21.15.26:6834/1099020571":"2024-03-07T07:25:32.004862+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa18406b500 msgr2=0x7fa18406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa18406b500 0x7fa18406d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa190001eb0 tx=0x7fa190009040 comp rx=0 tx=0).stop 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa198154c80 msgr2=0x7fa1981e6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa198154c80 0x7fa1981e6660 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009d90 tx=0x7fa18c004730 comp rx=0 tx=0).stop 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 shutdown_connections 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa18406b500 0x7fa18406d9c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1981e6ba0 0x7fa198149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa198154c80 0x7fa1981e6660 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 --2- 172.21.15.26:0/2561725780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa180004800 0x7fa1980c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 >> 172.21.15.26:0/2561725780 conn(0x7fa19806b1a0 msgr2=0x7fa19814ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:59.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 shutdown_connections 2024-03-06T07:25:59.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:25:59.964+0000 7fa19fe15640 1 -- 172.21.15.26:0/2561725780 wait complete. 2024-03-06T07:25:59.977 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:59.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.080+0000 7f36b43e9640 1 Processor -- start 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.080+0000 7f36b43e9640 1 -- start start 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b43e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f36ac152920 0x7f36ac152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b43e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b43e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36ac0a5eb0 0x7f36ac0c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b43e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac0c02b0 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b43e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac155f70 con 0x7f36ac152920 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b43e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac156250 con 0x7f36ac06d800 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36ac0a5eb0 0x7f36ac0c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36ac0a5eb0 0x7f36ac0c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48434/0 (socket says 172.21.15.26:48434) 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 -- 172.21.15.26:0/1333226773 learned_addr learned my addr 172.21.15.26:0/1333226773 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2185640 1 --2- 172.21.15.26:0/1333226773 >> v2:172.21.15.117:3300/0 conn(0x7f36ac152920 0x7f36ac152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b1984640 1 --2- 172.21.15.26:0/1333226773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 -- 172.21.15.26:0/1333226773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 msgr2=0x7f36ac0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 --2- 172.21.15.26:0/1333226773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 -- 172.21.15.26:0/1333226773 >> v2:172.21.15.117:3300/0 conn(0x7f36ac152920 msgr2=0x7f36ac152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 --2- 172.21.15.26:0/1333226773 >> v2:172.21.15.117:3300/0 conn(0x7f36ac152920 0x7f36ac152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.081+0000 7f36b2986640 1 -- 172.21.15.26:0/1333226773 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0c49a0 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.082+0000 7f36b2986640 1 --2- 172.21.15.26:0/1333226773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac0c6a30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0681b0 tx=0x7f36a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75635e6bc80d14e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.082+0000 7f36b1183640 1 -- 172.21.15.26:0/1333226773 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a402e070 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.082+0000 7f36b1183640 1 -- 172.21.15.26:0/1333226773 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4037070 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.082+0000 7f36b1183640 1 -- 172.21.15.26:0/1333226773 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4031a00 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 -- 172.21.15.26:0/1333226773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 msgr2=0x7f36ac0c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1333226773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac0c6a30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0681b0 tx=0x7f36a402b880 comp rx=0 tx=0).stop 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 -- 172.21.15.26:0/1333226773 shutdown_connections 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1333226773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1333226773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac0c6a30 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1333226773 >> v2:172.21.15.117:3300/0 conn(0x7f36ac152920 0x7f36ac152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 -- 172.21.15.26:0/1333226773 >> 172.21.15.26:0/1333226773 conn(0x7f36ac06cb60 msgr2=0x7f36ac0a86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 -- 172.21.15.26:0/1333226773 shutdown_connections 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.083+0000 7f36b43e9640 1 -- 172.21.15.26:0/1333226773 wait complete. 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.084+0000 7f36b43e9640 1 Processor -- start 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.084+0000 7f36b43e9640 1 -- start start 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b43e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac1e2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b43e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac1eb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b43e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ac152920 0x7f36ac14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b43e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0a6340 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b43e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0c4f60 con 0x7f36ac152920 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b43e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0a64c0 con 0x7f36ac06d800 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac1eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac1eb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48436/0 (socket says 172.21.15.26:48436) 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 -- 172.21.15.26:0/1751608375 learned_addr learned my addr 172.21.15.26:0/1751608375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b2185640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac1e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b2986640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ac152920 0x7f36ac14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 -- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 msgr2=0x7f36ac1e2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac1e2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 -- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ac152920 msgr2=0x7f36ac14fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ac152920 0x7f36ac14fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 -- 172.21.15.26:0/1751608375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac1ee930 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.085+0000 7f36b1984640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac1eb220 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f36a000eed0 tx=0x7f36a000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.086+0000 7f369effd640 1 -- 172.21.15.26:0/1751608375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0018020 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.086+0000 7f369effd640 1 -- 172.21.15.26:0/1751608375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a000bd90 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.086+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac1e1330 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.086+0000 7f369effd640 1 -- 172.21.15.26:0/1751608375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0014620 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.086+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac1e2540 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.087+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac069460 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.087+0000 7f369effd640 1 -- 172.21.15.26:0/1751608375 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36a0005d10 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.088+0000 7f369effd640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.088+0000 7f369effd640 1 -- 172.21.15.26:0/1751608375 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f36a008a870 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.088+0000 7f36b2185640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.088+0000 7f36b2185640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369006b4d0 0x7f369006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009b40 tx=0x7f36a8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.092+0000 7f369effd640 1 -- 172.21.15.26:0/1751608375 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f36ac069460 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.271+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36ac04f330 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.271+0000 7f369effd640 1 -- 172.21.15.26:0/1751608375 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36a005f020 con 0x7f36ac0a5eb0 2024-03-06T07:26:00.273 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:26:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369006b4d0 msgr2=0x7f369006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369006b4d0 0x7f369006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009b40 tx=0x7f36a8009190 comp rx=0 tx=0).stop 2024-03-06T07:26:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 msgr2=0x7f36ac1eb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac1eb220 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f36a000eed0 tx=0x7f36a000b480 comp rx=0 tx=0).stop 2024-03-06T07:26:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 shutdown_connections 2024-03-06T07:26:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ac152920 0x7f36ac14fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ac0a5eb0 0x7f36ac1eb220 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 --2- 172.21.15.26:0/1751608375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36ac06d800 0x7f36ac1e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 >> 172.21.15.26:0/1751608375 conn(0x7f36ac06cb60 msgr2=0x7f36ac0a86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 shutdown_connections 2024-03-06T07:26:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.273+0000 7f36b43e9640 1 -- 172.21.15.26:0/1751608375 wait complete. 2024-03-06T07:26:00.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla' 2024-03-06T07:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.386+0000 7f6c8d1d0640 1 Processor -- start 2024-03-06T07:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.386+0000 7f6c8d1d0640 1 -- start start 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8d1d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c880c7380 0x7f6c880c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8d1d0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c88072630 con 0x7f6c880c6780 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8d1d0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c88072910 con 0x7f6c8806da40 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8d1d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8806de20 con 0x7f6c880c7380 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c880c6780 0x7f6c88158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38012/0 (socket says 172.21.15.26:38012) 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8659c640 1 -- 172.21.15.26:0/4078959741 learned_addr learned my addr 172.21.15.26:0/4078959741 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994812217 0 0) 0x7f6c88072630 con 0x7f6c880c6780 2024-03-06T07:26:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8759e640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c880c7380 0x7f6c880c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c70003520 con 0x7f6c880c6780 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2093969019 0 0) 0x7f6c70003520 con 0x7f6c880c6780 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c880c7380 msgr2=0x7f6c880c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c880c7380 0x7f6c880c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 >> v1:172.21.15.117:6789/0 conn(0x7f6c8806da40 legacy=0x7f6c88155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c88072bf0 con 0x7f6c880c6780 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6c7c001670 con 0x7f6c880c6780 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 >> v1:172.21.15.26:6789/0 conn(0x7f6c880c6780 legacy=0x7f6c88158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c700039e0 0x7f6c70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c70004800 0x7f6c70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c70005210 0x7f6c7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8806de20 con 0x7f6c70005210 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7000c140 con 0x7f6c70004800 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7000c340 con 0x7f6c700039e0 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.387+0000 7f6c8759e640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c880c7380 0x7f6c880c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c8659c640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c70004800 0x7f6c70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c86d9d640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c700039e0 0x7f6c70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c8759e640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c70005210 0x7f6c7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c8759e640 1 -- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c700039e0 msgr2=0x7f6c70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c8759e640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c700039e0 0x7f6c70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c8759e640 1 -- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c70004800 msgr2=0x7f6c70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c8759e640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c70004800 0x7f6c70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.388+0000 7f6c8759e640 1 -- 172.21.15.26:0/4078959741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c70003520 con 0x7f6c70005210 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8759e640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c70005210 0x7f6c7000b9c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6c88155090 tx=0x7f6c70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd9eaefd8376dc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80009b60 con 0x7f6c70005210 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8002cc30 con 0x7f6c70005210 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c85d9b640 1 -- 172.21.15.26:0/4078959741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80003990 con 0x7f6c70005210 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/4078959741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c70005210 msgr2=0x7f6c7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8d1d0640 1 --2- 172.21.15.26:0/4078959741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c70005210 0x7f6c7000b9c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6c88155090 tx=0x7f6c70005830 comp rx=0 tx=0).stop 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8759e640 1 -- 172.21.15.26:0/4078959741 reap_dead start 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/4078959741 shutdown_connections 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/4078959741 >> 172.21.15.26:0/4078959741 conn(0x7f6c880aa630 msgr2=0x7f6c880aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/4078959741 shutdown_connections 2024-03-06T07:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.389+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/4078959741 wait complete. 2024-03-06T07:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 Processor -- start 2024-03-06T07:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 -- start start 2024-03-06T07:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c881575c0 0x7f6c880ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8814edb0 0x7f6c880abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c881ee010 0x7f6c881ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880c3280 con 0x7f6c8814edb0 2024-03-06T07:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c881594a0 con 0x7f6c881575c0 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8d1d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880c3400 con 0x7f6c881ee010 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8814edb0 0x7f6c880abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8814edb0 0x7f6c880abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48454/0 (socket says 172.21.15.26:48454) 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.391+0000 7f6c8659c640 1 -- 172.21.15.26:0/2162979344 learned_addr learned my addr 172.21.15.26:0/2162979344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8659c640 1 -- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c881ee010 msgr2=0x7f6c881ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8759e640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c881ee010 0x7f6c881ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c86d9d640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c881575c0 0x7f6c880ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8659c640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c881ee010 0x7f6c881ee4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8659c640 1 -- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c881575c0 msgr2=0x7f6c880ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8659c640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c881575c0 0x7f6c880ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8659c640 1 -- 172.21.15.26:0/2162979344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c881eaaa0 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8659c640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8814edb0 0x7f6c880abb60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0072c0 tx=0x7f6c7c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c6f7fe640 1 -- 172.21.15.26:0/2162979344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00f8e0 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c6f7fe640 1 -- 172.21.15.26:0/2162979344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c018070 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c880ae760 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c6f7fe640 1 -- 172.21.15.26:0/2162979344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00fca0 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.392+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c881eb040 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.393+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c881eb370 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.394+0000 7f6c6f7fe640 1 -- 172.21.15.26:0/2162979344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c7c01d460 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.394+0000 7f6c6f7fe640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c7406b4d0 0x7f6c7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.394+0000 7f6c6f7fe640 1 -- 172.21.15.26:0/2162979344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6c7c08a1e0 con 0x7f6c8814edb0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.394+0000 7f6c86d9d640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c7406b4d0 0x7f6c7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.395+0000 7f6c86d9d640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c7406b4d0 0x7f6c7406d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005590 tx=0x7f6c78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.398+0000 7f6c6f7fe640 1 -- 172.21.15.26:0/2162979344 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c7c0591b0 con 0x7f6c8814edb0 2024-03-06T07:26:00.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:00.650+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]} v 0) v1 -- 0x7f6c8804f330 con 0x7f6c8814edb0 2024-03-06T07:26:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.646+0000 7f6c6f7fe640 1 -- 172.21.15.26:0/2162979344 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]}]=0 v346) v1 ==== 149+0+0 (secure 0 0 0) 0x7f6c7c058ab0 con 0x7f6c8814edb0 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c7406b4d0 msgr2=0x7f6c7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c7406b4d0 0x7f6c7406d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005590 tx=0x7f6c78009040 comp rx=0 tx=0).stop 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8814edb0 msgr2=0x7f6c880abb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8814edb0 0x7f6c880abb60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0072c0 tx=0x7f6c7c007780 comp rx=0 tx=0).stop 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 shutdown_connections 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c7406b4d0 0x7f6c7406d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c881ee010 0x7f6c881ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8814edb0 0x7f6c880abb60 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 --2- 172.21.15.26:0/2162979344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c881575c0 0x7f6c880ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 >> 172.21.15.26:0/2162979344 conn(0x7f6c880aa630 msgr2=0x7f6c880aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 shutdown_connections 2024-03-06T07:26:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.649+0000 7f6c8d1d0640 1 -- 172.21.15.26:0/2162979344 wait complete. 2024-03-06T07:26:01.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:26:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.773+0000 7efdec3a0640 1 Processor -- start 2024-03-06T07:26:01.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.773+0000 7efdec3a0640 1 -- start start 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efdec3a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 0x7efde40724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efdec3a0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde4152cb0 con 0x7efde4072a10 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efdec3a0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde40c38b0 con 0x7efde40c67f0 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efdec3a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde41594f0 con 0x7efde41528d0 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde993b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efde4072a10 0x7efde4155220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38022/0 (socket says 172.21.15.26:38022) 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde993b640 1 -- 172.21.15.26:0/3661824278 learned_addr learned my addr 172.21.15.26:0/3661824278 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efdea13c640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 0x7efde40724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469477095 0 0) 0x7efde4152cb0 con 0x7efde4072a10 2024-03-06T07:26:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd0003520 con 0x7efde4072a10 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832955672 0 0) 0x7efde40c38b0 con 0x7efde40c67f0 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde4152cb0 con 0x7efde40c67f0 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3007804978 0 0) 0x7efdd0003520 con 0x7efde4072a10 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 msgr2=0x7efde40724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 0x7efde40724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 >> v1:172.21.15.117:6789/0 conn(0x7efde40c67f0 legacy=0x7efde415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40a7f70 con 0x7efde4072a10 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7efdd4001670 con 0x7efde4072a10 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 >> v1:172.21.15.26:6789/0 conn(0x7efde4072a10 legacy=0x7efde4155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.774+0000 7efde913a640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 0x7efdd0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde913a640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00040d0 0x7efdd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde913a640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd0004d40 0x7efdd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde41594f0 con 0x7efdd00040d0 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd000baa0 con 0x7efdd0004d40 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd000bca0 con 0x7efde41528d0 2024-03-06T07:26:01.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde993b640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00040d0 0x7efdd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efdea13c640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 0x7efdd0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efdea93d640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd0004d40 0x7efdd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde993b640 1 -- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 msgr2=0x7efdd0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde993b640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efde41528d0 0x7efdd0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde993b640 1 -- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd0004d40 msgr2=0x7efdd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde993b640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd0004d40 0x7efdd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efde993b640 1 -- 172.21.15.26:0/3661824278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0003520 con 0x7efdd00040d0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.775+0000 7efdea93d640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd0004d40 0x7efdd00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.776+0000 7efde993b640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00040d0 0x7efdd0004800 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c190 tx=0x7efdd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40799eb7df611046 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.776+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4007ce0 con 0x7efdd00040d0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.776+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd4030070 con 0x7efdd00040d0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.776+0000 7efde913a640 1 -- 172.21.15.26:0/3661824278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4005d90 con 0x7efdd00040d0 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.776+0000 7efdec3a0640 1 -- 172.21.15.26:0/3661824278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00040d0 msgr2=0x7efdd0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.776+0000 7efdec3a0640 1 --2- 172.21.15.26:0/3661824278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00040d0 0x7efdd0004800 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c190 tx=0x7efdd0005830 comp rx=0 tx=0).stop 2024-03-06T07:26:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.776+0000 7efdea93d640 1 -- 172.21.15.26:0/3661824278 reap_dead start 2024-03-06T07:26:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.777+0000 7efdec3a0640 1 -- 172.21.15.26:0/3661824278 shutdown_connections 2024-03-06T07:26:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.777+0000 7efdec3a0640 1 -- 172.21.15.26:0/3661824278 >> 172.21.15.26:0/3661824278 conn(0x7efde40aa6a0 msgr2=0x7efde40aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.777+0000 7efdec3a0640 1 -- 172.21.15.26:0/3661824278 shutdown_connections 2024-03-06T07:26:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.777+0000 7efdec3a0640 1 -- 172.21.15.26:0/3661824278 wait complete. 2024-03-06T07:26:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 Processor -- start 2024-03-06T07:26:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 -- start start 2024-03-06T07:26:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd00040d0 0x7efde406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde4151e50 0x7efde41eae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efde41500f0 0x7efde41eb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde41591a0 con 0x7efde4151e50 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40a8250 con 0x7efde41500f0 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.778+0000 7efdec3a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40c4ef0 con 0x7efdd00040d0 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde4151e50 0x7efde41eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde4151e50 0x7efde41eae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48462/0 (socket says 172.21.15.26:48462) 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 -- 172.21.15.26:0/3911234336 learned_addr learned my addr 172.21.15.26:0/3911234336 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efdea13c640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd00040d0 0x7efde406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efdea93d640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efde41500f0 0x7efde41eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 -- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd00040d0 msgr2=0x7efde406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd00040d0 0x7efde406fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 -- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efde41500f0 msgr2=0x7efde41eb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efde41500f0 0x7efde41eb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 -- 172.21.15.26:0/3911234336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4070990 con 0x7efde4151e50 2024-03-06T07:26:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efde993b640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde4151e50 0x7efde41eae30 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7efdd40018f0 tx=0x7efdd40019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efddaffd640 1 -- 172.21.15.26:0/3911234336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4001c10 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efddaffd640 1 -- 172.21.15.26:0/3911234336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd4004bf0 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efddaffd640 1 -- 172.21.15.26:0/3911234336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4007070 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde41477d0 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.779+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde41ef230 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.780+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde404f330 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.781+0000 7efddaffd640 1 -- 172.21.15.26:0/3911234336 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdd4007210 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.781+0000 7efddaffd640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efdc806b4d0 0x7efdc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.781+0000 7efddaffd640 1 -- 172.21.15.26:0/3911234336 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7efdd40ab030 con 0x7efde4151e50 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.781+0000 7efdea13c640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efdc806b4d0 0x7efdc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.782+0000 7efdea13c640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efdc806b4d0 0x7efdc806d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efde0009ae0 tx=0x7efde0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.785+0000 7efddaffd640 1 -- 172.21.15.26:0/3911234336 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efde4070990 con 0x7efde4151e50 2024-03-06T07:26:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.970+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efde4073120 con 0x7efde4151e50 2024-03-06T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.973+0000 7efddaffd640 1 -- 172.21.15.26:0/3911234336 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7efdd407a0c0 con 0x7efde4151e50 2024-03-06T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efdc806b4d0 msgr2=0x7efdc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efdc806b4d0 0x7efdc806d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efde0009ae0 tx=0x7efde0009190 comp rx=0 tx=0).stop 2024-03-06T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde4151e50 msgr2=0x7efde41eae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde4151e50 0x7efde41eae30 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7efdd40018f0 tx=0x7efdd40019c0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 shutdown_connections 2024-03-06T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efdc806b4d0 0x7efdc806d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efde41500f0 0x7efde41eb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde4151e50 0x7efde41eae30 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 --2- 172.21.15.26:0/3911234336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd00040d0 0x7efde406fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 >> 172.21.15.26:0/3911234336 conn(0x7efde40aa6a0 msgr2=0x7efde406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 shutdown_connections 2024-03-06T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:01.975+0000 7efdec3a0640 1 -- 172.21.15.26:0/3911234336 wait complete. 2024-03-06T07:26:01.988 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:26:01.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:02.029 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:02.082 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:26:02.082 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:26:02.082 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:02.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:26:02.097 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:26:02.097 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:02.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:26:02.114 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:26:02.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.237+0000 7f240f51e640 1 Processor -- start 2024-03-06T07:26:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.237+0000 7f240f51e640 1 -- start start 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240f51e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24080c6780 0x7f2408158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240f51e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2408072630 con 0x7f24081592d0 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240f51e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2408072910 con 0x7f240806da40 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240f51e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240806de20 con 0x7f24080c6780 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240cab9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24080c6780 0x7f2408158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240dabb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f24081592d0 0x7f24080c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38024/0 (socket says 172.21.15.26:38024) 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240dabb640 1 -- 172.21.15.26:0/1138546793 learned_addr learned my addr 172.21.15.26:0/1138546793 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240cab9640 1 -- 172.21.15.26:0/1138546793 >> v1:172.21.15.117:6789/0 conn(0x7f240806da40 legacy=0x7f2408155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240cab9640 1 -- 172.21.15.26:0/1138546793 >> v1:172.21.15.26:6789/0 conn(0x7f24081592d0 legacy=0x7f24080c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.238+0000 7f240cab9640 1 -- 172.21.15.26:0/1138546793 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2408072bf0 con 0x7f24080c6780 2024-03-06T07:26:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f240cab9640 1 --2- 172.21.15.26:0/1138546793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24080c6780 0x7f2408158d90 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f23f80097a0 tx=0x7f23f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b6ed952c758d43 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f23fffff640 1 -- 172.21.15.26:0/1138546793 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80315a0 con 0x7f24080c6780 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f23fffff640 1 -- 172.21.15.26:0/1138546793 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f8037070 con 0x7f24080c6780 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f23fffff640 1 -- 172.21.15.26:0/1138546793 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8031960 con 0x7f24080c6780 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f240f51e640 1 -- 172.21.15.26:0/1138546793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24080c6780 msgr2=0x7f2408158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f240f51e640 1 --2- 172.21.15.26:0/1138546793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24080c6780 0x7f2408158d90 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f23f80097a0 tx=0x7f23f802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f240f51e640 1 -- 172.21.15.26:0/1138546793 shutdown_connections 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.239+0000 7f240f51e640 1 --2- 172.21.15.26:0/1138546793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24080c6780 0x7f2408158d90 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.240+0000 7f240f51e640 1 -- 172.21.15.26:0/1138546793 >> 172.21.15.26:0/1138546793 conn(0x7f24080aa630 msgr2=0x7f24080aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.240+0000 7f240f51e640 1 -- 172.21.15.26:0/1138546793 shutdown_connections 2024-03-06T07:26:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.240+0000 7f240f51e640 1 -- 172.21.15.26:0/1138546793 wait complete. 2024-03-06T07:26:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 Processor -- start 2024-03-06T07:26:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 -- start start 2024-03-06T07:26:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240806da40 0x7f24080c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24080c6780 0x7f24081ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24081592d0 0x7f24081ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c3b10 con 0x7f240806da40 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c3990 con 0x7f24080c6780 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240f51e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c7380 con 0x7f24081592d0 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240d2ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240806da40 0x7f24080c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240d2ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240806da40 0x7f24080c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48492/0 (socket says 172.21.15.26:48492) 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.241+0000 7f240d2ba640 1 -- 172.21.15.26:0/4095806876 learned_addr learned my addr 172.21.15.26:0/4095806876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240d2ba640 1 -- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24081592d0 msgr2=0x7f24081ee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240d2ba640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24081592d0 0x7f24081ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240d2ba640 1 -- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24080c6780 msgr2=0x7f24081ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240d2ba640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24080c6780 0x7f24081ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240d2ba640 1 -- 172.21.15.26:0/4095806876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2408070020 con 0x7f240806da40 2024-03-06T07:26:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240d2ba640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240806da40 0x7f24080c7c10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f240400c430 tx=0x7f240400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f23fdffb640 1 -- 172.21.15.26:0/4095806876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404011790 con 0x7f240806da40 2024-03-06T07:26:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f23fdffb640 1 -- 172.21.15.26:0/4095806876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404017070 con 0x7f240806da40 2024-03-06T07:26:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f23fdffb640 1 -- 172.21.15.26:0/4095806876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404011bb0 con 0x7f240806da40 2024-03-06T07:26:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2408152000 con 0x7f240806da40 2024-03-06T07:26:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.242+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240814f7f0 con 0x7f240806da40 2024-03-06T07:26:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.243+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240804f330 con 0x7f240806da40 2024-03-06T07:26:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.244+0000 7f23fdffb640 1 -- 172.21.15.26:0/4095806876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f240401c460 con 0x7f240806da40 2024-03-06T07:26:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.244+0000 7f23fdffb640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f23ec06b500 0x7f23ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.244+0000 7f23fdffb640 1 -- 172.21.15.26:0/4095806876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2404089f90 con 0x7f240806da40 2024-03-06T07:26:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.244+0000 7f240cab9640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f23ec06b500 0x7f23ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.245+0000 7f240cab9640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f23ec06b500 0x7f23ec06d9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f23f8036000 tx=0x7f23f8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.248+0000 7f23fdffb640 1 -- 172.21.15.26:0/4095806876 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2404058f60 con 0x7f240806da40 2024-03-06T07:26:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.464+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2408073240 con 0x7f240806da40 2024-03-06T07:26:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.465+0000 7f23fdffb640 1 -- 172.21.15.26:0/4095806876 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+3710 (secure 0 0 0) 0x7f2404058860 con 0x7f240806da40 2024-03-06T07:26:02.467 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:02.467 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":133,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1553547443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1553547443},{"type":"v1","addr":"172.21.15.117:6837","nonce":1553547443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8101,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3312878094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3312878094},{"type":"v1","addr":"172.21.15.117:6836","nonce":3312878094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2142785555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2142785555},{"type":"v1","addr":"172.21.15.26:6833","nonce":2142785555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2316269046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2316269046},{"type":"v1","addr":"172.21.15.26:6835","nonce":2316269046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[]} 2024-03-06T07:26:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 133 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f23ec06b500 msgr2=0x7f23ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f23ec06b500 0x7f23ec06d9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f23f8036000 tx=0x7f23f8030000 comp rx=0 tx=0).stop 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240806da40 msgr2=0x7f24080c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240806da40 0x7f24080c7c10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f240400c430 tx=0x7f240400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 shutdown_connections 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f23ec06b500 0x7f23ec06d9c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24081592d0 0x7f24081ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24080c6780 0x7f24081ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 --2- 172.21.15.26:0/4095806876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240806da40 0x7f24080c7c10 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 >> 172.21.15.26:0/4095806876 conn(0x7f24080aa630 msgr2=0x7f2408154850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 shutdown_connections 2024-03-06T07:26:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.467+0000 7f240f51e640 1 -- 172.21.15.26:0/4095806876 wait complete. 2024-03-06T07:26:02.479 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:26:02.479 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:26:02.480 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:26:02.480 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:02.480 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:26:02.480 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:26:02.480 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:26:02.480 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:02.481 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:26:02.481 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:26:02.481 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:26:02.481 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:02.481 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:26:02.481 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:26:02.482 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:26:02.482 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:02.489 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:26:02.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:26:02.490 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:26:02.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:26:02.492 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:26:02.492 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:26:02.493 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:26:02.493 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:26:02.532 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:26:02.532 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:02.533 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:26:02.533 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:02.535 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:26:02.535 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:02.536 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:26:02.536 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:02.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:26:02.608 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:26:02.609 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:26:02.617 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:26:02.617 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:26:02.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.642+0000 7f6b0a58c640 1 Processor -- start 2024-03-06T07:26:02.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.642+0000 7f6b0a58c640 1 -- start start 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b0a58c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b040732d0 0x7f6b040c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b0a58c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b04070160 con 0x7f6b04072ef0 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b0a58c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b040c6a70 con 0x7f6b040c7270 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b0a58c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040c7030 con 0x7f6b040732d0 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b03fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b04072ef0 0x7f6b04156d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38044/0 (socket says 172.21.15.26:38044) 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b03fff640 1 -- 172.21.15.26:0/1378342527 learned_addr learned my addr 172.21.15.26:0/1378342527 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b037fe640 1 --2- 172.21.15.26:0/1378342527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b040732d0 0x7f6b040c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b02ffd640 1 -- 172.21.15.26:0/1378342527 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712611183 0 0) 0x7f6af8001110 con 0x7f6b04072ef0 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b02ffd640 1 -- 172.21.15.26:0/1378342527 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aec003520 con 0x7f6b04072ef0 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b02ffd640 1 -- 172.21.15.26:0/1378342527 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608030235 0 0) 0x7f6b040c6a70 con 0x7f6b040c7270 2024-03-06T07:26:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b02ffd640 1 -- 172.21.15.26:0/1378342527 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af8001110 con 0x7f6b040c7270 2024-03-06T07:26:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b037fe640 1 -- 172.21.15.26:0/1378342527 >> v1:172.21.15.117:6789/0 conn(0x7f6b040c7270 legacy=0x7f6b040c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b037fe640 1 -- 172.21.15.26:0/1378342527 >> v1:172.21.15.26:6789/0 conn(0x7f6b04072ef0 legacy=0x7f6b04156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.643+0000 7f6b037fe640 1 -- 172.21.15.26:0/1378342527 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040c6d50 con 0x7f6b040732d0 2024-03-06T07:26:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.644+0000 7f6b037fe640 1 --2- 172.21.15.26:0/1378342527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b040732d0 0x7f6b040c36b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6af40097d0 tx=0x7f6af402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e58fa9dadf96795 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.644+0000 7f6b02ffd640 1 -- 172.21.15.26:0/1378342527 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4035020 con 0x7f6b040732d0 2024-03-06T07:26:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.644+0000 7f6b02ffd640 1 -- 172.21.15.26:0/1378342527 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af4031800 con 0x7f6b040732d0 2024-03-06T07:26:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.644+0000 7f6b02ffd640 1 -- 172.21.15.26:0/1378342527 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4031bd0 con 0x7f6b040732d0 2024-03-06T07:26:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.645+0000 7f6b0a58c640 1 -- 172.21.15.26:0/1378342527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b040732d0 msgr2=0x7f6b040c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.645+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/1378342527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b040732d0 0x7f6b040c36b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6af40097d0 tx=0x7f6af402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.645+0000 7f6b0a58c640 1 -- 172.21.15.26:0/1378342527 shutdown_connections 2024-03-06T07:26:02.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.645+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/1378342527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b040732d0 0x7f6b040c36b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.645+0000 7f6b0a58c640 1 -- 172.21.15.26:0/1378342527 >> 172.21.15.26:0/1378342527 conn(0x7f6b0406ea40 msgr2=0x7f6b0406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:02.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.645+0000 7f6b0a58c640 1 -- 172.21.15.26:0/1378342527 shutdown_connections 2024-03-06T07:26:02.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.645+0000 7f6b0a58c640 1 -- 172.21.15.26:0/1378342527 wait complete. 2024-03-06T07:26:02.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.646+0000 7f6b0a58c640 1 Processor -- start 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.646+0000 7f6b0a58c640 1 -- start start 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b0a58c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b04072ef0 0x7f6b041ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b0a58c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b040c7270 0x7f6b041486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b0a58c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b04148c00 0x7f6b041ea560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b0a58c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0406fc40 con 0x7f6b04072ef0 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b0a58c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0406f960 con 0x7f6b040c7270 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b0a58c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04157480 con 0x7f6b04148c00 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b04072ef0 0x7f6b041ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b04072ef0 0x7f6b041ebf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48528/0 (socket says 172.21.15.26:48528) 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b03fff640 1 -- 172.21.15.26:0/2688571775 learned_addr learned my addr 172.21.15.26:0/2688571775 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b037fe640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b040c7270 0x7f6b041486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.647+0000 7f6b08b29640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b04148c00 0x7f6b041ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.648+0000 7f6b03fff640 1 -- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b04148c00 msgr2=0x7f6b041ea560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.648+0000 7f6b03fff640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b04148c00 0x7f6b041ea560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.648+0000 7f6b03fff640 1 -- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b040c7270 msgr2=0x7f6b041486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.648+0000 7f6b03fff640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b040c7270 0x7f6b041486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.648+0000 7f6b03fff640 1 -- 172.21.15.26:0/2688571775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b041ec770 con 0x7f6b04072ef0 2024-03-06T07:26:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.648+0000 7f6b03fff640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b04072ef0 0x7f6b041ebf30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f6af800c730 tx=0x7f6af800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.649+0000 7f6b00ff9640 1 -- 172.21.15.26:0/2688571775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8017070 con 0x7f6b04072ef0 2024-03-06T07:26:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.649+0000 7f6b00ff9640 1 -- 172.21.15.26:0/2688571775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af8005af0 con 0x7f6b04072ef0 2024-03-06T07:26:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.649+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b041e9d60 con 0x7f6b04072ef0 2024-03-06T07:26:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.649+0000 7f6b00ff9640 1 -- 172.21.15.26:0/2688571775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8007070 con 0x7f6b04072ef0 2024-03-06T07:26:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.650+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b041ec470 con 0x7f6b04072ef0 2024-03-06T07:26:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.651+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b04149010 con 0x7f6b04072ef0 2024-03-06T07:26:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.651+0000 7f6b00ff9640 1 -- 172.21.15.26:0/2688571775 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6af801a070 con 0x7f6b04072ef0 2024-03-06T07:26:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.652+0000 7f6b00ff9640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6af006b4d0 0x7f6af006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.652+0000 7f6b00ff9640 1 -- 172.21.15.26:0/2688571775 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6af8089470 con 0x7f6b04072ef0 2024-03-06T07:26:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.652+0000 7f6b037fe640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6af006b4d0 0x7f6af006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.652+0000 7f6b037fe640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6af006b4d0 0x7f6af006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6af4036000 tx=0x7f6af4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.658+0000 7f6b00ff9640 1 -- 172.21.15.26:0/2688571775 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6b04070160 con 0x7f6b04072ef0 2024-03-06T07:26:02.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:02.855+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6b0404f330 con 0x7f6b04072ef0 2024-03-06T07:26:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.663+0000 7f6b00ff9640 1 -- 172.21.15.26:0/2688571775 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v147) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6af8057eb0 con 0x7f6b04072ef0 2024-03-06T07:26:03.666 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:26:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6af006b4d0 msgr2=0x7f6af006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6af006b4d0 0x7f6af006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6af4036000 tx=0x7f6af4008030 comp rx=0 tx=0).stop 2024-03-06T07:26:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b04072ef0 msgr2=0x7f6b041ebf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b04072ef0 0x7f6b041ebf30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f6af800c730 tx=0x7f6af800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 shutdown_connections 2024-03-06T07:26:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6af006b4d0 0x7f6af006d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b04148c00 0x7f6b041ea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b040c7270 0x7f6b041486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 --2- 172.21.15.26:0/2688571775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b04072ef0 0x7f6b041ebf30 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 >> 172.21.15.26:0/2688571775 conn(0x7f6b0406ea40 msgr2=0x7f6b040c6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 shutdown_connections 2024-03-06T07:26:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.666+0000 7f6b0a58c640 1 -- 172.21.15.26:0/2688571775 wait complete. 2024-03-06T07:26:03.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:26:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.787+0000 7f4643565640 1 Processor -- start 2024-03-06T07:26:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.787+0000 7f4643565640 1 -- start start 2024-03-06T07:26:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.787+0000 7f4643565640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c1592d0 0x7f463c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.787+0000 7f4643565640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f463c0ac260 0x7f463c155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.787+0000 7f4643565640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c152d40 con 0x7f463c152960 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.787+0000 7f4643565640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463c066e30 con 0x7f463c0ac260 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.787+0000 7f4643565640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c155d50 con 0x7f463c1592d0 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4641b02640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f463c0ac260 0x7f463c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4641301640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f463c152960 0x7f463c06dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38046/0 (socket says 172.21.15.26:38046) 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4641301640 1 -- 172.21.15.26:0/133973133 learned_addr learned my addr 172.21.15.26:0/133973133 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4640b00640 1 --2- 172.21.15.26:0/133973133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c1592d0 0x7f463c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4641b02640 1 -- 172.21.15.26:0/133973133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c1592d0 msgr2=0x7f463c1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4641b02640 1 --2- 172.21.15.26:0/133973133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c1592d0 0x7f463c1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4641b02640 1 -- 172.21.15.26:0/133973133 >> v1:172.21.15.26:6789/0 conn(0x7f463c152960 legacy=0x7f463c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4641b02640 1 -- 172.21.15.26:0/133973133 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c0c1b50 con 0x7f463c0ac260 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f463bfff640 1 -- 172.21.15.26:0/133973133 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717541154 0 0) 0x7f463c152d40 con 0x7f463c152960 2024-03-06T07:26:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.788+0000 7f4640b00640 1 --2- 172.21.15.26:0/133973133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c1592d0 0x7f463c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4641b02640 1 --2- 172.21.15.26:0/133973133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c0ac260 0x7f463c155810 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7976f6cf0038fd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f463bfff640 1 -- 172.21.15.26:0/133973133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c039070 con 0x7f463c0ac260 2024-03-06T07:26:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f463bfff640 1 -- 172.21.15.26:0/133973133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c02cc50 con 0x7f463c0ac260 2024-03-06T07:26:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f463bfff640 1 -- 172.21.15.26:0/133973133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c034440 con 0x7f463c0ac260 2024-03-06T07:26:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 -- 172.21.15.26:0/133973133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c0ac260 msgr2=0x7f463c155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 --2- 172.21.15.26:0/133973133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c0ac260 0x7f463c155810 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 -- 172.21.15.26:0/133973133 shutdown_connections 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 --2- 172.21.15.26:0/133973133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c0ac260 0x7f463c155810 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 --2- 172.21.15.26:0/133973133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c1592d0 0x7f463c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 -- 172.21.15.26:0/133973133 >> 172.21.15.26:0/133973133 conn(0x7f463c0c70b0 msgr2=0x7f463c0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 -- 172.21.15.26:0/133973133 shutdown_connections 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.789+0000 7f4643565640 1 -- 172.21.15.26:0/133973133 wait complete. 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.790+0000 7f4643565640 1 Processor -- start 2024-03-06T07:26:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4643565640 1 -- start start 2024-03-06T07:26:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4643565640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c0ac260 0x7f463c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4643565640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f463c152960 0x7f463c157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4643565640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c1564e0 0x7f463c157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4643565640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c159760 con 0x7f463c1564e0 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4643565640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0672c0 con 0x7f463c152960 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4643565640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c06e3a0 con 0x7f463c0ac260 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c1564e0 0x7f463c157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c1564e0 0x7f463c157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48546/0 (socket says 172.21.15.26:48546) 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 -- 172.21.15.26:0/3227826515 learned_addr learned my addr 172.21.15.26:0/3227826515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 -- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c0ac260 msgr2=0x7f463c06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c0ac260 0x7f463c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 -- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f463c152960 msgr2=0x7f463c157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f463c152960 0x7f463c157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.791+0000 7f4641b02640 1 -- 172.21.15.26:0/3227826515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c1eed60 con 0x7f463c1564e0 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.792+0000 7f4641b02640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c1564e0 0x7f463c157c40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.792+0000 7f4639ffb640 1 -- 172.21.15.26:0/3227826515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c039050 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.792+0000 7f4639ffb640 1 -- 172.21.15.26:0/3227826515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c045070 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.792+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c1dea60 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.792+0000 7f4639ffb640 1 -- 172.21.15.26:0/3227826515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c03e610 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.792+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c06c340 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.793+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463c04f330 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.793+0000 7f4639ffb640 1 -- 172.21.15.26:0/3227826515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f462c03e7b0 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.794+0000 7f4639ffb640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b4d0 0x7f462806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.794+0000 7f4639ffb640 1 -- 172.21.15.26:0/3227826515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f462c0acc00 con 0x7f463c1564e0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.794+0000 7f4641301640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b4d0 0x7f462806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.795+0000 7f4641301640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b4d0 0x7f462806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4634007fd0 tx=0x7f4634007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:03.798+0000 7f4639ffb640 1 -- 172.21.15.26:0/3227826515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f462c0b3080 con 0x7f463c1564e0 2024-03-06T07:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.048+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f463c06ebc0 con 0x7f463c1564e0 2024-03-06T07:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.049+0000 7f4639ffb640 1 -- 172.21.15.26:0/3227826515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f462c032e20 con 0x7f463c1564e0 2024-03-06T07:26:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:26:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:26:04.063 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:26:04.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b4d0 msgr2=0x7f462806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b4d0 0x7f462806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4634007fd0 tx=0x7f4634007920 comp rx=0 tx=0).stop 2024-03-06T07:26:04.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c1564e0 msgr2=0x7f463c157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c1564e0 0x7f463c157c40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 shutdown_connections 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b4d0 0x7f462806d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f463c1564e0 0x7f463c157c40 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f463c152960 0x7f463c157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 --2- 172.21.15.26:0/3227826515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f463c0ac260 0x7f463c06be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 >> 172.21.15.26:0/3227826515 conn(0x7f463c0c70b0 msgr2=0x7f463c06dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 shutdown_connections 2024-03-06T07:26:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.051+0000 7f4643565640 1 -- 172.21.15.26:0/3227826515 wait complete. 2024-03-06T07:26:04.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:04.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.164+0000 7f7c48857640 1 Processor -- start 2024-03-06T07:26:04.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c48857640 1 -- start start 2024-03-06T07:26:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c48857640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c440c7180 0x7f7c44159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c48857640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440c7c60 con 0x7f7c440c88b0 2024-03-06T07:26:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c48857640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440c8060 con 0x7f7c441527b0 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c48857640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c44159640 con 0x7f7c440c7180 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c3bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c440c7180 0x7f7c44159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c42d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7c440c88b0 0x7f7c440c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38062/0 (socket says 172.21.15.26:38062) 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.165+0000 7f7c42d9d640 1 -- 172.21.15.26:0/417657374 learned_addr learned my addr 172.21.15.26:0/417657374 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.166+0000 7f7c3bfff640 1 -- 172.21.15.26:0/417657374 >> v1:172.21.15.117:6789/0 conn(0x7f7c441527b0 legacy=0x7f7c44155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.166+0000 7f7c3bfff640 1 -- 172.21.15.26:0/417657374 >> v1:172.21.15.26:6789/0 conn(0x7f7c440c88b0 legacy=0x7f7c440c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.166+0000 7f7c3bfff640 1 -- 172.21.15.26:0/417657374 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440c76a0 con 0x7f7c440c7180 2024-03-06T07:26:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.166+0000 7f7c41d9b640 1 -- 172.21.15.26:0/417657374 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961828685 0 0) 0x7f7c3c001490 con 0x7f7c440c88b0 2024-03-06T07:26:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.166+0000 7f7c3bfff640 1 --2- 172.21.15.26:0/417657374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c440c7180 0x7f7c44159100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0097a0 tx=0x7f7c2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfaeff9f359b00a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c41d9b640 1 -- 172.21.15.26:0/417657374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0315a0 con 0x7f7c440c7180 2024-03-06T07:26:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c41d9b640 1 -- 172.21.15.26:0/417657374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c037070 con 0x7f7c440c7180 2024-03-06T07:26:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c41d9b640 1 -- 172.21.15.26:0/417657374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c031960 con 0x7f7c440c7180 2024-03-06T07:26:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c48857640 1 -- 172.21.15.26:0/417657374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c440c7180 msgr2=0x7f7c44159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c48857640 1 --2- 172.21.15.26:0/417657374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c440c7180 0x7f7c44159100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0097a0 tx=0x7f7c2c02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c48857640 1 -- 172.21.15.26:0/417657374 shutdown_connections 2024-03-06T07:26:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c48857640 1 --2- 172.21.15.26:0/417657374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c440c7180 0x7f7c44159100 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c48857640 1 -- 172.21.15.26:0/417657374 >> 172.21.15.26:0/417657374 conn(0x7f7c440ac560 msgr2=0x7f7c440ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c48857640 1 -- 172.21.15.26:0/417657374 shutdown_connections 2024-03-06T07:26:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.167+0000 7f7c48857640 1 -- 172.21.15.26:0/417657374 wait complete. 2024-03-06T07:26:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.168+0000 7f7c48857640 1 Processor -- start 2024-03-06T07:26:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.168+0000 7f7c48857640 1 -- start start 2024-03-06T07:26:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c48857640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c440c7180 0x7f7c441df340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c48857640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c440c88b0 0x7f7c441570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c48857640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c441527b0 0x7f7c44152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c48857640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44155600 con 0x7f7c441527b0 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c48857640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44155320 con 0x7f7c440c88b0 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c48857640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44152b90 con 0x7f7c440c7180 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c441527b0 0x7f7c44152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c441527b0 0x7f7c44152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48570/0 (socket says 172.21.15.26:48570) 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 -- 172.21.15.26:0/1772817136 learned_addr learned my addr 172.21.15.26:0/1772817136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c4259c640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c440c7180 0x7f7c441df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c3bfff640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c440c88b0 0x7f7c441570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 -- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c440c7180 msgr2=0x7f7c441df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c440c7180 0x7f7c441df340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 -- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c440c88b0 msgr2=0x7f7c441570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c440c88b0 0x7f7c441570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.169+0000 7f7c42d9d640 1 -- 172.21.15.26:0/1772817136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c441e1060 con 0x7f7c441527b0 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c4259c640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c440c7180 0x7f7c441df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c3bfff640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c440c88b0 0x7f7c441570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c42d9d640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c441527b0 0x7f7c44152ff0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00c820 tx=0x7f7c3c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c3affd640 1 -- 172.21.15.26:0/1772817136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c018070 con 0x7f7c441527b0 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c3affd640 1 -- 172.21.15.26:0/1772817136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c00a5e0 con 0x7f7c441527b0 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c441e1340 con 0x7f7c441527b0 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c3affd640 1 -- 172.21.15.26:0/1772817136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00a9d0 con 0x7f7c441527b0 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.170+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c441e07a0 con 0x7f7c441527b0 2024-03-06T07:26:04.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.171+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4404f330 con 0x7f7c441527b0 2024-03-06T07:26:04.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.171+0000 7f7c3affd640 1 -- 172.21.15.26:0/1772817136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c3c01b070 con 0x7f7c441527b0 2024-03-06T07:26:04.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.172+0000 7f7c3affd640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2406b4d0 0x7f7c2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.172+0000 7f7c3affd640 1 -- 172.21.15.26:0/1772817136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7c3c089690 con 0x7f7c441527b0 2024-03-06T07:26:04.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.172+0000 7f7c4259c640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2406b4d0 0x7f7c2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.172+0000 7f7c4259c640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2406b4d0 0x7f7c2406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400a830 tx=0x7f7c34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.176+0000 7f7c3affd640 1 -- 172.21.15.26:0/1772817136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c440c7c60 con 0x7f7c441527b0 2024-03-06T07:26:04.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.358+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c440bdee0 con 0x7f7c441527b0 2024-03-06T07:26:04.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.359+0000 7f7c3affd640 1 -- 172.21.15.26:0/1772817136 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+13186 (secure 0 0 0) 0x7f7c3c0586b0 con 0x7f7c441527b0 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":147,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:03.663766+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2406b4d0 msgr2=0x7f7c2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2406b4d0 0x7f7c2406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400a830 tx=0x7f7c34008040 comp rx=0 tx=0).stop 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c441527b0 msgr2=0x7f7c44152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c441527b0 0x7f7c44152ff0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00c820 tx=0x7f7c3c00cce0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 shutdown_connections 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2406b4d0 0x7f7c2406d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c441527b0 0x7f7c44152ff0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c440c88b0 0x7f7c441570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 --2- 172.21.15.26:0/1772817136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c440c7180 0x7f7c441df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 >> 172.21.15.26:0/1772817136 conn(0x7f7c440ac560 msgr2=0x7f7c4406b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 shutdown_connections 2024-03-06T07:26:04.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.361+0000 7f7c48857640 1 -- 172.21.15.26:0/1772817136 wait complete. 2024-03-06T07:26:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:04.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:04.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.480+0000 7f189a172640 1 Processor -- start 2024-03-06T07:26:04.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.480+0000 7f189a172640 1 -- start start 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.480+0000 7f189a172640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 0x7f18941494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.480+0000 7f189a172640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1894154330 con 0x7f18940ab310 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.480+0000 7f189a172640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1894153dc0 con 0x7f18940a97c0 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.480+0000 7f189a172640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c46a0 con 0x7f18941538c0 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f18937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f18940ab310 0x7f18940ab6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38066/0 (socket says 172.21.15.26:38066) 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f18937fe640 1 -- 172.21.15.26:0/330987633 learned_addr learned my addr 172.21.15.26:0/330987633 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f188affd640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 0x7f18941494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205105183 0 0) 0x7f1894154330 con 0x7f18940ab310 2024-03-06T07:26:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1880003520 con 0x7f18940ab310 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180712657 0 0) 0x7f1894153dc0 con 0x7f18940a97c0 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1894154330 con 0x7f18940a97c0 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993048034 0 0) 0x7f1880003520 con 0x7f18940ab310 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 msgr2=0x7f18941494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 0x7f18941494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 >> v1:172.21.15.117:6789/0 conn(0x7f18940a97c0 legacy=0x7f18940c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940c5d30 con 0x7f18940ab310 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f1884001670 con 0x7f18940ab310 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 >> v1:172.21.15.26:6789/0 conn(0x7f18940ab310 legacy=0x7f18940ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1880003b70 0x7f1880003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 0x7f1880004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880004d40 0x7f18800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c46a0 con 0x7f1880003b70 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188000baf0 con 0x7f1880004d40 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1880010100 con 0x7f18941538c0 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f18937fe640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1880003b70 0x7f1880003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f188affd640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 0x7f1880004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.481+0000 7f1893fff640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880004d40 0x7f18800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f18937fe640 1 -- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 msgr2=0x7f1880004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f18937fe640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18941538c0 0x7f1880004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f18937fe640 1 -- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880004d40 msgr2=0x7f18800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f18937fe640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880004d40 0x7f18800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f18937fe640 1 -- 172.21.15.26:0/330987633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1880003520 con 0x7f1880003b70 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f18937fe640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1880003b70 0x7f1880003f90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f188400c210 tx=0x7f1880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cde83507584e881 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884007ce0 con 0x7f1880003b70 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.482+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1884030070 con 0x7f1880003b70 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f1892ffd640 1 -- 172.21.15.26:0/330987633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884005d50 con 0x7f1880003b70 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f189a172640 1 -- 172.21.15.26:0/330987633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1880003b70 msgr2=0x7f1880003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f189a172640 1 --2- 172.21.15.26:0/330987633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1880003b70 0x7f1880003f90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f188400c210 tx=0x7f1880005830 comp rx=0 tx=0).stop 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f1893fff640 1 -- 172.21.15.26:0/330987633 reap_dead start 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f189a172640 1 -- 172.21.15.26:0/330987633 shutdown_connections 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f189a172640 1 -- 172.21.15.26:0/330987633 >> 172.21.15.26:0/330987633 conn(0x7f18940c6710 msgr2=0x7f18940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f189a172640 1 -- 172.21.15.26:0/330987633 shutdown_connections 2024-03-06T07:26:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.483+0000 7f189a172640 1 -- 172.21.15.26:0/330987633 wait complete. 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.485+0000 7f189a172640 1 Processor -- start 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.485+0000 7f189a172640 1 -- start start 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f189a172640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880003b70 0x7f189414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f189a172640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f189406c440 0x7f18941e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f189a172640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18941e8180 0x7f18940bf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f189a172640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c5240 con 0x7f18941e8180 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f189a172640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c62f0 con 0x7f1880003b70 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f189a172640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c53c0 con 0x7f189406c440 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18941e8180 0x7f18940bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18941e8180 0x7f18940bf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48598/0 (socket says 172.21.15.26:48598) 2024-03-06T07:26:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 -- 172.21.15.26:0/590748088 learned_addr learned my addr 172.21.15.26:0/590748088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f18937fe640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880003b70 0x7f189414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f188affd640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f189406c440 0x7f18941e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 -- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f189406c440 msgr2=0x7f18941e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f189406c440 0x7f18941e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 -- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880003b70 msgr2=0x7f189414f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880003b70 0x7f189414f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 -- 172.21.15.26:0/590748088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18941ea5c0 con 0x7f18941e8180 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f18937fe640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880003b70 0x7f189414f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f188affd640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f189406c440 0x7f18941e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1893fff640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18941e8180 0x7f18940bf930 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f18940c7440 tx=0x7f188c00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1890ff9640 1 -- 172.21.15.26:0/590748088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c019070 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1890ff9640 1 -- 172.21.15.26:0/590748088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c006e20 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18940a63d0 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.486+0000 7f1890ff9640 1 -- 172.21.15.26:0/590748088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c003b90 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.487+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1894072640 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.488+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189404f330 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.488+0000 7f1890ff9640 1 -- 172.21.15.26:0/590748088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f188c01c070 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.488+0000 7f1890ff9640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f187406b500 0x7f187406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.488+0000 7f1890ff9640 1 -- 172.21.15.26:0/590748088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f188c089410 con 0x7f18941e8180 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.489+0000 7f18937fe640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f187406b500 0x7f187406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.489+0000 7f18937fe640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f187406b500 0x7f187406d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1884034000 tx=0x7f1884007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.493+0000 7f1890ff9640 1 -- 172.21.15.26:0/590748088 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f188c0584e0 con 0x7f18941e8180 2024-03-06T07:26:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.694+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18940c5a70 con 0x7f18941e8180 2024-03-06T07:26:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.694+0000 7f1890ff9640 1 -- 172.21.15.26:0/590748088 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7f188c057de0 con 0x7f18941e8180 2024-03-06T07:26:04.697 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:26:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f187406b500 msgr2=0x7f187406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f187406b500 0x7f187406d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1884034000 tx=0x7f1884007000 comp rx=0 tx=0).stop 2024-03-06T07:26:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18941e8180 msgr2=0x7f18940bf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18941e8180 0x7f18940bf930 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f18940c7440 tx=0x7f188c00fd60 comp rx=0 tx=0).stop 2024-03-06T07:26:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 shutdown_connections 2024-03-06T07:26:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f187406b500 0x7f187406d9c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18941e8180 0x7f18940bf930 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f189406c440 0x7f18941e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 --2- 172.21.15.26:0/590748088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880003b70 0x7f189414f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 >> 172.21.15.26:0/590748088 conn(0x7f18940c6710 msgr2=0x7f18940c2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 shutdown_connections 2024-03-06T07:26:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.696+0000 7f189a172640 1 -- 172.21.15.26:0/590748088 wait complete. 2024-03-06T07:26:04.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:26:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 Processor -- start 2024-03-06T07:26:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 -- start start 2024-03-06T07:26:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7700c5480 0x7fa770155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5860 0x7fa77006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7701586f0 0x7fa770158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77006df40 con 0x7fa7701586f0 2024-03-06T07:26:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700c5cc0 con 0x7fa7700c5480 2024-03-06T07:26:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa775dd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7701591c0 con 0x7fa7700c5860 2024-03-06T07:26:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa76ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7701586f0 0x7fa770158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa76ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7701586f0 0x7fa770158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48612/0 (socket says 172.21.15.26:48612) 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.811+0000 7fa76ffff640 1 -- 172.21.15.26:0/166612802 learned_addr learned my addr 172.21.15.26:0/166612802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.812+0000 7fa76f7fe640 1 --2- 172.21.15.26:0/166612802 >> v2:172.21.15.117:3300/0 conn(0x7fa7700c5480 0x7fa770155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.812+0000 7fa76ffff640 1 -- 172.21.15.26:0/166612802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5860 msgr2=0x7fa77006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.812+0000 7fa76ffff640 1 --2- 172.21.15.26:0/166612802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5860 0x7fa77006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.812+0000 7fa76ffff640 1 -- 172.21.15.26:0/166612802 >> v2:172.21.15.117:3300/0 conn(0x7fa7700c5480 msgr2=0x7fa770155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.812+0000 7fa76ffff640 1 --2- 172.21.15.26:0/166612802 >> v2:172.21.15.117:3300/0 conn(0x7fa7700c5480 0x7fa770155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.812+0000 7fa76ffff640 1 -- 172.21.15.26:0/166612802 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7701594a0 con 0x7fa7701586f0 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.812+0000 7fa76ffff640 1 --2- 172.21.15.26:0/166612802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa770158b80 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fa7700c1cf0 tx=0x7fa76802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d329c7846b2fe2e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa76e7fc640 1 -- 172.21.15.26:0/166612802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76802c9e0 con 0x7fa7701586f0 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa76e7fc640 1 -- 172.21.15.26:0/166612802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76802cb80 con 0x7fa7701586f0 2024-03-06T07:26:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa76e7fc640 1 -- 172.21.15.26:0/166612802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680343f0 con 0x7fa7701586f0 2024-03-06T07:26:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 -- 172.21.15.26:0/166612802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 msgr2=0x7fa770158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 --2- 172.21.15.26:0/166612802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa770158b80 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fa7700c1cf0 tx=0x7fa76802be40 comp rx=0 tx=0).stop 2024-03-06T07:26:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 -- 172.21.15.26:0/166612802 shutdown_connections 2024-03-06T07:26:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 --2- 172.21.15.26:0/166612802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5860 0x7fa77006f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 --2- 172.21.15.26:0/166612802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa770158b80 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 --2- 172.21.15.26:0/166612802 >> v2:172.21.15.117:3300/0 conn(0x7fa7700c5480 0x7fa770155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 -- 172.21.15.26:0/166612802 >> 172.21.15.26:0/166612802 conn(0x7fa77006b1a0 msgr2=0x7fa77006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 -- 172.21.15.26:0/166612802 shutdown_connections 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.813+0000 7fa775dd0640 1 -- 172.21.15.26:0/166612802 wait complete. 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.814+0000 7fa775dd0640 1 Processor -- start 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.814+0000 7fa775dd0640 1 -- start start 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa775dd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5480 0x7fa7700aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa775dd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700c5860 0x7fa7700a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa775dd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa7701e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa775dd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770152a20 con 0x7fa7701586f0 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa775dd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770155680 con 0x7fa7700c5860 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa775dd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa770155050 con 0x7fa7700c5480 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa7701e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa7701e1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48622/0 (socket says 172.21.15.26:48622) 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 -- 172.21.15.26:0/3586751477 learned_addr learned my addr 172.21.15.26:0/3586751477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76f7fe640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5480 0x7fa7700aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76effd640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700c5860 0x7fa7700a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 -- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5480 msgr2=0x7fa7700aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5480 0x7fa7700aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 -- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700c5860 msgr2=0x7fa7700a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700c5860 0x7fa7700a7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.815+0000 7fa76ffff640 1 -- 172.21.15.26:0/3586751477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770158300 con 0x7fa7701586f0 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.816+0000 7fa76effd640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700c5860 0x7fa7700a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.816+0000 7fa76ffff640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa7701e1a10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa768036040 tx=0x7fa768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.816+0000 7fa757fff640 1 -- 172.21.15.26:0/3586751477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768044020 con 0x7fa7701586f0 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.816+0000 7fa757fff640 1 -- 172.21.15.26:0/3586751477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768040690 con 0x7fa7701586f0 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.816+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7700a8280 con 0x7fa7701586f0 2024-03-06T07:26:04.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.816+0000 7fa757fff640 1 -- 172.21.15.26:0/3586751477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768009510 con 0x7fa7701586f0 2024-03-06T07:26:04.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.816+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7700aca30 con 0x7fa7701586f0 2024-03-06T07:26:04.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.817+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77004f330 con 0x7fa7701586f0 2024-03-06T07:26:04.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.817+0000 7fa757fff640 1 -- 172.21.15.26:0/3586751477 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa768047020 con 0x7fa7701586f0 2024-03-06T07:26:04.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.818+0000 7fa757fff640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa75c06b500 0x7fa75c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.818+0000 7fa757fff640 1 -- 172.21.15.26:0/3586751477 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa7680ac870 con 0x7fa7701586f0 2024-03-06T07:26:04.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.818+0000 7fa76f7fe640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa75c06b500 0x7fa75c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.819+0000 7fa76f7fe640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa75c06b500 0x7fa75c06d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b7c0 tx=0x7fa760008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:04.822+0000 7fa757fff640 1 -- 172.21.15.26:0/3586751477 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa76804e070 con 0x7fa7701586f0 2024-03-06T07:26:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.007+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa7700c7620 con 0x7fa7701586f0 2024-03-06T07:26:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.007+0000 7fa757fff640 1 -- 172.21.15.26:0/3586751477 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa76807b940 con 0x7fa7701586f0 2024-03-06T07:26:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:05.010 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:26:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:26:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:26:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:05.011 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:26:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.009+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa75c06b500 msgr2=0x7fa75c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.009+0000 7fa775dd0640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa75c06b500 0x7fa75c06d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b7c0 tx=0x7fa760008040 comp rx=0 tx=0).stop 2024-03-06T07:26:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.009+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 msgr2=0x7fa7701e1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.009+0000 7fa775dd0640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa7701e1a10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa768036040 tx=0x7fa768004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.009+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 shutdown_connections 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.010+0000 7fa775dd0640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa75c06b500 0x7fa75c06d9c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.010+0000 7fa775dd0640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7701586f0 0x7fa7701e1a10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.010+0000 7fa775dd0640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700c5860 0x7fa7700a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.010+0000 7fa775dd0640 1 --2- 172.21.15.26:0/3586751477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700c5480 0x7fa7700aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.010+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 >> 172.21.15.26:0/3586751477 conn(0x7fa77006b1a0 msgr2=0x7fa77006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.010+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 shutdown_connections 2024-03-06T07:26:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.010+0000 7fa775dd0640 1 -- 172.21.15.26:0/3586751477 wait complete. 2024-03-06T07:26:05.023 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:26:05.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:26:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.126+0000 7fc62dfae640 1 Processor -- start 2024-03-06T07:26:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.126+0000 7fc62dfae640 1 -- start start 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc62dfae640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc6280c2200 0x7fc6280c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc62dfae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6280a6cc0 0x7fc628155460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc62dfae640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6281596c0 con 0x7fc6281559a0 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc62dfae640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc628158ab0 con 0x7fc6280a6cc0 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc62dfae640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc628158d90 con 0x7fc6280c2200 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc626ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6280a6cc0 0x7fc628155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc627fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc6281559a0 0x7fc6280c0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38080/0 (socket says 172.21.15.26:38080) 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc627fff640 1 -- 172.21.15.26:0/1597450210 learned_addr learned my addr 172.21.15.26:0/1597450210 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc6277fe640 1 --2- 172.21.15.26:0/1597450210 >> v2:172.21.15.117:3300/0 conn(0x7fc6280c2200 0x7fc6280c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc626ffd640 1 -- 172.21.15.26:0/1597450210 >> v2:172.21.15.117:3300/0 conn(0x7fc6280c2200 msgr2=0x7fc6280c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc626ffd640 1 --2- 172.21.15.26:0/1597450210 >> v2:172.21.15.117:3300/0 conn(0x7fc6280c2200 0x7fc6280c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc626ffd640 1 -- 172.21.15.26:0/1597450210 >> v1:172.21.15.26:6789/0 conn(0x7fc6281559a0 legacy=0x7fc6280c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.127+0000 7fc626ffd640 1 -- 172.21.15.26:0/1597450210 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc628159070 con 0x7fc6280a6cc0 2024-03-06T07:26:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc626ffd640 1 --2- 172.21.15.26:0/1597450210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280a6cc0 0x7fc628155460 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc62806d190 tx=0x7fc61802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b533929e44f24460 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc6267fc640 1 -- 172.21.15.26:0/1597450210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180315a0 con 0x7fc6280a6cc0 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc6267fc640 1 -- 172.21.15.26:0/1597450210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc618037070 con 0x7fc6280a6cc0 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc6267fc640 1 -- 172.21.15.26:0/1597450210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618031960 con 0x7fc6280a6cc0 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc62dfae640 1 -- 172.21.15.26:0/1597450210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280a6cc0 msgr2=0x7fc628155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc62dfae640 1 --2- 172.21.15.26:0/1597450210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280a6cc0 0x7fc628155460 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc62806d190 tx=0x7fc61802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc62dfae640 1 -- 172.21.15.26:0/1597450210 shutdown_connections 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc62dfae640 1 --2- 172.21.15.26:0/1597450210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280a6cc0 0x7fc628155460 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc62dfae640 1 --2- 172.21.15.26:0/1597450210 >> v2:172.21.15.117:3300/0 conn(0x7fc6280c2200 0x7fc6280c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.128+0000 7fc62dfae640 1 -- 172.21.15.26:0/1597450210 >> 172.21.15.26:0/1597450210 conn(0x7fc6280c4300 msgr2=0x7fc6280c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.129+0000 7fc62dfae640 1 -- 172.21.15.26:0/1597450210 shutdown_connections 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.129+0000 7fc62dfae640 1 -- 172.21.15.26:0/1597450210 wait complete. 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 Processor -- start 2024-03-06T07:26:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 -- start start 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6280a6cc0 0x7fc6281546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c2200 0x7fc62814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6281559a0 0x7fc6281eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628073580 con 0x7fc6280c2200 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62806be10 con 0x7fc6280a6cc0 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.130+0000 7fc62dfae640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc628158740 con 0x7fc6281559a0 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c2200 0x7fc62814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c2200 0x7fc62814b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48636/0 (socket says 172.21.15.26:48636) 2024-03-06T07:26:05.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 -- 172.21.15.26:0/180263122 learned_addr learned my addr 172.21.15.26:0/180263122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc6277fe640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6280a6cc0 0x7fc6281546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc627fff640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6281559a0 0x7fc6281eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 -- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6281559a0 msgr2=0x7fc6281eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6281559a0 0x7fc6281eafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 -- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6280a6cc0 msgr2=0x7fc6281546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6280a6cc0 0x7fc6281546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 -- 172.21.15.26:0/180263122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc628154bf0 con 0x7fc6280c2200 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc626ffd640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c2200 0x7fc62814b390 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fc618000c00 tx=0x7fc6180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc61ffff640 1 -- 172.21.15.26:0/180263122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618032840 con 0x7fc6280c2200 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc61ffff640 1 -- 172.21.15.26:0/180263122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc618032a00 con 0x7fc6280c2200 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc61ffff640 1 -- 172.21.15.26:0/180263122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618032df0 con 0x7fc6280c2200 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc628154e90 con 0x7fc6280c2200 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.131+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6281eea20 con 0x7fc6280c2200 2024-03-06T07:26:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.132+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6281ee530 con 0x7fc6280c2200 2024-03-06T07:26:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.133+0000 7fc61ffff640 1 -- 172.21.15.26:0/180263122 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc61804d3f0 con 0x7fc6280c2200 2024-03-06T07:26:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.133+0000 7fc61ffff640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.133+0000 7fc61ffff640 1 -- 172.21.15.26:0/180263122 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc61802f030 con 0x7fc6280c2200 2024-03-06T07:26:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.133+0000 7fc6277fe640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.134+0000 7fc6277fe640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc610006fd0 tx=0x7fc610008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.137+0000 7fc61ffff640 1 -- 172.21.15.26:0/180263122 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc618040070 con 0x7fc6280c2200 2024-03-06T07:26:05.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.319+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc62804f330 con 0x7fc6280c2200 2024-03-06T07:26:05.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.670+0000 7fc61ffff640 1 -- 172.21.15.26:0/180263122 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v148) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc61807a550 con 0x7fc6280c2200 2024-03-06T07:26:05.672 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:26:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 msgr2=0x7fc60806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc610006fd0 tx=0x7fc610008040 comp rx=0 tx=0).stop 2024-03-06T07:26:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c2200 msgr2=0x7fc62814b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c2200 0x7fc62814b390 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fc618000c00 tx=0x7fc6180046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 shutdown_connections 2024-03-06T07:26:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6281559a0 0x7fc6281eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c2200 0x7fc62814b390 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 --2- 172.21.15.26:0/180263122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6280a6cc0 0x7fc6281546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.673+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 >> 172.21.15.26:0/180263122 conn(0x7fc6280c4300 msgr2=0x7fc6280c1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.675+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 shutdown_connections 2024-03-06T07:26:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.676+0000 7fc62dfae640 1 -- 172.21.15.26:0/180263122 wait complete. 2024-03-06T07:26:05.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:26:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.802+0000 7fc962b93640 1 Processor -- start 2024-03-06T07:26:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.802+0000 7fc962b93640 1 -- start start 2024-03-06T07:26:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.802+0000 7fc962b93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc95c0c5370 0x7fc95c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.802+0000 7fc962b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 0x7fc95c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.802+0000 7fc962b93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c152dd0 con 0x7fc95c1529f0 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.802+0000 7fc962b93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95c159710 con 0x7fc95c0c5370 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.802+0000 7fc962b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c0c68b0 con 0x7fc95c155530 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc96092f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc95c1529f0 0x7fc95c0c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38088/0 (socket says 172.21.15.26:38088) 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc96092f640 1 -- 172.21.15.26:0/3069981255 learned_addr learned my addr 172.21.15.26:0/3069981255 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306985470 0 0) 0x7fc95c152dd0 con 0x7fc95c1529f0 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc94c003520 con 0x7fc95c1529f0 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95bfff640 1 --2- 172.21.15.26:0/3069981255 >> v2:172.21.15.117:3300/0 conn(0x7fc95c0c5370 0x7fc95c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc961130640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 0x7fc95c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223595363 0 0) 0x7fc94c003520 con 0x7fc95c1529f0 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 msgr2=0x7fc95c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 0x7fc95c1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 >> v2:172.21.15.117:3300/0 conn(0x7fc95c0c5370 msgr2=0x7fc95c0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 --2- 172.21.15.26:0/3069981255 >> v2:172.21.15.117:3300/0 conn(0x7fc95c0c5370 0x7fc95c0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c158b00 con 0x7fc95c1529f0 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fc950001670 con 0x7fc95c1529f0 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 >> v1:172.21.15.26:6789/0 conn(0x7fc95c1529f0 legacy=0x7fc95c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 0x7fc94c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94c003f30 0x7fc94c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc94c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c159710 con 0x7fc94c004d40 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c0c68b0 con 0x7fc94c003f30 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c00bac0 con 0x7fc95c155530 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc961130640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc94c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc95bfff640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94c003f30 0x7fc94c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.803+0000 7fc96092f640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 0x7fc94c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc961130640 1 -- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 msgr2=0x7fc94c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc961130640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c155530 0x7fc94c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc961130640 1 -- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94c003f30 msgr2=0x7fc94c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc961130640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94c003f30 0x7fc94c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc961130640 1 -- 172.21.15.26:0/3069981255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c003520 con 0x7fc94c004d40 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc95bfff640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94c003f30 0x7fc94c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc961130640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc94c0051c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc954007390 tx=0x7fc94c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f497e51c6e32d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954039070 con 0x7fc94c004d40 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95402ccb0 con 0x7fc94c004d40 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.804+0000 7fc95b7fe640 1 -- 172.21.15.26:0/3069981255 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954009be0 con 0x7fc94c004d40 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.805+0000 7fc962b93640 1 -- 172.21.15.26:0/3069981255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 msgr2=0x7fc94c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.805+0000 7fc962b93640 1 --2- 172.21.15.26:0/3069981255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc94c0051c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc954007390 tx=0x7fc94c005830 comp rx=0 tx=0).stop 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.805+0000 7fc961130640 1 -- 172.21.15.26:0/3069981255 reap_dead start 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.805+0000 7fc962b93640 1 -- 172.21.15.26:0/3069981255 shutdown_connections 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.805+0000 7fc962b93640 1 -- 172.21.15.26:0/3069981255 >> 172.21.15.26:0/3069981255 conn(0x7fc95c0ac580 msgr2=0x7fc95c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.805+0000 7fc962b93640 1 -- 172.21.15.26:0/3069981255 shutdown_connections 2024-03-06T07:26:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.805+0000 7fc962b93640 1 -- 172.21.15.26:0/3069981255 wait complete. 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.806+0000 7fc962b93640 1 Processor -- start 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.806+0000 7fc962b93640 1 -- start start 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc962b93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc95c0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc962b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c1eea40 0x7fc95c0ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc962b93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc95c1534f0 0x7fc95c153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc962b93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1590c0 con 0x7fc94c004d40 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc962b93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c0c9000 con 0x7fc95c1534f0 2024-03-06T07:26:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc962b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1593a0 con 0x7fc95c1eea40 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc95c0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc95c0ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48658/0 (socket says 172.21.15.26:48658) 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 -- 172.21.15.26:0/2152691870 learned_addr learned my addr 172.21.15.26:0/2152691870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc961130640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc95c1534f0 0x7fc95c153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc95bfff640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c1eea40 0x7fc95c0ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 -- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c1eea40 msgr2=0x7fc95c0ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c1eea40 0x7fc95c0ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 -- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc95c1534f0 msgr2=0x7fc95c153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc95c1534f0 0x7fc95c153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc96092f640 1 -- 172.21.15.26:0/2152691870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c0cb140 con 0x7fc94c004d40 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.807+0000 7fc95bfff640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c1eea40 0x7fc95c0ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.808+0000 7fc96092f640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc95c0ca9d0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc9500017e0 tx=0x7fc9500077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.808+0000 7fc9597fa640 1 -- 172.21.15.26:0/2152691870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000fa80 con 0x7fc94c004d40 2024-03-06T07:26:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.808+0000 7fc9597fa640 1 -- 172.21.15.26:0/2152691870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950018070 con 0x7fc94c004d40 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.808+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95c1edf60 con 0x7fc94c004d40 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.808+0000 7fc9597fa640 1 -- 172.21.15.26:0/2152691870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000fea0 con 0x7fc94c004d40 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.808+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c1ee520 con 0x7fc94c004d40 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.809+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95c04f330 con 0x7fc94c004d40 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.809+0000 7fc9597fa640 1 -- 172.21.15.26:0/2152691870 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc95001d460 con 0x7fc94c004d40 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.810+0000 7fc9597fa640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc94006b500 0x7fc94006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.810+0000 7fc9597fa640 1 -- 172.21.15.26:0/2152691870 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fc95008a390 con 0x7fc94c004d40 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.810+0000 7fc95bfff640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc94006b500 0x7fc94006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.811+0000 7fc95bfff640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc94006b500 0x7fc94006d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc95c14e760 tx=0x7fc948008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.815+0000 7fc9597fa640 1 -- 172.21.15.26:0/2152691870 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc9500591d0 con 0x7fc94c004d40 2024-03-06T07:26:05.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:05.996+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc95c1ee6f0 con 0x7fc94c004d40 2024-03-06T07:26:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.675+0000 7fc9597fa640 1 -- 172.21.15.26:0/2152691870 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v149) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc950058ad0 con 0x7fc94c004d40 2024-03-06T07:26:06.729 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.678+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc94006b500 msgr2=0x7fc94006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.678+0000 7fc962b93640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc94006b500 0x7fc94006d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc95c14e760 tx=0x7fc948008040 comp rx=0 tx=0).stop 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.678+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 msgr2=0x7fc95c0ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.678+0000 7fc962b93640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc95c0ca9d0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc9500017e0 tx=0x7fc9500077a0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 shutdown_connections 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc94006b500 0x7fc94006d9c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc95c1534f0 0x7fc95c153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc95c1eea40 0x7fc95c0ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 --2- 172.21.15.26:0/2152691870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc94c004d40 0x7fc95c0ca9d0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 >> 172.21.15.26:0/2152691870 conn(0x7fc95c0ac580 msgr2=0x7fc95c151900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 shutdown_connections 2024-03-06T07:26:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.679+0000 7fc962b93640 1 -- 172.21.15.26:0/2152691870 wait complete. 2024-03-06T07:26:06.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:26:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.836+0000 7f7104525640 1 Processor -- start 2024-03-06T07:26:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.836+0000 7f7104525640 1 -- start start 2024-03-06T07:26:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f7104525640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 0x7f70fc0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f7104525640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0c1f90 con 0x7f70fc0a6290 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f7104525640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0c11d0 con 0x7f70fc0a8ff0 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f7104525640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc14f850 con 0x7f70fc155cf0 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f7101ac0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70fc0a6290 0x7f70fc0a9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38092/0 (socket says 172.21.15.26:38092) 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f7101ac0640 1 -- 172.21.15.26:0/712391384 learned_addr learned my addr 172.21.15.26:0/712391384 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f7102ac2640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 0x7f70fc0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438017171 0 0) 0x7f70fc0c1f90 con 0x7f70fc0a6290 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.837+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e8003520 con 0x7f70fc0a6290 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519170044 0 0) 0x7f70e8003520 con 0x7f70fc0a6290 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 msgr2=0x7f70fc0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 0x7f70fc0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 >> v1:172.21.15.117:6789/0 conn(0x7f70fc0a8ff0 legacy=0x7f70fc0ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0c14b0 con 0x7f70fc0a6290 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f70ec001670 con 0x7f70fc0a6290 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 >> v1:172.21.15.26:6789/0 conn(0x7f70fc0a6290 legacy=0x7f70fc0a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e80039e0 0x7f70e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 0x7f70e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e8004d40 0x7f70e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc14f850 con 0x7f70e80039e0 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e800dd90 con 0x7f70e8004d40 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e800df90 con 0x7f70fc155cf0 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71022c1640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e80039e0 0x7f70e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f7102ac2640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e8004d40 0x7f70e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f7101ac0640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 0x7f70e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71022c1640 1 -- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 msgr2=0x7f70e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71022c1640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70fc155cf0 0x7f70e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71022c1640 1 -- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e8004d40 msgr2=0x7f70e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:06.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71022c1640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e8004d40 0x7f70e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.838+0000 7f71022c1640 1 -- 172.21.15.26:0/712391384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e8003520 con 0x7f70e80039e0 2024-03-06T07:26:06.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.839+0000 7f71022c1640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e80039e0 0x7f70e8003e00 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f70f8001750 tx=0x7f70e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347b0e19a3c9d900 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:06.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.839+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8001370 con 0x7f70e80039e0 2024-03-06T07:26:06.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.839+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f80052d0 con 0x7f70e80039e0 2024-03-06T07:26:06.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.839+0000 7f71012bf640 1 -- 172.21.15.26:0/712391384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8003830 con 0x7f70e80039e0 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.840+0000 7f7104525640 1 -- 172.21.15.26:0/712391384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e80039e0 msgr2=0x7f70e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.840+0000 7f7104525640 1 --2- 172.21.15.26:0/712391384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e80039e0 0x7f70e8003e00 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f70f8001750 tx=0x7f70e8005830 comp rx=0 tx=0).stop 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.840+0000 7f7102ac2640 1 -- 172.21.15.26:0/712391384 reap_dead start 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.840+0000 7f7104525640 1 -- 172.21.15.26:0/712391384 shutdown_connections 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.840+0000 7f7104525640 1 -- 172.21.15.26:0/712391384 >> 172.21.15.26:0/712391384 conn(0x7f70fc06afc0 msgr2=0x7f70fc06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.840+0000 7f7104525640 1 -- 172.21.15.26:0/712391384 shutdown_connections 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.840+0000 7f7104525640 1 -- 172.21.15.26:0/712391384 wait complete. 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 Processor -- start 2024-03-06T07:26:06.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 -- start start 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e80039e0 0x7f70fc0bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70fc0c5eb0 0x7f70fc142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70fc152320 0x7f70fc142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0c1a70 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc156260 con 0x7f70fc152320 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.841+0000 7f7104525640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0ab900 con 0x7f70e80039e0 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70fc0c5eb0 0x7f70fc142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70fc0c5eb0 0x7f70fc142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48674/0 (socket says 172.21.15.26:48674) 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 -- 172.21.15.26:0/1557086704 learned_addr learned my addr 172.21.15.26:0/1557086704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f71022c1640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e80039e0 0x7f70fc0bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7102ac2640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70fc152320 0x7f70fc142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 -- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e80039e0 msgr2=0x7f70fc0bfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e80039e0 0x7f70fc0bfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 -- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70fc152320 msgr2=0x7f70fc142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70fc152320 0x7f70fc142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 -- 172.21.15.26:0/1557086704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc1eb8c0 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7101ac0640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70fc0c5eb0 0x7f70fc142810 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0072f0 tx=0x7f70ec0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f70f2ffd640 1 -- 172.21.15.26:0/1557086704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec01a070 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f70f2ffd640 1 -- 172.21.15.26:0/1557086704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00f960 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc14fd10 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f70f2ffd640 1 -- 172.21.15.26:0/1557086704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00fd10 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.842+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc14ffb0 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.843+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc04f330 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.844+0000 7f70f2ffd640 1 -- 172.21.15.26:0/1557086704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70ec01d070 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.844+0000 7f70f2ffd640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.844+0000 7f70f2ffd640 1 -- 172.21.15.26:0/1557086704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f70ec089cf0 con 0x7f70fc0c5eb0 2024-03-06T07:26:06.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.844+0000 7f71022c1640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.845+0000 7f71022c1640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70e006b4d0 0x7f70e006d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f70f8033000 tx=0x7f70f8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:06.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:06.849+0000 7f70f2ffd640 1 -- 172.21.15.26:0/1557086704 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70ec08e070 con 0x7f70fc0c5eb0 2024-03-06T07:26:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.069+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f70fc150180 con 0x7f70fc0c5eb0 2024-03-06T07:26:07.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.683+0000 7f70f2ffd640 1 -- 172.21.15.26:0/1557086704 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v135) v1 ==== 174+0+0 (secure 0 0 0) 0x7f70ec058ad0 con 0x7f70fc0c5eb0 2024-03-06T07:26:07.692 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 23 and data pool 24 2024-03-06T07:26:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.698+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70e006b4d0 msgr2=0x7f70e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.698+0000 7f7104525640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70e006b4d0 0x7f70e006d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f70f8033000 tx=0x7f70f8009030 comp rx=0 tx=0).stop 2024-03-06T07:26:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.698+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70fc0c5eb0 msgr2=0x7f70fc142810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.698+0000 7f7104525640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70fc0c5eb0 0x7f70fc142810 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0072f0 tx=0x7f70ec0077b0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.702+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 shutdown_connections 2024-03-06T07:26:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.702+0000 7f7104525640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.702+0000 7f7104525640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70fc152320 0x7f70fc142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.702+0000 7f7104525640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70fc0c5eb0 0x7f70fc142810 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.702+0000 7f7104525640 1 --2- 172.21.15.26:0/1557086704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e80039e0 0x7f70fc0bfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.702+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 >> 172.21.15.26:0/1557086704 conn(0x7f70fc06afc0 msgr2=0x7f70fc0ab1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.702+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 shutdown_connections 2024-03-06T07:26:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.703+0000 7f7104525640 1 -- 172.21.15.26:0/1557086704 wait complete. 2024-03-06T07:26:07.724 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:07.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.827+0000 7fba02735640 1 Processor -- start 2024-03-06T07:26:07.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.827+0000 7fba02735640 1 -- start start 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fba02735640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9fc0c6780 0x7fb9fc158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fba02735640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9fc1592d0 0x7fb9fc0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fba02735640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc072630 con 0x7fb9fc06da40 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fba02735640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc06de20 con 0x7fb9fc0c6780 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fba02735640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc072910 con 0x7fb9fc1592d0 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fb9fb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9fc0c6780 0x7fb9fc158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fb9fb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9fc0c6780 0x7fb9fc158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44666/0 (socket says 172.21.15.26:44666) 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/199269098 learned_addr learned my addr 172.21.15.26:0/199269098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fba00cd2640 1 --2- 172.21.15.26:0/199269098 >> v2:172.21.15.117:3300/0 conn(0x7fb9fc1592d0 0x7fb9fc0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/199269098 >> v2:172.21.15.117:3300/0 conn(0x7fb9fc1592d0 msgr2=0x7fb9fc0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fb9fb7fe640 1 --2- 172.21.15.26:0/199269098 >> v2:172.21.15.117:3300/0 conn(0x7fb9fc1592d0 0x7fb9fc0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/199269098 >> v1:172.21.15.117:6789/0 conn(0x7fb9fc06da40 legacy=0x7fb9fc155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/199269098 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc072bf0 con 0x7fb9fc0c6780 2024-03-06T07:26:07.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.828+0000 7fba00cd2640 1 --2- 172.21.15.26:0/199269098 >> v2:172.21.15.117:3300/0 conn(0x7fb9fc1592d0 0x7fb9fc0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:07.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.829+0000 7fb9fb7fe640 1 --2- 172.21.15.26:0/199269098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc158d90 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c4b80 tx=0x7fb9e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db99800f06fc99d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.829+0000 7fb9faffd640 1 -- 172.21.15.26:0/199269098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e402e070 con 0x7fb9fc0c6780 2024-03-06T07:26:07.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.829+0000 7fb9faffd640 1 -- 172.21.15.26:0/199269098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e4037070 con 0x7fb9fc0c6780 2024-03-06T07:26:07.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.829+0000 7fb9faffd640 1 -- 172.21.15.26:0/199269098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4031a00 con 0x7fb9fc0c6780 2024-03-06T07:26:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 -- 172.21.15.26:0/199269098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 msgr2=0x7fb9fc158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 --2- 172.21.15.26:0/199269098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc158d90 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c4b80 tx=0x7fb9e402b880 comp rx=0 tx=0).stop 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 -- 172.21.15.26:0/199269098 shutdown_connections 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 --2- 172.21.15.26:0/199269098 >> v2:172.21.15.117:3300/0 conn(0x7fb9fc1592d0 0x7fb9fc0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 --2- 172.21.15.26:0/199269098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc158d90 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 -- 172.21.15.26:0/199269098 >> 172.21.15.26:0/199269098 conn(0x7fb9fc0aa630 msgr2=0x7fb9fc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 -- 172.21.15.26:0/199269098 shutdown_connections 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.830+0000 7fba02735640 1 -- 172.21.15.26:0/199269098 wait complete. 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.831+0000 7fba02735640 1 Processor -- start 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.831+0000 7fba02735640 1 -- start start 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fba02735640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9fc06da40 0x7fb9fc0ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fba02735640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc1ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fba02735640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9fc1592d0 0x7fb9fc1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fba02735640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c3940 con 0x7fb9fc0c6780 2024-03-06T07:26:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fba02735640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c37c0 con 0x7fb9fc1592d0 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fba02735640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c3ac0 con 0x7fb9fc06da40 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc1ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc1ef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44670/0 (socket says 172.21.15.26:44670) 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/2571301911 learned_addr learned my addr 172.21.15.26:0/2571301911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fba00cd2640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9fc1592d0 0x7fb9fc1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fbfff640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9fc06da40 0x7fb9fc0ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9fc06da40 msgr2=0x7fb9fc0ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9fc06da40 0x7fb9fc0ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9fc1592d0 msgr2=0x7fb9fc1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9fc1592d0 0x7fb9fc1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 -- 172.21.15.26:0/2571301911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0acff0 con 0x7fb9fc0c6780 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.832+0000 7fb9fb7fe640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc1ef020 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4000c00 tx=0x7fb9e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.833+0000 7fb9f8ff9640 1 -- 172.21.15.26:0/2571301911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4032880 con 0x7fb9fc0c6780 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.833+0000 7fb9f8ff9640 1 -- 172.21.15.26:0/2571301911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e4032a20 con 0x7fb9fc0c6780 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.833+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc0ad2d0 con 0x7fb9fc0c6780 2024-03-06T07:26:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.833+0000 7fb9f8ff9640 1 -- 172.21.15.26:0/2571301911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4032e10 con 0x7fb9fc0c6780 2024-03-06T07:26:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.833+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc157cf0 con 0x7fb9fc0c6780 2024-03-06T07:26:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.834+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc0c5d40 con 0x7fb9fc0c6780 2024-03-06T07:26:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.834+0000 7fb9f8ff9640 1 -- 172.21.15.26:0/2571301911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9e40453f0 con 0x7fb9fc0c6780 2024-03-06T07:26:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.834+0000 7fb9f8ff9640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9dc06b4d0 0x7fb9dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.834+0000 7fb9f8ff9640 1 -- 172.21.15.26:0/2571301911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb9e40ac210 con 0x7fb9fc0c6780 2024-03-06T07:26:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.835+0000 7fb9fbfff640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9dc06b4d0 0x7fb9dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.835+0000 7fb9fbfff640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9dc06b4d0 0x7fb9dc06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec006fd0 tx=0x7fb9ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:07.839+0000 7fb9f8ff9640 1 -- 172.21.15.26:0/2571301911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb9fc0c5d40 con 0x7fb9fc0c6780 2024-03-06T07:26:08.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.022+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9fc0c5a60 con 0x7fb9fc0c6780 2024-03-06T07:26:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.023+0000 7fb9f8ff9640 1 -- 172.21.15.26:0/2571301911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16909 (secure 0 0 0) 0x7fb9fc0c5a60 con 0x7fb9fc0c6780 2024-03-06T07:26:08.026 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:08.026 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":150,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:07.071386+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9dc06b4d0 msgr2=0x7fb9dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9dc06b4d0 0x7fb9dc06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec006fd0 tx=0x7fb9ec008040 comp rx=0 tx=0).stop 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 msgr2=0x7fb9fc1ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc1ef020 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4000c00 tx=0x7fb9e40046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 shutdown_connections 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb9dc06b4d0 0x7fb9dc06d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9fc1592d0 0x7fb9fc1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9fc0c6780 0x7fb9fc1ef020 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 --2- 172.21.15.26:0/2571301911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9fc06da40 0x7fb9fc0ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 >> 172.21.15.26:0/2571301911 conn(0x7fb9fc0aa630 msgr2=0x7fb9fc154870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 shutdown_connections 2024-03-06T07:26:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.025+0000 7fba02735640 1 -- 172.21.15.26:0/2571301911 wait complete. 2024-03-06T07:26:08.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.138+0000 7f0fef393640 1 Processor -- start 2024-03-06T07:26:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.138+0000 7f0fef393640 1 -- start start 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fef393640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0fe80c8a00 0x7f0fe8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fef393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe8159460 0x7f0fe80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fef393640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe80c7760 con 0x7f0fe8152920 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fef393640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe81587b0 con 0x7f0fe80c8a00 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fef393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80c7a40 con 0x7f0fe8159460 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fec92e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0fe80c8a00 0x7f0fe8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fed12f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0fe8152920 0x7f0fe8155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54946/0 (socket says 172.21.15.26:54946) 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fed12f640 1 -- 172.21.15.26:0/1887280129 learned_addr learned my addr 172.21.15.26:0/1887280129 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fed930640 1 --2- 172.21.15.26:0/1887280129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe8159460 0x7f0fe80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fec92e640 1 -- 172.21.15.26:0/1887280129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe8159460 msgr2=0x7f0fe80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fec92e640 1 --2- 172.21.15.26:0/1887280129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe8159460 0x7f0fe80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fec92e640 1 -- 172.21.15.26:0/1887280129 >> v1:172.21.15.26:6789/0 conn(0x7f0fe8152920 legacy=0x7f0fe8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fec92e640 1 -- 172.21.15.26:0/1887280129 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80c7d20 con 0x7f0fe80c8a00 2024-03-06T07:26:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fdffff640 1 -- 172.21.15.26:0/1887280129 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452415162 0 0) 0x7f0fe80c7760 con 0x7f0fe8152920 2024-03-06T07:26:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.139+0000 7f0fed930640 1 --2- 172.21.15.26:0/1887280129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe8159460 0x7f0fe80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:08.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.140+0000 7f0fec92e640 1 --2- 172.21.15.26:0/1887280129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c8a00 0x7f0fe8158f20 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80097a0 tx=0x7f0fd802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e42a2cd3894122 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.140+0000 7f0fdffff640 1 -- 172.21.15.26:0/1887280129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80315a0 con 0x7f0fe80c8a00 2024-03-06T07:26:08.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.140+0000 7f0fdffff640 1 -- 172.21.15.26:0/1887280129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8037070 con 0x7f0fe80c8a00 2024-03-06T07:26:08.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.140+0000 7f0fdffff640 1 -- 172.21.15.26:0/1887280129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8031960 con 0x7f0fe80c8a00 2024-03-06T07:26:08.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 -- 172.21.15.26:0/1887280129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c8a00 msgr2=0x7f0fe8158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 --2- 172.21.15.26:0/1887280129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c8a00 0x7f0fe8158f20 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80097a0 tx=0x7f0fd802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 -- 172.21.15.26:0/1887280129 shutdown_connections 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 --2- 172.21.15.26:0/1887280129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c8a00 0x7f0fe8158f20 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 --2- 172.21.15.26:0/1887280129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe8159460 0x7f0fe80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 -- 172.21.15.26:0/1887280129 >> 172.21.15.26:0/1887280129 conn(0x7f0fe80ac560 msgr2=0x7f0fe80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 -- 172.21.15.26:0/1887280129 shutdown_connections 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.141+0000 7f0fef393640 1 -- 172.21.15.26:0/1887280129 wait complete. 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.142+0000 7f0fef393640 1 Processor -- start 2024-03-06T07:26:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.142+0000 7f0fef393640 1 -- start start 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.142+0000 7f0fef393640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80c8a00 0x7f0fe814df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fef393640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe8152920 0x7f0fe8147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fef393640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c2da0 0x7f0fe8148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fef393640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80c7390 con 0x7f0fe80c2da0 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fef393640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe806d030 con 0x7f0fe8152920 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fef393640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80c7510 con 0x7f0fe80c8a00 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c2da0 0x7f0fe8148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c2da0 0x7f0fe8148140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44696/0 (socket says 172.21.15.26:44696) 2024-03-06T07:26:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 -- 172.21.15.26:0/3347290913 learned_addr learned my addr 172.21.15.26:0/3347290913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 -- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80c8a00 msgr2=0x7f0fe814df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80c8a00 0x7f0fe814df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 -- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe8152920 msgr2=0x7f0fe8147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe8152920 0x7f0fe8147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 -- 172.21.15.26:0/3347290913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe81ef790 con 0x7f0fe80c2da0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fed930640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c2da0 0x7f0fe8148140 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007d90 tx=0x7f0fe400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fddffb640 1 -- 172.21.15.26:0/3347290913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe401a070 con 0x7f0fe80c2da0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fddffb640 1 -- 172.21.15.26:0/3347290913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe4003990 con 0x7f0fe80c2da0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe80c0d70 con 0x7f0fe80c2da0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fddffb640 1 -- 172.21.15.26:0/3347290913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4003d80 con 0x7f0fe80c2da0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.143+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe80c1330 con 0x7f0fe80c2da0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.145+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe8067910 con 0x7f0fe80c2da0 2024-03-06T07:26:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.145+0000 7f0fddffb640 1 -- 172.21.15.26:0/3347290913 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fe401d070 con 0x7f0fe80c2da0 2024-03-06T07:26:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.145+0000 7f0fddffb640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fcc06b500 0x7f0fcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.145+0000 7f0fddffb640 1 -- 172.21.15.26:0/3347290913 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fe408a4a0 con 0x7f0fe80c2da0 2024-03-06T07:26:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.146+0000 7f0fed12f640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fcc06b500 0x7f0fcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.146+0000 7f0fed12f640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fcc06b500 0x7f0fcc06d9c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007fd0 tx=0x7f0fe0007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.150+0000 7f0fddffb640 1 -- 172.21.15.26:0/3347290913 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0fe8067910 con 0x7f0fe80c2da0 2024-03-06T07:26:08.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.333+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fe8075300 con 0x7f0fe80c2da0 2024-03-06T07:26:08.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.334+0000 7f0fddffb640 1 -- 172.21.15.26:0/3347290913 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16909 (secure 0 0 0) 0x7f0fe8075300 con 0x7f0fe80c2da0 2024-03-06T07:26:08.337 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:08.337 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":150,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:07.071386+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fcc06b500 msgr2=0x7f0fcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fcc06b500 0x7f0fcc06d9c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007fd0 tx=0x7f0fe0007920 comp rx=0 tx=0).stop 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c2da0 msgr2=0x7f0fe8148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c2da0 0x7f0fe8148140 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007d90 tx=0x7f0fe400d4a0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 shutdown_connections 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0fcc06b500 0x7f0fcc06d9c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fe80c2da0 0x7f0fe8148140 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe8152920 0x7f0fe8147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 --2- 172.21.15.26:0/3347290913 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80c8a00 0x7f0fe814df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 >> 172.21.15.26:0/3347290913 conn(0x7f0fe80ac560 msgr2=0x7f0fe80ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 shutdown_connections 2024-03-06T07:26:08.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.336+0000 7f0fef393640 1 -- 172.21.15.26:0/3347290913 wait complete. 2024-03-06T07:26:08.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:26:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.451+0000 7f2814a6d640 1 Processor -- start 2024-03-06T07:26:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.451+0000 7f2814a6d640 1 -- start start 2024-03-06T07:26:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.452+0000 7f2814a6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f281006d750 0x7f28100732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.452+0000 7f2814a6d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28100c3a70 0x7f2810072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.452+0000 7f2814a6d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2810156060 con 0x7f281006db30 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.452+0000 7f2814a6d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2810155b30 con 0x7f28100c3a70 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.452+0000 7f2814a6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28100a8430 con 0x7f281006d750 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280dd9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28100c3a70 0x7f2810072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280e59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f281006d750 0x7f28100732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280dd9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28100c3a70 0x7f2810072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57732/0 (socket says 172.21.15.26:57732) 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280ed9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f281006db30 0x7f28100c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52198/0 (socket says 172.21.15.26:52198) 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280dd9b640 1 -- 172.21.15.26:0/2573789728 learned_addr learned my addr 172.21.15.26:0/2573789728 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280dd9b640 1 -- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f281006d750 msgr2=0x7f28100732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280dd9b640 1 --2- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f281006d750 0x7f28100732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280dd9b640 1 -- 172.21.15.26:0/2573789728 >> v1:172.21.15.117:6789/0 conn(0x7f281006db30 legacy=0x7f28100c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280d59a640 1 -- 172.21.15.26:0/2573789728 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181789175 0 0) 0x7f2810156060 con 0x7f281006db30 2024-03-06T07:26:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280dd9b640 1 -- 172.21.15.26:0/2573789728 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810155d80 con 0x7f28100c3a70 2024-03-06T07:26:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.453+0000 7f280e59c640 1 --2- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f281006d750 0x7f28100732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.454+0000 7f280dd9b640 1 --2- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28100c3a70 0x7f2810072680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a53569da16af4dd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.454+0000 7f280d59a640 1 -- 172.21.15.26:0/2573789728 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80317c0 con 0x7f28100c3a70 2024-03-06T07:26:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.454+0000 7f280d59a640 1 -- 172.21.15.26:0/2573789728 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8037070 con 0x7f28100c3a70 2024-03-06T07:26:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.454+0000 7f280d59a640 1 -- 172.21.15.26:0/2573789728 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8031b80 con 0x7f28100c3a70 2024-03-06T07:26:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.454+0000 7f2814a6d640 1 -- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28100c3a70 msgr2=0x7f2810072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.454+0000 7f2814a6d640 1 --2- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28100c3a70 0x7f2810072680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802b860 comp rx=0 tx=0).stop 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.455+0000 7f2814a6d640 1 -- 172.21.15.26:0/2573789728 shutdown_connections 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.455+0000 7f2814a6d640 1 --2- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28100c3a70 0x7f2810072680 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.455+0000 7f2814a6d640 1 --2- 172.21.15.26:0/2573789728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f281006d750 0x7f28100732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.455+0000 7f2814a6d640 1 -- 172.21.15.26:0/2573789728 >> 172.21.15.26:0/2573789728 conn(0x7f28100c5560 msgr2=0x7f28100c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.455+0000 7f2814a6d640 1 -- 172.21.15.26:0/2573789728 shutdown_connections 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.455+0000 7f2814a6d640 1 -- 172.21.15.26:0/2573789728 wait complete. 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 Processor -- start 2024-03-06T07:26:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 -- start start 2024-03-06T07:26:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281006d750 0x7f28100ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006db30 0x7f2810152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28100c3a70 0x7f2810153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810072d40 con 0x7f281006db30 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810072bc0 con 0x7f281006d750 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.456+0000 7f2814a6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2810072ec0 con 0x7f28100c3a70 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006db30 0x7f2810152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006db30 0x7f2810152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44704/0 (socket says 172.21.15.26:44704) 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 -- 172.21.15.26:0/442250285 learned_addr learned my addr 172.21.15.26:0/442250285 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280ed9d640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28100c3a70 0x7f2810153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280e59c640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281006d750 0x7f28100ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 -- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28100c3a70 msgr2=0x7f2810153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28100c3a70 0x7f2810153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 -- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281006d750 msgr2=0x7f28100ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281006d750 0x7f28100ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 -- 172.21.15.26:0/442250285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28100c2290 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f280dd9b640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006db30 0x7f2810152f20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f2806ffd640 1 -- 172.21.15.26:0/442250285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8004030 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f2806ffd640 1 -- 172.21.15.26:0/442250285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f80041f0 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28101545c0 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.457+0000 7f2806ffd640 1 -- 172.21.15.26:0/442250285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8003610 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.458+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28101ea800 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.458+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28101ea310 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.459+0000 7f2806ffd640 1 -- 172.21.15.26:0/442250285 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27f80037b0 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.459+0000 7f2806ffd640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f27f006b4d0 0x7f27f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.459+0000 7f2806ffd640 1 -- 172.21.15.26:0/442250285 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27f80abff0 con 0x7f281006db30 2024-03-06T07:26:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.459+0000 7f280e59c640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f27f006b4d0 0x7f27f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.460+0000 7f280e59c640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f27f006b4d0 0x7f27f006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2808009560 tx=0x7f2808009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.463+0000 7f2806ffd640 1 -- 172.21.15.26:0/442250285 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f27f8040070 con 0x7f281006db30 2024-03-06T07:26:08.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:08.684+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f281004f330 con 0x7f281006db30 2024-03-06T07:26:09.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.695+0000 7f2806ffd640 1 -- 172.21.15.26:0/442250285 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v137) v1 ==== 118+0+0 (secure 0 0 0) 0x7f27f807a7b0 con 0x7f281006db30 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f27f006b4d0 msgr2=0x7f27f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f27f006b4d0 0x7f27f006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2808009560 tx=0x7f2808009190 comp rx=0 tx=0).stop 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006db30 msgr2=0x7f2810152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006db30 0x7f2810152f20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f80046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 shutdown_connections 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f27f006b4d0 0x7f27f006d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28100c3a70 0x7f2810153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006db30 0x7f2810152f20 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 --2- 172.21.15.26:0/442250285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281006d750 0x7f28100ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 >> 172.21.15.26:0/442250285 conn(0x7f28100c5560 msgr2=0x7f2810151380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:09.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 shutdown_connections 2024-03-06T07:26:09.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.697+0000 7f2814a6d640 1 -- 172.21.15.26:0/442250285 wait complete. 2024-03-06T07:26:09.710 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:26:09.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:26:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.812+0000 7f4fd5b0c640 1 Processor -- start 2024-03-06T07:26:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.812+0000 7f4fd5b0c640 1 -- start start 2024-03-06T07:26:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.813+0000 7f4fd5b0c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fd00c4c60 0x7f4fd00c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.813+0000 7f4fd5b0c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd00c80b0 con 0x7f4fd0070a50 2024-03-06T07:26:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.813+0000 7f4fd5b0c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd00c8390 con 0x7f4fd0152e40 2024-03-06T07:26:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.813+0000 7f4fd5b0c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd0070ee0 con 0x7f4fd00c4c60 2024-03-06T07:26:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.813+0000 7f4fcf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fd00c4c60 0x7f4fd00c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.813+0000 7f4fceffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4fd0070a50 0x7f4fd0156dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54948/0 (socket says 172.21.15.26:54948) 2024-03-06T07:26:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.813+0000 7f4fceffd640 1 -- 172.21.15.26:0/2128184072 learned_addr learned my addr 172.21.15.26:0/2128184072 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.814+0000 7f4fcf7fe640 1 -- 172.21.15.26:0/2128184072 >> v1:172.21.15.117:6789/0 conn(0x7f4fd0152e40 legacy=0x7f4fd00724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.814+0000 7f4fcf7fe640 1 -- 172.21.15.26:0/2128184072 >> v1:172.21.15.26:6789/0 conn(0x7f4fd0070a50 legacy=0x7f4fd0156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.814+0000 7f4fce7fc640 1 -- 172.21.15.26:0/2128184072 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638580431 0 0) 0x7f4fd00c80b0 con 0x7f4fd0070a50 2024-03-06T07:26:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.814+0000 7f4fcf7fe640 1 -- 172.21.15.26:0/2128184072 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd01565d0 con 0x7f4fd00c4c60 2024-03-06T07:26:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.814+0000 7f4fcf7fe640 1 --2- 172.21.15.26:0/2128184072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd00c7a20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00097a0 tx=0x7f4fc002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e98e8e9176644e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fce7fc640 1 -- 172.21.15.26:0/2128184072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00315a0 con 0x7f4fd00c4c60 2024-03-06T07:26:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fce7fc640 1 -- 172.21.15.26:0/2128184072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc0037070 con 0x7f4fd00c4c60 2024-03-06T07:26:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fce7fc640 1 -- 172.21.15.26:0/2128184072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0031960 con 0x7f4fd00c4c60 2024-03-06T07:26:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/2128184072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 msgr2=0x7f4fd00c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/2128184072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd00c7a20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00097a0 tx=0x7f4fc002b840 comp rx=0 tx=0).stop 2024-03-06T07:26:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/2128184072 shutdown_connections 2024-03-06T07:26:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/2128184072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd00c7a20 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/2128184072 >> 172.21.15.26:0/2128184072 conn(0x7f4fd00aa6a0 msgr2=0x7f4fd00aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/2128184072 shutdown_connections 2024-03-06T07:26:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.815+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/2128184072 wait complete. 2024-03-06T07:26:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.816+0000 7f4fd5b0c640 1 Processor -- start 2024-03-06T07:26:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.816+0000 7f4fd5b0c640 1 -- start start 2024-03-06T07:26:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fd5b0c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd0070a50 0x7f4fd01e9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fd5b0c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd0151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fd5b0c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd0152e40 0x7f4fd0148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fd5b0c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0072be0 con 0x7f4fd00c4c60 2024-03-06T07:26:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fd5b0c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0156940 con 0x7f4fd0152e40 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fd5b0c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0072d60 con 0x7f4fd0070a50 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd0151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd0151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44720/0 (socket says 172.21.15.26:44720) 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fceffd640 1 -- 172.21.15.26:0/315855816 learned_addr learned my addr 172.21.15.26:0/315855816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fcf7fe640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd0070a50 0x7f4fd01e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fcffff640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd0152e40 0x7f4fd0148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.817+0000 7f4fceffd640 1 -- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd0070a50 msgr2=0x7f4fd01e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fceffd640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd0070a50 0x7f4fd01e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fceffd640 1 -- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd0152e40 msgr2=0x7f4fd0148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fceffd640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd0152e40 0x7f4fd0148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fceffd640 1 -- 172.21.15.26:0/315855816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0151220 con 0x7f4fd00c4c60 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fceffd640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd0151f80 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c4d0 tx=0x7f4fb800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fc7fff640 1 -- 172.21.15.26:0/315855816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8018070 con 0x7f4fd00c4c60 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fc7fff640 1 -- 172.21.15.26:0/315855816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb8007500 con 0x7f4fd00c4c60 2024-03-06T07:26:09.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd0151500 con 0x7f4fd00c4c60 2024-03-06T07:26:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fc7fff640 1 -- 172.21.15.26:0/315855816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8004a20 con 0x7f4fd00c4c60 2024-03-06T07:26:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.818+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd01540e0 con 0x7f4fd00c4c60 2024-03-06T07:26:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.819+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd0153ed0 con 0x7f4fd00c4c60 2024-03-06T07:26:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.819+0000 7f4fc7fff640 1 -- 172.21.15.26:0/315855816 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fb801b070 con 0x7f4fd00c4c60 2024-03-06T07:26:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.820+0000 7f4fc7fff640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.820+0000 7f4fc7fff640 1 -- 172.21.15.26:0/315855816 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4fb8089640 con 0x7f4fd00c4c60 2024-03-06T07:26:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.820+0000 7f4fcf7fe640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.821+0000 7f4fcf7fe640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4fb006b4d0 0x7f4fb006d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0036000 tx=0x7f4fc0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:09.824+0000 7f4fc7fff640 1 -- 172.21.15.26:0/315855816 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4fb80583c0 con 0x7f4fd00c4c60 2024-03-06T07:26:10.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.087+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4fd004f330 con 0x7f4fd00c4c60 2024-03-06T07:26:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.697+0000 7f4fc7fff640 1 -- 172.21.15.26:0/315855816 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v138) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fb8057cc0 con 0x7f4fd00c4c60 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4fb006b4d0 msgr2=0x7f4fb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4fb006b4d0 0x7f4fb006d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0036000 tx=0x7f4fc0008030 comp rx=0 tx=0).stop 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 msgr2=0x7f4fd0151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd0151f80 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c4d0 tx=0x7f4fb800c990 comp rx=0 tx=0).stop 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 shutdown_connections 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd0152e40 0x7f4fd0148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00c4c60 0x7f4fd0151f80 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 --2- 172.21.15.26:0/315855816 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd0070a50 0x7f4fd01e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 >> 172.21.15.26:0/315855816 conn(0x7f4fd00aa6a0 msgr2=0x7f4fd014bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:10.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 shutdown_connections 2024-03-06T07:26:10.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.700+0000 7f4fd5b0c640 1 -- 172.21.15.26:0/315855816 wait complete. 2024-03-06T07:26:10.713 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:26:10.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.820+0000 7fd4a5d93640 1 Processor -- start 2024-03-06T07:26:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.820+0000 7fd4a5d93640 1 -- start start 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.820+0000 7fd4a5d93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4a00c88c0 0x7fd4a0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.820+0000 7fd4a5d93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.820+0000 7fd4a5d93640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a00c73d0 con 0x7fd4a014f360 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.820+0000 7fd4a5d93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a00c8dd0 con 0x7fd4a00c88c0 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.820+0000 7fd4a5d93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a00be2e0 con 0x7fd4a0158b60 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd496ffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4a00c88c0 0x7fd4a0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd49ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd49f7fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd4a014f360 0x7fd4a00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52212/0 (socket says 172.21.15.26:52212) 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd49f7fe640 1 -- 172.21.15.26:0/990081654 learned_addr learned my addr 172.21.15.26:0/990081654 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd496ffd640 1 -- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 msgr2=0x7fd4a0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd496ffd640 1 --2- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a0158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd496ffd640 1 -- 172.21.15.26:0/990081654 >> v1:172.21.15.117:6789/0 conn(0x7fd4a014f360 legacy=0x7fd4a00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd49effd640 1 -- 172.21.15.26:0/990081654 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435172172 0 0) 0x7fd4a00c73d0 con 0x7fd4a014f360 2024-03-06T07:26:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd496ffd640 1 -- 172.21.15.26:0/990081654 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a00c7670 con 0x7fd4a00c88c0 2024-03-06T07:26:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.821+0000 7fd49ffff640 1 --2- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.822+0000 7fd496ffd640 1 --2- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a00c88c0 0x7fd4a0155c10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4880097a0 tx=0x7fd48802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702628dbf1f21ee0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.822+0000 7fd49effd640 1 -- 172.21.15.26:0/990081654 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880315a0 con 0x7fd4a00c88c0 2024-03-06T07:26:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.822+0000 7fd49effd640 1 -- 172.21.15.26:0/990081654 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488037070 con 0x7fd4a00c88c0 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.822+0000 7fd49effd640 1 -- 172.21.15.26:0/990081654 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488031960 con 0x7fd4a00c88c0 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 -- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a00c88c0 msgr2=0x7fd4a0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a00c88c0 0x7fd4a0155c10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4880097a0 tx=0x7fd48802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 -- 172.21.15.26:0/990081654 shutdown_connections 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/990081654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a00c88c0 0x7fd4a0155c10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 -- 172.21.15.26:0/990081654 >> 172.21.15.26:0/990081654 conn(0x7fd4a00ac570 msgr2=0x7fd4a00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 -- 172.21.15.26:0/990081654 shutdown_connections 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.823+0000 7fd4a5d93640 1 -- 172.21.15.26:0/990081654 wait complete. 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.824+0000 7fd4a5d93640 1 Processor -- start 2024-03-06T07:26:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.824+0000 7fd4a5d93640 1 -- start start 2024-03-06T07:26:10.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd4a5d93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a00c88c0 0x7fd4a01e8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd4a5d93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a014f360 0x7fd4a014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd4a5d93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd4a5d93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a00c5a50 con 0x7fd4a00c88c0 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd4a5d93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a00c7c30 con 0x7fd4a014f360 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd4a5d93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a00c5bd0 con 0x7fd4a0158b60 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a00c88c0 0x7fd4a01e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a00c88c0 0x7fd4a01e8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44732/0 (socket says 172.21.15.26:44732) 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 -- 172.21.15.26:0/1366721896 learned_addr learned my addr 172.21.15.26:0/1366721896 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49ffff640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd496ffd640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a014f360 0x7fd4a014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 -- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 msgr2=0x7fd4a01565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a01565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 -- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a014f360 msgr2=0x7fd4a014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a014f360 0x7fd4a014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 -- 172.21.15.26:0/1366721896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a01eac80 con 0x7fd4a00c88c0 2024-03-06T07:26:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd496ffd640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a014f360 0x7fd4a014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:10.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49f7fe640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a00c88c0 0x7fd4a01e8020 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c460 tx=0x7fd49000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49cff9640 1 -- 172.21.15.26:0/1366721896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4900074e0 con 0x7fd4a00c88c0 2024-03-06T07:26:10.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49cff9640 1 -- 172.21.15.26:0/1366721896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4900047e0 con 0x7fd4a00c88c0 2024-03-06T07:26:10.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a01eaf60 con 0x7fd4a00c88c0 2024-03-06T07:26:10.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.825+0000 7fd49cff9640 1 -- 172.21.15.26:0/1366721896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490004bd0 con 0x7fd4a00c88c0 2024-03-06T07:26:10.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.826+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a01eb520 con 0x7fd4a00c88c0 2024-03-06T07:26:10.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.827+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a004f330 con 0x7fd4a00c88c0 2024-03-06T07:26:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.827+0000 7fd49cff9640 1 -- 172.21.15.26:0/1366721896 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd490011070 con 0x7fd4a00c88c0 2024-03-06T07:26:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.828+0000 7fd49cff9640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd48006b500 0x7fd48006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.828+0000 7fd49cff9640 1 -- 172.21.15.26:0/1366721896 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd490089f90 con 0x7fd4a00c88c0 2024-03-06T07:26:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.828+0000 7fd496ffd640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd48006b500 0x7fd48006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.829+0000 7fd496ffd640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd48006b500 0x7fd48006d9c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd488036000 tx=0x7fd488008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:10.832+0000 7fd49cff9640 1 -- 172.21.15.26:0/1366721896 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd490058bf0 con 0x7fd4a00c88c0 2024-03-06T07:26:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.050+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4a006c470 con 0x7fd4a00c88c0 2024-03-06T07:26:11.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.051+0000 7fd49cff9640 1 -- 172.21.15.26:0/1366721896 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+4987 (secure 0 0 0) 0x7fd4900584f0 con 0x7fd4a00c88c0 2024-03-06T07:26:11.053 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:11.053 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":138,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":8293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:10.089399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8793":{"gid":8793,"name":"a","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6833/3029467671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3029467671},{"type":"v1","addr":"172.21.15.26:6833","nonce":3029467671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8793,"qdb_cluster":[8793]},"id":11}]} 2024-03-06T07:26:11.054 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 138 2024-03-06T07:26:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd48006b500 msgr2=0x7fd48006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd48006b500 0x7fd48006d9c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd488036000 tx=0x7fd488008030 comp rx=0 tx=0).stop 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a00c88c0 msgr2=0x7fd4a01e8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a00c88c0 0x7fd4a01e8020 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c460 tx=0x7fd49000c860 comp rx=0 tx=0).stop 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 shutdown_connections 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd48006b500 0x7fd48006d9c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a0158b60 0x7fd4a01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a014f360 0x7fd4a014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 --2- 172.21.15.26:0/1366721896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a00c88c0 0x7fd4a01e8020 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 >> 172.21.15.26:0/1366721896 conn(0x7fd4a00ac570 msgr2=0x7fd4a0158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 shutdown_connections 2024-03-06T07:26:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.053+0000 7fd4a5d93640 1 -- 172.21.15.26:0/1366721896 wait complete. 2024-03-06T07:26:11.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.165+0000 7f5de462a640 1 Processor -- start 2024-03-06T07:26:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.165+0000 7f5de462a640 1 -- start start 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de462a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ddc0a7f80 0x7f5ddc1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de462a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc153b20 0x7f5ddc15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de462a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc0c3320 con 0x7f5ddc1505f0 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de462a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc0abaf0 con 0x7f5ddc0a7f80 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de462a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0c7c30 con 0x7f5ddc153b20 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de1bc5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ddc0a7f80 0x7f5ddc1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de23c6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ddc1505f0 0x7f5ddc0c43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54950/0 (socket says 172.21.15.26:54950) 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de23c6640 1 -- 172.21.15.26:0/1935065937 learned_addr learned my addr 172.21.15.26:0/1935065937 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de2bc7640 1 --2- 172.21.15.26:0/1935065937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc153b20 0x7f5ddc15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de1bc5640 1 -- 172.21.15.26:0/1935065937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc153b20 msgr2=0x7f5ddc15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de1bc5640 1 --2- 172.21.15.26:0/1935065937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc153b20 0x7f5ddc15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de1bc5640 1 -- 172.21.15.26:0/1935065937 >> v1:172.21.15.26:6789/0 conn(0x7f5ddc1505f0 legacy=0x7f5ddc0c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de1bc5640 1 -- 172.21.15.26:0/1935065937 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc153f50 con 0x7f5ddc0a7f80 2024-03-06T07:26:11.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de13c4640 1 -- 172.21.15.26:0/1935065937 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411764695 0 0) 0x7f5ddc0c3320 con 0x7f5ddc1505f0 2024-03-06T07:26:11.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.166+0000 7f5de2bc7640 1 --2- 172.21.15.26:0/1935065937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc153b20 0x7f5ddc15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.167+0000 7f5de1bc5640 1 --2- 172.21.15.26:0/1935065937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc0a7f80 0x7f5ddc1535e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0abc70 tx=0x7f5dcc02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332db4b7aeb0db56 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.167+0000 7f5de13c4640 1 -- 172.21.15.26:0/1935065937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc039070 con 0x7f5ddc0a7f80 2024-03-06T07:26:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.167+0000 7f5de13c4640 1 -- 172.21.15.26:0/1935065937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc0316f0 con 0x7f5ddc0a7f80 2024-03-06T07:26:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.167+0000 7f5de13c4640 1 -- 172.21.15.26:0/1935065937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc031ac0 con 0x7f5ddc0a7f80 2024-03-06T07:26:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.167+0000 7f5de462a640 1 -- 172.21.15.26:0/1935065937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc0a7f80 msgr2=0x7f5ddc1535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.167+0000 7f5de462a640 1 --2- 172.21.15.26:0/1935065937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc0a7f80 0x7f5ddc1535e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0abc70 tx=0x7f5dcc02b790 comp rx=0 tx=0).stop 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.168+0000 7f5de462a640 1 -- 172.21.15.26:0/1935065937 shutdown_connections 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.168+0000 7f5de462a640 1 --2- 172.21.15.26:0/1935065937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc0a7f80 0x7f5ddc1535e0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.168+0000 7f5de462a640 1 --2- 172.21.15.26:0/1935065937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc153b20 0x7f5ddc15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.168+0000 7f5de462a640 1 -- 172.21.15.26:0/1935065937 >> 172.21.15.26:0/1935065937 conn(0x7f5ddc0aa5c0 msgr2=0x7f5ddc0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.168+0000 7f5de462a640 1 -- 172.21.15.26:0/1935065937 shutdown_connections 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.168+0000 7f5de462a640 1 -- 172.21.15.26:0/1935065937 wait complete. 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 Processor -- start 2024-03-06T07:26:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 -- start start 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc0a7f80 0x7f5ddc151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc1505f0 0x7f5ddc06ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc153b20 0x7f5ddc0c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0c7e80 con 0x7f5ddc153b20 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc153150 con 0x7f5ddc0a7f80 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de462a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc1509d0 con 0x7f5ddc1505f0 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de2bc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc153b20 0x7f5ddc0c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de2bc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc153b20 0x7f5ddc0c1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44756/0 (socket says 172.21.15.26:44756) 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.169+0000 7f5de2bc7640 1 -- 172.21.15.26:0/2444283752 learned_addr learned my addr 172.21.15.26:0/2444283752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de1bc5640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc1505f0 0x7f5ddc06ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de2bc7640 1 -- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc1505f0 msgr2=0x7f5ddc06ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de23c6640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc0a7f80 0x7f5ddc151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de2bc7640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc1505f0 0x7f5ddc06ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de2bc7640 1 -- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc0a7f80 msgr2=0x7f5ddc151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de2bc7640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc0a7f80 0x7f5ddc151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de2bc7640 1 -- 172.21.15.26:0/2444283752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc152880 con 0x7f5ddc153b20 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de2bc7640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc153b20 0x7f5ddc0c1f10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009fd0 tx=0x7f5dd400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5dd2ffd640 1 -- 172.21.15.26:0/2444283752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd401a070 con 0x7f5ddc153b20 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5dd2ffd640 1 -- 172.21.15.26:0/2444283752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd4003bb0 con 0x7f5ddc153b20 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc06fac0 con 0x7f5ddc153b20 2024-03-06T07:26:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5dd2ffd640 1 -- 172.21.15.26:0/2444283752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004030 con 0x7f5ddc153b20 2024-03-06T07:26:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.170+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc1e99a0 con 0x7f5ddc153b20 2024-03-06T07:26:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.171+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc04f330 con 0x7f5ddc153b20 2024-03-06T07:26:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.172+0000 7f5dd2ffd640 1 -- 172.21.15.26:0/2444283752 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dd401d070 con 0x7f5ddc153b20 2024-03-06T07:26:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.172+0000 7f5dd2ffd640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc006b4d0 0x7f5dc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.173+0000 7f5dd2ffd640 1 -- 172.21.15.26:0/2444283752 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5dd408a5f0 con 0x7f5ddc153b20 2024-03-06T07:26:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.173+0000 7f5de23c6640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc006b4d0 0x7f5dc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.173+0000 7f5de23c6640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc006b4d0 0x7f5dc006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007fd0 tx=0x7f5dd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.177+0000 7f5dd2ffd640 1 -- 172.21.15.26:0/2444283752 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5dd4010070 con 0x7f5ddc153b20 2024-03-06T07:26:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.360+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ddc073480 con 0x7f5ddc153b20 2024-03-06T07:26:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.361+0000 7f5dd2ffd640 1 -- 172.21.15.26:0/2444283752 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16896 (secure 0 0 0) 0x7f5dd4059370 con 0x7f5ddc153b20 2024-03-06T07:26:11.364 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:11.364 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":151,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:08.684485+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.363+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc006b4d0 msgr2=0x7f5dc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.363+0000 7f5de462a640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc006b4d0 0x7f5dc006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007fd0 tx=0x7f5dd8009040 comp rx=0 tx=0).stop 2024-03-06T07:26:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.363+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc153b20 msgr2=0x7f5ddc0c1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.363+0000 7f5de462a640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc153b20 0x7f5ddc0c1f10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009fd0 tx=0x7f5dd400d560 comp rx=0 tx=0).stop 2024-03-06T07:26:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 shutdown_connections 2024-03-06T07:26:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc006b4d0 0x7f5dc006d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ddc153b20 0x7f5ddc0c1f10 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ddc1505f0 0x7f5ddc06ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 --2- 172.21.15.26:0/2444283752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc0a7f80 0x7f5ddc151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 >> 172.21.15.26:0/2444283752 conn(0x7f5ddc0aa5c0 msgr2=0x7f5ddc0ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 shutdown_connections 2024-03-06T07:26:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.364+0000 7f5de462a640 1 -- 172.21.15.26:0/2444283752 wait complete. 2024-03-06T07:26:11.377 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:26:11.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.476+0000 7f7a866b5640 1 Processor -- start 2024-03-06T07:26:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.476+0000 7f7a866b5640 1 -- start start 2024-03-06T07:26:11.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a866b5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7a801506b0 0x7f7a80153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a866b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a80156ea0 0x7f7a800702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a866b5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a80070800 con 0x7f7a800acc00 2024-03-06T07:26:11.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a866b5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a80156450 con 0x7f7a801506b0 2024-03-06T07:26:11.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a866b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80156a10 con 0x7f7a80156ea0 2024-03-06T07:26:11.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7a801506b0 0x7f7a80153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7a801506b0 0x7f7a80153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57794/0 (socket says 172.21.15.26:57794) 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a80156ea0 0x7f7a800702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7ffff640 1 -- 172.21.15.26:0/3691463569 learned_addr learned my addr 172.21.15.26:0/3691463569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a84c52640 1 --1- 172.21.15.26:0/3691463569 >> v1:172.21.15.117:6789/0 conn(0x7f7a800acc00 0x7f7a800c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52220/0 (socket says 172.21.15.26:52220) 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7ffff640 1 -- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a80156ea0 msgr2=0x7f7a800702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7ffff640 1 --2- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a80156ea0 0x7f7a800702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7ffff640 1 -- 172.21.15.26:0/3691463569 >> v1:172.21.15.117:6789/0 conn(0x7f7a800acc00 legacy=0x7f7a800c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7ffff640 1 -- 172.21.15.26:0/3691463569 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80070aa0 con 0x7f7a801506b0 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7effd640 1 -- 172.21.15.26:0/3691463569 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927694928 0 0) 0x7f7a80070800 con 0x7f7a800acc00 2024-03-06T07:26:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.477+0000 7f7a7f7fe640 1 --2- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a80156ea0 0x7f7a800702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:11.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a7ffff640 1 --2- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a801506b0 0x7f7a80153990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7a801565d0 tx=0x7f7a7802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b321e28e228436 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a7effd640 1 -- 172.21.15.26:0/3691463569 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78034070 con 0x7f7a801506b0 2024-03-06T07:26:11.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a7effd640 1 -- 172.21.15.26:0/3691463569 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78030840 con 0x7f7a801506b0 2024-03-06T07:26:11.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a7effd640 1 -- 172.21.15.26:0/3691463569 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78030c10 con 0x7f7a801506b0 2024-03-06T07:26:11.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 -- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a801506b0 msgr2=0x7f7a80153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 --2- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a801506b0 0x7f7a80153990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7a801565d0 tx=0x7f7a7802b980 comp rx=0 tx=0).stop 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 -- 172.21.15.26:0/3691463569 shutdown_connections 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 --2- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a80156ea0 0x7f7a800702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 --2- 172.21.15.26:0/3691463569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a801506b0 0x7f7a80153990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 -- 172.21.15.26:0/3691463569 >> 172.21.15.26:0/3691463569 conn(0x7f7a8006b1a0 msgr2=0x7f7a8006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 -- 172.21.15.26:0/3691463569 shutdown_connections 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.479+0000 7f7a866b5640 1 -- 172.21.15.26:0/3691463569 wait complete. 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.480+0000 7f7a866b5640 1 Processor -- start 2024-03-06T07:26:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.481+0000 7f7a866b5640 1 -- start start 2024-03-06T07:26:11.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.481+0000 7f7a866b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a800acc00 0x7f7a801ea990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.481+0000 7f7a866b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a801506b0 0x7f7a800c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.481+0000 7f7a866b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a80156ea0 0x7f7a801ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.481+0000 7f7a866b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800c5bb0 con 0x7f7a80156ea0 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.481+0000 7f7a866b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801566a0 con 0x7f7a800acc00 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.481+0000 7f7a866b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800c5d30 con 0x7f7a801506b0 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a80156ea0 0x7f7a801ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a80156ea0 0x7f7a801ee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44768/0 (socket says 172.21.15.26:44768) 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 -- 172.21.15.26:0/2288132668 learned_addr learned my addr 172.21.15.26:0/2288132668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a7ffff640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a800acc00 0x7f7a801ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a7f7fe640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a801506b0 0x7f7a800c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 -- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a801506b0 msgr2=0x7f7a800c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a801506b0 0x7f7a800c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 -- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a800acc00 msgr2=0x7f7a801ea990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a800acc00 0x7f7a801ea990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 -- 172.21.15.26:0/2288132668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800c7a10 con 0x7f7a80156ea0 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a84c52640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a80156ea0 0x7f7a801ee700 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000c800 tx=0x7f7a7000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a7cff9640 1 -- 172.21.15.26:0/2288132668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70018070 con 0x7f7a80156ea0 2024-03-06T07:26:11.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a7cff9640 1 -- 172.21.15.26:0/2288132668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a70004d80 con 0x7f7a80156ea0 2024-03-06T07:26:11.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a800c7cf0 con 0x7f7a80156ea0 2024-03-06T07:26:11.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a7cff9640 1 -- 172.21.15.26:0/2288132668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000a880 con 0x7f7a80156ea0 2024-03-06T07:26:11.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.482+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a801edf00 con 0x7f7a80156ea0 2024-03-06T07:26:11.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.483+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8004f330 con 0x7f7a80156ea0 2024-03-06T07:26:11.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.484+0000 7f7a7cff9640 1 -- 172.21.15.26:0/2288132668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a7001b070 con 0x7f7a80156ea0 2024-03-06T07:26:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.484+0000 7f7a7cff9640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7a6006b4d0 0x7f7a6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.484+0000 7f7a7cff9640 1 -- 172.21.15.26:0/2288132668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7a70089890 con 0x7f7a80156ea0 2024-03-06T07:26:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.485+0000 7f7a7ffff640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7a6006b4d0 0x7f7a6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.485+0000 7f7a7ffff640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7a6006b4d0 0x7f7a6006d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7a78036000 tx=0x7f7a78008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.489+0000 7f7a7cff9640 1 -- 172.21.15.26:0/2288132668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7a700586d0 con 0x7f7a80156ea0 2024-03-06T07:26:11.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.665+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a800700a0 con 0x7f7a6006b4d0 2024-03-06T07:26:11.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.672+0000 7f7a7cff9640 1 -- 172.21.15.26:0/2288132668 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252525 (secure 0 0 0) 0x7f7a800700a0 con 0x7f7a6006b4d0 2024-03-06T07:26:11.677 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:11.679 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7a6006b4d0 msgr2=0x7f7a6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7a6006b4d0 0x7f7a6006d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7a78036000 tx=0x7f7a78008030 comp rx=0 tx=0).stop 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a80156ea0 msgr2=0x7f7a801ee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a80156ea0 0x7f7a801ee700 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000c800 tx=0x7f7a7000ccc0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 shutdown_connections 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7a6006b4d0 0x7f7a6006d990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a80156ea0 0x7f7a801ee700 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a801506b0 0x7f7a800c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.674+0000 7f7a866b5640 1 --2- 172.21.15.26:0/2288132668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a800acc00 0x7f7a801ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.675+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 >> 172.21.15.26:0/2288132668 conn(0x7f7a8006b1a0 msgr2=0x7f7a800724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.675+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 shutdown_connections 2024-03-06T07:26:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.675+0000 7f7a866b5640 1 -- 172.21.15.26:0/2288132668 wait complete. 2024-03-06T07:26:11.688 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-06T07:26:11.106806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264464,"kb_used_data":14992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710256,"statfs":{"total":773094113280,"available":772823302144,"internally_reserved":0,"allocated":15351808,"data_stored":8508716,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.434280"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699396+0000","last_change":"2024-03-06T07:26:07.695841+0000","last_active":"2024-03-06T07:26:07.699396+0000","last_peered":"2024-03-06T07:26:07.699396+0000","last_clean":"2024-03-06T07:26:07.699396+0000","last_became_active":"2024-03-06T07:26:07.695553+0000","last_became_peered":"2024-03-06T07:26:07.695553+0000","last_unstale":"2024-03-06T07:26:07.699396+0000","last_undegraded":"2024-03-06T07:26:07.699396+0000","last_fullsized":"2024-03-06T07:26:07.699396+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:32.222328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699592+0000","last_change":"2024-03-06T07:26:07.698101+0000","last_active":"2024-03-06T07:26:07.699592+0000","last_peered":"2024-03-06T07:26:07.699592+0000","last_clean":"2024-03-06T07:26:07.699592+0000","last_became_active":"2024-03-06T07:26:07.697796+0000","last_became_peered":"2024-03-06T07:26:07.697796+0000","last_unstale":"2024-03-06T07:26:07.699592+0000","last_undegraded":"2024-03-06T07:26:07.699592+0000","last_fullsized":"2024-03-06T07:26:07.699592+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697427+0000","last_change":"2024-03-06T07:26:07.691124+0000","last_active":"2024-03-06T07:26:07.697427+0000","last_peered":"2024-03-06T07:26:07.697427+0000","last_clean":"2024-03-06T07:26:07.697427+0000","last_became_active":"2024-03-06T07:26:07.690832+0000","last_became_peered":"2024-03-06T07:26:07.690832+0000","last_unstale":"2024-03-06T07:26:07.697427+0000","last_undegraded":"2024-03-06T07:26:07.697427+0000","last_fullsized":"2024-03-06T07:26:07.697427+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:31:41.658258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.695720+0000","last_change":"2024-03-06T07:26:07.688926+0000","last_active":"2024-03-06T07:26:07.695720+0000","last_peered":"2024-03-06T07:26:07.695720+0000","last_clean":"2024-03-06T07:26:07.695720+0000","last_became_active":"2024-03-06T07:26:07.688684+0000","last_became_peered":"2024-03-06T07:26:07.688684+0000","last_unstale":"2024-03-06T07:26:07.695720+0000","last_undegraded":"2024-03-06T07:26:07.695720+0000","last_fullsized":"2024-03-06T07:26:07.695720+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:56.031022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839607+0000","last_change":"2024-03-06T07:26:07.708495+0000","last_active":"2024-03-06T07:26:08.839607+0000","last_peered":"2024-03-06T07:26:08.839607+0000","last_clean":"2024-03-06T07:26:08.839607+0000","last_became_active":"2024-03-06T07:26:07.708183+0000","last_became_peered":"2024-03-06T07:26:07.708183+0000","last_unstale":"2024-03-06T07:26:08.839607+0000","last_undegraded":"2024-03-06T07:26:08.839607+0000","last_fullsized":"2024-03-06T07:26:08.839607+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:00:41.851158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697761+0000","last_change":"2024-03-06T07:26:07.696015+0000","last_active":"2024-03-06T07:26:07.697761+0000","last_peered":"2024-03-06T07:26:07.697761+0000","last_clean":"2024-03-06T07:26:07.697761+0000","last_became_active":"2024-03-06T07:26:07.695708+0000","last_became_peered":"2024-03-06T07:26:07.695708+0000","last_unstale":"2024-03-06T07:26:07.697761+0000","last_undegraded":"2024-03-06T07:26:07.697761+0000","last_fullsized":"2024-03-06T07:26:07.697761+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:02.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687706+0000","last_change":"2024-03-06T07:26:07.699338+0000","last_active":"2024-03-06T07:26:08.687706+0000","last_peered":"2024-03-06T07:26:08.687706+0000","last_clean":"2024-03-06T07:26:08.687706+0000","last_became_active":"2024-03-06T07:26:07.699065+0000","last_became_peered":"2024-03-06T07:26:07.699065+0000","last_unstale":"2024-03-06T07:26:08.687706+0000","last_undegraded":"2024-03-06T07:26:08.687706+0000","last_fullsized":"2024-03-06T07:26:08.687706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687697+0000","last_change":"2024-03-06T07:26:07.700571+0000","last_active":"2024-03-06T07:26:08.687697+0000","last_peered":"2024-03-06T07:26:08.687697+0000","last_clean":"2024-03-06T07:26:08.687697+0000","last_became_active":"2024-03-06T07:26:07.698195+0000","last_became_peered":"2024-03-06T07:26:07.698195+0000","last_unstale":"2024-03-06T07:26:08.687697+0000","last_undegraded":"2024-03-06T07:26:08.687697+0000","last_fullsized":"2024-03-06T07:26:08.687697+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688048+0000","last_change":"2024-03-06T07:26:07.700691+0000","last_active":"2024-03-06T07:26:08.688048+0000","last_peered":"2024-03-06T07:26:08.688048+0000","last_clean":"2024-03-06T07:26:08.688048+0000","last_became_active":"2024-03-06T07:26:07.700158+0000","last_became_peered":"2024-03-06T07:26:07.700158+0000","last_unstale":"2024-03-06T07:26:08.688048+0000","last_undegraded":"2024-03-06T07:26:08.688048+0000","last_fullsized":"2024-03-06T07:26:08.688048+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:17.093387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688084+0000","last_change":"2024-03-06T07:26:07.700442+0000","last_active":"2024-03-06T07:26:08.688084+0000","last_peered":"2024-03-06T07:26:08.688084+0000","last_clean":"2024-03-06T07:26:08.688084+0000","last_became_active":"2024-03-06T07:26:07.700116+0000","last_became_peered":"2024-03-06T07:26:07.700116+0000","last_unstale":"2024-03-06T07:26:08.688084+0000","last_undegraded":"2024-03-06T07:26:08.688084+0000","last_fullsized":"2024-03-06T07:26:08.688084+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:30.446061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691434+0000","last_change":"2024-03-06T07:26:07.707356+0000","last_active":"2024-03-06T07:26:08.691434+0000","last_peered":"2024-03-06T07:26:08.691434+0000","last_clean":"2024-03-06T07:26:08.691434+0000","last_became_active":"2024-03-06T07:26:07.707082+0000","last_became_peered":"2024-03-06T07:26:07.707082+0000","last_unstale":"2024-03-06T07:26:08.691434+0000","last_undegraded":"2024-03-06T07:26:08.691434+0000","last_fullsized":"2024-03-06T07:26:08.691434+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697402+0000","last_change":"2024-03-06T07:26:07.694337+0000","last_active":"2024-03-06T07:26:07.697402+0000","last_peered":"2024-03-06T07:26:07.697402+0000","last_clean":"2024-03-06T07:26:07.697402+0000","last_became_active":"2024-03-06T07:26:07.694050+0000","last_became_peered":"2024-03-06T07:26:07.694050+0000","last_unstale":"2024-03-06T07:26:07.697402+0000","last_undegraded":"2024-03-06T07:26:07.697402+0000","last_fullsized":"2024-03-06T07:26:07.697402+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:24:50.068113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700250+0000","last_change":"2024-03-06T07:26:07.696706+0000","last_active":"2024-03-06T07:26:08.700250+0000","last_peered":"2024-03-06T07:26:08.700250+0000","last_clean":"2024-03-06T07:26:08.700250+0000","last_became_active":"2024-03-06T07:26:07.696438+0000","last_became_peered":"2024-03-06T07:26:07.696438+0000","last_unstale":"2024-03-06T07:26:08.700250+0000","last_undegraded":"2024-03-06T07:26:08.700250+0000","last_fullsized":"2024-03-06T07:26:08.700250+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:32:23.418950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839382+0000","last_change":"2024-03-06T07:26:07.704226+0000","last_active":"2024-03-06T07:26:08.839382+0000","last_peered":"2024-03-06T07:26:08.839382+0000","last_clean":"2024-03-06T07:26:08.839382+0000","last_became_active":"2024-03-06T07:26:07.703839+0000","last_became_peered":"2024-03-06T07:26:07.703839+0000","last_unstale":"2024-03-06T07:26:08.839382+0000","last_undegraded":"2024-03-06T07:26:08.839382+0000","last_fullsized":"2024-03-06T07:26:08.839382+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:52.356683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691271+0000","last_change":"2024-03-06T07:26:07.710172+0000","last_active":"2024-03-06T07:26:08.691271+0000","last_peered":"2024-03-06T07:26:08.691271+0000","last_clean":"2024-03-06T07:26:08.691271+0000","last_became_active":"2024-03-06T07:26:07.709871+0000","last_became_peered":"2024-03-06T07:26:07.709871+0000","last_unstale":"2024-03-06T07:26:08.691271+0000","last_undegraded":"2024-03-06T07:26:08.691271+0000","last_fullsized":"2024-03-06T07:26:08.691271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:17:00.303994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.695384+0000","last_change":"2024-03-06T07:26:07.688786+0000","last_active":"2024-03-06T07:26:07.695384+0000","last_peered":"2024-03-06T07:26:07.695384+0000","last_clean":"2024-03-06T07:26:07.695384+0000","last_became_active":"2024-03-06T07:26:07.688085+0000","last_became_peered":"2024-03-06T07:26:07.688085+0000","last_unstale":"2024-03-06T07:26:07.695384+0000","last_undegraded":"2024-03-06T07:26:07.695384+0000","last_fullsized":"2024-03-06T07:26:07.695384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:16:30.735444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688066+0000","last_change":"2024-03-06T07:26:07.711076+0000","last_active":"2024-03-06T07:26:08.688066+0000","last_peered":"2024-03-06T07:26:08.688066+0000","last_clean":"2024-03-06T07:26:08.688066+0000","last_became_active":"2024-03-06T07:26:07.710796+0000","last_became_peered":"2024-03-06T07:26:07.710796+0000","last_unstale":"2024-03-06T07:26:08.688066+0000","last_undegraded":"2024-03-06T07:26:08.688066+0000","last_fullsized":"2024-03-06T07:26:08.688066+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:56.000904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.704744+0000","last_change":"2024-03-06T07:26:07.703707+0000","last_active":"2024-03-06T07:26:07.704744+0000","last_peered":"2024-03-06T07:26:07.704744+0000","last_clean":"2024-03-06T07:26:07.704744+0000","last_became_active":"2024-03-06T07:26:07.703402+0000","last_became_peered":"2024-03-06T07:26:07.703402+0000","last_unstale":"2024-03-06T07:26:07.704744+0000","last_undegraded":"2024-03-06T07:26:07.704744+0000","last_fullsized":"2024-03-06T07:26:07.704744+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:00.573637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691558+0000","last_change":"2024-03-06T07:26:07.710174+0000","last_active":"2024-03-06T07:26:08.691558+0000","last_peered":"2024-03-06T07:26:08.691558+0000","last_clean":"2024-03-06T07:26:08.691558+0000","last_became_active":"2024-03-06T07:26:07.709939+0000","last_became_peered":"2024-03-06T07:26:07.709939+0000","last_unstale":"2024-03-06T07:26:08.691558+0000","last_undegraded":"2024-03-06T07:26:08.691558+0000","last_fullsized":"2024-03-06T07:26:08.691558+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706147+0000","last_change":"2024-03-06T07:26:07.704730+0000","last_active":"2024-03-06T07:26:07.706147+0000","last_peered":"2024-03-06T07:26:07.706147+0000","last_clean":"2024-03-06T07:26:07.706147+0000","last_became_active":"2024-03-06T07:26:07.704429+0000","last_became_peered":"2024-03-06T07:26:07.704429+0000","last_unstale":"2024-03-06T07:26:07.706147+0000","last_undegraded":"2024-03-06T07:26:07.706147+0000","last_fullsized":"2024-03-06T07:26:07.706147+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:19.206370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700300+0000","last_change":"2024-03-06T07:26:07.685590+0000","last_active":"2024-03-06T07:26:08.700300+0000","last_peered":"2024-03-06T07:26:08.700300+0000","last_clean":"2024-03-06T07:26:08.700300+0000","last_became_active":"2024-03-06T07:26:07.685265+0000","last_became_peered":"2024-03-06T07:26:07.685265+0000","last_unstale":"2024-03-06T07:26:08.700300+0000","last_undegraded":"2024-03-06T07:26:08.700300+0000","last_fullsized":"2024-03-06T07:26:08.700300+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:30.427263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.701827+0000","last_change":"2024-03-06T07:26:07.699358+0000","last_active":"2024-03-06T07:26:07.701827+0000","last_peered":"2024-03-06T07:26:07.701827+0000","last_clean":"2024-03-06T07:26:07.701827+0000","last_became_active":"2024-03-06T07:26:07.699058+0000","last_became_peered":"2024-03-06T07:26:07.699058+0000","last_unstale":"2024-03-06T07:26:07.701827+0000","last_undegraded":"2024-03-06T07:26:07.701827+0000","last_fullsized":"2024-03-06T07:26:07.701827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839269+0000","last_change":"2024-03-06T07:26:07.700848+0000","last_active":"2024-03-06T07:26:08.839269+0000","last_peered":"2024-03-06T07:26:08.839269+0000","last_clean":"2024-03-06T07:26:08.839269+0000","last_became_active":"2024-03-06T07:26:07.700551+0000","last_became_peered":"2024-03-06T07:26:07.700551+0000","last_unstale":"2024-03-06T07:26:08.839269+0000","last_undegraded":"2024-03-06T07:26:08.839269+0000","last_fullsized":"2024-03-06T07:26:08.839269+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.178917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.693524+0000","last_change":"2024-03-06T07:26:07.687552+0000","last_active":"2024-03-06T07:26:07.693524+0000","last_peered":"2024-03-06T07:26:07.693524+0000","last_clean":"2024-03-06T07:26:07.693524+0000","last_became_active":"2024-03-06T07:26:07.687276+0000","last_became_peered":"2024-03-06T07:26:07.687276+0000","last_unstale":"2024-03-06T07:26:07.693524+0000","last_undegraded":"2024-03-06T07:26:07.693524+0000","last_fullsized":"2024-03-06T07:26:07.693524+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:01.894148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700594+0000","last_change":"2024-03-06T07:26:07.698089+0000","last_active":"2024-03-06T07:26:08.700594+0000","last_peered":"2024-03-06T07:26:08.700594+0000","last_clean":"2024-03-06T07:26:08.700594+0000","last_became_active":"2024-03-06T07:26:07.697806+0000","last_became_peered":"2024-03-06T07:26:07.697806+0000","last_unstale":"2024-03-06T07:26:08.700594+0000","last_undegraded":"2024-03-06T07:26:08.700594+0000","last_fullsized":"2024-03-06T07:26:08.700594+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:52.239620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705132+0000","last_change":"2024-03-06T07:26:07.699922+0000","last_active":"2024-03-06T07:26:07.705132+0000","last_peered":"2024-03-06T07:26:07.705132+0000","last_clean":"2024-03-06T07:26:07.705132+0000","last_became_active":"2024-03-06T07:26:07.699631+0000","last_became_peered":"2024-03-06T07:26:07.699631+0000","last_unstale":"2024-03-06T07:26:07.705132+0000","last_undegraded":"2024-03-06T07:26:07.705132+0000","last_fullsized":"2024-03-06T07:26:07.705132+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:30:29.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.709238+0000","last_change":"2024-03-06T07:26:07.706359+0000","last_active":"2024-03-06T07:26:07.709238+0000","last_peered":"2024-03-06T07:26:07.709238+0000","last_clean":"2024-03-06T07:26:07.709238+0000","last_became_active":"2024-03-06T07:26:07.706096+0000","last_became_peered":"2024-03-06T07:26:07.706096+0000","last_unstale":"2024-03-06T07:26:07.709238+0000","last_undegraded":"2024-03-06T07:26:07.709238+0000","last_fullsized":"2024-03-06T07:26:07.709238+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:17.093387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700282+0000","last_change":"2024-03-06T07:26:07.693096+0000","last_active":"2024-03-06T07:26:08.700282+0000","last_peered":"2024-03-06T07:26:08.700282+0000","last_clean":"2024-03-06T07:26:08.700282+0000","last_became_active":"2024-03-06T07:26:07.692834+0000","last_became_peered":"2024-03-06T07:26:07.692834+0000","last_unstale":"2024-03-06T07:26:08.700282+0000","last_undegraded":"2024-03-06T07:26:08.700282+0000","last_fullsized":"2024-03-06T07:26:08.700282+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:58.002384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.709208+0000","last_change":"2024-03-06T07:26:07.706042+0000","last_active":"2024-03-06T07:26:07.709208+0000","last_peered":"2024-03-06T07:26:07.709208+0000","last_clean":"2024-03-06T07:26:07.709208+0000","last_became_active":"2024-03-06T07:26:07.705455+0000","last_became_peered":"2024-03-06T07:26:07.705455+0000","last_unstale":"2024-03-06T07:26:07.709208+0000","last_undegraded":"2024-03-06T07:26:07.709208+0000","last_fullsized":"2024-03-06T07:26:07.709208+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:29.340249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.708074+0000","last_change":"2024-03-06T07:26:07.706790+0000","last_active":"2024-03-06T07:26:07.708074+0000","last_peered":"2024-03-06T07:26:07.708074+0000","last_clean":"2024-03-06T07:26:07.708074+0000","last_became_active":"2024-03-06T07:26:07.706476+0000","last_became_peered":"2024-03-06T07:26:07.706476+0000","last_unstale":"2024-03-06T07:26:07.708074+0000","last_undegraded":"2024-03-06T07:26:07.708074+0000","last_fullsized":"2024-03-06T07:26:07.708074+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:01.894148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691377+0000","last_change":"2024-03-06T07:26:07.698630+0000","last_active":"2024-03-06T07:26:08.691377+0000","last_peered":"2024-03-06T07:26:08.691377+0000","last_clean":"2024-03-06T07:26:08.691377+0000","last_became_active":"2024-03-06T07:26:07.698334+0000","last_became_peered":"2024-03-06T07:26:07.698334+0000","last_unstale":"2024-03-06T07:26:08.691377+0000","last_undegraded":"2024-03-06T07:26:08.691377+0000","last_fullsized":"2024-03-06T07:26:08.691377+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:50:39.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.704391+0000","last_change":"2024-03-06T07:26:07.699219+0000","last_active":"2024-03-06T07:26:07.704391+0000","last_peered":"2024-03-06T07:26:07.704391+0000","last_clean":"2024-03-06T07:26:07.704391+0000","last_became_active":"2024-03-06T07:26:07.698897+0000","last_became_peered":"2024-03-06T07:26:07.698897+0000","last_unstale":"2024-03-06T07:26:07.704391+0000","last_undegraded":"2024-03-06T07:26:07.704391+0000","last_fullsized":"2024-03-06T07:26:07.704391+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:18.201061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699627+0000","last_change":"2024-03-06T07:26:07.697611+0000","last_active":"2024-03-06T07:26:07.699627+0000","last_peered":"2024-03-06T07:26:07.699627+0000","last_clean":"2024-03-06T07:26:07.699627+0000","last_became_active":"2024-03-06T07:26:07.697343+0000","last_became_peered":"2024-03-06T07:26:07.697343+0000","last_unstale":"2024-03-06T07:26:07.699627+0000","last_undegraded":"2024-03-06T07:26:07.699627+0000","last_fullsized":"2024-03-06T07:26:07.699627+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:17:00.303994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"150'4","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706677+0000","last_change":"2024-03-06T07:26:06.698500+0000","last_active":"2024-03-06T07:26:07.706677+0000","last_peered":"2024-03-06T07:26:07.706677+0000","last_clean":"2024-03-06T07:26:07.706677+0000","last_became_active":"2024-03-06T07:26:06.698299+0000","last_became_peered":"2024-03-06T07:26:06.698299+0000","last_unstale":"2024-03-06T07:26:07.706677+0000","last_undegraded":"2024-03-06T07:26:07.706677+0000","last_fullsized":"2024-03-06T07:26:07.706677+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:17.195623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700140+0000","last_change":"2024-03-06T07:26:07.698545+0000","last_active":"2024-03-06T07:26:08.700140+0000","last_peered":"2024-03-06T07:26:08.700140+0000","last_clean":"2024-03-06T07:26:08.700140+0000","last_became_active":"2024-03-06T07:26:07.698270+0000","last_became_peered":"2024-03-06T07:26:07.698270+0000","last_unstale":"2024-03-06T07:26:08.700140+0000","last_undegraded":"2024-03-06T07:26:08.700140+0000","last_fullsized":"2024-03-06T07:26:08.700140+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:59:25.247264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839549+0000","last_change":"2024-03-06T07:26:06.702569+0000","last_active":"2024-03-06T07:26:08.839549+0000","last_peered":"2024-03-06T07:26:08.839549+0000","last_clean":"2024-03-06T07:26:08.839549+0000","last_became_active":"2024-03-06T07:26:06.702304+0000","last_became_peered":"2024-03-06T07:26:06.702304+0000","last_unstale":"2024-03-06T07:26:08.839549+0000","last_undegraded":"2024-03-06T07:26:08.839549+0000","last_fullsized":"2024-03-06T07:26:08.839549+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:16.087949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839398+0000","last_change":"2024-03-06T07:26:07.701966+0000","last_active":"2024-03-06T07:26:08.839398+0000","last_peered":"2024-03-06T07:26:08.839398+0000","last_clean":"2024-03-06T07:26:08.839398+0000","last_became_active":"2024-03-06T07:26:07.701671+0000","last_became_peered":"2024-03-06T07:26:07.701671+0000","last_unstale":"2024-03-06T07:26:08.839398+0000","last_undegraded":"2024-03-06T07:26:08.839398+0000","last_fullsized":"2024-03-06T07:26:08.839398+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:03.437751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691667+0000","last_change":"2024-03-06T07:26:06.691461+0000","last_active":"2024-03-06T07:26:08.691667+0000","last_peered":"2024-03-06T07:26:08.691667+0000","last_clean":"2024-03-06T07:26:08.691667+0000","last_became_active":"2024-03-06T07:26:06.691169+0000","last_became_peered":"2024-03-06T07:26:06.691169+0000","last_unstale":"2024-03-06T07:26:08.691667+0000","last_undegraded":"2024-03-06T07:26:08.691667+0000","last_fullsized":"2024-03-06T07:26:08.691667+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.708821+0000","last_change":"2024-03-06T07:26:07.705527+0000","last_active":"2024-03-06T07:26:07.708821+0000","last_peered":"2024-03-06T07:26:07.708821+0000","last_clean":"2024-03-06T07:26:07.708821+0000","last_became_active":"2024-03-06T07:26:07.705214+0000","last_became_peered":"2024-03-06T07:26:07.705214+0000","last_unstale":"2024-03-06T07:26:07.708821+0000","last_undegraded":"2024-03-06T07:26:07.708821+0000","last_fullsized":"2024-03-06T07:26:07.708821+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705686+0000","last_change":"2024-03-06T07:26:06.689558+0000","last_active":"2024-03-06T07:26:07.705686+0000","last_peered":"2024-03-06T07:26:07.705686+0000","last_clean":"2024-03-06T07:26:07.705686+0000","last_became_active":"2024-03-06T07:26:06.689291+0000","last_became_peered":"2024-03-06T07:26:06.689291+0000","last_unstale":"2024-03-06T07:26:07.705686+0000","last_undegraded":"2024-03-06T07:26:07.705686+0000","last_fullsized":"2024-03-06T07:26:07.705686+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:59.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687862+0000","last_change":"2024-03-06T07:26:07.697671+0000","last_active":"2024-03-06T07:26:08.687862+0000","last_peered":"2024-03-06T07:26:08.687862+0000","last_clean":"2024-03-06T07:26:08.687862+0000","last_became_active":"2024-03-06T07:26:07.697366+0000","last_became_peered":"2024-03-06T07:26:07.697366+0000","last_unstale":"2024-03-06T07:26:08.687862+0000","last_undegraded":"2024-03-06T07:26:08.687862+0000","last_fullsized":"2024-03-06T07:26:08.687862+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687861+0000","last_change":"2024-03-06T07:26:06.700139+0000","last_active":"2024-03-06T07:26:08.687861+0000","last_peered":"2024-03-06T07:26:08.687861+0000","last_clean":"2024-03-06T07:26:08.687861+0000","last_became_active":"2024-03-06T07:26:06.699876+0000","last_became_peered":"2024-03-06T07:26:06.699876+0000","last_unstale":"2024-03-06T07:26:08.687861+0000","last_undegraded":"2024-03-06T07:26:08.687861+0000","last_fullsized":"2024-03-06T07:26:08.687861+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:22.510699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706891+0000","last_change":"2024-03-06T07:26:07.705724+0000","last_active":"2024-03-06T07:26:07.706891+0000","last_peered":"2024-03-06T07:26:07.706891+0000","last_clean":"2024-03-06T07:26:07.706891+0000","last_became_active":"2024-03-06T07:26:07.705378+0000","last_became_peered":"2024-03-06T07:26:07.705378+0000","last_unstale":"2024-03-06T07:26:07.706891+0000","last_undegraded":"2024-03-06T07:26:07.706891+0000","last_fullsized":"2024-03-06T07:26:07.706891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:08.158891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839489+0000","last_change":"2024-03-06T07:26:06.702328+0000","last_active":"2024-03-06T07:26:08.839489+0000","last_peered":"2024-03-06T07:26:08.839489+0000","last_clean":"2024-03-06T07:26:08.839489+0000","last_became_active":"2024-03-06T07:26:06.702009+0000","last_became_peered":"2024-03-06T07:26:06.702009+0000","last_unstale":"2024-03-06T07:26:08.839489+0000","last_undegraded":"2024-03-06T07:26:08.839489+0000","last_fullsized":"2024-03-06T07:26:08.839489+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:14.494596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705111+0000","last_change":"2024-03-06T07:26:07.698151+0000","last_active":"2024-03-06T07:26:07.705111+0000","last_peered":"2024-03-06T07:26:07.705111+0000","last_clean":"2024-03-06T07:26:07.705111+0000","last_became_active":"2024-03-06T07:26:07.697876+0000","last_became_peered":"2024-03-06T07:26:07.697876+0000","last_unstale":"2024-03-06T07:26:07.705111+0000","last_undegraded":"2024-03-06T07:26:07.705111+0000","last_fullsized":"2024-03-06T07:26:07.705111+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:35:55.125635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.687404+0000","last_change":"2024-03-06T07:26:06.700042+0000","last_active":"2024-03-06T07:26:07.687404+0000","last_peered":"2024-03-06T07:26:07.687404+0000","last_clean":"2024-03-06T07:26:07.687404+0000","last_became_active":"2024-03-06T07:26:06.699781+0000","last_became_peered":"2024-03-06T07:26:06.699781+0000","last_unstale":"2024-03-06T07:26:07.687404+0000","last_undegraded":"2024-03-06T07:26:07.687404+0000","last_fullsized":"2024-03-06T07:26:07.687404+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:13.960358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699416+0000","last_change":"2024-03-06T07:26:07.696784+0000","last_active":"2024-03-06T07:26:07.699416+0000","last_peered":"2024-03-06T07:26:07.699416+0000","last_clean":"2024-03-06T07:26:07.699416+0000","last_became_active":"2024-03-06T07:26:07.696515+0000","last_became_peered":"2024-03-06T07:26:07.696515+0000","last_unstale":"2024-03-06T07:26:07.699416+0000","last_undegraded":"2024-03-06T07:26:07.699416+0000","last_fullsized":"2024-03-06T07:26:07.699416+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:45.409316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687900+0000","last_change":"2024-03-06T07:26:06.693663+0000","last_active":"2024-03-06T07:26:08.687900+0000","last_peered":"2024-03-06T07:26:08.687900+0000","last_clean":"2024-03-06T07:26:08.687900+0000","last_became_active":"2024-03-06T07:26:06.693251+0000","last_became_peered":"2024-03-06T07:26:06.693251+0000","last_unstale":"2024-03-06T07:26:08.687900+0000","last_undegraded":"2024-03-06T07:26:08.687900+0000","last_fullsized":"2024-03-06T07:26:08.687900+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699645+0000","last_change":"2024-03-06T07:26:07.698185+0000","last_active":"2024-03-06T07:26:07.699645+0000","last_peered":"2024-03-06T07:26:07.699645+0000","last_clean":"2024-03-06T07:26:07.699645+0000","last_became_active":"2024-03-06T07:26:07.696887+0000","last_became_peered":"2024-03-06T07:26:07.696887+0000","last_unstale":"2024-03-06T07:26:07.699645+0000","last_undegraded":"2024-03-06T07:26:07.699645+0000","last_fullsized":"2024-03-06T07:26:07.699645+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:50:39.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700493+0000","last_change":"2024-03-06T07:26:06.689754+0000","last_active":"2024-03-06T07:26:08.700493+0000","last_peered":"2024-03-06T07:26:08.700493+0000","last_clean":"2024-03-06T07:26:08.700493+0000","last_became_active":"2024-03-06T07:26:06.689386+0000","last_became_peered":"2024-03-06T07:26:06.689386+0000","last_unstale":"2024-03-06T07:26:08.700493+0000","last_undegraded":"2024-03-06T07:26:08.700493+0000","last_fullsized":"2024-03-06T07:26:08.700493+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700180+0000","last_change":"2024-03-06T07:26:07.697222+0000","last_active":"2024-03-06T07:26:08.700180+0000","last_peered":"2024-03-06T07:26:08.700180+0000","last_clean":"2024-03-06T07:26:08.700180+0000","last_became_active":"2024-03-06T07:26:07.696964+0000","last_became_peered":"2024-03-06T07:26:07.696964+0000","last_unstale":"2024-03-06T07:26:08.700180+0000","last_undegraded":"2024-03-06T07:26:08.700180+0000","last_fullsized":"2024-03-06T07:26:08.700180+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:59.699225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"150'3","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706357+0000","last_change":"2024-03-06T07:26:06.693324+0000","last_active":"2024-03-06T07:26:07.706357+0000","last_peered":"2024-03-06T07:26:07.706357+0000","last_clean":"2024-03-06T07:26:07.706357+0000","last_became_active":"2024-03-06T07:26:06.693047+0000","last_became_peered":"2024-03-06T07:26:06.693047+0000","last_unstale":"2024-03-06T07:26:07.706357+0000","last_undegraded":"2024-03-06T07:26:07.706357+0000","last_fullsized":"2024-03-06T07:26:07.706357+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:59.568199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706239+0000","last_change":"2024-03-06T07:26:07.704905+0000","last_active":"2024-03-06T07:26:07.706239+0000","last_peered":"2024-03-06T07:26:07.706239+0000","last_clean":"2024-03-06T07:26:07.706239+0000","last_became_active":"2024-03-06T07:26:07.704581+0000","last_became_peered":"2024-03-06T07:26:07.704581+0000","last_unstale":"2024-03-06T07:26:07.706239+0000","last_undegraded":"2024-03-06T07:26:07.706239+0000","last_fullsized":"2024-03-06T07:26:07.706239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:46.436137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"150'6","reported_seq":20,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700384+0000","last_change":"2024-03-06T07:26:06.686441+0000","last_active":"2024-03-06T07:26:08.700384+0000","last_peered":"2024-03-06T07:26:08.700384+0000","last_clean":"2024-03-06T07:26:08.700384+0000","last_became_active":"2024-03-06T07:26:06.686158+0000","last_became_peered":"2024-03-06T07:26:06.686158+0000","last_unstale":"2024-03-06T07:26:08.700384+0000","last_undegraded":"2024-03-06T07:26:08.700384+0000","last_fullsized":"2024-03-06T07:26:08.700384+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:05.262174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.702068+0000","last_change":"2024-03-06T07:26:07.698486+0000","last_active":"2024-03-06T07:26:07.702068+0000","last_peered":"2024-03-06T07:26:07.702068+0000","last_clean":"2024-03-06T07:26:07.702068+0000","last_became_active":"2024-03-06T07:26:07.698210+0000","last_became_peered":"2024-03-06T07:26:07.698210+0000","last_unstale":"2024-03-06T07:26:07.702068+0000","last_undegraded":"2024-03-06T07:26:07.702068+0000","last_fullsized":"2024-03-06T07:26:07.702068+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:38.877911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839584+0000","last_change":"2024-03-06T07:26:06.698989+0000","last_active":"2024-03-06T07:26:08.839584+0000","last_peered":"2024-03-06T07:26:08.839584+0000","last_clean":"2024-03-06T07:26:08.839584+0000","last_became_active":"2024-03-06T07:26:06.698656+0000","last_became_peered":"2024-03-06T07:26:06.698656+0000","last_unstale":"2024-03-06T07:26:08.839584+0000","last_undegraded":"2024-03-06T07:26:08.839584+0000","last_fullsized":"2024-03-06T07:26:08.839584+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688101+0000","last_change":"2024-03-06T07:26:07.709375+0000","last_active":"2024-03-06T07:26:08.688101+0000","last_peered":"2024-03-06T07:26:08.688101+0000","last_clean":"2024-03-06T07:26:08.688101+0000","last_became_active":"2024-03-06T07:26:07.709091+0000","last_became_peered":"2024-03-06T07:26:07.709091+0000","last_unstale":"2024-03-06T07:26:08.688101+0000","last_undegraded":"2024-03-06T07:26:08.688101+0000","last_fullsized":"2024-03-06T07:26:08.688101+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:41.934898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700175+0000","last_change":"2024-03-06T07:26:06.695622+0000","last_active":"2024-03-06T07:26:08.700175+0000","last_peered":"2024-03-06T07:26:08.700175+0000","last_clean":"2024-03-06T07:26:08.700175+0000","last_became_active":"2024-03-06T07:26:06.695369+0000","last_became_peered":"2024-03-06T07:26:06.695369+0000","last_unstale":"2024-03-06T07:26:08.700175+0000","last_undegraded":"2024-03-06T07:26:08.700175+0000","last_fullsized":"2024-03-06T07:26:08.700175+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:26.984542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691613+0000","last_change":"2024-03-06T07:26:07.698143+0000","last_active":"2024-03-06T07:26:08.691613+0000","last_peered":"2024-03-06T07:26:08.691613+0000","last_clean":"2024-03-06T07:26:08.691613+0000","last_became_active":"2024-03-06T07:26:07.697880+0000","last_became_peered":"2024-03-06T07:26:07.697880+0000","last_unstale":"2024-03-06T07:26:08.691613+0000","last_undegraded":"2024-03-06T07:26:08.691613+0000","last_fullsized":"2024-03-06T07:26:08.691613+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691525+0000","last_change":"2024-03-06T07:26:06.696386+0000","last_active":"2024-03-06T07:26:08.691525+0000","last_peered":"2024-03-06T07:26:08.691525+0000","last_clean":"2024-03-06T07:26:08.691525+0000","last_became_active":"2024-03-06T07:26:06.696108+0000","last_became_peered":"2024-03-06T07:26:06.696108+0000","last_unstale":"2024-03-06T07:26:08.691525+0000","last_undegraded":"2024-03-06T07:26:08.691525+0000","last_fullsized":"2024-03-06T07:26:08.691525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.696914+0000","last_change":"2024-03-06T07:26:07.694636+0000","last_active":"2024-03-06T07:26:07.696914+0000","last_peered":"2024-03-06T07:26:07.696914+0000","last_clean":"2024-03-06T07:26:07.696914+0000","last_became_active":"2024-03-06T07:26:07.694340+0000","last_became_peered":"2024-03-06T07:26:07.694340+0000","last_unstale":"2024-03-06T07:26:07.696914+0000","last_undegraded":"2024-03-06T07:26:07.696914+0000","last_fullsized":"2024-03-06T07:26:07.696914+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:39.420525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"150'8","reported_seq":22,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687810+0000","last_change":"2024-03-06T07:26:06.692364+0000","last_active":"2024-03-06T07:26:08.687810+0000","last_peered":"2024-03-06T07:26:08.687810+0000","last_clean":"2024-03-06T07:26:08.687810+0000","last_became_active":"2024-03-06T07:26:06.692119+0000","last_became_peered":"2024-03-06T07:26:06.692119+0000","last_unstale":"2024-03-06T07:26:08.687810+0000","last_undegraded":"2024-03-06T07:26:08.687810+0000","last_fullsized":"2024-03-06T07:26:08.687810+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687687+0000","last_change":"2024-03-06T07:26:07.698466+0000","last_active":"2024-03-06T07:26:08.687687+0000","last_peered":"2024-03-06T07:26:08.687687+0000","last_clean":"2024-03-06T07:26:08.687687+0000","last_became_active":"2024-03-06T07:26:07.697848+0000","last_became_peered":"2024-03-06T07:26:07.697848+0000","last_unstale":"2024-03-06T07:26:08.687687+0000","last_undegraded":"2024-03-06T07:26:08.687687+0000","last_fullsized":"2024-03-06T07:26:08.687687+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700582+0000","last_change":"2024-03-06T07:26:06.694226+0000","last_active":"2024-03-06T07:26:08.700582+0000","last_peered":"2024-03-06T07:26:08.700582+0000","last_clean":"2024-03-06T07:26:08.700582+0000","last_became_active":"2024-03-06T07:26:06.693966+0000","last_became_peered":"2024-03-06T07:26:06.693966+0000","last_unstale":"2024-03-06T07:26:08.700582+0000","last_undegraded":"2024-03-06T07:26:08.700582+0000","last_fullsized":"2024-03-06T07:26:08.700582+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:45.769621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697346+0000","last_change":"2024-03-06T07:26:07.695629+0000","last_active":"2024-03-06T07:26:07.697346+0000","last_peered":"2024-03-06T07:26:07.697346+0000","last_clean":"2024-03-06T07:26:07.697346+0000","last_became_active":"2024-03-06T07:26:07.695358+0000","last_became_peered":"2024-03-06T07:26:07.695358+0000","last_unstale":"2024-03-06T07:26:07.697346+0000","last_undegraded":"2024-03-06T07:26:07.697346+0000","last_fullsized":"2024-03-06T07:26:07.697346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:12.120009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688146+0000","last_change":"2024-03-06T07:26:06.698927+0000","last_active":"2024-03-06T07:26:08.688146+0000","last_peered":"2024-03-06T07:26:08.688146+0000","last_clean":"2024-03-06T07:26:08.688146+0000","last_became_active":"2024-03-06T07:26:06.698636+0000","last_became_peered":"2024-03-06T07:26:06.698636+0000","last_unstale":"2024-03-06T07:26:08.688146+0000","last_undegraded":"2024-03-06T07:26:08.688146+0000","last_fullsized":"2024-03-06T07:26:08.688146+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691742+0000","last_change":"2024-03-06T07:26:07.697409+0000","last_active":"2024-03-06T07:26:08.691742+0000","last_peered":"2024-03-06T07:26:08.691742+0000","last_clean":"2024-03-06T07:26:08.691742+0000","last_became_active":"2024-03-06T07:26:07.697084+0000","last_became_peered":"2024-03-06T07:26:07.697084+0000","last_unstale":"2024-03-06T07:26:08.691742+0000","last_undegraded":"2024-03-06T07:26:08.691742+0000","last_fullsized":"2024-03-06T07:26:08.691742+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839724+0000","last_change":"2024-03-06T07:26:06.695033+0000","last_active":"2024-03-06T07:26:08.839724+0000","last_peered":"2024-03-06T07:26:08.839724+0000","last_clean":"2024-03-06T07:26:08.839724+0000","last_became_active":"2024-03-06T07:26:06.694763+0000","last_became_peered":"2024-03-06T07:26:06.694763+0000","last_unstale":"2024-03-06T07:26:08.839724+0000","last_undegraded":"2024-03-06T07:26:08.839724+0000","last_fullsized":"2024-03-06T07:26:08.839724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:29.440623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691584+0000","last_change":"2024-03-06T07:26:07.706589+0000","last_active":"2024-03-06T07:26:08.691584+0000","last_peered":"2024-03-06T07:26:08.691584+0000","last_clean":"2024-03-06T07:26:08.691584+0000","last_became_active":"2024-03-06T07:26:07.706305+0000","last_became_peered":"2024-03-06T07:26:07.706305+0000","last_unstale":"2024-03-06T07:26:08.691584+0000","last_undegraded":"2024-03-06T07:26:08.691584+0000","last_fullsized":"2024-03-06T07:26:08.691584+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687999+0000","last_change":"2024-03-06T07:26:06.697335+0000","last_active":"2024-03-06T07:26:08.687999+0000","last_peered":"2024-03-06T07:26:08.687999+0000","last_clean":"2024-03-06T07:26:08.687999+0000","last_became_active":"2024-03-06T07:26:06.697028+0000","last_became_peered":"2024-03-06T07:26:06.697028+0000","last_unstale":"2024-03-06T07:26:08.687999+0000","last_undegraded":"2024-03-06T07:26:08.687999+0000","last_fullsized":"2024-03-06T07:26:08.687999+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700431+0000","last_change":"2024-03-06T07:26:07.692326+0000","last_active":"2024-03-06T07:26:08.700431+0000","last_peered":"2024-03-06T07:26:08.700431+0000","last_clean":"2024-03-06T07:26:08.700431+0000","last_became_active":"2024-03-06T07:26:07.692063+0000","last_became_peered":"2024-03-06T07:26:07.692063+0000","last_unstale":"2024-03-06T07:26:08.700431+0000","last_undegraded":"2024-03-06T07:26:08.700431+0000","last_fullsized":"2024-03-06T07:26:08.700431+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:42.601237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839694+0000","last_change":"2024-03-06T07:26:06.696078+0000","last_active":"2024-03-06T07:26:08.839694+0000","last_peered":"2024-03-06T07:26:08.839694+0000","last_clean":"2024-03-06T07:26:08.839694+0000","last_became_active":"2024-03-06T07:26:06.695834+0000","last_became_peered":"2024-03-06T07:26:06.695834+0000","last_unstale":"2024-03-06T07:26:08.839694+0000","last_undegraded":"2024-03-06T07:26:08.839694+0000","last_fullsized":"2024-03-06T07:26:08.839694+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839725+0000","last_change":"2024-03-06T07:26:07.713113+0000","last_active":"2024-03-06T07:26:08.839725+0000","last_peered":"2024-03-06T07:26:08.839725+0000","last_clean":"2024-03-06T07:26:08.839725+0000","last_became_active":"2024-03-06T07:26:07.712610+0000","last_became_peered":"2024-03-06T07:26:07.712610+0000","last_unstale":"2024-03-06T07:26:08.839725+0000","last_undegraded":"2024-03-06T07:26:08.839725+0000","last_fullsized":"2024-03-06T07:26:08.839725+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:03.957417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691378+0000","last_change":"2024-03-06T07:26:06.692781+0000","last_active":"2024-03-06T07:26:08.691378+0000","last_peered":"2024-03-06T07:26:08.691378+0000","last_clean":"2024-03-06T07:26:08.691378+0000","last_became_active":"2024-03-06T07:26:06.691992+0000","last_became_peered":"2024-03-06T07:26:06.691992+0000","last_unstale":"2024-03-06T07:26:08.691378+0000","last_undegraded":"2024-03-06T07:26:08.691378+0000","last_fullsized":"2024-03-06T07:26:08.691378+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839553+0000","last_change":"2024-03-06T07:26:07.713547+0000","last_active":"2024-03-06T07:26:08.839553+0000","last_peered":"2024-03-06T07:26:08.839553+0000","last_clean":"2024-03-06T07:26:08.839553+0000","last_became_active":"2024-03-06T07:26:07.712950+0000","last_became_peered":"2024-03-06T07:26:07.712950+0000","last_unstale":"2024-03-06T07:26:08.839553+0000","last_undegraded":"2024-03-06T07:26:08.839553+0000","last_fullsized":"2024-03-06T07:26:08.839553+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:00.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.687580+0000","last_change":"2024-03-06T07:26:06.695426+0000","last_active":"2024-03-06T07:26:07.687580+0000","last_peered":"2024-03-06T07:26:07.687580+0000","last_clean":"2024-03-06T07:26:07.687580+0000","last_became_active":"2024-03-06T07:26:06.695162+0000","last_became_peered":"2024-03-06T07:26:06.695162+0000","last_unstale":"2024-03-06T07:26:07.687580+0000","last_undegraded":"2024-03-06T07:26:07.687580+0000","last_fullsized":"2024-03-06T07:26:07.687580+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:30.485289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.696913+0000","last_change":"2024-03-06T07:26:07.691244+0000","last_active":"2024-03-06T07:26:07.696913+0000","last_peered":"2024-03-06T07:26:07.696913+0000","last_clean":"2024-03-06T07:26:07.696913+0000","last_became_active":"2024-03-06T07:26:07.690929+0000","last_became_peered":"2024-03-06T07:26:07.690929+0000","last_unstale":"2024-03-06T07:26:07.696913+0000","last_undegraded":"2024-03-06T07:26:07.696913+0000","last_fullsized":"2024-03-06T07:26:07.696913+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:36.240577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.685833+0000","last_change":"2024-03-06T07:26:06.694068+0000","last_active":"2024-03-06T07:26:07.685833+0000","last_peered":"2024-03-06T07:26:07.685833+0000","last_clean":"2024-03-06T07:26:07.685833+0000","last_became_active":"2024-03-06T07:26:06.693777+0000","last_became_peered":"2024-03-06T07:26:06.693777+0000","last_unstale":"2024-03-06T07:26:07.685833+0000","last_undegraded":"2024-03-06T07:26:07.685833+0000","last_fullsized":"2024-03-06T07:26:07.685833+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:06.186255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691245+0000","last_change":"2024-03-06T07:26:07.696197+0000","last_active":"2024-03-06T07:26:08.691245+0000","last_peered":"2024-03-06T07:26:08.691245+0000","last_clean":"2024-03-06T07:26:08.691245+0000","last_became_active":"2024-03-06T07:26:07.695895+0000","last_became_peered":"2024-03-06T07:26:07.695895+0000","last_unstale":"2024-03-06T07:26:08.691245+0000","last_undegraded":"2024-03-06T07:26:08.691245+0000","last_fullsized":"2024-03-06T07:26:08.691245+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:02.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839696+0000","last_change":"2024-03-06T07:26:06.701125+0000","last_active":"2024-03-06T07:26:08.839696+0000","last_peered":"2024-03-06T07:26:08.839696+0000","last_clean":"2024-03-06T07:26:08.839696+0000","last_became_active":"2024-03-06T07:26:06.700819+0000","last_became_peered":"2024-03-06T07:26:06.700819+0000","last_unstale":"2024-03-06T07:26:08.839696+0000","last_undegraded":"2024-03-06T07:26:08.839696+0000","last_fullsized":"2024-03-06T07:26:08.839696+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:54.995466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687769+0000","last_change":"2024-03-06T07:26:07.701964+0000","last_active":"2024-03-06T07:26:08.687769+0000","last_peered":"2024-03-06T07:26:08.687769+0000","last_clean":"2024-03-06T07:26:08.687769+0000","last_became_active":"2024-03-06T07:26:07.701660+0000","last_became_peered":"2024-03-06T07:26:07.701660+0000","last_unstale":"2024-03-06T07:26:08.687769+0000","last_undegraded":"2024-03-06T07:26:08.687769+0000","last_fullsized":"2024-03-06T07:26:08.687769+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"151'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.701623+0000","last_change":"2024-03-06T07:26:06.693341+0000","last_active":"2024-03-06T07:26:08.701623+0000","last_peered":"2024-03-06T07:26:08.701623+0000","last_clean":"2024-03-06T07:26:08.701623+0000","last_became_active":"2024-03-06T07:26:06.693014+0000","last_became_peered":"2024-03-06T07:26:06.693014+0000","last_unstale":"2024-03-06T07:26:08.701623+0000","last_undegraded":"2024-03-06T07:26:08.701623+0000","last_fullsized":"2024-03-06T07:26:08.701623+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:03.887560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.703444+0000","last_change":"2024-03-06T07:26:07.697077+0000","last_active":"2024-03-06T07:26:07.703444+0000","last_peered":"2024-03-06T07:26:07.703444+0000","last_clean":"2024-03-06T07:26:07.703444+0000","last_became_active":"2024-03-06T07:26:07.695444+0000","last_became_peered":"2024-03-06T07:26:07.695444+0000","last_unstale":"2024-03-06T07:26:07.703444+0000","last_undegraded":"2024-03-06T07:26:07.703444+0000","last_fullsized":"2024-03-06T07:26:07.703444+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:14.350502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700450+0000","last_change":"2024-03-06T07:26:06.689881+0000","last_active":"2024-03-06T07:26:08.700450+0000","last_peered":"2024-03-06T07:26:08.700450+0000","last_clean":"2024-03-06T07:26:08.700450+0000","last_became_active":"2024-03-06T07:26:06.689604+0000","last_became_peered":"2024-03-06T07:26:06.689604+0000","last_unstale":"2024-03-06T07:26:08.700450+0000","last_undegraded":"2024-03-06T07:26:08.700450+0000","last_fullsized":"2024-03-06T07:26:08.700450+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687869+0000","last_change":"2024-03-06T07:26:07.700732+0000","last_active":"2024-03-06T07:26:08.687869+0000","last_peered":"2024-03-06T07:26:08.687869+0000","last_clean":"2024-03-06T07:26:08.687869+0000","last_became_active":"2024-03-06T07:26:07.700382+0000","last_became_peered":"2024-03-06T07:26:07.700382+0000","last_unstale":"2024-03-06T07:26:08.687869+0000","last_undegraded":"2024-03-06T07:26:08.687869+0000","last_fullsized":"2024-03-06T07:26:08.687869+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:29.340249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.686330+0000","last_change":"2024-03-06T07:26:06.693374+0000","last_active":"2024-03-06T07:26:07.686330+0000","last_peered":"2024-03-06T07:26:07.686330+0000","last_clean":"2024-03-06T07:26:07.686330+0000","last_became_active":"2024-03-06T07:26:06.693113+0000","last_became_peered":"2024-03-06T07:26:06.693113+0000","last_unstale":"2024-03-06T07:26:07.686330+0000","last_undegraded":"2024-03-06T07:26:07.686330+0000","last_fullsized":"2024-03-06T07:26:07.686330+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:55.760111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705508+0000","last_change":"2024-03-06T07:26:07.701237+0000","last_active":"2024-03-06T07:26:07.705508+0000","last_peered":"2024-03-06T07:26:07.705508+0000","last_clean":"2024-03-06T07:26:07.705508+0000","last_became_active":"2024-03-06T07:26:07.700717+0000","last_became_peered":"2024-03-06T07:26:07.700717+0000","last_unstale":"2024-03-06T07:26:07.705508+0000","last_undegraded":"2024-03-06T07:26:07.705508+0000","last_fullsized":"2024-03-06T07:26:07.705508+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:57.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.684455+0000","last_change":"2024-03-06T07:26:06.695137+0000","last_active":"2024-03-06T07:26:07.684455+0000","last_peered":"2024-03-06T07:26:07.684455+0000","last_clean":"2024-03-06T07:26:07.684455+0000","last_became_active":"2024-03-06T07:26:06.694881+0000","last_became_peered":"2024-03-06T07:26:06.694881+0000","last_unstale":"2024-03-06T07:26:07.684455+0000","last_undegraded":"2024-03-06T07:26:07.684455+0000","last_fullsized":"2024-03-06T07:26:07.684455+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:37.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706049+0000","last_change":"2024-03-06T07:26:07.705151+0000","last_active":"2024-03-06T07:26:07.706049+0000","last_peered":"2024-03-06T07:26:07.706049+0000","last_clean":"2024-03-06T07:26:07.706049+0000","last_became_active":"2024-03-06T07:26:07.704856+0000","last_became_peered":"2024-03-06T07:26:07.704856+0000","last_unstale":"2024-03-06T07:26:07.706049+0000","last_undegraded":"2024-03-06T07:26:07.706049+0000","last_fullsized":"2024-03-06T07:26:07.706049+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700271+0000","last_change":"2024-03-06T07:26:06.684537+0000","last_active":"2024-03-06T07:26:08.700271+0000","last_peered":"2024-03-06T07:26:08.700271+0000","last_clean":"2024-03-06T07:26:08.700271+0000","last_became_active":"2024-03-06T07:26:06.683851+0000","last_became_peered":"2024-03-06T07:26:06.683851+0000","last_unstale":"2024-03-06T07:26:08.700271+0000","last_undegraded":"2024-03-06T07:26:08.700271+0000","last_fullsized":"2024-03-06T07:26:08.700271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:07.304752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700491+0000","last_change":"2024-03-06T07:26:07.691230+0000","last_active":"2024-03-06T07:26:08.700491+0000","last_peered":"2024-03-06T07:26:08.700491+0000","last_clean":"2024-03-06T07:26:08.700491+0000","last_became_active":"2024-03-06T07:26:07.690936+0000","last_became_peered":"2024-03-06T07:26:07.690936+0000","last_unstale":"2024-03-06T07:26:08.700491+0000","last_undegraded":"2024-03-06T07:26:08.700491+0000","last_fullsized":"2024-03-06T07:26:08.700491+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:09.494792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691668+0000","last_change":"2024-03-06T07:26:06.692326+0000","last_active":"2024-03-06T07:26:08.691668+0000","last_peered":"2024-03-06T07:26:08.691668+0000","last_clean":"2024-03-06T07:26:08.691668+0000","last_became_active":"2024-03-06T07:26:06.691972+0000","last_became_peered":"2024-03-06T07:26:06.691972+0000","last_unstale":"2024-03-06T07:26:08.691668+0000","last_undegraded":"2024-03-06T07:26:08.691668+0000","last_fullsized":"2024-03-06T07:26:08.691668+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706117+0000","last_change":"2024-03-06T07:26:07.704775+0000","last_active":"2024-03-06T07:26:07.706117+0000","last_peered":"2024-03-06T07:26:07.706117+0000","last_clean":"2024-03-06T07:26:07.706117+0000","last_became_active":"2024-03-06T07:26:07.704502+0000","last_became_peered":"2024-03-06T07:26:07.704502+0000","last_unstale":"2024-03-06T07:26:07.706117+0000","last_undegraded":"2024-03-06T07:26:07.706117+0000","last_fullsized":"2024-03-06T07:26:07.706117+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:02.129392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839177+0000","last_change":"2024-03-06T07:26:06.696947+0000","last_active":"2024-03-06T07:26:08.839177+0000","last_peered":"2024-03-06T07:26:08.839177+0000","last_clean":"2024-03-06T07:26:08.839177+0000","last_became_active":"2024-03-06T07:26:06.696715+0000","last_became_peered":"2024-03-06T07:26:06.696715+0000","last_unstale":"2024-03-06T07:26:08.839177+0000","last_undegraded":"2024-03-06T07:26:08.839177+0000","last_fullsized":"2024-03-06T07:26:08.839177+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:40.929460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.701853+0000","last_change":"2024-03-06T07:26:07.699110+0000","last_active":"2024-03-06T07:26:07.701853+0000","last_peered":"2024-03-06T07:26:07.701853+0000","last_clean":"2024-03-06T07:26:07.701853+0000","last_became_active":"2024-03-06T07:26:07.698799+0000","last_became_peered":"2024-03-06T07:26:07.698799+0000","last_unstale":"2024-03-06T07:26:07.701853+0000","last_undegraded":"2024-03-06T07:26:07.701853+0000","last_fullsized":"2024-03-06T07:26:07.701853+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700485+0000","last_change":"2024-03-06T07:26:06.691133+0000","last_active":"2024-03-06T07:26:08.700485+0000","last_peered":"2024-03-06T07:26:08.700485+0000","last_clean":"2024-03-06T07:26:08.700485+0000","last_became_active":"2024-03-06T07:26:06.690687+0000","last_became_peered":"2024-03-06T07:26:06.690687+0000","last_unstale":"2024-03-06T07:26:08.700485+0000","last_undegraded":"2024-03-06T07:26:08.700485+0000","last_fullsized":"2024-03-06T07:26:08.700485+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:08.352394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.685888+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:26:07.685888+0000","last_peered":"2024-03-06T07:26:07.685888+0000","last_clean":"2024-03-06T07:26:07.685888+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:26:07.685888+0000","last_undegraded":"2024-03-06T07:26:07.685888+0000","last_fullsized":"2024-03-06T07:26:07.685888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1953792,"data_stored":1502417,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1888256,"data_stored":912049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1908736,"data_stored":918865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1904640,"data_stored":912813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1921024,"data_stored":920957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":912049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1912832,"data_stored":920353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1974272,"data_stored":1509213,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:11.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.827+0000 7f7fe551e640 1 Processor -- start 2024-03-06T07:26:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.827+0000 7f7fe551e640 1 -- start start 2024-03-06T07:26:11.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.828+0000 7f7fe551e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7fe00c7180 0x7f7fe0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.828+0000 7f7fe551e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe00c7c60 con 0x7f7fe00c88b0 2024-03-06T07:26:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.828+0000 7f7fe551e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe00c8060 con 0x7f7fe01527b0 2024-03-06T07:26:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.828+0000 7f7fe551e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe0159640 con 0x7f7fe00c7180 2024-03-06T07:26:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.828+0000 7f7fdf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7fe00c88b0 0x7f7fe00c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54964/0 (socket says 172.21.15.26:54964) 2024-03-06T07:26:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.828+0000 7f7fdf7fe640 1 -- 172.21.15.26:0/2327827541 learned_addr learned my addr 172.21.15.26:0/2327827541 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fde7fc640 1 --2- 172.21.15.26:0/2327827541 >> v2:172.21.15.117:3300/0 conn(0x7f7fe00c7180 0x7f7fe0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657605967 0 0) 0x7f7fd8001490 con 0x7f7fe00c88b0 2024-03-06T07:26:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc8003520 con 0x7f7fe00c88b0 2024-03-06T07:26:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3802819594 0 0) 0x7f7fc8003520 con 0x7f7fe00c88b0 2024-03-06T07:26:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 >> v2:172.21.15.117:3300/0 conn(0x7f7fe00c7180 msgr2=0x7f7fe0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 --2- 172.21.15.26:0/2327827541 >> v2:172.21.15.117:3300/0 conn(0x7f7fe00c7180 0x7f7fe0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 >> v1:172.21.15.117:6789/0 conn(0x7f7fe01527b0 legacy=0x7f7fe0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe00c76a0 con 0x7f7fe00c88b0 2024-03-06T07:26:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f7fd8001bc0 con 0x7f7fe00c88b0 2024-03-06T07:26:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 >> v1:172.21.15.26:6789/0 conn(0x7f7fe00c88b0 legacy=0x7f7fe00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc80039b0 0x7f7fc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8004800 0x7f7fc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0159640 con 0x7f7fc80051c0 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc800c160 con 0x7f7fc8004800 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc800c360 con 0x7f7fc80039b0 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fdf7fe640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fde7fc640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8004800 0x7f7fc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fdeffd640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc80039b0 0x7f7fc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fdf7fe640 1 -- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc80039b0 msgr2=0x7f7fc8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fdf7fe640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc80039b0 0x7f7fc8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fdf7fe640 1 -- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8004800 msgr2=0x7f7fc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fdf7fe640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8004800 0x7f7fc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.829+0000 7f7fdf7fe640 1 -- 172.21.15.26:0/2327827541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8003520 con 0x7f7fc80051c0 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.830+0000 7f7fdf7fe640 1 -- 172.21.15.26:0/2327827541 reap_dead start 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fdf7fe640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fc800b9c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800aa50 tx=0x7f7fc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ffd062d19745e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8038070 con 0x7f7fc80051c0 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd8033650 con 0x7f7fc80051c0 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fddffb640 1 -- 172.21.15.26:0/2327827541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800a670 con 0x7f7fc80051c0 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fe551e640 1 -- 172.21.15.26:0/2327827541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 msgr2=0x7f7fc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fe551e640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fc800b9c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800aa50 tx=0x7f7fc8005830 comp rx=0 tx=0).stop 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fe551e640 1 -- 172.21.15.26:0/2327827541 shutdown_connections 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fe551e640 1 --2- 172.21.15.26:0/2327827541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fc800b9c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fe551e640 1 -- 172.21.15.26:0/2327827541 >> 172.21.15.26:0/2327827541 conn(0x7f7fe00ac560 msgr2=0x7f7fe00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fe551e640 1 -- 172.21.15.26:0/2327827541 shutdown_connections 2024-03-06T07:26:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.831+0000 7f7fe551e640 1 -- 172.21.15.26:0/2327827541 wait complete. 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.833+0000 7f7fe551e640 1 Processor -- start 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.833+0000 7f7fe551e640 1 -- start start 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fe551e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fe01ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fe551e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c3110 0x7f7fe014ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fe551e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe014cf90 0x7f7fe014bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fe551e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0155600 con 0x7f7fc80051c0 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fe551e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0155320 con 0x7f7fe014cf90 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fe551e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0152b90 con 0x7f7fe00c3110 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fe01ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fe01ee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44786/0 (socket says 172.21.15.26:44786) 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 -- 172.21.15.26:0/3860525111 learned_addr learned my addr 172.21.15.26:0/3860525111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdf7fe640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe014cf90 0x7f7fe014bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fde7fc640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c3110 0x7f7fe014ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 -- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c3110 msgr2=0x7f7fe014ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c3110 0x7f7fe014ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 -- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe014cf90 msgr2=0x7f7fe014bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe014cf90 0x7f7fe014bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.834+0000 7f7fdeffd640 1 -- 172.21.15.26:0/3860525111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe01eebf0 con 0x7f7fc80051c0 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.835+0000 7f7fdeffd640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fe01ee3b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400f650 tx=0x7f7fd400fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.835+0000 7f7fc77fe640 1 -- 172.21.15.26:0/3860525111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40052e0 con 0x7f7fc80051c0 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.835+0000 7f7fc77fe640 1 -- 172.21.15.26:0/3860525111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd4018070 con 0x7f7fc80051c0 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.835+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe00c2630 con 0x7f7fc80051c0 2024-03-06T07:26:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.835+0000 7f7fc77fe640 1 -- 172.21.15.26:0/3860525111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4003810 con 0x7f7fc80051c0 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.835+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe01ee8f0 con 0x7f7fc80051c0 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.836+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe00ca6e0 con 0x7f7fc80051c0 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.836+0000 7f7fc77fe640 1 -- 172.21.15.26:0/3860525111 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fd40039b0 con 0x7f7fc80051c0 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.836+0000 7f7fc77fe640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fcc06b500 0x7f7fcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.837+0000 7f7fc77fe640 1 -- 172.21.15.26:0/3860525111 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fd4089fb0 con 0x7f7fc80051c0 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.837+0000 7f7fde7fc640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fcc06b500 0x7f7fcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.837+0000 7f7fde7fc640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fcc06b500 0x7f7fcc06d9c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0158ed0 tx=0x7f7fd0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:11.841+0000 7f7fc77fe640 1 -- 172.21.15.26:0/3860525111 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7fe00ca6e0 con 0x7f7fc80051c0 2024-03-06T07:26:12.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.059+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fe00c2440 con 0x7f7fcc06b500 2024-03-06T07:26:12.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.066+0000 7f7fc77fe640 1 -- 172.21.15.26:0/3860525111 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252525 (secure 0 0 0) 0x7f7fe00c2440 con 0x7f7fcc06b500 2024-03-06T07:26:12.071 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:12.072 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:26:12.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fcc06b500 msgr2=0x7f7fcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fcc06b500 0x7f7fcc06d9c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0158ed0 tx=0x7f7fd0009190 comp rx=0 tx=0).stop 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 msgr2=0x7f7fe01ee3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fe01ee3b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400f650 tx=0x7f7fd400fa50 comp rx=0 tx=0).stop 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 shutdown_connections 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fcc06b500 0x7f7fcc06d9c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe014cf90 0x7f7fe014bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c3110 0x7f7fe014ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 --2- 172.21.15.26:0/3860525111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc80051c0 0x7f7fe01ee3b0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 >> 172.21.15.26:0/3860525111 conn(0x7f7fe00ac560 msgr2=0x7f7fe00c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 shutdown_connections 2024-03-06T07:26:12.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.069+0000 7f7fe551e640 1 -- 172.21.15.26:0/3860525111 wait complete. 2024-03-06T07:26:12.083 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-06T07:26:11.106806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264464,"kb_used_data":14992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710256,"statfs":{"total":773094113280,"available":772823302144,"internally_reserved":0,"allocated":15351808,"data_stored":8508716,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.434280"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699396+0000","last_change":"2024-03-06T07:26:07.695841+0000","last_active":"2024-03-06T07:26:07.699396+0000","last_peered":"2024-03-06T07:26:07.699396+0000","last_clean":"2024-03-06T07:26:07.699396+0000","last_became_active":"2024-03-06T07:26:07.695553+0000","last_became_peered":"2024-03-06T07:26:07.695553+0000","last_unstale":"2024-03-06T07:26:07.699396+0000","last_undegraded":"2024-03-06T07:26:07.699396+0000","last_fullsized":"2024-03-06T07:26:07.699396+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:32.222328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699592+0000","last_change":"2024-03-06T07:26:07.698101+0000","last_active":"2024-03-06T07:26:07.699592+0000","last_peered":"2024-03-06T07:26:07.699592+0000","last_clean":"2024-03-06T07:26:07.699592+0000","last_became_active":"2024-03-06T07:26:07.697796+0000","last_became_peered":"2024-03-06T07:26:07.697796+0000","last_unstale":"2024-03-06T07:26:07.699592+0000","last_undegraded":"2024-03-06T07:26:07.699592+0000","last_fullsized":"2024-03-06T07:26:07.699592+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697427+0000","last_change":"2024-03-06T07:26:07.691124+0000","last_active":"2024-03-06T07:26:07.697427+0000","last_peered":"2024-03-06T07:26:07.697427+0000","last_clean":"2024-03-06T07:26:07.697427+0000","last_became_active":"2024-03-06T07:26:07.690832+0000","last_became_peered":"2024-03-06T07:26:07.690832+0000","last_unstale":"2024-03-06T07:26:07.697427+0000","last_undegraded":"2024-03-06T07:26:07.697427+0000","last_fullsized":"2024-03-06T07:26:07.697427+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:31:41.658258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.695720+0000","last_change":"2024-03-06T07:26:07.688926+0000","last_active":"2024-03-06T07:26:07.695720+0000","last_peered":"2024-03-06T07:26:07.695720+0000","last_clean":"2024-03-06T07:26:07.695720+0000","last_became_active":"2024-03-06T07:26:07.688684+0000","last_became_peered":"2024-03-06T07:26:07.688684+0000","last_unstale":"2024-03-06T07:26:07.695720+0000","last_undegraded":"2024-03-06T07:26:07.695720+0000","last_fullsized":"2024-03-06T07:26:07.695720+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:56.031022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839607+0000","last_change":"2024-03-06T07:26:07.708495+0000","last_active":"2024-03-06T07:26:08.839607+0000","last_peered":"2024-03-06T07:26:08.839607+0000","last_clean":"2024-03-06T07:26:08.839607+0000","last_became_active":"2024-03-06T07:26:07.708183+0000","last_became_peered":"2024-03-06T07:26:07.708183+0000","last_unstale":"2024-03-06T07:26:08.839607+0000","last_undegraded":"2024-03-06T07:26:08.839607+0000","last_fullsized":"2024-03-06T07:26:08.839607+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:00:41.851158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697761+0000","last_change":"2024-03-06T07:26:07.696015+0000","last_active":"2024-03-06T07:26:07.697761+0000","last_peered":"2024-03-06T07:26:07.697761+0000","last_clean":"2024-03-06T07:26:07.697761+0000","last_became_active":"2024-03-06T07:26:07.695708+0000","last_became_peered":"2024-03-06T07:26:07.695708+0000","last_unstale":"2024-03-06T07:26:07.697761+0000","last_undegraded":"2024-03-06T07:26:07.697761+0000","last_fullsized":"2024-03-06T07:26:07.697761+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:02.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687706+0000","last_change":"2024-03-06T07:26:07.699338+0000","last_active":"2024-03-06T07:26:08.687706+0000","last_peered":"2024-03-06T07:26:08.687706+0000","last_clean":"2024-03-06T07:26:08.687706+0000","last_became_active":"2024-03-06T07:26:07.699065+0000","last_became_peered":"2024-03-06T07:26:07.699065+0000","last_unstale":"2024-03-06T07:26:08.687706+0000","last_undegraded":"2024-03-06T07:26:08.687706+0000","last_fullsized":"2024-03-06T07:26:08.687706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687697+0000","last_change":"2024-03-06T07:26:07.700571+0000","last_active":"2024-03-06T07:26:08.687697+0000","last_peered":"2024-03-06T07:26:08.687697+0000","last_clean":"2024-03-06T07:26:08.687697+0000","last_became_active":"2024-03-06T07:26:07.698195+0000","last_became_peered":"2024-03-06T07:26:07.698195+0000","last_unstale":"2024-03-06T07:26:08.687697+0000","last_undegraded":"2024-03-06T07:26:08.687697+0000","last_fullsized":"2024-03-06T07:26:08.687697+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688048+0000","last_change":"2024-03-06T07:26:07.700691+0000","last_active":"2024-03-06T07:26:08.688048+0000","last_peered":"2024-03-06T07:26:08.688048+0000","last_clean":"2024-03-06T07:26:08.688048+0000","last_became_active":"2024-03-06T07:26:07.700158+0000","last_became_peered":"2024-03-06T07:26:07.700158+0000","last_unstale":"2024-03-06T07:26:08.688048+0000","last_undegraded":"2024-03-06T07:26:08.688048+0000","last_fullsized":"2024-03-06T07:26:08.688048+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:17.093387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688084+0000","last_change":"2024-03-06T07:26:07.700442+0000","last_active":"2024-03-06T07:26:08.688084+0000","last_peered":"2024-03-06T07:26:08.688084+0000","last_clean":"2024-03-06T07:26:08.688084+0000","last_became_active":"2024-03-06T07:26:07.700116+0000","last_became_peered":"2024-03-06T07:26:07.700116+0000","last_unstale":"2024-03-06T07:26:08.688084+0000","last_undegraded":"2024-03-06T07:26:08.688084+0000","last_fullsized":"2024-03-06T07:26:08.688084+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:30.446061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691434+0000","last_change":"2024-03-06T07:26:07.707356+0000","last_active":"2024-03-06T07:26:08.691434+0000","last_peered":"2024-03-06T07:26:08.691434+0000","last_clean":"2024-03-06T07:26:08.691434+0000","last_became_active":"2024-03-06T07:26:07.707082+0000","last_became_peered":"2024-03-06T07:26:07.707082+0000","last_unstale":"2024-03-06T07:26:08.691434+0000","last_undegraded":"2024-03-06T07:26:08.691434+0000","last_fullsized":"2024-03-06T07:26:08.691434+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697402+0000","last_change":"2024-03-06T07:26:07.694337+0000","last_active":"2024-03-06T07:26:07.697402+0000","last_peered":"2024-03-06T07:26:07.697402+0000","last_clean":"2024-03-06T07:26:07.697402+0000","last_became_active":"2024-03-06T07:26:07.694050+0000","last_became_peered":"2024-03-06T07:26:07.694050+0000","last_unstale":"2024-03-06T07:26:07.697402+0000","last_undegraded":"2024-03-06T07:26:07.697402+0000","last_fullsized":"2024-03-06T07:26:07.697402+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:24:50.068113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700250+0000","last_change":"2024-03-06T07:26:07.696706+0000","last_active":"2024-03-06T07:26:08.700250+0000","last_peered":"2024-03-06T07:26:08.700250+0000","last_clean":"2024-03-06T07:26:08.700250+0000","last_became_active":"2024-03-06T07:26:07.696438+0000","last_became_peered":"2024-03-06T07:26:07.696438+0000","last_unstale":"2024-03-06T07:26:08.700250+0000","last_undegraded":"2024-03-06T07:26:08.700250+0000","last_fullsized":"2024-03-06T07:26:08.700250+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:32:23.418950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839382+0000","last_change":"2024-03-06T07:26:07.704226+0000","last_active":"2024-03-06T07:26:08.839382+0000","last_peered":"2024-03-06T07:26:08.839382+0000","last_clean":"2024-03-06T07:26:08.839382+0000","last_became_active":"2024-03-06T07:26:07.703839+0000","last_became_peered":"2024-03-06T07:26:07.703839+0000","last_unstale":"2024-03-06T07:26:08.839382+0000","last_undegraded":"2024-03-06T07:26:08.839382+0000","last_fullsized":"2024-03-06T07:26:08.839382+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:52.356683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691271+0000","last_change":"2024-03-06T07:26:07.710172+0000","last_active":"2024-03-06T07:26:08.691271+0000","last_peered":"2024-03-06T07:26:08.691271+0000","last_clean":"2024-03-06T07:26:08.691271+0000","last_became_active":"2024-03-06T07:26:07.709871+0000","last_became_peered":"2024-03-06T07:26:07.709871+0000","last_unstale":"2024-03-06T07:26:08.691271+0000","last_undegraded":"2024-03-06T07:26:08.691271+0000","last_fullsized":"2024-03-06T07:26:08.691271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:17:00.303994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.695384+0000","last_change":"2024-03-06T07:26:07.688786+0000","last_active":"2024-03-06T07:26:07.695384+0000","last_peered":"2024-03-06T07:26:07.695384+0000","last_clean":"2024-03-06T07:26:07.695384+0000","last_became_active":"2024-03-06T07:26:07.688085+0000","last_became_peered":"2024-03-06T07:26:07.688085+0000","last_unstale":"2024-03-06T07:26:07.695384+0000","last_undegraded":"2024-03-06T07:26:07.695384+0000","last_fullsized":"2024-03-06T07:26:07.695384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:16:30.735444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688066+0000","last_change":"2024-03-06T07:26:07.711076+0000","last_active":"2024-03-06T07:26:08.688066+0000","last_peered":"2024-03-06T07:26:08.688066+0000","last_clean":"2024-03-06T07:26:08.688066+0000","last_became_active":"2024-03-06T07:26:07.710796+0000","last_became_peered":"2024-03-06T07:26:07.710796+0000","last_unstale":"2024-03-06T07:26:08.688066+0000","last_undegraded":"2024-03-06T07:26:08.688066+0000","last_fullsized":"2024-03-06T07:26:08.688066+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:56.000904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.704744+0000","last_change":"2024-03-06T07:26:07.703707+0000","last_active":"2024-03-06T07:26:07.704744+0000","last_peered":"2024-03-06T07:26:07.704744+0000","last_clean":"2024-03-06T07:26:07.704744+0000","last_became_active":"2024-03-06T07:26:07.703402+0000","last_became_peered":"2024-03-06T07:26:07.703402+0000","last_unstale":"2024-03-06T07:26:07.704744+0000","last_undegraded":"2024-03-06T07:26:07.704744+0000","last_fullsized":"2024-03-06T07:26:07.704744+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:00.573637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691558+0000","last_change":"2024-03-06T07:26:07.710174+0000","last_active":"2024-03-06T07:26:08.691558+0000","last_peered":"2024-03-06T07:26:08.691558+0000","last_clean":"2024-03-06T07:26:08.691558+0000","last_became_active":"2024-03-06T07:26:07.709939+0000","last_became_peered":"2024-03-06T07:26:07.709939+0000","last_unstale":"2024-03-06T07:26:08.691558+0000","last_undegraded":"2024-03-06T07:26:08.691558+0000","last_fullsized":"2024-03-06T07:26:08.691558+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706147+0000","last_change":"2024-03-06T07:26:07.704730+0000","last_active":"2024-03-06T07:26:07.706147+0000","last_peered":"2024-03-06T07:26:07.706147+0000","last_clean":"2024-03-06T07:26:07.706147+0000","last_became_active":"2024-03-06T07:26:07.704429+0000","last_became_peered":"2024-03-06T07:26:07.704429+0000","last_unstale":"2024-03-06T07:26:07.706147+0000","last_undegraded":"2024-03-06T07:26:07.706147+0000","last_fullsized":"2024-03-06T07:26:07.706147+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:19.206370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700300+0000","last_change":"2024-03-06T07:26:07.685590+0000","last_active":"2024-03-06T07:26:08.700300+0000","last_peered":"2024-03-06T07:26:08.700300+0000","last_clean":"2024-03-06T07:26:08.700300+0000","last_became_active":"2024-03-06T07:26:07.685265+0000","last_became_peered":"2024-03-06T07:26:07.685265+0000","last_unstale":"2024-03-06T07:26:08.700300+0000","last_undegraded":"2024-03-06T07:26:08.700300+0000","last_fullsized":"2024-03-06T07:26:08.700300+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:30.427263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.701827+0000","last_change":"2024-03-06T07:26:07.699358+0000","last_active":"2024-03-06T07:26:07.701827+0000","last_peered":"2024-03-06T07:26:07.701827+0000","last_clean":"2024-03-06T07:26:07.701827+0000","last_became_active":"2024-03-06T07:26:07.699058+0000","last_became_peered":"2024-03-06T07:26:07.699058+0000","last_unstale":"2024-03-06T07:26:07.701827+0000","last_undegraded":"2024-03-06T07:26:07.701827+0000","last_fullsized":"2024-03-06T07:26:07.701827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839269+0000","last_change":"2024-03-06T07:26:07.700848+0000","last_active":"2024-03-06T07:26:08.839269+0000","last_peered":"2024-03-06T07:26:08.839269+0000","last_clean":"2024-03-06T07:26:08.839269+0000","last_became_active":"2024-03-06T07:26:07.700551+0000","last_became_peered":"2024-03-06T07:26:07.700551+0000","last_unstale":"2024-03-06T07:26:08.839269+0000","last_undegraded":"2024-03-06T07:26:08.839269+0000","last_fullsized":"2024-03-06T07:26:08.839269+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.178917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.693524+0000","last_change":"2024-03-06T07:26:07.687552+0000","last_active":"2024-03-06T07:26:07.693524+0000","last_peered":"2024-03-06T07:26:07.693524+0000","last_clean":"2024-03-06T07:26:07.693524+0000","last_became_active":"2024-03-06T07:26:07.687276+0000","last_became_peered":"2024-03-06T07:26:07.687276+0000","last_unstale":"2024-03-06T07:26:07.693524+0000","last_undegraded":"2024-03-06T07:26:07.693524+0000","last_fullsized":"2024-03-06T07:26:07.693524+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:01.894148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700594+0000","last_change":"2024-03-06T07:26:07.698089+0000","last_active":"2024-03-06T07:26:08.700594+0000","last_peered":"2024-03-06T07:26:08.700594+0000","last_clean":"2024-03-06T07:26:08.700594+0000","last_became_active":"2024-03-06T07:26:07.697806+0000","last_became_peered":"2024-03-06T07:26:07.697806+0000","last_unstale":"2024-03-06T07:26:08.700594+0000","last_undegraded":"2024-03-06T07:26:08.700594+0000","last_fullsized":"2024-03-06T07:26:08.700594+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:52.239620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705132+0000","last_change":"2024-03-06T07:26:07.699922+0000","last_active":"2024-03-06T07:26:07.705132+0000","last_peered":"2024-03-06T07:26:07.705132+0000","last_clean":"2024-03-06T07:26:07.705132+0000","last_became_active":"2024-03-06T07:26:07.699631+0000","last_became_peered":"2024-03-06T07:26:07.699631+0000","last_unstale":"2024-03-06T07:26:07.705132+0000","last_undegraded":"2024-03-06T07:26:07.705132+0000","last_fullsized":"2024-03-06T07:26:07.705132+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:30:29.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.709238+0000","last_change":"2024-03-06T07:26:07.706359+0000","last_active":"2024-03-06T07:26:07.709238+0000","last_peered":"2024-03-06T07:26:07.709238+0000","last_clean":"2024-03-06T07:26:07.709238+0000","last_became_active":"2024-03-06T07:26:07.706096+0000","last_became_peered":"2024-03-06T07:26:07.706096+0000","last_unstale":"2024-03-06T07:26:07.709238+0000","last_undegraded":"2024-03-06T07:26:07.709238+0000","last_fullsized":"2024-03-06T07:26:07.709238+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:17.093387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700282+0000","last_change":"2024-03-06T07:26:07.693096+0000","last_active":"2024-03-06T07:26:08.700282+0000","last_peered":"2024-03-06T07:26:08.700282+0000","last_clean":"2024-03-06T07:26:08.700282+0000","last_became_active":"2024-03-06T07:26:07.692834+0000","last_became_peered":"2024-03-06T07:26:07.692834+0000","last_unstale":"2024-03-06T07:26:08.700282+0000","last_undegraded":"2024-03-06T07:26:08.700282+0000","last_fullsized":"2024-03-06T07:26:08.700282+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:58.002384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.709208+0000","last_change":"2024-03-06T07:26:07.706042+0000","last_active":"2024-03-06T07:26:07.709208+0000","last_peered":"2024-03-06T07:26:07.709208+0000","last_clean":"2024-03-06T07:26:07.709208+0000","last_became_active":"2024-03-06T07:26:07.705455+0000","last_became_peered":"2024-03-06T07:26:07.705455+0000","last_unstale":"2024-03-06T07:26:07.709208+0000","last_undegraded":"2024-03-06T07:26:07.709208+0000","last_fullsized":"2024-03-06T07:26:07.709208+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:29.340249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.708074+0000","last_change":"2024-03-06T07:26:07.706790+0000","last_active":"2024-03-06T07:26:07.708074+0000","last_peered":"2024-03-06T07:26:07.708074+0000","last_clean":"2024-03-06T07:26:07.708074+0000","last_became_active":"2024-03-06T07:26:07.706476+0000","last_became_peered":"2024-03-06T07:26:07.706476+0000","last_unstale":"2024-03-06T07:26:07.708074+0000","last_undegraded":"2024-03-06T07:26:07.708074+0000","last_fullsized":"2024-03-06T07:26:07.708074+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:01.894148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691377+0000","last_change":"2024-03-06T07:26:07.698630+0000","last_active":"2024-03-06T07:26:08.691377+0000","last_peered":"2024-03-06T07:26:08.691377+0000","last_clean":"2024-03-06T07:26:08.691377+0000","last_became_active":"2024-03-06T07:26:07.698334+0000","last_became_peered":"2024-03-06T07:26:07.698334+0000","last_unstale":"2024-03-06T07:26:08.691377+0000","last_undegraded":"2024-03-06T07:26:08.691377+0000","last_fullsized":"2024-03-06T07:26:08.691377+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:50:39.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.704391+0000","last_change":"2024-03-06T07:26:07.699219+0000","last_active":"2024-03-06T07:26:07.704391+0000","last_peered":"2024-03-06T07:26:07.704391+0000","last_clean":"2024-03-06T07:26:07.704391+0000","last_became_active":"2024-03-06T07:26:07.698897+0000","last_became_peered":"2024-03-06T07:26:07.698897+0000","last_unstale":"2024-03-06T07:26:07.704391+0000","last_undegraded":"2024-03-06T07:26:07.704391+0000","last_fullsized":"2024-03-06T07:26:07.704391+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:18.201061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699627+0000","last_change":"2024-03-06T07:26:07.697611+0000","last_active":"2024-03-06T07:26:07.699627+0000","last_peered":"2024-03-06T07:26:07.699627+0000","last_clean":"2024-03-06T07:26:07.699627+0000","last_became_active":"2024-03-06T07:26:07.697343+0000","last_became_peered":"2024-03-06T07:26:07.697343+0000","last_unstale":"2024-03-06T07:26:07.699627+0000","last_undegraded":"2024-03-06T07:26:07.699627+0000","last_fullsized":"2024-03-06T07:26:07.699627+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:17:00.303994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"150'4","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706677+0000","last_change":"2024-03-06T07:26:06.698500+0000","last_active":"2024-03-06T07:26:07.706677+0000","last_peered":"2024-03-06T07:26:07.706677+0000","last_clean":"2024-03-06T07:26:07.706677+0000","last_became_active":"2024-03-06T07:26:06.698299+0000","last_became_peered":"2024-03-06T07:26:06.698299+0000","last_unstale":"2024-03-06T07:26:07.706677+0000","last_undegraded":"2024-03-06T07:26:07.706677+0000","last_fullsized":"2024-03-06T07:26:07.706677+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:17.195623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700140+0000","last_change":"2024-03-06T07:26:07.698545+0000","last_active":"2024-03-06T07:26:08.700140+0000","last_peered":"2024-03-06T07:26:08.700140+0000","last_clean":"2024-03-06T07:26:08.700140+0000","last_became_active":"2024-03-06T07:26:07.698270+0000","last_became_peered":"2024-03-06T07:26:07.698270+0000","last_unstale":"2024-03-06T07:26:08.700140+0000","last_undegraded":"2024-03-06T07:26:08.700140+0000","last_fullsized":"2024-03-06T07:26:08.700140+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:59:25.247264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839549+0000","last_change":"2024-03-06T07:26:06.702569+0000","last_active":"2024-03-06T07:26:08.839549+0000","last_peered":"2024-03-06T07:26:08.839549+0000","last_clean":"2024-03-06T07:26:08.839549+0000","last_became_active":"2024-03-06T07:26:06.702304+0000","last_became_peered":"2024-03-06T07:26:06.702304+0000","last_unstale":"2024-03-06T07:26:08.839549+0000","last_undegraded":"2024-03-06T07:26:08.839549+0000","last_fullsized":"2024-03-06T07:26:08.839549+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:16.087949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839398+0000","last_change":"2024-03-06T07:26:07.701966+0000","last_active":"2024-03-06T07:26:08.839398+0000","last_peered":"2024-03-06T07:26:08.839398+0000","last_clean":"2024-03-06T07:26:08.839398+0000","last_became_active":"2024-03-06T07:26:07.701671+0000","last_became_peered":"2024-03-06T07:26:07.701671+0000","last_unstale":"2024-03-06T07:26:08.839398+0000","last_undegraded":"2024-03-06T07:26:08.839398+0000","last_fullsized":"2024-03-06T07:26:08.839398+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:03.437751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691667+0000","last_change":"2024-03-06T07:26:06.691461+0000","last_active":"2024-03-06T07:26:08.691667+0000","last_peered":"2024-03-06T07:26:08.691667+0000","last_clean":"2024-03-06T07:26:08.691667+0000","last_became_active":"2024-03-06T07:26:06.691169+0000","last_became_peered":"2024-03-06T07:26:06.691169+0000","last_unstale":"2024-03-06T07:26:08.691667+0000","last_undegraded":"2024-03-06T07:26:08.691667+0000","last_fullsized":"2024-03-06T07:26:08.691667+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.708821+0000","last_change":"2024-03-06T07:26:07.705527+0000","last_active":"2024-03-06T07:26:07.708821+0000","last_peered":"2024-03-06T07:26:07.708821+0000","last_clean":"2024-03-06T07:26:07.708821+0000","last_became_active":"2024-03-06T07:26:07.705214+0000","last_became_peered":"2024-03-06T07:26:07.705214+0000","last_unstale":"2024-03-06T07:26:07.708821+0000","last_undegraded":"2024-03-06T07:26:07.708821+0000","last_fullsized":"2024-03-06T07:26:07.708821+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705686+0000","last_change":"2024-03-06T07:26:06.689558+0000","last_active":"2024-03-06T07:26:07.705686+0000","last_peered":"2024-03-06T07:26:07.705686+0000","last_clean":"2024-03-06T07:26:07.705686+0000","last_became_active":"2024-03-06T07:26:06.689291+0000","last_became_peered":"2024-03-06T07:26:06.689291+0000","last_unstale":"2024-03-06T07:26:07.705686+0000","last_undegraded":"2024-03-06T07:26:07.705686+0000","last_fullsized":"2024-03-06T07:26:07.705686+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:59.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687862+0000","last_change":"2024-03-06T07:26:07.697671+0000","last_active":"2024-03-06T07:26:08.687862+0000","last_peered":"2024-03-06T07:26:08.687862+0000","last_clean":"2024-03-06T07:26:08.687862+0000","last_became_active":"2024-03-06T07:26:07.697366+0000","last_became_peered":"2024-03-06T07:26:07.697366+0000","last_unstale":"2024-03-06T07:26:08.687862+0000","last_undegraded":"2024-03-06T07:26:08.687862+0000","last_fullsized":"2024-03-06T07:26:08.687862+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687861+0000","last_change":"2024-03-06T07:26:06.700139+0000","last_active":"2024-03-06T07:26:08.687861+0000","last_peered":"2024-03-06T07:26:08.687861+0000","last_clean":"2024-03-06T07:26:08.687861+0000","last_became_active":"2024-03-06T07:26:06.699876+0000","last_became_peered":"2024-03-06T07:26:06.699876+0000","last_unstale":"2024-03-06T07:26:08.687861+0000","last_undegraded":"2024-03-06T07:26:08.687861+0000","last_fullsized":"2024-03-06T07:26:08.687861+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:22.510699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706891+0000","last_change":"2024-03-06T07:26:07.705724+0000","last_active":"2024-03-06T07:26:07.706891+0000","last_peered":"2024-03-06T07:26:07.706891+0000","last_clean":"2024-03-06T07:26:07.706891+0000","last_became_active":"2024-03-06T07:26:07.705378+0000","last_became_peered":"2024-03-06T07:26:07.705378+0000","last_unstale":"2024-03-06T07:26:07.706891+0000","last_undegraded":"2024-03-06T07:26:07.706891+0000","last_fullsized":"2024-03-06T07:26:07.706891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:08.158891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839489+0000","last_change":"2024-03-06T07:26:06.702328+0000","last_active":"2024-03-06T07:26:08.839489+0000","last_peered":"2024-03-06T07:26:08.839489+0000","last_clean":"2024-03-06T07:26:08.839489+0000","last_became_active":"2024-03-06T07:26:06.702009+0000","last_became_peered":"2024-03-06T07:26:06.702009+0000","last_unstale":"2024-03-06T07:26:08.839489+0000","last_undegraded":"2024-03-06T07:26:08.839489+0000","last_fullsized":"2024-03-06T07:26:08.839489+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:14.494596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705111+0000","last_change":"2024-03-06T07:26:07.698151+0000","last_active":"2024-03-06T07:26:07.705111+0000","last_peered":"2024-03-06T07:26:07.705111+0000","last_clean":"2024-03-06T07:26:07.705111+0000","last_became_active":"2024-03-06T07:26:07.697876+0000","last_became_peered":"2024-03-06T07:26:07.697876+0000","last_unstale":"2024-03-06T07:26:07.705111+0000","last_undegraded":"2024-03-06T07:26:07.705111+0000","last_fullsized":"2024-03-06T07:26:07.705111+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:35:55.125635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.687404+0000","last_change":"2024-03-06T07:26:06.700042+0000","last_active":"2024-03-06T07:26:07.687404+0000","last_peered":"2024-03-06T07:26:07.687404+0000","last_clean":"2024-03-06T07:26:07.687404+0000","last_became_active":"2024-03-06T07:26:06.699781+0000","last_became_peered":"2024-03-06T07:26:06.699781+0000","last_unstale":"2024-03-06T07:26:07.687404+0000","last_undegraded":"2024-03-06T07:26:07.687404+0000","last_fullsized":"2024-03-06T07:26:07.687404+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:13.960358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699416+0000","last_change":"2024-03-06T07:26:07.696784+0000","last_active":"2024-03-06T07:26:07.699416+0000","last_peered":"2024-03-06T07:26:07.699416+0000","last_clean":"2024-03-06T07:26:07.699416+0000","last_became_active":"2024-03-06T07:26:07.696515+0000","last_became_peered":"2024-03-06T07:26:07.696515+0000","last_unstale":"2024-03-06T07:26:07.699416+0000","last_undegraded":"2024-03-06T07:26:07.699416+0000","last_fullsized":"2024-03-06T07:26:07.699416+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:45.409316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687900+0000","last_change":"2024-03-06T07:26:06.693663+0000","last_active":"2024-03-06T07:26:08.687900+0000","last_peered":"2024-03-06T07:26:08.687900+0000","last_clean":"2024-03-06T07:26:08.687900+0000","last_became_active":"2024-03-06T07:26:06.693251+0000","last_became_peered":"2024-03-06T07:26:06.693251+0000","last_unstale":"2024-03-06T07:26:08.687900+0000","last_undegraded":"2024-03-06T07:26:08.687900+0000","last_fullsized":"2024-03-06T07:26:08.687900+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.699645+0000","last_change":"2024-03-06T07:26:07.698185+0000","last_active":"2024-03-06T07:26:07.699645+0000","last_peered":"2024-03-06T07:26:07.699645+0000","last_clean":"2024-03-06T07:26:07.699645+0000","last_became_active":"2024-03-06T07:26:07.696887+0000","last_became_peered":"2024-03-06T07:26:07.696887+0000","last_unstale":"2024-03-06T07:26:07.699645+0000","last_undegraded":"2024-03-06T07:26:07.699645+0000","last_fullsized":"2024-03-06T07:26:07.699645+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:50:39.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700493+0000","last_change":"2024-03-06T07:26:06.689754+0000","last_active":"2024-03-06T07:26:08.700493+0000","last_peered":"2024-03-06T07:26:08.700493+0000","last_clean":"2024-03-06T07:26:08.700493+0000","last_became_active":"2024-03-06T07:26:06.689386+0000","last_became_peered":"2024-03-06T07:26:06.689386+0000","last_unstale":"2024-03-06T07:26:08.700493+0000","last_undegraded":"2024-03-06T07:26:08.700493+0000","last_fullsized":"2024-03-06T07:26:08.700493+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700180+0000","last_change":"2024-03-06T07:26:07.697222+0000","last_active":"2024-03-06T07:26:08.700180+0000","last_peered":"2024-03-06T07:26:08.700180+0000","last_clean":"2024-03-06T07:26:08.700180+0000","last_became_active":"2024-03-06T07:26:07.696964+0000","last_became_peered":"2024-03-06T07:26:07.696964+0000","last_unstale":"2024-03-06T07:26:08.700180+0000","last_undegraded":"2024-03-06T07:26:08.700180+0000","last_fullsized":"2024-03-06T07:26:08.700180+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:59.699225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"150'3","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706357+0000","last_change":"2024-03-06T07:26:06.693324+0000","last_active":"2024-03-06T07:26:07.706357+0000","last_peered":"2024-03-06T07:26:07.706357+0000","last_clean":"2024-03-06T07:26:07.706357+0000","last_became_active":"2024-03-06T07:26:06.693047+0000","last_became_peered":"2024-03-06T07:26:06.693047+0000","last_unstale":"2024-03-06T07:26:07.706357+0000","last_undegraded":"2024-03-06T07:26:07.706357+0000","last_fullsized":"2024-03-06T07:26:07.706357+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:59.568199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706239+0000","last_change":"2024-03-06T07:26:07.704905+0000","last_active":"2024-03-06T07:26:07.706239+0000","last_peered":"2024-03-06T07:26:07.706239+0000","last_clean":"2024-03-06T07:26:07.706239+0000","last_became_active":"2024-03-06T07:26:07.704581+0000","last_became_peered":"2024-03-06T07:26:07.704581+0000","last_unstale":"2024-03-06T07:26:07.706239+0000","last_undegraded":"2024-03-06T07:26:07.706239+0000","last_fullsized":"2024-03-06T07:26:07.706239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:46.436137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"150'6","reported_seq":20,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700384+0000","last_change":"2024-03-06T07:26:06.686441+0000","last_active":"2024-03-06T07:26:08.700384+0000","last_peered":"2024-03-06T07:26:08.700384+0000","last_clean":"2024-03-06T07:26:08.700384+0000","last_became_active":"2024-03-06T07:26:06.686158+0000","last_became_peered":"2024-03-06T07:26:06.686158+0000","last_unstale":"2024-03-06T07:26:08.700384+0000","last_undegraded":"2024-03-06T07:26:08.700384+0000","last_fullsized":"2024-03-06T07:26:08.700384+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:05.262174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.702068+0000","last_change":"2024-03-06T07:26:07.698486+0000","last_active":"2024-03-06T07:26:07.702068+0000","last_peered":"2024-03-06T07:26:07.702068+0000","last_clean":"2024-03-06T07:26:07.702068+0000","last_became_active":"2024-03-06T07:26:07.698210+0000","last_became_peered":"2024-03-06T07:26:07.698210+0000","last_unstale":"2024-03-06T07:26:07.702068+0000","last_undegraded":"2024-03-06T07:26:07.702068+0000","last_fullsized":"2024-03-06T07:26:07.702068+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:38.877911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839584+0000","last_change":"2024-03-06T07:26:06.698989+0000","last_active":"2024-03-06T07:26:08.839584+0000","last_peered":"2024-03-06T07:26:08.839584+0000","last_clean":"2024-03-06T07:26:08.839584+0000","last_became_active":"2024-03-06T07:26:06.698656+0000","last_became_peered":"2024-03-06T07:26:06.698656+0000","last_unstale":"2024-03-06T07:26:08.839584+0000","last_undegraded":"2024-03-06T07:26:08.839584+0000","last_fullsized":"2024-03-06T07:26:08.839584+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688101+0000","last_change":"2024-03-06T07:26:07.709375+0000","last_active":"2024-03-06T07:26:08.688101+0000","last_peered":"2024-03-06T07:26:08.688101+0000","last_clean":"2024-03-06T07:26:08.688101+0000","last_became_active":"2024-03-06T07:26:07.709091+0000","last_became_peered":"2024-03-06T07:26:07.709091+0000","last_unstale":"2024-03-06T07:26:08.688101+0000","last_undegraded":"2024-03-06T07:26:08.688101+0000","last_fullsized":"2024-03-06T07:26:08.688101+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:41.934898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700175+0000","last_change":"2024-03-06T07:26:06.695622+0000","last_active":"2024-03-06T07:26:08.700175+0000","last_peered":"2024-03-06T07:26:08.700175+0000","last_clean":"2024-03-06T07:26:08.700175+0000","last_became_active":"2024-03-06T07:26:06.695369+0000","last_became_peered":"2024-03-06T07:26:06.695369+0000","last_unstale":"2024-03-06T07:26:08.700175+0000","last_undegraded":"2024-03-06T07:26:08.700175+0000","last_fullsized":"2024-03-06T07:26:08.700175+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:26.984542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691613+0000","last_change":"2024-03-06T07:26:07.698143+0000","last_active":"2024-03-06T07:26:08.691613+0000","last_peered":"2024-03-06T07:26:08.691613+0000","last_clean":"2024-03-06T07:26:08.691613+0000","last_became_active":"2024-03-06T07:26:07.697880+0000","last_became_peered":"2024-03-06T07:26:07.697880+0000","last_unstale":"2024-03-06T07:26:08.691613+0000","last_undegraded":"2024-03-06T07:26:08.691613+0000","last_fullsized":"2024-03-06T07:26:08.691613+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691525+0000","last_change":"2024-03-06T07:26:06.696386+0000","last_active":"2024-03-06T07:26:08.691525+0000","last_peered":"2024-03-06T07:26:08.691525+0000","last_clean":"2024-03-06T07:26:08.691525+0000","last_became_active":"2024-03-06T07:26:06.696108+0000","last_became_peered":"2024-03-06T07:26:06.696108+0000","last_unstale":"2024-03-06T07:26:08.691525+0000","last_undegraded":"2024-03-06T07:26:08.691525+0000","last_fullsized":"2024-03-06T07:26:08.691525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.696914+0000","last_change":"2024-03-06T07:26:07.694636+0000","last_active":"2024-03-06T07:26:07.696914+0000","last_peered":"2024-03-06T07:26:07.696914+0000","last_clean":"2024-03-06T07:26:07.696914+0000","last_became_active":"2024-03-06T07:26:07.694340+0000","last_became_peered":"2024-03-06T07:26:07.694340+0000","last_unstale":"2024-03-06T07:26:07.696914+0000","last_undegraded":"2024-03-06T07:26:07.696914+0000","last_fullsized":"2024-03-06T07:26:07.696914+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:39.420525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"150'8","reported_seq":22,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687810+0000","last_change":"2024-03-06T07:26:06.692364+0000","last_active":"2024-03-06T07:26:08.687810+0000","last_peered":"2024-03-06T07:26:08.687810+0000","last_clean":"2024-03-06T07:26:08.687810+0000","last_became_active":"2024-03-06T07:26:06.692119+0000","last_became_peered":"2024-03-06T07:26:06.692119+0000","last_unstale":"2024-03-06T07:26:08.687810+0000","last_undegraded":"2024-03-06T07:26:08.687810+0000","last_fullsized":"2024-03-06T07:26:08.687810+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687687+0000","last_change":"2024-03-06T07:26:07.698466+0000","last_active":"2024-03-06T07:26:08.687687+0000","last_peered":"2024-03-06T07:26:08.687687+0000","last_clean":"2024-03-06T07:26:08.687687+0000","last_became_active":"2024-03-06T07:26:07.697848+0000","last_became_peered":"2024-03-06T07:26:07.697848+0000","last_unstale":"2024-03-06T07:26:08.687687+0000","last_undegraded":"2024-03-06T07:26:08.687687+0000","last_fullsized":"2024-03-06T07:26:08.687687+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700582+0000","last_change":"2024-03-06T07:26:06.694226+0000","last_active":"2024-03-06T07:26:08.700582+0000","last_peered":"2024-03-06T07:26:08.700582+0000","last_clean":"2024-03-06T07:26:08.700582+0000","last_became_active":"2024-03-06T07:26:06.693966+0000","last_became_peered":"2024-03-06T07:26:06.693966+0000","last_unstale":"2024-03-06T07:26:08.700582+0000","last_undegraded":"2024-03-06T07:26:08.700582+0000","last_fullsized":"2024-03-06T07:26:08.700582+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:45.769621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.697346+0000","last_change":"2024-03-06T07:26:07.695629+0000","last_active":"2024-03-06T07:26:07.697346+0000","last_peered":"2024-03-06T07:26:07.697346+0000","last_clean":"2024-03-06T07:26:07.697346+0000","last_became_active":"2024-03-06T07:26:07.695358+0000","last_became_peered":"2024-03-06T07:26:07.695358+0000","last_unstale":"2024-03-06T07:26:07.697346+0000","last_undegraded":"2024-03-06T07:26:07.697346+0000","last_fullsized":"2024-03-06T07:26:07.697346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:12.120009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.688146+0000","last_change":"2024-03-06T07:26:06.698927+0000","last_active":"2024-03-06T07:26:08.688146+0000","last_peered":"2024-03-06T07:26:08.688146+0000","last_clean":"2024-03-06T07:26:08.688146+0000","last_became_active":"2024-03-06T07:26:06.698636+0000","last_became_peered":"2024-03-06T07:26:06.698636+0000","last_unstale":"2024-03-06T07:26:08.688146+0000","last_undegraded":"2024-03-06T07:26:08.688146+0000","last_fullsized":"2024-03-06T07:26:08.688146+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691742+0000","last_change":"2024-03-06T07:26:07.697409+0000","last_active":"2024-03-06T07:26:08.691742+0000","last_peered":"2024-03-06T07:26:08.691742+0000","last_clean":"2024-03-06T07:26:08.691742+0000","last_became_active":"2024-03-06T07:26:07.697084+0000","last_became_peered":"2024-03-06T07:26:07.697084+0000","last_unstale":"2024-03-06T07:26:08.691742+0000","last_undegraded":"2024-03-06T07:26:08.691742+0000","last_fullsized":"2024-03-06T07:26:08.691742+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839724+0000","last_change":"2024-03-06T07:26:06.695033+0000","last_active":"2024-03-06T07:26:08.839724+0000","last_peered":"2024-03-06T07:26:08.839724+0000","last_clean":"2024-03-06T07:26:08.839724+0000","last_became_active":"2024-03-06T07:26:06.694763+0000","last_became_peered":"2024-03-06T07:26:06.694763+0000","last_unstale":"2024-03-06T07:26:08.839724+0000","last_undegraded":"2024-03-06T07:26:08.839724+0000","last_fullsized":"2024-03-06T07:26:08.839724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:29.440623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691584+0000","last_change":"2024-03-06T07:26:07.706589+0000","last_active":"2024-03-06T07:26:08.691584+0000","last_peered":"2024-03-06T07:26:08.691584+0000","last_clean":"2024-03-06T07:26:08.691584+0000","last_became_active":"2024-03-06T07:26:07.706305+0000","last_became_peered":"2024-03-06T07:26:07.706305+0000","last_unstale":"2024-03-06T07:26:08.691584+0000","last_undegraded":"2024-03-06T07:26:08.691584+0000","last_fullsized":"2024-03-06T07:26:08.691584+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687999+0000","last_change":"2024-03-06T07:26:06.697335+0000","last_active":"2024-03-06T07:26:08.687999+0000","last_peered":"2024-03-06T07:26:08.687999+0000","last_clean":"2024-03-06T07:26:08.687999+0000","last_became_active":"2024-03-06T07:26:06.697028+0000","last_became_peered":"2024-03-06T07:26:06.697028+0000","last_unstale":"2024-03-06T07:26:08.687999+0000","last_undegraded":"2024-03-06T07:26:08.687999+0000","last_fullsized":"2024-03-06T07:26:08.687999+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700431+0000","last_change":"2024-03-06T07:26:07.692326+0000","last_active":"2024-03-06T07:26:08.700431+0000","last_peered":"2024-03-06T07:26:08.700431+0000","last_clean":"2024-03-06T07:26:08.700431+0000","last_became_active":"2024-03-06T07:26:07.692063+0000","last_became_peered":"2024-03-06T07:26:07.692063+0000","last_unstale":"2024-03-06T07:26:08.700431+0000","last_undegraded":"2024-03-06T07:26:08.700431+0000","last_fullsized":"2024-03-06T07:26:08.700431+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:42.601237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839694+0000","last_change":"2024-03-06T07:26:06.696078+0000","last_active":"2024-03-06T07:26:08.839694+0000","last_peered":"2024-03-06T07:26:08.839694+0000","last_clean":"2024-03-06T07:26:08.839694+0000","last_became_active":"2024-03-06T07:26:06.695834+0000","last_became_peered":"2024-03-06T07:26:06.695834+0000","last_unstale":"2024-03-06T07:26:08.839694+0000","last_undegraded":"2024-03-06T07:26:08.839694+0000","last_fullsized":"2024-03-06T07:26:08.839694+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839725+0000","last_change":"2024-03-06T07:26:07.713113+0000","last_active":"2024-03-06T07:26:08.839725+0000","last_peered":"2024-03-06T07:26:08.839725+0000","last_clean":"2024-03-06T07:26:08.839725+0000","last_became_active":"2024-03-06T07:26:07.712610+0000","last_became_peered":"2024-03-06T07:26:07.712610+0000","last_unstale":"2024-03-06T07:26:08.839725+0000","last_undegraded":"2024-03-06T07:26:08.839725+0000","last_fullsized":"2024-03-06T07:26:08.839725+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:03.957417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691378+0000","last_change":"2024-03-06T07:26:06.692781+0000","last_active":"2024-03-06T07:26:08.691378+0000","last_peered":"2024-03-06T07:26:08.691378+0000","last_clean":"2024-03-06T07:26:08.691378+0000","last_became_active":"2024-03-06T07:26:06.691992+0000","last_became_peered":"2024-03-06T07:26:06.691992+0000","last_unstale":"2024-03-06T07:26:08.691378+0000","last_undegraded":"2024-03-06T07:26:08.691378+0000","last_fullsized":"2024-03-06T07:26:08.691378+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839553+0000","last_change":"2024-03-06T07:26:07.713547+0000","last_active":"2024-03-06T07:26:08.839553+0000","last_peered":"2024-03-06T07:26:08.839553+0000","last_clean":"2024-03-06T07:26:08.839553+0000","last_became_active":"2024-03-06T07:26:07.712950+0000","last_became_peered":"2024-03-06T07:26:07.712950+0000","last_unstale":"2024-03-06T07:26:08.839553+0000","last_undegraded":"2024-03-06T07:26:08.839553+0000","last_fullsized":"2024-03-06T07:26:08.839553+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:00.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.687580+0000","last_change":"2024-03-06T07:26:06.695426+0000","last_active":"2024-03-06T07:26:07.687580+0000","last_peered":"2024-03-06T07:26:07.687580+0000","last_clean":"2024-03-06T07:26:07.687580+0000","last_became_active":"2024-03-06T07:26:06.695162+0000","last_became_peered":"2024-03-06T07:26:06.695162+0000","last_unstale":"2024-03-06T07:26:07.687580+0000","last_undegraded":"2024-03-06T07:26:07.687580+0000","last_fullsized":"2024-03-06T07:26:07.687580+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:30.485289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.696913+0000","last_change":"2024-03-06T07:26:07.691244+0000","last_active":"2024-03-06T07:26:07.696913+0000","last_peered":"2024-03-06T07:26:07.696913+0000","last_clean":"2024-03-06T07:26:07.696913+0000","last_became_active":"2024-03-06T07:26:07.690929+0000","last_became_peered":"2024-03-06T07:26:07.690929+0000","last_unstale":"2024-03-06T07:26:07.696913+0000","last_undegraded":"2024-03-06T07:26:07.696913+0000","last_fullsized":"2024-03-06T07:26:07.696913+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:36.240577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.685833+0000","last_change":"2024-03-06T07:26:06.694068+0000","last_active":"2024-03-06T07:26:07.685833+0000","last_peered":"2024-03-06T07:26:07.685833+0000","last_clean":"2024-03-06T07:26:07.685833+0000","last_became_active":"2024-03-06T07:26:06.693777+0000","last_became_peered":"2024-03-06T07:26:06.693777+0000","last_unstale":"2024-03-06T07:26:07.685833+0000","last_undegraded":"2024-03-06T07:26:07.685833+0000","last_fullsized":"2024-03-06T07:26:07.685833+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:06.186255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691245+0000","last_change":"2024-03-06T07:26:07.696197+0000","last_active":"2024-03-06T07:26:08.691245+0000","last_peered":"2024-03-06T07:26:08.691245+0000","last_clean":"2024-03-06T07:26:08.691245+0000","last_became_active":"2024-03-06T07:26:07.695895+0000","last_became_peered":"2024-03-06T07:26:07.695895+0000","last_unstale":"2024-03-06T07:26:08.691245+0000","last_undegraded":"2024-03-06T07:26:08.691245+0000","last_fullsized":"2024-03-06T07:26:08.691245+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:02.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839696+0000","last_change":"2024-03-06T07:26:06.701125+0000","last_active":"2024-03-06T07:26:08.839696+0000","last_peered":"2024-03-06T07:26:08.839696+0000","last_clean":"2024-03-06T07:26:08.839696+0000","last_became_active":"2024-03-06T07:26:06.700819+0000","last_became_peered":"2024-03-06T07:26:06.700819+0000","last_unstale":"2024-03-06T07:26:08.839696+0000","last_undegraded":"2024-03-06T07:26:08.839696+0000","last_fullsized":"2024-03-06T07:26:08.839696+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:54.995466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687769+0000","last_change":"2024-03-06T07:26:07.701964+0000","last_active":"2024-03-06T07:26:08.687769+0000","last_peered":"2024-03-06T07:26:08.687769+0000","last_clean":"2024-03-06T07:26:08.687769+0000","last_became_active":"2024-03-06T07:26:07.701660+0000","last_became_peered":"2024-03-06T07:26:07.701660+0000","last_unstale":"2024-03-06T07:26:08.687769+0000","last_undegraded":"2024-03-06T07:26:08.687769+0000","last_fullsized":"2024-03-06T07:26:08.687769+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"151'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.701623+0000","last_change":"2024-03-06T07:26:06.693341+0000","last_active":"2024-03-06T07:26:08.701623+0000","last_peered":"2024-03-06T07:26:08.701623+0000","last_clean":"2024-03-06T07:26:08.701623+0000","last_became_active":"2024-03-06T07:26:06.693014+0000","last_became_peered":"2024-03-06T07:26:06.693014+0000","last_unstale":"2024-03-06T07:26:08.701623+0000","last_undegraded":"2024-03-06T07:26:08.701623+0000","last_fullsized":"2024-03-06T07:26:08.701623+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:03.887560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.703444+0000","last_change":"2024-03-06T07:26:07.697077+0000","last_active":"2024-03-06T07:26:07.703444+0000","last_peered":"2024-03-06T07:26:07.703444+0000","last_clean":"2024-03-06T07:26:07.703444+0000","last_became_active":"2024-03-06T07:26:07.695444+0000","last_became_peered":"2024-03-06T07:26:07.695444+0000","last_unstale":"2024-03-06T07:26:07.703444+0000","last_undegraded":"2024-03-06T07:26:07.703444+0000","last_fullsized":"2024-03-06T07:26:07.703444+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:14.350502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700450+0000","last_change":"2024-03-06T07:26:06.689881+0000","last_active":"2024-03-06T07:26:08.700450+0000","last_peered":"2024-03-06T07:26:08.700450+0000","last_clean":"2024-03-06T07:26:08.700450+0000","last_became_active":"2024-03-06T07:26:06.689604+0000","last_became_peered":"2024-03-06T07:26:06.689604+0000","last_unstale":"2024-03-06T07:26:08.700450+0000","last_undegraded":"2024-03-06T07:26:08.700450+0000","last_fullsized":"2024-03-06T07:26:08.700450+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.687869+0000","last_change":"2024-03-06T07:26:07.700732+0000","last_active":"2024-03-06T07:26:08.687869+0000","last_peered":"2024-03-06T07:26:08.687869+0000","last_clean":"2024-03-06T07:26:08.687869+0000","last_became_active":"2024-03-06T07:26:07.700382+0000","last_became_peered":"2024-03-06T07:26:07.700382+0000","last_unstale":"2024-03-06T07:26:08.687869+0000","last_undegraded":"2024-03-06T07:26:08.687869+0000","last_fullsized":"2024-03-06T07:26:08.687869+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:29.340249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.686330+0000","last_change":"2024-03-06T07:26:06.693374+0000","last_active":"2024-03-06T07:26:07.686330+0000","last_peered":"2024-03-06T07:26:07.686330+0000","last_clean":"2024-03-06T07:26:07.686330+0000","last_became_active":"2024-03-06T07:26:06.693113+0000","last_became_peered":"2024-03-06T07:26:06.693113+0000","last_unstale":"2024-03-06T07:26:07.686330+0000","last_undegraded":"2024-03-06T07:26:07.686330+0000","last_fullsized":"2024-03-06T07:26:07.686330+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:55.760111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.705508+0000","last_change":"2024-03-06T07:26:07.701237+0000","last_active":"2024-03-06T07:26:07.705508+0000","last_peered":"2024-03-06T07:26:07.705508+0000","last_clean":"2024-03-06T07:26:07.705508+0000","last_became_active":"2024-03-06T07:26:07.700717+0000","last_became_peered":"2024-03-06T07:26:07.700717+0000","last_unstale":"2024-03-06T07:26:07.705508+0000","last_undegraded":"2024-03-06T07:26:07.705508+0000","last_fullsized":"2024-03-06T07:26:07.705508+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:57.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.684455+0000","last_change":"2024-03-06T07:26:06.695137+0000","last_active":"2024-03-06T07:26:07.684455+0000","last_peered":"2024-03-06T07:26:07.684455+0000","last_clean":"2024-03-06T07:26:07.684455+0000","last_became_active":"2024-03-06T07:26:06.694881+0000","last_became_peered":"2024-03-06T07:26:06.694881+0000","last_unstale":"2024-03-06T07:26:07.684455+0000","last_undegraded":"2024-03-06T07:26:07.684455+0000","last_fullsized":"2024-03-06T07:26:07.684455+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:37.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706049+0000","last_change":"2024-03-06T07:26:07.705151+0000","last_active":"2024-03-06T07:26:07.706049+0000","last_peered":"2024-03-06T07:26:07.706049+0000","last_clean":"2024-03-06T07:26:07.706049+0000","last_became_active":"2024-03-06T07:26:07.704856+0000","last_became_peered":"2024-03-06T07:26:07.704856+0000","last_unstale":"2024-03-06T07:26:07.706049+0000","last_undegraded":"2024-03-06T07:26:07.706049+0000","last_fullsized":"2024-03-06T07:26:07.706049+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700271+0000","last_change":"2024-03-06T07:26:06.684537+0000","last_active":"2024-03-06T07:26:08.700271+0000","last_peered":"2024-03-06T07:26:08.700271+0000","last_clean":"2024-03-06T07:26:08.700271+0000","last_became_active":"2024-03-06T07:26:06.683851+0000","last_became_peered":"2024-03-06T07:26:06.683851+0000","last_unstale":"2024-03-06T07:26:08.700271+0000","last_undegraded":"2024-03-06T07:26:08.700271+0000","last_fullsized":"2024-03-06T07:26:08.700271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:07.304752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700491+0000","last_change":"2024-03-06T07:26:07.691230+0000","last_active":"2024-03-06T07:26:08.700491+0000","last_peered":"2024-03-06T07:26:08.700491+0000","last_clean":"2024-03-06T07:26:08.700491+0000","last_became_active":"2024-03-06T07:26:07.690936+0000","last_became_peered":"2024-03-06T07:26:07.690936+0000","last_unstale":"2024-03-06T07:26:08.700491+0000","last_undegraded":"2024-03-06T07:26:08.700491+0000","last_fullsized":"2024-03-06T07:26:08.700491+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:09.494792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.691668+0000","last_change":"2024-03-06T07:26:06.692326+0000","last_active":"2024-03-06T07:26:08.691668+0000","last_peered":"2024-03-06T07:26:08.691668+0000","last_clean":"2024-03-06T07:26:08.691668+0000","last_became_active":"2024-03-06T07:26:06.691972+0000","last_became_peered":"2024-03-06T07:26:06.691972+0000","last_unstale":"2024-03-06T07:26:08.691668+0000","last_undegraded":"2024-03-06T07:26:08.691668+0000","last_fullsized":"2024-03-06T07:26:08.691668+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.706117+0000","last_change":"2024-03-06T07:26:07.704775+0000","last_active":"2024-03-06T07:26:07.706117+0000","last_peered":"2024-03-06T07:26:07.706117+0000","last_clean":"2024-03-06T07:26:07.706117+0000","last_became_active":"2024-03-06T07:26:07.704502+0000","last_became_peered":"2024-03-06T07:26:07.704502+0000","last_unstale":"2024-03-06T07:26:07.706117+0000","last_undegraded":"2024-03-06T07:26:07.706117+0000","last_fullsized":"2024-03-06T07:26:07.706117+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:02.129392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.839177+0000","last_change":"2024-03-06T07:26:06.696947+0000","last_active":"2024-03-06T07:26:08.839177+0000","last_peered":"2024-03-06T07:26:08.839177+0000","last_clean":"2024-03-06T07:26:08.839177+0000","last_became_active":"2024-03-06T07:26:06.696715+0000","last_became_peered":"2024-03-06T07:26:06.696715+0000","last_unstale":"2024-03-06T07:26:08.839177+0000","last_undegraded":"2024-03-06T07:26:08.839177+0000","last_fullsized":"2024-03-06T07:26:08.839177+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:40.929460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.701853+0000","last_change":"2024-03-06T07:26:07.699110+0000","last_active":"2024-03-06T07:26:07.701853+0000","last_peered":"2024-03-06T07:26:07.701853+0000","last_clean":"2024-03-06T07:26:07.701853+0000","last_became_active":"2024-03-06T07:26:07.698799+0000","last_became_peered":"2024-03-06T07:26:07.698799+0000","last_unstale":"2024-03-06T07:26:07.701853+0000","last_undegraded":"2024-03-06T07:26:07.701853+0000","last_fullsized":"2024-03-06T07:26:07.701853+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-03-06T07:26:08.700485+0000","last_change":"2024-03-06T07:26:06.691133+0000","last_active":"2024-03-06T07:26:08.700485+0000","last_peered":"2024-03-06T07:26:08.700485+0000","last_clean":"2024-03-06T07:26:08.700485+0000","last_became_active":"2024-03-06T07:26:06.690687+0000","last_became_peered":"2024-03-06T07:26:06.690687+0000","last_unstale":"2024-03-06T07:26:08.700485+0000","last_undegraded":"2024-03-06T07:26:08.700485+0000","last_fullsized":"2024-03-06T07:26:08.700485+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:08.352394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:26:07.685888+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:26:07.685888+0000","last_peered":"2024-03-06T07:26:07.685888+0000","last_clean":"2024-03-06T07:26:07.685888+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:26:07.685888+0000","last_undegraded":"2024-03-06T07:26:07.685888+0000","last_fullsized":"2024-03-06T07:26:07.685888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1953792,"data_stored":1502417,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1888256,"data_stored":912049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1908736,"data_stored":918865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1904640,"data_stored":912813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1921024,"data_stored":920957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":912049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1912832,"data_stored":920353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1974272,"data_stored":1509213,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:12.087 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:26:12.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:26:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.230+0000 7f106e639640 1 Processor -- start 2024-03-06T07:26:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.230+0000 7f106e639640 1 -- start start 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.230+0000 7f106e639640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f10680a8290 0x7f10681539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.230+0000 7f106e639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f1068072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.230+0000 7f106e639640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10680c62a0 con 0x7f10680c4740 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.230+0000 7f106e639640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1068153f00 con 0x7f10680a8290 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.230+0000 7f106e639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10680c4c00 con 0x7f10680c69a0 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f1067fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f10680a8290 0x7f10681539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f1067fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f10680a8290 0x7f10681539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44790/0 (socket says 172.21.15.26:44790) 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f1067fff640 1 -- 172.21.15.26:0/3268523587 learned_addr learned my addr 172.21.15.26:0/3268523587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f106cbd6640 1 --2- 172.21.15.26:0/3268523587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f1068072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f1067fff640 1 -- 172.21.15.26:0/3268523587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 msgr2=0x7f1068072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f1067fff640 1 --2- 172.21.15.26:0/3268523587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f1068072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f1067fff640 1 -- 172.21.15.26:0/3268523587 >> v1:172.21.15.117:6789/0 conn(0x7f10680c4740 legacy=0x7f10680701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.231+0000 7f1067fff640 1 -- 172.21.15.26:0/3268523587 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10680c4ee0 con 0x7f10680a8290 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f1067fff640 1 --2- 172.21.15.26:0/3268523587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680a8290 0x7f10681539c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1058009870 tx=0x7f105802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876cc9435071ccd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f1066ffd640 1 -- 172.21.15.26:0/3268523587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10580317c0 con 0x7f10680a8290 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f1066ffd640 1 -- 172.21.15.26:0/3268523587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058037070 con 0x7f10680a8290 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f1066ffd640 1 -- 172.21.15.26:0/3268523587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058031b80 con 0x7f10680a8290 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 -- 172.21.15.26:0/3268523587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680a8290 msgr2=0x7f10681539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 --2- 172.21.15.26:0/3268523587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680a8290 0x7f10681539c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1058009870 tx=0x7f105802b860 comp rx=0 tx=0).stop 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 -- 172.21.15.26:0/3268523587 shutdown_connections 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 --2- 172.21.15.26:0/3268523587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f1068072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 --2- 172.21.15.26:0/3268523587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680a8290 0x7f10681539c0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 -- 172.21.15.26:0/3268523587 >> 172.21.15.26:0/3268523587 conn(0x7f10680c5560 msgr2=0x7f10680c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 -- 172.21.15.26:0/3268523587 shutdown_connections 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.232+0000 7f106e639640 1 -- 172.21.15.26:0/3268523587 wait complete. 2024-03-06T07:26:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.233+0000 7f106e639640 1 Processor -- start 2024-03-06T07:26:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f106e639640 1 -- start start 2024-03-06T07:26:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f106e639640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10680a8290 0x7f106814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f106e639640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680c4740 0x7f10680c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f106e639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f10681ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f106e639640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680c6e30 con 0x7f10680c4740 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f106e639640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106806fc80 con 0x7f10680a8290 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f106e639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1068072bc0 con 0x7f10680c69a0 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f10677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680c4740 0x7f10680c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f10677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680c4740 0x7f10680c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44796/0 (socket says 172.21.15.26:44796) 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.234+0000 7f10677fe640 1 -- 172.21.15.26:0/1124355412 learned_addr learned my addr 172.21.15.26:0/1124355412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f106cbd6640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f10681ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f1067fff640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10680a8290 0x7f106814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f10677fe640 1 -- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 msgr2=0x7f10681ee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f10677fe640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f10681ee380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f10677fe640 1 -- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10680a8290 msgr2=0x7f106814dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f10677fe640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10680a8290 0x7f106814dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f10677fe640 1 -- 172.21.15.26:0/1124355412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068156580 con 0x7f10680c4740 2024-03-06T07:26:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f1067fff640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10680a8290 0x7f106814dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f10677fe640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680c4740 0x7f10680c1e90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c400 tx=0x7f105c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f1064ff9640 1 -- 172.21.15.26:0/1124355412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0052d0 con 0x7f10680c4740 2024-03-06T07:26:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f1064ff9640 1 -- 172.21.15.26:0/1124355412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c005490 con 0x7f10680c4740 2024-03-06T07:26:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1068156820 con 0x7f10680c4740 2024-03-06T07:26:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f1064ff9640 1 -- 172.21.15.26:0/1124355412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c007260 con 0x7f10680c4740 2024-03-06T07:26:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.235+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1068156de0 con 0x7f10680c4740 2024-03-06T07:26:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.236+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106804f330 con 0x7f10680c4740 2024-03-06T07:26:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.237+0000 7f1064ff9640 1 -- 172.21.15.26:0/1124355412 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f105c007400 con 0x7f10680c4740 2024-03-06T07:26:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.237+0000 7f1064ff9640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105406b4d0 0x7f105406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.237+0000 7f1064ff9640 1 -- 172.21.15.26:0/1124355412 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f105c0898e0 con 0x7f10680c4740 2024-03-06T07:26:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.237+0000 7f1067fff640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105406b4d0 0x7f105406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.238+0000 7f1067fff640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105406b4d0 0x7f105406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1058036000 tx=0x7f1058008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.241+0000 7f1064ff9640 1 -- 172.21.15.26:0/1124355412 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f105c0585d0 con 0x7f10680c4740 2024-03-06T07:26:12.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:12.335 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:12.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.491+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f10680c8480 con 0x7f10680c4740 2024-03-06T07:26:12.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.493+0000 7f1064ff9640 1 -- 172.21.15.26:0/1124355412 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f105c057ed0 con 0x7f10680c4740 2024-03-06T07:26:12.495 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:26:12.496 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:26:12.496 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:26:12.496 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:26:12.496 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:26:12.496 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:26:12.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.495+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105406b4d0 msgr2=0x7f105406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.495+0000 7f106e639640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105406b4d0 0x7f105406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1058036000 tx=0x7f1058008030 comp rx=0 tx=0).stop 2024-03-06T07:26:12.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.495+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680c4740 msgr2=0x7f10680c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.495+0000 7f106e639640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680c4740 0x7f10680c1e90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c400 tx=0x7f105c00c8c0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 shutdown_connections 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f105406b4d0 0x7f105406d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10680c69a0 0x7f10681ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680c4740 0x7f10680c1e90 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 --2- 172.21.15.26:0/1124355412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10680a8290 0x7f106814dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 >> 172.21.15.26:0/1124355412 conn(0x7f10680c5560 msgr2=0x7f10681523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 shutdown_connections 2024-03-06T07:26:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.496+0000 7f106e639640 1 -- 172.21.15.26:0/1124355412 wait complete. 2024-03-06T07:26:12.509 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.608+0000 7fef7dd3e640 1 Processor -- start 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.608+0000 7fef7dd3e640 1 -- start start 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef7dd3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef78153990 0x7fef78156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef7dd3e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef780acc00 0x7fef780c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef7dd3e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef78070aa0 con 0x7fef781506b0 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef7dd3e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef78153e10 con 0x7fef780acc00 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef7dd3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef78156450 con 0x7fef78153990 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef780acc00 0x7fef780c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef780acc00 0x7fef780c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44802/0 (socket says 172.21.15.26:44802) 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.609+0000 7fef77fff640 1 -- 172.21.15.26:0/2980422496 learned_addr learned my addr 172.21.15.26:0/2980422496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.610+0000 7fef77fff640 1 -- 172.21.15.26:0/2980422496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef78153990 msgr2=0x7fef78156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.610+0000 7fef76ffd640 1 --2- 172.21.15.26:0/2980422496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef78153990 0x7fef78156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.610+0000 7fef77fff640 1 --2- 172.21.15.26:0/2980422496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef78153990 0x7fef78156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.610+0000 7fef77fff640 1 -- 172.21.15.26:0/2980422496 >> v1:172.21.15.117:6789/0 conn(0x7fef781506b0 legacy=0x7fef780702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.610+0000 7fef77fff640 1 -- 172.21.15.26:0/2980422496 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78070d80 con 0x7fef780acc00 2024-03-06T07:26:12.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.611+0000 7fef77fff640 1 --2- 172.21.15.26:0/2980422496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef780acc00 0x7fef780c54a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7739f48720c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.611+0000 7fef767fc640 1 -- 172.21.15.26:0/2980422496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70035020 con 0x7fef780acc00 2024-03-06T07:26:12.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.611+0000 7fef767fc640 1 -- 172.21.15.26:0/2980422496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70031640 con 0x7fef780acc00 2024-03-06T07:26:12.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.611+0000 7fef767fc640 1 -- 172.21.15.26:0/2980422496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7002d040 con 0x7fef780acc00 2024-03-06T07:26:12.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.611+0000 7fef7dd3e640 1 -- 172.21.15.26:0/2980422496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef780acc00 msgr2=0x7fef780c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.611+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/2980422496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef780acc00 0x7fef780c54a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.612+0000 7fef7dd3e640 1 -- 172.21.15.26:0/2980422496 shutdown_connections 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.612+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/2980422496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef780acc00 0x7fef780c54a0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.612+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/2980422496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef78153990 0x7fef78156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.612+0000 7fef7dd3e640 1 -- 172.21.15.26:0/2980422496 >> 172.21.15.26:0/2980422496 conn(0x7fef7806b1a0 msgr2=0x7fef7806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.612+0000 7fef7dd3e640 1 -- 172.21.15.26:0/2980422496 shutdown_connections 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.612+0000 7fef7dd3e640 1 -- 172.21.15.26:0/2980422496 wait complete. 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.613+0000 7fef7dd3e640 1 Processor -- start 2024-03-06T07:26:12.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.613+0000 7fef7dd3e640 1 -- start start 2024-03-06T07:26:12.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.613+0000 7fef7dd3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef780acc00 0x7fef780c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef7dd3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef781506b0 0x7fef780c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef7dd3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78153990 0x7fef781ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef7dd3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780ad070 con 0x7fef78153990 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef7dd3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78156a10 con 0x7fef780acc00 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef7dd3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780ad1f0 con 0x7fef781506b0 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78153990 0x7fef781ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78153990 0x7fef781ee410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44814/0 (socket says 172.21.15.26:44814) 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 -- 172.21.15.26:0/3834614047 learned_addr learned my addr 172.21.15.26:0/3834614047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef777fe640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef780acc00 0x7fef780c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef76ffd640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef781506b0 0x7fef780c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 -- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef781506b0 msgr2=0x7fef780c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef781506b0 0x7fef780c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 -- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef780acc00 msgr2=0x7fef780c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef780acc00 0x7fef780c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 -- 172.21.15.26:0/3834614047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780c4ed0 con 0x7fef78153990 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef77fff640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78153990 0x7fef781ee410 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fef7002b930 tx=0x7fef700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef5ffff640 1 -- 172.21.15.26:0/3834614047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70040070 con 0x7fef78153990 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef5ffff640 1 -- 172.21.15.26:0/3834614047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70039c10 con 0x7fef78153990 2024-03-06T07:26:12.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef781eb1b0 con 0x7fef78153990 2024-03-06T07:26:12.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef5ffff640 1 -- 172.21.15.26:0/3834614047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70035040 con 0x7fef78153990 2024-03-06T07:26:12.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.614+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef780c22a0 con 0x7fef78153990 2024-03-06T07:26:12.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.615+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7806b920 con 0x7fef78153990 2024-03-06T07:26:12.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.616+0000 7fef5ffff640 1 -- 172.21.15.26:0/3834614047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef70038020 con 0x7fef78153990 2024-03-06T07:26:12.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.616+0000 7fef5ffff640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fef6406b4d0 0x7fef6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.616+0000 7fef5ffff640 1 -- 172.21.15.26:0/3834614047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fef700abd60 con 0x7fef78153990 2024-03-06T07:26:12.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.617+0000 7fef777fe640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fef6406b4d0 0x7fef6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.617+0000 7fef777fe640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fef6406b4d0 0x7fef6406d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef780c25e0 tx=0x7fef68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.621+0000 7fef5ffff640 1 -- 172.21.15.26:0/3834614047 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fef7806b920 con 0x7fef78153990 2024-03-06T07:26:12.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.920+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef7804f330 con 0x7fef78153990 2024-03-06T07:26:12.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.920+0000 7fef5ffff640 1 -- 172.21.15.26:0/3834614047 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+4987 (secure 0 0 0) 0x7fef7007aae0 con 0x7fef78153990 2024-03-06T07:26:12.923 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:12.923 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":138,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":8293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:10.089399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8793":{"gid":8793,"name":"a","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6833/3029467671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3029467671},{"type":"v1","addr":"172.21.15.26:6833","nonce":3029467671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8793,"qdb_cluster":[8793]},"id":11}]} 2024-03-06T07:26:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 138 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fef6406b4d0 msgr2=0x7fef6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fef6406b4d0 0x7fef6406d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef780c25e0 tx=0x7fef68009040 comp rx=0 tx=0).stop 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78153990 msgr2=0x7fef781ee410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78153990 0x7fef781ee410 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fef7002b930 tx=0x7fef700046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 shutdown_connections 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fef6406b4d0 0x7fef6406d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78153990 0x7fef781ee410 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef781506b0 0x7fef780c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 --2- 172.21.15.26:0/3834614047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef780acc00 0x7fef780c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 >> 172.21.15.26:0/3834614047 conn(0x7fef7806b1a0 msgr2=0x7fef7814d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 shutdown_connections 2024-03-06T07:26:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:12.922+0000 7fef7dd3e640 1 -- 172.21.15.26:0/3834614047 wait complete. 2024-03-06T07:26:12.983 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-03-06T07:26:12.981+0000 7ff6ea5a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:13.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 138, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:07.071721+0000', 'modified': '2024-03-06T07:26:10.089399+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8793}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8793': {'gid': 8793, 'name': 'a', 'rank': 0, 'incarnation': 135, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6833/3029467671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3029467671}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3029467671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8793, 'qdb_cluster': [8793]} 2024-03-06T07:26:13.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:26:13.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:26:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.439+0000 7f080600e640 1 Processor -- start 2024-03-06T07:26:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.439+0000 7f080600e640 1 -- start start 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f080600e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08000c69f0 0x7f08000c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f080600e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f08000acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f080600e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0800155130 con 0x7f08000ad2b0 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f080600e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0800155440 con 0x7f08000c69f0 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f080600e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000c5d50 con 0x7f08000c8950 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07ff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08000c69f0 0x7f08000c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07fffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f08000ad2b0 0x7f08001558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54966/0 (socket says 172.21.15.26:54966) 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07fffff640 1 -- 172.21.15.26:0/3242011379 learned_addr learned my addr 172.21.15.26:0/3242011379 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07ff7fe640 1 -- 172.21.15.26:0/3242011379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 msgr2=0x7f08000acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07feffd640 1 --2- 172.21.15.26:0/3242011379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f08000acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3242011379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f08000acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07fe7fc640 1 -- 172.21.15.26:0/3242011379 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394680232 0 0) 0x7f07f8001110 con 0x7f08000ad2b0 2024-03-06T07:26:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07ff7fe640 1 -- 172.21.15.26:0/3242011379 >> v1:172.21.15.26:6789/0 conn(0x7f08000ad2b0 legacy=0x7f08001558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.440+0000 7f07ff7fe640 1 -- 172.21.15.26:0/3242011379 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000c6030 con 0x7f08000c69f0 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.441+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3242011379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000c69f0 0x7f08000c6dd0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009d90 tx=0x7f07f002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5311b7799403916d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.441+0000 7f07fe7fc640 1 -- 172.21.15.26:0/3242011379 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0039070 con 0x7f08000c69f0 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.441+0000 7f07fe7fc640 1 -- 172.21.15.26:0/3242011379 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f002cbc0 con 0x7f08000c69f0 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.441+0000 7f07fe7fc640 1 -- 172.21.15.26:0/3242011379 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0034440 con 0x7f08000c69f0 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 -- 172.21.15.26:0/3242011379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000c69f0 msgr2=0x7f08000c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 --2- 172.21.15.26:0/3242011379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000c69f0 0x7f08000c6dd0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009d90 tx=0x7f07f002be10 comp rx=0 tx=0).stop 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 -- 172.21.15.26:0/3242011379 shutdown_connections 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 --2- 172.21.15.26:0/3242011379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f08000acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 --2- 172.21.15.26:0/3242011379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000c69f0 0x7f08000c6dd0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 -- 172.21.15.26:0/3242011379 >> 172.21.15.26:0/3242011379 conn(0x7f08000cae70 msgr2=0x7f08000cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 -- 172.21.15.26:0/3242011379 shutdown_connections 2024-03-06T07:26:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.442+0000 7f080600e640 1 -- 172.21.15.26:0/3242011379 wait complete. 2024-03-06T07:26:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.443+0000 7f080600e640 1 Processor -- start 2024-03-06T07:26:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.443+0000 7f080600e640 1 -- start start 2024-03-06T07:26:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f080600e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000ad2b0 0x7f080006c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f080600e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08000c69f0 0x7f080014c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f080600e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f080014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f080600e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800152a20 con 0x7f08000ad2b0 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f080600e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800155fe0 con 0x7f08000c69f0 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f080600e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0800152ba0 con 0x7f08000c8950 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000ad2b0 0x7f080006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000ad2b0 0x7f080006c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44840/0 (socket says 172.21.15.26:44840) 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.444+0000 7f07ff7fe640 1 -- 172.21.15.26:0/4180974741 learned_addr learned my addr 172.21.15.26:0/4180974741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07fffff640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f080014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07feffd640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08000c69f0 0x7f080014c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07ff7fe640 1 -- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 msgr2=0x7f080014c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f080014c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07ff7fe640 1 -- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08000c69f0 msgr2=0x7f080014c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08000c69f0 0x7f080014c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07ff7fe640 1 -- 172.21.15.26:0/4180974741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000c3210 con 0x7f08000ad2b0 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07feffd640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08000c69f0 0x7f080014c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000ad2b0 0x7f080006c280 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009d90 tx=0x7f07f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07f7fff640 1 -- 172.21.15.26:0/4180974741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0039050 con 0x7f08000ad2b0 2024-03-06T07:26:13.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07f7fff640 1 -- 172.21.15.26:0/4180974741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f003a040 con 0x7f08000ad2b0 2024-03-06T07:26:13.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08000c34b0 con 0x7f08000ad2b0 2024-03-06T07:26:13.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f07f7fff640 1 -- 172.21.15.26:0/4180974741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f003e610 con 0x7f08000ad2b0 2024-03-06T07:26:13.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.445+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080006c7c0 con 0x7f08000ad2b0 2024-03-06T07:26:13.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.447+0000 7f07f7fff640 1 -- 172.21.15.26:0/4180974741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07f0047020 con 0x7f08000ad2b0 2024-03-06T07:26:13.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.447+0000 7f07f7fff640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f07e006b4d0 0x7f07e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.447+0000 7f07f7fff640 1 -- 172.21.15.26:0/4180974741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07f00ac800 con 0x7f08000ad2b0 2024-03-06T07:26:13.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.447+0000 7f07feffd640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f07e006b4d0 0x7f07e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.448+0000 7f07feffd640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f07e006b4d0 0x7f07e006d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f080006d260 tx=0x7f07e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.449+0000 7f0808bf7740 1 Processor -- start 2024-03-06T07:26:13.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.449+0000 7f0808bf7740 1 -- start start 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b8ebf50 0x55950b8e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ea4b0 0x55950b8e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f0808bf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950b8506b0 con 0x55950b8ebf50 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950b7f9a60 con 0x55950b8ec330 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55950b84c750 con 0x55950b8ea4b0 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b8ebf50 0x55950b8e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b8ebf50 0x55950b8e9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44844/0 (socket says 172.21.15.26:44844) 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f07ff7fe640 1 -- 172.21.15.26:0/3656346196 learned_addr learned my addr 172.21.15.26:0/3656346196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f07fffff640 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ea4b0 0x55950b8e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.450+0000 7f07feffd640 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07ff7fe640 1 -- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ea4b0 msgr2=0x55950b8e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ea4b0 0x55950b8e6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07ff7fe640 1 -- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 msgr2=0x55950b8e9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07ff7fe640 1 -- 172.21.15.26:0/3656346196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55950b817f30 con 0x55950b8ebf50 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b8ebf50 0x55950b8e9a30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f07f0037040 tx=0x7f07f007a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfaeb6310fe62db server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07f57fa640 1 -- 172.21.15.26:0/3656346196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0083590 con 0x55950b8ebf50 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07f57fa640 1 -- 172.21.15.26:0/3656346196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0079180 con 0x55950b8ebf50 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.451+0000 7f07f57fa640 1 -- 172.21.15.26:0/3656346196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0081070 con 0x55950b8ebf50 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 -- 172.21.15.26:0/3656346196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b8ebf50 msgr2=0x55950b8e9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b8ebf50 0x55950b8e9a30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f07f0037040 tx=0x7f07f007a8a0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 -- 172.21.15.26:0/3656346196 shutdown_connections 2024-03-06T07:26:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ea4b0 0x55950b8e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3656346196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b8ebf50 0x55950b8e9a30 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 -- 172.21.15.26:0/3656346196 >> 172.21.15.26:0/3656346196 conn(0x55950b8e87d0 msgr2=0x55950b90d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 -- 172.21.15.26:0/3656346196 shutdown_connections 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 -- 172.21.15.26:0/3656346196 wait complete. 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 -- start start 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.452+0000 7f0808bf7740 1 Processor -- start 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b9385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b938b10 0x55950b90fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f0808bf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950b8506b0 con 0x55950b938b10 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950b80cea0 con 0x55950b8ebf50 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55950b90e230 con 0x55950b8ec330 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f0808bf7740 10 client.0 _ll_register_callbacks cb 0x55950b80c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f07fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b938b10 0x55950b90fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f07fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b938b10 0x55950b90fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44858/0 (socket says 172.21.15.26:44858) 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f07fffff640 1 -- 172.21.15.26:0/2568233367 learned_addr learned my addr 172.21.15.26:0/2568233367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b9385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.453+0000 7f07feffd640 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07fffff640 1 -- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 msgr2=0x55950b90c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07fffff640 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07fffff640 1 -- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 msgr2=0x55950b9385d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07fffff640 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b9385d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07fffff640 1 -- 172.21.15.26:0/2568233367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55950b80d270 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07fffff640 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b938b10 0x55950b90fbc0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f07f8005b60 tx=0x7f07f80014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b3c6cfd0b554fe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07de7fc640 1 -- 172.21.15.26:0/2568233367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8005c50 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07de7fc640 1 -- 172.21.15.26:0/2568233367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07f8005df0 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07f0079180 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07de7fc640 1 -- 172.21.15.26:0/2568233367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f8021050 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 10 client.8940 fetch_fsmap learned FSMap version 138 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=2+}) v3 -- 0x55950b5652d0 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07de7fc640 1 -- 172.21.15.26:0/2568233367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0079180 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f07de7fc640 1 -- 172.21.15.26:0/2568233367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 138) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f07f8024020 con 0x55950b938b10 2024-03-06T07:26:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 10 client.8940 fetch_fsmap finished waiting for FSMap version 138 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 10 client.8940 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 20 client.8940 populate_metadata read hostname 'smithi026' 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b945210 0x55950b947600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 4 client.8940 mds_command: new command op to 8793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.454+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 --> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55950b85dbf0 con 0x55950b945210 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.455+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b945210 0x55950b947600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.455+0000 7f07de7fc640 10 client.8940 ms_handle_connect on v2:172.21.15.26:6832/3029467671 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.455+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b945210 0x55950b947600 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437ef7bd7b7ae11c server_cookie=7fdee14283b431c7 in_seq=0 out_seq=0 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f07de7fc640 1 -- 172.21.15.26:0/2568233367 <== mds.0 v2:172.21.15.26:6832/3029467671 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55950b85dbf0 con 0x55950b945210 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f07de7fc640 10 client.8940 handle_command_reply: tid=0 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 client.8940 shutdown 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b938b10 msgr2=0x55950b90fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b938b10 0x55950b90fbc0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f07f8005b60 tx=0x7f07f80014b0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 shutdown_connections 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b945210 0x55950b947600 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b938b10 0x55950b90fbc0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f07de7fc640 0 client.8940 ms_handle_reset on v2:172.21.15.26:6832/3029467671 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 --2- 172.21.15.26:0/2568233367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b9385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 >> 172.21.15.26:0/2568233367 conn(0x55950b8ea4b0 msgr2=0x55950b928b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 shutdown_connections 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 1 -- 172.21.15.26:0/2568233367 wait complete. 2024-03-06T07:26:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.456+0000 7f0808bf7740 20 client.8940 trim_cache size 0 max 16384 2024-03-06T07:26:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 Processor -- start 2024-03-06T07:26:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 -- start start 2024-03-06T07:26:13.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ebf50 0x55950b8e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b911a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f6ed0 con 0x55950b9360f0 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f6430 con 0x55950b8ec330 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.473+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f68d0 con 0x55950b8ebf50 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b911a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b911a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44870/0 (socket says 172.21.15.26:44870) 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 -- 172.21.15.26:0/3172378921 learned_addr learned my addr 172.21.15.26:0/3172378921 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ebf50 0x55950b8e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07feffd640 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 -- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ebf50 msgr2=0x55950b8e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ebf50 0x55950b8e76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 -- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 msgr2=0x55950b8e7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 -- 172.21.15.26:0/3172378921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55950b62f6b0 con 0x55950b9360f0 2024-03-06T07:26:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07fffff640 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b911a30 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f07f8014710 tx=0x7f07f800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e5d754a5a43bee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07f4ff9640 1 -- 172.21.15.26:0/3172378921 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8029070 con 0x55950b9360f0 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07f4ff9640 1 -- 172.21.15.26:0/3172378921 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8019070 con 0x55950b9360f0 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.474+0000 7f07f4ff9640 1 -- 172.21.15.26:0/3172378921 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8014790 con 0x55950b9360f0 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 -- 172.21.15.26:0/3172378921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 msgr2=0x55950b911a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b911a30 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f07f8014710 tx=0x7f07f800e000 comp rx=0 tx=0).stop 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 -- 172.21.15.26:0/3172378921 shutdown_connections 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b911a30 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ec330 0x55950b8e7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3172378921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ebf50 0x55950b8e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 -- 172.21.15.26:0/3172378921 >> 172.21.15.26:0/3172378921 conn(0x55950b8ea4b0 msgr2=0x55950b8e1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 -- 172.21.15.26:0/3172378921 shutdown_connections 2024-03-06T07:26:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 -- 172.21.15.26:0/3172378921 wait complete. 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 -- start start 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.475+0000 7f0808bf7740 1 Processor -- start 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b912340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f0808bf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b9969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f0808bf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950b910ee0 con 0x55950b9360f0 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950b90d1c0 con 0x55950b8ebf50 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f0808bf7740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55950b9096a0 con 0x55950b8ec330 2024-03-06T07:26:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f0808bf7740 10 client.0 _ll_register_callbacks cb 0x7f07f8007f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f07fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b9969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f07fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b9969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44872/0 (socket says 172.21.15.26:44872) 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f07fffff640 1 -- 172.21.15.26:0/3612269974 learned_addr learned my addr 172.21.15.26:0/3612269974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b912340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.476+0000 7f07feffd640 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07fffff640 1 -- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 msgr2=0x55950b90afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07fffff640 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07fffff640 1 -- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 msgr2=0x55950b912340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07fffff640 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b912340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07fffff640 1 -- 172.21.15.26:0/3612269974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55950b941bd0 con 0x55950b9360f0 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07fffff640 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b9969f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f07f8028000 tx=0x7f07f8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd5948816c29d6a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:26:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07de7fc640 1 -- 172.21.15.26:0/3612269974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8029050 con 0x55950b9360f0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07de7fc640 1 -- 172.21.15.26:0/3612269974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07f8014d40 con 0x55950b9360f0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07f8019070 con 0x55950b9360f0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07de7fc640 1 -- 172.21.15.26:0/3612269974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f803f050 con 0x55950b9360f0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 10 client.8946 fetch_fsmap learned FSMap version 138 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=2+}) v3 -- 0x55950b94cae0 con 0x55950b9360f0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07de7fc640 1 -- 172.21.15.26:0/3612269974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8019070 con 0x55950b9360f0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f07de7fc640 1 -- 172.21.15.26:0/3612269974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 138) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f07f8042020 con 0x55950b9360f0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 10 client.8946 fetch_fsmap finished waiting for FSMap version 138 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 10 client.8946 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 20 client.8946 populate_metadata read hostname 'smithi026' 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b997050 0x55950b905980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 4 client.8946 mds_command: new command op to 8793 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.477+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 --> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55950b857f40 con 0x55950b997050 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b997050 0x55950b905980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f07de7fc640 10 client.8946 ms_handle_connect on v2:172.21.15.26:6832/3029467671 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f07ff7fe640 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b997050 0x55950b905980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35409d977587e38 server_cookie=5c94cc2de77ff4dc in_seq=0 out_seq=0 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f07de7fc640 1 -- 172.21.15.26:0/3612269974 <== mds.0 v2:172.21.15.26:6832/3029467671 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55950b857f40 con 0x55950b997050 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f07de7fc640 10 client.8946 handle_command_reply: tid=0 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 client.8946 shutdown 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 msgr2=0x55950b9969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b9969f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f07f8028000 tx=0x7f07f8000ed0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 shutdown_connections 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:6832/3029467671,v1:172.21.15.26:6833/3029467671] conn(0x55950b997050 0x55950b905980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55950b9360f0 0x55950b9969f0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55950b8ec330 0x55950b90afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 --2- 172.21.15.26:0/3612269974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55950b8ebf50 0x55950b912340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 >> 172.21.15.26:0/3612269974 conn(0x55950b8ea4b0 msgr2=0x55950b8e95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f07de7fc640 0 client.8946 ms_handle_reset on v2:172.21.15.26:6832/3029467671 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 shutdown_connections 2024-03-06T07:26:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 1 -- 172.21.15.26:0/3612269974 wait complete. 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.478+0000 7f0808bf7740 20 client.8946 trim_cache size 0 max 16384 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8793, 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.7914537509999997, 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 138, 2024-03-06T07:26:13.487 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 151, 2024-03-06T07:26:13.488 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 151, 2024-03-06T07:26:13.488 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.871147819000001 2024-03-06T07:26:13.488 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:26:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f07e006b4d0 msgr2=0x7f07e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f07e006b4d0 0x7f07e006d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f080006d260 tx=0x7f07e8008040 comp rx=0 tx=0).stop 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000ad2b0 msgr2=0x7f080006c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000ad2b0 0x7f080006c280 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009d90 tx=0x7f07f0004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 shutdown_connections 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f07e006b4d0 0x7f07e006d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08000c8950 0x7f080014c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08000c69f0 0x7f080014c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 --2- 172.21.15.26:0/4180974741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08000ad2b0 0x7f080006c280 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 >> 172.21.15.26:0/4180974741 conn(0x7f08000cae70 msgr2=0x7f08001558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 shutdown_connections 2024-03-06T07:26:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.479+0000 7f080600e640 1 -- 172.21.15.26:0/4180974741 wait complete. 2024-03-06T07:26:13.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.591+0000 7f1a24fd7640 1 Processor -- start 2024-03-06T07:26:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.591+0000 7f1a24fd7640 1 -- start start 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.592+0000 7f1a24fd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 0x7f1a201529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.592+0000 7f1a24fd7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a20155a40 0x7f1a20155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a24fd7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a20156410 con 0x7f1a2014f790 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a24fd7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a200c2680 con 0x7f1a20155a40 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a24fd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200c28d0 con 0x7f1a200c1aa0 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a20155a40 0x7f1a20155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a20155a40 0x7f1a20155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44876/0 (socket says 172.21.15.26:44876) 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/3355132101 learned_addr learned my addr 172.21.15.26:0/3355132101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1e59c640 1 --1- 172.21.15.26:0/3355132101 >> v1:172.21.15.26:6789/0 conn(0x7f1a2014f790 0x7f1a200c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54982/0 (socket says 172.21.15.26:54982) 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1dd9b640 1 --2- 172.21.15.26:0/3355132101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 0x7f1a201529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/3355132101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 msgr2=0x7f1a201529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1ed9d640 1 --2- 172.21.15.26:0/3355132101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 0x7f1a201529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/3355132101 >> v1:172.21.15.26:6789/0 conn(0x7f1a2014f790 legacy=0x7f1a200c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/3355132101 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a200c2bb0 con 0x7f1a20155a40 2024-03-06T07:26:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.593+0000 7f1a1d59a640 1 -- 172.21.15.26:0/3355132101 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010436327 0 0) 0x7f1a20156410 con 0x7f1a2014f790 2024-03-06T07:26:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.594+0000 7f1a1ed9d640 1 --2- 172.21.15.26:0/3355132101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a20155ed0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009e50 tx=0x7f1a1802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31640ea27a6a186f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.594+0000 7f1a1d59a640 1 -- 172.21.15.26:0/3355132101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1802cb00 con 0x7f1a20155a40 2024-03-06T07:26:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.594+0000 7f1a1d59a640 1 -- 172.21.15.26:0/3355132101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1802cca0 con 0x7f1a20155a40 2024-03-06T07:26:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.594+0000 7f1a1d59a640 1 -- 172.21.15.26:0/3355132101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18034610 con 0x7f1a20155a40 2024-03-06T07:26:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.594+0000 7f1a24fd7640 1 -- 172.21.15.26:0/3355132101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 msgr2=0x7f1a20155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.594+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/3355132101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a20155ed0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009e50 tx=0x7f1a1802bf60 comp rx=0 tx=0).stop 2024-03-06T07:26:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.595+0000 7f1a24fd7640 1 -- 172.21.15.26:0/3355132101 shutdown_connections 2024-03-06T07:26:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.595+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/3355132101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a20155ed0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.595+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/3355132101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 0x7f1a201529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.595+0000 7f1a24fd7640 1 -- 172.21.15.26:0/3355132101 >> 172.21.15.26:0/3355132101 conn(0x7f1a200c6710 msgr2=0x7f1a200c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.595+0000 7f1a24fd7640 1 -- 172.21.15.26:0/3355132101 shutdown_connections 2024-03-06T07:26:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.595+0000 7f1a24fd7640 1 -- 172.21.15.26:0/3355132101 wait complete. 2024-03-06T07:26:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.596+0000 7f1a24fd7640 1 Processor -- start 2024-03-06T07:26:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.596+0000 7f1a24fd7640 1 -- start start 2024-03-06T07:26:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 0x7f1a200c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2014f790 0x7f1a2014cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a2014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200c0d60 con 0x7f1a20155a40 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20155520 con 0x7f1a2014f790 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200c5e30 con 0x7f1a200c1aa0 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a2014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a2014a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44882/0 (socket says 172.21.15.26:44882) 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/200400190 learned_addr learned my addr 172.21.15.26:0/200400190 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 msgr2=0x7f1a200c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 0x7f1a200c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2014f790 msgr2=0x7f1a2014cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2014f790 0x7f1a2014cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 -- 172.21.15.26:0/200400190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a201e9ed0 con 0x7f1a20155a40 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a1ed9d640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a2014a4d0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009e50 tx=0x7f1a18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a16ffd640 1 -- 172.21.15.26:0/200400190 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18043020 con 0x7f1a20155a40 2024-03-06T07:26:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a16ffd640 1 -- 172.21.15.26:0/200400190 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1802ce50 con 0x7f1a20155a40 2024-03-06T07:26:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a16ffd640 1 -- 172.21.15.26:0/200400190 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18009510 con 0x7f1a20155a40 2024-03-06T07:26:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a20071090 con 0x7f1a20155a40 2024-03-06T07:26:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.598+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a20151e60 con 0x7f1a20155a40 2024-03-06T07:26:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.599+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2004f330 con 0x7f1a20155a40 2024-03-06T07:26:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.600+0000 7f1a16ffd640 1 -- 172.21.15.26:0/200400190 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a18046020 con 0x7f1a20155a40 2024-03-06T07:26:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.601+0000 7f1a16ffd640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a0006b4d0 0x7f1a0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.601+0000 7f1a16ffd640 1 -- 172.21.15.26:0/200400190 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a180ac920 con 0x7f1a20155a40 2024-03-06T07:26:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.601+0000 7f1a1e59c640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a0006b4d0 0x7f1a0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.602+0000 7f1a1e59c640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a0006b4d0 0x7f1a0006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1a10007fd0 tx=0x7f1a10007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.605+0000 7f1a16ffd640 1 -- 172.21.15.26:0/200400190 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1a1804d030 con 0x7f1a20155a40 2024-03-06T07:26:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.789+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a200c1f40 con 0x7f1a20155a40 2024-03-06T07:26:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.790+0000 7f1a16ffd640 1 -- 172.21.15.26:0/200400190 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16896 (secure 0 0 0) 0x7f1a1807b6a0 con 0x7f1a20155a40 2024-03-06T07:26:13.792 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:13.793 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":151,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:08.684485+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a0006b4d0 msgr2=0x7f1a0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a0006b4d0 0x7f1a0006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1a10007fd0 tx=0x7f1a10007920 comp rx=0 tx=0).stop 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 msgr2=0x7f1a2014a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a2014a4d0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009e50 tx=0x7f1a18004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 shutdown_connections 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a0006b4d0 0x7f1a0006d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a20155a40 0x7f1a2014a4d0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2014f790 0x7f1a2014cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 --2- 172.21.15.26:0/200400190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a200c1aa0 0x7f1a200c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 >> 172.21.15.26:0/200400190 conn(0x7f1a200c6710 msgr2=0x7f1a200c4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 shutdown_connections 2024-03-06T07:26:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.792+0000 7f1a24fd7640 1 -- 172.21.15.26:0/200400190 wait complete. 2024-03-06T07:26:13.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:26:05.339258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:26:06.016366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:13.805 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.907+0000 7f942102a640 1 Processor -- start 2024-03-06T07:26:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.907+0000 7f942102a640 1 -- start start 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f942102a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c0b00d0 0x7f941c0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f942102a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f941c06b8e0 0x7f941c06bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f942102a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c0b04b0 con 0x7f941c0c7210 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f942102a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c156160 con 0x7f941c06b8e0 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f942102a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0c5710 con 0x7f941c0b00d0 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f9413fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f941c06b8e0 0x7f941c06bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f9413fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f941c06b8e0 0x7f941c06bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44890/0 (socket says 172.21.15.26:44890) 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f9413fff640 1 -- 172.21.15.26:0/1630124891 learned_addr learned my addr 172.21.15.26:0/1630124891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.908+0000 7f941ad9d640 1 --2- 172.21.15.26:0/1630124891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c0b00d0 0x7f941c0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.909+0000 7f9413fff640 1 -- 172.21.15.26:0/1630124891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c0b00d0 msgr2=0x7f941c0c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.909+0000 7f9413fff640 1 --2- 172.21.15.26:0/1630124891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c0b00d0 0x7f941c0c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.909+0000 7f9413fff640 1 -- 172.21.15.26:0/1630124891 >> v1:172.21.15.117:6789/0 conn(0x7f941c0c7210 legacy=0x7f941c0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.909+0000 7f9413fff640 1 -- 172.21.15.26:0/1630124891 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c06d210 con 0x7f941c06b8e0 2024-03-06T07:26:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.909+0000 7f9413fff640 1 --2- 172.21.15.26:0/1630124891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c06b8e0 0x7f941c06bd20 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f9404009870 tx=0x7f940402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86738ceafe855263 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.909+0000 7f941a59c640 1 -- 172.21.15.26:0/1630124891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404034070 con 0x7f941c06b8e0 2024-03-06T07:26:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.909+0000 7f941a59c640 1 -- 172.21.15.26:0/1630124891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404030840 con 0x7f941c06b8e0 2024-03-06T07:26:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f941a59c640 1 -- 172.21.15.26:0/1630124891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404030c10 con 0x7f941c06b8e0 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 -- 172.21.15.26:0/1630124891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c06b8e0 msgr2=0x7f941c06bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 --2- 172.21.15.26:0/1630124891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c06b8e0 0x7f941c06bd20 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f9404009870 tx=0x7f940402b980 comp rx=0 tx=0).stop 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 -- 172.21.15.26:0/1630124891 shutdown_connections 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 --2- 172.21.15.26:0/1630124891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c06b8e0 0x7f941c06bd20 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 --2- 172.21.15.26:0/1630124891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c0b00d0 0x7f941c0c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 -- 172.21.15.26:0/1630124891 >> 172.21.15.26:0/1630124891 conn(0x7f941c0ac810 msgr2=0x7f941c0acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 -- 172.21.15.26:0/1630124891 shutdown_connections 2024-03-06T07:26:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.910+0000 7f942102a640 1 -- 172.21.15.26:0/1630124891 wait complete. 2024-03-06T07:26:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.912+0000 7f942102a640 1 Processor -- start 2024-03-06T07:26:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.912+0000 7f942102a640 1 -- start start 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f942102a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c06b8e0 0x7f941c0be790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f942102a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c0b00d0 0x7f941c1e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f942102a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f941c0c7210 0x7f941c1e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f942102a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c06c260 con 0x7f941c0b00d0 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f942102a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c152c50 con 0x7f941c0c7210 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f942102a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0c5ba0 con 0x7f941c06b8e0 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f9413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c0b00d0 0x7f941c1e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f9413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c0b00d0 0x7f941c1e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44904/0 (socket says 172.21.15.26:44904) 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f9413fff640 1 -- 172.21.15.26:0/4232664033 learned_addr learned my addr 172.21.15.26:0/4232664033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.913+0000 7f941b59e640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f941c0c7210 0x7f941c1e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f941ad9d640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c06b8e0 0x7f941c0be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f9413fff640 1 -- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c06b8e0 msgr2=0x7f941c0be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f9413fff640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c06b8e0 0x7f941c0be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f9413fff640 1 -- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f941c0c7210 msgr2=0x7f941c1e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f9413fff640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f941c0c7210 0x7f941c1e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f9413fff640 1 -- 172.21.15.26:0/4232664033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c1eef60 con 0x7f941c0b00d0 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f941b59e640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f941c0c7210 0x7f941c1e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f9413fff640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c0b00d0 0x7f941c1e7af0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9404000c00 tx=0x7f9404004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f94137fe640 1 -- 172.21.15.26:0/4232664033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404048070 con 0x7f941c0b00d0 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f94137fe640 1 -- 172.21.15.26:0/4232664033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404030640 con 0x7f941c0b00d0 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f94137fe640 1 -- 172.21.15.26:0/4232664033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404034040 con 0x7f941c0b00d0 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c0c26a0 con 0x7f941c0b00d0 2024-03-06T07:26:13.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.914+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c0c2c20 con 0x7f941c0b00d0 2024-03-06T07:26:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.915+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c04f330 con 0x7f941c0b00d0 2024-03-06T07:26:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.916+0000 7f94137fe640 1 -- 172.21.15.26:0/4232664033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f940404b070 con 0x7f941c0b00d0 2024-03-06T07:26:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.916+0000 7f94137fe640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93fc06b4d0 0x7f93fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.916+0000 7f94137fe640 1 -- 172.21.15.26:0/4232664033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94040ac230 con 0x7f941c0b00d0 2024-03-06T07:26:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.916+0000 7f941ad9d640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93fc06b4d0 0x7f93fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.917+0000 7f941ad9d640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93fc06b4d0 0x7f93fc06d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f940c006fd0 tx=0x7f940c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:13.920+0000 7f94137fe640 1 -- 172.21.15.26:0/4232664033 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f940407b0d0 con 0x7f941c0b00d0 2024-03-06T07:26:14.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.099+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f941c0c8090 con 0x7f941c0b00d0 2024-03-06T07:26:14.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.099+0000 7f94137fe640 1 -- 172.21.15.26:0/4232664033 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7f940407a9d0 con 0x7f941c0b00d0 2024-03-06T07:26:14.102 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93fc06b4d0 msgr2=0x7f93fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93fc06b4d0 0x7f93fc06d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f940c006fd0 tx=0x7f940c008040 comp rx=0 tx=0).stop 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c0b00d0 msgr2=0x7f941c1e7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c0b00d0 0x7f941c1e7af0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9404000c00 tx=0x7f9404004730 comp rx=0 tx=0).stop 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 shutdown_connections 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93fc06b4d0 0x7f93fc06d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f941c0c7210 0x7f941c1e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f941c0b00d0 0x7f941c1e7af0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 --2- 172.21.15.26:0/4232664033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f941c06b8e0 0x7f941c0be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 >> 172.21.15.26:0/4232664033 conn(0x7f941c0ac810 msgr2=0x7f941c0ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 shutdown_connections 2024-03-06T07:26:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.101+0000 7f942102a640 1 -- 172.21.15.26:0/4232664033 wait complete. 2024-03-06T07:26:14.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:26:14.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.216+0000 7fb83c229640 1 Processor -- start 2024-03-06T07:26:14.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.217+0000 7fb83c229640 1 -- start start 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83c229640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8340a8290 0x7fb8341539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83c229640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c69a0 0x7fb834072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83c229640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8340c62a0 con 0x7fb8340c4740 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83c229640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb834153f00 con 0x7fb8340a8290 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83c229640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340c4c00 con 0x7fb8340c69a0 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83a7c6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c69a0 0x7fb834072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb839fc5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8340a8290 0x7fb8341539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83a7c6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c69a0 0x7fb834072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:54326/0 (socket says 172.21.15.26:54326) 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb8397c4640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb8340c4740 0x7fb8340701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52274/0 (socket says 172.21.15.26:52274) 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb83a7c6640 1 -- 172.21.15.26:0/3822323603 learned_addr learned my addr 172.21.15.26:0/3822323603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb839fc5640 1 -- 172.21.15.26:0/3822323603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c69a0 msgr2=0x7fb834072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb839fc5640 1 --2- 172.21.15.26:0/3822323603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c69a0 0x7fb834072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb839fc5640 1 -- 172.21.15.26:0/3822323603 >> v1:172.21.15.117:6789/0 conn(0x7fb8340c4740 legacy=0x7fb8340701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb839fc5640 1 -- 172.21.15.26:0/3822323603 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340c4ee0 con 0x7fb8340a8290 2024-03-06T07:26:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.218+0000 7fb838fc3640 1 -- 172.21.15.26:0/3822323603 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867056335 0 0) 0x7fb8340c62a0 con 0x7fb8340c4740 2024-03-06T07:26:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.219+0000 7fb839fc5640 1 --2- 172.21.15.26:0/3822323603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340a8290 0x7fb8341539c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0097a0 tx=0x7fb82c02b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d0ccfedf42c72c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.219+0000 7fb838fc3640 1 -- 172.21.15.26:0/3822323603 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c038070 con 0x7fb8340a8290 2024-03-06T07:26:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb838fc3640 1 -- 172.21.15.26:0/3822323603 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c031700 con 0x7fb8340a8290 2024-03-06T07:26:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb838fc3640 1 -- 172.21.15.26:0/3822323603 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c031ad0 con 0x7fb8340a8290 2024-03-06T07:26:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 -- 172.21.15.26:0/3822323603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340a8290 msgr2=0x7fb8341539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 --2- 172.21.15.26:0/3822323603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340a8290 0x7fb8341539c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0097a0 tx=0x7fb82c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 -- 172.21.15.26:0/3822323603 shutdown_connections 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 --2- 172.21.15.26:0/3822323603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c69a0 0x7fb834072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 --2- 172.21.15.26:0/3822323603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340a8290 0x7fb8341539c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 -- 172.21.15.26:0/3822323603 >> 172.21.15.26:0/3822323603 conn(0x7fb8340c5560 msgr2=0x7fb8340c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 -- 172.21.15.26:0/3822323603 shutdown_connections 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.220+0000 7fb83c229640 1 -- 172.21.15.26:0/3822323603 wait complete. 2024-03-06T07:26:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.222+0000 7fb83c229640 1 Processor -- start 2024-03-06T07:26:14.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.222+0000 7fb83c229640 1 -- start start 2024-03-06T07:26:14.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb83c229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8340a8290 0x7fb834156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb83c229640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c4740 0x7fb8340a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb83c229640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340c69a0 0x7fb83406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb83c229640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340c6e30 con 0x7fb8340a8290 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb83c229640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83406fc80 con 0x7fb8340c69a0 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb83c229640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb834072bc0 con 0x7fb8340c4740 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb839fc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8340a8290 0x7fb834156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb839fc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8340a8290 0x7fb834156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44912/0 (socket says 172.21.15.26:44912) 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb839fc5640 1 -- 172.21.15.26:0/718483833 learned_addr learned my addr 172.21.15.26:0/718483833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb83a7c6640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340c69a0 0x7fb83406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.223+0000 7fb8397c4640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c4740 0x7fb8340a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb839fc5640 1 -- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c4740 msgr2=0x7fb8340a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb839fc5640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c4740 0x7fb8340a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb839fc5640 1 -- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340c69a0 msgr2=0x7fb83406c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb839fc5640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340c69a0 0x7fb83406c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb839fc5640 1 -- 172.21.15.26:0/718483833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83414c1c0 con 0x7fb8340a8290 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb839fc5640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8340a8290 0x7fb834156940 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0097a0 tx=0x7fb82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb8327fc640 1 -- 172.21.15.26:0/718483833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c038040 con 0x7fb8340a8290 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb8327fc640 1 -- 172.21.15.26:0/718483833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c004030 con 0x7fb8340a8290 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83414bd00 con 0x7fb8340a8290 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb8327fc640 1 -- 172.21.15.26:0/718483833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c004420 con 0x7fb8340a8290 2024-03-06T07:26:14.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.224+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8341561b0 con 0x7fb8340a8290 2024-03-06T07:26:14.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.225+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83414bf20 con 0x7fb8340a8290 2024-03-06T07:26:14.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.226+0000 7fb8327fc640 1 -- 172.21.15.26:0/718483833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb82c033780 con 0x7fb8340a8290 2024-03-06T07:26:14.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.227+0000 7fb8327fc640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.227+0000 7fb8327fc640 1 -- 172.21.15.26:0/718483833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb82c0ac5e0 con 0x7fb8340a8290 2024-03-06T07:26:14.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.227+0000 7fb8397c4640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.228+0000 7fb8397c4640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb83414f800 tx=0x7fb828007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.231+0000 7fb8327fc640 1 -- 172.21.15.26:0/718483833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb82c030070 con 0x7fb8340a8290 2024-03-06T07:26:14.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.410+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb83404f330 con 0x7fb8340a8290 2024-03-06T07:26:14.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.410+0000 7fb8327fc640 1 -- 172.21.15.26:0/718483833 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb82c07abe0 con 0x7fb8340a8290 2024-03-06T07:26:14.413 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:26:14.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 msgr2=0x7fb82006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb83414f800 tx=0x7fb828007d50 comp rx=0 tx=0).stop 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8340a8290 msgr2=0x7fb834156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8340a8290 0x7fb834156940 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0097a0 tx=0x7fb82c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 shutdown_connections 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8340c69a0 0x7fb83406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c4740 0x7fb8340a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 --2- 172.21.15.26:0/718483833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8340a8290 0x7fb834156940 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 >> 172.21.15.26:0/718483833 conn(0x7fb8340c5560 msgr2=0x7fb834150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 shutdown_connections 2024-03-06T07:26:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.412+0000 7fb83c229640 1 -- 172.21.15.26:0/718483833 wait complete. 2024-03-06T07:26:14.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:26:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.530+0000 7fb89bc88640 1 Processor -- start 2024-03-06T07:26:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.531+0000 7fb89bc88640 1 -- start start 2024-03-06T07:26:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.531+0000 7fb89bc88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940a8290 0x7fb8941539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.531+0000 7fb89bc88640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8940c4740 0x7fb8940701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.531+0000 7fb89bc88640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8940c62a0 con 0x7fb8940c69a0 2024-03-06T07:26:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.531+0000 7fb89bc88640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb894153f00 con 0x7fb8940c4740 2024-03-06T07:26:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.531+0000 7fb89bc88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940706e0 con 0x7fb8940a8290 2024-03-06T07:26:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb899a24640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940a8290 0x7fb8941539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb899223640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8940c4740 0x7fb8940701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb89a225640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb8940c69a0 0x7fb894072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52278/0 (socket says 172.21.15.26:52278) 2024-03-06T07:26:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb89a225640 1 -- 172.21.15.26:0/3054103057 learned_addr learned my addr 172.21.15.26:0/3054103057 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb899a24640 1 -- 172.21.15.26:0/3054103057 >> v2:172.21.15.117:3300/0 conn(0x7fb8940c4740 msgr2=0x7fb8940701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb899a24640 1 --2- 172.21.15.26:0/3054103057 >> v2:172.21.15.117:3300/0 conn(0x7fb8940c4740 0x7fb8940701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb899a24640 1 -- 172.21.15.26:0/3054103057 >> v1:172.21.15.117:6789/0 conn(0x7fb8940c69a0 legacy=0x7fb894072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb899a24640 1 -- 172.21.15.26:0/3054103057 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940c4c00 con 0x7fb8940a8290 2024-03-06T07:26:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.532+0000 7fb899223640 1 --2- 172.21.15.26:0/3054103057 >> v2:172.21.15.117:3300/0 conn(0x7fb8940c4740 0x7fb8940701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.533+0000 7fb899a24640 1 --2- 172.21.15.26:0/3054103057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940a8290 0x7fb8941539c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb8900097a0 tx=0x7fb89002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e11388fe0ee66e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.533+0000 7fb898a22640 1 -- 172.21.15.26:0/3054103057 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890038070 con 0x7fb8940a8290 2024-03-06T07:26:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.533+0000 7fb898a22640 1 -- 172.21.15.26:0/3054103057 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890031700 con 0x7fb8940a8290 2024-03-06T07:26:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.533+0000 7fb898a22640 1 -- 172.21.15.26:0/3054103057 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890031ad0 con 0x7fb8940a8290 2024-03-06T07:26:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.533+0000 7fb89bc88640 1 -- 172.21.15.26:0/3054103057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940a8290 msgr2=0x7fb8941539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.533+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3054103057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940a8290 0x7fb8941539c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb8900097a0 tx=0x7fb89002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.534+0000 7fb89bc88640 1 -- 172.21.15.26:0/3054103057 shutdown_connections 2024-03-06T07:26:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.534+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3054103057 >> v2:172.21.15.117:3300/0 conn(0x7fb8940c4740 0x7fb8940701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.534+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3054103057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940a8290 0x7fb8941539c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.534+0000 7fb89bc88640 1 -- 172.21.15.26:0/3054103057 >> 172.21.15.26:0/3054103057 conn(0x7fb8940c5560 msgr2=0x7fb8940c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.534+0000 7fb89bc88640 1 -- 172.21.15.26:0/3054103057 shutdown_connections 2024-03-06T07:26:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.534+0000 7fb89bc88640 1 -- 172.21.15.26:0/3054103057 wait complete. 2024-03-06T07:26:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 Processor -- start 2024-03-06T07:26:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 -- start start 2024-03-06T07:26:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8940a8290 0x7fb894154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8940c4740 0x7fb8940c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c69a0 0x7fb89414d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89406fc80 con 0x7fb8940a8290 2024-03-06T07:26:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89406f9a0 con 0x7fb8940c4740 2024-03-06T07:26:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb89bc88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb894072d90 con 0x7fb8940c69a0 2024-03-06T07:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb899a24640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8940a8290 0x7fb894154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.535+0000 7fb899a24640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8940a8290 0x7fb894154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44914/0 (socket says 172.21.15.26:44914) 2024-03-06T07:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899a24640 1 -- 172.21.15.26:0/3503659204 learned_addr learned my addr 172.21.15.26:0/3503659204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb89a225640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c69a0 0x7fb89414d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899223640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8940c4740 0x7fb8940c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899a24640 1 -- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c69a0 msgr2=0x7fb89414d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899a24640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c69a0 0x7fb89414d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899a24640 1 -- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8940c4740 msgr2=0x7fb8940c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899a24640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8940c4740 0x7fb8940c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899a24640 1 -- 172.21.15.26:0/3503659204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8941ee4c0 con 0x7fb8940a8290 2024-03-06T07:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb899a24640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8940a8290 0x7fb894154c70 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb890036040 tx=0x7fb8900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb88a7fc640 1 -- 172.21.15.26:0/3503659204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890042070 con 0x7fb8940a8290 2024-03-06T07:26:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb88a7fc640 1 -- 172.21.15.26:0/3503659204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89003d950 con 0x7fb8940a8290 2024-03-06T07:26:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8941ee7a0 con 0x7fb8940a8290 2024-03-06T07:26:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb88a7fc640 1 -- 172.21.15.26:0/3503659204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890038040 con 0x7fb8940a8290 2024-03-06T07:26:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.536+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8941eed60 con 0x7fb8940a8290 2024-03-06T07:26:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.537+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89404f330 con 0x7fb8940a8290 2024-03-06T07:26:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.538+0000 7fb88a7fc640 1 -- 172.21.15.26:0/3503659204 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb89003b070 con 0x7fb8940a8290 2024-03-06T07:26:14.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.538+0000 7fb88a7fc640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb87806b4d0 0x7fb87806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.538+0000 7fb88a7fc640 1 -- 172.21.15.26:0/3503659204 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8900aba00 con 0x7fb8940a8290 2024-03-06T07:26:14.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.539+0000 7fb899223640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb87806b4d0 0x7fb87806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.539+0000 7fb899223640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb87806b4d0 0x7fb87806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb884009560 tx=0x7fb884009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.543+0000 7fb88a7fc640 1 -- 172.21.15.26:0/3503659204 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb890030070 con 0x7fb8940a8290 2024-03-06T07:26:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.729+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8940c8480 con 0x7fb8940a8290 2024-03-06T07:26:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.729+0000 7fb88a7fc640 1 -- 172.21.15.26:0/3503659204 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb89007a250 con 0x7fb8940a8290 2024-03-06T07:26:14.732 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb87806b4d0 msgr2=0x7fb87806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb87806b4d0 0x7fb87806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb884009560 tx=0x7fb884009190 comp rx=0 tx=0).stop 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8940a8290 msgr2=0x7fb894154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8940a8290 0x7fb894154c70 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb890036040 tx=0x7fb8900046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 shutdown_connections 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb87806b4d0 0x7fb87806d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c69a0 0x7fb89414d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8940c4740 0x7fb8940c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 --2- 172.21.15.26:0/3503659204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8940a8290 0x7fb894154c70 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 >> 172.21.15.26:0/3503659204 conn(0x7fb8940c5560 msgr2=0x7fb894152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 shutdown_connections 2024-03-06T07:26:14.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.731+0000 7fb89bc88640 1 -- 172.21.15.26:0/3503659204 wait complete. 2024-03-06T07:26:14.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:26:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.843+0000 7faed415e640 1 Processor -- start 2024-03-06T07:26:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.843+0000 7faed415e640 1 -- start start 2024-03-06T07:26:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.843+0000 7faed415e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faecc14f360 0x7faecc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed415e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faecc158b60 0x7faecc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed415e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc0c73d0 con 0x7faecc0c88c0 2024-03-06T07:26:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed415e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faecc0c8dd0 con 0x7faecc158b60 2024-03-06T07:26:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed415e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faecc0be2e0 con 0x7faecc14f360 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed26fb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faecc158b60 0x7faecc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed26fb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faecc158b60 0x7faecc158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44928/0 (socket says 172.21.15.26:44928) 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed26fb640 1 -- 172.21.15.26:0/3251140471 learned_addr learned my addr 172.21.15.26:0/3251140471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed1efa640 1 --2- 172.21.15.26:0/3251140471 >> v2:172.21.15.117:3300/0 conn(0x7faecc14f360 0x7faecc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed26fb640 1 -- 172.21.15.26:0/3251140471 >> v2:172.21.15.117:3300/0 conn(0x7faecc14f360 msgr2=0x7faecc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed26fb640 1 --2- 172.21.15.26:0/3251140471 >> v2:172.21.15.117:3300/0 conn(0x7faecc14f360 0x7faecc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed26fb640 1 -- 172.21.15.26:0/3251140471 >> v1:172.21.15.117:6789/0 conn(0x7faecc0c88c0 legacy=0x7faecc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.844+0000 7faed26fb640 1 -- 172.21.15.26:0/3251140471 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0c7670 con 0x7faecc158b60 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed26fb640 1 --2- 172.21.15.26:0/3251140471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc158ff0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7faecc08c840 tx=0x7faec402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c730d3f990af9ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed0ef8640 1 -- 172.21.15.26:0/3251140471 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec402ca80 con 0x7faecc158b60 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed0ef8640 1 -- 172.21.15.26:0/3251140471 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec402cc20 con 0x7faecc158b60 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed0ef8640 1 -- 172.21.15.26:0/3251140471 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec40343f0 con 0x7faecc158b60 2024-03-06T07:26:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed415e640 1 -- 172.21.15.26:0/3251140471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 msgr2=0x7faecc158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed415e640 1 --2- 172.21.15.26:0/3251140471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc158ff0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7faecc08c840 tx=0x7faec402bee0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed415e640 1 -- 172.21.15.26:0/3251140471 shutdown_connections 2024-03-06T07:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed415e640 1 --2- 172.21.15.26:0/3251140471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc158ff0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed415e640 1 --2- 172.21.15.26:0/3251140471 >> v2:172.21.15.117:3300/0 conn(0x7faecc14f360 0x7faecc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.845+0000 7faed415e640 1 -- 172.21.15.26:0/3251140471 >> 172.21.15.26:0/3251140471 conn(0x7faecc0ac570 msgr2=0x7faecc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.846+0000 7faed415e640 1 -- 172.21.15.26:0/3251140471 shutdown_connections 2024-03-06T07:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.846+0000 7faed415e640 1 -- 172.21.15.26:0/3251140471 wait complete. 2024-03-06T07:26:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 Processor -- start 2024-03-06T07:26:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 -- start start 2024-03-06T07:26:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faecc0c88c0 0x7faecc1ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc14f360 0x7faecc0bf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc1eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc0c5a50 con 0x7faecc158b60 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc0c7c30 con 0x7faecc14f360 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed415e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc0c5bd0 con 0x7faecc0c88c0 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed26fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc1eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed26fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc1eaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44934/0 (socket says 172.21.15.26:44934) 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.847+0000 7faed26fb640 1 -- 172.21.15.26:0/1770495286 learned_addr learned my addr 172.21.15.26:0/1770495286 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed16f9640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc14f360 0x7faecc0bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed1efa640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faecc0c88c0 0x7faecc1ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed26fb640 1 -- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faecc0c88c0 msgr2=0x7faecc1ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed26fb640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faecc0c88c0 0x7faecc1ebcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed26fb640 1 -- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc14f360 msgr2=0x7faecc0bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed26fb640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc14f360 0x7faecc0bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed26fb640 1 -- 172.21.15.26:0/1770495286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0c0d90 con 0x7faecc158b60 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed16f9640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc14f360 0x7faecc0bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed26fb640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc1eaaf0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7faec4036040 tx=0x7faec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faeca7fc640 1 -- 172.21.15.26:0/1770495286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4043020 con 0x7faecc158b60 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faeca7fc640 1 -- 172.21.15.26:0/1770495286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec403e450 con 0x7faecc158b60 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc0c1030 con 0x7faecc158b60 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faeca7fc640 1 -- 172.21.15.26:0/1770495286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4009510 con 0x7faecc158b60 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.848+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc0bf1e0 con 0x7faecc158b60 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.849+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc0c1200 con 0x7faecc158b60 2024-03-06T07:26:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.850+0000 7faeca7fc640 1 -- 172.21.15.26:0/1770495286 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faec4046020 con 0x7faecc158b60 2024-03-06T07:26:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.850+0000 7faeca7fc640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faeb806b4d0 0x7faeb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.850+0000 7faeca7fc640 1 -- 172.21.15.26:0/1770495286 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faec40acbc0 con 0x7faecc158b60 2024-03-06T07:26:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.851+0000 7faed1efa640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faeb806b4d0 0x7faeb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.851+0000 7faed1efa640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faeb806b4d0 0x7faeb806d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7faecc14d0d0 tx=0x7faebc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:14.855+0000 7faeca7fc640 1 -- 172.21.15.26:0/1770495286 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faec404d030 con 0x7faecc158b60 2024-03-06T07:26:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.037+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faecc04f330 con 0x7faecc158b60 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.037+0000 7faeca7fc640 1 -- 172.21.15.26:0/1770495286 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v151) v1 ==== 84+0+209 (secure 0 0 0) 0x7faec407ba60 con 0x7faecc158b60 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 23, 2024-03-06T07:26:15.040 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_metadata" 2024-03-06T07:26:15.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:15.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:15.041 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 24, 2024-03-06T07:26:15.041 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_data" 2024-03-06T07:26:15.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:15.041 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:26:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faeb806b4d0 msgr2=0x7faeb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faeb806b4d0 0x7faeb806d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7faecc14d0d0 tx=0x7faebc008040 comp rx=0 tx=0).stop 2024-03-06T07:26:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 msgr2=0x7faecc1eaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc1eaaf0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7faec4036040 tx=0x7faec4004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 shutdown_connections 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faeb806b4d0 0x7faeb806d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faecc158b60 0x7faecc1eaaf0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc14f360 0x7faecc0bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 --2- 172.21.15.26:0/1770495286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faecc0c88c0 0x7faecc1ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 >> 172.21.15.26:0/1770495286 conn(0x7faecc0ac570 msgr2=0x7faecc0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 shutdown_connections 2024-03-06T07:26:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.040+0000 7faed415e640 1 -- 172.21.15.26:0/1770495286 wait complete. 2024-03-06T07:26:15.053 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-06T07:26:15.053 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-06T07:26:15.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.156+0000 7f7c909c7640 1 Processor -- start 2024-03-06T07:26:15.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.156+0000 7f7c909c7640 1 -- start start 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c909c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c8c0c22b0 0x7f7c8c151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c909c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c909c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7c8c14f490 0x7f7c8c153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c909c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c06e850 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c909c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c06d2b0 con 0x7f7c8c14f490 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c909c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c06d590 con 0x7f7c8c0a9830 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c8a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c8c0c22b0 0x7f7c8c151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c8a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c8c0c22b0 0x7f7c8c151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44946/0 (socket says 172.21.15.26:44946) 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c8a59c640 1 -- 172.21.15.26:0/1830327911 learned_addr learned my addr 172.21.15.26:0/1830327911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c8ad9d640 1 --2- 172.21.15.26:0/1830327911 >> v2:172.21.15.117:3300/0 conn(0x7f7c8c14f490 0x7f7c8c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c89d9b640 1 --2- 172.21.15.26:0/1830327911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c8a59c640 1 -- 172.21.15.26:0/1830327911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 msgr2=0x7f7c8c14ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.157+0000 7f7c8a59c640 1 --2- 172.21.15.26:0/1830327911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c14ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c8a59c640 1 -- 172.21.15.26:0/1830327911 >> v2:172.21.15.117:3300/0 conn(0x7f7c8c14f490 msgr2=0x7f7c8c153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c8a59c640 1 --2- 172.21.15.26:0/1830327911 >> v2:172.21.15.117:3300/0 conn(0x7f7c8c14f490 0x7f7c8c153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c8a59c640 1 -- 172.21.15.26:0/1830327911 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c06d870 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c89d9b640 1 --2- 172.21.15.26:0/1830327911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c14ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c8a59c640 1 --2- 172.21.15.26:0/1830327911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c151e10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c06c1c0 tx=0x7f7c7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c466152f025c8688 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c8959a640 1 -- 172.21.15.26:0/1830327911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74035020 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c8959a640 1 -- 172.21.15.26:0/1830327911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c74031870 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.158+0000 7f7c8959a640 1 -- 172.21.15.26:0/1830327911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74031c40 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 -- 172.21.15.26:0/1830327911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 msgr2=0x7f7c8c151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 --2- 172.21.15.26:0/1830327911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c151e10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c06c1c0 tx=0x7f7c7402b910 comp rx=0 tx=0).stop 2024-03-06T07:26:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 -- 172.21.15.26:0/1830327911 shutdown_connections 2024-03-06T07:26:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 --2- 172.21.15.26:0/1830327911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c14ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 --2- 172.21.15.26:0/1830327911 >> v2:172.21.15.117:3300/0 conn(0x7f7c8c14f490 0x7f7c8c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 --2- 172.21.15.26:0/1830327911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c151e10 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 -- 172.21.15.26:0/1830327911 >> 172.21.15.26:0/1830327911 conn(0x7f7c8c0c65b0 msgr2=0x7f7c8c0c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 -- 172.21.15.26:0/1830327911 shutdown_connections 2024-03-06T07:26:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.159+0000 7f7c909c7640 1 -- 172.21.15.26:0/1830327911 wait complete. 2024-03-06T07:26:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.160+0000 7f7c909c7640 1 Processor -- start 2024-03-06T07:26:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.160+0000 7f7c909c7640 1 -- start start 2024-03-06T07:26:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c909c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c1e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c909c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c0c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c909c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c8c14f490 0x7f7c8c0bb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c909c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0c2810 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c909c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0c2690 con 0x7f7c8c14f490 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c909c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1538d0 con 0x7f7c8c0a9830 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c89d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c0c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c89d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c0c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44956/0 (socket says 172.21.15.26:44956) 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c89d9b640 1 -- 172.21.15.26:0/2915560990 learned_addr learned my addr 172.21.15.26:0/2915560990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c8ad9d640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c8c14f490 0x7f7c8c0bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c8a59c640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c1e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c89d9b640 1 -- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 msgr2=0x7f7c8c1e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c89d9b640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c1e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c89d9b640 1 -- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c8c14f490 msgr2=0x7f7c8c0bb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.161+0000 7f7c89d9b640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c8c14f490 0x7f7c8c0bb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.162+0000 7f7c89d9b640 1 -- 172.21.15.26:0/2915560990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c0c4fc0 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.162+0000 7f7c89d9b640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c0c4a80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00e840 tx=0x7f7c7c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.162+0000 7f7c82ffd640 1 -- 172.21.15.26:0/2915560990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00f040 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.162+0000 7f7c82ffd640 1 -- 172.21.15.26:0/2915560990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c004030 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.162+0000 7f7c82ffd640 1 -- 172.21.15.26:0/2915560990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c004400 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.162+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c0c5280 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.162+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c0c5800 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.163+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c04f330 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.163+0000 7f7c82ffd640 1 -- 172.21.15.26:0/2915560990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c7c00c6d0 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.164+0000 7f7c82ffd640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.164+0000 7f7c82ffd640 1 -- 172.21.15.26:0/2915560990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c7c089fc0 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.164+0000 7f7c8a59c640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.165+0000 7f7c8a59c640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7c74036000 tx=0x7f7c7403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.168+0000 7f7c82ffd640 1 -- 172.21.15.26:0/2915560990 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c7c017020 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.380+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7c8c0c5a00 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.715+0000 7f7c82ffd640 1 -- 172.21.15.26:0/2915560990 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v152) v1 ==== 140+0+0 (secure 0 0 0) 0x7f7c7c058d40 con 0x7f7c8c0c22b0 2024-03-06T07:26:15.718 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_metadata' created 2024-03-06T07:26:15.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c6c06b4d0 msgr2=0x7f7c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7c74036000 tx=0x7f7c7403c040 comp rx=0 tx=0).stop 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 msgr2=0x7f7c8c0c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c0c4a80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00e840 tx=0x7f7c7c00ed00 comp rx=0 tx=0).stop 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 shutdown_connections 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c8c14f490 0x7f7c8c0bb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c8c0c22b0 0x7f7c8c0c4a80 secure :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00e840 tx=0x7f7c7c00ed00 comp rx=0 tx=0).stop 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 --2- 172.21.15.26:0/2915560990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c8c0a9830 0x7f7c8c1e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.719+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 >> 172.21.15.26:0/2915560990 conn(0x7f7c8c0c65b0 msgr2=0x7f7c8c1519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.720+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 shutdown_connections 2024-03-06T07:26:15.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.721+0000 7f7c909c7640 1 -- 172.21.15.26:0/2915560990 wait complete. 2024-03-06T07:26:15.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:26:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.847+0000 7f79689a0640 1 Processor -- start 2024-03-06T07:26:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.847+0000 7f79689a0640 1 -- start start 2024-03-06T07:26:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f79689a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7964072230 0x7f79640c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f79689a0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79640726c0 con 0x7f796406f650 2024-03-06T07:26:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f79689a0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79640c31b0 con 0x7f7964150490 2024-03-06T07:26:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f79689a0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7964150870 con 0x7f7964072230 2024-03-06T07:26:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f7963fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7964072230 0x7f79640c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f7962ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f796406f650 0x7f7964153780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54988/0 (socket says 172.21.15.26:54988) 2024-03-06T07:26:15.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f7962ffd640 1 -- 172.21.15.26:0/1150746332 learned_addr learned my addr 172.21.15.26:0/1150746332 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:15.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f7963fff640 1 -- 172.21.15.26:0/1150746332 >> v1:172.21.15.117:6789/0 conn(0x7f7964150490 legacy=0x7f7964156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f7963fff640 1 -- 172.21.15.26:0/1150746332 >> v1:172.21.15.26:6789/0 conn(0x7f796406f650 legacy=0x7f7964153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f79627fc640 1 -- 172.21.15.26:0/1150746332 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680905848 0 0) 0x7f79640726c0 con 0x7f796406f650 2024-03-06T07:26:15.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.848+0000 7f7963fff640 1 -- 172.21.15.26:0/1150746332 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79640c3470 con 0x7f7964072230 2024-03-06T07:26:15.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.849+0000 7f7963fff640 1 --2- 172.21.15.26:0/1150746332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964072230 0x7f79640c50c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f795c009d90 tx=0x7f795c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff25635fe2acb19a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.849+0000 7f79627fc640 1 -- 172.21.15.26:0/1150746332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c039070 con 0x7f7964072230 2024-03-06T07:26:15.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.849+0000 7f79627fc640 1 -- 172.21.15.26:0/1150746332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c02cbd0 con 0x7f7964072230 2024-03-06T07:26:15.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.849+0000 7f79627fc640 1 -- 172.21.15.26:0/1150746332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c034440 con 0x7f7964072230 2024-03-06T07:26:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.850+0000 7f79689a0640 1 -- 172.21.15.26:0/1150746332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964072230 msgr2=0x7f79640c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.850+0000 7f79689a0640 1 --2- 172.21.15.26:0/1150746332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964072230 0x7f79640c50c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f795c009d90 tx=0x7f795c02be20 comp rx=0 tx=0).stop 2024-03-06T07:26:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.850+0000 7f79689a0640 1 -- 172.21.15.26:0/1150746332 shutdown_connections 2024-03-06T07:26:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.850+0000 7f79689a0640 1 --2- 172.21.15.26:0/1150746332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964072230 0x7f79640c50c0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.850+0000 7f79689a0640 1 -- 172.21.15.26:0/1150746332 >> 172.21.15.26:0/1150746332 conn(0x7f796406afc0 msgr2=0x7f796406b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.850+0000 7f79689a0640 1 -- 172.21.15.26:0/1150746332 shutdown_connections 2024-03-06T07:26:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.850+0000 7f79689a0640 1 -- 172.21.15.26:0/1150746332 wait complete. 2024-03-06T07:26:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.851+0000 7f79689a0640 1 Processor -- start 2024-03-06T07:26:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.851+0000 7f79689a0640 1 -- start start 2024-03-06T07:26:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f79689a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f796406f650 0x7f796414ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f79689a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964072230 0x7f796413ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f79689a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964150490 0x7f796413f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f79689a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7964153260 con 0x7f7964150490 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f79689a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7964152f80 con 0x7f7964072230 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f79689a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c5600 con 0x7f796406f650 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964150490 0x7f796413f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964150490 0x7f796413f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44976/0 (socket says 172.21.15.26:44976) 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 -- 172.21.15.26:0/2528846818 learned_addr learned my addr 172.21.15.26:0/2528846818 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f79637fe640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f796406f650 0x7f796414ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7962ffd640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964072230 0x7f796413ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 -- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f796406f650 msgr2=0x7f796414ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f796406f650 0x7f796414ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 -- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964072230 msgr2=0x7f796413ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964072230 0x7f796413ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 -- 172.21.15.26:0/2528846818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796413f740 con 0x7f7964150490 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.852+0000 7f7963fff640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964150490 0x7f796413f200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f795c009d90 tx=0x7f795c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.853+0000 7f794bfff640 1 -- 172.21.15.26:0/2528846818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c044020 con 0x7f7964150490 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.853+0000 7f794bfff640 1 -- 172.21.15.26:0/2528846818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c03f470 con 0x7f7964150490 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.853+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7964155640 con 0x7f7964150490 2024-03-06T07:26:15.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.853+0000 7f794bfff640 1 -- 172.21.15.26:0/2528846818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c039040 con 0x7f7964150490 2024-03-06T07:26:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.853+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7964155900 con 0x7f7964150490 2024-03-06T07:26:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.854+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79640693c0 con 0x7f7964150490 2024-03-06T07:26:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.854+0000 7f794bfff640 1 -- 172.21.15.26:0/2528846818 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f795c047020 con 0x7f7964150490 2024-03-06T07:26:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.855+0000 7f794bfff640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f795006b4d0 0x7f795006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.855+0000 7f794bfff640 1 -- 172.21.15.26:0/2528846818 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6472+0+0 (secure 0 0 0) 0x7f795c0ad150 con 0x7f7964150490 2024-03-06T07:26:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.855+0000 7f79637fe640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f795006b4d0 0x7f795006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.856+0000 7f79637fe640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f795006b4d0 0x7f795006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7954006fd0 tx=0x7f7954008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:15.860+0000 7f794bfff640 1 -- 172.21.15.26:0/2528846818 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f795c04e030 con 0x7f7964150490 2024-03-06T07:26:16.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.041+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7964156720 con 0x7f7964150490 2024-03-06T07:26:16.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.717+0000 7f794bfff640 1 -- 172.21.15.26:0/2528846818 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v153) v1 ==== 172+0+0 (secure 0 0 0) 0x7f795c07bcd0 con 0x7f7964150490 2024-03-06T07:26:16.720 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_data' created 2024-03-06T07:26:16.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f795006b4d0 msgr2=0x7f795006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f795006b4d0 0x7f795006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7954006fd0 tx=0x7f7954008040 comp rx=0 tx=0).stop 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964150490 msgr2=0x7f796413f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964150490 0x7f796413f200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f795c009d90 tx=0x7f795c004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 shutdown_connections 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f795006b4d0 0x7f795006d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7964150490 0x7f796413f200 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964072230 0x7f796413ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 --2- 172.21.15.26:0/2528846818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f796406f650 0x7f796414ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.720+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 >> 172.21.15.26:0/2528846818 conn(0x7f796406afc0 msgr2=0x7f7964156c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.721+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 shutdown_connections 2024-03-06T07:26:16.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.722+0000 7f79689a0640 1 -- 172.21.15.26:0/2528846818 wait complete. 2024-03-06T07:26:16.751 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-06T07:26:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.852+0000 7fa6a8a71640 1 Processor -- start 2024-03-06T07:26:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.852+0000 7fa6a8a71640 1 -- start start 2024-03-06T07:26:16.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.853+0000 7fa6a8a71640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6a40ca1a0 0x7fa6a40c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.853+0000 7fa6a8a71640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40c5310 0x7fa6a40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.853+0000 7fa6a8a71640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a4155d20 con 0x7fa6a40af720 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.853+0000 7fa6a8a71640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a40ad900 con 0x7fa6a40ca1a0 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.853+0000 7fa6a8a71640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a4155fc0 con 0x7fa6a40c5310 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6a40ca1a0 0x7fa6a40c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa6a40af720 0x7fa6a414ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54992/0 (socket says 172.21.15.26:54992) 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6a40ca1a0 0x7fa6a40c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44986/0 (socket says 172.21.15.26:44986) 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a259c640 1 -- 172.21.15.26:0/2749162622 learned_addr learned my addr 172.21.15.26:0/2749162622 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a2d9d640 1 --2- 172.21.15.26:0/2749162622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40c5310 0x7fa6a40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a1d9b640 1 -- 172.21.15.26:0/2749162622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40c5310 msgr2=0x7fa6a40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a1d9b640 1 --2- 172.21.15.26:0/2749162622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40c5310 0x7fa6a40c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a1d9b640 1 -- 172.21.15.26:0/2749162622 >> v1:172.21.15.26:6789/0 conn(0x7fa6a40af720 legacy=0x7fa6a414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.854+0000 7fa6a1d9b640 1 -- 172.21.15.26:0/2749162622 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40adb50 con 0x7fa6a40ca1a0 2024-03-06T07:26:16.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.855+0000 7fa6a1d9b640 1 --2- 172.21.15.26:0/2749162622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40ca1a0 0x7fa6a40c8b30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40ada80 tx=0x7fa68c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f28524bc3e0d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.855+0000 7fa6a159a640 1 -- 172.21.15.26:0/2749162622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c039070 con 0x7fa6a40ca1a0 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.855+0000 7fa6a159a640 1 -- 172.21.15.26:0/2749162622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c0316f0 con 0x7fa6a40ca1a0 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.855+0000 7fa6a159a640 1 -- 172.21.15.26:0/2749162622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c031ac0 con 0x7fa6a40ca1a0 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 -- 172.21.15.26:0/2749162622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40ca1a0 msgr2=0x7fa6a40c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/2749162622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40ca1a0 0x7fa6a40c8b30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40ada80 tx=0x7fa68c02b790 comp rx=0 tx=0).stop 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 -- 172.21.15.26:0/2749162622 shutdown_connections 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/2749162622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40ca1a0 0x7fa6a40c8b30 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/2749162622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40c5310 0x7fa6a40c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 -- 172.21.15.26:0/2749162622 >> 172.21.15.26:0/2749162622 conn(0x7fa6a40ac7e0 msgr2=0x7fa6a40acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 -- 172.21.15.26:0/2749162622 shutdown_connections 2024-03-06T07:26:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.856+0000 7fa6a8a71640 1 -- 172.21.15.26:0/2749162622 wait complete. 2024-03-06T07:26:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.857+0000 7fa6a8a71640 1 Processor -- start 2024-03-06T07:26:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.857+0000 7fa6a8a71640 1 -- start start 2024-03-06T07:26:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a8a71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40af720 0x7fa6a4154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a8a71640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6a40c5310 0x7fa6a406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a8a71640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40ca1a0 0x7fa6a414ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a8a71640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40adfb0 con 0x7fa6a40af720 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a8a71640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40ade30 con 0x7fa6a40c5310 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a8a71640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a414f4a0 con 0x7fa6a40ca1a0 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40af720 0x7fa6a4154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40af720 0x7fa6a4154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44998/0 (socket says 172.21.15.26:44998) 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 -- 172.21.15.26:0/920781955 learned_addr learned my addr 172.21.15.26:0/920781955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 -- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40ca1a0 msgr2=0x7fa6a414ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a2d9d640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40ca1a0 0x7fa6a414ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a1d9b640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6a40c5310 0x7fa6a406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40ca1a0 0x7fa6a414ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 -- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6a40c5310 msgr2=0x7fa6a406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6a40c5310 0x7fa6a406cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 -- 172.21.15.26:0/920781955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a406d170 con 0x7fa6a40af720 2024-03-06T07:26:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa6a259c640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40af720 0x7fa6a4154630 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00c3f0 tx=0x7fa69c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.858+0000 7fa69affd640 1 -- 172.21.15.26:0/920781955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c017070 con 0x7fa6a40af720 2024-03-06T07:26:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.859+0000 7fa69affd640 1 -- 172.21.15.26:0/920781955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c005ce0 con 0x7fa6a40af720 2024-03-06T07:26:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.859+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a406d6d0 con 0x7fa6a40af720 2024-03-06T07:26:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.859+0000 7fa69affd640 1 -- 172.21.15.26:0/920781955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c007310 con 0x7fa6a40af720 2024-03-06T07:26:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.859+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a406d3f0 con 0x7fa6a40af720 2024-03-06T07:26:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.860+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a404f330 con 0x7fa6a40af720 2024-03-06T07:26:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.860+0000 7fa69affd640 1 -- 172.21.15.26:0/920781955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa69c01a070 con 0x7fa6a40af720 2024-03-06T07:26:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.861+0000 7fa69affd640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68406b4d0 0x7fa68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.861+0000 7fa69affd640 1 -- 172.21.15.26:0/920781955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fa69c089970 con 0x7fa6a40af720 2024-03-06T07:26:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.861+0000 7fa6a1d9b640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68406b4d0 0x7fa68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.861+0000 7fa6a1d9b640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68406b4d0 0x7fa68406d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa68c036000 tx=0x7fa68c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:16.865+0000 7fa69affd640 1 -- 172.21.15.26:0/920781955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa6a406d170 con 0x7fa6a40af720 2024-03-06T07:26:17.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.115+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7fa6a40c9f80 con 0x7fa6a40af720 2024-03-06T07:26:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.737+0000 7fa69affd640 1 -- 172.21.15.26:0/920781955 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 25 and data pool 26 v139) v1 ==== 177+0+0 (secure 0 0 0) 0x7fa69c058400 con 0x7fa6a40af720 2024-03-06T07:26:17.741 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 25 and data pool 26 2024-03-06T07:26:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.747+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68406b4d0 msgr2=0x7fa68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.747+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68406b4d0 0x7fa68406d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa68c036000 tx=0x7fa68c008030 comp rx=0 tx=0).stop 2024-03-06T07:26:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.747+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40af720 msgr2=0x7fa6a4154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.747+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40af720 0x7fa6a4154630 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00c3f0 tx=0x7fa69c00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.748+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 shutdown_connections 2024-03-06T07:26:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.748+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68406b4d0 0x7fa68406d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.748+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6a40ca1a0 0x7fa6a414ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.748+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6a40c5310 0x7fa6a406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.748+0000 7fa6a8a71640 1 --2- 172.21.15.26:0/920781955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a40af720 0x7fa6a4154630 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.748+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 >> 172.21.15.26:0/920781955 conn(0x7fa6a40ac7e0 msgr2=0x7fa6a4149e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.750+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 shutdown_connections 2024-03-06T07:26:17.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.751+0000 7fa6a8a71640 1 -- 172.21.15.26:0/920781955 wait complete. 2024-03-06T07:26:17.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:17.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.874+0000 7fd37e31e640 1 Processor -- start 2024-03-06T07:26:17.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.874+0000 7fd37e31e640 1 -- start start 2024-03-06T07:26:17.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd37e31e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3781536c0 0x7fd37806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd37e31e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806b3a0 0x7fd37815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd37e31e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780a70f0 con 0x7fd37806afc0 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd37e31e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd378150a20 con 0x7fd3781536c0 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd37e31e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781565a0 con 0x7fd37806b3a0 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd3777fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3781536c0 0x7fd37806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd37c8bb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806b3a0 0x7fd37815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd377fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd37806afc0 0x7fd378152fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44606/0 (socket says 172.21.15.26:44606) 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.875+0000 7fd377fff640 1 -- 172.21.15.26:0/1640896139 learned_addr learned my addr 172.21.15.26:0/1640896139 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.876+0000 7fd3777fe640 1 -- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806b3a0 msgr2=0x7fd37815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.876+0000 7fd3777fe640 1 --2- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806b3a0 0x7fd37815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.876+0000 7fd3777fe640 1 -- 172.21.15.26:0/1640896139 >> v1:172.21.15.117:6789/0 conn(0x7fd37806afc0 legacy=0x7fd378152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.876+0000 7fd3777fe640 1 -- 172.21.15.26:0/1640896139 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3781567f0 con 0x7fd3781536c0 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.876+0000 7fd376ffd640 1 -- 172.21.15.26:0/1640896139 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975136279 0 0) 0x7fd3780a70f0 con 0x7fd37806afc0 2024-03-06T07:26:17.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.876+0000 7fd37c8bb640 1 --2- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806b3a0 0x7fd37815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd3777fe640 1 --2- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3781536c0 0x7fd37806f100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0097a0 tx=0x7fd36c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6578cce025ee5c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd376ffd640 1 -- 172.21.15.26:0/1640896139 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c039070 con 0x7fd3781536c0 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd376ffd640 1 -- 172.21.15.26:0/1640896139 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c0316f0 con 0x7fd3781536c0 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd376ffd640 1 -- 172.21.15.26:0/1640896139 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c031ac0 con 0x7fd3781536c0 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 -- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3781536c0 msgr2=0x7fd37806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 --2- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3781536c0 0x7fd37806f100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0097a0 tx=0x7fd36c02b790 comp rx=0 tx=0).stop 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 -- 172.21.15.26:0/1640896139 shutdown_connections 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 --2- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806b3a0 0x7fd37815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 --2- 172.21.15.26:0/1640896139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3781536c0 0x7fd37806f100 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 -- 172.21.15.26:0/1640896139 >> 172.21.15.26:0/1640896139 conn(0x7fd3780c4290 msgr2=0x7fd3780c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 -- 172.21.15.26:0/1640896139 shutdown_connections 2024-03-06T07:26:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.877+0000 7fd37e31e640 1 -- 172.21.15.26:0/1640896139 wait complete. 2024-03-06T07:26:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.878+0000 7fd37e31e640 1 Processor -- start 2024-03-06T07:26:17.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37e31e640 1 -- start start 2024-03-06T07:26:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37e31e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806afc0 0x7fd3780c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37e31e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37806b3a0 0x7fd3780c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37e31e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3781536c0 0x7fd378071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37e31e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd378156db0 con 0x7fd3781536c0 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37e31e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37806b810 con 0x7fd37806b3a0 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37e31e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780c7360 con 0x7fd37806afc0 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37c8bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3781536c0 0x7fd378071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37c8bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3781536c0 0x7fd378071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57924/0 (socket says 172.21.15.26:57924) 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.879+0000 7fd37c8bb640 1 -- 172.21.15.26:0/2714524766 learned_addr learned my addr 172.21.15.26:0/2714524766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd377fff640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806afc0 0x7fd3780c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd3777fe640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37806b3a0 0x7fd3780c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37c8bb640 1 -- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806afc0 msgr2=0x7fd3780c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37c8bb640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806afc0 0x7fd3780c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37c8bb640 1 -- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37806b3a0 msgr2=0x7fd3780c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37c8bb640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37806b3a0 0x7fd3780c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37c8bb640 1 -- 172.21.15.26:0/2714524766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3781529e0 con 0x7fd3781536c0 2024-03-06T07:26:17.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37c8bb640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3781536c0 0x7fd378071d40 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fd37000efc0 tx=0x7fd37000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd374ff9640 1 -- 172.21.15.26:0/2714524766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370018030 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd378152cc0 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd374ff9640 1 -- 172.21.15.26:0/2714524766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37000cdf0 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd374ff9640 1 -- 172.21.15.26:0/2714524766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370003bb0 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.880+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd378071820 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.881+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378152e90 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.882+0000 7fd374ff9640 1 -- 172.21.15.26:0/2714524766 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd37001b070 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.882+0000 7fd374ff9640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd36406b4d0 0x7fd36406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.882+0000 7fd374ff9640 1 -- 172.21.15.26:0/2714524766 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fd37008b000 con 0x7fd3781536c0 2024-03-06T07:26:17.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.882+0000 7fd377fff640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd36406b4d0 0x7fd36406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.883+0000 7fd377fff640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd36406b4d0 0x7fd36406d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd368007fd0 tx=0x7fd368007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:17.890+0000 7fd374ff9640 1 -- 172.21.15.26:0/2714524766 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd370010040 con 0x7fd3781536c0 2024-03-06T07:26:18.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.071+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd37804f330 con 0x7fd3781536c0 2024-03-06T07:26:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.072+0000 7fd374ff9640 1 -- 172.21.15.26:0/2714524766 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+20627 (secure 0 0 0) 0x7fd3700592d0 con 0x7fd3781536c0 2024-03-06T07:26:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:18.075 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":154,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:17.117856+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd36406b4d0 msgr2=0x7fd36406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd36406b4d0 0x7fd36406d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd368007fd0 tx=0x7fd368007de0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3781536c0 msgr2=0x7fd378071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3781536c0 0x7fd378071d40 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fd37000efc0 tx=0x7fd37000c490 comp rx=0 tx=0).stop 2024-03-06T07:26:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 shutdown_connections 2024-03-06T07:26:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd36406b4d0 0x7fd36406d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3781536c0 0x7fd378071d40 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37806b3a0 0x7fd3780c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 --2- 172.21.15.26:0/2714524766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd37806afc0 0x7fd3780c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 >> 172.21.15.26:0/2714524766 conn(0x7fd3780c4290 msgr2=0x7fd3780a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 shutdown_connections 2024-03-06T07:26:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.074+0000 7fd37e31e640 1 -- 172.21.15.26:0/2714524766 wait complete. 2024-03-06T07:26:18.088 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.187+0000 7f64ee3ac640 1 Processor -- start 2024-03-06T07:26:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.187+0000 7f64ee3ac640 1 -- start start 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.188+0000 7f64ee3ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f64e80c7180 0x7f64e8159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.188+0000 7f64ee3ac640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e80c7c60 con 0x7f64e80c88b0 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.188+0000 7f64ee3ac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e80c8060 con 0x7f64e81527b0 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.188+0000 7f64ee3ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e8159640 con 0x7f64e80c7180 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.188+0000 7f64ec949640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f64e80c88b0 0x7f64e80c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60658/0 (socket says 172.21.15.26:60658) 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.188+0000 7f64ec949640 1 -- 172.21.15.26:0/247720973 learned_addr learned my addr 172.21.15.26:0/247720973 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e77fe640 1 --2- 172.21.15.26:0/247720973 >> v2:172.21.15.117:3300/0 conn(0x7f64e80c7180 0x7f64e8159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e6ffd640 1 -- 172.21.15.26:0/247720973 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145875313 0 0) 0x7f64e0001490 con 0x7f64e80c88b0 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e6ffd640 1 -- 172.21.15.26:0/247720973 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d0003520 con 0x7f64e80c88b0 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e6ffd640 1 -- 172.21.15.26:0/247720973 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836191720 0 0) 0x7f64e80c8060 con 0x7f64e81527b0 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e6ffd640 1 -- 172.21.15.26:0/247720973 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e0001490 con 0x7f64e81527b0 2024-03-06T07:26:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e77fe640 1 -- 172.21.15.26:0/247720973 >> v1:172.21.15.117:6789/0 conn(0x7f64e81527b0 legacy=0x7f64e8155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e77fe640 1 -- 172.21.15.26:0/247720973 >> v1:172.21.15.26:6789/0 conn(0x7f64e80c88b0 legacy=0x7f64e80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.189+0000 7f64e77fe640 1 -- 172.21.15.26:0/247720973 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80c76a0 con 0x7f64e80c7180 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.190+0000 7f64e77fe640 1 --2- 172.21.15.26:0/247720973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e80c7180 0x7f64e8159100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0097a0 tx=0x7f64dc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c39a55f684a6d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.190+0000 7f64e6ffd640 1 -- 172.21.15.26:0/247720973 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc035020 con 0x7f64e80c7180 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.190+0000 7f64e6ffd640 1 -- 172.21.15.26:0/247720973 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc031830 con 0x7f64e80c7180 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.190+0000 7f64e6ffd640 1 -- 172.21.15.26:0/247720973 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc031c00 con 0x7f64e80c7180 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.191+0000 7f64ee3ac640 1 -- 172.21.15.26:0/247720973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e80c7180 msgr2=0x7f64e8159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.191+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/247720973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e80c7180 0x7f64e8159100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0097a0 tx=0x7f64dc02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.191+0000 7f64ee3ac640 1 -- 172.21.15.26:0/247720973 shutdown_connections 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.191+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/247720973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e80c7180 0x7f64e8159100 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.191+0000 7f64ee3ac640 1 -- 172.21.15.26:0/247720973 >> 172.21.15.26:0/247720973 conn(0x7f64e80ac560 msgr2=0x7f64e80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.191+0000 7f64ee3ac640 1 -- 172.21.15.26:0/247720973 shutdown_connections 2024-03-06T07:26:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.191+0000 7f64ee3ac640 1 -- 172.21.15.26:0/247720973 wait complete. 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.192+0000 7f64ee3ac640 1 Processor -- start 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.192+0000 7f64ee3ac640 1 -- start start 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64ee3ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64e80c7180 0x7f64e8154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64ee3ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64e80c88b0 0x7f64e81ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64ee3ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e81527b0 0x7f64e81ec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64ee3ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8155600 con 0x7f64e80c88b0 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64ee3ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8155320 con 0x7f64e81527b0 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64ee3ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8152b90 con 0x7f64e80c7180 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64e80c88b0 0x7f64e81ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64e80c88b0 0x7f64e81ec3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57940/0 (socket says 172.21.15.26:57940) 2024-03-06T07:26:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 -- 172.21.15.26:0/1852262209 learned_addr learned my addr 172.21.15.26:0/1852262209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64ec949640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e81527b0 0x7f64e81ec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e7fff640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64e80c7180 0x7f64e8154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 -- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64e80c7180 msgr2=0x7f64e8154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64e80c7180 0x7f64e8154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 -- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e81527b0 msgr2=0x7f64e81ec8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e81527b0 0x7f64e81ec8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.193+0000 7f64e77fe640 1 -- 172.21.15.26:0/1852262209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e8157430 con 0x7f64e80c88b0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.194+0000 7f64ec949640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e81527b0 0x7f64e81ec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.194+0000 7f64e77fe640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64e80c88b0 0x7f64e81ec3b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009890 tx=0x7f64dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.194+0000 7f64e4ff9640 1 -- 172.21.15.26:0/1852262209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc035040 con 0x7f64e80c88b0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.194+0000 7f64e4ff9640 1 -- 172.21.15.26:0/1852262209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc03b920 con 0x7f64e80c88b0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.194+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e8148d80 con 0x7f64e80c88b0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.194+0000 7f64e4ff9640 1 -- 172.21.15.26:0/1852262209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc03bd10 con 0x7f64e80c88b0 2024-03-06T07:26:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.194+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8157150 con 0x7f64e80c88b0 2024-03-06T07:26:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.195+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e804f330 con 0x7f64e80c88b0 2024-03-06T07:26:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.195+0000 7f64e4ff9640 1 -- 172.21.15.26:0/1852262209 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64dc038020 con 0x7f64e80c88b0 2024-03-06T07:26:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.196+0000 7f64e4ff9640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64d406b4d0 0x7f64d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.196+0000 7f64e4ff9640 1 -- 172.21.15.26:0/1852262209 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f64dc0ac6f0 con 0x7f64e80c88b0 2024-03-06T07:26:18.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.196+0000 7f64e7fff640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64d406b4d0 0x7f64d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.196+0000 7f64e7fff640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64d406b4d0 0x7f64d406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f64e80c4c80 tx=0x7f64d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.200+0000 7f64e4ff9640 1 -- 172.21.15.26:0/1852262209 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f64dc07b120 con 0x7f64e80c88b0 2024-03-06T07:26:18.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.382+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64e80c1950 con 0x7f64e80c88b0 2024-03-06T07:26:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.384+0000 7f64e4ff9640 1 -- 172.21.15.26:0/1852262209 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+20627 (secure 0 0 0) 0x7f64dc07aa20 con 0x7f64e80c88b0 2024-03-06T07:26:18.387 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:18.387 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":154,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:17.117856+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64d406b4d0 msgr2=0x7f64d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64d406b4d0 0x7f64d406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f64e80c4c80 tx=0x7f64d8009040 comp rx=0 tx=0).stop 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64e80c88b0 msgr2=0x7f64e81ec3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64e80c88b0 0x7f64e81ec3b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009890 tx=0x7f64dc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 shutdown_connections 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64d406b4d0 0x7f64d406d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e81527b0 0x7f64e81ec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64e80c88b0 0x7f64e81ec3b0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 --2- 172.21.15.26:0/1852262209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64e80c7180 0x7f64e8154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 >> 172.21.15.26:0/1852262209 conn(0x7f64e80ac560 msgr2=0x7f64e806b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 shutdown_connections 2024-03-06T07:26:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.386+0000 7f64ee3ac640 1 -- 172.21.15.26:0/1852262209 wait complete. 2024-03-06T07:26:18.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-06T07:26:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.501+0000 7f3685881640 1 Processor -- start 2024-03-06T07:26:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.501+0000 7f3685881640 1 -- start start 2024-03-06T07:26:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f3685881640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f36800b0070 0x7f3680155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f3685881640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 0x7f36800c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f3685881640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800b0450 con 0x7f368006ca70 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f3685881640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368006c020 con 0x7f36800b0070 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f3685881640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368006cf20 con 0x7f36800c5380 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f367e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f368006ca70 0x7f36800c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60660/0 (socket says 172.21.15.26:60660) 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f367e7fc640 1 -- 172.21.15.26:0/3564323187 learned_addr learned my addr 172.21.15.26:0/3564323187 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f367f7fe640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 0x7f36800c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f367effd640 1 --2- 172.21.15.26:0/3564323187 >> v2:172.21.15.117:3300/0 conn(0x7f36800b0070 0x7f3680155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238697959 0 0) 0x7f3670001110 con 0x7f368006ca70 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.502+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366c003520 con 0x7f368006ca70 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3856568241 0 0) 0x7f366c003520 con 0x7f368006ca70 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 msgr2=0x7f36800c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 0x7f36800c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 >> v2:172.21.15.117:3300/0 conn(0x7f36800b0070 msgr2=0x7f3680155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 --2- 172.21.15.26:0/3564323187 >> v2:172.21.15.117:3300/0 conn(0x7f36800b0070 0x7f3680155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800c77f0 con 0x7f368006ca70 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f3670001840 con 0x7f368006ca70 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367f7fe640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 0x7f36800c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 >> v1:172.21.15.26:6789/0 conn(0x7f368006ca70 legacy=0x7f36800c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 0x7f366c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c003ef0 0x7f366c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c004d40 0x7f366c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368006c020 con 0x7f366c003ef0 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368006cf20 con 0x7f366c004d40 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00b9c0 con 0x7f36800c5380 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367e7fc640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c003ef0 0x7f366c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367effd640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 0x7f366c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367f7fe640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c004d40 0x7f366c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367e7fc640 1 -- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 msgr2=0x7f366c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367e7fc640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36800c5380 0x7f366c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367e7fc640 1 -- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c004d40 msgr2=0x7f366c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367e7fc640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c004d40 0x7f366c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.503+0000 7f367e7fc640 1 -- 172.21.15.26:0/3564323187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c003520 con 0x7f366c003ef0 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.504+0000 7f367e7fc640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c003ef0 0x7f366c004800 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f367000c3e0 tx=0x7f366c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bae26389de69586 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.504+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367000c0e0 con 0x7f366c003ef0 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.504+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670030070 con 0x7f366c003ef0 2024-03-06T07:26:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.504+0000 7f367dffb640 1 -- 172.21.15.26:0/3564323187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36700058d0 con 0x7f366c003ef0 2024-03-06T07:26:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.505+0000 7f3685881640 1 -- 172.21.15.26:0/3564323187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c003ef0 msgr2=0x7f366c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.505+0000 7f3685881640 1 --2- 172.21.15.26:0/3564323187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c003ef0 0x7f366c004800 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f367000c3e0 tx=0x7f366c005830 comp rx=0 tx=0).stop 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.505+0000 7f367f7fe640 1 -- 172.21.15.26:0/3564323187 reap_dead start 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.505+0000 7f3685881640 1 -- 172.21.15.26:0/3564323187 shutdown_connections 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.505+0000 7f3685881640 1 -- 172.21.15.26:0/3564323187 >> 172.21.15.26:0/3564323187 conn(0x7f36800ac590 msgr2=0x7f36800ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.505+0000 7f3685881640 1 -- 172.21.15.26:0/3564323187 shutdown_connections 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.505+0000 7f3685881640 1 -- 172.21.15.26:0/3564323187 wait complete. 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.506+0000 7f3685881640 1 Processor -- start 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.506+0000 7f3685881640 1 -- start start 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3685881640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f366c003ef0 0x7f368014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3685881640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368014c0c0 0x7f3680144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3685881640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36801451f0 0x7f3680149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3685881640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368006c550 con 0x7f368014c0c0 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3685881640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368006c270 con 0x7f36801451f0 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3685881640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368006da80 con 0x7f366c003ef0 2024-03-06T07:26:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368014c0c0 0x7f3680144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368014c0c0 0x7f3680144cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57958/0 (socket says 172.21.15.26:57958) 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 -- 172.21.15.26:0/3836013188 learned_addr learned my addr 172.21.15.26:0/3836013188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367effd640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f366c003ef0 0x7f368014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367f7fe640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36801451f0 0x7f3680149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 -- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f366c003ef0 msgr2=0x7f368014cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f366c003ef0 0x7f368014cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 -- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36801451f0 msgr2=0x7f3680149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36801451f0 0x7f3680149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 -- 172.21.15.26:0/3836013188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368014bb00 con 0x7f368014c0c0 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367effd640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f366c003ef0 0x7f368014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367f7fe640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36801451f0 0x7f3680149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f367e7fc640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368014c0c0 0x7f3680144cb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f367000c500 tx=0x7f36800c7180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3677fff640 1 -- 172.21.15.26:0/3836013188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670004bf0 con 0x7f368014c0c0 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3677fff640 1 -- 172.21.15.26:0/3836013188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670004d90 con 0x7f368014c0c0 2024-03-06T07:26:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368014bde0 con 0x7f368014c0c0 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.507+0000 7f3677fff640 1 -- 172.21.15.26:0/3836013188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670007270 con 0x7f368014c0c0 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.508+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3680154d90 con 0x7f368014c0c0 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.509+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3680145600 con 0x7f368014c0c0 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.509+0000 7f3677fff640 1 -- 172.21.15.26:0/3836013188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3670007410 con 0x7f368014c0c0 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.509+0000 7f3677fff640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f366006b500 0x7f366006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.510+0000 7f3677fff640 1 -- 172.21.15.26:0/3836013188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f36700ac110 con 0x7f368014c0c0 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.510+0000 7f367effd640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f366006b500 0x7f366006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.510+0000 7f367effd640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f366006b500 0x7f366006d9c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f36800acae0 tx=0x7f3668008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.513+0000 7f3677fff640 1 -- 172.21.15.26:0/3836013188 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f36800b0450 con 0x7f368014c0c0 2024-03-06T07:26:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:18.890+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f368006d860 con 0x7f368014c0c0 2024-03-06T07:26:19.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.760+0000 7f3677fff640 1 -- 172.21.15.26:0/3836013188 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v142) v1 ==== 119+0+0 (secure 0 0 0) 0x7f367007ab40 con 0x7f368014c0c0 2024-03-06T07:26:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f366006b500 msgr2=0x7f366006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f366006b500 0x7f366006d9c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f36800acae0 tx=0x7f3668008040 comp rx=0 tx=0).stop 2024-03-06T07:26:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368014c0c0 msgr2=0x7f3680144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368014c0c0 0x7f3680144cb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f367000c500 tx=0x7f36800c7180 comp rx=0 tx=0).stop 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 shutdown_connections 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f366006b500 0x7f366006d9c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36801451f0 0x7f3680149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f368014c0c0 0x7f3680144cb0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 --2- 172.21.15.26:0/3836013188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f366c003ef0 0x7f368014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.762+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 >> 172.21.15.26:0/3836013188 conn(0x7f36800ac590 msgr2=0x7f36800c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.763+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 shutdown_connections 2024-03-06T07:26:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.763+0000 7f3685881640 1 -- 172.21.15.26:0/3836013188 wait complete. 2024-03-06T07:26:19.776 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:26:19.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-06T07:26:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.878+0000 7f87868cf640 1 Processor -- start 2024-03-06T07:26:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.878+0000 7f87868cf640 1 -- start start 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f87868cf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8780155c30 0x7f878006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f87868cf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87800b0450 con 0x7f87800b0070 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f87868cf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87801560e0 con 0x7f87800c5380 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f87868cf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878006cfb0 con 0x7f8780155c30 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87800b0070 0x7f87800c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60676/0 (socket says 172.21.15.26:60676) 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877ffff640 1 -- 172.21.15.26:0/2775904513 learned_addr learned my addr 172.21.15.26:0/2775904513 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877f7fe640 1 --2- 172.21.15.26:0/2775904513 >> v2:172.21.15.117:3300/0 conn(0x7f8780155c30 0x7f878006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877effd640 1 -- 172.21.15.26:0/2775904513 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603833225 0 0) 0x7f8770001110 con 0x7f87800b0070 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877effd640 1 -- 172.21.15.26:0/2775904513 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8768003520 con 0x7f87800b0070 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877effd640 1 -- 172.21.15.26:0/2775904513 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155715870 0 0) 0x7f87801560e0 con 0x7f87800c5380 2024-03-06T07:26:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877effd640 1 -- 172.21.15.26:0/2775904513 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8770001110 con 0x7f87800c5380 2024-03-06T07:26:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877f7fe640 1 -- 172.21.15.26:0/2775904513 >> v1:172.21.15.117:6789/0 conn(0x7f87800c5380 legacy=0x7f878014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877f7fe640 1 -- 172.21.15.26:0/2775904513 >> v1:172.21.15.26:6789/0 conn(0x7f87800b0070 legacy=0x7f87800c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.880+0000 7f877f7fe640 1 -- 172.21.15.26:0/2775904513 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87800c77f0 con 0x7f8780155c30 2024-03-06T07:26:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.881+0000 7f877f7fe640 1 --2- 172.21.15.26:0/2775904513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 0x7f878006ca70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f87740097a0 tx=0x7f877402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b75228800c3800 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.881+0000 7f877effd640 1 -- 172.21.15.26:0/2775904513 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774035020 con 0x7f8780155c30 2024-03-06T07:26:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.881+0000 7f877effd640 1 -- 172.21.15.26:0/2775904513 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8774031830 con 0x7f8780155c30 2024-03-06T07:26:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.881+0000 7f877effd640 1 -- 172.21.15.26:0/2775904513 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774031c00 con 0x7f8780155c30 2024-03-06T07:26:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.882+0000 7f87868cf640 1 -- 172.21.15.26:0/2775904513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 msgr2=0x7f878006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.882+0000 7f87868cf640 1 --2- 172.21.15.26:0/2775904513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 0x7f878006ca70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f87740097a0 tx=0x7f877402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.882+0000 7f87868cf640 1 -- 172.21.15.26:0/2775904513 shutdown_connections 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.882+0000 7f87868cf640 1 --2- 172.21.15.26:0/2775904513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 0x7f878006ca70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.882+0000 7f87868cf640 1 -- 172.21.15.26:0/2775904513 >> 172.21.15.26:0/2775904513 conn(0x7f87800ac590 msgr2=0x7f87800ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.882+0000 7f87868cf640 1 -- 172.21.15.26:0/2775904513 shutdown_connections 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.882+0000 7f87868cf640 1 -- 172.21.15.26:0/2775904513 wait complete. 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.883+0000 7f87868cf640 1 Processor -- start 2024-03-06T07:26:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.883+0000 7f87868cf640 1 -- start start 2024-03-06T07:26:19.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f87868cf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87800b0070 0x7f87800c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f87868cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87800c5380 0x7f87801ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f87868cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 0x7f8780154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f87868cf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878006c270 con 0x7f87800c5380 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f87868cf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878006c020 con 0x7f8780155c30 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f87868cf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878006c550 con 0x7f87800b0070 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87800c5380 0x7f87801ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87800c5380 0x7f87801ee7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57964/0 (socket says 172.21.15.26:57964) 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 -- 172.21.15.26:0/750625617 learned_addr learned my addr 172.21.15.26:0/750625617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f8784e6c640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 0x7f8780154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877ffff640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87800b0070 0x7f87800c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 -- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87800b0070 msgr2=0x7f87800c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87800b0070 0x7f87800c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 -- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 msgr2=0x7f8780154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 0x7f8780154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 -- 172.21.15.26:0/750625617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87801eed00 con 0x7f87800c5380 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877f7fe640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87800c5380 0x7f87801ee7c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f877402bee0 tx=0x7f87740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.884+0000 7f877cff9640 1 -- 172.21.15.26:0/750625617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774035040 con 0x7f87800c5380 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.885+0000 7f877cff9640 1 -- 172.21.15.26:0/750625617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877403b920 con 0x7f87800c5380 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.885+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87801596c0 con 0x7f87800c5380 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.885+0000 7f877cff9640 1 -- 172.21.15.26:0/750625617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877403bd10 con 0x7f87800c5380 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.885+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87801593e0 con 0x7f87800c5380 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.885+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878004f330 con 0x7f87800c5380 2024-03-06T07:26:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.886+0000 7f877cff9640 1 -- 172.21.15.26:0/750625617 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8774038020 con 0x7f87800c5380 2024-03-06T07:26:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.886+0000 7f877cff9640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876c06b500 0x7f876c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.887+0000 7f877cff9640 1 -- 172.21.15.26:0/750625617 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f87740ac5f0 con 0x7f87800c5380 2024-03-06T07:26:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.887+0000 7f877ffff640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876c06b500 0x7f876c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.887+0000 7f877ffff640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876c06b500 0x7f876c06d9c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8770000f60 tx=0x7f8770007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:19.891+0000 7f877cff9640 1 -- 172.21.15.26:0/750625617 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f87801eed00 con 0x7f87800c5380 2024-03-06T07:26:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.121+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f87800af5a0 con 0x7f87800c5380 2024-03-06T07:26:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.761+0000 7f877cff9640 1 -- 172.21.15.26:0/750625617 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v143) v1 ==== 123+0+0 (secure 0 0 0) 0x7f877407a930 con 0x7f87800c5380 2024-03-06T07:26:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876c06b500 msgr2=0x7f876c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876c06b500 0x7f876c06d9c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8770000f60 tx=0x7f8770007d50 comp rx=0 tx=0).stop 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87800c5380 msgr2=0x7f87801ee7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87800c5380 0x7f87801ee7c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f877402bee0 tx=0x7f87740046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 shutdown_connections 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f876c06b500 0x7f876c06d9c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8780155c30 0x7f8780154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87800c5380 0x7f87801ee7c0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 --2- 172.21.15.26:0/750625617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87800b0070 0x7f87800c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 >> 172.21.15.26:0/750625617 conn(0x7f87800ac590 msgr2=0x7f87800ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 shutdown_connections 2024-03-06T07:26:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.763+0000 7f87868cf640 1 -- 172.21.15.26:0/750625617 wait complete. 2024-03-06T07:26:20.776 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-06T07:26:20.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.882+0000 7f4f1cbac640 1 Processor -- start 2024-03-06T07:26:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.882+0000 7f4f1cbac640 1 -- start start 2024-03-06T07:26:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1cbac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f180c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1cbac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4f180c5330 0x7f4f180c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1cbac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f181595e0 con 0x7f4f18156cd0 2024-03-06T07:26:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1cbac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f181570b0 con 0x7f4f180c5330 2024-03-06T07:26:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1cbac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180c76f0 con 0x7f4f180bda70 2024-03-06T07:26:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f16d9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4f180c5330 0x7f4f180c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f15d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f180c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1659c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4f18156cd0 0x7f4f18159dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44634/0 (socket says 172.21.15.26:44634) 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1659c640 1 -- 172.21.15.26:0/672803658 learned_addr learned my addr 172.21.15.26:0/672803658 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f16d9d640 1 -- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 msgr2=0x7f4f180c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f16d9d640 1 --2- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f180c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f16d9d640 1 -- 172.21.15.26:0/672803658 >> v1:172.21.15.117:6789/0 conn(0x7f4f18156cd0 legacy=0x7f4f18159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f16d9d640 1 -- 172.21.15.26:0/672803658 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f180c79d0 con 0x7f4f180c5330 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f1559a640 1 -- 172.21.15.26:0/672803658 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093688327 0 0) 0x7f4f181595e0 con 0x7f4f18156cd0 2024-03-06T07:26:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.883+0000 7f4f15d9b640 1 --2- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f180c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:20.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f16d9d640 1 --2- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f180c5330 0x7f4f180c57c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac59a523b18eb97 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:20.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1559a640 1 -- 172.21.15.26:0/672803658 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08035020 con 0x7f4f180c5330 2024-03-06T07:26:20.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1559a640 1 -- 172.21.15.26:0/672803658 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f08031830 con 0x7f4f180c5330 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1559a640 1 -- 172.21.15.26:0/672803658 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08031c00 con 0x7f4f180c5330 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 -- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f180c5330 msgr2=0x7f4f180c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f180c5330 0x7f4f180c57c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 -- 172.21.15.26:0/672803658 shutdown_connections 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f180c5330 0x7f4f180c57c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/672803658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f180c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 -- 172.21.15.26:0/672803658 >> 172.21.15.26:0/672803658 conn(0x7f4f180ac560 msgr2=0x7f4f180ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 -- 172.21.15.26:0/672803658 shutdown_connections 2024-03-06T07:26:20.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.885+0000 7f4f1cbac640 1 -- 172.21.15.26:0/672803658 wait complete. 2024-03-06T07:26:20.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 Processor -- start 2024-03-06T07:26:20.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 -- start start 2024-03-06T07:26:20.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f1815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f180c5330 0x7f4f1815c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18156cd0 0x7f4f1815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1815a4d0 con 0x7f4f180c5330 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f181598a0 con 0x7f4f18156cd0 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1cbac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180be0c0 con 0x7f4f180bda70 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f15d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f180c5330 0x7f4f1815c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f15d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f180c5330 0x7f4f1815c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57968/0 (socket says 172.21.15.26:57968) 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f15d9b640 1 -- 172.21.15.26:0/474666315 learned_addr learned my addr 172.21.15.26:0/474666315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f16d9d640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18156cd0 0x7f4f1815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.887+0000 7f4f1659c640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f1815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f15d9b640 1 -- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 msgr2=0x7f4f1815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f15d9b640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f1815aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f15d9b640 1 -- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18156cd0 msgr2=0x7f4f1815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f15d9b640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18156cd0 0x7f4f1815cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f15d9b640 1 -- 172.21.15.26:0/474666315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f180c1810 con 0x7f4f180c5330 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f15d9b640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f180c5330 0x7f4f1815c8b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000ec10 tx=0x7f4f0000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:20.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f0effd640 1 -- 172.21.15.26:0/474666315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00017070 con 0x7f4f180c5330 2024-03-06T07:26:20.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f0effd640 1 -- 172.21.15.26:0/474666315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0000cc30 con 0x7f4f180c5330 2024-03-06T07:26:20.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f0effd640 1 -- 172.21.15.26:0/474666315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00013470 con 0x7f4f180c5330 2024-03-06T07:26:20.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f181f3390 con 0x7f4f180c5330 2024-03-06T07:26:20.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.888+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f181f25d0 con 0x7f4f180c5330 2024-03-06T07:26:20.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.889+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f180c6430 con 0x7f4f180c5330 2024-03-06T07:26:20.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.890+0000 7f4f0effd640 1 -- 172.21.15.26:0/474666315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f0001a070 con 0x7f4f180c5330 2024-03-06T07:26:20.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.890+0000 7f4f0effd640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ef806b4d0 0x7f4ef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.890+0000 7f4f0effd640 1 -- 172.21.15.26:0/474666315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f4f0008a630 con 0x7f4f180c5330 2024-03-06T07:26:20.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.891+0000 7f4f1659c640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ef806b4d0 0x7f4ef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.891+0000 7f4f1659c640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ef806b4d0 0x7f4ef806d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4f100012e0 tx=0x7f4f10001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:20.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:20.895+0000 7f4f0effd640 1 -- 172.21.15.26:0/474666315 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4f180c6430 con 0x7f4f180c5330 2024-03-06T07:26:21.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.112+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f180bd7e0 con 0x7f4f180c5330 2024-03-06T07:26:21.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.113+0000 7f4f0effd640 1 -- 172.21.15.26:0/474666315 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+6272 (secure 0 0 0) 0x7f4f000588e0 con 0x7f4f180c5330 2024-03-06T07:26:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:21.116 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":143,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:10.089399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8793":{"gid":8793,"name":"a","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6833/3029467671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3029467671},{"type":"v1","addr":"172.21.15.26:6833","nonce":3029467671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8793,"qdb_cluster":[8793]},"id":11},{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:20.123577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 143 2024-03-06T07:26:21.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ef806b4d0 msgr2=0x7f4ef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ef806b4d0 0x7f4ef806d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4f100012e0 tx=0x7f4f10001630 comp rx=0 tx=0).stop 2024-03-06T07:26:21.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f180c5330 msgr2=0x7f4f1815c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f180c5330 0x7f4f1815c8b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000ec10 tx=0x7f4f0000c520 comp rx=0 tx=0).stop 2024-03-06T07:26:21.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 shutdown_connections 2024-03-06T07:26:21.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4ef806b4d0 0x7f4ef806d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18156cd0 0x7f4f1815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f180c5330 0x7f4f1815c8b0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 --2- 172.21.15.26:0/474666315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f180bda70 0x7f4f1815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 >> 172.21.15.26:0/474666315 conn(0x7f4f180ac560 msgr2=0x7f4f18157450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 shutdown_connections 2024-03-06T07:26:21.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.115+0000 7f4f1cbac640 1 -- 172.21.15.26:0/474666315 wait complete. 2024-03-06T07:26:21.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:21.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.228+0000 7f5966f3e640 1 Processor -- start 2024-03-06T07:26:21.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.228+0000 7f5966f3e640 1 -- start start 2024-03-06T07:26:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5966f3e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5960152920 0x7f5960155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5966f3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c5330 0x7f59600c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5966f3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5960156180 con 0x7f59600c8a00 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5966f3e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59601587b0 con 0x7f5960152920 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5966f3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5960159630 con 0x7f59600c5330 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5964cda640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5960152920 0x7f5960155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f595ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f59600c8a00 0x7f5960158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60682/0 (socket says 172.21.15.26:60682) 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f595ffff640 1 -- 172.21.15.26:0/4087635128 learned_addr learned my addr 172.21.15.26:0/4087635128 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5964cda640 1 --2- 172.21.15.26:0/4087635128 >> v2:172.21.15.26:3300/0 conn(0x7f5960152920 0x7f5960155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57974/0 (socket says 172.21.15.26:57974) 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f59654db640 1 --2- 172.21.15.26:0/4087635128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c5330 0x7f59600c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5964cda640 1 -- 172.21.15.26:0/4087635128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c5330 msgr2=0x7f59600c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5964cda640 1 --2- 172.21.15.26:0/4087635128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c5330 0x7f59600c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5964cda640 1 -- 172.21.15.26:0/4087635128 >> v1:172.21.15.26:6789/0 conn(0x7f59600c8a00 legacy=0x7f5960158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f595f7fe640 1 -- 172.21.15.26:0/4087635128 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363451349 0 0) 0x7f594c001110 con 0x7f59600c8a00 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f5964cda640 1 -- 172.21.15.26:0/4087635128 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600c7760 con 0x7f5960152920 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.229+0000 7f59654db640 1 --2- 172.21.15.26:0/4087635128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c5330 0x7f59600c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.230+0000 7f5964cda640 1 --2- 172.21.15.26:0/4087635128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f5960155c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f59540097a0 tx=0x7f595402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75443f2bd24188c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.230+0000 7f595f7fe640 1 -- 172.21.15.26:0/4087635128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540315a0 con 0x7f5960152920 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.230+0000 7f595f7fe640 1 -- 172.21.15.26:0/4087635128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954037070 con 0x7f5960152920 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.230+0000 7f595f7fe640 1 -- 172.21.15.26:0/4087635128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954031960 con 0x7f5960152920 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 -- 172.21.15.26:0/4087635128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 msgr2=0x7f5960155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 --2- 172.21.15.26:0/4087635128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f5960155c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f59540097a0 tx=0x7f595402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 -- 172.21.15.26:0/4087635128 shutdown_connections 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 --2- 172.21.15.26:0/4087635128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c5330 0x7f59600c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 --2- 172.21.15.26:0/4087635128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f5960155c40 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 -- 172.21.15.26:0/4087635128 >> 172.21.15.26:0/4087635128 conn(0x7f59600ac560 msgr2=0x7f59600ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 -- 172.21.15.26:0/4087635128 shutdown_connections 2024-03-06T07:26:21.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.231+0000 7f5966f3e640 1 -- 172.21.15.26:0/4087635128 wait complete. 2024-03-06T07:26:21.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.232+0000 7f5966f3e640 1 Processor -- start 2024-03-06T07:26:21.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.232+0000 7f5966f3e640 1 -- start start 2024-03-06T07:26:21.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f5966f3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59600c5330 0x7f59601ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f5966f3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c8a00 0x7f59601e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f5966f3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f59600bd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f5966f3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5960158a00 con 0x7f5960152920 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f5966f3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c7d20 con 0x7f59600c5330 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f5966f3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596006d030 con 0x7f59600c8a00 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f59600bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f59600bd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57984/0 (socket says 172.21.15.26:57984) 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 -- 172.21.15.26:0/1902143525 learned_addr learned my addr 172.21.15.26:0/1902143525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f595ffff640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c8a00 0x7f59601e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f5964cda640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59600c5330 0x7f59601ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 -- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c8a00 msgr2=0x7f59601e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c8a00 0x7f59601e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 -- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59600c5330 msgr2=0x7f59601ebfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59600c5330 0x7f59601ebfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 -- 172.21.15.26:0/1902143525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960154490 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f595ffff640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c8a00 0x7f59601e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.233+0000 7f59654db640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f59600bd750 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f595800efc0 tx=0x7f595800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.234+0000 7f595d7fa640 1 -- 172.21.15.26:0/1902143525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958018030 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.234+0000 7f595d7fa640 1 -- 172.21.15.26:0/1902143525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595800cdf0 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.234+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596014cbc0 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.234+0000 7f595d7fa640 1 -- 172.21.15.26:0/1902143525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958003bb0 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.234+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59601533b0 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.235+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596004f330 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.235+0000 7f595d7fa640 1 -- 172.21.15.26:0/1902143525 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5958005d10 con 0x7f5960152920 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.236+0000 7f595d7fa640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f594406b500 0x7f594406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.236+0000 7f595d7fa640 1 -- 172.21.15.26:0/1902143525 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f595808ae70 con 0x7f5960152920 2024-03-06T07:26:21.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.236+0000 7f5964cda640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f594406b500 0x7f594406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.236+0000 7f5964cda640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f594406b500 0x7f594406d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5954036000 tx=0x7f5954008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.240+0000 7f595d7fa640 1 -- 172.21.15.26:0/1902143525 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5958022070 con 0x7f5960152920 2024-03-06T07:26:21.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.421+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59601535b0 con 0x7f5960152920 2024-03-06T07:26:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.423+0000 7f595d7fa640 1 -- 172.21.15.26:0/1902143525 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+20614 (secure 0 0 0) 0x7f5958010040 con 0x7f5960152920 2024-03-06T07:26:21.426 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:21.426 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":155,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:18.738673+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f594406b500 msgr2=0x7f594406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f594406b500 0x7f594406d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5954036000 tx=0x7f5954008030 comp rx=0 tx=0).stop 2024-03-06T07:26:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 msgr2=0x7f59600bd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f59600bd750 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f595800efc0 tx=0x7f595800c490 comp rx=0 tx=0).stop 2024-03-06T07:26:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 shutdown_connections 2024-03-06T07:26:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f594406b500 0x7f594406d9c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5960152920 0x7f59600bd750 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59600c8a00 0x7f59601e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 --2- 172.21.15.26:0/1902143525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59600c5330 0x7f59601ebfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 >> 172.21.15.26:0/1902143525 conn(0x7f59600ac560 msgr2=0x7f5960150140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 shutdown_connections 2024-03-06T07:26:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.425+0000 7f5966f3e640 1 -- 172.21.15.26:0/1902143525 wait complete. 2024-03-06T07:26:21.439 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:26:21.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.541+0000 7fd8b9f9a640 1 Processor -- start 2024-03-06T07:26:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b9f9a640 1 -- start start 2024-03-06T07:26:21.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b9f9a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8b4152920 0x7fd8b4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b9f9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b406afc0 con 0x7fd8b40c2190 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b9f9a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b406b2a0 con 0x7fd8b40c2620 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b9f9a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b4156160 con 0x7fd8b4152920 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b2ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd8b40c2190 0x7fd8b40c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60690/0 (socket says 172.21.15.26:60690) 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b2ffd640 1 -- 172.21.15.26:0/2586836643 learned_addr learned my addr 172.21.15.26:0/2586836643 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.542+0000 7fd8b37fe640 1 --2- 172.21.15.26:0/2586836643 >> v2:172.21.15.117:3300/0 conn(0x7fd8b4152920 0x7fd8b4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.543+0000 7fd8b27fc640 1 -- 172.21.15.26:0/2586836643 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284008534 0 0) 0x7fd8b406afc0 con 0x7fd8b40c2190 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.543+0000 7fd8b37fe640 1 -- 172.21.15.26:0/2586836643 >> v1:172.21.15.117:6789/0 conn(0x7fd8b40c2620 legacy=0x7fd8b415fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.543+0000 7fd8b37fe640 1 -- 172.21.15.26:0/2586836643 >> v1:172.21.15.26:6789/0 conn(0x7fd8b40c2190 legacy=0x7fd8b40c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.543+0000 7fd8b37fe640 1 -- 172.21.15.26:0/2586836643 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b406b580 con 0x7fd8b4152920 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b37fe640 1 --2- 172.21.15.26:0/2586836643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 0x7fd8b4155c20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40097a0 tx=0x7fd8a402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dfb43adbe43fab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b27fc640 1 -- 172.21.15.26:0/2586836643 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4034070 con 0x7fd8b4152920 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b27fc640 1 -- 172.21.15.26:0/2586836643 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a4030610 con 0x7fd8b4152920 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b27fc640 1 -- 172.21.15.26:0/2586836643 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40309e0 con 0x7fd8b4152920 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/2586836643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 msgr2=0x7fd8b4155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/2586836643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 0x7fd8b4155c20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40097a0 tx=0x7fd8a402b960 comp rx=0 tx=0).stop 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/2586836643 shutdown_connections 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/2586836643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 0x7fd8b4155c20 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/2586836643 >> 172.21.15.26:0/2586836643 conn(0x7fd8b40c4290 msgr2=0x7fd8b40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/2586836643 shutdown_connections 2024-03-06T07:26:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.544+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/2586836643 wait complete. 2024-03-06T07:26:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.545+0000 7fd8b9f9a640 1 Processor -- start 2024-03-06T07:26:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b9f9a640 1 -- start start 2024-03-06T07:26:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b9f9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8b40c2190 0x7fd8b4157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b9f9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b40c2620 0x7fd8b4154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b9f9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 0x7fd8b4156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b9f9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40a7100 con 0x7fd8b40c2620 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b9f9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40c0d60 con 0x7fd8b4152920 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b9f9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40a7280 con 0x7fd8b40c2190 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b40c2620 0x7fd8b4154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b40c2620 0x7fd8b4154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57996/0 (socket says 172.21.15.26:57996) 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b2ffd640 1 -- 172.21.15.26:0/1789105522 learned_addr learned my addr 172.21.15.26:0/1789105522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b37fe640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8b40c2190 0x7fd8b4157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.546+0000 7fd8b3fff640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 0x7fd8b4156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b2ffd640 1 -- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8b40c2190 msgr2=0x7fd8b4157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b2ffd640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8b40c2190 0x7fd8b4157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b2ffd640 1 -- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 msgr2=0x7fd8b4156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b2ffd640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 0x7fd8b4156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b2ffd640 1 -- 172.21.15.26:0/1789105522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4156590 con 0x7fd8b40c2620 2024-03-06T07:26:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b2ffd640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b40c2620 0x7fd8b4154870 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c4a0 tx=0x7fd8a800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd89bfff640 1 -- 172.21.15.26:0/1789105522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8018070 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd89bfff640 1 -- 172.21.15.26:0/1789105522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a80074e0 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b4157fb0 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd89bfff640 1 -- 172.21.15.26:0/1789105522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8004a20 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.547+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b40ad510 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.548+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b404f330 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.549+0000 7fd89bfff640 1 -- 172.21.15.26:0/1789105522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8a801b070 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.549+0000 7fd89bfff640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8a006b4d0 0x7fd8a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.549+0000 7fd89bfff640 1 -- 172.21.15.26:0/1789105522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fd8a8089940 con 0x7fd8b40c2620 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.549+0000 7fd8b37fe640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8a006b4d0 0x7fd8a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.550+0000 7fd8b37fe640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8a006b4d0 0x7fd8a006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4036000 tx=0x7fd8a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.553+0000 7fd89bfff640 1 -- 172.21.15.26:0/1789105522 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd8a8057c90 con 0x7fd8b40c2620 2024-03-06T07:26:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.731+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8b4068460 con 0x7fd8a006b4d0 2024-03-06T07:26:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.748+0000 7fd89bfff640 1 -- 172.21.15.26:0/1789105522 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+491523 (secure 0 0 0) 0x7fd8b4068460 con 0x7fd8a006b4d0 2024-03-06T07:26:21.756 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:21.760 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8a006b4d0 msgr2=0x7fd8a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8a006b4d0 0x7fd8a006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4036000 tx=0x7fd8a4008030 comp rx=0 tx=0).stop 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b40c2620 msgr2=0x7fd8b4154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b40c2620 0x7fd8b4154870 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c4a0 tx=0x7fd8a800c960 comp rx=0 tx=0).stop 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 shutdown_connections 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd8a006b4d0 0x7fd8a006d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8b4152920 0x7fd8b4156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8b40c2620 0x7fd8b4154870 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 --2- 172.21.15.26:0/1789105522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8b40c2190 0x7fd8b4157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 >> 172.21.15.26:0/1789105522 conn(0x7fd8b40c4290 msgr2=0x7fd8b414ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 shutdown_connections 2024-03-06T07:26:21.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.752+0000 7fd8b9f9a640 1 -- 172.21.15.26:0/1789105522 wait complete. 2024-03-06T07:26:21.766 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-03-06T07:26:21.109326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":316,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264924,"kb_used_data":15452,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709796,"statfs":{"total":773094113280,"available":772822831104,"internally_reserved":0,"allocated":15822848,"data_stored":8758929,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.391395"},"pg_stats":[{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882598+0000","last_change":"2024-03-06T07:26:17.751441+0000","last_active":"2024-03-06T07:26:18.882598+0000","last_peered":"2024-03-06T07:26:18.882598+0000","last_clean":"2024-03-06T07:26:18.882598+0000","last_became_active":"2024-03-06T07:26:17.751024+0000","last_became_peered":"2024-03-06T07:26:17.751024+0000","last_unstale":"2024-03-06T07:26:18.882598+0000","last_undegraded":"2024-03-06T07:26:18.882598+0000","last_fullsized":"2024-03-06T07:26:18.882598+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:30.905978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747351+0000","last_change":"2024-03-06T07:26:07.695841+0000","last_active":"2024-03-06T07:26:18.747351+0000","last_peered":"2024-03-06T07:26:18.747351+0000","last_clean":"2024-03-06T07:26:18.747351+0000","last_became_active":"2024-03-06T07:26:07.695553+0000","last_became_peered":"2024-03-06T07:26:07.695553+0000","last_unstale":"2024-03-06T07:26:18.747351+0000","last_undegraded":"2024-03-06T07:26:18.747351+0000","last_fullsized":"2024-03-06T07:26:18.747351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:17.847841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.758292+0000","last_change":"2024-03-06T07:26:17.752487+0000","last_active":"2024-03-06T07:26:17.758292+0000","last_peered":"2024-03-06T07:26:17.758292+0000","last_clean":"2024-03-06T07:26:17.758292+0000","last_became_active":"2024-03-06T07:26:17.752105+0000","last_became_peered":"2024-03-06T07:26:17.752105+0000","last_unstale":"2024-03-06T07:26:17.758292+0000","last_undegraded":"2024-03-06T07:26:17.758292+0000","last_fullsized":"2024-03-06T07:26:17.758292+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:26:27.566777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745195+0000","last_change":"2024-03-06T07:26:07.698101+0000","last_active":"2024-03-06T07:26:18.745195+0000","last_peered":"2024-03-06T07:26:18.745195+0000","last_clean":"2024-03-06T07:26:18.745195+0000","last_became_active":"2024-03-06T07:26:07.697796+0000","last_became_peered":"2024-03-06T07:26:07.697796+0000","last_unstale":"2024-03-06T07:26:18.745195+0000","last_undegraded":"2024-03-06T07:26:18.745195+0000","last_fullsized":"2024-03-06T07:26:18.745195+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:04.892999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749443+0000","last_change":"2024-03-06T07:26:17.739275+0000","last_active":"2024-03-06T07:26:19.749443+0000","last_peered":"2024-03-06T07:26:19.749443+0000","last_clean":"2024-03-06T07:26:19.749443+0000","last_became_active":"2024-03-06T07:26:17.739019+0000","last_became_peered":"2024-03-06T07:26:17.739019+0000","last_unstale":"2024-03-06T07:26:19.749443+0000","last_undegraded":"2024-03-06T07:26:19.749443+0000","last_fullsized":"2024-03-06T07:26:19.749443+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:10:47.968925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747261+0000","last_change":"2024-03-06T07:26:07.691124+0000","last_active":"2024-03-06T07:26:18.747261+0000","last_peered":"2024-03-06T07:26:18.747261+0000","last_clean":"2024-03-06T07:26:18.747261+0000","last_became_active":"2024-03-06T07:26:07.690832+0000","last_became_peered":"2024-03-06T07:26:07.690832+0000","last_unstale":"2024-03-06T07:26:18.747261+0000","last_undegraded":"2024-03-06T07:26:18.747261+0000","last_fullsized":"2024-03-06T07:26:18.747261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:29:40.327108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749443+0000","last_change":"2024-03-06T07:26:17.751375+0000","last_active":"2024-03-06T07:26:19.749443+0000","last_peered":"2024-03-06T07:26:19.749443+0000","last_clean":"2024-03-06T07:26:19.749443+0000","last_became_active":"2024-03-06T07:26:17.751123+0000","last_became_peered":"2024-03-06T07:26:17.751123+0000","last_unstale":"2024-03-06T07:26:19.749443+0000","last_undegraded":"2024-03-06T07:26:19.749443+0000","last_fullsized":"2024-03-06T07:26:19.749443+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:43.640582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747209+0000","last_change":"2024-03-06T07:26:07.688926+0000","last_active":"2024-03-06T07:26:18.747209+0000","last_peered":"2024-03-06T07:26:18.747209+0000","last_clean":"2024-03-06T07:26:18.747209+0000","last_became_active":"2024-03-06T07:26:07.688684+0000","last_became_peered":"2024-03-06T07:26:07.688684+0000","last_unstale":"2024-03-06T07:26:18.747209+0000","last_undegraded":"2024-03-06T07:26:18.747209+0000","last_fullsized":"2024-03-06T07:26:18.747209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:31.513519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001539+0000","last_change":"2024-03-06T07:26:17.749303+0000","last_active":"2024-03-06T07:26:19.001539+0000","last_peered":"2024-03-06T07:26:19.001539+0000","last_clean":"2024-03-06T07:26:19.001539+0000","last_became_active":"2024-03-06T07:26:17.749000+0000","last_became_peered":"2024-03-06T07:26:17.749000+0000","last_unstale":"2024-03-06T07:26:19.001539+0000","last_undegraded":"2024-03-06T07:26:19.001539+0000","last_fullsized":"2024-03-06T07:26:19.001539+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:31:51.703291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738387+0000","last_change":"2024-03-06T07:26:07.708495+0000","last_active":"2024-03-06T07:26:17.738387+0000","last_peered":"2024-03-06T07:26:17.738387+0000","last_clean":"2024-03-06T07:26:17.738387+0000","last_became_active":"2024-03-06T07:26:07.708183+0000","last_became_peered":"2024-03-06T07:26:07.708183+0000","last_unstale":"2024-03-06T07:26:17.738387+0000","last_undegraded":"2024-03-06T07:26:17.738387+0000","last_fullsized":"2024-03-06T07:26:17.738387+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:00:41.851158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744919+0000","last_change":"2024-03-06T07:26:17.754154+0000","last_active":"2024-03-06T07:26:18.744919+0000","last_peered":"2024-03-06T07:26:18.744919+0000","last_clean":"2024-03-06T07:26:18.744919+0000","last_became_active":"2024-03-06T07:26:17.753797+0000","last_became_peered":"2024-03-06T07:26:17.753797+0000","last_unstale":"2024-03-06T07:26:18.744919+0000","last_undegraded":"2024-03-06T07:26:18.744919+0000","last_fullsized":"2024-03-06T07:26:18.744919+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:50:45.483679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744637+0000","last_change":"2024-03-06T07:26:07.696015+0000","last_active":"2024-03-06T07:26:18.744637+0000","last_peered":"2024-03-06T07:26:18.744637+0000","last_clean":"2024-03-06T07:26:18.744637+0000","last_became_active":"2024-03-06T07:26:07.695708+0000","last_became_peered":"2024-03-06T07:26:07.695708+0000","last_unstale":"2024-03-06T07:26:18.744637+0000","last_undegraded":"2024-03-06T07:26:18.744637+0000","last_fullsized":"2024-03-06T07:26:18.744637+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:15.500034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806911+0000","last_change":"2024-03-06T07:26:17.746370+0000","last_active":"2024-03-06T07:26:18.806911+0000","last_peered":"2024-03-06T07:26:18.806911+0000","last_clean":"2024-03-06T07:26:18.806911+0000","last_became_active":"2024-03-06T07:26:17.744994+0000","last_became_peered":"2024-03-06T07:26:17.744994+0000","last_unstale":"2024-03-06T07:26:18.806911+0000","last_undegraded":"2024-03-06T07:26:18.806911+0000","last_fullsized":"2024-03-06T07:26:18.806911+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:24.779563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749074+0000","last_change":"2024-03-06T07:26:07.699338+0000","last_active":"2024-03-06T07:26:19.749074+0000","last_peered":"2024-03-06T07:26:19.749074+0000","last_clean":"2024-03-06T07:26:19.749074+0000","last_became_active":"2024-03-06T07:26:07.699065+0000","last_became_peered":"2024-03-06T07:26:07.699065+0000","last_unstale":"2024-03-06T07:26:19.749074+0000","last_undegraded":"2024-03-06T07:26:19.749074+0000","last_fullsized":"2024-03-06T07:26:19.749074+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806423+0000","last_change":"2024-03-06T07:26:17.747306+0000","last_active":"2024-03-06T07:26:18.806423+0000","last_peered":"2024-03-06T07:26:18.806423+0000","last_clean":"2024-03-06T07:26:18.806423+0000","last_became_active":"2024-03-06T07:26:17.746888+0000","last_became_peered":"2024-03-06T07:26:17.746888+0000","last_unstale":"2024-03-06T07:26:18.806423+0000","last_undegraded":"2024-03-06T07:26:18.806423+0000","last_fullsized":"2024-03-06T07:26:18.806423+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:27:28.288152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748699+0000","last_change":"2024-03-06T07:26:07.700571+0000","last_active":"2024-03-06T07:26:19.748699+0000","last_peered":"2024-03-06T07:26:19.748699+0000","last_clean":"2024-03-06T07:26:19.748699+0000","last_became_active":"2024-03-06T07:26:07.698195+0000","last_became_peered":"2024-03-06T07:26:07.698195+0000","last_unstale":"2024-03-06T07:26:19.748699+0000","last_undegraded":"2024-03-06T07:26:19.748699+0000","last_fullsized":"2024-03-06T07:26:19.748699+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.762269+0000","last_change":"2024-03-06T07:26:17.757260+0000","last_active":"2024-03-06T07:26:17.762269+0000","last_peered":"2024-03-06T07:26:17.762269+0000","last_clean":"2024-03-06T07:26:17.762269+0000","last_became_active":"2024-03-06T07:26:17.756964+0000","last_became_peered":"2024-03-06T07:26:17.756964+0000","last_unstale":"2024-03-06T07:26:17.762269+0000","last_undegraded":"2024-03-06T07:26:17.762269+0000","last_fullsized":"2024-03-06T07:26:17.762269+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:52.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749521+0000","last_change":"2024-03-06T07:26:07.700691+0000","last_active":"2024-03-06T07:26:19.749521+0000","last_peered":"2024-03-06T07:26:19.749521+0000","last_clean":"2024-03-06T07:26:19.749521+0000","last_became_active":"2024-03-06T07:26:07.700158+0000","last_became_peered":"2024-03-06T07:26:07.700158+0000","last_unstale":"2024-03-06T07:26:19.749521+0000","last_undegraded":"2024-03-06T07:26:19.749521+0000","last_fullsized":"2024-03-06T07:26:19.749521+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:17.093387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.758270+0000","last_change":"2024-03-06T07:26:17.752361+0000","last_active":"2024-03-06T07:26:17.758270+0000","last_peered":"2024-03-06T07:26:17.758270+0000","last_clean":"2024-03-06T07:26:17.758270+0000","last_became_active":"2024-03-06T07:26:17.751787+0000","last_became_peered":"2024-03-06T07:26:17.751787+0000","last_unstale":"2024-03-06T07:26:17.758270+0000","last_undegraded":"2024-03-06T07:26:17.758270+0000","last_fullsized":"2024-03-06T07:26:17.758270+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:04.831471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749559+0000","last_change":"2024-03-06T07:26:07.700442+0000","last_active":"2024-03-06T07:26:19.749559+0000","last_peered":"2024-03-06T07:26:19.749559+0000","last_clean":"2024-03-06T07:26:19.749559+0000","last_became_active":"2024-03-06T07:26:07.700116+0000","last_became_peered":"2024-03-06T07:26:07.700116+0000","last_unstale":"2024-03-06T07:26:19.749559+0000","last_undegraded":"2024-03-06T07:26:19.749559+0000","last_fullsized":"2024-03-06T07:26:19.749559+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:30.446061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749470+0000","last_change":"2024-03-06T07:26:17.753333+0000","last_active":"2024-03-06T07:26:19.749470+0000","last_peered":"2024-03-06T07:26:19.749470+0000","last_clean":"2024-03-06T07:26:19.749470+0000","last_became_active":"2024-03-06T07:26:17.753044+0000","last_became_peered":"2024-03-06T07:26:17.753044+0000","last_unstale":"2024-03-06T07:26:19.749470+0000","last_undegraded":"2024-03-06T07:26:19.749470+0000","last_fullsized":"2024-03-06T07:26:19.749470+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:02.145666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.745875+0000","last_change":"2024-03-06T07:26:07.707356+0000","last_active":"2024-03-06T07:26:17.745875+0000","last_peered":"2024-03-06T07:26:17.745875+0000","last_clean":"2024-03-06T07:26:17.745875+0000","last_became_active":"2024-03-06T07:26:07.707082+0000","last_became_peered":"2024-03-06T07:26:07.707082+0000","last_unstale":"2024-03-06T07:26:17.745875+0000","last_undegraded":"2024-03-06T07:26:17.745875+0000","last_fullsized":"2024-03-06T07:26:17.745875+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806879+0000","last_change":"2024-03-06T07:26:17.750773+0000","last_active":"2024-03-06T07:26:18.806879+0000","last_peered":"2024-03-06T07:26:18.806879+0000","last_clean":"2024-03-06T07:26:18.806879+0000","last_became_active":"2024-03-06T07:26:17.750338+0000","last_became_peered":"2024-03-06T07:26:17.750338+0000","last_unstale":"2024-03-06T07:26:18.806879+0000","last_undegraded":"2024-03-06T07:26:18.806879+0000","last_fullsized":"2024-03-06T07:26:18.806879+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:15.433217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745321+0000","last_change":"2024-03-06T07:26:07.694337+0000","last_active":"2024-03-06T07:26:18.745321+0000","last_peered":"2024-03-06T07:26:18.745321+0000","last_clean":"2024-03-06T07:26:18.745321+0000","last_became_active":"2024-03-06T07:26:07.694050+0000","last_became_peered":"2024-03-06T07:26:07.694050+0000","last_unstale":"2024-03-06T07:26:18.745321+0000","last_undegraded":"2024-03-06T07:26:18.745321+0000","last_fullsized":"2024-03-06T07:26:18.745321+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:46.775059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749128+0000","last_change":"2024-03-06T07:26:17.748683+0000","last_active":"2024-03-06T07:26:19.749128+0000","last_peered":"2024-03-06T07:26:19.749128+0000","last_clean":"2024-03-06T07:26:19.749128+0000","last_became_active":"2024-03-06T07:26:17.748402+0000","last_became_peered":"2024-03-06T07:26:17.748402+0000","last_unstale":"2024-03-06T07:26:19.749128+0000","last_undegraded":"2024-03-06T07:26:19.749128+0000","last_fullsized":"2024-03-06T07:26:19.749128+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:59:42.208048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806908+0000","last_change":"2024-03-06T07:26:07.696706+0000","last_active":"2024-03-06T07:26:18.806908+0000","last_peered":"2024-03-06T07:26:18.806908+0000","last_clean":"2024-03-06T07:26:18.806908+0000","last_became_active":"2024-03-06T07:26:07.696438+0000","last_became_peered":"2024-03-06T07:26:07.696438+0000","last_unstale":"2024-03-06T07:26:18.806908+0000","last_undegraded":"2024-03-06T07:26:18.806908+0000","last_fullsized":"2024-03-06T07:26:18.806908+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:32:23.418950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806414+0000","last_change":"2024-03-06T07:26:17.747667+0000","last_active":"2024-03-06T07:26:18.806414+0000","last_peered":"2024-03-06T07:26:18.806414+0000","last_clean":"2024-03-06T07:26:18.806414+0000","last_became_active":"2024-03-06T07:26:17.747376+0000","last_became_peered":"2024-03-06T07:26:17.747376+0000","last_unstale":"2024-03-06T07:26:18.806414+0000","last_undegraded":"2024-03-06T07:26:18.806414+0000","last_fullsized":"2024-03-06T07:26:18.806414+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:48.192361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743785+0000","last_change":"2024-03-06T07:26:07.704226+0000","last_active":"2024-03-06T07:26:17.743785+0000","last_peered":"2024-03-06T07:26:17.743785+0000","last_clean":"2024-03-06T07:26:17.743785+0000","last_became_active":"2024-03-06T07:26:07.703839+0000","last_became_peered":"2024-03-06T07:26:07.703839+0000","last_unstale":"2024-03-06T07:26:17.743785+0000","last_undegraded":"2024-03-06T07:26:17.743785+0000","last_fullsized":"2024-03-06T07:26:17.743785+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:52.356683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746419+0000","last_change":"2024-03-06T07:26:17.752169+0000","last_active":"2024-03-06T07:26:18.746419+0000","last_peered":"2024-03-06T07:26:18.746419+0000","last_clean":"2024-03-06T07:26:18.746419+0000","last_became_active":"2024-03-06T07:26:17.751883+0000","last_became_peered":"2024-03-06T07:26:17.751883+0000","last_unstale":"2024-03-06T07:26:18.746419+0000","last_undegraded":"2024-03-06T07:26:18.746419+0000","last_fullsized":"2024-03-06T07:26:18.746419+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:38.529399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744672+0000","last_change":"2024-03-06T07:26:07.710172+0000","last_active":"2024-03-06T07:26:17.744672+0000","last_peered":"2024-03-06T07:26:17.744672+0000","last_clean":"2024-03-06T07:26:17.744672+0000","last_became_active":"2024-03-06T07:26:07.709871+0000","last_became_peered":"2024-03-06T07:26:07.709871+0000","last_unstale":"2024-03-06T07:26:17.744672+0000","last_undegraded":"2024-03-06T07:26:17.744672+0000","last_fullsized":"2024-03-06T07:26:17.744672+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:17:00.303994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001140+0000","last_change":"2024-03-06T07:26:17.750774+0000","last_active":"2024-03-06T07:26:19.001140+0000","last_peered":"2024-03-06T07:26:19.001140+0000","last_clean":"2024-03-06T07:26:19.001140+0000","last_became_active":"2024-03-06T07:26:17.750510+0000","last_became_peered":"2024-03-06T07:26:17.750510+0000","last_unstale":"2024-03-06T07:26:19.001140+0000","last_undegraded":"2024-03-06T07:26:19.001140+0000","last_fullsized":"2024-03-06T07:26:19.001140+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:16:40.780478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746197+0000","last_change":"2024-03-06T07:26:07.688786+0000","last_active":"2024-03-06T07:26:18.746197+0000","last_peered":"2024-03-06T07:26:18.746197+0000","last_clean":"2024-03-06T07:26:18.746197+0000","last_became_active":"2024-03-06T07:26:07.688085+0000","last_became_peered":"2024-03-06T07:26:07.688085+0000","last_unstale":"2024-03-06T07:26:18.746197+0000","last_undegraded":"2024-03-06T07:26:18.746197+0000","last_fullsized":"2024-03-06T07:26:18.746197+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:55.281049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747369+0000","last_change":"2024-03-06T07:26:17.745834+0000","last_active":"2024-03-06T07:26:18.747369+0000","last_peered":"2024-03-06T07:26:18.747369+0000","last_clean":"2024-03-06T07:26:18.747369+0000","last_became_active":"2024-03-06T07:26:17.745566+0000","last_became_peered":"2024-03-06T07:26:17.745566+0000","last_unstale":"2024-03-06T07:26:18.747369+0000","last_undegraded":"2024-03-06T07:26:18.747369+0000","last_fullsized":"2024-03-06T07:26:18.747369+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:27.138420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749610+0000","last_change":"2024-03-06T07:26:07.711076+0000","last_active":"2024-03-06T07:26:19.749610+0000","last_peered":"2024-03-06T07:26:19.749610+0000","last_clean":"2024-03-06T07:26:19.749610+0000","last_became_active":"2024-03-06T07:26:07.710796+0000","last_became_peered":"2024-03-06T07:26:07.710796+0000","last_unstale":"2024-03-06T07:26:19.749610+0000","last_undegraded":"2024-03-06T07:26:19.749610+0000","last_fullsized":"2024-03-06T07:26:19.749610+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:56.000904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807100+0000","last_change":"2024-03-06T07:26:17.759373+0000","last_active":"2024-03-06T07:26:18.807100+0000","last_peered":"2024-03-06T07:26:18.807100+0000","last_clean":"2024-03-06T07:26:18.807100+0000","last_became_active":"2024-03-06T07:26:17.759122+0000","last_became_peered":"2024-03-06T07:26:17.759122+0000","last_unstale":"2024-03-06T07:26:18.807100+0000","last_undegraded":"2024-03-06T07:26:18.807100+0000","last_fullsized":"2024-03-06T07:26:18.807100+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:48:28.061942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882497+0000","last_change":"2024-03-06T07:26:07.703707+0000","last_active":"2024-03-06T07:26:18.882497+0000","last_peered":"2024-03-06T07:26:18.882497+0000","last_clean":"2024-03-06T07:26:18.882497+0000","last_became_active":"2024-03-06T07:26:07.703402+0000","last_became_peered":"2024-03-06T07:26:07.703402+0000","last_unstale":"2024-03-06T07:26:18.882497+0000","last_undegraded":"2024-03-06T07:26:18.882497+0000","last_fullsized":"2024-03-06T07:26:18.882497+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:45.409316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807429+0000","last_change":"2024-03-06T07:26:17.757430+0000","last_active":"2024-03-06T07:26:18.807429+0000","last_peered":"2024-03-06T07:26:18.807429+0000","last_clean":"2024-03-06T07:26:18.807429+0000","last_became_active":"2024-03-06T07:26:17.757156+0000","last_became_peered":"2024-03-06T07:26:17.757156+0000","last_unstale":"2024-03-06T07:26:18.807429+0000","last_undegraded":"2024-03-06T07:26:18.807429+0000","last_fullsized":"2024-03-06T07:26:18.807429+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:31.806156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.746012+0000","last_change":"2024-03-06T07:26:07.710174+0000","last_active":"2024-03-06T07:26:17.746012+0000","last_peered":"2024-03-06T07:26:17.746012+0000","last_clean":"2024-03-06T07:26:17.746012+0000","last_became_active":"2024-03-06T07:26:07.709939+0000","last_became_peered":"2024-03-06T07:26:07.709939+0000","last_unstale":"2024-03-06T07:26:17.746012+0000","last_undegraded":"2024-03-06T07:26:17.746012+0000","last_fullsized":"2024-03-06T07:26:17.746012+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747283+0000","last_change":"2024-03-06T07:26:17.740913+0000","last_active":"2024-03-06T07:26:18.747283+0000","last_peered":"2024-03-06T07:26:18.747283+0000","last_clean":"2024-03-06T07:26:18.747283+0000","last_became_active":"2024-03-06T07:26:17.740648+0000","last_became_peered":"2024-03-06T07:26:17.740648+0000","last_unstale":"2024-03-06T07:26:18.747283+0000","last_undegraded":"2024-03-06T07:26:18.747283+0000","last_fullsized":"2024-03-06T07:26:18.747283+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:39.385283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001457+0000","last_change":"2024-03-06T07:26:07.704730+0000","last_active":"2024-03-06T07:26:19.001457+0000","last_peered":"2024-03-06T07:26:19.001457+0000","last_clean":"2024-03-06T07:26:19.001457+0000","last_became_active":"2024-03-06T07:26:07.704429+0000","last_became_peered":"2024-03-06T07:26:07.704429+0000","last_unstale":"2024-03-06T07:26:19.001457+0000","last_undegraded":"2024-03-06T07:26:19.001457+0000","last_fullsized":"2024-03-06T07:26:19.001457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:14.776787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745095+0000","last_change":"2024-03-06T07:26:17.746811+0000","last_active":"2024-03-06T07:26:18.745095+0000","last_peered":"2024-03-06T07:26:18.745095+0000","last_clean":"2024-03-06T07:26:18.745095+0000","last_became_active":"2024-03-06T07:26:17.746510+0000","last_became_peered":"2024-03-06T07:26:17.746510+0000","last_unstale":"2024-03-06T07:26:18.745095+0000","last_undegraded":"2024-03-06T07:26:18.745095+0000","last_fullsized":"2024-03-06T07:26:18.745095+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:05.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806960+0000","last_change":"2024-03-06T07:26:07.685590+0000","last_active":"2024-03-06T07:26:18.806960+0000","last_peered":"2024-03-06T07:26:18.806960+0000","last_clean":"2024-03-06T07:26:18.806960+0000","last_became_active":"2024-03-06T07:26:07.685265+0000","last_became_peered":"2024-03-06T07:26:07.685265+0000","last_unstale":"2024-03-06T07:26:18.806960+0000","last_undegraded":"2024-03-06T07:26:18.806960+0000","last_fullsized":"2024-03-06T07:26:18.806960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:30.427263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748892+0000","last_change":"2024-03-06T07:26:17.740092+0000","last_active":"2024-03-06T07:26:19.748892+0000","last_peered":"2024-03-06T07:26:19.748892+0000","last_clean":"2024-03-06T07:26:19.748892+0000","last_became_active":"2024-03-06T07:26:17.739842+0000","last_became_peered":"2024-03-06T07:26:17.739842+0000","last_unstale":"2024-03-06T07:26:19.748892+0000","last_undegraded":"2024-03-06T07:26:19.748892+0000","last_fullsized":"2024-03-06T07:26:19.748892+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:19.539826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744661+0000","last_change":"2024-03-06T07:26:07.699358+0000","last_active":"2024-03-06T07:26:18.744661+0000","last_peered":"2024-03-06T07:26:18.744661+0000","last_clean":"2024-03-06T07:26:18.744661+0000","last_became_active":"2024-03-06T07:26:07.699058+0000","last_became_peered":"2024-03-06T07:26:07.699058+0000","last_unstale":"2024-03-06T07:26:18.744661+0000","last_undegraded":"2024-03-06T07:26:18.744661+0000","last_fullsized":"2024-03-06T07:26:18.744661+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:02.835571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749232+0000","last_change":"2024-03-06T07:26:17.752029+0000","last_active":"2024-03-06T07:26:19.749232+0000","last_peered":"2024-03-06T07:26:19.749232+0000","last_clean":"2024-03-06T07:26:19.749232+0000","last_became_active":"2024-03-06T07:26:17.751766+0000","last_became_peered":"2024-03-06T07:26:17.751766+0000","last_unstale":"2024-03-06T07:26:19.749232+0000","last_undegraded":"2024-03-06T07:26:19.749232+0000","last_fullsized":"2024-03-06T07:26:19.749232+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:34.812576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.739016+0000","last_change":"2024-03-06T07:26:07.700848+0000","last_active":"2024-03-06T07:26:17.739016+0000","last_peered":"2024-03-06T07:26:17.739016+0000","last_clean":"2024-03-06T07:26:17.739016+0000","last_became_active":"2024-03-06T07:26:07.700551+0000","last_became_peered":"2024-03-06T07:26:07.700551+0000","last_unstale":"2024-03-06T07:26:17.739016+0000","last_undegraded":"2024-03-06T07:26:17.739016+0000","last_fullsized":"2024-03-06T07:26:17.739016+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.178917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748998+0000","last_change":"2024-03-06T07:26:17.754908+0000","last_active":"2024-03-06T07:26:19.748998+0000","last_peered":"2024-03-06T07:26:19.748998+0000","last_clean":"2024-03-06T07:26:19.748998+0000","last_became_active":"2024-03-06T07:26:17.754430+0000","last_became_peered":"2024-03-06T07:26:17.754430+0000","last_unstale":"2024-03-06T07:26:19.748998+0000","last_undegraded":"2024-03-06T07:26:19.748998+0000","last_fullsized":"2024-03-06T07:26:19.748998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:50:45.483679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746061+0000","last_change":"2024-03-06T07:26:07.687552+0000","last_active":"2024-03-06T07:26:18.746061+0000","last_peered":"2024-03-06T07:26:18.746061+0000","last_clean":"2024-03-06T07:26:18.746061+0000","last_became_active":"2024-03-06T07:26:07.687276+0000","last_became_peered":"2024-03-06T07:26:07.687276+0000","last_unstale":"2024-03-06T07:26:18.746061+0000","last_undegraded":"2024-03-06T07:26:18.746061+0000","last_fullsized":"2024-03-06T07:26:18.746061+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:03.163515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745034+0000","last_change":"2024-03-06T07:26:17.740835+0000","last_active":"2024-03-06T07:26:18.745034+0000","last_peered":"2024-03-06T07:26:18.745034+0000","last_clean":"2024-03-06T07:26:18.745034+0000","last_became_active":"2024-03-06T07:26:17.740544+0000","last_became_peered":"2024-03-06T07:26:17.740544+0000","last_unstale":"2024-03-06T07:26:18.745034+0000","last_undegraded":"2024-03-06T07:26:18.745034+0000","last_fullsized":"2024-03-06T07:26:18.745034+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:59:42.208048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807551+0000","last_change":"2024-03-06T07:26:07.698089+0000","last_active":"2024-03-06T07:26:18.807551+0000","last_peered":"2024-03-06T07:26:18.807551+0000","last_clean":"2024-03-06T07:26:18.807551+0000","last_became_active":"2024-03-06T07:26:07.697806+0000","last_became_peered":"2024-03-06T07:26:07.697806+0000","last_unstale":"2024-03-06T07:26:18.807551+0000","last_undegraded":"2024-03-06T07:26:18.807551+0000","last_fullsized":"2024-03-06T07:26:18.807551+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:52.239620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806635+0000","last_change":"2024-03-06T07:26:17.756836+0000","last_active":"2024-03-06T07:26:18.806635+0000","last_peered":"2024-03-06T07:26:18.806635+0000","last_clean":"2024-03-06T07:26:18.806635+0000","last_became_active":"2024-03-06T07:26:17.756565+0000","last_became_peered":"2024-03-06T07:26:17.756565+0000","last_unstale":"2024-03-06T07:26:18.806635+0000","last_undegraded":"2024-03-06T07:26:18.806635+0000","last_fullsized":"2024-03-06T07:26:18.806635+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:44.181354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001511+0000","last_change":"2024-03-06T07:26:07.699922+0000","last_active":"2024-03-06T07:26:19.001511+0000","last_peered":"2024-03-06T07:26:19.001511+0000","last_clean":"2024-03-06T07:26:19.001511+0000","last_became_active":"2024-03-06T07:26:07.699631+0000","last_became_peered":"2024-03-06T07:26:07.699631+0000","last_unstale":"2024-03-06T07:26:19.001511+0000","last_undegraded":"2024-03-06T07:26:19.001511+0000","last_fullsized":"2024-03-06T07:26:19.001511+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:39.110553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001570+0000","last_change":"2024-03-06T07:26:17.748859+0000","last_active":"2024-03-06T07:26:19.001570+0000","last_peered":"2024-03-06T07:26:19.001570+0000","last_clean":"2024-03-06T07:26:19.001570+0000","last_became_active":"2024-03-06T07:26:17.748556+0000","last_became_peered":"2024-03-06T07:26:17.748556+0000","last_unstale":"2024-03-06T07:26:19.001570+0000","last_undegraded":"2024-03-06T07:26:19.001570+0000","last_fullsized":"2024-03-06T07:26:19.001570+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:29:50.372142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745228+0000","last_change":"2024-03-06T07:26:07.706359+0000","last_active":"2024-03-06T07:26:18.745228+0000","last_peered":"2024-03-06T07:26:18.745228+0000","last_clean":"2024-03-06T07:26:18.745228+0000","last_became_active":"2024-03-06T07:26:07.706096+0000","last_became_peered":"2024-03-06T07:26:07.706096+0000","last_unstale":"2024-03-06T07:26:18.745228+0000","last_undegraded":"2024-03-06T07:26:18.745228+0000","last_fullsized":"2024-03-06T07:26:18.745228+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:08.310190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.752823+0000","last_change":"2024-03-06T07:26:17.750402+0000","last_active":"2024-03-06T07:26:17.752823+0000","last_peered":"2024-03-06T07:26:17.752823+0000","last_clean":"2024-03-06T07:26:17.752823+0000","last_became_active":"2024-03-06T07:26:17.749658+0000","last_became_peered":"2024-03-06T07:26:17.749658+0000","last_unstale":"2024-03-06T07:26:17.752823+0000","last_undegraded":"2024-03-06T07:26:17.752823+0000","last_fullsized":"2024-03-06T07:26:17.752823+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:16.312646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806734+0000","last_change":"2024-03-06T07:26:07.693096+0000","last_active":"2024-03-06T07:26:18.806734+0000","last_peered":"2024-03-06T07:26:18.806734+0000","last_clean":"2024-03-06T07:26:18.806734+0000","last_became_active":"2024-03-06T07:26:07.692834+0000","last_became_peered":"2024-03-06T07:26:07.692834+0000","last_unstale":"2024-03-06T07:26:18.806734+0000","last_undegraded":"2024-03-06T07:26:18.806734+0000","last_fullsized":"2024-03-06T07:26:18.806734+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:58.002384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001192+0000","last_change":"2024-03-06T07:26:17.746481+0000","last_active":"2024-03-06T07:26:19.001192+0000","last_peered":"2024-03-06T07:26:19.001192+0000","last_clean":"2024-03-06T07:26:19.001192+0000","last_became_active":"2024-03-06T07:26:17.746226+0000","last_became_peered":"2024-03-06T07:26:17.746226+0000","last_unstale":"2024-03-06T07:26:19.001192+0000","last_undegraded":"2024-03-06T07:26:19.001192+0000","last_fullsized":"2024-03-06T07:26:19.001192+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:06.076055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744906+0000","last_change":"2024-03-06T07:26:07.706042+0000","last_active":"2024-03-06T07:26:18.744906+0000","last_peered":"2024-03-06T07:26:18.744906+0000","last_clean":"2024-03-06T07:26:18.744906+0000","last_became_active":"2024-03-06T07:26:07.705455+0000","last_became_peered":"2024-03-06T07:26:07.705455+0000","last_unstale":"2024-03-06T07:26:18.744906+0000","last_undegraded":"2024-03-06T07:26:18.744906+0000","last_fullsized":"2024-03-06T07:26:18.744906+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:48.048889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806419+0000","last_change":"2024-03-06T07:26:17.755210+0000","last_active":"2024-03-06T07:26:18.806419+0000","last_peered":"2024-03-06T07:26:18.806419+0000","last_clean":"2024-03-06T07:26:18.806419+0000","last_became_active":"2024-03-06T07:26:17.754985+0000","last_became_peered":"2024-03-06T07:26:17.754985+0000","last_unstale":"2024-03-06T07:26:18.806419+0000","last_undegraded":"2024-03-06T07:26:18.806419+0000","last_fullsized":"2024-03-06T07:26:18.806419+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:39.760526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882242+0000","last_change":"2024-03-06T07:26:07.706790+0000","last_active":"2024-03-06T07:26:18.882242+0000","last_peered":"2024-03-06T07:26:18.882242+0000","last_clean":"2024-03-06T07:26:18.882242+0000","last_became_active":"2024-03-06T07:26:07.706476+0000","last_became_peered":"2024-03-06T07:26:07.706476+0000","last_unstale":"2024-03-06T07:26:18.882242+0000","last_undegraded":"2024-03-06T07:26:18.882242+0000","last_fullsized":"2024-03-06T07:26:18.882242+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:57.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.762180+0000","last_change":"2024-03-06T07:26:17.755465+0000","last_active":"2024-03-06T07:26:17.762180+0000","last_peered":"2024-03-06T07:26:17.762180+0000","last_clean":"2024-03-06T07:26:17.762180+0000","last_became_active":"2024-03-06T07:26:17.755199+0000","last_became_peered":"2024-03-06T07:26:17.755199+0000","last_unstale":"2024-03-06T07:26:17.762180+0000","last_undegraded":"2024-03-06T07:26:17.762180+0000","last_fullsized":"2024-03-06T07:26:17.762180+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:45.169002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744755+0000","last_change":"2024-03-06T07:26:07.698630+0000","last_active":"2024-03-06T07:26:17.744755+0000","last_peered":"2024-03-06T07:26:17.744755+0000","last_clean":"2024-03-06T07:26:17.744755+0000","last_became_active":"2024-03-06T07:26:07.698334+0000","last_became_peered":"2024-03-06T07:26:07.698334+0000","last_unstale":"2024-03-06T07:26:17.744755+0000","last_undegraded":"2024-03-06T07:26:17.744755+0000","last_fullsized":"2024-03-06T07:26:17.744755+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:50:39.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760234+0000","last_change":"2024-03-06T07:26:17.752948+0000","last_active":"2024-03-06T07:26:17.760234+0000","last_peered":"2024-03-06T07:26:17.760234+0000","last_clean":"2024-03-06T07:26:17.760234+0000","last_became_active":"2024-03-06T07:26:17.752660+0000","last_became_peered":"2024-03-06T07:26:17.752660+0000","last_unstale":"2024-03-06T07:26:17.760234+0000","last_undegraded":"2024-03-06T07:26:17.760234+0000","last_fullsized":"2024-03-06T07:26:17.760234+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:38.035013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882321+0000","last_change":"2024-03-06T07:26:07.699219+0000","last_active":"2024-03-06T07:26:18.882321+0000","last_peered":"2024-03-06T07:26:18.882321+0000","last_clean":"2024-03-06T07:26:18.882321+0000","last_became_active":"2024-03-06T07:26:07.698897+0000","last_became_peered":"2024-03-06T07:26:07.698897+0000","last_unstale":"2024-03-06T07:26:18.882321+0000","last_undegraded":"2024-03-06T07:26:18.882321+0000","last_fullsized":"2024-03-06T07:26:18.882321+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:56.031022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746317+0000","last_change":"2024-03-06T07:26:17.749192+0000","last_active":"2024-03-06T07:26:18.746317+0000","last_peered":"2024-03-06T07:26:18.746317+0000","last_clean":"2024-03-06T07:26:18.746317+0000","last_became_active":"2024-03-06T07:26:17.748932+0000","last_became_peered":"2024-03-06T07:26:17.748932+0000","last_unstale":"2024-03-06T07:26:18.746317+0000","last_undegraded":"2024-03-06T07:26:18.746317+0000","last_fullsized":"2024-03-06T07:26:18.746317+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:35.899348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738947+0000","last_change":"2024-03-06T07:26:16.740205+0000","last_active":"2024-03-06T07:26:17.738947+0000","last_peered":"2024-03-06T07:26:17.738947+0000","last_clean":"2024-03-06T07:26:17.738947+0000","last_became_active":"2024-03-06T07:26:16.739946+0000","last_became_peered":"2024-03-06T07:26:16.739946+0000","last_unstale":"2024-03-06T07:26:17.738947+0000","last_undegraded":"2024-03-06T07:26:17.738947+0000","last_fullsized":"2024-03-06T07:26:17.738947+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:59:41.202625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744621+0000","last_change":"2024-03-06T07:26:07.697611+0000","last_active":"2024-03-06T07:26:18.744621+0000","last_peered":"2024-03-06T07:26:18.744621+0000","last_clean":"2024-03-06T07:26:18.744621+0000","last_became_active":"2024-03-06T07:26:07.697343+0000","last_became_peered":"2024-03-06T07:26:07.697343+0000","last_unstale":"2024-03-06T07:26:18.744621+0000","last_undegraded":"2024-03-06T07:26:18.744621+0000","last_fullsized":"2024-03-06T07:26:18.744621+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:41.934898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"150'4","reported_seq":26,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747506+0000","last_change":"2024-03-06T07:26:06.698500+0000","last_active":"2024-03-06T07:26:18.747506+0000","last_peered":"2024-03-06T07:26:18.747506+0000","last_clean":"2024-03-06T07:26:18.747506+0000","last_became_active":"2024-03-06T07:26:06.698299+0000","last_became_peered":"2024-03-06T07:26:06.698299+0000","last_unstale":"2024-03-06T07:26:18.747506+0000","last_undegraded":"2024-03-06T07:26:18.747506+0000","last_fullsized":"2024-03-06T07:26:18.747506+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:17.195623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746855+0000","last_change":"2024-03-06T07:26:17.747833+0000","last_active":"2024-03-06T07:26:18.746855+0000","last_peered":"2024-03-06T07:26:18.746855+0000","last_clean":"2024-03-06T07:26:18.746855+0000","last_became_active":"2024-03-06T07:26:17.747575+0000","last_became_peered":"2024-03-06T07:26:17.747575+0000","last_unstale":"2024-03-06T07:26:18.746855+0000","last_undegraded":"2024-03-06T07:26:18.746855+0000","last_fullsized":"2024-03-06T07:26:18.746855+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:09.554449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"154'8","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.765009+0000","last_change":"2024-03-06T07:26:16.741372+0000","last_active":"2024-03-06T07:26:17.765009+0000","last_peered":"2024-03-06T07:26:17.765009+0000","last_clean":"2024-03-06T07:26:17.765009+0000","last_became_active":"2024-03-06T07:26:16.741131+0000","last_became_peered":"2024-03-06T07:26:16.741131+0000","last_unstale":"2024-03-06T07:26:17.765009+0000","last_undegraded":"2024-03-06T07:26:17.765009+0000","last_fullsized":"2024-03-06T07:26:17.765009+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:11.875182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806422+0000","last_change":"2024-03-06T07:26:07.698545+0000","last_active":"2024-03-06T07:26:18.806422+0000","last_peered":"2024-03-06T07:26:18.806422+0000","last_clean":"2024-03-06T07:26:18.806422+0000","last_became_active":"2024-03-06T07:26:07.698270+0000","last_became_peered":"2024-03-06T07:26:07.698270+0000","last_unstale":"2024-03-06T07:26:18.806422+0000","last_undegraded":"2024-03-06T07:26:18.806422+0000","last_fullsized":"2024-03-06T07:26:18.806422+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:59:25.247264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"150'1","reported_seq":21,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738979+0000","last_change":"2024-03-06T07:26:06.702569+0000","last_active":"2024-03-06T07:26:17.738979+0000","last_peered":"2024-03-06T07:26:17.738979+0000","last_clean":"2024-03-06T07:26:17.738979+0000","last_became_active":"2024-03-06T07:26:06.702304+0000","last_became_peered":"2024-03-06T07:26:06.702304+0000","last_unstale":"2024-03-06T07:26:17.738979+0000","last_undegraded":"2024-03-06T07:26:17.738979+0000","last_fullsized":"2024-03-06T07:26:17.738979+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:16.087949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001008+0000","last_change":"2024-03-06T07:26:17.750415+0000","last_active":"2024-03-06T07:26:19.001008+0000","last_peered":"2024-03-06T07:26:19.001008+0000","last_clean":"2024-03-06T07:26:19.001008+0000","last_became_active":"2024-03-06T07:26:17.748976+0000","last_became_peered":"2024-03-06T07:26:17.748976+0000","last_unstale":"2024-03-06T07:26:19.001008+0000","last_undegraded":"2024-03-06T07:26:19.001008+0000","last_fullsized":"2024-03-06T07:26:19.001008+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:11.939182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746593+0000","last_change":"2024-03-06T07:26:16.740808+0000","last_active":"2024-03-06T07:26:18.746593+0000","last_peered":"2024-03-06T07:26:18.746593+0000","last_clean":"2024-03-06T07:26:18.746593+0000","last_became_active":"2024-03-06T07:26:16.740532+0000","last_became_peered":"2024-03-06T07:26:16.740532+0000","last_unstale":"2024-03-06T07:26:18.746593+0000","last_undegraded":"2024-03-06T07:26:18.746593+0000","last_fullsized":"2024-03-06T07:26:18.746593+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:59.758111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743865+0000","last_change":"2024-03-06T07:26:07.701966+0000","last_active":"2024-03-06T07:26:17.743865+0000","last_peered":"2024-03-06T07:26:17.743865+0000","last_clean":"2024-03-06T07:26:17.743865+0000","last_became_active":"2024-03-06T07:26:07.701671+0000","last_became_peered":"2024-03-06T07:26:07.701671+0000","last_unstale":"2024-03-06T07:26:17.743865+0000","last_undegraded":"2024-03-06T07:26:17.743865+0000","last_fullsized":"2024-03-06T07:26:17.743865+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:03.437751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.746289+0000","last_change":"2024-03-06T07:26:06.691461+0000","last_active":"2024-03-06T07:26:17.746289+0000","last_peered":"2024-03-06T07:26:17.746289+0000","last_clean":"2024-03-06T07:26:17.746289+0000","last_became_active":"2024-03-06T07:26:06.691169+0000","last_became_peered":"2024-03-06T07:26:06.691169+0000","last_unstale":"2024-03-06T07:26:17.746289+0000","last_undegraded":"2024-03-06T07:26:17.746289+0000","last_fullsized":"2024-03-06T07:26:17.746289+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806365+0000","last_change":"2024-03-06T07:26:17.756094+0000","last_active":"2024-03-06T07:26:18.806365+0000","last_peered":"2024-03-06T07:26:18.806365+0000","last_clean":"2024-03-06T07:26:18.806365+0000","last_became_active":"2024-03-06T07:26:17.755837+0000","last_became_peered":"2024-03-06T07:26:17.755837+0000","last_unstale":"2024-03-06T07:26:18.806365+0000","last_undegraded":"2024-03-06T07:26:18.806365+0000","last_fullsized":"2024-03-06T07:26:18.806365+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:02.797165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749384+0000","last_change":"2024-03-06T07:26:16.740881+0000","last_active":"2024-03-06T07:26:19.749384+0000","last_peered":"2024-03-06T07:26:19.749384+0000","last_clean":"2024-03-06T07:26:19.749384+0000","last_became_active":"2024-03-06T07:26:16.740610+0000","last_became_peered":"2024-03-06T07:26:16.740610+0000","last_unstale":"2024-03-06T07:26:19.749384+0000","last_undegraded":"2024-03-06T07:26:19.749384+0000","last_fullsized":"2024-03-06T07:26:19.749384+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:13.932609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745015+0000","last_change":"2024-03-06T07:26:07.705527+0000","last_active":"2024-03-06T07:26:18.745015+0000","last_peered":"2024-03-06T07:26:18.745015+0000","last_clean":"2024-03-06T07:26:18.745015+0000","last_became_active":"2024-03-06T07:26:07.705214+0000","last_became_peered":"2024-03-06T07:26:07.705214+0000","last_unstale":"2024-03-06T07:26:18.745015+0000","last_undegraded":"2024-03-06T07:26:18.745015+0000","last_fullsized":"2024-03-06T07:26:18.745015+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:09.357832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"150'2","reported_seq":24,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001562+0000","last_change":"2024-03-06T07:26:06.689558+0000","last_active":"2024-03-06T07:26:19.001562+0000","last_peered":"2024-03-06T07:26:19.001562+0000","last_clean":"2024-03-06T07:26:19.001562+0000","last_became_active":"2024-03-06T07:26:06.689291+0000","last_became_peered":"2024-03-06T07:26:06.689291+0000","last_unstale":"2024-03-06T07:26:19.001562+0000","last_undegraded":"2024-03-06T07:26:19.001562+0000","last_fullsized":"2024-03-06T07:26:19.001562+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:59.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747393+0000","last_change":"2024-03-06T07:26:17.747419+0000","last_active":"2024-03-06T07:26:18.747393+0000","last_peered":"2024-03-06T07:26:18.747393+0000","last_clean":"2024-03-06T07:26:18.747393+0000","last_became_active":"2024-03-06T07:26:17.747131+0000","last_became_peered":"2024-03-06T07:26:17.747131+0000","last_unstale":"2024-03-06T07:26:18.747393+0000","last_undegraded":"2024-03-06T07:26:18.747393+0000","last_fullsized":"2024-03-06T07:26:18.747393+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:06.045937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"154'3","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.761355+0000","last_change":"2024-03-06T07:26:16.742718+0000","last_active":"2024-03-06T07:26:17.761355+0000","last_peered":"2024-03-06T07:26:17.761355+0000","last_clean":"2024-03-06T07:26:17.761355+0000","last_became_active":"2024-03-06T07:26:16.742380+0000","last_became_peered":"2024-03-06T07:26:16.742380+0000","last_unstale":"2024-03-06T07:26:17.761355+0000","last_undegraded":"2024-03-06T07:26:17.761355+0000","last_fullsized":"2024-03-06T07:26:17.761355+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:11.700610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749126+0000","last_change":"2024-03-06T07:26:07.697671+0000","last_active":"2024-03-06T07:26:19.749126+0000","last_peered":"2024-03-06T07:26:19.749126+0000","last_clean":"2024-03-06T07:26:19.749126+0000","last_became_active":"2024-03-06T07:26:07.697366+0000","last_became_peered":"2024-03-06T07:26:07.697366+0000","last_unstale":"2024-03-06T07:26:19.749126+0000","last_undegraded":"2024-03-06T07:26:19.749126+0000","last_fullsized":"2024-03-06T07:26:19.749126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"150'1","reported_seq":23,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749286+0000","last_change":"2024-03-06T07:26:06.700139+0000","last_active":"2024-03-06T07:26:19.749286+0000","last_peered":"2024-03-06T07:26:19.749286+0000","last_clean":"2024-03-06T07:26:19.749286+0000","last_became_active":"2024-03-06T07:26:06.699876+0000","last_became_peered":"2024-03-06T07:26:06.699876+0000","last_unstale":"2024-03-06T07:26:19.749286+0000","last_undegraded":"2024-03-06T07:26:19.749286+0000","last_fullsized":"2024-03-06T07:26:19.749286+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:22.510699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749740+0000","last_change":"2024-03-06T07:26:17.746435+0000","last_active":"2024-03-06T07:26:19.749740+0000","last_peered":"2024-03-06T07:26:19.749740+0000","last_clean":"2024-03-06T07:26:19.749740+0000","last_became_active":"2024-03-06T07:26:17.746036+0000","last_became_peered":"2024-03-06T07:26:17.746036+0000","last_unstale":"2024-03-06T07:26:19.749740+0000","last_undegraded":"2024-03-06T07:26:19.749740+0000","last_fullsized":"2024-03-06T07:26:19.749740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:04.831471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"154'6","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746503+0000","last_change":"2024-03-06T07:26:16.737058+0000","last_active":"2024-03-06T07:26:18.746503+0000","last_peered":"2024-03-06T07:26:18.746503+0000","last_clean":"2024-03-06T07:26:18.746503+0000","last_became_active":"2024-03-06T07:26:16.736759+0000","last_became_peered":"2024-03-06T07:26:16.736759+0000","last_unstale":"2024-03-06T07:26:18.746503+0000","last_undegraded":"2024-03-06T07:26:18.746503+0000","last_fullsized":"2024-03-06T07:26:18.746503+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:19.128809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001438+0000","last_change":"2024-03-06T07:26:07.705724+0000","last_active":"2024-03-06T07:26:19.001438+0000","last_peered":"2024-03-06T07:26:19.001438+0000","last_clean":"2024-03-06T07:26:19.001438+0000","last_became_active":"2024-03-06T07:26:07.705378+0000","last_became_peered":"2024-03-06T07:26:07.705378+0000","last_unstale":"2024-03-06T07:26:19.001438+0000","last_undegraded":"2024-03-06T07:26:19.001438+0000","last_fullsized":"2024-03-06T07:26:19.001438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:38:29.508415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"150'1","reported_seq":21,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743946+0000","last_change":"2024-03-06T07:26:06.702328+0000","last_active":"2024-03-06T07:26:17.743946+0000","last_peered":"2024-03-06T07:26:17.743946+0000","last_clean":"2024-03-06T07:26:17.743946+0000","last_became_active":"2024-03-06T07:26:06.702009+0000","last_became_peered":"2024-03-06T07:26:06.702009+0000","last_unstale":"2024-03-06T07:26:17.743946+0000","last_undegraded":"2024-03-06T07:26:17.743946+0000","last_fullsized":"2024-03-06T07:26:17.743946+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:14.494596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745356+0000","last_change":"2024-03-06T07:26:17.739382+0000","last_active":"2024-03-06T07:26:18.745356+0000","last_peered":"2024-03-06T07:26:18.745356+0000","last_clean":"2024-03-06T07:26:18.745356+0000","last_became_active":"2024-03-06T07:26:17.739065+0000","last_became_peered":"2024-03-06T07:26:17.739065+0000","last_unstale":"2024-03-06T07:26:18.745356+0000","last_undegraded":"2024-03-06T07:26:18.745356+0000","last_fullsized":"2024-03-06T07:26:18.745356+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:10:47.968925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807433+0000","last_change":"2024-03-06T07:26:16.741605+0000","last_active":"2024-03-06T07:26:18.807433+0000","last_peered":"2024-03-06T07:26:18.807433+0000","last_clean":"2024-03-06T07:26:18.807433+0000","last_became_active":"2024-03-06T07:26:16.741355+0000","last_became_peered":"2024-03-06T07:26:16.741355+0000","last_unstale":"2024-03-06T07:26:18.807433+0000","last_undegraded":"2024-03-06T07:26:18.807433+0000","last_fullsized":"2024-03-06T07:26:18.807433+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:25.144021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001448+0000","last_change":"2024-03-06T07:26:07.698151+0000","last_active":"2024-03-06T07:26:19.001448+0000","last_peered":"2024-03-06T07:26:19.001448+0000","last_clean":"2024-03-06T07:26:19.001448+0000","last_became_active":"2024-03-06T07:26:07.697876+0000","last_became_peered":"2024-03-06T07:26:07.697876+0000","last_unstale":"2024-03-06T07:26:19.001448+0000","last_undegraded":"2024-03-06T07:26:19.001448+0000","last_fullsized":"2024-03-06T07:26:19.001448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:56.601965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001309+0000","last_change":"2024-03-06T07:26:06.700042+0000","last_active":"2024-03-06T07:26:19.001309+0000","last_peered":"2024-03-06T07:26:19.001309+0000","last_clean":"2024-03-06T07:26:19.001309+0000","last_became_active":"2024-03-06T07:26:06.699781+0000","last_became_peered":"2024-03-06T07:26:06.699781+0000","last_unstale":"2024-03-06T07:26:19.001309+0000","last_undegraded":"2024-03-06T07:26:19.001309+0000","last_fullsized":"2024-03-06T07:26:19.001309+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:13.960358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806959+0000","last_change":"2024-03-06T07:26:17.748519+0000","last_active":"2024-03-06T07:26:18.806959+0000","last_peered":"2024-03-06T07:26:18.806959+0000","last_clean":"2024-03-06T07:26:18.806959+0000","last_became_active":"2024-03-06T07:26:17.748239+0000","last_became_peered":"2024-03-06T07:26:17.748239+0000","last_unstale":"2024-03-06T07:26:18.806959+0000","last_undegraded":"2024-03-06T07:26:18.806959+0000","last_fullsized":"2024-03-06T07:26:18.806959+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:02.981819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749425+0000","last_change":"2024-03-06T07:26:16.742781+0000","last_active":"2024-03-06T07:26:19.749425+0000","last_peered":"2024-03-06T07:26:19.749425+0000","last_clean":"2024-03-06T07:26:19.749425+0000","last_became_active":"2024-03-06T07:26:16.742519+0000","last_became_peered":"2024-03-06T07:26:16.742519+0000","last_unstale":"2024-03-06T07:26:19.749425+0000","last_undegraded":"2024-03-06T07:26:19.749425+0000","last_fullsized":"2024-03-06T07:26:19.749425+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:15.307222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747373+0000","last_change":"2024-03-06T07:26:07.696784+0000","last_active":"2024-03-06T07:26:18.747373+0000","last_peered":"2024-03-06T07:26:18.747373+0000","last_clean":"2024-03-06T07:26:18.747373+0000","last_became_active":"2024-03-06T07:26:07.696515+0000","last_became_peered":"2024-03-06T07:26:07.696515+0000","last_unstale":"2024-03-06T07:26:18.747373+0000","last_undegraded":"2024-03-06T07:26:18.747373+0000","last_fullsized":"2024-03-06T07:26:18.747373+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:07.777819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748910+0000","last_change":"2024-03-06T07:26:06.693663+0000","last_active":"2024-03-06T07:26:19.748910+0000","last_peered":"2024-03-06T07:26:19.748910+0000","last_clean":"2024-03-06T07:26:19.748910+0000","last_became_active":"2024-03-06T07:26:06.693251+0000","last_became_peered":"2024-03-06T07:26:06.693251+0000","last_unstale":"2024-03-06T07:26:19.748910+0000","last_undegraded":"2024-03-06T07:26:19.748910+0000","last_fullsized":"2024-03-06T07:26:19.748910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749564+0000","last_change":"2024-03-06T07:26:17.751564+0000","last_active":"2024-03-06T07:26:19.749564+0000","last_peered":"2024-03-06T07:26:19.749564+0000","last_clean":"2024-03-06T07:26:19.749564+0000","last_became_active":"2024-03-06T07:26:17.751256+0000","last_became_peered":"2024-03-06T07:26:17.751256+0000","last_unstale":"2024-03-06T07:26:19.749564+0000","last_undegraded":"2024-03-06T07:26:19.749564+0000","last_fullsized":"2024-03-06T07:26:19.749564+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:29.345786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746227+0000","last_change":"2024-03-06T07:26:16.730081+0000","last_active":"2024-03-06T07:26:18.746227+0000","last_peered":"2024-03-06T07:26:18.746227+0000","last_clean":"2024-03-06T07:26:18.746227+0000","last_became_active":"2024-03-06T07:26:16.729647+0000","last_became_peered":"2024-03-06T07:26:16.729647+0000","last_unstale":"2024-03-06T07:26:18.746227+0000","last_undegraded":"2024-03-06T07:26:18.746227+0000","last_fullsized":"2024-03-06T07:26:18.746227+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:29.900555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744501+0000","last_change":"2024-03-06T07:26:07.698185+0000","last_active":"2024-03-06T07:26:18.744501+0000","last_peered":"2024-03-06T07:26:18.744501+0000","last_clean":"2024-03-06T07:26:18.744501+0000","last_became_active":"2024-03-06T07:26:07.696887+0000","last_became_peered":"2024-03-06T07:26:07.696887+0000","last_unstale":"2024-03-06T07:26:18.744501+0000","last_undegraded":"2024-03-06T07:26:18.744501+0000","last_fullsized":"2024-03-06T07:26:18.744501+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:30.446061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807520+0000","last_change":"2024-03-06T07:26:06.689754+0000","last_active":"2024-03-06T07:26:18.807520+0000","last_peered":"2024-03-06T07:26:18.807520+0000","last_clean":"2024-03-06T07:26:18.807520+0000","last_became_active":"2024-03-06T07:26:06.689386+0000","last_became_peered":"2024-03-06T07:26:06.689386+0000","last_unstale":"2024-03-06T07:26:18.807520+0000","last_undegraded":"2024-03-06T07:26:18.807520+0000","last_fullsized":"2024-03-06T07:26:18.807520+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745966+0000","last_change":"2024-03-06T07:26:17.749723+0000","last_active":"2024-03-06T07:26:18.745966+0000","last_peered":"2024-03-06T07:26:18.745966+0000","last_clean":"2024-03-06T07:26:18.745966+0000","last_became_active":"2024-03-06T07:26:17.749469+0000","last_became_peered":"2024-03-06T07:26:17.749469+0000","last_unstale":"2024-03-06T07:26:18.745966+0000","last_undegraded":"2024-03-06T07:26:18.745966+0000","last_fullsized":"2024-03-06T07:26:18.745966+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:47.445099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806588+0000","last_change":"2024-03-06T07:26:16.738778+0000","last_active":"2024-03-06T07:26:18.806588+0000","last_peered":"2024-03-06T07:26:18.806588+0000","last_clean":"2024-03-06T07:26:18.806588+0000","last_became_active":"2024-03-06T07:26:16.738504+0000","last_became_peered":"2024-03-06T07:26:16.738504+0000","last_unstale":"2024-03-06T07:26:18.806588+0000","last_undegraded":"2024-03-06T07:26:18.806588+0000","last_fullsized":"2024-03-06T07:26:18.806588+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:26:26.561354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806611+0000","last_change":"2024-03-06T07:26:07.697222+0000","last_active":"2024-03-06T07:26:18.806611+0000","last_peered":"2024-03-06T07:26:18.806611+0000","last_clean":"2024-03-06T07:26:18.806611+0000","last_became_active":"2024-03-06T07:26:07.696964+0000","last_became_peered":"2024-03-06T07:26:07.696964+0000","last_unstale":"2024-03-06T07:26:18.806611+0000","last_undegraded":"2024-03-06T07:26:18.806611+0000","last_fullsized":"2024-03-06T07:26:18.806611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:59.699225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"150'3","reported_seq":25,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747386+0000","last_change":"2024-03-06T07:26:06.693324+0000","last_active":"2024-03-06T07:26:18.747386+0000","last_peered":"2024-03-06T07:26:18.747386+0000","last_clean":"2024-03-06T07:26:18.747386+0000","last_became_active":"2024-03-06T07:26:06.693047+0000","last_became_peered":"2024-03-06T07:26:06.693047+0000","last_unstale":"2024-03-06T07:26:18.747386+0000","last_undegraded":"2024-03-06T07:26:18.747386+0000","last_fullsized":"2024-03-06T07:26:18.747386+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:59.568199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748941+0000","last_change":"2024-03-06T07:26:17.743284+0000","last_active":"2024-03-06T07:26:19.748941+0000","last_peered":"2024-03-06T07:26:19.748941+0000","last_clean":"2024-03-06T07:26:19.748941+0000","last_became_active":"2024-03-06T07:26:17.743033+0000","last_became_peered":"2024-03-06T07:26:17.743033+0000","last_unstale":"2024-03-06T07:26:19.748941+0000","last_undegraded":"2024-03-06T07:26:19.748941+0000","last_fullsized":"2024-03-06T07:26:19.748941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:02.284654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.761476+0000","last_change":"2024-03-06T07:26:16.745063+0000","last_active":"2024-03-06T07:26:17.761476+0000","last_peered":"2024-03-06T07:26:17.761476+0000","last_clean":"2024-03-06T07:26:17.761476+0000","last_became_active":"2024-03-06T07:26:16.744809+0000","last_became_peered":"2024-03-06T07:26:16.744809+0000","last_unstale":"2024-03-06T07:26:17.761476+0000","last_undegraded":"2024-03-06T07:26:17.761476+0000","last_fullsized":"2024-03-06T07:26:17.761476+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:28.340363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001164+0000","last_change":"2024-03-06T07:26:07.704905+0000","last_active":"2024-03-06T07:26:19.001164+0000","last_peered":"2024-03-06T07:26:19.001164+0000","last_clean":"2024-03-06T07:26:19.001164+0000","last_became_active":"2024-03-06T07:26:07.704581+0000","last_became_peered":"2024-03-06T07:26:07.704581+0000","last_unstale":"2024-03-06T07:26:19.001164+0000","last_undegraded":"2024-03-06T07:26:19.001164+0000","last_fullsized":"2024-03-06T07:26:19.001164+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:06:22.770076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"150'6","reported_seq":28,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807230+0000","last_change":"2024-03-06T07:26:06.686441+0000","last_active":"2024-03-06T07:26:18.807230+0000","last_peered":"2024-03-06T07:26:18.807230+0000","last_clean":"2024-03-06T07:26:18.807230+0000","last_became_active":"2024-03-06T07:26:06.686158+0000","last_became_peered":"2024-03-06T07:26:06.686158+0000","last_unstale":"2024-03-06T07:26:18.807230+0000","last_undegraded":"2024-03-06T07:26:18.807230+0000","last_fullsized":"2024-03-06T07:26:18.807230+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:05.262174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760489+0000","last_change":"2024-03-06T07:26:17.753430+0000","last_active":"2024-03-06T07:26:17.760489+0000","last_peered":"2024-03-06T07:26:17.760489+0000","last_clean":"2024-03-06T07:26:17.760489+0000","last_became_active":"2024-03-06T07:26:17.750028+0000","last_became_peered":"2024-03-06T07:26:17.750028+0000","last_unstale":"2024-03-06T07:26:17.760489+0000","last_undegraded":"2024-03-06T07:26:17.760489+0000","last_fullsized":"2024-03-06T07:26:17.760489+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:18.355223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.764879+0000","last_change":"2024-03-06T07:26:16.738161+0000","last_active":"2024-03-06T07:26:17.764879+0000","last_peered":"2024-03-06T07:26:17.764879+0000","last_clean":"2024-03-06T07:26:17.764879+0000","last_became_active":"2024-03-06T07:26:16.737876+0000","last_became_peered":"2024-03-06T07:26:16.737876+0000","last_unstale":"2024-03-06T07:26:17.764879+0000","last_undegraded":"2024-03-06T07:26:17.764879+0000","last_fullsized":"2024-03-06T07:26:17.764879+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:18.397443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882319+0000","last_change":"2024-03-06T07:26:07.698486+0000","last_active":"2024-03-06T07:26:18.882319+0000","last_peered":"2024-03-06T07:26:18.882319+0000","last_clean":"2024-03-06T07:26:18.882319+0000","last_became_active":"2024-03-06T07:26:07.698210+0000","last_became_peered":"2024-03-06T07:26:07.698210+0000","last_unstale":"2024-03-06T07:26:18.882319+0000","last_undegraded":"2024-03-06T07:26:18.882319+0000","last_fullsized":"2024-03-06T07:26:18.882319+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:31:41.658258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738610+0000","last_change":"2024-03-06T07:26:06.698989+0000","last_active":"2024-03-06T07:26:17.738610+0000","last_peered":"2024-03-06T07:26:17.738610+0000","last_clean":"2024-03-06T07:26:17.738610+0000","last_became_active":"2024-03-06T07:26:06.698656+0000","last_became_peered":"2024-03-06T07:26:06.698656+0000","last_unstale":"2024-03-06T07:26:17.738610+0000","last_undegraded":"2024-03-06T07:26:17.738610+0000","last_fullsized":"2024-03-06T07:26:17.738610+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882544+0000","last_change":"2024-03-06T07:26:17.752964+0000","last_active":"2024-03-06T07:26:18.882544+0000","last_peered":"2024-03-06T07:26:18.882544+0000","last_clean":"2024-03-06T07:26:18.882544+0000","last_became_active":"2024-03-06T07:26:17.751768+0000","last_became_peered":"2024-03-06T07:26:17.751768+0000","last_unstale":"2024-03-06T07:26:18.882544+0000","last_undegraded":"2024-03-06T07:26:18.882544+0000","last_fullsized":"2024-03-06T07:26:18.882544+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:45.573417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747102+0000","last_change":"2024-03-06T07:26:16.737220+0000","last_active":"2024-03-06T07:26:18.747102+0000","last_peered":"2024-03-06T07:26:18.747102+0000","last_clean":"2024-03-06T07:26:18.747102+0000","last_became_active":"2024-03-06T07:26:16.736946+0000","last_became_peered":"2024-03-06T07:26:16.736946+0000","last_unstale":"2024-03-06T07:26:18.747102+0000","last_undegraded":"2024-03-06T07:26:18.747102+0000","last_fullsized":"2024-03-06T07:26:18.747102+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:44.567994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749334+0000","last_change":"2024-03-06T07:26:07.709375+0000","last_active":"2024-03-06T07:26:19.749334+0000","last_peered":"2024-03-06T07:26:19.749334+0000","last_clean":"2024-03-06T07:26:19.749334+0000","last_became_active":"2024-03-06T07:26:07.709091+0000","last_became_peered":"2024-03-06T07:26:07.709091+0000","last_unstale":"2024-03-06T07:26:19.749334+0000","last_undegraded":"2024-03-06T07:26:19.749334+0000","last_fullsized":"2024-03-06T07:26:19.749334+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:41.934898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806630+0000","last_change":"2024-03-06T07:26:06.695622+0000","last_active":"2024-03-06T07:26:18.806630+0000","last_peered":"2024-03-06T07:26:18.806630+0000","last_clean":"2024-03-06T07:26:18.806630+0000","last_became_active":"2024-03-06T07:26:06.695369+0000","last_became_peered":"2024-03-06T07:26:06.695369+0000","last_unstale":"2024-03-06T07:26:18.806630+0000","last_undegraded":"2024-03-06T07:26:18.806630+0000","last_fullsized":"2024-03-06T07:26:18.806630+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:26.984542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745102+0000","last_change":"2024-03-06T07:26:17.748383+0000","last_active":"2024-03-06T07:26:18.745102+0000","last_peered":"2024-03-06T07:26:18.745102+0000","last_clean":"2024-03-06T07:26:18.745102+0000","last_became_active":"2024-03-06T07:26:17.748091+0000","last_became_peered":"2024-03-06T07:26:17.748091+0000","last_unstale":"2024-03-06T07:26:18.745102+0000","last_undegraded":"2024-03-06T07:26:18.745102+0000","last_fullsized":"2024-03-06T07:26:18.745102+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:34.812576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"154'4","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.759683+0000","last_change":"2024-03-06T07:26:16.743306+0000","last_active":"2024-03-06T07:26:17.759683+0000","last_peered":"2024-03-06T07:26:17.759683+0000","last_clean":"2024-03-06T07:26:17.759683+0000","last_became_active":"2024-03-06T07:26:16.741028+0000","last_became_peered":"2024-03-06T07:26:16.741028+0000","last_unstale":"2024-03-06T07:26:17.759683+0000","last_undegraded":"2024-03-06T07:26:17.759683+0000","last_fullsized":"2024-03-06T07:26:17.759683+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:04.976679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.745408+0000","last_change":"2024-03-06T07:26:07.698143+0000","last_active":"2024-03-06T07:26:17.745408+0000","last_peered":"2024-03-06T07:26:17.745408+0000","last_clean":"2024-03-06T07:26:17.745408+0000","last_became_active":"2024-03-06T07:26:07.697880+0000","last_became_peered":"2024-03-06T07:26:07.697880+0000","last_unstale":"2024-03-06T07:26:17.745408+0000","last_undegraded":"2024-03-06T07:26:17.745408+0000","last_fullsized":"2024-03-06T07:26:17.745408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744836+0000","last_change":"2024-03-06T07:26:06.696386+0000","last_active":"2024-03-06T07:26:17.744836+0000","last_peered":"2024-03-06T07:26:17.744836+0000","last_clean":"2024-03-06T07:26:17.744836+0000","last_became_active":"2024-03-06T07:26:06.696108+0000","last_became_peered":"2024-03-06T07:26:06.696108+0000","last_unstale":"2024-03-06T07:26:17.744836+0000","last_undegraded":"2024-03-06T07:26:17.744836+0000","last_fullsized":"2024-03-06T07:26:17.744836+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001269+0000","last_change":"2024-03-06T07:26:17.738958+0000","last_active":"2024-03-06T07:26:19.001269+0000","last_peered":"2024-03-06T07:26:19.001269+0000","last_clean":"2024-03-06T07:26:19.001269+0000","last_became_active":"2024-03-06T07:26:17.738699+0000","last_became_peered":"2024-03-06T07:26:17.738699+0000","last_unstale":"2024-03-06T07:26:19.001269+0000","last_undegraded":"2024-03-06T07:26:19.001269+0000","last_fullsized":"2024-03-06T07:26:19.001269+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:42.267362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807053+0000","last_change":"2024-03-06T07:26:16.738986+0000","last_active":"2024-03-06T07:26:18.807053+0000","last_peered":"2024-03-06T07:26:18.807053+0000","last_clean":"2024-03-06T07:26:18.807053+0000","last_became_active":"2024-03-06T07:26:16.738729+0000","last_became_peered":"2024-03-06T07:26:16.738729+0000","last_unstale":"2024-03-06T07:26:18.807053+0000","last_undegraded":"2024-03-06T07:26:18.807053+0000","last_fullsized":"2024-03-06T07:26:18.807053+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:22.333796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001707+0000","last_change":"2024-03-06T07:26:07.694636+0000","last_active":"2024-03-06T07:26:19.001707+0000","last_peered":"2024-03-06T07:26:19.001707+0000","last_clean":"2024-03-06T07:26:19.001707+0000","last_became_active":"2024-03-06T07:26:07.694340+0000","last_became_peered":"2024-03-06T07:26:07.694340+0000","last_unstale":"2024-03-06T07:26:19.001707+0000","last_undegraded":"2024-03-06T07:26:19.001707+0000","last_fullsized":"2024-03-06T07:26:19.001707+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:42.904456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"150'8","reported_seq":30,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748965+0000","last_change":"2024-03-06T07:26:06.692364+0000","last_active":"2024-03-06T07:26:19.748965+0000","last_peered":"2024-03-06T07:26:19.748965+0000","last_clean":"2024-03-06T07:26:19.748965+0000","last_became_active":"2024-03-06T07:26:06.692119+0000","last_became_peered":"2024-03-06T07:26:06.692119+0000","last_unstale":"2024-03-06T07:26:19.748965+0000","last_undegraded":"2024-03-06T07:26:19.748965+0000","last_fullsized":"2024-03-06T07:26:19.748965+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749159+0000","last_change":"2024-03-06T07:26:17.745908+0000","last_active":"2024-03-06T07:26:19.749159+0000","last_peered":"2024-03-06T07:26:19.749159+0000","last_clean":"2024-03-06T07:26:19.749159+0000","last_became_active":"2024-03-06T07:26:17.745649+0000","last_became_peered":"2024-03-06T07:26:17.745649+0000","last_unstale":"2024-03-06T07:26:19.749159+0000","last_undegraded":"2024-03-06T07:26:19.749159+0000","last_fullsized":"2024-03-06T07:26:19.749159+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:12.706034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.000990+0000","last_change":"2024-03-06T07:26:16.736287+0000","last_active":"2024-03-06T07:26:19.000990+0000","last_peered":"2024-03-06T07:26:19.000990+0000","last_clean":"2024-03-06T07:26:19.000990+0000","last_became_active":"2024-03-06T07:26:16.734127+0000","last_became_peered":"2024-03-06T07:26:16.734127+0000","last_unstale":"2024-03-06T07:26:19.000990+0000","last_undegraded":"2024-03-06T07:26:19.000990+0000","last_fullsized":"2024-03-06T07:26:19.000990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:27.821504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748818+0000","last_change":"2024-03-06T07:26:07.698466+0000","last_active":"2024-03-06T07:26:19.748818+0000","last_peered":"2024-03-06T07:26:19.748818+0000","last_clean":"2024-03-06T07:26:19.748818+0000","last_became_active":"2024-03-06T07:26:07.697848+0000","last_became_peered":"2024-03-06T07:26:07.697848+0000","last_unstale":"2024-03-06T07:26:19.748818+0000","last_undegraded":"2024-03-06T07:26:19.748818+0000","last_fullsized":"2024-03-06T07:26:19.748818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"150'1","reported_seq":23,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807568+0000","last_change":"2024-03-06T07:26:06.694226+0000","last_active":"2024-03-06T07:26:18.807568+0000","last_peered":"2024-03-06T07:26:18.807568+0000","last_clean":"2024-03-06T07:26:18.807568+0000","last_became_active":"2024-03-06T07:26:06.693966+0000","last_became_peered":"2024-03-06T07:26:06.693966+0000","last_unstale":"2024-03-06T07:26:18.807568+0000","last_undegraded":"2024-03-06T07:26:18.807568+0000","last_fullsized":"2024-03-06T07:26:18.807568+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:45.769621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744786+0000","last_change":"2024-03-06T07:26:17.746782+0000","last_active":"2024-03-06T07:26:18.744786+0000","last_peered":"2024-03-06T07:26:18.744786+0000","last_clean":"2024-03-06T07:26:18.744786+0000","last_became_active":"2024-03-06T07:26:17.746473+0000","last_became_peered":"2024-03-06T07:26:17.746473+0000","last_unstale":"2024-03-06T07:26:18.744786+0000","last_undegraded":"2024-03-06T07:26:18.744786+0000","last_fullsized":"2024-03-06T07:26:18.744786+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:02.284654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882207+0000","last_change":"2024-03-06T07:26:16.738787+0000","last_active":"2024-03-06T07:26:18.882207+0000","last_peered":"2024-03-06T07:26:18.882207+0000","last_clean":"2024-03-06T07:26:18.882207+0000","last_became_active":"2024-03-06T07:26:16.738428+0000","last_became_peered":"2024-03-06T07:26:16.738428+0000","last_unstale":"2024-03-06T07:26:18.882207+0000","last_undegraded":"2024-03-06T07:26:18.882207+0000","last_fullsized":"2024-03-06T07:26:18.882207+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:21.825588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882439+0000","last_change":"2024-03-06T07:26:07.695629+0000","last_active":"2024-03-06T07:26:18.882439+0000","last_peered":"2024-03-06T07:26:18.882439+0000","last_clean":"2024-03-06T07:26:18.882439+0000","last_became_active":"2024-03-06T07:26:07.695358+0000","last_became_peered":"2024-03-06T07:26:07.695358+0000","last_unstale":"2024-03-06T07:26:18.882439+0000","last_undegraded":"2024-03-06T07:26:18.882439+0000","last_fullsized":"2024-03-06T07:26:18.882439+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:32.222328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"150'2","reported_seq":24,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749766+0000","last_change":"2024-03-06T07:26:06.698927+0000","last_active":"2024-03-06T07:26:19.749766+0000","last_peered":"2024-03-06T07:26:19.749766+0000","last_clean":"2024-03-06T07:26:19.749766+0000","last_became_active":"2024-03-06T07:26:06.698636+0000","last_became_peered":"2024-03-06T07:26:06.698636+0000","last_unstale":"2024-03-06T07:26:19.749766+0000","last_undegraded":"2024-03-06T07:26:19.749766+0000","last_fullsized":"2024-03-06T07:26:19.749766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882217+0000","last_change":"2024-03-06T07:26:17.752227+0000","last_active":"2024-03-06T07:26:18.882217+0000","last_peered":"2024-03-06T07:26:18.882217+0000","last_clean":"2024-03-06T07:26:18.882217+0000","last_became_active":"2024-03-06T07:26:17.751966+0000","last_became_peered":"2024-03-06T07:26:17.751966+0000","last_unstale":"2024-03-06T07:26:18.882217+0000","last_undegraded":"2024-03-06T07:26:18.882217+0000","last_fullsized":"2024-03-06T07:26:18.882217+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:20.134232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744723+0000","last_change":"2024-03-06T07:26:16.734670+0000","last_active":"2024-03-06T07:26:18.744723+0000","last_peered":"2024-03-06T07:26:18.744723+0000","last_clean":"2024-03-06T07:26:18.744723+0000","last_became_active":"2024-03-06T07:26:16.733618+0000","last_became_peered":"2024-03-06T07:26:16.733618+0000","last_unstale":"2024-03-06T07:26:18.744723+0000","last_undegraded":"2024-03-06T07:26:18.744723+0000","last_fullsized":"2024-03-06T07:26:18.744723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:38.826593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744196+0000","last_change":"2024-03-06T07:26:07.697409+0000","last_active":"2024-03-06T07:26:17.744196+0000","last_peered":"2024-03-06T07:26:17.744196+0000","last_clean":"2024-03-06T07:26:17.744196+0000","last_became_active":"2024-03-06T07:26:07.697084+0000","last_became_peered":"2024-03-06T07:26:07.697084+0000","last_unstale":"2024-03-06T07:26:17.744196+0000","last_undegraded":"2024-03-06T07:26:17.744196+0000","last_fullsized":"2024-03-06T07:26:17.744196+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"150'1","reported_seq":21,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.739525+0000","last_change":"2024-03-06T07:26:06.695033+0000","last_active":"2024-03-06T07:26:17.739525+0000","last_peered":"2024-03-06T07:26:17.739525+0000","last_clean":"2024-03-06T07:26:17.739525+0000","last_became_active":"2024-03-06T07:26:06.694763+0000","last_became_peered":"2024-03-06T07:26:06.694763+0000","last_unstale":"2024-03-06T07:26:17.739525+0000","last_undegraded":"2024-03-06T07:26:17.739525+0000","last_fullsized":"2024-03-06T07:26:17.739525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:29.440623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746363+0000","last_change":"2024-03-06T07:26:17.752315+0000","last_active":"2024-03-06T07:26:18.746363+0000","last_peered":"2024-03-06T07:26:18.746363+0000","last_clean":"2024-03-06T07:26:18.746363+0000","last_became_active":"2024-03-06T07:26:17.752025+0000","last_became_peered":"2024-03-06T07:26:17.752025+0000","last_unstale":"2024-03-06T07:26:18.746363+0000","last_undegraded":"2024-03-06T07:26:18.746363+0000","last_fullsized":"2024-03-06T07:26:18.746363+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:28.209449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744560+0000","last_change":"2024-03-06T07:26:16.735510+0000","last_active":"2024-03-06T07:26:18.744560+0000","last_peered":"2024-03-06T07:26:18.744560+0000","last_clean":"2024-03-06T07:26:18.744560+0000","last_became_active":"2024-03-06T07:26:16.735238+0000","last_became_peered":"2024-03-06T07:26:16.735238+0000","last_unstale":"2024-03-06T07:26:18.744560+0000","last_undegraded":"2024-03-06T07:26:18.744560+0000","last_fullsized":"2024-03-06T07:26:18.744560+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:12.997028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744500+0000","last_change":"2024-03-06T07:26:07.706589+0000","last_active":"2024-03-06T07:26:17.744500+0000","last_peered":"2024-03-06T07:26:17.744500+0000","last_clean":"2024-03-06T07:26:17.744500+0000","last_became_active":"2024-03-06T07:26:07.706305+0000","last_became_peered":"2024-03-06T07:26:07.706305+0000","last_unstale":"2024-03-06T07:26:17.744500+0000","last_undegraded":"2024-03-06T07:26:17.744500+0000","last_fullsized":"2024-03-06T07:26:17.744500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749365+0000","last_change":"2024-03-06T07:26:06.697335+0000","last_active":"2024-03-06T07:26:19.749365+0000","last_peered":"2024-03-06T07:26:19.749365+0000","last_clean":"2024-03-06T07:26:19.749365+0000","last_became_active":"2024-03-06T07:26:06.697028+0000","last_became_peered":"2024-03-06T07:26:06.697028+0000","last_unstale":"2024-03-06T07:26:19.749365+0000","last_undegraded":"2024-03-06T07:26:19.749365+0000","last_fullsized":"2024-03-06T07:26:19.749365+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745088+0000","last_change":"2024-03-06T07:26:17.754920+0000","last_active":"2024-03-06T07:26:18.745088+0000","last_peered":"2024-03-06T07:26:18.745088+0000","last_clean":"2024-03-06T07:26:18.745088+0000","last_became_active":"2024-03-06T07:26:17.754660+0000","last_became_peered":"2024-03-06T07:26:17.754660+0000","last_unstale":"2024-03-06T07:26:18.745088+0000","last_undegraded":"2024-03-06T07:26:18.745088+0000","last_fullsized":"2024-03-06T07:26:18.745088+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:12.706034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806988+0000","last_change":"2024-03-06T07:26:16.738310+0000","last_active":"2024-03-06T07:26:18.806988+0000","last_peered":"2024-03-06T07:26:18.806988+0000","last_clean":"2024-03-06T07:26:18.806988+0000","last_became_active":"2024-03-06T07:26:16.738040+0000","last_became_peered":"2024-03-06T07:26:16.738040+0000","last_unstale":"2024-03-06T07:26:18.806988+0000","last_undegraded":"2024-03-06T07:26:18.806988+0000","last_fullsized":"2024-03-06T07:26:18.806988+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:28.199517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807146+0000","last_change":"2024-03-06T07:26:07.692326+0000","last_active":"2024-03-06T07:26:18.807146+0000","last_peered":"2024-03-06T07:26:18.807146+0000","last_clean":"2024-03-06T07:26:18.807146+0000","last_became_active":"2024-03-06T07:26:07.692063+0000","last_became_peered":"2024-03-06T07:26:07.692063+0000","last_unstale":"2024-03-06T07:26:18.807146+0000","last_undegraded":"2024-03-06T07:26:18.807146+0000","last_fullsized":"2024-03-06T07:26:18.807146+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:42.601237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738725+0000","last_change":"2024-03-06T07:26:06.696078+0000","last_active":"2024-03-06T07:26:17.738725+0000","last_peered":"2024-03-06T07:26:17.738725+0000","last_clean":"2024-03-06T07:26:17.738725+0000","last_became_active":"2024-03-06T07:26:06.695834+0000","last_became_peered":"2024-03-06T07:26:06.695834+0000","last_unstale":"2024-03-06T07:26:17.738725+0000","last_undegraded":"2024-03-06T07:26:17.738725+0000","last_fullsized":"2024-03-06T07:26:17.738725+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001738+0000","last_change":"2024-03-06T07:26:17.751409+0000","last_active":"2024-03-06T07:26:19.001738+0000","last_peered":"2024-03-06T07:26:19.001738+0000","last_clean":"2024-03-06T07:26:19.001738+0000","last_became_active":"2024-03-06T07:26:17.751151+0000","last_became_peered":"2024-03-06T07:26:17.751151+0000","last_unstale":"2024-03-06T07:26:19.001738+0000","last_undegraded":"2024-03-06T07:26:19.001738+0000","last_fullsized":"2024-03-06T07:26:19.001738+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:27.892875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807252+0000","last_change":"2024-03-06T07:26:16.740507+0000","last_active":"2024-03-06T07:26:18.807252+0000","last_peered":"2024-03-06T07:26:18.807252+0000","last_clean":"2024-03-06T07:26:18.807252+0000","last_became_active":"2024-03-06T07:26:16.740247+0000","last_became_peered":"2024-03-06T07:26:16.740247+0000","last_unstale":"2024-03-06T07:26:18.807252+0000","last_undegraded":"2024-03-06T07:26:18.807252+0000","last_fullsized":"2024-03-06T07:26:18.807252+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:50.785757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.739432+0000","last_change":"2024-03-06T07:26:07.713113+0000","last_active":"2024-03-06T07:26:17.739432+0000","last_peered":"2024-03-06T07:26:17.739432+0000","last_clean":"2024-03-06T07:26:17.739432+0000","last_became_active":"2024-03-06T07:26:07.712610+0000","last_became_peered":"2024-03-06T07:26:07.712610+0000","last_unstale":"2024-03-06T07:26:17.739432+0000","last_undegraded":"2024-03-06T07:26:17.739432+0000","last_fullsized":"2024-03-06T07:26:17.739432+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:03.957417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744286+0000","last_change":"2024-03-06T07:26:06.692781+0000","last_active":"2024-03-06T07:26:17.744286+0000","last_peered":"2024-03-06T07:26:17.744286+0000","last_clean":"2024-03-06T07:26:17.744286+0000","last_became_active":"2024-03-06T07:26:06.691992+0000","last_became_peered":"2024-03-06T07:26:06.691992+0000","last_unstale":"2024-03-06T07:26:17.744286+0000","last_undegraded":"2024-03-06T07:26:17.744286+0000","last_fullsized":"2024-03-06T07:26:17.744286+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001409+0000","last_change":"2024-03-06T07:26:17.746271+0000","last_active":"2024-03-06T07:26:19.001409+0000","last_peered":"2024-03-06T07:26:19.001409+0000","last_clean":"2024-03-06T07:26:19.001409+0000","last_became_active":"2024-03-06T07:26:17.746013+0000","last_became_peered":"2024-03-06T07:26:17.746013+0000","last_unstale":"2024-03-06T07:26:19.001409+0000","last_undegraded":"2024-03-06T07:26:19.001409+0000","last_fullsized":"2024-03-06T07:26:19.001409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:55.454349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760759+0000","last_change":"2024-03-06T07:26:16.742136+0000","last_active":"2024-03-06T07:26:17.760759+0000","last_peered":"2024-03-06T07:26:17.760759+0000","last_clean":"2024-03-06T07:26:17.760759+0000","last_became_active":"2024-03-06T07:26:16.740809+0000","last_became_peered":"2024-03-06T07:26:16.740809+0000","last_unstale":"2024-03-06T07:26:17.760759+0000","last_undegraded":"2024-03-06T07:26:17.760759+0000","last_fullsized":"2024-03-06T07:26:17.760759+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:10:46.963501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738881+0000","last_change":"2024-03-06T07:26:07.713547+0000","last_active":"2024-03-06T07:26:17.738881+0000","last_peered":"2024-03-06T07:26:17.738881+0000","last_clean":"2024-03-06T07:26:17.738881+0000","last_became_active":"2024-03-06T07:26:07.712950+0000","last_became_peered":"2024-03-06T07:26:07.712950+0000","last_unstale":"2024-03-06T07:26:17.738881+0000","last_undegraded":"2024-03-06T07:26:17.738881+0000","last_fullsized":"2024-03-06T07:26:17.738881+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:00.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001469+0000","last_change":"2024-03-06T07:26:06.695426+0000","last_active":"2024-03-06T07:26:19.001469+0000","last_peered":"2024-03-06T07:26:19.001469+0000","last_clean":"2024-03-06T07:26:19.001469+0000","last_became_active":"2024-03-06T07:26:06.695162+0000","last_became_peered":"2024-03-06T07:26:06.695162+0000","last_unstale":"2024-03-06T07:26:19.001469+0000","last_undegraded":"2024-03-06T07:26:19.001469+0000","last_fullsized":"2024-03-06T07:26:19.001469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:30.485289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749421+0000","last_change":"2024-03-06T07:26:17.740123+0000","last_active":"2024-03-06T07:26:19.749421+0000","last_peered":"2024-03-06T07:26:19.749421+0000","last_clean":"2024-03-06T07:26:19.749421+0000","last_became_active":"2024-03-06T07:26:17.739862+0000","last_became_peered":"2024-03-06T07:26:17.739862+0000","last_unstale":"2024-03-06T07:26:19.749421+0000","last_undegraded":"2024-03-06T07:26:19.749421+0000","last_fullsized":"2024-03-06T07:26:19.749421+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:05.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"155'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748293+0000","last_change":"2024-03-06T07:26:16.737585+0000","last_active":"2024-03-06T07:26:19.748293+0000","last_peered":"2024-03-06T07:26:19.748293+0000","last_clean":"2024-03-06T07:26:19.748293+0000","last_became_active":"2024-03-06T07:26:16.737117+0000","last_became_peered":"2024-03-06T07:26:16.737117+0000","last_unstale":"2024-03-06T07:26:19.748293+0000","last_undegraded":"2024-03-06T07:26:19.748293+0000","last_fullsized":"2024-03-06T07:26:19.748293+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:26.441757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745500+0000","last_change":"2024-03-06T07:26:07.691244+0000","last_active":"2024-03-06T07:26:18.745500+0000","last_peered":"2024-03-06T07:26:18.745500+0000","last_clean":"2024-03-06T07:26:18.745500+0000","last_became_active":"2024-03-06T07:26:07.690929+0000","last_became_peered":"2024-03-06T07:26:07.690929+0000","last_unstale":"2024-03-06T07:26:18.745500+0000","last_undegraded":"2024-03-06T07:26:18.745500+0000","last_fullsized":"2024-03-06T07:26:18.745500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:27.989980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745237+0000","last_change":"2024-03-06T07:26:06.694068+0000","last_active":"2024-03-06T07:26:18.745237+0000","last_peered":"2024-03-06T07:26:18.745237+0000","last_clean":"2024-03-06T07:26:18.745237+0000","last_became_active":"2024-03-06T07:26:06.693777+0000","last_became_peered":"2024-03-06T07:26:06.693777+0000","last_unstale":"2024-03-06T07:26:18.745237+0000","last_undegraded":"2024-03-06T07:26:18.745237+0000","last_fullsized":"2024-03-06T07:26:18.745237+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:06.186255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.758362+0000","last_change":"2024-03-06T07:26:17.753731+0000","last_active":"2024-03-06T07:26:17.758362+0000","last_peered":"2024-03-06T07:26:17.758362+0000","last_clean":"2024-03-06T07:26:17.758362+0000","last_became_active":"2024-03-06T07:26:17.753483+0000","last_became_peered":"2024-03-06T07:26:17.753483+0000","last_unstale":"2024-03-06T07:26:17.758362+0000","last_undegraded":"2024-03-06T07:26:17.758362+0000","last_fullsized":"2024-03-06T07:26:17.758362+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:23.339219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.000941+0000","last_change":"2024-03-06T07:26:16.736897+0000","last_active":"2024-03-06T07:26:19.000941+0000","last_peered":"2024-03-06T07:26:19.000941+0000","last_clean":"2024-03-06T07:26:19.000941+0000","last_became_active":"2024-03-06T07:26:16.736623+0000","last_became_peered":"2024-03-06T07:26:16.736623+0000","last_unstale":"2024-03-06T07:26:19.000941+0000","last_undegraded":"2024-03-06T07:26:19.000941+0000","last_fullsized":"2024-03-06T07:26:19.000941+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:03.929124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743967+0000","last_change":"2024-03-06T07:26:07.696197+0000","last_active":"2024-03-06T07:26:17.743967+0000","last_peered":"2024-03-06T07:26:17.743967+0000","last_clean":"2024-03-06T07:26:17.743967+0000","last_became_active":"2024-03-06T07:26:07.695895+0000","last_became_peered":"2024-03-06T07:26:07.695895+0000","last_unstale":"2024-03-06T07:26:17.743967+0000","last_undegraded":"2024-03-06T07:26:17.743967+0000","last_fullsized":"2024-03-06T07:26:17.743967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:02.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.740239+0000","last_change":"2024-03-06T07:26:06.701125+0000","last_active":"2024-03-06T07:26:17.740239+0000","last_peered":"2024-03-06T07:26:17.740239+0000","last_clean":"2024-03-06T07:26:17.740239+0000","last_became_active":"2024-03-06T07:26:06.700819+0000","last_became_peered":"2024-03-06T07:26:06.700819+0000","last_unstale":"2024-03-06T07:26:17.740239+0000","last_undegraded":"2024-03-06T07:26:17.740239+0000","last_fullsized":"2024-03-06T07:26:17.740239+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:54.995466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806854+0000","last_change":"2024-03-06T07:26:17.745253+0000","last_active":"2024-03-06T07:26:18.806854+0000","last_peered":"2024-03-06T07:26:18.806854+0000","last_clean":"2024-03-06T07:26:18.806854+0000","last_became_active":"2024-03-06T07:26:17.744711+0000","last_became_peered":"2024-03-06T07:26:17.744711+0000","last_unstale":"2024-03-06T07:26:18.806854+0000","last_undegraded":"2024-03-06T07:26:18.806854+0000","last_fullsized":"2024-03-06T07:26:18.806854+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:57.856041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882331+0000","last_change":"2024-03-06T07:26:16.738255+0000","last_active":"2024-03-06T07:26:18.882331+0000","last_peered":"2024-03-06T07:26:18.882331+0000","last_clean":"2024-03-06T07:26:18.882331+0000","last_became_active":"2024-03-06T07:26:16.737911+0000","last_became_peered":"2024-03-06T07:26:16.737911+0000","last_unstale":"2024-03-06T07:26:18.882331+0000","last_undegraded":"2024-03-06T07:26:18.882331+0000","last_fullsized":"2024-03-06T07:26:18.882331+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:42:35.210727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749183+0000","last_change":"2024-03-06T07:26:07.701964+0000","last_active":"2024-03-06T07:26:19.749183+0000","last_peered":"2024-03-06T07:26:19.749183+0000","last_clean":"2024-03-06T07:26:19.749183+0000","last_became_active":"2024-03-06T07:26:07.701660+0000","last_became_peered":"2024-03-06T07:26:07.701660+0000","last_unstale":"2024-03-06T07:26:19.749183+0000","last_undegraded":"2024-03-06T07:26:19.749183+0000","last_fullsized":"2024-03-06T07:26:19.749183+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"151'1","reported_seq":23,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807276+0000","last_change":"2024-03-06T07:26:06.693341+0000","last_active":"2024-03-06T07:26:18.807276+0000","last_peered":"2024-03-06T07:26:18.807276+0000","last_clean":"2024-03-06T07:26:18.807276+0000","last_became_active":"2024-03-06T07:26:06.693014+0000","last_became_peered":"2024-03-06T07:26:06.693014+0000","last_unstale":"2024-03-06T07:26:18.807276+0000","last_undegraded":"2024-03-06T07:26:18.807276+0000","last_fullsized":"2024-03-06T07:26:18.807276+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:03.887560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748771+0000","last_change":"2024-03-06T07:26:17.747098+0000","last_active":"2024-03-06T07:26:19.748771+0000","last_peered":"2024-03-06T07:26:19.748771+0000","last_clean":"2024-03-06T07:26:19.748771+0000","last_became_active":"2024-03-06T07:26:17.746849+0000","last_became_peered":"2024-03-06T07:26:17.746849+0000","last_unstale":"2024-03-06T07:26:19.748771+0000","last_undegraded":"2024-03-06T07:26:19.748771+0000","last_fullsized":"2024-03-06T07:26:19.748771+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:52.646271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746743+0000","last_change":"2024-03-06T07:26:16.742535+0000","last_active":"2024-03-06T07:26:18.746743+0000","last_peered":"2024-03-06T07:26:18.746743+0000","last_clean":"2024-03-06T07:26:18.746743+0000","last_became_active":"2024-03-06T07:26:16.742246+0000","last_became_peered":"2024-03-06T07:26:16.742246+0000","last_unstale":"2024-03-06T07:26:18.746743+0000","last_undegraded":"2024-03-06T07:26:18.746743+0000","last_fullsized":"2024-03-06T07:26:18.746743+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:15:44.025380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001197+0000","last_change":"2024-03-06T07:26:07.697077+0000","last_active":"2024-03-06T07:26:19.001197+0000","last_peered":"2024-03-06T07:26:19.001197+0000","last_clean":"2024-03-06T07:26:19.001197+0000","last_became_active":"2024-03-06T07:26:07.695444+0000","last_became_peered":"2024-03-06T07:26:07.695444+0000","last_unstale":"2024-03-06T07:26:19.001197+0000","last_undegraded":"2024-03-06T07:26:19.001197+0000","last_fullsized":"2024-03-06T07:26:19.001197+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:56.050281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807063+0000","last_change":"2024-03-06T07:26:06.689881+0000","last_active":"2024-03-06T07:26:18.807063+0000","last_peered":"2024-03-06T07:26:18.807063+0000","last_clean":"2024-03-06T07:26:18.807063+0000","last_became_active":"2024-03-06T07:26:06.689604+0000","last_became_peered":"2024-03-06T07:26:06.689604+0000","last_unstale":"2024-03-06T07:26:18.807063+0000","last_undegraded":"2024-03-06T07:26:18.807063+0000","last_fullsized":"2024-03-06T07:26:18.807063+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.752325+0000","last_change":"2024-03-06T07:26:17.751363+0000","last_active":"2024-03-06T07:26:17.752325+0000","last_peered":"2024-03-06T07:26:17.752325+0000","last_clean":"2024-03-06T07:26:17.752325+0000","last_became_active":"2024-03-06T07:26:17.751115+0000","last_became_peered":"2024-03-06T07:26:17.751115+0000","last_unstale":"2024-03-06T07:26:17.752325+0000","last_undegraded":"2024-03-06T07:26:17.752325+0000","last_fullsized":"2024-03-06T07:26:17.752325+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:49:23.743634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743400+0000","last_change":"2024-03-06T07:26:16.745483+0000","last_active":"2024-03-06T07:26:17.743400+0000","last_peered":"2024-03-06T07:26:17.743400+0000","last_clean":"2024-03-06T07:26:17.743400+0000","last_became_active":"2024-03-06T07:26:16.745181+0000","last_became_peered":"2024-03-06T07:26:16.745181+0000","last_unstale":"2024-03-06T07:26:17.743400+0000","last_undegraded":"2024-03-06T07:26:17.743400+0000","last_fullsized":"2024-03-06T07:26:17.743400+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:01.140243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749144+0000","last_change":"2024-03-06T07:26:07.700732+0000","last_active":"2024-03-06T07:26:19.749144+0000","last_peered":"2024-03-06T07:26:19.749144+0000","last_clean":"2024-03-06T07:26:19.749144+0000","last_became_active":"2024-03-06T07:26:07.700382+0000","last_became_peered":"2024-03-06T07:26:07.700382+0000","last_unstale":"2024-03-06T07:26:19.749144+0000","last_undegraded":"2024-03-06T07:26:19.749144+0000","last_fullsized":"2024-03-06T07:26:19.749144+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:29.340249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001565+0000","last_change":"2024-03-06T07:26:06.693374+0000","last_active":"2024-03-06T07:26:19.001565+0000","last_peered":"2024-03-06T07:26:19.001565+0000","last_clean":"2024-03-06T07:26:19.001565+0000","last_became_active":"2024-03-06T07:26:06.693113+0000","last_became_peered":"2024-03-06T07:26:06.693113+0000","last_unstale":"2024-03-06T07:26:19.001565+0000","last_undegraded":"2024-03-06T07:26:19.001565+0000","last_fullsized":"2024-03-06T07:26:19.001565+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:55.760111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.752802+0000","last_change":"2024-03-06T07:26:17.750146+0000","last_active":"2024-03-06T07:26:17.752802+0000","last_peered":"2024-03-06T07:26:17.752802+0000","last_clean":"2024-03-06T07:26:17.752802+0000","last_became_active":"2024-03-06T07:26:17.749567+0000","last_became_peered":"2024-03-06T07:26:17.749567+0000","last_unstale":"2024-03-06T07:26:17.752802+0000","last_undegraded":"2024-03-06T07:26:17.752802+0000","last_fullsized":"2024-03-06T07:26:17.752802+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:14.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882651+0000","last_change":"2024-03-06T07:26:16.742521+0000","last_active":"2024-03-06T07:26:18.882651+0000","last_peered":"2024-03-06T07:26:18.882651+0000","last_clean":"2024-03-06T07:26:18.882651+0000","last_became_active":"2024-03-06T07:26:16.742257+0000","last_became_peered":"2024-03-06T07:26:16.742257+0000","last_unstale":"2024-03-06T07:26:18.882651+0000","last_undegraded":"2024-03-06T07:26:18.882651+0000","last_fullsized":"2024-03-06T07:26:18.882651+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:58:48.101127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747248+0000","last_change":"2024-03-06T07:26:07.701237+0000","last_active":"2024-03-06T07:26:18.747248+0000","last_peered":"2024-03-06T07:26:18.747248+0000","last_clean":"2024-03-06T07:26:18.747248+0000","last_became_active":"2024-03-06T07:26:07.700717+0000","last_became_peered":"2024-03-06T07:26:07.700717+0000","last_unstale":"2024-03-06T07:26:18.747248+0000","last_undegraded":"2024-03-06T07:26:18.747248+0000","last_fullsized":"2024-03-06T07:26:18.747248+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:42:26.171117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746916+0000","last_change":"2024-03-06T07:26:06.695137+0000","last_active":"2024-03-06T07:26:18.746916+0000","last_peered":"2024-03-06T07:26:18.746916+0000","last_clean":"2024-03-06T07:26:18.746916+0000","last_became_active":"2024-03-06T07:26:06.694881+0000","last_became_peered":"2024-03-06T07:26:06.694881+0000","last_unstale":"2024-03-06T07:26:18.746916+0000","last_undegraded":"2024-03-06T07:26:18.746916+0000","last_fullsized":"2024-03-06T07:26:18.746916+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:37.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749590+0000","last_change":"2024-03-06T07:26:17.746350+0000","last_active":"2024-03-06T07:26:19.749590+0000","last_peered":"2024-03-06T07:26:19.749590+0000","last_clean":"2024-03-06T07:26:19.749590+0000","last_became_active":"2024-03-06T07:26:17.745703+0000","last_became_peered":"2024-03-06T07:26:17.745703+0000","last_unstale":"2024-03-06T07:26:19.749590+0000","last_undegraded":"2024-03-06T07:26:19.749590+0000","last_fullsized":"2024-03-06T07:26:19.749590+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:49:23.743634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.745179+0000","last_change":"2024-03-06T07:26:16.737133+0000","last_active":"2024-03-06T07:26:17.745179+0000","last_peered":"2024-03-06T07:26:17.745179+0000","last_clean":"2024-03-06T07:26:17.745179+0000","last_became_active":"2024-03-06T07:26:16.736855+0000","last_became_peered":"2024-03-06T07:26:16.736855+0000","last_unstale":"2024-03-06T07:26:17.745179+0000","last_undegraded":"2024-03-06T07:26:17.745179+0000","last_fullsized":"2024-03-06T07:26:17.745179+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:57.088499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745335+0000","last_change":"2024-03-06T07:26:07.705151+0000","last_active":"2024-03-06T07:26:18.745335+0000","last_peered":"2024-03-06T07:26:18.745335+0000","last_clean":"2024-03-06T07:26:18.745335+0000","last_became_active":"2024-03-06T07:26:07.704856+0000","last_became_peered":"2024-03-06T07:26:07.704856+0000","last_unstale":"2024-03-06T07:26:18.745335+0000","last_undegraded":"2024-03-06T07:26:18.745335+0000","last_fullsized":"2024-03-06T07:26:18.745335+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:06.267612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806747+0000","last_change":"2024-03-06T07:26:06.684537+0000","last_active":"2024-03-06T07:26:18.806747+0000","last_peered":"2024-03-06T07:26:18.806747+0000","last_clean":"2024-03-06T07:26:18.806747+0000","last_became_active":"2024-03-06T07:26:06.683851+0000","last_became_peered":"2024-03-06T07:26:06.683851+0000","last_unstale":"2024-03-06T07:26:18.806747+0000","last_undegraded":"2024-03-06T07:26:18.806747+0000","last_fullsized":"2024-03-06T07:26:18.806747+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:07.304752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001488+0000","last_change":"2024-03-06T07:26:17.744558+0000","last_active":"2024-03-06T07:26:19.001488+0000","last_peered":"2024-03-06T07:26:19.001488+0000","last_clean":"2024-03-06T07:26:19.001488+0000","last_became_active":"2024-03-06T07:26:17.744304+0000","last_became_peered":"2024-03-06T07:26:17.744304+0000","last_unstale":"2024-03-06T07:26:19.001488+0000","last_undegraded":"2024-03-06T07:26:19.001488+0000","last_fullsized":"2024-03-06T07:26:19.001488+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:07.801361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.764903+0000","last_change":"2024-03-06T07:26:16.744565+0000","last_active":"2024-03-06T07:26:17.764903+0000","last_peered":"2024-03-06T07:26:17.764903+0000","last_clean":"2024-03-06T07:26:17.764903+0000","last_became_active":"2024-03-06T07:26:16.744244+0000","last_became_peered":"2024-03-06T07:26:16.744244+0000","last_unstale":"2024-03-06T07:26:17.764903+0000","last_undegraded":"2024-03-06T07:26:17.764903+0000","last_fullsized":"2024-03-06T07:26:17.764903+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:55.814670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807105+0000","last_change":"2024-03-06T07:26:07.691230+0000","last_active":"2024-03-06T07:26:18.807105+0000","last_peered":"2024-03-06T07:26:18.807105+0000","last_clean":"2024-03-06T07:26:18.807105+0000","last_became_active":"2024-03-06T07:26:07.690936+0000","last_became_peered":"2024-03-06T07:26:07.690936+0000","last_unstale":"2024-03-06T07:26:18.807105+0000","last_undegraded":"2024-03-06T07:26:18.807105+0000","last_fullsized":"2024-03-06T07:26:18.807105+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:09.494792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744919+0000","last_change":"2024-03-06T07:26:06.692326+0000","last_active":"2024-03-06T07:26:17.744919+0000","last_peered":"2024-03-06T07:26:17.744919+0000","last_clean":"2024-03-06T07:26:17.744919+0000","last_became_active":"2024-03-06T07:26:06.691972+0000","last_became_peered":"2024-03-06T07:26:06.691972+0000","last_unstale":"2024-03-06T07:26:17.744919+0000","last_undegraded":"2024-03-06T07:26:17.744919+0000","last_fullsized":"2024-03-06T07:26:17.744919+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806870+0000","last_change":"2024-03-06T07:26:17.750249+0000","last_active":"2024-03-06T07:26:18.806870+0000","last_peered":"2024-03-06T07:26:18.806870+0000","last_clean":"2024-03-06T07:26:18.806870+0000","last_became_active":"2024-03-06T07:26:17.750006+0000","last_became_peered":"2024-03-06T07:26:17.750006+0000","last_unstale":"2024-03-06T07:26:18.806870+0000","last_undegraded":"2024-03-06T07:26:18.806870+0000","last_fullsized":"2024-03-06T07:26:18.806870+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:17.553214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760729+0000","last_change":"2024-03-06T07:26:16.740772+0000","last_active":"2024-03-06T07:26:17.760729+0000","last_peered":"2024-03-06T07:26:17.760729+0000","last_clean":"2024-03-06T07:26:17.760729+0000","last_became_active":"2024-03-06T07:26:16.740115+0000","last_became_peered":"2024-03-06T07:26:16.740115+0000","last_unstale":"2024-03-06T07:26:17.760729+0000","last_undegraded":"2024-03-06T07:26:17.760729+0000","last_fullsized":"2024-03-06T07:26:17.760729+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:33.807152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001780+0000","last_change":"2024-03-06T07:26:07.704775+0000","last_active":"2024-03-06T07:26:19.001780+0000","last_peered":"2024-03-06T07:26:19.001780+0000","last_clean":"2024-03-06T07:26:19.001780+0000","last_became_active":"2024-03-06T07:26:07.704502+0000","last_became_peered":"2024-03-06T07:26:07.704502+0000","last_unstale":"2024-03-06T07:26:19.001780+0000","last_undegraded":"2024-03-06T07:26:19.001780+0000","last_fullsized":"2024-03-06T07:26:19.001780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:04.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.740373+0000","last_change":"2024-03-06T07:26:06.696947+0000","last_active":"2024-03-06T07:26:17.740373+0000","last_peered":"2024-03-06T07:26:17.740373+0000","last_clean":"2024-03-06T07:26:17.740373+0000","last_became_active":"2024-03-06T07:26:06.696715+0000","last_became_peered":"2024-03-06T07:26:06.696715+0000","last_unstale":"2024-03-06T07:26:17.740373+0000","last_undegraded":"2024-03-06T07:26:17.740373+0000","last_fullsized":"2024-03-06T07:26:17.740373+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:40.929460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882613+0000","last_change":"2024-03-06T07:26:17.751662+0000","last_active":"2024-03-06T07:26:18.882613+0000","last_peered":"2024-03-06T07:26:18.882613+0000","last_clean":"2024-03-06T07:26:18.882613+0000","last_became_active":"2024-03-06T07:26:17.751263+0000","last_became_peered":"2024-03-06T07:26:17.751263+0000","last_unstale":"2024-03-06T07:26:18.882613+0000","last_undegraded":"2024-03-06T07:26:18.882613+0000","last_fullsized":"2024-03-06T07:26:18.882613+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:00.763534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.740547+0000","last_change":"2024-03-06T07:26:16.740573+0000","last_active":"2024-03-06T07:26:17.740547+0000","last_peered":"2024-03-06T07:26:17.740547+0000","last_clean":"2024-03-06T07:26:17.740547+0000","last_became_active":"2024-03-06T07:26:16.740049+0000","last_became_peered":"2024-03-06T07:26:16.740049+0000","last_unstale":"2024-03-06T07:26:17.740547+0000","last_undegraded":"2024-03-06T07:26:17.740547+0000","last_fullsized":"2024-03-06T07:26:17.740547+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:42.635159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745082+0000","last_change":"2024-03-06T07:26:07.699110+0000","last_active":"2024-03-06T07:26:18.745082+0000","last_peered":"2024-03-06T07:26:18.745082+0000","last_clean":"2024-03-06T07:26:18.745082+0000","last_became_active":"2024-03-06T07:26:07.698799+0000","last_became_peered":"2024-03-06T07:26:07.698799+0000","last_unstale":"2024-03-06T07:26:18.745082+0000","last_undegraded":"2024-03-06T07:26:18.745082+0000","last_fullsized":"2024-03-06T07:26:18.745082+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:56.000904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807190+0000","last_change":"2024-03-06T07:26:06.691133+0000","last_active":"2024-03-06T07:26:18.807190+0000","last_peered":"2024-03-06T07:26:18.807190+0000","last_clean":"2024-03-06T07:26:18.807190+0000","last_became_active":"2024-03-06T07:26:06.690687+0000","last_became_peered":"2024-03-06T07:26:06.690687+0000","last_unstale":"2024-03-06T07:26:18.807190+0000","last_undegraded":"2024-03-06T07:26:18.807190+0000","last_fullsized":"2024-03-06T07:26:18.807190+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:08.352394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":424,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745347+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:26:18.745347+0000","last_peered":"2024-03-06T07:26:18.745347+0000","last_clean":"2024-03-06T07:26:18.745347+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:26:18.745347+0000","last_undegraded":"2024-03-06T07:26:18.745347+0000","last_fullsized":"2024-03-06T07:26:18.745347+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2015232,"data_stored":1539268,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":948970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1961984,"data_stored":943109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1982464,"data_stored":951310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1978368,"data_stored":951276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":949482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1949696,"data_stored":942963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2027520,"data_stored":1532551,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:21.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.905+0000 7fcb40981640 1 Processor -- start 2024-03-06T07:26:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.905+0000 7fcb40981640 1 -- start start 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb40981640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb3c156ea0 0x7fcb3c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb40981640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0ad310 con 0x7fcb3c153990 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb40981640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c1572a0 con 0x7fcb3c1506b0 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb40981640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c156450 con 0x7fcb3c156ea0 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb3ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb3c156ea0 0x7fcb3c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb39d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcb3c153990 0x7fcb3c0acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60694/0 (socket says 172.21.15.26:60694) 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb39d9b640 1 -- 172.21.15.26:0/2387978221 learned_addr learned my addr 172.21.15.26:0/2387978221 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb3ad9d640 1 -- 172.21.15.26:0/2387978221 >> v1:172.21.15.117:6789/0 conn(0x7fcb3c1506b0 legacy=0x7fcb3c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb3ad9d640 1 -- 172.21.15.26:0/2387978221 >> v1:172.21.15.26:6789/0 conn(0x7fcb3c153990 legacy=0x7fcb3c0acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb3ad9d640 1 -- 172.21.15.26:0/2387978221 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c070aa0 con 0x7fcb3c156ea0 2024-03-06T07:26:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.906+0000 7fcb3959a640 1 -- 172.21.15.26:0/2387978221 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233852150 0 0) 0x7fcb3c0ad310 con 0x7fcb3c153990 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.907+0000 7fcb3ad9d640 1 --2- 172.21.15.26:0/2387978221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c156ea0 0x7fcb3c0c54a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009d90 tx=0x7fcb3402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ee2e45a4884d47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.907+0000 7fcb3959a640 1 -- 172.21.15.26:0/2387978221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34039070 con 0x7fcb3c156ea0 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.907+0000 7fcb3959a640 1 -- 172.21.15.26:0/2387978221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3402cd00 con 0x7fcb3c156ea0 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.907+0000 7fcb3959a640 1 -- 172.21.15.26:0/2387978221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34034660 con 0x7fcb3c156ea0 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.907+0000 7fcb40981640 1 -- 172.21.15.26:0/2387978221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c156ea0 msgr2=0x7fcb3c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.907+0000 7fcb40981640 1 --2- 172.21.15.26:0/2387978221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c156ea0 0x7fcb3c0c54a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009d90 tx=0x7fcb3402bf50 comp rx=0 tx=0).stop 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.908+0000 7fcb40981640 1 -- 172.21.15.26:0/2387978221 shutdown_connections 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.908+0000 7fcb40981640 1 --2- 172.21.15.26:0/2387978221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c156ea0 0x7fcb3c0c54a0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.908+0000 7fcb40981640 1 -- 172.21.15.26:0/2387978221 >> 172.21.15.26:0/2387978221 conn(0x7fcb3c06b1a0 msgr2=0x7fcb3c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.908+0000 7fcb40981640 1 -- 172.21.15.26:0/2387978221 shutdown_connections 2024-03-06T07:26:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.908+0000 7fcb40981640 1 -- 172.21.15.26:0/2387978221 wait complete. 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.909+0000 7fcb40981640 1 Processor -- start 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.909+0000 7fcb40981640 1 -- start start 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb3c1506b0 0x7fcb3c155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c153990 0x7fcb3c0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb3c156ea0 0x7fcb3c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c1566a0 con 0x7fcb3c153990 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c156a10 con 0x7fcb3c156ea0 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0c59e0 con 0x7fcb3c1506b0 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c153990 0x7fcb3c0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c153990 0x7fcb3c0c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58006/0 (socket says 172.21.15.26:58006) 2024-03-06T07:26:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 -- 172.21.15.26:0/3783369521 learned_addr learned my addr 172.21.15.26:0/3783369521 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb3a59c640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb3c1506b0 0x7fcb3c155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb3ad9d640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb3c156ea0 0x7fcb3c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 -- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb3c1506b0 msgr2=0x7fcb3c155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb3c1506b0 0x7fcb3c155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 -- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb3c156ea0 msgr2=0x7fcb3c1510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb3c156ea0 0x7fcb3c1510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 -- 172.21.15.26:0/3783369521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c151e50 con 0x7fcb3c153990 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb39d9b640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c153990 0x7fcb3c0c7a40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000c4d0 tx=0x7fcb3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb2affd640 1 -- 172.21.15.26:0/3783369521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30018070 con 0x7fcb3c153990 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb2affd640 1 -- 172.21.15.26:0/3783369521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb300074e0 con 0x7fcb3c153990 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c1541d0 con 0x7fcb3c153990 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb2affd640 1 -- 172.21.15.26:0/3783369521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30004a20 con 0x7fcb3c153990 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.910+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c154490 con 0x7fcb3c153990 2024-03-06T07:26:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.911+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c04f330 con 0x7fcb3c153990 2024-03-06T07:26:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.912+0000 7fcb2affd640 1 -- 172.21.15.26:0/3783369521 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb30004bc0 con 0x7fcb3c153990 2024-03-06T07:26:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.912+0000 7fcb2affd640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb1c06b4d0 0x7fcb1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.912+0000 7fcb2affd640 1 -- 172.21.15.26:0/3783369521 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fcb30089970 con 0x7fcb3c153990 2024-03-06T07:26:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.912+0000 7fcb3a59c640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb1c06b4d0 0x7fcb1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.913+0000 7fcb3a59c640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb1c06b4d0 0x7fcb1c06d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c000f60 tx=0x7fcb2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:21.921+0000 7fcb2affd640 1 -- 172.21.15.26:0/3783369521 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb30057cd0 con 0x7fcb3c153990 2024-03-06T07:26:22.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.099+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb3c0700a0 con 0x7fcb1c06b4d0 2024-03-06T07:26:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.110+0000 7fcb2affd640 1 -- 172.21.15.26:0/3783369521 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+491523 (secure 0 0 0) 0x7fcb3c0700a0 con 0x7fcb1c06b4d0 2024-03-06T07:26:22.118 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:22.122 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:26:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb1c06b4d0 msgr2=0x7fcb1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb1c06b4d0 0x7fcb1c06d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c000f60 tx=0x7fcb2c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c153990 msgr2=0x7fcb3c0c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c153990 0x7fcb3c0c7a40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000c4d0 tx=0x7fcb3000c990 comp rx=0 tx=0).stop 2024-03-06T07:26:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 shutdown_connections 2024-03-06T07:26:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb1c06b4d0 0x7fcb1c06d990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb3c156ea0 0x7fcb3c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c153990 0x7fcb3c0c7a40 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 --2- 172.21.15.26:0/3783369521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb3c1506b0 0x7fcb3c155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 >> 172.21.15.26:0/3783369521 conn(0x7fcb3c06b1a0 msgr2=0x7fcb3c152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 shutdown_connections 2024-03-06T07:26:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.114+0000 7fcb40981640 1 -- 172.21.15.26:0/3783369521 wait complete. 2024-03-06T07:26:22.128 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-03-06T07:26:21.109326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":316,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264924,"kb_used_data":15452,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709796,"statfs":{"total":773094113280,"available":772822831104,"internally_reserved":0,"allocated":15822848,"data_stored":8758929,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.391395"},"pg_stats":[{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882598+0000","last_change":"2024-03-06T07:26:17.751441+0000","last_active":"2024-03-06T07:26:18.882598+0000","last_peered":"2024-03-06T07:26:18.882598+0000","last_clean":"2024-03-06T07:26:18.882598+0000","last_became_active":"2024-03-06T07:26:17.751024+0000","last_became_peered":"2024-03-06T07:26:17.751024+0000","last_unstale":"2024-03-06T07:26:18.882598+0000","last_undegraded":"2024-03-06T07:26:18.882598+0000","last_fullsized":"2024-03-06T07:26:18.882598+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:30.905978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747351+0000","last_change":"2024-03-06T07:26:07.695841+0000","last_active":"2024-03-06T07:26:18.747351+0000","last_peered":"2024-03-06T07:26:18.747351+0000","last_clean":"2024-03-06T07:26:18.747351+0000","last_became_active":"2024-03-06T07:26:07.695553+0000","last_became_peered":"2024-03-06T07:26:07.695553+0000","last_unstale":"2024-03-06T07:26:18.747351+0000","last_undegraded":"2024-03-06T07:26:18.747351+0000","last_fullsized":"2024-03-06T07:26:18.747351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:17.847841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.758292+0000","last_change":"2024-03-06T07:26:17.752487+0000","last_active":"2024-03-06T07:26:17.758292+0000","last_peered":"2024-03-06T07:26:17.758292+0000","last_clean":"2024-03-06T07:26:17.758292+0000","last_became_active":"2024-03-06T07:26:17.752105+0000","last_became_peered":"2024-03-06T07:26:17.752105+0000","last_unstale":"2024-03-06T07:26:17.758292+0000","last_undegraded":"2024-03-06T07:26:17.758292+0000","last_fullsized":"2024-03-06T07:26:17.758292+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:26:27.566777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745195+0000","last_change":"2024-03-06T07:26:07.698101+0000","last_active":"2024-03-06T07:26:18.745195+0000","last_peered":"2024-03-06T07:26:18.745195+0000","last_clean":"2024-03-06T07:26:18.745195+0000","last_became_active":"2024-03-06T07:26:07.697796+0000","last_became_peered":"2024-03-06T07:26:07.697796+0000","last_unstale":"2024-03-06T07:26:18.745195+0000","last_undegraded":"2024-03-06T07:26:18.745195+0000","last_fullsized":"2024-03-06T07:26:18.745195+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:04.892999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749443+0000","last_change":"2024-03-06T07:26:17.739275+0000","last_active":"2024-03-06T07:26:19.749443+0000","last_peered":"2024-03-06T07:26:19.749443+0000","last_clean":"2024-03-06T07:26:19.749443+0000","last_became_active":"2024-03-06T07:26:17.739019+0000","last_became_peered":"2024-03-06T07:26:17.739019+0000","last_unstale":"2024-03-06T07:26:19.749443+0000","last_undegraded":"2024-03-06T07:26:19.749443+0000","last_fullsized":"2024-03-06T07:26:19.749443+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:10:47.968925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747261+0000","last_change":"2024-03-06T07:26:07.691124+0000","last_active":"2024-03-06T07:26:18.747261+0000","last_peered":"2024-03-06T07:26:18.747261+0000","last_clean":"2024-03-06T07:26:18.747261+0000","last_became_active":"2024-03-06T07:26:07.690832+0000","last_became_peered":"2024-03-06T07:26:07.690832+0000","last_unstale":"2024-03-06T07:26:18.747261+0000","last_undegraded":"2024-03-06T07:26:18.747261+0000","last_fullsized":"2024-03-06T07:26:18.747261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:29:40.327108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749443+0000","last_change":"2024-03-06T07:26:17.751375+0000","last_active":"2024-03-06T07:26:19.749443+0000","last_peered":"2024-03-06T07:26:19.749443+0000","last_clean":"2024-03-06T07:26:19.749443+0000","last_became_active":"2024-03-06T07:26:17.751123+0000","last_became_peered":"2024-03-06T07:26:17.751123+0000","last_unstale":"2024-03-06T07:26:19.749443+0000","last_undegraded":"2024-03-06T07:26:19.749443+0000","last_fullsized":"2024-03-06T07:26:19.749443+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:43.640582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747209+0000","last_change":"2024-03-06T07:26:07.688926+0000","last_active":"2024-03-06T07:26:18.747209+0000","last_peered":"2024-03-06T07:26:18.747209+0000","last_clean":"2024-03-06T07:26:18.747209+0000","last_became_active":"2024-03-06T07:26:07.688684+0000","last_became_peered":"2024-03-06T07:26:07.688684+0000","last_unstale":"2024-03-06T07:26:18.747209+0000","last_undegraded":"2024-03-06T07:26:18.747209+0000","last_fullsized":"2024-03-06T07:26:18.747209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:31.513519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001539+0000","last_change":"2024-03-06T07:26:17.749303+0000","last_active":"2024-03-06T07:26:19.001539+0000","last_peered":"2024-03-06T07:26:19.001539+0000","last_clean":"2024-03-06T07:26:19.001539+0000","last_became_active":"2024-03-06T07:26:17.749000+0000","last_became_peered":"2024-03-06T07:26:17.749000+0000","last_unstale":"2024-03-06T07:26:19.001539+0000","last_undegraded":"2024-03-06T07:26:19.001539+0000","last_fullsized":"2024-03-06T07:26:19.001539+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:31:51.703291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738387+0000","last_change":"2024-03-06T07:26:07.708495+0000","last_active":"2024-03-06T07:26:17.738387+0000","last_peered":"2024-03-06T07:26:17.738387+0000","last_clean":"2024-03-06T07:26:17.738387+0000","last_became_active":"2024-03-06T07:26:07.708183+0000","last_became_peered":"2024-03-06T07:26:07.708183+0000","last_unstale":"2024-03-06T07:26:17.738387+0000","last_undegraded":"2024-03-06T07:26:17.738387+0000","last_fullsized":"2024-03-06T07:26:17.738387+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:00:41.851158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744919+0000","last_change":"2024-03-06T07:26:17.754154+0000","last_active":"2024-03-06T07:26:18.744919+0000","last_peered":"2024-03-06T07:26:18.744919+0000","last_clean":"2024-03-06T07:26:18.744919+0000","last_became_active":"2024-03-06T07:26:17.753797+0000","last_became_peered":"2024-03-06T07:26:17.753797+0000","last_unstale":"2024-03-06T07:26:18.744919+0000","last_undegraded":"2024-03-06T07:26:18.744919+0000","last_fullsized":"2024-03-06T07:26:18.744919+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:50:45.483679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744637+0000","last_change":"2024-03-06T07:26:07.696015+0000","last_active":"2024-03-06T07:26:18.744637+0000","last_peered":"2024-03-06T07:26:18.744637+0000","last_clean":"2024-03-06T07:26:18.744637+0000","last_became_active":"2024-03-06T07:26:07.695708+0000","last_became_peered":"2024-03-06T07:26:07.695708+0000","last_unstale":"2024-03-06T07:26:18.744637+0000","last_undegraded":"2024-03-06T07:26:18.744637+0000","last_fullsized":"2024-03-06T07:26:18.744637+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:15.500034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806911+0000","last_change":"2024-03-06T07:26:17.746370+0000","last_active":"2024-03-06T07:26:18.806911+0000","last_peered":"2024-03-06T07:26:18.806911+0000","last_clean":"2024-03-06T07:26:18.806911+0000","last_became_active":"2024-03-06T07:26:17.744994+0000","last_became_peered":"2024-03-06T07:26:17.744994+0000","last_unstale":"2024-03-06T07:26:18.806911+0000","last_undegraded":"2024-03-06T07:26:18.806911+0000","last_fullsized":"2024-03-06T07:26:18.806911+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:24.779563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749074+0000","last_change":"2024-03-06T07:26:07.699338+0000","last_active":"2024-03-06T07:26:19.749074+0000","last_peered":"2024-03-06T07:26:19.749074+0000","last_clean":"2024-03-06T07:26:19.749074+0000","last_became_active":"2024-03-06T07:26:07.699065+0000","last_became_peered":"2024-03-06T07:26:07.699065+0000","last_unstale":"2024-03-06T07:26:19.749074+0000","last_undegraded":"2024-03-06T07:26:19.749074+0000","last_fullsized":"2024-03-06T07:26:19.749074+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806423+0000","last_change":"2024-03-06T07:26:17.747306+0000","last_active":"2024-03-06T07:26:18.806423+0000","last_peered":"2024-03-06T07:26:18.806423+0000","last_clean":"2024-03-06T07:26:18.806423+0000","last_became_active":"2024-03-06T07:26:17.746888+0000","last_became_peered":"2024-03-06T07:26:17.746888+0000","last_unstale":"2024-03-06T07:26:18.806423+0000","last_undegraded":"2024-03-06T07:26:18.806423+0000","last_fullsized":"2024-03-06T07:26:18.806423+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:27:28.288152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748699+0000","last_change":"2024-03-06T07:26:07.700571+0000","last_active":"2024-03-06T07:26:19.748699+0000","last_peered":"2024-03-06T07:26:19.748699+0000","last_clean":"2024-03-06T07:26:19.748699+0000","last_became_active":"2024-03-06T07:26:07.698195+0000","last_became_peered":"2024-03-06T07:26:07.698195+0000","last_unstale":"2024-03-06T07:26:19.748699+0000","last_undegraded":"2024-03-06T07:26:19.748699+0000","last_fullsized":"2024-03-06T07:26:19.748699+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.762269+0000","last_change":"2024-03-06T07:26:17.757260+0000","last_active":"2024-03-06T07:26:17.762269+0000","last_peered":"2024-03-06T07:26:17.762269+0000","last_clean":"2024-03-06T07:26:17.762269+0000","last_became_active":"2024-03-06T07:26:17.756964+0000","last_became_peered":"2024-03-06T07:26:17.756964+0000","last_unstale":"2024-03-06T07:26:17.762269+0000","last_undegraded":"2024-03-06T07:26:17.762269+0000","last_fullsized":"2024-03-06T07:26:17.762269+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:52.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749521+0000","last_change":"2024-03-06T07:26:07.700691+0000","last_active":"2024-03-06T07:26:19.749521+0000","last_peered":"2024-03-06T07:26:19.749521+0000","last_clean":"2024-03-06T07:26:19.749521+0000","last_became_active":"2024-03-06T07:26:07.700158+0000","last_became_peered":"2024-03-06T07:26:07.700158+0000","last_unstale":"2024-03-06T07:26:19.749521+0000","last_undegraded":"2024-03-06T07:26:19.749521+0000","last_fullsized":"2024-03-06T07:26:19.749521+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:17.093387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.758270+0000","last_change":"2024-03-06T07:26:17.752361+0000","last_active":"2024-03-06T07:26:17.758270+0000","last_peered":"2024-03-06T07:26:17.758270+0000","last_clean":"2024-03-06T07:26:17.758270+0000","last_became_active":"2024-03-06T07:26:17.751787+0000","last_became_peered":"2024-03-06T07:26:17.751787+0000","last_unstale":"2024-03-06T07:26:17.758270+0000","last_undegraded":"2024-03-06T07:26:17.758270+0000","last_fullsized":"2024-03-06T07:26:17.758270+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:04.831471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749559+0000","last_change":"2024-03-06T07:26:07.700442+0000","last_active":"2024-03-06T07:26:19.749559+0000","last_peered":"2024-03-06T07:26:19.749559+0000","last_clean":"2024-03-06T07:26:19.749559+0000","last_became_active":"2024-03-06T07:26:07.700116+0000","last_became_peered":"2024-03-06T07:26:07.700116+0000","last_unstale":"2024-03-06T07:26:19.749559+0000","last_undegraded":"2024-03-06T07:26:19.749559+0000","last_fullsized":"2024-03-06T07:26:19.749559+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:30.446061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749470+0000","last_change":"2024-03-06T07:26:17.753333+0000","last_active":"2024-03-06T07:26:19.749470+0000","last_peered":"2024-03-06T07:26:19.749470+0000","last_clean":"2024-03-06T07:26:19.749470+0000","last_became_active":"2024-03-06T07:26:17.753044+0000","last_became_peered":"2024-03-06T07:26:17.753044+0000","last_unstale":"2024-03-06T07:26:19.749470+0000","last_undegraded":"2024-03-06T07:26:19.749470+0000","last_fullsized":"2024-03-06T07:26:19.749470+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:02.145666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.745875+0000","last_change":"2024-03-06T07:26:07.707356+0000","last_active":"2024-03-06T07:26:17.745875+0000","last_peered":"2024-03-06T07:26:17.745875+0000","last_clean":"2024-03-06T07:26:17.745875+0000","last_became_active":"2024-03-06T07:26:07.707082+0000","last_became_peered":"2024-03-06T07:26:07.707082+0000","last_unstale":"2024-03-06T07:26:17.745875+0000","last_undegraded":"2024-03-06T07:26:17.745875+0000","last_fullsized":"2024-03-06T07:26:17.745875+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806879+0000","last_change":"2024-03-06T07:26:17.750773+0000","last_active":"2024-03-06T07:26:18.806879+0000","last_peered":"2024-03-06T07:26:18.806879+0000","last_clean":"2024-03-06T07:26:18.806879+0000","last_became_active":"2024-03-06T07:26:17.750338+0000","last_became_peered":"2024-03-06T07:26:17.750338+0000","last_unstale":"2024-03-06T07:26:18.806879+0000","last_undegraded":"2024-03-06T07:26:18.806879+0000","last_fullsized":"2024-03-06T07:26:18.806879+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:15.433217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745321+0000","last_change":"2024-03-06T07:26:07.694337+0000","last_active":"2024-03-06T07:26:18.745321+0000","last_peered":"2024-03-06T07:26:18.745321+0000","last_clean":"2024-03-06T07:26:18.745321+0000","last_became_active":"2024-03-06T07:26:07.694050+0000","last_became_peered":"2024-03-06T07:26:07.694050+0000","last_unstale":"2024-03-06T07:26:18.745321+0000","last_undegraded":"2024-03-06T07:26:18.745321+0000","last_fullsized":"2024-03-06T07:26:18.745321+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:46.775059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749128+0000","last_change":"2024-03-06T07:26:17.748683+0000","last_active":"2024-03-06T07:26:19.749128+0000","last_peered":"2024-03-06T07:26:19.749128+0000","last_clean":"2024-03-06T07:26:19.749128+0000","last_became_active":"2024-03-06T07:26:17.748402+0000","last_became_peered":"2024-03-06T07:26:17.748402+0000","last_unstale":"2024-03-06T07:26:19.749128+0000","last_undegraded":"2024-03-06T07:26:19.749128+0000","last_fullsized":"2024-03-06T07:26:19.749128+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:59:42.208048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806908+0000","last_change":"2024-03-06T07:26:07.696706+0000","last_active":"2024-03-06T07:26:18.806908+0000","last_peered":"2024-03-06T07:26:18.806908+0000","last_clean":"2024-03-06T07:26:18.806908+0000","last_became_active":"2024-03-06T07:26:07.696438+0000","last_became_peered":"2024-03-06T07:26:07.696438+0000","last_unstale":"2024-03-06T07:26:18.806908+0000","last_undegraded":"2024-03-06T07:26:18.806908+0000","last_fullsized":"2024-03-06T07:26:18.806908+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:32:23.418950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806414+0000","last_change":"2024-03-06T07:26:17.747667+0000","last_active":"2024-03-06T07:26:18.806414+0000","last_peered":"2024-03-06T07:26:18.806414+0000","last_clean":"2024-03-06T07:26:18.806414+0000","last_became_active":"2024-03-06T07:26:17.747376+0000","last_became_peered":"2024-03-06T07:26:17.747376+0000","last_unstale":"2024-03-06T07:26:18.806414+0000","last_undegraded":"2024-03-06T07:26:18.806414+0000","last_fullsized":"2024-03-06T07:26:18.806414+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:48.192361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743785+0000","last_change":"2024-03-06T07:26:07.704226+0000","last_active":"2024-03-06T07:26:17.743785+0000","last_peered":"2024-03-06T07:26:17.743785+0000","last_clean":"2024-03-06T07:26:17.743785+0000","last_became_active":"2024-03-06T07:26:07.703839+0000","last_became_peered":"2024-03-06T07:26:07.703839+0000","last_unstale":"2024-03-06T07:26:17.743785+0000","last_undegraded":"2024-03-06T07:26:17.743785+0000","last_fullsized":"2024-03-06T07:26:17.743785+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:52.356683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746419+0000","last_change":"2024-03-06T07:26:17.752169+0000","last_active":"2024-03-06T07:26:18.746419+0000","last_peered":"2024-03-06T07:26:18.746419+0000","last_clean":"2024-03-06T07:26:18.746419+0000","last_became_active":"2024-03-06T07:26:17.751883+0000","last_became_peered":"2024-03-06T07:26:17.751883+0000","last_unstale":"2024-03-06T07:26:18.746419+0000","last_undegraded":"2024-03-06T07:26:18.746419+0000","last_fullsized":"2024-03-06T07:26:18.746419+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:38.529399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744672+0000","last_change":"2024-03-06T07:26:07.710172+0000","last_active":"2024-03-06T07:26:17.744672+0000","last_peered":"2024-03-06T07:26:17.744672+0000","last_clean":"2024-03-06T07:26:17.744672+0000","last_became_active":"2024-03-06T07:26:07.709871+0000","last_became_peered":"2024-03-06T07:26:07.709871+0000","last_unstale":"2024-03-06T07:26:17.744672+0000","last_undegraded":"2024-03-06T07:26:17.744672+0000","last_fullsized":"2024-03-06T07:26:17.744672+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:17:00.303994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001140+0000","last_change":"2024-03-06T07:26:17.750774+0000","last_active":"2024-03-06T07:26:19.001140+0000","last_peered":"2024-03-06T07:26:19.001140+0000","last_clean":"2024-03-06T07:26:19.001140+0000","last_became_active":"2024-03-06T07:26:17.750510+0000","last_became_peered":"2024-03-06T07:26:17.750510+0000","last_unstale":"2024-03-06T07:26:19.001140+0000","last_undegraded":"2024-03-06T07:26:19.001140+0000","last_fullsized":"2024-03-06T07:26:19.001140+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:16:40.780478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746197+0000","last_change":"2024-03-06T07:26:07.688786+0000","last_active":"2024-03-06T07:26:18.746197+0000","last_peered":"2024-03-06T07:26:18.746197+0000","last_clean":"2024-03-06T07:26:18.746197+0000","last_became_active":"2024-03-06T07:26:07.688085+0000","last_became_peered":"2024-03-06T07:26:07.688085+0000","last_unstale":"2024-03-06T07:26:18.746197+0000","last_undegraded":"2024-03-06T07:26:18.746197+0000","last_fullsized":"2024-03-06T07:26:18.746197+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:55.281049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747369+0000","last_change":"2024-03-06T07:26:17.745834+0000","last_active":"2024-03-06T07:26:18.747369+0000","last_peered":"2024-03-06T07:26:18.747369+0000","last_clean":"2024-03-06T07:26:18.747369+0000","last_became_active":"2024-03-06T07:26:17.745566+0000","last_became_peered":"2024-03-06T07:26:17.745566+0000","last_unstale":"2024-03-06T07:26:18.747369+0000","last_undegraded":"2024-03-06T07:26:18.747369+0000","last_fullsized":"2024-03-06T07:26:18.747369+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:27.138420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749610+0000","last_change":"2024-03-06T07:26:07.711076+0000","last_active":"2024-03-06T07:26:19.749610+0000","last_peered":"2024-03-06T07:26:19.749610+0000","last_clean":"2024-03-06T07:26:19.749610+0000","last_became_active":"2024-03-06T07:26:07.710796+0000","last_became_peered":"2024-03-06T07:26:07.710796+0000","last_unstale":"2024-03-06T07:26:19.749610+0000","last_undegraded":"2024-03-06T07:26:19.749610+0000","last_fullsized":"2024-03-06T07:26:19.749610+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:56.000904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807100+0000","last_change":"2024-03-06T07:26:17.759373+0000","last_active":"2024-03-06T07:26:18.807100+0000","last_peered":"2024-03-06T07:26:18.807100+0000","last_clean":"2024-03-06T07:26:18.807100+0000","last_became_active":"2024-03-06T07:26:17.759122+0000","last_became_peered":"2024-03-06T07:26:17.759122+0000","last_unstale":"2024-03-06T07:26:18.807100+0000","last_undegraded":"2024-03-06T07:26:18.807100+0000","last_fullsized":"2024-03-06T07:26:18.807100+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:48:28.061942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882497+0000","last_change":"2024-03-06T07:26:07.703707+0000","last_active":"2024-03-06T07:26:18.882497+0000","last_peered":"2024-03-06T07:26:18.882497+0000","last_clean":"2024-03-06T07:26:18.882497+0000","last_became_active":"2024-03-06T07:26:07.703402+0000","last_became_peered":"2024-03-06T07:26:07.703402+0000","last_unstale":"2024-03-06T07:26:18.882497+0000","last_undegraded":"2024-03-06T07:26:18.882497+0000","last_fullsized":"2024-03-06T07:26:18.882497+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:45.409316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807429+0000","last_change":"2024-03-06T07:26:17.757430+0000","last_active":"2024-03-06T07:26:18.807429+0000","last_peered":"2024-03-06T07:26:18.807429+0000","last_clean":"2024-03-06T07:26:18.807429+0000","last_became_active":"2024-03-06T07:26:17.757156+0000","last_became_peered":"2024-03-06T07:26:17.757156+0000","last_unstale":"2024-03-06T07:26:18.807429+0000","last_undegraded":"2024-03-06T07:26:18.807429+0000","last_fullsized":"2024-03-06T07:26:18.807429+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:31.806156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.746012+0000","last_change":"2024-03-06T07:26:07.710174+0000","last_active":"2024-03-06T07:26:17.746012+0000","last_peered":"2024-03-06T07:26:17.746012+0000","last_clean":"2024-03-06T07:26:17.746012+0000","last_became_active":"2024-03-06T07:26:07.709939+0000","last_became_peered":"2024-03-06T07:26:07.709939+0000","last_unstale":"2024-03-06T07:26:17.746012+0000","last_undegraded":"2024-03-06T07:26:17.746012+0000","last_fullsized":"2024-03-06T07:26:17.746012+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:25.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747283+0000","last_change":"2024-03-06T07:26:17.740913+0000","last_active":"2024-03-06T07:26:18.747283+0000","last_peered":"2024-03-06T07:26:18.747283+0000","last_clean":"2024-03-06T07:26:18.747283+0000","last_became_active":"2024-03-06T07:26:17.740648+0000","last_became_peered":"2024-03-06T07:26:17.740648+0000","last_unstale":"2024-03-06T07:26:18.747283+0000","last_undegraded":"2024-03-06T07:26:18.747283+0000","last_fullsized":"2024-03-06T07:26:18.747283+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:39.385283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001457+0000","last_change":"2024-03-06T07:26:07.704730+0000","last_active":"2024-03-06T07:26:19.001457+0000","last_peered":"2024-03-06T07:26:19.001457+0000","last_clean":"2024-03-06T07:26:19.001457+0000","last_became_active":"2024-03-06T07:26:07.704429+0000","last_became_peered":"2024-03-06T07:26:07.704429+0000","last_unstale":"2024-03-06T07:26:19.001457+0000","last_undegraded":"2024-03-06T07:26:19.001457+0000","last_fullsized":"2024-03-06T07:26:19.001457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:14.776787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745095+0000","last_change":"2024-03-06T07:26:17.746811+0000","last_active":"2024-03-06T07:26:18.745095+0000","last_peered":"2024-03-06T07:26:18.745095+0000","last_clean":"2024-03-06T07:26:18.745095+0000","last_became_active":"2024-03-06T07:26:17.746510+0000","last_became_peered":"2024-03-06T07:26:17.746510+0000","last_unstale":"2024-03-06T07:26:18.745095+0000","last_undegraded":"2024-03-06T07:26:18.745095+0000","last_fullsized":"2024-03-06T07:26:18.745095+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:05.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806960+0000","last_change":"2024-03-06T07:26:07.685590+0000","last_active":"2024-03-06T07:26:18.806960+0000","last_peered":"2024-03-06T07:26:18.806960+0000","last_clean":"2024-03-06T07:26:18.806960+0000","last_became_active":"2024-03-06T07:26:07.685265+0000","last_became_peered":"2024-03-06T07:26:07.685265+0000","last_unstale":"2024-03-06T07:26:18.806960+0000","last_undegraded":"2024-03-06T07:26:18.806960+0000","last_fullsized":"2024-03-06T07:26:18.806960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:30.427263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748892+0000","last_change":"2024-03-06T07:26:17.740092+0000","last_active":"2024-03-06T07:26:19.748892+0000","last_peered":"2024-03-06T07:26:19.748892+0000","last_clean":"2024-03-06T07:26:19.748892+0000","last_became_active":"2024-03-06T07:26:17.739842+0000","last_became_peered":"2024-03-06T07:26:17.739842+0000","last_unstale":"2024-03-06T07:26:19.748892+0000","last_undegraded":"2024-03-06T07:26:19.748892+0000","last_fullsized":"2024-03-06T07:26:19.748892+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:19.539826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744661+0000","last_change":"2024-03-06T07:26:07.699358+0000","last_active":"2024-03-06T07:26:18.744661+0000","last_peered":"2024-03-06T07:26:18.744661+0000","last_clean":"2024-03-06T07:26:18.744661+0000","last_became_active":"2024-03-06T07:26:07.699058+0000","last_became_peered":"2024-03-06T07:26:07.699058+0000","last_unstale":"2024-03-06T07:26:18.744661+0000","last_undegraded":"2024-03-06T07:26:18.744661+0000","last_fullsized":"2024-03-06T07:26:18.744661+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:02.835571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749232+0000","last_change":"2024-03-06T07:26:17.752029+0000","last_active":"2024-03-06T07:26:19.749232+0000","last_peered":"2024-03-06T07:26:19.749232+0000","last_clean":"2024-03-06T07:26:19.749232+0000","last_became_active":"2024-03-06T07:26:17.751766+0000","last_became_peered":"2024-03-06T07:26:17.751766+0000","last_unstale":"2024-03-06T07:26:19.749232+0000","last_undegraded":"2024-03-06T07:26:19.749232+0000","last_fullsized":"2024-03-06T07:26:19.749232+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:34.812576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.739016+0000","last_change":"2024-03-06T07:26:07.700848+0000","last_active":"2024-03-06T07:26:17.739016+0000","last_peered":"2024-03-06T07:26:17.739016+0000","last_clean":"2024-03-06T07:26:17.739016+0000","last_became_active":"2024-03-06T07:26:07.700551+0000","last_became_peered":"2024-03-06T07:26:07.700551+0000","last_unstale":"2024-03-06T07:26:17.739016+0000","last_undegraded":"2024-03-06T07:26:17.739016+0000","last_fullsized":"2024-03-06T07:26:17.739016+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:42.178917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748998+0000","last_change":"2024-03-06T07:26:17.754908+0000","last_active":"2024-03-06T07:26:19.748998+0000","last_peered":"2024-03-06T07:26:19.748998+0000","last_clean":"2024-03-06T07:26:19.748998+0000","last_became_active":"2024-03-06T07:26:17.754430+0000","last_became_peered":"2024-03-06T07:26:17.754430+0000","last_unstale":"2024-03-06T07:26:19.748998+0000","last_undegraded":"2024-03-06T07:26:19.748998+0000","last_fullsized":"2024-03-06T07:26:19.748998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:50:45.483679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746061+0000","last_change":"2024-03-06T07:26:07.687552+0000","last_active":"2024-03-06T07:26:18.746061+0000","last_peered":"2024-03-06T07:26:18.746061+0000","last_clean":"2024-03-06T07:26:18.746061+0000","last_became_active":"2024-03-06T07:26:07.687276+0000","last_became_peered":"2024-03-06T07:26:07.687276+0000","last_unstale":"2024-03-06T07:26:18.746061+0000","last_undegraded":"2024-03-06T07:26:18.746061+0000","last_fullsized":"2024-03-06T07:26:18.746061+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:03.163515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745034+0000","last_change":"2024-03-06T07:26:17.740835+0000","last_active":"2024-03-06T07:26:18.745034+0000","last_peered":"2024-03-06T07:26:18.745034+0000","last_clean":"2024-03-06T07:26:18.745034+0000","last_became_active":"2024-03-06T07:26:17.740544+0000","last_became_peered":"2024-03-06T07:26:17.740544+0000","last_unstale":"2024-03-06T07:26:18.745034+0000","last_undegraded":"2024-03-06T07:26:18.745034+0000","last_fullsized":"2024-03-06T07:26:18.745034+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:59:42.208048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807551+0000","last_change":"2024-03-06T07:26:07.698089+0000","last_active":"2024-03-06T07:26:18.807551+0000","last_peered":"2024-03-06T07:26:18.807551+0000","last_clean":"2024-03-06T07:26:18.807551+0000","last_became_active":"2024-03-06T07:26:07.697806+0000","last_became_peered":"2024-03-06T07:26:07.697806+0000","last_unstale":"2024-03-06T07:26:18.807551+0000","last_undegraded":"2024-03-06T07:26:18.807551+0000","last_fullsized":"2024-03-06T07:26:18.807551+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:52.239620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806635+0000","last_change":"2024-03-06T07:26:17.756836+0000","last_active":"2024-03-06T07:26:18.806635+0000","last_peered":"2024-03-06T07:26:18.806635+0000","last_clean":"2024-03-06T07:26:18.806635+0000","last_became_active":"2024-03-06T07:26:17.756565+0000","last_became_peered":"2024-03-06T07:26:17.756565+0000","last_unstale":"2024-03-06T07:26:18.806635+0000","last_undegraded":"2024-03-06T07:26:18.806635+0000","last_fullsized":"2024-03-06T07:26:18.806635+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:44.181354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001511+0000","last_change":"2024-03-06T07:26:07.699922+0000","last_active":"2024-03-06T07:26:19.001511+0000","last_peered":"2024-03-06T07:26:19.001511+0000","last_clean":"2024-03-06T07:26:19.001511+0000","last_became_active":"2024-03-06T07:26:07.699631+0000","last_became_peered":"2024-03-06T07:26:07.699631+0000","last_unstale":"2024-03-06T07:26:19.001511+0000","last_undegraded":"2024-03-06T07:26:19.001511+0000","last_fullsized":"2024-03-06T07:26:19.001511+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:39.110553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001570+0000","last_change":"2024-03-06T07:26:17.748859+0000","last_active":"2024-03-06T07:26:19.001570+0000","last_peered":"2024-03-06T07:26:19.001570+0000","last_clean":"2024-03-06T07:26:19.001570+0000","last_became_active":"2024-03-06T07:26:17.748556+0000","last_became_peered":"2024-03-06T07:26:17.748556+0000","last_unstale":"2024-03-06T07:26:19.001570+0000","last_undegraded":"2024-03-06T07:26:19.001570+0000","last_fullsized":"2024-03-06T07:26:19.001570+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:29:50.372142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745228+0000","last_change":"2024-03-06T07:26:07.706359+0000","last_active":"2024-03-06T07:26:18.745228+0000","last_peered":"2024-03-06T07:26:18.745228+0000","last_clean":"2024-03-06T07:26:18.745228+0000","last_became_active":"2024-03-06T07:26:07.706096+0000","last_became_peered":"2024-03-06T07:26:07.706096+0000","last_unstale":"2024-03-06T07:26:18.745228+0000","last_undegraded":"2024-03-06T07:26:18.745228+0000","last_fullsized":"2024-03-06T07:26:18.745228+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:08.310190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.752823+0000","last_change":"2024-03-06T07:26:17.750402+0000","last_active":"2024-03-06T07:26:17.752823+0000","last_peered":"2024-03-06T07:26:17.752823+0000","last_clean":"2024-03-06T07:26:17.752823+0000","last_became_active":"2024-03-06T07:26:17.749658+0000","last_became_peered":"2024-03-06T07:26:17.749658+0000","last_unstale":"2024-03-06T07:26:17.752823+0000","last_undegraded":"2024-03-06T07:26:17.752823+0000","last_fullsized":"2024-03-06T07:26:17.752823+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:16.312646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806734+0000","last_change":"2024-03-06T07:26:07.693096+0000","last_active":"2024-03-06T07:26:18.806734+0000","last_peered":"2024-03-06T07:26:18.806734+0000","last_clean":"2024-03-06T07:26:18.806734+0000","last_became_active":"2024-03-06T07:26:07.692834+0000","last_became_peered":"2024-03-06T07:26:07.692834+0000","last_unstale":"2024-03-06T07:26:18.806734+0000","last_undegraded":"2024-03-06T07:26:18.806734+0000","last_fullsized":"2024-03-06T07:26:18.806734+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:58.002384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001192+0000","last_change":"2024-03-06T07:26:17.746481+0000","last_active":"2024-03-06T07:26:19.001192+0000","last_peered":"2024-03-06T07:26:19.001192+0000","last_clean":"2024-03-06T07:26:19.001192+0000","last_became_active":"2024-03-06T07:26:17.746226+0000","last_became_peered":"2024-03-06T07:26:17.746226+0000","last_unstale":"2024-03-06T07:26:19.001192+0000","last_undegraded":"2024-03-06T07:26:19.001192+0000","last_fullsized":"2024-03-06T07:26:19.001192+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:06.076055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744906+0000","last_change":"2024-03-06T07:26:07.706042+0000","last_active":"2024-03-06T07:26:18.744906+0000","last_peered":"2024-03-06T07:26:18.744906+0000","last_clean":"2024-03-06T07:26:18.744906+0000","last_became_active":"2024-03-06T07:26:07.705455+0000","last_became_peered":"2024-03-06T07:26:07.705455+0000","last_unstale":"2024-03-06T07:26:18.744906+0000","last_undegraded":"2024-03-06T07:26:18.744906+0000","last_fullsized":"2024-03-06T07:26:18.744906+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:48.048889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806419+0000","last_change":"2024-03-06T07:26:17.755210+0000","last_active":"2024-03-06T07:26:18.806419+0000","last_peered":"2024-03-06T07:26:18.806419+0000","last_clean":"2024-03-06T07:26:18.806419+0000","last_became_active":"2024-03-06T07:26:17.754985+0000","last_became_peered":"2024-03-06T07:26:17.754985+0000","last_unstale":"2024-03-06T07:26:18.806419+0000","last_undegraded":"2024-03-06T07:26:18.806419+0000","last_fullsized":"2024-03-06T07:26:18.806419+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:39.760526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882242+0000","last_change":"2024-03-06T07:26:07.706790+0000","last_active":"2024-03-06T07:26:18.882242+0000","last_peered":"2024-03-06T07:26:18.882242+0000","last_clean":"2024-03-06T07:26:18.882242+0000","last_became_active":"2024-03-06T07:26:07.706476+0000","last_became_peered":"2024-03-06T07:26:07.706476+0000","last_unstale":"2024-03-06T07:26:18.882242+0000","last_undegraded":"2024-03-06T07:26:18.882242+0000","last_fullsized":"2024-03-06T07:26:18.882242+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:57.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.762180+0000","last_change":"2024-03-06T07:26:17.755465+0000","last_active":"2024-03-06T07:26:17.762180+0000","last_peered":"2024-03-06T07:26:17.762180+0000","last_clean":"2024-03-06T07:26:17.762180+0000","last_became_active":"2024-03-06T07:26:17.755199+0000","last_became_peered":"2024-03-06T07:26:17.755199+0000","last_unstale":"2024-03-06T07:26:17.762180+0000","last_undegraded":"2024-03-06T07:26:17.762180+0000","last_fullsized":"2024-03-06T07:26:17.762180+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:45.169002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744755+0000","last_change":"2024-03-06T07:26:07.698630+0000","last_active":"2024-03-06T07:26:17.744755+0000","last_peered":"2024-03-06T07:26:17.744755+0000","last_clean":"2024-03-06T07:26:17.744755+0000","last_became_active":"2024-03-06T07:26:07.698334+0000","last_became_peered":"2024-03-06T07:26:07.698334+0000","last_unstale":"2024-03-06T07:26:17.744755+0000","last_undegraded":"2024-03-06T07:26:17.744755+0000","last_fullsized":"2024-03-06T07:26:17.744755+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:50:39.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760234+0000","last_change":"2024-03-06T07:26:17.752948+0000","last_active":"2024-03-06T07:26:17.760234+0000","last_peered":"2024-03-06T07:26:17.760234+0000","last_clean":"2024-03-06T07:26:17.760234+0000","last_became_active":"2024-03-06T07:26:17.752660+0000","last_became_peered":"2024-03-06T07:26:17.752660+0000","last_unstale":"2024-03-06T07:26:17.760234+0000","last_undegraded":"2024-03-06T07:26:17.760234+0000","last_fullsized":"2024-03-06T07:26:17.760234+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:38.035013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882321+0000","last_change":"2024-03-06T07:26:07.699219+0000","last_active":"2024-03-06T07:26:18.882321+0000","last_peered":"2024-03-06T07:26:18.882321+0000","last_clean":"2024-03-06T07:26:18.882321+0000","last_became_active":"2024-03-06T07:26:07.698897+0000","last_became_peered":"2024-03-06T07:26:07.698897+0000","last_unstale":"2024-03-06T07:26:18.882321+0000","last_undegraded":"2024-03-06T07:26:18.882321+0000","last_fullsized":"2024-03-06T07:26:18.882321+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:56.031022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746317+0000","last_change":"2024-03-06T07:26:17.749192+0000","last_active":"2024-03-06T07:26:18.746317+0000","last_peered":"2024-03-06T07:26:18.746317+0000","last_clean":"2024-03-06T07:26:18.746317+0000","last_became_active":"2024-03-06T07:26:17.748932+0000","last_became_peered":"2024-03-06T07:26:17.748932+0000","last_unstale":"2024-03-06T07:26:18.746317+0000","last_undegraded":"2024-03-06T07:26:18.746317+0000","last_fullsized":"2024-03-06T07:26:18.746317+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:35.899348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738947+0000","last_change":"2024-03-06T07:26:16.740205+0000","last_active":"2024-03-06T07:26:17.738947+0000","last_peered":"2024-03-06T07:26:17.738947+0000","last_clean":"2024-03-06T07:26:17.738947+0000","last_became_active":"2024-03-06T07:26:16.739946+0000","last_became_peered":"2024-03-06T07:26:16.739946+0000","last_unstale":"2024-03-06T07:26:17.738947+0000","last_undegraded":"2024-03-06T07:26:17.738947+0000","last_fullsized":"2024-03-06T07:26:17.738947+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:59:41.202625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744621+0000","last_change":"2024-03-06T07:26:07.697611+0000","last_active":"2024-03-06T07:26:18.744621+0000","last_peered":"2024-03-06T07:26:18.744621+0000","last_clean":"2024-03-06T07:26:18.744621+0000","last_became_active":"2024-03-06T07:26:07.697343+0000","last_became_peered":"2024-03-06T07:26:07.697343+0000","last_unstale":"2024-03-06T07:26:18.744621+0000","last_undegraded":"2024-03-06T07:26:18.744621+0000","last_fullsized":"2024-03-06T07:26:18.744621+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:41.934898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"150'4","reported_seq":26,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747506+0000","last_change":"2024-03-06T07:26:06.698500+0000","last_active":"2024-03-06T07:26:18.747506+0000","last_peered":"2024-03-06T07:26:18.747506+0000","last_clean":"2024-03-06T07:26:18.747506+0000","last_became_active":"2024-03-06T07:26:06.698299+0000","last_became_peered":"2024-03-06T07:26:06.698299+0000","last_unstale":"2024-03-06T07:26:18.747506+0000","last_undegraded":"2024-03-06T07:26:18.747506+0000","last_fullsized":"2024-03-06T07:26:18.747506+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:17.195623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746855+0000","last_change":"2024-03-06T07:26:17.747833+0000","last_active":"2024-03-06T07:26:18.746855+0000","last_peered":"2024-03-06T07:26:18.746855+0000","last_clean":"2024-03-06T07:26:18.746855+0000","last_became_active":"2024-03-06T07:26:17.747575+0000","last_became_peered":"2024-03-06T07:26:17.747575+0000","last_unstale":"2024-03-06T07:26:18.746855+0000","last_undegraded":"2024-03-06T07:26:18.746855+0000","last_fullsized":"2024-03-06T07:26:18.746855+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:09.554449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"154'8","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.765009+0000","last_change":"2024-03-06T07:26:16.741372+0000","last_active":"2024-03-06T07:26:17.765009+0000","last_peered":"2024-03-06T07:26:17.765009+0000","last_clean":"2024-03-06T07:26:17.765009+0000","last_became_active":"2024-03-06T07:26:16.741131+0000","last_became_peered":"2024-03-06T07:26:16.741131+0000","last_unstale":"2024-03-06T07:26:17.765009+0000","last_undegraded":"2024-03-06T07:26:17.765009+0000","last_fullsized":"2024-03-06T07:26:17.765009+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:11.875182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806422+0000","last_change":"2024-03-06T07:26:07.698545+0000","last_active":"2024-03-06T07:26:18.806422+0000","last_peered":"2024-03-06T07:26:18.806422+0000","last_clean":"2024-03-06T07:26:18.806422+0000","last_became_active":"2024-03-06T07:26:07.698270+0000","last_became_peered":"2024-03-06T07:26:07.698270+0000","last_unstale":"2024-03-06T07:26:18.806422+0000","last_undegraded":"2024-03-06T07:26:18.806422+0000","last_fullsized":"2024-03-06T07:26:18.806422+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:59:25.247264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"150'1","reported_seq":21,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738979+0000","last_change":"2024-03-06T07:26:06.702569+0000","last_active":"2024-03-06T07:26:17.738979+0000","last_peered":"2024-03-06T07:26:17.738979+0000","last_clean":"2024-03-06T07:26:17.738979+0000","last_became_active":"2024-03-06T07:26:06.702304+0000","last_became_peered":"2024-03-06T07:26:06.702304+0000","last_unstale":"2024-03-06T07:26:17.738979+0000","last_undegraded":"2024-03-06T07:26:17.738979+0000","last_fullsized":"2024-03-06T07:26:17.738979+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:16.087949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001008+0000","last_change":"2024-03-06T07:26:17.750415+0000","last_active":"2024-03-06T07:26:19.001008+0000","last_peered":"2024-03-06T07:26:19.001008+0000","last_clean":"2024-03-06T07:26:19.001008+0000","last_became_active":"2024-03-06T07:26:17.748976+0000","last_became_peered":"2024-03-06T07:26:17.748976+0000","last_unstale":"2024-03-06T07:26:19.001008+0000","last_undegraded":"2024-03-06T07:26:19.001008+0000","last_fullsized":"2024-03-06T07:26:19.001008+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:11.939182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746593+0000","last_change":"2024-03-06T07:26:16.740808+0000","last_active":"2024-03-06T07:26:18.746593+0000","last_peered":"2024-03-06T07:26:18.746593+0000","last_clean":"2024-03-06T07:26:18.746593+0000","last_became_active":"2024-03-06T07:26:16.740532+0000","last_became_peered":"2024-03-06T07:26:16.740532+0000","last_unstale":"2024-03-06T07:26:18.746593+0000","last_undegraded":"2024-03-06T07:26:18.746593+0000","last_fullsized":"2024-03-06T07:26:18.746593+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:59.758111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743865+0000","last_change":"2024-03-06T07:26:07.701966+0000","last_active":"2024-03-06T07:26:17.743865+0000","last_peered":"2024-03-06T07:26:17.743865+0000","last_clean":"2024-03-06T07:26:17.743865+0000","last_became_active":"2024-03-06T07:26:07.701671+0000","last_became_peered":"2024-03-06T07:26:07.701671+0000","last_unstale":"2024-03-06T07:26:17.743865+0000","last_undegraded":"2024-03-06T07:26:17.743865+0000","last_fullsized":"2024-03-06T07:26:17.743865+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:03.437751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.746289+0000","last_change":"2024-03-06T07:26:06.691461+0000","last_active":"2024-03-06T07:26:17.746289+0000","last_peered":"2024-03-06T07:26:17.746289+0000","last_clean":"2024-03-06T07:26:17.746289+0000","last_became_active":"2024-03-06T07:26:06.691169+0000","last_became_peered":"2024-03-06T07:26:06.691169+0000","last_unstale":"2024-03-06T07:26:17.746289+0000","last_undegraded":"2024-03-06T07:26:17.746289+0000","last_fullsized":"2024-03-06T07:26:17.746289+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806365+0000","last_change":"2024-03-06T07:26:17.756094+0000","last_active":"2024-03-06T07:26:18.806365+0000","last_peered":"2024-03-06T07:26:18.806365+0000","last_clean":"2024-03-06T07:26:18.806365+0000","last_became_active":"2024-03-06T07:26:17.755837+0000","last_became_peered":"2024-03-06T07:26:17.755837+0000","last_unstale":"2024-03-06T07:26:18.806365+0000","last_undegraded":"2024-03-06T07:26:18.806365+0000","last_fullsized":"2024-03-06T07:26:18.806365+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:02.797165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749384+0000","last_change":"2024-03-06T07:26:16.740881+0000","last_active":"2024-03-06T07:26:19.749384+0000","last_peered":"2024-03-06T07:26:19.749384+0000","last_clean":"2024-03-06T07:26:19.749384+0000","last_became_active":"2024-03-06T07:26:16.740610+0000","last_became_peered":"2024-03-06T07:26:16.740610+0000","last_unstale":"2024-03-06T07:26:19.749384+0000","last_undegraded":"2024-03-06T07:26:19.749384+0000","last_fullsized":"2024-03-06T07:26:19.749384+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:13.932609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745015+0000","last_change":"2024-03-06T07:26:07.705527+0000","last_active":"2024-03-06T07:26:18.745015+0000","last_peered":"2024-03-06T07:26:18.745015+0000","last_clean":"2024-03-06T07:26:18.745015+0000","last_became_active":"2024-03-06T07:26:07.705214+0000","last_became_peered":"2024-03-06T07:26:07.705214+0000","last_unstale":"2024-03-06T07:26:18.745015+0000","last_undegraded":"2024-03-06T07:26:18.745015+0000","last_fullsized":"2024-03-06T07:26:18.745015+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:09.357832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"150'2","reported_seq":24,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001562+0000","last_change":"2024-03-06T07:26:06.689558+0000","last_active":"2024-03-06T07:26:19.001562+0000","last_peered":"2024-03-06T07:26:19.001562+0000","last_clean":"2024-03-06T07:26:19.001562+0000","last_became_active":"2024-03-06T07:26:06.689291+0000","last_became_peered":"2024-03-06T07:26:06.689291+0000","last_unstale":"2024-03-06T07:26:19.001562+0000","last_undegraded":"2024-03-06T07:26:19.001562+0000","last_fullsized":"2024-03-06T07:26:19.001562+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:59.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747393+0000","last_change":"2024-03-06T07:26:17.747419+0000","last_active":"2024-03-06T07:26:18.747393+0000","last_peered":"2024-03-06T07:26:18.747393+0000","last_clean":"2024-03-06T07:26:18.747393+0000","last_became_active":"2024-03-06T07:26:17.747131+0000","last_became_peered":"2024-03-06T07:26:17.747131+0000","last_unstale":"2024-03-06T07:26:18.747393+0000","last_undegraded":"2024-03-06T07:26:18.747393+0000","last_fullsized":"2024-03-06T07:26:18.747393+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:06.045937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"154'3","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.761355+0000","last_change":"2024-03-06T07:26:16.742718+0000","last_active":"2024-03-06T07:26:17.761355+0000","last_peered":"2024-03-06T07:26:17.761355+0000","last_clean":"2024-03-06T07:26:17.761355+0000","last_became_active":"2024-03-06T07:26:16.742380+0000","last_became_peered":"2024-03-06T07:26:16.742380+0000","last_unstale":"2024-03-06T07:26:17.761355+0000","last_undegraded":"2024-03-06T07:26:17.761355+0000","last_fullsized":"2024-03-06T07:26:17.761355+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:11.700610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749126+0000","last_change":"2024-03-06T07:26:07.697671+0000","last_active":"2024-03-06T07:26:19.749126+0000","last_peered":"2024-03-06T07:26:19.749126+0000","last_clean":"2024-03-06T07:26:19.749126+0000","last_became_active":"2024-03-06T07:26:07.697366+0000","last_became_peered":"2024-03-06T07:26:07.697366+0000","last_unstale":"2024-03-06T07:26:19.749126+0000","last_undegraded":"2024-03-06T07:26:19.749126+0000","last_fullsized":"2024-03-06T07:26:19.749126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"150'1","reported_seq":23,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749286+0000","last_change":"2024-03-06T07:26:06.700139+0000","last_active":"2024-03-06T07:26:19.749286+0000","last_peered":"2024-03-06T07:26:19.749286+0000","last_clean":"2024-03-06T07:26:19.749286+0000","last_became_active":"2024-03-06T07:26:06.699876+0000","last_became_peered":"2024-03-06T07:26:06.699876+0000","last_unstale":"2024-03-06T07:26:19.749286+0000","last_undegraded":"2024-03-06T07:26:19.749286+0000","last_fullsized":"2024-03-06T07:26:19.749286+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:22.510699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749740+0000","last_change":"2024-03-06T07:26:17.746435+0000","last_active":"2024-03-06T07:26:19.749740+0000","last_peered":"2024-03-06T07:26:19.749740+0000","last_clean":"2024-03-06T07:26:19.749740+0000","last_became_active":"2024-03-06T07:26:17.746036+0000","last_became_peered":"2024-03-06T07:26:17.746036+0000","last_unstale":"2024-03-06T07:26:19.749740+0000","last_undegraded":"2024-03-06T07:26:19.749740+0000","last_fullsized":"2024-03-06T07:26:19.749740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:04.831471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"154'6","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746503+0000","last_change":"2024-03-06T07:26:16.737058+0000","last_active":"2024-03-06T07:26:18.746503+0000","last_peered":"2024-03-06T07:26:18.746503+0000","last_clean":"2024-03-06T07:26:18.746503+0000","last_became_active":"2024-03-06T07:26:16.736759+0000","last_became_peered":"2024-03-06T07:26:16.736759+0000","last_unstale":"2024-03-06T07:26:18.746503+0000","last_undegraded":"2024-03-06T07:26:18.746503+0000","last_fullsized":"2024-03-06T07:26:18.746503+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:19.128809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001438+0000","last_change":"2024-03-06T07:26:07.705724+0000","last_active":"2024-03-06T07:26:19.001438+0000","last_peered":"2024-03-06T07:26:19.001438+0000","last_clean":"2024-03-06T07:26:19.001438+0000","last_became_active":"2024-03-06T07:26:07.705378+0000","last_became_peered":"2024-03-06T07:26:07.705378+0000","last_unstale":"2024-03-06T07:26:19.001438+0000","last_undegraded":"2024-03-06T07:26:19.001438+0000","last_fullsized":"2024-03-06T07:26:19.001438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:38:29.508415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"150'1","reported_seq":21,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743946+0000","last_change":"2024-03-06T07:26:06.702328+0000","last_active":"2024-03-06T07:26:17.743946+0000","last_peered":"2024-03-06T07:26:17.743946+0000","last_clean":"2024-03-06T07:26:17.743946+0000","last_became_active":"2024-03-06T07:26:06.702009+0000","last_became_peered":"2024-03-06T07:26:06.702009+0000","last_unstale":"2024-03-06T07:26:17.743946+0000","last_undegraded":"2024-03-06T07:26:17.743946+0000","last_fullsized":"2024-03-06T07:26:17.743946+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:14.494596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745356+0000","last_change":"2024-03-06T07:26:17.739382+0000","last_active":"2024-03-06T07:26:18.745356+0000","last_peered":"2024-03-06T07:26:18.745356+0000","last_clean":"2024-03-06T07:26:18.745356+0000","last_became_active":"2024-03-06T07:26:17.739065+0000","last_became_peered":"2024-03-06T07:26:17.739065+0000","last_unstale":"2024-03-06T07:26:18.745356+0000","last_undegraded":"2024-03-06T07:26:18.745356+0000","last_fullsized":"2024-03-06T07:26:18.745356+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:10:47.968925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807433+0000","last_change":"2024-03-06T07:26:16.741605+0000","last_active":"2024-03-06T07:26:18.807433+0000","last_peered":"2024-03-06T07:26:18.807433+0000","last_clean":"2024-03-06T07:26:18.807433+0000","last_became_active":"2024-03-06T07:26:16.741355+0000","last_became_peered":"2024-03-06T07:26:16.741355+0000","last_unstale":"2024-03-06T07:26:18.807433+0000","last_undegraded":"2024-03-06T07:26:18.807433+0000","last_fullsized":"2024-03-06T07:26:18.807433+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:25.144021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001448+0000","last_change":"2024-03-06T07:26:07.698151+0000","last_active":"2024-03-06T07:26:19.001448+0000","last_peered":"2024-03-06T07:26:19.001448+0000","last_clean":"2024-03-06T07:26:19.001448+0000","last_became_active":"2024-03-06T07:26:07.697876+0000","last_became_peered":"2024-03-06T07:26:07.697876+0000","last_unstale":"2024-03-06T07:26:19.001448+0000","last_undegraded":"2024-03-06T07:26:19.001448+0000","last_fullsized":"2024-03-06T07:26:19.001448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:56.601965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001309+0000","last_change":"2024-03-06T07:26:06.700042+0000","last_active":"2024-03-06T07:26:19.001309+0000","last_peered":"2024-03-06T07:26:19.001309+0000","last_clean":"2024-03-06T07:26:19.001309+0000","last_became_active":"2024-03-06T07:26:06.699781+0000","last_became_peered":"2024-03-06T07:26:06.699781+0000","last_unstale":"2024-03-06T07:26:19.001309+0000","last_undegraded":"2024-03-06T07:26:19.001309+0000","last_fullsized":"2024-03-06T07:26:19.001309+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:13.960358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806959+0000","last_change":"2024-03-06T07:26:17.748519+0000","last_active":"2024-03-06T07:26:18.806959+0000","last_peered":"2024-03-06T07:26:18.806959+0000","last_clean":"2024-03-06T07:26:18.806959+0000","last_became_active":"2024-03-06T07:26:17.748239+0000","last_became_peered":"2024-03-06T07:26:17.748239+0000","last_unstale":"2024-03-06T07:26:18.806959+0000","last_undegraded":"2024-03-06T07:26:18.806959+0000","last_fullsized":"2024-03-06T07:26:18.806959+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:02.981819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749425+0000","last_change":"2024-03-06T07:26:16.742781+0000","last_active":"2024-03-06T07:26:19.749425+0000","last_peered":"2024-03-06T07:26:19.749425+0000","last_clean":"2024-03-06T07:26:19.749425+0000","last_became_active":"2024-03-06T07:26:16.742519+0000","last_became_peered":"2024-03-06T07:26:16.742519+0000","last_unstale":"2024-03-06T07:26:19.749425+0000","last_undegraded":"2024-03-06T07:26:19.749425+0000","last_fullsized":"2024-03-06T07:26:19.749425+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:15.307222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747373+0000","last_change":"2024-03-06T07:26:07.696784+0000","last_active":"2024-03-06T07:26:18.747373+0000","last_peered":"2024-03-06T07:26:18.747373+0000","last_clean":"2024-03-06T07:26:18.747373+0000","last_became_active":"2024-03-06T07:26:07.696515+0000","last_became_peered":"2024-03-06T07:26:07.696515+0000","last_unstale":"2024-03-06T07:26:18.747373+0000","last_undegraded":"2024-03-06T07:26:18.747373+0000","last_fullsized":"2024-03-06T07:26:18.747373+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:07.777819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748910+0000","last_change":"2024-03-06T07:26:06.693663+0000","last_active":"2024-03-06T07:26:19.748910+0000","last_peered":"2024-03-06T07:26:19.748910+0000","last_clean":"2024-03-06T07:26:19.748910+0000","last_became_active":"2024-03-06T07:26:06.693251+0000","last_became_peered":"2024-03-06T07:26:06.693251+0000","last_unstale":"2024-03-06T07:26:19.748910+0000","last_undegraded":"2024-03-06T07:26:19.748910+0000","last_fullsized":"2024-03-06T07:26:19.748910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749564+0000","last_change":"2024-03-06T07:26:17.751564+0000","last_active":"2024-03-06T07:26:19.749564+0000","last_peered":"2024-03-06T07:26:19.749564+0000","last_clean":"2024-03-06T07:26:19.749564+0000","last_became_active":"2024-03-06T07:26:17.751256+0000","last_became_peered":"2024-03-06T07:26:17.751256+0000","last_unstale":"2024-03-06T07:26:19.749564+0000","last_undegraded":"2024-03-06T07:26:19.749564+0000","last_fullsized":"2024-03-06T07:26:19.749564+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:29.345786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746227+0000","last_change":"2024-03-06T07:26:16.730081+0000","last_active":"2024-03-06T07:26:18.746227+0000","last_peered":"2024-03-06T07:26:18.746227+0000","last_clean":"2024-03-06T07:26:18.746227+0000","last_became_active":"2024-03-06T07:26:16.729647+0000","last_became_peered":"2024-03-06T07:26:16.729647+0000","last_unstale":"2024-03-06T07:26:18.746227+0000","last_undegraded":"2024-03-06T07:26:18.746227+0000","last_fullsized":"2024-03-06T07:26:18.746227+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:29.900555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744501+0000","last_change":"2024-03-06T07:26:07.698185+0000","last_active":"2024-03-06T07:26:18.744501+0000","last_peered":"2024-03-06T07:26:18.744501+0000","last_clean":"2024-03-06T07:26:18.744501+0000","last_became_active":"2024-03-06T07:26:07.696887+0000","last_became_peered":"2024-03-06T07:26:07.696887+0000","last_unstale":"2024-03-06T07:26:18.744501+0000","last_undegraded":"2024-03-06T07:26:18.744501+0000","last_fullsized":"2024-03-06T07:26:18.744501+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:30.446061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807520+0000","last_change":"2024-03-06T07:26:06.689754+0000","last_active":"2024-03-06T07:26:18.807520+0000","last_peered":"2024-03-06T07:26:18.807520+0000","last_clean":"2024-03-06T07:26:18.807520+0000","last_became_active":"2024-03-06T07:26:06.689386+0000","last_became_peered":"2024-03-06T07:26:06.689386+0000","last_unstale":"2024-03-06T07:26:18.807520+0000","last_undegraded":"2024-03-06T07:26:18.807520+0000","last_fullsized":"2024-03-06T07:26:18.807520+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745966+0000","last_change":"2024-03-06T07:26:17.749723+0000","last_active":"2024-03-06T07:26:18.745966+0000","last_peered":"2024-03-06T07:26:18.745966+0000","last_clean":"2024-03-06T07:26:18.745966+0000","last_became_active":"2024-03-06T07:26:17.749469+0000","last_became_peered":"2024-03-06T07:26:17.749469+0000","last_unstale":"2024-03-06T07:26:18.745966+0000","last_undegraded":"2024-03-06T07:26:18.745966+0000","last_fullsized":"2024-03-06T07:26:18.745966+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:47.445099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806588+0000","last_change":"2024-03-06T07:26:16.738778+0000","last_active":"2024-03-06T07:26:18.806588+0000","last_peered":"2024-03-06T07:26:18.806588+0000","last_clean":"2024-03-06T07:26:18.806588+0000","last_became_active":"2024-03-06T07:26:16.738504+0000","last_became_peered":"2024-03-06T07:26:16.738504+0000","last_unstale":"2024-03-06T07:26:18.806588+0000","last_undegraded":"2024-03-06T07:26:18.806588+0000","last_fullsized":"2024-03-06T07:26:18.806588+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:26:26.561354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806611+0000","last_change":"2024-03-06T07:26:07.697222+0000","last_active":"2024-03-06T07:26:18.806611+0000","last_peered":"2024-03-06T07:26:18.806611+0000","last_clean":"2024-03-06T07:26:18.806611+0000","last_became_active":"2024-03-06T07:26:07.696964+0000","last_became_peered":"2024-03-06T07:26:07.696964+0000","last_unstale":"2024-03-06T07:26:18.806611+0000","last_undegraded":"2024-03-06T07:26:18.806611+0000","last_fullsized":"2024-03-06T07:26:18.806611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:59.699225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"150'3","reported_seq":25,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747386+0000","last_change":"2024-03-06T07:26:06.693324+0000","last_active":"2024-03-06T07:26:18.747386+0000","last_peered":"2024-03-06T07:26:18.747386+0000","last_clean":"2024-03-06T07:26:18.747386+0000","last_became_active":"2024-03-06T07:26:06.693047+0000","last_became_peered":"2024-03-06T07:26:06.693047+0000","last_unstale":"2024-03-06T07:26:18.747386+0000","last_undegraded":"2024-03-06T07:26:18.747386+0000","last_fullsized":"2024-03-06T07:26:18.747386+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:59.568199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748941+0000","last_change":"2024-03-06T07:26:17.743284+0000","last_active":"2024-03-06T07:26:19.748941+0000","last_peered":"2024-03-06T07:26:19.748941+0000","last_clean":"2024-03-06T07:26:19.748941+0000","last_became_active":"2024-03-06T07:26:17.743033+0000","last_became_peered":"2024-03-06T07:26:17.743033+0000","last_unstale":"2024-03-06T07:26:19.748941+0000","last_undegraded":"2024-03-06T07:26:19.748941+0000","last_fullsized":"2024-03-06T07:26:19.748941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:02.284654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.761476+0000","last_change":"2024-03-06T07:26:16.745063+0000","last_active":"2024-03-06T07:26:17.761476+0000","last_peered":"2024-03-06T07:26:17.761476+0000","last_clean":"2024-03-06T07:26:17.761476+0000","last_became_active":"2024-03-06T07:26:16.744809+0000","last_became_peered":"2024-03-06T07:26:16.744809+0000","last_unstale":"2024-03-06T07:26:17.761476+0000","last_undegraded":"2024-03-06T07:26:17.761476+0000","last_fullsized":"2024-03-06T07:26:17.761476+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:28.340363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001164+0000","last_change":"2024-03-06T07:26:07.704905+0000","last_active":"2024-03-06T07:26:19.001164+0000","last_peered":"2024-03-06T07:26:19.001164+0000","last_clean":"2024-03-06T07:26:19.001164+0000","last_became_active":"2024-03-06T07:26:07.704581+0000","last_became_peered":"2024-03-06T07:26:07.704581+0000","last_unstale":"2024-03-06T07:26:19.001164+0000","last_undegraded":"2024-03-06T07:26:19.001164+0000","last_fullsized":"2024-03-06T07:26:19.001164+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:06:22.770076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"150'6","reported_seq":28,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807230+0000","last_change":"2024-03-06T07:26:06.686441+0000","last_active":"2024-03-06T07:26:18.807230+0000","last_peered":"2024-03-06T07:26:18.807230+0000","last_clean":"2024-03-06T07:26:18.807230+0000","last_became_active":"2024-03-06T07:26:06.686158+0000","last_became_peered":"2024-03-06T07:26:06.686158+0000","last_unstale":"2024-03-06T07:26:18.807230+0000","last_undegraded":"2024-03-06T07:26:18.807230+0000","last_fullsized":"2024-03-06T07:26:18.807230+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:05.262174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760489+0000","last_change":"2024-03-06T07:26:17.753430+0000","last_active":"2024-03-06T07:26:17.760489+0000","last_peered":"2024-03-06T07:26:17.760489+0000","last_clean":"2024-03-06T07:26:17.760489+0000","last_became_active":"2024-03-06T07:26:17.750028+0000","last_became_peered":"2024-03-06T07:26:17.750028+0000","last_unstale":"2024-03-06T07:26:17.760489+0000","last_undegraded":"2024-03-06T07:26:17.760489+0000","last_fullsized":"2024-03-06T07:26:17.760489+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:18.355223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.764879+0000","last_change":"2024-03-06T07:26:16.738161+0000","last_active":"2024-03-06T07:26:17.764879+0000","last_peered":"2024-03-06T07:26:17.764879+0000","last_clean":"2024-03-06T07:26:17.764879+0000","last_became_active":"2024-03-06T07:26:16.737876+0000","last_became_peered":"2024-03-06T07:26:16.737876+0000","last_unstale":"2024-03-06T07:26:17.764879+0000","last_undegraded":"2024-03-06T07:26:17.764879+0000","last_fullsized":"2024-03-06T07:26:17.764879+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:18.397443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882319+0000","last_change":"2024-03-06T07:26:07.698486+0000","last_active":"2024-03-06T07:26:18.882319+0000","last_peered":"2024-03-06T07:26:18.882319+0000","last_clean":"2024-03-06T07:26:18.882319+0000","last_became_active":"2024-03-06T07:26:07.698210+0000","last_became_peered":"2024-03-06T07:26:07.698210+0000","last_unstale":"2024-03-06T07:26:18.882319+0000","last_undegraded":"2024-03-06T07:26:18.882319+0000","last_fullsized":"2024-03-06T07:26:18.882319+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:31:41.658258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738610+0000","last_change":"2024-03-06T07:26:06.698989+0000","last_active":"2024-03-06T07:26:17.738610+0000","last_peered":"2024-03-06T07:26:17.738610+0000","last_clean":"2024-03-06T07:26:17.738610+0000","last_became_active":"2024-03-06T07:26:06.698656+0000","last_became_peered":"2024-03-06T07:26:06.698656+0000","last_unstale":"2024-03-06T07:26:17.738610+0000","last_undegraded":"2024-03-06T07:26:17.738610+0000","last_fullsized":"2024-03-06T07:26:17.738610+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882544+0000","last_change":"2024-03-06T07:26:17.752964+0000","last_active":"2024-03-06T07:26:18.882544+0000","last_peered":"2024-03-06T07:26:18.882544+0000","last_clean":"2024-03-06T07:26:18.882544+0000","last_became_active":"2024-03-06T07:26:17.751768+0000","last_became_peered":"2024-03-06T07:26:17.751768+0000","last_unstale":"2024-03-06T07:26:18.882544+0000","last_undegraded":"2024-03-06T07:26:18.882544+0000","last_fullsized":"2024-03-06T07:26:18.882544+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:45.573417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747102+0000","last_change":"2024-03-06T07:26:16.737220+0000","last_active":"2024-03-06T07:26:18.747102+0000","last_peered":"2024-03-06T07:26:18.747102+0000","last_clean":"2024-03-06T07:26:18.747102+0000","last_became_active":"2024-03-06T07:26:16.736946+0000","last_became_peered":"2024-03-06T07:26:16.736946+0000","last_unstale":"2024-03-06T07:26:18.747102+0000","last_undegraded":"2024-03-06T07:26:18.747102+0000","last_fullsized":"2024-03-06T07:26:18.747102+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:44.567994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749334+0000","last_change":"2024-03-06T07:26:07.709375+0000","last_active":"2024-03-06T07:26:19.749334+0000","last_peered":"2024-03-06T07:26:19.749334+0000","last_clean":"2024-03-06T07:26:19.749334+0000","last_became_active":"2024-03-06T07:26:07.709091+0000","last_became_peered":"2024-03-06T07:26:07.709091+0000","last_unstale":"2024-03-06T07:26:19.749334+0000","last_undegraded":"2024-03-06T07:26:19.749334+0000","last_fullsized":"2024-03-06T07:26:19.749334+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:41.934898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806630+0000","last_change":"2024-03-06T07:26:06.695622+0000","last_active":"2024-03-06T07:26:18.806630+0000","last_peered":"2024-03-06T07:26:18.806630+0000","last_clean":"2024-03-06T07:26:18.806630+0000","last_became_active":"2024-03-06T07:26:06.695369+0000","last_became_peered":"2024-03-06T07:26:06.695369+0000","last_unstale":"2024-03-06T07:26:18.806630+0000","last_undegraded":"2024-03-06T07:26:18.806630+0000","last_fullsized":"2024-03-06T07:26:18.806630+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:26.984542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745102+0000","last_change":"2024-03-06T07:26:17.748383+0000","last_active":"2024-03-06T07:26:18.745102+0000","last_peered":"2024-03-06T07:26:18.745102+0000","last_clean":"2024-03-06T07:26:18.745102+0000","last_became_active":"2024-03-06T07:26:17.748091+0000","last_became_peered":"2024-03-06T07:26:17.748091+0000","last_unstale":"2024-03-06T07:26:18.745102+0000","last_undegraded":"2024-03-06T07:26:18.745102+0000","last_fullsized":"2024-03-06T07:26:18.745102+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:34.812576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"154'4","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.759683+0000","last_change":"2024-03-06T07:26:16.743306+0000","last_active":"2024-03-06T07:26:17.759683+0000","last_peered":"2024-03-06T07:26:17.759683+0000","last_clean":"2024-03-06T07:26:17.759683+0000","last_became_active":"2024-03-06T07:26:16.741028+0000","last_became_peered":"2024-03-06T07:26:16.741028+0000","last_unstale":"2024-03-06T07:26:17.759683+0000","last_undegraded":"2024-03-06T07:26:17.759683+0000","last_fullsized":"2024-03-06T07:26:17.759683+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:04.976679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.745408+0000","last_change":"2024-03-06T07:26:07.698143+0000","last_active":"2024-03-06T07:26:17.745408+0000","last_peered":"2024-03-06T07:26:17.745408+0000","last_clean":"2024-03-06T07:26:17.745408+0000","last_became_active":"2024-03-06T07:26:07.697880+0000","last_became_peered":"2024-03-06T07:26:07.697880+0000","last_unstale":"2024-03-06T07:26:17.745408+0000","last_undegraded":"2024-03-06T07:26:17.745408+0000","last_fullsized":"2024-03-06T07:26:17.745408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744836+0000","last_change":"2024-03-06T07:26:06.696386+0000","last_active":"2024-03-06T07:26:17.744836+0000","last_peered":"2024-03-06T07:26:17.744836+0000","last_clean":"2024-03-06T07:26:17.744836+0000","last_became_active":"2024-03-06T07:26:06.696108+0000","last_became_peered":"2024-03-06T07:26:06.696108+0000","last_unstale":"2024-03-06T07:26:17.744836+0000","last_undegraded":"2024-03-06T07:26:17.744836+0000","last_fullsized":"2024-03-06T07:26:17.744836+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001269+0000","last_change":"2024-03-06T07:26:17.738958+0000","last_active":"2024-03-06T07:26:19.001269+0000","last_peered":"2024-03-06T07:26:19.001269+0000","last_clean":"2024-03-06T07:26:19.001269+0000","last_became_active":"2024-03-06T07:26:17.738699+0000","last_became_peered":"2024-03-06T07:26:17.738699+0000","last_unstale":"2024-03-06T07:26:19.001269+0000","last_undegraded":"2024-03-06T07:26:19.001269+0000","last_fullsized":"2024-03-06T07:26:19.001269+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:42.267362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807053+0000","last_change":"2024-03-06T07:26:16.738986+0000","last_active":"2024-03-06T07:26:18.807053+0000","last_peered":"2024-03-06T07:26:18.807053+0000","last_clean":"2024-03-06T07:26:18.807053+0000","last_became_active":"2024-03-06T07:26:16.738729+0000","last_became_peered":"2024-03-06T07:26:16.738729+0000","last_unstale":"2024-03-06T07:26:18.807053+0000","last_undegraded":"2024-03-06T07:26:18.807053+0000","last_fullsized":"2024-03-06T07:26:18.807053+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:22.333796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001707+0000","last_change":"2024-03-06T07:26:07.694636+0000","last_active":"2024-03-06T07:26:19.001707+0000","last_peered":"2024-03-06T07:26:19.001707+0000","last_clean":"2024-03-06T07:26:19.001707+0000","last_became_active":"2024-03-06T07:26:07.694340+0000","last_became_peered":"2024-03-06T07:26:07.694340+0000","last_unstale":"2024-03-06T07:26:19.001707+0000","last_undegraded":"2024-03-06T07:26:19.001707+0000","last_fullsized":"2024-03-06T07:26:19.001707+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:27:42.904456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"150'8","reported_seq":30,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748965+0000","last_change":"2024-03-06T07:26:06.692364+0000","last_active":"2024-03-06T07:26:19.748965+0000","last_peered":"2024-03-06T07:26:19.748965+0000","last_clean":"2024-03-06T07:26:19.748965+0000","last_became_active":"2024-03-06T07:26:06.692119+0000","last_became_peered":"2024-03-06T07:26:06.692119+0000","last_unstale":"2024-03-06T07:26:19.748965+0000","last_undegraded":"2024-03-06T07:26:19.748965+0000","last_fullsized":"2024-03-06T07:26:19.748965+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:09.083760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749159+0000","last_change":"2024-03-06T07:26:17.745908+0000","last_active":"2024-03-06T07:26:19.749159+0000","last_peered":"2024-03-06T07:26:19.749159+0000","last_clean":"2024-03-06T07:26:19.749159+0000","last_became_active":"2024-03-06T07:26:17.745649+0000","last_became_peered":"2024-03-06T07:26:17.745649+0000","last_unstale":"2024-03-06T07:26:19.749159+0000","last_undegraded":"2024-03-06T07:26:19.749159+0000","last_fullsized":"2024-03-06T07:26:19.749159+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:12.706034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.000990+0000","last_change":"2024-03-06T07:26:16.736287+0000","last_active":"2024-03-06T07:26:19.000990+0000","last_peered":"2024-03-06T07:26:19.000990+0000","last_clean":"2024-03-06T07:26:19.000990+0000","last_became_active":"2024-03-06T07:26:16.734127+0000","last_became_peered":"2024-03-06T07:26:16.734127+0000","last_unstale":"2024-03-06T07:26:19.000990+0000","last_undegraded":"2024-03-06T07:26:19.000990+0000","last_fullsized":"2024-03-06T07:26:19.000990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:27.821504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748818+0000","last_change":"2024-03-06T07:26:07.698466+0000","last_active":"2024-03-06T07:26:19.748818+0000","last_peered":"2024-03-06T07:26:19.748818+0000","last_clean":"2024-03-06T07:26:19.748818+0000","last_became_active":"2024-03-06T07:26:07.697848+0000","last_became_peered":"2024-03-06T07:26:07.697848+0000","last_unstale":"2024-03-06T07:26:19.748818+0000","last_undegraded":"2024-03-06T07:26:19.748818+0000","last_fullsized":"2024-03-06T07:26:19.748818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:37.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"150'1","reported_seq":23,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807568+0000","last_change":"2024-03-06T07:26:06.694226+0000","last_active":"2024-03-06T07:26:18.807568+0000","last_peered":"2024-03-06T07:26:18.807568+0000","last_clean":"2024-03-06T07:26:18.807568+0000","last_became_active":"2024-03-06T07:26:06.693966+0000","last_became_peered":"2024-03-06T07:26:06.693966+0000","last_unstale":"2024-03-06T07:26:18.807568+0000","last_undegraded":"2024-03-06T07:26:18.807568+0000","last_fullsized":"2024-03-06T07:26:18.807568+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:45.769621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744786+0000","last_change":"2024-03-06T07:26:17.746782+0000","last_active":"2024-03-06T07:26:18.744786+0000","last_peered":"2024-03-06T07:26:18.744786+0000","last_clean":"2024-03-06T07:26:18.744786+0000","last_became_active":"2024-03-06T07:26:17.746473+0000","last_became_peered":"2024-03-06T07:26:17.746473+0000","last_unstale":"2024-03-06T07:26:18.744786+0000","last_undegraded":"2024-03-06T07:26:18.744786+0000","last_fullsized":"2024-03-06T07:26:18.744786+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:02.284654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882207+0000","last_change":"2024-03-06T07:26:16.738787+0000","last_active":"2024-03-06T07:26:18.882207+0000","last_peered":"2024-03-06T07:26:18.882207+0000","last_clean":"2024-03-06T07:26:18.882207+0000","last_became_active":"2024-03-06T07:26:16.738428+0000","last_became_peered":"2024-03-06T07:26:16.738428+0000","last_unstale":"2024-03-06T07:26:18.882207+0000","last_undegraded":"2024-03-06T07:26:18.882207+0000","last_fullsized":"2024-03-06T07:26:18.882207+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:21.825588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882439+0000","last_change":"2024-03-06T07:26:07.695629+0000","last_active":"2024-03-06T07:26:18.882439+0000","last_peered":"2024-03-06T07:26:18.882439+0000","last_clean":"2024-03-06T07:26:18.882439+0000","last_became_active":"2024-03-06T07:26:07.695358+0000","last_became_peered":"2024-03-06T07:26:07.695358+0000","last_unstale":"2024-03-06T07:26:18.882439+0000","last_undegraded":"2024-03-06T07:26:18.882439+0000","last_fullsized":"2024-03-06T07:26:18.882439+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:32.222328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"150'2","reported_seq":24,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749766+0000","last_change":"2024-03-06T07:26:06.698927+0000","last_active":"2024-03-06T07:26:19.749766+0000","last_peered":"2024-03-06T07:26:19.749766+0000","last_clean":"2024-03-06T07:26:19.749766+0000","last_became_active":"2024-03-06T07:26:06.698636+0000","last_became_peered":"2024-03-06T07:26:06.698636+0000","last_unstale":"2024-03-06T07:26:19.749766+0000","last_undegraded":"2024-03-06T07:26:19.749766+0000","last_fullsized":"2024-03-06T07:26:19.749766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882217+0000","last_change":"2024-03-06T07:26:17.752227+0000","last_active":"2024-03-06T07:26:18.882217+0000","last_peered":"2024-03-06T07:26:18.882217+0000","last_clean":"2024-03-06T07:26:18.882217+0000","last_became_active":"2024-03-06T07:26:17.751966+0000","last_became_peered":"2024-03-06T07:26:17.751966+0000","last_unstale":"2024-03-06T07:26:18.882217+0000","last_undegraded":"2024-03-06T07:26:18.882217+0000","last_fullsized":"2024-03-06T07:26:18.882217+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:09:20.134232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744723+0000","last_change":"2024-03-06T07:26:16.734670+0000","last_active":"2024-03-06T07:26:18.744723+0000","last_peered":"2024-03-06T07:26:18.744723+0000","last_clean":"2024-03-06T07:26:18.744723+0000","last_became_active":"2024-03-06T07:26:16.733618+0000","last_became_peered":"2024-03-06T07:26:16.733618+0000","last_unstale":"2024-03-06T07:26:18.744723+0000","last_undegraded":"2024-03-06T07:26:18.744723+0000","last_fullsized":"2024-03-06T07:26:18.744723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:38.826593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744196+0000","last_change":"2024-03-06T07:26:07.697409+0000","last_active":"2024-03-06T07:26:17.744196+0000","last_peered":"2024-03-06T07:26:17.744196+0000","last_clean":"2024-03-06T07:26:17.744196+0000","last_became_active":"2024-03-06T07:26:07.697084+0000","last_became_peered":"2024-03-06T07:26:07.697084+0000","last_unstale":"2024-03-06T07:26:17.744196+0000","last_undegraded":"2024-03-06T07:26:17.744196+0000","last_fullsized":"2024-03-06T07:26:17.744196+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:36:59.509416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"150'1","reported_seq":21,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.739525+0000","last_change":"2024-03-06T07:26:06.695033+0000","last_active":"2024-03-06T07:26:17.739525+0000","last_peered":"2024-03-06T07:26:17.739525+0000","last_clean":"2024-03-06T07:26:17.739525+0000","last_became_active":"2024-03-06T07:26:06.694763+0000","last_became_peered":"2024-03-06T07:26:06.694763+0000","last_unstale":"2024-03-06T07:26:17.739525+0000","last_undegraded":"2024-03-06T07:26:17.739525+0000","last_fullsized":"2024-03-06T07:26:17.739525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:29.440623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746363+0000","last_change":"2024-03-06T07:26:17.752315+0000","last_active":"2024-03-06T07:26:18.746363+0000","last_peered":"2024-03-06T07:26:18.746363+0000","last_clean":"2024-03-06T07:26:18.746363+0000","last_became_active":"2024-03-06T07:26:17.752025+0000","last_became_peered":"2024-03-06T07:26:17.752025+0000","last_unstale":"2024-03-06T07:26:18.746363+0000","last_undegraded":"2024-03-06T07:26:18.746363+0000","last_fullsized":"2024-03-06T07:26:18.746363+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:28.209449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.744560+0000","last_change":"2024-03-06T07:26:16.735510+0000","last_active":"2024-03-06T07:26:18.744560+0000","last_peered":"2024-03-06T07:26:18.744560+0000","last_clean":"2024-03-06T07:26:18.744560+0000","last_became_active":"2024-03-06T07:26:16.735238+0000","last_became_peered":"2024-03-06T07:26:16.735238+0000","last_unstale":"2024-03-06T07:26:18.744560+0000","last_undegraded":"2024-03-06T07:26:18.744560+0000","last_fullsized":"2024-03-06T07:26:18.744560+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:12.997028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744500+0000","last_change":"2024-03-06T07:26:07.706589+0000","last_active":"2024-03-06T07:26:17.744500+0000","last_peered":"2024-03-06T07:26:17.744500+0000","last_clean":"2024-03-06T07:26:17.744500+0000","last_became_active":"2024-03-06T07:26:07.706305+0000","last_became_peered":"2024-03-06T07:26:07.706305+0000","last_unstale":"2024-03-06T07:26:17.744500+0000","last_undegraded":"2024-03-06T07:26:17.744500+0000","last_fullsized":"2024-03-06T07:26:17.744500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:18.164416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749365+0000","last_change":"2024-03-06T07:26:06.697335+0000","last_active":"2024-03-06T07:26:19.749365+0000","last_peered":"2024-03-06T07:26:19.749365+0000","last_clean":"2024-03-06T07:26:19.749365+0000","last_became_active":"2024-03-06T07:26:06.697028+0000","last_became_peered":"2024-03-06T07:26:06.697028+0000","last_unstale":"2024-03-06T07:26:19.749365+0000","last_undegraded":"2024-03-06T07:26:19.749365+0000","last_fullsized":"2024-03-06T07:26:19.749365+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:34.522946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745088+0000","last_change":"2024-03-06T07:26:17.754920+0000","last_active":"2024-03-06T07:26:18.745088+0000","last_peered":"2024-03-06T07:26:18.745088+0000","last_clean":"2024-03-06T07:26:18.745088+0000","last_became_active":"2024-03-06T07:26:17.754660+0000","last_became_peered":"2024-03-06T07:26:17.754660+0000","last_unstale":"2024-03-06T07:26:18.745088+0000","last_undegraded":"2024-03-06T07:26:18.745088+0000","last_fullsized":"2024-03-06T07:26:18.745088+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:12.706034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806988+0000","last_change":"2024-03-06T07:26:16.738310+0000","last_active":"2024-03-06T07:26:18.806988+0000","last_peered":"2024-03-06T07:26:18.806988+0000","last_clean":"2024-03-06T07:26:18.806988+0000","last_became_active":"2024-03-06T07:26:16.738040+0000","last_became_peered":"2024-03-06T07:26:16.738040+0000","last_unstale":"2024-03-06T07:26:18.806988+0000","last_undegraded":"2024-03-06T07:26:18.806988+0000","last_fullsized":"2024-03-06T07:26:18.806988+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:28.199517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807146+0000","last_change":"2024-03-06T07:26:07.692326+0000","last_active":"2024-03-06T07:26:18.807146+0000","last_peered":"2024-03-06T07:26:18.807146+0000","last_clean":"2024-03-06T07:26:18.807146+0000","last_became_active":"2024-03-06T07:26:07.692063+0000","last_became_peered":"2024-03-06T07:26:07.692063+0000","last_unstale":"2024-03-06T07:26:18.807146+0000","last_undegraded":"2024-03-06T07:26:18.807146+0000","last_fullsized":"2024-03-06T07:26:18.807146+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:42.601237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738725+0000","last_change":"2024-03-06T07:26:06.696078+0000","last_active":"2024-03-06T07:26:17.738725+0000","last_peered":"2024-03-06T07:26:17.738725+0000","last_clean":"2024-03-06T07:26:17.738725+0000","last_became_active":"2024-03-06T07:26:06.695834+0000","last_became_peered":"2024-03-06T07:26:06.695834+0000","last_unstale":"2024-03-06T07:26:17.738725+0000","last_undegraded":"2024-03-06T07:26:17.738725+0000","last_fullsized":"2024-03-06T07:26:17.738725+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:47.043451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001738+0000","last_change":"2024-03-06T07:26:17.751409+0000","last_active":"2024-03-06T07:26:19.001738+0000","last_peered":"2024-03-06T07:26:19.001738+0000","last_clean":"2024-03-06T07:26:19.001738+0000","last_became_active":"2024-03-06T07:26:17.751151+0000","last_became_peered":"2024-03-06T07:26:17.751151+0000","last_unstale":"2024-03-06T07:26:19.001738+0000","last_undegraded":"2024-03-06T07:26:19.001738+0000","last_fullsized":"2024-03-06T07:26:19.001738+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:27.892875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807252+0000","last_change":"2024-03-06T07:26:16.740507+0000","last_active":"2024-03-06T07:26:18.807252+0000","last_peered":"2024-03-06T07:26:18.807252+0000","last_clean":"2024-03-06T07:26:18.807252+0000","last_became_active":"2024-03-06T07:26:16.740247+0000","last_became_peered":"2024-03-06T07:26:16.740247+0000","last_unstale":"2024-03-06T07:26:18.807252+0000","last_undegraded":"2024-03-06T07:26:18.807252+0000","last_fullsized":"2024-03-06T07:26:18.807252+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:50.785757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.739432+0000","last_change":"2024-03-06T07:26:07.713113+0000","last_active":"2024-03-06T07:26:17.739432+0000","last_peered":"2024-03-06T07:26:17.739432+0000","last_clean":"2024-03-06T07:26:17.739432+0000","last_became_active":"2024-03-06T07:26:07.712610+0000","last_became_peered":"2024-03-06T07:26:07.712610+0000","last_unstale":"2024-03-06T07:26:17.739432+0000","last_undegraded":"2024-03-06T07:26:17.739432+0000","last_fullsized":"2024-03-06T07:26:17.739432+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:03.957417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744286+0000","last_change":"2024-03-06T07:26:06.692781+0000","last_active":"2024-03-06T07:26:17.744286+0000","last_peered":"2024-03-06T07:26:17.744286+0000","last_clean":"2024-03-06T07:26:17.744286+0000","last_became_active":"2024-03-06T07:26:06.691992+0000","last_became_peered":"2024-03-06T07:26:06.691992+0000","last_unstale":"2024-03-06T07:26:17.744286+0000","last_undegraded":"2024-03-06T07:26:17.744286+0000","last_fullsized":"2024-03-06T07:26:17.744286+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:24.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001409+0000","last_change":"2024-03-06T07:26:17.746271+0000","last_active":"2024-03-06T07:26:19.001409+0000","last_peered":"2024-03-06T07:26:19.001409+0000","last_clean":"2024-03-06T07:26:19.001409+0000","last_became_active":"2024-03-06T07:26:17.746013+0000","last_became_peered":"2024-03-06T07:26:17.746013+0000","last_unstale":"2024-03-06T07:26:19.001409+0000","last_undegraded":"2024-03-06T07:26:19.001409+0000","last_fullsized":"2024-03-06T07:26:19.001409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:55.454349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760759+0000","last_change":"2024-03-06T07:26:16.742136+0000","last_active":"2024-03-06T07:26:17.760759+0000","last_peered":"2024-03-06T07:26:17.760759+0000","last_clean":"2024-03-06T07:26:17.760759+0000","last_became_active":"2024-03-06T07:26:16.740809+0000","last_became_peered":"2024-03-06T07:26:16.740809+0000","last_unstale":"2024-03-06T07:26:17.760759+0000","last_undegraded":"2024-03-06T07:26:17.760759+0000","last_fullsized":"2024-03-06T07:26:17.760759+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:10:46.963501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.738881+0000","last_change":"2024-03-06T07:26:07.713547+0000","last_active":"2024-03-06T07:26:17.738881+0000","last_peered":"2024-03-06T07:26:17.738881+0000","last_clean":"2024-03-06T07:26:17.738881+0000","last_became_active":"2024-03-06T07:26:07.712950+0000","last_became_peered":"2024-03-06T07:26:07.712950+0000","last_unstale":"2024-03-06T07:26:17.738881+0000","last_undegraded":"2024-03-06T07:26:17.738881+0000","last_fullsized":"2024-03-06T07:26:17.738881+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:00.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001469+0000","last_change":"2024-03-06T07:26:06.695426+0000","last_active":"2024-03-06T07:26:19.001469+0000","last_peered":"2024-03-06T07:26:19.001469+0000","last_clean":"2024-03-06T07:26:19.001469+0000","last_became_active":"2024-03-06T07:26:06.695162+0000","last_became_peered":"2024-03-06T07:26:06.695162+0000","last_unstale":"2024-03-06T07:26:19.001469+0000","last_undegraded":"2024-03-06T07:26:19.001469+0000","last_fullsized":"2024-03-06T07:26:19.001469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:30.485289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749421+0000","last_change":"2024-03-06T07:26:17.740123+0000","last_active":"2024-03-06T07:26:19.749421+0000","last_peered":"2024-03-06T07:26:19.749421+0000","last_clean":"2024-03-06T07:26:19.749421+0000","last_became_active":"2024-03-06T07:26:17.739862+0000","last_became_peered":"2024-03-06T07:26:17.739862+0000","last_unstale":"2024-03-06T07:26:19.749421+0000","last_undegraded":"2024-03-06T07:26:19.749421+0000","last_fullsized":"2024-03-06T07:26:19.749421+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:05.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"155'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748293+0000","last_change":"2024-03-06T07:26:16.737585+0000","last_active":"2024-03-06T07:26:19.748293+0000","last_peered":"2024-03-06T07:26:19.748293+0000","last_clean":"2024-03-06T07:26:19.748293+0000","last_became_active":"2024-03-06T07:26:16.737117+0000","last_became_peered":"2024-03-06T07:26:16.737117+0000","last_unstale":"2024-03-06T07:26:19.748293+0000","last_undegraded":"2024-03-06T07:26:19.748293+0000","last_fullsized":"2024-03-06T07:26:19.748293+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:26.441757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745500+0000","last_change":"2024-03-06T07:26:07.691244+0000","last_active":"2024-03-06T07:26:18.745500+0000","last_peered":"2024-03-06T07:26:18.745500+0000","last_clean":"2024-03-06T07:26:18.745500+0000","last_became_active":"2024-03-06T07:26:07.690929+0000","last_became_peered":"2024-03-06T07:26:07.690929+0000","last_unstale":"2024-03-06T07:26:18.745500+0000","last_undegraded":"2024-03-06T07:26:18.745500+0000","last_fullsized":"2024-03-06T07:26:18.745500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:27.989980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745237+0000","last_change":"2024-03-06T07:26:06.694068+0000","last_active":"2024-03-06T07:26:18.745237+0000","last_peered":"2024-03-06T07:26:18.745237+0000","last_clean":"2024-03-06T07:26:18.745237+0000","last_became_active":"2024-03-06T07:26:06.693777+0000","last_became_peered":"2024-03-06T07:26:06.693777+0000","last_unstale":"2024-03-06T07:26:18.745237+0000","last_undegraded":"2024-03-06T07:26:18.745237+0000","last_fullsized":"2024-03-06T07:26:18.745237+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:06.186255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.758362+0000","last_change":"2024-03-06T07:26:17.753731+0000","last_active":"2024-03-06T07:26:17.758362+0000","last_peered":"2024-03-06T07:26:17.758362+0000","last_clean":"2024-03-06T07:26:17.758362+0000","last_became_active":"2024-03-06T07:26:17.753483+0000","last_became_peered":"2024-03-06T07:26:17.753483+0000","last_unstale":"2024-03-06T07:26:17.758362+0000","last_undegraded":"2024-03-06T07:26:17.758362+0000","last_fullsized":"2024-03-06T07:26:17.758362+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:23.339219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.000941+0000","last_change":"2024-03-06T07:26:16.736897+0000","last_active":"2024-03-06T07:26:19.000941+0000","last_peered":"2024-03-06T07:26:19.000941+0000","last_clean":"2024-03-06T07:26:19.000941+0000","last_became_active":"2024-03-06T07:26:16.736623+0000","last_became_peered":"2024-03-06T07:26:16.736623+0000","last_unstale":"2024-03-06T07:26:19.000941+0000","last_undegraded":"2024-03-06T07:26:19.000941+0000","last_fullsized":"2024-03-06T07:26:19.000941+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:03.929124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743967+0000","last_change":"2024-03-06T07:26:07.696197+0000","last_active":"2024-03-06T07:26:17.743967+0000","last_peered":"2024-03-06T07:26:17.743967+0000","last_clean":"2024-03-06T07:26:17.743967+0000","last_became_active":"2024-03-06T07:26:07.695895+0000","last_became_peered":"2024-03-06T07:26:07.695895+0000","last_unstale":"2024-03-06T07:26:17.743967+0000","last_undegraded":"2024-03-06T07:26:17.743967+0000","last_fullsized":"2024-03-06T07:26:17.743967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:02.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"150'2","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.740239+0000","last_change":"2024-03-06T07:26:06.701125+0000","last_active":"2024-03-06T07:26:17.740239+0000","last_peered":"2024-03-06T07:26:17.740239+0000","last_clean":"2024-03-06T07:26:17.740239+0000","last_became_active":"2024-03-06T07:26:06.700819+0000","last_became_peered":"2024-03-06T07:26:06.700819+0000","last_unstale":"2024-03-06T07:26:17.740239+0000","last_undegraded":"2024-03-06T07:26:17.740239+0000","last_fullsized":"2024-03-06T07:26:17.740239+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:54.995466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806854+0000","last_change":"2024-03-06T07:26:17.745253+0000","last_active":"2024-03-06T07:26:18.806854+0000","last_peered":"2024-03-06T07:26:18.806854+0000","last_clean":"2024-03-06T07:26:18.806854+0000","last_became_active":"2024-03-06T07:26:17.744711+0000","last_became_peered":"2024-03-06T07:26:17.744711+0000","last_unstale":"2024-03-06T07:26:18.806854+0000","last_undegraded":"2024-03-06T07:26:18.806854+0000","last_fullsized":"2024-03-06T07:26:18.806854+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:57.856041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882331+0000","last_change":"2024-03-06T07:26:16.738255+0000","last_active":"2024-03-06T07:26:18.882331+0000","last_peered":"2024-03-06T07:26:18.882331+0000","last_clean":"2024-03-06T07:26:18.882331+0000","last_became_active":"2024-03-06T07:26:16.737911+0000","last_became_peered":"2024-03-06T07:26:16.737911+0000","last_unstale":"2024-03-06T07:26:18.882331+0000","last_undegraded":"2024-03-06T07:26:18.882331+0000","last_fullsized":"2024-03-06T07:26:18.882331+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:42:35.210727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749183+0000","last_change":"2024-03-06T07:26:07.701964+0000","last_active":"2024-03-06T07:26:19.749183+0000","last_peered":"2024-03-06T07:26:19.749183+0000","last_clean":"2024-03-06T07:26:19.749183+0000","last_became_active":"2024-03-06T07:26:07.701660+0000","last_became_peered":"2024-03-06T07:26:07.701660+0000","last_unstale":"2024-03-06T07:26:19.749183+0000","last_undegraded":"2024-03-06T07:26:19.749183+0000","last_fullsized":"2024-03-06T07:26:19.749183+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:24:28.484365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"151'1","reported_seq":23,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807276+0000","last_change":"2024-03-06T07:26:06.693341+0000","last_active":"2024-03-06T07:26:18.807276+0000","last_peered":"2024-03-06T07:26:18.807276+0000","last_clean":"2024-03-06T07:26:18.807276+0000","last_became_active":"2024-03-06T07:26:06.693014+0000","last_became_peered":"2024-03-06T07:26:06.693014+0000","last_unstale":"2024-03-06T07:26:18.807276+0000","last_undegraded":"2024-03-06T07:26:18.807276+0000","last_fullsized":"2024-03-06T07:26:18.807276+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:03.887560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.748771+0000","last_change":"2024-03-06T07:26:17.747098+0000","last_active":"2024-03-06T07:26:19.748771+0000","last_peered":"2024-03-06T07:26:19.748771+0000","last_clean":"2024-03-06T07:26:19.748771+0000","last_became_active":"2024-03-06T07:26:17.746849+0000","last_became_peered":"2024-03-06T07:26:17.746849+0000","last_unstale":"2024-03-06T07:26:19.748771+0000","last_undegraded":"2024-03-06T07:26:19.748771+0000","last_fullsized":"2024-03-06T07:26:19.748771+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:52.646271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746743+0000","last_change":"2024-03-06T07:26:16.742535+0000","last_active":"2024-03-06T07:26:18.746743+0000","last_peered":"2024-03-06T07:26:18.746743+0000","last_clean":"2024-03-06T07:26:18.746743+0000","last_became_active":"2024-03-06T07:26:16.742246+0000","last_became_peered":"2024-03-06T07:26:16.742246+0000","last_unstale":"2024-03-06T07:26:18.746743+0000","last_undegraded":"2024-03-06T07:26:18.746743+0000","last_fullsized":"2024-03-06T07:26:18.746743+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:15:44.025380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001197+0000","last_change":"2024-03-06T07:26:07.697077+0000","last_active":"2024-03-06T07:26:19.001197+0000","last_peered":"2024-03-06T07:26:19.001197+0000","last_clean":"2024-03-06T07:26:19.001197+0000","last_became_active":"2024-03-06T07:26:07.695444+0000","last_became_peered":"2024-03-06T07:26:07.695444+0000","last_unstale":"2024-03-06T07:26:19.001197+0000","last_undegraded":"2024-03-06T07:26:19.001197+0000","last_fullsized":"2024-03-06T07:26:19.001197+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:56.050281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807063+0000","last_change":"2024-03-06T07:26:06.689881+0000","last_active":"2024-03-06T07:26:18.807063+0000","last_peered":"2024-03-06T07:26:18.807063+0000","last_clean":"2024-03-06T07:26:18.807063+0000","last_became_active":"2024-03-06T07:26:06.689604+0000","last_became_peered":"2024-03-06T07:26:06.689604+0000","last_unstale":"2024-03-06T07:26:18.807063+0000","last_undegraded":"2024-03-06T07:26:18.807063+0000","last_fullsized":"2024-03-06T07:26:18.807063+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.752325+0000","last_change":"2024-03-06T07:26:17.751363+0000","last_active":"2024-03-06T07:26:17.752325+0000","last_peered":"2024-03-06T07:26:17.752325+0000","last_clean":"2024-03-06T07:26:17.752325+0000","last_became_active":"2024-03-06T07:26:17.751115+0000","last_became_peered":"2024-03-06T07:26:17.751115+0000","last_unstale":"2024-03-06T07:26:17.752325+0000","last_undegraded":"2024-03-06T07:26:17.752325+0000","last_fullsized":"2024-03-06T07:26:17.752325+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:49:23.743634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.743400+0000","last_change":"2024-03-06T07:26:16.745483+0000","last_active":"2024-03-06T07:26:17.743400+0000","last_peered":"2024-03-06T07:26:17.743400+0000","last_clean":"2024-03-06T07:26:17.743400+0000","last_became_active":"2024-03-06T07:26:16.745181+0000","last_became_peered":"2024-03-06T07:26:16.745181+0000","last_unstale":"2024-03-06T07:26:17.743400+0000","last_undegraded":"2024-03-06T07:26:17.743400+0000","last_fullsized":"2024-03-06T07:26:17.743400+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:01.140243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749144+0000","last_change":"2024-03-06T07:26:07.700732+0000","last_active":"2024-03-06T07:26:19.749144+0000","last_peered":"2024-03-06T07:26:19.749144+0000","last_clean":"2024-03-06T07:26:19.749144+0000","last_became_active":"2024-03-06T07:26:07.700382+0000","last_became_peered":"2024-03-06T07:26:07.700382+0000","last_unstale":"2024-03-06T07:26:19.749144+0000","last_undegraded":"2024-03-06T07:26:19.749144+0000","last_fullsized":"2024-03-06T07:26:19.749144+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:29.340249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001565+0000","last_change":"2024-03-06T07:26:06.693374+0000","last_active":"2024-03-06T07:26:19.001565+0000","last_peered":"2024-03-06T07:26:19.001565+0000","last_clean":"2024-03-06T07:26:19.001565+0000","last_became_active":"2024-03-06T07:26:06.693113+0000","last_became_peered":"2024-03-06T07:26:06.693113+0000","last_unstale":"2024-03-06T07:26:19.001565+0000","last_undegraded":"2024-03-06T07:26:19.001565+0000","last_fullsized":"2024-03-06T07:26:19.001565+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:55.760111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.752802+0000","last_change":"2024-03-06T07:26:17.750146+0000","last_active":"2024-03-06T07:26:17.752802+0000","last_peered":"2024-03-06T07:26:17.752802+0000","last_clean":"2024-03-06T07:26:17.752802+0000","last_became_active":"2024-03-06T07:26:17.749567+0000","last_became_peered":"2024-03-06T07:26:17.749567+0000","last_unstale":"2024-03-06T07:26:17.752802+0000","last_undegraded":"2024-03-06T07:26:17.752802+0000","last_fullsized":"2024-03-06T07:26:17.752802+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:14.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882651+0000","last_change":"2024-03-06T07:26:16.742521+0000","last_active":"2024-03-06T07:26:18.882651+0000","last_peered":"2024-03-06T07:26:18.882651+0000","last_clean":"2024-03-06T07:26:18.882651+0000","last_became_active":"2024-03-06T07:26:16.742257+0000","last_became_peered":"2024-03-06T07:26:16.742257+0000","last_unstale":"2024-03-06T07:26:18.882651+0000","last_undegraded":"2024-03-06T07:26:18.882651+0000","last_fullsized":"2024-03-06T07:26:18.882651+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:58:48.101127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.747248+0000","last_change":"2024-03-06T07:26:07.701237+0000","last_active":"2024-03-06T07:26:18.747248+0000","last_peered":"2024-03-06T07:26:18.747248+0000","last_clean":"2024-03-06T07:26:18.747248+0000","last_became_active":"2024-03-06T07:26:07.700717+0000","last_became_peered":"2024-03-06T07:26:07.700717+0000","last_unstale":"2024-03-06T07:26:18.747248+0000","last_undegraded":"2024-03-06T07:26:18.747248+0000","last_fullsized":"2024-03-06T07:26:18.747248+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:42:26.171117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.746916+0000","last_change":"2024-03-06T07:26:06.695137+0000","last_active":"2024-03-06T07:26:18.746916+0000","last_peered":"2024-03-06T07:26:18.746916+0000","last_clean":"2024-03-06T07:26:18.746916+0000","last_became_active":"2024-03-06T07:26:06.694881+0000","last_became_peered":"2024-03-06T07:26:06.694881+0000","last_unstale":"2024-03-06T07:26:18.746916+0000","last_undegraded":"2024-03-06T07:26:18.746916+0000","last_fullsized":"2024-03-06T07:26:18.746916+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:37.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.749590+0000","last_change":"2024-03-06T07:26:17.746350+0000","last_active":"2024-03-06T07:26:19.749590+0000","last_peered":"2024-03-06T07:26:19.749590+0000","last_clean":"2024-03-06T07:26:19.749590+0000","last_became_active":"2024-03-06T07:26:17.745703+0000","last_became_peered":"2024-03-06T07:26:17.745703+0000","last_unstale":"2024-03-06T07:26:19.749590+0000","last_undegraded":"2024-03-06T07:26:19.749590+0000","last_fullsized":"2024-03-06T07:26:19.749590+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:49:23.743634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.745179+0000","last_change":"2024-03-06T07:26:16.737133+0000","last_active":"2024-03-06T07:26:17.745179+0000","last_peered":"2024-03-06T07:26:17.745179+0000","last_clean":"2024-03-06T07:26:17.745179+0000","last_became_active":"2024-03-06T07:26:16.736855+0000","last_became_peered":"2024-03-06T07:26:16.736855+0000","last_unstale":"2024-03-06T07:26:17.745179+0000","last_undegraded":"2024-03-06T07:26:17.745179+0000","last_fullsized":"2024-03-06T07:26:17.745179+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:57.088499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745335+0000","last_change":"2024-03-06T07:26:07.705151+0000","last_active":"2024-03-06T07:26:18.745335+0000","last_peered":"2024-03-06T07:26:18.745335+0000","last_clean":"2024-03-06T07:26:18.745335+0000","last_became_active":"2024-03-06T07:26:07.704856+0000","last_became_peered":"2024-03-06T07:26:07.704856+0000","last_unstale":"2024-03-06T07:26:18.745335+0000","last_undegraded":"2024-03-06T07:26:18.745335+0000","last_fullsized":"2024-03-06T07:26:18.745335+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:06.267612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806747+0000","last_change":"2024-03-06T07:26:06.684537+0000","last_active":"2024-03-06T07:26:18.806747+0000","last_peered":"2024-03-06T07:26:18.806747+0000","last_clean":"2024-03-06T07:26:18.806747+0000","last_became_active":"2024-03-06T07:26:06.683851+0000","last_became_peered":"2024-03-06T07:26:06.683851+0000","last_unstale":"2024-03-06T07:26:18.806747+0000","last_undegraded":"2024-03-06T07:26:18.806747+0000","last_fullsized":"2024-03-06T07:26:18.806747+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:07.304752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001488+0000","last_change":"2024-03-06T07:26:17.744558+0000","last_active":"2024-03-06T07:26:19.001488+0000","last_peered":"2024-03-06T07:26:19.001488+0000","last_clean":"2024-03-06T07:26:19.001488+0000","last_became_active":"2024-03-06T07:26:17.744304+0000","last_became_peered":"2024-03-06T07:26:17.744304+0000","last_unstale":"2024-03-06T07:26:19.001488+0000","last_undegraded":"2024-03-06T07:26:19.001488+0000","last_fullsized":"2024-03-06T07:26:19.001488+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:07.801361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.764903+0000","last_change":"2024-03-06T07:26:16.744565+0000","last_active":"2024-03-06T07:26:17.764903+0000","last_peered":"2024-03-06T07:26:17.764903+0000","last_clean":"2024-03-06T07:26:17.764903+0000","last_became_active":"2024-03-06T07:26:16.744244+0000","last_became_peered":"2024-03-06T07:26:16.744244+0000","last_unstale":"2024-03-06T07:26:17.764903+0000","last_undegraded":"2024-03-06T07:26:17.764903+0000","last_fullsized":"2024-03-06T07:26:17.764903+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:55.814670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807105+0000","last_change":"2024-03-06T07:26:07.691230+0000","last_active":"2024-03-06T07:26:18.807105+0000","last_peered":"2024-03-06T07:26:18.807105+0000","last_clean":"2024-03-06T07:26:18.807105+0000","last_became_active":"2024-03-06T07:26:07.690936+0000","last_became_peered":"2024-03-06T07:26:07.690936+0000","last_unstale":"2024-03-06T07:26:18.807105+0000","last_undegraded":"2024-03-06T07:26:18.807105+0000","last_fullsized":"2024-03-06T07:26:18.807105+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:09.494792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.744919+0000","last_change":"2024-03-06T07:26:06.692326+0000","last_active":"2024-03-06T07:26:17.744919+0000","last_peered":"2024-03-06T07:26:17.744919+0000","last_clean":"2024-03-06T07:26:17.744919+0000","last_became_active":"2024-03-06T07:26:06.691972+0000","last_became_peered":"2024-03-06T07:26:06.691972+0000","last_unstale":"2024-03-06T07:26:17.744919+0000","last_undegraded":"2024-03-06T07:26:17.744919+0000","last_fullsized":"2024-03-06T07:26:17.744919+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:25.159122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.806870+0000","last_change":"2024-03-06T07:26:17.750249+0000","last_active":"2024-03-06T07:26:18.806870+0000","last_peered":"2024-03-06T07:26:18.806870+0000","last_clean":"2024-03-06T07:26:18.806870+0000","last_became_active":"2024-03-06T07:26:17.750006+0000","last_became_peered":"2024-03-06T07:26:17.750006+0000","last_unstale":"2024-03-06T07:26:18.806870+0000","last_undegraded":"2024-03-06T07:26:18.806870+0000","last_fullsized":"2024-03-06T07:26:18.806870+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:17.553214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.760729+0000","last_change":"2024-03-06T07:26:16.740772+0000","last_active":"2024-03-06T07:26:17.760729+0000","last_peered":"2024-03-06T07:26:17.760729+0000","last_clean":"2024-03-06T07:26:17.760729+0000","last_became_active":"2024-03-06T07:26:16.740115+0000","last_became_peered":"2024-03-06T07:26:16.740115+0000","last_unstale":"2024-03-06T07:26:17.760729+0000","last_undegraded":"2024-03-06T07:26:17.760729+0000","last_fullsized":"2024-03-06T07:26:17.760729+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:33.807152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:19.001780+0000","last_change":"2024-03-06T07:26:07.704775+0000","last_active":"2024-03-06T07:26:19.001780+0000","last_peered":"2024-03-06T07:26:19.001780+0000","last_clean":"2024-03-06T07:26:19.001780+0000","last_became_active":"2024-03-06T07:26:07.704502+0000","last_became_peered":"2024-03-06T07:26:07.704502+0000","last_unstale":"2024-03-06T07:26:19.001780+0000","last_undegraded":"2024-03-06T07:26:19.001780+0000","last_fullsized":"2024-03-06T07:26:19.001780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:04.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.740373+0000","last_change":"2024-03-06T07:26:06.696947+0000","last_active":"2024-03-06T07:26:17.740373+0000","last_peered":"2024-03-06T07:26:17.740373+0000","last_clean":"2024-03-06T07:26:17.740373+0000","last_became_active":"2024-03-06T07:26:06.696715+0000","last_became_peered":"2024-03-06T07:26:06.696715+0000","last_unstale":"2024-03-06T07:26:17.740373+0000","last_undegraded":"2024-03-06T07:26:17.740373+0000","last_fullsized":"2024-03-06T07:26:17.740373+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:40.929460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.882613+0000","last_change":"2024-03-06T07:26:17.751662+0000","last_active":"2024-03-06T07:26:18.882613+0000","last_peered":"2024-03-06T07:26:18.882613+0000","last_clean":"2024-03-06T07:26:18.882613+0000","last_became_active":"2024-03-06T07:26:17.751263+0000","last_became_peered":"2024-03-06T07:26:17.751263+0000","last_unstale":"2024-03-06T07:26:18.882613+0000","last_undegraded":"2024-03-06T07:26:18.882613+0000","last_fullsized":"2024-03-06T07:26:18.882613+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:16.717041+0000","last_clean_scrub_stamp":"2024-03-06T07:26:16.717041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:13:00.763534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:17.740547+0000","last_change":"2024-03-06T07:26:16.740573+0000","last_active":"2024-03-06T07:26:17.740547+0000","last_peered":"2024-03-06T07:26:17.740547+0000","last_clean":"2024-03-06T07:26:17.740547+0000","last_became_active":"2024-03-06T07:26:16.740049+0000","last_became_peered":"2024-03-06T07:26:16.740049+0000","last_unstale":"2024-03-06T07:26:17.740547+0000","last_undegraded":"2024-03-06T07:26:17.740547+0000","last_fullsized":"2024-03-06T07:26:17.740547+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:15.711617+0000","last_clean_scrub_stamp":"2024-03-06T07:26:15.711617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:42.635159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745082+0000","last_change":"2024-03-06T07:26:07.699110+0000","last_active":"2024-03-06T07:26:18.745082+0000","last_peered":"2024-03-06T07:26:18.745082+0000","last_clean":"2024-03-06T07:26:18.745082+0000","last_became_active":"2024-03-06T07:26:07.698799+0000","last_became_peered":"2024-03-06T07:26:07.698799+0000","last_unstale":"2024-03-06T07:26:18.745082+0000","last_undegraded":"2024-03-06T07:26:18.745082+0000","last_fullsized":"2024-03-06T07:26:18.745082+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:06.672007+0000","last_clean_scrub_stamp":"2024-03-06T07:26:06.672007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:56.000904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.807190+0000","last_change":"2024-03-06T07:26:06.691133+0000","last_active":"2024-03-06T07:26:18.807190+0000","last_peered":"2024-03-06T07:26:18.807190+0000","last_clean":"2024-03-06T07:26:18.807190+0000","last_became_active":"2024-03-06T07:26:06.690687+0000","last_became_peered":"2024-03-06T07:26:06.690687+0000","last_unstale":"2024-03-06T07:26:18.807190+0000","last_undegraded":"2024-03-06T07:26:18.807190+0000","last_fullsized":"2024-03-06T07:26:18.807190+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:05.666569+0000","last_clean_scrub_stamp":"2024-03-06T07:26:05.666569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:08.352394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":424,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-03-06T07:26:18.745347+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:26:18.745347+0000","last_peered":"2024-03-06T07:26:18.745347+0000","last_clean":"2024-03-06T07:26:18.745347+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:26:18.745347+0000","last_undegraded":"2024-03-06T07:26:18.745347+0000","last_fullsized":"2024-03-06T07:26:18.745347+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2015232,"data_stored":1539268,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":948970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1961984,"data_stored":943109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1982464,"data_stored":951310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1978368,"data_stored":951276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":949482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1949696,"data_stored":942963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2027520,"data_stored":1532551,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:22.135 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:26:22.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.267+0000 7ff3ddea4640 1 Processor -- start 2024-03-06T07:26:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.267+0000 7ff3ddea4640 1 -- start start 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3ddea4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3ddea4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d80c5cc0 con 0x7ff3d81586f0 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3ddea4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d81591c0 con 0x7ff3d80c5860 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3ddea4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d806df40 con 0x7ff3d80c5480 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff3d81586f0 0x7ff3d815f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60708/0 (socket says 172.21.15.26:60708) 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d7fff640 1 -- 172.21.15.26:0/1038927541 learned_addr learned my addr 172.21.15.26:0/1038927541 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d77fe640 1 --2- 172.21.15.26:0/1038927541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d67fc640 1 -- 172.21.15.26:0/1038927541 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875592347 0 0) 0x7ff3d80c5cc0 con 0x7ff3d81586f0 2024-03-06T07:26:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d67fc640 1 -- 172.21.15.26:0/1038927541 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c4003520 con 0x7ff3d81586f0 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d67fc640 1 -- 172.21.15.26:0/1038927541 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775286678 0 0) 0x7ff3d81591c0 con 0x7ff3d80c5860 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d67fc640 1 -- 172.21.15.26:0/1038927541 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d80c5cc0 con 0x7ff3d80c5860 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d77fe640 1 -- 172.21.15.26:0/1038927541 >> v1:172.21.15.117:6789/0 conn(0x7ff3d80c5860 legacy=0x7ff3d806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d77fe640 1 -- 172.21.15.26:0/1038927541 >> v1:172.21.15.26:6789/0 conn(0x7ff3d81586f0 legacy=0x7ff3d815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.268+0000 7ff3d77fe640 1 -- 172.21.15.26:0/1038927541 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d81594a0 con 0x7ff3d80c5480 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.269+0000 7ff3d77fe640 1 --2- 172.21.15.26:0/1038927541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d8155ba0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80097d0 tx=0x7ff3c802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb52ce8025d25f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.269+0000 7ff3d67fc640 1 -- 172.21.15.26:0/1038927541 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8035020 con 0x7ff3d80c5480 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.269+0000 7ff3d67fc640 1 -- 172.21.15.26:0/1038927541 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c8031800 con 0x7ff3d80c5480 2024-03-06T07:26:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.269+0000 7ff3d67fc640 1 -- 172.21.15.26:0/1038927541 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8031bd0 con 0x7ff3d80c5480 2024-03-06T07:26:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.270+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1038927541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 msgr2=0x7ff3d8155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.270+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1038927541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d8155ba0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80097d0 tx=0x7ff3c802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.270+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1038927541 shutdown_connections 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.270+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1038927541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d8155ba0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.270+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1038927541 >> 172.21.15.26:0/1038927541 conn(0x7ff3d806b1a0 msgr2=0x7ff3d806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.270+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1038927541 shutdown_connections 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.270+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1038927541 wait complete. 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.271+0000 7ff3ddea4640 1 Processor -- start 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.271+0000 7ff3ddea4640 1 -- start start 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3ddea4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d81e2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3ddea4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3d80c5860 0x7ff3d8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3ddea4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d81586f0 0x7ff3d8157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3ddea4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d8155680 con 0x7ff3d80c5860 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3ddea4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d806fdb0 con 0x7ff3d81586f0 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3ddea4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d8152a20 con 0x7ff3d80c5480 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3d80c5860 0x7ff3d8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d81e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3d80c5860 0x7ff3d8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58020/0 (socket says 172.21.15.26:58020) 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 -- 172.21.15.26:0/1763012478 learned_addr learned my addr 172.21.15.26:0/1763012478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d7fff640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d81586f0 0x7ff3d8157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 -- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 msgr2=0x7ff3d81e2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d81e2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 -- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d81586f0 msgr2=0x7ff3d8157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d81586f0 0x7ff3d8157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d6ffd640 1 -- 172.21.15.26:0/1763012478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d81ee760 con 0x7ff3d80c5860 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.272+0000 7ff3d7fff640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d81586f0 0x7ff3d8157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.273+0000 7ff3d6ffd640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3d80c5860 0x7ff3d8156fd0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000c500 tx=0x7ff3c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.273+0000 7ff3cffff640 1 -- 172.21.15.26:0/1763012478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0017070 con 0x7ff3d80c5860 2024-03-06T07:26:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.273+0000 7ff3cffff640 1 -- 172.21.15.26:0/1763012478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c0005310 con 0x7ff3d80c5860 2024-03-06T07:26:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.273+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d81eea00 con 0x7ff3d80c5860 2024-03-06T07:26:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.273+0000 7ff3cffff640 1 -- 172.21.15.26:0/1763012478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0003830 con 0x7ff3d80c5860 2024-03-06T07:26:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.273+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d81564f0 con 0x7ff3d80c5860 2024-03-06T07:26:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.274+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d804f330 con 0x7ff3d80c5860 2024-03-06T07:26:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.274+0000 7ff3cffff640 1 -- 172.21.15.26:0/1763012478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3c001a070 con 0x7ff3d80c5860 2024-03-06T07:26:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.275+0000 7ff3cffff640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3b806b4d0 0x7ff3b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.275+0000 7ff3cffff640 1 -- 172.21.15.26:0/1763012478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7ff3c00899a0 con 0x7ff3d80c5860 2024-03-06T07:26:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.275+0000 7ff3d77fe640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3b806b4d0 0x7ff3b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.275+0000 7ff3d77fe640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3b806b4d0 0x7ff3b806d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8036000 tx=0x7ff3c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.280+0000 7ff3cffff640 1 -- 172.21.15.26:0/1763012478 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff3c0057d00 con 0x7ff3d80c5860 2024-03-06T07:26:22.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.497+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3d806e0c0 con 0x7ff3d80c5860 2024-03-06T07:26:22.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.497+0000 7ff3cffff640 1 -- 172.21.15.26:0/1763012478 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+6272 (secure 0 0 0) 0x7ff3c005bda0 con 0x7ff3d80c5860 2024-03-06T07:26:22.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:22.500 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":143,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:10.089399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8793":{"gid":8793,"name":"a","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6833/3029467671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3029467671},{"type":"v1","addr":"172.21.15.26:6833","nonce":3029467671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8793,"qdb_cluster":[8793]},"id":11},{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:20.123577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:22.500 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 143 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3b806b4d0 msgr2=0x7ff3b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3b806b4d0 0x7ff3b806d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8036000 tx=0x7ff3c8008030 comp rx=0 tx=0).stop 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3d80c5860 msgr2=0x7ff3d8156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3d80c5860 0x7ff3d8156fd0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000c500 tx=0x7ff3c000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 shutdown_connections 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff3b806b4d0 0x7ff3b806d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d81586f0 0x7ff3d8157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3d80c5860 0x7ff3d8156fd0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 --2- 172.21.15.26:0/1763012478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3d80c5480 0x7ff3d81e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 >> 172.21.15.26:0/1763012478 conn(0x7ff3d806b1a0 msgr2=0x7ff3d80c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 shutdown_connections 2024-03-06T07:26:22.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.499+0000 7ff3ddea4640 1 -- 172.21.15.26:0/1763012478 wait complete. 2024-03-06T07:26:22.513 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 11, "epoch": 143, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 11, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:07.071721+0000", "damaged": [], "data_pools": [ 24 ], "enabled": true, "epoch": 138, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8793": { "addr": "172.21.15.26:6833/3029467671", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3029467671, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3029467671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8793, "incarnation": 135, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 23, "modified": "2024-03-06T07:26:10.089399+0000", "qdb_cluster": [ 8793 ], "qdb_leader": 8793, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8793 } } }, { "id": 12, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:17.118167+0000", "damaged": [], "data_pools": [ 26 ], "enabled": true, "epoch": 143, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs2", "in": [ 0 ], "info": { "gid_8293": { "addr": "172.21.15.117:6836/296681906", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 296681906, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 296681906, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8293, "incarnation": 139, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 25, "modified": "2024-03-06T07:26:20.123577+0000", "qdb_cluster": [ 8293 ], "qdb_leader": 8293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8293 } } } ], "standbys": [ { "addr": "172.21.15.26:6835/3463955148", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 3463955148, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 3463955148, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 134, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7166, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/677194061", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 677194061, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 677194061, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 134, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7178, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:26:22.513 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:26:22.513 INFO:tasks.cephfs.test_failover:current = [{'gid': 8793, 'name': 'a', 'rank': 0, 'incarnation': 135, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6833/3029467671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3029467671}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3029467671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7178, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/677194061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 677194061}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 677194061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 134}, {'gid': 7166, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/3463955148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3463955148}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3463955148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 134}, {'gid': 8293, 'name': 'd', 'rank': 0, 'incarnation': 139, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6836/296681906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 296681906}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 296681906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:26:22.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.613+0000 7fbd35e4e640 1 Processor -- start 2024-03-06T07:26:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.613+0000 7fbd35e4e640 1 -- start start 2024-03-06T07:26:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd35e4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd301527b0 0x7fbd30159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd35e4e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd300c88b0 0x7fbd300c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd35e4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd300c7c60 con 0x7fbd300c7180 2024-03-06T07:26:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd35e4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd30159640 con 0x7fbd301527b0 2024-03-06T07:26:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd35e4e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd300c8060 con 0x7fbd300c88b0 2024-03-06T07:26:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd301527b0 0x7fbd30159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbd300c7180 0x7fbd30155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60712/0 (socket says 172.21.15.26:60712) 2024-03-06T07:26:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2effd640 1 -- 172.21.15.26:0/260839756 learned_addr learned my addr 172.21.15.26:0/260839756 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2ffff640 1 --2- 172.21.15.26:0/260839756 >> v2:172.21.15.117:3300/0 conn(0x7fbd300c88b0 0x7fbd300c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2f7fe640 1 -- 172.21.15.26:0/260839756 >> v2:172.21.15.117:3300/0 conn(0x7fbd300c88b0 msgr2=0x7fbd300c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2f7fe640 1 --2- 172.21.15.26:0/260839756 >> v2:172.21.15.117:3300/0 conn(0x7fbd300c88b0 0x7fbd300c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2f7fe640 1 -- 172.21.15.26:0/260839756 >> v1:172.21.15.26:6789/0 conn(0x7fbd300c7180 legacy=0x7fbd30155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2f7fe640 1 -- 172.21.15.26:0/260839756 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300c76a0 con 0x7fbd301527b0 2024-03-06T07:26:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.614+0000 7fbd2e7fc640 1 -- 172.21.15.26:0/260839756 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126631622 0 0) 0x7fbd24001110 con 0x7fbd300c7180 2024-03-06T07:26:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd2f7fe640 1 --2- 172.21.15.26:0/260839756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd30159100 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbd300ad600 tx=0x7fbd2002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77d3bf33fe40a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd2e7fc640 1 -- 172.21.15.26:0/260839756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200315a0 con 0x7fbd301527b0 2024-03-06T07:26:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd2e7fc640 1 -- 172.21.15.26:0/260839756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd20037070 con 0x7fbd301527b0 2024-03-06T07:26:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd2e7fc640 1 -- 172.21.15.26:0/260839756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20031960 con 0x7fbd301527b0 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 -- 172.21.15.26:0/260839756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 msgr2=0x7fbd30159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/260839756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd30159100 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbd300ad600 tx=0x7fbd2002b840 comp rx=0 tx=0).stop 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 -- 172.21.15.26:0/260839756 shutdown_connections 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/260839756 >> v2:172.21.15.117:3300/0 conn(0x7fbd300c88b0 0x7fbd300c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/260839756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd30159100 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 -- 172.21.15.26:0/260839756 >> 172.21.15.26:0/260839756 conn(0x7fbd300ac560 msgr2=0x7fbd300ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 -- 172.21.15.26:0/260839756 shutdown_connections 2024-03-06T07:26:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.616+0000 7fbd35e4e640 1 -- 172.21.15.26:0/260839756 wait complete. 2024-03-06T07:26:22.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.618+0000 7fbd35e4e640 1 Processor -- start 2024-03-06T07:26:22.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.618+0000 7fbd35e4e640 1 -- start start 2024-03-06T07:26:22.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd35e4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd300c7180 0x7fbd3014bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd35e4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd300c88b0 0x7fbd30150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd35e4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd300c3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd35e4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd30155600 con 0x7fbd301527b0 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd35e4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd30155320 con 0x7fbd300c88b0 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd35e4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd30152b90 con 0x7fbd300c7180 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd300c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd300c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58030/0 (socket says 172.21.15.26:58030) 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd2ffff640 1 -- 172.21.15.26:0/2781376465 learned_addr learned my addr 172.21.15.26:0/2781376465 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd2f7fe640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd300c7180 0x7fbd3014bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.619+0000 7fbd2effd640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd300c88b0 0x7fbd30150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd2ffff640 1 -- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd300c7180 msgr2=0x7fbd3014bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd2ffff640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd300c7180 0x7fbd3014bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd2ffff640 1 -- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd300c88b0 msgr2=0x7fbd30150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd2ffff640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd300c88b0 0x7fbd30150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd2ffff640 1 -- 172.21.15.26:0/2781376465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30157d90 con 0x7fbd301527b0 2024-03-06T07:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd2ffff640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd300c3140 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009dd0 tx=0x7fbd2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd17fff640 1 -- 172.21.15.26:0/2781376465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28018020 con 0x7fbd301527b0 2024-03-06T07:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd17fff640 1 -- 172.21.15.26:0/2781376465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28014480 con 0x7fbd301527b0 2024-03-06T07:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd301eea90 con 0x7fbd301527b0 2024-03-06T07:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd17fff640 1 -- 172.21.15.26:0/2781376465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28014870 con 0x7fbd301527b0 2024-03-06T07:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.620+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd30151530 con 0x7fbd301527b0 2024-03-06T07:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.621+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3004f330 con 0x7fbd301527b0 2024-03-06T07:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.622+0000 7fbd17fff640 1 -- 172.21.15.26:0/2781376465 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd28005d10 con 0x7fbd301527b0 2024-03-06T07:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.622+0000 7fbd17fff640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd1c06b500 0x7fbd1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.622+0000 7fbd17fff640 1 -- 172.21.15.26:0/2781376465 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fbd2808afa0 con 0x7fbd301527b0 2024-03-06T07:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.622+0000 7fbd2f7fe640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd1c06b500 0x7fbd1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.623+0000 7fbd2f7fe640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd1c06b500 0x7fbd1c06d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbd20036000 tx=0x7fbd2003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.626+0000 7fbd17fff640 1 -- 172.21.15.26:0/2781376465 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd28022070 con 0x7fbd301527b0 2024-03-06T07:26:22.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.850+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd3006ce00 con 0x7fbd301527b0 2024-03-06T07:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.850+0000 7fbd17fff640 1 -- 172.21.15.26:0/2781376465 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+6272 (secure 0 0 0) 0x7fbd28010040 con 0x7fbd301527b0 2024-03-06T07:26:22.853 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:22.853 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":143,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:10.089399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8793":{"gid":8793,"name":"a","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6833/3029467671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3029467671},{"type":"v1","addr":"172.21.15.26:6833","nonce":3029467671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8793,"qdb_cluster":[8793]},"id":11},{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:20.123577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 143 2024-03-06T07:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd1c06b500 msgr2=0x7fbd1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd1c06b500 0x7fbd1c06d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbd20036000 tx=0x7fbd2003d040 comp rx=0 tx=0).stop 2024-03-06T07:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 msgr2=0x7fbd300c3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd300c3140 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009dd0 tx=0x7fbd2800c490 comp rx=0 tx=0).stop 2024-03-06T07:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 shutdown_connections 2024-03-06T07:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd1c06b500 0x7fbd1c06d9c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd301527b0 0x7fbd300c3140 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd300c88b0 0x7fbd30150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 --2- 172.21.15.26:0/2781376465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd300c7180 0x7fbd3014bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 >> 172.21.15.26:0/2781376465 conn(0x7fbd300ac560 msgr2=0x7fbd3014ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 shutdown_connections 2024-03-06T07:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.852+0000 7fbd35e4e640 1 -- 172.21.15.26:0/2781376465 wait complete. 2024-03-06T07:26:22.866 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 11, "epoch": 143, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 11, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:07.071721+0000", "damaged": [], "data_pools": [ 24 ], "enabled": true, "epoch": 138, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8793": { "addr": "172.21.15.26:6833/3029467671", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 3029467671, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 3029467671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8793, "incarnation": 135, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 23, "modified": "2024-03-06T07:26:10.089399+0000", "qdb_cluster": [ 8793 ], "qdb_leader": 8793, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8793 } } }, { "id": 12, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:17.118167+0000", "damaged": [], "data_pools": [ 26 ], "enabled": true, "epoch": 143, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs2", "in": [ 0 ], "info": { "gid_8293": { "addr": "172.21.15.117:6836/296681906", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 296681906, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 296681906, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8293, "incarnation": 139, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 25, "modified": "2024-03-06T07:26:20.123577+0000", "qdb_cluster": [ 8293 ], "qdb_leader": 8293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8293 } } } ], "standbys": [ { "addr": "172.21.15.26:6835/3463955148", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 3463955148, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 3463955148, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 134, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7166, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6837/677194061", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 677194061, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 677194061, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 134, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7178, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:26:22.867 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:26:22.867 INFO:tasks.cephfs.test_failover:current = [{'gid': 8793, 'name': 'a', 'rank': 0, 'incarnation': 135, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6833/3029467671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3029467671}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 3029467671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7178, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.117:6837/677194061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 677194061}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 677194061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 134}, {'gid': 7166, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6835/3463955148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3463955148}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3463955148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 134}, {'gid': 8293, 'name': 'd', 'rank': 0, 'incarnation': 139, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6836/296681906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 296681906}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 296681906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:26:22.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.c mds_join_fs cephfs2 2024-03-06T07:26:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.970+0000 7f4907eed640 1 Processor -- start 2024-03-06T07:26:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.970+0000 7f4907eed640 1 -- start start 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.970+0000 7f4907eed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f4900156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.970+0000 7f4907eed640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49000c7f60 0x7f4900072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4907eed640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4900073580 con 0x7f4900072a50 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4907eed640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4900070b20 con 0x7f49000c7f60 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4907eed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000701f0 con 0x7f49001505f0 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4905488640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49000c7f60 0x7f4900072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4905c89640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f4900156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f490648a640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4900072a50 0x7f49000c6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44652/0 (socket says 172.21.15.26:44652) 2024-03-06T07:26:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f490648a640 1 -- 172.21.15.26:0/3545079423 learned_addr learned my addr 172.21.15.26:0/3545079423 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4905488640 1 -- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 msgr2=0x7f4900156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4905488640 1 --2- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f4900156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4905488640 1 -- 172.21.15.26:0/3545079423 >> v1:172.21.15.117:6789/0 conn(0x7f4900072a50 legacy=0x7f49000c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4905488640 1 -- 172.21.15.26:0/3545079423 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000704d0 con 0x7f49000c7f60 2024-03-06T07:26:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.971+0000 7f4905c89640 1 --2- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f4900156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.972+0000 7f4905488640 1 --2- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 0x7f4900072510 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f48f00097a0 tx=0x7f48f002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d962ed617614f228 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.972+0000 7f4904c87640 1 -- 172.21.15.26:0/3545079423 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0035020 con 0x7f49000c7f60 2024-03-06T07:26:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.972+0000 7f4904c87640 1 -- 172.21.15.26:0/3545079423 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0031810 con 0x7f49000c7f60 2024-03-06T07:26:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.972+0000 7f4904c87640 1 -- 172.21.15.26:0/3545079423 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0031be0 con 0x7f49000c7f60 2024-03-06T07:26:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.972+0000 7f4907eed640 1 -- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 msgr2=0x7f4900072510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.972+0000 7f4907eed640 1 --2- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 0x7f4900072510 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f48f00097a0 tx=0x7f48f002b8b0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.973+0000 7f4907eed640 1 -- 172.21.15.26:0/3545079423 shutdown_connections 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.973+0000 7f4907eed640 1 --2- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 0x7f4900072510 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.973+0000 7f4907eed640 1 --2- 172.21.15.26:0/3545079423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f4900156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.973+0000 7f4907eed640 1 -- 172.21.15.26:0/3545079423 >> 172.21.15.26:0/3545079423 conn(0x7f49000aa5c0 msgr2=0x7f49000aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.973+0000 7f4907eed640 1 -- 172.21.15.26:0/3545079423 shutdown_connections 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.973+0000 7f4907eed640 1 -- 172.21.15.26:0/3545079423 wait complete. 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 Processor -- start 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 -- start start 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4900072a50 0x7f49000c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 0x7f490006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f49001ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001509d0 con 0x7f4900072a50 2024-03-06T07:26:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490006fec0 con 0x7f49000c7f60 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4907eed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4900150b50 con 0x7f49001505f0 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4905c89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4900072a50 0x7f49000c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4905488640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 0x7f490006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4905c89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4900072a50 0x7f49000c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58034/0 (socket says 172.21.15.26:58034) 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.974+0000 7f4905c89640 1 -- 172.21.15.26:0/3436996469 learned_addr learned my addr 172.21.15.26:0/3436996469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f490648a640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f49001ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4905c89640 1 -- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 msgr2=0x7f49001ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4905c89640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f49001ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4905c89640 1 -- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 msgr2=0x7f490006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4905c89640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 0x7f490006c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4905c89640 1 -- 172.21.15.26:0/3436996469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49001eb6b0 con 0x7f4900072a50 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4905c89640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4900072a50 0x7f49000c1eb0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00e960 tx=0x7f48fc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f48f67fc640 1 -- 172.21.15.26:0/3436996469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc017070 con 0x7f4900072a50 2024-03-06T07:26:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f48f67fc640 1 -- 172.21.15.26:0/3436996469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc00c9a0 con 0x7f4900072a50 2024-03-06T07:26:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f48f67fc640 1 -- 172.21.15.26:0/3436996469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00cd70 con 0x7f4900072a50 2024-03-06T07:26:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49001eb990 con 0x7f4900072a50 2024-03-06T07:26:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.975+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4900155600 con 0x7f4900072a50 2024-03-06T07:26:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.976+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490004f330 con 0x7f4900072a50 2024-03-06T07:26:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.977+0000 7f48f67fc640 1 -- 172.21.15.26:0/3436996469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48fc01a070 con 0x7f4900072a50 2024-03-06T07:26:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.977+0000 7f48f67fc640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e406b530 0x7f48e406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.977+0000 7f48f67fc640 1 -- 172.21.15.26:0/3436996469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f48fc08a840 con 0x7f4900072a50 2024-03-06T07:26:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.977+0000 7f4905488640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e406b530 0x7f48e406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.978+0000 7f4905488640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e406b530 0x7f48e406d9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f48f0036000 tx=0x7f48f003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.984 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:26:22.982+0000 7f254b9a7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:22.986+0000 7f48f67fc640 1 -- 172.21.15.26:0/3436996469 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f48fc058b80 con 0x7f4900072a50 2024-03-06T07:26:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.168+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f49000a95b0 con 0x7f4900072a50 2024-03-06T07:26:23.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.171+0000 7f48f67fc640 1 -- 172.21.15.26:0/3436996469 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v52)=0 v52) v1 ==== 117+0+0 (secure 0 0 0) 0x7f48fc05cc20 con 0x7f4900072a50 2024-03-06T07:26:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e406b530 msgr2=0x7f48e406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e406b530 0x7f48e406d9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f48f0036000 tx=0x7f48f003c040 comp rx=0 tx=0).stop 2024-03-06T07:26:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4900072a50 msgr2=0x7f49000c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4900072a50 0x7f49000c1eb0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00e960 tx=0x7f48fc00ee20 comp rx=0 tx=0).stop 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 shutdown_connections 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f48e406b530 0x7f48e406d9f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49001505f0 0x7f49001ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49000c7f60 0x7f490006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 --2- 172.21.15.26:0/3436996469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4900072a50 0x7f49000c1eb0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 >> 172.21.15.26:0/3436996469 conn(0x7f49000aa5c0 msgr2=0x7f49000c3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 shutdown_connections 2024-03-06T07:26:23.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.173+0000 7f4907eed640 1 -- 172.21.15.26:0/3436996469 wait complete. 2024-03-06T07:26:23.186 INFO:tasks.cephfs.test_failover:assigning affinity to cephfs2 for active mds (mds.d) 2024-03-06T07:26:23.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs2 2024-03-06T07:26:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.289+0000 7f97b31d0640 1 Processor -- start 2024-03-06T07:26:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.289+0000 7f97b31d0640 1 -- start start 2024-03-06T07:26:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b31d0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97ac0af720 0x7f97ac14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b31d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97ac0ca1a0 0x7f97ac14f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b31d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0c5310 0x7f97ac0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b31d0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0ad900 con 0x7f97ac0ca1a0 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b31d0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac155d20 con 0x7f97ac0af720 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b31d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac155fc0 con 0x7f97ac0c5310 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97ac0ca1a0 0x7f97ac14f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97ac0ca1a0 0x7f97ac14f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58036/0 (socket says 172.21.15.26:58036) 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 -- 172.21.15.26:0/863266209 learned_addr learned my addr 172.21.15.26:0/863266209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b0f6c640 1 --2- 172.21.15.26:0/863266209 >> v2:172.21.15.117:3300/0 conn(0x7f97ac0af720 0x7f97ac14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97b176d640 1 --2- 172.21.15.26:0/863266209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0c5310 0x7f97ac0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 -- 172.21.15.26:0/863266209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0c5310 msgr2=0x7f97ac0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 --2- 172.21.15.26:0/863266209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0c5310 0x7f97ac0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 -- 172.21.15.26:0/863266209 >> v2:172.21.15.117:3300/0 conn(0x7f97ac0af720 msgr2=0x7f97ac14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 --2- 172.21.15.26:0/863266209 >> v2:172.21.15.117:3300/0 conn(0x7f97ac0af720 0x7f97ac14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.290+0000 7f97abfff640 1 -- 172.21.15.26:0/863266209 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0adb50 con 0x7f97ac0ca1a0 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.291+0000 7f97abfff640 1 --2- 172.21.15.26:0/863266209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0ca1a0 0x7f97ac14f2d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f97ac08c9b0 tx=0x7f97a002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937378ee3645633c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.291+0000 7f97ab7fe640 1 -- 172.21.15.26:0/863266209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0038070 con 0x7f97ac0ca1a0 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.291+0000 7f97ab7fe640 1 -- 172.21.15.26:0/863266209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a002f050 con 0x7f97ac0ca1a0 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.291+0000 7f97ab7fe640 1 -- 172.21.15.26:0/863266209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0031970 con 0x7f97ac0ca1a0 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 -- 172.21.15.26:0/863266209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0ca1a0 msgr2=0x7f97ac14f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 --2- 172.21.15.26:0/863266209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0ca1a0 0x7f97ac14f2d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f97ac08c9b0 tx=0x7f97a002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 -- 172.21.15.26:0/863266209 shutdown_connections 2024-03-06T07:26:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 --2- 172.21.15.26:0/863266209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0c5310 0x7f97ac0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 --2- 172.21.15.26:0/863266209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0ca1a0 0x7f97ac14f2d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 --2- 172.21.15.26:0/863266209 >> v2:172.21.15.117:3300/0 conn(0x7f97ac0af720 0x7f97ac14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 -- 172.21.15.26:0/863266209 >> 172.21.15.26:0/863266209 conn(0x7f97ac0ac7e0 msgr2=0x7f97ac0acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 -- 172.21.15.26:0/863266209 shutdown_connections 2024-03-06T07:26:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.292+0000 7f97b31d0640 1 -- 172.21.15.26:0/863266209 wait complete. 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.293+0000 7f97b31d0640 1 Processor -- start 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.293+0000 7f97b31d0640 1 -- start start 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b31d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0af720 0x7f97ac144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b31d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0c5310 0x7f97ac06c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b31d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac0ca1a0 0x7f97ac1eb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b31d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0adfb0 con 0x7f97ac0c5310 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b31d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0ade30 con 0x7f97ac0ca1a0 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b31d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0afb00 con 0x7f97ac0af720 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0c5310 0x7f97ac06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b0f6c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0af720 0x7f97ac144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0c5310 0x7f97ac06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58046/0 (socket says 172.21.15.26:58046) 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 -- 172.21.15.26:0/887029679 learned_addr learned my addr 172.21.15.26:0/887029679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b176d640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac0ca1a0 0x7f97ac1eb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 -- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0af720 msgr2=0x7f97ac144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0af720 0x7f97ac144660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 -- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac0ca1a0 msgr2=0x7f97ac1eb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac0ca1a0 0x7f97ac1eb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97abfff640 1 -- 172.21.15.26:0/887029679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac159630 con 0x7f97ac0c5310 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.294+0000 7f97b0f6c640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0af720 0x7f97ac144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.295+0000 7f97abfff640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0c5310 0x7f97ac06c780 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f97a002b830 tx=0x7f97a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.295+0000 7f97a97fa640 1 -- 172.21.15.26:0/887029679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0042070 con 0x7f97ac0c5310 2024-03-06T07:26:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.295+0000 7f97a97fa640 1 -- 172.21.15.26:0/887029679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a003d680 con 0x7f97ac0c5310 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.295+0000 7f97a97fa640 1 -- 172.21.15.26:0/887029679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0038040 con 0x7f97ac0c5310 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.295+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac156a00 con 0x7f97ac0c5310 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.295+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac159350 con 0x7f97ac0c5310 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.296+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac1eaf30 con 0x7f97ac0c5310 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.296+0000 7f97a97fa640 1 -- 172.21.15.26:0/887029679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97a0045070 con 0x7f97ac0c5310 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.297+0000 7f97a97fa640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f979806b4d0 0x7f979806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.297+0000 7f97a97fa640 1 -- 172.21.15.26:0/887029679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f97a00ac100 con 0x7f97ac0c5310 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.297+0000 7f97b0f6c640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f979806b4d0 0x7f979806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.297+0000 7f97b0f6c640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f979806b4d0 0x7f979806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009fd0 tx=0x7f97a400d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.301+0000 7f97a97fa640 1 -- 172.21.15.26:0/887029679 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f97ac1eaf30 con 0x7f97ac0c5310 2024-03-06T07:26:23.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.482+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f97ac156d00 con 0x7f97ac0c5310 2024-03-06T07:26:23.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.485+0000 7f97a97fa640 1 -- 172.21.15.26:0/887029679 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v53)=0 v53) v1 ==== 117+0+0 (secure 0 0 0) 0x7f97a007a600 con 0x7f97ac0c5310 2024-03-06T07:26:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f979806b4d0 msgr2=0x7f979806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f979806b4d0 0x7f979806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009fd0 tx=0x7f97a400d040 comp rx=0 tx=0).stop 2024-03-06T07:26:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0c5310 msgr2=0x7f97ac06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0c5310 0x7f97ac06c780 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f97a002b830 tx=0x7f97a00046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 shutdown_connections 2024-03-06T07:26:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f979806b4d0 0x7f979806d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac0ca1a0 0x7f97ac1eb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac0c5310 0x7f97ac06c780 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 --2- 172.21.15.26:0/887029679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0af720 0x7f97ac144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 >> 172.21.15.26:0/887029679 conn(0x7f97ac0ac7e0 msgr2=0x7f97ac0c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 shutdown_connections 2024-03-06T07:26:23.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.487+0000 7f97b31d0640 1 -- 172.21.15.26:0/887029679 wait complete. 2024-03-06T07:26:23.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 11:0 2024-03-06T07:26:23.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.600+0000 7fd1861d7640 1 Processor -- start 2024-03-06T07:26:23.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.600+0000 7fd1861d7640 1 -- start start 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.601+0000 7fd1861d7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd1800c69f0 0x7fd1800c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.601+0000 7fd1861d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1800c8950 0x7fd1800acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.601+0000 7fd1861d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd180155130 con 0x7fd1800ad2b0 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd1861d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd180155440 con 0x7fd1800c8950 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd1861d7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1800c5d50 con 0x7fd1800c69f0 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1800c8950 0x7fd1800acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd1800ad2b0 0x7fd1801558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60724/0 (socket says 172.21.15.26:60724) 2024-03-06T07:26:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17ffff640 1 -- 172.21.15.26:0/1292878501 learned_addr learned my addr 172.21.15.26:0/1292878501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1800c8950 0x7fd1800acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58058/0 (socket says 172.21.15.26:58058) 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17f7fe640 1 --2- 172.21.15.26:0/1292878501 >> v2:172.21.15.117:3300/0 conn(0x7fd1800c69f0 0x7fd1800c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17effd640 1 -- 172.21.15.26:0/1292878501 >> v2:172.21.15.117:3300/0 conn(0x7fd1800c69f0 msgr2=0x7fd1800c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17effd640 1 --2- 172.21.15.26:0/1292878501 >> v2:172.21.15.117:3300/0 conn(0x7fd1800c69f0 0x7fd1800c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17effd640 1 -- 172.21.15.26:0/1292878501 >> v1:172.21.15.26:6789/0 conn(0x7fd1800ad2b0 legacy=0x7fd1801558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17e7fc640 1 -- 172.21.15.26:0/1292878501 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071422612 0 0) 0x7fd178001490 con 0x7fd1800ad2b0 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.602+0000 7fd17effd640 1 -- 172.21.15.26:0/1292878501 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800c6030 con 0x7fd1800c8950 2024-03-06T07:26:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.603+0000 7fd17effd640 1 --2- 172.21.15.26:0/1292878501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1800acd70 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd1800aa430 tx=0x7fd17402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb72ecd9fe8042a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.603+0000 7fd17e7fc640 1 -- 172.21.15.26:0/1292878501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740315a0 con 0x7fd1800c8950 2024-03-06T07:26:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.603+0000 7fd17e7fc640 1 -- 172.21.15.26:0/1292878501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174037070 con 0x7fd1800c8950 2024-03-06T07:26:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.603+0000 7fd17e7fc640 1 -- 172.21.15.26:0/1292878501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174031960 con 0x7fd1800c8950 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.603+0000 7fd1861d7640 1 -- 172.21.15.26:0/1292878501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 msgr2=0x7fd1800acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.603+0000 7fd1861d7640 1 --2- 172.21.15.26:0/1292878501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1800acd70 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd1800aa430 tx=0x7fd17402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.604+0000 7fd1861d7640 1 -- 172.21.15.26:0/1292878501 shutdown_connections 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.604+0000 7fd1861d7640 1 --2- 172.21.15.26:0/1292878501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1800acd70 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.604+0000 7fd1861d7640 1 --2- 172.21.15.26:0/1292878501 >> v2:172.21.15.117:3300/0 conn(0x7fd1800c69f0 0x7fd1800c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.604+0000 7fd1861d7640 1 -- 172.21.15.26:0/1292878501 >> 172.21.15.26:0/1292878501 conn(0x7fd1800cae70 msgr2=0x7fd1800cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.604+0000 7fd1861d7640 1 -- 172.21.15.26:0/1292878501 shutdown_connections 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.604+0000 7fd1861d7640 1 -- 172.21.15.26:0/1292878501 wait complete. 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 Processor -- start 2024-03-06T07:26:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 -- start start 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1800ad2b0 0x7fd180154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1800c69f0 0x7fd1801eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1801eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180152a20 con 0x7fd1800c8950 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180155fe0 con 0x7fd1800c69f0 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.605+0000 7fd1861d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd180152ba0 con 0x7fd1800ad2b0 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1801eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1801eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58066/0 (socket says 172.21.15.26:58066) 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 -- 172.21.15.26:0/194890667 learned_addr learned my addr 172.21.15.26:0/194890667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17effd640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1800c69f0 0x7fd1801eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17f7fe640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1800ad2b0 0x7fd180154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 -- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1800ad2b0 msgr2=0x7fd180154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1800ad2b0 0x7fd180154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 -- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1800c69f0 msgr2=0x7fd1801eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1800c69f0 0x7fd1801eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 -- 172.21.15.26:0/194890667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800c4640 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd17ffff640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1801eb0a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c700 tx=0x7fd17800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd167fff640 1 -- 172.21.15.26:0/194890667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178018070 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd167fff640 1 -- 172.21.15.26:0/194890667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17800ce10 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd180159090 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd167fff640 1 -- 172.21.15.26:0/194890667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800a860 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.606+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd180158bc0 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.607+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1801586d0 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.608+0000 7fd167fff640 1 -- 172.21.15.26:0/194890667 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd17801b070 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.608+0000 7fd167fff640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd16c06b4d0 0x7fd16c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.608+0000 7fd167fff640 1 -- 172.21.15.26:0/194890667 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fd178089b50 con 0x7fd1800c8950 2024-03-06T07:26:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.608+0000 7fd17f7fe640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd16c06b4d0 0x7fd16c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.609+0000 7fd17f7fe640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd16c06b4d0 0x7fd16c06d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd170009b40 tx=0x7fd170009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.613+0000 7fd167fff640 1 -- 172.21.15.26:0/194890667 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd180155130 con 0x7fd1800c8950 2024-03-06T07:26:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.873+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "11:0"} v 0) v1 -- 0x7fd18004f330 con 0x7fd1800c8950 2024-03-06T07:26:23.876 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:26:23.874+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:26:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.875+0000 7fd167fff640 1 -- 172.21.15.26:0/194890667 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "11:0"}]=0 failed mds gid 8793 v144) v1 ==== 98+0+0 (secure 0 0 0) 0x7fd178057f30 con 0x7fd1800c8950 2024-03-06T07:26:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:failed mds gid 8793 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd16c06b4d0 msgr2=0x7fd16c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd16c06b4d0 0x7fd16c06d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd170009b40 tx=0x7fd170009190 comp rx=0 tx=0).stop 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 msgr2=0x7fd1801eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1801eb0a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c700 tx=0x7fd17800cbc0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 shutdown_connections 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd16c06b4d0 0x7fd16c06d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800c8950 0x7fd1801eb0a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1800c69f0 0x7fd1801eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 --2- 172.21.15.26:0/194890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1800ad2b0 0x7fd180154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 >> 172.21.15.26:0/194890667 conn(0x7fd1800cae70 msgr2=0x7fd1800bff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 shutdown_connections 2024-03-06T07:26:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.877+0000 7fd1861d7640 1 -- 172.21.15.26:0/194890667 wait complete. 2024-03-06T07:26:23.881 INFO:tasks.ceph.mds.a.smithi026.stderr: -408> 2024-03-06T07:26:12.981+0000 7ff6ea5a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:23.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:23.907 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:26:23.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.994+0000 7ff4fb30c640 1 Processor -- start 2024-03-06T07:26:23.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.994+0000 7ff4fb30c640 1 -- start start 2024-03-06T07:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4fb30c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f4153990 0x7ff4f4156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4fb30c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff4f40acc00 0x7ff4f40c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4fb30c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f4070aa0 con 0x7ff4f41506b0 2024-03-06T07:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4fb30c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f4153e10 con 0x7ff4f40acc00 2024-03-06T07:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4fb30c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f4156450 con 0x7ff4f4153990 2024-03-06T07:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4f90a8640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff4f41506b0 0x7ff4f40702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44668/0 (socket says 172.21.15.26:44668) 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4f90a8640 1 -- 172.21.15.26:0/2365403895 learned_addr learned my addr 172.21.15.26:0/2365403895 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4f98a9640 1 --2- 172.21.15.26:0/2365403895 >> v2:172.21.15.117:3300/0 conn(0x7ff4f40acc00 0x7ff4f40c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.995+0000 7ff4f88a7640 1 --2- 172.21.15.26:0/2365403895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f4153990 0x7ff4f4156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.996+0000 7ff4f3fff640 1 -- 172.21.15.26:0/2365403895 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356300837 0 0) 0x7ff4f4070aa0 con 0x7ff4f41506b0 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.996+0000 7ff4f3fff640 1 -- 172.21.15.26:0/2365403895 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4dc003520 con 0x7ff4f41506b0 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.996+0000 7ff4f88a7640 1 -- 172.21.15.26:0/2365403895 >> v2:172.21.15.117:3300/0 conn(0x7ff4f40acc00 msgr2=0x7ff4f40c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.996+0000 7ff4f88a7640 1 --2- 172.21.15.26:0/2365403895 >> v2:172.21.15.117:3300/0 conn(0x7ff4f40acc00 0x7ff4f40c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.996+0000 7ff4f88a7640 1 -- 172.21.15.26:0/2365403895 >> v1:172.21.15.117:6789/0 conn(0x7ff4f41506b0 legacy=0x7ff4f40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.996+0000 7ff4f88a7640 1 -- 172.21.15.26:0/2365403895 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f4070d80 con 0x7ff4f4153990 2024-03-06T07:26:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.996+0000 7ff4f88a7640 1 --2- 172.21.15.26:0/2365403895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f4153990 0x7ff4f4156ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80097a0 tx=0x7ff4e802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e6f8a21bff0b82 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4f3fff640 1 -- 172.21.15.26:0/2365403895 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8035020 con 0x7ff4f4153990 2024-03-06T07:26:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4f3fff640 1 -- 172.21.15.26:0/2365403895 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e8031830 con 0x7ff4f4153990 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4f3fff640 1 -- 172.21.15.26:0/2365403895 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8031be0 con 0x7ff4f4153990 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 -- 172.21.15.26:0/2365403895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f4153990 msgr2=0x7ff4f4156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/2365403895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f4153990 0x7ff4f4156ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80097a0 tx=0x7ff4e802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 -- 172.21.15.26:0/2365403895 shutdown_connections 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/2365403895 >> v2:172.21.15.117:3300/0 conn(0x7ff4f40acc00 0x7ff4f40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/2365403895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f4153990 0x7ff4f4156ea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 -- 172.21.15.26:0/2365403895 >> 172.21.15.26:0/2365403895 conn(0x7ff4f406b1a0 msgr2=0x7ff4f406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 -- 172.21.15.26:0/2365403895 shutdown_connections 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.997+0000 7ff4fb30c640 1 -- 172.21.15.26:0/2365403895 wait complete. 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.998+0000 7ff4fb30c640 1 Processor -- start 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.998+0000 7ff4fb30c640 1 -- start start 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4fb30c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f40acc00 0x7ff4f414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4fb30c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4f41506b0 0x7ff4f41e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4fb30c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4153990 0x7ff4f40aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4fb30c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f40ad070 con 0x7ff4f4153990 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4fb30c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f4156a10 con 0x7ff4f41506b0 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4fb30c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f40ad1f0 con 0x7ff4f40acc00 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f98a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4153990 0x7ff4f40aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4f41506b0 0x7ff4f41e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f90a8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f40acc00 0x7ff4f414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4f41506b0 0x7ff4f41e8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55980/0 (socket says 172.21.15.26:55980) 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f98a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4153990 0x7ff4f40aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58106/0 (socket says 172.21.15.26:58106) 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 -- 172.21.15.26:0/1389010303 learned_addr learned my addr 172.21.15.26:0/1389010303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 -- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f40acc00 msgr2=0x7ff4f414b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f40acc00 0x7ff4f414b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 -- 172.21.15.26:0/1389010303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4153990 msgr2=0x7ff4f40aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4153990 0x7ff4f40aab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f88a7640 1 -- 172.21.15.26:0/1389010303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f41ed040 con 0x7ff4f41506b0 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:23.999+0000 7ff4f98a9640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4153990 0x7ff4f40aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.000+0000 7ff4f88a7640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4f41506b0 0x7ff4f41e8e70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff4e802b910 tx=0x7ff4e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.000+0000 7ff4f1ffb640 1 -- 172.21.15.26:0/1389010303 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8040070 con 0x7ff4f41506b0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.000+0000 7ff4f1ffb640 1 -- 172.21.15.26:0/1389010303 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e803a910 con 0x7ff4f41506b0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.000+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f40a8e40 con 0x7ff4f41506b0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.000+0000 7ff4f1ffb640 1 -- 172.21.15.26:0/1389010303 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8035040 con 0x7ff4f41506b0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.000+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f414a950 con 0x7ff4f41506b0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.001+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f404f330 con 0x7ff4f41506b0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.002+0000 7ff4f1ffb640 1 -- 172.21.15.26:0/1389010303 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4e804b060 con 0x7ff4f41506b0 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.002+0000 7ff4f1ffb640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e006b4d0 0x7ff4e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.002+0000 7ff4f1ffb640 1 -- 172.21.15.26:0/1389010303 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7ff4e80abf10 con 0x7ff4f41506b0 2024-03-06T07:26:24.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.002+0000 7ff4f90a8640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e006b4d0 0x7ff4e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.003+0000 7ff4f90a8640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e006b4d0 0x7ff4e006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005880 tx=0x7ff4ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.010+0000 7ff4f1ffb640 1 -- 172.21.15.26:0/1389010303 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4e807a2b0 con 0x7ff4f41506b0 2024-03-06T07:26:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.233+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4f40700a0 con 0x7ff4f41506b0 2024-03-06T07:26:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.234+0000 7ff4f1ffb640 1 -- 172.21.15.26:0/1389010303 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5475 (secure 0 0 0) 0x7ff4e807e350 con 0x7ff4f41506b0 2024-03-06T07:26:24.236 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:24.236 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":145,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:23.880322+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":11},{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:20.123577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 145 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e006b4d0 msgr2=0x7ff4e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e006b4d0 0x7ff4e006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005880 tx=0x7ff4ec009040 comp rx=0 tx=0).stop 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4f41506b0 msgr2=0x7ff4f41e8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4f41506b0 0x7ff4f41e8e70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff4e802b910 tx=0x7ff4e80046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 shutdown_connections 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e006b4d0 0x7ff4e006d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4153990 0x7ff4f40aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4f41506b0 0x7ff4f41e8e70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 --2- 172.21.15.26:0/1389010303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4f40acc00 0x7ff4f414b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 >> 172.21.15.26:0/1389010303 conn(0x7ff4f406b1a0 msgr2=0x7ff4f4153240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 shutdown_connections 2024-03-06T07:26:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:24.236+0000 7ff4fb30c640 1 -- 172.21.15.26:0/1389010303 wait complete. 2024-03-06T07:26:24.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 145, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:07.071721+0000', 'modified': '2024-03-06T07:26:23.880322+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 156, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7178': {'gid': 7178, 'name': 'b', 'rank': 0, 'incarnation': 145, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.117:6837/677194061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 677194061}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 677194061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:24.652 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7178:up:replay 2024-03-06T07:26:25.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.766+0000 7fa3f8ef4640 1 Processor -- start 2024-03-06T07:26:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.766+0000 7fa3f8ef4640 1 -- start start 2024-03-06T07:26:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f8ef4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa3f40c3280 0x7fa3f40c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f8ef4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3f41551e0 0x7fa3f41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f8ef4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f4158800 con 0x7fa3f406d9f0 2024-03-06T07:26:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f8ef4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f4158ae0 con 0x7fa3f41551e0 2024-03-06T07:26:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f8ef4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f4158dc0 con 0x7fa3f40c3280 2024-03-06T07:26:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3f41551e0 0x7fa3f41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3f41551e0 0x7fa3f41555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58120/0 (socket says 172.21.15.26:58120) 2024-03-06T07:26:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f2d9d640 1 -- 172.21.15.26:0/2570325552 learned_addr learned my addr 172.21.15.26:0/2570325552 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3ebfff640 1 --2- 172.21.15.26:0/2570325552 >> v2:172.21.15.117:3300/0 conn(0x7fa3f40c3280 0x7fa3f40c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f2d9d640 1 -- 172.21.15.26:0/2570325552 >> v2:172.21.15.117:3300/0 conn(0x7fa3f40c3280 msgr2=0x7fa3f40c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f2d9d640 1 --2- 172.21.15.26:0/2570325552 >> v2:172.21.15.117:3300/0 conn(0x7fa3f40c3280 0x7fa3f40c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f2d9d640 1 -- 172.21.15.26:0/2570325552 >> v1:172.21.15.117:6789/0 conn(0x7fa3f406d9f0 legacy=0x7fa3f40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.767+0000 7fa3f2d9d640 1 -- 172.21.15.26:0/2570325552 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f41590a0 con 0x7fa3f41551e0 2024-03-06T07:26:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.768+0000 7fa3f2d9d640 1 --2- 172.21.15.26:0/2570325552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f41551e0 0x7fa3f41555f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40721a0 tx=0x7fa3ec02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e826a49c616b6b01 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.768+0000 7fa3f1d9b640 1 -- 172.21.15.26:0/2570325552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec02ca80 con 0x7fa3f41551e0 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.768+0000 7fa3f1d9b640 1 -- 172.21.15.26:0/2570325552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec02cc20 con 0x7fa3f41551e0 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.768+0000 7fa3f1d9b640 1 -- 172.21.15.26:0/2570325552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0343f0 con 0x7fa3f41551e0 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/2570325552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f41551e0 msgr2=0x7fa3f41555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/2570325552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f41551e0 0x7fa3f41555f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40721a0 tx=0x7fa3ec02bee0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/2570325552 shutdown_connections 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/2570325552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f41551e0 0x7fa3f41555f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/2570325552 >> v2:172.21.15.117:3300/0 conn(0x7fa3f40c3280 0x7fa3f40c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/2570325552 >> 172.21.15.26:0/2570325552 conn(0x7fa3f40aa630 msgr2=0x7fa3f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/2570325552 shutdown_connections 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.769+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/2570325552 wait complete. 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.770+0000 7fa3f8ef4640 1 Processor -- start 2024-03-06T07:26:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.770+0000 7fa3f8ef4640 1 -- start start 2024-03-06T07:26:25.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f8ef4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f406d9f0 0x7fa3f41eb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f8ef4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3f40c3280 0x7fa3f41ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f8ef4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3f41551e0 0x7fa3f406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f8ef4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406df50 con 0x7fa3f406d9f0 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f8ef4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406ddd0 con 0x7fa3f41551e0 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f8ef4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40c6910 con 0x7fa3f40c3280 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f406d9f0 0x7fa3f41eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f406d9f0 0x7fa3f41eb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58126/0 (socket says 172.21.15.26:58126) 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 -- 172.21.15.26:0/3287812077 learned_addr learned my addr 172.21.15.26:0/3287812077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f2d9d640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3f41551e0 0x7fa3f406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3ebfff640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3f40c3280 0x7fa3f41ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 -- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3f40c3280 msgr2=0x7fa3f41ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3f40c3280 0x7fa3f41ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 -- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3f41551e0 msgr2=0x7fa3f406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3f41551e0 0x7fa3f406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 -- 172.21.15.26:0/3287812077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40ab750 con 0x7fa3f406d9f0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.771+0000 7fa3f259c640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f406d9f0 0x7fa3f41eb330 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c430 tx=0x7fa3e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.772+0000 7fa3eaffd640 1 -- 172.21.15.26:0/3287812077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40052d0 con 0x7fa3f406d9f0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.772+0000 7fa3eaffd640 1 -- 172.21.15.26:0/3287812077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e4005490 con 0x7fa3f406d9f0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.772+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f41eedb0 con 0x7fa3f406d9f0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.772+0000 7fa3eaffd640 1 -- 172.21.15.26:0/3287812077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4007260 con 0x7fa3f406d9f0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.772+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f40ab9f0 con 0x7fa3f406d9f0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.773+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f404f330 con 0x7fa3f406d9f0 2024-03-06T07:26:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.773+0000 7fa3eaffd640 1 -- 172.21.15.26:0/3287812077 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3e4007400 con 0x7fa3f406d9f0 2024-03-06T07:26:25.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.774+0000 7fa3eaffd640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3d406b4d0 0x7fa3d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.774+0000 7fa3eaffd640 1 -- 172.21.15.26:0/3287812077 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa3e4089bd0 con 0x7fa3f406d9f0 2024-03-06T07:26:25.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.774+0000 7fa3ebfff640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3d406b4d0 0x7fa3d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.774+0000 7fa3ebfff640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3d406b4d0 0x7fa3d406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa3f406d090 tx=0x7fa3dc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:25.778+0000 7fa3eaffd640 1 -- 172.21.15.26:0/3287812077 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa3e4057eb0 con 0x7fa3f406d9f0 2024-03-06T07:26:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.000+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3f41524d0 con 0x7fa3f406d9f0 2024-03-06T07:26:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.000+0000 7fa3eaffd640 1 -- 172.21.15.26:0/3287812077 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 147 v147) v1 ==== 95+0+6264 (secure 0 0 0) 0x7fa3e405bf50 con 0x7fa3f406d9f0 2024-03-06T07:26:26.003 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:26.003 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":147,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:24.885582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":11},{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:25.766406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 147 2024-03-06T07:26:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3d406b4d0 msgr2=0x7fa3d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3d406b4d0 0x7fa3d406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa3f406d090 tx=0x7fa3dc009190 comp rx=0 tx=0).stop 2024-03-06T07:26:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f406d9f0 msgr2=0x7fa3f41eb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f406d9f0 0x7fa3f41eb330 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c430 tx=0x7fa3e400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 shutdown_connections 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa3d406b4d0 0x7fa3d406d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3f41551e0 0x7fa3f406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3f40c3280 0x7fa3f41ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.002+0000 7fa3f8ef4640 1 --2- 172.21.15.26:0/3287812077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f406d9f0 0x7fa3f41eb330 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.003+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 >> 172.21.15.26:0/3287812077 conn(0x7fa3f40aa630 msgr2=0x7fa3f40c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.003+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 shutdown_connections 2024-03-06T07:26:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:26.003+0000 7fa3f8ef4640 1 -- 172.21.15.26:0/3287812077 wait complete. 2024-03-06T07:26:26.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 147, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:07.071721+0000', 'modified': '2024-03-06T07:26:24.885582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 156, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7178': {'gid': 7178, 'name': 'b', 'rank': 0, 'incarnation': 145, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.117:6837/677194061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 677194061}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 677194061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:26.419 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7178:up:rejoin 2024-03-06T07:26:27.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.531+0000 7f164742a640 1 Processor -- start 2024-03-06T07:26:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.531+0000 7f164742a640 1 -- start start 2024-03-06T07:26:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.532+0000 7f164742a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f164006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.532+0000 7f164742a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400c1ae0 con 0x7f1640155c80 2024-03-06T07:26:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.532+0000 7f164742a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164006e450 con 0x7f16400720a0 2024-03-06T07:26:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.532+0000 7f164742a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400c1dc0 con 0x7f1640152520 2024-03-06T07:26:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.532+0000 7f16449c5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1640155c80 0x7f16400c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49468/0 (socket says 172.21.15.26:49468) 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.532+0000 7f16449c5640 1 -- 172.21.15.26:0/2891479056 learned_addr learned my addr 172.21.15.26:0/2891479056 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.532+0000 7f16451c6640 1 --2- 172.21.15.26:0/2891479056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f164006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.533+0000 7f1637fff640 1 -- 172.21.15.26:0/2891479056 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226481964 0 0) 0x7f16400c1ae0 con 0x7f1640155c80 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.533+0000 7f1637fff640 1 -- 172.21.15.26:0/2891479056 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f162c003520 con 0x7f1640155c80 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.533+0000 7f1637fff640 1 -- 172.21.15.26:0/2891479056 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144664232 0 0) 0x7f164006e450 con 0x7f16400720a0 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.533+0000 7f16451c6640 1 -- 172.21.15.26:0/2891479056 >> v1:172.21.15.117:6789/0 conn(0x7f16400720a0 legacy=0x7f164006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.533+0000 7f16451c6640 1 -- 172.21.15.26:0/2891479056 >> v1:172.21.15.26:6789/0 conn(0x7f1640155c80 legacy=0x7f16400c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.533+0000 7f16451c6640 1 -- 172.21.15.26:0/2891479056 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164006e730 con 0x7f1640152520 2024-03-06T07:26:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.533+0000 7f16451c6640 1 --2- 172.21.15.26:0/2891479056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f164006ef30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f16380097d0 tx=0x7f163802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d394b372433ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f1637fff640 1 -- 172.21.15.26:0/2891479056 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638035020 con 0x7f1640152520 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f1637fff640 1 -- 172.21.15.26:0/2891479056 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638031890 con 0x7f1640152520 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f1637fff640 1 -- 172.21.15.26:0/2891479056 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638031c40 con 0x7f1640152520 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f164742a640 1 -- 172.21.15.26:0/2891479056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 msgr2=0x7f164006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f164742a640 1 --2- 172.21.15.26:0/2891479056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f164006ef30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f16380097d0 tx=0x7f163802b930 comp rx=0 tx=0).stop 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f164742a640 1 -- 172.21.15.26:0/2891479056 shutdown_connections 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f164742a640 1 --2- 172.21.15.26:0/2891479056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f164006ef30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f164742a640 1 -- 172.21.15.26:0/2891479056 >> 172.21.15.26:0/2891479056 conn(0x7f16400c76b0 msgr2=0x7f16400c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f164742a640 1 -- 172.21.15.26:0/2891479056 shutdown_connections 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.534+0000 7f164742a640 1 -- 172.21.15.26:0/2891479056 wait complete. 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.535+0000 7f164742a640 1 Processor -- start 2024-03-06T07:26:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.535+0000 7f164742a640 1 -- start start 2024-03-06T07:26:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f164742a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16400720a0 0x7f16401eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f164742a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f16400a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f164742a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1640155c80 0x7f16400ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f164742a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164006b850 con 0x7f1640155c80 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f164742a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164006b6d0 con 0x7f16400720a0 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f164742a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1640152900 con 0x7f1640152520 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f16459c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1640155c80 0x7f16400ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f16451c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16400720a0 0x7f16401eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f16459c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1640155c80 0x7f16400ab290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42662/0 (socket says 172.21.15.26:42662) 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f16459c7640 1 -- 172.21.15.26:0/3568702633 learned_addr learned my addr 172.21.15.26:0/3568702633 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f16449c5640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f16400a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f16459c7640 1 -- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 msgr2=0x7f16400a6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.536+0000 7f16459c7640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f16400a6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f16459c7640 1 -- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16400720a0 msgr2=0x7f16401eaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f16459c7640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16400720a0 0x7f16401eaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f16459c7640 1 -- 172.21.15.26:0/3568702633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640157920 con 0x7f1640155c80 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f16451c6640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16400720a0 0x7f16401eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f16459c7640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1640155c80 0x7f16400ab290 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c780 tx=0x7f163c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f1635ffb640 1 -- 172.21.15.26:0/3568702633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c003b70 con 0x7f1640155c80 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f1635ffb640 1 -- 172.21.15.26:0/3568702633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c003d10 con 0x7f1640155c80 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1640157c00 con 0x7f1640155c80 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f1635ffb640 1 -- 172.21.15.26:0/3568702633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c005270 con 0x7f1640155c80 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.537+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16401581c0 con 0x7f1640155c80 2024-03-06T07:26:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.538+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164006e450 con 0x7f1640155c80 2024-03-06T07:26:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.539+0000 7f1635ffb640 1 -- 172.21.15.26:0/3568702633 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f163c005410 con 0x7f1640155c80 2024-03-06T07:26:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.539+0000 7f1635ffb640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f162406b500 0x7f162406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.539+0000 7f1635ffb640 1 -- 172.21.15.26:0/3568702633 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f163c08a3d0 con 0x7f1640155c80 2024-03-06T07:26:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.539+0000 7f16451c6640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f162406b500 0x7f162406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.540+0000 7f16451c6640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f162406b500 0x7f162406d9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1638036000 tx=0x7f163803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.544+0000 7f1635ffb640 1 -- 172.21.15.26:0/3568702633 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f164006e450 con 0x7f1640155c80 2024-03-06T07:26:27.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.762+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f164004f330 con 0x7f1640155c80 2024-03-06T07:26:27.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.762+0000 7f1635ffb640 1 -- 172.21.15.26:0/3568702633 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+6271 (secure 0 0 0) 0x7f163c058750 con 0x7f1640155c80 2024-03-06T07:26:27.765 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:27.765 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":148,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:26.887138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7178,"qdb_cluster":[7178]},"id":11},{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:25.766406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:27.765 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 148 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f162406b500 msgr2=0x7f162406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f162406b500 0x7f162406d9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1638036000 tx=0x7f163803c040 comp rx=0 tx=0).stop 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1640155c80 msgr2=0x7f16400ab290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1640155c80 0x7f16400ab290 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c780 tx=0x7f163c00cb80 comp rx=0 tx=0).stop 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 shutdown_connections 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f162406b500 0x7f162406d9c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1640155c80 0x7f16400ab290 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1640152520 0x7f16400a6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 --2- 172.21.15.26:0/3568702633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16400720a0 0x7f16401eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 >> 172.21.15.26:0/3568702633 conn(0x7f16400c76b0 msgr2=0x7f16400c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 shutdown_connections 2024-03-06T07:26:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:27.764+0000 7f164742a640 1 -- 172.21.15.26:0/3568702633 wait complete. 2024-03-06T07:26:27.987 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-03-06T07:26:27.983+0000 7eff1b895640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:28.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:07.071721+0000', 'modified': '2024-03-06T07:26:26.887138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 156, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7178': {'gid': 7178, 'name': 'b', 'rank': 0, 'incarnation': 145, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/677194061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 677194061}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 677194061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7178, 'qdb_cluster': [7178]} 2024-03-06T07:26:28.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:26:28.180 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:26:28.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.282+0000 7f0f236eb640 1 Processor -- start 2024-03-06T07:26:28.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.282+0000 7f0f236eb640 1 -- start start 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f236eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f1c1526d0 0x7f0f1c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f236eb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f1c0c35b0 0x7f0f1c0c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f236eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c06feb0 con 0x7f0f1c0c25f0 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f236eb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c06b960 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f236eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c06f0f0 con 0x7f0f1c1526d0 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f20c86640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f1c0c35b0 0x7f0f1c0c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f21c88640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0f1c0c25f0 0x7f0f1c155110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49482/0 (socket says 172.21.15.26:49482) 2024-03-06T07:26:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f21c88640 1 -- 172.21.15.26:0/936983895 learned_addr learned my addr 172.21.15.26:0/936983895 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:28.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f21487640 1 --2- 172.21.15.26:0/936983895 >> v2:172.21.15.117:3300/0 conn(0x7f0f1c1526d0 0x7f0f1c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f20c86640 1 -- 172.21.15.26:0/936983895 >> v2:172.21.15.117:3300/0 conn(0x7f0f1c1526d0 msgr2=0x7f0f1c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f20c86640 1 --2- 172.21.15.26:0/936983895 >> v2:172.21.15.117:3300/0 conn(0x7f0f1c1526d0 0x7f0f1c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f20c86640 1 -- 172.21.15.26:0/936983895 >> v1:172.21.15.26:6789/0 conn(0x7f0f1c0c25f0 legacy=0x7f0f1c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f20c86640 1 -- 172.21.15.26:0/936983895 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c06f3d0 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.283+0000 7f0f1bfff640 1 -- 172.21.15.26:0/936983895 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883031582 0 0) 0x7f0f1c06feb0 con 0x7f0f1c0c25f0 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.284+0000 7f0f20c86640 1 --2- 172.21.15.26:0/936983895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c20b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0ada20 tx=0x7f0f1002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121fddcc0bb5d536 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.284+0000 7f0f1bfff640 1 -- 172.21.15.26:0/936983895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f100315a0 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.284+0000 7f0f1bfff640 1 -- 172.21.15.26:0/936983895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10037070 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.284+0000 7f0f1bfff640 1 -- 172.21.15.26:0/936983895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10031960 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 -- 172.21.15.26:0/936983895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 msgr2=0x7f0f1c0c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 --2- 172.21.15.26:0/936983895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c20b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0ada20 tx=0x7f0f1002b840 comp rx=0 tx=0).stop 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 -- 172.21.15.26:0/936983895 shutdown_connections 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 --2- 172.21.15.26:0/936983895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c20b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 --2- 172.21.15.26:0/936983895 >> v2:172.21.15.117:3300/0 conn(0x7f0f1c1526d0 0x7f0f1c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 -- 172.21.15.26:0/936983895 >> 172.21.15.26:0/936983895 conn(0x7f0f1c0c41b0 msgr2=0x7f0f1c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 -- 172.21.15.26:0/936983895 shutdown_connections 2024-03-06T07:26:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.285+0000 7f0f236eb640 1 -- 172.21.15.26:0/936983895 wait complete. 2024-03-06T07:26:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.286+0000 7f0f236eb640 1 Processor -- start 2024-03-06T07:26:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.286+0000 7f0f236eb640 1 -- start start 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f236eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1c0c25f0 0x7f0f1c1474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f236eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f236eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1c1526d0 0x7f0f1c0c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f236eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c152ab0 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f236eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c06f990 con 0x7f0f1c1526d0 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f236eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c06b500 con 0x7f0f1c0c25f0 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42670/0 (socket says 172.21.15.26:42670) 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 -- 172.21.15.26:0/3822741473 learned_addr learned my addr 172.21.15.26:0/3822741473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f21487640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1c0c25f0 0x7f0f1c1474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f21c88640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1c1526d0 0x7f0f1c0c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 -- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1c0c25f0 msgr2=0x7f0f1c1474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1c0c25f0 0x7f0f1c1474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 -- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1c1526d0 msgr2=0x7f0f1c0c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1c1526d0 0x7f0f1c0c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 -- 172.21.15.26:0/3822741473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c1481e0 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f20c86640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c6560 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0f10000c00 tx=0x7f0f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f19ffb640 1 -- 172.21.15.26:0/3822741473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10032630 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f19ffb640 1 -- 172.21.15.26:0/3822741473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f100327f0 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c157b00 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.287+0000 7f0f19ffb640 1 -- 172.21.15.26:0/3822741473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10032be0 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.288+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c1ebd40 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.289+0000 7f0f19ffb640 1 -- 172.21.15.26:0/3822741473 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f1004d3f0 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.289+0000 7f0f19ffb640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0806b500 0x7f0f0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.289+0000 7f0f19ffb640 1 -- 172.21.15.26:0/3822741473 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f0f1002f030 con 0x7f0f1c0c35b0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.290+0000 7f0f21487640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0806b500 0x7f0f0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.290+0000 7f0f21487640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0806b500 0x7f0f0806d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009aa0 tx=0x7f0f14009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.290+0000 7f0f263ae740 1 Processor -- start 2024-03-06T07:26:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.290+0000 7f0f263ae740 1 -- start start 2024-03-06T07:26:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25acdf50 0x562b25acba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25acbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acc4b0 0x562b25ac8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f263ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25a326b0 con 0x562b25ace330 2024-03-06T07:26:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b259dba60 con 0x562b25acdf50 2024-03-06T07:26:28.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b25a2e750 con 0x562b25acc4b0 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f20c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25acbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f20c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25acbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42680/0 (socket says 172.21.15.26:42680) 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f20c86640 1 -- 172.21.15.26:0/2263338692 learned_addr learned my addr 172.21.15.26:0/2263338692 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f21487640 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25acdf50 0x562b25acba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f21c88640 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acc4b0 0x562b25ac8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f21487640 1 -- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acc4b0 msgr2=0x562b25ac8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f21487640 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acc4b0 0x562b25ac8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f21487640 1 -- 172.21.15.26:0/2263338692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 msgr2=0x562b25acbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f21487640 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25acbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f21487640 1 -- 172.21.15.26:0/2263338692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b259f9f30 con 0x562b25acdf50 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.291+0000 7f0f20c86640 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25acbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f21487640 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25acdf50 0x562b25acba30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f140043a0 tx=0x7f0f14014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb04faa79bf1b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f02ffd640 1 -- 172.21.15.26:0/2263338692 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1401b070 con 0x562b25acdf50 2024-03-06T07:26:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f02ffd640 1 -- 172.21.15.26:0/2263338692 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f14027070 con 0x562b25acdf50 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f02ffd640 1 -- 172.21.15.26:0/2263338692 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14003a20 con 0x562b25acdf50 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 -- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25acdf50 msgr2=0x562b25acba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25acdf50 0x562b25acba30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f140043a0 tx=0x7f0f14014000 comp rx=0 tx=0).stop 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 -- 172.21.15.26:0/2263338692 shutdown_connections 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acc4b0 0x562b25ac8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25acbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2263338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25acdf50 0x562b25acba30 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 -- 172.21.15.26:0/2263338692 >> 172.21.15.26:0/2263338692 conn(0x562b25aca7d0 msgr2=0x562b25aef3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 -- 172.21.15.26:0/2263338692 shutdown_connections 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.292+0000 7f0f263ae740 1 -- 172.21.15.26:0/2263338692 wait complete. 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.293+0000 7f0f263ae740 1 -- start start 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.293+0000 7f0f263ae740 1 Processor -- start 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25b1a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25b0a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25af3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f263ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25af0230 con 0x562b25acdf50 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b259eeea0 con 0x562b25ace330 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b25aefc70 con 0x562b25af3a30 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25b1a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f263ae740 10 client.0 _ll_register_callbacks cb 0x562b259ee870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25b1a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42682/0 (socket says 172.21.15.26:42682) 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 -- 172.21.15.26:0/2349206311 learned_addr learned my addr 172.21.15.26:0/2349206311 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f20c86640 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25b0a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21c88640 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25af3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 -- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 msgr2=0x562b25af3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25af3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 -- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 msgr2=0x562b25b0a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25b0a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 -- 172.21.15.26:0/2349206311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b259ef270 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0f21487640 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25b1a950 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0f14003da0 tx=0x7f0f14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad62d5f5b236679 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.294+0000 7f0efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0efbfff640 1 -- 172.21.15.26:0/2349206311 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1401b040 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0efbfff640 1 -- 172.21.15.26:0/2349206311 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f0f14003e00 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f14027070 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0efbfff640 1 -- 172.21.15.26:0/2349206311 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f14004020 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 10 client.9114 fetch_fsmap learned FSMap version 148 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=2+}) v3 -- 0x562b257472d0 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0efbfff640 1 -- 172.21.15.26:0/2349206311 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f14027070 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0efbfff640 1 -- 172.21.15.26:0/2349206311 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 148) v1 ==== 3929+0+0 (secure 0 0 0) 0x7f0f14041080 con 0x562b25acdf50 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 10 client.9114 fetch_fsmap finished waiting for FSMap version 148 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 10 client.9114 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:26:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 20 client.9114 populate_metadata read hostname 'smithi026' 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25af0fd0 0x562b25b295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 4 client.9114 mds_command: new command op to 7178 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 --> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b25a3fbf0 con 0x562b25af0fd0 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.295+0000 7f0f20c86640 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25af0fd0 0x562b25b295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f20c86640 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25af0fd0 0x562b25b295e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b59b0292d144155 server_cookie=3fabc220cbfdc526 in_seq=0 out_seq=0 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0efbfff640 10 client.9114 ms_handle_connect on v2:172.21.15.117:6835/677194061 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0efbfff640 1 -- 172.21.15.26:0/2349206311 <== mds.0 v2:172.21.15.117:6835/677194061 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562b25a3fbf0 con 0x562b25af0fd0 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0efbfff640 10 client.9114 handle_command_reply: tid=0 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 client.9114 shutdown 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 msgr2=0x562b25b1a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25b1a950 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0f14003da0 tx=0x7f0f14004730 comp rx=0 tx=0).stop 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 shutdown_connections 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25af0fd0 0x562b25b295e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25af3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25b0a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0efbfff640 0 client.9114 ms_handle_reset on v2:172.21.15.117:6835/677194061 2024-03-06T07:26:28.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2349206311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25b1a950 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.296+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 >> 172.21.15.26:0/2349206311 conn(0x562b25acc4b0 msgr2=0x562b25ac7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.297+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 shutdown_connections 2024-03-06T07:26:28.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.297+0000 7f0f263ae740 1 -- 172.21.15.26:0/2349206311 wait complete. 2024-03-06T07:26:28.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.297+0000 7f0f263ae740 20 client.9114 trim_cache size 0 max 16384 2024-03-06T07:26:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.313+0000 7f0f263ae740 1 Processor -- start 2024-03-06T07:26:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.313+0000 7f0f263ae740 1 -- start start 2024-03-06T07:26:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acdf50 0x562b25b07590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25ac7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25af3a30 0x562b25ac8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f263ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1f9c10 con 0x562b25ace330 2024-03-06T07:26:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1fab80 con 0x562b25af3a30 2024-03-06T07:26:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1f8840 con 0x562b25acdf50 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25ac7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25ac7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42684/0 (socket says 172.21.15.26:42684) 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 -- 172.21.15.26:0/2034281031 learned_addr learned my addr 172.21.15.26:0/2034281031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f21c88640 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25af3a30 0x562b25ac8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f21487640 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acdf50 0x562b25b07590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 -- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acdf50 msgr2=0x562b25b07590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acdf50 0x562b25b07590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 -- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25af3a30 msgr2=0x562b25ac8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25af3a30 0x562b25ac8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 -- 172.21.15.26:0/2034281031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b25b1e330 con 0x562b25ace330 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.314+0000 7f0f20c86640 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25ac7c00 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002be50 tx=0x7f0f100a9ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539678592915b110 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f027fc640 1 -- 172.21.15.26:0/2034281031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10087070 con 0x562b25ace330 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f027fc640 1 -- 172.21.15.26:0/2034281031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10037040 con 0x562b25ace330 2024-03-06T07:26:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f027fc640 1 -- 172.21.15.26:0/2034281031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10078720 con 0x562b25ace330 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 -- 172.21.15.26:0/2034281031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 msgr2=0x562b25ac7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25ac7c00 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002be50 tx=0x7f0f100a9ed0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 -- 172.21.15.26:0/2034281031 shutdown_connections 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25af3a30 0x562b25ac8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25ace330 0x562b25ac7c00 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 --2- 172.21.15.26:0/2034281031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25acdf50 0x562b25b07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 -- 172.21.15.26:0/2034281031 >> 172.21.15.26:0/2034281031 conn(0x562b25acc4b0 msgr2=0x562b25b1aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 -- 172.21.15.26:0/2034281031 shutdown_connections 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.315+0000 7f0f263ae740 1 -- 172.21.15.26:0/2034281031 wait complete. 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.316+0000 7f0f263ae740 1 -- start start 2024-03-06T07:26:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.316+0000 7f0f263ae740 1 Processor -- start 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25ac8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25aee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25b2de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25af4490 con 0x562b25acdf50 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1f8e40 con 0x562b25ace330 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b25af0b20 con 0x562b25af3a30 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 10 client.0 _ll_register_callbacks cb 0x7f0f0c004ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25ac8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25ac8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42688/0 (socket says 172.21.15.26:42688) 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 -- 172.21.15.26:0/1944241488 learned_addr learned my addr 172.21.15.26:0/1944241488 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21c88640 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25b2de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f20c86640 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25aee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 -- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 msgr2=0x562b25b2de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25b2de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 -- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 msgr2=0x562b25aee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25aee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 -- 172.21.15.26:0/1944241488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b258116b0 con 0x562b25acdf50 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f21487640 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25ac8fd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0f14011f50 tx=0x7f0f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f2bbbfe24155f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0efbfff640 1 -- 172.21.15.26:0/1944241488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1401b040 con 0x562b25acdf50 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0efbfff640 1 -- 172.21.15.26:0/1944241488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f0f140227f0 con 0x562b25acdf50 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f10037040 con 0x562b25acdf50 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0efbfff640 1 -- 172.21.15.26:0/1944241488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f1404e050 con 0x562b25acdf50 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 10 client.9120 fetch_fsmap learned FSMap version 148 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.317+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=2+}) v3 -- 0x562b25aee850 con 0x562b25acdf50 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0efbfff640 1 -- 172.21.15.26:0/1944241488 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10037040 con 0x562b25acdf50 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0efbfff640 1 -- 172.21.15.26:0/1944241488 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 148) v1 ==== 3929+0+0 (secure 0 0 0) 0x7f0f140229f0 con 0x562b25acdf50 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f263ae740 10 client.9120 fetch_fsmap finished waiting for FSMap version 148 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f263ae740 10 client.9120 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f263ae740 20 client.9120 populate_metadata read hostname 'smithi026' 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f263ae740 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25b77d10 0x562b25b173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f263ae740 4 client.9120 mds_command: new command op to 7178 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 --> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562b25a39f40 con 0x562b25b77d10 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f20c86640 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25b77d10 0x562b25b173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0efbfff640 10 client.9120 ms_handle_connect on v2:172.21.15.117:6835/677194061 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f20c86640 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25b77d10 0x562b25b173d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887ae95458efc5d1 server_cookie=157095f54f48511c in_seq=0 out_seq=0 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0efbfff640 1 -- 172.21.15.26:0/1944241488 <== mds.0 v2:172.21.15.117:6835/677194061 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x562b25a39f40 con 0x562b25b77d10 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0efbfff640 10 client.9120 handle_command_reply: tid=0 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.318+0000 7f0f263ae740 1 client.9120 shutdown 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 msgr2=0x562b25ac8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25ac8fd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0f14011f50 tx=0x7f0f140046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 shutdown_connections 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:6835/677194061,v1:172.21.15.117:6837/677194061] conn(0x562b25b77d10 0x562b25b173d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562b25af3a30 0x562b25b2de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0efbfff640 0 client.9120 ms_handle_reset on v2:172.21.15.117:6835/677194061 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b25ace330 0x562b25aee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 --2- 172.21.15.26:0/1944241488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b25acdf50 0x562b25ac8fd0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 >> 172.21.15.26:0/1944241488 conn(0x562b25acc4b0 msgr2=0x562b25acf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 shutdown_connections 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 1 -- 172.21.15.26:0/1944241488 wait complete. 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f263ae740 20 client.9120 trim_cache size 0 max 16384 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7178, 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.4377226609999996, 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 148, 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 156, 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 156, 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 25.702226363000001 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0806b500 msgr2=0x7f0f0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0806b500 0x7f0f0806d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009aa0 tx=0x7f0f14009190 comp rx=0 tx=0).stop 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 msgr2=0x7f0f1c0c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c6560 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0f10000c00 tx=0x7f0f100046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 shutdown_connections 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f0806b500 0x7f0f0806d9c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1c1526d0 0x7f0f1c0c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f1c0c35b0 0x7f0f1c0c6560 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 --2- 172.21.15.26:0/3822741473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1c0c25f0 0x7f0f1c1474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 >> 172.21.15.26:0/3822741473 conn(0x7f0f1c0c41b0 msgr2=0x7f0f1c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 shutdown_connections 2024-03-06T07:26:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.319+0000 7f0f236eb640 1 -- 172.21.15.26:0/3822741473 wait complete. 2024-03-06T07:26:28.332 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 11, "epoch": 148, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 11, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:07.071721+0000", "damaged": [], "data_pools": [ 24 ], "enabled": true, "epoch": 148, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7178": { "addr": "172.21.15.117:6837/677194061", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6835", "nonce": 677194061, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 677194061, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7178, "incarnation": 145, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 156, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 23, "modified": "2024-03-06T07:26:26.887138+0000", "qdb_cluster": [ 7178 ], "qdb_leader": 7178, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7178 } } }, { "id": 12, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:17.118167+0000", "damaged": [], "data_pools": [ 26 ], "enabled": true, "epoch": 147, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs2", "in": [ 0 ], "info": { "gid_8293": { "addr": "172.21.15.117:6836/296681906", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 296681906, "type": "v2" }, { "addr": "172.21.15.117:6836", "nonce": 296681906, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8293, "incarnation": 139, "join_fscid": 12, "name": "d", "rank": 0, "state": "up:active", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 25, "modified": "2024-03-06T07:26:25.766406+0000", "qdb_cluster": [ 8293 ], "qdb_leader": 8293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8293 } } } ], "standbys": [ { "addr": "172.21.15.26:6835/3463955148", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6834", "nonce": 3463955148, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 3463955148, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 148, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7166, "incarnation": 0, "join_fscid": 12, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 7 }, { "addr": "172.21.15.26:6833/743496767", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 743496767, "type": "v2" }, { "addr": "172.21.15.26:6833", "nonce": 743496767, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 146, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 9084, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:26:28.333 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b', 'state': 'up:active'}, {'join_fscid': 12, 'name': 'c'}, {'join_fscid': 12, 'name': 'd'}] 2024-03-06T07:26:28.334 INFO:tasks.cephfs.test_failover:current = [{'gid': 9084, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.26:6833/743496767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 743496767}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 743496767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 146}, {'gid': 7178, 'name': 'b', 'rank': 0, 'incarnation': 145, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/677194061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 677194061}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 677194061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7166, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 7, 'addr': '172.21.15.26:6835/3463955148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3463955148}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3463955148}]}, 'join_fscid': 12, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 148}, {'gid': 8293, 'name': 'd', 'rank': 0, 'incarnation': 139, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.117:6836/296681906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 296681906}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 296681906}]}, 'join_fscid': 12, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:26:28.334 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-06T07:26:28.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.432+0000 7f4b64755640 1 Processor -- start 2024-03-06T07:26:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.432+0000 7f4b64755640 1 -- start start 2024-03-06T07:26:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b64755640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4b5c0c7180 0x7f4b5c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b64755640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c88b0 0x7f4b5c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b64755640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5c0c7c60 con 0x7f4b5c1527b0 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b64755640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c159640 con 0x7f4b5c0c7180 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b64755640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c0c8060 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b624f1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4b5c1527b0 0x7f4b5c155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49484/0 (socket says 172.21.15.26:49484) 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b624f1640 1 -- 172.21.15.26:0/1238818155 learned_addr learned my addr 172.21.15.26:0/1238818155 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b62cf2640 1 --2- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c88b0 0x7f4b5c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.433+0000 7f4b61cf0640 1 --2- 172.21.15.26:0/1238818155 >> v2:172.21.15.117:3300/0 conn(0x7f4b5c0c7180 0x7f4b5c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b614ef640 1 -- 172.21.15.26:0/1238818155 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636923441 0 0) 0x7f4b58001110 con 0x7f4b5c1527b0 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b614ef640 1 -- 172.21.15.26:0/1238818155 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b48003520 con 0x7f4b5c1527b0 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b61cf0640 1 -- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c88b0 msgr2=0x7f4b5c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b61cf0640 1 --2- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c88b0 0x7f4b5c0c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b61cf0640 1 -- 172.21.15.26:0/1238818155 >> v1:172.21.15.26:6789/0 conn(0x7f4b5c1527b0 legacy=0x7f4b5c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b61cf0640 1 -- 172.21.15.26:0/1238818155 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c0c76a0 con 0x7f4b5c0c7180 2024-03-06T07:26:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b62cf2640 1 --2- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c88b0 0x7f4b5c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.434+0000 7f4b61cf0640 1 --2- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c0c7180 0x7f4b5c159100 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0097a0 tx=0x7f4b4c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6bba4f817ac0470 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b614ef640 1 -- 172.21.15.26:0/1238818155 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0315a0 con 0x7f4b5c0c7180 2024-03-06T07:26:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b614ef640 1 -- 172.21.15.26:0/1238818155 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c037070 con 0x7f4b5c0c7180 2024-03-06T07:26:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b614ef640 1 -- 172.21.15.26:0/1238818155 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c031940 con 0x7f4b5c0c7180 2024-03-06T07:26:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 -- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c0c7180 msgr2=0x7f4b5c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 --2- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c0c7180 0x7f4b5c159100 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0097a0 tx=0x7f4b4c02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 -- 172.21.15.26:0/1238818155 shutdown_connections 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 --2- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c0c7180 0x7f4b5c159100 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 --2- 172.21.15.26:0/1238818155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c88b0 0x7f4b5c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 -- 172.21.15.26:0/1238818155 >> 172.21.15.26:0/1238818155 conn(0x7f4b5c0ac560 msgr2=0x7f4b5c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 -- 172.21.15.26:0/1238818155 shutdown_connections 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.435+0000 7f4b64755640 1 -- 172.21.15.26:0/1238818155 wait complete. 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.436+0000 7f4b64755640 1 Processor -- start 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.436+0000 7f4b64755640 1 -- start start 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b64755640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c7180 0x7f4b5c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b64755640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0c88b0 0x7f4b5c1eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b64755640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c1527b0 0x7f4b5c1eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b64755640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c155600 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b64755640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c155320 con 0x7f4b5c1527b0 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b64755640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c0c8d60 con 0x7f4b5c0c7180 2024-03-06T07:26:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0c88b0 0x7f4b5c1eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0c88b0 0x7f4b5c1eac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42704/0 (socket says 172.21.15.26:42704) 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 -- 172.21.15.26:0/3947446609 learned_addr learned my addr 172.21.15.26:0/3947446609 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b624f1640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c7180 0x7f4b5c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b62cf2640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c1527b0 0x7f4b5c1eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 -- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c7180 msgr2=0x7f4b5c1543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c7180 0x7f4b5c1543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 -- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c1527b0 msgr2=0x7f4b5c1eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c1527b0 0x7f4b5c1eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.437+0000 7f4b61cf0640 1 -- 172.21.15.26:0/3947446609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c0b00d0 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.438+0000 7f4b61cf0640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0c88b0 0x7f4b5c1eac80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c034040 tx=0x7f4b4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.438+0000 7f4b52ffd640 1 -- 172.21.15.26:0/3947446609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c032430 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.438+0000 7f4b52ffd640 1 -- 172.21.15.26:0/3947446609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c0325d0 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.438+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c0af2a0 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.438+0000 7f4b52ffd640 1 -- 172.21.15.26:0/3947446609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0329a0 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.438+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c154930 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.439+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c04f330 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.439+0000 7f4b52ffd640 1 -- 172.21.15.26:0/3947446609 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b4c032b40 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.439+0000 7f4b52ffd640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b4006b500 0x7f4b4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.440+0000 7f4b52ffd640 1 -- 172.21.15.26:0/3947446609 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f4b4c02f030 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.440+0000 7f4b624f1640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b4006b500 0x7f4b4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.440+0000 7f4b624f1640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b4006b500 0x7f4b4006d9c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4b58005880 tx=0x7f4b58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.444+0000 7f4b52ffd640 1 -- 172.21.15.26:0/3947446609 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b4c040070 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.661+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b5c06ce00 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.662+0000 7f4b52ffd640 1 -- 172.21.15.26:0/3947446609 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+6271 (secure 0 0 0) 0x7f4b4c07ac90 con 0x7f4b5c0c88b0 2024-03-06T07:26:28.665 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:28.665 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":148,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:26.887138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7178,"qdb_cluster":[7178]},"id":11},{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:25.766406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 148 2024-03-06T07:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b4006b500 msgr2=0x7f4b4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b4006b500 0x7f4b4006d9c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4b58005880 tx=0x7f4b58009040 comp rx=0 tx=0).stop 2024-03-06T07:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0c88b0 msgr2=0x7f4b5c1eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0c88b0 0x7f4b5c1eac80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c034040 tx=0x7f4b4c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 shutdown_connections 2024-03-06T07:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b4006b500 0x7f4b4006d9c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b5c1527b0 0x7f4b5c1eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0c88b0 0x7f4b5c1eac80 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 --2- 172.21.15.26:0/3947446609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5c0c7180 0x7f4b5c1543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 >> 172.21.15.26:0/3947446609 conn(0x7f4b5c0ac560 msgr2=0x7f4b5c0c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 shutdown_connections 2024-03-06T07:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.664+0000 7f4b64755640 1 -- 172.21.15.26:0/3947446609 wait complete. 2024-03-06T07:26:28.677 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:26:28.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:28.717 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:28.764 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:26:28.765 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:26:28.765 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:28.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:26:28.779 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:26:28.779 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:28.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:26:28.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.918+0000 7fdce19a0640 1 Processor -- start 2024-03-06T07:26:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.918+0000 7fdce19a0640 1 -- start start 2024-03-06T07:26:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce19a0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdcdc1526d0 0x7fdcdc155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce19a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcdc0c87d0 0x7fdcdc0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce19a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0cb120 0x7fdcdc0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce19a0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0c8ce0 con 0x7fdcdc0c87d0 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce19a0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0c71e0 con 0x7fdcdc1526d0 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce19a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0c5460 con 0x7fdcdc0cb120 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdcdbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcdc0c87d0 0x7fdcdc0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdcdbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcdc0c87d0 0x7fdcdc0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42718/0 (socket says 172.21.15.26:42718) 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdcdbfff640 1 -- 172.21.15.26:0/2137662656 learned_addr learned my addr 172.21.15.26:0/2137662656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce119f640 1 --2- 172.21.15.26:0/2137662656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0cb120 0x7fdcdc0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.919+0000 7fdce099e640 1 --2- 172.21.15.26:0/2137662656 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc1526d0 0x7fdcdc155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdbfff640 1 -- 172.21.15.26:0/2137662656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0cb120 msgr2=0x7fdcdc0c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdbfff640 1 --2- 172.21.15.26:0/2137662656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0cb120 0x7fdcdc0c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdbfff640 1 -- 172.21.15.26:0/2137662656 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc1526d0 msgr2=0x7fdcdc155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdbfff640 1 --2- 172.21.15.26:0/2137662656 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc1526d0 0x7fdcdc155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdbfff640 1 -- 172.21.15.26:0/2137662656 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0c5740 con 0x7fdcdc0c87d0 2024-03-06T07:26:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdbfff640 1 --2- 172.21.15.26:0/2137662656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc0c87d0 0x7fdcdc0cabe0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ad540 tx=0x7fdcd002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9475d045368f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdb7fe640 1 -- 172.21.15.26:0/2137662656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0035020 con 0x7fdcdc0c87d0 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.920+0000 7fdcdb7fe640 1 -- 172.21.15.26:0/2137662656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0031870 con 0x7fdcdc0c87d0 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdcdb7fe640 1 -- 172.21.15.26:0/2137662656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0031c40 con 0x7fdcdc0c87d0 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 -- 172.21.15.26:0/2137662656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc0c87d0 msgr2=0x7fdcdc0cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 --2- 172.21.15.26:0/2137662656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc0c87d0 0x7fdcdc0cabe0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ad540 tx=0x7fdcd002b910 comp rx=0 tx=0).stop 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 -- 172.21.15.26:0/2137662656 shutdown_connections 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 --2- 172.21.15.26:0/2137662656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0cb120 0x7fdcdc0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 --2- 172.21.15.26:0/2137662656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc0c87d0 0x7fdcdc0cabe0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 --2- 172.21.15.26:0/2137662656 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc1526d0 0x7fdcdc155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 -- 172.21.15.26:0/2137662656 >> 172.21.15.26:0/2137662656 conn(0x7fdcdc0728d0 msgr2=0x7fdcdc072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 -- 172.21.15.26:0/2137662656 shutdown_connections 2024-03-06T07:26:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.921+0000 7fdce19a0640 1 -- 172.21.15.26:0/2137662656 wait complete. 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.922+0000 7fdce19a0640 1 Processor -- start 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.922+0000 7fdce19a0640 1 -- start start 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce19a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0c87d0 0x7fdcdc0c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce19a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc0cb120 0x7fdcdc1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce19a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc1526d0 0x7fdcdc148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce19a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc152ab0 con 0x7fdcdc1526d0 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce19a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc06d060 con 0x7fdcdc0cb120 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce19a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0c7bb0 con 0x7fdcdc0c87d0 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc1526d0 0x7fdcdc148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc1526d0 0x7fdcdc148ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42730/0 (socket says 172.21.15.26:42730) 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 -- 172.21.15.26:0/339256278 learned_addr learned my addr 172.21.15.26:0/339256278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce099e640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0c87d0 0x7fdcdc0c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdcdbfff640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc0cb120 0x7fdcdc1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 -- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0c87d0 msgr2=0x7fdcdc0c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0c87d0 0x7fdcdc0c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 -- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc0cb120 msgr2=0x7fdcdc1556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc0cb120 0x7fdcdc1556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.923+0000 7fdce119f640 1 -- 172.21.15.26:0/339256278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0c24e0 con 0x7fdcdc1526d0 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.924+0000 7fdce119f640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc1526d0 0x7fdcdc148ba0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00e830 tx=0x7fdccc00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.924+0000 7fdcd97fa640 1 -- 172.21.15.26:0/339256278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc017070 con 0x7fdcdc1526d0 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.924+0000 7fdcd97fa640 1 -- 172.21.15.26:0/339256278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc00c940 con 0x7fdcdc1526d0 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.924+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc0c27c0 con 0x7fdcdc1526d0 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.924+0000 7fdcd97fa640 1 -- 172.21.15.26:0/339256278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00cd30 con 0x7fdcdc1526d0 2024-03-06T07:26:28.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.924+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc0c48a0 con 0x7fdcdc1526d0 2024-03-06T07:26:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.925+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc06b090 con 0x7fdcdc1526d0 2024-03-06T07:26:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.925+0000 7fdcd97fa640 1 -- 172.21.15.26:0/339256278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdccc01a070 con 0x7fdcdc1526d0 2024-03-06T07:26:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.926+0000 7fdcd97fa640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcc806b4d0 0x7fdcc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.926+0000 7fdcd97fa640 1 -- 172.21.15.26:0/339256278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fdccc08a640 con 0x7fdcdc1526d0 2024-03-06T07:26:28.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.926+0000 7fdce099e640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcc806b4d0 0x7fdcc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.926+0000 7fdce099e640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcc806b4d0 0x7fdcc806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4011340 tx=0x7fdcd400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:28.931+0000 7fdcd97fa640 1 -- 172.21.15.26:0/339256278 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdcdc06b090 con 0x7fdcdc1526d0 2024-03-06T07:26:29.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.148+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcdc069430 con 0x7fdcdc1526d0 2024-03-06T07:26:29.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.148+0000 7fdcd97fa640 1 -- 172.21.15.26:0/339256278 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+6271 (secure 0 0 0) 0x7fdccc058930 con 0x7fdcdc1526d0 2024-03-06T07:26:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:29.151 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":148,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:26.887138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7178,"qdb_cluster":[7178]},"id":11},{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:25.766406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:29.151 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 148 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcc806b4d0 msgr2=0x7fdcc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcc806b4d0 0x7fdcc806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4011340 tx=0x7fdcd400b040 comp rx=0 tx=0).stop 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc1526d0 msgr2=0x7fdcdc148ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc1526d0 0x7fdcdc148ba0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00e830 tx=0x7fdccc00ecf0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 shutdown_connections 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcc806b4d0 0x7fdcc806d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcdc1526d0 0x7fdcdc148ba0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc0cb120 0x7fdcdc1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 --2- 172.21.15.26:0/339256278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc0c87d0 0x7fdcdc0c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 >> 172.21.15.26:0/339256278 conn(0x7fdcdc0728d0 msgr2=0x7fdcdc1517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 shutdown_connections 2024-03-06T07:26:29.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.150+0000 7fdce19a0640 1 -- 172.21.15.26:0/339256278 wait complete. 2024-03-06T07:26:29.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.263+0000 7fae2f30c640 1 Processor -- start 2024-03-06T07:26:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.263+0000 7fae2f30c640 1 -- start start 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2f30c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fae2814f310 0x7fae280c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2f30c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae28155bb0 con 0x7fae281550c0 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2f30c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae280aa660 con 0x7fae280aa180 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2f30c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae280c4610 con 0x7fae2814f310 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2d8a9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fae2814f310 0x7fae280c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2d0a8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae281550c0 0x7fae281554a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49490/0 (socket says 172.21.15.26:49490) 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2d0a8640 1 -- 172.21.15.26:0/1132435599 learned_addr learned my addr 172.21.15.26:0/1132435599 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae1ffff640 1 -- 172.21.15.26:0/1132435599 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985439430 0 0) 0x7fae28155bb0 con 0x7fae281550c0 2024-03-06T07:26:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae1ffff640 1 -- 172.21.15.26:0/1132435599 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae14003520 con 0x7fae281550c0 2024-03-06T07:26:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2d8a9640 1 -- 172.21.15.26:0/1132435599 >> v1:172.21.15.117:6789/0 conn(0x7fae280aa180 legacy=0x7fae2814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae1ffff640 1 -- 172.21.15.26:0/1132435599 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089411182 0 0) 0x7fae280aa660 con 0x7fae280aa180 2024-03-06T07:26:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2d8a9640 1 -- 172.21.15.26:0/1132435599 >> v1:172.21.15.26:6789/0 conn(0x7fae281550c0 legacy=0x7fae281554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2d8a9640 1 -- 172.21.15.26:0/1132435599 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280aa940 con 0x7fae2814f310 2024-03-06T07:26:29.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae2d8a9640 1 --2- 172.21.15.26:0/1132435599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2814f310 0x7fae280c5060 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fae2000aea0 tx=0x7fae2002cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c329dfdb57ab1efb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae1ffff640 1 -- 172.21.15.26:0/1132435599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2002dac0 con 0x7fae2814f310 2024-03-06T07:26:29.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae1ffff640 1 -- 172.21.15.26:0/1132435599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2002dc60 con 0x7fae2814f310 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.265+0000 7fae1ffff640 1 -- 172.21.15.26:0/1132435599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20008660 con 0x7fae2814f310 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.266+0000 7fae2f30c640 1 -- 172.21.15.26:0/1132435599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2814f310 msgr2=0x7fae280c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.266+0000 7fae2f30c640 1 --2- 172.21.15.26:0/1132435599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2814f310 0x7fae280c5060 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fae2000aea0 tx=0x7fae2002cf20 comp rx=0 tx=0).stop 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.266+0000 7fae2f30c640 1 -- 172.21.15.26:0/1132435599 shutdown_connections 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.266+0000 7fae2f30c640 1 --2- 172.21.15.26:0/1132435599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2814f310 0x7fae280c5060 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.266+0000 7fae2f30c640 1 -- 172.21.15.26:0/1132435599 >> 172.21.15.26:0/1132435599 conn(0x7fae280c6710 msgr2=0x7fae280c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.266+0000 7fae2f30c640 1 -- 172.21.15.26:0/1132435599 shutdown_connections 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.266+0000 7fae2f30c640 1 -- 172.21.15.26:0/1132435599 wait complete. 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.268+0000 7fae2f30c640 1 Processor -- start 2024-03-06T07:26:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.268+0000 7fae2f30c640 1 -- start start 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.268+0000 7fae2f30c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae280aa180 0x7fae2815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.268+0000 7fae2f30c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae2814f310 0x7fae281eb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2f30c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae281550c0 0x7fae2814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2f30c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c1490 con 0x7fae280aa180 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2f30c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c1310 con 0x7fae281550c0 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2f30c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c3210 con 0x7fae2814f310 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae280aa180 0x7fae2815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae280aa180 0x7fae2815dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42746/0 (socket says 172.21.15.26:42746) 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 -- 172.21.15.26:0/196424381 learned_addr learned my addr 172.21.15.26:0/196424381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2c8a7640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae2814f310 0x7fae281eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d8a9640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae281550c0 0x7fae2814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 -- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae2814f310 msgr2=0x7fae281eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae2814f310 0x7fae281eb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 -- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae281550c0 msgr2=0x7fae2814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae281550c0 0x7fae2814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 -- 172.21.15.26:0/196424381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2815ee40 con 0x7fae280aa180 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2d0a8640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae280aa180 0x7fae2815dd60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c3c0 tx=0x7fae2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae1dffb640 1 -- 172.21.15.26:0/196424381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24017070 con 0x7fae280aa180 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae1dffb640 1 -- 172.21.15.26:0/196424381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24005ce0 con 0x7fae280aa180 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2815d280 con 0x7fae280aa180 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae1dffb640 1 -- 172.21.15.26:0/196424381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240072f0 con 0x7fae280aa180 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2815d840 con 0x7fae280aa180 2024-03-06T07:26:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.269+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae280c2bd0 con 0x7fae280aa180 2024-03-06T07:26:29.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.270+0000 7fae1dffb640 1 -- 172.21.15.26:0/196424381 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae2401a070 con 0x7fae280aa180 2024-03-06T07:26:29.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.270+0000 7fae1dffb640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae0c06b4d0 0x7fae0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.270+0000 7fae1dffb640 1 -- 172.21.15.26:0/196424381 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fae24089a50 con 0x7fae280aa180 2024-03-06T07:26:29.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.271+0000 7fae2c8a7640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae0c06b4d0 0x7fae0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.271+0000 7fae2c8a7640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae0c06b4d0 0x7fae0c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fae28152a10 tx=0x7fae18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.275+0000 7fae1dffb640 1 -- 172.21.15.26:0/196424381 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae280c2bd0 con 0x7fae280aa180 2024-03-06T07:26:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.455+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae2804f330 con 0x7fae280aa180 2024-03-06T07:26:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.456+0000 7fae1dffb640 1 -- 172.21.15.26:0/196424381 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+20743 (secure 0 0 0) 0x7fae2405e080 con 0x7fae280aa180 2024-03-06T07:26:29.459 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:29.459 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":156,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:23.875901+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6833/3029467671":"2024-03-07T07:26:23.875878+0000","172.21.15.26:6832/3029467671":"2024-03-07T07:26:23.875878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae0c06b4d0 msgr2=0x7fae0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae0c06b4d0 0x7fae0c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fae28152a10 tx=0x7fae18009040 comp rx=0 tx=0).stop 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae280aa180 msgr2=0x7fae2815dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae280aa180 0x7fae2815dd60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c3c0 tx=0x7fae2400c880 comp rx=0 tx=0).stop 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 shutdown_connections 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae0c06b4d0 0x7fae0c06d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae281550c0 0x7fae2814cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae2814f310 0x7fae281eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 --2- 172.21.15.26:0/196424381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae280aa180 0x7fae2815dd60 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 >> 172.21.15.26:0/196424381 conn(0x7fae280c6710 msgr2=0x7fae2814d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 shutdown_connections 2024-03-06T07:26:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.459+0000 7fae2f30c640 1 -- 172.21.15.26:0/196424381 wait complete. 2024-03-06T07:26:29.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:26:05.339258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:26:06.016366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:26:15.400219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:26:16.061363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:29.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd29b5e640 1 Processor -- start 2024-03-06T07:26:29.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd29b5e640 1 -- start start 2024-03-06T07:26:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd29b5e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcd2406c900 0x7fcd240c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd29b5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd240c3ef0 con 0x7fcd240c5990 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd29b5e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd241556d0 con 0x7fcd240ab170 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd29b5e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd241559b0 con 0x7fcd2406c900 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd237fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcd240c5990 0x7fcd2406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49494/0 (socket says 172.21.15.26:49494) 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd237fe640 1 -- 172.21.15.26:0/2475516579 learned_addr learned my addr 172.21.15.26:0/2475516579 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd22ffd640 1 --2- 172.21.15.26:0/2475516579 >> v2:172.21.15.117:3300/0 conn(0x7fcd2406c900 0x7fcd240c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd227fc640 1 -- 172.21.15.26:0/2475516579 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322333652 0 0) 0x7fcd240c3ef0 con 0x7fcd240c5990 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd227fc640 1 -- 172.21.15.26:0/2475516579 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd0c003520 con 0x7fcd240c5990 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd227fc640 1 -- 172.21.15.26:0/2475516579 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145550272 0 0) 0x7fcd241556d0 con 0x7fcd240ab170 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd227fc640 1 -- 172.21.15.26:0/2475516579 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd240c3ef0 con 0x7fcd240ab170 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd22ffd640 1 -- 172.21.15.26:0/2475516579 >> v1:172.21.15.117:6789/0 conn(0x7fcd240ab170 legacy=0x7fcd240a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd22ffd640 1 -- 172.21.15.26:0/2475516579 >> v1:172.21.15.26:6789/0 conn(0x7fcd240c5990 legacy=0x7fcd2406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.575+0000 7fcd22ffd640 1 -- 172.21.15.26:0/2475516579 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24155c90 con 0x7fcd2406c900 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.576+0000 7fcd22ffd640 1 --2- 172.21.15.26:0/2475516579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406c900 0x7fcd240c0ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcd180097d0 tx=0x7fcd1802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f58a490abd68931 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.576+0000 7fcd227fc640 1 -- 172.21.15.26:0/2475516579 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18034070 con 0x7fcd2406c900 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.576+0000 7fcd227fc640 1 -- 172.21.15.26:0/2475516579 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18030610 con 0x7fcd2406c900 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.576+0000 7fcd227fc640 1 -- 172.21.15.26:0/2475516579 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd180309e0 con 0x7fcd2406c900 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.576+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2475516579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406c900 msgr2=0x7fcd240c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.577+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2475516579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406c900 0x7fcd240c0ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcd180097d0 tx=0x7fcd1802b960 comp rx=0 tx=0).stop 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.577+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2475516579 shutdown_connections 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.577+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2475516579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406c900 0x7fcd240c0ab0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.577+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2475516579 >> 172.21.15.26:0/2475516579 conn(0x7fcd240c6710 msgr2=0x7fcd240c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.577+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2475516579 shutdown_connections 2024-03-06T07:26:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.577+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2475516579 wait complete. 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.578+0000 7fcd29b5e640 1 Processor -- start 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.578+0000 7fcd29b5e640 1 -- start start 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.578+0000 7fcd29b5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2406c900 0x7fcd241ea760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd29b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd240ab170 0x7fcd241e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd29b5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd240c5990 0x7fcd241512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd29b5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd24156610 con 0x7fcd2406c900 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd29b5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd24156490 con 0x7fcd240ab170 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd29b5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240c3af0 con 0x7fcd240c5990 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2406c900 0x7fcd241ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2406c900 0x7fcd241ea760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42762/0 (socket says 172.21.15.26:42762) 2024-03-06T07:26:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 -- 172.21.15.26:0/2928206173 learned_addr learned my addr 172.21.15.26:0/2928206173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd22ffd640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd240ab170 0x7fcd241e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd23fff640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd240c5990 0x7fcd241512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 -- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd240c5990 msgr2=0x7fcd241512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd240c5990 0x7fcd241512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 -- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd240ab170 msgr2=0x7fcd241e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd240ab170 0x7fcd241e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 -- 172.21.15.26:0/2928206173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd241e6800 con 0x7fcd2406c900 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd237fe640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2406c900 0x7fcd241ea760 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c410 tx=0x7fcd1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd0bfff640 1 -- 172.21.15.26:0/2928206173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14017070 con 0x7fcd2406c900 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd0bfff640 1 -- 172.21.15.26:0/2928206173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd14005ce0 con 0x7fcd2406c900 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd24154df0 con 0x7fcd2406c900 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd0bfff640 1 -- 172.21.15.26:0/2928206173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd140072d0 con 0x7fcd2406c900 2024-03-06T07:26:29.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.579+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd241e6a80 con 0x7fcd2406c900 2024-03-06T07:26:29.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.580+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2404f330 con 0x7fcd2406c900 2024-03-06T07:26:29.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.581+0000 7fcd0bfff640 1 -- 172.21.15.26:0/2928206173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd1401a070 con 0x7fcd2406c900 2024-03-06T07:26:29.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.581+0000 7fcd0bfff640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd1006b4d0 0x7fcd1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.581+0000 7fcd0bfff640 1 -- 172.21.15.26:0/2928206173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fcd140899b0 con 0x7fcd2406c900 2024-03-06T07:26:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.582+0000 7fcd22ffd640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd1006b4d0 0x7fcd1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.582+0000 7fcd22ffd640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd1006b4d0 0x7fcd1006d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcd18036000 tx=0x7fcd1803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.586+0000 7fcd0bfff640 1 -- 172.21.15.26:0/2928206173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcd241e6800 con 0x7fcd2406c900 2024-03-06T07:26:29.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.768+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd240c1740 con 0x7fcd2406c900 2024-03-06T07:26:29.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.769+0000 7fcd0bfff640 1 -- 172.21.15.26:0/2928206173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd14057d30 con 0x7fcd2406c900 2024-03-06T07:26:29.771 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd1006b4d0 msgr2=0x7fcd1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd1006b4d0 0x7fcd1006d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcd18036000 tx=0x7fcd1803b040 comp rx=0 tx=0).stop 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2406c900 msgr2=0x7fcd241ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2406c900 0x7fcd241ea760 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c410 tx=0x7fcd1400c8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 shutdown_connections 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcd1006b4d0 0x7fcd1006d990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd240c5990 0x7fcd241512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd240ab170 0x7fcd241e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 --2- 172.21.15.26:0/2928206173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2406c900 0x7fcd241ea760 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 >> 172.21.15.26:0/2928206173 conn(0x7fcd240c6710 msgr2=0x7fcd240c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 shutdown_connections 2024-03-06T07:26:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.771+0000 7fcd29b5e640 1 -- 172.21.15.26:0/2928206173 wait complete. 2024-03-06T07:26:29.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:26:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.886+0000 7fe27d3d6640 1 Processor -- start 2024-03-06T07:26:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.886+0000 7fe27d3d6640 1 -- start start 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe27d3d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe278155c30 0x7fe27806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe27d3d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe2780c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe27d3d6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780b0450 con 0x7fe2780b0070 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe27d3d6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27806cfb0 con 0x7fe278155c30 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe27d3d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806c020 con 0x7fe2780c5380 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe2767fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe278155c30 0x7fe27806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe276ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe2780b0070 0x7fe2780c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49502/0 (socket says 172.21.15.26:49502) 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe276ffd640 1 -- 172.21.15.26:0/469014180 learned_addr learned my addr 172.21.15.26:0/469014180 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe2777fe640 1 --2- 172.21.15.26:0/469014180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe2780c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe2767fc640 1 -- 172.21.15.26:0/469014180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 msgr2=0x7fe2780c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe2767fc640 1 --2- 172.21.15.26:0/469014180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe2780c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe2767fc640 1 -- 172.21.15.26:0/469014180 >> v1:172.21.15.26:6789/0 conn(0x7fe2780b0070 legacy=0x7fe2780c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe2767fc640 1 -- 172.21.15.26:0/469014180 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2781560e0 con 0x7fe278155c30 2024-03-06T07:26:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.887+0000 7fe275ffb640 1 -- 172.21.15.26:0/469014180 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357357684 0 0) 0x7fe270001110 con 0x7fe2780b0070 2024-03-06T07:26:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.888+0000 7fe2767fc640 1 --2- 172.21.15.26:0/469014180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe278155c30 0x7fe27806ca70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0097a0 tx=0x7fe26c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824f8316db12b936 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.888+0000 7fe275ffb640 1 -- 172.21.15.26:0/469014180 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0315a0 con 0x7fe278155c30 2024-03-06T07:26:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.888+0000 7fe275ffb640 1 -- 172.21.15.26:0/469014180 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c037070 con 0x7fe278155c30 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.888+0000 7fe275ffb640 1 -- 172.21.15.26:0/469014180 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c031960 con 0x7fe278155c30 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 -- 172.21.15.26:0/469014180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe278155c30 msgr2=0x7fe27806ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/469014180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe278155c30 0x7fe27806ca70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0097a0 tx=0x7fe26c02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 -- 172.21.15.26:0/469014180 shutdown_connections 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/469014180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe278155c30 0x7fe27806ca70 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/469014180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe2780c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 -- 172.21.15.26:0/469014180 >> 172.21.15.26:0/469014180 conn(0x7fe2780ac590 msgr2=0x7fe2780ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 -- 172.21.15.26:0/469014180 shutdown_connections 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.889+0000 7fe27d3d6640 1 -- 172.21.15.26:0/469014180 wait complete. 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.890+0000 7fe27d3d6640 1 Processor -- start 2024-03-06T07:26:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.890+0000 7fe27d3d6640 1 -- start start 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2780b0070 0x7fe278158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe27814e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe278155c30 0x7fe27814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806c270 con 0x7fe2780b0070 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780c7ad0 con 0x7fe278155c30 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806c550 con 0x7fe2780c5380 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2780b0070 0x7fe278158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2780b0070 0x7fe278158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42778/0 (socket says 172.21.15.26:42778) 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 -- 172.21.15.26:0/3897495351 learned_addr learned my addr 172.21.15.26:0/3897495351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe2777fe640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe278155c30 0x7fe27814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe2767fc640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe27814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 -- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 msgr2=0x7fe27814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe27814e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 -- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe278155c30 msgr2=0x7fe27814ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe278155c30 0x7fe27814ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 -- 172.21.15.26:0/3897495351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe278156f90 con 0x7fe2780b0070 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe276ffd640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2780b0070 0x7fe278158d30 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c430 tx=0x7fe27000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe25f7fe640 1 -- 172.21.15.26:0/3897495351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700074e0 con 0x7fe2780b0070 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe25f7fe640 1 -- 172.21.15.26:0/3897495351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2700047e0 con 0x7fe2780b0070 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe278157270 con 0x7fe2780b0070 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe25f7fe640 1 -- 172.21.15.26:0/3897495351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270004bd0 con 0x7fe2780b0070 2024-03-06T07:26:29.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.891+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe278157830 con 0x7fe2780b0070 2024-03-06T07:26:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.892+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27804f330 con 0x7fe2780b0070 2024-03-06T07:26:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.893+0000 7fe25f7fe640 1 -- 172.21.15.26:0/3897495351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe270011070 con 0x7fe2780b0070 2024-03-06T07:26:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.893+0000 7fe25f7fe640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe26406b4d0 0x7fe26406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.893+0000 7fe25f7fe640 1 -- 172.21.15.26:0/3897495351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fe27008a1e0 con 0x7fe2780b0070 2024-03-06T07:26:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.893+0000 7fe2767fc640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe26406b4d0 0x7fe26406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.894+0000 7fe2767fc640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe26406b4d0 0x7fe26406d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe26c036000 tx=0x7fe26c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:29.898+0000 7fe25f7fe640 1 -- 172.21.15.26:0/3897495351 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe2780b0450 con 0x7fe2780b0070 2024-03-06T07:26:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.079+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe27806d860 con 0x7fe2780b0070 2024-03-06T07:26:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.079+0000 7fe25f7fe640 1 -- 172.21.15.26:0/3897495351 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe270058560 con 0x7fe2780b0070 2024-03-06T07:26:30.081 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:26:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe26406b4d0 msgr2=0x7fe26406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe26406b4d0 0x7fe26406d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe26c036000 tx=0x7fe26c03d040 comp rx=0 tx=0).stop 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2780b0070 msgr2=0x7fe278158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2780b0070 0x7fe278158d30 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c430 tx=0x7fe27000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 shutdown_connections 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe26406b4d0 0x7fe26406d990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe278155c30 0x7fe27814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2780c5380 0x7fe27814e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 --2- 172.21.15.26:0/3897495351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2780b0070 0x7fe278158d30 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 >> 172.21.15.26:0/3897495351 conn(0x7fe2780ac590 msgr2=0x7fe2780ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 shutdown_connections 2024-03-06T07:26:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.082+0000 7fe27d3d6640 1 -- 172.21.15.26:0/3897495351 wait complete. 2024-03-06T07:26:30.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:26:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.197+0000 7fcb67e39640 1 Processor -- start 2024-03-06T07:26:30.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.197+0000 7fcb67e39640 1 -- start start 2024-03-06T07:26:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb67e39640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb601529f0 0x7fcb600c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb67e39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb600c5370 0x7fcb600c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb67e39640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb60152dd0 con 0x7fcb60155530 2024-03-06T07:26:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb67e39640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb60159710 con 0x7fcb601529f0 2024-03-06T07:26:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb67e39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600c68b0 con 0x7fcb600c5370 2024-03-06T07:26:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb65bd5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb601529f0 0x7fcb600c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb653d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb600c5370 0x7fcb600c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb663d6640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fcb60155530 0x7fcb601559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60360/0 (socket says 172.21.15.26:60360) 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb663d6640 1 -- 172.21.15.26:0/3727032786 learned_addr learned my addr 172.21.15.26:0/3727032786 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb653d4640 1 -- 172.21.15.26:0/3727032786 >> v2:172.21.15.117:3300/0 conn(0x7fcb601529f0 msgr2=0x7fcb600c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb653d4640 1 --2- 172.21.15.26:0/3727032786 >> v2:172.21.15.117:3300/0 conn(0x7fcb601529f0 0x7fcb600c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.198+0000 7fcb653d4640 1 -- 172.21.15.26:0/3727032786 >> v1:172.21.15.117:6789/0 conn(0x7fcb60155530 legacy=0x7fcb601559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.199+0000 7fcb64bd3640 1 -- 172.21.15.26:0/3727032786 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320198004 0 0) 0x7fcb60152dd0 con 0x7fcb60155530 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.199+0000 7fcb653d4640 1 -- 172.21.15.26:0/3727032786 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60158b00 con 0x7fcb600c5370 2024-03-06T07:26:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.199+0000 7fcb65bd5640 1 --2- 172.21.15.26:0/3727032786 >> v2:172.21.15.117:3300/0 conn(0x7fcb601529f0 0x7fcb600c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.199+0000 7fcb653d4640 1 --2- 172.21.15.26:0/3727032786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb600c5370 0x7fcb600c57d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb540097a0 tx=0x7fcb5402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12702a63ba1dc179 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.199+0000 7fcb64bd3640 1 -- 172.21.15.26:0/3727032786 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540315a0 con 0x7fcb600c5370 2024-03-06T07:26:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb64bd3640 1 -- 172.21.15.26:0/3727032786 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54037070 con 0x7fcb600c5370 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb64bd3640 1 -- 172.21.15.26:0/3727032786 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54031960 con 0x7fcb600c5370 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 -- 172.21.15.26:0/3727032786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb600c5370 msgr2=0x7fcb600c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3727032786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb600c5370 0x7fcb600c57d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb540097a0 tx=0x7fcb5402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 -- 172.21.15.26:0/3727032786 shutdown_connections 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3727032786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb600c5370 0x7fcb600c57d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3727032786 >> v2:172.21.15.117:3300/0 conn(0x7fcb601529f0 0x7fcb600c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 -- 172.21.15.26:0/3727032786 >> 172.21.15.26:0/3727032786 conn(0x7fcb600ac580 msgr2=0x7fcb600ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 -- 172.21.15.26:0/3727032786 shutdown_connections 2024-03-06T07:26:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.200+0000 7fcb67e39640 1 -- 172.21.15.26:0/3727032786 wait complete. 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.201+0000 7fcb67e39640 1 Processor -- start 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.201+0000 7fcb67e39640 1 -- start start 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb67e39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb600c5370 0x7fcb6014c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb67e39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb601529f0 0x7fcb601ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb67e39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60155530 0x7fcb600cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb67e39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601593a0 con 0x7fcb60155530 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb67e39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601590c0 con 0x7fcb600c5370 2024-03-06T07:26:30.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb67e39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601560d0 con 0x7fcb601529f0 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb663d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60155530 0x7fcb600cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb663d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60155530 0x7fcb600cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42782/0 (socket says 172.21.15.26:42782) 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb663d6640 1 -- 172.21.15.26:0/3882102980 learned_addr learned my addr 172.21.15.26:0/3882102980 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb65bd5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb600c5370 0x7fcb6014c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.202+0000 7fcb653d4640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb601529f0 0x7fcb601ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb663d6640 1 -- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb601529f0 msgr2=0x7fcb601ee4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb663d6640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb601529f0 0x7fcb601ee4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb663d6640 1 -- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb600c5370 msgr2=0x7fcb6014c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb663d6640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb600c5370 0x7fcb6014c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb663d6640 1 -- 172.21.15.26:0/3882102980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6014b960 con 0x7fcb60155530 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb65bd5640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb600c5370 0x7fcb6014c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb663d6640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60155530 0x7fcb600cb200 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c700 tx=0x7fcb5000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb5e7fc640 1 -- 172.21.15.26:0/3882102980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50018070 con 0x7fcb60155530 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb5e7fc640 1 -- 172.21.15.26:0/3882102980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5000ce10 con 0x7fcb60155530 2024-03-06T07:26:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6014bc40 con 0x7fcb60155530 2024-03-06T07:26:30.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb5e7fc640 1 -- 172.21.15.26:0/3882102980 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000a860 con 0x7fcb60155530 2024-03-06T07:26:30.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.203+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb600cb740 con 0x7fcb60155530 2024-03-06T07:26:30.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.204+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6004f330 con 0x7fcb60155530 2024-03-06T07:26:30.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.204+0000 7fcb5e7fc640 1 -- 172.21.15.26:0/3882102980 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb5001b070 con 0x7fcb60155530 2024-03-06T07:26:30.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.205+0000 7fcb5e7fc640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb4c06b500 0x7fcb4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.205+0000 7fcb5e7fc640 1 -- 172.21.15.26:0/3882102980 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fcb50089bb0 con 0x7fcb60155530 2024-03-06T07:26:30.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.205+0000 7fcb65bd5640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb4c06b500 0x7fcb4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.205+0000 7fcb65bd5640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb4c06b500 0x7fcb4c06d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009560 tx=0x7fcb58009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.209+0000 7fcb5e7fc640 1 -- 172.21.15.26:0/3882102980 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb50057f30 con 0x7fcb60155530 2024-03-06T07:26:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.389+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb600c8890 con 0x7fcb60155530 2024-03-06T07:26:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.389+0000 7fcb5e7fc640 1 -- 172.21.15.26:0/3882102980 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb5005bfd0 con 0x7fcb60155530 2024-03-06T07:26:30.391 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb4c06b500 msgr2=0x7fcb4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb4c06b500 0x7fcb4c06d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009560 tx=0x7fcb58009190 comp rx=0 tx=0).stop 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60155530 msgr2=0x7fcb600cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60155530 0x7fcb600cb200 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c700 tx=0x7fcb5000cbc0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 shutdown_connections 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb4c06b500 0x7fcb4c06d9c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60155530 0x7fcb600cb200 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb601529f0 0x7fcb601ee4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 --2- 172.21.15.26:0/3882102980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb600c5370 0x7fcb6014c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 >> 172.21.15.26:0/3882102980 conn(0x7fcb600ac580 msgr2=0x7fcb600aac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 shutdown_connections 2024-03-06T07:26:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.391+0000 7fcb67e39640 1 -- 172.21.15.26:0/3882102980 wait complete. 2024-03-06T07:26:30.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:26:30.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.506+0000 7fc1cc4c2640 1 Processor -- start 2024-03-06T07:26:30.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.506+0000 7fc1cc4c2640 1 -- start start 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1cc4c2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1c4155bd0 0x7fc1c4152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1cc4c2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c40670c0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1cc4c2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c406b0f0 con 0x7fc1c40c26c0 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1cc4c2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c4156060 con 0x7fc1c4155bd0 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1c9a5d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1c40c2aa0 0x7fc1c40c0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49506/0 (socket says 172.21.15.26:49506) 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1c9a5d640 1 -- 172.21.15.26:0/2881001431 learned_addr learned my addr 172.21.15.26:0/2881001431 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1caa5f640 1 --2- 172.21.15.26:0/2881001431 >> v2:172.21.15.117:3300/0 conn(0x7fc1c4155bd0 0x7fc1c4152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1c925c640 1 -- 172.21.15.26:0/2881001431 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053418446 0 0) 0x7fc1c40670c0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1c925c640 1 -- 172.21.15.26:0/2881001431 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b0003520 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1c925c640 1 -- 172.21.15.26:0/2881001431 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975078822 0 0) 0x7fc1c406b0f0 con 0x7fc1c40c26c0 2024-03-06T07:26:30.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1c925c640 1 -- 172.21.15.26:0/2881001431 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c40670c0 con 0x7fc1c40c26c0 2024-03-06T07:26:30.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1caa5f640 1 -- 172.21.15.26:0/2881001431 >> v1:172.21.15.117:6789/0 conn(0x7fc1c40c26c0 legacy=0x7fc1c406bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1caa5f640 1 -- 172.21.15.26:0/2881001431 >> v1:172.21.15.26:6789/0 conn(0x7fc1c40c2aa0 legacy=0x7fc1c40c0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.507+0000 7fc1caa5f640 1 -- 172.21.15.26:0/2881001431 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c406b3d0 con 0x7fc1c4155bd0 2024-03-06T07:26:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.508+0000 7fc1caa5f640 1 --2- 172.21.15.26:0/2881001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 0x7fc1c4152350 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009d90 tx=0x7fc1bc02be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=834dac27b38e4c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.508+0000 7fc1c925c640 1 -- 172.21.15.26:0/2881001431 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc039070 con 0x7fc1c4155bd0 2024-03-06T07:26:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.508+0000 7fc1c925c640 1 -- 172.21.15.26:0/2881001431 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc02cbd0 con 0x7fc1c4155bd0 2024-03-06T07:26:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.508+0000 7fc1c925c640 1 -- 172.21.15.26:0/2881001431 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc034440 con 0x7fc1c4155bd0 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.509+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/2881001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 msgr2=0x7fc1c4152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.509+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/2881001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 0x7fc1c4152350 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009d90 tx=0x7fc1bc02be20 comp rx=0 tx=0).stop 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.509+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/2881001431 shutdown_connections 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.509+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/2881001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 0x7fc1c4152350 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.509+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/2881001431 >> 172.21.15.26:0/2881001431 conn(0x7fc1c40c3b40 msgr2=0x7fc1c40c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.509+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/2881001431 shutdown_connections 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.509+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/2881001431 wait complete. 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 Processor -- start 2024-03-06T07:26:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 -- start start 2024-03-06T07:26:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c40c26c0 0x7fc1c41491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c40c2aa0 0x7fc1c414ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 0x7fc1c41edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4066d00 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c406b740 con 0x7fc1c4155bd0 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.510+0000 7fc1cc4c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4066e80 con 0x7fc1c40c26c0 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c40c2aa0 0x7fc1c414ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c40c2aa0 0x7fc1c414ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42792/0 (socket says 172.21.15.26:42792) 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 -- 172.21.15.26:0/8366911 learned_addr learned my addr 172.21.15.26:0/8366911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1caa5f640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 0x7fc1c41edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1ca25e640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c40c26c0 0x7fc1c41491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 -- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c40c26c0 msgr2=0x7fc1c41491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c40c26c0 0x7fc1c41491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 -- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 msgr2=0x7fc1c41edeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 0x7fc1c41edeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 -- 172.21.15.26:0/8366911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4152c00 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1c9a5d640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c40c2aa0 0x7fc1c414ba80 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400c500 tx=0x7fc1b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1baffd640 1 -- 172.21.15.26:0/8366911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4017070 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1baffd640 1 -- 172.21.15.26:0/8366911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b4005ce0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1baffd640 1 -- 172.21.15.26:0/8366911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40072f0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c4152ee0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.511+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c414bfc0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.512+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c404f330 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.513+0000 7fc1baffd640 1 -- 172.21.15.26:0/8366911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1b401a070 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.513+0000 7fc1baffd640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc1a806b500 0x7fc1a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.513+0000 7fc1baffd640 1 -- 172.21.15.26:0/8366911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fc1b4089a30 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.513+0000 7fc1ca25e640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc1a806b500 0x7fc1a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.514+0000 7fc1ca25e640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc1a806b500 0x7fc1a806d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0007fd0 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.518+0000 7fc1baffd640 1 -- 172.21.15.26:0/8366911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc1b4057d10 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.703+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1c40c15e0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.703+0000 7fc1baffd640 1 -- 172.21.15.26:0/8366911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v156) v1 ==== 105+0+11 (secure 0 0 0) 0x7fc1b405bdb0 con 0x7fc1c40c2aa0 2024-03-06T07:26:30.705 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:26:30.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc1a806b500 msgr2=0x7fc1a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc1a806b500 0x7fc1a806d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0007fd0 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c40c2aa0 msgr2=0x7fc1c414ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c40c2aa0 0x7fc1c414ba80 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400c500 tx=0x7fc1b400c9c0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 shutdown_connections 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc1a806b500 0x7fc1a806d9c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c4155bd0 0x7fc1c41edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c40c2aa0 0x7fc1c414ba80 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 --2- 172.21.15.26:0/8366911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c40c26c0 0x7fc1c41491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 >> 172.21.15.26:0/8366911 conn(0x7fc1c40c3b40 msgr2=0x7fc1c414af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 shutdown_connections 2024-03-06T07:26:30.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.705+0000 7fc1cc4c2640 1 -- 172.21.15.26:0/8366911 wait complete. 2024-03-06T07:26:30.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:26:30.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.820+0000 7f349dba0640 1 Processor -- start 2024-03-06T07:26:30.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.820+0000 7f349dba0640 1 -- start start 2024-03-06T07:26:30.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349dba0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3498152a20 0x7f34981592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349dba0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f349806b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349dba0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34980c8640 0x7f34980c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349dba0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34981552c0 con 0x7f34980c8640 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349dba0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34980c6d30 con 0x7f3498152a20 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349dba0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3498155580 con 0x7f349806afc0 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34980c8640 0x7f34980c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34980c8640 0x7f34980c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42800/0 (socket says 172.21.15.26:42800) 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 -- 172.21.15.26:0/4189336597 learned_addr learned my addr 172.21.15.26:0/4189336597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f3497fff640 1 --2- 172.21.15.26:0/4189336597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f349806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349cb9e640 1 --2- 172.21.15.26:0/4189336597 >> v2:172.21.15.117:3300/0 conn(0x7f3498152a20 0x7f34981592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 -- 172.21.15.26:0/4189336597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 msgr2=0x7f349806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:30.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 --2- 172.21.15.26:0/4189336597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f349806b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 -- 172.21.15.26:0/4189336597 >> v2:172.21.15.117:3300/0 conn(0x7f3498152a20 msgr2=0x7f34981592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 --2- 172.21.15.26:0/4189336597 >> v2:172.21.15.117:3300/0 conn(0x7f3498152a20 0x7f34981592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.821+0000 7f349d39f640 1 -- 172.21.15.26:0/4189336597 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3498155860 con 0x7f34980c8640 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.822+0000 7f349d39f640 1 --2- 172.21.15.26:0/4189336597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34980c8640 0x7f34980c8ab0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f34980c8210 tx=0x7f348002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7f69d045fcc33a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.822+0000 7f34977fe640 1 -- 172.21.15.26:0/4189336597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480038070 con 0x7f34980c8640 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.822+0000 7f34977fe640 1 -- 172.21.15.26:0/4189336597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348002f050 con 0x7f34980c8640 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.822+0000 7f34977fe640 1 -- 172.21.15.26:0/4189336597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480031970 con 0x7f34980c8640 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 -- 172.21.15.26:0/4189336597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34980c8640 msgr2=0x7f34980c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 --2- 172.21.15.26:0/4189336597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34980c8640 0x7f34980c8ab0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f34980c8210 tx=0x7f348002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 -- 172.21.15.26:0/4189336597 shutdown_connections 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 --2- 172.21.15.26:0/4189336597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f349806b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 --2- 172.21.15.26:0/4189336597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34980c8640 0x7f34980c8ab0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 --2- 172.21.15.26:0/4189336597 >> v2:172.21.15.117:3300/0 conn(0x7f3498152a20 0x7f34981592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 -- 172.21.15.26:0/4189336597 >> 172.21.15.26:0/4189336597 conn(0x7f34980ca740 msgr2=0x7f34980cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 -- 172.21.15.26:0/4189336597 shutdown_connections 2024-03-06T07:26:30.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.823+0000 7f349dba0640 1 -- 172.21.15.26:0/4189336597 wait complete. 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.824+0000 7f349dba0640 1 Processor -- start 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.824+0000 7f349dba0640 1 -- start start 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349dba0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f34980bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349dba0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34980c8640 0x7f34980c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349dba0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3498152a20 0x7f34980c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349dba0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498156060 con 0x7f3498152a20 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349dba0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498152e00 con 0x7f34980c8640 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349dba0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34980c1810 con 0x7f349806afc0 2024-03-06T07:26:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349d39f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3498152a20 0x7f34980c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f34980bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349d39f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3498152a20 0x7f34980c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42804/0 (socket says 172.21.15.26:42804) 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f34980bccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:50748/0 (socket says 172.21.15.26:50748) 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349d39f640 1 -- 172.21.15.26:0/2297687072 learned_addr learned my addr 172.21.15.26:0/2297687072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f3497fff640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34980c8640 0x7f34980c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 -- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34980c8640 msgr2=0x7f34980c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34980c8640 0x7f34980c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 -- 172.21.15.26:0/2297687072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3498152a20 msgr2=0x7f34980c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3498152a20 0x7f34980c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 -- 172.21.15.26:0/2297687072 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3498151490 con 0x7f349806afc0 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349d39f640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3498152a20 0x7f34980c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f349cb9e640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f34980bccd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f349000e850 tx=0x7f349000ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.825+0000 7f34957fa640 1 -- 172.21.15.26:0/2297687072 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490017070 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.826+0000 7f34957fa640 1 -- 172.21.15.26:0/2297687072 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000c940 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.826+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34981511b0 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.826+0000 7f34957fa640 1 -- 172.21.15.26:0/2297687072 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000cd10 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.826+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34981eb900 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.826+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34980684b0 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.827+0000 7f34957fa640 1 -- 172.21.15.26:0/2297687072 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f349001a070 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.827+0000 7f34957fa640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f347806b4d0 0x7f347806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.827+0000 7f34957fa640 1 -- 172.21.15.26:0/2297687072 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f349008a500 con 0x7f349806afc0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.827+0000 7f3497fff640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f347806b4d0 0x7f347806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.828+0000 7f3497fff640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f347806b4d0 0x7f347806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f34981589d0 tx=0x7f348800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:30.832+0000 7f34957fa640 1 -- 172.21.15.26:0/2297687072 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3490058880 con 0x7f349806afc0 2024-03-06T07:26:31.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.012+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f34980735a0 con 0x7f349806afc0 2024-03-06T07:26:31.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.012+0000 7f34957fa640 1 -- 172.21.15.26:0/2297687072 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v156) v1 ==== 101+0+11 (secure 0 0 0) 0x7f34980735a0 con 0x7f349806afc0 2024-03-06T07:26:31.014 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:26:31.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f347806b4d0 msgr2=0x7f347806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f347806b4d0 0x7f347806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f34981589d0 tx=0x7f348800d040 comp rx=0 tx=0).stop 2024-03-06T07:26:31.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 msgr2=0x7f34980bccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f34980bccd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f349000e850 tx=0x7f349000ed10 comp rx=0 tx=0).stop 2024-03-06T07:26:31.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 shutdown_connections 2024-03-06T07:26:31.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f347806b4d0 0x7f347806d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3498152a20 0x7f34980c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34980c8640 0x7f34980c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 --2- 172.21.15.26:0/2297687072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f349806afc0 0x7f34980bccd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 >> 172.21.15.26:0/2297687072 conn(0x7f34980ca740 msgr2=0x7f34980cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.014+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 shutdown_connections 2024-03-06T07:26:31.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.015+0000 7f349dba0640 1 -- 172.21.15.26:0/2297687072 wait complete. 2024-03-06T07:26:31.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:31.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.126+0000 7f084b53e640 1 Processor -- start 2024-03-06T07:26:31.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.126+0000 7f084b53e640 1 -- start start 2024-03-06T07:26:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f084b53e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406afc0 0x7f084406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f084b53e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f084406b3a0 0x7f0844152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f084b53e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08441534f0 0x7f08440c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f084b53e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0844153980 con 0x7f08441534f0 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f084b53e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0844150a20 con 0x7f084406b3a0 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f084b53e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08441565a0 con 0x7f084406afc0 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f0849adb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08441534f0 0x7f08440c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f0849adb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08441534f0 0x7f08440c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42808/0 (socket says 172.21.15.26:42808) 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f0849adb640 1 -- 172.21.15.26:0/3663473313 learned_addr learned my addr 172.21.15.26:0/3663473313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f0848ad9640 1 --2- 172.21.15.26:0/3663473313 >> v2:172.21.15.117:3300/0 conn(0x7f084406b3a0 0x7f0844152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.127+0000 7f08492da640 1 --2- 172.21.15.26:0/3663473313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406afc0 0x7f084406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.128+0000 7f0849adb640 1 -- 172.21.15.26:0/3663473313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406afc0 msgr2=0x7f084406f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.128+0000 7f0849adb640 1 --2- 172.21.15.26:0/3663473313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406afc0 0x7f084406f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.128+0000 7f0849adb640 1 -- 172.21.15.26:0/3663473313 >> v2:172.21.15.117:3300/0 conn(0x7f084406b3a0 msgr2=0x7f0844152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.128+0000 7f0849adb640 1 --2- 172.21.15.26:0/3663473313 >> v2:172.21.15.117:3300/0 conn(0x7f084406b3a0 0x7f0844152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.128+0000 7f0849adb640 1 -- 172.21.15.26:0/3663473313 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084406b800 con 0x7f08441534f0 2024-03-06T07:26:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.128+0000 7f0849adb640 1 --2- 172.21.15.26:0/3663473313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08441534f0 0x7f08440c5690 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0844070dd0 tx=0x7f083402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828e06946a6df43d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f0843fff640 1 -- 172.21.15.26:0/3663473313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083402c950 con 0x7f08441534f0 2024-03-06T07:26:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f0843fff640 1 -- 172.21.15.26:0/3663473313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083402caf0 con 0x7f08441534f0 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f0843fff640 1 -- 172.21.15.26:0/3663473313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08340343f0 con 0x7f08441534f0 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 -- 172.21.15.26:0/3663473313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08441534f0 msgr2=0x7f08440c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 --2- 172.21.15.26:0/3663473313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08441534f0 0x7f08440c5690 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0844070dd0 tx=0x7f083402bdb0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 -- 172.21.15.26:0/3663473313 shutdown_connections 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 --2- 172.21.15.26:0/3663473313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08441534f0 0x7f08440c5690 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 --2- 172.21.15.26:0/3663473313 >> v2:172.21.15.117:3300/0 conn(0x7f084406b3a0 0x7f0844152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 --2- 172.21.15.26:0/3663473313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406afc0 0x7f084406f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 -- 172.21.15.26:0/3663473313 >> 172.21.15.26:0/3663473313 conn(0x7f08440c4290 msgr2=0x7f08440c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 -- 172.21.15.26:0/3663473313 shutdown_connections 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.129+0000 7f084b53e640 1 -- 172.21.15.26:0/3663473313 wait complete. 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.130+0000 7f084b53e640 1 Processor -- start 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.130+0000 7f084b53e640 1 -- start start 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f084b53e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f084406afc0 0x7f084406c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f084b53e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406b3a0 0x7f08441ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f084b53e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08441534f0 0x7f08441eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f084b53e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0844156ad0 con 0x7f084406afc0 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f084b53e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08441567f0 con 0x7f08441534f0 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f084b53e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0844156db0 con 0x7f084406b3a0 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f084406afc0 0x7f084406c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f084406afc0 0x7f084406c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42810/0 (socket says 172.21.15.26:42810) 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 -- 172.21.15.26:0/3244976999 learned_addr learned my addr 172.21.15.26:0/3244976999 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f0849adb640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08441534f0 0x7f08441eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f0848ad9640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406b3a0 0x7f08441ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 -- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406b3a0 msgr2=0x7f08441ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406b3a0 0x7f08441ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 -- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08441534f0 msgr2=0x7f08441eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08441534f0 0x7f08441eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.131+0000 7f08492da640 1 -- 172.21.15.26:0/3244976999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08441ef200 con 0x7f084406afc0 2024-03-06T07:26:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.132+0000 7f08492da640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f084406afc0 0x7f084406c2e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f083c00e9b0 tx=0x7f083c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.132+0000 7f0841ffb640 1 -- 172.21.15.26:0/3244976999 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c017070 con 0x7f084406afc0 2024-03-06T07:26:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.132+0000 7f0841ffb640 1 -- 172.21.15.26:0/3244976999 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c00c9a0 con 0x7f084406afc0 2024-03-06T07:26:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.132+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084414f4c0 con 0x7f084406afc0 2024-03-06T07:26:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.132+0000 7f0841ffb640 1 -- 172.21.15.26:0/3244976999 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c00cd90 con 0x7f084406afc0 2024-03-06T07:26:31.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.132+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084414fa80 con 0x7f084406afc0 2024-03-06T07:26:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.133+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0844067ad0 con 0x7f084406afc0 2024-03-06T07:26:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.133+0000 7f0841ffb640 1 -- 172.21.15.26:0/3244976999 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f083c01a070 con 0x7f084406afc0 2024-03-06T07:26:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.134+0000 7f0841ffb640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f083006b4d0 0x7f083006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.134+0000 7f0841ffb640 1 -- 172.21.15.26:0/3244976999 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f083c08a990 con 0x7f084406afc0 2024-03-06T07:26:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.134+0000 7f0848ad9640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f083006b4d0 0x7f083006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.134+0000 7f0848ad9640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f083006b4d0 0x7f083006d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f08441e8f70 tx=0x7f0838009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.138+0000 7f0841ffb640 1 -- 172.21.15.26:0/3244976999 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0844067ad0 con 0x7f084406afc0 2024-03-06T07:26:31.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.355+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f084404f330 con 0x7f084406afc0 2024-03-06T07:26:31.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.355+0000 7f0841ffb640 1 -- 172.21.15.26:0/3244976999 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+6271 (secure 0 0 0) 0x7f083c058d10 con 0x7f084406afc0 2024-03-06T07:26:31.357 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:31.357 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":148,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:26.887138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7178,"qdb_cluster":[7178]},"id":11},{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:25.766406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:31.357 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 148 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f083006b4d0 msgr2=0x7f083006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f083006b4d0 0x7f083006d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f08441e8f70 tx=0x7f0838009190 comp rx=0 tx=0).stop 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f084406afc0 msgr2=0x7f084406c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f084406afc0 0x7f084406c2e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f083c00e9b0 tx=0x7f083c00ee70 comp rx=0 tx=0).stop 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 shutdown_connections 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f083006b4d0 0x7f083006d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08441534f0 0x7f08441eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f084406b3a0 0x7f08441ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 --2- 172.21.15.26:0/3244976999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f084406afc0 0x7f084406c2e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 >> 172.21.15.26:0/3244976999 conn(0x7f08440c4290 msgr2=0x7f08440a7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 shutdown_connections 2024-03-06T07:26:31.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.357+0000 7f084b53e640 1 -- 172.21.15.26:0/3244976999 wait complete. 2024-03-06T07:26:31.370 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:31.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.469+0000 7fa8d4eed640 1 Processor -- start 2024-03-06T07:26:31.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.469+0000 7fa8d4eed640 1 -- start start 2024-03-06T07:26:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8d4eed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d00c1320 0x7fa8d01566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8d4eed640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa8d00c6000 0x7fa8d0156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8d4eed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d0153a60 con 0x7fa8d0150580 2024-03-06T07:26:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8d4eed640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d0153d40 con 0x7fa8d00c6000 2024-03-06T07:26:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8d4eed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d006c2c0 con 0x7fa8d00c1320 2024-03-06T07:26:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8ce59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa8d0150580 0x7fa8d006dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49510/0 (socket says 172.21.15.26:49510) 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8ce59c640 1 -- 172.21.15.26:0/98061381 learned_addr learned my addr 172.21.15.26:0/98061381 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8ced9d640 1 --2- 172.21.15.26:0/98061381 >> v2:172.21.15.117:3300/0 conn(0x7fa8d00c6000 0x7fa8d0156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8cdd9b640 1 --2- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d00c1320 0x7fa8d01566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8cd59a640 1 -- 172.21.15.26:0/98061381 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431556034 0 0) 0x7fa8d0153a60 con 0x7fa8d0150580 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8cd59a640 1 -- 172.21.15.26:0/98061381 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b8003520 con 0x7fa8d0150580 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8ced9d640 1 -- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d00c1320 msgr2=0x7fa8d01566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8ced9d640 1 --2- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d00c1320 0x7fa8d01566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8ced9d640 1 -- 172.21.15.26:0/98061381 >> v1:172.21.15.26:6789/0 conn(0x7fa8d0150580 legacy=0x7fa8d006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8ced9d640 1 -- 172.21.15.26:0/98061381 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d006c5a0 con 0x7fa8d00c6000 2024-03-06T07:26:31.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.470+0000 7fa8cdd9b640 1 --2- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d00c1320 0x7fa8d01566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.471+0000 7fa8ced9d640 1 --2- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c6000 0x7fa8d0156bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009d90 tx=0x7fa8c802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0ecad42c11d0d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.471+0000 7fa8cd59a640 1 -- 172.21.15.26:0/98061381 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8039070 con 0x7fa8d00c6000 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.471+0000 7fa8cd59a640 1 -- 172.21.15.26:0/98061381 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c802cbc0 con 0x7fa8d00c6000 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.471+0000 7fa8cd59a640 1 -- 172.21.15.26:0/98061381 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8034440 con 0x7fa8d00c6000 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 -- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c6000 msgr2=0x7fa8d0156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c6000 0x7fa8d0156bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009d90 tx=0x7fa8c802be10 comp rx=0 tx=0).stop 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 -- 172.21.15.26:0/98061381 shutdown_connections 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c6000 0x7fa8d0156bf0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/98061381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d00c1320 0x7fa8d01566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 -- 172.21.15.26:0/98061381 >> 172.21.15.26:0/98061381 conn(0x7fa8d00c6710 msgr2=0x7fa8d00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 -- 172.21.15.26:0/98061381 shutdown_connections 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.472+0000 7fa8d4eed640 1 -- 172.21.15.26:0/98061381 wait complete. 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.473+0000 7fa8d4eed640 1 Processor -- start 2024-03-06T07:26:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.473+0000 7fa8d4eed640 1 -- start start 2024-03-06T07:26:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8d4eed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c1320 0x7fa8d00a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8d4eed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8d00c6000 0x7fa8d0154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8d4eed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d0150580 0x7fa8d00c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8d4eed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d006e5a0 con 0x7fa8d00c6000 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8d4eed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d006e420 con 0x7fa8d00c1320 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8d4eed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00c0c40 con 0x7fa8d0150580 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8cdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8d00c6000 0x7fa8d0154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8ce59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c1320 0x7fa8d00a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8ced9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d0150580 0x7fa8d00c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8cdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8d00c6000 0x7fa8d0154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42826/0 (socket says 172.21.15.26:42826) 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.474+0000 7fa8cdd9b640 1 -- 172.21.15.26:0/3421909670 learned_addr learned my addr 172.21.15.26:0/3421909670 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8cdd9b640 1 -- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d0150580 msgr2=0x7fa8d00c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8cdd9b640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d0150580 0x7fa8d00c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8cdd9b640 1 -- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c1320 msgr2=0x7fa8d00a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8cdd9b640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c1320 0x7fa8d00a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8cdd9b640 1 -- 172.21.15.26:0/3421909670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0151830 con 0x7fa8d00c6000 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8ced9d640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d0150580 0x7fa8d00c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8ce59c640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c1320 0x7fa8d00a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8cdd9b640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8d00c6000 0x7fa8d0154300 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ebe0 tx=0x7fa8c400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8beffd640 1 -- 172.21.15.26:0/3421909670 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4017070 con 0x7fa8d00c6000 2024-03-06T07:26:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8beffd640 1 -- 172.21.15.26:0/3421909670 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c400cc30 con 0x7fa8d00c6000 2024-03-06T07:26:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d0156380 con 0x7fa8d00c6000 2024-03-06T07:26:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8beffd640 1 -- 172.21.15.26:0/3421909670 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4013470 con 0x7fa8d00c6000 2024-03-06T07:26:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.475+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d0151550 con 0x7fa8d00c6000 2024-03-06T07:26:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.476+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d004f330 con 0x7fa8d00c6000 2024-03-06T07:26:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.476+0000 7fa8beffd640 1 -- 172.21.15.26:0/3421909670 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8c401a070 con 0x7fa8d00c6000 2024-03-06T07:26:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.477+0000 7fa8beffd640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.477+0000 7fa8beffd640 1 -- 172.21.15.26:0/3421909670 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa8c408a630 con 0x7fa8d00c6000 2024-03-06T07:26:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.477+0000 7fa8ce59c640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.477+0000 7fa8ce59c640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0005880 tx=0x7fa8c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.481+0000 7fa8beffd640 1 -- 172.21.15.26:0/3421909670 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa8c4058910 con 0x7fa8d00c6000 2024-03-06T07:26:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.663+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8d00c5de0 con 0x7fa8d00c6000 2024-03-06T07:26:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.664+0000 7fa8beffd640 1 -- 172.21.15.26:0/3421909670 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+20743 (secure 0 0 0) 0x7fa8c405c9b0 con 0x7fa8d00c6000 2024-03-06T07:26:31.666 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:31.666 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":156,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:23.875901+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6833/3029467671":"2024-03-07T07:26:23.875878+0000","172.21.15.26:6832/3029467671":"2024-03-07T07:26:23.875878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 msgr2=0x7fa8b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0005880 tx=0x7fa8c0009040 comp rx=0 tx=0).stop 2024-03-06T07:26:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8d00c6000 msgr2=0x7fa8d0154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8d00c6000 0x7fa8d0154300 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ebe0 tx=0x7fa8c400c520 comp rx=0 tx=0).stop 2024-03-06T07:26:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 shutdown_connections 2024-03-06T07:26:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa8b006b4d0 0x7fa8b006d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d0150580 0x7fa8d00c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8d00c6000 0x7fa8d0154300 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 --2- 172.21.15.26:0/3421909670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d00c1320 0x7fa8d00a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 >> 172.21.15.26:0/3421909670 conn(0x7fa8d00c6710 msgr2=0x7fa8d0153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 shutdown_connections 2024-03-06T07:26:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.666+0000 7fa8d4eed640 1 -- 172.21.15.26:0/3421909670 wait complete. 2024-03-06T07:26:31.679 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:26:31.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:31.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 Processor -- start 2024-03-06T07:26:31.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 -- start start 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c814f360 0x7f28c8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28c80c88c0 0x7f28c80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28c80c5880 0x7f28c8158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80c8dd0 con 0x7f28c80c5880 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80c73d0 con 0x7f28c80c88c0 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cf135640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80be2e0 con 0x7f28c814f360 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.781+0000 7f28cd6d2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28c80c5880 0x7f28c8158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cd6d2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28c80c5880 0x7f28c8158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42828/0 (socket says 172.21.15.26:42828) 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cd6d2640 1 -- 172.21.15.26:0/1836104569 learned_addr learned my addr 172.21.15.26:0/1836104569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28c7fff640 1 --2- 172.21.15.26:0/1836104569 >> v2:172.21.15.117:3300/0 conn(0x7f28c80c88c0 0x7f28c80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cced1640 1 --2- 172.21.15.26:0/1836104569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c814f360 0x7f28c8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cd6d2640 1 -- 172.21.15.26:0/1836104569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c814f360 msgr2=0x7f28c8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cd6d2640 1 --2- 172.21.15.26:0/1836104569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c814f360 0x7f28c8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cd6d2640 1 -- 172.21.15.26:0/1836104569 >> v2:172.21.15.117:3300/0 conn(0x7f28c80c88c0 msgr2=0x7f28c80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cd6d2640 1 --2- 172.21.15.26:0/1836104569 >> v2:172.21.15.117:3300/0 conn(0x7f28c80c88c0 0x7f28c80c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.782+0000 7f28cd6d2640 1 -- 172.21.15.26:0/1836104569 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80c7670 con 0x7f28c80c5880 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cd6d2640 1 --2- 172.21.15.26:0/1836104569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c80c5880 0x7f28c8158b60 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f28c80cbac0 tx=0x7f28b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10505dc0d7352f2d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28c77fe640 1 -- 172.21.15.26:0/1836104569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b002e070 con 0x7f28c80c5880 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28c77fe640 1 -- 172.21.15.26:0/1836104569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0037070 con 0x7f28c80c5880 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28c77fe640 1 -- 172.21.15.26:0/1836104569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0031a00 con 0x7f28c80c5880 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 -- 172.21.15.26:0/1836104569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c80c5880 msgr2=0x7f28c8158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 --2- 172.21.15.26:0/1836104569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c80c5880 0x7f28c8158b60 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f28c80cbac0 tx=0x7f28b002b880 comp rx=0 tx=0).stop 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 -- 172.21.15.26:0/1836104569 shutdown_connections 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 --2- 172.21.15.26:0/1836104569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c80c5880 0x7f28c8158b60 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 --2- 172.21.15.26:0/1836104569 >> v2:172.21.15.117:3300/0 conn(0x7f28c80c88c0 0x7f28c80c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 --2- 172.21.15.26:0/1836104569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c814f360 0x7f28c8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 -- 172.21.15.26:0/1836104569 >> 172.21.15.26:0/1836104569 conn(0x7f28c80ac570 msgr2=0x7f28c80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 -- 172.21.15.26:0/1836104569 shutdown_connections 2024-03-06T07:26:31.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.783+0000 7f28cf135640 1 -- 172.21.15.26:0/1836104569 wait complete. 2024-03-06T07:26:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.784+0000 7f28cf135640 1 Processor -- start 2024-03-06T07:26:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.784+0000 7f28cf135640 1 -- start start 2024-03-06T07:26:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cf135640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c80c5880 0x7f28c8156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cf135640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c80c88c0 0x7f28c8148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cf135640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c814f360 0x7f28c81490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cf135640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c81590a0 con 0x7f28c814f360 2024-03-06T07:26:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cf135640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80c7c30 con 0x7f28c80c88c0 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cf135640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8159220 con 0x7f28c80c5880 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cd6d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c814f360 0x7f28c81490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cd6d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c814f360 0x7f28c81490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42842/0 (socket says 172.21.15.26:42842) 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cd6d2640 1 -- 172.21.15.26:0/4148523891 learned_addr learned my addr 172.21.15.26:0/4148523891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28cced1640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c80c5880 0x7f28c8156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.785+0000 7f28c7fff640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c80c88c0 0x7f28c8148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cd6d2640 1 -- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c80c5880 msgr2=0x7f28c8156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cd6d2640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c80c5880 0x7f28c8156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cd6d2640 1 -- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c80c88c0 msgr2=0x7f28c8148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cd6d2640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c80c88c0 0x7f28c8148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cd6d2640 1 -- 172.21.15.26:0/4148523891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c8151d80 con 0x7f28c814f360 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cd6d2640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c814f360 0x7f28c81490f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f28b00090b0 tx=0x7f28b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28c57fa640 1 -- 172.21.15.26:0/4148523891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0032670 con 0x7f28c814f360 2024-03-06T07:26:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28c57fa640 1 -- 172.21.15.26:0/4148523891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0032810 con 0x7f28c814f360 2024-03-06T07:26:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c8152060 con 0x7f28c814f360 2024-03-06T07:26:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28c57fa640 1 -- 172.21.15.26:0/4148523891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0032c00 con 0x7f28c814f360 2024-03-06T07:26:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.786+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c814fe80 con 0x7f28c814f360 2024-03-06T07:26:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.787+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8152580 con 0x7f28c814f360 2024-03-06T07:26:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.788+0000 7f28c57fa640 1 -- 172.21.15.26:0/4148523891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28b00453f0 con 0x7f28c814f360 2024-03-06T07:26:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.788+0000 7f28c57fa640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28a806b4d0 0x7f28a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.788+0000 7f28c57fa640 1 -- 172.21.15.26:0/4148523891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f28b00ac300 con 0x7f28c814f360 2024-03-06T07:26:31.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.788+0000 7f28cced1640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28a806b4d0 0x7f28a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.789+0000 7f28cced1640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28a806b4d0 0x7f28a806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f28c0011340 tx=0x7f28c000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:31.792+0000 7f28c57fa640 1 -- 172.21.15.26:0/4148523891 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28b0040070 con 0x7f28c814f360 2024-03-06T07:26:32.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.011+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28c804f330 con 0x7f28c814f360 2024-03-06T07:26:32.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.012+0000 7f28c57fa640 1 -- 172.21.15.26:0/4148523891 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+6271 (secure 0 0 0) 0x7f28b007a710 con 0x7f28c814f360 2024-03-06T07:26:32.013 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:32.013 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":148,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:26.887138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7178,"qdb_cluster":[7178]},"id":11},{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:25.766406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:32.014 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 148 2024-03-06T07:26:32.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.013+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28a806b4d0 msgr2=0x7f28a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28a806b4d0 0x7f28a806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f28c0011340 tx=0x7f28c000b040 comp rx=0 tx=0).stop 2024-03-06T07:26:32.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c814f360 msgr2=0x7f28c81490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c814f360 0x7f28c81490f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f28b00090b0 tx=0x7f28b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 shutdown_connections 2024-03-06T07:26:32.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28a806b4d0 0x7f28a806d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c814f360 0x7f28c81490f0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c80c88c0 0x7f28c8148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 --2- 172.21.15.26:0/4148523891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c80c5880 0x7f28c8156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 >> 172.21.15.26:0/4148523891 conn(0x7f28c80ac570 msgr2=0x7f28c80c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:32.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 shutdown_connections 2024-03-06T07:26:32.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.014+0000 7f28cf135640 1 -- 172.21.15.26:0/4148523891 wait complete. 2024-03-06T07:26:32.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.127+0000 7f05fa629640 1 Processor -- start 2024-03-06T07:26:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.127+0000 7f05fa629640 1 -- start start 2024-03-06T07:26:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.127+0000 7f05fa629640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05f41508c0 0x7f05f40c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.127+0000 7f05fa629640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f4157340 con 0x7f05f406f7b0 2024-03-06T07:26:32.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.127+0000 7f05fa629640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f4156d80 con 0x7f05f4072720 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.127+0000 7f05fa629640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f4156b40 con 0x7f05f41508c0 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f05f406f7b0 0x7f05f41501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49522/0 (socket says 172.21.15.26:49522) 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f37fe640 1 -- 172.21.15.26:0/2558375690 learned_addr learned my addr 172.21.15.26:0/2558375690 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f8bc6640 1 --2- 172.21.15.26:0/2558375690 >> v2:172.21.15.117:3300/0 conn(0x7f05f41508c0 0x7f05f40c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f2ffd640 1 -- 172.21.15.26:0/2558375690 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932130180 0 0) 0x7f05f4157340 con 0x7f05f406f7b0 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f2ffd640 1 -- 172.21.15.26:0/2558375690 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05dc003520 con 0x7f05f406f7b0 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f2ffd640 1 -- 172.21.15.26:0/2558375690 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082153643 0 0) 0x7f05f4156d80 con 0x7f05f4072720 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f2ffd640 1 -- 172.21.15.26:0/2558375690 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f4157340 con 0x7f05f4072720 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f8bc6640 1 -- 172.21.15.26:0/2558375690 >> v1:172.21.15.117:6789/0 conn(0x7f05f4072720 legacy=0x7f05f4156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f8bc6640 1 -- 172.21.15.26:0/2558375690 >> v1:172.21.15.26:6789/0 conn(0x7f05f406f7b0 legacy=0x7f05f41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.128+0000 7f05f8bc6640 1 -- 172.21.15.26:0/2558375690 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4157060 con 0x7f05f41508c0 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.129+0000 7f05f8bc6640 1 --2- 172.21.15.26:0/2558375690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 0x7f05f40c3bd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009d90 tx=0x7f05ec02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1462f14087a5d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.129+0000 7f05f2ffd640 1 -- 172.21.15.26:0/2558375690 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec039070 con 0x7f05f41508c0 2024-03-06T07:26:32.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.129+0000 7f05f2ffd640 1 -- 172.21.15.26:0/2558375690 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec02cc60 con 0x7f05f41508c0 2024-03-06T07:26:32.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.129+0000 7f05f2ffd640 1 -- 172.21.15.26:0/2558375690 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec034640 con 0x7f05f41508c0 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.129+0000 7f05fa629640 1 -- 172.21.15.26:0/2558375690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 msgr2=0x7f05f40c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.129+0000 7f05fa629640 1 --2- 172.21.15.26:0/2558375690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 0x7f05f40c3bd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009d90 tx=0x7f05ec02beb0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.130+0000 7f05fa629640 1 -- 172.21.15.26:0/2558375690 shutdown_connections 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.130+0000 7f05fa629640 1 --2- 172.21.15.26:0/2558375690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 0x7f05f40c3bd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.130+0000 7f05fa629640 1 -- 172.21.15.26:0/2558375690 >> 172.21.15.26:0/2558375690 conn(0x7f05f40aa630 msgr2=0x7f05f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.130+0000 7f05fa629640 1 -- 172.21.15.26:0/2558375690 shutdown_connections 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.130+0000 7f05fa629640 1 -- 172.21.15.26:0/2558375690 wait complete. 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 Processor -- start 2024-03-06T07:26:32.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 -- start start 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05f406f7b0 0x7f05f40c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f4072720 0x7f05f4148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 0x7f05f4148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4072b00 con 0x7f05f4072720 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40c3740 con 0x7f05f41508c0 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.131+0000 7f05fa629640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4072c80 con 0x7f05f406f7b0 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f4072720 0x7f05f4148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f4072720 0x7f05f4148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42844/0 (socket says 172.21.15.26:42844) 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 -- 172.21.15.26:0/58368196 learned_addr learned my addr 172.21.15.26:0/58368196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f8bc6640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 0x7f05f4148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f3fff640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05f406f7b0 0x7f05f40c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 -- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05f406f7b0 msgr2=0x7f05f40c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05f406f7b0 0x7f05f40c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 -- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 msgr2=0x7f05f4148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 0x7f05f4148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 -- 172.21.15.26:0/58368196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f41ee850 con 0x7f05f4072720 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f37fe640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f4072720 0x7f05f4148810 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f05e800c500 tx=0x7f05e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f0ff9640 1 -- 172.21.15.26:0/58368196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8017070 con 0x7f05f4072720 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f0ff9640 1 -- 172.21.15.26:0/58368196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8005ce0 con 0x7f05f4072720 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f406dcc0 con 0x7f05f4072720 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05f0ff9640 1 -- 172.21.15.26:0/58368196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e80072f0 con 0x7f05f4072720 2024-03-06T07:26:32.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.132+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f40c2460 con 0x7f05f4072720 2024-03-06T07:26:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.133+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f404f330 con 0x7f05f4072720 2024-03-06T07:26:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.134+0000 7f05f0ff9640 1 -- 172.21.15.26:0/58368196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05e801a070 con 0x7f05f4072720 2024-03-06T07:26:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.134+0000 7f05f0ff9640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e006b4d0 0x7f05e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.134+0000 7f05f0ff9640 1 -- 172.21.15.26:0/58368196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f05e8089ac0 con 0x7f05f4072720 2024-03-06T07:26:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.134+0000 7f05f3fff640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e006b4d0 0x7f05e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.135+0000 7f05f3fff640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e006b4d0 0x7f05e006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f05e4007fd0 tx=0x7f05e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.138+0000 7f05f0ff9640 1 -- 172.21.15.26:0/58368196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f05e8057dc0 con 0x7f05f4072720 2024-03-06T07:26:32.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.357+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05f40c2960 con 0x7f05f4072720 2024-03-06T07:26:32.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.357+0000 7f05f0ff9640 1 -- 172.21.15.26:0/58368196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+6271 (secure 0 0 0) 0x7f05e805be60 con 0x7f05f4072720 2024-03-06T07:26:32.359 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:32.359 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":148,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:07.071721+0000","modified":"2024-03-06T07:26:26.887138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7178":{"gid":7178,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/677194061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":677194061},{"type":"v1","addr":"172.21.15.117:6837","nonce":677194061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7178,"qdb_cluster":[7178]},"id":11},{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:25.766406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8293":{"gid":8293,"name":"d","rank":0,"incarnation":139,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6836/296681906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":296681906},{"type":"v1","addr":"172.21.15.117:6836","nonce":296681906}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8293,"qdb_cluster":[8293]},"id":12}]} 2024-03-06T07:26:32.359 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 148 2024-03-06T07:26:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e006b4d0 msgr2=0x7f05e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e006b4d0 0x7f05e006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f05e4007fd0 tx=0x7f05e4009040 comp rx=0 tx=0).stop 2024-03-06T07:26:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f4072720 msgr2=0x7f05f4148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f4072720 0x7f05f4148810 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f05e800c500 tx=0x7f05e800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 shutdown_connections 2024-03-06T07:26:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f05e006b4d0 0x7f05e006d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f41508c0 0x7f05f4148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f4072720 0x7f05f4148810 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 --2- 172.21.15.26:0/58368196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05f406f7b0 0x7f05f40c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 >> 172.21.15.26:0/58368196 conn(0x7f05f40aa630 msgr2=0x7f05f40a8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 shutdown_connections 2024-03-06T07:26:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.359+0000 7f05fa629640 1 -- 172.21.15.26:0/58368196 wait complete. 2024-03-06T07:26:32.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:32.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.471+0000 7f9276f7a640 1 Processor -- start 2024-03-06T07:26:32.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.471+0000 7f9276f7a640 1 -- start start 2024-03-06T07:26:32.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.471+0000 7f9276f7a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f927006da40 0x7f9270155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.471+0000 7f9276f7a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92700c7380 0x7f92700c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.471+0000 7f9276f7a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9270156050 con 0x7f92700c6780 2024-03-06T07:26:32.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.471+0000 7f9276f7a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927006de20 con 0x7f92700c7380 2024-03-06T07:26:32.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.471+0000 7f9276f7a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9270072630 con 0x7f927006da40 2024-03-06T07:26:32.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9275517640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92700c7380 0x7f92700c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9275517640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92700c7380 0x7f92700c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42854/0 (socket says 172.21.15.26:42854) 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9275517640 1 -- 172.21.15.26:0/2959207178 learned_addr learned my addr 172.21.15.26:0/2959207178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9274d16640 1 --2- 172.21.15.26:0/2959207178 >> v2:172.21.15.117:3300/0 conn(0x7f927006da40 0x7f9270155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9275517640 1 -- 172.21.15.26:0/2959207178 >> v2:172.21.15.117:3300/0 conn(0x7f927006da40 msgr2=0x7f9270155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9275517640 1 --2- 172.21.15.26:0/2959207178 >> v2:172.21.15.117:3300/0 conn(0x7f927006da40 0x7f9270155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9275517640 1 -- 172.21.15.26:0/2959207178 >> v1:172.21.15.117:6789/0 conn(0x7f92700c6780 legacy=0x7f9270158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f926f7fe640 1 -- 172.21.15.26:0/2959207178 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604862814 0 0) 0x7f925c001110 con 0x7f92700c6780 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.472+0000 7f9275517640 1 -- 172.21.15.26:0/2959207178 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270072910 con 0x7f92700c7380 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9275517640 1 --2- 172.21.15.26:0/2959207178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92700c7380 0x7f92700c3280 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f92700c98a0 tx=0x7f926802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95569aee7c87898 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f926f7fe640 1 -- 172.21.15.26:0/2959207178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926802ca80 con 0x7f92700c7380 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f926f7fe640 1 -- 172.21.15.26:0/2959207178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926802cc20 con 0x7f92700c7380 2024-03-06T07:26:32.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f926f7fe640 1 -- 172.21.15.26:0/2959207178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92680343f0 con 0x7f92700c7380 2024-03-06T07:26:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 -- 172.21.15.26:0/2959207178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92700c7380 msgr2=0x7f92700c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 --2- 172.21.15.26:0/2959207178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92700c7380 0x7f92700c3280 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f92700c98a0 tx=0x7f926802bee0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 -- 172.21.15.26:0/2959207178 shutdown_connections 2024-03-06T07:26:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 --2- 172.21.15.26:0/2959207178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92700c7380 0x7f92700c3280 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 --2- 172.21.15.26:0/2959207178 >> v2:172.21.15.117:3300/0 conn(0x7f927006da40 0x7f9270155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 -- 172.21.15.26:0/2959207178 >> 172.21.15.26:0/2959207178 conn(0x7f92700aa630 msgr2=0x7f92700aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 -- 172.21.15.26:0/2959207178 shutdown_connections 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.473+0000 7f9276f7a640 1 -- 172.21.15.26:0/2959207178 wait complete. 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.474+0000 7f9276f7a640 1 Processor -- start 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.474+0000 7f9276f7a640 1 -- start start 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9276f7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927006da40 0x7f92701ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9276f7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92700c6780 0x7f92701df440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9276f7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92700c7380 0x7f927014e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9276f7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92701594a0 con 0x7f927006da40 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9276f7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9270158890 con 0x7f92700c6780 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9276f7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92700c37c0 con 0x7f92700c7380 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9274d16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927006da40 0x7f92701ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9274d16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927006da40 0x7f92701ebab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42864/0 (socket says 172.21.15.26:42864) 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9274d16640 1 -- 172.21.15.26:0/3475684732 learned_addr learned my addr 172.21.15.26:0/3475684732 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f9275517640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92700c7380 0x7f927014e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.475+0000 7f926ffff640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92700c6780 0x7f92701df440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9274d16640 1 -- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92700c7380 msgr2=0x7f927014e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9274d16640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92700c7380 0x7f927014e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9274d16640 1 -- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92700c6780 msgr2=0x7f92701df440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9274d16640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92700c6780 0x7f92701df440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9274d16640 1 -- 172.21.15.26:0/3475684732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927014ffd0 con 0x7f927006da40 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f926ffff640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92700c6780 0x7f92701df440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9274d16640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927006da40 0x7f92701ebab0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f926400e9e0 tx=0x7f926400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f926d7fa640 1 -- 172.21.15.26:0/3475684732 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264017070 con 0x7f927006da40 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f926d7fa640 1 -- 172.21.15.26:0/3475684732 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926400c9d0 con 0x7f927006da40 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9270150240 con 0x7f927006da40 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f926d7fa640 1 -- 172.21.15.26:0/3475684732 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926400cdc0 con 0x7f927006da40 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.476+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9270154d40 con 0x7f927006da40 2024-03-06T07:26:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.477+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927004f330 con 0x7f927006da40 2024-03-06T07:26:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.477+0000 7f926d7fa640 1 -- 172.21.15.26:0/3475684732 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f926401a070 con 0x7f927006da40 2024-03-06T07:26:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.478+0000 7f926d7fa640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925406b4d0 0x7f925406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.478+0000 7f926d7fa640 1 -- 172.21.15.26:0/3475684732 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f926408a6e0 con 0x7f927006da40 2024-03-06T07:26:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.478+0000 7f926ffff640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925406b4d0 0x7f925406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.478+0000 7f926ffff640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925406b4d0 0x7f925406d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f92700c4da0 tx=0x7f925c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.482+0000 7f926d7fa640 1 -- 172.21.15.26:0/3475684732 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f92640589c0 con 0x7f927006da40 2024-03-06T07:26:32.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.665+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9270073240 con 0x7f927006da40 2024-03-06T07:26:32.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.667+0000 7f926d7fa640 1 -- 172.21.15.26:0/3475684732 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+20743 (secure 0 0 0) 0x7f926405ca60 con 0x7f927006da40 2024-03-06T07:26:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:32.669 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":156,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:23.875901+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:05.339258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:06.016366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6833/3029467671":"2024-03-07T07:26:23.875878+0000","172.21.15.26:6832/3029467671":"2024-03-07T07:26:23.875878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925406b4d0 msgr2=0x7f925406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925406b4d0 0x7f925406d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f92700c4da0 tx=0x7f925c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927006da40 msgr2=0x7f92701ebab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927006da40 0x7f92701ebab0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f926400e9e0 tx=0x7f926400eea0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 shutdown_connections 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925406b4d0 0x7f925406d990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92700c7380 0x7f927014e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92700c6780 0x7f92701df440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 --2- 172.21.15.26:0/3475684732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927006da40 0x7f92701ebab0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 >> 172.21.15.26:0/3475684732 conn(0x7f92700aa630 msgr2=0x7f92700c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:32.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 shutdown_connections 2024-03-06T07:26:32.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.669+0000 7f9276f7a640 1 -- 172.21.15.26:0/3475684732 wait complete. 2024-03-06T07:26:32.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:26:32.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.784+0000 7fc433df7640 1 Processor -- start 2024-03-06T07:26:32.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.784+0000 7fc433df7640 1 -- start start 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc433df7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc433df7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c0c5140 con 0x7fc42c1526c0 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc433df7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c155bb0 con 0x7fc42c0c2b20 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc433df7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0c3790 con 0x7fc42c0c69b0 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc431b93640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc42c1526c0 0x7fc42c0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49538/0 (socket says 172.21.15.26:49538) 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc431b93640 1 -- 172.21.15.26:0/3920583171 learned_addr learned my addr 172.21.15.26:0/3920583171 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc432394640 1 --2- 172.21.15.26:0/3920583171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc430b91640 1 -- 172.21.15.26:0/3920583171 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953943894 0 0) 0x7fc42c0c5140 con 0x7fc42c1526c0 2024-03-06T07:26:32.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc430b91640 1 -- 172.21.15.26:0/3920583171 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc418003520 con 0x7fc42c1526c0 2024-03-06T07:26:32.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc430b91640 1 -- 172.21.15.26:0/3920583171 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789237277 0 0) 0x7fc42c155bb0 con 0x7fc42c0c2b20 2024-03-06T07:26:32.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc430b91640 1 -- 172.21.15.26:0/3920583171 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc42c0c5140 con 0x7fc42c0c2b20 2024-03-06T07:26:32.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc432394640 1 -- 172.21.15.26:0/3920583171 >> v1:172.21.15.117:6789/0 conn(0x7fc42c0c2b20 legacy=0x7fc42c1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc432394640 1 -- 172.21.15.26:0/3920583171 >> v1:172.21.15.26:6789/0 conn(0x7fc42c1526c0 legacy=0x7fc42c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.785+0000 7fc432394640 1 -- 172.21.15.26:0/3920583171 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c155e90 con 0x7fc42c0c69b0 2024-03-06T07:26:32.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.786+0000 7fc432394640 1 --2- 172.21.15.26:0/3920583171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c158ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc424009dc0 tx=0x7fc42402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01b5e68c93c7bb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.786+0000 7fc430b91640 1 -- 172.21.15.26:0/3920583171 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc424039070 con 0x7fc42c0c69b0 2024-03-06T07:26:32.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.786+0000 7fc430b91640 1 -- 172.21.15.26:0/3920583171 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42402cc40 con 0x7fc42c0c69b0 2024-03-06T07:26:32.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.786+0000 7fc430b91640 1 -- 172.21.15.26:0/3920583171 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc424034440 con 0x7fc42c0c69b0 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.787+0000 7fc433df7640 1 -- 172.21.15.26:0/3920583171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 msgr2=0x7fc42c158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.787+0000 7fc433df7640 1 --2- 172.21.15.26:0/3920583171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c158ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc424009dc0 tx=0x7fc42402be90 comp rx=0 tx=0).stop 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.787+0000 7fc433df7640 1 -- 172.21.15.26:0/3920583171 shutdown_connections 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.787+0000 7fc433df7640 1 --2- 172.21.15.26:0/3920583171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c158ef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.787+0000 7fc433df7640 1 -- 172.21.15.26:0/3920583171 >> 172.21.15.26:0/3920583171 conn(0x7fc42c0aa810 msgr2=0x7fc42c0aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.787+0000 7fc433df7640 1 -- 172.21.15.26:0/3920583171 shutdown_connections 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.787+0000 7fc433df7640 1 -- 172.21.15.26:0/3920583171 wait complete. 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.788+0000 7fc433df7640 1 Processor -- start 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.788+0000 7fc433df7640 1 -- start start 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.788+0000 7fc433df7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc42c0c2b20 0x7fc42c06d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.788+0000 7fc433df7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.788+0000 7fc433df7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc42c1526c0 0x7fc42c1e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.788+0000 7fc433df7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c152aa0 con 0x7fc42c0c2b20 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.788+0000 7fc433df7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c159430 con 0x7fc42c1526c0 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc433df7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0c9040 con 0x7fc42c0c69b0 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc42c0c2b20 0x7fc42c06d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc42c0c2b20 0x7fc42c06d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42874/0 (socket says 172.21.15.26:42874) 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 -- 172.21.15.26:0/617091719 learned_addr learned my addr 172.21.15.26:0/617091719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc432394640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc42c1526c0 0x7fc42c1e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431392640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 -- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 msgr2=0x7fc42c14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c14acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 -- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc42c1526c0 msgr2=0x7fc42c1e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc42c1526c0 0x7fc42c1e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 -- 172.21.15.26:0/617091719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c06d770 con 0x7fc42c0c2b20 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc431b93640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc42c0c2b20 0x7fc42c06d230 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fc42800c460 tx=0x7fc42800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc4227fc640 1 -- 172.21.15.26:0/617091719 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc428017070 con 0x7fc42c0c2b20 2024-03-06T07:26:32.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc4227fc640 1 -- 172.21.15.26:0/617091719 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc428005ce0 con 0x7fc42c0c2b20 2024-03-06T07:26:32.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c06fb90 con 0x7fc42c0c2b20 2024-03-06T07:26:32.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc4227fc640 1 -- 172.21.15.26:0/617091719 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280072b0 con 0x7fc42c0c2b20 2024-03-06T07:26:32.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.789+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c070150 con 0x7fc42c0c2b20 2024-03-06T07:26:32.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.790+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c04f330 con 0x7fc42c0c2b20 2024-03-06T07:26:32.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.791+0000 7fc4227fc640 1 -- 172.21.15.26:0/617091719 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc42801a070 con 0x7fc42c0c2b20 2024-03-06T07:26:32.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.791+0000 7fc4227fc640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41006b4d0 0x7fc41006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:32.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.791+0000 7fc4227fc640 1 -- 172.21.15.26:0/617091719 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fc428089990 con 0x7fc42c0c2b20 2024-03-06T07:26:32.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.791+0000 7fc431392640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41006b4d0 0x7fc41006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:32.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.792+0000 7fc431392640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41006b4d0 0x7fc41006d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc41c007fd0 tx=0x7fc41c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:32.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:32.795+0000 7fc4227fc640 1 -- 172.21.15.26:0/617091719 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc428057c80 con 0x7fc42c0c2b20 2024-03-06T07:26:33.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.014+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc42c14f900 con 0x7fc42c0c2b20 2024-03-06T07:26:33.016 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:26:33.014+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:26:33.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.015+0000 7fc4227fc640 1 -- 172.21.15.26:0/617091719 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v149) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc42805bd20 con 0x7fc42c0c2b20 2024-03-06T07:26:33.017 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41006b4d0 msgr2=0x7fc41006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41006b4d0 0x7fc41006d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc41c007fd0 tx=0x7fc41c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc42c0c2b20 msgr2=0x7fc42c06d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc42c0c2b20 0x7fc42c06d230 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fc42800c460 tx=0x7fc42800c920 comp rx=0 tx=0).stop 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 shutdown_connections 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41006b4d0 0x7fc41006d990 secure :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc41c007fd0 tx=0x7fc41c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc42c1526c0 0x7fc42c1e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42c0c69b0 0x7fc42c14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 --2- 172.21.15.26:0/617091719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc42c0c2b20 0x7fc42c06d230 secure :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fc42800c460 tx=0x7fc42800c920 comp rx=0 tx=0).stop 2024-03-06T07:26:33.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 >> 172.21.15.26:0/617091719 conn(0x7fc42c0aa810 msgr2=0x7fc42c0a8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:33.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 shutdown_connections 2024-03-06T07:26:33.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.018+0000 7fc433df7640 1 -- 172.21.15.26:0/617091719 wait complete. 2024-03-06T07:26:33.022 INFO:tasks.ceph.mds.b.smithi117.stderr: -199> 2024-03-06T07:26:27.983+0000 7eff1b895640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:33.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:26:33.046 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:26:33.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.132+0000 7fde50637640 1 Processor -- start 2024-03-06T07:26:33.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.132+0000 7fde50637640 1 -- start start 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.133+0000 7fde50637640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.133+0000 7fde50637640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde480a7cd0 0x7fde48155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.133+0000 7fde50637640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde48152b10 con 0x7fde48152730 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.133+0000 7fde50637640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde48158ad0 con 0x7fde480a7cd0 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.133+0000 7fde50637640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde48158db0 con 0x7fde480c6710 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.133+0000 7fde4e3d3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde48152730 0x7fde480c3210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49542/0 (socket says 172.21.15.26:49542) 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4e3d3640 1 -- 172.21.15.26:0/2058053708 learned_addr learned my addr 172.21.15.26:0/2058053708 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4ebd4640 1 --2- 172.21.15.26:0/2058053708 >> v2:172.21.15.117:3300/0 conn(0x7fde480a7cd0 0x7fde48155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4dbd2640 1 --2- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4d3d1640 1 -- 172.21.15.26:0/2058053708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134671625 0 0) 0x7fde48152b10 con 0x7fde48152730 2024-03-06T07:26:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4d3d1640 1 -- 172.21.15.26:0/2058053708 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde34003520 con 0x7fde48152730 2024-03-06T07:26:33.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4ebd4640 1 -- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 msgr2=0x7fde481554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:33.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4ebd4640 1 --2- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4ebd4640 1 -- 172.21.15.26:0/2058053708 >> v1:172.21.15.26:6789/0 conn(0x7fde48152730 legacy=0x7fde480c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4ebd4640 1 -- 172.21.15.26:0/2058053708 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48159090 con 0x7fde480a7cd0 2024-03-06T07:26:33.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.134+0000 7fde4dbd2640 1 --2- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde4ebd4640 1 --2- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48155a00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fde40009d90 tx=0x7fde4002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709d1e96f413d868 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde4d3d1640 1 -- 172.21.15.26:0/2058053708 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40039070 con 0x7fde480a7cd0 2024-03-06T07:26:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde4d3d1640 1 -- 172.21.15.26:0/2058053708 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4002cbc0 con 0x7fde480a7cd0 2024-03-06T07:26:33.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde4d3d1640 1 -- 172.21.15.26:0/2058053708 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40034440 con 0x7fde480a7cd0 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 -- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 msgr2=0x7fde48155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 --2- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48155a00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fde40009d90 tx=0x7fde4002be10 comp rx=0 tx=0).stop 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 -- 172.21.15.26:0/2058053708 shutdown_connections 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 --2- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48155a00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 --2- 172.21.15.26:0/2058053708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 -- 172.21.15.26:0/2058053708 >> 172.21.15.26:0/2058053708 conn(0x7fde480aa5c0 msgr2=0x7fde480aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 -- 172.21.15.26:0/2058053708 shutdown_connections 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.135+0000 7fde50637640 1 -- 172.21.15.26:0/2058053708 wait complete. 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.136+0000 7fde50637640 1 Processor -- start 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde50637640 1 -- start start 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde50637640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde50637640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde50637640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde48152730 0x7fde48154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde50637640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c3aa0 con 0x7fde48152730 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde50637640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c3920 con 0x7fde480a7cd0 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde50637640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde481586f0 con 0x7fde480c6710 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde4ebd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde48152730 0x7fde48154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde4ebd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde48152730 0x7fde48154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42888/0 (socket says 172.21.15.26:42888) 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde4ebd4640 1 -- 172.21.15.26:0/3900612617 learned_addr learned my addr 172.21.15.26:0/3900612617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde4dbd2640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.137+0000 7fde4e3d3640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde4ebd4640 1 -- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 msgr2=0x7fde481edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde4ebd4640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde4ebd4640 1 -- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 msgr2=0x7fde48070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde4ebd4640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde4ebd4640 1 -- 172.21.15.26:0/3900612617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48070e90 con 0x7fde48152730 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde4e3d3640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:33.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde4ebd4640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde48152730 0x7fde48154230 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fde40009d90 tx=0x7fde40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde3effd640 1 -- 172.21.15.26:0/3900612617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40039050 con 0x7fde48152730 2024-03-06T07:26:33.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde3effd640 1 -- 172.21.15.26:0/3900612617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40045070 con 0x7fde48152730 2024-03-06T07:26:33.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4806df60 con 0x7fde48152730 2024-03-06T07:26:33.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde3effd640 1 -- 172.21.15.26:0/3900612617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40009450 con 0x7fde48152730 2024-03-06T07:26:33.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.138+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde48070150 con 0x7fde48152730 2024-03-06T07:26:33.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.139+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4804f330 con 0x7fde48152730 2024-03-06T07:26:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.139+0000 7fde3effd640 1 -- 172.21.15.26:0/3900612617 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde4003e850 con 0x7fde48152730 2024-03-06T07:26:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.140+0000 7fde3effd640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.140+0000 7fde3effd640 1 -- 172.21.15.26:0/3900612617 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fde400ad6a0 con 0x7fde48152730 2024-03-06T07:26:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.140+0000 7fde4e3d3640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.140+0000 7fde4e3d3640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde2c06b4d0 0x7fde2c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.144+0000 7fde3effd640 1 -- 172.21.15.26:0/3900612617 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fde4007c030 con 0x7fde48152730 2024-03-06T07:26:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:33.362+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fde48073330 con 0x7fde48152730 2024-03-06T07:26:34.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.019+0000 7fde3effd640 1 -- 172.21.15.26:0/3900612617 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v150) v1 ==== 104+0+0 (secure 0 0 0) 0x7fde4007b930 con 0x7fde48152730 2024-03-06T07:26:34.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde2c06b4d0 msgr2=0x7fde2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde2c06b4d0 0x7fde2c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44009040 comp rx=0 tx=0).stop 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde48152730 msgr2=0x7fde48154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde48152730 0x7fde48154230 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fde40009d90 tx=0x7fde40004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 shutdown_connections 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde48152730 0x7fde48154230 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde480c6710 0x7fde481edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 --2- 172.21.15.26:0/3900612617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde480a7cd0 0x7fde48070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 >> 172.21.15.26:0/3900612617 conn(0x7fde480aa5c0 msgr2=0x7fde48072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 shutdown_connections 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.022+0000 7fde50637640 1 -- 172.21.15.26:0/3900612617 wait complete. 2024-03-06T07:26:34.028 INFO:tasks.ceph.mds.d.smithi117.stderr: -401> 2024-03-06T07:26:22.982+0000 7f254b9a7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:34.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:26:34.053 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:26:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.150+0000 7f66f65a0640 1 Processor -- start 2024-03-06T07:26:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.150+0000 7f66f65a0640 1 -- start start 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f65a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66f014f350 0x7f66f00c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f65a0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f66f006c080 0x7f66f00c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f65a0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00c5d40 con 0x7f66f00c51c0 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f65a0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f006bbf0 con 0x7f66f014f350 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f65a0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f00c5780 con 0x7f66f006c080 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f559e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66f014f350 0x7f66f00c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f4d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f66f00c51c0 0x7f66f0155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49544/0 (socket says 172.21.15.26:49544) 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f4d9d640 1 -- 172.21.15.26:0/2134011966 learned_addr learned my addr 172.21.15.26:0/2134011966 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f559e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66f014f350 0x7f66f00c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42894/0 (socket says 172.21.15.26:42894) 2024-03-06T07:26:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f5d9f640 1 --2- 172.21.15.26:0/2134011966 >> v2:172.21.15.117:3300/0 conn(0x7f66f006c080 0x7f66f00c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f559e640 1 -- 172.21.15.26:0/2134011966 >> v2:172.21.15.117:3300/0 conn(0x7f66f006c080 msgr2=0x7f66f00c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f559e640 1 --2- 172.21.15.26:0/2134011966 >> v2:172.21.15.117:3300/0 conn(0x7f66f006c080 0x7f66f00c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f559e640 1 -- 172.21.15.26:0/2134011966 >> v1:172.21.15.26:6789/0 conn(0x7f66f00c51c0 legacy=0x7f66f0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66f559e640 1 -- 172.21.15.26:0/2134011966 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f00c5a60 con 0x7f66f014f350 2024-03-06T07:26:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.151+0000 7f66e7fff640 1 -- 172.21.15.26:0/2134011966 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181648597 0 0) 0x7f66f00c5d40 con 0x7f66f00c51c0 2024-03-06T07:26:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.152+0000 7f66f559e640 1 --2- 172.21.15.26:0/2134011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f014f350 0x7f66f00c8bc0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f66f00aa2d0 tx=0x7f66e802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fadcf20d9470ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.152+0000 7f66e7fff640 1 -- 172.21.15.26:0/2134011966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e80315a0 con 0x7f66f014f350 2024-03-06T07:26:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.152+0000 7f66e7fff640 1 -- 172.21.15.26:0/2134011966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8037070 con 0x7f66f014f350 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.152+0000 7f66e7fff640 1 -- 172.21.15.26:0/2134011966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8031960 con 0x7f66f014f350 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 -- 172.21.15.26:0/2134011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f014f350 msgr2=0x7f66f00c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 --2- 172.21.15.26:0/2134011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f014f350 0x7f66f00c8bc0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f66f00aa2d0 tx=0x7f66e802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 -- 172.21.15.26:0/2134011966 shutdown_connections 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 --2- 172.21.15.26:0/2134011966 >> v2:172.21.15.117:3300/0 conn(0x7f66f006c080 0x7f66f00c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 --2- 172.21.15.26:0/2134011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f014f350 0x7f66f00c8bc0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 -- 172.21.15.26:0/2134011966 >> 172.21.15.26:0/2134011966 conn(0x7f66f00ac3d0 msgr2=0x7f66f00ac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 -- 172.21.15.26:0/2134011966 shutdown_connections 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.153+0000 7f66f65a0640 1 -- 172.21.15.26:0/2134011966 wait complete. 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.154+0000 7f66f65a0640 1 Processor -- start 2024-03-06T07:26:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.154+0000 7f66f65a0640 1 -- start start 2024-03-06T07:26:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f65a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f006c080 0x7f66f0158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f65a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f00c51c0 0x7f66f00c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f65a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f0145140 0x7f66f01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f65a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f006c650 con 0x7f66f0145140 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f65a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f006c4d0 con 0x7f66f006c080 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f65a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00c7760 con 0x7f66f00c51c0 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f0145140 0x7f66f01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f0145140 0x7f66f01445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42906/0 (socket says 172.21.15.26:42906) 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 -- 172.21.15.26:0/3012188599 learned_addr learned my addr 172.21.15.26:0/3012188599 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f559e640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f006c080 0x7f66f0158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f4d9d640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f00c51c0 0x7f66f00c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 -- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f00c51c0 msgr2=0x7f66f00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f00c51c0 0x7f66f00c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 -- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f006c080 msgr2=0x7f66f0158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f006c080 0x7f66f0158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 -- 172.21.15.26:0/3012188599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f01eaed0 con 0x7f66f0145140 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f5d9f640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f0145140 0x7f66f01445a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00efc0 tx=0x7f66ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66e5ffb640 1 -- 172.21.15.26:0/3012188599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec018020 con 0x7f66f0145140 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66e5ffb640 1 -- 172.21.15.26:0/3012188599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec014480 con 0x7f66f0145140 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f01eb1b0 con 0x7f66f0145140 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.155+0000 7f66e5ffb640 1 -- 172.21.15.26:0/3012188599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec014870 con 0x7f66f0145140 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.156+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f01eb770 con 0x7f66f0145140 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.157+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f00684b0 con 0x7f66f0145140 2024-03-06T07:26:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.157+0000 7f66e5ffb640 1 -- 172.21.15.26:0/3012188599 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66ec01b020 con 0x7f66f0145140 2024-03-06T07:26:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.157+0000 7f66e5ffb640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.158+0000 7f66e5ffb640 1 -- 172.21.15.26:0/3012188599 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7215+0+0 (secure 0 0 0) 0x7f66ec08b330 con 0x7f66f0145140 2024-03-06T07:26:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.158+0000 7f66f559e640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.158+0000 7f66f559e640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66d406b4d0 0x7f66d406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f66e8036000 tx=0x7f66e8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.162+0000 7f66e5ffb640 1 -- 172.21.15.26:0/3012188599 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f66ec022030 con 0x7f66f0145140 2024-03-06T07:26:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:34.343+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66f00c8740 con 0x7f66f0145140 2024-03-06T07:26:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.024+0000 7f66e5ffb640 1 -- 172.21.15.26:0/3012188599 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v159) v1 ==== 181+0+0 (secure 0 0 0) 0x7f66ec010040 con 0x7f66f0145140 2024-03-06T07:26:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66d406b4d0 msgr2=0x7f66d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66d406b4d0 0x7f66d406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f66e8036000 tx=0x7f66e8008030 comp rx=0 tx=0).stop 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f0145140 msgr2=0x7f66f01445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f0145140 0x7f66f01445a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00efc0 tx=0x7f66ec00c490 comp rx=0 tx=0).stop 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 shutdown_connections 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66f0145140 0x7f66f01445a0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f00c51c0 0x7f66f00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 --2- 172.21.15.26:0/3012188599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f006c080 0x7f66f0158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 >> 172.21.15.26:0/3012188599 conn(0x7f66f00ac3d0 msgr2=0x7f66f014bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 shutdown_connections 2024-03-06T07:26:35.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.029+0000 7f66f65a0640 1 -- 172.21.15.26:0/3012188599 wait complete. 2024-03-06T07:26:35.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:26:35.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.156+0000 7f291ea2a640 1 Processor -- start 2024-03-06T07:26:35.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.156+0000 7f291ea2a640 1 -- start start 2024-03-06T07:26:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.156+0000 7f291ea2a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2918152920 0x7f2918155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.156+0000 7f291ea2a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2918155eb0 0x7f29180c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.156+0000 7f291ea2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2918159740 con 0x7f29180c80d0 2024-03-06T07:26:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.156+0000 7f291ea2a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29181551d0 con 0x7f2918155eb0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.156+0000 7f291ea2a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180c8ab0 con 0x7f2918152920 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f29177fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2918155eb0 0x7f29180c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f291cfc7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f29180c80d0 0x7f29180724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49554/0 (socket says 172.21.15.26:49554) 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f291cfc7640 1 -- 172.21.15.26:0/2850655221 learned_addr learned my addr 172.21.15.26:0/2850655221 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f2917fff640 1 --2- 172.21.15.26:0/2850655221 >> v2:172.21.15.117:3300/0 conn(0x7f2918152920 0x7f2918155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f29177fe640 1 -- 172.21.15.26:0/2850655221 >> v2:172.21.15.117:3300/0 conn(0x7f2918152920 msgr2=0x7f2918155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f29177fe640 1 --2- 172.21.15.26:0/2850655221 >> v2:172.21.15.117:3300/0 conn(0x7f2918152920 0x7f2918155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f29177fe640 1 -- 172.21.15.26:0/2850655221 >> v1:172.21.15.26:6789/0 conn(0x7f29180c80d0 legacy=0x7f29180724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f29177fe640 1 -- 172.21.15.26:0/2850655221 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180c8d90 con 0x7f2918155eb0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.157+0000 7f2916ffd640 1 -- 172.21.15.26:0/2850655221 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198770776 0 0) 0x7f2918159740 con 0x7f29180c80d0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f29177fe640 1 --2- 172.21.15.26:0/2850655221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918155eb0 0x7f29180c7b90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2918073610 tx=0x7f290c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d60ac76d637ac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f2916ffd640 1 -- 172.21.15.26:0/2850655221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0315a0 con 0x7f2918155eb0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f2916ffd640 1 -- 172.21.15.26:0/2850655221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c037070 con 0x7f2918155eb0 2024-03-06T07:26:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f2916ffd640 1 -- 172.21.15.26:0/2850655221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c031960 con 0x7f2918155eb0 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 -- 172.21.15.26:0/2850655221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918155eb0 msgr2=0x7f29180c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 --2- 172.21.15.26:0/2850655221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918155eb0 0x7f29180c7b90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2918073610 tx=0x7f290c02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 -- 172.21.15.26:0/2850655221 shutdown_connections 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 --2- 172.21.15.26:0/2850655221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918155eb0 0x7f29180c7b90 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 --2- 172.21.15.26:0/2850655221 >> v2:172.21.15.117:3300/0 conn(0x7f2918152920 0x7f2918155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 -- 172.21.15.26:0/2850655221 >> 172.21.15.26:0/2850655221 conn(0x7f29180aa5c0 msgr2=0x7f29180aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 -- 172.21.15.26:0/2850655221 shutdown_connections 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.158+0000 7f291ea2a640 1 -- 172.21.15.26:0/2850655221 wait complete. 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 Processor -- start 2024-03-06T07:26:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 -- start start 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29180c80d0 0x7f291806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918152920 0x7f29180ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2918155eb0 0x7f29180a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29180c86c0 con 0x7f2918152920 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2918155450 con 0x7f29180c80d0 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.160+0000 7f291ea2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29180c8840 con 0x7f2918155eb0 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918152920 0x7f29180ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918152920 0x7f29180ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42920/0 (socket says 172.21.15.26:42920) 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 -- 172.21.15.26:0/326812959 learned_addr learned my addr 172.21.15.26:0/326812959 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f2917fff640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29180c80d0 0x7f291806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f291cfc7640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2918155eb0 0x7f29180a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 -- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2918155eb0 msgr2=0x7f29180a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2918155eb0 0x7f29180a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 -- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29180c80d0 msgr2=0x7f291806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29180c80d0 0x7f291806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 -- 172.21.15.26:0/326812959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29181eb180 con 0x7f2918152920 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f29177fe640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918152920 0x7f29180ac700 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f290c009890 tx=0x7f290c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f2914ff9640 1 -- 172.21.15.26:0/326812959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c032640 con 0x7f2918152920 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f2914ff9640 1 -- 172.21.15.26:0/326812959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c032800 con 0x7f2918152920 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2918156760 con 0x7f2918152920 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f2914ff9640 1 -- 172.21.15.26:0/326812959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c032bf0 con 0x7f2918152920 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.161+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29180c2690 con 0x7f2918152920 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.162+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29181576b0 con 0x7f2918152920 2024-03-06T07:26:35.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.163+0000 7f2914ff9640 1 -- 172.21.15.26:0/326812959 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f290c04c020 con 0x7f2918152920 2024-03-06T07:26:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.163+0000 7f2914ff9640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f290406b4d0 0x7f290406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.163+0000 7f2914ff9640 1 -- 172.21.15.26:0/326812959 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f290c02f020 con 0x7f2918152920 2024-03-06T07:26:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.163+0000 7f2917fff640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f290406b4d0 0x7f290406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.164+0000 7f2917fff640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f290406b4d0 0x7f290406d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2908006fd0 tx=0x7f2908008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.167+0000 7f2914ff9640 1 -- 172.21.15.26:0/326812959 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f29181576b0 con 0x7f2918152920 2024-03-06T07:26:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:35.349+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f291804f330 con 0x7f2918152920 2024-03-06T07:26:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.040+0000 7f2914ff9640 1 -- 172.21.15.26:0/326812959 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v160) v1 ==== 169+0+0 (secure 0 0 0) 0x7f290c040070 con 0x7f2918152920 2024-03-06T07:26:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:26:36.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f290406b4d0 msgr2=0x7f290406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f290406b4d0 0x7f290406d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2908006fd0 tx=0x7f2908008040 comp rx=0 tx=0).stop 2024-03-06T07:26:36.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918152920 msgr2=0x7f29180ac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918152920 0x7f29180ac700 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f290c009890 tx=0x7f290c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 shutdown_connections 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f290406b4d0 0x7f290406d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2918155eb0 0x7f29180a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2918152920 0x7f29180ac700 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 --2- 172.21.15.26:0/326812959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29180c80d0 0x7f291806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.042+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 >> 172.21.15.26:0/326812959 conn(0x7f29180aa5c0 msgr2=0x7f2918151f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.043+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 shutdown_connections 2024-03-06T07:26:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.043+0000 7f291ea2a640 1 -- 172.21.15.26:0/326812959 wait complete. 2024-03-06T07:26:36.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.163+0000 7f04a924e640 1 Processor -- start 2024-03-06T07:26:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.163+0000 7f04a924e640 1 -- start start 2024-03-06T07:26:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a924e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f04a40c6780 0x7f04a4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a924e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04a41592d0 0x7f04a40c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a924e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a4072630 con 0x7f04a406da40 2024-03-06T07:26:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a924e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a406de20 con 0x7f04a41592d0 2024-03-06T07:26:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a924e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a4072910 con 0x7f04a40c6780 2024-03-06T07:26:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f04a406da40 0x7f04a4155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49570/0 (socket says 172.21.15.26:49570) 2024-03-06T07:26:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04a41592d0 0x7f04a40c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a2d9d640 1 -- 172.21.15.26:0/651831990 learned_addr learned my addr 172.21.15.26:0/651831990 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a1d9b640 1 -- 172.21.15.26:0/651831990 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215952817 0 0) 0x7f04a4072630 con 0x7f04a406da40 2024-03-06T07:26:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a1d9b640 1 -- 172.21.15.26:0/651831990 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0490003520 con 0x7f04a406da40 2024-03-06T07:26:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a359e640 1 -- 172.21.15.26:0/651831990 >> v2:172.21.15.117:3300/0 conn(0x7f04a40c6780 msgr2=0x7f04a4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a359e640 1 --2- 172.21.15.26:0/651831990 >> v2:172.21.15.117:3300/0 conn(0x7f04a40c6780 0x7f04a4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a359e640 1 -- 172.21.15.26:0/651831990 >> v1:172.21.15.26:6789/0 conn(0x7f04a406da40 legacy=0x7f04a4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.164+0000 7f04a359e640 1 -- 172.21.15.26:0/651831990 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a4072bf0 con 0x7f04a41592d0 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.165+0000 7f04a359e640 1 --2- 172.21.15.26:0/651831990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a40c3280 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f04a40c7120 tx=0x7f049402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb38b485a2c89490 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.165+0000 7f04a1d9b640 1 -- 172.21.15.26:0/651831990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494039070 con 0x7f04a41592d0 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.165+0000 7f04a1d9b640 1 -- 172.21.15.26:0/651831990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049402cbf0 con 0x7f04a41592d0 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.165+0000 7f04a1d9b640 1 -- 172.21.15.26:0/651831990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494034440 con 0x7f04a41592d0 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 -- 172.21.15.26:0/651831990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 msgr2=0x7f04a40c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 --2- 172.21.15.26:0/651831990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a40c3280 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f04a40c7120 tx=0x7f049402be40 comp rx=0 tx=0).stop 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 -- 172.21.15.26:0/651831990 shutdown_connections 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 --2- 172.21.15.26:0/651831990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a40c3280 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 --2- 172.21.15.26:0/651831990 >> v2:172.21.15.117:3300/0 conn(0x7f04a40c6780 0x7f04a4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 -- 172.21.15.26:0/651831990 >> 172.21.15.26:0/651831990 conn(0x7f04a40aa630 msgr2=0x7f04a40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 -- 172.21.15.26:0/651831990 shutdown_connections 2024-03-06T07:26:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.166+0000 7f04a924e640 1 -- 172.21.15.26:0/651831990 wait complete. 2024-03-06T07:26:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.167+0000 7f04a924e640 1 Processor -- start 2024-03-06T07:26:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.167+0000 7f04a924e640 1 -- start start 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a924e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a406da40 0x7f04a4157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a924e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a40c6780 0x7f04a40c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a924e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a41513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a924e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c3940 con 0x7f04a41592d0 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a924e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c37c0 con 0x7f04a40c6780 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a924e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c3ac0 con 0x7f04a406da40 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a41513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a41513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42940/0 (socket says 172.21.15.26:42940) 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 -- 172.21.15.26:0/665223937 learned_addr learned my addr 172.21.15.26:0/665223937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a2d9d640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a406da40 0x7f04a4157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a259c640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a40c6780 0x7f04a40c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 -- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a406da40 msgr2=0x7f04a4157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a406da40 0x7f04a4157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 -- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a40c6780 msgr2=0x7f04a40c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a40c6780 0x7f04a40c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a359e640 1 -- 172.21.15.26:0/665223937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a41586c0 con 0x7f04a41592d0 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.168+0000 7f04a2d9d640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a406da40 0x7f04a4157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.169+0000 7f04a359e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a41513a0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f0494009e80 tx=0x7f0494004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.169+0000 7f049b7fe640 1 -- 172.21.15.26:0/665223937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494039050 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.169+0000 7f049b7fe640 1 -- 172.21.15.26:0/665223937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494045070 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.169+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a414f7e0 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.169+0000 7f049b7fe640 1 -- 172.21.15.26:0/665223937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494009450 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.169+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a40ae5c0 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.170+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a404f330 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.170+0000 7f049b7fe640 1 -- 172.21.15.26:0/665223937 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f049403e830 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.171+0000 7f049b7fe640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048406b4d0 0x7f048406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.171+0000 7f049b7fe640 1 -- 172.21.15.26:0/665223937 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f04940ad250 con 0x7f04a41592d0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.171+0000 7f04a2d9d640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048406b4d0 0x7f048406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.171+0000 7f04a2d9d640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048406b4d0 0x7f048406d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f049c005880 tx=0x7f049c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.175+0000 7f049b7fe640 1 -- 172.21.15.26:0/665223937 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f04a414f7e0 con 0x7f04a41592d0 2024-03-06T07:26:36.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.356+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04a4073240 con 0x7f04a41592d0 2024-03-06T07:26:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.357+0000 7f049b7fe640 1 -- 172.21.15.26:0/665223937 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+17293 (secure 0 0 0) 0x7f0494032e20 con 0x7f04a41592d0 2024-03-06T07:26:36.359 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:36.359 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":160,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:36.034371+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.117:6834/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.117:6835/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6833/3029467671":"2024-03-07T07:26:23.875878+0000","172.21.15.117:6837/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6832/3029467671":"2024-03-07T07:26:23.875878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048406b4d0 msgr2=0x7f048406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048406b4d0 0x7f048406d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f049c005880 tx=0x7f049c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 msgr2=0x7f04a41513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a41513a0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f0494009e80 tx=0x7f0494004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 shutdown_connections 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f048406b4d0 0x7f048406d990 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a41592d0 0x7f04a41513a0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a40c6780 0x7f04a40c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 --2- 172.21.15.26:0/665223937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04a406da40 0x7f04a4157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 >> 172.21.15.26:0/665223937 conn(0x7f04a40aa630 msgr2=0x7f04a406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 shutdown_connections 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.359+0000 7f04a924e640 1 -- 172.21.15.26:0/665223937 wait complete. 2024-03-06T07:26:36.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:26:15.400219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:26:16.061363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:36.372 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.478+0000 7efc60ba0640 1 Processor -- start 2024-03-06T07:26:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.478+0000 7efc60ba0640 1 -- start start 2024-03-06T07:26:36.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc60ba0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 0x7efc5c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc60ba0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c1552c0 con 0x7efc5c06afc0 2024-03-06T07:26:36.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc60ba0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c0c6d30 con 0x7efc5c0c8d50 2024-03-06T07:26:36.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc60ba0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c155580 con 0x7efc5c152a20 2024-03-06T07:26:36.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc5affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc5c06afc0 0x7efc5c0c8640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49572/0 (socket says 172.21.15.26:49572) 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc5affd640 1 -- 172.21.15.26:0/2741736385 learned_addr learned my addr 172.21.15.26:0/2741736385 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 0x7efc5c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899419099 0 0) 0x7efc5c1552c0 con 0x7efc5c06afc0 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.479+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc44003520 con 0x7efc5c06afc0 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991296408 0 0) 0x7efc44003520 con 0x7efc5c06afc0 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 msgr2=0x7efc5c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 0x7efc5c1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 >> v1:172.21.15.117:6789/0 conn(0x7efc5c0c8d50 legacy=0x7efc5c15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c155860 con 0x7efc5c06afc0 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7efc50001670 con 0x7efc5c06afc0 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 >> v1:172.21.15.26:6789/0 conn(0x7efc5c06afc0 legacy=0x7efc5c0c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 0x7efc44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc44004d40 0x7efc440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c155580 con 0x7efc440039e0 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4400bb50 con 0x7efc44004d40 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4400bd20 con 0x7efc5c152a20 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5affd640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 0x7efc44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5bfff640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc44004d40 0x7efc440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5b7fe640 1 -- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 msgr2=0x7efc44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 0x7efc44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5b7fe640 1 -- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc44004d40 msgr2=0x7efc440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc44004d40 0x7efc440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5b7fe640 1 -- 172.21.15.26:0/2741736385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc44003520 con 0x7efc440039e0 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5bfff640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc44004d40 0x7efc440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.480+0000 7efc5affd640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c152a20 0x7efc44004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.481+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc44003dc0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009af0 tx=0x7efc44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c9f54e3d15ecb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.481+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c039070 con 0x7efc440039e0 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.481+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c02c6b0 con 0x7efc440039e0 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.481+0000 7efc5a7fc640 1 -- 172.21.15.26:0/2741736385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c009690 con 0x7efc440039e0 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.482+0000 7efc60ba0640 1 -- 172.21.15.26:0/2741736385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 msgr2=0x7efc44003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.482+0000 7efc60ba0640 1 --2- 172.21.15.26:0/2741736385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc44003dc0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009af0 tx=0x7efc44005830 comp rx=0 tx=0).stop 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.482+0000 7efc5bfff640 1 -- 172.21.15.26:0/2741736385 reap_dead start 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.482+0000 7efc60ba0640 1 -- 172.21.15.26:0/2741736385 shutdown_connections 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.482+0000 7efc60ba0640 1 -- 172.21.15.26:0/2741736385 >> 172.21.15.26:0/2741736385 conn(0x7efc5c0ca740 msgr2=0x7efc5c0cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.482+0000 7efc60ba0640 1 -- 172.21.15.26:0/2741736385 shutdown_connections 2024-03-06T07:26:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.482+0000 7efc60ba0640 1 -- 172.21.15.26:0/2741736385 wait complete. 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.483+0000 7efc60ba0640 1 Processor -- start 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.483+0000 7efc60ba0640 1 -- start start 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc5c1dc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc5c1e1e50 0x7efc5c0c0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c1ea840 0x7efc5c150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c156060 con 0x7efc440039e0 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c152e00 con 0x7efc5c1e1e50 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0bff50 con 0x7efc5c1ea840 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc5c1dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc5c1dc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42970/0 (socket says 172.21.15.26:42970) 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 -- 172.21.15.26:0/2636445998 learned_addr learned my addr 172.21.15.26:0/2636445998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5bfff640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c1ea840 0x7efc5c150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5affd640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc5c1e1e50 0x7efc5c0c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 -- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c1ea840 msgr2=0x7efc5c150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c1ea840 0x7efc5c150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 -- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc5c1e1e50 msgr2=0x7efc5c0c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc5c1e1e50 0x7efc5c0c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 -- 172.21.15.26:0/2636445998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0c1340 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc5b7fe640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc5c1dc030 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009af0 tx=0x7efc4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc43fff640 1 -- 172.21.15.26:0/2636445998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c039050 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc43fff640 1 -- 172.21.15.26:0/2636445998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c045070 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c0bf990 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc43fff640 1 -- 172.21.15.26:0/2636445998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c003a50 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.484+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c0c0970 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.485+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c0684b0 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.486+0000 7efc43fff640 1 -- 172.21.15.26:0/2636445998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc4c003bf0 con 0x7efc440039e0 2024-03-06T07:26:36.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.486+0000 7efc43fff640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc4806b4d0 0x7efc4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.486+0000 7efc43fff640 1 -- 172.21.15.26:0/2636445998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7efc4c0abf70 con 0x7efc440039e0 2024-03-06T07:26:36.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.486+0000 7efc5affd640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc4806b4d0 0x7efc4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.487+0000 7efc5affd640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc4806b4d0 0x7efc4806d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efc50001200 tx=0x7efc50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.491+0000 7efc43fff640 1 -- 172.21.15.26:0/2636445998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc4c04e020 con 0x7efc440039e0 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.681+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc5c0cb150 con 0x7efc440039e0 2024-03-06T07:26:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.681+0000 7efc43fff640 1 -- 172.21.15.26:0/2636445998 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc5c0cb150 con 0x7efc440039e0 2024-03-06T07:26:36.683 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc4806b4d0 msgr2=0x7efc4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc4806b4d0 0x7efc4806d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efc50001200 tx=0x7efc50009040 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 msgr2=0x7efc5c1dc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc5c1dc030 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009af0 tx=0x7efc4c004730 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 shutdown_connections 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc4806b4d0 0x7efc4806d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc5c1ea840 0x7efc5c150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc5c1e1e50 0x7efc5c0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 --2- 172.21.15.26:0/2636445998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc440039e0 0x7efc5c1dc030 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 >> 172.21.15.26:0/2636445998 conn(0x7efc5c0ca740 msgr2=0x7efc5c0c7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 shutdown_connections 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.684+0000 7efc60ba0640 1 -- 172.21.15.26:0/2636445998 wait complete. 2024-03-06T07:26:36.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:26:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.799+0000 7f70b5365640 1 Processor -- start 2024-03-06T07:26:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.799+0000 7f70b5365640 1 -- start start 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70b5365640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f70b00c5370 0x7f70b00c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70b5365640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f70b0155530 0x7f70b01559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70b5365640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b0152dd0 con 0x7f70b01529f0 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70b5365640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b0159710 con 0x7f70b0155530 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70b5365640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b00c68b0 con 0x7f70b00c5370 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70af7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f70b0155530 0x7f70b01559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70aeffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70b01529f0 0x7f70b00c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49586/0 (socket says 172.21.15.26:49586) 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70aeffd640 1 -- 172.21.15.26:0/3606263246 learned_addr learned my addr 172.21.15.26:0/3606263246 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70ae7fc640 1 --2- 172.21.15.26:0/3606263246 >> v2:172.21.15.117:3300/0 conn(0x7f70b00c5370 0x7f70b00c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70af7fe640 1 -- 172.21.15.26:0/3606263246 >> v2:172.21.15.117:3300/0 conn(0x7f70b00c5370 msgr2=0x7f70b00c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70af7fe640 1 --2- 172.21.15.26:0/3606263246 >> v2:172.21.15.117:3300/0 conn(0x7f70b00c5370 0x7f70b00c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70af7fe640 1 -- 172.21.15.26:0/3606263246 >> v1:172.21.15.26:6789/0 conn(0x7f70b01529f0 legacy=0x7f70b00c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70adffb640 1 -- 172.21.15.26:0/3606263246 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346075728 0 0) 0x7f70b0152dd0 con 0x7f70b01529f0 2024-03-06T07:26:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.800+0000 7f70af7fe640 1 -- 172.21.15.26:0/3606263246 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0158b00 con 0x7f70b0155530 2024-03-06T07:26:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.801+0000 7f70af7fe640 1 --2- 172.21.15.26:0/3606263246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b0155530 0x7f70b01559c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f70b008c9b0 tx=0x7f70a002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa98cfbaa2a86a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.801+0000 7f70adffb640 1 -- 172.21.15.26:0/3606263246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0039070 con 0x7f70b0155530 2024-03-06T07:26:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.801+0000 7f70adffb640 1 -- 172.21.15.26:0/3606263246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a002cbf0 con 0x7f70b0155530 2024-03-06T07:26:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.801+0000 7f70adffb640 1 -- 172.21.15.26:0/3606263246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0034440 con 0x7f70b0155530 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 -- 172.21.15.26:0/3606263246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b0155530 msgr2=0x7f70b01559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 --2- 172.21.15.26:0/3606263246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b0155530 0x7f70b01559c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f70b008c9b0 tx=0x7f70a002be40 comp rx=0 tx=0).stop 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 -- 172.21.15.26:0/3606263246 shutdown_connections 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 --2- 172.21.15.26:0/3606263246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b0155530 0x7f70b01559c0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 --2- 172.21.15.26:0/3606263246 >> v2:172.21.15.117:3300/0 conn(0x7f70b00c5370 0x7f70b00c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 -- 172.21.15.26:0/3606263246 >> 172.21.15.26:0/3606263246 conn(0x7f70b00ac580 msgr2=0x7f70b00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 -- 172.21.15.26:0/3606263246 shutdown_connections 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.802+0000 7f70b5365640 1 -- 172.21.15.26:0/3606263246 wait complete. 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.803+0000 7f70b5365640 1 Processor -- start 2024-03-06T07:26:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.803+0000 7f70b5365640 1 -- start start 2024-03-06T07:26:36.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.803+0000 7f70b5365640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b00c5370 0x7f70b00bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70b5365640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b01529f0 0x7f70b01ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70b5365640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b0155530 0x7f70b0157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70b5365640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b01590c0 con 0x7f70b01529f0 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70b5365640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00c9000 con 0x7f70b00c5370 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70b5365640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b01593a0 con 0x7f70b0155530 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b01529f0 0x7f70b01ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b01529f0 0x7f70b01ee2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42990/0 (socket says 172.21.15.26:42990) 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 -- 172.21.15.26:0/3489014792 learned_addr learned my addr 172.21.15.26:0/3489014792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70af7fe640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b0155530 0x7f70b0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70aeffd640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b00c5370 0x7f70b00bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 -- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b0155530 msgr2=0x7f70b0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b0155530 0x7f70b0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 -- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b00c5370 msgr2=0x7f70b00bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b00c5370 0x7f70b00bfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 -- 172.21.15.26:0/3489014792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b00adaa0 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70ae7fc640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b01529f0 0x7f70b01ee2b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f70a400d800 tx=0x7f70a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70977fe640 1 -- 172.21.15.26:0/3489014792 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400b6c0 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70977fe640 1 -- 172.21.15.26:0/3489014792 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4019070 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70977fe640 1 -- 172.21.15.26:0/3489014792 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400ba80 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b00add80 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.804+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b0158710 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.805+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b004f330 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.806+0000 7f70977fe640 1 -- 172.21.15.26:0/3489014792 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70a400bc20 con 0x7f70b01529f0 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.806+0000 7f70977fe640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f709c06b4d0 0x7f709c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.806+0000 7f70977fe640 1 -- 172.21.15.26:0/3489014792 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f70a408a8e0 con 0x7f70b01529f0 2024-03-06T07:26:36.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.806+0000 7f70aeffd640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f709c06b4d0 0x7f709c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.807+0000 7f70aeffd640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f709c06b4d0 0x7f709c06d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000f60 tx=0x7f70a8007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.811+0000 7f70977fe640 1 -- 172.21.15.26:0/3489014792 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70a4059560 con 0x7f70b01529f0 2024-03-06T07:26:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.992+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70b0158430 con 0x7f70b01529f0 2024-03-06T07:26:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.992+0000 7f70977fe640 1 -- 172.21.15.26:0/3489014792 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v160) v1 ==== 105+0+11 (secure 0 0 0) 0x7f70a4058e60 con 0x7f70b01529f0 2024-03-06T07:26:36.994 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f709c06b4d0 msgr2=0x7f709c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f709c06b4d0 0x7f709c06d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000f60 tx=0x7f70a8007b50 comp rx=0 tx=0).stop 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b01529f0 msgr2=0x7f70b01ee2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b01529f0 0x7f70b01ee2b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f70a400d800 tx=0x7f70a400dcc0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 shutdown_connections 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f709c06b4d0 0x7f709c06d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b0155530 0x7f70b0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b01529f0 0x7f70b01ee2b0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 --2- 172.21.15.26:0/3489014792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b00c5370 0x7f70b00bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 >> 172.21.15.26:0/3489014792 conn(0x7f70b00ac580 msgr2=0x7f70b00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 shutdown_connections 2024-03-06T07:26:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:36.994+0000 7f70b5365640 1 -- 172.21.15.26:0/3489014792 wait complete. 2024-03-06T07:26:37.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:26:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.111+0000 7ff9ba5c7640 1 Processor -- start 2024-03-06T07:26:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.111+0000 7ff9ba5c7640 1 -- start start 2024-03-06T07:26:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9ba5c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b4155c30 0x7ff9b406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9ba5c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9b40c5380 0x7ff9b40c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9ba5c7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40b0450 con 0x7ff9b40b0070 2024-03-06T07:26:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9ba5c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b406cfb0 con 0x7ff9b40c5380 2024-03-06T07:26:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9ba5c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b406c020 con 0x7ff9b4155c30 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b8b64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9b40c5380 0x7ff9b40c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff9b40b0070 0x7ff9b40c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57550/0 (socket says 172.21.15.26:57550) 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b3fff640 1 -- 172.21.15.26:0/2221287463 learned_addr learned my addr 172.21.15.26:0/2221287463 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b37fe640 1 --2- 172.21.15.26:0/2221287463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b4155c30 0x7ff9b406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b8b64640 1 -- 172.21.15.26:0/2221287463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b4155c30 msgr2=0x7ff9b406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b8b64640 1 --2- 172.21.15.26:0/2221287463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b4155c30 0x7ff9b406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b8b64640 1 -- 172.21.15.26:0/2221287463 >> v1:172.21.15.26:6789/0 conn(0x7ff9b40b0070 legacy=0x7ff9b40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.112+0000 7ff9b8b64640 1 -- 172.21.15.26:0/2221287463 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b41560e0 con 0x7ff9b40c5380 2024-03-06T07:26:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.113+0000 7ff9b37fe640 1 --2- 172.21.15.26:0/2221287463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b4155c30 0x7ff9b406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:37.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.113+0000 7ff9b8b64640 1 --2- 172.21.15.26:0/2221287463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b40c5380 0x7ff9b40c57f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009d90 tx=0x7ff9ac02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c4af24fe7ae28c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.113+0000 7ff9b2ffd640 1 -- 172.21.15.26:0/2221287463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac039070 con 0x7ff9b40c5380 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.113+0000 7ff9b2ffd640 1 -- 172.21.15.26:0/2221287463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac02cbd0 con 0x7ff9b40c5380 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.113+0000 7ff9b2ffd640 1 -- 172.21.15.26:0/2221287463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac034440 con 0x7ff9b40c5380 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/2221287463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b40c5380 msgr2=0x7ff9b40c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/2221287463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b40c5380 0x7ff9b40c57f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009d90 tx=0x7ff9ac02be20 comp rx=0 tx=0).stop 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/2221287463 shutdown_connections 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/2221287463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b40c5380 0x7ff9b40c57f0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/2221287463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b4155c30 0x7ff9b406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/2221287463 >> 172.21.15.26:0/2221287463 conn(0x7ff9b40ac590 msgr2=0x7ff9b40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/2221287463 shutdown_connections 2024-03-06T07:26:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.114+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/2221287463 wait complete. 2024-03-06T07:26:37.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.116+0000 7ff9ba5c7640 1 Processor -- start 2024-03-06T07:26:37.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.116+0000 7ff9ba5c7640 1 -- start start 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9ba5c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b40b0070 0x7ff9b40c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9ba5c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b40c5380 0x7ff9b40bef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9ba5c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b4155c30 0x7ff9b41ee150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9ba5c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b406c270 con 0x7ff9b4155c30 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9ba5c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40c7ad0 con 0x7ff9b40c5380 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9ba5c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b406c550 con 0x7ff9b40b0070 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9b8b64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b4155c30 0x7ff9b41ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9b8b64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b4155c30 0x7ff9b41ee150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53002/0 (socket says 172.21.15.26:53002) 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9b8b64640 1 -- 172.21.15.26:0/3073962627 learned_addr learned my addr 172.21.15.26:0/3073962627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.117+0000 7ff9b37fe640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b40c5380 0x7ff9b40bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b3fff640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b40b0070 0x7ff9b40c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b8b64640 1 -- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b40b0070 msgr2=0x7ff9b40c0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b8b64640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b40b0070 0x7ff9b40c0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b8b64640 1 -- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b40c5380 msgr2=0x7ff9b40bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b8b64640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b40c5380 0x7ff9b40bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b8b64640 1 -- 172.21.15.26:0/3073962627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40c13b0 con 0x7ff9b4155c30 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b37fe640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b40c5380 0x7ff9b40bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b8b64640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b4155c30 0x7ff9b41ee150 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009eb0 tx=0x7ff9ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b0ff9640 1 -- 172.21.15.26:0/3073962627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac039050 con 0x7ff9b4155c30 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b0ff9640 1 -- 172.21.15.26:0/3073962627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac045070 con 0x7ff9b4155c30 2024-03-06T07:26:37.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b414ef80 con 0x7ff9b4155c30 2024-03-06T07:26:37.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9b0ff9640 1 -- 172.21.15.26:0/3073962627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac03e830 con 0x7ff9b4155c30 2024-03-06T07:26:37.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.118+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b40bf460 con 0x7ff9b4155c30 2024-03-06T07:26:37.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.119+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b41eefe0 con 0x7ff9b4155c30 2024-03-06T07:26:37.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.120+0000 7ff9b0ff9640 1 -- 172.21.15.26:0/3073962627 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9ac03e9d0 con 0x7ff9b4155c30 2024-03-06T07:26:37.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.120+0000 7ff9b0ff9640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9a006b4d0 0x7ff9a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.120+0000 7ff9b0ff9640 1 -- 172.21.15.26:0/3073962627 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7ff9ac0ad220 con 0x7ff9b4155c30 2024-03-06T07:26:37.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.120+0000 7ff9b3fff640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9a006b4d0 0x7ff9a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.121+0000 7ff9b3fff640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9a006b4d0 0x7ff9a006d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000f60 tx=0x7ff9a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.129+0000 7ff9b0ff9640 1 -- 172.21.15.26:0/3073962627 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff9b41eefe0 con 0x7ff9b4155c30 2024-03-06T07:26:37.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.338+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7ff9b404f330 con 0x7ff9b4155c30 2024-03-06T07:26:37.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.338+0000 7ff9b0ff9640 1 -- 172.21.15.26:0/3073962627 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v160) v1 ==== 101+0+11 (secure 0 0 0) 0x7ff9ac032e20 con 0x7ff9b4155c30 2024-03-06T07:26:37.340 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9a006b4d0 msgr2=0x7ff9a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9a006b4d0 0x7ff9a006d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000f60 tx=0x7ff9a8009040 comp rx=0 tx=0).stop 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b4155c30 msgr2=0x7ff9b41ee150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b4155c30 0x7ff9b41ee150 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009eb0 tx=0x7ff9ac004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 shutdown_connections 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff9a006b4d0 0x7ff9a006d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9b4155c30 0x7ff9b41ee150 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b40c5380 0x7ff9b40bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 --2- 172.21.15.26:0/3073962627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9b40b0070 0x7ff9b40c0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 >> 172.21.15.26:0/3073962627 conn(0x7ff9b40ac590 msgr2=0x7ff9b40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 shutdown_connections 2024-03-06T07:26:37.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.340+0000 7ff9ba5c7640 1 -- 172.21.15.26:0/3073962627 wait complete. 2024-03-06T07:26:37.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.453+0000 7f2ffcdbd640 1 Processor -- start 2024-03-06T07:26:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.453+0000 7f2ffcdbd640 1 -- start start 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ffcdbd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ff8153a00 0x7f2ff80c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ffcdbd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff8153e90 con 0x7f2ff81571f0 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ffcdbd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80c8440 con 0x7f2ff8072720 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ffcdbd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff81564c0 con 0x7f2ff8153a00 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ff6d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2ff8153a00 0x7f2ff80c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2fedd9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2ff81571f0 0x7f2ff80c67f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57562/0 (socket says 172.21.15.26:57562) 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2fedd9b640 1 -- 172.21.15.26:0/2263829487 learned_addr learned my addr 172.21.15.26:0/2263829487 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ff6d9d640 1 -- 172.21.15.26:0/2263829487 >> v1:172.21.15.117:6789/0 conn(0x7f2ff8072720 legacy=0x7f2ff80c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ff6d9d640 1 -- 172.21.15.26:0/2263829487 >> v1:172.21.15.26:6789/0 conn(0x7f2ff81571f0 legacy=0x7f2ff80c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ff5d9b640 1 -- 172.21.15.26:0/2263829487 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665915654 0 0) 0x7f2fe0001110 con 0x7f2ff81571f0 2024-03-06T07:26:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.454+0000 7f2ff6d9d640 1 -- 172.21.15.26:0/2263829487 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8156710 con 0x7f2ff8153a00 2024-03-06T07:26:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ff6d9d640 1 --2- 172.21.15.26:0/2263829487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff80c7d10 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009d90 tx=0x7f2ff002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4685d1c2de89f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ff5d9b640 1 -- 172.21.15.26:0/2263829487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0039070 con 0x7f2ff8153a00 2024-03-06T07:26:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ff5d9b640 1 -- 172.21.15.26:0/2263829487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff002cbd0 con 0x7f2ff8153a00 2024-03-06T07:26:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ff5d9b640 1 -- 172.21.15.26:0/2263829487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0034440 con 0x7f2ff8153a00 2024-03-06T07:26:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/2263829487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 msgr2=0x7f2ff80c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/2263829487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff80c7d10 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009d90 tx=0x7f2ff002be20 comp rx=0 tx=0).stop 2024-03-06T07:26:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/2263829487 shutdown_connections 2024-03-06T07:26:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/2263829487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff80c7d10 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/2263829487 >> 172.21.15.26:0/2263829487 conn(0x7f2ff80aa6a0 msgr2=0x7f2ff80aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/2263829487 shutdown_connections 2024-03-06T07:26:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.456+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/2263829487 wait complete. 2024-03-06T07:26:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 Processor -- start 2024-03-06T07:26:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 -- start start 2024-03-06T07:26:37.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff8072720 0x7f2ff81ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff81571f0 0x7f2ff81e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c4ed0 con 0x7f2ff8153a00 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8156cd0 con 0x7f2ff8072720 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2ffcdbd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c5050 con 0x7f2ff81571f0 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.458+0000 7f2fedd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff8155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53020/0 (socket says 172.21.15.26:53020) 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 -- 172.21.15.26:0/3548578907 learned_addr learned my addr 172.21.15.26:0/3548578907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2ff659c640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff8072720 0x7f2ff81ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2ff6d9d640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff81571f0 0x7f2ff81e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 -- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff81571f0 msgr2=0x7f2ff81e8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff81571f0 0x7f2ff81e8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 -- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff8072720 msgr2=0x7f2ff81ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff8072720 0x7f2ff81ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 -- 172.21.15.26:0/3548578907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8151c50 con 0x7f2ff8153a00 2024-03-06T07:26:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fedd9b640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff8155c40 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c3f0 tx=0x7f2fe000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fef7fe640 1 -- 172.21.15.26:0/3548578907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00074e0 con 0x7f2ff8153a00 2024-03-06T07:26:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fef7fe640 1 -- 172.21.15.26:0/3548578907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe00047e0 con 0x7f2ff8153a00 2024-03-06T07:26:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff81516d0 con 0x7f2ff8153a00 2024-03-06T07:26:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2fef7fe640 1 -- 172.21.15.26:0/3548578907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0004bd0 con 0x7f2ff8153a00 2024-03-06T07:26:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.459+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff8151970 con 0x7f2ff8153a00 2024-03-06T07:26:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.460+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff804f330 con 0x7f2ff8153a00 2024-03-06T07:26:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.461+0000 7f2fef7fe640 1 -- 172.21.15.26:0/3548578907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fe0011070 con 0x7f2ff8153a00 2024-03-06T07:26:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.461+0000 7f2fef7fe640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2fd806b4d0 0x7f2fd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.461+0000 7f2fef7fe640 1 -- 172.21.15.26:0/3548578907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f2fe0089ee0 con 0x7f2ff8153a00 2024-03-06T07:26:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.461+0000 7f2ff659c640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2fd806b4d0 0x7f2fd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.462+0000 7f2ff659c640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2fd806b4d0 0x7f2fd806d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8000f60 tx=0x7f2fe8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.466+0000 7f2fef7fe640 1 -- 172.21.15.26:0/3548578907 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ff8151c50 con 0x7f2ff8153a00 2024-03-06T07:26:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.684+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ff80735c0 con 0x7f2ff8153a00 2024-03-06T07:26:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.685+0000 7f2fef7fe640 1 -- 172.21.15.26:0/3548578907 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4994 (secure 0 0 0) 0x7f2ff8153e90 con 0x7f2ff8153a00 2024-03-06T07:26:37.687 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:37.687 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":154,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7454,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1583030325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1583030325},{"type":"v1","addr":"172.21.15.117:6837","nonce":1583030325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":8668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1620185363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1620185363},{"type":"v1","addr":"172.21.15.117:6836","nonce":1620185363}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:37.040749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":158,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7166":{"gid":7166,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7166,"qdb_cluster":[7166]},"id":12}]} 2024-03-06T07:26:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 154 2024-03-06T07:26:37.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2fd806b4d0 msgr2=0x7f2fd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2fd806b4d0 0x7f2fd806d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8000f60 tx=0x7f2fe8009040 comp rx=0 tx=0).stop 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 msgr2=0x7f2ff8155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff8155c40 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c3f0 tx=0x7f2fe000c8b0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 shutdown_connections 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2fd806b4d0 0x7f2fd806d990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff81571f0 0x7f2ff81e8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff8153a00 0x7f2ff8155c40 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 --2- 172.21.15.26:0/3548578907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff8072720 0x7f2ff81ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 >> 172.21.15.26:0/3548578907 conn(0x7f2ff80aa6a0 msgr2=0x7f2ff80c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 shutdown_connections 2024-03-06T07:26:37.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.687+0000 7f2ffcdbd640 1 -- 172.21.15.26:0/3548578907 wait complete. 2024-03-06T07:26:37.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.799+0000 7fcf90839640 1 Processor -- start 2024-03-06T07:26:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.799+0000 7fcf90839640 1 -- start start 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.799+0000 7fcf90839640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c0c88c0 0x7fcf8c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.799+0000 7fcf90839640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcf8c158b60 0x7fcf8c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.799+0000 7fcf90839640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c0c73d0 con 0x7fcf8c14f360 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.799+0000 7fcf90839640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8c0c8dd0 con 0x7fcf8c158b60 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.799+0000 7fcf90839640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0be2e0 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf8ad9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcf8c158b60 0x7fcf8c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf81d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c0c88c0 0x7fcf8c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf8a59c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fcf8c14f360 0x7fcf8c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:50364/0 (socket says 172.21.15.26:50364) 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf8a59c640 1 -- 172.21.15.26:0/1780234914 learned_addr learned my addr 172.21.15.26:0/1780234914 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf81d9b640 1 -- 172.21.15.26:0/1780234914 >> v2:172.21.15.117:3300/0 conn(0x7fcf8c158b60 msgr2=0x7fcf8c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf81d9b640 1 --2- 172.21.15.26:0/1780234914 >> v2:172.21.15.117:3300/0 conn(0x7fcf8c158b60 0x7fcf8c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf81d9b640 1 -- 172.21.15.26:0/1780234914 >> v1:172.21.15.117:6789/0 conn(0x7fcf8c14f360 legacy=0x7fcf8c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf89d9b640 1 -- 172.21.15.26:0/1780234914 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31502566 0 0) 0x7fcf8c0c73d0 con 0x7fcf8c14f360 2024-03-06T07:26:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf81d9b640 1 -- 172.21.15.26:0/1780234914 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0c7670 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.800+0000 7fcf8ad9d640 1 --2- 172.21.15.26:0/1780234914 >> v2:172.21.15.117:3300/0 conn(0x7fcf8c158b60 0x7fcf8c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.801+0000 7fcf81d9b640 1 --2- 172.21.15.26:0/1780234914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c0c88c0 0x7fcf8c155c10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf740097a0 tx=0x7fcf7402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b39c13cccd2b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.801+0000 7fcf89d9b640 1 -- 172.21.15.26:0/1780234914 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf740315a0 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.801+0000 7fcf89d9b640 1 -- 172.21.15.26:0/1780234914 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf74037070 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.801+0000 7fcf89d9b640 1 -- 172.21.15.26:0/1780234914 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74031960 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 -- 172.21.15.26:0/1780234914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c0c88c0 msgr2=0x7fcf8c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 --2- 172.21.15.26:0/1780234914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c0c88c0 0x7fcf8c155c10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf740097a0 tx=0x7fcf7402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 -- 172.21.15.26:0/1780234914 shutdown_connections 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 --2- 172.21.15.26:0/1780234914 >> v2:172.21.15.117:3300/0 conn(0x7fcf8c158b60 0x7fcf8c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 --2- 172.21.15.26:0/1780234914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c0c88c0 0x7fcf8c155c10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 -- 172.21.15.26:0/1780234914 >> 172.21.15.26:0/1780234914 conn(0x7fcf8c0ac570 msgr2=0x7fcf8c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 -- 172.21.15.26:0/1780234914 shutdown_connections 2024-03-06T07:26:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.802+0000 7fcf90839640 1 -- 172.21.15.26:0/1780234914 wait complete. 2024-03-06T07:26:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.804+0000 7fcf90839640 1 Processor -- start 2024-03-06T07:26:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.804+0000 7fcf90839640 1 -- start start 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf90839640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf8c0c88c0 0x7fcf8c154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf90839640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf8c14f360 0x7fcf8c1e7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf90839640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c158b60 0x7fcf8c1e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf90839640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0c5a50 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf90839640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0c7c30 con 0x7fcf8c14f360 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf90839640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0c5bd0 con 0x7fcf8c158b60 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf8a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf8c0c88c0 0x7fcf8c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf8a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf8c0c88c0 0x7fcf8c154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53030/0 (socket says 172.21.15.26:53030) 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf8a59c640 1 -- 172.21.15.26:0/471939709 learned_addr learned my addr 172.21.15.26:0/471939709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf81d9b640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf8c14f360 0x7fcf8c1e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.805+0000 7fcf8ad9d640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c158b60 0x7fcf8c1e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf8a59c640 1 -- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c158b60 msgr2=0x7fcf8c1e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf8a59c640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c158b60 0x7fcf8c1e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf8a59c640 1 -- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf8c14f360 msgr2=0x7fcf8c1e7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf8a59c640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf8c14f360 0x7fcf8c1e7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf8a59c640 1 -- 172.21.15.26:0/471939709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c158630 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf8ad9d640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c158b60 0x7fcf8c1e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf8a59c640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf8c0c88c0 0x7fcf8c154280 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c4d0 tx=0x7fcf7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf837fe640 1 -- 172.21.15.26:0/471939709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c018070 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf837fe640 1 -- 172.21.15.26:0/471939709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c007500 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c157d90 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf837fe640 1 -- 172.21.15.26:0/471939709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c004a20 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.806+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c158350 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.807+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c04f330 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.807+0000 7fcf837fe640 1 -- 172.21.15.26:0/471939709 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf7c004bc0 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.808+0000 7fcf837fe640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.808+0000 7fcf837fe640 1 -- 172.21.15.26:0/471939709 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fcf7c0897e0 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.808+0000 7fcf81d9b640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.809+0000 7fcf81d9b640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcf74036000 tx=0x7fcf74008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.812+0000 7fcf837fe640 1 -- 172.21.15.26:0/471939709 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcf7c057d00 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.987 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-03-06T07:26:37.983+0000 7f059a823640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.993+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf8c06cda0 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.995+0000 7fcf837fe640 1 -- 172.21.15.26:0/471939709 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+17293 (secure 0 0 0) 0x7fcf7c05bda0 con 0x7fcf8c0c88c0 2024-03-06T07:26:37.997 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:37.997 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":160,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:36.034371+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.117:6834/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.117:6835/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6833/3029467671":"2024-03-07T07:26:23.875878+0000","172.21.15.117:6837/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6832/3029467671":"2024-03-07T07:26:23.875878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 msgr2=0x7fcf6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcf74036000 tx=0x7fcf74008030 comp rx=0 tx=0).stop 2024-03-06T07:26:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf8c0c88c0 msgr2=0x7fcf8c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf8c0c88c0 0x7fcf8c154280 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c4d0 tx=0x7fcf7c00c990 comp rx=0 tx=0).stop 2024-03-06T07:26:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 shutdown_connections 2024-03-06T07:26:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf8c158b60 0x7fcf8c1e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf8c14f360 0x7fcf8c1e7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 --2- 172.21.15.26:0/471939709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf8c0c88c0 0x7fcf8c154280 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 >> 172.21.15.26:0/471939709 conn(0x7fcf8c0ac570 msgr2=0x7fcf8c0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 shutdown_connections 2024-03-06T07:26:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:37.997+0000 7fcf90839640 1 -- 172.21.15.26:0/471939709 wait complete. 2024-03-06T07:26:38.010 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-06T07:26:38.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:38.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.112+0000 7f4489488640 1 Processor -- start 2024-03-06T07:26:38.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.112+0000 7f4489488640 1 -- start start 2024-03-06T07:26:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f4489488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f4484155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f4489488640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44840c7380 0x7f44840c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f4489488640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4484156050 con 0x7f44840c6780 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f4489488640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448406de20 con 0x7f44840c7380 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f4489488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4484072630 con 0x7f448406da40 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f44837fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44840c7380 0x7f44840c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f44827fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f44840c6780 0x7f4484158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57576/0 (socket says 172.21.15.26:57576) 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f44827fc640 1 -- 172.21.15.26:0/1907894350 learned_addr learned my addr 172.21.15.26:0/1907894350 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f4482ffd640 1 --2- 172.21.15.26:0/1907894350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f4484155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f44837fe640 1 -- 172.21.15.26:0/1907894350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 msgr2=0x7f4484155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f44837fe640 1 --2- 172.21.15.26:0/1907894350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f4484155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f44837fe640 1 -- 172.21.15.26:0/1907894350 >> v1:172.21.15.26:6789/0 conn(0x7f44840c6780 legacy=0x7f4484158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f44837fe640 1 -- 172.21.15.26:0/1907894350 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484072910 con 0x7f44840c7380 2024-03-06T07:26:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.113+0000 7f4482ffd640 1 --2- 172.21.15.26:0/1907894350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f4484155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.114+0000 7f44837fe640 1 --2- 172.21.15.26:0/1907894350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c7380 0x7f44840c3280 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f447c009d90 tx=0x7f447c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c87a37a08118fc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.114+0000 7f4481ffb640 1 -- 172.21.15.26:0/1907894350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c039070 con 0x7f44840c7380 2024-03-06T07:26:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.114+0000 7f4481ffb640 1 -- 172.21.15.26:0/1907894350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c02cbc0 con 0x7f44840c7380 2024-03-06T07:26:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.114+0000 7f4481ffb640 1 -- 172.21.15.26:0/1907894350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c034440 con 0x7f44840c7380 2024-03-06T07:26:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 -- 172.21.15.26:0/1907894350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c7380 msgr2=0x7f44840c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 --2- 172.21.15.26:0/1907894350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c7380 0x7f44840c3280 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f447c009d90 tx=0x7f447c02be10 comp rx=0 tx=0).stop 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 -- 172.21.15.26:0/1907894350 shutdown_connections 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 --2- 172.21.15.26:0/1907894350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c7380 0x7f44840c3280 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 --2- 172.21.15.26:0/1907894350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f4484155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 -- 172.21.15.26:0/1907894350 >> 172.21.15.26:0/1907894350 conn(0x7f44840aa630 msgr2=0x7f44840aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 -- 172.21.15.26:0/1907894350 shutdown_connections 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.115+0000 7f4489488640 1 -- 172.21.15.26:0/1907894350 wait complete. 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.116+0000 7f4489488640 1 Processor -- start 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.116+0000 7f4489488640 1 -- start start 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f448406fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c6780 0x7f44841eef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840c7380 0x7f44840a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44841594a0 con 0x7f44840c6780 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4484158890 con 0x7f44840c7380 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840c37c0 con 0x7f448406da40 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c6780 0x7f44841eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c6780 0x7f44841eef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53046/0 (socket says 172.21.15.26:53046) 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 -- 172.21.15.26:0/1051795134 learned_addr learned my addr 172.21.15.26:0/1051795134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4482ffd640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f448406fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44837fe640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840c7380 0x7f44840a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 -- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 msgr2=0x7f448406fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f448406fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 -- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840c7380 msgr2=0x7f44840a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840c7380 0x7f44840a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 -- 172.21.15.26:0/1051795134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44841ef450 con 0x7f44840c6780 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f44827fc640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c6780 0x7f44841eef10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c470 tx=0x7f446c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f447b7fe640 1 -- 172.21.15.26:0/1051795134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c017070 con 0x7f44840c6780 2024-03-06T07:26:38.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f447b7fe640 1 -- 172.21.15.26:0/1051795134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c005ce0 con 0x7f44840c6780 2024-03-06T07:26:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4484070090 con 0x7f44840c6780 2024-03-06T07:26:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f447b7fe640 1 -- 172.21.15.26:0/1051795134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0072f0 con 0x7f44840c6780 2024-03-06T07:26:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.117+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44840acc00 con 0x7f44840c6780 2024-03-06T07:26:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.119+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448404f330 con 0x7f44840c6780 2024-03-06T07:26:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.119+0000 7f447b7fe640 1 -- 172.21.15.26:0/1051795134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f446c01a070 con 0x7f44840c6780 2024-03-06T07:26:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.120+0000 7f447b7fe640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b500 0x7f446406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.120+0000 7f447b7fe640 1 -- 172.21.15.26:0/1051795134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f446c089990 con 0x7f44840c6780 2024-03-06T07:26:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.120+0000 7f4482ffd640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b500 0x7f446406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.120+0000 7f4482ffd640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b500 0x7f446406d9c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f447400a990 tx=0x7f4474008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.124+0000 7f447b7fe640 1 -- 172.21.15.26:0/1051795134 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44841ef450 con 0x7f44840c6780 2024-03-06T07:26:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.411+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44840c5900 con 0x7f44840c6780 2024-03-06T07:26:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.412+0000 7f447b7fe640 1 -- 172.21.15.26:0/1051795134 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4994 (secure 0 0 0) 0x7f4484156050 con 0x7f44840c6780 2024-03-06T07:26:38.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:38.414 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":154,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7454,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1583030325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1583030325},{"type":"v1","addr":"172.21.15.117:6837","nonce":1583030325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":8668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1620185363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1620185363},{"type":"v1","addr":"172.21.15.117:6836","nonce":1620185363}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:37.040749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":158,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7166":{"gid":7166,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7166,"qdb_cluster":[7166]},"id":12}]} 2024-03-06T07:26:38.414 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 154 2024-03-06T07:26:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b500 msgr2=0x7f446406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b500 0x7f446406d9c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f447400a990 tx=0x7f4474008040 comp rx=0 tx=0).stop 2024-03-06T07:26:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c6780 msgr2=0x7f44841eef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c6780 0x7f44841eef10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c470 tx=0x7f446c001e70 comp rx=0 tx=0).stop 2024-03-06T07:26:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 shutdown_connections 2024-03-06T07:26:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f446406b500 0x7f446406d9c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840c7380 0x7f44840a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44840c6780 0x7f44841eef10 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 --2- 172.21.15.26:0/1051795134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448406da40 0x7f448406fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.414+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 >> 172.21.15.26:0/1051795134 conn(0x7f44840aa630 msgr2=0x7f44840c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.415+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 shutdown_connections 2024-03-06T07:26:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.415+0000 7f4489488640 1 -- 172.21.15.26:0/1051795134 wait complete. 2024-03-06T07:26:38.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.526+0000 7f8a0a720640 1 Processor -- start 2024-03-06T07:26:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.526+0000 7f8a0a720640 1 -- start start 2024-03-06T07:26:38.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.527+0000 7f8a0a720640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a040c3150 0x7f8a040c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.527+0000 7f8a0a720640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a04072440 con 0x7f8a0414f650 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.527+0000 7f8a0a720640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a040c2b10 con 0x7f8a04152c20 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.527+0000 7f8a0a720640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a04152140 con 0x7f8a040c3150 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.527+0000 7f8a03fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8a0414f650 0x7f8a04071880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57588/0 (socket says 172.21.15.26:57588) 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.527+0000 7f8a03fff640 1 -- 172.21.15.26:0/926669299 learned_addr learned my addr 172.21.15.26:0/926669299 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a037fe640 1 --2- 172.21.15.26:0/926669299 >> v2:172.21.15.117:3300/0 conn(0x7f8a040c3150 0x7f8a040c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103516981 0 0) 0x7f89f8001110 con 0x7f8a0414f650 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89ec003520 con 0x7f8a0414f650 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 250512110 0 0) 0x7f89ec003520 con 0x7f8a0414f650 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 >> v2:172.21.15.117:3300/0 conn(0x7f8a040c3150 msgr2=0x7f8a040c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 --2- 172.21.15.26:0/926669299 >> v2:172.21.15.117:3300/0 conn(0x7f8a040c3150 0x7f8a040c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 >> v1:172.21.15.117:6789/0 conn(0x7f8a04152c20 legacy=0x7f8a0415c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04152420 con 0x7f8a0414f650 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f89f8001840 con 0x7f8a0414f650 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 >> v1:172.21.15.26:6789/0 conn(0x7f8a0414f650 legacy=0x7f8a04071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f89ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ec004800 0x7f89ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ec0051c0 0x7f89ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a04152140 con 0x7f89ec0039b0 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec00c160 con 0x7f89ec0051c0 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec00e570 con 0x7f89ec004800 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a03fff640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f89ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a037fe640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ec004800 0x7f89ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a08cbd640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ec0051c0 0x7f89ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a03fff640 1 -- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ec004800 msgr2=0x7f89ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:38.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a03fff640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ec004800 0x7f89ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a03fff640 1 -- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ec0051c0 msgr2=0x7f89ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a03fff640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ec0051c0 0x7f89ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a08cbd640 1 -- 172.21.15.26:0/926669299 reap_dead start 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.528+0000 7f8a03fff640 1 -- 172.21.15.26:0/926669299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec003520 con 0x7f89ec0039b0 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.529+0000 7f8a03fff640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f89ec003dd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c4c0 tx=0x7f89ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac389ec2c19eee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.529+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f800c0e0 con 0x7f89ec0039b0 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.529+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8030070 con 0x7f89ec0039b0 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.529+0000 7f8a02ffd640 1 -- 172.21.15.26:0/926669299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80058d0 con 0x7f89ec0039b0 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.530+0000 7f8a0a720640 1 -- 172.21.15.26:0/926669299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 msgr2=0x7f89ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.530+0000 7f8a0a720640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f89ec003dd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c4c0 tx=0x7f89ec005830 comp rx=0 tx=0).stop 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.530+0000 7f8a0a720640 1 -- 172.21.15.26:0/926669299 shutdown_connections 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.530+0000 7f8a0a720640 1 --2- 172.21.15.26:0/926669299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f89ec003dd0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.530+0000 7f8a0a720640 1 -- 172.21.15.26:0/926669299 >> 172.21.15.26:0/926669299 conn(0x7f8a040c4290 msgr2=0x7f8a040c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.530+0000 7f8a0a720640 1 -- 172.21.15.26:0/926669299 shutdown_connections 2024-03-06T07:26:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.530+0000 7f8a0a720640 1 -- 172.21.15.26:0/926669299 wait complete. 2024-03-06T07:26:38.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.531+0000 7f8a0a720640 1 Processor -- start 2024-03-06T07:26:38.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.531+0000 7f8a0a720640 1 -- start start 2024-03-06T07:26:38.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a0a720640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f8a040c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a0a720640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a041db4f0 0x7f8a04148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a0a720640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0414aad0 0x7f8a0414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a0a720640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c40c0 con 0x7f89ec0039b0 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a0a720640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c3f40 con 0x7f8a0414aad0 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a0a720640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c2520 con 0x7f8a041db4f0 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f8a040c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f8a040c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53060/0 (socket says 172.21.15.26:53060) 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 -- 172.21.15.26:0/2914923333 learned_addr learned my addr 172.21.15.26:0/2914923333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a08cbd640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0414aad0 0x7f8a0414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a037fe640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a041db4f0 0x7f8a04148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 -- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a041db4f0 msgr2=0x7f8a04148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a041db4f0 0x7f8a04148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 -- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0414aad0 msgr2=0x7f8a0414af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0414aad0 0x7f8a0414af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.532+0000 7f8a03fff640 1 -- 172.21.15.26:0/2914923333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a041ea9d0 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.533+0000 7f8a03fff640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f8a040c1ab0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f89f802c720 tx=0x7f89f802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.533+0000 7f8a00ff9640 1 -- 172.21.15.26:0/2914923333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8007270 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.533+0000 7f8a00ff9640 1 -- 172.21.15.26:0/2914923333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8007410 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.533+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a04151760 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.533+0000 7f8a00ff9640 1 -- 172.21.15.26:0/2914923333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80077c0 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.533+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a041492c0 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.534+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a04151c80 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.534+0000 7f8a00ff9640 1 -- 172.21.15.26:0/2914923333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89f8035470 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.535+0000 7f8a00ff9640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89f006b6a0 0x7f89f006db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.535+0000 7f8a00ff9640 1 -- 172.21.15.26:0/2914923333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f89f80abc80 con 0x7f89ec0039b0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.535+0000 7f8a037fe640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89f006b6a0 0x7f89f006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.536+0000 7f8a037fe640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89f006b6a0 0x7f89f006db60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8a0414dc10 tx=0x7f89f4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.539+0000 7f8a00ff9640 1 -- 172.21.15.26:0/2914923333 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8a04151c80 con 0x7f89ec0039b0 2024-03-06T07:26:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.758+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a0404f330 con 0x7f89ec0039b0 2024-03-06T07:26:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.759+0000 7f8a00ff9640 1 -- 172.21.15.26:0/2914923333 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4994 (secure 0 0 0) 0x7f8a04072440 con 0x7f89ec0039b0 2024-03-06T07:26:38.760 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:38.761 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":154,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7454,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1583030325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1583030325},{"type":"v1","addr":"172.21.15.117:6837","nonce":1583030325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":8668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1620185363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1620185363},{"type":"v1","addr":"172.21.15.117:6836","nonce":1620185363}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:17.118167+0000","modified":"2024-03-06T07:26:37.040749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":158,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7166":{"gid":7166,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3463955148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3463955148},{"type":"v1","addr":"172.21.15.26:6835","nonce":3463955148}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7166,"qdb_cluster":[7166]},"id":12}]} 2024-03-06T07:26:38.761 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 154 2024-03-06T07:26:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89f006b6a0 msgr2=0x7f89f006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89f006b6a0 0x7f89f006db60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8a0414dc10 tx=0x7f89f4009190 comp rx=0 tx=0).stop 2024-03-06T07:26:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 msgr2=0x7f8a040c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f8a040c1ab0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f89f802c720 tx=0x7f89f802e070 comp rx=0 tx=0).stop 2024-03-06T07:26:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 shutdown_connections 2024-03-06T07:26:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f89f006b6a0 0x7f89f006db60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0414aad0 0x7f8a0414af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a041db4f0 0x7f8a04148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 --2- 172.21.15.26:0/2914923333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89ec0039b0 0x7f8a040c1ab0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 >> 172.21.15.26:0/2914923333 conn(0x7f8a040c4290 msgr2=0x7f8a0414e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 shutdown_connections 2024-03-06T07:26:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.761+0000 7f8a0a720640 1 -- 172.21.15.26:0/2914923333 wait complete. 2024-03-06T07:26:38.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.876+0000 7fecd7e64640 1 Processor -- start 2024-03-06T07:26:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.877+0000 7fecd7e64640 1 -- start start 2024-03-06T07:26:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.877+0000 7fecd7e64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd0152920 0x7fecd0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.877+0000 7fecd7e64640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fecd00c5330 0x7fecd00c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.877+0000 7fecd7e64640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd0156180 con 0x7fecd00c8a00 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.877+0000 7fecd7e64640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd01587b0 con 0x7fecd00c5330 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.877+0000 7fecd7e64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0159630 con 0x7fecd0152920 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd5c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd0152920 0x7fecd0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd53ff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fecd00c8a00 0x7fecd0158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:50378/0 (socket says 172.21.15.26:50378) 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd53ff640 1 -- 172.21.15.26:0/1466877026 learned_addr learned my addr 172.21.15.26:0/1466877026 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd6401640 1 --2- 172.21.15.26:0/1466877026 >> v2:172.21.15.117:3300/0 conn(0x7fecd00c5330 0x7fecd00c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd5c00640 1 -- 172.21.15.26:0/1466877026 >> v2:172.21.15.117:3300/0 conn(0x7fecd00c5330 msgr2=0x7fecd00c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd5c00640 1 --2- 172.21.15.26:0/1466877026 >> v2:172.21.15.117:3300/0 conn(0x7fecd00c5330 0x7fecd00c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd5c00640 1 -- 172.21.15.26:0/1466877026 >> v1:172.21.15.117:6789/0 conn(0x7fecd00c8a00 legacy=0x7fecd0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd4bfe640 1 -- 172.21.15.26:0/1466877026 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446451143 0 0) 0x7fecc4001110 con 0x7fecd00c8a00 2024-03-06T07:26:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.878+0000 7fecd5c00640 1 -- 172.21.15.26:0/1466877026 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd00c7760 con 0x7fecd0152920 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.879+0000 7fecd5c00640 1 --2- 172.21.15.26:0/1466877026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd0152920 0x7fecd0155c40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009d90 tx=0x7fecc802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7183be813a93129e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.879+0000 7fecd4bfe640 1 -- 172.21.15.26:0/1466877026 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8039070 con 0x7fecd0152920 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.879+0000 7fecd4bfe640 1 -- 172.21.15.26:0/1466877026 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc802cbc0 con 0x7fecd0152920 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.879+0000 7fecd4bfe640 1 -- 172.21.15.26:0/1466877026 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8034440 con 0x7fecd0152920 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 -- 172.21.15.26:0/1466877026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd0152920 msgr2=0x7fecd0155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 --2- 172.21.15.26:0/1466877026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd0152920 0x7fecd0155c40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009d90 tx=0x7fecc802be10 comp rx=0 tx=0).stop 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 -- 172.21.15.26:0/1466877026 shutdown_connections 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 --2- 172.21.15.26:0/1466877026 >> v2:172.21.15.117:3300/0 conn(0x7fecd00c5330 0x7fecd00c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 --2- 172.21.15.26:0/1466877026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd0152920 0x7fecd0155c40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 -- 172.21.15.26:0/1466877026 >> 172.21.15.26:0/1466877026 conn(0x7fecd00ac560 msgr2=0x7fecd00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 -- 172.21.15.26:0/1466877026 shutdown_connections 2024-03-06T07:26:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.880+0000 7fecd7e64640 1 -- 172.21.15.26:0/1466877026 wait complete. 2024-03-06T07:26:38.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.882+0000 7fecd7e64640 1 Processor -- start 2024-03-06T07:26:38.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.882+0000 7fecd7e64640 1 -- start start 2024-03-06T07:26:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd7e64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd00c5330 0x7fecd0154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd7e64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd00c8a00 0x7fecd014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd7e64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecd0152920 0x7fecd014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd7e64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0158a00 con 0x7fecd0152920 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd7e64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd00c7d20 con 0x7fecd00c8a00 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd7e64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd006d030 con 0x7fecd00c5330 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecd0152920 0x7fecd014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecd0152920 0x7fecd014e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53074/0 (socket says 172.21.15.26:53074) 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 -- 172.21.15.26:0/2733204710 learned_addr learned my addr 172.21.15.26:0/2733204710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd53ff640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd00c8a00 0x7fecd014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd5c00640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd00c5330 0x7fecd0154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 -- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd00c5330 msgr2=0x7fecd0154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd00c5330 0x7fecd0154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 -- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd00c8a00 msgr2=0x7fecd014e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd00c8a00 0x7fecd014e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 -- 172.21.15.26:0/2733204710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd01530d0 con 0x7fecd0152920 2024-03-06T07:26:38.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.883+0000 7fecd6401640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecd0152920 0x7fecd014e5f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e7f0 tx=0x7fecc000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.884+0000 7fecce7fc640 1 -- 172.21.15.26:0/2733204710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0017070 con 0x7fecd0152920 2024-03-06T07:26:38.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.884+0000 7fecce7fc640 1 -- 172.21.15.26:0/2733204710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc000ee40 con 0x7fecd0152920 2024-03-06T07:26:38.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.884+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd014aca0 con 0x7fecd0152920 2024-03-06T07:26:38.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.884+0000 7fecce7fc640 1 -- 172.21.15.26:0/2733204710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000cb80 con 0x7fecd0152920 2024-03-06T07:26:38.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.884+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd0154640 con 0x7fecd0152920 2024-03-06T07:26:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.885+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd004f330 con 0x7fecd0152920 2024-03-06T07:26:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.885+0000 7fecce7fc640 1 -- 172.21.15.26:0/2733204710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecc001a070 con 0x7fecd0152920 2024-03-06T07:26:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.886+0000 7fecce7fc640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecbc06b4d0 0x7fecbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.886+0000 7fecce7fc640 1 -- 172.21.15.26:0/2733204710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fecc008a290 con 0x7fecd0152920 2024-03-06T07:26:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.886+0000 7fecd5c00640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecbc06b4d0 0x7fecbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.887+0000 7fecd5c00640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecbc06b4d0 0x7fecbc06d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fecc8036000 tx=0x7fecc8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:38.890+0000 7fecce7fc640 1 -- 172.21.15.26:0/2733204710 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fecc0058820 con 0x7fecd0152920 2024-03-06T07:26:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.075+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecd006cda0 con 0x7fecd0152920 2024-03-06T07:26:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.076+0000 7fecce7fc640 1 -- 172.21.15.26:0/2733204710 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+17293 (secure 0 0 0) 0x7fecc005c8c0 con 0x7fecd0152920 2024-03-06T07:26:39.078 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:39.078 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":160,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:36.034371+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:15.400219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:16.061363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.117:6834/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.117:6835/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6833/3029467671":"2024-03-07T07:26:23.875878+0000","172.21.15.117:6837/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6832/3029467671":"2024-03-07T07:26:23.875878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecbc06b4d0 msgr2=0x7fecbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecbc06b4d0 0x7fecbc06d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fecc8036000 tx=0x7fecc8030000 comp rx=0 tx=0).stop 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecd0152920 msgr2=0x7fecd014e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecd0152920 0x7fecd014e5f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e7f0 tx=0x7fecc000ebf0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 shutdown_connections 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecbc06b4d0 0x7fecbc06d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecd0152920 0x7fecd014e5f0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd00c8a00 0x7fecd014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 --2- 172.21.15.26:0/2733204710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecd00c5330 0x7fecd0154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 >> 172.21.15.26:0/2733204710 conn(0x7fecd00ac560 msgr2=0x7fecd01508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 shutdown_connections 2024-03-06T07:26:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.078+0000 7fecd7e64640 1 -- 172.21.15.26:0/2733204710 wait complete. 2024-03-06T07:26:39.091 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-06T07:26:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.191+0000 7fcb74eb8640 1 Processor -- start 2024-03-06T07:26:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.191+0000 7fcb74eb8640 1 -- start start 2024-03-06T07:26:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb74eb8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb700c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb74eb8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb701558d0 0x7fcb70155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb74eb8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb70155130 con 0x7fcb700c69f0 2024-03-06T07:26:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb74eb8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb70155440 con 0x7fcb701558d0 2024-03-06T07:26:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb74eb8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700ad480 con 0x7fcb700c8950 2024-03-06T07:26:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb6dd9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb700c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb6ed9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb701558d0 0x7fcb70155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb6dd9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb700c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46450/0 (socket says 172.21.15.26:46450) 2024-03-06T07:26:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb6e59c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fcb700c69f0 0x7fcb700acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:50382/0 (socket says 172.21.15.26:50382) 2024-03-06T07:26:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.192+0000 7fcb6dd9b640 1 -- 172.21.15.26:0/3347184902 learned_addr learned my addr 172.21.15.26:0/3347184902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.193+0000 7fcb6dd9b640 1 -- 172.21.15.26:0/3347184902 >> v2:172.21.15.117:3300/0 conn(0x7fcb701558d0 msgr2=0x7fcb70155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.193+0000 7fcb6dd9b640 1 --2- 172.21.15.26:0/3347184902 >> v2:172.21.15.117:3300/0 conn(0x7fcb701558d0 0x7fcb70155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.193+0000 7fcb6dd9b640 1 -- 172.21.15.26:0/3347184902 >> v1:172.21.15.117:6789/0 conn(0x7fcb700c69f0 legacy=0x7fcb700acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.193+0000 7fcb6dd9b640 1 -- 172.21.15.26:0/3347184902 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700c5d50 con 0x7fcb700c8950 2024-03-06T07:26:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb6dd9b640 1 --2- 172.21.15.26:0/3347184902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb700c6dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb6402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e90a835dbe4a29d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb6d59a640 1 -- 172.21.15.26:0/3347184902 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64035020 con 0x7fcb700c8950 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb6d59a640 1 -- 172.21.15.26:0/3347184902 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb64031640 con 0x7fcb700c8950 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb6d59a640 1 -- 172.21.15.26:0/3347184902 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6402d040 con 0x7fcb700c8950 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3347184902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 msgr2=0x7fcb700c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3347184902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb700c6dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb6402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3347184902 shutdown_connections 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3347184902 >> v2:172.21.15.117:3300/0 conn(0x7fcb701558d0 0x7fcb70155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3347184902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb700c6dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3347184902 >> 172.21.15.26:0/3347184902 conn(0x7fcb700cae70 msgr2=0x7fcb700cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3347184902 shutdown_connections 2024-03-06T07:26:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.194+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3347184902 wait complete. 2024-03-06T07:26:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 Processor -- start 2024-03-06T07:26:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 -- start start 2024-03-06T07:26:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb700c69f0 0x7fcb7014a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb70148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb701558d0 0x7fcb70148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70152a20 con 0x7fcb701558d0 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700c6310 con 0x7fcb700c69f0 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb74eb8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70152ba0 con 0x7fcb700c8950 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb6ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb701558d0 0x7fcb70148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb6ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb701558d0 0x7fcb70148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53088/0 (socket says 172.21.15.26:53088) 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.196+0000 7fcb6ed9d640 1 -- 172.21.15.26:0/3840621580 learned_addr learned my addr 172.21.15.26:0/3840621580 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6dd9b640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb70148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6e59c640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb700c69f0 0x7fcb7014a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6ed9d640 1 -- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 msgr2=0x7fcb70148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6ed9d640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb70148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6ed9d640 1 -- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb700c69f0 msgr2=0x7fcb7014a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6ed9d640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb700c69f0 0x7fcb7014a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6ed9d640 1 -- 172.21.15.26:0/3840621580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700abf00 con 0x7fcb701558d0 2024-03-06T07:26:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb6ed9d640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb701558d0 0x7fcb70148d50 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009dd0 tx=0x7fcb6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb5effd640 1 -- 172.21.15.26:0/3840621580 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60018020 con 0x7fcb701558d0 2024-03-06T07:26:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb5effd640 1 -- 172.21.15.26:0/3840621580 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb60014480 con 0x7fcb701558d0 2024-03-06T07:26:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb700ac1e0 con 0x7fcb701558d0 2024-03-06T07:26:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb5effd640 1 -- 172.21.15.26:0/3840621580 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60014870 con 0x7fcb701558d0 2024-03-06T07:26:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.197+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb701569e0 con 0x7fcb701558d0 2024-03-06T07:26:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.198+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70156520 con 0x7fcb701558d0 2024-03-06T07:26:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.199+0000 7fcb5effd640 1 -- 172.21.15.26:0/3840621580 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb60005d10 con 0x7fcb701558d0 2024-03-06T07:26:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.199+0000 7fcb5effd640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb5006b4d0 0x7fcb5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.199+0000 7fcb5effd640 1 -- 172.21.15.26:0/3840621580 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fcb6008ae60 con 0x7fcb701558d0 2024-03-06T07:26:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.200+0000 7fcb6e59c640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb5006b4d0 0x7fcb5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.200+0000 7fcb6e59c640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb5006b4d0 0x7fcb5006d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcb68007fd0 tx=0x7fcb68007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.203+0000 7fcb5effd640 1 -- 172.21.15.26:0/3840621580 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb70156520 con 0x7fcb701558d0 2024-03-06T07:26:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.437+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7fcb70069460 con 0x7fcb701558d0 2024-03-06T07:26:39.439 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:26:39.438+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:26:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.439+0000 7fcb5effd640 1 -- 172.21.15.26:0/3840621580 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v155) v1 ==== 163+0+0 (secure 0 0 0) 0x7fcb70069460 con 0x7fcb701558d0 2024-03-06T07:26:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:26:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb5006b4d0 msgr2=0x7fcb5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb5006b4d0 0x7fcb5006d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcb68007fd0 tx=0x7fcb68007e80 comp rx=0 tx=0).stop 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb701558d0 msgr2=0x7fcb70148d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb701558d0 0x7fcb70148d50 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009dd0 tx=0x7fcb6000c490 comp rx=0 tx=0).stop 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 shutdown_connections 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcb5006b4d0 0x7fcb5006d990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb701558d0 0x7fcb70148d50 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb700c8950 0x7fcb70148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 --2- 172.21.15.26:0/3840621580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb700c69f0 0x7fcb7014a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 >> 172.21.15.26:0/3840621580 conn(0x7fcb700cae70 msgr2=0x7fcb700cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 shutdown_connections 2024-03-06T07:26:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.442+0000 7fcb74eb8640 1 -- 172.21.15.26:0/3840621580 wait complete. 2024-03-06T07:26:39.445 INFO:tasks.ceph.mds.c.smithi026.stderr: -64> 2024-03-06T07:26:37.983+0000 7f059a823640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:39.455 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-06T07:26:39.467 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:26:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.557+0000 7faf84b5e640 1 Processor -- start 2024-03-06T07:26:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.557+0000 7faf84b5e640 1 -- start start 2024-03-06T07:26:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf84b5e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf800c88f0 0x7faf80155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf84b5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf801596f0 con 0x7faf801527f0 2024-03-06T07:26:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf84b5e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800c82a0 con 0x7faf80155750 2024-03-06T07:26:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf84b5e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800c9130 con 0x7faf800c88f0 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf7e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf801527f0 0x7faf800c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57606/0 (socket says 172.21.15.26:57606) 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf7e59c640 1 -- 172.21.15.26:0/3966733448 learned_addr learned my addr 172.21.15.26:0/3966733448 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf7dd9b640 1 --2- 172.21.15.26:0/3966733448 >> v2:172.21.15.117:3300/0 conn(0x7faf800c88f0 0x7faf80155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf7d59a640 1 -- 172.21.15.26:0/3966733448 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396381327 0 0) 0x7faf801596f0 con 0x7faf801527f0 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.558+0000 7faf7d59a640 1 -- 172.21.15.26:0/3966733448 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf6c003520 con 0x7faf801527f0 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.559+0000 7faf7dd9b640 1 -- 172.21.15.26:0/3966733448 >> v1:172.21.15.117:6789/0 conn(0x7faf80155750 legacy=0x7faf8015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.559+0000 7faf7d59a640 1 -- 172.21.15.26:0/3966733448 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554869 0 0) 0x7faf6c003520 con 0x7faf801527f0 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.559+0000 7faf7dd9b640 1 -- 172.21.15.26:0/3966733448 >> v1:172.21.15.26:6789/0 conn(0x7faf801527f0 legacy=0x7faf800c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.559+0000 7faf7dd9b640 1 -- 172.21.15.26:0/3966733448 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80158930 con 0x7faf800c88f0 2024-03-06T07:26:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.559+0000 7faf7dd9b640 1 --2- 172.21.15.26:0/3966733448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 0x7faf80155210 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faf680097a0 tx=0x7faf6802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0e865d10f4a2411 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf7d59a640 1 -- 172.21.15.26:0/3966733448 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68037070 con 0x7faf800c88f0 2024-03-06T07:26:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf7d59a640 1 -- 172.21.15.26:0/3966733448 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf68030510 con 0x7faf800c88f0 2024-03-06T07:26:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf7d59a640 1 -- 172.21.15.26:0/3966733448 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78001670 con 0x7faf800c88f0 2024-03-06T07:26:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf84b5e640 1 -- 172.21.15.26:0/3966733448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 msgr2=0x7faf80155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf84b5e640 1 --2- 172.21.15.26:0/3966733448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 0x7faf80155210 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faf680097a0 tx=0x7faf6802b710 comp rx=0 tx=0).stop 2024-03-06T07:26:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf84b5e640 1 -- 172.21.15.26:0/3966733448 shutdown_connections 2024-03-06T07:26:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf84b5e640 1 --2- 172.21.15.26:0/3966733448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 0x7faf80155210 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf84b5e640 1 -- 172.21.15.26:0/3966733448 >> 172.21.15.26:0/3966733448 conn(0x7faf800ac580 msgr2=0x7faf800ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf84b5e640 1 -- 172.21.15.26:0/3966733448 shutdown_connections 2024-03-06T07:26:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.560+0000 7faf84b5e640 1 -- 172.21.15.26:0/3966733448 wait complete. 2024-03-06T07:26:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.561+0000 7faf84b5e640 1 Processor -- start 2024-03-06T07:26:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf84b5e640 1 -- start start 2024-03-06T07:26:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf84b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 0x7faf80157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf84b5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf801527f0 0x7faf801533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf84b5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf80155750 0x7faf801538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf84b5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800c6b90 con 0x7faf80155750 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf84b5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf80159260 con 0x7faf800c88f0 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf84b5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf80158ef0 con 0x7faf801527f0 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf7ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf80155750 0x7faf801538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf7ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf80155750 0x7faf801538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53124/0 (socket says 172.21.15.26:53124) 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf7ed9d640 1 -- 172.21.15.26:0/2132478128 learned_addr learned my addr 172.21.15.26:0/2132478128 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf7dd9b640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf801527f0 0x7faf801533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.562+0000 7faf7e59c640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 0x7faf80157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf7ed9d640 1 -- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf801527f0 msgr2=0x7faf801533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf7ed9d640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf801527f0 0x7faf801533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf7ed9d640 1 -- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 msgr2=0x7faf80157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf7ed9d640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 0x7faf80157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf7ed9d640 1 -- 172.21.15.26:0/2132478128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf801ee140 con 0x7faf80155750 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf7ed9d640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf80155750 0x7faf801538e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c7a0 tx=0x7faf7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf76ffd640 1 -- 172.21.15.26:0/2132478128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70003b70 con 0x7faf80155750 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf76ffd640 1 -- 172.21.15.26:0/2132478128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70003d30 con 0x7faf80155750 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf801ee420 con 0x7faf80155750 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf76ffd640 1 -- 172.21.15.26:0/2132478128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7000aa90 con 0x7faf80155750 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.563+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf801ee9e0 con 0x7faf80155750 2024-03-06T07:26:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.564+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8004f330 con 0x7faf80155750 2024-03-06T07:26:39.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.564+0000 7faf76ffd640 1 -- 172.21.15.26:0/2132478128 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf7000ac30 con 0x7faf80155750 2024-03-06T07:26:39.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.565+0000 7faf76ffd640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf6006b4d0 0x7faf6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.565+0000 7faf76ffd640 1 -- 172.21.15.26:0/2132478128 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6449+0+0 (secure 0 0 0) 0x7faf7008a2f0 con 0x7faf80155750 2024-03-06T07:26:39.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.565+0000 7faf7e59c640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf6006b4d0 0x7faf6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.566+0000 7faf7e59c640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf6006b4d0 0x7faf6006d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7faf78005880 tx=0x7faf780057f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.569+0000 7faf76ffd640 1 -- 172.21.15.26:0/2132478128 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf70058830 con 0x7faf80155750 2024-03-06T07:26:39.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:39.790+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faf800c8080 con 0x7faf80155750 2024-03-06T07:26:40.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.440+0000 7faf76ffd640 1 -- 172.21.15.26:0/2132478128 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v156) v1 ==== 105+0+0 (secure 0 0 0) 0x7faf7005c8d0 con 0x7faf80155750 2024-03-06T07:26:40.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf6006b4d0 msgr2=0x7faf6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf6006b4d0 0x7faf6006d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7faf78005880 tx=0x7faf780057f0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf80155750 msgr2=0x7faf801538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf80155750 0x7faf801538e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c7a0 tx=0x7faf7000cc60 comp rx=0 tx=0).stop 2024-03-06T07:26:40.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 shutdown_connections 2024-03-06T07:26:40.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf6006b4d0 0x7faf6006d990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf80155750 0x7faf801538e0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf801527f0 0x7faf801533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 --2- 172.21.15.26:0/2132478128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf800c88f0 0x7faf80157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 >> 172.21.15.26:0/2132478128 conn(0x7faf800ac580 msgr2=0x7faf800c7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:40.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 shutdown_connections 2024-03-06T07:26:40.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.444+0000 7faf84b5e640 1 -- 172.21.15.26:0/2132478128 wait complete. 2024-03-06T07:26:40.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-06T07:26:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.611+0000 7f8a51e4e640 1 Processor -- start 2024-03-06T07:26:40.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.611+0000 7f8a51e4e640 1 -- start start 2024-03-06T07:26:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a51e4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c0c2330 0x7f8a4c0c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a51e4e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a4c1550c0 0x7f8a4c155530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a51e4e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c159260 con 0x7f8a4c14bb20 2024-03-06T07:26:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a51e4e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4c0c3070 con 0x7f8a4c1550c0 2024-03-06T07:26:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a51e4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0a80e0 con 0x7f8a4c0c2330 2024-03-06T07:26:40.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a4affd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c0c2330 0x7f8a4c0c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a4bfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a4c1550c0 0x7f8a4c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a4affd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c0c2330 0x7f8a4c0c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46476/0 (socket says 172.21.15.26:46476) 2024-03-06T07:26:40.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.612+0000 7f8a4affd640 1 -- 172.21.15.26:0/538802284 learned_addr learned my addr 172.21.15.26:0/538802284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:40.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.613+0000 7f8a4affd640 1 -- 172.21.15.26:0/538802284 >> v2:172.21.15.117:3300/0 conn(0x7f8a4c1550c0 msgr2=0x7f8a4c155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:40.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.613+0000 7f8a4affd640 1 --2- 172.21.15.26:0/538802284 >> v2:172.21.15.117:3300/0 conn(0x7f8a4c1550c0 0x7f8a4c155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.613+0000 7f8a4affd640 1 -- 172.21.15.26:0/538802284 >> v1:172.21.15.117:6789/0 conn(0x7f8a4c14bb20 legacy=0x7f8a4c0c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.613+0000 7f8a4affd640 1 -- 172.21.15.26:0/538802284 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0a7d70 con 0x7f8a4c0c2330 2024-03-06T07:26:40.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a4affd640 1 --2- 172.21.15.26:0/538802284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c0c2330 0x7f8a4c0c5a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009870 tx=0x7f8a4002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a558560600ed69 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:40.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a4a7fc640 1 -- 172.21.15.26:0/538802284 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40035020 con 0x7f8a4c0c2330 2024-03-06T07:26:40.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a4a7fc640 1 -- 172.21.15.26:0/538802284 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a40031640 con 0x7f8a4c0c2330 2024-03-06T07:26:40.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a4a7fc640 1 -- 172.21.15.26:0/538802284 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4002d040 con 0x7f8a4c0c2330 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 -- 172.21.15.26:0/538802284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c0c2330 msgr2=0x7f8a4c0c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/538802284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c0c2330 0x7f8a4c0c5a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009870 tx=0x7f8a4002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 -- 172.21.15.26:0/538802284 shutdown_connections 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/538802284 >> v2:172.21.15.117:3300/0 conn(0x7f8a4c1550c0 0x7f8a4c155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/538802284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c0c2330 0x7f8a4c0c5a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 -- 172.21.15.26:0/538802284 >> 172.21.15.26:0/538802284 conn(0x7f8a4c0c88f0 msgr2=0x7f8a4c0c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 -- 172.21.15.26:0/538802284 shutdown_connections 2024-03-06T07:26:40.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.614+0000 7f8a51e4e640 1 -- 172.21.15.26:0/538802284 wait complete. 2024-03-06T07:26:40.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.615+0000 7f8a51e4e640 1 Processor -- start 2024-03-06T07:26:40.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a51e4e640 1 -- start start 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a51e4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c0c2330 0x7f8a4c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a51e4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c14bb20 0x7f8a4c0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a51e4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c1550c0 0x7f8a4c157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a51e4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c14bf00 con 0x7f8a4c1550c0 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a51e4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0c5f60 con 0x7f8a4c0c2330 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a51e4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0c67f0 con 0x7f8a4c14bb20 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.616+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c1550c0 0x7f8a4c157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c1550c0 0x7f8a4c157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53130/0 (socket says 172.21.15.26:53130) 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 -- 172.21.15.26:0/1863173700 learned_addr learned my addr 172.21.15.26:0/1863173700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4affd640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c14bb20 0x7f8a4c0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4b7fe640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c0c2330 0x7f8a4c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 -- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c14bb20 msgr2=0x7f8a4c0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c14bb20 0x7f8a4c0abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 -- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c0c2330 msgr2=0x7f8a4c153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c0c2330 0x7f8a4c153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 -- 172.21.15.26:0/1863173700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c157170 con 0x7f8a4c1550c0 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4b7fe640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c0c2330 0x7f8a4c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a4bfff640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c1550c0 0x7f8a4c157670 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009dd0 tx=0x7f8a4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a33fff640 1 -- 172.21.15.26:0/1863173700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44018020 con 0x7f8a4c1550c0 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a33fff640 1 -- 172.21.15.26:0/1863173700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a44014480 con 0x7f8a4c1550c0 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c1eb7d0 con 0x7f8a4c1550c0 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a33fff640 1 -- 172.21.15.26:0/1863173700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44014870 con 0x7f8a4c1550c0 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.617+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c14c4e0 con 0x7f8a4c1550c0 2024-03-06T07:26:40.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.618+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c04f330 con 0x7f8a4c1550c0 2024-03-06T07:26:40.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.619+0000 7f8a33fff640 1 -- 172.21.15.26:0/1863173700 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a44005d10 con 0x7f8a4c1550c0 2024-03-06T07:26:40.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.619+0000 7f8a33fff640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.619+0000 7f8a33fff640 1 -- 172.21.15.26:0/1863173700 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f8a4408af70 con 0x7f8a4c1550c0 2024-03-06T07:26:40.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.619+0000 7f8a4b7fe640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.620+0000 7f8a4b7fe640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3806b4d0 0x7f8a3806d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c007fd0 tx=0x7f8a3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:40.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.624+0000 7f8a33fff640 1 -- 172.21.15.26:0/1863173700 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8a44022070 con 0x7f8a4c1550c0 2024-03-06T07:26:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:40.804+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8a4c0c6b90 con 0x7f8a4c1550c0 2024-03-06T07:26:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.442+0000 7f8a33fff640 1 -- 172.21.15.26:0/1863173700 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v162) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8a44010040 con 0x7f8a4c1550c0 2024-03-06T07:26:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_metadata' removed 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3806b4d0 msgr2=0x7f8a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3806b4d0 0x7f8a3806d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c007fd0 tx=0x7f8a3c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c1550c0 msgr2=0x7f8a4c157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c1550c0 0x7f8a4c157670 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009dd0 tx=0x7f8a4400c490 comp rx=0 tx=0).stop 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 shutdown_connections 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a4c1550c0 0x7f8a4c157670 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4c14bb20 0x7f8a4c0abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 --2- 172.21.15.26:0/1863173700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c0c2330 0x7f8a4c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 >> 172.21.15.26:0/1863173700 conn(0x7f8a4c0c88f0 msgr2=0x7f8a4c0a9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 shutdown_connections 2024-03-06T07:26:41.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.445+0000 7f8a51e4e640 1 -- 172.21.15.26:0/1863173700 wait complete. 2024-03-06T07:26:41.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-06T07:26:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.570+0000 7fd7952c0640 1 Processor -- start 2024-03-06T07:26:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.570+0000 7fd7952c0640 1 -- start start 2024-03-06T07:26:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd7952c0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 0x7fd7900c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd7952c0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7900c5400 con 0x7fd790152870 2024-03-06T07:26:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd7952c0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7900c56e0 con 0x7fd7900c7930 2024-03-06T07:26:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd7952c0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79006d320 con 0x7fd7900c88e0 2024-03-06T07:26:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd790152870 0x7fd790155bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57614/0 (socket says 172.21.15.26:57614) 2024-03-06T07:26:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78effd640 1 -- 172.21.15.26:0/46748751 learned_addr learned my addr 172.21.15.26:0/46748751 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032070906 0 0) 0x7fd7900c5400 con 0x7fd790152870 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd77c003520 con 0x7fd790152870 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd787fff640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 0x7fd7900c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 168194401 0 0) 0x7fd77c003520 con 0x7fd790152870 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 msgr2=0x7fd7900c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 0x7fd7900c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 >> v1:172.21.15.117:6789/0 conn(0x7fd7900c7930 legacy=0x7fd79006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7900c59c0 con 0x7fd790152870 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd780001670 con 0x7fd790152870 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.571+0000 7fd787fff640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 0x7fd7900c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 >> v1:172.21.15.26:6789/0 conn(0x7fd790152870 legacy=0x7fd790155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78e7fc640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 0x7fd77c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78e7fc640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c003fd0 0x7fd77c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78e7fc640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd77c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79006d320 con 0x7fd77c004d40 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c00bbb0 con 0x7fd77c003fd0 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c00bdb0 con 0x7fd7900c88e0 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78f7fe640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd77c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78effd640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 0x7fd77c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd787fff640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c003fd0 0x7fd77c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78f7fe640 1 -- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 msgr2=0x7fd77c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78f7fe640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7900c88e0 0x7fd77c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78f7fe640 1 -- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c003fd0 msgr2=0x7fd77c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78f7fe640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c003fd0 0x7fd77c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd78f7fe640 1 -- 172.21.15.26:0/46748751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c003520 con 0x7fd77c004d40 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.572+0000 7fd787fff640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c003fd0 0x7fd77c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd78f7fe640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd77c0051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd7880044b0 tx=0x7fd77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68c8a24af53a623 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788034070 con 0x7fd77c004d40 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788030620 con 0x7fd77c004d40 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd78e7fc640 1 -- 172.21.15.26:0/46748751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788004080 con 0x7fd77c004d40 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd7952c0640 1 -- 172.21.15.26:0/46748751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 msgr2=0x7fd77c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd7952c0640 1 --2- 172.21.15.26:0/46748751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd77c0051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd7880044b0 tx=0x7fd77c005830 comp rx=0 tx=0).stop 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd78f7fe640 1 -- 172.21.15.26:0/46748751 reap_dead start 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd7952c0640 1 -- 172.21.15.26:0/46748751 shutdown_connections 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd7952c0640 1 -- 172.21.15.26:0/46748751 >> 172.21.15.26:0/46748751 conn(0x7fd7900ac5f0 msgr2=0x7fd7900aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd7952c0640 1 -- 172.21.15.26:0/46748751 shutdown_connections 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.573+0000 7fd7952c0640 1 -- 172.21.15.26:0/46748751 wait complete. 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 Processor -- start 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 -- start start 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd7901ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7901ee590 0x7fd79014ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd79014f380 0x7fd790147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900c9470 con 0x7fd77c004d40 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79006b8d0 con 0x7fd7901ee590 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd7952c0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900c95f0 con 0x7fd79014f380 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd78effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd7901ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd78effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd7901ee050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53144/0 (socket says 172.21.15.26:53144) 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd78effd640 1 -- 172.21.15.26:0/274521955 learned_addr learned my addr 172.21.15.26:0/274521955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.575+0000 7fd787fff640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7901ee590 0x7fd79014ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78f7fe640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd79014f380 0x7fd790147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78effd640 1 -- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd79014f380 msgr2=0x7fd790147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78effd640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd79014f380 0x7fd790147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78effd640 1 -- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7901ee590 msgr2=0x7fd79014ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78effd640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7901ee590 0x7fd79014ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78effd640 1 -- 172.21.15.26:0/274521955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd790152c50 con 0x7fd77c004d40 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78f7fe640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd79014f380 0x7fd790147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd787fff640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7901ee590 0x7fd79014ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd78effd640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd7901ee050 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd780007310 tx=0x7fd7800077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd7877fe640 1 -- 172.21.15.26:0/274521955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000f9a0 con 0x7fd77c004d40 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd7877fe640 1 -- 172.21.15.26:0/274521955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd780018070 con 0x7fd77c004d40 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd790152f30 con 0x7fd77c004d40 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd7877fe640 1 -- 172.21.15.26:0/274521955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000fda0 con 0x7fd77c004d40 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.576+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79014d800 con 0x7fd77c004d40 2024-03-06T07:26:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.577+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79004f330 con 0x7fd77c004d40 2024-03-06T07:26:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.578+0000 7fd7877fe640 1 -- 172.21.15.26:0/274521955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd78001d460 con 0x7fd77c004d40 2024-03-06T07:26:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.578+0000 7fd7877fe640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd77006b4d0 0x7fd77006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.578+0000 7fd7877fe640 1 -- 172.21.15.26:0/274521955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6009+0+0 (secure 0 0 0) 0x7fd78008a470 con 0x7fd77c004d40 2024-03-06T07:26:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.578+0000 7fd787fff640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd77006b4d0 0x7fd77006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.579+0000 7fd787fff640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd77006b4d0 0x7fd77006d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd7900b0a30 tx=0x7fd778008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.582+0000 7fd7877fe640 1 -- 172.21.15.26:0/274521955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd780059250 con 0x7fd77c004d40 2024-03-06T07:26:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:41.765+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7900b0bd0 con 0x7fd77c004d40 2024-03-06T07:26:42.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.447+0000 7fd7877fe640 1 -- 172.21.15.26:0/274521955 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v163) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd780058b50 con 0x7fd77c004d40 2024-03-06T07:26:42.449 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs2_data' removed 2024-03-06T07:26:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd77006b4d0 msgr2=0x7fd77006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd77006b4d0 0x7fd77006d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd7900b0a30 tx=0x7fd778008040 comp rx=0 tx=0).stop 2024-03-06T07:26:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 msgr2=0x7fd7901ee050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd7901ee050 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd780007310 tx=0x7fd7800077d0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 shutdown_connections 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd77006b4d0 0x7fd77006d990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd79014f380 0x7fd790147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7901ee590 0x7fd79014ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 --2- 172.21.15.26:0/274521955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77c004d40 0x7fd7901ee050 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.449+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 >> 172.21.15.26:0/274521955 conn(0x7fd7900ac5f0 msgr2=0x7fd7900aa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.450+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 shutdown_connections 2024-03-06T07:26:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.450+0000 7fd7952c0640 1 -- 172.21.15.26:0/274521955 wait complete. 2024-03-06T07:26:42.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.c mds_join_fs 2024-03-06T07:26:42.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:42.513 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:42.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.573+0000 7f24e77e9640 1 Processor -- start 2024-03-06T07:26:42.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.573+0000 7f24e77e9640 1 -- start start 2024-03-06T07:26:42.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.574+0000 7f24e77e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24e00b0070 0x7f24e0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.574+0000 7f24e77e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e00b0450 con 0x7f24e006ca70 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.574+0000 7f24e77e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e006cf20 con 0x7f24e00c5380 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.574+0000 7f24e77e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e006c020 con 0x7f24e00b0070 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.574+0000 7f24e4d84640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f24e006ca70 0x7f24e00c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57622/0 (socket says 172.21.15.26:57622) 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.574+0000 7f24e4d84640 1 -- 172.21.15.26:0/948239023 learned_addr learned my addr 172.21.15.26:0/948239023 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643255471 0 0) 0x7f24d4001110 con 0x7f24e006ca70 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5585640 1 --2- 172.21.15.26:0/948239023 >> v2:172.21.15.117:3300/0 conn(0x7f24e00b0070 0x7f24e0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c8003520 con 0x7f24e006ca70 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3762992184 0 0) 0x7f24c8003520 con 0x7f24e006ca70 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 >> v2:172.21.15.117:3300/0 conn(0x7f24e00b0070 msgr2=0x7f24e0155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 --2- 172.21.15.26:0/948239023 >> v2:172.21.15.117:3300/0 conn(0x7f24e00b0070 0x7f24e0155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 >> v1:172.21.15.117:6789/0 conn(0x7f24e00c5380 legacy=0x7f24e014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00c77f0 con 0x7f24e006ca70 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f24d4001840 con 0x7f24e006ca70 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 >> v1:172.21.15.26:6789/0 conn(0x7f24e006ca70 legacy=0x7f24e00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c80039b0 0x7f24c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c8004800 0x7f24c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c80051c0 0x7f24c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e006c020 con 0x7f24c80051c0 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c800c160 con 0x7f24c80039b0 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c800c360 con 0x7f24c8004800 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5d86640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c80051c0 0x7f24c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5585640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c80039b0 0x7f24c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e4d84640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c8004800 0x7f24c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5d86640 1 -- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c8004800 msgr2=0x7f24c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5d86640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c8004800 0x7f24c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5d86640 1 -- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c80039b0 msgr2=0x7f24c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5d86640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c80039b0 0x7f24c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.575+0000 7f24e5d86640 1 -- 172.21.15.26:0/948239023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8003520 con 0x7f24c80051c0 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.576+0000 7f24e5d86640 1 -- 172.21.15.26:0/948239023 reap_dead start 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.576+0000 7f24e5d86640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c80051c0 0x7f24c800b9c0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f24d0001e50 tx=0x7f24c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a68b01534646b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.576+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0001a70 con 0x7f24c80051c0 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.576+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d0003b90 con 0x7f24c80051c0 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.576+0000 7f24dffff640 1 -- 172.21.15.26:0/948239023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d00050d0 con 0x7f24c80051c0 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.577+0000 7f24e77e9640 1 -- 172.21.15.26:0/948239023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c80051c0 msgr2=0x7f24c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.577+0000 7f24e77e9640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c80051c0 0x7f24c800b9c0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f24d0001e50 tx=0x7f24c8005830 comp rx=0 tx=0).stop 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.577+0000 7f24e77e9640 1 -- 172.21.15.26:0/948239023 shutdown_connections 2024-03-06T07:26:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.577+0000 7f24e77e9640 1 --2- 172.21.15.26:0/948239023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c80051c0 0x7f24c800b9c0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.577+0000 7f24e77e9640 1 -- 172.21.15.26:0/948239023 >> 172.21.15.26:0/948239023 conn(0x7f24e00ac590 msgr2=0x7f24e00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.577+0000 7f24e77e9640 1 -- 172.21.15.26:0/948239023 shutdown_connections 2024-03-06T07:26:42.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.577+0000 7f24e77e9640 1 -- 172.21.15.26:0/948239023 wait complete. 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.578+0000 7f24e77e9640 1 Processor -- start 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.578+0000 7f24e77e9640 1 -- start start 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e77e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c80051c0 0x7f24e01ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e77e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24e0151be0 0x7f24e00ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e77e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0159140 0x7f24e0150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e77e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e006c550 con 0x7f24e0159140 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e77e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e006c270 con 0x7f24e0151be0 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e77e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e006da80 con 0x7f24c80051c0 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0159140 0x7f24e0150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0159140 0x7f24e0150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53158/0 (socket says 172.21.15.26:53158) 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 -- 172.21.15.26:0/386890610 learned_addr learned my addr 172.21.15.26:0/386890610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e4d84640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24e0151be0 0x7f24e00ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5585640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c80051c0 0x7f24e01ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 -- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c80051c0 msgr2=0x7f24e01ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c80051c0 0x7f24e01ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 -- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24e0151be0 msgr2=0x7f24e00ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24e0151be0 0x7f24e00ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 -- 172.21.15.26:0/386890610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0151940 con 0x7f24e0159140 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e4d84640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24e0151be0 0x7f24e00ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5585640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c80051c0 0x7f24e01ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24e5d86640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0159140 0x7f24e0150be0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f24d0001f40 tx=0x7f24d00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24ddffb640 1 -- 172.21.15.26:0/386890610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0007db0 con 0x7f24e0159140 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.579+0000 7f24ddffb640 1 -- 172.21.15.26:0/386890610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d0006d50 con 0x7f24e0159140 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.580+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e00c1440 con 0x7f24e0159140 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.580+0000 7f24ddffb640 1 -- 172.21.15.26:0/386890610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0039680 con 0x7f24e0159140 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.580+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00aee80 con 0x7f24e0159140 2024-03-06T07:26:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.581+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e004f330 con 0x7f24e0159140 2024-03-06T07:26:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.581+0000 7f24ddffb640 1 -- 172.21.15.26:0/386890610 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24d0039820 con 0x7f24e0159140 2024-03-06T07:26:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.582+0000 7f24ddffb640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f24cc06b4d0 0x7f24cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.582+0000 7f24ddffb640 1 -- 172.21.15.26:0/386890610 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f24d00ac1c0 con 0x7f24e0159140 2024-03-06T07:26:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.582+0000 7f24e5585640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f24cc06b4d0 0x7f24cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.582+0000 7f24e5585640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f24cc06b4d0 0x7f24cc06d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f24d80093d0 tx=0x7f24d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.586+0000 7f24ddffb640 1 -- 172.21.15.26:0/386890610 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f24d0033030 con 0x7f24e0159140 2024-03-06T07:26:42.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.768+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"} v 0) v1 -- 0x7f24e006d860 con 0x7f24e0159140 2024-03-06T07:26:42.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.771+0000 7f24ddffb640 1 -- 172.21.15.26:0/386890610 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"}]=0 v54) v1 ==== 96+0+0 (secure 0 0 0) 0x7f24d007b0b0 con 0x7f24e0159140 2024-03-06T07:26:42.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f24cc06b4d0 msgr2=0x7f24cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f24cc06b4d0 0x7f24cc06d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f24d80093d0 tx=0x7f24d8008040 comp rx=0 tx=0).stop 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0159140 msgr2=0x7f24e0150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0159140 0x7f24e0150be0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f24d0001f40 tx=0x7f24d00014a0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 shutdown_connections 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f24cc06b4d0 0x7f24cc06d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24e0159140 0x7f24e0150be0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24e0151be0 0x7f24e00ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 --2- 172.21.15.26:0/386890610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c80051c0 0x7f24e01ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 >> 172.21.15.26:0/386890610 conn(0x7f24e00ac590 msgr2=0x7f24e00abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 shutdown_connections 2024-03-06T07:26:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.774+0000 7f24e77e9640 1 -- 172.21.15.26:0/386890610 wait complete. 2024-03-06T07:26:42.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:26:42.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.932+0000 7f3a2eceb640 1 Processor -- start 2024-03-06T07:26:42.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.932+0000 7f3a2eceb640 1 -- start start 2024-03-06T07:26:42.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a2eceb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a281527f0 0x7f3a280c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a2eceb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a280c58b0 0x7f3a28155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a2eceb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a281596f0 con 0x7f3a280c88f0 2024-03-06T07:26:42.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a2eceb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a280c9130 con 0x7f3a280c58b0 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a2eceb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a28159260 con 0x7f3a281527f0 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a2d288640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a280c58b0 0x7f3a28155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a27fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3a280c88f0 0x7f3a28155210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57632/0 (socket says 172.21.15.26:57632) 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.933+0000 7f3a27fff640 1 -- 172.21.15.26:0/3155916614 learned_addr learned my addr 172.21.15.26:0/3155916614 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a2ca87640 1 --2- 172.21.15.26:0/3155916614 >> v2:172.21.15.117:3300/0 conn(0x7f3a281527f0 0x7f3a280c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a2d288640 1 -- 172.21.15.26:0/3155916614 >> v2:172.21.15.117:3300/0 conn(0x7f3a281527f0 msgr2=0x7f3a280c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a2d288640 1 --2- 172.21.15.26:0/3155916614 >> v2:172.21.15.117:3300/0 conn(0x7f3a281527f0 0x7f3a280c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a2d288640 1 -- 172.21.15.26:0/3155916614 >> v1:172.21.15.26:6789/0 conn(0x7f3a280c88f0 legacy=0x7f3a28155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a2d288640 1 -- 172.21.15.26:0/3155916614 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a280c82a0 con 0x7f3a280c58b0 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a277fe640 1 -- 172.21.15.26:0/3155916614 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765883553 0 0) 0x7f3a281596f0 con 0x7f3a280c88f0 2024-03-06T07:26:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a2ca87640 1 --2- 172.21.15.26:0/3155916614 >> v2:172.21.15.117:3300/0 conn(0x7f3a281527f0 0x7f3a280c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.934+0000 7f3a2d288640 1 --2- 172.21.15.26:0/3155916614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a280c58b0 0x7f3a28155920 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f3a28073690 tx=0x7f3a1802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99d9ac5cdc5e16a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a277fe640 1 -- 172.21.15.26:0/3155916614 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18038070 con 0x7f3a280c58b0 2024-03-06T07:26:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a277fe640 1 -- 172.21.15.26:0/3155916614 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18031710 con 0x7f3a280c58b0 2024-03-06T07:26:42.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a277fe640 1 -- 172.21.15.26:0/3155916614 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18031ae0 con 0x7f3a280c58b0 2024-03-06T07:26:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3155916614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a280c58b0 msgr2=0x7f3a28155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3155916614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a280c58b0 0x7f3a28155920 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f3a28073690 tx=0x7f3a1802b7b0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3155916614 shutdown_connections 2024-03-06T07:26:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3155916614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a280c58b0 0x7f3a28155920 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3155916614 >> v2:172.21.15.117:3300/0 conn(0x7f3a281527f0 0x7f3a280c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3155916614 >> 172.21.15.26:0/3155916614 conn(0x7f3a280ac580 msgr2=0x7f3a280ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3155916614 shutdown_connections 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.935+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3155916614 wait complete. 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.936+0000 7f3a2eceb640 1 Processor -- start 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.936+0000 7f3a2eceb640 1 -- start start 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2eceb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280c58b0 0x7f3a2814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2eceb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a280c88f0 0x7f3a28157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2eceb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a281527f0 0x7f3a2814bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2eceb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28158c10 con 0x7f3a281527f0 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2eceb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280c6b90 con 0x7f3a280c58b0 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2eceb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28158ef0 con 0x7f3a280c88f0 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a281527f0 0x7f3a2814bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a281527f0 0x7f3a2814bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53174/0 (socket says 172.21.15.26:53174) 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 -- 172.21.15.26:0/3453257820 learned_addr learned my addr 172.21.15.26:0/3453257820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2ca87640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280c58b0 0x7f3a2814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a27fff640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a280c88f0 0x7f3a28157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 -- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a280c88f0 msgr2=0x7f3a28157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a280c88f0 0x7f3a28157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 -- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280c58b0 msgr2=0x7f3a2814d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280c58b0 0x7f3a2814d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2d288640 1 -- 172.21.15.26:0/3453257820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2814d9e0 con 0x7f3a281527f0 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.937+0000 7f3a2ca87640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280c58b0 0x7f3a2814d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.938+0000 7f3a2d288640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a281527f0 0x7f3a2814bb80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009890 tx=0x7f3a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.938+0000 7f3a257fa640 1 -- 172.21.15.26:0/3453257820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18038040 con 0x7f3a281527f0 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.938+0000 7f3a257fa640 1 -- 172.21.15.26:0/3453257820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18033a40 con 0x7f3a281527f0 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.938+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2814dfc0 con 0x7f3a281527f0 2024-03-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.938+0000 7f3a257fa640 1 -- 172.21.15.26:0/3453257820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18006e10 con 0x7f3a281527f0 2024-03-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.938+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a281ebe10 con 0x7f3a281527f0 2024-03-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.939+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2814e4e0 con 0x7f3a281527f0 2024-03-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.939+0000 7f3a257fa640 1 -- 172.21.15.26:0/3453257820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a1803b070 con 0x7f3a281527f0 2024-03-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.940+0000 7f3a257fa640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a1406b500 0x7f3a1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.940+0000 7f3a257fa640 1 -- 172.21.15.26:0/3453257820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f3a180abe60 con 0x7f3a281527f0 2024-03-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.940+0000 7f3a2ca87640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a1406b500 0x7f3a1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.940+0000 7f3a2ca87640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a1406b500 0x7f3a1406d9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000e3d0 tx=0x7f3a20007a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:42.944+0000 7f3a257fa640 1 -- 172.21.15.26:0/3453257820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a2814e4e0 con 0x7f3a281527f0 2024-03-06T07:26:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.125+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3a2804f330 con 0x7f3a281527f0 2024-03-06T07:26:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.129+0000 7f3a257fa640 1 -- 172.21.15.26:0/3453257820 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3a18042020 con 0x7f3a281527f0 2024-03-06T07:26:43.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a1406b500 msgr2=0x7f3a1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a1406b500 0x7f3a1406d9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000e3d0 tx=0x7f3a20007a10 comp rx=0 tx=0).stop 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a281527f0 msgr2=0x7f3a2814bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a281527f0 0x7f3a2814bb80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009890 tx=0x7f3a180046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 shutdown_connections 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a1406b500 0x7f3a1406d9c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a281527f0 0x7f3a2814bb80 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a280c88f0 0x7f3a28157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 --2- 172.21.15.26:0/3453257820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280c58b0 0x7f3a2814d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 >> 172.21.15.26:0/3453257820 conn(0x7f3a280ac580 msgr2=0x7f3a280ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 shutdown_connections 2024-03-06T07:26:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.131+0000 7f3a2eceb640 1 -- 172.21.15.26:0/3453257820 wait complete. 2024-03-06T07:26:43.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:26:43.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.245+0000 7f2a663e2640 1 Processor -- start 2024-03-06T07:26:43.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.245+0000 7f2a663e2640 1 -- start start 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a663e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600c6da0 0x7f2a6006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a663e2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600c7430 con 0x7f2a6006afc0 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a663e2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a60157450 con 0x7f2a601536a0 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a663e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60156fc0 con 0x7f2a600c6da0 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a5ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a6006afc0 0x7f2a60152f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57634/0 (socket says 172.21.15.26:57634) 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a5ffff640 1 -- 172.21.15.26:0/1578610731 learned_addr learned my addr 172.21.15.26:0/1578610731 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a6497f640 1 --2- 172.21.15.26:0/1578610731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600c6da0 0x7f2a6006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a5effd640 1 -- 172.21.15.26:0/1578610731 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973236027 0 0) 0x7f2a600c7430 con 0x7f2a6006afc0 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a5effd640 1 -- 172.21.15.26:0/1578610731 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a48003520 con 0x7f2a6006afc0 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a5effd640 1 -- 172.21.15.26:0/1578610731 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540939091 0 0) 0x7f2a60157450 con 0x7f2a601536a0 2024-03-06T07:26:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a5effd640 1 -- 172.21.15.26:0/1578610731 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a600c7430 con 0x7f2a601536a0 2024-03-06T07:26:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a6497f640 1 -- 172.21.15.26:0/1578610731 >> v1:172.21.15.117:6789/0 conn(0x7f2a601536a0 legacy=0x7f2a6015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a6497f640 1 -- 172.21.15.26:0/1578610731 >> v1:172.21.15.26:6789/0 conn(0x7f2a6006afc0 legacy=0x7f2a60152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.246+0000 7f2a6497f640 1 -- 172.21.15.26:0/1578610731 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60156690 con 0x7f2a600c6da0 2024-03-06T07:26:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.247+0000 7f2a6497f640 1 --2- 172.21.15.26:0/1578610731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600c6da0 0x7f2a6006b3a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009dc0 tx=0x7f2a5802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ea8a5f529d4fbc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.247+0000 7f2a5effd640 1 -- 172.21.15.26:0/1578610731 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58039070 con 0x7f2a600c6da0 2024-03-06T07:26:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.247+0000 7f2a5effd640 1 -- 172.21.15.26:0/1578610731 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5802cc40 con 0x7f2a600c6da0 2024-03-06T07:26:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.247+0000 7f2a5effd640 1 -- 172.21.15.26:0/1578610731 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58034440 con 0x7f2a600c6da0 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.248+0000 7f2a663e2640 1 -- 172.21.15.26:0/1578610731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600c6da0 msgr2=0x7f2a6006b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.248+0000 7f2a663e2640 1 --2- 172.21.15.26:0/1578610731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600c6da0 0x7f2a6006b3a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009dc0 tx=0x7f2a5802be90 comp rx=0 tx=0).stop 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.248+0000 7f2a663e2640 1 -- 172.21.15.26:0/1578610731 shutdown_connections 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.248+0000 7f2a663e2640 1 --2- 172.21.15.26:0/1578610731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600c6da0 0x7f2a6006b3a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.248+0000 7f2a663e2640 1 -- 172.21.15.26:0/1578610731 >> 172.21.15.26:0/1578610731 conn(0x7f2a600c4220 msgr2=0x7f2a600c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.248+0000 7f2a663e2640 1 -- 172.21.15.26:0/1578610731 shutdown_connections 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.248+0000 7f2a663e2640 1 -- 172.21.15.26:0/1578610731 wait complete. 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.249+0000 7f2a663e2640 1 Processor -- start 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.249+0000 7f2a663e2640 1 -- start start 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a663e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6006afc0 0x7f2a600c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a663e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600c6da0 0x7f2a601ec610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a663e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a601536a0 0x7f2a601ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a663e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60150840 con 0x7f2a6006afc0 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a663e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60156c50 con 0x7f2a600c6da0 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a663e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a601509c0 con 0x7f2a601536a0 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6006afc0 0x7f2a600c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6006afc0 0x7f2a600c0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53178/0 (socket says 172.21.15.26:53178) 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 -- 172.21.15.26:0/943426982 learned_addr learned my addr 172.21.15.26:0/943426982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5f7fe640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600c6da0 0x7f2a601ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a6497f640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a601536a0 0x7f2a601ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 -- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a601536a0 msgr2=0x7f2a601ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a601536a0 0x7f2a601ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 -- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600c6da0 msgr2=0x7f2a601ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600c6da0 0x7f2a601ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.250+0000 7f2a5ffff640 1 -- 172.21.15.26:0/943426982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60153280 con 0x7f2a6006afc0 2024-03-06T07:26:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.251+0000 7f2a5ffff640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6006afc0 0x7f2a600c0dc0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c500 tx=0x7f2a5400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.251+0000 7f2a5cff9640 1 -- 172.21.15.26:0/943426982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54017070 con 0x7f2a6006afc0 2024-03-06T07:26:43.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.251+0000 7f2a5cff9640 1 -- 172.21.15.26:0/943426982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a54005ce0 con 0x7f2a6006afc0 2024-03-06T07:26:43.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.251+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a601ee710 con 0x7f2a6006afc0 2024-03-06T07:26:43.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.251+0000 7f2a5cff9640 1 -- 172.21.15.26:0/943426982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540072f0 con 0x7f2a6006afc0 2024-03-06T07:26:43.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.251+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a601eecd0 con 0x7f2a6006afc0 2024-03-06T07:26:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.252+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6004f330 con 0x7f2a6006afc0 2024-03-06T07:26:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.252+0000 7f2a5cff9640 1 -- 172.21.15.26:0/943426982 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a5401a070 con 0x7f2a6006afc0 2024-03-06T07:26:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.253+0000 7f2a5cff9640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.253+0000 7f2a5cff9640 1 -- 172.21.15.26:0/943426982 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f2a540894e0 con 0x7f2a6006afc0 2024-03-06T07:26:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.253+0000 7f2a5f7fe640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.253+0000 7f2a5f7fe640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2a50007fd0 tx=0x7f2a50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.257+0000 7f2a5cff9640 1 -- 172.21.15.26:0/943426982 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2a54057d90 con 0x7f2a6006afc0 2024-03-06T07:26:43.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.438+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7f2a6014efb0 con 0x7f2a6006afc0 2024-03-06T07:26:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.441+0000 7f2a5cff9640 1 -- 172.21.15.26:0/943426982 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v56) v1 ==== 96+0+0 (secure 0 0 0) 0x7f2a5405be30 con 0x7f2a6006afc0 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a4c06b4d0 msgr2=0x7f2a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2a50007fd0 tx=0x7f2a50009040 comp rx=0 tx=0).stop 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6006afc0 msgr2=0x7f2a600c0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6006afc0 0x7f2a600c0dc0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c500 tx=0x7f2a5400c9c0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 shutdown_connections 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a601536a0 0x7f2a601ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600c6da0 0x7f2a601ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 --2- 172.21.15.26:0/943426982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6006afc0 0x7f2a600c0dc0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 >> 172.21.15.26:0/943426982 conn(0x7f2a600c4220 msgr2=0x7f2a6006b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 shutdown_connections 2024-03-06T07:26:43.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.443+0000 7f2a663e2640 1 -- 172.21.15.26:0/943426982 wait complete. 2024-03-06T07:26:43.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla' 2024-03-06T07:26:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.556+0000 7fc309a0e640 1 Processor -- start 2024-03-06T07:26:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.556+0000 7fc309a0e640 1 -- start start 2024-03-06T07:26:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc309a0e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 0x7fc3040c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc309a0e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc304155110 con 0x7fc3040c5330 2024-03-06T07:26:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc309a0e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040c8f40 con 0x7fc304158db0 2024-03-06T07:26:43.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc309a0e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041559e0 con 0x7fc3040c88b0 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc302ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc3040c5330 0x7fc3041586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57648/0 (socket says 172.21.15.26:57648) 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc302ffd640 1 -- 172.21.15.26:0/1683773675 learned_addr learned my addr 172.21.15.26:0/1683773675 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196438950 0 0) 0x7fc2f4001490 con 0x7fc3040c5330 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f0003520 con 0x7fc3040c5330 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc3037fe640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 0x7fc3040c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1297189846 0 0) 0x7fc2f0003520 con 0x7fc3040c5330 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 msgr2=0x7fc3040c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc301ffb640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 0x7fc3040c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.557+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 >> v1:172.21.15.117:6789/0 conn(0x7fc304158db0 legacy=0x7fc30415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304155390 con 0x7fc3040c5330 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fc2f4001bc0 con 0x7fc3040c5330 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 >> v1:172.21.15.26:6789/0 conn(0x7fc3040c5330 legacy=0x7fc3041586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 0x7fc2f0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0003f80 0x7fc2f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0004d40 0x7fc2f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041559e0 con 0x7fc2f0003f80 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f000bb80 con 0x7fc2f0004d40 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f000bd80 con 0x7fc3040c88b0 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3027fc640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0003f80 0x7fc2f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc302ffd640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 0x7fc2f0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3037fe640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0004d40 0x7fc2f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3027fc640 1 -- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 msgr2=0x7fc2f0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3027fc640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3040c88b0 0x7fc2f0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3027fc640 1 -- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0004d40 msgr2=0x7fc2f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3027fc640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0004d40 0x7fc2f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3027fc640 1 -- 172.21.15.26:0/1683773675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0003520 con 0x7fc2f0003f80 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.558+0000 7fc3037fe640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0004d40 0x7fc2f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc3027fc640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0003f80 0x7fc2f0004800 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec001750 tx=0x7fc2f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4c9a131efea106 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec001370 con 0x7fc2f0003f80 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec0052d0 con 0x7fc2f0003f80 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc301ffb640 1 -- 172.21.15.26:0/1683773675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec003830 con 0x7fc2f0003f80 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc309a0e640 1 -- 172.21.15.26:0/1683773675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0003f80 msgr2=0x7fc2f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc309a0e640 1 --2- 172.21.15.26:0/1683773675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0003f80 0x7fc2f0004800 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec001750 tx=0x7fc2f0005830 comp rx=0 tx=0).stop 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc3037fe640 1 -- 172.21.15.26:0/1683773675 reap_dead start 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc309a0e640 1 -- 172.21.15.26:0/1683773675 shutdown_connections 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc309a0e640 1 -- 172.21.15.26:0/1683773675 >> 172.21.15.26:0/1683773675 conn(0x7fc3040ac560 msgr2=0x7fc3040ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.559+0000 7fc309a0e640 1 -- 172.21.15.26:0/1683773675 shutdown_connections 2024-03-06T07:26:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.560+0000 7fc309a0e640 1 -- 172.21.15.26:0/1683773675 wait complete. 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 Processor -- start 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 -- start start 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0003f80 0x7fc304156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30414e030 0x7fc30414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30414bef0 0x7fc3040c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041529d0 con 0x7fc30414e030 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406d090 con 0x7fc2f0003f80 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc309a0e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc304152b50 con 0x7fc30414bef0 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc3027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30414e030 0x7fc30414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.561+0000 7fc3027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30414e030 0x7fc30414b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53196/0 (socket says 172.21.15.26:53196) 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc3027fc640 1 -- 172.21.15.26:0/3652270316 learned_addr learned my addr 172.21.15.26:0/3652270316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc3027fc640 1 -- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30414bef0 msgr2=0x7fc3040c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc302ffd640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0003f80 0x7fc304156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc3027fc640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30414bef0 0x7fc3040c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc3027fc640 1 -- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0003f80 msgr2=0x7fc304156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc3027fc640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0003f80 0x7fc304156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc3027fc640 1 -- 172.21.15.26:0/3652270316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304157370 con 0x7fc30414e030 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc3027fc640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30414e030 0x7fc30414b9b0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec001870 tx=0x7fc2ec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc2fbfff640 1 -- 172.21.15.26:0/3652270316 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec041070 con 0x7fc30414e030 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc2fbfff640 1 -- 172.21.15.26:0/3652270316 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec005480 con 0x7fc30414e030 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc2fbfff640 1 -- 172.21.15.26:0/3652270316 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec001370 con 0x7fc30414e030 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc304157e80 con 0x7fc30414e030 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.562+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304158440 con 0x7fc30414e030 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.563+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3040abbe0 con 0x7fc30414e030 2024-03-06T07:26:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.563+0000 7fc2fbfff640 1 -- 172.21.15.26:0/3652270316 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2ec044070 con 0x7fc30414e030 2024-03-06T07:26:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.564+0000 7fc2fbfff640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2e406b500 0x7fc2e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.564+0000 7fc2fbfff640 1 -- 172.21.15.26:0/3652270316 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fc2ec0aba00 con 0x7fc30414e030 2024-03-06T07:26:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.564+0000 7fc302ffd640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2e406b500 0x7fc2e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.565+0000 7fc302ffd640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2e406b500 0x7fc2e406d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4001b60 tx=0x7fc2f4006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.568+0000 7fc2fbfff640 1 -- 172.21.15.26:0/3652270316 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc3040abbe0 con 0x7fc30414e030 2024-03-06T07:26:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:43.820+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]} v 0) v1 -- 0x7fc304067910 con 0x7fc30414e030 2024-03-06T07:26:44.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.773+0000 7fc2fbfff640 1 -- 172.21.15.26:0/3652270316 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]}]=0 v388) v1 ==== 146+0+0 (secure 0 0 0) 0x7fc304067910 con 0x7fc30414e030 2024-03-06T07:26:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2e406b500 msgr2=0x7fc2e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2e406b500 0x7fc2e406d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4001b60 tx=0x7fc2f4006040 comp rx=0 tx=0).stop 2024-03-06T07:26:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30414e030 msgr2=0x7fc30414b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30414e030 0x7fc30414b9b0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec001870 tx=0x7fc2ec001120 comp rx=0 tx=0).stop 2024-03-06T07:26:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 shutdown_connections 2024-03-06T07:26:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2e406b500 0x7fc2e406d9c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30414bef0 0x7fc3040c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30414e030 0x7fc30414b9b0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 --2- 172.21.15.26:0/3652270316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f0003f80 0x7fc304156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.776+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 >> 172.21.15.26:0/3652270316 conn(0x7fc3040ac560 msgr2=0x7fc304153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:44.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.777+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 shutdown_connections 2024-03-06T07:26:44.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.777+0000 7fc309a0e640 1 -- 172.21.15.26:0/3652270316 wait complete. 2024-03-06T07:26:44.791 INFO:tasks.cephfs_test_runner:That a vanilla standby is preferred over others with mds_join_fs set to another fs. ... ok 2024-03-06T07:26:44.792 INFO:tasks.cephfs_test_runner:Starting test: test_down (tasks.cephfs.test_failover.TestClusterResize) That down setting toggles and sets max_mds appropriately. 2024-03-06T07:26:44.792 INFO:tasks.cephfs_test_runner:test_down (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:26:44.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa489688640 1 Processor -- start 2024-03-06T07:26:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa489688640 1 -- start start 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa489688640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa4841550c0 0x7fa4841554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa489688640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 0x7fa48414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa489688640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4840aa660 con 0x7fa4840aa180 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa489688640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4840c4610 con 0x7fa4841550c0 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa489688640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840aa940 con 0x7fa4841559e0 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa4837fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa4840aa180 0x7fa4840c5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57658/0 (socket says 172.21.15.26:57658) 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.899+0000 7fa4837fe640 1 -- 172.21.15.26:0/1537411509 learned_addr learned my addr 172.21.15.26:0/1537411509 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa4827fc640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 0x7fa48414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa482ffd640 1 --2- 172.21.15.26:0/1537411509 >> v2:172.21.15.117:3300/0 conn(0x7fa4841550c0 0x7fa4841554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083453702 0 0) 0x7fa4840aa660 con 0x7fa4840aa180 2024-03-06T07:26:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa46c003520 con 0x7fa4840aa180 2024-03-06T07:26:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497275911 0 0) 0x7fa46c003520 con 0x7fa4840aa180 2024-03-06T07:26:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 msgr2=0x7fa48414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 0x7fa48414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 >> v2:172.21.15.117:3300/0 conn(0x7fa4841550c0 msgr2=0x7fa4841554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 --2- 172.21.15.26:0/1537411509 >> v2:172.21.15.117:3300/0 conn(0x7fa4841550c0 0x7fa4841554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840c4860 con 0x7fa4840aa180 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fa47c0019f0 con 0x7fa4840aa180 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 >> v1:172.21.15.26:6789/0 conn(0x7fa4840aa180 legacy=0x7fa4840c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa46c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 0x7fa46c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46c004d40 0x7fa46c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840c4610 con 0x7fa46c0039b0 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840aa940 con 0x7fa46c004d40 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c00ba70 con 0x7fa4841559e0 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa482ffd640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa46c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa4827fc640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 0x7fa46c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.900+0000 7fa4837fe640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46c004d40 0x7fa46c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.901+0000 7fa482ffd640 1 -- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 msgr2=0x7fa46c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.901+0000 7fa482ffd640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4841559e0 0x7fa46c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.901+0000 7fa482ffd640 1 -- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46c004d40 msgr2=0x7fa46c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.901+0000 7fa482ffd640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46c004d40 0x7fa46c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.901+0000 7fa482ffd640 1 -- 172.21.15.26:0/1537411509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c003520 con 0x7fa46c0039b0 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.901+0000 7fa482ffd640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa46c003dd0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa474009ac0 tx=0x7fa46c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f6d09a5417bfa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.901+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474039070 con 0x7fa46c0039b0 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47402c710 con 0x7fa46c0039b0 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa481ffb640 1 -- 172.21.15.26:0/1537411509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474009690 con 0x7fa46c0039b0 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa489688640 1 -- 172.21.15.26:0/1537411509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 msgr2=0x7fa46c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa489688640 1 --2- 172.21.15.26:0/1537411509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa46c003dd0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa474009ac0 tx=0x7fa46c005830 comp rx=0 tx=0).stop 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa4837fe640 1 -- 172.21.15.26:0/1537411509 reap_dead start 2024-03-06T07:26:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa489688640 1 -- 172.21.15.26:0/1537411509 shutdown_connections 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa489688640 1 -- 172.21.15.26:0/1537411509 >> 172.21.15.26:0/1537411509 conn(0x7fa4840c6710 msgr2=0x7fa4840c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa489688640 1 -- 172.21.15.26:0/1537411509 shutdown_connections 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.902+0000 7fa489688640 1 -- 172.21.15.26:0/1537411509 wait complete. 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.903+0000 7fa489688640 1 Processor -- start 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.903+0000 7fa489688640 1 -- start start 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa489688640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa48414b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa489688640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4840a8690 0x7fa4840a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa489688640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa484144c10 0x7fa484145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa489688640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840c5770 con 0x7fa46c0039b0 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa489688640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840c1520 con 0x7fa484144c10 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa489688640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840c58f0 con 0x7fa4840a8690 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa48414b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa48414b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53214/0 (socket says 172.21.15.26:53214) 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 -- 172.21.15.26:0/2824069797 learned_addr learned my addr 172.21.15.26:0/2824069797 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa4827fc640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4840a8690 0x7fa4840a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa4837fe640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa484144c10 0x7fa484145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 -- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4840a8690 msgr2=0x7fa4840a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4840a8690 0x7fa4840a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 -- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa484144c10 msgr2=0x7fa484145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa484144c10 0x7fa484145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.904+0000 7fa482ffd640 1 -- 172.21.15.26:0/2824069797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484070d70 con 0x7fa46c0039b0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.905+0000 7fa482ffd640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa48414b8d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa474037040 tx=0x7fa474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.905+0000 7fa46b7fe640 1 -- 172.21.15.26:0/2824069797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474042020 con 0x7fa46c0039b0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.905+0000 7fa46b7fe640 1 -- 172.21.15.26:0/2824069797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47403a040 con 0x7fa46c0039b0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.905+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa484072480 con 0x7fa46c0039b0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.905+0000 7fa46b7fe640 1 -- 172.21.15.26:0/2824069797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474039070 con 0x7fa46c0039b0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.905+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa484072a40 con 0x7fa46c0039b0 2024-03-06T07:26:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.906+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa484073440 con 0x7fa46c0039b0 2024-03-06T07:26:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.906+0000 7fa46b7fe640 1 -- 172.21.15.26:0/2824069797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa474045020 con 0x7fa46c0039b0 2024-03-06T07:26:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.907+0000 7fa46b7fe640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa47006b4d0 0x7fa47006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.907+0000 7fa46b7fe640 1 -- 172.21.15.26:0/2824069797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fa4740ab7a0 con 0x7fa46c0039b0 2024-03-06T07:26:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.907+0000 7fa4827fc640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa47006b4d0 0x7fa47006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.907+0000 7fa4827fc640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa47006b4d0 0x7fa47006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa4840c0830 tx=0x7fa47800a2a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:44.911+0000 7fa46b7fe640 1 -- 172.21.15.26:0/2824069797 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa484073440 con 0x7fa46c0039b0 2024-03-06T07:26:45.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.092+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa48404f330 con 0x7fa46c0039b0 2024-03-06T07:26:45.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.093+0000 7fa46b7fe640 1 -- 172.21.15.26:0/2824069797 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+13705 (secure 0 0 0) 0x7fa47404c030 con 0x7fa46c0039b0 2024-03-06T07:26:45.095 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:45.095 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":163,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:42.443164+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/3463955148":"2024-03-07T07:26:39.438335+0000","172.21.15.117:6836/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.26:6834/3463955148":"2024-03-07T07:26:39.438335+0000","172.21.15.117:6834/296681906":"2024-03-07T07:26:34.021889+0000","172.21.15.117:6835/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6833/3029467671":"2024-03-07T07:26:23.875878+0000","172.21.15.117:6837/677194061":"2024-03-07T07:26:33.014989+0000","172.21.15.26:6832/3029467671":"2024-03-07T07:26:23.875878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:45.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa47006b4d0 msgr2=0x7fa47006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa47006b4d0 0x7fa47006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa4840c0830 tx=0x7fa47800a2a0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 msgr2=0x7fa48414b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa48414b8d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa474037040 tx=0x7fa474004730 comp rx=0 tx=0).stop 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 shutdown_connections 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa47006b4d0 0x7fa47006d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa484144c10 0x7fa484145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4840a8690 0x7fa4840a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 --2- 172.21.15.26:0/2824069797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa46c0039b0 0x7fa48414b8d0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 >> 172.21.15.26:0/2824069797 conn(0x7fa4840c6710 msgr2=0x7fa48414c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 shutdown_connections 2024-03-06T07:26:45.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.095+0000 7fa489688640 1 -- 172.21.15.26:0/2824069797 wait complete. 2024-03-06T07:26:45.108 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:45.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:45.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.211+0000 7fa52270c640 1 Processor -- start 2024-03-06T07:26:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.211+0000 7fa52270c640 1 -- start start 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa52270c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa51c06c1f0 0x7fa51c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa52270c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c15a1a0 con 0x7fa51c0c4880 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa52270c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c15a480 con 0x7fa51c06dc50 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa52270c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c0c6170 con 0x7fa51c06c1f0 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa51b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa51c06c1f0 0x7fa51c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa51bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa51c0c4880 0x7fa51c0c4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57664/0 (socket says 172.21.15.26:57664) 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa51bfff640 1 -- 172.21.15.26:0/3598965170 learned_addr learned my addr 172.21.15.26:0/3598965170 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa51b7fe640 1 -- 172.21.15.26:0/3598965170 >> v1:172.21.15.117:6789/0 conn(0x7fa51c06dc50 legacy=0x7fa51c156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa51b7fe640 1 -- 172.21.15.26:0/3598965170 >> v1:172.21.15.26:6789/0 conn(0x7fa51c0c4880 legacy=0x7fa51c0c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.212+0000 7fa51b7fe640 1 -- 172.21.15.26:0/3598965170 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c06c6f0 con 0x7fa51c06c1f0 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/3598965170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06c1f0 0x7fa51c15ab00 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0097a0 tx=0x7fa50c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84662be28e27eb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa51affd640 1 -- 172.21.15.26:0/3598965170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c035020 con 0x7fa51c06c1f0 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa51affd640 1 -- 172.21.15.26:0/3598965170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c031830 con 0x7fa51c06c1f0 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa51affd640 1 -- 172.21.15.26:0/3598965170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c031c00 con 0x7fa51c06c1f0 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa52270c640 1 -- 172.21.15.26:0/3598965170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06c1f0 msgr2=0x7fa51c15ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa52270c640 1 --2- 172.21.15.26:0/3598965170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06c1f0 0x7fa51c15ab00 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0097a0 tx=0x7fa50c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa52270c640 1 -- 172.21.15.26:0/3598965170 shutdown_connections 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa52270c640 1 --2- 172.21.15.26:0/3598965170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06c1f0 0x7fa51c15ab00 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.213+0000 7fa52270c640 1 -- 172.21.15.26:0/3598965170 >> 172.21.15.26:0/3598965170 conn(0x7fa51c0c6710 msgr2=0x7fa51c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.214+0000 7fa52270c640 1 -- 172.21.15.26:0/3598965170 shutdown_connections 2024-03-06T07:26:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.214+0000 7fa52270c640 1 -- 172.21.15.26:0/3598965170 wait complete. 2024-03-06T07:26:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 Processor -- start 2024-03-06T07:26:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 -- start start 2024-03-06T07:26:45.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51c06c1f0 0x7fa51c0aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c06dc50 0x7fa51c071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0c4880 0x7fa51c154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0c5370 con 0x7fa51c0c4880 2024-03-06T07:26:45.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0c5e80 con 0x7fa51c06dc50 2024-03-06T07:26:45.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa52270c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0c0cb0 con 0x7fa51c06c1f0 2024-03-06T07:26:45.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.215+0000 7fa520ca9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0c4880 0x7fa51c154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0c4880 0x7fa51c154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53224/0 (socket says 172.21.15.26:53224) 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 -- 172.21.15.26:0/3361276298 learned_addr learned my addr 172.21.15.26:0/3361276298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c06dc50 0x7fa51c071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa51bfff640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51c06c1f0 0x7fa51c0aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 -- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51c06c1f0 msgr2=0x7fa51c0aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51c06c1f0 0x7fa51c0aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 -- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c06dc50 msgr2=0x7fa51c071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c06dc50 0x7fa51c071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 -- 172.21.15.26:0/3361276298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c153b00 con 0x7fa51c0c4880 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c06dc50 0x7fa51c071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa520ca9640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0c4880 0x7fa51c154340 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c7b0 tx=0x7fa51400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa518ff9640 1 -- 172.21.15.26:0/3361276298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514004b60 con 0x7fa51c0c4880 2024-03-06T07:26:45.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa518ff9640 1 -- 172.21.15.26:0/3361276298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa514004d00 con 0x7fa51c0c4880 2024-03-06T07:26:45.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c0731d0 con 0x7fa51c0c4880 2024-03-06T07:26:45.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa518ff9640 1 -- 172.21.15.26:0/3361276298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400aa90 con 0x7fa51c0c4880 2024-03-06T07:26:45.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.216+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c0aaa30 con 0x7fa51c0c4880 2024-03-06T07:26:45.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.217+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c04f330 con 0x7fa51c0c4880 2024-03-06T07:26:45.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.218+0000 7fa518ff9640 1 -- 172.21.15.26:0/3361276298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa51400ac30 con 0x7fa51c0c4880 2024-03-06T07:26:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.218+0000 7fa518ff9640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa50806b4d0 0x7fa50806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.218+0000 7fa518ff9640 1 -- 172.21.15.26:0/3361276298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fa51408a680 con 0x7fa51c0c4880 2024-03-06T07:26:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.218+0000 7fa51bfff640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa50806b4d0 0x7fa50806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.219+0000 7fa51bfff640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa50806b4d0 0x7fa50806d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa510000f60 tx=0x7fa510009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.222+0000 7fa518ff9640 1 -- 172.21.15.26:0/3361276298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa514059600 con 0x7fa51c0c4880 2024-03-06T07:26:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.404+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa51c0c5b20 con 0x7fa51c0c4880 2024-03-06T07:26:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.404+0000 7fa518ff9640 1 -- 172.21.15.26:0/3361276298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa514058f00 con 0x7fa51c0c4880 2024-03-06T07:26:45.406 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa50806b4d0 msgr2=0x7fa50806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa50806b4d0 0x7fa50806d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa510000f60 tx=0x7fa510009040 comp rx=0 tx=0).stop 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0c4880 msgr2=0x7fa51c154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0c4880 0x7fa51c154340 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c7b0 tx=0x7fa51400cc70 comp rx=0 tx=0).stop 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 shutdown_connections 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa50806b4d0 0x7fa50806d990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0c4880 0x7fa51c154340 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c06dc50 0x7fa51c071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 --2- 172.21.15.26:0/3361276298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51c06c1f0 0x7fa51c0aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 >> 172.21.15.26:0/3361276298 conn(0x7fa51c0c6710 msgr2=0x7fa51c1582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 shutdown_connections 2024-03-06T07:26:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.406+0000 7fa52270c640 1 -- 172.21.15.26:0/3361276298 wait complete. 2024-03-06T07:26:45.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down' 2024-03-06T07:26:45.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.519+0000 7f1e9584e640 1 Processor -- start 2024-03-06T07:26:45.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.519+0000 7f1e9584e640 1 -- start start 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e9584e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e90155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e9584e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1e900c8a00 0x7f1e90158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e9584e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e90156180 con 0x7f1e90159460 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e9584e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e901587b0 con 0x7f1e900c8a00 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e9584e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900c7760 con 0x7f1e90152920 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e87fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1e900c8a00 0x7f1e90158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e8effd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e90155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e8f7fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1e90159460 0x7f1e900c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:50456/0 (socket says 172.21.15.26:50456) 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e8f7fe640 1 -- 172.21.15.26:0/1852022454 learned_addr learned my addr 172.21.15.26:0/1852022454 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e8e7fc640 1 -- 172.21.15.26:0/1852022454 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903331259 0 0) 0x7f1e88001490 con 0x7f1e90159460 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e8e7fc640 1 -- 172.21.15.26:0/1852022454 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e7c003520 con 0x7f1e90159460 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e87fff640 1 -- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 msgr2=0x7f1e90155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e87fff640 1 --2- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e90155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e87fff640 1 -- 172.21.15.26:0/1852022454 >> v1:172.21.15.117:6789/0 conn(0x7f1e90159460 legacy=0x7f1e900c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e87fff640 1 -- 172.21.15.26:0/1852022454 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e900c7a40 con 0x7f1e900c8a00 2024-03-06T07:26:45.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.520+0000 7f1e8effd640 1 --2- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e90155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.521+0000 7f1e87fff640 1 --2- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c8a00 0x7f1e90158f20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1e780097a0 tx=0x7f1e7802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c9b5325e0732c0a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.521+0000 7f1e8e7fc640 1 -- 172.21.15.26:0/1852022454 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780315a0 con 0x7f1e900c8a00 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.521+0000 7f1e8e7fc640 1 -- 172.21.15.26:0/1852022454 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e78037070 con 0x7f1e900c8a00 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.521+0000 7f1e8e7fc640 1 -- 172.21.15.26:0/1852022454 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78031940 con 0x7f1e900c8a00 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 -- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c8a00 msgr2=0x7f1e90158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 --2- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c8a00 0x7f1e90158f20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1e780097a0 tx=0x7f1e7802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 -- 172.21.15.26:0/1852022454 shutdown_connections 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 --2- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c8a00 0x7f1e90158f20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 --2- 172.21.15.26:0/1852022454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e90155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 -- 172.21.15.26:0/1852022454 >> 172.21.15.26:0/1852022454 conn(0x7f1e900ac560 msgr2=0x7f1e900ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 -- 172.21.15.26:0/1852022454 shutdown_connections 2024-03-06T07:26:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.522+0000 7f1e9584e640 1 -- 172.21.15.26:0/1852022454 wait complete. 2024-03-06T07:26:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.523+0000 7f1e9584e640 1 Processor -- start 2024-03-06T07:26:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.523+0000 7f1e9584e640 1 -- start start 2024-03-06T07:26:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e9584e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e900c8a00 0x7f1e901532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e9584e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e901563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e9584e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c43b0 0x7f1e901491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e9584e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9006d030 con 0x7f1e900c8a00 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e9584e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e90158a00 con 0x7f1e900c43b0 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e9584e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900c5a40 con 0x7f1e90152920 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e900c8a00 0x7f1e901532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e900c8a00 0x7f1e901532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53238/0 (socket says 172.21.15.26:53238) 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 -- 172.21.15.26:0/3293110716 learned_addr learned my addr 172.21.15.26:0/3293110716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e87fff640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e901563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8f7fe640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c43b0 0x7f1e901491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 -- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 msgr2=0x7f1e901563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e901563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 -- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c43b0 msgr2=0x7f1e901491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c43b0 0x7f1e901491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.524+0000 7f1e8effd640 1 -- 172.21.15.26:0/3293110716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e900c3310 con 0x7f1e900c8a00 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.525+0000 7f1e8effd640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e900c8a00 0x7f1e901532e0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000ec10 tx=0x7f1e8000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.525+0000 7f1e8cff9640 1 -- 172.21.15.26:0/3293110716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80017070 con 0x7f1e900c8a00 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.525+0000 7f1e8cff9640 1 -- 172.21.15.26:0/3293110716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8000cc30 con 0x7f1e900c8a00 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.525+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e900c35f0 con 0x7f1e900c8a00 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.525+0000 7f1e8cff9640 1 -- 172.21.15.26:0/3293110716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80013470 con 0x7f1e900c8a00 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.525+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e900c3bb0 con 0x7f1e900c8a00 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.526+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e90075300 con 0x7f1e900c8a00 2024-03-06T07:26:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.526+0000 7f1e8cff9640 1 -- 172.21.15.26:0/3293110716 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e8001a070 con 0x7f1e900c8a00 2024-03-06T07:26:45.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.527+0000 7f1e8cff9640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e7006b4d0 0x7f1e7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.527+0000 7f1e8cff9640 1 -- 172.21.15.26:0/3293110716 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f1e8008a410 con 0x7f1e900c8a00 2024-03-06T07:26:45.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.527+0000 7f1e87fff640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e7006b4d0 0x7f1e7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.527+0000 7f1e87fff640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e7006b4d0 0x7f1e7006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1e78036000 tx=0x7f1e7803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.531+0000 7f1e8cff9640 1 -- 172.21.15.26:0/3293110716 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e90075300 con 0x7f1e900c8a00 2024-03-06T07:26:45.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:45.783+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down"]} v 0) v1 -- 0x7f1e90151030 con 0x7f1e900c8a00 2024-03-06T07:26:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.779+0000 7f1e8cff9640 1 -- 172.21.15.26:0/3293110716 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down"]}]=0 v390) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1e90151030 con 0x7f1e900c8a00 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e7006b4d0 msgr2=0x7f1e7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e7006b4d0 0x7f1e7006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1e78036000 tx=0x7f1e7803d040 comp rx=0 tx=0).stop 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e900c8a00 msgr2=0x7f1e901532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e900c8a00 0x7f1e901532e0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000ec10 tx=0x7f1e8000c520 comp rx=0 tx=0).stop 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 shutdown_connections 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1e7006b4d0 0x7f1e7006d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e900c43b0 0x7f1e901491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e90152920 0x7f1e901563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 --2- 172.21.15.26:0/3293110716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e900c8a00 0x7f1e901532e0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 >> 172.21.15.26:0/3293110716 conn(0x7f1e900ac560 msgr2=0x7f1e9014f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 shutdown_connections 2024-03-06T07:26:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.782+0000 7f1e9584e640 1 -- 172.21.15.26:0/3293110716 wait complete. 2024-03-06T07:26:46.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:26:46.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.907+0000 7f58eea55640 1 Processor -- start 2024-03-06T07:26:46.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.907+0000 7f58eea55640 1 -- start start 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58eea55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 0x7f58e8155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58eea55640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58e80c4ef0 0x7f58e8155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58eea55640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e8152cb0 con 0x7f58e81528d0 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58eea55640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e8155ca0 con 0x7f58e80c4ef0 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58eea55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80c38b0 con 0x7f58e80c67f0 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f58e81528d0 0x7f58e80724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57678/0 (socket says 172.21.15.26:57678) 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e7fff640 1 -- 172.21.15.26:0/3455535050 learned_addr learned my addr 172.21.15.26:0/3455535050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:46.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58ecff2640 1 --2- 172.21.15.26:0/3455535050 >> v2:172.21.15.117:3300/0 conn(0x7f58e80c4ef0 0x7f58e8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e77fe640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 0x7f58e8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502960360 0 0) 0x7f58e8152cb0 con 0x7f58e81528d0 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58d4003520 con 0x7f58e81528d0 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784974311 0 0) 0x7f58d4003520 con 0x7f58e81528d0 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 msgr2=0x7f58e8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e6ffd640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 0x7f58e8155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.908+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 >> v2:172.21.15.117:3300/0 conn(0x7f58e80c4ef0 msgr2=0x7f58e8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 --2- 172.21.15.26:0/3455535050 >> v2:172.21.15.117:3300/0 conn(0x7f58e80c4ef0 0x7f58e8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e81594f0 con 0x7f58e81528d0 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f58d0001670 con 0x7f58e81528d0 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 >> v1:172.21.15.26:6789/0 conn(0x7f58e81528d0 legacy=0x7f58e80724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58d40039b0 0x7f58d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d4004800 0x7f58d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 0x7f58d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e8155ca0 con 0x7f58d40039b0 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80c38b0 con 0x7f58d4004800 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d400b9c0 con 0x7f58e80c67f0 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e7fff640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58d40039b0 0x7f58d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58ecff2640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 0x7f58d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e77fe640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d4004800 0x7f58d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e7fff640 1 -- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 msgr2=0x7f58d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e7fff640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 0x7f58d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e7fff640 1 -- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d4004800 msgr2=0x7f58d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e7fff640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d4004800 0x7f58d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e7fff640 1 -- 172.21.15.26:0/3455535050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4003520 con 0x7f58d40039b0 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58ecff2640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e80c67f0 0x7f58d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.909+0000 7f58e77fe640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d4004800 0x7f58d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:46.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.910+0000 7f58e7fff640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58d40039b0 0x7f58d4003dd0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f58d0007fd0 tx=0x7f58d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d4356970af35b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.910+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0007d50 con 0x7f58d40039b0 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.910+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0030070 con 0x7f58d40039b0 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.910+0000 7f58e6ffd640 1 -- 172.21.15.26:0/3455535050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d002cd40 con 0x7f58d40039b0 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.910+0000 7f58eea55640 1 -- 172.21.15.26:0/3455535050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58d40039b0 msgr2=0x7f58d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.910+0000 7f58eea55640 1 --2- 172.21.15.26:0/3455535050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58d40039b0 0x7f58d4003dd0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f58d0007fd0 tx=0x7f58d4005830 comp rx=0 tx=0).stop 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.910+0000 7f58ecff2640 1 -- 172.21.15.26:0/3455535050 reap_dead start 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.911+0000 7f58eea55640 1 -- 172.21.15.26:0/3455535050 shutdown_connections 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.911+0000 7f58eea55640 1 -- 172.21.15.26:0/3455535050 >> 172.21.15.26:0/3455535050 conn(0x7f58e80aa6a0 msgr2=0x7f58e80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.911+0000 7f58eea55640 1 -- 172.21.15.26:0/3455535050 shutdown_connections 2024-03-06T07:26:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.911+0000 7f58eea55640 1 -- 172.21.15.26:0/3455535050 wait complete. 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 Processor -- start 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 -- start start 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d40039b0 0x7f58e81425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e81ee380 0x7f58e8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e8070460 0x7f58e80708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80a8250 con 0x7f58e8070460 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80c3390 con 0x7f58d40039b0 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.912+0000 7f58eea55640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81591a0 con 0x7f58e81ee380 2024-03-06T07:26:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e8070460 0x7f58e80708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e8070460 0x7f58e80708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53254/0 (socket says 172.21.15.26:53254) 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 -- 172.21.15.26:0/3172693252 learned_addr learned my addr 172.21.15.26:0/3172693252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58e7fff640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d40039b0 0x7f58e81425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58e77fe640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e81ee380 0x7f58e8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 -- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e81ee380 msgr2=0x7f58e8156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e81ee380 0x7f58e8156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 -- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d40039b0 msgr2=0x7f58e81425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d40039b0 0x7f58e81425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 -- 172.21.15.26:0/3172693252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e81ee8d0 con 0x7f58e8070460 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58e77fe640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e81ee380 0x7f58e8156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58ecff2640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e8070460 0x7f58e80708e0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009ed0 tx=0x7f58e0010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58e4ff9640 1 -- 172.21.15.26:0/3172693252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e001a030 con 0x7f58e8070460 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58e4ff9640 1 -- 172.21.15.26:0/3172693252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0003990 con 0x7f58e8070460 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e8157760 con 0x7f58e8070460 2024-03-06T07:26:46.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58e4ff9640 1 -- 172.21.15.26:0/3172693252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0003d80 con 0x7f58e8070460 2024-03-06T07:26:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.913+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8157120 con 0x7f58e8070460 2024-03-06T07:26:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.914+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e804f330 con 0x7f58e8070460 2024-03-06T07:26:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.915+0000 7f58e4ff9640 1 -- 172.21.15.26:0/3172693252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58e001d070 con 0x7f58e8070460 2024-03-06T07:26:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.915+0000 7f58e4ff9640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58c806b4d0 0x7f58c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.915+0000 7f58e4ff9640 1 -- 172.21.15.26:0/3172693252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f58e008ab90 con 0x7f58e8070460 2024-03-06T07:26:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.915+0000 7f58e7fff640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58c806b4d0 0x7f58c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.916+0000 7f58e7fff640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58c806b4d0 0x7f58c806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f58d0034000 tx=0x7f58d0007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:46.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:46.919+0000 7f58e4ff9640 1 -- 172.21.15.26:0/3172693252 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f58e0059b10 con 0x7f58e8070460 2024-03-06T07:26:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.099+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f58e8073120 con 0x7f58e8070460 2024-03-06T07:26:47.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.102+0000 7f58e4ff9640 1 -- 172.21.15.26:0/3172693252 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v57)=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f58e0059410 con 0x7f58e8070460 2024-03-06T07:26:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58c806b4d0 msgr2=0x7f58c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58c806b4d0 0x7f58c806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f58d0034000 tx=0x7f58d0007000 comp rx=0 tx=0).stop 2024-03-06T07:26:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e8070460 msgr2=0x7f58e80708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e8070460 0x7f58e80708e0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009ed0 tx=0x7f58e0010590 comp rx=0 tx=0).stop 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 shutdown_connections 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f58c806b4d0 0x7f58c806d990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58e8070460 0x7f58e80708e0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58e81ee380 0x7f58e8156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 --2- 172.21.15.26:0/3172693252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58d40039b0 0x7f58e81425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.104+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 >> 172.21.15.26:0/3172693252 conn(0x7f58e80aa6a0 msgr2=0x7f58e8072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.105+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 shutdown_connections 2024-03-06T07:26:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.105+0000 7f58eea55640 1 -- 172.21.15.26:0/3172693252 wait complete. 2024-03-06T07:26:47.117 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:26:47.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:47.162 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:47.209 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:26:47.209 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:26:47.209 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:47.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:26:47.224 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:26:47.224 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:47.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:26:47.242 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:26:47.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.363+0000 7fa888720640 1 Processor -- start 2024-03-06T07:26:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.363+0000 7fa888720640 1 -- start start 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa888720640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa880159270 0x7fa880155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa888720640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8800c73d0 con 0x7fa8800c88c0 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa888720640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8800be2e0 con 0x7fa88014f360 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa888720640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa880159700 con 0x7fa880159270 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa885cbb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa8800c88c0 0x7fa880158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36840/0 (socket says 172.21.15.26:36840) 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa885cbb640 1 -- 172.21.15.26:0/141538958 learned_addr learned my addr 172.21.15.26:0/141538958 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250961182 0 0) 0x7fa8800c73d0 con 0x7fa8800c88c0 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa86c003520 con 0x7fa8800c88c0 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.364+0000 7fa886cbd640 1 --2- 172.21.15.26:0/141538958 >> v2:172.21.15.117:3300/0 conn(0x7fa880159270 0x7fa880155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 522702326 0 0) 0x7fa86c003520 con 0x7fa8800c88c0 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 >> v2:172.21.15.117:3300/0 conn(0x7fa880159270 msgr2=0x7fa880155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 --2- 172.21.15.26:0/141538958 >> v2:172.21.15.117:3300/0 conn(0x7fa880159270 0x7fa880155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 >> v1:172.21.15.117:6789/0 conn(0x7fa88014f360 legacy=0x7fa8800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800c7670 con 0x7fa8800c88c0 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fa870001670 con 0x7fa8800c88c0 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 >> v1:172.21.15.26:6789/0 conn(0x7fa8800c88c0 legacy=0x7fa880158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa86c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa86c004800 0x7fa86c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa86c0051c0 0x7fa86c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880159700 con 0x7fa86c0039b0 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c00c160 con 0x7fa86c0051c0 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c00c360 con 0x7fa86c004800 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8864bc640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa86c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa886cbd640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa86c0051c0 0x7fa86c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa885cbb640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa86c004800 0x7fa86c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8864bc640 1 -- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa86c004800 msgr2=0x7fa86c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8864bc640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa86c004800 0x7fa86c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8864bc640 1 -- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa86c0051c0 msgr2=0x7fa86c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8864bc640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa86c0051c0 0x7fa86c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa886cbd640 1 -- 172.21.15.26:0/141538958 reap_dead start 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.365+0000 7fa8864bc640 1 -- 172.21.15.26:0/141538958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c003520 con 0x7fa86c0039b0 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.366+0000 7fa8864bc640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa86c003dd0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa878001e50 tx=0x7fa86c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9b92b18b0c9f33 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.366+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878001a70 con 0x7fa86c0039b0 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.366+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa878003b70 con 0x7fa86c0039b0 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.366+0000 7fa8854ba640 1 -- 172.21.15.26:0/141538958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780050d0 con 0x7fa86c0039b0 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.367+0000 7fa888720640 1 -- 172.21.15.26:0/141538958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 msgr2=0x7fa86c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.367+0000 7fa888720640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa86c003dd0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa878001e50 tx=0x7fa86c005830 comp rx=0 tx=0).stop 2024-03-06T07:26:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.367+0000 7fa888720640 1 -- 172.21.15.26:0/141538958 shutdown_connections 2024-03-06T07:26:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.367+0000 7fa888720640 1 --2- 172.21.15.26:0/141538958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa86c003dd0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.367+0000 7fa888720640 1 -- 172.21.15.26:0/141538958 >> 172.21.15.26:0/141538958 conn(0x7fa8800ac570 msgr2=0x7fa8800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.367+0000 7fa888720640 1 -- 172.21.15.26:0/141538958 shutdown_connections 2024-03-06T07:26:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.367+0000 7fa888720640 1 -- 172.21.15.26:0/141538958 wait complete. 2024-03-06T07:26:47.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.368+0000 7fa888720640 1 Processor -- start 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.368+0000 7fa888720640 1 -- start start 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.368+0000 7fa888720640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa8800bf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.368+0000 7fa888720640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8801ee060 0x7fa8800ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.368+0000 7fa888720640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8800c09f0 0x7fa88014bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.368+0000 7fa888720640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800c7c30 con 0x7fa86c0039b0 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa888720640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880155c10 con 0x7fa8800c09f0 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa888720640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880155d90 con 0x7fa8801ee060 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa8800bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa8800bf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48158/0 (socket says 172.21.15.26:48158) 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 -- 172.21.15.26:0/3380831988 learned_addr learned my addr 172.21.15.26:0/3380831988 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa885cbb640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8801ee060 0x7fa8800ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa886cbd640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8800c09f0 0x7fa88014bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 -- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8801ee060 msgr2=0x7fa8800ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8801ee060 0x7fa8800ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 -- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8800c09f0 msgr2=0x7fa88014bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8800c09f0 0x7fa88014bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 -- 172.21.15.26:0/3380831988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800bfe70 con 0x7fa86c0039b0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa885cbb640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8801ee060 0x7fa8800ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa886cbd640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8800c09f0 0x7fa88014bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa8864bc640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa8800bf930 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa878000f80 tx=0x7fa8780014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa876ffd640 1 -- 172.21.15.26:0/3380831988 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878006b50 con 0x7fa86c0039b0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa876ffd640 1 -- 172.21.15.26:0/3380831988 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa878006cf0 con 0x7fa86c0039b0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa876ffd640 1 -- 172.21.15.26:0/3380831988 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878007250 con 0x7fa86c0039b0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8800be890 con 0x7fa86c0039b0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.369+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8800bee50 con 0x7fa86c0039b0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.370+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88004f330 con 0x7fa86c0039b0 2024-03-06T07:26:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.371+0000 7fa876ffd640 1 -- 172.21.15.26:0/3380831988 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa878003a30 con 0x7fa86c0039b0 2024-03-06T07:26:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.371+0000 7fa876ffd640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa86406b4d0 0x7fa86406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.371+0000 7fa876ffd640 1 -- 172.21.15.26:0/3380831988 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fa8780abda0 con 0x7fa86c0039b0 2024-03-06T07:26:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.371+0000 7fa885cbb640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa86406b4d0 0x7fa86406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.372+0000 7fa885cbb640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa86406b4d0 0x7fa86406d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa870001eb0 tx=0x7fa870001d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.375+0000 7fa876ffd640 1 -- 172.21.15.26:0/3380831988 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa878033020 con 0x7fa86c0039b0 2024-03-06T07:26:47.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.594+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8800c8d50 con 0x7fa86c0039b0 2024-03-06T07:26:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.594+0000 7fa876ffd640 1 -- 172.21.15.26:0/3380831988 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+3707 (secure 0 0 0) 0x7fa87807ac90 con 0x7fa86c0039b0 2024-03-06T07:26:47.596 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:47.596 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":156,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7454,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1583030325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1583030325},{"type":"v1","addr":"172.21.15.117:6837","nonce":1583030325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":8668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1620185363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1620185363},{"type":"v1","addr":"172.21.15.117:6836","nonce":1620185363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156},{"gid":9084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/743496767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":743496767},{"type":"v1","addr":"172.21.15.26:6833","nonce":743496767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146},{"gid":9321,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3901640027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3901640027},{"type":"v1","addr":"172.21.15.26:6835","nonce":3901640027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[]} 2024-03-06T07:26:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 156 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa86406b4d0 msgr2=0x7fa86406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa86406b4d0 0x7fa86406d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa870001eb0 tx=0x7fa870001d80 comp rx=0 tx=0).stop 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 msgr2=0x7fa8800bf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa8800bf930 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa878000f80 tx=0x7fa8780014a0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 shutdown_connections 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa86406b4d0 0x7fa86406d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8800c09f0 0x7fa88014bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8801ee060 0x7fa8800ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 --2- 172.21.15.26:0/3380831988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa86c0039b0 0x7fa8800bf930 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 >> 172.21.15.26:0/3380831988 conn(0x7fa8800ac570 msgr2=0x7fa88014d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:47.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 shutdown_connections 2024-03-06T07:26:47.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.596+0000 7fa888720640 1 -- 172.21.15.26:0/3380831988 wait complete. 2024-03-06T07:26:47.609 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:26:47.609 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:26:47.609 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:26:47.609 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:47.609 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:26:47.610 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:26:47.610 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:26:47.610 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:47.610 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:26:47.610 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:26:47.610 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:26:47.610 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:47.610 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:26:47.611 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:26:47.611 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:26:47.611 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:47.618 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:26:47.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:26:47.619 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:26:47.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:26:47.622 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:26:47.622 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:26:47.623 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:26:47.623 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:26:47.660 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:26:47.661 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:47.661 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:26:47.661 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:47.664 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:26:47.664 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:47.665 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:26:47.665 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:47.665 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:26:47.736 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:26:47.736 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:26:47.747 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:26:47.747 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:26:47.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.770+0000 7ff2c3356640 1 Processor -- start 2024-03-06T07:26:47.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.770+0000 7ff2c3356640 1 -- start start 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c3356640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc06ff10 0x7ff2bc153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c3356640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2bc0a8390 con 0x7ff2bc0c8f20 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c3356640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2bc0730f0 con 0x7ff2bc0c7170 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c3356640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0c7620 con 0x7ff2bc06ff10 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c18f3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff2bc0c8f20 0x7ff2bc15dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36868/0 (socket says 172.21.15.26:36868) 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c18f3640 1 -- 172.21.15.26:0/2286363193 learned_addr learned my addr 172.21.15.26:0/2286363193 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c10f2640 1 --2- 172.21.15.26:0/2286363193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc06ff10 0x7ff2bc153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2b3fff640 1 -- 172.21.15.26:0/2286363193 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583698520 0 0) 0x7ff2bc0a8390 con 0x7ff2bc0c8f20 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2b3fff640 1 -- 172.21.15.26:0/2286363193 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2a8003520 con 0x7ff2bc0c8f20 2024-03-06T07:26:47.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2b3fff640 1 -- 172.21.15.26:0/2286363193 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562999086 0 0) 0x7ff2bc0730f0 con 0x7ff2bc0c7170 2024-03-06T07:26:47.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c10f2640 1 -- 172.21.15.26:0/2286363193 >> v1:172.21.15.117:6789/0 conn(0x7ff2bc0c7170 legacy=0x7ff2bc0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c10f2640 1 -- 172.21.15.26:0/2286363193 >> v1:172.21.15.26:6789/0 conn(0x7ff2bc0c8f20 legacy=0x7ff2bc15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.771+0000 7ff2c10f2640 1 -- 172.21.15.26:0/2286363193 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc0733d0 con 0x7ff2bc06ff10 2024-03-06T07:26:47.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.772+0000 7ff2c10f2640 1 --2- 172.21.15.26:0/2286363193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc06ff10 0x7ff2bc153c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80097d0 tx=0x7ff2b802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1423c32c1f694d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:47.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.772+0000 7ff2b3fff640 1 -- 172.21.15.26:0/2286363193 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8031770 con 0x7ff2bc06ff10 2024-03-06T07:26:47.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.772+0000 7ff2b3fff640 1 -- 172.21.15.26:0/2286363193 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b8037070 con 0x7ff2bc06ff10 2024-03-06T07:26:47.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.772+0000 7ff2b3fff640 1 -- 172.21.15.26:0/2286363193 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8031b30 con 0x7ff2bc06ff10 2024-03-06T07:26:47.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.773+0000 7ff2c3356640 1 -- 172.21.15.26:0/2286363193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc06ff10 msgr2=0x7ff2bc153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.773+0000 7ff2c3356640 1 --2- 172.21.15.26:0/2286363193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc06ff10 0x7ff2bc153c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80097d0 tx=0x7ff2b802b810 comp rx=0 tx=0).stop 2024-03-06T07:26:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.773+0000 7ff2c3356640 1 -- 172.21.15.26:0/2286363193 shutdown_connections 2024-03-06T07:26:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.773+0000 7ff2c3356640 1 --2- 172.21.15.26:0/2286363193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc06ff10 0x7ff2bc153c20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.773+0000 7ff2c3356640 1 -- 172.21.15.26:0/2286363193 >> 172.21.15.26:0/2286363193 conn(0x7ff2bc06b970 msgr2=0x7ff2bc0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.773+0000 7ff2c3356640 1 -- 172.21.15.26:0/2286363193 shutdown_connections 2024-03-06T07:26:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.773+0000 7ff2c3356640 1 -- 172.21.15.26:0/2286363193 wait complete. 2024-03-06T07:26:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.774+0000 7ff2c3356640 1 Processor -- start 2024-03-06T07:26:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.774+0000 7ff2c3356640 1 -- start start 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.774+0000 7ff2c3356640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2bc06ff10 0x7ff2bc156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.774+0000 7ff2c3356640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc0c7170 0x7ff2bc0a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.774+0000 7ff2c3356640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc0c8f20 0x7ff2bc1ed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.774+0000 7ff2c3356640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc070470 con 0x7ff2bc06ff10 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.774+0000 7ff2c3356640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0702f0 con 0x7ff2bc0c7170 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c3356640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0c6860 con 0x7ff2bc0c8f20 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2bc06ff10 0x7ff2bc156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2bc06ff10 0x7ff2bc156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48202/0 (socket says 172.21.15.26:48202) 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 -- 172.21.15.26:0/3892083656 learned_addr learned my addr 172.21.15.26:0/3892083656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c18f3640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc0c8f20 0x7ff2bc1ed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c08f1640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc0c7170 0x7ff2bc0a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 -- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc0c8f20 msgr2=0x7ff2bc1ed9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc0c8f20 0x7ff2bc1ed9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 -- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc0c7170 msgr2=0x7ff2bc0a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc0c7170 0x7ff2bc0a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 -- 172.21.15.26:0/3892083656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc155d90 con 0x7ff2bc06ff10 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c10f2640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2bc06ff10 0x7ff2bc156340 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80098f0 tx=0x7ff2b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2b1ffb640 1 -- 172.21.15.26:0/3892083656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8004030 con 0x7ff2bc06ff10 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2b1ffb640 1 -- 172.21.15.26:0/3892083656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b80041f0 con 0x7ff2bc06ff10 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc1eaf10 con 0x7ff2bc06ff10 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2b1ffb640 1 -- 172.21.15.26:0/3892083656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8003610 con 0x7ff2bc06ff10 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.775+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc0a8d30 con 0x7ff2bc06ff10 2024-03-06T07:26:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.776+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc04f330 con 0x7ff2bc06ff10 2024-03-06T07:26:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.777+0000 7ff2b1ffb640 1 -- 172.21.15.26:0/3892083656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2b804c060 con 0x7ff2bc06ff10 2024-03-06T07:26:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.777+0000 7ff2b1ffb640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b500 0x7ff2a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.777+0000 7ff2b1ffb640 1 -- 172.21.15.26:0/3892083656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5593+0+0 (secure 0 0 0) 0x7ff2b80ab540 con 0x7ff2bc06ff10 2024-03-06T07:26:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.777+0000 7ff2c08f1640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b500 0x7ff2a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.778+0000 7ff2c08f1640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b500 0x7ff2a006d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc09bd00 tx=0x7ff2ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.782+0000 7ff2b1ffb640 1 -- 172.21.15.26:0/3892083656 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2b8040070 con 0x7ff2bc06ff10 2024-03-06T07:26:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:47.986+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff2bc06fa80 con 0x7ff2bc06ff10 2024-03-06T07:26:48.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.794+0000 7ff2b1ffb640 1 -- 172.21.15.26:0/3892083656 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v168) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff2b807a390 con 0x7ff2bc06ff10 2024-03-06T07:26:48.795 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b500 msgr2=0x7ff2a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b500 0x7ff2a006d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc09bd00 tx=0x7ff2ac009040 comp rx=0 tx=0).stop 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2bc06ff10 msgr2=0x7ff2bc156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2bc06ff10 0x7ff2bc156340 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80098f0 tx=0x7ff2b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 shutdown_connections 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff2a006b500 0x7ff2a006d9c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2bc0c8f20 0x7ff2bc1ed9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc0c7170 0x7ff2bc0a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 --2- 172.21.15.26:0/3892083656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2bc06ff10 0x7ff2bc156340 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 >> 172.21.15.26:0/3892083656 conn(0x7ff2bc06b970 msgr2=0x7ff2bc0ad730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:48.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 shutdown_connections 2024-03-06T07:26:48.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.797+0000 7ff2c3356640 1 -- 172.21.15.26:0/3892083656 wait complete. 2024-03-06T07:26:48.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:26:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.918+0000 7fb2d53e9640 1 Processor -- start 2024-03-06T07:26:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.918+0000 7fb2d53e9640 1 -- start start 2024-03-06T07:26:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2d53e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d01506b0 0x7fb2d0153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2d53e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2d0156ea0 0x7fb2d00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2d53e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d0070800 con 0x7fb2d00acc00 2024-03-06T07:26:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2d53e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d0157320 con 0x7fb2d0156ea0 2024-03-06T07:26:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2d53e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0156450 con 0x7fb2d01506b0 2024-03-06T07:26:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2ce7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2d0156ea0 0x7fb2d00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2ce7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2d0156ea0 0x7fb2d00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48216/0 (socket says 172.21.15.26:48216) 2024-03-06T07:26:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/3814710751 learned_addr learned my addr 172.21.15.26:0/3814710751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/3814710751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d01506b0 msgr2=0x7fb2d0153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.919+0000 7fb2ceffd640 1 --2- 172.21.15.26:0/3814710751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d01506b0 0x7fb2d0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.920+0000 7fb2ce7fc640 1 --2- 172.21.15.26:0/3814710751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d01506b0 0x7fb2d0153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.920+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/3814710751 >> v1:172.21.15.117:6789/0 conn(0x7fb2d00acc00 legacy=0x7fb2d00c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.920+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/3814710751 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0070aa0 con 0x7fb2d0156ea0 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.920+0000 7fb2ce7fc640 1 --2- 172.21.15.26:0/3814710751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d0156ea0 0x7fb2d00702c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00c62c0 tx=0x7fb2c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c5aa5ff3536931 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.920+0000 7fb2cdffb640 1 -- 172.21.15.26:0/3814710751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4035020 con 0x7fb2d0156ea0 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.920+0000 7fb2cdffb640 1 -- 172.21.15.26:0/3814710751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4031640 con 0x7fb2d0156ea0 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.920+0000 7fb2cdffb640 1 -- 172.21.15.26:0/3814710751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c402d040 con 0x7fb2d0156ea0 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 -- 172.21.15.26:0/3814710751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d0156ea0 msgr2=0x7fb2d00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/3814710751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d0156ea0 0x7fb2d00702c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00c62c0 tx=0x7fb2c402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 -- 172.21.15.26:0/3814710751 shutdown_connections 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/3814710751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d0156ea0 0x7fb2d00702c0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/3814710751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d01506b0 0x7fb2d0153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 -- 172.21.15.26:0/3814710751 >> 172.21.15.26:0/3814710751 conn(0x7fb2d006b1a0 msgr2=0x7fb2d006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 -- 172.21.15.26:0/3814710751 shutdown_connections 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.921+0000 7fb2d53e9640 1 -- 172.21.15.26:0/3814710751 wait complete. 2024-03-06T07:26:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.922+0000 7fb2d53e9640 1 Processor -- start 2024-03-06T07:26:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.922+0000 7fb2d53e9640 1 -- start start 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2d53e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d00acc00 0x7fb2d0153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2d53e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d01506b0 0x7fb2d0151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2d53e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2d0156ea0 0x7fb2d0143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2d53e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d01566a0 con 0x7fb2d01506b0 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2d53e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0156a10 con 0x7fb2d0156ea0 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2d53e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00c5bb0 con 0x7fb2d00acc00 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d01506b0 0x7fb2d0151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d01506b0 0x7fb2d0151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48224/0 (socket says 172.21.15.26:48224) 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/2293063471 learned_addr learned my addr 172.21.15.26:0/2293063471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ceffd640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d00acc00 0x7fb2d0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2cf7fe640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2d0156ea0 0x7fb2d0143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d00acc00 msgr2=0x7fb2d0153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d00acc00 0x7fb2d0153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2d0156ea0 msgr2=0x7fb2d0143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2d0156ea0 0x7fb2d0143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ce7fc640 1 -- 172.21.15.26:0/2293063471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0150e70 con 0x7fb2d01506b0 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2ceffd640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d00acc00 0x7fb2d0153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.923+0000 7fb2cf7fe640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2d0156ea0 0x7fb2d0143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.924+0000 7fb2ce7fc640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d01506b0 0x7fb2d0151d20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.924+0000 7fb2b77fe640 1 -- 172.21.15.26:0/2293063471 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4040070 con 0x7fb2d01506b0 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.924+0000 7fb2b77fe640 1 -- 172.21.15.26:0/2293063471 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4039bf0 con 0x7fb2d01506b0 2024-03-06T07:26:48.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.924+0000 7fb2b77fe640 1 -- 172.21.15.26:0/2293063471 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4035040 con 0x7fb2d01506b0 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.924+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d01e9490 con 0x7fb2d01506b0 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.924+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d0151130 con 0x7fb2d01506b0 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.925+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d004f330 con 0x7fb2d01506b0 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.925+0000 7fb2b77fe640 1 -- 172.21.15.26:0/2293063471 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2c4038020 con 0x7fb2d01506b0 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.926+0000 7fb2b77fe640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.926+0000 7fb2b77fe640 1 -- 172.21.15.26:0/2293063471 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb2c40abbd0 con 0x7fb2d01506b0 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.926+0000 7fb2ceffd640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.926+0000 7fb2ceffd640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800a9d0 tx=0x7fb2c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:48.930+0000 7fb2b77fe640 1 -- 172.21.15.26:0/2293063471 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb2c4031610 con 0x7fb2d01506b0 2024-03-06T07:26:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.178+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb2d00700a0 con 0x7fb2d01506b0 2024-03-06T07:26:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.178+0000 7fb2b77fe640 1 -- 172.21.15.26:0/2293063471 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4724 (secure 0 0 0) 0x7fb2c407ac10 con 0x7fb2d01506b0 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:49.180 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.181 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:26:49.182 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.183 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:26:49.184 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:26:49.186 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:49.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:26:49.189 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:26:49.191 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2bc06b4d0 msgr2=0x7fb2bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800a9d0 tx=0x7fb2c8008040 comp rx=0 tx=0).stop 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d01506b0 msgr2=0x7fb2d0151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d01506b0 0x7fb2d0151d20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c40046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 shutdown_connections 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2d0156ea0 0x7fb2d0143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2d01506b0 0x7fb2d0151d20 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 --2- 172.21.15.26:0/2293063471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2d00acc00 0x7fb2d0153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 >> 172.21.15.26:0/2293063471 conn(0x7fb2d006b1a0 msgr2=0x7fb2d014e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 shutdown_connections 2024-03-06T07:26:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.180+0000 7fb2d53e9640 1 -- 172.21.15.26:0/2293063471 wait complete. 2024-03-06T07:26:49.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.294+0000 7f3e79255640 1 Processor -- start 2024-03-06T07:26:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e79255640 1 -- start start 2024-03-06T07:26:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e79255640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e740c5860 0x7f3e741609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e79255640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e741591c0 con 0x7f3e740c5480 2024-03-06T07:26:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e79255640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e741594a0 con 0x7f3e74155ba0 2024-03-06T07:26:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e79255640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7406df40 con 0x7f3e740c5860 2024-03-06T07:26:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e7359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e740c5860 0x7f3e741609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e72d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3e740c5480 0x7f3e7406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36870/0 (socket says 172.21.15.26:36870) 2024-03-06T07:26:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.295+0000 7f3e72d9d640 1 -- 172.21.15.26:0/1888036009 learned_addr learned my addr 172.21.15.26:0/1888036009 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:49.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.296+0000 7f3e7359e640 1 -- 172.21.15.26:0/1888036009 >> v1:172.21.15.117:6789/0 conn(0x7f3e74155ba0 legacy=0x7f3e741586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:49.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.296+0000 7f3e7359e640 1 -- 172.21.15.26:0/1888036009 >> v1:172.21.15.26:6789/0 conn(0x7f3e740c5480 legacy=0x7f3e7406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.296+0000 7f3e7359e640 1 -- 172.21.15.26:0/1888036009 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e741553a0 con 0x7f3e740c5860 2024-03-06T07:26:49.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.296+0000 7f3e7259c640 1 -- 172.21.15.26:0/1888036009 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657121879 0 0) 0x7f3e741591c0 con 0x7f3e740c5480 2024-03-06T07:26:49.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.296+0000 7f3e7359e640 1 --2- 172.21.15.26:0/1888036009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740c5860 0x7f3e741609e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1b6312ed9ec298 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e7259c640 1 -- 172.21.15.26:0/1888036009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c039070 con 0x7f3e740c5860 2024-03-06T07:26:49.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e7259c640 1 -- 172.21.15.26:0/1888036009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c02cbd0 con 0x7f3e740c5860 2024-03-06T07:26:49.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e7259c640 1 -- 172.21.15.26:0/1888036009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c034440 con 0x7f3e740c5860 2024-03-06T07:26:49.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e79255640 1 -- 172.21.15.26:0/1888036009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740c5860 msgr2=0x7f3e741609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e79255640 1 --2- 172.21.15.26:0/1888036009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740c5860 0x7f3e741609e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c02be20 comp rx=0 tx=0).stop 2024-03-06T07:26:49.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e79255640 1 -- 172.21.15.26:0/1888036009 shutdown_connections 2024-03-06T07:26:49.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e79255640 1 --2- 172.21.15.26:0/1888036009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740c5860 0x7f3e741609e0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e79255640 1 -- 172.21.15.26:0/1888036009 >> 172.21.15.26:0/1888036009 conn(0x7f3e7406b1a0 msgr2=0x7f3e7406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:49.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e79255640 1 -- 172.21.15.26:0/1888036009 shutdown_connections 2024-03-06T07:26:49.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.297+0000 7f3e79255640 1 -- 172.21.15.26:0/1888036009 wait complete. 2024-03-06T07:26:49.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.298+0000 7f3e79255640 1 Processor -- start 2024-03-06T07:26:49.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.298+0000 7f3e79255640 1 -- start start 2024-03-06T07:26:49.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e79255640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e740c5480 0x7f3e741dea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e79255640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e740c5860 0x7f3e741def80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e79255640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e74155ba0 0x7f3e741dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e79255640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740c5cf0 con 0x7f3e74155ba0 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e79255640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7406fdb0 con 0x7f3e740c5480 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e79255640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e74156000 con 0x7f3e740c5860 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e74155ba0 0x7f3e741dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e74155ba0 0x7f3e741dfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48234/0 (socket says 172.21.15.26:48234) 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 -- 172.21.15.26:0/3451075808 learned_addr learned my addr 172.21.15.26:0/3451075808 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e72d9d640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e740c5480 0x7f3e741dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 -- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e740c5860 msgr2=0x7f3e741def80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e740c5860 0x7f3e741def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 -- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e740c5480 msgr2=0x7f3e741dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e740c5480 0x7f3e741dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e7359e640 1 -- 172.21.15.26:0/3451075808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740ad7b0 con 0x7f3e74155ba0 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.299+0000 7f3e72d9d640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e740c5480 0x7f3e741dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:49.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.300+0000 7f3e7359e640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e74155ba0 0x7f3e741dfb40 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.300+0000 7f3e6b7fe640 1 -- 172.21.15.26:0/3451075808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c044020 con 0x7f3e74155ba0 2024-03-06T07:26:49.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.300+0000 7f3e6b7fe640 1 -- 172.21.15.26:0/3451075808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c03f470 con 0x7f3e74155ba0 2024-03-06T07:26:49.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.300+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e74156cc0 con 0x7f3e74155ba0 2024-03-06T07:26:49.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.300+0000 7f3e6b7fe640 1 -- 172.21.15.26:0/3451075808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c039040 con 0x7f3e74155ba0 2024-03-06T07:26:49.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.300+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e74157280 con 0x7f3e74155ba0 2024-03-06T07:26:49.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.301+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e74157480 con 0x7f3e74155ba0 2024-03-06T07:26:49.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.301+0000 7f3e6b7fe640 1 -- 172.21.15.26:0/3451075808 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e6c047020 con 0x7f3e74155ba0 2024-03-06T07:26:49.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.301+0000 7f3e6b7fe640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e5406b4d0 0x7f3e5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.302+0000 7f3e6b7fe640 1 -- 172.21.15.26:0/3451075808 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3e6c0acb60 con 0x7f3e74155ba0 2024-03-06T07:26:49.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.302+0000 7f3e72d9d640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e5406b4d0 0x7f3e5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.302+0000 7f3e72d9d640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e5406b4d0 0x7f3e5406d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000f60 tx=0x7f3e64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.306+0000 7f3e6b7fe640 1 -- 172.21.15.26:0/3451075808 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3e74157480 con 0x7f3e74155ba0 2024-03-06T07:26:49.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.488+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e7404f330 con 0x7f3e74155ba0 2024-03-06T07:26:49.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.489+0000 7f3e6b7fe640 1 -- 172.21.15.26:0/3451075808 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+13186 (secure 0 0 0) 0x7f3e6c04e030 con 0x7f3e74155ba0 2024-03-06T07:26:49.491 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:49.491 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":168,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:48.793225+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e5406b4d0 msgr2=0x7f3e5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e5406b4d0 0x7f3e5406d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000f60 tx=0x7f3e64009040 comp rx=0 tx=0).stop 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e74155ba0 msgr2=0x7f3e741dfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e74155ba0 0x7f3e741dfb40 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 shutdown_connections 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3e5406b4d0 0x7f3e5406d990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e74155ba0 0x7f3e741dfb40 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e740c5860 0x7f3e741def80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 --2- 172.21.15.26:0/3451075808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e740c5480 0x7f3e741dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 >> 172.21.15.26:0/3451075808 conn(0x7f3e7406b1a0 msgr2=0x7f3e740c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:49.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 shutdown_connections 2024-03-06T07:26:49.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.491+0000 7f3e79255640 1 -- 172.21.15.26:0/3451075808 wait complete. 2024-03-06T07:26:49.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:49.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:49.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.607+0000 7fc31efe9640 1 Processor -- start 2024-03-06T07:26:49.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.607+0000 7fc31efe9640 1 -- start start 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31efe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31806afc0 0x7fc31806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31efe9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3180c2580 0x7fc318159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31efe9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3180c5840 con 0x7fc3180c2120 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31efe9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc318158790 con 0x7fc3180c2580 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31efe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180c5b20 con 0x7fc31806afc0 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31d586640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3180c2580 0x7fc318159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31d586640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3180c2580 0x7fc318159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48236/0 (socket says 172.21.15.26:48236) 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31d586640 1 -- 172.21.15.26:0/1500966397 learned_addr learned my addr 172.21.15.26:0/1500966397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31cd85640 1 --2- 172.21.15.26:0/1500966397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31806afc0 0x7fc31806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31d586640 1 -- 172.21.15.26:0/1500966397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31806afc0 msgr2=0x7fc31806b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31d586640 1 --2- 172.21.15.26:0/1500966397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31806afc0 0x7fc31806b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31d586640 1 -- 172.21.15.26:0/1500966397 >> v1:172.21.15.117:6789/0 conn(0x7fc3180c2120 legacy=0x7fc318158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31d586640 1 -- 172.21.15.26:0/1500966397 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3180c5e00 con 0x7fc3180c2580 2024-03-06T07:26:49.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.608+0000 7fc31cd85640 1 --2- 172.21.15.26:0/1500966397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31806afc0 0x7fc31806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:49.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.609+0000 7fc31d586640 1 --2- 172.21.15.26:0/1500966397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc318159330 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc308009870 tx=0x7fc30802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abcb005e89ea2df server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.609+0000 7fc3177fe640 1 -- 172.21.15.26:0/1500966397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308035020 con 0x7fc3180c2580 2024-03-06T07:26:49.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.609+0000 7fc3177fe640 1 -- 172.21.15.26:0/1500966397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308031640 con 0x7fc3180c2580 2024-03-06T07:26:49.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.609+0000 7fc3177fe640 1 -- 172.21.15.26:0/1500966397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30802d040 con 0x7fc3180c2580 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 -- 172.21.15.26:0/1500966397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 msgr2=0x7fc318159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 --2- 172.21.15.26:0/1500966397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc318159330 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc308009870 tx=0x7fc30802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 -- 172.21.15.26:0/1500966397 shutdown_connections 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 --2- 172.21.15.26:0/1500966397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc318159330 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 --2- 172.21.15.26:0/1500966397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc31806afc0 0x7fc31806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 -- 172.21.15.26:0/1500966397 >> 172.21.15.26:0/1500966397 conn(0x7fc3180c4220 msgr2=0x7fc3180c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 -- 172.21.15.26:0/1500966397 shutdown_connections 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.610+0000 7fc31efe9640 1 -- 172.21.15.26:0/1500966397 wait complete. 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.611+0000 7fc31efe9640 1 Processor -- start 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.611+0000 7fc31efe9640 1 -- start start 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31efe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31806afc0 0x7fc3181ee730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31efe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3180c2120 0x7fc318157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31efe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc3181494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31efe9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180c2b30 con 0x7fc3180c2580 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31efe9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc318155220 con 0x7fc31806afc0 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31efe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180c2cb0 con 0x7fc3180c2120 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc3181494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc3181494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48242/0 (socket says 172.21.15.26:48242) 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 -- 172.21.15.26:0/3645234549 learned_addr learned my addr 172.21.15.26:0/3645234549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31cd85640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31806afc0 0x7fc3181ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc317fff640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3180c2120 0x7fc318157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 -- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3180c2120 msgr2=0x7fc318157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3180c2120 0x7fc318157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 -- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31806afc0 msgr2=0x7fc3181ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31806afc0 0x7fc3181ee730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 -- 172.21.15.26:0/3645234549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31814a0b0 con 0x7fc3180c2580 2024-03-06T07:26:49.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.612+0000 7fc31d586640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc3181494c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc30802b930 tx=0x7fc3080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.613+0000 7fc3157fa640 1 -- 172.21.15.26:0/3645234549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30803aa60 con 0x7fc3180c2580 2024-03-06T07:26:49.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.613+0000 7fc3157fa640 1 -- 172.21.15.26:0/3645234549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30803ac20 con 0x7fc3180c2580 2024-03-06T07:26:49.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.613+0000 7fc3157fa640 1 -- 172.21.15.26:0/3645234549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080443f0 con 0x7fc3180c2580 2024-03-06T07:26:49.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.613+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc318157060 con 0x7fc3180c2580 2024-03-06T07:26:49.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.613+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc318157970 con 0x7fc3180c2580 2024-03-06T07:26:49.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.614+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc318155010 con 0x7fc3180c2580 2024-03-06T07:26:49.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.614+0000 7fc3157fa640 1 -- 172.21.15.26:0/3645234549 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc30804b060 con 0x7fc3180c2580 2024-03-06T07:26:49.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.614+0000 7fc3157fa640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc30406b4d0 0x7fc30406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.614+0000 7fc3157fa640 1 -- 172.21.15.26:0/3645234549 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc3080abba0 con 0x7fc3180c2580 2024-03-06T07:26:49.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.615+0000 7fc31cd85640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc30406b4d0 0x7fc30406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.615+0000 7fc31cd85640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc30406b4d0 0x7fc30406d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc310003fd0 tx=0x7fc310003990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.619+0000 7fc3157fa640 1 -- 172.21.15.26:0/3645234549 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc318155010 con 0x7fc3180c2580 2024-03-06T07:26:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.803+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3180c31e0 con 0x7fc3180c2580 2024-03-06T07:26:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.803+0000 7fc3157fa640 1 -- 172.21.15.26:0/3645234549 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3180c31e0 con 0x7fc3180c2580 2024-03-06T07:26:49.804 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:26:49.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc30406b4d0 msgr2=0x7fc30406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc30406b4d0 0x7fc30406d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc310003fd0 tx=0x7fc310003990 comp rx=0 tx=0).stop 2024-03-06T07:26:49.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 msgr2=0x7fc3181494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc3181494c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc30802b930 tx=0x7fc3080046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 shutdown_connections 2024-03-06T07:26:49.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc30406b4d0 0x7fc30406d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3180c2580 0x7fc3181494c0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3180c2120 0x7fc318157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 --2- 172.21.15.26:0/3645234549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31806afc0 0x7fc3181ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 >> 172.21.15.26:0/3645234549 conn(0x7fc3180c4220 msgr2=0x7fc318154900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:49.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 shutdown_connections 2024-03-06T07:26:49.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.805+0000 7fc31efe9640 1 -- 172.21.15.26:0/3645234549 wait complete. 2024-03-06T07:26:49.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:26:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.918+0000 7fa012284640 1 Processor -- start 2024-03-06T07:26:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.918+0000 7fa012284640 1 -- start start 2024-03-06T07:26:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa012284640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa00c14f360 0x7fa00c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa012284640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa00c158b60 0x7fa00c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa012284640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00c0c73d0 con 0x7fa00c0c88c0 2024-03-06T07:26:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa012284640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c0c8dd0 con 0x7fa00c14f360 2024-03-06T07:26:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa012284640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c0be2e0 con 0x7fa00c158b60 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa00c14f360 0x7fa00c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa00c0c88c0 0x7fa00c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36878/0 (socket says 172.21.15.26:36878) 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00b7fe640 1 -- 172.21.15.26:0/2124146929 learned_addr learned my addr 172.21.15.26:0/2124146929 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00bfff640 1 -- 172.21.15.26:0/2124146929 >> v2:172.21.15.117:3300/0 conn(0x7fa00c158b60 msgr2=0x7fa00c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00bfff640 1 --2- 172.21.15.26:0/2124146929 >> v2:172.21.15.117:3300/0 conn(0x7fa00c158b60 0x7fa00c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00bfff640 1 -- 172.21.15.26:0/2124146929 >> v1:172.21.15.26:6789/0 conn(0x7fa00c0c88c0 legacy=0x7fa00c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00affd640 1 -- 172.21.15.26:0/2124146929 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119274318 0 0) 0x7fa00c0c73d0 con 0x7fa00c0c88c0 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.919+0000 7fa00bfff640 1 -- 172.21.15.26:0/2124146929 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0c7670 con 0x7fa00c14f360 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa00bfff640 1 --2- 172.21.15.26:0/2124146929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c14f360 0x7fa00c155c10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa00c152d60 tx=0x7fa00002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405f951804eda991 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa00affd640 1 -- 172.21.15.26:0/2124146929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0000315a0 con 0x7fa00c14f360 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa00affd640 1 -- 172.21.15.26:0/2124146929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000037070 con 0x7fa00c14f360 2024-03-06T07:26:49.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa00affd640 1 -- 172.21.15.26:0/2124146929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000031960 con 0x7fa00c14f360 2024-03-06T07:26:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa012284640 1 -- 172.21.15.26:0/2124146929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c14f360 msgr2=0x7fa00c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa012284640 1 --2- 172.21.15.26:0/2124146929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c14f360 0x7fa00c155c10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa00c152d60 tx=0x7fa00002b840 comp rx=0 tx=0).stop 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa012284640 1 -- 172.21.15.26:0/2124146929 shutdown_connections 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa012284640 1 --2- 172.21.15.26:0/2124146929 >> v2:172.21.15.117:3300/0 conn(0x7fa00c158b60 0x7fa00c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa012284640 1 --2- 172.21.15.26:0/2124146929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c14f360 0x7fa00c155c10 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.920+0000 7fa012284640 1 -- 172.21.15.26:0/2124146929 >> 172.21.15.26:0/2124146929 conn(0x7fa00c0ac570 msgr2=0x7fa00c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.921+0000 7fa012284640 1 -- 172.21.15.26:0/2124146929 shutdown_connections 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.921+0000 7fa012284640 1 -- 172.21.15.26:0/2124146929 wait complete. 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 Processor -- start 2024-03-06T07:26:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 -- start start 2024-03-06T07:26:49.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa00c0c88c0 0x7fa00c0c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa00c14f360 0x7fa00c0afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c158b60 0x7fa00c0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0c5a50 con 0x7fa00c158b60 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0c7c30 con 0x7fa00c0c88c0 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.922+0000 7fa012284640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0c5bd0 con 0x7fa00c14f360 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c158b60 0x7fa00c0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c158b60 0x7fa00c0b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48262/0 (socket says 172.21.15.26:48262) 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 -- 172.21.15.26:0/880080466 learned_addr learned my addr 172.21.15.26:0/880080466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa00bfff640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa00c0c88c0 0x7fa00c0c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa00b7fe640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa00c14f360 0x7fa00c0afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 -- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa00c14f360 msgr2=0x7fa00c0afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa00c14f360 0x7fa00c0afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 -- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa00c0c88c0 msgr2=0x7fa00c0c0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa00c0c88c0 0x7fa00c0c0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 -- 172.21.15.26:0/880080466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c1ebd80 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa010821640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c158b60 0x7fa00c0b0290 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa00400b970 tx=0x7fa00400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa008ff9640 1 -- 172.21.15.26:0/880080466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004019070 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa008ff9640 1 -- 172.21.15.26:0/880080466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00400cb60 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c0be740 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa008ff9640 1 -- 172.21.15.26:0/880080466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004014440 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.923+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c1eba10 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.924+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c04f330 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.925+0000 7fa008ff9640 1 -- 172.21.15.26:0/880080466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa00401c070 con 0x7fa00c158b60 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.925+0000 7fa008ff9640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ff806b4d0 0x7f9ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.925+0000 7fa008ff9640 1 -- 172.21.15.26:0/880080466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa004089d40 con 0x7fa00c158b60 2024-03-06T07:26:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.925+0000 7fa00bfff640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ff806b4d0 0x7f9ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.926+0000 7fa00bfff640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ff806b4d0 0x7f9ff806d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa000036000 tx=0x7fa000008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:49.930+0000 7fa008ff9640 1 -- 172.21.15.26:0/880080466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa004058d70 con 0x7fa00c158b60 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.114+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa00c1596b0 con 0x7fa00c158b60 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.114+0000 7fa008ff9640 1 -- 172.21.15.26:0/880080466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v168) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa004058670 con 0x7fa00c158b60 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:26:50.116 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:26:50.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ff806b4d0 msgr2=0x7f9ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ff806b4d0 0x7f9ff806d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa000036000 tx=0x7fa000008030 comp rx=0 tx=0).stop 2024-03-06T07:26:50.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c158b60 msgr2=0x7fa00c0b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c158b60 0x7fa00c0b0290 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa00400b970 tx=0x7fa00400be30 comp rx=0 tx=0).stop 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 shutdown_connections 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ff806b4d0 0x7f9ff806d990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa00c158b60 0x7fa00c0b0290 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa00c14f360 0x7fa00c0afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 --2- 172.21.15.26:0/880080466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa00c0c88c0 0x7fa00c0c0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 >> 172.21.15.26:0/880080466 conn(0x7fa00c0ac570 msgr2=0x7fa00c14da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 shutdown_connections 2024-03-06T07:26:50.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.117+0000 7fa012284640 1 -- 172.21.15.26:0/880080466 wait complete. 2024-03-06T07:26:50.129 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:26:50.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:26:50.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.233+0000 7f86ffde2640 1 Processor -- start 2024-03-06T07:26:50.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.234+0000 7f86ffde2640 1 -- start start 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.234+0000 7f86ffde2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 0x7f86f8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.234+0000 7f86ffde2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f8156150 con 0x7f86f8158780 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.234+0000 7f86ffde2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f80a7d70 con 0x7f86f80c6780 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.234+0000 7f86ffde2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80a8050 con 0x7f86f806fe20 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.234+0000 7f86fe37f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86f8158780 0x7f86f8158bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36882/0 (socket says 172.21.15.26:36882) 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.234+0000 7f86fe37f640 1 -- 172.21.15.26:0/1634139521 learned_addr learned my addr 172.21.15.26:0/1634139521 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fdb7e640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 0x7f86f8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857523824 0 0) 0x7f86f0001490 con 0x7f86f8158780 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e4003520 con 0x7f86f8158780 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279135218 0 0) 0x7f86f80a7d70 con 0x7f86f80c6780 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0001490 con 0x7f86f80c6780 2024-03-06T07:26:50.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1979322445 0 0) 0x7f86e4003520 con 0x7f86f8158780 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 msgr2=0x7f86f8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 0x7f86f8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 >> v1:172.21.15.117:6789/0 conn(0x7f86f80c6780 legacy=0x7f86f8072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80a8330 con 0x7f86f8158780 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f86f0001bc0 con 0x7f86f8158780 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 >> v1:172.21.15.26:6789/0 conn(0x7f86f8158780 legacy=0x7f86f8158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86e4003b70 0x7f86e4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 0x7f86e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4004d40 0x7f86e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80a8050 con 0x7f86e4003b70 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e400bac0 con 0x7f86e4004d40 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e400bcc0 con 0x7f86f806fe20 2024-03-06T07:26:50.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fdb7e640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86e4003b70 0x7f86e4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fd37d640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 0x7f86e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fe37f640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4004d40 0x7f86e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fdb7e640 1 -- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 msgr2=0x7f86e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fdb7e640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 0x7f86e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fdb7e640 1 -- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4004d40 msgr2=0x7f86e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fdb7e640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4004d40 0x7f86e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.235+0000 7f86fdb7e640 1 -- 172.21.15.26:0/1634139521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4003520 con 0x7f86e4003b70 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.236+0000 7f86fd37d640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f806fe20 0x7f86e4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.236+0000 7f86fdb7e640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86e4003b70 0x7f86e4003f70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009ac0 tx=0x7f86e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5146b6e2ae30e51 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.236+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4039070 con 0x7f86e4003b70 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.236+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f402c6b0 con 0x7f86e4003b70 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.236+0000 7f86fcb7c640 1 -- 172.21.15.26:0/1634139521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4009690 con 0x7f86e4003b70 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.237+0000 7f86ffde2640 1 -- 172.21.15.26:0/1634139521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86e4003b70 msgr2=0x7f86e4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.237+0000 7f86ffde2640 1 --2- 172.21.15.26:0/1634139521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86e4003b70 0x7f86e4003f70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009ac0 tx=0x7f86e4005830 comp rx=0 tx=0).stop 2024-03-06T07:26:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.237+0000 7f86fe37f640 1 -- 172.21.15.26:0/1634139521 reap_dead start 2024-03-06T07:26:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.237+0000 7f86ffde2640 1 -- 172.21.15.26:0/1634139521 shutdown_connections 2024-03-06T07:26:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.237+0000 7f86ffde2640 1 -- 172.21.15.26:0/1634139521 >> 172.21.15.26:0/1634139521 conn(0x7f86f80aa630 msgr2=0x7f86f80aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.237+0000 7f86ffde2640 1 -- 172.21.15.26:0/1634139521 shutdown_connections 2024-03-06T07:26:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.237+0000 7f86ffde2640 1 -- 172.21.15.26:0/1634139521 wait complete. 2024-03-06T07:26:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.238+0000 7f86ffde2640 1 Processor -- start 2024-03-06T07:26:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.238+0000 7f86ffde2640 1 -- start start 2024-03-06T07:26:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4003b70 0x7f86f81ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8153980 0x7f86f8154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f8154cc0 0x7f86f8151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8070200 con 0x7f86f8153980 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80c3600 con 0x7f86e4003b70 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8072c20 con 0x7f86f8154cc0 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8153980 0x7f86f8154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8153980 0x7f86f8154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48282/0 (socket says 172.21.15.26:48282) 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 -- 172.21.15.26:0/479407264 learned_addr learned my addr 172.21.15.26:0/479407264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 -- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f8154cc0 msgr2=0x7f86f8151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f8154cc0 0x7f86f8151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 -- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4003b70 msgr2=0x7f86f81ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4003b70 0x7f86f81ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 -- 172.21.15.26:0/479407264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f81ef570 con 0x7f86f8153980 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86fd37d640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8153980 0x7f86f8154780 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e780 tx=0x7f86e800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ee7fc640 1 -- 172.21.15.26:0/479407264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e80050d0 con 0x7f86f8153980 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ee7fc640 1 -- 172.21.15.26:0/479407264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8005270 con 0x7f86f8153980 2024-03-06T07:26:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f8153f00 con 0x7f86f8153980 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ee7fc640 1 -- 172.21.15.26:0/479407264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8005420 con 0x7f86f8153980 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.239+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f81ef290 con 0x7f86f8153980 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.240+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f80acf70 con 0x7f86f8153980 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.241+0000 7f86ee7fc640 1 -- 172.21.15.26:0/479407264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86e801a020 con 0x7f86f8153980 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.241+0000 7f86ee7fc640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86dc06b4d0 0x7f86dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.241+0000 7f86ee7fc640 1 -- 172.21.15.26:0/479407264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f86e8089710 con 0x7f86f8153980 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.242+0000 7f86fdb7e640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86dc06b4d0 0x7f86dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.242+0000 7f86fdb7e640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86dc06b4d0 0x7f86dc06d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f86f4036000 tx=0x7f86f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.246+0000 7f86ee7fc640 1 -- 172.21.15.26:0/479407264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f86f80acf70 con 0x7f86f8153980 2024-03-06T07:26:50.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.426+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f86f80c2f50 con 0x7f86f8153980 2024-03-06T07:26:50.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.808+0000 7f86ee7fc640 1 -- 172.21.15.26:0/479407264 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v169) v1 ==== 138+0+0 (secure 0 0 0) 0x7f86e80587e0 con 0x7f86f8153980 2024-03-06T07:26:50.810 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:26:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86dc06b4d0 msgr2=0x7f86dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86dc06b4d0 0x7f86dc06d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f86f4036000 tx=0x7f86f402e000 comp rx=0 tx=0).stop 2024-03-06T07:26:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8153980 msgr2=0x7f86f8154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8153980 0x7f86f8154780 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e780 tx=0x7f86e800ec40 comp rx=0 tx=0).stop 2024-03-06T07:26:50.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 shutdown_connections 2024-03-06T07:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f86dc06b4d0 0x7f86dc06d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86f8154cc0 0x7f86f8151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8153980 0x7f86f8154780 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 --2- 172.21.15.26:0/479407264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86e4003b70 0x7f86f81ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.811+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 >> 172.21.15.26:0/479407264 conn(0x7f86f80aa630 msgr2=0x7f86f806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.812+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 shutdown_connections 2024-03-06T07:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.812+0000 7f86ffde2640 1 -- 172.21.15.26:0/479407264 wait complete. 2024-03-06T07:26:50.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:26:50.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.935+0000 7f15078c3640 1 Processor -- start 2024-03-06T07:26:50.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.935+0000 7f15078c3640 1 -- start start 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f15078c3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f150014f250 0x7f1500155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f15078c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15000c8960 0x7f15000c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f15078c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150014f630 con 0x7f15000c5920 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f15078c3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15000c7470 con 0x7f150014f250 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f15078c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15000c7f60 con 0x7f15000c8960 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f150565f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f150014f250 0x7f1500155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f1505e60640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15000c5920 0x7f1500158aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36890/0 (socket says 172.21.15.26:36890) 2024-03-06T07:26:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f1505e60640 1 -- 172.21.15.26:0/656433255 learned_addr learned my addr 172.21.15.26:0/656433255 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.936+0000 7f1504e5e640 1 --2- 172.21.15.26:0/656433255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15000c8960 0x7f15000c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f150565f640 1 -- 172.21.15.26:0/656433255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15000c8960 msgr2=0x7f15000c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f150565f640 1 --2- 172.21.15.26:0/656433255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15000c8960 0x7f15000c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f150565f640 1 -- 172.21.15.26:0/656433255 >> v1:172.21.15.26:6789/0 conn(0x7f15000c5920 legacy=0x7f1500158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f150565f640 1 -- 172.21.15.26:0/656433255 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000c76c0 con 0x7f150014f250 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f150565f640 1 --2- 172.21.15.26:0/656433255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f150014f250 0x7f1500155bc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbc0121a0ee8519 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f14fffff640 1 -- 172.21.15.26:0/656433255 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8038070 con 0x7f150014f250 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f14fffff640 1 -- 172.21.15.26:0/656433255 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f8031700 con 0x7f150014f250 2024-03-06T07:26:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.937+0000 7f14fffff640 1 -- 172.21.15.26:0/656433255 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8031ad0 con 0x7f150014f250 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 -- 172.21.15.26:0/656433255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f150014f250 msgr2=0x7f1500155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 --2- 172.21.15.26:0/656433255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f150014f250 0x7f1500155bc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 -- 172.21.15.26:0/656433255 shutdown_connections 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 --2- 172.21.15.26:0/656433255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15000c8960 0x7f15000c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 --2- 172.21.15.26:0/656433255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f150014f250 0x7f1500155bc0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 -- 172.21.15.26:0/656433255 >> 172.21.15.26:0/656433255 conn(0x7f15000ac610 msgr2=0x7f15000aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 -- 172.21.15.26:0/656433255 shutdown_connections 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.938+0000 7f15078c3640 1 -- 172.21.15.26:0/656433255 wait complete. 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.939+0000 7f15078c3640 1 Processor -- start 2024-03-06T07:26:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.939+0000 7f15078c3640 1 -- start start 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f15078c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15000c5920 0x7f15000ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f15078c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000c8960 0x7f15000c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f15078c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150014f250 0x7f150014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f15078c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000be310 con 0x7f15000c5920 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f15078c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000c7c80 con 0x7f15000c8960 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f15078c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15001591b0 con 0x7f150014f250 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15000c5920 0x7f15000ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15000c5920 0x7f15000ae420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48300/0 (socket says 172.21.15.26:48300) 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 -- 172.21.15.26:0/1358495503 learned_addr learned my addr 172.21.15.26:0/1358495503 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f1505e60640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150014f250 0x7f150014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f1504e5e640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000c8960 0x7f15000c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 -- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150014f250 msgr2=0x7f150014b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150014f250 0x7f150014b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 -- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000c8960 msgr2=0x7f15000c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000c8960 0x7f15000c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.940+0000 7f150565f640 1 -- 172.21.15.26:0/1358495503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000c3b70 con 0x7f15000c5920 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.941+0000 7f150565f640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15000c5920 0x7f15000ae420 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f14f8036040 tx=0x7f14f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.941+0000 7f14fdffb640 1 -- 172.21.15.26:0/1358495503 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8042070 con 0x7f15000c5920 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.941+0000 7f14fdffb640 1 -- 172.21.15.26:0/1358495503 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f803d890 con 0x7f15000c5920 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.941+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15000c4a50 con 0x7f15000c5920 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.941+0000 7f14fdffb640 1 -- 172.21.15.26:0/1358495503 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8038040 con 0x7f15000c5920 2024-03-06T07:26:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.941+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15001ef240 con 0x7f15000c5920 2024-03-06T07:26:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.942+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150004f330 con 0x7f15000c5920 2024-03-06T07:26:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.942+0000 7f14fdffb640 1 -- 172.21.15.26:0/1358495503 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14f803b070 con 0x7f15000c5920 2024-03-06T07:26:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.943+0000 7f14fdffb640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14ec06b4d0 0x7f14ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.943+0000 7f14fdffb640 1 -- 172.21.15.26:0/1358495503 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f14f80ab7d0 con 0x7f15000c5920 2024-03-06T07:26:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.943+0000 7f1504e5e640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14ec06b4d0 0x7f14ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.944+0000 7f1504e5e640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14ec06b4d0 0x7f14ec06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f15000aa810 tx=0x7f14f4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:50.947+0000 7f14fdffb640 1 -- 172.21.15.26:0/1358495503 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14f8030070 con 0x7f15000c5920 2024-03-06T07:26:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.128+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f15000c56d0 con 0x7f15000c5920 2024-03-06T07:26:51.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.813+0000 7f14fdffb640 1 -- 172.21.15.26:0/1358495503 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v170) v1 ==== 170+0+0 (secure 0 0 0) 0x7f14f807a170 con 0x7f15000c5920 2024-03-06T07:26:51.815 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14ec06b4d0 msgr2=0x7f14ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14ec06b4d0 0x7f14ec06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f15000aa810 tx=0x7f14f4009190 comp rx=0 tx=0).stop 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15000c5920 msgr2=0x7f15000ae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15000c5920 0x7f15000ae420 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f14f8036040 tx=0x7f14f80046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 shutdown_connections 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f14ec06b4d0 0x7f14ec06d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f150014f250 0x7f150014b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000c8960 0x7f15000c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 --2- 172.21.15.26:0/1358495503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15000c5920 0x7f15000ae420 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.816+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 >> 172.21.15.26:0/1358495503 conn(0x7f15000ac610 msgr2=0x7f15000ab9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:51.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.817+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 shutdown_connections 2024-03-06T07:26:51.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.817+0000 7f15078c3640 1 -- 172.21.15.26:0/1358495503 wait complete. 2024-03-06T07:26:51.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:26:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.946+0000 7f3a0c4b5640 1 Processor -- start 2024-03-06T07:26:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.946+0000 7f3a0c4b5640 1 -- start start 2024-03-06T07:26:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.947+0000 7f3a0c4b5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a040c7370 0x7f3a040c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.947+0000 7f3a0c4b5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a04155260 con 0x7f3a041527f0 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.947+0000 7f3a0c4b5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a04155540 con 0x7f3a0406c0b0 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.947+0000 7f3a0c4b5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a040c8f70 con 0x7f3a040c7370 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.947+0000 7f3a0a251640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3a041527f0 0x7f3a040c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36896/0 (socket says 172.21.15.26:36896) 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.947+0000 7f3a0a251640 1 -- 172.21.15.26:0/1223799644 learned_addr learned my addr 172.21.15.26:0/1223799644 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.948+0000 7f3a09a50640 1 --2- 172.21.15.26:0/1223799644 >> v2:172.21.15.117:3300/0 conn(0x7f3a040c7370 0x7f3a040c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.948+0000 7f3a0924f640 1 -- 172.21.15.26:0/1223799644 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762476147 0 0) 0x7f3a04155260 con 0x7f3a041527f0 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.948+0000 7f3a0924f640 1 -- 172.21.15.26:0/1223799644 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f0003520 con 0x7f3a041527f0 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.948+0000 7f3a0924f640 1 -- 172.21.15.26:0/1223799644 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2614394281 0 0) 0x7f39f0003520 con 0x7f3a041527f0 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.948+0000 7f3a09a50640 1 -- 172.21.15.26:0/1223799644 >> v1:172.21.15.117:6789/0 conn(0x7f3a0406c0b0 legacy=0x7f3a0406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.948+0000 7f3a09a50640 1 -- 172.21.15.26:0/1223799644 >> v1:172.21.15.26:6789/0 conn(0x7f3a041527f0 legacy=0x7f3a040c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.948+0000 7f3a09a50640 1 -- 172.21.15.26:0/1223799644 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a04155820 con 0x7f3a040c7370 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.949+0000 7f3a09a50640 1 --2- 172.21.15.26:0/1223799644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a040c7370 0x7f3a040c7800 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a000097a0 tx=0x7f3a0002b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a842294afe440a76 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.949+0000 7f3a0924f640 1 -- 172.21.15.26:0/1223799644 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00037070 con 0x7f3a040c7370 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.949+0000 7f3a0924f640 1 -- 172.21.15.26:0/1223799644 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a00030510 con 0x7f3a040c7370 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.949+0000 7f3a0924f640 1 -- 172.21.15.26:0/1223799644 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8001690 con 0x7f3a040c7370 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.949+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/1223799644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a040c7370 msgr2=0x7f3a040c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.949+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/1223799644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a040c7370 0x7f3a040c7800 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a000097a0 tx=0x7f3a0002b570 comp rx=0 tx=0).stop 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.950+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/1223799644 shutdown_connections 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.950+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/1223799644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a040c7370 0x7f3a040c7800 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.950+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/1223799644 >> 172.21.15.26:0/1223799644 conn(0x7f3a040ac4f0 msgr2=0x7f3a040ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.950+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/1223799644 shutdown_connections 2024-03-06T07:26:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.950+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/1223799644 wait complete. 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 Processor -- start 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 -- start start 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a0406c0b0 0x7f3a041eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a040c7370 0x7f3a0414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a041527f0 0x7f3a041ed500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0406cbd0 con 0x7f3a040c7370 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040be1a0 con 0x7f3a041527f0 2024-03-06T07:26:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a0c4b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0406cd50 con 0x7f3a0406c0b0 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a09a50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a040c7370 0x7f3a0414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a09a50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a040c7370 0x7f3a0414b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48314/0 (socket says 172.21.15.26:48314) 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.951+0000 7f3a09a50640 1 -- 172.21.15.26:0/4284505663 learned_addr learned my addr 172.21.15.26:0/4284505663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a0a251640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a0406c0b0 0x7f3a041eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a0aa52640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a041527f0 0x7f3a041ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a09a50640 1 -- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a0406c0b0 msgr2=0x7f3a041eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a09a50640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a0406c0b0 0x7f3a041eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a09a50640 1 -- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a041527f0 msgr2=0x7f3a041ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a09a50640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a041527f0 0x7f3a041ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a09a50640 1 -- 172.21.15.26:0/4284505663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a041eadb0 con 0x7f3a040c7370 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a0aa52640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a041527f0 0x7f3a041ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a0a251640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a0406c0b0 0x7f3a041eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a09a50640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a040c7370 0x7f3a0414b280 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3a00035040 tx=0x7f3a000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:51.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f39f6ffd640 1 -- 172.21.15.26:0/4284505663 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00042020 con 0x7f3a040c7370 2024-03-06T07:26:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f39f6ffd640 1 -- 172.21.15.26:0/4284505663 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a000044c0 con 0x7f3a040c7370 2024-03-06T07:26:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f39f6ffd640 1 -- 172.21.15.26:0/4284505663 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00037040 con 0x7f3a040c7370 2024-03-06T07:26:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a041eb070 con 0x7f3a040c7370 2024-03-06T07:26:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.952+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a041eb630 con 0x7f3a040c7370 2024-03-06T07:26:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.953+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0404f330 con 0x7f3a040c7370 2024-03-06T07:26:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.954+0000 7f39f6ffd640 1 -- 172.21.15.26:0/4284505663 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a00045020 con 0x7f3a040c7370 2024-03-06T07:26:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.954+0000 7f39f6ffd640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39e806b4d0 0x7f39e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.954+0000 7f39f6ffd640 1 -- 172.21.15.26:0/4284505663 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3a000ac080 con 0x7f3a040c7370 2024-03-06T07:26:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.954+0000 7f3a0a251640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39e806b4d0 0x7f39e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.955+0000 7f3a0a251640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39e806b4d0 0x7f39e806d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f39f8001eb0 tx=0x7f39f8001ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:51.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:51.959+0000 7f39f6ffd640 1 -- 172.21.15.26:0/4284505663 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a0004c030 con 0x7f3a040c7370 2024-03-06T07:26:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.177+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3a0406d860 con 0x7f3a040c7370 2024-03-06T07:26:52.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.815+0000 7f39f6ffd640 1 -- 172.21.15.26:0/4284505663 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v158) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3a0007ad30 con 0x7f3a040c7370 2024-03-06T07:26:52.817 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 27 and data pool 28 2024-03-06T07:26:52.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.821+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39e806b4d0 msgr2=0x7f39e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:52.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.821+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39e806b4d0 0x7f39e806d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f39f8001eb0 tx=0x7f39f8001ca0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.821+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a040c7370 msgr2=0x7f3a0414b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:52.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.821+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a040c7370 0x7f3a0414b280 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3a00035040 tx=0x7f3a000046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.827+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 shutdown_connections 2024-03-06T07:26:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.827+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39e806b4d0 0x7f39e806d990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.827+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a041527f0 0x7f3a041ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.827+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a040c7370 0x7f3a0414b280 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.827+0000 7f3a0c4b5640 1 --2- 172.21.15.26:0/4284505663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a0406c0b0 0x7f3a041eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.828+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 >> 172.21.15.26:0/4284505663 conn(0x7f3a040ac4f0 msgr2=0x7f3a040c7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:52.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.829+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 shutdown_connections 2024-03-06T07:26:52.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.829+0000 7f3a0c4b5640 1 -- 172.21.15.26:0/4284505663 wait complete. 2024-03-06T07:26:52.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.962+0000 7fbc86913640 1 Processor -- start 2024-03-06T07:26:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.962+0000 7fbc86913640 1 -- start start 2024-03-06T07:26:52.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc86913640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbc8006da40 0x7fbc80155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:52.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc86913640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbc800c6780 0x7fbc80158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:52.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc86913640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc80156050 con 0x7fbc801592d0 2024-03-06T07:26:52.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc86913640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8006de20 con 0x7fbc8006da40 2024-03-06T07:26:52.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc86913640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc80072630 con 0x7fbc800c6780 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbc8006da40 0x7fbc80155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc84eb0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbc801592d0 0x7fbc800c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36902/0 (socket says 172.21.15.26:36902) 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc84eb0640 1 -- 172.21.15.26:0/922978497 learned_addr learned my addr 172.21.15.26:0/922978497 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7f7fe640 1 --2- 172.21.15.26:0/922978497 >> v2:172.21.15.117:3300/0 conn(0x7fbc800c6780 0x7fbc80158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7ffff640 1 -- 172.21.15.26:0/922978497 >> v2:172.21.15.117:3300/0 conn(0x7fbc800c6780 msgr2=0x7fbc80158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7ffff640 1 --2- 172.21.15.26:0/922978497 >> v2:172.21.15.117:3300/0 conn(0x7fbc800c6780 0x7fbc80158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7ffff640 1 -- 172.21.15.26:0/922978497 >> v1:172.21.15.26:6789/0 conn(0x7fbc801592d0 legacy=0x7fbc800c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7ffff640 1 -- 172.21.15.26:0/922978497 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80072910 con 0x7fbc8006da40 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7effd640 1 -- 172.21.15.26:0/922978497 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692753847 0 0) 0x7fbc78001490 con 0x7fbc801592d0 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.963+0000 7fbc7f7fe640 1 --2- 172.21.15.26:0/922978497 >> v2:172.21.15.117:3300/0 conn(0x7fbc800c6780 0x7fbc80158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc7ffff640 1 --2- 172.21.15.26:0/922978497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8006da40 0x7fbc80155b10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbc800c7120 tx=0x7fbc7402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea197e9db72cf44a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc7effd640 1 -- 172.21.15.26:0/922978497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740315a0 con 0x7fbc8006da40 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc7effd640 1 -- 172.21.15.26:0/922978497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74037070 con 0x7fbc8006da40 2024-03-06T07:26:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc7effd640 1 -- 172.21.15.26:0/922978497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74031960 con 0x7fbc8006da40 2024-03-06T07:26:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc86913640 1 -- 172.21.15.26:0/922978497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8006da40 msgr2=0x7fbc80155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc86913640 1 --2- 172.21.15.26:0/922978497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8006da40 0x7fbc80155b10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbc800c7120 tx=0x7fbc7402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc86913640 1 -- 172.21.15.26:0/922978497 shutdown_connections 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.964+0000 7fbc86913640 1 --2- 172.21.15.26:0/922978497 >> v2:172.21.15.117:3300/0 conn(0x7fbc800c6780 0x7fbc80158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.965+0000 7fbc86913640 1 --2- 172.21.15.26:0/922978497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8006da40 0x7fbc80155b10 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.965+0000 7fbc86913640 1 -- 172.21.15.26:0/922978497 >> 172.21.15.26:0/922978497 conn(0x7fbc800aa630 msgr2=0x7fbc800aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.965+0000 7fbc86913640 1 -- 172.21.15.26:0/922978497 shutdown_connections 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.965+0000 7fbc86913640 1 -- 172.21.15.26:0/922978497 wait complete. 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 Processor -- start 2024-03-06T07:26:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 -- start start 2024-03-06T07:26:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8006da40 0x7fbc801eec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc800c6780 0x7fbc801541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc801592d0 0x7fbc800ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800c3990 con 0x7fbc801592d0 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc80158890 con 0x7fbc8006da40 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.966+0000 7fbc86913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800c3b10 con 0x7fbc800c6780 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc801592d0 0x7fbc800ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc801592d0 0x7fbc800ab7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48326/0 (socket says 172.21.15.26:48326) 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 -- 172.21.15.26:0/3390414532 learned_addr learned my addr 172.21.15.26:0/3390414532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc7ffff640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8006da40 0x7fbc801eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc7f7fe640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc800c6780 0x7fbc801541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 -- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc800c6780 msgr2=0x7fbc801541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc800c6780 0x7fbc801541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 -- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8006da40 msgr2=0x7fbc801eec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8006da40 0x7fbc801eec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 -- 172.21.15.26:0/3390414532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800aeb50 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc7ffff640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8006da40 0x7fbc801eec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc84eb0640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc801592d0 0x7fbc800ab7f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c780 tx=0x7fbc7800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc7cff9640 1 -- 172.21.15.26:0/3390414532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78018070 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc7cff9640 1 -- 172.21.15.26:0/3390414532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7800ce30 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc801ee140 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc7cff9640 1 -- 172.21.15.26:0/3390414532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800a840 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.967+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc80154700 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.968+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8004f330 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.969+0000 7fbc7cff9640 1 -- 172.21.15.26:0/3390414532 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc7801b070 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.969+0000 7fbc7cff9640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.969+0000 7fbc7cff9640 1 -- 172.21.15.26:0/3390414532 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc78089960 con 0x7fbc801592d0 2024-03-06T07:26:52.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.969+0000 7fbc7ffff640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.970+0000 7fbc7ffff640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbc74036000 tx=0x7fbc74008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:52.974+0000 7fbc7cff9640 1 -- 172.21.15.26:0/3390414532 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbc80156050 con 0x7fbc801592d0 2024-03-06T07:26:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.158+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc80073240 con 0x7fbc801592d0 2024-03-06T07:26:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.159+0000 7fbc7cff9640 1 -- 172.21.15.26:0/3390414532 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16977 (secure 0 0 0) 0x7fbc78058650 con 0x7fbc801592d0 2024-03-06T07:26:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:53.161 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":171,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:52.178820+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:50.446132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:51.147227+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6c06b4d0 msgr2=0x7fbc6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbc74036000 tx=0x7fbc74008030 comp rx=0 tx=0).stop 2024-03-06T07:26:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc801592d0 msgr2=0x7fbc800ab7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc801592d0 0x7fbc800ab7f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c780 tx=0x7fbc7800cc40 comp rx=0 tx=0).stop 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 shutdown_connections 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc801592d0 0x7fbc800ab7f0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc800c6780 0x7fbc801541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 --2- 172.21.15.26:0/3390414532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8006da40 0x7fbc801eec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 >> 172.21.15.26:0/3390414532 conn(0x7fbc800aa630 msgr2=0x7fbc800c7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 shutdown_connections 2024-03-06T07:26:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.161+0000 7fbc86913640 1 -- 172.21.15.26:0/3390414532 wait complete. 2024-03-06T07:26:53.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.273+0000 7f325390f640 1 Processor -- start 2024-03-06T07:26:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.273+0000 7f325390f640 1 -- start start 2024-03-06T07:26:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f325390f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 0x7f324c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f325390f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f324c158b60 0x7f324c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f325390f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324c0c73d0 con 0x7f324c0c88c0 2024-03-06T07:26:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f325390f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c0c8dd0 con 0x7f324c158b60 2024-03-06T07:26:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f325390f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0be2e0 con 0x7f324c14f360 2024-03-06T07:26:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3250eaa640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f324c0c88c0 0x7f324c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36904/0 (socket says 172.21.15.26:36904) 2024-03-06T07:26:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3250eaa640 1 -- 172.21.15.26:0/3980825946 learned_addr learned my addr 172.21.15.26:0/3980825946 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3251eac640 1 --2- 172.21.15.26:0/3980825946 >> v2:172.21.15.117:3300/0 conn(0x7f324c158b60 0x7f324c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f32516ab640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 0x7f324c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062311056 0 0) 0x7f324c0c73d0 con 0x7f324c0c88c0 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3238003520 con 0x7f324c0c88c0 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2655248960 0 0) 0x7f3238003520 con 0x7f324c0c88c0 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 msgr2=0x7f324c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 0x7f324c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 >> v2:172.21.15.117:3300/0 conn(0x7f324c158b60 msgr2=0x7f324c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 --2- 172.21.15.26:0/3980825946 >> v2:172.21.15.117:3300/0 conn(0x7f324c158b60 0x7f324c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c0c7670 con 0x7f324c0c88c0 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f323c001670 con 0x7f324c0c88c0 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3251eac640 1 --2- 172.21.15.26:0/3980825946 >> v2:172.21.15.117:3300/0 conn(0x7f324c158b60 0x7f324c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 >> v1:172.21.15.26:6789/0 conn(0x7f324c0c88c0 legacy=0x7f324c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f3238003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3238004800 0x7f3238004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 0x7f32380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0be2e0 con 0x7f32380039b0 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323800b9c0 con 0x7f3238004800 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.274+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323800bb40 con 0x7f324c14f360 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f32516ab640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f3238003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f3250eaa640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3238004800 0x7f3238004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f3251eac640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 0x7f32380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f32516ab640 1 -- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 msgr2=0x7f32380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f32516ab640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c14f360 0x7f32380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f32516ab640 1 -- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3238004800 msgr2=0x7f3238004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f32516ab640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3238004800 0x7f3238004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.275+0000 7f32516ab640 1 -- 172.21.15.26:0/3980825946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238003520 con 0x7f32380039b0 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f32516ab640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f3238003dd0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f3248009af0 tx=0x7f3238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c525b69f49761ff server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248039070 con 0x7f32380039b0 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324802c6b0 con 0x7f32380039b0 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f3243fff640 1 -- 172.21.15.26:0/3980825946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248009690 con 0x7f32380039b0 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f325390f640 1 -- 172.21.15.26:0/3980825946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 msgr2=0x7f3238003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f325390f640 1 --2- 172.21.15.26:0/3980825946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f3238003dd0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f3248009af0 tx=0x7f3238005830 comp rx=0 tx=0).stop 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f3251eac640 1 -- 172.21.15.26:0/3980825946 reap_dead start 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f325390f640 1 -- 172.21.15.26:0/3980825946 shutdown_connections 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f325390f640 1 -- 172.21.15.26:0/3980825946 >> 172.21.15.26:0/3980825946 conn(0x7f324c0ac570 msgr2=0x7f324c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f325390f640 1 -- 172.21.15.26:0/3980825946 shutdown_connections 2024-03-06T07:26:53.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.276+0000 7f325390f640 1 -- 172.21.15.26:0/3980825946 wait complete. 2024-03-06T07:26:53.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.277+0000 7f325390f640 1 Processor -- start 2024-03-06T07:26:53.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.277+0000 7f325390f640 1 -- start start 2024-03-06T07:26:53.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f325390f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f324c0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f325390f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ad650 0x7f324c1eb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f325390f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c0ab6a0 0x7f324c1ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f325390f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0c5a50 con 0x7f32380039b0 2024-03-06T07:26:53.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f325390f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0c7c30 con 0x7f324c0ad650 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f325390f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0c5bd0 con 0x7f324c0ab6a0 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f324c0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f324c0adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48346/0 (socket says 172.21.15.26:48346) 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 -- 172.21.15.26:0/516342451 learned_addr learned my addr 172.21.15.26:0/516342451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f3250eaa640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ad650 0x7f324c1eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f3251eac640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c0ab6a0 0x7f324c1ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 -- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c0ab6a0 msgr2=0x7f324c1ee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c0ab6a0 0x7f324c1ee3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 -- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ad650 msgr2=0x7f324c1eb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ad650 0x7f324c1eb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f32516ab640 1 -- 172.21.15.26:0/516342451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c1eb5a0 con 0x7f32380039b0 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.278+0000 7f3250eaa640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ad650 0x7f324c1eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:53.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.279+0000 7f32516ab640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f324c0adb10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f3248009af0 tx=0x7f3248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.279+0000 7f3241ffb640 1 -- 172.21.15.26:0/516342451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248042020 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.279+0000 7f3241ffb640 1 -- 172.21.15.26:0/516342451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324803a040 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.279+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c158150 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.279+0000 7f3241ffb640 1 -- 172.21.15.26:0/516342451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248039070 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.279+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c158710 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.280+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c04f330 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.280+0000 7f3241ffb640 1 -- 172.21.15.26:0/516342451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3248045020 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.281+0000 7f3241ffb640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f323006b500 0x7f323006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.281+0000 7f3241ffb640 1 -- 172.21.15.26:0/516342451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32480ab8b0 con 0x7f32380039b0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.281+0000 7f3250eaa640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f323006b500 0x7f323006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.282+0000 7f3250eaa640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f323006b500 0x7f323006d9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f324c0b05c0 tx=0x7f323c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.285+0000 7f3241ffb640 1 -- 172.21.15.26:0/516342451 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f324804c070 con 0x7f32380039b0 2024-03-06T07:26:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.468+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f324c06cda0 con 0x7f32380039b0 2024-03-06T07:26:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.469+0000 7f3241ffb640 1 -- 172.21.15.26:0/516342451 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16977 (secure 0 0 0) 0x7f324807a510 con 0x7f32380039b0 2024-03-06T07:26:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:53.471 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":171,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:52.178820+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:50.446132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:51.147227+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f323006b500 msgr2=0x7f323006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f323006b500 0x7f323006d9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f324c0b05c0 tx=0x7f323c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 msgr2=0x7f324c0adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f324c0adb10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f3248009af0 tx=0x7f3248004730 comp rx=0 tx=0).stop 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 shutdown_connections 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f323006b500 0x7f323006d9c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f324c0ab6a0 0x7f324c1ee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ad650 0x7f324c1eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 --2- 172.21.15.26:0/516342451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32380039b0 0x7f324c0adb10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 >> 172.21.15.26:0/516342451 conn(0x7f324c0ac570 msgr2=0x7f324c0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 shutdown_connections 2024-03-06T07:26:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.472+0000 7f325390f640 1 -- 172.21.15.26:0/516342451 wait complete. 2024-03-06T07:26:53.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:26:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.588+0000 7fe5f95e9640 1 Processor -- start 2024-03-06T07:26:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.588+0000 7fe5f95e9640 1 -- start start 2024-03-06T07:26:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f95e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5f41529f0 0x7fe5f40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f95e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe5f40c5370 0x7fe5f40c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f95e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f4152dd0 con 0x7fe5f4155530 2024-03-06T07:26:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f95e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f4159710 con 0x7fe5f41529f0 2024-03-06T07:26:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f95e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f40c68b0 con 0x7fe5f40c5370 2024-03-06T07:26:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5f41529f0 0x7fe5f40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5f41529f0 0x7fe5f40c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48350/0 (socket says 172.21.15.26:48350) 2024-03-06T07:26:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f2ffd640 1 -- 172.21.15.26:0/1249890723 learned_addr learned my addr 172.21.15.26:0/1249890723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5ebfff640 1 --2- 172.21.15.26:0/1249890723 >> v2:172.21.15.117:3300/0 conn(0x7fe5f40c5370 0x7fe5f40c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f2ffd640 1 -- 172.21.15.26:0/1249890723 >> v2:172.21.15.117:3300/0 conn(0x7fe5f40c5370 msgr2=0x7fe5f40c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f2ffd640 1 --2- 172.21.15.26:0/1249890723 >> v2:172.21.15.117:3300/0 conn(0x7fe5f40c5370 0x7fe5f40c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f2ffd640 1 -- 172.21.15.26:0/1249890723 >> v1:172.21.15.117:6789/0 conn(0x7fe5f4155530 legacy=0x7fe5f41559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.589+0000 7fe5f2ffd640 1 -- 172.21.15.26:0/1249890723 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4158b00 con 0x7fe5f41529f0 2024-03-06T07:26:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.590+0000 7fe5f2ffd640 1 --2- 172.21.15.26:0/1249890723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f41529f0 0x7fe5f40c88f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe5f408c9b0 tx=0x7fe5e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f6024f644cbc5b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.590+0000 7fe5f27fc640 1 -- 172.21.15.26:0/1249890723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4035020 con 0x7fe5f41529f0 2024-03-06T07:26:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.590+0000 7fe5f27fc640 1 -- 172.21.15.26:0/1249890723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4031870 con 0x7fe5f41529f0 2024-03-06T07:26:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.590+0000 7fe5f27fc640 1 -- 172.21.15.26:0/1249890723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4031c40 con 0x7fe5f41529f0 2024-03-06T07:26:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 -- 172.21.15.26:0/1249890723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f41529f0 msgr2=0x7fe5f40c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/1249890723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f41529f0 0x7fe5f40c88f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe5f408c9b0 tx=0x7fe5e402b910 comp rx=0 tx=0).stop 2024-03-06T07:26:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 -- 172.21.15.26:0/1249890723 shutdown_connections 2024-03-06T07:26:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/1249890723 >> v2:172.21.15.117:3300/0 conn(0x7fe5f40c5370 0x7fe5f40c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/1249890723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f41529f0 0x7fe5f40c88f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 -- 172.21.15.26:0/1249890723 >> 172.21.15.26:0/1249890723 conn(0x7fe5f40ac580 msgr2=0x7fe5f40ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 -- 172.21.15.26:0/1249890723 shutdown_connections 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.591+0000 7fe5f95e9640 1 -- 172.21.15.26:0/1249890723 wait complete. 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 Processor -- start 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 -- start start 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5f40c5370 0x7fe5f40c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f41529f0 0x7fe5f4153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f4155530 0x7fe5f4153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f41593a0 con 0x7fe5f4155530 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f41590c0 con 0x7fe5f41529f0 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f95e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f41560d0 con 0x7fe5f40c5370 2024-03-06T07:26:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.592+0000 7fe5f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f4155530 0x7fe5f4153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f4155530 0x7fe5f4153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48364/0 (socket says 172.21.15.26:48364) 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 -- 172.21.15.26:0/4090837822 learned_addr learned my addr 172.21.15.26:0/4090837822 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5ebfff640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f41529f0 0x7fe5f4153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f2ffd640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5f40c5370 0x7fe5f40c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 -- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5f40c5370 msgr2=0x7fe5f40c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5f40c5370 0x7fe5f40c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 -- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f41529f0 msgr2=0x7fe5f4153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f41529f0 0x7fe5f4153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 -- 172.21.15.26:0/4090837822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4151900 con 0x7fe5f4155530 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5ebfff640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f41529f0 0x7fe5f4153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f37fe640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f4155530 0x7fe5f4153590 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c6e0 tx=0x7fe5ec00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5eb7fe640 1 -- 172.21.15.26:0/4090837822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec003b70 con 0x7fe5f4155530 2024-03-06T07:26:53.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5eb7fe640 1 -- 172.21.15.26:0/4090837822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec003d30 con 0x7fe5f4155530 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f41541d0 con 0x7fe5f4155530 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5eb7fe640 1 -- 172.21.15.26:0/4090837822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec005310 con 0x7fe5f4155530 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.593+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f4154790 con 0x7fe5f4155530 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.594+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f404f330 con 0x7fe5f4155530 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.595+0000 7fe5eb7fe640 1 -- 172.21.15.26:0/4090837822 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5ec0054b0 con 0x7fe5f4155530 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.595+0000 7fe5eb7fe640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe5d406b500 0x7fe5d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.595+0000 7fe5eb7fe640 1 -- 172.21.15.26:0/4090837822 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5ec08a180 con 0x7fe5f4155530 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.595+0000 7fe5f2ffd640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe5d406b500 0x7fe5d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.596+0000 7fe5f2ffd640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe5d406b500 0x7fe5d406d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4036000 tx=0x7fe5e403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.599+0000 7fe5eb7fe640 1 -- 172.21.15.26:0/4090837822 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe5ec058f00 con 0x7fe5f4155530 2024-03-06T07:26:53.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.821+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe5f40c9ff0 con 0x7fe5f4155530 2024-03-06T07:26:53.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.828+0000 7fe5eb7fe640 1 -- 172.21.15.26:0/4090837822 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v159) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe5ec058800 con 0x7fe5f4155530 2024-03-06T07:26:53.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.830+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe5d406b500 msgr2=0x7fe5d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.830+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe5d406b500 0x7fe5d406d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4036000 tx=0x7fe5e403c040 comp rx=0 tx=0).stop 2024-03-06T07:26:53.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.830+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f4155530 msgr2=0x7fe5f4153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.830+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f4155530 0x7fe5f4153590 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c6e0 tx=0x7fe5ec00cba0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 shutdown_connections 2024-03-06T07:26:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe5d406b500 0x7fe5d406d9c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f4155530 0x7fe5f4153590 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f41529f0 0x7fe5f4153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 --2- 172.21.15.26:0/4090837822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5f40c5370 0x7fe5f40c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 >> 172.21.15.26:0/4090837822 conn(0x7fe5f40ac580 msgr2=0x7fe5f406be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 shutdown_connections 2024-03-06T07:26:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.831+0000 7fe5f95e9640 1 -- 172.21.15.26:0/4090837822 wait complete. 2024-03-06T07:26:53.843 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:26:53.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:26:53.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.949+0000 7f4154488640 1 Processor -- start 2024-03-06T07:26:53.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.949+0000 7f4154488640 1 -- start start 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4154488640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f414c0a8290 0x7f414c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4154488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4154488640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414c0c62a0 con 0x7f414c0c69a0 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4154488640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414c153f00 con 0x7f414c0a8290 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4154488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c0706e0 con 0x7f414c0c4740 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4152224640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f414c0a8290 0x7f414c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4152224640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f414c0a8290 0x7f414c1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48376/0 (socket says 172.21.15.26:48376) 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4152224640 1 -- 172.21.15.26:0/1666579966 learned_addr learned my addr 172.21.15.26:0/1666579966 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:53.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4152224640 1 -- 172.21.15.26:0/1666579966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 msgr2=0x7f414c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4151a23640 1 --2- 172.21.15.26:0/1666579966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4152224640 1 --2- 172.21.15.26:0/1666579966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c0701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4152224640 1 -- 172.21.15.26:0/1666579966 >> v1:172.21.15.117:6789/0 conn(0x7f414c0c69a0 legacy=0x7f414c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.951+0000 7f4152224640 1 -- 172.21.15.26:0/1666579966 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c0c4c00 con 0x7f414c0a8290 2024-03-06T07:26:53.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.952+0000 7f4152224640 1 --2- 172.21.15.26:0/1666579966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1539c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4148009870 tx=0x7f414802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df35af31fe2eafe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.952+0000 7f4151222640 1 -- 172.21.15.26:0/1666579966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148038070 con 0x7f414c0a8290 2024-03-06T07:26:53.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.952+0000 7f4151222640 1 -- 172.21.15.26:0/1666579966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148031740 con 0x7f414c0a8290 2024-03-06T07:26:53.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.952+0000 7f4151222640 1 -- 172.21.15.26:0/1666579966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414802f050 con 0x7f414c0a8290 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 -- 172.21.15.26:0/1666579966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 msgr2=0x7f414c1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 --2- 172.21.15.26:0/1666579966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1539c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4148009870 tx=0x7f414802b7e0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 -- 172.21.15.26:0/1666579966 shutdown_connections 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 --2- 172.21.15.26:0/1666579966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 --2- 172.21.15.26:0/1666579966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1539c0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 -- 172.21.15.26:0/1666579966 >> 172.21.15.26:0/1666579966 conn(0x7f414c0c5560 msgr2=0x7f414c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 -- 172.21.15.26:0/1666579966 shutdown_connections 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.953+0000 7f4154488640 1 -- 172.21.15.26:0/1666579966 wait complete. 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 Processor -- start 2024-03-06T07:26:53.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 -- start start 2024-03-06T07:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f414c0c69a0 0x7f414c1eed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c06fc80 con 0x7f414c0a8290 2024-03-06T07:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c06f9a0 con 0x7f414c0c69a0 2024-03-06T07:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.954+0000 7f4154488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c072d90 con 0x7f414c0c4740 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48392/0 (socket says 172.21.15.26:48392) 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 -- 172.21.15.26:0/1325350254 learned_addr learned my addr 172.21.15.26:0/1325350254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4151a23640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152a25640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f414c0c69a0 0x7f414c1eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 -- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 msgr2=0x7f414c151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 -- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f414c0c69a0 msgr2=0x7f414c1eed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f414c0c69a0 0x7f414c1eed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 -- 172.21.15.26:0/1325350254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c1543e0 con 0x7f414c0a8290 2024-03-06T07:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4152224640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1ebb80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4148009870 tx=0x7f41480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4142ffd640 1 -- 172.21.15.26:0/1325350254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148042070 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4142ffd640 1 -- 172.21.15.26:0/1325350254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148033c10 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c1546c0 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4142ffd640 1 -- 172.21.15.26:0/1325350254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148038040 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.955+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c151c80 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.956+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c075300 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.957+0000 7f4142ffd640 1 -- 172.21.15.26:0/1325350254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4148045070 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.957+0000 7f4142ffd640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f413006b4d0 0x7f413006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.957+0000 7f4142ffd640 1 -- 172.21.15.26:0/1325350254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f41480ac020 con 0x7f414c0a8290 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.957+0000 7f4151a23640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f413006b4d0 0x7f413006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.958+0000 7f4151a23640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f413006b4d0 0x7f413006d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f414c0c5b70 tx=0x7f413c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:53.962+0000 7f4142ffd640 1 -- 172.21.15.26:0/1325350254 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f414c075300 con 0x7f414c0a8290 2024-03-06T07:26:54.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.183+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f414c1ee6b0 con 0x7f414c0a8290 2024-03-06T07:26:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.833+0000 7f4142ffd640 1 -- 172.21.15.26:0/1325350254 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v160) v1 ==== 122+0+0 (secure 0 0 0) 0x7f414c1ee6b0 con 0x7f414c0a8290 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f413006b4d0 msgr2=0x7f413006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f413006b4d0 0x7f413006d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f414c0c5b70 tx=0x7f413c008040 comp rx=0 tx=0).stop 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 msgr2=0x7f414c1ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1ebb80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4148009870 tx=0x7f41480046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 shutdown_connections 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f413006b4d0 0x7f413006d990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f414c0c69a0 0x7f414c1eed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f414c0c4740 0x7f414c151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 --2- 172.21.15.26:0/1325350254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f414c0a8290 0x7f414c1ebb80 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 >> 172.21.15.26:0/1325350254 conn(0x7f414c0c5560 msgr2=0x7f414c0a94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 shutdown_connections 2024-03-06T07:26:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.835+0000 7f4154488640 1 -- 172.21.15.26:0/1325350254 wait complete. 2024-03-06T07:26:54.848 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:26:54.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:54.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.948+0000 7fa11f70d640 1 Processor -- start 2024-03-06T07:26:54.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.948+0000 7fa11f70d640 1 -- start start 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11f70d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 0x7fa118155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11f70d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1180c2190 0x7fa1180c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11f70d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11806afc0 con 0x7fa1180c1280 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11f70d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa118156160 con 0x7fa1180c2190 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11f70d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180c1e20 con 0x7fa118152920 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11dcaa640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa1180c1280 0x7fa1180c1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36916/0 (socket says 172.21.15.26:36916) 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11dcaa640 1 -- 172.21.15.26:0/2888158865 learned_addr learned my addr 172.21.15.26:0/2888158865 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490585371 0 0) 0x7fa11806afc0 con 0x7fa1180c1280 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11d4a9640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 0x7fa118155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa11cca8640 1 --2- 172.21.15.26:0/2888158865 >> v2:172.21.15.117:3300/0 conn(0x7fa1180c2190 0x7fa1180c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.949+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa104003520 con 0x7fa1180c1280 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1934128237 0 0) 0x7fa104003520 con 0x7fa1180c1280 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 msgr2=0x7fa118155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 0x7fa118155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 >> v2:172.21.15.117:3300/0 conn(0x7fa1180c2190 msgr2=0x7fa1180c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 --2- 172.21.15.26:0/2888158865 >> v2:172.21.15.117:3300/0 conn(0x7fa1180c2190 0x7fa1180c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11806b2a0 con 0x7fa1180c1280 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11d4a9640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 0x7fa118155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fa1100019f0 con 0x7fa1180c1280 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 >> v1:172.21.15.26:6789/0 conn(0x7fa1180c1280 legacy=0x7fa1180c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 0x7fa104004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa1040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa118156160 con 0x7fa104004d40 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180c1e20 con 0x7fa1040039b0 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10400b9c0 con 0x7fa118152920 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11dcaa640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa1040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11cca8640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 0x7fa104004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11d4a9640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11dcaa640 1 -- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 msgr2=0x7fa104004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11dcaa640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa118152920 0x7fa104004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11dcaa640 1 -- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040039b0 msgr2=0x7fa104003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11dcaa640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.950+0000 7fa11dcaa640 1 -- 172.21.15.26:0/2888158865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104003520 con 0x7fa104004d40 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.951+0000 7fa11dcaa640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa1040051c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa11000a920 tx=0x7fa104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e05405dfd7567ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.951+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11000a4c0 con 0x7fa104004d40 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.951+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa110034630 con 0x7fa104004d40 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.951+0000 7fa10ffff640 1 -- 172.21.15.26:0/2888158865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110034a60 con 0x7fa104004d40 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.952+0000 7fa11f70d640 1 -- 172.21.15.26:0/2888158865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 msgr2=0x7fa1040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.952+0000 7fa11f70d640 1 --2- 172.21.15.26:0/2888158865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa1040051c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa11000a920 tx=0x7fa104005830 comp rx=0 tx=0).stop 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.952+0000 7fa11dcaa640 1 -- 172.21.15.26:0/2888158865 reap_dead start 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.952+0000 7fa11f70d640 1 -- 172.21.15.26:0/2888158865 shutdown_connections 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.952+0000 7fa11f70d640 1 -- 172.21.15.26:0/2888158865 >> 172.21.15.26:0/2888158865 conn(0x7fa1180c4290 msgr2=0x7fa1180c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.952+0000 7fa11f70d640 1 -- 172.21.15.26:0/2888158865 shutdown_connections 2024-03-06T07:26:54.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.952+0000 7fa11f70d640 1 -- 172.21.15.26:0/2888158865 wait complete. 2024-03-06T07:26:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.953+0000 7fa11f70d640 1 Processor -- start 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.953+0000 7fa11f70d640 1 -- start start 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11f70d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa118145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11f70d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1180a7d90 0x7fa11814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11f70d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1181ec880 0x7fa1181ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11f70d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180c0d60 con 0x7fa104004d40 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11f70d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11806b860 con 0x7fa1181ec880 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11f70d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180a7100 con 0x7fa1180a7d90 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa118145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa118145ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48406/0 (socket says 172.21.15.26:48406) 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 -- 172.21.15.26:0/344224379 learned_addr learned my addr 172.21.15.26:0/344224379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11dcaa640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1181ec880 0x7fa1181ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11cca8640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1180a7d90 0x7fa11814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 -- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1180a7d90 msgr2=0x7fa11814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1180a7d90 0x7fa11814a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 -- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1181ec880 msgr2=0x7fa1181ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1181ec880 0x7fa1181ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 -- 172.21.15.26:0/344224379 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180a8300 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11dcaa640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1181ec880 0x7fa1181ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11d4a9640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa118145ab0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa114011590 tx=0x7fa114011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa10dffb640 1 -- 172.21.15.26:0/344224379 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114019070 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa10dffb640 1 -- 172.21.15.26:0/344224379 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa114004220 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1181ec0c0 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.954+0000 7fa10dffb640 1 -- 172.21.15.26:0/344224379 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400f040 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.955+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1181eefb0 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.956+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11804f330 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.956+0000 7fa10dffb640 1 -- 172.21.15.26:0/344224379 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa114015670 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.956+0000 7fa10dffb640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.956+0000 7fa10dffb640 1 -- 172.21.15.26:0/344224379 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa11408a640 con 0x7fa104004d40 2024-03-06T07:26:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.957+0000 7fa11cca8640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:54.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.957+0000 7fa11cca8640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa1181477f0 tx=0x7fa108008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:54.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:54.961+0000 7fa10dffb640 1 -- 172.21.15.26:0/344224379 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa114059320 con 0x7fa104004d40 2024-03-06T07:26:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.182+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa118068460 con 0x7fa104004d40 2024-03-06T07:26:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.182+0000 7fa10dffb640 1 -- 172.21.15.26:0/344224379 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+4990 (secure 0 0 0) 0x7fa114058c20 con 0x7fa104004d40 2024-03-06T07:26:55.184 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:55.184 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":160,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:54.184709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9402,"qdb_cluster":[9402]},"id":13}]} 2024-03-06T07:26:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 160 2024-03-06T07:26:55.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.184+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0fc06b4d0 msgr2=0x7fa0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.184+0000 7fa11f70d640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa1181477f0 tx=0x7fa108008040 comp rx=0 tx=0).stop 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.184+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 msgr2=0x7fa118145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.184+0000 7fa11f70d640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa118145ab0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa114011590 tx=0x7fa114011a50 comp rx=0 tx=0).stop 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 shutdown_connections 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1181ec880 0x7fa1181ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1180a7d90 0x7fa11814a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 --2- 172.21.15.26:0/344224379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa104004d40 0x7fa118145ab0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 >> 172.21.15.26:0/344224379 conn(0x7fa1180c4290 msgr2=0x7fa1180c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 shutdown_connections 2024-03-06T07:26:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.185+0000 7fa11f70d640 1 -- 172.21.15.26:0/344224379 wait complete. 2024-03-06T07:26:55.197 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95e6e2a640 1 Processor -- start 2024-03-06T07:26:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95e6e2a640 1 -- start start 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95e6e2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 0x7f95e01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95e6e2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00c28d0 con 0x7f95e00c1aa0 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95e6e2a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00c2bb0 con 0x7f95e0155a40 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95e6e2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00c2680 con 0x7f95e014f790 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95dffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95e00c1aa0 0x7f95e00c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36926/0 (socket says 172.21.15.26:36926) 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.301+0000 7f95dffff640 1 -- 172.21.15.26:0/1512424170 learned_addr learned my addr 172.21.15.26:0/1512424170 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95e4bc6640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 0x7f95e01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084378779 0 0) 0x7f95e00c28d0 con 0x7f95e00c1aa0 2024-03-06T07:26:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc003520 con 0x7f95e00c1aa0 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435460477 0 0) 0x7f95cc003520 con 0x7f95e00c1aa0 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 msgr2=0x7f95e01529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 0x7f95e01529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 >> v1:172.21.15.117:6789/0 conn(0x7f95e0155a40 legacy=0x7f95e0155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e00c2e90 con 0x7f95e00c1aa0 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f95c8001670 con 0x7f95e00c1aa0 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 >> v1:172.21.15.26:6789/0 conn(0x7f95e00c1aa0 legacy=0x7f95e00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 0x7f95cc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00c2680 con 0x7f95cc004010 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00bb80 con 0x7f95cc004d40 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00bd80 con 0x7f95e014f790 2024-03-06T07:26:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95dffff640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95e4bc6640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 0x7f95cc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95e53c7640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.302+0000 7f95dffff640 1 -- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 msgr2=0x7f95cc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.303+0000 7f95dffff640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e014f790 0x7f95cc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.303+0000 7f95dffff640 1 -- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95cc004d40 msgr2=0x7f95cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.303+0000 7f95dffff640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.303+0000 7f95dffff640 1 -- 172.21.15.26:0/1512424170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc003520 con 0x7f95cc004010 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.303+0000 7f95e53c7640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95dffff640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95cc004800 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c240 tx=0x7f95cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac652e3b9c10c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8007d10 con 0x7f95cc004010 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8030070 con 0x7f95cc004010 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95df7fe640 1 -- 172.21.15.26:0/1512424170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c802cdc0 con 0x7f95cc004010 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95e6e2a640 1 -- 172.21.15.26:0/1512424170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 msgr2=0x7f95cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95e6e2a640 1 --2- 172.21.15.26:0/1512424170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95cc004800 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c240 tx=0x7f95cc005830 comp rx=0 tx=0).stop 2024-03-06T07:26:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95e53c7640 1 -- 172.21.15.26:0/1512424170 reap_dead start 2024-03-06T07:26:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95e6e2a640 1 -- 172.21.15.26:0/1512424170 shutdown_connections 2024-03-06T07:26:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95e6e2a640 1 -- 172.21.15.26:0/1512424170 >> 172.21.15.26:0/1512424170 conn(0x7f95e00c6710 msgr2=0x7f95e00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95e6e2a640 1 -- 172.21.15.26:0/1512424170 shutdown_connections 2024-03-06T07:26:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.304+0000 7f95e6e2a640 1 -- 172.21.15.26:0/1512424170 wait complete. 2024-03-06T07:26:55.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.305+0000 7f95e6e2a640 1 Processor -- start 2024-03-06T07:26:55.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.305+0000 7f95e6e2a640 1 -- start start 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e6e2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95e00bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e6e2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e01509c0 0x7f95e01eb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e6e2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e01eb5d0 0x7f95e01494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e6e2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00c5e30 con 0x7f95cc004010 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e6e2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00c0d60 con 0x7f95e01509c0 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e6e2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00c5fb0 con 0x7f95e01eb5d0 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e4bc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95e00bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e4bc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95e00bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48428/0 (socket says 172.21.15.26:48428) 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.306+0000 7f95e4bc6640 1 -- 172.21.15.26:0/4153933198 learned_addr learned my addr 172.21.15.26:0/4153933198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e53c7640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e01eb5d0 0x7f95e01494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95dffff640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e01509c0 0x7f95e01eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e4bc6640 1 -- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e01eb5d0 msgr2=0x7f95e01494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e4bc6640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e01eb5d0 0x7f95e01494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e4bc6640 1 -- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e01509c0 msgr2=0x7f95e01eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e4bc6640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e01509c0 0x7f95e01eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e4bc6640 1 -- 172.21.15.26:0/4153933198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0150e50 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e4bc6640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95e00bfdb0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b7f0 tx=0x7f95d000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95dd7fa640 1 -- 172.21.15.26:0/4153933198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0018070 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95dd7fa640 1 -- 172.21.15.26:0/4153933198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0010830 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e01ee600 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95dd7fa640 1 -- 172.21.15.26:0/4153933198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0010c00 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.307+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e01eebc0 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.308+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e004f330 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.309+0000 7f95dd7fa640 1 -- 172.21.15.26:0/4153933198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95d001b070 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.309+0000 7f95dd7fa640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95c006b500 0x7f95c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.309+0000 7f95dd7fa640 1 -- 172.21.15.26:0/4153933198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95d008a7f0 con 0x7f95cc004010 2024-03-06T07:26:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.309+0000 7f95dffff640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95c006b500 0x7f95c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.310+0000 7f95dffff640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95c006b500 0x7f95c006d9c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f95c8034000 tx=0x7f95c8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.314+0000 7f95dd7fa640 1 -- 172.21.15.26:0/4153933198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95d00594f0 con 0x7f95cc004010 2024-03-06T07:26:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.497+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95e00c0990 con 0x7f95cc004010 2024-03-06T07:26:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.498+0000 7f95dd7fa640 1 -- 172.21.15.26:0/4153933198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f95d0058df0 con 0x7f95cc004010 2024-03-06T07:26:55.499 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:55.500 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":172,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:53.824659+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:50.446132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:51.147227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95c006b500 msgr2=0x7f95c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95c006b500 0x7f95c006d9c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f95c8034000 tx=0x7f95c8009030 comp rx=0 tx=0).stop 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 msgr2=0x7f95e00bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95e00bfdb0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b7f0 tx=0x7f95d000bcb0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 shutdown_connections 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f95c006b500 0x7f95c006d9c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95e01eb5d0 0x7f95e01494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e01509c0 0x7f95e01eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 --2- 172.21.15.26:0/4153933198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95cc004010 0x7f95e00bfdb0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 >> 172.21.15.26:0/4153933198 conn(0x7f95e00c6710 msgr2=0x7f95e014dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 shutdown_connections 2024-03-06T07:26:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.500+0000 7f95e6e2a640 1 -- 172.21.15.26:0/4153933198 wait complete. 2024-03-06T07:26:55.513 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:26:55.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.616+0000 7f1a9b829640 1 Processor -- start 2024-03-06T07:26:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a9b829640 1 -- start start 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a9b829640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1a9414f360 0x7f1a94155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a9b829640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 0x7f1a94158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a9b829640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a940c73d0 con 0x7f1a940c88c0 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a9b829640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a940c8dd0 con 0x7f1a9414f360 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a9b829640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940be2e0 con 0x7f1a94158b60 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a98dc4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1a940c88c0 0x7f1a940c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36940/0 (socket says 172.21.15.26:36940) 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.617+0000 7f1a98dc4640 1 -- 172.21.15.26:0/3040392844 learned_addr learned my addr 172.21.15.26:0/3040392844 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a995c5640 1 --2- 172.21.15.26:0/3040392844 >> v2:172.21.15.117:3300/0 conn(0x7f1a9414f360 0x7f1a94155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 0x7f1a94158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128347488 0 0) 0x7f1a940c73d0 con 0x7f1a940c88c0 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a80003520 con 0x7f1a940c88c0 2024-03-06T07:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 256485228 0 0) 0x7f1a80003520 con 0x7f1a940c88c0 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 msgr2=0x7f1a94158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 0x7f1a94158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 >> v2:172.21.15.117:3300/0 conn(0x7f1a9414f360 msgr2=0x7f1a94155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 --2- 172.21.15.26:0/3040392844 >> v2:172.21.15.117:3300/0 conn(0x7f1a9414f360 0x7f1a94155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940c7670 con 0x7f1a940c88c0 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f1a84001670 con 0x7f1a940c88c0 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 >> v1:172.21.15.26:6789/0 conn(0x7f1a940c88c0 legacy=0x7f1a940c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800039b0 0x7f1a80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 0x7f1a80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940be2e0 con 0x7f1a80004d40 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8000b9c0 con 0x7f1a800039b0 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8000bb70 con 0x7f1a94158b60 2024-03-06T07:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a995c5640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800039b0 0x7f1a80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.618+0000 7f1a98dc4640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 0x7f1a80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a99dc6640 1 -- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 msgr2=0x7f1a80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a94158b60 0x7f1a80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a99dc6640 1 -- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800039b0 msgr2=0x7f1a80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800039b0 0x7f1a80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a99dc6640 1 -- 172.21.15.26:0/3040392844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80003520 con 0x7f1a80004d40 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009fc0 tx=0x7f1a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276b156230e6000f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c039070 con 0x7f1a80004d40 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c02cc20 con 0x7f1a80004d40 2024-03-06T07:26:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.619+0000 7f1a8bfff640 1 -- 172.21.15.26:0/3040392844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c009c10 con 0x7f1a80004d40 2024-03-06T07:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.620+0000 7f1a9b829640 1 -- 172.21.15.26:0/3040392844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80004d40 msgr2=0x7f1a800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.620+0000 7f1a9b829640 1 --2- 172.21.15.26:0/3040392844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009fc0 tx=0x7f1a80005830 comp rx=0 tx=0).stop 2024-03-06T07:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.620+0000 7f1a99dc6640 1 -- 172.21.15.26:0/3040392844 reap_dead start 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.620+0000 7f1a9b829640 1 -- 172.21.15.26:0/3040392844 shutdown_connections 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.620+0000 7f1a9b829640 1 -- 172.21.15.26:0/3040392844 >> 172.21.15.26:0/3040392844 conn(0x7f1a940ac570 msgr2=0x7f1a940ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.620+0000 7f1a9b829640 1 -- 172.21.15.26:0/3040392844 shutdown_connections 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.620+0000 7f1a9b829640 1 -- 172.21.15.26:0/3040392844 wait complete. 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.621+0000 7f1a9b829640 1 Processor -- start 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a9b829640 1 -- start start 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a9b829640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a941ee330 0x7f1a941ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a9b829640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a940af8e0 0x7f1a9414bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a9b829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9414c2e0 0x7f1a941ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a9b829640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c5a50 con 0x7f1a9414c2e0 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a9b829640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c7c30 con 0x7f1a941ee330 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a9b829640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c5bd0 con 0x7f1a940af8e0 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a99dc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9414c2e0 0x7f1a941ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a99dc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9414c2e0 0x7f1a941ecc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48440/0 (socket says 172.21.15.26:48440) 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a99dc6640 1 -- 172.21.15.26:0/335036581 learned_addr learned my addr 172.21.15.26:0/335036581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a995c5640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a941ee330 0x7f1a941ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.622+0000 7f1a98dc4640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a940af8e0 0x7f1a9414bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a99dc6640 1 -- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a940af8e0 msgr2=0x7f1a9414bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a940af8e0 0x7f1a9414bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a99dc6640 1 -- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a941ee330 msgr2=0x7f1a941ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a941ee330 0x7f1a941ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a99dc6640 1 -- 172.21.15.26:0/335036581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940afde0 con 0x7f1a9414c2e0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a99dc6640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9414c2e0 0x7f1a941ecc40 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e200 tx=0x7f1a8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a89ffb640 1 -- 172.21.15.26:0/335036581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c039070 con 0x7f1a9414c2e0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a89ffb640 1 -- 172.21.15.26:0/335036581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c044070 con 0x7f1a9414c2e0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a89ffb640 1 -- 172.21.15.26:0/335036581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c009c10 con 0x7f1a9414c2e0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a941ec160 con 0x7f1a9414c2e0 2024-03-06T07:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.623+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a941ec720 con 0x7f1a9414c2e0 2024-03-06T07:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.624+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a941ec920 con 0x7f1a9414c2e0 2024-03-06T07:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.625+0000 7f1a89ffb640 1 -- 172.21.15.26:0/335036581 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a8c005d10 con 0x7f1a9414c2e0 2024-03-06T07:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.625+0000 7f1a89ffb640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a7806b500 0x7f1a7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.625+0000 7f1a89ffb640 1 -- 172.21.15.26:0/335036581 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a8c0ac200 con 0x7f1a9414c2e0 2024-03-06T07:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.625+0000 7f1a995c5640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a7806b500 0x7f1a7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.626+0000 7f1a995c5640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a7806b500 0x7f1a7806d9c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009c40 tx=0x7f1a90009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.634+0000 7f1a89ffb640 1 -- 172.21.15.26:0/335036581 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1a941ec920 con 0x7f1a9414c2e0 2024-03-06T07:26:55.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.862+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a9404f330 con 0x7f1a7806b500 2024-03-06T07:26:55.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.868+0000 7f1a89ffb640 1 -- 172.21.15.26:0/335036581 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252566 (secure 0 0 0) 0x7f1a9404f330 con 0x7f1a7806b500 2024-03-06T07:26:55.873 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:26:55.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a7806b500 msgr2=0x7f1a7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a7806b500 0x7f1a7806d9c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009c40 tx=0x7f1a90009190 comp rx=0 tx=0).stop 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9414c2e0 msgr2=0x7f1a941ecc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9414c2e0 0x7f1a941ecc40 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e200 tx=0x7f1a8c009750 comp rx=0 tx=0).stop 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 shutdown_connections 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a7806b500 0x7f1a7806d9c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9414c2e0 0x7f1a941ecc40 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a940af8e0 0x7f1a9414bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 --2- 172.21.15.26:0/335036581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a941ee330 0x7f1a941ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.871+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 >> 172.21.15.26:0/335036581 conn(0x7f1a940ac570 msgr2=0x7f1a940c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.872+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 shutdown_connections 2024-03-06T07:26:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:55.872+0000 7f1a9b829640 1 -- 172.21.15.26:0/335036581 wait complete. 2024-03-06T07:26:55.884 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-03-06T07:26:55.116787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266656,"kb_used_data":17184,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708064,"statfs":{"total":773094113280,"available":772821057536,"internally_reserved":0,"allocated":17596416,"data_stored":9671940,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.306830"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112099+0000","last_change":"2024-03-06T07:26:52.839051+0000","last_active":"2024-03-06T07:26:54.112099+0000","last_peered":"2024-03-06T07:26:54.112099+0000","last_clean":"2024-03-06T07:26:54.112099+0000","last_became_active":"2024-03-06T07:26:52.838710+0000","last_became_peered":"2024-03-06T07:26:52.838710+0000","last_unstale":"2024-03-06T07:26:54.112099+0000","last_undegraded":"2024-03-06T07:26:54.112099+0000","last_fullsized":"2024-03-06T07:26:54.112099+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:37.889646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831250+0000","last_change":"2024-03-06T07:26:52.826808+0000","last_active":"2024-03-06T07:26:53.831250+0000","last_peered":"2024-03-06T07:26:53.831250+0000","last_clean":"2024-03-06T07:26:53.831250+0000","last_became_active":"2024-03-06T07:26:52.826508+0000","last_became_peered":"2024-03-06T07:26:52.826508+0000","last_unstale":"2024-03-06T07:26:53.831250+0000","last_undegraded":"2024-03-06T07:26:53.831250+0000","last_fullsized":"2024-03-06T07:26:53.831250+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:11:06.620959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112032+0000","last_change":"2024-03-06T07:26:52.836625+0000","last_active":"2024-03-06T07:26:54.112032+0000","last_peered":"2024-03-06T07:26:54.112032+0000","last_clean":"2024-03-06T07:26:54.112032+0000","last_became_active":"2024-03-06T07:26:52.836349+0000","last_became_peered":"2024-03-06T07:26:52.836349+0000","last_unstale":"2024-03-06T07:26:54.112032+0000","last_undegraded":"2024-03-06T07:26:54.112032+0000","last_fullsized":"2024-03-06T07:26:54.112032+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:50.525698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112536+0000","last_change":"2024-03-06T07:26:52.840797+0000","last_active":"2024-03-06T07:26:54.112536+0000","last_peered":"2024-03-06T07:26:54.112536+0000","last_clean":"2024-03-06T07:26:54.112536+0000","last_became_active":"2024-03-06T07:26:52.840539+0000","last_became_peered":"2024-03-06T07:26:52.840539+0000","last_unstale":"2024-03-06T07:26:54.112536+0000","last_undegraded":"2024-03-06T07:26:54.112536+0000","last_fullsized":"2024-03-06T07:26:54.112536+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:59.872043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111392+0000","last_change":"2024-03-06T07:26:52.845997+0000","last_active":"2024-03-06T07:26:54.111392+0000","last_peered":"2024-03-06T07:26:54.111392+0000","last_clean":"2024-03-06T07:26:54.111392+0000","last_became_active":"2024-03-06T07:26:52.845746+0000","last_became_peered":"2024-03-06T07:26:52.845746+0000","last_unstale":"2024-03-06T07:26:54.111392+0000","last_undegraded":"2024-03-06T07:26:54.111392+0000","last_fullsized":"2024-03-06T07:26:54.111392+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:14.477763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111793+0000","last_change":"2024-03-06T07:26:52.841694+0000","last_active":"2024-03-06T07:26:54.111793+0000","last_peered":"2024-03-06T07:26:54.111793+0000","last_clean":"2024-03-06T07:26:54.111793+0000","last_became_active":"2024-03-06T07:26:52.841393+0000","last_became_peered":"2024-03-06T07:26:52.841393+0000","last_unstale":"2024-03-06T07:26:54.111793+0000","last_undegraded":"2024-03-06T07:26:54.111793+0000","last_fullsized":"2024-03-06T07:26:54.111793+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:32.948522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111239+0000","last_change":"2024-03-06T07:26:52.845949+0000","last_active":"2024-03-06T07:26:54.111239+0000","last_peered":"2024-03-06T07:26:54.111239+0000","last_clean":"2024-03-06T07:26:54.111239+0000","last_became_active":"2024-03-06T07:26:52.845692+0000","last_became_peered":"2024-03-06T07:26:52.845692+0000","last_unstale":"2024-03-06T07:26:54.111239+0000","last_undegraded":"2024-03-06T07:26:54.111239+0000","last_fullsized":"2024-03-06T07:26:54.111239+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:44.646930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112989+0000","last_change":"2024-03-06T07:26:52.840961+0000","last_active":"2024-03-06T07:26:54.112989+0000","last_peered":"2024-03-06T07:26:54.112989+0000","last_clean":"2024-03-06T07:26:54.112989+0000","last_became_active":"2024-03-06T07:26:52.837551+0000","last_became_peered":"2024-03-06T07:26:52.837551+0000","last_unstale":"2024-03-06T07:26:54.112989+0000","last_undegraded":"2024-03-06T07:26:54.112989+0000","last_fullsized":"2024-03-06T07:26:54.112989+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:42:14.924497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112350+0000","last_change":"2024-03-06T07:26:52.835924+0000","last_active":"2024-03-06T07:26:54.112350+0000","last_peered":"2024-03-06T07:26:54.112350+0000","last_clean":"2024-03-06T07:26:54.112350+0000","last_became_active":"2024-03-06T07:26:52.835547+0000","last_became_peered":"2024-03-06T07:26:52.835547+0000","last_unstale":"2024-03-06T07:26:54.112350+0000","last_undegraded":"2024-03-06T07:26:54.112350+0000","last_fullsized":"2024-03-06T07:26:54.112350+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:57.620289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112005+0000","last_change":"2024-03-06T07:26:52.836820+0000","last_active":"2024-03-06T07:26:54.112005+0000","last_peered":"2024-03-06T07:26:54.112005+0000","last_clean":"2024-03-06T07:26:54.112005+0000","last_became_active":"2024-03-06T07:26:52.836496+0000","last_became_peered":"2024-03-06T07:26:52.836496+0000","last_unstale":"2024-03-06T07:26:54.112005+0000","last_undegraded":"2024-03-06T07:26:54.112005+0000","last_fullsized":"2024-03-06T07:26:54.112005+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:37.494197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112172+0000","last_change":"2024-03-06T07:26:52.839475+0000","last_active":"2024-03-06T07:26:54.112172+0000","last_peered":"2024-03-06T07:26:54.112172+0000","last_clean":"2024-03-06T07:26:54.112172+0000","last_became_active":"2024-03-06T07:26:52.839019+0000","last_became_peered":"2024-03-06T07:26:52.839019+0000","last_unstale":"2024-03-06T07:26:54.112172+0000","last_undegraded":"2024-03-06T07:26:54.112172+0000","last_fullsized":"2024-03-06T07:26:54.112172+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:59.872043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112104+0000","last_change":"2024-03-06T07:26:52.843451+0000","last_active":"2024-03-06T07:26:54.112104+0000","last_peered":"2024-03-06T07:26:54.112104+0000","last_clean":"2024-03-06T07:26:54.112104+0000","last_became_active":"2024-03-06T07:26:52.843177+0000","last_became_peered":"2024-03-06T07:26:52.843177+0000","last_unstale":"2024-03-06T07:26:54.112104+0000","last_undegraded":"2024-03-06T07:26:54.112104+0000","last_fullsized":"2024-03-06T07:26:54.112104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:15.583575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112660+0000","last_change":"2024-03-06T07:26:52.839732+0000","last_active":"2024-03-06T07:26:54.112660+0000","last_peered":"2024-03-06T07:26:54.112660+0000","last_clean":"2024-03-06T07:26:54.112660+0000","last_became_active":"2024-03-06T07:26:52.839470+0000","last_became_peered":"2024-03-06T07:26:52.839470+0000","last_unstale":"2024-03-06T07:26:54.112660+0000","last_undegraded":"2024-03-06T07:26:54.112660+0000","last_fullsized":"2024-03-06T07:26:54.112660+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:06.898636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111702+0000","last_change":"2024-03-06T07:26:52.844734+0000","last_active":"2024-03-06T07:26:54.111702+0000","last_peered":"2024-03-06T07:26:54.111702+0000","last_clean":"2024-03-06T07:26:54.111702+0000","last_became_active":"2024-03-06T07:26:52.843191+0000","last_became_peered":"2024-03-06T07:26:52.843191+0000","last_unstale":"2024-03-06T07:26:54.111702+0000","last_undegraded":"2024-03-06T07:26:54.111702+0000","last_fullsized":"2024-03-06T07:26:54.111702+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:20.261482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833742+0000","last_change":"2024-03-06T07:26:52.837625+0000","last_active":"2024-03-06T07:26:53.833742+0000","last_peered":"2024-03-06T07:26:53.833742+0000","last_clean":"2024-03-06T07:26:53.833742+0000","last_became_active":"2024-03-06T07:26:52.837363+0000","last_became_peered":"2024-03-06T07:26:52.837363+0000","last_unstale":"2024-03-06T07:26:53.833742+0000","last_undegraded":"2024-03-06T07:26:53.833742+0000","last_fullsized":"2024-03-06T07:26:53.833742+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:41.074583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111455+0000","last_change":"2024-03-06T07:26:52.835458+0000","last_active":"2024-03-06T07:26:54.111455+0000","last_peered":"2024-03-06T07:26:54.111455+0000","last_clean":"2024-03-06T07:26:54.111455+0000","last_became_active":"2024-03-06T07:26:52.835095+0000","last_became_peered":"2024-03-06T07:26:52.835095+0000","last_unstale":"2024-03-06T07:26:54.111455+0000","last_undegraded":"2024-03-06T07:26:54.111455+0000","last_fullsized":"2024-03-06T07:26:54.111455+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:32.948522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112475+0000","last_change":"2024-03-06T07:26:52.833356+0000","last_active":"2024-03-06T07:26:54.112475+0000","last_peered":"2024-03-06T07:26:54.112475+0000","last_clean":"2024-03-06T07:26:54.112475+0000","last_became_active":"2024-03-06T07:26:52.833031+0000","last_became_peered":"2024-03-06T07:26:52.833031+0000","last_unstale":"2024-03-06T07:26:54.112475+0000","last_undegraded":"2024-03-06T07:26:54.112475+0000","last_fullsized":"2024-03-06T07:26:54.112475+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:13.271264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111913+0000","last_change":"2024-03-06T07:26:52.836979+0000","last_active":"2024-03-06T07:26:54.111913+0000","last_peered":"2024-03-06T07:26:54.111913+0000","last_clean":"2024-03-06T07:26:54.111913+0000","last_became_active":"2024-03-06T07:26:52.836693+0000","last_became_peered":"2024-03-06T07:26:52.836693+0000","last_unstale":"2024-03-06T07:26:54.111913+0000","last_undegraded":"2024-03-06T07:26:54.111913+0000","last_fullsized":"2024-03-06T07:26:54.111913+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:11.712562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112647+0000","last_change":"2024-03-06T07:26:52.843510+0000","last_active":"2024-03-06T07:26:54.112647+0000","last_peered":"2024-03-06T07:26:54.112647+0000","last_clean":"2024-03-06T07:26:54.112647+0000","last_became_active":"2024-03-06T07:26:52.843135+0000","last_became_peered":"2024-03-06T07:26:52.843135+0000","last_unstale":"2024-03-06T07:26:54.112647+0000","last_undegraded":"2024-03-06T07:26:54.112647+0000","last_fullsized":"2024-03-06T07:26:54.112647+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:26.988672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111531+0000","last_change":"2024-03-06T07:26:52.847561+0000","last_active":"2024-03-06T07:26:54.111531+0000","last_peered":"2024-03-06T07:26:54.111531+0000","last_clean":"2024-03-06T07:26:54.111531+0000","last_became_active":"2024-03-06T07:26:52.846304+0000","last_became_peered":"2024-03-06T07:26:52.846304+0000","last_unstale":"2024-03-06T07:26:54.111531+0000","last_undegraded":"2024-03-06T07:26:54.111531+0000","last_fullsized":"2024-03-06T07:26:54.111531+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:13.127494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111535+0000","last_change":"2024-03-06T07:26:52.842836+0000","last_active":"2024-03-06T07:26:54.111535+0000","last_peered":"2024-03-06T07:26:54.111535+0000","last_clean":"2024-03-06T07:26:54.111535+0000","last_became_active":"2024-03-06T07:26:52.842536+0000","last_became_peered":"2024-03-06T07:26:52.842536+0000","last_unstale":"2024-03-06T07:26:54.111535+0000","last_undegraded":"2024-03-06T07:26:54.111535+0000","last_fullsized":"2024-03-06T07:26:54.111535+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:53.447704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111870+0000","last_change":"2024-03-06T07:26:52.845253+0000","last_active":"2024-03-06T07:26:54.111870+0000","last_peered":"2024-03-06T07:26:54.111870+0000","last_clean":"2024-03-06T07:26:54.111870+0000","last_became_active":"2024-03-06T07:26:52.844952+0000","last_became_peered":"2024-03-06T07:26:52.844952+0000","last_unstale":"2024-03-06T07:26:54.111870+0000","last_undegraded":"2024-03-06T07:26:54.111870+0000","last_fullsized":"2024-03-06T07:26:54.111870+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:27.316431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830921+0000","last_change":"2024-03-06T07:26:52.842770+0000","last_active":"2024-03-06T07:26:53.830921+0000","last_peered":"2024-03-06T07:26:53.830921+0000","last_clean":"2024-03-06T07:26:53.830921+0000","last_became_active":"2024-03-06T07:26:52.842460+0000","last_became_peered":"2024-03-06T07:26:52.842460+0000","last_unstale":"2024-03-06T07:26:53.830921+0000","last_undegraded":"2024-03-06T07:26:53.830921+0000","last_fullsized":"2024-03-06T07:26:53.830921+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:48:55.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.110827+0000","last_change":"2024-03-06T07:26:52.848322+0000","last_active":"2024-03-06T07:26:54.110827+0000","last_peered":"2024-03-06T07:26:54.110827+0000","last_clean":"2024-03-06T07:26:54.110827+0000","last_became_active":"2024-03-06T07:26:52.847974+0000","last_became_peered":"2024-03-06T07:26:52.847974+0000","last_unstale":"2024-03-06T07:26:54.110827+0000","last_undegraded":"2024-03-06T07:26:54.110827+0000","last_fullsized":"2024-03-06T07:26:54.110827+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:13.621879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112587+0000","last_change":"2024-03-06T07:26:52.837162+0000","last_active":"2024-03-06T07:26:54.112587+0000","last_peered":"2024-03-06T07:26:54.112587+0000","last_clean":"2024-03-06T07:26:54.112587+0000","last_became_active":"2024-03-06T07:26:52.836816+0000","last_became_peered":"2024-03-06T07:26:52.836816+0000","last_unstale":"2024-03-06T07:26:54.112587+0000","last_undegraded":"2024-03-06T07:26:54.112587+0000","last_fullsized":"2024-03-06T07:26:54.112587+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:19.280786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111754+0000","last_change":"2024-03-06T07:26:52.846391+0000","last_active":"2024-03-06T07:26:54.111754+0000","last_peered":"2024-03-06T07:26:54.111754+0000","last_clean":"2024-03-06T07:26:54.111754+0000","last_became_active":"2024-03-06T07:26:52.846040+0000","last_became_peered":"2024-03-06T07:26:52.846040+0000","last_unstale":"2024-03-06T07:26:54.111754+0000","last_undegraded":"2024-03-06T07:26:54.111754+0000","last_fullsized":"2024-03-06T07:26:54.111754+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:46.086368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112331+0000","last_change":"2024-03-06T07:26:52.839034+0000","last_active":"2024-03-06T07:26:54.112331+0000","last_peered":"2024-03-06T07:26:54.112331+0000","last_clean":"2024-03-06T07:26:54.112331+0000","last_became_active":"2024-03-06T07:26:52.838335+0000","last_became_peered":"2024-03-06T07:26:52.838335+0000","last_unstale":"2024-03-06T07:26:54.112331+0000","last_undegraded":"2024-03-06T07:26:54.112331+0000","last_fullsized":"2024-03-06T07:26:54.112331+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:38.074300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112303+0000","last_change":"2024-03-06T07:26:52.847424+0000","last_active":"2024-03-06T07:26:54.112303+0000","last_peered":"2024-03-06T07:26:54.112303+0000","last_clean":"2024-03-06T07:26:54.112303+0000","last_became_active":"2024-03-06T07:26:52.846812+0000","last_became_peered":"2024-03-06T07:26:52.846812+0000","last_unstale":"2024-03-06T07:26:54.112303+0000","last_undegraded":"2024-03-06T07:26:54.112303+0000","last_fullsized":"2024-03-06T07:26:54.112303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:27.072412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111771+0000","last_change":"2024-03-06T07:26:52.832234+0000","last_active":"2024-03-06T07:26:54.111771+0000","last_peered":"2024-03-06T07:26:54.111771+0000","last_clean":"2024-03-06T07:26:54.111771+0000","last_became_active":"2024-03-06T07:26:52.831908+0000","last_became_peered":"2024-03-06T07:26:52.831908+0000","last_unstale":"2024-03-06T07:26:54.111771+0000","last_undegraded":"2024-03-06T07:26:54.111771+0000","last_fullsized":"2024-03-06T07:26:54.111771+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:49.943342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833336+0000","last_change":"2024-03-06T07:26:52.832207+0000","last_active":"2024-03-06T07:26:53.833336+0000","last_peered":"2024-03-06T07:26:53.833336+0000","last_clean":"2024-03-06T07:26:53.833336+0000","last_became_active":"2024-03-06T07:26:52.831879+0000","last_became_peered":"2024-03-06T07:26:52.831879+0000","last_unstale":"2024-03-06T07:26:53.833336+0000","last_undegraded":"2024-03-06T07:26:53.833336+0000","last_fullsized":"2024-03-06T07:26:53.833336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:17.300528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111558+0000","last_change":"2024-03-06T07:26:52.832027+0000","last_active":"2024-03-06T07:26:54.111558+0000","last_peered":"2024-03-06T07:26:54.111558+0000","last_clean":"2024-03-06T07:26:54.111558+0000","last_became_active":"2024-03-06T07:26:52.831754+0000","last_became_peered":"2024-03-06T07:26:52.831754+0000","last_unstale":"2024-03-06T07:26:54.111558+0000","last_undegraded":"2024-03-06T07:26:54.111558+0000","last_fullsized":"2024-03-06T07:26:54.111558+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:21:19.273834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111392+0000","last_change":"2024-03-06T07:26:52.832539+0000","last_active":"2024-03-06T07:26:54.111392+0000","last_peered":"2024-03-06T07:26:54.111392+0000","last_clean":"2024-03-06T07:26:54.111392+0000","last_became_active":"2024-03-06T07:26:52.832151+0000","last_became_peered":"2024-03-06T07:26:52.832151+0000","last_unstale":"2024-03-06T07:26:54.111392+0000","last_undegraded":"2024-03-06T07:26:54.111392+0000","last_fullsized":"2024-03-06T07:26:54.111392+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:03.380633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830892+0000","last_change":"2024-03-06T07:26:52.843827+0000","last_active":"2024-03-06T07:26:53.830892+0000","last_peered":"2024-03-06T07:26:53.830892+0000","last_clean":"2024-03-06T07:26:53.830892+0000","last_became_active":"2024-03-06T07:26:52.843493+0000","last_became_peered":"2024-03-06T07:26:52.843493+0000","last_unstale":"2024-03-06T07:26:53.830892+0000","last_undegraded":"2024-03-06T07:26:53.830892+0000","last_fullsized":"2024-03-06T07:26:53.830892+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:31.688070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"171'4","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833661+0000","last_change":"2024-03-06T07:26:51.825945+0000","last_active":"2024-03-06T07:26:53.833661+0000","last_peered":"2024-03-06T07:26:53.833661+0000","last_clean":"2024-03-06T07:26:53.833661+0000","last_became_active":"2024-03-06T07:26:51.825667+0000","last_became_peered":"2024-03-06T07:26:51.825667+0000","last_unstale":"2024-03-06T07:26:53.833661+0000","last_undegraded":"2024-03-06T07:26:53.833661+0000","last_fullsized":"2024-03-06T07:26:53.833661+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:09.377291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111462+0000","last_change":"2024-03-06T07:26:52.842594+0000","last_active":"2024-03-06T07:26:54.111462+0000","last_peered":"2024-03-06T07:26:54.111462+0000","last_clean":"2024-03-06T07:26:54.111462+0000","last_became_active":"2024-03-06T07:26:52.842136+0000","last_became_peered":"2024-03-06T07:26:52.842136+0000","last_unstale":"2024-03-06T07:26:54.111462+0000","last_undegraded":"2024-03-06T07:26:54.111462+0000","last_fullsized":"2024-03-06T07:26:54.111462+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:10.157118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111836+0000","last_change":"2024-03-06T07:26:51.834206+0000","last_active":"2024-03-06T07:26:54.111836+0000","last_peered":"2024-03-06T07:26:54.111836+0000","last_clean":"2024-03-06T07:26:54.111836+0000","last_became_active":"2024-03-06T07:26:51.833890+0000","last_became_peered":"2024-03-06T07:26:51.833890+0000","last_unstale":"2024-03-06T07:26:54.111836+0000","last_undegraded":"2024-03-06T07:26:54.111836+0000","last_fullsized":"2024-03-06T07:26:54.111836+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:00.234438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112054+0000","last_change":"2024-03-06T07:26:52.845024+0000","last_active":"2024-03-06T07:26:54.112054+0000","last_peered":"2024-03-06T07:26:54.112054+0000","last_clean":"2024-03-06T07:26:54.112054+0000","last_became_active":"2024-03-06T07:26:52.843567+0000","last_became_peered":"2024-03-06T07:26:52.843567+0000","last_unstale":"2024-03-06T07:26:54.112054+0000","last_undegraded":"2024-03-06T07:26:54.112054+0000","last_fullsized":"2024-03-06T07:26:54.112054+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:48.575265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830915+0000","last_change":"2024-03-06T07:26:51.830196+0000","last_active":"2024-03-06T07:26:53.830915+0000","last_peered":"2024-03-06T07:26:53.830915+0000","last_clean":"2024-03-06T07:26:53.830915+0000","last_became_active":"2024-03-06T07:26:51.829892+0000","last_became_peered":"2024-03-06T07:26:51.829892+0000","last_unstale":"2024-03-06T07:26:53.830915+0000","last_undegraded":"2024-03-06T07:26:53.830915+0000","last_fullsized":"2024-03-06T07:26:53.830915+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:12.263778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111525+0000","last_change":"2024-03-06T07:26:52.847259+0000","last_active":"2024-03-06T07:26:54.111525+0000","last_peered":"2024-03-06T07:26:54.111525+0000","last_clean":"2024-03-06T07:26:54.111525+0000","last_became_active":"2024-03-06T07:26:52.846983+0000","last_became_peered":"2024-03-06T07:26:52.846983+0000","last_unstale":"2024-03-06T07:26:54.111525+0000","last_undegraded":"2024-03-06T07:26:54.111525+0000","last_fullsized":"2024-03-06T07:26:54.111525+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:35:02.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112199+0000","last_change":"2024-03-06T07:26:51.831054+0000","last_active":"2024-03-06T07:26:54.112199+0000","last_peered":"2024-03-06T07:26:54.112199+0000","last_clean":"2024-03-06T07:26:54.112199+0000","last_became_active":"2024-03-06T07:26:51.830791+0000","last_became_peered":"2024-03-06T07:26:51.830791+0000","last_unstale":"2024-03-06T07:26:54.112199+0000","last_undegraded":"2024-03-06T07:26:54.112199+0000","last_fullsized":"2024-03-06T07:26:54.112199+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:58.864556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833659+0000","last_change":"2024-03-06T07:26:52.840186+0000","last_active":"2024-03-06T07:26:53.833659+0000","last_peered":"2024-03-06T07:26:53.833659+0000","last_clean":"2024-03-06T07:26:53.833659+0000","last_became_active":"2024-03-06T07:26:52.839100+0000","last_became_peered":"2024-03-06T07:26:52.839100+0000","last_unstale":"2024-03-06T07:26:53.833659+0000","last_undegraded":"2024-03-06T07:26:53.833659+0000","last_fullsized":"2024-03-06T07:26:53.833659+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:18:09.905056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112480+0000","last_change":"2024-03-06T07:26:51.829167+0000","last_active":"2024-03-06T07:26:54.112480+0000","last_peered":"2024-03-06T07:26:54.112480+0000","last_clean":"2024-03-06T07:26:54.112480+0000","last_became_active":"2024-03-06T07:26:51.828901+0000","last_became_peered":"2024-03-06T07:26:51.828901+0000","last_unstale":"2024-03-06T07:26:54.112480+0000","last_undegraded":"2024-03-06T07:26:54.112480+0000","last_fullsized":"2024-03-06T07:26:54.112480+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:52.402709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833522+0000","last_change":"2024-03-06T07:26:52.849185+0000","last_active":"2024-03-06T07:26:53.833522+0000","last_peered":"2024-03-06T07:26:53.833522+0000","last_clean":"2024-03-06T07:26:53.833522+0000","last_became_active":"2024-03-06T07:26:52.848858+0000","last_became_peered":"2024-03-06T07:26:52.848858+0000","last_unstale":"2024-03-06T07:26:53.833522+0000","last_undegraded":"2024-03-06T07:26:53.833522+0000","last_fullsized":"2024-03-06T07:26:53.833522+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:47.798514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111532+0000","last_change":"2024-03-06T07:26:51.828224+0000","last_active":"2024-03-06T07:26:54.111532+0000","last_peered":"2024-03-06T07:26:54.111532+0000","last_clean":"2024-03-06T07:26:54.111532+0000","last_became_active":"2024-03-06T07:26:51.827914+0000","last_became_peered":"2024-03-06T07:26:51.827914+0000","last_unstale":"2024-03-06T07:26:54.111532+0000","last_undegraded":"2024-03-06T07:26:54.111532+0000","last_fullsized":"2024-03-06T07:26:54.111532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:48:24.153761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112597+0000","last_change":"2024-03-06T07:26:52.829714+0000","last_active":"2024-03-06T07:26:54.112597+0000","last_peered":"2024-03-06T07:26:54.112597+0000","last_clean":"2024-03-06T07:26:54.112597+0000","last_became_active":"2024-03-06T07:26:52.829393+0000","last_became_peered":"2024-03-06T07:26:52.829393+0000","last_unstale":"2024-03-06T07:26:54.112597+0000","last_undegraded":"2024-03-06T07:26:54.112597+0000","last_fullsized":"2024-03-06T07:26:54.112597+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:15:29.095588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111548+0000","last_change":"2024-03-06T07:26:51.829143+0000","last_active":"2024-03-06T07:26:54.111548+0000","last_peered":"2024-03-06T07:26:54.111548+0000","last_clean":"2024-03-06T07:26:54.111548+0000","last_became_active":"2024-03-06T07:26:51.828874+0000","last_became_peered":"2024-03-06T07:26:51.828874+0000","last_unstale":"2024-03-06T07:26:54.111548+0000","last_undegraded":"2024-03-06T07:26:54.111548+0000","last_fullsized":"2024-03-06T07:26:54.111548+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:52.402709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833339+0000","last_change":"2024-03-06T07:26:52.841104+0000","last_active":"2024-03-06T07:26:53.833339+0000","last_peered":"2024-03-06T07:26:53.833339+0000","last_clean":"2024-03-06T07:26:53.833339+0000","last_became_active":"2024-03-06T07:26:52.840802+0000","last_became_peered":"2024-03-06T07:26:52.840802+0000","last_unstale":"2024-03-06T07:26:53.833339+0000","last_undegraded":"2024-03-06T07:26:53.833339+0000","last_fullsized":"2024-03-06T07:26:53.833339+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:51:20.576160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111329+0000","last_change":"2024-03-06T07:26:51.829319+0000","last_active":"2024-03-06T07:26:54.111329+0000","last_peered":"2024-03-06T07:26:54.111329+0000","last_clean":"2024-03-06T07:26:54.111329+0000","last_became_active":"2024-03-06T07:26:51.829075+0000","last_became_peered":"2024-03-06T07:26:51.829075+0000","last_unstale":"2024-03-06T07:26:54.111329+0000","last_undegraded":"2024-03-06T07:26:54.111329+0000","last_fullsized":"2024-03-06T07:26:54.111329+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:40.296510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111431+0000","last_change":"2024-03-06T07:26:52.844532+0000","last_active":"2024-03-06T07:26:54.111431+0000","last_peered":"2024-03-06T07:26:54.111431+0000","last_clean":"2024-03-06T07:26:54.111431+0000","last_became_active":"2024-03-06T07:26:52.844250+0000","last_became_peered":"2024-03-06T07:26:52.844250+0000","last_unstale":"2024-03-06T07:26:54.111431+0000","last_undegraded":"2024-03-06T07:26:54.111431+0000","last_fullsized":"2024-03-06T07:26:54.111431+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:21:19.273834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112364+0000","last_change":"2024-03-06T07:26:51.836021+0000","last_active":"2024-03-06T07:26:54.112364+0000","last_peered":"2024-03-06T07:26:54.112364+0000","last_clean":"2024-03-06T07:26:54.112364+0000","last_became_active":"2024-03-06T07:26:51.835563+0000","last_became_peered":"2024-03-06T07:26:51.835563+0000","last_unstale":"2024-03-06T07:26:54.112364+0000","last_undegraded":"2024-03-06T07:26:54.112364+0000","last_fullsized":"2024-03-06T07:26:54.112364+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:29.287753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111445+0000","last_change":"2024-03-06T07:26:52.835859+0000","last_active":"2024-03-06T07:26:54.111445+0000","last_peered":"2024-03-06T07:26:54.111445+0000","last_clean":"2024-03-06T07:26:54.111445+0000","last_became_active":"2024-03-06T07:26:52.835397+0000","last_became_peered":"2024-03-06T07:26:52.835397+0000","last_unstale":"2024-03-06T07:26:54.111445+0000","last_undegraded":"2024-03-06T07:26:54.111445+0000","last_fullsized":"2024-03-06T07:26:54.111445+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:00.646800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"171'3","reported_seq":17,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112433+0000","last_change":"2024-03-06T07:26:51.828522+0000","last_active":"2024-03-06T07:26:54.112433+0000","last_peered":"2024-03-06T07:26:54.112433+0000","last_clean":"2024-03-06T07:26:54.112433+0000","last_became_active":"2024-03-06T07:26:51.828200+0000","last_became_peered":"2024-03-06T07:26:51.828200+0000","last_unstale":"2024-03-06T07:26:54.112433+0000","last_undegraded":"2024-03-06T07:26:54.112433+0000","last_fullsized":"2024-03-06T07:26:54.112433+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:02.146936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111667+0000","last_change":"2024-03-06T07:26:52.848134+0000","last_active":"2024-03-06T07:26:54.111667+0000","last_peered":"2024-03-06T07:26:54.111667+0000","last_clean":"2024-03-06T07:26:54.111667+0000","last_became_active":"2024-03-06T07:26:52.847615+0000","last_became_peered":"2024-03-06T07:26:52.847615+0000","last_unstale":"2024-03-06T07:26:54.111667+0000","last_undegraded":"2024-03-06T07:26:54.111667+0000","last_fullsized":"2024-03-06T07:26:54.111667+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:41.138418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"171'6","reported_seq":20,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112345+0000","last_change":"2024-03-06T07:26:51.832621+0000","last_active":"2024-03-06T07:26:54.112345+0000","last_peered":"2024-03-06T07:26:54.112345+0000","last_clean":"2024-03-06T07:26:54.112345+0000","last_became_active":"2024-03-06T07:26:51.832323+0000","last_became_peered":"2024-03-06T07:26:51.832323+0000","last_unstale":"2024-03-06T07:26:54.112345+0000","last_undegraded":"2024-03-06T07:26:54.112345+0000","last_fullsized":"2024-03-06T07:26:54.112345+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:19.927308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830966+0000","last_change":"2024-03-06T07:26:52.849363+0000","last_active":"2024-03-06T07:26:53.830966+0000","last_peered":"2024-03-06T07:26:53.830966+0000","last_clean":"2024-03-06T07:26:53.830966+0000","last_became_active":"2024-03-06T07:26:52.849003+0000","last_became_peered":"2024-03-06T07:26:52.849003+0000","last_unstale":"2024-03-06T07:26:53.830966+0000","last_undegraded":"2024-03-06T07:26:53.830966+0000","last_fullsized":"2024-03-06T07:26:53.830966+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:25.516252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831356+0000","last_change":"2024-03-06T07:26:51.835855+0000","last_active":"2024-03-06T07:26:53.831356+0000","last_peered":"2024-03-06T07:26:53.831356+0000","last_clean":"2024-03-06T07:26:53.831356+0000","last_became_active":"2024-03-06T07:26:51.835519+0000","last_became_peered":"2024-03-06T07:26:51.835519+0000","last_unstale":"2024-03-06T07:26:53.831356+0000","last_undegraded":"2024-03-06T07:26:53.831356+0000","last_fullsized":"2024-03-06T07:26:53.831356+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:56.612802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112484+0000","last_change":"2024-03-06T07:26:52.841166+0000","last_active":"2024-03-06T07:26:54.112484+0000","last_peered":"2024-03-06T07:26:54.112484+0000","last_clean":"2024-03-06T07:26:54.112484+0000","last_became_active":"2024-03-06T07:26:52.840844+0000","last_became_peered":"2024-03-06T07:26:52.840844+0000","last_unstale":"2024-03-06T07:26:54.112484+0000","last_undegraded":"2024-03-06T07:26:54.112484+0000","last_fullsized":"2024-03-06T07:26:54.112484+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:03.154423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111645+0000","last_change":"2024-03-06T07:26:51.824784+0000","last_active":"2024-03-06T07:26:54.111645+0000","last_peered":"2024-03-06T07:26:54.111645+0000","last_clean":"2024-03-06T07:26:54.111645+0000","last_became_active":"2024-03-06T07:26:51.824504+0000","last_became_peered":"2024-03-06T07:26:51.824504+0000","last_unstale":"2024-03-06T07:26:54.111645+0000","last_undegraded":"2024-03-06T07:26:54.111645+0000","last_fullsized":"2024-03-06T07:26:54.111645+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:19.927308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112045+0000","last_change":"2024-03-06T07:26:52.844410+0000","last_active":"2024-03-06T07:26:54.112045+0000","last_peered":"2024-03-06T07:26:54.112045+0000","last_clean":"2024-03-06T07:26:54.112045+0000","last_became_active":"2024-03-06T07:26:52.844085+0000","last_became_peered":"2024-03-06T07:26:52.844085+0000","last_unstale":"2024-03-06T07:26:54.112045+0000","last_undegraded":"2024-03-06T07:26:54.112045+0000","last_fullsized":"2024-03-06T07:26:54.112045+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:26:18.994417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111161+0000","last_change":"2024-03-06T07:26:51.834404+0000","last_active":"2024-03-06T07:26:54.111161+0000","last_peered":"2024-03-06T07:26:54.111161+0000","last_clean":"2024-03-06T07:26:54.111161+0000","last_became_active":"2024-03-06T07:26:51.834166+0000","last_became_peered":"2024-03-06T07:26:51.834166+0000","last_unstale":"2024-03-06T07:26:54.111161+0000","last_undegraded":"2024-03-06T07:26:54.111161+0000","last_fullsized":"2024-03-06T07:26:54.111161+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:49.518211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111933+0000","last_change":"2024-03-06T07:26:52.844056+0000","last_active":"2024-03-06T07:26:54.111933+0000","last_peered":"2024-03-06T07:26:54.111933+0000","last_clean":"2024-03-06T07:26:54.111933+0000","last_became_active":"2024-03-06T07:26:52.843754+0000","last_became_peered":"2024-03-06T07:26:52.843754+0000","last_unstale":"2024-03-06T07:26:54.111933+0000","last_undegraded":"2024-03-06T07:26:54.111933+0000","last_fullsized":"2024-03-06T07:26:54.111933+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:52.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"171'8","reported_seq":22,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.110940+0000","last_change":"2024-03-06T07:26:51.829602+0000","last_active":"2024-03-06T07:26:54.110940+0000","last_peered":"2024-03-06T07:26:54.110940+0000","last_clean":"2024-03-06T07:26:54.110940+0000","last_became_active":"2024-03-06T07:26:51.829165+0000","last_became_peered":"2024-03-06T07:26:51.829165+0000","last_unstale":"2024-03-06T07:26:54.110940+0000","last_undegraded":"2024-03-06T07:26:54.110940+0000","last_fullsized":"2024-03-06T07:26:54.110940+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:19.115797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.113177+0000","last_change":"2024-03-06T07:26:52.829644+0000","last_active":"2024-03-06T07:26:54.113177+0000","last_peered":"2024-03-06T07:26:54.113177+0000","last_clean":"2024-03-06T07:26:54.113177+0000","last_became_active":"2024-03-06T07:26:52.829320+0000","last_became_peered":"2024-03-06T07:26:52.829320+0000","last_unstale":"2024-03-06T07:26:54.113177+0000","last_undegraded":"2024-03-06T07:26:54.113177+0000","last_fullsized":"2024-03-06T07:26:54.113177+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:44.836739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112176+0000","last_change":"2024-03-06T07:26:51.836792+0000","last_active":"2024-03-06T07:26:54.112176+0000","last_peered":"2024-03-06T07:26:54.112176+0000","last_clean":"2024-03-06T07:26:54.112176+0000","last_became_active":"2024-03-06T07:26:51.836520+0000","last_became_peered":"2024-03-06T07:26:51.836520+0000","last_unstale":"2024-03-06T07:26:54.112176+0000","last_undegraded":"2024-03-06T07:26:54.112176+0000","last_fullsized":"2024-03-06T07:26:54.112176+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:57.424213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112181+0000","last_change":"2024-03-06T07:26:52.838273+0000","last_active":"2024-03-06T07:26:54.112181+0000","last_peered":"2024-03-06T07:26:54.112181+0000","last_clean":"2024-03-06T07:26:54.112181+0000","last_became_active":"2024-03-06T07:26:52.837925+0000","last_became_peered":"2024-03-06T07:26:52.837925+0000","last_unstale":"2024-03-06T07:26:54.112181+0000","last_undegraded":"2024-03-06T07:26:54.112181+0000","last_fullsized":"2024-03-06T07:26:54.112181+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:56:23.284841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111869+0000","last_change":"2024-03-06T07:26:51.836626+0000","last_active":"2024-03-06T07:26:54.111869+0000","last_peered":"2024-03-06T07:26:54.111869+0000","last_clean":"2024-03-06T07:26:54.111869+0000","last_became_active":"2024-03-06T07:26:51.836177+0000","last_became_peered":"2024-03-06T07:26:51.836177+0000","last_unstale":"2024-03-06T07:26:54.111869+0000","last_undegraded":"2024-03-06T07:26:54.111869+0000","last_fullsized":"2024-03-06T07:26:54.111869+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:03.289934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111589+0000","last_change":"2024-03-06T07:26:52.834881+0000","last_active":"2024-03-06T07:26:54.111589+0000","last_peered":"2024-03-06T07:26:54.111589+0000","last_clean":"2024-03-06T07:26:54.111589+0000","last_became_active":"2024-03-06T07:26:52.834533+0000","last_became_peered":"2024-03-06T07:26:52.834533+0000","last_unstale":"2024-03-06T07:26:54.111589+0000","last_undegraded":"2024-03-06T07:26:54.111589+0000","last_fullsized":"2024-03-06T07:26:54.111589+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:35.066881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831122+0000","last_change":"2024-03-06T07:26:51.827242+0000","last_active":"2024-03-06T07:26:53.831122+0000","last_peered":"2024-03-06T07:26:53.831122+0000","last_clean":"2024-03-06T07:26:53.831122+0000","last_became_active":"2024-03-06T07:26:51.826968+0000","last_became_peered":"2024-03-06T07:26:51.826968+0000","last_unstale":"2024-03-06T07:26:53.831122+0000","last_undegraded":"2024-03-06T07:26:53.831122+0000","last_fullsized":"2024-03-06T07:26:53.831122+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:18.273299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112714+0000","last_change":"2024-03-06T07:26:52.833432+0000","last_active":"2024-03-06T07:26:54.112714+0000","last_peered":"2024-03-06T07:26:54.112714+0000","last_clean":"2024-03-06T07:26:54.112714+0000","last_became_active":"2024-03-06T07:26:52.832887+0000","last_became_peered":"2024-03-06T07:26:52.832887+0000","last_unstale":"2024-03-06T07:26:54.112714+0000","last_undegraded":"2024-03-06T07:26:54.112714+0000","last_fullsized":"2024-03-06T07:26:54.112714+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:49.094931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112389+0000","last_change":"2024-03-06T07:26:51.835241+0000","last_active":"2024-03-06T07:26:54.112389+0000","last_peered":"2024-03-06T07:26:54.112389+0000","last_clean":"2024-03-06T07:26:54.112389+0000","last_became_active":"2024-03-06T07:26:51.834955+0000","last_became_peered":"2024-03-06T07:26:51.834955+0000","last_unstale":"2024-03-06T07:26:54.112389+0000","last_undegraded":"2024-03-06T07:26:54.112389+0000","last_fullsized":"2024-03-06T07:26:54.112389+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:20.370604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831117+0000","last_change":"2024-03-06T07:26:52.827584+0000","last_active":"2024-03-06T07:26:53.831117+0000","last_peered":"2024-03-06T07:26:53.831117+0000","last_clean":"2024-03-06T07:26:53.831117+0000","last_became_active":"2024-03-06T07:26:52.827276+0000","last_became_peered":"2024-03-06T07:26:52.827276+0000","last_unstale":"2024-03-06T07:26:53.831117+0000","last_undegraded":"2024-03-06T07:26:53.831117+0000","last_fullsized":"2024-03-06T07:26:53.831117+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:17.952849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111765+0000","last_change":"2024-03-06T07:26:51.828369+0000","last_active":"2024-03-06T07:26:54.111765+0000","last_peered":"2024-03-06T07:26:54.111765+0000","last_clean":"2024-03-06T07:26:54.111765+0000","last_became_active":"2024-03-06T07:26:51.828087+0000","last_became_peered":"2024-03-06T07:26:51.828087+0000","last_unstale":"2024-03-06T07:26:54.111765+0000","last_undegraded":"2024-03-06T07:26:54.111765+0000","last_fullsized":"2024-03-06T07:26:54.111765+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:51.321721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112601+0000","last_change":"2024-03-06T07:26:52.833175+0000","last_active":"2024-03-06T07:26:54.112601+0000","last_peered":"2024-03-06T07:26:54.112601+0000","last_clean":"2024-03-06T07:26:54.112601+0000","last_became_active":"2024-03-06T07:26:52.832777+0000","last_became_peered":"2024-03-06T07:26:52.832777+0000","last_unstale":"2024-03-06T07:26:54.112601+0000","last_undegraded":"2024-03-06T07:26:54.112601+0000","last_fullsized":"2024-03-06T07:26:54.112601+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:03.154423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831059+0000","last_change":"2024-03-06T07:26:51.831566+0000","last_active":"2024-03-06T07:26:53.831059+0000","last_peered":"2024-03-06T07:26:53.831059+0000","last_clean":"2024-03-06T07:26:53.831059+0000","last_became_active":"2024-03-06T07:26:51.831112+0000","last_became_peered":"2024-03-06T07:26:51.831112+0000","last_unstale":"2024-03-06T07:26:53.831059+0000","last_undegraded":"2024-03-06T07:26:53.831059+0000","last_fullsized":"2024-03-06T07:26:53.831059+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:45:32.193195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112175+0000","last_change":"2024-03-06T07:26:52.828752+0000","last_active":"2024-03-06T07:26:54.112175+0000","last_peered":"2024-03-06T07:26:54.112175+0000","last_clean":"2024-03-06T07:26:54.112175+0000","last_became_active":"2024-03-06T07:26:52.828460+0000","last_became_peered":"2024-03-06T07:26:52.828460+0000","last_unstale":"2024-03-06T07:26:54.112175+0000","last_undegraded":"2024-03-06T07:26:54.112175+0000","last_fullsized":"2024-03-06T07:26:54.112175+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:45:33.200682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833346+0000","last_change":"2024-03-06T07:26:51.826614+0000","last_active":"2024-03-06T07:26:53.833346+0000","last_peered":"2024-03-06T07:26:53.833346+0000","last_clean":"2024-03-06T07:26:53.833346+0000","last_became_active":"2024-03-06T07:26:51.826341+0000","last_became_peered":"2024-03-06T07:26:51.826341+0000","last_unstale":"2024-03-06T07:26:53.833346+0000","last_undegraded":"2024-03-06T07:26:53.833346+0000","last_fullsized":"2024-03-06T07:26:53.833346+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:07.548977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.113155+0000","last_change":"2024-03-06T07:26:52.838395+0000","last_active":"2024-03-06T07:26:54.113155+0000","last_peered":"2024-03-06T07:26:54.113155+0000","last_clean":"2024-03-06T07:26:54.113155+0000","last_became_active":"2024-03-06T07:26:52.838112+0000","last_became_peered":"2024-03-06T07:26:52.838112+0000","last_unstale":"2024-03-06T07:26:54.113155+0000","last_undegraded":"2024-03-06T07:26:54.113155+0000","last_fullsized":"2024-03-06T07:26:54.113155+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:10.384778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111281+0000","last_change":"2024-03-06T07:26:51.837690+0000","last_active":"2024-03-06T07:26:54.111281+0000","last_peered":"2024-03-06T07:26:54.111281+0000","last_clean":"2024-03-06T07:26:54.111281+0000","last_became_active":"2024-03-06T07:26:51.837390+0000","last_became_peered":"2024-03-06T07:26:51.837390+0000","last_unstale":"2024-03-06T07:26:54.111281+0000","last_undegraded":"2024-03-06T07:26:54.111281+0000","last_fullsized":"2024-03-06T07:26:54.111281+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:01.651771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.113327+0000","last_change":"2024-03-06T07:26:52.842712+0000","last_active":"2024-03-06T07:26:54.113327+0000","last_peered":"2024-03-06T07:26:54.113327+0000","last_clean":"2024-03-06T07:26:54.113327+0000","last_became_active":"2024-03-06T07:26:52.842401+0000","last_became_peered":"2024-03-06T07:26:52.842401+0000","last_unstale":"2024-03-06T07:26:54.113327+0000","last_undegraded":"2024-03-06T07:26:54.113327+0000","last_fullsized":"2024-03-06T07:26:54.113327+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:08.556464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112687+0000","last_change":"2024-03-06T07:26:51.832673+0000","last_active":"2024-03-06T07:26:54.112687+0000","last_peered":"2024-03-06T07:26:54.112687+0000","last_clean":"2024-03-06T07:26:54.112687+0000","last_became_active":"2024-03-06T07:26:51.831785+0000","last_became_peered":"2024-03-06T07:26:51.831785+0000","last_unstale":"2024-03-06T07:26:54.112687+0000","last_undegraded":"2024-03-06T07:26:54.112687+0000","last_fullsized":"2024-03-06T07:26:54.112687+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:25.181264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111574+0000","last_change":"2024-03-06T07:26:52.837198+0000","last_active":"2024-03-06T07:26:54.111574+0000","last_peered":"2024-03-06T07:26:54.111574+0000","last_clean":"2024-03-06T07:26:54.111574+0000","last_became_active":"2024-03-06T07:26:52.836899+0000","last_became_peered":"2024-03-06T07:26:52.836899+0000","last_unstale":"2024-03-06T07:26:54.111574+0000","last_undegraded":"2024-03-06T07:26:54.111574+0000","last_fullsized":"2024-03-06T07:26:54.111574+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:52.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"172'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.823777+0000","last_change":"2024-03-06T07:26:51.830363+0000","last_active":"2024-03-06T07:26:54.823777+0000","last_peered":"2024-03-06T07:26:54.823777+0000","last_clean":"2024-03-06T07:26:54.823777+0000","last_became_active":"2024-03-06T07:26:51.830065+0000","last_became_peered":"2024-03-06T07:26:51.830065+0000","last_unstale":"2024-03-06T07:26:54.823777+0000","last_undegraded":"2024-03-06T07:26:54.823777+0000","last_fullsized":"2024-03-06T07:26:54.823777+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:04:02.294443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111319+0000","last_change":"2024-03-06T07:26:52.839745+0000","last_active":"2024-03-06T07:26:54.111319+0000","last_peered":"2024-03-06T07:26:54.111319+0000","last_clean":"2024-03-06T07:26:54.111319+0000","last_became_active":"2024-03-06T07:26:52.839451+0000","last_became_peered":"2024-03-06T07:26:52.839451+0000","last_unstale":"2024-03-06T07:26:54.111319+0000","last_undegraded":"2024-03-06T07:26:54.111319+0000","last_fullsized":"2024-03-06T07:26:54.111319+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:49.560091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833465+0000","last_change":"2024-03-06T07:26:51.829436+0000","last_active":"2024-03-06T07:26:53.833465+0000","last_peered":"2024-03-06T07:26:53.833465+0000","last_clean":"2024-03-06T07:26:53.833465+0000","last_became_active":"2024-03-06T07:26:51.829173+0000","last_became_peered":"2024-03-06T07:26:51.829173+0000","last_unstale":"2024-03-06T07:26:53.833465+0000","last_undegraded":"2024-03-06T07:26:53.833465+0000","last_fullsized":"2024-03-06T07:26:53.833465+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:43.829252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111435+0000","last_change":"2024-03-06T07:26:52.839493+0000","last_active":"2024-03-06T07:26:54.111435+0000","last_peered":"2024-03-06T07:26:54.111435+0000","last_clean":"2024-03-06T07:26:54.111435+0000","last_became_active":"2024-03-06T07:26:52.838980+0000","last_became_peered":"2024-03-06T07:26:52.838980+0000","last_unstale":"2024-03-06T07:26:54.111435+0000","last_undegraded":"2024-03-06T07:26:54.111435+0000","last_fullsized":"2024-03-06T07:26:54.111435+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:13.961992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111474+0000","last_change":"2024-03-06T07:26:51.834526+0000","last_active":"2024-03-06T07:26:54.111474+0000","last_peered":"2024-03-06T07:26:54.111474+0000","last_clean":"2024-03-06T07:26:54.111474+0000","last_became_active":"2024-03-06T07:26:51.834275+0000","last_became_peered":"2024-03-06T07:26:51.834275+0000","last_unstale":"2024-03-06T07:26:54.111474+0000","last_undegraded":"2024-03-06T07:26:54.111474+0000","last_fullsized":"2024-03-06T07:26:54.111474+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:38.916465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112267+0000","last_change":"2024-03-06T07:26:52.839348+0000","last_active":"2024-03-06T07:26:54.112267+0000","last_peered":"2024-03-06T07:26:54.112267+0000","last_clean":"2024-03-06T07:26:54.112267+0000","last_became_active":"2024-03-06T07:26:52.838769+0000","last_became_peered":"2024-03-06T07:26:52.838769+0000","last_unstale":"2024-03-06T07:26:54.112267+0000","last_undegraded":"2024-03-06T07:26:54.112267+0000","last_fullsized":"2024-03-06T07:26:54.112267+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:45.355927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112808+0000","last_change":"2024-03-06T07:26:51.831751+0000","last_active":"2024-03-06T07:26:54.112808+0000","last_peered":"2024-03-06T07:26:54.112808+0000","last_clean":"2024-03-06T07:26:54.112808+0000","last_became_active":"2024-03-06T07:26:51.831349+0000","last_became_peered":"2024-03-06T07:26:51.831349+0000","last_unstale":"2024-03-06T07:26:54.112808+0000","last_undegraded":"2024-03-06T07:26:54.112808+0000","last_fullsized":"2024-03-06T07:26:54.112808+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:12.614392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112586+0000","last_change":"2024-03-06T07:26:52.849983+0000","last_active":"2024-03-06T07:26:54.112586+0000","last_peered":"2024-03-06T07:26:54.112586+0000","last_clean":"2024-03-06T07:26:54.112586+0000","last_became_active":"2024-03-06T07:26:52.849675+0000","last_became_peered":"2024-03-06T07:26:52.849675+0000","last_unstale":"2024-03-06T07:26:54.112586+0000","last_undegraded":"2024-03-06T07:26:54.112586+0000","last_fullsized":"2024-03-06T07:26:54.112586+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:38.074300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111622+0000","last_change":"2024-03-06T07:26:51.831817+0000","last_active":"2024-03-06T07:26:54.111622+0000","last_peered":"2024-03-06T07:26:54.111622+0000","last_clean":"2024-03-06T07:26:54.111622+0000","last_became_active":"2024-03-06T07:26:51.831548+0000","last_became_peered":"2024-03-06T07:26:51.831548+0000","last_unstale":"2024-03-06T07:26:54.111622+0000","last_undegraded":"2024-03-06T07:26:54.111622+0000","last_fullsized":"2024-03-06T07:26:54.111622+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:37.066813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112641+0000","last_change":"2024-03-06T07:26:52.849388+0000","last_active":"2024-03-06T07:26:54.112641+0000","last_peered":"2024-03-06T07:26:54.112641+0000","last_clean":"2024-03-06T07:26:54.112641+0000","last_became_active":"2024-03-06T07:26:52.849107+0000","last_became_peered":"2024-03-06T07:26:52.849107+0000","last_unstale":"2024-03-06T07:26:54.112641+0000","last_undegraded":"2024-03-06T07:26:54.112641+0000","last_fullsized":"2024-03-06T07:26:54.112641+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:50.525698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111707+0000","last_change":"2024-03-06T07:26:51.835844+0000","last_active":"2024-03-06T07:26:54.111707+0000","last_peered":"2024-03-06T07:26:54.111707+0000","last_clean":"2024-03-06T07:26:54.111707+0000","last_became_active":"2024-03-06T07:26:51.835560+0000","last_became_peered":"2024-03-06T07:26:51.835560+0000","last_unstale":"2024-03-06T07:26:54.111707+0000","last_undegraded":"2024-03-06T07:26:54.111707+0000","last_fullsized":"2024-03-06T07:26:54.111707+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:25.876174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111041+0000","last_change":"2024-03-06T07:26:52.837237+0000","last_active":"2024-03-06T07:26:54.111041+0000","last_peered":"2024-03-06T07:26:54.111041+0000","last_clean":"2024-03-06T07:26:54.111041+0000","last_became_active":"2024-03-06T07:26:52.836962+0000","last_became_peered":"2024-03-06T07:26:52.836962+0000","last_unstale":"2024-03-06T07:26:54.111041+0000","last_undegraded":"2024-03-06T07:26:54.111041+0000","last_fullsized":"2024-03-06T07:26:54.111041+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:03.380633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111042+0000","last_change":"2024-03-06T07:26:51.829446+0000","last_active":"2024-03-06T07:26:54.111042+0000","last_peered":"2024-03-06T07:26:54.111042+0000","last_clean":"2024-03-06T07:26:54.111042+0000","last_became_active":"2024-03-06T07:26:51.828753+0000","last_became_peered":"2024-03-06T07:26:51.828753+0000","last_unstale":"2024-03-06T07:26:54.111042+0000","last_undegraded":"2024-03-06T07:26:54.111042+0000","last_fullsized":"2024-03-06T07:26:54.111042+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:08.841551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111595+0000","last_change":"2024-03-06T07:26:52.845130+0000","last_active":"2024-03-06T07:26:54.111595+0000","last_peered":"2024-03-06T07:26:54.111595+0000","last_clean":"2024-03-06T07:26:54.111595+0000","last_became_active":"2024-03-06T07:26:52.844790+0000","last_became_peered":"2024-03-06T07:26:52.844790+0000","last_unstale":"2024-03-06T07:26:54.111595+0000","last_undegraded":"2024-03-06T07:26:54.111595+0000","last_fullsized":"2024-03-06T07:26:54.111595+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:37.717248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111536+0000","last_change":"2024-03-06T07:26:51.830404+0000","last_active":"2024-03-06T07:26:54.111536+0000","last_peered":"2024-03-06T07:26:54.111536+0000","last_clean":"2024-03-06T07:26:54.111536+0000","last_became_active":"2024-03-06T07:26:51.830109+0000","last_became_peered":"2024-03-06T07:26:51.830109+0000","last_unstale":"2024-03-06T07:26:54.111536+0000","last_undegraded":"2024-03-06T07:26:54.111536+0000","last_fullsized":"2024-03-06T07:26:54.111536+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:05.891150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":456,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111381+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:26:54.111381+0000","last_peered":"2024-03-06T07:26:54.111381+0000","last_clean":"2024-03-06T07:26:54.111381+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:26:54.111381+0000","last_undegraded":"2024-03-06T07:26:54.111381+0000","last_fullsized":"2024-03-06T07:26:54.111381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2256896,"data_stored":1652848,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2191360,"data_stored":1062992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2183168,"data_stored":1060854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1061380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2187264,"data_stored":1060924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2179072,"data_stored":1060888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2174976,"data_stored":1060798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33372,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338468,"statfs":{"total":96636764160,"available":96602591232,"internally_reserved":0,"allocated":2244608,"data_stored":1651256,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:55.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.024+0000 7fd4595b6640 1 Processor -- start 2024-03-06T07:26:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.024+0000 7fd4595b6640 1 -- start start 2024-03-06T07:26:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd4595b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 0x7fd454155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd4595b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd454156180 con 0x7fd4540c8a00 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd4595b6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd454159630 con 0x7fd4540c5330 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd4595b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4541587b0 con 0x7fd454152920 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd4527fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd4540c8a00 0x7fd454158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36950/0 (socket says 172.21.15.26:36950) 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd4527fc640 1 -- 172.21.15.26:0/3453918999 learned_addr learned my addr 172.21.15.26:0/3453918999 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd452ffd640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 0x7fd454155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592155152 0 0) 0x7fd43c001110 con 0x7fd4540c8a00 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd440003520 con 0x7fd4540c8a00 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1604045489 0 0) 0x7fd440003520 con 0x7fd4540c8a00 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 msgr2=0x7fd454155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 0x7fd454155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 >> v1:172.21.15.117:6789/0 conn(0x7fd4540c5330 legacy=0x7fd45415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540c7760 con 0x7fd4540c8a00 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.025+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd43c001840 con 0x7fd4540c8a00 2024-03-06T07:26:56.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 >> v1:172.21.15.26:6789/0 conn(0x7fd4540c8a00 legacy=0x7fd454158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd451ffb640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 0x7fd4400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd451ffb640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd440003f20 0x7fd440004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd451ffb640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd440004d40 0x7fd4400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4541587b0 con 0x7fd440004d40 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44000bad0 con 0x7fd440003f20 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44000bcd0 con 0x7fd454152920 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd4537fe640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd440004d40 0x7fd4400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd452ffd640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 0x7fd4400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd4527fc640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd440003f20 0x7fd440004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd4537fe640 1 -- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 msgr2=0x7fd4400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd4537fe640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd454152920 0x7fd4400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd4537fe640 1 -- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd440003f20 msgr2=0x7fd440004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd4537fe640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd440003f20 0x7fd440004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.026+0000 7fd4537fe640 1 -- 172.21.15.26:0/3453918999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440003520 con 0x7fd440004d40 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd4537fe640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd440004d40 0x7fd4400051c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd44c001f60 tx=0x7fd440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2114955b1accf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c001b30 con 0x7fd440004d40 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c02d710 con 0x7fd440004d40 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd451ffb640 1 -- 172.21.15.26:0/3453918999 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c02db00 con 0x7fd440004d40 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd4595b6640 1 -- 172.21.15.26:0/3453918999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd440004d40 msgr2=0x7fd4400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd4595b6640 1 --2- 172.21.15.26:0/3453918999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd440004d40 0x7fd4400051c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd44c001f60 tx=0x7fd440005830 comp rx=0 tx=0).stop 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd4537fe640 1 -- 172.21.15.26:0/3453918999 reap_dead start 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd4595b6640 1 -- 172.21.15.26:0/3453918999 shutdown_connections 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd4595b6640 1 -- 172.21.15.26:0/3453918999 >> 172.21.15.26:0/3453918999 conn(0x7fd4540ac560 msgr2=0x7fd4540ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.027+0000 7fd4595b6640 1 -- 172.21.15.26:0/3453918999 shutdown_connections 2024-03-06T07:26:56.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.028+0000 7fd4595b6640 1 -- 172.21.15.26:0/3453918999 wait complete. 2024-03-06T07:26:56.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 Processor -- start 2024-03-06T07:26:56.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 -- start start 2024-03-06T07:26:56.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd45414e390 0x7fd4540c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4540c3810 0x7fd4541ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd45414be80 0x7fd4541ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd454158a00 con 0x7fd45414e390 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540c7d20 con 0x7fd4540c3810 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.029+0000 7fd4595b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45406d030 con 0x7fd45414be80 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd45414e390 0x7fd4540c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd45414e390 0x7fd4540c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48450/0 (socket says 172.21.15.26:48450) 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 -- 172.21.15.26:0/3358086764 learned_addr learned my addr 172.21.15.26:0/3358086764 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd4527fc640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4540c3810 0x7fd4541ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd4537fe640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd45414be80 0x7fd4541ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 -- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd45414be80 msgr2=0x7fd4541ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd45414be80 0x7fd4541ee610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 -- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4540c3810 msgr2=0x7fd4541ee0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4540c3810 0x7fd4541ee0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 -- 172.21.15.26:0/3358086764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540c48a0 con 0x7fd45414e390 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd452ffd640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd45414e390 0x7fd4540c32d0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd4440119a0 tx=0x7fd444011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd44b7fe640 1 -- 172.21.15.26:0/3358086764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400faf0 con 0x7fd45414e390 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd44b7fe640 1 -- 172.21.15.26:0/3358086764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44400fc90 con 0x7fd45414e390 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd44b7fe640 1 -- 172.21.15.26:0/3358086764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444014440 con 0x7fd45414e390 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4540c43a0 con 0x7fd45414e390 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.030+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4541eb5e0 con 0x7fd45414e390 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.031+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45404f330 con 0x7fd45414e390 2024-03-06T07:26:56.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.032+0000 7fd44b7fe640 1 -- 172.21.15.26:0/3358086764 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd444012040 con 0x7fd45414e390 2024-03-06T07:26:56.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.032+0000 7fd44b7fe640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.032+0000 7fd44b7fe640 1 -- 172.21.15.26:0/3358086764 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd44408aa20 con 0x7fd45414e390 2024-03-06T07:26:56.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.032+0000 7fd4527fc640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.033+0000 7fd4527fc640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd43406b4d0 0x7fd43406d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd4540bd950 tx=0x7fd43c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.036+0000 7fd44b7fe640 1 -- 172.21.15.26:0/3358086764 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd444019030 con 0x7fd45414e390 2024-03-06T07:26:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.213+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd45414b930 con 0x7fd43406b4d0 2024-03-06T07:26:56.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.219+0000 7fd44b7fe640 1 -- 172.21.15.26:0/3358086764 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252566 (secure 0 0 0) 0x7fd45414b930 con 0x7fd43406b4d0 2024-03-06T07:26:56.223 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:56.224 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd43406b4d0 msgr2=0x7fd43406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd43406b4d0 0x7fd43406d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd4540bd950 tx=0x7fd43c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd45414e390 msgr2=0x7fd4540c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd45414e390 0x7fd4540c32d0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd4440119a0 tx=0x7fd444011e60 comp rx=0 tx=0).stop 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 shutdown_connections 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd45414be80 0x7fd4541ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4540c3810 0x7fd4541ee0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 --2- 172.21.15.26:0/3358086764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd45414e390 0x7fd4540c32d0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 >> 172.21.15.26:0/3358086764 conn(0x7fd4540ac560 msgr2=0x7fd4540ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:56.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 shutdown_connections 2024-03-06T07:26:56.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.222+0000 7fd4595b6640 1 -- 172.21.15.26:0/3358086764 wait complete. 2024-03-06T07:26:56.238 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-03-06T07:26:55.116787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266656,"kb_used_data":17184,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708064,"statfs":{"total":773094113280,"available":772821057536,"internally_reserved":0,"allocated":17596416,"data_stored":9671940,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.306830"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112099+0000","last_change":"2024-03-06T07:26:52.839051+0000","last_active":"2024-03-06T07:26:54.112099+0000","last_peered":"2024-03-06T07:26:54.112099+0000","last_clean":"2024-03-06T07:26:54.112099+0000","last_became_active":"2024-03-06T07:26:52.838710+0000","last_became_peered":"2024-03-06T07:26:52.838710+0000","last_unstale":"2024-03-06T07:26:54.112099+0000","last_undegraded":"2024-03-06T07:26:54.112099+0000","last_fullsized":"2024-03-06T07:26:54.112099+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:54:37.889646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831250+0000","last_change":"2024-03-06T07:26:52.826808+0000","last_active":"2024-03-06T07:26:53.831250+0000","last_peered":"2024-03-06T07:26:53.831250+0000","last_clean":"2024-03-06T07:26:53.831250+0000","last_became_active":"2024-03-06T07:26:52.826508+0000","last_became_peered":"2024-03-06T07:26:52.826508+0000","last_unstale":"2024-03-06T07:26:53.831250+0000","last_undegraded":"2024-03-06T07:26:53.831250+0000","last_fullsized":"2024-03-06T07:26:53.831250+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:11:06.620959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112032+0000","last_change":"2024-03-06T07:26:52.836625+0000","last_active":"2024-03-06T07:26:54.112032+0000","last_peered":"2024-03-06T07:26:54.112032+0000","last_clean":"2024-03-06T07:26:54.112032+0000","last_became_active":"2024-03-06T07:26:52.836349+0000","last_became_peered":"2024-03-06T07:26:52.836349+0000","last_unstale":"2024-03-06T07:26:54.112032+0000","last_undegraded":"2024-03-06T07:26:54.112032+0000","last_fullsized":"2024-03-06T07:26:54.112032+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:50.525698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112536+0000","last_change":"2024-03-06T07:26:52.840797+0000","last_active":"2024-03-06T07:26:54.112536+0000","last_peered":"2024-03-06T07:26:54.112536+0000","last_clean":"2024-03-06T07:26:54.112536+0000","last_became_active":"2024-03-06T07:26:52.840539+0000","last_became_peered":"2024-03-06T07:26:52.840539+0000","last_unstale":"2024-03-06T07:26:54.112536+0000","last_undegraded":"2024-03-06T07:26:54.112536+0000","last_fullsized":"2024-03-06T07:26:54.112536+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:59.872043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111392+0000","last_change":"2024-03-06T07:26:52.845997+0000","last_active":"2024-03-06T07:26:54.111392+0000","last_peered":"2024-03-06T07:26:54.111392+0000","last_clean":"2024-03-06T07:26:54.111392+0000","last_became_active":"2024-03-06T07:26:52.845746+0000","last_became_peered":"2024-03-06T07:26:52.845746+0000","last_unstale":"2024-03-06T07:26:54.111392+0000","last_undegraded":"2024-03-06T07:26:54.111392+0000","last_fullsized":"2024-03-06T07:26:54.111392+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:14.477763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111793+0000","last_change":"2024-03-06T07:26:52.841694+0000","last_active":"2024-03-06T07:26:54.111793+0000","last_peered":"2024-03-06T07:26:54.111793+0000","last_clean":"2024-03-06T07:26:54.111793+0000","last_became_active":"2024-03-06T07:26:52.841393+0000","last_became_peered":"2024-03-06T07:26:52.841393+0000","last_unstale":"2024-03-06T07:26:54.111793+0000","last_undegraded":"2024-03-06T07:26:54.111793+0000","last_fullsized":"2024-03-06T07:26:54.111793+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:32.948522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111239+0000","last_change":"2024-03-06T07:26:52.845949+0000","last_active":"2024-03-06T07:26:54.111239+0000","last_peered":"2024-03-06T07:26:54.111239+0000","last_clean":"2024-03-06T07:26:54.111239+0000","last_became_active":"2024-03-06T07:26:52.845692+0000","last_became_peered":"2024-03-06T07:26:52.845692+0000","last_unstale":"2024-03-06T07:26:54.111239+0000","last_undegraded":"2024-03-06T07:26:54.111239+0000","last_fullsized":"2024-03-06T07:26:54.111239+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:44.646930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112989+0000","last_change":"2024-03-06T07:26:52.840961+0000","last_active":"2024-03-06T07:26:54.112989+0000","last_peered":"2024-03-06T07:26:54.112989+0000","last_clean":"2024-03-06T07:26:54.112989+0000","last_became_active":"2024-03-06T07:26:52.837551+0000","last_became_peered":"2024-03-06T07:26:52.837551+0000","last_unstale":"2024-03-06T07:26:54.112989+0000","last_undegraded":"2024-03-06T07:26:54.112989+0000","last_fullsized":"2024-03-06T07:26:54.112989+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:42:14.924497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112350+0000","last_change":"2024-03-06T07:26:52.835924+0000","last_active":"2024-03-06T07:26:54.112350+0000","last_peered":"2024-03-06T07:26:54.112350+0000","last_clean":"2024-03-06T07:26:54.112350+0000","last_became_active":"2024-03-06T07:26:52.835547+0000","last_became_peered":"2024-03-06T07:26:52.835547+0000","last_unstale":"2024-03-06T07:26:54.112350+0000","last_undegraded":"2024-03-06T07:26:54.112350+0000","last_fullsized":"2024-03-06T07:26:54.112350+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:57.620289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112005+0000","last_change":"2024-03-06T07:26:52.836820+0000","last_active":"2024-03-06T07:26:54.112005+0000","last_peered":"2024-03-06T07:26:54.112005+0000","last_clean":"2024-03-06T07:26:54.112005+0000","last_became_active":"2024-03-06T07:26:52.836496+0000","last_became_peered":"2024-03-06T07:26:52.836496+0000","last_unstale":"2024-03-06T07:26:54.112005+0000","last_undegraded":"2024-03-06T07:26:54.112005+0000","last_fullsized":"2024-03-06T07:26:54.112005+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:37.494197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112172+0000","last_change":"2024-03-06T07:26:52.839475+0000","last_active":"2024-03-06T07:26:54.112172+0000","last_peered":"2024-03-06T07:26:54.112172+0000","last_clean":"2024-03-06T07:26:54.112172+0000","last_became_active":"2024-03-06T07:26:52.839019+0000","last_became_peered":"2024-03-06T07:26:52.839019+0000","last_unstale":"2024-03-06T07:26:54.112172+0000","last_undegraded":"2024-03-06T07:26:54.112172+0000","last_fullsized":"2024-03-06T07:26:54.112172+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:59.872043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112104+0000","last_change":"2024-03-06T07:26:52.843451+0000","last_active":"2024-03-06T07:26:54.112104+0000","last_peered":"2024-03-06T07:26:54.112104+0000","last_clean":"2024-03-06T07:26:54.112104+0000","last_became_active":"2024-03-06T07:26:52.843177+0000","last_became_peered":"2024-03-06T07:26:52.843177+0000","last_unstale":"2024-03-06T07:26:54.112104+0000","last_undegraded":"2024-03-06T07:26:54.112104+0000","last_fullsized":"2024-03-06T07:26:54.112104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:15.583575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112660+0000","last_change":"2024-03-06T07:26:52.839732+0000","last_active":"2024-03-06T07:26:54.112660+0000","last_peered":"2024-03-06T07:26:54.112660+0000","last_clean":"2024-03-06T07:26:54.112660+0000","last_became_active":"2024-03-06T07:26:52.839470+0000","last_became_peered":"2024-03-06T07:26:52.839470+0000","last_unstale":"2024-03-06T07:26:54.112660+0000","last_undegraded":"2024-03-06T07:26:54.112660+0000","last_fullsized":"2024-03-06T07:26:54.112660+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:06.898636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111702+0000","last_change":"2024-03-06T07:26:52.844734+0000","last_active":"2024-03-06T07:26:54.111702+0000","last_peered":"2024-03-06T07:26:54.111702+0000","last_clean":"2024-03-06T07:26:54.111702+0000","last_became_active":"2024-03-06T07:26:52.843191+0000","last_became_peered":"2024-03-06T07:26:52.843191+0000","last_unstale":"2024-03-06T07:26:54.111702+0000","last_undegraded":"2024-03-06T07:26:54.111702+0000","last_fullsized":"2024-03-06T07:26:54.111702+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:20.261482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833742+0000","last_change":"2024-03-06T07:26:52.837625+0000","last_active":"2024-03-06T07:26:53.833742+0000","last_peered":"2024-03-06T07:26:53.833742+0000","last_clean":"2024-03-06T07:26:53.833742+0000","last_became_active":"2024-03-06T07:26:52.837363+0000","last_became_peered":"2024-03-06T07:26:52.837363+0000","last_unstale":"2024-03-06T07:26:53.833742+0000","last_undegraded":"2024-03-06T07:26:53.833742+0000","last_fullsized":"2024-03-06T07:26:53.833742+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:41.074583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111455+0000","last_change":"2024-03-06T07:26:52.835458+0000","last_active":"2024-03-06T07:26:54.111455+0000","last_peered":"2024-03-06T07:26:54.111455+0000","last_clean":"2024-03-06T07:26:54.111455+0000","last_became_active":"2024-03-06T07:26:52.835095+0000","last_became_peered":"2024-03-06T07:26:52.835095+0000","last_unstale":"2024-03-06T07:26:54.111455+0000","last_undegraded":"2024-03-06T07:26:54.111455+0000","last_fullsized":"2024-03-06T07:26:54.111455+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:32.948522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112475+0000","last_change":"2024-03-06T07:26:52.833356+0000","last_active":"2024-03-06T07:26:54.112475+0000","last_peered":"2024-03-06T07:26:54.112475+0000","last_clean":"2024-03-06T07:26:54.112475+0000","last_became_active":"2024-03-06T07:26:52.833031+0000","last_became_peered":"2024-03-06T07:26:52.833031+0000","last_unstale":"2024-03-06T07:26:54.112475+0000","last_undegraded":"2024-03-06T07:26:54.112475+0000","last_fullsized":"2024-03-06T07:26:54.112475+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:13.271264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111913+0000","last_change":"2024-03-06T07:26:52.836979+0000","last_active":"2024-03-06T07:26:54.111913+0000","last_peered":"2024-03-06T07:26:54.111913+0000","last_clean":"2024-03-06T07:26:54.111913+0000","last_became_active":"2024-03-06T07:26:52.836693+0000","last_became_peered":"2024-03-06T07:26:52.836693+0000","last_unstale":"2024-03-06T07:26:54.111913+0000","last_undegraded":"2024-03-06T07:26:54.111913+0000","last_fullsized":"2024-03-06T07:26:54.111913+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:11.712562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112647+0000","last_change":"2024-03-06T07:26:52.843510+0000","last_active":"2024-03-06T07:26:54.112647+0000","last_peered":"2024-03-06T07:26:54.112647+0000","last_clean":"2024-03-06T07:26:54.112647+0000","last_became_active":"2024-03-06T07:26:52.843135+0000","last_became_peered":"2024-03-06T07:26:52.843135+0000","last_unstale":"2024-03-06T07:26:54.112647+0000","last_undegraded":"2024-03-06T07:26:54.112647+0000","last_fullsized":"2024-03-06T07:26:54.112647+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:26.988672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111531+0000","last_change":"2024-03-06T07:26:52.847561+0000","last_active":"2024-03-06T07:26:54.111531+0000","last_peered":"2024-03-06T07:26:54.111531+0000","last_clean":"2024-03-06T07:26:54.111531+0000","last_became_active":"2024-03-06T07:26:52.846304+0000","last_became_peered":"2024-03-06T07:26:52.846304+0000","last_unstale":"2024-03-06T07:26:54.111531+0000","last_undegraded":"2024-03-06T07:26:54.111531+0000","last_fullsized":"2024-03-06T07:26:54.111531+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:13.127494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111535+0000","last_change":"2024-03-06T07:26:52.842836+0000","last_active":"2024-03-06T07:26:54.111535+0000","last_peered":"2024-03-06T07:26:54.111535+0000","last_clean":"2024-03-06T07:26:54.111535+0000","last_became_active":"2024-03-06T07:26:52.842536+0000","last_became_peered":"2024-03-06T07:26:52.842536+0000","last_unstale":"2024-03-06T07:26:54.111535+0000","last_undegraded":"2024-03-06T07:26:54.111535+0000","last_fullsized":"2024-03-06T07:26:54.111535+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:53.447704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111870+0000","last_change":"2024-03-06T07:26:52.845253+0000","last_active":"2024-03-06T07:26:54.111870+0000","last_peered":"2024-03-06T07:26:54.111870+0000","last_clean":"2024-03-06T07:26:54.111870+0000","last_became_active":"2024-03-06T07:26:52.844952+0000","last_became_peered":"2024-03-06T07:26:52.844952+0000","last_unstale":"2024-03-06T07:26:54.111870+0000","last_undegraded":"2024-03-06T07:26:54.111870+0000","last_fullsized":"2024-03-06T07:26:54.111870+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:27.316431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830921+0000","last_change":"2024-03-06T07:26:52.842770+0000","last_active":"2024-03-06T07:26:53.830921+0000","last_peered":"2024-03-06T07:26:53.830921+0000","last_clean":"2024-03-06T07:26:53.830921+0000","last_became_active":"2024-03-06T07:26:52.842460+0000","last_became_peered":"2024-03-06T07:26:52.842460+0000","last_unstale":"2024-03-06T07:26:53.830921+0000","last_undegraded":"2024-03-06T07:26:53.830921+0000","last_fullsized":"2024-03-06T07:26:53.830921+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:48:55.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.110827+0000","last_change":"2024-03-06T07:26:52.848322+0000","last_active":"2024-03-06T07:26:54.110827+0000","last_peered":"2024-03-06T07:26:54.110827+0000","last_clean":"2024-03-06T07:26:54.110827+0000","last_became_active":"2024-03-06T07:26:52.847974+0000","last_became_peered":"2024-03-06T07:26:52.847974+0000","last_unstale":"2024-03-06T07:26:54.110827+0000","last_undegraded":"2024-03-06T07:26:54.110827+0000","last_fullsized":"2024-03-06T07:26:54.110827+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:13.621879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112587+0000","last_change":"2024-03-06T07:26:52.837162+0000","last_active":"2024-03-06T07:26:54.112587+0000","last_peered":"2024-03-06T07:26:54.112587+0000","last_clean":"2024-03-06T07:26:54.112587+0000","last_became_active":"2024-03-06T07:26:52.836816+0000","last_became_peered":"2024-03-06T07:26:52.836816+0000","last_unstale":"2024-03-06T07:26:54.112587+0000","last_undegraded":"2024-03-06T07:26:54.112587+0000","last_fullsized":"2024-03-06T07:26:54.112587+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:19.280786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111754+0000","last_change":"2024-03-06T07:26:52.846391+0000","last_active":"2024-03-06T07:26:54.111754+0000","last_peered":"2024-03-06T07:26:54.111754+0000","last_clean":"2024-03-06T07:26:54.111754+0000","last_became_active":"2024-03-06T07:26:52.846040+0000","last_became_peered":"2024-03-06T07:26:52.846040+0000","last_unstale":"2024-03-06T07:26:54.111754+0000","last_undegraded":"2024-03-06T07:26:54.111754+0000","last_fullsized":"2024-03-06T07:26:54.111754+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:46.086368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112331+0000","last_change":"2024-03-06T07:26:52.839034+0000","last_active":"2024-03-06T07:26:54.112331+0000","last_peered":"2024-03-06T07:26:54.112331+0000","last_clean":"2024-03-06T07:26:54.112331+0000","last_became_active":"2024-03-06T07:26:52.838335+0000","last_became_peered":"2024-03-06T07:26:52.838335+0000","last_unstale":"2024-03-06T07:26:54.112331+0000","last_undegraded":"2024-03-06T07:26:54.112331+0000","last_fullsized":"2024-03-06T07:26:54.112331+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:38.074300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112303+0000","last_change":"2024-03-06T07:26:52.847424+0000","last_active":"2024-03-06T07:26:54.112303+0000","last_peered":"2024-03-06T07:26:54.112303+0000","last_clean":"2024-03-06T07:26:54.112303+0000","last_became_active":"2024-03-06T07:26:52.846812+0000","last_became_peered":"2024-03-06T07:26:52.846812+0000","last_unstale":"2024-03-06T07:26:54.112303+0000","last_undegraded":"2024-03-06T07:26:54.112303+0000","last_fullsized":"2024-03-06T07:26:54.112303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:27.072412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111771+0000","last_change":"2024-03-06T07:26:52.832234+0000","last_active":"2024-03-06T07:26:54.111771+0000","last_peered":"2024-03-06T07:26:54.111771+0000","last_clean":"2024-03-06T07:26:54.111771+0000","last_became_active":"2024-03-06T07:26:52.831908+0000","last_became_peered":"2024-03-06T07:26:52.831908+0000","last_unstale":"2024-03-06T07:26:54.111771+0000","last_undegraded":"2024-03-06T07:26:54.111771+0000","last_fullsized":"2024-03-06T07:26:54.111771+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:49.943342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833336+0000","last_change":"2024-03-06T07:26:52.832207+0000","last_active":"2024-03-06T07:26:53.833336+0000","last_peered":"2024-03-06T07:26:53.833336+0000","last_clean":"2024-03-06T07:26:53.833336+0000","last_became_active":"2024-03-06T07:26:52.831879+0000","last_became_peered":"2024-03-06T07:26:52.831879+0000","last_unstale":"2024-03-06T07:26:53.833336+0000","last_undegraded":"2024-03-06T07:26:53.833336+0000","last_fullsized":"2024-03-06T07:26:53.833336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:17.300528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111558+0000","last_change":"2024-03-06T07:26:52.832027+0000","last_active":"2024-03-06T07:26:54.111558+0000","last_peered":"2024-03-06T07:26:54.111558+0000","last_clean":"2024-03-06T07:26:54.111558+0000","last_became_active":"2024-03-06T07:26:52.831754+0000","last_became_peered":"2024-03-06T07:26:52.831754+0000","last_unstale":"2024-03-06T07:26:54.111558+0000","last_undegraded":"2024-03-06T07:26:54.111558+0000","last_fullsized":"2024-03-06T07:26:54.111558+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:21:19.273834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111392+0000","last_change":"2024-03-06T07:26:52.832539+0000","last_active":"2024-03-06T07:26:54.111392+0000","last_peered":"2024-03-06T07:26:54.111392+0000","last_clean":"2024-03-06T07:26:54.111392+0000","last_became_active":"2024-03-06T07:26:52.832151+0000","last_became_peered":"2024-03-06T07:26:52.832151+0000","last_unstale":"2024-03-06T07:26:54.111392+0000","last_undegraded":"2024-03-06T07:26:54.111392+0000","last_fullsized":"2024-03-06T07:26:54.111392+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:03.380633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830892+0000","last_change":"2024-03-06T07:26:52.843827+0000","last_active":"2024-03-06T07:26:53.830892+0000","last_peered":"2024-03-06T07:26:53.830892+0000","last_clean":"2024-03-06T07:26:53.830892+0000","last_became_active":"2024-03-06T07:26:52.843493+0000","last_became_peered":"2024-03-06T07:26:52.843493+0000","last_unstale":"2024-03-06T07:26:53.830892+0000","last_undegraded":"2024-03-06T07:26:53.830892+0000","last_fullsized":"2024-03-06T07:26:53.830892+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:31.688070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"171'4","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833661+0000","last_change":"2024-03-06T07:26:51.825945+0000","last_active":"2024-03-06T07:26:53.833661+0000","last_peered":"2024-03-06T07:26:53.833661+0000","last_clean":"2024-03-06T07:26:53.833661+0000","last_became_active":"2024-03-06T07:26:51.825667+0000","last_became_peered":"2024-03-06T07:26:51.825667+0000","last_unstale":"2024-03-06T07:26:53.833661+0000","last_undegraded":"2024-03-06T07:26:53.833661+0000","last_fullsized":"2024-03-06T07:26:53.833661+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:09.377291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111462+0000","last_change":"2024-03-06T07:26:52.842594+0000","last_active":"2024-03-06T07:26:54.111462+0000","last_peered":"2024-03-06T07:26:54.111462+0000","last_clean":"2024-03-06T07:26:54.111462+0000","last_became_active":"2024-03-06T07:26:52.842136+0000","last_became_peered":"2024-03-06T07:26:52.842136+0000","last_unstale":"2024-03-06T07:26:54.111462+0000","last_undegraded":"2024-03-06T07:26:54.111462+0000","last_fullsized":"2024-03-06T07:26:54.111462+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:10.157118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111836+0000","last_change":"2024-03-06T07:26:51.834206+0000","last_active":"2024-03-06T07:26:54.111836+0000","last_peered":"2024-03-06T07:26:54.111836+0000","last_clean":"2024-03-06T07:26:54.111836+0000","last_became_active":"2024-03-06T07:26:51.833890+0000","last_became_peered":"2024-03-06T07:26:51.833890+0000","last_unstale":"2024-03-06T07:26:54.111836+0000","last_undegraded":"2024-03-06T07:26:54.111836+0000","last_fullsized":"2024-03-06T07:26:54.111836+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:00.234438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112054+0000","last_change":"2024-03-06T07:26:52.845024+0000","last_active":"2024-03-06T07:26:54.112054+0000","last_peered":"2024-03-06T07:26:54.112054+0000","last_clean":"2024-03-06T07:26:54.112054+0000","last_became_active":"2024-03-06T07:26:52.843567+0000","last_became_peered":"2024-03-06T07:26:52.843567+0000","last_unstale":"2024-03-06T07:26:54.112054+0000","last_undegraded":"2024-03-06T07:26:54.112054+0000","last_fullsized":"2024-03-06T07:26:54.112054+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:48.575265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830915+0000","last_change":"2024-03-06T07:26:51.830196+0000","last_active":"2024-03-06T07:26:53.830915+0000","last_peered":"2024-03-06T07:26:53.830915+0000","last_clean":"2024-03-06T07:26:53.830915+0000","last_became_active":"2024-03-06T07:26:51.829892+0000","last_became_peered":"2024-03-06T07:26:51.829892+0000","last_unstale":"2024-03-06T07:26:53.830915+0000","last_undegraded":"2024-03-06T07:26:53.830915+0000","last_fullsized":"2024-03-06T07:26:53.830915+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:12.263778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111525+0000","last_change":"2024-03-06T07:26:52.847259+0000","last_active":"2024-03-06T07:26:54.111525+0000","last_peered":"2024-03-06T07:26:54.111525+0000","last_clean":"2024-03-06T07:26:54.111525+0000","last_became_active":"2024-03-06T07:26:52.846983+0000","last_became_peered":"2024-03-06T07:26:52.846983+0000","last_unstale":"2024-03-06T07:26:54.111525+0000","last_undegraded":"2024-03-06T07:26:54.111525+0000","last_fullsized":"2024-03-06T07:26:54.111525+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:35:02.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112199+0000","last_change":"2024-03-06T07:26:51.831054+0000","last_active":"2024-03-06T07:26:54.112199+0000","last_peered":"2024-03-06T07:26:54.112199+0000","last_clean":"2024-03-06T07:26:54.112199+0000","last_became_active":"2024-03-06T07:26:51.830791+0000","last_became_peered":"2024-03-06T07:26:51.830791+0000","last_unstale":"2024-03-06T07:26:54.112199+0000","last_undegraded":"2024-03-06T07:26:54.112199+0000","last_fullsized":"2024-03-06T07:26:54.112199+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:58.864556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833659+0000","last_change":"2024-03-06T07:26:52.840186+0000","last_active":"2024-03-06T07:26:53.833659+0000","last_peered":"2024-03-06T07:26:53.833659+0000","last_clean":"2024-03-06T07:26:53.833659+0000","last_became_active":"2024-03-06T07:26:52.839100+0000","last_became_peered":"2024-03-06T07:26:52.839100+0000","last_unstale":"2024-03-06T07:26:53.833659+0000","last_undegraded":"2024-03-06T07:26:53.833659+0000","last_fullsized":"2024-03-06T07:26:53.833659+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:18:09.905056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112480+0000","last_change":"2024-03-06T07:26:51.829167+0000","last_active":"2024-03-06T07:26:54.112480+0000","last_peered":"2024-03-06T07:26:54.112480+0000","last_clean":"2024-03-06T07:26:54.112480+0000","last_became_active":"2024-03-06T07:26:51.828901+0000","last_became_peered":"2024-03-06T07:26:51.828901+0000","last_unstale":"2024-03-06T07:26:54.112480+0000","last_undegraded":"2024-03-06T07:26:54.112480+0000","last_fullsized":"2024-03-06T07:26:54.112480+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:52.402709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833522+0000","last_change":"2024-03-06T07:26:52.849185+0000","last_active":"2024-03-06T07:26:53.833522+0000","last_peered":"2024-03-06T07:26:53.833522+0000","last_clean":"2024-03-06T07:26:53.833522+0000","last_became_active":"2024-03-06T07:26:52.848858+0000","last_became_peered":"2024-03-06T07:26:52.848858+0000","last_unstale":"2024-03-06T07:26:53.833522+0000","last_undegraded":"2024-03-06T07:26:53.833522+0000","last_fullsized":"2024-03-06T07:26:53.833522+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:47.798514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111532+0000","last_change":"2024-03-06T07:26:51.828224+0000","last_active":"2024-03-06T07:26:54.111532+0000","last_peered":"2024-03-06T07:26:54.111532+0000","last_clean":"2024-03-06T07:26:54.111532+0000","last_became_active":"2024-03-06T07:26:51.827914+0000","last_became_peered":"2024-03-06T07:26:51.827914+0000","last_unstale":"2024-03-06T07:26:54.111532+0000","last_undegraded":"2024-03-06T07:26:54.111532+0000","last_fullsized":"2024-03-06T07:26:54.111532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:48:24.153761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112597+0000","last_change":"2024-03-06T07:26:52.829714+0000","last_active":"2024-03-06T07:26:54.112597+0000","last_peered":"2024-03-06T07:26:54.112597+0000","last_clean":"2024-03-06T07:26:54.112597+0000","last_became_active":"2024-03-06T07:26:52.829393+0000","last_became_peered":"2024-03-06T07:26:52.829393+0000","last_unstale":"2024-03-06T07:26:54.112597+0000","last_undegraded":"2024-03-06T07:26:54.112597+0000","last_fullsized":"2024-03-06T07:26:54.112597+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:15:29.095588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111548+0000","last_change":"2024-03-06T07:26:51.829143+0000","last_active":"2024-03-06T07:26:54.111548+0000","last_peered":"2024-03-06T07:26:54.111548+0000","last_clean":"2024-03-06T07:26:54.111548+0000","last_became_active":"2024-03-06T07:26:51.828874+0000","last_became_peered":"2024-03-06T07:26:51.828874+0000","last_unstale":"2024-03-06T07:26:54.111548+0000","last_undegraded":"2024-03-06T07:26:54.111548+0000","last_fullsized":"2024-03-06T07:26:54.111548+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:52.402709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833339+0000","last_change":"2024-03-06T07:26:52.841104+0000","last_active":"2024-03-06T07:26:53.833339+0000","last_peered":"2024-03-06T07:26:53.833339+0000","last_clean":"2024-03-06T07:26:53.833339+0000","last_became_active":"2024-03-06T07:26:52.840802+0000","last_became_peered":"2024-03-06T07:26:52.840802+0000","last_unstale":"2024-03-06T07:26:53.833339+0000","last_undegraded":"2024-03-06T07:26:53.833339+0000","last_fullsized":"2024-03-06T07:26:53.833339+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:51:20.576160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111329+0000","last_change":"2024-03-06T07:26:51.829319+0000","last_active":"2024-03-06T07:26:54.111329+0000","last_peered":"2024-03-06T07:26:54.111329+0000","last_clean":"2024-03-06T07:26:54.111329+0000","last_became_active":"2024-03-06T07:26:51.829075+0000","last_became_peered":"2024-03-06T07:26:51.829075+0000","last_unstale":"2024-03-06T07:26:54.111329+0000","last_undegraded":"2024-03-06T07:26:54.111329+0000","last_fullsized":"2024-03-06T07:26:54.111329+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:40.296510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111431+0000","last_change":"2024-03-06T07:26:52.844532+0000","last_active":"2024-03-06T07:26:54.111431+0000","last_peered":"2024-03-06T07:26:54.111431+0000","last_clean":"2024-03-06T07:26:54.111431+0000","last_became_active":"2024-03-06T07:26:52.844250+0000","last_became_peered":"2024-03-06T07:26:52.844250+0000","last_unstale":"2024-03-06T07:26:54.111431+0000","last_undegraded":"2024-03-06T07:26:54.111431+0000","last_fullsized":"2024-03-06T07:26:54.111431+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:21:19.273834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112364+0000","last_change":"2024-03-06T07:26:51.836021+0000","last_active":"2024-03-06T07:26:54.112364+0000","last_peered":"2024-03-06T07:26:54.112364+0000","last_clean":"2024-03-06T07:26:54.112364+0000","last_became_active":"2024-03-06T07:26:51.835563+0000","last_became_peered":"2024-03-06T07:26:51.835563+0000","last_unstale":"2024-03-06T07:26:54.112364+0000","last_undegraded":"2024-03-06T07:26:54.112364+0000","last_fullsized":"2024-03-06T07:26:54.112364+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:29.287753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111445+0000","last_change":"2024-03-06T07:26:52.835859+0000","last_active":"2024-03-06T07:26:54.111445+0000","last_peered":"2024-03-06T07:26:54.111445+0000","last_clean":"2024-03-06T07:26:54.111445+0000","last_became_active":"2024-03-06T07:26:52.835397+0000","last_became_peered":"2024-03-06T07:26:52.835397+0000","last_unstale":"2024-03-06T07:26:54.111445+0000","last_undegraded":"2024-03-06T07:26:54.111445+0000","last_fullsized":"2024-03-06T07:26:54.111445+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:00.646800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"171'3","reported_seq":17,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112433+0000","last_change":"2024-03-06T07:26:51.828522+0000","last_active":"2024-03-06T07:26:54.112433+0000","last_peered":"2024-03-06T07:26:54.112433+0000","last_clean":"2024-03-06T07:26:54.112433+0000","last_became_active":"2024-03-06T07:26:51.828200+0000","last_became_peered":"2024-03-06T07:26:51.828200+0000","last_unstale":"2024-03-06T07:26:54.112433+0000","last_undegraded":"2024-03-06T07:26:54.112433+0000","last_fullsized":"2024-03-06T07:26:54.112433+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:02.146936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111667+0000","last_change":"2024-03-06T07:26:52.848134+0000","last_active":"2024-03-06T07:26:54.111667+0000","last_peered":"2024-03-06T07:26:54.111667+0000","last_clean":"2024-03-06T07:26:54.111667+0000","last_became_active":"2024-03-06T07:26:52.847615+0000","last_became_peered":"2024-03-06T07:26:52.847615+0000","last_unstale":"2024-03-06T07:26:54.111667+0000","last_undegraded":"2024-03-06T07:26:54.111667+0000","last_fullsized":"2024-03-06T07:26:54.111667+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:41.138418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"171'6","reported_seq":20,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112345+0000","last_change":"2024-03-06T07:26:51.832621+0000","last_active":"2024-03-06T07:26:54.112345+0000","last_peered":"2024-03-06T07:26:54.112345+0000","last_clean":"2024-03-06T07:26:54.112345+0000","last_became_active":"2024-03-06T07:26:51.832323+0000","last_became_peered":"2024-03-06T07:26:51.832323+0000","last_unstale":"2024-03-06T07:26:54.112345+0000","last_undegraded":"2024-03-06T07:26:54.112345+0000","last_fullsized":"2024-03-06T07:26:54.112345+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:19.927308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.830966+0000","last_change":"2024-03-06T07:26:52.849363+0000","last_active":"2024-03-06T07:26:53.830966+0000","last_peered":"2024-03-06T07:26:53.830966+0000","last_clean":"2024-03-06T07:26:53.830966+0000","last_became_active":"2024-03-06T07:26:52.849003+0000","last_became_peered":"2024-03-06T07:26:52.849003+0000","last_unstale":"2024-03-06T07:26:53.830966+0000","last_undegraded":"2024-03-06T07:26:53.830966+0000","last_fullsized":"2024-03-06T07:26:53.830966+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:00:25.516252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831356+0000","last_change":"2024-03-06T07:26:51.835855+0000","last_active":"2024-03-06T07:26:53.831356+0000","last_peered":"2024-03-06T07:26:53.831356+0000","last_clean":"2024-03-06T07:26:53.831356+0000","last_became_active":"2024-03-06T07:26:51.835519+0000","last_became_peered":"2024-03-06T07:26:51.835519+0000","last_unstale":"2024-03-06T07:26:53.831356+0000","last_undegraded":"2024-03-06T07:26:53.831356+0000","last_fullsized":"2024-03-06T07:26:53.831356+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:56.612802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112484+0000","last_change":"2024-03-06T07:26:52.841166+0000","last_active":"2024-03-06T07:26:54.112484+0000","last_peered":"2024-03-06T07:26:54.112484+0000","last_clean":"2024-03-06T07:26:54.112484+0000","last_became_active":"2024-03-06T07:26:52.840844+0000","last_became_peered":"2024-03-06T07:26:52.840844+0000","last_unstale":"2024-03-06T07:26:54.112484+0000","last_undegraded":"2024-03-06T07:26:54.112484+0000","last_fullsized":"2024-03-06T07:26:54.112484+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:03.154423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111645+0000","last_change":"2024-03-06T07:26:51.824784+0000","last_active":"2024-03-06T07:26:54.111645+0000","last_peered":"2024-03-06T07:26:54.111645+0000","last_clean":"2024-03-06T07:26:54.111645+0000","last_became_active":"2024-03-06T07:26:51.824504+0000","last_became_peered":"2024-03-06T07:26:51.824504+0000","last_unstale":"2024-03-06T07:26:54.111645+0000","last_undegraded":"2024-03-06T07:26:54.111645+0000","last_fullsized":"2024-03-06T07:26:54.111645+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:19.927308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112045+0000","last_change":"2024-03-06T07:26:52.844410+0000","last_active":"2024-03-06T07:26:54.112045+0000","last_peered":"2024-03-06T07:26:54.112045+0000","last_clean":"2024-03-06T07:26:54.112045+0000","last_became_active":"2024-03-06T07:26:52.844085+0000","last_became_peered":"2024-03-06T07:26:52.844085+0000","last_unstale":"2024-03-06T07:26:54.112045+0000","last_undegraded":"2024-03-06T07:26:54.112045+0000","last_fullsized":"2024-03-06T07:26:54.112045+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:26:18.994417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111161+0000","last_change":"2024-03-06T07:26:51.834404+0000","last_active":"2024-03-06T07:26:54.111161+0000","last_peered":"2024-03-06T07:26:54.111161+0000","last_clean":"2024-03-06T07:26:54.111161+0000","last_became_active":"2024-03-06T07:26:51.834166+0000","last_became_peered":"2024-03-06T07:26:51.834166+0000","last_unstale":"2024-03-06T07:26:54.111161+0000","last_undegraded":"2024-03-06T07:26:54.111161+0000","last_fullsized":"2024-03-06T07:26:54.111161+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:49.518211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111933+0000","last_change":"2024-03-06T07:26:52.844056+0000","last_active":"2024-03-06T07:26:54.111933+0000","last_peered":"2024-03-06T07:26:54.111933+0000","last_clean":"2024-03-06T07:26:54.111933+0000","last_became_active":"2024-03-06T07:26:52.843754+0000","last_became_peered":"2024-03-06T07:26:52.843754+0000","last_unstale":"2024-03-06T07:26:54.111933+0000","last_undegraded":"2024-03-06T07:26:54.111933+0000","last_fullsized":"2024-03-06T07:26:54.111933+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:52.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"171'8","reported_seq":22,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.110940+0000","last_change":"2024-03-06T07:26:51.829602+0000","last_active":"2024-03-06T07:26:54.110940+0000","last_peered":"2024-03-06T07:26:54.110940+0000","last_clean":"2024-03-06T07:26:54.110940+0000","last_became_active":"2024-03-06T07:26:51.829165+0000","last_became_peered":"2024-03-06T07:26:51.829165+0000","last_unstale":"2024-03-06T07:26:54.110940+0000","last_undegraded":"2024-03-06T07:26:54.110940+0000","last_fullsized":"2024-03-06T07:26:54.110940+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:19.115797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.113177+0000","last_change":"2024-03-06T07:26:52.829644+0000","last_active":"2024-03-06T07:26:54.113177+0000","last_peered":"2024-03-06T07:26:54.113177+0000","last_clean":"2024-03-06T07:26:54.113177+0000","last_became_active":"2024-03-06T07:26:52.829320+0000","last_became_peered":"2024-03-06T07:26:52.829320+0000","last_unstale":"2024-03-06T07:26:54.113177+0000","last_undegraded":"2024-03-06T07:26:54.113177+0000","last_fullsized":"2024-03-06T07:26:54.113177+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:44.836739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112176+0000","last_change":"2024-03-06T07:26:51.836792+0000","last_active":"2024-03-06T07:26:54.112176+0000","last_peered":"2024-03-06T07:26:54.112176+0000","last_clean":"2024-03-06T07:26:54.112176+0000","last_became_active":"2024-03-06T07:26:51.836520+0000","last_became_peered":"2024-03-06T07:26:51.836520+0000","last_unstale":"2024-03-06T07:26:54.112176+0000","last_undegraded":"2024-03-06T07:26:54.112176+0000","last_fullsized":"2024-03-06T07:26:54.112176+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:57.424213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112181+0000","last_change":"2024-03-06T07:26:52.838273+0000","last_active":"2024-03-06T07:26:54.112181+0000","last_peered":"2024-03-06T07:26:54.112181+0000","last_clean":"2024-03-06T07:26:54.112181+0000","last_became_active":"2024-03-06T07:26:52.837925+0000","last_became_peered":"2024-03-06T07:26:52.837925+0000","last_unstale":"2024-03-06T07:26:54.112181+0000","last_undegraded":"2024-03-06T07:26:54.112181+0000","last_fullsized":"2024-03-06T07:26:54.112181+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:56:23.284841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111869+0000","last_change":"2024-03-06T07:26:51.836626+0000","last_active":"2024-03-06T07:26:54.111869+0000","last_peered":"2024-03-06T07:26:54.111869+0000","last_clean":"2024-03-06T07:26:54.111869+0000","last_became_active":"2024-03-06T07:26:51.836177+0000","last_became_peered":"2024-03-06T07:26:51.836177+0000","last_unstale":"2024-03-06T07:26:54.111869+0000","last_undegraded":"2024-03-06T07:26:54.111869+0000","last_fullsized":"2024-03-06T07:26:54.111869+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:03.289934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111589+0000","last_change":"2024-03-06T07:26:52.834881+0000","last_active":"2024-03-06T07:26:54.111589+0000","last_peered":"2024-03-06T07:26:54.111589+0000","last_clean":"2024-03-06T07:26:54.111589+0000","last_became_active":"2024-03-06T07:26:52.834533+0000","last_became_peered":"2024-03-06T07:26:52.834533+0000","last_unstale":"2024-03-06T07:26:54.111589+0000","last_undegraded":"2024-03-06T07:26:54.111589+0000","last_fullsized":"2024-03-06T07:26:54.111589+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:35.066881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831122+0000","last_change":"2024-03-06T07:26:51.827242+0000","last_active":"2024-03-06T07:26:53.831122+0000","last_peered":"2024-03-06T07:26:53.831122+0000","last_clean":"2024-03-06T07:26:53.831122+0000","last_became_active":"2024-03-06T07:26:51.826968+0000","last_became_peered":"2024-03-06T07:26:51.826968+0000","last_unstale":"2024-03-06T07:26:53.831122+0000","last_undegraded":"2024-03-06T07:26:53.831122+0000","last_fullsized":"2024-03-06T07:26:53.831122+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:18.273299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112714+0000","last_change":"2024-03-06T07:26:52.833432+0000","last_active":"2024-03-06T07:26:54.112714+0000","last_peered":"2024-03-06T07:26:54.112714+0000","last_clean":"2024-03-06T07:26:54.112714+0000","last_became_active":"2024-03-06T07:26:52.832887+0000","last_became_peered":"2024-03-06T07:26:52.832887+0000","last_unstale":"2024-03-06T07:26:54.112714+0000","last_undegraded":"2024-03-06T07:26:54.112714+0000","last_fullsized":"2024-03-06T07:26:54.112714+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:49.094931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112389+0000","last_change":"2024-03-06T07:26:51.835241+0000","last_active":"2024-03-06T07:26:54.112389+0000","last_peered":"2024-03-06T07:26:54.112389+0000","last_clean":"2024-03-06T07:26:54.112389+0000","last_became_active":"2024-03-06T07:26:51.834955+0000","last_became_peered":"2024-03-06T07:26:51.834955+0000","last_unstale":"2024-03-06T07:26:54.112389+0000","last_undegraded":"2024-03-06T07:26:54.112389+0000","last_fullsized":"2024-03-06T07:26:54.112389+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:20.370604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831117+0000","last_change":"2024-03-06T07:26:52.827584+0000","last_active":"2024-03-06T07:26:53.831117+0000","last_peered":"2024-03-06T07:26:53.831117+0000","last_clean":"2024-03-06T07:26:53.831117+0000","last_became_active":"2024-03-06T07:26:52.827276+0000","last_became_peered":"2024-03-06T07:26:52.827276+0000","last_unstale":"2024-03-06T07:26:53.831117+0000","last_undegraded":"2024-03-06T07:26:53.831117+0000","last_fullsized":"2024-03-06T07:26:53.831117+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:17.952849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111765+0000","last_change":"2024-03-06T07:26:51.828369+0000","last_active":"2024-03-06T07:26:54.111765+0000","last_peered":"2024-03-06T07:26:54.111765+0000","last_clean":"2024-03-06T07:26:54.111765+0000","last_became_active":"2024-03-06T07:26:51.828087+0000","last_became_peered":"2024-03-06T07:26:51.828087+0000","last_unstale":"2024-03-06T07:26:54.111765+0000","last_undegraded":"2024-03-06T07:26:54.111765+0000","last_fullsized":"2024-03-06T07:26:54.111765+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:51.321721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112601+0000","last_change":"2024-03-06T07:26:52.833175+0000","last_active":"2024-03-06T07:26:54.112601+0000","last_peered":"2024-03-06T07:26:54.112601+0000","last_clean":"2024-03-06T07:26:54.112601+0000","last_became_active":"2024-03-06T07:26:52.832777+0000","last_became_peered":"2024-03-06T07:26:52.832777+0000","last_unstale":"2024-03-06T07:26:54.112601+0000","last_undegraded":"2024-03-06T07:26:54.112601+0000","last_fullsized":"2024-03-06T07:26:54.112601+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:03.154423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.831059+0000","last_change":"2024-03-06T07:26:51.831566+0000","last_active":"2024-03-06T07:26:53.831059+0000","last_peered":"2024-03-06T07:26:53.831059+0000","last_clean":"2024-03-06T07:26:53.831059+0000","last_became_active":"2024-03-06T07:26:51.831112+0000","last_became_peered":"2024-03-06T07:26:51.831112+0000","last_unstale":"2024-03-06T07:26:53.831059+0000","last_undegraded":"2024-03-06T07:26:53.831059+0000","last_fullsized":"2024-03-06T07:26:53.831059+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:45:32.193195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112175+0000","last_change":"2024-03-06T07:26:52.828752+0000","last_active":"2024-03-06T07:26:54.112175+0000","last_peered":"2024-03-06T07:26:54.112175+0000","last_clean":"2024-03-06T07:26:54.112175+0000","last_became_active":"2024-03-06T07:26:52.828460+0000","last_became_peered":"2024-03-06T07:26:52.828460+0000","last_unstale":"2024-03-06T07:26:54.112175+0000","last_undegraded":"2024-03-06T07:26:54.112175+0000","last_fullsized":"2024-03-06T07:26:54.112175+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:45:33.200682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833346+0000","last_change":"2024-03-06T07:26:51.826614+0000","last_active":"2024-03-06T07:26:53.833346+0000","last_peered":"2024-03-06T07:26:53.833346+0000","last_clean":"2024-03-06T07:26:53.833346+0000","last_became_active":"2024-03-06T07:26:51.826341+0000","last_became_peered":"2024-03-06T07:26:51.826341+0000","last_unstale":"2024-03-06T07:26:53.833346+0000","last_undegraded":"2024-03-06T07:26:53.833346+0000","last_fullsized":"2024-03-06T07:26:53.833346+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:07.548977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.113155+0000","last_change":"2024-03-06T07:26:52.838395+0000","last_active":"2024-03-06T07:26:54.113155+0000","last_peered":"2024-03-06T07:26:54.113155+0000","last_clean":"2024-03-06T07:26:54.113155+0000","last_became_active":"2024-03-06T07:26:52.838112+0000","last_became_peered":"2024-03-06T07:26:52.838112+0000","last_unstale":"2024-03-06T07:26:54.113155+0000","last_undegraded":"2024-03-06T07:26:54.113155+0000","last_fullsized":"2024-03-06T07:26:54.113155+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:00:10.384778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111281+0000","last_change":"2024-03-06T07:26:51.837690+0000","last_active":"2024-03-06T07:26:54.111281+0000","last_peered":"2024-03-06T07:26:54.111281+0000","last_clean":"2024-03-06T07:26:54.111281+0000","last_became_active":"2024-03-06T07:26:51.837390+0000","last_became_peered":"2024-03-06T07:26:51.837390+0000","last_unstale":"2024-03-06T07:26:54.111281+0000","last_undegraded":"2024-03-06T07:26:54.111281+0000","last_fullsized":"2024-03-06T07:26:54.111281+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:01.651771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.113327+0000","last_change":"2024-03-06T07:26:52.842712+0000","last_active":"2024-03-06T07:26:54.113327+0000","last_peered":"2024-03-06T07:26:54.113327+0000","last_clean":"2024-03-06T07:26:54.113327+0000","last_became_active":"2024-03-06T07:26:52.842401+0000","last_became_peered":"2024-03-06T07:26:52.842401+0000","last_unstale":"2024-03-06T07:26:54.113327+0000","last_undegraded":"2024-03-06T07:26:54.113327+0000","last_fullsized":"2024-03-06T07:26:54.113327+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:08.556464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112687+0000","last_change":"2024-03-06T07:26:51.832673+0000","last_active":"2024-03-06T07:26:54.112687+0000","last_peered":"2024-03-06T07:26:54.112687+0000","last_clean":"2024-03-06T07:26:54.112687+0000","last_became_active":"2024-03-06T07:26:51.831785+0000","last_became_peered":"2024-03-06T07:26:51.831785+0000","last_unstale":"2024-03-06T07:26:54.112687+0000","last_undegraded":"2024-03-06T07:26:54.112687+0000","last_fullsized":"2024-03-06T07:26:54.112687+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:25.181264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111574+0000","last_change":"2024-03-06T07:26:52.837198+0000","last_active":"2024-03-06T07:26:54.111574+0000","last_peered":"2024-03-06T07:26:54.111574+0000","last_clean":"2024-03-06T07:26:54.111574+0000","last_became_active":"2024-03-06T07:26:52.836899+0000","last_became_peered":"2024-03-06T07:26:52.836899+0000","last_unstale":"2024-03-06T07:26:54.111574+0000","last_undegraded":"2024-03-06T07:26:54.111574+0000","last_fullsized":"2024-03-06T07:26:54.111574+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:52.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"172'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.823777+0000","last_change":"2024-03-06T07:26:51.830363+0000","last_active":"2024-03-06T07:26:54.823777+0000","last_peered":"2024-03-06T07:26:54.823777+0000","last_clean":"2024-03-06T07:26:54.823777+0000","last_became_active":"2024-03-06T07:26:51.830065+0000","last_became_peered":"2024-03-06T07:26:51.830065+0000","last_unstale":"2024-03-06T07:26:54.823777+0000","last_undegraded":"2024-03-06T07:26:54.823777+0000","last_fullsized":"2024-03-06T07:26:54.823777+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:04:02.294443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111319+0000","last_change":"2024-03-06T07:26:52.839745+0000","last_active":"2024-03-06T07:26:54.111319+0000","last_peered":"2024-03-06T07:26:54.111319+0000","last_clean":"2024-03-06T07:26:54.111319+0000","last_became_active":"2024-03-06T07:26:52.839451+0000","last_became_peered":"2024-03-06T07:26:52.839451+0000","last_unstale":"2024-03-06T07:26:54.111319+0000","last_undegraded":"2024-03-06T07:26:54.111319+0000","last_fullsized":"2024-03-06T07:26:54.111319+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:49.560091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:53.833465+0000","last_change":"2024-03-06T07:26:51.829436+0000","last_active":"2024-03-06T07:26:53.833465+0000","last_peered":"2024-03-06T07:26:53.833465+0000","last_clean":"2024-03-06T07:26:53.833465+0000","last_became_active":"2024-03-06T07:26:51.829173+0000","last_became_peered":"2024-03-06T07:26:51.829173+0000","last_unstale":"2024-03-06T07:26:53.833465+0000","last_undegraded":"2024-03-06T07:26:53.833465+0000","last_fullsized":"2024-03-06T07:26:53.833465+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:43.829252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111435+0000","last_change":"2024-03-06T07:26:52.839493+0000","last_active":"2024-03-06T07:26:54.111435+0000","last_peered":"2024-03-06T07:26:54.111435+0000","last_clean":"2024-03-06T07:26:54.111435+0000","last_became_active":"2024-03-06T07:26:52.838980+0000","last_became_peered":"2024-03-06T07:26:52.838980+0000","last_unstale":"2024-03-06T07:26:54.111435+0000","last_undegraded":"2024-03-06T07:26:54.111435+0000","last_fullsized":"2024-03-06T07:26:54.111435+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:13.961992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111474+0000","last_change":"2024-03-06T07:26:51.834526+0000","last_active":"2024-03-06T07:26:54.111474+0000","last_peered":"2024-03-06T07:26:54.111474+0000","last_clean":"2024-03-06T07:26:54.111474+0000","last_became_active":"2024-03-06T07:26:51.834275+0000","last_became_peered":"2024-03-06T07:26:51.834275+0000","last_unstale":"2024-03-06T07:26:54.111474+0000","last_undegraded":"2024-03-06T07:26:54.111474+0000","last_fullsized":"2024-03-06T07:26:54.111474+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:38.916465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112267+0000","last_change":"2024-03-06T07:26:52.839348+0000","last_active":"2024-03-06T07:26:54.112267+0000","last_peered":"2024-03-06T07:26:54.112267+0000","last_clean":"2024-03-06T07:26:54.112267+0000","last_became_active":"2024-03-06T07:26:52.838769+0000","last_became_peered":"2024-03-06T07:26:52.838769+0000","last_unstale":"2024-03-06T07:26:54.112267+0000","last_undegraded":"2024-03-06T07:26:54.112267+0000","last_fullsized":"2024-03-06T07:26:54.112267+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:45.355927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112808+0000","last_change":"2024-03-06T07:26:51.831751+0000","last_active":"2024-03-06T07:26:54.112808+0000","last_peered":"2024-03-06T07:26:54.112808+0000","last_clean":"2024-03-06T07:26:54.112808+0000","last_became_active":"2024-03-06T07:26:51.831349+0000","last_became_peered":"2024-03-06T07:26:51.831349+0000","last_unstale":"2024-03-06T07:26:54.112808+0000","last_undegraded":"2024-03-06T07:26:54.112808+0000","last_fullsized":"2024-03-06T07:26:54.112808+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:25:12.614392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112586+0000","last_change":"2024-03-06T07:26:52.849983+0000","last_active":"2024-03-06T07:26:54.112586+0000","last_peered":"2024-03-06T07:26:54.112586+0000","last_clean":"2024-03-06T07:26:54.112586+0000","last_became_active":"2024-03-06T07:26:52.849675+0000","last_became_peered":"2024-03-06T07:26:52.849675+0000","last_unstale":"2024-03-06T07:26:54.112586+0000","last_undegraded":"2024-03-06T07:26:54.112586+0000","last_fullsized":"2024-03-06T07:26:54.112586+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:38.074300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111622+0000","last_change":"2024-03-06T07:26:51.831817+0000","last_active":"2024-03-06T07:26:54.111622+0000","last_peered":"2024-03-06T07:26:54.111622+0000","last_clean":"2024-03-06T07:26:54.111622+0000","last_became_active":"2024-03-06T07:26:51.831548+0000","last_became_peered":"2024-03-06T07:26:51.831548+0000","last_unstale":"2024-03-06T07:26:54.111622+0000","last_undegraded":"2024-03-06T07:26:54.111622+0000","last_fullsized":"2024-03-06T07:26:54.111622+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:37.066813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.112641+0000","last_change":"2024-03-06T07:26:52.849388+0000","last_active":"2024-03-06T07:26:54.112641+0000","last_peered":"2024-03-06T07:26:54.112641+0000","last_clean":"2024-03-06T07:26:54.112641+0000","last_became_active":"2024-03-06T07:26:52.849107+0000","last_became_peered":"2024-03-06T07:26:52.849107+0000","last_unstale":"2024-03-06T07:26:54.112641+0000","last_undegraded":"2024-03-06T07:26:54.112641+0000","last_fullsized":"2024-03-06T07:26:54.112641+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:05:50.525698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111707+0000","last_change":"2024-03-06T07:26:51.835844+0000","last_active":"2024-03-06T07:26:54.111707+0000","last_peered":"2024-03-06T07:26:54.111707+0000","last_clean":"2024-03-06T07:26:54.111707+0000","last_became_active":"2024-03-06T07:26:51.835560+0000","last_became_peered":"2024-03-06T07:26:51.835560+0000","last_unstale":"2024-03-06T07:26:54.111707+0000","last_undegraded":"2024-03-06T07:26:54.111707+0000","last_fullsized":"2024-03-06T07:26:54.111707+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:25.876174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111041+0000","last_change":"2024-03-06T07:26:52.837237+0000","last_active":"2024-03-06T07:26:54.111041+0000","last_peered":"2024-03-06T07:26:54.111041+0000","last_clean":"2024-03-06T07:26:54.111041+0000","last_became_active":"2024-03-06T07:26:52.836962+0000","last_became_peered":"2024-03-06T07:26:52.836962+0000","last_unstale":"2024-03-06T07:26:54.111041+0000","last_undegraded":"2024-03-06T07:26:54.111041+0000","last_fullsized":"2024-03-06T07:26:54.111041+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:03.380633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111042+0000","last_change":"2024-03-06T07:26:51.829446+0000","last_active":"2024-03-06T07:26:54.111042+0000","last_peered":"2024-03-06T07:26:54.111042+0000","last_clean":"2024-03-06T07:26:54.111042+0000","last_became_active":"2024-03-06T07:26:51.828753+0000","last_became_peered":"2024-03-06T07:26:51.828753+0000","last_unstale":"2024-03-06T07:26:54.111042+0000","last_undegraded":"2024-03-06T07:26:54.111042+0000","last_fullsized":"2024-03-06T07:26:54.111042+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:08.841551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111595+0000","last_change":"2024-03-06T07:26:52.845130+0000","last_active":"2024-03-06T07:26:54.111595+0000","last_peered":"2024-03-06T07:26:54.111595+0000","last_clean":"2024-03-06T07:26:54.111595+0000","last_became_active":"2024-03-06T07:26:52.844790+0000","last_became_peered":"2024-03-06T07:26:52.844790+0000","last_unstale":"2024-03-06T07:26:54.111595+0000","last_undegraded":"2024-03-06T07:26:54.111595+0000","last_fullsized":"2024-03-06T07:26:54.111595+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:51.809521+0000","last_clean_scrub_stamp":"2024-03-06T07:26:51.809521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:37.717248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111536+0000","last_change":"2024-03-06T07:26:51.830404+0000","last_active":"2024-03-06T07:26:54.111536+0000","last_peered":"2024-03-06T07:26:54.111536+0000","last_clean":"2024-03-06T07:26:54.111536+0000","last_became_active":"2024-03-06T07:26:51.830109+0000","last_became_peered":"2024-03-06T07:26:51.830109+0000","last_unstale":"2024-03-06T07:26:54.111536+0000","last_undegraded":"2024-03-06T07:26:54.111536+0000","last_fullsized":"2024-03-06T07:26:54.111536+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:50.802034+0000","last_clean_scrub_stamp":"2024-03-06T07:26:50.802034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:05.891150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":456,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:54.111381+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:26:54.111381+0000","last_peered":"2024-03-06T07:26:54.111381+0000","last_clean":"2024-03-06T07:26:54.111381+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:26:54.111381+0000","last_undegraded":"2024-03-06T07:26:54.111381+0000","last_fullsized":"2024-03-06T07:26:54.111381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2256896,"data_stored":1652848,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2191360,"data_stored":1062992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2183168,"data_stored":1060854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1061380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2187264,"data_stored":1060924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2179072,"data_stored":1060888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2174976,"data_stored":1060798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33372,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338468,"statfs":{"total":96636764160,"available":96602591232,"internally_reserved":0,"allocated":2244608,"data_stored":1651256,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:56.241 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:26:56.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.385+0000 7f800e581640 1 Processor -- start 2024-03-06T07:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.385+0000 7f800e581640 1 -- start start 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.385+0000 7f800e581640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8008156060 0x7f80080c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.385+0000 7f800e581640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80080c5640 0x7f80080c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.385+0000 7f800e581640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080c3f90 con 0x7f80080c3160 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.385+0000 7f800e581640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8008156440 con 0x7f80080c5640 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.385+0000 7f800e581640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8008157030 con 0x7f8008156060 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f800cb1e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80080c5640 0x7f80080c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f800cb1e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80080c5640 0x7f80080c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48456/0 (socket says 172.21.15.26:48456) 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f800cb1e640 1 -- 172.21.15.26:0/3561728049 learned_addr learned my addr 172.21.15.26:0/3561728049 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f8007fff640 1 --2- 172.21.15.26:0/3561728049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8008156060 0x7f80080c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f800cb1e640 1 -- 172.21.15.26:0/3561728049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8008156060 msgr2=0x7f80080c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f800cb1e640 1 --2- 172.21.15.26:0/3561728049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8008156060 0x7f80080c39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f800cb1e640 1 -- 172.21.15.26:0/3561728049 >> v1:172.21.15.117:6789/0 conn(0x7f80080c3160 legacy=0x7f800806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.386+0000 7f800cb1e640 1 -- 172.21.15.26:0/3561728049 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008156700 con 0x7f80080c5640 2024-03-06T07:26:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800cb1e640 1 --2- 172.21.15.26:0/3561728049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80080c5640 0x7f80080c5ad0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8000009e50 tx=0x7f800002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dc6fd734f5acf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f8006ffd640 1 -- 172.21.15.26:0/3561728049 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800002c9e0 con 0x7f80080c5640 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f8006ffd640 1 -- 172.21.15.26:0/3561728049 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800002cb80 con 0x7f80080c5640 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f8006ffd640 1 -- 172.21.15.26:0/3561728049 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000343f0 con 0x7f80080c5640 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 -- 172.21.15.26:0/3561728049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80080c5640 msgr2=0x7f80080c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 --2- 172.21.15.26:0/3561728049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80080c5640 0x7f80080c5ad0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8000009e50 tx=0x7f800002be40 comp rx=0 tx=0).stop 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 -- 172.21.15.26:0/3561728049 shutdown_connections 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 --2- 172.21.15.26:0/3561728049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80080c5640 0x7f80080c5ad0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 --2- 172.21.15.26:0/3561728049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8008156060 0x7f80080c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 -- 172.21.15.26:0/3561728049 >> 172.21.15.26:0/3561728049 conn(0x7f80080a94d0 msgr2=0x7f80080a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 -- 172.21.15.26:0/3561728049 shutdown_connections 2024-03-06T07:26:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.387+0000 7f800e581640 1 -- 172.21.15.26:0/3561728049 wait complete. 2024-03-06T07:26:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.389+0000 7f800e581640 1 Processor -- start 2024-03-06T07:26:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.389+0000 7f800e581640 1 -- start start 2024-03-06T07:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.390+0000 7f800e581640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80080c3160 0x7f80081e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.390+0000 7f800e581640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80080c5640 0x7f80081ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.390+0000 7f800e581640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008156060 0x7f800814e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.390+0000 7f800e581640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c22d0 con 0x7f8008156060 2024-03-06T07:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.390+0000 7f800e581640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008156cc0 con 0x7f80080c5640 2024-03-06T07:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.390+0000 7f800e581640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80081574c0 con 0x7f80080c3160 2024-03-06T07:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.390+0000 7f800cb1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008156060 0x7f800814e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008156060 0x7f800814e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48464/0 (socket says 172.21.15.26:48464) 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 -- 172.21.15.26:0/3798718459 learned_addr learned my addr 172.21.15.26:0/3798718459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f8007fff640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80080c3160 0x7f80081e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f80077fe640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80080c5640 0x7f80081ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 -- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80080c3160 msgr2=0x7f80081e5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80080c3160 0x7f80081e5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 -- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80080c5640 msgr2=0x7f80081ebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80080c5640 0x7f80081ebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 -- 172.21.15.26:0/3798718459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080aa6c0 con 0x7f8008156060 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f80077fe640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80080c5640 0x7f80081ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f8007fff640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80080c3160 0x7f80081e5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800cb1e640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008156060 0x7f800814e150 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f8000009e50 tx=0x7f8000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f8004ff9640 1 -- 172.21.15.26:0/3798718459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000043020 con 0x7f8008156060 2024-03-06T07:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f8004ff9640 1 -- 172.21.15.26:0/3798718459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800003e450 con 0x7f8008156060 2024-03-06T07:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80081e8920 con 0x7f8008156060 2024-03-06T07:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f8004ff9640 1 -- 172.21.15.26:0/3798718459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800003e860 con 0x7f8008156060 2024-03-06T07:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.391+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80081e8ee0 con 0x7f8008156060 2024-03-06T07:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.392+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80081e8730 con 0x7f8008156060 2024-03-06T07:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.393+0000 7f8004ff9640 1 -- 172.21.15.26:0/3798718459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8000046020 con 0x7f8008156060 2024-03-06T07:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.393+0000 7f8004ff9640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff406b500 0x7f7ff406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.393+0000 7f8004ff9640 1 -- 172.21.15.26:0/3798718459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80000aca80 con 0x7f8008156060 2024-03-06T07:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.393+0000 7f8007fff640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff406b500 0x7f7ff406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.394+0000 7f8007fff640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff406b500 0x7f7ff406d9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8006fd0 tx=0x7f7ff8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.397+0000 7f8004ff9640 1 -- 172.21.15.26:0/3798718459 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f800004d070 con 0x7f8008156060 2024-03-06T07:26:56.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.650+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f800804f330 con 0x7f8008156060 2024-03-06T07:26:56.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.652+0000 7f8004ff9640 1 -- 172.21.15.26:0/3798718459 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8000033e40 con 0x7f8008156060 2024-03-06T07:26:56.654 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:26:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:26:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:26:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:26:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:26:56.655 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:26:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff406b500 msgr2=0x7f7ff406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff406b500 0x7f7ff406d9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8006fd0 tx=0x7f7ff8008040 comp rx=0 tx=0).stop 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008156060 msgr2=0x7f800814e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008156060 0x7f800814e150 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f8000009e50 tx=0x7f8000004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 shutdown_connections 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff406b500 0x7f7ff406d9c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008156060 0x7f800814e150 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80080c5640 0x7f80081ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 --2- 172.21.15.26:0/3798718459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80080c3160 0x7f80081e5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 >> 172.21.15.26:0/3798718459 conn(0x7f80080a94d0 msgr2=0x7f8008154760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 shutdown_connections 2024-03-06T07:26:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.654+0000 7f800e581640 1 -- 172.21.15.26:0/3798718459 wait complete. 2024-03-06T07:26:56.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.771+0000 7ff323bc7640 1 Processor -- start 2024-03-06T07:26:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.771+0000 7ff323bc7640 1 -- start start 2024-03-06T07:26:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff323bc7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff31c152920 0x7ff31c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff323bc7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c0c2190 0x7ff31c0c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff323bc7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c06afc0 con 0x7ff31c0c1280 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff323bc7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c156160 con 0x7ff31c152920 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff323bc7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0c1e20 con 0x7ff31c0c2190 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff321963640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff31c152920 0x7ff31c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff321963640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff31c152920 0x7ff31c155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48470/0 (socket says 172.21.15.26:48470) 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff321963640 1 -- 172.21.15.26:0/2534876771 learned_addr learned my addr 172.21.15.26:0/2534876771 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff321963640 1 -- 172.21.15.26:0/2534876771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c0c2190 msgr2=0x7ff31c0c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff321963640 1 --2- 172.21.15.26:0/2534876771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c0c2190 0x7ff31c0c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff321963640 1 -- 172.21.15.26:0/2534876771 >> v1:172.21.15.117:6789/0 conn(0x7ff31c0c1280 legacy=0x7ff31c0c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.773+0000 7ff321963640 1 -- 172.21.15.26:0/2534876771 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c06b2a0 con 0x7ff31c152920 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.774+0000 7ff321963640 1 --2- 172.21.15.26:0/2534876771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c152920 0x7ff31c155c20 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff318009870 tx=0x7ff31802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46303f1d7a3fa59f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.774+0000 7ff320961640 1 -- 172.21.15.26:0/2534876771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318035020 con 0x7ff31c152920 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.774+0000 7ff320961640 1 -- 172.21.15.26:0/2534876771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff318031640 con 0x7ff31c152920 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.774+0000 7ff320961640 1 -- 172.21.15.26:0/2534876771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31802d040 con 0x7ff31c152920 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 -- 172.21.15.26:0/2534876771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c152920 msgr2=0x7ff31c155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 --2- 172.21.15.26:0/2534876771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c152920 0x7ff31c155c20 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff318009870 tx=0x7ff31802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 -- 172.21.15.26:0/2534876771 shutdown_connections 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 --2- 172.21.15.26:0/2534876771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c0c2190 0x7ff31c0c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 --2- 172.21.15.26:0/2534876771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c152920 0x7ff31c155c20 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 -- 172.21.15.26:0/2534876771 >> 172.21.15.26:0/2534876771 conn(0x7ff31c0c4290 msgr2=0x7ff31c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 -- 172.21.15.26:0/2534876771 shutdown_connections 2024-03-06T07:26:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.775+0000 7ff323bc7640 1 -- 172.21.15.26:0/2534876771 wait complete. 2024-03-06T07:26:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.777+0000 7ff323bc7640 1 Processor -- start 2024-03-06T07:26:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.777+0000 7ff323bc7640 1 -- start start 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff323bc7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff31c0c1280 0x7ff31c14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff323bc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c0c2190 0x7ff31c0a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff323bc7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c152920 0x7ff31c157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff323bc7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0c0d60 con 0x7ff31c0c2190 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff323bc7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c06b860 con 0x7ff31c0c1280 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff323bc7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0a7100 con 0x7ff31c152920 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c0c2190 0x7ff31c0a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c0c2190 0x7ff31c0a7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48480/0 (socket says 172.21.15.26:48480) 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 -- 172.21.15.26:0/4218735745 learned_addr learned my addr 172.21.15.26:0/4218735745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff322164640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c152920 0x7ff31c157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321963640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff31c0c1280 0x7ff31c14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 -- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c152920 msgr2=0x7ff31c157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c152920 0x7ff31c157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 -- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff31c0c1280 msgr2=0x7ff31c14c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff31c0c1280 0x7ff31c14c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.778+0000 7ff321162640 1 -- 172.21.15.26:0/4218735745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c06f450 con 0x7ff31c0c2190 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.779+0000 7ff321162640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c0c2190 0x7ff31c0a7e30 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00b4a0 tx=0x7ff30c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.779+0000 7ff3127fc640 1 -- 172.21.15.26:0/4218735745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c014070 con 0x7ff31c0c2190 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.779+0000 7ff3127fc640 1 -- 172.21.15.26:0/4218735745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c010610 con 0x7ff31c0c2190 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.779+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c1eb3a0 con 0x7ff31c0c2190 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.779+0000 7ff3127fc640 1 -- 172.21.15.26:0/4218735745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c010a00 con 0x7ff31c0c2190 2024-03-06T07:26:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.779+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c1eb960 con 0x7ff31c0c2190 2024-03-06T07:26:56.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.780+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c04f330 con 0x7ff31c0c2190 2024-03-06T07:26:56.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.780+0000 7ff3127fc640 1 -- 172.21.15.26:0/4218735745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff30c017070 con 0x7ff31c0c2190 2024-03-06T07:26:56.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.781+0000 7ff3127fc640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff30006b4d0 0x7ff30006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.781+0000 7ff3127fc640 1 -- 172.21.15.26:0/4218735745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff30c089d40 con 0x7ff31c0c2190 2024-03-06T07:26:56.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.781+0000 7ff321963640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff30006b4d0 0x7ff30006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.781+0000 7ff321963640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff30006b4d0 0x7ff30006d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff318036000 tx=0x7ff31803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:56.785+0000 7ff3127fc640 1 -- 172.21.15.26:0/4218735745 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff30c058ac0 con 0x7ff31c0c2190 2024-03-06T07:26:57.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.108+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff31c068460 con 0x7ff31c0c2190 2024-03-06T07:26:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.109+0000 7ff3127fc640 1 -- 172.21.15.26:0/4218735745 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+4990 (secure 0 0 0) 0x7ff30c0583c0 con 0x7ff31c0c2190 2024-03-06T07:26:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:57.111 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":160,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:54.184709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9402,"qdb_cluster":[9402]},"id":13}]} 2024-03-06T07:26:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 160 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff30006b4d0 msgr2=0x7ff30006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff30006b4d0 0x7ff30006d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff318036000 tx=0x7ff31803c040 comp rx=0 tx=0).stop 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c0c2190 msgr2=0x7ff31c0a7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c0c2190 0x7ff31c0a7e30 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00b4a0 tx=0x7ff30c00b960 comp rx=0 tx=0).stop 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 shutdown_connections 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff30006b4d0 0x7ff30006d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff31c152920 0x7ff31c157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff31c0c2190 0x7ff31c0a7e30 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 --2- 172.21.15.26:0/4218735745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff31c0c1280 0x7ff31c14c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 >> 172.21.15.26:0/4218735745 conn(0x7ff31c0c4290 msgr2=0x7ff31c1540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 shutdown_connections 2024-03-06T07:26:57.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.111+0000 7ff323bc7640 1 -- 172.21.15.26:0/4218735745 wait complete. 2024-03-06T07:26:57.528 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 160, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:54.184709+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9402, 'qdb_cluster': [9402]} 2024-03-06T07:26:57.528 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:26:57.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:26:57.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.630+0000 7f4c49b93640 1 Processor -- start 2024-03-06T07:26:57.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.630+0000 7f4c49b93640 1 -- start start 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c49b93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4c440c69f0 0x7f4c440c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c49b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 0x7f4c44155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c49b93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c44155130 con 0x7f4c440c8950 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c49b93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c44155440 con 0x7f4c440c69f0 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c49b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440ad480 con 0x7f4c441558d0 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c42ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4c440c8950 0x7f4c440acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56322/0 (socket says 172.21.15.26:56322) 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c42ffd640 1 -- 172.21.15.26:0/3565448086 learned_addr learned my addr 172.21.15.26:0/3565448086 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838540311 0 0) 0x7f4c38001110 con 0x7f4c440c8950 2024-03-06T07:26:57.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c2c003520 con 0x7f4c440c8950 2024-03-06T07:26:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c43fff640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 0x7f4c44155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c437fe640 1 --2- 172.21.15.26:0/3565448086 >> v2:172.21.15.117:3300/0 conn(0x7f4c440c69f0 0x7f4c440c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428505003 0 0) 0x7f4c2c003520 con 0x7f4c440c8950 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 msgr2=0x7f4c44155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 0x7f4c44155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 >> v2:172.21.15.117:3300/0 conn(0x7f4c440c69f0 msgr2=0x7f4c440c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 --2- 172.21.15.26:0/3565448086 >> v2:172.21.15.117:3300/0 conn(0x7f4c440c69f0 0x7f4c440c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440c5d50 con 0x7f4c440c8950 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4c38001840 con 0x7f4c440c8950 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 >> v1:172.21.15.26:6789/0 conn(0x7f4c440c8950 legacy=0x7f4c440acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0039b0 0x7f4c2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 0x7f4c2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44155440 con 0x7f4c2c004800 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440ad480 con 0x7f4c2c0039b0 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c00bb20 con 0x7f4c441558d0 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.631+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c437fe640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0039b0 0x7f4c2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c43fff640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 0x7f4c2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c42ffd640 1 -- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 msgr2=0x7f4c2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 0x7f4c2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c42ffd640 1 -- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0039b0 msgr2=0x7f4c2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0039b0 0x7f4c2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c42ffd640 1 -- 172.21.15.26:0/3565448086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c003520 con 0x7f4c2c004800 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c43fff640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441558d0 0x7f4c2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.632+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c2c004c80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001050 tx=0x7f4c2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2715abb1e3202132 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800c0e0 con 0x7f4c2c004800 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38030070 con 0x7f4c2c004800 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c427fc640 1 -- 172.21.15.26:0/3565448086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380058d0 con 0x7f4c2c004800 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c49b93640 1 -- 172.21.15.26:0/3565448086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 msgr2=0x7f4c2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c49b93640 1 --2- 172.21.15.26:0/3565448086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c2c004c80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001050 tx=0x7f4c2c005830 comp rx=0 tx=0).stop 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c43fff640 1 -- 172.21.15.26:0/3565448086 reap_dead start 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c49b93640 1 -- 172.21.15.26:0/3565448086 shutdown_connections 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c49b93640 1 -- 172.21.15.26:0/3565448086 >> 172.21.15.26:0/3565448086 conn(0x7f4c440cae70 msgr2=0x7f4c440cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c49b93640 1 -- 172.21.15.26:0/3565448086 shutdown_connections 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.633+0000 7f4c49b93640 1 -- 172.21.15.26:0/3565448086 wait complete. 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.635+0000 7f4c49b93640 1 Processor -- start 2024-03-06T07:26:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.635+0000 7f4c49b93640 1 -- start start 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c49b93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c440c2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c49b93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4414f2c0 0x7f4c44150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c49b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4414f7f0 0x7f4c440af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c49b93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44152a20 con 0x7f4c2c004800 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c49b93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c6310 con 0x7f4c4414f2c0 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c49b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44152ba0 con 0x7f4c4414f7f0 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c440c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c440c2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46102/0 (socket says 172.21.15.26:46102) 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c437fe640 1 -- 172.21.15.26:0/1507553682 learned_addr learned my addr 172.21.15.26:0/1507553682 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4414f2c0 0x7f4c44150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.636+0000 7f4c43fff640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4414f7f0 0x7f4c440af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c437fe640 1 -- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4414f7f0 msgr2=0x7f4c440af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c437fe640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4414f7f0 0x7f4c440af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c437fe640 1 -- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4414f2c0 msgr2=0x7f4c44150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c437fe640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4414f2c0 0x7f4c44150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c437fe640 1 -- 172.21.15.26:0/1507553682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440ae430 con 0x7f4c2c004800 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4414f2c0 0x7f4c44150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c437fe640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c440c2470 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400b850 tx=0x7f4c3400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c2bfff640 1 -- 172.21.15.26:0/1507553682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34018070 con 0x7f4c2c004800 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c2bfff640 1 -- 172.21.15.26:0/1507553682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400be40 con 0x7f4c2c004800 2024-03-06T07:26:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c440c29b0 con 0x7f4c2c004800 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c2bfff640 1 -- 172.21.15.26:0/1507553682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34010c00 con 0x7f4c2c004800 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.637+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c440c4bc0 con 0x7f4c2c004800 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.639+0000 7f4c2bfff640 1 -- 172.21.15.26:0/1507553682 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c3401b070 con 0x7f4c2c004800 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.639+0000 7f4c2bfff640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c3006b500 0x7f4c3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.639+0000 7f4c2bfff640 1 -- 172.21.15.26:0/1507553682 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c3408a7e0 con 0x7f4c2c004800 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.640+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c3006b500 0x7f4c3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.640+0000 7f4c4c77d740 1 Processor -- start 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.640+0000 7f4c4c77d740 1 -- start start 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.640+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c3006b500 0x7f4c3006d9c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4c38034000 tx=0x7f4c38007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be813a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be816330 0x55a4be813f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be8144b0 0x55a4be810680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4be77a6b0 con 0x55a4be8144b0 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4be723a60 con 0x55a4be816330 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4be776750 con 0x55a4be815f50 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be8144b0 0x55a4be810680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be8144b0 0x55a4be810680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46116/0 (socket says 172.21.15.26:46116) 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 -- 172.21.15.26:0/3286191923 learned_addr learned my addr 172.21.15.26:0/3286191923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be816330 0x55a4be813f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c437fe640 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be813a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 -- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 msgr2=0x55a4be813a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be813a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 -- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be816330 msgr2=0x55a4be813f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be816330 0x55a4be813f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 -- 172.21.15.26:0/3286191923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4be741f30 con 0x55a4be8144b0 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c437fe640 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be813a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be816330 0x55a4be813f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c43fff640 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be8144b0 0x55a4be810680 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4c440ca620 tx=0x7f4c3c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfeda2a4b4f2f2e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c297fa640 1 -- 172.21.15.26:0/3286191923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c019020 con 0x55a4be8144b0 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c297fa640 1 -- 172.21.15.26:0/3286191923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c004720 con 0x55a4be8144b0 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.641+0000 7f4c297fa640 1 -- 172.21.15.26:0/3286191923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c0155f0 con 0x55a4be8144b0 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3286191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be8144b0 msgr2=0x55a4be810680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be8144b0 0x55a4be810680 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4c440ca620 tx=0x7f4c3c009c80 comp rx=0 tx=0).stop 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3286191923 shutdown_connections 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be8144b0 0x55a4be810680 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be816330 0x55a4be813f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3286191923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be813a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3286191923 >> 172.21.15.26:0/3286191923 conn(0x55a4be8127d0 msgr2=0x55a4be8373c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3286191923 shutdown_connections 2024-03-06T07:26:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.642+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3286191923 wait complete. 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.643+0000 7f4c4c77d740 1 -- start start 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.643+0000 7f4c4c77d740 1 Processor -- start 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be8366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 0x55a4be8623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be832fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4be838230 con 0x55a4be815f50 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4be736ea0 con 0x55a4be83d8a0 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4be837c70 con 0x55a4be816330 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c4c77d740 10 client.0 _ll_register_callbacks cb 0x55a4be736870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be8366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be8366d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46120/0 (socket says 172.21.15.26:46120) 2024-03-06T07:26:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.644+0000 7f4c437fe640 1 -- 172.21.15.26:0/418021254 learned_addr learned my addr 172.21.15.26:0/418021254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c43fff640 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be832fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 0x55a4be8623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c437fe640 1 -- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 msgr2=0x55a4be8623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c437fe640 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 0x55a4be8623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c437fe640 1 -- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 msgr2=0x55a4be832fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c437fe640 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be832fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c437fe640 1 -- 172.21.15.26:0/418021254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4be737270 con 0x55a4be815f50 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c437fe640 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be8366d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f4c340551a0 tx=0x7f4c34058e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6eac0c5a0a8d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c227fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c227fc640 1 -- 172.21.15.26:0/418021254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34022040 con 0x55a4be815f50 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c227fc640 1 -- 172.21.15.26:0/418021254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c34083d20 con 0x55a4be815f50 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c3c004720 con 0x55a4be815f50 2024-03-06T07:26:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c227fc640 1 -- 172.21.15.26:0/418021254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c34070b60 con 0x55a4be815f50 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c4c77d740 10 client.9543 fetch_fsmap learned FSMap version 160 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=2+}) v3 -- 0x55a4be48f2d0 con 0x55a4be815f50 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.645+0000 7f4c227fc640 1 -- 172.21.15.26:0/418021254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c004720 con 0x55a4be815f50 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c227fc640 1 -- 172.21.15.26:0/418021254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 160) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f4c34058190 con 0x55a4be815f50 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c4c77d740 10 client.9543 fetch_fsmap finished waiting for FSMap version 160 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c4c77d740 10 client.9543 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c4c77d740 20 client.9543 populate_metadata read hostname 'smithi026' 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be838fe0 0x55a4be871030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c4c77d740 4 client.9543 mds_command: new command op to 9402 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 --> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4be787bf0 con 0x55a4be838fe0 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be838fe0 0x55a4be871030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be838fe0 0x55a4be871030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e1f9ce4e2a48f8 server_cookie=d4db4100b9a7b81e in_seq=0 out_seq=0 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.646+0000 7f4c227fc640 10 client.9543 ms_handle_connect on v2:172.21.15.26:6833/156189543 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c227fc640 1 -- 172.21.15.26:0/418021254 <== mds.0 v2:172.21.15.26:6833/156189543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a4be787bf0 con 0x55a4be838fe0 2024-03-06T07:26:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c227fc640 10 client.9543 handle_command_reply: tid=0 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 client.9543 shutdown 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 msgr2=0x55a4be8366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be8366d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f4c340551a0 tx=0x7f4c34058e00 comp rx=0 tx=0).stop 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 shutdown_connections 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be838fe0 0x55a4be871030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be832fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c227fc640 0 client.9543 ms_handle_reset on v2:172.21.15.26:6833/156189543 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 0x55a4be8623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/418021254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be8366d0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 >> 172.21.15.26:0/418021254 conn(0x55a4be8144b0 msgr2=0x55a4be8148e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 shutdown_connections 2024-03-06T07:26:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 1 -- 172.21.15.26:0/418021254 wait complete. 2024-03-06T07:26:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.647+0000 7f4c4c77d740 20 client.9543 trim_cache size 0 max 16384 2024-03-06T07:26:57.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.680+0000 7f4c4c77d740 1 Processor -- start 2024-03-06T07:26:57.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.680+0000 7f4c4c77d740 1 -- start start 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be836ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be816330 0x55a4be8135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be813ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441f76b0 con 0x55a4be816330 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441eeca0 con 0x55a4be83d8a0 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441f7110 con 0x55a4be815f50 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be816330 0x55a4be8135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be816330 0x55a4be8135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46122/0 (socket says 172.21.15.26:46122) 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 -- 172.21.15.26:0/1100787080 learned_addr learned my addr 172.21.15.26:0/1100787080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 -- 172.21.15.26:0/1100787080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 msgr2=0x55a4be836ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/1100787080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be836ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 -- 172.21.15.26:0/1100787080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 msgr2=0x55a4be813ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/1100787080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be813ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 -- 172.21.15.26:0/1100787080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4be5596b0 con 0x55a4be816330 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/1100787080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be816330 0x55a4be8135a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4c3802de40 tx=0x7f4c3802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5232f43455b2cf97 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c28ff9640 1 -- 172.21.15.26:0/1100787080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38001110 con 0x55a4be816330 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c28ff9640 1 -- 172.21.15.26:0/1100787080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38053120 con 0x55a4be816330 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.681+0000 7f4c28ff9640 1 -- 172.21.15.26:0/1100787080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38004e30 con 0x55a4be816330 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 -- 172.21.15.26:0/1100787080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be816330 msgr2=0x55a4be8135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/1100787080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be816330 0x55a4be8135a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4c3802de40 tx=0x7f4c3802e000 comp rx=0 tx=0).stop 2024-03-06T07:26:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 -- 172.21.15.26:0/1100787080 shutdown_connections 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/1100787080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be813ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/1100787080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be816330 0x55a4be8135a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/1100787080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be815f50 0x55a4be836ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 -- 172.21.15.26:0/1100787080 >> 172.21.15.26:0/1100787080 conn(0x55a4be8144b0 msgr2=0x55a4be80be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 -- 172.21.15.26:0/1100787080 shutdown_connections 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.682+0000 7f4c4c77d740 1 -- 172.21.15.26:0/1100787080 wait complete. 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 -- start start 2024-03-06T07:26:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 Processor -- start 2024-03-06T07:26:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be813e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 0x55a4be82f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be86c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4be839e20 con 0x55a4be815f50 2024-03-06T07:26:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4be836e20 con 0x55a4be83d8a0 2024-03-06T07:26:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4be8338b0 con 0x55a4be816330 2024-03-06T07:26:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c4c77d740 10 client.0 _ll_register_callbacks cb 0x7f4c34054fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be813e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be813e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46136/0 (socket says 172.21.15.26:46136) 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.683+0000 7f4c437fe640 1 -- 172.21.15.26:0/3592858925 learned_addr learned my addr 172.21.15.26:0/3592858925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c437fe640 1 -- 172.21.15.26:0/3592858925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 msgr2=0x55a4be82f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c437fe640 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 0x55a4be82f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c437fe640 1 -- 172.21.15.26:0/3592858925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 msgr2=0x55a4be86c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c437fe640 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be86c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c437fe640 1 -- 172.21.15.26:0/3592858925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4be82ffa0 con 0x55a4be815f50 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c437fe640 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be813e90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4c34060fb0 tx=0x7f4c34088a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9ca2b1ca898620 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c227fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c227fc640 1 -- 172.21.15.26:0/3592858925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34022040 con 0x55a4be815f50 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c227fc640 1 -- 172.21.15.26:0/3592858925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c34054ca0 con 0x55a4be815f50 2024-03-06T07:26:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c38053120 con 0x55a4be815f50 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c227fc640 1 -- 172.21.15.26:0/3592858925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c34080050 con 0x55a4be815f50 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 10 client.9549 fetch_fsmap learned FSMap version 160 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=2+}) v3 -- 0x55a4be84f810 con 0x55a4be815f50 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c227fc640 1 -- 172.21.15.26:0/3592858925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38053120 con 0x55a4be815f50 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c227fc640 1 -- 172.21.15.26:0/3592858925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 160) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f4c34057c00 con 0x55a4be815f50 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 10 client.9549 fetch_fsmap finished waiting for FSMap version 160 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 10 client.9549 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 20 client.9549 populate_metadata read hostname 'smithi026' 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be8bf170 0x55a4be8bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 4 client.9549 mds_command: new command op to 9402 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.684+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 --> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a4be781f40 con 0x55a4be8bf170 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be8bf170 0x55a4be8bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c42ffd640 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be8bf170 0x55a4be8bf550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a574b3368b7c61e server_cookie=99aa07bd6d1e9e71 in_seq=0 out_seq=0 2024-03-06T07:26:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c227fc640 10 client.9549 ms_handle_connect on v2:172.21.15.26:6833/156189543 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c227fc640 1 -- 172.21.15.26:0/3592858925 <== mds.0 v2:172.21.15.26:6833/156189543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55a4be781f40 con 0x55a4be8bf170 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c227fc640 10 client.9549 handle_command_reply: tid=0 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 client.9549 shutdown 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 msgr2=0x55a4be813e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be813e90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4c34060fb0 tx=0x7f4c34088a50 comp rx=0 tx=0).stop 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 shutdown_connections 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:6833/156189543,v1:172.21.15.26:6835/156189543] conn(0x55a4be8bf170 0x55a4be8bf550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4be83d8a0 0x55a4be86c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4be816330 0x55a4be82f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 --2- 172.21.15.26:0/3592858925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4be815f50 0x55a4be813e90 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c227fc640 0 client.9549 ms_handle_reset on v2:172.21.15.26:6833/156189543 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 >> 172.21.15.26:0/3592858925 conn(0x55a4be8144b0 msgr2=0x55a4be836ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 shutdown_connections 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 1 -- 172.21.15.26:0/3592858925 wait complete. 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.685+0000 7f4c4c77d740 20 client.9549 trim_cache size 0 max 16384 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9402, 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:26:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:26:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.8693933600000001, 2024-03-06T07:26:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 160, 2024-03-06T07:26:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 172, 2024-03-06T07:26:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 172, 2024-03-06T07:26:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 9.9489517690000007 2024-03-06T07:26:57.695 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c3006b500 msgr2=0x7f4c3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c3006b500 0x7f4c3006d9c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4c38034000 tx=0x7f4c38007000 comp rx=0 tx=0).stop 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 msgr2=0x7f4c440c2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c440c2470 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400b850 tx=0x7f4c3400bd10 comp rx=0 tx=0).stop 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 shutdown_connections 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c3006b500 0x7f4c3006d9c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4414f7f0 0x7f4c440af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4414f2c0 0x7f4c44150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 --2- 172.21.15.26:0/1507553682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c004800 0x7f4c440c2470 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 >> 172.21.15.26:0/1507553682 conn(0x7f4c440cae70 msgr2=0x7f4c44154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 shutdown_connections 2024-03-06T07:26:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.686+0000 7f4c49b93640 1 -- 172.21.15.26:0/1507553682 wait complete. 2024-03-06T07:26:57.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:26:57.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.801+0000 7f9517156640 1 Processor -- start 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.801+0000 7f9517156640 1 -- start start 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f9517156640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f951006d750 0x7f95100732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f9517156640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95100c3a70 0x7f9510072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f9517156640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9510156060 con 0x7f951006db30 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f9517156640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9510155b30 con 0x7f95100c3a70 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f9517156640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95100a8430 con 0x7f951006d750 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f950ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95100c3a70 0x7f9510072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f950ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95100c3a70 0x7f9510072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46148/0 (socket says 172.21.15.26:46148) 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f95156f3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f951006db30 0x7f95100c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56334/0 (socket says 172.21.15.26:56334) 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f950ffff640 1 -- 172.21.15.26:0/2326786145 learned_addr learned my addr 172.21.15.26:0/2326786145 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.802+0000 7f9514ef2640 1 --2- 172.21.15.26:0/2326786145 >> v2:172.21.15.117:3300/0 conn(0x7f951006d750 0x7f95100732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950ffff640 1 -- 172.21.15.26:0/2326786145 >> v2:172.21.15.117:3300/0 conn(0x7f951006d750 msgr2=0x7f95100732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:57.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950ffff640 1 --2- 172.21.15.26:0/2326786145 >> v2:172.21.15.117:3300/0 conn(0x7f951006d750 0x7f95100732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950ffff640 1 -- 172.21.15.26:0/2326786145 >> v1:172.21.15.26:6789/0 conn(0x7f951006db30 legacy=0x7f95100c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950ffff640 1 -- 172.21.15.26:0/2326786145 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9510155d80 con 0x7f95100c3a70 2024-03-06T07:26:57.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950ffff640 1 --2- 172.21.15.26:0/2326786145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f9510072680 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f95100c5440 tx=0x7f94fc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff1e80f85afb43c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950f7fe640 1 -- 172.21.15.26:0/2326786145 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc02e070 con 0x7f95100c3a70 2024-03-06T07:26:57.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950f7fe640 1 -- 172.21.15.26:0/2326786145 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc037070 con 0x7f95100c3a70 2024-03-06T07:26:57.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.803+0000 7f950f7fe640 1 -- 172.21.15.26:0/2326786145 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc031a00 con 0x7f95100c3a70 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 -- 172.21.15.26:0/2326786145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 msgr2=0x7f9510072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 --2- 172.21.15.26:0/2326786145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f9510072680 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f95100c5440 tx=0x7f94fc02b880 comp rx=0 tx=0).stop 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 -- 172.21.15.26:0/2326786145 shutdown_connections 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 --2- 172.21.15.26:0/2326786145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f9510072680 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 --2- 172.21.15.26:0/2326786145 >> v2:172.21.15.117:3300/0 conn(0x7f951006d750 0x7f95100732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 -- 172.21.15.26:0/2326786145 >> 172.21.15.26:0/2326786145 conn(0x7f95100c5560 msgr2=0x7f95100c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 -- 172.21.15.26:0/2326786145 shutdown_connections 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.804+0000 7f9517156640 1 -- 172.21.15.26:0/2326786145 wait complete. 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.805+0000 7f9517156640 1 Processor -- start 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.805+0000 7f9517156640 1 -- start start 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f9517156640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f951006d750 0x7f9510154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f9517156640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f951006db30 0x7f951014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f9517156640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f951014cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f9517156640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9510072bc0 con 0x7f95100c3a70 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f9517156640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95100c27b0 con 0x7f951006db30 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f9517156640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9510072d40 con 0x7f951006d750 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f951014cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f951014cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46152/0 (socket says 172.21.15.26:46152) 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 -- 172.21.15.26:0/3283615104 learned_addr learned my addr 172.21.15.26:0/3283615104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f9514ef2640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f951006d750 0x7f9510154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f950ffff640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f951006db30 0x7f951014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 -- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f951006d750 msgr2=0x7f9510154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f951006d750 0x7f9510154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 -- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f951006db30 msgr2=0x7f951014cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f951006db30 0x7f951014cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.806+0000 7f95156f3640 1 -- 172.21.15.26:0/3283615104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951014db40 con 0x7f95100c3a70 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.807+0000 7f95156f3640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f951014cff0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f950800c730 tx=0x7f950800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.807+0000 7f950d7fa640 1 -- 172.21.15.26:0/3283615104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508018070 con 0x7f95100c3a70 2024-03-06T07:26:57.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.807+0000 7f950d7fa640 1 -- 172.21.15.26:0/3283615104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9508004d60 con 0x7f95100c3a70 2024-03-06T07:26:57.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.807+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95100abf00 con 0x7f95100c3a70 2024-03-06T07:26:57.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.807+0000 7f950d7fa640 1 -- 172.21.15.26:0/3283615104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950800a860 con 0x7f95100c3a70 2024-03-06T07:26:57.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.807+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95100abc20 con 0x7f95100c3a70 2024-03-06T07:26:57.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.808+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95100aedf0 con 0x7f95100c3a70 2024-03-06T07:26:57.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.808+0000 7f950d7fa640 1 -- 172.21.15.26:0/3283615104 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f950801b070 con 0x7f95100c3a70 2024-03-06T07:26:57.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.809+0000 7f950d7fa640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:57.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.809+0000 7f950d7fa640 1 -- 172.21.15.26:0/3283615104 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9508089920 con 0x7f95100c3a70 2024-03-06T07:26:57.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.809+0000 7f9514ef2640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:57.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.810+0000 7f9514ef2640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f95040079a0 tx=0x7f9504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:57.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:57.813+0000 7f950d7fa640 1 -- 172.21.15.26:0/3283615104 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95100aedf0 con 0x7f95100c3a70 2024-03-06T07:26:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.038+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7f951004f330 con 0x7f95100c3a70 2024-03-06T07:26:58.111 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-03-06T07:26:58.110+0000 7fdadff2a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.844+0000 7f950d7fa640 1 -- 172.21.15.26:0/3283615104 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v161) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9508058050 con 0x7f95100c3a70 2024-03-06T07:26:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked down. 2024-03-06T07:26:58.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 msgr2=0x7f94f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f95040079a0 tx=0x7f9504008040 comp rx=0 tx=0).stop 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 msgr2=0x7f951014cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f951014cff0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f950800c730 tx=0x7f950800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 shutdown_connections 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c3a70 0x7f951014cff0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f951006db30 0x7f951014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 --2- 172.21.15.26:0/3283615104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f951006d750 0x7f9510154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 >> 172.21.15.26:0/3283615104 conn(0x7f95100c5560 msgr2=0x7f95100c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 shutdown_connections 2024-03-06T07:26:58.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.847+0000 7f9517156640 1 -- 172.21.15.26:0/3283615104 wait complete. 2024-03-06T07:26:58.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:58.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.959+0000 7f68cbd1e640 1 Processor -- start 2024-03-06T07:26:58.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.959+0000 7f68cbd1e640 1 -- start start 2024-03-06T07:26:58.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68cbd1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 0x7f68c41609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68cbd1e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c41591c0 con 0x7f68c40c5480 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68cbd1e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c41594a0 con 0x7f68c4155ba0 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68cbd1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c406df40 con 0x7f68c40c5860 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68c9aba640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f68c40c5480 0x7f68c406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56342/0 (socket says 172.21.15.26:56342) 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68c9aba640 1 -- 172.21.15.26:0/4156785106 learned_addr learned my addr 172.21.15.26:0/4156785106 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68ca2bb640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 0x7f68c41609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724158018 0 0) 0x7f68c41591c0 con 0x7f68c40c5480 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.960+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68ac003520 con 0x7f68c40c5480 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4167414526 0 0) 0x7f68ac003520 con 0x7f68c40c5480 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 msgr2=0x7f68c41609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 0x7f68c41609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 >> v1:172.21.15.117:6789/0 conn(0x7f68c4155ba0 legacy=0x7f68c41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c41553a0 con 0x7f68c40c5480 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f68bc001670 con 0x7f68c40c5480 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 >> v1:172.21.15.26:6789/0 conn(0x7f68c40c5480 legacy=0x7f68c406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 0x7f68ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c406df40 con 0x7f68ac0039e0 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac00baa0 con 0x7f68ac004d40 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac00bca0 con 0x7f68c40c5860 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c9aba640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c92b9640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 0x7f68ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68ca2bb640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c9aba640 1 -- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 msgr2=0x7f68ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c9aba640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c40c5860 0x7f68ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c9aba640 1 -- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ac004d40 msgr2=0x7f68ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c9aba640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68c9aba640 1 -- 172.21.15.26:0/4156785106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac003520 con 0x7f68ac0039e0 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.961+0000 7f68ca2bb640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.962+0000 7f68c9aba640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68ac003dc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00c190 tx=0x7f68ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c7949d2ec0008b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.962+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc007cc0 con 0x7f68ac0039e0 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.962+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc030070 con 0x7f68ac0039e0 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.962+0000 7f68c8ab8640 1 -- 172.21.15.26:0/4156785106 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc005d10 con 0x7f68ac0039e0 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.963+0000 7f68cbd1e640 1 -- 172.21.15.26:0/4156785106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 msgr2=0x7f68ac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.963+0000 7f68cbd1e640 1 --2- 172.21.15.26:0/4156785106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68ac003dc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00c190 tx=0x7f68ac005830 comp rx=0 tx=0).stop 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.963+0000 7f68ca2bb640 1 -- 172.21.15.26:0/4156785106 reap_dead start 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.963+0000 7f68cbd1e640 1 -- 172.21.15.26:0/4156785106 shutdown_connections 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.963+0000 7f68cbd1e640 1 -- 172.21.15.26:0/4156785106 >> 172.21.15.26:0/4156785106 conn(0x7f68c406b1a0 msgr2=0x7f68c406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.963+0000 7f68cbd1e640 1 -- 172.21.15.26:0/4156785106 shutdown_connections 2024-03-06T07:26:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.963+0000 7f68cbd1e640 1 -- 172.21.15.26:0/4156785106 wait complete. 2024-03-06T07:26:58.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.964+0000 7f68cbd1e640 1 Processor -- start 2024-03-06T07:26:58.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.964+0000 7f68cbd1e640 1 -- start start 2024-03-06T07:26:58.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.964+0000 7f68cbd1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68c41e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.964+0000 7f68cbd1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c4156180 0x7f68c414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68cbd1e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c414c060 0x7f68c41eb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68cbd1e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40c5cf0 con 0x7f68ac0039e0 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68cbd1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c406fdb0 con 0x7f68c4156180 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68cbd1e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4156000 con 0x7f68c414c060 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68c41e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68c41e1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46170/0 (socket says 172.21.15.26:46170) 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 -- 172.21.15.26:0/579397043 learned_addr learned my addr 172.21.15.26:0/579397043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c92b9640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c4156180 0x7f68c414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68ca2bb640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c414c060 0x7f68c41eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 -- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c414c060 msgr2=0x7f68c41eb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c414c060 0x7f68c41eb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 -- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c4156180 msgr2=0x7f68c414bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c4156180 0x7f68c414bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 -- 172.21.15.26:0/579397043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c41e1020 con 0x7f68ac0039e0 2024-03-06T07:26:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68ca2bb640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c414c060 0x7f68c41eb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c92b9640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c4156180 0x7f68c414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c9aba640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68c41e1aa0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f68bc02c450 tx=0x7f68bc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c27fc640 1 -- 172.21.15.26:0/579397043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc045070 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c27fc640 1 -- 172.21.15.26:0/579397043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc02c9e0 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c40ad1c0 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68c27fc640 1 -- 172.21.15.26:0/579397043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc02cdb0 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.965+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c41e1fe0 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.966+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c404f330 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.967+0000 7f68c27fc640 1 -- 172.21.15.26:0/579397043 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68bc007070 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.967+0000 7f68c27fc640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68b006b4d0 0x7f68b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.967+0000 7f68c27fc640 1 -- 172.21.15.26:0/579397043 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68bc0ab5c0 con 0x7f68ac0039e0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.967+0000 7f68c92b9640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68b006b4d0 0x7f68b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.968+0000 7f68c92b9640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68b006b4d0 0x7f68b006d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f68c41569f0 tx=0x7f68b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:58.971+0000 7f68c27fc640 1 -- 172.21.15.26:0/579397043 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68bc07a2b0 con 0x7f68ac0039e0 2024-03-06T07:26:59.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.190+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68c40c6db0 con 0x7f68ac0039e0 2024-03-06T07:26:59.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.191+0000 7f68c27fc640 1 -- 172.21.15.26:0/579397043 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f68bc079bb0 con 0x7f68ac0039e0 2024-03-06T07:26:59.193 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:26:59.193 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:59.193 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68b006b4d0 msgr2=0x7f68b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68b006b4d0 0x7f68b006d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f68c41569f0 tx=0x7f68b8009040 comp rx=0 tx=0).stop 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 msgr2=0x7f68c41e1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68c41e1aa0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f68bc02c450 tx=0x7f68bc0019c0 comp rx=0 tx=0).stop 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 shutdown_connections 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68b006b4d0 0x7f68b006d990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68c414c060 0x7f68c41eb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c4156180 0x7f68c414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 --2- 172.21.15.26:0/579397043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac0039e0 0x7f68c41e1aa0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 >> 172.21.15.26:0/579397043 conn(0x7f68c406b1a0 msgr2=0x7f68c406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 shutdown_connections 2024-03-06T07:26:59.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:26:59.193+0000 7f68cbd1e640 1 -- 172.21.15.26:0/579397043 wait complete. 2024-03-06T07:26:59.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:59.608 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:00.609 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.721+0000 7f2319125640 1 Processor -- start 2024-03-06T07:27:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.721+0000 7f2319125640 1 -- start start 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2319125640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23140c5330 0x7f23140c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2319125640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2314155110 con 0x7f2314158db0 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2319125640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23140c8f40 con 0x7f23140c88b0 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2319125640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23141559e0 con 0x7f23140c5330 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2312d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23140c5330 0x7f23140c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f231359e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2314158db0 0x7f231415fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56354/0 (socket says 172.21.15.26:56354) 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f231359e640 1 -- 172.21.15.26:0/509556789 learned_addr learned my addr 172.21.15.26:0/509556789 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2312d9d640 1 -- 172.21.15.26:0/509556789 >> v1:172.21.15.117:6789/0 conn(0x7f23140c88b0 legacy=0x7f23141586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2312d9d640 1 -- 172.21.15.26:0/509556789 >> v1:172.21.15.26:6789/0 conn(0x7f2314158db0 legacy=0x7f231415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2312d9d640 1 -- 172.21.15.26:0/509556789 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314155390 con 0x7f23140c5330 2024-03-06T07:27:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.722+0000 7f2311d9b640 1 -- 172.21.15.26:0/509556789 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865546255 0 0) 0x7f230c001110 con 0x7f2314158db0 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.723+0000 7f2312d9d640 1 --2- 172.21.15.26:0/509556789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c5330 0x7f23140c5710 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f2304009d90 tx=0x7f230402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fdf92019e6119f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.723+0000 7f2311d9b640 1 -- 172.21.15.26:0/509556789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304039070 con 0x7f23140c5330 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.723+0000 7f2311d9b640 1 -- 172.21.15.26:0/509556789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230402cbd0 con 0x7f23140c5330 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.723+0000 7f2311d9b640 1 -- 172.21.15.26:0/509556789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304034440 con 0x7f23140c5330 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.724+0000 7f2319125640 1 -- 172.21.15.26:0/509556789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c5330 msgr2=0x7f23140c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.724+0000 7f2319125640 1 --2- 172.21.15.26:0/509556789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c5330 0x7f23140c5710 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f2304009d90 tx=0x7f230402be20 comp rx=0 tx=0).stop 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.724+0000 7f2319125640 1 -- 172.21.15.26:0/509556789 shutdown_connections 2024-03-06T07:27:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.724+0000 7f2319125640 1 --2- 172.21.15.26:0/509556789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c5330 0x7f23140c5710 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.724+0000 7f2319125640 1 -- 172.21.15.26:0/509556789 >> 172.21.15.26:0/509556789 conn(0x7f23140ac560 msgr2=0x7f23140ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.724+0000 7f2319125640 1 -- 172.21.15.26:0/509556789 shutdown_connections 2024-03-06T07:27:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.724+0000 7f2319125640 1 -- 172.21.15.26:0/509556789 wait complete. 2024-03-06T07:27:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.725+0000 7f2319125640 1 Processor -- start 2024-03-06T07:27:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f2319125640 1 -- start start 2024-03-06T07:27:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f2319125640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23140c5330 0x7f23141567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f2319125640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c88b0 0x7f23141ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f2319125640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23141e2500 0x7f231414e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f2319125640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141529d0 con 0x7f23140c88b0 2024-03-06T07:27:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f2319125640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231406d090 con 0x7f23140c5330 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f2319125640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2314152b50 con 0x7f23141e2500 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f231259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c88b0 0x7f23141ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f231259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c88b0 0x7f23141ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46198/0 (socket says 172.21.15.26:46198) 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.726+0000 7f231259c640 1 -- 172.21.15.26:0/3400630966 learned_addr learned my addr 172.21.15.26:0/3400630966 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f2312d9d640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23140c5330 0x7f23141567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f231359e640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23141e2500 0x7f231414e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f231259c640 1 -- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23141e2500 msgr2=0x7f231414e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f231259c640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23141e2500 0x7f231414e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f231259c640 1 -- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23140c5330 msgr2=0x7f23141567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f231259c640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23140c5330 0x7f23141567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f231259c640 1 -- 172.21.15.26:0/3400630966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231414d0f0 con 0x7f23140c88b0 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f231259c640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c88b0 0x7f23141ee480 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c400 tx=0x7f22fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f230b7fe640 1 -- 172.21.15.26:0/3400630966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc017070 con 0x7f23140c88b0 2024-03-06T07:27:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f230b7fe640 1 -- 172.21.15.26:0/3400630966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc0035f0 con 0x7f23140c88b0 2024-03-06T07:27:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231414ce10 con 0x7f23140c88b0 2024-03-06T07:27:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f230b7fe640 1 -- 172.21.15.26:0/3400630966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc003a20 con 0x7f23140c88b0 2024-03-06T07:27:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.727+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23141eefa0 con 0x7f23140c88b0 2024-03-06T07:27:00.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.728+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23140cb830 con 0x7f23140c88b0 2024-03-06T07:27:00.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.729+0000 7f230b7fe640 1 -- 172.21.15.26:0/3400630966 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22fc01a070 con 0x7f23140c88b0 2024-03-06T07:27:00.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.729+0000 7f230b7fe640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22f406b500 0x7f22f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.729+0000 7f230b7fe640 1 -- 172.21.15.26:0/3400630966 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22fc089620 con 0x7f23140c88b0 2024-03-06T07:27:00.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.729+0000 7f2312d9d640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22f406b500 0x7f22f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.730+0000 7f2312d9d640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22f406b500 0x7f22f406d9c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2304036000 tx=0x7f2304030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.733+0000 7f230b7fe640 1 -- 172.21.15.26:0/3400630966 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f23140cb830 con 0x7f23140c88b0 2024-03-06T07:27:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.956+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f231404f330 con 0x7f23140c88b0 2024-03-06T07:27:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.956+0000 7f230b7fe640 1 -- 172.21.15.26:0/3400630966 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f22fc057cc0 con 0x7f23140c88b0 2024-03-06T07:27:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:00.958 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:27:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22f406b500 msgr2=0x7f22f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22f406b500 0x7f22f406d9c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2304036000 tx=0x7f2304030000 comp rx=0 tx=0).stop 2024-03-06T07:27:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c88b0 msgr2=0x7f23141ee480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c88b0 0x7f23141ee480 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c400 tx=0x7f22fc00c8c0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 shutdown_connections 2024-03-06T07:27:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22f406b500 0x7f22f406d9c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23141e2500 0x7f231414e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140c88b0 0x7f23141ee480 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 --2- 172.21.15.26:0/3400630966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23140c5330 0x7f23141567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 >> 172.21.15.26:0/3400630966 conn(0x7f23140ac560 msgr2=0x7f23140c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 shutdown_connections 2024-03-06T07:27:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:00.958+0000 7f2319125640 1 -- 172.21.15.26:0/3400630966 wait complete. 2024-03-06T07:27:01.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:01.373 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:02.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:02.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.486+0000 7fadfdec2640 1 Processor -- start 2024-03-06T07:27:02.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.486+0000 7fadfdec2640 1 -- start start 2024-03-06T07:27:02.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadfdec2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fadf8152920 0x7fadf8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadfdec2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fadf80c5330 0x7fadf80c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadfdec2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf8156180 con 0x7fadf80c8a00 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadfdec2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf81587b0 con 0x7fadf80c5330 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadfdec2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf8159630 con 0x7fadf8152920 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fadf80c5330 0x7fadf80c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fadf80c8a00 0x7fadf8158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56362/0 (socket says 172.21.15.26:56362) 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf6ffd640 1 -- 172.21.15.26:0/3367923874 learned_addr learned my addr 172.21.15.26:0/3367923874 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf77fe640 1 --2- 172.21.15.26:0/3367923874 >> v2:172.21.15.117:3300/0 conn(0x7fadf8152920 0x7fadf8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf7fff640 1 -- 172.21.15.26:0/3367923874 >> v2:172.21.15.117:3300/0 conn(0x7fadf8152920 msgr2=0x7fadf8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf7fff640 1 --2- 172.21.15.26:0/3367923874 >> v2:172.21.15.117:3300/0 conn(0x7fadf8152920 0x7fadf8155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf7fff640 1 -- 172.21.15.26:0/3367923874 >> v1:172.21.15.26:6789/0 conn(0x7fadf80c8a00 legacy=0x7fadf8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf67fc640 1 -- 172.21.15.26:0/3367923874 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061146145 0 0) 0x7fadec001110 con 0x7fadf80c8a00 2024-03-06T07:27:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.487+0000 7fadf7fff640 1 -- 172.21.15.26:0/3367923874 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80c7760 con 0x7fadf80c5330 2024-03-06T07:27:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.488+0000 7fadf7fff640 1 --2- 172.21.15.26:0/3367923874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c5330 0x7fadf80c57a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fadf80be490 tx=0x7fadf002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e3122f7bf249c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.488+0000 7fadf67fc640 1 -- 172.21.15.26:0/3367923874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0039070 con 0x7fadf80c5330 2024-03-06T07:27:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.488+0000 7fadf67fc640 1 -- 172.21.15.26:0/3367923874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf002cbf0 con 0x7fadf80c5330 2024-03-06T07:27:02.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.488+0000 7fadf67fc640 1 -- 172.21.15.26:0/3367923874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0034440 con 0x7fadf80c5330 2024-03-06T07:27:02.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 -- 172.21.15.26:0/3367923874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c5330 msgr2=0x7fadf80c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 --2- 172.21.15.26:0/3367923874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c5330 0x7fadf80c57a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fadf80be490 tx=0x7fadf002be40 comp rx=0 tx=0).stop 2024-03-06T07:27:02.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 -- 172.21.15.26:0/3367923874 shutdown_connections 2024-03-06T07:27:02.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 --2- 172.21.15.26:0/3367923874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c5330 0x7fadf80c57a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 --2- 172.21.15.26:0/3367923874 >> v2:172.21.15.117:3300/0 conn(0x7fadf8152920 0x7fadf8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 -- 172.21.15.26:0/3367923874 >> 172.21.15.26:0/3367923874 conn(0x7fadf80ac560 msgr2=0x7fadf80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:02.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 -- 172.21.15.26:0/3367923874 shutdown_connections 2024-03-06T07:27:02.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.489+0000 7fadfdec2640 1 -- 172.21.15.26:0/3367923874 wait complete. 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.490+0000 7fadfdec2640 1 Processor -- start 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.490+0000 7fadfdec2640 1 -- start start 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadfdec2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf80c5330 0x7fadf806bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadfdec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c8a00 0x7fadf814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadfdec2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf8152920 0x7fadf814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadfdec2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8158a00 con 0x7fadf80c8a00 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadfdec2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80c7d20 con 0x7fadf80c5330 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadfdec2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf806d030 con 0x7fadf8152920 2024-03-06T07:27:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c8a00 0x7fadf814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c8a00 0x7fadf814bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46224/0 (socket says 172.21.15.26:46224) 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 -- 172.21.15.26:0/1305744489 learned_addr learned my addr 172.21.15.26:0/1305744489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf77fe640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf80c5330 0x7fadf806bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf7fff640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf8152920 0x7fadf814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 -- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf8152920 msgr2=0x7fadf814c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf8152920 0x7fadf814c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 -- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf80c5330 msgr2=0x7fadf806bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf80c5330 0x7fadf806bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 -- 172.21.15.26:0/1305744489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80c2630 con 0x7fadf80c8a00 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.491+0000 7fadf6ffd640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c8a00 0x7fadf814bc40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c460 tx=0x7fadec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.492+0000 7faddffff640 1 -- 172.21.15.26:0/1305744489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec018070 con 0x7fadf80c8a00 2024-03-06T07:27:02.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.492+0000 7faddffff640 1 -- 172.21.15.26:0/1305744489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec007500 con 0x7fadf80c8a00 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.492+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf8156700 con 0x7fadf80c8a00 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.492+0000 7faddffff640 1 -- 172.21.15.26:0/1305744489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec004a20 con 0x7fadf80c8a00 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.492+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf806c0d0 con 0x7fadf80c8a00 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.493+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8156c20 con 0x7fadf80c8a00 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.493+0000 7faddffff640 1 -- 172.21.15.26:0/1305744489 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadec01b070 con 0x7fadf80c8a00 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.494+0000 7faddffff640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fade406b4d0 0x7fade406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.494+0000 7faddffff640 1 -- 172.21.15.26:0/1305744489 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fadec089520 con 0x7fadf80c8a00 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.494+0000 7fadf77fe640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fade406b4d0 0x7fade406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.495+0000 7fadf77fe640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fade406b4d0 0x7fade406d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fade80079a0 tx=0x7fade8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.498+0000 7faddffff640 1 -- 172.21.15.26:0/1305744489 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fadf8156c20 con 0x7fadf80c8a00 2024-03-06T07:27:02.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.718+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadf81564f0 con 0x7fadf80c8a00 2024-03-06T07:27:02.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.719+0000 7faddffff640 1 -- 172.21.15.26:0/1305744489 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7fadf8156180 con 0x7fadf80c8a00 2024-03-06T07:27:02.720 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:02.721 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:02.721 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fade406b4d0 msgr2=0x7fade406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fade406b4d0 0x7fade406d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fade80079a0 tx=0x7fade8008040 comp rx=0 tx=0).stop 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c8a00 msgr2=0x7fadf814bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c8a00 0x7fadf814bc40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c460 tx=0x7fadec00c920 comp rx=0 tx=0).stop 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 shutdown_connections 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fade406b4d0 0x7fade406d990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf8152920 0x7fadf814c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf80c8a00 0x7fadf814bc40 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 --2- 172.21.15.26:0/1305744489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf80c5330 0x7fadf806bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 >> 172.21.15.26:0/1305744489 conn(0x7fadf80ac560 msgr2=0x7fadf814d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:02.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 shutdown_connections 2024-03-06T07:27:02.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:02.721+0000 7fadfdec2640 1 -- 172.21.15.26:0/1305744489 wait complete. 2024-03-06T07:27:03.136 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:03.136 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:04.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:04.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.250+0000 7f643324e640 1 Processor -- start 2024-03-06T07:27:04.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.250+0000 7f643324e640 1 -- start start 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f643324e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f642c06f7b0 0x7f642c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f643324e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c157340 con 0x7f642c072720 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f643324e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c156d80 con 0x7f642c1506f0 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f643324e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c150b60 con 0x7f642c06f7b0 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f642bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f642c06f7b0 0x7f642c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f6430fea640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f642c072720 0x7f642c156430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56368/0 (socket says 172.21.15.26:56368) 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f6430fea640 1 -- 172.21.15.26:0/1057648229 learned_addr learned my addr 172.21.15.26:0/1057648229 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f642bfff640 1 -- 172.21.15.26:0/1057648229 >> v1:172.21.15.117:6789/0 conn(0x7f642c1506f0 legacy=0x7f642c0c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f642bfff640 1 -- 172.21.15.26:0/1057648229 >> v1:172.21.15.26:6789/0 conn(0x7f642c072720 legacy=0x7f642c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.251+0000 7f642bfff640 1 -- 172.21.15.26:0/1057648229 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c157060 con 0x7f642c06f7b0 2024-03-06T07:27:04.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.252+0000 7f642bfff640 1 --2- 172.21.15.26:0/1057648229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c1501b0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f64180097a0 tx=0x7f641802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a45f46c644c8977 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.252+0000 7f642b7fe640 1 -- 172.21.15.26:0/1057648229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180315a0 con 0x7f642c06f7b0 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.252+0000 7f642b7fe640 1 -- 172.21.15.26:0/1057648229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6418037070 con 0x7f642c06f7b0 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.252+0000 7f642b7fe640 1 -- 172.21.15.26:0/1057648229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418031960 con 0x7f642c06f7b0 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.253+0000 7f643324e640 1 -- 172.21.15.26:0/1057648229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 msgr2=0x7f642c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.253+0000 7f643324e640 1 --2- 172.21.15.26:0/1057648229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c1501b0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f64180097a0 tx=0x7f641802b840 comp rx=0 tx=0).stop 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.253+0000 7f643324e640 1 -- 172.21.15.26:0/1057648229 shutdown_connections 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.253+0000 7f643324e640 1 --2- 172.21.15.26:0/1057648229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c1501b0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.253+0000 7f643324e640 1 -- 172.21.15.26:0/1057648229 >> 172.21.15.26:0/1057648229 conn(0x7f642c0aa630 msgr2=0x7f642c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.253+0000 7f643324e640 1 -- 172.21.15.26:0/1057648229 shutdown_connections 2024-03-06T07:27:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.253+0000 7f643324e640 1 -- 172.21.15.26:0/1057648229 wait complete. 2024-03-06T07:27:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.254+0000 7f643324e640 1 Processor -- start 2024-03-06T07:27:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.254+0000 7f643324e640 1 -- start start 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f643324e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f643324e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f642c072720 0x7f642c154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f643324e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f642c1506f0 0x7f642c154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f643324e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0c3740 con 0x7f642c06f7b0 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f643324e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c156b40 con 0x7f642c072720 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f643324e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c072b00 con 0x7f642c1506f0 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c0c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46246/0 (socket says 172.21.15.26:46246) 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 -- 172.21.15.26:0/3123229639 learned_addr learned my addr 172.21.15.26:0/3123229639 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 -- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f642c1506f0 msgr2=0x7f642c154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f642bfff640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f642c072720 0x7f642c154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f642c1506f0 0x7f642c154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 -- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f642c072720 msgr2=0x7f642c154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f642c072720 0x7f642c154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 -- 172.21.15.26:0/3123229639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0c8390 con 0x7f642c06f7b0 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f6430fea640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c0c7730 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f642000c400 tx=0x7f642000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.255+0000 7f64297fa640 1 -- 172.21.15.26:0/3123229639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420017070 con 0x7f642c06f7b0 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.256+0000 7f64297fa640 1 -- 172.21.15.26:0/3123229639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6420005ce0 con 0x7f642c06f7b0 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.256+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c155d50 con 0x7f642c06f7b0 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.256+0000 7f64297fa640 1 -- 172.21.15.26:0/3123229639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64200072f0 con 0x7f642c06f7b0 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.256+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c156310 con 0x7f642c06f7b0 2024-03-06T07:27:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.256+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c04f330 con 0x7f642c06f7b0 2024-03-06T07:27:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.257+0000 7f64297fa640 1 -- 172.21.15.26:0/3123229639 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f642001a070 con 0x7f642c06f7b0 2024-03-06T07:27:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.257+0000 7f64297fa640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f641006b500 0x7f641006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.257+0000 7f64297fa640 1 -- 172.21.15.26:0/3123229639 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f64200896f0 con 0x7f642c06f7b0 2024-03-06T07:27:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.258+0000 7f642bfff640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f641006b500 0x7f641006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.258+0000 7f642bfff640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f641006b500 0x7f641006d9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6418036000 tx=0x7f6418008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.262+0000 7f64297fa640 1 -- 172.21.15.26:0/3123229639 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f642c0c8390 con 0x7f642c06f7b0 2024-03-06T07:27:04.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.480+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f642c14f2d0 con 0x7f642c06f7b0 2024-03-06T07:27:04.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.480+0000 7f64297fa640 1 -- 172.21.15.26:0/3123229639 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f6420058470 con 0x7f642c06f7b0 2024-03-06T07:27:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:04.482 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:04.483 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:27:04.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f641006b500 msgr2=0x7f641006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f641006b500 0x7f641006d9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6418036000 tx=0x7f6418008030 comp rx=0 tx=0).stop 2024-03-06T07:27:04.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 msgr2=0x7f642c0c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c0c7730 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f642000c400 tx=0x7f642000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 shutdown_connections 2024-03-06T07:27:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f641006b500 0x7f641006d9c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f642c1506f0 0x7f642c154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f642c072720 0x7f642c154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 --2- 172.21.15.26:0/3123229639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f642c06f7b0 0x7f642c0c7730 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 >> 172.21.15.26:0/3123229639 conn(0x7f642c0aa630 msgr2=0x7f642c0c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 shutdown_connections 2024-03-06T07:27:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:04.482+0000 7f643324e640 1 -- 172.21.15.26:0/3123229639 wait complete. 2024-03-06T07:27:04.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:04.898 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:05.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:06.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.011+0000 7f51c5135640 1 Processor -- start 2024-03-06T07:27:06.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.011+0000 7f51c5135640 1 -- start start 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51c5135640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51c00c88b0 0x7f51c00c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51c5135640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c0155110 con 0x7f51c00c5330 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51c5135640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c00c8f40 con 0x7f51c0158db0 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51c5135640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c01559e0 con 0x7f51c00c88b0 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51bed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51c00c5330 0x7f51c01586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56378/0 (socket says 172.21.15.26:56378) 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51bed9d640 1 -- 172.21.15.26:0/3946439864 learned_addr learned my addr 172.21.15.26:0/3946439864 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050152528 0 0) 0x7f51b8001110 con 0x7f51c00c5330 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51ac003520 con 0x7f51c00c5330 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.012+0000 7f51bf59e640 1 --2- 172.21.15.26:0/3946439864 >> v2:172.21.15.117:3300/0 conn(0x7f51c00c88b0 0x7f51c00c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1607859643 0 0) 0x7f51ac003520 con 0x7f51c00c5330 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 >> v2:172.21.15.117:3300/0 conn(0x7f51c00c88b0 msgr2=0x7f51c00c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 --2- 172.21.15.26:0/3946439864 >> v2:172.21.15.117:3300/0 conn(0x7f51c00c88b0 0x7f51c00c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 >> v1:172.21.15.117:6789/0 conn(0x7f51c0158db0 legacy=0x7f51c015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c0155390 con 0x7f51c00c5330 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f51b8001840 con 0x7f51c00c5330 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 >> v1:172.21.15.26:6789/0 conn(0x7f51c00c5330 legacy=0x7f51c01586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51ac0039b0 0x7f51ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac004800 0x7f51ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51ac0051c0 0x7f51ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01559e0 con 0x7f51ac0039b0 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac00c160 con 0x7f51ac004800 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac00c360 con 0x7f51ac0051c0 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bed9d640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51ac0039b0 0x7f51ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51be59c640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac004800 0x7f51ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bf59e640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51ac0051c0 0x7f51ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bed9d640 1 -- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51ac0051c0 msgr2=0x7f51ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bed9d640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51ac0051c0 0x7f51ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bed9d640 1 -- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac004800 msgr2=0x7f51ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bed9d640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac004800 0x7f51ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bed9d640 1 -- 172.21.15.26:0/3946439864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac003520 con 0x7f51ac0039b0 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bf59e640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51ac0051c0 0x7f51ac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51be59c640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac004800 0x7f51ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.013+0000 7f51bf59e640 1 -- 172.21.15.26:0/3946439864 reap_dead start 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51bed9d640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51ac0039b0 0x7f51ac003dd0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f51b800c4c0 tx=0x7f51ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c236615bf7a2978 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800c0e0 con 0x7f51ac0039b0 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b8030070 con 0x7f51ac0039b0 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51bdd9b640 1 -- 172.21.15.26:0/3946439864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80058d0 con 0x7f51ac0039b0 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51c5135640 1 -- 172.21.15.26:0/3946439864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51ac0039b0 msgr2=0x7f51ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51c5135640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51ac0039b0 0x7f51ac003dd0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f51b800c4c0 tx=0x7f51ac005830 comp rx=0 tx=0).stop 2024-03-06T07:27:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51c5135640 1 -- 172.21.15.26:0/3946439864 shutdown_connections 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51c5135640 1 --2- 172.21.15.26:0/3946439864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51ac0039b0 0x7f51ac003dd0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.014+0000 7f51c5135640 1 -- 172.21.15.26:0/3946439864 >> 172.21.15.26:0/3946439864 conn(0x7f51c00ac560 msgr2=0x7f51c00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.015+0000 7f51c5135640 1 -- 172.21.15.26:0/3946439864 shutdown_connections 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.015+0000 7f51c5135640 1 -- 172.21.15.26:0/3946439864 wait complete. 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.016+0000 7f51c5135640 1 Processor -- start 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.016+0000 7f51c5135640 1 -- start start 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51c5135640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac0039b0 0x7f51c01ee790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51c5135640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c00c1370 0x7f51c01eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51c5135640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51c00c1a80 0x7f51c014ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51c5135640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01529d0 con 0x7f51c00c1a80 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51c5135640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c006d090 con 0x7f51ac0039b0 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51c5135640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0152b50 con 0x7f51c00c1370 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51c00c1a80 0x7f51c014ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51c00c1a80 0x7f51c014ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46256/0 (socket says 172.21.15.26:46256) 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 -- 172.21.15.26:0/4115009324 learned_addr learned my addr 172.21.15.26:0/4115009324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 -- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c00c1370 msgr2=0x7f51c01eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c00c1370 0x7f51c01eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bed9d640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac0039b0 0x7f51c01ee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 -- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac0039b0 msgr2=0x7f51c01ee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac0039b0 0x7f51c01ee790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 -- 172.21.15.26:0/4115009324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c014e9c0 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51bf59e640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51c00c1a80 0x7f51c014ba40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f51c00aa480 tx=0x7f51b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.017+0000 7f51b77fe640 1 -- 172.21.15.26:0/4115009324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b001a020 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.018+0000 7f51b77fe640 1 -- 172.21.15.26:0/4115009324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0011d70 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.018+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c014ec30 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.018+0000 7f51b77fe640 1 -- 172.21.15.26:0/4115009324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b00166b0 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.018+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c014bf80 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.019+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c004f330 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.019+0000 7f51b77fe640 1 -- 172.21.15.26:0/4115009324 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51b001d020 con 0x7f51c00c1a80 2024-03-06T07:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.020+0000 7f51b77fe640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51a006b4d0 0x7f51a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.020+0000 7f51b77fe640 1 -- 172.21.15.26:0/4115009324 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51b008afb0 con 0x7f51c00c1a80 2024-03-06T07:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.020+0000 7f51bed9d640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51a006b4d0 0x7f51a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.020+0000 7f51bed9d640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51a006b4d0 0x7f51a006d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f51b8034000 tx=0x7f51b8039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.024+0000 7f51b77fe640 1 -- 172.21.15.26:0/4115009324 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f51b0024020 con 0x7f51c00c1a80 2024-03-06T07:27:06.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.242+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51c01582d0 con 0x7f51c00c1a80 2024-03-06T07:27:06.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.243+0000 7f51b77fe640 1 -- 172.21.15.26:0/4115009324 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f51b0059d30 con 0x7f51c00c1a80 2024-03-06T07:27:06.244 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:06.245 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:06.245 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:27:06.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51a006b4d0 msgr2=0x7f51a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51a006b4d0 0x7f51a006d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f51b8034000 tx=0x7f51b8039040 comp rx=0 tx=0).stop 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51c00c1a80 msgr2=0x7f51c014ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51c00c1a80 0x7f51c014ba40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f51c00aa480 tx=0x7f51b0009f90 comp rx=0 tx=0).stop 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 shutdown_connections 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f51a006b4d0 0x7f51a006d990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51c00c1a80 0x7f51c014ba40 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c00c1370 0x7f51c01eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 --2- 172.21.15.26:0/4115009324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51ac0039b0 0x7f51c01ee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 >> 172.21.15.26:0/4115009324 conn(0x7f51c00ac560 msgr2=0x7f51c00c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 shutdown_connections 2024-03-06T07:27:06.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:06.245+0000 7f51c5135640 1 -- 172.21.15.26:0/4115009324 wait complete. 2024-03-06T07:27:06.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:06.660 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:07.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.772+0000 7fc387439640 1 Processor -- start 2024-03-06T07:27:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.772+0000 7fc387439640 1 -- start start 2024-03-06T07:27:07.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc387439640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc38014f360 0x7fc380155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc387439640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3800c88c0 0x7fc3800c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc387439640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c5880 0x7fc380158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc387439640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3800c8dd0 con 0x7fc3800c88c0 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc387439640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3800c73d0 con 0x7fc38014f360 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc387439640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3800be2e0 con 0x7fc3800c5880 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc3849d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3800c88c0 0x7fc3800c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc3849d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3800c88c0 0x7fc3800c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38350/0 (socket says 172.21.15.26:38350) 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.773+0000 7fc3849d4640 1 -- 172.21.15.26:0/591178056 learned_addr learned my addr 172.21.15.26:0/591178056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:07.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3859d6640 1 --2- 172.21.15.26:0/591178056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c5880 0x7fc380158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3851d5640 1 --2- 172.21.15.26:0/591178056 >> v2:172.21.15.117:3300/0 conn(0x7fc38014f360 0x7fc380155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3849d4640 1 -- 172.21.15.26:0/591178056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c5880 msgr2=0x7fc380158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:07.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3849d4640 1 --2- 172.21.15.26:0/591178056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c5880 0x7fc380158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3849d4640 1 -- 172.21.15.26:0/591178056 >> v2:172.21.15.117:3300/0 conn(0x7fc38014f360 msgr2=0x7fc380155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3849d4640 1 --2- 172.21.15.26:0/591178056 >> v2:172.21.15.117:3300/0 conn(0x7fc38014f360 0x7fc380155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3849d4640 1 -- 172.21.15.26:0/591178056 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800c7670 con 0x7fc3800c88c0 2024-03-06T07:27:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3859d6640 1 --2- 172.21.15.26:0/591178056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c5880 0x7fc380158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:07.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.774+0000 7fc3851d5640 1 --2- 172.21.15.26:0/591178056 >> v2:172.21.15.117:3300/0 conn(0x7fc38014f360 0x7fc380155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc3849d4640 1 --2- 172.21.15.26:0/591178056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c88c0 0x7fc3800c5340 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fc38008c840 tx=0x7fc37402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38838c21a6f7a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc37ffff640 1 -- 172.21.15.26:0/591178056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37402e070 con 0x7fc3800c88c0 2024-03-06T07:27:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc37ffff640 1 -- 172.21.15.26:0/591178056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc374037070 con 0x7fc3800c88c0 2024-03-06T07:27:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc37ffff640 1 -- 172.21.15.26:0/591178056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374031a00 con 0x7fc3800c88c0 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 -- 172.21.15.26:0/591178056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c88c0 msgr2=0x7fc3800c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 --2- 172.21.15.26:0/591178056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c88c0 0x7fc3800c5340 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fc38008c840 tx=0x7fc37402b880 comp rx=0 tx=0).stop 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 -- 172.21.15.26:0/591178056 shutdown_connections 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 --2- 172.21.15.26:0/591178056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c5880 0x7fc380158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 --2- 172.21.15.26:0/591178056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c88c0 0x7fc3800c5340 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 --2- 172.21.15.26:0/591178056 >> v2:172.21.15.117:3300/0 conn(0x7fc38014f360 0x7fc380155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 -- 172.21.15.26:0/591178056 >> 172.21.15.26:0/591178056 conn(0x7fc3800ac570 msgr2=0x7fc3800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 -- 172.21.15.26:0/591178056 shutdown_connections 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.775+0000 7fc387439640 1 -- 172.21.15.26:0/591178056 wait complete. 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.776+0000 7fc387439640 1 Processor -- start 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc387439640 1 -- start start 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc387439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c5880 0x7fc3800bf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc387439640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c88c0 0x7fc380147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc387439640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc38014f360 0x7fc380148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc387439640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3801590a0 con 0x7fc3800c5880 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc387439640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3800c7c30 con 0x7fc38014f360 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc387439640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc380159220 con 0x7fc3800c88c0 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc3851d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c5880 0x7fc3800bf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc3851d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c5880 0x7fc3800bf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38354/0 (socket says 172.21.15.26:38354) 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc3851d5640 1 -- 172.21.15.26:0/1932818790 learned_addr learned my addr 172.21.15.26:0/1932818790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc3849d4640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c88c0 0x7fc380147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.777+0000 7fc3859d6640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc38014f360 0x7fc380148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc3851d5640 1 -- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c88c0 msgr2=0x7fc380147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc3851d5640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c88c0 0x7fc380147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc3851d5640 1 -- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc38014f360 msgr2=0x7fc380148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc3851d5640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc38014f360 0x7fc380148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc3851d5640 1 -- 172.21.15.26:0/1932818790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800bef10 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc3849d4640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c88c0 0x7fc380147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc3851d5640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c5880 0x7fc3800bf6c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fc37800c7f0 tx=0x7fc37800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc37dffb640 1 -- 172.21.15.26:0/1932818790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378015070 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc37dffb640 1 -- 172.21.15.26:0/1932818790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378007820 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3800bec70 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc37dffb640 1 -- 172.21.15.26:0/1932818790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378007bd0 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.778+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3800bfc00 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.779+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38004f330 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.780+0000 7fc37dffb640 1 -- 172.21.15.26:0/1932818790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc378018070 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.780+0000 7fc37dffb640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc36c06b4d0 0x7fc36c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.780+0000 7fc37dffb640 1 -- 172.21.15.26:0/1932818790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc37808a380 con 0x7fc3800c5880 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.780+0000 7fc3849d4640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc36c06b4d0 0x7fc36c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.781+0000 7fc3849d4640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc36c06b4d0 0x7fc36c06d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc374036000 tx=0x7fc37403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:07.784+0000 7fc37dffb640 1 -- 172.21.15.26:0/1932818790 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc378058ab0 con 0x7fc3800c5880 2024-03-06T07:27:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.006+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3800b0320 con 0x7fc3800c5880 2024-03-06T07:27:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.007+0000 7fc37dffb640 1 -- 172.21.15.26:0/1932818790 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7fc37805cb50 con 0x7fc3800c5880 2024-03-06T07:27:08.008 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:08.009 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:08.009 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc36c06b4d0 msgr2=0x7fc36c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc36c06b4d0 0x7fc36c06d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc374036000 tx=0x7fc37403d040 comp rx=0 tx=0).stop 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c5880 msgr2=0x7fc3800bf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c5880 0x7fc3800bf6c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fc37800c7f0 tx=0x7fc37800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 shutdown_connections 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc36c06b4d0 0x7fc36c06d990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc38014f360 0x7fc380148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3800c88c0 0x7fc380147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 --2- 172.21.15.26:0/1932818790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3800c5880 0x7fc3800bf6c0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 >> 172.21.15.26:0/1932818790 conn(0x7fc3800ac570 msgr2=0x7fc3800c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 shutdown_connections 2024-03-06T07:27:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:08.009+0000 7fc387439640 1 -- 172.21.15.26:0/1932818790 wait complete. 2024-03-06T07:27:08.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:08.424 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:09.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.537+0000 7ffa9f837640 1 Processor -- start 2024-03-06T07:27:09.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.537+0000 7ffa9f837640 1 -- start start 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9f837640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa98152520 0x7ffa9806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9f837640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa98155c80 0x7ffa980c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9f837640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa980c1ae0 con 0x7ffa980720a0 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9f837640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa980c1dc0 con 0x7ffa98152520 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9f837640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9806e450 con 0x7ffa98155c80 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9d5d3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa98152520 0x7ffa9806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9ddd4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffa980720a0 0x7ffa9806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38750/0 (socket says 172.21.15.26:38750) 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9ddd4640 1 -- 172.21.15.26:0/339277263 learned_addr learned my addr 172.21.15.26:0/339277263 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:09.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9cdd2640 1 --2- 172.21.15.26:0/339277263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa98155c80 0x7ffa980c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9d5d3640 1 -- 172.21.15.26:0/339277263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa98155c80 msgr2=0x7ffa980c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9d5d3640 1 --2- 172.21.15.26:0/339277263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa98155c80 0x7ffa980c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9d5d3640 1 -- 172.21.15.26:0/339277263 >> v1:172.21.15.26:6789/0 conn(0x7ffa980720a0 legacy=0x7ffa9806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa9d5d3640 1 -- 172.21.15.26:0/339277263 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9806e730 con 0x7ffa98152520 2024-03-06T07:27:09.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.538+0000 7ffa8ffff640 1 -- 172.21.15.26:0/339277263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725182664 0 0) 0x7ffa980c1ae0 con 0x7ffa980720a0 2024-03-06T07:27:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.539+0000 7ffa9d5d3640 1 --2- 172.21.15.26:0/339277263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98152520 0x7ffa9806ef30 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ffa940097a0 tx=0x7ffa9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9c94020f0331f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.539+0000 7ffa8ffff640 1 -- 172.21.15.26:0/339277263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94035020 con 0x7ffa98152520 2024-03-06T07:27:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.539+0000 7ffa8ffff640 1 -- 172.21.15.26:0/339277263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa94031830 con 0x7ffa98152520 2024-03-06T07:27:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.539+0000 7ffa8ffff640 1 -- 172.21.15.26:0/339277263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94031c00 con 0x7ffa98152520 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.539+0000 7ffa9f837640 1 -- 172.21.15.26:0/339277263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98152520 msgr2=0x7ffa9806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.539+0000 7ffa9f837640 1 --2- 172.21.15.26:0/339277263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98152520 0x7ffa9806ef30 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ffa940097a0 tx=0x7ffa9402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.540+0000 7ffa9f837640 1 -- 172.21.15.26:0/339277263 shutdown_connections 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.540+0000 7ffa9f837640 1 --2- 172.21.15.26:0/339277263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa98155c80 0x7ffa980c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.540+0000 7ffa9f837640 1 --2- 172.21.15.26:0/339277263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98152520 0x7ffa9806ef30 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.540+0000 7ffa9f837640 1 -- 172.21.15.26:0/339277263 >> 172.21.15.26:0/339277263 conn(0x7ffa980c76b0 msgr2=0x7ffa980c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.540+0000 7ffa9f837640 1 -- 172.21.15.26:0/339277263 shutdown_connections 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.540+0000 7ffa9f837640 1 -- 172.21.15.26:0/339277263 wait complete. 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.541+0000 7ffa9f837640 1 Processor -- start 2024-03-06T07:27:09.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.541+0000 7ffa9f837640 1 -- start start 2024-03-06T07:27:09.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.541+0000 7ffa9f837640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa980720a0 0x7ffa981ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9f837640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa98152520 0x7ffa98158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9f837640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98155c80 0x7ffa98158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9f837640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9806b6d0 con 0x7ffa98155c80 2024-03-06T07:27:09.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9f837640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa980c17f0 con 0x7ffa98152520 2024-03-06T07:27:09.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9f837640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9806b850 con 0x7ffa980720a0 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98155c80 0x7ffa98158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98155c80 0x7ffa98158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38366/0 (socket says 172.21.15.26:38366) 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 -- 172.21.15.26:0/667664447 learned_addr learned my addr 172.21.15.26:0/667664447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9d5d3640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa980720a0 0x7ffa981ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9cdd2640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa98152520 0x7ffa98158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 -- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa980720a0 msgr2=0x7ffa981ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa980720a0 0x7ffa981ee470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 -- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa98152520 msgr2=0x7ffa98158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa98152520 0x7ffa98158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 -- 172.21.15.26:0/667664447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa981590f0 con 0x7ffa98155c80 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9ddd4640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98155c80 0x7ffa98158970 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c700 tx=0x7ffa9000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa8dffb640 1 -- 172.21.15.26:0/667664447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90018070 con 0x7ffa98155c80 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa8dffb640 1 -- 172.21.15.26:0/667664447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9000ce10 con 0x7ffa98155c80 2024-03-06T07:27:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa981593d0 con 0x7ffa98155c80 2024-03-06T07:27:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.542+0000 7ffa8dffb640 1 -- 172.21.15.26:0/667664447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000a800 con 0x7ffa98155c80 2024-03-06T07:27:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.543+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa981df2b0 con 0x7ffa98155c80 2024-03-06T07:27:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.544+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa981ef110 con 0x7ffa98155c80 2024-03-06T07:27:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.544+0000 7ffa8dffb640 1 -- 172.21.15.26:0/667664447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa9001b070 con 0x7ffa98155c80 2024-03-06T07:27:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.544+0000 7ffa8dffb640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa7c06b4d0 0x7ffa7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.545+0000 7ffa8dffb640 1 -- 172.21.15.26:0/667664447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffa90089840 con 0x7ffa98155c80 2024-03-06T07:27:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.545+0000 7ffa9d5d3640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa7c06b4d0 0x7ffa7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.545+0000 7ffa9d5d3640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa7c06b4d0 0x7ffa7c06d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffa94036000 tx=0x7ffa94008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.549+0000 7ffa8dffb640 1 -- 172.21.15.26:0/667664447 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffa981ef110 con 0x7ffa98155c80 2024-03-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.769+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa9804f330 con 0x7ffa98155c80 2024-03-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.770+0000 7ffa8dffb640 1 -- 172.21.15.26:0/667664447 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7ffa900585c0 con 0x7ffa98155c80 2024-03-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa7c06b4d0 msgr2=0x7ffa7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa7c06b4d0 0x7ffa7c06d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffa94036000 tx=0x7ffa94008030 comp rx=0 tx=0).stop 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98155c80 msgr2=0x7ffa98158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98155c80 0x7ffa98158970 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c700 tx=0x7ffa9000cbc0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 shutdown_connections 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa7c06b4d0 0x7ffa7c06d990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa98155c80 0x7ffa98158970 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa98152520 0x7ffa98158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 --2- 172.21.15.26:0/667664447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa980720a0 0x7ffa981ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 >> 172.21.15.26:0/667664447 conn(0x7ffa980c76b0 msgr2=0x7ffa980c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 shutdown_connections 2024-03-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:09.772+0000 7ffa9f837640 1 -- 172.21.15.26:0/667664447 wait complete. 2024-03-06T07:27:10.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:10.188 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:11.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.301+0000 7fed57842640 1 Processor -- start 2024-03-06T07:27:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed57842640 1 -- start start 2024-03-06T07:27:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed57842640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fed5014f360 0x7fed50155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed57842640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed500c88c0 0x7fed500c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed57842640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed500c73d0 con 0x7fed500c5880 2024-03-06T07:27:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed57842640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500c8dd0 con 0x7fed500c88c0 2024-03-06T07:27:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed57842640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500be2e0 con 0x7fed5014f360 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed54ddd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed500c88c0 0x7fed500c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed55ddf640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fed500c5880 0x7fed50158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38766/0 (socket says 172.21.15.26:38766) 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.302+0000 7fed55ddf640 1 -- 172.21.15.26:0/1909617005 learned_addr learned my addr 172.21.15.26:0/1909617005 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.303+0000 7fed555de640 1 --2- 172.21.15.26:0/1909617005 >> v2:172.21.15.117:3300/0 conn(0x7fed5014f360 0x7fed50155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.303+0000 7fed54ddd640 1 -- 172.21.15.26:0/1909617005 >> v2:172.21.15.117:3300/0 conn(0x7fed5014f360 msgr2=0x7fed50155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.303+0000 7fed54ddd640 1 --2- 172.21.15.26:0/1909617005 >> v2:172.21.15.117:3300/0 conn(0x7fed5014f360 0x7fed50155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.303+0000 7fed54ddd640 1 -- 172.21.15.26:0/1909617005 >> v1:172.21.15.26:6789/0 conn(0x7fed500c5880 legacy=0x7fed50158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.303+0000 7fed47fff640 1 -- 172.21.15.26:0/1909617005 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726484070 0 0) 0x7fed500c73d0 con 0x7fed500c5880 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.303+0000 7fed54ddd640 1 -- 172.21.15.26:0/1909617005 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500c7670 con 0x7fed500c88c0 2024-03-06T07:27:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.303+0000 7fed555de640 1 --2- 172.21.15.26:0/1909617005 >> v2:172.21.15.117:3300/0 conn(0x7fed5014f360 0x7fed50155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed54ddd640 1 --2- 172.21.15.26:0/1909617005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed500c88c0 0x7fed500c5340 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fed5008c840 tx=0x7fed4002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d1d7e1ae2178da server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed47fff640 1 -- 172.21.15.26:0/1909617005 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed400315a0 con 0x7fed500c88c0 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed47fff640 1 -- 172.21.15.26:0/1909617005 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed40037070 con 0x7fed500c88c0 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed47fff640 1 -- 172.21.15.26:0/1909617005 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40031960 con 0x7fed500c88c0 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed57842640 1 -- 172.21.15.26:0/1909617005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed500c88c0 msgr2=0x7fed500c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed57842640 1 --2- 172.21.15.26:0/1909617005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed500c88c0 0x7fed500c5340 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fed5008c840 tx=0x7fed4002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed57842640 1 -- 172.21.15.26:0/1909617005 shutdown_connections 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed57842640 1 --2- 172.21.15.26:0/1909617005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed500c88c0 0x7fed500c5340 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed57842640 1 --2- 172.21.15.26:0/1909617005 >> v2:172.21.15.117:3300/0 conn(0x7fed5014f360 0x7fed50155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.304+0000 7fed57842640 1 -- 172.21.15.26:0/1909617005 >> 172.21.15.26:0/1909617005 conn(0x7fed500ac570 msgr2=0x7fed500ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.305+0000 7fed57842640 1 -- 172.21.15.26:0/1909617005 shutdown_connections 2024-03-06T07:27:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.305+0000 7fed57842640 1 -- 172.21.15.26:0/1909617005 wait complete. 2024-03-06T07:27:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 Processor -- start 2024-03-06T07:27:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 -- start start 2024-03-06T07:27:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed500c5880 0x7fed500aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed500c88c0 0x7fed501519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5014f360 0x7fed501ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50159270 con 0x7fed5014f360 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed500c7c30 con 0x7fed500c88c0 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed57842640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed501593f0 con 0x7fed500c5880 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed55ddf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5014f360 0x7fed501ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed55ddf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5014f360 0x7fed501ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38378/0 (socket says 172.21.15.26:38378) 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.306+0000 7fed55ddf640 1 -- 172.21.15.26:0/2287764617 learned_addr learned my addr 172.21.15.26:0/2287764617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed55ddf640 1 -- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed500c5880 msgr2=0x7fed500aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed55ddf640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed500c5880 0x7fed500aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed55ddf640 1 -- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed500c88c0 msgr2=0x7fed501519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed54ddd640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed500c88c0 0x7fed501519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed55ddf640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed500c88c0 0x7fed501519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed55ddf640 1 -- 172.21.15.26:0/2287764617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50151f20 con 0x7fed5014f360 2024-03-06T07:27:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed55ddf640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5014f360 0x7fed501ee6d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fed4800c7a0 tx=0x7fed4800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:11.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed45ffb640 1 -- 172.21.15.26:0/2287764617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48018070 con 0x7fed5014f360 2024-03-06T07:27:11.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed45ffb640 1 -- 172.21.15.26:0/2287764617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4800a640 con 0x7fed5014f360 2024-03-06T07:27:11.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed45ffb640 1 -- 172.21.15.26:0/2287764617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4800aab0 con 0x7fed5014f360 2024-03-06T07:27:11.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed501eef10 con 0x7fed5014f360 2024-03-06T07:27:11.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.307+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed500ae2c0 con 0x7fed5014f360 2024-03-06T07:27:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.308+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5004f330 con 0x7fed5014f360 2024-03-06T07:27:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.309+0000 7fed45ffb640 1 -- 172.21.15.26:0/2287764617 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed4801b070 con 0x7fed5014f360 2024-03-06T07:27:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.309+0000 7fed45ffb640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed3406b500 0x7fed3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.309+0000 7fed45ffb640 1 -- 172.21.15.26:0/2287764617 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed480898b0 con 0x7fed5014f360 2024-03-06T07:27:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.309+0000 7fed555de640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed3406b500 0x7fed3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.310+0000 7fed555de640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed3406b500 0x7fed3406d9c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006fd0 tx=0x7fed4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:11.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.314+0000 7fed45ffb640 1 -- 172.21.15.26:0/2287764617 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed48058630 con 0x7fed5014f360 2024-03-06T07:27:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.533+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed50156030 con 0x7fed5014f360 2024-03-06T07:27:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.533+0000 7fed45ffb640 1 -- 172.21.15.26:0/2287764617 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4985 (secure 0 0 0) 0x7fed48057f30 con 0x7fed5014f360 2024-03-06T07:27:11.535 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:11.535 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:26:58.848063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"a","rank":0,"incarnation":158,"state":"up:stopping","state_seq":3,"addr":"172.21.15.26:6835/156189543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":156189543},{"type":"v1","addr":"172.21.15.26:6835","nonce":156189543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 162 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed3406b500 msgr2=0x7fed3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed3406b500 0x7fed3406d9c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006fd0 tx=0x7fed4c008040 comp rx=0 tx=0).stop 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5014f360 msgr2=0x7fed501ee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5014f360 0x7fed501ee6d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fed4800c7a0 tx=0x7fed4800cc60 comp rx=0 tx=0).stop 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 shutdown_connections 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed3406b500 0x7fed3406d9c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5014f360 0x7fed501ee6d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed500c88c0 0x7fed501519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 --2- 172.21.15.26:0/2287764617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed500c5880 0x7fed500aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 >> 172.21.15.26:0/2287764617 conn(0x7fed500ac570 msgr2=0x7fed501539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 shutdown_connections 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:11.535+0000 7fed57842640 1 -- 172.21.15.26:0/2287764617 wait complete. 2024-03-06T07:27:11.950 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:26:58.848063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.26:6835/156189543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 156189543}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 156189543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:11.950 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:stopping 2024-03-06T07:27:12.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:12.575 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:12.893 INFO:tasks.ceph.mds.a.smithi026.stderr: -608> 2024-03-06T07:26:58.110+0000 7fdadff2a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:12.921 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:27:12.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.058+0000 7fdad92b5640 1 Processor -- start 2024-03-06T07:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.058+0000 7fdad92b5640 1 -- start start 2024-03-06T07:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad92b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad41527b0 0x7fdad4159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad92b5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdad40c88b0 0x7fdad40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad92b5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40c7c60 con 0x7fdad40c7180 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad92b5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad4159640 con 0x7fdad40c88b0 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad92b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40c8060 con 0x7fdad41527b0 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad27fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdad40c7180 0x7fdad4155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38776/0 (socket says 172.21.15.26:38776) 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad27fc640 1 -- 172.21.15.26:0/890469151 learned_addr learned my addr 172.21.15.26:0/890469151 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad37fe640 1 --2- 172.21.15.26:0/890469151 >> v2:172.21.15.117:3300/0 conn(0x7fdad40c88b0 0x7fdad40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.059+0000 7fdad2ffd640 1 --2- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad41527b0 0x7fdad4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.060+0000 7fdad1ffb640 1 -- 172.21.15.26:0/890469151 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202910866 0 0) 0x7fdac8001110 con 0x7fdad40c7180 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.060+0000 7fdad1ffb640 1 -- 172.21.15.26:0/890469151 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdabc003520 con 0x7fdad40c7180 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.060+0000 7fdad37fe640 1 -- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad41527b0 msgr2=0x7fdad4159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.060+0000 7fdad37fe640 1 --2- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad41527b0 0x7fdad4159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.060+0000 7fdad37fe640 1 -- 172.21.15.26:0/890469151 >> v1:172.21.15.26:6789/0 conn(0x7fdad40c7180 legacy=0x7fdad4155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.060+0000 7fdad37fe640 1 -- 172.21.15.26:0/890469151 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40c76a0 con 0x7fdad40c88b0 2024-03-06T07:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad37fe640 1 --2- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad40c88b0 0x7fdad40c5330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009d90 tx=0x7fdac402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f0ba4134e5bee0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad1ffb640 1 -- 172.21.15.26:0/890469151 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4039070 con 0x7fdad40c88b0 2024-03-06T07:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad1ffb640 1 -- 172.21.15.26:0/890469151 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac402cbc0 con 0x7fdad40c88b0 2024-03-06T07:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad1ffb640 1 -- 172.21.15.26:0/890469151 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4034440 con 0x7fdad40c88b0 2024-03-06T07:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 -- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad40c88b0 msgr2=0x7fdad40c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 --2- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad40c88b0 0x7fdad40c5330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009d90 tx=0x7fdac402be10 comp rx=0 tx=0).stop 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 -- 172.21.15.26:0/890469151 shutdown_connections 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 --2- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad40c88b0 0x7fdad40c5330 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 --2- 172.21.15.26:0/890469151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad41527b0 0x7fdad4159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 -- 172.21.15.26:0/890469151 >> 172.21.15.26:0/890469151 conn(0x7fdad40ac560 msgr2=0x7fdad40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 -- 172.21.15.26:0/890469151 shutdown_connections 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.061+0000 7fdad92b5640 1 -- 172.21.15.26:0/890469151 wait complete. 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 Processor -- start 2024-03-06T07:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 -- start start 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad40c7180 0x7fdad41eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad40c88b0 0x7fdad40c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad41527b0 0x7fdad40bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4155600 con 0x7fdad40c7180 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4155320 con 0x7fdad41527b0 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.063+0000 7fdad92b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40c8d60 con 0x7fdad40c88b0 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad40c7180 0x7fdad41eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad40c7180 0x7fdad41eaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38404/0 (socket says 172.21.15.26:38404) 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 -- 172.21.15.26:0/2603739668 learned_addr learned my addr 172.21.15.26:0/2603739668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad27fc640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad40c88b0 0x7fdad40c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad37fe640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad41527b0 0x7fdad40bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 -- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad40c88b0 msgr2=0x7fdad40c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad40c88b0 0x7fdad40c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 -- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad41527b0 msgr2=0x7fdad40bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad41527b0 0x7fdad40bfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 -- 172.21.15.26:0/2603739668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40aba10 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad2ffd640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad40c7180 0x7fdad41eaaf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00e8a0 tx=0x7fdacc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdabb7fe640 1 -- 172.21.15.26:0/2603739668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc017070 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdabb7fe640 1 -- 172.21.15.26:0/2603739668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc00c940 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad41579d0 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdabb7fe640 1 -- 172.21.15.26:0/2603739668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00cd30 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.064+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad4157f90 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.065+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad404f330 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.066+0000 7fdabb7fe640 1 -- 172.21.15.26:0/2603739668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdacc01a070 con 0x7fdad40c7180 2024-03-06T07:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.066+0000 7fdabb7fe640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdac006b500 0x7fdac006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.066+0000 7fdabb7fe640 1 -- 172.21.15.26:0/2603739668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdacc08a0c0 con 0x7fdad40c7180 2024-03-06T07:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.066+0000 7fdad27fc640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdac006b500 0x7fdac006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.067+0000 7fdad27fc640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdac006b500 0x7fdac006d9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdad4159160 tx=0x7fdac8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.071+0000 7fdabb7fe640 1 -- 172.21.15.26:0/2603739668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdacc0587f0 con 0x7fdad40c7180 2024-03-06T07:27:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.289+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdad406ce00 con 0x7fdad40c7180 2024-03-06T07:27:13.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.289+0000 7fdabb7fe640 1 -- 172.21.15.26:0/2603739668 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) v1 ==== 95+0+4184 (secure 0 0 0) 0x7fdacc05c890 con 0x7fdad40c7180 2024-03-06T07:27:13.291 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:13.291 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":163,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:12.740127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:13.291 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 163 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdac006b500 msgr2=0x7fdac006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdac006b500 0x7fdac006d9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdad4159160 tx=0x7fdac8009040 comp rx=0 tx=0).stop 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad40c7180 msgr2=0x7fdad41eaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad40c7180 0x7fdad41eaaf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00e8a0 tx=0x7fdacc00ed60 comp rx=0 tx=0).stop 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 shutdown_connections 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdac006b500 0x7fdac006d9c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad41527b0 0x7fdad40bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad40c88b0 0x7fdad40c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 --2- 172.21.15.26:0/2603739668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad40c7180 0x7fdad41eaaf0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 >> 172.21.15.26:0/2603739668 conn(0x7fdad40ac560 msgr2=0x7fdad4154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 shutdown_connections 2024-03-06T07:27:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.291+0000 7fdad92b5640 1 -- 172.21.15.26:0/2603739668 wait complete. 2024-03-06T07:27:13.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 163, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:27:12.740127+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [], 'up': {}, 'failed': [], 'damaged': [], 'stopped': [0], 'info': {}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:13.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/0 2024-03-06T07:27:13.706 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.812+0000 7f5b76a4e640 1 Processor -- start 2024-03-06T07:27:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.812+0000 7f5b76a4e640 1 -- start start 2024-03-06T07:27:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.812+0000 7f5b76a4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b70072720 0x7f5b70156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.812+0000 7f5b76a4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b701508c0 0x7f5b700c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.812+0000 7f5b76a4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b70157340 con 0x7f5b7006f7b0 2024-03-06T07:27:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.812+0000 7f5b76a4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b700c3740 con 0x7f5b701508c0 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.812+0000 7f5b76a4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70156d80 con 0x7f5b70072720 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b74feb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b701508c0 0x7f5b700c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b6f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5b7006f7b0 0x7f5b701501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38790/0 (socket says 172.21.15.26:38790) 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b6f7fe640 1 -- 172.21.15.26:0/1651908307 learned_addr learned my addr 172.21.15.26:0/1651908307 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b74feb640 1 -- 172.21.15.26:0/1651908307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b70072720 msgr2=0x7f5b70156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b6ffff640 1 --2- 172.21.15.26:0/1651908307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b70072720 0x7f5b70156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b74feb640 1 --2- 172.21.15.26:0/1651908307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b70072720 0x7f5b70156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b6effd640 1 -- 172.21.15.26:0/1651908307 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205901076 0 0) 0x7f5b70157340 con 0x7f5b7006f7b0 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b74feb640 1 -- 172.21.15.26:0/1651908307 >> v1:172.21.15.26:6789/0 conn(0x7f5b7006f7b0 legacy=0x7f5b701501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b74feb640 1 -- 172.21.15.26:0/1651908307 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b70157060 con 0x7f5b701508c0 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.813+0000 7f5b6ffff640 1 --2- 172.21.15.26:0/1651908307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b70072720 0x7f5b70156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b74feb640 1 --2- 172.21.15.26:0/1651908307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c3bd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b6802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cb5df2d1de966e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b6effd640 1 -- 172.21.15.26:0/1651908307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68039070 con 0x7f5b701508c0 2024-03-06T07:27:13.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b6effd640 1 -- 172.21.15.26:0/1651908307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6802cbc0 con 0x7f5b701508c0 2024-03-06T07:27:13.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b6effd640 1 -- 172.21.15.26:0/1651908307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68034440 con 0x7f5b701508c0 2024-03-06T07:27:13.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 -- 172.21.15.26:0/1651908307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 msgr2=0x7f5b700c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/1651908307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c3bd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b6802be10 comp rx=0 tx=0).stop 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 -- 172.21.15.26:0/1651908307 shutdown_connections 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/1651908307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c3bd0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/1651908307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b70072720 0x7f5b70156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 -- 172.21.15.26:0/1651908307 >> 172.21.15.26:0/1651908307 conn(0x7f5b700aa630 msgr2=0x7f5b700aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 -- 172.21.15.26:0/1651908307 shutdown_connections 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.814+0000 7f5b76a4e640 1 -- 172.21.15.26:0/1651908307 wait complete. 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.815+0000 7f5b76a4e640 1 Processor -- start 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b76a4e640 1 -- start start 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b76a4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b7006f7b0 0x7f5b701ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b76a4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70072720 0x7f5b700c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b76a4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b76a4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70156af0 con 0x7f5b701508c0 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b76a4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70156970 con 0x7f5b70072720 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b76a4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70072b00 con 0x7f5b7006f7b0 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b74feb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b74feb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38428/0 (socket says 172.21.15.26:38428) 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b74feb640 1 -- 172.21.15.26:0/2750005068 learned_addr learned my addr 172.21.15.26:0/2750005068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.816+0000 7f5b6ffff640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b7006f7b0 0x7f5b701ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b6f7fe640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70072720 0x7f5b700c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b74feb640 1 -- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b7006f7b0 msgr2=0x7f5b701ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b74feb640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b7006f7b0 0x7f5b701ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b74feb640 1 -- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70072720 msgr2=0x7f5b700c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b74feb640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70072720 0x7f5b700c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b74feb640 1 -- 172.21.15.26:0/2750005068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b70156240 con 0x7f5b701508c0 2024-03-06T07:27:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b6f7fe640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70072720 0x7f5b700c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b74feb640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c7f30 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009eb0 tx=0x7f5b68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b6cff9640 1 -- 172.21.15.26:0/2750005068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68039050 con 0x7f5b701508c0 2024-03-06T07:27:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b6cff9640 1 -- 172.21.15.26:0/2750005068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68045070 con 0x7f5b701508c0 2024-03-06T07:27:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b701ee870 con 0x7f5b701508c0 2024-03-06T07:27:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b6cff9640 1 -- 172.21.15.26:0/2750005068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6803e850 con 0x7f5b701508c0 2024-03-06T07:27:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.817+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b700c5cf0 con 0x7f5b701508c0 2024-03-06T07:27:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.818+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7004f330 con 0x7f5b701508c0 2024-03-06T07:27:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.819+0000 7f5b6cff9640 1 -- 172.21.15.26:0/2750005068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b6803e9f0 con 0x7f5b701508c0 2024-03-06T07:27:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.819+0000 7f5b6cff9640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b5006b4d0 0x7f5b5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.819+0000 7f5b6cff9640 1 -- 172.21.15.26:0/2750005068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b680ad130 con 0x7f5b701508c0 2024-03-06T07:27:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.819+0000 7f5b6ffff640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b5006b4d0 0x7f5b5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.820+0000 7f5b6ffff640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b5006b4d0 0x7f5b5006d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5b700aeae0 tx=0x7f5b58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:13.823+0000 7f5b6cff9640 1 -- 172.21.15.26:0/2750005068 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5b6807c090 con 0x7f5b701508c0 2024-03-06T07:27:14.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.043+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b7014abf0 con 0x7f5b701508c0 2024-03-06T07:27:14.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.043+0000 7f5b6cff9640 1 -- 172.21.15.26:0/2750005068 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+4976 (secure 0 0 0) 0x7f5b6807b990 con 0x7f5b701508c0 2024-03-06T07:27:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:14.045 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":164,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":9627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":164}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:12.740127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:14.045 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 164 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b5006b4d0 msgr2=0x7f5b5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b5006b4d0 0x7f5b5006d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5b700aeae0 tx=0x7f5b58008040 comp rx=0 tx=0).stop 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 msgr2=0x7f5b700c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c7f30 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009eb0 tx=0x7f5b68004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 shutdown_connections 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b5006b4d0 0x7f5b5006d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b701508c0 0x7f5b700c7f30 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70072720 0x7f5b700c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 --2- 172.21.15.26:0/2750005068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b7006f7b0 0x7f5b701ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 >> 172.21.15.26:0/2750005068 conn(0x7f5b700aa630 msgr2=0x7f5b7014c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 shutdown_connections 2024-03-06T07:27:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.045+0000 7f5b76a4e640 1 -- 172.21.15.26:0/2750005068 wait complete. 2024-03-06T07:27:14.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down false 2024-03-06T07:27:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.160+0000 7f947afd7640 1 Processor -- start 2024-03-06T07:27:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.160+0000 7f947afd7640 1 -- start start 2024-03-06T07:27:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f947afd7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94741528d0 0x7f94740724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f947afd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94740c67f0 0x7f9474155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f947afd7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9474152cb0 con 0x7f9474072a10 2024-03-06T07:27:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f947afd7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94740c38b0 con 0x7f94741528d0 2024-03-06T07:27:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f947afd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94741594f0 con 0x7f94740c67f0 2024-03-06T07:27:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9478d73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94741528d0 0x7f94740724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9478d73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94741528d0 0x7f94740724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38440/0 (socket says 172.21.15.26:38440) 2024-03-06T07:27:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9478d73640 1 -- 172.21.15.26:0/1872225869 learned_addr learned my addr 172.21.15.26:0/1872225869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9479574640 1 --2- 172.21.15.26:0/1872225869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94740c67f0 0x7f9474155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9478d73640 1 -- 172.21.15.26:0/1872225869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94740c67f0 msgr2=0x7f9474155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9478d73640 1 --2- 172.21.15.26:0/1872225869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94740c67f0 0x7f9474155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9478d73640 1 -- 172.21.15.26:0/1872225869 >> v1:172.21.15.117:6789/0 conn(0x7f9474072a10 legacy=0x7f9474155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.161+0000 7f9478d73640 1 -- 172.21.15.26:0/1872225869 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740a7f70 con 0x7f94741528d0 2024-03-06T07:27:14.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.162+0000 7f9478d73640 1 --2- 172.21.15.26:0/1872225869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94741528d0 0x7f94740724d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f946c009e50 tx=0x7f946c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdcfc1d42e674dc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.162+0000 7f94737fe640 1 -- 172.21.15.26:0/1872225869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c02cb90 con 0x7f94741528d0 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.162+0000 7f94737fe640 1 -- 172.21.15.26:0/1872225869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c02cd30 con 0x7f94741528d0 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.162+0000 7f94737fe640 1 -- 172.21.15.26:0/1872225869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c034610 con 0x7f94741528d0 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 -- 172.21.15.26:0/1872225869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94741528d0 msgr2=0x7f94740724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 --2- 172.21.15.26:0/1872225869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94741528d0 0x7f94740724d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f946c009e50 tx=0x7f946c02bff0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 -- 172.21.15.26:0/1872225869 shutdown_connections 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 --2- 172.21.15.26:0/1872225869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94740c67f0 0x7f9474155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 --2- 172.21.15.26:0/1872225869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94741528d0 0x7f94740724d0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 -- 172.21.15.26:0/1872225869 >> 172.21.15.26:0/1872225869 conn(0x7f94740aa6a0 msgr2=0x7f94740aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 -- 172.21.15.26:0/1872225869 shutdown_connections 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.163+0000 7f947afd7640 1 -- 172.21.15.26:0/1872225869 wait complete. 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.164+0000 7f947afd7640 1 Processor -- start 2024-03-06T07:27:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.164+0000 7f947afd7640 1 -- start start 2024-03-06T07:27:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f947afd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9474072a10 0x7f9474154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f947afd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94740c67f0 0x7f9474148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f947afd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94741528d0 0x7f9474149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f947afd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94741591a0 con 0x7f94740c67f0 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f947afd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740a8250 con 0x7f94741528d0 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f947afd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94740c4ef0 con 0x7f9474072a10 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94740c67f0 0x7f9474148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94740c67f0 0x7f9474148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38442/0 (socket says 172.21.15.26:38442) 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 -- 172.21.15.26:0/2983169536 learned_addr learned my addr 172.21.15.26:0/2983169536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9478d73640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9474072a10 0x7f9474154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9479574640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94741528d0 0x7f9474149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 -- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9474072a10 msgr2=0x7f9474154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9474072a10 0x7f9474154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 -- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94741528d0 msgr2=0x7f9474149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94741528d0 0x7f9474149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 -- 172.21.15.26:0/2983169536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94741ea8c0 con 0x7f94740c67f0 2024-03-06T07:27:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9479574640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94741528d0 0x7f9474149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.165+0000 7f9473fff640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94740c67f0 0x7f9474148ef0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f946800c400 tx=0x7f946800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.166+0000 7f94717fa640 1 -- 172.21.15.26:0/2983169536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680052d0 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.166+0000 7f94717fa640 1 -- 172.21.15.26:0/2983169536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468005490 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.166+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94741eaba0 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.166+0000 7f94717fa640 1 -- 172.21.15.26:0/2983169536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468007260 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.166+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94741551b0 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.167+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947404f330 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.167+0000 7f94717fa640 1 -- 172.21.15.26:0/2983169536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9468007400 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.167+0000 7f94717fa640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.167+0000 7f94717fa640 1 -- 172.21.15.26:0/2983169536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9468089780 con 0x7f94740c67f0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.168+0000 7f9478d73640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.168+0000 7f9478d73640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f946006b4d0 0x7f946006d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f946c035000 tx=0x7f946c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.172+0000 7f94717fa640 1 -- 172.21.15.26:0/2983169536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9468057eb0 con 0x7f94740c67f0 2024-03-06T07:27:14.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.390+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "false"} v 0) v1 -- 0x7f94740c5e60 con 0x7f94740c67f0 2024-03-06T07:27:14.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.648+0000 7f94717fa640 1 -- 172.21.15.26:0/2983169536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "false"}]=0 cephfs marked up, max_mds = 1 v165) v1 ==== 135+0+0 (secure 0 0 0) 0x7f946805bf50 con 0x7f94740c67f0 2024-03-06T07:27:14.657 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked up, max_mds = 1 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f946006b4d0 msgr2=0x7f946006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f946006b4d0 0x7f946006d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f946c035000 tx=0x7f946c030000 comp rx=0 tx=0).stop 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94740c67f0 msgr2=0x7f9474148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94740c67f0 0x7f9474148ef0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f946800c400 tx=0x7f946800c8c0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 shutdown_connections 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94741528d0 0x7f9474149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94740c67f0 0x7f9474148ef0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 --2- 172.21.15.26:0/2983169536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9474072a10 0x7f9474154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 >> 172.21.15.26:0/2983169536 conn(0x7f94740aa6a0 msgr2=0x7f94740c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 shutdown_connections 2024-03-06T07:27:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.650+0000 7f947afd7640 1 -- 172.21.15.26:0/2983169536 wait complete. 2024-03-06T07:27:14.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.764+0000 7fae6b89a640 1 Processor -- start 2024-03-06T07:27:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.765+0000 7fae6b89a640 1 -- start start 2024-03-06T07:27:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.765+0000 7fae6b89a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae640c6780 0x7fae64158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.765+0000 7fae6b89a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 0x7fae640c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.765+0000 7fae6b89a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae64072630 con 0x7fae6406da40 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.765+0000 7fae6b89a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6406de20 con 0x7fae640c6780 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.765+0000 7fae6b89a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae64072910 con 0x7fae641592d0 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae69636640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae6406da40 0x7fae64155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38798/0 (socket says 172.21.15.26:38798) 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae69636640 1 -- 172.21.15.26:0/4138573279 learned_addr learned my addr 172.21.15.26:0/4138573279 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae69e37640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 0x7fae640c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae68e35640 1 --2- 172.21.15.26:0/4138573279 >> v2:172.21.15.117:3300/0 conn(0x7fae640c6780 0x7fae64158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082457931 0 0) 0x7fae64072630 con 0x7fae6406da40 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae4c003520 con 0x7fae6406da40 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 170586046 0 0) 0x7fae4c003520 con 0x7fae6406da40 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 msgr2=0x7fae640c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 0x7fae640c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 >> v2:172.21.15.117:3300/0 conn(0x7fae640c6780 msgr2=0x7fae64158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 --2- 172.21.15.26:0/4138573279 >> v2:172.21.15.117:3300/0 conn(0x7fae640c6780 0x7fae64158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae64072bf0 con 0x7fae6406da40 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fae5c001670 con 0x7fae6406da40 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 >> v1:172.21.15.26:6789/0 conn(0x7fae6406da40 legacy=0x7fae64155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0039b0 0x7fae4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 0x7fae4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c004d40 0x7fae4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406de20 con 0x7fae4c004d40 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64072910 con 0x7fae4c0039b0 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c00b9c0 con 0x7fae641592d0 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae69e37640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c004d40 0x7fae4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae69636640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0039b0 0x7fae4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.766+0000 7fae68e35640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 0x7fae4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae69e37640 1 -- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 msgr2=0x7fae4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae69e37640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae641592d0 0x7fae4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae69e37640 1 -- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0039b0 msgr2=0x7fae4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae69e37640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4c0039b0 0x7fae4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae69e37640 1 -- 172.21.15.26:0/4138573279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c003520 con 0x7fae4c004d40 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae69e37640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c004d40 0x7fae4c0051c0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fae54009fc0 tx=0x7fae4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ca68bb821b9438 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54039070 con 0x7fae4c004d40 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5402cc20 con 0x7fae4c004d40 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.767+0000 7fae63fff640 1 -- 172.21.15.26:0/4138573279 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54009c10 con 0x7fae4c004d40 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.768+0000 7fae6b89a640 1 -- 172.21.15.26:0/4138573279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c004d40 msgr2=0x7fae4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.768+0000 7fae6b89a640 1 --2- 172.21.15.26:0/4138573279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4c004d40 0x7fae4c0051c0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fae54009fc0 tx=0x7fae4c005830 comp rx=0 tx=0).stop 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.768+0000 7fae69e37640 1 -- 172.21.15.26:0/4138573279 reap_dead start 2024-03-06T07:27:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.768+0000 7fae6b89a640 1 -- 172.21.15.26:0/4138573279 shutdown_connections 2024-03-06T07:27:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.768+0000 7fae6b89a640 1 -- 172.21.15.26:0/4138573279 >> 172.21.15.26:0/4138573279 conn(0x7fae640aa630 msgr2=0x7fae640aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.768+0000 7fae6b89a640 1 -- 172.21.15.26:0/4138573279 shutdown_connections 2024-03-06T07:27:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.768+0000 7fae6b89a640 1 -- 172.21.15.26:0/4138573279 wait complete. 2024-03-06T07:27:14.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.769+0000 7fae6b89a640 1 Processor -- start 2024-03-06T07:27:14.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.769+0000 7fae6b89a640 1 -- start start 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae6b89a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae64157720 0x7fae640adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae6b89a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640ac7d0 0x7fae640ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae6b89a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae640ae740 0x7fae641ee240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae6b89a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae640c7500 con 0x7fae640ae740 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae6b89a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae640c7380 con 0x7fae64157720 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae6b89a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae640c7680 con 0x7fae640ac7d0 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae640ae740 0x7fae641ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae640ae740 0x7fae641ee240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38468/0 (socket says 172.21.15.26:38468) 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 -- 172.21.15.26:0/1754204213 learned_addr learned my addr 172.21.15.26:0/1754204213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 -- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640ac7d0 msgr2=0x7fae640ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640ac7d0 0x7fae640ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 -- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae64157720 msgr2=0x7fae640adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae64157720 0x7fae640adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 -- 172.21.15.26:0/1754204213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae640ac400 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae69e37640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae640ae740 0x7fae641ee240 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e230 tx=0x7fae54009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.770+0000 7fae61ffb640 1 -- 172.21.15.26:0/1754204213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54039070 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.771+0000 7fae61ffb640 1 -- 172.21.15.26:0/1754204213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae54044070 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.771+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae641de940 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.771+0000 7fae61ffb640 1 -- 172.21.15.26:0/1754204213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54009c10 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.771+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae640ad7c0 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.772+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6404f330 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.772+0000 7fae61ffb640 1 -- 172.21.15.26:0/1754204213 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae54005d10 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.773+0000 7fae61ffb640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.773+0000 7fae61ffb640 1 -- 172.21.15.26:0/1754204213 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae540ac350 con 0x7fae640ae740 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.773+0000 7fae69636640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.773+0000 7fae69636640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fae5c000fb0 tx=0x7fae5c0047e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.777+0000 7fae61ffb640 1 -- 172.21.15.26:0/1754204213 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae5404d020 con 0x7fae640ae740 2024-03-06T07:27:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.998+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae64070640 con 0x7fae640ae740 2024-03-06T07:27:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:14.999+0000 7fae61ffb640 1 -- 172.21.15.26:0/1754204213 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+4988 (secure 0 0 0) 0x7fae5407aa80 con 0x7fae640ae740 2024-03-06T07:27:15.011 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:15.011 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":165,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:14.647403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:starting","state_seq":1,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 165 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 msgr2=0x7fae5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fae5c000fb0 tx=0x7fae5c0047e0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae640ae740 msgr2=0x7fae641ee240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae640ae740 0x7fae641ee240 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e230 tx=0x7fae54009750 comp rx=0 tx=0).stop 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 shutdown_connections 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae5006b500 0x7fae5006d9c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae640ae740 0x7fae641ee240 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae640ac7d0 0x7fae640ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 --2- 172.21.15.26:0/1754204213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae64157720 0x7fae640adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 >> 172.21.15.26:0/1754204213 conn(0x7fae640aa630 msgr2=0x7fae640aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 shutdown_connections 2024-03-06T07:27:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.001+0000 7fae6b89a640 1 -- 172.21.15.26:0/1754204213 wait complete. 2024-03-06T07:27:15.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:15.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.114+0000 7f0b01193640 1 Processor -- start 2024-03-06T07:27:15.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.114+0000 7f0b01193640 1 -- start start 2024-03-06T07:27:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0b01193640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0afc0c88c0 0x7f0afc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0b01193640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0afc158b60 0x7f0afc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0b01193640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0afc0c73d0 con 0x7f0afc14f360 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0b01193640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0afc0c8dd0 con 0x7f0afc0c88c0 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0b01193640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0afc0be2e0 con 0x7f0afc158b60 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0afa59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0afc0c88c0 0x7f0afc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0afa59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0afc0c88c0 0x7f0afc155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38476/0 (socket says 172.21.15.26:38476) 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0afa59c640 1 -- 172.21.15.26:0/3455712591 learned_addr learned my addr 172.21.15.26:0/3455712591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.115+0000 7f0afb59e640 1 --2- 172.21.15.26:0/3455712591 >> v2:172.21.15.117:3300/0 conn(0x7f0afc158b60 0x7f0afc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0afa59c640 1 -- 172.21.15.26:0/3455712591 >> v2:172.21.15.117:3300/0 conn(0x7f0afc158b60 msgr2=0x7f0afc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0afa59c640 1 --2- 172.21.15.26:0/3455712591 >> v2:172.21.15.117:3300/0 conn(0x7f0afc158b60 0x7f0afc158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0afa59c640 1 -- 172.21.15.26:0/3455712591 >> v1:172.21.15.117:6789/0 conn(0x7f0afc14f360 legacy=0x7f0afc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0afa59c640 1 -- 172.21.15.26:0/3455712591 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc0c7670 con 0x7f0afc0c88c0 2024-03-06T07:27:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0afa59c640 1 --2- 172.21.15.26:0/3455712591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc155c10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0afc152d60 tx=0x7f0af002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d099da7507466b48 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0af9d9b640 1 -- 172.21.15.26:0/3455712591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0035020 con 0x7f0afc0c88c0 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0af9d9b640 1 -- 172.21.15.26:0/3455712591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af0031870 con 0x7f0afc0c88c0 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.116+0000 7f0af9d9b640 1 -- 172.21.15.26:0/3455712591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0031c40 con 0x7f0afc0c88c0 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 -- 172.21.15.26:0/3455712591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 msgr2=0x7f0afc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 --2- 172.21.15.26:0/3455712591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc155c10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0afc152d60 tx=0x7f0af002b910 comp rx=0 tx=0).stop 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 -- 172.21.15.26:0/3455712591 shutdown_connections 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 --2- 172.21.15.26:0/3455712591 >> v2:172.21.15.117:3300/0 conn(0x7f0afc158b60 0x7f0afc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 --2- 172.21.15.26:0/3455712591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc155c10 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 -- 172.21.15.26:0/3455712591 >> 172.21.15.26:0/3455712591 conn(0x7f0afc0ac570 msgr2=0x7f0afc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 -- 172.21.15.26:0/3455712591 shutdown_connections 2024-03-06T07:27:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.117+0000 7f0b01193640 1 -- 172.21.15.26:0/3455712591 wait complete. 2024-03-06T07:27:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.118+0000 7f0b01193640 1 Processor -- start 2024-03-06T07:27:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.118+0000 7f0b01193640 1 -- start start 2024-03-06T07:27:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0b01193640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc1eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0b01193640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc14f360 0x7f0afc1eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0b01193640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc158b60 0x7f0afc1ec320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0b01193640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc0c5a50 con 0x7f0afc0c88c0 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0b01193640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc0c7c30 con 0x7f0afc14f360 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0b01193640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc0c5bd0 con 0x7f0afc158b60 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc1eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc1eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38490/0 (socket says 172.21.15.26:38490) 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 -- 172.21.15.26:0/3010575044 learned_addr learned my addr 172.21.15.26:0/3010575044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afa59c640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc14f360 0x7f0afc1eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afb59e640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc158b60 0x7f0afc1ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 -- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc158b60 msgr2=0x7f0afc1ec320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc158b60 0x7f0afc1ec320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 -- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc14f360 msgr2=0x7f0afc1eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc14f360 0x7f0afc1eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afad9d640 1 -- 172.21.15.26:0/3010575044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc1eaeb0 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.119+0000 7f0afa59c640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc14f360 0x7f0afc1eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.120+0000 7f0afad9d640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc1eb930 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00c430 tx=0x7f0aec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.120+0000 7f0ae37fe640 1 -- 172.21.15.26:0/3010575044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0052d0 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.120+0000 7f0ae37fe640 1 -- 172.21.15.26:0/3010575044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec005490 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.120+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc1ef320 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.120+0000 7f0ae37fe640 1 -- 172.21.15.26:0/3010575044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec007260 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.120+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc0c0f60 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.121+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc04f330 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.121+0000 7f0ae37fe640 1 -- 172.21.15.26:0/3010575044 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0aec007400 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.122+0000 7f0ae37fe640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b4d0 0x7f0ae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.122+0000 7f0ae37fe640 1 -- 172.21.15.26:0/3010575044 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0aec089830 con 0x7f0afc0c88c0 2024-03-06T07:27:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.122+0000 7f0afa59c640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b4d0 0x7f0ae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.122+0000 7f0afa59c640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b4d0 0x7f0ae806d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0af0036000 tx=0x7f0af0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:15.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.127+0000 7f0ae37fe640 1 -- 172.21.15.26:0/3010575044 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0afc1eaeb0 con 0x7f0afc0c88c0 2024-03-06T07:27:15.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.346+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0afc156030 con 0x7f0afc0c88c0 2024-03-06T07:27:15.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.346+0000 7f0ae37fe640 1 -- 172.21.15.26:0/3010575044 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f0aec0585b0 con 0x7f0afc0c88c0 2024-03-06T07:27:15.348 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:15.348 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":165,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:14.647403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:starting","state_seq":1,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:15.349 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 165 2024-03-06T07:27:15.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b4d0 msgr2=0x7f0ae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b4d0 0x7f0ae806d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0af0036000 tx=0x7f0af0008030 comp rx=0 tx=0).stop 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 msgr2=0x7f0afc1eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc1eb930 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00c430 tx=0x7f0aec00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 shutdown_connections 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b4d0 0x7f0ae806d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc158b60 0x7f0afc1ec320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc14f360 0x7f0afc1eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 --2- 172.21.15.26:0/3010575044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc0c88c0 0x7f0afc1eb930 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.348+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 >> 172.21.15.26:0/3010575044 conn(0x7f0afc0ac570 msgr2=0x7f0afc0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.349+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 shutdown_connections 2024-03-06T07:27:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:15.349+0000 7f0b01193640 1 -- 172.21.15.26:0/3010575044 wait complete. 2024-03-06T07:27:15.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 165, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:27:14.647403+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9627}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9627': {'gid': 9627, 'name': 'a', 'rank': 0, 'incarnation': 165, 'state': 'up:starting', 'state_seq': 1, 'addr': '172.21.15.26:6835/1519487439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1519487439}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1519487439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:15.763 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9627:up:starting 2024-03-06T07:27:16.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:16.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.866+0000 7fb1abc2a640 1 Processor -- start 2024-03-06T07:27:16.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.866+0000 7fb1abc2a640 1 -- start start 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1abc2a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1a40c88c0 0x7fb1a4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1abc2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a4158b60 0x7fb1a4158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1abc2a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40c73d0 con 0x7fb1a414f360 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1abc2a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a40c8dd0 con 0x7fb1a40c88c0 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1abc2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40be2e0 con 0x7fb1a4158b60 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1a91c5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1a40c88c0 0x7fb1a4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1a91c5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1a40c88c0 0x7fb1a4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38506/0 (socket says 172.21.15.26:38506) 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1a91c5640 1 -- 172.21.15.26:0/2023848173 learned_addr learned my addr 172.21.15.26:0/2023848173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:16.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1aa1c7640 1 --2- 172.21.15.26:0/2023848173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a4158b60 0x7fb1a4158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1a91c5640 1 -- 172.21.15.26:0/2023848173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a4158b60 msgr2=0x7fb1a4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:16.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1a91c5640 1 --2- 172.21.15.26:0/2023848173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a4158b60 0x7fb1a4158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1a91c5640 1 -- 172.21.15.26:0/2023848173 >> v1:172.21.15.117:6789/0 conn(0x7fb1a414f360 legacy=0x7fb1a40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.867+0000 7fb1a91c5640 1 -- 172.21.15.26:0/2023848173 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40c7670 con 0x7fb1a40c88c0 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.868+0000 7fb1a91c5640 1 --2- 172.21.15.26:0/2023848173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40c88c0 0x7fb1a4155c10 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb194009870 tx=0x7fb19402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8863bd8976d47db1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.868+0000 7fb1a89c4640 1 -- 172.21.15.26:0/2023848173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194035020 con 0x7fb1a40c88c0 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.868+0000 7fb1a89c4640 1 -- 172.21.15.26:0/2023848173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194031640 con 0x7fb1a40c88c0 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.868+0000 7fb1a89c4640 1 -- 172.21.15.26:0/2023848173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19402d040 con 0x7fb1a40c88c0 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 -- 172.21.15.26:0/2023848173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40c88c0 msgr2=0x7fb1a4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/2023848173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40c88c0 0x7fb1a4155c10 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb194009870 tx=0x7fb19402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 -- 172.21.15.26:0/2023848173 shutdown_connections 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/2023848173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a4158b60 0x7fb1a4158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/2023848173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40c88c0 0x7fb1a4155c10 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 -- 172.21.15.26:0/2023848173 >> 172.21.15.26:0/2023848173 conn(0x7fb1a40ac570 msgr2=0x7fb1a40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:16.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 -- 172.21.15.26:0/2023848173 shutdown_connections 2024-03-06T07:27:16.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.869+0000 7fb1abc2a640 1 -- 172.21.15.26:0/2023848173 wait complete. 2024-03-06T07:27:16.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.871+0000 7fb1abc2a640 1 Processor -- start 2024-03-06T07:27:16.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.871+0000 7fb1abc2a640 1 -- start start 2024-03-06T07:27:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1abc2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a40c88c0 0x7fb1a4158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1abc2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a414f360 0x7fb1a414de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1abc2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a4158b60 0x7fb1a41ead10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1abc2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c5a50 con 0x7fb1a414f360 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1abc2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c7c30 con 0x7fb1a4158b60 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1abc2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c5bd0 con 0x7fb1a40c88c0 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a414f360 0x7fb1a414de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a414f360 0x7fb1a414de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38512/0 (socket says 172.21.15.26:38512) 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 -- 172.21.15.26:0/547650934 learned_addr learned my addr 172.21.15.26:0/547650934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1aa1c7640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a4158b60 0x7fb1a41ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a99c6640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a40c88c0 0x7fb1a4158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 -- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a40c88c0 msgr2=0x7fb1a4158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a40c88c0 0x7fb1a4158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 -- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a4158b60 msgr2=0x7fb1a41ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a4158b60 0x7fb1a41ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 -- 172.21.15.26:0/547650934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4152590 con 0x7fb1a414f360 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1aa1c7640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a4158b60 0x7fb1a41ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a99c6640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a40c88c0 0x7fb1a4158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.872+0000 7fb1a91c5640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a414f360 0x7fb1a414de40 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fb19402b930 tx=0x7fb1940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.873+0000 7fb19a7fc640 1 -- 172.21.15.26:0/547650934 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194040070 con 0x7fb1a414f360 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.873+0000 7fb19a7fc640 1 -- 172.21.15.26:0/547650934 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19403e070 con 0x7fb1a414f360 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.873+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a40c1b30 con 0x7fb1a414f360 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.873+0000 7fb19a7fc640 1 -- 172.21.15.26:0/547650934 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194035040 con 0x7fb1a414f360 2024-03-06T07:27:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.873+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a414e380 con 0x7fb1a414f360 2024-03-06T07:27:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.874+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a404f330 con 0x7fb1a414f360 2024-03-06T07:27:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.874+0000 7fb19a7fc640 1 -- 172.21.15.26:0/547650934 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb194038020 con 0x7fb1a414f360 2024-03-06T07:27:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.875+0000 7fb19a7fc640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb18806b4d0 0x7fb18806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.875+0000 7fb19a7fc640 1 -- 172.21.15.26:0/547650934 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb1940abd50 con 0x7fb1a414f360 2024-03-06T07:27:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.875+0000 7fb1a99c6640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb18806b4d0 0x7fb18806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.876+0000 7fb1a99c6640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb18806b4d0 0x7fb18806d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb1a414e6c0 tx=0x7fb1a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:16.879+0000 7fb19a7fc640 1 -- 172.21.15.26:0/547650934 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb1940af030 con 0x7fb1a414f360 2024-03-06T07:27:17.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.156+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1a4155ed0 con 0x7fb1a414f360 2024-03-06T07:27:17.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.156+0000 7fb19a7fc640 1 -- 172.21.15.26:0/547650934 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7fb19407aad0 con 0x7fb1a414f360 2024-03-06T07:27:17.158 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:15.648611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9627,"qdb_cluster":[9627]},"id":13}]} 2024-03-06T07:27:17.158 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 166 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb18806b4d0 msgr2=0x7fb18806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb18806b4d0 0x7fb18806d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb1a414e6c0 tx=0x7fb1a0009040 comp rx=0 tx=0).stop 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a414f360 msgr2=0x7fb1a414de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a414f360 0x7fb1a414de40 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fb19402b930 tx=0x7fb1940046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 shutdown_connections 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb18806b4d0 0x7fb18806d990 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a4158b60 0x7fb1a41ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a414f360 0x7fb1a414de40 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 --2- 172.21.15.26:0/547650934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a40c88c0 0x7fb1a4158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 >> 172.21.15.26:0/547650934 conn(0x7fb1a40ac570 msgr2=0x7fb1a40c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 shutdown_connections 2024-03-06T07:27:17.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.158+0000 7fb1abc2a640 1 -- 172.21.15.26:0/547650934 wait complete. 2024-03-06T07:27:17.573 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 166, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:52.179125+0000', 'modified': '2024-03-06T07:27:15.648611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9627}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9627': {'gid': 9627, 'name': 'a', 'rank': 0, 'incarnation': 165, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.26:6835/1519487439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1519487439}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1519487439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9627, 'qdb_cluster': [9627]} 2024-03-06T07:27:17.573 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:27:17.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:27:17.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.690+0000 7f0896db5640 1 Processor -- start 2024-03-06T07:27:17.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.690+0000 7f0896db5640 1 -- start start 2024-03-06T07:27:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0896db5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f08901529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0896db5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08901561a0 0x7f08900a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0896db5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08900aa430 0x7f089014f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0896db5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0890155d10 con 0x7f08900aa430 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0896db5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08900a9990 con 0x7f08901561a0 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0896db5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08901556c0 con 0x7f08900c6ab0 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08900aa430 0x7f089014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08900aa430 0x7f089014f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56572/0 (socket says 172.21.15.26:56572) 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 -- 172.21.15.26:0/2095660863 learned_addr learned my addr 172.21.15.26:0/2095660863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0894b51640 1 --2- 172.21.15.26:0/2095660863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f08901529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f088ffff640 1 --2- 172.21.15.26:0/2095660863 >> v2:172.21.15.117:3300/0 conn(0x7f08901561a0 0x7f08900a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 -- 172.21.15.26:0/2095660863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 msgr2=0x7f08901529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 --2- 172.21.15.26:0/2095660863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f08901529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 -- 172.21.15.26:0/2095660863 >> v2:172.21.15.117:3300/0 conn(0x7f08901561a0 msgr2=0x7f08900a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 --2- 172.21.15.26:0/2095660863 >> v2:172.21.15.117:3300/0 conn(0x7f08901561a0 0x7f08900a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f0895352640 1 -- 172.21.15.26:0/2095660863 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08901559a0 con 0x7f08900aa430 2024-03-06T07:27:17.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.691+0000 7f088ffff640 1 --2- 172.21.15.26:0/2095660863 >> v2:172.21.15.117:3300/0 conn(0x7f08901561a0 0x7f08900a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:17.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.692+0000 7f0895352640 1 --2- 172.21.15.26:0/2095660863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f089014f110 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f089006bf60 tx=0x7f088002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fe85ec1c1dd2b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.692+0000 7f088f7fe640 1 -- 172.21.15.26:0/2095660863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880038070 con 0x7f08900aa430 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.692+0000 7f088f7fe640 1 -- 172.21.15.26:0/2095660863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088002f050 con 0x7f08900aa430 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.692+0000 7f088f7fe640 1 -- 172.21.15.26:0/2095660863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031970 con 0x7f08900aa430 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 -- 172.21.15.26:0/2095660863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 msgr2=0x7f089014f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 --2- 172.21.15.26:0/2095660863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f089014f110 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f089006bf60 tx=0x7f088002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 -- 172.21.15.26:0/2095660863 shutdown_connections 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 --2- 172.21.15.26:0/2095660863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f089014f110 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 --2- 172.21.15.26:0/2095660863 >> v2:172.21.15.117:3300/0 conn(0x7f08901561a0 0x7f08900a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 --2- 172.21.15.26:0/2095660863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f08901529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 -- 172.21.15.26:0/2095660863 >> 172.21.15.26:0/2095660863 conn(0x7f089006b1a0 msgr2=0x7f089006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 -- 172.21.15.26:0/2095660863 shutdown_connections 2024-03-06T07:27:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.693+0000 7f0896db5640 1 -- 172.21.15.26:0/2095660863 wait complete. 2024-03-06T07:27:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.694+0000 7f0896db5640 1 Processor -- start 2024-03-06T07:27:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.694+0000 7f0896db5640 1 -- start start 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f08900a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f0890151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08901561a0 0x7f08900c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900c2c10 con 0x7f08900aa430 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900c2a90 con 0x7f08901561a0 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900c6e90 con 0x7f08900c6ab0 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f08900a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f08900a8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56580/0 (socket says 172.21.15.26:56580) 2024-03-06T07:27:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 -- 172.21.15.26:0/2287079656 learned_addr learned my addr 172.21.15.26:0/2287079656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f088ffff640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f0890151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0895352640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08901561a0 0x7f08900c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 -- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 msgr2=0x7f0890151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f0890151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 -- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08901561a0 msgr2=0x7f08900c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08901561a0 0x7f08900c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 -- 172.21.15.26:0/2287079656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08901e8bb0 con 0x7f08900aa430 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0894b51640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f08900a8d50 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f088800eed0 tx=0x7f088800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f088d7fa640 1 -- 172.21.15.26:0/2287079656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888018020 con 0x7f08900aa430 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f088d7fa640 1 -- 172.21.15.26:0/2287079656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088800bd90 con 0x7f08900aa430 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08901e8e90 con 0x7f08900aa430 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f088d7fa640 1 -- 172.21.15.26:0/2287079656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888014620 con 0x7f08900aa430 2024-03-06T07:27:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.696+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08901e9450 con 0x7f08900aa430 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.698+0000 7f088d7fa640 1 -- 172.21.15.26:0/2287079656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0888005d10 con 0x7f08900aa430 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.699+0000 7f088d7fa640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f087c06b500 0x7f087c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.699+0000 7f088d7fa640 1 -- 172.21.15.26:0/2287079656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f088808abf0 con 0x7f08900aa430 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.699+0000 7f088ffff640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f087c06b500 0x7f087c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.699+0000 7f0899995740 1 Processor -- start 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.699+0000 7f0899995740 1 -- start start 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f088ffff640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f087c06b500 0x7f087c06d9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0890155580 tx=0x7f0884008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b14010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b14550 0x561012b5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0899995740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b4e420 0x561012b0e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0899995740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561012a786b0 con 0x561012b4e420 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561012a21a60 con 0x561012b14550 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561012a74750 con 0x561012b5d800 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0895352640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b4e420 0x561012b0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0895352640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b4e420 0x561012b0e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56592/0 (socket says 172.21.15.26:56592) 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0895352640 1 -- 172.21.15.26:0/1358518835 learned_addr learned my addr 172.21.15.26:0/1358518835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0894b51640 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b14010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f088ffff640 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b14550 0x561012b5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0895352640 1 -- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 msgr2=0x561012b14010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0895352640 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b14010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0895352640 1 -- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b14550 msgr2=0x561012b5dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.700+0000 7f0895352640 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b14550 0x561012b5dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.701+0000 7f0895352640 1 -- 172.21.15.26:0/1358518835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561012a3ff30 con 0x561012b4e420 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.701+0000 7f0895352640 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b4e420 0x561012b0e680 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f0880009960 tx=0x7f0880030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3ffccb86dc3b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.701+0000 7f08767fc640 1 -- 172.21.15.26:0/1358518835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880038040 con 0x561012b4e420 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.701+0000 7f08767fc640 1 -- 172.21.15.26:0/1358518835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800317d0 con 0x561012b4e420 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.701+0000 7f08767fc640 1 -- 172.21.15.26:0/1358518835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031b80 con 0x561012b4e420 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.701+0000 7f0899995740 1 -- 172.21.15.26:0/1358518835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b4e420 msgr2=0x561012b0e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.701+0000 7f0899995740 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b4e420 0x561012b0e680 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f0880009960 tx=0x7f0880030000 comp rx=0 tx=0).stop 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 -- 172.21.15.26:0/1358518835 shutdown_connections 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b4e420 0x561012b0e680 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b14550 0x561012b5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 --2- 172.21.15.26:0/1358518835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b14010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 -- 172.21.15.26:0/1358518835 >> 172.21.15.26:0/1358518835 conn(0x561012b107d0 msgr2=0x561012b353c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 -- 172.21.15.26:0/1358518835 shutdown_connections 2024-03-06T07:27:17.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 -- 172.21.15.26:0/1358518835 wait complete. 2024-03-06T07:27:17.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 -- start start 2024-03-06T07:27:17.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.702+0000 7f0899995740 1 Processor -- start 2024-03-06T07:27:17.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b68a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0899995740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b4fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50150 0x561012b58ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0899995740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561012b52590 con 0x561012b5d800 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561012a34ea0 con 0x561012b50150 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561012b35db0 con 0x561012b4e420 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0899995740 10 client.0 _ll_register_callbacks cb 0x561012a34870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f088ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b4fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f088ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b4fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56594/0 (socket says 172.21.15.26:56594) 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f088ffff640 1 -- 172.21.15.26:0/3161402949 learned_addr learned my addr 172.21.15.26:0/3161402949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0894b51640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b68a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.703+0000 7f0895352640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50150 0x561012b58ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f088ffff640 1 -- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 msgr2=0x561012b68a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f088ffff640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b68a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f088ffff640 1 -- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50150 msgr2=0x561012b58ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f088ffff640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50150 0x561012b58ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f088ffff640 1 -- 172.21.15.26:0/3161402949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561012a35270 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0894b51640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b68a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f088ffff640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b4fc10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0884006fd0 tx=0x7f08840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2afb9e692f07b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f086f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f086f7fe640 1 -- 172.21.15.26:0/3161402949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884026070 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f086f7fe640 1 -- 172.21.15.26:0/3161402949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0884016a30 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08800317d0 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f086f7fe640 1 -- 172.21.15.26:0/3161402949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7f088401c040 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 10 client.9678 fetch_fsmap learned FSMap version 166 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=166,monmap=2+}) v3 -- 0x56101278d2d0 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f086f7fe640 1 -- 172.21.15.26:0/3161402949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800317d0 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f086f7fe640 1 -- 172.21.15.26:0/3161402949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 166) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0884034080 con 0x561012b5d800 2024-03-06T07:27:17.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 10 client.9678 fetch_fsmap finished waiting for FSMap version 166 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 10 client.9678 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 20 client.9678 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012b5cd60 0x561012b73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 4 client.9678 mds_command: new command op to 9627 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.704+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 --> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561012a85bf0 con 0x561012b5cd60 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.705+0000 7f0894b51640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012b5cd60 0x561012b73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.705+0000 7f086f7fe640 10 client.9678 ms_handle_connect on v2:172.21.15.26:6833/1519487439 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.705+0000 7f0894b51640 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012b5cd60 0x561012b73cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfffff38bf9e6e6 server_cookie=5715da855b24db04 in_seq=0 out_seq=0 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f086f7fe640 1 -- 172.21.15.26:0/3161402949 <== mds.0 v2:172.21.15.26:6833/1519487439 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561012a85bf0 con 0x561012b5cd60 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f086f7fe640 10 client.9678 handle_command_reply: tid=0 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 client.9678 shutdown 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 msgr2=0x561012b4fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b4fc10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0884006fd0 tx=0x7f08840046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 shutdown_connections 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50150 0x561012b58ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012b5cd60 0x561012b73cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f086f7fe640 0 client.9678 ms_handle_reset on v2:172.21.15.26:6833/1519487439 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b4fc10 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 --2- 172.21.15.26:0/3161402949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b68a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 >> 172.21.15.26:0/3161402949 conn(0x561012b14550 msgr2=0x561012b35630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 shutdown_connections 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 1 -- 172.21.15.26:0/3161402949 wait complete. 2024-03-06T07:27:17.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.706+0000 7f0899995740 20 client.9678 trim_cache size 0 max 16384 2024-03-06T07:27:17.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 Processor -- start 2024-03-06T07:27:17.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 -- start start 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b35610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b5a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50bc0 0x561012b5a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890154f20 con 0x561012b5d800 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089006e240 con 0x561012b50bc0 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0890154920 con 0x561012b4e420 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f088ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b5a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f088ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b5a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56602/0 (socket says 172.21.15.26:56602) 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.723+0000 7f088ffff640 1 -- 172.21.15.26:0/3292937178 learned_addr learned my addr 172.21.15.26:0/3292937178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f088ffff640 1 -- 172.21.15.26:0/3292937178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 msgr2=0x561012b35610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f088ffff640 1 --2- 172.21.15.26:0/3292937178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b35610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f088ffff640 1 -- 172.21.15.26:0/3292937178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50bc0 msgr2=0x561012b5a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f088ffff640 1 --2- 172.21.15.26:0/3292937178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50bc0 0x561012b5a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f088ffff640 1 -- 172.21.15.26:0/3292937178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610128576b0 con 0x561012b5d800 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f088ffff640 1 --2- 172.21.15.26:0/3292937178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b5a260 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0884016d70 tx=0x7f0884023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e47a8126bba7abf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f0875ffb640 1 -- 172.21.15.26:0/3292937178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884026040 con 0x561012b5d800 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f0875ffb640 1 -- 172.21.15.26:0/3292937178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088401d040 con 0x561012b5d800 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.724+0000 7f0875ffb640 1 -- 172.21.15.26:0/3292937178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400f040 con 0x561012b5d800 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 -- 172.21.15.26:0/3292937178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 msgr2=0x561012b5a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 --2- 172.21.15.26:0/3292937178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b5a260 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0884016d70 tx=0x7f0884023000 comp rx=0 tx=0).stop 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 -- 172.21.15.26:0/3292937178 shutdown_connections 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 --2- 172.21.15.26:0/3292937178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b50bc0 0x561012b5a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 --2- 172.21.15.26:0/3292937178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012b5d800 0x561012b5a260 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 --2- 172.21.15.26:0/3292937178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b4e420 0x561012b35610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 -- 172.21.15.26:0/3292937178 >> 172.21.15.26:0/3292937178 conn(0x561012b14550 msgr2=0x561012b65c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 -- 172.21.15.26:0/3292937178 shutdown_connections 2024-03-06T07:27:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 -- 172.21.15.26:0/3292937178 wait complete. 2024-03-06T07:27:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 -- start start 2024-03-06T07:27:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.725+0000 7f0899995740 1 Processor -- start 2024-03-06T07:27:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b4e420 0x561012b5b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0899995740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b4cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0899995740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012bc8d00 0x561012bc9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0899995740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561012b69080 con 0x561012bc8d00 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561012b5b3a0 con 0x561012b4e420 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0899995740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561012b65040 con 0x561012b5d800 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0899995740 10 client.0 _ll_register_callbacks cb 0x7f0884010840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0895352640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012bc8d00 0x561012bc9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0895352640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012bc8d00 0x561012bc9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56608/0 (socket says 172.21.15.26:56608) 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0895352640 1 -- 172.21.15.26:0/3643383882 learned_addr learned my addr 172.21.15.26:0/3643383882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f088ffff640 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b4cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.726+0000 7f0894b51640 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b4e420 0x561012b5b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0895352640 1 -- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 msgr2=0x561012b4cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0895352640 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b4cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0895352640 1 -- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b4e420 msgr2=0x561012b5b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0895352640 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b4e420 0x561012b5b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0895352640 1 -- 172.21.15.26:0/3643383882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561012b51ce0 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0895352640 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012bc8d00 0x561012bc9180 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0880009930 tx=0x7f08800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb133a4e2a91c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f086f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f086f7fe640 1 -- 172.21.15.26:0/3643383882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880038040 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f086f7fe640 1 -- 172.21.15.26:0/3643383882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0880003810 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f088401d040 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f086f7fe640 1 -- 172.21.15.26:0/3643383882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7f088002be70 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 10 client.9684 fetch_fsmap learned FSMap version 166 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=166,monmap=2+}) v3 -- 0x561012b324a0 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f086f7fe640 1 -- 172.21.15.26:0/3643383882 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088401d040 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f086f7fe640 1 -- 172.21.15.26:0/3643383882 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 166) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f088004e020 con 0x561012bc8d00 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 10 client.9684 fetch_fsmap finished waiting for FSMap version 166 2024-03-06T07:27:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 10 client.9684 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 20 client.9684 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012bc9760 0x561012bc25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 4 client.9684 mds_command: new command op to 9627 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.727+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 --> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561012b14d40 con 0x561012bc9760 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0894b51640 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012bc9760 0x561012bc25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f086f7fe640 10 client.9684 ms_handle_connect on v2:172.21.15.26:6833/1519487439 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0894b51640 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012bc9760 0x561012bc25f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2be7e8405d1d8e0 server_cookie=89032194b5190a82 in_seq=0 out_seq=0 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f086f7fe640 1 -- 172.21.15.26:0/3643383882 <== mds.0 v2:172.21.15.26:6833/1519487439 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x561012b14d40 con 0x561012bc9760 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f086f7fe640 10 client.9684 handle_command_reply: tid=0 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 client.9684 shutdown 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012bc8d00 msgr2=0x561012bc9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012bc8d00 0x561012bc9180 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0880009930 tx=0x7f08800046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 shutdown_connections 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:6833/1519487439,v1:172.21.15.26:6835/1519487439] conn(0x561012bc9760 0x561012bc25f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561012bc8d00 0x561012bc9180 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561012b5d800 0x561012b4cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f086f7fe640 0 client.9684 ms_handle_reset on v2:172.21.15.26:6833/1519487439 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 --2- 172.21.15.26:0/3643383882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561012b4e420 0x561012b5b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 >> 172.21.15.26:0/3643383882 conn(0x561012b14550 msgr2=0x561012b65ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 shutdown_connections 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.728+0000 7f0899995740 1 -- 172.21.15.26:0/3643383882 wait complete. 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0899995740 20 client.9684 trim_cache size 0 max 16384 2024-03-06T07:27:17.737 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9627, 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 3.0809451509999999, 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 166, 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 172, 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 172, 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 4.8077259699999999 2024-03-06T07:27:17.738 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:27:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f087c06b500 msgr2=0x7f087c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f087c06b500 0x7f087c06d9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0890155580 tx=0x7f0884008040 comp rx=0 tx=0).stop 2024-03-06T07:27:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 msgr2=0x7f08900a8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f08900a8d50 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f088800eed0 tx=0x7f088800b480 comp rx=0 tx=0).stop 2024-03-06T07:27:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 shutdown_connections 2024-03-06T07:27:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f087c06b500 0x7f087c06d9c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08901561a0 0x7f08900c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08900c6ab0 0x7f0890151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 --2- 172.21.15.26:0/2287079656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08900aa430 0x7f08900a8d50 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 >> 172.21.15.26:0/2287079656 conn(0x7f089006b1a0 msgr2=0x7f08900a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 shutdown_connections 2024-03-06T07:27:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.729+0000 7f0896db5640 1 -- 172.21.15.26:0/2287079656 wait complete. 2024-03-06T07:27:17.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.843+0000 7f17416c3640 1 Processor -- start 2024-03-06T07:27:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.843+0000 7f17416c3640 1 -- start start 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.843+0000 7f17416c3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f173c1527b0 0x7f173c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.843+0000 7f17416c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173c0c7c60 con 0x7f173c0c88b0 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.843+0000 7f17416c3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173c0c8060 con 0x7f173c0c7180 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.843+0000 7f17416c3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173c159640 con 0x7f173c1527b0 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f173affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f173c1527b0 0x7f173c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f173b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f173c0c88b0 0x7f173c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35410/0 (socket says 172.21.15.26:35410) 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f173b7fe640 1 -- 172.21.15.26:0/3866744747 learned_addr learned my addr 172.21.15.26:0/3866744747 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f1739ffb640 1 -- 172.21.15.26:0/3866744747 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228168341 0 0) 0x7f1734001490 con 0x7f173c0c88b0 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f1739ffb640 1 -- 172.21.15.26:0/3866744747 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1724003520 con 0x7f173c0c88b0 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f173affd640 1 -- 172.21.15.26:0/3866744747 >> v1:172.21.15.117:6789/0 conn(0x7f173c0c7180 legacy=0x7f173c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f173affd640 1 -- 172.21.15.26:0/3866744747 >> v1:172.21.15.26:6789/0 conn(0x7f173c0c88b0 legacy=0x7f173c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.844+0000 7f173affd640 1 -- 172.21.15.26:0/3866744747 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c0c76a0 con 0x7f173c1527b0 2024-03-06T07:27:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.845+0000 7f173affd640 1 --2- 172.21.15.26:0/3866744747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c1527b0 0x7f173c159100 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097a0 tx=0x7f172c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b352e7fd42bb108 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.845+0000 7f1739ffb640 1 -- 172.21.15.26:0/3866744747 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c035020 con 0x7f173c1527b0 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.845+0000 7f1739ffb640 1 -- 172.21.15.26:0/3866744747 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c031830 con 0x7f173c1527b0 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.845+0000 7f1739ffb640 1 -- 172.21.15.26:0/3866744747 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c031be0 con 0x7f173c1527b0 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.846+0000 7f17416c3640 1 -- 172.21.15.26:0/3866744747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c1527b0 msgr2=0x7f173c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.846+0000 7f17416c3640 1 --2- 172.21.15.26:0/3866744747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c1527b0 0x7f173c159100 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097a0 tx=0x7f172c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.846+0000 7f17416c3640 1 -- 172.21.15.26:0/3866744747 shutdown_connections 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.846+0000 7f17416c3640 1 --2- 172.21.15.26:0/3866744747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c1527b0 0x7f173c159100 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.846+0000 7f17416c3640 1 -- 172.21.15.26:0/3866744747 >> 172.21.15.26:0/3866744747 conn(0x7f173c0ac560 msgr2=0x7f173c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.846+0000 7f17416c3640 1 -- 172.21.15.26:0/3866744747 shutdown_connections 2024-03-06T07:27:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.846+0000 7f17416c3640 1 -- 172.21.15.26:0/3866744747 wait complete. 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.847+0000 7f17416c3640 1 Processor -- start 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.847+0000 7f17416c3640 1 -- start start 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f17416c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c0c7180 0x7f173c154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f17416c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173c0c88b0 0x7f173c0c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f17416c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173c1527b0 0x7f173c0c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f17416c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c155600 con 0x7f173c0c7180 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f17416c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c155320 con 0x7f173c1527b0 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f17416c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c152b90 con 0x7f173c0c88b0 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c0c7180 0x7f173c154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c0c7180 0x7f173c154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56620/0 (socket says 172.21.15.26:56620) 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 -- 172.21.15.26:0/1818562214 learned_addr learned my addr 172.21.15.26:0/1818562214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173b7fe640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173c1527b0 0x7f173c0c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173a7fc640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173c0c88b0 0x7f173c0c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 -- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173c0c88b0 msgr2=0x7f173c0c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173c0c88b0 0x7f173c0c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 -- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173c1527b0 msgr2=0x7f173c0c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173c1527b0 0x7f173c0c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 -- 172.21.15.26:0/1818562214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c154e50 con 0x7f173c0c7180 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173a7fc640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173c0c88b0 0x7f173c0c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f173affd640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c0c7180 0x7f173c154220 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097a0 tx=0x7f172c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.848+0000 7f17237fe640 1 -- 172.21.15.26:0/1818562214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c035040 con 0x7f173c0c7180 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.849+0000 7f17237fe640 1 -- 172.21.15.26:0/1818562214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c004250 con 0x7f173c0c7180 2024-03-06T07:27:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.849+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c0c8ec0 con 0x7f173c0c7180 2024-03-06T07:27:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.849+0000 7f17237fe640 1 -- 172.21.15.26:0/1818562214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c03a640 con 0x7f173c0c7180 2024-03-06T07:27:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.849+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c0c9480 con 0x7f173c0c7180 2024-03-06T07:27:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.850+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c04f330 con 0x7f173c0c7180 2024-03-06T07:27:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.850+0000 7f17237fe640 1 -- 172.21.15.26:0/1818562214 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f172c038020 con 0x7f173c0c7180 2024-03-06T07:27:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.851+0000 7f17237fe640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f172806b4d0 0x7f172806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.851+0000 7f17237fe640 1 -- 172.21.15.26:0/1818562214 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f172c0ac110 con 0x7f173c0c7180 2024-03-06T07:27:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.851+0000 7f173a7fc640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f172806b4d0 0x7f172806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.851+0000 7f173a7fc640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f172806b4d0 0x7f172806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f173c14c4b0 tx=0x7f1730009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:17.855+0000 7f17237fe640 1 -- 172.21.15.26:0/1818562214 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f172c07ae00 con 0x7f173c0c7180 2024-03-06T07:27:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.179+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f173c06ce00 con 0x7f173c0c7180 2024-03-06T07:27:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.180+0000 7f17237fe640 1 -- 172.21.15.26:0/1818562214 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f172c07a700 con 0x7f173c0c7180 2024-03-06T07:27:18.182 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:18.182 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:15.648611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9627,"qdb_cluster":[9627]},"id":13}]} 2024-03-06T07:27:18.183 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 166 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f172806b4d0 msgr2=0x7f172806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f172806b4d0 0x7f172806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f173c14c4b0 tx=0x7f1730009040 comp rx=0 tx=0).stop 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c0c7180 msgr2=0x7f173c154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c0c7180 0x7f173c154220 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097a0 tx=0x7f172c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 shutdown_connections 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f172806b4d0 0x7f172806d990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173c1527b0 0x7f173c0c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173c0c88b0 0x7f173c0c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 --2- 172.21.15.26:0/1818562214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c0c7180 0x7f173c154220 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 >> 172.21.15.26:0/1818562214 conn(0x7f173c0ac560 msgr2=0x7f173c0aade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 shutdown_connections 2024-03-06T07:27:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.182+0000 7f17416c3640 1 -- 172.21.15.26:0/1818562214 wait complete. 2024-03-06T07:27:18.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:27:18.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:18.233 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:18.288 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:27:18.288 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:27:18.288 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:18.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:27:18.304 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:27:18.304 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:18.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:27:18.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.442+0000 7f9ce0325640 1 Processor -- start 2024-03-06T07:27:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.442+0000 7f9ce0325640 1 -- start start 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9ce0325640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9cd80c88c0 0x7f9cd8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9ce0325640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cd8158b60 0x7f9cd8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9ce0325640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd80c73d0 con 0x7f9cd814f360 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9ce0325640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd80c8dd0 con 0x7f9cd8158b60 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9ce0325640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd80be2e0 con 0x7f9cd80c88c0 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9cde8c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cd8158b60 0x7f9cd8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9cde8c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cd8158b60 0x7f9cd8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56624/0 (socket says 172.21.15.26:56624) 2024-03-06T07:27:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.443+0000 7f9cde8c2640 1 -- 172.21.15.26:0/2675824469 learned_addr learned my addr 172.21.15.26:0/2675824469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.444+0000 7f9cde8c2640 1 -- 172.21.15.26:0/2675824469 >> v2:172.21.15.117:3300/0 conn(0x7f9cd80c88c0 msgr2=0x7f9cd8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.444+0000 7f9cdd8c0640 1 --2- 172.21.15.26:0/2675824469 >> v2:172.21.15.117:3300/0 conn(0x7f9cd80c88c0 0x7f9cd8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.444+0000 7f9cde8c2640 1 --2- 172.21.15.26:0/2675824469 >> v2:172.21.15.117:3300/0 conn(0x7f9cd80c88c0 0x7f9cd8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.444+0000 7f9cde8c2640 1 -- 172.21.15.26:0/2675824469 >> v1:172.21.15.117:6789/0 conn(0x7f9cd814f360 legacy=0x7f9cd80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.444+0000 7f9cde8c2640 1 -- 172.21.15.26:0/2675824469 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd80c7670 con 0x7f9cd8158b60 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.444+0000 7f9cdd8c0640 1 --2- 172.21.15.26:0/2675824469 >> v2:172.21.15.117:3300/0 conn(0x7f9cd80c88c0 0x7f9cd8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.444+0000 7f9cde8c2640 1 --2- 172.21.15.26:0/2675824469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8158ff0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9cd80c84c0 tx=0x7f9cd002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc57ce24dfe15414 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9cdd0bf640 1 -- 172.21.15.26:0/2675824469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd002e070 con 0x7f9cd8158b60 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9cdd0bf640 1 -- 172.21.15.26:0/2675824469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd0037070 con 0x7f9cd8158b60 2024-03-06T07:27:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9cdd0bf640 1 -- 172.21.15.26:0/2675824469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0031a00 con 0x7f9cd8158b60 2024-03-06T07:27:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 -- 172.21.15.26:0/2675824469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 msgr2=0x7f9cd8158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 --2- 172.21.15.26:0/2675824469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8158ff0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9cd80c84c0 tx=0x7f9cd002b880 comp rx=0 tx=0).stop 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 -- 172.21.15.26:0/2675824469 shutdown_connections 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 --2- 172.21.15.26:0/2675824469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8158ff0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 --2- 172.21.15.26:0/2675824469 >> v2:172.21.15.117:3300/0 conn(0x7f9cd80c88c0 0x7f9cd8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 -- 172.21.15.26:0/2675824469 >> 172.21.15.26:0/2675824469 conn(0x7f9cd80ac570 msgr2=0x7f9cd80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 -- 172.21.15.26:0/2675824469 shutdown_connections 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.445+0000 7f9ce0325640 1 -- 172.21.15.26:0/2675824469 wait complete. 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.447+0000 7f9ce0325640 1 Processor -- start 2024-03-06T07:27:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.447+0000 7f9ce0325640 1 -- start start 2024-03-06T07:27:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9ce0325640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cd80c88c0 0x7f9cd81519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9ce0325640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cd814f360 0x7f9cd81504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9ce0325640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9ce0325640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80c5a50 con 0x7f9cd8158b60 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9ce0325640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80c7c30 con 0x7f9cd80c88c0 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9ce0325640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80c5bd0 con 0x7f9cd814f360 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56632/0 (socket says 172.21.15.26:56632) 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 -- 172.21.15.26:0/1507574051 learned_addr learned my addr 172.21.15.26:0/1507574051 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde0c1640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cd80c88c0 0x7f9cd81519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cdd8c0640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cd814f360 0x7f9cd81504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 -- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cd814f360 msgr2=0x7f9cd81504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cd814f360 0x7f9cd81504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 -- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cd80c88c0 msgr2=0x7f9cd81519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cd80c88c0 0x7f9cd81519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 -- 172.21.15.26:0/1507574051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd814f740 con 0x7f9cd8158b60 2024-03-06T07:27:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde0c1640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cd80c88c0 0x7f9cd81519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.448+0000 7f9cde8c2640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8150a20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f9cd002b8c0 tx=0x7f9cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.449+0000 7f9cceffd640 1 -- 172.21.15.26:0/1507574051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0032880 con 0x7f9cd8158b60 2024-03-06T07:27:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.449+0000 7f9cceffd640 1 -- 172.21.15.26:0/1507574051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd0032a40 con 0x7f9cd8158b60 2024-03-06T07:27:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.449+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd81ebb30 con 0x7f9cd8158b60 2024-03-06T07:27:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.449+0000 7f9cceffd640 1 -- 172.21.15.26:0/1507574051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0032e30 con 0x7f9cd8158b60 2024-03-06T07:27:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.449+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd814fce0 con 0x7f9cd8158b60 2024-03-06T07:27:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.450+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd804f330 con 0x7f9cd8158b60 2024-03-06T07:27:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.450+0000 7f9cceffd640 1 -- 172.21.15.26:0/1507574051 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cd00453f0 con 0x7f9cd8158b60 2024-03-06T07:27:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.451+0000 7f9cceffd640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9cbc06b4d0 0x7f9cbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.451+0000 7f9cceffd640 1 -- 172.21.15.26:0/1507574051 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9cd00ac3c0 con 0x7f9cd8158b60 2024-03-06T07:27:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.451+0000 7f9cde0c1640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9cbc06b4d0 0x7f9cbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.451+0000 7f9cde0c1640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9cbc06b4d0 0x7f9cbc06d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40045b0 tx=0x7f9cd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.455+0000 7f9cceffd640 1 -- 172.21.15.26:0/1507574051 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9cd0040070 con 0x7f9cd8158b60 2024-03-06T07:27:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.672+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cd81553c0 con 0x7f9cd8158b60 2024-03-06T07:27:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.673+0000 7f9cceffd640 1 -- 172.21.15.26:0/1507574051 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f9cd007b140 con 0x7f9cd8158b60 2024-03-06T07:27:18.675 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:18.675 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:15.648611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9627,"qdb_cluster":[9627]},"id":13}]} 2024-03-06T07:27:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 166 2024-03-06T07:27:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9cbc06b4d0 msgr2=0x7f9cbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9cbc06b4d0 0x7f9cbc06d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40045b0 tx=0x7f9cd4009040 comp rx=0 tx=0).stop 2024-03-06T07:27:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 msgr2=0x7f9cd8150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8150a20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f9cd002b8c0 tx=0x7f9cd00046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 shutdown_connections 2024-03-06T07:27:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9cbc06b4d0 0x7f9cbc06d990 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cd8158b60 0x7f9cd8150a20 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cd814f360 0x7f9cd81504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 --2- 172.21.15.26:0/1507574051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cd80c88c0 0x7f9cd81519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.674+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 >> 172.21.15.26:0/1507574051 conn(0x7f9cd80ac570 msgr2=0x7f9cd80ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.675+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 shutdown_connections 2024-03-06T07:27:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.675+0000 7f9ce0325640 1 -- 172.21.15.26:0/1507574051 wait complete. 2024-03-06T07:27:18.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:18.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.787+0000 7f84e6f32640 1 Processor -- start 2024-03-06T07:27:18.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.787+0000 7f84e6f32640 1 -- start start 2024-03-06T07:27:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e6f32640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f84e014f360 0x7f84e0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e6f32640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84e00c88c0 0x7f84e00c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e6f32640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c5880 0x7f84e0158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e6f32640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e00c8dd0 con 0x7f84e00c88c0 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e6f32640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e00c73d0 con 0x7f84e014f360 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e6f32640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e00be2e0 con 0x7f84e00c5880 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84e00c88c0 0x7f84e00c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84e00c88c0 0x7f84e00c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56636/0 (socket says 172.21.15.26:56636) 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 -- 172.21.15.26:0/1444704636 learned_addr learned my addr 172.21.15.26:0/1444704636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e54cf640 1 --2- 172.21.15.26:0/1444704636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c5880 0x7f84e0158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e4cce640 1 --2- 172.21.15.26:0/1444704636 >> v2:172.21.15.117:3300/0 conn(0x7f84e014f360 0x7f84e0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 -- 172.21.15.26:0/1444704636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c5880 msgr2=0x7f84e0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 --2- 172.21.15.26:0/1444704636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c5880 0x7f84e0158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 -- 172.21.15.26:0/1444704636 >> v2:172.21.15.117:3300/0 conn(0x7f84e014f360 msgr2=0x7f84e0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 --2- 172.21.15.26:0/1444704636 >> v2:172.21.15.117:3300/0 conn(0x7f84e014f360 0x7f84e0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84dffff640 1 -- 172.21.15.26:0/1444704636 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e00c7670 con 0x7f84e00c88c0 2024-03-06T07:27:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e54cf640 1 --2- 172.21.15.26:0/1444704636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c5880 0x7f84e0158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.788+0000 7f84e4cce640 1 --2- 172.21.15.26:0/1444704636 >> v2:172.21.15.117:3300/0 conn(0x7f84e014f360 0x7f84e0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.789+0000 7f84dffff640 1 --2- 172.21.15.26:0/1444704636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e00c88c0 0x7f84e00c5340 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f84e008c840 tx=0x7f84cc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e30ba6629797ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.789+0000 7f84df7fe640 1 -- 172.21.15.26:0/1444704636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0317d0 con 0x7f84e00c88c0 2024-03-06T07:27:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.789+0000 7f84df7fe640 1 -- 172.21.15.26:0/1444704636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84cc037070 con 0x7f84e00c88c0 2024-03-06T07:27:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.789+0000 7f84df7fe640 1 -- 172.21.15.26:0/1444704636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc031b90 con 0x7f84e00c88c0 2024-03-06T07:27:18.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 -- 172.21.15.26:0/1444704636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e00c88c0 msgr2=0x7f84e00c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 --2- 172.21.15.26:0/1444704636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e00c88c0 0x7f84e00c5340 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f84e008c840 tx=0x7f84cc02b870 comp rx=0 tx=0).stop 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 -- 172.21.15.26:0/1444704636 shutdown_connections 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 --2- 172.21.15.26:0/1444704636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c5880 0x7f84e0158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 --2- 172.21.15.26:0/1444704636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e00c88c0 0x7f84e00c5340 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 --2- 172.21.15.26:0/1444704636 >> v2:172.21.15.117:3300/0 conn(0x7f84e014f360 0x7f84e0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 -- 172.21.15.26:0/1444704636 >> 172.21.15.26:0/1444704636 conn(0x7f84e00ac570 msgr2=0x7f84e00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 -- 172.21.15.26:0/1444704636 shutdown_connections 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.790+0000 7f84e6f32640 1 -- 172.21.15.26:0/1444704636 wait complete. 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.791+0000 7f84e6f32640 1 Processor -- start 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.791+0000 7f84e6f32640 1 -- start start 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84e00c5880 0x7f84e00abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c88c0 0x7f84e01e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e014f360 0x7f84e01e8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e01590a0 con 0x7f84e014f360 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e00c7c30 con 0x7f84e00c5880 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e0159220 con 0x7f84e00c88c0 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e014f360 0x7f84e01e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e014f360 0x7f84e01e8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56638/0 (socket says 172.21.15.26:56638) 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 -- 172.21.15.26:0/442022752 learned_addr learned my addr 172.21.15.26:0/442022752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84dffff640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c88c0 0x7f84e01e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e4cce640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84e00c5880 0x7f84e00abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 -- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c88c0 msgr2=0x7f84e01e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c88c0 0x7f84e01e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 -- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84e00c5880 msgr2=0x7f84e00abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84e00c5880 0x7f84e00abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 -- 172.21.15.26:0/442022752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e01ee140 con 0x7f84e014f360 2024-03-06T07:27:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e54cf640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e014f360 0x7f84e01e8180 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f84d8007e10 tx=0x7f84d800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:18.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84dd7fa640 1 -- 172.21.15.26:0/442022752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d801a070 con 0x7f84e014f360 2024-03-06T07:27:18.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84dd7fa640 1 -- 172.21.15.26:0/442022752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8003b90 con 0x7f84e014f360 2024-03-06T07:27:18.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e01ee420 con 0x7f84e014f360 2024-03-06T07:27:18.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84dd7fa640 1 -- 172.21.15.26:0/442022752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8004030 con 0x7f84e014f360 2024-03-06T07:27:18.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.792+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e0156700 con 0x7f84e014f360 2024-03-06T07:27:18.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.793+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e004f330 con 0x7f84e014f360 2024-03-06T07:27:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.794+0000 7f84dd7fa640 1 -- 172.21.15.26:0/442022752 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84d801d070 con 0x7f84e014f360 2024-03-06T07:27:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.794+0000 7f84dd7fa640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f84c406b500 0x7f84c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.794+0000 7f84dd7fa640 1 -- 172.21.15.26:0/442022752 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84d808a5b0 con 0x7f84e014f360 2024-03-06T07:27:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.794+0000 7f84e4cce640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f84c406b500 0x7f84c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.795+0000 7f84e4cce640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f84c406b500 0x7f84c406d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f84d4006fd0 tx=0x7f84d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.803+0000 7f84dd7fa640 1 -- 172.21.15.26:0/442022752 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f84d8010070 con 0x7f84e014f360 2024-03-06T07:27:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.986+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84e006cda0 con 0x7f84e014f360 2024-03-06T07:27:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.987+0000 7f84dd7fa640 1 -- 172.21.15.26:0/442022752 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f84d8059330 con 0x7f84e014f360 2024-03-06T07:27:18.990 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:18.990 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":172,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:53.824659+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:50.446132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:51.147227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f84c406b500 msgr2=0x7f84c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f84c406b500 0x7f84c406d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f84d4006fd0 tx=0x7f84d4008040 comp rx=0 tx=0).stop 2024-03-06T07:27:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e014f360 msgr2=0x7f84e01e8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e014f360 0x7f84e01e8180 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f84d8007e10 tx=0x7f84d800d4a0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 shutdown_connections 2024-03-06T07:27:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f84c406b500 0x7f84c406d9c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84e014f360 0x7f84e01e8180 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84e00c88c0 0x7f84e01e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 --2- 172.21.15.26:0/442022752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84e00c5880 0x7f84e00abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 >> 172.21.15.26:0/442022752 conn(0x7f84e00ac570 msgr2=0x7f84e00c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 shutdown_connections 2024-03-06T07:27:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:18.989+0000 7f84e6f32640 1 -- 172.21.15.26:0/442022752 wait complete. 2024-03-06T07:27:19.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:26:50.446132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:26:51.147227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:27:19.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:27:19.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.106+0000 7ffb105d7640 1 Processor -- start 2024-03-06T07:27:19.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.106+0000 7ffb105d7640 1 -- start start 2024-03-06T07:27:19.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb105d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 0x7ffb080c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb105d7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffb08073190 0x7ffb08156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb105d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb08072600 con 0x7ffb080c5880 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb105d7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb080723b0 con 0x7ffb08073190 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb105d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080728e0 con 0x7ffb080c5c60 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb0e373640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffb080c5880 0x7ffb080c6ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35424/0 (socket says 172.21.15.26:35424) 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb0e373640 1 -- 172.21.15.26:0/717611919 learned_addr learned my addr 172.21.15.26:0/717611919 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb0db72640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 0x7ffb080c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb0eb74640 1 --2- 172.21.15.26:0/717611919 >> v2:172.21.15.117:3300/0 conn(0x7ffb08073190 0x7ffb08156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992528626 0 0) 0x7ffb08072600 con 0x7ffb080c5880 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.107+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf4003520 con 0x7ffb080c5880 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 161130888 0 0) 0x7ffaf4003520 con 0x7ffb080c5880 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 msgr2=0x7ffb080c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 0x7ffb080c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 >> v2:172.21.15.117:3300/0 conn(0x7ffb08073190 msgr2=0x7ffb08156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 --2- 172.21.15.26:0/717611919 >> v2:172.21.15.117:3300/0 conn(0x7ffb08073190 0x7ffb08156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb08153af0 con 0x7ffb080c5880 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7ffb04001670 con 0x7ffb080c5880 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 >> v1:172.21.15.26:6789/0 conn(0x7ffb080c5880 legacy=0x7ffb080c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40039b0 0x7ffaf4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4004800 0x7ffaf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 0x7ffaf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080723b0 con 0x7ffaf40039b0 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080728e0 con 0x7ffaf4004800 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf400ba40 con 0x7ffb080c5c60 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0e373640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40039b0 0x7ffaf4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0db72640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4004800 0x7ffaf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0eb74640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 0x7ffaf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0e373640 1 -- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 msgr2=0x7ffaf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:19.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0e373640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb080c5c60 0x7ffaf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0e373640 1 -- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4004800 msgr2=0x7ffaf4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0e373640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4004800 0x7ffaf4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.108+0000 7ffb0e373640 1 -- 172.21.15.26:0/717611919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4003520 con 0x7ffaf40039b0 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.109+0000 7ffb0e373640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40039b0 0x7ffaf4003dd0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ffb04007fd0 tx=0x7ffaf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7bdd9fe8395d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.109+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04007d50 con 0x7ffaf40039b0 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.109+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb04030070 con 0x7ffaf40039b0 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.109+0000 7ffb0d371640 1 -- 172.21.15.26:0/717611919 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0402cd40 con 0x7ffaf40039b0 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.110+0000 7ffb105d7640 1 -- 172.21.15.26:0/717611919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40039b0 msgr2=0x7ffaf4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.110+0000 7ffb105d7640 1 --2- 172.21.15.26:0/717611919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40039b0 0x7ffaf4003dd0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ffb04007fd0 tx=0x7ffaf4005830 comp rx=0 tx=0).stop 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.110+0000 7ffb0eb74640 1 -- 172.21.15.26:0/717611919 reap_dead start 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.110+0000 7ffb105d7640 1 -- 172.21.15.26:0/717611919 shutdown_connections 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.110+0000 7ffb105d7640 1 -- 172.21.15.26:0/717611919 >> 172.21.15.26:0/717611919 conn(0x7ffb080aa630 msgr2=0x7ffb080aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.110+0000 7ffb105d7640 1 -- 172.21.15.26:0/717611919 shutdown_connections 2024-03-06T07:27:19.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.110+0000 7ffb105d7640 1 -- 172.21.15.26:0/717611919 wait complete. 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.111+0000 7ffb105d7640 1 Processor -- start 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.111+0000 7ffb105d7640 1 -- start start 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf40039b0 0x7ffb080c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb080ac140 0x7ffb08148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb08148b60 0x7ffb0814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080c73b0 con 0x7ffb080ac140 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080c6780 con 0x7ffaf40039b0 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080c7530 con 0x7ffb08148b60 2024-03-06T07:27:19.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb080ac140 0x7ffb08148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb080ac140 0x7ffb08148620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56648/0 (socket says 172.21.15.26:56648) 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 -- 172.21.15.26:0/2193383183 learned_addr learned my addr 172.21.15.26:0/2193383183 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0e373640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf40039b0 0x7ffb080c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 -- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb08148b60 msgr2=0x7ffb0814bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb08148b60 0x7ffb0814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 -- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf40039b0 msgr2=0x7ffb080c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf40039b0 0x7ffb080c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 -- 172.21.15.26:0/2193383183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb080aba70 con 0x7ffb080ac140 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb0db72640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb080ac140 0x7ffb08148620 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8011590 tx=0x7ffaf8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffafeffd640 1 -- 172.21.15.26:0/2193383183 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8019070 con 0x7ffb080ac140 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffafeffd640 1 -- 172.21.15.26:0/2193383183 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8004220 con 0x7ffb080ac140 2024-03-06T07:27:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0806d8e0 con 0x7ffb080ac140 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffafeffd640 1 -- 172.21.15.26:0/2193383183 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8015730 con 0x7ffb080ac140 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.112+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0806dea0 con 0x7ffb080ac140 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.113+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0806e0a0 con 0x7ffb080ac140 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.114+0000 7ffafeffd640 1 -- 172.21.15.26:0/2193383183 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaf80158d0 con 0x7ffb080ac140 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.114+0000 7ffafeffd640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffaec06b500 0x7ffaec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.115+0000 7ffafeffd640 1 -- 172.21.15.26:0/2193383183 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffaf808a830 con 0x7ffb080ac140 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.115+0000 7ffb0e373640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffaec06b500 0x7ffaec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.115+0000 7ffb0e373640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffaec06b500 0x7ffaec06d9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ffb04034000 tx=0x7ffb04039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.122+0000 7ffafeffd640 1 -- 172.21.15.26:0/2193383183 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffb0806e0a0 con 0x7ffb080ac140 2024-03-06T07:27:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.347+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb0804f330 con 0x7ffb080ac140 2024-03-06T07:27:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.348+0000 7ffafeffd640 1 -- 172.21.15.26:0/2193383183 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffaf8059520 con 0x7ffb080ac140 2024-03-06T07:27:19.351 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:27:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffaec06b500 msgr2=0x7ffaec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffaec06b500 0x7ffaec06d9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ffb04034000 tx=0x7ffb04039040 comp rx=0 tx=0).stop 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb080ac140 msgr2=0x7ffb08148620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb080ac140 0x7ffb08148620 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8011590 tx=0x7ffaf8011a50 comp rx=0 tx=0).stop 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 shutdown_connections 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffaec06b500 0x7ffaec06d9c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb08148b60 0x7ffb0814bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb080ac140 0x7ffb08148620 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 --2- 172.21.15.26:0/2193383183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf40039b0 0x7ffb080c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 >> 172.21.15.26:0/2193383183 conn(0x7ffb080aa630 msgr2=0x7ffb08151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 shutdown_connections 2024-03-06T07:27:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.350+0000 7ffb105d7640 1 -- 172.21.15.26:0/2193383183 wait complete. 2024-03-06T07:27:19.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:27:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.468+0000 7fac0be81640 1 Processor -- start 2024-03-06T07:27:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.468+0000 7fac0be81640 1 -- start start 2024-03-06T07:27:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac0be81640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fac040a8290 0x7fac041539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac0be81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040c62a0 con 0x7fac040c4740 2024-03-06T07:27:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac0be81640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040c4c00 con 0x7fac040c69a0 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac0be81640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac04153f00 con 0x7fac040a8290 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac0941c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fac040c4740 0x7fac040701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35434/0 (socket says 172.21.15.26:35434) 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac0941c640 1 -- 172.21.15.26:0/3325457736 learned_addr learned my addr 172.21.15.26:0/3325457736 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac09c1d640 1 --2- 172.21.15.26:0/3325457736 >> v2:172.21.15.117:3300/0 conn(0x7fac040a8290 0x7fac041539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac08c1b640 1 -- 172.21.15.26:0/3325457736 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986853234 0 0) 0x7fac040c62a0 con 0x7fac040c4740 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.469+0000 7fac08c1b640 1 -- 172.21.15.26:0/3325457736 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf0003520 con 0x7fac040c4740 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.470+0000 7fac08c1b640 1 -- 172.21.15.26:0/3325457736 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708562283 0 0) 0x7fac040c4c00 con 0x7fac040c69a0 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.470+0000 7fac08c1b640 1 -- 172.21.15.26:0/3325457736 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac040c62a0 con 0x7fac040c69a0 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.470+0000 7fac09c1d640 1 -- 172.21.15.26:0/3325457736 >> v1:172.21.15.117:6789/0 conn(0x7fac040c69a0 legacy=0x7fac04072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.470+0000 7fac09c1d640 1 -- 172.21.15.26:0/3325457736 >> v1:172.21.15.26:6789/0 conn(0x7fac040c4740 legacy=0x7fac040701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.470+0000 7fac09c1d640 1 -- 172.21.15.26:0/3325457736 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040c4ee0 con 0x7fac040a8290 2024-03-06T07:27:19.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.471+0000 7fac09c1d640 1 --2- 172.21.15.26:0/3325457736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040a8290 0x7fac041539c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac0002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a8ae77960ced34 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.471+0000 7fac08c1b640 1 -- 172.21.15.26:0/3325457736 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac000315a0 con 0x7fac040a8290 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.471+0000 7fac08c1b640 1 -- 172.21.15.26:0/3325457736 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00037070 con 0x7fac040a8290 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.471+0000 7fac08c1b640 1 -- 172.21.15.26:0/3325457736 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00031940 con 0x7fac040a8290 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.472+0000 7fac0be81640 1 -- 172.21.15.26:0/3325457736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040a8290 msgr2=0x7fac041539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.472+0000 7fac0be81640 1 --2- 172.21.15.26:0/3325457736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040a8290 0x7fac041539c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac0002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.472+0000 7fac0be81640 1 -- 172.21.15.26:0/3325457736 shutdown_connections 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.472+0000 7fac0be81640 1 --2- 172.21.15.26:0/3325457736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040a8290 0x7fac041539c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.472+0000 7fac0be81640 1 -- 172.21.15.26:0/3325457736 >> 172.21.15.26:0/3325457736 conn(0x7fac040c5560 msgr2=0x7fac040c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.472+0000 7fac0be81640 1 -- 172.21.15.26:0/3325457736 shutdown_connections 2024-03-06T07:27:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.472+0000 7fac0be81640 1 -- 172.21.15.26:0/3325457736 wait complete. 2024-03-06T07:27:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 Processor -- start 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 -- start start 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac040a8290 0x7fac041ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040c4740 0x7fac0414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac040c69a0 0x7fac0414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04072d90 con 0x7fac040a8290 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0406fc80 con 0x7fac040c4740 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac0be81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac04072f10 con 0x7fac040c69a0 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac09c1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac040a8290 0x7fac041ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.473+0000 7fac09c1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac040a8290 0x7fac041ee4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56658/0 (socket says 172.21.15.26:56658) 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac09c1d640 1 -- 172.21.15.26:0/1598988851 learned_addr learned my addr 172.21.15.26:0/1598988851 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac0a41e640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac040c69a0 0x7fac0414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac0941c640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040c4740 0x7fac0414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac09c1d640 1 -- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac040c69a0 msgr2=0x7fac0414dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac09c1d640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac040c69a0 0x7fac0414dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac09c1d640 1 -- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040c4740 msgr2=0x7fac0414d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac09c1d640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040c4740 0x7fac0414d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac09c1d640 1 -- 172.21.15.26:0/1598988851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac041eefd0 con 0x7fac040a8290 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac0941c640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040c4740 0x7fac0414d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac09c1d640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac040a8290 0x7fac041ee4f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fabfa7fc640 1 -- 172.21.15.26:0/1598988851 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00032630 con 0x7fac040a8290 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fabfa7fc640 1 -- 172.21.15.26:0/1598988851 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac000327d0 con 0x7fac040a8290 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac041eed30 con 0x7fac040a8290 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fabfa7fc640 1 -- 172.21.15.26:0/1598988851 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00032ba0 con 0x7fac040a8290 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.474+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac04156820 con 0x7fac040a8290 2024-03-06T07:27:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.475+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac04067ad0 con 0x7fac040a8290 2024-03-06T07:27:19.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.476+0000 7fabfa7fc640 1 -- 172.21.15.26:0/1598988851 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac0003b3f0 con 0x7fac040a8290 2024-03-06T07:27:19.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.476+0000 7fabfa7fc640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabe806b500 0x7fabe806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.476+0000 7fabfa7fc640 1 -- 172.21.15.26:0/1598988851 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac0002f070 con 0x7fac040a8290 2024-03-06T07:27:19.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.476+0000 7fac0941c640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabe806b500 0x7fabe806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.477+0000 7fac0941c640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabe806b500 0x7fabe806d9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000f60 tx=0x7fabf4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.481+0000 7fabfa7fc640 1 -- 172.21.15.26:0/1598988851 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fac04067ad0 con 0x7fac040a8290 2024-03-06T07:27:19.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.660+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fac04152d60 con 0x7fac040a8290 2024-03-06T07:27:19.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.661+0000 7fabfa7fc640 1 -- 172.21.15.26:0/1598988851 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7fac04152d60 con 0x7fac040a8290 2024-03-06T07:27:19.664 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:27:19.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabe806b500 msgr2=0x7fabe806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabe806b500 0x7fabe806d9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000f60 tx=0x7fabf4009040 comp rx=0 tx=0).stop 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac040a8290 msgr2=0x7fac041ee4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac040a8290 0x7fac041ee4f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac000046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 shutdown_connections 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fabe806b500 0x7fabe806d9c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac040c69a0 0x7fac0414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac040c4740 0x7fac0414d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 --2- 172.21.15.26:0/1598988851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac040a8290 0x7fac041ee4f0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 >> 172.21.15.26:0/1598988851 conn(0x7fac040c5560 msgr2=0x7fac040c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 shutdown_connections 2024-03-06T07:27:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.663+0000 7fac0be81640 1 -- 172.21.15.26:0/1598988851 wait complete. 2024-03-06T07:27:19.676 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:27:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.778+0000 7f73a8b65640 1 Processor -- start 2024-03-06T07:27:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.778+0000 7f73a8b65640 1 -- start start 2024-03-06T07:27:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a8b65640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73a40c3a70 0x7f73a40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a8b65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a40c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a8b65640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a4156060 con 0x7f73a406d750 2024-03-06T07:27:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a8b65640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a4155b30 con 0x7f73a40c3a70 2024-03-06T07:27:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a8b65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40a8430 con 0x7f73a406db30 2024-03-06T07:27:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a40c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a1d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73a40c3a70 0x7f73a40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a259c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f73a406d750 0x7f73a4072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:34720/0 (socket says 172.21.15.26:34720) 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a1d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73a40c3a70 0x7f73a40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56362/0 (socket says 172.21.15.26:56362) 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a259c640 1 -- 172.21.15.26:0/2552174711 learned_addr learned my addr 172.21.15.26:0/2552174711 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.779+0000 7f73a2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a40c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:44814/0 (socket says 172.21.15.26:44814) 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.780+0000 7f73a2d9d640 1 -- 172.21.15.26:0/2552174711 >> v2:172.21.15.117:3300/0 conn(0x7f73a40c3a70 msgr2=0x7f73a40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.780+0000 7f73a2d9d640 1 --2- 172.21.15.26:0/2552174711 >> v2:172.21.15.117:3300/0 conn(0x7f73a40c3a70 0x7f73a40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.780+0000 7f73a2d9d640 1 -- 172.21.15.26:0/2552174711 >> v1:172.21.15.117:6789/0 conn(0x7f73a406d750 legacy=0x7f73a4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.780+0000 7f73a2d9d640 1 -- 172.21.15.26:0/2552174711 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4155d80 con 0x7f73a406db30 2024-03-06T07:27:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.780+0000 7f73a2d9d640 1 --2- 172.21.15.26:0/2552174711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a40c1e90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f739c009d90 tx=0x7f739c02bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1a42a52fb369d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a159a640 1 -- 172.21.15.26:0/2552174711 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c039070 con 0x7f73a406db30 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a159a640 1 -- 172.21.15.26:0/2552174711 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c02cb30 con 0x7f73a406db30 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a159a640 1 -- 172.21.15.26:0/2552174711 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c034440 con 0x7f73a406db30 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 -- 172.21.15.26:0/2552174711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 msgr2=0x7f73a40c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 --2- 172.21.15.26:0/2552174711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a40c1e90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f739c009d90 tx=0x7f739c02bd80 comp rx=0 tx=0).stop 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 -- 172.21.15.26:0/2552174711 shutdown_connections 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 --2- 172.21.15.26:0/2552174711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a40c1e90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 --2- 172.21.15.26:0/2552174711 >> v2:172.21.15.117:3300/0 conn(0x7f73a40c3a70 0x7f73a40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 -- 172.21.15.26:0/2552174711 >> 172.21.15.26:0/2552174711 conn(0x7f73a40c5560 msgr2=0x7f73a40c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 -- 172.21.15.26:0/2552174711 shutdown_connections 2024-03-06T07:27:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.781+0000 7f73a8b65640 1 -- 172.21.15.26:0/2552174711 wait complete. 2024-03-06T07:27:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.782+0000 7f73a8b65640 1 Processor -- start 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a8b65640 1 -- start start 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a8b65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a406d750 0x7f73a41547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a8b65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a41ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a8b65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40c3a70 0x7f73a41508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a8b65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40a7e50 con 0x7f73a406d750 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a8b65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40a7cd0 con 0x7f73a40c3a70 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a8b65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40a7fd0 con 0x7f73a406db30 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a406d750 0x7f73a41547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a406d750 0x7f73a41547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56670/0 (socket says 172.21.15.26:56670) 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a259c640 1 -- 172.21.15.26:0/578830340 learned_addr learned my addr 172.21.15.26:0/578830340 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.783+0000 7f73a1d9b640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a41ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a2d9d640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40c3a70 0x7f73a41508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a259c640 1 -- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 msgr2=0x7f73a41ebc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a259c640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a41ebc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a259c640 1 -- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40c3a70 msgr2=0x7f73a41508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a259c640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40c3a70 0x7f73a41508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a259c640 1 -- 172.21.15.26:0/578830340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4153d20 con 0x7f73a406d750 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a2d9d640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40c3a70 0x7f73a41508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a1d9b640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a41ebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a259c640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a406d750 0x7f73a41547a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f739400c3f0 tx=0x7f739400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f739affd640 1 -- 172.21.15.26:0/578830340 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394017070 con 0x7f73a406d750 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f739affd640 1 -- 172.21.15.26:0/578830340 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394005ce0 con 0x7f73a406d750 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a414f120 con 0x7f73a406d750 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f739affd640 1 -- 172.21.15.26:0/578830340 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940072f0 con 0x7f73a406d750 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.784+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a4153fa0 con 0x7f73a406d750 2024-03-06T07:27:19.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.785+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a404f330 con 0x7f73a406d750 2024-03-06T07:27:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.786+0000 7f739affd640 1 -- 172.21.15.26:0/578830340 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f739401a070 con 0x7f73a406d750 2024-03-06T07:27:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.786+0000 7f739affd640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f738406b4d0 0x7f738406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.786+0000 7f739affd640 1 -- 172.21.15.26:0/578830340 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7394089680 con 0x7f73a406d750 2024-03-06T07:27:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.786+0000 7f73a1d9b640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f738406b4d0 0x7f738406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.787+0000 7f73a1d9b640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f738406b4d0 0x7f738406d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f73a4154fb0 tx=0x7f738c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.790+0000 7f739affd640 1 -- 172.21.15.26:0/578830340 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73a4153d20 con 0x7f73a406d750 2024-03-06T07:27:19.903 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-03-06T07:27:19.901+0000 7f1f8cca2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.974+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73a4073160 con 0x7f73a406d750 2024-03-06T07:27:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.974+0000 7f739affd640 1 -- 172.21.15.26:0/578830340 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7394058400 con 0x7f73a406d750 2024-03-06T07:27:19.977 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f738406b4d0 msgr2=0x7f738406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f738406b4d0 0x7f738406d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f73a4154fb0 tx=0x7f738c008040 comp rx=0 tx=0).stop 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a406d750 msgr2=0x7f73a41547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a406d750 0x7f73a41547a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f739400c3f0 tx=0x7f739400c8b0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 shutdown_connections 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f738406b4d0 0x7f738406d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40c3a70 0x7f73a41508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a406db30 0x7f73a41ebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 --2- 172.21.15.26:0/578830340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a406d750 0x7f73a41547a0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 >> 172.21.15.26:0/578830340 conn(0x7f73a40c5560 msgr2=0x7f73a406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 shutdown_connections 2024-03-06T07:27:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:19.976+0000 7f73a8b65640 1 -- 172.21.15.26:0/578830340 wait complete. 2024-03-06T07:27:19.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:20.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.115+0000 7f7d6971e640 1 Processor -- start 2024-03-06T07:27:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.115+0000 7f7d6971e640 1 -- start start 2024-03-06T07:27:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d6971e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d64152920 0x7f7d64155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d6971e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d640c8a00 0x7f7d64158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d6971e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d64156180 con 0x7f7d64159460 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d6971e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d641587b0 con 0x7f7d64152920 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d6971e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640c7760 con 0x7f7d640c8a00 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d62ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d64152920 0x7f7d64155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d637fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7d64159460 0x7f7d640c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35450/0 (socket says 172.21.15.26:35450) 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.116+0000 7f7d637fe640 1 -- 172.21.15.26:0/2635203900 learned_addr learned my addr 172.21.15.26:0/2635203900 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d62ffd640 1 -- 172.21.15.26:0/2635203900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d640c8a00 msgr2=0x7f7d64158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d627fc640 1 --2- 172.21.15.26:0/2635203900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d640c8a00 0x7f7d64158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d62ffd640 1 --2- 172.21.15.26:0/2635203900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d640c8a00 0x7f7d64158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d61ffb640 1 -- 172.21.15.26:0/2635203900 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921810032 0 0) 0x7f7d5c001490 con 0x7f7d64159460 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d62ffd640 1 -- 172.21.15.26:0/2635203900 >> v1:172.21.15.26:6789/0 conn(0x7f7d64159460 legacy=0x7f7d640c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d62ffd640 1 -- 172.21.15.26:0/2635203900 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640c7a40 con 0x7f7d64152920 2024-03-06T07:27:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d627fc640 1 --2- 172.21.15.26:0/2635203900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d640c8a00 0x7f7d64158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d62ffd640 1 --2- 172.21.15.26:0/2635203900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d64152920 0x7f7d64155c40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7d580097a0 tx=0x7f7d5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250187ccfaefd184 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.117+0000 7f7d61ffb640 1 -- 172.21.15.26:0/2635203900 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580315a0 con 0x7f7d64152920 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d61ffb640 1 -- 172.21.15.26:0/2635203900 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58037070 con 0x7f7d64152920 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d61ffb640 1 -- 172.21.15.26:0/2635203900 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58031960 con 0x7f7d64152920 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 -- 172.21.15.26:0/2635203900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d64152920 msgr2=0x7f7d64155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 --2- 172.21.15.26:0/2635203900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d64152920 0x7f7d64155c40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7d580097a0 tx=0x7f7d5802b840 comp rx=0 tx=0).stop 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 -- 172.21.15.26:0/2635203900 shutdown_connections 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 --2- 172.21.15.26:0/2635203900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d640c8a00 0x7f7d64158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 --2- 172.21.15.26:0/2635203900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d64152920 0x7f7d64155c40 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 -- 172.21.15.26:0/2635203900 >> 172.21.15.26:0/2635203900 conn(0x7f7d640ac560 msgr2=0x7f7d640ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 -- 172.21.15.26:0/2635203900 shutdown_connections 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.118+0000 7f7d6971e640 1 -- 172.21.15.26:0/2635203900 wait complete. 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.119+0000 7f7d6971e640 1 Processor -- start 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.119+0000 7f7d6971e640 1 -- start start 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d6971e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d640c8a00 0x7f7d641ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d6971e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d64152920 0x7f7d641ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d6971e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d6414e8d0 0x7f7d64147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d6971e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6406d030 con 0x7f7d640c8a00 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d6971e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64158a00 con 0x7f7d64152920 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d6971e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640c5a40 con 0x7f7d6414e8d0 2024-03-06T07:27:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d640c8a00 0x7f7d641ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d640c8a00 0x7f7d641ee3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56684/0 (socket says 172.21.15.26:56684) 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 -- 172.21.15.26:0/3478646205 learned_addr learned my addr 172.21.15.26:0/3478646205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d627fc640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d64152920 0x7f7d641ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d637fe640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d6414e8d0 0x7f7d64147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 -- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d6414e8d0 msgr2=0x7f7d64147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d6414e8d0 0x7f7d64147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 -- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d64152920 msgr2=0x7f7d641ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d64152920 0x7f7d641ee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d62ffd640 1 -- 172.21.15.26:0/3478646205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640ae100 con 0x7f7d640c8a00 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d627fc640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d64152920 0x7f7d641ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.120+0000 7f7d637fe640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d6414e8d0 0x7f7d64147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.121+0000 7f7d62ffd640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d640c8a00 0x7f7d641ee3d0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7d5802be50 tx=0x7f7d580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.121+0000 7f7d4b7fe640 1 -- 172.21.15.26:0/3478646205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58032840 con 0x7f7d640c8a00 2024-03-06T07:27:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.121+0000 7f7d4b7fe640 1 -- 172.21.15.26:0/3478646205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58032a00 con 0x7f7d640c8a00 2024-03-06T07:27:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.121+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d641eee50 con 0x7f7d640c8a00 2024-03-06T07:27:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.121+0000 7f7d4b7fe640 1 -- 172.21.15.26:0/3478646205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58032df0 con 0x7f7d640c8a00 2024-03-06T07:27:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.121+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d640ae3a0 con 0x7f7d640c8a00 2024-03-06T07:27:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.122+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6404f330 con 0x7f7d640c8a00 2024-03-06T07:27:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.122+0000 7f7d4b7fe640 1 -- 172.21.15.26:0/3478646205 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d5804c020 con 0x7f7d640c8a00 2024-03-06T07:27:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.123+0000 7f7d4b7fe640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d5006b500 0x7f7d5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.123+0000 7f7d4b7fe640 1 -- 172.21.15.26:0/3478646205 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d5802f020 con 0x7f7d640c8a00 2024-03-06T07:27:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.123+0000 7f7d627fc640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d5006b500 0x7f7d5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.123+0000 7f7d627fc640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d5006b500 0x7f7d5006d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.131+0000 7f7d4b7fe640 1 -- 172.21.15.26:0/3478646205 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d58040070 con 0x7f7d640c8a00 2024-03-06T07:27:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.352+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d6406cda0 con 0x7f7d640c8a00 2024-03-06T07:27:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.352+0000 7f7d4b7fe640 1 -- 172.21.15.26:0/3478646205 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f7d5807ae50 con 0x7f7d640c8a00 2024-03-06T07:27:20.355 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:20.355 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:15.648611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9627,"qdb_cluster":[9627]},"id":13}]} 2024-03-06T07:27:20.355 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 166 2024-03-06T07:27:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d5006b500 msgr2=0x7f7d5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d5006b500 0x7f7d5006d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54008040 comp rx=0 tx=0).stop 2024-03-06T07:27:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d640c8a00 msgr2=0x7f7d641ee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d640c8a00 0x7f7d641ee3d0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7d5802be50 tx=0x7f7d580046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 shutdown_connections 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7d5006b500 0x7f7d5006d9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d6414e8d0 0x7f7d64147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d64152920 0x7f7d641ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 --2- 172.21.15.26:0/3478646205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d640c8a00 0x7f7d641ee3d0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.354+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 >> 172.21.15.26:0/3478646205 conn(0x7f7d640ac560 msgr2=0x7f7d640ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.355+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 shutdown_connections 2024-03-06T07:27:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.355+0000 7f7d6971e640 1 -- 172.21.15.26:0/3478646205 wait complete. 2024-03-06T07:27:20.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.470+0000 7fbb01481640 1 Processor -- start 2024-03-06T07:27:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.470+0000 7fbb01481640 1 -- start start 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.471+0000 7fbb01481640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0c7180 0x7fbafc159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.471+0000 7fbb01481640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbafc0c88b0 0x7fbafc0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.471+0000 7fbb01481640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc0c7c60 con 0x7fbafc1527b0 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.471+0000 7fbb01481640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc159640 con 0x7fbafc0c7180 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.471+0000 7fbb01481640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc0c8060 con 0x7fbafc0c88b0 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafa7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0c7180 0x7fbafc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafa7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0c7180 0x7fbafc159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56694/0 (socket says 172.21.15.26:56694) 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafa7fc640 1 -- 172.21.15.26:0/1487310365 learned_addr learned my addr 172.21.15.26:0/1487310365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/1487310365 >> v2:172.21.15.117:3300/0 conn(0x7fbafc0c88b0 0x7fbafc0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafa7fc640 1 -- 172.21.15.26:0/1487310365 >> v2:172.21.15.117:3300/0 conn(0x7fbafc0c88b0 msgr2=0x7fbafc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafa7fc640 1 --2- 172.21.15.26:0/1487310365 >> v2:172.21.15.117:3300/0 conn(0x7fbafc0c88b0 0x7fbafc0c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafa7fc640 1 -- 172.21.15.26:0/1487310365 >> v1:172.21.15.117:6789/0 conn(0x7fbafc1527b0 legacy=0x7fbafc155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.472+0000 7fbafa7fc640 1 -- 172.21.15.26:0/1487310365 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc0c76a0 con 0x7fbafc0c7180 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.473+0000 7fbafa7fc640 1 --2- 172.21.15.26:0/1487310365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc159100 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0ad600 tx=0x7fbaf002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8589999406009d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.473+0000 7fbaf9ffb640 1 -- 172.21.15.26:0/1487310365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0035020 con 0x7fbafc0c7180 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.473+0000 7fbaf9ffb640 1 -- 172.21.15.26:0/1487310365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf0031870 con 0x7fbafc0c7180 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.473+0000 7fbaf9ffb640 1 -- 172.21.15.26:0/1487310365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0031c40 con 0x7fbafc0c7180 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 -- 172.21.15.26:0/1487310365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 msgr2=0x7fbafc159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 --2- 172.21.15.26:0/1487310365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc159100 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0ad600 tx=0x7fbaf002b910 comp rx=0 tx=0).stop 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 -- 172.21.15.26:0/1487310365 shutdown_connections 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 --2- 172.21.15.26:0/1487310365 >> v2:172.21.15.117:3300/0 conn(0x7fbafc0c88b0 0x7fbafc0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 --2- 172.21.15.26:0/1487310365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc159100 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 -- 172.21.15.26:0/1487310365 >> 172.21.15.26:0/1487310365 conn(0x7fbafc0ac560 msgr2=0x7fbafc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 -- 172.21.15.26:0/1487310365 shutdown_connections 2024-03-06T07:27:20.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.474+0000 7fbb01481640 1 -- 172.21.15.26:0/1487310365 wait complete. 2024-03-06T07:27:20.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.476+0000 7fbb01481640 1 Processor -- start 2024-03-06T07:27:20.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.476+0000 7fbb01481640 1 -- start start 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbb01481640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc1ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbb01481640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbafc0c88b0 0x7fbafc0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbb01481640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbafc1527b0 0x7fbafc1ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbb01481640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc155600 con 0x7fbafc0c7180 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbb01481640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc155320 con 0x7fbafc1527b0 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbb01481640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc152b90 con 0x7fbafc0c88b0 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc1ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc1ebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56704/0 (socket says 172.21.15.26:56704) 2024-03-06T07:27:20.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 -- 172.21.15.26:0/2134135432 learned_addr learned my addr 172.21.15.26:0/2134135432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbafc1527b0 0x7fbafc1ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafa7fc640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbafc0c88b0 0x7fbafc0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 -- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbafc0c88b0 msgr2=0x7fbafc0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbafc0c88b0 0x7fbafc0ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 -- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbafc1527b0 msgr2=0x7fbafc1ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbafc1527b0 0x7fbafc1ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafaffd640 1 -- 172.21.15.26:0/2134135432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc1ef1f0 con 0x7fbafc0c7180 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbafc1527b0 0x7fbafc1ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.477+0000 7fbafa7fc640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbafc0c88b0 0x7fbafc0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.478+0000 7fbafaffd640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc1ebc90 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c3c0 tx=0x7fbaec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.478+0000 7fbae37fe640 1 -- 172.21.15.26:0/2134135432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0052d0 con 0x7fbafc0c7180 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.478+0000 7fbae37fe640 1 -- 172.21.15.26:0/2134135432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec005490 con 0x7fbafc0c7180 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.478+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc1579e0 con 0x7fbafc0c7180 2024-03-06T07:27:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.478+0000 7fbae37fe640 1 -- 172.21.15.26:0/2134135432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec007260 con 0x7fbafc0c7180 2024-03-06T07:27:20.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.478+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc1eb490 con 0x7fbafc0c7180 2024-03-06T07:27:20.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.479+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc04f330 con 0x7fbafc0c7180 2024-03-06T07:27:20.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.479+0000 7fbae37fe640 1 -- 172.21.15.26:0/2134135432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaec007400 con 0x7fbafc0c7180 2024-03-06T07:27:20.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.479+0000 7fbae37fe640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbae806b4d0 0x7fbae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.480+0000 7fbae37fe640 1 -- 172.21.15.26:0/2134135432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbaec0897e0 con 0x7fbafc0c7180 2024-03-06T07:27:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.480+0000 7fbafa7fc640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbae806b4d0 0x7fbae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.480+0000 7fbafa7fc640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbae806b4d0 0x7fbae806d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0036000 tx=0x7fbaf003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.484+0000 7fbae37fe640 1 -- 172.21.15.26:0/2134135432 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbafc0c7c60 con 0x7fbafc0c7180 2024-03-06T07:27:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.666+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbafc0ace60 con 0x7fbafc0c7180 2024-03-06T07:27:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.667+0000 7fbae37fe640 1 -- 172.21.15.26:0/2134135432 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16964 (secure 0 0 0) 0x7fbaec058560 con 0x7fbafc0c7180 2024-03-06T07:27:20.670 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:20.670 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":172,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:53.824659+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:50.446132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:51.147227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbae806b4d0 msgr2=0x7fbae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbae806b4d0 0x7fbae806d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0036000 tx=0x7fbaf003c040 comp rx=0 tx=0).stop 2024-03-06T07:27:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 msgr2=0x7fbafc1ebc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc1ebc90 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c3c0 tx=0x7fbaec00c880 comp rx=0 tx=0).stop 2024-03-06T07:27:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 shutdown_connections 2024-03-06T07:27:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbae806b4d0 0x7fbae806d990 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbafc1527b0 0x7fbafc1ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbafc0c88b0 0x7fbafc0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 --2- 172.21.15.26:0/2134135432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0c7180 0x7fbafc1ebc90 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 >> 172.21.15.26:0/2134135432 conn(0x7fbafc0ac560 msgr2=0x7fbafc151180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 shutdown_connections 2024-03-06T07:27:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.670+0000 7fbb01481640 1 -- 172.21.15.26:0/2134135432 wait complete. 2024-03-06T07:27:20.684 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:27:20.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:20.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.782+0000 7f4b4552c640 1 Processor -- start 2024-03-06T07:27:20.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.782+0000 7f4b4552c640 1 -- start start 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b4552c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b40155ba0 0x7f4b400c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b4552c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b401586f0 0x7f4b40158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b4552c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b401591c0 con 0x7f4b400c5480 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b4552c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4006df40 con 0x7f4b40155ba0 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b4552c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4006fdb0 con 0x7f4b401586f0 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b40155ba0 0x7f4b400c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b40155ba0 0x7f4b400c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56708/0 (socket says 172.21.15.26:56708) 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2768163649 learned_addr learned my addr 172.21.15.26:0/2768163649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3f7fe640 1 --2- 172.21.15.26:0/2768163649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b401586f0 0x7f4b40158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2768163649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b401586f0 msgr2=0x7f4b40158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3e7fc640 1 --2- 172.21.15.26:0/2768163649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b401586f0 0x7f4b40158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2768163649 >> v1:172.21.15.117:6789/0 conn(0x7f4b400c5480 legacy=0x7f4b4006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.784+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2768163649 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b401594a0 con 0x7f4b40155ba0 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.785+0000 7f4b3e7fc640 1 --2- 172.21.15.26:0/2768163649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b400c5860 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009870 tx=0x7f4b3402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cf3d859a64da2d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.785+0000 7f4b3dffb640 1 -- 172.21.15.26:0/2768163649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34035020 con 0x7f4b40155ba0 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.785+0000 7f4b3dffb640 1 -- 172.21.15.26:0/2768163649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b34031640 con 0x7f4b40155ba0 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.785+0000 7f4b3dffb640 1 -- 172.21.15.26:0/2768163649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3402d040 con 0x7f4b40155ba0 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.785+0000 7f4b4552c640 1 -- 172.21.15.26:0/2768163649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 msgr2=0x7f4b400c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.785+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2768163649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b400c5860 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009870 tx=0x7f4b3402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.786+0000 7f4b4552c640 1 -- 172.21.15.26:0/2768163649 shutdown_connections 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.786+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2768163649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b401586f0 0x7f4b40158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.786+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2768163649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b400c5860 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.786+0000 7f4b4552c640 1 -- 172.21.15.26:0/2768163649 >> 172.21.15.26:0/2768163649 conn(0x7f4b4006b1a0 msgr2=0x7f4b4006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.786+0000 7f4b4552c640 1 -- 172.21.15.26:0/2768163649 shutdown_connections 2024-03-06T07:27:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.786+0000 7f4b4552c640 1 -- 172.21.15.26:0/2768163649 wait complete. 2024-03-06T07:27:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.787+0000 7f4b4552c640 1 Processor -- start 2024-03-06T07:27:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.787+0000 7f4b4552c640 1 -- start start 2024-03-06T07:27:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.787+0000 7f4b4552c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b400c5480 0x7f4b40156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.787+0000 7f4b4552c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b40148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b4552c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b401586f0 0x7f4b40149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b4552c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40155680 con 0x7f4b40155ba0 2024-03-06T07:27:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b4552c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40156000 con 0x7f4b401586f0 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b4552c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40152a20 con 0x7f4b400c5480 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b40148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b40148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56720/0 (socket says 172.21.15.26:56720) 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2819107788 learned_addr learned my addr 172.21.15.26:0/2819107788 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3f7fe640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b401586f0 0x7f4b40149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3effd640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b400c5480 0x7f4b40156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b400c5480 msgr2=0x7f4b40156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b400c5480 0x7f4b40156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b401586f0 msgr2=0x7f4b40149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b401586f0 0x7f4b40149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 -- 172.21.15.26:0/2819107788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4014e2e0 con 0x7f4b40155ba0 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b3e7fc640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b40148af0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402b930 tx=0x7f4b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b277fe640 1 -- 172.21.15.26:0/2819107788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34040070 con 0x7f4b40155ba0 2024-03-06T07:27:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b277fe640 1 -- 172.21.15.26:0/2819107788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3403e070 con 0x7f4b40155ba0 2024-03-06T07:27:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4014e5c0 con 0x7f4b40155ba0 2024-03-06T07:27:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b277fe640 1 -- 172.21.15.26:0/2819107788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34035040 con 0x7f4b40155ba0 2024-03-06T07:27:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.788+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4006cbf0 con 0x7f4b40155ba0 2024-03-06T07:27:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.790+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4004f330 con 0x7f4b40155ba0 2024-03-06T07:27:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.790+0000 7f4b277fe640 1 -- 172.21.15.26:0/2819107788 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b34038020 con 0x7f4b40155ba0 2024-03-06T07:27:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.790+0000 7f4b277fe640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.791+0000 7f4b277fe640 1 -- 172.21.15.26:0/2819107788 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b340abdd0 con 0x7f4b40155ba0 2024-03-06T07:27:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.791+0000 7f4b3effd640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.791+0000 7f4b3effd640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b38007fd0 tx=0x7f4b38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:20.795+0000 7f4b277fe640 1 -- 172.21.15.26:0/2819107788 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b3407ab50 con 0x7f4b40155ba0 2024-03-06T07:27:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.013+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b4014e790 con 0x7f4b40155ba0 2024-03-06T07:27:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.013+0000 7f4b277fe640 1 -- 172.21.15.26:0/2819107788 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f4b3407a450 con 0x7f4b40155ba0 2024-03-06T07:27:21.016 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:21.016 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:15.648611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9627,"qdb_cluster":[9627]},"id":13}]} 2024-03-06T07:27:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 166 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b2c06b4d0 msgr2=0x7f4b2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b38007fd0 tx=0x7f4b38009040 comp rx=0 tx=0).stop 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 msgr2=0x7f4b40148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b40148af0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402b930 tx=0x7f4b340046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 shutdown_connections 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b401586f0 0x7f4b40149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40155ba0 0x7f4b40148af0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 --2- 172.21.15.26:0/2819107788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b400c5480 0x7f4b40156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 >> 172.21.15.26:0/2819107788 conn(0x7f4b4006b1a0 msgr2=0x7f4b400707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 shutdown_connections 2024-03-06T07:27:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.015+0000 7f4b4552c640 1 -- 172.21.15.26:0/2819107788 wait complete. 2024-03-06T07:27:21.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.127+0000 7fcab7e6f640 1 Processor -- start 2024-03-06T07:27:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.127+0000 7fcab7e6f640 1 -- start start 2024-03-06T07:27:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab7e6f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcab01527b0 0x7fcab0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab7e6f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcab00c88b0 0x7fcab00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab7e6f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab00c7c60 con 0x7fcab00c7180 2024-03-06T07:27:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab7e6f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab0159640 con 0x7fcab00c88b0 2024-03-06T07:27:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab7e6f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab00c8060 con 0x7fcab01527b0 2024-03-06T07:27:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab540a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcab00c7180 0x7fcab0155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35460/0 (socket says 172.21.15.26:35460) 2024-03-06T07:27:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab640c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcab00c88b0 0x7fcab00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab540a640 1 -- 172.21.15.26:0/3914547147 learned_addr learned my addr 172.21.15.26:0/3914547147 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab640c640 1 -- 172.21.15.26:0/3914547147 >> v2:172.21.15.117:3300/0 conn(0x7fcab01527b0 msgr2=0x7fcab0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab640c640 1 --2- 172.21.15.26:0/3914547147 >> v2:172.21.15.117:3300/0 conn(0x7fcab01527b0 0x7fcab0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab640c640 1 -- 172.21.15.26:0/3914547147 >> v1:172.21.15.26:6789/0 conn(0x7fcab00c7180 legacy=0x7fcab0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.129+0000 7fcab640c640 1 -- 172.21.15.26:0/3914547147 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00c76a0 con 0x7fcab00c88b0 2024-03-06T07:27:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.128+0000 7fcab4c09640 1 -- 172.21.15.26:0/3914547147 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197840898 0 0) 0x7fcaa0001110 con 0x7fcab00c7180 2024-03-06T07:27:21.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.129+0000 7fcab640c640 1 --2- 172.21.15.26:0/3914547147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab00c5330 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cba80 tx=0x7fcaa802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66da6b07801dc8c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.129+0000 7fcab4c09640 1 -- 172.21.15.26:0/3914547147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8039070 con 0x7fcab00c88b0 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.129+0000 7fcab4c09640 1 -- 172.21.15.26:0/3914547147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa802cbf0 con 0x7fcab00c88b0 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.129+0000 7fcab4c09640 1 -- 172.21.15.26:0/3914547147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8034440 con 0x7fcab00c88b0 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 -- 172.21.15.26:0/3914547147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 msgr2=0x7fcab00c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/3914547147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab00c5330 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cba80 tx=0x7fcaa802be40 comp rx=0 tx=0).stop 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 -- 172.21.15.26:0/3914547147 shutdown_connections 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/3914547147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab00c5330 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/3914547147 >> v2:172.21.15.117:3300/0 conn(0x7fcab01527b0 0x7fcab0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 -- 172.21.15.26:0/3914547147 >> 172.21.15.26:0/3914547147 conn(0x7fcab00ac560 msgr2=0x7fcab00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 -- 172.21.15.26:0/3914547147 shutdown_connections 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.130+0000 7fcab7e6f640 1 -- 172.21.15.26:0/3914547147 wait complete. 2024-03-06T07:27:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.131+0000 7fcab7e6f640 1 Processor -- start 2024-03-06T07:27:21.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.131+0000 7fcab7e6f640 1 -- start start 2024-03-06T07:27:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab7e6f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcab00c7180 0x7fcab00c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab7e6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab014ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab7e6f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcab01527b0 0x7fcab014b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab7e6f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0155600 con 0x7fcab00c88b0 2024-03-06T07:27:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab7e6f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0155320 con 0x7fcab00c7180 2024-03-06T07:27:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab7e6f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0152b90 con 0x7fcab01527b0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab014ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab014ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56732/0 (socket says 172.21.15.26:56732) 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 -- 172.21.15.26:0/4224843027 learned_addr learned my addr 172.21.15.26:0/4224843027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab640c640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcab01527b0 0x7fcab014b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab5c0b640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcab00c7180 0x7fcab00c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 -- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcab01527b0 msgr2=0x7fcab014b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcab01527b0 0x7fcab014b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 -- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcab00c7180 msgr2=0x7fcab00c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcab00c7180 0x7fcab00c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.132+0000 7fcab540a640 1 -- 172.21.15.26:0/4224843027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00c0bc0 con 0x7fcab00c88b0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.133+0000 7fcab540a640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab014ca50 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000c780 tx=0x7fcaa000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.133+0000 7fcaa67fc640 1 -- 172.21.15.26:0/4224843027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0018070 con 0x7fcab00c88b0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.133+0000 7fcaa67fc640 1 -- 172.21.15.26:0/4224843027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa00072d0 con 0x7fcab00c88b0 2024-03-06T07:27:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.133+0000 7fcaa67fc640 1 -- 172.21.15.26:0/4224843027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00047e0 con 0x7fcab00c88b0 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.133+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab01eba00 con 0x7fcab00c88b0 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.133+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab00c1140 con 0x7fcab00c88b0 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.134+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab004f330 con 0x7fcab00c88b0 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.134+0000 7fcaa67fc640 1 -- 172.21.15.26:0/4224843027 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcaa001b070 con 0x7fcab00c88b0 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.134+0000 7fcaa67fc640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fca9406b4d0 0x7fca9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.134+0000 7fcaa67fc640 1 -- 172.21.15.26:0/4224843027 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcaa0089560 con 0x7fcab00c88b0 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.135+0000 7fcab5c0b640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fca9406b4d0 0x7fca9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.135+0000 7fcab5c0b640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fca9406b4d0 0x7fca9406d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0098e0 tx=0x7fcaac009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.139+0000 7fcaa67fc640 1 -- 172.21.15.26:0/4224843027 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcab00c7c60 con 0x7fcab00c88b0 2024-03-06T07:27:21.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.393+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcab0152d70 con 0x7fcab00c88b0 2024-03-06T07:27:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.393+0000 7fcaa67fc640 1 -- 172.21.15.26:0/4224843027 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7fcaa0057c90 con 0x7fcab00c88b0 2024-03-06T07:27:21.396 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:21.396 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:52.179125+0000","modified":"2024-03-06T07:27:15.648611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9627":{"gid":9627,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.26:6835/1519487439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1519487439},{"type":"v1","addr":"172.21.15.26:6835","nonce":1519487439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9627,"qdb_cluster":[9627]},"id":13}]} 2024-03-06T07:27:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 166 2024-03-06T07:27:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fca9406b4d0 msgr2=0x7fca9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fca9406b4d0 0x7fca9406d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0098e0 tx=0x7fcaac009060 comp rx=0 tx=0).stop 2024-03-06T07:27:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 msgr2=0x7fcab014ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab014ca50 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000c780 tx=0x7fcaa000cc40 comp rx=0 tx=0).stop 2024-03-06T07:27:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 shutdown_connections 2024-03-06T07:27:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fca9406b4d0 0x7fca9406d990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcab01527b0 0x7fcab014b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab00c88b0 0x7fcab014ca50 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 --2- 172.21.15.26:0/4224843027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcab00c7180 0x7fcab00c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 >> 172.21.15.26:0/4224843027 conn(0x7fcab00ac560 msgr2=0x7fcab00ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 shutdown_connections 2024-03-06T07:27:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.395+0000 7fcab7e6f640 1 -- 172.21.15.26:0/4224843027 wait complete. 2024-03-06T07:27:21.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.508+0000 7fb874d56640 1 Processor -- start 2024-03-06T07:27:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.508+0000 7fb874d56640 1 -- start start 2024-03-06T07:27:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb874d56640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb87014f360 0x7fb870155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb874d56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c88c0 0x7fb8700c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb874d56640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8700c73d0 con 0x7fb8700c5880 2024-03-06T07:27:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb874d56640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8700c8dd0 con 0x7fb87014f360 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb874d56640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700be2e0 con 0x7fb8700c88c0 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb86e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb87014f360 0x7fb870155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb86ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8700c5880 0x7fb870158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35474/0 (socket says 172.21.15.26:35474) 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb86ed9d640 1 -- 172.21.15.26:0/3521199338 learned_addr learned my addr 172.21.15.26:0/3521199338 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.509+0000 7fb867fff640 1 --2- 172.21.15.26:0/3521199338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c88c0 0x7fb8700c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86e59c640 1 -- 172.21.15.26:0/3521199338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c88c0 msgr2=0x7fb8700c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86e59c640 1 --2- 172.21.15.26:0/3521199338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c88c0 0x7fb8700c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86e59c640 1 -- 172.21.15.26:0/3521199338 >> v1:172.21.15.26:6789/0 conn(0x7fb8700c5880 legacy=0x7fb870158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86dd9b640 1 -- 172.21.15.26:0/3521199338 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261792270 0 0) 0x7fb8700c73d0 con 0x7fb8700c5880 2024-03-06T07:27:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86e59c640 1 -- 172.21.15.26:0/3521199338 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700c7670 con 0x7fb87014f360 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86e59c640 1 --2- 172.21.15.26:0/3521199338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87014f360 0x7fb870155c10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fb8600097a0 tx=0x7fb86002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff00f8273559bd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86dd9b640 1 -- 172.21.15.26:0/3521199338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8600315a0 con 0x7fb87014f360 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86dd9b640 1 -- 172.21.15.26:0/3521199338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb860037070 con 0x7fb87014f360 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.510+0000 7fb86dd9b640 1 -- 172.21.15.26:0/3521199338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860031960 con 0x7fb87014f360 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 -- 172.21.15.26:0/3521199338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87014f360 msgr2=0x7fb870155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 --2- 172.21.15.26:0/3521199338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87014f360 0x7fb870155c10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fb8600097a0 tx=0x7fb86002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 -- 172.21.15.26:0/3521199338 shutdown_connections 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 --2- 172.21.15.26:0/3521199338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c88c0 0x7fb8700c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 --2- 172.21.15.26:0/3521199338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87014f360 0x7fb870155c10 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 -- 172.21.15.26:0/3521199338 >> 172.21.15.26:0/3521199338 conn(0x7fb8700ac570 msgr2=0x7fb8700ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 -- 172.21.15.26:0/3521199338 shutdown_connections 2024-03-06T07:27:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.511+0000 7fb874d56640 1 -- 172.21.15.26:0/3521199338 wait complete. 2024-03-06T07:27:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.512+0000 7fb874d56640 1 Processor -- start 2024-03-06T07:27:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.512+0000 7fb874d56640 1 -- start start 2024-03-06T07:27:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb874d56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c5880 0x7fb8701ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb874d56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8700c88c0 0x7fb8701546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb874d56640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87014f360 0x7fb8700af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb874d56640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb870159270 con 0x7fb8700c88c0 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb874d56640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700c7c30 con 0x7fb87014f360 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb874d56640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701593f0 con 0x7fb8700c5880 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8700c88c0 0x7fb8701546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8700c88c0 0x7fb8701546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56752/0 (socket says 172.21.15.26:56752) 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 -- 172.21.15.26:0/1699125704 learned_addr learned my addr 172.21.15.26:0/1699125704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb86e59c640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c5880 0x7fb8701ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb86ed9d640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87014f360 0x7fb8700af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 -- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c5880 msgr2=0x7fb8701ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c5880 0x7fb8701ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 -- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87014f360 msgr2=0x7fb8700af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87014f360 0x7fb8700af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb867fff640 1 -- 172.21.15.26:0/1699125704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700c03b0 con 0x7fb8700c88c0 2024-03-06T07:27:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.513+0000 7fb86e59c640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c5880 0x7fb8701ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.514+0000 7fb867fff640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8700c88c0 0x7fb8701546b0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb85800d7d0 tx=0x7fb85800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.514+0000 7fb866ffd640 1 -- 172.21.15.26:0/1699125704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800b6c0 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.514+0000 7fb866ffd640 1 -- 172.21.15.26:0/1699125704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb858019070 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.514+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8700c11e0 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.514+0000 7fb866ffd640 1 -- 172.21.15.26:0/1699125704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800bac0 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.514+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8700c0670 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.515+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87004f330 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.516+0000 7fb866ffd640 1 -- 172.21.15.26:0/1699125704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb85800bc60 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.516+0000 7fb866ffd640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb85006b4d0 0x7fb85006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.516+0000 7fb866ffd640 1 -- 172.21.15.26:0/1699125704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb85808a850 con 0x7fb8700c88c0 2024-03-06T07:27:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.516+0000 7fb86e59c640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb85006b4d0 0x7fb85006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.517+0000 7fb86e59c640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb85006b4d0 0x7fb85006d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb860036000 tx=0x7fb860008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.524+0000 7fb866ffd640 1 -- 172.21.15.26:0/1699125704 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb858059540 con 0x7fb8700c88c0 2024-03-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.707+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb87006cda0 con 0x7fb8700c88c0 2024-03-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.708+0000 7fb866ffd640 1 -- 172.21.15.26:0/1699125704 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16964 (secure 0 0 0) 0x7fb858058e40 con 0x7fb8700c88c0 2024-03-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":172,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:26:53.824659+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:50.446132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:51.147227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.710+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb85006b4d0 msgr2=0x7fb85006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.710+0000 7fb874d56640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb85006b4d0 0x7fb85006d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb860036000 tx=0x7fb860008030 comp rx=0 tx=0).stop 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.710+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8700c88c0 msgr2=0x7fb8701546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.710+0000 7fb874d56640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8700c88c0 0x7fb8701546b0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb85800d7d0 tx=0x7fb85800dc90 comp rx=0 tx=0).stop 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 shutdown_connections 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb85006b4d0 0x7fb85006d990 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87014f360 0x7fb8700af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8700c88c0 0x7fb8701546b0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 --2- 172.21.15.26:0/1699125704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8700c5880 0x7fb8701ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 >> 172.21.15.26:0/1699125704 conn(0x7fb8700ac570 msgr2=0x7fb87014d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 shutdown_connections 2024-03-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.711+0000 7fb874d56640 1 -- 172.21.15.26:0/1699125704 wait complete. 2024-03-06T07:27:21.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:27:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.828+0000 7f5141da0640 1 Processor -- start 2024-03-06T07:27:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.828+0000 7f5141da0640 1 -- start start 2024-03-06T07:27:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f5141da0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c155950 0x7f513c0c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f5141da0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c06f680 con 0x7f513c152520 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f5141da0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c06f250 con 0x7f513c0c90e0 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f5141da0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c158900 con 0x7f513c155950 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f5140d9e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f513c152520 0x7f513c158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35480/0 (socket says 172.21.15.26:35480) 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f5140d9e640 1 -- 172.21.15.26:0/3588779557 learned_addr learned my addr 172.21.15.26:0/3588779557 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f514159f640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c155950 0x7f513c0c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167457370 0 0) 0x7f513c06f680 con 0x7f513c152520 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.829+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f512c003520 con 0x7f513c152520 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849391976 0 0) 0x7f513c06f250 con 0x7f513c0c90e0 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f513c06f680 con 0x7f513c0c90e0 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970646342 0 0) 0x7f512c003520 con 0x7f513c152520 2024-03-06T07:27:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c155950 msgr2=0x7f513c0c1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c155950 0x7f513c0c1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 >> v1:172.21.15.117:6789/0 conn(0x7f513c0c90e0 legacy=0x7f513c0aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c06f920 con 0x7f513c152520 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f5130001690 con 0x7f513c152520 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 >> v1:172.21.15.26:6789/0 conn(0x7f513c152520 legacy=0x7f513c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f512c003b70 0x7f512c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f512c004800 0x7f512c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c0051c0 0x7f512c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c158900 con 0x7f512c004800 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c00c090 con 0x7f512c0051c0 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c00c290 con 0x7f512c003b70 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f514159f640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c155950 0x7f513c0c1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513bfff640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f512c004800 0x7f512c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f5140d9e640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f512c003b70 0x7f512c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f514159f640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c0051c0 0x7f512c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513bfff640 1 -- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f512c003b70 msgr2=0x7f512c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513bfff640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f512c003b70 0x7f512c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513bfff640 1 -- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c0051c0 msgr2=0x7f512c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513bfff640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c0051c0 0x7f512c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f513bfff640 1 -- 172.21.15.26:0/3588779557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c003520 con 0x7f512c004800 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f5140d9e640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f512c003b70 0x7f512c003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.830+0000 7f514159f640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c0051c0 0x7f512c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.831+0000 7f513bfff640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f512c004800 0x7f512c004c80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f51280018c0 tx=0x7f512c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf652596157bb285 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128037070 con 0x7f512c004800 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51280052f0 con 0x7f512c004800 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f513b7fe640 1 -- 172.21.15.26:0/3588779557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280014e0 con 0x7f512c004800 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f5141da0640 1 -- 172.21.15.26:0/3588779557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f512c004800 msgr2=0x7f512c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f5141da0640 1 --2- 172.21.15.26:0/3588779557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f512c004800 0x7f512c004c80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f51280018c0 tx=0x7f512c005830 comp rx=0 tx=0).stop 2024-03-06T07:27:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f514159f640 1 -- 172.21.15.26:0/3588779557 reap_dead start 2024-03-06T07:27:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f5141da0640 1 -- 172.21.15.26:0/3588779557 shutdown_connections 2024-03-06T07:27:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f5141da0640 1 -- 172.21.15.26:0/3588779557 >> 172.21.15.26:0/3588779557 conn(0x7f513c067300 msgr2=0x7f513c0a9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f5141da0640 1 -- 172.21.15.26:0/3588779557 shutdown_connections 2024-03-06T07:27:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.832+0000 7f5141da0640 1 -- 172.21.15.26:0/3588779557 wait complete. 2024-03-06T07:27:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.834+0000 7f5141da0640 1 Processor -- start 2024-03-06T07:27:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.834+0000 7f5141da0640 1 -- start start 2024-03-06T07:27:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f5141da0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c004800 0x7f513c149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f5141da0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c154a80 0x7f513c14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f5141da0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c1eb2f0 0x7f513c1e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f5141da0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c06fc00 con 0x7f513c154a80 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f5141da0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c152900 con 0x7f512c004800 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f5141da0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0aaaf0 con 0x7f513c1eb2f0 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f513bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c154a80 0x7f513c14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f513bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c154a80 0x7f513c14b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56772/0 (socket says 172.21.15.26:56772) 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f513bfff640 1 -- 172.21.15.26:0/353682740 learned_addr learned my addr 172.21.15.26:0/353682740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f5140d9e640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c004800 0x7f513c149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.835+0000 7f514159f640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c1eb2f0 0x7f513c1e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f513bfff640 1 -- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c1eb2f0 msgr2=0x7f513c1e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f513bfff640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c1eb2f0 0x7f513c1e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f513bfff640 1 -- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c004800 msgr2=0x7f513c149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f513bfff640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c004800 0x7f513c149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f513bfff640 1 -- 172.21.15.26:0/353682740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c14bea0 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f513bfff640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c154a80 0x7f513c14b280 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5128005040 tx=0x7f51280012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f51397fa640 1 -- 172.21.15.26:0/353682740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128042020 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f51397fa640 1 -- 172.21.15.26:0/353682740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128038040 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c153ee0 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f51397fa640 1 -- 172.21.15.26:0/353682740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128037070 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.836+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c158030 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.837+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c0693c0 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.837+0000 7f51397fa640 1 -- 172.21.15.26:0/353682740 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5128045020 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.838+0000 7f51397fa640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512006b500 0x7f512006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.838+0000 7f51397fa640 1 -- 172.21.15.26:0/353682740 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51280abea0 con 0x7f513c154a80 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.838+0000 7f5140d9e640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512006b500 0x7f512006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.839+0000 7f5140d9e640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512006b500 0x7f512006d9c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5130001220 tx=0x7f5130009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:21.842+0000 7f51397fa640 1 -- 172.21.15.26:0/353682740 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f512804c030 con 0x7f513c154a80 2024-03-06T07:27:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.063+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f513c068460 con 0x7f513c154a80 2024-03-06T07:27:22.068 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:27:22.064+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:27:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.065+0000 7f51397fa640 1 -- 172.21.15.26:0/353682740 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v167) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5128006e20 con 0x7f513c154a80 2024-03-06T07:27:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:27:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.067+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512006b500 msgr2=0x7f512006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.067+0000 7f5141da0640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512006b500 0x7f512006d9c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5130001220 tx=0x7f5130009040 comp rx=0 tx=0).stop 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c154a80 msgr2=0x7f513c14b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c154a80 0x7f513c14b280 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5128005040 tx=0x7f51280012d0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 shutdown_connections 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512006b500 0x7f512006d9c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f513c1eb2f0 0x7f513c1e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c154a80 0x7f513c14b280 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 --2- 172.21.15.26:0/353682740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c004800 0x7f513c149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 >> 172.21.15.26:0/353682740 conn(0x7f513c067300 msgr2=0x7f513c0a9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 shutdown_connections 2024-03-06T07:27:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.068+0000 7f5141da0640 1 -- 172.21.15.26:0/353682740 wait complete. 2024-03-06T07:27:22.072 INFO:tasks.ceph.mds.a.smithi026.stderr: -69> 2024-03-06T07:27:19.901+0000 7f1f8cca2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:22.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:27:22.099 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:27:22.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.188+0000 7ff9838d4640 1 Processor -- start 2024-03-06T07:27:22.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.188+0000 7ff9838d4640 1 -- start start 2024-03-06T07:27:22.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.189+0000 7ff9838d4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff97c0c69a0 0x7ff97c073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.189+0000 7ff9838d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 0x7ff97c0c2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.189+0000 7ff9838d4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff97c159580 con 0x7ff97c06d840 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.189+0000 7ff9838d4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff97c0c3460 con 0x7ff97c0c69a0 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.189+0000 7ff9838d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0c2050 con 0x7ff97c0c24e0 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.189+0000 7ff981670640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff97c06d840 0x7ff97c158690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35494/0 (socket says 172.21.15.26:35494) 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.189+0000 7ff981670640 1 -- 172.21.15.26:0/18713138 learned_addr learned my addr 172.21.15.26:0/18713138 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff980e6f640 1 --2- 172.21.15.26:0/18713138 >> v2:172.21.15.117:3300/0 conn(0x7ff97c0c69a0 0x7ff97c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981e71640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 0x7ff97c0c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921088427 0 0) 0x7ff97c159580 con 0x7ff97c06d840 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff968003520 con 0x7ff97c06d840 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143409239 0 0) 0x7ff968003520 con 0x7ff97c06d840 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 msgr2=0x7ff97c0c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 0x7ff97c0c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 >> v2:172.21.15.117:3300/0 conn(0x7ff97c0c69a0 msgr2=0x7ff97c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 --2- 172.21.15.26:0/18713138 >> v2:172.21.15.117:3300/0 conn(0x7ff97c0c69a0 0x7ff97c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c0c36b0 con 0x7ff97c06d840 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7ff970001670 con 0x7ff97c06d840 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 >> v1:172.21.15.26:6789/0 conn(0x7ff97c06d840 legacy=0x7ff97c158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9680039b0 0x7ff968003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 0x7ff968004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff968004d40 0x7ff9680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0c3460 con 0x7ff9680039b0 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0c2050 con 0x7ff968004d40 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96800ba40 con 0x7ff97c0c24e0 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981670640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9680039b0 0x7ff968003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981e71640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff968004d40 0x7ff9680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff980e6f640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 0x7ff968004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981670640 1 -- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 msgr2=0x7ff968004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981670640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 0x7ff968004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981670640 1 -- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff968004d40 msgr2=0x7ff9680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981670640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff968004d40 0x7ff9680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff981670640 1 -- 172.21.15.26:0/18713138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968003520 con 0x7ff9680039b0 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.190+0000 7ff980e6f640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff97c0c24e0 0x7ff968004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.191+0000 7ff981670640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9680039b0 0x7ff968003dd0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c210 tx=0x7ff968005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce9dc87e775b366 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.191+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970007d50 con 0x7ff9680039b0 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.191+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff970030070 con 0x7ff9680039b0 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.191+0000 7ff96ffff640 1 -- 172.21.15.26:0/18713138 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97002cda0 con 0x7ff9680039b0 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.192+0000 7ff9838d4640 1 -- 172.21.15.26:0/18713138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9680039b0 msgr2=0x7ff968003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.192+0000 7ff9838d4640 1 --2- 172.21.15.26:0/18713138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9680039b0 0x7ff968003dd0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c210 tx=0x7ff968005830 comp rx=0 tx=0).stop 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.192+0000 7ff981e71640 1 -- 172.21.15.26:0/18713138 reap_dead start 2024-03-06T07:27:22.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.192+0000 7ff9838d4640 1 -- 172.21.15.26:0/18713138 shutdown_connections 2024-03-06T07:27:22.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.192+0000 7ff9838d4640 1 -- 172.21.15.26:0/18713138 >> 172.21.15.26:0/18713138 conn(0x7ff97c0c5560 msgr2=0x7ff97c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:22.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.192+0000 7ff9838d4640 1 -- 172.21.15.26:0/18713138 shutdown_connections 2024-03-06T07:27:22.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.192+0000 7ff9838d4640 1 -- 172.21.15.26:0/18713138 wait complete. 2024-03-06T07:27:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.193+0000 7ff9838d4640 1 Processor -- start 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.193+0000 7ff9838d4640 1 -- start start 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff9838d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9680039b0 0x7ff97c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff9838d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff97c1eac20 0x7ff97c1eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff9838d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97c0acb60 0x7ff97c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff9838d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c06dc20 con 0x7ff97c1eac20 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff9838d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0c3c70 con 0x7ff97c0acb60 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff9838d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c06dda0 con 0x7ff9680039b0 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff97c1eac20 0x7ff97c1eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff97c1eac20 0x7ff97c1eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56810/0 (socket says 172.21.15.26:56810) 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 -- 172.21.15.26:0/809239201 learned_addr learned my addr 172.21.15.26:0/809239201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff981670640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9680039b0 0x7ff97c06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff981e71640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97c0acb60 0x7ff97c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 -- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9680039b0 msgr2=0x7ff97c06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9680039b0 0x7ff97c06f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 -- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97c0acb60 msgr2=0x7ff97c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97c0acb60 0x7ff97c0ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 -- 172.21.15.26:0/809239201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c1ebc00 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff981e71640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97c0acb60 0x7ff97c0ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff980e6f640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff97c1eac20 0x7ff97c1eb0a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff978007f30 tx=0x7ff978010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff96dffb640 1 -- 172.21.15.26:0/809239201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97801a020 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.194+0000 7ff96dffb640 1 -- 172.21.15.26:0/809239201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff978015640 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.195+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97c14fa50 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.195+0000 7ff96dffb640 1 -- 172.21.15.26:0/809239201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978015a30 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.195+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97c1eea90 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.196+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97c04f330 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.196+0000 7ff96dffb640 1 -- 172.21.15.26:0/809239201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff978005d10 con 0x7ff97c1eac20 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.197+0000 7ff96dffb640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff96006b4d0 0x7ff96006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.197+0000 7ff96dffb640 1 -- 172.21.15.26:0/809239201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff97808b500 con 0x7ff97c1eac20 2024-03-06T07:27:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.197+0000 7ff981670640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff96006b4d0 0x7ff96006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.197+0000 7ff981670640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff96006b4d0 0x7ff96006d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff970034000 tx=0x7ff970007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.201+0000 7ff96dffb640 1 -- 172.21.15.26:0/809239201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff978024030 con 0x7ff97c1eac20 2024-03-06T07:27:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:22.420+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff97c0a89d0 con 0x7ff97c1eac20 2024-03-06T07:27:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.067+0000 7ff96dffb640 1 -- 172.21.15.26:0/809239201 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v168) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff97c1ebc00 con 0x7ff97c1eac20 2024-03-06T07:27:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff96006b4d0 msgr2=0x7ff96006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff96006b4d0 0x7ff96006d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff970034000 tx=0x7ff970007000 comp rx=0 tx=0).stop 2024-03-06T07:27:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff97c1eac20 msgr2=0x7ff97c1eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff97c1eac20 0x7ff97c1eb0a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff978007f30 tx=0x7ff978010650 comp rx=0 tx=0).stop 2024-03-06T07:27:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 shutdown_connections 2024-03-06T07:27:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff96006b4d0 0x7ff96006d990 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97c0acb60 0x7ff97c0ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff97c1eac20 0x7ff97c1eb0a0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 --2- 172.21.15.26:0/809239201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9680039b0 0x7ff97c06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.069+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 >> 172.21.15.26:0/809239201 conn(0x7ff97c0c5560 msgr2=0x7ff97c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.070+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 shutdown_connections 2024-03-06T07:27:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.070+0000 7ff9838d4640 1 -- 172.21.15.26:0/809239201 wait complete. 2024-03-06T07:27:23.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:27:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.236+0000 7fa073135640 1 Processor -- start 2024-03-06T07:27:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa073135640 1 -- start start 2024-03-06T07:27:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa073135640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa06c152860 0x7fa06c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa073135640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa06c0c7470 0x7fa06c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa073135640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c0c8ea0 con 0x7fa06c159230 2024-03-06T07:27:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa073135640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c158da0 con 0x7fa06c152860 2024-03-06T07:27:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa073135640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c156260 con 0x7fa06c0c7470 2024-03-06T07:27:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa070ed1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa06c152860 0x7fa06c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa0716d2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa06c159230 0x7fa06c0c53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35500/0 (socket says 172.21.15.26:35500) 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa070ed1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa06c152860 0x7fa06c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56826/0 (socket says 172.21.15.26:56826) 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.237+0000 7fa0716d2640 1 -- 172.21.15.26:0/2743435979 learned_addr learned my addr 172.21.15.26:0/2743435979 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.238+0000 7fa06bfff640 1 --2- 172.21.15.26:0/2743435979 >> v2:172.21.15.117:3300/0 conn(0x7fa06c0c7470 0x7fa06c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.238+0000 7fa070ed1640 1 -- 172.21.15.26:0/2743435979 >> v2:172.21.15.117:3300/0 conn(0x7fa06c0c7470 msgr2=0x7fa06c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.238+0000 7fa070ed1640 1 --2- 172.21.15.26:0/2743435979 >> v2:172.21.15.117:3300/0 conn(0x7fa06c0c7470 0x7fa06c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.238+0000 7fa070ed1640 1 -- 172.21.15.26:0/2743435979 >> v1:172.21.15.26:6789/0 conn(0x7fa06c159230 legacy=0x7fa06c0c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.238+0000 7fa06b7fe640 1 -- 172.21.15.26:0/2743435979 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589313422 0 0) 0x7fa06c0c8ea0 con 0x7fa06c159230 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.238+0000 7fa070ed1640 1 -- 172.21.15.26:0/2743435979 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c0c7a10 con 0x7fa06c152860 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa070ed1640 1 --2- 172.21.15.26:0/2743435979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c152860 0x7fa06c0c8960 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fa06c08c410 tx=0x7fa06002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f42050ef1e68b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa06b7fe640 1 -- 172.21.15.26:0/2743435979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0600315a0 con 0x7fa06c152860 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa06b7fe640 1 -- 172.21.15.26:0/2743435979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa060037070 con 0x7fa06c152860 2024-03-06T07:27:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa06b7fe640 1 -- 172.21.15.26:0/2743435979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060031960 con 0x7fa06c152860 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 -- 172.21.15.26:0/2743435979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c152860 msgr2=0x7fa06c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 --2- 172.21.15.26:0/2743435979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c152860 0x7fa06c0c8960 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fa06c08c410 tx=0x7fa06002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 -- 172.21.15.26:0/2743435979 shutdown_connections 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 --2- 172.21.15.26:0/2743435979 >> v2:172.21.15.117:3300/0 conn(0x7fa06c0c7470 0x7fa06c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 --2- 172.21.15.26:0/2743435979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c152860 0x7fa06c0c8960 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 -- 172.21.15.26:0/2743435979 >> 172.21.15.26:0/2743435979 conn(0x7fa06c0ac630 msgr2=0x7fa06c0aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 -- 172.21.15.26:0/2743435979 shutdown_connections 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.239+0000 7fa073135640 1 -- 172.21.15.26:0/2743435979 wait complete. 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.240+0000 7fa073135640 1 Processor -- start 2024-03-06T07:27:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.240+0000 7fa073135640 1 -- start start 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa073135640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c0c7470 0x7fa06c1eb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa073135640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa06c152860 0x7fa06c1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa073135640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa06c159230 0x7fa06c148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa073135640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c152c40 con 0x7fa06c0c7470 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa073135640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c158a30 con 0x7fa06c152860 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa073135640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0c5af0 con 0x7fa06c159230 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa070ed1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c0c7470 0x7fa06c1eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa070ed1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c0c7470 0x7fa06c1eb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56828/0 (socket says 172.21.15.26:56828) 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.241+0000 7fa070ed1640 1 -- 172.21.15.26:0/3364946769 learned_addr learned my addr 172.21.15.26:0/3364946769 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa06bfff640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa06c152860 0x7fa06c1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa070ed1640 1 -- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa06c159230 msgr2=0x7fa06c148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa070ed1640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa06c159230 0x7fa06c148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa070ed1640 1 -- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa06c152860 msgr2=0x7fa06c1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa070ed1640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa06c152860 0x7fa06c1558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa070ed1640 1 -- 172.21.15.26:0/3364946769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c1eb9a0 con 0x7fa06c0c7470 2024-03-06T07:27:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa070ed1640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c0c7470 0x7fa06c1eb460 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fa060009890 tx=0x7fa0600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa0697fa640 1 -- 172.21.15.26:0/3364946769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060032840 con 0x7fa06c0c7470 2024-03-06T07:27:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa0697fa640 1 -- 172.21.15.26:0/3364946769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa060032a00 con 0x7fa06c0c7470 2024-03-06T07:27:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c0c3b80 con 0x7fa06c0c7470 2024-03-06T07:27:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa0697fa640 1 -- 172.21.15.26:0/3364946769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060032df0 con 0x7fa06c0c7470 2024-03-06T07:27:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.242+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c0c1de0 con 0x7fa06c0c7470 2024-03-06T07:27:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.243+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c04f330 con 0x7fa06c0c7470 2024-03-06T07:27:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.244+0000 7fa0697fa640 1 -- 172.21.15.26:0/3364946769 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa06004c020 con 0x7fa06c0c7470 2024-03-06T07:27:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.244+0000 7fa0697fa640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa05006b500 0x7fa05006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.244+0000 7fa0697fa640 1 -- 172.21.15.26:0/3364946769 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa06002f020 con 0x7fa06c0c7470 2024-03-06T07:27:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.244+0000 7fa06bfff640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa05006b500 0x7fa05006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.245+0000 7fa06bfff640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa05006b500 0x7fa05006d9c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0c17c0 tx=0x7fa058008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.248+0000 7fa0697fa640 1 -- 172.21.15.26:0/3364946769 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa060040070 con 0x7fa06c0c7470 2024-03-06T07:27:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:23.429+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa06c0b0960 con 0x7fa06c0c7470 2024-03-06T07:27:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.074+0000 7fa0697fa640 1 -- 172.21.15.26:0/3364946769 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v174) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa06007add0 con 0x7fa06c0c7470 2024-03-06T07:27:24.076 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:27:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.076+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa05006b500 msgr2=0x7fa05006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.076+0000 7fa073135640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa05006b500 0x7fa05006d9c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0c17c0 tx=0x7fa058008040 comp rx=0 tx=0).stop 2024-03-06T07:27:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.076+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c0c7470 msgr2=0x7fa06c1eb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.076+0000 7fa073135640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c0c7470 0x7fa06c1eb460 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fa060009890 tx=0x7fa0600046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 shutdown_connections 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa05006b500 0x7fa05006d9c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa06c159230 0x7fa06c148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa06c152860 0x7fa06c1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 --2- 172.21.15.26:0/3364946769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa06c0c7470 0x7fa06c1eb460 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 >> 172.21.15.26:0/3364946769 conn(0x7fa06c0ac630 msgr2=0x7fa06c0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 shutdown_connections 2024-03-06T07:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.077+0000 7fa073135640 1 -- 172.21.15.26:0/3364946769 wait complete. 2024-03-06T07:27:24.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:27:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.202+0000 7f28c99e9640 1 Processor -- start 2024-03-06T07:27:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.202+0000 7f28c99e9640 1 -- start start 2024-03-06T07:27:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c99e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28c4151e10 0x7f28c4153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c99e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c406e850 con 0x7f28c40a9830 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c99e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c406d2b0 con 0x7f28c40c22b0 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c99e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c406d590 con 0x7f28c4151e10 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28c4151e10 0x7f28c4153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c27fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f28c40a9830 0x7f28c4153390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35512/0 (socket says 172.21.15.26:35512) 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c27fc640 1 -- 172.21.15.26:0/2721226126 learned_addr learned my addr 172.21.15.26:0/2721226126 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c37fe640 1 -- 172.21.15.26:0/2721226126 >> v1:172.21.15.117:6789/0 conn(0x7f28c40c22b0 legacy=0x7f28c414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c37fe640 1 -- 172.21.15.26:0/2721226126 >> v1:172.21.15.26:6789/0 conn(0x7f28c40a9830 legacy=0x7f28c4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c1ffb640 1 -- 172.21.15.26:0/2721226126 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792681901 0 0) 0x7f28b8001110 con 0x7f28c40a9830 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.203+0000 7f28c37fe640 1 -- 172.21.15.26:0/2721226126 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c406d870 con 0x7f28c4151e10 2024-03-06T07:27:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c37fe640 1 --2- 172.21.15.26:0/2721226126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c4151e10 0x7f28c4153aa0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009dc0 tx=0x7f28bc02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622d7620a49873d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c1ffb640 1 -- 172.21.15.26:0/2721226126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc039070 con 0x7f28c4151e10 2024-03-06T07:27:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c1ffb640 1 -- 172.21.15.26:0/2721226126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc02cc40 con 0x7f28c4151e10 2024-03-06T07:27:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c1ffb640 1 -- 172.21.15.26:0/2721226126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc034440 con 0x7f28c4151e10 2024-03-06T07:27:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c99e9640 1 -- 172.21.15.26:0/2721226126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c4151e10 msgr2=0x7f28c4153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c99e9640 1 --2- 172.21.15.26:0/2721226126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c4151e10 0x7f28c4153aa0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009dc0 tx=0x7f28bc02be90 comp rx=0 tx=0).stop 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c99e9640 1 -- 172.21.15.26:0/2721226126 shutdown_connections 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c99e9640 1 --2- 172.21.15.26:0/2721226126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c4151e10 0x7f28c4153aa0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.204+0000 7f28c99e9640 1 -- 172.21.15.26:0/2721226126 >> 172.21.15.26:0/2721226126 conn(0x7f28c40c65b0 msgr2=0x7f28c40c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.205+0000 7f28c99e9640 1 -- 172.21.15.26:0/2721226126 shutdown_connections 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.205+0000 7f28c99e9640 1 -- 172.21.15.26:0/2721226126 wait complete. 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 Processor -- start 2024-03-06T07:27:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 -- start start 2024-03-06T07:27:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c40a9830 0x7f28c4150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c40c22b0 0x7f28c414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c4151e10 0x7f28c414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c414f660 con 0x7f28c40a9830 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4153fe0 con 0x7f28c4151e10 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.206+0000 7f28c99e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c414f7e0 con 0x7f28c40c22b0 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c40a9830 0x7f28c4150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c40a9830 0x7f28c4150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56838/0 (socket says 172.21.15.26:56838) 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 -- 172.21.15.26:0/885911807 learned_addr learned my addr 172.21.15.26:0/885911807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c37fe640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c4151e10 0x7f28c414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c27fc640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c40c22b0 0x7f28c414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 -- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c40c22b0 msgr2=0x7f28c414ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c40c22b0 0x7f28c414ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 -- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c4151e10 msgr2=0x7f28c414d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c4151e10 0x7f28c414d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 -- 172.21.15.26:0/885911807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c414a1e0 con 0x7f28c40a9830 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c2ffd640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c40a9830 0x7f28c4150cd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f28b400c410 tx=0x7f28b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28abfff640 1 -- 172.21.15.26:0/885911807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4017070 con 0x7f28c40a9830 2024-03-06T07:27:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28abfff640 1 -- 172.21.15.26:0/885911807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b40035f0 con 0x7f28c40a9830 2024-03-06T07:27:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28abfff640 1 -- 172.21.15.26:0/885911807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4003a20 con 0x7f28c40a9830 2024-03-06T07:27:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c41eb700 con 0x7f28c40a9830 2024-03-06T07:27:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.207+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c414df00 con 0x7f28c40a9830 2024-03-06T07:27:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.208+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c404f330 con 0x7f28c40a9830 2024-03-06T07:27:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.209+0000 7f28abfff640 1 -- 172.21.15.26:0/885911807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28b401a070 con 0x7f28c40a9830 2024-03-06T07:27:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.209+0000 7f28abfff640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b006b4d0 0x7f28b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.209+0000 7f28abfff640 1 -- 172.21.15.26:0/885911807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f28b4089630 con 0x7f28c40a9830 2024-03-06T07:27:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.209+0000 7f28c27fc640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b006b4d0 0x7f28b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.210+0000 7f28c27fc640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b006b4d0 0x7f28b006d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f28c41e2340 tx=0x7f28b8007970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.213+0000 7f28abfff640 1 -- 172.21.15.26:0/885911807 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28b4058510 con 0x7f28c40a9830 2024-03-06T07:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:24.394+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28c40abd50 con 0x7f28c40a9830 2024-03-06T07:27:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.075+0000 7f28abfff640 1 -- 172.21.15.26:0/885911807 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v175) v1 ==== 169+0+0 (secure 0 0 0) 0x7f28b4057e10 con 0x7f28c40a9830 2024-03-06T07:27:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:27:25.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.078+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b006b4d0 msgr2=0x7f28b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.078+0000 7f28c99e9640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b006b4d0 0x7f28b006d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f28c41e2340 tx=0x7f28b8007970 comp rx=0 tx=0).stop 2024-03-06T07:27:25.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.078+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c40a9830 msgr2=0x7f28c4150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.078+0000 7f28c99e9640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c40a9830 0x7f28c4150cd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f28b400c410 tx=0x7f28b400c8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.079+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 shutdown_connections 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.079+0000 7f28c99e9640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f28b006b4d0 0x7f28b006d990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.079+0000 7f28c99e9640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c4151e10 0x7f28c414d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.079+0000 7f28c99e9640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c40c22b0 0x7f28c414ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.079+0000 7f28c99e9640 1 --2- 172.21.15.26:0/885911807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28c40a9830 0x7f28c4150cd0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.080+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 >> 172.21.15.26:0/885911807 conn(0x7f28c40c65b0 msgr2=0x7f28c40bfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.080+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 shutdown_connections 2024-03-06T07:27:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.080+0000 7f28c99e9640 1 -- 172.21.15.26:0/885911807 wait complete. 2024-03-06T07:27:25.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:27:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.205+0000 7f5fdecd4640 1 Processor -- start 2024-03-06T07:27:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.205+0000 7f5fdecd4640 1 -- start start 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdecd4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 0x7f5fd80c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdecd4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5fd80c88f0 0x7f5fd8155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdecd4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fd80c58b0 0x7f5fd8155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdecd4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd80c9130 con 0x7f5fd80c58b0 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdecd4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd8159260 con 0x7f5fd80c88f0 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdecd4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80c6b90 con 0x7f5fd81527f0 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fd80c58b0 0x7f5fd8155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fd80c58b0 0x7f5fd8155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56840/0 (socket says 172.21.15.26:56840) 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 -- 172.21.15.26:0/1707643157 learned_addr learned my addr 172.21.15.26:0/1707643157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 -- 172.21.15.26:0/1707643157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 msgr2=0x7f5fd80c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 --2- 172.21.15.26:0/1707643157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 0x7f5fd80c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 -- 172.21.15.26:0/1707643157 >> v2:172.21.15.117:3300/0 conn(0x7f5fd80c88f0 msgr2=0x7f5fd8155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 --2- 172.21.15.26:0/1707643157 >> v2:172.21.15.117:3300/0 conn(0x7f5fd80c88f0 0x7f5fd8155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.206+0000 7f5fdd271640 1 -- 172.21.15.26:0/1707643157 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd81596f0 con 0x7f5fd80c58b0 2024-03-06T07:27:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.207+0000 7f5fdd271640 1 --2- 172.21.15.26:0/1707643157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c58b0 0x7f5fd8155750 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5fd808c4f0 tx=0x7f5fc002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50c799437d4e954 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.207+0000 7f5fd77fe640 1 -- 172.21.15.26:0/1707643157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0038070 con 0x7f5fd80c58b0 2024-03-06T07:27:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.207+0000 7f5fd77fe640 1 -- 172.21.15.26:0/1707643157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc002f050 con 0x7f5fd80c58b0 2024-03-06T07:27:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.207+0000 7f5fd77fe640 1 -- 172.21.15.26:0/1707643157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0031970 con 0x7f5fd80c58b0 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 -- 172.21.15.26:0/1707643157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c58b0 msgr2=0x7f5fd8155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/1707643157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c58b0 0x7f5fd8155750 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5fd808c4f0 tx=0x7f5fc002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 -- 172.21.15.26:0/1707643157 shutdown_connections 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/1707643157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c58b0 0x7f5fd8155750 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/1707643157 >> v2:172.21.15.117:3300/0 conn(0x7f5fd80c88f0 0x7f5fd8155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/1707643157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 0x7f5fd80c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 -- 172.21.15.26:0/1707643157 >> 172.21.15.26:0/1707643157 conn(0x7f5fd80ac580 msgr2=0x7f5fd80ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 -- 172.21.15.26:0/1707643157 shutdown_connections 2024-03-06T07:27:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.208+0000 7f5fdecd4640 1 -- 172.21.15.26:0/1707643157 wait complete. 2024-03-06T07:27:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.209+0000 7f5fdecd4640 1 Processor -- start 2024-03-06T07:27:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.209+0000 7f5fdecd4640 1 -- start start 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdecd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd80c58b0 0x7f5fd814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdecd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c88f0 0x7f5fd80bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdecd4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 0x7f5fd81ead90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdecd4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8158c10 con 0x7f5fd80c88f0 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdecd4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8158930 con 0x7f5fd80c58b0 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdecd4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8158ef0 con 0x7f5fd81527f0 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c88f0 0x7f5fd80bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c88f0 0x7f5fd80bcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56856/0 (socket says 172.21.15.26:56856) 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 -- 172.21.15.26:0/2755383634 learned_addr learned my addr 172.21.15.26:0/2755383634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdca70640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd80c58b0 0x7f5fd814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fdd271640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 0x7f5fd81ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 -- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 msgr2=0x7f5fd81ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 0x7f5fd81ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 -- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd80c58b0 msgr2=0x7f5fd814f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd80c58b0 0x7f5fd814f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.210+0000 7f5fd7fff640 1 -- 172.21.15.26:0/2755383634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd81585c0 con 0x7f5fd80c88f0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.211+0000 7f5fd7fff640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c88f0 0x7f5fd80bcdb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ba50 tx=0x7f5fc800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.211+0000 7f5fd57fa640 1 -- 172.21.15.26:0/2755383634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8019070 con 0x7f5fd80c88f0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.211+0000 7f5fd57fa640 1 -- 172.21.15.26:0/2755383634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc800cb40 con 0x7f5fd80c88f0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.211+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd81ee3d0 con 0x7f5fd80c88f0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.211+0000 7f5fd57fa640 1 -- 172.21.15.26:0/2755383634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8014440 con 0x7f5fd80c88f0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.211+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd81ee990 con 0x7f5fd80c88f0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.212+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd804f330 con 0x7f5fd80c88f0 2024-03-06T07:27:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.212+0000 7f5fd57fa640 1 -- 172.21.15.26:0/2755383634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fc801c070 con 0x7f5fd80c88f0 2024-03-06T07:27:25.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.213+0000 7f5fd57fa640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fb806b500 0x7f5fb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.213+0000 7f5fd57fa640 1 -- 172.21.15.26:0/2755383634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5fc8089bf0 con 0x7f5fd80c88f0 2024-03-06T07:27:25.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.213+0000 7f5fdca70640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fb806b500 0x7f5fb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.214+0000 7f5fdca70640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fb806b500 0x7f5fb806d9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009990 tx=0x7f5fd0009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.217+0000 7f5fd57fa640 1 -- 172.21.15.26:0/2755383634 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5fc8058620 con 0x7f5fd80c88f0 2024-03-06T07:27:25.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.397+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5fd8151ce0 con 0x7f5fd80c88f0 2024-03-06T07:27:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.400+0000 7f5fd57fa640 1 -- 172.21.15.26:0/2755383634 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5fc805c6c0 con 0x7f5fd80c88f0 2024-03-06T07:27:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fb806b500 msgr2=0x7f5fb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fb806b500 0x7f5fb806d9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009990 tx=0x7f5fd0009110 comp rx=0 tx=0).stop 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c88f0 msgr2=0x7f5fd80bcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c88f0 0x7f5fd80bcdb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ba50 tx=0x7f5fc800be50 comp rx=0 tx=0).stop 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 shutdown_connections 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fb806b500 0x7f5fb806d9c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fd81527f0 0x7f5fd81ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80c88f0 0x7f5fd80bcdb0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 --2- 172.21.15.26:0/2755383634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd80c58b0 0x7f5fd814f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 >> 172.21.15.26:0/2755383634 conn(0x7f5fd80ac580 msgr2=0x7f5fd80ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 shutdown_connections 2024-03-06T07:27:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.403+0000 7f5fdecd4640 1 -- 172.21.15.26:0/2755383634 wait complete. 2024-03-06T07:27:25.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down' 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.514+0000 7f39b915c640 1 Processor -- start 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b915c640 1 -- start start 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b915c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39b406da40 0x7f39b4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b915c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39b40c7380 0x7f39b40c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b915c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b4156050 con 0x7f39b40c6780 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b915c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b406de20 con 0x7f39b40c7380 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b915c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b4072630 con 0x7f39b406da40 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39b40c7380 0x7f39b40c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39b40c7380 0x7f39b40c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56858/0 (socket says 172.21.15.26:56858) 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.515+0000 7f39b359e640 1 -- 172.21.15.26:0/993867124 learned_addr learned my addr 172.21.15.26:0/993867124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.516+0000 7f39b2d9d640 1 --2- 172.21.15.26:0/993867124 >> v2:172.21.15.117:3300/0 conn(0x7f39b406da40 0x7f39b4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.516+0000 7f39b359e640 1 -- 172.21.15.26:0/993867124 >> v2:172.21.15.117:3300/0 conn(0x7f39b406da40 msgr2=0x7f39b4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.516+0000 7f39b359e640 1 --2- 172.21.15.26:0/993867124 >> v2:172.21.15.117:3300/0 conn(0x7f39b406da40 0x7f39b4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.516+0000 7f39b359e640 1 -- 172.21.15.26:0/993867124 >> v1:172.21.15.117:6789/0 conn(0x7f39b40c6780 legacy=0x7f39b4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.516+0000 7f39b359e640 1 -- 172.21.15.26:0/993867124 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4072910 con 0x7f39b40c7380 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.516+0000 7f39b2d9d640 1 --2- 172.21.15.26:0/993867124 >> v2:172.21.15.117:3300/0 conn(0x7f39b406da40 0x7f39b4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:25.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.516+0000 7f39b359e640 1 --2- 172.21.15.26:0/993867124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c7380 0x7f39b40c3280 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f39b40c98a0 tx=0x7f39a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64daf696337bb90e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b1d9b640 1 -- 172.21.15.26:0/993867124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a402ca80 con 0x7f39b40c7380 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b1d9b640 1 -- 172.21.15.26:0/993867124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a402cc20 con 0x7f39b40c7380 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b1d9b640 1 -- 172.21.15.26:0/993867124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40343f0 con 0x7f39b40c7380 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 -- 172.21.15.26:0/993867124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c7380 msgr2=0x7f39b40c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 --2- 172.21.15.26:0/993867124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c7380 0x7f39b40c3280 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f39b40c98a0 tx=0x7f39a402bee0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 -- 172.21.15.26:0/993867124 shutdown_connections 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 --2- 172.21.15.26:0/993867124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c7380 0x7f39b40c3280 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 --2- 172.21.15.26:0/993867124 >> v2:172.21.15.117:3300/0 conn(0x7f39b406da40 0x7f39b4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 -- 172.21.15.26:0/993867124 >> 172.21.15.26:0/993867124 conn(0x7f39b40aa630 msgr2=0x7f39b40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 -- 172.21.15.26:0/993867124 shutdown_connections 2024-03-06T07:27:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.517+0000 7f39b915c640 1 -- 172.21.15.26:0/993867124 wait complete. 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.518+0000 7f39b915c640 1 Processor -- start 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b915c640 1 -- start start 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b915c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b406da40 0x7f39b414d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b915c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c6780 0x7f39b406c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b915c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b40c7380 0x7f39b41eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b915c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b41594a0 con 0x7f39b40c6780 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b915c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4158890 con 0x7f39b406da40 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b915c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40c37c0 con 0x7f39b40c7380 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c6780 0x7f39b406c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c6780 0x7f39b406c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56870/0 (socket says 172.21.15.26:56870) 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b259c640 1 -- 172.21.15.26:0/1392907602 learned_addr learned my addr 172.21.15.26:0/1392907602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b2d9d640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b406da40 0x7f39b414d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.519+0000 7f39b359e640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b40c7380 0x7f39b41eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b259c640 1 -- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b40c7380 msgr2=0x7f39b41eeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b259c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b40c7380 0x7f39b41eeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b259c640 1 -- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b406da40 msgr2=0x7f39b414d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b259c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b406da40 0x7f39b414d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b259c640 1 -- 172.21.15.26:0/1392907602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40a99a0 con 0x7f39b40c6780 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b259c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c6780 0x7f39b406c850 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f399c00c480 tx=0x7f399c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39ab7fe640 1 -- 172.21.15.26:0/1392907602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0052d0 con 0x7f39b40c6780 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39ab7fe640 1 -- 172.21.15.26:0/1392907602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c005470 con 0x7f39b40c6780 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b40a9c60 con 0x7f39b40c6780 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39ab7fe640 1 -- 172.21.15.26:0/1392907602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c012870 con 0x7f39b40c6780 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.520+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b40aa220 con 0x7f39b40c6780 2024-03-06T07:27:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.521+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b404f330 con 0x7f39b40c6780 2024-03-06T07:27:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.522+0000 7f39ab7fe640 1 -- 172.21.15.26:0/1392907602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f399c012a10 con 0x7f39b40c6780 2024-03-06T07:27:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.522+0000 7f39ab7fe640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f399406b4d0 0x7f399406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.522+0000 7f39ab7fe640 1 -- 172.21.15.26:0/1392907602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f399c089ba0 con 0x7f39b40c6780 2024-03-06T07:27:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.522+0000 7f39b2d9d640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f399406b4d0 0x7f399406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.523+0000 7f39b2d9d640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f399406b4d0 0x7f399406d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f39b406d0d0 tx=0x7f39ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.526+0000 7f39ab7fe640 1 -- 172.21.15.26:0/1392907602 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f39b4156050 con 0x7f39b40c6780 2024-03-06T07:27:25.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:25.782+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down"]} v 0) v1 -- 0x7f39b4073240 con 0x7f39b40c6780 2024-03-06T07:27:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.085+0000 7f39ab7fe640 1 -- 172.21.15.26:0/1392907602 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down"]}]=0 v426) v1 ==== 133+0+0 (secure 0 0 0) 0x7f399c058550 con 0x7f39b40c6780 2024-03-06T07:27:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f399406b4d0 msgr2=0x7f399406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f399406b4d0 0x7f399406d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f39b406d0d0 tx=0x7f39ac008040 comp rx=0 tx=0).stop 2024-03-06T07:27:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c6780 msgr2=0x7f39b406c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c6780 0x7f39b406c850 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f399c00c480 tx=0x7f399c00c940 comp rx=0 tx=0).stop 2024-03-06T07:27:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 shutdown_connections 2024-03-06T07:27:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f399406b4d0 0x7f399406d990 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b40c7380 0x7f39b41eeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39b40c6780 0x7f39b406c850 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 --2- 172.21.15.26:0/1392907602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b406da40 0x7f39b414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 >> 172.21.15.26:0/1392907602 conn(0x7f39b40aa630 msgr2=0x7f39b40c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 shutdown_connections 2024-03-06T07:27:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.087+0000 7f39b915c640 1 -- 172.21.15.26:0/1392907602 wait complete. 2024-03-06T07:27:26.101 INFO:tasks.cephfs_test_runner:That down setting toggles and sets max_mds appropriately. ... ok 2024-03-06T07:27:26.102 INFO:tasks.cephfs_test_runner:Starting test: test_down_grow (tasks.cephfs.test_failover.TestClusterResize) That setting max_mds undoes down. 2024-03-06T07:27:26.102 INFO:tasks.cephfs_test_runner:test_down_grow (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:27:26.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.203+0000 7f33d231e640 1 Processor -- start 2024-03-06T07:27:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.203+0000 7f33d231e640 1 -- start start 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.203+0000 7f33d231e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33cc150920 0x7f33cc0c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.203+0000 7f33d231e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33cc0c2e70 0x7f33cc15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.203+0000 7f33d231e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33cc067340 con 0x7f33cc06c720 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.203+0000 7f33d231e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33cc150d00 con 0x7f33cc150920 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.203+0000 7f33d231e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33cc0c2010 con 0x7f33cc0c2e70 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33cc150920 0x7f33cc0c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33cb7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f33cc06c720 0x7f33cc071980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35528/0 (socket says 172.21.15.26:35528) 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33cb7fe640 1 -- 172.21.15.26:0/4269354677 learned_addr learned my addr 172.21.15.26:0/4269354677 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33cbfff640 1 -- 172.21.15.26:0/4269354677 >> v2:172.21.15.117:3300/0 conn(0x7f33cc0c2e70 msgr2=0x7f33cc15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33cbfff640 1 --2- 172.21.15.26:0/4269354677 >> v2:172.21.15.117:3300/0 conn(0x7f33cc0c2e70 0x7f33cc15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33cbfff640 1 -- 172.21.15.26:0/4269354677 >> v1:172.21.15.26:6789/0 conn(0x7f33cc06c720 legacy=0x7f33cc071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33caffd640 1 -- 172.21.15.26:0/4269354677 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938707161 0 0) 0x7f33cc067340 con 0x7f33cc06c720 2024-03-06T07:27:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.204+0000 7f33cbfff640 1 -- 172.21.15.26:0/4269354677 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc067060 con 0x7f33cc150920 2024-03-06T07:27:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33cbfff640 1 --2- 172.21.15.26:0/4269354677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc150920 0x7f33cc0c2930 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f33cc1543f0 tx=0x7f33bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fc7d8431372948 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33caffd640 1 -- 172.21.15.26:0/4269354677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0315a0 con 0x7f33cc150920 2024-03-06T07:27:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33caffd640 1 -- 172.21.15.26:0/4269354677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc037070 con 0x7f33cc150920 2024-03-06T07:27:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33caffd640 1 -- 172.21.15.26:0/4269354677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc031960 con 0x7f33cc150920 2024-03-06T07:27:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 -- 172.21.15.26:0/4269354677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc150920 msgr2=0x7f33cc0c2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 --2- 172.21.15.26:0/4269354677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc150920 0x7f33cc0c2930 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f33cc1543f0 tx=0x7f33bc02b840 comp rx=0 tx=0).stop 2024-03-06T07:27:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 -- 172.21.15.26:0/4269354677 shutdown_connections 2024-03-06T07:27:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 --2- 172.21.15.26:0/4269354677 >> v2:172.21.15.117:3300/0 conn(0x7f33cc0c2e70 0x7f33cc15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 --2- 172.21.15.26:0/4269354677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc150920 0x7f33cc0c2930 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 -- 172.21.15.26:0/4269354677 >> 172.21.15.26:0/4269354677 conn(0x7f33cc0c70b0 msgr2=0x7f33cc0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 -- 172.21.15.26:0/4269354677 shutdown_connections 2024-03-06T07:27:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.205+0000 7f33d231e640 1 -- 172.21.15.26:0/4269354677 wait complete. 2024-03-06T07:27:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.206+0000 7f33d231e640 1 Processor -- start 2024-03-06T07:27:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33d231e640 1 -- start start 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33d231e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33cc06c720 0x7f33cc1eb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33d231e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc0c2e70 0x7f33cc145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33d231e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33cc150920 0x7f33cc0c04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33d231e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc073210 con 0x7f33cc0c2e70 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33d231e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc073090 con 0x7f33cc150920 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33d231e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0c1b40 con 0x7f33cc06c720 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc0c2e70 0x7f33cc145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc0c2e70 0x7f33cc145d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56900/0 (socket says 172.21.15.26:56900) 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.207+0000 7f33cb7fe640 1 -- 172.21.15.26:0/407531975 learned_addr learned my addr 172.21.15.26:0/407531975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33cb7fe640 1 -- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33cc06c720 msgr2=0x7f33cc1eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33cbfff640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33cc06c720 0x7f33cc1eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33cb7fe640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33cc06c720 0x7f33cc1eb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33d08bb640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33cc150920 0x7f33cc0c04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33cb7fe640 1 -- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33cc150920 msgr2=0x7f33cc0c04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33cb7fe640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33cc150920 0x7f33cc0c04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33cb7fe640 1 -- 172.21.15.26:0/407531975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc146240 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33d08bb640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33cc150920 0x7f33cc0c04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33cb7fe640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc0c2e70 0x7f33cc145d00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f33c000c410 tx=0x7f33c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33c8ff9640 1 -- 172.21.15.26:0/407531975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c00074e0 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33c8ff9640 1 -- 172.21.15.26:0/407531975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c00047e0 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc0ac250 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33c8ff9640 1 -- 172.21.15.26:0/407531975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c0004bd0 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.208+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc0abf70 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.209+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc04f330 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.209+0000 7f33c8ff9640 1 -- 172.21.15.26:0/407531975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33c0011070 con 0x7f33cc0c2e70 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.210+0000 7f33c8ff9640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33b806b4d0 0x7f33b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.210+0000 7f33c8ff9640 1 -- 172.21.15.26:0/407531975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f33c0089bc0 con 0x7f33cc0c2e70 2024-03-06T07:27:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.210+0000 7f33cbfff640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33b806b4d0 0x7f33b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.210+0000 7f33cbfff640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33b806b4d0 0x7f33b806d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f33bc036000 tx=0x7f33bc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.214+0000 7f33c8ff9640 1 -- 172.21.15.26:0/407531975 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33c000e280 con 0x7f33cc0c2e70 2024-03-06T07:27:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.394+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33cc06eca0 con 0x7f33cc0c2e70 2024-03-06T07:27:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.395+0000 7f33c8ff9640 1 -- 172.21.15.26:0/407531975 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+13315 (secure 0 0 0) 0x7f33c0058c40 con 0x7f33cc0c2e70 2024-03-06T07:27:26.398 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:26.398 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":175,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:25.075731+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1519487439":"2024-03-07T07:27:22.065597+0000","172.21.15.26:6833/1519487439":"2024-03-07T07:27:22.065597+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33b806b4d0 msgr2=0x7f33b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33b806b4d0 0x7f33b806d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f33bc036000 tx=0x7f33bc008030 comp rx=0 tx=0).stop 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc0c2e70 msgr2=0x7f33cc145d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc0c2e70 0x7f33cc145d00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f33c000c410 tx=0x7f33c000c8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 shutdown_connections 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f33b806b4d0 0x7f33b806d990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33cc150920 0x7f33cc0c04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33cc0c2e70 0x7f33cc145d00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 --2- 172.21.15.26:0/407531975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33cc06c720 0x7f33cc1eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 >> 172.21.15.26:0/407531975 conn(0x7f33cc0c70b0 msgr2=0x7f33cc14fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 shutdown_connections 2024-03-06T07:27:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.397+0000 7f33d231e640 1 -- 172.21.15.26:0/407531975 wait complete. 2024-03-06T07:27:26.411 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:27:26.411 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:27:26.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.513+0000 7f8ff34d4640 1 Processor -- start 2024-03-06T07:27:26.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.513+0000 7f8ff34d4640 1 -- start start 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff34d4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8fec0ab170 0x7f8fec0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff34d4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec0c3ef0 con 0x7f8fec0c5990 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff34d4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec1556d0 con 0x7f8fec06c900 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff34d4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec1559b0 con 0x7f8fec0ab170 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff1270640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8fec0c5990 0x7f8fec06c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35542/0 (socket says 172.21.15.26:35542) 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff1270640 1 -- 172.21.15.26:0/3811080177 learned_addr learned my addr 172.21.15.26:0/3811080177 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff1a71640 1 --2- 172.21.15.26:0/3811080177 >> v2:172.21.15.117:3300/0 conn(0x7f8fec0ab170 0x7f8fec0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959880144 0 0) 0x7f8fec0c3ef0 con 0x7f8fec0c5990 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd8005730 con 0x7f8fec0c5990 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835600032 0 0) 0x7f8fec1556d0 con 0x7f8fec06c900 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fec0c3ef0 con 0x7f8fec06c900 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff1a71640 1 -- 172.21.15.26:0/3811080177 >> v1:172.21.15.117:6789/0 conn(0x7f8fec06c900 legacy=0x7f8fec0a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177112805 0 0) 0x7f8fd8005730 con 0x7f8fec0c5990 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff1a71640 1 -- 172.21.15.26:0/3811080177 >> v1:172.21.15.26:6789/0 conn(0x7f8fec0c5990 legacy=0x7f8fec06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.514+0000 7f8ff1a71640 1 -- 172.21.15.26:0/3811080177 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec155c90 con 0x7f8fec0ab170 2024-03-06T07:27:26.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.515+0000 7f8ff1a71640 1 --2- 172.21.15.26:0/3811080177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 0x7f8fec0c0ab0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009dc0 tx=0x7f8fe402be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db997689021e617 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.515+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4039070 con 0x7f8fec0ab170 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.515+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe402c930 con 0x7f8fec0ab170 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.515+0000 7f8fe3fff640 1 -- 172.21.15.26:0/3811080177 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8001670 con 0x7f8fec0ab170 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.516+0000 7f8ff34d4640 1 -- 172.21.15.26:0/3811080177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 msgr2=0x7f8fec0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.516+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/3811080177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 0x7f8fec0c0ab0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009dc0 tx=0x7f8fe402be00 comp rx=0 tx=0).stop 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.516+0000 7f8ff34d4640 1 -- 172.21.15.26:0/3811080177 shutdown_connections 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.516+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/3811080177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 0x7f8fec0c0ab0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.516+0000 7f8ff34d4640 1 -- 172.21.15.26:0/3811080177 >> 172.21.15.26:0/3811080177 conn(0x7f8fec0c6710 msgr2=0x7f8fec0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.516+0000 7f8ff34d4640 1 -- 172.21.15.26:0/3811080177 shutdown_connections 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.516+0000 7f8ff34d4640 1 -- 172.21.15.26:0/3811080177 wait complete. 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.517+0000 7f8ff34d4640 1 Processor -- start 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.517+0000 7f8ff34d4640 1 -- start start 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff34d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fec06c900 0x7f8fec144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff34d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 0x7f8fec0c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff34d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0c5990 0x7f8fec0c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff34d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec156610 con 0x7f8fec06c900 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff34d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec156490 con 0x7f8fec0ab170 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff34d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c3af0 con 0x7f8fec0c5990 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fec06c900 0x7f8fec144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fec06c900 0x7f8fec144fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56902/0 (socket says 172.21.15.26:56902) 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 -- 172.21.15.26:0/1019442170 learned_addr learned my addr 172.21.15.26:0/1019442170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1a71640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0c5990 0x7f8fec0c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff0a6f640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 0x7f8fec0c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 -- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0c5990 msgr2=0x7f8fec0c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0c5990 0x7f8fec0c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 -- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 msgr2=0x7f8fec0c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 0x7f8fec0c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 -- 172.21.15.26:0/1019442170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec14cc50 con 0x7f8fec06c900 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1a71640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0c5990 0x7f8fec0c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff1270640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fec06c900 0x7f8fec144fc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c520 tx=0x7f8fe800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8fe1ffb640 1 -- 172.21.15.26:0/1019442170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8017070 con 0x7f8fec06c900 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8fe1ffb640 1 -- 172.21.15.26:0/1019442170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8005d00 con 0x7f8fec06c900 2024-03-06T07:27:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec0c32e0 con 0x7f8fec06c900 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.518+0000 7f8fe1ffb640 1 -- 172.21.15.26:0/1019442170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80072b0 con 0x7f8fec06c900 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.519+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1543d0 con 0x7f8fec06c900 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.520+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec04f330 con 0x7f8fec06c900 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.520+0000 7f8fe1ffb640 1 -- 172.21.15.26:0/1019442170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fe801a070 con 0x7f8fec06c900 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.520+0000 7f8fe1ffb640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd006b4d0 0x7f8fd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.520+0000 7f8fe1ffb640 1 -- 172.21.15.26:0/1019442170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8fe80894d0 con 0x7f8fec06c900 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.521+0000 7f8ff0a6f640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd006b4d0 0x7f8fd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.521+0000 7f8ff0a6f640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd006b4d0 0x7f8fd006d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8fec1470e0 tx=0x7f8fdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.525+0000 7f8fe1ffb640 1 -- 172.21.15.26:0/1019442170 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8fe8058550 con 0x7f8fec06c900 2024-03-06T07:27:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.703+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fec0c1740 con 0x7f8fec06c900 2024-03-06T07:27:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.704+0000 7f8fe1ffb640 1 -- 172.21.15.26:0/1019442170 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fe8057e50 con 0x7f8fec06c900 2024-03-06T07:27:26.706 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:27:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd006b4d0 msgr2=0x7f8fd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd006b4d0 0x7f8fd006d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8fec1470e0 tx=0x7f8fdc009040 comp rx=0 tx=0).stop 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fec06c900 msgr2=0x7f8fec144fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fec06c900 0x7f8fec144fc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c520 tx=0x7f8fe800c9e0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 shutdown_connections 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd006b4d0 0x7f8fd006d990 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0c5990 0x7f8fec0c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0ab170 0x7f8fec0c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 --2- 172.21.15.26:0/1019442170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fec06c900 0x7f8fec144fc0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 >> 172.21.15.26:0/1019442170 conn(0x7f8fec0c6710 msgr2=0x7f8fec14ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 shutdown_connections 2024-03-06T07:27:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.706+0000 7f8ff34d4640 1 -- 172.21.15.26:0/1019442170 wait complete. 2024-03-06T07:27:26.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_grow' 2024-03-06T07:27:26.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.818+0000 7f2315b13640 1 Processor -- start 2024-03-06T07:27:26.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.818+0000 7f2315b13640 1 -- start start 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.818+0000 7f2315b13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.818+0000 7f2315b13640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23100c8a00 0x7f2310158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.818+0000 7f2315b13640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2310156180 con 0x7f2310159460 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.818+0000 7f2315b13640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23101587b0 con 0x7f23100c8a00 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.818+0000 7f2315b13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100c7760 con 0x7f2310152920 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230effd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23100c8a00 0x7f2310158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:45074/0 (socket says 172.21.15.26:45074) 2024-03-06T07:27:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230f7fe640 1 -- 172.21.15.26:0/69905455 learned_addr learned my addr 172.21.15.26:0/69905455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230ffff640 1 --1- 172.21.15.26:0/69905455 >> v1:172.21.15.117:6789/0 conn(0x7f2310159460 0x7f23100c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:34770/0 (socket says 172.21.15.26:34770) 2024-03-06T07:27:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230f7fe640 1 -- 172.21.15.26:0/69905455 >> v2:172.21.15.117:3300/0 conn(0x7f23100c8a00 msgr2=0x7f2310158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230f7fe640 1 --2- 172.21.15.26:0/69905455 >> v2:172.21.15.117:3300/0 conn(0x7f23100c8a00 0x7f2310158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230f7fe640 1 -- 172.21.15.26:0/69905455 >> v1:172.21.15.117:6789/0 conn(0x7f2310159460 legacy=0x7f23100c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230f7fe640 1 -- 172.21.15.26:0/69905455 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23100c7a40 con 0x7f2310152920 2024-03-06T07:27:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230e7fc640 1 -- 172.21.15.26:0/69905455 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633807572 0 0) 0x7f2308001490 con 0x7f2310159460 2024-03-06T07:27:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.819+0000 7f230effd640 1 --2- 172.21.15.26:0/69905455 >> v2:172.21.15.117:3300/0 conn(0x7f23100c8a00 0x7f2310158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:26.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.820+0000 7f230f7fe640 1 --2- 172.21.15.26:0/69905455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310155c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2300009870 tx=0x7f230002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39920ca6e0b1dba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.820+0000 7f230e7fc640 1 -- 172.21.15.26:0/69905455 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300035020 con 0x7f2310152920 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.820+0000 7f230e7fc640 1 -- 172.21.15.26:0/69905455 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300031640 con 0x7f2310152920 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.820+0000 7f230e7fc640 1 -- 172.21.15.26:0/69905455 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230002d040 con 0x7f2310152920 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 -- 172.21.15.26:0/69905455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 msgr2=0x7f2310155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 --2- 172.21.15.26:0/69905455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310155c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2300009870 tx=0x7f230002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 -- 172.21.15.26:0/69905455 shutdown_connections 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 --2- 172.21.15.26:0/69905455 >> v2:172.21.15.117:3300/0 conn(0x7f23100c8a00 0x7f2310158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 --2- 172.21.15.26:0/69905455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310155c40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 -- 172.21.15.26:0/69905455 >> 172.21.15.26:0/69905455 conn(0x7f23100ac560 msgr2=0x7f23100ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 -- 172.21.15.26:0/69905455 shutdown_connections 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.821+0000 7f2315b13640 1 -- 172.21.15.26:0/69905455 wait complete. 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.822+0000 7f2315b13640 1 Processor -- start 2024-03-06T07:27:26.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.822+0000 7f2315b13640 1 -- start start 2024-03-06T07:27:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23100c8a00 0x7f231014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23100ada30 0x7f2310148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231006d030 con 0x7f23100ada30 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2310158a00 con 0x7f23100c8a00 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100c5a40 con 0x7f2310152920 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23100ada30 0x7f2310148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23100ada30 0x7f2310148140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56910/0 (socket says 172.21.15.26:56910) 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 -- 172.21.15.26:0/199415496 learned_addr learned my addr 172.21.15.26:0/199415496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230f7fe640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23100c8a00 0x7f231014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230effd640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 -- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 msgr2=0x7f2310147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 -- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23100c8a00 msgr2=0x7f231014e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23100c8a00 0x7f231014e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 -- 172.21.15.26:0/199415496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23100aeab0 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230effd640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f230ffff640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23100ada30 0x7f2310148140 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f230800c780 tx=0x7f230800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f22f7fff640 1 -- 172.21.15.26:0/199415496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308018070 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f22f7fff640 1 -- 172.21.15.26:0/199415496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2308004da0 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23100b00e0 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f22f7fff640 1 -- 172.21.15.26:0/199415496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230800a820 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.823+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2310155c30 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.824+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2310067910 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.825+0000 7f22f7fff640 1 -- 172.21.15.26:0/199415496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f230801b070 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.825+0000 7f22f7fff640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22fc06b500 0x7f22fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.825+0000 7f22f7fff640 1 -- 172.21.15.26:0/199415496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2308089550 con 0x7f23100ada30 2024-03-06T07:27:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.825+0000 7f230f7fe640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22fc06b500 0x7f22fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.826+0000 7f230f7fe640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22fc06b500 0x7f22fc06d9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2300036000 tx=0x7f230003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:26.829+0000 7f22f7fff640 1 -- 172.21.15.26:0/199415496 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2310067910 con 0x7f23100ada30 2024-03-06T07:27:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.079+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]} v 0) v1 -- 0x7f231004f330 con 0x7f23100ada30 2024-03-06T07:27:27.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.087+0000 7f22f7fff640 1 -- 172.21.15.26:0/199415496 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]}]=0 v427) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2310156180 con 0x7f23100ada30 2024-03-06T07:27:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22fc06b500 msgr2=0x7f22fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22fc06b500 0x7f22fc06d9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2300036000 tx=0x7f230003c040 comp rx=0 tx=0).stop 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23100ada30 msgr2=0x7f2310148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23100ada30 0x7f2310148140 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f230800c780 tx=0x7f230800cc40 comp rx=0 tx=0).stop 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 shutdown_connections 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22fc06b500 0x7f22fc06d9c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23100ada30 0x7f2310148140 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2310152920 0x7f2310147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 --2- 172.21.15.26:0/199415496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23100c8a00 0x7f231014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 >> 172.21.15.26:0/199415496 conn(0x7f23100ac560 msgr2=0x7f23100c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 shutdown_connections 2024-03-06T07:27:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.089+0000 7f2315b13640 1 -- 172.21.15.26:0/199415496 wait complete. 2024-03-06T07:27:27.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:27:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.204+0000 7f7cf83a1640 1 Processor -- start 2024-03-06T07:27:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.204+0000 7f7cf83a1640 1 -- start start 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf83a1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cf0155c30 0x7f7cf006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf83a1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7cf00c5380 0x7f7cf00c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf83a1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf00b0450 con 0x7f7cf00b0070 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf83a1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf006cfb0 con 0x7f7cf0155c30 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf83a1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf006c020 con 0x7f7cf00c5380 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf593c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cf0155c30 0x7f7cf006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf613d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7cf00b0070 0x7f7cf00c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57674/0 (socket says 172.21.15.26:57674) 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.205+0000 7f7cf613d640 1 -- 172.21.15.26:0/2576833184 learned_addr learned my addr 172.21.15.26:0/2576833184 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.206+0000 7f7cf593c640 1 -- 172.21.15.26:0/2576833184 >> v2:172.21.15.117:3300/0 conn(0x7f7cf00c5380 msgr2=0x7f7cf00c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.206+0000 7f7cf593c640 1 --2- 172.21.15.26:0/2576833184 >> v2:172.21.15.117:3300/0 conn(0x7f7cf00c5380 0x7f7cf00c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.206+0000 7f7cf593c640 1 -- 172.21.15.26:0/2576833184 >> v1:172.21.15.26:6789/0 conn(0x7f7cf00b0070 legacy=0x7f7cf00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.206+0000 7f7cf593c640 1 -- 172.21.15.26:0/2576833184 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf01560e0 con 0x7f7cf0155c30 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.206+0000 7f7cf513b640 1 -- 172.21.15.26:0/2576833184 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395044952 0 0) 0x7f7cec001110 con 0x7f7cf00b0070 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.206+0000 7f7cf593c640 1 --2- 172.21.15.26:0/2576833184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf006ca70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00be030 tx=0x7f7ce402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832974ec20d3240 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf513b640 1 -- 172.21.15.26:0/2576833184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40315a0 con 0x7f7cf0155c30 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf513b640 1 -- 172.21.15.26:0/2576833184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4037070 con 0x7f7cf0155c30 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf513b640 1 -- 172.21.15.26:0/2576833184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4031960 con 0x7f7cf0155c30 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2576833184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 msgr2=0x7f7cf006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2576833184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf006ca70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00be030 tx=0x7f7ce402b840 comp rx=0 tx=0).stop 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2576833184 shutdown_connections 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2576833184 >> v2:172.21.15.117:3300/0 conn(0x7f7cf00c5380 0x7f7cf00c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2576833184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf006ca70 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2576833184 >> 172.21.15.26:0/2576833184 conn(0x7f7cf00ac590 msgr2=0x7f7cf00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2576833184 shutdown_connections 2024-03-06T07:27:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.207+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2576833184 wait complete. 2024-03-06T07:27:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.208+0000 7f7cf83a1640 1 Processor -- start 2024-03-06T07:27:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.208+0000 7f7cf83a1640 1 -- start start 2024-03-06T07:27:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf83a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf00b0070 0x7f7cf01edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf83a1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf00c5380 0x7f7cf014eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf83a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf83a1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf006c270 con 0x7f7cf0155c30 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf83a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00c7ad0 con 0x7f7cf00b0070 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf83a1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf006c550 con 0x7f7cf00c5380 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf693e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf693e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf0154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58234/0 (socket says 172.21.15.26:58234) 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf693e640 1 -- 172.21.15.26:0/2612955139 learned_addr learned my addr 172.21.15.26:0/2612955139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf613d640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf00b0070 0x7f7cf01edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf593c640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf00c5380 0x7f7cf014eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf693e640 1 -- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf00c5380 msgr2=0x7f7cf014eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.209+0000 7f7cf693e640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf00c5380 0x7f7cf014eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7cf693e640 1 -- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf00b0070 msgr2=0x7f7cf01edcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7cf693e640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf00b0070 0x7f7cf01edcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7cf693e640 1 -- 172.21.15.26:0/2612955139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf01ee2d0 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7cf693e640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf0154770 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800b9a0 tx=0x7f7ce800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7ce2ffd640 1 -- 172.21.15.26:0/2612955139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8019070 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7ce2ffd640 1 -- 172.21.15.26:0/2612955139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce800cbc0 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf01ee5b0 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7ce2ffd640 1 -- 172.21.15.26:0/2612955139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8014440 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.210+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf01eeb70 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.211+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf004f330 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.211+0000 7f7ce2ffd640 1 -- 172.21.15.26:0/2612955139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ce801c070 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.212+0000 7f7ce2ffd640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cd406b4d0 0x7f7cd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.212+0000 7f7ce2ffd640 1 -- 172.21.15.26:0/2612955139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7ce8089d90 con 0x7f7cf0155c30 2024-03-06T07:27:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.212+0000 7f7cf613d640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cd406b4d0 0x7f7cd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.212+0000 7f7cf613d640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cd406b4d0 0x7f7cd406d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7cec000f60 tx=0x7f7cec007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.216+0000 7f7ce2ffd640 1 -- 172.21.15.26:0/2612955139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ce8058e10 con 0x7f7cf0155c30 2024-03-06T07:27:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.395+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7cf014fd90 con 0x7f7cf0155c30 2024-03-06T07:27:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.399+0000 7f7ce2ffd640 1 -- 172.21.15.26:0/2612955139 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v59)=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ce8058710 con 0x7f7cf0155c30 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cd406b4d0 msgr2=0x7f7cd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cd406b4d0 0x7f7cd406d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7cec000f60 tx=0x7f7cec007c00 comp rx=0 tx=0).stop 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 msgr2=0x7f7cf0154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf0154770 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800b9a0 tx=0x7f7ce800be60 comp rx=0 tx=0).stop 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 shutdown_connections 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cd406b4d0 0x7f7cd406d990 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0155c30 0x7f7cf0154770 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf00c5380 0x7f7cf014eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 --2- 172.21.15.26:0/2612955139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf00b0070 0x7f7cf01edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 >> 172.21.15.26:0/2612955139 conn(0x7f7cf00ac590 msgr2=0x7f7cf00ae780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 shutdown_connections 2024-03-06T07:27:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.401+0000 7f7cf83a1640 1 -- 172.21.15.26:0/2612955139 wait complete. 2024-03-06T07:27:27.414 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:27:27.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:27.463 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:27.511 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:27:27.511 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:27:27.511 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:27.526 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:27:27.526 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:27:27.526 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:27.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:27:27.544 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:27:27.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.665+0000 7f0cc92df640 1 Processor -- start 2024-03-06T07:27:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.665+0000 7f0cc92df640 1 -- start start 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc92df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0cc4156150 0x7f0cc41529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc92df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40c28d0 con 0x7f0cc40c1aa0 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc92df640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40c2bb0 con 0x7f0cc414f790 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc92df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc40c2680 con 0x7f0cc4156150 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0cc4156150 0x7f0cc41529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc27fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0cc40c1aa0 0x7f0cc4155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57684/0 (socket says 172.21.15.26:57684) 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc27fc640 1 -- 172.21.15.26:0/3709887906 learned_addr learned my addr 172.21.15.26:0/3709887906 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc37fe640 1 -- 172.21.15.26:0/3709887906 >> v1:172.21.15.117:6789/0 conn(0x7f0cc414f790 legacy=0x7f0cc40c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc37fe640 1 -- 172.21.15.26:0/3709887906 >> v1:172.21.15.26:6789/0 conn(0x7f0cc40c1aa0 legacy=0x7f0cc4155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc37fe640 1 -- 172.21.15.26:0/3709887906 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40c2e90 con 0x7f0cc4156150 2024-03-06T07:27:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.666+0000 7f0cc1ffb640 1 -- 172.21.15.26:0/3709887906 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632621061 0 0) 0x7f0cc40c28d0 con 0x7f0cc40c1aa0 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc37fe640 1 --2- 172.21.15.26:0/3709887906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc41529d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40c2800 tx=0x7f0cbc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbe494f0c874ee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc1ffb640 1 -- 172.21.15.26:0/3709887906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc035020 con 0x7f0cc4156150 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc1ffb640 1 -- 172.21.15.26:0/3709887906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc031830 con 0x7f0cc4156150 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc1ffb640 1 -- 172.21.15.26:0/3709887906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc031c00 con 0x7f0cc4156150 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc92df640 1 -- 172.21.15.26:0/3709887906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 msgr2=0x7f0cc41529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc92df640 1 --2- 172.21.15.26:0/3709887906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc41529d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40c2800 tx=0x7f0cbc02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc92df640 1 -- 172.21.15.26:0/3709887906 shutdown_connections 2024-03-06T07:27:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc92df640 1 --2- 172.21.15.26:0/3709887906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc41529d0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.667+0000 7f0cc92df640 1 -- 172.21.15.26:0/3709887906 >> 172.21.15.26:0/3709887906 conn(0x7f0cc40c6710 msgr2=0x7f0cc40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.668+0000 7f0cc92df640 1 -- 172.21.15.26:0/3709887906 shutdown_connections 2024-03-06T07:27:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.668+0000 7f0cc92df640 1 -- 172.21.15.26:0/3709887906 wait complete. 2024-03-06T07:27:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 Processor -- start 2024-03-06T07:27:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 -- start start 2024-03-06T07:27:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cc40c1aa0 0x7f0cc414eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc414f790 0x7f0cc4142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc4143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40c5e30 con 0x7f0cc4156150 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40c0d60 con 0x7f0cc414f790 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc92df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40c5fb0 con 0x7f0cc40c1aa0 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.669+0000 7f0cc37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc4143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc4143350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58262/0 (socket says 172.21.15.26:58262) 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 -- 172.21.15.26:0/544969457 learned_addr learned my addr 172.21.15.26:0/544969457 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 -- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cc40c1aa0 msgr2=0x7f0cc414eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cc40c1aa0 0x7f0cc414eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 -- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc414f790 msgr2=0x7f0cc4142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc414f790 0x7f0cc4142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 -- 172.21.15.26:0/544969457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc4160b70 con 0x7f0cc4156150 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc37fe640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc4143350 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009890 tx=0x7f0cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cbb7fe640 1 -- 172.21.15.26:0/544969457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc03a930 con 0x7f0cc4156150 2024-03-06T07:27:27.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cbb7fe640 1 -- 172.21.15.26:0/544969457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc03aad0 con 0x7f0cc4156150 2024-03-06T07:27:27.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc4147f90 con 0x7f0cc4156150 2024-03-06T07:27:27.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cbb7fe640 1 -- 172.21.15.26:0/544969457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0443f0 con 0x7f0cc4156150 2024-03-06T07:27:27.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.670+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc415fa30 con 0x7f0cc4156150 2024-03-06T07:27:27.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.671+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc404f330 con 0x7f0cc4156150 2024-03-06T07:27:27.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.672+0000 7f0cbb7fe640 1 -- 172.21.15.26:0/544969457 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cbc038020 con 0x7f0cc4156150 2024-03-06T07:27:27.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.672+0000 7f0cbb7fe640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ca406b4d0 0x7f0ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.672+0000 7f0cbb7fe640 1 -- 172.21.15.26:0/544969457 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0cbc0abda0 con 0x7f0cc4156150 2024-03-06T07:27:27.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.672+0000 7f0cc2ffd640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ca406b4d0 0x7f0ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.673+0000 7f0cc2ffd640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ca406b4d0 0x7f0ca406d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4151f20 tx=0x7f0cb4007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.676+0000 7f0cbb7fe640 1 -- 172.21.15.26:0/544969457 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0cbc03f070 con 0x7f0cc4156150 2024-03-06T07:27:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.896+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cc4147820 con 0x7f0cc4156150 2024-03-06T07:27:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.897+0000 7f0cbb7fe640 1 -- 172.21.15.26:0/544969457 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+3710 (secure 0 0 0) 0x7f0cbc07aea0 con 0x7f0cc4156150 2024-03-06T07:27:27.899 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:27.900 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":168,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4097072817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4097072817},{"type":"v1","addr":"172.21.15.117:6836","nonce":4097072817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":7583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1990150014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1990150014},{"type":"v1","addr":"172.21.15.117:6837","nonce":1990150014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":8824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4226578925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4226578925},{"type":"v1","addr":"172.21.15.26:6834","nonce":4226578925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":157},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2731958826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2731958826},{"type":"v1","addr":"172.21.15.26:6835","nonce":2731958826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[]} 2024-03-06T07:27:27.900 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 168 2024-03-06T07:27:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ca406b4d0 msgr2=0x7f0ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ca406b4d0 0x7f0ca406d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4151f20 tx=0x7f0cb4007d10 comp rx=0 tx=0).stop 2024-03-06T07:27:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 msgr2=0x7f0cc4143350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc4143350 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009890 tx=0x7f0cbc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 shutdown_connections 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ca406b4d0 0x7f0ca406d990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cc4156150 0x7f0cc4143350 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc414f790 0x7f0cc4142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 --2- 172.21.15.26:0/544969457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cc40c1aa0 0x7f0cc414eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 >> 172.21.15.26:0/544969457 conn(0x7f0cc40c6710 msgr2=0x7f0cc4155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 shutdown_connections 2024-03-06T07:27:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:27.899+0000 7f0cc92df640 1 -- 172.21.15.26:0/544969457 wait complete. 2024-03-06T07:27:27.912 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:27:27.913 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:27:27.913 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:27:27.913 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:27.913 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:27:27.913 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:27:27.913 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:27:27.913 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:27.914 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:27:27.914 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:27:27.914 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:27:27.914 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:27.914 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:27:27.914 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:27:27.914 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:27:27.914 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:27.922 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:27:27.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:27:27.922 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:27:27.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:27:27.925 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:27:27.925 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:27:27.926 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:27:27.926 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:27:27.965 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:27:27.965 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:27.965 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:27:27.965 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:27.968 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:27:27.968 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:27.969 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:27:27.969 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:27.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:27:28.040 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:27:28.041 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:27:28.052 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:27:28.052 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:27:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.074+0000 7fbcd410c640 1 Processor -- start 2024-03-06T07:27:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.074+0000 7fbcd410c640 1 -- start start 2024-03-06T07:27:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd410c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbccc0c8950 0x7fbccc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd410c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc1558d0 0x7fbccc155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd410c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc155130 con 0x7fbccc0c69f0 2024-03-06T07:27:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd410c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc155440 con 0x7fbccc0c8950 2024-03-06T07:27:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd410c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0ad480 con 0x7fbccc1558d0 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd16a7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbccc0c8950 0x7fbccc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd1ea8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbccc0c69f0 0x7fbccc0acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57686/0 (socket says 172.21.15.26:57686) 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd1ea8640 1 -- 172.21.15.26:0/2793820906 learned_addr learned my addr 172.21.15.26:0/2793820906 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.075+0000 7fbcd26a9640 1 --2- 172.21.15.26:0/2793820906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc1558d0 0x7fbccc155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.076+0000 7fbcd16a7640 1 -- 172.21.15.26:0/2793820906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc1558d0 msgr2=0x7fbccc155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.076+0000 7fbcd16a7640 1 --2- 172.21.15.26:0/2793820906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc1558d0 0x7fbccc155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.076+0000 7fbcd16a7640 1 -- 172.21.15.26:0/2793820906 >> v1:172.21.15.26:6789/0 conn(0x7fbccc0c69f0 legacy=0x7fbccc0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.076+0000 7fbcd16a7640 1 -- 172.21.15.26:0/2793820906 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0c5d50 con 0x7fbccc0c8950 2024-03-06T07:27:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.076+0000 7fbcd0ea6640 1 -- 172.21.15.26:0/2793820906 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748067446 0 0) 0x7fbcc8001110 con 0x7fbccc0c69f0 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.076+0000 7fbcd16a7640 1 --2- 172.21.15.26:0/2793820906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0c8950 0x7fbccc0c6dd0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097a0 tx=0x7fbcc002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b728db2edc9259 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.076+0000 7fbcd0ea6640 1 -- 172.21.15.26:0/2793820906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00315a0 con 0x7fbccc0c8950 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd0ea6640 1 -- 172.21.15.26:0/2793820906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc0037070 con 0x7fbccc0c8950 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd0ea6640 1 -- 172.21.15.26:0/2793820906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0031960 con 0x7fbccc0c8950 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 -- 172.21.15.26:0/2793820906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0c8950 msgr2=0x7fbccc0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 --2- 172.21.15.26:0/2793820906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0c8950 0x7fbccc0c6dd0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097a0 tx=0x7fbcc002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 -- 172.21.15.26:0/2793820906 shutdown_connections 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 --2- 172.21.15.26:0/2793820906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0c8950 0x7fbccc0c6dd0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 --2- 172.21.15.26:0/2793820906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc1558d0 0x7fbccc155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 -- 172.21.15.26:0/2793820906 >> 172.21.15.26:0/2793820906 conn(0x7fbccc0cae70 msgr2=0x7fbccc0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 -- 172.21.15.26:0/2793820906 shutdown_connections 2024-03-06T07:27:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.077+0000 7fbcd410c640 1 -- 172.21.15.26:0/2793820906 wait complete. 2024-03-06T07:27:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.078+0000 7fbcd410c640 1 Processor -- start 2024-03-06T07:27:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.078+0000 7fbcd410c640 1 -- start start 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd410c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc0c69f0 0x7fbccc1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd410c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc0c8950 0x7fbccc06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd410c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1558d0 0x7fbccc14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd410c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc152a20 con 0x7fbccc1558d0 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd410c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0c6310 con 0x7fbccc0c69f0 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd410c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc152ba0 con 0x7fbccc0c8950 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1558d0 0x7fbccc14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1558d0 0x7fbccc14c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58318/0 (socket says 172.21.15.26:58318) 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 -- 172.21.15.26:0/3840789684 learned_addr learned my addr 172.21.15.26:0/3840789684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd16a7640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc0c8950 0x7fbccc06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd1ea8640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc0c69f0 0x7fbccc1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 -- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc0c8950 msgr2=0x7fbccc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc0c8950 0x7fbccc06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 -- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc0c69f0 msgr2=0x7fbccc1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc0c69f0 0x7fbccc1586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 -- 172.21.15.26:0/3840789684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc1590b0 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.079+0000 7fbcd26a9640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1558d0 0x7fbccc14c580 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4007e70 tx=0x7fbcc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.080+0000 7fbcbe7fc640 1 -- 172.21.15.26:0/3840789684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4014620 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.080+0000 7fbcbe7fc640 1 -- 172.21.15.26:0/3840789684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4019070 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.080+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc1eb8d0 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.080+0000 7fbcbe7fc640 1 -- 172.21.15.26:0/3840789684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4014a20 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.080+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc14d5a0 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.081+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc1eb6e0 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.081+0000 7fbcbe7fc640 1 -- 172.21.15.26:0/3840789684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcc4014bc0 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.081+0000 7fbcbe7fc640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcb006b4d0 0x7fbcb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.081+0000 7fbcbe7fc640 1 -- 172.21.15.26:0/3840789684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbcc408a7a0 con 0x7fbccc1558d0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.082+0000 7fbcd1ea8640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcb006b4d0 0x7fbcb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.082+0000 7fbcd1ea8640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcb006b4d0 0x7fbcb006d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8007fd0 tx=0x7fbcc8007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.086+0000 7fbcbe7fc640 1 -- 172.21.15.26:0/3840789684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbcc40591d0 con 0x7fbccc1558d0 2024-03-06T07:27:28.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:28.285+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbccc04f330 con 0x7fbccc1558d0 2024-03-06T07:27:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.105+0000 7fbcbe7fc640 1 -- 172.21.15.26:0/3840789684 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v180) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbcc405d270 con 0x7fbccc1558d0 2024-03-06T07:27:29.108 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:27:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcb006b4d0 msgr2=0x7fbcb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcb006b4d0 0x7fbcb006d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8007fd0 tx=0x7fbcc8007de0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1558d0 msgr2=0x7fbccc14c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1558d0 0x7fbccc14c580 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4007e70 tx=0x7fbcc400e4b0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 shutdown_connections 2024-03-06T07:27:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcb006b4d0 0x7fbcb006d990 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1558d0 0x7fbccc14c580 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbccc0c8950 0x7fbccc06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 --2- 172.21.15.26:0/3840789684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc0c69f0 0x7fbccc1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 >> 172.21.15.26:0/3840789684 conn(0x7fbccc0cae70 msgr2=0x7fbccc0acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 shutdown_connections 2024-03-06T07:27:29.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.108+0000 7fbcd410c640 1 -- 172.21.15.26:0/3840789684 wait complete. 2024-03-06T07:27:29.124 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:27:29.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.229+0000 7fd85ffa1640 1 Processor -- start 2024-03-06T07:27:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.229+0000 7fd85ffa1640 1 -- start start 2024-03-06T07:27:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85ffa1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd85806d9f0 0x7fd8580a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85ffa1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8580c3280 0x7fd8580c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85ffa1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd858158800 con 0x7fd8581551e0 2024-03-06T07:27:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85ffa1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd858158ae0 con 0x7fd85806d9f0 2024-03-06T07:27:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85ffa1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd858158dc0 con 0x7fd8580c3280 2024-03-06T07:27:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85dd3d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd85806d9f0 0x7fd8580a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85d53c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8580c3280 0x7fd8580c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85d53c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8580c3280 0x7fd8580c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:43722/0 (socket says 172.21.15.26:43722) 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85dd3d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd85806d9f0 0x7fd8580a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53458/0 (socket says 172.21.15.26:53458) 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85d53c640 1 -- 172.21.15.26:0/3646281233 learned_addr learned my addr 172.21.15.26:0/3646281233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.230+0000 7fd85e53e640 1 --1- 172.21.15.26:0/3646281233 >> v1:172.21.15.117:6789/0 conn(0x7fd8581551e0 0x7fd858155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:34060/0 (socket says 172.21.15.26:34060) 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.231+0000 7fd85d53c640 1 -- 172.21.15.26:0/3646281233 >> v2:172.21.15.117:3300/0 conn(0x7fd85806d9f0 msgr2=0x7fd8580a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.231+0000 7fd85d53c640 1 --2- 172.21.15.26:0/3646281233 >> v2:172.21.15.117:3300/0 conn(0x7fd85806d9f0 0x7fd8580a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.231+0000 7fd85d53c640 1 -- 172.21.15.26:0/3646281233 >> v1:172.21.15.117:6789/0 conn(0x7fd8581551e0 legacy=0x7fd858155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.231+0000 7fd85d53c640 1 -- 172.21.15.26:0/3646281233 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8581590a0 con 0x7fd8580c3280 2024-03-06T07:27:29.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.231+0000 7fd85dd3d640 1 --2- 172.21.15.26:0/3646281233 >> v2:172.21.15.117:3300/0 conn(0x7fd85806d9f0 0x7fd8580a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.231+0000 7fd85d53c640 1 --2- 172.21.15.26:0/3646281233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8580c3280 0x7fd8580c36e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd848009870 tx=0x7fd84802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce27adfde584a960 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85cd3b640 1 -- 172.21.15.26:0/3646281233 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848035020 con 0x7fd8580c3280 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85cd3b640 1 -- 172.21.15.26:0/3646281233 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848031640 con 0x7fd8580c3280 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85cd3b640 1 -- 172.21.15.26:0/3646281233 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84802d040 con 0x7fd8580c3280 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 -- 172.21.15.26:0/3646281233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8580c3280 msgr2=0x7fd8580c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/3646281233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8580c3280 0x7fd8580c36e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd848009870 tx=0x7fd84802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 -- 172.21.15.26:0/3646281233 shutdown_connections 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/3646281233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8580c3280 0x7fd8580c36e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/3646281233 >> v2:172.21.15.117:3300/0 conn(0x7fd85806d9f0 0x7fd8580a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 -- 172.21.15.26:0/3646281233 >> 172.21.15.26:0/3646281233 conn(0x7fd8580aa630 msgr2=0x7fd8580aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 -- 172.21.15.26:0/3646281233 shutdown_connections 2024-03-06T07:27:29.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.232+0000 7fd85ffa1640 1 -- 172.21.15.26:0/3646281233 wait complete. 2024-03-06T07:27:29.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.233+0000 7fd85ffa1640 1 Processor -- start 2024-03-06T07:27:29.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85ffa1640 1 -- start start 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85ffa1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd85806d9f0 0x7fd8580a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85ffa1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8580c3280 0x7fd85814f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85ffa1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8581551e0 0x7fd85814f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85ffa1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85806df50 con 0x7fd85806d9f0 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85ffa1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85806ddd0 con 0x7fd8580c3280 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85ffa1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580a8210 con 0x7fd8581551e0 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85dd3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd85806d9f0 0x7fd8580a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85dd3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd85806d9f0 0x7fd8580a9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58334/0 (socket says 172.21.15.26:58334) 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85dd3d640 1 -- 172.21.15.26:0/780518776 learned_addr learned my addr 172.21.15.26:0/780518776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85e53e640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8581551e0 0x7fd85814f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.234+0000 7fd85d53c640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8580c3280 0x7fd85814f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85dd3d640 1 -- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8581551e0 msgr2=0x7fd85814f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85dd3d640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8581551e0 0x7fd85814f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85dd3d640 1 -- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8580c3280 msgr2=0x7fd85814f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85dd3d640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8580c3280 0x7fd85814f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85dd3d640 1 -- 172.21.15.26:0/780518776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85814faf0 con 0x7fd85806d9f0 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85dd3d640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd85806d9f0 0x7fd8580a9d80 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd85400e990 tx=0x7fd85400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd84e7fc640 1 -- 172.21.15.26:0/780518776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854017070 con 0x7fd85806d9f0 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd84e7fc640 1 -- 172.21.15.26:0/780518776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85400c9e0 con 0x7fd85806d9f0 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8580c7da0 con 0x7fd85806d9f0 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd84e7fc640 1 -- 172.21.15.26:0/780518776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85400cdb0 con 0x7fd85806d9f0 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.235+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8580ae2a0 con 0x7fd85806d9f0 2024-03-06T07:27:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.236+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85804f330 con 0x7fd85806d9f0 2024-03-06T07:27:29.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.237+0000 7fd84e7fc640 1 -- 172.21.15.26:0/780518776 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd85401a070 con 0x7fd85806d9f0 2024-03-06T07:27:29.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.237+0000 7fd84e7fc640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd83c06b4d0 0x7fd83c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.237+0000 7fd84e7fc640 1 -- 172.21.15.26:0/780518776 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd854089f00 con 0x7fd85806d9f0 2024-03-06T07:27:29.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.237+0000 7fd85d53c640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd83c06b4d0 0x7fd83c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.238+0000 7fd85d53c640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd83c06b4d0 0x7fd83c06d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd848036000 tx=0x7fd84803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.242+0000 7fd84e7fc640 1 -- 172.21.15.26:0/780518776 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd85814faf0 con 0x7fd85806d9f0 2024-03-06T07:27:29.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.493+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd858073240 con 0x7fd85806d9f0 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.494+0000 7fd84e7fc640 1 -- 172.21.15.26:0/780518776 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+4724 (secure 0 0 0) 0x7fd854058870 con 0x7fd85806d9f0 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:27:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.498 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:27:29.499 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.501 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.502 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:29.504 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:27:29.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.506 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:29.508 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:27:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:27:29.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd83c06b4d0 msgr2=0x7fd83c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd83c06b4d0 0x7fd83c06d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd848036000 tx=0x7fd84803c040 comp rx=0 tx=0).stop 2024-03-06T07:27:29.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd85806d9f0 msgr2=0x7fd8580a9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd85806d9f0 0x7fd8580a9d80 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd85400e990 tx=0x7fd85400ee50 comp rx=0 tx=0).stop 2024-03-06T07:27:29.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 shutdown_connections 2024-03-06T07:27:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd83c06b4d0 0x7fd83c06d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8581551e0 0x7fd85814f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8580c3280 0x7fd85814f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 --2- 172.21.15.26:0/780518776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd85806d9f0 0x7fd8580a9d80 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 >> 172.21.15.26:0/780518776 conn(0x7fd8580aa630 msgr2=0x7fd85806c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 shutdown_connections 2024-03-06T07:27:29.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.496+0000 7fd85ffa1640 1 -- 172.21.15.26:0/780518776 wait complete. 2024-03-06T07:27:29.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.611+0000 7f66fb59e640 1 Processor -- start 2024-03-06T07:27:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.611+0000 7f66fb59e640 1 -- start start 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fb59e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc0c8690 0x7f66fc1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fb59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66fc158a40 0x7f66fc158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fb59e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc14be70 con 0x7f66fc14ba90 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fb59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc1593d0 con 0x7f66fc158a40 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fb59e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0cad30 con 0x7f66fc0c8690 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66fc158a40 0x7f66fc158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66fc158a40 0x7f66fc158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58338/0 (socket says 172.21.15.26:58338) 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fad9d640 1 -- 172.21.15.26:0/3320246914 learned_addr learned my addr 172.21.15.26:0/3320246914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66f9d9b640 1 --2- 172.21.15.26:0/3320246914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc0c8690 0x7f66fc1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fad9d640 1 -- 172.21.15.26:0/3320246914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc0c8690 msgr2=0x7f66fc1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fad9d640 1 --2- 172.21.15.26:0/3320246914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc0c8690 0x7f66fc1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fad9d640 1 -- 172.21.15.26:0/3320246914 >> v1:172.21.15.117:6789/0 conn(0x7f66fc14ba90 legacy=0x7f66fc0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:29.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.612+0000 7f66fad9d640 1 -- 172.21.15.26:0/3320246914 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0cb010 con 0x7f66fc158a40 2024-03-06T07:27:29.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.613+0000 7f66fad9d640 1 --2- 172.21.15.26:0/3320246914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc158a40 0x7f66fc158e90 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009e50 tx=0x7f66f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f779fbf2f544252 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.613+0000 7f66f959a640 1 -- 172.21.15.26:0/3320246914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f402ca70 con 0x7f66fc158a40 2024-03-06T07:27:29.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.613+0000 7f66f959a640 1 -- 172.21.15.26:0/3320246914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f402cc10 con 0x7f66fc158a40 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.613+0000 7f66f959a640 1 -- 172.21.15.26:0/3320246914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40343f0 con 0x7f66fc158a40 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 -- 172.21.15.26:0/3320246914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc158a40 msgr2=0x7f66fc158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3320246914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc158a40 0x7f66fc158e90 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009e50 tx=0x7f66f402bed0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 -- 172.21.15.26:0/3320246914 shutdown_connections 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3320246914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc158a40 0x7f66fc158e90 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3320246914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc0c8690 0x7f66fc1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 -- 172.21.15.26:0/3320246914 >> 172.21.15.26:0/3320246914 conn(0x7f66fc0728d0 msgr2=0x7f66fc072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 -- 172.21.15.26:0/3320246914 shutdown_connections 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.614+0000 7f66fb59e640 1 -- 172.21.15.26:0/3320246914 wait complete. 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.615+0000 7f66fb59e640 1 Processor -- start 2024-03-06T07:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.615+0000 7f66fb59e640 1 -- start start 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc0c8690 0x7f66fc1eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fb59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc14ba90 0x7f66fc157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fb59e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc158a40 0x7f66fc145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fb59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc06cf60 con 0x7f66fc0c8690 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fb59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc158520 con 0x7f66fc14ba90 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fb59e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c5820 con 0x7f66fc158a40 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc0c8690 0x7f66fc1eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc0c8690 0x7f66fc1eedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58340/0 (socket says 172.21.15.26:58340) 2024-03-06T07:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 -- 172.21.15.26:0/3090913762 learned_addr learned my addr 172.21.15.26:0/3090913762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 -- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc158a40 msgr2=0x7f66fc145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc158a40 0x7f66fc145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 -- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc14ba90 msgr2=0x7f66fc157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66f9d9b640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc14ba90 0x7f66fc157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc14ba90 0x7f66fc157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 -- 172.21.15.26:0/3090913762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0c4790 con 0x7f66fc0c8690 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66f9d9b640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc14ba90 0x7f66fc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66fa59c640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc0c8690 0x7f66fc1eedb0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c430 tx=0x7f66ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.616+0000 7f66eaffd640 1 -- 172.21.15.26:0/3090913762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec011790 con 0x7f66fc0c8690 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.617+0000 7f66eaffd640 1 -- 172.21.15.26:0/3090913762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec017070 con 0x7f66fc0c8690 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.617+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc0c41d0 con 0x7f66fc0c8690 2024-03-06T07:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.617+0000 7f66eaffd640 1 -- 172.21.15.26:0/3090913762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec011bb0 con 0x7f66fc0c8690 2024-03-06T07:27:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.617+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc0c2360 con 0x7f66fc0c8690 2024-03-06T07:27:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.618+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc04f330 con 0x7f66fc0c8690 2024-03-06T07:27:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.618+0000 7f66eaffd640 1 -- 172.21.15.26:0/3090913762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66ec01c460 con 0x7f66fc0c8690 2024-03-06T07:27:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.618+0000 7f66eaffd640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66dc06b4d0 0x7f66dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.619+0000 7f66eaffd640 1 -- 172.21.15.26:0/3090913762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f66ec089ee0 con 0x7f66fc0c8690 2024-03-06T07:27:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.619+0000 7f66f9d9b640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66dc06b4d0 0x7f66dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.619+0000 7f66f9d9b640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66dc06b4d0 0x7f66dc06d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0c2200 tx=0x7f66f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.623+0000 7f66eaffd640 1 -- 172.21.15.26:0/3090913762 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f66fc04f330 con 0x7f66fc0c8690 2024-03-06T07:27:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.804+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66fc069430 con 0x7f66fc0c8690 2024-03-06T07:27:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.805+0000 7f66eaffd640 1 -- 172.21.15.26:0/3090913762 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+13186 (secure 0 0 0) 0x7f66ec058f20 con 0x7f66fc0c8690 2024-03-06T07:27:29.808 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:29.808 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":180,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:29.105984+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66dc06b4d0 msgr2=0x7f66dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66dc06b4d0 0x7f66dc06d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0c2200 tx=0x7f66f0008040 comp rx=0 tx=0).stop 2024-03-06T07:27:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc0c8690 msgr2=0x7f66fc1eedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc0c8690 0x7f66fc1eedb0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c430 tx=0x7f66ec00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 shutdown_connections 2024-03-06T07:27:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f66dc06b4d0 0x7f66dc06d990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66fc158a40 0x7f66fc145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc14ba90 0x7f66fc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 --2- 172.21.15.26:0/3090913762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66fc0c8690 0x7f66fc1eedb0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 >> 172.21.15.26:0/3090913762 conn(0x7f66fc0728d0 msgr2=0x7f66fc0c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 shutdown_connections 2024-03-06T07:27:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.807+0000 7f66fb59e640 1 -- 172.21.15.26:0/3090913762 wait complete. 2024-03-06T07:27:29.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:27:29.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.923+0000 7fb92016b640 1 Processor -- start 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.923+0000 7fb92016b640 1 -- start start 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb92016b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9181536a0 0x7fb9180c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb92016b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 0x7fb91806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb92016b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9180c7430 con 0x7fb91806afc0 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb92016b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91806fde0 con 0x7fb9181536a0 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb92016b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91806b830 con 0x7fb91806b3a0 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91d706640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9181536a0 0x7fb9180c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91d706640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9181536a0 0x7fb9180c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58356/0 (socket says 172.21.15.26:58356) 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91d706640 1 -- 172.21.15.26:0/1364820455 learned_addr learned my addr 172.21.15.26:0/1364820455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91e708640 1 --2- 172.21.15.26:0/1364820455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 0x7fb91806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91d706640 1 -- 172.21.15.26:0/1364820455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 msgr2=0x7fb91806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91d706640 1 --2- 172.21.15.26:0/1364820455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 0x7fb91806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91d706640 1 -- 172.21.15.26:0/1364820455 >> v1:172.21.15.117:6789/0 conn(0x7fb91806afc0 legacy=0x7fb918152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.924+0000 7fb91d706640 1 -- 172.21.15.26:0/1364820455 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918157450 con 0x7fb9181536a0 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.925+0000 7fb91d706640 1 --2- 172.21.15.26:0/1364820455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9180c6da0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb908009870 tx=0x7fb90802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f0ad319634b8b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.925+0000 7fb91cf05640 1 -- 172.21.15.26:0/1364820455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080317c0 con 0x7fb9181536a0 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.925+0000 7fb91cf05640 1 -- 172.21.15.26:0/1364820455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb908037070 con 0x7fb9181536a0 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.925+0000 7fb91cf05640 1 -- 172.21.15.26:0/1364820455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908031b80 con 0x7fb9181536a0 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.925+0000 7fb92016b640 1 -- 172.21.15.26:0/1364820455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 msgr2=0x7fb9180c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.925+0000 7fb92016b640 1 --2- 172.21.15.26:0/1364820455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9180c6da0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb908009870 tx=0x7fb90802b860 comp rx=0 tx=0).stop 2024-03-06T07:27:29.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.926+0000 7fb92016b640 1 -- 172.21.15.26:0/1364820455 shutdown_connections 2024-03-06T07:27:29.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.926+0000 7fb92016b640 1 --2- 172.21.15.26:0/1364820455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 0x7fb91806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.926+0000 7fb92016b640 1 --2- 172.21.15.26:0/1364820455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9180c6da0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.926+0000 7fb92016b640 1 -- 172.21.15.26:0/1364820455 >> 172.21.15.26:0/1364820455 conn(0x7fb9180c4220 msgr2=0x7fb9180c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.926+0000 7fb92016b640 1 -- 172.21.15.26:0/1364820455 shutdown_connections 2024-03-06T07:27:29.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.926+0000 7fb92016b640 1 -- 172.21.15.26:0/1364820455 wait complete. 2024-03-06T07:27:29.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 Processor -- start 2024-03-06T07:27:29.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 -- start start 2024-03-06T07:27:29.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb91806afc0 0x7fb9180ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 0x7fb9180aa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9181ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918156970 con 0x7fb9181536a0 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918156fc0 con 0x7fb91806afc0 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.927+0000 7fb92016b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb918156c50 con 0x7fb91806b3a0 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9181ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9181ec5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58364/0 (socket says 172.21.15.26:58364) 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 -- 172.21.15.26:0/1965438231 learned_addr learned my addr 172.21.15.26:0/1965438231 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91df07640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb91806afc0 0x7fb9180ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 -- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 msgr2=0x7fb9180aa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 0x7fb9180aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 -- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb91806afc0 msgr2=0x7fb9180ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb91806afc0 0x7fb9180ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 -- 172.21.15.26:0/1965438231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180ab760 con 0x7fb9181536a0 2024-03-06T07:27:29.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb91e708640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9181ec5e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb910007e40 tx=0x7fb91000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb90e7fc640 1 -- 172.21.15.26:0/1965438231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910014620 con 0x7fb9181536a0 2024-03-06T07:27:29.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb90e7fc640 1 -- 172.21.15.26:0/1965438231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910019070 con 0x7fb9181536a0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9181ed0d0 con 0x7fb9181536a0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb90e7fc640 1 -- 172.21.15.26:0/1965438231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910014a20 con 0x7fb9181536a0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.928+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9180ab480 con 0x7fb9181536a0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.929+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9180aaf90 con 0x7fb9181536a0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.930+0000 7fb90e7fc640 1 -- 172.21.15.26:0/1965438231 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb910014bc0 con 0x7fb9181536a0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.930+0000 7fb90e7fc640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8fc06b500 0x7fb8fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.930+0000 7fb90e7fc640 1 -- 172.21.15.26:0/1965438231 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb91008a950 con 0x7fb9181536a0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.930+0000 7fb91df07640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8fc06b500 0x7fb8fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.931+0000 7fb91df07640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8fc06b500 0x7fb8fc06d9c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb914003d90 tx=0x7fb914009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:29.935+0000 7fb90e7fc640 1 -- 172.21.15.26:0/1965438231 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb910059990 con 0x7fb9181536a0 2024-03-06T07:27:30.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.114+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb91804f330 con 0x7fb9181536a0 2024-03-06T07:27:30.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.114+0000 7fb90e7fc640 1 -- 172.21.15.26:0/1965438231 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb910059290 con 0x7fb9181536a0 2024-03-06T07:27:30.117 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:27:30.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8fc06b500 msgr2=0x7fb8fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8fc06b500 0x7fb8fc06d9c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb914003d90 tx=0x7fb914009040 comp rx=0 tx=0).stop 2024-03-06T07:27:30.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 msgr2=0x7fb9181ec5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9181ec5e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb910007e40 tx=0x7fb91000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 shutdown_connections 2024-03-06T07:27:30.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb8fc06b500 0x7fb8fc06d9c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9181536a0 0x7fb9181ec5e0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb91806b3a0 0x7fb9180aa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 --2- 172.21.15.26:0/1965438231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb91806afc0 0x7fb9180ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 >> 172.21.15.26:0/1965438231 conn(0x7fb9180c4220 msgr2=0x7fb9180c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 shutdown_connections 2024-03-06T07:27:30.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.116+0000 7fb92016b640 1 -- 172.21.15.26:0/1965438231 wait complete. 2024-03-06T07:27:30.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:27:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.228+0000 7fbce770f640 1 Processor -- start 2024-03-06T07:27:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.228+0000 7fbce770f640 1 -- start start 2024-03-06T07:27:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.229+0000 7fbce770f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 0x7fbce0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.229+0000 7fbce770f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce00c73d0 con 0x7fbce014f360 2024-03-06T07:27:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.229+0000 7fbce770f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce00be2e0 con 0x7fbce00c88c0 2024-03-06T07:27:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.229+0000 7fbce770f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce0159700 con 0x7fbce0159270 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.229+0000 7fbce54ab640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbce014f360 0x7fbce00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57688/0 (socket says 172.21.15.26:57688) 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.229+0000 7fbce54ab640 1 -- 172.21.15.26:0/632839175 learned_addr learned my addr 172.21.15.26:0/632839175 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283850052 0 0) 0x7fbce00c73d0 con 0x7fbce014f360 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbccc003520 con 0x7fbce014f360 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce5cac640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 0x7fbce0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2856125625 0 0) 0x7fbccc003520 con 0x7fbce014f360 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 msgr2=0x7fbce0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 0x7fbce0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 >> v1:172.21.15.117:6789/0 conn(0x7fbce00c88c0 legacy=0x7fbce0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00c7670 con 0x7fbce014f360 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fbcdc001670 con 0x7fbce014f360 2024-03-06T07:27:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 >> v1:172.21.15.26:6789/0 conn(0x7fbce014f360 legacy=0x7fbce00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbccc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc004800 0x7fbccc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 0x7fbccc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce0159700 con 0x7fbccc0039e0 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc005550 con 0x7fbccc004800 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc00bf00 con 0x7fbce0159270 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce54ab640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbccc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce4caa640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc004800 0x7fbccc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce5cac640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 0x7fbccc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce54ab640 1 -- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 msgr2=0x7fbccc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce54ab640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0159270 0x7fbccc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce54ab640 1 -- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc004800 msgr2=0x7fbccc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce54ab640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc004800 0x7fbccc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.230+0000 7fbce54ab640 1 -- 172.21.15.26:0/632839175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc003520 con 0x7fbccc0039e0 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.231+0000 7fbce54ab640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbccc003e00 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c210 tx=0x7fbccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544ee2573ca190df server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.231+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc007d30 con 0x7fbccc0039e0 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.231+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc030070 con 0x7fbccc0039e0 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.231+0000 7fbcd7fff640 1 -- 172.21.15.26:0/632839175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc02cd60 con 0x7fbccc0039e0 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.232+0000 7fbce770f640 1 -- 172.21.15.26:0/632839175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 msgr2=0x7fbccc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.232+0000 7fbce770f640 1 --2- 172.21.15.26:0/632839175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbccc003e00 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c210 tx=0x7fbccc005830 comp rx=0 tx=0).stop 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.232+0000 7fbce5cac640 1 -- 172.21.15.26:0/632839175 reap_dead start 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.232+0000 7fbce770f640 1 -- 172.21.15.26:0/632839175 shutdown_connections 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.232+0000 7fbce770f640 1 -- 172.21.15.26:0/632839175 >> 172.21.15.26:0/632839175 conn(0x7fbce00ac570 msgr2=0x7fbce00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.232+0000 7fbce770f640 1 -- 172.21.15.26:0/632839175 shutdown_connections 2024-03-06T07:27:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.232+0000 7fbce770f640 1 -- 172.21.15.26:0/632839175 wait complete. 2024-03-06T07:27:30.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 Processor -- start 2024-03-06T07:27:30.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 -- start start 2024-03-06T07:27:30.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbce014bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce014bab0 0x7fbce00c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0154720 0x7fbce0154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c7c30 con 0x7fbccc0039e0 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c5a50 con 0x7fbce014bab0 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.233+0000 7fbce770f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c5bd0 con 0x7fbce0154720 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbce014bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbce014bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58394/0 (socket says 172.21.15.26:58394) 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 -- 172.21.15.26:0/3571853577 learned_addr learned my addr 172.21.15.26:0/3571853577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce5cac640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0154720 0x7fbce0154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce4caa640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce014bab0 0x7fbce00c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 -- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0154720 msgr2=0x7fbce0154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0154720 0x7fbce0154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 -- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce014bab0 msgr2=0x7fbce00c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce014bab0 0x7fbce00c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 -- 172.21.15.26:0/3571853577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00c1270 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce4caa640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce014bab0 0x7fbce00c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce54ab640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbce014bf90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000c00 tx=0x7fbcdc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbcd5ffb640 1 -- 172.21.15.26:0/3571853577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc046070 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbcd5ffb640 1 -- 172.21.15.26:0/3571853577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc004c10 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbcd5ffb640 1 -- 172.21.15.26:0/3571853577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc007d10 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.234+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce00c1550 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.235+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce01eef30 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.235+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce004f330 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.236+0000 7fbcd5ffb640 1 -- 172.21.15.26:0/3571853577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcdc007070 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.236+0000 7fbcd5ffb640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc406b500 0x7fbcc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.236+0000 7fbcd5ffb640 1 -- 172.21.15.26:0/3571853577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbcdc0ab210 con 0x7fbccc0039e0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.237+0000 7fbce4caa640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc406b500 0x7fbcc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.237+0000 7fbce4caa640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc406b500 0x7fbcc406d9c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbce006c5c0 tx=0x7fbcd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.241+0000 7fbcd5ffb640 1 -- 172.21.15.26:0/3571853577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbcdc07a3c0 con 0x7fbccc0039e0 2024-03-06T07:27:30.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.425+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbce00c8d50 con 0x7fbccc0039e0 2024-03-06T07:27:30.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.425+0000 7fbcd5ffb640 1 -- 172.21.15.26:0/3571853577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbcdc079cc0 con 0x7fbccc0039e0 2024-03-06T07:27:30.428 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:30.428 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:27:30.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:27:30.428 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:27:30.428 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:27:30.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:27:30.429 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc406b500 msgr2=0x7fbcc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc406b500 0x7fbcc406d9c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbce006c5c0 tx=0x7fbcd0009040 comp rx=0 tx=0).stop 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 msgr2=0x7fbce014bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbce014bf90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000c00 tx=0x7fbcdc001a40 comp rx=0 tx=0).stop 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 shutdown_connections 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbcc406b500 0x7fbcc406d9c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbce0154720 0x7fbce0154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbce014bab0 0x7fbce00c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 --2- 172.21.15.26:0/3571853577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc0039e0 0x7fbce014bf90 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 >> 172.21.15.26:0/3571853577 conn(0x7fbce00ac570 msgr2=0x7fbce00c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 shutdown_connections 2024-03-06T07:27:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.427+0000 7fbce770f640 1 -- 172.21.15.26:0/3571853577 wait complete. 2024-03-06T07:27:30.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:27:30.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:27:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.544+0000 7f5b8a3bc640 1 Processor -- start 2024-03-06T07:27:30.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.544+0000 7f5b8a3bc640 1 -- start start 2024-03-06T07:27:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b8a3bc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 0x7f5b840bf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b8a3bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b840c0f30 0x7f5b8406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b8a3bc640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b841554e0 con 0x7f5b840a8d70 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b8a3bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b841557c0 con 0x7f5b840c0f30 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b8a3bc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84155aa0 con 0x7f5b8414f1a0 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b837fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b840c0f30 0x7f5b8406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b837fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b840c0f30 0x7f5b8406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58404/0 (socket says 172.21.15.26:58404) 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b837fe640 1 -- 172.21.15.26:0/2045767133 learned_addr learned my addr 172.21.15.26:0/2045767133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b837fe640 1 -- 172.21.15.26:0/2045767133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 msgr2=0x7f5b840bf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b837fe640 1 --2- 172.21.15.26:0/2045767133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 0x7f5b840bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b837fe640 1 -- 172.21.15.26:0/2045767133 >> v1:172.21.15.117:6789/0 conn(0x7f5b840a8d70 legacy=0x7f5b8406d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.545+0000 7f5b837fe640 1 -- 172.21.15.26:0/2045767133 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b84155d80 con 0x7f5b840c0f30 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.546+0000 7f5b837fe640 1 --2- 172.21.15.26:0/2045767133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b8406d3d0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009870 tx=0x7f5b7402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3bcaf260248c5a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.546+0000 7f5b82ffd640 1 -- 172.21.15.26:0/2045767133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740317c0 con 0x7f5b840c0f30 2024-03-06T07:27:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.546+0000 7f5b82ffd640 1 -- 172.21.15.26:0/2045767133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74037070 con 0x7f5b840c0f30 2024-03-06T07:27:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.546+0000 7f5b82ffd640 1 -- 172.21.15.26:0/2045767133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74031b80 con 0x7f5b840c0f30 2024-03-06T07:27:30.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2045767133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 msgr2=0x7f5b8406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2045767133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b8406d3d0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009870 tx=0x7f5b7402b860 comp rx=0 tx=0).stop 2024-03-06T07:27:30.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2045767133 shutdown_connections 2024-03-06T07:27:30.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2045767133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b8406d3d0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2045767133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 0x7f5b840bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2045767133 >> 172.21.15.26:0/2045767133 conn(0x7f5b840c65b0 msgr2=0x7f5b840c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2045767133 shutdown_connections 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.547+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2045767133 wait complete. 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.548+0000 7f5b8a3bc640 1 Processor -- start 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.548+0000 7f5b8a3bc640 1 -- start start 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840a8d70 0x7f5b8414b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b841ec670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 0x7f5b841ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8406e020 con 0x7f5b840c0f30 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84156060 con 0x7f5b840a8d70 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84155120 con 0x7f5b8414f1a0 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b841ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b841ec670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58416/0 (socket says 172.21.15.26:58416) 2024-03-06T07:27:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 -- 172.21.15.26:0/2480910017 learned_addr learned my addr 172.21.15.26:0/2480910017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 -- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 msgr2=0x7f5b841ecbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 0x7f5b841ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 -- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840a8d70 msgr2=0x7f5b8414b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840a8d70 0x7f5b8414b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 -- 172.21.15.26:0/2480910017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840c54b0 con 0x7f5b840c0f30 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b837fe640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b841ec670 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009870 tx=0x7f5b740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b80ff9640 1 -- 172.21.15.26:0/2480910017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74032860 con 0x7f5b840c0f30 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b80ff9640 1 -- 172.21.15.26:0/2480910017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74032a00 con 0x7f5b840c0f30 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b841ef150 con 0x7f5b840c0f30 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b80ff9640 1 -- 172.21.15.26:0/2480910017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74032df0 con 0x7f5b840c0f30 2024-03-06T07:27:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.549+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8414bc00 con 0x7f5b840c0f30 2024-03-06T07:27:30.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.550+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b840c5b30 con 0x7f5b840c0f30 2024-03-06T07:27:30.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.551+0000 7f5b80ff9640 1 -- 172.21.15.26:0/2480910017 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b7404c060 con 0x7f5b840c0f30 2024-03-06T07:27:30.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.551+0000 7f5b80ff9640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b6406b4d0 0x7f5b6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.551+0000 7f5b80ff9640 1 -- 172.21.15.26:0/2480910017 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5b740ab900 con 0x7f5b840c0f30 2024-03-06T07:27:30.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.551+0000 7f5b83fff640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b6406b4d0 0x7f5b6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.552+0000 7f5b83fff640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b6406b4d0 0x7f5b6406d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009710 tx=0x7f5b6c006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.556+0000 7f5b80ff9640 1 -- 172.21.15.26:0/2480910017 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5b840c5b30 con 0x7f5b840c0f30 2024-03-06T07:27:30.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:30.737+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5b84153eb0 con 0x7f5b840c0f30 2024-03-06T07:27:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.116+0000 7f5b80ff9640 1 -- 172.21.15.26:0/2480910017 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5b74040070 con 0x7f5b840c0f30 2024-03-06T07:27:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:27:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b6406b4d0 msgr2=0x7f5b6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b6406b4d0 0x7f5b6406d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009710 tx=0x7f5b6c006c90 comp rx=0 tx=0).stop 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 msgr2=0x7f5b841ec670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b841ec670 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009870 tx=0x7f5b740046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 shutdown_connections 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5b6406b4d0 0x7f5b6406d990 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8414f1a0 0x7f5b841ecbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b840c0f30 0x7f5b841ec670 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 --2- 172.21.15.26:0/2480910017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840a8d70 0x7f5b8414b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.118+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 >> 172.21.15.26:0/2480910017 conn(0x7f5b840c65b0 msgr2=0x7f5b84153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.119+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 shutdown_connections 2024-03-06T07:27:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.119+0000 7f5b8a3bc640 1 -- 172.21.15.26:0/2480910017 wait complete. 2024-03-06T07:27:31.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:27:31.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.246+0000 7fe8bd01c640 1 Processor -- start 2024-03-06T07:27:31.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.246+0000 7fe8bd01c640 1 -- start start 2024-03-06T07:27:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.246+0000 7fe8bd01c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8b81529f0 0x7fe8b80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.246+0000 7fe8bd01c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c57d0 0x7fe8b8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.246+0000 7fe8bd01c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b8152dd0 con 0x7fe8b80c5370 2024-03-06T07:27:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.246+0000 7fe8bd01c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b8159710 con 0x7fe8b81529f0 2024-03-06T07:27:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.246+0000 7fe8bd01c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b8156180 con 0x7fe8b80c57d0 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8b6d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8b81529f0 0x7fe8b80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8affff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe8b80c5370 0x7fe8b8155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57700/0 (socket says 172.21.15.26:57700) 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8affff640 1 -- 172.21.15.26:0/1787363427 learned_addr learned my addr 172.21.15.26:0/1787363427 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8b759e640 1 --2- 172.21.15.26:0/1787363427 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c57d0 0x7fe8b8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8b6d9d640 1 -- 172.21.15.26:0/1787363427 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c57d0 msgr2=0x7fe8b8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8b6d9d640 1 --2- 172.21.15.26:0/1787363427 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c57d0 0x7fe8b8155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8b6d9d640 1 -- 172.21.15.26:0/1787363427 >> v1:172.21.15.26:6789/0 conn(0x7fe8b80c5370 legacy=0x7fe8b8155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.247+0000 7fe8b6d9d640 1 -- 172.21.15.26:0/1787363427 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8158b00 con 0x7fe8b81529f0 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8b6d9d640 1 --2- 172.21.15.26:0/1787363427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81529f0 0x7fe8b80c88f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80735c0 tx=0x7fe8a802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddeaf0895f19a7d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8b659c640 1 -- 172.21.15.26:0/1787363427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80315a0 con 0x7fe8b81529f0 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8b659c640 1 -- 172.21.15.26:0/1787363427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8037070 con 0x7fe8b81529f0 2024-03-06T07:27:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8b659c640 1 -- 172.21.15.26:0/1787363427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8031960 con 0x7fe8b81529f0 2024-03-06T07:27:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1787363427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81529f0 msgr2=0x7fe8b80c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1787363427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81529f0 0x7fe8b80c88f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80735c0 tx=0x7fe8a802b840 comp rx=0 tx=0).stop 2024-03-06T07:27:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1787363427 shutdown_connections 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1787363427 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c57d0 0x7fe8b8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1787363427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81529f0 0x7fe8b80c88f0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1787363427 >> 172.21.15.26:0/1787363427 conn(0x7fe8b80ac580 msgr2=0x7fe8b80ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1787363427 shutdown_connections 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.248+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1787363427 wait complete. 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 Processor -- start 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 -- start start 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c5370 0x7fe8b8158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b80c57d0 0x7fe8b8148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b81529f0 0x7fe8b814b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81590c0 con 0x7fe8b80c57d0 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80c68b0 con 0x7fe8b80c5370 2024-03-06T07:27:31.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8bd01c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81593a0 con 0x7fe8b81529f0 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b80c57d0 0x7fe8b8148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b80c57d0 0x7fe8b8148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58440/0 (socket says 172.21.15.26:58440) 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.250+0000 7fe8affff640 1 -- 172.21.15.26:0/1408825891 learned_addr learned my addr 172.21.15.26:0/1408825891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8affff640 1 -- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b81529f0 msgr2=0x7fe8b814b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8affff640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b81529f0 0x7fe8b814b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8affff640 1 -- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c5370 msgr2=0x7fe8b8158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8affff640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c5370 0x7fe8b8158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8affff640 1 -- 172.21.15.26:0/1408825891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8149690 con 0x7fe8b80c57d0 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8affff640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b80c57d0 0x7fe8b8148f90 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c400 tx=0x7fe8a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8af7fe640 1 -- 172.21.15.26:0/1408825891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0017070 con 0x7fe8b80c57d0 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8af7fe640 1 -- 172.21.15.26:0/1408825891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0005ce0 con 0x7fe8b80c57d0 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b81eab60 con 0x7fe8b80c57d0 2024-03-06T07:27:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8af7fe640 1 -- 172.21.15.26:0/1408825891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00072f0 con 0x7fe8b80c57d0 2024-03-06T07:27:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.251+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b814b770 con 0x7fe8b80c57d0 2024-03-06T07:27:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.252+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b81ea970 con 0x7fe8b80c57d0 2024-03-06T07:27:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.252+0000 7fe8af7fe640 1 -- 172.21.15.26:0/1408825891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8a001a070 con 0x7fe8b80c57d0 2024-03-06T07:27:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.253+0000 7fe8af7fe640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe89806b4d0 0x7fe89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.253+0000 7fe8af7fe640 1 -- 172.21.15.26:0/1408825891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe8a0089560 con 0x7fe8b80c57d0 2024-03-06T07:27:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.253+0000 7fe8b6d9d640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe89806b4d0 0x7fe89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.254+0000 7fe8b6d9d640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe89806b4d0 0x7fe89806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8036000 tx=0x7fe8a8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.257+0000 7fe8af7fe640 1 -- 172.21.15.26:0/1408825891 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8a00584c0 con 0x7fe8b80c57d0 2024-03-06T07:27:31.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:31.438+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe8b804f330 con 0x7fe8b80c57d0 2024-03-06T07:27:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.122+0000 7fe8af7fe640 1 -- 172.21.15.26:0/1408825891 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8a0057dc0 con 0x7fe8b80c57d0 2024-03-06T07:27:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:27:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.124+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe89806b4d0 msgr2=0x7fe89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.124+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe89806b4d0 0x7fe89806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8036000 tx=0x7fe8a8030000 comp rx=0 tx=0).stop 2024-03-06T07:27:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b80c57d0 msgr2=0x7fe8b8148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b80c57d0 0x7fe8b8148f90 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c400 tx=0x7fe8a000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 shutdown_connections 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe89806b4d0 0x7fe89806d990 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b81529f0 0x7fe8b814b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b80c57d0 0x7fe8b8148f90 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 --2- 172.21.15.26:0/1408825891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c5370 0x7fe8b8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 >> 172.21.15.26:0/1408825891 conn(0x7fe8b80ac580 msgr2=0x7fe8b8150010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 shutdown_connections 2024-03-06T07:27:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.125+0000 7fe8bd01c640 1 -- 172.21.15.26:0/1408825891 wait complete. 2024-03-06T07:27:32.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:27:32.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.257+0000 7f76d27bc640 1 Processor -- start 2024-03-06T07:27:32.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.257+0000 7f76d27bc640 1 -- start start 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76d27bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76cc152920 0x7f76cc155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76d27bc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76d27bc640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76cc156180 con 0x7f76cc0c8a00 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76d27bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76cc1587b0 con 0x7f76cc152920 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76d27bc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc159630 con 0x7f76cc0c5330 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76cc152920 0x7f76cc155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76cc152920 0x7f76cc155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58454/0 (socket says 172.21.15.26:58454) 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76cbfff640 1 -- 172.21.15.26:0/1655008288 learned_addr learned my addr 172.21.15.26:0/1655008288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76d0d59640 1 --2- 172.21.15.26:0/1655008288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76cbfff640 1 -- 172.21.15.26:0/1655008288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 msgr2=0x7f76cc0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76cbfff640 1 --2- 172.21.15.26:0/1655008288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc0c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76cbfff640 1 -- 172.21.15.26:0/1655008288 >> v1:172.21.15.117:6789/0 conn(0x7f76cc0c8a00 legacy=0x7f76cc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:32.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.258+0000 7f76cbfff640 1 -- 172.21.15.26:0/1655008288 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc0c7760 con 0x7f76cc152920 2024-03-06T07:27:32.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.259+0000 7f76cbfff640 1 --2- 172.21.15.26:0/1655008288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc152920 0x7f76cc155c40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009870 tx=0x7f76b402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e450a619c452cc33 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:32.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.259+0000 7f76caffd640 1 -- 172.21.15.26:0/1655008288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b40317c0 con 0x7f76cc152920 2024-03-06T07:27:32.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.259+0000 7f76caffd640 1 -- 172.21.15.26:0/1655008288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b4037070 con 0x7f76cc152920 2024-03-06T07:27:32.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.259+0000 7f76caffd640 1 -- 172.21.15.26:0/1655008288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4031b80 con 0x7f76cc152920 2024-03-06T07:27:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 -- 172.21.15.26:0/1655008288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc152920 msgr2=0x7f76cc155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 --2- 172.21.15.26:0/1655008288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc152920 0x7f76cc155c40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009870 tx=0x7f76b402b860 comp rx=0 tx=0).stop 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 -- 172.21.15.26:0/1655008288 shutdown_connections 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 --2- 172.21.15.26:0/1655008288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 --2- 172.21.15.26:0/1655008288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc152920 0x7f76cc155c40 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 -- 172.21.15.26:0/1655008288 >> 172.21.15.26:0/1655008288 conn(0x7f76cc0ac560 msgr2=0x7f76cc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 -- 172.21.15.26:0/1655008288 shutdown_connections 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.260+0000 7f76d27bc640 1 -- 172.21.15.26:0/1655008288 wait complete. 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.261+0000 7f76d27bc640 1 Processor -- start 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.261+0000 7f76d27bc640 1 -- start start 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d27bc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc1eae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d27bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc0c8a00 0x7f76cc0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d27bc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76cc152920 0x7f76cc157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d27bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc158a00 con 0x7f76cc0c8a00 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d27bc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc0c7d20 con 0x7f76cc152920 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d27bc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc06d030 con 0x7f76cc0c5330 2024-03-06T07:27:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc0c8a00 0x7f76cc0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc0c8a00 0x7f76cc0ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58456/0 (socket says 172.21.15.26:58456) 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 -- 172.21.15.26:0/915835771 learned_addr learned my addr 172.21.15.26:0/915835771 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cbfff640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc1eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d0d59640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76cc152920 0x7f76cc157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 -- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 msgr2=0x7f76cc1eae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc1eae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 -- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76cc152920 msgr2=0x7f76cc157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76cc152920 0x7f76cc157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76cb7fe640 1 -- 172.21.15.26:0/915835771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc0bdaf0 con 0x7f76cc0c8a00 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.262+0000 7f76d0d59640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76cc152920 0x7f76cc157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.263+0000 7f76cb7fe640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc0c8a00 0x7f76cc0ab770 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00c460 tx=0x7f76bc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.263+0000 7f76c8ff9640 1 -- 172.21.15.26:0/915835771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc016020 con 0x7f76cc0c8a00 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.263+0000 7f76c8ff9640 1 -- 172.21.15.26:0/915835771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc012870 con 0x7f76cc0c8a00 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.263+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc145ea0 con 0x7f76cc0c8a00 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.263+0000 7f76c8ff9640 1 -- 172.21.15.26:0/915835771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc012c60 con 0x7f76cc0c8a00 2024-03-06T07:27:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.263+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc0abcb0 con 0x7f76cc0c8a00 2024-03-06T07:27:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.264+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc0c7390 con 0x7f76cc0c8a00 2024-03-06T07:27:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.264+0000 7f76c8ff9640 1 -- 172.21.15.26:0/915835771 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76bc019020 con 0x7f76cc0c8a00 2024-03-06T07:27:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.265+0000 7f76c8ff9640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76ac06b4d0 0x7f76ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.265+0000 7f76c8ff9640 1 -- 172.21.15.26:0/915835771 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f76bc08a230 con 0x7f76cc0c8a00 2024-03-06T07:27:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.265+0000 7f76cbfff640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76ac06b4d0 0x7f76ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.265+0000 7f76cbfff640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76ac06b4d0 0x7f76ac06d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036000 tx=0x7f76b403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.269+0000 7f76c8ff9640 1 -- 172.21.15.26:0/915835771 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f76cc0c7390 con 0x7f76cc0c8a00 2024-03-06T07:27:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:32.491+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f76cc0c8580 con 0x7f76cc0c8a00 2024-03-06T07:27:33.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.124+0000 7f76c8ff9640 1 -- 172.21.15.26:0/915835771 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v170) v1 ==== 174+0+0 (secure 0 0 0) 0x7f76cc0c8580 con 0x7f76cc0c8a00 2024-03-06T07:27:33.126 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 29 and data pool 30 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76ac06b4d0 msgr2=0x7f76ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76ac06b4d0 0x7f76ac06d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036000 tx=0x7f76b403d040 comp rx=0 tx=0).stop 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc0c8a00 msgr2=0x7f76cc0ab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc0c8a00 0x7f76cc0ab770 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00c460 tx=0x7f76bc00c920 comp rx=0 tx=0).stop 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 shutdown_connections 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f76ac06b4d0 0x7f76ac06d990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76cc152920 0x7f76cc157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76cc0c8a00 0x7f76cc0ab770 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 --2- 172.21.15.26:0/915835771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76cc0c5330 0x7f76cc1eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 >> 172.21.15.26:0/915835771 conn(0x7f76cc0ac560 msgr2=0x7f76cc151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:33.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 shutdown_connections 2024-03-06T07:27:33.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.127+0000 7f76d27bc640 1 -- 172.21.15.26:0/915835771 wait complete. 2024-03-06T07:27:33.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:33.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.262+0000 7f289193e640 1 Processor -- start 2024-03-06T07:27:33.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.262+0000 7f289193e640 1 -- start start 2024-03-06T07:27:33.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f289193e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f288c150710 0x7f288c0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f289193e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c156f90 con 0x7f288c072720 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f289193e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c157270 con 0x7f288c150290 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f289193e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0c38d0 con 0x7f288c150710 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f288affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f288c072720 0x7f288c1564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57710/0 (socket says 172.21.15.26:57710) 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f288affd640 1 -- 172.21.15.26:0/2098921568 learned_addr learned my addr 172.21.15.26:0/2098921568 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f288b7fe640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f288c150710 0x7f288c0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703141488 0 0) 0x7f288c156f90 con 0x7f288c072720 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2878003520 con 0x7f288c072720 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084554159 0 0) 0x7f288c157270 con 0x7f288c150290 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f288c156f90 con 0x7f288c150290 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 394282066 0 0) 0x7f2878003520 con 0x7f288c072720 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f288c150710 msgr2=0x7f288c0c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:33.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f288c150710 0x7f288c0c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 >> v1:172.21.15.117:6789/0 conn(0x7f288c150290 legacy=0x7f288c0c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.263+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0c3280 con 0x7f288c072720 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f287c0019f0 con 0x7f288c072720 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 >> v1:172.21.15.26:6789/0 conn(0x7f288c072720 legacy=0x7f288c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2878003b70 0x7f2878003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2878004800 0x7f2878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28780051c0 0x7f287800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0c38d0 con 0x7f2878004800 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287800c030 con 0x7f2878003b70 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287800c230 con 0x7f28780051c0 2024-03-06T07:27:33.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288b7fe640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f288c150710 0x7f288c0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288a7fc640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2878004800 0x7f2878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288affd640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2878003b70 0x7f2878003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288b7fe640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28780051c0 0x7f287800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288a7fc640 1 -- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28780051c0 msgr2=0x7f287800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288a7fc640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28780051c0 0x7f287800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288a7fc640 1 -- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2878003b70 msgr2=0x7f2878003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288a7fc640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2878003b70 0x7f2878003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.264+0000 7f288a7fc640 1 -- 172.21.15.26:0/2098921568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2878003520 con 0x7f2878004800 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f288a7fc640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2878004800 0x7f2878004c80 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f28740018c0 tx=0x7f2878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e26e4335e061b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874037070 con 0x7f2878004800 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28740052f0 con 0x7f2878004800 2024-03-06T07:27:33.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f2889ffb640 1 -- 172.21.15.26:0/2098921568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28740014e0 con 0x7f2878004800 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f289193e640 1 -- 172.21.15.26:0/2098921568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2878004800 msgr2=0x7f2878004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f289193e640 1 --2- 172.21.15.26:0/2098921568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2878004800 0x7f2878004c80 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f28740018c0 tx=0x7f2878005830 comp rx=0 tx=0).stop 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f288b7fe640 1 -- 172.21.15.26:0/2098921568 reap_dead start 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f289193e640 1 -- 172.21.15.26:0/2098921568 shutdown_connections 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.265+0000 7f289193e640 1 -- 172.21.15.26:0/2098921568 >> 172.21.15.26:0/2098921568 conn(0x7f288c0aa630 msgr2=0x7f288c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.266+0000 7f289193e640 1 -- 172.21.15.26:0/2098921568 shutdown_connections 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.266+0000 7f289193e640 1 -- 172.21.15.26:0/2098921568 wait complete. 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 Processor -- start 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 -- start start 2024-03-06T07:27:33.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2878004800 0x7f288c0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f288c1e8960 0x7f288c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f288c0ad390 0x7f288c1e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c156bd0 con 0x7f288c1e8960 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c06f8f0 con 0x7f288c0ad390 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.267+0000 7f289193e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c156d50 con 0x7f2878004800 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f288c1e8960 0x7f288c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f288c1e8960 0x7f288c0c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58480/0 (socket says 172.21.15.26:58480) 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 -- 172.21.15.26:0/2827515507 learned_addr learned my addr 172.21.15.26:0/2827515507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288affd640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2878004800 0x7f288c0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288b7fe640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f288c0ad390 0x7f288c1e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 -- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2878004800 msgr2=0x7f288c0ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2878004800 0x7f288c0ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 -- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f288c0ad390 msgr2=0x7f288c1e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f288c0ad390 0x7f288c1e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 -- 172.21.15.26:0/2827515507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c153880 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288b7fe640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f288c0ad390 0x7f288c1e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288affd640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2878004800 0x7f288c0ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f288a7fc640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f288c1e8960 0x7f288c0c74a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2874008fd0 tx=0x7f28740012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f2883fff640 1 -- 172.21.15.26:0/2827515507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874042020 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f2883fff640 1 -- 172.21.15.26:0/2827515507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874038040 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c153b60 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f2883fff640 1 -- 172.21.15.26:0/2827515507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874037070 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.268+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c0c79e0 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.269+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c0c7cd0 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.270+0000 7f2883fff640 1 -- 172.21.15.26:0/2827515507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2874045020 con 0x7f288c1e8960 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.270+0000 7f2883fff640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f286c06b4d0 0x7f286c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.270+0000 7f2883fff640 1 -- 172.21.15.26:0/2827515507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28740abfa0 con 0x7f288c1e8960 2024-03-06T07:27:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.270+0000 7f288affd640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f286c06b4d0 0x7f286c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.271+0000 7f288affd640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f286c06b4d0 0x7f286c06d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f287c004d00 tx=0x7f287c006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.274+0000 7f2883fff640 1 -- 172.21.15.26:0/2827515507 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f288c0c7cd0 con 0x7f288c1e8960 2024-03-06T07:27:33.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.455+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f288c04f330 con 0x7f288c1e8960 2024-03-06T07:27:33.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.456+0000 7f2883fff640 1 -- 172.21.15.26:0/2827515507 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16909 (secure 0 0 0) 0x7f287404c030 con 0x7f288c1e8960 2024-03-06T07:27:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:33.459 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":183,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:32.493103+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:30.757647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:31.458902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:33.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f286c06b4d0 msgr2=0x7f286c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f286c06b4d0 0x7f286c06d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f287c004d00 tx=0x7f287c006040 comp rx=0 tx=0).stop 2024-03-06T07:27:33.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f288c1e8960 msgr2=0x7f288c0c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f288c1e8960 0x7f288c0c74a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2874008fd0 tx=0x7f28740012d0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 shutdown_connections 2024-03-06T07:27:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f286c06b4d0 0x7f286c06d990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f288c0ad390 0x7f288c1e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f288c1e8960 0x7f288c0c74a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 --2- 172.21.15.26:0/2827515507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2878004800 0x7f288c0ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 >> 172.21.15.26:0/2827515507 conn(0x7f288c0aa630 msgr2=0x7f288c14b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 shutdown_connections 2024-03-06T07:27:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.458+0000 7f289193e640 1 -- 172.21.15.26:0/2827515507 wait complete. 2024-03-06T07:27:33.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.571+0000 7ff1a1415640 1 Processor -- start 2024-03-06T07:27:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.571+0000 7ff1a1415640 1 -- start start 2024-03-06T07:27:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff1a1415640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff19c0c7380 0x7ff19c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff1a1415640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c072630 con 0x7ff19c06da40 2024-03-06T07:27:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff1a1415640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c072910 con 0x7ff19c0c6780 2024-03-06T07:27:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff1a1415640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c06de20 con 0x7ff19c0c7380 2024-03-06T07:27:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff19b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff19c0c7380 0x7ff19c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff19affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff19c06da40 0x7ff19c155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57720/0 (socket says 172.21.15.26:57720) 2024-03-06T07:27:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff19affd640 1 -- 172.21.15.26:0/3801749517 learned_addr learned my addr 172.21.15.26:0/3801749517 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff19b7fe640 1 -- 172.21.15.26:0/3801749517 >> v1:172.21.15.117:6789/0 conn(0x7ff19c0c6780 legacy=0x7ff19c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff19b7fe640 1 -- 172.21.15.26:0/3801749517 >> v1:172.21.15.26:6789/0 conn(0x7ff19c06da40 legacy=0x7ff19c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.572+0000 7ff19b7fe640 1 -- 172.21.15.26:0/3801749517 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c072bf0 con 0x7ff19c0c7380 2024-03-06T07:27:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.573+0000 7ff19b7fe640 1 --2- 172.21.15.26:0/3801749517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c0c3280 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009d90 tx=0x7ff18c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5519c5bac9e145e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.573+0000 7ff199ffb640 1 -- 172.21.15.26:0/3801749517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c039070 con 0x7ff19c0c7380 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.573+0000 7ff199ffb640 1 -- 172.21.15.26:0/3801749517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c02cc60 con 0x7ff19c0c7380 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.573+0000 7ff199ffb640 1 -- 172.21.15.26:0/3801749517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c034660 con 0x7ff19c0c7380 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.574+0000 7ff1a1415640 1 -- 172.21.15.26:0/3801749517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 msgr2=0x7ff19c0c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.574+0000 7ff1a1415640 1 --2- 172.21.15.26:0/3801749517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c0c3280 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009d90 tx=0x7ff18c02beb0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.574+0000 7ff1a1415640 1 -- 172.21.15.26:0/3801749517 shutdown_connections 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.574+0000 7ff1a1415640 1 --2- 172.21.15.26:0/3801749517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c0c3280 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.574+0000 7ff1a1415640 1 -- 172.21.15.26:0/3801749517 >> 172.21.15.26:0/3801749517 conn(0x7ff19c0aa630 msgr2=0x7ff19c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.574+0000 7ff1a1415640 1 -- 172.21.15.26:0/3801749517 shutdown_connections 2024-03-06T07:27:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.574+0000 7ff1a1415640 1 -- 172.21.15.26:0/3801749517 wait complete. 2024-03-06T07:27:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.575+0000 7ff1a1415640 1 Processor -- start 2024-03-06T07:27:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.575+0000 7ff1a1415640 1 -- start start 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1a1415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c06da40 0x7ff19c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1a1415640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0c6780 0x7ff19c14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1a1415640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c1ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1a1415640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c37c0 con 0x7ff19c0c7380 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1a1415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1594a0 con 0x7ff19c06da40 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1a1415640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c3940 con 0x7ff19c0c6780 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c1ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c1ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58510/0 (socket says 172.21.15.26:58510) 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 -- 172.21.15.26:0/1665949101 learned_addr learned my addr 172.21.15.26:0/1665949101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19affd640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c06da40 0x7ff19c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19a7fc640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0c6780 0x7ff19c14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 -- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0c6780 msgr2=0x7ff19c14ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0c6780 0x7ff19c14ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 -- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c06da40 msgr2=0x7ff19c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c06da40 0x7ff19c157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 -- 172.21.15.26:0/1665949101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c151fe0 con 0x7ff19c0c7380 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19a7fc640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0c6780 0x7ff19c14ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19affd640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c06da40 0x7ff19c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff19b7fe640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c1ee880 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009d90 tx=0x7ff18c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1937fe640 1 -- 172.21.15.26:0/1665949101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c039050 con 0x7ff19c0c7380 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.576+0000 7ff1937fe640 1 -- 172.21.15.26:0/1665949101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c045070 con 0x7ff19c0c7380 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.577+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c1522c0 con 0x7ff19c0c7380 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.577+0000 7ff1937fe640 1 -- 172.21.15.26:0/1665949101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c03e850 con 0x7ff19c0c7380 2024-03-06T07:27:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.577+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c152880 con 0x7ff19c0c7380 2024-03-06T07:27:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.577+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c04f330 con 0x7ff19c0c7380 2024-03-06T07:27:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.578+0000 7ff1937fe640 1 -- 172.21.15.26:0/1665949101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff18c03e9f0 con 0x7ff19c0c7380 2024-03-06T07:27:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.578+0000 7ff1937fe640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b4d0 0x7ff17c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.579+0000 7ff1937fe640 1 -- 172.21.15.26:0/1665949101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff18c0ad110 con 0x7ff19c0c7380 2024-03-06T07:27:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.579+0000 7ff19affd640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b4d0 0x7ff17c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.579+0000 7ff19affd640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b4d0 0x7ff17c06d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff194005880 tx=0x7ff194009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.583+0000 7ff1937fe640 1 -- 172.21.15.26:0/1665949101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff18c07beb0 con 0x7ff19c0c7380 2024-03-06T07:27:33.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.770+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff19c0c8660 con 0x7ff19c0c7380 2024-03-06T07:27:33.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.771+0000 7ff1937fe640 1 -- 172.21.15.26:0/1665949101 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16909 (secure 0 0 0) 0x7ff18c07b7b0 con 0x7ff19c0c7380 2024-03-06T07:27:33.773 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:33.774 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":183,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:32.493103+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:30.757647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:31.458902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b4d0 msgr2=0x7ff17c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b4d0 0x7ff17c06d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff194005880 tx=0x7ff194009040 comp rx=0 tx=0).stop 2024-03-06T07:27:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 msgr2=0x7ff19c1ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c1ee880 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009d90 tx=0x7ff18c004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 shutdown_connections 2024-03-06T07:27:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff17c06b4d0 0x7ff17c06d990 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19c0c7380 0x7ff19c1ee880 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff19c0c6780 0x7ff19c14ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 --2- 172.21.15.26:0/1665949101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff19c06da40 0x7ff19c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 >> 172.21.15.26:0/1665949101 conn(0x7ff19c0aa630 msgr2=0x7ff19c154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 shutdown_connections 2024-03-06T07:27:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.773+0000 7ff1a1415640 1 -- 172.21.15.26:0/1665949101 wait complete. 2024-03-06T07:27:33.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:27:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.889+0000 7fc42d9ac640 1 Processor -- start 2024-03-06T07:27:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.889+0000 7fc42d9ac640 1 -- start start 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc42d9ac640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4280c5990 0x7fc4280c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc42d9ac640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4280c3ef0 con 0x7fc42806c900 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc42d9ac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4281556d0 con 0x7fc4280ab170 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc42d9ac640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4281559b0 con 0x7fc4280c5990 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc426ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4280c5990 0x7fc4280c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc4277fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc42806c900 0x7fc4280a9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57728/0 (socket says 172.21.15.26:57728) 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc4277fe640 1 -- 172.21.15.26:0/3698946249 learned_addr learned my addr 172.21.15.26:0/3698946249 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc426ffd640 1 -- 172.21.15.26:0/3698946249 >> v1:172.21.15.117:6789/0 conn(0x7fc4280ab170 legacy=0x7fc42806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc426ffd640 1 -- 172.21.15.26:0/3698946249 >> v1:172.21.15.26:6789/0 conn(0x7fc42806c900 legacy=0x7fc4280a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc426ffd640 1 -- 172.21.15.26:0/3698946249 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428155c90 con 0x7fc4280c5990 2024-03-06T07:27:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.890+0000 7fc425ffb640 1 -- 172.21.15.26:0/3698946249 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104062296 0 0) 0x7fc4280c3ef0 con 0x7fc42806c900 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.891+0000 7fc426ffd640 1 --2- 172.21.15.26:0/3698946249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280c5990 0x7fc4280c0ab0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097d0 tx=0x7fc42002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1891800f8e974dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.891+0000 7fc425ffb640 1 -- 172.21.15.26:0/3698946249 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420035020 con 0x7fc4280c5990 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.891+0000 7fc425ffb640 1 -- 172.21.15.26:0/3698946249 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc420031830 con 0x7fc4280c5990 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.891+0000 7fc425ffb640 1 -- 172.21.15.26:0/3698946249 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420031c00 con 0x7fc4280c5990 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.891+0000 7fc42d9ac640 1 -- 172.21.15.26:0/3698946249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280c5990 msgr2=0x7fc4280c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.891+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/3698946249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280c5990 0x7fc4280c0ab0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097d0 tx=0x7fc42002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.891+0000 7fc42d9ac640 1 -- 172.21.15.26:0/3698946249 shutdown_connections 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.892+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/3698946249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280c5990 0x7fc4280c0ab0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.892+0000 7fc42d9ac640 1 -- 172.21.15.26:0/3698946249 >> 172.21.15.26:0/3698946249 conn(0x7fc4280c6710 msgr2=0x7fc4280c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.892+0000 7fc42d9ac640 1 -- 172.21.15.26:0/3698946249 shutdown_connections 2024-03-06T07:27:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.892+0000 7fc42d9ac640 1 -- 172.21.15.26:0/3698946249 wait complete. 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 Processor -- start 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 -- start start 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42806c900 0x7fc4280bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280ab170 0x7fc428148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4280c5990 0x7fc428148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc428156610 con 0x7fc4280ab170 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc428156490 con 0x7fc4280c5990 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.893+0000 7fc42d9ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280c3af0 con 0x7fc42806c900 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280ab170 0x7fc428148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280ab170 0x7fc428148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58522/0 (socket says 172.21.15.26:58522) 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 -- 172.21.15.26:0/1603190942 learned_addr learned my addr 172.21.15.26:0/1603190942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc426ffd640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42806c900 0x7fc4280bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4277fe640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4280c5990 0x7fc428148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 -- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42806c900 msgr2=0x7fc4280bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42806c900 0x7fc4280bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 -- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4280c5990 msgr2=0x7fc428148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4280c5990 0x7fc428148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 -- 172.21.15.26:0/1603190942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42814eb00 con 0x7fc4280ab170 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4277fe640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4280c5990 0x7fc428148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc4267fc640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280ab170 0x7fc428148950 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c440 tx=0x7fc41c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc40ffff640 1 -- 172.21.15.26:0/1603190942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c017070 con 0x7fc4280ab170 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc40ffff640 1 -- 172.21.15.26:0/1603190942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c0035f0 con 0x7fc4280ab170 2024-03-06T07:27:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4281549e0 con 0x7fc4280ab170 2024-03-06T07:27:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc40ffff640 1 -- 172.21.15.26:0/1603190942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0039e0 con 0x7fc4280ab170 2024-03-06T07:27:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.894+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc428071220 con 0x7fc4280ab170 2024-03-06T07:27:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.895+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc428067ad0 con 0x7fc4280ab170 2024-03-06T07:27:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.896+0000 7fc40ffff640 1 -- 172.21.15.26:0/1603190942 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc41c01a070 con 0x7fc4280ab170 2024-03-06T07:27:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.896+0000 7fc40ffff640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41406b4d0 0x7fc41406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.896+0000 7fc40ffff640 1 -- 172.21.15.26:0/1603190942 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc41c089750 con 0x7fc4280ab170 2024-03-06T07:27:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.896+0000 7fc426ffd640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41406b4d0 0x7fc41406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.897+0000 7fc426ffd640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41406b4d0 0x7fc41406d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc420036000 tx=0x7fc420008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:33.900+0000 7fc40ffff640 1 -- 172.21.15.26:0/1603190942 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc428067ad0 con 0x7fc4280ab170 2024-03-06T07:27:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.120+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc4280aa400 con 0x7fc4280ab170 2024-03-06T07:27:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.139+0000 7fc40ffff640 1 -- 172.21.15.26:0/1603190942 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v171) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc4280aa400 con 0x7fc4280ab170 2024-03-06T07:27:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41406b4d0 msgr2=0x7fc41406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41406b4d0 0x7fc41406d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc420036000 tx=0x7fc420008030 comp rx=0 tx=0).stop 2024-03-06T07:27:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280ab170 msgr2=0x7fc428148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280ab170 0x7fc428148950 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c440 tx=0x7fc41c00c900 comp rx=0 tx=0).stop 2024-03-06T07:27:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 shutdown_connections 2024-03-06T07:27:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc41406b4d0 0x7fc41406d990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4280c5990 0x7fc428148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4280ab170 0x7fc428148950 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 --2- 172.21.15.26:0/1603190942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc42806c900 0x7fc4280bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 >> 172.21.15.26:0/1603190942 conn(0x7fc4280c6710 msgr2=0x7fc42814f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 shutdown_connections 2024-03-06T07:27:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.141+0000 7fc42d9ac640 1 -- 172.21.15.26:0/1603190942 wait complete. 2024-03-06T07:27:34.155 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:27:34.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:27:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.257+0000 7fb294fd0640 1 Processor -- start 2024-03-06T07:27:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.257+0000 7fb294fd0640 1 -- start start 2024-03-06T07:27:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb294fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900c5990 0x7fb2900c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb294fd0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2900ab170 0x7fb29006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb294fd0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2900c3ef0 con 0x7fb29006c730 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb294fd0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29006cbe0 con 0x7fb2900ab170 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb294fd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901556d0 con 0x7fb2900c5990 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb287fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2900ab170 0x7fb29006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb28ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb29006c730 0x7fb2901523f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57734/0 (socket says 172.21.15.26:57734) 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb28ed9d640 1 -- 172.21.15.26:0/3607125811 learned_addr learned my addr 172.21.15.26:0/3607125811 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb28e59c640 1 --2- 172.21.15.26:0/3607125811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900c5990 0x7fb2900c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb287fff640 1 -- 172.21.15.26:0/3607125811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900c5990 msgr2=0x7fb2900c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb287fff640 1 --2- 172.21.15.26:0/3607125811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900c5990 0x7fb2900c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb287fff640 1 -- 172.21.15.26:0/3607125811 >> v1:172.21.15.26:6789/0 conn(0x7fb29006c730 legacy=0x7fb2901523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb287fff640 1 -- 172.21.15.26:0/3607125811 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2901559b0 con 0x7fb2900ab170 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.258+0000 7fb28dd9b640 1 -- 172.21.15.26:0/3607125811 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227609774 0 0) 0x7fb2900c3ef0 con 0x7fb29006c730 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.259+0000 7fb287fff640 1 --2- 172.21.15.26:0/3607125811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2900ab170 0x7fb29006c1f0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fb2780097a0 tx=0x7fb27802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcef88c67061b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.259+0000 7fb28dd9b640 1 -- 172.21.15.26:0/3607125811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780315a0 con 0x7fb2900ab170 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.259+0000 7fb28dd9b640 1 -- 172.21.15.26:0/3607125811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278037070 con 0x7fb2900ab170 2024-03-06T07:27:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.259+0000 7fb28dd9b640 1 -- 172.21.15.26:0/3607125811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278031960 con 0x7fb2900ab170 2024-03-06T07:27:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.259+0000 7fb294fd0640 1 -- 172.21.15.26:0/3607125811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2900ab170 msgr2=0x7fb29006c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.260+0000 7fb294fd0640 1 --2- 172.21.15.26:0/3607125811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2900ab170 0x7fb29006c1f0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fb2780097a0 tx=0x7fb27802b840 comp rx=0 tx=0).stop 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.260+0000 7fb294fd0640 1 -- 172.21.15.26:0/3607125811 shutdown_connections 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.260+0000 7fb294fd0640 1 --2- 172.21.15.26:0/3607125811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2900ab170 0x7fb29006c1f0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.260+0000 7fb294fd0640 1 --2- 172.21.15.26:0/3607125811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900c5990 0x7fb2900c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.260+0000 7fb294fd0640 1 -- 172.21.15.26:0/3607125811 >> 172.21.15.26:0/3607125811 conn(0x7fb2900c6710 msgr2=0x7fb2900c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.260+0000 7fb294fd0640 1 -- 172.21.15.26:0/3607125811 shutdown_connections 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.260+0000 7fb294fd0640 1 -- 172.21.15.26:0/3607125811 wait complete. 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 Processor -- start 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 -- start start 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006c730 0x7fb290147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900ab170 0x7fb290144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2900c5990 0x7fb2901452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290156490 con 0x7fb29006c730 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290155f70 con 0x7fb2900c5990 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.261+0000 7fb294fd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb290156610 con 0x7fb2900ab170 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006c730 0x7fb290147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006c730 0x7fb290147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58550/0 (socket says 172.21.15.26:58550) 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 -- 172.21.15.26:0/2378262734 learned_addr learned my addr 172.21.15.26:0/2378262734 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb287fff640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900ab170 0x7fb290144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28ed9d640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2900c5990 0x7fb2901452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 -- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900ab170 msgr2=0x7fb290144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900ab170 0x7fb290144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 -- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2900c5990 msgr2=0x7fb2901452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2900c5990 0x7fb2901452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 -- 172.21.15.26:0/2378262734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900a8280 con 0x7fb29006c730 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb28e59c640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006c730 0x7fb290147020 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e8a0 tx=0x7fb28000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb286ffd640 1 -- 172.21.15.26:0/2378262734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280017070 con 0x7fb29006c730 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb286ffd640 1 -- 172.21.15.26:0/2378262734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28000c940 con 0x7fb29006c730 2024-03-06T07:27:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb290154210 con 0x7fb29006c730 2024-03-06T07:27:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb286ffd640 1 -- 172.21.15.26:0/2378262734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28000cd30 con 0x7fb29006c730 2024-03-06T07:27:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.262+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2900a67f0 con 0x7fb29006c730 2024-03-06T07:27:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.263+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29004f330 con 0x7fb29006c730 2024-03-06T07:27:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.264+0000 7fb286ffd640 1 -- 172.21.15.26:0/2378262734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb28001a070 con 0x7fb29006c730 2024-03-06T07:27:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.264+0000 7fb286ffd640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb27006b4d0 0x7fb27006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.264+0000 7fb286ffd640 1 -- 172.21.15.26:0/2378262734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb28008a0e0 con 0x7fb29006c730 2024-03-06T07:27:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.264+0000 7fb287fff640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb27006b4d0 0x7fb27006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.265+0000 7fb287fff640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb27006b4d0 0x7fb27006d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb278036000 tx=0x7fb27803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:34.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.269+0000 7fb286ffd640 1 -- 172.21.15.26:0/2378262734 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb280058810 con 0x7fb29006c730 2024-03-06T07:27:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:34.489+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb2900c1740 con 0x7fb29006c730 2024-03-06T07:27:35.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.147+0000 7fb286ffd640 1 -- 172.21.15.26:0/2378262734 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v172) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb28005c8b0 con 0x7fb29006c730 2024-03-06T07:27:35.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.149+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb27006b4d0 msgr2=0x7fb27006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.149+0000 7fb294fd0640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb27006b4d0 0x7fb27006d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb278036000 tx=0x7fb27803d040 comp rx=0 tx=0).stop 2024-03-06T07:27:35.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.149+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006c730 msgr2=0x7fb290147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.149+0000 7fb294fd0640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006c730 0x7fb290147020 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e8a0 tx=0x7fb28000ed60 comp rx=0 tx=0).stop 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 shutdown_connections 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb27006b4d0 0x7fb27006d990 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2900c5990 0x7fb2901452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2900ab170 0x7fb290144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 --2- 172.21.15.26:0/2378262734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006c730 0x7fb290147020 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 >> 172.21.15.26:0/2378262734 conn(0x7fb2900c6710 msgr2=0x7fb29014f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 shutdown_connections 2024-03-06T07:27:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.150+0000 7fb294fd0640 1 -- 172.21.15.26:0/2378262734 wait complete. 2024-03-06T07:27:35.165 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:27:35.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:35.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.271+0000 7efdaf847640 1 Processor -- start 2024-03-06T07:27:35.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.271+0000 7efdaf847640 1 -- start start 2024-03-06T07:27:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdaf847640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efda8155c80 0x7efda806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdaf847640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda80720a0 0x7efda806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdaf847640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80c1ae0 con 0x7efda8152520 2024-03-06T07:27:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdaf847640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80c1dc0 con 0x7efda8155c80 2024-03-06T07:27:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdaf847640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda806e450 con 0x7efda80720a0 2024-03-06T07:27:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdad5e3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efda8152520 0x7efda80c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57736/0 (socket says 172.21.15.26:57736) 2024-03-06T07:27:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdad5e3640 1 -- 172.21.15.26:0/139840949 learned_addr learned my addr 172.21.15.26:0/139840949 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdacde2640 1 --2- 172.21.15.26:0/139840949 >> v2:172.21.15.117:3300/0 conn(0x7efda8155c80 0x7efda806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdadde4640 1 --2- 172.21.15.26:0/139840949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda80720a0 0x7efda806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efda7fff640 1 -- 172.21.15.26:0/139840949 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017198528 0 0) 0x7efda80c1ae0 con 0x7efda8152520 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efda7fff640 1 -- 172.21.15.26:0/139840949 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd90003520 con 0x7efda8152520 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdadde4640 1 -- 172.21.15.26:0/139840949 >> v2:172.21.15.117:3300/0 conn(0x7efda8155c80 msgr2=0x7efda806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdadde4640 1 --2- 172.21.15.26:0/139840949 >> v2:172.21.15.117:3300/0 conn(0x7efda8155c80 0x7efda806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdadde4640 1 -- 172.21.15.26:0/139840949 >> v1:172.21.15.26:6789/0 conn(0x7efda8152520 legacy=0x7efda80c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdadde4640 1 -- 172.21.15.26:0/139840949 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda806e730 con 0x7efda80720a0 2024-03-06T07:27:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.272+0000 7efdacde2640 1 --2- 172.21.15.26:0/139840949 >> v2:172.21.15.117:3300/0 conn(0x7efda8155c80 0x7efda806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.273+0000 7efdadde4640 1 --2- 172.21.15.26:0/139840949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda80720a0 0x7efda806afc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd98009d90 tx=0x7efd9802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83a67ff241d9f1d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.273+0000 7efda7fff640 1 -- 172.21.15.26:0/139840949 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98039070 con 0x7efda80720a0 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.273+0000 7efda7fff640 1 -- 172.21.15.26:0/139840949 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9802cc50 con 0x7efda80720a0 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.273+0000 7efda7fff640 1 -- 172.21.15.26:0/139840949 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98034440 con 0x7efda80720a0 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 -- 172.21.15.26:0/139840949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda80720a0 msgr2=0x7efda806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 --2- 172.21.15.26:0/139840949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda80720a0 0x7efda806afc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd98009d90 tx=0x7efd9802bea0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 -- 172.21.15.26:0/139840949 shutdown_connections 2024-03-06T07:27:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 --2- 172.21.15.26:0/139840949 >> v2:172.21.15.117:3300/0 conn(0x7efda8155c80 0x7efda806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 --2- 172.21.15.26:0/139840949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda80720a0 0x7efda806afc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 -- 172.21.15.26:0/139840949 >> 172.21.15.26:0/139840949 conn(0x7efda80c76b0 msgr2=0x7efda80c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 -- 172.21.15.26:0/139840949 shutdown_connections 2024-03-06T07:27:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.274+0000 7efdaf847640 1 -- 172.21.15.26:0/139840949 wait complete. 2024-03-06T07:27:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.275+0000 7efdaf847640 1 Processor -- start 2024-03-06T07:27:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.275+0000 7efdaf847640 1 -- start start 2024-03-06T07:27:35.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdaf847640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda80720a0 0x7efda8154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdaf847640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda8152520 0x7efda81edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdaf847640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda8155c80 0x7efda81ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdaf847640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda8152a80 con 0x7efda8152520 2024-03-06T07:27:35.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdaf847640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda8152900 con 0x7efda80720a0 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdaf847640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda806b500 con 0x7efda8155c80 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda8152520 0x7efda81edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdad5e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda80720a0 0x7efda8154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda8152520 0x7efda81edc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58562/0 (socket says 172.21.15.26:58562) 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdadde4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda8155c80 0x7efda81ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 -- 172.21.15.26:0/22100987 learned_addr learned my addr 172.21.15.26:0/22100987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 -- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda8155c80 msgr2=0x7efda81ee170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda8155c80 0x7efda81ee170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 -- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda80720a0 msgr2=0x7efda8154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda80720a0 0x7efda8154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdacde2640 1 -- 172.21.15.26:0/22100987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda8157d90 con 0x7efda8152520 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.276+0000 7efdadde4640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda8155c80 0x7efda81ee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.277+0000 7efdacde2640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda8152520 0x7efda81edc30 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00ec10 tx=0x7efd9c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.277+0000 7efda5ffb640 1 -- 172.21.15.26:0/22100987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c017070 con 0x7efda8152520 2024-03-06T07:27:35.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.277+0000 7efda5ffb640 1 -- 172.21.15.26:0/22100987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c00cc30 con 0x7efda8152520 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.277+0000 7efda5ffb640 1 -- 172.21.15.26:0/22100987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c013470 con 0x7efda8152520 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.277+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda8158070 con 0x7efda8152520 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.277+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda8155280 con 0x7efda8152520 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.278+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda804f330 con 0x7efda8152520 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.278+0000 7efda5ffb640 1 -- 172.21.15.26:0/22100987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd9c01a070 con 0x7efda8152520 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.279+0000 7efda5ffb640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9406b4d0 0x7efd9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.279+0000 7efda5ffb640 1 -- 172.21.15.26:0/22100987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd9c08a520 con 0x7efda8152520 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.279+0000 7efdad5e3640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9406b4d0 0x7efd9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.279+0000 7efdad5e3640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9406b4d0 0x7efd9406d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efda0005880 tx=0x7efda0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.283+0000 7efda5ffb640 1 -- 172.21.15.26:0/22100987 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd9c058c50 con 0x7efda8152520 2024-03-06T07:27:35.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.502+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda81e2460 con 0x7efda8152520 2024-03-06T07:27:35.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.502+0000 7efda5ffb640 1 -- 172.21.15.26:0/22100987 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+4993 (secure 0 0 0) 0x7efd9c05ccf0 con 0x7efda8152520 2024-03-06T07:27:35.505 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:35.505 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":172,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:34.491446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9867,"qdb_cluster":[9867]},"id":14}]} 2024-03-06T07:27:35.505 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 172 2024-03-06T07:27:35.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9406b4d0 msgr2=0x7efd9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9406b4d0 0x7efd9406d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efda0005880 tx=0x7efda0009040 comp rx=0 tx=0).stop 2024-03-06T07:27:35.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda8152520 msgr2=0x7efda81edc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda8152520 0x7efda81edc30 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00ec10 tx=0x7efd9c00c520 comp rx=0 tx=0).stop 2024-03-06T07:27:35.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 shutdown_connections 2024-03-06T07:27:35.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd9406b4d0 0x7efd9406d990 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda8155c80 0x7efda81ee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda8152520 0x7efda81edc30 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 --2- 172.21.15.26:0/22100987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda80720a0 0x7efda8154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 >> 172.21.15.26:0/22100987 conn(0x7efda80c76b0 msgr2=0x7efda8151e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 shutdown_connections 2024-03-06T07:27:35.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.504+0000 7efdaf847640 1 -- 172.21.15.26:0/22100987 wait complete. 2024-03-06T07:27:35.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:35.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.616+0000 7f4d65c8f640 1 Processor -- start 2024-03-06T07:27:35.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.616+0000 7f4d65c8f640 1 -- start start 2024-03-06T07:27:35.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d65c8f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c20b0 0x7f4d6006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d65c8f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d600c25c0 0x7f4d60153890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d65c8f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d60153dd0 con 0x7f4d6006afc0 2024-03-06T07:27:35.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d65c8f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d600a9530 con 0x7f4d600c25c0 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d65c8f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600a9810 con 0x7f4d600c20b0 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d600c25c0 0x7f4d60153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5effd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c20b0 0x7f4d6006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d600c25c0 0x7f4d60153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53592/0 (socket says 172.21.15.26:53592) 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5ffff640 1 -- 172.21.15.26:0/191076594 learned_addr learned my addr 172.21.15.26:0/191076594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5effd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c20b0 0x7f4d6006b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:43928/0 (socket says 172.21.15.26:43928) 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5effd640 1 -- 172.21.15.26:0/191076594 >> v2:172.21.15.117:3300/0 conn(0x7f4d600c25c0 msgr2=0x7f4d60153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5effd640 1 --2- 172.21.15.26:0/191076594 >> v2:172.21.15.117:3300/0 conn(0x7f4d600c25c0 0x7f4d60153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5effd640 1 -- 172.21.15.26:0/191076594 >> v1:172.21.15.117:6789/0 conn(0x7f4d6006afc0 legacy=0x7f4d60153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.617+0000 7f4d5effd640 1 -- 172.21.15.26:0/191076594 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600a9af0 con 0x7f4d600c20b0 2024-03-06T07:27:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.618+0000 7f4d5ffff640 1 --2- 172.21.15.26:0/191076594 >> v2:172.21.15.117:3300/0 conn(0x7f4d600c25c0 0x7f4d60153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.618+0000 7f4d5effd640 1 --2- 172.21.15.26:0/191076594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c20b0 0x7f4d6006b3a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097a0 tx=0x7f4d5402b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e63f6cf7031e74e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.618+0000 7f4d5e7fc640 1 -- 172.21.15.26:0/191076594 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54038070 con 0x7f4d600c20b0 2024-03-06T07:27:35.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.618+0000 7f4d5e7fc640 1 -- 172.21.15.26:0/191076594 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54031700 con 0x7f4d600c20b0 2024-03-06T07:27:35.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.618+0000 7f4d5e7fc640 1 -- 172.21.15.26:0/191076594 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54031ad0 con 0x7f4d600c20b0 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 -- 172.21.15.26:0/191076594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c20b0 msgr2=0x7f4d6006b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/191076594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c20b0 0x7f4d6006b3a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097a0 tx=0x7f4d5402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 -- 172.21.15.26:0/191076594 shutdown_connections 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/191076594 >> v2:172.21.15.117:3300/0 conn(0x7f4d600c25c0 0x7f4d60153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/191076594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c20b0 0x7f4d6006b3a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 -- 172.21.15.26:0/191076594 >> 172.21.15.26:0/191076594 conn(0x7f4d600c76b0 msgr2=0x7f4d600c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 -- 172.21.15.26:0/191076594 shutdown_connections 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.619+0000 7f4d65c8f640 1 -- 172.21.15.26:0/191076594 wait complete. 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.620+0000 7f4d65c8f640 1 Processor -- start 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.620+0000 7f4d65c8f640 1 -- start start 2024-03-06T07:27:35.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d65c8f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d6006afc0 0x7f4d6014cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d65c8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d600c20b0 0x7f4d601de700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d65c8f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c25c0 0x7f4d60155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d65c8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60150980 con 0x7f4d600c20b0 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d65c8f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600aa0b0 con 0x7f4d6006afc0 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d65c8f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60150b00 con 0x7f4d600c25c0 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d600c20b0 0x7f4d601de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c25c0 0x7f4d60155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d6006afc0 0x7f4d6014cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d600c20b0 0x7f4d601de700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58568/0 (socket says 172.21.15.26:58568) 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 -- 172.21.15.26:0/1230189566 learned_addr learned my addr 172.21.15.26:0/1230189566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:35.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 -- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c25c0 msgr2=0x7f4d60155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c25c0 0x7f4d60155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 -- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d6006afc0 msgr2=0x7f4d6014cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d6006afc0 0x7f4d6014cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5effd640 1 -- 172.21.15.26:0/1230189566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d601572e0 con 0x7f4d600c20b0 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5ffff640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c25c0 0x7f4d60155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.621+0000 7f4d5f7fe640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d6006afc0 0x7f4d6014cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.622+0000 7f4d5effd640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d600c20b0 0x7f4d601de700 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036040 tx=0x7f4d540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.622+0000 7f4d47fff640 1 -- 172.21.15.26:0/1230189566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54042070 con 0x7f4d600c20b0 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.622+0000 7f4d47fff640 1 -- 172.21.15.26:0/1230189566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5403d950 con 0x7f4d600c20b0 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.622+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6014d300 con 0x7f4d600c20b0 2024-03-06T07:27:35.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.622+0000 7f4d47fff640 1 -- 172.21.15.26:0/1230189566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54038040 con 0x7f4d600c20b0 2024-03-06T07:27:35.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.622+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6014b910 con 0x7f4d600c20b0 2024-03-06T07:27:35.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.623+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6006fb00 con 0x7f4d600c20b0 2024-03-06T07:27:35.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.623+0000 7f4d47fff640 1 -- 172.21.15.26:0/1230189566 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d5403b070 con 0x7f4d600c20b0 2024-03-06T07:27:35.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.624+0000 7f4d47fff640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d4c06b4d0 0x7f4d4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.624+0000 7f4d47fff640 1 -- 172.21.15.26:0/1230189566 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d540ab9b0 con 0x7f4d600c20b0 2024-03-06T07:27:35.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.624+0000 7f4d5f7fe640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d4c06b4d0 0x7f4d4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.624+0000 7f4d5f7fe640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d4c06b4d0 0x7f4d4c06d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4d50001ad0 tx=0x7f4d50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.628+0000 7f4d47fff640 1 -- 172.21.15.26:0/1230189566 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d6006fb00 con 0x7f4d600c20b0 2024-03-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.809+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d6004f330 con 0x7f4d600c20b0 2024-03-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.810+0000 7f4d47fff640 1 -- 172.21.15.26:0/1230189566 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16896 (secure 0 0 0) 0x7f4d601572e0 con 0x7f4d600c20b0 2024-03-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":184,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:34.125547+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:30.757647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:31.458902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d4c06b4d0 msgr2=0x7f4d4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d4c06b4d0 0x7f4d4c06d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4d50001ad0 tx=0x7f4d50009040 comp rx=0 tx=0).stop 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d600c20b0 msgr2=0x7f4d601de700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d600c20b0 0x7f4d601de700 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036040 tx=0x7f4d540046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 shutdown_connections 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d4c06b4d0 0x7f4d4c06d990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d600c25c0 0x7f4d60155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d600c20b0 0x7f4d601de700 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 --2- 172.21.15.26:0/1230189566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d6006afc0 0x7f4d6014cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 >> 172.21.15.26:0/1230189566 conn(0x7f4d600c76b0 msgr2=0x7f4d6006e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 shutdown_connections 2024-03-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.812+0000 7f4d65c8f640 1 -- 172.21.15.26:0/1230189566 wait complete. 2024-03-06T07:27:35.826 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:27:35.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:27:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.925+0000 7f7d0a355640 1 Processor -- start 2024-03-06T07:27:35.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.925+0000 7f7d0a355640 1 -- start start 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d0a355640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d040c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d0a355640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d04072630 con 0x7f7d040c6780 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d0a355640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d04072910 con 0x7f7d0406da40 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d0a355640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0406de20 con 0x7f7d040c7380 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d037fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7d040c6780 0x7f7d04158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57750/0 (socket says 172.21.15.26:57750) 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d037fe640 1 -- 172.21.15.26:0/669697372 learned_addr learned my addr 172.21.15.26:0/669697372 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d088f2640 1 --2- 172.21.15.26:0/669697372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d040c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d02ffd640 1 -- 172.21.15.26:0/669697372 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354486923 0 0) 0x7f7d04072630 con 0x7f7d040c6780 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d02ffd640 1 -- 172.21.15.26:0/669697372 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cec003520 con 0x7f7d040c6780 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d02ffd640 1 -- 172.21.15.26:0/669697372 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376504161 0 0) 0x7f7d04072910 con 0x7f7d0406da40 2024-03-06T07:27:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d088f2640 1 -- 172.21.15.26:0/669697372 >> v1:172.21.15.117:6789/0 conn(0x7f7d0406da40 legacy=0x7f7d04155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d088f2640 1 -- 172.21.15.26:0/669697372 >> v1:172.21.15.26:6789/0 conn(0x7f7d040c6780 legacy=0x7f7d04158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.926+0000 7f7d088f2640 1 -- 172.21.15.26:0/669697372 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04072bf0 con 0x7f7d040c7380 2024-03-06T07:27:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.927+0000 7f7d088f2640 1 --2- 172.21.15.26:0/669697372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d040c3280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009dc0 tx=0x7f7cfc02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fa34a2ef42a4329 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.927+0000 7f7d02ffd640 1 -- 172.21.15.26:0/669697372 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc039070 con 0x7f7d040c7380 2024-03-06T07:27:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.927+0000 7f7d02ffd640 1 -- 172.21.15.26:0/669697372 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc02cc40 con 0x7f7d040c7380 2024-03-06T07:27:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.927+0000 7f7d02ffd640 1 -- 172.21.15.26:0/669697372 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc034440 con 0x7f7d040c7380 2024-03-06T07:27:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.928+0000 7f7d0a355640 1 -- 172.21.15.26:0/669697372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 msgr2=0x7f7d040c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.928+0000 7f7d0a355640 1 --2- 172.21.15.26:0/669697372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d040c3280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009dc0 tx=0x7f7cfc02be90 comp rx=0 tx=0).stop 2024-03-06T07:27:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.928+0000 7f7d0a355640 1 -- 172.21.15.26:0/669697372 shutdown_connections 2024-03-06T07:27:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.928+0000 7f7d0a355640 1 --2- 172.21.15.26:0/669697372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d040c3280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.928+0000 7f7d0a355640 1 -- 172.21.15.26:0/669697372 >> 172.21.15.26:0/669697372 conn(0x7f7d040aa630 msgr2=0x7f7d040aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.928+0000 7f7d0a355640 1 -- 172.21.15.26:0/669697372 shutdown_connections 2024-03-06T07:27:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.928+0000 7f7d0a355640 1 -- 172.21.15.26:0/669697372 wait complete. 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.929+0000 7f7d0a355640 1 Processor -- start 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.929+0000 7f7d0a355640 1 -- start start 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d0a355640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0406da40 0x7f7d04152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d0a355640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040c6780 0x7f7d041edd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d0a355640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d0414f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d0a355640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040c37c0 con 0x7f7d040c6780 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d0a355640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041594a0 con 0x7f7d0406da40 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d0a355640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040c3940 con 0x7f7d040c7380 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040c6780 0x7f7d041edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040c6780 0x7f7d041edd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58584/0 (socket says 172.21.15.26:58584) 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 -- 172.21.15.26:0/20605313 learned_addr learned my addr 172.21.15.26:0/20605313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d03fff640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0406da40 0x7f7d04152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d088f2640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d0414f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 -- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 msgr2=0x7f7d0414f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d0414f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 -- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0406da40 msgr2=0x7f7d04152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0406da40 0x7f7d04152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 -- 172.21.15.26:0/20605313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d041519c0 con 0x7f7d040c6780 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d088f2640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d0414f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d037fe640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040c6780 0x7f7d041edd70 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c390 tx=0x7f7cf800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d00ff9640 1 -- 172.21.15.26:0/20605313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8017070 con 0x7f7d040c6780 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.930+0000 7f7d00ff9640 1 -- 172.21.15.26:0/20605313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8005ce0 con 0x7f7d040c6780 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.931+0000 7f7d00ff9640 1 -- 172.21.15.26:0/20605313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80072f0 con 0x7f7d040c6780 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.931+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d04152e30 con 0x7f7d040c6780 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.931+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d040ae880 con 0x7f7d040c6780 2024-03-06T07:27:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.932+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d04075300 con 0x7f7d040c6780 2024-03-06T07:27:35.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.932+0000 7f7d00ff9640 1 -- 172.21.15.26:0/20605313 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cf801a070 con 0x7f7d040c6780 2024-03-06T07:27:35.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.932+0000 7f7d00ff9640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cf006b500 0x7f7cf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.933+0000 7f7d00ff9640 1 -- 172.21.15.26:0/20605313 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7cf8089600 con 0x7f7d040c6780 2024-03-06T07:27:35.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.933+0000 7f7d03fff640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cf006b500 0x7f7cf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.933+0000 7f7d03fff640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cf006b500 0x7f7cf006d9c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4000f60 tx=0x7f7cf4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:35.937+0000 7f7d00ff9640 1 -- 172.21.15.26:0/20605313 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d04075300 con 0x7f7d040c6780 2024-03-06T07:27:36.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.114+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d0404f330 con 0x7f7cf006b500 2024-03-06T07:27:36.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.121+0000 7f7d00ff9640 1 -- 172.21.15.26:0/20605313 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252517 (secure 0 0 0) 0x7f7d0404f330 con 0x7f7cf006b500 2024-03-06T07:27:36.125 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:27:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cf006b500 msgr2=0x7f7cf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cf006b500 0x7f7cf006d9c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4000f60 tx=0x7f7cf4009040 comp rx=0 tx=0).stop 2024-03-06T07:27:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040c6780 msgr2=0x7f7d041edd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040c6780 0x7f7d041edd70 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c390 tx=0x7f7cf800c850 comp rx=0 tx=0).stop 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 shutdown_connections 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7cf006b500 0x7f7cf006d9c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c7380 0x7f7d0414f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040c6780 0x7f7d041edd70 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 --2- 172.21.15.26:0/20605313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0406da40 0x7f7d04152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.123+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 >> 172.21.15.26:0/20605313 conn(0x7f7d040aa630 msgr2=0x7f7d040aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.124+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 shutdown_connections 2024-03-06T07:27:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.124+0000 7f7d0a355640 1 -- 172.21.15.26:0/20605313 wait complete. 2024-03-06T07:27:36.137 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-03-06T07:27:35.126252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267804,"kb_used_data":18328,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754706916,"statfs":{"total":773094113280,"available":772819881984,"internally_reserved":0,"allocated":18767872,"data_stored":10217540,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.005994"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.158179+0000","last_change":"2024-03-06T07:27:33.155279+0000","last_active":"2024-03-06T07:27:33.158179+0000","last_peered":"2024-03-06T07:27:33.158179+0000","last_clean":"2024-03-06T07:27:33.158179+0000","last_became_active":"2024-03-06T07:27:33.155013+0000","last_became_peered":"2024-03-06T07:27:33.155013+0000","last_unstale":"2024-03-06T07:27:33.158179+0000","last_undegraded":"2024-03-06T07:27:33.158179+0000","last_fullsized":"2024-03-06T07:27:33.158179+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:20.914191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.154874+0000","last_change":"2024-03-06T07:27:33.151127+0000","last_active":"2024-03-06T07:27:33.154874+0000","last_peered":"2024-03-06T07:27:33.154874+0000","last_clean":"2024-03-06T07:27:33.154874+0000","last_became_active":"2024-03-06T07:27:33.150121+0000","last_became_peered":"2024-03-06T07:27:33.150121+0000","last_unstale":"2024-03-06T07:27:33.154874+0000","last_undegraded":"2024-03-06T07:27:33.154874+0000","last_fullsized":"2024-03-06T07:27:33.154874+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:29:45.203419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128633+0000","last_change":"2024-03-06T07:27:33.149107+0000","last_active":"2024-03-06T07:27:34.128633+0000","last_peered":"2024-03-06T07:27:34.128633+0000","last_clean":"2024-03-06T07:27:34.128633+0000","last_became_active":"2024-03-06T07:27:33.148836+0000","last_became_peered":"2024-03-06T07:27:33.148836+0000","last_unstale":"2024-03-06T07:27:34.128633+0000","last_undegraded":"2024-03-06T07:27:34.128633+0000","last_fullsized":"2024-03-06T07:27:34.128633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:27:33.192746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129802+0000","last_change":"2024-03-06T07:27:33.153052+0000","last_active":"2024-03-06T07:27:34.129802+0000","last_peered":"2024-03-06T07:27:34.129802+0000","last_clean":"2024-03-06T07:27:34.129802+0000","last_became_active":"2024-03-06T07:27:33.152766+0000","last_became_peered":"2024-03-06T07:27:33.152766+0000","last_unstale":"2024-03-06T07:27:34.129802+0000","last_undegraded":"2024-03-06T07:27:34.129802+0000","last_fullsized":"2024-03-06T07:27:34.129802+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:18.026072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.148352+0000","last_change":"2024-03-06T07:27:33.140305+0000","last_active":"2024-03-06T07:27:33.148352+0000","last_peered":"2024-03-06T07:27:33.148352+0000","last_clean":"2024-03-06T07:27:33.148352+0000","last_became_active":"2024-03-06T07:27:33.139973+0000","last_became_peered":"2024-03-06T07:27:33.139973+0000","last_unstale":"2024-03-06T07:27:33.148352+0000","last_undegraded":"2024-03-06T07:27:33.148352+0000","last_fullsized":"2024-03-06T07:27:33.148352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:36.868705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160157+0000","last_change":"2024-03-06T07:27:33.152163+0000","last_active":"2024-03-06T07:27:33.160157+0000","last_peered":"2024-03-06T07:27:33.160157+0000","last_clean":"2024-03-06T07:27:33.160157+0000","last_became_active":"2024-03-06T07:27:33.151314+0000","last_became_peered":"2024-03-06T07:27:33.151314+0000","last_unstale":"2024-03-06T07:27:33.160157+0000","last_undegraded":"2024-03-06T07:27:33.160157+0000","last_fullsized":"2024-03-06T07:27:33.160157+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:29.868915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128434+0000","last_change":"2024-03-06T07:27:33.148746+0000","last_active":"2024-03-06T07:27:34.128434+0000","last_peered":"2024-03-06T07:27:34.128434+0000","last_clean":"2024-03-06T07:27:34.128434+0000","last_became_active":"2024-03-06T07:27:33.148443+0000","last_became_peered":"2024-03-06T07:27:33.148443+0000","last_unstale":"2024-03-06T07:27:34.128434+0000","last_undegraded":"2024-03-06T07:27:34.128434+0000","last_fullsized":"2024-03-06T07:27:34.128434+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:51:59.786014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160230+0000","last_change":"2024-03-06T07:27:33.152080+0000","last_active":"2024-03-06T07:27:33.160230+0000","last_peered":"2024-03-06T07:27:33.160230+0000","last_clean":"2024-03-06T07:27:33.160230+0000","last_became_active":"2024-03-06T07:27:33.151379+0000","last_became_peered":"2024-03-06T07:27:33.151379+0000","last_unstale":"2024-03-06T07:27:33.160230+0000","last_undegraded":"2024-03-06T07:27:33.160230+0000","last_fullsized":"2024-03-06T07:27:33.160230+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:38.325184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.136517+0000","last_change":"2024-03-06T07:27:33.135277+0000","last_active":"2024-03-06T07:27:33.136517+0000","last_peered":"2024-03-06T07:27:33.136517+0000","last_clean":"2024-03-06T07:27:33.136517+0000","last_became_active":"2024-03-06T07:27:33.134995+0000","last_became_peered":"2024-03-06T07:27:33.134995+0000","last_unstale":"2024-03-06T07:27:33.136517+0000","last_undegraded":"2024-03-06T07:27:33.136517+0000","last_fullsized":"2024-03-06T07:27:33.136517+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:44:17.685958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128677+0000","last_change":"2024-03-06T07:27:33.151543+0000","last_active":"2024-03-06T07:27:34.128677+0000","last_peered":"2024-03-06T07:27:34.128677+0000","last_clean":"2024-03-06T07:27:34.128677+0000","last_became_active":"2024-03-06T07:27:33.151261+0000","last_became_peered":"2024-03-06T07:27:33.151261+0000","last_unstale":"2024-03-06T07:27:34.128677+0000","last_undegraded":"2024-03-06T07:27:34.128677+0000","last_fullsized":"2024-03-06T07:27:34.128677+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:55.803752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129917+0000","last_change":"2024-03-06T07:27:33.148963+0000","last_active":"2024-03-06T07:27:34.129917+0000","last_peered":"2024-03-06T07:27:34.129917+0000","last_clean":"2024-03-06T07:27:34.129917+0000","last_became_active":"2024-03-06T07:27:33.148670+0000","last_became_peered":"2024-03-06T07:27:33.148670+0000","last_unstale":"2024-03-06T07:27:34.129917+0000","last_undegraded":"2024-03-06T07:27:34.129917+0000","last_fullsized":"2024-03-06T07:27:34.129917+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:55:18.198469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.168805+0000","last_change":"2024-03-06T07:27:33.164228+0000","last_active":"2024-03-06T07:27:33.168805+0000","last_peered":"2024-03-06T07:27:33.168805+0000","last_clean":"2024-03-06T07:27:33.168805+0000","last_became_active":"2024-03-06T07:27:33.163971+0000","last_became_peered":"2024-03-06T07:27:33.163971+0000","last_unstale":"2024-03-06T07:27:33.168805+0000","last_undegraded":"2024-03-06T07:27:33.168805+0000","last_fullsized":"2024-03-06T07:27:33.168805+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:18.383123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157404+0000","last_change":"2024-03-06T07:27:33.151226+0000","last_active":"2024-03-06T07:27:33.157404+0000","last_peered":"2024-03-06T07:27:33.157404+0000","last_clean":"2024-03-06T07:27:33.157404+0000","last_became_active":"2024-03-06T07:27:33.150917+0000","last_became_peered":"2024-03-06T07:27:33.150917+0000","last_unstale":"2024-03-06T07:27:33.157404+0000","last_undegraded":"2024-03-06T07:27:33.157404+0000","last_fullsized":"2024-03-06T07:27:33.157404+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:04.973457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163976+0000","last_change":"2024-03-06T07:27:33.159390+0000","last_active":"2024-03-06T07:27:33.163976+0000","last_peered":"2024-03-06T07:27:33.163976+0000","last_clean":"2024-03-06T07:27:33.163976+0000","last_became_active":"2024-03-06T07:27:33.158040+0000","last_became_peered":"2024-03-06T07:27:33.158040+0000","last_unstale":"2024-03-06T07:27:33.163976+0000","last_undegraded":"2024-03-06T07:27:33.163976+0000","last_fullsized":"2024-03-06T07:27:33.163976+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:14:13.257345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147917+0000","last_change":"2024-03-06T07:27:33.139936+0000","last_active":"2024-03-06T07:27:33.147917+0000","last_peered":"2024-03-06T07:27:33.147917+0000","last_clean":"2024-03-06T07:27:33.147917+0000","last_became_active":"2024-03-06T07:27:33.139659+0000","last_became_peered":"2024-03-06T07:27:33.139659+0000","last_unstale":"2024-03-06T07:27:33.147917+0000","last_undegraded":"2024-03-06T07:27:33.147917+0000","last_fullsized":"2024-03-06T07:27:33.147917+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:11.996894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157613+0000","last_change":"2024-03-06T07:27:33.151729+0000","last_active":"2024-03-06T07:27:33.157613+0000","last_peered":"2024-03-06T07:27:33.157613+0000","last_clean":"2024-03-06T07:27:33.157613+0000","last_became_active":"2024-03-06T07:27:33.151035+0000","last_became_peered":"2024-03-06T07:27:33.151035+0000","last_unstale":"2024-03-06T07:27:33.157613+0000","last_undegraded":"2024-03-06T07:27:33.157613+0000","last_fullsized":"2024-03-06T07:27:33.157613+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:00.743621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164138+0000","last_change":"2024-03-06T07:27:33.159283+0000","last_active":"2024-03-06T07:27:33.164138+0000","last_peered":"2024-03-06T07:27:33.164138+0000","last_clean":"2024-03-06T07:27:33.164138+0000","last_became_active":"2024-03-06T07:27:33.157565+0000","last_became_peered":"2024-03-06T07:27:33.157565+0000","last_unstale":"2024-03-06T07:27:33.164138+0000","last_undegraded":"2024-03-06T07:27:33.164138+0000","last_fullsized":"2024-03-06T07:27:33.164138+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:21:59.582658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128725+0000","last_change":"2024-03-06T07:27:33.146830+0000","last_active":"2024-03-06T07:27:34.128725+0000","last_peered":"2024-03-06T07:27:34.128725+0000","last_clean":"2024-03-06T07:27:34.128725+0000","last_became_active":"2024-03-06T07:27:33.146567+0000","last_became_peered":"2024-03-06T07:27:33.146567+0000","last_unstale":"2024-03-06T07:27:34.128725+0000","last_undegraded":"2024-03-06T07:27:34.128725+0000","last_fullsized":"2024-03-06T07:27:34.128725+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:49.961714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152281+0000","last_change":"2024-03-06T07:27:33.149381+0000","last_active":"2024-03-06T07:27:33.152281+0000","last_peered":"2024-03-06T07:27:33.152281+0000","last_clean":"2024-03-06T07:27:33.152281+0000","last_became_active":"2024-03-06T07:27:33.149111+0000","last_became_peered":"2024-03-06T07:27:33.149111+0000","last_unstale":"2024-03-06T07:27:33.152281+0000","last_undegraded":"2024-03-06T07:27:33.152281+0000","last_fullsized":"2024-03-06T07:27:33.152281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:58.261673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160124+0000","last_change":"2024-03-06T07:27:33.152075+0000","last_active":"2024-03-06T07:27:33.160124+0000","last_peered":"2024-03-06T07:27:33.160124+0000","last_clean":"2024-03-06T07:27:33.160124+0000","last_became_active":"2024-03-06T07:27:33.151788+0000","last_became_peered":"2024-03-06T07:27:33.151788+0000","last_unstale":"2024-03-06T07:27:33.160124+0000","last_undegraded":"2024-03-06T07:27:33.160124+0000","last_fullsized":"2024-03-06T07:27:33.160124+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:39.354949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128441+0000","last_change":"2024-03-06T07:27:33.146275+0000","last_active":"2024-03-06T07:27:34.128441+0000","last_peered":"2024-03-06T07:27:34.128441+0000","last_clean":"2024-03-06T07:27:34.128441+0000","last_became_active":"2024-03-06T07:27:33.145953+0000","last_became_peered":"2024-03-06T07:27:33.145953+0000","last_unstale":"2024-03-06T07:27:34.128441+0000","last_undegraded":"2024-03-06T07:27:34.128441+0000","last_fullsized":"2024-03-06T07:27:34.128441+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:51:20.509542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.155357+0000","last_change":"2024-03-06T07:27:33.154623+0000","last_active":"2024-03-06T07:27:33.155357+0000","last_peered":"2024-03-06T07:27:33.155357+0000","last_clean":"2024-03-06T07:27:33.155357+0000","last_became_active":"2024-03-06T07:27:33.154316+0000","last_became_peered":"2024-03-06T07:27:33.154316+0000","last_unstale":"2024-03-06T07:27:33.155357+0000","last_undegraded":"2024-03-06T07:27:33.155357+0000","last_fullsized":"2024-03-06T07:27:33.155357+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:58.261673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.135669+0000","last_change":"2024-03-06T07:27:33.134532+0000","last_active":"2024-03-06T07:27:33.135669+0000","last_peered":"2024-03-06T07:27:33.135669+0000","last_clean":"2024-03-06T07:27:33.135669+0000","last_became_active":"2024-03-06T07:27:33.134257+0000","last_became_peered":"2024-03-06T07:27:33.134257+0000","last_unstale":"2024-03-06T07:27:33.135669+0000","last_undegraded":"2024-03-06T07:27:33.135669+0000","last_fullsized":"2024-03-06T07:27:33.135669+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:02:34.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.149729+0000","last_change":"2024-03-06T07:27:33.143532+0000","last_active":"2024-03-06T07:27:33.149729+0000","last_peered":"2024-03-06T07:27:33.149729+0000","last_clean":"2024-03-06T07:27:33.149729+0000","last_became_active":"2024-03-06T07:27:33.143251+0000","last_became_peered":"2024-03-06T07:27:33.143251+0000","last_unstale":"2024-03-06T07:27:33.149729+0000","last_undegraded":"2024-03-06T07:27:33.149729+0000","last_fullsized":"2024-03-06T07:27:33.149729+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:35.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.149832+0000","last_change":"2024-03-06T07:27:33.143491+0000","last_active":"2024-03-06T07:27:33.149832+0000","last_peered":"2024-03-06T07:27:33.149832+0000","last_clean":"2024-03-06T07:27:33.149832+0000","last_became_active":"2024-03-06T07:27:33.143131+0000","last_became_peered":"2024-03-06T07:27:33.143131+0000","last_unstale":"2024-03-06T07:27:33.149832+0000","last_undegraded":"2024-03-06T07:27:33.149832+0000","last_fullsized":"2024-03-06T07:27:33.149832+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:11.996894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128772+0000","last_change":"2024-03-06T07:27:33.158732+0000","last_active":"2024-03-06T07:27:34.128772+0000","last_peered":"2024-03-06T07:27:34.128772+0000","last_clean":"2024-03-06T07:27:34.128772+0000","last_became_active":"2024-03-06T07:27:33.158491+0000","last_became_peered":"2024-03-06T07:27:33.158491+0000","last_unstale":"2024-03-06T07:27:34.128772+0000","last_undegraded":"2024-03-06T07:27:34.128772+0000","last_fullsized":"2024-03-06T07:27:34.128772+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:02.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129918+0000","last_change":"2024-03-06T07:27:33.146057+0000","last_active":"2024-03-06T07:27:34.129918+0000","last_peered":"2024-03-06T07:27:34.129918+0000","last_clean":"2024-03-06T07:27:34.129918+0000","last_became_active":"2024-03-06T07:27:33.145784+0000","last_became_peered":"2024-03-06T07:27:33.145784+0000","last_unstale":"2024-03-06T07:27:34.129918+0000","last_undegraded":"2024-03-06T07:27:34.129918+0000","last_fullsized":"2024-03-06T07:27:34.129918+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:57:03.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164369+0000","last_change":"2024-03-06T07:27:33.159871+0000","last_active":"2024-03-06T07:27:33.164369+0000","last_peered":"2024-03-06T07:27:33.164369+0000","last_clean":"2024-03-06T07:27:33.164369+0000","last_became_active":"2024-03-06T07:27:33.159570+0000","last_became_peered":"2024-03-06T07:27:33.159570+0000","last_unstale":"2024-03-06T07:27:33.164369+0000","last_undegraded":"2024-03-06T07:27:33.164369+0000","last_fullsized":"2024-03-06T07:27:33.164369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:56:32.954518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.154246+0000","last_change":"2024-03-06T07:27:33.149076+0000","last_active":"2024-03-06T07:27:33.154246+0000","last_peered":"2024-03-06T07:27:33.154246+0000","last_clean":"2024-03-06T07:27:33.154246+0000","last_became_active":"2024-03-06T07:27:33.148391+0000","last_became_peered":"2024-03-06T07:27:33.148391+0000","last_unstale":"2024-03-06T07:27:33.154246+0000","last_undegraded":"2024-03-06T07:27:33.154246+0000","last_fullsized":"2024-03-06T07:27:33.154246+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:39.410877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163043+0000","last_change":"2024-03-06T07:27:33.158523+0000","last_active":"2024-03-06T07:27:33.163043+0000","last_peered":"2024-03-06T07:27:33.163043+0000","last_clean":"2024-03-06T07:27:33.163043+0000","last_became_active":"2024-03-06T07:27:33.158268+0000","last_became_peered":"2024-03-06T07:27:33.158268+0000","last_unstale":"2024-03-06T07:27:33.163043+0000","last_undegraded":"2024-03-06T07:27:33.163043+0000","last_fullsized":"2024-03-06T07:27:33.163043+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:40.955624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152756+0000","last_change":"2024-03-06T07:27:33.151064+0000","last_active":"2024-03-06T07:27:33.152756+0000","last_peered":"2024-03-06T07:27:33.152756+0000","last_clean":"2024-03-06T07:27:33.152756+0000","last_became_active":"2024-03-06T07:27:33.150737+0000","last_became_peered":"2024-03-06T07:27:33.150737+0000","last_unstale":"2024-03-06T07:27:33.152756+0000","last_undegraded":"2024-03-06T07:27:33.152756+0000","last_fullsized":"2024-03-06T07:27:33.152756+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:18:50.213880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163571+0000","last_change":"2024-03-06T07:27:33.161368+0000","last_active":"2024-03-06T07:27:33.163571+0000","last_peered":"2024-03-06T07:27:33.163571+0000","last_clean":"2024-03-06T07:27:33.163571+0000","last_became_active":"2024-03-06T07:27:33.161117+0000","last_became_peered":"2024-03-06T07:27:33.161117+0000","last_unstale":"2024-03-06T07:27:33.163571+0000","last_undegraded":"2024-03-06T07:27:33.163571+0000","last_fullsized":"2024-03-06T07:27:33.163571+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:30.252166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128482+0000","last_change":"2024-03-06T07:27:33.152683+0000","last_active":"2024-03-06T07:27:34.128482+0000","last_peered":"2024-03-06T07:27:34.128482+0000","last_clean":"2024-03-06T07:27:34.128482+0000","last_became_active":"2024-03-06T07:27:33.152403+0000","last_became_peered":"2024-03-06T07:27:33.152403+0000","last_unstale":"2024-03-06T07:27:34.128482+0000","last_undegraded":"2024-03-06T07:27:34.128482+0000","last_fullsized":"2024-03-06T07:27:34.128482+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:12.871068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"183'4","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152066+0000","last_change":"2024-03-06T07:27:32.142861+0000","last_active":"2024-03-06T07:27:33.152066+0000","last_peered":"2024-03-06T07:27:33.152066+0000","last_clean":"2024-03-06T07:27:33.152066+0000","last_became_active":"2024-03-06T07:27:32.142557+0000","last_became_peered":"2024-03-06T07:27:32.142557+0000","last_unstale":"2024-03-06T07:27:33.152066+0000","last_undegraded":"2024-03-06T07:27:33.152066+0000","last_fullsized":"2024-03-06T07:27:33.152066+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.042398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128621+0000","last_change":"2024-03-06T07:27:33.138462+0000","last_active":"2024-03-06T07:27:34.128621+0000","last_peered":"2024-03-06T07:27:34.128621+0000","last_clean":"2024-03-06T07:27:34.128621+0000","last_became_active":"2024-03-06T07:27:33.138144+0000","last_became_peered":"2024-03-06T07:27:33.138144+0000","last_unstale":"2024-03-06T07:27:34.128621+0000","last_undegraded":"2024-03-06T07:27:34.128621+0000","last_fullsized":"2024-03-06T07:27:34.128621+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:16.293356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161522+0000","last_change":"2024-03-06T07:27:32.141407+0000","last_active":"2024-03-06T07:27:33.161522+0000","last_peered":"2024-03-06T07:27:33.161522+0000","last_clean":"2024-03-06T07:27:33.161522+0000","last_became_active":"2024-03-06T07:27:32.141083+0000","last_became_peered":"2024-03-06T07:27:32.141083+0000","last_unstale":"2024-03-06T07:27:33.161522+0000","last_undegraded":"2024-03-06T07:27:33.161522+0000","last_fullsized":"2024-03-06T07:27:33.161522+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:23.040151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161670+0000","last_change":"2024-03-06T07:27:33.157812+0000","last_active":"2024-03-06T07:27:33.161670+0000","last_peered":"2024-03-06T07:27:33.161670+0000","last_clean":"2024-03-06T07:27:33.161670+0000","last_became_active":"2024-03-06T07:27:33.157520+0000","last_became_peered":"2024-03-06T07:27:33.157520+0000","last_unstale":"2024-03-06T07:27:33.161670+0000","last_undegraded":"2024-03-06T07:27:33.161670+0000","last_fullsized":"2024-03-06T07:27:33.161670+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:55.161831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162142+0000","last_change":"2024-03-06T07:27:32.146210+0000","last_active":"2024-03-06T07:27:33.162142+0000","last_peered":"2024-03-06T07:27:33.162142+0000","last_clean":"2024-03-06T07:27:33.162142+0000","last_became_active":"2024-03-06T07:27:32.145911+0000","last_became_peered":"2024-03-06T07:27:32.145911+0000","last_unstale":"2024-03-06T07:27:33.162142+0000","last_undegraded":"2024-03-06T07:27:33.162142+0000","last_fullsized":"2024-03-06T07:27:33.162142+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:02:59.864073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128317+0000","last_change":"2024-03-06T07:27:33.152715+0000","last_active":"2024-03-06T07:27:34.128317+0000","last_peered":"2024-03-06T07:27:34.128317+0000","last_clean":"2024-03-06T07:27:34.128317+0000","last_became_active":"2024-03-06T07:27:33.152040+0000","last_became_peered":"2024-03-06T07:27:33.152040+0000","last_unstale":"2024-03-06T07:27:34.128317+0000","last_undegraded":"2024-03-06T07:27:34.128317+0000","last_fullsized":"2024-03-06T07:27:34.128317+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:08.820948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157969+0000","last_change":"2024-03-06T07:27:32.146403+0000","last_active":"2024-03-06T07:27:33.157969+0000","last_peered":"2024-03-06T07:27:33.157969+0000","last_clean":"2024-03-06T07:27:33.157969+0000","last_became_active":"2024-03-06T07:27:32.146175+0000","last_became_peered":"2024-03-06T07:27:32.146175+0000","last_unstale":"2024-03-06T07:27:33.157969+0000","last_undegraded":"2024-03-06T07:27:33.157969+0000","last_fullsized":"2024-03-06T07:27:33.157969+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:15.618377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164370+0000","last_change":"2024-03-06T07:27:33.160177+0000","last_active":"2024-03-06T07:27:33.164370+0000","last_peered":"2024-03-06T07:27:33.164370+0000","last_clean":"2024-03-06T07:27:33.164370+0000","last_became_active":"2024-03-06T07:27:33.159892+0000","last_became_peered":"2024-03-06T07:27:33.159892+0000","last_unstale":"2024-03-06T07:27:33.164370+0000","last_undegraded":"2024-03-06T07:27:33.164370+0000","last_fullsized":"2024-03-06T07:27:33.164370+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:06:30.834522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.155770+0000","last_change":"2024-03-06T07:27:32.142277+0000","last_active":"2024-03-06T07:27:33.155770+0000","last_peered":"2024-03-06T07:27:33.155770+0000","last_clean":"2024-03-06T07:27:33.155770+0000","last_became_active":"2024-03-06T07:27:32.142006+0000","last_became_peered":"2024-03-06T07:27:32.142006+0000","last_unstale":"2024-03-06T07:27:33.155770+0000","last_undegraded":"2024-03-06T07:27:33.155770+0000","last_fullsized":"2024-03-06T07:27:33.155770+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:52.574912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147968+0000","last_change":"2024-03-06T07:27:33.145176+0000","last_active":"2024-03-06T07:27:33.147968+0000","last_peered":"2024-03-06T07:27:33.147968+0000","last_clean":"2024-03-06T07:27:33.147968+0000","last_became_active":"2024-03-06T07:27:33.144880+0000","last_became_peered":"2024-03-06T07:27:33.144880+0000","last_unstale":"2024-03-06T07:27:33.147968+0000","last_undegraded":"2024-03-06T07:27:33.147968+0000","last_fullsized":"2024-03-06T07:27:33.147968+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:35.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129417+0000","last_change":"2024-03-06T07:27:32.143984+0000","last_active":"2024-03-06T07:27:34.129417+0000","last_peered":"2024-03-06T07:27:34.129417+0000","last_clean":"2024-03-06T07:27:34.129417+0000","last_became_active":"2024-03-06T07:27:32.143652+0000","last_became_peered":"2024-03-06T07:27:32.143652+0000","last_unstale":"2024-03-06T07:27:34.129417+0000","last_undegraded":"2024-03-06T07:27:34.129417+0000","last_fullsized":"2024-03-06T07:27:34.129417+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:00.238442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147141+0000","last_change":"2024-03-06T07:27:33.144398+0000","last_active":"2024-03-06T07:27:33.147141+0000","last_peered":"2024-03-06T07:27:33.147141+0000","last_clean":"2024-03-06T07:27:33.147141+0000","last_became_active":"2024-03-06T07:27:33.143888+0000","last_became_peered":"2024-03-06T07:27:33.143888+0000","last_unstale":"2024-03-06T07:27:33.147141+0000","last_undegraded":"2024-03-06T07:27:33.147141+0000","last_fullsized":"2024-03-06T07:27:33.147141+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:57.609352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128417+0000","last_change":"2024-03-06T07:27:32.140035+0000","last_active":"2024-03-06T07:27:34.128417+0000","last_peered":"2024-03-06T07:27:34.128417+0000","last_clean":"2024-03-06T07:27:34.128417+0000","last_became_active":"2024-03-06T07:27:32.139707+0000","last_became_peered":"2024-03-06T07:27:32.139707+0000","last_unstale":"2024-03-06T07:27:34.128417+0000","last_undegraded":"2024-03-06T07:27:34.128417+0000","last_fullsized":"2024-03-06T07:27:34.128417+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:00.272125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147537+0000","last_change":"2024-03-06T07:27:33.139529+0000","last_active":"2024-03-06T07:27:33.147537+0000","last_peered":"2024-03-06T07:27:33.147537+0000","last_clean":"2024-03-06T07:27:33.147537+0000","last_became_active":"2024-03-06T07:27:33.139277+0000","last_became_peered":"2024-03-06T07:27:33.139277+0000","last_unstale":"2024-03-06T07:27:33.147537+0000","last_undegraded":"2024-03-06T07:27:33.147537+0000","last_fullsized":"2024-03-06T07:27:33.147537+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:57.762554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.135145+0000","last_change":"2024-03-06T07:27:32.138854+0000","last_active":"2024-03-06T07:27:33.135145+0000","last_peered":"2024-03-06T07:27:33.135145+0000","last_clean":"2024-03-06T07:27:33.135145+0000","last_became_active":"2024-03-06T07:27:32.138549+0000","last_became_peered":"2024-03-06T07:27:32.138549+0000","last_unstale":"2024-03-06T07:27:33.135145+0000","last_undegraded":"2024-03-06T07:27:33.135145+0000","last_fullsized":"2024-03-06T07:27:33.135145+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:10.155545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.139692+0000","last_change":"2024-03-06T07:27:33.137977+0000","last_active":"2024-03-06T07:27:33.139692+0000","last_peered":"2024-03-06T07:27:33.139692+0000","last_clean":"2024-03-06T07:27:33.139692+0000","last_became_active":"2024-03-06T07:27:33.137691+0000","last_became_peered":"2024-03-06T07:27:33.137691+0000","last_unstale":"2024-03-06T07:27:33.139692+0000","last_undegraded":"2024-03-06T07:27:33.139692+0000","last_fullsized":"2024-03-06T07:27:33.139692+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:36.868705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129647+0000","last_change":"2024-03-06T07:27:32.143181+0000","last_active":"2024-03-06T07:27:34.129647+0000","last_peered":"2024-03-06T07:27:34.129647+0000","last_clean":"2024-03-06T07:27:34.129647+0000","last_became_active":"2024-03-06T07:27:32.142749+0000","last_became_peered":"2024-03-06T07:27:32.142749+0000","last_unstale":"2024-03-06T07:27:34.129647+0000","last_undegraded":"2024-03-06T07:27:34.129647+0000","last_fullsized":"2024-03-06T07:27:34.129647+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:04.464894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128873+0000","last_change":"2024-03-06T07:27:33.142549+0000","last_active":"2024-03-06T07:27:34.128873+0000","last_peered":"2024-03-06T07:27:34.128873+0000","last_clean":"2024-03-06T07:27:34.128873+0000","last_became_active":"2024-03-06T07:27:33.142266+0000","last_became_peered":"2024-03-06T07:27:33.142266+0000","last_unstale":"2024-03-06T07:27:34.128873+0000","last_undegraded":"2024-03-06T07:27:34.128873+0000","last_fullsized":"2024-03-06T07:27:34.128873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:58.695535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"183'3","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.151099+0000","last_change":"2024-03-06T07:27:32.138437+0000","last_active":"2024-03-06T07:27:33.151099+0000","last_peered":"2024-03-06T07:27:33.151099+0000","last_clean":"2024-03-06T07:27:33.151099+0000","last_became_active":"2024-03-06T07:27:32.138155+0000","last_became_peered":"2024-03-06T07:27:32.138155+0000","last_unstale":"2024-03-06T07:27:33.151099+0000","last_undegraded":"2024-03-06T07:27:33.151099+0000","last_fullsized":"2024-03-06T07:27:33.151099+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:15.618377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129807+0000","last_change":"2024-03-06T07:27:33.146764+0000","last_active":"2024-03-06T07:27:34.129807+0000","last_peered":"2024-03-06T07:27:34.129807+0000","last_clean":"2024-03-06T07:27:34.129807+0000","last_became_active":"2024-03-06T07:27:33.146475+0000","last_became_peered":"2024-03-06T07:27:33.146475+0000","last_unstale":"2024-03-06T07:27:34.129807+0000","last_undegraded":"2024-03-06T07:27:34.129807+0000","last_fullsized":"2024-03-06T07:27:34.129807+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:26.395192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"183'6","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.168992+0000","last_change":"2024-03-06T07:27:32.143742+0000","last_active":"2024-03-06T07:27:33.168992+0000","last_peered":"2024-03-06T07:27:33.168992+0000","last_clean":"2024-03-06T07:27:33.168992+0000","last_became_active":"2024-03-06T07:27:32.143445+0000","last_became_peered":"2024-03-06T07:27:32.143445+0000","last_unstale":"2024-03-06T07:27:33.168992+0000","last_undegraded":"2024-03-06T07:27:33.168992+0000","last_fullsized":"2024-03-06T07:27:33.168992+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:27.898581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.154206+0000","last_change":"2024-03-06T07:27:33.148835+0000","last_active":"2024-03-06T07:27:33.154206+0000","last_peered":"2024-03-06T07:27:33.154206+0000","last_clean":"2024-03-06T07:27:33.154206+0000","last_became_active":"2024-03-06T07:27:33.148547+0000","last_became_peered":"2024-03-06T07:27:33.148547+0000","last_unstale":"2024-03-06T07:27:33.154206+0000","last_undegraded":"2024-03-06T07:27:33.154206+0000","last_fullsized":"2024-03-06T07:27:33.154206+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:43.806939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160433+0000","last_change":"2024-03-06T07:27:32.144140+0000","last_active":"2024-03-06T07:27:33.160433+0000","last_peered":"2024-03-06T07:27:33.160433+0000","last_clean":"2024-03-06T07:27:33.160433+0000","last_became_active":"2024-03-06T07:27:32.143372+0000","last_became_peered":"2024-03-06T07:27:32.143372+0000","last_unstale":"2024-03-06T07:27:33.160433+0000","last_undegraded":"2024-03-06T07:27:33.160433+0000","last_fullsized":"2024-03-06T07:27:33.160433+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:04.464894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.165955+0000","last_change":"2024-03-06T07:27:33.162480+0000","last_active":"2024-03-06T07:27:33.165955+0000","last_peered":"2024-03-06T07:27:33.165955+0000","last_clean":"2024-03-06T07:27:33.165955+0000","last_became_active":"2024-03-06T07:27:33.162164+0000","last_became_peered":"2024-03-06T07:27:33.162164+0000","last_unstale":"2024-03-06T07:27:33.165955+0000","last_undegraded":"2024-03-06T07:27:33.165955+0000","last_fullsized":"2024-03-06T07:27:33.165955+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:40.180867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.143889+0000","last_change":"2024-03-06T07:27:32.142116+0000","last_active":"2024-03-06T07:27:33.143889+0000","last_peered":"2024-03-06T07:27:33.143889+0000","last_clean":"2024-03-06T07:27:33.143889+0000","last_became_active":"2024-03-06T07:27:32.141795+0000","last_became_peered":"2024-03-06T07:27:32.141795+0000","last_unstale":"2024-03-06T07:27:33.143889+0000","last_undegraded":"2024-03-06T07:27:33.143889+0000","last_fullsized":"2024-03-06T07:27:33.143889+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:02:41.620288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128443+0000","last_change":"2024-03-06T07:27:33.147899+0000","last_active":"2024-03-06T07:27:34.128443+0000","last_peered":"2024-03-06T07:27:34.128443+0000","last_clean":"2024-03-06T07:27:34.128443+0000","last_became_active":"2024-03-06T07:27:33.147606+0000","last_became_peered":"2024-03-06T07:27:33.147606+0000","last_unstale":"2024-03-06T07:27:34.128443+0000","last_undegraded":"2024-03-06T07:27:34.128443+0000","last_fullsized":"2024-03-06T07:27:34.128443+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:40.375547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.144453+0000","last_change":"2024-03-06T07:27:32.140983+0000","last_active":"2024-03-06T07:27:33.144453+0000","last_peered":"2024-03-06T07:27:33.144453+0000","last_clean":"2024-03-06T07:27:33.144453+0000","last_became_active":"2024-03-06T07:27:32.140674+0000","last_became_peered":"2024-03-06T07:27:32.140674+0000","last_unstale":"2024-03-06T07:27:33.144453+0000","last_undegraded":"2024-03-06T07:27:33.144453+0000","last_fullsized":"2024-03-06T07:27:33.144453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:05.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152289+0000","last_change":"2024-03-06T07:27:33.150411+0000","last_active":"2024-03-06T07:27:33.152289+0000","last_peered":"2024-03-06T07:27:33.152289+0000","last_clean":"2024-03-06T07:27:33.152289+0000","last_became_active":"2024-03-06T07:27:33.150125+0000","last_became_peered":"2024-03-06T07:27:33.150125+0000","last_unstale":"2024-03-06T07:27:33.152289+0000","last_undegraded":"2024-03-06T07:27:33.152289+0000","last_fullsized":"2024-03-06T07:27:33.152289+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:05.825076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"183'8","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164473+0000","last_change":"2024-03-06T07:27:32.141346+0000","last_active":"2024-03-06T07:27:33.164473+0000","last_peered":"2024-03-06T07:27:33.164473+0000","last_clean":"2024-03-06T07:27:33.164473+0000","last_became_active":"2024-03-06T07:27:32.141073+0000","last_became_peered":"2024-03-06T07:27:32.141073+0000","last_unstale":"2024-03-06T07:27:33.164473+0000","last_undegraded":"2024-03-06T07:27:33.164473+0000","last_fullsized":"2024-03-06T07:27:33.164473+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:49.460766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161145+0000","last_change":"2024-03-06T07:27:33.160347+0000","last_active":"2024-03-06T07:27:33.161145+0000","last_peered":"2024-03-06T07:27:33.161145+0000","last_clean":"2024-03-06T07:27:33.161145+0000","last_became_active":"2024-03-06T07:27:33.160041+0000","last_became_peered":"2024-03-06T07:27:33.160041+0000","last_unstale":"2024-03-06T07:27:33.161145+0000","last_undegraded":"2024-03-06T07:27:33.161145+0000","last_fullsized":"2024-03-06T07:27:33.161145+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:16.662051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128742+0000","last_change":"2024-03-06T07:27:32.144182+0000","last_active":"2024-03-06T07:27:34.128742+0000","last_peered":"2024-03-06T07:27:34.128742+0000","last_clean":"2024-03-06T07:27:34.128742+0000","last_became_active":"2024-03-06T07:27:32.143889+0000","last_became_peered":"2024-03-06T07:27:32.143889+0000","last_unstale":"2024-03-06T07:27:34.128742+0000","last_undegraded":"2024-03-06T07:27:34.128742+0000","last_fullsized":"2024-03-06T07:27:34.128742+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:03:19.997514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.151036+0000","last_change":"2024-03-06T07:27:33.142695+0000","last_active":"2024-03-06T07:27:33.151036+0000","last_peered":"2024-03-06T07:27:33.151036+0000","last_clean":"2024-03-06T07:27:33.151036+0000","last_became_active":"2024-03-06T07:27:33.142365+0000","last_became_peered":"2024-03-06T07:27:33.142365+0000","last_unstale":"2024-03-06T07:27:33.151036+0000","last_undegraded":"2024-03-06T07:27:33.151036+0000","last_fullsized":"2024-03-06T07:27:33.151036+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:57.762554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.143695+0000","last_change":"2024-03-06T07:27:32.144307+0000","last_active":"2024-03-06T07:27:33.143695+0000","last_peered":"2024-03-06T07:27:33.143695+0000","last_clean":"2024-03-06T07:27:33.143695+0000","last_became_active":"2024-03-06T07:27:32.143567+0000","last_became_peered":"2024-03-06T07:27:32.143567+0000","last_unstale":"2024-03-06T07:27:33.143695+0000","last_undegraded":"2024-03-06T07:27:33.143695+0000","last_fullsized":"2024-03-06T07:27:33.143695+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:45:54.868424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128870+0000","last_change":"2024-03-06T07:27:33.147699+0000","last_active":"2024-03-06T07:27:34.128870+0000","last_peered":"2024-03-06T07:27:34.128870+0000","last_clean":"2024-03-06T07:27:34.128870+0000","last_became_active":"2024-03-06T07:27:33.147389+0000","last_became_peered":"2024-03-06T07:27:33.147389+0000","last_unstale":"2024-03-06T07:27:34.128870+0000","last_undegraded":"2024-03-06T07:27:34.128870+0000","last_fullsized":"2024-03-06T07:27:34.128870+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:43.533093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163829+0000","last_change":"2024-03-06T07:27:32.146355+0000","last_active":"2024-03-06T07:27:33.163829+0000","last_peered":"2024-03-06T07:27:33.163829+0000","last_clean":"2024-03-06T07:27:33.163829+0000","last_became_active":"2024-03-06T07:27:32.145671+0000","last_became_peered":"2024-03-06T07:27:32.145671+0000","last_unstale":"2024-03-06T07:27:33.163829+0000","last_undegraded":"2024-03-06T07:27:33.163829+0000","last_fullsized":"2024-03-06T07:27:33.163829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:09.598887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161512+0000","last_change":"2024-03-06T07:27:33.156755+0000","last_active":"2024-03-06T07:27:33.161512+0000","last_peered":"2024-03-06T07:27:33.161512+0000","last_clean":"2024-03-06T07:27:33.161512+0000","last_became_active":"2024-03-06T07:27:33.156131+0000","last_became_peered":"2024-03-06T07:27:33.156131+0000","last_unstale":"2024-03-06T07:27:33.161512+0000","last_undegraded":"2024-03-06T07:27:33.161512+0000","last_fullsized":"2024-03-06T07:27:33.161512+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:57:03.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.145093+0000","last_change":"2024-03-06T07:27:32.146717+0000","last_active":"2024-03-06T07:27:33.145093+0000","last_peered":"2024-03-06T07:27:33.145093+0000","last_clean":"2024-03-06T07:27:33.145093+0000","last_became_active":"2024-03-06T07:27:32.146350+0000","last_became_peered":"2024-03-06T07:27:32.146350+0000","last_unstale":"2024-03-06T07:27:33.145093+0000","last_undegraded":"2024-03-06T07:27:33.145093+0000","last_fullsized":"2024-03-06T07:27:33.145093+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:00.238442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162354+0000","last_change":"2024-03-06T07:27:33.158330+0000","last_active":"2024-03-06T07:27:33.162354+0000","last_peered":"2024-03-06T07:27:33.162354+0000","last_clean":"2024-03-06T07:27:33.162354+0000","last_became_active":"2024-03-06T07:27:33.157841+0000","last_became_peered":"2024-03-06T07:27:33.157841+0000","last_unstale":"2024-03-06T07:27:33.162354+0000","last_undegraded":"2024-03-06T07:27:33.162354+0000","last_fullsized":"2024-03-06T07:27:33.162354+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:43.689457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162426+0000","last_change":"2024-03-06T07:27:32.144252+0000","last_active":"2024-03-06T07:27:33.162426+0000","last_peered":"2024-03-06T07:27:33.162426+0000","last_clean":"2024-03-06T07:27:33.162426+0000","last_became_active":"2024-03-06T07:27:32.143710+0000","last_became_peered":"2024-03-06T07:27:32.143710+0000","last_unstale":"2024-03-06T07:27:33.162426+0000","last_undegraded":"2024-03-06T07:27:33.162426+0000","last_fullsized":"2024-03-06T07:27:33.162426+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:17.020895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152255+0000","last_change":"2024-03-06T07:27:33.147708+0000","last_active":"2024-03-06T07:27:33.152255+0000","last_peered":"2024-03-06T07:27:33.152255+0000","last_clean":"2024-03-06T07:27:33.152255+0000","last_became_active":"2024-03-06T07:27:33.147270+0000","last_became_peered":"2024-03-06T07:27:33.147270+0000","last_unstale":"2024-03-06T07:27:33.152255+0000","last_undegraded":"2024-03-06T07:27:33.152255+0000","last_fullsized":"2024-03-06T07:27:33.152255+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:05.825076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.133588+0000","last_change":"2024-03-06T07:27:32.137543+0000","last_active":"2024-03-06T07:27:33.133588+0000","last_peered":"2024-03-06T07:27:33.133588+0000","last_clean":"2024-03-06T07:27:33.133588+0000","last_became_active":"2024-03-06T07:27:32.137261+0000","last_became_peered":"2024-03-06T07:27:32.137261+0000","last_unstale":"2024-03-06T07:27:33.133588+0000","last_undegraded":"2024-03-06T07:27:33.133588+0000","last_fullsized":"2024-03-06T07:27:33.133588+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:10.155545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161477+0000","last_change":"2024-03-06T07:27:33.161184+0000","last_active":"2024-03-06T07:27:33.161477+0000","last_peered":"2024-03-06T07:27:33.161477+0000","last_clean":"2024-03-06T07:27:33.161477+0000","last_became_active":"2024-03-06T07:27:33.160887+0000","last_became_peered":"2024-03-06T07:27:33.160887+0000","last_unstale":"2024-03-06T07:27:33.161477+0000","last_undegraded":"2024-03-06T07:27:33.161477+0000","last_fullsized":"2024-03-06T07:27:33.161477+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:08.036174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.132440+0000","last_change":"2024-03-06T07:27:32.143486+0000","last_active":"2024-03-06T07:27:33.132440+0000","last_peered":"2024-03-06T07:27:33.132440+0000","last_clean":"2024-03-06T07:27:33.132440+0000","last_became_active":"2024-03-06T07:27:32.143204+0000","last_became_peered":"2024-03-06T07:27:32.143204+0000","last_unstale":"2024-03-06T07:27:33.132440+0000","last_undegraded":"2024-03-06T07:27:33.132440+0000","last_fullsized":"2024-03-06T07:27:33.132440+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:08.399236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128587+0000","last_change":"2024-03-06T07:27:33.156644+0000","last_active":"2024-03-06T07:27:34.128587+0000","last_peered":"2024-03-06T07:27:34.128587+0000","last_clean":"2024-03-06T07:27:34.128587+0000","last_became_active":"2024-03-06T07:27:33.156354+0000","last_became_peered":"2024-03-06T07:27:33.156354+0000","last_unstale":"2024-03-06T07:27:34.128587+0000","last_undegraded":"2024-03-06T07:27:34.128587+0000","last_fullsized":"2024-03-06T07:27:34.128587+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:05.053228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.128235+0000","last_change":"2024-03-06T07:27:32.142934+0000","last_active":"2024-03-06T07:27:33.128235+0000","last_peered":"2024-03-06T07:27:33.128235+0000","last_clean":"2024-03-06T07:27:33.128235+0000","last_became_active":"2024-03-06T07:27:32.142648+0000","last_became_peered":"2024-03-06T07:27:32.142648+0000","last_unstale":"2024-03-06T07:27:33.128235+0000","last_undegraded":"2024-03-06T07:27:33.128235+0000","last_fullsized":"2024-03-06T07:27:33.128235+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:25:22.443545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.148131+0000","last_change":"2024-03-06T07:27:33.146187+0000","last_active":"2024-03-06T07:27:33.148131+0000","last_peered":"2024-03-06T07:27:33.148131+0000","last_clean":"2024-03-06T07:27:33.148131+0000","last_became_active":"2024-03-06T07:27:33.145851+0000","last_became_peered":"2024-03-06T07:27:33.145851+0000","last_unstale":"2024-03-06T07:27:33.148131+0000","last_undegraded":"2024-03-06T07:27:33.148131+0000","last_fullsized":"2024-03-06T07:27:33.148131+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:16.452494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162528+0000","last_change":"2024-03-06T07:27:32.147353+0000","last_active":"2024-03-06T07:27:33.162528+0000","last_peered":"2024-03-06T07:27:33.162528+0000","last_clean":"2024-03-06T07:27:33.162528+0000","last_became_active":"2024-03-06T07:27:32.146919+0000","last_became_peered":"2024-03-06T07:27:32.146919+0000","last_unstale":"2024-03-06T07:27:33.162528+0000","last_undegraded":"2024-03-06T07:27:33.162528+0000","last_fullsized":"2024-03-06T07:27:33.162528+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:25.390016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.155698+0000","last_change":"2024-03-06T07:27:33.151234+0000","last_active":"2024-03-06T07:27:33.155698+0000","last_peered":"2024-03-06T07:27:33.155698+0000","last_clean":"2024-03-06T07:27:33.155698+0000","last_became_active":"2024-03-06T07:27:33.150673+0000","last_became_peered":"2024-03-06T07:27:33.150673+0000","last_unstale":"2024-03-06T07:27:33.155698+0000","last_undegraded":"2024-03-06T07:27:33.155698+0000","last_fullsized":"2024-03-06T07:27:33.155698+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:11:46.929783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.131688+0000","last_change":"2024-03-06T07:27:32.144203+0000","last_active":"2024-03-06T07:27:33.131688+0000","last_peered":"2024-03-06T07:27:33.131688+0000","last_clean":"2024-03-06T07:27:33.131688+0000","last_became_active":"2024-03-06T07:27:32.143872+0000","last_became_peered":"2024-03-06T07:27:32.143872+0000","last_unstale":"2024-03-06T07:27:33.131688+0000","last_undegraded":"2024-03-06T07:27:33.131688+0000","last_fullsized":"2024-03-06T07:27:33.131688+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:52.574912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163540+0000","last_change":"2024-03-06T07:27:33.162500+0000","last_active":"2024-03-06T07:27:33.163540+0000","last_peered":"2024-03-06T07:27:33.163540+0000","last_clean":"2024-03-06T07:27:33.163540+0000","last_became_active":"2024-03-06T07:27:33.162249+0000","last_became_peered":"2024-03-06T07:27:33.162249+0000","last_unstale":"2024-03-06T07:27:33.163540+0000","last_undegraded":"2024-03-06T07:27:33.163540+0000","last_fullsized":"2024-03-06T07:27:33.163540+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:52.021386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.132350+0000","last_change":"2024-03-06T07:27:32.143271+0000","last_active":"2024-03-06T07:27:33.132350+0000","last_peered":"2024-03-06T07:27:33.132350+0000","last_clean":"2024-03-06T07:27:33.132350+0000","last_became_active":"2024-03-06T07:27:32.142807+0000","last_became_peered":"2024-03-06T07:27:32.142807+0000","last_unstale":"2024-03-06T07:27:33.132350+0000","last_undegraded":"2024-03-06T07:27:33.132350+0000","last_fullsized":"2024-03-06T07:27:33.132350+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:58.584433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.146616+0000","last_change":"2024-03-06T07:27:33.146046+0000","last_active":"2024-03-06T07:27:33.146616+0000","last_peered":"2024-03-06T07:27:33.146616+0000","last_clean":"2024-03-06T07:27:33.146616+0000","last_became_active":"2024-03-06T07:27:33.145807+0000","last_became_peered":"2024-03-06T07:27:33.145807+0000","last_unstale":"2024-03-06T07:27:33.146616+0000","last_undegraded":"2024-03-06T07:27:33.146616+0000","last_fullsized":"2024-03-06T07:27:33.146616+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:28.107338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.145141+0000","last_change":"2024-03-06T07:27:32.141653+0000","last_active":"2024-03-06T07:27:33.145141+0000","last_peered":"2024-03-06T07:27:33.145141+0000","last_clean":"2024-03-06T07:27:33.145141+0000","last_became_active":"2024-03-06T07:27:32.141130+0000","last_became_peered":"2024-03-06T07:27:32.141130+0000","last_unstale":"2024-03-06T07:27:33.145141+0000","last_undegraded":"2024-03-06T07:27:33.145141+0000","last_fullsized":"2024-03-06T07:27:33.145141+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.713843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128312+0000","last_change":"2024-03-06T07:27:33.157105+0000","last_active":"2024-03-06T07:27:34.128312+0000","last_peered":"2024-03-06T07:27:34.128312+0000","last_clean":"2024-03-06T07:27:34.128312+0000","last_became_active":"2024-03-06T07:27:33.156756+0000","last_became_peered":"2024-03-06T07:27:33.156756+0000","last_unstale":"2024-03-06T07:27:34.128312+0000","last_undegraded":"2024-03-06T07:27:34.128312+0000","last_fullsized":"2024-03-06T07:27:34.128312+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:25:15.451845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.145763+0000","last_change":"2024-03-06T07:27:32.143960+0000","last_active":"2024-03-06T07:27:33.145763+0000","last_peered":"2024-03-06T07:27:33.145763+0000","last_clean":"2024-03-06T07:27:33.145763+0000","last_became_active":"2024-03-06T07:27:32.143013+0000","last_became_peered":"2024-03-06T07:27:32.143013+0000","last_unstale":"2024-03-06T07:27:33.145763+0000","last_undegraded":"2024-03-06T07:27:33.145763+0000","last_fullsized":"2024-03-06T07:27:33.145763+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:53.265639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128400+0000","last_change":"2024-03-06T07:27:33.143010+0000","last_active":"2024-03-06T07:27:34.128400+0000","last_peered":"2024-03-06T07:27:34.128400+0000","last_clean":"2024-03-06T07:27:34.128400+0000","last_became_active":"2024-03-06T07:27:33.142737+0000","last_became_peered":"2024-03-06T07:27:33.142737+0000","last_unstale":"2024-03-06T07:27:34.128400+0000","last_undegraded":"2024-03-06T07:27:34.128400+0000","last_fullsized":"2024-03-06T07:27:34.128400+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:33.082995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129497+0000","last_change":"2024-03-06T07:27:32.139836+0000","last_active":"2024-03-06T07:27:34.129497+0000","last_peered":"2024-03-06T07:27:34.129497+0000","last_clean":"2024-03-06T07:27:34.129497+0000","last_became_active":"2024-03-06T07:27:32.139329+0000","last_became_peered":"2024-03-06T07:27:32.139329+0000","last_unstale":"2024-03-06T07:27:34.129497+0000","last_undegraded":"2024-03-06T07:27:34.129497+0000","last_fullsized":"2024-03-06T07:27:34.129497+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:09.598887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.144496+0000","last_change":"2024-03-06T07:27:33.143303+0000","last_active":"2024-03-06T07:27:33.144496+0000","last_peered":"2024-03-06T07:27:33.144496+0000","last_clean":"2024-03-06T07:27:33.144496+0000","last_became_active":"2024-03-06T07:27:33.141593+0000","last_became_peered":"2024-03-06T07:27:33.141593+0000","last_unstale":"2024-03-06T07:27:33.144496+0000","last_undegraded":"2024-03-06T07:27:33.144496+0000","last_fullsized":"2024-03-06T07:27:33.144496+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:00.884983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.133014+0000","last_change":"2024-03-06T07:27:32.139094+0000","last_active":"2024-03-06T07:27:33.133014+0000","last_peered":"2024-03-06T07:27:33.133014+0000","last_clean":"2024-03-06T07:27:33.133014+0000","last_became_active":"2024-03-06T07:27:32.138785+0000","last_became_peered":"2024-03-06T07:27:32.138785+0000","last_unstale":"2024-03-06T07:27:33.133014+0000","last_undegraded":"2024-03-06T07:27:33.133014+0000","last_fullsized":"2024-03-06T07:27:33.133014+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:58.584433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157511+0000","last_change":"2024-03-06T07:27:33.154098+0000","last_active":"2024-03-06T07:27:33.157511+0000","last_peered":"2024-03-06T07:27:33.157511+0000","last_clean":"2024-03-06T07:27:33.157511+0000","last_became_active":"2024-03-06T07:27:33.153776+0000","last_became_peered":"2024-03-06T07:27:33.153776+0000","last_unstale":"2024-03-06T07:27:33.157511+0000","last_undegraded":"2024-03-06T07:27:33.157511+0000","last_fullsized":"2024-03-06T07:27:33.157511+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:55:18.198469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.146265+0000","last_change":"2024-03-06T07:27:32.142549+0000","last_active":"2024-03-06T07:27:33.146265+0000","last_peered":"2024-03-06T07:27:33.146265+0000","last_clean":"2024-03-06T07:27:33.146265+0000","last_became_active":"2024-03-06T07:27:32.142259+0000","last_became_peered":"2024-03-06T07:27:32.142259+0000","last_unstale":"2024-03-06T07:27:33.146265+0000","last_undegraded":"2024-03-06T07:27:33.146265+0000","last_fullsized":"2024-03-06T07:27:33.146265+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:14.370528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128801+0000","last_change":"2024-03-06T07:27:33.160085+0000","last_active":"2024-03-06T07:27:34.128801+0000","last_peered":"2024-03-06T07:27:34.128801+0000","last_clean":"2024-03-06T07:27:34.128801+0000","last_became_active":"2024-03-06T07:27:33.159825+0000","last_became_peered":"2024-03-06T07:27:33.159825+0000","last_unstale":"2024-03-06T07:27:34.128801+0000","last_undegraded":"2024-03-06T07:27:34.128801+0000","last_fullsized":"2024-03-06T07:27:34.128801+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:44.688077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.150880+0000","last_change":"2024-03-06T07:27:32.140356+0000","last_active":"2024-03-06T07:27:33.150880+0000","last_peered":"2024-03-06T07:27:33.150880+0000","last_clean":"2024-03-06T07:27:33.150880+0000","last_became_active":"2024-03-06T07:27:32.139971+0000","last_became_peered":"2024-03-06T07:27:32.139971+0000","last_unstale":"2024-03-06T07:27:33.150880+0000","last_undegraded":"2024-03-06T07:27:33.150880+0000","last_fullsized":"2024-03-06T07:27:33.150880+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:08.399236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":475,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.132853+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:27:33.132853+0000","last_peered":"2024-03-06T07:27:33.132853+0000","last_clean":"2024-03-06T07:27:33.132853+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:27:33.132853+0000","last_undegraded":"2024-03-06T07:27:33.132853+0000","last_fullsized":"2024-03-06T07:27:33.132853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33524,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338316,"statfs":{"total":96636764160,"available":96602435584,"internally_reserved":0,"allocated":2392064,"data_stored":1718365,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2342912,"data_stored":1129063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1129007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1134107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1134039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2326528,"data_stored":1127941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33448,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338392,"statfs":{"total":96636764160,"available":96602513408,"internally_reserved":0,"allocated":2318336,"data_stored":1127325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2383872,"data_stored":1717693,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:27:36.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:27:36.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.276+0000 7fcc63feb640 1 Processor -- start 2024-03-06T07:27:36.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.276+0000 7fcc63feb640 1 -- start start 2024-03-06T07:27:36.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.276+0000 7fcc63feb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcc5c0c3280 0x7fcc5c0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.276+0000 7fcc63feb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c1551e0 0x7fcc5c155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.276+0000 7fcc63feb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c158800 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.276+0000 7fcc63feb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c158ae0 con 0x7fcc5c0c3280 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.276+0000 7fcc63feb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c158dc0 con 0x7fcc5c1551e0 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc61586640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcc5c0c3280 0x7fcc5c0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc62588640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c1551e0 0x7fcc5c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc61586640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcc5c0c3280 0x7fcc5c0c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53624/0 (socket says 172.21.15.26:53624) 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc61586640 1 -- 172.21.15.26:0/660857525 learned_addr learned my addr 172.21.15.26:0/660857525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc61586640 1 -- 172.21.15.26:0/660857525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c1551e0 msgr2=0x7fcc5c155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc61586640 1 --2- 172.21.15.26:0/660857525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c1551e0 0x7fcc5c155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc61586640 1 -- 172.21.15.26:0/660857525 >> v1:172.21.15.117:6789/0 conn(0x7fcc5c06d9f0 legacy=0x7fcc5c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.277+0000 7fcc61586640 1 -- 172.21.15.26:0/660857525 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c1590a0 con 0x7fcc5c0c3280 2024-03-06T07:27:36.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc61586640 1 --2- 172.21.15.26:0/660857525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c0c3280 0x7fcc5c0c3710 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009870 tx=0x7fcc4c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e04f57cc70e07d30 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc60d85640 1 -- 172.21.15.26:0/660857525 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c035020 con 0x7fcc5c0c3280 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc60d85640 1 -- 172.21.15.26:0/660857525 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c031640 con 0x7fcc5c0c3280 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc60d85640 1 -- 172.21.15.26:0/660857525 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c02d040 con 0x7fcc5c0c3280 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc63feb640 1 -- 172.21.15.26:0/660857525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c0c3280 msgr2=0x7fcc5c0c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc63feb640 1 --2- 172.21.15.26:0/660857525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c0c3280 0x7fcc5c0c3710 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009870 tx=0x7fcc4c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc63feb640 1 -- 172.21.15.26:0/660857525 shutdown_connections 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc63feb640 1 --2- 172.21.15.26:0/660857525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c1551e0 0x7fcc5c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc63feb640 1 --2- 172.21.15.26:0/660857525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c0c3280 0x7fcc5c0c3710 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.278+0000 7fcc63feb640 1 -- 172.21.15.26:0/660857525 >> 172.21.15.26:0/660857525 conn(0x7fcc5c0aa630 msgr2=0x7fcc5c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.279+0000 7fcc63feb640 1 -- 172.21.15.26:0/660857525 shutdown_connections 2024-03-06T07:27:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.279+0000 7fcc63feb640 1 -- 172.21.15.26:0/660857525 wait complete. 2024-03-06T07:27:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 Processor -- start 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 -- start start 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5c06d9f0 0x7fcc5c14f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c0c3280 0x7fcc5c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c1551e0 0x7fcc5c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c06df50 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c06ddd0 con 0x7fcc5c1551e0 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc63feb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0c6910 con 0x7fcc5c0c3280 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc61d87640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5c06d9f0 0x7fcc5c14f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc61d87640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5c06d9f0 0x7fcc5c14f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58588/0 (socket says 172.21.15.26:58588) 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc61d87640 1 -- 172.21.15.26:0/544295556 learned_addr learned my addr 172.21.15.26:0/544295556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.280+0000 7fcc61586640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c0c3280 0x7fcc5c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc62588640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c1551e0 0x7fcc5c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc61d87640 1 -- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c0c3280 msgr2=0x7fcc5c157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc61d87640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c0c3280 0x7fcc5c157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc61d87640 1 -- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c1551e0 msgr2=0x7fcc5c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc61d87640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c1551e0 0x7fcc5c14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc61d87640 1 -- 172.21.15.26:0/544295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c1e15d0 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc61d87640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5c06d9f0 0x7fcc5c14f3e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c430 tx=0x7fcc5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc527fc640 1 -- 172.21.15.26:0/544295556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc580052d0 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc527fc640 1 -- 172.21.15.26:0/544295556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58005490 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c0adde0 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc527fc640 1 -- 172.21.15.26:0/544295556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58007260 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.281+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c0ae3a0 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.282+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c04f330 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.283+0000 7fcc527fc640 1 -- 172.21.15.26:0/544295556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc58007400 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.283+0000 7fcc527fc640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc4006b500 0x7fcc4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.283+0000 7fcc527fc640 1 -- 172.21.15.26:0/544295556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc58089740 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.283+0000 7fcc61586640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc4006b500 0x7fcc4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.284+0000 7fcc61586640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc4006b500 0x7fcc4006d9c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c036000 tx=0x7fcc4c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.287+0000 7fcc527fc640 1 -- 172.21.15.26:0/544295556 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcc5c1e15d0 con 0x7fcc5c06d9f0 2024-03-06T07:27:36.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.466+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc5c073240 con 0x7fcc4006b500 2024-03-06T07:27:36.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.472+0000 7fcc527fc640 1 -- 172.21.15.26:0/544295556 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252517 (secure 0 0 0) 0x7fcc4c085a80 con 0x7fcc4006b500 2024-03-06T07:27:36.477 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:36.478 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:27:36.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc4006b500 msgr2=0x7fcc4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc4006b500 0x7fcc4006d9c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c036000 tx=0x7fcc4c03c040 comp rx=0 tx=0).stop 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5c06d9f0 msgr2=0x7fcc5c14f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5c06d9f0 0x7fcc5c14f3e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c430 tx=0x7fcc5800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 shutdown_connections 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc4006b500 0x7fcc4006d9c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc5c1551e0 0x7fcc5c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc5c0c3280 0x7fcc5c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 --2- 172.21.15.26:0/544295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5c06d9f0 0x7fcc5c14f3e0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 >> 172.21.15.26:0/544295556 conn(0x7fcc5c0aa630 msgr2=0x7fcc5c0c7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 shutdown_connections 2024-03-06T07:27:36.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.475+0000 7fcc63feb640 1 -- 172.21.15.26:0/544295556 wait complete. 2024-03-06T07:27:36.488 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-03-06T07:27:35.126252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267804,"kb_used_data":18328,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754706916,"statfs":{"total":773094113280,"available":772819881984,"internally_reserved":0,"allocated":18767872,"data_stored":10217540,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.005994"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.158179+0000","last_change":"2024-03-06T07:27:33.155279+0000","last_active":"2024-03-06T07:27:33.158179+0000","last_peered":"2024-03-06T07:27:33.158179+0000","last_clean":"2024-03-06T07:27:33.158179+0000","last_became_active":"2024-03-06T07:27:33.155013+0000","last_became_peered":"2024-03-06T07:27:33.155013+0000","last_unstale":"2024-03-06T07:27:33.158179+0000","last_undegraded":"2024-03-06T07:27:33.158179+0000","last_fullsized":"2024-03-06T07:27:33.158179+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:45:20.914191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.154874+0000","last_change":"2024-03-06T07:27:33.151127+0000","last_active":"2024-03-06T07:27:33.154874+0000","last_peered":"2024-03-06T07:27:33.154874+0000","last_clean":"2024-03-06T07:27:33.154874+0000","last_became_active":"2024-03-06T07:27:33.150121+0000","last_became_peered":"2024-03-06T07:27:33.150121+0000","last_unstale":"2024-03-06T07:27:33.154874+0000","last_undegraded":"2024-03-06T07:27:33.154874+0000","last_fullsized":"2024-03-06T07:27:33.154874+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:29:45.203419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128633+0000","last_change":"2024-03-06T07:27:33.149107+0000","last_active":"2024-03-06T07:27:34.128633+0000","last_peered":"2024-03-06T07:27:34.128633+0000","last_clean":"2024-03-06T07:27:34.128633+0000","last_became_active":"2024-03-06T07:27:33.148836+0000","last_became_peered":"2024-03-06T07:27:33.148836+0000","last_unstale":"2024-03-06T07:27:34.128633+0000","last_undegraded":"2024-03-06T07:27:34.128633+0000","last_fullsized":"2024-03-06T07:27:34.128633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:27:33.192746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129802+0000","last_change":"2024-03-06T07:27:33.153052+0000","last_active":"2024-03-06T07:27:34.129802+0000","last_peered":"2024-03-06T07:27:34.129802+0000","last_clean":"2024-03-06T07:27:34.129802+0000","last_became_active":"2024-03-06T07:27:33.152766+0000","last_became_peered":"2024-03-06T07:27:33.152766+0000","last_unstale":"2024-03-06T07:27:34.129802+0000","last_undegraded":"2024-03-06T07:27:34.129802+0000","last_fullsized":"2024-03-06T07:27:34.129802+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:18.026072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.148352+0000","last_change":"2024-03-06T07:27:33.140305+0000","last_active":"2024-03-06T07:27:33.148352+0000","last_peered":"2024-03-06T07:27:33.148352+0000","last_clean":"2024-03-06T07:27:33.148352+0000","last_became_active":"2024-03-06T07:27:33.139973+0000","last_became_peered":"2024-03-06T07:27:33.139973+0000","last_unstale":"2024-03-06T07:27:33.148352+0000","last_undegraded":"2024-03-06T07:27:33.148352+0000","last_fullsized":"2024-03-06T07:27:33.148352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:36.868705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160157+0000","last_change":"2024-03-06T07:27:33.152163+0000","last_active":"2024-03-06T07:27:33.160157+0000","last_peered":"2024-03-06T07:27:33.160157+0000","last_clean":"2024-03-06T07:27:33.160157+0000","last_became_active":"2024-03-06T07:27:33.151314+0000","last_became_peered":"2024-03-06T07:27:33.151314+0000","last_unstale":"2024-03-06T07:27:33.160157+0000","last_undegraded":"2024-03-06T07:27:33.160157+0000","last_fullsized":"2024-03-06T07:27:33.160157+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:29.868915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128434+0000","last_change":"2024-03-06T07:27:33.148746+0000","last_active":"2024-03-06T07:27:34.128434+0000","last_peered":"2024-03-06T07:27:34.128434+0000","last_clean":"2024-03-06T07:27:34.128434+0000","last_became_active":"2024-03-06T07:27:33.148443+0000","last_became_peered":"2024-03-06T07:27:33.148443+0000","last_unstale":"2024-03-06T07:27:34.128434+0000","last_undegraded":"2024-03-06T07:27:34.128434+0000","last_fullsized":"2024-03-06T07:27:34.128434+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:51:59.786014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160230+0000","last_change":"2024-03-06T07:27:33.152080+0000","last_active":"2024-03-06T07:27:33.160230+0000","last_peered":"2024-03-06T07:27:33.160230+0000","last_clean":"2024-03-06T07:27:33.160230+0000","last_became_active":"2024-03-06T07:27:33.151379+0000","last_became_peered":"2024-03-06T07:27:33.151379+0000","last_unstale":"2024-03-06T07:27:33.160230+0000","last_undegraded":"2024-03-06T07:27:33.160230+0000","last_fullsized":"2024-03-06T07:27:33.160230+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:38.325184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.136517+0000","last_change":"2024-03-06T07:27:33.135277+0000","last_active":"2024-03-06T07:27:33.136517+0000","last_peered":"2024-03-06T07:27:33.136517+0000","last_clean":"2024-03-06T07:27:33.136517+0000","last_became_active":"2024-03-06T07:27:33.134995+0000","last_became_peered":"2024-03-06T07:27:33.134995+0000","last_unstale":"2024-03-06T07:27:33.136517+0000","last_undegraded":"2024-03-06T07:27:33.136517+0000","last_fullsized":"2024-03-06T07:27:33.136517+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:44:17.685958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128677+0000","last_change":"2024-03-06T07:27:33.151543+0000","last_active":"2024-03-06T07:27:34.128677+0000","last_peered":"2024-03-06T07:27:34.128677+0000","last_clean":"2024-03-06T07:27:34.128677+0000","last_became_active":"2024-03-06T07:27:33.151261+0000","last_became_peered":"2024-03-06T07:27:33.151261+0000","last_unstale":"2024-03-06T07:27:34.128677+0000","last_undegraded":"2024-03-06T07:27:34.128677+0000","last_fullsized":"2024-03-06T07:27:34.128677+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:55.803752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129917+0000","last_change":"2024-03-06T07:27:33.148963+0000","last_active":"2024-03-06T07:27:34.129917+0000","last_peered":"2024-03-06T07:27:34.129917+0000","last_clean":"2024-03-06T07:27:34.129917+0000","last_became_active":"2024-03-06T07:27:33.148670+0000","last_became_peered":"2024-03-06T07:27:33.148670+0000","last_unstale":"2024-03-06T07:27:34.129917+0000","last_undegraded":"2024-03-06T07:27:34.129917+0000","last_fullsized":"2024-03-06T07:27:34.129917+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:55:18.198469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.168805+0000","last_change":"2024-03-06T07:27:33.164228+0000","last_active":"2024-03-06T07:27:33.168805+0000","last_peered":"2024-03-06T07:27:33.168805+0000","last_clean":"2024-03-06T07:27:33.168805+0000","last_became_active":"2024-03-06T07:27:33.163971+0000","last_became_peered":"2024-03-06T07:27:33.163971+0000","last_unstale":"2024-03-06T07:27:33.168805+0000","last_undegraded":"2024-03-06T07:27:33.168805+0000","last_fullsized":"2024-03-06T07:27:33.168805+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:18.383123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157404+0000","last_change":"2024-03-06T07:27:33.151226+0000","last_active":"2024-03-06T07:27:33.157404+0000","last_peered":"2024-03-06T07:27:33.157404+0000","last_clean":"2024-03-06T07:27:33.157404+0000","last_became_active":"2024-03-06T07:27:33.150917+0000","last_became_peered":"2024-03-06T07:27:33.150917+0000","last_unstale":"2024-03-06T07:27:33.157404+0000","last_undegraded":"2024-03-06T07:27:33.157404+0000","last_fullsized":"2024-03-06T07:27:33.157404+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:04.973457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163976+0000","last_change":"2024-03-06T07:27:33.159390+0000","last_active":"2024-03-06T07:27:33.163976+0000","last_peered":"2024-03-06T07:27:33.163976+0000","last_clean":"2024-03-06T07:27:33.163976+0000","last_became_active":"2024-03-06T07:27:33.158040+0000","last_became_peered":"2024-03-06T07:27:33.158040+0000","last_unstale":"2024-03-06T07:27:33.163976+0000","last_undegraded":"2024-03-06T07:27:33.163976+0000","last_fullsized":"2024-03-06T07:27:33.163976+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:14:13.257345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147917+0000","last_change":"2024-03-06T07:27:33.139936+0000","last_active":"2024-03-06T07:27:33.147917+0000","last_peered":"2024-03-06T07:27:33.147917+0000","last_clean":"2024-03-06T07:27:33.147917+0000","last_became_active":"2024-03-06T07:27:33.139659+0000","last_became_peered":"2024-03-06T07:27:33.139659+0000","last_unstale":"2024-03-06T07:27:33.147917+0000","last_undegraded":"2024-03-06T07:27:33.147917+0000","last_fullsized":"2024-03-06T07:27:33.147917+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:11.996894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157613+0000","last_change":"2024-03-06T07:27:33.151729+0000","last_active":"2024-03-06T07:27:33.157613+0000","last_peered":"2024-03-06T07:27:33.157613+0000","last_clean":"2024-03-06T07:27:33.157613+0000","last_became_active":"2024-03-06T07:27:33.151035+0000","last_became_peered":"2024-03-06T07:27:33.151035+0000","last_unstale":"2024-03-06T07:27:33.157613+0000","last_undegraded":"2024-03-06T07:27:33.157613+0000","last_fullsized":"2024-03-06T07:27:33.157613+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:00.743621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164138+0000","last_change":"2024-03-06T07:27:33.159283+0000","last_active":"2024-03-06T07:27:33.164138+0000","last_peered":"2024-03-06T07:27:33.164138+0000","last_clean":"2024-03-06T07:27:33.164138+0000","last_became_active":"2024-03-06T07:27:33.157565+0000","last_became_peered":"2024-03-06T07:27:33.157565+0000","last_unstale":"2024-03-06T07:27:33.164138+0000","last_undegraded":"2024-03-06T07:27:33.164138+0000","last_fullsized":"2024-03-06T07:27:33.164138+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:21:59.582658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128725+0000","last_change":"2024-03-06T07:27:33.146830+0000","last_active":"2024-03-06T07:27:34.128725+0000","last_peered":"2024-03-06T07:27:34.128725+0000","last_clean":"2024-03-06T07:27:34.128725+0000","last_became_active":"2024-03-06T07:27:33.146567+0000","last_became_peered":"2024-03-06T07:27:33.146567+0000","last_unstale":"2024-03-06T07:27:34.128725+0000","last_undegraded":"2024-03-06T07:27:34.128725+0000","last_fullsized":"2024-03-06T07:27:34.128725+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:49.961714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152281+0000","last_change":"2024-03-06T07:27:33.149381+0000","last_active":"2024-03-06T07:27:33.152281+0000","last_peered":"2024-03-06T07:27:33.152281+0000","last_clean":"2024-03-06T07:27:33.152281+0000","last_became_active":"2024-03-06T07:27:33.149111+0000","last_became_peered":"2024-03-06T07:27:33.149111+0000","last_unstale":"2024-03-06T07:27:33.152281+0000","last_undegraded":"2024-03-06T07:27:33.152281+0000","last_fullsized":"2024-03-06T07:27:33.152281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:58.261673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160124+0000","last_change":"2024-03-06T07:27:33.152075+0000","last_active":"2024-03-06T07:27:33.160124+0000","last_peered":"2024-03-06T07:27:33.160124+0000","last_clean":"2024-03-06T07:27:33.160124+0000","last_became_active":"2024-03-06T07:27:33.151788+0000","last_became_peered":"2024-03-06T07:27:33.151788+0000","last_unstale":"2024-03-06T07:27:33.160124+0000","last_undegraded":"2024-03-06T07:27:33.160124+0000","last_fullsized":"2024-03-06T07:27:33.160124+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:39.354949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128441+0000","last_change":"2024-03-06T07:27:33.146275+0000","last_active":"2024-03-06T07:27:34.128441+0000","last_peered":"2024-03-06T07:27:34.128441+0000","last_clean":"2024-03-06T07:27:34.128441+0000","last_became_active":"2024-03-06T07:27:33.145953+0000","last_became_peered":"2024-03-06T07:27:33.145953+0000","last_unstale":"2024-03-06T07:27:34.128441+0000","last_undegraded":"2024-03-06T07:27:34.128441+0000","last_fullsized":"2024-03-06T07:27:34.128441+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:51:20.509542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.155357+0000","last_change":"2024-03-06T07:27:33.154623+0000","last_active":"2024-03-06T07:27:33.155357+0000","last_peered":"2024-03-06T07:27:33.155357+0000","last_clean":"2024-03-06T07:27:33.155357+0000","last_became_active":"2024-03-06T07:27:33.154316+0000","last_became_peered":"2024-03-06T07:27:33.154316+0000","last_unstale":"2024-03-06T07:27:33.155357+0000","last_undegraded":"2024-03-06T07:27:33.155357+0000","last_fullsized":"2024-03-06T07:27:33.155357+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:58.261673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.135669+0000","last_change":"2024-03-06T07:27:33.134532+0000","last_active":"2024-03-06T07:27:33.135669+0000","last_peered":"2024-03-06T07:27:33.135669+0000","last_clean":"2024-03-06T07:27:33.135669+0000","last_became_active":"2024-03-06T07:27:33.134257+0000","last_became_peered":"2024-03-06T07:27:33.134257+0000","last_unstale":"2024-03-06T07:27:33.135669+0000","last_undegraded":"2024-03-06T07:27:33.135669+0000","last_fullsized":"2024-03-06T07:27:33.135669+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:02:34.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.149729+0000","last_change":"2024-03-06T07:27:33.143532+0000","last_active":"2024-03-06T07:27:33.149729+0000","last_peered":"2024-03-06T07:27:33.149729+0000","last_clean":"2024-03-06T07:27:33.149729+0000","last_became_active":"2024-03-06T07:27:33.143251+0000","last_became_peered":"2024-03-06T07:27:33.143251+0000","last_unstale":"2024-03-06T07:27:33.149729+0000","last_undegraded":"2024-03-06T07:27:33.149729+0000","last_fullsized":"2024-03-06T07:27:33.149729+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:35.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.149832+0000","last_change":"2024-03-06T07:27:33.143491+0000","last_active":"2024-03-06T07:27:33.149832+0000","last_peered":"2024-03-06T07:27:33.149832+0000","last_clean":"2024-03-06T07:27:33.149832+0000","last_became_active":"2024-03-06T07:27:33.143131+0000","last_became_peered":"2024-03-06T07:27:33.143131+0000","last_unstale":"2024-03-06T07:27:33.149832+0000","last_undegraded":"2024-03-06T07:27:33.149832+0000","last_fullsized":"2024-03-06T07:27:33.149832+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:11.996894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128772+0000","last_change":"2024-03-06T07:27:33.158732+0000","last_active":"2024-03-06T07:27:34.128772+0000","last_peered":"2024-03-06T07:27:34.128772+0000","last_clean":"2024-03-06T07:27:34.128772+0000","last_became_active":"2024-03-06T07:27:33.158491+0000","last_became_peered":"2024-03-06T07:27:33.158491+0000","last_unstale":"2024-03-06T07:27:34.128772+0000","last_undegraded":"2024-03-06T07:27:34.128772+0000","last_fullsized":"2024-03-06T07:27:34.128772+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:02.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129918+0000","last_change":"2024-03-06T07:27:33.146057+0000","last_active":"2024-03-06T07:27:34.129918+0000","last_peered":"2024-03-06T07:27:34.129918+0000","last_clean":"2024-03-06T07:27:34.129918+0000","last_became_active":"2024-03-06T07:27:33.145784+0000","last_became_peered":"2024-03-06T07:27:33.145784+0000","last_unstale":"2024-03-06T07:27:34.129918+0000","last_undegraded":"2024-03-06T07:27:34.129918+0000","last_fullsized":"2024-03-06T07:27:34.129918+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:57:03.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164369+0000","last_change":"2024-03-06T07:27:33.159871+0000","last_active":"2024-03-06T07:27:33.164369+0000","last_peered":"2024-03-06T07:27:33.164369+0000","last_clean":"2024-03-06T07:27:33.164369+0000","last_became_active":"2024-03-06T07:27:33.159570+0000","last_became_peered":"2024-03-06T07:27:33.159570+0000","last_unstale":"2024-03-06T07:27:33.164369+0000","last_undegraded":"2024-03-06T07:27:33.164369+0000","last_fullsized":"2024-03-06T07:27:33.164369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:56:32.954518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.154246+0000","last_change":"2024-03-06T07:27:33.149076+0000","last_active":"2024-03-06T07:27:33.154246+0000","last_peered":"2024-03-06T07:27:33.154246+0000","last_clean":"2024-03-06T07:27:33.154246+0000","last_became_active":"2024-03-06T07:27:33.148391+0000","last_became_peered":"2024-03-06T07:27:33.148391+0000","last_unstale":"2024-03-06T07:27:33.154246+0000","last_undegraded":"2024-03-06T07:27:33.154246+0000","last_fullsized":"2024-03-06T07:27:33.154246+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:39.410877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163043+0000","last_change":"2024-03-06T07:27:33.158523+0000","last_active":"2024-03-06T07:27:33.163043+0000","last_peered":"2024-03-06T07:27:33.163043+0000","last_clean":"2024-03-06T07:27:33.163043+0000","last_became_active":"2024-03-06T07:27:33.158268+0000","last_became_peered":"2024-03-06T07:27:33.158268+0000","last_unstale":"2024-03-06T07:27:33.163043+0000","last_undegraded":"2024-03-06T07:27:33.163043+0000","last_fullsized":"2024-03-06T07:27:33.163043+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:40.955624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152756+0000","last_change":"2024-03-06T07:27:33.151064+0000","last_active":"2024-03-06T07:27:33.152756+0000","last_peered":"2024-03-06T07:27:33.152756+0000","last_clean":"2024-03-06T07:27:33.152756+0000","last_became_active":"2024-03-06T07:27:33.150737+0000","last_became_peered":"2024-03-06T07:27:33.150737+0000","last_unstale":"2024-03-06T07:27:33.152756+0000","last_undegraded":"2024-03-06T07:27:33.152756+0000","last_fullsized":"2024-03-06T07:27:33.152756+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:18:50.213880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163571+0000","last_change":"2024-03-06T07:27:33.161368+0000","last_active":"2024-03-06T07:27:33.163571+0000","last_peered":"2024-03-06T07:27:33.163571+0000","last_clean":"2024-03-06T07:27:33.163571+0000","last_became_active":"2024-03-06T07:27:33.161117+0000","last_became_peered":"2024-03-06T07:27:33.161117+0000","last_unstale":"2024-03-06T07:27:33.163571+0000","last_undegraded":"2024-03-06T07:27:33.163571+0000","last_fullsized":"2024-03-06T07:27:33.163571+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:30.252166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128482+0000","last_change":"2024-03-06T07:27:33.152683+0000","last_active":"2024-03-06T07:27:34.128482+0000","last_peered":"2024-03-06T07:27:34.128482+0000","last_clean":"2024-03-06T07:27:34.128482+0000","last_became_active":"2024-03-06T07:27:33.152403+0000","last_became_peered":"2024-03-06T07:27:33.152403+0000","last_unstale":"2024-03-06T07:27:34.128482+0000","last_undegraded":"2024-03-06T07:27:34.128482+0000","last_fullsized":"2024-03-06T07:27:34.128482+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:12.871068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"183'4","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152066+0000","last_change":"2024-03-06T07:27:32.142861+0000","last_active":"2024-03-06T07:27:33.152066+0000","last_peered":"2024-03-06T07:27:33.152066+0000","last_clean":"2024-03-06T07:27:33.152066+0000","last_became_active":"2024-03-06T07:27:32.142557+0000","last_became_peered":"2024-03-06T07:27:32.142557+0000","last_unstale":"2024-03-06T07:27:33.152066+0000","last_undegraded":"2024-03-06T07:27:33.152066+0000","last_fullsized":"2024-03-06T07:27:33.152066+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.042398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128621+0000","last_change":"2024-03-06T07:27:33.138462+0000","last_active":"2024-03-06T07:27:34.128621+0000","last_peered":"2024-03-06T07:27:34.128621+0000","last_clean":"2024-03-06T07:27:34.128621+0000","last_became_active":"2024-03-06T07:27:33.138144+0000","last_became_peered":"2024-03-06T07:27:33.138144+0000","last_unstale":"2024-03-06T07:27:34.128621+0000","last_undegraded":"2024-03-06T07:27:34.128621+0000","last_fullsized":"2024-03-06T07:27:34.128621+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:16.293356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161522+0000","last_change":"2024-03-06T07:27:32.141407+0000","last_active":"2024-03-06T07:27:33.161522+0000","last_peered":"2024-03-06T07:27:33.161522+0000","last_clean":"2024-03-06T07:27:33.161522+0000","last_became_active":"2024-03-06T07:27:32.141083+0000","last_became_peered":"2024-03-06T07:27:32.141083+0000","last_unstale":"2024-03-06T07:27:33.161522+0000","last_undegraded":"2024-03-06T07:27:33.161522+0000","last_fullsized":"2024-03-06T07:27:33.161522+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:23.040151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161670+0000","last_change":"2024-03-06T07:27:33.157812+0000","last_active":"2024-03-06T07:27:33.161670+0000","last_peered":"2024-03-06T07:27:33.161670+0000","last_clean":"2024-03-06T07:27:33.161670+0000","last_became_active":"2024-03-06T07:27:33.157520+0000","last_became_peered":"2024-03-06T07:27:33.157520+0000","last_unstale":"2024-03-06T07:27:33.161670+0000","last_undegraded":"2024-03-06T07:27:33.161670+0000","last_fullsized":"2024-03-06T07:27:33.161670+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:55.161831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162142+0000","last_change":"2024-03-06T07:27:32.146210+0000","last_active":"2024-03-06T07:27:33.162142+0000","last_peered":"2024-03-06T07:27:33.162142+0000","last_clean":"2024-03-06T07:27:33.162142+0000","last_became_active":"2024-03-06T07:27:32.145911+0000","last_became_peered":"2024-03-06T07:27:32.145911+0000","last_unstale":"2024-03-06T07:27:33.162142+0000","last_undegraded":"2024-03-06T07:27:33.162142+0000","last_fullsized":"2024-03-06T07:27:33.162142+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:02:59.864073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128317+0000","last_change":"2024-03-06T07:27:33.152715+0000","last_active":"2024-03-06T07:27:34.128317+0000","last_peered":"2024-03-06T07:27:34.128317+0000","last_clean":"2024-03-06T07:27:34.128317+0000","last_became_active":"2024-03-06T07:27:33.152040+0000","last_became_peered":"2024-03-06T07:27:33.152040+0000","last_unstale":"2024-03-06T07:27:34.128317+0000","last_undegraded":"2024-03-06T07:27:34.128317+0000","last_fullsized":"2024-03-06T07:27:34.128317+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:08.820948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157969+0000","last_change":"2024-03-06T07:27:32.146403+0000","last_active":"2024-03-06T07:27:33.157969+0000","last_peered":"2024-03-06T07:27:33.157969+0000","last_clean":"2024-03-06T07:27:33.157969+0000","last_became_active":"2024-03-06T07:27:32.146175+0000","last_became_peered":"2024-03-06T07:27:32.146175+0000","last_unstale":"2024-03-06T07:27:33.157969+0000","last_undegraded":"2024-03-06T07:27:33.157969+0000","last_fullsized":"2024-03-06T07:27:33.157969+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:15.618377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164370+0000","last_change":"2024-03-06T07:27:33.160177+0000","last_active":"2024-03-06T07:27:33.164370+0000","last_peered":"2024-03-06T07:27:33.164370+0000","last_clean":"2024-03-06T07:27:33.164370+0000","last_became_active":"2024-03-06T07:27:33.159892+0000","last_became_peered":"2024-03-06T07:27:33.159892+0000","last_unstale":"2024-03-06T07:27:33.164370+0000","last_undegraded":"2024-03-06T07:27:33.164370+0000","last_fullsized":"2024-03-06T07:27:33.164370+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:06:30.834522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.155770+0000","last_change":"2024-03-06T07:27:32.142277+0000","last_active":"2024-03-06T07:27:33.155770+0000","last_peered":"2024-03-06T07:27:33.155770+0000","last_clean":"2024-03-06T07:27:33.155770+0000","last_became_active":"2024-03-06T07:27:32.142006+0000","last_became_peered":"2024-03-06T07:27:32.142006+0000","last_unstale":"2024-03-06T07:27:33.155770+0000","last_undegraded":"2024-03-06T07:27:33.155770+0000","last_fullsized":"2024-03-06T07:27:33.155770+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:52.574912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147968+0000","last_change":"2024-03-06T07:27:33.145176+0000","last_active":"2024-03-06T07:27:33.147968+0000","last_peered":"2024-03-06T07:27:33.147968+0000","last_clean":"2024-03-06T07:27:33.147968+0000","last_became_active":"2024-03-06T07:27:33.144880+0000","last_became_peered":"2024-03-06T07:27:33.144880+0000","last_unstale":"2024-03-06T07:27:33.147968+0000","last_undegraded":"2024-03-06T07:27:33.147968+0000","last_fullsized":"2024-03-06T07:27:33.147968+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:35.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129417+0000","last_change":"2024-03-06T07:27:32.143984+0000","last_active":"2024-03-06T07:27:34.129417+0000","last_peered":"2024-03-06T07:27:34.129417+0000","last_clean":"2024-03-06T07:27:34.129417+0000","last_became_active":"2024-03-06T07:27:32.143652+0000","last_became_peered":"2024-03-06T07:27:32.143652+0000","last_unstale":"2024-03-06T07:27:34.129417+0000","last_undegraded":"2024-03-06T07:27:34.129417+0000","last_fullsized":"2024-03-06T07:27:34.129417+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:00.238442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147141+0000","last_change":"2024-03-06T07:27:33.144398+0000","last_active":"2024-03-06T07:27:33.147141+0000","last_peered":"2024-03-06T07:27:33.147141+0000","last_clean":"2024-03-06T07:27:33.147141+0000","last_became_active":"2024-03-06T07:27:33.143888+0000","last_became_peered":"2024-03-06T07:27:33.143888+0000","last_unstale":"2024-03-06T07:27:33.147141+0000","last_undegraded":"2024-03-06T07:27:33.147141+0000","last_fullsized":"2024-03-06T07:27:33.147141+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:00:57.609352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128417+0000","last_change":"2024-03-06T07:27:32.140035+0000","last_active":"2024-03-06T07:27:34.128417+0000","last_peered":"2024-03-06T07:27:34.128417+0000","last_clean":"2024-03-06T07:27:34.128417+0000","last_became_active":"2024-03-06T07:27:32.139707+0000","last_became_peered":"2024-03-06T07:27:32.139707+0000","last_unstale":"2024-03-06T07:27:34.128417+0000","last_undegraded":"2024-03-06T07:27:34.128417+0000","last_fullsized":"2024-03-06T07:27:34.128417+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:00.272125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.147537+0000","last_change":"2024-03-06T07:27:33.139529+0000","last_active":"2024-03-06T07:27:33.147537+0000","last_peered":"2024-03-06T07:27:33.147537+0000","last_clean":"2024-03-06T07:27:33.147537+0000","last_became_active":"2024-03-06T07:27:33.139277+0000","last_became_peered":"2024-03-06T07:27:33.139277+0000","last_unstale":"2024-03-06T07:27:33.147537+0000","last_undegraded":"2024-03-06T07:27:33.147537+0000","last_fullsized":"2024-03-06T07:27:33.147537+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:57.762554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.135145+0000","last_change":"2024-03-06T07:27:32.138854+0000","last_active":"2024-03-06T07:27:33.135145+0000","last_peered":"2024-03-06T07:27:33.135145+0000","last_clean":"2024-03-06T07:27:33.135145+0000","last_became_active":"2024-03-06T07:27:32.138549+0000","last_became_peered":"2024-03-06T07:27:32.138549+0000","last_unstale":"2024-03-06T07:27:33.135145+0000","last_undegraded":"2024-03-06T07:27:33.135145+0000","last_fullsized":"2024-03-06T07:27:33.135145+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:10.155545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.139692+0000","last_change":"2024-03-06T07:27:33.137977+0000","last_active":"2024-03-06T07:27:33.139692+0000","last_peered":"2024-03-06T07:27:33.139692+0000","last_clean":"2024-03-06T07:27:33.139692+0000","last_became_active":"2024-03-06T07:27:33.137691+0000","last_became_peered":"2024-03-06T07:27:33.137691+0000","last_unstale":"2024-03-06T07:27:33.139692+0000","last_undegraded":"2024-03-06T07:27:33.139692+0000","last_fullsized":"2024-03-06T07:27:33.139692+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:10:36.868705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129647+0000","last_change":"2024-03-06T07:27:32.143181+0000","last_active":"2024-03-06T07:27:34.129647+0000","last_peered":"2024-03-06T07:27:34.129647+0000","last_clean":"2024-03-06T07:27:34.129647+0000","last_became_active":"2024-03-06T07:27:32.142749+0000","last_became_peered":"2024-03-06T07:27:32.142749+0000","last_unstale":"2024-03-06T07:27:34.129647+0000","last_undegraded":"2024-03-06T07:27:34.129647+0000","last_fullsized":"2024-03-06T07:27:34.129647+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:04.464894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128873+0000","last_change":"2024-03-06T07:27:33.142549+0000","last_active":"2024-03-06T07:27:34.128873+0000","last_peered":"2024-03-06T07:27:34.128873+0000","last_clean":"2024-03-06T07:27:34.128873+0000","last_became_active":"2024-03-06T07:27:33.142266+0000","last_became_peered":"2024-03-06T07:27:33.142266+0000","last_unstale":"2024-03-06T07:27:34.128873+0000","last_undegraded":"2024-03-06T07:27:34.128873+0000","last_fullsized":"2024-03-06T07:27:34.128873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:58.695535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"183'3","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.151099+0000","last_change":"2024-03-06T07:27:32.138437+0000","last_active":"2024-03-06T07:27:33.151099+0000","last_peered":"2024-03-06T07:27:33.151099+0000","last_clean":"2024-03-06T07:27:33.151099+0000","last_became_active":"2024-03-06T07:27:32.138155+0000","last_became_peered":"2024-03-06T07:27:32.138155+0000","last_unstale":"2024-03-06T07:27:33.151099+0000","last_undegraded":"2024-03-06T07:27:33.151099+0000","last_fullsized":"2024-03-06T07:27:33.151099+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:15.618377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129807+0000","last_change":"2024-03-06T07:27:33.146764+0000","last_active":"2024-03-06T07:27:34.129807+0000","last_peered":"2024-03-06T07:27:34.129807+0000","last_clean":"2024-03-06T07:27:34.129807+0000","last_became_active":"2024-03-06T07:27:33.146475+0000","last_became_peered":"2024-03-06T07:27:33.146475+0000","last_unstale":"2024-03-06T07:27:34.129807+0000","last_undegraded":"2024-03-06T07:27:34.129807+0000","last_fullsized":"2024-03-06T07:27:34.129807+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:26.395192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"183'6","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.168992+0000","last_change":"2024-03-06T07:27:32.143742+0000","last_active":"2024-03-06T07:27:33.168992+0000","last_peered":"2024-03-06T07:27:33.168992+0000","last_clean":"2024-03-06T07:27:33.168992+0000","last_became_active":"2024-03-06T07:27:32.143445+0000","last_became_peered":"2024-03-06T07:27:32.143445+0000","last_unstale":"2024-03-06T07:27:33.168992+0000","last_undegraded":"2024-03-06T07:27:33.168992+0000","last_fullsized":"2024-03-06T07:27:33.168992+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:27.898581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.154206+0000","last_change":"2024-03-06T07:27:33.148835+0000","last_active":"2024-03-06T07:27:33.154206+0000","last_peered":"2024-03-06T07:27:33.154206+0000","last_clean":"2024-03-06T07:27:33.154206+0000","last_became_active":"2024-03-06T07:27:33.148547+0000","last_became_peered":"2024-03-06T07:27:33.148547+0000","last_unstale":"2024-03-06T07:27:33.154206+0000","last_undegraded":"2024-03-06T07:27:33.154206+0000","last_fullsized":"2024-03-06T07:27:33.154206+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:43.806939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.160433+0000","last_change":"2024-03-06T07:27:32.144140+0000","last_active":"2024-03-06T07:27:33.160433+0000","last_peered":"2024-03-06T07:27:33.160433+0000","last_clean":"2024-03-06T07:27:33.160433+0000","last_became_active":"2024-03-06T07:27:32.143372+0000","last_became_peered":"2024-03-06T07:27:32.143372+0000","last_unstale":"2024-03-06T07:27:33.160433+0000","last_undegraded":"2024-03-06T07:27:33.160433+0000","last_fullsized":"2024-03-06T07:27:33.160433+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:04.464894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.165955+0000","last_change":"2024-03-06T07:27:33.162480+0000","last_active":"2024-03-06T07:27:33.165955+0000","last_peered":"2024-03-06T07:27:33.165955+0000","last_clean":"2024-03-06T07:27:33.165955+0000","last_became_active":"2024-03-06T07:27:33.162164+0000","last_became_peered":"2024-03-06T07:27:33.162164+0000","last_unstale":"2024-03-06T07:27:33.165955+0000","last_undegraded":"2024-03-06T07:27:33.165955+0000","last_fullsized":"2024-03-06T07:27:33.165955+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:40.180867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.143889+0000","last_change":"2024-03-06T07:27:32.142116+0000","last_active":"2024-03-06T07:27:33.143889+0000","last_peered":"2024-03-06T07:27:33.143889+0000","last_clean":"2024-03-06T07:27:33.143889+0000","last_became_active":"2024-03-06T07:27:32.141795+0000","last_became_peered":"2024-03-06T07:27:32.141795+0000","last_unstale":"2024-03-06T07:27:33.143889+0000","last_undegraded":"2024-03-06T07:27:33.143889+0000","last_fullsized":"2024-03-06T07:27:33.143889+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:02:41.620288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128443+0000","last_change":"2024-03-06T07:27:33.147899+0000","last_active":"2024-03-06T07:27:34.128443+0000","last_peered":"2024-03-06T07:27:34.128443+0000","last_clean":"2024-03-06T07:27:34.128443+0000","last_became_active":"2024-03-06T07:27:33.147606+0000","last_became_peered":"2024-03-06T07:27:33.147606+0000","last_unstale":"2024-03-06T07:27:34.128443+0000","last_undegraded":"2024-03-06T07:27:34.128443+0000","last_fullsized":"2024-03-06T07:27:34.128443+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:40.375547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.144453+0000","last_change":"2024-03-06T07:27:32.140983+0000","last_active":"2024-03-06T07:27:33.144453+0000","last_peered":"2024-03-06T07:27:33.144453+0000","last_clean":"2024-03-06T07:27:33.144453+0000","last_became_active":"2024-03-06T07:27:32.140674+0000","last_became_peered":"2024-03-06T07:27:32.140674+0000","last_unstale":"2024-03-06T07:27:33.144453+0000","last_undegraded":"2024-03-06T07:27:33.144453+0000","last_fullsized":"2024-03-06T07:27:33.144453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:05.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152289+0000","last_change":"2024-03-06T07:27:33.150411+0000","last_active":"2024-03-06T07:27:33.152289+0000","last_peered":"2024-03-06T07:27:33.152289+0000","last_clean":"2024-03-06T07:27:33.152289+0000","last_became_active":"2024-03-06T07:27:33.150125+0000","last_became_peered":"2024-03-06T07:27:33.150125+0000","last_unstale":"2024-03-06T07:27:33.152289+0000","last_undegraded":"2024-03-06T07:27:33.152289+0000","last_fullsized":"2024-03-06T07:27:33.152289+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:05.825076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"183'8","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.164473+0000","last_change":"2024-03-06T07:27:32.141346+0000","last_active":"2024-03-06T07:27:33.164473+0000","last_peered":"2024-03-06T07:27:33.164473+0000","last_clean":"2024-03-06T07:27:33.164473+0000","last_became_active":"2024-03-06T07:27:32.141073+0000","last_became_peered":"2024-03-06T07:27:32.141073+0000","last_unstale":"2024-03-06T07:27:33.164473+0000","last_undegraded":"2024-03-06T07:27:33.164473+0000","last_fullsized":"2024-03-06T07:27:33.164473+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:49.460766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161145+0000","last_change":"2024-03-06T07:27:33.160347+0000","last_active":"2024-03-06T07:27:33.161145+0000","last_peered":"2024-03-06T07:27:33.161145+0000","last_clean":"2024-03-06T07:27:33.161145+0000","last_became_active":"2024-03-06T07:27:33.160041+0000","last_became_peered":"2024-03-06T07:27:33.160041+0000","last_unstale":"2024-03-06T07:27:33.161145+0000","last_undegraded":"2024-03-06T07:27:33.161145+0000","last_fullsized":"2024-03-06T07:27:33.161145+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:16.662051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128742+0000","last_change":"2024-03-06T07:27:32.144182+0000","last_active":"2024-03-06T07:27:34.128742+0000","last_peered":"2024-03-06T07:27:34.128742+0000","last_clean":"2024-03-06T07:27:34.128742+0000","last_became_active":"2024-03-06T07:27:32.143889+0000","last_became_peered":"2024-03-06T07:27:32.143889+0000","last_unstale":"2024-03-06T07:27:34.128742+0000","last_undegraded":"2024-03-06T07:27:34.128742+0000","last_fullsized":"2024-03-06T07:27:34.128742+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:03:19.997514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.151036+0000","last_change":"2024-03-06T07:27:33.142695+0000","last_active":"2024-03-06T07:27:33.151036+0000","last_peered":"2024-03-06T07:27:33.151036+0000","last_clean":"2024-03-06T07:27:33.151036+0000","last_became_active":"2024-03-06T07:27:33.142365+0000","last_became_peered":"2024-03-06T07:27:33.142365+0000","last_unstale":"2024-03-06T07:27:33.151036+0000","last_undegraded":"2024-03-06T07:27:33.151036+0000","last_fullsized":"2024-03-06T07:27:33.151036+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:57.762554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.143695+0000","last_change":"2024-03-06T07:27:32.144307+0000","last_active":"2024-03-06T07:27:33.143695+0000","last_peered":"2024-03-06T07:27:33.143695+0000","last_clean":"2024-03-06T07:27:33.143695+0000","last_became_active":"2024-03-06T07:27:32.143567+0000","last_became_peered":"2024-03-06T07:27:32.143567+0000","last_unstale":"2024-03-06T07:27:33.143695+0000","last_undegraded":"2024-03-06T07:27:33.143695+0000","last_fullsized":"2024-03-06T07:27:33.143695+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:45:54.868424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128870+0000","last_change":"2024-03-06T07:27:33.147699+0000","last_active":"2024-03-06T07:27:34.128870+0000","last_peered":"2024-03-06T07:27:34.128870+0000","last_clean":"2024-03-06T07:27:34.128870+0000","last_became_active":"2024-03-06T07:27:33.147389+0000","last_became_peered":"2024-03-06T07:27:33.147389+0000","last_unstale":"2024-03-06T07:27:34.128870+0000","last_undegraded":"2024-03-06T07:27:34.128870+0000","last_fullsized":"2024-03-06T07:27:34.128870+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:43.533093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163829+0000","last_change":"2024-03-06T07:27:32.146355+0000","last_active":"2024-03-06T07:27:33.163829+0000","last_peered":"2024-03-06T07:27:33.163829+0000","last_clean":"2024-03-06T07:27:33.163829+0000","last_became_active":"2024-03-06T07:27:32.145671+0000","last_became_peered":"2024-03-06T07:27:32.145671+0000","last_unstale":"2024-03-06T07:27:33.163829+0000","last_undegraded":"2024-03-06T07:27:33.163829+0000","last_fullsized":"2024-03-06T07:27:33.163829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:09.598887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161512+0000","last_change":"2024-03-06T07:27:33.156755+0000","last_active":"2024-03-06T07:27:33.161512+0000","last_peered":"2024-03-06T07:27:33.161512+0000","last_clean":"2024-03-06T07:27:33.161512+0000","last_became_active":"2024-03-06T07:27:33.156131+0000","last_became_peered":"2024-03-06T07:27:33.156131+0000","last_unstale":"2024-03-06T07:27:33.161512+0000","last_undegraded":"2024-03-06T07:27:33.161512+0000","last_fullsized":"2024-03-06T07:27:33.161512+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:57:03.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.145093+0000","last_change":"2024-03-06T07:27:32.146717+0000","last_active":"2024-03-06T07:27:33.145093+0000","last_peered":"2024-03-06T07:27:33.145093+0000","last_clean":"2024-03-06T07:27:33.145093+0000","last_became_active":"2024-03-06T07:27:32.146350+0000","last_became_peered":"2024-03-06T07:27:32.146350+0000","last_unstale":"2024-03-06T07:27:33.145093+0000","last_undegraded":"2024-03-06T07:27:33.145093+0000","last_fullsized":"2024-03-06T07:27:33.145093+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:16:00.238442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162354+0000","last_change":"2024-03-06T07:27:33.158330+0000","last_active":"2024-03-06T07:27:33.162354+0000","last_peered":"2024-03-06T07:27:33.162354+0000","last_clean":"2024-03-06T07:27:33.162354+0000","last_became_active":"2024-03-06T07:27:33.157841+0000","last_became_peered":"2024-03-06T07:27:33.157841+0000","last_unstale":"2024-03-06T07:27:33.162354+0000","last_undegraded":"2024-03-06T07:27:33.162354+0000","last_fullsized":"2024-03-06T07:27:33.162354+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:28:43.689457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162426+0000","last_change":"2024-03-06T07:27:32.144252+0000","last_active":"2024-03-06T07:27:33.162426+0000","last_peered":"2024-03-06T07:27:33.162426+0000","last_clean":"2024-03-06T07:27:33.162426+0000","last_became_active":"2024-03-06T07:27:32.143710+0000","last_became_peered":"2024-03-06T07:27:32.143710+0000","last_unstale":"2024-03-06T07:27:33.162426+0000","last_undegraded":"2024-03-06T07:27:33.162426+0000","last_fullsized":"2024-03-06T07:27:33.162426+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:17.020895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.152255+0000","last_change":"2024-03-06T07:27:33.147708+0000","last_active":"2024-03-06T07:27:33.152255+0000","last_peered":"2024-03-06T07:27:33.152255+0000","last_clean":"2024-03-06T07:27:33.152255+0000","last_became_active":"2024-03-06T07:27:33.147270+0000","last_became_peered":"2024-03-06T07:27:33.147270+0000","last_unstale":"2024-03-06T07:27:33.152255+0000","last_undegraded":"2024-03-06T07:27:33.152255+0000","last_fullsized":"2024-03-06T07:27:33.152255+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:05.825076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.133588+0000","last_change":"2024-03-06T07:27:32.137543+0000","last_active":"2024-03-06T07:27:33.133588+0000","last_peered":"2024-03-06T07:27:33.133588+0000","last_clean":"2024-03-06T07:27:33.133588+0000","last_became_active":"2024-03-06T07:27:32.137261+0000","last_became_peered":"2024-03-06T07:27:32.137261+0000","last_unstale":"2024-03-06T07:27:33.133588+0000","last_undegraded":"2024-03-06T07:27:33.133588+0000","last_fullsized":"2024-03-06T07:27:33.133588+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:10.155545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.161477+0000","last_change":"2024-03-06T07:27:33.161184+0000","last_active":"2024-03-06T07:27:33.161477+0000","last_peered":"2024-03-06T07:27:33.161477+0000","last_clean":"2024-03-06T07:27:33.161477+0000","last_became_active":"2024-03-06T07:27:33.160887+0000","last_became_peered":"2024-03-06T07:27:33.160887+0000","last_unstale":"2024-03-06T07:27:33.161477+0000","last_undegraded":"2024-03-06T07:27:33.161477+0000","last_fullsized":"2024-03-06T07:27:33.161477+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:08.036174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.132440+0000","last_change":"2024-03-06T07:27:32.143486+0000","last_active":"2024-03-06T07:27:33.132440+0000","last_peered":"2024-03-06T07:27:33.132440+0000","last_clean":"2024-03-06T07:27:33.132440+0000","last_became_active":"2024-03-06T07:27:32.143204+0000","last_became_peered":"2024-03-06T07:27:32.143204+0000","last_unstale":"2024-03-06T07:27:33.132440+0000","last_undegraded":"2024-03-06T07:27:33.132440+0000","last_fullsized":"2024-03-06T07:27:33.132440+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:08.399236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128587+0000","last_change":"2024-03-06T07:27:33.156644+0000","last_active":"2024-03-06T07:27:34.128587+0000","last_peered":"2024-03-06T07:27:34.128587+0000","last_clean":"2024-03-06T07:27:34.128587+0000","last_became_active":"2024-03-06T07:27:33.156354+0000","last_became_peered":"2024-03-06T07:27:33.156354+0000","last_unstale":"2024-03-06T07:27:34.128587+0000","last_undegraded":"2024-03-06T07:27:34.128587+0000","last_fullsized":"2024-03-06T07:27:34.128587+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:05.053228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.128235+0000","last_change":"2024-03-06T07:27:32.142934+0000","last_active":"2024-03-06T07:27:33.128235+0000","last_peered":"2024-03-06T07:27:33.128235+0000","last_clean":"2024-03-06T07:27:33.128235+0000","last_became_active":"2024-03-06T07:27:32.142648+0000","last_became_peered":"2024-03-06T07:27:32.142648+0000","last_unstale":"2024-03-06T07:27:33.128235+0000","last_undegraded":"2024-03-06T07:27:33.128235+0000","last_fullsized":"2024-03-06T07:27:33.128235+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:25:22.443545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.148131+0000","last_change":"2024-03-06T07:27:33.146187+0000","last_active":"2024-03-06T07:27:33.148131+0000","last_peered":"2024-03-06T07:27:33.148131+0000","last_clean":"2024-03-06T07:27:33.148131+0000","last_became_active":"2024-03-06T07:27:33.145851+0000","last_became_peered":"2024-03-06T07:27:33.145851+0000","last_unstale":"2024-03-06T07:27:33.148131+0000","last_undegraded":"2024-03-06T07:27:33.148131+0000","last_fullsized":"2024-03-06T07:27:33.148131+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:16.452494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.162528+0000","last_change":"2024-03-06T07:27:32.147353+0000","last_active":"2024-03-06T07:27:33.162528+0000","last_peered":"2024-03-06T07:27:33.162528+0000","last_clean":"2024-03-06T07:27:33.162528+0000","last_became_active":"2024-03-06T07:27:32.146919+0000","last_became_peered":"2024-03-06T07:27:32.146919+0000","last_unstale":"2024-03-06T07:27:33.162528+0000","last_undegraded":"2024-03-06T07:27:33.162528+0000","last_fullsized":"2024-03-06T07:27:33.162528+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:25.390016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.155698+0000","last_change":"2024-03-06T07:27:33.151234+0000","last_active":"2024-03-06T07:27:33.155698+0000","last_peered":"2024-03-06T07:27:33.155698+0000","last_clean":"2024-03-06T07:27:33.155698+0000","last_became_active":"2024-03-06T07:27:33.150673+0000","last_became_peered":"2024-03-06T07:27:33.150673+0000","last_unstale":"2024-03-06T07:27:33.155698+0000","last_undegraded":"2024-03-06T07:27:33.155698+0000","last_fullsized":"2024-03-06T07:27:33.155698+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:11:46.929783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.131688+0000","last_change":"2024-03-06T07:27:32.144203+0000","last_active":"2024-03-06T07:27:33.131688+0000","last_peered":"2024-03-06T07:27:33.131688+0000","last_clean":"2024-03-06T07:27:33.131688+0000","last_became_active":"2024-03-06T07:27:32.143872+0000","last_became_peered":"2024-03-06T07:27:32.143872+0000","last_unstale":"2024-03-06T07:27:33.131688+0000","last_undegraded":"2024-03-06T07:27:33.131688+0000","last_fullsized":"2024-03-06T07:27:33.131688+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:52.574912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.163540+0000","last_change":"2024-03-06T07:27:33.162500+0000","last_active":"2024-03-06T07:27:33.163540+0000","last_peered":"2024-03-06T07:27:33.163540+0000","last_clean":"2024-03-06T07:27:33.163540+0000","last_became_active":"2024-03-06T07:27:33.162249+0000","last_became_peered":"2024-03-06T07:27:33.162249+0000","last_unstale":"2024-03-06T07:27:33.163540+0000","last_undegraded":"2024-03-06T07:27:33.163540+0000","last_fullsized":"2024-03-06T07:27:33.163540+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:52.021386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.132350+0000","last_change":"2024-03-06T07:27:32.143271+0000","last_active":"2024-03-06T07:27:33.132350+0000","last_peered":"2024-03-06T07:27:33.132350+0000","last_clean":"2024-03-06T07:27:33.132350+0000","last_became_active":"2024-03-06T07:27:32.142807+0000","last_became_peered":"2024-03-06T07:27:32.142807+0000","last_unstale":"2024-03-06T07:27:33.132350+0000","last_undegraded":"2024-03-06T07:27:33.132350+0000","last_fullsized":"2024-03-06T07:27:33.132350+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:58.584433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.146616+0000","last_change":"2024-03-06T07:27:33.146046+0000","last_active":"2024-03-06T07:27:33.146616+0000","last_peered":"2024-03-06T07:27:33.146616+0000","last_clean":"2024-03-06T07:27:33.146616+0000","last_became_active":"2024-03-06T07:27:33.145807+0000","last_became_peered":"2024-03-06T07:27:33.145807+0000","last_unstale":"2024-03-06T07:27:33.146616+0000","last_undegraded":"2024-03-06T07:27:33.146616+0000","last_fullsized":"2024-03-06T07:27:33.146616+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:28.107338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.145141+0000","last_change":"2024-03-06T07:27:32.141653+0000","last_active":"2024-03-06T07:27:33.145141+0000","last_peered":"2024-03-06T07:27:33.145141+0000","last_clean":"2024-03-06T07:27:33.145141+0000","last_became_active":"2024-03-06T07:27:32.141130+0000","last_became_peered":"2024-03-06T07:27:32.141130+0000","last_unstale":"2024-03-06T07:27:33.145141+0000","last_undegraded":"2024-03-06T07:27:33.145141+0000","last_fullsized":"2024-03-06T07:27:33.145141+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:32.713843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128312+0000","last_change":"2024-03-06T07:27:33.157105+0000","last_active":"2024-03-06T07:27:34.128312+0000","last_peered":"2024-03-06T07:27:34.128312+0000","last_clean":"2024-03-06T07:27:34.128312+0000","last_became_active":"2024-03-06T07:27:33.156756+0000","last_became_peered":"2024-03-06T07:27:33.156756+0000","last_unstale":"2024-03-06T07:27:34.128312+0000","last_undegraded":"2024-03-06T07:27:34.128312+0000","last_fullsized":"2024-03-06T07:27:34.128312+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:25:15.451845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.145763+0000","last_change":"2024-03-06T07:27:32.143960+0000","last_active":"2024-03-06T07:27:33.145763+0000","last_peered":"2024-03-06T07:27:33.145763+0000","last_clean":"2024-03-06T07:27:33.145763+0000","last_became_active":"2024-03-06T07:27:32.143013+0000","last_became_peered":"2024-03-06T07:27:32.143013+0000","last_unstale":"2024-03-06T07:27:33.145763+0000","last_undegraded":"2024-03-06T07:27:33.145763+0000","last_fullsized":"2024-03-06T07:27:33.145763+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:53.265639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128400+0000","last_change":"2024-03-06T07:27:33.143010+0000","last_active":"2024-03-06T07:27:34.128400+0000","last_peered":"2024-03-06T07:27:34.128400+0000","last_clean":"2024-03-06T07:27:34.128400+0000","last_became_active":"2024-03-06T07:27:33.142737+0000","last_became_peered":"2024-03-06T07:27:33.142737+0000","last_unstale":"2024-03-06T07:27:34.128400+0000","last_undegraded":"2024-03-06T07:27:34.128400+0000","last_fullsized":"2024-03-06T07:27:34.128400+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:33.082995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.129497+0000","last_change":"2024-03-06T07:27:32.139836+0000","last_active":"2024-03-06T07:27:34.129497+0000","last_peered":"2024-03-06T07:27:34.129497+0000","last_clean":"2024-03-06T07:27:34.129497+0000","last_became_active":"2024-03-06T07:27:32.139329+0000","last_became_peered":"2024-03-06T07:27:32.139329+0000","last_unstale":"2024-03-06T07:27:34.129497+0000","last_undegraded":"2024-03-06T07:27:34.129497+0000","last_fullsized":"2024-03-06T07:27:34.129497+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:09.598887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.144496+0000","last_change":"2024-03-06T07:27:33.143303+0000","last_active":"2024-03-06T07:27:33.144496+0000","last_peered":"2024-03-06T07:27:33.144496+0000","last_clean":"2024-03-06T07:27:33.144496+0000","last_became_active":"2024-03-06T07:27:33.141593+0000","last_became_peered":"2024-03-06T07:27:33.141593+0000","last_unstale":"2024-03-06T07:27:33.144496+0000","last_undegraded":"2024-03-06T07:27:33.144496+0000","last_fullsized":"2024-03-06T07:27:33.144496+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:00.884983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.133014+0000","last_change":"2024-03-06T07:27:32.139094+0000","last_active":"2024-03-06T07:27:33.133014+0000","last_peered":"2024-03-06T07:27:33.133014+0000","last_clean":"2024-03-06T07:27:33.133014+0000","last_became_active":"2024-03-06T07:27:32.138785+0000","last_became_peered":"2024-03-06T07:27:32.138785+0000","last_unstale":"2024-03-06T07:27:33.133014+0000","last_undegraded":"2024-03-06T07:27:33.133014+0000","last_fullsized":"2024-03-06T07:27:33.133014+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:58.584433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.157511+0000","last_change":"2024-03-06T07:27:33.154098+0000","last_active":"2024-03-06T07:27:33.157511+0000","last_peered":"2024-03-06T07:27:33.157511+0000","last_clean":"2024-03-06T07:27:33.157511+0000","last_became_active":"2024-03-06T07:27:33.153776+0000","last_became_peered":"2024-03-06T07:27:33.153776+0000","last_unstale":"2024-03-06T07:27:33.157511+0000","last_undegraded":"2024-03-06T07:27:33.157511+0000","last_fullsized":"2024-03-06T07:27:33.157511+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:55:18.198469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.146265+0000","last_change":"2024-03-06T07:27:32.142549+0000","last_active":"2024-03-06T07:27:33.146265+0000","last_peered":"2024-03-06T07:27:33.146265+0000","last_clean":"2024-03-06T07:27:33.146265+0000","last_became_active":"2024-03-06T07:27:32.142259+0000","last_became_peered":"2024-03-06T07:27:32.142259+0000","last_unstale":"2024-03-06T07:27:33.146265+0000","last_undegraded":"2024-03-06T07:27:33.146265+0000","last_fullsized":"2024-03-06T07:27:33.146265+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:14.370528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:34.128801+0000","last_change":"2024-03-06T07:27:33.160085+0000","last_active":"2024-03-06T07:27:34.128801+0000","last_peered":"2024-03-06T07:27:34.128801+0000","last_clean":"2024-03-06T07:27:34.128801+0000","last_became_active":"2024-03-06T07:27:33.159825+0000","last_became_peered":"2024-03-06T07:27:33.159825+0000","last_unstale":"2024-03-06T07:27:34.128801+0000","last_undegraded":"2024-03-06T07:27:34.128801+0000","last_fullsized":"2024-03-06T07:27:34.128801+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:32.118345+0000","last_clean_scrub_stamp":"2024-03-06T07:27:32.118345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:44.688077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.150880+0000","last_change":"2024-03-06T07:27:32.140356+0000","last_active":"2024-03-06T07:27:33.150880+0000","last_peered":"2024-03-06T07:27:33.150880+0000","last_clean":"2024-03-06T07:27:33.150880+0000","last_became_active":"2024-03-06T07:27:32.139971+0000","last_became_peered":"2024-03-06T07:27:32.139971+0000","last_unstale":"2024-03-06T07:27:33.150880+0000","last_undegraded":"2024-03-06T07:27:33.150880+0000","last_fullsized":"2024-03-06T07:27:33.150880+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:31.113168+0000","last_clean_scrub_stamp":"2024-03-06T07:27:31.113168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:08.399236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":475,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-03-06T07:27:33.132853+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:27:33.132853+0000","last_peered":"2024-03-06T07:27:33.132853+0000","last_clean":"2024-03-06T07:27:33.132853+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:27:33.132853+0000","last_undegraded":"2024-03-06T07:27:33.132853+0000","last_fullsized":"2024-03-06T07:27:33.132853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33524,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338316,"statfs":{"total":96636764160,"available":96602435584,"internally_reserved":0,"allocated":2392064,"data_stored":1718365,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2342912,"data_stored":1129063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1129007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1134107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1134039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2326528,"data_stored":1127941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33448,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338392,"statfs":{"total":96636764160,"available":96602513408,"internally_reserved":0,"allocated":2318336,"data_stored":1127325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2383872,"data_stored":1717693,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:27:36.492 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:27:36.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:27:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 Processor -- start 2024-03-06T07:27:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 -- start start 2024-03-06T07:27:36.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77440c5990 0x7f77440c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440ab170 0x7f774406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f774406c730 0x7f77441523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774406cbe0 con 0x7f77440c5990 2024-03-06T07:27:36.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77440c3ef0 con 0x7f774406c730 2024-03-06T07:27:36.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f7749d19640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77441556d0 con 0x7f77440ab170 2024-03-06T07:27:36.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.635+0000 7f77437fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77440c5990 0x7f77440c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f77437fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77440c5990 0x7f77440c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58600/0 (socket says 172.21.15.26:58600) 2024-03-06T07:27:36.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f77437fe640 1 -- 172.21.15.26:0/4216081093 learned_addr learned my addr 172.21.15.26:0/4216081093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:36.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f7743fff640 1 --2- 172.21.15.26:0/4216081093 >> v2:172.21.15.117:3300/0 conn(0x7f774406c730 0x7f77441523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f7742ffd640 1 --2- 172.21.15.26:0/4216081093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440ab170 0x7f774406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f77437fe640 1 -- 172.21.15.26:0/4216081093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440ab170 msgr2=0x7f774406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:36.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f77437fe640 1 --2- 172.21.15.26:0/4216081093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440ab170 0x7f774406c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f77437fe640 1 -- 172.21.15.26:0/4216081093 >> v2:172.21.15.117:3300/0 conn(0x7f774406c730 msgr2=0x7f77441523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f77437fe640 1 --2- 172.21.15.26:0/4216081093 >> v2:172.21.15.117:3300/0 conn(0x7f774406c730 0x7f77441523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f77437fe640 1 -- 172.21.15.26:0/4216081093 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77441559b0 con 0x7f77440c5990 2024-03-06T07:27:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.636+0000 7f7743fff640 1 --2- 172.21.15.26:0/4216081093 >> v2:172.21.15.117:3300/0 conn(0x7f774406c730 0x7f77441523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f77437fe640 1 --2- 172.21.15.26:0/4216081093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440c5990 0x7f77440c0ab0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f77440c4100 tx=0x7f772c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a973e5af4a0e3d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f77427fc640 1 -- 172.21.15.26:0/4216081093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c03b070 con 0x7f77440c5990 2024-03-06T07:27:36.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f77427fc640 1 -- 172.21.15.26:0/4216081093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c036440 con 0x7f77440c5990 2024-03-06T07:27:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f77427fc640 1 -- 172.21.15.26:0/4216081093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c036810 con 0x7f77440c5990 2024-03-06T07:27:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 -- 172.21.15.26:0/4216081093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440c5990 msgr2=0x7f77440c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 --2- 172.21.15.26:0/4216081093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440c5990 0x7f77440c0ab0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f77440c4100 tx=0x7f772c02e0c0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 -- 172.21.15.26:0/4216081093 shutdown_connections 2024-03-06T07:27:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 --2- 172.21.15.26:0/4216081093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440ab170 0x7f774406c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 --2- 172.21.15.26:0/4216081093 >> v2:172.21.15.117:3300/0 conn(0x7f774406c730 0x7f77441523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 --2- 172.21.15.26:0/4216081093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440c5990 0x7f77440c0ab0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 -- 172.21.15.26:0/4216081093 >> 172.21.15.26:0/4216081093 conn(0x7f77440c6710 msgr2=0x7f77440c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 -- 172.21.15.26:0/4216081093 shutdown_connections 2024-03-06T07:27:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.637+0000 7f7749d19640 1 -- 172.21.15.26:0/4216081093 wait complete. 2024-03-06T07:27:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.638+0000 7f7749d19640 1 Processor -- start 2024-03-06T07:27:36.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7749d19640 1 -- start start 2024-03-06T07:27:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7749d19640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774406c730 0x7f77441488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7749d19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440ab170 0x7f7744148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7749d19640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440c5990 0x7f77441e7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7749d19640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744156490 con 0x7f77440ab170 2024-03-06T07:27:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7749d19640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744155f70 con 0x7f774406c730 2024-03-06T07:27:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7749d19640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7744156610 con 0x7f77440c5990 2024-03-06T07:27:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7742ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440ab170 0x7f7744148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7742ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440ab170 0x7f7744148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58604/0 (socket says 172.21.15.26:58604) 2024-03-06T07:27:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7742ffd640 1 -- 172.21.15.26:0/1356802233 learned_addr learned my addr 172.21.15.26:0/1356802233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f77437fe640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774406c730 0x7f77441488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.639+0000 7f7743fff640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440c5990 0x7f77441e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f77437fe640 1 -- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440c5990 msgr2=0x7f77441e7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f77437fe640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440c5990 0x7f77441e7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f77437fe640 1 -- 172.21.15.26:0/1356802233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440ab170 msgr2=0x7f7744148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f77437fe640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440ab170 0x7f7744148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f77437fe640 1 -- 172.21.15.26:0/1356802233 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744154db0 con 0x7f774406c730 2024-03-06T07:27:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f7742ffd640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440ab170 0x7f7744148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f77437fe640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774406c730 0x7f77441488e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f772c035000 tx=0x7f772c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f773bfff640 1 -- 172.21.15.26:0/1356802233 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c045020 con 0x7f774406c730 2024-03-06T07:27:36.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f773bfff640 1 -- 172.21.15.26:0/1356802233 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c041640 con 0x7f774406c730 2024-03-06T07:27:36.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774414da70 con 0x7f774406c730 2024-03-06T07:27:36.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f773bfff640 1 -- 172.21.15.26:0/1356802233 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c03b040 con 0x7f774406c730 2024-03-06T07:27:36.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.640+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77441542d0 con 0x7f774406c730 2024-03-06T07:27:36.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.641+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774404f330 con 0x7f774406c730 2024-03-06T07:27:36.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.641+0000 7f773bfff640 1 -- 172.21.15.26:0/1356802233 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f772c048020 con 0x7f774406c730 2024-03-06T07:27:36.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.642+0000 7f773bfff640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f772406b4d0 0x7f772406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.642+0000 7f773bfff640 1 -- 172.21.15.26:0/1356802233 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f772c0aea40 con 0x7f774406c730 2024-03-06T07:27:36.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.642+0000 7f7742ffd640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f772406b4d0 0x7f772406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.642+0000 7f7742ffd640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f772406b4d0 0x7f772406d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f774414f2f0 tx=0x7f7734009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.646+0000 7f773bfff640 1 -- 172.21.15.26:0/1356802233 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f772c04f030 con 0x7f774406c730 2024-03-06T07:27:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.895+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f77440c28c0 con 0x7f774406c730 2024-03-06T07:27:36.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.900+0000 7f773bfff640 1 -- 172.21.15.26:0/1356802233 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f772c0aee20 con 0x7f774406c730 2024-03-06T07:27:36.903 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:27:36.903 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:27:36.903 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:27:36.903 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:27:36.903 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:27:36.903 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f772406b4d0 msgr2=0x7f772406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f772406b4d0 0x7f772406d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f774414f2f0 tx=0x7f7734009190 comp rx=0 tx=0).stop 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774406c730 msgr2=0x7f77441488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774406c730 0x7f77441488e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f772c035000 tx=0x7f772c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 shutdown_connections 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f772406b4d0 0x7f772406d990 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77440c5990 0x7f77441e7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77440ab170 0x7f7744148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 --2- 172.21.15.26:0/1356802233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774406c730 0x7f77441488e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.902+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 >> 172.21.15.26:0/1356802233 conn(0x7f77440c6710 msgr2=0x7f77440a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.903+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 shutdown_connections 2024-03-06T07:27:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:36.903+0000 7f7749d19640 1 -- 172.21.15.26:0/1356802233 wait complete. 2024-03-06T07:27:36.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.015+0000 7f0c011f7640 1 Processor -- start 2024-03-06T07:27:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.015+0000 7f0c011f7640 1 -- start start 2024-03-06T07:27:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.015+0000 7f0c011f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bfc14f790 0x7f0bfc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.015+0000 7f0c011f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0bfc155a40 0x7f0bfc155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.015+0000 7f0c011f7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc0c28d0 con 0x7f0bfc0c1aa0 2024-03-06T07:27:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.015+0000 7f0c011f7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc0c2680 con 0x7f0bfc14f790 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.015+0000 7f0c011f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc0c2bb0 con 0x7f0bfc155a40 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bfc14f790 0x7f0bfc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bfc14f790 0x7f0bfc1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58610/0 (socket says 172.21.15.26:58610) 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfad9d640 1 -- 172.21.15.26:0/1575141810 learned_addr learned my addr 172.21.15.26:0/1575141810 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfb59e640 1 --2- 172.21.15.26:0/1575141810 >> v2:172.21.15.117:3300/0 conn(0x7f0bfc155a40 0x7f0bfc155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfad9d640 1 -- 172.21.15.26:0/1575141810 >> v2:172.21.15.117:3300/0 conn(0x7f0bfc155a40 msgr2=0x7f0bfc155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfad9d640 1 --2- 172.21.15.26:0/1575141810 >> v2:172.21.15.117:3300/0 conn(0x7f0bfc155a40 0x7f0bfc155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfad9d640 1 -- 172.21.15.26:0/1575141810 >> v1:172.21.15.117:6789/0 conn(0x7f0bfc0c1aa0 legacy=0x7f0bfc0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.016+0000 7f0bfad9d640 1 -- 172.21.15.26:0/1575141810 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0c2e90 con 0x7f0bfc14f790 2024-03-06T07:27:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0bfad9d640 1 --2- 172.21.15.26:0/1575141810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc14f790 0x7f0bfc1529d0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc06c960 tx=0x7f0bec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510c3d5ab0f06564 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0bfa59c640 1 -- 172.21.15.26:0/1575141810 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec034070 con 0x7f0bfc14f790 2024-03-06T07:27:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0bfa59c640 1 -- 172.21.15.26:0/1575141810 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec030860 con 0x7f0bfc14f790 2024-03-06T07:27:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0bfa59c640 1 -- 172.21.15.26:0/1575141810 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec030c30 con 0x7f0bfc14f790 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 -- 172.21.15.26:0/1575141810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc14f790 msgr2=0x7f0bfc1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 --2- 172.21.15.26:0/1575141810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc14f790 0x7f0bfc1529d0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc06c960 tx=0x7f0bec02b9a0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 -- 172.21.15.26:0/1575141810 shutdown_connections 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 --2- 172.21.15.26:0/1575141810 >> v2:172.21.15.117:3300/0 conn(0x7f0bfc155a40 0x7f0bfc155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 --2- 172.21.15.26:0/1575141810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc14f790 0x7f0bfc1529d0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 -- 172.21.15.26:0/1575141810 >> 172.21.15.26:0/1575141810 conn(0x7f0bfc0c6710 msgr2=0x7f0bfc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 -- 172.21.15.26:0/1575141810 shutdown_connections 2024-03-06T07:27:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.017+0000 7f0c011f7640 1 -- 172.21.15.26:0/1575141810 wait complete. 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 Processor -- start 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 -- start start 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bfc0c1aa0 0x7f0bfc1e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bfc14f790 0x7f0bfc1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc155a40 0x7f0bfc14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0c0d60 con 0x7f0bfc155a40 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc156390 con 0x7f0bfc14f790 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0c011f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0c5e30 con 0x7f0bfc0c1aa0 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0bfb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc155a40 0x7f0bfc14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0bfb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc155a40 0x7f0bfc14b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58616/0 (socket says 172.21.15.26:58616) 2024-03-06T07:27:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0bfb59e640 1 -- 172.21.15.26:0/578625090 learned_addr learned my addr 172.21.15.26:0/578625090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0bfad9d640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bfc0c1aa0 0x7f0bfc1e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.019+0000 7f0bf3fff640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bfc14f790 0x7f0bfc1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bfb59e640 1 -- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bfc0c1aa0 msgr2=0x7f0bfc1e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bfb59e640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bfc0c1aa0 0x7f0bfc1e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bfb59e640 1 -- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bfc14f790 msgr2=0x7f0bfc1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bfb59e640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bfc14f790 0x7f0bfc1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bfb59e640 1 -- 172.21.15.26:0/578625090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc154820 con 0x7f0bfc155a40 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bfb59e640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc155a40 0x7f0bfc14b4e0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400efc0 tx=0x7f0bf400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bf37fe640 1 -- 172.21.15.26:0/578625090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4018020 con 0x7f0bfc155a40 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bf37fe640 1 -- 172.21.15.26:0/578625090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf4014480 con 0x7f0bfc155a40 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc1df040 con 0x7f0bfc155a40 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0bf37fe640 1 -- 172.21.15.26:0/578625090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4014870 con 0x7f0bfc155a40 2024-03-06T07:27:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.020+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc154f20 con 0x7f0bfc155a40 2024-03-06T07:27:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.021+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc04f330 con 0x7f0bfc155a40 2024-03-06T07:27:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.022+0000 7f0bf37fe640 1 -- 172.21.15.26:0/578625090 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bf4005d10 con 0x7f0bfc155a40 2024-03-06T07:27:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.022+0000 7f0bf37fe640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bdc06b500 0x7f0bdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.022+0000 7f0bf37fe640 1 -- 172.21.15.26:0/578625090 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bf408aba0 con 0x7f0bfc155a40 2024-03-06T07:27:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.022+0000 7f0bfad9d640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bdc06b500 0x7f0bdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.022+0000 7f0bfad9d640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bdc06b500 0x7f0bdc06d9c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0bec036000 tx=0x7f0bec008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.026+0000 7f0bf37fe640 1 -- 172.21.15.26:0/578625090 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0bf4022070 con 0x7f0bfc155a40 2024-03-06T07:27:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.243+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bfc0c0b40 con 0x7f0bfc155a40 2024-03-06T07:27:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.244+0000 7f0bf37fe640 1 -- 172.21.15.26:0/578625090 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f0bf4010040 con 0x7f0bfc155a40 2024-03-06T07:27:37.246 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:37.246 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":172,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:34.491446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9867,"qdb_cluster":[9867]},"id":14}]} 2024-03-06T07:27:37.247 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 172 2024-03-06T07:27:37.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.245+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bdc06b500 msgr2=0x7f0bdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.245+0000 7f0c011f7640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bdc06b500 0x7f0bdc06d9c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0bec036000 tx=0x7f0bec008030 comp rx=0 tx=0).stop 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc155a40 msgr2=0x7f0bfc14b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc155a40 0x7f0bfc14b4e0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400efc0 tx=0x7f0bf400c490 comp rx=0 tx=0).stop 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 shutdown_connections 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bdc06b500 0x7f0bdc06d9c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc155a40 0x7f0bfc14b4e0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bfc14f790 0x7f0bfc1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 --2- 172.21.15.26:0/578625090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bfc0c1aa0 0x7f0bfc1e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 >> 172.21.15.26:0/578625090 conn(0x7f0bfc0c6710 msgr2=0x7f0bfc0a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 shutdown_connections 2024-03-06T07:27:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.246+0000 7f0c011f7640 1 -- 172.21.15.26:0/578625090 wait complete. 2024-03-06T07:27:37.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 172, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:34.491446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9867, 'qdb_cluster': [9867]} 2024-03-06T07:27:37.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:27:37.663 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:27:37.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2eb72640 1 Processor -- start 2024-03-06T07:27:37.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2eb72640 1 -- start start 2024-03-06T07:27:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2eb72640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7b280c2b20 0x7f7b280c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2eb72640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b280c5140 con 0x7f7b281587e0 2024-03-06T07:27:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2eb72640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b28155bb0 con 0x7f7b281526c0 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2eb72640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b280c3790 con 0x7f7b280c2b20 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2d10f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7b281587e0 0x7f7b28158c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47402/0 (socket says 172.21.15.26:47402) 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.767+0000 7f7b2d10f640 1 -- 172.21.15.26:0/143136666 learned_addr learned my addr 172.21.15.26:0/143136666 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.768+0000 7f7b27fff640 1 --2- 172.21.15.26:0/143136666 >> v2:172.21.15.117:3300/0 conn(0x7f7b280c2b20 0x7f7b280c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.768+0000 7f7b277fe640 1 -- 172.21.15.26:0/143136666 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409702941 0 0) 0x7f7b280c5140 con 0x7f7b281587e0 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.768+0000 7f7b277fe640 1 -- 172.21.15.26:0/143136666 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b18003520 con 0x7f7b281587e0 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.768+0000 7f7b27fff640 1 -- 172.21.15.26:0/143136666 >> v1:172.21.15.117:6789/0 conn(0x7f7b281526c0 legacy=0x7f7b280c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.768+0000 7f7b27fff640 1 -- 172.21.15.26:0/143136666 >> v1:172.21.15.26:6789/0 conn(0x7f7b281587e0 legacy=0x7f7b28158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.768+0000 7f7b27fff640 1 -- 172.21.15.26:0/143136666 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28155e90 con 0x7f7b280c2b20 2024-03-06T07:27:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b27fff640 1 --2- 172.21.15.26:0/143136666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b280c2b20 0x7f7b280c69b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7b140097a0 tx=0x7f7b1402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3444e5645ed39 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b277fe640 1 -- 172.21.15.26:0/143136666 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b14035020 con 0x7f7b280c2b20 2024-03-06T07:27:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b277fe640 1 -- 172.21.15.26:0/143136666 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b14031830 con 0x7f7b280c2b20 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b277fe640 1 -- 172.21.15.26:0/143136666 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b14031be0 con 0x7f7b280c2b20 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b2eb72640 1 -- 172.21.15.26:0/143136666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b280c2b20 msgr2=0x7f7b280c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/143136666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b280c2b20 0x7f7b280c69b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7b140097a0 tx=0x7f7b1402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b2eb72640 1 -- 172.21.15.26:0/143136666 shutdown_connections 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/143136666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b280c2b20 0x7f7b280c69b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b2eb72640 1 -- 172.21.15.26:0/143136666 >> 172.21.15.26:0/143136666 conn(0x7f7b280aa810 msgr2=0x7f7b280aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b2eb72640 1 -- 172.21.15.26:0/143136666 shutdown_connections 2024-03-06T07:27:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.769+0000 7f7b2eb72640 1 -- 172.21.15.26:0/143136666 wait complete. 2024-03-06T07:27:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.770+0000 7f7b2eb72640 1 Processor -- start 2024-03-06T07:27:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.770+0000 7f7b2eb72640 1 -- start start 2024-03-06T07:27:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2eb72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b280c2b20 0x7f7b2806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2eb72640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b281526c0 0x7f7b28156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2eb72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b281587e0 0x7f7b281470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2eb72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280c9040 con 0x7f7b280c2b20 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2eb72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b28152aa0 con 0x7f7b281587e0 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2eb72640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280c91c0 con 0x7f7b281526c0 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2c90e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b280c2b20 0x7f7b2806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2c90e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b280c2b20 0x7f7b2806c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53034/0 (socket says 172.21.15.26:53034) 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2c90e640 1 -- 172.21.15.26:0/1735131723 learned_addr learned my addr 172.21.15.26:0/1735131723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b281587e0 0x7f7b281470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.771+0000 7f7b27fff640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b281526c0 0x7f7b28156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2c90e640 1 -- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b281526c0 msgr2=0x7f7b28156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b281526c0 0x7f7b28156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2c90e640 1 -- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b281587e0 msgr2=0x7f7b281470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b281587e0 0x7f7b281470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2c90e640 1 -- 172.21.15.26:0/1735131723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28156860 con 0x7f7b280c2b20 2024-03-06T07:27:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b27fff640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b281526c0 0x7f7b28156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b280c2b20 0x7f7b2806c680 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00c430 tx=0x7f7b1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b257fa640 1 -- 172.21.15.26:0/1735131723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0052d0 con 0x7f7b280c2b20 2024-03-06T07:27:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b257fa640 1 -- 172.21.15.26:0/1735131723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c005490 con 0x7f7b280c2b20 2024-03-06T07:27:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2806cec0 con 0x7f7b280c2b20 2024-03-06T07:27:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b257fa640 1 -- 172.21.15.26:0/1735131723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c007260 con 0x7f7b280c2b20 2024-03-06T07:27:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.772+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2806cbc0 con 0x7f7b280c2b20 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.773+0000 7f7b257fa640 1 -- 172.21.15.26:0/1735131723 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b1c007400 con 0x7f7b280c2b20 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.774+0000 7f7b257fa640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0c06b500 0x7f7b0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.774+0000 7f7b257fa640 1 -- 172.21.15.26:0/1735131723 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b1c089860 con 0x7f7b280c2b20 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.774+0000 7f7b27fff640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0c06b500 0x7f7b0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.774+0000 7f7b27fff640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0c06b500 0x7f7b0c06d9c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7b14036000 tx=0x7f7b14008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 Processor -- start 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 -- start start 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffaa9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffaa9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaaa4b0 0x55ecffaa6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecffa106b0 con 0x55ecffaac330 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecff9b9a60 con 0x55ecffaaa4b0 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.775+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecffa0c750 con 0x55ecffaabf50 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffaa9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffaa9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53048/0 (socket says 172.21.15.26:53048) 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 -- 172.21.15.26:0/393553866 learned_addr learned my addr 172.21.15.26:0/393553866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaaa4b0 0x55ecffaa6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffaa9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 -- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 msgr2=0x55ecffaa9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffaa9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 -- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaaa4b0 msgr2=0x55ecffaa6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaaa4b0 0x55ecffaa6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 -- 172.21.15.26:0/393553866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecff9d7f30 con 0x55ecffaac330 2024-03-06T07:27:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b27fff640 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffaa9f70 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009890 tx=0x7f7b14030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b495b418d013a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b127fc640 1 -- 172.21.15.26:0/393553866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b140036d0 con 0x55ecffaac330 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b127fc640 1 -- 172.21.15.26:0/393553866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b14003870 con 0x55ecffaac330 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.776+0000 7f7b127fc640 1 -- 172.21.15.26:0/393553866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b14003c20 con 0x55ecffaac330 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 -- 172.21.15.26:0/393553866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 msgr2=0x55ecffaa9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffaa9f70 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009890 tx=0x7f7b14030000 comp rx=0 tx=0).stop 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 -- 172.21.15.26:0/393553866 shutdown_connections 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaaa4b0 0x55ecffaa6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffaa9f70 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 --2- 172.21.15.26:0/393553866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffaa9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 -- 172.21.15.26:0/393553866 >> 172.21.15.26:0/393553866 conn(0x55ecffaa87d0 msgr2=0x55ecffacd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 -- 172.21.15.26:0/393553866 shutdown_connections 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 -- 172.21.15.26:0/393553866 wait complete. 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.777+0000 7f7b31831740 1 -- start start 2024-03-06T07:27:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 1 Processor -- start 2024-03-06T07:27:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffac9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffafbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffacc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfface230 con 0x55ecffaac330 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecff9ccea0 con 0x55ecffad13e0 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecffacdc70 con 0x55ecffaabf50 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b31831740 10 client.0 _ll_register_callbacks cb 0x55ecff9cc870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffafbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffafbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53062/0 (socket says 172.21.15.26:53062) 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b27fff640 1 -- 172.21.15.26:0/352139465 learned_addr learned my addr 172.21.15.26:0/352139465 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffac9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.778+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffacc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b27fff640 1 -- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 msgr2=0x55ecffac9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b27fff640 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffac9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b27fff640 1 -- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 msgr2=0x55ecffacc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b27fff640 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffacc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b27fff640 1 -- 172.21.15.26:0/352139465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecff9cd270 con 0x55ecffaac330 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b27fff640 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffafbc00 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7b1402b910 tx=0x7f7b14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16991426dd9062d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b077fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b077fe640 1 -- 172.21.15.26:0/352139465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b14003a20 con 0x55ecffaac330 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b077fe640 1 -- 172.21.15.26:0/352139465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b14003bc0 con 0x55ecffaac330 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b14003870 con 0x55ecffaac330 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b077fe640 1 -- 172.21.15.26:0/352139465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b14063050 con 0x55ecffaac330 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 10 client.9996 fetch_fsmap learned FSMap version 172 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=2+}) v3 -- 0x55ecff7252d0 con 0x55ecffaac330 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b077fe640 1 -- 172.21.15.26:0/352139465 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b14003870 con 0x55ecffaac330 2024-03-06T07:27:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b077fe640 1 -- 172.21.15.26:0/352139465 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 172) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f7b14052480 con 0x55ecffaac330 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 10 client.9996 fetch_fsmap finished waiting for FSMap version 172 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 10 client.9996 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 20 client.9996 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffaceef0 0x55ecffb071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 4 client.9996 mds_command: new command op to 9867 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.779+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 --> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecffa1dbf0 con 0x55ecffaceef0 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.780+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffaceef0 0x55ecffb071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.780+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffaceef0 0x55ecffb071a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a4f99a1f6a2041f server_cookie=11a4b58e6284ba71 in_seq=0 out_seq=0 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.780+0000 7f7b077fe640 10 client.9996 ms_handle_connect on v2:172.21.15.117:6834/1623983507 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b077fe640 1 -- 172.21.15.26:0/352139465 <== mds.0 v2:172.21.15.117:6834/1623983507 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ecffa1dbf0 con 0x55ecffaceef0 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b077fe640 10 client.9996 handle_command_reply: tid=0 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 client.9996 shutdown 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 msgr2=0x55ecffafbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffafbc00 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7b1402b910 tx=0x7f7b14004730 comp rx=0 tx=0).stop 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 shutdown_connections 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffaceef0 0x55ecffb071a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffacc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaac330 0x55ecffafbc00 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b077fe640 0 client.9996 ms_handle_reset on v2:172.21.15.117:6834/1623983507 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 --2- 172.21.15.26:0/352139465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaabf50 0x55ecffac9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 >> 172.21.15.26:0/352139465 conn(0x55ecffaaa4b0 msgr2=0x55ecffaa5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 shutdown_connections 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 1 -- 172.21.15.26:0/352139465 wait complete. 2024-03-06T07:27:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.781+0000 7f7b31831740 20 client.9996 trim_cache size 0 max 16384 2024-03-06T07:27:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.798+0000 7f7b31831740 1 Processor -- start 2024-03-06T07:27:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.798+0000 7f7b31831740 1 -- start start 2024-03-06T07:27:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.798+0000 7f7b31831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaabf50 0x55ecffb0c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffaa94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b31831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281f6b70 con 0x55ecffaabf50 2024-03-06T07:27:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281eea40 con 0x55ecffad13e0 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281f6570 con 0x55ecffaac330 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaabf50 0x55ecffb0c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaabf50 0x55ecffb0c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53070/0 (socket says 172.21.15.26:53070) 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 -- 172.21.15.26:0/55566512 learned_addr learned my addr 172.21.15.26:0/55566512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b27fff640 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffaa94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 -- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 msgr2=0x55ecffacbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 -- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 msgr2=0x55ecffaa94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffaa94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 -- 172.21.15.26:0/55566512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecff7ef6b0 con 0x55ecffaabf50 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffaa94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaabf50 0x55ecffb0c1d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c087c00 tx=0x7f7b1c087ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409802df756a3a50 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b11ffb640 1 -- 172.21.15.26:0/55566512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c074070 con 0x55ecffaabf50 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b11ffb640 1 -- 172.21.15.26:0/55566512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c055f10 con 0x55ecffaabf50 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.799+0000 7f7b11ffb640 1 -- 172.21.15.26:0/55566512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0544e0 con 0x55ecffaabf50 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 -- 172.21.15.26:0/55566512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaabf50 msgr2=0x55ecffb0c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaabf50 0x55ecffb0c1d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c087c00 tx=0x7f7b1c087ad0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 -- 172.21.15.26:0/55566512 shutdown_connections 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffad13e0 0x55ecffaa94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 --2- 172.21.15.26:0/55566512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffaabf50 0x55ecffb0c1d0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 -- 172.21.15.26:0/55566512 >> 172.21.15.26:0/55566512 conn(0x55ecffaaa4b0 msgr2=0x55ecffaf8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 -- 172.21.15.26:0/55566512 shutdown_connections 2024-03-06T07:27:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 -- 172.21.15.26:0/55566512 wait complete. 2024-03-06T07:27:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 -- start start 2024-03-06T07:27:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.800+0000 7f7b31831740 1 Processor -- start 2024-03-06T07:27:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaabf50 0x55ecffaa6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b31831740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b31831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffad13e0 0x55ecffb58770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b31831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecffad2680 con 0x55ecffad13e0 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfface230 con 0x55ecffaabf50 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b31831740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecffafc620 con 0x55ecffaac330 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b31831740 10 client.0 _ll_register_callbacks cb 0x7f7b20006cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b2d10f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffad13e0 0x55ecffb58770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b2d10f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffad13e0 0x55ecffb58770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53080/0 (socket says 172.21.15.26:53080) 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b2c90e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaabf50 0x55ecffaa6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b2d10f640 1 -- 172.21.15.26:0/354111609 learned_addr learned my addr 172.21.15.26:0/354111609 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.801+0000 7f7b27fff640 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b2d10f640 1 -- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 msgr2=0x55ecffacf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b2d10f640 1 -- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaabf50 msgr2=0x55ecffaa6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaabf50 0x55ecffaa6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b2d10f640 1 -- 172.21.15.26:0/354111609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecffad0e50 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b2d10f640 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffad13e0 0x55ecffb58770 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f7b20017a10 tx=0x7f7b20017ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9428dbf5bc60daa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b077fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b077fe640 1 -- 172.21.15.26:0/354111609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20007990 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b077fe640 1 -- 172.21.15.26:0/354111609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b20021c80 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b1c055f10 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b077fe640 1 -- 172.21.15.26:0/354111609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b2001f3e0 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 10 client.10002 fetch_fsmap learned FSMap version 172 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=2+}) v3 -- 0x55ecffaab690 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b077fe640 1 -- 172.21.15.26:0/354111609 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c055f10 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b077fe640 1 -- 172.21.15.26:0/354111609 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 172) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f7b20028050 con 0x55ecffad13e0 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 10 client.10002 fetch_fsmap finished waiting for FSMap version 172 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 10 client.10002 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 20 client.10002 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffb0b1c0 0x55ecffb0b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 4 client.10002 mds_command: new command op to 9867 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.802+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 --> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ecffaab9a0 con 0x55ecffb0b1c0 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffb0b1c0 0x55ecffb0b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b077fe640 10 client.10002 ms_handle_connect on v2:172.21.15.117:6834/1623983507 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b2c90e640 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffb0b1c0 0x55ecffb0b5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a08fea4012144a server_cookie=9527d3ac7ed6d9e7 in_seq=0 out_seq=0 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b077fe640 1 -- 172.21.15.26:0/354111609 <== mds.0 v2:172.21.15.117:6834/1623983507 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55ecffaab9a0 con 0x55ecffb0b1c0 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b077fe640 10 client.10002 handle_command_reply: tid=0 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 client.10002 shutdown 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffad13e0 msgr2=0x55ecffb58770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffad13e0 0x55ecffb58770 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f7b20017a10 tx=0x7f7b20017ed0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 shutdown_connections 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecffad13e0 0x55ecffb58770 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecffaac330 0x55ecffacf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:6834/1623983507,v1:172.21.15.117:6836/1623983507] conn(0x55ecffb0b1c0 0x55ecffb0b5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 --2- 172.21.15.26:0/354111609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecffaabf50 0x55ecffaa6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b077fe640 0 client.10002 ms_handle_reset on v2:172.21.15.117:6834/1623983507 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 >> 172.21.15.26:0/354111609 conn(0x55ecffaaa4b0 msgr2=0x55ecffb0c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 shutdown_connections 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 1 -- 172.21.15.26:0/354111609 wait complete. 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.803+0000 7f7b31831740 20 client.10002 trim_cache size 0 max 16384 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:27:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9867, 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.6786868029999997, 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 172, 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 184, 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 184, 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 9.7522959870000001 2024-03-06T07:27:37.814 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0c06b500 msgr2=0x7f7b0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0c06b500 0x7f7b0c06d9c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7b14036000 tx=0x7f7b14008030 comp rx=0 tx=0).stop 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b280c2b20 msgr2=0x7f7b2806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b280c2b20 0x7f7b2806c680 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00c430 tx=0x7f7b1c00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 shutdown_connections 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0c06b500 0x7f7b0c06d9c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b281587e0 0x7f7b281470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b281526c0 0x7f7b28156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 --2- 172.21.15.26:0/1735131723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b280c2b20 0x7f7b2806c680 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 >> 172.21.15.26:0/1735131723 conn(0x7f7b280aa810 msgr2=0x7f7b2814bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 shutdown_connections 2024-03-06T07:27:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.804+0000 7f7b2eb72640 1 -- 172.21.15.26:0/1735131723 wait complete. 2024-03-06T07:27:37.817 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:27:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.920+0000 7f3ded11e640 1 Processor -- start 2024-03-06T07:27:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.920+0000 7f3ded11e640 1 -- start start 2024-03-06T07:27:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ded11e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3de80c67f0 0x7f3de8155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ded11e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ded11e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de8152cb0 con 0x7f3de81528d0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ded11e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de8155ca0 con 0x7f3de80c67f0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ded11e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80c38b0 con 0x7f3de80c4ef0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ddffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3de80c67f0 0x7f3de8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3de6d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3de81528d0 0x7f3de80724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47412/0 (socket says 172.21.15.26:47412) 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3de6d9d640 1 -- 172.21.15.26:0/2172372799 learned_addr learned my addr 172.21.15.26:0/2172372799 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3de759e640 1 --2- 172.21.15.26:0/2172372799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ddffff640 1 -- 172.21.15.26:0/2172372799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 msgr2=0x7f3de8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ddffff640 1 --2- 172.21.15.26:0/2172372799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ddffff640 1 -- 172.21.15.26:0/2172372799 >> v1:172.21.15.26:6789/0 conn(0x7f3de81528d0 legacy=0x7f3de80724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3de659c640 1 -- 172.21.15.26:0/2172372799 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776031682 0 0) 0x7f3de8152cb0 con 0x7f3de81528d0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.921+0000 7f3ddffff640 1 -- 172.21.15.26:0/2172372799 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de81594f0 con 0x7f3de80c67f0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.922+0000 7f3ddffff640 1 --2- 172.21.15.26:0/2172372799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de8155220 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00097a0 tx=0x7f3dd002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc04071ae8cfacda server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.922+0000 7f3de659c640 1 -- 172.21.15.26:0/2172372799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0034070 con 0x7f3de80c67f0 2024-03-06T07:27:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.922+0000 7f3de659c640 1 -- 172.21.15.26:0/2172372799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0030610 con 0x7f3de80c67f0 2024-03-06T07:27:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.922+0000 7f3de659c640 1 -- 172.21.15.26:0/2172372799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00309e0 con 0x7f3de80c67f0 2024-03-06T07:27:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.922+0000 7f3ded11e640 1 -- 172.21.15.26:0/2172372799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 msgr2=0x7f3de8155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.922+0000 7f3ded11e640 1 --2- 172.21.15.26:0/2172372799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de8155220 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00097a0 tx=0x7f3dd002b960 comp rx=0 tx=0).stop 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.923+0000 7f3ded11e640 1 -- 172.21.15.26:0/2172372799 shutdown_connections 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.923+0000 7f3ded11e640 1 --2- 172.21.15.26:0/2172372799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de8155220 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.923+0000 7f3ded11e640 1 --2- 172.21.15.26:0/2172372799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.923+0000 7f3ded11e640 1 -- 172.21.15.26:0/2172372799 >> 172.21.15.26:0/2172372799 conn(0x7f3de80aa6a0 msgr2=0x7f3de80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.923+0000 7f3ded11e640 1 -- 172.21.15.26:0/2172372799 shutdown_connections 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.923+0000 7f3ded11e640 1 -- 172.21.15.26:0/2172372799 wait complete. 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 Processor -- start 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 -- start start 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de81ee130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81528d0 0x7f3de81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80a8250 con 0x7f3de80c67f0 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80c3390 con 0x7f3de81528d0 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.924+0000 7f3ded11e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de81591a0 con 0x7f3de80c4ef0 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de81ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de81ee130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53092/0 (socket says 172.21.15.26:53092) 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 -- 172.21.15.26:0/295593925 learned_addr learned my addr 172.21.15.26:0/295593925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3de6d9d640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3de759e640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81528d0 0x7f3de81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 -- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 msgr2=0x7f3de8151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 -- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81528d0 msgr2=0x7f3de81500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81528d0 0x7f3de81500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 -- 172.21.15.26:0/295593925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8157550 con 0x7f3de80c67f0 2024-03-06T07:27:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddffff640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de81ee130 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000c00 tx=0x7f3dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddf7fe640 1 -- 172.21.15.26:0/295593925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0034040 con 0x7f3de80c67f0 2024-03-06T07:27:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddf7fe640 1 -- 172.21.15.26:0/295593925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0030410 con 0x7f3de80c67f0 2024-03-06T07:27:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de8157830 con 0x7f3de80c67f0 2024-03-06T07:27:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ddf7fe640 1 -- 172.21.15.26:0/295593925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0004250 con 0x7f3de80c67f0 2024-03-06T07:27:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.925+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de81edc10 con 0x7f3de80c67f0 2024-03-06T07:27:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.926+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de804f330 con 0x7f3de80c67f0 2024-03-06T07:27:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.927+0000 7f3ddf7fe640 1 -- 172.21.15.26:0/295593925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3dd0037070 con 0x7f3de80c67f0 2024-03-06T07:27:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.927+0000 7f3ddf7fe640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3dc806b4d0 0x7f3dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.927+0000 7f3ddf7fe640 1 -- 172.21.15.26:0/295593925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3dd00abda0 con 0x7f3de80c67f0 2024-03-06T07:27:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.927+0000 7f3de6d9d640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3dc806b4d0 0x7f3dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.928+0000 7f3de6d9d640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3dc806b4d0 0x7f3dc806d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8000f60 tx=0x7f3dd8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:37.931+0000 7f3ddf7fe640 1 -- 172.21.15.26:0/295593925 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3dd00305e0 con 0x7f3de80c67f0 2024-03-06T07:27:38.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:38.152+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7f3de8155550 con 0x7f3de80c67f0 2024-03-06T07:27:38.408 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-06T07:27:38.405+0000 7f5e660ca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.150+0000 7f3ddf7fe640 1 -- 172.21.15.26:0/295593925 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v173) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3dd007a560 con 0x7f3de80c67f0 2024-03-06T07:27:39.153 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked down. 2024-03-06T07:27:39.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.152+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3dc806b4d0 msgr2=0x7f3dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3dc806b4d0 0x7f3dc806d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8000f60 tx=0x7f3dd8007d50 comp rx=0 tx=0).stop 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 msgr2=0x7f3de81ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de81ee130 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000c00 tx=0x7f3dd00046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 shutdown_connections 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3dc806b4d0 0x7f3dc806d990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81528d0 0x7f3de81500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3de80c67f0 0x7f3de81ee130 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 --2- 172.21.15.26:0/295593925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c4ef0 0x7f3de8151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 >> 172.21.15.26:0/295593925 conn(0x7f3de80aa6a0 msgr2=0x7f3de806db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 shutdown_connections 2024-03-06T07:27:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.153+0000 7f3ded11e640 1 -- 172.21.15.26:0/295593925 wait complete. 2024-03-06T07:27:39.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 Processor -- start 2024-03-06T07:27:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 -- start start 2024-03-06T07:27:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88dc06d9f0 0x7f88dc0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f88dc0c3280 0x7f88dc0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc1551e0 0x7f88dc155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88dc158800 con 0x7f88dc06d9f0 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88dc158ae0 con 0x7f88dc0c3280 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.275+0000 7f88e261c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc158dc0 con 0x7f88dc1551e0 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88dc06d9f0 0x7f88dc0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88dc06d9f0 0x7f88dc0a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53100/0 (socket says 172.21.15.26:53100) 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 -- 172.21.15.26:0/3924978608 learned_addr learned my addr 172.21.15.26:0/3924978608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88e0bb9640 1 --2- 172.21.15.26:0/3924978608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc1551e0 0x7f88dc155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88db7fe640 1 --2- 172.21.15.26:0/3924978608 >> v2:172.21.15.117:3300/0 conn(0x7f88dc0c3280 0x7f88dc0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 -- 172.21.15.26:0/3924978608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc1551e0 msgr2=0x7f88dc155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 --2- 172.21.15.26:0/3924978608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc1551e0 0x7f88dc155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 -- 172.21.15.26:0/3924978608 >> v2:172.21.15.117:3300/0 conn(0x7f88dc0c3280 msgr2=0x7f88dc0c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 --2- 172.21.15.26:0/3924978608 >> v2:172.21.15.117:3300/0 conn(0x7f88dc0c3280 0x7f88dc0c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88dbfff640 1 -- 172.21.15.26:0/3924978608 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc1590a0 con 0x7f88dc06d9f0 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.276+0000 7f88db7fe640 1 --2- 172.21.15.26:0/3924978608 >> v2:172.21.15.117:3300/0 conn(0x7f88dc0c3280 0x7f88dc0c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88dbfff640 1 --2- 172.21.15.26:0/3924978608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06d9f0 0x7f88dc0a7cd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0721a0 tx=0x7f88cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84418da2c66f6bad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88daffd640 1 -- 172.21.15.26:0/3924978608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc02e070 con 0x7f88dc06d9f0 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88daffd640 1 -- 172.21.15.26:0/3924978608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc037070 con 0x7f88dc06d9f0 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88daffd640 1 -- 172.21.15.26:0/3924978608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc031a00 con 0x7f88dc06d9f0 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 -- 172.21.15.26:0/3924978608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06d9f0 msgr2=0x7f88dc0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 --2- 172.21.15.26:0/3924978608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06d9f0 0x7f88dc0a7cd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0721a0 tx=0x7f88cc02b880 comp rx=0 tx=0).stop 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 -- 172.21.15.26:0/3924978608 shutdown_connections 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 --2- 172.21.15.26:0/3924978608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc1551e0 0x7f88dc155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 --2- 172.21.15.26:0/3924978608 >> v2:172.21.15.117:3300/0 conn(0x7f88dc0c3280 0x7f88dc0c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 --2- 172.21.15.26:0/3924978608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06d9f0 0x7f88dc0a7cd0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 -- 172.21.15.26:0/3924978608 >> 172.21.15.26:0/3924978608 conn(0x7f88dc0aa630 msgr2=0x7f88dc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.277+0000 7f88e261c640 1 -- 172.21.15.26:0/3924978608 shutdown_connections 2024-03-06T07:27:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.278+0000 7f88e261c640 1 -- 172.21.15.26:0/3924978608 wait complete. 2024-03-06T07:27:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 Processor -- start 2024-03-06T07:27:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 -- start start 2024-03-06T07:27:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc06d9f0 0x7f88dc1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc0c3280 0x7f88dc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1551e0 0x7f88dc06e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc06df50 con 0x7f88dc0c3280 2024-03-06T07:27:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc06ddd0 con 0x7f88dc1551e0 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88e261c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc0a8210 con 0x7f88dc06d9f0 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.279+0000 7f88db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc0c3280 0x7f88dc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc0c3280 0x7f88dc06e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53114/0 (socket says 172.21.15.26:53114) 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 -- 172.21.15.26:0/1420797620 learned_addr learned my addr 172.21.15.26:0/1420797620 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88e0bb9640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1551e0 0x7f88dc06e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 -- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc06d9f0 msgr2=0x7f88dc1544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88dbfff640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc06d9f0 0x7f88dc1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc06d9f0 0x7f88dc1544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 -- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1551e0 msgr2=0x7f88dc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1551e0 0x7f88dc06e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 -- 172.21.15.26:0/1420797620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc157ff0 con 0x7f88dc0c3280 2024-03-06T07:27:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88db7fe640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc0c3280 0x7f88dc06e0d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f88d000c820 tx=0x7f88d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88d8ff9640 1 -- 172.21.15.26:0/1420797620 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0015070 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88d8ff9640 1 -- 172.21.15.26:0/1420797620 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d0007a30 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88dc1582d0 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88d8ff9640 1 -- 172.21.15.26:0/1420797620 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0007e00 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.280+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88dc06f1a0 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.281+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88dc04f330 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.282+0000 7f88d8ff9640 1 -- 172.21.15.26:0/1420797620 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88d0018070 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.282+0000 7f88d8ff9640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f88c806b4d0 0x7f88c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.282+0000 7f88d8ff9640 1 -- 172.21.15.26:0/1420797620 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f88d008a290 con 0x7f88dc0c3280 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.282+0000 7f88dbfff640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f88c806b4d0 0x7f88c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.283+0000 7f88dbfff640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f88c806b4d0 0x7f88c806d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f88cc036000 tx=0x7f88cc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.286+0000 7f88d8ff9640 1 -- 172.21.15.26:0/1420797620 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f88d0059010 con 0x7f88dc0c3280 2024-03-06T07:27:39.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.502+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88dc073240 con 0x7f88dc0c3280 2024-03-06T07:27:39.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.503+0000 7f88d8ff9640 1 -- 172.21.15.26:0/1420797620 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f88d0058910 con 0x7f88dc0c3280 2024-03-06T07:27:39.506 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:39.506 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:39.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f88c806b4d0 msgr2=0x7f88c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f88c806b4d0 0x7f88c806d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f88cc036000 tx=0x7f88cc008030 comp rx=0 tx=0).stop 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc0c3280 msgr2=0x7f88dc06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc0c3280 0x7f88dc06e0d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f88d000c820 tx=0x7f88d000cc20 comp rx=0 tx=0).stop 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 shutdown_connections 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f88c806b4d0 0x7f88c806d990 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1551e0 0x7f88dc06e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc0c3280 0x7f88dc06e0d0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 --2- 172.21.15.26:0/1420797620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88dc06d9f0 0x7f88dc1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 >> 172.21.15.26:0/1420797620 conn(0x7f88dc0aa630 msgr2=0x7f88dc0c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 shutdown_connections 2024-03-06T07:27:39.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:39.505+0000 7f88e261c640 1 -- 172.21.15.26:0/1420797620 wait complete. 2024-03-06T07:27:39.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:39.921 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:40.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.033+0000 7fc44d302640 1 Processor -- start 2024-03-06T07:27:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.033+0000 7fc44d302640 1 -- start start 2024-03-06T07:27:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc44d302640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 0x7fc448156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc44d302640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc448157340 con 0x7fc44806f7b0 2024-03-06T07:27:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc44d302640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc448156d80 con 0x7fc4481508c0 2024-03-06T07:27:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc44d302640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480c3740 con 0x7fc448072720 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc43e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc44806f7b0 0x7fc4481501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47420/0 (socket says 172.21.15.26:47420) 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc43e7fc640 1 -- 172.21.15.26:0/4292127254 learned_addr learned my addr 172.21.15.26:0/4292127254 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256421933 0 0) 0x7fc448157340 con 0x7fc44806f7b0 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc446ffd640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 0x7fc448156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc434003520 con 0x7fc44806f7b0 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 181594432 0 0) 0x7fc434003520 con 0x7fc44806f7b0 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 msgr2=0x7fc448156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 0x7fc448156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 >> v1:172.21.15.117:6789/0 conn(0x7fc4481508c0 legacy=0x7fc4480c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc448157060 con 0x7fc44806f7b0 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fc430001670 con 0x7fc44806f7b0 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 >> v1:172.21.15.26:6789/0 conn(0x7fc44806f7b0 legacy=0x7fc4481501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 0x7fc4340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc434003f60 0x7fc434004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434004d40 0x7fc4340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480c3740 con 0x7fc434004d40 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43400bb50 con 0x7fc434003f60 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.034+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43400bd50 con 0x7fc448072720 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc4477fe640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434004d40 0x7fc4340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc446ffd640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 0x7fc4340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc43e7fc640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc434003f60 0x7fc434004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc4477fe640 1 -- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 msgr2=0x7fc4340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc4477fe640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 0x7fc4340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc4477fe640 1 -- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc434003f60 msgr2=0x7fc434004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc4477fe640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc434003f60 0x7fc434004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc4477fe640 1 -- 172.21.15.26:0/4292127254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434003520 con 0x7fc434004d40 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.035+0000 7fc446ffd640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc448072720 0x7fc4340039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc4477fe640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434004d40 0x7fc4340051c0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc448072460 tx=0x7fc434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca7e19110f6c2bc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440001a70 con 0x7fc434004d40 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc440003b70 con 0x7fc434004d40 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc4467fc640 1 -- 172.21.15.26:0/4292127254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400050d0 con 0x7fc434004d40 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc44d302640 1 -- 172.21.15.26:0/4292127254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434004d40 msgr2=0x7fc4340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc44d302640 1 --2- 172.21.15.26:0/4292127254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434004d40 0x7fc4340051c0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc448072460 tx=0x7fc434005830 comp rx=0 tx=0).stop 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc4477fe640 1 -- 172.21.15.26:0/4292127254 reap_dead start 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc44d302640 1 -- 172.21.15.26:0/4292127254 shutdown_connections 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.036+0000 7fc44d302640 1 -- 172.21.15.26:0/4292127254 >> 172.21.15.26:0/4292127254 conn(0x7fc4480aa630 msgr2=0x7fc4480aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.037+0000 7fc44d302640 1 -- 172.21.15.26:0/4292127254 shutdown_connections 2024-03-06T07:27:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.037+0000 7fc44d302640 1 -- 172.21.15.26:0/4292127254 wait complete. 2024-03-06T07:27:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.038+0000 7fc44d302640 1 Processor -- start 2024-03-06T07:27:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.038+0000 7fc44d302640 1 -- start start 2024-03-06T07:27:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4481ee3d0 0x7fc4481ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4480c82f0 0x7fc4480c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4480c6c70 0x7fc4481eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc448072c80 con 0x7fc4481ee3d0 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc448072b00 con 0x7fc4480c82f0 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480c42e0 con 0x7fc4480c6c70 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4481ee3d0 0x7fc4481ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4481ee3d0 0x7fc4481ee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53122/0 (socket says 172.21.15.26:53122) 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 -- 172.21.15.26:0/701525757 learned_addr learned my addr 172.21.15.26:0/701525757 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc43e7fc640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4480c82f0 0x7fc4480c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc4477fe640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4480c6c70 0x7fc4481eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 -- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4480c6c70 msgr2=0x7fc4481eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4480c6c70 0x7fc4481eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 -- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4480c82f0 msgr2=0x7fc4480c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4480c82f0 0x7fc4480c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 -- 172.21.15.26:0/701525757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44806d610 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc4477fe640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4480c6c70 0x7fc4481eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc446ffd640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4481ee3d0 0x7fc4481ee7b0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc438011ca0 tx=0x7fc43800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc43ffff640 1 -- 172.21.15.26:0/701525757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43800fd20 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc43ffff640 1 -- 172.21.15.26:0/701525757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438014440 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4481eedc0 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc43ffff640 1 -- 172.21.15.26:0/701525757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438014830 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.039+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44806d330 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.040+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44804f330 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.041+0000 7fc43ffff640 1 -- 172.21.15.26:0/701525757 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc438012040 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.041+0000 7fc43ffff640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.041+0000 7fc43ffff640 1 -- 172.21.15.26:0/701525757 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc43808ab50 con 0x7fc4481ee3d0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.042+0000 7fc43e7fc640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.042+0000 7fc43e7fc640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc42806b4d0 0x7fc42806d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc430001050 tx=0x7fc430009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.046+0000 7fc43ffff640 1 -- 172.21.15.26:0/701525757 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc438019030 con 0x7fc4481ee3d0 2024-03-06T07:27:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.268+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4480aacf0 con 0x7fc4481ee3d0 2024-03-06T07:27:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.269+0000 7fc43ffff640 1 -- 172.21.15.26:0/701525757 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7fc438059280 con 0x7fc4481ee3d0 2024-03-06T07:27:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:41.272 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:41.272 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:41.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc42806b4d0 msgr2=0x7fc42806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc42806b4d0 0x7fc42806d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc430001050 tx=0x7fc430009040 comp rx=0 tx=0).stop 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4481ee3d0 msgr2=0x7fc4481ee7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4481ee3d0 0x7fc4481ee7b0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc438011ca0 tx=0x7fc43800f610 comp rx=0 tx=0).stop 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 shutdown_connections 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4480c6c70 0x7fc4481eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4480c82f0 0x7fc4480c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 --2- 172.21.15.26:0/701525757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4481ee3d0 0x7fc4481ee7b0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 >> 172.21.15.26:0/701525757 conn(0x7fc4480aa630 msgr2=0x7fc4480a85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 shutdown_connections 2024-03-06T07:27:41.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:41.271+0000 7fc44d302640 1 -- 172.21.15.26:0/701525757 wait complete. 2024-03-06T07:27:41.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:41.686 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:42.688 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:42.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:42.734 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:42.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.800+0000 7fd9aa05c640 1 Processor -- start 2024-03-06T07:27:42.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.800+0000 7fd9aa05c640 1 -- start start 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.800+0000 7fd9aa05c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9a40c3280 0x7fd9a40c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.800+0000 7fd9aa05c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd9a41551e0 0x7fd9a41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.800+0000 7fd9aa05c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a4158800 con 0x7fd9a406d9f0 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.800+0000 7fd9aa05c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a4158ae0 con 0x7fd9a40c3280 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.800+0000 7fd9aa05c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a4158dc0 con 0x7fd9a41551e0 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd9a406d9f0 0x7fd9a40a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47428/0 (socket says 172.21.15.26:47428) 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a37fe640 1 -- 172.21.15.26:0/4216621200 learned_addr learned my addr 172.21.15.26:0/4216621200 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a2ffd640 1 --2- 172.21.15.26:0/4216621200 >> v2:172.21.15.26:3300/0 conn(0x7fd9a40c3280 0x7fd9a40c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a3fff640 1 --2- 172.21.15.26:0/4216621200 >> v2:172.21.15.117:3300/0 conn(0x7fd9a41551e0 0x7fd9a41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a27fc640 1 -- 172.21.15.26:0/4216621200 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462321296 0 0) 0x7fd9a4158800 con 0x7fd9a406d9f0 2024-03-06T07:27:42.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a27fc640 1 -- 172.21.15.26:0/4216621200 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd98c003520 con 0x7fd9a406d9f0 2024-03-06T07:27:42.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a2ffd640 1 -- 172.21.15.26:0/4216621200 >> v2:172.21.15.117:3300/0 conn(0x7fd9a41551e0 msgr2=0x7fd9a41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:42.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a2ffd640 1 --2- 172.21.15.26:0/4216621200 >> v2:172.21.15.117:3300/0 conn(0x7fd9a41551e0 0x7fd9a41555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a2ffd640 1 -- 172.21.15.26:0/4216621200 >> v1:172.21.15.26:6789/0 conn(0x7fd9a406d9f0 legacy=0x7fd9a40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a2ffd640 1 -- 172.21.15.26:0/4216621200 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a41590a0 con 0x7fd9a40c3280 2024-03-06T07:27:42.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.801+0000 7fd9a3fff640 1 --2- 172.21.15.26:0/4216621200 >> v2:172.21.15.117:3300/0 conn(0x7fd9a41551e0 0x7fd9a41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:42.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.802+0000 7fd9a2ffd640 1 --2- 172.21.15.26:0/4216621200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a40c3710 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4072f50 tx=0x7fd99802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332b50be74d78a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.802+0000 7fd9a27fc640 1 -- 172.21.15.26:0/4216621200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980315a0 con 0x7fd9a40c3280 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.802+0000 7fd9a27fc640 1 -- 172.21.15.26:0/4216621200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd998037070 con 0x7fd9a40c3280 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.802+0000 7fd9a27fc640 1 -- 172.21.15.26:0/4216621200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998031960 con 0x7fd9a40c3280 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 -- 172.21.15.26:0/4216621200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 msgr2=0x7fd9a40c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/4216621200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a40c3710 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4072f50 tx=0x7fd99802b840 comp rx=0 tx=0).stop 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 -- 172.21.15.26:0/4216621200 shutdown_connections 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/4216621200 >> v2:172.21.15.117:3300/0 conn(0x7fd9a41551e0 0x7fd9a41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/4216621200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a40c3710 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 -- 172.21.15.26:0/4216621200 >> 172.21.15.26:0/4216621200 conn(0x7fd9a40aa630 msgr2=0x7fd9a40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 -- 172.21.15.26:0/4216621200 shutdown_connections 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.803+0000 7fd9aa05c640 1 -- 172.21.15.26:0/4216621200 wait complete. 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.804+0000 7fd9aa05c640 1 Processor -- start 2024-03-06T07:27:42.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.804+0000 7fd9aa05c640 1 -- start start 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9aa05c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a406d9f0 0x7fd9a41ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9aa05c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a4154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9aa05c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a41551e0 0x7fd9a41eab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9aa05c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a406df50 con 0x7fd9a40c3280 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9aa05c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a406ddd0 con 0x7fd9a406d9f0 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9aa05c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40c6910 con 0x7fd9a41551e0 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a4154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a4154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53142/0 (socket says 172.21.15.26:53142) 2024-03-06T07:27:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 -- 172.21.15.26:0/2814755105 learned_addr learned my addr 172.21.15.26:0/2814755105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a37fe640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a406d9f0 0x7fd9a41ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a3fff640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a41551e0 0x7fd9a41eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 -- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a41551e0 msgr2=0x7fd9a41eab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a41551e0 0x7fd9a41eab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 -- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a406d9f0 msgr2=0x7fd9a41ee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a406d9f0 0x7fd9a41ee8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 -- 172.21.15.26:0/2814755105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40ac670 con 0x7fd9a40c3280 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9a2ffd640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a4154560 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fd998009890 tx=0x7fd9980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd98bfff640 1 -- 172.21.15.26:0/2814755105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998032630 con 0x7fd9a40c3280 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd98bfff640 1 -- 172.21.15.26:0/2814755105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9980327f0 con 0x7fd9a40c3280 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a40ac950 con 0x7fd9a40c3280 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.805+0000 7fd98bfff640 1 -- 172.21.15.26:0/2814755105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998032be0 con 0x7fd9a40c3280 2024-03-06T07:27:42.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.806+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a40acf10 con 0x7fd9a40c3280 2024-03-06T07:27:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.806+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a404f330 con 0x7fd9a40c3280 2024-03-06T07:27:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.807+0000 7fd98bfff640 1 -- 172.21.15.26:0/2814755105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd99804c020 con 0x7fd9a40c3280 2024-03-06T07:27:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.807+0000 7fd98bfff640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd99006b500 0x7fd99006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.807+0000 7fd98bfff640 1 -- 172.21.15.26:0/2814755105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd99802f020 con 0x7fd9a40c3280 2024-03-06T07:27:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.808+0000 7fd9a37fe640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd99006b500 0x7fd99006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.808+0000 7fd9a37fe640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd99006b500 0x7fd99006d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd994005880 tx=0x7fd994009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:42.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:42.812+0000 7fd98bfff640 1 -- 172.21.15.26:0/2814755105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd998040070 con 0x7fd9a40c3280 2024-03-06T07:27:43.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.031+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9a4151290 con 0x7fd9a40c3280 2024-03-06T07:27:43.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.032+0000 7fd98bfff640 1 -- 172.21.15.26:0/2814755105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7fd99807acb0 con 0x7fd9a40c3280 2024-03-06T07:27:43.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:43.035 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:43.035 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd99006b500 msgr2=0x7fd99006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd99006b500 0x7fd99006d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd994005880 tx=0x7fd994009040 comp rx=0 tx=0).stop 2024-03-06T07:27:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 msgr2=0x7fd9a4154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a4154560 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fd998009890 tx=0x7fd9980046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 shutdown_connections 2024-03-06T07:27:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd99006b500 0x7fd99006d9c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a41551e0 0x7fd9a41eab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a40c3280 0x7fd9a4154560 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 --2- 172.21.15.26:0/2814755105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a406d9f0 0x7fd9a41ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 >> 172.21.15.26:0/2814755105 conn(0x7fd9a40aa630 msgr2=0x7fd9a4150500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 shutdown_connections 2024-03-06T07:27:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:43.034+0000 7fd9aa05c640 1 -- 172.21.15.26:0/2814755105 wait complete. 2024-03-06T07:27:43.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:43.451 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:44.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:44.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.562+0000 7f2ec4666640 1 Processor -- start 2024-03-06T07:27:44.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.562+0000 7f2ec4666640 1 -- start start 2024-03-06T07:27:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.563+0000 7f2ec4666640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ebc155ba0 0x7f2ebc0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.563+0000 7f2ec4666640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc1591c0 con 0x7f2ebc0c5480 2024-03-06T07:27:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.563+0000 7f2ec4666640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc1594a0 con 0x7f2ebc1586f0 2024-03-06T07:27:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.563+0000 7f2ec4666640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc06df40 con 0x7f2ebc155ba0 2024-03-06T07:27:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec2402640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2ebc0c5480 0x7f2ebc06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47444/0 (socket says 172.21.15.26:47444) 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec2402640 1 -- 172.21.15.26:0/898426654 learned_addr learned my addr 172.21.15.26:0/898426654 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1c01640 1 --2- 172.21.15.26:0/898426654 >> v2:172.21.15.117:3300/0 conn(0x7f2ebc155ba0 0x7f2ebc0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783209920 0 0) 0x7f2ebc1591c0 con 0x7f2ebc0c5480 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea4003520 con 0x7f2ebc0c5480 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233203720 0 0) 0x7f2ea4003520 con 0x7f2ebc0c5480 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 >> v2:172.21.15.117:3300/0 conn(0x7f2ebc155ba0 msgr2=0x7f2ebc0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 --2- 172.21.15.26:0/898426654 >> v2:172.21.15.117:3300/0 conn(0x7f2ebc155ba0 0x7f2ebc0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 >> v1:172.21.15.117:6789/0 conn(0x7f2ebc1586f0 legacy=0x7f2ebc15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc1553a0 con 0x7f2ebc0c5480 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f2eb4001670 con 0x7f2ebc0c5480 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 >> v1:172.21.15.26:6789/0 conn(0x7f2ebc0c5480 legacy=0x7f2ebc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ea4004800 0x7f2ea4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ea40051c0 0x7f2ea400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06df40 con 0x7f2ea40039b0 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea400c160 con 0x7f2ea4004800 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.564+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea400c360 con 0x7f2ea40051c0 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2402640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec1c01640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ea4004800 0x7f2ea4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2c03640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ea40051c0 0x7f2ea400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2402640 1 -- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ea40051c0 msgr2=0x7f2ea400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2402640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ea40051c0 0x7f2ea400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2402640 1 -- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ea4004800 msgr2=0x7f2ea4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2402640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ea4004800 0x7f2ea4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2402640 1 -- 172.21.15.26:0/898426654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea4003520 con 0x7f2ea40039b0 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.565+0000 7f2ec2c03640 1 -- 172.21.15.26:0/898426654 reap_dead start 2024-03-06T07:27:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.566+0000 7f2ec2402640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c240 tx=0x7f2ea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14a61ed8fa3108 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:44.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.566+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4007d10 con 0x7f2ea40039b0 2024-03-06T07:27:44.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.566+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4030070 con 0x7f2ea40039b0 2024-03-06T07:27:44.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.566+0000 7f2ec1400640 1 -- 172.21.15.26:0/898426654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb402cda0 con 0x7f2ea40039b0 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.566+0000 7f2ec4666640 1 -- 172.21.15.26:0/898426654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 msgr2=0x7f2ea4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.566+0000 7f2ec4666640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c240 tx=0x7f2ea4005830 comp rx=0 tx=0).stop 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.567+0000 7f2ec4666640 1 -- 172.21.15.26:0/898426654 shutdown_connections 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.567+0000 7f2ec4666640 1 --2- 172.21.15.26:0/898426654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.567+0000 7f2ec4666640 1 -- 172.21.15.26:0/898426654 >> 172.21.15.26:0/898426654 conn(0x7f2ebc06b1a0 msgr2=0x7f2ebc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.567+0000 7f2ec4666640 1 -- 172.21.15.26:0/898426654 shutdown_connections 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.567+0000 7f2ec4666640 1 -- 172.21.15.26:0/898426654 wait complete. 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.568+0000 7f2ec4666640 1 Processor -- start 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.568+0000 7f2ec4666640 1 -- start start 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.568+0000 7f2ec4666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ebc14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.568+0000 7f2ec4666640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0c4920 0x7f2ebc0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec4666640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0c9220 0x7f2ebc1eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec4666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc156000 con 0x7f2ea40039b0 2024-03-06T07:27:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec4666640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06fdb0 con 0x7f2ebc0c4920 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec4666640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc152a20 con 0x7f2ebc0c9220 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ebc14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ebc14db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53156/0 (socket says 172.21.15.26:53156) 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 -- 172.21.15.26:0/3313706280 learned_addr learned my addr 172.21.15.26:0/3313706280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec1c01640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0c4920 0x7f2ebc0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2c03640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0c9220 0x7f2ebc1eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 -- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0c9220 msgr2=0x7f2ebc1eaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0c9220 0x7f2ebc1eaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 -- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0c4920 msgr2=0x7f2ebc0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0c4920 0x7f2ebc0c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 -- 172.21.15.26:0/3313706280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc1ee0e0 con 0x7f2ea40039b0 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec1c01640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0c4920 0x7f2ebc0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec2402640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ebc14db60 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c360 tx=0x7f2eb402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ebaffd640 1 -- 172.21.15.26:0/3313706280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4045070 con 0x7f2ea40039b0 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ebaffd640 1 -- 172.21.15.26:0/3313706280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4004bf0 con 0x7f2ea40039b0 2024-03-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc0aae90 con 0x7f2ea40039b0 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ebaffd640 1 -- 172.21.15.26:0/3313706280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4007070 con 0x7f2ea40039b0 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.569+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc1ee630 con 0x7f2ea40039b0 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.570+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc0c5c90 con 0x7f2ea40039b0 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.571+0000 7f2ebaffd640 1 -- 172.21.15.26:0/3313706280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2eb4007210 con 0x7f2ea40039b0 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.571+0000 7f2ebaffd640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.571+0000 7f2ebaffd640 1 -- 172.21.15.26:0/3313706280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2eb40ab360 con 0x7f2ea40039b0 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.571+0000 7f2ec1c01640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.572+0000 7f2ec1c01640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc155190 tx=0x7f2eb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.576+0000 7f2ebaffd640 1 -- 172.21.15.26:0/3313706280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ebc0c5c90 con 0x7f2ea40039b0 2024-03-06T07:27:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.793+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ebc04f330 con 0x7f2ea40039b0 2024-03-06T07:27:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.794+0000 7f2ebaffd640 1 -- 172.21.15.26:0/3313706280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f2eb4079a90 con 0x7f2ea40039b0 2024-03-06T07:27:44.796 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:44.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 msgr2=0x7f2ea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc155190 tx=0x7f2eb0008040 comp rx=0 tx=0).stop 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 msgr2=0x7f2ebc14db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ebc14db60 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c360 tx=0x7f2eb402e070 comp rx=0 tx=0).stop 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 shutdown_connections 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ea806b4d0 0x7f2ea806d990 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0c9220 0x7f2ebc1eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0c4920 0x7f2ebc0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 --2- 172.21.15.26:0/3313706280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ea40039b0 0x7f2ebc14db60 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 >> 172.21.15.26:0/3313706280 conn(0x7f2ebc06b1a0 msgr2=0x7f2ebc06ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 shutdown_connections 2024-03-06T07:27:44.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:44.796+0000 7f2ec4666640 1 -- 172.21.15.26:0/3313706280 wait complete. 2024-03-06T07:27:45.211 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:45.211 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:46.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:46.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.326+0000 7f73c2fd0640 1 Processor -- start 2024-03-06T07:27:46.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.326+0000 7f73c2fd0640 1 -- start start 2024-03-06T07:27:46.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.327+0000 7f73c2fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc0c1aa0 0x7f73bc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.327+0000 7f73c2fd0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73bc155a40 0x7f73bc155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.327+0000 7f73c2fd0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc156410 con 0x7f73bc14f790 2024-03-06T07:27:46.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.327+0000 7f73c2fd0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73bc0c2680 con 0x7f73bc155a40 2024-03-06T07:27:46.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.327+0000 7f73c2fd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0c28d0 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.327+0000 7f73c156d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73bc155a40 0x7f73bc155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.327+0000 7f73bbfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc0c1aa0 0x7f73bc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73c0d6c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f73bc14f790 0x7f73bc0c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:43790/0 (socket says 172.21.15.26:43790) 2024-03-06T07:27:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73bbfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc0c1aa0 0x7f73bc1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:41236/0 (socket says 172.21.15.26:41236) 2024-03-06T07:27:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73c0d6c640 1 -- 172.21.15.26:0/3931465510 learned_addr learned my addr 172.21.15.26:0/3931465510 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73c156d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73bc155a40 0x7f73bc155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:46560/0 (socket says 172.21.15.26:46560) 2024-03-06T07:27:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73c156d640 1 -- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc0c1aa0 msgr2=0x7f73bc1529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:46.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73c156d640 1 --2- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc0c1aa0 0x7f73bc1529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73c156d640 1 -- 172.21.15.26:0/3931465510 >> v1:172.21.15.117:6789/0 conn(0x7f73bc14f790 legacy=0x7f73bc0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73c156d640 1 -- 172.21.15.26:0/3931465510 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0c2bb0 con 0x7f73bc155a40 2024-03-06T07:27:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.328+0000 7f73bbfff640 1 --2- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc0c1aa0 0x7f73bc1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.329+0000 7f73c156d640 1 --2- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc155ed0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f73a40097a0 tx=0x7f73a402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8468724bb8e7e71 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:46.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.329+0000 7f73bb7fe640 1 -- 172.21.15.26:0/3931465510 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4035020 con 0x7f73bc155a40 2024-03-06T07:27:46.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.329+0000 7f73bb7fe640 1 -- 172.21.15.26:0/3931465510 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a4031830 con 0x7f73bc155a40 2024-03-06T07:27:46.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.329+0000 7f73bb7fe640 1 -- 172.21.15.26:0/3931465510 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4031c00 con 0x7f73bc155a40 2024-03-06T07:27:46.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 -- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 msgr2=0x7f73bc155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc155ed0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f73a40097a0 tx=0x7f73a402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 -- 172.21.15.26:0/3931465510 shutdown_connections 2024-03-06T07:27:46.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc155ed0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/3931465510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc0c1aa0 0x7f73bc1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 -- 172.21.15.26:0/3931465510 >> 172.21.15.26:0/3931465510 conn(0x7f73bc0c6710 msgr2=0x7f73bc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 -- 172.21.15.26:0/3931465510 shutdown_connections 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.330+0000 7f73c2fd0640 1 -- 172.21.15.26:0/3931465510 wait complete. 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.331+0000 7f73c2fd0640 1 Processor -- start 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.331+0000 7f73c2fd0640 1 -- start start 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c2fd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc0c1aa0 0x7f73bc1ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c2fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc14f790 0x7f73bc0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c2fd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc14be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c2fd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0c0d60 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c2fd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc155520 con 0x7f73bc155a40 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c2fd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0c5e30 con 0x7f73bc14f790 2024-03-06T07:27:46.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc0c1aa0 0x7f73bc1ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc0c1aa0 0x7f73bc1ecbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53166/0 (socket says 172.21.15.26:53166) 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 -- 172.21.15.26:0/2775677322 learned_addr learned my addr 172.21.15.26:0/2775677322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73bbfff640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc14f790 0x7f73bc0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c156d640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc14be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 -- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc14f790 msgr2=0x7f73bc0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc14f790 0x7f73bc0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 -- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 msgr2=0x7f73bc14be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc14be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c0d6c640 1 -- 172.21.15.26:0/2775677322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc1513c0 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.332+0000 7f73c156d640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc14be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.333+0000 7f73c0d6c640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc0c1aa0 0x7f73bc1ecbe0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f73b400cb40 tx=0x7f73b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.333+0000 7f73b97fa640 1 -- 172.21.15.26:0/2775677322 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4017070 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.333+0000 7f73b97fa640 1 -- 172.21.15.26:0/2775677322 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b400aca0 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.333+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc1516a0 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.333+0000 7f73b97fa640 1 -- 172.21.15.26:0/2775677322 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4012620 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.333+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc0c4dc0 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.334+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc04f330 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.334+0000 7f73b97fa640 1 -- 172.21.15.26:0/2775677322 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73b401a070 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.335+0000 7f73b97fa640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f739c06b4d0 0x7f739c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.335+0000 7f73b97fa640 1 -- 172.21.15.26:0/2775677322 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73b408a060 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.335+0000 7f73bbfff640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f739c06b4d0 0x7f739c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.335+0000 7f73bbfff640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f739c06b4d0 0x7f739c06d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009560 tx=0x7f73ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:46.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.339+0000 7f73b97fa640 1 -- 172.21.15.26:0/2775677322 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73b4058de0 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.558+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73bc06c770 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.559+0000 7f73b97fa640 1 -- 172.21.15.26:0/2775677322 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f73b40586e0 con 0x7f73bc0c1aa0 2024-03-06T07:27:46.561 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:46.561 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f739c06b4d0 msgr2=0x7f739c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f739c06b4d0 0x7f739c06d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009560 tx=0x7f73ac008040 comp rx=0 tx=0).stop 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc0c1aa0 msgr2=0x7f73bc1ecbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc0c1aa0 0x7f73bc1ecbe0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f73b400cb40 tx=0x7f73b400a060 comp rx=0 tx=0).stop 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 shutdown_connections 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f739c06b4d0 0x7f739c06d990 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73bc155a40 0x7f73bc14be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73bc14f790 0x7f73bc0c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 --2- 172.21.15.26:0/2775677322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc0c1aa0 0x7f73bc1ecbe0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 >> 172.21.15.26:0/2775677322 conn(0x7f73bc0c6710 msgr2=0x7f73bc0a6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 shutdown_connections 2024-03-06T07:27:46.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:46.561+0000 7f73c2fd0640 1 -- 172.21.15.26:0/2775677322 wait complete. 2024-03-06T07:27:46.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:46.975 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:47.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.089+0000 7f9d3858a640 1 Processor -- start 2024-03-06T07:27:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.089+0000 7f9d3858a640 1 -- start start 2024-03-06T07:27:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d3858a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 0x7f9d30159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d3858a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9d300c7180 0x7f9d300c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d3858a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d300c7c60 con 0x7f9d30155b20 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d3858a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d30159640 con 0x7f9d300c7180 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d3858a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300c8df0 con 0x7f9d301527b0 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d36b27640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d30155b20 0x7f9d300c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39370/0 (socket says 172.21.15.26:39370) 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d36b27640 1 -- 172.21.15.26:0/1101965479 learned_addr learned my addr 172.21.15.26:0/1101965479 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d35b25640 1 --2- 172.21.15.26:0/1101965479 >> v2:172.21.15.117:3300/0 conn(0x7f9d300c7180 0x7f9d300c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d36326640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 0x7f9d30159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18068828 0 0) 0x7f9d28001490 con 0x7f9d30155b20 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.090+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d1c003520 con 0x7f9d30155b20 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562820329 0 0) 0x7f9d1c003520 con 0x7f9d30155b20 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 msgr2=0x7f9d30159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 0x7f9d30159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 >> v2:172.21.15.117:3300/0 conn(0x7f9d300c7180 msgr2=0x7f9d300c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 --2- 172.21.15.26:0/1101965479 >> v2:172.21.15.117:3300/0 conn(0x7f9d300c7180 0x7f9d300c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d300c8060 con 0x7f9d30155b20 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f9d28001bc0 con 0x7f9d30155b20 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 >> v1:172.21.15.26:6789/0 conn(0x7f9d30155b20 legacy=0x7f9d300c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d1c0039b0 0x7f9d1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 0x7f9d1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30159640 con 0x7f9d1c004800 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300c8df0 con 0x7f9d1c0039b0 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c00b9c0 con 0x7f9d301527b0 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35b25640 1 --2- 172.21.15.26:0/1101965479 >> v2:172.21.15.117:3300/0 conn(0x7f9d300c7180 0x7f9d300c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d36326640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d1c0039b0 0x7f9d1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35b25640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d36b27640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 0x7f9d1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35b25640 1 -- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 msgr2=0x7f9d1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35b25640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301527b0 0x7f9d1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35b25640 1 -- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d1c0039b0 msgr2=0x7f9d1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35b25640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d1c0039b0 0x7f9d1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.091+0000 7f9d35b25640 1 -- 172.21.15.26:0/1101965479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c003520 con 0x7f9d1c004800 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.092+0000 7f9d35b25640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009ca0 tx=0x7f9d1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f13133ebe5d2ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.092+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c009840 con 0x7f9d1c004800 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.092+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c004020 con 0x7f9d1c004800 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.092+0000 7f9d35324640 1 -- 172.21.15.26:0/1101965479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004410 con 0x7f9d1c004800 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.093+0000 7f9d3858a640 1 -- 172.21.15.26:0/1101965479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 msgr2=0x7f9d1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.093+0000 7f9d3858a640 1 --2- 172.21.15.26:0/1101965479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009ca0 tx=0x7f9d1c005830 comp rx=0 tx=0).stop 2024-03-06T07:27:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.093+0000 7f9d36b27640 1 -- 172.21.15.26:0/1101965479 reap_dead start 2024-03-06T07:27:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.093+0000 7f9d3858a640 1 -- 172.21.15.26:0/1101965479 shutdown_connections 2024-03-06T07:27:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.093+0000 7f9d3858a640 1 -- 172.21.15.26:0/1101965479 >> 172.21.15.26:0/1101965479 conn(0x7f9d300ac560 msgr2=0x7f9d300ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.093+0000 7f9d3858a640 1 -- 172.21.15.26:0/1101965479 shutdown_connections 2024-03-06T07:27:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.093+0000 7f9d3858a640 1 -- 172.21.15.26:0/1101965479 wait complete. 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.094+0000 7f9d3858a640 1 Processor -- start 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.094+0000 7f9d3858a640 1 -- start start 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d30157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d30158470 0x7f9d301ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d301ee350 0x7f9d30154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30155320 con 0x7f9d1c004800 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300c7980 con 0x7f9d301ee350 2024-03-06T07:27:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30155600 con 0x7f9d30158470 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d30157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d30157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50346/0 (socket says 172.21.15.26:50346) 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 -- 172.21.15.26:0/2783973066 learned_addr learned my addr 172.21.15.26:0/2783973066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d35b25640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d30158470 0x7f9d301ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36b27640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d301ee350 0x7f9d30154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 -- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d30158470 msgr2=0x7f9d301ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d30158470 0x7f9d301ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 -- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d301ee350 msgr2=0x7f9d30154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d301ee350 0x7f9d30154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 -- 172.21.15.26:0/2783973066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d30158ea0 con 0x7f9d1c004800 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d36326640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d30157290 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f9d24011940 tx=0x7f9d24011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d22ffd640 1 -- 172.21.15.26:0/2783973066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400fa90 con 0x7f9d1c004800 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d22ffd640 1 -- 172.21.15.26:0/2783973066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2400fc30 con 0x7f9d1c004800 2024-03-06T07:27:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d301593f0 con 0x7f9d1c004800 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d22ffd640 1 -- 172.21.15.26:0/2783973066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24014440 con 0x7f9d1c004800 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.095+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d30150070 con 0x7f9d1c004800 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.097+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d30152b90 con 0x7f9d1c004800 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.097+0000 7f9d22ffd640 1 -- 172.21.15.26:0/2783973066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d24012040 con 0x7f9d1c004800 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.097+0000 7f9d22ffd640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d1406b4d0 0x7f9d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.097+0000 7f9d22ffd640 1 -- 172.21.15.26:0/2783973066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d2408a970 con 0x7f9d1c004800 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.098+0000 7f9d35b25640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d1406b4d0 0x7f9d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.098+0000 7f9d35b25640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d1406b4d0 0x7f9d1406d990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c035000 tx=0x7f9d2c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.102+0000 7f9d22ffd640 1 -- 172.21.15.26:0/2783973066 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d30152b90 con 0x7f9d1c004800 2024-03-06T07:27:48.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.321+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d3004f330 con 0x7f9d1c004800 2024-03-06T07:27:48.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.321+0000 7f9d22ffd640 1 -- 172.21.15.26:0/2783973066 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f9d24019030 con 0x7f9d1c004800 2024-03-06T07:27:48.323 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:48.323 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:48.324 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:48.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d1406b4d0 msgr2=0x7f9d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d1406b4d0 0x7f9d1406d990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c035000 tx=0x7f9d2c030000 comp rx=0 tx=0).stop 2024-03-06T07:27:48.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 msgr2=0x7f9d30157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d30157290 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f9d24011940 tx=0x7f9d24011e00 comp rx=0 tx=0).stop 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 shutdown_connections 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d1406b4d0 0x7f9d1406d990 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d301ee350 0x7f9d30154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d30158470 0x7f9d301ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 --2- 172.21.15.26:0/2783973066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c004800 0x7f9d30157290 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 >> 172.21.15.26:0/2783973066 conn(0x7f9d300ac560 msgr2=0x7f9d3006c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.323+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 shutdown_connections 2024-03-06T07:27:48.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:48.324+0000 7f9d3858a640 1 -- 172.21.15.26:0/2783973066 wait complete. 2024-03-06T07:27:48.738 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:48.738 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:49.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.850+0000 7f668d4c9640 1 Processor -- start 2024-03-06T07:27:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.850+0000 7f668d4c9640 1 -- start start 2024-03-06T07:27:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f668d4c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66880c1aa0 0x7f66881529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f668d4c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6688155a40 0x7f6688155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f668d4c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6688156410 con 0x7f668814f790 2024-03-06T07:27:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f668d4c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66880c2680 con 0x7f6688155a40 2024-03-06T07:27:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f668d4c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c28d0 con 0x7f66880c1aa0 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f6686ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f668814f790 0x7f66880c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39380/0 (socket says 172.21.15.26:39380) 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f6686ffd640 1 -- 172.21.15.26:0/957849936 learned_addr learned my addr 172.21.15.26:0/957849936 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f66867fc640 1 --2- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66880c1aa0 0x7f66881529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f66877fe640 1 --2- 172.21.15.26:0/957849936 >> v2:172.21.15.117:3300/0 conn(0x7f6688155a40 0x7f6688155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f6685ffb640 1 -- 172.21.15.26:0/957849936 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382935011 0 0) 0x7f6688156410 con 0x7f668814f790 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f6685ffb640 1 -- 172.21.15.26:0/957849936 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6670003520 con 0x7f668814f790 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f6685ffb640 1 -- 172.21.15.26:0/957849936 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 909431980 0 0) 0x7f6670003520 con 0x7f668814f790 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f66877fe640 1 -- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66880c1aa0 msgr2=0x7f66881529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.851+0000 7f66877fe640 1 --2- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66880c1aa0 0x7f66881529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.852+0000 7f66877fe640 1 -- 172.21.15.26:0/957849936 >> v1:172.21.15.26:6789/0 conn(0x7f668814f790 legacy=0x7f66880c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.852+0000 7f66877fe640 1 -- 172.21.15.26:0/957849936 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880c2bb0 con 0x7f6688155a40 2024-03-06T07:27:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.852+0000 7f66867fc640 1 --2- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66880c1aa0 0x7f66881529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.852+0000 7f66877fe640 1 --2- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688155a40 0x7f6688155ed0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6680009d90 tx=0x7f668002bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ff2c768787481e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f6685ffb640 1 -- 172.21.15.26:0/957849936 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680039070 con 0x7f6688155a40 2024-03-06T07:27:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f6685ffb640 1 -- 172.21.15.26:0/957849936 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668002c8a0 con 0x7f6688155a40 2024-03-06T07:27:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f6685ffb640 1 -- 172.21.15.26:0/957849936 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678001670 con 0x7f6688155a40 2024-03-06T07:27:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 -- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688155a40 msgr2=0x7f6688155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 --2- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688155a40 0x7f6688155ed0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6680009d90 tx=0x7f668002bd70 comp rx=0 tx=0).stop 2024-03-06T07:27:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 -- 172.21.15.26:0/957849936 shutdown_connections 2024-03-06T07:27:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 --2- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6688155a40 0x7f6688155ed0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 --2- 172.21.15.26:0/957849936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66880c1aa0 0x7f66881529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 -- 172.21.15.26:0/957849936 >> 172.21.15.26:0/957849936 conn(0x7f66880c6710 msgr2=0x7f66880c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 -- 172.21.15.26:0/957849936 shutdown_connections 2024-03-06T07:27:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.853+0000 7f668d4c9640 1 -- 172.21.15.26:0/957849936 wait complete. 2024-03-06T07:27:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 Processor -- start 2024-03-06T07:27:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 -- start start 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c1aa0 0x7f66880c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f668814f790 0x7f66881ec700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6688155a40 0x7f66881ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c0d60 con 0x7f6688155a40 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6688155520 con 0x7f66880c1aa0 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f668d4c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c5e30 con 0x7f668814f790 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f66877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6688155a40 0x7f66881ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f66877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6688155a40 0x7f66881ecc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50362/0 (socket says 172.21.15.26:50362) 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.855+0000 7f66877fe640 1 -- 172.21.15.26:0/4186989105 learned_addr learned my addr 172.21.15.26:0/4186989105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66867fc640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f668814f790 0x7f66881ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f6686ffd640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c1aa0 0x7f66880c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66877fe640 1 -- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f668814f790 msgr2=0x7f66881ec700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66877fe640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f668814f790 0x7f66881ec700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66877fe640 1 -- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c1aa0 msgr2=0x7f66880c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66877fe640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c1aa0 0x7f66880c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66877fe640 1 -- 172.21.15.26:0/4186989105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66881dec10 con 0x7f6688155a40 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f6686ffd640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c1aa0 0x7f66880c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66867fc640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f668814f790 0x7f66881ec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f66877fe640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6688155a40 0x7f66881ecc40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6680009d90 tx=0x7f6680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f666f7fe640 1 -- 172.21.15.26:0/4186989105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680039050 con 0x7f6688155a40 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f666f7fe640 1 -- 172.21.15.26:0/4186989105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680045070 con 0x7f6688155a40 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66881df3e0 con 0x7f6688155a40 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f666f7fe640 1 -- 172.21.15.26:0/4186989105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680009450 con 0x7f6688155a40 2024-03-06T07:27:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.856+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66881df140 con 0x7f6688155a40 2024-03-06T07:27:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.857+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668804f330 con 0x7f6688155a40 2024-03-06T07:27:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.858+0000 7f666f7fe640 1 -- 172.21.15.26:0/4186989105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6680040690 con 0x7f6688155a40 2024-03-06T07:27:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.858+0000 7f666f7fe640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b4d0 0x7f667406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.858+0000 7f666f7fe640 1 -- 172.21.15.26:0/4186989105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f66800acb80 con 0x7f6688155a40 2024-03-06T07:27:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.858+0000 7f6686ffd640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b4d0 0x7f667406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.859+0000 7f6686ffd640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b4d0 0x7f667406d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6688152160 tx=0x7f6678009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:49.862+0000 7f666f7fe640 1 -- 172.21.15.26:0/4186989105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6680032340 con 0x7f6688155a40 2024-03-06T07:27:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.080+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66880c4000 con 0x7f6688155a40 2024-03-06T07:27:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.081+0000 7f666f7fe640 1 -- 172.21.15.26:0/4186989105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f668007b900 con 0x7f6688155a40 2024-03-06T07:27:50.083 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:50.083 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b4d0 msgr2=0x7f667406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b4d0 0x7f667406d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6688152160 tx=0x7f6678009040 comp rx=0 tx=0).stop 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6688155a40 msgr2=0x7f66881ecc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6688155a40 0x7f66881ecc40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6680009d90 tx=0x7f6680004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 shutdown_connections 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f667406b4d0 0x7f667406d990 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6688155a40 0x7f66881ecc40 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f668814f790 0x7f66881ec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 --2- 172.21.15.26:0/4186989105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66880c1aa0 0x7f66880c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 >> 172.21.15.26:0/4186989105 conn(0x7f66880c6710 msgr2=0x7f668806dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 shutdown_connections 2024-03-06T07:27:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:50.083+0000 7f668d4c9640 1 -- 172.21.15.26:0/4186989105 wait complete. 2024-03-06T07:27:50.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:50.498 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:51.499 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:51.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.611+0000 7febca39a640 1 Processor -- start 2024-03-06T07:27:51.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.611+0000 7febca39a640 1 -- start start 2024-03-06T07:27:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febca39a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7febc40c3710 0x7febc41558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febca39a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc4158800 con 0x7febc406d9f0 2024-03-06T07:27:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febca39a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc4158ae0 con 0x7febc40c3280 2024-03-06T07:27:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febca39a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc4158dc0 con 0x7febc40c3710 2024-03-06T07:27:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7febc406d9f0 0x7febc40a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39388/0 (socket says 172.21.15.26:39388) 2024-03-06T07:27:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc3fff640 1 -- 172.21.15.26:0/1161358846 learned_addr learned my addr 172.21.15.26:0/1161358846 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc8937640 1 --2- 172.21.15.26:0/1161358846 >> v2:172.21.15.117:3300/0 conn(0x7febc40c3710 0x7febc41558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815025662 0 0) 0x7febc4158800 con 0x7febc406d9f0 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febac003520 con 0x7febc406d9f0 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273757405 0 0) 0x7febac003520 con 0x7febc406d9f0 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 >> v2:172.21.15.117:3300/0 conn(0x7febc40c3710 msgr2=0x7febc41558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 --2- 172.21.15.26:0/1161358846 >> v2:172.21.15.117:3300/0 conn(0x7febc40c3710 0x7febc41558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 >> v1:172.21.15.117:6789/0 conn(0x7febc40c3280 legacy=0x7febc41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc41590a0 con 0x7febc406d9f0 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7febb4001670 con 0x7febc406d9f0 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 >> v1:172.21.15.26:6789/0 conn(0x7febc406d9f0 legacy=0x7febc40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febac0039b0 0x7febac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac004800 0x7febac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0051c0 0x7febac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc4158dc0 con 0x7febac0039b0 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac00c160 con 0x7febac004800 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.612+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac00c360 con 0x7febac0051c0 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc3fff640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febac0039b0 0x7febac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc37fe640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac004800 0x7febac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc8937640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0051c0 0x7febac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc3fff640 1 -- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0051c0 msgr2=0x7febac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc3fff640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0051c0 0x7febac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc3fff640 1 -- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac004800 msgr2=0x7febac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc3fff640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac004800 0x7febac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc3fff640 1 -- 172.21.15.26:0/1161358846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac003520 con 0x7febac0039b0 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.613+0000 7febc8937640 1 -- 172.21.15.26:0/1161358846 reap_dead start 2024-03-06T07:27:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febc3fff640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febac0039b0 0x7febac003dd0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7febb400c240 tx=0x7febac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71aac4bb3e0b943 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:51.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4007d10 con 0x7febac0039b0 2024-03-06T07:27:51.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb4030070 con 0x7febac0039b0 2024-03-06T07:27:51.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febc2ffd640 1 -- 172.21.15.26:0/1161358846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4005d90 con 0x7febac0039b0 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febca39a640 1 -- 172.21.15.26:0/1161358846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febac0039b0 msgr2=0x7febac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febca39a640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febac0039b0 0x7febac003dd0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7febb400c240 tx=0x7febac005830 comp rx=0 tx=0).stop 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febca39a640 1 -- 172.21.15.26:0/1161358846 shutdown_connections 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febca39a640 1 --2- 172.21.15.26:0/1161358846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febac0039b0 0x7febac003dd0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febca39a640 1 -- 172.21.15.26:0/1161358846 >> 172.21.15.26:0/1161358846 conn(0x7febc40aa630 msgr2=0x7febc40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febca39a640 1 -- 172.21.15.26:0/1161358846 shutdown_connections 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.614+0000 7febca39a640 1 -- 172.21.15.26:0/1161358846 wait complete. 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.615+0000 7febca39a640 1 Processor -- start 2024-03-06T07:27:51.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febca39a640 1 -- start start 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febca39a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0039b0 0x7febc406f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febca39a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc40a8ba0 0x7febc40abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febca39a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febc41ee360 0x7febc4146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febca39a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc406df50 con 0x7febc41ee360 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febca39a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc406ddd0 con 0x7febc40a8ba0 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febca39a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc40c6910 con 0x7febac0039b0 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.616+0000 7febc8937640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febc41ee360 0x7febc4146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febc41ee360 0x7febc4146880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50382/0 (socket says 172.21.15.26:50382) 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 -- 172.21.15.26:0/3086809027 learned_addr learned my addr 172.21.15.26:0/3086809027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc37fe640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc40a8ba0 0x7febc40abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc3fff640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0039b0 0x7febc406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 -- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0039b0 msgr2=0x7febc406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0039b0 0x7febc406f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 -- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc40a8ba0 msgr2=0x7febc40abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc40a8ba0 0x7febc40abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 -- 172.21.15.26:0/3086809027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc41ee8d0 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc3fff640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0039b0 0x7febc406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc8937640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febc41ee360 0x7febc4146880 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7febbc007cf0 tx=0x7febbc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc0ff9640 1 -- 172.21.15.26:0/3086809027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc011de0 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc0ff9640 1 -- 172.21.15.26:0/3086809027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc01b070 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc40a7de0 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febc0ff9640 1 -- 172.21.15.26:0/3086809027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc003990 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.617+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc40a83a0 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.618+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc404f330 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.619+0000 7febc0ff9640 1 -- 172.21.15.26:0/3086809027 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febbc003b30 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.619+0000 7febc0ff9640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febb006b500 0x7febb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.619+0000 7febc0ff9640 1 -- 172.21.15.26:0/3086809027 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6217+0+0 (secure 0 0 0) 0x7febbc08aa00 con 0x7febc41ee360 2024-03-06T07:27:51.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.619+0000 7febc3fff640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febb006b500 0x7febb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.620+0000 7febc3fff640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febb006b500 0x7febb006d9c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7febb4034000 tx=0x7febb4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:51.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.623+0000 7febc0ff9640 1 -- 172.21.15.26:0/3086809027 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7febbc003e20 con 0x7febc41ee360 2024-03-06T07:27:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.845+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febc4073240 con 0x7febc41ee360 2024-03-06T07:27:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.846+0000 7febc0ff9640 1 -- 172.21.15.26:0/3086809027 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7febbc059710 con 0x7febc41ee360 2024-03-06T07:27:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febb006b500 msgr2=0x7febb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febb006b500 0x7febb006d9c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7febb4034000 tx=0x7febb4009030 comp rx=0 tx=0).stop 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febc41ee360 msgr2=0x7febc4146880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febc41ee360 0x7febc4146880 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7febbc007cf0 tx=0x7febbc0115a0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 shutdown_connections 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febb006b500 0x7febb006d9c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febc41ee360 0x7febc4146880 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc40a8ba0 0x7febc40abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 --2- 172.21.15.26:0/3086809027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0039b0 0x7febc406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 >> 172.21.15.26:0/3086809027 conn(0x7febc40aa630 msgr2=0x7febc406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 shutdown_connections 2024-03-06T07:27:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:51.848+0000 7febca39a640 1 -- 172.21.15.26:0/3086809027 wait complete. 2024-03-06T07:27:52.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:52.263 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:53.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:53.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.376+0000 7fab3d09a640 1 Processor -- start 2024-03-06T07:27:53.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.376+0000 7fab3d09a640 1 -- start start 2024-03-06T07:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3d09a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab3806d9f0 0x7fab380a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3d09a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab380c36e0 0x7fab381558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3d09a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab38158800 con 0x7fab380c3280 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3d09a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab380c3b10 con 0x7fab380c36e0 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3d09a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab38158ae0 con 0x7fab3806d9f0 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3759e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab380c36e0 0x7fab381558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab36d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab3806d9f0 0x7fab380a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3759e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab380c36e0 0x7fab381558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59948/0 (socket says 172.21.15.26:59948) 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab36d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab3806d9f0 0x7fab380a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:44932/0 (socket says 172.21.15.26:44932) 2024-03-06T07:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab3759e640 1 -- 172.21.15.26:0/2830150948 learned_addr learned my addr 172.21.15.26:0/2830150948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab36d9d640 1 -- 172.21.15.26:0/2830150948 >> v2:172.21.15.117:3300/0 conn(0x7fab380c36e0 msgr2=0x7fab381558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab36d9d640 1 --2- 172.21.15.26:0/2830150948 >> v2:172.21.15.117:3300/0 conn(0x7fab380c36e0 0x7fab381558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab36d9d640 1 -- 172.21.15.26:0/2830150948 >> v1:172.21.15.117:6789/0 conn(0x7fab380c3280 legacy=0x7fab381551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.377+0000 7fab36d9d640 1 -- 172.21.15.26:0/2830150948 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38158dc0 con 0x7fab3806d9f0 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.378+0000 7fab3759e640 1 --2- 172.21.15.26:0/2830150948 >> v2:172.21.15.117:3300/0 conn(0x7fab380c36e0 0x7fab381558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.378+0000 7fab36d9d640 1 --2- 172.21.15.26:0/2830150948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab3806d9f0 0x7fab380a7cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab280097a0 tx=0x7fab2802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275fef51f932cf64 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.378+0000 7fab35d9b640 1 -- 172.21.15.26:0/2830150948 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28031780 con 0x7fab3806d9f0 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab35d9b640 1 -- 172.21.15.26:0/2830150948 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28037070 con 0x7fab3806d9f0 2024-03-06T07:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab35d9b640 1 -- 172.21.15.26:0/2830150948 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28031b40 con 0x7fab3806d9f0 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 -- 172.21.15.26:0/2830150948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab3806d9f0 msgr2=0x7fab380a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 --2- 172.21.15.26:0/2830150948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab3806d9f0 0x7fab380a7cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab280097a0 tx=0x7fab2802b820 comp rx=0 tx=0).stop 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 -- 172.21.15.26:0/2830150948 shutdown_connections 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 --2- 172.21.15.26:0/2830150948 >> v2:172.21.15.117:3300/0 conn(0x7fab380c36e0 0x7fab381558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 --2- 172.21.15.26:0/2830150948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab3806d9f0 0x7fab380a7cd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 -- 172.21.15.26:0/2830150948 >> 172.21.15.26:0/2830150948 conn(0x7fab380aa630 msgr2=0x7fab380aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 -- 172.21.15.26:0/2830150948 shutdown_connections 2024-03-06T07:27:53.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.379+0000 7fab3d09a640 1 -- 172.21.15.26:0/2830150948 wait complete. 2024-03-06T07:27:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 Processor -- start 2024-03-06T07:27:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 -- start start 2024-03-06T07:27:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3806d9f0 0x7fab380ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380c3280 0x7fab381ede20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab380c36e0 0x7fab3806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38155e30 con 0x7fab3806d9f0 2024-03-06T07:27:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38159380 con 0x7fab380c3280 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab3d09a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3806ddd0 con 0x7fab380c36e0 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab36d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3806d9f0 0x7fab380ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab36d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3806d9f0 0x7fab380ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50384/0 (socket says 172.21.15.26:50384) 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.381+0000 7fab36d9d640 1 -- 172.21.15.26:0/3064652612 learned_addr learned my addr 172.21.15.26:0/3064652612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab3659c640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380c3280 0x7fab381ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab3759e640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab380c36e0 0x7fab3806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab36d9d640 1 -- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab380c36e0 msgr2=0x7fab3806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab36d9d640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab380c36e0 0x7fab3806ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab36d9d640 1 -- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380c3280 msgr2=0x7fab381ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab36d9d640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380c3280 0x7fab381ede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab36d9d640 1 -- 172.21.15.26:0/3064652612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38154ce0 con 0x7fab3806d9f0 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab3759e640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab380c36e0 0x7fab3806ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:53.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab36d9d640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3806d9f0 0x7fab380ae330 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fab28034040 tx=0x7fab280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:53.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab1f7fe640 1 -- 172.21.15.26:0/3064652612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28032610 con 0x7fab3806d9f0 2024-03-06T07:27:53.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab1f7fe640 1 -- 172.21.15.26:0/3064652612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab280327b0 con 0x7fab3806d9f0 2024-03-06T07:27:53.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab1f7fe640 1 -- 172.21.15.26:0/3064652612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28032ba0 con 0x7fab3806d9f0 2024-03-06T07:27:53.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab38070360 con 0x7fab3806d9f0 2024-03-06T07:27:53.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.382+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab381ee660 con 0x7fab3806d9f0 2024-03-06T07:27:53.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.383+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3804f330 con 0x7fab3806d9f0 2024-03-06T07:27:53.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.384+0000 7fab1f7fe640 1 -- 172.21.15.26:0/3064652612 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab2804e440 con 0x7fab3806d9f0 2024-03-06T07:27:53.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.384+0000 7fab1f7fe640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab2406b500 0x7fab2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.384+0000 7fab1f7fe640 1 -- 172.21.15.26:0/3064652612 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fab280ac130 con 0x7fab3806d9f0 2024-03-06T07:27:53.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.384+0000 7fab3659c640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab2406b500 0x7fab2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.385+0000 7fab3659c640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab2406b500 0x7fab2406d9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fab3814e610 tx=0x7fab2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:53.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.388+0000 7fab1f7fe640 1 -- 172.21.15.26:0/3064652612 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab38154ce0 con 0x7fab3806d9f0 2024-03-06T07:27:53.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.609+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab38073240 con 0x7fab3806d9f0 2024-03-06T07:27:53.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.609+0000 7fab1f7fe640 1 -- 172.21.15.26:0/3064652612 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+4988 (secure 0 0 0) 0x7fab28040070 con 0x7fab3806d9f0 2024-03-06T07:27:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:53.611 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":174,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:39.154723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9867":{"gid":9867,"name":"d","rank":0,"incarnation":170,"state":"up:stopping","state_seq":3,"addr":"172.21.15.117:6836/1623983507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1623983507},{"type":"v1","addr":"172.21.15.117:6836","nonce":1623983507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:53.612 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 174 2024-03-06T07:27:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab2406b500 msgr2=0x7fab2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab2406b500 0x7fab2406d9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fab3814e610 tx=0x7fab2c009040 comp rx=0 tx=0).stop 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3806d9f0 msgr2=0x7fab380ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3806d9f0 0x7fab380ae330 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fab28034040 tx=0x7fab280046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 shutdown_connections 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab2406b500 0x7fab2406d9c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab380c36e0 0x7fab3806ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380c3280 0x7fab381ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 --2- 172.21.15.26:0/3064652612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3806d9f0 0x7fab380ae330 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 >> 172.21.15.26:0/3064652612 conn(0x7fab380aa630 msgr2=0x7fab380c7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 shutdown_connections 2024-03-06T07:27:53.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:53.612+0000 7fab3d09a640 1 -- 172.21.15.26:0/3064652612 wait complete. 2024-03-06T07:27:53.681 INFO:tasks.ceph.mds.d.smithi117.stderr: -619> 2024-03-06T07:27:38.405+0000 7f5e660ca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:53.708 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:27:54.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:39.154723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9867': {'gid': 9867, 'name': 'd', 'rank': 0, 'incarnation': 170, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.117:6836/1623983507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1623983507}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 1623983507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:54.030 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9867:up:stopping 2024-03-06T07:27:55.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:55.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.143+0000 7fe6f93e2640 1 Processor -- start 2024-03-06T07:27:55.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.143+0000 7fe6f93e2640 1 -- start start 2024-03-06T07:27:55.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f93e2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6f41527b0 0x7fe6f4159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f93e2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe6f40c7180 0x7fe6f40c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f93e2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f40c7c60 con 0x7fe6f4155b20 2024-03-06T07:27:55.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f93e2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f4159640 con 0x7fe6f40c7180 2024-03-06T07:27:55.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f93e2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f40c8df0 con 0x7fe6f41527b0 2024-03-06T07:27:55.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe6f40c7180 0x7fe6f40c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe6f40c7180 0x7fe6f40c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50392/0 (socket says 172.21.15.26:50392) 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f27fc640 1 -- 172.21.15.26:0/2132145157 learned_addr learned my addr 172.21.15.26:0/2132145157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.144+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/2132145157 >> v2:172.21.15.117:3300/0 conn(0x7fe6f41527b0 0x7fe6f4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.145+0000 7fe6f27fc640 1 -- 172.21.15.26:0/2132145157 >> v2:172.21.15.117:3300/0 conn(0x7fe6f41527b0 msgr2=0x7fe6f4159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.145+0000 7fe6f27fc640 1 --2- 172.21.15.26:0/2132145157 >> v2:172.21.15.117:3300/0 conn(0x7fe6f41527b0 0x7fe6f4159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.145+0000 7fe6f27fc640 1 -- 172.21.15.26:0/2132145157 >> v1:172.21.15.117:6789/0 conn(0x7fe6f4155b20 legacy=0x7fe6f40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.145+0000 7fe6f27fc640 1 -- 172.21.15.26:0/2132145157 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f40c8060 con 0x7fe6f40c7180 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.145+0000 7fe6f27fc640 1 --2- 172.21.15.26:0/2132145157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f40c7180 0x7fe6f40c88b0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40cba80 tx=0x7fe6e802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627c9006b5b6a2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f1ffb640 1 -- 172.21.15.26:0/2132145157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e802e070 con 0x7fe6f40c7180 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f1ffb640 1 -- 172.21.15.26:0/2132145157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8037070 con 0x7fe6f40c7180 2024-03-06T07:27:55.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f1ffb640 1 -- 172.21.15.26:0/2132145157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8031a00 con 0x7fe6f40c7180 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 -- 172.21.15.26:0/2132145157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f40c7180 msgr2=0x7fe6f40c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/2132145157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f40c7180 0x7fe6f40c88b0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40cba80 tx=0x7fe6e802b880 comp rx=0 tx=0).stop 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 -- 172.21.15.26:0/2132145157 shutdown_connections 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/2132145157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f40c7180 0x7fe6f40c88b0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/2132145157 >> v2:172.21.15.117:3300/0 conn(0x7fe6f41527b0 0x7fe6f4159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 -- 172.21.15.26:0/2132145157 >> 172.21.15.26:0/2132145157 conn(0x7fe6f40ac560 msgr2=0x7fe6f40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 -- 172.21.15.26:0/2132145157 shutdown_connections 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.146+0000 7fe6f93e2640 1 -- 172.21.15.26:0/2132145157 wait complete. 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 Processor -- start 2024-03-06T07:27:55.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 -- start start 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6f40c7180 0x7fe6f40c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6f41527b0 0x7fe6f41ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4155b20 0x7fe6f41eed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4155320 con 0x7fe6f4155b20 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40c7980 con 0x7fe6f41527b0 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f93e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4155600 con 0x7fe6f40c7180 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4155b20 0x7fe6f41eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4155b20 0x7fe6f41eed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50404/0 (socket says 172.21.15.26:50404) 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.148+0000 7fe6f37fe640 1 -- 172.21.15.26:0/117572030 learned_addr learned my addr 172.21.15.26:0/117572030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f27fc640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6f41527b0 0x7fe6f41ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f37fe640 1 -- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6f40c7180 msgr2=0x7fe6f40c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6f40c7180 0x7fe6f40c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6f40c7180 0x7fe6f40c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f37fe640 1 -- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6f41527b0 msgr2=0x7fe6f41ee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6f41527b0 0x7fe6f41ee7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f37fe640 1 -- 172.21.15.26:0/117572030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f40c48f0 con 0x7fe6f4155b20 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4155b20 0x7fe6f41eed20 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c780 tx=0x7fe6ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6db7fe640 1 -- 172.21.15.26:0/117572030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec004b60 con 0x7fe6f4155b20 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6db7fe640 1 -- 172.21.15.26:0/117572030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec004d00 con 0x7fe6f4155b20 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f40c4330 con 0x7fe6f4155b20 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6db7fe640 1 -- 172.21.15.26:0/117572030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00aa90 con 0x7fe6f4155b20 2024-03-06T07:27:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.149+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f41ee2c0 con 0x7fe6f4155b20 2024-03-06T07:27:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.150+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f404f330 con 0x7fe6f4155b20 2024-03-06T07:27:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.151+0000 7fe6db7fe640 1 -- 172.21.15.26:0/117572030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6ec00ac30 con 0x7fe6f4155b20 2024-03-06T07:27:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.151+0000 7fe6db7fe640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6e006b500 0x7fe6e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.151+0000 7fe6db7fe640 1 -- 172.21.15.26:0/117572030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe6ec08a580 con 0x7fe6f4155b20 2024-03-06T07:27:55.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.151+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6e006b500 0x7fe6e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.152+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6e006b500 0x7fe6e006d9c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006fd0 tx=0x7fe6e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.155+0000 7fe6db7fe640 1 -- 172.21.15.26:0/117572030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe6ec020070 con 0x7fe6f4155b20 2024-03-06T07:27:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.373+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6f406ce00 con 0x7fe6f4155b20 2024-03-06T07:27:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.374+0000 7fe6db7fe640 1 -- 172.21.15.26:0/117572030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4973 (secure 0 0 0) 0x7fe6ec059290 con 0x7fe6f4155b20 2024-03-06T07:27:55.375 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:55.375 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":176,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:53.055774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 176 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6e006b500 msgr2=0x7fe6e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6e006b500 0x7fe6e006d9c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006fd0 tx=0x7fe6e4008040 comp rx=0 tx=0).stop 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4155b20 msgr2=0x7fe6f41eed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4155b20 0x7fe6f41eed20 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c780 tx=0x7fe6ec00cc40 comp rx=0 tx=0).stop 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 shutdown_connections 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6e006b500 0x7fe6e006d9c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4155b20 0x7fe6f41eed20 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6f41527b0 0x7fe6f41ee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 --2- 172.21.15.26:0/117572030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6f40c7180 0x7fe6f40c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 >> 172.21.15.26:0/117572030 conn(0x7fe6f40ac560 msgr2=0x7fe6f40ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:55.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 shutdown_connections 2024-03-06T07:27:55.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.376+0000 7fe6f93e2640 1 -- 172.21.15.26:0/117572030 wait complete. 2024-03-06T07:27:55.791 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:53.055774+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [], 'up': {}, 'failed': [], 'damaged': [], 'stopped': [0], 'info': {}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:55.791 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/0 2024-03-06T07:27:55.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.895+0000 7fcbf6839640 1 Processor -- start 2024-03-06T07:27:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.895+0000 7fcbf6839640 1 -- start start 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf6839640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcbf00c8a00 0x7fcbf0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf6839640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0159460 0x7fcbf00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf6839640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00c7760 con 0x7fcbf0152920 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf6839640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf01587b0 con 0x7fcbf00c8a00 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf6839640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00c7a40 con 0x7fcbf0159460 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf4dd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0159460 0x7fcbf00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbef7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcbf00c8a00 0x7fcbf0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf4dd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0159460 0x7fcbf00c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:44956/0 (socket says 172.21.15.26:44956) 2024-03-06T07:27:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbef7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcbf00c8a00 0x7fcbf0158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59984/0 (socket says 172.21.15.26:59984) 2024-03-06T07:27:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.896+0000 7fcbf4dd6640 1 -- 172.21.15.26:0/3502858500 learned_addr learned my addr 172.21.15.26:0/3502858500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.897+0000 7fcbef7fe640 1 -- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0159460 msgr2=0x7fcbf00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.897+0000 7fcbef7fe640 1 --2- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0159460 0x7fcbf00c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.897+0000 7fcbef7fe640 1 -- 172.21.15.26:0/3502858500 >> v1:172.21.15.117:6789/0 conn(0x7fcbf0152920 legacy=0x7fcbf0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.897+0000 7fcbef7fe640 1 -- 172.21.15.26:0/3502858500 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00c7d20 con 0x7fcbf00c8a00 2024-03-06T07:27:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.897+0000 7fcbf4dd6640 1 --2- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0159460 0x7fcbf00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:55.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.897+0000 7fcbef7fe640 1 --2- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 0x7fcbf0158f20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40097a0 tx=0x7fcbe402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e4292048ff7879c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.897+0000 7fcbeeffd640 1 -- 172.21.15.26:0/3502858500 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4031780 con 0x7fcbf00c8a00 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbeeffd640 1 -- 172.21.15.26:0/3502858500 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe4037070 con 0x7fcbf00c8a00 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbeeffd640 1 -- 172.21.15.26:0/3502858500 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4031b40 con 0x7fcbf00c8a00 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 -- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 msgr2=0x7fcbf0158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 --2- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 0x7fcbf0158f20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40097a0 tx=0x7fcbe402b820 comp rx=0 tx=0).stop 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 -- 172.21.15.26:0/3502858500 shutdown_connections 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 --2- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0159460 0x7fcbf00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 --2- 172.21.15.26:0/3502858500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 0x7fcbf0158f20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 -- 172.21.15.26:0/3502858500 >> 172.21.15.26:0/3502858500 conn(0x7fcbf00ac560 msgr2=0x7fcbf00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 -- 172.21.15.26:0/3502858500 shutdown_connections 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.898+0000 7fcbf6839640 1 -- 172.21.15.26:0/3502858500 wait complete. 2024-03-06T07:27:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.899+0000 7fcbf6839640 1 Processor -- start 2024-03-06T07:27:55.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.899+0000 7fcbf6839640 1 -- start start 2024-03-06T07:27:55.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf6839640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 0x7fcbf0147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf6839640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0152920 0x7fcbf00c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf6839640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf00ab630 0x7fcbf00abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf6839640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00c7390 con 0x7fcbf00ab630 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf6839640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006d030 con 0x7fcbf00c8a00 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf6839640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00c7510 con 0x7fcbf0152920 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf00ab630 0x7fcbf00abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf00ab630 0x7fcbf00abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50414/0 (socket says 172.21.15.26:50414) 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 -- 172.21.15.26:0/776023925 learned_addr learned my addr 172.21.15.26:0/776023925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbeffff640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 0x7fcbf0147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbef7fe640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0152920 0x7fcbf00c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 -- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0152920 msgr2=0x7fcbf00c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0152920 0x7fcbf00c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 -- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 msgr2=0x7fcbf0147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 0x7fcbf0147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.900+0000 7fcbf4dd6640 1 -- 172.21.15.26:0/776023925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0148290 con 0x7fcbf00ab630 2024-03-06T07:27:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.901+0000 7fcbf4dd6640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf00ab630 0x7fcbf00abae0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800ef90 tx=0x7fcbe800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.901+0000 7fcbecff9640 1 -- 172.21.15.26:0/776023925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800cb20 con 0x7fcbf00ab630 2024-03-06T07:27:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.901+0000 7fcbecff9640 1 -- 172.21.15.26:0/776023925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe800ccc0 con 0x7fcbf00ab630 2024-03-06T07:27:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.901+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf01eeff0 con 0x7fcbf00ab630 2024-03-06T07:27:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.901+0000 7fcbecff9640 1 -- 172.21.15.26:0/776023925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8003bb0 con 0x7fcbf00ab630 2024-03-06T07:27:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.901+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf00be7c0 con 0x7fcbf00ab630 2024-03-06T07:27:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.902+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf004f330 con 0x7fcbf00ab630 2024-03-06T07:27:55.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.902+0000 7fcbecff9640 1 -- 172.21.15.26:0/776023925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbe801b030 con 0x7fcbf00ab630 2024-03-06T07:27:55.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.903+0000 7fcbecff9640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.903+0000 7fcbecff9640 1 -- 172.21.15.26:0/776023925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fcbe808a6f0 con 0x7fcbf00ab630 2024-03-06T07:27:55.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.903+0000 7fcbeffff640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.903+0000 7fcbeffff640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0007fd0 tx=0x7fcbe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:55.907+0000 7fcbecff9640 1 -- 172.21.15.26:0/776023925 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbe8022020 con 0x7fcbf00ab630 2024-03-06T07:27:56.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.127+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbf006cda0 con 0x7fcbf00ab630 2024-03-06T07:27:56.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.128+0000 7fcbecff9640 1 -- 172.21.15.26:0/776023925 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4973 (secure 0 0 0) 0x7fcbe8010040 con 0x7fcbf00ab630 2024-03-06T07:27:56.130 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:56.130 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":176,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9852,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:53.055774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:56.130 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 176 2024-03-06T07:27:56.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbdc06b4d0 msgr2=0x7fcbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0007fd0 tx=0x7fcbe0009040 comp rx=0 tx=0).stop 2024-03-06T07:27:56.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf00ab630 msgr2=0x7fcbf00abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf00ab630 0x7fcbf00abae0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800ef90 tx=0x7fcbe800c490 comp rx=0 tx=0).stop 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 shutdown_connections 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf00ab630 0x7fcbf00abae0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbf0152920 0x7fcbf00c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 --2- 172.21.15.26:0/776023925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbf00c8a00 0x7fcbf0147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 >> 172.21.15.26:0/776023925 conn(0x7fcbf00ac560 msgr2=0x7fcbf00c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 shutdown_connections 2024-03-06T07:27:56.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.130+0000 7fcbf6839640 1 -- 172.21.15.26:0/776023925 wait complete. 2024-03-06T07:27:56.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-03-06T07:27:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.245+0000 7f3d3edd6640 1 Processor -- start 2024-03-06T07:27:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.245+0000 7f3d3edd6640 1 -- start start 2024-03-06T07:27:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3edd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d380ab310 0x7f3d380ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3edd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 0x7f3d381494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3edd6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d38154330 con 0x7f3d380a97c0 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3edd6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d380c46a0 con 0x7f3d380ab310 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3edd6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d38153dc0 con 0x7f3d381538c0 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3cb72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d380ab310 0x7f3d380ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3cb72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d380ab310 0x7f3d380ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50420/0 (socket says 172.21.15.26:50420) 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3cb72640 1 -- 172.21.15.26:0/4060026946 learned_addr learned my addr 172.21.15.26:0/4060026946 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3cb72640 1 -- 172.21.15.26:0/4060026946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 msgr2=0x7f3d381494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3cb72640 1 --2- 172.21.15.26:0/4060026946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 0x7f3d381494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3cb72640 1 -- 172.21.15.26:0/4060026946 >> v1:172.21.15.117:6789/0 conn(0x7f3d380a97c0 legacy=0x7f3d380c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.246+0000 7f3d3cb72640 1 -- 172.21.15.26:0/4060026946 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d380c5d30 con 0x7f3d380ab310 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.247+0000 7f3d3cb72640 1 --2- 172.21.15.26:0/4060026946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d380ab6f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f3d380c4820 tx=0x7f3d3402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4112bfa168bfb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.247+0000 7f3d2f7fe640 1 -- 172.21.15.26:0/4060026946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340317c0 con 0x7f3d380ab310 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.247+0000 7f3d2f7fe640 1 -- 172.21.15.26:0/4060026946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d34037070 con 0x7f3d380ab310 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.247+0000 7f3d2f7fe640 1 -- 172.21.15.26:0/4060026946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34031b80 con 0x7f3d380ab310 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.247+0000 7f3d3edd6640 1 -- 172.21.15.26:0/4060026946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 msgr2=0x7f3d380ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.247+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/4060026946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d380ab6f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f3d380c4820 tx=0x7f3d3402b860 comp rx=0 tx=0).stop 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.248+0000 7f3d3edd6640 1 -- 172.21.15.26:0/4060026946 shutdown_connections 2024-03-06T07:27:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.248+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/4060026946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 0x7f3d381494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.248+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/4060026946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d380ab6f0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.248+0000 7f3d3edd6640 1 -- 172.21.15.26:0/4060026946 >> 172.21.15.26:0/4060026946 conn(0x7f3d380c6710 msgr2=0x7f3d380c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.248+0000 7f3d3edd6640 1 -- 172.21.15.26:0/4060026946 shutdown_connections 2024-03-06T07:27:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.248+0000 7f3d3edd6640 1 -- 172.21.15.26:0/4060026946 wait complete. 2024-03-06T07:27:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 Processor -- start 2024-03-06T07:27:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 -- start start 2024-03-06T07:27:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d380a97c0 0x7f3d380bc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d381ec180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 0x7f3d381de030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380abc30 con 0x7f3d380ab310 2024-03-06T07:27:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380c62f0 con 0x7f3d380a97c0 2024-03-06T07:27:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.249+0000 7f3d3edd6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380c5240 con 0x7f3d381538c0 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d381ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d381ec180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50428/0 (socket says 172.21.15.26:50428) 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 -- 172.21.15.26:0/1353828424 learned_addr learned my addr 172.21.15.26:0/1353828424 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d3cb72640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d380a97c0 0x7f3d380bc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d3d373640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 0x7f3d381de030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 -- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 msgr2=0x7f3d381de030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 0x7f3d381de030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 -- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d380a97c0 msgr2=0x7f3d380bc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d380a97c0 0x7f3d380bc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 -- 172.21.15.26:0/1353828424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3814f950 con 0x7f3d380ab310 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2ffff640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d381ec180 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b410 tx=0x7f3d2800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2d7fa640 1 -- 172.21.15.26:0/1353828424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28015020 con 0x7f3d380ab310 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2d7fa640 1 -- 172.21.15.26:0/1353828424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d28011850 con 0x7f3d380ab310 2024-03-06T07:27:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d381ecc00 con 0x7f3d380ab310 2024-03-06T07:27:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d2d7fa640 1 -- 172.21.15.26:0/1353828424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28011c40 con 0x7f3d380ab310 2024-03-06T07:27:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.250+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d381df120 con 0x7f3d380ab310 2024-03-06T07:27:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.251+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3804f330 con 0x7f3d380ab310 2024-03-06T07:27:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.252+0000 7f3d2d7fa640 1 -- 172.21.15.26:0/1353828424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d28018020 con 0x7f3d380ab310 2024-03-06T07:27:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.252+0000 7f3d2d7fa640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d2406b500 0x7f3d2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.252+0000 7f3d2d7fa640 1 -- 172.21.15.26:0/1353828424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f3d2808a2d0 con 0x7f3d380ab310 2024-03-06T07:27:56.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.252+0000 7f3d3cb72640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d2406b500 0x7f3d2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.253+0000 7f3d3cb72640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d2406b500 0x7f3d2406d9c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3d34036000 tx=0x7f3d34008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.257+0000 7f3d2d7fa640 1 -- 172.21.15.26:0/1353828424 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3d2801f020 con 0x7f3d380ab310 2024-03-06T07:27:56.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.476+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f3d380c5a70 con 0x7f3d380ab310 2024-03-06T07:27:56.687 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:27:56.686+0000 7f8035816640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:27:56.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.687+0000 7f3d2d7fa640 1 -- 172.21.15.26:0/1353828424 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v177) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3d280172c0 con 0x7f3d380ab310 2024-03-06T07:27:56.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.689+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d2406b500 msgr2=0x7f3d2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.689+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d2406b500 0x7f3d2406d9c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3d34036000 tx=0x7f3d34008030 comp rx=0 tx=0).stop 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.689+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 msgr2=0x7f3d381ec180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.689+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d381ec180 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b410 tx=0x7f3d2800b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 shutdown_connections 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d2406b500 0x7f3d2406d9c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d381538c0 0x7f3d381de030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d380ab310 0x7f3d381ec180 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 --2- 172.21.15.26:0/1353828424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d380a97c0 0x7f3d380bc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 >> 172.21.15.26:0/1353828424 conn(0x7f3d380c6710 msgr2=0x7f3d38156560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 shutdown_connections 2024-03-06T07:27:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.690+0000 7f3d3edd6640 1 -- 172.21.15.26:0/1353828424 wait complete. 2024-03-06T07:27:56.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:56.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.804+0000 7f482eb77640 1 Processor -- start 2024-03-06T07:27:56.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.804+0000 7f482eb77640 1 -- start start 2024-03-06T07:27:56.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f482eb77640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 0x7f4828156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f482eb77640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f48281508c0 0x7f48280c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f482eb77640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4828157340 con 0x7f482806f7b0 2024-03-06T07:27:56.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f482eb77640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48280c3740 con 0x7f48281508c0 2024-03-06T07:27:56.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f482eb77640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4828156d80 con 0x7f4828072720 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f4827fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f482806f7b0 0x7f48281501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39404/0 (socket says 172.21.15.26:39404) 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f4827fff640 1 -- 172.21.15.26:0/663630313 learned_addr learned my addr 172.21.15.26:0/663630313 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282537358 0 0) 0x7f4828157340 con 0x7f482806f7b0 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4814003520 con 0x7f482806f7b0 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f482c913640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 0x7f4828156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f482d114640 1 --2- 172.21.15.26:0/663630313 >> v2:172.21.15.117:3300/0 conn(0x7f48281508c0 0x7f48280c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895988048 0 0) 0x7f4814003520 con 0x7f482806f7b0 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 msgr2=0x7f4828156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 0x7f4828156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 >> v2:172.21.15.117:3300/0 conn(0x7f48281508c0 msgr2=0x7f48280c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 --2- 172.21.15.26:0/663630313 >> v2:172.21.15.117:3300/0 conn(0x7f48281508c0 0x7f48280c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828157060 con 0x7f482806f7b0 2024-03-06T07:27:56.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f48180019f0 con 0x7f482806f7b0 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.805+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 >> v1:172.21.15.26:6789/0 conn(0x7f482806f7b0 legacy=0x7f48281501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f48277fe640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48140039b0 0x7f4814003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f48277fe640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4814004800 0x7f4814004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f48277fe640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 0x7f48140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48280c3740 con 0x7f4814004800 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828156d80 con 0x7f48140039b0 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481400bb20 con 0x7f4828072720 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f4827fff640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4814004800 0x7f4814004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f482c913640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48140039b0 0x7f4814003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f482d114640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 0x7f48140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f4827fff640 1 -- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 msgr2=0x7f48140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f4827fff640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4828072720 0x7f48140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f4827fff640 1 -- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48140039b0 msgr2=0x7f4814003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f4827fff640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48140039b0 0x7f4814003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.806+0000 7f4827fff640 1 -- 172.21.15.26:0/663630313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814003520 con 0x7f4814004800 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f4827fff640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4814004800 0x7f4814004c80 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f481800a8f0 tx=0x7f4814005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820e437764a7b658 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800a4c0 con 0x7f4814004800 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4818034630 con 0x7f4814004800 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f48277fe640 1 -- 172.21.15.26:0/663630313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818034a60 con 0x7f4814004800 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f482eb77640 1 -- 172.21.15.26:0/663630313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4814004800 msgr2=0x7f4814004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f482eb77640 1 --2- 172.21.15.26:0/663630313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4814004800 0x7f4814004c80 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f481800a8f0 tx=0x7f4814005830 comp rx=0 tx=0).stop 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f482d114640 1 -- 172.21.15.26:0/663630313 reap_dead start 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f482eb77640 1 -- 172.21.15.26:0/663630313 shutdown_connections 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f482eb77640 1 -- 172.21.15.26:0/663630313 >> 172.21.15.26:0/663630313 conn(0x7f48280aa630 msgr2=0x7f48280aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f482eb77640 1 -- 172.21.15.26:0/663630313 shutdown_connections 2024-03-06T07:27:56.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.807+0000 7f482eb77640 1 -- 172.21.15.26:0/663630313 wait complete. 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 Processor -- start 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 -- start start 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4814004800 0x7f482814ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482814e580 0x7f482814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482814b020 0x7f48281e8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828072c80 con 0x7f482814e580 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828072b00 con 0x7f482814b020 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f482eb77640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4828156430 con 0x7f4814004800 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.809+0000 7f4827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482814e580 0x7f482814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482814e580 0x7f482814aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50438/0 (socket says 172.21.15.26:50438) 2024-03-06T07:27:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 -- 172.21.15.26:0/345953612 learned_addr learned my addr 172.21.15.26:0/345953612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f482d114640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482814b020 0x7f48281e8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f482c913640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4814004800 0x7f482814ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 -- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4814004800 msgr2=0x7f482814ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4814004800 0x7f482814ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 -- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482814b020 msgr2=0x7f48281e8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482814b020 0x7f48281e8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 -- 172.21.15.26:0/345953612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48280ad7e0 con 0x7f482814e580 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f4827fff640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482814e580 0x7f482814aae0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f481800a9e0 tx=0x7f4828153e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f48257fa640 1 -- 172.21.15.26:0/345953612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818045070 con 0x7f482814e580 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f48257fa640 1 -- 172.21.15.26:0/345953612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481803a040 con 0x7f482814e580 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482814bf60 con 0x7f482814e580 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f48257fa640 1 -- 172.21.15.26:0/345953612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818002000 con 0x7f482814e580 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.810+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48280acd00 con 0x7f482814e580 2024-03-06T07:27:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.811+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482804f330 con 0x7f482814e580 2024-03-06T07:27:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.811+0000 7f48257fa640 1 -- 172.21.15.26:0/345953612 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4818048070 con 0x7f482814e580 2024-03-06T07:27:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.812+0000 7f48257fa640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f480806b500 0x7f480806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.812+0000 7f48257fa640 1 -- 172.21.15.26:0/345953612 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f48180ac380 con 0x7f482814e580 2024-03-06T07:27:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.812+0000 7f482c913640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f480806b500 0x7f480806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.813+0000 7f482c913640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f480806b500 0x7f480806d9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4820006fd0 tx=0x7f4820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:56.816+0000 7f48257fa640 1 -- 172.21.15.26:0/345953612 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f481807af60 con 0x7f482814e580 2024-03-06T07:27:57.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.037+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48280aa260 con 0x7f482814e580 2024-03-06T07:27:57.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.037+0000 7f48257fa640 1 -- 172.21.15.26:0/345953612 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f481807a860 con 0x7f482814e580 2024-03-06T07:27:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:57.039 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":178,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:56.690406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0],"up":{"mds_0":9852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":178,"state":"up:starting","state_seq":1,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:57.040 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 178 2024-03-06T07:27:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f480806b500 msgr2=0x7f480806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f480806b500 0x7f480806d9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4820006fd0 tx=0x7f4820008040 comp rx=0 tx=0).stop 2024-03-06T07:27:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482814e580 msgr2=0x7f482814aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482814e580 0x7f482814aae0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f481800a9e0 tx=0x7f4828153e90 comp rx=0 tx=0).stop 2024-03-06T07:27:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 shutdown_connections 2024-03-06T07:27:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f480806b500 0x7f480806d9c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f482814b020 0x7f48281e8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f482814e580 0x7f482814aae0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 --2- 172.21.15.26:0/345953612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4814004800 0x7f482814ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 >> 172.21.15.26:0/345953612 conn(0x7f48280aa630 msgr2=0x7f48280a9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 shutdown_connections 2024-03-06T07:27:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:57.039+0000 7f482eb77640 1 -- 172.21.15.26:0/345953612 wait complete. 2024-03-06T07:27:57.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 178, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:56.690406+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0], 'up': {'mds_0': 9852}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9852': {'gid': 9852, 'name': 'c', 'rank': 0, 'incarnation': 178, 'state': 'up:starting', 'state_seq': 1, 'addr': '172.21.15.26:6834/3840578019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3840578019}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 3840578019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:57.455 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9852:up:starting 2024-03-06T07:27:58.407 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-03-06T07:27:58.405+0000 7f6881960640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:58.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.556+0000 7f4bb0155640 1 Processor -- start 2024-03-06T07:27:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4bb0155640 1 -- start start 2024-03-06T07:27:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4bb0155640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ba814f360 0x7f4ba8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4bb0155640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ba80c88c0 0x7f4ba80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4bb0155640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba80c73d0 con 0x7f4ba80c5880 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4bb0155640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80c8dd0 con 0x7f4ba814f360 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4bb0155640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80be2e0 con 0x7f4ba80c88c0 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4badef1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ba814f360 0x7f4ba8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4badef1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ba814f360 0x7f4ba8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60864/0 (socket says 172.21.15.26:60864) 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.557+0000 7f4badef1640 1 -- 172.21.15.26:0/3759010001 learned_addr learned my addr 172.21.15.26:0/3759010001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.558+0000 7f4bad6f0640 1 --2- 172.21.15.26:0/3759010001 >> v2:172.21.15.117:3300/0 conn(0x7f4ba80c88c0 0x7f4ba80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.558+0000 7f4badef1640 1 -- 172.21.15.26:0/3759010001 >> v2:172.21.15.117:3300/0 conn(0x7f4ba80c88c0 msgr2=0x7f4ba80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.558+0000 7f4badef1640 1 --2- 172.21.15.26:0/3759010001 >> v2:172.21.15.117:3300/0 conn(0x7f4ba80c88c0 0x7f4ba80c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.558+0000 7f4badef1640 1 -- 172.21.15.26:0/3759010001 >> v1:172.21.15.117:6789/0 conn(0x7f4ba80c5880 legacy=0x7f4ba8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.558+0000 7f4badef1640 1 -- 172.21.15.26:0/3759010001 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80c7670 con 0x7f4ba814f360 2024-03-06T07:27:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.558+0000 7f4bad6f0640 1 --2- 172.21.15.26:0/3759010001 >> v2:172.21.15.117:3300/0 conn(0x7f4ba80c88c0 0x7f4ba80c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.558+0000 7f4badef1640 1 --2- 172.21.15.26:0/3759010001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba8155c10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80c84c0 tx=0x7f4b9802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e6f1b82276c525 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4baceef640 1 -- 172.21.15.26:0/3759010001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9802c9f0 con 0x7f4ba814f360 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4baceef640 1 -- 172.21.15.26:0/3759010001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9802cb90 con 0x7f4ba814f360 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4baceef640 1 -- 172.21.15.26:0/3759010001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980343f0 con 0x7f4ba814f360 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 -- 172.21.15.26:0/3759010001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 msgr2=0x7f4ba8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3759010001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba8155c10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80c84c0 tx=0x7f4b9802be50 comp rx=0 tx=0).stop 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 -- 172.21.15.26:0/3759010001 shutdown_connections 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3759010001 >> v2:172.21.15.117:3300/0 conn(0x7f4ba80c88c0 0x7f4ba80c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3759010001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba8155c10 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 -- 172.21.15.26:0/3759010001 >> 172.21.15.26:0/3759010001 conn(0x7f4ba80ac570 msgr2=0x7f4ba80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 -- 172.21.15.26:0/3759010001 shutdown_connections 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.559+0000 7f4bb0155640 1 -- 172.21.15.26:0/3759010001 wait complete. 2024-03-06T07:27:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.560+0000 7f4bb0155640 1 Processor -- start 2024-03-06T07:27:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.560+0000 7f4bb0155640 1 -- start start 2024-03-06T07:27:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bb0155640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba80c5880 0x7f4ba8150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bb0155640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80c88c0 0x7f4ba80be920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bb0155640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bb0155640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8159270 con 0x7f4ba814f360 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bb0155640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80c7c30 con 0x7f4ba80c5880 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bb0155640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba81593f0 con 0x7f4ba80c88c0 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba814bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60876/0 (socket says 172.21.15.26:60876) 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 -- 172.21.15.26:0/3448595331 learned_addr learned my addr 172.21.15.26:0/3448595331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4badef1640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba80c5880 0x7f4ba8150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bad6f0640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80c88c0 0x7f4ba80be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 -- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80c88c0 msgr2=0x7f4ba80be920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80c88c0 0x7f4ba80be920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 -- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba80c5880 msgr2=0x7f4ba8150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba80c5880 0x7f4ba8150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4bae6f2640 1 -- 172.21.15.26:0/3448595331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8158740 con 0x7f4ba814f360 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.561+0000 7f4badef1640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba80c5880 0x7f4ba8150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.562+0000 7f4bae6f2640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba814bbe0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c410 tx=0x7f4ba000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.562+0000 7f4ba67fc640 1 -- 172.21.15.26:0/3448595331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0011790 con 0x7f4ba814f360 2024-03-06T07:27:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.562+0000 7f4ba67fc640 1 -- 172.21.15.26:0/3448595331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba0017070 con 0x7f4ba814f360 2024-03-06T07:27:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.562+0000 7f4ba67fc640 1 -- 172.21.15.26:0/3448595331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0011b70 con 0x7f4ba814f360 2024-03-06T07:27:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.562+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba8152190 con 0x7f4ba814f360 2024-03-06T07:27:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.562+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba8150360 con 0x7f4ba814f360 2024-03-06T07:27:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.563+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba8150150 con 0x7f4ba814f360 2024-03-06T07:27:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.564+0000 7f4ba67fc640 1 -- 172.21.15.26:0/3448595331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ba0011d10 con 0x7f4ba814f360 2024-03-06T07:27:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.564+0000 7f4ba67fc640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b9406b4d0 0x7f4b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.564+0000 7f4ba67fc640 1 -- 172.21.15.26:0/3448595331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f4ba008a040 con 0x7f4ba814f360 2024-03-06T07:27:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.565+0000 7f4badef1640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b9406b4d0 0x7f4b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.565+0000 7f4badef1640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b9406b4d0 0x7f4b9406d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4b98035000 tx=0x7f4b98030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.569+0000 7f4ba67fc640 1 -- 172.21.15.26:0/3448595331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4ba0021020 con 0x7f4ba814f360 2024-03-06T07:27:58.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.787+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ba804f330 con 0x7f4ba814f360 2024-03-06T07:27:58.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.788+0000 7f4ba67fc640 1 -- 172.21.15.26:0/3448595331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5009 (secure 0 0 0) 0x7f4ba0058cb0 con 0x7f4ba814f360 2024-03-06T07:27:58.790 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:58.790 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":181,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:58.700534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9127":{"gid":9127,"name":"b","rank":1,"incarnation":180,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":178,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9127,9852]},"id":14}]} 2024-03-06T07:27:58.790 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 181 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b9406b4d0 msgr2=0x7f4b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b9406b4d0 0x7f4b9406d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4b98035000 tx=0x7f4b98030000 comp rx=0 tx=0).stop 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 msgr2=0x7f4ba814bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba814bbe0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c410 tx=0x7f4ba000c8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 shutdown_connections 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b9406b4d0 0x7f4b9406d990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ba814f360 0x7f4ba814bbe0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba80c88c0 0x7f4ba80be920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 --2- 172.21.15.26:0/3448595331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba80c5880 0x7f4ba8150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 >> 172.21.15.26:0/3448595331 conn(0x7f4ba80ac570 msgr2=0x7f4ba80ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 shutdown_connections 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:58.790+0000 7f4bb0155640 1 -- 172.21.15.26:0/3448595331 wait complete. 2024-03-06T07:27:59.204 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 181, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:58.700534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9852, 'mds_1': 9127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9127': {'gid': 9127, 'name': 'b', 'rank': 1, 'incarnation': 180, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/3568105233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3568105233}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3568105233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9852': {'gid': 9852, 'name': 'c', 'rank': 0, 'incarnation': 178, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.26:6834/3840578019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3840578019}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 3840578019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9852, 'qdb_cluster': [9127, 9852]} 2024-03-06T07:27:59.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-03-06T07:27:59.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:27:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.306+0000 7fb00e6c2640 1 Processor -- start 2024-03-06T07:27:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.306+0000 7fb00e6c2640 1 -- start start 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.306+0000 7fb00e6c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb008150600 0x7fb008157090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.306+0000 7fb00e6c2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0080c4940 0x7fb0080c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.306+0000 7fb00e6c2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0081575d0 con 0x7fb0081535e0 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.306+0000 7fb00e6c2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0080a72f0 con 0x7fb008150600 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.306+0000 7fb00e6c2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0080a75d0 con 0x7fb0080c4940 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb007fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb008150600 0x7fb008157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb00cc5f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb0081535e0 0x7fb008153a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46114/0 (socket says 172.21.15.26:46114) 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb00cc5f640 1 -- 172.21.15.26:0/800136220 learned_addr learned my addr 172.21.15.26:0/800136220 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb0077fe640 1 --2- 172.21.15.26:0/800136220 >> v2:172.21.15.117:3300/0 conn(0x7fb0080c4940 0x7fb0080c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb007fff640 1 -- 172.21.15.26:0/800136220 >> v2:172.21.15.117:3300/0 conn(0x7fb0080c4940 msgr2=0x7fb0080c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb007fff640 1 --2- 172.21.15.26:0/800136220 >> v2:172.21.15.117:3300/0 conn(0x7fb0080c4940 0x7fb0080c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb007fff640 1 -- 172.21.15.26:0/800136220 >> v1:172.21.15.26:6789/0 conn(0x7fb0081535e0 legacy=0x7fb008153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb007fff640 1 -- 172.21.15.26:0/800136220 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080a78b0 con 0x7fb008150600 2024-03-06T07:27:59.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.307+0000 7fb006ffd640 1 -- 172.21.15.26:0/800136220 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358458516 0 0) 0x7fb0081575d0 con 0x7fb0081535e0 2024-03-06T07:27:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.308+0000 7fb007fff640 1 --2- 172.21.15.26:0/800136220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb008157090 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fb0080c1db0 tx=0x7faff802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ab1980c78528cd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.308+0000 7fb006ffd640 1 -- 172.21.15.26:0/800136220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8035020 con 0x7fb008150600 2024-03-06T07:27:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.308+0000 7fb006ffd640 1 -- 172.21.15.26:0/800136220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8031830 con 0x7fb008150600 2024-03-06T07:27:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.308+0000 7fb006ffd640 1 -- 172.21.15.26:0/800136220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8031c00 con 0x7fb008150600 2024-03-06T07:27:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 -- 172.21.15.26:0/800136220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 msgr2=0x7fb008157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/800136220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb008157090 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fb0080c1db0 tx=0x7faff802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 -- 172.21.15.26:0/800136220 shutdown_connections 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/800136220 >> v2:172.21.15.117:3300/0 conn(0x7fb0080c4940 0x7fb0080c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/800136220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb008157090 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 -- 172.21.15.26:0/800136220 >> 172.21.15.26:0/800136220 conn(0x7fb0080c7de0 msgr2=0x7fb0080c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 -- 172.21.15.26:0/800136220 shutdown_connections 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.309+0000 7fb00e6c2640 1 -- 172.21.15.26:0/800136220 wait complete. 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.310+0000 7fb00e6c2640 1 Processor -- start 2024-03-06T07:27:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.310+0000 7fb00e6c2640 1 -- start start 2024-03-06T07:27:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0080c4940 0x7fb00806b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb0081e9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0081535e0 0x7fb0081e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080a9240 con 0x7fb008150600 2024-03-06T07:27:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0081509e0 con 0x7fb0081535e0 2024-03-06T07:27:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080a93c0 con 0x7fb0080c4940 2024-03-06T07:27:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb0081e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb0081e9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60898/0 (socket says 172.21.15.26:60898) 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 -- 172.21.15.26:0/1721863881 learned_addr learned my addr 172.21.15.26:0/1721863881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0081535e0 0x7fb0081e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb007fff640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0080c4940 0x7fb00806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 -- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0080c4940 msgr2=0x7fb00806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0080c4940 0x7fb00806b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 -- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0081535e0 msgr2=0x7fb0081e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0081535e0 0x7fb0081e9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 -- 172.21.15.26:0/1721863881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080c5b60 con 0x7fb008150600 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb0077fe640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb0081e9900 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9e0 tx=0x7faffc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb004ff9640 1 -- 172.21.15.26:0/1721863881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc017070 con 0x7fb008150600 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb004ff9640 1 -- 172.21.15.26:0/1721863881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc00c9d0 con 0x7fb008150600 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb008156d80 con 0x7fb008150600 2024-03-06T07:27:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb004ff9640 1 -- 172.21.15.26:0/1721863881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00cdc0 con 0x7fb008150600 2024-03-06T07:27:59.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.311+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0081ebc50 con 0x7fb008150600 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.313+0000 7fb004ff9640 1 -- 172.21.15.26:0/1721863881 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faffc01a070 con 0x7fb008150600 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.314+0000 7fb004ff9640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faff406b4d0 0x7faff406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.314+0000 7fb004ff9640 1 -- 172.21.15.26:0/1721863881 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7faffc08a710 con 0x7fb008150600 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.314+0000 7fb007fff640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faff406b4d0 0x7faff406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.314+0000 7fb007fff640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faff406b4d0 0x7faff406d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7faff8036000 tx=0x7faff8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.314+0000 7fb011389740 1 Processor -- start 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb011389740 1 -- start start 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c151800 0x55f86c108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c108550 0x55f86c151be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb011389740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c142420 0x55f86c102680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb011389740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86c06c6b0 con 0x55f86c142420 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86c015a60 con 0x55f86c108550 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f86c068750 con 0x55f86c151800 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c142420 0x55f86c102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c142420 0x55f86c102680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60902/0 (socket says 172.21.15.26:60902) 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2367276525 learned_addr learned my addr 172.21.15.26:0/2367276525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb0077fe640 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c108550 0x55f86c151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.315+0000 7fb007fff640 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c151800 0x55f86c108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c151800 msgr2=0x55f86c108010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c151800 0x55f86c108010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c108550 msgr2=0x55f86c151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c108550 0x55f86c151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2367276525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f86c033f30 con 0x55f86c142420 2024-03-06T07:27:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fb007fff640 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c151800 0x55f86c108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c142420 0x55f86c102680 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb000001bc0 tx=0x7fb000002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf8f695a6292a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fafedffb640 1 -- 172.21.15.26:0/2367276525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000007850 con 0x55f86c142420 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fafedffb640 1 -- 172.21.15.26:0/2367276525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000018070 con 0x55f86c142420 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.316+0000 7fafedffb640 1 -- 172.21.15.26:0/2367276525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000135f0 con 0x55f86c142420 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 -- 172.21.15.26:0/2367276525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c142420 msgr2=0x55f86c102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c142420 0x55f86c102680 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb000001bc0 tx=0x7fb000002080 comp rx=0 tx=0).stop 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 -- 172.21.15.26:0/2367276525 shutdown_connections 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c142420 0x55f86c102680 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c108550 0x55f86c151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 --2- 172.21.15.26:0/2367276525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c151800 0x55f86c108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 -- 172.21.15.26:0/2367276525 >> 172.21.15.26:0/2367276525 conn(0x55f86c1047d0 msgr2=0x55f86c1293c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 -- 172.21.15.26:0/2367276525 shutdown_connections 2024-03-06T07:27:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 -- 172.21.15.26:0/2367276525 wait complete. 2024-03-06T07:27:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 -- start start 2024-03-06T07:27:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.317+0000 7fb011389740 1 Processor -- start 2024-03-06T07:27:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c151800 0x55f86c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb011389740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c14feb0 0x55f86c143b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb011389740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86c146590 con 0x55f86c14feb0 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86c028ea0 con 0x55f86c151800 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f86c129db0 con 0x55f86c142420 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb011389740 10 client.0 _ll_register_callbacks cb 0x55f86c028870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c14feb0 0x55f86c143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c14feb0 0x55f86c143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60906/0 (socket says 172.21.15.26:60906) 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb00cc5f640 1 -- 172.21.15.26:0/232199636 learned_addr learned my addr 172.21.15.26:0/232199636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb007fff640 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.318+0000 7fb0077fe640 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c151800 0x55f86c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb00cc5f640 1 -- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 msgr2=0x55f86c15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb00cc5f640 1 -- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c151800 msgr2=0x55f86c14f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c151800 0x55f86c14f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb00cc5f640 1 -- 172.21.15.26:0/232199636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f86c029270 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c14feb0 0x55f86c143b50 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb000002230 tx=0x7fb0000015c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0aa6e41a7b9854 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fafe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fafe6ffd640 1 -- 172.21.15.26:0/232199636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000013bb0 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb000018070 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fafe6ffd640 1 -- 172.21.15.26:0/232199636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb000013d50 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fafe6ffd640 1 -- 172.21.15.26:0/232199636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb00002c050 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb011389740 10 client.10110 fetch_fsmap learned FSMap version 181 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x55f86bd812d0 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fafe6ffd640 1 -- 172.21.15.26:0/232199636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000018070 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.319+0000 7fafe6ffd640 1 -- 172.21.15.26:0/232199636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fb00002f020 con 0x55f86c14feb0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb011389740 10 client.10110 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb011389740 10 client.10110 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb011389740 20 client.10110 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb011389740 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c12ca00 0x55f86c159290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb011389740 4 client.10110 mds_command: new command op to 9127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 --> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f86c079bf0 con 0x55f86c12ca00 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb007fff640 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c12ca00 0x55f86c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fafe6ffd640 10 client.10110 ms_handle_connect on v2:172.21.15.117:6835/3568105233 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.320+0000 7fb007fff640 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c12ca00 0x55f86c159290 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b0e393a1965d7d1 server_cookie=5eaa6d5d933c1ee0 in_seq=0 out_seq=0 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fafe6ffd640 1 -- 172.21.15.26:0/232199636 <== mds.1 v2:172.21.15.117:6835/3568105233 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f86c079bf0 con 0x55f86c12ca00 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fafe6ffd640 10 client.10110 handle_command_reply: tid=0 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 client.10110 shutdown 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c14feb0 msgr2=0x55f86c143b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c14feb0 0x55f86c143b50 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb000002230 tx=0x7fb0000015c0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 shutdown_connections 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c12ca00 0x55f86c159290 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c14feb0 0x55f86c143b50 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c151800 0x55f86c14f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 --2- 172.21.15.26:0/232199636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 >> 172.21.15.26:0/232199636 conn(0x55f86c108550 msgr2=0x55f86c1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fafe6ffd640 0 client.10110 ms_handle_reset on v2:172.21.15.117:6835/3568105233 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 shutdown_connections 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 1 -- 172.21.15.26:0/232199636 wait complete. 2024-03-06T07:27:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.321+0000 7fb011389740 20 client.10110 trim_cache size 0 max 16384 2024-03-06T07:27:59.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 Processor -- start 2024-03-06T07:27:59.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 -- start start 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c103970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c14feb0 0x55f86c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080ac090 con 0x55f86c151800 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080725b0 con 0x55f86c14feb0 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb008072b80 con 0x55f86c142420 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.338+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60922/0 (socket says 172.21.15.26:60922) 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2565974374 learned_addr learned my addr 172.21.15.26:0/2565974374 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb0077fe640 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c14feb0 0x55f86c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb007fff640 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 msgr2=0x55f86c103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c14feb0 msgr2=0x55f86c150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c14feb0 0x55f86c150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb00cc5f640 1 -- 172.21.15.26:0/2565974374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f86be4b6b0 con 0x55f86c151800 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c150f70 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fb000007ba0 tx=0x7fb00001f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6903f9c263f66db0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fafed7fa640 1 -- 172.21.15.26:0/2565974374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000139b0 con 0x55f86c151800 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fafed7fa640 1 -- 172.21.15.26:0/2565974374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000018070 con 0x55f86c151800 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.339+0000 7fafed7fa640 1 -- 172.21.15.26:0/2565974374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000013dd0 con 0x55f86c151800 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 -- 172.21.15.26:0/2565974374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 msgr2=0x55f86c150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c150f70 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fb000007ba0 tx=0x7fb00001f070 comp rx=0 tx=0).stop 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 -- 172.21.15.26:0/2565974374 shutdown_connections 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c150f70 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c14feb0 0x55f86c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 --2- 172.21.15.26:0/2565974374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c142420 0x55f86c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 -- 172.21.15.26:0/2565974374 >> 172.21.15.26:0/2565974374 conn(0x55f86c108550 msgr2=0x55f86c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 -- 172.21.15.26:0/2565974374 shutdown_connections 2024-03-06T07:27:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 -- 172.21.15.26:0/2565974374 wait complete. 2024-03-06T07:27:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 -- start start 2024-03-06T07:27:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.340+0000 7fb011389740 1 Processor -- start 2024-03-06T07:27:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c142420 0x55f86c16d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb011389740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c14feb0 0x55f86c148240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb011389740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c1b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb011389740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86c1272b0 con 0x55f86c151800 2024-03-06T07:27:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86c128940 con 0x55f86c142420 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb011389740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f86c1504a0 con 0x55f86c14feb0 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb011389740 10 client.0 _ll_register_callbacks cb 0x7fb000001a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c1b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c1b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60924/0 (socket says 172.21.15.26:60924) 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 -- 172.21.15.26:0/641689918 learned_addr learned my addr 172.21.15.26:0/641689918 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb0077fe640 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c14feb0 0x55f86c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb007fff640 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c142420 0x55f86c16d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 -- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c14feb0 msgr2=0x55f86c148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c14feb0 0x55f86c148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 -- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c142420 msgr2=0x55f86c16d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c142420 0x55f86c16d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.341+0000 7fb00cc5f640 1 -- 172.21.15.26:0/641689918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f86c16d9b0 con 0x55f86c151800 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb00cc5f640 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c1b61c0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fb00001c440 tx=0x7fb000004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b69623879b9fda9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fafe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fafe6ffd640 1 -- 172.21.15.26:0/641689918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000139b0 con 0x55f86c151800 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fafe6ffd640 1 -- 172.21.15.26:0/641689918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb000013b50 con 0x55f86c151800 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb000018070 con 0x55f86c151800 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fafe6ffd640 1 -- 172.21.15.26:0/641689918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb000047050 con 0x55f86c151800 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 10 client.10116 fetch_fsmap learned FSMap version 181 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x55f86c148780 con 0x55f86c151800 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fafe6ffd640 1 -- 172.21.15.26:0/641689918 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000018070 con 0x55f86c151800 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fafe6ffd640 1 -- 172.21.15.26:0/641689918 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fb00004a020 con 0x55f86c151800 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 10 client.10116 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 10 client.10116 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 20 client.10116 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c1b6700 0x55f86c13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 4 client.10116 mds_command: new command op to 9127 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 --> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f86c073f40 con 0x55f86c1b6700 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.342+0000 7fb007fff640 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c1b6700 0x55f86c13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fafe6ffd640 10 client.10116 ms_handle_connect on v2:172.21.15.117:6835/3568105233 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb007fff640 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c1b6700 0x55f86c13d3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ab7e5e753cf4f12 server_cookie=88a79d0d45d322e in_seq=0 out_seq=0 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fafe6ffd640 1 -- 172.21.15.26:0/641689918 <== mds.1 v2:172.21.15.117:6835/3568105233 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55f86c073f40 con 0x55f86c1b6700 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fafe6ffd640 10 client.10116 handle_command_reply: tid=0 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 client.10116 shutdown 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 msgr2=0x55f86c1b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c1b61c0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fb00001c440 tx=0x7fb000004cf0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 shutdown_connections 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x55f86c1b6700 0x55f86c13d3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f86c151800 0x55f86c1b61c0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f86c14feb0 0x55f86c148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 --2- 172.21.15.26:0/641689918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f86c142420 0x55f86c16d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 >> 172.21.15.26:0/641689918 conn(0x55f86c108550 msgr2=0x55f86c127660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fafe6ffd640 0 client.10116 ms_handle_reset on v2:172.21.15.117:6835/3568105233 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 shutdown_connections 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 1 -- 172.21.15.26:0/641689918 wait complete. 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.343+0000 7fb011389740 20 client.10116 trim_cache size 0 max 16384 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9127, 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:27:59.351 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:27:59.352 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 1.64287253, 2024-03-06T07:27:59.352 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 181, 2024-03-06T07:27:59.352 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 186, 2024-03-06T07:27:59.352 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 186, 2024-03-06T07:27:59.352 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 31.291201490999999 2024-03-06T07:27:59.352 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faff406b4d0 msgr2=0x7faff406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faff406b4d0 0x7faff406d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7faff8036000 tx=0x7faff8008030 comp rx=0 tx=0).stop 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 msgr2=0x7fb0081e9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb0081e9900 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9e0 tx=0x7faffc00eea0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 shutdown_connections 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faff406b4d0 0x7faff406d990 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0081535e0 0x7fb0081e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008150600 0x7fb0081e9900 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 --2- 172.21.15.26:0/1721863881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0080c4940 0x7fb00806b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 >> 172.21.15.26:0/1721863881 conn(0x7fb0080c7de0 msgr2=0x7fb00806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 shutdown_connections 2024-03-06T07:27:59.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.344+0000 7fb00e6c2640 1 -- 172.21.15.26:0/1721863881 wait complete. 2024-03-06T07:27:59.356 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:27:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.456+0000 7f898b3f7640 1 Processor -- start 2024-03-06T07:27:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.456+0000 7f898b3f7640 1 -- start start 2024-03-06T07:27:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f898b3f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89840c69f0 0x7f89840c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f898b3f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89841558d0 0x7f8984155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f898b3f7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8984155130 con 0x7f89840c8950 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f898b3f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8984155440 con 0x7f89841558d0 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f898b3f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89840ad480 con 0x7f89840c69f0 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f8989193640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89840c69f0 0x7f89840c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f8988992640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f89840c8950 0x7f89840acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46124/0 (socket says 172.21.15.26:46124) 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f8988992640 1 -- 172.21.15.26:0/2361352409 learned_addr learned my addr 172.21.15.26:0/2361352409 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.457+0000 7f8989994640 1 --2- 172.21.15.26:0/2361352409 >> v2:172.21.15.117:3300/0 conn(0x7f89841558d0 0x7f8984155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.458+0000 7f8989193640 1 -- 172.21.15.26:0/2361352409 >> v2:172.21.15.117:3300/0 conn(0x7f89841558d0 msgr2=0x7f8984155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.458+0000 7f8989193640 1 --2- 172.21.15.26:0/2361352409 >> v2:172.21.15.117:3300/0 conn(0x7f89841558d0 0x7f8984155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.458+0000 7f8989193640 1 -- 172.21.15.26:0/2361352409 >> v1:172.21.15.26:6789/0 conn(0x7f89840c8950 legacy=0x7f89840acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.458+0000 7f8989193640 1 -- 172.21.15.26:0/2361352409 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89840c5d50 con 0x7f89840c69f0 2024-03-06T07:27:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.458+0000 7f897bfff640 1 -- 172.21.15.26:0/2361352409 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840213916 0 0) 0x7f8974001110 con 0x7f89840c8950 2024-03-06T07:27:59.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.458+0000 7f8989193640 1 --2- 172.21.15.26:0/2361352409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89840c69f0 0x7f89840c6dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89800097a0 tx=0x7f898002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e2c067b2045155 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f897bfff640 1 -- 172.21.15.26:0/2361352409 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89800315a0 con 0x7f89840c69f0 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f897bfff640 1 -- 172.21.15.26:0/2361352409 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8980037070 con 0x7f89840c69f0 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f897bfff640 1 -- 172.21.15.26:0/2361352409 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980031960 con 0x7f89840c69f0 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 -- 172.21.15.26:0/2361352409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89840c69f0 msgr2=0x7f89840c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 --2- 172.21.15.26:0/2361352409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89840c69f0 0x7f89840c6dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89800097a0 tx=0x7f898002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 -- 172.21.15.26:0/2361352409 shutdown_connections 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 --2- 172.21.15.26:0/2361352409 >> v2:172.21.15.117:3300/0 conn(0x7f89841558d0 0x7f8984155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 --2- 172.21.15.26:0/2361352409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89840c69f0 0x7f89840c6dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 -- 172.21.15.26:0/2361352409 >> 172.21.15.26:0/2361352409 conn(0x7f89840cae70 msgr2=0x7f89840cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 -- 172.21.15.26:0/2361352409 shutdown_connections 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.459+0000 7f898b3f7640 1 -- 172.21.15.26:0/2361352409 wait complete. 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.460+0000 7f898b3f7640 1 Processor -- start 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.460+0000 7f898b3f7640 1 -- start start 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f898b3f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89840c69f0 0x7f8984145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f898b3f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89840c8950 0x7f8984153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f898b3f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89841558d0 0x7f89840ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f898b3f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8984152a20 con 0x7f89840c8950 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f898b3f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89840c6310 con 0x7f89840c69f0 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f898b3f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8984152ba0 con 0x7f89841558d0 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8988992640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89840c8950 0x7f8984153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89840c69f0 0x7f8984145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89840c69f0 0x7f8984145710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53472/0 (socket says 172.21.15.26:53472) 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 -- 172.21.15.26:0/3187688320 learned_addr learned my addr 172.21.15.26:0/3187688320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8988992640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89840c8950 0x7f8984153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60932/0 (socket says 172.21.15.26:60932) 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989994640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89841558d0 0x7f89840ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 -- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89841558d0 msgr2=0x7f89840ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89841558d0 0x7f89840ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 -- 172.21.15.26:0/3187688320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89840c8950 msgr2=0x7f8984153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89840c8950 0x7f8984153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989193640 1 -- 172.21.15.26:0/3187688320 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8984154090 con 0x7f89840c69f0 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8988992640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89840c8950 0x7f8984153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.461+0000 7f8989994640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89841558d0 0x7f89840ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.462+0000 7f8989193640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89840c69f0 0x7f8984145710 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8980034040 tx=0x7f89800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.462+0000 7f8979ffb640 1 -- 172.21.15.26:0/3187688320 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980032630 con 0x7f89840c69f0 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.462+0000 7f8979ffb640 1 -- 172.21.15.26:0/3187688320 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89800327f0 con 0x7f89840c69f0 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.462+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89841530b0 con 0x7f89840c69f0 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.462+0000 7f8979ffb640 1 -- 172.21.15.26:0/3187688320 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980032be0 con 0x7f89840c69f0 2024-03-06T07:27:59.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.462+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8984153350 con 0x7f89840c69f0 2024-03-06T07:27:59.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.463+0000 7f8979ffb640 1 -- 172.21.15.26:0/3187688320 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f898004e430 con 0x7f89840c69f0 2024-03-06T07:27:59.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.464+0000 7f8979ffb640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.464+0000 7f8979ffb640 1 -- 172.21.15.26:0/3187688320 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f898002f030 con 0x7f89840c69f0 2024-03-06T07:27:59.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.464+0000 7f8988992640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.464+0000 7f8988992640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f896806b4d0 0x7f896806d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8984150c90 tx=0x7f8974007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.466+0000 7f898dfe3740 1 Processor -- start 2024-03-06T07:27:59.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.466+0000 7f898dfe3740 1 -- start start 2024-03-06T07:27:59.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f638354a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638354f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383554b0 0x55f638351680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f898dfe3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6382bb6b0 con 0x55f6383554b0 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f638264a60 con 0x55f638357330 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6382b7750 con 0x55f638356f50 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383554b0 0x55f638351680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383554b0 0x55f638351680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60936/0 (socket says 172.21.15.26:60936) 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 -- 172.21.15.26:0/1788012464 learned_addr learned my addr 172.21.15.26:0/1788012464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989193640 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f638354a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8988992640 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638354f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 -- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 msgr2=0x55f638354a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f638354a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 -- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 msgr2=0x55f638354f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638354f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989994640 1 -- 172.21.15.26:0/1788012464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f638282f30 con 0x55f6383554b0 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.467+0000 7f8989193640 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f638354a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f8989994640 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383554b0 0x55f638351680 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e7f0 tx=0x7f897c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5fcf3846c0576 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f896effd640 1 -- 172.21.15.26:0/1788012464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c01b030 con 0x55f6383554b0 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f896effd640 1 -- 172.21.15.26:0/1788012464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c007aa0 con 0x55f6383554b0 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f896effd640 1 -- 172.21.15.26:0/1788012464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c007e70 con 0x55f6383554b0 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 -- 172.21.15.26:0/1788012464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383554b0 msgr2=0x55f638351680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383554b0 0x55f638351680 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e7f0 tx=0x7f897c009750 comp rx=0 tx=0).stop 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 -- 172.21.15.26:0/1788012464 shutdown_connections 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383554b0 0x55f638351680 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638354f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 --2- 172.21.15.26:0/1788012464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f638354a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 -- 172.21.15.26:0/1788012464 >> 172.21.15.26:0/1788012464 conn(0x55f6383537d0 msgr2=0x55f6383783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 -- 172.21.15.26:0/1788012464 shutdown_connections 2024-03-06T07:27:59.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.468+0000 7f898dfe3740 1 -- 172.21.15.26:0/1788012464 wait complete. 2024-03-06T07:27:59.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.469+0000 7f898dfe3740 1 -- start start 2024-03-06T07:27:59.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.469+0000 7f898dfe3740 1 Processor -- start 2024-03-06T07:27:59.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.470+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f6383777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.470+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f6383a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.470+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f638374080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.470+0000 7f898dfe3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f638379230 con 0x55f638357330 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.470+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f638277ea0 con 0x55f638356f50 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.470+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f638378c70 con 0x55f6383908d0 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f898dfe3740 10 client.0 _ll_register_callbacks cb 0x55f638277870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f6383a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f6383a3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60938/0 (socket says 172.21.15.26:60938) 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 -- 172.21.15.26:0/3040457433 learned_addr learned my addr 172.21.15.26:0/3040457433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8989994640 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f638374080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8989193640 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f6383777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 -- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 msgr2=0x55f638374080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f638374080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 -- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 msgr2=0x55f6383777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f6383777b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 -- 172.21.15.26:0/3040457433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f638278270 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8988992640 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f6383a3480 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f8974000f60 tx=0x7f8974000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4698254c4513f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8963fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8963fff640 1 -- 172.21.15.26:0/3040457433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974012a30 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8963fff640 1 -- 172.21.15.26:0/3040457433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f8974012bd0 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f897c007aa0 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f8963fff640 1 -- 172.21.15.26:0/3040457433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7f897402e050 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f898dfe3740 10 client.10128 fetch_fsmap learned FSMap version 181 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.471+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x55f637fd02d0 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f8963fff640 1 -- 172.21.15.26:0/3040457433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c007aa0 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f8963fff640 1 -- 172.21.15.26:0/3040457433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f8974031020 con 0x55f638357330 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f898dfe3740 10 client.10128 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:27:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f898dfe3740 10 client.10128 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f898dfe3740 20 client.10128 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f898dfe3740 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f63837a000 0x55f6383b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f898dfe3740 4 client.10128 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 --> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6382c8bf0 con 0x55f63837a000 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f8989193640 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f63837a000 0x55f6383b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f8989193640 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f63837a000 0x55f6383b2070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1282f5a25e29eb server_cookie=2de55c6422949348 in_seq=0 out_seq=0 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.472+0000 7f8963fff640 10 client.10128 ms_handle_connect on v2:172.21.15.26:6832/3840578019 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f8963fff640 1 -- 172.21.15.26:0/3040457433 <== mds.0 v2:172.21.15.26:6832/3840578019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f6382c8bf0 con 0x55f63837a000 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f8963fff640 10 client.10128 handle_command_reply: tid=0 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 client.10128 shutdown 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 msgr2=0x55f6383a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f6383a3480 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f8974000f60 tx=0x7f8974000ed0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 shutdown_connections 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f638374080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f6383a3480 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f63837a000 0x55f6383b2070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 --2- 172.21.15.26:0/3040457433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f6383777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 >> 172.21.15.26:0/3040457433 conn(0x55f6383554b0 msgr2=0x55f638350510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f8963fff640 0 client.10128 ms_handle_reset on v2:172.21.15.26:6832/3840578019 2024-03-06T07:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 shutdown_connections 2024-03-06T07:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.473+0000 7f898dfe3740 1 -- 172.21.15.26:0/3040457433 wait complete. 2024-03-06T07:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.474+0000 7f898dfe3740 20 client.10128 trim_cache size 0 max 16384 2024-03-06T07:27:59.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 Processor -- start 2024-03-06T07:27:59.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 -- start start 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f638350930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f63837ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f6383513e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89841f7f20 con 0x55f638357330 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89841eef20 con 0x55f638356f50 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.494+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89841f6a50 con 0x55f6383908d0 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f63837ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f63837ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60954/0 (socket says 172.21.15.26:60954) 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 -- 172.21.15.26:0/2036177038 learned_addr learned my addr 172.21.15.26:0/2036177038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8989994640 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f6383513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8989193640 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f638350930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 -- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 msgr2=0x55f6383513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f6383513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 -- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 msgr2=0x55f638350930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f638350930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 -- 172.21.15.26:0/2036177038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63809a6b0 con 0x55f638357330 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f8988992640 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f63837ba60 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f8974012bf0 tx=0x7f8974021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f43703e4c053f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f896e7fc640 1 -- 172.21.15.26:0/2036177038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89740323f0 con 0x55f638357330 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f896e7fc640 1 -- 172.21.15.26:0/2036177038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974026070 con 0x55f638357330 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.495+0000 7f896e7fc640 1 -- 172.21.15.26:0/2036177038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974032790 con 0x55f638357330 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 -- 172.21.15.26:0/2036177038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 msgr2=0x55f63837ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f63837ba60 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f8974012bf0 tx=0x7f8974021000 comp rx=0 tx=0).stop 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 -- 172.21.15.26:0/2036177038 shutdown_connections 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f6383908d0 0x55f6383513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f638357330 0x55f63837ba60 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2036177038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638356f50 0x55f638350930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 -- 172.21.15.26:0/2036177038 >> 172.21.15.26:0/2036177038 conn(0x55f6383554b0 msgr2=0x55f638377fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 -- 172.21.15.26:0/2036177038 shutdown_connections 2024-03-06T07:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 -- 172.21.15.26:0/2036177038 wait complete. 2024-03-06T07:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 -- start start 2024-03-06T07:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.496+0000 7f898dfe3740 1 Processor -- start 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.497+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f6383b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.497+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638356280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.497+0000 7f898dfe3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383908d0 0x55f638400cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f898dfe3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6383a3b10 con 0x55f6383908d0 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63837ce60 con 0x55f638357330 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f898dfe3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f63837dc10 con 0x55f638356f50 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f898dfe3740 10 client.0 _ll_register_callbacks cb 0x7f89740136d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383908d0 0x55f638400cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8988992640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638356280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989193640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f6383b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383908d0 0x55f638400cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60964/0 (socket says 172.21.15.26:60964) 2024-03-06T07:27:59.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 -- 172.21.15.26:0/2814248296 learned_addr learned my addr 172.21.15.26:0/2814248296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 -- 172.21.15.26:0/2814248296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 msgr2=0x55f6383b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f6383b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 -- 172.21.15.26:0/2814248296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 msgr2=0x55f638356280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638356280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 -- 172.21.15.26:0/2814248296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6383a6f80 con 0x55f6383908d0 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989193640 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f6383b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8989994640 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383908d0 0x55f638400cf0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f897c01a040 tx=0x7f897c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641e1940d7a0fd15 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8963fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8963fff640 1 -- 172.21.15.26:0/2814248296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c030060 con 0x55f6383908d0 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8963fff640 1 -- 172.21.15.26:0/2814248296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f897c004040 con 0x55f6383908d0 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8974026070 con 0x55f6383908d0 2024-03-06T07:27:59.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.498+0000 7f8963fff640 1 -- 172.21.15.26:0/2814248296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7f897c036050 con 0x55f6383908d0 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 10 client.10134 fetch_fsmap learned FSMap version 181 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x55f638376450 con 0x55f6383908d0 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f8963fff640 1 -- 172.21.15.26:0/2814248296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974026070 con 0x55f6383908d0 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f8963fff640 1 -- 172.21.15.26:0/2814248296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f897c039020 con 0x55f6383908d0 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 10 client.10134 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 10 client.10134 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 20 client.10134 populate_metadata read hostname 'smithi026' 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f6384072d0 0x55f6384076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 4 client.10134 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 --> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f6382c2f40 con 0x55f6384072d0 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f8989193640 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f6384072d0 0x55f6384076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f8963fff640 10 client.10134 ms_handle_connect on v2:172.21.15.26:6832/3840578019 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f8989193640 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f6384072d0 0x55f6384076b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7891a1d15fe8befa server_cookie=5f4dcaf6db5a78c1 in_seq=0 out_seq=0 2024-03-06T07:27:59.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f8963fff640 1 -- 172.21.15.26:0/2814248296 <== mds.0 v2:172.21.15.26:6832/3840578019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55f6382c2f40 con 0x55f6384072d0 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f8963fff640 10 client.10134 handle_command_reply: tid=0 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.499+0000 7f898dfe3740 1 client.10134 shutdown 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383908d0 msgr2=0x55f638400cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383908d0 0x55f638400cf0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f897c01a040 tx=0x7f897c014000 comp rx=0 tx=0).stop 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 shutdown_connections 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6383908d0 0x55f638400cf0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x55f6384072d0 0x55f6384076b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f638357330 0x55f638356280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 --2- 172.21.15.26:0/2814248296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f638356f50 0x55f6383b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f8963fff640 0 client.10134 ms_handle_reset on v2:172.21.15.26:6832/3840578019 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 >> 172.21.15.26:0/2814248296 conn(0x55f6383554b0 msgr2=0x55f638377af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 shutdown_connections 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 1 -- 172.21.15.26:0/2814248296 wait complete. 2024-03-06T07:27:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898dfe3740 20 client.10134 trim_cache size 0 max 16384 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9852, 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8083986429999999, 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 181, 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 186, 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 186, 2024-03-06T07:27:59.509 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 31.457712267000002 2024-03-06T07:27:59.510 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:27:59.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f896806b4d0 msgr2=0x7f896806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898b3f7640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f896806b4d0 0x7f896806d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8984150c90 tx=0x7f8974007b50 comp rx=0 tx=0).stop 2024-03-06T07:27:59.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89840c69f0 msgr2=0x7f8984145710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.500+0000 7f898b3f7640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89840c69f0 0x7f8984145710 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8980034040 tx=0x7f89800046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 shutdown_connections 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89841558d0 0x7f89840ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89840c8950 0x7f8984153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 --2- 172.21.15.26:0/3187688320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89840c69f0 0x7f8984145710 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 >> 172.21.15.26:0/3187688320 conn(0x7f89840cae70 msgr2=0x7f89840acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 shutdown_connections 2024-03-06T07:27:59.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.501+0000 7f898b3f7640 1 -- 172.21.15.26:0/3187688320 wait complete. 2024-03-06T07:27:59.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.613+0000 7f50e6e1c640 1 Processor -- start 2024-03-06T07:27:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.613+0000 7f50e6e1c640 1 -- start start 2024-03-06T07:27:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.614+0000 7f50e6e1c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50e00c88c0 0x7f50e0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.614+0000 7f50e6e1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e00c73d0 con 0x7f50e014f360 2024-03-06T07:27:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.614+0000 7f50e6e1c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e00be2e0 con 0x7f50e0158b60 2024-03-06T07:27:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.614+0000 7f50e6e1c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e0159700 con 0x7f50e00c88c0 2024-03-06T07:27:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.614+0000 7f50e4bb8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f50e014f360 0x7f50e00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46132/0 (socket says 172.21.15.26:46132) 2024-03-06T07:27:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.614+0000 7f50e4bb8640 1 -- 172.21.15.26:0/492895681 learned_addr learned my addr 172.21.15.26:0/492895681 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:27:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.614+0000 7f50dffff640 1 --2- 172.21.15.26:0/492895681 >> v2:172.21.15.117:3300/0 conn(0x7f50e00c88c0 0x7f50e0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.615+0000 7f50df7fe640 1 -- 172.21.15.26:0/492895681 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530063682 0 0) 0x7f50e00c73d0 con 0x7f50e014f360 2024-03-06T07:27:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.615+0000 7f50dffff640 1 -- 172.21.15.26:0/492895681 >> v1:172.21.15.117:6789/0 conn(0x7f50e0158b60 legacy=0x7f50e0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.615+0000 7f50dffff640 1 -- 172.21.15.26:0/492895681 >> v1:172.21.15.26:6789/0 conn(0x7f50e014f360 legacy=0x7f50e00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.615+0000 7f50dffff640 1 -- 172.21.15.26:0/492895681 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e00c7670 con 0x7f50e00c88c0 2024-03-06T07:27:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50dffff640 1 --2- 172.21.15.26:0/492895681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 0x7f50e0155c10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f50d40097a0 tx=0x7f50d402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b78d0bf43fb7a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50df7fe640 1 -- 172.21.15.26:0/492895681 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4035020 con 0x7f50e00c88c0 2024-03-06T07:27:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50df7fe640 1 -- 172.21.15.26:0/492895681 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d4031830 con 0x7f50e00c88c0 2024-03-06T07:27:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50df7fe640 1 -- 172.21.15.26:0/492895681 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4031c00 con 0x7f50e00c88c0 2024-03-06T07:27:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50e6e1c640 1 -- 172.21.15.26:0/492895681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 msgr2=0x7f50e0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/492895681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 0x7f50e0155c10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f50d40097a0 tx=0x7f50d402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50e6e1c640 1 -- 172.21.15.26:0/492895681 shutdown_connections 2024-03-06T07:27:59.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.616+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/492895681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 0x7f50e0155c10 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.617+0000 7f50e6e1c640 1 -- 172.21.15.26:0/492895681 >> 172.21.15.26:0/492895681 conn(0x7f50e00ac570 msgr2=0x7f50e00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.617+0000 7f50e6e1c640 1 -- 172.21.15.26:0/492895681 shutdown_connections 2024-03-06T07:27:59.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.617+0000 7f50e6e1c640 1 -- 172.21.15.26:0/492895681 wait complete. 2024-03-06T07:27:59.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.618+0000 7f50e6e1c640 1 Processor -- start 2024-03-06T07:27:59.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.618+0000 7f50e6e1c640 1 -- start start 2024-03-06T07:27:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.619+0000 7f50e6e1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 0x7f50e01eb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.619+0000 7f50e6e1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50e014f360 0x7f50e014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.619+0000 7f50e6e1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50e0158b60 0x7f50e0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.619+0000 7f50e6e1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00c7c30 con 0x7f50e014f360 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50e6e1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00c5a50 con 0x7f50e00c88c0 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50e6e1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00c5bd0 con 0x7f50e0158b60 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50e014f360 0x7f50e014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50e014f360 0x7f50e014f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60968/0 (socket says 172.21.15.26:60968) 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 -- 172.21.15.26:0/2255067047 learned_addr learned my addr 172.21.15.26:0/2255067047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50e4bb8640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 0x7f50e01eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50e53b9640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50e0158b60 0x7f50e0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 -- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50e0158b60 msgr2=0x7f50e0151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50e0158b60 0x7f50e0151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 -- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 msgr2=0x7f50e01eb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 0x7f50e01eb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 -- 172.21.15.26:0/2255067047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e014fe40 con 0x7f50e014f360 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dffff640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50e014f360 0x7f50e014f740 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f50d40098c0 tx=0x7f50d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dd7fa640 1 -- 172.21.15.26:0/2255067047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4041070 con 0x7f50e014f360 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dd7fa640 1 -- 172.21.15.26:0/2255067047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d403aa30 con 0x7f50e014f360 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50dd7fa640 1 -- 172.21.15.26:0/2255067047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4035040 con 0x7f50e014f360 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e0150120 con 0x7f50e014f360 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.620+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e01506e0 con 0x7f50e014f360 2024-03-06T07:27:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.621+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e014ebc0 con 0x7f50e014f360 2024-03-06T07:27:59.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.622+0000 7f50dd7fa640 1 -- 172.21.15.26:0/2255067047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50d404c060 con 0x7f50e014f360 2024-03-06T07:27:59.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.622+0000 7f50dd7fa640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50cc06b4d0 0x7f50cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.622+0000 7f50dd7fa640 1 -- 172.21.15.26:0/2255067047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f50d40abda0 con 0x7f50e014f360 2024-03-06T07:27:59.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.622+0000 7f50e4bb8640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50cc06b4d0 0x7f50cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.623+0000 7f50e4bb8640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50cc06b4d0 0x7f50cc06d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f50d8001920 tx=0x7f50d80018b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.626+0000 7f50dd7fa640 1 -- 172.21.15.26:0/2255067047 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f50e014ebc0 con 0x7f50e014f360 2024-03-06T07:27:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.988+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50e0069460 con 0x7f50e014f360 2024-03-06T07:27:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.989+0000 7f50dd7fa640 1 -- 172.21.15.26:0/2255067047 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5009 (secure 0 0 0) 0x7f50e0069460 con 0x7f50e014f360 2024-03-06T07:27:59.991 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:27:59.991 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":181,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:58.700534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9127":{"gid":9127,"name":"b","rank":1,"incarnation":180,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":178,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9127,9852]},"id":14}]} 2024-03-06T07:27:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 181 2024-03-06T07:27:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50cc06b4d0 msgr2=0x7f50cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50cc06b4d0 0x7f50cc06d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f50d8001920 tx=0x7f50d80018b0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50e014f360 msgr2=0x7f50e014f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50e014f360 0x7f50e014f740 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f50d40098c0 tx=0x7f50d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 shutdown_connections 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f50cc06b4d0 0x7f50cc06d990 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50e0158b60 0x7f50e0151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50e014f360 0x7f50e014f740 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 --2- 172.21.15.26:0/2255067047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e00c88c0 0x7f50e01eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 >> 172.21.15.26:0/2255067047 conn(0x7f50e00ac570 msgr2=0x7f50e014e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 shutdown_connections 2024-03-06T07:27:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:27:59.991+0000 7f50e6e1c640 1 -- 172.21.15.26:0/2255067047 wait complete. 2024-03-06T07:28:00.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 181, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:32.493419+0000', 'modified': '2024-03-06T07:27:58.700534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9852, 'mds_1': 9127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9127': {'gid': 9127, 'name': 'b', 'rank': 1, 'incarnation': 180, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/3568105233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3568105233}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3568105233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9852': {'gid': 9852, 'name': 'c', 'rank': 0, 'incarnation': 178, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.26:6834/3840578019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3840578019}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 3840578019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9852, 'qdb_cluster': [9127, 9852]} 2024-03-06T07:28:00.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-03-06T07:28:00.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:28:00.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.509+0000 7fd5ae40e640 1 Processor -- start 2024-03-06T07:28:00.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.509+0000 7fd5ae40e640 1 -- start start 2024-03-06T07:28:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5ae40e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c6dd0 0x7fd5a8160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5ae40e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a8155130 con 0x7fd5a80c8950 2024-03-06T07:28:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5ae40e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a80ad480 con 0x7fd5a80c69f0 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5ae40e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8155440 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5a77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5a80c8950 0x7fd5a81558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46148/0 (socket says 172.21.15.26:46148) 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5a77fe640 1 -- 172.21.15.26:0/1887508571 learned_addr learned my addr 172.21.15.26:0/1887508571 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/1887508571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c6dd0 0x7fd5a8160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5a6ffd640 1 -- 172.21.15.26:0/1887508571 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017367674 0 0) 0x7fd59c001110 con 0x7fd5a80c8950 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5a6ffd640 1 -- 172.21.15.26:0/1887508571 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd590003520 con 0x7fd5a80c8950 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5a6ffd640 1 -- 172.21.15.26:0/1887508571 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376827842 0 0) 0x7fd5a80ad480 con 0x7fd5a80c69f0 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.510+0000 7fd5a6ffd640 1 -- 172.21.15.26:0/1887508571 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd59c001110 con 0x7fd5a80c69f0 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.511+0000 7fd5ac9ab640 1 -- 172.21.15.26:0/1887508571 >> v1:172.21.15.117:6789/0 conn(0x7fd5a80c69f0 legacy=0x7fd5a80acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.511+0000 7fd5ac9ab640 1 -- 172.21.15.26:0/1887508571 >> v1:172.21.15.26:6789/0 conn(0x7fd5a80c8950 legacy=0x7fd5a81558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.511+0000 7fd5ac9ab640 1 -- 172.21.15.26:0/1887508571 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80c5d50 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.511+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/1887508571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c6dd0 0x7fd5a8160ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009dc0 tx=0x7fd5a002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993c4a7ba58d6492 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5a6ffd640 1 -- 172.21.15.26:0/1887508571 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0039070 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5a6ffd640 1 -- 172.21.15.26:0/1887508571 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a002cc40 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5a6ffd640 1 -- 172.21.15.26:0/1887508571 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0034440 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5ae40e640 1 -- 172.21.15.26:0/1887508571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c6dd0 msgr2=0x7fd5a8160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/1887508571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c6dd0 0x7fd5a8160ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009dc0 tx=0x7fd5a002be90 comp rx=0 tx=0).stop 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5ae40e640 1 -- 172.21.15.26:0/1887508571 shutdown_connections 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/1887508571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c6dd0 0x7fd5a8160ac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5ae40e640 1 -- 172.21.15.26:0/1887508571 >> 172.21.15.26:0/1887508571 conn(0x7fd5a80cae70 msgr2=0x7fd5a80cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5ae40e640 1 -- 172.21.15.26:0/1887508571 shutdown_connections 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.512+0000 7fd5ae40e640 1 -- 172.21.15.26:0/1887508571 wait complete. 2024-03-06T07:28:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.513+0000 7fd5ae40e640 1 Processor -- start 2024-03-06T07:28:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.513+0000 7fd5ae40e640 1 -- start start 2024-03-06T07:28:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5ae40e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c69f0 0x7fd5a806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5ae40e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a80c6dd0 0x7fd5a81ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5ae40e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a80c8950 0x7fd5a81ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5ae40e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80c6310 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5ae40e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80c7240 con 0x7fd5a80c8950 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5ae40e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8155fe0 con 0x7fd5a80c69f0 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a80c6dd0 0x7fd5a81ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a80c6dd0 0x7fd5a81ee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60974/0 (socket says 172.21.15.26:60974) 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3488980626 learned_addr learned my addr 172.21.15.26:0/3488980626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c69f0 0x7fd5a806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a80c8950 0x7fd5a81ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c69f0 msgr2=0x7fd5a806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c69f0 0x7fd5a806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a80c8950 msgr2=0x7fd5a81ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a80c8950 0x7fd5a81ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.514+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3488980626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a806d090 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.515+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a80c6dd0 0x7fd5a81ee0d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c680 tx=0x7fd59c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.515+0000 7fd5a4ff9640 1 -- 172.21.15.26:0/3488980626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c017070 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.515+0000 7fd5a4ff9640 1 -- 172.21.15.26:0/3488980626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c005ab0 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.515+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a80c1b30 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.515+0000 7fd5a4ff9640 1 -- 172.21.15.26:0/3488980626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c007070 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.515+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a80c2540 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.517+0000 7fd5a4ff9640 1 -- 172.21.15.26:0/3488980626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59c01a070 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.517+0000 7fd5a4ff9640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59406b4d0 0x7fd59406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.517+0000 7fd5a4ff9640 1 -- 172.21.15.26:0/3488980626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd59c0899a0 con 0x7fd5a80c6dd0 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.517+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59406b4d0 0x7fd59406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 Processor -- start 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 -- start start 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59406b4d0 0x7fd59406d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd598007fd0 tx=0x7fd598009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a7ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a7f4b0 0x556943a7b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569439e56b0 con 0x556943a81330 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694398ea60 con 0x556943a7f4b0 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569439e1750 con 0x556943a80f50 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a7ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a7ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60986/0 (socket says 172.21.15.26:60986) 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5a77fe640 1 -- 172.21.15.26:0/2207364698 learned_addr learned my addr 172.21.15.26:0/2207364698 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.518+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a7f4b0 0x556943a7b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5a77fe640 1 -- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 msgr2=0x556943a7ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5a77fe640 1 -- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a7f4b0 msgr2=0x556943a7b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a7f4b0 0x556943a7b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5a77fe640 1 -- 172.21.15.26:0/2207364698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569439acf30 con 0x556943a81330 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a7ef70 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fd59c060130 tx=0x7fd59c05c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2f493a042497bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd58dffb640 1 -- 172.21.15.26:0/2207364698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c074070 con 0x556943a81330 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd58dffb640 1 -- 172.21.15.26:0/2207364698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c056300 con 0x556943a81330 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.519+0000 7fd58dffb640 1 -- 172.21.15.26:0/2207364698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0601d0 con 0x556943a81330 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/2207364698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 msgr2=0x556943a7ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a7ef70 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fd59c060130 tx=0x7fd59c05c0a0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/2207364698 shutdown_connections 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a7f4b0 0x556943a7b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a7ef70 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/2207364698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/2207364698 >> 172.21.15.26:0/2207364698 conn(0x556943a7d7d0 msgr2=0x556943aa23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/2207364698 shutdown_connections 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/2207364698 wait complete. 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 -- start start 2024-03-06T07:28:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.520+0000 7fd5b0ff6740 1 Processor -- start 2024-03-06T07:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a80f50 0x556943a9edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943acd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943aa4a70 0x556943ad0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556943aa3230 con 0x556943a80f50 2024-03-06T07:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569439a1ea0 con 0x556943a81330 2024-03-06T07:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556943aa2c70 con 0x556943aa4a70 2024-03-06T07:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a80f50 0x556943a9edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5b0ff6740 10 client.0 _ll_register_callbacks cb 0x5569439a1870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a80f50 0x556943a9edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32768/0 (socket says 172.21.15.26:32768) 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5a7fff640 1 -- 172.21.15.26:0/3269702069 learned_addr learned my addr 172.21.15.26:0/3269702069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943aa4a70 0x556943ad0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.521+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943acd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5a7fff640 1 -- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943aa4a70 msgr2=0x556943ad0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943aa4a70 0x556943ad0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5a7fff640 1 -- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 msgr2=0x556943acd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943acd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5a7fff640 1 -- 172.21.15.26:0/3269702069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569439a2270 con 0x556943a80f50 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a80f50 0x556943a9edb0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fd5980146d0 tx=0x7fd598000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe003f758c92e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd586ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd586ffd640 1 -- 172.21.15.26:0/3269702069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598025020 con 0x556943a80f50 2024-03-06T07:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd586ffd640 1 -- 172.21.15.26:0/3269702069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd598013740 con 0x556943a80f50 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd59c056300 con 0x556943a80f50 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd586ffd640 1 -- 172.21.15.26:0/3269702069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd59802f050 con 0x556943a80f50 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 10 client.10152 fetch_fsmap learned FSMap version 181 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x5569436fa2d0 con 0x556943a80f50 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd586ffd640 1 -- 172.21.15.26:0/3269702069 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c056300 con 0x556943a80f50 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd586ffd640 1 -- 172.21.15.26:0/3269702069 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fd598032020 con 0x556943a80f50 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 10 client.10152 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 10 client.10152 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 20 client.10152 populate_metadata read hostname 'smithi026' 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ad17f0 0x556943adc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 4 client.10152 mds_command: new command op to 9127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.522+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 --> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569439f2bf0 con 0x556943ad17f0 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.523+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ad17f0 0x556943adc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.523+0000 7fd586ffd640 10 client.10152 ms_handle_connect on v2:172.21.15.117:6835/3568105233 2024-03-06T07:28:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.523+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ad17f0 0x556943adc340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f16ce9c0851cdcec server_cookie=4214cb063de80c1d in_seq=0 out_seq=0 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd586ffd640 1 -- 172.21.15.26:0/3269702069 <== mds.1 v2:172.21.15.117:6835/3568105233 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5569439f2bf0 con 0x556943ad17f0 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd586ffd640 10 client.10152 handle_command_reply: tid=0 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 client.10152 shutdown 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a80f50 msgr2=0x556943a9edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a80f50 0x556943a9edb0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fd5980146d0 tx=0x7fd598000ed0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 shutdown_connections 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ad17f0 0x556943adc340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943aa4a70 0x556943ad0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943acd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd586ffd640 0 client.10152 ms_handle_reset on v2:172.21.15.117:6835/3568105233 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3269702069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a80f50 0x556943a9edb0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 >> 172.21.15.26:0/3269702069 conn(0x556943a7f4b0 msgr2=0x556943a803f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 shutdown_connections 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3269702069 wait complete. 2024-03-06T07:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.524+0000 7fd5b0ff6740 20 client.10152 trim_cache size 0 max 16384 2024-03-06T07:28:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.541+0000 7fd5b0ff6740 1 Processor -- start 2024-03-06T07:28:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.541+0000 7fd5b0ff6740 1 -- start start 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a80f50 0x556943a7f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a81cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943abfef0 0x556943a82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81f7880 con 0x556943a81330 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81f7ee0 con 0x556943a80f50 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a814d0f0 con 0x556943abfef0 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a81cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a81cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32784/0 (socket says 172.21.15.26:32784) 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3854475101 learned_addr learned my addr 172.21.15.26:0/3854475101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943abfef0 0x556943a82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a80f50 0x556943a7f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943abfef0 msgr2=0x556943a82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943abfef0 0x556943a82210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a80f50 msgr2=0x556943a7f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a80f50 0x556943a7f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 -- 172.21.15.26:0/3854475101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569437c46b0 con 0x556943a81330 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a81cd0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fd59c060130 tx=0x7fd59c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a9ff93eced5513 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd58d7fa640 1 -- 172.21.15.26:0/3854475101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c074040 con 0x556943a81330 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd58d7fa640 1 -- 172.21.15.26:0/3854475101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c085070 con 0x556943a81330 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.542+0000 7fd58d7fa640 1 -- 172.21.15.26:0/3854475101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0601d0 con 0x556943a81330 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3854475101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 msgr2=0x556943a81cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a81cd0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fd59c060130 tx=0x7fd59c01d000 comp rx=0 tx=0).stop 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3854475101 shutdown_connections 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943abfef0 0x556943a82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943a81330 0x556943a81cd0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/3854475101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a80f50 0x556943a7f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3854475101 >> 172.21.15.26:0/3854475101 conn(0x556943a7f4b0 msgr2=0x556943a803f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3854475101 shutdown_connections 2024-03-06T07:28:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/3854475101 wait complete. 2024-03-06T07:28:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.543+0000 7fd5b0ff6740 1 -- start start 2024-03-06T07:28:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 1 Processor -- start 2024-03-06T07:28:00.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943b2d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943abfef0 0x556943aa39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556943ace320 con 0x556943abfef0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81f6a00 con 0x556943a81330 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556943acdd60 con 0x556943a80f50 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5b0ff6740 10 client.0 _ll_register_callbacks cb 0x7fd59800fec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.544+0000 7fd5ac9ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943abfef0 0x556943aa39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943abfef0 0x556943aa39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32794/0 (socket says 172.21.15.26:32794) 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 -- 172.21.15.26:0/536352102 learned_addr learned my addr 172.21.15.26:0/536352102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5a77fe640 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943b2d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 -- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 msgr2=0x556943a7b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 -- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 msgr2=0x556943b2d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943b2d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 -- 172.21.15.26:0/536352102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556943aa3f20 con 0x556943abfef0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5ac9ab640 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943abfef0 0x556943aa39e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0037040 tx=0x7fd5a002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a81dfc51f0ea66 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd586ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:28:00.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd586ffd640 1 -- 172.21.15.26:0/536352102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0050060 con 0x556943abfef0 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd586ffd640 1 -- 172.21.15.26:0/536352102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd5a0041910 con 0x556943abfef0 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd59c085070 con 0x556943abfef0 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd586ffd640 1 -- 172.21.15.26:0/536352102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5a0056050 con 0x556943abfef0 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5b0ff6740 10 client.10158 fetch_fsmap learned FSMap version 181 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.545+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x556943b2d990 con 0x556943abfef0 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd586ffd640 1 -- 172.21.15.26:0/536352102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c085070 con 0x556943abfef0 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd586ffd640 1 -- 172.21.15.26:0/536352102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fd5a0059020 con 0x556943abfef0 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5b0ff6740 10 client.10158 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5b0ff6740 10 client.10158 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5b0ff6740 20 client.10158 populate_metadata read hostname 'smithi026' 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ac0770 0x556943ac0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5b0ff6740 4 client.10158 mds_command: new command op to 9127 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 --> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5569439ecf40 con 0x556943ac0770 2024-03-06T07:28:00.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ac0770 0x556943ac0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd586ffd640 10 client.10158 ms_handle_connect on v2:172.21.15.117:6835/3568105233 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5a7fff640 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ac0770 0x556943ac0b50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e26aa398048dcc52 server_cookie=d0afe4fdd50991f9 in_seq=0 out_seq=0 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd586ffd640 1 -- 172.21.15.26:0/536352102 <== mds.1 v2:172.21.15.117:6835/3568105233 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5569439ecf40 con 0x556943ac0770 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd586ffd640 10 client.10158 handle_command_reply: tid=0 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.546+0000 7fd5b0ff6740 1 client.10158 shutdown 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943abfef0 msgr2=0x556943aa39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943abfef0 0x556943aa39e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0037040 tx=0x7fd5a002e070 comp rx=0 tx=0).stop 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 shutdown_connections 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:6835/3568105233,v1:172.21.15.117:6837/3568105233] conn(0x556943ac0770 0x556943ac0b50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556943abfef0 0x556943aa39e0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556943a81330 0x556943b2d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 --2- 172.21.15.26:0/536352102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556943a80f50 0x556943a7b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 >> 172.21.15.26:0/536352102 conn(0x556943a7f4b0 msgr2=0x556943a81cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd586ffd640 0 client.10158 ms_handle_reset on v2:172.21.15.117:6835/3568105233 2024-03-06T07:28:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 shutdown_connections 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 1 -- 172.21.15.26:0/536352102 wait complete. 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5b0ff6740 20 client.10158 trim_cache size 0 max 16384 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9127, 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.846281098, 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 181, 2024-03-06T07:28:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 186, 2024-03-06T07:28:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 186, 2024-03-06T07:28:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 32.494613553999997 2024-03-06T07:28:00.555 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:28:00.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59406b4d0 msgr2=0x7fd59406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59406b4d0 0x7fd59406d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd598007fd0 tx=0x7fd598009040 comp rx=0 tx=0).stop 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a80c6dd0 msgr2=0x7fd5a81ee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a80c6dd0 0x7fd5a81ee0d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c680 tx=0x7fd59c00cb40 comp rx=0 tx=0).stop 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 shutdown_connections 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd59406b4d0 0x7fd59406d990 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a80c8950 0x7fd5a81ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a80c6dd0 0x7fd5a81ee0d0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 --2- 172.21.15.26:0/3488980626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80c69f0 0x7fd5a806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 >> 172.21.15.26:0/3488980626 conn(0x7fd5a80cae70 msgr2=0x7fd5a8151790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 shutdown_connections 2024-03-06T07:28:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.547+0000 7fd5ae40e640 1 -- 172.21.15.26:0/3488980626 wait complete. 2024-03-06T07:28:00.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:28:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.661+0000 7fba88719640 1 Processor -- start 2024-03-06T07:28:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.661+0000 7fba88719640 1 -- start start 2024-03-06T07:28:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba88719640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba800abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba88719640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba8006b6d0 0x7fba8015d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba88719640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba80153eb0 con 0x7fba800ab810 2024-03-06T07:28:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba88719640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba800ac130 con 0x7fba8006b6d0 2024-03-06T07:28:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba88719640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba800ac350 con 0x7fba800730e0 2024-03-06T07:28:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba86cb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba8006b6d0 0x7fba8015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba86cb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba8006b6d0 0x7fba8015d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32808/0 (socket says 172.21.15.26:32808) 2024-03-06T07:28:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba86cb6640 1 -- 172.21.15.26:0/779273606 learned_addr learned my addr 172.21.15.26:0/779273606 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.662+0000 7fba85cb4640 1 --2- 172.21.15.26:0/779273606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba800abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.663+0000 7fba86cb6640 1 -- 172.21.15.26:0/779273606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 msgr2=0x7fba800abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.663+0000 7fba86cb6640 1 --2- 172.21.15.26:0/779273606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba800abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.663+0000 7fba86cb6640 1 -- 172.21.15.26:0/779273606 >> v1:172.21.15.117:6789/0 conn(0x7fba800ab810 legacy=0x7fba8006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.663+0000 7fba86cb6640 1 -- 172.21.15.26:0/779273606 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80153bd0 con 0x7fba8006b6d0 2024-03-06T07:28:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.663+0000 7fba86cb6640 1 --2- 172.21.15.26:0/779273606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba8015d930 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fba78009e50 tx=0x7fba7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05ae6f75e7ae0aa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.663+0000 7fba854b3640 1 -- 172.21.15.26:0/779273606 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7802ca70 con 0x7fba8006b6d0 2024-03-06T07:28:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba854b3640 1 -- 172.21.15.26:0/779273606 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7802cc10 con 0x7fba8006b6d0 2024-03-06T07:28:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba854b3640 1 -- 172.21.15.26:0/779273606 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780343f0 con 0x7fba8006b6d0 2024-03-06T07:28:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 -- 172.21.15.26:0/779273606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 msgr2=0x7fba8015d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 --2- 172.21.15.26:0/779273606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba8015d930 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fba78009e50 tx=0x7fba7802bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 -- 172.21.15.26:0/779273606 shutdown_connections 2024-03-06T07:28:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 --2- 172.21.15.26:0/779273606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba8015d930 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 --2- 172.21.15.26:0/779273606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba800abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 -- 172.21.15.26:0/779273606 >> 172.21.15.26:0/779273606 conn(0x7fba800c3bb0 msgr2=0x7fba800c3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 -- 172.21.15.26:0/779273606 shutdown_connections 2024-03-06T07:28:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.664+0000 7fba88719640 1 -- 172.21.15.26:0/779273606 wait complete. 2024-03-06T07:28:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.665+0000 7fba88719640 1 Processor -- start 2024-03-06T07:28:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.665+0000 7fba88719640 1 -- start start 2024-03-06T07:28:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba88719640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba80141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba88719640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba80143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba88719640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba800ab810 0x7fba801e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba88719640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800ad250 con 0x7fba8006b6d0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba88719640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800ad0d0 con 0x7fba800ab810 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba88719640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba80149c80 con 0x7fba800730e0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba80141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba80141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32810/0 (socket says 172.21.15.26:32810) 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 -- 172.21.15.26:0/1713902187 learned_addr learned my addr 172.21.15.26:0/1713902187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba86cb6640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba800ab810 0x7fba801e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba85cb4640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba80143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 -- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 msgr2=0x7fba80143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba80143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 -- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba800ab810 msgr2=0x7fba801e9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba800ab810 0x7fba801e9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.666+0000 7fba864b5640 1 -- 172.21.15.26:0/1713902187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80143b70 con 0x7fba8006b6d0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.667+0000 7fba864b5640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba80141490 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c3f0 tx=0x7fba7c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.667+0000 7fba76ffd640 1 -- 172.21.15.26:0/1713902187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0052d0 con 0x7fba8006b6d0 2024-03-06T07:28:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.667+0000 7fba76ffd640 1 -- 172.21.15.26:0/1713902187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c005490 con 0x7fba8006b6d0 2024-03-06T07:28:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.667+0000 7fba76ffd640 1 -- 172.21.15.26:0/1713902187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c007260 con 0x7fba8006b6d0 2024-03-06T07:28:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.667+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba80151d40 con 0x7fba8006b6d0 2024-03-06T07:28:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.667+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba80152300 con 0x7fba8006b6d0 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.668+0000 7fba76ffd640 1 -- 172.21.15.26:0/1713902187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba7c007400 con 0x7fba8006b6d0 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.668+0000 7fba76ffd640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.669+0000 7fba76ffd640 1 -- 172.21.15.26:0/1713902187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fba7c089780 con 0x7fba8006b6d0 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.669+0000 7fba85cb4640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.669+0000 7fba85cb4640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fba801ebe30 tx=0x7fba70009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.670+0000 7fba8b3d6740 1 Processor -- start 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.670+0000 7fba8b3d6740 1 -- start start 2024-03-06T07:28:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133beea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf1330 0x556133beef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bef4b0 0x556133beb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133b556b0 con 0x556133bf1330 2024-03-06T07:28:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133afea60 con 0x556133bf0f50 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556133b51750 con 0x556133bef4b0 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf1330 0x556133beef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf1330 0x556133beef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32814/0 (socket says 172.21.15.26:32814) 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba864b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133beea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba86cb6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bef4b0 0x556133beb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 -- 172.21.15.26:0/2156623085 learned_addr learned my addr 172.21.15.26:0/2156623085 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 -- 172.21.15.26:0/2156623085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bef4b0 msgr2=0x556133beb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bef4b0 0x556133beb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 -- 172.21.15.26:0/2156623085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 msgr2=0x556133beea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133beea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba85cb4640 1 -- 172.21.15.26:0/2156623085 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556133b1cf30 con 0x556133bf1330 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.671+0000 7fba864b5640 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133beea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba85cb4640 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf1330 0x556133beef70 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fba70006d40 tx=0x7fba70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44552445522d17a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba63fff640 1 -- 172.21.15.26:0/2156623085 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7001b070 con 0x556133bf1330 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba63fff640 1 -- 172.21.15.26:0/2156623085 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70027070 con 0x556133bf1330 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba63fff640 1 -- 172.21.15.26:0/2156623085 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70003a20 con 0x556133bf1330 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 -- 172.21.15.26:0/2156623085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf1330 msgr2=0x556133beef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf1330 0x556133beef70 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fba70006d40 tx=0x7fba70014000 comp rx=0 tx=0).stop 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 -- 172.21.15.26:0/2156623085 shutdown_connections 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bef4b0 0x556133beb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf1330 0x556133beef70 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/2156623085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133beea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 -- 172.21.15.26:0/2156623085 >> 172.21.15.26:0/2156623085 conn(0x556133bed7d0 msgr2=0x556133c123c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 -- 172.21.15.26:0/2156623085 shutdown_connections 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.672+0000 7fba8b3d6740 1 -- 172.21.15.26:0/2156623085 wait complete. 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.673+0000 7fba8b3d6740 1 -- start start 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.673+0000 7fba8b3d6740 1 Processor -- start 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133c14d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf1330 0x556133c30a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133c41000 0x556133c0de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133c13230 con 0x556133bf0f50 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133b11ea0 con 0x556133bf1330 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556133c12c70 con 0x556133c41000 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba8b3d6740 10 client.0 _ll_register_callbacks cb 0x556133b11870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba864b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133c14d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba864b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133c14d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32818/0 (socket says 172.21.15.26:32818) 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba864b5640 1 -- 172.21.15.26:0/1466759232 learned_addr learned my addr 172.21.15.26:0/1466759232 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba85cb4640 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf1330 0x556133c30a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.674+0000 7fba86cb6640 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133c41000 0x556133c0de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba864b5640 1 -- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133c41000 msgr2=0x556133c0de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba864b5640 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133c41000 0x556133c0de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba864b5640 1 -- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf1330 msgr2=0x556133c30a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba864b5640 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf1330 0x556133c30a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba864b5640 1 -- 172.21.15.26:0/1466759232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556133b12270 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba864b5640 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133c14d90 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c3c0 tx=0x7fba7c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7a5efa6dfd189b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba617fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba617fa640 1 -- 172.21.15.26:0/1466759232 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c056170 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba617fa640 1 -- 172.21.15.26:0/1466759232 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fba7c082cc0 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba70027070 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba617fa640 1 -- 172.21.15.26:0/1466759232 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba7c057200 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba8b3d6740 10 client.10170 fetch_fsmap learned FSMap version 181 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x55613386a2d0 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba617fa640 1 -- 172.21.15.26:0/1466759232 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70027070 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba617fa640 1 -- 172.21.15.26:0/1466759232 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fba7c0573a0 con 0x556133bf0f50 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba8b3d6740 10 client.10170 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba8b3d6740 10 client.10170 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba8b3d6740 20 client.10170 populate_metadata read hostname 'smithi026' 2024-03-06T07:28:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.675+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c13ee0 0x556133c4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.676+0000 7fba8b3d6740 4 client.10170 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.676+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 --> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556133b62bf0 con 0x556133c13ee0 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.676+0000 7fba85cb4640 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c13ee0 0x556133c4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.676+0000 7fba617fa640 10 client.10170 ms_handle_connect on v2:172.21.15.26:6832/3840578019 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.676+0000 7fba85cb4640 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c13ee0 0x556133c4c6a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1009163048fb8d9f server_cookie=8d5273320aca3ef5 in_seq=0 out_seq=0 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba617fa640 1 -- 172.21.15.26:0/1466759232 <== mds.0 v2:172.21.15.26:6832/3840578019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556133b62bf0 con 0x556133c13ee0 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba617fa640 10 client.10170 handle_command_reply: tid=0 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 client.10170 shutdown 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 msgr2=0x556133c14d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133c14d90 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c3c0 tx=0x7fba7c000ed0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 shutdown_connections 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133c41000 0x556133c0de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c13ee0 0x556133c4c6a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf1330 0x556133c30a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1466759232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133c14d90 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba617fa640 0 client.10170 ms_handle_reset on v2:172.21.15.26:6832/3840578019 2024-03-06T07:28:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 >> 172.21.15.26:0/1466759232 conn(0x556133bef4b0 msgr2=0x556133beb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 shutdown_connections 2024-03-06T07:28:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1466759232 wait complete. 2024-03-06T07:28:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.677+0000 7fba8b3d6740 20 client.10170 trim_cache size 0 max 16384 2024-03-06T07:28:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.694+0000 7fba8b3d6740 1 Processor -- start 2024-03-06T07:28:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.694+0000 7fba8b3d6740 1 -- start start 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133bea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133c17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133c41000 0x556133c17f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801f5d30 con 0x556133bf0f50 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801ecc40 con 0x556133c41000 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba801f47d0 con 0x556133bf1330 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133bea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133bea700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32834/0 (socket says 172.21.15.26:32834) 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 -- 172.21.15.26:0/1072506991 learned_addr learned my addr 172.21.15.26:0/1072506991 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba86cb6640 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133c41000 0x556133c17f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba85cb4640 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133c17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 -- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 msgr2=0x556133c17a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133c17a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 -- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133c41000 msgr2=0x556133c17f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133c41000 0x556133c17f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 -- 172.21.15.26:0/1072506991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561339346b0 con 0x556133bf0f50 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba864b5640 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133bea700 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fba7c05a040 tx=0x7fba7c087980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b696fbe00ceea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba637fe640 1 -- 172.21.15.26:0/1072506991 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c054420 con 0x556133bf0f50 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba637fe640 1 -- 172.21.15.26:0/1072506991 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c0573a0 con 0x556133bf0f50 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.695+0000 7fba637fe640 1 -- 172.21.15.26:0/1072506991 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c076e10 con 0x556133bf0f50 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1072506991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 msgr2=0x556133bea700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133bea700 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fba7c05a040 tx=0x7fba7c087980 comp rx=0 tx=0).stop 2024-03-06T07:28:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1072506991 shutdown_connections 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133c41000 0x556133c17f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133c17a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/1072506991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133bf0f50 0x556133bea700 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1072506991 >> 172.21.15.26:0/1072506991 conn(0x556133bef4b0 msgr2=0x556133c2f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1072506991 shutdown_connections 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.696+0000 7fba8b3d6740 1 -- 172.21.15.26:0/1072506991 wait complete. 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 -- start start 2024-03-06T07:28:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 Processor -- start 2024-03-06T07:28:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133c17e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133ca21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133c41000 0x556133ca2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133c0e840 con 0x556133c41000 2024-03-06T07:28:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133c3e270 con 0x556133bf0f50 2024-03-06T07:28:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556133c163d0 con 0x556133bf1330 2024-03-06T07:28:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.697+0000 7fba8b3d6740 10 client.0 _ll_register_callbacks cb 0x7fba7c057ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133c41000 0x556133ca2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133c41000 0x556133ca2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32850/0 (socket says 172.21.15.26:32850) 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 -- 172.21.15.26:0/4107075559 learned_addr learned my addr 172.21.15.26:0/4107075559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba864b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133c17e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba85cb4640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133ca21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 -- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 msgr2=0x556133ca21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133ca21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 -- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 msgr2=0x556133c17e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133c17e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 -- 172.21.15.26:0/4107075559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556133bec4f0 con 0x556133c41000 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba864b5640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133c17e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba85cb4640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133ca21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba86cb6640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133c41000 0x556133ca2720 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fba78009f70 tx=0x7fba7802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbfdf287ea89a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba617fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba617fa640 1 -- 172.21.15.26:0/4107075559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78040af0 con 0x556133c41000 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba617fa640 1 -- 172.21.15.26:0/4107075559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fba78040c90 con 0x556133c41000 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba7c0573a0 con 0x556133c41000 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba617fa640 1 -- 172.21.15.26:0/4107075559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba78003990 con 0x556133c41000 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba8b3d6740 10 client.10176 fetch_fsmap learned FSMap version 181 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.698+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) v3 -- 0x556133c2a190 con 0x556133c41000 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba617fa640 1 -- 172.21.15.26:0/4107075559 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c0573a0 con 0x556133c41000 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba617fa640 1 -- 172.21.15.26:0/4107075559 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 181) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fba78003b30 con 0x556133c41000 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba8b3d6740 10 client.10176 fetch_fsmap finished waiting for FSMap version 181 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba8b3d6740 10 client.10176 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba8b3d6740 20 client.10176 populate_metadata read hostname 'smithi026' 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c9ee80 0x556133c9f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba8b3d6740 4 client.10176 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:28:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 --> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556133b5cf40 con 0x556133c9ee80 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba864b5640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c9ee80 0x556133c9f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba617fa640 10 client.10176 ms_handle_connect on v2:172.21.15.26:6832/3840578019 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba864b5640 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c9ee80 0x556133c9f260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d5fc78480e0f49 server_cookie=c652bb7329d96429 in_seq=0 out_seq=0 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba617fa640 1 -- 172.21.15.26:0/4107075559 <== mds.0 v2:172.21.15.26:6832/3840578019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x556133b5cf40 con 0x556133c9ee80 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.699+0000 7fba617fa640 10 client.10176 handle_command_reply: tid=0 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 client.10176 shutdown 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133c41000 msgr2=0x556133ca2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133c41000 0x556133ca2720 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fba78009f70 tx=0x7fba7802e070 comp rx=0 tx=0).stop 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 shutdown_connections 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556133c41000 0x556133ca2720 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556133bf1330 0x556133ca21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.26:6832/3840578019,v1:172.21.15.26:6834/3840578019] conn(0x556133c9ee80 0x556133c9f260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 --2- 172.21.15.26:0/4107075559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556133bf0f50 0x556133c17e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 >> 172.21.15.26:0/4107075559 conn(0x556133bef4b0 msgr2=0x556133bea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba617fa640 0 client.10176 ms_handle_reset on v2:172.21.15.26:6832/3840578019 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 shutdown_connections 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 1 -- 172.21.15.26:0/4107075559 wait complete. 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba8b3d6740 20 client.10176 trim_cache size 0 max 16384 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9852, 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:28:00.708 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:28:00.709 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:28:00.709 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.0084047199999997, 2024-03-06T07:28:00.709 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 181, 2024-03-06T07:28:00.709 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 186, 2024-03-06T07:28:00.709 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 186, 2024-03-06T07:28:00.709 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 32.657717949000002 2024-03-06T07:28:00.709 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba6406b4d0 msgr2=0x7fba6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fba801ebe30 tx=0x7fba70009190 comp rx=0 tx=0).stop 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 msgr2=0x7fba80141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba80141490 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c3f0 tx=0x7fba7c00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 shutdown_connections 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba800ab810 0x7fba801e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba800730e0 0x7fba80143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 --2- 172.21.15.26:0/1713902187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba8006b6d0 0x7fba80141490 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.700+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 >> 172.21.15.26:0/1713902187 conn(0x7fba800c3bb0 msgr2=0x7fba800aa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.701+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 shutdown_connections 2024-03-06T07:28:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.701+0000 7fba88719640 1 -- 172.21.15.26:0/1713902187 wait complete. 2024-03-06T07:28:00.713 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:28:00.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:00.757 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:00.802 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:28:00.802 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:28:00.802 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:00.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:28:00.816 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:28:00.817 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:00.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:28:00.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.955+0000 7f161e466640 1 Processor -- start 2024-03-06T07:28:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.955+0000 7f161e466640 1 -- start start 2024-03-06T07:28:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f161e466640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16181527b0 0x7f1618159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f161e466640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f16180c88b0 0x7f16180c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f161e466640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16180c7c60 con 0x7f16180c7180 2024-03-06T07:28:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f161e466640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1618159640 con 0x7f16181527b0 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f161e466640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16180c8060 con 0x7f16180c88b0 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f1617fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16181527b0 0x7f1618159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f1617fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16181527b0 0x7f1618159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32858/0 (socket says 172.21.15.26:32858) 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f1617fff640 1 -- 172.21.15.26:0/3470448032 learned_addr learned my addr 172.21.15.26:0/3470448032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.956+0000 7f161ca03640 1 --2- 172.21.15.26:0/3470448032 >> v2:172.21.15.117:3300/0 conn(0x7f16180c88b0 0x7f16180c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.957+0000 7f1617fff640 1 -- 172.21.15.26:0/3470448032 >> v2:172.21.15.117:3300/0 conn(0x7f16180c88b0 msgr2=0x7f16180c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.957+0000 7f1617fff640 1 --2- 172.21.15.26:0/3470448032 >> v2:172.21.15.117:3300/0 conn(0x7f16180c88b0 0x7f16180c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.957+0000 7f1617fff640 1 -- 172.21.15.26:0/3470448032 >> v1:172.21.15.117:6789/0 conn(0x7f16180c7180 legacy=0x7f1618155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.957+0000 7f1617fff640 1 -- 172.21.15.26:0/3470448032 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16180c76a0 con 0x7f16181527b0 2024-03-06T07:28:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.957+0000 7f161ca03640 1 --2- 172.21.15.26:0/3470448032 >> v2:172.21.15.117:3300/0 conn(0x7f16180c88b0 0x7f16180c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.957+0000 7f1617fff640 1 --2- 172.21.15.26:0/3470448032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f1618159100 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f16180ad600 tx=0x7f160802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691e2a90d281f356 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f1616ffd640 1 -- 172.21.15.26:0/3470448032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608035020 con 0x7f16181527b0 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f1616ffd640 1 -- 172.21.15.26:0/3470448032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1608031870 con 0x7f16181527b0 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f1616ffd640 1 -- 172.21.15.26:0/3470448032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608031c40 con 0x7f16181527b0 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 -- 172.21.15.26:0/3470448032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 msgr2=0x7f1618159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 --2- 172.21.15.26:0/3470448032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f1618159100 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f16180ad600 tx=0x7f160802b910 comp rx=0 tx=0).stop 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 -- 172.21.15.26:0/3470448032 shutdown_connections 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 --2- 172.21.15.26:0/3470448032 >> v2:172.21.15.117:3300/0 conn(0x7f16180c88b0 0x7f16180c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 --2- 172.21.15.26:0/3470448032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f1618159100 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 -- 172.21.15.26:0/3470448032 >> 172.21.15.26:0/3470448032 conn(0x7f16180ac560 msgr2=0x7f16180ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 -- 172.21.15.26:0/3470448032 shutdown_connections 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.958+0000 7f161e466640 1 -- 172.21.15.26:0/3470448032 wait complete. 2024-03-06T07:28:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.959+0000 7f161e466640 1 Processor -- start 2024-03-06T07:28:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.959+0000 7f161e466640 1 -- start start 2024-03-06T07:28:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161e466640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16180c7180 0x7f161814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161e466640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16180c88b0 0x7f1618151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161e466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f16181564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161e466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618155600 con 0x7f16181527b0 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161e466640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618155320 con 0x7f16180c7180 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161e466640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1618152b90 con 0x7f16180c88b0 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161ca03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f16181564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161ca03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f16181564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32866/0 (socket says 172.21.15.26:32866) 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f161ca03640 1 -- 172.21.15.26:0/2295274421 learned_addr learned my addr 172.21.15.26:0/2295274421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f1617fff640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16180c7180 0x7f161814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.960+0000 7f16177fe640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16180c88b0 0x7f1618151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161ca03640 1 -- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16180c88b0 msgr2=0x7f1618151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161ca03640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16180c88b0 0x7f1618151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161ca03640 1 -- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16180c7180 msgr2=0x7f161814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161ca03640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16180c7180 0x7f161814be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161ca03640 1 -- 172.21.15.26:0/2295274421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161814c560 con 0x7f16181527b0 2024-03-06T07:28:00.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161ca03640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f16181564f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f1610009fa0 tx=0x7f161000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f1614ff9640 1 -- 172.21.15.26:0/2295274421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161001a070 con 0x7f16181527b0 2024-03-06T07:28:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f1614ff9640 1 -- 172.21.15.26:0/2295274421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1610003bb0 con 0x7f16181527b0 2024-03-06T07:28:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f1614ff9640 1 -- 172.21.15.26:0/2295274421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610004030 con 0x7f16181527b0 2024-03-06T07:28:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161814bb60 con 0x7f16181527b0 2024-03-06T07:28:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.961+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16181eafc0 con 0x7f16181527b0 2024-03-06T07:28:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.962+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16181eaaf0 con 0x7f16181527b0 2024-03-06T07:28:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.963+0000 7f1614ff9640 1 -- 172.21.15.26:0/2295274421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f161001d070 con 0x7f16181527b0 2024-03-06T07:28:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.963+0000 7f1614ff9640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f160406b4d0 0x7f160406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.963+0000 7f1614ff9640 1 -- 172.21.15.26:0/2295274421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f161008a5a0 con 0x7f16181527b0 2024-03-06T07:28:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.963+0000 7f1617fff640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f160406b4d0 0x7f160406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.964+0000 7f1617fff640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f160406b4d0 0x7f160406d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1608036000 tx=0x7f160803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:00.967+0000 7f1614ff9640 1 -- 172.21.15.26:0/2295274421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1610010070 con 0x7f16181527b0 2024-03-06T07:28:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.185+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f161804f330 con 0x7f16181527b0 2024-03-06T07:28:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.186+0000 7f1614ff9640 1 -- 172.21.15.26:0/2295274421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5009 (secure 0 0 0) 0x7f1610058c60 con 0x7f16181527b0 2024-03-06T07:28:01.188 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:01.188 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":181,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:58.700534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9127":{"gid":9127,"name":"b","rank":1,"incarnation":180,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":178,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9127,9852]},"id":14}]} 2024-03-06T07:28:01.188 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 181 2024-03-06T07:28:01.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f160406b4d0 msgr2=0x7f160406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f160406b4d0 0x7f160406d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1608036000 tx=0x7f160803c040 comp rx=0 tx=0).stop 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 msgr2=0x7f16181564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f16181564f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f1610009fa0 tx=0x7f161000d560 comp rx=0 tx=0).stop 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 shutdown_connections 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f160406b4d0 0x7f160406d990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16181527b0 0x7f16181564f0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16180c88b0 0x7f1618151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 --2- 172.21.15.26:0/2295274421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16180c7180 0x7f161814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 >> 172.21.15.26:0/2295274421 conn(0x7f16180ac560 msgr2=0x7f16180c9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 shutdown_connections 2024-03-06T07:28:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.188+0000 7f161e466640 1 -- 172.21.15.26:0/2295274421 wait complete. 2024-03-06T07:28:01.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.304+0000 7f4123d65640 1 Processor -- start 2024-03-06T07:28:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.304+0000 7f4123d65640 1 -- start start 2024-03-06T07:28:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4123d65640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f411c0c20e0 0x7f411c1493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4123d65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c0c3240 0x7f411c149920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4123d65640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c0ab420 con 0x7f411c06c240 2024-03-06T07:28:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4123d65640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0ab700 con 0x7f411c0c20e0 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4123d65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0ab9e0 con 0x7f411c0c3240 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4121300640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f411c0c20e0 0x7f411c1493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4121b01640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f411c06c240 0x7f411c0c19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46160/0 (socket says 172.21.15.26:46160) 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4121b01640 1 -- 172.21.15.26:0/1340008309 learned_addr learned my addr 172.21.15.26:0/1340008309 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4122302640 1 --2- 172.21.15.26:0/1340008309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c0c3240 0x7f411c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4121300640 1 -- 172.21.15.26:0/1340008309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c0c3240 msgr2=0x7f411c149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4121300640 1 --2- 172.21.15.26:0/1340008309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c0c3240 0x7f411c149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4121300640 1 -- 172.21.15.26:0/1340008309 >> v1:172.21.15.26:6789/0 conn(0x7f411c06c240 legacy=0x7f411c0c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4120aff640 1 -- 172.21.15.26:0/1340008309 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937341661 0 0) 0x7f411c0ab420 con 0x7f411c06c240 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.306+0000 7f4121300640 1 -- 172.21.15.26:0/1340008309 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0abcc0 con 0x7f411c0c20e0 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.307+0000 7f4121300640 1 --2- 172.21.15.26:0/1340008309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c20e0 0x7f411c1493e0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f410c0097a0 tx=0x7f410c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db66ebf5968288a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.307+0000 7f4120aff640 1 -- 172.21.15.26:0/1340008309 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c035020 con 0x7f411c0c20e0 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.307+0000 7f4120aff640 1 -- 172.21.15.26:0/1340008309 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c031830 con 0x7f411c0c20e0 2024-03-06T07:28:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.307+0000 7f4120aff640 1 -- 172.21.15.26:0/1340008309 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c031c00 con 0x7f411c0c20e0 2024-03-06T07:28:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 -- 172.21.15.26:0/1340008309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c20e0 msgr2=0x7f411c1493e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 --2- 172.21.15.26:0/1340008309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c20e0 0x7f411c1493e0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f410c0097a0 tx=0x7f410c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 -- 172.21.15.26:0/1340008309 shutdown_connections 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 --2- 172.21.15.26:0/1340008309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c20e0 0x7f411c1493e0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 --2- 172.21.15.26:0/1340008309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c0c3240 0x7f411c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 -- 172.21.15.26:0/1340008309 >> 172.21.15.26:0/1340008309 conn(0x7f411c0c3ad0 msgr2=0x7f411c0c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 -- 172.21.15.26:0/1340008309 shutdown_connections 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.308+0000 7f4123d65640 1 -- 172.21.15.26:0/1340008309 wait complete. 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.309+0000 7f4123d65640 1 Processor -- start 2024-03-06T07:28:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.309+0000 7f4123d65640 1 -- start start 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.309+0000 7f4123d65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c06c240 0x7f411c0c0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.309+0000 7f4123d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c3240 0x7f411c1eab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.309+0000 7f4123d65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c070b30 0x7f411c1eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4123d65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0ac1e0 con 0x7f411c0c3240 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4123d65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c149e60 con 0x7f411c06c240 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4123d65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c14a070 con 0x7f411c070b30 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c3240 0x7f411c1eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c3240 0x7f411c1eab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32886/0 (socket says 172.21.15.26:32886) 2024-03-06T07:28:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 -- 172.21.15.26:0/3929353780 learned_addr learned my addr 172.21.15.26:0/3929353780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121b01640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c06c240 0x7f411c0c0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4122302640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c070b30 0x7f411c1eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 -- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c070b30 msgr2=0x7f411c1eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c070b30 0x7f411c1eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 -- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c06c240 msgr2=0x7f411c0c0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c06c240 0x7f411c0c0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 -- 172.21.15.26:0/3929353780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c1eed10 con 0x7f411c0c3240 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4121300640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c3240 0x7f411c1eab40 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f410c02bee0 tx=0x7f410c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f41127fc640 1 -- 172.21.15.26:0/3929353780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c035040 con 0x7f411c0c3240 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f41127fc640 1 -- 172.21.15.26:0/3929353780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c03b920 con 0x7f411c0c3240 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f41127fc640 1 -- 172.21.15.26:0/3929353780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c03bd10 con 0x7f411c0c3240 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c157030 con 0x7f411c0c3240 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.310+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c14b590 con 0x7f411c0c3240 2024-03-06T07:28:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.312+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c04f330 con 0x7f411c0c3240 2024-03-06T07:28:01.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.312+0000 7f41127fc640 1 -- 172.21.15.26:0/3929353780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f410c038020 con 0x7f411c0c3240 2024-03-06T07:28:01.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.312+0000 7f41127fc640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410006b4d0 0x7f410006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.312+0000 7f41127fc640 1 -- 172.21.15.26:0/3929353780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f410c0ac3e0 con 0x7f411c0c3240 2024-03-06T07:28:01.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.312+0000 7f4121b01640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410006b4d0 0x7f410006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.313+0000 7f4121b01640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410006b4d0 0x7f410006d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4118007fd0 tx=0x7f4118007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.316+0000 7f41127fc640 1 -- 172.21.15.26:0/3929353780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f410c031800 con 0x7f411c0c3240 2024-03-06T07:28:01.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.500+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f411c0ac360 con 0x7f411c0c3240 2024-03-06T07:28:01.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.501+0000 7f41127fc640 1 -- 172.21.15.26:0/3929353780 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17081 (secure 0 0 0) 0x7f410c07aa20 con 0x7f411c0c3240 2024-03-06T07:28:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:01.503 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":186,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:51.668092+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:30.757647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:31.458902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]},{"pgid":"29.1d","mappings":[{"from":6,"to":0}]},{"pgid":"29.1e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:01.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410006b4d0 msgr2=0x7f410006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410006b4d0 0x7f410006d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4118007fd0 tx=0x7f4118007de0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c3240 msgr2=0x7f411c1eab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c3240 0x7f411c1eab40 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f410c02bee0 tx=0x7f410c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 shutdown_connections 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f410006b4d0 0x7f410006d990 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f411c070b30 0x7f411c1eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f411c0c3240 0x7f411c1eab40 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 --2- 172.21.15.26:0/3929353780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c06c240 0x7f411c0c0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 >> 172.21.15.26:0/3929353780 conn(0x7f411c0c3ad0 msgr2=0x7f411c151850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 shutdown_connections 2024-03-06T07:28:01.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.504+0000 7f4123d65640 1 -- 172.21.15.26:0/3929353780 wait complete. 2024-03-06T07:28:01.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:27:30.757647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:27:31.458902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:28:01.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:28:01.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.619+0000 7fbe54c4e640 1 Processor -- start 2024-03-06T07:28:01.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.619+0000 7fbe54c4e640 1 -- start start 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.619+0000 7fbe54c4e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe500c3a70 0x7fbe500732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.619+0000 7fbe54c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006db30 0x7fbe500c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.619+0000 7fbe54c4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe50156060 con 0x7fbe5006d750 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.619+0000 7fbe54c4e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe50155b30 con 0x7fbe500c3a70 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.619+0000 7fbe54c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500a8430 con 0x7fbe5006db30 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe4e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbe5006d750 0x7fbe50072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46168/0 (socket says 172.21.15.26:46168) 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe4e59c640 1 -- 172.21.15.26:0/20871115 learned_addr learned my addr 172.21.15.26:0/20871115 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe45d9b640 1 --2- 172.21.15.26:0/20871115 >> v2:172.21.15.117:3300/0 conn(0x7fbe500c3a70 0x7fbe500732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe4ed9d640 1 --2- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006db30 0x7fbe500c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe4dd9b640 1 -- 172.21.15.26:0/20871115 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736884439 0 0) 0x7fbe50156060 con 0x7fbe5006d750 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe4dd9b640 1 -- 172.21.15.26:0/20871115 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3c003520 con 0x7fbe5006d750 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe45d9b640 1 -- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006db30 msgr2=0x7fbe500c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe45d9b640 1 --2- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006db30 0x7fbe500c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe45d9b640 1 -- 172.21.15.26:0/20871115 >> v1:172.21.15.26:6789/0 conn(0x7fbe5006d750 legacy=0x7fbe50072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe45d9b640 1 -- 172.21.15.26:0/20871115 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe50155d80 con 0x7fbe500c3a70 2024-03-06T07:28:01.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.620+0000 7fbe4ed9d640 1 --2- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006db30 0x7fbe500c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe45d9b640 1 --2- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 0x7fbe500732a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbe380097a0 tx=0x7fbe3802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda2f2829336a71d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe4dd9b640 1 -- 172.21.15.26:0/20871115 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38038070 con 0x7fbe500c3a70 2024-03-06T07:28:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe4dd9b640 1 -- 172.21.15.26:0/20871115 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38031700 con 0x7fbe500c3a70 2024-03-06T07:28:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe4dd9b640 1 -- 172.21.15.26:0/20871115 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38031ad0 con 0x7fbe500c3a70 2024-03-06T07:28:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe54c4e640 1 -- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 msgr2=0x7fbe500732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 0x7fbe500732a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbe380097a0 tx=0x7fbe3802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe54c4e640 1 -- 172.21.15.26:0/20871115 shutdown_connections 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 0x7fbe500732a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/20871115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006db30 0x7fbe500c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.621+0000 7fbe54c4e640 1 -- 172.21.15.26:0/20871115 >> 172.21.15.26:0/20871115 conn(0x7fbe500c5560 msgr2=0x7fbe500c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.622+0000 7fbe54c4e640 1 -- 172.21.15.26:0/20871115 shutdown_connections 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.622+0000 7fbe54c4e640 1 -- 172.21.15.26:0/20871115 wait complete. 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.623+0000 7fbe54c4e640 1 Processor -- start 2024-03-06T07:28:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.623+0000 7fbe54c4e640 1 -- start start 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006d750 0x7fbe5014cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe5006db30 0x7fbe501eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 0x7fbe501eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500a7e50 con 0x7fbe5006db30 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500a7cd0 con 0x7fbe500c3a70 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500a7fd0 con 0x7fbe5006d750 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe5006db30 0x7fbe501eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe5006db30 0x7fbe501eaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32892/0 (socket says 172.21.15.26:32892) 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 -- 172.21.15.26:0/3857134006 learned_addr learned my addr 172.21.15.26:0/3857134006 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe4e59c640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006d750 0x7fbe5014cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe4ed9d640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 0x7fbe501eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 -- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006d750 msgr2=0x7fbe5014cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006d750 0x7fbe5014cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 -- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 msgr2=0x7fbe501eafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 0x7fbe501eafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 -- 172.21.15.26:0/3857134006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe501edfc0 con 0x7fbe5006db30 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe45d9b640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe5006db30 0x7fbe501eaa90 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fbe38036040 tx=0x7fbe380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe477fe640 1 -- 172.21.15.26:0/3857134006 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38042070 con 0x7fbe5006db30 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe477fe640 1 -- 172.21.15.26:0/3857134006 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3803d890 con 0x7fbe5006db30 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe501ee2a0 con 0x7fbe5006db30 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe477fe640 1 -- 172.21.15.26:0/3857134006 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38038040 con 0x7fbe5006db30 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.624+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe501ee860 con 0x7fbe5006db30 2024-03-06T07:28:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.625+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5004f330 con 0x7fbe5006db30 2024-03-06T07:28:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.626+0000 7fbe477fe640 1 -- 172.21.15.26:0/3857134006 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe3803b070 con 0x7fbe5006db30 2024-03-06T07:28:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.626+0000 7fbe477fe640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe3006b4d0 0x7fbe3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.627+0000 7fbe477fe640 1 -- 172.21.15.26:0/3857134006 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fbe380ab9b0 con 0x7fbe5006db30 2024-03-06T07:28:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.627+0000 7fbe4e59c640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe3006b4d0 0x7fbe3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.627+0000 7fbe4e59c640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe3006b4d0 0x7fbe3006d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005880 tx=0x7fbe40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.631+0000 7fbe477fe640 1 -- 172.21.15.26:0/3857134006 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe38030070 con 0x7fbe5006db30 2024-03-06T07:28:01.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.813+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe500a9b50 con 0x7fbe5006db30 2024-03-06T07:28:01.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.814+0000 7fbe477fe640 1 -- 172.21.15.26:0/3857134006 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe38079fd0 con 0x7fbe5006db30 2024-03-06T07:28:01.815 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:28:01.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe3006b4d0 msgr2=0x7fbe3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe3006b4d0 0x7fbe3006d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005880 tx=0x7fbe40009040 comp rx=0 tx=0).stop 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe5006db30 msgr2=0x7fbe501eaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe5006db30 0x7fbe501eaa90 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fbe38036040 tx=0x7fbe380046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 shutdown_connections 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe3006b4d0 0x7fbe3006d990 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe500c3a70 0x7fbe501eafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe5006db30 0x7fbe501eaa90 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 --2- 172.21.15.26:0/3857134006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe5006d750 0x7fbe5014cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 >> 172.21.15.26:0/3857134006 conn(0x7fbe500c5560 msgr2=0x7fbe5006eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 shutdown_connections 2024-03-06T07:28:01.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.816+0000 7fbe54c4e640 1 -- 172.21.15.26:0/3857134006 wait complete. 2024-03-06T07:28:01.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:28:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.934+0000 7fb83a193640 1 Processor -- start 2024-03-06T07:28:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.934+0000 7fb83a193640 1 -- start start 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.934+0000 7fb83a193640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8341529d0 0x7fb8340c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.934+0000 7fb83a193640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb834072cd0 0x7fb834155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.934+0000 7fb83a193640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb834152db0 con 0x7fb8340c57e0 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.934+0000 7fb83a193640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340c6dd0 con 0x7fb8341529d0 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.934+0000 7fb83a193640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340c6940 con 0x7fb834072cd0 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb832ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8340c57e0 0x7fb834155120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46178/0 (socket says 172.21.15.26:46178) 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb832ffd640 1 -- 172.21.15.26:0/2661196938 learned_addr learned my addr 172.21.15.26:0/2661196938 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb833fff640 1 --2- 172.21.15.26:0/2661196938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb834072cd0 0x7fb834155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb8337fe640 1 --2- 172.21.15.26:0/2661196938 >> v2:172.21.15.117:3300/0 conn(0x7fb8341529d0 0x7fb8340c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb8327fc640 1 -- 172.21.15.26:0/2661196938 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774309250 0 0) 0x7fb828001110 con 0x7fb8340c57e0 2024-03-06T07:28:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb8327fc640 1 -- 172.21.15.26:0/2661196938 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb81c003520 con 0x7fb8340c57e0 2024-03-06T07:28:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb833fff640 1 -- 172.21.15.26:0/2661196938 >> v2:172.21.15.117:3300/0 conn(0x7fb8341529d0 msgr2=0x7fb8340c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb833fff640 1 --2- 172.21.15.26:0/2661196938 >> v2:172.21.15.117:3300/0 conn(0x7fb8341529d0 0x7fb8340c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb833fff640 1 -- 172.21.15.26:0/2661196938 >> v1:172.21.15.26:6789/0 conn(0x7fb8340c57e0 legacy=0x7fb834155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb833fff640 1 -- 172.21.15.26:0/2661196938 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8341590f0 con 0x7fb834072cd0 2024-03-06T07:28:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.935+0000 7fb8337fe640 1 --2- 172.21.15.26:0/2661196938 >> v2:172.21.15.117:3300/0 conn(0x7fb8341529d0 0x7fb8340c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.936+0000 7fb833fff640 1 --2- 172.21.15.26:0/2661196938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb834072cd0 0x7fb834155830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009d90 tx=0x7fb82c02bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84751f6a38f0a60d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.936+0000 7fb8327fc640 1 -- 172.21.15.26:0/2661196938 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c039070 con 0x7fb834072cd0 2024-03-06T07:28:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.936+0000 7fb8327fc640 1 -- 172.21.15.26:0/2661196938 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c02cb30 con 0x7fb834072cd0 2024-03-06T07:28:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.936+0000 7fb8327fc640 1 -- 172.21.15.26:0/2661196938 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c034440 con 0x7fb834072cd0 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.936+0000 7fb83a193640 1 -- 172.21.15.26:0/2661196938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb834072cd0 msgr2=0x7fb834155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.936+0000 7fb83a193640 1 --2- 172.21.15.26:0/2661196938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb834072cd0 0x7fb834155830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009d90 tx=0x7fb82c02bd80 comp rx=0 tx=0).stop 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.937+0000 7fb83a193640 1 -- 172.21.15.26:0/2661196938 shutdown_connections 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.937+0000 7fb83a193640 1 --2- 172.21.15.26:0/2661196938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb834072cd0 0x7fb834155830 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.937+0000 7fb83a193640 1 --2- 172.21.15.26:0/2661196938 >> v2:172.21.15.117:3300/0 conn(0x7fb8341529d0 0x7fb8340c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.937+0000 7fb83a193640 1 -- 172.21.15.26:0/2661196938 >> 172.21.15.26:0/2661196938 conn(0x7fb8340aa630 msgr2=0x7fb8340aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.937+0000 7fb83a193640 1 -- 172.21.15.26:0/2661196938 shutdown_connections 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.937+0000 7fb83a193640 1 -- 172.21.15.26:0/2661196938 wait complete. 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.938+0000 7fb83a193640 1 Processor -- start 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.938+0000 7fb83a193640 1 -- start start 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.938+0000 7fb83a193640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb834072cd0 0x7fb8341dfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.938+0000 7fb83a193640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c57e0 0x7fb834157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb83a193640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8341529d0 0x7fb8341ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb83a193640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb834072720 con 0x7fb8341529d0 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb83a193640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb834155d70 con 0x7fb834072cd0 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb83a193640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb834159370 con 0x7fb8340c57e0 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8341529d0 0x7fb8341ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8341529d0 0x7fb8341ee3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32908/0 (socket says 172.21.15.26:32908) 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 -- 172.21.15.26:0/2541442436 learned_addr learned my addr 172.21.15.26:0/2541442436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb8337fe640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb834072cd0 0x7fb8341dfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb832ffd640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c57e0 0x7fb834157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 -- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c57e0 msgr2=0x7fb834157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c57e0 0x7fb834157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 -- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb834072cd0 msgr2=0x7fb8341dfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb834072cd0 0x7fb8341dfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 -- 172.21.15.26:0/2541442436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8341df690 con 0x7fb8341529d0 2024-03-06T07:28:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb833fff640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8341529d0 0x7fb8341ee3f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009eb0 tx=0x7fb82c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb81bfff640 1 -- 172.21.15.26:0/2541442436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c039050 con 0x7fb8341529d0 2024-03-06T07:28:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb81bfff640 1 -- 172.21.15.26:0/2541442436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c045070 con 0x7fb8341529d0 2024-03-06T07:28:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8340ab930 con 0x7fb8341529d0 2024-03-06T07:28:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb81bfff640 1 -- 172.21.15.26:0/2541442436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c009450 con 0x7fb8341529d0 2024-03-06T07:28:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.939+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8340abef0 con 0x7fb8341529d0 2024-03-06T07:28:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.940+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83404f330 con 0x7fb8341529d0 2024-03-06T07:28:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.941+0000 7fb81bfff640 1 -- 172.21.15.26:0/2541442436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb82c03e850 con 0x7fb8341529d0 2024-03-06T07:28:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.941+0000 7fb81bfff640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.941+0000 7fb81bfff640 1 -- 172.21.15.26:0/2541442436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb82c0ad110 con 0x7fb8341529d0 2024-03-06T07:28:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.942+0000 7fb8337fe640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.942+0000 7fb8337fe640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb834154700 tx=0x7fb824009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:01.946+0000 7fb81bfff640 1 -- 172.21.15.26:0/2541442436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb8341590f0 con 0x7fb8341529d0 2024-03-06T07:28:02.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.126+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb834152180 con 0x7fb8341529d0 2024-03-06T07:28:02.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.127+0000 7fb81bfff640 1 -- 172.21.15.26:0/2541442436 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb82c07b860 con 0x7fb8341529d0 2024-03-06T07:28:02.128 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:28:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 msgr2=0x7fb82006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb834154700 tx=0x7fb824009190 comp rx=0 tx=0).stop 2024-03-06T07:28:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8341529d0 msgr2=0x7fb8341ee3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8341529d0 0x7fb8341ee3f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009eb0 tx=0x7fb82c004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 shutdown_connections 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8341529d0 0x7fb8341ee3f0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8340c57e0 0x7fb834157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 --2- 172.21.15.26:0/2541442436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb834072cd0 0x7fb8341dfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.129+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 >> 172.21.15.26:0/2541442436 conn(0x7fb8340aa630 msgr2=0x7fb8340a9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.130+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 shutdown_connections 2024-03-06T07:28:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.130+0000 7fb83a193640 1 -- 172.21.15.26:0/2541442436 wait complete. 2024-03-06T07:28:02.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:28:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.246+0000 7f2bc6f35640 1 Processor -- start 2024-03-06T07:28:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.246+0000 7f2bc6f35640 1 -- start start 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bc6f35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2bc006bff0 0x7f2bc00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bc6f35640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2bc00c7210 0x7f2bc00c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bc6f35640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc00b04b0 con 0x7f2bc00b00d0 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bc6f35640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc0156160 con 0x7f2bc006bff0 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bc6f35640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc00c5710 con 0x7f2bc00c7210 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bbffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2bc006bff0 0x7f2bc00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bbffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2bc006bff0 0x7f2bc00c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32922/0 (socket says 172.21.15.26:32922) 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bbffff640 1 -- 172.21.15.26:0/2379025720 learned_addr learned my addr 172.21.15.26:0/2379025720 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bc54d2640 1 --2- 172.21.15.26:0/2379025720 >> v2:172.21.15.117:3300/0 conn(0x7f2bc00c7210 0x7f2bc00c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bbffff640 1 -- 172.21.15.26:0/2379025720 >> v2:172.21.15.117:3300/0 conn(0x7f2bc00c7210 msgr2=0x7f2bc00c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bbffff640 1 --2- 172.21.15.26:0/2379025720 >> v2:172.21.15.117:3300/0 conn(0x7f2bc00c7210 0x7f2bc00c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bbffff640 1 -- 172.21.15.26:0/2379025720 >> v1:172.21.15.117:6789/0 conn(0x7f2bc00b00d0 legacy=0x7f2bc006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.247+0000 7f2bbffff640 1 -- 172.21.15.26:0/2379025720 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc006d210 con 0x7f2bc006bff0 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.248+0000 7f2bbffff640 1 --2- 172.21.15.26:0/2379025720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc006bff0 0x7f2bc00c8ff0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00be2e0 tx=0x7f2bac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b4a5096eac9998 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.248+0000 7f2bbf7fe640 1 -- 172.21.15.26:0/2379025720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac034070 con 0x7f2bc006bff0 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.248+0000 7f2bbf7fe640 1 -- 172.21.15.26:0/2379025720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac030860 con 0x7f2bc006bff0 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.248+0000 7f2bbf7fe640 1 -- 172.21.15.26:0/2379025720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac030c30 con 0x7f2bc006bff0 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.248+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2379025720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc006bff0 msgr2=0x7f2bc00c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.248+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2379025720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc006bff0 0x7f2bc00c8ff0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00be2e0 tx=0x7f2bac02b9a0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.249+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2379025720 shutdown_connections 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.249+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2379025720 >> v2:172.21.15.117:3300/0 conn(0x7f2bc00c7210 0x7f2bc00c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.249+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2379025720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc006bff0 0x7f2bc00c8ff0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.249+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2379025720 >> 172.21.15.26:0/2379025720 conn(0x7f2bc00ac810 msgr2=0x7f2bc00acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.249+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2379025720 shutdown_connections 2024-03-06T07:28:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.249+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2379025720 wait complete. 2024-03-06T07:28:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.250+0000 7f2bc6f35640 1 Processor -- start 2024-03-06T07:28:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.250+0000 7f2bc6f35640 1 -- start start 2024-03-06T07:28:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc6f35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc006bff0 0x7f2bc014d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc6f35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc00b00d0 0x7f2bc00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc6f35640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc00c7210 0x7f2bc00be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc6f35640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00c5ba0 con 0x7f2bc00b00d0 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc6f35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0152c50 con 0x7f2bc006bff0 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc6f35640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00c5d20 con 0x7f2bc00c7210 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc00b00d0 0x7f2bc00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc4cd1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc006bff0 0x7f2bc014d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc00b00d0 0x7f2bc00ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32924/0 (socket says 172.21.15.26:32924) 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 -- 172.21.15.26:0/2490351454 learned_addr learned my addr 172.21.15.26:0/2490351454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc54d2640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc00c7210 0x7f2bc00be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 -- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc00c7210 msgr2=0x7f2bc00be9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc00c7210 0x7f2bc00be9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 -- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc006bff0 msgr2=0x7f2bc014d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc006bff0 0x7f2bc014d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 -- 172.21.15.26:0/2490351454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc014dc90 con 0x7f2bc00b00d0 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bc4cd1640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc006bff0 0x7f2bc014d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbffff640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc00b00d0 0x7f2bc00ca040 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f2bac02b9e0 tx=0x7f2bac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbd7fa640 1 -- 172.21.15.26:0/2490351454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac048070 con 0x7f2bc00b00d0 2024-03-06T07:28:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.252+0000 7f2bbd7fa640 1 -- 172.21.15.26:0/2490351454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac030660 con 0x7f2bc00b00d0 2024-03-06T07:28:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.253+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc014e0f0 con 0x7f2bc00b00d0 2024-03-06T07:28:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.253+0000 7f2bbd7fa640 1 -- 172.21.15.26:0/2490351454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac034040 con 0x7f2bc00b00d0 2024-03-06T07:28:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.253+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc01eb3b0 con 0x7f2bc00b00d0 2024-03-06T07:28:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.254+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc01547a0 con 0x7f2bc00b00d0 2024-03-06T07:28:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.254+0000 7f2bbd7fa640 1 -- 172.21.15.26:0/2490351454 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bac04b070 con 0x7f2bc00b00d0 2024-03-06T07:28:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.255+0000 7f2bbd7fa640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ba406b4d0 0x7f2ba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.255+0000 7f2bbd7fa640 1 -- 172.21.15.26:0/2490351454 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2bac0ac1b0 con 0x7f2bc00b00d0 2024-03-06T07:28:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.255+0000 7f2bc4cd1640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ba406b4d0 0x7f2ba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.255+0000 7f2bc4cd1640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ba406b4d0 0x7f2ba406d990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00ad160 tx=0x7f2bb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.259+0000 7f2bbd7fa640 1 -- 172.21.15.26:0/2490351454 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2bc01547a0 con 0x7f2bc00b00d0 2024-03-06T07:28:02.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.462+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bc004f330 con 0x7f2bc00b00d0 2024-03-06T07:28:02.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.462+0000 7f2bbd7fa640 1 -- 172.21.15.26:0/2490351454 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bac07a870 con 0x7f2bc00b00d0 2024-03-06T07:28:02.464 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:28:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ba406b4d0 msgr2=0x7f2ba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ba406b4d0 0x7f2ba406d990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00ad160 tx=0x7f2bb4009040 comp rx=0 tx=0).stop 2024-03-06T07:28:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc00b00d0 msgr2=0x7f2bc00ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc00b00d0 0x7f2bc00ca040 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f2bac02b9e0 tx=0x7f2bac004730 comp rx=0 tx=0).stop 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 shutdown_connections 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2ba406b4d0 0x7f2ba406d990 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc00c7210 0x7f2bc00be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc00b00d0 0x7f2bc00ca040 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 --2- 172.21.15.26:0/2490351454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc006bff0 0x7f2bc014d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 >> 172.21.15.26:0/2490351454 conn(0x7f2bc00ac810 msgr2=0x7f2bc0150c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.464+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 shutdown_connections 2024-03-06T07:28:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.465+0000 7f2bc6f35640 1 -- 172.21.15.26:0/2490351454 wait complete. 2024-03-06T07:28:02.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:02.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.576+0000 7f066a42a640 1 Processor -- start 2024-03-06T07:28:02.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.576+0000 7f066a42a640 1 -- start start 2024-03-06T07:28:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f066a42a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06641527b0 0x7f0664159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f066a42a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f06640c7180 0x7f06640c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f066a42a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06640c7c60 con 0x7f0664155b20 2024-03-06T07:28:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f066a42a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0664159640 con 0x7f06641527b0 2024-03-06T07:28:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f066a42a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06640c8df0 con 0x7f06640c7180 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f0663fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06641527b0 0x7f0664159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f06689c7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0664155b20 0x7f06640c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46190/0 (socket says 172.21.15.26:46190) 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f06689c7640 1 -- 172.21.15.26:0/2635105524 learned_addr learned my addr 172.21.15.26:0/2635105524 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.577+0000 7f06637fe640 1 --2- 172.21.15.26:0/2635105524 >> v2:172.21.15.117:3300/0 conn(0x7f06640c7180 0x7f06640c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0663fff640 1 -- 172.21.15.26:0/2635105524 >> v2:172.21.15.117:3300/0 conn(0x7f06640c7180 msgr2=0x7f06640c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0663fff640 1 --2- 172.21.15.26:0/2635105524 >> v2:172.21.15.117:3300/0 conn(0x7f06640c7180 0x7f06640c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0663fff640 1 -- 172.21.15.26:0/2635105524 >> v1:172.21.15.26:6789/0 conn(0x7f0664155b20 legacy=0x7f06640c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0663fff640 1 -- 172.21.15.26:0/2635105524 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06640c8060 con 0x7f06641527b0 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0662ffd640 1 -- 172.21.15.26:0/2635105524 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215514384 0 0) 0x7f065c001490 con 0x7f0664155b20 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0663fff640 1 --2- 172.21.15.26:0/2635105524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06641527b0 0x7f0664159100 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f066408cac0 tx=0x7f065402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8123c23ca254a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0662ffd640 1 -- 172.21.15.26:0/2635105524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06540315a0 con 0x7f06641527b0 2024-03-06T07:28:02.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.578+0000 7f0662ffd640 1 -- 172.21.15.26:0/2635105524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654037070 con 0x7f06641527b0 2024-03-06T07:28:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f0662ffd640 1 -- 172.21.15.26:0/2635105524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654031960 con 0x7f06641527b0 2024-03-06T07:28:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 -- 172.21.15.26:0/2635105524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06641527b0 msgr2=0x7f0664159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 --2- 172.21.15.26:0/2635105524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06641527b0 0x7f0664159100 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f066408cac0 tx=0x7f065402b840 comp rx=0 tx=0).stop 2024-03-06T07:28:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 -- 172.21.15.26:0/2635105524 shutdown_connections 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 --2- 172.21.15.26:0/2635105524 >> v2:172.21.15.117:3300/0 conn(0x7f06640c7180 0x7f06640c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 --2- 172.21.15.26:0/2635105524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06641527b0 0x7f0664159100 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 -- 172.21.15.26:0/2635105524 >> 172.21.15.26:0/2635105524 conn(0x7f06640ac560 msgr2=0x7f06640ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 -- 172.21.15.26:0/2635105524 shutdown_connections 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.579+0000 7f066a42a640 1 -- 172.21.15.26:0/2635105524 wait complete. 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.580+0000 7f066a42a640 1 Processor -- start 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f066a42a640 1 -- start start 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f066a42a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640c7180 0x7f06641eec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f066a42a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06641527b0 0x7f06641eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f066a42a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0664155b20 0x7f0664154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f066a42a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0664155320 con 0x7f06640c7180 2024-03-06T07:28:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f066a42a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06640c7980 con 0x7f0664155b20 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f066a42a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0664155600 con 0x7f06641527b0 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f0663fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640c7180 0x7f06641eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f0663fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640c7180 0x7f06641eec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32948/0 (socket says 172.21.15.26:32948) 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f0663fff640 1 -- 172.21.15.26:0/4083348579 learned_addr learned my addr 172.21.15.26:0/4083348579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f06637fe640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06641527b0 0x7f06641eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.581+0000 7f06689c7640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0664155b20 0x7f0664154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0663fff640 1 -- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06641527b0 msgr2=0x7f06641eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0663fff640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06641527b0 0x7f06641eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0663fff640 1 -- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0664155b20 msgr2=0x7f0664154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0663fff640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0664155b20 0x7f0664154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0663fff640 1 -- 172.21.15.26:0/4083348579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06641574b0 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f06637fe640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06641527b0 0x7f06641eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f06689c7640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0664155b20 0x7f0664154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0663fff640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640c7180 0x7f06641eec80 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f0654009890 tx=0x7f06540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0660ff9640 1 -- 172.21.15.26:0/4083348579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654032630 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0660ff9640 1 -- 172.21.15.26:0/4083348579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06540327d0 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06641ebc30 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f0660ff9640 1 -- 172.21.15.26:0/4083348579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654032bc0 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.582+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06641587e0 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.583+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06641589b0 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.583+0000 7f0660ff9640 1 -- 172.21.15.26:0/4083348579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f065404d3f0 con 0x7f06640c7180 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.584+0000 7f0660ff9640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f065006b4d0 0x7f065006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.584+0000 7f0660ff9640 1 -- 172.21.15.26:0/4083348579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f065402f030 con 0x7f06640c7180 2024-03-06T07:28:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.584+0000 7f06637fe640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f065006b4d0 0x7f065006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.584+0000 7f06637fe640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f065006b4d0 0x7f065006d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f06640ac960 tx=0x7f0658008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.592+0000 7f0660ff9640 1 -- 172.21.15.26:0/4083348579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f06641574b0 con 0x7f06640c7180 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.868+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f066404f330 con 0x7f06640c7180 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.868+0000 7f0660ff9640 1 -- 172.21.15.26:0/4083348579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5009 (secure 0 0 0) 0x7f0654040070 con 0x7f06640c7180 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":181,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:58.700534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9127":{"gid":9127,"name":"b","rank":1,"incarnation":180,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":178,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9127,9852]},"id":14}]} 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 181 2024-03-06T07:28:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f065006b4d0 msgr2=0x7f065006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f065006b4d0 0x7f065006d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f06640ac960 tx=0x7f0658008040 comp rx=0 tx=0).stop 2024-03-06T07:28:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640c7180 msgr2=0x7f06641eec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640c7180 0x7f06641eec80 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f0654009890 tx=0x7f06540046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 shutdown_connections 2024-03-06T07:28:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f065006b4d0 0x7f065006d990 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0664155b20 0x7f0664154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06641527b0 0x7f06641eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 --2- 172.21.15.26:0/4083348579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640c7180 0x7f06641eec80 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.870+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 >> 172.21.15.26:0/4083348579 conn(0x7f06640ac560 msgr2=0x7f06640aad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.871+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 shutdown_connections 2024-03-06T07:28:02.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.871+0000 7f066a42a640 1 -- 172.21.15.26:0/4083348579 wait complete. 2024-03-06T07:28:02.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.983+0000 7efc9620e640 1 Processor -- start 2024-03-06T07:28:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.983+0000 7efc9620e640 1 -- start start 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc9620e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc900c88c0 0x7efc90155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc9620e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc900c73d0 con 0x7efc90158b60 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc9620e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc900be2e0 con 0x7efc9014f360 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc9620e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc90159700 con 0x7efc900c88c0 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc90158b60 0x7efc90158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46202/0 (socket says 172.21.15.26:46202) 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8ffff640 1 -- 172.21.15.26:0/2605848797 learned_addr learned my addr 172.21.15.26:0/2605848797 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8effd640 1 --2- 172.21.15.26:0/2605848797 >> v2:172.21.15.117:3300/0 conn(0x7efc900c88c0 0x7efc90155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8e7fc640 1 -- 172.21.15.26:0/2605848797 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537970584 0 0) 0x7efc900c73d0 con 0x7efc90158b60 2024-03-06T07:28:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8e7fc640 1 -- 172.21.15.26:0/2605848797 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc78003520 con 0x7efc90158b60 2024-03-06T07:28:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8effd640 1 -- 172.21.15.26:0/2605848797 >> v1:172.21.15.117:6789/0 conn(0x7efc9014f360 legacy=0x7efc900c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8effd640 1 -- 172.21.15.26:0/2605848797 >> v1:172.21.15.26:6789/0 conn(0x7efc90158b60 legacy=0x7efc90158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.984+0000 7efc8effd640 1 -- 172.21.15.26:0/2605848797 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900c7670 con 0x7efc900c88c0 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.985+0000 7efc8effd640 1 --2- 172.21.15.26:0/2605848797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 0x7efc90155c10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efc800097a0 tx=0x7efc8002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69abc569f1ae0347 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.985+0000 7efc8e7fc640 1 -- 172.21.15.26:0/2605848797 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80035020 con 0x7efc900c88c0 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.985+0000 7efc8e7fc640 1 -- 172.21.15.26:0/2605848797 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80031830 con 0x7efc900c88c0 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.985+0000 7efc8e7fc640 1 -- 172.21.15.26:0/2605848797 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80031be0 con 0x7efc900c88c0 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.986+0000 7efc9620e640 1 -- 172.21.15.26:0/2605848797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 msgr2=0x7efc90155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.986+0000 7efc9620e640 1 --2- 172.21.15.26:0/2605848797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 0x7efc90155c10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efc800097a0 tx=0x7efc8002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.986+0000 7efc9620e640 1 -- 172.21.15.26:0/2605848797 shutdown_connections 2024-03-06T07:28:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.986+0000 7efc9620e640 1 --2- 172.21.15.26:0/2605848797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 0x7efc90155c10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.986+0000 7efc9620e640 1 -- 172.21.15.26:0/2605848797 >> 172.21.15.26:0/2605848797 conn(0x7efc900ac570 msgr2=0x7efc900ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.986+0000 7efc9620e640 1 -- 172.21.15.26:0/2605848797 shutdown_connections 2024-03-06T07:28:02.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.986+0000 7efc9620e640 1 -- 172.21.15.26:0/2605848797 wait complete. 2024-03-06T07:28:02.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.987+0000 7efc9620e640 1 Processor -- start 2024-03-06T07:28:02.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.988+0000 7efc9620e640 1 -- start start 2024-03-06T07:28:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc9620e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 0x7efc900af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc9620e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc9014f360 0x7efc900bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc9620e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc90158b60 0x7efc900c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc9620e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900c7c30 con 0x7efc90158b60 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc9620e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900c5a50 con 0x7efc900c88c0 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc9620e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900c5bd0 con 0x7efc9014f360 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc90158b60 0x7efc900c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc90158b60 0x7efc900c0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32952/0 (socket says 172.21.15.26:32952) 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 -- 172.21.15.26:0/1726323800 learned_addr learned my addr 172.21.15.26:0/1726323800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8effd640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc9014f360 0x7efc900bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8f7fe640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 0x7efc900af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 -- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc9014f360 msgr2=0x7efc900bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc9014f360 0x7efc900bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 -- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 msgr2=0x7efc900af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 0x7efc900af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 -- 172.21.15.26:0/1726323800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900ad3c0 con 0x7efc90158b60 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc8ffff640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc90158b60 0x7efc900c0360 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7efc8800cad0 tx=0x7efc8800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.989+0000 7efc77fff640 1 -- 172.21.15.26:0/1726323800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88017070 con 0x7efc90158b60 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.990+0000 7efc77fff640 1 -- 172.21.15.26:0/1726323800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8800ac30 con 0x7efc90158b60 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.990+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc901ebb20 con 0x7efc90158b60 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.990+0000 7efc77fff640 1 -- 172.21.15.26:0/1726323800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88012400 con 0x7efc90158b60 2024-03-06T07:28:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.990+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc900ad940 con 0x7efc90158b60 2024-03-06T07:28:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.991+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9004f330 con 0x7efc90158b60 2024-03-06T07:28:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.991+0000 7efc77fff640 1 -- 172.21.15.26:0/1726323800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc8801a070 con 0x7efc90158b60 2024-03-06T07:28:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.991+0000 7efc77fff640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc7c06b4d0 0x7efc7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.991+0000 7efc77fff640 1 -- 172.21.15.26:0/1726323800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7efc8808a070 con 0x7efc90158b60 2024-03-06T07:28:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.992+0000 7efc8f7fe640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc7c06b4d0 0x7efc7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.992+0000 7efc8f7fe640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc7c06b4d0 0x7efc7c06d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7efc84001780 tx=0x7efc84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:02.996+0000 7efc77fff640 1 -- 172.21.15.26:0/1726323800 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc88058730 con 0x7efc90158b60 2024-03-06T07:28:03.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.177+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc900c8d50 con 0x7efc90158b60 2024-03-06T07:28:03.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.178+0000 7efc77fff640 1 -- 172.21.15.26:0/1726323800 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17081 (secure 0 0 0) 0x7efc8805c7d0 con 0x7efc90158b60 2024-03-06T07:28:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":186,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:51.668092+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:30.757647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:31.458902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]},{"pgid":"29.1d","mappings":[{"from":6,"to":0}]},{"pgid":"29.1e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc7c06b4d0 msgr2=0x7efc7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc7c06b4d0 0x7efc7c06d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7efc84001780 tx=0x7efc84009040 comp rx=0 tx=0).stop 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc90158b60 msgr2=0x7efc900c0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc90158b60 0x7efc900c0360 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7efc8800cad0 tx=0x7efc8800cf90 comp rx=0 tx=0).stop 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 shutdown_connections 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc7c06b4d0 0x7efc7c06d990 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc90158b60 0x7efc900c0360 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc9014f360 0x7efc900bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 --2- 172.21.15.26:0/1726323800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc900c88c0 0x7efc900af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 >> 172.21.15.26:0/1726323800 conn(0x7efc900ac570 msgr2=0x7efc900c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 shutdown_connections 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.180+0000 7efc9620e640 1 -- 172.21.15.26:0/1726323800 wait complete. 2024-03-06T07:28:03.193 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:28:03.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:03.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.293+0000 7f499eb5c640 1 Processor -- start 2024-03-06T07:28:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.293+0000 7f499eb5c640 1 -- start start 2024-03-06T07:28:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.293+0000 7f499eb5c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4998152920 0x7f4998155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f499eb5c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49980c2190 0x7f49980c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f499eb5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49980c1280 0x7f49980c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f499eb5c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4998156160 con 0x7f49980c2190 2024-03-06T07:28:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f499eb5c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499806afc0 con 0x7f4998152920 2024-03-06T07:28:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f499eb5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499806b2a0 con 0x7f49980c1280 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49980c2190 0x7f49980c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49980c2190 0x7f49980c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32958/0 (socket says 172.21.15.26:32958) 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 -- 172.21.15.26:0/1123883827 learned_addr learned my addr 172.21.15.26:0/1123883827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f499d0f9640 1 --2- 172.21.15.26:0/1123883827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49980c1280 0x7f49980c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f499c8f8640 1 --2- 172.21.15.26:0/1123883827 >> v2:172.21.15.117:3300/0 conn(0x7f4998152920 0x7f4998155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 -- 172.21.15.26:0/1123883827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49980c1280 msgr2=0x7f49980c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 --2- 172.21.15.26:0/1123883827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49980c1280 0x7f49980c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 -- 172.21.15.26:0/1123883827 >> v2:172.21.15.117:3300/0 conn(0x7f4998152920 msgr2=0x7f4998155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 --2- 172.21.15.26:0/1123883827 >> v2:172.21.15.117:3300/0 conn(0x7f4998152920 0x7f4998155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.294+0000 7f4997fff640 1 -- 172.21.15.26:0/1123883827 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499806b580 con 0x7f49980c2190 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f4997fff640 1 --2- 172.21.15.26:0/1123883827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49980c2620 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f4984009870 tx=0x7f498402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d926164ac26e41e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f49977fe640 1 -- 172.21.15.26:0/1123883827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984035020 con 0x7f49980c2190 2024-03-06T07:28:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f49977fe640 1 -- 172.21.15.26:0/1123883827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984031640 con 0x7f49980c2190 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f49977fe640 1 -- 172.21.15.26:0/1123883827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498402d040 con 0x7f49980c2190 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f499eb5c640 1 -- 172.21.15.26:0/1123883827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 msgr2=0x7f49980c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f499eb5c640 1 --2- 172.21.15.26:0/1123883827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49980c2620 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f4984009870 tx=0x7f498402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f499eb5c640 1 -- 172.21.15.26:0/1123883827 shutdown_connections 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f499eb5c640 1 --2- 172.21.15.26:0/1123883827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49980c1280 0x7f49980c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f499eb5c640 1 --2- 172.21.15.26:0/1123883827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49980c2620 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f499eb5c640 1 --2- 172.21.15.26:0/1123883827 >> v2:172.21.15.117:3300/0 conn(0x7f4998152920 0x7f4998155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.295+0000 7f499eb5c640 1 -- 172.21.15.26:0/1123883827 >> 172.21.15.26:0/1123883827 conn(0x7f49980c4290 msgr2=0x7f49980c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.296+0000 7f499eb5c640 1 -- 172.21.15.26:0/1123883827 shutdown_connections 2024-03-06T07:28:03.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.296+0000 7f499eb5c640 1 -- 172.21.15.26:0/1123883827 wait complete. 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.297+0000 7f499eb5c640 1 Processor -- start 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.297+0000 7f499eb5c640 1 -- start start 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.297+0000 7f499eb5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980c1280 0x7f499814b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.297+0000 7f499eb5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49981549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499eb5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4998152920 0x7f49980c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499eb5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980c1c50 con 0x7f49980c2190 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499eb5c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980c0d60 con 0x7f49980c1280 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499eb5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980c1dd0 con 0x7f4998152920 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49981549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49981549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32970/0 (socket says 172.21.15.26:32970) 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 -- 172.21.15.26:0/2470119627 learned_addr learned my addr 172.21.15.26:0/2470119627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499d0f9640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4998152920 0x7f49980c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499c8f8640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980c1280 0x7f499814b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 -- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4998152920 msgr2=0x7f49980c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4998152920 0x7f49980c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 -- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980c1280 msgr2=0x7f499814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980c1280 0x7f499814b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 -- 172.21.15.26:0/2470119627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4998153dc0 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f4997fff640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49981549d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f49840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f49957fa640 1 -- 172.21.15.26:0/2470119627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984040070 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f49957fa640 1 -- 172.21.15.26:0/2470119627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49840399f0 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f49957fa640 1 -- 172.21.15.26:0/2470119627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984035040 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4998155390 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.298+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49980c6b20 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.299+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499804f330 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.300+0000 7f49957fa640 1 -- 172.21.15.26:0/2470119627 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f498404b060 con 0x7f49980c2190 2024-03-06T07:28:03.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.300+0000 7f49957fa640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.300+0000 7f49957fa640 1 -- 172.21.15.26:0/2470119627 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f49840ab9a0 con 0x7f49980c2190 2024-03-06T07:28:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.300+0000 7f499c8f8640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.301+0000 7f499c8f8640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f497c06b4d0 0x7f497c06d990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f498c009ac0 tx=0x7f498c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.304+0000 7f49957fa640 1 -- 172.21.15.26:0/2470119627 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f498407a060 con 0x7f49980c2190 2024-03-06T07:28:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.525+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4998068460 con 0x7f49980c2190 2024-03-06T07:28:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.525+0000 7f49957fa640 1 -- 172.21.15.26:0/2470119627 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5009 (secure 0 0 0) 0x7f498407e100 con 0x7f49980c2190 2024-03-06T07:28:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:03.527 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":181,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:58.700534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9127":{"gid":9127,"name":"b","rank":1,"incarnation":180,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":178,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9127,9852]},"id":14}]} 2024-03-06T07:28:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 181 2024-03-06T07:28:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f497c06b4d0 msgr2=0x7f497c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f497c06b4d0 0x7f497c06d990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f498c009ac0 tx=0x7f498c009190 comp rx=0 tx=0).stop 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 msgr2=0x7f49981549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49981549d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f49840046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 shutdown_connections 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4998152920 0x7f49980c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49980c2190 0x7f49981549d0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 --2- 172.21.15.26:0/2470119627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49980c1280 0x7f499814b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 >> 172.21.15.26:0/2470119627 conn(0x7f49980c4290 msgr2=0x7f499814ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 shutdown_connections 2024-03-06T07:28:03.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.528+0000 7f499eb5c640 1 -- 172.21.15.26:0/2470119627 wait complete. 2024-03-06T07:28:03.540 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:03.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 Processor -- start 2024-03-06T07:28:03.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 -- start start 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff50414f360 0x7ff504155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5040c88c0 0x7ff5040c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5040c5880 0x7ff504158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5040c8dd0 con 0x7ff5040c5880 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5040c73d0 con 0x7ff50414f360 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.640+0000 7ff50c332640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040be2e0 con 0x7ff5040c88c0 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a8cf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5040c5880 0x7ff504158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a8cf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5040c5880 0x7ff504158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32980/0 (socket says 172.21.15.26:32980) 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a8cf640 1 -- 172.21.15.26:0/894536018 learned_addr learned my addr 172.21.15.26:0/894536018 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a0ce640 1 --2- 172.21.15.26:0/894536018 >> v2:172.21.15.117:3300/0 conn(0x7ff50414f360 0x7ff504155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff5098cd640 1 --2- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5040c88c0 0x7ff5040c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a0ce640 1 -- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5040c88c0 msgr2=0x7ff5040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:03.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a0ce640 1 --2- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5040c88c0 0x7ff5040c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a0ce640 1 -- 172.21.15.26:0/894536018 >> v2:172.21.15.26:3300/0 conn(0x7ff5040c5880 msgr2=0x7ff504158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:03.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a0ce640 1 --2- 172.21.15.26:0/894536018 >> v2:172.21.15.26:3300/0 conn(0x7ff5040c5880 0x7ff504158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a0ce640 1 -- 172.21.15.26:0/894536018 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040c7670 con 0x7ff50414f360 2024-03-06T07:28:03.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.641+0000 7ff50a8cf640 1 --2- 172.21.15.26:0/894536018 >> v2:172.21.15.26:3300/0 conn(0x7ff5040c5880 0x7ff504158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50a0ce640 1 --2- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50414f360 0x7ff504155c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff50408c840 tx=0x7ff50002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24cd6bd1418dc676 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff5090cc640 1 -- 172.21.15.26:0/894536018 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500038070 con 0x7ff50414f360 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff5090cc640 1 -- 172.21.15.26:0/894536018 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff500031700 con 0x7ff50414f360 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff5090cc640 1 -- 172.21.15.26:0/894536018 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500031ad0 con 0x7ff50414f360 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50c332640 1 -- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50414f360 msgr2=0x7ff504155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50c332640 1 --2- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50414f360 0x7ff504155c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff50408c840 tx=0x7ff50002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50c332640 1 -- 172.21.15.26:0/894536018 shutdown_connections 2024-03-06T07:28:03.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50c332640 1 --2- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5040c88c0 0x7ff5040c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50c332640 1 --2- 172.21.15.26:0/894536018 >> v2:172.21.15.26:3300/0 conn(0x7ff5040c5880 0x7ff504158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50c332640 1 --2- 172.21.15.26:0/894536018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50414f360 0x7ff504155c10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.642+0000 7ff50c332640 1 -- 172.21.15.26:0/894536018 >> 172.21.15.26:0/894536018 conn(0x7ff5040ac570 msgr2=0x7ff5040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.643+0000 7ff50c332640 1 -- 172.21.15.26:0/894536018 shutdown_connections 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.643+0000 7ff50c332640 1 -- 172.21.15.26:0/894536018 wait complete. 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 Processor -- start 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 -- start start 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5040c5880 0x7ff50414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5040c88c0 0x7ff504157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff50414f360 0x7ff5040c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5041590a0 con 0x7ff5040c88c0 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040c7c30 con 0x7ff5040c5880 2024-03-06T07:28:03.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50c332640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff504159220 con 0x7ff50414f360 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff5098cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5040c88c0 0x7ff504157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff5098cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5040c88c0 0x7ff504157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32994/0 (socket says 172.21.15.26:32994) 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff5098cd640 1 -- 172.21.15.26:0/3879890463 learned_addr learned my addr 172.21.15.26:0/3879890463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50a0ce640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5040c5880 0x7ff50414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.644+0000 7ff50a8cf640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff50414f360 0x7ff5040c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff5098cd640 1 -- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff50414f360 msgr2=0x7ff5040c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff5098cd640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff50414f360 0x7ff5040c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff5098cd640 1 -- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5040c5880 msgr2=0x7ff50414f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff5098cd640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5040c5880 0x7ff50414f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff5098cd640 1 -- 172.21.15.26:0/3879890463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040c4410 con 0x7ff5040c88c0 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff5098cd640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5040c88c0 0x7ff504157a20 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400e980 tx=0x7ff4f400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff4faffd640 1 -- 172.21.15.26:0/3879890463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4017070 con 0x7ff5040c88c0 2024-03-06T07:28:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff4faffd640 1 -- 172.21.15.26:0/3879890463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f400ca00 con 0x7ff5040c88c0 2024-03-06T07:28:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5040c29d0 con 0x7ff5040c88c0 2024-03-06T07:28:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff4faffd640 1 -- 172.21.15.26:0/3879890463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f400cdd0 con 0x7ff5040c88c0 2024-03-06T07:28:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.645+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff504157f60 con 0x7ff5040c88c0 2024-03-06T07:28:03.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.646+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50404f330 con 0x7ff5040c88c0 2024-03-06T07:28:03.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.647+0000 7ff4faffd640 1 -- 172.21.15.26:0/3879890463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4f401a070 con 0x7ff5040c88c0 2024-03-06T07:28:03.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.647+0000 7ff4faffd640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e806b4d0 0x7ff4e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.647+0000 7ff4faffd640 1 -- 172.21.15.26:0/3879890463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff4f408a3d0 con 0x7ff5040c88c0 2024-03-06T07:28:03.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.648+0000 7ff50a0ce640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e806b4d0 0x7ff4e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.648+0000 7ff50a0ce640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e806b4d0 0x7ff4e806d990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff500035000 tx=0x7ff500008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.652+0000 7ff4faffd640 1 -- 172.21.15.26:0/3879890463 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4f4058970 con 0x7ff5040c88c0 2024-03-06T07:28:03.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.875+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff50406cda0 con 0x7ff5040c88c0 2024-03-06T07:28:03.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.875+0000 7ff4faffd640 1 -- 172.21.15.26:0/3879890463 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5009 (secure 0 0 0) 0x7ff4f405ca10 con 0x7ff5040c88c0 2024-03-06T07:28:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:03.877 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":181,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:32.493419+0000","modified":"2024-03-06T07:27:58.700534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9127":{"gid":9127,"name":"b","rank":1,"incarnation":180,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/3568105233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3568105233},{"type":"v1","addr":"172.21.15.117:6837","nonce":3568105233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":178,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6834/3840578019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3840578019},{"type":"v1","addr":"172.21.15.26:6834","nonce":3840578019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9127,9852]},"id":14}]} 2024-03-06T07:28:03.877 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 181 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e806b4d0 msgr2=0x7ff4e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e806b4d0 0x7ff4e806d990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff500035000 tx=0x7ff500008030 comp rx=0 tx=0).stop 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5040c88c0 msgr2=0x7ff504157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5040c88c0 0x7ff504157a20 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400e980 tx=0x7ff4f400ee40 comp rx=0 tx=0).stop 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 shutdown_connections 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff4e806b4d0 0x7ff4e806d990 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff50414f360 0x7ff5040c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5040c88c0 0x7ff504157a20 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 --2- 172.21.15.26:0/3879890463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5040c5880 0x7ff50414f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 >> 172.21.15.26:0/3879890463 conn(0x7ff5040ac570 msgr2=0x7ff504153160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 shutdown_connections 2024-03-06T07:28:03.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.877+0000 7ff50c332640 1 -- 172.21.15.26:0/3879890463 wait complete. 2024-03-06T07:28:03.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.989+0000 7ff428afa640 1 Processor -- start 2024-03-06T07:28:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.989+0000 7ff428afa640 1 -- start start 2024-03-06T07:28:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff428afa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff42414f360 0x7ff424155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff428afa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4240c88c0 0x7ff4240c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff428afa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff4240c5880 0x7ff424158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff428afa640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240c8dd0 con 0x7ff42414f360 2024-03-06T07:28:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff428afa640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240c73d0 con 0x7ff4240c5880 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff428afa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240be2e0 con 0x7ff4240c88c0 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff42414f360 0x7ff424155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff42414f360 0x7ff424155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33008/0 (socket says 172.21.15.26:33008) 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 -- 172.21.15.26:0/1645972494 learned_addr learned my addr 172.21.15.26:0/1645972494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff41bfff640 1 --2- 172.21.15.26:0/1645972494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4240c88c0 0x7ff4240c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff422d9d640 1 --2- 172.21.15.26:0/1645972494 >> v2:172.21.15.117:3300/0 conn(0x7ff4240c5880 0x7ff424158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 -- 172.21.15.26:0/1645972494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4240c88c0 msgr2=0x7ff4240c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 --2- 172.21.15.26:0/1645972494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4240c88c0 0x7ff4240c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 -- 172.21.15.26:0/1645972494 >> v2:172.21.15.117:3300/0 conn(0x7ff4240c5880 msgr2=0x7ff424158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 --2- 172.21.15.26:0/1645972494 >> v2:172.21.15.117:3300/0 conn(0x7ff4240c5880 0x7ff424158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.990+0000 7ff42259c640 1 -- 172.21.15.26:0/1645972494 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240c7670 con 0x7ff42414f360 2024-03-06T07:28:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.991+0000 7ff42259c640 1 --2- 172.21.15.26:0/1645972494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff42414f360 0x7ff424155c10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff42408cc10 tx=0x7ff41402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0e1b66341f0fd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.991+0000 7ff421d9b640 1 -- 172.21.15.26:0/1645972494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41402e070 con 0x7ff42414f360 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.991+0000 7ff421d9b640 1 -- 172.21.15.26:0/1645972494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414037070 con 0x7ff42414f360 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.991+0000 7ff421d9b640 1 -- 172.21.15.26:0/1645972494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414031a00 con 0x7ff42414f360 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 -- 172.21.15.26:0/1645972494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff42414f360 msgr2=0x7ff424155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 --2- 172.21.15.26:0/1645972494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff42414f360 0x7ff424155c10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff42408cc10 tx=0x7ff41402b880 comp rx=0 tx=0).stop 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 -- 172.21.15.26:0/1645972494 shutdown_connections 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 --2- 172.21.15.26:0/1645972494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4240c88c0 0x7ff4240c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 --2- 172.21.15.26:0/1645972494 >> v2:172.21.15.117:3300/0 conn(0x7ff4240c5880 0x7ff424158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 --2- 172.21.15.26:0/1645972494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff42414f360 0x7ff424155c10 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 -- 172.21.15.26:0/1645972494 >> 172.21.15.26:0/1645972494 conn(0x7ff4240ac570 msgr2=0x7ff4240ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 -- 172.21.15.26:0/1645972494 shutdown_connections 2024-03-06T07:28:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.992+0000 7ff428afa640 1 -- 172.21.15.26:0/1645972494 wait complete. 2024-03-06T07:28:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.994+0000 7ff428afa640 1 Processor -- start 2024-03-06T07:28:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.994+0000 7ff428afa640 1 -- start start 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff428afa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4240c5880 0x7ff4241ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff428afa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4240c88c0 0x7ff424151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff428afa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff42414f360 0x7ff4241524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff428afa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241590a0 con 0x7ff4240c5880 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff428afa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240c7c30 con 0x7ff4240c88c0 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff428afa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff424159220 con 0x7ff42414f360 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff42259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4240c5880 0x7ff4241ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff42259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4240c5880 0x7ff4241ee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33014/0 (socket says 172.21.15.26:33014) 2024-03-06T07:28:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff42259c640 1 -- 172.21.15.26:0/160111852 learned_addr learned my addr 172.21.15.26:0/160111852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff422d9d640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff42414f360 0x7ff4241524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.995+0000 7ff41bfff640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4240c88c0 0x7ff424151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff42259c640 1 -- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff42414f360 msgr2=0x7ff4241524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff42259c640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff42414f360 0x7ff4241524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff42259c640 1 -- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4240c88c0 msgr2=0x7ff424151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff42259c640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4240c88c0 0x7ff424151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff42259c640 1 -- 172.21.15.26:0/160111852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240af980 con 0x7ff4240c5880 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff42259c640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4240c5880 0x7ff4241ee0d0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff414000c00 tx=0x7ff4140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff41affd640 1 -- 172.21.15.26:0/160111852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414032670 con 0x7ff4240c5880 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff41affd640 1 -- 172.21.15.26:0/160111852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414032810 con 0x7ff4240c5880 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4240af6e0 con 0x7ff4240c5880 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff41affd640 1 -- 172.21.15.26:0/160111852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414032c00 con 0x7ff4240c5880 2024-03-06T07:28:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.996+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241eec60 con 0x7ff4240c5880 2024-03-06T07:28:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.997+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42404f330 con 0x7ff4240c5880 2024-03-06T07:28:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.997+0000 7ff41affd640 1 -- 172.21.15.26:0/160111852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff41404c060 con 0x7ff4240c5880 2024-03-06T07:28:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.998+0000 7ff41affd640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff40406b4d0 0x7ff40406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.998+0000 7ff41affd640 1 -- 172.21.15.26:0/160111852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff4140abbf0 con 0x7ff4240c5880 2024-03-06T07:28:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.998+0000 7ff41bfff640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff40406b4d0 0x7ff40406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:03.999+0000 7ff41bfff640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff40406b4d0 0x7ff40406d990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff4240c82c0 tx=0x7ff40c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.002+0000 7ff41affd640 1 -- 172.21.15.26:0/160111852 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff414040070 con 0x7ff4240c5880 2024-03-06T07:28:04.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.190+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff42406cda0 con 0x7ff4240c5880 2024-03-06T07:28:04.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.191+0000 7ff41affd640 1 -- 172.21.15.26:0/160111852 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17081 (secure 0 0 0) 0x7ff41407a250 con 0x7ff4240c5880 2024-03-06T07:28:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:04.193 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":186,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:27:51.668092+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:30.757647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:31.458902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]},{"pgid":"29.1d","mappings":[{"from":6,"to":0}]},{"pgid":"29.1e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff40406b4d0 msgr2=0x7ff40406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff40406b4d0 0x7ff40406d990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff4240c82c0 tx=0x7ff40c009190 comp rx=0 tx=0).stop 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4240c5880 msgr2=0x7ff4241ee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4240c5880 0x7ff4241ee0d0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff414000c00 tx=0x7ff4140046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 shutdown_connections 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff40406b4d0 0x7ff40406d990 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff42414f360 0x7ff4241524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4240c88c0 0x7ff424151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 --2- 172.21.15.26:0/160111852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4240c5880 0x7ff4241ee0d0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 >> 172.21.15.26:0/160111852 conn(0x7ff4240ac570 msgr2=0x7ff42406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 shutdown_connections 2024-03-06T07:28:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.194+0000 7ff428afa640 1 -- 172.21.15.26:0/160111852 wait complete. 2024-03-06T07:28:04.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:28:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.307+0000 7fa23b3b6640 1 Processor -- start 2024-03-06T07:28:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.307+0000 7fa23b3b6640 1 -- start start 2024-03-06T07:28:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa23b3b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2340c69f0 0x7fa2340c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa23b3b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c8950 0x7fa2340acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa23b3b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa234155130 con 0x7fa2340ad2b0 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa23b3b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa234155440 con 0x7fa2340c69f0 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa23b3b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340c5d50 con 0x7fa2340c8950 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa239152640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2340c69f0 0x7fa2340c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa239953640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa2340ad2b0 0x7fa2341558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46218/0 (socket says 172.21.15.26:46218) 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa239953640 1 -- 172.21.15.26:0/1370826633 learned_addr learned my addr 172.21.15.26:0/1370826633 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.308+0000 7fa238951640 1 --2- 172.21.15.26:0/1370826633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c8950 0x7fa2340acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.309+0000 7fa239152640 1 -- 172.21.15.26:0/1370826633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c8950 msgr2=0x7fa2340acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.309+0000 7fa239152640 1 --2- 172.21.15.26:0/1370826633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c8950 0x7fa2340acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.309+0000 7fa239152640 1 -- 172.21.15.26:0/1370826633 >> v1:172.21.15.26:6789/0 conn(0x7fa2340ad2b0 legacy=0x7fa2341558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.309+0000 7fa233fff640 1 -- 172.21.15.26:0/1370826633 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795109007 0 0) 0x7fa224001490 con 0x7fa2340ad2b0 2024-03-06T07:28:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.309+0000 7fa239152640 1 -- 172.21.15.26:0/1370826633 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340c6030 con 0x7fa2340c69f0 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.309+0000 7fa239152640 1 --2- 172.21.15.26:0/1370826633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c69f0 0x7fa2340c6dd0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0097a0 tx=0x7fa22c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6288c53278ab6cd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa233fff640 1 -- 172.21.15.26:0/1370826633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0315a0 con 0x7fa2340c69f0 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa233fff640 1 -- 172.21.15.26:0/1370826633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c037070 con 0x7fa2340c69f0 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa233fff640 1 -- 172.21.15.26:0/1370826633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c031960 con 0x7fa2340c69f0 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 -- 172.21.15.26:0/1370826633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c69f0 msgr2=0x7fa2340c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/1370826633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c69f0 0x7fa2340c6dd0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0097a0 tx=0x7fa22c02b840 comp rx=0 tx=0).stop 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 -- 172.21.15.26:0/1370826633 shutdown_connections 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/1370826633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c8950 0x7fa2340acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/1370826633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c69f0 0x7fa2340c6dd0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 -- 172.21.15.26:0/1370826633 >> 172.21.15.26:0/1370826633 conn(0x7fa2340cae70 msgr2=0x7fa2340cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 -- 172.21.15.26:0/1370826633 shutdown_connections 2024-03-06T07:28:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.310+0000 7fa23b3b6640 1 -- 172.21.15.26:0/1370826633 wait complete. 2024-03-06T07:28:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.311+0000 7fa23b3b6640 1 Processor -- start 2024-03-06T07:28:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.311+0000 7fa23b3b6640 1 -- start start 2024-03-06T07:28:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa23b3b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2340ad2b0 0x7fa234158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa23b3b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c69f0 0x7fa2340c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa23b3b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c8950 0x7fa23406c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa23b3b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234152a20 con 0x7fa2340c8950 2024-03-06T07:28:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa23b3b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234155fe0 con 0x7fa2340ad2b0 2024-03-06T07:28:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa23b3b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa234152ba0 con 0x7fa2340c69f0 2024-03-06T07:28:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c8950 0x7fa23406c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c8950 0x7fa23406c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33036/0 (socket says 172.21.15.26:33036) 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 -- 172.21.15.26:0/885322282 learned_addr learned my addr 172.21.15.26:0/885322282 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa238951640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c69f0 0x7fa2340c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239152640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2340ad2b0 0x7fa234158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 -- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c69f0 msgr2=0x7fa2340c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c69f0 0x7fa2340c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 -- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2340ad2b0 msgr2=0x7fa234158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2340ad2b0 0x7fa234158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.312+0000 7fa239953640 1 -- 172.21.15.26:0/885322282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340c4ac0 con 0x7fa2340c8950 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.313+0000 7fa239953640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c8950 0x7fa23406c280 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c6d0 tx=0x7fa22400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.313+0000 7fa231ffb640 1 -- 172.21.15.26:0/885322282 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224018070 con 0x7fa2340c8950 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.313+0000 7fa231ffb640 1 -- 172.21.15.26:0/885322282 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224004d60 con 0x7fa2340c8950 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.313+0000 7fa231ffb640 1 -- 172.21.15.26:0/885322282 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400a820 con 0x7fa2340c8950 2024-03-06T07:28:04.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.313+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa234157be0 con 0x7fa2340c8950 2024-03-06T07:28:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.313+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341581a0 con 0x7fa2340c8950 2024-03-06T07:28:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.314+0000 7fa231ffb640 1 -- 172.21.15.26:0/885322282 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa22401b070 con 0x7fa2340c8950 2024-03-06T07:28:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.314+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23404f330 con 0x7fa2340c8950 2024-03-06T07:28:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.315+0000 7fa231ffb640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa22006b4d0 0x7fa22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.315+0000 7fa231ffb640 1 -- 172.21.15.26:0/885322282 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fa224089700 con 0x7fa2340c8950 2024-03-06T07:28:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.315+0000 7fa239152640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa22006b4d0 0x7fa22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.315+0000 7fa239152640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa22006b4d0 0x7fa22006d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa22c036000 tx=0x7fa22c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.319+0000 7fa231ffb640 1 -- 172.21.15.26:0/885322282 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa234155130 con 0x7fa2340c8950 2024-03-06T07:28:04.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.641+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa234159550 con 0x7fa2340c8950 2024-03-06T07:28:04.643 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-03-06T07:28:04.642+0000 7f8033011640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:28:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.643+0000 7fa231ffb640 1 -- 172.21.15.26:0/885322282 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v182) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa224057dc0 con 0x7fa2340c8950 2024-03-06T07:28:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:28:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa22006b4d0 msgr2=0x7fa22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa22006b4d0 0x7fa22006d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa22c036000 tx=0x7fa22c03d040 comp rx=0 tx=0).stop 2024-03-06T07:28:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c8950 msgr2=0x7fa23406c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c8950 0x7fa23406c280 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c6d0 tx=0x7fa22400cb90 comp rx=0 tx=0).stop 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 shutdown_connections 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa22006b4d0 0x7fa22006d990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2340c8950 0x7fa23406c280 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2340c69f0 0x7fa2340c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 --2- 172.21.15.26:0/885322282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2340ad2b0 0x7fa234158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.646+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 >> 172.21.15.26:0/885322282 conn(0x7fa2340cae70 msgr2=0x7fa234152d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.647+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 shutdown_connections 2024-03-06T07:28:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.647+0000 7fa23b3b6640 1 -- 172.21.15.26:0/885322282 wait complete. 2024-03-06T07:28:04.649 INFO:tasks.ceph.mds.c.smithi026.stderr: -363> 2024-03-06T07:27:58.405+0000 7f6881960640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:28:04.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:28:04.671 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:28:04.673 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:28:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.764+0000 7f2a2ecdf640 1 Processor -- start 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.764+0000 7f2a2ecdf640 1 -- start start 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a2ecdf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2a280c1820 0x7f2a2806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a2ecdf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a280c2790 con 0x7f2a2806c1f0 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a2ecdf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a280c1540 con 0x7f2a2814ef70 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a2ecdf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a280c3090 con 0x7f2a280c1820 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a27fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a2806c1f0 0x7f2a28154c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46226/0 (socket says 172.21.15.26:46226) 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a27fff640 1 -- 172.21.15.26:0/282261237 learned_addr learned my addr 172.21.15.26:0/282261237 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180304646 0 0) 0x7f2a280c2790 con 0x7f2a2806c1f0 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a2d27c640 1 --2- 172.21.15.26:0/282261237 >> v2:172.21.15.117:3300/0 conn(0x7f2a280c1820 0x7f2a2806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.765+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a10003520 con 0x7f2a2806c1f0 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754644450 0 0) 0x7f2a10003520 con 0x7f2a2806c1f0 2024-03-06T07:28:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 >> v2:172.21.15.117:3300/0 conn(0x7f2a280c1820 msgr2=0x7f2a2806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 --2- 172.21.15.26:0/282261237 >> v2:172.21.15.117:3300/0 conn(0x7f2a280c1820 0x7f2a2806cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 >> v1:172.21.15.117:6789/0 conn(0x7f2a2814ef70 legacy=0x7f2a280c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280c24b0 con 0x7f2a2806c1f0 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f2a1c001670 con 0x7f2a2806c1f0 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 >> v1:172.21.15.26:6789/0 conn(0x7f2a2806c1f0 legacy=0x7f2a28154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100039b0 0x7f2a10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a10004800 0x7f2a10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a100051c0 0x7f2a1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c3090 con 0x7f2a100051c0 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1000c160 con 0x7f2a100039b0 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1000c360 con 0x7f2a10004800 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a2d27c640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a100051c0 0x7f2a1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a2ca7b640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100039b0 0x7f2a10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a27fff640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a10004800 0x7f2a10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a2d27c640 1 -- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a10004800 msgr2=0x7f2a10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a2d27c640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a10004800 0x7f2a10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a2d27c640 1 -- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100039b0 msgr2=0x7f2a10003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a2d27c640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100039b0 0x7f2a10003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.766+0000 7f2a2d27c640 1 -- 172.21.15.26:0/282261237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a10003520 con 0x7f2a100051c0 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.767+0000 7f2a2d27c640 1 -- 172.21.15.26:0/282261237 reap_dead start 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.767+0000 7f2a2d27c640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a100051c0 0x7f2a1000b9c0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009ac0 tx=0x7f2a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768de2045fad64a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.767+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18009690 con 0x7f2a100051c0 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.767+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a18004020 con 0x7f2a100051c0 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.767+0000 7f2a277fe640 1 -- 172.21.15.26:0/282261237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18004410 con 0x7f2a100051c0 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.768+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/282261237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a100051c0 msgr2=0x7f2a1000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.768+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a100051c0 0x7f2a1000b9c0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009ac0 tx=0x7f2a10005830 comp rx=0 tx=0).stop 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.768+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/282261237 shutdown_connections 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.768+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/282261237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a100051c0 0x7f2a1000b9c0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.768+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/282261237 >> 172.21.15.26:0/282261237 conn(0x7f2a280c6710 msgr2=0x7f2a280c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.768+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/282261237 shutdown_connections 2024-03-06T07:28:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.768+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/282261237 wait complete. 2024-03-06T07:28:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.769+0000 7f2a2ecdf640 1 Processor -- start 2024-03-06T07:28:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.769+0000 7f2a2ecdf640 1 -- start start 2024-03-06T07:28:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.769+0000 7f2a2ecdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100051c0 0x7f2a281e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ecdf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a2814f890 0x7f2a28153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ecdf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a2814cf00 0x7f2a2814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ecdf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c1c70 con 0x7f2a2814f890 2024-03-06T07:28:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ecdf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2814f350 con 0x7f2a100051c0 2024-03-06T07:28:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ecdf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2806cad0 con 0x7f2a2814cf00 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a2814f890 0x7f2a28153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a2814f890 0x7f2a28153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33080/0 (socket says 172.21.15.26:33080) 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 -- 172.21.15.26:0/2943127237 learned_addr learned my addr 172.21.15.26:0/2943127237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ca7b640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100051c0 0x7f2a281e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2d27c640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a2814cf00 0x7f2a2814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 -- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a2814cf00 msgr2=0x7f2a2814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a2814cf00 0x7f2a2814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 -- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100051c0 msgr2=0x7f2a281e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100051c0 0x7f2a281e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 -- 172.21.15.26:0/2943127237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28149480 con 0x7f2a2814f890 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a27fff640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a2814f890 0x7f2a28153cf0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c007290 tx=0x7f2a1c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a257fa640 1 -- 172.21.15.26:0/2943127237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00f960 con 0x7f2a2814f890 2024-03-06T07:28:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a257fa640 1 -- 172.21.15.26:0/2943127237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c018070 con 0x7f2a2814f890 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a257fa640 1 -- 172.21.15.26:0/2943127237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00fd60 con 0x7f2a2814f890 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2814df90 con 0x7f2a2814f890 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.770+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2814e550 con 0x7f2a2814f890 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.771+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2804f330 con 0x7f2a2814f890 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.772+0000 7f2a257fa640 1 -- 172.21.15.26:0/2943127237 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a1c01d460 con 0x7f2a2814f890 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.772+0000 7f2a257fa640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a1406b500 0x7f2a1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.772+0000 7f2a257fa640 1 -- 172.21.15.26:0/2943127237 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2a1c08a270 con 0x7f2a2814f890 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.772+0000 7f2a2ca7b640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a1406b500 0x7f2a1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.773+0000 7f2a2ca7b640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a1406b500 0x7f2a1406d9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000f80 tx=0x7f2a20001610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.777+0000 7f2a257fa640 1 -- 172.21.15.26:0/2943127237 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2a1c058880 con 0x7f2a2814f890 2024-03-06T07:28:04.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:04.995+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2a280c1110 con 0x7f2a2814f890 2024-03-06T07:28:05.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.645+0000 7f2a257fa640 1 -- 172.21.15.26:0/2943127237 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v183) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2a1c05c920 con 0x7f2a2814f890 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a1406b500 msgr2=0x7f2a1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a1406b500 0x7f2a1406d9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000f80 tx=0x7f2a20001610 comp rx=0 tx=0).stop 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a2814f890 msgr2=0x7f2a28153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a2814f890 0x7f2a28153cf0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c007290 tx=0x7f2a1c007750 comp rx=0 tx=0).stop 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 shutdown_connections 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2a1406b500 0x7f2a1406d9c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a2814cf00 0x7f2a2814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a2814f890 0x7f2a28153cf0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 --2- 172.21.15.26:0/2943127237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a100051c0 0x7f2a281e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 >> 172.21.15.26:0/2943127237 conn(0x7f2a280c6710 msgr2=0x7f2a2814a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:05.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 shutdown_connections 2024-03-06T07:28:05.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.647+0000 7f2a2ecdf640 1 -- 172.21.15.26:0/2943127237 wait complete. 2024-03-06T07:28:05.662 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.816+0000 7f464c85c640 1 Processor -- start 2024-03-06T07:28:05.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.816+0000 7f464c85c640 1 -- start start 2024-03-06T07:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f464c85c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 0x7f46480c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f464c85c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4648159260 con 0x7f46481550c0 2024-03-06T07:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f464c85c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480a7d70 con 0x7f46480c2330 2024-03-06T07:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f464c85c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46480c3070 con 0x7f464814bb20 2024-03-06T07:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4646d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f46481550c0 0x7f4648155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46236/0 (socket says 172.21.15.26:46236) 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4646d9d640 1 -- 172.21.15.26:0/3820832798 learned_addr learned my addr 172.21.15.26:0/3820832798 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f464659c640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 0x7f46480c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271755554 0 0) 0x7f4648159260 con 0x7f46481550c0 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4634003520 con 0x7f46481550c0 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465297100 0 0) 0x7f46480a7d70 con 0x7f46480c2330 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4648159260 con 0x7f46480c2330 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809725994 0 0) 0x7f4634003520 con 0x7f46481550c0 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 msgr2=0x7f46480c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 0x7f46480c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 >> v1:172.21.15.117:6789/0 conn(0x7f46480c2330 legacy=0x7f46480c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4648159540 con 0x7f46481550c0 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4640001a10 con 0x7f46481550c0 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 >> v1:172.21.15.26:6789/0 conn(0x7f46481550c0 legacy=0x7f4648155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4634003b70 0x7f4634003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 0x7f4634004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4634004d40 0x7f46340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480c3070 con 0x7f4634004d40 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463400bac0 con 0x7f4634003b70 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.817+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463400bcc0 con 0x7f464814bb20 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f4646d9d640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4634004d40 0x7f46340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f464659c640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4634003b70 0x7f4634003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f463dd9b640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 0x7f4634004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f4646d9d640 1 -- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 msgr2=0x7f4634004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f4646d9d640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f464814bb20 0x7f4634004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f4646d9d640 1 -- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4634003b70 msgr2=0x7f4634003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f4646d9d640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4634003b70 0x7f4634003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.818+0000 7f4646d9d640 1 -- 172.21.15.26:0/3820832798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634003520 con 0x7f4634004d40 2024-03-06T07:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f4646d9d640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4634004d40 0x7f46340051c0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f464000a810 tx=0x7f4634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b6ab448a74fc08 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464000a460 con 0x7f4634004d40 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4640034630 con 0x7f4634004d40 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f4645d9b640 1 -- 172.21.15.26:0/3820832798 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464002ce60 con 0x7f4634004d40 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f464c85c640 1 -- 172.21.15.26:0/3820832798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4634004d40 msgr2=0x7f46340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f464c85c640 1 --2- 172.21.15.26:0/3820832798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4634004d40 0x7f46340051c0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f464000a810 tx=0x7f4634005830 comp rx=0 tx=0).stop 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f4646d9d640 1 -- 172.21.15.26:0/3820832798 reap_dead start 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f464c85c640 1 -- 172.21.15.26:0/3820832798 shutdown_connections 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f464c85c640 1 -- 172.21.15.26:0/3820832798 >> 172.21.15.26:0/3820832798 conn(0x7f46480c88f0 msgr2=0x7f46480c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f464c85c640 1 -- 172.21.15.26:0/3820832798 shutdown_connections 2024-03-06T07:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.819+0000 7f464c85c640 1 -- 172.21.15.26:0/3820832798 wait complete. 2024-03-06T07:28:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 Processor -- start 2024-03-06T07:28:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 -- start start 2024-03-06T07:28:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634004d40 0x7f46480c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480aea50 0x7f46480a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46480aa0e0 0x7f46481ee390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480c5f60 con 0x7f46480aea50 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480a80e0 con 0x7f46480aa0e0 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.821+0000 7f464c85c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46480c67f0 con 0x7f4634004d40 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480aea50 0x7f46480a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480aea50 0x7f46480a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33098/0 (socket says 172.21.15.26:33098) 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 -- 172.21.15.26:0/2928564302 learned_addr learned my addr 172.21.15.26:0/2928564302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f464659c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634004d40 0x7f46480c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f4646d9d640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46480aa0e0 0x7f46481ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 -- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634004d40 msgr2=0x7f46480c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634004d40 0x7f46480c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 -- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46480aa0e0 msgr2=0x7f46481ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46480aa0e0 0x7f46481ee390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 -- 172.21.15.26:0/2928564302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46481488a0 con 0x7f46480aea50 2024-03-06T07:28:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f464659c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634004d40 0x7f46480c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f4646d9d640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46480aa0e0 0x7f46481ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463dd9b640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480aea50 0x7f46480a9ba0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f463000f7a0 tx=0x7f463000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463f7fe640 1 -- 172.21.15.26:0/2928564302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000fe50 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463f7fe640 1 -- 172.21.15.26:0/2928564302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4630018070 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4648148b80 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f463f7fe640 1 -- 172.21.15.26:0/2928564302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000d6d0 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.822+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46480a8570 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.823+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4648154ee0 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.824+0000 7f463f7fe640 1 -- 172.21.15.26:0/2928564302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f463000d870 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.824+0000 7f463f7fe640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.824+0000 7f463f7fe640 1 -- 172.21.15.26:0/2928564302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f463008a3e0 con 0x7f46480aea50 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.824+0000 7f464659c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.825+0000 7f464659c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b500 0x7f462806d9c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4638009ab0 tx=0x7f4638008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:05.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:05.829+0000 7f463f7fe640 1 -- 172.21.15.26:0/2928564302 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4648154ee0 con 0x7f46480aea50 2024-03-06T07:28:06.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.010+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f464804f330 con 0x7f46480aea50 2024-03-06T07:28:06.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.727+0000 7f463f7fe640 1 -- 172.21.15.26:0/2928564302 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v188) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4630058fb0 con 0x7f46480aea50 2024-03-06T07:28:06.729 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:28:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b500 msgr2=0x7f462806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b500 0x7f462806d9c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4638009ab0 tx=0x7f4638008040 comp rx=0 tx=0).stop 2024-03-06T07:28:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480aea50 msgr2=0x7f46480a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480aea50 0x7f46480a9ba0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f463000f7a0 tx=0x7f463000fc60 comp rx=0 tx=0).stop 2024-03-06T07:28:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 shutdown_connections 2024-03-06T07:28:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f462806b500 0x7f462806d9c0 secure :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4638009ab0 tx=0x7f4638008040 comp rx=0 tx=0).stop 2024-03-06T07:28:06.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46480aa0e0 0x7f46481ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480aea50 0x7f46480a9ba0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 --2- 172.21.15.26:0/2928564302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634004d40 0x7f46480c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.730+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 >> 172.21.15.26:0/2928564302 conn(0x7f46480c88f0 msgr2=0x7f46480c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.731+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 shutdown_connections 2024-03-06T07:28:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.731+0000 7f464c85c640 1 -- 172.21.15.26:0/2928564302 wait complete. 2024-03-06T07:28:06.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:28:06.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.856+0000 7f399fa78640 1 Processor -- start 2024-03-06T07:28:06.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.856+0000 7f399fa78640 1 -- start start 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399fa78640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39980c2200 0x7f39980c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399fa78640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39980a6cc0 0x7f3998155460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399fa78640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39981596c0 con 0x7f39981559a0 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399fa78640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3998158ab0 con 0x7f39980c2200 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399fa78640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3998158d90 con 0x7f39980a6cc0 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399d814640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39980c2200 0x7f39980c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399d013640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39980a6cc0 0x7f3998155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399d814640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39980c2200 0x7f39980c25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53876/0 (socket says 172.21.15.26:53876) 2024-03-06T07:28:06.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399e015640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f39981559a0 0x7f39980c0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:52000/0 (socket says 172.21.15.26:52000) 2024-03-06T07:28:06.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399d814640 1 -- 172.21.15.26:0/4131026619 learned_addr learned my addr 172.21.15.26:0/4131026619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:06.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399d013640 1 -- 172.21.15.26:0/4131026619 >> v2:172.21.15.117:3300/0 conn(0x7f39980c2200 msgr2=0x7f39980c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:06.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.857+0000 7f399d013640 1 --2- 172.21.15.26:0/4131026619 >> v2:172.21.15.117:3300/0 conn(0x7f39980c2200 0x7f39980c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.858+0000 7f399d013640 1 -- 172.21.15.26:0/4131026619 >> v1:172.21.15.117:6789/0 conn(0x7f39981559a0 legacy=0x7f39980c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.858+0000 7f399d013640 1 -- 172.21.15.26:0/4131026619 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998159070 con 0x7f39980a6cc0 2024-03-06T07:28:06.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.858+0000 7f399c812640 1 -- 172.21.15.26:0/4131026619 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955540676 0 0) 0x7f39981596c0 con 0x7f39981559a0 2024-03-06T07:28:06.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.858+0000 7f399d814640 1 --2- 172.21.15.26:0/4131026619 >> v2:172.21.15.117:3300/0 conn(0x7f39980c2200 0x7f39980c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.858+0000 7f399d013640 1 --2- 172.21.15.26:0/4131026619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39980a6cc0 0x7f3998155460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f39880097a0 tx=0x7f398802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ded048acfd8ce4e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.858+0000 7f399c812640 1 -- 172.21.15.26:0/4131026619 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880315a0 con 0x7f39980a6cc0 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399c812640 1 -- 172.21.15.26:0/4131026619 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3988037070 con 0x7f39980a6cc0 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399c812640 1 -- 172.21.15.26:0/4131026619 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988031960 con 0x7f39980a6cc0 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 -- 172.21.15.26:0/4131026619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39980a6cc0 msgr2=0x7f3998155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 --2- 172.21.15.26:0/4131026619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39980a6cc0 0x7f3998155460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f39880097a0 tx=0x7f398802b840 comp rx=0 tx=0).stop 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 -- 172.21.15.26:0/4131026619 shutdown_connections 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 --2- 172.21.15.26:0/4131026619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39980a6cc0 0x7f3998155460 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 --2- 172.21.15.26:0/4131026619 >> v2:172.21.15.117:3300/0 conn(0x7f39980c2200 0x7f39980c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 -- 172.21.15.26:0/4131026619 >> 172.21.15.26:0/4131026619 conn(0x7f39980c4300 msgr2=0x7f39980c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 -- 172.21.15.26:0/4131026619 shutdown_connections 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.859+0000 7f399fa78640 1 -- 172.21.15.26:0/4131026619 wait complete. 2024-03-06T07:28:06.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.860+0000 7f399fa78640 1 Processor -- start 2024-03-06T07:28:06.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.860+0000 7f399fa78640 1 -- start start 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399fa78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39980a6cc0 0x7f39980a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399fa78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39980c2200 0x7f39981ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399fa78640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39981559a0 0x7f39981ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399fa78640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3998073580 con 0x7f39980c2200 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399fa78640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399806be10 con 0x7f39980a6cc0 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399fa78640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3998158740 con 0x7f39981559a0 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d013640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39980c2200 0x7f39981ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d814640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39980a6cc0 0x7f39980a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d814640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39980a6cc0 0x7f39980a86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:53892/0 (socket says 172.21.15.26:53892) 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d013640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39980c2200 0x7f39981ee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33100/0 (socket says 172.21.15.26:33100) 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d814640 1 -- 172.21.15.26:0/3008433887 learned_addr learned my addr 172.21.15.26:0/3008433887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399e015640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39981559a0 0x7f39981ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d013640 1 -- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39981559a0 msgr2=0x7f39981ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d013640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39981559a0 0x7f39981ee880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d013640 1 -- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39980a6cc0 msgr2=0x7f39980a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d013640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39980a6cc0 0x7f39980a86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.861+0000 7f399d013640 1 -- 172.21.15.26:0/3008433887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998157aa0 con 0x7f39980c2200 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.862+0000 7f399d013640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39980c2200 0x7f39981ee340 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f3988033000 tx=0x7f39880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.862+0000 7f398e7fc640 1 -- 172.21.15.26:0/3008433887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988004230 con 0x7f39980c2200 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.862+0000 7f398e7fc640 1 -- 172.21.15.26:0/3008433887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880043d0 con 0x7f39980c2200 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.862+0000 7f398e7fc640 1 -- 172.21.15.26:0/3008433887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988003850 con 0x7f39980c2200 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.862+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3998157d80 con 0x7f39980c2200 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.862+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3998158340 con 0x7f39980c2200 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.863+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399804f330 con 0x7f39980c2200 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.863+0000 7f398e7fc640 1 -- 172.21.15.26:0/3008433887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39880039f0 con 0x7f39980c2200 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.864+0000 7f398e7fc640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f397c06b4d0 0x7f397c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.864+0000 7f398e7fc640 1 -- 172.21.15.26:0/3008433887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f398802f030 con 0x7f39980c2200 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.864+0000 7f399d814640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f397c06b4d0 0x7f397c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.864+0000 7f399d814640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f397c06b4d0 0x7f397c06d990 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3994009630 tx=0x7f3994009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:06.868+0000 7f398e7fc640 1 -- 172.21.15.26:0/3008433887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3988041070 con 0x7f39980c2200 2024-03-06T07:28:07.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.053+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f399806bbf0 con 0x7f39980c2200 2024-03-06T07:28:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.738+0000 7f398e7fc640 1 -- 172.21.15.26:0/3008433887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v189) v1 ==== 169+0+0 (secure 0 0 0) 0x7f398807a7d0 con 0x7f39980c2200 2024-03-06T07:28:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:28:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f397c06b4d0 msgr2=0x7f397c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f397c06b4d0 0x7f397c06d990 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3994009630 tx=0x7f3994009240 comp rx=0 tx=0).stop 2024-03-06T07:28:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39980c2200 msgr2=0x7f39981ee340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39980c2200 0x7f39981ee340 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f3988033000 tx=0x7f39880046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 shutdown_connections 2024-03-06T07:28:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f397c06b4d0 0x7f397c06d990 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39981559a0 0x7f39981ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39980c2200 0x7f39981ee340 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.740+0000 7f399fa78640 1 --2- 172.21.15.26:0/3008433887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39980a6cc0 0x7f39980a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.741+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 >> 172.21.15.26:0/3008433887 conn(0x7f39980c4300 msgr2=0x7f399806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.741+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 shutdown_connections 2024-03-06T07:28:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.741+0000 7f399fa78640 1 -- 172.21.15.26:0/3008433887 wait complete. 2024-03-06T07:28:07.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.863+0000 7fd6d515e640 1 Processor -- start 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.863+0000 7fd6d515e640 1 -- start start 2024-03-06T07:28:07.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6d515e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d01527f0 0x7fd6d00c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6d515e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6d00c88f0 0x7fd6d0155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6d515e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d01596f0 con 0x7fd6d00c58b0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6d515e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d00c9130 con 0x7fd6d00c88f0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6d515e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0159260 con 0x7fd6d01527f0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6ce59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd6d00c88f0 0x7fd6d0155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6cf59e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd6d00c58b0 0x7fd6d0155750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39314/0 (socket says 172.21.15.26:39314) 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6cf59e640 1 -- 172.21.15.26:0/1466320416 learned_addr learned my addr 172.21.15.26:0/1466320416 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6ced9d640 1 --2- 172.21.15.26:0/1466320416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d01527f0 0x7fd6d00c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6ce59c640 1 -- 172.21.15.26:0/1466320416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d01527f0 msgr2=0x7fd6d00c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6ce59c640 1 --2- 172.21.15.26:0/1466320416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d01527f0 0x7fd6d00c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6ce59c640 1 -- 172.21.15.26:0/1466320416 >> v1:172.21.15.26:6789/0 conn(0x7fd6d00c58b0 legacy=0x7fd6d0155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6ce59c640 1 -- 172.21.15.26:0/1466320416 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00c82a0 con 0x7fd6d00c88f0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6cdd9b640 1 -- 172.21.15.26:0/1466320416 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727826708 0 0) 0x7fd6d01596f0 con 0x7fd6d00c58b0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.864+0000 7fd6ced9d640 1 --2- 172.21.15.26:0/1466320416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d01527f0 0x7fd6d00c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:07.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.865+0000 7fd6ce59c640 1 --2- 172.21.15.26:0/1466320416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00c88f0 0x7fd6d0155210 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80097a0 tx=0x7fd6b802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc381b891e522081 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.865+0000 7fd6cdd9b640 1 -- 172.21.15.26:0/1466320416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8038070 con 0x7fd6d00c88f0 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.865+0000 7fd6cdd9b640 1 -- 172.21.15.26:0/1466320416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b8031700 con 0x7fd6d00c88f0 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.865+0000 7fd6cdd9b640 1 -- 172.21.15.26:0/1466320416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8031ad0 con 0x7fd6d00c88f0 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 -- 172.21.15.26:0/1466320416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00c88f0 msgr2=0x7fd6d0155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 --2- 172.21.15.26:0/1466320416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00c88f0 0x7fd6d0155210 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80097a0 tx=0x7fd6b802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 -- 172.21.15.26:0/1466320416 shutdown_connections 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 --2- 172.21.15.26:0/1466320416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00c88f0 0x7fd6d0155210 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 --2- 172.21.15.26:0/1466320416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d01527f0 0x7fd6d00c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 -- 172.21.15.26:0/1466320416 >> 172.21.15.26:0/1466320416 conn(0x7fd6d00ac580 msgr2=0x7fd6d00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:07.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 -- 172.21.15.26:0/1466320416 shutdown_connections 2024-03-06T07:28:07.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.866+0000 7fd6d515e640 1 -- 172.21.15.26:0/1466320416 wait complete. 2024-03-06T07:28:07.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.867+0000 7fd6d515e640 1 Processor -- start 2024-03-06T07:28:07.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.867+0000 7fd6d515e640 1 -- start start 2024-03-06T07:28:07.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6d515e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d00c58b0 0x7fd6d014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6d515e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d00c88f0 0x7fd6d014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6d515e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d01527f0 0x7fd6d014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6d515e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0158c10 con 0x7fd6d01527f0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6d515e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00c6b90 con 0x7fd6d00c58b0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6d515e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0158ef0 con 0x7fd6d00c88f0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6cf59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d01527f0 0x7fd6d014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6cf59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d01527f0 0x7fd6d014bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48102/0 (socket says 172.21.15.26:48102) 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6cf59e640 1 -- 172.21.15.26:0/83226421 learned_addr learned my addr 172.21.15.26:0/83226421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6ced9d640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d00c58b0 0x7fd6d014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.868+0000 7fd6ce59c640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d00c88f0 0x7fd6d014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6cf59e640 1 -- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d00c88f0 msgr2=0x7fd6d014b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6cf59e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d00c88f0 0x7fd6d014b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6cf59e640 1 -- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d00c58b0 msgr2=0x7fd6d014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6cf59e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d00c58b0 0x7fd6d014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6cf59e640 1 -- 172.21.15.26:0/83226421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d01eb320 con 0x7fd6d01527f0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6cf59e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d01527f0 0x7fd6d014bf30 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c410 tx=0x7fd6c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6c77fe640 1 -- 172.21.15.26:0/83226421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80074e0 con 0x7fd6d01527f0 2024-03-06T07:28:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6c77fe640 1 -- 172.21.15.26:0/83226421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c80047e0 con 0x7fd6d01527f0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01eb600 con 0x7fd6d01527f0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6c77fe640 1 -- 172.21.15.26:0/83226421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8004bd0 con 0x7fd6d01527f0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.869+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01546a0 con 0x7fd6d01527f0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.870+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d004f330 con 0x7fd6d01527f0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.871+0000 7fd6c77fe640 1 -- 172.21.15.26:0/83226421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6c8011070 con 0x7fd6d01527f0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.871+0000 7fd6c77fe640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.871+0000 7fd6c77fe640 1 -- 172.21.15.26:0/83226421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd6c8089af0 con 0x7fd6d01527f0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.871+0000 7fd6ced9d640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.872+0000 7fd6ced9d640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0003b80 tx=0x7fd6c0009890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:07.875+0000 7fd6c77fe640 1 -- 172.21.15.26:0/83226421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd6c8058450 con 0x7fd6d01527f0 2024-03-06T07:28:08.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.066+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd6d00c8080 con 0x7fd6d01527f0 2024-03-06T07:28:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.069+0000 7fd6c77fe640 1 -- 172.21.15.26:0/83226421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd6c805c4f0 con 0x7fd6d01527f0 2024-03-06T07:28:08.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 msgr2=0x7fd6b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0003b80 tx=0x7fd6c0009890 comp rx=0 tx=0).stop 2024-03-06T07:28:08.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d01527f0 msgr2=0x7fd6d014bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d01527f0 0x7fd6d014bf30 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c410 tx=0x7fd6c800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 shutdown_connections 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d01527f0 0x7fd6d014bf30 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d00c88f0 0x7fd6d014b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 --2- 172.21.15.26:0/83226421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d00c58b0 0x7fd6d014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.071+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 >> 172.21.15.26:0/83226421 conn(0x7fd6d00ac580 msgr2=0x7fd6d0151ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.072+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 shutdown_connections 2024-03-06T07:28:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.072+0000 7fd6d515e640 1 -- 172.21.15.26:0/83226421 wait complete. 2024-03-06T07:28:08.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_grow' 2024-03-06T07:28:08.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.184+0000 7fe30588d640 1 Processor -- start 2024-03-06T07:28:08.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.184+0000 7fe30588d640 1 -- start start 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe30588d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3001550c0 0x7fe3001554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe30588d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe3000aa180 0x7fe3000c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe30588d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3000aa660 con 0x7fe3001559e0 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe30588d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3000c4610 con 0x7fe3000aa180 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe30588d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3000aa940 con 0x7fe3001550c0 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2ff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe3000aa180 0x7fe3000c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2f7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe3001559e0 0x7fe30014ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39330/0 (socket says 172.21.15.26:39330) 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2f7fff640 1 -- 172.21.15.26:0/738576663 learned_addr learned my addr 172.21.15.26:0/738576663 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2feffd640 1 --2- 172.21.15.26:0/738576663 >> v2:172.21.15.117:3300/0 conn(0x7fe3001550c0 0x7fe3001554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2ff7fe640 1 -- 172.21.15.26:0/738576663 >> v2:172.21.15.117:3300/0 conn(0x7fe3001550c0 msgr2=0x7fe3001554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2ff7fe640 1 --2- 172.21.15.26:0/738576663 >> v2:172.21.15.117:3300/0 conn(0x7fe3001550c0 0x7fe3001554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2ff7fe640 1 -- 172.21.15.26:0/738576663 >> v1:172.21.15.26:6789/0 conn(0x7fe3001559e0 legacy=0x7fe30014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2fe7fc640 1 -- 172.21.15.26:0/738576663 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937948718 0 0) 0x7fe3000aa660 con 0x7fe3001559e0 2024-03-06T07:28:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.185+0000 7fe2ff7fe640 1 -- 172.21.15.26:0/738576663 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3000c4860 con 0x7fe3000aa180 2024-03-06T07:28:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.186+0000 7fe2ff7fe640 1 --2- 172.21.15.26:0/738576663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000c5060 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fe3000c3fc0 tx=0x7fe2f802dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5324ac284cb3d917 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.186+0000 7fe2fe7fc640 1 -- 172.21.15.26:0/738576663 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f802e950 con 0x7fe3000aa180 2024-03-06T07:28:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.186+0000 7fe2fe7fc640 1 -- 172.21.15.26:0/738576663 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f802eaf0 con 0x7fe3000aa180 2024-03-06T07:28:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.186+0000 7fe2fe7fc640 1 -- 172.21.15.26:0/738576663 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8009540 con 0x7fe3000aa180 2024-03-06T07:28:08.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 -- 172.21.15.26:0/738576663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 msgr2=0x7fe3000c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 --2- 172.21.15.26:0/738576663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000c5060 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fe3000c3fc0 tx=0x7fe2f802dfb0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 -- 172.21.15.26:0/738576663 shutdown_connections 2024-03-06T07:28:08.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 --2- 172.21.15.26:0/738576663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000c5060 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 --2- 172.21.15.26:0/738576663 >> v2:172.21.15.117:3300/0 conn(0x7fe3001550c0 0x7fe3001554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 -- 172.21.15.26:0/738576663 >> 172.21.15.26:0/738576663 conn(0x7fe3000c6710 msgr2=0x7fe3000c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 -- 172.21.15.26:0/738576663 shutdown_connections 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.187+0000 7fe30588d640 1 -- 172.21.15.26:0/738576663 wait complete. 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.188+0000 7fe30588d640 1 Processor -- start 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.188+0000 7fe30588d640 1 -- start start 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe30588d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe30588d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3001550c0 0x7fe300148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe30588d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3001559e0 0x7fe3001489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe30588d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30014f310 con 0x7fe3000aa180 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe30588d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000c1560 con 0x7fe3001559e0 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe30588d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30014f490 con 0x7fe3001550c0 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48122/0 (socket says 172.21.15.26:48122) 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 -- 172.21.15.26:0/1264821242 learned_addr learned my addr 172.21.15.26:0/1264821242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2ff7fe640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3001559e0 0x7fe3001489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2f7fff640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3001550c0 0x7fe300148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 -- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3001550c0 msgr2=0x7fe300148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3001550c0 0x7fe300148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 -- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3001559e0 msgr2=0x7fe3001489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3001559e0 0x7fe3001489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 -- 172.21.15.26:0/1264821242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300148f10 con 0x7fe3000aa180 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2f7fff640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3001550c0 0x7fe300148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.189+0000 7fe2feffd640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000bfdb0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000d620 tx=0x7fe2f000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.190+0000 7fe2fcff9640 1 -- 172.21.15.26:0/1264821242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000b620 con 0x7fe3000aa180 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.190+0000 7fe2fcff9640 1 -- 172.21.15.26:0/1264821242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f0019070 con 0x7fe3000aa180 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.190+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3000a85e0 con 0x7fe3000aa180 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.190+0000 7fe2fcff9640 1 -- 172.21.15.26:0/1264821242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000ba00 con 0x7fe3000aa180 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.190+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3000a8880 con 0x7fe3000aa180 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.191+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3000a9360 con 0x7fe3000aa180 2024-03-06T07:28:08.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.191+0000 7fe2fcff9640 1 -- 172.21.15.26:0/1264821242 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2f000bba0 con 0x7fe3000aa180 2024-03-06T07:28:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.191+0000 7fe2fcff9640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2e006b4d0 0x7fe2e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.191+0000 7fe2fcff9640 1 -- 172.21.15.26:0/1264821242 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe2f008a500 con 0x7fe3000aa180 2024-03-06T07:28:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.192+0000 7fe2f7fff640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2e006b4d0 0x7fe2e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.192+0000 7fe2f7fff640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2e006b4d0 0x7fe2e006d990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe3001eb820 tx=0x7fe2e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.196+0000 7fe2fcff9640 1 -- 172.21.15.26:0/1264821242 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe300148f10 con 0x7fe3000aa180 2024-03-06T07:28:08.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.446+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]} v 0) v1 -- 0x7fe30004f330 con 0x7fe3000aa180 2024-03-06T07:28:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.739+0000 7fe2fcff9640 1 -- 172.21.15.26:0/1264821242 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]}]=0 v467) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe2f00594a0 con 0x7fe3000aa180 2024-03-06T07:28:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2e006b4d0 msgr2=0x7fe2e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2e006b4d0 0x7fe2e006d990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe3001eb820 tx=0x7fe2e8009040 comp rx=0 tx=0).stop 2024-03-06T07:28:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 msgr2=0x7fe3000bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000bfdb0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000d620 tx=0x7fe2f000dae0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 shutdown_connections 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe2e006b4d0 0x7fe2e006d990 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3001559e0 0x7fe3001489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3001550c0 0x7fe300148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 --2- 172.21.15.26:0/1264821242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3000aa180 0x7fe3000bfdb0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 >> 172.21.15.26:0/1264821242 conn(0x7fe3000c6710 msgr2=0x7fe3000c31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 shutdown_connections 2024-03-06T07:28:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.742+0000 7fe30588d640 1 -- 172.21.15.26:0/1264821242 wait complete. 2024-03-06T07:28:08.755 INFO:tasks.cephfs_test_runner:That setting max_mds undoes down. ... ok 2024-03-06T07:28:08.755 INFO:tasks.cephfs_test_runner:Starting test: test_down_health (tasks.cephfs.test_failover.TestClusterResize) That marking a FS down does not generate a health warning 2024-03-06T07:28:08.755 INFO:tasks.cephfs_test_runner:test_down_health (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:28:08.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.856+0000 7fcfd6ced640 1 Processor -- start 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.856+0000 7fcfd6ced640 1 -- start start 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfd6ced640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 0x7fcfd00a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfd6ced640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcfd00c36e0 0x7fcfd01558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfd6ced640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd0158800 con 0x7fcfd00c3280 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfd6ced640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd00c3b10 con 0x7fcfd00c36e0 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfd6ced640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0158ae0 con 0x7fcfd006d9f0 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcfd00c3280 0x7fcfd01551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39336/0 (socket says 172.21.15.26:39336) 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcffff640 1 -- 172.21.15.26:0/681548244 learned_addr learned my addr 172.21.15.26:0/681548244 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfd528a640 1 --2- 172.21.15.26:0/681548244 >> v2:172.21.15.117:3300/0 conn(0x7fcfd00c36e0 0x7fcfd01558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfd4a89640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 0x7fcfd00a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874578048 0 0) 0x7fcfd0158800 con 0x7fcfd00c3280 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb8003520 con 0x7fcfd00c3280 2024-03-06T07:28:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 263105429 0 0) 0x7fcfb8003520 con 0x7fcfd00c3280 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 msgr2=0x7fcfd00a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 0x7fcfd00a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 >> v2:172.21.15.117:3300/0 conn(0x7fcfd00c36e0 msgr2=0x7fcfd01558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 --2- 172.21.15.26:0/681548244 >> v2:172.21.15.117:3300/0 conn(0x7fcfd00c36e0 0x7fcfd01558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.857+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd0158dc0 con 0x7fcfd00c3280 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fcfc4001670 con 0x7fcfd00c3280 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 >> v1:172.21.15.26:6789/0 conn(0x7fcfd00c3280 legacy=0x7fcfd01551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 0x7fcfb80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb8004d40 0x7fcfb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00c3b10 con 0x7fcfb8003f30 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0158ae0 con 0x7fcfb8004d40 2024-03-06T07:28:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb800ba10 con 0x7fcfd006d9f0 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfd528a640 1 --2- 172.21.15.26:0/681548244 >> v2:172.21.15.117:3300/0 conn(0x7fcfd00c36e0 0x7fcfd01558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcffff640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfd4a89640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 0x7fcfb80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfd528a640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb8004d40 0x7fcfb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcffff640 1 -- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 msgr2=0x7fcfb80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcffff640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd006d9f0 0x7fcfb80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcffff640 1 -- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb8004d40 msgr2=0x7fcfb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcffff640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb8004d40 0x7fcfb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.858+0000 7fcfcffff640 1 -- 172.21.15.26:0/681548244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8003520 con 0x7fcfb8003f30 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfcffff640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfb8004800 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c240 tx=0x7fcfb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab37c02f05785767 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4007d50 con 0x7fcfb8003f30 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4030070 con 0x7fcfb8003f30 2024-03-06T07:28:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfcf7fe640 1 -- 172.21.15.26:0/681548244 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc402cd80 con 0x7fcfb8003f30 2024-03-06T07:28:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfd6ced640 1 -- 172.21.15.26:0/681548244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 msgr2=0x7fcfb8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfd6ced640 1 --2- 172.21.15.26:0/681548244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfb8004800 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c240 tx=0x7fcfb8005830 comp rx=0 tx=0).stop 2024-03-06T07:28:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfd528a640 1 -- 172.21.15.26:0/681548244 reap_dead start 2024-03-06T07:28:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfd6ced640 1 -- 172.21.15.26:0/681548244 shutdown_connections 2024-03-06T07:28:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfd6ced640 1 -- 172.21.15.26:0/681548244 >> 172.21.15.26:0/681548244 conn(0x7fcfd00aa630 msgr2=0x7fcfd00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfd6ced640 1 -- 172.21.15.26:0/681548244 shutdown_connections 2024-03-06T07:28:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.859+0000 7fcfd6ced640 1 -- 172.21.15.26:0/681548244 wait complete. 2024-03-06T07:28:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.860+0000 7fcfd6ced640 1 Processor -- start 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd6ced640 1 -- start start 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd6ced640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfd006f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd6ced640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd01ede70 0x7fcfd00aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd6ced640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd00acbf0 0x7fcfd01447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd6ced640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0155e30 con 0x7fcfb8003f30 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd6ced640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0159380 con 0x7fcfd00acbf0 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd6ced640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd006ddd0 con 0x7fcfd01ede70 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd4a89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfd006f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd4a89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfd006f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48130/0 (socket says 172.21.15.26:48130) 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.861+0000 7fcfd4a89640 1 -- 172.21.15.26:0/2561992996 learned_addr learned my addr 172.21.15.26:0/2561992996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd4a89640 1 -- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd01ede70 msgr2=0x7fcfd00aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd4a89640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd01ede70 0x7fcfd00aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd528a640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd00acbf0 0x7fcfd01447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd4a89640 1 -- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd00acbf0 msgr2=0x7fcfd01447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd4a89640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd00acbf0 0x7fcfd01447b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd4a89640 1 -- 172.21.15.26:0/2561992996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd006e2e0 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd4a89640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfd006f0a0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8011e30 tx=0x7fcfc800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfcd7fa640 1 -- 172.21.15.26:0/2561992996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8015630 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfcd7fa640 1 -- 172.21.15.26:0/2561992996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc80157d0 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfcd7fa640 1 -- 172.21.15.26:0/2561992996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8015ba0 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd006e5c0 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.862+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd006fc10 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.863+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd004f330 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.863+0000 7fcfcd7fa640 1 -- 172.21.15.26:0/2561992996 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfc801d400 con 0x7fcfb8003f30 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.864+0000 7fcfcd7fa640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.864+0000 7fcfcd7fa640 1 -- 172.21.15.26:0/2561992996 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fcfc808b220 con 0x7fcfb8003f30 2024-03-06T07:28:08.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.864+0000 7fcfcffff640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.865+0000 7fcfcffff640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4034000 tx=0x7fcfc4007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:08.868+0000 7fcfcd7fa640 1 -- 172.21.15.26:0/2561992996 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcfc8019030 con 0x7fcfb8003f30 2024-03-06T07:28:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.051+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfd0153460 con 0x7fcfb8003f30 2024-03-06T07:28:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.052+0000 7fcfcd7fa640 1 -- 172.21.15.26:0/2561992996 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+13447 (secure 0 0 0) 0x7fcfc8010040 con 0x7fcfb8003f30 2024-03-06T07:28:09.054 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:09.054 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":189,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:07.734805+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/3840578019":"2024-03-07T07:28:04.642600+0000","172.21.15.26:6832/3840578019":"2024-03-07T07:28:04.642600+0000","172.21.15.117:6837/3568105233":"2024-03-07T07:28:04.642584+0000","172.21.15.117:6835/3568105233":"2024-03-07T07:28:04.642584+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfbc06b4d0 msgr2=0x7fcfbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4034000 tx=0x7fcfc4007000 comp rx=0 tx=0).stop 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 msgr2=0x7fcfd006f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfd006f0a0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8011e30 tx=0x7fcfc800f580 comp rx=0 tx=0).stop 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 shutdown_connections 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd00acbf0 0x7fcfd01447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfd01ede70 0x7fcfd00aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 --2- 172.21.15.26:0/2561992996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb8003f30 0x7fcfd006f0a0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 >> 172.21.15.26:0/2561992996 conn(0x7fcfd00aa630 msgr2=0x7fcfd01550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 shutdown_connections 2024-03-06T07:28:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.054+0000 7fcfd6ced640 1 -- 172.21.15.26:0/2561992996 wait complete. 2024-03-06T07:28:09.067 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:28:09.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:28:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.169+0000 7fa26788f640 1 Processor -- start 2024-03-06T07:28:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.169+0000 7fa26788f640 1 -- start start 2024-03-06T07:28:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.170+0000 7fa26788f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 0x7fa260155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.170+0000 7fa26788f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa26006ca70 0x7fa2600c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.170+0000 7fa26788f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2600b0450 con 0x7fa2600c5380 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.170+0000 7fa26788f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26006c020 con 0x7fa26006ca70 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.170+0000 7fa26788f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600c77f0 con 0x7fa2600b0070 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.170+0000 7fa265e2c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa2600c5380 0x7fa26014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39348/0 (socket says 172.21.15.26:39348) 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.170+0000 7fa265e2c640 1 -- 172.21.15.26:0/2184784551 learned_addr learned my addr 172.21.15.26:0/2184784551 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2184784551 >> v2:172.21.15.117:3300/0 conn(0x7fa26006ca70 0x7fa2600c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa26562b640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 0x7fa260155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2555490463 0 0) 0x7fa250001490 con 0x7fa2600c5380 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa248003520 con 0x7fa2600c5380 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307319862 0 0) 0x7fa248003520 con 0x7fa2600c5380 2024-03-06T07:28:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 msgr2=0x7fa260155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 0x7fa260155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 >> v2:172.21.15.117:3300/0 conn(0x7fa26006ca70 msgr2=0x7fa2600c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 --2- 172.21.15.26:0/2184784551 >> v2:172.21.15.117:3300/0 conn(0x7fa26006ca70 0x7fa2600c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600c7ad0 con 0x7fa2600c5380 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fa250001bc0 con 0x7fa2600c5380 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 >> v1:172.21.15.26:6789/0 conn(0x7fa2600c5380 legacy=0x7fa26014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 0x7fa2480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa248004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa248004d40 0x7fa2480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26006c020 con 0x7fa248003f30 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600c77f0 con 0x7fa248004d40 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24800ba10 con 0x7fa2600b0070 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa248004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa26562b640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 0x7fa2480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa265e2c640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa248004d40 0x7fa2480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa264e2a640 1 -- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 msgr2=0x7fa2480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2600b0070 0x7fa2480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa264e2a640 1 -- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa248004d40 msgr2=0x7fa2480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa248004d40 0x7fa2480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.171+0000 7fa264e2a640 1 -- 172.21.15.26:0/2184784551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa248003520 con 0x7fa248003f30 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.172+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa248004800 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fa254009af0 tx=0x7fa248005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3607d95b227f2006 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.172+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254039070 con 0x7fa248003f30 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.172+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25402c710 con 0x7fa248003f30 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.172+0000 7fa25ffff640 1 -- 172.21.15.26:0/2184784551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254009690 con 0x7fa248003f30 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.173+0000 7fa26788f640 1 -- 172.21.15.26:0/2184784551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 msgr2=0x7fa248004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.173+0000 7fa26788f640 1 --2- 172.21.15.26:0/2184784551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa248004800 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fa254009af0 tx=0x7fa248005830 comp rx=0 tx=0).stop 2024-03-06T07:28:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.173+0000 7fa265e2c640 1 -- 172.21.15.26:0/2184784551 reap_dead start 2024-03-06T07:28:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.173+0000 7fa26788f640 1 -- 172.21.15.26:0/2184784551 shutdown_connections 2024-03-06T07:28:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.173+0000 7fa26788f640 1 -- 172.21.15.26:0/2184784551 >> 172.21.15.26:0/2184784551 conn(0x7fa2600ac590 msgr2=0x7fa2600ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.173+0000 7fa26788f640 1 -- 172.21.15.26:0/2184784551 shutdown_connections 2024-03-06T07:28:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.173+0000 7fa26788f640 1 -- 172.21.15.26:0/2184784551 wait complete. 2024-03-06T07:28:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.174+0000 7fa26788f640 1 Processor -- start 2024-03-06T07:28:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.174+0000 7fa26788f640 1 -- start start 2024-03-06T07:28:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26788f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa2600c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26788f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2601540c0 0x7fa2601ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26788f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2601ee7a0 0x7fa26014d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26788f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26006cef0 con 0x7fa248003f30 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26788f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26006c550 con 0x7fa2601ee7a0 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26788f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26006da80 con 0x7fa2601540c0 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa2600c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa2600c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48144/0 (socket says 172.21.15.26:48144) 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 -- 172.21.15.26:0/2686515300 learned_addr learned my addr 172.21.15.26:0/2686515300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa265e2c640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2601ee7a0 0x7fa26014d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2601540c0 0x7fa2601ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 -- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2601540c0 msgr2=0x7fa2601ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2601540c0 0x7fa2601ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 -- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2601ee7a0 msgr2=0x7fa26014d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2601ee7a0 0x7fa26014d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 -- 172.21.15.26:0/2686515300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600c29a0 con 0x7fa248003f30 2024-03-06T07:28:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa265e2c640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2601ee7a0 0x7fa26014d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2601540c0 0x7fa2601ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26562b640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa2600c2230 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fa2580119d0 tx=0x7fa258011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa25dffb640 1 -- 172.21.15.26:0/2686515300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25800fa90 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa25dffb640 1 -- 172.21.15.26:0/2686515300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25800fc30 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2601566d0 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.175+0000 7fa25dffb640 1 -- 172.21.15.26:0/2686515300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258014440 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.176+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa260156c90 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.177+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26004f330 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.177+0000 7fa25dffb640 1 -- 172.21.15.26:0/2686515300 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa258012040 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.177+0000 7fa25dffb640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa24c06b500 0x7fa24c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.178+0000 7fa25dffb640 1 -- 172.21.15.26:0/2686515300 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa25808a620 con 0x7fa248003f30 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.178+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa24c06b500 0x7fa24c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.178+0000 7fa264e2a640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa24c06b500 0x7fa24c06d9c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa254036000 tx=0x7fa254030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.182+0000 7fa25dffb640 1 -- 172.21.15.26:0/2686515300 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa258019030 con 0x7fa248003f30 2024-03-06T07:28:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.362+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa26006d860 con 0x7fa248003f30 2024-03-06T07:28:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.362+0000 7fa25dffb640 1 -- 172.21.15.26:0/2686515300 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa258059520 con 0x7fa248003f30 2024-03-06T07:28:09.364 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa24c06b500 msgr2=0x7fa24c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa24c06b500 0x7fa24c06d9c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa254036000 tx=0x7fa254030000 comp rx=0 tx=0).stop 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 msgr2=0x7fa2600c2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa2600c2230 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fa2580119d0 tx=0x7fa258011e90 comp rx=0 tx=0).stop 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 shutdown_connections 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa24c06b500 0x7fa24c06d9c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2601ee7a0 0x7fa26014d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2601540c0 0x7fa2601ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 --2- 172.21.15.26:0/2686515300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa248003f30 0x7fa2600c2230 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 >> 172.21.15.26:0/2686515300 conn(0x7fa2600ac590 msgr2=0x7fa2600ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 shutdown_connections 2024-03-06T07:28:09.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.365+0000 7fa26788f640 1 -- 172.21.15.26:0/2686515300 wait complete. 2024-03-06T07:28:09.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_health' 2024-03-06T07:28:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.477+0000 7fe21d05c640 1 Processor -- start 2024-03-06T07:28:09.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.477+0000 7fe21d05c640 1 -- start start 2024-03-06T07:28:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe21d05c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2180c5330 0x7fe2180c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe21d05c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe218158db0 0x7fe21815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe21d05c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe218155110 con 0x7fe2180c88b0 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe21d05c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2181559e0 con 0x7fe2180c5330 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe21d05c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180c8f40 con 0x7fe218158db0 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe216d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2180c5330 0x7fe2180c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe216d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2180c5330 0x7fe2180c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48146/0 (socket says 172.21.15.26:48146) 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe216d9d640 1 -- 172.21.15.26:0/4040773932 learned_addr learned my addr 172.21.15.26:0/4040773932 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe21759e640 1 --2- 172.21.15.26:0/4040773932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe218158db0 0x7fe21815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe216d9d640 1 -- 172.21.15.26:0/4040773932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe218158db0 msgr2=0x7fe21815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe216d9d640 1 --2- 172.21.15.26:0/4040773932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe218158db0 0x7fe21815fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe216d9d640 1 -- 172.21.15.26:0/4040773932 >> v1:172.21.15.117:6789/0 conn(0x7fe2180c88b0 legacy=0x7fe2181586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.478+0000 7fe216d9d640 1 -- 172.21.15.26:0/4040773932 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218155390 con 0x7fe2180c5330 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.479+0000 7fe216d9d640 1 --2- 172.21.15.26:0/4040773932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c5330 0x7fe2180c5710 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe210009870 tx=0x7fe21002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca268953a76ccc7b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.479+0000 7fe215d9b640 1 -- 172.21.15.26:0/4040773932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210035020 con 0x7fe2180c5330 2024-03-06T07:28:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.479+0000 7fe215d9b640 1 -- 172.21.15.26:0/4040773932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe210031640 con 0x7fe2180c5330 2024-03-06T07:28:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.479+0000 7fe215d9b640 1 -- 172.21.15.26:0/4040773932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21002d040 con 0x7fe2180c5330 2024-03-06T07:28:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.479+0000 7fe21d05c640 1 -- 172.21.15.26:0/4040773932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c5330 msgr2=0x7fe2180c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.479+0000 7fe21d05c640 1 --2- 172.21.15.26:0/4040773932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c5330 0x7fe2180c5710 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe210009870 tx=0x7fe21002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.480+0000 7fe21d05c640 1 -- 172.21.15.26:0/4040773932 shutdown_connections 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.480+0000 7fe21d05c640 1 --2- 172.21.15.26:0/4040773932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe218158db0 0x7fe21815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.480+0000 7fe21d05c640 1 --2- 172.21.15.26:0/4040773932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c5330 0x7fe2180c5710 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.480+0000 7fe21d05c640 1 -- 172.21.15.26:0/4040773932 >> 172.21.15.26:0/4040773932 conn(0x7fe2180ac560 msgr2=0x7fe2180ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.480+0000 7fe21d05c640 1 -- 172.21.15.26:0/4040773932 shutdown_connections 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.480+0000 7fe21d05c640 1 -- 172.21.15.26:0/4040773932 wait complete. 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.481+0000 7fe21d05c640 1 Processor -- start 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.481+0000 7fe21d05c640 1 -- start start 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21d05c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2180c5330 0x7fe2181573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21d05c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c88b0 0x7fe218151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21d05c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2181eb5a0 0x7fe2181ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21d05c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2181529d0 con 0x7fe2180c88b0 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21d05c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21806d090 con 0x7fe2180c5330 2024-03-06T07:28:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21d05c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe218152b50 con 0x7fe2181eb5a0 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c88b0 0x7fe218151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c88b0 0x7fe218151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48152/0 (socket says 172.21.15.26:48152) 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 -- 172.21.15.26:0/1158213781 learned_addr learned my addr 172.21.15.26:0/1158213781 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe216d9d640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2180c5330 0x7fe2181573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21759e640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2181eb5a0 0x7fe2181ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 -- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2181eb5a0 msgr2=0x7fe2181ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2181eb5a0 0x7fe2181ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 -- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2180c5330 msgr2=0x7fe2181573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2180c5330 0x7fe2181573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 -- 172.21.15.26:0/1158213781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218156620 con 0x7fe2180c88b0 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe216d9d640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2180c5330 0x7fe2181573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe21659c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c88b0 0x7fe218151720 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe20000c440 tx=0x7fe200001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe20f7fe640 1 -- 172.21.15.26:0/1158213781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2000052d0 con 0x7fe2180c88b0 2024-03-06T07:28:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.482+0000 7fe20f7fe640 1 -- 172.21.15.26:0/1158213781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe200005470 con 0x7fe2180c88b0 2024-03-06T07:28:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.483+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe218156900 con 0x7fe2180c88b0 2024-03-06T07:28:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.483+0000 7fe20f7fe640 1 -- 172.21.15.26:0/1158213781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200007260 con 0x7fe2180c88b0 2024-03-06T07:28:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.483+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe218151c60 con 0x7fe2180c88b0 2024-03-06T07:28:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.483+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21804f330 con 0x7fe2180c88b0 2024-03-06T07:28:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.484+0000 7fe20f7fe640 1 -- 172.21.15.26:0/1158213781 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe200007400 con 0x7fe2180c88b0 2024-03-06T07:28:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.484+0000 7fe20f7fe640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe1f806b4d0 0x7fe1f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.484+0000 7fe20f7fe640 1 -- 172.21.15.26:0/1158213781 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe200089590 con 0x7fe2180c88b0 2024-03-06T07:28:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.485+0000 7fe216d9d640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe1f806b4d0 0x7fe1f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.485+0000 7fe216d9d640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe1f806b4d0 0x7fe1f806d990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe210036000 tx=0x7fe21003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.489+0000 7fe20f7fe640 1 -- 172.21.15.26:0/1158213781 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe218155110 con 0x7fe2180c88b0 2024-03-06T07:28:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.737+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]} v 0) v1 -- 0x7fe21806cdc0 con 0x7fe2180c88b0 2024-03-06T07:28:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.743+0000 7fe20f7fe640 1 -- 172.21.15.26:0/1158213781 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]}]=0 v468) v1 ==== 143+0+0 (secure 0 0 0) 0x7fe2000585c0 con 0x7fe2180c88b0 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe1f806b4d0 msgr2=0x7fe1f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe1f806b4d0 0x7fe1f806d990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe210036000 tx=0x7fe21003c040 comp rx=0 tx=0).stop 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c88b0 msgr2=0x7fe218151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c88b0 0x7fe218151720 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe20000c440 tx=0x7fe200001e70 comp rx=0 tx=0).stop 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 shutdown_connections 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe1f806b4d0 0x7fe1f806d990 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2181eb5a0 0x7fe2181ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2180c88b0 0x7fe218151720 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 --2- 172.21.15.26:0/1158213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2180c5330 0x7fe2181573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 >> 172.21.15.26:0/1158213781 conn(0x7fe2180ac560 msgr2=0x7fe218153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 shutdown_connections 2024-03-06T07:28:09.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.746+0000 7fe21d05c640 1 -- 172.21.15.26:0/1158213781 wait complete. 2024-03-06T07:28:09.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:28:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.862+0000 7fe14500e640 1 Processor -- start 2024-03-06T07:28:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.862+0000 7fe14500e640 1 -- start start 2024-03-06T07:28:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.862+0000 7fe14500e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 0x7fe140155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe14500e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe140152cb0 con 0x7fe1401528d0 2024-03-06T07:28:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe14500e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1400c38b0 con 0x7fe1400c67f0 2024-03-06T07:28:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe14500e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1401594f0 con 0x7fe1400c4ef0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe1401528d0 0x7fe1400724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39350/0 (socket says 172.21.15.26:39350) 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13ed9d640 1 -- 172.21.15.26:0/2363196944 learned_addr learned my addr 172.21.15.26:0/2363196944 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13f59e640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 0x7fe140155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251524100 0 0) 0x7fe140152cb0 con 0x7fe1401528d0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe128003520 con 0x7fe1401528d0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087253054 0 0) 0x7fe1400c38b0 con 0x7fe1400c67f0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe140152cb0 con 0x7fe1400c67f0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3829124993 0 0) 0x7fe128003520 con 0x7fe1401528d0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 msgr2=0x7fe140155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 0x7fe140155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 >> v1:172.21.15.117:6789/0 conn(0x7fe1400c67f0 legacy=0x7fe140155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1400a7f70 con 0x7fe1401528d0 2024-03-06T07:28:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe130001670 con 0x7fe1401528d0 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 >> v1:172.21.15.26:6789/0 conn(0x7fe1401528d0 legacy=0x7fe1400724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe128003b70 0x7fe128003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 0x7fe1280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1401594f0 con 0x7fe128004800 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12800ba70 con 0x7fe128003b70 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12800bc70 con 0x7fe1400c4ef0 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.863+0000 7fe13e59c640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13ed9d640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe128003b70 0x7fe128003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13f59e640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 0x7fe1280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13e59c640 1 -- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 msgr2=0x7fe1280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13e59c640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1400c4ef0 0x7fe1280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13e59c640 1 -- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe128003b70 msgr2=0x7fe128003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13e59c640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe128003b70 0x7fe128003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13e59c640 1 -- 172.21.15.26:0/2363196944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128003520 con 0x7fe128004800 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.864+0000 7fe13e59c640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe128004c80 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe1340018d0 tx=0x7fe128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6d5ac0770d1cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134037070 con 0x7fe128004800 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1340052d0 con 0x7fe128004800 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe13dd9b640 1 -- 172.21.15.26:0/2363196944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1340014f0 con 0x7fe128004800 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe14500e640 1 -- 172.21.15.26:0/2363196944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 msgr2=0x7fe128004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe14500e640 1 --2- 172.21.15.26:0/2363196944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe128004c80 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe1340018d0 tx=0x7fe128005830 comp rx=0 tx=0).stop 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe13f59e640 1 -- 172.21.15.26:0/2363196944 reap_dead start 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe14500e640 1 -- 172.21.15.26:0/2363196944 shutdown_connections 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe14500e640 1 -- 172.21.15.26:0/2363196944 >> 172.21.15.26:0/2363196944 conn(0x7fe1400aa6a0 msgr2=0x7fe1400aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe14500e640 1 -- 172.21.15.26:0/2363196944 shutdown_connections 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.865+0000 7fe14500e640 1 -- 172.21.15.26:0/2363196944 wait complete. 2024-03-06T07:28:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.866+0000 7fe14500e640 1 Processor -- start 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe14500e640 1 -- start start 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe14500e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe140153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe14500e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe140153fe0 0x7fe14014a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe14500e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14014ac50 0x7fe14014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe14500e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1401591a0 con 0x7fe128004800 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe14500e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1400a8250 con 0x7fe14014ac50 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe14500e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe140072be0 con 0x7fe140153fe0 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe13ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe140153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe13ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe140153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48162/0 (socket says 172.21.15.26:48162) 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.867+0000 7fe13ed9d640 1 -- 172.21.15.26:0/2130734658 learned_addr learned my addr 172.21.15.26:0/2130734658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13e59c640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe140153fe0 0x7fe14014a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13f59e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14014ac50 0x7fe14014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13ed9d640 1 -- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe140153fe0 msgr2=0x7fe14014a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13ed9d640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe140153fe0 0x7fe14014a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13ed9d640 1 -- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14014ac50 msgr2=0x7fe14014e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13ed9d640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14014ac50 0x7fe14014e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13ed9d640 1 -- 172.21.15.26:0/2130734658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14014b540 con 0x7fe128004800 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13f59e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14014ac50 0x7fe14014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe13ed9d640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe140153aa0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe130004ef0 tx=0x7fe130007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe1277fe640 1 -- 172.21.15.26:0/2130734658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000f7b0 con 0x7fe128004800 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe1277fe640 1 -- 172.21.15.26:0/2130734658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130018070 con 0x7fe128004800 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe140155040 con 0x7fe128004800 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe1277fe640 1 -- 172.21.15.26:0/2130734658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000fbd0 con 0x7fe128004800 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.868+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14014df40 con 0x7fe128004800 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.869+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe140154e30 con 0x7fe128004800 2024-03-06T07:28:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.870+0000 7fe1277fe640 1 -- 172.21.15.26:0/2130734658 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe13001d460 con 0x7fe128004800 2024-03-06T07:28:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.870+0000 7fe1277fe640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe12c06b500 0x7fe12c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.870+0000 7fe1277fe640 1 -- 172.21.15.26:0/2130734658 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe13008a070 con 0x7fe128004800 2024-03-06T07:28:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.870+0000 7fe13e59c640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe12c06b500 0x7fe12c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.871+0000 7fe13e59c640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe12c06b500 0x7fe12c06d9c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe134034000 tx=0x7fe134031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:09.874+0000 7fe1277fe640 1 -- 172.21.15.26:0/2130734658 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1300590a0 con 0x7fe128004800 2024-03-06T07:28:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.058+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe14004f330 con 0x7fe128004800 2024-03-06T07:28:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.061+0000 7fe1277fe640 1 -- 172.21.15.26:0/2130734658 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v61)=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe1300589a0 con 0x7fe128004800 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe12c06b500 msgr2=0x7fe12c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe12c06b500 0x7fe12c06d9c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe134034000 tx=0x7fe134031000 comp rx=0 tx=0).stop 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 msgr2=0x7fe140153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe140153aa0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe130004ef0 tx=0x7fe130007430 comp rx=0 tx=0).stop 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 shutdown_connections 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe12c06b500 0x7fe12c06d9c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14014ac50 0x7fe14014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe140153fe0 0x7fe14014a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 --2- 172.21.15.26:0/2130734658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe128004800 0x7fe140153aa0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 >> 172.21.15.26:0/2130734658 conn(0x7fe1400aa6a0 msgr2=0x7fe1400a8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 shutdown_connections 2024-03-06T07:28:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.063+0000 7fe14500e640 1 -- 172.21.15.26:0/2130734658 wait complete. 2024-03-06T07:28:10.076 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:28:10.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:10.121 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:10.166 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:28:10.166 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:28:10.166 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:10.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:28:10.181 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:28:10.182 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:10.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:28:10.199 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:28:10.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.321+0000 7f2e509e4640 1 Processor -- start 2024-03-06T07:28:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.321+0000 7f2e509e4640 1 -- start start 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e509e4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e4c0c6780 0x7f2e4c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e509e4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e509e4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c072630 con 0x7f2e4c06da40 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e509e4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c06de20 con 0x7f2e4c0c6780 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e509e4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c072910 con 0x7f2e4c1592d0 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e49d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e4c0c6780 0x7f2e4c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e49d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e4c0c6780 0x7f2e4c158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48168/0 (socket says 172.21.15.26:48168) 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e49d9b640 1 -- 172.21.15.26:0/3479254518 learned_addr learned my addr 172.21.15.26:0/3479254518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e4ad9d640 1 --2- 172.21.15.26:0/3479254518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e49d9b640 1 -- 172.21.15.26:0/3479254518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 msgr2=0x7f2e4c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e49d9b640 1 --2- 172.21.15.26:0/3479254518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e49d9b640 1 -- 172.21.15.26:0/3479254518 >> v1:172.21.15.117:6789/0 conn(0x7f2e4c06da40 legacy=0x7f2e4c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e49d9b640 1 -- 172.21.15.26:0/3479254518 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c072bf0 con 0x7f2e4c0c6780 2024-03-06T07:28:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.322+0000 7f2e4ad9d640 1 --2- 172.21.15.26:0/3479254518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.323+0000 7f2e49d9b640 1 --2- 172.21.15.26:0/3479254518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c158d90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700d26b99e9fd94d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.323+0000 7f2e4959a640 1 -- 172.21.15.26:0/3479254518 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e340317c0 con 0x7f2e4c0c6780 2024-03-06T07:28:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.323+0000 7f2e4959a640 1 -- 172.21.15.26:0/3479254518 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34037070 con 0x7f2e4c0c6780 2024-03-06T07:28:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.323+0000 7f2e4959a640 1 -- 172.21.15.26:0/3479254518 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34031b80 con 0x7f2e4c0c6780 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 -- 172.21.15.26:0/3479254518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 msgr2=0x7f2e4c158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 --2- 172.21.15.26:0/3479254518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c158d90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402b860 comp rx=0 tx=0).stop 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 -- 172.21.15.26:0/3479254518 shutdown_connections 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 --2- 172.21.15.26:0/3479254518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 --2- 172.21.15.26:0/3479254518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c158d90 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 -- 172.21.15.26:0/3479254518 >> 172.21.15.26:0/3479254518 conn(0x7f2e4c0aa630 msgr2=0x7f2e4c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 -- 172.21.15.26:0/3479254518 shutdown_connections 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.324+0000 7f2e509e4640 1 -- 172.21.15.26:0/3479254518 wait complete. 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.325+0000 7f2e509e4640 1 Processor -- start 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.325+0000 7f2e509e4640 1 -- start start 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e509e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4c06da40 0x7f2e4c0a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e509e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e509e4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e509e4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c3940 con 0x7f2e4c0c6780 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e509e4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c37c0 con 0x7f2e4c06da40 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e509e4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c3ac0 con 0x7f2e4c1592d0 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c148cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48180/0 (socket says 172.21.15.26:48180) 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 -- 172.21.15.26:0/897084846 learned_addr learned my addr 172.21.15.26:0/897084846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e4a59c640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4c06da40 0x7f2e4c0a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e4ad9d640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 -- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 msgr2=0x7f2e4c149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 -- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4c06da40 msgr2=0x7f2e4c0a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4c06da40 0x7f2e4c0a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 -- 172.21.15.26:0/897084846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c1ee010 con 0x7f2e4c0c6780 2024-03-06T07:28:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.326+0000 7f2e49d9b640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c148cf0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000c00 tx=0x7f2e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.327+0000 7f2e42ffd640 1 -- 172.21.15.26:0/897084846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34032860 con 0x7f2e4c0c6780 2024-03-06T07:28:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.327+0000 7f2e42ffd640 1 -- 172.21.15.26:0/897084846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34032a20 con 0x7f2e4c0c6780 2024-03-06T07:28:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.327+0000 7f2e42ffd640 1 -- 172.21.15.26:0/897084846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34032e10 con 0x7f2e4c0c6780 2024-03-06T07:28:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.327+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c06f220 con 0x7f2e4c0c6780 2024-03-06T07:28:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.327+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c06fe10 con 0x7f2e4c0c6780 2024-03-06T07:28:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.328+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c04f330 con 0x7f2e4c0c6780 2024-03-06T07:28:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.328+0000 7f2e42ffd640 1 -- 172.21.15.26:0/897084846 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e3404c060 con 0x7f2e4c0c6780 2024-03-06T07:28:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.328+0000 7f2e42ffd640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.329+0000 7f2e42ffd640 1 -- 172.21.15.26:0/897084846 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2e340abda0 con 0x7f2e4c0c6780 2024-03-06T07:28:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.329+0000 7f2e4a59c640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.329+0000 7f2e4a59c640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2e44006fd0 tx=0x7f2e44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.333+0000 7f2e42ffd640 1 -- 172.21.15.26:0/897084846 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2e34040070 con 0x7f2e4c0c6780 2024-03-06T07:28:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.555+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e4c073240 con 0x7f2e4c0c6780 2024-03-06T07:28:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.555+0000 7f2e42ffd640 1 -- 172.21.15.26:0/897084846 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f2e3407add0 con 0x7f2e4c0c6780 2024-03-06T07:28:10.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:10.557 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":183,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/918256521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":918256521},{"type":"v1","addr":"172.21.15.117:6836","nonce":918256521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":176},{"gid":9115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1518138607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1518138607},{"type":"v1","addr":"172.21.15.26:6835","nonce":1518138607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9463,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3930767664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3930767664},{"type":"v1","addr":"172.21.15.117:6837","nonce":3930767664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":183},{"gid":10263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1369024896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1369024896},{"type":"v1","addr":"172.21.15.26:6834","nonce":1369024896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":183}],"filesystems":[]} 2024-03-06T07:28:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 183 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e2c06b4d0 msgr2=0x7f2e2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2e44006fd0 tx=0x7f2e44008040 comp rx=0 tx=0).stop 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 msgr2=0x7f2e4c148cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c148cf0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000c00 tx=0x7f2e340046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 shutdown_connections 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4c1592d0 0x7f2e4c149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e4c0c6780 0x7f2e4c148cf0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 --2- 172.21.15.26:0/897084846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4c06da40 0x7f2e4c0a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 >> 172.21.15.26:0/897084846 conn(0x7f2e4c0aa630 msgr2=0x7f2e4c154950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 shutdown_connections 2024-03-06T07:28:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.557+0000 7f2e509e4640 1 -- 172.21.15.26:0/897084846 wait complete. 2024-03-06T07:28:10.570 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:28:10.570 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:28:10.570 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:28:10.570 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.570 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:28:10.570 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:28:10.571 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:28:10.571 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.571 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:28:10.571 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:28:10.571 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:28:10.571 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.571 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:28:10.571 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:28:10.572 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:28:10.572 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.579 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:28:10.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:28:10.579 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:28:10.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:28:10.583 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:28:10.583 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:28:10.584 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:28:10.584 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:28:10.622 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:28:10.622 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.623 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:28:10.623 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.625 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:28:10.625 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.626 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:28:10.626 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:28:10.699 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:28:10.699 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:28:10.707 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:28:10.707 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:28:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.731+0000 7f1ac9cfa640 1 Processor -- start 2024-03-06T07:28:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.731+0000 7f1ac9cfa640 1 -- start start 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac9cfa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 0x7f1ac40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac9cfa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ac41558d0 0x7f1ac4155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac9cfa640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac4155130 con 0x7f1ac40c69f0 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac9cfa640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac4155440 con 0x7f1ac41558d0 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac9cfa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40ad480 con 0x7f1ac40c8950 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ac40c69f0 0x7f1ac40acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39382/0 (socket says 172.21.15.26:39382) 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac37fe640 1 -- 172.21.15.26:0/1966010371 learned_addr learned my addr 172.21.15.26:0/1966010371 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac3fff640 1 --2- 172.21.15.26:0/1966010371 >> v2:172.21.15.117:3300/0 conn(0x7f1ac41558d0 0x7f1ac4155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac2ffd640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 0x7f1ac40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.732+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486137460 0 0) 0x7f1ab4001110 con 0x7f1ac40c69f0 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aac003520 con 0x7f1ac40c69f0 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472960889 0 0) 0x7f1aac003520 con 0x7f1ac40c69f0 2024-03-06T07:28:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 msgr2=0x7f1ac40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 0x7f1ac40c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 >> v2:172.21.15.117:3300/0 conn(0x7f1ac41558d0 msgr2=0x7f1ac4155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 --2- 172.21.15.26:0/1966010371 >> v2:172.21.15.117:3300/0 conn(0x7f1ac41558d0 0x7f1ac4155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac40c5d50 con 0x7f1ac40c69f0 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f1ab4001840 con 0x7f1ac40c69f0 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac2ffd640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 0x7f1ac40c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 >> v1:172.21.15.26:6789/0 conn(0x7f1ac40c69f0 legacy=0x7f1ac40acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aac0039b0 0x7f1aac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 0x7f1aac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac004d40 0x7f1aac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac4155440 con 0x7f1aac0039b0 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40ad480 con 0x7f1aac004d40 2024-03-06T07:28:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac00ba10 con 0x7f1ac40c8950 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aac0039b0 0x7f1aac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac2ffd640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 0x7f1aac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac3fff640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac004d40 0x7f1aac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac37fe640 1 -- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 msgr2=0x7f1aac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac40c8950 0x7f1aac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac37fe640 1 -- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac004d40 msgr2=0x7f1aac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac004d40 0x7f1aac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac37fe640 1 -- 172.21.15.26:0/1966010371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac003520 con 0x7f1aac0039b0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.733+0000 7f1ac3fff640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac004d40 0x7f1aac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.734+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aac0039b0 0x7f1aac003dd0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c3e0 tx=0x7f1aac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876927d673ca9df2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.734+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400c0e0 con 0x7f1aac0039b0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.734+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab4030070 con 0x7f1aac0039b0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.734+0000 7f1ac27fc640 1 -- 172.21.15.26:0/1966010371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40058d0 con 0x7f1aac0039b0 2024-03-06T07:28:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.735+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/1966010371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aac0039b0 msgr2=0x7f1aac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.735+0000 7f1ac9cfa640 1 --2- 172.21.15.26:0/1966010371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aac0039b0 0x7f1aac003dd0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c3e0 tx=0x7f1aac005830 comp rx=0 tx=0).stop 2024-03-06T07:28:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.735+0000 7f1ac3fff640 1 -- 172.21.15.26:0/1966010371 reap_dead start 2024-03-06T07:28:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.735+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/1966010371 shutdown_connections 2024-03-06T07:28:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.735+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/1966010371 >> 172.21.15.26:0/1966010371 conn(0x7f1ac40cae70 msgr2=0x7f1ac40cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.735+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/1966010371 shutdown_connections 2024-03-06T07:28:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.735+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/1966010371 wait complete. 2024-03-06T07:28:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.736+0000 7f1ac9cfa640 1 Processor -- start 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.736+0000 7f1ac9cfa640 1 -- start start 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac0039b0 0x7f1ac414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac41dfac0 0x7f1ac41e0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac414cc70 0x7f1ac41e0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac4152a20 con 0x7f1ac41dfac0 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40c6310 con 0x7f1aac0039b0 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac4152ba0 con 0x7f1ac414cc70 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac41dfac0 0x7f1ac41e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac41dfac0 0x7f1ac41e0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48212/0 (socket says 172.21.15.26:48212) 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 -- 172.21.15.26:0/659322946 learned_addr learned my addr 172.21.15.26:0/659322946 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac0039b0 0x7f1ac414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac3fff640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac414cc70 0x7f1ac41e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 -- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac414cc70 msgr2=0x7f1ac41e0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac414cc70 0x7f1ac41e0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 -- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac0039b0 msgr2=0x7f1ac414c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac0039b0 0x7f1ac414c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 -- 172.21.15.26:0/659322946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac40c1b30 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac3fff640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac414cc70 0x7f1ac41e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac0039b0 0x7f1ac414c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac2ffd640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac41dfac0 0x7f1ac41e0130 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80115f0 tx=0x7f1ab8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1aabfff640 1 -- 172.21.15.26:0/659322946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8019070 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1aabfff640 1 -- 172.21.15.26:0/659322946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab8004220 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac40c1e10 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1aabfff640 1 -- 172.21.15.26:0/659322946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab800f040 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.737+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac41de790 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.738+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac40c2050 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.739+0000 7f1aabfff640 1 -- 172.21.15.26:0/659322946 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ab800f6f0 con 0x7f1ac41dfac0 2024-03-06T07:28:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.739+0000 7f1aabfff640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ab006b500 0x7f1ab006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.739+0000 7f1aabfff640 1 -- 172.21.15.26:0/659322946 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1ab808a280 con 0x7f1ac41dfac0 2024-03-06T07:28:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.739+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ab006b500 0x7f1ab006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.740+0000 7f1ac37fe640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ab006b500 0x7f1ab006d9c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c3b0 tx=0x7f1ab4007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.743+0000 7f1aabfff640 1 -- 172.21.15.26:0/659322946 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1ab8059220 con 0x7f1ac41dfac0 2024-03-06T07:28:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:10.938+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1ac41eee00 con 0x7f1ac41dfac0 2024-03-06T07:28:11.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.758+0000 7f1aabfff640 1 -- 172.21.15.26:0/659322946 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v194) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1ab8058b20 con 0x7f1ac41dfac0 2024-03-06T07:28:11.760 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:28:11.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ab006b500 msgr2=0x7f1ab006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ab006b500 0x7f1ab006d9c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c3b0 tx=0x7f1ab4007000 comp rx=0 tx=0).stop 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac41dfac0 msgr2=0x7f1ac41e0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac41dfac0 0x7f1ac41e0130 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80115f0 tx=0x7f1ab8011ab0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 shutdown_connections 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1ab006b500 0x7f1ab006d9c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac414cc70 0x7f1ac41e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac41dfac0 0x7f1ac41e0130 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 --2- 172.21.15.26:0/659322946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1aac0039b0 0x7f1ac414c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 >> 172.21.15.26:0/659322946 conn(0x7f1ac40cae70 msgr2=0x7f1ac4151910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 shutdown_connections 2024-03-06T07:28:11.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.761+0000 7f1ac9cfa640 1 -- 172.21.15.26:0/659322946 wait complete. 2024-03-06T07:28:11.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:28:11.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.881+0000 7f0d2a53a640 1 Processor -- start 2024-03-06T07:28:11.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.881+0000 7f0d2a53a640 1 -- start start 2024-03-06T07:28:11.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.882+0000 7f0d2a53a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 0x7f0d240c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.882+0000 7f0d2a53a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d241591c0 con 0x7f0d241586f0 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.882+0000 7f0d2a53a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d241594a0 con 0x7f0d240c5480 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.882+0000 7f0d2a53a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2406df40 con 0x7f0d24155ba0 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.882+0000 7f0d28ad7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0d241586f0 0x7f0d2415f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39384/0 (socket says 172.21.15.26:39384) 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.882+0000 7f0d28ad7640 1 -- 172.21.15.26:0/1060584741 learned_addr learned my addr 172.21.15.26:0/1060584741 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.882+0000 7f0d237fe640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 0x7f0d240c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431621610 0 0) 0x7f0d241591c0 con 0x7f0d241586f0 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d10003520 con 0x7f0d241586f0 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201394022 0 0) 0x7f0d241594a0 con 0x7f0d240c5480 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d241591c0 con 0x7f0d240c5480 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3919435341 0 0) 0x7f0d10003520 con 0x7f0d241586f0 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 msgr2=0x7f0d240c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 0x7f0d240c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 >> v1:172.21.15.117:6789/0 conn(0x7f0d240c5480 legacy=0x7f0d2406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d241553a0 con 0x7f0d241586f0 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f0d1c0019f0 con 0x7f0d241586f0 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 >> v1:172.21.15.26:6789/0 conn(0x7f0d241586f0 legacy=0x7f0d2415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 0x7f0d10003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d10004120 0x7f0d10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d10004d40 0x7f0d100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2406df40 con 0x7f0d10004d40 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1000bb80 con 0x7f0d10004120 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1000bd80 con 0x7f0d24155ba0 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d28ad7640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d10004d40 0x7f0d100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d23fff640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 0x7f0d10003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d237fe640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d10004120 0x7f0d10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d28ad7640 1 -- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 msgr2=0x7f0d10003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d28ad7640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d24155ba0 0x7f0d10003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d28ad7640 1 -- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d10004120 msgr2=0x7f0d10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d28ad7640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d10004120 0x7f0d10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.883+0000 7f0d28ad7640 1 -- 172.21.15.26:0/1060584741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10003520 con 0x7f0d10004d40 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.884+0000 7f0d28ad7640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d10004d40 0x7f0d100051c0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00a720 tx=0x7f0d10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705d6ff04b4c2886 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.884+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00a370 con 0x7f0d10004d40 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.884+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c034650 con 0x7f0d10004d40 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.884+0000 7f0d22ffd640 1 -- 172.21.15.26:0/1060584741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c034a80 con 0x7f0d10004d40 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.885+0000 7f0d2a53a640 1 -- 172.21.15.26:0/1060584741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d10004d40 msgr2=0x7f0d100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.885+0000 7f0d2a53a640 1 --2- 172.21.15.26:0/1060584741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d10004d40 0x7f0d100051c0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00a720 tx=0x7f0d10005830 comp rx=0 tx=0).stop 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.885+0000 7f0d28ad7640 1 -- 172.21.15.26:0/1060584741 reap_dead start 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.885+0000 7f0d2a53a640 1 -- 172.21.15.26:0/1060584741 shutdown_connections 2024-03-06T07:28:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.885+0000 7f0d2a53a640 1 -- 172.21.15.26:0/1060584741 >> 172.21.15.26:0/1060584741 conn(0x7f0d2406b1a0 msgr2=0x7f0d2406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.885+0000 7f0d2a53a640 1 -- 172.21.15.26:0/1060584741 shutdown_connections 2024-03-06T07:28:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.885+0000 7f0d2a53a640 1 -- 172.21.15.26:0/1060584741 wait complete. 2024-03-06T07:28:11.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.886+0000 7f0d2a53a640 1 Processor -- start 2024-03-06T07:28:11.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.886+0000 7f0d2a53a640 1 -- start start 2024-03-06T07:28:11.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d2a53a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d10004d40 0x7f0d241eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d2a53a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d240ab560 0x7f0d2414db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d2a53a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d240c49e0 0x7f0d241501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d2a53a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24156000 con 0x7f0d240ab560 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d2a53a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2406fdb0 con 0x7f0d240c49e0 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d2a53a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24152a20 con 0x7f0d10004d40 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d240ab560 0x7f0d2414db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d240ab560 0x7f0d2414db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48234/0 (socket says 172.21.15.26:48234) 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 -- 172.21.15.26:0/3064953575 learned_addr learned my addr 172.21.15.26:0/3064953575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d23fff640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d10004d40 0x7f0d241eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d28ad7640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d240c49e0 0x7f0d241501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 -- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d10004d40 msgr2=0x7f0d241eacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d10004d40 0x7f0d241eacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 -- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d240c49e0 msgr2=0x7f0d241501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d240c49e0 0x7f0d241501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 -- 172.21.15.26:0/3064953575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d240aba40 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d23fff640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d10004d40 0x7f0d241eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d28ad7640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d240c49e0 0x7f0d241501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.887+0000 7f0d237fe640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d240ab560 0x7f0d2414db60 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0119d0 tx=0x7f0d0c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.888+0000 7f0d20ff9640 1 -- 172.21.15.26:0/3064953575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00fb50 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.888+0000 7f0d20ff9640 1 -- 172.21.15.26:0/3064953575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00fcf0 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.888+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d240ad040 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.888+0000 7f0d20ff9640 1 -- 172.21.15.26:0/3064953575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00fea0 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.888+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d241ef0b0 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.889+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2404f330 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.889+0000 7f0d20ff9640 1 -- 172.21.15.26:0/3064953575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d0c012040 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.889+0000 7f0d20ff9640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0d0406b4d0 0x7f0d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.890+0000 7f0d20ff9640 1 -- 172.21.15.26:0/3064953575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0d0c08a6f0 con 0x7f0d240ab560 2024-03-06T07:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.890+0000 7f0d23fff640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0d0406b4d0 0x7f0d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.890+0000 7f0d23fff640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0d0406b4d0 0x7f0d0406d990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:11.894+0000 7f0d20ff9640 1 -- 172.21.15.26:0/3064953575 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0d240aba40 con 0x7f0d240ab560 2024-03-06T07:28:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.147+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0d2406e0c0 con 0x7f0d240ab560 2024-03-06T07:28:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.147+0000 7f0d20ff9640 1 -- 172.21.15.26:0/3064953575 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f0d0c019030 con 0x7f0d240ab560 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:28:12.149 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:28:12.151 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.152 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.153 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:28:12.155 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.157 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.159 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:12.160 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:28:12.161 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:28:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0d0406b4d0 msgr2=0x7f0d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0d0406b4d0 0x7f0d0406d990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14009040 comp rx=0 tx=0).stop 2024-03-06T07:28:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d240ab560 msgr2=0x7f0d2414db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d240ab560 0x7f0d2414db60 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0119d0 tx=0x7f0d0c011e90 comp rx=0 tx=0).stop 2024-03-06T07:28:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 shutdown_connections 2024-03-06T07:28:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0d0406b4d0 0x7f0d0406d990 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d240c49e0 0x7f0d241501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d240ab560 0x7f0d2414db60 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 --2- 172.21.15.26:0/3064953575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d10004d40 0x7f0d241eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 >> 172.21.15.26:0/3064953575 conn(0x7f0d2406b1a0 msgr2=0x7f0d240c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 shutdown_connections 2024-03-06T07:28:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.149+0000 7f0d2a53a640 1 -- 172.21.15.26:0/3064953575 wait complete. 2024-03-06T07:28:12.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:12.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.264+0000 7fbe4af3e640 1 Processor -- start 2024-03-06T07:28:12.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.264+0000 7fbe4af3e640 1 -- start start 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.264+0000 7fbe4af3e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe4406ef30 0x7fbe4406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.264+0000 7fbe4af3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe440c1ae0 con 0x7fbe44155c80 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.264+0000 7fbe4af3e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4406e450 con 0x7fbe44152520 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.264+0000 7fbe4af3e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe440c1dc0 con 0x7fbe4406ef30 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe43fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbe44155c80 0x7fbe440720a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39390/0 (socket says 172.21.15.26:39390) 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe43fff640 1 -- 172.21.15.26:0/333902757 learned_addr learned my addr 172.21.15.26:0/333902757 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe494db640 1 --2- 172.21.15.26:0/333902757 >> v2:172.21.15.117:3300/0 conn(0x7fbe4406ef30 0x7fbe4406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe437fe640 1 -- 172.21.15.26:0/333902757 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063132715 0 0) 0x7fbe440c1ae0 con 0x7fbe44155c80 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe437fe640 1 -- 172.21.15.26:0/333902757 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe34003520 con 0x7fbe44155c80 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe437fe640 1 -- 172.21.15.26:0/333902757 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987361240 0 0) 0x7fbe34003520 con 0x7fbe44155c80 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe494db640 1 -- 172.21.15.26:0/333902757 >> v1:172.21.15.117:6789/0 conn(0x7fbe44152520 legacy=0x7fbe440c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe494db640 1 -- 172.21.15.26:0/333902757 >> v1:172.21.15.26:6789/0 conn(0x7fbe44155c80 legacy=0x7fbe440720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.265+0000 7fbe494db640 1 -- 172.21.15.26:0/333902757 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4406e730 con 0x7fbe4406ef30 2024-03-06T07:28:12.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.266+0000 7fbe494db640 1 --2- 172.21.15.26:0/333902757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 0x7fbe4406afc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009d90 tx=0x7fbe3c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6175da517af48aa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.266+0000 7fbe437fe640 1 -- 172.21.15.26:0/333902757 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c039070 con 0x7fbe4406ef30 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.266+0000 7fbe437fe640 1 -- 172.21.15.26:0/333902757 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c02c940 con 0x7fbe4406ef30 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.266+0000 7fbe437fe640 1 -- 172.21.15.26:0/333902757 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30001670 con 0x7fbe4406ef30 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.267+0000 7fbe4af3e640 1 -- 172.21.15.26:0/333902757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 msgr2=0x7fbe4406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.267+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/333902757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 0x7fbe4406afc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009d90 tx=0x7fbe3c02be10 comp rx=0 tx=0).stop 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.267+0000 7fbe4af3e640 1 -- 172.21.15.26:0/333902757 shutdown_connections 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.267+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/333902757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 0x7fbe4406afc0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.267+0000 7fbe4af3e640 1 -- 172.21.15.26:0/333902757 >> 172.21.15.26:0/333902757 conn(0x7fbe440c76b0 msgr2=0x7fbe440c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.267+0000 7fbe4af3e640 1 -- 172.21.15.26:0/333902757 shutdown_connections 2024-03-06T07:28:12.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.267+0000 7fbe4af3e640 1 -- 172.21.15.26:0/333902757 wait complete. 2024-03-06T07:28:12.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.268+0000 7fbe4af3e640 1 Processor -- start 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.268+0000 7fbe4af3e640 1 -- start start 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe4af3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 0x7fbe440c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe4af3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe44152520 0x7fbe44157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe4af3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe44155c80 0x7fbe44157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe4af3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44152900 con 0x7fbe44152520 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe4af3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440727b0 con 0x7fbe4406ef30 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe4af3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44152a80 con 0x7fbe44155c80 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe44152520 0x7fbe44157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe44152520 0x7fbe44157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48236/0 (socket says 172.21.15.26:48236) 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 -- 172.21.15.26:0/3283909594 learned_addr learned my addr 172.21.15.26:0/3283909594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe494db640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe44155c80 0x7fbe44157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe48cda640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 0x7fbe440c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 -- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe44155c80 msgr2=0x7fbe44157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe44155c80 0x7fbe44157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 -- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 msgr2=0x7fbe440c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 0x7fbe440c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 -- 172.21.15.26:0/3283909594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440c81f0 con 0x7fbe44152520 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.269+0000 7fbe43fff640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe44152520 0x7fbe44157420 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c760 tx=0x7fbe3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.270+0000 7fbe417fa640 1 -- 172.21.15.26:0/3283909594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30017070 con 0x7fbe44152520 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.270+0000 7fbe417fa640 1 -- 172.21.15.26:0/3283909594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30005af0 con 0x7fbe44152520 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.270+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe440ac790 con 0x7fbe44152520 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.270+0000 7fbe417fa640 1 -- 172.21.15.26:0/3283909594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30007070 con 0x7fbe44152520 2024-03-06T07:28:12.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.270+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe440acd50 con 0x7fbe44152520 2024-03-06T07:28:12.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.271+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4404f330 con 0x7fbe44152520 2024-03-06T07:28:12.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.271+0000 7fbe417fa640 1 -- 172.21.15.26:0/3283909594 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe3001a070 con 0x7fbe44152520 2024-03-06T07:28:12.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.271+0000 7fbe417fa640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe2806b500 0x7fbe2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.271+0000 7fbe417fa640 1 -- 172.21.15.26:0/3283909594 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe30089480 con 0x7fbe44152520 2024-03-06T07:28:12.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.272+0000 7fbe48cda640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe2806b500 0x7fbe2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.272+0000 7fbe48cda640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe2806b500 0x7fbe2806d9c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbe440ad220 tx=0x7fbe38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.276+0000 7fbe417fa640 1 -- 172.21.15.26:0/3283909594 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe30057f00 con 0x7fbe44152520 2024-03-06T07:28:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.459+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe440c0b70 con 0x7fbe44152520 2024-03-06T07:28:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.460+0000 7fbe417fa640 1 -- 172.21.15.26:0/3283909594 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+13186 (secure 0 0 0) 0x7fbe3005bfa0 con 0x7fbe44152520 2024-03-06T07:28:12.461 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:12.462 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":194,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:11.757357+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe2806b500 msgr2=0x7fbe2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe2806b500 0x7fbe2806d9c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbe440ad220 tx=0x7fbe38009040 comp rx=0 tx=0).stop 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe44152520 msgr2=0x7fbe44157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe44152520 0x7fbe44157420 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c760 tx=0x7fbe3000cc20 comp rx=0 tx=0).stop 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 shutdown_connections 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe2806b500 0x7fbe2806d9c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe44155c80 0x7fbe44157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe44152520 0x7fbe44157420 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 --2- 172.21.15.26:0/3283909594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe4406ef30 0x7fbe440c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 >> 172.21.15.26:0/3283909594 conn(0x7fbe440c76b0 msgr2=0x7fbe440c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 shutdown_connections 2024-03-06T07:28:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.462+0000 7fbe4af3e640 1 -- 172.21.15.26:0/3283909594 wait complete. 2024-03-06T07:28:12.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:28:12.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:28:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.577+0000 7fe141683640 1 Processor -- start 2024-03-06T07:28:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.577+0000 7fe141683640 1 -- start start 2024-03-06T07:28:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.578+0000 7fe141683640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe13c0c8ff0 0x7fe13c0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.578+0000 7fe141683640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c0b04b0 con 0x7fe13c06bff0 2024-03-06T07:28:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.578+0000 7fe141683640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c06d210 con 0x7fe13c0b00d0 2024-03-06T07:28:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.578+0000 7fe141683640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c156160 con 0x7fe13c0c8ff0 2024-03-06T07:28:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.578+0000 7fe13b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe13c0c8ff0 0x7fe13c0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.578+0000 7fe13a7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe13c06bff0 0x7fe13c0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39396/0 (socket says 172.21.15.26:39396) 2024-03-06T07:28:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.578+0000 7fe13a7fc640 1 -- 172.21.15.26:0/1274350156 learned_addr learned my addr 172.21.15.26:0/1274350156 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe13b7fe640 1 -- 172.21.15.26:0/1274350156 >> v1:172.21.15.117:6789/0 conn(0x7fe13c0b00d0 legacy=0x7fe13c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe13b7fe640 1 -- 172.21.15.26:0/1274350156 >> v1:172.21.15.26:6789/0 conn(0x7fe13c06bff0 legacy=0x7fe13c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe139ffb640 1 -- 172.21.15.26:0/1274350156 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843554214 0 0) 0x7fe13c0b04b0 con 0x7fe13c06bff0 2024-03-06T07:28:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe13b7fe640 1 -- 172.21.15.26:0/1274350156 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c155e80 con 0x7fe13c0c8ff0 2024-03-06T07:28:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe13b7fe640 1 --2- 172.21.15.26:0/1274350156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c0c8ff0 0x7fe13c0c7920 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fe13c1562e0 tx=0x7fe12c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d565ff7ce13c3c84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe139ffb640 1 -- 172.21.15.26:0/1274350156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c039070 con 0x7fe13c0c8ff0 2024-03-06T07:28:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe139ffb640 1 -- 172.21.15.26:0/1274350156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c02cc50 con 0x7fe13c0c8ff0 2024-03-06T07:28:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.579+0000 7fe139ffb640 1 -- 172.21.15.26:0/1274350156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c034440 con 0x7fe13c0c8ff0 2024-03-06T07:28:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.580+0000 7fe141683640 1 -- 172.21.15.26:0/1274350156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c0c8ff0 msgr2=0x7fe13c0c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.580+0000 7fe141683640 1 --2- 172.21.15.26:0/1274350156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c0c8ff0 0x7fe13c0c7920 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fe13c1562e0 tx=0x7fe12c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.580+0000 7fe141683640 1 -- 172.21.15.26:0/1274350156 shutdown_connections 2024-03-06T07:28:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.580+0000 7fe141683640 1 --2- 172.21.15.26:0/1274350156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c0c8ff0 0x7fe13c0c7920 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.580+0000 7fe141683640 1 -- 172.21.15.26:0/1274350156 >> 172.21.15.26:0/1274350156 conn(0x7fe13c0ac810 msgr2=0x7fe13c0acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.580+0000 7fe141683640 1 -- 172.21.15.26:0/1274350156 shutdown_connections 2024-03-06T07:28:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.580+0000 7fe141683640 1 -- 172.21.15.26:0/1274350156 wait complete. 2024-03-06T07:28:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.581+0000 7fe141683640 1 Processor -- start 2024-03-06T07:28:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.581+0000 7fe141683640 1 -- start start 2024-03-06T07:28:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe141683640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c06bff0 0x7fe13c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe141683640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe13c0b00d0 0x7fe13c157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe141683640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe13c0c8ff0 0x7fe13c1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe141683640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c152c50 con 0x7fe13c06bff0 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe141683640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0c5710 con 0x7fe13c0b00d0 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe141683640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0c5ba0 con 0x7fe13c0c8ff0 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c06bff0 0x7fe13c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c06bff0 0x7fe13c0af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48266/0 (socket says 172.21.15.26:48266) 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 -- 172.21.15.26:0/2238434752 learned_addr learned my addr 172.21.15.26:0/2238434752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13a7fc640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe13c0b00d0 0x7fe13c157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13b7fe640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe13c0c8ff0 0x7fe13c1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 -- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe13c0c8ff0 msgr2=0x7fe13c1576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe13c0c8ff0 0x7fe13c1576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 -- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe13c0b00d0 msgr2=0x7fe13c157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe13c0b00d0 0x7fe13c157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 -- 172.21.15.26:0/2238434752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0c3210 con 0x7fe13c06bff0 2024-03-06T07:28:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13b7fe640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe13c0c8ff0 0x7fe13c1576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.582+0000 7fe13affd640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c06bff0 0x7fe13c0af490 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fe13400c460 tx=0x7fe13400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.583+0000 7fe1337fe640 1 -- 172.21.15.26:0/2238434752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134016020 con 0x7fe13c06bff0 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.583+0000 7fe1337fe640 1 -- 172.21.15.26:0/2238434752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe134012870 con 0x7fe13c06bff0 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.583+0000 7fe1337fe640 1 -- 172.21.15.26:0/2238434752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134012c60 con 0x7fe13c06bff0 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.583+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c0c34f0 con 0x7fe13c06bff0 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.583+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c0c3ab0 con 0x7fe13c06bff0 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.584+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c04f330 con 0x7fe13c06bff0 2024-03-06T07:28:12.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.584+0000 7fe1337fe640 1 -- 172.21.15.26:0/2238434752 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe134019020 con 0x7fe13c06bff0 2024-03-06T07:28:12.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.584+0000 7fe1337fe640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe11c06b500 0x7fe11c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.585+0000 7fe1337fe640 1 -- 172.21.15.26:0/2238434752 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe134089fc0 con 0x7fe13c06bff0 2024-03-06T07:28:12.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.585+0000 7fe13a7fc640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe11c06b500 0x7fe11c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.585+0000 7fe13a7fc640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe11c06b500 0x7fe11c06d9c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0c4f40 tx=0x7fe124007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.589+0000 7fe1337fe640 1 -- 172.21.15.26:0/2238434752 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe134020020 con 0x7fe13c06bff0 2024-03-06T07:28:12.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.767+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe13c06ce50 con 0x7fe13c06bff0 2024-03-06T07:28:12.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.767+0000 7fe1337fe640 1 -- 172.21.15.26:0/2238434752 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v194) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1340182c0 con 0x7fe13c06bff0 2024-03-06T07:28:12.769 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:28:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe11c06b500 msgr2=0x7fe11c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe11c06b500 0x7fe11c06d9c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0c4f40 tx=0x7fe124007c00 comp rx=0 tx=0).stop 2024-03-06T07:28:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c06bff0 msgr2=0x7fe13c0af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c06bff0 0x7fe13c0af490 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fe13400c460 tx=0x7fe13400c920 comp rx=0 tx=0).stop 2024-03-06T07:28:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 shutdown_connections 2024-03-06T07:28:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe11c06b500 0x7fe11c06d9c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe13c0c8ff0 0x7fe13c1576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe13c0b00d0 0x7fe13c157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 --2- 172.21.15.26:0/2238434752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe13c06bff0 0x7fe13c0af490 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 >> 172.21.15.26:0/2238434752 conn(0x7fe13c0ac810 msgr2=0x7fe13c153380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 shutdown_connections 2024-03-06T07:28:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.769+0000 7fe141683640 1 -- 172.21.15.26:0/2238434752 wait complete. 2024-03-06T07:28:12.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:28:12.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:12.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.883+0000 7f1845afa640 1 Processor -- start 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.883+0000 7f1845afa640 1 -- start start 2024-03-06T07:28:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.883+0000 7f1845afa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1840152920 0x7f1840155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.883+0000 7f1845afa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18400c8a00 0x7f1840158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.883+0000 7f1845afa640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1840156180 con 0x7f1840159460 2024-03-06T07:28:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.883+0000 7f1845afa640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18401587b0 con 0x7f1840152920 2024-03-06T07:28:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.883+0000 7f1845afa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18400c7760 con 0x7f18400c8a00 2024-03-06T07:28:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1840152920 0x7f1840155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1840159460 0x7f18400c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39400/0 (socket says 172.21.15.26:39400) 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183ffff640 1 -- 172.21.15.26:0/4029931219 learned_addr learned my addr 172.21.15.26:0/4029931219 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183effd640 1 --2- 172.21.15.26:0/4029931219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18400c8a00 0x7f1840158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183f7fe640 1 -- 172.21.15.26:0/4029931219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18400c8a00 msgr2=0x7f1840158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183f7fe640 1 --2- 172.21.15.26:0/4029931219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18400c8a00 0x7f1840158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183f7fe640 1 -- 172.21.15.26:0/4029931219 >> v1:172.21.15.26:6789/0 conn(0x7f1840159460 legacy=0x7f18400c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183e7fc640 1 -- 172.21.15.26:0/4029931219 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210983588 0 0) 0x7f1838001490 con 0x7f1840159460 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.884+0000 7f183f7fe640 1 -- 172.21.15.26:0/4029931219 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18400c7a40 con 0x7f1840152920 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f183f7fe640 1 --2- 172.21.15.26:0/4029931219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1840152920 0x7f1840155c40 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f18300097a0 tx=0x7f183002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c316d9db9619a48d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f183e7fc640 1 -- 172.21.15.26:0/4029931219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18300315a0 con 0x7f1840152920 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f183e7fc640 1 -- 172.21.15.26:0/4029931219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1830037070 con 0x7f1840152920 2024-03-06T07:28:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f183e7fc640 1 -- 172.21.15.26:0/4029931219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830031960 con 0x7f1840152920 2024-03-06T07:28:12.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 -- 172.21.15.26:0/4029931219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1840152920 msgr2=0x7f1840155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 --2- 172.21.15.26:0/4029931219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1840152920 0x7f1840155c40 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f18300097a0 tx=0x7f183002b840 comp rx=0 tx=0).stop 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 -- 172.21.15.26:0/4029931219 shutdown_connections 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 --2- 172.21.15.26:0/4029931219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18400c8a00 0x7f1840158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 --2- 172.21.15.26:0/4029931219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1840152920 0x7f1840155c40 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 -- 172.21.15.26:0/4029931219 >> 172.21.15.26:0/4029931219 conn(0x7f18400ac560 msgr2=0x7f18400ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 -- 172.21.15.26:0/4029931219 shutdown_connections 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.885+0000 7f1845afa640 1 -- 172.21.15.26:0/4029931219 wait complete. 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.886+0000 7f1845afa640 1 Processor -- start 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f1845afa640 1 -- start start 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f1845afa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18400c8a00 0x7f18400ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f1845afa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1840152920 0x7f1840148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f1845afa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184014cb90 0x7f1840149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f1845afa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184006d030 con 0x7f184014cb90 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f1845afa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1840158a00 con 0x7f18400c8a00 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f1845afa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18400c5a40 con 0x7f1840152920 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f183ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184014cb90 0x7f1840149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f183ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184014cb90 0x7f1840149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48274/0 (socket says 172.21.15.26:48274) 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f183ffff640 1 -- 172.21.15.26:0/3537433650 learned_addr learned my addr 172.21.15.26:0/3537433650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f183effd640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1840152920 0x7f1840148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.887+0000 7f183f7fe640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18400c8a00 0x7f18400ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f183ffff640 1 -- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1840152920 msgr2=0x7f1840148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f183ffff640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1840152920 0x7f1840148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f183ffff640 1 -- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18400c8a00 msgr2=0x7f18400ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f183ffff640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18400c8a00 0x7f18400ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f183ffff640 1 -- 172.21.15.26:0/3537433650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840150840 con 0x7f184014cb90 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f183ffff640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184014cb90 0x7f1840149380 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f183800c7b0 tx=0x7f183800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f1827fff640 1 -- 172.21.15.26:0/3537433650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838018070 con 0x7f184014cb90 2024-03-06T07:28:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f1827fff640 1 -- 172.21.15.26:0/3537433650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183800a640 con 0x7f184014cb90 2024-03-06T07:28:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184014d010 con 0x7f184014cb90 2024-03-06T07:28:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f1827fff640 1 -- 172.21.15.26:0/3537433650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800ce60 con 0x7f184014cb90 2024-03-06T07:28:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.888+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18401552e0 con 0x7f184014cb90 2024-03-06T07:28:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.889+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184004f330 con 0x7f184014cb90 2024-03-06T07:28:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.890+0000 7f1827fff640 1 -- 172.21.15.26:0/3537433650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f183801b070 con 0x7f184014cb90 2024-03-06T07:28:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.890+0000 7f1827fff640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.890+0000 7f1827fff640 1 -- 172.21.15.26:0/3537433650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1838089390 con 0x7f184014cb90 2024-03-06T07:28:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.890+0000 7f183f7fe640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.891+0000 7f183f7fe640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f182c06b4d0 0x7f182c06d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1830036000 tx=0x7f183003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:12.894+0000 7f1827fff640 1 -- 172.21.15.26:0/3537433650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1840156180 con 0x7f184014cb90 2024-03-06T07:28:13.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.076+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f18400cae10 con 0x7f184014cb90 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.076+0000 7f1827fff640 1 -- 172.21.15.26:0/3537433650 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v194) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1838057e10 con 0x7f184014cb90 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:28:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:28:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f182c06b4d0 msgr2=0x7f182c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f182c06b4d0 0x7f182c06d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1830036000 tx=0x7f183003d040 comp rx=0 tx=0).stop 2024-03-06T07:28:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184014cb90 msgr2=0x7f1840149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184014cb90 0x7f1840149380 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f183800c7b0 tx=0x7f183800cc70 comp rx=0 tx=0).stop 2024-03-06T07:28:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 shutdown_connections 2024-03-06T07:28:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184014cb90 0x7f1840149380 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1840152920 0x7f1840148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 --2- 172.21.15.26:0/3537433650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18400c8a00 0x7f18400ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 >> 172.21.15.26:0/3537433650 conn(0x7f18400ac560 msgr2=0x7f18400ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 shutdown_connections 2024-03-06T07:28:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.078+0000 7f1845afa640 1 -- 172.21.15.26:0/3537433650 wait complete. 2024-03-06T07:28:13.090 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:28:13.091 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:28:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.235+0000 7f4c5e5bd640 1 Processor -- start 2024-03-06T07:28:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.235+0000 7f4c5e5bd640 1 -- start start 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.235+0000 7f4c5e5bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c580a9830 0x7f4c58151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.235+0000 7f4c5e5bd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5806e850 con 0x7f4c580c22b0 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.235+0000 7f4c5e5bd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5806d2b0 con 0x7f4c58153390 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.235+0000 7f4c5e5bd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5806d590 con 0x7f4c580a9830 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c57fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4c580c22b0 0x7f4c5814ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39406/0 (socket says 172.21.15.26:39406) 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c57fff640 1 -- 172.21.15.26:0/801000308 learned_addr learned my addr 172.21.15.26:0/801000308 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c577fe640 1 --2- 172.21.15.26:0/801000308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c580a9830 0x7f4c58151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c56ffd640 1 -- 172.21.15.26:0/801000308 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191915820 0 0) 0x7f4c48001110 con 0x7f4c580c22b0 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c56ffd640 1 -- 172.21.15.26:0/801000308 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c40003520 con 0x7f4c580c22b0 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c56ffd640 1 -- 172.21.15.26:0/801000308 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460938170 0 0) 0x7f4c5806d2b0 con 0x7f4c58153390 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c56ffd640 1 -- 172.21.15.26:0/801000308 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c48001110 con 0x7f4c58153390 2024-03-06T07:28:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c577fe640 1 -- 172.21.15.26:0/801000308 >> v1:172.21.15.117:6789/0 conn(0x7f4c58153390 legacy=0x7f4c58153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c577fe640 1 -- 172.21.15.26:0/801000308 >> v1:172.21.15.26:6789/0 conn(0x7f4c580c22b0 legacy=0x7f4c5814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.236+0000 7f4c577fe640 1 -- 172.21.15.26:0/801000308 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5806d870 con 0x7f4c580a9830 2024-03-06T07:28:13.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.237+0000 7f4c577fe640 1 --2- 172.21.15.26:0/801000308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c580a9830 0x7f4c58151e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0097d0 tx=0x7f4c4c02db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8522bee143eeb2d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.237+0000 7f4c56ffd640 1 -- 172.21.15.26:0/801000308 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c02f900 con 0x7f4c580a9830 2024-03-06T07:28:13.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.237+0000 7f4c56ffd640 1 -- 172.21.15.26:0/801000308 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c02faa0 con 0x7f4c580a9830 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.237+0000 7f4c56ffd640 1 -- 172.21.15.26:0/801000308 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c02fe50 con 0x7f4c580a9830 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.238+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/801000308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c580a9830 msgr2=0x7f4c58151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.238+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/801000308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c580a9830 0x7f4c58151e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0097d0 tx=0x7f4c4c02db40 comp rx=0 tx=0).stop 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.238+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/801000308 shutdown_connections 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.238+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/801000308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c580a9830 0x7f4c58151e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.238+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/801000308 >> 172.21.15.26:0/801000308 conn(0x7f4c580c65b0 msgr2=0x7f4c580c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.238+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/801000308 shutdown_connections 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.238+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/801000308 wait complete. 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.239+0000 7f4c5e5bd640 1 Processor -- start 2024-03-06T07:28:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.239+0000 7f4c5e5bd640 1 -- start start 2024-03-06T07:28:13.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.239+0000 7f4c5e5bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c580a9830 0x7f4c580c0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c5e5bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c580c22b0 0x7f4c580bf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c5e5bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c58153390 0x7f4c580bf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c5e5bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58153f30 con 0x7f4c580c22b0 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c5e5bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5814f660 con 0x7f4c580a9830 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c5e5bd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5814ea10 con 0x7f4c58153390 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c580c22b0 0x7f4c580bf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c580c22b0 0x7f4c580bf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48280/0 (socket says 172.21.15.26:48280) 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 -- 172.21.15.26:0/582030178 learned_addr learned my addr 172.21.15.26:0/582030178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c57fff640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c580a9830 0x7f4c580c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c5cb5a640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c58153390 0x7f4c580bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 -- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c58153390 msgr2=0x7f4c580bf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c58153390 0x7f4c580bf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 -- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c580a9830 msgr2=0x7f4c580c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c580a9830 0x7f4c580c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 -- 172.21.15.26:0/582030178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580bfd90 con 0x7f4c580c22b0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c577fe640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c580c22b0 0x7f4c580bf310 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c02db80 tx=0x7f4c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c54ff9640 1 -- 172.21.15.26:0/582030178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c047070 con 0x7f4c580c22b0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c54ff9640 1 -- 172.21.15.26:0/582030178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c037d20 con 0x7f4c580c22b0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c581e6b30 con 0x7f4c580c22b0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.240+0000 7f4c54ff9640 1 -- 172.21.15.26:0/582030178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c040640 con 0x7f4c580c22b0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.241+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c58151c90 con 0x7f4c580c22b0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.242+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5804f330 con 0x7f4c580c22b0 2024-03-06T07:28:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.242+0000 7f4c54ff9640 1 -- 172.21.15.26:0/582030178 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c4c04a070 con 0x7f4c580c22b0 2024-03-06T07:28:13.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.242+0000 7f4c54ff9640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c4406b4d0 0x7f4c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.242+0000 7f4c54ff9640 1 -- 172.21.15.26:0/582030178 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4c4c0ae1d0 con 0x7f4c580c22b0 2024-03-06T07:28:13.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.243+0000 7f4c57fff640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c4406b4d0 0x7f4c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.243+0000 7f4c57fff640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c4406b4d0 0x7f4c4406d990 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000f60 tx=0x7f4c48007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.247+0000 7f4c54ff9640 1 -- 172.21.15.26:0/582030178 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4c4c051030 con 0x7f4c580c22b0 2024-03-06T07:28:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.427+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4c580c0950 con 0x7f4c580c22b0 2024-03-06T07:28:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.764+0000 7f4c54ff9640 1 -- 172.21.15.26:0/582030178 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v195) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4c4c07cbd0 con 0x7f4c580c22b0 2024-03-06T07:28:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c4406b4d0 msgr2=0x7f4c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c4406b4d0 0x7f4c4406d990 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000f60 tx=0x7f4c48007d50 comp rx=0 tx=0).stop 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c580c22b0 msgr2=0x7f4c580bf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c580c22b0 0x7f4c580bf310 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c02db80 tx=0x7f4c4c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 shutdown_connections 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4c4406b4d0 0x7f4c4406d990 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c58153390 0x7f4c580bf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c580c22b0 0x7f4c580bf310 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 --2- 172.21.15.26:0/582030178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c580a9830 0x7f4c580c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 >> 172.21.15.26:0/582030178 conn(0x7f4c580c65b0 msgr2=0x7f4c5814dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 shutdown_connections 2024-03-06T07:28:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.766+0000 7f4c5e5bd640 1 -- 172.21.15.26:0/582030178 wait complete. 2024-03-06T07:28:13.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:28:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.891+0000 7f95642af640 1 Processor -- start 2024-03-06T07:28:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.891+0000 7f95642af640 1 -- start start 2024-03-06T07:28:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.892+0000 7f95642af640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f955c0b0070 0x7f955c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.892+0000 7f95642af640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0b0450 con 0x7f955c06ca70 2024-03-06T07:28:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.892+0000 7f95642af640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c06cf20 con 0x7f955c0c5380 2024-03-06T07:28:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.892+0000 7f95642af640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955c06c020 con 0x7f955c0b0070 2024-03-06T07:28:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.892+0000 7f956204b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f955c0b0070 0x7f955c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.892+0000 7f956184a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f955c06ca70 0x7f955c0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39418/0 (socket says 172.21.15.26:39418) 2024-03-06T07:28:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.892+0000 7f956184a640 1 -- 172.21.15.26:0/914300516 learned_addr learned my addr 172.21.15.26:0/914300516 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.893+0000 7f956204b640 1 -- 172.21.15.26:0/914300516 >> v1:172.21.15.117:6789/0 conn(0x7f955c0c5380 legacy=0x7f955c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.893+0000 7f956204b640 1 -- 172.21.15.26:0/914300516 >> v1:172.21.15.26:6789/0 conn(0x7f955c06ca70 legacy=0x7f955c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.893+0000 7f956204b640 1 -- 172.21.15.26:0/914300516 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0c77f0 con 0x7f955c0b0070 2024-03-06T07:28:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.893+0000 7f956204b640 1 --2- 172.21.15.26:0/914300516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c155c30 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f95580097a0 tx=0x7f955802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0221077f8493e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.893+0000 7f9561049640 1 -- 172.21.15.26:0/914300516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95580315a0 con 0x7f955c0b0070 2024-03-06T07:28:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f9561049640 1 -- 172.21.15.26:0/914300516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558037070 con 0x7f955c0b0070 2024-03-06T07:28:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f9561049640 1 -- 172.21.15.26:0/914300516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558031960 con 0x7f955c0b0070 2024-03-06T07:28:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f95642af640 1 -- 172.21.15.26:0/914300516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 msgr2=0x7f955c155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f95642af640 1 --2- 172.21.15.26:0/914300516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c155c30 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f95580097a0 tx=0x7f955802b840 comp rx=0 tx=0).stop 2024-03-06T07:28:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f95642af640 1 -- 172.21.15.26:0/914300516 shutdown_connections 2024-03-06T07:28:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f95642af640 1 --2- 172.21.15.26:0/914300516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c155c30 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f95642af640 1 -- 172.21.15.26:0/914300516 >> 172.21.15.26:0/914300516 conn(0x7f955c0ac590 msgr2=0x7f955c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f95642af640 1 -- 172.21.15.26:0/914300516 shutdown_connections 2024-03-06T07:28:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.894+0000 7f95642af640 1 -- 172.21.15.26:0/914300516 wait complete. 2024-03-06T07:28:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.895+0000 7f95642af640 1 Processor -- start 2024-03-06T07:28:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.895+0000 7f95642af640 1 -- start start 2024-03-06T07:28:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f95642af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955c06ca70 0x7f955c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f95642af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f95642af640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f955c0c5380 0x7f955c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f95642af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06c550 con 0x7f955c0b0070 2024-03-06T07:28:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f95642af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06c270 con 0x7f955c06ca70 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f95642af640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06da80 con 0x7f955c0c5380 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48308/0 (socket says 172.21.15.26:48308) 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 -- 172.21.15.26:0/1521383000 learned_addr learned my addr 172.21.15.26:0/1521383000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956204b640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955c06ca70 0x7f955c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956284c640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f955c0c5380 0x7f955c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 -- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f955c0c5380 msgr2=0x7f955c158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f955c0c5380 0x7f955c158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 -- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955c06ca70 msgr2=0x7f955c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955c06ca70 0x7f955c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 -- 172.21.15.26:0/1521383000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c1eada0 con 0x7f955c0b0070 2024-03-06T07:28:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956204b640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955c06ca70 0x7f955c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956284c640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f955c0c5380 0x7f955c158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.896+0000 7f956184a640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c158580 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c3c0 tx=0x7f954c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.897+0000 7f9552ffd640 1 -- 172.21.15.26:0/1521383000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c017070 con 0x7f955c0b0070 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.897+0000 7f9552ffd640 1 -- 172.21.15.26:0/1521383000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c005ce0 con 0x7f955c0b0070 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.897+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c1ee4b0 con 0x7f955c0b0070 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.897+0000 7f9552ffd640 1 -- 172.21.15.26:0/1521383000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0072f0 con 0x7f955c0b0070 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.897+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c1eb340 con 0x7f955c0b0070 2024-03-06T07:28:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.898+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c04f330 con 0x7f955c0b0070 2024-03-06T07:28:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.898+0000 7f9552ffd640 1 -- 172.21.15.26:0/1521383000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f954c01a070 con 0x7f955c0b0070 2024-03-06T07:28:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.899+0000 7f9552ffd640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f954006b4d0 0x7f954006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.899+0000 7f9552ffd640 1 -- 172.21.15.26:0/1521383000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f954c089560 con 0x7f955c0b0070 2024-03-06T07:28:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.899+0000 7f956204b640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f954006b4d0 0x7f954006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.900+0000 7f956204b640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f954006b4d0 0x7f954006d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9558036000 tx=0x7f955803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:13.903+0000 7f9552ffd640 1 -- 172.21.15.26:0/1521383000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f955c0b0450 con 0x7f955c0b0070 2024-03-06T07:28:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.086+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f955c0ae200 con 0x7f955c0b0070 2024-03-06T07:28:14.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.766+0000 7f9552ffd640 1 -- 172.21.15.26:0/1521383000 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v196) v1 ==== 170+0+0 (secure 0 0 0) 0x7f954c058420 con 0x7f955c0b0070 2024-03-06T07:28:14.768 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:28:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f954006b4d0 msgr2=0x7f954006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f954006b4d0 0x7f954006d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9558036000 tx=0x7f955803d040 comp rx=0 tx=0).stop 2024-03-06T07:28:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 msgr2=0x7f955c158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c158580 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c3c0 tx=0x7f954c00c880 comp rx=0 tx=0).stop 2024-03-06T07:28:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 shutdown_connections 2024-03-06T07:28:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f954006b4d0 0x7f954006d990 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f955c0c5380 0x7f955c158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c0b0070 0x7f955c158580 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 --2- 172.21.15.26:0/1521383000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955c06ca70 0x7f955c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.769+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 >> 172.21.15.26:0/1521383000 conn(0x7f955c0ac590 msgr2=0x7f955c1514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.770+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 shutdown_connections 2024-03-06T07:28:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.770+0000 7f95642af640 1 -- 172.21.15.26:0/1521383000 wait complete. 2024-03-06T07:28:14.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:28:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.907+0000 7f624d59a640 1 Processor -- start 2024-03-06T07:28:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.907+0000 7f624d59a640 1 -- start start 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f624d59a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624806d9f0 0x7f62480a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f624d59a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62480c3280 0x7f62480c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f624d59a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62481551e0 0x7f6248155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f624d59a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6248158800 con 0x7f62481551e0 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f624d59a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6248158ae0 con 0x7f62480c3280 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f624d59a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6248158dc0 con 0x7f624806d9f0 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f62477fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62481551e0 0x7f6248155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f62477fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62481551e0 0x7f6248155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48322/0 (socket says 172.21.15.26:48322) 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f62477fe640 1 -- 172.21.15.26:0/2194077920 learned_addr learned my addr 172.21.15.26:0/2194077920 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f62467fc640 1 --2- 172.21.15.26:0/2194077920 >> v2:172.21.15.117:3300/0 conn(0x7f62480c3280 0x7f62480c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.908+0000 7f6246ffd640 1 --2- 172.21.15.26:0/2194077920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624806d9f0 0x7f62480a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.909+0000 7f62477fe640 1 -- 172.21.15.26:0/2194077920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624806d9f0 msgr2=0x7f62480a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.909+0000 7f62477fe640 1 --2- 172.21.15.26:0/2194077920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624806d9f0 0x7f62480a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.909+0000 7f62477fe640 1 -- 172.21.15.26:0/2194077920 >> v2:172.21.15.117:3300/0 conn(0x7f62480c3280 msgr2=0x7f62480c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.909+0000 7f62477fe640 1 --2- 172.21.15.26:0/2194077920 >> v2:172.21.15.117:3300/0 conn(0x7f62480c3280 0x7f62480c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.909+0000 7f62477fe640 1 -- 172.21.15.26:0/2194077920 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62481590a0 con 0x7f62481551e0 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.909+0000 7f6246ffd640 1 --2- 172.21.15.26:0/2194077920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624806d9f0 0x7f62480a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.909+0000 7f62477fe640 1 --2- 172.21.15.26:0/2194077920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62481551e0 0x7f6248155610 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f62480c98d0 tx=0x7f623802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733a723a375ab2da server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f6245ffb640 1 -- 172.21.15.26:0/2194077920 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623802c9e0 con 0x7f62481551e0 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f6245ffb640 1 -- 172.21.15.26:0/2194077920 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623802cb80 con 0x7f62481551e0 2024-03-06T07:28:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f6245ffb640 1 -- 172.21.15.26:0/2194077920 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380343f0 con 0x7f62481551e0 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 -- 172.21.15.26:0/2194077920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62481551e0 msgr2=0x7f6248155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 --2- 172.21.15.26:0/2194077920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62481551e0 0x7f6248155610 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f62480c98d0 tx=0x7f623802be40 comp rx=0 tx=0).stop 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 -- 172.21.15.26:0/2194077920 shutdown_connections 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 --2- 172.21.15.26:0/2194077920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62481551e0 0x7f6248155610 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 --2- 172.21.15.26:0/2194077920 >> v2:172.21.15.117:3300/0 conn(0x7f62480c3280 0x7f62480c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 --2- 172.21.15.26:0/2194077920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624806d9f0 0x7f62480a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 -- 172.21.15.26:0/2194077920 >> 172.21.15.26:0/2194077920 conn(0x7f62480aa630 msgr2=0x7f62480aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:14.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 -- 172.21.15.26:0/2194077920 shutdown_connections 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.910+0000 7f624d59a640 1 -- 172.21.15.26:0/2194077920 wait complete. 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.911+0000 7f624d59a640 1 Processor -- start 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.911+0000 7f624d59a640 1 -- start start 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f624d59a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f624806d9f0 0x7f624806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f624d59a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62480c3280 0x7f6248151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f624d59a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62481551e0 0x7f62481f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f624d59a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624806df50 con 0x7f624806d9f0 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f624d59a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624806ddd0 con 0x7f62480c3280 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f624d59a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480a8210 con 0x7f62481551e0 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f624806d9f0 0x7f624806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f624806d9f0 0x7f624806efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48326/0 (socket says 172.21.15.26:48326) 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 -- 172.21.15.26:0/2376309018 learned_addr learned my addr 172.21.15.26:0/2376309018 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f62477fe640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62481551e0 0x7f62481f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f62467fc640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62480c3280 0x7f6248151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 -- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62481551e0 msgr2=0x7f62481f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62481551e0 0x7f62481f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 -- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62480c3280 msgr2=0x7f6248151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62480c3280 0x7f6248151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.912+0000 7f6246ffd640 1 -- 172.21.15.26:0/2376309018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62480a78a0 con 0x7f624806d9f0 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.913+0000 7f6246ffd640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f624806d9f0 0x7f624806efb0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f624000d7d0 tx=0x7f624000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.913+0000 7f623f7fe640 1 -- 172.21.15.26:0/2376309018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240004030 con 0x7f624806d9f0 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.913+0000 7f623f7fe640 1 -- 172.21.15.26:0/2376309018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62400041d0 con 0x7f624806d9f0 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.913+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624806fd10 con 0x7f624806d9f0 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.913+0000 7f623f7fe640 1 -- 172.21.15.26:0/2376309018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62400096a0 con 0x7f624806d9f0 2024-03-06T07:28:14.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.913+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624806fff0 con 0x7f624806d9f0 2024-03-06T07:28:14.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.914+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624804f330 con 0x7f624806d9f0 2024-03-06T07:28:14.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.915+0000 7f623f7fe640 1 -- 172.21.15.26:0/2376309018 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6240009840 con 0x7f624806d9f0 2024-03-06T07:28:14.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.915+0000 7f623f7fe640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f622806b4d0 0x7f622806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.915+0000 7f623f7fe640 1 -- 172.21.15.26:0/2376309018 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f624008a3d0 con 0x7f624806d9f0 2024-03-06T07:28:14.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.915+0000 7f62467fc640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f622806b4d0 0x7f622806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.916+0000 7f62467fc640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f622806b4d0 0x7f622806d990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f62481e1d50 tx=0x7f6230009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:14.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:14.920+0000 7f623f7fe640 1 -- 172.21.15.26:0/2376309018 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6240058b60 con 0x7f624806d9f0 2024-03-06T07:28:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.140+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6248072a00 con 0x7f624806d9f0 2024-03-06T07:28:15.799 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 31 and data pool 32 2024-03-06T07:28:15.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.795+0000 7f623f7fe640 1 -- 172.21.15.26:0/2376309018 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v185) v1 ==== 174+0+0 (secure 0 0 0) 0x7f624005cc00 con 0x7f624806d9f0 2024-03-06T07:28:15.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.801+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f622806b4d0 msgr2=0x7f622806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.801+0000 7f624d59a640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f622806b4d0 0x7f622806d990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f62481e1d50 tx=0x7f6230009190 comp rx=0 tx=0).stop 2024-03-06T07:28:15.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.801+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f624806d9f0 msgr2=0x7f624806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.801+0000 7f624d59a640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f624806d9f0 0x7f624806efb0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f624000d7d0 tx=0x7f624000dc90 comp rx=0 tx=0).stop 2024-03-06T07:28:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.803+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 shutdown_connections 2024-03-06T07:28:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.803+0000 7f624d59a640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f622806b4d0 0x7f622806d990 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.803+0000 7f624d59a640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62481551e0 0x7f62481f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.803+0000 7f624d59a640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62480c3280 0x7f6248151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.803+0000 7f624d59a640 1 --2- 172.21.15.26:0/2376309018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f624806d9f0 0x7f624806efb0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.803+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 >> 172.21.15.26:0/2376309018 conn(0x7f62480aa630 msgr2=0x7f624806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.806+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 shutdown_connections 2024-03-06T07:28:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.808+0000 7f624d59a640 1 -- 172.21.15.26:0/2376309018 wait complete. 2024-03-06T07:28:15.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:15.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.933+0000 7fd5044e1640 1 Processor -- start 2024-03-06T07:28:15.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.933+0000 7fd5044e1640 1 -- start start 2024-03-06T07:28:15.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.934+0000 7fd5044e1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4fc155c20 0x7fd4fc0c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.934+0000 7fd5044e1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc06afc0 con 0x7fd4fc0c17c0 2024-03-06T07:28:15.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.934+0000 7fd5044e1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc06b2a0 con 0x7fd4fc152920 2024-03-06T07:28:15.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.934+0000 7fd5044e1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc06b580 con 0x7fd4fc155c20 2024-03-06T07:28:15.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.934+0000 7fd501a7c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4fc155c20 0x7fd4fc0c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.934+0000 7fd502a7e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd4fc0c17c0 0x7fd4fc15fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39428/0 (socket says 172.21.15.26:39428) 2024-03-06T07:28:15.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.934+0000 7fd502a7e640 1 -- 172.21.15.26:0/162297671 learned_addr learned my addr 172.21.15.26:0/162297671 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:15.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.935+0000 7fd50127b640 1 -- 172.21.15.26:0/162297671 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289694237 0 0) 0x7fd4fc06afc0 con 0x7fd4fc0c17c0 2024-03-06T07:28:15.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.935+0000 7fd50127b640 1 -- 172.21.15.26:0/162297671 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e4003520 con 0x7fd4fc0c17c0 2024-03-06T07:28:15.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.935+0000 7fd501a7c640 1 -- 172.21.15.26:0/162297671 >> v1:172.21.15.117:6789/0 conn(0x7fd4fc152920 legacy=0x7fd4fc0c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.935+0000 7fd501a7c640 1 -- 172.21.15.26:0/162297671 >> v1:172.21.15.26:6789/0 conn(0x7fd4fc0c17c0 legacy=0x7fd4fc15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.935+0000 7fd501a7c640 1 -- 172.21.15.26:0/162297671 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc06b860 con 0x7fd4fc155c20 2024-03-06T07:28:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.935+0000 7fd501a7c640 1 --2- 172.21.15.26:0/162297671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc155c20 0x7fd4fc0c1280 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00097a0 tx=0x7fd4f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b7d83e04cd8b0f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd50127b640 1 -- 172.21.15.26:0/162297671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0034070 con 0x7fd4fc155c20 2024-03-06T07:28:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd50127b640 1 -- 172.21.15.26:0/162297671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f0030610 con 0x7fd4fc155c20 2024-03-06T07:28:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd50127b640 1 -- 172.21.15.26:0/162297671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00309e0 con 0x7fd4fc155c20 2024-03-06T07:28:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd5044e1640 1 -- 172.21.15.26:0/162297671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc155c20 msgr2=0x7fd4fc0c1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd5044e1640 1 --2- 172.21.15.26:0/162297671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc155c20 0x7fd4fc0c1280 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00097a0 tx=0x7fd4f002b960 comp rx=0 tx=0).stop 2024-03-06T07:28:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd5044e1640 1 -- 172.21.15.26:0/162297671 shutdown_connections 2024-03-06T07:28:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd5044e1640 1 --2- 172.21.15.26:0/162297671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc155c20 0x7fd4fc0c1280 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd5044e1640 1 -- 172.21.15.26:0/162297671 >> 172.21.15.26:0/162297671 conn(0x7fd4fc0c4290 msgr2=0x7fd4fc0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd5044e1640 1 -- 172.21.15.26:0/162297671 shutdown_connections 2024-03-06T07:28:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.936+0000 7fd5044e1640 1 -- 172.21.15.26:0/162297671 wait complete. 2024-03-06T07:28:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.937+0000 7fd5044e1640 1 Processor -- start 2024-03-06T07:28:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.937+0000 7fd5044e1640 1 -- start start 2024-03-06T07:28:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd5044e1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4fc0c17c0 0x7fd4fc0aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd5044e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc152920 0x7fd4fc14b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd5044e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc155c20 0x7fd4fc06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd5044e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c28a0 con 0x7fd4fc152920 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd5044e1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc1560b0 con 0x7fd4fc155c20 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd5044e1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c2a20 con 0x7fd4fc0c17c0 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc152920 0x7fd4fc14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc152920 0x7fd4fc14b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48338/0 (socket says 172.21.15.26:48338) 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 -- 172.21.15.26:0/926435783 learned_addr learned my addr 172.21.15.26:0/926435783 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd502a7e640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc155c20 0x7fd4fc06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd50227d640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4fc0c17c0 0x7fd4fc0aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 -- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4fc0c17c0 msgr2=0x7fd4fc0aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4fc0c17c0 0x7fd4fc0aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 -- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc155c20 msgr2=0x7fd4fc06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc155c20 0x7fd4fc06c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.938+0000 7fd501a7c640 1 -- 172.21.15.26:0/926435783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc14c000 con 0x7fd4fc152920 2024-03-06T07:28:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.939+0000 7fd501a7c640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc152920 0x7fd4fc14b530 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.939+0000 7fd4faffd640 1 -- 172.21.15.26:0/926435783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0034040 con 0x7fd4fc152920 2024-03-06T07:28:15.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.939+0000 7fd4faffd640 1 -- 172.21.15.26:0/926435783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f003aaa0 con 0x7fd4fc152920 2024-03-06T07:28:15.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.939+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc0abb70 con 0x7fd4fc152920 2024-03-06T07:28:15.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.939+0000 7fd4faffd640 1 -- 172.21.15.26:0/926435783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00443f0 con 0x7fd4fc152920 2024-03-06T07:28:15.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.939+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc0ac130 con 0x7fd4fc152920 2024-03-06T07:28:15.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.940+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc04f330 con 0x7fd4fc152920 2024-03-06T07:28:15.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.940+0000 7fd4faffd640 1 -- 172.21.15.26:0/926435783 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4f004b060 con 0x7fd4fc152920 2024-03-06T07:28:15.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.941+0000 7fd4faffd640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.941+0000 7fd4faffd640 1 -- 172.21.15.26:0/926435783 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4f00abf40 con 0x7fd4fc152920 2024-03-06T07:28:15.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.941+0000 7fd50227d640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.941+0000 7fd50227d640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4001780 tx=0x7fd4f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:15.945+0000 7fd4faffd640 1 -- 172.21.15.26:0/926435783 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd4f003f070 con 0x7fd4fc152920 2024-03-06T07:28:16.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.127+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4fc068460 con 0x7fd4fc152920 2024-03-06T07:28:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.129+0000 7fd4faffd640 1 -- 172.21.15.26:0/926435783 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16977 (secure 0 0 0) 0x7fd4f007a670 con 0x7fd4fc152920 2024-03-06T07:28:16.130 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:16.131 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":197,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:15.141921+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.447072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:14.106103+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:16.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.130+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 msgr2=0x7fd4e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.130+0000 7fd5044e1640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4001780 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2024-03-06T07:28:16.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.130+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc152920 msgr2=0x7fd4fc14b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.130+0000 7fd5044e1640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc152920 0x7fd4fc14b530 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f00046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 shutdown_connections 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd4e806b4d0 0x7fd4e806d990 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4fc155c20 0x7fd4fc06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4fc152920 0x7fd4fc14b530 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 --2- 172.21.15.26:0/926435783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4fc0c17c0 0x7fd4fc0aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 >> 172.21.15.26:0/926435783 conn(0x7fd4fc0c4290 msgr2=0x7fd4fc150140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 shutdown_connections 2024-03-06T07:28:16.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.131+0000 7fd5044e1640 1 -- 172.21.15.26:0/926435783 wait complete. 2024-03-06T07:28:16.143 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:16.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.243+0000 7f894153e640 1 Processor -- start 2024-03-06T07:28:16.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.243+0000 7f894153e640 1 -- start start 2024-03-06T07:28:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f894153e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f893c152920 0x7f893c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f894153e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 0x7f893c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f894153e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c156180 con 0x7f893c159460 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f894153e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893c1587b0 con 0x7f893c152920 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f894153e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0c7760 con 0x7f893c0c8a00 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f893c159460 0x7f893c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39436/0 (socket says 172.21.15.26:39436) 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893b7fe640 1 -- 172.21.15.26:0/3622834103 learned_addr learned my addr 172.21.15.26:0/3622834103 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893affd640 1 --2- 172.21.15.26:0/3622834103 >> v2:172.21.15.117:3300/0 conn(0x7f893c152920 0x7f893c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f8933fff640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 0x7f893c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990177872 0 0) 0x7f8934001490 con 0x7f893c159460 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8928003520 con 0x7f893c159460 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3322930605 0 0) 0x7f8928003520 con 0x7f893c159460 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 msgr2=0x7f893c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893a7fc640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 0x7f893c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 >> v2:172.21.15.117:3300/0 conn(0x7f893c152920 msgr2=0x7f893c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.244+0000 7f893a7fc640 1 --2- 172.21.15.26:0/3622834103 >> v2:172.21.15.117:3300/0 conn(0x7f893c152920 0x7f893c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0c7a40 con 0x7f893c159460 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f8934001bc0 con 0x7f893c159460 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893affd640 1 --2- 172.21.15.26:0/3622834103 >> v2:172.21.15.117:3300/0 conn(0x7f893c152920 0x7f893c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 >> v1:172.21.15.26:6789/0 conn(0x7f893c159460 legacy=0x7f893c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89280039b0 0x7f8928003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 0x7f8928004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928004d40 0x7f89280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1587b0 con 0x7f8928004d40 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0c7760 con 0x7f89280039b0 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892800b9c0 con 0x7f893c0c8a00 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893b7fe640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928004d40 0x7f89280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893affd640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89280039b0 0x7f8928003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f8933fff640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 0x7f8928004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893b7fe640 1 -- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 msgr2=0x7f8928004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893b7fe640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c8a00 0x7f8928004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893b7fe640 1 -- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89280039b0 msgr2=0x7f8928003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893b7fe640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89280039b0 0x7f8928003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.245+0000 7f893b7fe640 1 -- 172.21.15.26:0/3622834103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928003520 con 0x7f8928004d40 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f893b7fe640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928004d40 0x7f89280051c0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f893c0c4c60 tx=0x7f8928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25240428aa8a969a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400a6b0 con 0x7f8928004d40 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89340072d0 con 0x7f8928004d40 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f893a7fc640 1 -- 172.21.15.26:0/3622834103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89340076c0 con 0x7f8928004d40 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f894153e640 1 -- 172.21.15.26:0/3622834103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928004d40 msgr2=0x7f89280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f894153e640 1 --2- 172.21.15.26:0/3622834103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928004d40 0x7f89280051c0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f893c0c4c60 tx=0x7f8928005830 comp rx=0 tx=0).stop 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f893b7fe640 1 -- 172.21.15.26:0/3622834103 reap_dead start 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f894153e640 1 -- 172.21.15.26:0/3622834103 shutdown_connections 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.246+0000 7f894153e640 1 -- 172.21.15.26:0/3622834103 >> 172.21.15.26:0/3622834103 conn(0x7f893c0ac560 msgr2=0x7f893c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.247+0000 7f894153e640 1 -- 172.21.15.26:0/3622834103 shutdown_connections 2024-03-06T07:28:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.247+0000 7f894153e640 1 -- 172.21.15.26:0/3622834103 wait complete. 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 Processor -- start 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 -- start start 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c0be700 0x7f893c1e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c1e0a10 0x7f893c1edf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c10f0 0x7f893c1ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c06d030 con 0x7f893c1e0a10 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c158a00 con 0x7f893c0be700 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f894153e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0c5a40 con 0x7f893c0c10f0 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f8933fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c1e0a10 0x7f893c1edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f8933fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c1e0a10 0x7f893c1edf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48352/0 (socket says 172.21.15.26:48352) 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.248+0000 7f8933fff640 1 -- 172.21.15.26:0/2411274660 learned_addr learned my addr 172.21.15.26:0/2411274660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f893affd640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c0be700 0x7f893c1e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f893b7fe640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c10f0 0x7f893c1ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f8933fff640 1 -- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c10f0 msgr2=0x7f893c1ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f8933fff640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c10f0 0x7f893c1ee4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f8933fff640 1 -- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c0be700 msgr2=0x7f893c1e04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f8933fff640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c0be700 0x7f893c1e04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f8933fff640 1 -- 172.21.15.26:0/2411274660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c14bab0 con 0x7f893c1e0a10 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f8933fff640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c1e0a10 0x7f893c1edf80 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f8924011a00 tx=0x7f8924011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f89337fe640 1 -- 172.21.15.26:0/2411274660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892400faf0 con 0x7f893c1e0a10 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f89337fe640 1 -- 172.21.15.26:0/2411274660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892400fc90 con 0x7f893c1e0a10 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f89337fe640 1 -- 172.21.15.26:0/2411274660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924014440 con 0x7f893c1e0a10 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c0c0610 con 0x7f893c1e0a10 2024-03-06T07:28:16.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.249+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c1dfa40 con 0x7f893c1e0a10 2024-03-06T07:28:16.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.250+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c04f330 con 0x7f893c1e0a10 2024-03-06T07:28:16.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.251+0000 7f89337fe640 1 -- 172.21.15.26:0/2411274660 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8924012040 con 0x7f893c1e0a10 2024-03-06T07:28:16.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.251+0000 7f89337fe640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f891c06b500 0x7f891c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.251+0000 7f89337fe640 1 -- 172.21.15.26:0/2411274660 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f892408a970 con 0x7f893c1e0a10 2024-03-06T07:28:16.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.251+0000 7f893affd640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f891c06b500 0x7f891c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.252+0000 7f893affd640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f891c06b500 0x7f891c06d9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f892c009c90 tx=0x7f892c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.255+0000 7f89337fe640 1 -- 172.21.15.26:0/2411274660 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8924019030 con 0x7f893c1e0a10 2024-03-06T07:28:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.441+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f893c06cda0 con 0x7f893c1e0a10 2024-03-06T07:28:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.442+0000 7f89337fe640 1 -- 172.21.15.26:0/2411274660 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16977 (secure 0 0 0) 0x7f89240596f0 con 0x7f893c1e0a10 2024-03-06T07:28:16.444 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:16.444 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":197,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:15.141921+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.447072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:14.106103+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f891c06b500 msgr2=0x7f891c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f891c06b500 0x7f891c06d9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f892c009c90 tx=0x7f892c008040 comp rx=0 tx=0).stop 2024-03-06T07:28:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c1e0a10 msgr2=0x7f893c1edf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c1e0a10 0x7f893c1edf80 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f8924011a00 tx=0x7f8924011ec0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 shutdown_connections 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f891c06b500 0x7f891c06d9c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893c0c10f0 0x7f893c1ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f893c1e0a10 0x7f893c1edf80 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 --2- 172.21.15.26:0/2411274660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c0be700 0x7f893c1e04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 >> 172.21.15.26:0/2411274660 conn(0x7f893c0ac560 msgr2=0x7f893c0aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 shutdown_connections 2024-03-06T07:28:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.445+0000 7f894153e640 1 -- 172.21.15.26:0/2411274660 wait complete. 2024-03-06T07:28:16.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:28:16.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.560+0000 7fb994839640 1 Processor -- start 2024-03-06T07:28:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.560+0000 7fb994839640 1 -- start start 2024-03-06T07:28:16.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb994839640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9900c5990 0x7fb9900c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb994839640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb99006c900 0x7fb9900a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb994839640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9900c3ef0 con 0x7fb9900ab170 2024-03-06T07:28:16.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb994839640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9901556d0 con 0x7fb9900c5990 2024-03-06T07:28:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb994839640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9901559b0 con 0x7fb99006c900 2024-03-06T07:28:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9900c5990 0x7fb9900c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb987fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb9900ab170 0x7fb99006c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39448/0 (socket says 172.21.15.26:39448) 2024-03-06T07:28:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb987fff640 1 -- 172.21.15.26:0/1339777689 learned_addr learned my addr 172.21.15.26:0/1339777689 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98ed9d640 1 --2- 172.21.15.26:0/1339777689 >> v2:172.21.15.117:3300/0 conn(0x7fb99006c900 0x7fb9900a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98e59c640 1 -- 172.21.15.26:0/1339777689 >> v2:172.21.15.117:3300/0 conn(0x7fb99006c900 msgr2=0x7fb9900a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98e59c640 1 --2- 172.21.15.26:0/1339777689 >> v2:172.21.15.117:3300/0 conn(0x7fb99006c900 0x7fb9900a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98e59c640 1 -- 172.21.15.26:0/1339777689 >> v1:172.21.15.26:6789/0 conn(0x7fb9900ab170 legacy=0x7fb99006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98dd9b640 1 -- 172.21.15.26:0/1339777689 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962546687 0 0) 0x7fb9900c3ef0 con 0x7fb9900ab170 2024-03-06T07:28:16.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98e59c640 1 -- 172.21.15.26:0/1339777689 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb990155c90 con 0x7fb9900c5990 2024-03-06T07:28:16.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.561+0000 7fb98ed9d640 1 --2- 172.21.15.26:0/1339777689 >> v2:172.21.15.117:3300/0 conn(0x7fb99006c900 0x7fb9900a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb98e59c640 1 --2- 172.21.15.26:0/1339777689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900c5990 0x7fb9900c0ab0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fb9800097a0 tx=0x7fb98002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edf16ad7518ae27 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb98dd9b640 1 -- 172.21.15.26:0/1339777689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980035020 con 0x7fb9900c5990 2024-03-06T07:28:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb98dd9b640 1 -- 172.21.15.26:0/1339777689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb980031830 con 0x7fb9900c5990 2024-03-06T07:28:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb98dd9b640 1 -- 172.21.15.26:0/1339777689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980031c00 con 0x7fb9900c5990 2024-03-06T07:28:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb994839640 1 -- 172.21.15.26:0/1339777689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900c5990 msgr2=0x7fb9900c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb994839640 1 --2- 172.21.15.26:0/1339777689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900c5990 0x7fb9900c0ab0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fb9800097a0 tx=0x7fb98002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb994839640 1 -- 172.21.15.26:0/1339777689 shutdown_connections 2024-03-06T07:28:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb994839640 1 --2- 172.21.15.26:0/1339777689 >> v2:172.21.15.117:3300/0 conn(0x7fb99006c900 0x7fb9900a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb994839640 1 --2- 172.21.15.26:0/1339777689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900c5990 0x7fb9900c0ab0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.562+0000 7fb994839640 1 -- 172.21.15.26:0/1339777689 >> 172.21.15.26:0/1339777689 conn(0x7fb9900c6710 msgr2=0x7fb9900c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.563+0000 7fb994839640 1 -- 172.21.15.26:0/1339777689 shutdown_connections 2024-03-06T07:28:16.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.563+0000 7fb994839640 1 -- 172.21.15.26:0/1339777689 wait complete. 2024-03-06T07:28:16.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 Processor -- start 2024-03-06T07:28:16.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 -- start start 2024-03-06T07:28:16.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb99006c900 0x7fb9900c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900ab170 0x7fb990147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9900c5990 0x7fb990143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990156610 con 0x7fb9900ab170 2024-03-06T07:28:16.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990156490 con 0x7fb9900c5990 2024-03-06T07:28:16.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.564+0000 7fb994839640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9900c3af0 con 0x7fb99006c900 2024-03-06T07:28:16.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900ab170 0x7fb990147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900ab170 0x7fb990147920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48376/0 (socket says 172.21.15.26:48376) 2024-03-06T07:28:16.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 -- 172.21.15.26:0/1788703752 learned_addr learned my addr 172.21.15.26:0/1788703752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:16.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb98e59c640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb99006c900 0x7fb9900c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb98ed9d640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9900c5990 0x7fb990143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 -- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb99006c900 msgr2=0x7fb9900c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb99006c900 0x7fb9900c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 -- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9900c5990 msgr2=0x7fb990143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9900c5990 0x7fb990143440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 -- 172.21.15.26:0/1788703752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9900c1ea0 con 0x7fb9900ab170 2024-03-06T07:28:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb98e59c640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb99006c900 0x7fb9900c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb987fff640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900ab170 0x7fb990147920 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c500 tx=0x7fb97800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb986ffd640 1 -- 172.21.15.26:0/1788703752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978018070 con 0x7fb9900ab170 2024-03-06T07:28:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb986ffd640 1 -- 172.21.15.26:0/1788703752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb978007580 con 0x7fb9900ab170 2024-03-06T07:28:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9900c2140 con 0x7fb9900ab170 2024-03-06T07:28:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb986ffd640 1 -- 172.21.15.26:0/1788703752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978004a20 con 0x7fb9900ab170 2024-03-06T07:28:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.565+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9900c2700 con 0x7fb9900ab170 2024-03-06T07:28:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.566+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99004f330 con 0x7fb9900ab170 2024-03-06T07:28:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.567+0000 7fb986ffd640 1 -- 172.21.15.26:0/1788703752 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb97801b070 con 0x7fb9900ab170 2024-03-06T07:28:16.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.567+0000 7fb986ffd640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb97006b4d0 0x7fb97006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.567+0000 7fb986ffd640 1 -- 172.21.15.26:0/1788703752 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb978089630 con 0x7fb9900ab170 2024-03-06T07:28:16.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.568+0000 7fb98e59c640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb97006b4d0 0x7fb97006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.568+0000 7fb98e59c640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb97006b4d0 0x7fb97006d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb980036000 tx=0x7fb980008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.572+0000 7fb986ffd640 1 -- 172.21.15.26:0/1788703752 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb9780583b0 con 0x7fb9900ab170 2024-03-06T07:28:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:16.791+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb990072e70 con 0x7fb9900ab170 2024-03-06T07:28:17.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.805+0000 7fb986ffd640 1 -- 172.21.15.26:0/1788703752 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v187) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb978057cb0 con 0x7fb9900ab170 2024-03-06T07:28:17.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb97006b4d0 msgr2=0x7fb97006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb97006b4d0 0x7fb97006d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb980036000 tx=0x7fb980008030 comp rx=0 tx=0).stop 2024-03-06T07:28:17.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900ab170 msgr2=0x7fb990147920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900ab170 0x7fb990147920 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c500 tx=0x7fb97800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 shutdown_connections 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb97006b4d0 0x7fb97006d990 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9900c5990 0x7fb990143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9900ab170 0x7fb990147920 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 --2- 172.21.15.26:0/1788703752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb99006c900 0x7fb9900c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 >> 172.21.15.26:0/1788703752 conn(0x7fb9900c6710 msgr2=0x7fb990072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 shutdown_connections 2024-03-06T07:28:17.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.807+0000 7fb994839640 1 -- 172.21.15.26:0/1788703752 wait complete. 2024-03-06T07:28:17.820 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:28:17.820 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:28:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.927+0000 7feda6833640 1 Processor -- start 2024-03-06T07:28:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.927+0000 7feda6833640 1 -- start start 2024-03-06T07:28:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda6833640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda00c5990 0x7feda00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda6833640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feda00ab170 0x7feda006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda6833640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda006c730 0x7feda01523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda6833640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda006cbe0 con 0x7feda006c730 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda6833640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda00c3ef0 con 0x7feda00ab170 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda6833640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda01556d0 con 0x7feda00c5990 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda006c730 0x7feda01523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda006c730 0x7feda01523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43860/0 (socket says 172.21.15.26:43860) 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 -- 172.21.15.26:0/1917808299 learned_addr learned my addr 172.21.15.26:0/1917808299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 -- 172.21.15.26:0/1917808299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda00c5990 msgr2=0x7feda00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 --2- 172.21.15.26:0/1917808299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda00c5990 0x7feda00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 -- 172.21.15.26:0/1917808299 >> v2:172.21.15.117:3300/0 conn(0x7feda00ab170 msgr2=0x7feda006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 --2- 172.21.15.26:0/1917808299 >> v2:172.21.15.117:3300/0 conn(0x7feda00ab170 0x7feda006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.928+0000 7feda4dd0640 1 -- 172.21.15.26:0/1917808299 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda01559b0 con 0x7feda006c730 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.929+0000 7feda4dd0640 1 --2- 172.21.15.26:0/1917808299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda006c730 0x7feda01523f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7feda00c4100 tx=0x7fed9802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbe276bdd267803 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.929+0000 7fed9effd640 1 -- 172.21.15.26:0/1917808299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9802ca70 con 0x7feda006c730 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.929+0000 7fed9effd640 1 -- 172.21.15.26:0/1917808299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9802cc10 con 0x7feda006c730 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.929+0000 7fed9effd640 1 -- 172.21.15.26:0/1917808299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed980343f0 con 0x7feda006c730 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 -- 172.21.15.26:0/1917808299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda006c730 msgr2=0x7feda01523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 --2- 172.21.15.26:0/1917808299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda006c730 0x7feda01523f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7feda00c4100 tx=0x7fed9802bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 -- 172.21.15.26:0/1917808299 shutdown_connections 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 --2- 172.21.15.26:0/1917808299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda006c730 0x7feda01523f0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 --2- 172.21.15.26:0/1917808299 >> v2:172.21.15.117:3300/0 conn(0x7feda00ab170 0x7feda006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 --2- 172.21.15.26:0/1917808299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda00c5990 0x7feda00c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 -- 172.21.15.26:0/1917808299 >> 172.21.15.26:0/1917808299 conn(0x7feda00c6710 msgr2=0x7feda00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 -- 172.21.15.26:0/1917808299 shutdown_connections 2024-03-06T07:28:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.930+0000 7feda6833640 1 -- 172.21.15.26:0/1917808299 wait complete. 2024-03-06T07:28:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.931+0000 7feda6833640 1 Processor -- start 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.931+0000 7feda6833640 1 -- start start 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7feda6833640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda006c730 0x7feda014f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7feda6833640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda00ab170 0x7feda0145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7feda6833640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda00c5990 0x7feda0150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7feda6833640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda0156490 con 0x7feda00ab170 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7feda6833640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda0155f70 con 0x7feda00c5990 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7feda6833640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda0156610 con 0x7feda006c730 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda00ab170 0x7feda0145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda00ab170 0x7feda0145990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43874/0 (socket says 172.21.15.26:43874) 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 -- 172.21.15.26:0/205576701 learned_addr learned my addr 172.21.15.26:0/205576701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7feda4dd0640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda00c5990 0x7feda0150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9ffff640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda006c730 0x7feda014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 -- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda006c730 msgr2=0x7feda014f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda006c730 0x7feda014f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 -- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda00c5990 msgr2=0x7feda0150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda00c5990 0x7feda0150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 -- 172.21.15.26:0/205576701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda0146540 con 0x7feda00ab170 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9ffff640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda006c730 0x7feda014f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9f7fe640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda00ab170 0x7feda0145990 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fed9000b450 tx=0x7fed9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.932+0000 7fed9cff9640 1 -- 172.21.15.26:0/205576701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90015020 con 0x7feda00ab170 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.933+0000 7fed9cff9640 1 -- 172.21.15.26:0/205576701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90011890 con 0x7feda00ab170 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.933+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda01ec290 con 0x7feda00ab170 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.933+0000 7fed9cff9640 1 -- 172.21.15.26:0/205576701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90011c80 con 0x7feda00ab170 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.933+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda01ec850 con 0x7feda00ab170 2024-03-06T07:28:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.934+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda014fcc0 con 0x7feda00ab170 2024-03-06T07:28:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.934+0000 7fed9cff9640 1 -- 172.21.15.26:0/205576701 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed90018020 con 0x7feda00ab170 2024-03-06T07:28:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.934+0000 7fed9cff9640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed8006b4d0 0x7fed8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.934+0000 7fed9cff9640 1 -- 172.21.15.26:0/205576701 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed9008a4f0 con 0x7feda00ab170 2024-03-06T07:28:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.935+0000 7fed9ffff640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed8006b4d0 0x7fed8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.935+0000 7fed9ffff640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed8006b4d0 0x7fed8006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fed8800a870 tx=0x7fed88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:17.939+0000 7fed9cff9640 1 -- 172.21.15.26:0/205576701 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed9001f020 con 0x7feda00ab170 2024-03-06T07:28:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.220+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7feda00c1740 con 0x7feda00ab170 2024-03-06T07:28:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.806+0000 7fed9cff9640 1 -- 172.21.15.26:0/205576701 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v188) v1 ==== 122+0+0 (secure 0 0 0) 0x7fed9000d040 con 0x7feda00ab170 2024-03-06T07:28:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed8006b4d0 msgr2=0x7fed8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed8006b4d0 0x7fed8006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fed8800a870 tx=0x7fed88008040 comp rx=0 tx=0).stop 2024-03-06T07:28:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda00ab170 msgr2=0x7feda0145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda00ab170 0x7feda0145990 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fed9000b450 tx=0x7fed9000b910 comp rx=0 tx=0).stop 2024-03-06T07:28:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 shutdown_connections 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed8006b4d0 0x7fed8006d990 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda00c5990 0x7feda0150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda00ab170 0x7feda0145990 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 --2- 172.21.15.26:0/205576701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda006c730 0x7feda014f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.808+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 >> 172.21.15.26:0/205576701 conn(0x7feda00c6710 msgr2=0x7feda00c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.809+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 shutdown_connections 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.809+0000 7feda6833640 1 -- 172.21.15.26:0/205576701 wait complete. 2024-03-06T07:28:18.821 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:28:18.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:18.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f19a2466640 1 Processor -- start 2024-03-06T07:28:18.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f19a2466640 1 -- start start 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f19a2466640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f19a2466640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f199c158b60 0x7f199c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f19a2466640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c0c73d0 con 0x7f199c0c88c0 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f19a2466640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199c0c8dd0 con 0x7f199c158b60 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f19a2466640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0be2e0 con 0x7f199c14f360 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f199b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f199c0c88c0 0x7f199c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57704/0 (socket says 172.21.15.26:57704) 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.921+0000 7f199b7fe640 1 -- 172.21.15.26:0/4189337231 learned_addr learned my addr 172.21.15.26:0/4189337231 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f19a0a03640 1 --2- 172.21.15.26:0/4189337231 >> v2:172.21.15.26:3300/0 conn(0x7f199c158b60 0x7f199c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f199bfff640 1 --2- 172.21.15.26:0/4189337231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f199affd640 1 -- 172.21.15.26:0/4189337231 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452638636 0 0) 0x7f199c0c73d0 con 0x7f199c0c88c0 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f199affd640 1 -- 172.21.15.26:0/4189337231 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1988003520 con 0x7f199c0c88c0 2024-03-06T07:28:18.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f19a0a03640 1 -- 172.21.15.26:0/4189337231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 msgr2=0x7f199c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f19a0a03640 1 --2- 172.21.15.26:0/4189337231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f19a0a03640 1 -- 172.21.15.26:0/4189337231 >> v1:172.21.15.26:6789/0 conn(0x7f199c0c88c0 legacy=0x7f199c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.922+0000 7f19a0a03640 1 -- 172.21.15.26:0/4189337231 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0c7670 con 0x7f199c158b60 2024-03-06T07:28:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a0a03640 1 --2- 172.21.15.26:0/4189337231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c158b60 0x7f199c158ff0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f1994009d90 tx=0x7f199402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bdeddf4fa08872 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f199affd640 1 -- 172.21.15.26:0/4189337231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994039070 con 0x7f199c158b60 2024-03-06T07:28:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f199affd640 1 -- 172.21.15.26:0/4189337231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199402cbd0 con 0x7f199c158b60 2024-03-06T07:28:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f199affd640 1 -- 172.21.15.26:0/4189337231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994034440 con 0x7f199c158b60 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 -- 172.21.15.26:0/4189337231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c158b60 msgr2=0x7f199c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 --2- 172.21.15.26:0/4189337231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c158b60 0x7f199c158ff0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f1994009d90 tx=0x7f199402be20 comp rx=0 tx=0).stop 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 -- 172.21.15.26:0/4189337231 shutdown_connections 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 --2- 172.21.15.26:0/4189337231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c158b60 0x7f199c158ff0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 --2- 172.21.15.26:0/4189337231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 -- 172.21.15.26:0/4189337231 >> 172.21.15.26:0/4189337231 conn(0x7f199c0ac570 msgr2=0x7f199c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 -- 172.21.15.26:0/4189337231 shutdown_connections 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.923+0000 7f19a2466640 1 -- 172.21.15.26:0/4189337231 wait complete. 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.924+0000 7f19a2466640 1 Processor -- start 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f19a2466640 1 -- start start 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f19a2466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c0c88c0 0x7f199c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f19a2466640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c0aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f19a2466640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f199c158b60 0x7f199c0ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f19a2466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0c5a50 con 0x7f199c0c88c0 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f19a2466640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0c7c30 con 0x7f199c158b60 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f19a2466640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0c5bd0 con 0x7f199c14f360 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f199bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c0c88c0 0x7f199c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f199bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c0c88c0 0x7f199c14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43888/0 (socket says 172.21.15.26:43888) 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.925+0000 7f199bfff640 1 -- 172.21.15.26:0/313832136 learned_addr learned my addr 172.21.15.26:0/313832136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f19a0a03640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f199c158b60 0x7f199c0ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f199b7fe640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c0aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f199bfff640 1 -- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 msgr2=0x7f199c0aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f199bfff640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c0aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f199bfff640 1 -- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f199c158b60 msgr2=0x7f199c0ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f199bfff640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f199c158b60 0x7f199c0ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f199bfff640 1 -- 172.21.15.26:0/313832136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c1eb6b0 con 0x7f199c0c88c0 2024-03-06T07:28:18.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f199bfff640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c0c88c0 0x7f199c14b790 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e870 tx=0x7f198c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f1998ff9640 1 -- 172.21.15.26:0/313832136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c017070 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f1998ff9640 1 -- 172.21.15.26:0/313832136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c00c940 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c151aa0 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f1998ff9640 1 -- 172.21.15.26:0/313832136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00cd30 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.926+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c14c000 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.927+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c04f330 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.928+0000 7f1998ff9640 1 -- 172.21.15.26:0/313832136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f198c01a070 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.928+0000 7f1998ff9640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f197c06b4d0 0x7f197c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.928+0000 7f1998ff9640 1 -- 172.21.15.26:0/313832136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f198c08a580 con 0x7f199c0c88c0 2024-03-06T07:28:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.928+0000 7f199b7fe640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f197c06b4d0 0x7f197c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.929+0000 7f199b7fe640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f197c06b4d0 0x7f197c06d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f199c152d30 tx=0x7f1984009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:18.933+0000 7f1998ff9640 1 -- 172.21.15.26:0/313832136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f198c059300 con 0x7f199c0c88c0 2024-03-06T07:28:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.152+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f199c151850 con 0x7f199c0c88c0 2024-03-06T07:28:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.152+0000 7f1998ff9640 1 -- 172.21.15.26:0/313832136 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 188 v188) v1 ==== 95+0+4998 (secure 0 0 0) 0x7f198c058c00 con 0x7f199c0c88c0 2024-03-06T07:28:19.154 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:19.154 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":188,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2181907552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2181907552},{"type":"v1","addr":"172.21.15.117:6836","nonce":2181907552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":8129,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2243060002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2243060002},{"type":"v1","addr":"172.21.15.117:6837","nonce":2243060002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":9529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4068022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4068022212},{"type":"v1","addr":"172.21.15.26:6834","nonce":4068022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":188,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:15.142228+0000","modified":"2024-03-06T07:28:18.221296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10338":{"gid":10338,"name":"a","rank":0,"incarnation":185,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1578728485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1578728485},{"type":"v1","addr":"172.21.15.26:6835","nonce":1578728485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10338,"qdb_cluster":[10338]},"id":15}]} 2024-03-06T07:28:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 188 2024-03-06T07:28:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f197c06b4d0 msgr2=0x7f197c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f197c06b4d0 0x7f197c06d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f199c152d30 tx=0x7f1984009040 comp rx=0 tx=0).stop 2024-03-06T07:28:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c0c88c0 msgr2=0x7f199c14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c0c88c0 0x7f199c14b790 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e870 tx=0x7f198c00ed30 comp rx=0 tx=0).stop 2024-03-06T07:28:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 shutdown_connections 2024-03-06T07:28:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f197c06b4d0 0x7f197c06d990 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f199c158b60 0x7f199c0ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f199c14f360 0x7f199c0aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 --2- 172.21.15.26:0/313832136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f199c0c88c0 0x7f199c14b790 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 >> 172.21.15.26:0/313832136 conn(0x7f199c0ac570 msgr2=0x7f199c1546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 shutdown_connections 2024-03-06T07:28:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.154+0000 7f19a2466640 1 -- 172.21.15.26:0/313832136 wait complete. 2024-03-06T07:28:19.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:19.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.270+0000 7f75b7a5c640 1 Processor -- start 2024-03-06T07:28:19.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.270+0000 7f75b7a5c640 1 -- start start 2024-03-06T07:28:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b7a5c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75b00c88c0 0x7f75b0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b7a5c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b00c73d0 con 0x7f75b014f360 2024-03-06T07:28:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b7a5c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b00be2e0 con 0x7f75b0158b60 2024-03-06T07:28:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b7a5c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b0159700 con 0x7f75b00c88c0 2024-03-06T07:28:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b4ff7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75b00c88c0 0x7f75b0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b57f8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f75b014f360 0x7f75b00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57714/0 (socket says 172.21.15.26:57714) 2024-03-06T07:28:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b57f8640 1 -- 172.21.15.26:0/411700193 learned_addr learned my addr 172.21.15.26:0/411700193 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:19.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b4ff7640 1 -- 172.21.15.26:0/411700193 >> v1:172.21.15.117:6789/0 conn(0x7f75b0158b60 legacy=0x7f75b0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:19.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75a7fff640 1 -- 172.21.15.26:0/411700193 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134710609 0 0) 0x7f75b00c73d0 con 0x7f75b014f360 2024-03-06T07:28:19.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b4ff7640 1 -- 172.21.15.26:0/411700193 >> v1:172.21.15.26:6789/0 conn(0x7f75b014f360 legacy=0x7f75b00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.271+0000 7f75b4ff7640 1 -- 172.21.15.26:0/411700193 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00c7670 con 0x7f75b00c88c0 2024-03-06T07:28:19.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.272+0000 7f75b4ff7640 1 --2- 172.21.15.26:0/411700193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b0155c10 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f75a00097a0 tx=0x7f75a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b51004bb8aaaa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.272+0000 7f75a7fff640 1 -- 172.21.15.26:0/411700193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0035020 con 0x7f75b00c88c0 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.272+0000 7f75a7fff640 1 -- 172.21.15.26:0/411700193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0031830 con 0x7f75b00c88c0 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.272+0000 7f75a7fff640 1 -- 172.21.15.26:0/411700193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0031c00 con 0x7f75b00c88c0 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.273+0000 7f75b7a5c640 1 -- 172.21.15.26:0/411700193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 msgr2=0x7f75b0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.273+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/411700193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b0155c10 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f75a00097a0 tx=0x7f75a002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.273+0000 7f75b7a5c640 1 -- 172.21.15.26:0/411700193 shutdown_connections 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.273+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/411700193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b0155c10 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.273+0000 7f75b7a5c640 1 -- 172.21.15.26:0/411700193 >> 172.21.15.26:0/411700193 conn(0x7f75b00ac570 msgr2=0x7f75b00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.273+0000 7f75b7a5c640 1 -- 172.21.15.26:0/411700193 shutdown_connections 2024-03-06T07:28:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.273+0000 7f75b7a5c640 1 -- 172.21.15.26:0/411700193 wait complete. 2024-03-06T07:28:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.274+0000 7f75b7a5c640 1 Processor -- start 2024-03-06T07:28:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.274+0000 7f75b7a5c640 1 -- start start 2024-03-06T07:28:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.274+0000 7f75b7a5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b01eb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.274+0000 7f75b7a5c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75b014f360 0x7f75b014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b7a5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75b0158b60 0x7f75b0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b7a5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00c7c30 con 0x7f75b00c88c0 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b7a5c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00c5a50 con 0x7f75b0158b60 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b7a5c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00c5bd0 con 0x7f75b014f360 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b01eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b01eb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43906/0 (socket says 172.21.15.26:43906) 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 -- 172.21.15.26:0/2483893792 learned_addr learned my addr 172.21.15.26:0/2483893792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 -- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75b014f360 msgr2=0x7f75b014f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75b014f360 0x7f75b014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 -- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75b0158b60 msgr2=0x7f75b0151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b5ff9640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75b0158b60 0x7f75b0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75b0158b60 0x7f75b0151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 -- 172.21.15.26:0/2483893792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b014fe40 con 0x7f75b00c88c0 2024-03-06T07:28:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b57f8640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b01eb210 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00c4a0 tx=0x7f75ac00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75a5ffb640 1 -- 172.21.15.26:0/2483893792 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac018070 con 0x7f75b00c88c0 2024-03-06T07:28:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75a5ffb640 1 -- 172.21.15.26:0/2483893792 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac0074e0 con 0x7f75b00c88c0 2024-03-06T07:28:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75a5ffb640 1 -- 172.21.15.26:0/2483893792 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac004a20 con 0x7f75b00c88c0 2024-03-06T07:28:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.275+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b0150120 con 0x7f75b00c88c0 2024-03-06T07:28:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.276+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b01506e0 con 0x7f75b00c88c0 2024-03-06T07:28:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.277+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b014ebc0 con 0x7f75b00c88c0 2024-03-06T07:28:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.277+0000 7f75a5ffb640 1 -- 172.21.15.26:0/2483893792 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75ac01b070 con 0x7f75b00c88c0 2024-03-06T07:28:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.278+0000 7f75a5ffb640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f759406b4d0 0x7f759406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.278+0000 7f75a5ffb640 1 -- 172.21.15.26:0/2483893792 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75ac089670 con 0x7f75b00c88c0 2024-03-06T07:28:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.278+0000 7f75b4ff7640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f759406b4d0 0x7f759406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.278+0000 7f75b4ff7640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f759406b4d0 0x7f759406d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f75a0036000 tx=0x7f75a0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.282+0000 7f75a5ffb640 1 -- 172.21.15.26:0/2483893792 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f75b014ebc0 con 0x7f75b00c88c0 2024-03-06T07:28:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.467+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75b0069460 con 0x7f75b00c88c0 2024-03-06T07:28:19.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.468+0000 7f75a5ffb640 1 -- 172.21.15.26:0/2483893792 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f75b0069460 con 0x7f75b00c88c0 2024-03-06T07:28:19.470 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:19.470 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":198,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:16.787189+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.447072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:14.106103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f759406b4d0 msgr2=0x7f759406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f759406b4d0 0x7f759406d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f75a0036000 tx=0x7f75a0030000 comp rx=0 tx=0).stop 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 msgr2=0x7f75b01eb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b01eb210 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00c4a0 tx=0x7f75ac00c960 comp rx=0 tx=0).stop 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 shutdown_connections 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f759406b4d0 0x7f759406d990 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75b0158b60 0x7f75b0151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75b014f360 0x7f75b014f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 --2- 172.21.15.26:0/2483893792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75b00c88c0 0x7f75b01eb210 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 >> 172.21.15.26:0/2483893792 conn(0x7f75b00ac570 msgr2=0x7f75b014e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 shutdown_connections 2024-03-06T07:28:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.470+0000 7f75b7a5c640 1 -- 172.21.15.26:0/2483893792 wait complete. 2024-03-06T07:28:19.482 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:28:19.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:28:19.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.582+0000 7f1c6369a640 1 Processor -- start 2024-03-06T07:28:19.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.582+0000 7f1c6369a640 1 -- start start 2024-03-06T07:28:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c6369a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c150580 0x7f1c5c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c6369a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1c5c0c6000 0x7f1c5c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c6369a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c153a60 con 0x7f1c5c0c1320 2024-03-06T07:28:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c6369a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c153d40 con 0x7f1c5c0c6000 2024-03-06T07:28:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c6369a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c06c2c0 con 0x7f1c5c150580 2024-03-06T07:28:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61c37640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1c5c0c6000 0x7f1c5c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61c37640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1c5c0c6000 0x7f1c5c06e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43908/0 (socket says 172.21.15.26:43908) 2024-03-06T07:28:19.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61c37640 1 -- 172.21.15.26:0/3804820833 learned_addr learned my addr 172.21.15.26:0/3804820833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:19.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61436640 1 --2- 172.21.15.26:0/3804820833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c150580 0x7f1c5c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61c37640 1 -- 172.21.15.26:0/3804820833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c150580 msgr2=0x7f1c5c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:19.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61c37640 1 --2- 172.21.15.26:0/3804820833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c150580 0x7f1c5c06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61c37640 1 -- 172.21.15.26:0/3804820833 >> v1:172.21.15.117:6789/0 conn(0x7f1c5c0c1320 legacy=0x7f1c5c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.583+0000 7f1c61c37640 1 -- 172.21.15.26:0/3804820833 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c06c5a0 con 0x7f1c5c0c6000 2024-03-06T07:28:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.584+0000 7f1c61c37640 1 --2- 172.21.15.26:0/3804820833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c6000 0x7f1c5c06e250 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009e50 tx=0x7f1c5402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156d7b1837a37637 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c53fff640 1 -- 172.21.15.26:0/3804820833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5402ca70 con 0x7f1c5c0c6000 2024-03-06T07:28:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c53fff640 1 -- 172.21.15.26:0/3804820833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5402cc10 con 0x7f1c5c0c6000 2024-03-06T07:28:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c53fff640 1 -- 172.21.15.26:0/3804820833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540343f0 con 0x7f1c5c0c6000 2024-03-06T07:28:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 -- 172.21.15.26:0/3804820833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c6000 msgr2=0x7f1c5c06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 --2- 172.21.15.26:0/3804820833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c6000 0x7f1c5c06e250 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009e50 tx=0x7f1c5402bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 -- 172.21.15.26:0/3804820833 shutdown_connections 2024-03-06T07:28:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 --2- 172.21.15.26:0/3804820833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c6000 0x7f1c5c06e250 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 --2- 172.21.15.26:0/3804820833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c150580 0x7f1c5c06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 -- 172.21.15.26:0/3804820833 >> 172.21.15.26:0/3804820833 conn(0x7f1c5c0c6710 msgr2=0x7f1c5c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 -- 172.21.15.26:0/3804820833 shutdown_connections 2024-03-06T07:28:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.585+0000 7f1c6369a640 1 -- 172.21.15.26:0/3804820833 wait complete. 2024-03-06T07:28:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.586+0000 7f1c6369a640 1 Processor -- start 2024-03-06T07:28:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.586+0000 7f1c6369a640 1 -- start start 2024-03-06T07:28:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c6369a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c1320 0x7f1c5c070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c6369a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c0c6000 0x7f1c5c0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c6369a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c5c150580 0x7f1c5c1ec620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c6369a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0c0dc0 con 0x7f1c5c0c1320 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c6369a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0c0c40 con 0x7f1c5c150580 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c6369a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c156dc0 con 0x7f1c5c0c6000 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c1320 0x7f1c5c070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c1320 0x7f1c5c070950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43918/0 (socket says 172.21.15.26:43918) 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 -- 172.21.15.26:0/2986974578 learned_addr learned my addr 172.21.15.26:0/2986974578 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c60c35640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c0c6000 0x7f1c5c0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61c37640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c5c150580 0x7f1c5c1ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 -- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c0c6000 msgr2=0x7f1c5c0c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c0c6000 0x7f1c5c0c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 -- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c5c150580 msgr2=0x7f1c5c1ec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c5c150580 0x7f1c5c1ec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.587+0000 7f1c61436640 1 -- 172.21.15.26:0/2986974578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c073410 con 0x7f1c5c0c1320 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.588+0000 7f1c61436640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c1320 0x7f1c5c070950 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e870 tx=0x7f1c5800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.588+0000 7f1c51ffb640 1 -- 172.21.15.26:0/2986974578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58017070 con 0x7f1c5c0c1320 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.588+0000 7f1c51ffb640 1 -- 172.21.15.26:0/2986974578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800c940 con 0x7f1c5c0c1320 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.588+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c072e50 con 0x7f1c5c0c1320 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.588+0000 7f1c51ffb640 1 -- 172.21.15.26:0/2986974578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800cd30 con 0x7f1c5c0c1320 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.588+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c070390 con 0x7f1c5c0c1320 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.589+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c04f330 con 0x7f1c5c0c1320 2024-03-06T07:28:19.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.589+0000 7f1c51ffb640 1 -- 172.21.15.26:0/2986974578 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c5801a070 con 0x7f1c5c0c1320 2024-03-06T07:28:19.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.590+0000 7f1c51ffb640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c4006b4d0 0x7f1c4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.590+0000 7f1c51ffb640 1 -- 172.21.15.26:0/2986974578 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c5808a110 con 0x7f1c5c0c1320 2024-03-06T07:28:19.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.590+0000 7f1c60c35640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c4006b4d0 0x7f1c4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.591+0000 7f1c60c35640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c4006b4d0 0x7f1c4006d990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c14c080 tx=0x7f1c4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.594+0000 7f1c51ffb640 1 -- 172.21.15.26:0/2986974578 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1c58058840 con 0x7f1c5c0c1320 2024-03-06T07:28:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.773+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c5c070730 con 0x7f1c4006b4d0 2024-03-06T07:28:19.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.779+0000 7f1c51ffb640 1 -- 172.21.15.26:0/2986974578 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247174 (secure 0 0 0) 0x7f1c5c070730 con 0x7f1c4006b4d0 2024-03-06T07:28:19.784 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c4006b4d0 msgr2=0x7f1c4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c4006b4d0 0x7f1c4006d990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c14c080 tx=0x7f1c4c009040 comp rx=0 tx=0).stop 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c1320 msgr2=0x7f1c5c070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c1320 0x7f1c5c070950 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e870 tx=0x7f1c5800ed30 comp rx=0 tx=0).stop 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 shutdown_connections 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c4006b4d0 0x7f1c4006d990 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c5c150580 0x7f1c5c1ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c5c0c6000 0x7f1c5c0c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 --2- 172.21.15.26:0/2986974578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c5c0c1320 0x7f1c5c070950 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 >> 172.21.15.26:0/2986974578 conn(0x7f1c5c0c6710 msgr2=0x7f1c5c0c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 shutdown_connections 2024-03-06T07:28:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.782+0000 7f1c6369a640 1 -- 172.21.15.26:0/2986974578 wait complete. 2024-03-06T07:28:19.795 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-03-06T07:28:19.135828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281416,"kb_used_data":19480,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754693304,"statfs":{"total":773094113280,"available":772805943296,"internally_reserved":0,"allocated":19947520,"data_stored":10799078,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.370025"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773969+0000","last_change":"2024-03-06T07:28:14.773969+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773969+0000","last_undegraded":"2024-03-06T07:28:14.773969+0000","last_fullsized":"2024-03-06T07:28:14.773969+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774111+0000","last_change":"2024-03-06T07:28:14.774111+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774111+0000","last_undegraded":"2024-03-06T07:28:14.774111+0000","last_fullsized":"2024-03-06T07:28:14.774111+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776824+0000","last_change":"2024-03-06T07:28:14.776824+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776824+0000","last_undegraded":"2024-03-06T07:28:14.776824+0000","last_fullsized":"2024-03-06T07:28:14.776824+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774117+0000","last_change":"2024-03-06T07:28:14.774117+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774117+0000","last_undegraded":"2024-03-06T07:28:14.774117+0000","last_fullsized":"2024-03-06T07:28:14.774117+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776834+0000","last_change":"2024-03-06T07:28:14.776834+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776834+0000","last_undegraded":"2024-03-06T07:28:14.776834+0000","last_fullsized":"2024-03-06T07:28:14.776834+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773087+0000","last_change":"2024-03-06T07:28:14.773087+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773087+0000","last_undegraded":"2024-03-06T07:28:14.773087+0000","last_fullsized":"2024-03-06T07:28:14.773087+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775879+0000","last_change":"2024-03-06T07:28:14.775879+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775879+0000","last_undegraded":"2024-03-06T07:28:14.775879+0000","last_fullsized":"2024-03-06T07:28:14.775879+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775942+0000","last_change":"2024-03-06T07:28:14.775942+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775942+0000","last_undegraded":"2024-03-06T07:28:14.775942+0000","last_fullsized":"2024-03-06T07:28:14.775942+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773504+0000","last_change":"2024-03-06T07:28:14.773504+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773504+0000","last_undegraded":"2024-03-06T07:28:14.773504+0000","last_fullsized":"2024-03-06T07:28:14.773504+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776672+0000","last_change":"2024-03-06T07:28:14.776672+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776672+0000","last_undegraded":"2024-03-06T07:28:14.776672+0000","last_fullsized":"2024-03-06T07:28:14.776672+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776343+0000","last_change":"2024-03-06T07:28:14.776343+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776343+0000","last_undegraded":"2024-03-06T07:28:14.776343+0000","last_fullsized":"2024-03-06T07:28:14.776343+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772516+0000","last_change":"2024-03-06T07:28:14.772516+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772516+0000","last_undegraded":"2024-03-06T07:28:14.772516+0000","last_fullsized":"2024-03-06T07:28:14.772516+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775564+0000","last_change":"2024-03-06T07:28:14.775564+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775564+0000","last_undegraded":"2024-03-06T07:28:14.775564+0000","last_fullsized":"2024-03-06T07:28:14.775564+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778948+0000","last_change":"2024-03-06T07:28:14.778948+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778948+0000","last_undegraded":"2024-03-06T07:28:14.778948+0000","last_fullsized":"2024-03-06T07:28:14.778948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772911+0000","last_change":"2024-03-06T07:28:14.772911+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772911+0000","last_undegraded":"2024-03-06T07:28:14.772911+0000","last_fullsized":"2024-03-06T07:28:14.772911+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772552+0000","last_change":"2024-03-06T07:28:14.772552+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772552+0000","last_undegraded":"2024-03-06T07:28:14.772552+0000","last_fullsized":"2024-03-06T07:28:14.772552+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777866+0000","last_change":"2024-03-06T07:28:14.777866+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777866+0000","last_undegraded":"2024-03-06T07:28:14.777866+0000","last_fullsized":"2024-03-06T07:28:14.777866+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773906+0000","last_change":"2024-03-06T07:28:14.773906+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773906+0000","last_undegraded":"2024-03-06T07:28:14.773906+0000","last_fullsized":"2024-03-06T07:28:14.773906+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773952+0000","last_change":"2024-03-06T07:28:14.773952+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773952+0000","last_undegraded":"2024-03-06T07:28:14.773952+0000","last_fullsized":"2024-03-06T07:28:14.773952+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772662+0000","last_change":"2024-03-06T07:28:14.772662+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772662+0000","last_undegraded":"2024-03-06T07:28:14.772662+0000","last_fullsized":"2024-03-06T07:28:14.772662+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777224+0000","last_change":"2024-03-06T07:28:14.777224+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777224+0000","last_undegraded":"2024-03-06T07:28:14.777224+0000","last_fullsized":"2024-03-06T07:28:14.777224+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775345+0000","last_change":"2024-03-06T07:28:14.775345+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775345+0000","last_undegraded":"2024-03-06T07:28:14.775345+0000","last_fullsized":"2024-03-06T07:28:14.775345+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772357+0000","last_change":"2024-03-06T07:28:14.772357+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772357+0000","last_undegraded":"2024-03-06T07:28:14.772357+0000","last_fullsized":"2024-03-06T07:28:14.772357+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777328+0000","last_change":"2024-03-06T07:28:14.777328+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777328+0000","last_undegraded":"2024-03-06T07:28:14.777328+0000","last_fullsized":"2024-03-06T07:28:14.777328+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773893+0000","last_change":"2024-03-06T07:28:14.773893+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773893+0000","last_undegraded":"2024-03-06T07:28:14.773893+0000","last_fullsized":"2024-03-06T07:28:14.773893+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778348+0000","last_change":"2024-03-06T07:28:14.778348+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778348+0000","last_undegraded":"2024-03-06T07:28:14.778348+0000","last_fullsized":"2024-03-06T07:28:14.778348+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778695+0000","last_change":"2024-03-06T07:28:14.778695+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778695+0000","last_undegraded":"2024-03-06T07:28:14.778695+0000","last_fullsized":"2024-03-06T07:28:14.778695+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.771877+0000","last_change":"2024-03-06T07:28:14.771877+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.771877+0000","last_undegraded":"2024-03-06T07:28:14.771877+0000","last_fullsized":"2024-03-06T07:28:14.771877+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779909+0000","last_change":"2024-03-06T07:28:14.779909+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779909+0000","last_undegraded":"2024-03-06T07:28:14.779909+0000","last_fullsized":"2024-03-06T07:28:14.779909+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778508+0000","last_change":"2024-03-06T07:28:14.778508+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778508+0000","last_undegraded":"2024-03-06T07:28:14.778508+0000","last_fullsized":"2024-03-06T07:28:14.778508+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773904+0000","last_change":"2024-03-06T07:28:14.773904+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773904+0000","last_undegraded":"2024-03-06T07:28:14.773904+0000","last_fullsized":"2024-03-06T07:28:14.773904+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.771975+0000","last_change":"2024-03-06T07:28:14.771975+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.771975+0000","last_undegraded":"2024-03-06T07:28:14.771975+0000","last_fullsized":"2024-03-06T07:28:14.771975+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773009+0000","last_change":"2024-03-06T07:28:14.773009+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773009+0000","last_undegraded":"2024-03-06T07:28:14.773009+0000","last_fullsized":"2024-03-06T07:28:14.773009+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786785+0000","last_change":"2024-03-06T07:28:14.781972+0000","last_active":"2024-03-06T07:28:14.786785+0000","last_peered":"2024-03-06T07:28:14.786785+0000","last_clean":"2024-03-06T07:28:14.786785+0000","last_became_active":"2024-03-06T07:28:14.781691+0000","last_became_peered":"2024-03-06T07:28:14.781691+0000","last_unstale":"2024-03-06T07:28:14.786785+0000","last_undegraded":"2024-03-06T07:28:14.786785+0000","last_fullsized":"2024-03-06T07:28:14.786785+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:13.953720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772471+0000","last_change":"2024-03-06T07:28:14.772471+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772471+0000","last_undegraded":"2024-03-06T07:28:14.772471+0000","last_fullsized":"2024-03-06T07:28:14.772471+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791749+0000","last_change":"2024-03-06T07:28:14.788442+0000","last_active":"2024-03-06T07:28:14.791749+0000","last_peered":"2024-03-06T07:28:14.791749+0000","last_clean":"2024-03-06T07:28:14.791749+0000","last_became_active":"2024-03-06T07:28:14.788075+0000","last_became_peered":"2024-03-06T07:28:14.788075+0000","last_unstale":"2024-03-06T07:28:14.791749+0000","last_undegraded":"2024-03-06T07:28:14.791749+0000","last_fullsized":"2024-03-06T07:28:14.791749+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:42.385844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776275+0000","last_change":"2024-03-06T07:28:14.776275+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776275+0000","last_undegraded":"2024-03-06T07:28:14.776275+0000","last_fullsized":"2024-03-06T07:28:14.776275+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.792515+0000","last_change":"2024-03-06T07:28:14.791895+0000","last_active":"2024-03-06T07:28:14.792515+0000","last_peered":"2024-03-06T07:28:14.792515+0000","last_clean":"2024-03-06T07:28:14.792515+0000","last_became_active":"2024-03-06T07:28:14.791606+0000","last_became_peered":"2024-03-06T07:28:14.791606+0000","last_unstale":"2024-03-06T07:28:14.792515+0000","last_undegraded":"2024-03-06T07:28:14.792515+0000","last_fullsized":"2024-03-06T07:28:14.792515+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:02.151765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.781725+0000","last_change":"2024-03-06T07:28:14.781725+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.781725+0000","last_undegraded":"2024-03-06T07:28:14.781725+0000","last_fullsized":"2024-03-06T07:28:14.781725+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.785659+0000","last_change":"2024-03-06T07:28:14.783699+0000","last_active":"2024-03-06T07:28:14.785659+0000","last_peered":"2024-03-06T07:28:14.785659+0000","last_clean":"2024-03-06T07:28:14.785659+0000","last_became_active":"2024-03-06T07:28:14.783452+0000","last_became_peered":"2024-03-06T07:28:14.783452+0000","last_unstale":"2024-03-06T07:28:14.785659+0000","last_undegraded":"2024-03-06T07:28:14.785659+0000","last_fullsized":"2024-03-06T07:28:14.785659+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:41.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776389+0000","last_change":"2024-03-06T07:28:14.776389+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776389+0000","last_undegraded":"2024-03-06T07:28:14.776389+0000","last_fullsized":"2024-03-06T07:28:14.776389+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789132+0000","last_change":"2024-03-06T07:28:14.786831+0000","last_active":"2024-03-06T07:28:14.789132+0000","last_peered":"2024-03-06T07:28:14.789132+0000","last_clean":"2024-03-06T07:28:14.789132+0000","last_became_active":"2024-03-06T07:28:14.786548+0000","last_became_peered":"2024-03-06T07:28:14.786548+0000","last_unstale":"2024-03-06T07:28:14.789132+0000","last_undegraded":"2024-03-06T07:28:14.789132+0000","last_fullsized":"2024-03-06T07:28:14.789132+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:25:57.094067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777088+0000","last_change":"2024-03-06T07:28:14.777088+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777088+0000","last_undegraded":"2024-03-06T07:28:14.777088+0000","last_fullsized":"2024-03-06T07:28:14.777088+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789637+0000","last_change":"2024-03-06T07:28:14.782671+0000","last_active":"2024-03-06T07:28:14.789637+0000","last_peered":"2024-03-06T07:28:14.789637+0000","last_clean":"2024-03-06T07:28:14.789637+0000","last_became_active":"2024-03-06T07:28:14.782382+0000","last_became_peered":"2024-03-06T07:28:14.782382+0000","last_unstale":"2024-03-06T07:28:14.789637+0000","last_undegraded":"2024-03-06T07:28:14.789637+0000","last_fullsized":"2024-03-06T07:28:14.789637+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:07.331967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778436+0000","last_change":"2024-03-06T07:28:14.778436+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778436+0000","last_undegraded":"2024-03-06T07:28:14.778436+0000","last_fullsized":"2024-03-06T07:28:14.778436+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.777796+0000","last_change":"2024-03-06T07:28:14.776124+0000","last_active":"2024-03-06T07:28:14.777796+0000","last_peered":"2024-03-06T07:28:14.777796+0000","last_clean":"2024-03-06T07:28:14.777796+0000","last_became_active":"2024-03-06T07:28:14.775819+0000","last_became_peered":"2024-03-06T07:28:14.775819+0000","last_unstale":"2024-03-06T07:28:14.777796+0000","last_undegraded":"2024-03-06T07:28:14.777796+0000","last_fullsized":"2024-03-06T07:28:14.777796+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:30:20.528287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779244+0000","last_change":"2024-03-06T07:28:14.779244+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779244+0000","last_undegraded":"2024-03-06T07:28:14.779244+0000","last_fullsized":"2024-03-06T07:28:14.779244+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790821+0000","last_change":"2024-03-06T07:28:14.789097+0000","last_active":"2024-03-06T07:28:14.790821+0000","last_peered":"2024-03-06T07:28:14.790821+0000","last_clean":"2024-03-06T07:28:14.790821+0000","last_became_active":"2024-03-06T07:28:14.788833+0000","last_became_peered":"2024-03-06T07:28:14.788833+0000","last_unstale":"2024-03-06T07:28:14.790821+0000","last_undegraded":"2024-03-06T07:28:14.790821+0000","last_fullsized":"2024-03-06T07:28:14.790821+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:14.725218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779004+0000","last_change":"2024-03-06T07:28:14.779004+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779004+0000","last_undegraded":"2024-03-06T07:28:14.779004+0000","last_fullsized":"2024-03-06T07:28:14.779004+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789611+0000","last_change":"2024-03-06T07:28:14.787649+0000","last_active":"2024-03-06T07:28:14.789611+0000","last_peered":"2024-03-06T07:28:14.789611+0000","last_clean":"2024-03-06T07:28:14.789611+0000","last_became_active":"2024-03-06T07:28:14.787348+0000","last_became_peered":"2024-03-06T07:28:14.787348+0000","last_unstale":"2024-03-06T07:28:14.789611+0000","last_undegraded":"2024-03-06T07:28:14.789611+0000","last_fullsized":"2024-03-06T07:28:14.789611+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:38:13.142714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773103+0000","last_change":"2024-03-06T07:28:14.773103+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773103+0000","last_undegraded":"2024-03-06T07:28:14.773103+0000","last_fullsized":"2024-03-06T07:28:14.773103+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787342+0000","last_change":"2024-03-06T07:28:14.785132+0000","last_active":"2024-03-06T07:28:14.787342+0000","last_peered":"2024-03-06T07:28:14.787342+0000","last_clean":"2024-03-06T07:28:14.787342+0000","last_became_active":"2024-03-06T07:28:14.784849+0000","last_became_peered":"2024-03-06T07:28:14.784849+0000","last_unstale":"2024-03-06T07:28:14.787342+0000","last_undegraded":"2024-03-06T07:28:14.787342+0000","last_fullsized":"2024-03-06T07:28:14.787342+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:14:41.231832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777607+0000","last_change":"2024-03-06T07:28:14.777607+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777607+0000","last_undegraded":"2024-03-06T07:28:14.777607+0000","last_fullsized":"2024-03-06T07:28:14.777607+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786919+0000","last_change":"2024-03-06T07:28:14.785822+0000","last_active":"2024-03-06T07:28:14.786919+0000","last_peered":"2024-03-06T07:28:14.786919+0000","last_clean":"2024-03-06T07:28:14.786919+0000","last_became_active":"2024-03-06T07:28:14.785540+0000","last_became_peered":"2024-03-06T07:28:14.785540+0000","last_unstale":"2024-03-06T07:28:14.786919+0000","last_undegraded":"2024-03-06T07:28:14.786919+0000","last_fullsized":"2024-03-06T07:28:14.786919+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:03.928508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773152+0000","last_change":"2024-03-06T07:28:14.773152+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773152+0000","last_undegraded":"2024-03-06T07:28:14.773152+0000","last_fullsized":"2024-03-06T07:28:14.773152+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.785679+0000","last_change":"2024-03-06T07:28:14.784265+0000","last_active":"2024-03-06T07:28:14.785679+0000","last_peered":"2024-03-06T07:28:14.785679+0000","last_clean":"2024-03-06T07:28:14.785679+0000","last_became_active":"2024-03-06T07:28:14.783985+0000","last_became_peered":"2024-03-06T07:28:14.783985+0000","last_unstale":"2024-03-06T07:28:14.785679+0000","last_undegraded":"2024-03-06T07:28:14.785679+0000","last_fullsized":"2024-03-06T07:28:14.785679+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.428223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776788+0000","last_change":"2024-03-06T07:28:14.776788+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776788+0000","last_undegraded":"2024-03-06T07:28:14.776788+0000","last_fullsized":"2024-03-06T07:28:14.776788+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787370+0000","last_change":"2024-03-06T07:28:14.784209+0000","last_active":"2024-03-06T07:28:14.787370+0000","last_peered":"2024-03-06T07:28:14.787370+0000","last_clean":"2024-03-06T07:28:14.787370+0000","last_became_active":"2024-03-06T07:28:14.783932+0000","last_became_peered":"2024-03-06T07:28:14.783932+0000","last_unstale":"2024-03-06T07:28:14.787370+0000","last_undegraded":"2024-03-06T07:28:14.787370+0000","last_fullsized":"2024-03-06T07:28:14.787370+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:35.222311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773068+0000","last_change":"2024-03-06T07:28:14.773068+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773068+0000","last_undegraded":"2024-03-06T07:28:14.773068+0000","last_fullsized":"2024-03-06T07:28:14.773068+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791974+0000","last_change":"2024-03-06T07:28:14.787534+0000","last_active":"2024-03-06T07:28:14.791974+0000","last_peered":"2024-03-06T07:28:14.791974+0000","last_clean":"2024-03-06T07:28:14.791974+0000","last_became_active":"2024-03-06T07:28:14.787286+0000","last_became_peered":"2024-03-06T07:28:14.787286+0000","last_unstale":"2024-03-06T07:28:14.791974+0000","last_undegraded":"2024-03-06T07:28:14.791974+0000","last_fullsized":"2024-03-06T07:28:14.791974+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:58.265777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778848+0000","last_change":"2024-03-06T07:28:14.778848+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778848+0000","last_undegraded":"2024-03-06T07:28:14.778848+0000","last_fullsized":"2024-03-06T07:28:14.778848+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791451+0000","last_change":"2024-03-06T07:28:14.788682+0000","last_active":"2024-03-06T07:28:14.791451+0000","last_peered":"2024-03-06T07:28:14.791451+0000","last_clean":"2024-03-06T07:28:14.791451+0000","last_became_active":"2024-03-06T07:28:14.788381+0000","last_became_peered":"2024-03-06T07:28:14.788381+0000","last_unstale":"2024-03-06T07:28:14.791451+0000","last_undegraded":"2024-03-06T07:28:14.791451+0000","last_fullsized":"2024-03-06T07:28:14.791451+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:23:05.687550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776586+0000","last_change":"2024-03-06T07:28:14.776586+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776586+0000","last_undegraded":"2024-03-06T07:28:14.776586+0000","last_fullsized":"2024-03-06T07:28:14.776586+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789078+0000","last_change":"2024-03-06T07:28:14.787636+0000","last_active":"2024-03-06T07:28:14.789078+0000","last_peered":"2024-03-06T07:28:14.789078+0000","last_clean":"2024-03-06T07:28:14.789078+0000","last_became_active":"2024-03-06T07:28:14.787367+0000","last_became_peered":"2024-03-06T07:28:14.787367+0000","last_unstale":"2024-03-06T07:28:14.789078+0000","last_undegraded":"2024-03-06T07:28:14.789078+0000","last_fullsized":"2024-03-06T07:28:14.789078+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:21.492957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777633+0000","last_change":"2024-03-06T07:28:14.777633+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777633+0000","last_undegraded":"2024-03-06T07:28:14.777633+0000","last_fullsized":"2024-03-06T07:28:14.777633+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787834+0000","last_change":"2024-03-06T07:28:14.787083+0000","last_active":"2024-03-06T07:28:14.787834+0000","last_peered":"2024-03-06T07:28:14.787834+0000","last_clean":"2024-03-06T07:28:14.787834+0000","last_became_active":"2024-03-06T07:28:14.786827+0000","last_became_peered":"2024-03-06T07:28:14.786827+0000","last_unstale":"2024-03-06T07:28:14.787834+0000","last_undegraded":"2024-03-06T07:28:14.787834+0000","last_fullsized":"2024-03-06T07:28:14.787834+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:15.361243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774343+0000","last_change":"2024-03-06T07:28:14.774343+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774343+0000","last_undegraded":"2024-03-06T07:28:14.774343+0000","last_fullsized":"2024-03-06T07:28:14.774343+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791723+0000","last_change":"2024-03-06T07:28:14.788050+0000","last_active":"2024-03-06T07:28:14.791723+0000","last_peered":"2024-03-06T07:28:14.791723+0000","last_clean":"2024-03-06T07:28:14.791723+0000","last_became_active":"2024-03-06T07:28:14.787770+0000","last_became_peered":"2024-03-06T07:28:14.787770+0000","last_unstale":"2024-03-06T07:28:14.791723+0000","last_undegraded":"2024-03-06T07:28:14.791723+0000","last_fullsized":"2024-03-06T07:28:14.791723+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:42.511473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773686+0000","last_change":"2024-03-06T07:28:14.773686+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773686+0000","last_undegraded":"2024-03-06T07:28:14.773686+0000","last_fullsized":"2024-03-06T07:28:14.773686+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.783299+0000","last_change":"2024-03-06T07:28:14.782491+0000","last_active":"2024-03-06T07:28:14.783299+0000","last_peered":"2024-03-06T07:28:14.783299+0000","last_clean":"2024-03-06T07:28:14.783299+0000","last_became_active":"2024-03-06T07:28:14.782223+0000","last_became_peered":"2024-03-06T07:28:14.782223+0000","last_unstale":"2024-03-06T07:28:14.783299+0000","last_undegraded":"2024-03-06T07:28:14.783299+0000","last_fullsized":"2024-03-06T07:28:14.783299+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:55.179141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775653+0000","last_change":"2024-03-06T07:28:14.775653+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775653+0000","last_undegraded":"2024-03-06T07:28:14.775653+0000","last_fullsized":"2024-03-06T07:28:14.775653+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790941+0000","last_change":"2024-03-06T07:28:14.785927+0000","last_active":"2024-03-06T07:28:14.790941+0000","last_peered":"2024-03-06T07:28:14.790941+0000","last_clean":"2024-03-06T07:28:14.790941+0000","last_became_active":"2024-03-06T07:28:14.785666+0000","last_became_peered":"2024-03-06T07:28:14.785666+0000","last_unstale":"2024-03-06T07:28:14.790941+0000","last_undegraded":"2024-03-06T07:28:14.790941+0000","last_fullsized":"2024-03-06T07:28:14.790941+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.036054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773460+0000","last_change":"2024-03-06T07:28:14.773460+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773460+0000","last_undegraded":"2024-03-06T07:28:14.773460+0000","last_fullsized":"2024-03-06T07:28:14.773460+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791763+0000","last_change":"2024-03-06T07:28:14.788414+0000","last_active":"2024-03-06T07:28:14.791763+0000","last_peered":"2024-03-06T07:28:14.791763+0000","last_clean":"2024-03-06T07:28:14.791763+0000","last_became_active":"2024-03-06T07:28:14.788047+0000","last_became_peered":"2024-03-06T07:28:14.788047+0000","last_unstale":"2024-03-06T07:28:14.791763+0000","last_undegraded":"2024-03-06T07:28:14.791763+0000","last_fullsized":"2024-03-06T07:28:14.791763+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:46.615680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774832+0000","last_change":"2024-03-06T07:28:14.774832+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774832+0000","last_undegraded":"2024-03-06T07:28:14.774832+0000","last_fullsized":"2024-03-06T07:28:14.774832+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787528+0000","last_change":"2024-03-06T07:28:14.782233+0000","last_active":"2024-03-06T07:28:14.787528+0000","last_peered":"2024-03-06T07:28:14.787528+0000","last_clean":"2024-03-06T07:28:14.787528+0000","last_became_active":"2024-03-06T07:28:14.781927+0000","last_became_peered":"2024-03-06T07:28:14.781927+0000","last_unstale":"2024-03-06T07:28:14.787528+0000","last_undegraded":"2024-03-06T07:28:14.787528+0000","last_fullsized":"2024-03-06T07:28:14.787528+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:48.443105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779417+0000","last_change":"2024-03-06T07:28:14.779417+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779417+0000","last_undegraded":"2024-03-06T07:28:14.779417+0000","last_fullsized":"2024-03-06T07:28:14.779417+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787170+0000","last_change":"2024-03-06T07:28:14.785960+0000","last_active":"2024-03-06T07:28:14.787170+0000","last_peered":"2024-03-06T07:28:14.787170+0000","last_clean":"2024-03-06T07:28:14.787170+0000","last_became_active":"2024-03-06T07:28:14.785692+0000","last_became_peered":"2024-03-06T07:28:14.785692+0000","last_unstale":"2024-03-06T07:28:14.787170+0000","last_undegraded":"2024-03-06T07:28:14.787170+0000","last_fullsized":"2024-03-06T07:28:14.787170+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:04.218118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779640+0000","last_change":"2024-03-06T07:28:14.779640+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779640+0000","last_undegraded":"2024-03-06T07:28:14.779640+0000","last_fullsized":"2024-03-06T07:28:14.779640+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789580+0000","last_change":"2024-03-06T07:28:14.784580+0000","last_active":"2024-03-06T07:28:14.789580+0000","last_peered":"2024-03-06T07:28:14.789580+0000","last_clean":"2024-03-06T07:28:14.789580+0000","last_became_active":"2024-03-06T07:28:14.784245+0000","last_became_peered":"2024-03-06T07:28:14.784245+0000","last_unstale":"2024-03-06T07:28:14.789580+0000","last_undegraded":"2024-03-06T07:28:14.789580+0000","last_fullsized":"2024-03-06T07:28:14.789580+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:10.104421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773331+0000","last_change":"2024-03-06T07:28:14.773331+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773331+0000","last_undegraded":"2024-03-06T07:28:14.773331+0000","last_fullsized":"2024-03-06T07:28:14.773331+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789348+0000","last_change":"2024-03-06T07:28:14.786720+0000","last_active":"2024-03-06T07:28:14.789348+0000","last_peered":"2024-03-06T07:28:14.789348+0000","last_clean":"2024-03-06T07:28:14.789348+0000","last_became_active":"2024-03-06T07:28:14.786452+0000","last_became_peered":"2024-03-06T07:28:14.786452+0000","last_unstale":"2024-03-06T07:28:14.789348+0000","last_undegraded":"2024-03-06T07:28:14.789348+0000","last_fullsized":"2024-03-06T07:28:14.789348+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:51.046635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773547+0000","last_change":"2024-03-06T07:28:14.773547+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773547+0000","last_undegraded":"2024-03-06T07:28:14.773547+0000","last_fullsized":"2024-03-06T07:28:14.773547+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790155+0000","last_change":"2024-03-06T07:28:14.789522+0000","last_active":"2024-03-06T07:28:14.790155+0000","last_peered":"2024-03-06T07:28:14.790155+0000","last_clean":"2024-03-06T07:28:14.790155+0000","last_became_active":"2024-03-06T07:28:14.789267+0000","last_became_peered":"2024-03-06T07:28:14.789267+0000","last_unstale":"2024-03-06T07:28:14.790155+0000","last_undegraded":"2024-03-06T07:28:14.790155+0000","last_fullsized":"2024-03-06T07:28:14.790155+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:41.428236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777263+0000","last_change":"2024-03-06T07:28:14.777263+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777263+0000","last_undegraded":"2024-03-06T07:28:14.777263+0000","last_fullsized":"2024-03-06T07:28:14.777263+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789903+0000","last_change":"2024-03-06T07:28:14.788385+0000","last_active":"2024-03-06T07:28:14.789903+0000","last_peered":"2024-03-06T07:28:14.789903+0000","last_clean":"2024-03-06T07:28:14.789903+0000","last_became_active":"2024-03-06T07:28:14.788113+0000","last_became_peered":"2024-03-06T07:28:14.788113+0000","last_unstale":"2024-03-06T07:28:14.789903+0000","last_undegraded":"2024-03-06T07:28:14.789903+0000","last_fullsized":"2024-03-06T07:28:14.789903+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:50.463171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.782151+0000","last_change":"2024-03-06T07:28:14.782151+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.782151+0000","last_undegraded":"2024-03-06T07:28:14.782151+0000","last_fullsized":"2024-03-06T07:28:14.782151+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.794736+0000","last_change":"2024-03-06T07:28:14.792256+0000","last_active":"2024-03-06T07:28:14.794736+0000","last_peered":"2024-03-06T07:28:14.794736+0000","last_clean":"2024-03-06T07:28:14.794736+0000","last_became_active":"2024-03-06T07:28:14.792032+0000","last_became_peered":"2024-03-06T07:28:14.792032+0000","last_unstale":"2024-03-06T07:28:14.794736+0000","last_undegraded":"2024-03-06T07:28:14.794736+0000","last_fullsized":"2024-03-06T07:28:14.794736+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:20.997172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775584+0000","last_change":"2024-03-06T07:28:14.775584+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775584+0000","last_undegraded":"2024-03-06T07:28:14.775584+0000","last_fullsized":"2024-03-06T07:28:14.775584+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.792007+0000","last_change":"2024-03-06T07:28:14.788477+0000","last_active":"2024-03-06T07:28:14.792007+0000","last_peered":"2024-03-06T07:28:14.792007+0000","last_clean":"2024-03-06T07:28:14.792007+0000","last_became_active":"2024-03-06T07:28:14.788223+0000","last_became_peered":"2024-03-06T07:28:14.788223+0000","last_unstale":"2024-03-06T07:28:14.792007+0000","last_undegraded":"2024-03-06T07:28:14.792007+0000","last_fullsized":"2024-03-06T07:28:14.792007+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:52.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773666+0000","last_change":"2024-03-06T07:28:14.773666+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773666+0000","last_undegraded":"2024-03-06T07:28:14.773666+0000","last_fullsized":"2024-03-06T07:28:14.773666+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789661+0000","last_change":"2024-03-06T07:28:14.788356+0000","last_active":"2024-03-06T07:28:14.789661+0000","last_peered":"2024-03-06T07:28:14.789661+0000","last_clean":"2024-03-06T07:28:14.789661+0000","last_became_active":"2024-03-06T07:28:14.788093+0000","last_became_peered":"2024-03-06T07:28:14.788093+0000","last_unstale":"2024-03-06T07:28:14.789661+0000","last_undegraded":"2024-03-06T07:28:14.789661+0000","last_fullsized":"2024-03-06T07:28:14.789661+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:31.566144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.780498+0000","last_change":"2024-03-06T07:28:14.780498+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.780498+0000","last_undegraded":"2024-03-06T07:28:14.780498+0000","last_fullsized":"2024-03-06T07:28:14.780498+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789297+0000","last_change":"2024-03-06T07:28:14.786359+0000","last_active":"2024-03-06T07:28:14.789297+0000","last_peered":"2024-03-06T07:28:14.789297+0000","last_clean":"2024-03-06T07:28:14.789297+0000","last_became_active":"2024-03-06T07:28:14.786068+0000","last_became_peered":"2024-03-06T07:28:14.786068+0000","last_unstale":"2024-03-06T07:28:14.789297+0000","last_undegraded":"2024-03-06T07:28:14.789297+0000","last_fullsized":"2024-03-06T07:28:14.789297+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:10.545981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775080+0000","last_change":"2024-03-06T07:28:14.775080+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775080+0000","last_undegraded":"2024-03-06T07:28:14.775080+0000","last_fullsized":"2024-03-06T07:28:14.775080+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786611+0000","last_change":"2024-03-06T07:28:14.785657+0000","last_active":"2024-03-06T07:28:14.786611+0000","last_peered":"2024-03-06T07:28:14.786611+0000","last_clean":"2024-03-06T07:28:14.786611+0000","last_became_active":"2024-03-06T07:28:14.785292+0000","last_became_peered":"2024-03-06T07:28:14.785292+0000","last_unstale":"2024-03-06T07:28:14.786611+0000","last_undegraded":"2024-03-06T07:28:14.786611+0000","last_fullsized":"2024-03-06T07:28:14.786611+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:21.053100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":498,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.774290+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:28:14.774290+0000","last_peered":"2024-03-06T07:28:14.774290+0000","last_clean":"2024-03-06T07:28:14.774290+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:28:14.774290+0000","last_undegraded":"2024-03-06T07:28:14.774290+0000","last_fullsized":"2024-03-06T07:28:14.774290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2539520,"data_stored":1790994,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33612,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338228,"statfs":{"total":96636764160,"available":96602345472,"internally_reserved":0,"allocated":2486272,"data_stored":1207293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33612,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338228,"statfs":{"total":96636764160,"available":96602345472,"internally_reserved":0,"allocated":2486272,"data_stored":1207293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37764,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334076,"statfs":{"total":96636764160,"available":96598093824,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2539520,"data_stored":1790994,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:28:19.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:28:19.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.936+0000 7f508b556640 1 Processor -- start 2024-03-06T07:28:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.936+0000 7f508b556640 1 -- start start 2024-03-06T07:28:19.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.937+0000 7f508b556640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f508414f360 0x7f5084155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.937+0000 7f508b556640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5084158b60 0x7f5084158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.937+0000 7f508b556640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50840c73d0 con 0x7f50840c88c0 2024-03-06T07:28:19.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.937+0000 7f508b556640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840c8dd0 con 0x7f5084158b60 2024-03-06T07:28:19.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.937+0000 7f508b556640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840be2e0 con 0x7f508414f360 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.937+0000 7f5089af3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5084158b60 0x7f5084158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.937+0000 7f5089af3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5084158b60 0x7f5084158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43920/0 (socket says 172.21.15.26:43920) 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f5089af3640 1 -- 172.21.15.26:0/1357876864 learned_addr learned my addr 172.21.15.26:0/1357876864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f50892f2640 1 --2- 172.21.15.26:0/1357876864 >> v2:172.21.15.117:3300/0 conn(0x7f508414f360 0x7f5084155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f5089af3640 1 -- 172.21.15.26:0/1357876864 >> v2:172.21.15.117:3300/0 conn(0x7f508414f360 msgr2=0x7f5084155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f5089af3640 1 --2- 172.21.15.26:0/1357876864 >> v2:172.21.15.117:3300/0 conn(0x7f508414f360 0x7f5084155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f5089af3640 1 -- 172.21.15.26:0/1357876864 >> v1:172.21.15.117:6789/0 conn(0x7f50840c88c0 legacy=0x7f50840c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f5089af3640 1 -- 172.21.15.26:0/1357876864 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840c7670 con 0x7f5084158b60 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f50892f2640 1 --2- 172.21.15.26:0/1357876864 >> v2:172.21.15.117:3300/0 conn(0x7f508414f360 0x7f5084155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.938+0000 7f5089af3640 1 --2- 172.21.15.26:0/1357876864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f5084158ff0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f508408c840 tx=0x7f507c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f915aa6e167fae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f507bfff640 1 -- 172.21.15.26:0/1357876864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c02ca80 con 0x7f5084158b60 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f507bfff640 1 -- 172.21.15.26:0/1357876864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c02cc20 con 0x7f5084158b60 2024-03-06T07:28:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f507bfff640 1 -- 172.21.15.26:0/1357876864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0343f0 con 0x7f5084158b60 2024-03-06T07:28:19.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f508b556640 1 -- 172.21.15.26:0/1357876864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 msgr2=0x7f5084158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f508b556640 1 --2- 172.21.15.26:0/1357876864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f5084158ff0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f508408c840 tx=0x7f507c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f508b556640 1 -- 172.21.15.26:0/1357876864 shutdown_connections 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f508b556640 1 --2- 172.21.15.26:0/1357876864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f5084158ff0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f508b556640 1 --2- 172.21.15.26:0/1357876864 >> v2:172.21.15.117:3300/0 conn(0x7f508414f360 0x7f5084155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.939+0000 7f508b556640 1 -- 172.21.15.26:0/1357876864 >> 172.21.15.26:0/1357876864 conn(0x7f50840ac570 msgr2=0x7f50840ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.940+0000 7f508b556640 1 -- 172.21.15.26:0/1357876864 shutdown_connections 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.940+0000 7f508b556640 1 -- 172.21.15.26:0/1357876864 wait complete. 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.941+0000 7f508b556640 1 Processor -- start 2024-03-06T07:28:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.941+0000 7f508b556640 1 -- start start 2024-03-06T07:28:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.942+0000 7f508b556640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50840c88c0 0x7f50840bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.942+0000 7f508b556640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f508414f360 0x7f508414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.942+0000 7f508b556640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f50840c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.942+0000 7f508b556640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840c5a50 con 0x7f5084158b60 2024-03-06T07:28:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.942+0000 7f508b556640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840c7c30 con 0x7f50840c88c0 2024-03-06T07:28:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.942+0000 7f508b556640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840c5bd0 con 0x7f508414f360 2024-03-06T07:28:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f50840c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f50840c2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43924/0 (socket says 172.21.15.26:43924) 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 -- 172.21.15.26:0/2106419696 learned_addr learned my addr 172.21.15.26:0/2106419696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f50892f2640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50840c88c0 0x7f50840bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5088af1640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f508414f360 0x7f508414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 -- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f508414f360 msgr2=0x7f508414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f508414f360 0x7f508414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 -- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50840c88c0 msgr2=0x7f50840bfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50840c88c0 0x7f50840bfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 -- 172.21.15.26:0/2106419696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840bf240 con 0x7f5084158b60 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5089af3640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f50840c2660 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f507c036040 tx=0x7f507c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5079ffb640 1 -- 172.21.15.26:0/2106419696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c043020 con 0x7f5084158b60 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5079ffb640 1 -- 172.21.15.26:0/2106419696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c03e450 con 0x7f5084158b60 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f5079ffb640 1 -- 172.21.15.26:0/2106419696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c009510 con 0x7f5084158b60 2024-03-06T07:28:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.943+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50841ef0b0 con 0x7f5084158b60 2024-03-06T07:28:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.944+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50840bf7c0 con 0x7f5084158b60 2024-03-06T07:28:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.944+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508404f330 con 0x7f5084158b60 2024-03-06T07:28:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.945+0000 7f5079ffb640 1 -- 172.21.15.26:0/2106419696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f507c046020 con 0x7f5084158b60 2024-03-06T07:28:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.945+0000 7f5079ffb640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f506806b4d0 0x7f506806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.945+0000 7f5079ffb640 1 -- 172.21.15.26:0/2106419696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f507c0ac9e0 con 0x7f5084158b60 2024-03-06T07:28:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.945+0000 7f50892f2640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f506806b4d0 0x7f506806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.946+0000 7f50892f2640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f506806b4d0 0x7f506806d990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5080006fd0 tx=0x7f5080008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:19.950+0000 7f5079ffb640 1 -- 172.21.15.26:0/2106419696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f507c04d030 con 0x7f5084158b60 2024-03-06T07:28:20.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.128+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f508414c200 con 0x7f506806b4d0 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.135+0000 7f5079ffb640 1 -- 172.21.15.26:0/2106419696 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247174 (secure 0 0 0) 0x7f508000b720 con 0x7f506806b4d0 2024-03-06T07:28:20.139 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:20.140 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:28:20.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f506806b4d0 msgr2=0x7f506806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f506806b4d0 0x7f506806d990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5080006fd0 tx=0x7f5080008040 comp rx=0 tx=0).stop 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 msgr2=0x7f50840c2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f50840c2660 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f507c036040 tx=0x7f507c004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 shutdown_connections 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f506806b4d0 0x7f506806d990 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084158b60 0x7f50840c2660 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f508414f360 0x7f508414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 --2- 172.21.15.26:0/2106419696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50840c88c0 0x7f50840bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 >> 172.21.15.26:0/2106419696 conn(0x7f50840ac570 msgr2=0x7f50840ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 shutdown_connections 2024-03-06T07:28:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.138+0000 7f508b556640 1 -- 172.21.15.26:0/2106419696 wait complete. 2024-03-06T07:28:20.154 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-03-06T07:28:19.135828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281416,"kb_used_data":19480,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754693304,"statfs":{"total":773094113280,"available":772805943296,"internally_reserved":0,"allocated":19947520,"data_stored":10799078,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.370025"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773969+0000","last_change":"2024-03-06T07:28:14.773969+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773969+0000","last_undegraded":"2024-03-06T07:28:14.773969+0000","last_fullsized":"2024-03-06T07:28:14.773969+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774111+0000","last_change":"2024-03-06T07:28:14.774111+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774111+0000","last_undegraded":"2024-03-06T07:28:14.774111+0000","last_fullsized":"2024-03-06T07:28:14.774111+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776824+0000","last_change":"2024-03-06T07:28:14.776824+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776824+0000","last_undegraded":"2024-03-06T07:28:14.776824+0000","last_fullsized":"2024-03-06T07:28:14.776824+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774117+0000","last_change":"2024-03-06T07:28:14.774117+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774117+0000","last_undegraded":"2024-03-06T07:28:14.774117+0000","last_fullsized":"2024-03-06T07:28:14.774117+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776834+0000","last_change":"2024-03-06T07:28:14.776834+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776834+0000","last_undegraded":"2024-03-06T07:28:14.776834+0000","last_fullsized":"2024-03-06T07:28:14.776834+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773087+0000","last_change":"2024-03-06T07:28:14.773087+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773087+0000","last_undegraded":"2024-03-06T07:28:14.773087+0000","last_fullsized":"2024-03-06T07:28:14.773087+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775879+0000","last_change":"2024-03-06T07:28:14.775879+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775879+0000","last_undegraded":"2024-03-06T07:28:14.775879+0000","last_fullsized":"2024-03-06T07:28:14.775879+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775942+0000","last_change":"2024-03-06T07:28:14.775942+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775942+0000","last_undegraded":"2024-03-06T07:28:14.775942+0000","last_fullsized":"2024-03-06T07:28:14.775942+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773504+0000","last_change":"2024-03-06T07:28:14.773504+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773504+0000","last_undegraded":"2024-03-06T07:28:14.773504+0000","last_fullsized":"2024-03-06T07:28:14.773504+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776672+0000","last_change":"2024-03-06T07:28:14.776672+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776672+0000","last_undegraded":"2024-03-06T07:28:14.776672+0000","last_fullsized":"2024-03-06T07:28:14.776672+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776343+0000","last_change":"2024-03-06T07:28:14.776343+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776343+0000","last_undegraded":"2024-03-06T07:28:14.776343+0000","last_fullsized":"2024-03-06T07:28:14.776343+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772516+0000","last_change":"2024-03-06T07:28:14.772516+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772516+0000","last_undegraded":"2024-03-06T07:28:14.772516+0000","last_fullsized":"2024-03-06T07:28:14.772516+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775564+0000","last_change":"2024-03-06T07:28:14.775564+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775564+0000","last_undegraded":"2024-03-06T07:28:14.775564+0000","last_fullsized":"2024-03-06T07:28:14.775564+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778948+0000","last_change":"2024-03-06T07:28:14.778948+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778948+0000","last_undegraded":"2024-03-06T07:28:14.778948+0000","last_fullsized":"2024-03-06T07:28:14.778948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772911+0000","last_change":"2024-03-06T07:28:14.772911+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772911+0000","last_undegraded":"2024-03-06T07:28:14.772911+0000","last_fullsized":"2024-03-06T07:28:14.772911+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772552+0000","last_change":"2024-03-06T07:28:14.772552+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772552+0000","last_undegraded":"2024-03-06T07:28:14.772552+0000","last_fullsized":"2024-03-06T07:28:14.772552+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777866+0000","last_change":"2024-03-06T07:28:14.777866+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777866+0000","last_undegraded":"2024-03-06T07:28:14.777866+0000","last_fullsized":"2024-03-06T07:28:14.777866+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773906+0000","last_change":"2024-03-06T07:28:14.773906+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773906+0000","last_undegraded":"2024-03-06T07:28:14.773906+0000","last_fullsized":"2024-03-06T07:28:14.773906+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773952+0000","last_change":"2024-03-06T07:28:14.773952+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773952+0000","last_undegraded":"2024-03-06T07:28:14.773952+0000","last_fullsized":"2024-03-06T07:28:14.773952+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772662+0000","last_change":"2024-03-06T07:28:14.772662+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772662+0000","last_undegraded":"2024-03-06T07:28:14.772662+0000","last_fullsized":"2024-03-06T07:28:14.772662+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777224+0000","last_change":"2024-03-06T07:28:14.777224+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777224+0000","last_undegraded":"2024-03-06T07:28:14.777224+0000","last_fullsized":"2024-03-06T07:28:14.777224+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775345+0000","last_change":"2024-03-06T07:28:14.775345+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775345+0000","last_undegraded":"2024-03-06T07:28:14.775345+0000","last_fullsized":"2024-03-06T07:28:14.775345+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772357+0000","last_change":"2024-03-06T07:28:14.772357+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772357+0000","last_undegraded":"2024-03-06T07:28:14.772357+0000","last_fullsized":"2024-03-06T07:28:14.772357+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777328+0000","last_change":"2024-03-06T07:28:14.777328+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777328+0000","last_undegraded":"2024-03-06T07:28:14.777328+0000","last_fullsized":"2024-03-06T07:28:14.777328+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773893+0000","last_change":"2024-03-06T07:28:14.773893+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773893+0000","last_undegraded":"2024-03-06T07:28:14.773893+0000","last_fullsized":"2024-03-06T07:28:14.773893+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778348+0000","last_change":"2024-03-06T07:28:14.778348+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778348+0000","last_undegraded":"2024-03-06T07:28:14.778348+0000","last_fullsized":"2024-03-06T07:28:14.778348+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778695+0000","last_change":"2024-03-06T07:28:14.778695+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778695+0000","last_undegraded":"2024-03-06T07:28:14.778695+0000","last_fullsized":"2024-03-06T07:28:14.778695+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.771877+0000","last_change":"2024-03-06T07:28:14.771877+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.771877+0000","last_undegraded":"2024-03-06T07:28:14.771877+0000","last_fullsized":"2024-03-06T07:28:14.771877+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779909+0000","last_change":"2024-03-06T07:28:14.779909+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779909+0000","last_undegraded":"2024-03-06T07:28:14.779909+0000","last_fullsized":"2024-03-06T07:28:14.779909+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778508+0000","last_change":"2024-03-06T07:28:14.778508+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778508+0000","last_undegraded":"2024-03-06T07:28:14.778508+0000","last_fullsized":"2024-03-06T07:28:14.778508+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773904+0000","last_change":"2024-03-06T07:28:14.773904+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773904+0000","last_undegraded":"2024-03-06T07:28:14.773904+0000","last_fullsized":"2024-03-06T07:28:14.773904+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.771975+0000","last_change":"2024-03-06T07:28:14.771975+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.771975+0000","last_undegraded":"2024-03-06T07:28:14.771975+0000","last_fullsized":"2024-03-06T07:28:14.771975+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773009+0000","last_change":"2024-03-06T07:28:14.773009+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773009+0000","last_undegraded":"2024-03-06T07:28:14.773009+0000","last_fullsized":"2024-03-06T07:28:14.773009+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786785+0000","last_change":"2024-03-06T07:28:14.781972+0000","last_active":"2024-03-06T07:28:14.786785+0000","last_peered":"2024-03-06T07:28:14.786785+0000","last_clean":"2024-03-06T07:28:14.786785+0000","last_became_active":"2024-03-06T07:28:14.781691+0000","last_became_peered":"2024-03-06T07:28:14.781691+0000","last_unstale":"2024-03-06T07:28:14.786785+0000","last_undegraded":"2024-03-06T07:28:14.786785+0000","last_fullsized":"2024-03-06T07:28:14.786785+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:13.953720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772471+0000","last_change":"2024-03-06T07:28:14.772471+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772471+0000","last_undegraded":"2024-03-06T07:28:14.772471+0000","last_fullsized":"2024-03-06T07:28:14.772471+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791749+0000","last_change":"2024-03-06T07:28:14.788442+0000","last_active":"2024-03-06T07:28:14.791749+0000","last_peered":"2024-03-06T07:28:14.791749+0000","last_clean":"2024-03-06T07:28:14.791749+0000","last_became_active":"2024-03-06T07:28:14.788075+0000","last_became_peered":"2024-03-06T07:28:14.788075+0000","last_unstale":"2024-03-06T07:28:14.791749+0000","last_undegraded":"2024-03-06T07:28:14.791749+0000","last_fullsized":"2024-03-06T07:28:14.791749+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:42.385844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776275+0000","last_change":"2024-03-06T07:28:14.776275+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776275+0000","last_undegraded":"2024-03-06T07:28:14.776275+0000","last_fullsized":"2024-03-06T07:28:14.776275+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.792515+0000","last_change":"2024-03-06T07:28:14.791895+0000","last_active":"2024-03-06T07:28:14.792515+0000","last_peered":"2024-03-06T07:28:14.792515+0000","last_clean":"2024-03-06T07:28:14.792515+0000","last_became_active":"2024-03-06T07:28:14.791606+0000","last_became_peered":"2024-03-06T07:28:14.791606+0000","last_unstale":"2024-03-06T07:28:14.792515+0000","last_undegraded":"2024-03-06T07:28:14.792515+0000","last_fullsized":"2024-03-06T07:28:14.792515+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:02.151765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.781725+0000","last_change":"2024-03-06T07:28:14.781725+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.781725+0000","last_undegraded":"2024-03-06T07:28:14.781725+0000","last_fullsized":"2024-03-06T07:28:14.781725+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.785659+0000","last_change":"2024-03-06T07:28:14.783699+0000","last_active":"2024-03-06T07:28:14.785659+0000","last_peered":"2024-03-06T07:28:14.785659+0000","last_clean":"2024-03-06T07:28:14.785659+0000","last_became_active":"2024-03-06T07:28:14.783452+0000","last_became_peered":"2024-03-06T07:28:14.783452+0000","last_unstale":"2024-03-06T07:28:14.785659+0000","last_undegraded":"2024-03-06T07:28:14.785659+0000","last_fullsized":"2024-03-06T07:28:14.785659+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:41.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776389+0000","last_change":"2024-03-06T07:28:14.776389+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776389+0000","last_undegraded":"2024-03-06T07:28:14.776389+0000","last_fullsized":"2024-03-06T07:28:14.776389+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789132+0000","last_change":"2024-03-06T07:28:14.786831+0000","last_active":"2024-03-06T07:28:14.789132+0000","last_peered":"2024-03-06T07:28:14.789132+0000","last_clean":"2024-03-06T07:28:14.789132+0000","last_became_active":"2024-03-06T07:28:14.786548+0000","last_became_peered":"2024-03-06T07:28:14.786548+0000","last_unstale":"2024-03-06T07:28:14.789132+0000","last_undegraded":"2024-03-06T07:28:14.789132+0000","last_fullsized":"2024-03-06T07:28:14.789132+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:25:57.094067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777088+0000","last_change":"2024-03-06T07:28:14.777088+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777088+0000","last_undegraded":"2024-03-06T07:28:14.777088+0000","last_fullsized":"2024-03-06T07:28:14.777088+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789637+0000","last_change":"2024-03-06T07:28:14.782671+0000","last_active":"2024-03-06T07:28:14.789637+0000","last_peered":"2024-03-06T07:28:14.789637+0000","last_clean":"2024-03-06T07:28:14.789637+0000","last_became_active":"2024-03-06T07:28:14.782382+0000","last_became_peered":"2024-03-06T07:28:14.782382+0000","last_unstale":"2024-03-06T07:28:14.789637+0000","last_undegraded":"2024-03-06T07:28:14.789637+0000","last_fullsized":"2024-03-06T07:28:14.789637+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:07.331967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778436+0000","last_change":"2024-03-06T07:28:14.778436+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778436+0000","last_undegraded":"2024-03-06T07:28:14.778436+0000","last_fullsized":"2024-03-06T07:28:14.778436+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.777796+0000","last_change":"2024-03-06T07:28:14.776124+0000","last_active":"2024-03-06T07:28:14.777796+0000","last_peered":"2024-03-06T07:28:14.777796+0000","last_clean":"2024-03-06T07:28:14.777796+0000","last_became_active":"2024-03-06T07:28:14.775819+0000","last_became_peered":"2024-03-06T07:28:14.775819+0000","last_unstale":"2024-03-06T07:28:14.777796+0000","last_undegraded":"2024-03-06T07:28:14.777796+0000","last_fullsized":"2024-03-06T07:28:14.777796+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:30:20.528287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779244+0000","last_change":"2024-03-06T07:28:14.779244+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779244+0000","last_undegraded":"2024-03-06T07:28:14.779244+0000","last_fullsized":"2024-03-06T07:28:14.779244+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790821+0000","last_change":"2024-03-06T07:28:14.789097+0000","last_active":"2024-03-06T07:28:14.790821+0000","last_peered":"2024-03-06T07:28:14.790821+0000","last_clean":"2024-03-06T07:28:14.790821+0000","last_became_active":"2024-03-06T07:28:14.788833+0000","last_became_peered":"2024-03-06T07:28:14.788833+0000","last_unstale":"2024-03-06T07:28:14.790821+0000","last_undegraded":"2024-03-06T07:28:14.790821+0000","last_fullsized":"2024-03-06T07:28:14.790821+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:14.725218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779004+0000","last_change":"2024-03-06T07:28:14.779004+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779004+0000","last_undegraded":"2024-03-06T07:28:14.779004+0000","last_fullsized":"2024-03-06T07:28:14.779004+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789611+0000","last_change":"2024-03-06T07:28:14.787649+0000","last_active":"2024-03-06T07:28:14.789611+0000","last_peered":"2024-03-06T07:28:14.789611+0000","last_clean":"2024-03-06T07:28:14.789611+0000","last_became_active":"2024-03-06T07:28:14.787348+0000","last_became_peered":"2024-03-06T07:28:14.787348+0000","last_unstale":"2024-03-06T07:28:14.789611+0000","last_undegraded":"2024-03-06T07:28:14.789611+0000","last_fullsized":"2024-03-06T07:28:14.789611+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:38:13.142714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773103+0000","last_change":"2024-03-06T07:28:14.773103+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773103+0000","last_undegraded":"2024-03-06T07:28:14.773103+0000","last_fullsized":"2024-03-06T07:28:14.773103+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787342+0000","last_change":"2024-03-06T07:28:14.785132+0000","last_active":"2024-03-06T07:28:14.787342+0000","last_peered":"2024-03-06T07:28:14.787342+0000","last_clean":"2024-03-06T07:28:14.787342+0000","last_became_active":"2024-03-06T07:28:14.784849+0000","last_became_peered":"2024-03-06T07:28:14.784849+0000","last_unstale":"2024-03-06T07:28:14.787342+0000","last_undegraded":"2024-03-06T07:28:14.787342+0000","last_fullsized":"2024-03-06T07:28:14.787342+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:14:41.231832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777607+0000","last_change":"2024-03-06T07:28:14.777607+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777607+0000","last_undegraded":"2024-03-06T07:28:14.777607+0000","last_fullsized":"2024-03-06T07:28:14.777607+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786919+0000","last_change":"2024-03-06T07:28:14.785822+0000","last_active":"2024-03-06T07:28:14.786919+0000","last_peered":"2024-03-06T07:28:14.786919+0000","last_clean":"2024-03-06T07:28:14.786919+0000","last_became_active":"2024-03-06T07:28:14.785540+0000","last_became_peered":"2024-03-06T07:28:14.785540+0000","last_unstale":"2024-03-06T07:28:14.786919+0000","last_undegraded":"2024-03-06T07:28:14.786919+0000","last_fullsized":"2024-03-06T07:28:14.786919+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:03.928508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773152+0000","last_change":"2024-03-06T07:28:14.773152+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773152+0000","last_undegraded":"2024-03-06T07:28:14.773152+0000","last_fullsized":"2024-03-06T07:28:14.773152+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.785679+0000","last_change":"2024-03-06T07:28:14.784265+0000","last_active":"2024-03-06T07:28:14.785679+0000","last_peered":"2024-03-06T07:28:14.785679+0000","last_clean":"2024-03-06T07:28:14.785679+0000","last_became_active":"2024-03-06T07:28:14.783985+0000","last_became_peered":"2024-03-06T07:28:14.783985+0000","last_unstale":"2024-03-06T07:28:14.785679+0000","last_undegraded":"2024-03-06T07:28:14.785679+0000","last_fullsized":"2024-03-06T07:28:14.785679+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.428223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776788+0000","last_change":"2024-03-06T07:28:14.776788+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776788+0000","last_undegraded":"2024-03-06T07:28:14.776788+0000","last_fullsized":"2024-03-06T07:28:14.776788+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787370+0000","last_change":"2024-03-06T07:28:14.784209+0000","last_active":"2024-03-06T07:28:14.787370+0000","last_peered":"2024-03-06T07:28:14.787370+0000","last_clean":"2024-03-06T07:28:14.787370+0000","last_became_active":"2024-03-06T07:28:14.783932+0000","last_became_peered":"2024-03-06T07:28:14.783932+0000","last_unstale":"2024-03-06T07:28:14.787370+0000","last_undegraded":"2024-03-06T07:28:14.787370+0000","last_fullsized":"2024-03-06T07:28:14.787370+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:35.222311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773068+0000","last_change":"2024-03-06T07:28:14.773068+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773068+0000","last_undegraded":"2024-03-06T07:28:14.773068+0000","last_fullsized":"2024-03-06T07:28:14.773068+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791974+0000","last_change":"2024-03-06T07:28:14.787534+0000","last_active":"2024-03-06T07:28:14.791974+0000","last_peered":"2024-03-06T07:28:14.791974+0000","last_clean":"2024-03-06T07:28:14.791974+0000","last_became_active":"2024-03-06T07:28:14.787286+0000","last_became_peered":"2024-03-06T07:28:14.787286+0000","last_unstale":"2024-03-06T07:28:14.791974+0000","last_undegraded":"2024-03-06T07:28:14.791974+0000","last_fullsized":"2024-03-06T07:28:14.791974+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:58.265777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778848+0000","last_change":"2024-03-06T07:28:14.778848+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778848+0000","last_undegraded":"2024-03-06T07:28:14.778848+0000","last_fullsized":"2024-03-06T07:28:14.778848+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791451+0000","last_change":"2024-03-06T07:28:14.788682+0000","last_active":"2024-03-06T07:28:14.791451+0000","last_peered":"2024-03-06T07:28:14.791451+0000","last_clean":"2024-03-06T07:28:14.791451+0000","last_became_active":"2024-03-06T07:28:14.788381+0000","last_became_peered":"2024-03-06T07:28:14.788381+0000","last_unstale":"2024-03-06T07:28:14.791451+0000","last_undegraded":"2024-03-06T07:28:14.791451+0000","last_fullsized":"2024-03-06T07:28:14.791451+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:23:05.687550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776586+0000","last_change":"2024-03-06T07:28:14.776586+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776586+0000","last_undegraded":"2024-03-06T07:28:14.776586+0000","last_fullsized":"2024-03-06T07:28:14.776586+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789078+0000","last_change":"2024-03-06T07:28:14.787636+0000","last_active":"2024-03-06T07:28:14.789078+0000","last_peered":"2024-03-06T07:28:14.789078+0000","last_clean":"2024-03-06T07:28:14.789078+0000","last_became_active":"2024-03-06T07:28:14.787367+0000","last_became_peered":"2024-03-06T07:28:14.787367+0000","last_unstale":"2024-03-06T07:28:14.789078+0000","last_undegraded":"2024-03-06T07:28:14.789078+0000","last_fullsized":"2024-03-06T07:28:14.789078+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:21.492957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777633+0000","last_change":"2024-03-06T07:28:14.777633+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777633+0000","last_undegraded":"2024-03-06T07:28:14.777633+0000","last_fullsized":"2024-03-06T07:28:14.777633+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787834+0000","last_change":"2024-03-06T07:28:14.787083+0000","last_active":"2024-03-06T07:28:14.787834+0000","last_peered":"2024-03-06T07:28:14.787834+0000","last_clean":"2024-03-06T07:28:14.787834+0000","last_became_active":"2024-03-06T07:28:14.786827+0000","last_became_peered":"2024-03-06T07:28:14.786827+0000","last_unstale":"2024-03-06T07:28:14.787834+0000","last_undegraded":"2024-03-06T07:28:14.787834+0000","last_fullsized":"2024-03-06T07:28:14.787834+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:15.361243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774343+0000","last_change":"2024-03-06T07:28:14.774343+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774343+0000","last_undegraded":"2024-03-06T07:28:14.774343+0000","last_fullsized":"2024-03-06T07:28:14.774343+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791723+0000","last_change":"2024-03-06T07:28:14.788050+0000","last_active":"2024-03-06T07:28:14.791723+0000","last_peered":"2024-03-06T07:28:14.791723+0000","last_clean":"2024-03-06T07:28:14.791723+0000","last_became_active":"2024-03-06T07:28:14.787770+0000","last_became_peered":"2024-03-06T07:28:14.787770+0000","last_unstale":"2024-03-06T07:28:14.791723+0000","last_undegraded":"2024-03-06T07:28:14.791723+0000","last_fullsized":"2024-03-06T07:28:14.791723+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:42.511473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773686+0000","last_change":"2024-03-06T07:28:14.773686+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773686+0000","last_undegraded":"2024-03-06T07:28:14.773686+0000","last_fullsized":"2024-03-06T07:28:14.773686+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.783299+0000","last_change":"2024-03-06T07:28:14.782491+0000","last_active":"2024-03-06T07:28:14.783299+0000","last_peered":"2024-03-06T07:28:14.783299+0000","last_clean":"2024-03-06T07:28:14.783299+0000","last_became_active":"2024-03-06T07:28:14.782223+0000","last_became_peered":"2024-03-06T07:28:14.782223+0000","last_unstale":"2024-03-06T07:28:14.783299+0000","last_undegraded":"2024-03-06T07:28:14.783299+0000","last_fullsized":"2024-03-06T07:28:14.783299+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:55.179141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775653+0000","last_change":"2024-03-06T07:28:14.775653+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775653+0000","last_undegraded":"2024-03-06T07:28:14.775653+0000","last_fullsized":"2024-03-06T07:28:14.775653+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790941+0000","last_change":"2024-03-06T07:28:14.785927+0000","last_active":"2024-03-06T07:28:14.790941+0000","last_peered":"2024-03-06T07:28:14.790941+0000","last_clean":"2024-03-06T07:28:14.790941+0000","last_became_active":"2024-03-06T07:28:14.785666+0000","last_became_peered":"2024-03-06T07:28:14.785666+0000","last_unstale":"2024-03-06T07:28:14.790941+0000","last_undegraded":"2024-03-06T07:28:14.790941+0000","last_fullsized":"2024-03-06T07:28:14.790941+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.036054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773460+0000","last_change":"2024-03-06T07:28:14.773460+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773460+0000","last_undegraded":"2024-03-06T07:28:14.773460+0000","last_fullsized":"2024-03-06T07:28:14.773460+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791763+0000","last_change":"2024-03-06T07:28:14.788414+0000","last_active":"2024-03-06T07:28:14.791763+0000","last_peered":"2024-03-06T07:28:14.791763+0000","last_clean":"2024-03-06T07:28:14.791763+0000","last_became_active":"2024-03-06T07:28:14.788047+0000","last_became_peered":"2024-03-06T07:28:14.788047+0000","last_unstale":"2024-03-06T07:28:14.791763+0000","last_undegraded":"2024-03-06T07:28:14.791763+0000","last_fullsized":"2024-03-06T07:28:14.791763+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:46.615680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774832+0000","last_change":"2024-03-06T07:28:14.774832+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774832+0000","last_undegraded":"2024-03-06T07:28:14.774832+0000","last_fullsized":"2024-03-06T07:28:14.774832+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787528+0000","last_change":"2024-03-06T07:28:14.782233+0000","last_active":"2024-03-06T07:28:14.787528+0000","last_peered":"2024-03-06T07:28:14.787528+0000","last_clean":"2024-03-06T07:28:14.787528+0000","last_became_active":"2024-03-06T07:28:14.781927+0000","last_became_peered":"2024-03-06T07:28:14.781927+0000","last_unstale":"2024-03-06T07:28:14.787528+0000","last_undegraded":"2024-03-06T07:28:14.787528+0000","last_fullsized":"2024-03-06T07:28:14.787528+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:48.443105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779417+0000","last_change":"2024-03-06T07:28:14.779417+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779417+0000","last_undegraded":"2024-03-06T07:28:14.779417+0000","last_fullsized":"2024-03-06T07:28:14.779417+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787170+0000","last_change":"2024-03-06T07:28:14.785960+0000","last_active":"2024-03-06T07:28:14.787170+0000","last_peered":"2024-03-06T07:28:14.787170+0000","last_clean":"2024-03-06T07:28:14.787170+0000","last_became_active":"2024-03-06T07:28:14.785692+0000","last_became_peered":"2024-03-06T07:28:14.785692+0000","last_unstale":"2024-03-06T07:28:14.787170+0000","last_undegraded":"2024-03-06T07:28:14.787170+0000","last_fullsized":"2024-03-06T07:28:14.787170+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:04.218118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779640+0000","last_change":"2024-03-06T07:28:14.779640+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779640+0000","last_undegraded":"2024-03-06T07:28:14.779640+0000","last_fullsized":"2024-03-06T07:28:14.779640+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789580+0000","last_change":"2024-03-06T07:28:14.784580+0000","last_active":"2024-03-06T07:28:14.789580+0000","last_peered":"2024-03-06T07:28:14.789580+0000","last_clean":"2024-03-06T07:28:14.789580+0000","last_became_active":"2024-03-06T07:28:14.784245+0000","last_became_peered":"2024-03-06T07:28:14.784245+0000","last_unstale":"2024-03-06T07:28:14.789580+0000","last_undegraded":"2024-03-06T07:28:14.789580+0000","last_fullsized":"2024-03-06T07:28:14.789580+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:10.104421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773331+0000","last_change":"2024-03-06T07:28:14.773331+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773331+0000","last_undegraded":"2024-03-06T07:28:14.773331+0000","last_fullsized":"2024-03-06T07:28:14.773331+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789348+0000","last_change":"2024-03-06T07:28:14.786720+0000","last_active":"2024-03-06T07:28:14.789348+0000","last_peered":"2024-03-06T07:28:14.789348+0000","last_clean":"2024-03-06T07:28:14.789348+0000","last_became_active":"2024-03-06T07:28:14.786452+0000","last_became_peered":"2024-03-06T07:28:14.786452+0000","last_unstale":"2024-03-06T07:28:14.789348+0000","last_undegraded":"2024-03-06T07:28:14.789348+0000","last_fullsized":"2024-03-06T07:28:14.789348+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:51.046635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773547+0000","last_change":"2024-03-06T07:28:14.773547+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773547+0000","last_undegraded":"2024-03-06T07:28:14.773547+0000","last_fullsized":"2024-03-06T07:28:14.773547+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790155+0000","last_change":"2024-03-06T07:28:14.789522+0000","last_active":"2024-03-06T07:28:14.790155+0000","last_peered":"2024-03-06T07:28:14.790155+0000","last_clean":"2024-03-06T07:28:14.790155+0000","last_became_active":"2024-03-06T07:28:14.789267+0000","last_became_peered":"2024-03-06T07:28:14.789267+0000","last_unstale":"2024-03-06T07:28:14.790155+0000","last_undegraded":"2024-03-06T07:28:14.790155+0000","last_fullsized":"2024-03-06T07:28:14.790155+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:41.428236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777263+0000","last_change":"2024-03-06T07:28:14.777263+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777263+0000","last_undegraded":"2024-03-06T07:28:14.777263+0000","last_fullsized":"2024-03-06T07:28:14.777263+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789903+0000","last_change":"2024-03-06T07:28:14.788385+0000","last_active":"2024-03-06T07:28:14.789903+0000","last_peered":"2024-03-06T07:28:14.789903+0000","last_clean":"2024-03-06T07:28:14.789903+0000","last_became_active":"2024-03-06T07:28:14.788113+0000","last_became_peered":"2024-03-06T07:28:14.788113+0000","last_unstale":"2024-03-06T07:28:14.789903+0000","last_undegraded":"2024-03-06T07:28:14.789903+0000","last_fullsized":"2024-03-06T07:28:14.789903+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:50.463171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.782151+0000","last_change":"2024-03-06T07:28:14.782151+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.782151+0000","last_undegraded":"2024-03-06T07:28:14.782151+0000","last_fullsized":"2024-03-06T07:28:14.782151+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.794736+0000","last_change":"2024-03-06T07:28:14.792256+0000","last_active":"2024-03-06T07:28:14.794736+0000","last_peered":"2024-03-06T07:28:14.794736+0000","last_clean":"2024-03-06T07:28:14.794736+0000","last_became_active":"2024-03-06T07:28:14.792032+0000","last_became_peered":"2024-03-06T07:28:14.792032+0000","last_unstale":"2024-03-06T07:28:14.794736+0000","last_undegraded":"2024-03-06T07:28:14.794736+0000","last_fullsized":"2024-03-06T07:28:14.794736+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:20.997172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775584+0000","last_change":"2024-03-06T07:28:14.775584+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775584+0000","last_undegraded":"2024-03-06T07:28:14.775584+0000","last_fullsized":"2024-03-06T07:28:14.775584+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.792007+0000","last_change":"2024-03-06T07:28:14.788477+0000","last_active":"2024-03-06T07:28:14.792007+0000","last_peered":"2024-03-06T07:28:14.792007+0000","last_clean":"2024-03-06T07:28:14.792007+0000","last_became_active":"2024-03-06T07:28:14.788223+0000","last_became_peered":"2024-03-06T07:28:14.788223+0000","last_unstale":"2024-03-06T07:28:14.792007+0000","last_undegraded":"2024-03-06T07:28:14.792007+0000","last_fullsized":"2024-03-06T07:28:14.792007+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:52.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773666+0000","last_change":"2024-03-06T07:28:14.773666+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773666+0000","last_undegraded":"2024-03-06T07:28:14.773666+0000","last_fullsized":"2024-03-06T07:28:14.773666+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789661+0000","last_change":"2024-03-06T07:28:14.788356+0000","last_active":"2024-03-06T07:28:14.789661+0000","last_peered":"2024-03-06T07:28:14.789661+0000","last_clean":"2024-03-06T07:28:14.789661+0000","last_became_active":"2024-03-06T07:28:14.788093+0000","last_became_peered":"2024-03-06T07:28:14.788093+0000","last_unstale":"2024-03-06T07:28:14.789661+0000","last_undegraded":"2024-03-06T07:28:14.789661+0000","last_fullsized":"2024-03-06T07:28:14.789661+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:31.566144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.780498+0000","last_change":"2024-03-06T07:28:14.780498+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.780498+0000","last_undegraded":"2024-03-06T07:28:14.780498+0000","last_fullsized":"2024-03-06T07:28:14.780498+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789297+0000","last_change":"2024-03-06T07:28:14.786359+0000","last_active":"2024-03-06T07:28:14.789297+0000","last_peered":"2024-03-06T07:28:14.789297+0000","last_clean":"2024-03-06T07:28:14.789297+0000","last_became_active":"2024-03-06T07:28:14.786068+0000","last_became_peered":"2024-03-06T07:28:14.786068+0000","last_unstale":"2024-03-06T07:28:14.789297+0000","last_undegraded":"2024-03-06T07:28:14.789297+0000","last_fullsized":"2024-03-06T07:28:14.789297+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:10.545981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775080+0000","last_change":"2024-03-06T07:28:14.775080+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775080+0000","last_undegraded":"2024-03-06T07:28:14.775080+0000","last_fullsized":"2024-03-06T07:28:14.775080+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786611+0000","last_change":"2024-03-06T07:28:14.785657+0000","last_active":"2024-03-06T07:28:14.786611+0000","last_peered":"2024-03-06T07:28:14.786611+0000","last_clean":"2024-03-06T07:28:14.786611+0000","last_became_active":"2024-03-06T07:28:14.785292+0000","last_became_peered":"2024-03-06T07:28:14.785292+0000","last_unstale":"2024-03-06T07:28:14.786611+0000","last_undegraded":"2024-03-06T07:28:14.786611+0000","last_fullsized":"2024-03-06T07:28:14.786611+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:21.053100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":498,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.774290+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:28:14.774290+0000","last_peered":"2024-03-06T07:28:14.774290+0000","last_clean":"2024-03-06T07:28:14.774290+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:28:14.774290+0000","last_undegraded":"2024-03-06T07:28:14.774290+0000","last_fullsized":"2024-03-06T07:28:14.774290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2539520,"data_stored":1790994,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33612,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338228,"statfs":{"total":96636764160,"available":96602345472,"internally_reserved":0,"allocated":2486272,"data_stored":1207293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33612,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338228,"statfs":{"total":96636764160,"available":96602345472,"internally_reserved":0,"allocated":2486272,"data_stored":1207293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37764,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334076,"statfs":{"total":96636764160,"available":96598093824,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2539520,"data_stored":1790994,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:28:20.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:28:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.294+0000 7f3a6db3e640 1 Processor -- start 2024-03-06T07:28:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.294+0000 7f3a6db3e640 1 -- start start 2024-03-06T07:28:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.294+0000 7f3a6db3e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a6806d9f0 0x7f3a680a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.294+0000 7f3a6db3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a68158800 con 0x7f3a680c3280 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.294+0000 7f3a6db3e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a68158ae0 con 0x7f3a680c3710 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.294+0000 7f3a6db3e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a68158dc0 con 0x7f3a6806d9f0 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.295+0000 7f3a677fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a6806d9f0 0x7f3a680a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.295+0000 7f3a66ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3a680c3280 0x7f3a681551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57718/0 (socket says 172.21.15.26:57718) 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.295+0000 7f3a66ffd640 1 -- 172.21.15.26:0/2884929449 learned_addr learned my addr 172.21.15.26:0/2884929449 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.295+0000 7f3a677fe640 1 -- 172.21.15.26:0/2884929449 >> v1:172.21.15.117:6789/0 conn(0x7f3a680c3710 legacy=0x7f3a681558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.295+0000 7f3a677fe640 1 -- 172.21.15.26:0/2884929449 >> v1:172.21.15.26:6789/0 conn(0x7f3a680c3280 legacy=0x7f3a681551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.295+0000 7f3a677fe640 1 -- 172.21.15.26:0/2884929449 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a681590a0 con 0x7f3a6806d9f0 2024-03-06T07:28:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.295+0000 7f3a667fc640 1 -- 172.21.15.26:0/2884929449 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829710439 0 0) 0x7f3a68158800 con 0x7f3a680c3280 2024-03-06T07:28:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.296+0000 7f3a677fe640 1 --2- 172.21.15.26:0/2884929449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680a7cd0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3a580097a0 tx=0x7f3a5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f794a63aa221570 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.296+0000 7f3a667fc640 1 -- 172.21.15.26:0/2884929449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a580315a0 con 0x7f3a6806d9f0 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.296+0000 7f3a667fc640 1 -- 172.21.15.26:0/2884929449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58037070 con 0x7f3a6806d9f0 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.296+0000 7f3a667fc640 1 -- 172.21.15.26:0/2884929449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58031960 con 0x7f3a6806d9f0 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.297+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2884929449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 msgr2=0x7f3a680a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.297+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2884929449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680a7cd0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3a580097a0 tx=0x7f3a5802b840 comp rx=0 tx=0).stop 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.297+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2884929449 shutdown_connections 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.297+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2884929449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680a7cd0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.297+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2884929449 >> 172.21.15.26:0/2884929449 conn(0x7f3a680aa630 msgr2=0x7f3a680aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.297+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2884929449 shutdown_connections 2024-03-06T07:28:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.297+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2884929449 wait complete. 2024-03-06T07:28:20.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.298+0000 7f3a6db3e640 1 Processor -- start 2024-03-06T07:28:20.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.298+0000 7f3a6db3e640 1 -- start start 2024-03-06T07:28:20.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a6db3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a6db3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a680c3280 0x7f3a681e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a6db3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a680c3710 0x7f3a681e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a6db3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6806df50 con 0x7f3a6806d9f0 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a6db3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6806ddd0 con 0x7f3a680c3710 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a6db3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680a8210 con 0x7f3a680c3280 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43944/0 (socket says 172.21.15.26:43944) 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 -- 172.21.15.26:0/2669558772 learned_addr learned my addr 172.21.15.26:0/2669558772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a67fff640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a680c3710 0x7f3a681e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a66ffd640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a680c3280 0x7f3a681e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 -- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a680c3280 msgr2=0x7f3a681e7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a680c3280 0x7f3a681e7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 -- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a680c3710 msgr2=0x7f3a681e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a680c3710 0x7f3a681e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 -- 172.21.15.26:0/2669558772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68070500 con 0x7f3a6806d9f0 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.299+0000 7f3a677fe640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680c5580 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f3a580097a0 tx=0x7f3a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.300+0000 7f3a4ffff640 1 -- 172.21.15.26:0/2669558772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58004230 con 0x7f3a6806d9f0 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.300+0000 7f3a4ffff640 1 -- 172.21.15.26:0/2669558772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a580043f0 con 0x7f3a6806d9f0 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.300+0000 7f3a4ffff640 1 -- 172.21.15.26:0/2669558772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58032880 con 0x7f3a6806d9f0 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.300+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a680a9a50 con 0x7f3a6806d9f0 2024-03-06T07:28:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.300+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680aa010 con 0x7f3a6806d9f0 2024-03-06T07:28:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.301+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6804f330 con 0x7f3a6806d9f0 2024-03-06T07:28:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.301+0000 7f3a4ffff640 1 -- 172.21.15.26:0/2669558772 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a58032a20 con 0x7f3a6806d9f0 2024-03-06T07:28:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.302+0000 7f3a4ffff640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5406b4d0 0x7f3a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.302+0000 7f3a4ffff640 1 -- 172.21.15.26:0/2669558772 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a5802f030 con 0x7f3a6806d9f0 2024-03-06T07:28:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.302+0000 7f3a66ffd640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5406b4d0 0x7f3a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.303+0000 7f3a66ffd640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5406b4d0 0x7f3a5406d990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3a6814e0a0 tx=0x7f3a5c007b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.306+0000 7f3a4ffff640 1 -- 172.21.15.26:0/2669558772 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a58041070 con 0x7f3a6806d9f0 2024-03-06T07:28:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.484+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a68073240 con 0x7f3a5406b4d0 2024-03-06T07:28:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.490+0000 7f3a4ffff640 1 -- 172.21.15.26:0/2669558772 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247174 (secure 0 0 0) 0x7f3a68073240 con 0x7f3a5406b4d0 2024-03-06T07:28:20.494 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:20.496 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:28:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5406b4d0 msgr2=0x7f3a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5406b4d0 0x7f3a5406d990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3a6814e0a0 tx=0x7f3a5c007b90 comp rx=0 tx=0).stop 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 msgr2=0x7f3a680c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680c5580 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f3a580097a0 tx=0x7f3a580046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 shutdown_connections 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5406b4d0 0x7f3a5406d990 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a680c3710 0x7f3a681e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a680c3280 0x7f3a681e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 --2- 172.21.15.26:0/2669558772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6806d9f0 0x7f3a680c5580 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 >> 172.21.15.26:0/2669558772 conn(0x7f3a680aa630 msgr2=0x7f3a68154290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 shutdown_connections 2024-03-06T07:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.493+0000 7f3a6db3e640 1 -- 172.21.15.26:0/2669558772 wait complete. 2024-03-06T07:28:20.506 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-03-06T07:28:19.135828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281416,"kb_used_data":19480,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754693304,"statfs":{"total":773094113280,"available":772805943296,"internally_reserved":0,"allocated":19947520,"data_stored":10799078,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.370025"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773969+0000","last_change":"2024-03-06T07:28:14.773969+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773969+0000","last_undegraded":"2024-03-06T07:28:14.773969+0000","last_fullsized":"2024-03-06T07:28:14.773969+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774111+0000","last_change":"2024-03-06T07:28:14.774111+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774111+0000","last_undegraded":"2024-03-06T07:28:14.774111+0000","last_fullsized":"2024-03-06T07:28:14.774111+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776824+0000","last_change":"2024-03-06T07:28:14.776824+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776824+0000","last_undegraded":"2024-03-06T07:28:14.776824+0000","last_fullsized":"2024-03-06T07:28:14.776824+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774117+0000","last_change":"2024-03-06T07:28:14.774117+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774117+0000","last_undegraded":"2024-03-06T07:28:14.774117+0000","last_fullsized":"2024-03-06T07:28:14.774117+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776834+0000","last_change":"2024-03-06T07:28:14.776834+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776834+0000","last_undegraded":"2024-03-06T07:28:14.776834+0000","last_fullsized":"2024-03-06T07:28:14.776834+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773087+0000","last_change":"2024-03-06T07:28:14.773087+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773087+0000","last_undegraded":"2024-03-06T07:28:14.773087+0000","last_fullsized":"2024-03-06T07:28:14.773087+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775879+0000","last_change":"2024-03-06T07:28:14.775879+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775879+0000","last_undegraded":"2024-03-06T07:28:14.775879+0000","last_fullsized":"2024-03-06T07:28:14.775879+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775942+0000","last_change":"2024-03-06T07:28:14.775942+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775942+0000","last_undegraded":"2024-03-06T07:28:14.775942+0000","last_fullsized":"2024-03-06T07:28:14.775942+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773504+0000","last_change":"2024-03-06T07:28:14.773504+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773504+0000","last_undegraded":"2024-03-06T07:28:14.773504+0000","last_fullsized":"2024-03-06T07:28:14.773504+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776672+0000","last_change":"2024-03-06T07:28:14.776672+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776672+0000","last_undegraded":"2024-03-06T07:28:14.776672+0000","last_fullsized":"2024-03-06T07:28:14.776672+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776343+0000","last_change":"2024-03-06T07:28:14.776343+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776343+0000","last_undegraded":"2024-03-06T07:28:14.776343+0000","last_fullsized":"2024-03-06T07:28:14.776343+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772516+0000","last_change":"2024-03-06T07:28:14.772516+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772516+0000","last_undegraded":"2024-03-06T07:28:14.772516+0000","last_fullsized":"2024-03-06T07:28:14.772516+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775564+0000","last_change":"2024-03-06T07:28:14.775564+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775564+0000","last_undegraded":"2024-03-06T07:28:14.775564+0000","last_fullsized":"2024-03-06T07:28:14.775564+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778948+0000","last_change":"2024-03-06T07:28:14.778948+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778948+0000","last_undegraded":"2024-03-06T07:28:14.778948+0000","last_fullsized":"2024-03-06T07:28:14.778948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772911+0000","last_change":"2024-03-06T07:28:14.772911+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772911+0000","last_undegraded":"2024-03-06T07:28:14.772911+0000","last_fullsized":"2024-03-06T07:28:14.772911+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772552+0000","last_change":"2024-03-06T07:28:14.772552+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772552+0000","last_undegraded":"2024-03-06T07:28:14.772552+0000","last_fullsized":"2024-03-06T07:28:14.772552+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777866+0000","last_change":"2024-03-06T07:28:14.777866+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777866+0000","last_undegraded":"2024-03-06T07:28:14.777866+0000","last_fullsized":"2024-03-06T07:28:14.777866+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773906+0000","last_change":"2024-03-06T07:28:14.773906+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773906+0000","last_undegraded":"2024-03-06T07:28:14.773906+0000","last_fullsized":"2024-03-06T07:28:14.773906+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773952+0000","last_change":"2024-03-06T07:28:14.773952+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773952+0000","last_undegraded":"2024-03-06T07:28:14.773952+0000","last_fullsized":"2024-03-06T07:28:14.773952+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772662+0000","last_change":"2024-03-06T07:28:14.772662+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772662+0000","last_undegraded":"2024-03-06T07:28:14.772662+0000","last_fullsized":"2024-03-06T07:28:14.772662+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777224+0000","last_change":"2024-03-06T07:28:14.777224+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777224+0000","last_undegraded":"2024-03-06T07:28:14.777224+0000","last_fullsized":"2024-03-06T07:28:14.777224+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775345+0000","last_change":"2024-03-06T07:28:14.775345+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775345+0000","last_undegraded":"2024-03-06T07:28:14.775345+0000","last_fullsized":"2024-03-06T07:28:14.775345+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772357+0000","last_change":"2024-03-06T07:28:14.772357+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772357+0000","last_undegraded":"2024-03-06T07:28:14.772357+0000","last_fullsized":"2024-03-06T07:28:14.772357+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777328+0000","last_change":"2024-03-06T07:28:14.777328+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777328+0000","last_undegraded":"2024-03-06T07:28:14.777328+0000","last_fullsized":"2024-03-06T07:28:14.777328+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773893+0000","last_change":"2024-03-06T07:28:14.773893+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773893+0000","last_undegraded":"2024-03-06T07:28:14.773893+0000","last_fullsized":"2024-03-06T07:28:14.773893+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778348+0000","last_change":"2024-03-06T07:28:14.778348+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778348+0000","last_undegraded":"2024-03-06T07:28:14.778348+0000","last_fullsized":"2024-03-06T07:28:14.778348+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778695+0000","last_change":"2024-03-06T07:28:14.778695+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778695+0000","last_undegraded":"2024-03-06T07:28:14.778695+0000","last_fullsized":"2024-03-06T07:28:14.778695+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.771877+0000","last_change":"2024-03-06T07:28:14.771877+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.771877+0000","last_undegraded":"2024-03-06T07:28:14.771877+0000","last_fullsized":"2024-03-06T07:28:14.771877+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779909+0000","last_change":"2024-03-06T07:28:14.779909+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779909+0000","last_undegraded":"2024-03-06T07:28:14.779909+0000","last_fullsized":"2024-03-06T07:28:14.779909+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778508+0000","last_change":"2024-03-06T07:28:14.778508+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778508+0000","last_undegraded":"2024-03-06T07:28:14.778508+0000","last_fullsized":"2024-03-06T07:28:14.778508+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773904+0000","last_change":"2024-03-06T07:28:14.773904+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773904+0000","last_undegraded":"2024-03-06T07:28:14.773904+0000","last_fullsized":"2024-03-06T07:28:14.773904+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.771975+0000","last_change":"2024-03-06T07:28:14.771975+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.771975+0000","last_undegraded":"2024-03-06T07:28:14.771975+0000","last_fullsized":"2024-03-06T07:28:14.771975+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773009+0000","last_change":"2024-03-06T07:28:14.773009+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773009+0000","last_undegraded":"2024-03-06T07:28:14.773009+0000","last_fullsized":"2024-03-06T07:28:14.773009+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786785+0000","last_change":"2024-03-06T07:28:14.781972+0000","last_active":"2024-03-06T07:28:14.786785+0000","last_peered":"2024-03-06T07:28:14.786785+0000","last_clean":"2024-03-06T07:28:14.786785+0000","last_became_active":"2024-03-06T07:28:14.781691+0000","last_became_peered":"2024-03-06T07:28:14.781691+0000","last_unstale":"2024-03-06T07:28:14.786785+0000","last_undegraded":"2024-03-06T07:28:14.786785+0000","last_fullsized":"2024-03-06T07:28:14.786785+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:13.953720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.772471+0000","last_change":"2024-03-06T07:28:14.772471+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.772471+0000","last_undegraded":"2024-03-06T07:28:14.772471+0000","last_fullsized":"2024-03-06T07:28:14.772471+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791749+0000","last_change":"2024-03-06T07:28:14.788442+0000","last_active":"2024-03-06T07:28:14.791749+0000","last_peered":"2024-03-06T07:28:14.791749+0000","last_clean":"2024-03-06T07:28:14.791749+0000","last_became_active":"2024-03-06T07:28:14.788075+0000","last_became_peered":"2024-03-06T07:28:14.788075+0000","last_unstale":"2024-03-06T07:28:14.791749+0000","last_undegraded":"2024-03-06T07:28:14.791749+0000","last_fullsized":"2024-03-06T07:28:14.791749+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:42.385844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776275+0000","last_change":"2024-03-06T07:28:14.776275+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776275+0000","last_undegraded":"2024-03-06T07:28:14.776275+0000","last_fullsized":"2024-03-06T07:28:14.776275+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.792515+0000","last_change":"2024-03-06T07:28:14.791895+0000","last_active":"2024-03-06T07:28:14.792515+0000","last_peered":"2024-03-06T07:28:14.792515+0000","last_clean":"2024-03-06T07:28:14.792515+0000","last_became_active":"2024-03-06T07:28:14.791606+0000","last_became_peered":"2024-03-06T07:28:14.791606+0000","last_unstale":"2024-03-06T07:28:14.792515+0000","last_undegraded":"2024-03-06T07:28:14.792515+0000","last_fullsized":"2024-03-06T07:28:14.792515+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:02.151765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.781725+0000","last_change":"2024-03-06T07:28:14.781725+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.781725+0000","last_undegraded":"2024-03-06T07:28:14.781725+0000","last_fullsized":"2024-03-06T07:28:14.781725+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.785659+0000","last_change":"2024-03-06T07:28:14.783699+0000","last_active":"2024-03-06T07:28:14.785659+0000","last_peered":"2024-03-06T07:28:14.785659+0000","last_clean":"2024-03-06T07:28:14.785659+0000","last_became_active":"2024-03-06T07:28:14.783452+0000","last_became_peered":"2024-03-06T07:28:14.783452+0000","last_unstale":"2024-03-06T07:28:14.785659+0000","last_undegraded":"2024-03-06T07:28:14.785659+0000","last_fullsized":"2024-03-06T07:28:14.785659+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:41.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776389+0000","last_change":"2024-03-06T07:28:14.776389+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776389+0000","last_undegraded":"2024-03-06T07:28:14.776389+0000","last_fullsized":"2024-03-06T07:28:14.776389+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789132+0000","last_change":"2024-03-06T07:28:14.786831+0000","last_active":"2024-03-06T07:28:14.789132+0000","last_peered":"2024-03-06T07:28:14.789132+0000","last_clean":"2024-03-06T07:28:14.789132+0000","last_became_active":"2024-03-06T07:28:14.786548+0000","last_became_peered":"2024-03-06T07:28:14.786548+0000","last_unstale":"2024-03-06T07:28:14.789132+0000","last_undegraded":"2024-03-06T07:28:14.789132+0000","last_fullsized":"2024-03-06T07:28:14.789132+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:25:57.094067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777088+0000","last_change":"2024-03-06T07:28:14.777088+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777088+0000","last_undegraded":"2024-03-06T07:28:14.777088+0000","last_fullsized":"2024-03-06T07:28:14.777088+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789637+0000","last_change":"2024-03-06T07:28:14.782671+0000","last_active":"2024-03-06T07:28:14.789637+0000","last_peered":"2024-03-06T07:28:14.789637+0000","last_clean":"2024-03-06T07:28:14.789637+0000","last_became_active":"2024-03-06T07:28:14.782382+0000","last_became_peered":"2024-03-06T07:28:14.782382+0000","last_unstale":"2024-03-06T07:28:14.789637+0000","last_undegraded":"2024-03-06T07:28:14.789637+0000","last_fullsized":"2024-03-06T07:28:14.789637+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:19:07.331967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778436+0000","last_change":"2024-03-06T07:28:14.778436+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778436+0000","last_undegraded":"2024-03-06T07:28:14.778436+0000","last_fullsized":"2024-03-06T07:28:14.778436+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.777796+0000","last_change":"2024-03-06T07:28:14.776124+0000","last_active":"2024-03-06T07:28:14.777796+0000","last_peered":"2024-03-06T07:28:14.777796+0000","last_clean":"2024-03-06T07:28:14.777796+0000","last_became_active":"2024-03-06T07:28:14.775819+0000","last_became_peered":"2024-03-06T07:28:14.775819+0000","last_unstale":"2024-03-06T07:28:14.777796+0000","last_undegraded":"2024-03-06T07:28:14.777796+0000","last_fullsized":"2024-03-06T07:28:14.777796+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:30:20.528287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779244+0000","last_change":"2024-03-06T07:28:14.779244+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779244+0000","last_undegraded":"2024-03-06T07:28:14.779244+0000","last_fullsized":"2024-03-06T07:28:14.779244+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790821+0000","last_change":"2024-03-06T07:28:14.789097+0000","last_active":"2024-03-06T07:28:14.790821+0000","last_peered":"2024-03-06T07:28:14.790821+0000","last_clean":"2024-03-06T07:28:14.790821+0000","last_became_active":"2024-03-06T07:28:14.788833+0000","last_became_peered":"2024-03-06T07:28:14.788833+0000","last_unstale":"2024-03-06T07:28:14.790821+0000","last_undegraded":"2024-03-06T07:28:14.790821+0000","last_fullsized":"2024-03-06T07:28:14.790821+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:14.725218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779004+0000","last_change":"2024-03-06T07:28:14.779004+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779004+0000","last_undegraded":"2024-03-06T07:28:14.779004+0000","last_fullsized":"2024-03-06T07:28:14.779004+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789611+0000","last_change":"2024-03-06T07:28:14.787649+0000","last_active":"2024-03-06T07:28:14.789611+0000","last_peered":"2024-03-06T07:28:14.789611+0000","last_clean":"2024-03-06T07:28:14.789611+0000","last_became_active":"2024-03-06T07:28:14.787348+0000","last_became_peered":"2024-03-06T07:28:14.787348+0000","last_unstale":"2024-03-06T07:28:14.789611+0000","last_undegraded":"2024-03-06T07:28:14.789611+0000","last_fullsized":"2024-03-06T07:28:14.789611+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:38:13.142714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773103+0000","last_change":"2024-03-06T07:28:14.773103+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773103+0000","last_undegraded":"2024-03-06T07:28:14.773103+0000","last_fullsized":"2024-03-06T07:28:14.773103+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787342+0000","last_change":"2024-03-06T07:28:14.785132+0000","last_active":"2024-03-06T07:28:14.787342+0000","last_peered":"2024-03-06T07:28:14.787342+0000","last_clean":"2024-03-06T07:28:14.787342+0000","last_became_active":"2024-03-06T07:28:14.784849+0000","last_became_peered":"2024-03-06T07:28:14.784849+0000","last_unstale":"2024-03-06T07:28:14.787342+0000","last_undegraded":"2024-03-06T07:28:14.787342+0000","last_fullsized":"2024-03-06T07:28:14.787342+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:14:41.231832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777607+0000","last_change":"2024-03-06T07:28:14.777607+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777607+0000","last_undegraded":"2024-03-06T07:28:14.777607+0000","last_fullsized":"2024-03-06T07:28:14.777607+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786919+0000","last_change":"2024-03-06T07:28:14.785822+0000","last_active":"2024-03-06T07:28:14.786919+0000","last_peered":"2024-03-06T07:28:14.786919+0000","last_clean":"2024-03-06T07:28:14.786919+0000","last_became_active":"2024-03-06T07:28:14.785540+0000","last_became_peered":"2024-03-06T07:28:14.785540+0000","last_unstale":"2024-03-06T07:28:14.786919+0000","last_undegraded":"2024-03-06T07:28:14.786919+0000","last_fullsized":"2024-03-06T07:28:14.786919+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:03.928508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773152+0000","last_change":"2024-03-06T07:28:14.773152+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773152+0000","last_undegraded":"2024-03-06T07:28:14.773152+0000","last_fullsized":"2024-03-06T07:28:14.773152+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.785679+0000","last_change":"2024-03-06T07:28:14.784265+0000","last_active":"2024-03-06T07:28:14.785679+0000","last_peered":"2024-03-06T07:28:14.785679+0000","last_clean":"2024-03-06T07:28:14.785679+0000","last_became_active":"2024-03-06T07:28:14.783985+0000","last_became_peered":"2024-03-06T07:28:14.783985+0000","last_unstale":"2024-03-06T07:28:14.785679+0000","last_undegraded":"2024-03-06T07:28:14.785679+0000","last_fullsized":"2024-03-06T07:28:14.785679+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.428223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776788+0000","last_change":"2024-03-06T07:28:14.776788+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776788+0000","last_undegraded":"2024-03-06T07:28:14.776788+0000","last_fullsized":"2024-03-06T07:28:14.776788+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787370+0000","last_change":"2024-03-06T07:28:14.784209+0000","last_active":"2024-03-06T07:28:14.787370+0000","last_peered":"2024-03-06T07:28:14.787370+0000","last_clean":"2024-03-06T07:28:14.787370+0000","last_became_active":"2024-03-06T07:28:14.783932+0000","last_became_peered":"2024-03-06T07:28:14.783932+0000","last_unstale":"2024-03-06T07:28:14.787370+0000","last_undegraded":"2024-03-06T07:28:14.787370+0000","last_fullsized":"2024-03-06T07:28:14.787370+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:35.222311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773068+0000","last_change":"2024-03-06T07:28:14.773068+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773068+0000","last_undegraded":"2024-03-06T07:28:14.773068+0000","last_fullsized":"2024-03-06T07:28:14.773068+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791974+0000","last_change":"2024-03-06T07:28:14.787534+0000","last_active":"2024-03-06T07:28:14.791974+0000","last_peered":"2024-03-06T07:28:14.791974+0000","last_clean":"2024-03-06T07:28:14.791974+0000","last_became_active":"2024-03-06T07:28:14.787286+0000","last_became_peered":"2024-03-06T07:28:14.787286+0000","last_unstale":"2024-03-06T07:28:14.791974+0000","last_undegraded":"2024-03-06T07:28:14.791974+0000","last_fullsized":"2024-03-06T07:28:14.791974+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:58.265777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.778848+0000","last_change":"2024-03-06T07:28:14.778848+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.778848+0000","last_undegraded":"2024-03-06T07:28:14.778848+0000","last_fullsized":"2024-03-06T07:28:14.778848+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791451+0000","last_change":"2024-03-06T07:28:14.788682+0000","last_active":"2024-03-06T07:28:14.791451+0000","last_peered":"2024-03-06T07:28:14.791451+0000","last_clean":"2024-03-06T07:28:14.791451+0000","last_became_active":"2024-03-06T07:28:14.788381+0000","last_became_peered":"2024-03-06T07:28:14.788381+0000","last_unstale":"2024-03-06T07:28:14.791451+0000","last_undegraded":"2024-03-06T07:28:14.791451+0000","last_fullsized":"2024-03-06T07:28:14.791451+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:23:05.687550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.776586+0000","last_change":"2024-03-06T07:28:14.776586+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.776586+0000","last_undegraded":"2024-03-06T07:28:14.776586+0000","last_fullsized":"2024-03-06T07:28:14.776586+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789078+0000","last_change":"2024-03-06T07:28:14.787636+0000","last_active":"2024-03-06T07:28:14.789078+0000","last_peered":"2024-03-06T07:28:14.789078+0000","last_clean":"2024-03-06T07:28:14.789078+0000","last_became_active":"2024-03-06T07:28:14.787367+0000","last_became_peered":"2024-03-06T07:28:14.787367+0000","last_unstale":"2024-03-06T07:28:14.789078+0000","last_undegraded":"2024-03-06T07:28:14.789078+0000","last_fullsized":"2024-03-06T07:28:14.789078+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:21.492957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777633+0000","last_change":"2024-03-06T07:28:14.777633+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777633+0000","last_undegraded":"2024-03-06T07:28:14.777633+0000","last_fullsized":"2024-03-06T07:28:14.777633+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787834+0000","last_change":"2024-03-06T07:28:14.787083+0000","last_active":"2024-03-06T07:28:14.787834+0000","last_peered":"2024-03-06T07:28:14.787834+0000","last_clean":"2024-03-06T07:28:14.787834+0000","last_became_active":"2024-03-06T07:28:14.786827+0000","last_became_peered":"2024-03-06T07:28:14.786827+0000","last_unstale":"2024-03-06T07:28:14.787834+0000","last_undegraded":"2024-03-06T07:28:14.787834+0000","last_fullsized":"2024-03-06T07:28:14.787834+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:15.361243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774343+0000","last_change":"2024-03-06T07:28:14.774343+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774343+0000","last_undegraded":"2024-03-06T07:28:14.774343+0000","last_fullsized":"2024-03-06T07:28:14.774343+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791723+0000","last_change":"2024-03-06T07:28:14.788050+0000","last_active":"2024-03-06T07:28:14.791723+0000","last_peered":"2024-03-06T07:28:14.791723+0000","last_clean":"2024-03-06T07:28:14.791723+0000","last_became_active":"2024-03-06T07:28:14.787770+0000","last_became_peered":"2024-03-06T07:28:14.787770+0000","last_unstale":"2024-03-06T07:28:14.791723+0000","last_undegraded":"2024-03-06T07:28:14.791723+0000","last_fullsized":"2024-03-06T07:28:14.791723+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:42.511473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773686+0000","last_change":"2024-03-06T07:28:14.773686+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773686+0000","last_undegraded":"2024-03-06T07:28:14.773686+0000","last_fullsized":"2024-03-06T07:28:14.773686+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.783299+0000","last_change":"2024-03-06T07:28:14.782491+0000","last_active":"2024-03-06T07:28:14.783299+0000","last_peered":"2024-03-06T07:28:14.783299+0000","last_clean":"2024-03-06T07:28:14.783299+0000","last_became_active":"2024-03-06T07:28:14.782223+0000","last_became_peered":"2024-03-06T07:28:14.782223+0000","last_unstale":"2024-03-06T07:28:14.783299+0000","last_undegraded":"2024-03-06T07:28:14.783299+0000","last_fullsized":"2024-03-06T07:28:14.783299+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:55.179141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775653+0000","last_change":"2024-03-06T07:28:14.775653+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775653+0000","last_undegraded":"2024-03-06T07:28:14.775653+0000","last_fullsized":"2024-03-06T07:28:14.775653+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790941+0000","last_change":"2024-03-06T07:28:14.785927+0000","last_active":"2024-03-06T07:28:14.790941+0000","last_peered":"2024-03-06T07:28:14.790941+0000","last_clean":"2024-03-06T07:28:14.790941+0000","last_became_active":"2024-03-06T07:28:14.785666+0000","last_became_peered":"2024-03-06T07:28:14.785666+0000","last_unstale":"2024-03-06T07:28:14.790941+0000","last_undegraded":"2024-03-06T07:28:14.790941+0000","last_fullsized":"2024-03-06T07:28:14.790941+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.036054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773460+0000","last_change":"2024-03-06T07:28:14.773460+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773460+0000","last_undegraded":"2024-03-06T07:28:14.773460+0000","last_fullsized":"2024-03-06T07:28:14.773460+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.791763+0000","last_change":"2024-03-06T07:28:14.788414+0000","last_active":"2024-03-06T07:28:14.791763+0000","last_peered":"2024-03-06T07:28:14.791763+0000","last_clean":"2024-03-06T07:28:14.791763+0000","last_became_active":"2024-03-06T07:28:14.788047+0000","last_became_peered":"2024-03-06T07:28:14.788047+0000","last_unstale":"2024-03-06T07:28:14.791763+0000","last_undegraded":"2024-03-06T07:28:14.791763+0000","last_fullsized":"2024-03-06T07:28:14.791763+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:46.615680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.774832+0000","last_change":"2024-03-06T07:28:14.774832+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.774832+0000","last_undegraded":"2024-03-06T07:28:14.774832+0000","last_fullsized":"2024-03-06T07:28:14.774832+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787528+0000","last_change":"2024-03-06T07:28:14.782233+0000","last_active":"2024-03-06T07:28:14.787528+0000","last_peered":"2024-03-06T07:28:14.787528+0000","last_clean":"2024-03-06T07:28:14.787528+0000","last_became_active":"2024-03-06T07:28:14.781927+0000","last_became_peered":"2024-03-06T07:28:14.781927+0000","last_unstale":"2024-03-06T07:28:14.787528+0000","last_undegraded":"2024-03-06T07:28:14.787528+0000","last_fullsized":"2024-03-06T07:28:14.787528+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:48.443105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779417+0000","last_change":"2024-03-06T07:28:14.779417+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779417+0000","last_undegraded":"2024-03-06T07:28:14.779417+0000","last_fullsized":"2024-03-06T07:28:14.779417+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.787170+0000","last_change":"2024-03-06T07:28:14.785960+0000","last_active":"2024-03-06T07:28:14.787170+0000","last_peered":"2024-03-06T07:28:14.787170+0000","last_clean":"2024-03-06T07:28:14.787170+0000","last_became_active":"2024-03-06T07:28:14.785692+0000","last_became_peered":"2024-03-06T07:28:14.785692+0000","last_unstale":"2024-03-06T07:28:14.787170+0000","last_undegraded":"2024-03-06T07:28:14.787170+0000","last_fullsized":"2024-03-06T07:28:14.787170+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:04.218118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.779640+0000","last_change":"2024-03-06T07:28:14.779640+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.779640+0000","last_undegraded":"2024-03-06T07:28:14.779640+0000","last_fullsized":"2024-03-06T07:28:14.779640+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789580+0000","last_change":"2024-03-06T07:28:14.784580+0000","last_active":"2024-03-06T07:28:14.789580+0000","last_peered":"2024-03-06T07:28:14.789580+0000","last_clean":"2024-03-06T07:28:14.789580+0000","last_became_active":"2024-03-06T07:28:14.784245+0000","last_became_peered":"2024-03-06T07:28:14.784245+0000","last_unstale":"2024-03-06T07:28:14.789580+0000","last_undegraded":"2024-03-06T07:28:14.789580+0000","last_fullsized":"2024-03-06T07:28:14.789580+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:10.104421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773331+0000","last_change":"2024-03-06T07:28:14.773331+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773331+0000","last_undegraded":"2024-03-06T07:28:14.773331+0000","last_fullsized":"2024-03-06T07:28:14.773331+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789348+0000","last_change":"2024-03-06T07:28:14.786720+0000","last_active":"2024-03-06T07:28:14.789348+0000","last_peered":"2024-03-06T07:28:14.789348+0000","last_clean":"2024-03-06T07:28:14.789348+0000","last_became_active":"2024-03-06T07:28:14.786452+0000","last_became_peered":"2024-03-06T07:28:14.786452+0000","last_unstale":"2024-03-06T07:28:14.789348+0000","last_undegraded":"2024-03-06T07:28:14.789348+0000","last_fullsized":"2024-03-06T07:28:14.789348+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:51.046635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773547+0000","last_change":"2024-03-06T07:28:14.773547+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773547+0000","last_undegraded":"2024-03-06T07:28:14.773547+0000","last_fullsized":"2024-03-06T07:28:14.773547+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.790155+0000","last_change":"2024-03-06T07:28:14.789522+0000","last_active":"2024-03-06T07:28:14.790155+0000","last_peered":"2024-03-06T07:28:14.790155+0000","last_clean":"2024-03-06T07:28:14.790155+0000","last_became_active":"2024-03-06T07:28:14.789267+0000","last_became_peered":"2024-03-06T07:28:14.789267+0000","last_unstale":"2024-03-06T07:28:14.790155+0000","last_undegraded":"2024-03-06T07:28:14.790155+0000","last_fullsized":"2024-03-06T07:28:14.790155+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:41.428236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.777263+0000","last_change":"2024-03-06T07:28:14.777263+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.777263+0000","last_undegraded":"2024-03-06T07:28:14.777263+0000","last_fullsized":"2024-03-06T07:28:14.777263+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789903+0000","last_change":"2024-03-06T07:28:14.788385+0000","last_active":"2024-03-06T07:28:14.789903+0000","last_peered":"2024-03-06T07:28:14.789903+0000","last_clean":"2024-03-06T07:28:14.789903+0000","last_became_active":"2024-03-06T07:28:14.788113+0000","last_became_peered":"2024-03-06T07:28:14.788113+0000","last_unstale":"2024-03-06T07:28:14.789903+0000","last_undegraded":"2024-03-06T07:28:14.789903+0000","last_fullsized":"2024-03-06T07:28:14.789903+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:22:50.463171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.782151+0000","last_change":"2024-03-06T07:28:14.782151+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.782151+0000","last_undegraded":"2024-03-06T07:28:14.782151+0000","last_fullsized":"2024-03-06T07:28:14.782151+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.794736+0000","last_change":"2024-03-06T07:28:14.792256+0000","last_active":"2024-03-06T07:28:14.794736+0000","last_peered":"2024-03-06T07:28:14.794736+0000","last_clean":"2024-03-06T07:28:14.794736+0000","last_became_active":"2024-03-06T07:28:14.792032+0000","last_became_peered":"2024-03-06T07:28:14.792032+0000","last_unstale":"2024-03-06T07:28:14.794736+0000","last_undegraded":"2024-03-06T07:28:14.794736+0000","last_fullsized":"2024-03-06T07:28:14.794736+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:20.997172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775584+0000","last_change":"2024-03-06T07:28:14.775584+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775584+0000","last_undegraded":"2024-03-06T07:28:14.775584+0000","last_fullsized":"2024-03-06T07:28:14.775584+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.792007+0000","last_change":"2024-03-06T07:28:14.788477+0000","last_active":"2024-03-06T07:28:14.792007+0000","last_peered":"2024-03-06T07:28:14.792007+0000","last_clean":"2024-03-06T07:28:14.792007+0000","last_became_active":"2024-03-06T07:28:14.788223+0000","last_became_peered":"2024-03-06T07:28:14.788223+0000","last_unstale":"2024-03-06T07:28:14.792007+0000","last_undegraded":"2024-03-06T07:28:14.792007+0000","last_fullsized":"2024-03-06T07:28:14.792007+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:52.802944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.773666+0000","last_change":"2024-03-06T07:28:14.773666+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.773666+0000","last_undegraded":"2024-03-06T07:28:14.773666+0000","last_fullsized":"2024-03-06T07:28:14.773666+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789661+0000","last_change":"2024-03-06T07:28:14.788356+0000","last_active":"2024-03-06T07:28:14.789661+0000","last_peered":"2024-03-06T07:28:14.789661+0000","last_clean":"2024-03-06T07:28:14.789661+0000","last_became_active":"2024-03-06T07:28:14.788093+0000","last_became_peered":"2024-03-06T07:28:14.788093+0000","last_unstale":"2024-03-06T07:28:14.789661+0000","last_undegraded":"2024-03-06T07:28:14.789661+0000","last_fullsized":"2024-03-06T07:28:14.789661+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:31.566144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.780498+0000","last_change":"2024-03-06T07:28:14.780498+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.780498+0000","last_undegraded":"2024-03-06T07:28:14.780498+0000","last_fullsized":"2024-03-06T07:28:14.780498+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.789297+0000","last_change":"2024-03-06T07:28:14.786359+0000","last_active":"2024-03-06T07:28:14.789297+0000","last_peered":"2024-03-06T07:28:14.789297+0000","last_clean":"2024-03-06T07:28:14.789297+0000","last_became_active":"2024-03-06T07:28:14.786068+0000","last_became_peered":"2024-03-06T07:28:14.786068+0000","last_unstale":"2024-03-06T07:28:14.789297+0000","last_undegraded":"2024-03-06T07:28:14.789297+0000","last_fullsized":"2024-03-06T07:28:14.789297+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:02:10.545981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2024-03-06T07:28:14.775080+0000","last_change":"2024-03-06T07:28:14.775080+0000","last_active":"2024-03-06T07:28:14.764755+0000","last_peered":"2024-03-06T07:28:14.764755+0000","last_clean":"2024-03-06T07:28:14.764755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:28:14.775080+0000","last_undegraded":"2024-03-06T07:28:14.775080+0000","last_fullsized":"2024-03-06T07:28:14.775080+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.786611+0000","last_change":"2024-03-06T07:28:14.785657+0000","last_active":"2024-03-06T07:28:14.786611+0000","last_peered":"2024-03-06T07:28:14.786611+0000","last_clean":"2024-03-06T07:28:14.786611+0000","last_became_active":"2024-03-06T07:28:14.785292+0000","last_became_peered":"2024-03-06T07:28:14.785292+0000","last_unstale":"2024-03-06T07:28:14.786611+0000","last_undegraded":"2024-03-06T07:28:14.786611+0000","last_fullsized":"2024-03-06T07:28:14.786611+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:21.053100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":498,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-03-06T07:28:14.774290+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:28:14.774290+0000","last_peered":"2024-03-06T07:28:14.774290+0000","last_clean":"2024-03-06T07:28:14.774290+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:28:14.774290+0000","last_undegraded":"2024-03-06T07:28:14.774290+0000","last_fullsized":"2024-03-06T07:28:14.774290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2539520,"data_stored":1790994,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33612,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338228,"statfs":{"total":96636764160,"available":96602345472,"internally_reserved":0,"allocated":2486272,"data_stored":1207293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33612,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338228,"statfs":{"total":96636764160,"available":96602345472,"internally_reserved":0,"allocated":2486272,"data_stored":1207293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37764,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334076,"statfs":{"total":96636764160,"available":96598093824,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2473984,"data_stored":1200626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2539520,"data_stored":1790994,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:28:20.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1f is not active+clean 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773969+0000', 'last_change': '2024-03-06T07:28:14.773969+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773969+0000', 'last_undegraded': '2024-03-06T07:28:14.773969+0000', 'last_fullsized': '2024-03-06T07:28:14.773969+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1e is not active+clean 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.774111+0000', 'last_change': '2024-03-06T07:28:14.774111+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.774111+0000', 'last_undegraded': '2024-03-06T07:28:14.774111+0000', 'last_fullsized': '2024-03-06T07:28:14.774111+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1d is not active+clean 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776824+0000', 'last_change': '2024-03-06T07:28:14.776824+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776824+0000', 'last_undegraded': '2024-03-06T07:28:14.776824+0000', 'last_fullsized': '2024-03-06T07:28:14.776824+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1c is not active+clean 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.774117+0000', 'last_change': '2024-03-06T07:28:14.774117+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.774117+0000', 'last_undegraded': '2024-03-06T07:28:14.774117+0000', 'last_fullsized': '2024-03-06T07:28:14.774117+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1b is not active+clean 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776834+0000', 'last_change': '2024-03-06T07:28:14.776834+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776834+0000', 'last_undegraded': '2024-03-06T07:28:14.776834+0000', 'last_fullsized': '2024-03-06T07:28:14.776834+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1a is not active+clean 2024-03-06T07:28:20.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773087+0000', 'last_change': '2024-03-06T07:28:14.773087+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773087+0000', 'last_undegraded': '2024-03-06T07:28:14.773087+0000', 'last_fullsized': '2024-03-06T07:28:14.773087+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.19 is not active+clean 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.775879+0000', 'last_change': '2024-03-06T07:28:14.775879+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.775879+0000', 'last_undegraded': '2024-03-06T07:28:14.775879+0000', 'last_fullsized': '2024-03-06T07:28:14.775879+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.18 is not active+clean 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.775942+0000', 'last_change': '2024-03-06T07:28:14.775942+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.775942+0000', 'last_undegraded': '2024-03-06T07:28:14.775942+0000', 'last_fullsized': '2024-03-06T07:28:14.775942+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.17 is not active+clean 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773504+0000', 'last_change': '2024-03-06T07:28:14.773504+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773504+0000', 'last_undegraded': '2024-03-06T07:28:14.773504+0000', 'last_fullsized': '2024-03-06T07:28:14.773504+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.16 is not active+clean 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776672+0000', 'last_change': '2024-03-06T07:28:14.776672+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776672+0000', 'last_undegraded': '2024-03-06T07:28:14.776672+0000', 'last_fullsized': '2024-03-06T07:28:14.776672+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.15 is not active+clean 2024-03-06T07:28:20.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776343+0000', 'last_change': '2024-03-06T07:28:14.776343+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776343+0000', 'last_undegraded': '2024-03-06T07:28:14.776343+0000', 'last_fullsized': '2024-03-06T07:28:14.776343+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.14 is not active+clean 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.772516+0000', 'last_change': '2024-03-06T07:28:14.772516+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.772516+0000', 'last_undegraded': '2024-03-06T07:28:14.772516+0000', 'last_fullsized': '2024-03-06T07:28:14.772516+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.13 is not active+clean 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.775564+0000', 'last_change': '2024-03-06T07:28:14.775564+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.775564+0000', 'last_undegraded': '2024-03-06T07:28:14.775564+0000', 'last_fullsized': '2024-03-06T07:28:14.775564+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.12 is not active+clean 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.778948+0000', 'last_change': '2024-03-06T07:28:14.778948+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.778948+0000', 'last_undegraded': '2024-03-06T07:28:14.778948+0000', 'last_fullsized': '2024-03-06T07:28:14.778948+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.11 is not active+clean 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.772911+0000', 'last_change': '2024-03-06T07:28:14.772911+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.772911+0000', 'last_undegraded': '2024-03-06T07:28:14.772911+0000', 'last_fullsized': '2024-03-06T07:28:14.772911+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.10 is not active+clean 2024-03-06T07:28:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.772552+0000', 'last_change': '2024-03-06T07:28:14.772552+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.772552+0000', 'last_undegraded': '2024-03-06T07:28:14.772552+0000', 'last_fullsized': '2024-03-06T07:28:14.772552+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.f is not active+clean 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.777866+0000', 'last_change': '2024-03-06T07:28:14.777866+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.777866+0000', 'last_undegraded': '2024-03-06T07:28:14.777866+0000', 'last_fullsized': '2024-03-06T07:28:14.777866+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.e is not active+clean 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773906+0000', 'last_change': '2024-03-06T07:28:14.773906+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773906+0000', 'last_undegraded': '2024-03-06T07:28:14.773906+0000', 'last_fullsized': '2024-03-06T07:28:14.773906+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.d is not active+clean 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773952+0000', 'last_change': '2024-03-06T07:28:14.773952+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773952+0000', 'last_undegraded': '2024-03-06T07:28:14.773952+0000', 'last_fullsized': '2024-03-06T07:28:14.773952+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.c is not active+clean 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.772662+0000', 'last_change': '2024-03-06T07:28:14.772662+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.772662+0000', 'last_undegraded': '2024-03-06T07:28:14.772662+0000', 'last_fullsized': '2024-03-06T07:28:14.772662+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.b is not active+clean 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.777224+0000', 'last_change': '2024-03-06T07:28:14.777224+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.777224+0000', 'last_undegraded': '2024-03-06T07:28:14.777224+0000', 'last_fullsized': '2024-03-06T07:28:14.777224+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.a is not active+clean 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.775345+0000', 'last_change': '2024-03-06T07:28:14.775345+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.775345+0000', 'last_undegraded': '2024-03-06T07:28:14.775345+0000', 'last_fullsized': '2024-03-06T07:28:14.775345+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.9 is not active+clean 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.772357+0000', 'last_change': '2024-03-06T07:28:14.772357+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.772357+0000', 'last_undegraded': '2024-03-06T07:28:14.772357+0000', 'last_fullsized': '2024-03-06T07:28:14.772357+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.8 is not active+clean 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.777328+0000', 'last_change': '2024-03-06T07:28:14.777328+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.777328+0000', 'last_undegraded': '2024-03-06T07:28:14.777328+0000', 'last_fullsized': '2024-03-06T07:28:14.777328+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.7 is not active+clean 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773893+0000', 'last_change': '2024-03-06T07:28:14.773893+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773893+0000', 'last_undegraded': '2024-03-06T07:28:14.773893+0000', 'last_fullsized': '2024-03-06T07:28:14.773893+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.6 is not active+clean 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.778348+0000', 'last_change': '2024-03-06T07:28:14.778348+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.778348+0000', 'last_undegraded': '2024-03-06T07:28:14.778348+0000', 'last_fullsized': '2024-03-06T07:28:14.778348+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.5 is not active+clean 2024-03-06T07:28:20.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.778695+0000', 'last_change': '2024-03-06T07:28:14.778695+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.778695+0000', 'last_undegraded': '2024-03-06T07:28:14.778695+0000', 'last_fullsized': '2024-03-06T07:28:14.778695+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.4 is not active+clean 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.771877+0000', 'last_change': '2024-03-06T07:28:14.771877+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.771877+0000', 'last_undegraded': '2024-03-06T07:28:14.771877+0000', 'last_fullsized': '2024-03-06T07:28:14.771877+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3 is not active+clean 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.779909+0000', 'last_change': '2024-03-06T07:28:14.779909+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.779909+0000', 'last_undegraded': '2024-03-06T07:28:14.779909+0000', 'last_fullsized': '2024-03-06T07:28:14.779909+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2 is not active+clean 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.778508+0000', 'last_change': '2024-03-06T07:28:14.778508+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.778508+0000', 'last_undegraded': '2024-03-06T07:28:14.778508+0000', 'last_fullsized': '2024-03-06T07:28:14.778508+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1 is not active+clean 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773904+0000', 'last_change': '2024-03-06T07:28:14.773904+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773904+0000', 'last_undegraded': '2024-03-06T07:28:14.773904+0000', 'last_fullsized': '2024-03-06T07:28:14.773904+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.0 is not active+clean 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.771975+0000', 'last_change': '2024-03-06T07:28:14.771975+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.771975+0000', 'last_undegraded': '2024-03-06T07:28:14.771975+0000', 'last_fullsized': '2024-03-06T07:28:14.771975+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.20 is not active+clean 2024-03-06T07:28:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773009+0000', 'last_change': '2024-03-06T07:28:14.773009+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773009+0000', 'last_undegraded': '2024-03-06T07:28:14.773009+0000', 'last_fullsized': '2024-03-06T07:28:14.773009+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.22 is not active+clean 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.772471+0000', 'last_change': '2024-03-06T07:28:14.772471+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.772471+0000', 'last_undegraded': '2024-03-06T07:28:14.772471+0000', 'last_fullsized': '2024-03-06T07:28:14.772471+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.23 is not active+clean 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776275+0000', 'last_change': '2024-03-06T07:28:14.776275+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776275+0000', 'last_undegraded': '2024-03-06T07:28:14.776275+0000', 'last_fullsized': '2024-03-06T07:28:14.776275+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.24 is not active+clean 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.781725+0000', 'last_change': '2024-03-06T07:28:14.781725+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.781725+0000', 'last_undegraded': '2024-03-06T07:28:14.781725+0000', 'last_fullsized': '2024-03-06T07:28:14.781725+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.25 is not active+clean 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776389+0000', 'last_change': '2024-03-06T07:28:14.776389+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776389+0000', 'last_undegraded': '2024-03-06T07:28:14.776389+0000', 'last_fullsized': '2024-03-06T07:28:14.776389+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.26 is not active+clean 2024-03-06T07:28:20.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.777088+0000', 'last_change': '2024-03-06T07:28:14.777088+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.777088+0000', 'last_undegraded': '2024-03-06T07:28:14.777088+0000', 'last_fullsized': '2024-03-06T07:28:14.777088+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.27 is not active+clean 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.778436+0000', 'last_change': '2024-03-06T07:28:14.778436+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.778436+0000', 'last_undegraded': '2024-03-06T07:28:14.778436+0000', 'last_fullsized': '2024-03-06T07:28:14.778436+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.28 is not active+clean 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.779244+0000', 'last_change': '2024-03-06T07:28:14.779244+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.779244+0000', 'last_undegraded': '2024-03-06T07:28:14.779244+0000', 'last_fullsized': '2024-03-06T07:28:14.779244+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.29 is not active+clean 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.779004+0000', 'last_change': '2024-03-06T07:28:14.779004+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.779004+0000', 'last_undegraded': '2024-03-06T07:28:14.779004+0000', 'last_fullsized': '2024-03-06T07:28:14.779004+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2a is not active+clean 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773103+0000', 'last_change': '2024-03-06T07:28:14.773103+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773103+0000', 'last_undegraded': '2024-03-06T07:28:14.773103+0000', 'last_fullsized': '2024-03-06T07:28:14.773103+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2b is not active+clean 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.777607+0000', 'last_change': '2024-03-06T07:28:14.777607+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.777607+0000', 'last_undegraded': '2024-03-06T07:28:14.777607+0000', 'last_fullsized': '2024-03-06T07:28:14.777607+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2c is not active+clean 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773152+0000', 'last_change': '2024-03-06T07:28:14.773152+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773152+0000', 'last_undegraded': '2024-03-06T07:28:14.773152+0000', 'last_fullsized': '2024-03-06T07:28:14.773152+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2d is not active+clean 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776788+0000', 'last_change': '2024-03-06T07:28:14.776788+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776788+0000', 'last_undegraded': '2024-03-06T07:28:14.776788+0000', 'last_fullsized': '2024-03-06T07:28:14.776788+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2e is not active+clean 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773068+0000', 'last_change': '2024-03-06T07:28:14.773068+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773068+0000', 'last_undegraded': '2024-03-06T07:28:14.773068+0000', 'last_fullsized': '2024-03-06T07:28:14.773068+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2f is not active+clean 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.778848+0000', 'last_change': '2024-03-06T07:28:14.778848+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.778848+0000', 'last_undegraded': '2024-03-06T07:28:14.778848+0000', 'last_fullsized': '2024-03-06T07:28:14.778848+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.30 is not active+clean 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.776586+0000', 'last_change': '2024-03-06T07:28:14.776586+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.776586+0000', 'last_undegraded': '2024-03-06T07:28:14.776586+0000', 'last_fullsized': '2024-03-06T07:28:14.776586+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.31 is not active+clean 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.777633+0000', 'last_change': '2024-03-06T07:28:14.777633+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.777633+0000', 'last_undegraded': '2024-03-06T07:28:14.777633+0000', 'last_fullsized': '2024-03-06T07:28:14.777633+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.32 is not active+clean 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.774343+0000', 'last_change': '2024-03-06T07:28:14.774343+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.774343+0000', 'last_undegraded': '2024-03-06T07:28:14.774343+0000', 'last_fullsized': '2024-03-06T07:28:14.774343+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.33 is not active+clean 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773686+0000', 'last_change': '2024-03-06T07:28:14.773686+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773686+0000', 'last_undegraded': '2024-03-06T07:28:14.773686+0000', 'last_fullsized': '2024-03-06T07:28:14.773686+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.34 is not active+clean 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.775653+0000', 'last_change': '2024-03-06T07:28:14.775653+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.775653+0000', 'last_undegraded': '2024-03-06T07:28:14.775653+0000', 'last_fullsized': '2024-03-06T07:28:14.775653+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.35 is not active+clean 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773460+0000', 'last_change': '2024-03-06T07:28:14.773460+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773460+0000', 'last_undegraded': '2024-03-06T07:28:14.773460+0000', 'last_fullsized': '2024-03-06T07:28:14.773460+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.36 is not active+clean 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.774832+0000', 'last_change': '2024-03-06T07:28:14.774832+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.774832+0000', 'last_undegraded': '2024-03-06T07:28:14.774832+0000', 'last_fullsized': '2024-03-06T07:28:14.774832+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.37 is not active+clean 2024-03-06T07:28:20.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.779417+0000', 'last_change': '2024-03-06T07:28:14.779417+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.779417+0000', 'last_undegraded': '2024-03-06T07:28:14.779417+0000', 'last_fullsized': '2024-03-06T07:28:14.779417+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.38 is not active+clean 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.779640+0000', 'last_change': '2024-03-06T07:28:14.779640+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.779640+0000', 'last_undegraded': '2024-03-06T07:28:14.779640+0000', 'last_fullsized': '2024-03-06T07:28:14.779640+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.39 is not active+clean 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773331+0000', 'last_change': '2024-03-06T07:28:14.773331+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773331+0000', 'last_undegraded': '2024-03-06T07:28:14.773331+0000', 'last_fullsized': '2024-03-06T07:28:14.773331+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3a is not active+clean 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773547+0000', 'last_change': '2024-03-06T07:28:14.773547+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773547+0000', 'last_undegraded': '2024-03-06T07:28:14.773547+0000', 'last_fullsized': '2024-03-06T07:28:14.773547+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3b is not active+clean 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.777263+0000', 'last_change': '2024-03-06T07:28:14.777263+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.777263+0000', 'last_undegraded': '2024-03-06T07:28:14.777263+0000', 'last_fullsized': '2024-03-06T07:28:14.777263+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3c is not active+clean 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.782151+0000', 'last_change': '2024-03-06T07:28:14.782151+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.782151+0000', 'last_undegraded': '2024-03-06T07:28:14.782151+0000', 'last_fullsized': '2024-03-06T07:28:14.782151+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:28:20.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3d is not active+clean 2024-03-06T07:28:20.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.775584+0000', 'last_change': '2024-03-06T07:28:14.775584+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.775584+0000', 'last_undegraded': '2024-03-06T07:28:14.775584+0000', 'last_fullsized': '2024-03-06T07:28:14.775584+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3e is not active+clean 2024-03-06T07:28:20.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.773666+0000', 'last_change': '2024-03-06T07:28:14.773666+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.773666+0000', 'last_undegraded': '2024-03-06T07:28:14.773666+0000', 'last_fullsized': '2024-03-06T07:28:14.773666+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:28:20.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3f is not active+clean 2024-03-06T07:28:20.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.780498+0000', 'last_change': '2024-03-06T07:28:14.780498+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.780498+0000', 'last_undegraded': '2024-03-06T07:28:14.780498+0000', 'last_fullsized': '2024-03-06T07:28:14.780498+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:28:20.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.21 is not active+clean 2024-03-06T07:28:20.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 196, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:28:14.775080+0000', 'last_change': '2024-03-06T07:28:14.775080+0000', 'last_active': '2024-03-06T07:28:14.764755+0000', 'last_peered': '2024-03-06T07:28:14.764755+0000', 'last_clean': '2024-03-06T07:28:14.764755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:28:14.775080+0000', 'last_undegraded': '2024-03-06T07:28:14.775080+0000', 'last_fullsized': '2024-03-06T07:28:14.775080+0000', 'mapping_epoch': 196, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 196, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'last_clean_scrub_stamp': '2024-03-06T07:28:14.764755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:28:20.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:28:20.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.646+0000 7fae96d8c640 1 Processor -- start 2024-03-06T07:28:20.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.646+0000 7fae96d8c640 1 -- start start 2024-03-06T07:28:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae96d8c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 0x7fae900c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae96d8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae901572f0 con 0x7fae90153990 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae96d8c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae900ad310 con 0x7fae901506b0 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae96d8c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae90156450 con 0x7fae90156ea0 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae8ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae90153990 0x7fae900acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57726/0 (socket says 172.21.15.26:57726) 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae8ffff640 1 -- 172.21.15.26:0/481287488 learned_addr learned my addr 172.21.15.26:0/481287488 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae95329640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 0x7fae900c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364885243 0 0) 0x7fae901572f0 con 0x7fae90153990 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.647+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae78003520 con 0x7fae90153990 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397173388 0 0) 0x7fae78003520 con 0x7fae90153990 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 msgr2=0x7fae900c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 0x7fae900c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 >> v1:172.21.15.117:6789/0 conn(0x7fae901506b0 legacy=0x7fae900702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae90070aa0 con 0x7fae90153990 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fae84001690 con 0x7fae90153990 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 >> v1:172.21.15.26:6789/0 conn(0x7fae90153990 legacy=0x7fae900acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 0x7fae780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae78003f20 0x7fae78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae78004d40 0x7fae780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae90156450 con 0x7fae78004d40 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7800ba20 con 0x7fae78003f20 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7800bc20 con 0x7fae90156ea0 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae95329640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae78004d40 0x7fae780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae94b28640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 0x7fae780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae8ffff640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae78003f20 0x7fae78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae95329640 1 -- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 msgr2=0x7fae780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae95329640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae90156ea0 0x7fae780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae95329640 1 -- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae78003f20 msgr2=0x7fae78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae95329640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae78003f20 0x7fae78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.648+0000 7fae95329640 1 -- 172.21.15.26:0/481287488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae78003520 con 0x7fae78004d40 2024-03-06T07:28:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.649+0000 7fae95329640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae78004d40 0x7fae780051c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fae88009fc0 tx=0x7fae78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c75a468cd255b53 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.649+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88039070 con 0x7fae78004d40 2024-03-06T07:28:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.649+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8802cc20 con 0x7fae78004d40 2024-03-06T07:28:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.649+0000 7fae8f7fe640 1 -- 172.21.15.26:0/481287488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88009c10 con 0x7fae78004d40 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.650+0000 7fae96d8c640 1 -- 172.21.15.26:0/481287488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae78004d40 msgr2=0x7fae780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.650+0000 7fae96d8c640 1 --2- 172.21.15.26:0/481287488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae78004d40 0x7fae780051c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fae88009fc0 tx=0x7fae78005830 comp rx=0 tx=0).stop 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.650+0000 7fae95329640 1 -- 172.21.15.26:0/481287488 reap_dead start 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.650+0000 7fae96d8c640 1 -- 172.21.15.26:0/481287488 shutdown_connections 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.650+0000 7fae96d8c640 1 -- 172.21.15.26:0/481287488 >> 172.21.15.26:0/481287488 conn(0x7fae9006b1a0 msgr2=0x7fae9006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.650+0000 7fae96d8c640 1 -- 172.21.15.26:0/481287488 shutdown_connections 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.650+0000 7fae96d8c640 1 -- 172.21.15.26:0/481287488 wait complete. 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.651+0000 7fae96d8c640 1 Processor -- start 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.651+0000 7fae96d8c640 1 -- start start 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae900c1d60 0x7fae9014a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae901eaac0 0x7fae901eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae90146ab0 0x7fae90146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae901566a0 con 0x7fae900c1d60 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae90156a10 con 0x7fae90146ab0 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae900c54a0 con 0x7fae901eaac0 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae900c1d60 0x7fae9014a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae900c1d60 0x7fae9014a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43962/0 (socket says 172.21.15.26:43962) 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 -- 172.21.15.26:0/4139205033 learned_addr learned my addr 172.21.15.26:0/4139205033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae8ffff640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae901eaac0 0x7fae901eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae95329640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae90146ab0 0x7fae90146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 -- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae901eaac0 msgr2=0x7fae901eaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae901eaac0 0x7fae901eaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 -- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae90146ab0 msgr2=0x7fae90146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae90146ab0 0x7fae90146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 -- 172.21.15.26:0/4139205033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae901eb480 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae95329640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae90146ab0 0x7fae90146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae94b28640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae900c1d60 0x7fae9014a950 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fae8000f890 tx=0x7fae8000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae8d7fa640 1 -- 172.21.15.26:0/4139205033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000fe80 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae8d7fa640 1 -- 172.21.15.26:0/4139205033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80018070 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9006dea0 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae8d7fa640 1 -- 172.21.15.26:0/4139205033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000d6d0 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.652+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9014c4e0 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.653+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9004f330 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.654+0000 7fae8d7fa640 1 -- 172.21.15.26:0/4139205033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae8000d870 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.655+0000 7fae8d7fa640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae7c06b4d0 0x7fae7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.655+0000 7fae8d7fa640 1 -- 172.21.15.26:0/4139205033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae8008a240 con 0x7fae900c1d60 2024-03-06T07:28:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.655+0000 7fae8ffff640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae7c06b4d0 0x7fae7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.655+0000 7fae8ffff640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae7c06b4d0 0x7fae7c06d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fae900c3580 tx=0x7fae840078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:20.659+0000 7fae8d7fa640 1 -- 172.21.15.26:0/4139205033 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae80058f20 con 0x7fae900c1d60 2024-03-06T07:28:21.068 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-03-06T07:28:21.066+0000 7fe5ccab5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:28:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.077+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fae900700a0 con 0x7fae900c1d60 2024-03-06T07:28:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.078+0000 7fae8d7fa640 1 -- 172.21.15.26:0/4139205033 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1330 (secure 0 0 0) 0x7fae80058820 con 0x7fae900c1d60 2024-03-06T07:28:21.079 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:21.079 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":516,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":198,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709595,"num_in_osds":8,"osd_in_since":1709709586,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":288169984,"bytes_avail":772805943296,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":188,"id":15,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":15,"rank":0,"name":"a","status":"up:active","gid":10338}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-03-06T07:28:05.132583+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.079+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae7c06b4d0 msgr2=0x7fae7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae7c06b4d0 0x7fae7c06d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fae900c3580 tx=0x7fae840078f0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae900c1d60 msgr2=0x7fae9014a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae900c1d60 0x7fae9014a950 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fae8000f890 tx=0x7fae8000fc90 comp rx=0 tx=0).stop 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 shutdown_connections 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fae7c06b4d0 0x7fae7c06d990 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae90146ab0 0x7fae90146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae901eaac0 0x7fae901eaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 --2- 172.21.15.26:0/4139205033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae900c1d60 0x7fae9014a950 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 >> 172.21.15.26:0/4139205033 conn(0x7fae9006b1a0 msgr2=0x7fae90152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 shutdown_connections 2024-03-06T07:28:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.080+0000 7fae96d8c640 1 -- 172.21.15.26:0/4139205033 wait complete. 2024-03-06T07:28:21.093 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:28:21.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:28:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.193+0000 7f0f84e6d640 1 Processor -- start 2024-03-06T07:28:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.193+0000 7f0f84e6d640 1 -- start start 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f84e6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f80155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f84e6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f800c5860 0x7f0f8006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f84e6d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f800c5cc0 con 0x7f0f801586f0 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f84e6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8006df40 con 0x7f0f800c5860 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f84e6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801591c0 con 0x7f0f800c5480 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f800c5860 0x7f0f8006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f77fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f800c5860 0x7f0f8006f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43972/0 (socket says 172.21.15.26:43972) 2024-03-06T07:28:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f77fff640 1 -- 172.21.15.26:0/3483391155 learned_addr learned my addr 172.21.15.26:0/3483391155 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f7e59c640 1 --2- 172.21.15.26:0/3483391155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f80155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f77fff640 1 -- 172.21.15.26:0/3483391155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 msgr2=0x7f0f80155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f77fff640 1 --2- 172.21.15.26:0/3483391155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f80155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f77fff640 1 -- 172.21.15.26:0/3483391155 >> v1:172.21.15.117:6789/0 conn(0x7f0f801586f0 legacy=0x7f0f800724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.194+0000 7f0f77fff640 1 -- 172.21.15.26:0/3483391155 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f801594a0 con 0x7f0f800c5860 2024-03-06T07:28:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.195+0000 7f0f77fff640 1 --2- 172.21.15.26:0/3483391155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f800c5860 0x7f0f8006f6a0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009870 tx=0x7f0f6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f67ee4b22fc9aba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f7dd9b640 1 -- 172.21.15.26:0/3483391155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68035020 con 0x7f0f800c5860 2024-03-06T07:28:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f7dd9b640 1 -- 172.21.15.26:0/3483391155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68031640 con 0x7f0f800c5860 2024-03-06T07:28:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f7dd9b640 1 -- 172.21.15.26:0/3483391155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6802d040 con 0x7f0f800c5860 2024-03-06T07:28:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3483391155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f800c5860 msgr2=0x7f0f8006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3483391155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f800c5860 0x7f0f8006f6a0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009870 tx=0x7f0f6802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3483391155 shutdown_connections 2024-03-06T07:28:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3483391155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f800c5860 0x7f0f8006f6a0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3483391155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f80155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3483391155 >> 172.21.15.26:0/3483391155 conn(0x7f0f8006b1a0 msgr2=0x7f0f8006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3483391155 shutdown_connections 2024-03-06T07:28:21.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.196+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3483391155 wait complete. 2024-03-06T07:28:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.197+0000 7f0f84e6d640 1 Processor -- start 2024-03-06T07:28:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f84e6d640 1 -- start start 2024-03-06T07:28:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f84e6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f801eac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f84e6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f800c5860 0x7f0f8014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f84e6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f801586f0 0x7f0f80153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f84e6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80152a20 con 0x7f0f801586f0 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f84e6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80155680 con 0x7f0f800c5860 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f84e6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80155050 con 0x7f0f800c5480 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f7ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f801586f0 0x7f0f80153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f7ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f801586f0 0x7f0f80153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43976/0 (socket says 172.21.15.26:43976) 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f7ed9d640 1 -- 172.21.15.26:0/3111812699 learned_addr learned my addr 172.21.15.26:0/3111812699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f7e59c640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f801eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.198+0000 7f0f77fff640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f800c5860 0x7f0f8014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f7ed9d640 1 -- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 msgr2=0x7f0f801eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f7ed9d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f801eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f7ed9d640 1 -- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f800c5860 msgr2=0x7f0f8014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f7ed9d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f800c5860 0x7f0f8014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f7ed9d640 1 -- 172.21.15.26:0/3111812699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f801eb340 con 0x7f0f801586f0 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f77fff640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f800c5860 0x7f0f8014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f7ed9d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f801586f0 0x7f0f80153a60 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c730 tx=0x7f0f7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f76ffd640 1 -- 172.21.15.26:0/3111812699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78003b70 con 0x7f0f801586f0 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f76ffd640 1 -- 172.21.15.26:0/3111812699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f78003d30 con 0x7f0f801586f0 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f801eb620 con 0x7f0f801586f0 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f76ffd640 1 -- 172.21.15.26:0/3111812699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78005310 con 0x7f0f801586f0 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.199+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8014c560 con 0x7f0f801586f0 2024-03-06T07:28:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.200+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8004f330 con 0x7f0f801586f0 2024-03-06T07:28:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.200+0000 7f0f76ffd640 1 -- 172.21.15.26:0/3111812699 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f780054b0 con 0x7f0f801586f0 2024-03-06T07:28:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.201+0000 7f0f76ffd640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f6006b4d0 0x7f0f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.201+0000 7f0f76ffd640 1 -- 172.21.15.26:0/3111812699 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f7808a200 con 0x7f0f801586f0 2024-03-06T07:28:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.201+0000 7f0f7e59c640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f6006b4d0 0x7f0f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.202+0000 7f0f7e59c640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f6006b4d0 0x7f0f6006d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0f70006fd0 tx=0x7f0f70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:21.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.205+0000 7f0f76ffd640 1 -- 172.21.15.26:0/3111812699 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f78058f00 con 0x7f0f801586f0 2024-03-06T07:28:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.383+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f8006e0c0 con 0x7f0f6006b4d0 2024-03-06T07:28:21.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.389+0000 7f0f76ffd640 1 -- 172.21.15.26:0/3111812699 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252567 (secure 0 0 0) 0x7f0f8006e0c0 con 0x7f0f6006b4d0 2024-03-06T07:28:21.394 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:28:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f6006b4d0 msgr2=0x7f0f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f6006b4d0 0x7f0f6006d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0f70006fd0 tx=0x7f0f70008040 comp rx=0 tx=0).stop 2024-03-06T07:28:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f801586f0 msgr2=0x7f0f80153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f801586f0 0x7f0f80153a60 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c730 tx=0x7f0f7800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 shutdown_connections 2024-03-06T07:28:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f6006b4d0 0x7f0f6006d990 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f801586f0 0x7f0f80153a60 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f800c5860 0x7f0f8014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 --2- 172.21.15.26:0/3111812699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f800c5480 0x7f0f801eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 >> 172.21.15.26:0/3111812699 conn(0x7f0f8006b1a0 msgr2=0x7f0f8006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 shutdown_connections 2024-03-06T07:28:21.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:21.392+0000 7f0f84e6d640 1 -- 172.21.15.26:0/3111812699 wait complete. 2024-03-06T07:28:21.405 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-03-06T07:28:21.136226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281744,"kb_used_data":19808,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754692976,"statfs":{"total":773094113280,"available":772805607424,"internally_reserved":0,"allocated":20283392,"data_stored":10954772,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.365465"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398067+0000","last_change":"2024-03-06T07:28:15.802088+0000","last_active":"2024-03-06T07:28:17.398067+0000","last_peered":"2024-03-06T07:28:17.398067+0000","last_clean":"2024-03-06T07:28:17.398067+0000","last_became_active":"2024-03-06T07:28:15.801526+0000","last_became_peered":"2024-03-06T07:28:15.801526+0000","last_unstale":"2024-03-06T07:28:17.398067+0000","last_undegraded":"2024-03-06T07:28:17.398067+0000","last_fullsized":"2024-03-06T07:28:17.398067+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:25.951020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798698+0000","last_change":"2024-03-06T07:28:15.802062+0000","last_active":"2024-03-06T07:28:16.798698+0000","last_peered":"2024-03-06T07:28:16.798698+0000","last_clean":"2024-03-06T07:28:16.798698+0000","last_became_active":"2024-03-06T07:28:15.801697+0000","last_became_peered":"2024-03-06T07:28:15.801697+0000","last_unstale":"2024-03-06T07:28:16.798698+0000","last_undegraded":"2024-03-06T07:28:16.798698+0000","last_fullsized":"2024-03-06T07:28:16.798698+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:36.476028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398994+0000","last_change":"2024-03-06T07:28:15.803607+0000","last_active":"2024-03-06T07:28:17.398994+0000","last_peered":"2024-03-06T07:28:17.398994+0000","last_clean":"2024-03-06T07:28:17.398994+0000","last_became_active":"2024-03-06T07:28:15.803339+0000","last_became_peered":"2024-03-06T07:28:15.803339+0000","last_unstale":"2024-03-06T07:28:17.398994+0000","last_undegraded":"2024-03-06T07:28:17.398994+0000","last_fullsized":"2024-03-06T07:28:17.398994+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:16:27.334487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398825+0000","last_change":"2024-03-06T07:28:15.800331+0000","last_active":"2024-03-06T07:28:17.398825+0000","last_peered":"2024-03-06T07:28:17.398825+0000","last_clean":"2024-03-06T07:28:17.398825+0000","last_became_active":"2024-03-06T07:28:15.800042+0000","last_became_peered":"2024-03-06T07:28:15.800042+0000","last_unstale":"2024-03-06T07:28:17.398825+0000","last_undegraded":"2024-03-06T07:28:17.398825+0000","last_fullsized":"2024-03-06T07:28:17.398825+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:32.608124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399009+0000","last_change":"2024-03-06T07:28:15.796723+0000","last_active":"2024-03-06T07:28:17.399009+0000","last_peered":"2024-03-06T07:28:17.399009+0000","last_clean":"2024-03-06T07:28:17.399009+0000","last_became_active":"2024-03-06T07:28:15.796474+0000","last_became_peered":"2024-03-06T07:28:15.796474+0000","last_unstale":"2024-03-06T07:28:17.399009+0000","last_undegraded":"2024-03-06T07:28:17.399009+0000","last_fullsized":"2024-03-06T07:28:17.399009+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:48.433133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398415+0000","last_change":"2024-03-06T07:28:15.796822+0000","last_active":"2024-03-06T07:28:17.398415+0000","last_peered":"2024-03-06T07:28:17.398415+0000","last_clean":"2024-03-06T07:28:17.398415+0000","last_became_active":"2024-03-06T07:28:15.796531+0000","last_became_peered":"2024-03-06T07:28:15.796531+0000","last_unstale":"2024-03-06T07:28:17.398415+0000","last_undegraded":"2024-03-06T07:28:17.398415+0000","last_fullsized":"2024-03-06T07:28:17.398415+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:11.333135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801572+0000","last_change":"2024-03-06T07:28:15.796769+0000","last_active":"2024-03-06T07:28:16.801572+0000","last_peered":"2024-03-06T07:28:16.801572+0000","last_clean":"2024-03-06T07:28:16.801572+0000","last_became_active":"2024-03-06T07:28:15.796493+0000","last_became_peered":"2024-03-06T07:28:15.796493+0000","last_unstale":"2024-03-06T07:28:16.801572+0000","last_undegraded":"2024-03-06T07:28:16.801572+0000","last_fullsized":"2024-03-06T07:28:16.801572+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:45.040241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398535+0000","last_change":"2024-03-06T07:28:15.795513+0000","last_active":"2024-03-06T07:28:17.398535+0000","last_peered":"2024-03-06T07:28:17.398535+0000","last_clean":"2024-03-06T07:28:17.398535+0000","last_became_active":"2024-03-06T07:28:15.795195+0000","last_became_peered":"2024-03-06T07:28:15.795195+0000","last_unstale":"2024-03-06T07:28:17.398535+0000","last_undegraded":"2024-03-06T07:28:17.398535+0000","last_fullsized":"2024-03-06T07:28:17.398535+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:50.682583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398254+0000","last_change":"2024-03-06T07:28:15.801827+0000","last_active":"2024-03-06T07:28:17.398254+0000","last_peered":"2024-03-06T07:28:17.398254+0000","last_clean":"2024-03-06T07:28:17.398254+0000","last_became_active":"2024-03-06T07:28:15.801167+0000","last_became_peered":"2024-03-06T07:28:15.801167+0000","last_unstale":"2024-03-06T07:28:17.398254+0000","last_undegraded":"2024-03-06T07:28:17.398254+0000","last_fullsized":"2024-03-06T07:28:17.398254+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.449435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399028+0000","last_change":"2024-03-06T07:28:15.805896+0000","last_active":"2024-03-06T07:28:17.399028+0000","last_peered":"2024-03-06T07:28:17.399028+0000","last_clean":"2024-03-06T07:28:17.399028+0000","last_became_active":"2024-03-06T07:28:15.805001+0000","last_became_peered":"2024-03-06T07:28:15.805001+0000","last_unstale":"2024-03-06T07:28:17.399028+0000","last_undegraded":"2024-03-06T07:28:17.399028+0000","last_fullsized":"2024-03-06T07:28:17.399028+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:58:41.341945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398988+0000","last_change":"2024-03-06T07:28:15.801304+0000","last_active":"2024-03-06T07:28:17.398988+0000","last_peered":"2024-03-06T07:28:17.398988+0000","last_clean":"2024-03-06T07:28:17.398988+0000","last_became_active":"2024-03-06T07:28:15.801023+0000","last_became_peered":"2024-03-06T07:28:15.801023+0000","last_unstale":"2024-03-06T07:28:17.398988+0000","last_undegraded":"2024-03-06T07:28:17.398988+0000","last_fullsized":"2024-03-06T07:28:17.398988+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:44.998092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398285+0000","last_change":"2024-03-06T07:28:15.798814+0000","last_active":"2024-03-06T07:28:17.398285+0000","last_peered":"2024-03-06T07:28:17.398285+0000","last_clean":"2024-03-06T07:28:17.398285+0000","last_became_active":"2024-03-06T07:28:15.798465+0000","last_became_peered":"2024-03-06T07:28:15.798465+0000","last_unstale":"2024-03-06T07:28:17.398285+0000","last_undegraded":"2024-03-06T07:28:17.398285+0000","last_fullsized":"2024-03-06T07:28:17.398285+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:37:43.332054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399408+0000","last_change":"2024-03-06T07:28:15.800830+0000","last_active":"2024-03-06T07:28:17.399408+0000","last_peered":"2024-03-06T07:28:17.399408+0000","last_clean":"2024-03-06T07:28:17.399408+0000","last_became_active":"2024-03-06T07:28:15.800409+0000","last_became_peered":"2024-03-06T07:28:15.800409+0000","last_unstale":"2024-03-06T07:28:17.399408+0000","last_undegraded":"2024-03-06T07:28:17.399408+0000","last_fullsized":"2024-03-06T07:28:17.399408+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:00:15.287292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397625+0000","last_change":"2024-03-06T07:28:15.801204+0000","last_active":"2024-03-06T07:28:17.397625+0000","last_peered":"2024-03-06T07:28:17.397625+0000","last_clean":"2024-03-06T07:28:17.397625+0000","last_became_active":"2024-03-06T07:28:15.800407+0000","last_became_peered":"2024-03-06T07:28:15.800407+0000","last_unstale":"2024-03-06T07:28:17.397625+0000","last_undegraded":"2024-03-06T07:28:17.397625+0000","last_fullsized":"2024-03-06T07:28:17.397625+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:57.609973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398671+0000","last_change":"2024-03-06T07:28:15.795936+0000","last_active":"2024-03-06T07:28:17.398671+0000","last_peered":"2024-03-06T07:28:17.398671+0000","last_clean":"2024-03-06T07:28:17.398671+0000","last_became_active":"2024-03-06T07:28:15.795650+0000","last_became_peered":"2024-03-06T07:28:15.795650+0000","last_unstale":"2024-03-06T07:28:17.398671+0000","last_undegraded":"2024-03-06T07:28:17.398671+0000","last_fullsized":"2024-03-06T07:28:17.398671+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:47.699638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399085+0000","last_change":"2024-03-06T07:28:15.799715+0000","last_active":"2024-03-06T07:28:17.399085+0000","last_peered":"2024-03-06T07:28:17.399085+0000","last_clean":"2024-03-06T07:28:17.399085+0000","last_became_active":"2024-03-06T07:28:15.799422+0000","last_became_peered":"2024-03-06T07:28:15.799422+0000","last_unstale":"2024-03-06T07:28:17.399085+0000","last_undegraded":"2024-03-06T07:28:17.399085+0000","last_fullsized":"2024-03-06T07:28:17.399085+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:57.609973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399219+0000","last_change":"2024-03-06T07:28:15.797121+0000","last_active":"2024-03-06T07:28:17.399219+0000","last_peered":"2024-03-06T07:28:17.399219+0000","last_clean":"2024-03-06T07:28:17.399219+0000","last_became_active":"2024-03-06T07:28:15.796775+0000","last_became_peered":"2024-03-06T07:28:15.796775+0000","last_unstale":"2024-03-06T07:28:17.399219+0000","last_undegraded":"2024-03-06T07:28:17.399219+0000","last_fullsized":"2024-03-06T07:28:17.399219+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:46:03.560601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399024+0000","last_change":"2024-03-06T07:28:15.810661+0000","last_active":"2024-03-06T07:28:17.399024+0000","last_peered":"2024-03-06T07:28:17.399024+0000","last_clean":"2024-03-06T07:28:17.399024+0000","last_became_active":"2024-03-06T07:28:15.805348+0000","last_became_peered":"2024-03-06T07:28:15.805348+0000","last_unstale":"2024-03-06T07:28:17.399024+0000","last_undegraded":"2024-03-06T07:28:17.399024+0000","last_fullsized":"2024-03-06T07:28:17.399024+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:35:26.179502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398898+0000","last_change":"2024-03-06T07:28:15.803699+0000","last_active":"2024-03-06T07:28:17.398898+0000","last_peered":"2024-03-06T07:28:17.398898+0000","last_clean":"2024-03-06T07:28:17.398898+0000","last_became_active":"2024-03-06T07:28:15.803220+0000","last_became_peered":"2024-03-06T07:28:15.803220+0000","last_unstale":"2024-03-06T07:28:17.398898+0000","last_undegraded":"2024-03-06T07:28:17.398898+0000","last_fullsized":"2024-03-06T07:28:17.398898+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:15.839156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798600+0000","last_change":"2024-03-06T07:28:15.806052+0000","last_active":"2024-03-06T07:28:16.798600+0000","last_peered":"2024-03-06T07:28:16.798600+0000","last_clean":"2024-03-06T07:28:16.798600+0000","last_became_active":"2024-03-06T07:28:15.805178+0000","last_became_peered":"2024-03-06T07:28:15.805178+0000","last_unstale":"2024-03-06T07:28:16.798600+0000","last_undegraded":"2024-03-06T07:28:16.798600+0000","last_fullsized":"2024-03-06T07:28:16.798600+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:36.959393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801734+0000","last_change":"2024-03-06T07:28:15.789673+0000","last_active":"2024-03-06T07:28:16.801734+0000","last_peered":"2024-03-06T07:28:16.801734+0000","last_clean":"2024-03-06T07:28:16.801734+0000","last_became_active":"2024-03-06T07:28:15.789403+0000","last_became_peered":"2024-03-06T07:28:15.789403+0000","last_unstale":"2024-03-06T07:28:16.801734+0000","last_undegraded":"2024-03-06T07:28:16.801734+0000","last_fullsized":"2024-03-06T07:28:16.801734+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:32.570331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398532+0000","last_change":"2024-03-06T07:28:15.789477+0000","last_active":"2024-03-06T07:28:17.398532+0000","last_peered":"2024-03-06T07:28:17.398532+0000","last_clean":"2024-03-06T07:28:17.398532+0000","last_became_active":"2024-03-06T07:28:15.788910+0000","last_became_peered":"2024-03-06T07:28:15.788910+0000","last_unstale":"2024-03-06T07:28:17.398532+0000","last_undegraded":"2024-03-06T07:28:17.398532+0000","last_fullsized":"2024-03-06T07:28:17.398532+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:18.231291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398835+0000","last_change":"2024-03-06T07:28:15.797621+0000","last_active":"2024-03-06T07:28:17.398835+0000","last_peered":"2024-03-06T07:28:17.398835+0000","last_clean":"2024-03-06T07:28:17.398835+0000","last_became_active":"2024-03-06T07:28:15.797313+0000","last_became_peered":"2024-03-06T07:28:15.797313+0000","last_unstale":"2024-03-06T07:28:17.398835+0000","last_undegraded":"2024-03-06T07:28:17.398835+0000","last_fullsized":"2024-03-06T07:28:17.398835+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:58.939766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397764+0000","last_change":"2024-03-06T07:28:15.789861+0000","last_active":"2024-03-06T07:28:17.397764+0000","last_peered":"2024-03-06T07:28:17.397764+0000","last_clean":"2024-03-06T07:28:17.397764+0000","last_became_active":"2024-03-06T07:28:15.789574+0000","last_became_peered":"2024-03-06T07:28:15.789574+0000","last_unstale":"2024-03-06T07:28:17.397764+0000","last_undegraded":"2024-03-06T07:28:17.397764+0000","last_fullsized":"2024-03-06T07:28:17.397764+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:40.408964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798918+0000","last_change":"2024-03-06T07:28:15.807619+0000","last_active":"2024-03-06T07:28:16.798918+0000","last_peered":"2024-03-06T07:28:16.798918+0000","last_clean":"2024-03-06T07:28:16.798918+0000","last_became_active":"2024-03-06T07:28:15.806880+0000","last_became_peered":"2024-03-06T07:28:15.806880+0000","last_unstale":"2024-03-06T07:28:16.798918+0000","last_undegraded":"2024-03-06T07:28:16.798918+0000","last_fullsized":"2024-03-06T07:28:16.798918+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:49.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.802128+0000","last_change":"2024-03-06T07:28:15.796807+0000","last_active":"2024-03-06T07:28:16.802128+0000","last_peered":"2024-03-06T07:28:16.802128+0000","last_clean":"2024-03-06T07:28:16.802128+0000","last_became_active":"2024-03-06T07:28:15.796525+0000","last_became_peered":"2024-03-06T07:28:15.796525+0000","last_unstale":"2024-03-06T07:28:16.802128+0000","last_undegraded":"2024-03-06T07:28:16.802128+0000","last_fullsized":"2024-03-06T07:28:16.802128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:56.183328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398207+0000","last_change":"2024-03-06T07:28:15.801409+0000","last_active":"2024-03-06T07:28:17.398207+0000","last_peered":"2024-03-06T07:28:17.398207+0000","last_clean":"2024-03-06T07:28:17.398207+0000","last_became_active":"2024-03-06T07:28:15.801120+0000","last_became_peered":"2024-03-06T07:28:15.801120+0000","last_unstale":"2024-03-06T07:28:17.398207+0000","last_undegraded":"2024-03-06T07:28:17.398207+0000","last_fullsized":"2024-03-06T07:28:17.398207+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:27.849829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398547+0000","last_change":"2024-03-06T07:28:15.800350+0000","last_active":"2024-03-06T07:28:17.398547+0000","last_peered":"2024-03-06T07:28:17.398547+0000","last_clean":"2024-03-06T07:28:17.398547+0000","last_became_active":"2024-03-06T07:28:15.799655+0000","last_became_peered":"2024-03-06T07:28:15.799655+0000","last_unstale":"2024-03-06T07:28:17.398547+0000","last_undegraded":"2024-03-06T07:28:17.398547+0000","last_fullsized":"2024-03-06T07:28:17.398547+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:39:06.987968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398768+0000","last_change":"2024-03-06T07:28:15.799608+0000","last_active":"2024-03-06T07:28:17.398768+0000","last_peered":"2024-03-06T07:28:17.398768+0000","last_clean":"2024-03-06T07:28:17.398768+0000","last_became_active":"2024-03-06T07:28:15.799303+0000","last_became_peered":"2024-03-06T07:28:15.799303+0000","last_unstale":"2024-03-06T07:28:17.398768+0000","last_undegraded":"2024-03-06T07:28:17.398768+0000","last_fullsized":"2024-03-06T07:28:17.398768+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:44:02.229882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397785+0000","last_change":"2024-03-06T07:28:15.799983+0000","last_active":"2024-03-06T07:28:17.397785+0000","last_peered":"2024-03-06T07:28:17.397785+0000","last_clean":"2024-03-06T07:28:17.397785+0000","last_became_active":"2024-03-06T07:28:15.799645+0000","last_became_peered":"2024-03-06T07:28:15.799645+0000","last_unstale":"2024-03-06T07:28:17.397785+0000","last_undegraded":"2024-03-06T07:28:17.397785+0000","last_fullsized":"2024-03-06T07:28:17.397785+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:25.561669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798492+0000","last_change":"2024-03-06T07:28:15.794790+0000","last_active":"2024-03-06T07:28:16.798492+0000","last_peered":"2024-03-06T07:28:16.798492+0000","last_clean":"2024-03-06T07:28:16.798492+0000","last_became_active":"2024-03-06T07:28:15.794361+0000","last_became_peered":"2024-03-06T07:28:15.794361+0000","last_unstale":"2024-03-06T07:28:16.798492+0000","last_undegraded":"2024-03-06T07:28:16.798492+0000","last_fullsized":"2024-03-06T07:28:16.798492+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:58.576387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397663+0000","last_change":"2024-03-06T07:28:15.802386+0000","last_active":"2024-03-06T07:28:17.397663+0000","last_peered":"2024-03-06T07:28:17.397663+0000","last_clean":"2024-03-06T07:28:17.397663+0000","last_became_active":"2024-03-06T07:28:15.799297+0000","last_became_peered":"2024-03-06T07:28:15.799297+0000","last_unstale":"2024-03-06T07:28:17.397663+0000","last_undegraded":"2024-03-06T07:28:17.397663+0000","last_fullsized":"2024-03-06T07:28:17.397663+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:31:54.984328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798314+0000","last_change":"2024-03-06T07:28:15.801031+0000","last_active":"2024-03-06T07:28:16.798314+0000","last_peered":"2024-03-06T07:28:16.798314+0000","last_clean":"2024-03-06T07:28:16.798314+0000","last_became_active":"2024-03-06T07:28:15.800738+0000","last_became_peered":"2024-03-06T07:28:15.800738+0000","last_unstale":"2024-03-06T07:28:16.798314+0000","last_undegraded":"2024-03-06T07:28:16.798314+0000","last_fullsized":"2024-03-06T07:28:16.798314+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:22.827277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"197'4","reported_seq":18,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398077+0000","last_change":"2024-03-06T07:28:14.781972+0000","last_active":"2024-03-06T07:28:17.398077+0000","last_peered":"2024-03-06T07:28:17.398077+0000","last_clean":"2024-03-06T07:28:17.398077+0000","last_became_active":"2024-03-06T07:28:14.781691+0000","last_became_peered":"2024-03-06T07:28:14.781691+0000","last_unstale":"2024-03-06T07:28:17.398077+0000","last_undegraded":"2024-03-06T07:28:17.398077+0000","last_fullsized":"2024-03-06T07:28:17.398077+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:29.750140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398155+0000","last_change":"2024-03-06T07:28:15.798509+0000","last_active":"2024-03-06T07:28:17.398155+0000","last_peered":"2024-03-06T07:28:17.398155+0000","last_clean":"2024-03-06T07:28:17.398155+0000","last_became_active":"2024-03-06T07:28:15.798222+0000","last_became_peered":"2024-03-06T07:28:15.798222+0000","last_unstale":"2024-03-06T07:28:17.398155+0000","last_undegraded":"2024-03-06T07:28:17.398155+0000","last_fullsized":"2024-03-06T07:28:17.398155+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:23.021957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398143+0000","last_change":"2024-03-06T07:28:14.788442+0000","last_active":"2024-03-06T07:28:17.398143+0000","last_peered":"2024-03-06T07:28:17.398143+0000","last_clean":"2024-03-06T07:28:17.398143+0000","last_became_active":"2024-03-06T07:28:14.788075+0000","last_became_peered":"2024-03-06T07:28:14.788075+0000","last_unstale":"2024-03-06T07:28:17.398143+0000","last_undegraded":"2024-03-06T07:28:17.398143+0000","last_fullsized":"2024-03-06T07:28:17.398143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:11.511138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398279+0000","last_change":"2024-03-06T07:28:15.796288+0000","last_active":"2024-03-06T07:28:17.398279+0000","last_peered":"2024-03-06T07:28:17.398279+0000","last_clean":"2024-03-06T07:28:17.398279+0000","last_became_active":"2024-03-06T07:28:15.796000+0000","last_became_peered":"2024-03-06T07:28:15.796000+0000","last_unstale":"2024-03-06T07:28:17.398279+0000","last_undegraded":"2024-03-06T07:28:17.398279+0000","last_fullsized":"2024-03-06T07:28:17.398279+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:52.339463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398142+0000","last_change":"2024-03-06T07:28:14.791895+0000","last_active":"2024-03-06T07:28:17.398142+0000","last_peered":"2024-03-06T07:28:17.398142+0000","last_clean":"2024-03-06T07:28:17.398142+0000","last_became_active":"2024-03-06T07:28:14.791606+0000","last_became_peered":"2024-03-06T07:28:14.791606+0000","last_unstale":"2024-03-06T07:28:17.398142+0000","last_undegraded":"2024-03-06T07:28:17.398142+0000","last_fullsized":"2024-03-06T07:28:17.398142+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:54.513290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398053+0000","last_change":"2024-03-06T07:28:15.808840+0000","last_active":"2024-03-06T07:28:17.398053+0000","last_peered":"2024-03-06T07:28:17.398053+0000","last_clean":"2024-03-06T07:28:17.398053+0000","last_became_active":"2024-03-06T07:28:15.808536+0000","last_became_peered":"2024-03-06T07:28:15.808536+0000","last_unstale":"2024-03-06T07:28:17.398053+0000","last_undegraded":"2024-03-06T07:28:17.398053+0000","last_fullsized":"2024-03-06T07:28:17.398053+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:51:55.356703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398571+0000","last_change":"2024-03-06T07:28:14.783699+0000","last_active":"2024-03-06T07:28:17.398571+0000","last_peered":"2024-03-06T07:28:17.398571+0000","last_clean":"2024-03-06T07:28:17.398571+0000","last_became_active":"2024-03-06T07:28:14.783452+0000","last_became_peered":"2024-03-06T07:28:14.783452+0000","last_unstale":"2024-03-06T07:28:17.398571+0000","last_undegraded":"2024-03-06T07:28:17.398571+0000","last_fullsized":"2024-03-06T07:28:17.398571+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:15.422272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801921+0000","last_change":"2024-03-06T07:28:15.792944+0000","last_active":"2024-03-06T07:28:16.801921+0000","last_peered":"2024-03-06T07:28:16.801921+0000","last_clean":"2024-03-06T07:28:16.801921+0000","last_became_active":"2024-03-06T07:28:15.792695+0000","last_became_peered":"2024-03-06T07:28:15.792695+0000","last_unstale":"2024-03-06T07:28:16.801921+0000","last_undegraded":"2024-03-06T07:28:16.801921+0000","last_fullsized":"2024-03-06T07:28:16.801921+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:51.553861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397867+0000","last_change":"2024-03-06T07:28:14.786831+0000","last_active":"2024-03-06T07:28:17.397867+0000","last_peered":"2024-03-06T07:28:17.397867+0000","last_clean":"2024-03-06T07:28:17.397867+0000","last_became_active":"2024-03-06T07:28:14.786548+0000","last_became_peered":"2024-03-06T07:28:14.786548+0000","last_unstale":"2024-03-06T07:28:17.397867+0000","last_undegraded":"2024-03-06T07:28:17.397867+0000","last_fullsized":"2024-03-06T07:28:17.397867+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:37.445975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397710+0000","last_change":"2024-03-06T07:28:15.810738+0000","last_active":"2024-03-06T07:28:17.397710+0000","last_peered":"2024-03-06T07:28:17.397710+0000","last_clean":"2024-03-06T07:28:17.397710+0000","last_became_active":"2024-03-06T07:28:15.810446+0000","last_became_peered":"2024-03-06T07:28:15.810446+0000","last_unstale":"2024-03-06T07:28:17.397710+0000","last_undegraded":"2024-03-06T07:28:17.397710+0000","last_fullsized":"2024-03-06T07:28:17.397710+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:55.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397732+0000","last_change":"2024-03-06T07:28:14.782671+0000","last_active":"2024-03-06T07:28:17.397732+0000","last_peered":"2024-03-06T07:28:17.397732+0000","last_clean":"2024-03-06T07:28:17.397732+0000","last_became_active":"2024-03-06T07:28:14.782382+0000","last_became_peered":"2024-03-06T07:28:14.782382+0000","last_unstale":"2024-03-06T07:28:17.397732+0000","last_undegraded":"2024-03-06T07:28:17.397732+0000","last_fullsized":"2024-03-06T07:28:17.397732+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:41.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398705+0000","last_change":"2024-03-06T07:28:15.802769+0000","last_active":"2024-03-06T07:28:17.398705+0000","last_peered":"2024-03-06T07:28:17.398705+0000","last_clean":"2024-03-06T07:28:17.398705+0000","last_became_active":"2024-03-06T07:28:15.802369+0000","last_became_peered":"2024-03-06T07:28:15.802369+0000","last_unstale":"2024-03-06T07:28:17.398705+0000","last_undegraded":"2024-03-06T07:28:17.398705+0000","last_fullsized":"2024-03-06T07:28:17.398705+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:35.086433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397754+0000","last_change":"2024-03-06T07:28:14.776124+0000","last_active":"2024-03-06T07:28:17.397754+0000","last_peered":"2024-03-06T07:28:17.397754+0000","last_clean":"2024-03-06T07:28:17.397754+0000","last_became_active":"2024-03-06T07:28:14.775819+0000","last_became_peered":"2024-03-06T07:28:14.775819+0000","last_unstale":"2024-03-06T07:28:17.397754+0000","last_undegraded":"2024-03-06T07:28:17.397754+0000","last_fullsized":"2024-03-06T07:28:17.397754+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:41.621389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801571+0000","last_change":"2024-03-06T07:28:15.793560+0000","last_active":"2024-03-06T07:28:16.801571+0000","last_peered":"2024-03-06T07:28:16.801571+0000","last_clean":"2024-03-06T07:28:16.801571+0000","last_became_active":"2024-03-06T07:28:15.793292+0000","last_became_peered":"2024-03-06T07:28:15.793292+0000","last_unstale":"2024-03-06T07:28:16.801571+0000","last_undegraded":"2024-03-06T07:28:16.801571+0000","last_fullsized":"2024-03-06T07:28:16.801571+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:38:14.146901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398462+0000","last_change":"2024-03-06T07:28:14.789097+0000","last_active":"2024-03-06T07:28:17.398462+0000","last_peered":"2024-03-06T07:28:17.398462+0000","last_clean":"2024-03-06T07:28:17.398462+0000","last_became_active":"2024-03-06T07:28:14.788833+0000","last_became_peered":"2024-03-06T07:28:14.788833+0000","last_unstale":"2024-03-06T07:28:17.398462+0000","last_undegraded":"2024-03-06T07:28:17.398462+0000","last_fullsized":"2024-03-06T07:28:17.398462+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:14.834969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398034+0000","last_change":"2024-03-06T07:28:15.794447+0000","last_active":"2024-03-06T07:28:17.398034+0000","last_peered":"2024-03-06T07:28:17.398034+0000","last_clean":"2024-03-06T07:28:17.398034+0000","last_became_active":"2024-03-06T07:28:15.794115+0000","last_became_peered":"2024-03-06T07:28:15.794115+0000","last_unstale":"2024-03-06T07:28:17.398034+0000","last_undegraded":"2024-03-06T07:28:17.398034+0000","last_fullsized":"2024-03-06T07:28:17.398034+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:54.643304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399137+0000","last_change":"2024-03-06T07:28:14.787649+0000","last_active":"2024-03-06T07:28:17.399137+0000","last_peered":"2024-03-06T07:28:17.399137+0000","last_clean":"2024-03-06T07:28:17.399137+0000","last_became_active":"2024-03-06T07:28:14.787348+0000","last_became_peered":"2024-03-06T07:28:14.787348+0000","last_unstale":"2024-03-06T07:28:17.399137+0000","last_undegraded":"2024-03-06T07:28:17.399137+0000","last_fullsized":"2024-03-06T07:28:17.399137+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:50.549674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398121+0000","last_change":"2024-03-06T07:28:15.798455+0000","last_active":"2024-03-06T07:28:17.398121+0000","last_peered":"2024-03-06T07:28:17.398121+0000","last_clean":"2024-03-06T07:28:17.398121+0000","last_became_active":"2024-03-06T07:28:15.798172+0000","last_became_peered":"2024-03-06T07:28:15.798172+0000","last_unstale":"2024-03-06T07:28:17.398121+0000","last_undegraded":"2024-03-06T07:28:17.398121+0000","last_fullsized":"2024-03-06T07:28:17.398121+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:58.939766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"197'3","reported_seq":17,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398315+0000","last_change":"2024-03-06T07:28:14.785132+0000","last_active":"2024-03-06T07:28:17.398315+0000","last_peered":"2024-03-06T07:28:17.398315+0000","last_clean":"2024-03-06T07:28:17.398315+0000","last_became_active":"2024-03-06T07:28:14.784849+0000","last_became_peered":"2024-03-06T07:28:14.784849+0000","last_unstale":"2024-03-06T07:28:17.398315+0000","last_undegraded":"2024-03-06T07:28:17.398315+0000","last_fullsized":"2024-03-06T07:28:17.398315+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:17.227104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801727+0000","last_change":"2024-03-06T07:28:15.797828+0000","last_active":"2024-03-06T07:28:16.801727+0000","last_peered":"2024-03-06T07:28:16.801727+0000","last_clean":"2024-03-06T07:28:16.801727+0000","last_became_active":"2024-03-06T07:28:15.797537+0000","last_became_peered":"2024-03-06T07:28:15.797537+0000","last_unstale":"2024-03-06T07:28:16.801727+0000","last_undegraded":"2024-03-06T07:28:16.801727+0000","last_fullsized":"2024-03-06T07:28:16.801727+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:12.511505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"197'6","reported_seq":20,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398398+0000","last_change":"2024-03-06T07:28:14.785822+0000","last_active":"2024-03-06T07:28:17.398398+0000","last_peered":"2024-03-06T07:28:17.398398+0000","last_clean":"2024-03-06T07:28:17.398398+0000","last_became_active":"2024-03-06T07:28:14.785540+0000","last_became_peered":"2024-03-06T07:28:14.785540+0000","last_unstale":"2024-03-06T07:28:17.398398+0000","last_undegraded":"2024-03-06T07:28:17.398398+0000","last_fullsized":"2024-03-06T07:28:17.398398+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:35:52.319705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798568+0000","last_change":"2024-03-06T07:28:15.806312+0000","last_active":"2024-03-06T07:28:16.798568+0000","last_peered":"2024-03-06T07:28:16.798568+0000","last_clean":"2024-03-06T07:28:16.798568+0000","last_became_active":"2024-03-06T07:28:15.805665+0000","last_became_peered":"2024-03-06T07:28:15.805665+0000","last_unstale":"2024-03-06T07:28:16.798568+0000","last_undegraded":"2024-03-06T07:28:16.798568+0000","last_fullsized":"2024-03-06T07:28:16.798568+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:19:49.143984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398683+0000","last_change":"2024-03-06T07:28:14.784265+0000","last_active":"2024-03-06T07:28:17.398683+0000","last_peered":"2024-03-06T07:28:17.398683+0000","last_clean":"2024-03-06T07:28:17.398683+0000","last_became_active":"2024-03-06T07:28:14.783985+0000","last_became_peered":"2024-03-06T07:28:14.783985+0000","last_unstale":"2024-03-06T07:28:17.398683+0000","last_undegraded":"2024-03-06T07:28:17.398683+0000","last_fullsized":"2024-03-06T07:28:17.398683+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:53:56.571551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801758+0000","last_change":"2024-03-06T07:28:15.801792+0000","last_active":"2024-03-06T07:28:16.801758+0000","last_peered":"2024-03-06T07:28:16.801758+0000","last_clean":"2024-03-06T07:28:16.801758+0000","last_became_active":"2024-03-06T07:28:15.801469+0000","last_became_peered":"2024-03-06T07:28:15.801469+0000","last_unstale":"2024-03-06T07:28:16.801758+0000","last_undegraded":"2024-03-06T07:28:16.801758+0000","last_fullsized":"2024-03-06T07:28:16.801758+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:26:51.432968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398477+0000","last_change":"2024-03-06T07:28:14.784209+0000","last_active":"2024-03-06T07:28:17.398477+0000","last_peered":"2024-03-06T07:28:17.398477+0000","last_clean":"2024-03-06T07:28:17.398477+0000","last_became_active":"2024-03-06T07:28:14.783932+0000","last_became_peered":"2024-03-06T07:28:14.783932+0000","last_unstale":"2024-03-06T07:28:17.398477+0000","last_undegraded":"2024-03-06T07:28:17.398477+0000","last_fullsized":"2024-03-06T07:28:17.398477+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:53.639117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399034+0000","last_change":"2024-03-06T07:28:15.794406+0000","last_active":"2024-03-06T07:28:17.399034+0000","last_peered":"2024-03-06T07:28:17.399034+0000","last_clean":"2024-03-06T07:28:17.399034+0000","last_became_active":"2024-03-06T07:28:15.794097+0000","last_became_peered":"2024-03-06T07:28:15.794097+0000","last_unstale":"2024-03-06T07:28:17.399034+0000","last_undegraded":"2024-03-06T07:28:17.399034+0000","last_fullsized":"2024-03-06T07:28:17.399034+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:38.450162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397969+0000","last_change":"2024-03-06T07:28:14.787534+0000","last_active":"2024-03-06T07:28:17.397969+0000","last_peered":"2024-03-06T07:28:17.397969+0000","last_clean":"2024-03-06T07:28:17.397969+0000","last_became_active":"2024-03-06T07:28:14.787286+0000","last_became_peered":"2024-03-06T07:28:14.787286+0000","last_unstale":"2024-03-06T07:28:17.397969+0000","last_undegraded":"2024-03-06T07:28:17.397969+0000","last_fullsized":"2024-03-06T07:28:17.397969+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:18.510928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398812+0000","last_change":"2024-03-06T07:28:15.806082+0000","last_active":"2024-03-06T07:28:17.398812+0000","last_peered":"2024-03-06T07:28:17.398812+0000","last_clean":"2024-03-06T07:28:17.398812+0000","last_became_active":"2024-03-06T07:28:15.805099+0000","last_became_peered":"2024-03-06T07:28:15.805099+0000","last_unstale":"2024-03-06T07:28:17.398812+0000","last_undegraded":"2024-03-06T07:28:17.398812+0000","last_fullsized":"2024-03-06T07:28:17.398812+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:38:05.515279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"197'8","reported_seq":22,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798420+0000","last_change":"2024-03-06T07:28:14.788682+0000","last_active":"2024-03-06T07:28:16.798420+0000","last_peered":"2024-03-06T07:28:16.798420+0000","last_clean":"2024-03-06T07:28:16.798420+0000","last_became_active":"2024-03-06T07:28:14.788381+0000","last_became_peered":"2024-03-06T07:28:14.788381+0000","last_unstale":"2024-03-06T07:28:16.798420+0000","last_undegraded":"2024-03-06T07:28:16.798420+0000","last_fullsized":"2024-03-06T07:28:16.798420+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:57.017927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398247+0000","last_change":"2024-03-06T07:28:15.796200+0000","last_active":"2024-03-06T07:28:17.398247+0000","last_peered":"2024-03-06T07:28:17.398247+0000","last_clean":"2024-03-06T07:28:17.398247+0000","last_became_active":"2024-03-06T07:28:15.795919+0000","last_became_peered":"2024-03-06T07:28:15.795919+0000","last_unstale":"2024-03-06T07:28:17.398247+0000","last_undegraded":"2024-03-06T07:28:17.398247+0000","last_fullsized":"2024-03-06T07:28:17.398247+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:59.308461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398196+0000","last_change":"2024-03-06T07:28:14.787636+0000","last_active":"2024-03-06T07:28:17.398196+0000","last_peered":"2024-03-06T07:28:17.398196+0000","last_clean":"2024-03-06T07:28:17.398196+0000","last_became_active":"2024-03-06T07:28:14.787367+0000","last_became_peered":"2024-03-06T07:28:14.787367+0000","last_unstale":"2024-03-06T07:28:17.398196+0000","last_undegraded":"2024-03-06T07:28:17.398196+0000","last_fullsized":"2024-03-06T07:28:17.398196+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:22.374111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397858+0000","last_change":"2024-03-06T07:28:15.797503+0000","last_active":"2024-03-06T07:28:17.397858+0000","last_peered":"2024-03-06T07:28:17.397858+0000","last_clean":"2024-03-06T07:28:17.397858+0000","last_became_active":"2024-03-06T07:28:15.796055+0000","last_became_peered":"2024-03-06T07:28:15.796055+0000","last_unstale":"2024-03-06T07:28:17.397858+0000","last_undegraded":"2024-03-06T07:28:17.397858+0000","last_fullsized":"2024-03-06T07:28:17.397858+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:22.057287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798665+0000","last_change":"2024-03-06T07:28:14.787083+0000","last_active":"2024-03-06T07:28:16.798665+0000","last_peered":"2024-03-06T07:28:16.798665+0000","last_clean":"2024-03-06T07:28:16.798665+0000","last_became_active":"2024-03-06T07:28:14.786827+0000","last_became_peered":"2024-03-06T07:28:14.786827+0000","last_unstale":"2024-03-06T07:28:16.798665+0000","last_undegraded":"2024-03-06T07:28:16.798665+0000","last_fullsized":"2024-03-06T07:28:16.798665+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:59.668294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798329+0000","last_change":"2024-03-06T07:28:15.808458+0000","last_active":"2024-03-06T07:28:16.798329+0000","last_peered":"2024-03-06T07:28:16.798329+0000","last_clean":"2024-03-06T07:28:16.798329+0000","last_became_active":"2024-03-06T07:28:15.808179+0000","last_became_peered":"2024-03-06T07:28:15.808179+0000","last_unstale":"2024-03-06T07:28:16.798329+0000","last_undegraded":"2024-03-06T07:28:16.798329+0000","last_fullsized":"2024-03-06T07:28:16.798329+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:01.029533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398242+0000","last_change":"2024-03-06T07:28:14.788050+0000","last_active":"2024-03-06T07:28:17.398242+0000","last_peered":"2024-03-06T07:28:17.398242+0000","last_clean":"2024-03-06T07:28:17.398242+0000","last_became_active":"2024-03-06T07:28:14.787770+0000","last_became_peered":"2024-03-06T07:28:14.787770+0000","last_unstale":"2024-03-06T07:28:17.398242+0000","last_undegraded":"2024-03-06T07:28:17.398242+0000","last_fullsized":"2024-03-06T07:28:17.398242+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:22.597847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398350+0000","last_change":"2024-03-06T07:28:15.801317+0000","last_active":"2024-03-06T07:28:17.398350+0000","last_peered":"2024-03-06T07:28:17.398350+0000","last_clean":"2024-03-06T07:28:17.398350+0000","last_became_active":"2024-03-06T07:28:15.801080+0000","last_became_peered":"2024-03-06T07:28:15.801080+0000","last_unstale":"2024-03-06T07:28:17.398350+0000","last_undegraded":"2024-03-06T07:28:17.398350+0000","last_fullsized":"2024-03-06T07:28:17.398350+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:15.839156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398350+0000","last_change":"2024-03-06T07:28:14.782491+0000","last_active":"2024-03-06T07:28:17.398350+0000","last_peered":"2024-03-06T07:28:17.398350+0000","last_clean":"2024-03-06T07:28:17.398350+0000","last_became_active":"2024-03-06T07:28:14.782223+0000","last_became_peered":"2024-03-06T07:28:14.782223+0000","last_unstale":"2024-03-06T07:28:17.398350+0000","last_undegraded":"2024-03-06T07:28:17.398350+0000","last_fullsized":"2024-03-06T07:28:17.398350+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:58.645677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398679+0000","last_change":"2024-03-06T07:28:15.796329+0000","last_active":"2024-03-06T07:28:17.398679+0000","last_peered":"2024-03-06T07:28:17.398679+0000","last_clean":"2024-03-06T07:28:17.398679+0000","last_became_active":"2024-03-06T07:28:15.795993+0000","last_became_peered":"2024-03-06T07:28:15.795993+0000","last_unstale":"2024-03-06T07:28:17.398679+0000","last_undegraded":"2024-03-06T07:28:17.398679+0000","last_fullsized":"2024-03-06T07:28:17.398679+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:38.450162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398439+0000","last_change":"2024-03-06T07:28:14.785927+0000","last_active":"2024-03-06T07:28:17.398439+0000","last_peered":"2024-03-06T07:28:17.398439+0000","last_clean":"2024-03-06T07:28:17.398439+0000","last_became_active":"2024-03-06T07:28:14.785666+0000","last_became_peered":"2024-03-06T07:28:14.785666+0000","last_unstale":"2024-03-06T07:28:17.398439+0000","last_undegraded":"2024-03-06T07:28:17.398439+0000","last_fullsized":"2024-03-06T07:28:17.398439+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:55.179141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798507+0000","last_change":"2024-03-06T07:28:15.796518+0000","last_active":"2024-03-06T07:28:16.798507+0000","last_peered":"2024-03-06T07:28:16.798507+0000","last_clean":"2024-03-06T07:28:16.798507+0000","last_became_active":"2024-03-06T07:28:15.796242+0000","last_became_peered":"2024-03-06T07:28:15.796242+0000","last_unstale":"2024-03-06T07:28:16.798507+0000","last_undegraded":"2024-03-06T07:28:16.798507+0000","last_fullsized":"2024-03-06T07:28:16.798507+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:26.109657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398635+0000","last_change":"2024-03-06T07:28:14.788414+0000","last_active":"2024-03-06T07:28:17.398635+0000","last_peered":"2024-03-06T07:28:17.398635+0000","last_clean":"2024-03-06T07:28:17.398635+0000","last_became_active":"2024-03-06T07:28:14.788047+0000","last_became_peered":"2024-03-06T07:28:14.788047+0000","last_unstale":"2024-03-06T07:28:17.398635+0000","last_undegraded":"2024-03-06T07:28:17.398635+0000","last_fullsized":"2024-03-06T07:28:17.398635+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:40.945464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398671+0000","last_change":"2024-03-06T07:28:15.796320+0000","last_active":"2024-03-06T07:28:17.398671+0000","last_peered":"2024-03-06T07:28:17.398671+0000","last_clean":"2024-03-06T07:28:17.398671+0000","last_became_active":"2024-03-06T07:28:15.796013+0000","last_became_peered":"2024-03-06T07:28:15.796013+0000","last_unstale":"2024-03-06T07:28:17.398671+0000","last_undegraded":"2024-03-06T07:28:17.398671+0000","last_fullsized":"2024-03-06T07:28:17.398671+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:48.471486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398120+0000","last_change":"2024-03-06T07:28:14.782233+0000","last_active":"2024-03-06T07:28:17.398120+0000","last_peered":"2024-03-06T07:28:17.398120+0000","last_clean":"2024-03-06T07:28:17.398120+0000","last_became_active":"2024-03-06T07:28:14.781927+0000","last_became_peered":"2024-03-06T07:28:14.781927+0000","last_unstale":"2024-03-06T07:28:17.398120+0000","last_undegraded":"2024-03-06T07:28:17.398120+0000","last_fullsized":"2024-03-06T07:28:17.398120+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:11.894389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399173+0000","last_change":"2024-03-06T07:28:15.785648+0000","last_active":"2024-03-06T07:28:17.399173+0000","last_peered":"2024-03-06T07:28:17.399173+0000","last_clean":"2024-03-06T07:28:17.399173+0000","last_became_active":"2024-03-06T07:28:15.785364+0000","last_became_peered":"2024-03-06T07:28:15.785364+0000","last_unstale":"2024-03-06T07:28:17.399173+0000","last_undegraded":"2024-03-06T07:28:17.399173+0000","last_fullsized":"2024-03-06T07:28:17.399173+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:29.576193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397463+0000","last_change":"2024-03-06T07:28:14.785960+0000","last_active":"2024-03-06T07:28:17.397463+0000","last_peered":"2024-03-06T07:28:17.397463+0000","last_clean":"2024-03-06T07:28:17.397463+0000","last_became_active":"2024-03-06T07:28:14.785692+0000","last_became_peered":"2024-03-06T07:28:14.785692+0000","last_unstale":"2024-03-06T07:28:17.397463+0000","last_undegraded":"2024-03-06T07:28:17.397463+0000","last_fullsized":"2024-03-06T07:28:17.397463+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:57.935579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801768+0000","last_change":"2024-03-06T07:28:15.793292+0000","last_active":"2024-03-06T07:28:16.801768+0000","last_peered":"2024-03-06T07:28:16.801768+0000","last_clean":"2024-03-06T07:28:16.801768+0000","last_became_active":"2024-03-06T07:28:15.793015+0000","last_became_peered":"2024-03-06T07:28:15.793015+0000","last_unstale":"2024-03-06T07:28:16.801768+0000","last_undegraded":"2024-03-06T07:28:16.801768+0000","last_fullsized":"2024-03-06T07:28:16.801768+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:17.075519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399124+0000","last_change":"2024-03-06T07:28:14.784580+0000","last_active":"2024-03-06T07:28:17.399124+0000","last_peered":"2024-03-06T07:28:17.399124+0000","last_clean":"2024-03-06T07:28:17.399124+0000","last_became_active":"2024-03-06T07:28:14.784245+0000","last_became_peered":"2024-03-06T07:28:14.784245+0000","last_unstale":"2024-03-06T07:28:17.399124+0000","last_undegraded":"2024-03-06T07:28:17.399124+0000","last_fullsized":"2024-03-06T07:28:17.399124+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:16.071332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398501+0000","last_change":"2024-03-06T07:28:15.798632+0000","last_active":"2024-03-06T07:28:17.398501+0000","last_peered":"2024-03-06T07:28:17.398501+0000","last_clean":"2024-03-06T07:28:17.398501+0000","last_became_active":"2024-03-06T07:28:15.798283+0000","last_became_peered":"2024-03-06T07:28:15.798283+0000","last_unstale":"2024-03-06T07:28:17.398501+0000","last_undegraded":"2024-03-06T07:28:17.398501+0000","last_fullsized":"2024-03-06T07:28:17.398501+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:55.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"198'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399018+0000","last_change":"2024-03-06T07:28:14.786720+0000","last_active":"2024-03-06T07:28:17.399018+0000","last_peered":"2024-03-06T07:28:17.399018+0000","last_clean":"2024-03-06T07:28:17.399018+0000","last_became_active":"2024-03-06T07:28:14.786452+0000","last_became_peered":"2024-03-06T07:28:14.786452+0000","last_unstale":"2024-03-06T07:28:17.399018+0000","last_undegraded":"2024-03-06T07:28:17.399018+0000","last_fullsized":"2024-03-06T07:28:17.399018+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:39.404777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397982+0000","last_change":"2024-03-06T07:28:15.799073+0000","last_active":"2024-03-06T07:28:17.397982+0000","last_peered":"2024-03-06T07:28:17.397982+0000","last_clean":"2024-03-06T07:28:17.397982+0000","last_became_active":"2024-03-06T07:28:15.798797+0000","last_became_peered":"2024-03-06T07:28:15.798797+0000","last_unstale":"2024-03-06T07:28:17.397982+0000","last_undegraded":"2024-03-06T07:28:17.397982+0000","last_fullsized":"2024-03-06T07:28:17.397982+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:47.699638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397725+0000","last_change":"2024-03-06T07:28:14.789522+0000","last_active":"2024-03-06T07:28:17.397725+0000","last_peered":"2024-03-06T07:28:17.397725+0000","last_clean":"2024-03-06T07:28:17.397725+0000","last_became_active":"2024-03-06T07:28:14.789267+0000","last_became_peered":"2024-03-06T07:28:14.789267+0000","last_unstale":"2024-03-06T07:28:17.397725+0000","last_undegraded":"2024-03-06T07:28:17.397725+0000","last_fullsized":"2024-03-06T07:28:17.397725+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:22.017770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399110+0000","last_change":"2024-03-06T07:28:15.805864+0000","last_active":"2024-03-06T07:28:17.399110+0000","last_peered":"2024-03-06T07:28:17.399110+0000","last_clean":"2024-03-06T07:28:17.399110+0000","last_became_active":"2024-03-06T07:28:15.805047+0000","last_became_peered":"2024-03-06T07:28:15.805047+0000","last_unstale":"2024-03-06T07:28:17.399110+0000","last_undegraded":"2024-03-06T07:28:17.399110+0000","last_fullsized":"2024-03-06T07:28:17.399110+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:43.923712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398163+0000","last_change":"2024-03-06T07:28:14.788385+0000","last_active":"2024-03-06T07:28:17.398163+0000","last_peered":"2024-03-06T07:28:17.398163+0000","last_clean":"2024-03-06T07:28:17.398163+0000","last_became_active":"2024-03-06T07:28:14.788113+0000","last_became_peered":"2024-03-06T07:28:14.788113+0000","last_unstale":"2024-03-06T07:28:17.398163+0000","last_undegraded":"2024-03-06T07:28:17.398163+0000","last_fullsized":"2024-03-06T07:28:17.398163+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:46.695450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398008+0000","last_change":"2024-03-06T07:28:15.802764+0000","last_active":"2024-03-06T07:28:17.398008+0000","last_peered":"2024-03-06T07:28:17.398008+0000","last_clean":"2024-03-06T07:28:17.398008+0000","last_became_active":"2024-03-06T07:28:15.802494+0000","last_became_peered":"2024-03-06T07:28:15.802494+0000","last_unstale":"2024-03-06T07:28:17.398008+0000","last_undegraded":"2024-03-06T07:28:17.398008+0000","last_fullsized":"2024-03-06T07:28:17.398008+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:59.147501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399401+0000","last_change":"2024-03-06T07:28:14.792256+0000","last_active":"2024-03-06T07:28:17.399401+0000","last_peered":"2024-03-06T07:28:17.399401+0000","last_clean":"2024-03-06T07:28:17.399401+0000","last_became_active":"2024-03-06T07:28:14.792032+0000","last_became_peered":"2024-03-06T07:28:14.792032+0000","last_unstale":"2024-03-06T07:28:17.399401+0000","last_undegraded":"2024-03-06T07:28:17.399401+0000","last_fullsized":"2024-03-06T07:28:17.399401+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:33.663609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399186+0000","last_change":"2024-03-06T07:28:15.803745+0000","last_active":"2024-03-06T07:28:17.399186+0000","last_peered":"2024-03-06T07:28:17.399186+0000","last_clean":"2024-03-06T07:28:17.399186+0000","last_became_active":"2024-03-06T07:28:15.803431+0000","last_became_peered":"2024-03-06T07:28:15.803431+0000","last_unstale":"2024-03-06T07:28:17.399186+0000","last_undegraded":"2024-03-06T07:28:17.399186+0000","last_fullsized":"2024-03-06T07:28:17.399186+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:15:17.380724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398583+0000","last_change":"2024-03-06T07:28:14.788477+0000","last_active":"2024-03-06T07:28:17.398583+0000","last_peered":"2024-03-06T07:28:17.398583+0000","last_clean":"2024-03-06T07:28:17.398583+0000","last_became_active":"2024-03-06T07:28:14.788223+0000","last_became_peered":"2024-03-06T07:28:14.788223+0000","last_unstale":"2024-03-06T07:28:17.398583+0000","last_undegraded":"2024-03-06T07:28:17.398583+0000","last_fullsized":"2024-03-06T07:28:17.398583+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:47.467299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798559+0000","last_change":"2024-03-06T07:28:15.800969+0000","last_active":"2024-03-06T07:28:16.798559+0000","last_peered":"2024-03-06T07:28:16.798559+0000","last_clean":"2024-03-06T07:28:16.798559+0000","last_became_active":"2024-03-06T07:28:15.800595+0000","last_became_peered":"2024-03-06T07:28:15.800595+0000","last_unstale":"2024-03-06T07:28:16.798559+0000","last_undegraded":"2024-03-06T07:28:16.798559+0000","last_fullsized":"2024-03-06T07:28:16.798559+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:29.853870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397821+0000","last_change":"2024-03-06T07:28:14.788356+0000","last_active":"2024-03-06T07:28:17.397821+0000","last_peered":"2024-03-06T07:28:17.397821+0000","last_clean":"2024-03-06T07:28:17.397821+0000","last_became_active":"2024-03-06T07:28:14.788093+0000","last_became_peered":"2024-03-06T07:28:14.788093+0000","last_unstale":"2024-03-06T07:28:17.397821+0000","last_undegraded":"2024-03-06T07:28:17.397821+0000","last_fullsized":"2024-03-06T07:28:17.397821+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.428223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399029+0000","last_change":"2024-03-06T07:28:15.795594+0000","last_active":"2024-03-06T07:28:17.399029+0000","last_peered":"2024-03-06T07:28:17.399029+0000","last_clean":"2024-03-06T07:28:17.399029+0000","last_became_active":"2024-03-06T07:28:15.795236+0000","last_became_peered":"2024-03-06T07:28:15.795236+0000","last_unstale":"2024-03-06T07:28:17.399029+0000","last_undegraded":"2024-03-06T07:28:17.399029+0000","last_fullsized":"2024-03-06T07:28:17.399029+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:40.908082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397691+0000","last_change":"2024-03-06T07:28:14.786359+0000","last_active":"2024-03-06T07:28:17.397691+0000","last_peered":"2024-03-06T07:28:17.397691+0000","last_clean":"2024-03-06T07:28:17.397691+0000","last_became_active":"2024-03-06T07:28:14.786068+0000","last_became_peered":"2024-03-06T07:28:14.786068+0000","last_unstale":"2024-03-06T07:28:17.397691+0000","last_undegraded":"2024-03-06T07:28:17.397691+0000","last_fullsized":"2024-03-06T07:28:17.397691+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:19.967406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398031+0000","last_change":"2024-03-06T07:28:15.806625+0000","last_active":"2024-03-06T07:28:17.398031+0000","last_peered":"2024-03-06T07:28:17.398031+0000","last_clean":"2024-03-06T07:28:17.398031+0000","last_became_active":"2024-03-06T07:28:15.806283+0000","last_became_peered":"2024-03-06T07:28:15.806283+0000","last_unstale":"2024-03-06T07:28:17.398031+0000","last_undegraded":"2024-03-06T07:28:17.398031+0000","last_fullsized":"2024-03-06T07:28:17.398031+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:05.645953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.802094+0000","last_change":"2024-03-06T07:28:14.785657+0000","last_active":"2024-03-06T07:28:16.802094+0000","last_peered":"2024-03-06T07:28:16.802094+0000","last_clean":"2024-03-06T07:28:16.802094+0000","last_became_active":"2024-03-06T07:28:14.785292+0000","last_became_peered":"2024-03-06T07:28:14.785292+0000","last_unstale":"2024-03-06T07:28:16.802094+0000","last_undegraded":"2024-03-06T07:28:16.802094+0000","last_fullsized":"2024-03-06T07:28:16.802094+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:26.845642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":502,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397826+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:28:17.397826+0000","last_peered":"2024-03-06T07:28:17.397826+0000","last_clean":"2024-03-06T07:28:17.397826+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:28:17.397826+0000","last_undegraded":"2024-03-06T07:28:17.397826+0000","last_fullsized":"2024-03-06T07:28:17.397826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2584576,"data_stored":1811644,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":2519040,"data_stored":1221264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2510848,"data_stored":1221152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":2514944,"data_stored":1221222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2535424,"data_stored":1223402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":2519040,"data_stored":1221804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37792,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334048,"statfs":{"total":96636764160,"available":96598065152,"internally_reserved":0,"allocated":2510848,"data_stored":1221152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37872,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333968,"statfs":{"total":96636764160,"available":96597983232,"internally_reserved":0,"allocated":2588672,"data_stored":1813132,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:28:24.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:28:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.509+0000 7f420e888640 1 Processor -- start 2024-03-06T07:28:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.509+0000 7f420e888640 1 -- start start 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.510+0000 7f420e888640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42081527b0 0x7f4208159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.510+0000 7f420e888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 0x7f42080c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.510+0000 7f420e888640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42080c7c60 con 0x7f42080c7180 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.510+0000 7f420e888640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4208159640 con 0x7f42081527b0 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.510+0000 7f420e888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42080c8060 con 0x7f42080c88b0 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.510+0000 7f42077fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f42080c7180 0x7f4208155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57742/0 (socket says 172.21.15.26:57742) 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.510+0000 7f42077fe640 1 -- 172.21.15.26:0/3532201331 learned_addr learned my addr 172.21.15.26:0/3532201331 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4207fff640 1 --2- 172.21.15.26:0/3532201331 >> v2:172.21.15.117:3300/0 conn(0x7f42081527b0 0x7f4208159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f420ce25640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 0x7f42080c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507052121 0 0) 0x7f41fc001110 con 0x7f42080c7180 2024-03-06T07:28:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f0003520 con 0x7f42080c7180 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3732708838 0 0) 0x7f41f0003520 con 0x7f42080c7180 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 msgr2=0x7f42080c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 0x7f42080c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 >> v2:172.21.15.117:3300/0 conn(0x7f42081527b0 msgr2=0x7f4208159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 --2- 172.21.15.26:0/3532201331 >> v2:172.21.15.117:3300/0 conn(0x7f42081527b0 0x7f4208159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080c76a0 con 0x7f42080c7180 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f41fc001840 con 0x7f42080c7180 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 >> v1:172.21.15.26:6789/0 conn(0x7f42080c7180 legacy=0x7f4208155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 0x7f41f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f41f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0004d40 0x7f41f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208159640 con 0x7f41f0003fe0 2024-03-06T07:28:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42080c8060 con 0x7f41f0004d40 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f000bb80 con 0x7f42080c88b0 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4207fff640 1 --2- 172.21.15.26:0/3532201331 >> v2:172.21.15.117:3300/0 conn(0x7f42081527b0 0x7f4208159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f42077fe640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f41f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f420ce25640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0004d40 0x7f41f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f4207fff640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 0x7f41f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f42077fe640 1 -- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 msgr2=0x7f41f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f42077fe640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080c88b0 0x7f41f00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f42077fe640 1 -- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0004d40 msgr2=0x7f41f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f42077fe640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0004d40 0x7f41f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f42077fe640 1 -- 172.21.15.26:0/3532201331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0003520 con 0x7f41f0003fe0 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.511+0000 7f420ce25640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0004d40 0x7f41f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.512+0000 7f42077fe640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f41f0004800 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c3e0 tx=0x7f41f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e594550d446ccc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.512+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00c0e0 con 0x7f41f0003fe0 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.512+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc030070 con 0x7f41f0003fe0 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.512+0000 7f4206ffd640 1 -- 172.21.15.26:0/3532201331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0058d0 con 0x7f41f0003fe0 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.513+0000 7f420e888640 1 -- 172.21.15.26:0/3532201331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 msgr2=0x7f41f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.513+0000 7f420e888640 1 --2- 172.21.15.26:0/3532201331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f41f0004800 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c3e0 tx=0x7f41f0005830 comp rx=0 tx=0).stop 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.513+0000 7f420ce25640 1 -- 172.21.15.26:0/3532201331 reap_dead start 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.513+0000 7f420e888640 1 -- 172.21.15.26:0/3532201331 shutdown_connections 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.513+0000 7f420e888640 1 -- 172.21.15.26:0/3532201331 >> 172.21.15.26:0/3532201331 conn(0x7f42080ac560 msgr2=0x7f42080ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.513+0000 7f420e888640 1 -- 172.21.15.26:0/3532201331 shutdown_connections 2024-03-06T07:28:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.513+0000 7f420e888640 1 -- 172.21.15.26:0/3532201331 wait complete. 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.514+0000 7f420e888640 1 Processor -- start 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.514+0000 7f420e888640 1 -- start start 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f4208149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42081580f0 0x7f420814af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42081eedf0 0x7f42081eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208155600 con 0x7f41f0003fe0 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208155320 con 0x7f42081580f0 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42080c8d60 con 0x7f42081eedf0 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f4208149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f4208149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43986/0 (socket says 172.21.15.26:43986) 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 -- 172.21.15.26:0/551408437 learned_addr learned my addr 172.21.15.26:0/551408437 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 -- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42081eedf0 msgr2=0x7f42081eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42081eedf0 0x7f42081eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 -- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42081580f0 msgr2=0x7f420814af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f42077fe640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42081580f0 0x7f420814af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42081580f0 0x7f420814af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 -- 172.21.15.26:0/551408437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208156b50 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4207fff640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f4208149040 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f41f8010760 tx=0x7f41f8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4204ff9640 1 -- 172.21.15.26:0/551408437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f801a020 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4204ff9640 1 -- 172.21.15.26:0/551408437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f800eb60 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4208158620 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f4204ff9640 1 -- 172.21.15.26:0/551408437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800f040 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.515+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42081ee5f0 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.517+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420804f330 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.517+0000 7f4204ff9640 1 -- 172.21.15.26:0/551408437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41f801d020 con 0x7f41f0003fe0 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.517+0000 7f4204ff9640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f406b4d0 0x7f41f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.518+0000 7f4204ff9640 1 -- 172.21.15.26:0/551408437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f41f808ac00 con 0x7f41f0003fe0 2024-03-06T07:28:24.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.518+0000 7f42077fe640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f406b4d0 0x7f41f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.518+0000 7f42077fe640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f406b4d0 0x7f41f406d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f41fc034000 tx=0x7f41fc007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:24.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.522+0000 7f4204ff9640 1 -- 172.21.15.26:0/551408437 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f41f8091080 con 0x7f41f0003fe0 2024-03-06T07:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.699+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42080c1e90 con 0x7f41f406b4d0 2024-03-06T07:28:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.706+0000 7f4204ff9640 1 -- 172.21.15.26:0/551408437 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252567 (secure 0 0 0) 0x7f42080c1e90 con 0x7f41f406b4d0 2024-03-06T07:28:24.738 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:28:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f406b4d0 msgr2=0x7f41f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f406b4d0 0x7f41f406d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f41fc034000 tx=0x7f41fc007000 comp rx=0 tx=0).stop 2024-03-06T07:28:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 msgr2=0x7f4208149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f4208149040 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f41f8010760 tx=0x7f41f8010c20 comp rx=0 tx=0).stop 2024-03-06T07:28:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 shutdown_connections 2024-03-06T07:28:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f406b4d0 0x7f41f406d990 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42081eedf0 0x7f42081eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42081580f0 0x7f420814af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 --2- 172.21.15.26:0/551408437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003fe0 0x7f4208149040 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 >> 172.21.15.26:0/551408437 conn(0x7f42080ac560 msgr2=0x7f42081516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 shutdown_connections 2024-03-06T07:28:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.708+0000 7f420e888640 1 -- 172.21.15.26:0/551408437 wait complete. 2024-03-06T07:28:24.742 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-03-06T07:28:23.136832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281744,"kb_used_data":19808,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754692976,"statfs":{"total":773094113280,"available":772805607424,"internally_reserved":0,"allocated":20283392,"data_stored":10954772,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002133"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398067+0000","last_change":"2024-03-06T07:28:15.802088+0000","last_active":"2024-03-06T07:28:17.398067+0000","last_peered":"2024-03-06T07:28:17.398067+0000","last_clean":"2024-03-06T07:28:17.398067+0000","last_became_active":"2024-03-06T07:28:15.801526+0000","last_became_peered":"2024-03-06T07:28:15.801526+0000","last_unstale":"2024-03-06T07:28:17.398067+0000","last_undegraded":"2024-03-06T07:28:17.398067+0000","last_fullsized":"2024-03-06T07:28:17.398067+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:25.951020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798698+0000","last_change":"2024-03-06T07:28:15.802062+0000","last_active":"2024-03-06T07:28:16.798698+0000","last_peered":"2024-03-06T07:28:16.798698+0000","last_clean":"2024-03-06T07:28:16.798698+0000","last_became_active":"2024-03-06T07:28:15.801697+0000","last_became_peered":"2024-03-06T07:28:15.801697+0000","last_unstale":"2024-03-06T07:28:16.798698+0000","last_undegraded":"2024-03-06T07:28:16.798698+0000","last_fullsized":"2024-03-06T07:28:16.798698+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:36.476028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398994+0000","last_change":"2024-03-06T07:28:15.803607+0000","last_active":"2024-03-06T07:28:17.398994+0000","last_peered":"2024-03-06T07:28:17.398994+0000","last_clean":"2024-03-06T07:28:17.398994+0000","last_became_active":"2024-03-06T07:28:15.803339+0000","last_became_peered":"2024-03-06T07:28:15.803339+0000","last_unstale":"2024-03-06T07:28:17.398994+0000","last_undegraded":"2024-03-06T07:28:17.398994+0000","last_fullsized":"2024-03-06T07:28:17.398994+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:16:27.334487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398825+0000","last_change":"2024-03-06T07:28:15.800331+0000","last_active":"2024-03-06T07:28:17.398825+0000","last_peered":"2024-03-06T07:28:17.398825+0000","last_clean":"2024-03-06T07:28:17.398825+0000","last_became_active":"2024-03-06T07:28:15.800042+0000","last_became_peered":"2024-03-06T07:28:15.800042+0000","last_unstale":"2024-03-06T07:28:17.398825+0000","last_undegraded":"2024-03-06T07:28:17.398825+0000","last_fullsized":"2024-03-06T07:28:17.398825+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:32.608124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399009+0000","last_change":"2024-03-06T07:28:15.796723+0000","last_active":"2024-03-06T07:28:17.399009+0000","last_peered":"2024-03-06T07:28:17.399009+0000","last_clean":"2024-03-06T07:28:17.399009+0000","last_became_active":"2024-03-06T07:28:15.796474+0000","last_became_peered":"2024-03-06T07:28:15.796474+0000","last_unstale":"2024-03-06T07:28:17.399009+0000","last_undegraded":"2024-03-06T07:28:17.399009+0000","last_fullsized":"2024-03-06T07:28:17.399009+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:48.433133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398415+0000","last_change":"2024-03-06T07:28:15.796822+0000","last_active":"2024-03-06T07:28:17.398415+0000","last_peered":"2024-03-06T07:28:17.398415+0000","last_clean":"2024-03-06T07:28:17.398415+0000","last_became_active":"2024-03-06T07:28:15.796531+0000","last_became_peered":"2024-03-06T07:28:15.796531+0000","last_unstale":"2024-03-06T07:28:17.398415+0000","last_undegraded":"2024-03-06T07:28:17.398415+0000","last_fullsized":"2024-03-06T07:28:17.398415+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:45:11.333135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801572+0000","last_change":"2024-03-06T07:28:15.796769+0000","last_active":"2024-03-06T07:28:16.801572+0000","last_peered":"2024-03-06T07:28:16.801572+0000","last_clean":"2024-03-06T07:28:16.801572+0000","last_became_active":"2024-03-06T07:28:15.796493+0000","last_became_peered":"2024-03-06T07:28:15.796493+0000","last_unstale":"2024-03-06T07:28:16.801572+0000","last_undegraded":"2024-03-06T07:28:16.801572+0000","last_fullsized":"2024-03-06T07:28:16.801572+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:45.040241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398535+0000","last_change":"2024-03-06T07:28:15.795513+0000","last_active":"2024-03-06T07:28:17.398535+0000","last_peered":"2024-03-06T07:28:17.398535+0000","last_clean":"2024-03-06T07:28:17.398535+0000","last_became_active":"2024-03-06T07:28:15.795195+0000","last_became_peered":"2024-03-06T07:28:15.795195+0000","last_unstale":"2024-03-06T07:28:17.398535+0000","last_undegraded":"2024-03-06T07:28:17.398535+0000","last_fullsized":"2024-03-06T07:28:17.398535+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:50.682583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398254+0000","last_change":"2024-03-06T07:28:15.801827+0000","last_active":"2024-03-06T07:28:17.398254+0000","last_peered":"2024-03-06T07:28:17.398254+0000","last_clean":"2024-03-06T07:28:17.398254+0000","last_became_active":"2024-03-06T07:28:15.801167+0000","last_became_peered":"2024-03-06T07:28:15.801167+0000","last_unstale":"2024-03-06T07:28:17.398254+0000","last_undegraded":"2024-03-06T07:28:17.398254+0000","last_fullsized":"2024-03-06T07:28:17.398254+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.449435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399028+0000","last_change":"2024-03-06T07:28:15.805896+0000","last_active":"2024-03-06T07:28:17.399028+0000","last_peered":"2024-03-06T07:28:17.399028+0000","last_clean":"2024-03-06T07:28:17.399028+0000","last_became_active":"2024-03-06T07:28:15.805001+0000","last_became_peered":"2024-03-06T07:28:15.805001+0000","last_unstale":"2024-03-06T07:28:17.399028+0000","last_undegraded":"2024-03-06T07:28:17.399028+0000","last_fullsized":"2024-03-06T07:28:17.399028+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:58:41.341945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398988+0000","last_change":"2024-03-06T07:28:15.801304+0000","last_active":"2024-03-06T07:28:17.398988+0000","last_peered":"2024-03-06T07:28:17.398988+0000","last_clean":"2024-03-06T07:28:17.398988+0000","last_became_active":"2024-03-06T07:28:15.801023+0000","last_became_peered":"2024-03-06T07:28:15.801023+0000","last_unstale":"2024-03-06T07:28:17.398988+0000","last_undegraded":"2024-03-06T07:28:17.398988+0000","last_fullsized":"2024-03-06T07:28:17.398988+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:44.998092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398285+0000","last_change":"2024-03-06T07:28:15.798814+0000","last_active":"2024-03-06T07:28:17.398285+0000","last_peered":"2024-03-06T07:28:17.398285+0000","last_clean":"2024-03-06T07:28:17.398285+0000","last_became_active":"2024-03-06T07:28:15.798465+0000","last_became_peered":"2024-03-06T07:28:15.798465+0000","last_unstale":"2024-03-06T07:28:17.398285+0000","last_undegraded":"2024-03-06T07:28:17.398285+0000","last_fullsized":"2024-03-06T07:28:17.398285+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:37:43.332054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399408+0000","last_change":"2024-03-06T07:28:15.800830+0000","last_active":"2024-03-06T07:28:17.399408+0000","last_peered":"2024-03-06T07:28:17.399408+0000","last_clean":"2024-03-06T07:28:17.399408+0000","last_became_active":"2024-03-06T07:28:15.800409+0000","last_became_peered":"2024-03-06T07:28:15.800409+0000","last_unstale":"2024-03-06T07:28:17.399408+0000","last_undegraded":"2024-03-06T07:28:17.399408+0000","last_fullsized":"2024-03-06T07:28:17.399408+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:00:15.287292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397625+0000","last_change":"2024-03-06T07:28:15.801204+0000","last_active":"2024-03-06T07:28:17.397625+0000","last_peered":"2024-03-06T07:28:17.397625+0000","last_clean":"2024-03-06T07:28:17.397625+0000","last_became_active":"2024-03-06T07:28:15.800407+0000","last_became_peered":"2024-03-06T07:28:15.800407+0000","last_unstale":"2024-03-06T07:28:17.397625+0000","last_undegraded":"2024-03-06T07:28:17.397625+0000","last_fullsized":"2024-03-06T07:28:17.397625+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:57.609973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398671+0000","last_change":"2024-03-06T07:28:15.795936+0000","last_active":"2024-03-06T07:28:17.398671+0000","last_peered":"2024-03-06T07:28:17.398671+0000","last_clean":"2024-03-06T07:28:17.398671+0000","last_became_active":"2024-03-06T07:28:15.795650+0000","last_became_peered":"2024-03-06T07:28:15.795650+0000","last_unstale":"2024-03-06T07:28:17.398671+0000","last_undegraded":"2024-03-06T07:28:17.398671+0000","last_fullsized":"2024-03-06T07:28:17.398671+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:47.699638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399085+0000","last_change":"2024-03-06T07:28:15.799715+0000","last_active":"2024-03-06T07:28:17.399085+0000","last_peered":"2024-03-06T07:28:17.399085+0000","last_clean":"2024-03-06T07:28:17.399085+0000","last_became_active":"2024-03-06T07:28:15.799422+0000","last_became_peered":"2024-03-06T07:28:15.799422+0000","last_unstale":"2024-03-06T07:28:17.399085+0000","last_undegraded":"2024-03-06T07:28:17.399085+0000","last_fullsized":"2024-03-06T07:28:17.399085+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:57.609973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399219+0000","last_change":"2024-03-06T07:28:15.797121+0000","last_active":"2024-03-06T07:28:17.399219+0000","last_peered":"2024-03-06T07:28:17.399219+0000","last_clean":"2024-03-06T07:28:17.399219+0000","last_became_active":"2024-03-06T07:28:15.796775+0000","last_became_peered":"2024-03-06T07:28:15.796775+0000","last_unstale":"2024-03-06T07:28:17.399219+0000","last_undegraded":"2024-03-06T07:28:17.399219+0000","last_fullsized":"2024-03-06T07:28:17.399219+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:46:03.560601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399024+0000","last_change":"2024-03-06T07:28:15.810661+0000","last_active":"2024-03-06T07:28:17.399024+0000","last_peered":"2024-03-06T07:28:17.399024+0000","last_clean":"2024-03-06T07:28:17.399024+0000","last_became_active":"2024-03-06T07:28:15.805348+0000","last_became_peered":"2024-03-06T07:28:15.805348+0000","last_unstale":"2024-03-06T07:28:17.399024+0000","last_undegraded":"2024-03-06T07:28:17.399024+0000","last_fullsized":"2024-03-06T07:28:17.399024+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:35:26.179502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398898+0000","last_change":"2024-03-06T07:28:15.803699+0000","last_active":"2024-03-06T07:28:17.398898+0000","last_peered":"2024-03-06T07:28:17.398898+0000","last_clean":"2024-03-06T07:28:17.398898+0000","last_became_active":"2024-03-06T07:28:15.803220+0000","last_became_peered":"2024-03-06T07:28:15.803220+0000","last_unstale":"2024-03-06T07:28:17.398898+0000","last_undegraded":"2024-03-06T07:28:17.398898+0000","last_fullsized":"2024-03-06T07:28:17.398898+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:15.839156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798600+0000","last_change":"2024-03-06T07:28:15.806052+0000","last_active":"2024-03-06T07:28:16.798600+0000","last_peered":"2024-03-06T07:28:16.798600+0000","last_clean":"2024-03-06T07:28:16.798600+0000","last_became_active":"2024-03-06T07:28:15.805178+0000","last_became_peered":"2024-03-06T07:28:15.805178+0000","last_unstale":"2024-03-06T07:28:16.798600+0000","last_undegraded":"2024-03-06T07:28:16.798600+0000","last_fullsized":"2024-03-06T07:28:16.798600+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:36.959393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801734+0000","last_change":"2024-03-06T07:28:15.789673+0000","last_active":"2024-03-06T07:28:16.801734+0000","last_peered":"2024-03-06T07:28:16.801734+0000","last_clean":"2024-03-06T07:28:16.801734+0000","last_became_active":"2024-03-06T07:28:15.789403+0000","last_became_peered":"2024-03-06T07:28:15.789403+0000","last_unstale":"2024-03-06T07:28:16.801734+0000","last_undegraded":"2024-03-06T07:28:16.801734+0000","last_fullsized":"2024-03-06T07:28:16.801734+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:32.570331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398532+0000","last_change":"2024-03-06T07:28:15.789477+0000","last_active":"2024-03-06T07:28:17.398532+0000","last_peered":"2024-03-06T07:28:17.398532+0000","last_clean":"2024-03-06T07:28:17.398532+0000","last_became_active":"2024-03-06T07:28:15.788910+0000","last_became_peered":"2024-03-06T07:28:15.788910+0000","last_unstale":"2024-03-06T07:28:17.398532+0000","last_undegraded":"2024-03-06T07:28:17.398532+0000","last_fullsized":"2024-03-06T07:28:17.398532+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:18.231291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398835+0000","last_change":"2024-03-06T07:28:15.797621+0000","last_active":"2024-03-06T07:28:17.398835+0000","last_peered":"2024-03-06T07:28:17.398835+0000","last_clean":"2024-03-06T07:28:17.398835+0000","last_became_active":"2024-03-06T07:28:15.797313+0000","last_became_peered":"2024-03-06T07:28:15.797313+0000","last_unstale":"2024-03-06T07:28:17.398835+0000","last_undegraded":"2024-03-06T07:28:17.398835+0000","last_fullsized":"2024-03-06T07:28:17.398835+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:58.939766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397764+0000","last_change":"2024-03-06T07:28:15.789861+0000","last_active":"2024-03-06T07:28:17.397764+0000","last_peered":"2024-03-06T07:28:17.397764+0000","last_clean":"2024-03-06T07:28:17.397764+0000","last_became_active":"2024-03-06T07:28:15.789574+0000","last_became_peered":"2024-03-06T07:28:15.789574+0000","last_unstale":"2024-03-06T07:28:17.397764+0000","last_undegraded":"2024-03-06T07:28:17.397764+0000","last_fullsized":"2024-03-06T07:28:17.397764+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:40.408964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798918+0000","last_change":"2024-03-06T07:28:15.807619+0000","last_active":"2024-03-06T07:28:16.798918+0000","last_peered":"2024-03-06T07:28:16.798918+0000","last_clean":"2024-03-06T07:28:16.798918+0000","last_became_active":"2024-03-06T07:28:15.806880+0000","last_became_peered":"2024-03-06T07:28:15.806880+0000","last_unstale":"2024-03-06T07:28:16.798918+0000","last_undegraded":"2024-03-06T07:28:16.798918+0000","last_fullsized":"2024-03-06T07:28:16.798918+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:49.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.802128+0000","last_change":"2024-03-06T07:28:15.796807+0000","last_active":"2024-03-06T07:28:16.802128+0000","last_peered":"2024-03-06T07:28:16.802128+0000","last_clean":"2024-03-06T07:28:16.802128+0000","last_became_active":"2024-03-06T07:28:15.796525+0000","last_became_peered":"2024-03-06T07:28:15.796525+0000","last_unstale":"2024-03-06T07:28:16.802128+0000","last_undegraded":"2024-03-06T07:28:16.802128+0000","last_fullsized":"2024-03-06T07:28:16.802128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:56.183328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398207+0000","last_change":"2024-03-06T07:28:15.801409+0000","last_active":"2024-03-06T07:28:17.398207+0000","last_peered":"2024-03-06T07:28:17.398207+0000","last_clean":"2024-03-06T07:28:17.398207+0000","last_became_active":"2024-03-06T07:28:15.801120+0000","last_became_peered":"2024-03-06T07:28:15.801120+0000","last_unstale":"2024-03-06T07:28:17.398207+0000","last_undegraded":"2024-03-06T07:28:17.398207+0000","last_fullsized":"2024-03-06T07:28:17.398207+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:27.849829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398547+0000","last_change":"2024-03-06T07:28:15.800350+0000","last_active":"2024-03-06T07:28:17.398547+0000","last_peered":"2024-03-06T07:28:17.398547+0000","last_clean":"2024-03-06T07:28:17.398547+0000","last_became_active":"2024-03-06T07:28:15.799655+0000","last_became_peered":"2024-03-06T07:28:15.799655+0000","last_unstale":"2024-03-06T07:28:17.398547+0000","last_undegraded":"2024-03-06T07:28:17.398547+0000","last_fullsized":"2024-03-06T07:28:17.398547+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:39:06.987968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398768+0000","last_change":"2024-03-06T07:28:15.799608+0000","last_active":"2024-03-06T07:28:17.398768+0000","last_peered":"2024-03-06T07:28:17.398768+0000","last_clean":"2024-03-06T07:28:17.398768+0000","last_became_active":"2024-03-06T07:28:15.799303+0000","last_became_peered":"2024-03-06T07:28:15.799303+0000","last_unstale":"2024-03-06T07:28:17.398768+0000","last_undegraded":"2024-03-06T07:28:17.398768+0000","last_fullsized":"2024-03-06T07:28:17.398768+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:44:02.229882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397785+0000","last_change":"2024-03-06T07:28:15.799983+0000","last_active":"2024-03-06T07:28:17.397785+0000","last_peered":"2024-03-06T07:28:17.397785+0000","last_clean":"2024-03-06T07:28:17.397785+0000","last_became_active":"2024-03-06T07:28:15.799645+0000","last_became_peered":"2024-03-06T07:28:15.799645+0000","last_unstale":"2024-03-06T07:28:17.397785+0000","last_undegraded":"2024-03-06T07:28:17.397785+0000","last_fullsized":"2024-03-06T07:28:17.397785+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:25.561669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798492+0000","last_change":"2024-03-06T07:28:15.794790+0000","last_active":"2024-03-06T07:28:16.798492+0000","last_peered":"2024-03-06T07:28:16.798492+0000","last_clean":"2024-03-06T07:28:16.798492+0000","last_became_active":"2024-03-06T07:28:15.794361+0000","last_became_peered":"2024-03-06T07:28:15.794361+0000","last_unstale":"2024-03-06T07:28:16.798492+0000","last_undegraded":"2024-03-06T07:28:16.798492+0000","last_fullsized":"2024-03-06T07:28:16.798492+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:58.576387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397663+0000","last_change":"2024-03-06T07:28:15.802386+0000","last_active":"2024-03-06T07:28:17.397663+0000","last_peered":"2024-03-06T07:28:17.397663+0000","last_clean":"2024-03-06T07:28:17.397663+0000","last_became_active":"2024-03-06T07:28:15.799297+0000","last_became_peered":"2024-03-06T07:28:15.799297+0000","last_unstale":"2024-03-06T07:28:17.397663+0000","last_undegraded":"2024-03-06T07:28:17.397663+0000","last_fullsized":"2024-03-06T07:28:17.397663+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:31:54.984328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798314+0000","last_change":"2024-03-06T07:28:15.801031+0000","last_active":"2024-03-06T07:28:16.798314+0000","last_peered":"2024-03-06T07:28:16.798314+0000","last_clean":"2024-03-06T07:28:16.798314+0000","last_became_active":"2024-03-06T07:28:15.800738+0000","last_became_peered":"2024-03-06T07:28:15.800738+0000","last_unstale":"2024-03-06T07:28:16.798314+0000","last_undegraded":"2024-03-06T07:28:16.798314+0000","last_fullsized":"2024-03-06T07:28:16.798314+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:22.827277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"197'4","reported_seq":18,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398077+0000","last_change":"2024-03-06T07:28:14.781972+0000","last_active":"2024-03-06T07:28:17.398077+0000","last_peered":"2024-03-06T07:28:17.398077+0000","last_clean":"2024-03-06T07:28:17.398077+0000","last_became_active":"2024-03-06T07:28:14.781691+0000","last_became_peered":"2024-03-06T07:28:14.781691+0000","last_unstale":"2024-03-06T07:28:17.398077+0000","last_undegraded":"2024-03-06T07:28:17.398077+0000","last_fullsized":"2024-03-06T07:28:17.398077+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:19:29.750140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398155+0000","last_change":"2024-03-06T07:28:15.798509+0000","last_active":"2024-03-06T07:28:17.398155+0000","last_peered":"2024-03-06T07:28:17.398155+0000","last_clean":"2024-03-06T07:28:17.398155+0000","last_became_active":"2024-03-06T07:28:15.798222+0000","last_became_peered":"2024-03-06T07:28:15.798222+0000","last_unstale":"2024-03-06T07:28:17.398155+0000","last_undegraded":"2024-03-06T07:28:17.398155+0000","last_fullsized":"2024-03-06T07:28:17.398155+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:23.021957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398143+0000","last_change":"2024-03-06T07:28:14.788442+0000","last_active":"2024-03-06T07:28:17.398143+0000","last_peered":"2024-03-06T07:28:17.398143+0000","last_clean":"2024-03-06T07:28:17.398143+0000","last_became_active":"2024-03-06T07:28:14.788075+0000","last_became_peered":"2024-03-06T07:28:14.788075+0000","last_unstale":"2024-03-06T07:28:17.398143+0000","last_undegraded":"2024-03-06T07:28:17.398143+0000","last_fullsized":"2024-03-06T07:28:17.398143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:11.511138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398279+0000","last_change":"2024-03-06T07:28:15.796288+0000","last_active":"2024-03-06T07:28:17.398279+0000","last_peered":"2024-03-06T07:28:17.398279+0000","last_clean":"2024-03-06T07:28:17.398279+0000","last_became_active":"2024-03-06T07:28:15.796000+0000","last_became_peered":"2024-03-06T07:28:15.796000+0000","last_unstale":"2024-03-06T07:28:17.398279+0000","last_undegraded":"2024-03-06T07:28:17.398279+0000","last_fullsized":"2024-03-06T07:28:17.398279+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:52.339463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398142+0000","last_change":"2024-03-06T07:28:14.791895+0000","last_active":"2024-03-06T07:28:17.398142+0000","last_peered":"2024-03-06T07:28:17.398142+0000","last_clean":"2024-03-06T07:28:17.398142+0000","last_became_active":"2024-03-06T07:28:14.791606+0000","last_became_peered":"2024-03-06T07:28:14.791606+0000","last_unstale":"2024-03-06T07:28:17.398142+0000","last_undegraded":"2024-03-06T07:28:17.398142+0000","last_fullsized":"2024-03-06T07:28:17.398142+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:54.513290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398053+0000","last_change":"2024-03-06T07:28:15.808840+0000","last_active":"2024-03-06T07:28:17.398053+0000","last_peered":"2024-03-06T07:28:17.398053+0000","last_clean":"2024-03-06T07:28:17.398053+0000","last_became_active":"2024-03-06T07:28:15.808536+0000","last_became_peered":"2024-03-06T07:28:15.808536+0000","last_unstale":"2024-03-06T07:28:17.398053+0000","last_undegraded":"2024-03-06T07:28:17.398053+0000","last_fullsized":"2024-03-06T07:28:17.398053+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:51:55.356703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398571+0000","last_change":"2024-03-06T07:28:14.783699+0000","last_active":"2024-03-06T07:28:17.398571+0000","last_peered":"2024-03-06T07:28:17.398571+0000","last_clean":"2024-03-06T07:28:17.398571+0000","last_became_active":"2024-03-06T07:28:14.783452+0000","last_became_peered":"2024-03-06T07:28:14.783452+0000","last_unstale":"2024-03-06T07:28:17.398571+0000","last_undegraded":"2024-03-06T07:28:17.398571+0000","last_fullsized":"2024-03-06T07:28:17.398571+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:15.422272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801921+0000","last_change":"2024-03-06T07:28:15.792944+0000","last_active":"2024-03-06T07:28:16.801921+0000","last_peered":"2024-03-06T07:28:16.801921+0000","last_clean":"2024-03-06T07:28:16.801921+0000","last_became_active":"2024-03-06T07:28:15.792695+0000","last_became_peered":"2024-03-06T07:28:15.792695+0000","last_unstale":"2024-03-06T07:28:16.801921+0000","last_undegraded":"2024-03-06T07:28:16.801921+0000","last_fullsized":"2024-03-06T07:28:16.801921+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:51.553861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397867+0000","last_change":"2024-03-06T07:28:14.786831+0000","last_active":"2024-03-06T07:28:17.397867+0000","last_peered":"2024-03-06T07:28:17.397867+0000","last_clean":"2024-03-06T07:28:17.397867+0000","last_became_active":"2024-03-06T07:28:14.786548+0000","last_became_peered":"2024-03-06T07:28:14.786548+0000","last_unstale":"2024-03-06T07:28:17.397867+0000","last_undegraded":"2024-03-06T07:28:17.397867+0000","last_fullsized":"2024-03-06T07:28:17.397867+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:37.445975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397710+0000","last_change":"2024-03-06T07:28:15.810738+0000","last_active":"2024-03-06T07:28:17.397710+0000","last_peered":"2024-03-06T07:28:17.397710+0000","last_clean":"2024-03-06T07:28:17.397710+0000","last_became_active":"2024-03-06T07:28:15.810446+0000","last_became_peered":"2024-03-06T07:28:15.810446+0000","last_unstale":"2024-03-06T07:28:17.397710+0000","last_undegraded":"2024-03-06T07:28:17.397710+0000","last_fullsized":"2024-03-06T07:28:17.397710+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:55.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397732+0000","last_change":"2024-03-06T07:28:14.782671+0000","last_active":"2024-03-06T07:28:17.397732+0000","last_peered":"2024-03-06T07:28:17.397732+0000","last_clean":"2024-03-06T07:28:17.397732+0000","last_became_active":"2024-03-06T07:28:14.782382+0000","last_became_peered":"2024-03-06T07:28:14.782382+0000","last_unstale":"2024-03-06T07:28:17.397732+0000","last_undegraded":"2024-03-06T07:28:17.397732+0000","last_fullsized":"2024-03-06T07:28:17.397732+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:41.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398705+0000","last_change":"2024-03-06T07:28:15.802769+0000","last_active":"2024-03-06T07:28:17.398705+0000","last_peered":"2024-03-06T07:28:17.398705+0000","last_clean":"2024-03-06T07:28:17.398705+0000","last_became_active":"2024-03-06T07:28:15.802369+0000","last_became_peered":"2024-03-06T07:28:15.802369+0000","last_unstale":"2024-03-06T07:28:17.398705+0000","last_undegraded":"2024-03-06T07:28:17.398705+0000","last_fullsized":"2024-03-06T07:28:17.398705+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:35.086433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397754+0000","last_change":"2024-03-06T07:28:14.776124+0000","last_active":"2024-03-06T07:28:17.397754+0000","last_peered":"2024-03-06T07:28:17.397754+0000","last_clean":"2024-03-06T07:28:17.397754+0000","last_became_active":"2024-03-06T07:28:14.775819+0000","last_became_peered":"2024-03-06T07:28:14.775819+0000","last_unstale":"2024-03-06T07:28:17.397754+0000","last_undegraded":"2024-03-06T07:28:17.397754+0000","last_fullsized":"2024-03-06T07:28:17.397754+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:41.621389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801571+0000","last_change":"2024-03-06T07:28:15.793560+0000","last_active":"2024-03-06T07:28:16.801571+0000","last_peered":"2024-03-06T07:28:16.801571+0000","last_clean":"2024-03-06T07:28:16.801571+0000","last_became_active":"2024-03-06T07:28:15.793292+0000","last_became_peered":"2024-03-06T07:28:15.793292+0000","last_unstale":"2024-03-06T07:28:16.801571+0000","last_undegraded":"2024-03-06T07:28:16.801571+0000","last_fullsized":"2024-03-06T07:28:16.801571+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:38:14.146901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398462+0000","last_change":"2024-03-06T07:28:14.789097+0000","last_active":"2024-03-06T07:28:17.398462+0000","last_peered":"2024-03-06T07:28:17.398462+0000","last_clean":"2024-03-06T07:28:17.398462+0000","last_became_active":"2024-03-06T07:28:14.788833+0000","last_became_peered":"2024-03-06T07:28:14.788833+0000","last_unstale":"2024-03-06T07:28:17.398462+0000","last_undegraded":"2024-03-06T07:28:17.398462+0000","last_fullsized":"2024-03-06T07:28:17.398462+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:14.834969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398034+0000","last_change":"2024-03-06T07:28:15.794447+0000","last_active":"2024-03-06T07:28:17.398034+0000","last_peered":"2024-03-06T07:28:17.398034+0000","last_clean":"2024-03-06T07:28:17.398034+0000","last_became_active":"2024-03-06T07:28:15.794115+0000","last_became_peered":"2024-03-06T07:28:15.794115+0000","last_unstale":"2024-03-06T07:28:17.398034+0000","last_undegraded":"2024-03-06T07:28:17.398034+0000","last_fullsized":"2024-03-06T07:28:17.398034+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:54.643304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399137+0000","last_change":"2024-03-06T07:28:14.787649+0000","last_active":"2024-03-06T07:28:17.399137+0000","last_peered":"2024-03-06T07:28:17.399137+0000","last_clean":"2024-03-06T07:28:17.399137+0000","last_became_active":"2024-03-06T07:28:14.787348+0000","last_became_peered":"2024-03-06T07:28:14.787348+0000","last_unstale":"2024-03-06T07:28:17.399137+0000","last_undegraded":"2024-03-06T07:28:17.399137+0000","last_fullsized":"2024-03-06T07:28:17.399137+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:50.549674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398121+0000","last_change":"2024-03-06T07:28:15.798455+0000","last_active":"2024-03-06T07:28:17.398121+0000","last_peered":"2024-03-06T07:28:17.398121+0000","last_clean":"2024-03-06T07:28:17.398121+0000","last_became_active":"2024-03-06T07:28:15.798172+0000","last_became_peered":"2024-03-06T07:28:15.798172+0000","last_unstale":"2024-03-06T07:28:17.398121+0000","last_undegraded":"2024-03-06T07:28:17.398121+0000","last_fullsized":"2024-03-06T07:28:17.398121+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:58.939766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"197'3","reported_seq":17,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398315+0000","last_change":"2024-03-06T07:28:14.785132+0000","last_active":"2024-03-06T07:28:17.398315+0000","last_peered":"2024-03-06T07:28:17.398315+0000","last_clean":"2024-03-06T07:28:17.398315+0000","last_became_active":"2024-03-06T07:28:14.784849+0000","last_became_peered":"2024-03-06T07:28:14.784849+0000","last_unstale":"2024-03-06T07:28:17.398315+0000","last_undegraded":"2024-03-06T07:28:17.398315+0000","last_fullsized":"2024-03-06T07:28:17.398315+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:17.227104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801727+0000","last_change":"2024-03-06T07:28:15.797828+0000","last_active":"2024-03-06T07:28:16.801727+0000","last_peered":"2024-03-06T07:28:16.801727+0000","last_clean":"2024-03-06T07:28:16.801727+0000","last_became_active":"2024-03-06T07:28:15.797537+0000","last_became_peered":"2024-03-06T07:28:15.797537+0000","last_unstale":"2024-03-06T07:28:16.801727+0000","last_undegraded":"2024-03-06T07:28:16.801727+0000","last_fullsized":"2024-03-06T07:28:16.801727+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:12.511505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"197'6","reported_seq":20,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398398+0000","last_change":"2024-03-06T07:28:14.785822+0000","last_active":"2024-03-06T07:28:17.398398+0000","last_peered":"2024-03-06T07:28:17.398398+0000","last_clean":"2024-03-06T07:28:17.398398+0000","last_became_active":"2024-03-06T07:28:14.785540+0000","last_became_peered":"2024-03-06T07:28:14.785540+0000","last_unstale":"2024-03-06T07:28:17.398398+0000","last_undegraded":"2024-03-06T07:28:17.398398+0000","last_fullsized":"2024-03-06T07:28:17.398398+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:35:52.319705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798568+0000","last_change":"2024-03-06T07:28:15.806312+0000","last_active":"2024-03-06T07:28:16.798568+0000","last_peered":"2024-03-06T07:28:16.798568+0000","last_clean":"2024-03-06T07:28:16.798568+0000","last_became_active":"2024-03-06T07:28:15.805665+0000","last_became_peered":"2024-03-06T07:28:15.805665+0000","last_unstale":"2024-03-06T07:28:16.798568+0000","last_undegraded":"2024-03-06T07:28:16.798568+0000","last_fullsized":"2024-03-06T07:28:16.798568+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:19:49.143984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398683+0000","last_change":"2024-03-06T07:28:14.784265+0000","last_active":"2024-03-06T07:28:17.398683+0000","last_peered":"2024-03-06T07:28:17.398683+0000","last_clean":"2024-03-06T07:28:17.398683+0000","last_became_active":"2024-03-06T07:28:14.783985+0000","last_became_peered":"2024-03-06T07:28:14.783985+0000","last_unstale":"2024-03-06T07:28:17.398683+0000","last_undegraded":"2024-03-06T07:28:17.398683+0000","last_fullsized":"2024-03-06T07:28:17.398683+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:53:56.571551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801758+0000","last_change":"2024-03-06T07:28:15.801792+0000","last_active":"2024-03-06T07:28:16.801758+0000","last_peered":"2024-03-06T07:28:16.801758+0000","last_clean":"2024-03-06T07:28:16.801758+0000","last_became_active":"2024-03-06T07:28:15.801469+0000","last_became_peered":"2024-03-06T07:28:15.801469+0000","last_unstale":"2024-03-06T07:28:16.801758+0000","last_undegraded":"2024-03-06T07:28:16.801758+0000","last_fullsized":"2024-03-06T07:28:16.801758+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:26:51.432968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398477+0000","last_change":"2024-03-06T07:28:14.784209+0000","last_active":"2024-03-06T07:28:17.398477+0000","last_peered":"2024-03-06T07:28:17.398477+0000","last_clean":"2024-03-06T07:28:17.398477+0000","last_became_active":"2024-03-06T07:28:14.783932+0000","last_became_peered":"2024-03-06T07:28:14.783932+0000","last_unstale":"2024-03-06T07:28:17.398477+0000","last_undegraded":"2024-03-06T07:28:17.398477+0000","last_fullsized":"2024-03-06T07:28:17.398477+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:44:53.639117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399034+0000","last_change":"2024-03-06T07:28:15.794406+0000","last_active":"2024-03-06T07:28:17.399034+0000","last_peered":"2024-03-06T07:28:17.399034+0000","last_clean":"2024-03-06T07:28:17.399034+0000","last_became_active":"2024-03-06T07:28:15.794097+0000","last_became_peered":"2024-03-06T07:28:15.794097+0000","last_unstale":"2024-03-06T07:28:17.399034+0000","last_undegraded":"2024-03-06T07:28:17.399034+0000","last_fullsized":"2024-03-06T07:28:17.399034+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:38.450162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397969+0000","last_change":"2024-03-06T07:28:14.787534+0000","last_active":"2024-03-06T07:28:17.397969+0000","last_peered":"2024-03-06T07:28:17.397969+0000","last_clean":"2024-03-06T07:28:17.397969+0000","last_became_active":"2024-03-06T07:28:14.787286+0000","last_became_peered":"2024-03-06T07:28:14.787286+0000","last_unstale":"2024-03-06T07:28:17.397969+0000","last_undegraded":"2024-03-06T07:28:17.397969+0000","last_fullsized":"2024-03-06T07:28:17.397969+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:11:18.510928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398812+0000","last_change":"2024-03-06T07:28:15.806082+0000","last_active":"2024-03-06T07:28:17.398812+0000","last_peered":"2024-03-06T07:28:17.398812+0000","last_clean":"2024-03-06T07:28:17.398812+0000","last_became_active":"2024-03-06T07:28:15.805099+0000","last_became_peered":"2024-03-06T07:28:15.805099+0000","last_unstale":"2024-03-06T07:28:17.398812+0000","last_undegraded":"2024-03-06T07:28:17.398812+0000","last_fullsized":"2024-03-06T07:28:17.398812+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:38:05.515279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"197'8","reported_seq":22,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798420+0000","last_change":"2024-03-06T07:28:14.788682+0000","last_active":"2024-03-06T07:28:16.798420+0000","last_peered":"2024-03-06T07:28:16.798420+0000","last_clean":"2024-03-06T07:28:16.798420+0000","last_became_active":"2024-03-06T07:28:14.788381+0000","last_became_peered":"2024-03-06T07:28:14.788381+0000","last_unstale":"2024-03-06T07:28:16.798420+0000","last_undegraded":"2024-03-06T07:28:16.798420+0000","last_fullsized":"2024-03-06T07:28:16.798420+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:57.017927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398247+0000","last_change":"2024-03-06T07:28:15.796200+0000","last_active":"2024-03-06T07:28:17.398247+0000","last_peered":"2024-03-06T07:28:17.398247+0000","last_clean":"2024-03-06T07:28:17.398247+0000","last_became_active":"2024-03-06T07:28:15.795919+0000","last_became_peered":"2024-03-06T07:28:15.795919+0000","last_unstale":"2024-03-06T07:28:17.398247+0000","last_undegraded":"2024-03-06T07:28:17.398247+0000","last_fullsized":"2024-03-06T07:28:17.398247+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:59.308461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398196+0000","last_change":"2024-03-06T07:28:14.787636+0000","last_active":"2024-03-06T07:28:17.398196+0000","last_peered":"2024-03-06T07:28:17.398196+0000","last_clean":"2024-03-06T07:28:17.398196+0000","last_became_active":"2024-03-06T07:28:14.787367+0000","last_became_peered":"2024-03-06T07:28:14.787367+0000","last_unstale":"2024-03-06T07:28:17.398196+0000","last_undegraded":"2024-03-06T07:28:17.398196+0000","last_fullsized":"2024-03-06T07:28:17.398196+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:22.374111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397858+0000","last_change":"2024-03-06T07:28:15.797503+0000","last_active":"2024-03-06T07:28:17.397858+0000","last_peered":"2024-03-06T07:28:17.397858+0000","last_clean":"2024-03-06T07:28:17.397858+0000","last_became_active":"2024-03-06T07:28:15.796055+0000","last_became_peered":"2024-03-06T07:28:15.796055+0000","last_unstale":"2024-03-06T07:28:17.397858+0000","last_undegraded":"2024-03-06T07:28:17.397858+0000","last_fullsized":"2024-03-06T07:28:17.397858+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:22.057287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798665+0000","last_change":"2024-03-06T07:28:14.787083+0000","last_active":"2024-03-06T07:28:16.798665+0000","last_peered":"2024-03-06T07:28:16.798665+0000","last_clean":"2024-03-06T07:28:16.798665+0000","last_became_active":"2024-03-06T07:28:14.786827+0000","last_became_peered":"2024-03-06T07:28:14.786827+0000","last_unstale":"2024-03-06T07:28:16.798665+0000","last_undegraded":"2024-03-06T07:28:16.798665+0000","last_fullsized":"2024-03-06T07:28:16.798665+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:40:59.668294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798329+0000","last_change":"2024-03-06T07:28:15.808458+0000","last_active":"2024-03-06T07:28:16.798329+0000","last_peered":"2024-03-06T07:28:16.798329+0000","last_clean":"2024-03-06T07:28:16.798329+0000","last_became_active":"2024-03-06T07:28:15.808179+0000","last_became_peered":"2024-03-06T07:28:15.808179+0000","last_unstale":"2024-03-06T07:28:16.798329+0000","last_undegraded":"2024-03-06T07:28:16.798329+0000","last_fullsized":"2024-03-06T07:28:16.798329+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:01.029533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398242+0000","last_change":"2024-03-06T07:28:14.788050+0000","last_active":"2024-03-06T07:28:17.398242+0000","last_peered":"2024-03-06T07:28:17.398242+0000","last_clean":"2024-03-06T07:28:17.398242+0000","last_became_active":"2024-03-06T07:28:14.787770+0000","last_became_peered":"2024-03-06T07:28:14.787770+0000","last_unstale":"2024-03-06T07:28:17.398242+0000","last_undegraded":"2024-03-06T07:28:17.398242+0000","last_fullsized":"2024-03-06T07:28:17.398242+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:22.597847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398350+0000","last_change":"2024-03-06T07:28:15.801317+0000","last_active":"2024-03-06T07:28:17.398350+0000","last_peered":"2024-03-06T07:28:17.398350+0000","last_clean":"2024-03-06T07:28:17.398350+0000","last_became_active":"2024-03-06T07:28:15.801080+0000","last_became_peered":"2024-03-06T07:28:15.801080+0000","last_unstale":"2024-03-06T07:28:17.398350+0000","last_undegraded":"2024-03-06T07:28:17.398350+0000","last_fullsized":"2024-03-06T07:28:17.398350+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:15.839156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398350+0000","last_change":"2024-03-06T07:28:14.782491+0000","last_active":"2024-03-06T07:28:17.398350+0000","last_peered":"2024-03-06T07:28:17.398350+0000","last_clean":"2024-03-06T07:28:17.398350+0000","last_became_active":"2024-03-06T07:28:14.782223+0000","last_became_peered":"2024-03-06T07:28:14.782223+0000","last_unstale":"2024-03-06T07:28:17.398350+0000","last_undegraded":"2024-03-06T07:28:17.398350+0000","last_fullsized":"2024-03-06T07:28:17.398350+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:58.645677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398679+0000","last_change":"2024-03-06T07:28:15.796329+0000","last_active":"2024-03-06T07:28:17.398679+0000","last_peered":"2024-03-06T07:28:17.398679+0000","last_clean":"2024-03-06T07:28:17.398679+0000","last_became_active":"2024-03-06T07:28:15.795993+0000","last_became_peered":"2024-03-06T07:28:15.795993+0000","last_unstale":"2024-03-06T07:28:17.398679+0000","last_undegraded":"2024-03-06T07:28:17.398679+0000","last_fullsized":"2024-03-06T07:28:17.398679+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:38.450162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398439+0000","last_change":"2024-03-06T07:28:14.785927+0000","last_active":"2024-03-06T07:28:17.398439+0000","last_peered":"2024-03-06T07:28:17.398439+0000","last_clean":"2024-03-06T07:28:17.398439+0000","last_became_active":"2024-03-06T07:28:14.785666+0000","last_became_peered":"2024-03-06T07:28:14.785666+0000","last_unstale":"2024-03-06T07:28:17.398439+0000","last_undegraded":"2024-03-06T07:28:17.398439+0000","last_fullsized":"2024-03-06T07:28:17.398439+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:55.179141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798507+0000","last_change":"2024-03-06T07:28:15.796518+0000","last_active":"2024-03-06T07:28:16.798507+0000","last_peered":"2024-03-06T07:28:16.798507+0000","last_clean":"2024-03-06T07:28:16.798507+0000","last_became_active":"2024-03-06T07:28:15.796242+0000","last_became_peered":"2024-03-06T07:28:15.796242+0000","last_unstale":"2024-03-06T07:28:16.798507+0000","last_undegraded":"2024-03-06T07:28:16.798507+0000","last_fullsized":"2024-03-06T07:28:16.798507+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:26.109657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398635+0000","last_change":"2024-03-06T07:28:14.788414+0000","last_active":"2024-03-06T07:28:17.398635+0000","last_peered":"2024-03-06T07:28:17.398635+0000","last_clean":"2024-03-06T07:28:17.398635+0000","last_became_active":"2024-03-06T07:28:14.788047+0000","last_became_peered":"2024-03-06T07:28:14.788047+0000","last_unstale":"2024-03-06T07:28:17.398635+0000","last_undegraded":"2024-03-06T07:28:17.398635+0000","last_fullsized":"2024-03-06T07:28:17.398635+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:27:40.945464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398671+0000","last_change":"2024-03-06T07:28:15.796320+0000","last_active":"2024-03-06T07:28:17.398671+0000","last_peered":"2024-03-06T07:28:17.398671+0000","last_clean":"2024-03-06T07:28:17.398671+0000","last_became_active":"2024-03-06T07:28:15.796013+0000","last_became_peered":"2024-03-06T07:28:15.796013+0000","last_unstale":"2024-03-06T07:28:17.398671+0000","last_undegraded":"2024-03-06T07:28:17.398671+0000","last_fullsized":"2024-03-06T07:28:17.398671+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:48.471486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398120+0000","last_change":"2024-03-06T07:28:14.782233+0000","last_active":"2024-03-06T07:28:17.398120+0000","last_peered":"2024-03-06T07:28:17.398120+0000","last_clean":"2024-03-06T07:28:17.398120+0000","last_became_active":"2024-03-06T07:28:14.781927+0000","last_became_peered":"2024-03-06T07:28:14.781927+0000","last_unstale":"2024-03-06T07:28:17.398120+0000","last_undegraded":"2024-03-06T07:28:17.398120+0000","last_fullsized":"2024-03-06T07:28:17.398120+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:11.894389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399173+0000","last_change":"2024-03-06T07:28:15.785648+0000","last_active":"2024-03-06T07:28:17.399173+0000","last_peered":"2024-03-06T07:28:17.399173+0000","last_clean":"2024-03-06T07:28:17.399173+0000","last_became_active":"2024-03-06T07:28:15.785364+0000","last_became_peered":"2024-03-06T07:28:15.785364+0000","last_unstale":"2024-03-06T07:28:17.399173+0000","last_undegraded":"2024-03-06T07:28:17.399173+0000","last_fullsized":"2024-03-06T07:28:17.399173+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:29.576193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397463+0000","last_change":"2024-03-06T07:28:14.785960+0000","last_active":"2024-03-06T07:28:17.397463+0000","last_peered":"2024-03-06T07:28:17.397463+0000","last_clean":"2024-03-06T07:28:17.397463+0000","last_became_active":"2024-03-06T07:28:14.785692+0000","last_became_peered":"2024-03-06T07:28:14.785692+0000","last_unstale":"2024-03-06T07:28:17.397463+0000","last_undegraded":"2024-03-06T07:28:17.397463+0000","last_fullsized":"2024-03-06T07:28:17.397463+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:57.935579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.801768+0000","last_change":"2024-03-06T07:28:15.793292+0000","last_active":"2024-03-06T07:28:16.801768+0000","last_peered":"2024-03-06T07:28:16.801768+0000","last_clean":"2024-03-06T07:28:16.801768+0000","last_became_active":"2024-03-06T07:28:15.793015+0000","last_became_peered":"2024-03-06T07:28:15.793015+0000","last_unstale":"2024-03-06T07:28:16.801768+0000","last_undegraded":"2024-03-06T07:28:16.801768+0000","last_fullsized":"2024-03-06T07:28:16.801768+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:17.075519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"197'2","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399124+0000","last_change":"2024-03-06T07:28:14.784580+0000","last_active":"2024-03-06T07:28:17.399124+0000","last_peered":"2024-03-06T07:28:17.399124+0000","last_clean":"2024-03-06T07:28:17.399124+0000","last_became_active":"2024-03-06T07:28:14.784245+0000","last_became_peered":"2024-03-06T07:28:14.784245+0000","last_unstale":"2024-03-06T07:28:17.399124+0000","last_undegraded":"2024-03-06T07:28:17.399124+0000","last_fullsized":"2024-03-06T07:28:17.399124+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:16.071332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398501+0000","last_change":"2024-03-06T07:28:15.798632+0000","last_active":"2024-03-06T07:28:17.398501+0000","last_peered":"2024-03-06T07:28:17.398501+0000","last_clean":"2024-03-06T07:28:17.398501+0000","last_became_active":"2024-03-06T07:28:15.798283+0000","last_became_peered":"2024-03-06T07:28:15.798283+0000","last_unstale":"2024-03-06T07:28:17.398501+0000","last_undegraded":"2024-03-06T07:28:17.398501+0000","last_fullsized":"2024-03-06T07:28:17.398501+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:55.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"198'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399018+0000","last_change":"2024-03-06T07:28:14.786720+0000","last_active":"2024-03-06T07:28:17.399018+0000","last_peered":"2024-03-06T07:28:17.399018+0000","last_clean":"2024-03-06T07:28:17.399018+0000","last_became_active":"2024-03-06T07:28:14.786452+0000","last_became_peered":"2024-03-06T07:28:14.786452+0000","last_unstale":"2024-03-06T07:28:17.399018+0000","last_undegraded":"2024-03-06T07:28:17.399018+0000","last_fullsized":"2024-03-06T07:28:17.399018+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:39.404777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397982+0000","last_change":"2024-03-06T07:28:15.799073+0000","last_active":"2024-03-06T07:28:17.397982+0000","last_peered":"2024-03-06T07:28:17.397982+0000","last_clean":"2024-03-06T07:28:17.397982+0000","last_became_active":"2024-03-06T07:28:15.798797+0000","last_became_peered":"2024-03-06T07:28:15.798797+0000","last_unstale":"2024-03-06T07:28:17.397982+0000","last_undegraded":"2024-03-06T07:28:17.397982+0000","last_fullsized":"2024-03-06T07:28:17.397982+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:47.699638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397725+0000","last_change":"2024-03-06T07:28:14.789522+0000","last_active":"2024-03-06T07:28:17.397725+0000","last_peered":"2024-03-06T07:28:17.397725+0000","last_clean":"2024-03-06T07:28:17.397725+0000","last_became_active":"2024-03-06T07:28:14.789267+0000","last_became_peered":"2024-03-06T07:28:14.789267+0000","last_unstale":"2024-03-06T07:28:17.397725+0000","last_undegraded":"2024-03-06T07:28:17.397725+0000","last_fullsized":"2024-03-06T07:28:17.397725+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:22.017770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399110+0000","last_change":"2024-03-06T07:28:15.805864+0000","last_active":"2024-03-06T07:28:17.399110+0000","last_peered":"2024-03-06T07:28:17.399110+0000","last_clean":"2024-03-06T07:28:17.399110+0000","last_became_active":"2024-03-06T07:28:15.805047+0000","last_became_peered":"2024-03-06T07:28:15.805047+0000","last_unstale":"2024-03-06T07:28:17.399110+0000","last_undegraded":"2024-03-06T07:28:17.399110+0000","last_fullsized":"2024-03-06T07:28:17.399110+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:43:43.923712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398163+0000","last_change":"2024-03-06T07:28:14.788385+0000","last_active":"2024-03-06T07:28:17.398163+0000","last_peered":"2024-03-06T07:28:17.398163+0000","last_clean":"2024-03-06T07:28:17.398163+0000","last_became_active":"2024-03-06T07:28:14.788113+0000","last_became_peered":"2024-03-06T07:28:14.788113+0000","last_unstale":"2024-03-06T07:28:17.398163+0000","last_undegraded":"2024-03-06T07:28:17.398163+0000","last_fullsized":"2024-03-06T07:28:17.398163+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:30:46.695450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398008+0000","last_change":"2024-03-06T07:28:15.802764+0000","last_active":"2024-03-06T07:28:17.398008+0000","last_peered":"2024-03-06T07:28:17.398008+0000","last_clean":"2024-03-06T07:28:17.398008+0000","last_became_active":"2024-03-06T07:28:15.802494+0000","last_became_peered":"2024-03-06T07:28:15.802494+0000","last_unstale":"2024-03-06T07:28:17.398008+0000","last_undegraded":"2024-03-06T07:28:17.398008+0000","last_fullsized":"2024-03-06T07:28:17.398008+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:59.147501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399401+0000","last_change":"2024-03-06T07:28:14.792256+0000","last_active":"2024-03-06T07:28:17.399401+0000","last_peered":"2024-03-06T07:28:17.399401+0000","last_clean":"2024-03-06T07:28:17.399401+0000","last_became_active":"2024-03-06T07:28:14.792032+0000","last_became_peered":"2024-03-06T07:28:14.792032+0000","last_unstale":"2024-03-06T07:28:17.399401+0000","last_undegraded":"2024-03-06T07:28:17.399401+0000","last_fullsized":"2024-03-06T07:28:17.399401+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:33.663609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399186+0000","last_change":"2024-03-06T07:28:15.803745+0000","last_active":"2024-03-06T07:28:17.399186+0000","last_peered":"2024-03-06T07:28:17.399186+0000","last_clean":"2024-03-06T07:28:17.399186+0000","last_became_active":"2024-03-06T07:28:15.803431+0000","last_became_peered":"2024-03-06T07:28:15.803431+0000","last_unstale":"2024-03-06T07:28:17.399186+0000","last_undegraded":"2024-03-06T07:28:17.399186+0000","last_fullsized":"2024-03-06T07:28:17.399186+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:15:17.380724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398583+0000","last_change":"2024-03-06T07:28:14.788477+0000","last_active":"2024-03-06T07:28:17.398583+0000","last_peered":"2024-03-06T07:28:17.398583+0000","last_clean":"2024-03-06T07:28:17.398583+0000","last_became_active":"2024-03-06T07:28:14.788223+0000","last_became_peered":"2024-03-06T07:28:14.788223+0000","last_unstale":"2024-03-06T07:28:17.398583+0000","last_undegraded":"2024-03-06T07:28:17.398583+0000","last_fullsized":"2024-03-06T07:28:17.398583+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:01:47.467299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.798559+0000","last_change":"2024-03-06T07:28:15.800969+0000","last_active":"2024-03-06T07:28:16.798559+0000","last_peered":"2024-03-06T07:28:16.798559+0000","last_clean":"2024-03-06T07:28:16.798559+0000","last_became_active":"2024-03-06T07:28:15.800595+0000","last_became_peered":"2024-03-06T07:28:15.800595+0000","last_unstale":"2024-03-06T07:28:16.798559+0000","last_undegraded":"2024-03-06T07:28:16.798559+0000","last_fullsized":"2024-03-06T07:28:16.798559+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:29.853870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397821+0000","last_change":"2024-03-06T07:28:14.788356+0000","last_active":"2024-03-06T07:28:17.397821+0000","last_peered":"2024-03-06T07:28:17.397821+0000","last_clean":"2024-03-06T07:28:17.397821+0000","last_became_active":"2024-03-06T07:28:14.788093+0000","last_became_peered":"2024-03-06T07:28:14.788093+0000","last_unstale":"2024-03-06T07:28:17.397821+0000","last_undegraded":"2024-03-06T07:28:17.397821+0000","last_fullsized":"2024-03-06T07:28:17.397821+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.428223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.399029+0000","last_change":"2024-03-06T07:28:15.795594+0000","last_active":"2024-03-06T07:28:17.399029+0000","last_peered":"2024-03-06T07:28:17.399029+0000","last_clean":"2024-03-06T07:28:17.399029+0000","last_became_active":"2024-03-06T07:28:15.795236+0000","last_became_peered":"2024-03-06T07:28:15.795236+0000","last_unstale":"2024-03-06T07:28:17.399029+0000","last_undegraded":"2024-03-06T07:28:17.399029+0000","last_fullsized":"2024-03-06T07:28:17.399029+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:40.908082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397691+0000","last_change":"2024-03-06T07:28:14.786359+0000","last_active":"2024-03-06T07:28:17.397691+0000","last_peered":"2024-03-06T07:28:17.397691+0000","last_clean":"2024-03-06T07:28:17.397691+0000","last_became_active":"2024-03-06T07:28:14.786068+0000","last_became_peered":"2024-03-06T07:28:14.786068+0000","last_unstale":"2024-03-06T07:28:17.397691+0000","last_undegraded":"2024-03-06T07:28:17.397691+0000","last_fullsized":"2024-03-06T07:28:17.397691+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:19.967406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.398031+0000","last_change":"2024-03-06T07:28:15.806625+0000","last_active":"2024-03-06T07:28:17.398031+0000","last_peered":"2024-03-06T07:28:17.398031+0000","last_clean":"2024-03-06T07:28:17.398031+0000","last_became_active":"2024-03-06T07:28:15.806283+0000","last_became_peered":"2024-03-06T07:28:15.806283+0000","last_unstale":"2024-03-06T07:28:17.398031+0000","last_undegraded":"2024-03-06T07:28:17.398031+0000","last_fullsized":"2024-03-06T07:28:17.398031+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.764755+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.764755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:22:05.645953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"197'1","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.802094+0000","last_change":"2024-03-06T07:28:14.785657+0000","last_active":"2024-03-06T07:28:16.802094+0000","last_peered":"2024-03-06T07:28:16.802094+0000","last_clean":"2024-03-06T07:28:16.802094+0000","last_became_active":"2024-03-06T07:28:14.785292+0000","last_became_peered":"2024-03-06T07:28:14.785292+0000","last_unstale":"2024-03-06T07:28:16.802094+0000","last_undegraded":"2024-03-06T07:28:16.802094+0000","last_fullsized":"2024-03-06T07:28:16.802094+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.760568+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.760568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:26.845642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":502,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-03-06T07:28:17.397826+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:28:17.397826+0000","last_peered":"2024-03-06T07:28:17.397826+0000","last_clean":"2024-03-06T07:28:17.397826+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:28:17.397826+0000","last_undegraded":"2024-03-06T07:28:17.397826+0000","last_fullsized":"2024-03-06T07:28:17.397826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2584576,"data_stored":1811644,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":2519040,"data_stored":1221264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2510848,"data_stored":1221152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":2514944,"data_stored":1221222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2535424,"data_stored":1223402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":2519040,"data_stored":1221804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37792,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334048,"statfs":{"total":96636764160,"available":96598065152,"internally_reserved":0,"allocated":2510848,"data_stored":1221152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37872,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333968,"statfs":{"total":96636764160,"available":96597983232,"internally_reserved":0,"allocated":2588672,"data_stored":1813132,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:28:24.746 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:28:24.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:28:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.874+0000 7fbe3ece1640 1 Processor -- start 2024-03-06T07:28:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.874+0000 7fbe3ece1640 1 -- start start 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3ece1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe38153420 0x7fbe38155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3ece1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe38153850 con 0x7fbe3806dca0 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3ece1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380c2d80 con 0x7fbe3814f580 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3ece1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe380c2b30 con 0x7fbe38153420 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3d27e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe38153420 0x7fbe38155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe37fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbe3806dca0 0x7fbe38155520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57754/0 (socket says 172.21.15.26:57754) 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3d27e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe38153420 0x7fbe38155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43998/0 (socket says 172.21.15.26:43998) 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe37fff640 1 -- 172.21.15.26:0/2241405516 learned_addr learned my addr 172.21.15.26:0/2241405516 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3d27e640 1 -- 172.21.15.26:0/2241405516 >> v1:172.21.15.117:6789/0 conn(0x7fbe3814f580 legacy=0x7fbe3806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3d27e640 1 -- 172.21.15.26:0/2241405516 >> v1:172.21.15.26:6789/0 conn(0x7fbe3806dca0 legacy=0x7fbe38155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.876+0000 7fbe3d27e640 1 -- 172.21.15.26:0/2241405516 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380c3a00 con 0x7fbe38153420 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.877+0000 7fbe3d27e640 1 --2- 172.21.15.26:0/2241405516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38153420 0x7fbe38155c30 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097a0 tx=0x7fbe3002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ede68e5fbf9cedd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.877+0000 7fbe377fe640 1 -- 172.21.15.26:0/2241405516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe300315a0 con 0x7fbe38153420 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.877+0000 7fbe377fe640 1 -- 172.21.15.26:0/2241405516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30037070 con 0x7fbe38153420 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.877+0000 7fbe377fe640 1 -- 172.21.15.26:0/2241405516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30031960 con 0x7fbe38153420 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.878+0000 7fbe3ece1640 1 -- 172.21.15.26:0/2241405516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38153420 msgr2=0x7fbe38155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.878+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/2241405516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38153420 0x7fbe38155c30 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097a0 tx=0x7fbe3002b840 comp rx=0 tx=0).stop 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.878+0000 7fbe3ece1640 1 -- 172.21.15.26:0/2241405516 shutdown_connections 2024-03-06T07:28:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.878+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/2241405516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38153420 0x7fbe38155c30 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.878+0000 7fbe3ece1640 1 -- 172.21.15.26:0/2241405516 >> 172.21.15.26:0/2241405516 conn(0x7fbe380a94d0 msgr2=0x7fbe380a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.878+0000 7fbe3ece1640 1 -- 172.21.15.26:0/2241405516 shutdown_connections 2024-03-06T07:28:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.878+0000 7fbe3ece1640 1 -- 172.21.15.26:0/2241405516 wait complete. 2024-03-06T07:28:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.880+0000 7fbe3ece1640 1 Processor -- start 2024-03-06T07:28:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.880+0000 7fbe3ece1640 1 -- start start 2024-03-06T07:28:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe3ece1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe3806dca0 0x7fbe38149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe3ece1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3814f580 0x7fbe381eae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe3ece1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe38153420 0x7fbe381eb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe3ece1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3814f960 con 0x7fbe3814f580 2024-03-06T07:28:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe3ece1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380c3d70 con 0x7fbe3806dca0 2024-03-06T07:28:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe3ece1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380c3060 con 0x7fbe38153420 2024-03-06T07:28:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3814f580 0x7fbe381eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3814f580 0x7fbe381eae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44002/0 (socket says 172.21.15.26:44002) 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe37fff640 1 -- 172.21.15.26:0/1761127156 learned_addr learned my addr 172.21.15.26:0/1761127156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.881+0000 7fbe3ca7d640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe3806dca0 0x7fbe38149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe3d27e640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe38153420 0x7fbe381eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe37fff640 1 -- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe38153420 msgr2=0x7fbe381eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe37fff640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe38153420 0x7fbe381eb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe37fff640 1 -- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe3806dca0 msgr2=0x7fbe38149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe37fff640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe3806dca0 0x7fbe38149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe37fff640 1 -- 172.21.15.26:0/1761127156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3814a4b0 con 0x7fbe3814f580 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe3d27e640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe38153420 0x7fbe381eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe37fff640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3814f580 0x7fbe381eae50 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800cba0 tx=0x7fbe2800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe357fa640 1 -- 172.21.15.26:0/1761127156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28017070 con 0x7fbe3814f580 2024-03-06T07:28:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe357fa640 1 -- 172.21.15.26:0/1761127156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2800ad40 con 0x7fbe3814f580 2024-03-06T07:28:24.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe357fa640 1 -- 172.21.15.26:0/1761127156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28012600 con 0x7fbe3814f580 2024-03-06T07:28:24.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe381eb990 con 0x7fbe3814f580 2024-03-06T07:28:24.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.882+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe38150090 con 0x7fbe3814f580 2024-03-06T07:28:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.883+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe380735b0 con 0x7fbe3814f580 2024-03-06T07:28:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.884+0000 7fbe357fa640 1 -- 172.21.15.26:0/1761127156 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe2801a070 con 0x7fbe3814f580 2024-03-06T07:28:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.884+0000 7fbe357fa640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe1806b500 0x7fbe1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.884+0000 7fbe357fa640 1 -- 172.21.15.26:0/1761127156 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe28089f80 con 0x7fbe3814f580 2024-03-06T07:28:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.884+0000 7fbe3ca7d640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe1806b500 0x7fbe1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.885+0000 7fbe3ca7d640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe1806b500 0x7fbe1806d9c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f60 tx=0x7fbe20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:24.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:24.888+0000 7fbe357fa640 1 -- 172.21.15.26:0/1761127156 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe380735b0 con 0x7fbe3814f580 2024-03-06T07:28:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.159+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbe3804f330 con 0x7fbe3814f580 2024-03-06T07:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.161+0000 7fbe357fa640 1 -- 172.21.15.26:0/1761127156 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbe280586b0 con 0x7fbe3814f580 2024-03-06T07:28:25.163 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:28:25.163 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:28:25.163 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:28:25.163 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:28:25.163 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:28:25.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.163+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe1806b500 msgr2=0x7fbe1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.163+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe1806b500 0x7fbe1806d9c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f60 tx=0x7fbe20009040 comp rx=0 tx=0).stop 2024-03-06T07:28:25.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.163+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3814f580 msgr2=0x7fbe381eae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.163+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3814f580 0x7fbe381eae50 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800cba0 tx=0x7fbe2800a430 comp rx=0 tx=0).stop 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.163+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 shutdown_connections 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.163+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbe1806b500 0x7fbe1806d9c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.163+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe38153420 0x7fbe381eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.164+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3814f580 0x7fbe381eae50 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.164+0000 7fbe3ece1640 1 --2- 172.21.15.26:0/1761127156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe3806dca0 0x7fbe38149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.164+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 >> 172.21.15.26:0/1761127156 conn(0x7fbe380a94d0 msgr2=0x7fbe38072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.164+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 shutdown_connections 2024-03-06T07:28:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.164+0000 7fbe3ece1640 1 -- 172.21.15.26:0/1761127156 wait complete. 2024-03-06T07:28:25.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:25.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.275+0000 7f2305b93640 1 Processor -- start 2024-03-06T07:28:25.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.275+0000 7f2305b93640 1 -- start start 2024-03-06T07:28:25.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.276+0000 7f2305b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 0x7f2300155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.276+0000 7f2305b93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23000c73d0 con 0x7f230014f360 2024-03-06T07:28:25.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.276+0000 7f2305b93640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23000be2e0 con 0x7f23000c88c0 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.276+0000 7f2305b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300159700 con 0x7f2300159270 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.276+0000 7f22ff7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f230014f360 0x7f23000c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57764/0 (socket says 172.21.15.26:57764) 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.276+0000 7f22ff7fe640 1 -- 172.21.15.26:0/3944243933 learned_addr learned my addr 172.21.15.26:0/3944243933 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22fffff640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 0x7f2300155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249317583 0 0) 0x7f23000c73d0 con 0x7f230014f360 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22ec003520 con 0x7f230014f360 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482719221 0 0) 0x7f23000be2e0 con 0x7f23000c88c0 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23000c73d0 con 0x7f23000c88c0 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 323191417 0 0) 0x7f22ec003520 con 0x7f230014f360 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 msgr2=0x7f2300155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 0x7f2300155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 >> v1:172.21.15.117:6789/0 conn(0x7f23000c88c0 legacy=0x7f2300158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23000c7670 con 0x7f230014f360 2024-03-06T07:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f22f0001670 con 0x7f230014f360 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22fffff640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 0x7f2300155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 >> v1:172.21.15.26:6789/0 conn(0x7f230014f360 legacy=0x7f23000c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f22ec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 0x7f22ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec004d40 0x7f22ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300159700 con 0x7f22ec003b70 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22ff7fe640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f22ec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec00bad0 con 0x7f22ec004d40 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec00bcd0 con 0x7f2300159270 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22f7fff640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 0x7f22ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22fffff640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec004d40 0x7f22ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22ff7fe640 1 -- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 msgr2=0x7f22ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22ff7fe640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2300159270 0x7f22ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22ff7fe640 1 -- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec004d40 msgr2=0x7f22ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22ff7fe640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec004d40 0x7f22ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.277+0000 7f22ff7fe640 1 -- 172.21.15.26:0/3944243933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec003520 con 0x7f22ec003b70 2024-03-06T07:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.279+0000 7f22ff7fe640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f22ec003f70 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c1c0 tx=0x7f22ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49ea7e045c7e364 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.279+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0007cc0 con 0x7f22ec003b70 2024-03-06T07:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.279+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f0030070 con 0x7f22ec003b70 2024-03-06T07:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.279+0000 7f22feffd640 1 -- 172.21.15.26:0/3944243933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0005d10 con 0x7f22ec003b70 2024-03-06T07:28:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.279+0000 7f2305b93640 1 -- 172.21.15.26:0/3944243933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 msgr2=0x7f22ec003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.279+0000 7f2305b93640 1 --2- 172.21.15.26:0/3944243933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f22ec003f70 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c1c0 tx=0x7f22ec005830 comp rx=0 tx=0).stop 2024-03-06T07:28:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.279+0000 7f22fffff640 1 -- 172.21.15.26:0/3944243933 reap_dead start 2024-03-06T07:28:25.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.280+0000 7f2305b93640 1 -- 172.21.15.26:0/3944243933 shutdown_connections 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.280+0000 7f2305b93640 1 -- 172.21.15.26:0/3944243933 >> 172.21.15.26:0/3944243933 conn(0x7f23000ac570 msgr2=0x7f23000ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.280+0000 7f2305b93640 1 -- 172.21.15.26:0/3944243933 shutdown_connections 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.280+0000 7f2305b93640 1 -- 172.21.15.26:0/3944243933 wait complete. 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 Processor -- start 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 -- start start 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f230014b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23001edf50 0x7f23001ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23000be5d0 0x7f23000bea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000c7c30 con 0x7f22ec003b70 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000c5a50 con 0x7f23001edf50 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f2305b93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23000c5bd0 con 0x7f23000be5d0 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f230014b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f230014b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44020/0 (socket says 172.21.15.26:44020) 2024-03-06T07:28:25.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.281+0000 7f22ff7fe640 1 -- 172.21.15.26:0/2718322087 learned_addr learned my addr 172.21.15.26:0/2718322087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22fffff640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23000be5d0 0x7f23000bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22ff7fe640 1 -- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23000be5d0 msgr2=0x7f23000bea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22f7fff640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23001edf50 0x7f23001ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22ff7fe640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23000be5d0 0x7f23000bea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22ff7fe640 1 -- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23001edf50 msgr2=0x7f23001ee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22ff7fe640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23001edf50 0x7f23001ee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22ff7fe640 1 -- 172.21.15.26:0/2718322087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230014c090 con 0x7f22ec003b70 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22ff7fe640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f230014b850 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f22f0000c00 tx=0x7f22f00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22fcff9640 1 -- 172.21.15.26:0/2718322087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0045070 con 0x7f22ec003b70 2024-03-06T07:28:25.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22fcff9640 1 -- 172.21.15.26:0/2718322087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f00049e0 con 0x7f22ec003b70 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23000af400 con 0x7f22ec003b70 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f22fcff9640 1 -- 172.21.15.26:0/2718322087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0004dd0 con 0x7f22ec003b70 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.282+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23000b0190 con 0x7f22ec003b70 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.283+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23000af1f0 con 0x7f22ec003b70 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.284+0000 7f22fcff9640 1 -- 172.21.15.26:0/2718322087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22f0007070 con 0x7f22ec003b70 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.284+0000 7f22fcff9640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22e006b500 0x7f22e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.284+0000 7f22fcff9640 1 -- 172.21.15.26:0/2718322087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22f00ab040 con 0x7f22ec003b70 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.284+0000 7f22f7fff640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22e006b500 0x7f22e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:25.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.285+0000 7f22f7fff640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22e006b500 0x7f22e006d9c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f23000c9720 tx=0x7f22e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:25.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.288+0000 7f22fcff9640 1 -- 172.21.15.26:0/2718322087 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f22f0079ea0 con 0x7f22ec003b70 2024-03-06T07:28:25.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.509+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f230004f330 con 0x7f22ec003b70 2024-03-06T07:28:25.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.510+0000 7f22fcff9640 1 -- 172.21.15.26:0/2718322087 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 188 v188) v1 ==== 95+0+4998 (secure 0 0 0) 0x7f22f00797a0 con 0x7f22ec003b70 2024-03-06T07:28:25.511 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:25.511 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":188,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2181907552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2181907552},{"type":"v1","addr":"172.21.15.117:6836","nonce":2181907552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":8129,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2243060002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2243060002},{"type":"v1","addr":"172.21.15.117:6837","nonce":2243060002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":9529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4068022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4068022212},{"type":"v1","addr":"172.21.15.26:6834","nonce":4068022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":188,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:15.142228+0000","modified":"2024-03-06T07:28:18.221296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10338":{"gid":10338,"name":"a","rank":0,"incarnation":185,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1578728485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1578728485},{"type":"v1","addr":"172.21.15.26:6835","nonce":1578728485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10338,"qdb_cluster":[10338]},"id":15}]} 2024-03-06T07:28:25.512 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 188 2024-03-06T07:28:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22e006b500 msgr2=0x7f22e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22e006b500 0x7f22e006d9c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f23000c9720 tx=0x7f22e8009040 comp rx=0 tx=0).stop 2024-03-06T07:28:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 msgr2=0x7f230014b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f230014b850 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f22f0000c00 tx=0x7f22f00019c0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 shutdown_connections 2024-03-06T07:28:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f22e006b500 0x7f22e006d9c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23000be5d0 0x7f23000bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23001edf50 0x7f23001ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 --2- 172.21.15.26:0/2718322087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ec003b70 0x7f230014b850 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 >> 172.21.15.26:0/2718322087 conn(0x7f23000ac570 msgr2=0x7f230014d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 shutdown_connections 2024-03-06T07:28:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:25.512+0000 7f2305b93640 1 -- 172.21.15.26:0/2718322087 wait complete. 2024-03-06T07:28:25.927 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 188, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:28:15.142228+0000', 'modified': '2024-03-06T07:28:18.221296+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10338}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10338': {'gid': 10338, 'name': 'a', 'rank': 0, 'incarnation': 185, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1578728485', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1578728485}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1578728485}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10338, 'qdb_cluster': [10338]} 2024-03-06T07:28:25.927 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:28:25.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:28:26.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.031+0000 7f0bec871640 1 Processor -- start 2024-03-06T07:28:26.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.031+0000 7f0bec871640 1 -- start start 2024-03-06T07:28:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.032+0000 7f0bec871640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be80c8950 0x7f0be80c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.032+0000 7f0bec871640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be8155130 con 0x7f0be80c69f0 2024-03-06T07:28:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.032+0000 7f0bec871640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80ad480 con 0x7f0be81558d0 2024-03-06T07:28:26.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.032+0000 7f0bec871640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be8155440 con 0x7f0be80c8950 2024-03-06T07:28:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.032+0000 7f0be5d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be80c8950 0x7f0be80c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.032+0000 7f0be659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0be80c69f0 0x7f0be80acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57780/0 (socket says 172.21.15.26:57780) 2024-03-06T07:28:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.032+0000 7f0be659c640 1 -- 172.21.15.26:0/2686725742 learned_addr learned my addr 172.21.15.26:0/2686725742 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.033+0000 7f0be5d9b640 1 -- 172.21.15.26:0/2686725742 >> v1:172.21.15.117:6789/0 conn(0x7f0be81558d0 legacy=0x7f0be815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.033+0000 7f0be5d9b640 1 -- 172.21.15.26:0/2686725742 >> v1:172.21.15.26:6789/0 conn(0x7f0be80c69f0 legacy=0x7f0be80acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.033+0000 7f0be5d9b640 1 -- 172.21.15.26:0/2686725742 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80c5d50 con 0x7f0be80c8950 2024-03-06T07:28:26.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.033+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/2686725742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c6dd0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00097a0 tx=0x7f0bd002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79a9b089008e311 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0be559a640 1 -- 172.21.15.26:0/2686725742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00315a0 con 0x7f0be80c8950 2024-03-06T07:28:26.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0be559a640 1 -- 172.21.15.26:0/2686725742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0037070 con 0x7f0be80c8950 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0be559a640 1 -- 172.21.15.26:0/2686725742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0031960 con 0x7f0be80c8950 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0bec871640 1 -- 172.21.15.26:0/2686725742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 msgr2=0x7f0be80c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0bec871640 1 --2- 172.21.15.26:0/2686725742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c6dd0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00097a0 tx=0x7f0bd002b840 comp rx=0 tx=0).stop 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0bec871640 1 -- 172.21.15.26:0/2686725742 shutdown_connections 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0bec871640 1 --2- 172.21.15.26:0/2686725742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c6dd0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0bec871640 1 -- 172.21.15.26:0/2686725742 >> 172.21.15.26:0/2686725742 conn(0x7f0be80cae70 msgr2=0x7f0be80cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0bec871640 1 -- 172.21.15.26:0/2686725742 shutdown_connections 2024-03-06T07:28:26.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.034+0000 7f0bec871640 1 -- 172.21.15.26:0/2686725742 wait complete. 2024-03-06T07:28:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.035+0000 7f0bec871640 1 Processor -- start 2024-03-06T07:28:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0bec871640 1 -- start start 2024-03-06T07:28:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0bec871640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be80c69f0 0x7f0be80adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0bec871640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0bec871640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be81558d0 0x7f0be80c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0bec871640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8152a20 con 0x7f0be80c8950 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0bec871640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80c6310 con 0x7f0be81558d0 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0bec871640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8152ba0 con 0x7f0be80c69f0 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0be5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0be5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44032/0 (socket says 172.21.15.26:44032) 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0be5d9b640 1 -- 172.21.15.26:0/3959776382 learned_addr learned my addr 172.21.15.26:0/3959776382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be81558d0 0x7f0be80c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.036+0000 7f0be659c640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be80c69f0 0x7f0be80adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0be5d9b640 1 -- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be80c69f0 msgr2=0x7f0be80adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be80c69f0 0x7f0be80adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0be5d9b640 1 -- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be81558d0 msgr2=0x7f0be80c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be81558d0 0x7f0be80c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0be5d9b640 1 -- 172.21.15.26:0/3959776382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80c3d90 con 0x7f0be80c8950 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be81558d0 0x7f0be80c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c3310 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00098c0 tx=0x7f0bd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0bdeffd640 1 -- 172.21.15.26:0/3959776382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0032840 con 0x7f0be80c8950 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0bdeffd640 1 -- 172.21.15.26:0/3959776382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0032a00 con 0x7f0be80c8950 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0bdeffd640 1 -- 172.21.15.26:0/3959776382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0032df0 con 0x7f0be80c8950 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be80c48e0 con 0x7f0be80c8950 2024-03-06T07:28:26.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.037+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be80c2df0 con 0x7f0be80c8950 2024-03-06T07:28:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.039+0000 7f0bdeffd640 1 -- 172.21.15.26:0/3959776382 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bd003b3f0 con 0x7f0be80c8950 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.039+0000 7f0bdeffd640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bc806b4d0 0x7f0bc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.039+0000 7f0bdeffd640 1 -- 172.21.15.26:0/3959776382 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bd002f030 con 0x7f0be80c8950 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.039+0000 7f0be659c640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bc806b4d0 0x7f0bc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.040+0000 7f0bef457740 1 Processor -- start 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.040+0000 7f0be659c640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bc806b4d0 0x7f0bc806d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0be80c10c0 tx=0x7f0bd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.040+0000 7f0bef457740 1 -- start start 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.040+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550310f50 0x55b55030ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.040+0000 7f0bef457740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550311330 0x55b55030ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.040+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b55030f4b0 0x55b55030b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.040+0000 7f0bef457740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5502756b0 con 0x55b550311330 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55021ea60 con 0x55b55030f4b0 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b550271750 con 0x55b550310f50 2024-03-06T07:28:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550311330 0x55b55030ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550311330 0x55b55030ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44034/0 (socket says 172.21.15.26:44034) 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 -- 172.21.15.26:0/2374059142 learned_addr learned my addr 172.21.15.26:0/2374059142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be659c640 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550310f50 0x55b55030ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b55030f4b0 0x55b55030b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 -- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550310f50 msgr2=0x55b55030ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550310f50 0x55b55030ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 -- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b55030f4b0 msgr2=0x55b55030b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b55030f4b0 0x55b55030b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 -- 172.21.15.26:0/2374059142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b55023cf30 con 0x55b550311330 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550311330 0x55b55030ef70 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0076d70 tx=0x7f0bd007a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b54125fabc9723 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0bc7fff640 1 -- 172.21.15.26:0/2374059142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0083020 con 0x55b550311330 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0bc7fff640 1 -- 172.21.15.26:0/2374059142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0037050 con 0x55b550311330 2024-03-06T07:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.041+0000 7f0bc7fff640 1 -- 172.21.15.26:0/2374059142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0078c10 con 0x55b550311330 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 -- 172.21.15.26:0/2374059142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550311330 msgr2=0x55b55030ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550311330 0x55b55030ef70 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0076d70 tx=0x7f0bd007a370 comp rx=0 tx=0).stop 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 -- 172.21.15.26:0/2374059142 shutdown_connections 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b55030f4b0 0x55b55030b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550311330 0x55b55030ef70 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 --2- 172.21.15.26:0/2374059142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550310f50 0x55b55030ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 -- 172.21.15.26:0/2374059142 >> 172.21.15.26:0/2374059142 conn(0x55b55030d7d0 msgr2=0x55b5503323c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 -- 172.21.15.26:0/2374059142 shutdown_connections 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.042+0000 7f0bef457740 1 -- 172.21.15.26:0/2374059142 wait complete. 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.043+0000 7f0bef457740 1 -- start start 2024-03-06T07:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.043+0000 7f0bef457740 1 Processor -- start 2024-03-06T07:28:26.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bef457740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b5503317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b550333840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550337b30 0x55b55032de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bef457740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5502756b0 con 0x55b550310f50 2024-03-06T07:28:26.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b550231ea0 con 0x55b550337b30 2024-03-06T07:28:26.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b550333230 con 0x55b550311330 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bef457740 10 client.0 _ll_register_callbacks cb 0x55b550231870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b5503317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b5503317b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44050/0 (socket says 172.21.15.26:44050) 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 -- 172.21.15.26:0/1790379186 learned_addr learned my addr 172.21.15.26:0/1790379186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550337b30 0x55b55032de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b550333840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 -- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 msgr2=0x55b550333840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b550333840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 -- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550337b30 msgr2=0x55b55032de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550337b30 0x55b55032de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 -- 172.21.15.26:0/1790379186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b550232270 con 0x55b550310f50 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0be659c640 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b5503317b0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8007680 tx=0x7f0bd8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9360992b084faa25 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bc57fa640 1 -- 172.21.15.26:0/1790379186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8027060 con 0x55b550310f50 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.044+0000 7f0bc57fa640 1 -- 172.21.15.26:0/1790379186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bd8010040 con 0x55b550310f50 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bd0037050 con 0x55b550310f50 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bc57fa640 1 -- 172.21.15.26:0/1790379186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bd802d050 con 0x55b550310f50 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 10 client.10524 fetch_fsmap learned FSMap version 188 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=2+}) v3 -- 0x55b54ff8a2d0 con 0x55b550310f50 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bc57fa640 1 -- 172.21.15.26:0/1790379186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0037050 con 0x55b550310f50 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bc57fa640 1 -- 172.21.15.26:0/1790379186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 188) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0bd8030020 con 0x55b550310f50 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 10 client.10524 fetch_fsmap finished waiting for FSMap version 188 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 10 client.10524 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 20 client.10524 populate_metadata read hostname 'smithi026' 2024-03-06T07:28:26.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b5503618f0 0x55b55032e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 4 client.10524 mds_command: new command op to 10338 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 --> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b550282bf0 con 0x55b5503618f0 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b5503618f0 0x55b55032e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0bc57fa640 10 client.10524 ms_handle_connect on v2:172.21.15.26:6833/1578728485 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.045+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b5503618f0 0x55b55032e510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9725554fcc3e4e server_cookie=bfd6192f82c9a9d8 in_seq=0 out_seq=0 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bc57fa640 1 -- 172.21.15.26:0/1790379186 <== mds.0 v2:172.21.15.26:6833/1578728485 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b550282bf0 con 0x55b5503618f0 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bc57fa640 10 client.10524 handle_command_reply: tid=0 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 client.10524 shutdown 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 msgr2=0x55b5503317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b5503317b0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8007680 tx=0x7f0bd8001250 comp rx=0 tx=0).stop 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 shutdown_connections 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b5503618f0 0x55b55032e510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550337b30 0x55b55032de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b550333840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 --2- 172.21.15.26:0/1790379186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b5503317b0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 >> 172.21.15.26:0/1790379186 conn(0x55b55030f4b0 msgr2=0x55b55030a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bc57fa640 0 client.10524 ms_handle_reset on v2:172.21.15.26:6833/1578728485 2024-03-06T07:28:26.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 shutdown_connections 2024-03-06T07:28:26.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.046+0000 7f0bef457740 1 -- 172.21.15.26:0/1790379186 wait complete. 2024-03-06T07:28:26.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.047+0000 7f0bef457740 20 client.10524 trim_cache size 0 max 16384 2024-03-06T07:28:26.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.063+0000 7f0bef457740 1 Processor -- start 2024-03-06T07:28:26.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.063+0000 7f0bef457740 1 -- start start 2024-03-06T07:28:26.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550310f50 0x55b55030e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b55030b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0bef457740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550337b30 0x55b55030ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0bef457740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be81f7050 con 0x55b550337b30 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be814fa30 con 0x55b550310f50 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be81f6a80 con 0x55b550311330 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550337b30 0x55b55030ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550337b30 0x55b55030ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44060/0 (socket says 172.21.15.26:44060) 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 -- 172.21.15.26:0/3005562430 learned_addr learned my addr 172.21.15.26:0/3005562430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be659c640 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550310f50 0x55b55030e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b55030b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 -- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 msgr2=0x55b55030b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b55030b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 -- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550310f50 msgr2=0x55b55030e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550310f50 0x55b55030e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.064+0000 7f0be6d9d640 1 -- 172.21.15.26:0/3005562430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5500546b0 con 0x55b550337b30 2024-03-06T07:28:26.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.065+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550337b30 0x55b55030ba60 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f0be0004e60 tx=0x7f0be0005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5ff4dc4c0d64ea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.065+0000 7f0bc77fe640 1 -- 172.21.15.26:0/3005562430 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0014b30 con 0x55b550337b30 2024-03-06T07:28:26.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.065+0000 7f0bc77fe640 1 -- 172.21.15.26:0/3005562430 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0014cd0 con 0x55b550337b30 2024-03-06T07:28:26.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.065+0000 7f0bc77fe640 1 -- 172.21.15.26:0/3005562430 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0007250 con 0x55b550337b30 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 -- 172.21.15.26:0/3005562430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550337b30 msgr2=0x55b55030ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550337b30 0x55b55030ba60 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f0be0004e60 tx=0x7f0be0005320 comp rx=0 tx=0).stop 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 -- 172.21.15.26:0/3005562430 shutdown_connections 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550337b30 0x55b55030ba60 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550311330 0x55b55030b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 --2- 172.21.15.26:0/3005562430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550310f50 0x55b55030e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 -- 172.21.15.26:0/3005562430 >> 172.21.15.26:0/3005562430 conn(0x55b55030f4b0 msgr2=0x55b550336f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 -- 172.21.15.26:0/3005562430 shutdown_connections 2024-03-06T07:28:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 -- 172.21.15.26:0/3005562430 wait complete. 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.066+0000 7f0bef457740 1 -- start start 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 1 Processor -- start 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b550370d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550311330 0x55b55030bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550337b30 0x55b5503ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55032e390 con 0x55b550310f50 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5503322a0 con 0x55b550311330 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b550360d20 con 0x55b550337b30 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0bef457740 10 client.0 _ll_register_callbacks cb 0x7f0bd8007300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0be659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b550370d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.067+0000 7f0be659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b550370d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44074/0 (socket says 172.21.15.26:44074) 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be659c640 1 -- 172.21.15.26:0/542442393 learned_addr learned my addr 172.21.15.26:0/542442393 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be6d9d640 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550337b30 0x55b5503ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550311330 0x55b55030bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be659c640 1 -- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550337b30 msgr2=0x55b5503ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be659c640 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550337b30 0x55b5503ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be659c640 1 -- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550311330 msgr2=0x55b55030bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be659c640 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550311330 0x55b55030bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be659c640 1 -- 172.21.15.26:0/542442393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b550337fa0 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0be659c640 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b550370d20 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005c00 tx=0x7f0bd8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3454b3ec78bc24a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0bc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0bc57fa640 1 -- 172.21.15.26:0/542442393 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8036060 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0bc57fa640 1 -- 172.21.15.26:0/542442393 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bd8022450 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0be0014cd0 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0bc57fa640 1 -- 172.21.15.26:0/542442393 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bd803c050 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0bef457740 10 client.10530 fetch_fsmap learned FSMap version 188 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.068+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=2+}) v3 -- 0x55b5503c1520 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bc57fa640 1 -- 172.21.15.26:0/542442393 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0014cd0 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bc57fa640 1 -- 172.21.15.26:0/542442393 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 188) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0bd803f020 con 0x55b550310f50 2024-03-06T07:28:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bef457740 10 client.10530 fetch_fsmap finished waiting for FSMap version 188 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bef457740 10 client.10530 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bef457740 20 client.10530 populate_metadata read hostname 'smithi026' 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bef457740 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b550359c30 0x55b55035a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bef457740 4 client.10530 mds_command: new command op to 10338 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 --> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b55027cf40 con 0x55b550359c30 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b550359c30 0x55b55035a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bc57fa640 10 client.10530 ms_handle_connect on v2:172.21.15.26:6833/1578728485 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0be5d9b640 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b550359c30 0x55b55035a010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df48816e25346af7 server_cookie=a8a2720bdf53d5b in_seq=0 out_seq=0 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bc57fa640 1 -- 172.21.15.26:0/542442393 <== mds.0 v2:172.21.15.26:6833/1578728485 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55b55027cf40 con 0x55b550359c30 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bc57fa640 10 client.10530 handle_command_reply: tid=0 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.069+0000 7f0bef457740 1 client.10530 shutdown 2024-03-06T07:28:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 msgr2=0x55b550370d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b550370d20 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005c00 tx=0x7f0bd8020000 comp rx=0 tx=0).stop 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 shutdown_connections 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:6833/1578728485,v1:172.21.15.26:6835/1578728485] conn(0x55b550359c30 0x55b55035a010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b550337b30 0x55b5503ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b550311330 0x55b55030bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bc57fa640 0 client.10530 ms_handle_reset on v2:172.21.15.26:6833/1578728485 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 --2- 172.21.15.26:0/542442393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b550310f50 0x55b550370d20 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 >> 172.21.15.26:0/542442393 conn(0x55b55030f4b0 msgr2=0x55b55034ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 shutdown_connections 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 1 -- 172.21.15.26:0/542442393 wait complete. 2024-03-06T07:28:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bef457740 20 client.10530 trim_cache size 0 max 16384 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 10338, 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.2823049, 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 188, 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 198, 2024-03-06T07:28:26.078 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 198, 2024-03-06T07:28:26.079 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.370093546 2024-03-06T07:28:26.079 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:28:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bc806b4d0 msgr2=0x7f0bc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bc806b4d0 0x7f0bc806d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0be80c10c0 tx=0x7f0bd8009040 comp rx=0 tx=0).stop 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 msgr2=0x7f0be80c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c3310 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00098c0 tx=0x7f0bd00046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 shutdown_connections 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0bc806b4d0 0x7f0bc806d990 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be81558d0 0x7f0be80c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be80c8950 0x7f0be80c3310 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 --2- 172.21.15.26:0/3959776382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be80c69f0 0x7f0be80adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.070+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 >> 172.21.15.26:0/3959776382 conn(0x7f0be80cae70 msgr2=0x7f0be81509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.071+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 shutdown_connections 2024-03-06T07:28:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.071+0000 7f0bec871640 1 -- 172.21.15.26:0/3959776382 wait complete. 2024-03-06T07:28:26.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:28:26.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.186+0000 7f947d815640 1 Processor -- start 2024-03-06T07:28:26.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.186+0000 7f947d815640 1 -- start start 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f947d815640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94780b00d0 0x7f94780c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f947d815640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f947806b8e0 0x7f947806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f947d815640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780b04b0 con 0x7f94780c7210 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f947d815640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9478156160 con 0x7f947806b8e0 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f947d815640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780c5710 con 0x7f94780b00d0 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f946ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f947806b8e0 0x7f947806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f94777fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f94780c7210 0x7f94780c7680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57790/0 (socket says 172.21.15.26:57790) 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f94777fe640 1 -- 172.21.15.26:0/4193668863 learned_addr learned my addr 172.21.15.26:0/4193668863 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f9476ffd640 1 --2- 172.21.15.26:0/4193668863 >> v2:172.21.15.117:3300/0 conn(0x7f94780b00d0 0x7f94780c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f946ffff640 1 -- 172.21.15.26:0/4193668863 >> v2:172.21.15.117:3300/0 conn(0x7f94780b00d0 msgr2=0x7f94780c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f946ffff640 1 --2- 172.21.15.26:0/4193668863 >> v2:172.21.15.117:3300/0 conn(0x7f94780b00d0 0x7f94780c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f946ffff640 1 -- 172.21.15.26:0/4193668863 >> v1:172.21.15.26:6789/0 conn(0x7f94780c7210 legacy=0x7f94780c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f94767fc640 1 -- 172.21.15.26:0/4193668863 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340152916 0 0) 0x7f94780b04b0 con 0x7f94780c7210 2024-03-06T07:28:26.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f946ffff640 1 -- 172.21.15.26:0/4193668863 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947806d210 con 0x7f947806b8e0 2024-03-06T07:28:26.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.187+0000 7f9476ffd640 1 --2- 172.21.15.26:0/4193668863 >> v2:172.21.15.117:3300/0 conn(0x7f94780b00d0 0x7f94780c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:26.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.188+0000 7f946ffff640 1 --2- 172.21.15.26:0/4193668863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947806b8e0 0x7f947806bd20 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f94780ad3f0 tx=0x7f946002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce2e43eb462f8f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.188+0000 7f94767fc640 1 -- 172.21.15.26:0/4193668863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460035020 con 0x7f947806b8e0 2024-03-06T07:28:26.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.188+0000 7f94767fc640 1 -- 172.21.15.26:0/4193668863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9460031830 con 0x7f947806b8e0 2024-03-06T07:28:26.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.188+0000 7f94767fc640 1 -- 172.21.15.26:0/4193668863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460031c00 con 0x7f947806b8e0 2024-03-06T07:28:26.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 -- 172.21.15.26:0/4193668863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947806b8e0 msgr2=0x7f947806bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 --2- 172.21.15.26:0/4193668863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947806b8e0 0x7f947806bd20 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f94780ad3f0 tx=0x7f946002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 -- 172.21.15.26:0/4193668863 shutdown_connections 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 --2- 172.21.15.26:0/4193668863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947806b8e0 0x7f947806bd20 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 --2- 172.21.15.26:0/4193668863 >> v2:172.21.15.117:3300/0 conn(0x7f94780b00d0 0x7f94780c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 -- 172.21.15.26:0/4193668863 >> 172.21.15.26:0/4193668863 conn(0x7f94780ac810 msgr2=0x7f94780acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 -- 172.21.15.26:0/4193668863 shutdown_connections 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.189+0000 7f947d815640 1 -- 172.21.15.26:0/4193668863 wait complete. 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 Processor -- start 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 -- start start 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947806b8e0 0x7f94780c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b00d0 0x7f9478158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94780c7210 0x7f94780c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947806c260 con 0x7f94780c7210 2024-03-06T07:28:26.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9478152c50 con 0x7f947806b8e0 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f947d815640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94780c5ba0 con 0x7f94780b00d0 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f94777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94780c7210 0x7f94780c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.190+0000 7f94777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94780c7210 0x7f94780c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44102/0 (socket says 172.21.15.26:44102) 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f94777fe640 1 -- 172.21.15.26:0/251591473 learned_addr learned my addr 172.21.15.26:0/251591473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f9476ffd640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947806b8e0 0x7f94780c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f946ffff640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b00d0 0x7f9478158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f94777fe640 1 -- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b00d0 msgr2=0x7f9478158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f94777fe640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b00d0 0x7f9478158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f94777fe640 1 -- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947806b8e0 msgr2=0x7f94780c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f94777fe640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947806b8e0 0x7f94780c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f94777fe640 1 -- 172.21.15.26:0/251591473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780beb70 con 0x7f94780c7210 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f9476ffd640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947806b8e0 0x7f94780c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f946ffff640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b00d0 0x7f9478158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f94777fe640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94780c7210 0x7f94780c0da0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f947000c7d0 tx=0x7f947000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f9474ff9640 1 -- 172.21.15.26:0/251591473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470018070 con 0x7f94780c7210 2024-03-06T07:28:26.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f9474ff9640 1 -- 172.21.15.26:0/251591473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947000a640 con 0x7f94780c7210 2024-03-06T07:28:26.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f9474ff9640 1 -- 172.21.15.26:0/251591473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000aa70 con 0x7f94780c7210 2024-03-06T07:28:26.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94780bf2f0 con 0x7f94780c7210 2024-03-06T07:28:26.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.191+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9478158f40 con 0x7f94780c7210 2024-03-06T07:28:26.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.192+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947804f330 con 0x7f94780c7210 2024-03-06T07:28:26.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.193+0000 7f9474ff9640 1 -- 172.21.15.26:0/251591473 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f947001b070 con 0x7f94780c7210 2024-03-06T07:28:26.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.193+0000 7f9474ff9640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f945806b500 0x7f945806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.193+0000 7f9474ff9640 1 -- 172.21.15.26:0/251591473 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94700898f0 con 0x7f94780c7210 2024-03-06T07:28:26.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.193+0000 7f9476ffd640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f945806b500 0x7f945806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.194+0000 7f9476ffd640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f945806b500 0x7f945806d9c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f946800aa30 tx=0x7f9468008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.197+0000 7f9474ff9640 1 -- 172.21.15.26:0/251591473 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9470058670 con 0x7f94780c7210 2024-03-06T07:28:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.418+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7f94781528f0 con 0x7f94780c7210 2024-03-06T07:28:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.831+0000 7f9474ff9640 1 -- 172.21.15.26:0/251591473 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v189) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9470057f70 con 0x7f94780c7210 2024-03-06T07:28:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked down. 2024-03-06T07:28:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f945806b500 msgr2=0x7f945806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f945806b500 0x7f945806d9c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f946800aa30 tx=0x7f9468008040 comp rx=0 tx=0).stop 2024-03-06T07:28:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94780c7210 msgr2=0x7f94780c0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94780c7210 0x7f94780c0da0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f947000c7d0 tx=0x7f947000cc90 comp rx=0 tx=0).stop 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 shutdown_connections 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f945806b500 0x7f945806d9c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94780c7210 0x7f94780c0da0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b00d0 0x7f9478158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 --2- 172.21.15.26:0/251591473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947806b8e0 0x7f94780c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 >> 172.21.15.26:0/251591473 conn(0x7f94780ac810 msgr2=0x7f94780acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 shutdown_connections 2024-03-06T07:28:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.833+0000 7f947d815640 1 -- 172.21.15.26:0/251591473 wait complete. 2024-03-06T07:28:26.846 INFO:tasks.ceph_test_case:waiting 30s for health warning matching 2024-03-06T07:28:26.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.945+0000 7f9d8742a640 1 Processor -- start 2024-03-06T07:28:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.945+0000 7f9d8742a640 1 -- start start 2024-03-06T07:28:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.946+0000 7f9d8742a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d8014f360 0x7f9d80155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.946+0000 7f9d8742a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800c73d0 con 0x7f9d800c88c0 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.946+0000 7f9d8742a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800be2e0 con 0x7f9d80158b60 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.946+0000 7f9d8742a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80159700 con 0x7f9d8014f360 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.946+0000 7f9d849c5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d800c88c0 0x7f9d800c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57806/0 (socket says 172.21.15.26:57806) 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.946+0000 7f9d849c5640 1 -- 172.21.15.26:0/3869232101 learned_addr learned my addr 172.21.15.26:0/3869232101 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.946+0000 7f9d851c6640 1 --2- 172.21.15.26:0/3869232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d8014f360 0x7f9d80155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023859742 0 0) 0x7f9d800c73d0 con 0x7f9d800c88c0 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d6c003520 con 0x7f9d800c88c0 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554236904 0 0) 0x7f9d800be2e0 con 0x7f9d80158b60 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d800c73d0 con 0x7f9d80158b60 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1305270914 0 0) 0x7f9d6c003520 con 0x7f9d800c88c0 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d851c6640 1 -- 172.21.15.26:0/3869232101 >> v1:172.21.15.117:6789/0 conn(0x7f9d80158b60 legacy=0x7f9d80158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d851c6640 1 -- 172.21.15.26:0/3869232101 >> v1:172.21.15.26:6789/0 conn(0x7f9d800c88c0 legacy=0x7f9d800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.947+0000 7f9d851c6640 1 -- 172.21.15.26:0/3869232101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800c7670 con 0x7f9d8014f360 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d851c6640 1 --2- 172.21.15.26:0/3869232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d8014f360 0x7f9d80155c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d780097d0 tx=0x7f9d7802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e41a6889734085b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78037070 con 0x7f9d8014f360 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78030510 con 0x7f9d8014f360 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d77fff640 1 -- 172.21.15.26:0/3869232101 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70001670 con 0x7f9d8014f360 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d8742a640 1 -- 172.21.15.26:0/3869232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d8014f360 msgr2=0x7f9d80155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d8742a640 1 --2- 172.21.15.26:0/3869232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d8014f360 0x7f9d80155c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d780097d0 tx=0x7f9d7802b770 comp rx=0 tx=0).stop 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d8742a640 1 -- 172.21.15.26:0/3869232101 shutdown_connections 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d8742a640 1 --2- 172.21.15.26:0/3869232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d8014f360 0x7f9d80155c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d8742a640 1 -- 172.21.15.26:0/3869232101 >> 172.21.15.26:0/3869232101 conn(0x7f9d800ac570 msgr2=0x7f9d800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d8742a640 1 -- 172.21.15.26:0/3869232101 shutdown_connections 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.948+0000 7f9d8742a640 1 -- 172.21.15.26:0/3869232101 wait complete. 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.949+0000 7f9d8742a640 1 Processor -- start 2024-03-06T07:28:26.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.949+0000 7f9d8742a640 1 -- start start 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d8742a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d800c88c0 0x7f9d801df400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d8742a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8014f360 0x7f9d800aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d8742a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d80158b60 0x7f9d800aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d8742a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c7c30 con 0x7f9d8014f360 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d8742a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c5a50 con 0x7f9d800c88c0 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d8742a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c5bd0 con 0x7f9d80158b60 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d849c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8014f360 0x7f9d800aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d849c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8014f360 0x7f9d800aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44110/0 (socket says 172.21.15.26:44110) 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d849c5640 1 -- 172.21.15.26:0/550199234 learned_addr learned my addr 172.21.15.26:0/550199234 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d851c6640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d800c88c0 0x7f9d801df400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.950+0000 7f9d859c7640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d80158b60 0x7f9d800aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d849c5640 1 -- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d80158b60 msgr2=0x7f9d800aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d849c5640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d80158b60 0x7f9d800aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d849c5640 1 -- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d800c88c0 msgr2=0x7f9d801df400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d849c5640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d800c88c0 0x7f9d801df400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d849c5640 1 -- 172.21.15.26:0/550199234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d801e12a0 con 0x7f9d8014f360 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d859c7640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d80158b60 0x7f9d800aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d849c5640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8014f360 0x7f9d800aa3e0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f9d70000fd0 tx=0x7f9d7000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d75ffb640 1 -- 172.21.15.26:0/550199234 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70017070 con 0x7f9d8014f360 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d75ffb640 1 -- 172.21.15.26:0/550199234 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d70005ad0 con 0x7f9d8014f360 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d801e1580 con 0x7f9d8014f360 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d75ffb640 1 -- 172.21.15.26:0/550199234 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70007070 con 0x7f9d8014f360 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.951+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d801dead0 con 0x7f9d8014f360 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.952+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8004f330 con 0x7f9d8014f360 2024-03-06T07:28:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.952+0000 7f9d75ffb640 1 -- 172.21.15.26:0/550199234 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d7001a070 con 0x7f9d8014f360 2024-03-06T07:28:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.953+0000 7f9d75ffb640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d6406b500 0x7f9d6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.953+0000 7f9d75ffb640 1 -- 172.21.15.26:0/550199234 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d70089790 con 0x7f9d8014f360 2024-03-06T07:28:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.953+0000 7f9d851c6640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d6406b500 0x7f9d6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.953+0000 7f9d851c6640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d6406b500 0x7f9d6406d9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9d78034000 tx=0x7f9d78008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:26.957+0000 7f9d75ffb640 1 -- 172.21.15.26:0/550199234 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d70057db0 con 0x7f9d8014f360 2024-03-06T07:28:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.177+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9d800c8d50 con 0x7f9d8014f360 2024-03-06T07:28:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.177+0000 7f9d75ffb640 1 -- 172.21.15.26:0/550199234 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9d7005be50 con 0x7f9d8014f360 2024-03-06T07:28:27.179 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:27.179 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d6406b500 msgr2=0x7f9d6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d6406b500 0x7f9d6406d9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9d78034000 tx=0x7f9d78008030 comp rx=0 tx=0).stop 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8014f360 msgr2=0x7f9d800aa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8014f360 0x7f9d800aa3e0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f9d70000fd0 tx=0x7f9d7000cbe0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 shutdown_connections 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d6406b500 0x7f9d6406d9c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d80158b60 0x7f9d800aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d8014f360 0x7f9d800aa3e0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 --2- 172.21.15.26:0/550199234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d800c88c0 0x7f9d801df400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 >> 172.21.15.26:0/550199234 conn(0x7f9d800ac570 msgr2=0x7f9d800ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 shutdown_connections 2024-03-06T07:28:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:27.179+0000 7f9d8742a640 1 -- 172.21.15.26:0/550199234 wait complete. 2024-03-06T07:28:27.191 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:27.191 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:32.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:32.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.305+0000 7f941ae9a640 1 Processor -- start 2024-03-06T07:28:32.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.305+0000 7f941ae9a640 1 -- start start 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f941ae9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9414152920 0x7f9414155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f941ae9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94140c8a00 0x7f9414158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f941ae9a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9414156180 con 0x7f9414159460 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f941ae9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94141587b0 con 0x7f9414152920 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f941ae9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94140c7760 con 0x7f94140c8a00 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9418c36640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9414152920 0x7f9414155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9418c36640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9414152920 0x7f9414155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58608/0 (socket says 172.21.15.26:58608) 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9418c36640 1 -- 172.21.15.26:0/3534435956 learned_addr learned my addr 172.21.15.26:0/3534435956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:32.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9413fff640 1 --2- 172.21.15.26:0/3534435956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94140c8a00 0x7f9414158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9418c36640 1 -- 172.21.15.26:0/3534435956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94140c8a00 msgr2=0x7f9414158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:32.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9418c36640 1 --2- 172.21.15.26:0/3534435956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94140c8a00 0x7f9414158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9418c36640 1 -- 172.21.15.26:0/3534435956 >> v1:172.21.15.117:6789/0 conn(0x7f9414159460 legacy=0x7f94140c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:32.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.306+0000 7f9418c36640 1 -- 172.21.15.26:0/3534435956 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94140c7a40 con 0x7f9414152920 2024-03-06T07:28:32.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.307+0000 7f9418c36640 1 --2- 172.21.15.26:0/3534435956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f9414155c40 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f9404009870 tx=0x7f940402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97e563f44aa3851 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:32.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.307+0000 7f94137fe640 1 -- 172.21.15.26:0/3534435956 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94040317c0 con 0x7f9414152920 2024-03-06T07:28:32.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.307+0000 7f94137fe640 1 -- 172.21.15.26:0/3534435956 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404037070 con 0x7f9414152920 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.307+0000 7f94137fe640 1 -- 172.21.15.26:0/3534435956 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404031b80 con 0x7f9414152920 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.307+0000 7f941ae9a640 1 -- 172.21.15.26:0/3534435956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 msgr2=0x7f9414155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.307+0000 7f941ae9a640 1 --2- 172.21.15.26:0/3534435956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f9414155c40 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f9404009870 tx=0x7f940402b860 comp rx=0 tx=0).stop 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.308+0000 7f941ae9a640 1 -- 172.21.15.26:0/3534435956 shutdown_connections 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.308+0000 7f941ae9a640 1 --2- 172.21.15.26:0/3534435956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94140c8a00 0x7f9414158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.308+0000 7f941ae9a640 1 --2- 172.21.15.26:0/3534435956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f9414155c40 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.308+0000 7f941ae9a640 1 -- 172.21.15.26:0/3534435956 >> 172.21.15.26:0/3534435956 conn(0x7f94140ac560 msgr2=0x7f94140ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.308+0000 7f941ae9a640 1 -- 172.21.15.26:0/3534435956 shutdown_connections 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.308+0000 7f941ae9a640 1 -- 172.21.15.26:0/3534435956 wait complete. 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.309+0000 7f941ae9a640 1 Processor -- start 2024-03-06T07:28:32.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.309+0000 7f941ae9a640 1 -- start start 2024-03-06T07:28:32.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94140c8a00 0x7f94141494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f94141eb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94141536a0 0x7f9414153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941406d030 con 0x7f9414152920 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9414158a00 con 0x7f94140c8a00 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94140c5a40 con 0x7f94141536a0 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f94141eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f94141eb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58618/0 (socket says 172.21.15.26:58618) 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 -- 172.21.15.26:0/2194412668 learned_addr learned my addr 172.21.15.26:0/2194412668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9418c36640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94140c8a00 0x7f94141494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9419437640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94141536a0 0x7f9414153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 -- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94141536a0 msgr2=0x7f9414153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94141536a0 0x7f9414153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 -- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94140c8a00 msgr2=0x7f94141494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94140c8a00 0x7f94141494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 -- 172.21.15.26:0/2194412668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94141eb860 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f9413fff640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f94141eb320 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00c7f0 tx=0x7f93fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f94117fa640 1 -- 172.21.15.26:0/2194412668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc015070 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f94117fa640 1 -- 172.21.15.26:0/2194412668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93fc004240 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9414155690 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f94117fa640 1 -- 172.21.15.26:0/2194412668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc007620 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.310+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94140c4620 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.312+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94140c7390 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.312+0000 7f94117fa640 1 -- 172.21.15.26:0/2194412668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93fc0079d0 con 0x7f9414152920 2024-03-06T07:28:32.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.312+0000 7f94117fa640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93f406b4d0 0x7f93f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.313+0000 7f94117fa640 1 -- 172.21.15.26:0/2194412668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93fc08a3e0 con 0x7f9414152920 2024-03-06T07:28:32.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.313+0000 7f9418c36640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93f406b4d0 0x7f93f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.313+0000 7f9418c36640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93f406b4d0 0x7f93f406d990 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9404036000 tx=0x7f940403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:32.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.317+0000 7f94117fa640 1 -- 172.21.15.26:0/2194412668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f94140c7390 con 0x7f9414152920 2024-03-06T07:28:32.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.541+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f941404f330 con 0x7f9414152920 2024-03-06T07:28:32.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.541+0000 7f94117fa640 1 -- 172.21.15.26:0/2194412668 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f93fc059160 con 0x7f9414152920 2024-03-06T07:28:32.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:32.543 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93f406b4d0 msgr2=0x7f93f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93f406b4d0 0x7f93f406d990 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9404036000 tx=0x7f940403d040 comp rx=0 tx=0).stop 2024-03-06T07:28:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 msgr2=0x7f94141eb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f94141eb320 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00c7f0 tx=0x7f93fc00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 shutdown_connections 2024-03-06T07:28:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f93f406b4d0 0x7f93f406d990 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94141536a0 0x7f9414153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9414152920 0x7f94141eb320 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 --2- 172.21.15.26:0/2194412668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94140c8a00 0x7f94141494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 >> 172.21.15.26:0/2194412668 conn(0x7f94140ac560 msgr2=0x7f94140ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 shutdown_connections 2024-03-06T07:28:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:32.543+0000 7f941ae9a640 1 -- 172.21.15.26:0/2194412668 wait complete. 2024-03-06T07:28:32.556 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:32.556 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:37.557 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:37.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.667+0000 7f5754e2a640 1 Processor -- start 2024-03-06T07:28:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.667+0000 7f5754e2a640 1 -- start start 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f5754e2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 0x7f575006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f5754e2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57500c1ae0 con 0x7f5750152520 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f5754e2a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575006e450 con 0x7f5750155c80 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f5754e2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57500c1dc0 con 0x7f575006ef30 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f574e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5750152520 0x7f57500c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45640/0 (socket says 172.21.15.26:45640) 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f574e59c640 1 -- 172.21.15.26:0/1001455610 learned_addr learned my addr 172.21.15.26:0/1001455610 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093844972 0 0) 0x7f57500c1ae0 con 0x7f5750152520 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f574ed9d640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 0x7f575006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.668+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f573c003520 con 0x7f5750152520 2024-03-06T07:28:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3817153897 0 0) 0x7f573c003520 con 0x7f5750152520 2024-03-06T07:28:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 msgr2=0x7f575006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 0x7f575006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 >> v1:172.21.15.117:6789/0 conn(0x7f5750155c80 legacy=0x7f57500720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575006e730 con 0x7f5750152520 2024-03-06T07:28:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f57400019f0 con 0x7f5750152520 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 >> v1:172.21.15.26:6789/0 conn(0x7f5750152520 legacy=0x7f57500c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 0x7f573c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c004d40 0x7f573c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57500c1dc0 con 0x7f573c0039e0 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c00ba70 con 0x7f573c004d40 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c00bc70 con 0x7f575006ef30 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574e59c640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574dd9b640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 0x7f573c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574ed9d640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c004d40 0x7f573c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574e59c640 1 -- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 msgr2=0x7f573c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574e59c640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 0x7f573c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574e59c640 1 -- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c004d40 msgr2=0x7f573c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574e59c640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c004d40 0x7f573c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574e59c640 1 -- 172.21.15.26:0/1001455610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c003520 con 0x7f573c0039e0 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.669+0000 7f574dd9b640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f575006ef30 0x7f573c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.670+0000 7f574e59c640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f574000a810 tx=0x7f573c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832880f723b5fe7a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.670+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000a460 con 0x7f573c0039e0 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.670+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5740034650 con 0x7f573c0039e0 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.670+0000 7f574d59a640 1 -- 172.21.15.26:0/1001455610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740034a40 con 0x7f573c0039e0 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.670+0000 7f5754e2a640 1 -- 172.21.15.26:0/1001455610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573c0039e0 msgr2=0x7f573c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.670+0000 7f5754e2a640 1 --2- 172.21.15.26:0/1001455610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f574000a810 tx=0x7f573c005830 comp rx=0 tx=0).stop 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.670+0000 7f574ed9d640 1 -- 172.21.15.26:0/1001455610 reap_dead start 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.671+0000 7f5754e2a640 1 -- 172.21.15.26:0/1001455610 shutdown_connections 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.671+0000 7f5754e2a640 1 -- 172.21.15.26:0/1001455610 >> 172.21.15.26:0/1001455610 conn(0x7f57500c76b0 msgr2=0x7f57500c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.671+0000 7f5754e2a640 1 -- 172.21.15.26:0/1001455610 shutdown_connections 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.671+0000 7f5754e2a640 1 -- 172.21.15.26:0/1001455610 wait complete. 2024-03-06T07:28:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.672+0000 7f5754e2a640 1 Processor -- start 2024-03-06T07:28:37.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.672+0000 7f5754e2a640 1 -- start start 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5754e2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c0039e0 0x7f575014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5754e2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575014b540 0x7f57501ee530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5754e2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5750147600 0x7f5750157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5754e2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5750152900 con 0x7f575014b540 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5754e2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57500727b0 con 0x7f573c0039e0 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5754e2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5750152a80 con 0x7f5750147600 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575014b540 0x7f57501ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575014b540 0x7f57501ee530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45686/0 (socket says 172.21.15.26:45686) 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 -- 172.21.15.26:0/3417199627 learned_addr learned my addr 172.21.15.26:0/3417199627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574ed9d640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5750147600 0x7f5750157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574e59c640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c0039e0 0x7f575014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 -- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5750147600 msgr2=0x7f5750157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5750147600 0x7f5750157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 -- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c0039e0 msgr2=0x7f575014f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c0039e0 0x7f575014f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 -- 172.21.15.26:0/3417199627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5750157420 con 0x7f575014b540 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f574dd9b640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575014b540 0x7f57501ee530 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f573800e600 tx=0x7f573800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5746ffd640 1 -- 172.21.15.26:0/3417199627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380052e0 con 0x7f575014b540 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5746ffd640 1 -- 172.21.15.26:0/3417199627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738018070 con 0x7f575014b540 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57501eed80 con 0x7f575014b540 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.673+0000 7f5746ffd640 1 -- 172.21.15.26:0/3417199627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738003810 con 0x7f575014b540 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.674+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57501ef340 con 0x7f575014b540 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.674+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57500c17e0 con 0x7f575014b540 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.675+0000 7f5746ffd640 1 -- 172.21.15.26:0/3417199627 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57380039b0 con 0x7f575014b540 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.675+0000 7f5746ffd640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573006b500 0x7f573006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.676+0000 7f5746ffd640 1 -- 172.21.15.26:0/3417199627 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f573808a050 con 0x7f575014b540 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.676+0000 7f574e59c640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573006b500 0x7f573006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.676+0000 7f574e59c640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573006b500 0x7f573006d9c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5740034000 tx=0x7f5740009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.680+0000 7f5746ffd640 1 -- 172.21.15.26:0/3417199627 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57500c17e0 con 0x7f575014b540 2024-03-06T07:28:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.897+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f57500c0b70 con 0x7f575014b540 2024-03-06T07:28:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.898+0000 7f5746ffd640 1 -- 172.21.15.26:0/3417199627 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5738058d40 con 0x7f575014b540 2024-03-06T07:28:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:37.901 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573006b500 msgr2=0x7f573006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573006b500 0x7f573006d9c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5740034000 tx=0x7f5740009030 comp rx=0 tx=0).stop 2024-03-06T07:28:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575014b540 msgr2=0x7f57501ee530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575014b540 0x7f57501ee530 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f573800e600 tx=0x7f573800eac0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 shutdown_connections 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f573006b500 0x7f573006d9c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5750147600 0x7f5750157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575014b540 0x7f57501ee530 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 --2- 172.21.15.26:0/3417199627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c0039e0 0x7f575014f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.899+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 >> 172.21.15.26:0/3417199627 conn(0x7f57500c76b0 msgr2=0x7f575006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.900+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 shutdown_connections 2024-03-06T07:28:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:37.900+0000 7f5754e2a640 1 -- 172.21.15.26:0/3417199627 wait complete. 2024-03-06T07:28:37.913 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:37.913 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:40.862 INFO:tasks.ceph.mds.a.smithi026.stderr: -795> 2024-03-06T07:28:21.066+0000 7fe5ccab5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:28:40.891 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:28:42.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:42.899 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:42.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.045+0000 7f94658fa640 1 Processor -- start 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.045+0000 7f94658fa640 1 -- start start 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f94658fa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94601527b0 0x7f9460159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f94658fa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94600c7180 0x7f94600c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f94658fa640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600c7c60 con 0x7f9460155b20 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f94658fa640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9460159640 con 0x7f94600c7180 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f94658fa640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600c8df0 con 0x7f94601527b0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f945e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94600c7180 0x7f94600c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f945e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94600c7180 0x7f94600c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45706/0 (socket says 172.21.15.26:45706) 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f945e7fc640 1 -- 172.21.15.26:0/1379386060 learned_addr learned my addr 172.21.15.26:0/1379386060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f945e7fc640 1 -- 172.21.15.26:0/1379386060 >> v2:172.21.15.117:3300/0 conn(0x7f94601527b0 msgr2=0x7f9460159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f945e7fc640 1 --2- 172.21.15.26:0/1379386060 >> v2:172.21.15.117:3300/0 conn(0x7f94601527b0 0x7f9460159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f945e7fc640 1 -- 172.21.15.26:0/1379386060 >> v1:172.21.15.117:6789/0 conn(0x7f9460155b20 legacy=0x7f94600c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.046+0000 7f945e7fc640 1 -- 172.21.15.26:0/1379386060 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600c8060 con 0x7f94600c7180 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.047+0000 7f945e7fc640 1 --2- 172.21.15.26:0/1379386060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94600c7180 0x7f94600c88b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f94600cba80 tx=0x7f945402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bcb65b71e1530c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.047+0000 7f945dffb640 1 -- 172.21.15.26:0/1379386060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945402e070 con 0x7f94600c7180 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.047+0000 7f945dffb640 1 -- 172.21.15.26:0/1379386060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454037070 con 0x7f94600c7180 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.047+0000 7f945dffb640 1 -- 172.21.15.26:0/1379386060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454031a00 con 0x7f94600c7180 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 -- 172.21.15.26:0/1379386060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94600c7180 msgr2=0x7f94600c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 --2- 172.21.15.26:0/1379386060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94600c7180 0x7f94600c88b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f94600cba80 tx=0x7f945402b880 comp rx=0 tx=0).stop 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 -- 172.21.15.26:0/1379386060 shutdown_connections 2024-03-06T07:28:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 --2- 172.21.15.26:0/1379386060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94600c7180 0x7f94600c88b0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 --2- 172.21.15.26:0/1379386060 >> v2:172.21.15.117:3300/0 conn(0x7f94601527b0 0x7f9460159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 -- 172.21.15.26:0/1379386060 >> 172.21.15.26:0/1379386060 conn(0x7f94600ac560 msgr2=0x7f94600ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 -- 172.21.15.26:0/1379386060 shutdown_connections 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.048+0000 7f94658fa640 1 -- 172.21.15.26:0/1379386060 wait complete. 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.049+0000 7f94658fa640 1 Processor -- start 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.049+0000 7f94658fa640 1 -- start start 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f94658fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94600c7180 0x7f94601584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f94658fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94601527b0 0x7f94601edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f94658fa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9460155b20 0x7f946014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f94658fa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460155320 con 0x7f94601527b0 2024-03-06T07:28:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f94658fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600c7980 con 0x7f94600c7180 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f94658fa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9460155600 con 0x7f9460155b20 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94601527b0 0x7f94601edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94601527b0 0x7f94601edfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45712/0 (socket says 172.21.15.26:45712) 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 -- 172.21.15.26:0/393814845 learned_addr learned my addr 172.21.15.26:0/393814845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945effd640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94600c7180 0x7f94601584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945f7fe640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9460155b20 0x7f946014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 -- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9460155b20 msgr2=0x7f946014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9460155b20 0x7f946014b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 -- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94600c7180 msgr2=0x7f94601584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94600c7180 0x7f94601584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945e7fc640 1 -- 172.21.15.26:0/393814845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460158a30 con 0x7f94601527b0 2024-03-06T07:28:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.050+0000 7f945effd640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94600c7180 0x7f94601584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.051+0000 7f945e7fc640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94601527b0 0x7f94601edfd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f945402b8c0 tx=0x7f94540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.051+0000 7f9447fff640 1 -- 172.21.15.26:0/393814845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454032880 con 0x7f94601527b0 2024-03-06T07:28:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.051+0000 7f9447fff640 1 -- 172.21.15.26:0/393814845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454032a20 con 0x7f94601527b0 2024-03-06T07:28:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.051+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9460145b70 con 0x7f94601527b0 2024-03-06T07:28:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.051+0000 7f9447fff640 1 -- 172.21.15.26:0/393814845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454032e10 con 0x7f94601527b0 2024-03-06T07:28:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.051+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601e1410 con 0x7f94601527b0 2024-03-06T07:28:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.052+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94601e0f20 con 0x7f94601527b0 2024-03-06T07:28:43.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.052+0000 7f9447fff640 1 -- 172.21.15.26:0/393814845 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f945404c060 con 0x7f94601527b0 2024-03-06T07:28:43.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.052+0000 7f9447fff640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f944c06b500 0x7f944c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.053+0000 7f9447fff640 1 -- 172.21.15.26:0/393814845 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94540abf40 con 0x7f94601527b0 2024-03-06T07:28:43.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.053+0000 7f945effd640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f944c06b500 0x7f944c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.053+0000 7f945effd640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f944c06b500 0x7f944c06d9c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f946014c2d0 tx=0x7f9458008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.057+0000 7f9447fff640 1 -- 172.21.15.26:0/393814845 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9454040070 con 0x7f94601527b0 2024-03-06T07:28:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.276+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f946004f330 con 0x7f94601527b0 2024-03-06T07:28:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.276+0000 7f9447fff640 1 -- 172.21.15.26:0/393814845 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f94600c7c60 con 0x7f94601527b0 2024-03-06T07:28:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:43.279 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:43.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f944c06b500 msgr2=0x7f944c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f944c06b500 0x7f944c06d9c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f946014c2d0 tx=0x7f9458008040 comp rx=0 tx=0).stop 2024-03-06T07:28:43.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94601527b0 msgr2=0x7f94601edfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94601527b0 0x7f94601edfd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f945402b8c0 tx=0x7f94540046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 shutdown_connections 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f944c06b500 0x7f944c06d9c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9460155b20 0x7f946014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94601527b0 0x7f94601edfd0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 --2- 172.21.15.26:0/393814845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94600c7180 0x7f94601584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 >> 172.21.15.26:0/393814845 conn(0x7f94600ac560 msgr2=0x7f94600ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 shutdown_connections 2024-03-06T07:28:43.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:43.278+0000 7f94658fa640 1 -- 172.21.15.26:0/393814845 wait complete. 2024-03-06T07:28:43.292 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:43.292 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:48.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.403+0000 7fe82a3b6640 1 Processor -- start 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.403+0000 7fe82a3b6640 1 -- start start 2024-03-06T07:28:48.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe82a3b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 0x7fe8241559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe82a3b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe824152d40 con 0x7fe8241592d0 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe82a3b6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8240c1b50 con 0x7fe824152960 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe82a3b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe824066e30 con 0x7fe8240ac260 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe8237fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe8241592d0 0x7fe8241552d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50962/0 (socket says 172.21.15.26:50962) 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe8237fe640 1 -- 172.21.15.26:0/3297213781 learned_addr learned my addr 172.21.15.26:0/3297213781 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe828953640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 0x7fe8241559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338553109 0 0) 0x7fe824152d40 con 0x7fe8241592d0 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.404+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe810003520 con 0x7fe8241592d0 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3740499077 0 0) 0x7fe810003520 con 0x7fe8241592d0 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 msgr2=0x7fe8241559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 0x7fe8241559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 >> v1:172.21.15.117:6789/0 conn(0x7fe824152960 legacy=0x7fe82406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8240c1dc0 con 0x7fe8241592d0 2024-03-06T07:28:48.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe80c001670 con 0x7fe8241592d0 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 >> v1:172.21.15.26:6789/0 conn(0x7fe8241592d0 legacy=0x7fe8241552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8100039e0 0x7fe810003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 0x7fe810004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe810004d40 0x7fe8100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe824066e30 con 0x7fe810004d40 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81000b9c0 con 0x7fe8100039e0 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81000bbc0 con 0x7fe8240ac260 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe828953640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe810004d40 0x7fe8100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe8237fe640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 0x7fe810004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe823fff640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8100039e0 0x7fe810003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe828953640 1 -- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 msgr2=0x7fe810004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe828953640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240ac260 0x7fe810004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe828953640 1 -- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8100039e0 msgr2=0x7fe810003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe828953640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8100039e0 0x7fe810003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.405+0000 7fe828953640 1 -- 172.21.15.26:0/3297213781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810003520 con 0x7fe810004d40 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.406+0000 7fe828953640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe810004d40 0x7fe8100051c0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009f90 tx=0x7fe810005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467d83d39abea48a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.406+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c039070 con 0x7fe810004d40 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.406+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c02cbd0 con 0x7fe810004d40 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.406+0000 7fe822ffd640 1 -- 172.21.15.26:0/3297213781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c009c10 con 0x7fe810004d40 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.407+0000 7fe82a3b6640 1 -- 172.21.15.26:0/3297213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe810004d40 msgr2=0x7fe8100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.407+0000 7fe82a3b6640 1 --2- 172.21.15.26:0/3297213781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe810004d40 0x7fe8100051c0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009f90 tx=0x7fe810005830 comp rx=0 tx=0).stop 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.407+0000 7fe828953640 1 -- 172.21.15.26:0/3297213781 reap_dead start 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.407+0000 7fe82a3b6640 1 -- 172.21.15.26:0/3297213781 shutdown_connections 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.407+0000 7fe82a3b6640 1 -- 172.21.15.26:0/3297213781 >> 172.21.15.26:0/3297213781 conn(0x7fe8240c70b0 msgr2=0x7fe8240c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.407+0000 7fe82a3b6640 1 -- 172.21.15.26:0/3297213781 shutdown_connections 2024-03-06T07:28:48.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.407+0000 7fe82a3b6640 1 -- 172.21.15.26:0/3297213781 wait complete. 2024-03-06T07:28:48.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.408+0000 7fe82a3b6640 1 Processor -- start 2024-03-06T07:28:48.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.408+0000 7fe82a3b6640 1 -- start start 2024-03-06T07:28:48.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe82a3b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240c3fa0 0x7fe824070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe82a3b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8241540f0 0x7fe8241576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe82a3b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe824157c00 0x7fe824158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe82a3b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82406e3a0 con 0x7fe824157c00 2024-03-06T07:28:48.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe82a3b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe824159760 con 0x7fe8241540f0 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe82a3b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82406e520 con 0x7fe8240c3fa0 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe824157c00 0x7fe824158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe824157c00 0x7fe824158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58552/0 (socket says 172.21.15.26:58552) 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 -- 172.21.15.26:0/2729762810 learned_addr learned my addr 172.21.15.26:0/2729762810 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe823fff640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240c3fa0 0x7fe824070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe8237fe640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8241540f0 0x7fe8241576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 -- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240c3fa0 msgr2=0x7fe824070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240c3fa0 0x7fe824070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 -- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8241540f0 msgr2=0x7fe8241576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8241540f0 0x7fe8241576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 -- 172.21.15.26:0/2729762810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe824073160 con 0x7fe824157c00 2024-03-06T07:28:48.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe823fff640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240c3fa0 0x7fe824070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe8237fe640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8241540f0 0x7fe8241576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.409+0000 7fe828953640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe824157c00 0x7fe824158080 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009f90 tx=0x7fe81c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.410+0000 7fe820ff9640 1 -- 172.21.15.26:0/2729762810 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c039070 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.410+0000 7fe820ff9640 1 -- 172.21.15.26:0/2729762810 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c044070 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.410+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe824071190 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.410+0000 7fe820ff9640 1 -- 172.21.15.26:0/2729762810 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c009c10 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.410+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8241e2100 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.411+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82404f330 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.411+0000 7fe820ff9640 1 -- 172.21.15.26:0/2729762810 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe81c005d10 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.412+0000 7fe820ff9640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80406b4d0 0x7fe80406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.412+0000 7fe820ff9640 1 -- 172.21.15.26:0/2729762810 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe81c0ac390 con 0x7fe824157c00 2024-03-06T07:28:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.412+0000 7fe823fff640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80406b4d0 0x7fe80406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.412+0000 7fe823fff640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80406b4d0 0x7fe80406d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe814003700 tx=0x7fe814009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.416+0000 7fe820ff9640 1 -- 172.21.15.26:0/2729762810 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe81c04d020 con 0x7fe824157c00 2024-03-06T07:28:48.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.633+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe824156270 con 0x7fe824157c00 2024-03-06T07:28:48.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.634+0000 7fe820ff9640 1 -- 172.21.15.26:0/2729762810 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe81c07b110 con 0x7fe824157c00 2024-03-06T07:28:48.636 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:48.636 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80406b4d0 msgr2=0x7fe80406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80406b4d0 0x7fe80406d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe814003700 tx=0x7fe814009040 comp rx=0 tx=0).stop 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe824157c00 msgr2=0x7fe824158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe824157c00 0x7fe824158080 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009f90 tx=0x7fe81c009750 comp rx=0 tx=0).stop 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 shutdown_connections 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe80406b4d0 0x7fe80406d990 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe824157c00 0x7fe824158080 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8241540f0 0x7fe8241576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 --2- 172.21.15.26:0/2729762810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8240c3fa0 0x7fe824070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.635+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 >> 172.21.15.26:0/2729762810 conn(0x7fe8240c70b0 msgr2=0x7fe82406dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.636+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 shutdown_connections 2024-03-06T07:28:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:48.636+0000 7fe82a3b6640 1 -- 172.21.15.26:0/2729762810 wait complete. 2024-03-06T07:28:48.649 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:48.649 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:53.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.760+0000 7f2b467bf640 1 Processor -- start 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.760+0000 7f2b467bf640 1 -- start start 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.760+0000 7f2b467bf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b4006da40 0x7f2b40155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.760+0000 7f2b467bf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b400c6780 0x7f2b40158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.760+0000 7f2b467bf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b40156050 con 0x7f2b401592d0 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.760+0000 7f2b467bf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b4006de20 con 0x7f2b400c6780 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.760+0000 7f2b467bf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b40072630 con 0x7f2b4006da40 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b400c6780 0x7f2b40158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b44d5c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2b401592d0 0x7f2b400c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50968/0 (socket says 172.21.15.26:50968) 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b44d5c640 1 -- 172.21.15.26:0/4159916633 learned_addr learned my addr 172.21.15.26:0/4159916633 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3f7fe640 1 -- 172.21.15.26:0/4159916633 >> v2:172.21.15.117:3300/0 conn(0x7f2b4006da40 msgr2=0x7f2b40155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3ffff640 1 --2- 172.21.15.26:0/4159916633 >> v2:172.21.15.117:3300/0 conn(0x7f2b4006da40 0x7f2b40155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3f7fe640 1 --2- 172.21.15.26:0/4159916633 >> v2:172.21.15.117:3300/0 conn(0x7f2b4006da40 0x7f2b40155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3effd640 1 -- 172.21.15.26:0/4159916633 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471096356 0 0) 0x7f2b38001490 con 0x7f2b401592d0 2024-03-06T07:28:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3f7fe640 1 -- 172.21.15.26:0/4159916633 >> v1:172.21.15.26:6789/0 conn(0x7f2b401592d0 legacy=0x7f2b400c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3f7fe640 1 -- 172.21.15.26:0/4159916633 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40072910 con 0x7f2b400c6780 2024-03-06T07:28:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.761+0000 7f2b3ffff640 1 --2- 172.21.15.26:0/4159916633 >> v2:172.21.15.117:3300/0 conn(0x7f2b4006da40 0x7f2b40155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b3f7fe640 1 --2- 172.21.15.26:0/4159916633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b400c6780 0x7f2b40158d90 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f2b400c98a0 tx=0x7f2b3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956e1f228bf436 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b3effd640 1 -- 172.21.15.26:0/4159916633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340315a0 con 0x7f2b400c6780 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b3effd640 1 -- 172.21.15.26:0/4159916633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b34037070 con 0x7f2b400c6780 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b3effd640 1 -- 172.21.15.26:0/4159916633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34031960 con 0x7f2b400c6780 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 -- 172.21.15.26:0/4159916633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b400c6780 msgr2=0x7f2b40158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 --2- 172.21.15.26:0/4159916633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b400c6780 0x7f2b40158d90 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f2b400c98a0 tx=0x7f2b3402b840 comp rx=0 tx=0).stop 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 -- 172.21.15.26:0/4159916633 shutdown_connections 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 --2- 172.21.15.26:0/4159916633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b400c6780 0x7f2b40158d90 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 --2- 172.21.15.26:0/4159916633 >> v2:172.21.15.117:3300/0 conn(0x7f2b4006da40 0x7f2b40155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 -- 172.21.15.26:0/4159916633 >> 172.21.15.26:0/4159916633 conn(0x7f2b400aa630 msgr2=0x7f2b400aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 -- 172.21.15.26:0/4159916633 shutdown_connections 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.762+0000 7f2b467bf640 1 -- 172.21.15.26:0/4159916633 wait complete. 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.764+0000 7f2b467bf640 1 Processor -- start 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.764+0000 7f2b467bf640 1 -- start start 2024-03-06T07:28:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.764+0000 7f2b467bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4006da40 0x7f2b401eac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.764+0000 7f2b467bf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b400c6780 0x7f2b4014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.764+0000 7f2b467bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b401592d0 0x7f2b40151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b467bf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400c3990 con 0x7f2b401592d0 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b467bf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40158890 con 0x7f2b4006da40 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b467bf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400c3b10 con 0x7f2b400c6780 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b401592d0 0x7f2b40151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b401592d0 0x7f2b40151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58572/0 (socket says 172.21.15.26:58572) 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 -- 172.21.15.26:0/1782499284 learned_addr learned my addr 172.21.15.26:0/1782499284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b3ffff640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4006da40 0x7f2b401eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b3f7fe640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b400c6780 0x7f2b4014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 -- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b400c6780 msgr2=0x7f2b4014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b400c6780 0x7f2b4014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 -- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4006da40 msgr2=0x7f2b401eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4006da40 0x7f2b401eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 -- 172.21.15.26:0/1782499284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400ad110 con 0x7f2b401592d0 2024-03-06T07:28:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b44d5c640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b401592d0 0x7f2b40151c60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c7b0 tx=0x7f2b3800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b3cff9640 1 -- 172.21.15.26:0/1782499284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38018070 con 0x7f2b401592d0 2024-03-06T07:28:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b3cff9640 1 -- 172.21.15.26:0/1782499284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3800a640 con 0x7f2b401592d0 2024-03-06T07:28:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b401eb3b0 con 0x7f2b401592d0 2024-03-06T07:28:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b3cff9640 1 -- 172.21.15.26:0/1782499284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800ce60 con 0x7f2b401592d0 2024-03-06T07:28:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.765+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b401eb970 con 0x7f2b401592d0 2024-03-06T07:28:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.766+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b401ebbb0 con 0x7f2b401592d0 2024-03-06T07:28:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.767+0000 7f2b3cff9640 1 -- 172.21.15.26:0/1782499284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b3801b070 con 0x7f2b401592d0 2024-03-06T07:28:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.767+0000 7f2b3cff9640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b2c06b500 0x7f2b2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.768+0000 7f2b3cff9640 1 -- 172.21.15.26:0/1782499284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2b380899a0 con 0x7f2b401592d0 2024-03-06T07:28:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.768+0000 7f2b3ffff640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b2c06b500 0x7f2b2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.768+0000 7f2b3ffff640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b2c06b500 0x7f2b2c06d9c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2b400ac980 tx=0x7f2b30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:53.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.772+0000 7f2b3cff9640 1 -- 172.21.15.26:0/1782499284 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2b400ad110 con 0x7f2b401592d0 2024-03-06T07:28:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.990+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2b4004f330 con 0x7f2b401592d0 2024-03-06T07:28:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.990+0000 7f2b3cff9640 1 -- 172.21.15.26:0/1782499284 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2b40156050 con 0x7f2b401592d0 2024-03-06T07:28:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:53.993 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.992+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b2c06b500 msgr2=0x7f2b2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.992+0000 7f2b467bf640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b2c06b500 0x7f2b2c06d9c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2b400ac980 tx=0x7f2b30008040 comp rx=0 tx=0).stop 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.992+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b401592d0 msgr2=0x7f2b40151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.992+0000 7f2b467bf640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b401592d0 0x7f2b40151c60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c7b0 tx=0x7f2b3800cc70 comp rx=0 tx=0).stop 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.992+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 shutdown_connections 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.992+0000 7f2b467bf640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2b2c06b500 0x7f2b2c06d9c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.993+0000 7f2b467bf640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b401592d0 0x7f2b40151c60 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.993+0000 7f2b467bf640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b400c6780 0x7f2b4014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.993+0000 7f2b467bf640 1 --2- 172.21.15.26:0/1782499284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4006da40 0x7f2b401eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.993+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 >> 172.21.15.26:0/1782499284 conn(0x7f2b400aa630 msgr2=0x7f2b400a95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.993+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 shutdown_connections 2024-03-06T07:28:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:53.993+0000 7f2b467bf640 1 -- 172.21.15.26:0/1782499284 wait complete. 2024-03-06T07:28:54.006 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:54.006 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:59.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.118+0000 7f6c4a415640 1 Processor -- start 2024-03-06T07:28:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.118+0000 7f6c4a415640 1 -- start start 2024-03-06T07:28:59.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c4a415640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6c44152520 0x7f6c4406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c4a415640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44155c80 0x7f6c440c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c4a415640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c440720a0 0x7f6c4406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c4a415640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440c1dc0 con 0x7f6c440720a0 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c4a415640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440c1ae0 con 0x7f6c44152520 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c4a415640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4406e450 con 0x7f6c44155c80 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c440720a0 0x7f6c4406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c440720a0 0x7f6c4406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49534/0 (socket says 172.21.15.26:49534) 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 -- 172.21.15.26:0/38457027 learned_addr learned my addr 172.21.15.26:0/38457027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c43fff640 1 --2- 172.21.15.26:0/38457027 >> v2:172.21.15.117:3300/0 conn(0x7f6c44152520 0x7f6c4406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c437fe640 1 --2- 172.21.15.26:0/38457027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44155c80 0x7f6c440c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 -- 172.21.15.26:0/38457027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44155c80 msgr2=0x7f6c440c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 --2- 172.21.15.26:0/38457027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44155c80 0x7f6c440c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 -- 172.21.15.26:0/38457027 >> v2:172.21.15.117:3300/0 conn(0x7f6c44152520 msgr2=0x7f6c4406ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 --2- 172.21.15.26:0/38457027 >> v2:172.21.15.117:3300/0 conn(0x7f6c44152520 0x7f6c4406ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c489b2640 1 -- 172.21.15.26:0/38457027 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4406e730 con 0x7f6c440720a0 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c43fff640 1 --2- 172.21.15.26:0/38457027 >> v2:172.21.15.117:3300/0 conn(0x7f6c44152520 0x7f6c4406ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.119+0000 7f6c437fe640 1 --2- 172.21.15.26:0/38457027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44155c80 0x7f6c440c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.120+0000 7f6c489b2640 1 --2- 172.21.15.26:0/38457027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c440720a0 0x7f6c4406afc0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f6c440c1f80 tx=0x7f6c3c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4858e30a6614597 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.120+0000 7f6c42ffd640 1 -- 172.21.15.26:0/38457027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c02ca80 con 0x7f6c440720a0 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.120+0000 7f6c42ffd640 1 -- 172.21.15.26:0/38457027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c02cc20 con 0x7f6c440720a0 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.120+0000 7f6c42ffd640 1 -- 172.21.15.26:0/38457027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0343f0 con 0x7f6c440720a0 2024-03-06T07:28:59.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 -- 172.21.15.26:0/38457027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c440720a0 msgr2=0x7f6c4406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 --2- 172.21.15.26:0/38457027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c440720a0 0x7f6c4406afc0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f6c440c1f80 tx=0x7f6c3c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 -- 172.21.15.26:0/38457027 shutdown_connections 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 --2- 172.21.15.26:0/38457027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44155c80 0x7f6c440c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 --2- 172.21.15.26:0/38457027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c440720a0 0x7f6c4406afc0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 --2- 172.21.15.26:0/38457027 >> v2:172.21.15.117:3300/0 conn(0x7f6c44152520 0x7f6c4406ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 -- 172.21.15.26:0/38457027 >> 172.21.15.26:0/38457027 conn(0x7f6c440c76b0 msgr2=0x7f6c440c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 -- 172.21.15.26:0/38457027 shutdown_connections 2024-03-06T07:28:59.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.121+0000 7f6c4a415640 1 -- 172.21.15.26:0/38457027 wait complete. 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.122+0000 7f6c4a415640 1 Processor -- start 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.122+0000 7f6c4a415640 1 -- start start 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c4a415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c440720a0 0x7f6c441ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c4a415640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44152520 0x7f6c4414d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c4a415640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c44155c80 0x7f6c4414ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c4a415640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44152900 con 0x7f6c44155c80 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c4a415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c17f0 con 0x7f6c440720a0 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c4a415640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44152a80 con 0x7f6c44152520 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c44155c80 0x7f6c4414ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c44155c80 0x7f6c4414ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49544/0 (socket says 172.21.15.26:49544) 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 -- 172.21.15.26:0/3267166102 learned_addr learned my addr 172.21.15.26:0/3267166102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:59.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c437fe640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44152520 0x7f6c4414d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c43fff640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c440720a0 0x7f6c441ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 -- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44152520 msgr2=0x7f6c4414d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44152520 0x7f6c4414d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 -- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c440720a0 msgr2=0x7f6c441ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c440720a0 0x7f6c441ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 -- 172.21.15.26:0/3267166102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440c6450 con 0x7f6c44155c80 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c43fff640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c440720a0 0x7f6c441ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c489b2640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c44155c80 0x7f6c4414ccd0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02c4f0 tx=0x7f6c3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.123+0000 7f6c40ff9640 1 -- 172.21.15.26:0/3267166102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c043020 con 0x7f6c44155c80 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.124+0000 7f6c40ff9640 1 -- 172.21.15.26:0/3267166102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c03e450 con 0x7f6c44155c80 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.124+0000 7f6c40ff9640 1 -- 172.21.15.26:0/3267166102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c009510 con 0x7f6c44155c80 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.124+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c441de6e0 con 0x7f6c44155c80 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.124+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c441eee60 con 0x7f6c44155c80 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.125+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c441ef060 con 0x7f6c44155c80 2024-03-06T07:28:59.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.125+0000 7f6c40ff9640 1 -- 172.21.15.26:0/3267166102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c3c046020 con 0x7f6c44155c80 2024-03-06T07:28:59.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.126+0000 7f6c40ff9640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c3006b4d0 0x7f6c3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.126+0000 7f6c40ff9640 1 -- 172.21.15.26:0/3267166102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6c3c0ad0b0 con 0x7f6c44155c80 2024-03-06T07:28:59.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.126+0000 7f6c43fff640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c3006b4d0 0x7f6c3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.126+0000 7f6c43fff640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c3006b4d0 0x7f6c3006d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6c4406fb90 tx=0x7f6c34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.130+0000 7f6c40ff9640 1 -- 172.21.15.26:0/3267166102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c3c04d020 con 0x7f6c44155c80 2024-03-06T07:28:59.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.347+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6c4404f330 con 0x7f6c44155c80 2024-03-06T07:28:59.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.348+0000 7f6c40ff9640 1 -- 172.21.15.26:0/3267166102 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6c3c07bd50 con 0x7f6c44155c80 2024-03-06T07:28:59.350 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:59.350 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c3006b4d0 msgr2=0x7f6c3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c3006b4d0 0x7f6c3006d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6c4406fb90 tx=0x7f6c34008040 comp rx=0 tx=0).stop 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c44155c80 msgr2=0x7f6c4414ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c44155c80 0x7f6c4414ccd0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02c4f0 tx=0x7f6c3c004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 shutdown_connections 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c3006b4d0 0x7f6c3006d990 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c44155c80 0x7f6c4414ccd0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c44152520 0x7f6c4414d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 --2- 172.21.15.26:0/3267166102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c440720a0 0x7f6c441ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.349+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 >> 172.21.15.26:0/3267166102 conn(0x7f6c440c76b0 msgr2=0x7f6c440c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.350+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 shutdown_connections 2024-03-06T07:28:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.350+0000 7f6c4a415640 1 -- 172.21.15.26:0/3267166102 wait complete. 2024-03-06T07:28:59.363 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:59.363 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:28:59.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:59.403 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:59.462 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:28:59.462 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:28:59.462 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:59.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:28:59.477 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:28:59.478 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:59.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:28:59.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.615+0000 7fab91066640 1 Processor -- start 2024-03-06T07:28:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.615+0000 7fab91066640 1 -- start start 2024-03-06T07:28:59.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab91066640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 0x7fab8c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab91066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8c0c7c60 con 0x7fab8c0c88b0 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab91066640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8c0c8060 con 0x7fab8c1527b0 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab91066640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c159640 con 0x7fab8c0c7180 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab8b59e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fab8c0c88b0 0x7fab8c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43538/0 (socket says 172.21.15.26:43538) 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab8b59e640 1 -- 172.21.15.26:0/3976377575 learned_addr learned my addr 172.21.15.26:0/3976377575 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161110022 0 0) 0x7fab7c001490 con 0x7fab8c0c88b0 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab74003520 con 0x7fab8c0c88b0 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.616+0000 7fab8a59c640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 0x7fab8c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3628274048 0 0) 0x7fab74003520 con 0x7fab8c0c88b0 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 msgr2=0x7fab8c159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 0x7fab8c159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 >> v1:172.21.15.117:6789/0 conn(0x7fab8c1527b0 legacy=0x7fab8c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c0c76a0 con 0x7fab8c0c88b0 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fab7c001bc0 con 0x7fab8c0c88b0 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 >> v1:172.21.15.26:6789/0 conn(0x7fab8c0c88b0 legacy=0x7fab8c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 0x7fab740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab74004010 0x7fab74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c159640 con 0x7fab74004d40 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7400bc00 con 0x7fab74004010 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7400be00 con 0x7fab8c0c7180 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8b59e640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8ad9d640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 0x7fab740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8a59c640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab74004010 0x7fab74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8b59e640 1 -- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 msgr2=0x7fab740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8b59e640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c0c7180 0x7fab740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8b59e640 1 -- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab74004010 msgr2=0x7fab74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8b59e640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab74004010 0x7fab74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.617+0000 7fab8b59e640 1 -- 172.21.15.26:0/3976377575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74003520 con 0x7fab74004d40 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.618+0000 7fab8b59e640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab740051c0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00aa50 tx=0x7fab74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440e9af4a5853495 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.618+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c038070 con 0x7fab74004d40 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.618+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c033650 con 0x7fab74004d40 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.618+0000 7fab89d9b640 1 -- 172.21.15.26:0/3976377575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00a670 con 0x7fab74004d40 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.618+0000 7fab91066640 1 -- 172.21.15.26:0/3976377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 msgr2=0x7fab740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.618+0000 7fab91066640 1 --2- 172.21.15.26:0/3976377575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab740051c0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00aa50 tx=0x7fab74005830 comp rx=0 tx=0).stop 2024-03-06T07:28:59.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.619+0000 7fab8b59e640 1 -- 172.21.15.26:0/3976377575 reap_dead start 2024-03-06T07:28:59.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.619+0000 7fab91066640 1 -- 172.21.15.26:0/3976377575 shutdown_connections 2024-03-06T07:28:59.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.619+0000 7fab91066640 1 -- 172.21.15.26:0/3976377575 >> 172.21.15.26:0/3976377575 conn(0x7fab8c0ac560 msgr2=0x7fab8c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:59.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.619+0000 7fab91066640 1 -- 172.21.15.26:0/3976377575 shutdown_connections 2024-03-06T07:28:59.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.619+0000 7fab91066640 1 -- 172.21.15.26:0/3976377575 wait complete. 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.620+0000 7fab91066640 1 Processor -- start 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.620+0000 7fab91066640 1 -- start start 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab91066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab8c14d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab91066640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c158770 0x7fab8c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab91066640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c1ee410 0x7fab8c0c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab91066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c155600 con 0x7fab74004d40 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab91066640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c155320 con 0x7fab8c158770 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab91066640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c0c8d60 con 0x7fab8c1ee410 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab8c14d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab8c14d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49568/0 (socket says 172.21.15.26:49568) 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 -- 172.21.15.26:0/3087528831 learned_addr learned my addr 172.21.15.26:0/3087528831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8a59c640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c158770 0x7fab8c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8b59e640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c1ee410 0x7fab8c0c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 -- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c1ee410 msgr2=0x7fab8c0c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c1ee410 0x7fab8c0c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 -- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c158770 msgr2=0x7fab8c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c158770 0x7fab8c0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 -- 172.21.15.26:0/3087528831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c1edc30 con 0x7fab74004d40 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8b59e640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c1ee410 0x7fab8c0c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab8ad9d640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab8c14d5e0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fab8400f820 tx=0x7fab8400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.621+0000 7fab737fe640 1 -- 172.21.15.26:0/3087528831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840050d0 con 0x7fab74004d40 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.622+0000 7fab737fe640 1 -- 172.21.15.26:0/3087528831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab84018070 con 0x7fab74004d40 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.622+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8c156980 con 0x7fab74004d40 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.622+0000 7fab737fe640 1 -- 172.21.15.26:0/3087528831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84005490 con 0x7fab74004d40 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.622+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c14ca50 con 0x7fab74004d40 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.622+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8c04f330 con 0x7fab74004d40 2024-03-06T07:28:59.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.623+0000 7fab737fe640 1 -- 172.21.15.26:0/3087528831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab8400d6d0 con 0x7fab74004d40 2024-03-06T07:28:59.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.623+0000 7fab737fe640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b500 0x7fab7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.623+0000 7fab737fe640 1 -- 172.21.15.26:0/3087528831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fab8408a100 con 0x7fab74004d40 2024-03-06T07:28:59.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.624+0000 7fab8a59c640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b500 0x7fab7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.624+0000 7fab8a59c640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b500 0x7fab7806d9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fab8c152e00 tx=0x7fab80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.628+0000 7fab737fe640 1 -- 172.21.15.26:0/3087528831 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab84058da0 con 0x7fab74004d40 2024-03-06T07:28:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.846+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab8c06ce00 con 0x7fab74004d40 2024-03-06T07:28:59.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.847+0000 7fab737fe640 1 -- 172.21.15.26:0/3087528831 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4977 (secure 0 0 0) 0x7fab840586a0 con 0x7fab74004d40 2024-03-06T07:28:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:28:59.849 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":192,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2181907552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2181907552},{"type":"v1","addr":"172.21.15.117:6836","nonce":2181907552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":8129,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2243060002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2243060002},{"type":"v1","addr":"172.21.15.117:6837","nonce":2243060002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":9529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4068022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4068022212},{"type":"v1","addr":"172.21.15.26:6834","nonce":4068022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":10563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2089138655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2089138655},{"type":"v1","addr":"172.21.15.26:6835","nonce":2089138655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:15.142228+0000","modified":"2024-03-06T07:28:40.703192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:28:59.850 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 192 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b500 msgr2=0x7fab7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b500 0x7fab7806d9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fab8c152e00 tx=0x7fab80008040 comp rx=0 tx=0).stop 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 msgr2=0x7fab8c14d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab8c14d5e0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fab8400f820 tx=0x7fab8400fce0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 shutdown_connections 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7806b500 0x7fab7806d9c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab8c1ee410 0x7fab8c0c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c158770 0x7fab8c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 --2- 172.21.15.26:0/3087528831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74004d40 0x7fab8c14d5e0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 >> 172.21.15.26:0/3087528831 conn(0x7fab8c0ac560 msgr2=0x7fab8c14feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:59.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 shutdown_connections 2024-03-06T07:28:59.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.849+0000 7fab91066640 1 -- 172.21.15.26:0/3087528831 wait complete. 2024-03-06T07:28:59.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.962+0000 7f6077288640 1 Processor -- start 2024-03-06T07:28:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.962+0000 7f6077288640 1 -- start start 2024-03-06T07:28:59.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6077288640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 0x7f6070155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6077288640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60700c7380 0x7f60700c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6077288640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6070156050 con 0x7f60700c6780 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6077288640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607006de20 con 0x7f60700c7380 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6077288640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6070072630 con 0x7f607006da40 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6074823640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f60700c6780 0x7f6070158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43540/0 (socket says 172.21.15.26:43540) 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6074823640 1 -- 172.21.15.26:0/632261494 learned_addr learned my addr 172.21.15.26:0/632261494 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6075024640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 0x7f6070155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6075825640 1 --2- 172.21.15.26:0/632261494 >> v2:172.21.15.117:3300/0 conn(0x7f60700c7380 0x7f60700c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737134452 0 0) 0x7f6060001110 con 0x7f60700c6780 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f605c003520 con 0x7f60700c6780 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.963+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695453360 0 0) 0x7f605c003520 con 0x7f60700c6780 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 msgr2=0x7f6070155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 0x7f6070155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 >> v2:172.21.15.117:3300/0 conn(0x7f60700c7380 msgr2=0x7f60700c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 --2- 172.21.15.26:0/632261494 >> v2:172.21.15.117:3300/0 conn(0x7f60700c7380 0x7f60700c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070072910 con 0x7f60700c6780 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6060001840 con 0x7f60700c6780 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075024640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 0x7f6070155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 >> v1:172.21.15.26:6789/0 conn(0x7f60700c6780 legacy=0x7f6070158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c0039b0 0x7f605c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 0x7f605c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f605c004d40 0x7f605c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607006de20 con 0x7f605c004d40 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070072630 con 0x7f605c0039b0 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c00b9c0 con 0x7f607006da40 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075825640 1 --2- 172.21.15.26:0/632261494 >> v2:172.21.15.117:3300/0 conn(0x7f60700c7380 0x7f60700c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075825640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f605c004d40 0x7f605c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6074823640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 0x7f605c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075024640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c0039b0 0x7f605c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075825640 1 -- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 msgr2=0x7f605c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075825640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f607006da40 0x7f605c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075825640 1 -- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c0039b0 msgr2=0x7f605c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075825640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c0039b0 0x7f605c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.964+0000 7f6075825640 1 -- 172.21.15.26:0/632261494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c003520 con 0x7f605c004d40 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6075825640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f605c004d40 0x7f605c0051c0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f606c00e320 tx=0x7f605c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c742d9ecd136cf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c039070 con 0x7f605c004d40 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c02cdc0 con 0x7f605c004d40 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6067fff640 1 -- 172.21.15.26:0/632261494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c009de0 con 0x7f605c004d40 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6077288640 1 -- 172.21.15.26:0/632261494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f605c004d40 msgr2=0x7f605c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6077288640 1 --2- 172.21.15.26:0/632261494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f605c004d40 0x7f605c0051c0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f606c00e320 tx=0x7f605c005830 comp rx=0 tx=0).stop 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6075825640 1 -- 172.21.15.26:0/632261494 reap_dead start 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6077288640 1 -- 172.21.15.26:0/632261494 shutdown_connections 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.965+0000 7f6077288640 1 -- 172.21.15.26:0/632261494 >> 172.21.15.26:0/632261494 conn(0x7f60700aa630 msgr2=0x7f60700aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:59.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.966+0000 7f6077288640 1 -- 172.21.15.26:0/632261494 shutdown_connections 2024-03-06T07:28:59.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.966+0000 7f6077288640 1 -- 172.21.15.26:0/632261494 wait complete. 2024-03-06T07:28:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 Processor -- start 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 -- start start 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c004d40 0x7f6070148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60701563c0 0x7f6070152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60701531a0 0x7f60700c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701594a0 con 0x7f60701563c0 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070158890 con 0x7f605c004d40 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6077288640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60700c37c0 con 0x7f60701531a0 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6074823640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60701563c0 0x7f6070152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6074823640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60701563c0 0x7f6070152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49592/0 (socket says 172.21.15.26:49592) 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.967+0000 7f6074823640 1 -- 172.21.15.26:0/803529166 learned_addr learned my addr 172.21.15.26:0/803529166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6075825640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60701531a0 0x7f60700c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6074823640 1 -- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60701531a0 msgr2=0x7f60700c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6074823640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60701531a0 0x7f60700c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6075024640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c004d40 0x7f6070148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6074823640 1 -- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c004d40 msgr2=0x7f6070148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6074823640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c004d40 0x7f6070148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6074823640 1 -- 172.21.15.26:0/803529166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070148c10 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6075825640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60701531a0 0x7f60700c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6075024640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c004d40 0x7f6070148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6074823640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60701563c0 0x7f6070152c60 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f6060007340 tx=0x7f6060007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6065ffb640 1 -- 172.21.15.26:0/803529166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000f9c0 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6065ffb640 1 -- 172.21.15.26:0/803529166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6060018070 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60700a9dd0 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6065ffb640 1 -- 172.21.15.26:0/803529166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000fdc0 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.968+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701ee140 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.969+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60700c3580 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.969+0000 7f6065ffb640 1 -- 172.21.15.26:0/803529166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f606001d460 con 0x7f60701563c0 2024-03-06T07:28:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.970+0000 7f6065ffb640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f605406b4d0 0x7f605406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.970+0000 7f6065ffb640 1 -- 172.21.15.26:0/803529166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f606008a540 con 0x7f60701563c0 2024-03-06T07:28:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.970+0000 7f6075024640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f605406b4d0 0x7f605406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.971+0000 7f6075024640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f605406b4d0 0x7f605406d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6068009a30 tx=0x7f6068008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:28:59.975+0000 7f6065ffb640 1 -- 172.21.15.26:0/803529166 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f60700c3580 con 0x7f60701563c0 2024-03-06T07:29:00.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.157+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60701ee340 con 0x7f60701563c0 2024-03-06T07:29:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.159+0000 7f6065ffb640 1 -- 172.21.15.26:0/803529166 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17109 (secure 0 0 0) 0x7f60701ee340 con 0x7f60701563c0 2024-03-06T07:29:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:00.162 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":200,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:51.890050+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.447072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:14.106103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.2","mappings":[{"from":6,"to":7}]},{"pgid":"32.20","mappings":[{"from":2,"to":0}]},{"pgid":"32.36","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f605406b4d0 msgr2=0x7f605406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f605406b4d0 0x7f605406d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6068009a30 tx=0x7f6068008040 comp rx=0 tx=0).stop 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60701563c0 msgr2=0x7f6070152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60701563c0 0x7f6070152c60 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f6060007340 tx=0x7f6060007800 comp rx=0 tx=0).stop 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 shutdown_connections 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f605406b4d0 0x7f605406d990 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60701531a0 0x7f60700c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60701563c0 0x7f6070152c60 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 --2- 172.21.15.26:0/803529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f605c004d40 0x7f6070148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 >> 172.21.15.26:0/803529166 conn(0x7f60700aa630 msgr2=0x7f607006ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 shutdown_connections 2024-03-06T07:29:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.161+0000 7f6077288640 1 -- 172.21.15.26:0/803529166 wait complete. 2024-03-06T07:29:00.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:28:13.447072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:28:14.106103+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:29:00.175 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:29:00.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.277+0000 7f9a8c75e640 1 Processor -- start 2024-03-06T07:29:00.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.277+0000 7f9a8c75e640 1 -- start start 2024-03-06T07:29:00.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.277+0000 7f9a8c75e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a841529f0 0x7f9a840c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.277+0000 7f9a8c75e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a84152dd0 con 0x7f9a840c5370 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.277+0000 7f9a8c75e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a84158b00 con 0x7f9a840c57d0 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.277+0000 7f9a8c75e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a84159710 con 0x7f9a841529f0 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.278+0000 7f9a8a4fa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a841529f0 0x7f9a840c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.278+0000 7f9a89cf9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9a840c5370 0x7f9a84155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43554/0 (socket says 172.21.15.26:43554) 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.278+0000 7f9a89cf9640 1 -- 172.21.15.26:0/4012805361 learned_addr learned my addr 172.21.15.26:0/4012805361 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.278+0000 7f9a8a4fa640 1 -- 172.21.15.26:0/4012805361 >> v1:172.21.15.117:6789/0 conn(0x7f9a840c57d0 legacy=0x7f9a8415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.278+0000 7f9a8a4fa640 1 -- 172.21.15.26:0/4012805361 >> v1:172.21.15.26:6789/0 conn(0x7f9a840c5370 legacy=0x7f9a84155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.278+0000 7f9a8a4fa640 1 -- 172.21.15.26:0/4012805361 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84158de0 con 0x7f9a841529f0 2024-03-06T07:29:00.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.278+0000 7f9a894f8640 1 -- 172.21.15.26:0/4012805361 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402403082 0 0) 0x7f9a84152dd0 con 0x7f9a840c5370 2024-03-06T07:29:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8a4fa640 1 --2- 172.21.15.26:0/4012805361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a841529f0 0x7f9a840c88f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0097a0 tx=0x7f9a7c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35bde71fb65fbd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a894f8640 1 -- 172.21.15.26:0/4012805361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0315a0 con 0x7f9a841529f0 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a894f8640 1 -- 172.21.15.26:0/4012805361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c037070 con 0x7f9a841529f0 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a894f8640 1 -- 172.21.15.26:0/4012805361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c031960 con 0x7f9a841529f0 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8c75e640 1 -- 172.21.15.26:0/4012805361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a841529f0 msgr2=0x7f9a840c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/4012805361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a841529f0 0x7f9a840c88f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0097a0 tx=0x7f9a7c02b840 comp rx=0 tx=0).stop 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8c75e640 1 -- 172.21.15.26:0/4012805361 shutdown_connections 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/4012805361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a841529f0 0x7f9a840c88f0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8c75e640 1 -- 172.21.15.26:0/4012805361 >> 172.21.15.26:0/4012805361 conn(0x7f9a840ac580 msgr2=0x7f9a840ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8c75e640 1 -- 172.21.15.26:0/4012805361 shutdown_connections 2024-03-06T07:29:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.279+0000 7f9a8c75e640 1 -- 172.21.15.26:0/4012805361 wait complete. 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.280+0000 7f9a8c75e640 1 Processor -- start 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8c75e640 1 -- start start 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8c75e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a840c5370 0x7f9a841ee560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8c75e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a840c57d0 0x7f9a8414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8c75e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a841529f0 0x7f9a8414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8c75e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841593a0 con 0x7f9a840c5370 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8c75e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a840c68b0 con 0x7f9a841529f0 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8c75e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a84158740 con 0x7f9a840c57d0 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8a4fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a840c5370 0x7f9a841ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8a4fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a840c5370 0x7f9a841ee560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49616/0 (socket says 172.21.15.26:49616) 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.281+0000 7f9a8a4fa640 1 -- 172.21.15.26:0/795174099 learned_addr learned my addr 172.21.15.26:0/795174099 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8acfb640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a841529f0 0x7f9a8414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a89cf9640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a840c57d0 0x7f9a8414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8a4fa640 1 -- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a840c57d0 msgr2=0x7f9a8414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8a4fa640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a840c57d0 0x7f9a8414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8a4fa640 1 -- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a841529f0 msgr2=0x7f9a8414e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8a4fa640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a841529f0 0x7f9a8414e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8a4fa640 1 -- 172.21.15.26:0/795174099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a840ae950 con 0x7f9a840c5370 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a89cf9640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a840c57d0 0x7f9a8414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8acfb640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a841529f0 0x7f9a8414e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8a4fa640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a840c5370 0x7f9a841ee560 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c000c00 tx=0x7f9a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a82ffd640 1 -- 172.21.15.26:0/795174099 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c032840 con 0x7f9a840c5370 2024-03-06T07:29:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a82ffd640 1 -- 172.21.15.26:0/795174099 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0329e0 con 0x7f9a840c5370 2024-03-06T07:29:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a82ffd640 1 -- 172.21.15.26:0/795174099 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c032dd0 con 0x7f9a840c5370 2024-03-06T07:29:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a841eef50 con 0x7f9a840c5370 2024-03-06T07:29:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.282+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a840aeed0 con 0x7f9a840c5370 2024-03-06T07:29:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.283+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8404f330 con 0x7f9a840c5370 2024-03-06T07:29:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.283+0000 7f9a82ffd640 1 -- 172.21.15.26:0/795174099 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a7c04d3f0 con 0x7f9a840c5370 2024-03-06T07:29:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.284+0000 7f9a82ffd640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.284+0000 7f9a82ffd640 1 -- 172.21.15.26:0/795174099 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9a7c02f030 con 0x7f9a840c5370 2024-03-06T07:29:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.284+0000 7f9a89cf9640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.285+0000 7f9a89cf9640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a7006b4d0 0x7f9a7006d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9a84151930 tx=0x7f9a78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.289+0000 7f9a82ffd640 1 -- 172.21.15.26:0/795174099 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9a7c040070 con 0x7f9a840c5370 2024-03-06T07:29:00.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.467+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a840ac960 con 0x7f9a840c5370 2024-03-06T07:29:00.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.468+0000 7f9a82ffd640 1 -- 172.21.15.26:0/795174099 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a7c07ac40 con 0x7f9a840c5370 2024-03-06T07:29:00.470 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:29:00.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a7006b4d0 msgr2=0x7f9a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a7006b4d0 0x7f9a7006d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9a84151930 tx=0x7f9a78009040 comp rx=0 tx=0).stop 2024-03-06T07:29:00.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a840c5370 msgr2=0x7f9a841ee560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a840c5370 0x7f9a841ee560 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c000c00 tx=0x7f9a7c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 shutdown_connections 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a841529f0 0x7f9a8414e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a840c57d0 0x7f9a8414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 --2- 172.21.15.26:0/795174099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a840c5370 0x7f9a841ee560 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 >> 172.21.15.26:0/795174099 conn(0x7f9a840ac580 msgr2=0x7f9a840ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 shutdown_connections 2024-03-06T07:29:00.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.470+0000 7f9a8c75e640 1 -- 172.21.15.26:0/795174099 wait complete. 2024-03-06T07:29:00.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:29:00.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d2913640 1 Processor -- start 2024-03-06T07:29:00.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d2913640 1 -- start start 2024-03-06T07:29:00.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d2913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc0b0070 0x7fb1cc155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d2913640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1cc0c5380 0x7fb1cc0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d2913640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc0b0450 con 0x7fb1cc06ca70 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d2913640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1cc06c020 con 0x7fb1cc0c5380 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d2913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0c77f0 con 0x7fb1cc0b0070 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d0eb0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1cc0c5380 0x7fb1cc0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d0eb0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1cc0c5380 0x7fb1cc0c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49620/0 (socket says 172.21.15.26:49620) 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/1257277736 learned_addr learned my addr 172.21.15.26:0/1257277736 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1cbfff640 1 --2- 172.21.15.26:0/1257277736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc0b0070 0x7fb1cc155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/1257277736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc0b0070 msgr2=0x7fb1cc155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d0eb0640 1 --2- 172.21.15.26:0/1257277736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc0b0070 0x7fb1cc155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/1257277736 >> v1:172.21.15.117:6789/0 conn(0x7fb1cc06ca70 legacy=0x7fb1cc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.586+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/1257277736 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0c7ad0 con 0x7fb1cc0c5380 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d0eb0640 1 --2- 172.21.15.26:0/1257277736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc0c57d0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009e50 tx=0x7fb1c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60758da609b4d29 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1caffd640 1 -- 172.21.15.26:0/1257277736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c402ca70 con 0x7fb1cc0c5380 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1caffd640 1 -- 172.21.15.26:0/1257277736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c402cc10 con 0x7fb1cc0c5380 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1caffd640 1 -- 172.21.15.26:0/1257277736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40343f0 con 0x7fb1cc0c5380 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 -- 172.21.15.26:0/1257277736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 msgr2=0x7fb1cc0c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 --2- 172.21.15.26:0/1257277736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc0c57d0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009e50 tx=0x7fb1c402bed0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 -- 172.21.15.26:0/1257277736 shutdown_connections 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 --2- 172.21.15.26:0/1257277736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc0c57d0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 --2- 172.21.15.26:0/1257277736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc0b0070 0x7fb1cc155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 -- 172.21.15.26:0/1257277736 >> 172.21.15.26:0/1257277736 conn(0x7fb1cc0ac590 msgr2=0x7fb1cc0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 -- 172.21.15.26:0/1257277736 shutdown_connections 2024-03-06T07:29:00.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.587+0000 7fb1d2913640 1 -- 172.21.15.26:0/1257277736 wait complete. 2024-03-06T07:29:00.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.588+0000 7fb1d2913640 1 Processor -- start 2024-03-06T07:29:00.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d2913640 1 -- start start 2024-03-06T07:29:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d2913640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc06ca70 0x7fb1cc1eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d2913640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1cc0b0070 0x7fb1cc1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d2913640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d2913640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc06cef0 con 0x7fb1cc0c5380 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d2913640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc06c550 con 0x7fb1cc0b0070 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d2913640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc06da80 con 0x7fb1cc06ca70 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d0eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d0eb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49622/0 (socket says 172.21.15.26:49622) 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.589+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/473407225 learned_addr learned my addr 172.21.15.26:0/473407225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1cbfff640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc06ca70 0x7fb1cc1eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1cb7fe640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1cc0b0070 0x7fb1cc1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc06ca70 msgr2=0x7fb1cc1eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d0eb0640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc06ca70 0x7fb1cc1eb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1cc0b0070 msgr2=0x7fb1cc1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d0eb0640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1cc0b0070 0x7fb1cc1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d0eb0640 1 -- 172.21.15.26:0/473407225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0c3bc0 con 0x7fb1cc0c5380 2024-03-06T07:29:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1cb7fe640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1cc0b0070 0x7fb1cc1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1cbfff640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc06ca70 0x7fb1cc1eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d0eb0640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc156b30 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009e50 tx=0x7fb1c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1c8ff9640 1 -- 172.21.15.26:0/473407225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4043020 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1c8ff9640 1 -- 172.21.15.26:0/473407225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c403e450 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc0c45a0 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1c8ff9640 1 -- 172.21.15.26:0/473407225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4009510 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.590+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc0c4b60 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.591+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc04f330 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.591+0000 7fb1c8ff9640 1 -- 172.21.15.26:0/473407225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1c4046020 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.592+0000 7fb1c8ff9640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1b806b4d0 0x7fb1b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.592+0000 7fb1c8ff9640 1 -- 172.21.15.26:0/473407225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fb1c40acb90 con 0x7fb1cc0c5380 2024-03-06T07:29:00.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.592+0000 7fb1cbfff640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1b806b4d0 0x7fb1b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.594+0000 7fb1cbfff640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1b806b4d0 0x7fb1b806d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0006fd0 tx=0x7fb1c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.597+0000 7fb1c8ff9640 1 -- 172.21.15.26:0/473407225 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb1c404d030 con 0x7fb1cc0c5380 2024-03-06T07:29:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.776+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1cc06d860 con 0x7fb1cc0c5380 2024-03-06T07:29:00.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.776+0000 7fb1c8ff9640 1 -- 172.21.15.26:0/473407225 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v200) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1cc0c3bc0 con 0x7fb1cc0c5380 2024-03-06T07:29:00.779 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:29:00.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1b806b4d0 msgr2=0x7fb1b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1b806b4d0 0x7fb1b806d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0006fd0 tx=0x7fb1c0008040 comp rx=0 tx=0).stop 2024-03-06T07:29:00.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 msgr2=0x7fb1cc156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc156b30 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009e50 tx=0x7fb1c4004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 shutdown_connections 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb1b806b4d0 0x7fb1b806d990 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc0c5380 0x7fb1cc156b30 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1cc0b0070 0x7fb1cc1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 --2- 172.21.15.26:0/473407225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1cc06ca70 0x7fb1cc1eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 >> 172.21.15.26:0/473407225 conn(0x7fb1cc0ac590 msgr2=0x7fb1cc151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.778+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 shutdown_connections 2024-03-06T07:29:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.779+0000 7fb1d2913640 1 -- 172.21.15.26:0/473407225 wait complete. 2024-03-06T07:29:00.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:29:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.893+0000 7f801384e640 1 Processor -- start 2024-03-06T07:29:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.893+0000 7f801384e640 1 -- start start 2024-03-06T07:29:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f801384e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f801384e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f800c0c5380 0x7f800c0c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f801384e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c0b0450 con 0x7f800c0b0070 2024-03-06T07:29:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f801384e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c06cfb0 con 0x7f800c0c5380 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f801384e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c06c020 con 0x7f800c155c30 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f8011deb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f800c0c5380 0x7f800c0c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f80115ea640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f800c0b0070 0x7f800c0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43560/0 (socket says 172.21.15.26:43560) 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.894+0000 7f80115ea640 1 -- 172.21.15.26:0/3989533907 learned_addr learned my addr 172.21.15.26:0/3989533907 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.895+0000 7f8010de9640 1 --2- 172.21.15.26:0/3989533907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.895+0000 7f8011deb640 1 -- 172.21.15.26:0/3989533907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 msgr2=0x7f800c06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.895+0000 7f8011deb640 1 --2- 172.21.15.26:0/3989533907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c06ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.895+0000 7f8011deb640 1 -- 172.21.15.26:0/3989533907 >> v1:172.21.15.26:6789/0 conn(0x7f800c0b0070 legacy=0x7f800c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.895+0000 7f8003fff640 1 -- 172.21.15.26:0/3989533907 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35918472 0 0) 0x7f8008001110 con 0x7f800c0b0070 2024-03-06T07:29:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.895+0000 7f8011deb640 1 -- 172.21.15.26:0/3989533907 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c1560e0 con 0x7f800c0c5380 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.895+0000 7f8011deb640 1 --2- 172.21.15.26:0/3989533907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c0c57f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8004009d90 tx=0x7f800402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30c46cee9cad7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f8003fff640 1 -- 172.21.15.26:0/3989533907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004039070 con 0x7f800c0c5380 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f8003fff640 1 -- 172.21.15.26:0/3989533907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800402cbc0 con 0x7f800c0c5380 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f8003fff640 1 -- 172.21.15.26:0/3989533907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004034440 con 0x7f800c0c5380 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 -- 172.21.15.26:0/3989533907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 msgr2=0x7f800c0c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 --2- 172.21.15.26:0/3989533907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c0c57f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8004009d90 tx=0x7f800402be10 comp rx=0 tx=0).stop 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 -- 172.21.15.26:0/3989533907 shutdown_connections 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 --2- 172.21.15.26:0/3989533907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c0c57f0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 --2- 172.21.15.26:0/3989533907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c06ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 -- 172.21.15.26:0/3989533907 >> 172.21.15.26:0/3989533907 conn(0x7f800c0ac590 msgr2=0x7f800c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 -- 172.21.15.26:0/3989533907 shutdown_connections 2024-03-06T07:29:00.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.896+0000 7f801384e640 1 -- 172.21.15.26:0/3989533907 wait complete. 2024-03-06T07:29:00.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.897+0000 7f801384e640 1 Processor -- start 2024-03-06T07:29:00.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.897+0000 7f801384e640 1 -- start start 2024-03-06T07:29:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f801384e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c0b0070 0x7f800c0c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f801384e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f801384e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c1ee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f801384e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c06c270 con 0x7f800c0c5380 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f801384e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0c7ad0 con 0x7f800c0b0070 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f801384e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c06c550 con 0x7f800c155c30 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49630/0 (socket says 172.21.15.26:49630) 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 -- 172.21.15.26:0/256956056 learned_addr learned my addr 172.21.15.26:0/256956056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f80115ea640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c0b0070 0x7f800c0c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8011deb640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c1ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 -- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 msgr2=0x7f800c1ee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c1ee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 -- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c0b0070 msgr2=0x7f800c0c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c0b0070 0x7f800c0c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f8010de9640 1 -- 172.21.15.26:0/256956056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c0c4940 con 0x7f800c0c5380 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.898+0000 7f80115ea640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c0b0070 0x7f800c0c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.899+0000 7f8010de9640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c156560 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c820 tx=0x7f7ffc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.899+0000 7f8001ffb640 1 -- 172.21.15.26:0/256956056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc015070 con 0x7f800c0c5380 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.899+0000 7f8001ffb640 1 -- 172.21.15.26:0/256956056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc004240 con 0x7f800c0c5380 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.899+0000 7f8001ffb640 1 -- 172.21.15.26:0/256956056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc00ce10 con 0x7f800c0c5380 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.899+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c0c4c20 con 0x7f800c0c5380 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.899+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c156aa0 con 0x7f800c0c5380 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.900+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c1eeca0 con 0x7f800c0c5380 2024-03-06T07:29:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.900+0000 7f8001ffb640 1 -- 172.21.15.26:0/256956056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ffc007830 con 0x7f800c0c5380 2024-03-06T07:29:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.901+0000 7f8001ffb640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff006b500 0x7f7ff006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.901+0000 7f8001ffb640 1 -- 172.21.15.26:0/256956056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f7ffc08a210 con 0x7f800c0c5380 2024-03-06T07:29:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.901+0000 7f80115ea640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff006b500 0x7f7ff006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.901+0000 7f80115ea640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff006b500 0x7f7ff006d9c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8008000f60 tx=0x7f8008007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:00.906+0000 7f8001ffb640 1 -- 172.21.15.26:0/256956056 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f800c1eeca0 con 0x7f800c0c5380 2024-03-06T07:29:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.086+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f800c0c3970 con 0x7f800c0c5380 2024-03-06T07:29:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.086+0000 7f8001ffb640 1 -- 172.21.15.26:0/256956056 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v200) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ffc058f40 con 0x7f800c0c5380 2024-03-06T07:29:01.088 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff006b500 msgr2=0x7f7ff006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff006b500 0x7f7ff006d9c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8008000f60 tx=0x7f8008007ca0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 msgr2=0x7f800c156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c156560 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c820 tx=0x7f7ffc00cce0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 shutdown_connections 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ff006b500 0x7f7ff006d9c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f800c155c30 0x7f800c1ee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c0c5380 0x7f800c156560 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 --2- 172.21.15.26:0/256956056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c0b0070 0x7f800c0c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 >> 172.21.15.26:0/256956056 conn(0x7f800c0ac590 msgr2=0x7f800c0c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 shutdown_connections 2024-03-06T07:29:01.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.088+0000 7f801384e640 1 -- 172.21.15.26:0/256956056 wait complete. 2024-03-06T07:29:01.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:01.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.200+0000 7f9e30065640 1 Processor -- start 2024-03-06T07:29:01.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.200+0000 7f9e30065640 1 -- start start 2024-03-06T07:29:01.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e30065640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9e28072720 0x7f9e281564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e30065640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28156a00 0x7f9e28150290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e30065640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e281507d0 0x7f9e280c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e30065640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e280c38d0 con 0x7f9e28072720 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e30065640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e28156f90 con 0x7f9e281507d0 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e30065640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28157270 con 0x7f9e28156a00 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e2de01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9e28072720 0x7f9e281564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.201+0000 7f9e2de01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9e28072720 0x7f9e281564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49634/0 (socket says 172.21.15.26:49634) 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2de01640 1 -- 172.21.15.26:0/4224644096 learned_addr learned my addr 172.21.15.26:0/4224644096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2d600640 1 --2- 172.21.15.26:0/4224644096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28156a00 0x7f9e28150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2e602640 1 --2- 172.21.15.26:0/4224644096 >> v2:172.21.15.117:3300/0 conn(0x7f9e281507d0 0x7f9e280c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2de01640 1 -- 172.21.15.26:0/4224644096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28156a00 msgr2=0x7f9e28150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2de01640 1 --2- 172.21.15.26:0/4224644096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28156a00 0x7f9e28150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2de01640 1 -- 172.21.15.26:0/4224644096 >> v2:172.21.15.117:3300/0 conn(0x7f9e281507d0 msgr2=0x7f9e280c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2de01640 1 --2- 172.21.15.26:0/4224644096 >> v2:172.21.15.117:3300/0 conn(0x7f9e281507d0 0x7f9e280c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2de01640 1 -- 172.21.15.26:0/4224644096 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e280c3280 con 0x7f9e28072720 2024-03-06T07:29:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.202+0000 7f9e2e602640 1 --2- 172.21.15.26:0/4224644096 >> v2:172.21.15.117:3300/0 conn(0x7f9e281507d0 0x7f9e280c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e2de01640 1 --2- 172.21.15.26:0/4224644096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e28072720 0x7f9e281564c0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009870 tx=0x7f9e2002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7953ae30e45780c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e2cdff640 1 -- 172.21.15.26:0/4224644096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20038070 con 0x7f9e28072720 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e2cdff640 1 -- 172.21.15.26:0/4224644096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e20031740 con 0x7f9e28072720 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e2cdff640 1 -- 172.21.15.26:0/4224644096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2002f050 con 0x7f9e28072720 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e30065640 1 -- 172.21.15.26:0/4224644096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e28072720 msgr2=0x7f9e281564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e30065640 1 --2- 172.21.15.26:0/4224644096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e28072720 0x7f9e281564c0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009870 tx=0x7f9e2002b7e0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e30065640 1 -- 172.21.15.26:0/4224644096 shutdown_connections 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e30065640 1 --2- 172.21.15.26:0/4224644096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28156a00 0x7f9e28150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e30065640 1 --2- 172.21.15.26:0/4224644096 >> v2:172.21.15.117:3300/0 conn(0x7f9e281507d0 0x7f9e280c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e30065640 1 --2- 172.21.15.26:0/4224644096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e28072720 0x7f9e281564c0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.203+0000 7f9e30065640 1 -- 172.21.15.26:0/4224644096 >> 172.21.15.26:0/4224644096 conn(0x7f9e280aa630 msgr2=0x7f9e280aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.204+0000 7f9e30065640 1 -- 172.21.15.26:0/4224644096 shutdown_connections 2024-03-06T07:29:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.204+0000 7f9e30065640 1 -- 172.21.15.26:0/4224644096 wait complete. 2024-03-06T07:29:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 Processor -- start 2024-03-06T07:29:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 -- start start 2024-03-06T07:29:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28072720 0x7f9e280c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e281507d0 0x7f9e281eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e28156a00 0x7f9e281eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28072b00 con 0x7f9e281507d0 2024-03-06T07:29:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2806f8f0 con 0x7f9e28156a00 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.205+0000 7f9e30065640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28072c80 con 0x7f9e28072720 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e281507d0 0x7f9e281eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e281507d0 0x7f9e281eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49642/0 (socket says 172.21.15.26:49642) 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 -- 172.21.15.26:0/2684700345 learned_addr learned my addr 172.21.15.26:0/2684700345 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2e602640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e28156a00 0x7f9e281eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2de01640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28072720 0x7f9e280c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 -- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28072720 msgr2=0x7f9e280c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28072720 0x7f9e280c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 -- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e28156a00 msgr2=0x7f9e281eb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e28156a00 0x7f9e281eb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 -- 172.21.15.26:0/2684700345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2806cf30 con 0x7f9e281507d0 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e2d600640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e281507d0 0x7f9e281eb0a0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00e9b0 tx=0x7f9e1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e267fc640 1 -- 172.21.15.26:0/2684700345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c017070 con 0x7f9e281507d0 2024-03-06T07:29:01.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e267fc640 1 -- 172.21.15.26:0/2684700345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00c9a0 con 0x7f9e281507d0 2024-03-06T07:29:01.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e267fc640 1 -- 172.21.15.26:0/2684700345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00cd90 con 0x7f9e281507d0 2024-03-06T07:29:01.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2806c0d0 con 0x7f9e281507d0 2024-03-06T07:29:01.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.206+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2806c690 con 0x7f9e281507d0 2024-03-06T07:29:01.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.207+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2804f330 con 0x7f9e281507d0 2024-03-06T07:29:01.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.208+0000 7f9e267fc640 1 -- 172.21.15.26:0/2684700345 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e1c01a070 con 0x7f9e281507d0 2024-03-06T07:29:01.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.208+0000 7f9e267fc640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e1406b500 0x7f9e1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.208+0000 7f9e267fc640 1 -- 172.21.15.26:0/2684700345 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9e1c08a810 con 0x7f9e281507d0 2024-03-06T07:29:01.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.208+0000 7f9e2de01640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e1406b500 0x7f9e1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.209+0000 7f9e2de01640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e1406b500 0x7f9e1406d9c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9e20035000 tx=0x7f9e20008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.212+0000 7f9e267fc640 1 -- 172.21.15.26:0/2684700345 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e1c059410 con 0x7f9e281507d0 2024-03-06T07:29:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.430+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e280c6080 con 0x7f9e281507d0 2024-03-06T07:29:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.431+0000 7f9e267fc640 1 -- 172.21.15.26:0/2684700345 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4977 (secure 0 0 0) 0x7f9e1c058d10 con 0x7f9e281507d0 2024-03-06T07:29:01.433 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:01.433 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":192,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2181907552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2181907552},{"type":"v1","addr":"172.21.15.117:6836","nonce":2181907552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":8129,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2243060002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2243060002},{"type":"v1","addr":"172.21.15.117:6837","nonce":2243060002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":9529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4068022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4068022212},{"type":"v1","addr":"172.21.15.26:6834","nonce":4068022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":10563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2089138655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2089138655},{"type":"v1","addr":"172.21.15.26:6835","nonce":2089138655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:15.142228+0000","modified":"2024-03-06T07:28:40.703192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:29:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 192 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e1406b500 msgr2=0x7f9e1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e1406b500 0x7f9e1406d9c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9e20035000 tx=0x7f9e20008030 comp rx=0 tx=0).stop 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e281507d0 msgr2=0x7f9e281eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e281507d0 0x7f9e281eb0a0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00e9b0 tx=0x7f9e1c00ee70 comp rx=0 tx=0).stop 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 shutdown_connections 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9e1406b500 0x7f9e1406d9c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e28156a00 0x7f9e281eb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e281507d0 0x7f9e281eb0a0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 --2- 172.21.15.26:0/2684700345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e28072720 0x7f9e280c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 >> 172.21.15.26:0/2684700345 conn(0x7f9e280aa630 msgr2=0x7f9e2814bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 shutdown_connections 2024-03-06T07:29:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.433+0000 7f9e30065640 1 -- 172.21.15.26:0/2684700345 wait complete. 2024-03-06T07:29:01.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.544+0000 7f2427916640 1 Processor -- start 2024-03-06T07:29:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.545+0000 7f2427916640 1 -- start start 2024-03-06T07:29:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.545+0000 7f2427916640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24200c5480 0x7f2420155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.545+0000 7f2427916640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24200c5cc0 con 0x7f24201586f0 2024-03-06T07:29:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.545+0000 7f2427916640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24201591c0 con 0x7f24200c5860 2024-03-06T07:29:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.545+0000 7f2427916640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242006df40 con 0x7f24200c5480 2024-03-06T07:29:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f2425eb3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f24201586f0 0x7f242015f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43566/0 (socket says 172.21.15.26:43566) 2024-03-06T07:29:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f2425eb3640 1 -- 172.21.15.26:0/3881826006 learned_addr learned my addr 172.21.15.26:0/3881826006 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f24256b2640 1 --2- 172.21.15.26:0/3881826006 >> v2:172.21.15.117:3300/0 conn(0x7f24200c5480 0x7f2420155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360710709 0 0) 0x7f24200c5cc0 con 0x7f24201586f0 2024-03-06T07:29:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2408003520 con 0x7f24201586f0 2024-03-06T07:29:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554898102 0 0) 0x7f24201591c0 con 0x7f24200c5860 2024-03-06T07:29:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24200c5cc0 con 0x7f24200c5860 2024-03-06T07:29:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1670613689 0 0) 0x7f2408003520 con 0x7f24201586f0 2024-03-06T07:29:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f24256b2640 1 -- 172.21.15.26:0/3881826006 >> v1:172.21.15.117:6789/0 conn(0x7f24200c5860 legacy=0x7f242006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f24256b2640 1 -- 172.21.15.26:0/3881826006 >> v1:172.21.15.26:6789/0 conn(0x7f24201586f0 legacy=0x7f242015f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.546+0000 7f24256b2640 1 -- 172.21.15.26:0/3881826006 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24201594a0 con 0x7f24200c5480 2024-03-06T07:29:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.547+0000 7f24256b2640 1 --2- 172.21.15.26:0/3881826006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200c5480 0x7f2420155ba0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6bf42155230b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.547+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418037070 con 0x7f24200c5480 2024-03-06T07:29:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.547+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418030510 con 0x7f24200c5480 2024-03-06T07:29:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.547+0000 7f241ffff640 1 -- 172.21.15.26:0/3881826006 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100019f0 con 0x7f24200c5480 2024-03-06T07:29:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.548+0000 7f2427916640 1 -- 172.21.15.26:0/3881826006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200c5480 msgr2=0x7f2420155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.548+0000 7f2427916640 1 --2- 172.21.15.26:0/3881826006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200c5480 0x7f2420155ba0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802b790 comp rx=0 tx=0).stop 2024-03-06T07:29:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.548+0000 7f2427916640 1 -- 172.21.15.26:0/3881826006 shutdown_connections 2024-03-06T07:29:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.548+0000 7f2427916640 1 --2- 172.21.15.26:0/3881826006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200c5480 0x7f2420155ba0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.548+0000 7f2427916640 1 -- 172.21.15.26:0/3881826006 >> 172.21.15.26:0/3881826006 conn(0x7f242006b1a0 msgr2=0x7f242006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.548+0000 7f2427916640 1 -- 172.21.15.26:0/3881826006 shutdown_connections 2024-03-06T07:29:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.548+0000 7f2427916640 1 -- 172.21.15.26:0/3881826006 wait complete. 2024-03-06T07:29:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 Processor -- start 2024-03-06T07:29:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 -- start start 2024-03-06T07:29:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c5480 0x7f2420153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24200c5860 0x7f24201e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24201586f0 0x7f24201e21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420155680 con 0x7f24200c5860 2024-03-06T07:29:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242006fdb0 con 0x7f24201586f0 2024-03-06T07:29:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.549+0000 7f2427916640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2420152a20 con 0x7f24200c5480 2024-03-06T07:29:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24200c5860 0x7f24201e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24200c5860 0x7f24201e1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49656/0 (socket says 172.21.15.26:49656) 2024-03-06T07:29:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 -- 172.21.15.26:0/4179578059 learned_addr learned my addr 172.21.15.26:0/4179578059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2425eb3640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24201586f0 0x7f24201e21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 -- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c5480 msgr2=0x7f2420153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c5480 0x7f2420153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 -- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24201586f0 msgr2=0x7f24201e21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24201586f0 0x7f24201e21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 -- 172.21.15.26:0/4179578059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24201ee5d0 con 0x7f24200c5860 2024-03-06T07:29:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2425eb3640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24201586f0 0x7f24201e21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2424eb1640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24200c5860 0x7f24201e1c60 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f241400c490 tx=0x7f241400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f241dffb640 1 -- 172.21.15.26:0/4179578059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140052d0 con 0x7f24200c5860 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f241dffb640 1 -- 172.21.15.26:0/4179578059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2414005470 con 0x7f24200c5860 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24201ee8b0 con 0x7f24200c5860 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f241dffb640 1 -- 172.21.15.26:0/4179578059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414012870 con 0x7f24200c5860 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.550+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24201eee70 con 0x7f24200c5860 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.551+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242004f330 con 0x7f24200c5860 2024-03-06T07:29:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.552+0000 7f241dffb640 1 -- 172.21.15.26:0/4179578059 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2414012a10 con 0x7f24200c5860 2024-03-06T07:29:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.552+0000 7f241dffb640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.552+0000 7f241dffb640 1 -- 172.21.15.26:0/4179578059 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f241408a020 con 0x7f24200c5860 2024-03-06T07:29:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.552+0000 7f24256b2640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.553+0000 7f24256b2640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2418034000 tx=0x7f241802d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.557+0000 7f241dffb640 1 -- 172.21.15.26:0/4179578059 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2414058d50 con 0x7f24200c5860 2024-03-06T07:29:01.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.736+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24200c2670 con 0x7f24200c5860 2024-03-06T07:29:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.737+0000 7f241dffb640 1 -- 172.21.15.26:0/4179578059 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17109 (secure 0 0 0) 0x7f2414058650 con 0x7f24200c5860 2024-03-06T07:29:01.739 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:01.739 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":200,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:51.890050+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.447072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:14.106103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.2","mappings":[{"from":6,"to":7}]},{"pgid":"32.20","mappings":[{"from":2,"to":0}]},{"pgid":"32.36","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f240c06b4d0 msgr2=0x7f240c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2418034000 tx=0x7f241802d000 comp rx=0 tx=0).stop 2024-03-06T07:29:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24200c5860 msgr2=0x7f24201e1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24200c5860 0x7f24201e1c60 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f241400c490 tx=0x7f241400c950 comp rx=0 tx=0).stop 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 shutdown_connections 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24201586f0 0x7f24201e21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24200c5860 0x7f24201e1c60 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 --2- 172.21.15.26:0/4179578059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c5480 0x7f2420153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 >> 172.21.15.26:0/4179578059 conn(0x7f242006b1a0 msgr2=0x7f242006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 shutdown_connections 2024-03-06T07:29:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.739+0000 7f2427916640 1 -- 172.21.15.26:0/4179578059 wait complete. 2024-03-06T07:29:01.752 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:29:01.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.852+0000 7f9cadf81640 1 Processor -- start 2024-03-06T07:29:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.852+0000 7f9cadf81640 1 -- start start 2024-03-06T07:29:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9cadf81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 0x7f9ca80c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9cadf81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8072630 con 0x7f9ca80c6780 2024-03-06T07:29:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9cadf81640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8072910 con 0x7f9ca806da40 2024-03-06T07:29:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9cadf81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca806de20 con 0x7f9ca80c7380 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9ca6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ca80c6780 0x7f9ca8158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43580/0 (socket says 172.21.15.26:43580) 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9ca6ffd640 1 -- 172.21.15.26:0/2257163187 learned_addr learned my addr 172.21.15.26:0/2257163187 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9ca7fff640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 0x7f9ca80c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803083436 0 0) 0x7f9ca8072630 con 0x7f9ca80c6780 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.853+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c90003520 con 0x7f9ca80c6780 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127120054 0 0) 0x7f9c90003520 con 0x7f9ca80c6780 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 msgr2=0x7f9ca80c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 0x7f9ca80c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 >> v1:172.21.15.117:6789/0 conn(0x7f9ca806da40 legacy=0x7f9ca8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8072bf0 con 0x7f9ca80c6780 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f9c9c001670 con 0x7f9ca80c6780 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 >> v1:172.21.15.26:6789/0 conn(0x7f9ca80c6780 legacy=0x7f9ca8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c900039e0 0x7f9c90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 0x7f9c90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c90004d40 0x7f9c900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca806de20 con 0x7f9c900039e0 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9000baa0 con 0x7f9c90004d40 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9000bca0 con 0x7f9ca80c7380 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca77fe640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c900039e0 0x7f9c90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca7fff640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c90004d40 0x7f9c900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca6ffd640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 0x7f9c90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca77fe640 1 -- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 msgr2=0x7f9c90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca77fe640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca80c7380 0x7f9c90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca77fe640 1 -- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c90004d40 msgr2=0x7f9c900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca77fe640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c90004d40 0x7f9c900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.854+0000 7f9ca77fe640 1 -- 172.21.15.26:0/2257163187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c90003520 con 0x7f9c900039e0 2024-03-06T07:29:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.855+0000 7f9ca77fe640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c900039e0 0x7f9c90003dc0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f9c98001860 tx=0x7f9c90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834fc8c748d030c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.855+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9802f050 con 0x7f9c900039e0 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.855+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98033650 con 0x7f9c900039e0 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.855+0000 7f9ca67fc640 1 -- 172.21.15.26:0/2257163187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98001430 con 0x7f9c900039e0 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.855+0000 7f9cadf81640 1 -- 172.21.15.26:0/2257163187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c900039e0 msgr2=0x7f9c90003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.855+0000 7f9cadf81640 1 --2- 172.21.15.26:0/2257163187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c900039e0 0x7f9c90003dc0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f9c98001860 tx=0x7f9c90005830 comp rx=0 tx=0).stop 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.855+0000 7f9ca7fff640 1 -- 172.21.15.26:0/2257163187 reap_dead start 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.856+0000 7f9cadf81640 1 -- 172.21.15.26:0/2257163187 shutdown_connections 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.856+0000 7f9cadf81640 1 -- 172.21.15.26:0/2257163187 >> 172.21.15.26:0/2257163187 conn(0x7f9ca80aa630 msgr2=0x7f9ca80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.856+0000 7f9cadf81640 1 -- 172.21.15.26:0/2257163187 shutdown_connections 2024-03-06T07:29:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.856+0000 7f9cadf81640 1 -- 172.21.15.26:0/2257163187 wait complete. 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 Processor -- start 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 -- start start 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca8157630 0x7f9ca814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80ad900 0x7f9ca8150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca81ee010 0x7f9ca81ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80c3280 con 0x7f9ca80ad900 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca81594a0 con 0x7f9ca81ee010 2024-03-06T07:29:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.857+0000 7f9cadf81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80c3400 con 0x7f9ca8157630 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80ad900 0x7f9ca8150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80ad900 0x7f9ca8150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49676/0 (socket says 172.21.15.26:49676) 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 -- 172.21.15.26:0/4065373525 learned_addr learned my addr 172.21.15.26:0/4065373525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca77fe640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca8157630 0x7f9ca814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca7fff640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca81ee010 0x7f9ca81ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 -- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca8157630 msgr2=0x7f9ca814fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca8157630 0x7f9ca814fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 -- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca81ee010 msgr2=0x7f9ca81ee4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca81ee010 0x7f9ca81ee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 -- 172.21.15.26:0/4065373525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80ac260 con 0x7f9ca80ad900 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9ca6ffd640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80ad900 0x7f9ca8150330 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c007060 tx=0x7f9c9c007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9c8ffff640 1 -- 172.21.15.26:0/4065373525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c01a070 con 0x7f9ca80ad900 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9c8ffff640 1 -- 172.21.15.26:0/4065373525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00f9c0 con 0x7f9ca80ad900 2024-03-06T07:29:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9c8ffff640 1 -- 172.21.15.26:0/4065373525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00fdb0 con 0x7f9ca80ad900 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca81eb520 con 0x7f9ca80ad900 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.858+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca814f8d0 con 0x7f9ca80ad900 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.859+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca814fad0 con 0x7f9ca80ad900 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.860+0000 7f9c8ffff640 1 -- 172.21.15.26:0/4065373525 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c9c01d070 con 0x7f9ca80ad900 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.860+0000 7f9c8ffff640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c9406b4d0 0x7f9c9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.860+0000 7f9c8ffff640 1 -- 172.21.15.26:0/4065373525 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9c9c089d90 con 0x7f9ca80ad900 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.860+0000 7f9ca77fe640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c9406b4d0 0x7f9c9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.861+0000 7f9ca77fe640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c9406b4d0 0x7f9c9406d990 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9c98036000 tx=0x7f9c98031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:01.864+0000 7f9c8ffff640 1 -- 172.21.15.26:0/4065373525 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ca814fad0 con 0x7f9ca80ad900 2024-03-06T07:29:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.082+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ca804f330 con 0x7f9ca80ad900 2024-03-06T07:29:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.082+0000 7f9c8ffff640 1 -- 172.21.15.26:0/4065373525 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4977 (secure 0 0 0) 0x7f9c9c058a30 con 0x7f9ca80ad900 2024-03-06T07:29:02.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:02.084 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":192,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2181907552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2181907552},{"type":"v1","addr":"172.21.15.117:6836","nonce":2181907552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":8129,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2243060002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2243060002},{"type":"v1","addr":"172.21.15.117:6837","nonce":2243060002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":9529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4068022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4068022212},{"type":"v1","addr":"172.21.15.26:6834","nonce":4068022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":10563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2089138655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2089138655},{"type":"v1","addr":"172.21.15.26:6835","nonce":2089138655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:15.142228+0000","modified":"2024-03-06T07:28:40.703192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:29:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 192 2024-03-06T07:29:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c9406b4d0 msgr2=0x7f9c9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c9406b4d0 0x7f9c9406d990 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9c98036000 tx=0x7f9c98031000 comp rx=0 tx=0).stop 2024-03-06T07:29:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80ad900 msgr2=0x7f9ca8150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80ad900 0x7f9ca8150330 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c007060 tx=0x7f9c9c007520 comp rx=0 tx=0).stop 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 shutdown_connections 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9c9406b4d0 0x7f9c9406d990 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca81ee010 0x7f9ca81ee4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80ad900 0x7f9ca8150330 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 --2- 172.21.15.26:0/4065373525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca8157630 0x7f9ca814fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.084+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 >> 172.21.15.26:0/4065373525 conn(0x7f9ca80aa630 msgr2=0x7f9ca8152ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.085+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 shutdown_connections 2024-03-06T07:29:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.085+0000 7f9cadf81640 1 -- 172.21.15.26:0/4065373525 wait complete. 2024-03-06T07:29:02.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.197+0000 7f70efb5e640 1 Processor -- start 2024-03-06T07:29:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.197+0000 7f70efb5e640 1 -- start start 2024-03-06T07:29:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70efb5e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f70e80c5330 0x7f70e80c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70efb5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e8159630 con 0x7f70e80c8a00 2024-03-06T07:29:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70efb5e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e80c7760 con 0x7f70e8152920 2024-03-06T07:29:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70efb5e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e81587b0 con 0x7f70e80c5330 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ed0f9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70e80c8a00 0x7f70e8158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43594/0 (socket says 172.21.15.26:43594) 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ed0f9640 1 -- 172.21.15.26:0/2947073708 learned_addr learned my addr 172.21.15.26:0/2947073708 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ee0fb640 1 --2- 172.21.15.26:0/2947073708 >> v2:172.21.15.117:3300/0 conn(0x7f70e80c5330 0x7f70e80c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542176361 0 0) 0x7f70e8159630 con 0x7f70e80c8a00 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70d4003520 con 0x7f70e80c8a00 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216142891 0 0) 0x7f70d4003520 con 0x7f70e80c8a00 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 >> v2:172.21.15.117:3300/0 conn(0x7f70e80c5330 msgr2=0x7f70e80c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 --2- 172.21.15.26:0/2947073708 >> v2:172.21.15.117:3300/0 conn(0x7f70e80c5330 0x7f70e80c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 >> v1:172.21.15.117:6789/0 conn(0x7f70e8152920 legacy=0x7f70e8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e80c7a40 con 0x7f70e80c8a00 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f70d8001670 con 0x7f70e80c8a00 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 >> v1:172.21.15.26:6789/0 conn(0x7f70e80c8a00 legacy=0x7f70e8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40039b0 0x7f70d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70d4004800 0x7f70d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d40051c0 0x7f70d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e81587b0 con 0x7f70d40051c0 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d400c160 con 0x7f70d40039b0 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d400c360 con 0x7f70d4004800 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.198+0000 7f70ee0fb640 1 --2- 172.21.15.26:0/2947073708 >> v2:172.21.15.117:3300/0 conn(0x7f70e80c5330 0x7f70e80c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ee0fb640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d40051c0 0x7f70d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ed8fa640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40039b0 0x7f70d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ed0f9640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70d4004800 0x7f70d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ee0fb640 1 -- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70d4004800 msgr2=0x7f70d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ee0fb640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70d4004800 0x7f70d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ee0fb640 1 -- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40039b0 msgr2=0x7f70d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ee0fb640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40039b0 0x7f70d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ee0fb640 1 -- 172.21.15.26:0/2947073708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4003520 con 0x7f70d40051c0 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.199+0000 7f70ee0fb640 1 -- 172.21.15.26:0/2947073708 reap_dead start 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70ee0fb640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d40051c0 0x7f70d400b9c0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f70e000e350 tx=0x7f70d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93ed908153d95b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0009dc0 con 0x7f70d40051c0 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e002ce00 con 0x7f70d40051c0 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70ec8f8640 1 -- 172.21.15.26:0/2947073708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0003bd0 con 0x7f70d40051c0 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70efb5e640 1 -- 172.21.15.26:0/2947073708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d40051c0 msgr2=0x7f70d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d40051c0 0x7f70d400b9c0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f70e000e350 tx=0x7f70d4005830 comp rx=0 tx=0).stop 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70efb5e640 1 -- 172.21.15.26:0/2947073708 shutdown_connections 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2947073708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d40051c0 0x7f70d400b9c0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70efb5e640 1 -- 172.21.15.26:0/2947073708 >> 172.21.15.26:0/2947073708 conn(0x7f70e80ac560 msgr2=0x7f70e80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70efb5e640 1 -- 172.21.15.26:0/2947073708 shutdown_connections 2024-03-06T07:29:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.200+0000 7f70efb5e640 1 -- 172.21.15.26:0/2947073708 wait complete. 2024-03-06T07:29:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.201+0000 7f70efb5e640 1 Processor -- start 2024-03-06T07:29:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.201+0000 7f70efb5e640 1 -- start start 2024-03-06T07:29:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70efb5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40051c0 0x7f70e8149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70efb5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e8156fc0 0x7f70e8146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70efb5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e8147e50 0x7f70e814cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70efb5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e806d030 con 0x7f70e8156fc0 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70efb5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e8158a00 con 0x7f70d40051c0 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70efb5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80c7390 con 0x7f70e8147e50 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e8156fc0 0x7f70e8146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e8156fc0 0x7f70e8146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49688/0 (socket says 172.21.15.26:49688) 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 -- 172.21.15.26:0/2335123820 learned_addr learned my addr 172.21.15.26:0/2335123820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 -- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e8147e50 msgr2=0x7f70e814cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e8147e50 0x7f70e814cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 -- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40051c0 msgr2=0x7f70e8149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40051c0 0x7f70e8149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.202+0000 7f70ed0f9640 1 -- 172.21.15.26:0/2335123820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e81461d0 con 0x7f70e8156fc0 2024-03-06T07:29:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.203+0000 7f70ed0f9640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e8156fc0 0x7f70e8146680 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f70d8007290 tx=0x7f70d8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.203+0000 7f70de7fc640 1 -- 172.21.15.26:0/2335123820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8007e10 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.203+0000 7f70de7fc640 1 -- 172.21.15.26:0/2335123820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8018070 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.203+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e8145b80 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.203+0000 7f70de7fc640 1 -- 172.21.15.26:0/2335123820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800fc00 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.203+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e814a3f0 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.204+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e8151390 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.204+0000 7f70de7fc640 1 -- 172.21.15.26:0/2335123820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70d801d460 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.205+0000 7f70de7fc640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70cc06b4d0 0x7f70cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.205+0000 7f70de7fc640 1 -- 172.21.15.26:0/2335123820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f70d808a410 con 0x7f70e8156fc0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.205+0000 7f70ed8fa640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70cc06b4d0 0x7f70cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.205+0000 7f70ed8fa640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70cc06b4d0 0x7f70cc06d990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000c00 tx=0x7f70e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.209+0000 7f70de7fc640 1 -- 172.21.15.26:0/2335123820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70e8151390 con 0x7f70e8156fc0 2024-03-06T07:29:02.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.428+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70e804f330 con 0x7f70e8156fc0 2024-03-06T07:29:02.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.429+0000 7f70de7fc640 1 -- 172.21.15.26:0/2335123820 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4977 (secure 0 0 0) 0x7f70d80590b0 con 0x7f70e8156fc0 2024-03-06T07:29:02.430 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:02.431 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":192,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2181907552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2181907552},{"type":"v1","addr":"172.21.15.117:6836","nonce":2181907552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":8129,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2243060002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2243060002},{"type":"v1","addr":"172.21.15.117:6837","nonce":2243060002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":9529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4068022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4068022212},{"type":"v1","addr":"172.21.15.26:6834","nonce":4068022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":10563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2089138655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2089138655},{"type":"v1","addr":"172.21.15.26:6835","nonce":2089138655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:15.142228+0000","modified":"2024-03-06T07:28:40.703192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:29:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 192 2024-03-06T07:29:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.430+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70cc06b4d0 msgr2=0x7f70cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.430+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70cc06b4d0 0x7f70cc06d990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000c00 tx=0x7f70e4009040 comp rx=0 tx=0).stop 2024-03-06T07:29:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.430+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e8156fc0 msgr2=0x7f70e8146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.430+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e8156fc0 0x7f70e8146680 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f70d8007290 tx=0x7f70d8007750 comp rx=0 tx=0).stop 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 shutdown_connections 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f70cc06b4d0 0x7f70cc06d990 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70e8147e50 0x7f70e814cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e8156fc0 0x7f70e8146680 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 --2- 172.21.15.26:0/2335123820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70d40051c0 0x7f70e8149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 >> 172.21.15.26:0/2335123820 conn(0x7f70e80ac560 msgr2=0x7f70e80caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 shutdown_connections 2024-03-06T07:29:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.431+0000 7f70efb5e640 1 -- 172.21.15.26:0/2335123820 wait complete. 2024-03-06T07:29:02.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.542+0000 7feae400e640 1 Processor -- start 2024-03-06T07:29:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.542+0000 7feae400e640 1 -- start start 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae400e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feadc0c88c0 0x7feadc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae400e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 0x7feadc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae400e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c73d0 con 0x7feadc14f360 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae400e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc0c8dd0 con 0x7feadc0c88c0 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae400e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0be2e0 con 0x7feadc158b60 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae1daa640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feadc14f360 0x7feadc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43596/0 (socket says 172.21.15.26:43596) 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae1daa640 1 -- 172.21.15.26:0/2956395992 learned_addr learned my addr 172.21.15.26:0/2956395992 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077539651 0 0) 0x7feadc0c73d0 con 0x7feadc14f360 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac4003520 con 0x7feadc14f360 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.543+0000 7feae25ab640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 0x7feadc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae15a9640 1 --2- 172.21.15.26:0/2956395992 >> v2:172.21.15.117:3300/0 conn(0x7feadc0c88c0 0x7feadc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921558587 0 0) 0x7feac4003520 con 0x7feadc14f360 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 msgr2=0x7feadc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 0x7feadc158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 >> v2:172.21.15.117:3300/0 conn(0x7feadc0c88c0 msgr2=0x7feadc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 --2- 172.21.15.26:0/2956395992 >> v2:172.21.15.117:3300/0 conn(0x7feadc0c88c0 0x7feadc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc0c7670 con 0x7feadc14f360 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7feacc0019f0 con 0x7feadc14f360 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 >> v1:172.21.15.26:6789/0 conn(0x7feadc14f360 legacy=0x7feadc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac40039b0 0x7feac4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 0x7feac4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac4004d40 0x7feac40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0be2e0 con 0x7feac40039b0 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac400b9c0 con 0x7feac4004d40 2024-03-06T07:29:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac400bb70 con 0x7feadc158b60 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae1daa640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac40039b0 0x7feac4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae15a9640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 0x7feac4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae25ab640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac4004d40 0x7feac40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae1daa640 1 -- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 msgr2=0x7feac4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae1daa640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc158b60 0x7feac4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae1daa640 1 -- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac4004d40 msgr2=0x7feac40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae1daa640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac4004d40 0x7feac40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.544+0000 7feae1daa640 1 -- 172.21.15.26:0/2956395992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4003520 con 0x7feac40039b0 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.545+0000 7feae1daa640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac40039b0 0x7feac4003dd0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7feacc0013d0 tx=0x7feac4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4f6ede48b79ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.545+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00a4e0 con 0x7feac40039b0 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.545+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc034650 con 0x7feac40039b0 2024-03-06T07:29:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.545+0000 7feae0da8640 1 -- 172.21.15.26:0/2956395992 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc034a40 con 0x7feac40039b0 2024-03-06T07:29:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.546+0000 7feae400e640 1 -- 172.21.15.26:0/2956395992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac40039b0 msgr2=0x7feac4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.546+0000 7feae400e640 1 --2- 172.21.15.26:0/2956395992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac40039b0 0x7feac4003dd0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7feacc0013d0 tx=0x7feac4005830 comp rx=0 tx=0).stop 2024-03-06T07:29:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.546+0000 7feae25ab640 1 -- 172.21.15.26:0/2956395992 reap_dead start 2024-03-06T07:29:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.546+0000 7feae400e640 1 -- 172.21.15.26:0/2956395992 shutdown_connections 2024-03-06T07:29:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.546+0000 7feae400e640 1 -- 172.21.15.26:0/2956395992 >> 172.21.15.26:0/2956395992 conn(0x7feadc0ac570 msgr2=0x7feadc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.546+0000 7feae400e640 1 -- 172.21.15.26:0/2956395992 shutdown_connections 2024-03-06T07:29:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.546+0000 7feae400e640 1 -- 172.21.15.26:0/2956395992 wait complete. 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.547+0000 7feae400e640 1 Processor -- start 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.547+0000 7feae400e640 1 -- start start 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.547+0000 7feae400e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feadc14b790 0x7feadc1ead90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.547+0000 7feae400e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feadc1e8db0 0x7feadc14bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.547+0000 7feae400e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc0bf5f0 0x7feadc0bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae400e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0c5a50 con 0x7feadc1e8db0 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae400e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0c7c30 con 0x7feadc14b790 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae400e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0c5bd0 con 0x7feadc0bf5f0 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feadc1e8db0 0x7feadc14bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feadc1e8db0 0x7feadc14bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49702/0 (socket says 172.21.15.26:49702) 2024-03-06T07:29:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 -- 172.21.15.26:0/3207932398 learned_addr learned my addr 172.21.15.26:0/3207932398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 -- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc0bf5f0 msgr2=0x7feadc0bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc0bf5f0 0x7feadc0bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 -- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feadc14b790 msgr2=0x7feadc1ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feadc14b790 0x7feadc1ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 -- 172.21.15.26:0/3207932398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc0be590 con 0x7feadc1e8db0 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae15a9640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feadc1e8db0 0x7feadc14bb70 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fead0011980 tx=0x7fead0011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feada7fc640 1 -- 172.21.15.26:0/3207932398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0019070 con 0x7feadc1e8db0 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feada7fc640 1 -- 172.21.15.26:0/3207932398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead0004240 con 0x7feadc1e8db0 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feadc0be830 con 0x7feadc1e8db0 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feada7fc640 1 -- 172.21.15.26:0/3207932398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000f800 con 0x7feadc1e8db0 2024-03-06T07:29:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.548+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc0bedf0 con 0x7feadc1e8db0 2024-03-06T07:29:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.549+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc04f330 con 0x7feadc1e8db0 2024-03-06T07:29:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.550+0000 7feada7fc640 1 -- 172.21.15.26:0/3207932398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fead001c070 con 0x7feadc1e8db0 2024-03-06T07:29:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.550+0000 7feada7fc640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feac806b500 0x7feac806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.550+0000 7feada7fc640 1 -- 172.21.15.26:0/3207932398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fead008a700 con 0x7feadc1e8db0 2024-03-06T07:29:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.550+0000 7feae1daa640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feac806b500 0x7feac806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.551+0000 7feae1daa640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feac806b500 0x7feac806d9c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7feacc034000 tx=0x7feacc007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.554+0000 7feada7fc640 1 -- 172.21.15.26:0/3207932398 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fead0059430 con 0x7feadc1e8db0 2024-03-06T07:29:02.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.736+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feadc06cda0 con 0x7feadc1e8db0 2024-03-06T07:29:02.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.737+0000 7feada7fc640 1 -- 172.21.15.26:0/3207932398 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17109 (secure 0 0 0) 0x7fead0058d30 con 0x7feadc1e8db0 2024-03-06T07:29:02.739 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:02.739 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":200,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:28:51.890050+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.447072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:14.106103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.2","mappings":[{"from":6,"to":7}]},{"pgid":"32.20","mappings":[{"from":2,"to":0}]},{"pgid":"32.36","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feac806b500 msgr2=0x7feac806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feac806b500 0x7feac806d9c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7feacc034000 tx=0x7feacc007000 comp rx=0 tx=0).stop 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feadc1e8db0 msgr2=0x7feadc14bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feadc1e8db0 0x7feadc14bb70 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fead0011980 tx=0x7fead0011e40 comp rx=0 tx=0).stop 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 shutdown_connections 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feac806b500 0x7feac806d9c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feadc0bf5f0 0x7feadc0bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feadc1e8db0 0x7feadc14bb70 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 --2- 172.21.15.26:0/3207932398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feadc14b790 0x7feadc1ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 >> 172.21.15.26:0/3207932398 conn(0x7feadc0ac570 msgr2=0x7feadc14e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.739+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 shutdown_connections 2024-03-06T07:29:02.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.740+0000 7feae400e640 1 -- 172.21.15.26:0/3207932398 wait complete. 2024-03-06T07:29:02.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:29:02.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 Processor -- start 2024-03-06T07:29:02.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 -- start start 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe0152720 0x7f7fe01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7fe006f550 0x7f7fe00c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe00c7d20 0x7f7fe006fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe0155ca0 con 0x7f7fe00c7d20 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe01561d0 con 0x7f7fe006f550 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.854+0000 7f7fe83e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00733f0 con 0x7f7fe0152720 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe00c7d20 0x7f7fe006fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe00c7d20 0x7f7fe006fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49704/0 (socket says 172.21.15.26:49704) 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 -- 172.21.15.26:0/2459089707 learned_addr learned my addr 172.21.15.26:0/2459089707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6185640 1 --2- 172.21.15.26:0/2459089707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe0152720 0x7f7fe01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 -- 172.21.15.26:0/2459089707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe0152720 msgr2=0x7f7fe01593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 --2- 172.21.15.26:0/2459089707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe0152720 0x7f7fe01593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 -- 172.21.15.26:0/2459089707 >> v2:172.21.15.117:3300/0 conn(0x7f7fe006f550 msgr2=0x7f7fe00c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 --2- 172.21.15.26:0/2459089707 >> v2:172.21.15.117:3300/0 conn(0x7f7fe006f550 0x7f7fe00c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.855+0000 7f7fe6986640 1 -- 172.21.15.26:0/2459089707 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0155ef0 con 0x7f7fe00c7d20 2024-03-06T07:29:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe6986640 1 --2- 172.21.15.26:0/2459089707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe00c7d20 0x7f7fe006fd90 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f7fe006b9e0 tx=0x7f7fd802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b01c3ad75a61f12 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe5183640 1 -- 172.21.15.26:0/2459089707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd802c9e0 con 0x7f7fe00c7d20 2024-03-06T07:29:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe5183640 1 -- 172.21.15.26:0/2459089707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd802cb80 con 0x7f7fe00c7d20 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe5183640 1 -- 172.21.15.26:0/2459089707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd80343f0 con 0x7f7fe00c7d20 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 -- 172.21.15.26:0/2459089707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe00c7d20 msgr2=0x7f7fe006fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/2459089707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe00c7d20 0x7f7fe006fd90 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f7fe006b9e0 tx=0x7f7fd802be40 comp rx=0 tx=0).stop 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 -- 172.21.15.26:0/2459089707 shutdown_connections 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/2459089707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe00c7d20 0x7f7fe006fd90 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/2459089707 >> v2:172.21.15.117:3300/0 conn(0x7f7fe006f550 0x7f7fe00c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/2459089707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe0152720 0x7f7fe01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 -- 172.21.15.26:0/2459089707 >> 172.21.15.26:0/2459089707 conn(0x7f7fe006ea40 msgr2=0x7f7fe006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 -- 172.21.15.26:0/2459089707 shutdown_connections 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.856+0000 7f7fe83e9640 1 -- 172.21.15.26:0/2459089707 wait complete. 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.857+0000 7f7fe83e9640 1 Processor -- start 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe83e9640 1 -- start start 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe83e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe006f550 0x7f7fe006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe83e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c7d20 0x7f7fe006da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe83e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe0152720 0x7f7fe014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe83e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0158ea0 con 0x7f7fe0152720 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe83e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0158bc0 con 0x7f7fe006f550 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe83e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0152b00 con 0x7f7fe00c7d20 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe6986640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe0152720 0x7f7fe014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe6986640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe0152720 0x7f7fe014aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49720/0 (socket says 172.21.15.26:49720) 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe6986640 1 -- 172.21.15.26:0/209183998 learned_addr learned my addr 172.21.15.26:0/209183998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe5984640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c7d20 0x7f7fe006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.858+0000 7f7fe6185640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe006f550 0x7f7fe006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe6986640 1 -- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c7d20 msgr2=0x7f7fe006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe6986640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c7d20 0x7f7fe006da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe6986640 1 -- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe006f550 msgr2=0x7f7fe006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe6986640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe006f550 0x7f7fe006d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe6986640 1 -- 172.21.15.26:0/209183998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe014c000 con 0x7f7fe0152720 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe5984640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c7d20 0x7f7fe006da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe6185640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe006f550 0x7f7fe006d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe6986640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe0152720 0x7f7fe014aed0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f7fd802c450 tx=0x7f7fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fd6ffd640 1 -- 172.21.15.26:0/209183998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8043020 con 0x7f7fe0152720 2024-03-06T07:29:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fd6ffd640 1 -- 172.21.15.26:0/209183998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd803e450 con 0x7f7fe0152720 2024-03-06T07:29:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe00c8990 con 0x7f7fe0152720 2024-03-06T07:29:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fd6ffd640 1 -- 172.21.15.26:0/209183998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8009510 con 0x7f7fe0152720 2024-03-06T07:29:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.859+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe00c5900 con 0x7f7fe0152720 2024-03-06T07:29:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.860+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe004f330 con 0x7f7fe0152720 2024-03-06T07:29:02.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.860+0000 7f7fd6ffd640 1 -- 172.21.15.26:0/209183998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fd8046020 con 0x7f7fe0152720 2024-03-06T07:29:02.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.861+0000 7f7fd6ffd640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc406b4d0 0x7f7fc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.861+0000 7f7fd6ffd640 1 -- 172.21.15.26:0/209183998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f7fd80accc0 con 0x7f7fe0152720 2024-03-06T07:29:02.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.861+0000 7f7fe6185640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc406b4d0 0x7f7fc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.862+0000 7f7fe6185640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc406b4d0 0x7f7fc406d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006fd0 tx=0x7f7fdc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:02.865+0000 7f7fd6ffd640 1 -- 172.21.15.26:0/209183998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7fd804d030 con 0x7f7fe0152720 2024-03-06T07:29:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.085+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7fe0072ab0 con 0x7f7fe0152720 2024-03-06T07:29:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.930+0000 7f7fd6ffd640 1 -- 172.21.15.26:0/209183998 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v193) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7fd807b9f0 con 0x7f7fe0152720 2024-03-06T07:29:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:29:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc406b4d0 msgr2=0x7f7fc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc406b4d0 0x7f7fc406d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006fd0 tx=0x7f7fdc008040 comp rx=0 tx=0).stop 2024-03-06T07:29:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe0152720 msgr2=0x7f7fe014aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe0152720 0x7f7fe014aed0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f7fd802c450 tx=0x7f7fd8004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 shutdown_connections 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7fc406b4d0 0x7f7fc406d990 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe0152720 0x7f7fe014aed0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fe00c7d20 0x7f7fe006da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 --2- 172.21.15.26:0/209183998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe006f550 0x7f7fe006d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.933+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 >> 172.21.15.26:0/209183998 conn(0x7f7fe006ea40 msgr2=0x7f7fe006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.934+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 shutdown_connections 2024-03-06T07:29:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:03.934+0000 7f7fe83e9640 1 -- 172.21.15.26:0/209183998 wait complete. 2024-03-06T07:29:03.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:29:04.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.056+0000 7fd5c590c640 1 Processor -- start 2024-03-06T07:29:04.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.056+0000 7fd5c590c640 1 -- start start 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5c590c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5c01527f0 0x7fd5c00c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5c590c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c88f0 0x7fd5c0155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5c590c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c01596f0 con 0x7fd5c00c58b0 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5c590c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00c9130 con 0x7fd5c01527f0 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5c590c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0159260 con 0x7fd5c00c88f0 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5beffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5c01527f0 0x7fd5c00c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5bf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5c00c58b0 0x7fd5c0155750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43610/0 (socket says 172.21.15.26:43610) 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.057+0000 7fd5bf7fe640 1 -- 172.21.15.26:0/1297309562 learned_addr learned my addr 172.21.15.26:0/1297309562 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:04.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5be7fc640 1 --2- 172.21.15.26:0/1297309562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c88f0 0x7fd5c0155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5beffd640 1 -- 172.21.15.26:0/1297309562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c88f0 msgr2=0x7fd5c0155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5beffd640 1 --2- 172.21.15.26:0/1297309562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c88f0 0x7fd5c0155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5beffd640 1 -- 172.21.15.26:0/1297309562 >> v1:172.21.15.26:6789/0 conn(0x7fd5c00c58b0 legacy=0x7fd5c0155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5bdffb640 1 -- 172.21.15.26:0/1297309562 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253027411 0 0) 0x7fd5c01596f0 con 0x7fd5c00c58b0 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5beffd640 1 -- 172.21.15.26:0/1297309562 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00c82a0 con 0x7fd5c01527f0 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5be7fc640 1 --2- 172.21.15.26:0/1297309562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c88f0 0x7fd5c0155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5beffd640 1 --2- 172.21.15.26:0/1297309562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c00c5370 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80097a0 tx=0x7fd5b802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fbfbb4baa66f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.058+0000 7fd5bdffb640 1 -- 172.21.15.26:0/1297309562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8039070 con 0x7fd5c01527f0 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5bdffb640 1 -- 172.21.15.26:0/1297309562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b80316f0 con 0x7fd5c01527f0 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5bdffb640 1 -- 172.21.15.26:0/1297309562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8031ac0 con 0x7fd5c01527f0 2024-03-06T07:29:04.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 -- 172.21.15.26:0/1297309562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 msgr2=0x7fd5c00c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:04.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 --2- 172.21.15.26:0/1297309562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c00c5370 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80097a0 tx=0x7fd5b802b790 comp rx=0 tx=0).stop 2024-03-06T07:29:04.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 -- 172.21.15.26:0/1297309562 shutdown_connections 2024-03-06T07:29:04.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 --2- 172.21.15.26:0/1297309562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c88f0 0x7fd5c0155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 --2- 172.21.15.26:0/1297309562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c00c5370 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 -- 172.21.15.26:0/1297309562 >> 172.21.15.26:0/1297309562 conn(0x7fd5c00ac580 msgr2=0x7fd5c00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:04.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 -- 172.21.15.26:0/1297309562 shutdown_connections 2024-03-06T07:29:04.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.059+0000 7fd5c590c640 1 -- 172.21.15.26:0/1297309562 wait complete. 2024-03-06T07:29:04.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.060+0000 7fd5c590c640 1 Processor -- start 2024-03-06T07:29:04.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.060+0000 7fd5c590c640 1 -- start start 2024-03-06T07:29:04.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5c590c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c58b0 0x7fd5c01494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:04.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5c590c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c00c88f0 0x7fd5c014af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5c590c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c01566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5c590c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0158c10 con 0x7fd5c01527f0 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5c590c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00c6b90 con 0x7fd5c00c88f0 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5c590c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0158ef0 con 0x7fd5c00c58b0 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c01566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49734/0 (socket says 172.21.15.26:49734) 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 -- 172.21.15.26:0/151390710 learned_addr learned my addr 172.21.15.26:0/151390710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5be7fc640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c00c88f0 0x7fd5c014af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5beffd640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c58b0 0x7fd5c01494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 -- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c58b0 msgr2=0x7fd5c01494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c58b0 0x7fd5c01494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 -- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c00c88f0 msgr2=0x7fd5c014af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c00c88f0 0x7fd5c014af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.061+0000 7fd5bf7fe640 1 -- 172.21.15.26:0/151390710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c01ee1d0 con 0x7fd5c01527f0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.062+0000 7fd5bf7fe640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c01566e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000c730 tx=0x7fd5b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.062+0000 7fd5b7fff640 1 -- 172.21.15.26:0/151390710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0018070 con 0x7fd5c01527f0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.062+0000 7fd5b7fff640 1 -- 172.21.15.26:0/151390710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b000ce40 con 0x7fd5c01527f0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.062+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c01ee4b0 con 0x7fd5c01527f0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.062+0000 7fd5b7fff640 1 -- 172.21.15.26:0/151390710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000a860 con 0x7fd5c01527f0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.062+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c01eea70 con 0x7fd5c01527f0 2024-03-06T07:29:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.063+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c004f330 con 0x7fd5c01527f0 2024-03-06T07:29:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.063+0000 7fd5b7fff640 1 -- 172.21.15.26:0/151390710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5b001b070 con 0x7fd5c01527f0 2024-03-06T07:29:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.064+0000 7fd5b7fff640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd5a006b500 0x7fd5a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.064+0000 7fd5b7fff640 1 -- 172.21.15.26:0/151390710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd5b00898d0 con 0x7fd5c01527f0 2024-03-06T07:29:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.064+0000 7fd5beffd640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd5a006b500 0x7fd5a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.065+0000 7fd5beffd640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd5a006b500 0x7fd5a006d9c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8036000 tx=0x7fd5b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.068+0000 7fd5b7fff640 1 -- 172.21.15.26:0/151390710 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5b0058600 con 0x7fd5c01527f0 2024-03-06T07:29:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.287+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5c014c640 con 0x7fd5c01527f0 2024-03-06T07:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.932+0000 7fd5b7fff640 1 -- 172.21.15.26:0/151390710 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v194) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd5b0057f00 con 0x7fd5c01527f0 2024-03-06T07:29:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd5a006b500 msgr2=0x7fd5a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd5a006b500 0x7fd5a006d9c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8036000 tx=0x7fd5b8008030 comp rx=0 tx=0).stop 2024-03-06T07:29:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 msgr2=0x7fd5c01566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c01566e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000c730 tx=0x7fd5b000cbf0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 shutdown_connections 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd5a006b500 0x7fd5a006d9c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5c01527f0 0x7fd5c01566e0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c00c88f0 0x7fd5c014af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 --2- 172.21.15.26:0/151390710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00c58b0 0x7fd5c01494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.935+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 >> 172.21.15.26:0/151390710 conn(0x7fd5c00ac580 msgr2=0x7fd5c014f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.936+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 shutdown_connections 2024-03-06T07:29:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:04.936+0000 7fd5c590c640 1 -- 172.21.15.26:0/151390710 wait complete. 2024-03-06T07:29:04.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:29:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.061+0000 7f372a1c9640 1 Processor -- start 2024-03-06T07:29:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.061+0000 7f372a1c9640 1 -- start start 2024-03-06T07:29:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f372a1c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 0x7f3724072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f372a1c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3724072a50 0x7f3724158780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f372a1c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240a7d70 con 0x7f372406fe20 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f372a1c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37240a8050 con 0x7f3724072a50 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f372a1c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37240a8330 con 0x7f37240c6780 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f37237fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f372406fe20 0x7f3724155c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43612/0 (socket says 172.21.15.26:43612) 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f37237fe640 1 -- 172.21.15.26:0/2502679959 learned_addr learned my addr 172.21.15.26:0/2502679959 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f3722ffd640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 0x7f3724072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f3723fff640 1 --2- 172.21.15.26:0/2502679959 >> v2:172.21.15.117:3300/0 conn(0x7f3724072a50 0x7f3724158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161649890 0 0) 0x7f37240a7d70 con 0x7f372406fe20 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.062+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c003520 con 0x7f372406fe20 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 7802138 0 0) 0x7f370c003520 con 0x7f372406fe20 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 msgr2=0x7f3724072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 0x7f3724072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 >> v2:172.21.15.117:3300/0 conn(0x7f3724072a50 msgr2=0x7f3724158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 --2- 172.21.15.26:0/2502679959 >> v2:172.21.15.117:3300/0 conn(0x7f3724072a50 0x7f3724158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240c3320 con 0x7f372406fe20 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f3714001670 con 0x7f372406fe20 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 >> v1:172.21.15.26:6789/0 conn(0x7f372406fe20 legacy=0x7f3724155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f370c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f370c004800 0x7f370c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 0x7f370c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37240a8050 con 0x7f370c0039b0 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37240a8330 con 0x7f370c004800 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c00b9f0 con 0x7f37240c6780 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37237fe640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f370c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f3722ffd640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f370c004800 0x7f370c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f3723fff640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 0x7f370c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37237fe640 1 -- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 msgr2=0x7f370c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37237fe640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37240c6780 0x7f370c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37237fe640 1 -- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f370c004800 msgr2=0x7f370c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37237fe640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f370c004800 0x7f370c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.063+0000 7f37237fe640 1 -- 172.21.15.26:0/2502679959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c003520 con 0x7f370c0039b0 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.064+0000 7f37237fe640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f370c003dd0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f371400c240 tx=0x7f370c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e604661cbda4834 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.064+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714007d30 con 0x7f370c0039b0 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.064+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3714030070 con 0x7f370c0039b0 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.064+0000 7f37227fc640 1 -- 172.21.15.26:0/2502679959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371402cd60 con 0x7f370c0039b0 2024-03-06T07:29:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.065+0000 7f372a1c9640 1 -- 172.21.15.26:0/2502679959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 msgr2=0x7f370c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.065+0000 7f372a1c9640 1 --2- 172.21.15.26:0/2502679959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f370c003dd0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f371400c240 tx=0x7f370c005830 comp rx=0 tx=0).stop 2024-03-06T07:29:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.065+0000 7f3723fff640 1 -- 172.21.15.26:0/2502679959 reap_dead start 2024-03-06T07:29:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.065+0000 7f372a1c9640 1 -- 172.21.15.26:0/2502679959 shutdown_connections 2024-03-06T07:29:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.065+0000 7f372a1c9640 1 -- 172.21.15.26:0/2502679959 >> 172.21.15.26:0/2502679959 conn(0x7f37240aa630 msgr2=0x7f37240aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.065+0000 7f372a1c9640 1 -- 172.21.15.26:0/2502679959 shutdown_connections 2024-03-06T07:29:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.065+0000 7f372a1c9640 1 -- 172.21.15.26:0/2502679959 wait complete. 2024-03-06T07:29:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.066+0000 7f372a1c9640 1 Processor -- start 2024-03-06T07:29:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.066+0000 7f372a1c9640 1 -- start start 2024-03-06T07:29:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f372a1c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f372414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f372a1c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3724159000 0x7f37241ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f372a1c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724159480 0x7f37241ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f372a1c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724072510 con 0x7f370c0039b0 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f372a1c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724070200 con 0x7f3724159480 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f372a1c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3724072690 con 0x7f3724159000 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f372414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f372414ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49758/0 (socket says 172.21.15.26:49758) 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 -- 172.21.15.26:0/1035109233 learned_addr learned my addr 172.21.15.26:0/1035109233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f3722ffd640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3724159000 0x7f37241ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f3723fff640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724159480 0x7f37241ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 -- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3724159000 msgr2=0x7f37241ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3724159000 0x7f37241ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 -- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724159480 msgr2=0x7f37241ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724159480 0x7f37241ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 -- 172.21.15.26:0/1035109233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724150cf0 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f3723fff640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724159480 0x7f37241ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f37237fe640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f372414ec70 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f371402c5d0 tx=0x7f3724155190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f370bfff640 1 -- 172.21.15.26:0/1035109233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714001c70 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f370bfff640 1 -- 172.21.15.26:0/1035109233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3714004bf0 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f370bfff640 1 -- 172.21.15.26:0/1035109233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714007070 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.067+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3724156c60 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.068+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37241ef080 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.069+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372406c690 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.069+0000 7f370bfff640 1 -- 172.21.15.26:0/1035109233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3714047020 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.069+0000 7f370bfff640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f371006b4d0 0x7f371006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.069+0000 7f370bfff640 1 -- 172.21.15.26:0/1035109233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f37140ab4e0 con 0x7f370c0039b0 2024-03-06T07:29:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.069+0000 7f3722ffd640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f371006b4d0 0x7f371006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.070+0000 7f3722ffd640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f371006b4d0 0x7f371006d990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f372414f500 tx=0x7f3718009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.074+0000 7f370bfff640 1 -- 172.21.15.26:0/1035109233 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f372406c690 con 0x7f370c0039b0 2024-03-06T07:29:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.254+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f372404f330 con 0x7f370c0039b0 2024-03-06T07:29:05.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.948+0000 7f370bfff640 1 -- 172.21.15.26:0/1035109233 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v201) v1 ==== 181+0+0 (secure 0 0 0) 0x7f371407a2d0 con 0x7f370c0039b0 2024-03-06T07:29:05.950 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:29:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.951+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f371006b4d0 msgr2=0x7f371006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.951+0000 7f372a1c9640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f371006b4d0 0x7f371006d990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f372414f500 tx=0x7f3718009190 comp rx=0 tx=0).stop 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.951+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 msgr2=0x7f372414ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.951+0000 7f372a1c9640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f372414ec70 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f371402c5d0 tx=0x7f3724155190 comp rx=0 tx=0).stop 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 shutdown_connections 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f371006b4d0 0x7f371006d990 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724159480 0x7f37241ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3724159000 0x7f37241ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 --2- 172.21.15.26:0/1035109233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f370c0039b0 0x7f372414ec70 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 >> 172.21.15.26:0/1035109233 conn(0x7f37240aa630 msgr2=0x7f37240c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 shutdown_connections 2024-03-06T07:29:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:05.952+0000 7f372a1c9640 1 -- 172.21.15.26:0/1035109233 wait complete. 2024-03-06T07:29:05.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:29:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4dae55e640 1 Processor -- start 2024-03-06T07:29:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4dae55e640 1 -- start start 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4dae55e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 0x7f4da80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4dae55e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80c8ea0 con 0x7f4da80c7470 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4dae55e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80c7a10 con 0x7f4da8159230 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4dae55e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8158da0 con 0x7f4da8152860 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4da77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4da80c7470 0x7f4da8155d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43620/0 (socket says 172.21.15.26:43620) 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.077+0000 7f4da77fe640 1 -- 172.21.15.26:0/3257933285 learned_addr learned my addr 172.21.15.26:0/3257933285 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da7fff640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 0x7f4da80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687046709 0 0) 0x7f4da80c8ea0 con 0x7f4da80c7470 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d90003520 con 0x7f4da80c7470 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 824152691 0 0) 0x7f4d90003520 con 0x7f4da80c7470 2024-03-06T07:29:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 msgr2=0x7f4da80c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 0x7f4da80c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 >> v1:172.21.15.117:6789/0 conn(0x7f4da8159230 legacy=0x7f4da80c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80c7cf0 con 0x7f4da80c7470 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4d9c001670 con 0x7f4da80c7470 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 >> v1:172.21.15.26:6789/0 conn(0x7f4da80c7470 legacy=0x7f4da8155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d900039e0 0x7f4d90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d90004390 0x7f4d90004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 0x7f4d90005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8158da0 con 0x7f4d900039e0 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000ba70 con 0x7f4d90004390 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000bc70 con 0x7f4da8152860 2024-03-06T07:29:06.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da7fff640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d900039e0 0x7f4d90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da77fe640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d90004390 0x7f4d90004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4dacafb640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 0x7f4d90005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da7fff640 1 -- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 msgr2=0x7f4d90005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da7fff640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 0x7f4d90005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da7fff640 1 -- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d90004390 msgr2=0x7f4d90004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da7fff640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d90004390 0x7f4d90004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da7fff640 1 -- 172.21.15.26:0/3257933285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d90003520 con 0x7f4d900039e0 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4dacafb640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da8152860 0x7f4d90005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.078+0000 7f4da77fe640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d90004390 0x7f4d90004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.079+0000 7f4da7fff640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d900039e0 0x7f4d90003dc0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009ac0 tx=0x7f4d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22361cd5949d5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.079+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98039070 con 0x7f4d900039e0 2024-03-06T07:29:06.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.079+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9802c6b0 con 0x7f4d900039e0 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.079+0000 7f4da6ffd640 1 -- 172.21.15.26:0/3257933285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98009690 con 0x7f4d900039e0 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.080+0000 7f4dae55e640 1 -- 172.21.15.26:0/3257933285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d900039e0 msgr2=0x7f4d90003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.080+0000 7f4dae55e640 1 --2- 172.21.15.26:0/3257933285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d900039e0 0x7f4d90003dc0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009ac0 tx=0x7f4d90005830 comp rx=0 tx=0).stop 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.080+0000 7f4dacafb640 1 -- 172.21.15.26:0/3257933285 reap_dead start 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.080+0000 7f4dae55e640 1 -- 172.21.15.26:0/3257933285 shutdown_connections 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.080+0000 7f4dae55e640 1 -- 172.21.15.26:0/3257933285 >> 172.21.15.26:0/3257933285 conn(0x7f4da80ac630 msgr2=0x7f4da80aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.080+0000 7f4dae55e640 1 -- 172.21.15.26:0/3257933285 shutdown_connections 2024-03-06T07:29:06.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.080+0000 7f4dae55e640 1 -- 172.21.15.26:0/3257933285 wait complete. 2024-03-06T07:29:06.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.081+0000 7f4dae55e640 1 Processor -- start 2024-03-06T07:29:06.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.081+0000 7f4dae55e640 1 -- start start 2024-03-06T07:29:06.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4dae55e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da814dec0 0x7f4da81ee850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4dae55e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da8158600 0x7f4da81e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4dae55e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da81e8100 0x7f4da81eae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4dae55e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80c5af0 con 0x7f4da8158600 2024-03-06T07:29:06.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4dae55e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8152c40 con 0x7f4da814dec0 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4dae55e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80c5c70 con 0x7f4da81e8100 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da8158600 0x7f4da81e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da8158600 0x7f4da81e7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49766/0 (socket says 172.21.15.26:49766) 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 -- 172.21.15.26:0/457651880 learned_addr learned my addr 172.21.15.26:0/457651880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da7fff640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da814dec0 0x7f4da81ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4dacafb640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da81e8100 0x7f4da81eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 -- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da81e8100 msgr2=0x7f4da81eae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da81e8100 0x7f4da81eae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 -- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da814dec0 msgr2=0x7f4da81ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da814dec0 0x7f4da81ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 -- 172.21.15.26:0/457651880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da81e8580 con 0x7f4da8158600 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.082+0000 7f4da77fe640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da8158600 0x7f4da81e7bc0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00eaf0 tx=0x7f4d9c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.083+0000 7f4da4ff9640 1 -- 172.21.15.26:0/457651880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00c870 con 0x7f4da8158600 2024-03-06T07:29:06.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.083+0000 7f4da4ff9640 1 -- 172.21.15.26:0/457651880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c018070 con 0x7f4da8158600 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.083+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da8157800 con 0x7f4da8158600 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.083+0000 7f4da4ff9640 1 -- 172.21.15.26:0/457651880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00cc10 con 0x7f4da8158600 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.083+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da8157dc0 con 0x7f4da8158600 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.084+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da8067910 con 0x7f4da8158600 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.084+0000 7f4da4ff9640 1 -- 172.21.15.26:0/457651880 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d9c013420 con 0x7f4da8158600 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.084+0000 7f4da4ff9640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d9406b500 0x7f4d9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.085+0000 7f4da4ff9640 1 -- 172.21.15.26:0/457651880 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5750+0+0 (secure 0 0 0) 0x7f4d9c089c70 con 0x7f4da8158600 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.085+0000 7f4da7fff640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d9406b500 0x7f4d9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.085+0000 7f4da7fff640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d9406b500 0x7f4d9406d9c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4d98036000 tx=0x7f4d98030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:06.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.089+0000 7f4da4ff9640 1 -- 172.21.15.26:0/457651880 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4da8067910 con 0x7f4da8158600 2024-03-06T07:29:06.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.269+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4da80c8110 con 0x7f4da8158600 2024-03-06T07:29:06.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.950+0000 7f4da4ff9640 1 -- 172.21.15.26:0/457651880 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v202) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4d9c058ab0 con 0x7f4da8158600 2024-03-06T07:29:06.952 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:29:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d9406b500 msgr2=0x7f4d9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d9406b500 0x7f4d9406d9c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4d98036000 tx=0x7f4d98030000 comp rx=0 tx=0).stop 2024-03-06T07:29:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da8158600 msgr2=0x7f4da81e7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da8158600 0x7f4da81e7bc0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00eaf0 tx=0x7f4d9c00eef0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 shutdown_connections 2024-03-06T07:29:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4d9406b500 0x7f4d9406d9c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4da81e8100 0x7f4da81eae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4da8158600 0x7f4da81e7bc0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 --2- 172.21.15.26:0/457651880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da814dec0 0x7f4da81ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.952+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 >> 172.21.15.26:0/457651880 conn(0x7f4da80ac630 msgr2=0x7f4da806c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.953+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 shutdown_connections 2024-03-06T07:29:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:06.953+0000 7f4dae55e640 1 -- 172.21.15.26:0/457651880 wait complete. 2024-03-06T07:29:06.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:29:07.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.077+0000 7f15325a0640 1 Processor -- start 2024-03-06T07:29:07.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.077+0000 7f15325a0640 1 -- start start 2024-03-06T07:29:07.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f15325a0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f152c0b0070 0x7f152c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f15325a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 0x7f152c0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f15325a0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c0b0450 con 0x7f152c06ca70 2024-03-06T07:29:07.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f15325a0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c06c020 con 0x7f152c0b0070 2024-03-06T07:29:07.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f15325a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c06cf20 con 0x7f152c0c5380 2024-03-06T07:29:07.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f152c06ca70 0x7f152c0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49502/0 (socket says 172.21.15.26:49502) 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152b7fe640 1 -- 172.21.15.26:0/4116698109 learned_addr learned my addr 172.21.15.26:0/4116698109 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671549865 0 0) 0x7f1514001110 con 0x7f152c06ca70 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1518003520 con 0x7f152c06ca70 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152bfff640 1 --2- 172.21.15.26:0/4116698109 >> v2:172.21.15.117:3300/0 conn(0x7f152c0b0070 0x7f152c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f1530b3d640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 0x7f152c0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243420120 0 0) 0x7f1518003520 con 0x7f152c06ca70 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 msgr2=0x7f152c0c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 0x7f152c0c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 >> v2:172.21.15.117:3300/0 conn(0x7f152c0b0070 msgr2=0x7f152c155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 --2- 172.21.15.26:0/4116698109 >> v2:172.21.15.117:3300/0 conn(0x7f152c0b0070 0x7f152c155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0c77f0 con 0x7f152c06ca70 2024-03-06T07:29:07.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f1514001840 con 0x7f152c06ca70 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f1530b3d640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 0x7f152c0c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 >> v1:172.21.15.26:6789/0 conn(0x7f152c06ca70 legacy=0x7f152c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15180039b0 0x7f1518003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518004800 0x7f1518004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 0x7f15180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c06c020 con 0x7f15180039b0 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c06cf20 con 0x7f1518004800 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.078+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151800ba40 con 0x7f152c0c5380 2024-03-06T07:29:07.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f152bfff640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15180039b0 0x7f1518003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f152b7fe640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518004800 0x7f1518004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f1530b3d640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 0x7f15180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f152bfff640 1 -- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 msgr2=0x7f15180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f152bfff640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0c5380 0x7f15180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f152bfff640 1 -- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518004800 msgr2=0x7f1518004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f152bfff640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518004800 0x7f1518004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.079+0000 7f152bfff640 1 -- 172.21.15.26:0/4116698109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518003520 con 0x7f15180039b0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f152bfff640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15180039b0 0x7f1518003dd0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f151c009ac0 tx=0x7f1518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6900f222562f17c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c039070 con 0x7f15180039b0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c02c710 con 0x7f15180039b0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f152affd640 1 -- 172.21.15.26:0/4116698109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c009690 con 0x7f15180039b0 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f15325a0640 1 -- 172.21.15.26:0/4116698109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15180039b0 msgr2=0x7f1518003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f15325a0640 1 --2- 172.21.15.26:0/4116698109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15180039b0 0x7f1518003dd0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f151c009ac0 tx=0x7f1518005830 comp rx=0 tx=0).stop 2024-03-06T07:29:07.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f1530b3d640 1 -- 172.21.15.26:0/4116698109 reap_dead start 2024-03-06T07:29:07.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f15325a0640 1 -- 172.21.15.26:0/4116698109 shutdown_connections 2024-03-06T07:29:07.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f15325a0640 1 -- 172.21.15.26:0/4116698109 >> 172.21.15.26:0/4116698109 conn(0x7f152c0ac590 msgr2=0x7f152c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:07.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f15325a0640 1 -- 172.21.15.26:0/4116698109 shutdown_connections 2024-03-06T07:29:07.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.080+0000 7f15325a0640 1 -- 172.21.15.26:0/4116698109 wait complete. 2024-03-06T07:29:07.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.081+0000 7f15325a0640 1 Processor -- start 2024-03-06T07:29:07.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f15325a0640 1 -- start start 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f15325a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180039b0 0x7f152c1edf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f15325a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c14ada0 0x7f152c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f15325a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f152c158310 0x7f152c149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f15325a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c06c550 con 0x7f152c158310 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f15325a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c06c270 con 0x7f152c14ada0 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f15325a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c06da80 con 0x7f15180039b0 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f1530b3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f152c158310 0x7f152c149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f1530b3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f152c158310 0x7f152c149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38026/0 (socket says 172.21.15.26:38026) 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.082+0000 7f1530b3d640 1 -- 172.21.15.26:0/2165109608 learned_addr learned my addr 172.21.15.26:0/2165109608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f152b7fe640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c14ada0 0x7f152c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f152bfff640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180039b0 0x7f152c1edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1530b3d640 1 -- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180039b0 msgr2=0x7f152c1edf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1530b3d640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180039b0 0x7f152c1edf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1530b3d640 1 -- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c14ada0 msgr2=0x7f152c1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1530b3d640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c14ada0 0x7f152c1590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1530b3d640 1 -- 172.21.15.26:0/2165109608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0c4c20 con 0x7f152c158310 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f152b7fe640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c14ada0 0x7f152c1590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f152bfff640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180039b0 0x7f152c1edf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1530b3d640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f152c158310 0x7f152c149c70 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f152c0cba60 tx=0x7f1524010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1528ff9640 1 -- 172.21.15.26:0/2165109608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524010cc0 con 0x7f152c158310 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1528ff9640 1 -- 172.21.15.26:0/2165109608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1524003990 con 0x7f152c158310 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c0c2ef0 con 0x7f152c158310 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f1528ff9640 1 -- 172.21.15.26:0/2165109608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524010ea0 con 0x7f152c158310 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.083+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c159610 con 0x7f152c158310 2024-03-06T07:29:07.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.084+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c04f330 con 0x7f152c158310 2024-03-06T07:29:07.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.085+0000 7f1528ff9640 1 -- 172.21.15.26:0/2165109608 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f152401e020 con 0x7f152c158310 2024-03-06T07:29:07.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.085+0000 7f1528ff9640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f150c06b4d0 0x7f150c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.085+0000 7f1528ff9640 1 -- 172.21.15.26:0/2165109608 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f152408a5d0 con 0x7f152c158310 2024-03-06T07:29:07.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.086+0000 7f152bfff640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f150c06b4d0 0x7f150c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.086+0000 7f152bfff640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f150c06b4d0 0x7f150c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f151c036000 tx=0x7f151c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:07.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.090+0000 7f1528ff9640 1 -- 172.21.15.26:0/2165109608 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1524025020 con 0x7f152c158310 2024-03-06T07:29:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.270+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f152c0c1890 con 0x7f152c158310 2024-03-06T07:29:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.274+0000 7f1528ff9640 1 -- 172.21.15.26:0/2165109608 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1524008e40 con 0x7f152c158310 2024-03-06T07:29:07.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f150c06b4d0 msgr2=0x7f150c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f150c06b4d0 0x7f150c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f151c036000 tx=0x7f151c030000 comp rx=0 tx=0).stop 2024-03-06T07:29:07.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f152c158310 msgr2=0x7f152c149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f152c158310 0x7f152c149c70 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f152c0cba60 tx=0x7f1524010530 comp rx=0 tx=0).stop 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 shutdown_connections 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f150c06b4d0 0x7f150c06d990 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f152c158310 0x7f152c149c70 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c14ada0 0x7f152c1590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 --2- 172.21.15.26:0/2165109608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180039b0 0x7f152c1edf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 >> 172.21.15.26:0/2165109608 conn(0x7f152c0ac590 msgr2=0x7f152c151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 shutdown_connections 2024-03-06T07:29:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.277+0000 7f15325a0640 1 -- 172.21.15.26:0/2165109608 wait complete. 2024-03-06T07:29:07.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_health' 2024-03-06T07:29:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.390+0000 7fecd23bc640 1 Processor -- start 2024-03-06T07:29:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.390+0000 7fecd23bc640 1 -- start start 2024-03-06T07:29:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7fecd23bc640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feccc06da40 0x7feccc155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7fecd23bc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 0x7feccc0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7fecd23bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feccc156050 con 0x7feccc0c6780 2024-03-06T07:29:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7fecd23bc640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc06de20 con 0x7feccc06da40 2024-03-06T07:29:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7fecd23bc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc072630 con 0x7feccc0c7380 2024-03-06T07:29:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccb7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feccc0c6780 0x7feccc158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49508/0 (socket says 172.21.15.26:49508) 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccb7fe640 1 -- 172.21.15.26:0/2545852027 learned_addr learned my addr 172.21.15.26:0/2545852027 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781214339 0 0) 0x7fecbc001110 con 0x7feccc0c6780 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7fecd0959640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 0x7feccc0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb4003520 con 0x7feccc0c6780 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccbfff640 1 --2- 172.21.15.26:0/2545852027 >> v2:172.21.15.117:3300/0 conn(0x7feccc06da40 0x7feccc155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 22558626 0 0) 0x7fecb4003520 con 0x7feccc0c6780 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 msgr2=0x7feccc0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 0x7feccc0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 >> v2:172.21.15.117:3300/0 conn(0x7feccc06da40 msgr2=0x7feccc155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 --2- 172.21.15.26:0/2545852027 >> v2:172.21.15.117:3300/0 conn(0x7feccc06da40 0x7feccc155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc072910 con 0x7feccc0c6780 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fecbc001840 con 0x7feccc0c6780 2024-03-06T07:29:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.391+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 >> v1:172.21.15.26:6789/0 conn(0x7feccc0c6780 legacy=0x7feccc158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccaffd640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 0x7fecb40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccaffd640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb4003f30 0x7fecb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccaffd640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecb4004d40 0x7fecb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc06de20 con 0x7fecb4004d40 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc072630 con 0x7fecb4003f30 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb400ba10 con 0x7feccc0c7380 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7fecd0959640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecb4004d40 0x7fecb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccbfff640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 0x7fecb40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccb7fe640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb4003f30 0x7fecb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7fecd0959640 1 -- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 msgr2=0x7fecb40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7fecd0959640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feccc0c7380 0x7fecb40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7fecd0959640 1 -- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb4003f30 msgr2=0x7fecb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7fecd0959640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb4003f30 0x7fecb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7fecd0959640 1 -- 172.21.15.26:0/2545852027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4003520 con 0x7fecb4004d40 2024-03-06T07:29:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.392+0000 7feccb7fe640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb4003f30 0x7fecb4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd0959640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecb4004d40 0x7fecb40051c0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009fc0 tx=0x7fecb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda3c58a9f700ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4039070 con 0x7fecb4004d40 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc402cbd0 con 0x7fecb4004d40 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7feccaffd640 1 -- 172.21.15.26:0/2545852027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4009c10 con 0x7fecb4004d40 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd23bc640 1 -- 172.21.15.26:0/2545852027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecb4004d40 msgr2=0x7fecb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd23bc640 1 --2- 172.21.15.26:0/2545852027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecb4004d40 0x7fecb40051c0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009fc0 tx=0x7fecb4005830 comp rx=0 tx=0).stop 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd0959640 1 -- 172.21.15.26:0/2545852027 reap_dead start 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd23bc640 1 -- 172.21.15.26:0/2545852027 shutdown_connections 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd23bc640 1 -- 172.21.15.26:0/2545852027 >> 172.21.15.26:0/2545852027 conn(0x7feccc0aa630 msgr2=0x7feccc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd23bc640 1 -- 172.21.15.26:0/2545852027 shutdown_connections 2024-03-06T07:29:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.393+0000 7fecd23bc640 1 -- 172.21.15.26:0/2545852027 wait complete. 2024-03-06T07:29:07.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.394+0000 7fecd23bc640 1 Processor -- start 2024-03-06T07:29:07.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7fecd23bc640 1 -- start start 2024-03-06T07:29:07.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7fecd23bc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecb4004d40 0x7feccc0a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7fecd23bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc1e1f70 0x7feccc14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7fecd23bc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ae720 0x7feccc14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7fecd23bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1594a0 con 0x7feccc1e1f70 2024-03-06T07:29:07.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7fecd23bc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc158890 con 0x7feccc0ae720 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7fecd23bc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0c3280 con 0x7fecb4004d40 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc1e1f70 0x7feccc14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc1e1f70 0x7feccc14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38046/0 (socket says 172.21.15.26:38046) 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.395+0000 7feccb7fe640 1 -- 172.21.15.26:0/1788802721 learned_addr learned my addr 172.21.15.26:0/1788802721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7feccbfff640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecb4004d40 0x7feccc0a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7fecd0959640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ae720 0x7feccc14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7feccb7fe640 1 -- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecb4004d40 msgr2=0x7feccc0a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7feccb7fe640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecb4004d40 0x7feccc0a97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7feccb7fe640 1 -- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ae720 msgr2=0x7feccc14efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7feccb7fe640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ae720 0x7feccc14efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7feccb7fe640 1 -- 172.21.15.26:0/1788802721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc1e2530 con 0x7feccc1e1f70 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7fecd0959640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ae720 0x7feccc14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7feccb7fe640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc1e1f70 0x7feccc14ea90 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e8d0 tx=0x7fecbc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7fecc8ff9640 1 -- 172.21.15.26:0/1788802721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00c8d0 con 0x7feccc1e1f70 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7fecc8ff9640 1 -- 172.21.15.26:0/1788802721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc018070 con 0x7feccc1e1f70 2024-03-06T07:29:07.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc153060 con 0x7feccc1e1f70 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7fecc8ff9640 1 -- 172.21.15.26:0/1788802721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00cc90 con 0x7feccc1e1f70 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.396+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc0ad710 con 0x7feccc1e1f70 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.397+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc04f330 con 0x7feccc1e1f70 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.398+0000 7fecc8ff9640 1 -- 172.21.15.26:0/1788802721 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecbc01d460 con 0x7feccc1e1f70 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.398+0000 7fecc8ff9640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.398+0000 7fecc8ff9640 1 -- 172.21.15.26:0/1788802721 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fecbc089f30 con 0x7feccc1e1f70 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.398+0000 7feccbfff640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.399+0000 7feccbfff640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecb806b4d0 0x7fecb806d990 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009b80 tx=0x7fecc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:07.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.402+0000 7fecc8ff9640 1 -- 172.21.15.26:0/1788802721 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feccc156050 con 0x7feccc1e1f70 2024-03-06T07:29:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.660+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]} v 0) v1 -- 0x7feccc0aec10 con 0x7feccc1e1f70 2024-03-06T07:29:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.965+0000 7fecc8ff9640 1 -- 172.21.15.26:0/1788802721 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]}]=0 v513) v1 ==== 140+0+0 (secure 0 0 0) 0x7fecbc0590c0 con 0x7feccc1e1f70 2024-03-06T07:29:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.967+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecb806b4d0 msgr2=0x7fecb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.967+0000 7fecd23bc640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecb806b4d0 0x7fecb806d990 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009b80 tx=0x7fecc0008040 comp rx=0 tx=0).stop 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.967+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc1e1f70 msgr2=0x7feccc14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc1e1f70 0x7feccc14ea90 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e8d0 tx=0x7fecbc00ed90 comp rx=0 tx=0).stop 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 shutdown_connections 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ae720 0x7feccc14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc1e1f70 0x7feccc14ea90 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 --2- 172.21.15.26:0/1788802721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fecb4004d40 0x7feccc0a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 >> 172.21.15.26:0/1788802721 conn(0x7feccc0aa630 msgr2=0x7feccc1540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 shutdown_connections 2024-03-06T07:29:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:07.968+0000 7fecd23bc640 1 -- 172.21.15.26:0/1788802721 wait complete. 2024-03-06T07:29:07.980 INFO:tasks.cephfs_test_runner:That marking a FS down does not generate a health warning ... ok 2024-03-06T07:29:07.981 INFO:tasks.cephfs_test_runner:Starting test: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) That marking a FS down twice does not wipe old_max_mds. 2024-03-06T07:29:07.981 INFO:tasks.cephfs_test_runner:test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:29:07.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:08.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.081+0000 7f163b2fb640 1 Processor -- start 2024-03-06T07:29:08.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.081+0000 7f163b2fb640 1 -- start start 2024-03-06T07:29:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f163b2fb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f163414f790 0x7f16341529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f163b2fb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16340c1aa0 0x7f16340c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f163b2fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16340c5c60 0x7f1634155a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f163b2fb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340c2680 con 0x7f16340c1aa0 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f163b2fb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340c28d0 con 0x7f163414f790 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f163b2fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340c2bb0 con 0x7f16340c5c60 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f1638896640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16340c1aa0 0x7f16340c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f1638896640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16340c1aa0 0x7f16340c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38050/0 (socket says 172.21.15.26:38050) 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.082+0000 7f1638896640 1 -- 172.21.15.26:0/2492855771 learned_addr learned my addr 172.21.15.26:0/2492855771 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1639097640 1 --2- 172.21.15.26:0/2492855771 >> v2:172.21.15.117:3300/0 conn(0x7f163414f790 0x7f16341529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1639898640 1 --2- 172.21.15.26:0/2492855771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16340c5c60 0x7f1634155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1638896640 1 -- 172.21.15.26:0/2492855771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16340c5c60 msgr2=0x7f1634155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1638896640 1 --2- 172.21.15.26:0/2492855771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16340c5c60 0x7f1634155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1638896640 1 -- 172.21.15.26:0/2492855771 >> v2:172.21.15.117:3300/0 conn(0x7f163414f790 msgr2=0x7f16341529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1638896640 1 --2- 172.21.15.26:0/2492855771 >> v2:172.21.15.117:3300/0 conn(0x7f163414f790 0x7f16341529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1638896640 1 -- 172.21.15.26:0/2492855771 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340c2e90 con 0x7f16340c1aa0 2024-03-06T07:29:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1639898640 1 --2- 172.21.15.26:0/2492855771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16340c5c60 0x7f1634155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.083+0000 7f1638896640 1 --2- 172.21.15.26:0/2492855771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c1aa0 0x7f16340c5720 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f16340c3170 tx=0x7f162402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f753736532301bac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f162bfff640 1 -- 172.21.15.26:0/2492855771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162403b070 con 0x7f16340c1aa0 2024-03-06T07:29:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f162bfff640 1 -- 172.21.15.26:0/2492855771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624036440 con 0x7f16340c1aa0 2024-03-06T07:29:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f162bfff640 1 -- 172.21.15.26:0/2492855771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624036810 con 0x7f16340c1aa0 2024-03-06T07:29:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 -- 172.21.15.26:0/2492855771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c1aa0 msgr2=0x7f16340c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2492855771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c1aa0 0x7f16340c5720 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f16340c3170 tx=0x7f162402e0e0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 -- 172.21.15.26:0/2492855771 shutdown_connections 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2492855771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16340c5c60 0x7f1634155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2492855771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c1aa0 0x7f16340c5720 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2492855771 >> v2:172.21.15.117:3300/0 conn(0x7f163414f790 0x7f16341529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 -- 172.21.15.26:0/2492855771 >> 172.21.15.26:0/2492855771 conn(0x7f16340c6710 msgr2=0x7f16340c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 -- 172.21.15.26:0/2492855771 shutdown_connections 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.084+0000 7f163b2fb640 1 -- 172.21.15.26:0/2492855771 wait complete. 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.085+0000 7f163b2fb640 1 Processor -- start 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.085+0000 7f163b2fb640 1 -- start start 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f163b2fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16340c1aa0 0x7f16340a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f163b2fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c5c60 0x7f163414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f163b2fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163414f790 0x7f163414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f163b2fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163406cc00 con 0x7f16340c5c60 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f163b2fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340c0d60 con 0x7f16340c1aa0 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f163b2fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163406cd80 con 0x7f163414f790 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c5c60 0x7f163414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c5c60 0x7f163414b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38066/0 (socket says 172.21.15.26:38066) 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 -- 172.21.15.26:0/2547022387 learned_addr learned my addr 172.21.15.26:0/2547022387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1639097640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16340c1aa0 0x7f16340a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1639898640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163414f790 0x7f163414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 -- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163414f790 msgr2=0x7f163414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163414f790 0x7f163414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 -- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16340c1aa0 msgr2=0x7f16340a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16340c1aa0 0x7f16340a76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 -- 172.21.15.26:0/2547022387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340a81f0 con 0x7f16340c5c60 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1639097640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16340c1aa0 0x7f16340a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.086+0000 7f1638896640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c5c60 0x7f163414b130 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f162402e120 tx=0x7f16240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.087+0000 7f1629ffb640 1 -- 172.21.15.26:0/2547022387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624046070 con 0x7f16340c5c60 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.087+0000 7f1629ffb640 1 -- 172.21.15.26:0/2547022387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162403d050 con 0x7f16340c5c60 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.087+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16341ee340 con 0x7f16340c5c60 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.087+0000 7f1629ffb640 1 -- 172.21.15.26:0/2547022387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162403b030 con 0x7f16340c5c60 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.087+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16341ee900 con 0x7f16340c5c60 2024-03-06T07:29:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.088+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163404f330 con 0x7f16340c5c60 2024-03-06T07:29:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.088+0000 7f1629ffb640 1 -- 172.21.15.26:0/2547022387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1624049070 con 0x7f16340c5c60 2024-03-06T07:29:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.089+0000 7f1629ffb640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161806b500 0x7f161806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.089+0000 7f1629ffb640 1 -- 172.21.15.26:0/2547022387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f16240abab0 con 0x7f16340c5c60 2024-03-06T07:29:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.089+0000 7f1639097640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161806b500 0x7f161806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.090+0000 7f1639097640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161806b500 0x7f161806d9c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1630006fd0 tx=0x7f1630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.093+0000 7f1629ffb640 1 -- 172.21.15.26:0/2547022387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f162407ab80 con 0x7f16340c5c60 2024-03-06T07:29:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.279+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16340c1f40 con 0x7f16340c5c60 2024-03-06T07:29:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.280+0000 7f1629ffb640 1 -- 172.21.15.26:0/2547022387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+13186 (secure 0 0 0) 0x7f162407a480 con 0x7f16340c5c60 2024-03-06T07:29:08.281 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:08.281 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:29:06.949061+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161806b500 msgr2=0x7f161806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161806b500 0x7f161806d9c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1630006fd0 tx=0x7f1630008040 comp rx=0 tx=0).stop 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c5c60 msgr2=0x7f163414b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c5c60 0x7f163414b130 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f162402e120 tx=0x7f16240046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 shutdown_connections 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161806b500 0x7f161806d9c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163414f790 0x7f163414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16340c5c60 0x7f163414b130 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 --2- 172.21.15.26:0/2547022387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16340c1aa0 0x7f16340a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 >> 172.21.15.26:0/2547022387 conn(0x7f16340c6710 msgr2=0x7f16341dd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 shutdown_connections 2024-03-06T07:29:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.282+0000 7f163b2fb640 1 -- 172.21.15.26:0/2547022387 wait complete. 2024-03-06T07:29:08.294 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:29:08.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:29:08.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.397+0000 7f22a0181640 1 Processor -- start 2024-03-06T07:29:08.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.397+0000 7f22a0181640 1 -- start start 2024-03-06T07:29:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.398+0000 7f22a0181640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f229806d750 0x7f22980732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.398+0000 7f22a0181640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2298156060 con 0x7f22980c3a70 2024-03-06T07:29:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.398+0000 7f22a0181640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22980a8430 con 0x7f229806db30 2024-03-06T07:29:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.398+0000 7f22a0181640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2298155b30 con 0x7f229806d750 2024-03-06T07:29:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.398+0000 7f229df1d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f229806d750 0x7f22980732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.398+0000 7f229d71c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f22980c3a70 0x7f2298072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49518/0 (socket says 172.21.15.26:49518) 2024-03-06T07:29:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.398+0000 7f229d71c640 1 -- 172.21.15.26:0/2761914117 learned_addr learned my addr 172.21.15.26:0/2761914117 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.399+0000 7f229df1d640 1 -- 172.21.15.26:0/2761914117 >> v1:172.21.15.117:6789/0 conn(0x7f229806db30 legacy=0x7f22980c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.399+0000 7f229df1d640 1 -- 172.21.15.26:0/2761914117 >> v1:172.21.15.26:6789/0 conn(0x7f22980c3a70 legacy=0x7f2298072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.399+0000 7f229df1d640 1 -- 172.21.15.26:0/2761914117 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298155d80 con 0x7f229806d750 2024-03-06T07:29:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.399+0000 7f229cf1b640 1 -- 172.21.15.26:0/2761914117 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431496712 0 0) 0x7f2298156060 con 0x7f22980c3a70 2024-03-06T07:29:08.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.399+0000 7f229df1d640 1 --2- 172.21.15.26:0/2761914117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f229806d750 0x7f22980732a0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f22900097a0 tx=0x7f229002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a2b06ccb512c91 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f229cf1b640 1 -- 172.21.15.26:0/2761914117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2290038070 con 0x7f229806d750 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f229cf1b640 1 -- 172.21.15.26:0/2761914117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2290031700 con 0x7f229806d750 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f229cf1b640 1 -- 172.21.15.26:0/2761914117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2290031ad0 con 0x7f229806d750 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f22a0181640 1 -- 172.21.15.26:0/2761914117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f229806d750 msgr2=0x7f22980732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f22a0181640 1 --2- 172.21.15.26:0/2761914117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f229806d750 0x7f22980732a0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f22900097a0 tx=0x7f229002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f22a0181640 1 -- 172.21.15.26:0/2761914117 shutdown_connections 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f22a0181640 1 --2- 172.21.15.26:0/2761914117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f229806d750 0x7f22980732a0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f22a0181640 1 -- 172.21.15.26:0/2761914117 >> 172.21.15.26:0/2761914117 conn(0x7f22980c5560 msgr2=0x7f22980c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f22a0181640 1 -- 172.21.15.26:0/2761914117 shutdown_connections 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.400+0000 7f22a0181640 1 -- 172.21.15.26:0/2761914117 wait complete. 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.401+0000 7f22a0181640 1 Processor -- start 2024-03-06T07:29:08.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.401+0000 7f22a0181640 1 -- start start 2024-03-06T07:29:08.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f22a0181640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f229806d750 0x7f229814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f22a0181640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f229806db30 0x7f2298153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f22a0181640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22980c3a70 0x7f2298158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f22a0181640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22980c2720 con 0x7f22980c3a70 2024-03-06T07:29:08.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f22a0181640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22980c25a0 con 0x7f229806d750 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f22a0181640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980a7cd0 con 0x7f229806db30 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22980c3a70 0x7f2298158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22980c3a70 0x7f2298158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38090/0 (socket says 172.21.15.26:38090) 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 -- 172.21.15.26:0/2064931037 learned_addr learned my addr 172.21.15.26:0/2064931037 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229df1d640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f229806d750 0x7f229814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229d71c640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f229806db30 0x7f2298153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 -- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f229806db30 msgr2=0x7f2298153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f229806db30 0x7f2298153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 -- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f229806d750 msgr2=0x7f229814d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f229806d750 0x7f229814d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229e71e640 1 -- 172.21.15.26:0/2064931037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229814dd50 con 0x7f22980c3a70 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.402+0000 7f229df1d640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f229806d750 0x7f229814d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:08.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.403+0000 7f229e71e640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22980c3a70 0x7f2298158550 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f228800c780 tx=0x7f228800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.403+0000 7f22967fc640 1 -- 172.21.15.26:0/2064931037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288017070 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.403+0000 7f22967fc640 1 -- 172.21.15.26:0/2064931037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2288003da0 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.403+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22981edd90 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.403+0000 7f22967fc640 1 -- 172.21.15.26:0/2064931037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22880052b0 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.403+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22981ee350 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.404+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22981ea850 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.404+0000 7f22967fc640 1 -- 172.21.15.26:0/2064931037 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f228801a070 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.404+0000 7f22967fc640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f228406b4d0 0x7f228406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.405+0000 7f22967fc640 1 -- 172.21.15.26:0/2064931037 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f22880895b0 con 0x7f22980c3a70 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.405+0000 7f229df1d640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f228406b4d0 0x7f228406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.405+0000 7f229df1d640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f228406b4d0 0x7f228406d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2290035000 tx=0x7f2290008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.409+0000 7f22967fc640 1 -- 172.21.15.26:0/2064931037 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f22981ea850 con 0x7f22980c3a70 2024-03-06T07:29:08.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.589+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f229804f330 con 0x7f22980c3a70 2024-03-06T07:29:08.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.589+0000 7f22967fc640 1 -- 172.21.15.26:0/2064931037 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v202) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2288058680 con 0x7f22980c3a70 2024-03-06T07:29:08.591 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:29:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f228406b4d0 msgr2=0x7f228406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f228406b4d0 0x7f228406d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2290035000 tx=0x7f2290008030 comp rx=0 tx=0).stop 2024-03-06T07:29:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22980c3a70 msgr2=0x7f2298158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22980c3a70 0x7f2298158550 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f228800c780 tx=0x7f228800cc40 comp rx=0 tx=0).stop 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 shutdown_connections 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f228406b4d0 0x7f228406d990 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22980c3a70 0x7f2298158550 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f229806db30 0x7f2298153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 --2- 172.21.15.26:0/2064931037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f229806d750 0x7f229814d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.591+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 >> 172.21.15.26:0/2064931037 conn(0x7f22980c5560 msgr2=0x7f2298072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.592+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 shutdown_connections 2024-03-06T07:29:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.592+0000 7f22a0181640 1 -- 172.21.15.26:0/2064931037 wait complete. 2024-03-06T07:29:08.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_twice' 2024-03-06T07:29:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.703+0000 7f93a4716640 1 Processor -- start 2024-03-06T07:29:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.703+0000 7f93a4716640 1 -- start start 2024-03-06T07:29:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a4716640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f939c1527b0 0x7f939c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a4716640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f939c0c7180 0x7f939c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a4716640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c0c7c60 con 0x7f939c155b20 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a4716640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c159640 con 0x7f939c0c7180 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a4716640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c0c8df0 con 0x7f939c1527b0 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a1cb1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f939c0c7180 0x7f939c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a2cb3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f939c155b20 0x7f939c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49534/0 (socket says 172.21.15.26:49534) 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.704+0000 7f93a2cb3640 1 -- 172.21.15.26:0/2741598553 learned_addr learned my addr 172.21.15.26:0/2741598553 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.705+0000 7f93a1cb1640 1 -- 172.21.15.26:0/2741598553 >> v2:172.21.15.117:3300/0 conn(0x7f939c1527b0 msgr2=0x7f939c159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.705+0000 7f93a24b2640 1 --2- 172.21.15.26:0/2741598553 >> v2:172.21.15.117:3300/0 conn(0x7f939c1527b0 0x7f939c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.705+0000 7f93a1cb1640 1 --2- 172.21.15.26:0/2741598553 >> v2:172.21.15.117:3300/0 conn(0x7f939c1527b0 0x7f939c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.705+0000 7f93a1cb1640 1 -- 172.21.15.26:0/2741598553 >> v1:172.21.15.26:6789/0 conn(0x7f939c155b20 legacy=0x7f939c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.705+0000 7f93a14b0640 1 -- 172.21.15.26:0/2741598553 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310177925 0 0) 0x7f9394001490 con 0x7f939c155b20 2024-03-06T07:29:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.705+0000 7f93a1cb1640 1 -- 172.21.15.26:0/2741598553 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c0c8060 con 0x7f939c0c7180 2024-03-06T07:29:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a1cb1640 1 --2- 172.21.15.26:0/2741598553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c0c7180 0x7f939c0c88b0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f939c0cba80 tx=0x7f938c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebbe65ee9348bd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a14b0640 1 -- 172.21.15.26:0/2741598553 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c0315a0 con 0x7f939c0c7180 2024-03-06T07:29:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a14b0640 1 -- 172.21.15.26:0/2741598553 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c037070 con 0x7f939c0c7180 2024-03-06T07:29:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a14b0640 1 -- 172.21.15.26:0/2741598553 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c031960 con 0x7f939c0c7180 2024-03-06T07:29:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 -- 172.21.15.26:0/2741598553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c0c7180 msgr2=0x7f939c0c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 --2- 172.21.15.26:0/2741598553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c0c7180 0x7f939c0c88b0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f939c0cba80 tx=0x7f938c02b840 comp rx=0 tx=0).stop 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 -- 172.21.15.26:0/2741598553 shutdown_connections 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 --2- 172.21.15.26:0/2741598553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c0c7180 0x7f939c0c88b0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 --2- 172.21.15.26:0/2741598553 >> v2:172.21.15.117:3300/0 conn(0x7f939c1527b0 0x7f939c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 -- 172.21.15.26:0/2741598553 >> 172.21.15.26:0/2741598553 conn(0x7f939c0ac560 msgr2=0x7f939c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 -- 172.21.15.26:0/2741598553 shutdown_connections 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.706+0000 7f93a4716640 1 -- 172.21.15.26:0/2741598553 wait complete. 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.707+0000 7f93a4716640 1 Processor -- start 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.707+0000 7f93a4716640 1 -- start start 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a4716640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f939c0c7180 0x7f939c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a4716640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f939c1527b0 0x7f939c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a4716640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c155b20 0x7f939c1ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a4716640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c155320 con 0x7f939c155b20 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a4716640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0c7980 con 0x7f939c0c7180 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a4716640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c155600 con 0x7f939c1527b0 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c155b20 0x7f939c1ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c155b20 0x7f939c1ee2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38104/0 (socket says 172.21.15.26:38104) 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 -- 172.21.15.26:0/1984616991 learned_addr learned my addr 172.21.15.26:0/1984616991 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a24b2640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f939c0c7180 0x7f939c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a1cb1640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f939c1527b0 0x7f939c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 -- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f939c1527b0 msgr2=0x7f939c0c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f939c1527b0 0x7f939c0c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 -- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f939c0c7180 msgr2=0x7f939c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f939c0c7180 0x7f939c158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a2cb3640 1 -- 172.21.15.26:0/1984616991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c1531d0 con 0x7f939c155b20 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.708+0000 7f93a1cb1640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f939c1527b0 0x7f939c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:08.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.709+0000 7f93a2cb3640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c155b20 0x7f939c1ee2d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f939400c7f0 tx=0x7f939400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.709+0000 7f9392ffd640 1 -- 172.21.15.26:0/1984616991 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394018070 con 0x7f939c155b20 2024-03-06T07:29:08.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.709+0000 7f9392ffd640 1 -- 172.21.15.26:0/1984616991 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939400a5e0 con 0x7f939c155b20 2024-03-06T07:29:08.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.709+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c1eecc0 con 0x7f939c155b20 2024-03-06T07:29:08.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.709+0000 7f9392ffd640 1 -- 172.21.15.26:0/1984616991 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400aa10 con 0x7f939c155b20 2024-03-06T07:29:08.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.709+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c0c4730 con 0x7f939c155b20 2024-03-06T07:29:08.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.710+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c04f330 con 0x7f939c155b20 2024-03-06T07:29:08.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.710+0000 7f9392ffd640 1 -- 172.21.15.26:0/1984616991 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f939401b070 con 0x7f939c155b20 2024-03-06T07:29:08.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.711+0000 7f9392ffd640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f938006b4d0 0x7f938006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:08.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.711+0000 7f9392ffd640 1 -- 172.21.15.26:0/1984616991 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9394089560 con 0x7f939c155b20 2024-03-06T07:29:08.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.711+0000 7f93a24b2640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f938006b4d0 0x7f938006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:08.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.712+0000 7f93a24b2640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f938006b4d0 0x7f938006d990 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f939c14c3e0 tx=0x7f9398008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:08.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.715+0000 7f9392ffd640 1 -- 172.21.15.26:0/1984616991 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f939c1531d0 con 0x7f939c155b20 2024-03-06T07:29:08.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.963+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]} v 0) v1 -- 0x7f939c06ce00 con 0x7f939c155b20 2024-03-06T07:29:08.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.966+0000 7f9392ffd640 1 -- 172.21.15.26:0/1984616991 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]}]=0 v514) v1 ==== 142+0+0 (secure 0 0 0) 0x7f939c0c7c60 con 0x7f939c155b20 2024-03-06T07:29:08.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f938006b4d0 msgr2=0x7f938006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f938006b4d0 0x7f938006d990 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f939c14c3e0 tx=0x7f9398008040 comp rx=0 tx=0).stop 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c155b20 msgr2=0x7f939c1ee2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c155b20 0x7f939c1ee2d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f939400c7f0 tx=0x7f939400ccb0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 shutdown_connections 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f938006b4d0 0x7f938006d990 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c155b20 0x7f939c1ee2d0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f939c1527b0 0x7f939c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 --2- 172.21.15.26:0/1984616991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f939c0c7180 0x7f939c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 >> 172.21.15.26:0/1984616991 conn(0x7f939c0ac560 msgr2=0x7f939c0c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 shutdown_connections 2024-03-06T07:29:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:08.970+0000 7f93a4716640 1 -- 172.21.15.26:0/1984616991 wait complete. 2024-03-06T07:29:08.982 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:29:09.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.085+0000 7f5851779640 1 Processor -- start 2024-03-06T07:29:09.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.085+0000 7f5851779640 1 -- start start 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f5851779640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f584c1528d0 0x7f584c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f5851779640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c072a10 0x7f584c155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f5851779640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f584c0c67f0 0x7f584c155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f5851779640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c152cb0 con 0x7f584c0c67f0 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f5851779640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c155ca0 con 0x7f584c1528d0 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f5851779640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0c38b0 con 0x7f584c072a10 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f584c0c67f0 0x7f584c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f584c0c67f0 0x7f584c155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38106/0 (socket says 172.21.15.26:38106) 2024-03-06T07:29:09.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 -- 172.21.15.26:0/2293757318 learned_addr learned my addr 172.21.15.26:0/2293757318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584affd640 1 --2- 172.21.15.26:0/2293757318 >> v2:172.21.15.117:3300/0 conn(0x7f584c1528d0 0x7f584c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584a7fc640 1 --2- 172.21.15.26:0/2293757318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c072a10 0x7f584c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 -- 172.21.15.26:0/2293757318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c072a10 msgr2=0x7f584c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 --2- 172.21.15.26:0/2293757318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c072a10 0x7f584c155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 -- 172.21.15.26:0/2293757318 >> v2:172.21.15.117:3300/0 conn(0x7f584c1528d0 msgr2=0x7f584c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 --2- 172.21.15.26:0/2293757318 >> v2:172.21.15.117:3300/0 conn(0x7f584c1528d0 0x7f584c0724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584b7fe640 1 -- 172.21.15.26:0/2293757318 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c1594f0 con 0x7f584c0c67f0 2024-03-06T07:29:09.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.086+0000 7f584affd640 1 --2- 172.21.15.26:0/2293757318 >> v2:172.21.15.117:3300/0 conn(0x7f584c1528d0 0x7f584c0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:09.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f584b7fe640 1 --2- 172.21.15.26:0/2293757318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c0c67f0 0x7f584c155760 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f584c06e3e0 tx=0x7f584402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17a60204c94e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5849ffb640 1 -- 172.21.15.26:0/2293757318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584402cb00 con 0x7f584c0c67f0 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5849ffb640 1 -- 172.21.15.26:0/2293757318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584402cca0 con 0x7f584c0c67f0 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5849ffb640 1 -- 172.21.15.26:0/2293757318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844034610 con 0x7f584c0c67f0 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5851779640 1 -- 172.21.15.26:0/2293757318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c0c67f0 msgr2=0x7f584c155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5851779640 1 --2- 172.21.15.26:0/2293757318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c0c67f0 0x7f584c155760 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f584c06e3e0 tx=0x7f584402bf60 comp rx=0 tx=0).stop 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5851779640 1 -- 172.21.15.26:0/2293757318 shutdown_connections 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5851779640 1 --2- 172.21.15.26:0/2293757318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c072a10 0x7f584c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5851779640 1 --2- 172.21.15.26:0/2293757318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c0c67f0 0x7f584c155760 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5851779640 1 --2- 172.21.15.26:0/2293757318 >> v2:172.21.15.117:3300/0 conn(0x7f584c1528d0 0x7f584c0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.087+0000 7f5851779640 1 -- 172.21.15.26:0/2293757318 >> 172.21.15.26:0/2293757318 conn(0x7f584c0aa6a0 msgr2=0x7f584c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.088+0000 7f5851779640 1 -- 172.21.15.26:0/2293757318 shutdown_connections 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.088+0000 7f5851779640 1 -- 172.21.15.26:0/2293757318 wait complete. 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 Processor -- start 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 -- start start 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584c072a10 0x7f584c0abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c0c67f0 0x7f584c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c1528d0 0x7f584c1eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0a8250 con 0x7f584c1528d0 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0c3390 con 0x7f584c072a10 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f5851779640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c1591a0 con 0x7f584c0c67f0 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c1528d0 0x7f584c1eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.089+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c1528d0 0x7f584c1eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38110/0 (socket says 172.21.15.26:38110) 2024-03-06T07:29:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584b7fe640 1 -- 172.21.15.26:0/1231249366 learned_addr learned my addr 172.21.15.26:0/1231249366 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584affd640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584c072a10 0x7f584c0abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584a7fc640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c0c67f0 0x7f584c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584b7fe640 1 -- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c0c67f0 msgr2=0x7f584c1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584b7fe640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c0c67f0 0x7f584c1550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584b7fe640 1 -- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584c072a10 msgr2=0x7f584c0abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584b7fe640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584c072a10 0x7f584c0abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584b7fe640 1 -- 172.21.15.26:0/1231249366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c0ad960 con 0x7f584c1528d0 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584affd640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584c072a10 0x7f584c0abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f584b7fe640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c1528d0 0x7f584c1eab60 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f5844009f40 tx=0x7f5844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f58337fe640 1 -- 172.21.15.26:0/1231249366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844043020 con 0x7f584c1528d0 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f58337fe640 1 -- 172.21.15.26:0/1231249366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584402ce50 con 0x7f584c1528d0 2024-03-06T07:29:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c0adc40 con 0x7f584c1528d0 2024-03-06T07:29:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f58337fe640 1 -- 172.21.15.26:0/1231249366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844009510 con 0x7f584c1528d0 2024-03-06T07:29:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.090+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c0ae200 con 0x7f584c1528d0 2024-03-06T07:29:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.091+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c04f330 con 0x7f584c1528d0 2024-03-06T07:29:09.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.092+0000 7f58337fe640 1 -- 172.21.15.26:0/1231249366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5844046020 con 0x7f584c1528d0 2024-03-06T07:29:09.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.092+0000 7f58337fe640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f583806b4d0 0x7f583806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.092+0000 7f58337fe640 1 -- 172.21.15.26:0/1231249366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f58440ac790 con 0x7f584c1528d0 2024-03-06T07:29:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.093+0000 7f584affd640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f583806b4d0 0x7f583806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.093+0000 7f584affd640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f583806b4d0 0x7f583806d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f583c0088b0 tx=0x7f583c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.097+0000 7f58337fe640 1 -- 172.21.15.26:0/1231249366 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f584404d030 con 0x7f584c1528d0 2024-03-06T07:29:09.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.276+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f584c0c2e70 con 0x7f584c1528d0 2024-03-06T07:29:09.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.279+0000 7f58337fe640 1 -- 172.21.15.26:0/1231249366 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v63)=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7f584407b860 con 0x7f584c1528d0 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f583806b4d0 msgr2=0x7f583806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f583806b4d0 0x7f583806d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f583c0088b0 tx=0x7f583c008040 comp rx=0 tx=0).stop 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c1528d0 msgr2=0x7f584c1eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c1528d0 0x7f584c1eab60 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f5844009f40 tx=0x7f5844004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 shutdown_connections 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f583806b4d0 0x7f583806d990 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584c1528d0 0x7f584c1eab60 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f584c0c67f0 0x7f584c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 --2- 172.21.15.26:0/1231249366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584c072a10 0x7f584c0abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 >> 172.21.15.26:0/1231249366 conn(0x7f584c0aa6a0 msgr2=0x7f584c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 shutdown_connections 2024-03-06T07:29:09.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.281+0000 7f5851779640 1 -- 172.21.15.26:0/1231249366 wait complete. 2024-03-06T07:29:09.293 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:29:09.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:29:09.341 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:29:09.386 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:29:09.386 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:29:09.386 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:29:09.401 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:29:09.401 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:29:09.401 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:29:09.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:29:09.419 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:29:09.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.540+0000 7f96acb81640 1 Processor -- start 2024-03-06T07:29:09.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.540+0000 7f96acb81640 1 -- start start 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96acb81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 0x7f96a8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96acb81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a80c73d0 con 0x7f96a80c88c0 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96acb81640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a80be2e0 con 0x7f96a8158b60 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96acb81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8159700 con 0x7f96a814f360 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96a5d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f96a80c88c0 0x7f96a80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49542/0 (socket says 172.21.15.26:49542) 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96a5d9b640 1 -- 172.21.15.26:0/2959774936 learned_addr learned my addr 172.21.15.26:0/2959774936 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96a659c640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 0x7f96a8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.541+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021876230 0 0) 0x7f96a80c73d0 con 0x7f96a80c88c0 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9690003520 con 0x7f96a80c88c0 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2613963145 0 0) 0x7f9690003520 con 0x7f96a80c88c0 2024-03-06T07:29:09.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 msgr2=0x7f96a8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 0x7f96a8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 >> v1:172.21.15.117:6789/0 conn(0x7f96a8158b60 legacy=0x7f96a8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80c7670 con 0x7f96a80c88c0 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f969c001670 con 0x7f96a80c88c0 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 >> v1:172.21.15.26:6789/0 conn(0x7f96a80c88c0 legacy=0x7f96a80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 0x7f96900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690003f20 0x7f9690004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9690004d40 0x7f96900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8159700 con 0x7f9690004d40 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969000bad0 con 0x7f9690003f20 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969000bcd0 con 0x7f96a814f360 2024-03-06T07:29:09.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a6d9d640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9690004d40 0x7f96900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a659c640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 0x7f96900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a5d9b640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690003f20 0x7f9690004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a6d9d640 1 -- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 msgr2=0x7f96900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a6d9d640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a814f360 0x7f96900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a6d9d640 1 -- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690003f20 msgr2=0x7f9690004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a6d9d640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690003f20 0x7f9690004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.542+0000 7f96a6d9d640 1 -- 172.21.15.26:0/2959774936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690003520 con 0x7f9690004d40 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.543+0000 7f96a6d9d640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9690004d40 0x7f96900051c0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9698001f60 tx=0x7f9690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a8f0df3633c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.543+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698001b30 con 0x7f9690004d40 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.543+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969802d710 con 0x7f9690004d40 2024-03-06T07:29:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.543+0000 7f96a559a640 1 -- 172.21.15.26:0/2959774936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969802db00 con 0x7f9690004d40 2024-03-06T07:29:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.544+0000 7f96acb81640 1 -- 172.21.15.26:0/2959774936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9690004d40 msgr2=0x7f96900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.544+0000 7f96acb81640 1 --2- 172.21.15.26:0/2959774936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9690004d40 0x7f96900051c0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9698001f60 tx=0x7f9690005830 comp rx=0 tx=0).stop 2024-03-06T07:29:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.544+0000 7f96a6d9d640 1 -- 172.21.15.26:0/2959774936 reap_dead start 2024-03-06T07:29:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.544+0000 7f96acb81640 1 -- 172.21.15.26:0/2959774936 shutdown_connections 2024-03-06T07:29:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.544+0000 7f96acb81640 1 -- 172.21.15.26:0/2959774936 >> 172.21.15.26:0/2959774936 conn(0x7f96a80ac570 msgr2=0x7f96a80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.544+0000 7f96acb81640 1 -- 172.21.15.26:0/2959774936 shutdown_connections 2024-03-06T07:29:09.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.544+0000 7f96acb81640 1 -- 172.21.15.26:0/2959774936 wait complete. 2024-03-06T07:29:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.545+0000 7f96acb81640 1 Processor -- start 2024-03-06T07:29:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.545+0000 7f96acb81640 1 -- start start 2024-03-06T07:29:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96acb81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690004d40 0x7f96a80ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96acb81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a80c0ff0 0x7f96a81eb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96acb81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a814c010 0x7f96a81ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96acb81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80c7c30 con 0x7f96a814c010 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96acb81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80c5a50 con 0x7f9690004d40 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96acb81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80c5bd0 con 0x7f96a80c0ff0 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a814c010 0x7f96a81ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a814c010 0x7f96a81ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38128/0 (socket says 172.21.15.26:38128) 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 -- 172.21.15.26:0/1781134332 learned_addr learned my addr 172.21.15.26:0/1781134332 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a659c640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690004d40 0x7f96a80ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a5d9b640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a80c0ff0 0x7f96a81eb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 -- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a80c0ff0 msgr2=0x7f96a81eb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a80c0ff0 0x7f96a81eb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 -- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690004d40 msgr2=0x7f96a80ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690004d40 0x7f96a80ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 -- 172.21.15.26:0/1781134332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a81eb770 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a5d9b640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a80c0ff0 0x7f96a81eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96a6d9d640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a814c010 0x7f96a81ee640 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9698008fd0 tx=0x7f96980015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f9696ffd640 1 -- 172.21.15.26:0/1781134332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698001b30 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f9696ffd640 1 -- 172.21.15.26:0/1781134332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9698045070 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a80c0200 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.546+0000 7f9696ffd640 1 -- 172.21.15.26:0/1781134332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698003bb0 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.547+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a80c07c0 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.548+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a804f330 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.548+0000 7f9696ffd640 1 -- 172.21.15.26:0/1781134332 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9698005070 con 0x7f96a814c010 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.548+0000 7f9696ffd640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f968806b500 0x7f968806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.548+0000 7f9696ffd640 1 -- 172.21.15.26:0/1781134332 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f96980ac030 con 0x7f96a814c010 2024-03-06T07:29:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.549+0000 7f96a659c640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f968806b500 0x7f968806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.549+0000 7f96a659c640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f968806b500 0x7f968806d9c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009bc0 tx=0x7f96a0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.553+0000 7f9696ffd640 1 -- 172.21.15.26:0/1781134332 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96a81eb770 con 0x7f96a814c010 2024-03-06T07:29:09.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.774+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96a80c8d50 con 0x7f96a814c010 2024-03-06T07:29:09.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.775+0000 7f9696ffd640 1 -- 172.21.15.26:0/1781134332 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+3711 (secure 0 0 0) 0x7f969807b100 con 0x7f96a814c010 2024-03-06T07:29:09.777 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:09.777 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":194,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2181907552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2181907552},{"type":"v1","addr":"172.21.15.117:6836","nonce":2181907552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":8129,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2243060002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2243060002},{"type":"v1","addr":"172.21.15.117:6837","nonce":2243060002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":9529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4068022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":4068022212},{"type":"v1","addr":"172.21.15.26:6834","nonce":4068022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":10563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2089138655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2089138655},{"type":"v1","addr":"172.21.15.26:6835","nonce":2089138655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":192}],"filesystems":[]} 2024-03-06T07:29:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 194 2024-03-06T07:29:09.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f968806b500 msgr2=0x7f968806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f968806b500 0x7f968806d9c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009bc0 tx=0x7f96a0009190 comp rx=0 tx=0).stop 2024-03-06T07:29:09.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a814c010 msgr2=0x7f96a81ee640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a814c010 0x7f96a81ee640 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9698008fd0 tx=0x7f96980015b0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 shutdown_connections 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f968806b500 0x7f968806d9c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a814c010 0x7f96a81ee640 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a80c0ff0 0x7f96a81eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 --2- 172.21.15.26:0/1781134332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690004d40 0x7f96a80ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 >> 172.21.15.26:0/1781134332 conn(0x7f96a80ac570 msgr2=0x7f96a80c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 shutdown_connections 2024-03-06T07:29:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.777+0000 7f96acb81640 1 -- 172.21.15.26:0/1781134332 wait complete. 2024-03-06T07:29:09.789 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:29:09.790 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:29:09.790 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:29:09.790 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:09.790 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:29:09.790 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:29:09.791 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:29:09.791 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:09.791 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:29:09.791 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:29:09.791 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:29:09.791 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:09.791 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:29:09.792 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:29:09.792 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:29:09.792 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:09.799 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:29:09.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:29:09.800 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:29:09.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:29:09.802 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:29:09.803 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:29:09.804 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:29:09.804 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:29:09.842 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:29:09.842 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:09.843 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:29:09.843 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:09.845 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:29:09.845 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:09.845 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:29:09.846 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:09.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:29:09.919 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-03-06T07:29:09.919 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:29:09.927 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-06T07:29:09.927 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-06T07:29:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58c1313640 1 Processor -- start 2024-03-06T07:29:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58c1313640 1 -- start start 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58c1313640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58bc152890 0x7f58bc159450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58c1313640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58bc0c1170 0x7f58bc155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58c1313640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc1589b0 con 0x7f58bc072120 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58c1313640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc1562f0 con 0x7f58bc152890 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58c1313640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc0c1800 con 0x7f58bc0c1170 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58baffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58bc152890 0x7f58bc159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.952+0000 7f58baffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58bc152890 0x7f58bc159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38162/0 (socket says 172.21.15.26:38162) 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.953+0000 7f58baffd640 1 -- 172.21.15.26:0/155377248 learned_addr learned my addr 172.21.15.26:0/155377248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.953+0000 7f58ba7fc640 1 --2- 172.21.15.26:0/155377248 >> v2:172.21.15.117:3300/0 conn(0x7f58bc0c1170 0x7f58bc155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.953+0000 7f58baffd640 1 -- 172.21.15.26:0/155377248 >> v2:172.21.15.117:3300/0 conn(0x7f58bc0c1170 msgr2=0x7f58bc155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.953+0000 7f58baffd640 1 --2- 172.21.15.26:0/155377248 >> v2:172.21.15.117:3300/0 conn(0x7f58bc0c1170 0x7f58bc155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.953+0000 7f58baffd640 1 -- 172.21.15.26:0/155377248 >> v1:172.21.15.117:6789/0 conn(0x7f58bc072120 legacy=0x7f58bc0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.953+0000 7f58baffd640 1 -- 172.21.15.26:0/155377248 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc158c50 con 0x7f58bc152890 2024-03-06T07:29:09.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58baffd640 1 --2- 172.21.15.26:0/155377248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc152890 0x7f58bc159450 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0c8690 tx=0x7f58b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffdb50292f73e56 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58b9ffb640 1 -- 172.21.15.26:0/155377248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4035020 con 0x7f58bc152890 2024-03-06T07:29:09.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58b9ffb640 1 -- 172.21.15.26:0/155377248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4031870 con 0x7f58bc152890 2024-03-06T07:29:09.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58b9ffb640 1 -- 172.21.15.26:0/155377248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4031c40 con 0x7f58bc152890 2024-03-06T07:29:09.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 -- 172.21.15.26:0/155377248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc152890 msgr2=0x7f58bc159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 --2- 172.21.15.26:0/155377248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc152890 0x7f58bc159450 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0c8690 tx=0x7f58b402b910 comp rx=0 tx=0).stop 2024-03-06T07:29:09.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 -- 172.21.15.26:0/155377248 shutdown_connections 2024-03-06T07:29:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 --2- 172.21.15.26:0/155377248 >> v2:172.21.15.117:3300/0 conn(0x7f58bc0c1170 0x7f58bc155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 --2- 172.21.15.26:0/155377248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc152890 0x7f58bc159450 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 -- 172.21.15.26:0/155377248 >> 172.21.15.26:0/155377248 conn(0x7f58bc0c7de0 msgr2=0x7f58bc0c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 -- 172.21.15.26:0/155377248 shutdown_connections 2024-03-06T07:29:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.954+0000 7f58c1313640 1 -- 172.21.15.26:0/155377248 wait complete. 2024-03-06T07:29:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 Processor -- start 2024-03-06T07:29:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 -- start start 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc072120 0x7f58bc150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc0c1170 0x7f58bc1e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc152890 0x7f58bc14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0c5050 con 0x7f58bc0c1170 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc152c70 con 0x7f58bc072120 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58c1313640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1e1e00 con 0x7f58bc152890 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.956+0000 7f58ba7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc0c1170 0x7f58bc1e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc0c1170 0x7f58bc1e11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38174/0 (socket says 172.21.15.26:38174) 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 -- 172.21.15.26:0/1581341861 learned_addr learned my addr 172.21.15.26:0/1581341861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58baffd640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc072120 0x7f58bc150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58bb7fe640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc152890 0x7f58bc14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 -- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc152890 msgr2=0x7f58bc14c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc152890 0x7f58bc14c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 -- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc072120 msgr2=0x7f58bc150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc072120 0x7f58bc150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 -- 172.21.15.26:0/1581341861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc1e2030 con 0x7f58bc0c1170 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58ba7fc640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc0c1170 0x7f58bc1e11b0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f58a400e9e0 tx=0x7f58a400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58b37fe640 1 -- 172.21.15.26:0/1581341861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4017070 con 0x7f58bc0c1170 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58b37fe640 1 -- 172.21.15.26:0/1581341861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a400c9d0 con 0x7f58bc0c1170 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc1e2310 con 0x7f58bc0c1170 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58b37fe640 1 -- 172.21.15.26:0/1581341861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a400cdc0 con 0x7f58bc0c1170 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.957+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1decb0 con 0x7f58bc0c1170 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.958+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc04f330 con 0x7f58bc0c1170 2024-03-06T07:29:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.959+0000 7f58b37fe640 1 -- 172.21.15.26:0/1581341861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58a401a070 con 0x7f58bc0c1170 2024-03-06T07:29:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.959+0000 7f58b37fe640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f589c06b500 0x7f589c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.959+0000 7f58b37fe640 1 -- 172.21.15.26:0/1581341861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f58a408a340 con 0x7f58bc0c1170 2024-03-06T07:29:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.959+0000 7f58baffd640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f589c06b500 0x7f589c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.960+0000 7f58baffd640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f589c06b500 0x7f589c06d9c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f58b4036000 tx=0x7f58b403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:09.964+0000 7f58b37fe640 1 -- 172.21.15.26:0/1581341861 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f58a4058d30 con 0x7f58bc0c1170 2024-03-06T07:29:10.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.161+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f58bc06f270 con 0x7f58bc0c1170 2024-03-06T07:29:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.981+0000 7f58b37fe640 1 -- 172.21.15.26:0/1581341861 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v207) v1 ==== 97+0+0 (secure 0 0 0) 0x7f58a405cdd0 con 0x7f58bc0c1170 2024-03-06T07:29:10.983 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-03-06T07:29:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f589c06b500 msgr2=0x7f589c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f589c06b500 0x7f589c06d9c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f58b4036000 tx=0x7f58b403c040 comp rx=0 tx=0).stop 2024-03-06T07:29:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc0c1170 msgr2=0x7f58bc1e11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc0c1170 0x7f58bc1e11b0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f58a400e9e0 tx=0x7f58a400eea0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 shutdown_connections 2024-03-06T07:29:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f589c06b500 0x7f589c06d9c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc152890 0x7f58bc14c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58bc0c1170 0x7f58bc1e11b0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 --2- 172.21.15.26:0/1581341861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc072120 0x7f58bc150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 >> 172.21.15.26:0/1581341861 conn(0x7f58bc0c7de0 msgr2=0x7f58bc1543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 shutdown_connections 2024-03-06T07:29:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:10.984+0000 7f58c1313640 1 -- 172.21.15.26:0/1581341861 wait complete. 2024-03-06T07:29:10.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:29:11.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.105+0000 7f43da4df640 1 Processor -- start 2024-03-06T07:29:11.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.105+0000 7f43da4df640 1 -- start start 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43da4df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43d4150290 0x7f43d4150710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43da4df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d40c3d60 0x7f43d40c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43da4df640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d4156f90 con 0x7f43d4072720 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43da4df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d40c38d0 con 0x7f43d4150290 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43da4df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4157270 con 0x7f43d40c3d60 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43d4150290 0x7f43d4150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43d4150290 0x7f43d4150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38180/0 (socket says 172.21.15.26:38180) 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d37fe640 1 -- 172.21.15.26:0/859516232 learned_addr learned my addr 172.21.15.26:0/859516232 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d8a7c640 1 --2- 172.21.15.26:0/859516232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d40c3d60 0x7f43d40c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d37fe640 1 -- 172.21.15.26:0/859516232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d40c3d60 msgr2=0x7f43d40c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d37fe640 1 --2- 172.21.15.26:0/859516232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d40c3d60 0x7f43d40c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d37fe640 1 -- 172.21.15.26:0/859516232 >> v1:172.21.15.117:6789/0 conn(0x7f43d4072720 legacy=0x7f43d41564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d37fe640 1 -- 172.21.15.26:0/859516232 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40c3280 con 0x7f43d4150290 2024-03-06T07:29:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.106+0000 7f43d8a7c640 1 --2- 172.21.15.26:0/859516232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d40c3d60 0x7f43d40c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.107+0000 7f43d37fe640 1 --2- 172.21.15.26:0/859516232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d4150290 0x7f43d4150710 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009870 tx=0x7f43c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d490999f706c7122 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.107+0000 7f43d2ffd640 1 -- 172.21.15.26:0/859516232 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4035020 con 0x7f43d4150290 2024-03-06T07:29:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.107+0000 7f43d2ffd640 1 -- 172.21.15.26:0/859516232 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4031640 con 0x7f43d4150290 2024-03-06T07:29:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.107+0000 7f43d2ffd640 1 -- 172.21.15.26:0/859516232 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c402d040 con 0x7f43d4150290 2024-03-06T07:29:11.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 -- 172.21.15.26:0/859516232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d4150290 msgr2=0x7f43d4150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 --2- 172.21.15.26:0/859516232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d4150290 0x7f43d4150710 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009870 tx=0x7f43c402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 -- 172.21.15.26:0/859516232 shutdown_connections 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 --2- 172.21.15.26:0/859516232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d40c3d60 0x7f43d40c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 --2- 172.21.15.26:0/859516232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d4150290 0x7f43d4150710 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 -- 172.21.15.26:0/859516232 >> 172.21.15.26:0/859516232 conn(0x7f43d40aa630 msgr2=0x7f43d40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 -- 172.21.15.26:0/859516232 shutdown_connections 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.108+0000 7f43da4df640 1 -- 172.21.15.26:0/859516232 wait complete. 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.109+0000 7f43da4df640 1 Processor -- start 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.109+0000 7f43da4df640 1 -- start start 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43da4df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d4072720 0x7f43d40c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43da4df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d40c3d60 0x7f43d41ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43da4df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d4150290 0x7f43d41eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43da4df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4156bd0 con 0x7f43d40c3d60 2024-03-06T07:29:11.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43da4df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d406f8f0 con 0x7f43d4150290 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43da4df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4156d50 con 0x7f43d4072720 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d40c3d60 0x7f43d41ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d40c3d60 0x7f43d41ee670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38186/0 (socket says 172.21.15.26:38186) 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 -- 172.21.15.26:0/3413106887 learned_addr learned my addr 172.21.15.26:0/3413106887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d3fff640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d4072720 0x7f43d40c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d8a7c640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d4150290 0x7f43d41eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 -- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d4072720 msgr2=0x7f43d40c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d4072720 0x7f43d40c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 -- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d4150290 msgr2=0x7f43d41eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d4150290 0x7f43d41eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 -- 172.21.15.26:0/3413106887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40c8640 con 0x7f43d40c3d60 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.110+0000 7f43d37fe640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d40c3d60 0x7f43d41ee670 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f43c4000c00 tx=0x7f43c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.111+0000 7f43d0ff9640 1 -- 172.21.15.26:0/3413106887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4040070 con 0x7f43d40c3d60 2024-03-06T07:29:11.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.111+0000 7f43d0ff9640 1 -- 172.21.15.26:0/3413106887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4039bf0 con 0x7f43d40c3d60 2024-03-06T07:29:11.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.111+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d4150fa0 con 0x7f43d40c3d60 2024-03-06T07:29:11.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.111+0000 7f43d0ff9640 1 -- 172.21.15.26:0/3413106887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4035040 con 0x7f43d40c3d60 2024-03-06T07:29:11.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.111+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d41eb790 con 0x7f43d40c3d60 2024-03-06T07:29:11.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.112+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d404f330 con 0x7f43d40c3d60 2024-03-06T07:29:11.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.112+0000 7f43d0ff9640 1 -- 172.21.15.26:0/3413106887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43c404b060 con 0x7f43d40c3d60 2024-03-06T07:29:11.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.113+0000 7f43d0ff9640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f43c006b500 0x7f43c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.113+0000 7f43d0ff9640 1 -- 172.21.15.26:0/3413106887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f43c40ab900 con 0x7f43d40c3d60 2024-03-06T07:29:11.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.113+0000 7f43d3fff640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f43c006b500 0x7f43c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.113+0000 7f43d3fff640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f43c006b500 0x7f43c006d9c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f43c8001640 tx=0x7f43c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.117+0000 7f43d0ff9640 1 -- 172.21.15.26:0/3413106887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f43c402f300 con 0x7f43d40c3d60 2024-03-06T07:29:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.371+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f43d41ef0f0 con 0x7f43d40c3d60 2024-03-06T07:29:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.372+0000 7f43d0ff9640 1 -- 172.21.15.26:0/3413106887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f43c407a940 con 0x7f43d40c3d60 2024-03-06T07:29:11.373 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:11.373 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:29:11.373 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-03-06T07:29:11.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlwFmeEhAA3Na7TJJDcop4ifee8FTPqw==", 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhliczsHhAAclzrtGw2rJvnCYCzwZhX5Q==", 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlHab6GRAAQRjrNzBTQuCNWfrw44YdOQ==", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlMr1ZJhAApG4BaKoLPAnY1crPwWArUg==", 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.376 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDzGOhlxRHMBBAAFuyGnGCZQZEVqoJh5OZsoQ==", 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD2GOhlKNIMBBAAy5P2UhqpsmvpKeVKcBaEHQ==", 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD5GOhlk4z2AhAAgO7zhOhHqM6+VeDTMQZmWw==", 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQD8GOhlraoGARAAiTa6jrXZgMZtDZW/pxZDCA==", 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQACGehltr+3CRAA50BxiW1nl/+9nZIsrhZs7Q==", 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAFGehlkj1rCBAAxWttp5Lu5eZdCWMisa44yA==", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQAIGehlncibBxAAh3UYb0SbBMrWT6rA5BSN1g==", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQALGehlbXqfBRAAiOBXTiOePdj2TPEZZA6v3w==", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA==", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlkigLLhAAiuDr8CnTxj633QRJN7YYDg==", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDwGOhlISSsMRAAwyg0bnc4SJMKlkYwrB9jdQ==", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhlxNYvDxAAQ9orXDTQbVF4jXpvF69NGg==", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDxGOhl9s9UCxAAxdBu13nPcIziC7Hvwme8DA==", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.373+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f43c006b500 msgr2=0x7f43c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.373+0000 7f43da4df640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f43c006b500 0x7f43c006d9c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f43c8001640 tx=0x7f43c8009040 comp rx=0 tx=0).stop 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.373+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d40c3d60 msgr2=0x7f43d41ee670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.373+0000 7f43da4df640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d40c3d60 0x7f43d41ee670 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f43c4000c00 tx=0x7f43c40046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 shutdown_connections 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f43c006b500 0x7f43c006d9c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d4150290 0x7f43d41eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43d40c3d60 0x7f43d41ee670 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 --2- 172.21.15.26:0/3413106887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d4072720 0x7f43d40c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 >> 172.21.15.26:0/3413106887 conn(0x7f43d40aa630 msgr2=0x7f43d40a9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 shutdown_connections 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.374+0000 7f43da4df640 1 -- 172.21.15.26:0/3413106887 wait complete. 2024-03-06T07:29:11.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.488+0000 7ff469eec640 1 Processor -- start 2024-03-06T07:29:11.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.488+0000 7ff469eec640 1 -- start start 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff469eec640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 0x7ff464155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff469eec640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff4640c8a00 0x7ff464158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff469eec640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff464156180 con 0x7ff464159460 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff469eec640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4641587b0 con 0x7ff4640c8a00 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff469eec640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640c7760 con 0x7ff464152920 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff463fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff464159460 0x7ff4640c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49562/0 (socket says 172.21.15.26:49562) 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff463fff640 1 -- 172.21.15.26:0/469971946 learned_addr learned my addr 172.21.15.26:0/469971946 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288003836 0 0) 0x7ff45c001490 con 0x7ff464159460 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff462ffd640 1 --2- 172.21.15.26:0/469971946 >> v2:172.21.15.117:3300/0 conn(0x7ff4640c8a00 0x7ff464158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4637fe640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 0x7ff464155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff44c003520 con 0x7ff464159460 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2437796576 0 0) 0x7ff44c003520 con 0x7ff464159460 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 msgr2=0x7ff464155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 0x7ff464155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 >> v2:172.21.15.117:3300/0 conn(0x7ff4640c8a00 msgr2=0x7ff464158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 --2- 172.21.15.26:0/469971946 >> v2:172.21.15.117:3300/0 conn(0x7ff4640c8a00 0x7ff464158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640c7a40 con 0x7ff464159460 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7ff45c001bc0 con 0x7ff464159460 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 >> v1:172.21.15.26:6789/0 conn(0x7ff464159460 legacy=0x7ff4640c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0039b0 0x7ff44c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 0x7ff44c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.489+0000 7ff4627fc640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c004d40 0x7ff44c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4641587b0 con 0x7ff44c0039b0 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640c7760 con 0x7ff44c004d40 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c00bb20 con 0x7ff464152920 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4637fe640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0039b0 0x7ff44c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff463fff640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c004d40 0x7ff44c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff462ffd640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 0x7ff44c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4637fe640 1 -- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 msgr2=0x7ff44c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4637fe640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 0x7ff44c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4637fe640 1 -- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c004d40 msgr2=0x7ff44c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4637fe640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c004d40 0x7ff44c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff4637fe640 1 -- 172.21.15.26:0/469971946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c003520 con 0x7ff44c0039b0 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff462ffd640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464152920 0x7ff44c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.490+0000 7ff463fff640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c004d40 0x7ff44c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff4637fe640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0039b0 0x7ff44c003dd0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ba30 tx=0x7ff44c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eac579b0ec926d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454036020 con 0x7ff44c0039b0 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454032640 con 0x7ff44c0039b0 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff4627fc640 1 -- 172.21.15.26:0/469971946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45400b600 con 0x7ff44c0039b0 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff469eec640 1 -- 172.21.15.26:0/469971946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0039b0 msgr2=0x7ff44c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff469eec640 1 --2- 172.21.15.26:0/469971946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0039b0 0x7ff44c003dd0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ba30 tx=0x7ff44c005830 comp rx=0 tx=0).stop 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff463fff640 1 -- 172.21.15.26:0/469971946 reap_dead start 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff469eec640 1 -- 172.21.15.26:0/469971946 shutdown_connections 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff469eec640 1 -- 172.21.15.26:0/469971946 >> 172.21.15.26:0/469971946 conn(0x7ff4640ac560 msgr2=0x7ff4640ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff469eec640 1 -- 172.21.15.26:0/469971946 shutdown_connections 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.491+0000 7ff469eec640 1 -- 172.21.15.26:0/469971946 wait complete. 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.492+0000 7ff469eec640 1 Processor -- start 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff469eec640 1 -- start start 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff469eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0039b0 0x7ff46414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff469eec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4641537f0 0x7ff46414a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff469eec640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464155c70 0x7ff46414e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff469eec640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46406d030 con 0x7ff4641537f0 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff469eec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff464158a00 con 0x7ff44c0039b0 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff469eec640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640c5a40 con 0x7ff464155c70 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4641537f0 0x7ff46414a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4641537f0 0x7ff46414a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38204/0 (socket says 172.21.15.26:38204) 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff462ffd640 1 -- 172.21.15.26:0/4084363891 learned_addr learned my addr 172.21.15.26:0/4084363891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.493+0000 7ff4637fe640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0039b0 0x7ff46414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff463fff640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464155c70 0x7ff46414e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff462ffd640 1 -- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464155c70 msgr2=0x7ff46414e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff462ffd640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464155c70 0x7ff46414e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff462ffd640 1 -- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0039b0 msgr2=0x7ff46414d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff462ffd640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0039b0 0x7ff46414d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff462ffd640 1 -- 172.21.15.26:0/4084363891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464153030 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff462ffd640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4641537f0 0x7ff46414a060 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7ff45800f580 tx=0x7ff45800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff44bfff640 1 -- 172.21.15.26:0/4084363891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458019020 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff44bfff640 1 -- 172.21.15.26:0/4084363891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458014870 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff464153310 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff44bfff640 1 -- 172.21.15.26:0/4084363891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458014c60 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.494+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46414cfd0 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.495+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46404f330 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.495+0000 7ff44bfff640 1 -- 172.21.15.26:0/4084363891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff45801c020 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.496+0000 7ff44bfff640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45006b500 0x7ff45006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.496+0000 7ff44bfff640 1 -- 172.21.15.26:0/4084363891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff45808a730 con 0x7ff4641537f0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.496+0000 7ff4637fe640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45006b500 0x7ff45006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.497+0000 7ff4637fe640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45006b500 0x7ff45006d9c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ba00 tx=0x7ff454030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.500+0000 7ff44bfff640 1 -- 172.21.15.26:0/4084363891 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff458023020 con 0x7ff4641537f0 2024-03-06T07:29:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.681+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff46406cda0 con 0x7ff4641537f0 2024-03-06T07:29:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.682+0000 7ff44bfff640 1 -- 172.21.15.26:0/4084363891 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+13186 (secure 0 0 0) 0x7ff458059760 con 0x7ff4641537f0 2024-03-06T07:29:11.683 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:11.684 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":207,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:29:10.980932+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45006b500 msgr2=0x7ff45006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45006b500 0x7ff45006d9c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ba00 tx=0x7ff454030000 comp rx=0 tx=0).stop 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4641537f0 msgr2=0x7ff46414a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4641537f0 0x7ff46414a060 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7ff45800f580 tx=0x7ff45800fa40 comp rx=0 tx=0).stop 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 shutdown_connections 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff45006b500 0x7ff45006d9c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464155c70 0x7ff46414e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4641537f0 0x7ff46414a060 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 --2- 172.21.15.26:0/4084363891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0039b0 0x7ff46414d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 >> 172.21.15.26:0/4084363891 conn(0x7ff4640ac560 msgr2=0x7ff4640ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 shutdown_connections 2024-03-06T07:29:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.684+0000 7ff469eec640 1 -- 172.21.15.26:0/4084363891 wait complete. 2024-03-06T07:29:11.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:29:11.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:29:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.799+0000 7fed379d7640 1 Processor -- start 2024-03-06T07:29:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.800+0000 7fed379d7640 1 -- start start 2024-03-06T07:29:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.800+0000 7fed379d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed30155c30 0x7fed3006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.800+0000 7fed379d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed300b0450 con 0x7fed300b0070 2024-03-06T07:29:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.800+0000 7fed379d7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed301560e0 con 0x7fed300c5380 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.800+0000 7fed379d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3006cfb0 con 0x7fed30155c30 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.800+0000 7fed34f72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed30155c30 0x7fed3006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.800+0000 7fed35773640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fed300b0070 0x7fed300c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49572/0 (socket says 172.21.15.26:49572) 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.801+0000 7fed35773640 1 -- 172.21.15.26:0/4097752490 learned_addr learned my addr 172.21.15.26:0/4097752490 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.801+0000 7fed34f72640 1 -- 172.21.15.26:0/4097752490 >> v1:172.21.15.117:6789/0 conn(0x7fed300c5380 legacy=0x7fed3014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.801+0000 7fed34f72640 1 -- 172.21.15.26:0/4097752490 >> v1:172.21.15.26:6789/0 conn(0x7fed300b0070 legacy=0x7fed300c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.801+0000 7fed27fff640 1 -- 172.21.15.26:0/4097752490 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644675085 0 0) 0x7fed2c001110 con 0x7fed300b0070 2024-03-06T07:29:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.801+0000 7fed34f72640 1 -- 172.21.15.26:0/4097752490 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300c77f0 con 0x7fed30155c30 2024-03-06T07:29:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed34f72640 1 --2- 172.21.15.26:0/4097752490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed3006ca70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fed200097a0 tx=0x7fed2002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b121b1bf7447ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed27fff640 1 -- 172.21.15.26:0/4097752490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20035020 con 0x7fed30155c30 2024-03-06T07:29:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed27fff640 1 -- 172.21.15.26:0/4097752490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed20031830 con 0x7fed30155c30 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed27fff640 1 -- 172.21.15.26:0/4097752490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031c00 con 0x7fed30155c30 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed379d7640 1 -- 172.21.15.26:0/4097752490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 msgr2=0x7fed3006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed379d7640 1 --2- 172.21.15.26:0/4097752490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed3006ca70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fed200097a0 tx=0x7fed2002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed379d7640 1 -- 172.21.15.26:0/4097752490 shutdown_connections 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed379d7640 1 --2- 172.21.15.26:0/4097752490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed3006ca70 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed379d7640 1 -- 172.21.15.26:0/4097752490 >> 172.21.15.26:0/4097752490 conn(0x7fed300ac590 msgr2=0x7fed300ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed379d7640 1 -- 172.21.15.26:0/4097752490 shutdown_connections 2024-03-06T07:29:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.802+0000 7fed379d7640 1 -- 172.21.15.26:0/4097752490 wait complete. 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.803+0000 7fed379d7640 1 Processor -- start 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.803+0000 7fed379d7640 1 -- start start 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed379d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed300b0070 0x7fed300c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed379d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed300c5380 0x7fed301ec020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed379d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed301ec560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed379d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3006c270 con 0x7fed30155c30 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed379d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3006c020 con 0x7fed300c5380 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed379d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3006c550 con 0x7fed300b0070 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed35f74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed301ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed35f74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed301ec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38212/0 (socket says 172.21.15.26:38212) 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed35f74640 1 -- 172.21.15.26:0/986829691 learned_addr learned my addr 172.21.15.26:0/986829691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed34f72640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed300c5380 0x7fed301ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.804+0000 7fed35773640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed300b0070 0x7fed300c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed35f74640 1 -- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed300b0070 msgr2=0x7fed300c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed35f74640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed300b0070 0x7fed300c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed35f74640 1 -- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed300c5380 msgr2=0x7fed301ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed35f74640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed300c5380 0x7fed301ec020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed35f74640 1 -- 172.21.15.26:0/986829691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300c2580 con 0x7fed30155c30 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed35773640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed300b0070 0x7fed300c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed35f74640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed301ec560 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c7a0 tx=0x7fed2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed25ffb640 1 -- 172.21.15.26:0/986829691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28003b70 con 0x7fed30155c30 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed25ffb640 1 -- 172.21.15.26:0/986829691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28003d30 con 0x7fed30155c30 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed25ffb640 1 -- 172.21.15.26:0/986829691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800aa90 con 0x7fed30155c30 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed300c2860 con 0x7fed30155c30 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.805+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed300c2e20 con 0x7fed30155c30 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.806+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3004f330 con 0x7fed30155c30 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.806+0000 7fed25ffb640 1 -- 172.21.15.26:0/986829691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed2800ac30 con 0x7fed30155c30 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.807+0000 7fed25ffb640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed1406b4d0 0x7fed1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.807+0000 7fed25ffb640 1 -- 172.21.15.26:0/986829691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fed28089e30 con 0x7fed30155c30 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.807+0000 7fed35773640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed1406b4d0 0x7fed1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.807+0000 7fed35773640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed1406b4d0 0x7fed1406d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fed2c007fd0 tx=0x7fed2c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.811+0000 7fed25ffb640 1 -- 172.21.15.26:0/986829691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed28058f00 con 0x7fed30155c30 2024-03-06T07:29:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.991+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed30152b70 con 0x7fed30155c30 2024-03-06T07:29:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.991+0000 7fed25ffb640 1 -- 172.21.15.26:0/986829691 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v207) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed28058800 con 0x7fed30155c30 2024-03-06T07:29:11.993 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:29:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed1406b4d0 msgr2=0x7fed1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed1406b4d0 0x7fed1406d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fed2c007fd0 tx=0x7fed2c007de0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 msgr2=0x7fed301ec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed301ec560 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c7a0 tx=0x7fed2800cc60 comp rx=0 tx=0).stop 2024-03-06T07:29:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 shutdown_connections 2024-03-06T07:29:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fed1406b4d0 0x7fed1406d990 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30155c30 0x7fed301ec560 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed300c5380 0x7fed301ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 --2- 172.21.15.26:0/986829691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed300b0070 0x7fed300c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 >> 172.21.15.26:0/986829691 conn(0x7fed300ac590 msgr2=0x7fed300ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 shutdown_connections 2024-03-06T07:29:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:11.993+0000 7fed379d7640 1 -- 172.21.15.26:0/986829691 wait complete. 2024-03-06T07:29:12.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:29:12.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.105+0000 7f5259d0c640 1 Processor -- start 2024-03-06T07:29:12.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.105+0000 7f5259d0c640 1 -- start start 2024-03-06T07:29:12.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f5259d0c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5254156ea0 0x7f52540c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f5259d0c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52540ad310 con 0x7f5254153990 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f5259d0c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52541572a0 con 0x7f52541506b0 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f5259d0c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5254156450 con 0x7f5254156ea0 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f5252ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5254153990 0x7f52540acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49574/0 (socket says 172.21.15.26:49574) 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f5252ffd640 1 -- 172.21.15.26:0/2964409008 learned_addr learned my addr 172.21.15.26:0/2964409008 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419590593 0 0) 0x7f52540ad310 con 0x7f5254153990 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f523c003520 con 0x7f5254153990 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.106+0000 7f5253fff640 1 --2- 172.21.15.26:0/2964409008 >> v2:172.21.15.117:3300/0 conn(0x7f5254156ea0 0x7f52540c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2687144076 0 0) 0x7f523c003520 con 0x7f5254153990 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 >> v2:172.21.15.117:3300/0 conn(0x7f5254156ea0 msgr2=0x7f52540c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 --2- 172.21.15.26:0/2964409008 >> v2:172.21.15.117:3300/0 conn(0x7f5254156ea0 0x7f52540c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 >> v1:172.21.15.117:6789/0 conn(0x7f52541506b0 legacy=0x7f52540702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5254070aa0 con 0x7f5254153990 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f5248001670 con 0x7f5254153990 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 >> v1:172.21.15.26:6789/0 conn(0x7f5254153990 legacy=0x7f52540acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c0039b0 0x7f523c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c004800 0x7f523c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523c0051c0 0x7f523c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254156450 con 0x7f523c004800 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c00c060 con 0x7f523c0039b0 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c00c260 con 0x7f523c0051c0 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5252ffd640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c004800 0x7f523c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f52537fe640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c0039b0 0x7f523c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5253fff640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523c0051c0 0x7f523c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5252ffd640 1 -- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523c0051c0 msgr2=0x7f523c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5252ffd640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523c0051c0 0x7f523c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5252ffd640 1 -- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c0039b0 msgr2=0x7f523c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5252ffd640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c0039b0 0x7f523c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5252ffd640 1 -- 172.21.15.26:0/2964409008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c003520 con 0x7f523c004800 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5253fff640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523c0051c0 0x7f523c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.107+0000 7f5253fff640 1 -- 172.21.15.26:0/2964409008 reap_dead start 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.108+0000 7f5252ffd640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c004800 0x7f523c004c80 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f524800c240 tx=0x7f523c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbda81a65daf6fa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.108+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248007cf0 con 0x7f523c004800 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.108+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5248030070 con 0x7f523c004800 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.108+0000 7f52527fc640 1 -- 172.21.15.26:0/2964409008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248005d70 con 0x7f523c004800 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.109+0000 7f5259d0c640 1 -- 172.21.15.26:0/2964409008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c004800 msgr2=0x7f523c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.109+0000 7f5259d0c640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c004800 0x7f523c004c80 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f524800c240 tx=0x7f523c005830 comp rx=0 tx=0).stop 2024-03-06T07:29:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.109+0000 7f5259d0c640 1 -- 172.21.15.26:0/2964409008 shutdown_connections 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.109+0000 7f5259d0c640 1 --2- 172.21.15.26:0/2964409008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f523c004800 0x7f523c004c80 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.109+0000 7f5259d0c640 1 -- 172.21.15.26:0/2964409008 >> 172.21.15.26:0/2964409008 conn(0x7f525406b1a0 msgr2=0x7f525406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.109+0000 7f5259d0c640 1 -- 172.21.15.26:0/2964409008 shutdown_connections 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.109+0000 7f5259d0c640 1 -- 172.21.15.26:0/2964409008 wait complete. 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.110+0000 7f5259d0c640 1 Processor -- start 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.110+0000 7f5259d0c640 1 -- start start 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c004800 0x7f5254155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52541ebf00 0x7f52541ec360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52541ec8a0 0x7f525414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52541566a0 con 0x7f52541ebf00 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254156a10 con 0x7f523c004800 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540702c0 con 0x7f52541ec8a0 2024-03-06T07:29:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52541ebf00 0x7f52541ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52541ebf00 0x7f52541ec360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38222/0 (socket says 172.21.15.26:38222) 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 -- 172.21.15.26:0/207206817 learned_addr learned my addr 172.21.15.26:0/207206817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f52537fe640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c004800 0x7f5254155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5253fff640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52541ec8a0 0x7f525414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 -- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52541ec8a0 msgr2=0x7f525414d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52541ec8a0 0x7f525414d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 -- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c004800 msgr2=0x7f5254155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c004800 0x7f5254155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 -- 172.21.15.26:0/207206817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5254143c00 con 0x7f52541ebf00 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5252ffd640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52541ebf00 0x7f52541ec360 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f524800c240 tx=0x7f5254071060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f523bfff640 1 -- 172.21.15.26:0/207206817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248001c30 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f523bfff640 1 -- 172.21.15.26:0/207206817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5248004bf0 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540ab6b0 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f523bfff640 1 -- 172.21.15.26:0/207206817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248007070 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.111+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5254149320 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.112+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5254070650 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.113+0000 7f523bfff640 1 -- 172.21.15.26:0/207206817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5248048020 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.113+0000 7f523bfff640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.114+0000 7f523bfff640 1 -- 172.21.15.26:0/207206817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f52480ab4e0 con 0x7f52541ebf00 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.114+0000 7f52537fe640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.114+0000 7f52537fe640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5244000c00 tx=0x7f5244009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.118+0000 7f523bfff640 1 -- 172.21.15.26:0/207206817 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5254070650 con 0x7f52541ebf00 2024-03-06T07:29:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.300+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f525404f330 con 0x7f52541ebf00 2024-03-06T07:29:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.301+0000 7f523bfff640 1 -- 172.21.15.26:0/207206817 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v207) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5254143c00 con 0x7f52541ebf00 2024-03-06T07:29:12.303 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:12.303 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-03-06T07:29:12.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-03-06T07:29:12.303 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-03-06T07:29:12.303 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-03-06T07:29:12.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-06T07:29:12.303 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 msgr2=0x7f524006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5244000c00 tx=0x7f5244009040 comp rx=0 tx=0).stop 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52541ebf00 msgr2=0x7f52541ec360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52541ebf00 0x7f52541ec360 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f524800c240 tx=0x7f5254071060 comp rx=0 tx=0).stop 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 shutdown_connections 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52541ec8a0 0x7f525414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52541ebf00 0x7f52541ec360 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 --2- 172.21.15.26:0/207206817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c004800 0x7f5254155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 >> 172.21.15.26:0/207206817 conn(0x7f525406b1a0 msgr2=0x7f52540c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 shutdown_connections 2024-03-06T07:29:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.303+0000 7f5259d0c640 1 -- 172.21.15.26:0/207206817 wait complete. 2024-03-06T07:29:12.315 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:29:12.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:29:12.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.419+0000 7fd63db7a640 1 Processor -- start 2024-03-06T07:29:12.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.419+0000 7fd63db7a640 1 -- start start 2024-03-06T07:29:12.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.420+0000 7fd63db7a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6380c88e0 0x7fd6380c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.420+0000 7fd63db7a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380c5400 con 0x7fd6380c7930 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.420+0000 7fd63db7a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380c56e0 con 0x7fd638152870 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.420+0000 7fd63db7a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63806d320 con 0x7fd6380c88e0 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.420+0000 7fd637fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd6380c7930 0x7fd63806bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49588/0 (socket says 172.21.15.26:49588) 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.420+0000 7fd637fff640 1 -- 172.21.15.26:0/428628673 learned_addr learned my addr 172.21.15.26:0/428628673 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd636ffd640 1 --2- 172.21.15.26:0/428628673 >> v2:172.21.15.117:3300/0 conn(0x7fd6380c88e0 0x7fd6380c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984953792 0 0) 0x7fd6380c5400 con 0x7fd6380c7930 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd620003520 con 0x7fd6380c7930 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347139941 0 0) 0x7fd6380c56e0 con 0x7fd638152870 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6380c5400 con 0x7fd638152870 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd636ffd640 1 -- 172.21.15.26:0/428628673 >> v1:172.21.15.117:6789/0 conn(0x7fd638152870 legacy=0x7fd638155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd636ffd640 1 -- 172.21.15.26:0/428628673 >> v1:172.21.15.26:6789/0 conn(0x7fd6380c7930 legacy=0x7fd63806bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3175343232 0 0) 0x7fd620003520 con 0x7fd6380c7930 2024-03-06T07:29:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.421+0000 7fd636ffd640 1 -- 172.21.15.26:0/428628673 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380c59c0 con 0x7fd6380c88e0 2024-03-06T07:29:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd636ffd640 1 --2- 172.21.15.26:0/428628673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6380c88e0 0x7fd6380c73f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0097a0 tx=0x7fd62c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea40f0dcf22f251 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c037070 con 0x7fd6380c88e0 2024-03-06T07:29:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c030510 con 0x7fd6380c88e0 2024-03-06T07:29:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd6367fc640 1 -- 172.21.15.26:0/428628673 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6300019f0 con 0x7fd6380c88e0 2024-03-06T07:29:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd63db7a640 1 -- 172.21.15.26:0/428628673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6380c88e0 msgr2=0x7fd6380c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd63db7a640 1 --2- 172.21.15.26:0/428628673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6380c88e0 0x7fd6380c73f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0097a0 tx=0x7fd62c02b790 comp rx=0 tx=0).stop 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd63db7a640 1 -- 172.21.15.26:0/428628673 shutdown_connections 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd63db7a640 1 --2- 172.21.15.26:0/428628673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6380c88e0 0x7fd6380c73f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.422+0000 7fd63db7a640 1 -- 172.21.15.26:0/428628673 >> 172.21.15.26:0/428628673 conn(0x7fd6380ac5f0 msgr2=0x7fd6380aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.423+0000 7fd63db7a640 1 -- 172.21.15.26:0/428628673 shutdown_connections 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.423+0000 7fd63db7a640 1 -- 172.21.15.26:0/428628673 wait complete. 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 Processor -- start 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 -- start start 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6380c7930 0x7fd638141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6380c88e0 0x7fd6380b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd638152870 0x7fd6381ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380c9470 con 0x7fd6380c7930 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63806b8d0 con 0x7fd638152870 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd63db7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380c95f0 con 0x7fd6380c88e0 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd6377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6380c7930 0x7fd638141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.424+0000 7fd6377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6380c7930 0x7fd638141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38232/0 (socket says 172.21.15.26:38232) 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd6377fe640 1 -- 172.21.15.26:0/2164967000 learned_addr learned my addr 172.21.15.26:0/2164967000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd636ffd640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6380c88e0 0x7fd6380b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd637fff640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd638152870 0x7fd6381ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd6377fe640 1 -- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6380c88e0 msgr2=0x7fd6380b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd6377fe640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6380c88e0 0x7fd6380b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd6377fe640 1 -- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd638152870 msgr2=0x7fd6381ebe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd6377fe640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd638152870 0x7fd6381ebe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd6377fe640 1 -- 172.21.15.26:0/2164967000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380aef10 con 0x7fd6380c7930 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd637fff640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd638152870 0x7fd6381ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd6377fe640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6380c7930 0x7fd638141c80 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c490 tx=0x7fd62800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd61ffff640 1 -- 172.21.15.26:0/2164967000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280052d0 con 0x7fd6380c7930 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd61ffff640 1 -- 172.21.15.26:0/2164967000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628005470 con 0x7fd6380c7930 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6380ae190 con 0x7fd6380c7930 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd61ffff640 1 -- 172.21.15.26:0/2164967000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628012850 con 0x7fd6380c7930 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.425+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6380ae430 con 0x7fd6380c7930 2024-03-06T07:29:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.426+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd638075300 con 0x7fd6380c7930 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.427+0000 7fd61ffff640 1 -- 172.21.15.26:0/2164967000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6280129f0 con 0x7fd6380c7930 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.427+0000 7fd61ffff640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd62406b4d0 0x7fd62406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.427+0000 7fd61ffff640 1 -- 172.21.15.26:0/2164967000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd628089c90 con 0x7fd6380c7930 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.427+0000 7fd636ffd640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd62406b4d0 0x7fd62406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.428+0000 7fd636ffd640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd62406b4d0 0x7fd62406d990 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd62c034000 tx=0x7fd62c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.431+0000 7fd61ffff640 1 -- 172.21.15.26:0/2164967000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd638075300 con 0x7fd6380c7930 2024-03-06T07:29:12.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.613+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd63804f330 con 0x7fd6380c7930 2024-03-06T07:29:12.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:12.989 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:13.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:12.998+0000 7fd61ffff640 1 -- 172.21.15.26:0/2164967000 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v208) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd628058ce0 con 0x7fd6380c7930 2024-03-06T07:29:13.000 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-03-06T07:29:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd62406b4d0 msgr2=0x7fd62406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd62406b4d0 0x7fd62406d990 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd62c034000 tx=0x7fd62c02f000 comp rx=0 tx=0).stop 2024-03-06T07:29:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6380c7930 msgr2=0x7fd638141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6380c7930 0x7fd638141c80 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c490 tx=0x7fd62800c950 comp rx=0 tx=0).stop 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 shutdown_connections 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd62406b4d0 0x7fd62406d990 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd638152870 0x7fd6381ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6380c88e0 0x7fd6380b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 --2- 172.21.15.26:0/2164967000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6380c7930 0x7fd638141c80 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.001+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 >> 172.21.15.26:0/2164967000 conn(0x7fd6380ac5f0 msgr2=0x7fd6380ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.002+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 shutdown_connections 2024-03-06T07:29:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.002+0000 7fd63db7a640 1 -- 172.21.15.26:0/2164967000 wait complete. 2024-03-06T07:29:13.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:29:13.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.197+0000 7f0b0413e640 1 Processor -- start 2024-03-06T07:29:13.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.197+0000 7f0b0413e640 1 -- start start 2024-03-06T07:29:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b0413e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc0c5370 0x7f0afc0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b0413e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0afc155530 0x7f0afc1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b0413e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0afc152dd0 con 0x7f0afc1529f0 2024-03-06T07:29:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b0413e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0afc159710 con 0x7f0afc155530 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b0413e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc0c68b0 con 0x7f0afc0c5370 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b026db640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0afc155530 0x7f0afc1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b026db640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0afc155530 0x7f0afc1559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38248/0 (socket says 172.21.15.26:38248) 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b026db640 1 -- 172.21.15.26:0/639915707 learned_addr learned my addr 172.21.15.26:0/639915707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b016d9640 1 --2- 172.21.15.26:0/639915707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc0c5370 0x7f0afc0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b026db640 1 -- 172.21.15.26:0/639915707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc0c5370 msgr2=0x7f0afc0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b026db640 1 --2- 172.21.15.26:0/639915707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc0c5370 0x7f0afc0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b026db640 1 -- 172.21.15.26:0/639915707 >> v1:172.21.15.117:6789/0 conn(0x7f0afc1529f0 legacy=0x7f0afc0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b026db640 1 -- 172.21.15.26:0/639915707 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc158b00 con 0x7f0afc155530 2024-03-06T07:29:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.198+0000 7f0b016d9640 1 --2- 172.21.15.26:0/639915707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc0c5370 0x7f0afc0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:13.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.199+0000 7f0b026db640 1 --2- 172.21.15.26:0/639915707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc155530 0x7f0afc1559c0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e50 tx=0x7f0aec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6f42fe5d750aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.199+0000 7f0b00ed8640 1 -- 172.21.15.26:0/639915707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec02c9e0 con 0x7f0afc155530 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.199+0000 7f0b00ed8640 1 -- 172.21.15.26:0/639915707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec02cb80 con 0x7f0afc155530 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.199+0000 7f0b00ed8640 1 -- 172.21.15.26:0/639915707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0343f0 con 0x7f0afc155530 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 -- 172.21.15.26:0/639915707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc155530 msgr2=0x7f0afc1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 --2- 172.21.15.26:0/639915707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc155530 0x7f0afc1559c0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e50 tx=0x7f0aec02be40 comp rx=0 tx=0).stop 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 -- 172.21.15.26:0/639915707 shutdown_connections 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 --2- 172.21.15.26:0/639915707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc155530 0x7f0afc1559c0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 --2- 172.21.15.26:0/639915707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc0c5370 0x7f0afc0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 -- 172.21.15.26:0/639915707 >> 172.21.15.26:0/639915707 conn(0x7f0afc0ac580 msgr2=0x7f0afc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 -- 172.21.15.26:0/639915707 shutdown_connections 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.200+0000 7f0b0413e640 1 -- 172.21.15.26:0/639915707 wait complete. 2024-03-06T07:29:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.201+0000 7f0b0413e640 1 Processor -- start 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.201+0000 7f0b0413e640 1 -- start start 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b0413e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc0c5370 0x7f0afc0af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b0413e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1529f0 0x7f0afc154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b0413e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc155530 0x7f0afc1ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b0413e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc1590c0 con 0x7f0afc1529f0 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b0413e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc0c9000 con 0x7f0afc0c5370 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b0413e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc1593a0 con 0x7f0afc155530 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1529f0 0x7f0afc154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1529f0 0x7f0afc154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38258/0 (socket says 172.21.15.26:38258) 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 -- 172.21.15.26:0/402375878 learned_addr learned my addr 172.21.15.26:0/402375878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b01eda640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc0c5370 0x7f0afc0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b026db640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc155530 0x7f0afc1ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 -- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc155530 msgr2=0x7f0afc1ead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc155530 0x7f0afc1ead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 -- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc0c5370 msgr2=0x7f0afc0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc0c5370 0x7f0afc0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 -- 172.21.15.26:0/402375878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc149010 con 0x7f0afc1529f0 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.202+0000 7f0b016d9640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1529f0 0x7f0afc154040 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c820 tx=0x7f0af000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.203+0000 7f0afa7fc640 1 -- 172.21.15.26:0/402375878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0015070 con 0x7f0afc1529f0 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.203+0000 7f0afa7fc640 1 -- 172.21.15.26:0/402375878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af0007a30 con 0x7f0afc1529f0 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.203+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc1ee500 con 0x7f0afc1529f0 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.203+0000 7f0afa7fc640 1 -- 172.21.15.26:0/402375878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0007e00 con 0x7f0afc1529f0 2024-03-06T07:29:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.203+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc1eefe0 con 0x7f0afc1529f0 2024-03-06T07:29:13.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.203+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc0c5960 con 0x7f0afc1529f0 2024-03-06T07:29:13.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.204+0000 7f0afa7fc640 1 -- 172.21.15.26:0/402375878 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0af0018070 con 0x7f0afc1529f0 2024-03-06T07:29:13.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.204+0000 7f0afa7fc640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b500 0x7f0ae806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.204+0000 7f0afa7fc640 1 -- 172.21.15.26:0/402375878 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0af008a000 con 0x7f0afc1529f0 2024-03-06T07:29:13.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.205+0000 7f0b01eda640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b500 0x7f0ae806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.205+0000 7f0b01eda640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b500 0x7f0ae806d9c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0afc149d00 tx=0x7f0af4007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:13.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.209+0000 7f0afa7fc640 1 -- 172.21.15.26:0/402375878 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0afc0c5960 con 0x7f0afc1529f0 2024-03-06T07:29:13.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:13.392+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0afc04f330 con 0x7f0afc1529f0 2024-03-06T07:29:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.007+0000 7f0afa7fc640 1 -- 172.21.15.26:0/402375878 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v209) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0af0058910 con 0x7f0afc1529f0 2024-03-06T07:29:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-03-06T07:29:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b500 msgr2=0x7f0ae806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b500 0x7f0ae806d9c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0afc149d00 tx=0x7f0af4007e30 comp rx=0 tx=0).stop 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1529f0 msgr2=0x7f0afc154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1529f0 0x7f0afc154040 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c820 tx=0x7f0af000cc20 comp rx=0 tx=0).stop 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 shutdown_connections 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0ae806b500 0x7f0ae806d9c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0afc155530 0x7f0afc1ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1529f0 0x7f0afc154040 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 --2- 172.21.15.26:0/402375878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0afc0c5370 0x7f0afc0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 >> 172.21.15.26:0/402375878 conn(0x7f0afc0ac580 msgr2=0x7f0afc14f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.010+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 shutdown_connections 2024-03-06T07:29:14.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.011+0000 7f0b0413e640 1 -- 172.21.15.26:0/402375878 wait complete. 2024-03-06T07:29:14.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:29:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.147+0000 7f9ee8450640 1 Processor -- start 2024-03-06T07:29:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.147+0000 7f9ee8450640 1 -- start start 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee8450640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ee0152920 0x7f9ee0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee8450640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee00c5330 0x7f9ee00c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee8450640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee0156180 con 0x7f9ee00c8a00 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee8450640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee01587b0 con 0x7f9ee0152920 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee8450640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0159630 con 0x7f9ee00c5330 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee61ec640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ee0152920 0x7f9ee0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee61ec640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ee0152920 0x7f9ee0155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38264/0 (socket says 172.21.15.26:38264) 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee61ec640 1 -- 172.21.15.26:0/2422376486 learned_addr learned my addr 172.21.15.26:0/2422376486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee69ed640 1 --2- 172.21.15.26:0/2422376486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee00c5330 0x7f9ee00c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee61ec640 1 -- 172.21.15.26:0/2422376486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee00c5330 msgr2=0x7f9ee00c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee61ec640 1 --2- 172.21.15.26:0/2422376486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee00c5330 0x7f9ee00c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee61ec640 1 -- 172.21.15.26:0/2422376486 >> v1:172.21.15.117:6789/0 conn(0x7f9ee00c8a00 legacy=0x7f9ee0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.148+0000 7f9ee61ec640 1 -- 172.21.15.26:0/2422376486 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee00c7760 con 0x7f9ee0152920 2024-03-06T07:29:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.149+0000 7f9ee61ec640 1 --2- 172.21.15.26:0/2422376486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0152920 0x7f9ee0155c40 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009870 tx=0x7f9ed802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb857a912251022a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.149+0000 7f9ee51ea640 1 -- 172.21.15.26:0/2422376486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8035020 con 0x7f9ee0152920 2024-03-06T07:29:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.149+0000 7f9ee51ea640 1 -- 172.21.15.26:0/2422376486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8031640 con 0x7f9ee0152920 2024-03-06T07:29:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.149+0000 7f9ee51ea640 1 -- 172.21.15.26:0/2422376486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed802d040 con 0x7f9ee0152920 2024-03-06T07:29:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 -- 172.21.15.26:0/2422376486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0152920 msgr2=0x7f9ee0155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 --2- 172.21.15.26:0/2422376486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0152920 0x7f9ee0155c40 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009870 tx=0x7f9ed802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 -- 172.21.15.26:0/2422376486 shutdown_connections 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 --2- 172.21.15.26:0/2422376486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee00c5330 0x7f9ee00c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 --2- 172.21.15.26:0/2422376486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0152920 0x7f9ee0155c40 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 -- 172.21.15.26:0/2422376486 >> 172.21.15.26:0/2422376486 conn(0x7f9ee00ac560 msgr2=0x7f9ee00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 -- 172.21.15.26:0/2422376486 shutdown_connections 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.150+0000 7f9ee8450640 1 -- 172.21.15.26:0/2422376486 wait complete. 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.151+0000 7f9ee8450640 1 Processor -- start 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.151+0000 7f9ee8450640 1 -- start start 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee8450640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ee00c5330 0x7f9ee01dee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee8450640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee00c8a00 0x7f9ee01463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee8450640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee0152920 0x7f9ee0146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee8450640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0158a00 con 0x7f9ee00c8a00 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee8450640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00c7d20 con 0x7f9ee00c5330 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee8450640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee006d030 con 0x7f9ee0152920 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee00c8a00 0x7f9ee01463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee00c8a00 0x7f9ee01463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38266/0 (socket says 172.21.15.26:38266) 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 -- 172.21.15.26:0/3442905668 learned_addr learned my addr 172.21.15.26:0/3442905668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee61ec640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ee00c5330 0x7f9ee01dee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee69ed640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee0152920 0x7f9ee0146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 -- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee0152920 msgr2=0x7f9ee0146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee0152920 0x7f9ee0146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 -- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ee00c5330 msgr2=0x7f9ee01dee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ee00c5330 0x7f9ee01dee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 -- 172.21.15.26:0/3442905668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0146e70 con 0x7f9ee00c8a00 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee69ed640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee0152920 0x7f9ee0146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee59eb640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee00c8a00 0x7f9ee01463f0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c430 tx=0x7f9ed400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9edeffd640 1 -- 172.21.15.26:0/3442905668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40052d0 con 0x7f9ee00c8a00 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9edeffd640 1 -- 172.21.15.26:0/3442905668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4005490 con 0x7f9ee00c8a00 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.152+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee0158130 con 0x7f9ee00c8a00 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.153+0000 7f9edeffd640 1 -- 172.21.15.26:0/3442905668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4007260 con 0x7f9ee00c8a00 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.153+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee0145b50 con 0x7f9ee00c8a00 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.154+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0075300 con 0x7f9ee00c8a00 2024-03-06T07:29:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.154+0000 7f9edeffd640 1 -- 172.21.15.26:0/3442905668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ed4007400 con 0x7f9ee00c8a00 2024-03-06T07:29:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.155+0000 7f9edeffd640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ecc06b500 0x7f9ecc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.155+0000 7f9edeffd640 1 -- 172.21.15.26:0/3442905668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f9ed40896e0 con 0x7f9ee00c8a00 2024-03-06T07:29:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.155+0000 7f9ee61ec640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ecc06b500 0x7f9ecc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.155+0000 7f9ee61ec640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ecc06b500 0x7f9ecc06d9c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8036000 tx=0x7f9ed803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.159+0000 7f9edeffd640 1 -- 172.21.15.26:0/3442905668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ee0075300 con 0x7f9ee00c8a00 2024-03-06T07:29:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:14.376+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9ee004f330 con 0x7f9ee00c8a00 2024-03-06T07:29:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.008+0000 7f9edeffd640 1 -- 172.21.15.26:0/3442905668 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 33 and data pool 34 v196) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9ee0146e70 con 0x7f9ee00c8a00 2024-03-06T07:29:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 33 and data pool 34 2024-03-06T07:29:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.012+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ecc06b500 msgr2=0x7f9ecc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.012+0000 7f9ee8450640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ecc06b500 0x7f9ecc06d9c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8036000 tx=0x7f9ed803c040 comp rx=0 tx=0).stop 2024-03-06T07:29:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.012+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee00c8a00 msgr2=0x7f9ee01463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.012+0000 7f9ee8450640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee00c8a00 0x7f9ee01463f0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c430 tx=0x7f9ed400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 shutdown_connections 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ecc06b500 0x7f9ecc06d9c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ee0152920 0x7f9ee0146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee00c8a00 0x7f9ee01463f0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 --2- 172.21.15.26:0/3442905668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ee00c5330 0x7f9ee01dee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 >> 172.21.15.26:0/3442905668 conn(0x7f9ee00ac560 msgr2=0x7f9ee0151c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 shutdown_connections 2024-03-06T07:29:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.013+0000 7f9ee8450640 1 -- 172.21.15.26:0/3442905668 wait complete. 2024-03-06T07:29:15.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:15.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.151+0000 7f20e8a9d640 1 Processor -- start 2024-03-06T07:29:15.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.151+0000 7f20e8a9d640 1 -- start start 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e8a9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e8a9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20e40c7180 0x7f20e40c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e8a9d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e40c7c60 con 0x7f20e4155b20 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e8a9d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e4159640 con 0x7f20e40c7180 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e8a9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40c8df0 con 0x7f20e41527b0 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e1d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20e40c7180 0x7f20e40c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e2d9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f20e4155b20 0x7f20e40c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:40590/0 (socket says 172.21.15.26:40590) 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e1d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20e40c7180 0x7f20e40c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55342/0 (socket says 172.21.15.26:55342) 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:36474/0 (socket says 172.21.15.26:36474) 2024-03-06T07:29:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.152+0000 7f20e2d9d640 1 -- 172.21.15.26:0/163158353 learned_addr learned my addr 172.21.15.26:0/163158353 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.153+0000 7f20e1d9b640 1 -- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 msgr2=0x7f20e4159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.153+0000 7f20e1d9b640 1 --2- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.153+0000 7f20e1d9b640 1 -- 172.21.15.26:0/163158353 >> v1:172.21.15.117:6789/0 conn(0x7f20e4155b20 legacy=0x7f20e40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.153+0000 7f20e1d9b640 1 -- 172.21.15.26:0/163158353 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40c8060 con 0x7f20e40c7180 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.153+0000 7f20e159a640 1 -- 172.21.15.26:0/163158353 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753606355 0 0) 0x7f20dc001490 con 0x7f20e4155b20 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.153+0000 7f20e259c640 1 --2- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e1d9b640 1 --2- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e40c88b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097a0 tx=0x7f20d802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acfc4fd11ded279 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e159a640 1 -- 172.21.15.26:0/163158353 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80315a0 con 0x7f20e40c7180 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e159a640 1 -- 172.21.15.26:0/163158353 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8037070 con 0x7f20e40c7180 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e159a640 1 -- 172.21.15.26:0/163158353 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8031960 con 0x7f20e40c7180 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 -- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 msgr2=0x7f20e40c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e40c88b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097a0 tx=0x7f20d802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 -- 172.21.15.26:0/163158353 shutdown_connections 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e40c88b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/163158353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 -- 172.21.15.26:0/163158353 >> 172.21.15.26:0/163158353 conn(0x7f20e40ac560 msgr2=0x7f20e40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 -- 172.21.15.26:0/163158353 shutdown_connections 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.154+0000 7f20e8a9d640 1 -- 172.21.15.26:0/163158353 wait complete. 2024-03-06T07:29:15.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 Processor -- start 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 -- start start 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e41566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20e4155b20 0x7f20e41432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4155320 con 0x7f20e4155b20 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40c7980 con 0x7f20e40c7180 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e8a9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4155600 con 0x7f20e41527b0 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20e4155b20 0x7f20e41432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.156+0000 7f20e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20e4155b20 0x7f20e41432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38272/0 (socket says 172.21.15.26:38272) 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e2d9d640 1 -- 172.21.15.26:0/2858993830 learned_addr learned my addr 172.21.15.26:0/2858993830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e259c640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e41566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e1d9b640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e2d9d640 1 -- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 msgr2=0x7f20e4142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e2d9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e2d9d640 1 -- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 msgr2=0x7f20e41566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e2d9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e41566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e2d9d640 1 -- 172.21.15.26:0/2858993830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e41576a0 con 0x7f20e4155b20 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e259c640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e41566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e2d9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20e4155b20 0x7f20e41432b0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c730 tx=0x7f20dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20d2ffd640 1 -- 172.21.15.26:0/2858993830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc018070 con 0x7f20e4155b20 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20d2ffd640 1 -- 172.21.15.26:0/2858993830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc00ce40 con 0x7f20e4155b20 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e41437f0 con 0x7f20e4155b20 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20d2ffd640 1 -- 172.21.15.26:0/2858993830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00a860 con 0x7f20e4155b20 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.157+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e41573c0 con 0x7f20e4155b20 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.158+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e404f330 con 0x7f20e4155b20 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.159+0000 7f20d2ffd640 1 -- 172.21.15.26:0/2858993830 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20dc01b070 con 0x7f20e4155b20 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.159+0000 7f20d2ffd640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f20c406b500 0x7f20c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.159+0000 7f20d2ffd640 1 -- 172.21.15.26:0/2858993830 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20dc089780 con 0x7f20e4155b20 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.159+0000 7f20e259c640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f20c406b500 0x7f20c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.160+0000 7f20e259c640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f20c406b500 0x7f20c406d9c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009560 tx=0x7f20d4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.164+0000 7f20d2ffd640 1 -- 172.21.15.26:0/2858993830 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f20e40c7c60 con 0x7f20e4155b20 2024-03-06T07:29:15.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.346+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20e406ce00 con 0x7f20e4155b20 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.347+0000 7f20d2ffd640 1 -- 172.21.15.26:0/2858993830 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) v1 ==== 74+0+16913 (secure 0 0 0) 0x7f20dc057eb0 con 0x7f20e4155b20 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":210,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:29:14.377889+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.632872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.411822+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f20c406b500 msgr2=0x7f20c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f20c406b500 0x7f20c406d9c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009560 tx=0x7f20d4009190 comp rx=0 tx=0).stop 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20e4155b20 msgr2=0x7f20e41432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20e4155b20 0x7f20e41432b0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c730 tx=0x7f20dc00cbf0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 shutdown_connections 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f20c406b500 0x7f20c406d9c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20e4155b20 0x7f20e41432b0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20e41527b0 0x7f20e4142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 --2- 172.21.15.26:0/2858993830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20e40c7180 0x7f20e41566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.349+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 >> 172.21.15.26:0/2858993830 conn(0x7f20e40ac560 msgr2=0x7f20e414aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.350+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 shutdown_connections 2024-03-06T07:29:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.350+0000 7f20e8a9d640 1 -- 172.21.15.26:0/2858993830 wait complete. 2024-03-06T07:29:15.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.462+0000 7f3968d3e640 1 Processor -- start 2024-03-06T07:29:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.462+0000 7f3968d3e640 1 -- start start 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.462+0000 7f3968d3e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39640c3280 0x7f39640c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.462+0000 7f3968d3e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3964158800 con 0x7f39641551e0 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.462+0000 7f3968d3e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3964158ae0 con 0x7f396406d9f0 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.462+0000 7f3968d3e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3964158dc0 con 0x7f39640c3280 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f3962d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f39641551e0 0x7f39641555f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49590/0 (socket says 172.21.15.26:49590) 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f3962d9d640 1 -- 172.21.15.26:0/2606161779 learned_addr learned my addr 172.21.15.26:0/2606161779 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f3961d9b640 1 --2- 172.21.15.26:0/2606161779 >> v2:172.21.15.117:3300/0 conn(0x7f39640c3280 0x7f39640c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295325151 0 0) 0x7f3964158800 con 0x7f39641551e0 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f394c003520 con 0x7f39641551e0 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3637044887 0 0) 0x7f394c003520 con 0x7f39641551e0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 >> v2:172.21.15.117:3300/0 conn(0x7f39640c3280 msgr2=0x7f39640c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 --2- 172.21.15.26:0/2606161779 >> v2:172.21.15.117:3300/0 conn(0x7f39640c3280 0x7f39640c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 >> v1:172.21.15.117:6789/0 conn(0x7f396406d9f0 legacy=0x7f39640a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39641590a0 con 0x7f39641551e0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f395c0019f0 con 0x7f39641551e0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 >> v1:172.21.15.26:6789/0 conn(0x7f39641551e0 legacy=0x7f39641555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c0039b0 0x7f394c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c004800 0x7f394c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c0051c0 0x7f394c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3964158dc0 con 0x7f394c004800 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c00c160 con 0x7f394c0039b0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c00c360 con 0x7f394c0051c0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f3961d9b640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c004800 0x7f394c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.463+0000 7f396259c640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c0039b0 0x7f394c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3962d9d640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c0051c0 0x7f394c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3961d9b640 1 -- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c0051c0 msgr2=0x7f394c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3961d9b640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c0051c0 0x7f394c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3961d9b640 1 -- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c0039b0 msgr2=0x7f394c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3961d9b640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c0039b0 0x7f394c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3961d9b640 1 -- 172.21.15.26:0/2606161779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c003520 con 0x7f394c004800 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f396259c640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c0039b0 0x7f394c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3962d9d640 1 -- 172.21.15.26:0/2606161779 reap_dead start 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f3961d9b640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c004800 0x7f394c004c80 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f3958009ac0 tx=0x7f394c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29605850c30b6682 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958039070 con 0x7f394c004800 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.464+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395802c6b0 con 0x7f394c004800 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f396159a640 1 -- 172.21.15.26:0/2606161779 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958009690 con 0x7f394c004800 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f3968d3e640 1 -- 172.21.15.26:0/2606161779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c004800 msgr2=0x7f394c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f3968d3e640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c004800 0x7f394c004c80 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f3958009ac0 tx=0x7f394c005830 comp rx=0 tx=0).stop 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f3968d3e640 1 -- 172.21.15.26:0/2606161779 shutdown_connections 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f3968d3e640 1 --2- 172.21.15.26:0/2606161779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c004800 0x7f394c004c80 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f3968d3e640 1 -- 172.21.15.26:0/2606161779 >> 172.21.15.26:0/2606161779 conn(0x7f39640aa630 msgr2=0x7f39640aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f3968d3e640 1 -- 172.21.15.26:0/2606161779 shutdown_connections 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.465+0000 7f3968d3e640 1 -- 172.21.15.26:0/2606161779 wait complete. 2024-03-06T07:29:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.466+0000 7f3968d3e640 1 Processor -- start 2024-03-06T07:29:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.466+0000 7f3968d3e640 1 -- start start 2024-03-06T07:29:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3968d3e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c004800 0x7f39641e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3968d3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39641ee280 0x7f396406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3968d3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f396414ea70 0x7f396414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3968d3e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396406df50 con 0x7f396414ea70 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3968d3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396406ddd0 con 0x7f39641ee280 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3968d3e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39640c6910 con 0x7f394c004800 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f396414ea70 0x7f396414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f396414ea70 0x7f396414ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38284/0 (socket says 172.21.15.26:38284) 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 -- 172.21.15.26:0/1169913790 learned_addr learned my addr 172.21.15.26:0/1169913790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3961d9b640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39641ee280 0x7f396406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f396259c640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c004800 0x7f39641e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 -- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c004800 msgr2=0x7f39641e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c004800 0x7f39641e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 -- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39641ee280 msgr2=0x7f396406f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39641ee280 0x7f396406f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3962d9d640 1 -- 172.21.15.26:0/1169913790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39641eeac0 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.467+0000 7f3961d9b640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39641ee280 0x7f396406f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.468+0000 7f3962d9d640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f396414ea70 0x7f396414ef00 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f39641555c0 tx=0x7f395c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.468+0000 7f3952ffd640 1 -- 172.21.15.26:0/1169913790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c00ea80 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.468+0000 7f3952ffd640 1 -- 172.21.15.26:0/1169913790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c018070 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.468+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39641eed60 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.468+0000 7f3952ffd640 1 -- 172.21.15.26:0/1169913790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c00eea0 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.468+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3964070130 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.469+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396404f330 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.469+0000 7f3952ffd640 1 -- 172.21.15.26:0/1169913790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f395c01d460 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.470+0000 7f3952ffd640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f394406b500 0x7f394406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.470+0000 7f3952ffd640 1 -- 172.21.15.26:0/1169913790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f395c08a5b0 con 0x7f396414ea70 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.470+0000 7f396259c640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f394406b500 0x7f394406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.470+0000 7f396259c640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f394406b500 0x7f394406d9c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3954000c00 tx=0x7f3954009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.474+0000 7f3952ffd640 1 -- 172.21.15.26:0/1169913790 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f395c059330 con 0x7f396414ea70 2024-03-06T07:29:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.655+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39640aa1d0 con 0x7f396414ea70 2024-03-06T07:29:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.656+0000 7f3952ffd640 1 -- 172.21.15.26:0/1169913790 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) v1 ==== 74+0+16913 (secure 0 0 0) 0x7f395c058c30 con 0x7f396414ea70 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":210,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:29:14.377889+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.632872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.411822+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f394406b500 msgr2=0x7f394406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f394406b500 0x7f394406d9c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3954000c00 tx=0x7f3954009040 comp rx=0 tx=0).stop 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f396414ea70 msgr2=0x7f396414ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f396414ea70 0x7f396414ef00 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f39641555c0 tx=0x7f395c007bc0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 shutdown_connections 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f394406b500 0x7f394406d9c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f396414ea70 0x7f396414ef00 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39641ee280 0x7f396406f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 --2- 172.21.15.26:0/1169913790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f394c004800 0x7f39641e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 >> 172.21.15.26:0/1169913790 conn(0x7f39640aa630 msgr2=0x7f3964153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 shutdown_connections 2024-03-06T07:29:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.658+0000 7f3968d3e640 1 -- 172.21.15.26:0/1169913790 wait complete. 2024-03-06T07:29:15.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:29:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.774+0000 7f5e27888640 1 Processor -- start 2024-03-06T07:29:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.774+0000 7f5e27888640 1 -- start start 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e27888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e200c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e27888640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e200c57e0 0x7f5e20155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e27888640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5e20072cd0 0x7f5e20155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e27888640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e200c6dd0 con 0x7f5e200c57e0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e27888640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e200c6940 con 0x7f5e20072cd0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e27888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20152db0 con 0x7f5e201529d0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e200c57e0 0x7f5e20155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e200c57e0 0x7f5e20155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38296/0 (socket says 172.21.15.26:38296) 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 -- 172.21.15.26:0/1762998431 learned_addr learned my addr 172.21.15.26:0/1762998431 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e25e25640 1 --2- 172.21.15.26:0/1762998431 >> v2:172.21.15.117:3300/0 conn(0x7f5e20072cd0 0x7f5e20155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e25624640 1 --2- 172.21.15.26:0/1762998431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e200c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 -- 172.21.15.26:0/1762998431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 msgr2=0x7f5e200c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 --2- 172.21.15.26:0/1762998431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e200c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 -- 172.21.15.26:0/1762998431 >> v2:172.21.15.117:3300/0 conn(0x7f5e20072cd0 msgr2=0x7f5e20155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 --2- 172.21.15.26:0/1762998431 >> v2:172.21.15.117:3300/0 conn(0x7f5e20072cd0 0x7f5e20155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.775+0000 7f5e24e23640 1 -- 172.21.15.26:0/1762998431 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e201590f0 con 0x7f5e200c57e0 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.776+0000 7f5e24e23640 1 --2- 172.21.15.26:0/1762998431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e200c57e0 0x7f5e20155120 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f5e200c73f0 tx=0x7f5e1002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c702120f99256f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.776+0000 7f5e17fff640 1 -- 172.21.15.26:0/1762998431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10038070 con 0x7f5e200c57e0 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.776+0000 7f5e17fff640 1 -- 172.21.15.26:0/1762998431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1002f050 con 0x7f5e200c57e0 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.776+0000 7f5e17fff640 1 -- 172.21.15.26:0/1762998431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10031970 con 0x7f5e200c57e0 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 -- 172.21.15.26:0/1762998431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e200c57e0 msgr2=0x7f5e20155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 --2- 172.21.15.26:0/1762998431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e200c57e0 0x7f5e20155120 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f5e200c73f0 tx=0x7f5e1002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 -- 172.21.15.26:0/1762998431 shutdown_connections 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 --2- 172.21.15.26:0/1762998431 >> v2:172.21.15.117:3300/0 conn(0x7f5e20072cd0 0x7f5e20155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 --2- 172.21.15.26:0/1762998431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e200c57e0 0x7f5e20155120 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 --2- 172.21.15.26:0/1762998431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e200c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 -- 172.21.15.26:0/1762998431 >> 172.21.15.26:0/1762998431 conn(0x7f5e200aa630 msgr2=0x7f5e200aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 -- 172.21.15.26:0/1762998431 shutdown_connections 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.777+0000 7f5e27888640 1 -- 172.21.15.26:0/1762998431 wait complete. 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 Processor -- start 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 -- start start 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e20072cd0 0x7f5e20147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e200c57e0 0x7f5e2006c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e201ee530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20159370 con 0x7f5e20072cd0 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20072720 con 0x7f5e200c57e0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.778+0000 7f5e27888640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200723b0 con 0x7f5e201529d0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e20072cd0 0x7f5e20147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e20072cd0 0x7f5e20147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38302/0 (socket says 172.21.15.26:38302) 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 -- 172.21.15.26:0/2671091470 learned_addr learned my addr 172.21.15.26:0/2671091470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e24e23640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e200c57e0 0x7f5e2006c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25e25640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e201ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 -- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 msgr2=0x7f5e201ee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e201ee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 -- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e200c57e0 msgr2=0x7f5e2006c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e200c57e0 0x7f5e2006c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 -- 172.21.15.26:0/2671091470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e200acea0 con 0x7f5e20072cd0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e25624640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e20072cd0 0x7f5e20147610 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00d800 tx=0x7f5e1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e15ffb640 1 -- 172.21.15.26:0/2671091470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00b6c0 con 0x7f5e20072cd0 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e15ffb640 1 -- 172.21.15.26:0/2671091470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c019070 con 0x7f5e20072cd0 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e200c3140 con 0x7f5e20072cd0 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e15ffb640 1 -- 172.21.15.26:0/2671091470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00ba80 con 0x7f5e20072cd0 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.779+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e201eb680 con 0x7f5e20072cd0 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.780+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e20073540 con 0x7f5e20072cd0 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.781+0000 7f5e15ffb640 1 -- 172.21.15.26:0/2671091470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e1c00bc20 con 0x7f5e20072cd0 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.781+0000 7f5e15ffb640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b500 0x7f5e0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.781+0000 7f5e15ffb640 1 -- 172.21.15.26:0/2671091470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e1c08a6c0 con 0x7f5e20072cd0 2024-03-06T07:29:15.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.781+0000 7f5e24e23640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b500 0x7f5e0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.782+0000 7f5e24e23640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b500 0x7f5e0406d9c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5e10035000 tx=0x7f5e10008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:15.785+0000 7f5e15ffb640 1 -- 172.21.15.26:0/2671091470 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5e20073540 con 0x7f5e20072cd0 2024-03-06T07:29:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.004+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5e200a9310 con 0x7f5e20072cd0 2024-03-06T07:29:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.022+0000 7f5e15ffb640 1 -- 172.21.15.26:0/2671091470 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v197) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5e200a9310 con 0x7f5e20072cd0 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b500 msgr2=0x7f5e0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b500 0x7f5e0406d9c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5e10035000 tx=0x7f5e10008030 comp rx=0 tx=0).stop 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e20072cd0 msgr2=0x7f5e20147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e20072cd0 0x7f5e20147610 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00d800 tx=0x7f5e1c00dcc0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 shutdown_connections 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5e0406b500 0x7f5e0406d9c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e201529d0 0x7f5e201ee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e200c57e0 0x7f5e2006c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 --2- 172.21.15.26:0/2671091470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e20072cd0 0x7f5e20147610 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 >> 172.21.15.26:0/2671091470 conn(0x7f5e200aa630 msgr2=0x7f5e200c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 shutdown_connections 2024-03-06T07:29:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.025+0000 7f5e27888640 1 -- 172.21.15.26:0/2671091470 wait complete. 2024-03-06T07:29:16.037 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:29:16.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:29:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.143+0000 7fda2f3c7640 1 Processor -- start 2024-03-06T07:29:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.143+0000 7fda2f3c7640 1 -- start start 2024-03-06T07:29:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2f3c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda280b0070 0x7fda28155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2f3c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda280c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2f3c7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280b0450 con 0x7fda280c5380 2024-03-06T07:29:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2f3c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2806c020 con 0x7fda280b0070 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2f3c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2806cf20 con 0x7fda2806ca70 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2d163640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda280b0070 0x7fda28155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2d964640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fda280c5380 0x7fda2814e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49600/0 (socket says 172.21.15.26:49600) 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2d964640 1 -- 172.21.15.26:0/2753059778 learned_addr learned my addr 172.21.15.26:0/2753059778 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.144+0000 7fda2c962640 1 --2- 172.21.15.26:0/2753059778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda280c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.145+0000 7fda2d163640 1 -- 172.21.15.26:0/2753059778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 msgr2=0x7fda280c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.145+0000 7fda2d163640 1 --2- 172.21.15.26:0/2753059778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda280c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.145+0000 7fda2d163640 1 -- 172.21.15.26:0/2753059778 >> v1:172.21.15.26:6789/0 conn(0x7fda280c5380 legacy=0x7fda2814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.145+0000 7fda2d163640 1 -- 172.21.15.26:0/2753059778 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280c77f0 con 0x7fda280b0070 2024-03-06T07:29:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.145+0000 7fda2d163640 1 --2- 172.21.15.26:0/2753059778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280b0070 0x7fda28155c30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda2002bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf83b754e2e6e2eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.145+0000 7fda27fff640 1 -- 172.21.15.26:0/2753059778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20039070 con 0x7fda280b0070 2024-03-06T07:29:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda27fff640 1 -- 172.21.15.26:0/2753059778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2002c980 con 0x7fda280b0070 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda27fff640 1 -- 172.21.15.26:0/2753059778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2002cd50 con 0x7fda280b0070 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2753059778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280b0070 msgr2=0x7fda28155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2753059778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280b0070 0x7fda28155c30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda2002bc50 comp rx=0 tx=0).stop 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2753059778 shutdown_connections 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2753059778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda280c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2753059778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280b0070 0x7fda28155c30 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2753059778 >> 172.21.15.26:0/2753059778 conn(0x7fda280ac590 msgr2=0x7fda280ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2753059778 shutdown_connections 2024-03-06T07:29:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.146+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2753059778 wait complete. 2024-03-06T07:29:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.147+0000 7fda2f3c7640 1 Processor -- start 2024-03-06T07:29:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.147+0000 7fda2f3c7640 1 -- start start 2024-03-06T07:29:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2f3c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda281eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2f3c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280b0070 0x7fda280abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2f3c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280c5380 0x7fda28157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2f3c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2806c550 con 0x7fda280c5380 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2f3c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2806c270 con 0x7fda280b0070 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2f3c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2806da80 con 0x7fda2806ca70 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2d964640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280c5380 0x7fda28157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2d964640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280c5380 0x7fda28157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38316/0 (socket says 172.21.15.26:38316) 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2d964640 1 -- 172.21.15.26:0/2719182684 learned_addr learned my addr 172.21.15.26:0/2719182684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2d163640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda281eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.148+0000 7fda2c962640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280b0070 0x7fda280abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2d964640 1 -- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 msgr2=0x7fda281eb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2d964640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda281eb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2d964640 1 -- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280b0070 msgr2=0x7fda280abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2d964640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280b0070 0x7fda280abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2d964640 1 -- 172.21.15.26:0/2719182684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28156890 con 0x7fda280c5380 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2d163640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda281eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2c962640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280b0070 0x7fda280abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2d964640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280c5380 0x7fda28157fa0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c780 tx=0x7fda1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda25ffb640 1 -- 172.21.15.26:0/2719182684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18018070 con 0x7fda280c5380 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda25ffb640 1 -- 172.21.15.26:0/2719182684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda180074e0 con 0x7fda280c5380 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda25ffb640 1 -- 172.21.15.26:0/2719182684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18007690 con 0x7fda280c5380 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda28154170 con 0x7fda280c5380 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.149+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda280ac2b0 con 0x7fda280c5380 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.150+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2804f330 con 0x7fda280c5380 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.151+0000 7fda25ffb640 1 -- 172.21.15.26:0/2719182684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda18004a20 con 0x7fda280c5380 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.151+0000 7fda25ffb640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fda1406b4d0 0x7fda1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.151+0000 7fda25ffb640 1 -- 172.21.15.26:0/2719182684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda18089a70 con 0x7fda280c5380 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.151+0000 7fda2d163640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fda1406b4d0 0x7fda1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.152+0000 7fda2d163640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fda1406b4d0 0x7fda1406d990 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fda20036000 tx=0x7fda20030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.160+0000 7fda25ffb640 1 -- 172.21.15.26:0/2719182684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fda280b0450 con 0x7fda280c5380 2024-03-06T07:29:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:16.378+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fda2806d860 con 0x7fda280c5380 2024-03-06T07:29:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.027+0000 7fda25ffb640 1 -- 172.21.15.26:0/2719182684 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v198) v1 ==== 122+0+0 (secure 0 0 0) 0x7fda180586d0 con 0x7fda280c5380 2024-03-06T07:29:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fda1406b4d0 msgr2=0x7fda1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fda1406b4d0 0x7fda1406d990 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fda20036000 tx=0x7fda20030000 comp rx=0 tx=0).stop 2024-03-06T07:29:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280c5380 msgr2=0x7fda28157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280c5380 0x7fda28157fa0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c780 tx=0x7fda1800cc40 comp rx=0 tx=0).stop 2024-03-06T07:29:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 shutdown_connections 2024-03-06T07:29:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fda1406b4d0 0x7fda1406d990 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda280c5380 0x7fda28157fa0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280b0070 0x7fda280abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 --2- 172.21.15.26:0/2719182684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda2806ca70 0x7fda281eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 >> 172.21.15.26:0/2719182684 conn(0x7fda280ac590 msgr2=0x7fda28151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 shutdown_connections 2024-03-06T07:29:17.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.029+0000 7fda2f3c7640 1 -- 172.21.15.26:0/2719182684 wait complete. 2024-03-06T07:29:17.042 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:29:17.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.141+0000 7f59a3e4e640 1 Processor -- start 2024-03-06T07:29:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.141+0000 7f59a3e4e640 1 -- start start 2024-03-06T07:29:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a3e4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f599c0c3280 0x7f599c0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a3e4e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f599c1551e0 0x7f599c1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a3e4e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c158800 con 0x7f599c06d9f0 2024-03-06T07:29:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a3e4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c158ae0 con 0x7f599c0c3280 2024-03-06T07:29:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a3e4e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c158dc0 con 0x7f599c1551e0 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a13e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f599c0c3280 0x7f599c0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a13e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f599c0c3280 0x7f599c0c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37396/0 (socket says 172.21.15.26:37396) 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a13e9640 1 -- 172.21.15.26:0/697426485 learned_addr learned my addr 172.21.15.26:0/697426485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a23eb640 1 --2- 172.21.15.26:0/697426485 >> v2:172.21.15.117:3300/0 conn(0x7f599c1551e0 0x7f599c1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a13e9640 1 -- 172.21.15.26:0/697426485 >> v2:172.21.15.117:3300/0 conn(0x7f599c1551e0 msgr2=0x7f599c1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a13e9640 1 --2- 172.21.15.26:0/697426485 >> v2:172.21.15.117:3300/0 conn(0x7f599c1551e0 0x7f599c1555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a13e9640 1 -- 172.21.15.26:0/697426485 >> v1:172.21.15.117:6789/0 conn(0x7f599c06d9f0 legacy=0x7f599c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a13e9640 1 -- 172.21.15.26:0/697426485 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c1590a0 con 0x7f599c0c3280 2024-03-06T07:29:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.142+0000 7f59a23eb640 1 --2- 172.21.15.26:0/697426485 >> v2:172.21.15.117:3300/0 conn(0x7f599c1551e0 0x7f599c1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.143+0000 7f59a13e9640 1 --2- 172.21.15.26:0/697426485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0c3280 0x7f599c0c3710 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f599c072f50 tx=0x7f599802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bf3348ce859e88 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a0be8640 1 -- 172.21.15.26:0/697426485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998035020 con 0x7f599c0c3280 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a0be8640 1 -- 172.21.15.26:0/697426485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998031870 con 0x7f599c0c3280 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a0be8640 1 -- 172.21.15.26:0/697426485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998031c40 con 0x7f599c0c3280 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 -- 172.21.15.26:0/697426485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0c3280 msgr2=0x7f599c0c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/697426485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0c3280 0x7f599c0c3710 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f599c072f50 tx=0x7f599802b910 comp rx=0 tx=0).stop 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 -- 172.21.15.26:0/697426485 shutdown_connections 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/697426485 >> v2:172.21.15.117:3300/0 conn(0x7f599c1551e0 0x7f599c1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/697426485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0c3280 0x7f599c0c3710 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 -- 172.21.15.26:0/697426485 >> 172.21.15.26:0/697426485 conn(0x7f599c0aa630 msgr2=0x7f599c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 -- 172.21.15.26:0/697426485 shutdown_connections 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.144+0000 7f59a3e4e640 1 -- 172.21.15.26:0/697426485 wait complete. 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.145+0000 7f59a3e4e640 1 Processor -- start 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.145+0000 7f59a3e4e640 1 -- start start 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a3e4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f599c06d9f0 0x7f599c06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a3e4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f599c0c3280 0x7f599c1edd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a3e4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c1551e0 0x7f599c0ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a3e4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c06df50 con 0x7f599c1551e0 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a3e4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c06ddd0 con 0x7f599c06d9f0 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a3e4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c6910 con 0x7f599c0c3280 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c1551e0 0x7f599c0ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c1551e0 0x7f599c0ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37408/0 (socket says 172.21.15.26:37408) 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 -- 172.21.15.26:0/2109170436 learned_addr learned my addr 172.21.15.26:0/2109170436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a1bea640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f599c06d9f0 0x7f599c06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a13e9640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f599c0c3280 0x7f599c1edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 -- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f599c0c3280 msgr2=0x7f599c1edd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f599c0c3280 0x7f599c1edd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 -- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f599c06d9f0 msgr2=0x7f599c06f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f599c06d9f0 0x7f599c06f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.146+0000 7f59a23eb640 1 -- 172.21.15.26:0/2109170436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c1ee5e0 con 0x7f599c1551e0 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.147+0000 7f59a23eb640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c1551e0 0x7f599c0ac510 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f5994007e90 tx=0x7f599400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.147+0000 7f598e7fc640 1 -- 172.21.15.26:0/2109170436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599401a070 con 0x7f599c1551e0 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.147+0000 7f598e7fc640 1 -- 172.21.15.26:0/2109170436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5994003990 con 0x7f599c1551e0 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.147+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c1ee8c0 con 0x7f599c1551e0 2024-03-06T07:29:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.147+0000 7f598e7fc640 1 -- 172.21.15.26:0/2109170436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994003d60 con 0x7f599c1551e0 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.147+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c06edc0 con 0x7f599c1551e0 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.148+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c04f330 con 0x7f599c1551e0 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.148+0000 7f598e7fc640 1 -- 172.21.15.26:0/2109170436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f599401d070 con 0x7f599c1551e0 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.149+0000 7f598e7fc640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f598006b4d0 0x7f598006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.149+0000 7f598e7fc640 1 -- 172.21.15.26:0/2109170436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f599408a4d0 con 0x7f599c1551e0 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.149+0000 7f59a1bea640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f598006b4d0 0x7f598006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.149+0000 7f59a1bea640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f598006b4d0 0x7f598006d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5990007fd0 tx=0x7f5990009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.153+0000 7f598e7fc640 1 -- 172.21.15.26:0/2109170436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5994010070 con 0x7f599c1551e0 2024-03-06T07:29:17.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.371+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f599c073240 con 0x7f599c1551e0 2024-03-06T07:29:17.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.371+0000 7f598e7fc640 1 -- 172.21.15.26:0/2109170436 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f5994059250 con 0x7f599c1551e0 2024-03-06T07:29:17.373 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:17.373 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":198,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":9799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:16.379626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 198 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f598006b4d0 msgr2=0x7f598006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f598006b4d0 0x7f598006d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5990007fd0 tx=0x7f5990009040 comp rx=0 tx=0).stop 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c1551e0 msgr2=0x7f599c0ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c1551e0 0x7f599c0ac510 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f5994007e90 tx=0x7f599400d4a0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 shutdown_connections 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f598006b4d0 0x7f598006d990 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c1551e0 0x7f599c0ac510 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f599c0c3280 0x7f599c1edd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 --2- 172.21.15.26:0/2109170436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f599c06d9f0 0x7f599c06f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 >> 172.21.15.26:0/2109170436 conn(0x7f599c0aa630 msgr2=0x7f599c06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 shutdown_connections 2024-03-06T07:29:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.374+0000 7f59a3e4e640 1 -- 172.21.15.26:0/2109170436 wait complete. 2024-03-06T07:29:17.386 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.490+0000 7fd36b4af640 1 Processor -- start 2024-03-06T07:29:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.490+0000 7fd36b4af640 1 -- start start 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.491+0000 7fd36b4af640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd36414f360 0x7fd364155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.491+0000 7fd36b4af640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd364158b60 0x7fd364158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.491+0000 7fd36b4af640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640c73d0 con 0x7fd3640c88c0 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.491+0000 7fd36b4af640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640c8dd0 con 0x7fd364158b60 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.491+0000 7fd36b4af640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640be2e0 con 0x7fd36414f360 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd36924b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd36414f360 0x7fd364155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd369a4c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd364158b60 0x7fd364158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd36924b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd36414f360 0x7fd364155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:37778/0 (socket says 172.21.15.26:37778) 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd368a4a640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd3640c88c0 0x7fd3640c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:43624/0 (socket says 172.21.15.26:43624) 2024-03-06T07:29:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd36924b640 1 -- 172.21.15.26:0/2576255101 learned_addr learned my addr 172.21.15.26:0/2576255101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd35bfff640 1 -- 172.21.15.26:0/2576255101 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894218723 0 0) 0x7fd3640c73d0 con 0x7fd3640c88c0 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd36924b640 1 -- 172.21.15.26:0/2576255101 >> v2:172.21.15.117:3300/0 conn(0x7fd364158b60 msgr2=0x7fd364158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd36924b640 1 --2- 172.21.15.26:0/2576255101 >> v2:172.21.15.117:3300/0 conn(0x7fd364158b60 0x7fd364158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd36924b640 1 -- 172.21.15.26:0/2576255101 >> v1:172.21.15.117:6789/0 conn(0x7fd3640c88c0 legacy=0x7fd3640c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd36924b640 1 -- 172.21.15.26:0/2576255101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640c7670 con 0x7fd36414f360 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.492+0000 7fd369a4c640 1 --2- 172.21.15.26:0/2576255101 >> v2:172.21.15.117:3300/0 conn(0x7fd364158b60 0x7fd364158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.493+0000 7fd36924b640 1 --2- 172.21.15.26:0/2576255101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd36414f360 0x7fd364155c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd360009870 tx=0x7fd36002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63e263fdb8d4b29 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.493+0000 7fd35bfff640 1 -- 172.21.15.26:0/2576255101 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360035020 con 0x7fd36414f360 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.493+0000 7fd35bfff640 1 -- 172.21.15.26:0/2576255101 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360031640 con 0x7fd36414f360 2024-03-06T07:29:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.493+0000 7fd35bfff640 1 -- 172.21.15.26:0/2576255101 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36002d040 con 0x7fd36414f360 2024-03-06T07:29:17.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 -- 172.21.15.26:0/2576255101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd36414f360 msgr2=0x7fd364155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 --2- 172.21.15.26:0/2576255101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd36414f360 0x7fd364155c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd360009870 tx=0x7fd36002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 -- 172.21.15.26:0/2576255101 shutdown_connections 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 --2- 172.21.15.26:0/2576255101 >> v2:172.21.15.117:3300/0 conn(0x7fd364158b60 0x7fd364158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 --2- 172.21.15.26:0/2576255101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd36414f360 0x7fd364155c10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 -- 172.21.15.26:0/2576255101 >> 172.21.15.26:0/2576255101 conn(0x7fd3640ac570 msgr2=0x7fd3640ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 -- 172.21.15.26:0/2576255101 shutdown_connections 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.494+0000 7fd36b4af640 1 -- 172.21.15.26:0/2576255101 wait complete. 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.496+0000 7fd36b4af640 1 Processor -- start 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.496+0000 7fd36b4af640 1 -- start start 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd36b4af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3640c88c0 0x7fd3640ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd36b4af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36414f360 0x7fd3641ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd36b4af640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd364158b60 0x7fd3641ec220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd36b4af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640c5a50 con 0x7fd36414f360 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd36b4af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640c7c30 con 0x7fd3640c88c0 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd36b4af640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640c5bd0 con 0x7fd364158b60 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36414f360 0x7fd3641ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36414f360 0x7fd3641ebce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37414/0 (socket says 172.21.15.26:37414) 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 -- 172.21.15.26:0/1646349216 learned_addr learned my addr 172.21.15.26:0/1646349216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd369a4c640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd364158b60 0x7fd3641ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd36924b640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3640c88c0 0x7fd3640ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 -- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd364158b60 msgr2=0x7fd3641ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd364158b60 0x7fd3641ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 -- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3640c88c0 msgr2=0x7fd3640ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3640c88c0 0x7fd3640ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 -- 172.21.15.26:0/1646349216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640ac2c0 con 0x7fd36414f360 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.497+0000 7fd368a4a640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36414f360 0x7fd3641ebce0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd35400c440 tx=0x7fd35400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.498+0000 7fd359ffb640 1 -- 172.21.15.26:0/1646349216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540074e0 con 0x7fd36414f360 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.498+0000 7fd359ffb640 1 -- 172.21.15.26:0/1646349216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3540047e0 con 0x7fd36414f360 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.498+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3641eb700 con 0x7fd36414f360 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.498+0000 7fd359ffb640 1 -- 172.21.15.26:0/1646349216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354004bd0 con 0x7fd36414f360 2024-03-06T07:29:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.498+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641ee160 con 0x7fd36414f360 2024-03-06T07:29:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.499+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36404f330 con 0x7fd36414f360 2024-03-06T07:29:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.499+0000 7fd359ffb640 1 -- 172.21.15.26:0/1646349216 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd354011070 con 0x7fd36414f360 2024-03-06T07:29:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.500+0000 7fd359ffb640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd34806b4d0 0x7fd34806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.500+0000 7fd359ffb640 1 -- 172.21.15.26:0/1646349216 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd354089dd0 con 0x7fd36414f360 2024-03-06T07:29:17.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.500+0000 7fd36924b640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd34806b4d0 0x7fd34806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.501+0000 7fd36924b640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd34806b4d0 0x7fd34806d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd360036000 tx=0x7fd360008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.504+0000 7fd359ffb640 1 -- 172.21.15.26:0/1646349216 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd354058b50 con 0x7fd36414f360 2024-03-06T07:29:17.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.738+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd36414c5f0 con 0x7fd36414f360 2024-03-06T07:29:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.739+0000 7fd359ffb640 1 -- 172.21.15.26:0/1646349216 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16900 (secure 0 0 0) 0x7fd354058450 con 0x7fd36414f360 2024-03-06T07:29:17.740 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:17.741 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":211,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:29:16.016890+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.632872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.411822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:17.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd34806b4d0 msgr2=0x7fd34806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd34806b4d0 0x7fd34806d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd360036000 tx=0x7fd360008030 comp rx=0 tx=0).stop 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36414f360 msgr2=0x7fd3641ebce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36414f360 0x7fd3641ebce0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd35400c440 tx=0x7fd35400c840 comp rx=0 tx=0).stop 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 shutdown_connections 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd34806b4d0 0x7fd34806d990 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd364158b60 0x7fd3641ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd36414f360 0x7fd3641ebce0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 --2- 172.21.15.26:0/1646349216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3640c88c0 0x7fd3640ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 >> 172.21.15.26:0/1646349216 conn(0x7fd3640ac570 msgr2=0x7fd36414e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 shutdown_connections 2024-03-06T07:29:17.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.741+0000 7fd36b4af640 1 -- 172.21.15.26:0/1646349216 wait complete. 2024-03-06T07:29:17.753 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:29:17.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:29:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.853+0000 7f9ecb484640 1 Processor -- start 2024-03-06T07:29:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ecb484640 1 -- start start 2024-03-06T07:29:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ecb484640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ec414f650 0x7f9ec4071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ecb484640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ec40c3150 0x7f9ec40c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ecb484640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec4152c20 0x7f9ec40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ecb484640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec4072440 con 0x7f9ec40c3150 2024-03-06T07:29:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ecb484640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec40c2b10 con 0x7f9ec414f650 2024-03-06T07:29:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ecb484640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec4152140 con 0x7f9ec4152c20 2024-03-06T07:29:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.854+0000 7f9ec8a1f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ec40c3150 0x7f9ec40c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec8a1f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ec40c3150 0x7f9ec40c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37424/0 (socket says 172.21.15.26:37424) 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec8a1f640 1 -- 172.21.15.26:0/2105291794 learned_addr learned my addr 172.21.15.26:0/2105291794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec9a21640 1 --2- 172.21.15.26:0/2105291794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec4152c20 0x7f9ec40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec8a1f640 1 -- 172.21.15.26:0/2105291794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec4152c20 msgr2=0x7f9ec40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec8a1f640 1 --2- 172.21.15.26:0/2105291794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec4152c20 0x7f9ec40c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec8a1f640 1 -- 172.21.15.26:0/2105291794 >> v2:172.21.15.117:3300/0 conn(0x7f9ec414f650 msgr2=0x7f9ec4071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec9220640 1 --2- 172.21.15.26:0/2105291794 >> v2:172.21.15.117:3300/0 conn(0x7f9ec414f650 0x7f9ec4071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec8a1f640 1 --2- 172.21.15.26:0/2105291794 >> v2:172.21.15.117:3300/0 conn(0x7f9ec414f650 0x7f9ec4071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec8a1f640 1 -- 172.21.15.26:0/2105291794 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4152420 con 0x7f9ec40c3150 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.855+0000 7f9ec9220640 1 --2- 172.21.15.26:0/2105291794 >> v2:172.21.15.117:3300/0 conn(0x7f9ec414f650 0x7f9ec4071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ec8a1f640 1 --2- 172.21.15.26:0/2105291794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40c5690 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4070a90 tx=0x7f9eb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea8b1b14fb3f228 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ebbfff640 1 -- 172.21.15.26:0/2105291794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4035020 con 0x7f9ec40c3150 2024-03-06T07:29:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ebbfff640 1 -- 172.21.15.26:0/2105291794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb4031870 con 0x7f9ec40c3150 2024-03-06T07:29:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ebbfff640 1 -- 172.21.15.26:0/2105291794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4031c40 con 0x7f9ec40c3150 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 -- 172.21.15.26:0/2105291794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 msgr2=0x7f9ec40c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2105291794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40c5690 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4070a90 tx=0x7f9eb402b910 comp rx=0 tx=0).stop 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 -- 172.21.15.26:0/2105291794 shutdown_connections 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2105291794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec4152c20 0x7f9ec40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2105291794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40c5690 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2105291794 >> v2:172.21.15.117:3300/0 conn(0x7f9ec414f650 0x7f9ec4071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 -- 172.21.15.26:0/2105291794 >> 172.21.15.26:0/2105291794 conn(0x7f9ec40c4290 msgr2=0x7f9ec40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 -- 172.21.15.26:0/2105291794 shutdown_connections 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.856+0000 7f9ecb484640 1 -- 172.21.15.26:0/2105291794 wait complete. 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.858+0000 7f9ecb484640 1 Processor -- start 2024-03-06T07:29:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.858+0000 7f9ecb484640 1 -- start start 2024-03-06T07:29:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ecb484640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ecb484640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec414f650 0x7f9ec40aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ecb484640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ec4152c20 0x7f9ec40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ecb484640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40c40c0 con 0x7f9ec40c3150 2024-03-06T07:29:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ecb484640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40c3f40 con 0x7f9ec4152c20 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ecb484640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec4071dc0 con 0x7f9ec414f650 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ec9220640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ec9220640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40a81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37432/0 (socket says 172.21.15.26:37432) 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.859+0000 7f9ec9220640 1 -- 172.21.15.26:0/2261544150 learned_addr learned my addr 172.21.15.26:0/2261544150 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ec9220640 1 -- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec414f650 msgr2=0x7f9ec40aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ec9220640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec414f650 0x7f9ec40aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ec9220640 1 -- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ec4152c20 msgr2=0x7f9ec40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ec9220640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ec4152c20 0x7f9ec40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ec9220640 1 -- 172.21.15.26:0/2261544150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec406b710 con 0x7f9ec40c3150 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ec9220640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40a81b0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c7f0 tx=0x7f9ec000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9eb9ffb640 1 -- 172.21.15.26:0/2261544150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0015070 con 0x7f9ec40c3150 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9eb9ffb640 1 -- 172.21.15.26:0/2261544150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0007a30 con 0x7f9ec40c3150 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec406b9f0 con 0x7f9ec40c3150 2024-03-06T07:29:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9eb9ffb640 1 -- 172.21.15.26:0/2261544150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0007e00 con 0x7f9ec40c3150 2024-03-06T07:29:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.860+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec40a86f0 con 0x7f9ec40c3150 2024-03-06T07:29:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.861+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec4069460 con 0x7f9ec40c3150 2024-03-06T07:29:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.862+0000 7f9eb9ffb640 1 -- 172.21.15.26:0/2261544150 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ec0018070 con 0x7f9ec40c3150 2024-03-06T07:29:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.862+0000 7f9eb9ffb640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ea806b500 0x7f9ea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.862+0000 7f9eb9ffb640 1 -- 172.21.15.26:0/2261544150 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ec008a1e0 con 0x7f9ec40c3150 2024-03-06T07:29:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.862+0000 7f9ec8a1f640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ea806b500 0x7f9ea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.863+0000 7f9ec8a1f640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ea806b500 0x7f9ea806d9c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4036000 tx=0x7f9eb4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:17.866+0000 7f9eb9ffb640 1 -- 172.21.15.26:0/2261544150 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ec4069460 con 0x7f9ec40c3150 2024-03-06T07:29:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.055+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ec404f330 con 0x7f9ea806b500 2024-03-06T07:29:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.061+0000 7f9eb9ffb640 1 -- 172.21.15.26:0/2261544150 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252499 (secure 0 0 0) 0x7f9ec404f330 con 0x7f9ea806b500 2024-03-06T07:29:18.065 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ea806b500 msgr2=0x7f9ea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ea806b500 0x7f9ea806d9c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4036000 tx=0x7f9eb4030000 comp rx=0 tx=0).stop 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 msgr2=0x7f9ec40a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40a81b0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c7f0 tx=0x7f9ec000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 shutdown_connections 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9ea806b500 0x7f9ea806d9c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ec4152c20 0x7f9ec40ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ec414f650 0x7f9ec40aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 --2- 172.21.15.26:0/2261544150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ec40c3150 0x7f9ec40a81b0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 >> 172.21.15.26:0/2261544150 conn(0x7f9ec40c4290 msgr2=0x7f9ec4151340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 shutdown_connections 2024-03-06T07:29:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.064+0000 7f9ecb484640 1 -- 172.21.15.26:0/2261544150 wait complete. 2024-03-06T07:29:18.077 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-03-06T07:29:17.150622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282804,"kb_used_data":20864,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754691916,"statfs":{"total":773094113280,"available":772804521984,"internally_reserved":0,"allocated":21364736,"data_stored":11472496,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.148190"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031259+0000","last_change":"2024-03-06T07:29:15.030359+0000","last_active":"2024-03-06T07:29:15.031259+0000","last_peered":"2024-03-06T07:29:15.031259+0000","last_clean":"2024-03-06T07:29:15.031259+0000","last_became_active":"2024-03-06T07:29:15.030107+0000","last_became_peered":"2024-03-06T07:29:15.030107+0000","last_unstale":"2024-03-06T07:29:15.031259+0000","last_undegraded":"2024-03-06T07:29:15.031259+0000","last_fullsized":"2024-03-06T07:29:15.031259+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:56.810673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046685+0000","last_change":"2024-03-06T07:29:15.044690+0000","last_active":"2024-03-06T07:29:15.046685+0000","last_peered":"2024-03-06T07:29:15.046685+0000","last_clean":"2024-03-06T07:29:15.046685+0000","last_became_active":"2024-03-06T07:29:15.044382+0000","last_became_peered":"2024-03-06T07:29:15.044382+0000","last_unstale":"2024-03-06T07:29:15.046685+0000","last_undegraded":"2024-03-06T07:29:15.046685+0000","last_fullsized":"2024-03-06T07:29:15.046685+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:16.615660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.041892+0000","last_change":"2024-03-06T07:29:15.041290+0000","last_active":"2024-03-06T07:29:15.041892+0000","last_peered":"2024-03-06T07:29:15.041892+0000","last_clean":"2024-03-06T07:29:15.041892+0000","last_became_active":"2024-03-06T07:29:15.041026+0000","last_became_peered":"2024-03-06T07:29:15.041026+0000","last_unstale":"2024-03-06T07:29:15.041892+0000","last_undegraded":"2024-03-06T07:29:15.041892+0000","last_fullsized":"2024-03-06T07:29:15.041892+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:56:00.693221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031182+0000","last_change":"2024-03-06T07:29:15.029112+0000","last_active":"2024-03-06T07:29:15.031182+0000","last_peered":"2024-03-06T07:29:15.031182+0000","last_clean":"2024-03-06T07:29:15.031182+0000","last_became_active":"2024-03-06T07:29:15.028838+0000","last_became_peered":"2024-03-06T07:29:15.028838+0000","last_unstale":"2024-03-06T07:29:15.031182+0000","last_undegraded":"2024-03-06T07:29:15.031182+0000","last_fullsized":"2024-03-06T07:29:15.031182+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:55.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026239+0000","last_change":"2024-03-06T07:29:15.024030+0000","last_active":"2024-03-06T07:29:15.026239+0000","last_peered":"2024-03-06T07:29:15.026239+0000","last_clean":"2024-03-06T07:29:15.026239+0000","last_became_active":"2024-03-06T07:29:15.023518+0000","last_became_peered":"2024-03-06T07:29:15.023518+0000","last_unstale":"2024-03-06T07:29:15.026239+0000","last_undegraded":"2024-03-06T07:29:15.026239+0000","last_fullsized":"2024-03-06T07:29:15.026239+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:07:20.498162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030277+0000","last_change":"2024-03-06T07:29:15.028312+0000","last_active":"2024-03-06T07:29:15.030277+0000","last_peered":"2024-03-06T07:29:15.030277+0000","last_clean":"2024-03-06T07:29:15.030277+0000","last_became_active":"2024-03-06T07:29:15.028015+0000","last_became_peered":"2024-03-06T07:29:15.028015+0000","last_unstale":"2024-03-06T07:29:15.030277+0000","last_undegraded":"2024-03-06T07:29:15.030277+0000","last_fullsized":"2024-03-06T07:29:15.030277+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:14.164253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.023929+0000","last_change":"2024-03-06T07:29:15.023058+0000","last_active":"2024-03-06T07:29:15.023929+0000","last_peered":"2024-03-06T07:29:15.023929+0000","last_clean":"2024-03-06T07:29:15.023929+0000","last_became_active":"2024-03-06T07:29:15.022728+0000","last_became_peered":"2024-03-06T07:29:15.022728+0000","last_unstale":"2024-03-06T07:29:15.023929+0000","last_undegraded":"2024-03-06T07:29:15.023929+0000","last_fullsized":"2024-03-06T07:29:15.023929+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:44:58.382436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042941+0000","last_change":"2024-03-06T07:29:15.027529+0000","last_active":"2024-03-06T07:29:15.042941+0000","last_peered":"2024-03-06T07:29:15.042941+0000","last_clean":"2024-03-06T07:29:15.042941+0000","last_became_active":"2024-03-06T07:29:15.027245+0000","last_became_peered":"2024-03-06T07:29:15.027245+0000","last_unstale":"2024-03-06T07:29:15.042941+0000","last_undegraded":"2024-03-06T07:29:15.042941+0000","last_fullsized":"2024-03-06T07:29:15.042941+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:53.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030261+0000","last_change":"2024-03-06T07:29:15.026744+0000","last_active":"2024-03-06T07:29:15.030261+0000","last_peered":"2024-03-06T07:29:15.030261+0000","last_clean":"2024-03-06T07:29:15.030261+0000","last_became_active":"2024-03-06T07:29:15.026477+0000","last_became_peered":"2024-03-06T07:29:15.026477+0000","last_unstale":"2024-03-06T07:29:15.030261+0000","last_undegraded":"2024-03-06T07:29:15.030261+0000","last_fullsized":"2024-03-06T07:29:15.030261+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:22.613234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031356+0000","last_change":"2024-03-06T07:29:15.030879+0000","last_active":"2024-03-06T07:29:15.031356+0000","last_peered":"2024-03-06T07:29:15.031356+0000","last_clean":"2024-03-06T07:29:15.031356+0000","last_became_active":"2024-03-06T07:29:15.030605+0000","last_became_peered":"2024-03-06T07:29:15.030605+0000","last_unstale":"2024-03-06T07:29:15.031356+0000","last_undegraded":"2024-03-06T07:29:15.031356+0000","last_fullsized":"2024-03-06T07:29:15.031356+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:41:37.685097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.027020+0000","last_change":"2024-03-06T07:29:15.023196+0000","last_active":"2024-03-06T07:29:15.027020+0000","last_peered":"2024-03-06T07:29:15.027020+0000","last_clean":"2024-03-06T07:29:15.027020+0000","last_became_active":"2024-03-06T07:29:15.022906+0000","last_became_peered":"2024-03-06T07:29:15.022906+0000","last_unstale":"2024-03-06T07:29:15.027020+0000","last_undegraded":"2024-03-06T07:29:15.027020+0000","last_fullsized":"2024-03-06T07:29:15.027020+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:50.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.029548+0000","last_change":"2024-03-06T07:29:15.026121+0000","last_active":"2024-03-06T07:29:15.029548+0000","last_peered":"2024-03-06T07:29:15.029548+0000","last_clean":"2024-03-06T07:29:15.029548+0000","last_became_active":"2024-03-06T07:29:15.025864+0000","last_became_peered":"2024-03-06T07:29:15.025864+0000","last_unstale":"2024-03-06T07:29:15.029548+0000","last_undegraded":"2024-03-06T07:29:15.029548+0000","last_fullsized":"2024-03-06T07:29:15.029548+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:29.989262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044764+0000","last_change":"2024-03-06T07:29:15.036581+0000","last_active":"2024-03-06T07:29:15.044764+0000","last_peered":"2024-03-06T07:29:15.044764+0000","last_clean":"2024-03-06T07:29:15.044764+0000","last_became_active":"2024-03-06T07:29:15.031104+0000","last_became_peered":"2024-03-06T07:29:15.031104+0000","last_unstale":"2024-03-06T07:29:15.044764+0000","last_undegraded":"2024-03-06T07:29:15.044764+0000","last_fullsized":"2024-03-06T07:29:15.044764+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:09:13.391272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045843+0000","last_change":"2024-03-06T07:29:15.042749+0000","last_active":"2024-03-06T07:29:15.045843+0000","last_peered":"2024-03-06T07:29:15.045843+0000","last_clean":"2024-03-06T07:29:15.045843+0000","last_became_active":"2024-03-06T07:29:15.042484+0000","last_became_peered":"2024-03-06T07:29:15.042484+0000","last_unstale":"2024-03-06T07:29:15.045843+0000","last_undegraded":"2024-03-06T07:29:15.045843+0000","last_fullsized":"2024-03-06T07:29:15.045843+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:38:42.566990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044937+0000","last_change":"2024-03-06T07:29:15.041280+0000","last_active":"2024-03-06T07:29:15.044937+0000","last_peered":"2024-03-06T07:29:15.044937+0000","last_clean":"2024-03-06T07:29:15.044937+0000","last_became_active":"2024-03-06T07:29:15.040956+0000","last_became_peered":"2024-03-06T07:29:15.040956+0000","last_unstale":"2024-03-06T07:29:15.044937+0000","last_undegraded":"2024-03-06T07:29:15.044937+0000","last_fullsized":"2024-03-06T07:29:15.044937+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:04.880888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038790+0000","last_change":"2024-03-06T07:29:15.037372+0000","last_active":"2024-03-06T07:29:15.038790+0000","last_peered":"2024-03-06T07:29:15.038790+0000","last_clean":"2024-03-06T07:29:15.038790+0000","last_became_active":"2024-03-06T07:29:15.037082+0000","last_became_peered":"2024-03-06T07:29:15.037082+0000","last_unstale":"2024-03-06T07:29:15.038790+0000","last_undegraded":"2024-03-06T07:29:15.038790+0000","last_fullsized":"2024-03-06T07:29:15.038790+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:17:26.569423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033184+0000","last_change":"2024-03-06T07:29:15.032203+0000","last_active":"2024-03-06T07:29:15.033184+0000","last_peered":"2024-03-06T07:29:15.033184+0000","last_clean":"2024-03-06T07:29:15.033184+0000","last_became_active":"2024-03-06T07:29:15.031852+0000","last_became_peered":"2024-03-06T07:29:15.031852+0000","last_unstale":"2024-03-06T07:29:15.033184+0000","last_undegraded":"2024-03-06T07:29:15.033184+0000","last_fullsized":"2024-03-06T07:29:15.033184+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:44.233028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039915+0000","last_change":"2024-03-06T07:29:15.029839+0000","last_active":"2024-03-06T07:29:15.039915+0000","last_peered":"2024-03-06T07:29:15.039915+0000","last_clean":"2024-03-06T07:29:15.039915+0000","last_became_active":"2024-03-06T07:29:15.029541+0000","last_became_peered":"2024-03-06T07:29:15.029541+0000","last_unstale":"2024-03-06T07:29:15.039915+0000","last_undegraded":"2024-03-06T07:29:15.039915+0000","last_fullsized":"2024-03-06T07:29:15.039915+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:27.084764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026437+0000","last_change":"2024-03-06T07:29:15.023928+0000","last_active":"2024-03-06T07:29:15.026437+0000","last_peered":"2024-03-06T07:29:15.026437+0000","last_clean":"2024-03-06T07:29:15.026437+0000","last_became_active":"2024-03-06T07:29:15.023662+0000","last_became_peered":"2024-03-06T07:29:15.023662+0000","last_unstale":"2024-03-06T07:29:15.026437+0000","last_undegraded":"2024-03-06T07:29:15.026437+0000","last_fullsized":"2024-03-06T07:29:15.026437+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:00:58.174701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032940+0000","last_change":"2024-03-06T07:29:15.031416+0000","last_active":"2024-03-06T07:29:15.032940+0000","last_peered":"2024-03-06T07:29:15.032940+0000","last_clean":"2024-03-06T07:29:15.032940+0000","last_became_active":"2024-03-06T07:29:15.030352+0000","last_became_peered":"2024-03-06T07:29:15.030352+0000","last_unstale":"2024-03-06T07:29:15.032940+0000","last_undegraded":"2024-03-06T07:29:15.032940+0000","last_fullsized":"2024-03-06T07:29:15.032940+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:53.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026216+0000","last_change":"2024-03-06T07:29:15.023795+0000","last_active":"2024-03-06T07:29:15.026216+0000","last_peered":"2024-03-06T07:29:15.026216+0000","last_clean":"2024-03-06T07:29:15.026216+0000","last_became_active":"2024-03-06T07:29:15.023151+0000","last_became_peered":"2024-03-06T07:29:15.023151+0000","last_unstale":"2024-03-06T07:29:15.026216+0000","last_undegraded":"2024-03-06T07:29:15.026216+0000","last_fullsized":"2024-03-06T07:29:15.026216+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:15:43.422349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033599+0000","last_change":"2024-03-06T07:29:15.032126+0000","last_active":"2024-03-06T07:29:15.033599+0000","last_peered":"2024-03-06T07:29:15.033599+0000","last_clean":"2024-03-06T07:29:15.033599+0000","last_became_active":"2024-03-06T07:29:15.031854+0000","last_became_peered":"2024-03-06T07:29:15.031854+0000","last_unstale":"2024-03-06T07:29:15.033599+0000","last_undegraded":"2024-03-06T07:29:15.033599+0000","last_fullsized":"2024-03-06T07:29:15.033599+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:17:13.696237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045866+0000","last_change":"2024-03-06T07:29:15.041977+0000","last_active":"2024-03-06T07:29:15.045866+0000","last_peered":"2024-03-06T07:29:15.045866+0000","last_clean":"2024-03-06T07:29:15.045866+0000","last_became_active":"2024-03-06T07:29:15.041219+0000","last_became_peered":"2024-03-06T07:29:15.041219+0000","last_unstale":"2024-03-06T07:29:15.045866+0000","last_undegraded":"2024-03-06T07:29:15.045866+0000","last_fullsized":"2024-03-06T07:29:15.045866+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:41.684370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.022830+0000","last_change":"2024-03-06T07:29:15.022028+0000","last_active":"2024-03-06T07:29:15.022830+0000","last_peered":"2024-03-06T07:29:15.022830+0000","last_clean":"2024-03-06T07:29:15.022830+0000","last_became_active":"2024-03-06T07:29:15.021744+0000","last_became_peered":"2024-03-06T07:29:15.021744+0000","last_unstale":"2024-03-06T07:29:15.022830+0000","last_undegraded":"2024-03-06T07:29:15.022830+0000","last_fullsized":"2024-03-06T07:29:15.022830+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:16.310455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032500+0000","last_change":"2024-03-06T07:29:15.028134+0000","last_active":"2024-03-06T07:29:15.032500+0000","last_peered":"2024-03-06T07:29:15.032500+0000","last_clean":"2024-03-06T07:29:15.032500+0000","last_became_active":"2024-03-06T07:29:15.027846+0000","last_became_peered":"2024-03-06T07:29:15.027846+0000","last_unstale":"2024-03-06T07:29:15.032500+0000","last_undegraded":"2024-03-06T07:29:15.032500+0000","last_fullsized":"2024-03-06T07:29:15.032500+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.591638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.040756+0000","last_change":"2024-03-06T07:29:15.034571+0000","last_active":"2024-03-06T07:29:15.040756+0000","last_peered":"2024-03-06T07:29:15.040756+0000","last_clean":"2024-03-06T07:29:15.040756+0000","last_became_active":"2024-03-06T07:29:15.034277+0000","last_became_peered":"2024-03-06T07:29:15.034277+0000","last_unstale":"2024-03-06T07:29:15.040756+0000","last_undegraded":"2024-03-06T07:29:15.040756+0000","last_fullsized":"2024-03-06T07:29:15.040756+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:01.748829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033965+0000","last_change":"2024-03-06T07:29:15.033322+0000","last_active":"2024-03-06T07:29:15.033965+0000","last_peered":"2024-03-06T07:29:15.033965+0000","last_clean":"2024-03-06T07:29:15.033965+0000","last_became_active":"2024-03-06T07:29:15.032742+0000","last_became_peered":"2024-03-06T07:29:15.032742+0000","last_unstale":"2024-03-06T07:29:15.033965+0000","last_undegraded":"2024-03-06T07:29:15.033965+0000","last_fullsized":"2024-03-06T07:29:15.033965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:44:04.457241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031233+0000","last_change":"2024-03-06T07:29:15.029682+0000","last_active":"2024-03-06T07:29:15.031233+0000","last_peered":"2024-03-06T07:29:15.031233+0000","last_clean":"2024-03-06T07:29:15.031233+0000","last_became_active":"2024-03-06T07:29:15.029422+0000","last_became_peered":"2024-03-06T07:29:15.029422+0000","last_unstale":"2024-03-06T07:29:15.031233+0000","last_undegraded":"2024-03-06T07:29:15.031233+0000","last_fullsized":"2024-03-06T07:29:15.031233+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:26:15.661395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030255+0000","last_change":"2024-03-06T07:29:15.027659+0000","last_active":"2024-03-06T07:29:15.030255+0000","last_peered":"2024-03-06T07:29:15.030255+0000","last_clean":"2024-03-06T07:29:15.030255+0000","last_became_active":"2024-03-06T07:29:15.027385+0000","last_became_peered":"2024-03-06T07:29:15.027385+0000","last_unstale":"2024-03-06T07:29:15.030255+0000","last_undegraded":"2024-03-06T07:29:15.030255+0000","last_fullsized":"2024-03-06T07:29:15.030255+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:25.493710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.048014+0000","last_change":"2024-03-06T07:29:15.047343+0000","last_active":"2024-03-06T07:29:15.048014+0000","last_peered":"2024-03-06T07:29:15.048014+0000","last_clean":"2024-03-06T07:29:15.048014+0000","last_became_active":"2024-03-06T07:29:15.047045+0000","last_became_peered":"2024-03-06T07:29:15.047045+0000","last_unstale":"2024-03-06T07:29:15.048014+0000","last_undegraded":"2024-03-06T07:29:15.048014+0000","last_fullsized":"2024-03-06T07:29:15.048014+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:24.336860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044726+0000","last_change":"2024-03-06T07:29:15.041521+0000","last_active":"2024-03-06T07:29:15.044726+0000","last_peered":"2024-03-06T07:29:15.044726+0000","last_clean":"2024-03-06T07:29:15.044726+0000","last_became_active":"2024-03-06T07:29:15.041009+0000","last_became_peered":"2024-03-06T07:29:15.041009+0000","last_unstale":"2024-03-06T07:29:15.044726+0000","last_undegraded":"2024-03-06T07:29:15.044726+0000","last_fullsized":"2024-03-06T07:29:15.044726+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:25.185955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028340+0000","last_change":"2024-03-06T07:29:15.025968+0000","last_active":"2024-03-06T07:29:15.028340+0000","last_peered":"2024-03-06T07:29:15.028340+0000","last_clean":"2024-03-06T07:29:15.028340+0000","last_became_active":"2024-03-06T07:29:15.025691+0000","last_became_peered":"2024-03-06T07:29:15.025691+0000","last_unstale":"2024-03-06T07:29:15.028340+0000","last_undegraded":"2024-03-06T07:29:15.028340+0000","last_fullsized":"2024-03-06T07:29:15.028340+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:15.074091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"210'4","reported_seq":16,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045627+0000","last_change":"2024-03-06T07:29:14.025579+0000","last_active":"2024-03-06T07:29:15.045627+0000","last_peered":"2024-03-06T07:29:15.045627+0000","last_clean":"2024-03-06T07:29:15.045627+0000","last_became_active":"2024-03-06T07:29:14.025299+0000","last_became_peered":"2024-03-06T07:29:14.025299+0000","last_unstale":"2024-03-06T07:29:15.045627+0000","last_undegraded":"2024-03-06T07:29:15.045627+0000","last_fullsized":"2024-03-06T07:29:15.045627+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:01.785227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039852+0000","last_change":"2024-03-06T07:29:15.027795+0000","last_active":"2024-03-06T07:29:15.039852+0000","last_peered":"2024-03-06T07:29:15.039852+0000","last_clean":"2024-03-06T07:29:15.039852+0000","last_became_active":"2024-03-06T07:29:15.027503+0000","last_became_peered":"2024-03-06T07:29:15.027503+0000","last_unstale":"2024-03-06T07:29:15.039852+0000","last_undegraded":"2024-03-06T07:29:15.039852+0000","last_fullsized":"2024-03-06T07:29:15.039852+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:21.292222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.036436+0000","last_change":"2024-03-06T07:29:14.024503+0000","last_active":"2024-03-06T07:29:15.036436+0000","last_peered":"2024-03-06T07:29:15.036436+0000","last_clean":"2024-03-06T07:29:15.036436+0000","last_became_active":"2024-03-06T07:29:14.024197+0000","last_became_peered":"2024-03-06T07:29:14.024197+0000","last_unstale":"2024-03-06T07:29:15.036436+0000","last_undegraded":"2024-03-06T07:29:15.036436+0000","last_fullsized":"2024-03-06T07:29:15.036436+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:53.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026387+0000","last_change":"2024-03-06T07:29:15.025683+0000","last_active":"2024-03-06T07:29:15.026387+0000","last_peered":"2024-03-06T07:29:15.026387+0000","last_clean":"2024-03-06T07:29:15.026387+0000","last_became_active":"2024-03-06T07:29:15.021856+0000","last_became_peered":"2024-03-06T07:29:15.021856+0000","last_unstale":"2024-03-06T07:29:15.026387+0000","last_undegraded":"2024-03-06T07:29:15.026387+0000","last_fullsized":"2024-03-06T07:29:15.026387+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:58:25.688284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.041617+0000","last_change":"2024-03-06T07:29:14.026365+0000","last_active":"2024-03-06T07:29:15.041617+0000","last_peered":"2024-03-06T07:29:15.041617+0000","last_clean":"2024-03-06T07:29:15.041617+0000","last_became_active":"2024-03-06T07:29:14.026078+0000","last_became_peered":"2024-03-06T07:29:14.026078+0000","last_unstale":"2024-03-06T07:29:15.041617+0000","last_undegraded":"2024-03-06T07:29:15.041617+0000","last_fullsized":"2024-03-06T07:29:15.041617+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:27:49.657594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.040752+0000","last_change":"2024-03-06T07:29:15.035819+0000","last_active":"2024-03-06T07:29:15.040752+0000","last_peered":"2024-03-06T07:29:15.040752+0000","last_clean":"2024-03-06T07:29:15.040752+0000","last_became_active":"2024-03-06T07:29:15.035436+0000","last_became_peered":"2024-03-06T07:29:15.035436+0000","last_unstale":"2024-03-06T07:29:15.040752+0000","last_undegraded":"2024-03-06T07:29:15.040752+0000","last_fullsized":"2024-03-06T07:29:15.040752+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.594027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038458+0000","last_change":"2024-03-06T07:29:14.027623+0000","last_active":"2024-03-06T07:29:15.038458+0000","last_peered":"2024-03-06T07:29:15.038458+0000","last_clean":"2024-03-06T07:29:15.038458+0000","last_became_active":"2024-03-06T07:29:14.027352+0000","last_became_peered":"2024-03-06T07:29:14.027352+0000","last_unstale":"2024-03-06T07:29:15.038458+0000","last_undegraded":"2024-03-06T07:29:15.038458+0000","last_fullsized":"2024-03-06T07:29:15.038458+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.380578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039194+0000","last_change":"2024-03-06T07:29:15.032654+0000","last_active":"2024-03-06T07:29:15.039194+0000","last_peered":"2024-03-06T07:29:15.039194+0000","last_clean":"2024-03-06T07:29:15.039194+0000","last_became_active":"2024-03-06T07:29:15.032286+0000","last_became_peered":"2024-03-06T07:29:15.032286+0000","last_unstale":"2024-03-06T07:29:15.039194+0000","last_undegraded":"2024-03-06T07:29:15.039194+0000","last_fullsized":"2024-03-06T07:29:15.039194+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:53.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038537+0000","last_change":"2024-03-06T07:29:14.029242+0000","last_active":"2024-03-06T07:29:15.038537+0000","last_peered":"2024-03-06T07:29:15.038537+0000","last_clean":"2024-03-06T07:29:15.038537+0000","last_became_active":"2024-03-06T07:29:14.028975+0000","last_became_peered":"2024-03-06T07:29:14.028975+0000","last_unstale":"2024-03-06T07:29:15.038537+0000","last_undegraded":"2024-03-06T07:29:15.038537+0000","last_fullsized":"2024-03-06T07:29:15.038537+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:30.997292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042406+0000","last_change":"2024-03-06T07:29:15.035383+0000","last_active":"2024-03-06T07:29:15.042406+0000","last_peered":"2024-03-06T07:29:15.042406+0000","last_clean":"2024-03-06T07:29:15.042406+0000","last_became_active":"2024-03-06T07:29:15.034879+0000","last_became_peered":"2024-03-06T07:29:15.034879+0000","last_unstale":"2024-03-06T07:29:15.042406+0000","last_undegraded":"2024-03-06T07:29:15.042406+0000","last_fullsized":"2024-03-06T07:29:15.042406+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:07:20.498162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046675+0000","last_change":"2024-03-06T07:29:14.033003+0000","last_active":"2024-03-06T07:29:15.046675+0000","last_peered":"2024-03-06T07:29:15.046675+0000","last_clean":"2024-03-06T07:29:15.046675+0000","last_became_active":"2024-03-06T07:29:14.032291+0000","last_became_peered":"2024-03-06T07:29:14.032291+0000","last_unstale":"2024-03-06T07:29:15.046675+0000","last_undegraded":"2024-03-06T07:29:15.046675+0000","last_fullsized":"2024-03-06T07:29:15.046675+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:30.997292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038160+0000","last_change":"2024-03-06T07:29:15.037310+0000","last_active":"2024-03-06T07:29:15.038160+0000","last_peered":"2024-03-06T07:29:15.038160+0000","last_clean":"2024-03-06T07:29:15.038160+0000","last_became_active":"2024-03-06T07:29:15.036999+0000","last_became_peered":"2024-03-06T07:29:15.036999+0000","last_unstale":"2024-03-06T07:29:15.038160+0000","last_undegraded":"2024-03-06T07:29:15.038160+0000","last_fullsized":"2024-03-06T07:29:15.038160+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:45:01.464817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017153+0000","last_change":"2024-03-06T07:29:14.025905+0000","last_active":"2024-03-06T07:29:15.017153+0000","last_peered":"2024-03-06T07:29:15.017153+0000","last_clean":"2024-03-06T07:29:15.017153+0000","last_became_active":"2024-03-06T07:29:14.025619+0000","last_became_peered":"2024-03-06T07:29:14.025619+0000","last_unstale":"2024-03-06T07:29:15.017153+0000","last_undegraded":"2024-03-06T07:29:15.017153+0000","last_fullsized":"2024-03-06T07:29:15.017153+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:21.246584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044843+0000","last_change":"2024-03-06T07:29:15.031414+0000","last_active":"2024-03-06T07:29:15.044843+0000","last_peered":"2024-03-06T07:29:15.044843+0000","last_clean":"2024-03-06T07:29:15.044843+0000","last_became_active":"2024-03-06T07:29:15.029802+0000","last_became_peered":"2024-03-06T07:29:15.029802+0000","last_unstale":"2024-03-06T07:29:15.044843+0000","last_undegraded":"2024-03-06T07:29:15.044843+0000","last_fullsized":"2024-03-06T07:29:15.044843+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:35.463008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.011892+0000","last_change":"2024-03-06T07:29:14.028519+0000","last_active":"2024-03-06T07:29:15.011892+0000","last_peered":"2024-03-06T07:29:15.011892+0000","last_clean":"2024-03-06T07:29:15.011892+0000","last_became_active":"2024-03-06T07:29:14.027431+0000","last_became_peered":"2024-03-06T07:29:14.027431+0000","last_unstale":"2024-03-06T07:29:15.011892+0000","last_undegraded":"2024-03-06T07:29:15.011892+0000","last_fullsized":"2024-03-06T07:29:15.011892+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:22.677326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045056+0000","last_change":"2024-03-06T07:29:15.037780+0000","last_active":"2024-03-06T07:29:15.045056+0000","last_peered":"2024-03-06T07:29:15.045056+0000","last_clean":"2024-03-06T07:29:15.045056+0000","last_became_active":"2024-03-06T07:29:15.034710+0000","last_became_peered":"2024-03-06T07:29:15.034710+0000","last_unstale":"2024-03-06T07:29:15.045056+0000","last_undegraded":"2024-03-06T07:29:15.045056+0000","last_fullsized":"2024-03-06T07:29:15.045056+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:50.764109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017291+0000","last_change":"2024-03-06T07:29:14.029442+0000","last_active":"2024-03-06T07:29:15.017291+0000","last_peered":"2024-03-06T07:29:15.017291+0000","last_clean":"2024-03-06T07:29:15.017291+0000","last_became_active":"2024-03-06T07:29:14.029168+0000","last_became_peered":"2024-03-06T07:29:14.029168+0000","last_unstale":"2024-03-06T07:29:15.017291+0000","last_undegraded":"2024-03-06T07:29:15.017291+0000","last_fullsized":"2024-03-06T07:29:15.017291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:56.121417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039875+0000","last_change":"2024-03-06T07:29:15.029999+0000","last_active":"2024-03-06T07:29:15.039875+0000","last_peered":"2024-03-06T07:29:15.039875+0000","last_clean":"2024-03-06T07:29:15.039875+0000","last_became_active":"2024-03-06T07:29:15.029691+0000","last_became_peered":"2024-03-06T07:29:15.029691+0000","last_unstale":"2024-03-06T07:29:15.039875+0000","last_undegraded":"2024-03-06T07:29:15.039875+0000","last_fullsized":"2024-03-06T07:29:15.039875+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:07:58.543396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.019279+0000","last_change":"2024-03-06T07:29:14.025372+0000","last_active":"2024-03-06T07:29:15.019279+0000","last_peered":"2024-03-06T07:29:15.019279+0000","last_clean":"2024-03-06T07:29:15.019279+0000","last_became_active":"2024-03-06T07:29:14.025042+0000","last_became_peered":"2024-03-06T07:29:14.025042+0000","last_unstale":"2024-03-06T07:29:15.019279+0000","last_undegraded":"2024-03-06T07:29:15.019279+0000","last_fullsized":"2024-03-06T07:29:15.019279+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:45.924264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030092+0000","last_change":"2024-03-06T07:29:15.028930+0000","last_active":"2024-03-06T07:29:15.030092+0000","last_peered":"2024-03-06T07:29:15.030092+0000","last_clean":"2024-03-06T07:29:15.030092+0000","last_became_active":"2024-03-06T07:29:15.028645+0000","last_became_peered":"2024-03-06T07:29:15.028645+0000","last_unstale":"2024-03-06T07:29:15.030092+0000","last_undegraded":"2024-03-06T07:29:15.030092+0000","last_fullsized":"2024-03-06T07:29:15.030092+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:23.687635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"210'3","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044128+0000","last_change":"2024-03-06T07:29:14.028366+0000","last_active":"2024-03-06T07:29:15.044128+0000","last_peered":"2024-03-06T07:29:15.044128+0000","last_clean":"2024-03-06T07:29:15.044128+0000","last_became_active":"2024-03-06T07:29:14.027685+0000","last_became_peered":"2024-03-06T07:29:14.027685+0000","last_unstale":"2024-03-06T07:29:15.044128+0000","last_undegraded":"2024-03-06T07:29:15.044128+0000","last_fullsized":"2024-03-06T07:29:15.044128+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:10.535807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028575+0000","last_change":"2024-03-06T07:29:15.026042+0000","last_active":"2024-03-06T07:29:15.028575+0000","last_peered":"2024-03-06T07:29:15.028575+0000","last_clean":"2024-03-06T07:29:15.028575+0000","last_became_active":"2024-03-06T07:29:15.020873+0000","last_became_peered":"2024-03-06T07:29:15.020873+0000","last_unstale":"2024-03-06T07:29:15.028575+0000","last_undegraded":"2024-03-06T07:29:15.028575+0000","last_fullsized":"2024-03-06T07:29:15.028575+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:14:30.667346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"210'6","reported_seq":18,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.048450+0000","last_change":"2024-03-06T07:29:14.027068+0000","last_active":"2024-03-06T07:29:15.048450+0000","last_peered":"2024-03-06T07:29:15.048450+0000","last_clean":"2024-03-06T07:29:15.048450+0000","last_became_active":"2024-03-06T07:29:14.026793+0000","last_became_peered":"2024-03-06T07:29:14.026793+0000","last_unstale":"2024-03-06T07:29:15.048450+0000","last_undegraded":"2024-03-06T07:29:15.048450+0000","last_fullsized":"2024-03-06T07:29:15.048450+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:35:49.995437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026797+0000","last_change":"2024-03-06T07:29:15.024412+0000","last_active":"2024-03-06T07:29:15.026797+0000","last_peered":"2024-03-06T07:29:15.026797+0000","last_clean":"2024-03-06T07:29:15.026797+0000","last_became_active":"2024-03-06T07:29:15.024114+0000","last_became_peered":"2024-03-06T07:29:15.024114+0000","last_unstale":"2024-03-06T07:29:15.026797+0000","last_undegraded":"2024-03-06T07:29:15.026797+0000","last_fullsized":"2024-03-06T07:29:15.026797+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.594027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038047+0000","last_change":"2024-03-06T07:29:14.030141+0000","last_active":"2024-03-06T07:29:15.038047+0000","last_peered":"2024-03-06T07:29:15.038047+0000","last_clean":"2024-03-06T07:29:15.038047+0000","last_became_active":"2024-03-06T07:29:14.029877+0000","last_became_peered":"2024-03-06T07:29:14.029877+0000","last_unstale":"2024-03-06T07:29:15.038047+0000","last_undegraded":"2024-03-06T07:29:15.038047+0000","last_fullsized":"2024-03-06T07:29:15.038047+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:52.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.043723+0000","last_change":"2024-03-06T07:29:15.043384+0000","last_active":"2024-03-06T07:29:15.043723+0000","last_peered":"2024-03-06T07:29:15.043723+0000","last_clean":"2024-03-06T07:29:15.043723+0000","last_became_active":"2024-03-06T07:29:15.043156+0000","last_became_peered":"2024-03-06T07:29:15.043156+0000","last_unstale":"2024-03-06T07:29:15.043723+0000","last_undegraded":"2024-03-06T07:29:15.043723+0000","last_fullsized":"2024-03-06T07:29:15.043723+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:23:50.702294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.020821+0000","last_change":"2024-03-06T07:29:14.025602+0000","last_active":"2024-03-06T07:29:15.020821+0000","last_peered":"2024-03-06T07:29:15.020821+0000","last_clean":"2024-03-06T07:29:15.020821+0000","last_became_active":"2024-03-06T07:29:14.025161+0000","last_became_peered":"2024-03-06T07:29:14.025161+0000","last_unstale":"2024-03-06T07:29:15.020821+0000","last_undegraded":"2024-03-06T07:29:15.020821+0000","last_fullsized":"2024-03-06T07:29:15.020821+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:50.289371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.027893+0000","last_change":"2024-03-06T07:29:15.027434+0000","last_active":"2024-03-06T07:29:15.027893+0000","last_peered":"2024-03-06T07:29:15.027893+0000","last_clean":"2024-03-06T07:29:15.027893+0000","last_became_active":"2024-03-06T07:29:15.027155+0000","last_became_peered":"2024-03-06T07:29:15.027155+0000","last_unstale":"2024-03-06T07:29:15.027893+0000","last_undegraded":"2024-03-06T07:29:15.027893+0000","last_fullsized":"2024-03-06T07:29:15.027893+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:54.219263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017557+0000","last_change":"2024-03-06T07:29:14.022979+0000","last_active":"2024-03-06T07:29:15.017557+0000","last_peered":"2024-03-06T07:29:15.017557+0000","last_clean":"2024-03-06T07:29:15.017557+0000","last_became_active":"2024-03-06T07:29:14.022666+0000","last_became_peered":"2024-03-06T07:29:14.022666+0000","last_unstale":"2024-03-06T07:29:15.017557+0000","last_undegraded":"2024-03-06T07:29:15.017557+0000","last_fullsized":"2024-03-06T07:29:15.017557+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:30:33.696105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038913+0000","last_change":"2024-03-06T07:29:15.037471+0000","last_active":"2024-03-06T07:29:15.038913+0000","last_peered":"2024-03-06T07:29:15.038913+0000","last_clean":"2024-03-06T07:29:15.038913+0000","last_became_active":"2024-03-06T07:29:15.037184+0000","last_became_peered":"2024-03-06T07:29:15.037184+0000","last_unstale":"2024-03-06T07:29:15.038913+0000","last_undegraded":"2024-03-06T07:29:15.038913+0000","last_fullsized":"2024-03-06T07:29:15.038913+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:04.880888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"210'8","reported_seq":20,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.048843+0000","last_change":"2024-03-06T07:29:14.016029+0000","last_active":"2024-03-06T07:29:15.048843+0000","last_peered":"2024-03-06T07:29:15.048843+0000","last_clean":"2024-03-06T07:29:15.048843+0000","last_became_active":"2024-03-06T07:29:14.015756+0000","last_became_peered":"2024-03-06T07:29:14.015756+0000","last_unstale":"2024-03-06T07:29:15.048843+0000","last_undegraded":"2024-03-06T07:29:15.048843+0000","last_fullsized":"2024-03-06T07:29:15.048843+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:19.827128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030688+0000","last_change":"2024-03-06T07:29:15.027834+0000","last_active":"2024-03-06T07:29:15.030688+0000","last_peered":"2024-03-06T07:29:15.030688+0000","last_clean":"2024-03-06T07:29:15.030688+0000","last_became_active":"2024-03-06T07:29:15.026590+0000","last_became_peered":"2024-03-06T07:29:15.026590+0000","last_unstale":"2024-03-06T07:29:15.030688+0000","last_undegraded":"2024-03-06T07:29:15.030688+0000","last_fullsized":"2024-03-06T07:29:15.030688+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:32:58.884800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046036+0000","last_change":"2024-03-06T07:29:14.027082+0000","last_active":"2024-03-06T07:29:15.046036+0000","last_peered":"2024-03-06T07:29:15.046036+0000","last_clean":"2024-03-06T07:29:15.046036+0000","last_became_active":"2024-03-06T07:29:14.026824+0000","last_became_peered":"2024-03-06T07:29:14.026824+0000","last_unstale":"2024-03-06T07:29:15.046036+0000","last_undegraded":"2024-03-06T07:29:15.046036+0000","last_fullsized":"2024-03-06T07:29:15.046036+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:40.850202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028684+0000","last_change":"2024-03-06T07:29:15.023321+0000","last_active":"2024-03-06T07:29:15.028684+0000","last_peered":"2024-03-06T07:29:15.028684+0000","last_clean":"2024-03-06T07:29:15.028684+0000","last_became_active":"2024-03-06T07:29:15.023004+0000","last_became_peered":"2024-03-06T07:29:15.023004+0000","last_unstale":"2024-03-06T07:29:15.028684+0000","last_undegraded":"2024-03-06T07:29:15.028684+0000","last_fullsized":"2024-03-06T07:29:15.028684+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:10:55.418263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045959+0000","last_change":"2024-03-06T07:29:14.028584+0000","last_active":"2024-03-06T07:29:15.045959+0000","last_peered":"2024-03-06T07:29:15.045959+0000","last_clean":"2024-03-06T07:29:15.045959+0000","last_became_active":"2024-03-06T07:29:14.028176+0000","last_became_peered":"2024-03-06T07:29:14.028176+0000","last_unstale":"2024-03-06T07:29:15.045959+0000","last_undegraded":"2024-03-06T07:29:15.045959+0000","last_fullsized":"2024-03-06T07:29:15.045959+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:13.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044290+0000","last_change":"2024-03-06T07:29:15.042753+0000","last_active":"2024-03-06T07:29:15.044290+0000","last_peered":"2024-03-06T07:29:15.044290+0000","last_clean":"2024-03-06T07:29:15.044290+0000","last_became_active":"2024-03-06T07:29:15.035626+0000","last_became_peered":"2024-03-06T07:29:15.035626+0000","last_unstale":"2024-03-06T07:29:15.044290+0000","last_undegraded":"2024-03-06T07:29:15.044290+0000","last_fullsized":"2024-03-06T07:29:15.044290+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:33.060067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045061+0000","last_change":"2024-03-06T07:29:14.028516+0000","last_active":"2024-03-06T07:29:15.045061+0000","last_peered":"2024-03-06T07:29:15.045061+0000","last_clean":"2024-03-06T07:29:15.045061+0000","last_became_active":"2024-03-06T07:29:14.028292+0000","last_became_peered":"2024-03-06T07:29:14.028292+0000","last_unstale":"2024-03-06T07:29:15.045061+0000","last_undegraded":"2024-03-06T07:29:15.045061+0000","last_fullsized":"2024-03-06T07:29:15.045061+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:55.834599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033959+0000","last_change":"2024-03-06T07:29:15.033315+0000","last_active":"2024-03-06T07:29:15.033959+0000","last_peered":"2024-03-06T07:29:15.033959+0000","last_clean":"2024-03-06T07:29:15.033959+0000","last_became_active":"2024-03-06T07:29:15.032417+0000","last_became_peered":"2024-03-06T07:29:15.032417+0000","last_unstale":"2024-03-06T07:29:15.033959+0000","last_undegraded":"2024-03-06T07:29:15.033959+0000","last_fullsized":"2024-03-06T07:29:15.033959+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:26:57.333190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016801+0000","last_change":"2024-03-06T07:29:14.021450+0000","last_active":"2024-03-06T07:29:15.016801+0000","last_peered":"2024-03-06T07:29:15.016801+0000","last_clean":"2024-03-06T07:29:15.016801+0000","last_became_active":"2024-03-06T07:29:14.021155+0000","last_became_peered":"2024-03-06T07:29:14.021155+0000","last_unstale":"2024-03-06T07:29:15.016801+0000","last_undegraded":"2024-03-06T07:29:15.016801+0000","last_fullsized":"2024-03-06T07:29:15.016801+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:57.323530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.040776+0000","last_change":"2024-03-06T07:29:15.036423+0000","last_active":"2024-03-06T07:29:15.040776+0000","last_peered":"2024-03-06T07:29:15.040776+0000","last_clean":"2024-03-06T07:29:15.040776+0000","last_became_active":"2024-03-06T07:29:15.036149+0000","last_became_peered":"2024-03-06T07:29:15.036149+0000","last_unstale":"2024-03-06T07:29:15.040776+0000","last_undegraded":"2024-03-06T07:29:15.040776+0000","last_fullsized":"2024-03-06T07:29:15.040776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:54.219263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045271+0000","last_change":"2024-03-06T07:29:14.032698+0000","last_active":"2024-03-06T07:29:15.045271+0000","last_peered":"2024-03-06T07:29:15.045271+0000","last_clean":"2024-03-06T07:29:15.045271+0000","last_became_active":"2024-03-06T07:29:14.032370+0000","last_became_peered":"2024-03-06T07:29:14.032370+0000","last_unstale":"2024-03-06T07:29:15.045271+0000","last_undegraded":"2024-03-06T07:29:15.045271+0000","last_fullsized":"2024-03-06T07:29:15.045271+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:00.738520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028534+0000","last_change":"2024-03-06T07:29:15.026389+0000","last_active":"2024-03-06T07:29:15.028534+0000","last_peered":"2024-03-06T07:29:15.028534+0000","last_clean":"2024-03-06T07:29:15.028534+0000","last_became_active":"2024-03-06T07:29:15.026089+0000","last_became_peered":"2024-03-06T07:29:15.026089+0000","last_unstale":"2024-03-06T07:29:15.028534+0000","last_undegraded":"2024-03-06T07:29:15.028534+0000","last_fullsized":"2024-03-06T07:29:15.028534+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:41.954664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.012295+0000","last_change":"2024-03-06T07:29:14.031021+0000","last_active":"2024-03-06T07:29:15.012295+0000","last_peered":"2024-03-06T07:29:15.012295+0000","last_clean":"2024-03-06T07:29:15.012295+0000","last_became_active":"2024-03-06T07:29:14.030732+0000","last_became_peered":"2024-03-06T07:29:14.030732+0000","last_unstale":"2024-03-06T07:29:15.012295+0000","last_undegraded":"2024-03-06T07:29:15.012295+0000","last_fullsized":"2024-03-06T07:29:15.012295+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:48.168788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.024694+0000","last_change":"2024-03-06T07:29:15.022946+0000","last_active":"2024-03-06T07:29:15.024694+0000","last_peered":"2024-03-06T07:29:15.024694+0000","last_clean":"2024-03-06T07:29:15.024694+0000","last_became_active":"2024-03-06T07:29:15.022673+0000","last_became_peered":"2024-03-06T07:29:15.022673+0000","last_unstale":"2024-03-06T07:29:15.024694+0000","last_undegraded":"2024-03-06T07:29:15.024694+0000","last_fullsized":"2024-03-06T07:29:15.024694+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:26.929627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.034646+0000","last_change":"2024-03-06T07:29:14.028405+0000","last_active":"2024-03-06T07:29:15.034646+0000","last_peered":"2024-03-06T07:29:15.034646+0000","last_clean":"2024-03-06T07:29:15.034646+0000","last_became_active":"2024-03-06T07:29:14.028115+0000","last_became_peered":"2024-03-06T07:29:14.028115+0000","last_unstale":"2024-03-06T07:29:15.034646+0000","last_undegraded":"2024-03-06T07:29:15.034646+0000","last_fullsized":"2024-03-06T07:29:15.034646+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:15:42.412040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046733+0000","last_change":"2024-03-06T07:29:15.044137+0000","last_active":"2024-03-06T07:29:15.046733+0000","last_peered":"2024-03-06T07:29:15.046733+0000","last_clean":"2024-03-06T07:29:15.046733+0000","last_became_active":"2024-03-06T07:29:15.043875+0000","last_became_peered":"2024-03-06T07:29:15.043875+0000","last_unstale":"2024-03-06T07:29:15.046733+0000","last_undegraded":"2024-03-06T07:29:15.046733+0000","last_fullsized":"2024-03-06T07:29:15.046733+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:40:06.222903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017522+0000","last_change":"2024-03-06T07:29:14.018984+0000","last_active":"2024-03-06T07:29:15.017522+0000","last_peered":"2024-03-06T07:29:15.017522+0000","last_clean":"2024-03-06T07:29:15.017522+0000","last_became_active":"2024-03-06T07:29:14.018696+0000","last_became_peered":"2024-03-06T07:29:14.018696+0000","last_unstale":"2024-03-06T07:29:15.017522+0000","last_undegraded":"2024-03-06T07:29:15.017522+0000","last_fullsized":"2024-03-06T07:29:15.017522+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:24:47.511788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032920+0000","last_change":"2024-03-06T07:29:15.031145+0000","last_active":"2024-03-06T07:29:15.032920+0000","last_peered":"2024-03-06T07:29:15.032920+0000","last_clean":"2024-03-06T07:29:15.032920+0000","last_became_active":"2024-03-06T07:29:15.030371+0000","last_became_peered":"2024-03-06T07:29:15.030371+0000","last_unstale":"2024-03-06T07:29:15.032920+0000","last_undegraded":"2024-03-06T07:29:15.032920+0000","last_fullsized":"2024-03-06T07:29:15.032920+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:32.007601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044633+0000","last_change":"2024-03-06T07:29:14.020479+0000","last_active":"2024-03-06T07:29:15.044633+0000","last_peered":"2024-03-06T07:29:15.044633+0000","last_clean":"2024-03-06T07:29:15.044633+0000","last_became_active":"2024-03-06T07:29:14.020183+0000","last_became_peered":"2024-03-06T07:29:14.020183+0000","last_unstale":"2024-03-06T07:29:15.044633+0000","last_undegraded":"2024-03-06T07:29:15.044633+0000","last_fullsized":"2024-03-06T07:29:15.044633+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:22:21.722072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.029615+0000","last_change":"2024-03-06T07:29:15.025585+0000","last_active":"2024-03-06T07:29:15.029615+0000","last_peered":"2024-03-06T07:29:15.029615+0000","last_clean":"2024-03-06T07:29:15.029615+0000","last_became_active":"2024-03-06T07:29:15.025211+0000","last_became_peered":"2024-03-06T07:29:15.025211+0000","last_unstale":"2024-03-06T07:29:15.029615+0000","last_undegraded":"2024-03-06T07:29:15.029615+0000","last_fullsized":"2024-03-06T07:29:15.029615+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:52.558827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016800+0000","last_change":"2024-03-06T07:29:14.026232+0000","last_active":"2024-03-06T07:29:15.016800+0000","last_peered":"2024-03-06T07:29:15.016800+0000","last_clean":"2024-03-06T07:29:15.016800+0000","last_became_active":"2024-03-06T07:29:14.025973+0000","last_became_peered":"2024-03-06T07:29:14.025973+0000","last_unstale":"2024-03-06T07:29:15.016800+0000","last_undegraded":"2024-03-06T07:29:15.016800+0000","last_fullsized":"2024-03-06T07:29:15.016800+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:04:23.496501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.022894+0000","last_change":"2024-03-06T07:29:15.021329+0000","last_active":"2024-03-06T07:29:15.022894+0000","last_peered":"2024-03-06T07:29:15.022894+0000","last_clean":"2024-03-06T07:29:15.022894+0000","last_became_active":"2024-03-06T07:29:15.021026+0000","last_became_peered":"2024-03-06T07:29:15.021026+0000","last_unstale":"2024-03-06T07:29:15.022894+0000","last_undegraded":"2024-03-06T07:29:15.022894+0000","last_fullsized":"2024-03-06T07:29:15.022894+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:24.796604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017937+0000","last_change":"2024-03-06T07:29:14.025589+0000","last_active":"2024-03-06T07:29:15.017937+0000","last_peered":"2024-03-06T07:29:15.017937+0000","last_clean":"2024-03-06T07:29:15.017937+0000","last_became_active":"2024-03-06T07:29:14.025310+0000","last_became_peered":"2024-03-06T07:29:14.025310+0000","last_unstale":"2024-03-06T07:29:15.017937+0000","last_undegraded":"2024-03-06T07:29:15.017937+0000","last_fullsized":"2024-03-06T07:29:15.017937+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:39.566571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031850+0000","last_change":"2024-03-06T07:29:15.030510+0000","last_active":"2024-03-06T07:29:15.031850+0000","last_peered":"2024-03-06T07:29:15.031850+0000","last_clean":"2024-03-06T07:29:15.031850+0000","last_became_active":"2024-03-06T07:29:15.030081+0000","last_became_peered":"2024-03-06T07:29:15.030081+0000","last_unstale":"2024-03-06T07:29:15.031850+0000","last_undegraded":"2024-03-06T07:29:15.031850+0000","last_fullsized":"2024-03-06T07:29:15.031850+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:49.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016737+0000","last_change":"2024-03-06T07:29:14.027773+0000","last_active":"2024-03-06T07:29:15.016737+0000","last_peered":"2024-03-06T07:29:15.016737+0000","last_clean":"2024-03-06T07:29:15.016737+0000","last_became_active":"2024-03-06T07:29:14.027437+0000","last_became_peered":"2024-03-06T07:29:14.027437+0000","last_unstale":"2024-03-06T07:29:15.016737+0000","last_undegraded":"2024-03-06T07:29:15.016737+0000","last_fullsized":"2024-03-06T07:29:15.016737+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:17:25.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.022165+0000","last_change":"2024-03-06T07:29:15.021567+0000","last_active":"2024-03-06T07:29:15.022165+0000","last_peered":"2024-03-06T07:29:15.022165+0000","last_clean":"2024-03-06T07:29:15.022165+0000","last_became_active":"2024-03-06T07:29:15.021293+0000","last_became_peered":"2024-03-06T07:29:15.021293+0000","last_unstale":"2024-03-06T07:29:15.022165+0000","last_undegraded":"2024-03-06T07:29:15.022165+0000","last_fullsized":"2024-03-06T07:29:15.022165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:41:45.090477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016131+0000","last_change":"2024-03-06T07:29:14.022154+0000","last_active":"2024-03-06T07:29:15.016131+0000","last_peered":"2024-03-06T07:29:15.016131+0000","last_clean":"2024-03-06T07:29:15.016131+0000","last_became_active":"2024-03-06T07:29:14.021888+0000","last_became_peered":"2024-03-06T07:29:14.021888+0000","last_unstale":"2024-03-06T07:29:15.016131+0000","last_undegraded":"2024-03-06T07:29:15.016131+0000","last_fullsized":"2024-03-06T07:29:15.016131+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:27.800819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045890+0000","last_change":"2024-03-06T07:29:15.042221+0000","last_active":"2024-03-06T07:29:15.045890+0000","last_peered":"2024-03-06T07:29:15.045890+0000","last_clean":"2024-03-06T07:29:15.045890+0000","last_became_active":"2024-03-06T07:29:15.041660+0000","last_became_peered":"2024-03-06T07:29:15.041660+0000","last_unstale":"2024-03-06T07:29:15.045890+0000","last_undegraded":"2024-03-06T07:29:15.045890+0000","last_fullsized":"2024-03-06T07:29:15.045890+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:51.574399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.015949+0000","last_change":"2024-03-06T07:29:14.016086+0000","last_active":"2024-03-06T07:29:15.015949+0000","last_peered":"2024-03-06T07:29:15.015949+0000","last_clean":"2024-03-06T07:29:15.015949+0000","last_became_active":"2024-03-06T07:29:14.015807+0000","last_became_peered":"2024-03-06T07:29:14.015807+0000","last_unstale":"2024-03-06T07:29:15.015949+0000","last_undegraded":"2024-03-06T07:29:15.015949+0000","last_fullsized":"2024-03-06T07:29:15.015949+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:39.132709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045866+0000","last_change":"2024-03-06T07:29:15.043046+0000","last_active":"2024-03-06T07:29:15.045866+0000","last_peered":"2024-03-06T07:29:15.045866+0000","last_clean":"2024-03-06T07:29:15.045866+0000","last_became_active":"2024-03-06T07:29:15.041341+0000","last_became_peered":"2024-03-06T07:29:15.041341+0000","last_unstale":"2024-03-06T07:29:15.045866+0000","last_undegraded":"2024-03-06T07:29:15.045866+0000","last_fullsized":"2024-03-06T07:29:15.045866+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:10.568070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017277+0000","last_change":"2024-03-06T07:29:14.025780+0000","last_active":"2024-03-06T07:29:15.017277+0000","last_peered":"2024-03-06T07:29:15.017277+0000","last_clean":"2024-03-06T07:29:15.017277+0000","last_became_active":"2024-03-06T07:29:14.025537+0000","last_became_peered":"2024-03-06T07:29:14.025537+0000","last_unstale":"2024-03-06T07:29:15.017277+0000","last_undegraded":"2024-03-06T07:29:15.017277+0000","last_fullsized":"2024-03-06T07:29:15.017277+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:24.404129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033171+0000","last_change":"2024-03-06T07:29:15.032208+0000","last_active":"2024-03-06T07:29:15.033171+0000","last_peered":"2024-03-06T07:29:15.033171+0000","last_clean":"2024-03-06T07:29:15.033171+0000","last_became_active":"2024-03-06T07:29:15.031914+0000","last_became_peered":"2024-03-06T07:29:15.031914+0000","last_unstale":"2024-03-06T07:29:15.033171+0000","last_undegraded":"2024-03-06T07:29:15.033171+0000","last_fullsized":"2024-03-06T07:29:15.033171+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:31.843059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.015573+0000","last_change":"2024-03-06T07:29:14.026890+0000","last_active":"2024-03-06T07:29:15.015573+0000","last_peered":"2024-03-06T07:29:15.015573+0000","last_clean":"2024-03-06T07:29:15.015573+0000","last_became_active":"2024-03-06T07:29:14.026583+0000","last_became_peered":"2024-03-06T07:29:14.026583+0000","last_unstale":"2024-03-06T07:29:15.015573+0000","last_undegraded":"2024-03-06T07:29:15.015573+0000","last_fullsized":"2024-03-06T07:29:15.015573+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:46.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044863+0000","last_change":"2024-03-06T07:29:15.037476+0000","last_active":"2024-03-06T07:29:15.044863+0000","last_peered":"2024-03-06T07:29:15.044863+0000","last_clean":"2024-03-06T07:29:15.044863+0000","last_became_active":"2024-03-06T07:29:15.036305+0000","last_became_peered":"2024-03-06T07:29:15.036305+0000","last_unstale":"2024-03-06T07:29:15.044863+0000","last_undegraded":"2024-03-06T07:29:15.044863+0000","last_fullsized":"2024-03-06T07:29:15.044863+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:07:19.230153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":524,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016541+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:29:15.016541+0000","last_peered":"2024-03-06T07:29:15.016541+0000","last_clean":"2024-03-06T07:29:15.016541+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:29:15.016541+0000","last_undegraded":"2024-03-06T07:29:15.016541+0000","last_fullsized":"2024-03-06T07:29:15.016541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33840,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338000,"statfs":{"total":96636764160,"available":96602112000,"internally_reserved":0,"allocated":2719744,"data_stored":1876838,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33772,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338068,"statfs":{"total":96636764160,"available":96602181632,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33772,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338068,"statfs":{"total":96636764160,"available":96602181632,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33772,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338068,"statfs":{"total":96636764160,"available":96602181632,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37940,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333900,"statfs":{"total":96636764160,"available":96597913600,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33776,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338064,"statfs":{"total":96636764160,"available":96602177536,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574962,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2719744,"data_stored":1876838,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:29:18.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:29:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.218+0000 7f2d5357a640 1 Processor -- start 2024-03-06T07:29:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.218+0000 7f2d5357a640 1 -- start start 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d5357a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d4c156ea0 0x7f2d4c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d5357a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0ad310 con 0x7f2d4c1506b0 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d5357a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c1572a0 con 0x7f2d4c153990 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d5357a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c156450 con 0x7f2d4c156ea0 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d51b17640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d4c156ea0 0x7f2d4c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d51316640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2d4c1506b0 0x7f2d4c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39130/0 (socket says 172.21.15.26:39130) 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d51316640 1 -- 172.21.15.26:0/1420150109 learned_addr learned my addr 172.21.15.26:0/1420150109 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d51b17640 1 -- 172.21.15.26:0/1420150109 >> v1:172.21.15.117:6789/0 conn(0x7f2d4c153990 legacy=0x7f2d4c0acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d51b17640 1 -- 172.21.15.26:0/1420150109 >> v1:172.21.15.26:6789/0 conn(0x7f2d4c1506b0 legacy=0x7f2d4c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d43fff640 1 -- 172.21.15.26:0/1420150109 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141940491 0 0) 0x7f2d4c0ad310 con 0x7f2d4c1506b0 2024-03-06T07:29:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.219+0000 7f2d51b17640 1 -- 172.21.15.26:0/1420150109 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c070aa0 con 0x7f2d4c156ea0 2024-03-06T07:29:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.220+0000 7f2d51b17640 1 --2- 172.21.15.26:0/1420150109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c156ea0 0x7f2d4c0c54a0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009d90 tx=0x7f2d4402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b16dc8127726fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.220+0000 7f2d43fff640 1 -- 172.21.15.26:0/1420150109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44039070 con 0x7f2d4c156ea0 2024-03-06T07:29:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.220+0000 7f2d43fff640 1 -- 172.21.15.26:0/1420150109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4402cc60 con 0x7f2d4c156ea0 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.220+0000 7f2d43fff640 1 -- 172.21.15.26:0/1420150109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44034660 con 0x7f2d4c156ea0 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.221+0000 7f2d5357a640 1 -- 172.21.15.26:0/1420150109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c156ea0 msgr2=0x7f2d4c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.221+0000 7f2d5357a640 1 --2- 172.21.15.26:0/1420150109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c156ea0 0x7f2d4c0c54a0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009d90 tx=0x7f2d4402beb0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.221+0000 7f2d5357a640 1 -- 172.21.15.26:0/1420150109 shutdown_connections 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.221+0000 7f2d5357a640 1 --2- 172.21.15.26:0/1420150109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c156ea0 0x7f2d4c0c54a0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.221+0000 7f2d5357a640 1 -- 172.21.15.26:0/1420150109 >> 172.21.15.26:0/1420150109 conn(0x7f2d4c06b1a0 msgr2=0x7f2d4c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.221+0000 7f2d5357a640 1 -- 172.21.15.26:0/1420150109 shutdown_connections 2024-03-06T07:29:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.221+0000 7f2d5357a640 1 -- 172.21.15.26:0/1420150109 wait complete. 2024-03-06T07:29:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.222+0000 7f2d5357a640 1 Processor -- start 2024-03-06T07:29:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.223+0000 7f2d5357a640 1 -- start start 2024-03-06T07:29:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d5357a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c1506b0 0x7f2d4c1ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d5357a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d4c153990 0x7f2d4c152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d5357a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d4c156ea0 0x7f2d4c1e9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d5357a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c1566a0 con 0x7f2d4c1506b0 2024-03-06T07:29:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d5357a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c156a10 con 0x7f2d4c153990 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d5357a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c59e0 con 0x7f2d4c156ea0 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c1506b0 0x7f2d4c1ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c1506b0 0x7f2d4c1ebc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37452/0 (socket says 172.21.15.26:37452) 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 -- 172.21.15.26:0/2930451928 learned_addr learned my addr 172.21.15.26:0/2930451928 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d50b15640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d4c153990 0x7f2d4c152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51b17640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d4c156ea0 0x7f2d4c1e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 -- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d4c156ea0 msgr2=0x7f2d4c1e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d4c156ea0 0x7f2d4c1e9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 -- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d4c153990 msgr2=0x7f2d4c152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d4c153990 0x7f2d4c152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 -- 172.21.15.26:0/2930451928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c149a60 con 0x7f2d4c1506b0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51b17640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d4c156ea0 0x7f2d4c1e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d51316640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c1506b0 0x7f2d4c1ebc80 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c4d0 tx=0x7f2d3c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d41ffb640 1 -- 172.21.15.26:0/2930451928 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c018070 con 0x7f2d4c1506b0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d41ffb640 1 -- 172.21.15.26:0/2930451928 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c007500 con 0x7f2d4c1506b0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c148f10 con 0x7f2d4c1506b0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.224+0000 7f2d41ffb640 1 -- 172.21.15.26:0/2930451928 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004a20 con 0x7f2d4c1506b0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.225+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c1494d0 con 0x7f2d4c1506b0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.225+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c04f330 con 0x7f2d4c1506b0 2024-03-06T07:29:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.226+0000 7f2d41ffb640 1 -- 172.21.15.26:0/2930451928 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d3c01b070 con 0x7f2d4c1506b0 2024-03-06T07:29:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.226+0000 7f2d41ffb640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d3006b4d0 0x7f2d3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.226+0000 7f2d41ffb640 1 -- 172.21.15.26:0/2930451928 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d3c089660 con 0x7f2d4c1506b0 2024-03-06T07:29:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.226+0000 7f2d50b15640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d3006b4d0 0x7f2d3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.227+0000 7f2d50b15640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d3006b4d0 0x7f2d3006d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0c2930 tx=0x7f2d48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.235+0000 7f2d41ffb640 1 -- 172.21.15.26:0/2930451928 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d4c149a60 con 0x7f2d4c1506b0 2024-03-06T07:29:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.413+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d4c0700a0 con 0x7f2d3006b4d0 2024-03-06T07:29:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.419+0000 7f2d41ffb640 1 -- 172.21.15.26:0/2930451928 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252499 (secure 0 0 0) 0x7f2d4c0700a0 con 0x7f2d3006b4d0 2024-03-06T07:29:18.423 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:18.425 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:29:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.422+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d3006b4d0 msgr2=0x7f2d3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.422+0000 7f2d5357a640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d3006b4d0 0x7f2d3006d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0c2930 tx=0x7f2d48009040 comp rx=0 tx=0).stop 2024-03-06T07:29:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.422+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c1506b0 msgr2=0x7f2d4c1ebc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.422+0000 7f2d5357a640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c1506b0 0x7f2d4c1ebc80 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c4d0 tx=0x7f2d3c00c990 comp rx=0 tx=0).stop 2024-03-06T07:29:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 shutdown_connections 2024-03-06T07:29:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2d3006b4d0 0x7f2d3006d990 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d4c156ea0 0x7f2d4c1e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d4c153990 0x7f2d4c152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 --2- 172.21.15.26:0/2930451928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d4c1506b0 0x7f2d4c1ebc80 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 >> 172.21.15.26:0/2930451928 conn(0x7f2d4c06b1a0 msgr2=0x7f2d4c152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 shutdown_connections 2024-03-06T07:29:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.423+0000 7f2d5357a640 1 -- 172.21.15.26:0/2930451928 wait complete. 2024-03-06T07:29:18.435 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-03-06T07:29:17.150622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282804,"kb_used_data":20864,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754691916,"statfs":{"total":773094113280,"available":772804521984,"internally_reserved":0,"allocated":21364736,"data_stored":11472496,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.148190"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031259+0000","last_change":"2024-03-06T07:29:15.030359+0000","last_active":"2024-03-06T07:29:15.031259+0000","last_peered":"2024-03-06T07:29:15.031259+0000","last_clean":"2024-03-06T07:29:15.031259+0000","last_became_active":"2024-03-06T07:29:15.030107+0000","last_became_peered":"2024-03-06T07:29:15.030107+0000","last_unstale":"2024-03-06T07:29:15.031259+0000","last_undegraded":"2024-03-06T07:29:15.031259+0000","last_fullsized":"2024-03-06T07:29:15.031259+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:56.810673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046685+0000","last_change":"2024-03-06T07:29:15.044690+0000","last_active":"2024-03-06T07:29:15.046685+0000","last_peered":"2024-03-06T07:29:15.046685+0000","last_clean":"2024-03-06T07:29:15.046685+0000","last_became_active":"2024-03-06T07:29:15.044382+0000","last_became_peered":"2024-03-06T07:29:15.044382+0000","last_unstale":"2024-03-06T07:29:15.046685+0000","last_undegraded":"2024-03-06T07:29:15.046685+0000","last_fullsized":"2024-03-06T07:29:15.046685+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:16.615660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.041892+0000","last_change":"2024-03-06T07:29:15.041290+0000","last_active":"2024-03-06T07:29:15.041892+0000","last_peered":"2024-03-06T07:29:15.041892+0000","last_clean":"2024-03-06T07:29:15.041892+0000","last_became_active":"2024-03-06T07:29:15.041026+0000","last_became_peered":"2024-03-06T07:29:15.041026+0000","last_unstale":"2024-03-06T07:29:15.041892+0000","last_undegraded":"2024-03-06T07:29:15.041892+0000","last_fullsized":"2024-03-06T07:29:15.041892+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:56:00.693221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031182+0000","last_change":"2024-03-06T07:29:15.029112+0000","last_active":"2024-03-06T07:29:15.031182+0000","last_peered":"2024-03-06T07:29:15.031182+0000","last_clean":"2024-03-06T07:29:15.031182+0000","last_became_active":"2024-03-06T07:29:15.028838+0000","last_became_peered":"2024-03-06T07:29:15.028838+0000","last_unstale":"2024-03-06T07:29:15.031182+0000","last_undegraded":"2024-03-06T07:29:15.031182+0000","last_fullsized":"2024-03-06T07:29:15.031182+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:55.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026239+0000","last_change":"2024-03-06T07:29:15.024030+0000","last_active":"2024-03-06T07:29:15.026239+0000","last_peered":"2024-03-06T07:29:15.026239+0000","last_clean":"2024-03-06T07:29:15.026239+0000","last_became_active":"2024-03-06T07:29:15.023518+0000","last_became_peered":"2024-03-06T07:29:15.023518+0000","last_unstale":"2024-03-06T07:29:15.026239+0000","last_undegraded":"2024-03-06T07:29:15.026239+0000","last_fullsized":"2024-03-06T07:29:15.026239+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:07:20.498162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030277+0000","last_change":"2024-03-06T07:29:15.028312+0000","last_active":"2024-03-06T07:29:15.030277+0000","last_peered":"2024-03-06T07:29:15.030277+0000","last_clean":"2024-03-06T07:29:15.030277+0000","last_became_active":"2024-03-06T07:29:15.028015+0000","last_became_peered":"2024-03-06T07:29:15.028015+0000","last_unstale":"2024-03-06T07:29:15.030277+0000","last_undegraded":"2024-03-06T07:29:15.030277+0000","last_fullsized":"2024-03-06T07:29:15.030277+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:52:14.164253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.023929+0000","last_change":"2024-03-06T07:29:15.023058+0000","last_active":"2024-03-06T07:29:15.023929+0000","last_peered":"2024-03-06T07:29:15.023929+0000","last_clean":"2024-03-06T07:29:15.023929+0000","last_became_active":"2024-03-06T07:29:15.022728+0000","last_became_peered":"2024-03-06T07:29:15.022728+0000","last_unstale":"2024-03-06T07:29:15.023929+0000","last_undegraded":"2024-03-06T07:29:15.023929+0000","last_fullsized":"2024-03-06T07:29:15.023929+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:44:58.382436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042941+0000","last_change":"2024-03-06T07:29:15.027529+0000","last_active":"2024-03-06T07:29:15.042941+0000","last_peered":"2024-03-06T07:29:15.042941+0000","last_clean":"2024-03-06T07:29:15.042941+0000","last_became_active":"2024-03-06T07:29:15.027245+0000","last_became_peered":"2024-03-06T07:29:15.027245+0000","last_unstale":"2024-03-06T07:29:15.042941+0000","last_undegraded":"2024-03-06T07:29:15.042941+0000","last_fullsized":"2024-03-06T07:29:15.042941+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:53.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030261+0000","last_change":"2024-03-06T07:29:15.026744+0000","last_active":"2024-03-06T07:29:15.030261+0000","last_peered":"2024-03-06T07:29:15.030261+0000","last_clean":"2024-03-06T07:29:15.030261+0000","last_became_active":"2024-03-06T07:29:15.026477+0000","last_became_peered":"2024-03-06T07:29:15.026477+0000","last_unstale":"2024-03-06T07:29:15.030261+0000","last_undegraded":"2024-03-06T07:29:15.030261+0000","last_fullsized":"2024-03-06T07:29:15.030261+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:22.613234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031356+0000","last_change":"2024-03-06T07:29:15.030879+0000","last_active":"2024-03-06T07:29:15.031356+0000","last_peered":"2024-03-06T07:29:15.031356+0000","last_clean":"2024-03-06T07:29:15.031356+0000","last_became_active":"2024-03-06T07:29:15.030605+0000","last_became_peered":"2024-03-06T07:29:15.030605+0000","last_unstale":"2024-03-06T07:29:15.031356+0000","last_undegraded":"2024-03-06T07:29:15.031356+0000","last_fullsized":"2024-03-06T07:29:15.031356+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:41:37.685097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.027020+0000","last_change":"2024-03-06T07:29:15.023196+0000","last_active":"2024-03-06T07:29:15.027020+0000","last_peered":"2024-03-06T07:29:15.027020+0000","last_clean":"2024-03-06T07:29:15.027020+0000","last_became_active":"2024-03-06T07:29:15.022906+0000","last_became_peered":"2024-03-06T07:29:15.022906+0000","last_unstale":"2024-03-06T07:29:15.027020+0000","last_undegraded":"2024-03-06T07:29:15.027020+0000","last_fullsized":"2024-03-06T07:29:15.027020+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:50.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.029548+0000","last_change":"2024-03-06T07:29:15.026121+0000","last_active":"2024-03-06T07:29:15.029548+0000","last_peered":"2024-03-06T07:29:15.029548+0000","last_clean":"2024-03-06T07:29:15.029548+0000","last_became_active":"2024-03-06T07:29:15.025864+0000","last_became_peered":"2024-03-06T07:29:15.025864+0000","last_unstale":"2024-03-06T07:29:15.029548+0000","last_undegraded":"2024-03-06T07:29:15.029548+0000","last_fullsized":"2024-03-06T07:29:15.029548+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:29.989262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044764+0000","last_change":"2024-03-06T07:29:15.036581+0000","last_active":"2024-03-06T07:29:15.044764+0000","last_peered":"2024-03-06T07:29:15.044764+0000","last_clean":"2024-03-06T07:29:15.044764+0000","last_became_active":"2024-03-06T07:29:15.031104+0000","last_became_peered":"2024-03-06T07:29:15.031104+0000","last_unstale":"2024-03-06T07:29:15.044764+0000","last_undegraded":"2024-03-06T07:29:15.044764+0000","last_fullsized":"2024-03-06T07:29:15.044764+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:09:13.391272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045843+0000","last_change":"2024-03-06T07:29:15.042749+0000","last_active":"2024-03-06T07:29:15.045843+0000","last_peered":"2024-03-06T07:29:15.045843+0000","last_clean":"2024-03-06T07:29:15.045843+0000","last_became_active":"2024-03-06T07:29:15.042484+0000","last_became_peered":"2024-03-06T07:29:15.042484+0000","last_unstale":"2024-03-06T07:29:15.045843+0000","last_undegraded":"2024-03-06T07:29:15.045843+0000","last_fullsized":"2024-03-06T07:29:15.045843+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:38:42.566990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044937+0000","last_change":"2024-03-06T07:29:15.041280+0000","last_active":"2024-03-06T07:29:15.044937+0000","last_peered":"2024-03-06T07:29:15.044937+0000","last_clean":"2024-03-06T07:29:15.044937+0000","last_became_active":"2024-03-06T07:29:15.040956+0000","last_became_peered":"2024-03-06T07:29:15.040956+0000","last_unstale":"2024-03-06T07:29:15.044937+0000","last_undegraded":"2024-03-06T07:29:15.044937+0000","last_fullsized":"2024-03-06T07:29:15.044937+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:04.880888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038790+0000","last_change":"2024-03-06T07:29:15.037372+0000","last_active":"2024-03-06T07:29:15.038790+0000","last_peered":"2024-03-06T07:29:15.038790+0000","last_clean":"2024-03-06T07:29:15.038790+0000","last_became_active":"2024-03-06T07:29:15.037082+0000","last_became_peered":"2024-03-06T07:29:15.037082+0000","last_unstale":"2024-03-06T07:29:15.038790+0000","last_undegraded":"2024-03-06T07:29:15.038790+0000","last_fullsized":"2024-03-06T07:29:15.038790+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:17:26.569423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033184+0000","last_change":"2024-03-06T07:29:15.032203+0000","last_active":"2024-03-06T07:29:15.033184+0000","last_peered":"2024-03-06T07:29:15.033184+0000","last_clean":"2024-03-06T07:29:15.033184+0000","last_became_active":"2024-03-06T07:29:15.031852+0000","last_became_peered":"2024-03-06T07:29:15.031852+0000","last_unstale":"2024-03-06T07:29:15.033184+0000","last_undegraded":"2024-03-06T07:29:15.033184+0000","last_fullsized":"2024-03-06T07:29:15.033184+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:44.233028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039915+0000","last_change":"2024-03-06T07:29:15.029839+0000","last_active":"2024-03-06T07:29:15.039915+0000","last_peered":"2024-03-06T07:29:15.039915+0000","last_clean":"2024-03-06T07:29:15.039915+0000","last_became_active":"2024-03-06T07:29:15.029541+0000","last_became_peered":"2024-03-06T07:29:15.029541+0000","last_unstale":"2024-03-06T07:29:15.039915+0000","last_undegraded":"2024-03-06T07:29:15.039915+0000","last_fullsized":"2024-03-06T07:29:15.039915+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:27.084764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026437+0000","last_change":"2024-03-06T07:29:15.023928+0000","last_active":"2024-03-06T07:29:15.026437+0000","last_peered":"2024-03-06T07:29:15.026437+0000","last_clean":"2024-03-06T07:29:15.026437+0000","last_became_active":"2024-03-06T07:29:15.023662+0000","last_became_peered":"2024-03-06T07:29:15.023662+0000","last_unstale":"2024-03-06T07:29:15.026437+0000","last_undegraded":"2024-03-06T07:29:15.026437+0000","last_fullsized":"2024-03-06T07:29:15.026437+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:00:58.174701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032940+0000","last_change":"2024-03-06T07:29:15.031416+0000","last_active":"2024-03-06T07:29:15.032940+0000","last_peered":"2024-03-06T07:29:15.032940+0000","last_clean":"2024-03-06T07:29:15.032940+0000","last_became_active":"2024-03-06T07:29:15.030352+0000","last_became_peered":"2024-03-06T07:29:15.030352+0000","last_unstale":"2024-03-06T07:29:15.032940+0000","last_undegraded":"2024-03-06T07:29:15.032940+0000","last_fullsized":"2024-03-06T07:29:15.032940+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:53.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026216+0000","last_change":"2024-03-06T07:29:15.023795+0000","last_active":"2024-03-06T07:29:15.026216+0000","last_peered":"2024-03-06T07:29:15.026216+0000","last_clean":"2024-03-06T07:29:15.026216+0000","last_became_active":"2024-03-06T07:29:15.023151+0000","last_became_peered":"2024-03-06T07:29:15.023151+0000","last_unstale":"2024-03-06T07:29:15.026216+0000","last_undegraded":"2024-03-06T07:29:15.026216+0000","last_fullsized":"2024-03-06T07:29:15.026216+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:15:43.422349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033599+0000","last_change":"2024-03-06T07:29:15.032126+0000","last_active":"2024-03-06T07:29:15.033599+0000","last_peered":"2024-03-06T07:29:15.033599+0000","last_clean":"2024-03-06T07:29:15.033599+0000","last_became_active":"2024-03-06T07:29:15.031854+0000","last_became_peered":"2024-03-06T07:29:15.031854+0000","last_unstale":"2024-03-06T07:29:15.033599+0000","last_undegraded":"2024-03-06T07:29:15.033599+0000","last_fullsized":"2024-03-06T07:29:15.033599+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:17:13.696237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045866+0000","last_change":"2024-03-06T07:29:15.041977+0000","last_active":"2024-03-06T07:29:15.045866+0000","last_peered":"2024-03-06T07:29:15.045866+0000","last_clean":"2024-03-06T07:29:15.045866+0000","last_became_active":"2024-03-06T07:29:15.041219+0000","last_became_peered":"2024-03-06T07:29:15.041219+0000","last_unstale":"2024-03-06T07:29:15.045866+0000","last_undegraded":"2024-03-06T07:29:15.045866+0000","last_fullsized":"2024-03-06T07:29:15.045866+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:41.684370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.022830+0000","last_change":"2024-03-06T07:29:15.022028+0000","last_active":"2024-03-06T07:29:15.022830+0000","last_peered":"2024-03-06T07:29:15.022830+0000","last_clean":"2024-03-06T07:29:15.022830+0000","last_became_active":"2024-03-06T07:29:15.021744+0000","last_became_peered":"2024-03-06T07:29:15.021744+0000","last_unstale":"2024-03-06T07:29:15.022830+0000","last_undegraded":"2024-03-06T07:29:15.022830+0000","last_fullsized":"2024-03-06T07:29:15.022830+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:16.310455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032500+0000","last_change":"2024-03-06T07:29:15.028134+0000","last_active":"2024-03-06T07:29:15.032500+0000","last_peered":"2024-03-06T07:29:15.032500+0000","last_clean":"2024-03-06T07:29:15.032500+0000","last_became_active":"2024-03-06T07:29:15.027846+0000","last_became_peered":"2024-03-06T07:29:15.027846+0000","last_unstale":"2024-03-06T07:29:15.032500+0000","last_undegraded":"2024-03-06T07:29:15.032500+0000","last_fullsized":"2024-03-06T07:29:15.032500+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:52:54.591638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.040756+0000","last_change":"2024-03-06T07:29:15.034571+0000","last_active":"2024-03-06T07:29:15.040756+0000","last_peered":"2024-03-06T07:29:15.040756+0000","last_clean":"2024-03-06T07:29:15.040756+0000","last_became_active":"2024-03-06T07:29:15.034277+0000","last_became_peered":"2024-03-06T07:29:15.034277+0000","last_unstale":"2024-03-06T07:29:15.040756+0000","last_undegraded":"2024-03-06T07:29:15.040756+0000","last_fullsized":"2024-03-06T07:29:15.040756+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:01.748829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033965+0000","last_change":"2024-03-06T07:29:15.033322+0000","last_active":"2024-03-06T07:29:15.033965+0000","last_peered":"2024-03-06T07:29:15.033965+0000","last_clean":"2024-03-06T07:29:15.033965+0000","last_became_active":"2024-03-06T07:29:15.032742+0000","last_became_peered":"2024-03-06T07:29:15.032742+0000","last_unstale":"2024-03-06T07:29:15.033965+0000","last_undegraded":"2024-03-06T07:29:15.033965+0000","last_fullsized":"2024-03-06T07:29:15.033965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:44:04.457241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031233+0000","last_change":"2024-03-06T07:29:15.029682+0000","last_active":"2024-03-06T07:29:15.031233+0000","last_peered":"2024-03-06T07:29:15.031233+0000","last_clean":"2024-03-06T07:29:15.031233+0000","last_became_active":"2024-03-06T07:29:15.029422+0000","last_became_peered":"2024-03-06T07:29:15.029422+0000","last_unstale":"2024-03-06T07:29:15.031233+0000","last_undegraded":"2024-03-06T07:29:15.031233+0000","last_fullsized":"2024-03-06T07:29:15.031233+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:26:15.661395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030255+0000","last_change":"2024-03-06T07:29:15.027659+0000","last_active":"2024-03-06T07:29:15.030255+0000","last_peered":"2024-03-06T07:29:15.030255+0000","last_clean":"2024-03-06T07:29:15.030255+0000","last_became_active":"2024-03-06T07:29:15.027385+0000","last_became_peered":"2024-03-06T07:29:15.027385+0000","last_unstale":"2024-03-06T07:29:15.030255+0000","last_undegraded":"2024-03-06T07:29:15.030255+0000","last_fullsized":"2024-03-06T07:29:15.030255+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:39:25.493710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.048014+0000","last_change":"2024-03-06T07:29:15.047343+0000","last_active":"2024-03-06T07:29:15.048014+0000","last_peered":"2024-03-06T07:29:15.048014+0000","last_clean":"2024-03-06T07:29:15.048014+0000","last_became_active":"2024-03-06T07:29:15.047045+0000","last_became_peered":"2024-03-06T07:29:15.047045+0000","last_unstale":"2024-03-06T07:29:15.048014+0000","last_undegraded":"2024-03-06T07:29:15.048014+0000","last_fullsized":"2024-03-06T07:29:15.048014+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:24.336860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044726+0000","last_change":"2024-03-06T07:29:15.041521+0000","last_active":"2024-03-06T07:29:15.044726+0000","last_peered":"2024-03-06T07:29:15.044726+0000","last_clean":"2024-03-06T07:29:15.044726+0000","last_became_active":"2024-03-06T07:29:15.041009+0000","last_became_peered":"2024-03-06T07:29:15.041009+0000","last_unstale":"2024-03-06T07:29:15.044726+0000","last_undegraded":"2024-03-06T07:29:15.044726+0000","last_fullsized":"2024-03-06T07:29:15.044726+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:25.185955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028340+0000","last_change":"2024-03-06T07:29:15.025968+0000","last_active":"2024-03-06T07:29:15.028340+0000","last_peered":"2024-03-06T07:29:15.028340+0000","last_clean":"2024-03-06T07:29:15.028340+0000","last_became_active":"2024-03-06T07:29:15.025691+0000","last_became_peered":"2024-03-06T07:29:15.025691+0000","last_unstale":"2024-03-06T07:29:15.028340+0000","last_undegraded":"2024-03-06T07:29:15.028340+0000","last_fullsized":"2024-03-06T07:29:15.028340+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:15.074091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"210'4","reported_seq":16,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045627+0000","last_change":"2024-03-06T07:29:14.025579+0000","last_active":"2024-03-06T07:29:15.045627+0000","last_peered":"2024-03-06T07:29:15.045627+0000","last_clean":"2024-03-06T07:29:15.045627+0000","last_became_active":"2024-03-06T07:29:14.025299+0000","last_became_peered":"2024-03-06T07:29:14.025299+0000","last_unstale":"2024-03-06T07:29:15.045627+0000","last_undegraded":"2024-03-06T07:29:15.045627+0000","last_fullsized":"2024-03-06T07:29:15.045627+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:01.785227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039852+0000","last_change":"2024-03-06T07:29:15.027795+0000","last_active":"2024-03-06T07:29:15.039852+0000","last_peered":"2024-03-06T07:29:15.039852+0000","last_clean":"2024-03-06T07:29:15.039852+0000","last_became_active":"2024-03-06T07:29:15.027503+0000","last_became_peered":"2024-03-06T07:29:15.027503+0000","last_unstale":"2024-03-06T07:29:15.039852+0000","last_undegraded":"2024-03-06T07:29:15.039852+0000","last_fullsized":"2024-03-06T07:29:15.039852+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:21.292222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.036436+0000","last_change":"2024-03-06T07:29:14.024503+0000","last_active":"2024-03-06T07:29:15.036436+0000","last_peered":"2024-03-06T07:29:15.036436+0000","last_clean":"2024-03-06T07:29:15.036436+0000","last_became_active":"2024-03-06T07:29:14.024197+0000","last_became_peered":"2024-03-06T07:29:14.024197+0000","last_unstale":"2024-03-06T07:29:15.036436+0000","last_undegraded":"2024-03-06T07:29:15.036436+0000","last_fullsized":"2024-03-06T07:29:15.036436+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:53.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026387+0000","last_change":"2024-03-06T07:29:15.025683+0000","last_active":"2024-03-06T07:29:15.026387+0000","last_peered":"2024-03-06T07:29:15.026387+0000","last_clean":"2024-03-06T07:29:15.026387+0000","last_became_active":"2024-03-06T07:29:15.021856+0000","last_became_peered":"2024-03-06T07:29:15.021856+0000","last_unstale":"2024-03-06T07:29:15.026387+0000","last_undegraded":"2024-03-06T07:29:15.026387+0000","last_fullsized":"2024-03-06T07:29:15.026387+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:58:25.688284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.041617+0000","last_change":"2024-03-06T07:29:14.026365+0000","last_active":"2024-03-06T07:29:15.041617+0000","last_peered":"2024-03-06T07:29:15.041617+0000","last_clean":"2024-03-06T07:29:15.041617+0000","last_became_active":"2024-03-06T07:29:14.026078+0000","last_became_peered":"2024-03-06T07:29:14.026078+0000","last_unstale":"2024-03-06T07:29:15.041617+0000","last_undegraded":"2024-03-06T07:29:15.041617+0000","last_fullsized":"2024-03-06T07:29:15.041617+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:27:49.657594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.040752+0000","last_change":"2024-03-06T07:29:15.035819+0000","last_active":"2024-03-06T07:29:15.040752+0000","last_peered":"2024-03-06T07:29:15.040752+0000","last_clean":"2024-03-06T07:29:15.040752+0000","last_became_active":"2024-03-06T07:29:15.035436+0000","last_became_peered":"2024-03-06T07:29:15.035436+0000","last_unstale":"2024-03-06T07:29:15.040752+0000","last_undegraded":"2024-03-06T07:29:15.040752+0000","last_fullsized":"2024-03-06T07:29:15.040752+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.594027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038458+0000","last_change":"2024-03-06T07:29:14.027623+0000","last_active":"2024-03-06T07:29:15.038458+0000","last_peered":"2024-03-06T07:29:15.038458+0000","last_clean":"2024-03-06T07:29:15.038458+0000","last_became_active":"2024-03-06T07:29:14.027352+0000","last_became_peered":"2024-03-06T07:29:14.027352+0000","last_unstale":"2024-03-06T07:29:15.038458+0000","last_undegraded":"2024-03-06T07:29:15.038458+0000","last_fullsized":"2024-03-06T07:29:15.038458+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:53:01.380578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039194+0000","last_change":"2024-03-06T07:29:15.032654+0000","last_active":"2024-03-06T07:29:15.039194+0000","last_peered":"2024-03-06T07:29:15.039194+0000","last_clean":"2024-03-06T07:29:15.039194+0000","last_became_active":"2024-03-06T07:29:15.032286+0000","last_became_peered":"2024-03-06T07:29:15.032286+0000","last_unstale":"2024-03-06T07:29:15.039194+0000","last_undegraded":"2024-03-06T07:29:15.039194+0000","last_fullsized":"2024-03-06T07:29:15.039194+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:53.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038537+0000","last_change":"2024-03-06T07:29:14.029242+0000","last_active":"2024-03-06T07:29:15.038537+0000","last_peered":"2024-03-06T07:29:15.038537+0000","last_clean":"2024-03-06T07:29:15.038537+0000","last_became_active":"2024-03-06T07:29:14.028975+0000","last_became_peered":"2024-03-06T07:29:14.028975+0000","last_unstale":"2024-03-06T07:29:15.038537+0000","last_undegraded":"2024-03-06T07:29:15.038537+0000","last_fullsized":"2024-03-06T07:29:15.038537+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:30.997292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042406+0000","last_change":"2024-03-06T07:29:15.035383+0000","last_active":"2024-03-06T07:29:15.042406+0000","last_peered":"2024-03-06T07:29:15.042406+0000","last_clean":"2024-03-06T07:29:15.042406+0000","last_became_active":"2024-03-06T07:29:15.034879+0000","last_became_peered":"2024-03-06T07:29:15.034879+0000","last_unstale":"2024-03-06T07:29:15.042406+0000","last_undegraded":"2024-03-06T07:29:15.042406+0000","last_fullsized":"2024-03-06T07:29:15.042406+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:07:20.498162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046675+0000","last_change":"2024-03-06T07:29:14.033003+0000","last_active":"2024-03-06T07:29:15.046675+0000","last_peered":"2024-03-06T07:29:15.046675+0000","last_clean":"2024-03-06T07:29:15.046675+0000","last_became_active":"2024-03-06T07:29:14.032291+0000","last_became_peered":"2024-03-06T07:29:14.032291+0000","last_unstale":"2024-03-06T07:29:15.046675+0000","last_undegraded":"2024-03-06T07:29:15.046675+0000","last_fullsized":"2024-03-06T07:29:15.046675+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:30.997292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038160+0000","last_change":"2024-03-06T07:29:15.037310+0000","last_active":"2024-03-06T07:29:15.038160+0000","last_peered":"2024-03-06T07:29:15.038160+0000","last_clean":"2024-03-06T07:29:15.038160+0000","last_became_active":"2024-03-06T07:29:15.036999+0000","last_became_peered":"2024-03-06T07:29:15.036999+0000","last_unstale":"2024-03-06T07:29:15.038160+0000","last_undegraded":"2024-03-06T07:29:15.038160+0000","last_fullsized":"2024-03-06T07:29:15.038160+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:45:01.464817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017153+0000","last_change":"2024-03-06T07:29:14.025905+0000","last_active":"2024-03-06T07:29:15.017153+0000","last_peered":"2024-03-06T07:29:15.017153+0000","last_clean":"2024-03-06T07:29:15.017153+0000","last_became_active":"2024-03-06T07:29:14.025619+0000","last_became_peered":"2024-03-06T07:29:14.025619+0000","last_unstale":"2024-03-06T07:29:15.017153+0000","last_undegraded":"2024-03-06T07:29:15.017153+0000","last_fullsized":"2024-03-06T07:29:15.017153+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:21.246584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044843+0000","last_change":"2024-03-06T07:29:15.031414+0000","last_active":"2024-03-06T07:29:15.044843+0000","last_peered":"2024-03-06T07:29:15.044843+0000","last_clean":"2024-03-06T07:29:15.044843+0000","last_became_active":"2024-03-06T07:29:15.029802+0000","last_became_peered":"2024-03-06T07:29:15.029802+0000","last_unstale":"2024-03-06T07:29:15.044843+0000","last_undegraded":"2024-03-06T07:29:15.044843+0000","last_fullsized":"2024-03-06T07:29:15.044843+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:35.463008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.011892+0000","last_change":"2024-03-06T07:29:14.028519+0000","last_active":"2024-03-06T07:29:15.011892+0000","last_peered":"2024-03-06T07:29:15.011892+0000","last_clean":"2024-03-06T07:29:15.011892+0000","last_became_active":"2024-03-06T07:29:14.027431+0000","last_became_peered":"2024-03-06T07:29:14.027431+0000","last_unstale":"2024-03-06T07:29:15.011892+0000","last_undegraded":"2024-03-06T07:29:15.011892+0000","last_fullsized":"2024-03-06T07:29:15.011892+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:22.677326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045056+0000","last_change":"2024-03-06T07:29:15.037780+0000","last_active":"2024-03-06T07:29:15.045056+0000","last_peered":"2024-03-06T07:29:15.045056+0000","last_clean":"2024-03-06T07:29:15.045056+0000","last_became_active":"2024-03-06T07:29:15.034710+0000","last_became_peered":"2024-03-06T07:29:15.034710+0000","last_unstale":"2024-03-06T07:29:15.045056+0000","last_undegraded":"2024-03-06T07:29:15.045056+0000","last_fullsized":"2024-03-06T07:29:15.045056+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:39:50.764109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017291+0000","last_change":"2024-03-06T07:29:14.029442+0000","last_active":"2024-03-06T07:29:15.017291+0000","last_peered":"2024-03-06T07:29:15.017291+0000","last_clean":"2024-03-06T07:29:15.017291+0000","last_became_active":"2024-03-06T07:29:14.029168+0000","last_became_peered":"2024-03-06T07:29:14.029168+0000","last_unstale":"2024-03-06T07:29:15.017291+0000","last_undegraded":"2024-03-06T07:29:15.017291+0000","last_fullsized":"2024-03-06T07:29:15.017291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:09:56.121417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039875+0000","last_change":"2024-03-06T07:29:15.029999+0000","last_active":"2024-03-06T07:29:15.039875+0000","last_peered":"2024-03-06T07:29:15.039875+0000","last_clean":"2024-03-06T07:29:15.039875+0000","last_became_active":"2024-03-06T07:29:15.029691+0000","last_became_peered":"2024-03-06T07:29:15.029691+0000","last_unstale":"2024-03-06T07:29:15.039875+0000","last_undegraded":"2024-03-06T07:29:15.039875+0000","last_fullsized":"2024-03-06T07:29:15.039875+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:07:58.543396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.019279+0000","last_change":"2024-03-06T07:29:14.025372+0000","last_active":"2024-03-06T07:29:15.019279+0000","last_peered":"2024-03-06T07:29:15.019279+0000","last_clean":"2024-03-06T07:29:15.019279+0000","last_became_active":"2024-03-06T07:29:14.025042+0000","last_became_peered":"2024-03-06T07:29:14.025042+0000","last_unstale":"2024-03-06T07:29:15.019279+0000","last_undegraded":"2024-03-06T07:29:15.019279+0000","last_fullsized":"2024-03-06T07:29:15.019279+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:45.924264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030092+0000","last_change":"2024-03-06T07:29:15.028930+0000","last_active":"2024-03-06T07:29:15.030092+0000","last_peered":"2024-03-06T07:29:15.030092+0000","last_clean":"2024-03-06T07:29:15.030092+0000","last_became_active":"2024-03-06T07:29:15.028645+0000","last_became_peered":"2024-03-06T07:29:15.028645+0000","last_unstale":"2024-03-06T07:29:15.030092+0000","last_undegraded":"2024-03-06T07:29:15.030092+0000","last_fullsized":"2024-03-06T07:29:15.030092+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:23.687635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"210'3","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044128+0000","last_change":"2024-03-06T07:29:14.028366+0000","last_active":"2024-03-06T07:29:15.044128+0000","last_peered":"2024-03-06T07:29:15.044128+0000","last_clean":"2024-03-06T07:29:15.044128+0000","last_became_active":"2024-03-06T07:29:14.027685+0000","last_became_peered":"2024-03-06T07:29:14.027685+0000","last_unstale":"2024-03-06T07:29:15.044128+0000","last_undegraded":"2024-03-06T07:29:15.044128+0000","last_fullsized":"2024-03-06T07:29:15.044128+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:10.535807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028575+0000","last_change":"2024-03-06T07:29:15.026042+0000","last_active":"2024-03-06T07:29:15.028575+0000","last_peered":"2024-03-06T07:29:15.028575+0000","last_clean":"2024-03-06T07:29:15.028575+0000","last_became_active":"2024-03-06T07:29:15.020873+0000","last_became_peered":"2024-03-06T07:29:15.020873+0000","last_unstale":"2024-03-06T07:29:15.028575+0000","last_undegraded":"2024-03-06T07:29:15.028575+0000","last_fullsized":"2024-03-06T07:29:15.028575+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:14:30.667346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"210'6","reported_seq":18,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.048450+0000","last_change":"2024-03-06T07:29:14.027068+0000","last_active":"2024-03-06T07:29:15.048450+0000","last_peered":"2024-03-06T07:29:15.048450+0000","last_clean":"2024-03-06T07:29:15.048450+0000","last_became_active":"2024-03-06T07:29:14.026793+0000","last_became_peered":"2024-03-06T07:29:14.026793+0000","last_unstale":"2024-03-06T07:29:15.048450+0000","last_undegraded":"2024-03-06T07:29:15.048450+0000","last_fullsized":"2024-03-06T07:29:15.048450+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:35:49.995437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.026797+0000","last_change":"2024-03-06T07:29:15.024412+0000","last_active":"2024-03-06T07:29:15.026797+0000","last_peered":"2024-03-06T07:29:15.026797+0000","last_clean":"2024-03-06T07:29:15.026797+0000","last_became_active":"2024-03-06T07:29:15.024114+0000","last_became_peered":"2024-03-06T07:29:15.024114+0000","last_unstale":"2024-03-06T07:29:15.026797+0000","last_undegraded":"2024-03-06T07:29:15.026797+0000","last_fullsized":"2024-03-06T07:29:15.026797+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.594027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038047+0000","last_change":"2024-03-06T07:29:14.030141+0000","last_active":"2024-03-06T07:29:15.038047+0000","last_peered":"2024-03-06T07:29:15.038047+0000","last_clean":"2024-03-06T07:29:15.038047+0000","last_became_active":"2024-03-06T07:29:14.029877+0000","last_became_peered":"2024-03-06T07:29:14.029877+0000","last_unstale":"2024-03-06T07:29:15.038047+0000","last_undegraded":"2024-03-06T07:29:15.038047+0000","last_fullsized":"2024-03-06T07:29:15.038047+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:52.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.043723+0000","last_change":"2024-03-06T07:29:15.043384+0000","last_active":"2024-03-06T07:29:15.043723+0000","last_peered":"2024-03-06T07:29:15.043723+0000","last_clean":"2024-03-06T07:29:15.043723+0000","last_became_active":"2024-03-06T07:29:15.043156+0000","last_became_peered":"2024-03-06T07:29:15.043156+0000","last_unstale":"2024-03-06T07:29:15.043723+0000","last_undegraded":"2024-03-06T07:29:15.043723+0000","last_fullsized":"2024-03-06T07:29:15.043723+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:23:50.702294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.020821+0000","last_change":"2024-03-06T07:29:14.025602+0000","last_active":"2024-03-06T07:29:15.020821+0000","last_peered":"2024-03-06T07:29:15.020821+0000","last_clean":"2024-03-06T07:29:15.020821+0000","last_became_active":"2024-03-06T07:29:14.025161+0000","last_became_peered":"2024-03-06T07:29:14.025161+0000","last_unstale":"2024-03-06T07:29:15.020821+0000","last_undegraded":"2024-03-06T07:29:15.020821+0000","last_fullsized":"2024-03-06T07:29:15.020821+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:50.289371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.027893+0000","last_change":"2024-03-06T07:29:15.027434+0000","last_active":"2024-03-06T07:29:15.027893+0000","last_peered":"2024-03-06T07:29:15.027893+0000","last_clean":"2024-03-06T07:29:15.027893+0000","last_became_active":"2024-03-06T07:29:15.027155+0000","last_became_peered":"2024-03-06T07:29:15.027155+0000","last_unstale":"2024-03-06T07:29:15.027893+0000","last_undegraded":"2024-03-06T07:29:15.027893+0000","last_fullsized":"2024-03-06T07:29:15.027893+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:54.219263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017557+0000","last_change":"2024-03-06T07:29:14.022979+0000","last_active":"2024-03-06T07:29:15.017557+0000","last_peered":"2024-03-06T07:29:15.017557+0000","last_clean":"2024-03-06T07:29:15.017557+0000","last_became_active":"2024-03-06T07:29:14.022666+0000","last_became_peered":"2024-03-06T07:29:14.022666+0000","last_unstale":"2024-03-06T07:29:15.017557+0000","last_undegraded":"2024-03-06T07:29:15.017557+0000","last_fullsized":"2024-03-06T07:29:15.017557+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:30:33.696105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038913+0000","last_change":"2024-03-06T07:29:15.037471+0000","last_active":"2024-03-06T07:29:15.038913+0000","last_peered":"2024-03-06T07:29:15.038913+0000","last_clean":"2024-03-06T07:29:15.038913+0000","last_became_active":"2024-03-06T07:29:15.037184+0000","last_became_peered":"2024-03-06T07:29:15.037184+0000","last_unstale":"2024-03-06T07:29:15.038913+0000","last_undegraded":"2024-03-06T07:29:15.038913+0000","last_fullsized":"2024-03-06T07:29:15.038913+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:04.880888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"210'8","reported_seq":20,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.048843+0000","last_change":"2024-03-06T07:29:14.016029+0000","last_active":"2024-03-06T07:29:15.048843+0000","last_peered":"2024-03-06T07:29:15.048843+0000","last_clean":"2024-03-06T07:29:15.048843+0000","last_became_active":"2024-03-06T07:29:14.015756+0000","last_became_peered":"2024-03-06T07:29:14.015756+0000","last_unstale":"2024-03-06T07:29:15.048843+0000","last_undegraded":"2024-03-06T07:29:15.048843+0000","last_fullsized":"2024-03-06T07:29:15.048843+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:19.827128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.030688+0000","last_change":"2024-03-06T07:29:15.027834+0000","last_active":"2024-03-06T07:29:15.030688+0000","last_peered":"2024-03-06T07:29:15.030688+0000","last_clean":"2024-03-06T07:29:15.030688+0000","last_became_active":"2024-03-06T07:29:15.026590+0000","last_became_peered":"2024-03-06T07:29:15.026590+0000","last_unstale":"2024-03-06T07:29:15.030688+0000","last_undegraded":"2024-03-06T07:29:15.030688+0000","last_fullsized":"2024-03-06T07:29:15.030688+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:32:58.884800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046036+0000","last_change":"2024-03-06T07:29:14.027082+0000","last_active":"2024-03-06T07:29:15.046036+0000","last_peered":"2024-03-06T07:29:15.046036+0000","last_clean":"2024-03-06T07:29:15.046036+0000","last_became_active":"2024-03-06T07:29:14.026824+0000","last_became_peered":"2024-03-06T07:29:14.026824+0000","last_unstale":"2024-03-06T07:29:15.046036+0000","last_undegraded":"2024-03-06T07:29:15.046036+0000","last_fullsized":"2024-03-06T07:29:15.046036+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:40.850202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028684+0000","last_change":"2024-03-06T07:29:15.023321+0000","last_active":"2024-03-06T07:29:15.028684+0000","last_peered":"2024-03-06T07:29:15.028684+0000","last_clean":"2024-03-06T07:29:15.028684+0000","last_became_active":"2024-03-06T07:29:15.023004+0000","last_became_peered":"2024-03-06T07:29:15.023004+0000","last_unstale":"2024-03-06T07:29:15.028684+0000","last_undegraded":"2024-03-06T07:29:15.028684+0000","last_fullsized":"2024-03-06T07:29:15.028684+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:10:55.418263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045959+0000","last_change":"2024-03-06T07:29:14.028584+0000","last_active":"2024-03-06T07:29:15.045959+0000","last_peered":"2024-03-06T07:29:15.045959+0000","last_clean":"2024-03-06T07:29:15.045959+0000","last_became_active":"2024-03-06T07:29:14.028176+0000","last_became_peered":"2024-03-06T07:29:14.028176+0000","last_unstale":"2024-03-06T07:29:15.045959+0000","last_undegraded":"2024-03-06T07:29:15.045959+0000","last_fullsized":"2024-03-06T07:29:15.045959+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:01:13.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044290+0000","last_change":"2024-03-06T07:29:15.042753+0000","last_active":"2024-03-06T07:29:15.044290+0000","last_peered":"2024-03-06T07:29:15.044290+0000","last_clean":"2024-03-06T07:29:15.044290+0000","last_became_active":"2024-03-06T07:29:15.035626+0000","last_became_peered":"2024-03-06T07:29:15.035626+0000","last_unstale":"2024-03-06T07:29:15.044290+0000","last_undegraded":"2024-03-06T07:29:15.044290+0000","last_fullsized":"2024-03-06T07:29:15.044290+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:33.060067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"210'1","reported_seq":13,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045061+0000","last_change":"2024-03-06T07:29:14.028516+0000","last_active":"2024-03-06T07:29:15.045061+0000","last_peered":"2024-03-06T07:29:15.045061+0000","last_clean":"2024-03-06T07:29:15.045061+0000","last_became_active":"2024-03-06T07:29:14.028292+0000","last_became_peered":"2024-03-06T07:29:14.028292+0000","last_unstale":"2024-03-06T07:29:15.045061+0000","last_undegraded":"2024-03-06T07:29:15.045061+0000","last_fullsized":"2024-03-06T07:29:15.045061+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:11:55.834599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033959+0000","last_change":"2024-03-06T07:29:15.033315+0000","last_active":"2024-03-06T07:29:15.033959+0000","last_peered":"2024-03-06T07:29:15.033959+0000","last_clean":"2024-03-06T07:29:15.033959+0000","last_became_active":"2024-03-06T07:29:15.032417+0000","last_became_peered":"2024-03-06T07:29:15.032417+0000","last_unstale":"2024-03-06T07:29:15.033959+0000","last_undegraded":"2024-03-06T07:29:15.033959+0000","last_fullsized":"2024-03-06T07:29:15.033959+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:26:57.333190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016801+0000","last_change":"2024-03-06T07:29:14.021450+0000","last_active":"2024-03-06T07:29:15.016801+0000","last_peered":"2024-03-06T07:29:15.016801+0000","last_clean":"2024-03-06T07:29:15.016801+0000","last_became_active":"2024-03-06T07:29:14.021155+0000","last_became_peered":"2024-03-06T07:29:14.021155+0000","last_unstale":"2024-03-06T07:29:15.016801+0000","last_undegraded":"2024-03-06T07:29:15.016801+0000","last_fullsized":"2024-03-06T07:29:15.016801+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:55:57.323530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.040776+0000","last_change":"2024-03-06T07:29:15.036423+0000","last_active":"2024-03-06T07:29:15.040776+0000","last_peered":"2024-03-06T07:29:15.040776+0000","last_clean":"2024-03-06T07:29:15.040776+0000","last_became_active":"2024-03-06T07:29:15.036149+0000","last_became_peered":"2024-03-06T07:29:15.036149+0000","last_unstale":"2024-03-06T07:29:15.040776+0000","last_undegraded":"2024-03-06T07:29:15.040776+0000","last_fullsized":"2024-03-06T07:29:15.040776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:54.219263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045271+0000","last_change":"2024-03-06T07:29:14.032698+0000","last_active":"2024-03-06T07:29:15.045271+0000","last_peered":"2024-03-06T07:29:15.045271+0000","last_clean":"2024-03-06T07:29:15.045271+0000","last_became_active":"2024-03-06T07:29:14.032370+0000","last_became_peered":"2024-03-06T07:29:14.032370+0000","last_unstale":"2024-03-06T07:29:15.045271+0000","last_undegraded":"2024-03-06T07:29:15.045271+0000","last_fullsized":"2024-03-06T07:29:15.045271+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:40:00.738520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.028534+0000","last_change":"2024-03-06T07:29:15.026389+0000","last_active":"2024-03-06T07:29:15.028534+0000","last_peered":"2024-03-06T07:29:15.028534+0000","last_clean":"2024-03-06T07:29:15.028534+0000","last_became_active":"2024-03-06T07:29:15.026089+0000","last_became_peered":"2024-03-06T07:29:15.026089+0000","last_unstale":"2024-03-06T07:29:15.028534+0000","last_undegraded":"2024-03-06T07:29:15.028534+0000","last_fullsized":"2024-03-06T07:29:15.028534+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:41.954664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.012295+0000","last_change":"2024-03-06T07:29:14.031021+0000","last_active":"2024-03-06T07:29:15.012295+0000","last_peered":"2024-03-06T07:29:15.012295+0000","last_clean":"2024-03-06T07:29:15.012295+0000","last_became_active":"2024-03-06T07:29:14.030732+0000","last_became_peered":"2024-03-06T07:29:14.030732+0000","last_unstale":"2024-03-06T07:29:15.012295+0000","last_undegraded":"2024-03-06T07:29:15.012295+0000","last_fullsized":"2024-03-06T07:29:15.012295+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:48.168788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.024694+0000","last_change":"2024-03-06T07:29:15.022946+0000","last_active":"2024-03-06T07:29:15.024694+0000","last_peered":"2024-03-06T07:29:15.024694+0000","last_clean":"2024-03-06T07:29:15.024694+0000","last_became_active":"2024-03-06T07:29:15.022673+0000","last_became_peered":"2024-03-06T07:29:15.022673+0000","last_unstale":"2024-03-06T07:29:15.024694+0000","last_undegraded":"2024-03-06T07:29:15.024694+0000","last_fullsized":"2024-03-06T07:29:15.024694+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:26.929627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.034646+0000","last_change":"2024-03-06T07:29:14.028405+0000","last_active":"2024-03-06T07:29:15.034646+0000","last_peered":"2024-03-06T07:29:15.034646+0000","last_clean":"2024-03-06T07:29:15.034646+0000","last_became_active":"2024-03-06T07:29:14.028115+0000","last_became_peered":"2024-03-06T07:29:14.028115+0000","last_unstale":"2024-03-06T07:29:15.034646+0000","last_undegraded":"2024-03-06T07:29:15.034646+0000","last_fullsized":"2024-03-06T07:29:15.034646+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:15:42.412040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046733+0000","last_change":"2024-03-06T07:29:15.044137+0000","last_active":"2024-03-06T07:29:15.046733+0000","last_peered":"2024-03-06T07:29:15.046733+0000","last_clean":"2024-03-06T07:29:15.046733+0000","last_became_active":"2024-03-06T07:29:15.043875+0000","last_became_peered":"2024-03-06T07:29:15.043875+0000","last_unstale":"2024-03-06T07:29:15.046733+0000","last_undegraded":"2024-03-06T07:29:15.046733+0000","last_fullsized":"2024-03-06T07:29:15.046733+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:40:06.222903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017522+0000","last_change":"2024-03-06T07:29:14.018984+0000","last_active":"2024-03-06T07:29:15.017522+0000","last_peered":"2024-03-06T07:29:15.017522+0000","last_clean":"2024-03-06T07:29:15.017522+0000","last_became_active":"2024-03-06T07:29:14.018696+0000","last_became_peered":"2024-03-06T07:29:14.018696+0000","last_unstale":"2024-03-06T07:29:15.017522+0000","last_undegraded":"2024-03-06T07:29:15.017522+0000","last_fullsized":"2024-03-06T07:29:15.017522+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:24:47.511788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032920+0000","last_change":"2024-03-06T07:29:15.031145+0000","last_active":"2024-03-06T07:29:15.032920+0000","last_peered":"2024-03-06T07:29:15.032920+0000","last_clean":"2024-03-06T07:29:15.032920+0000","last_became_active":"2024-03-06T07:29:15.030371+0000","last_became_peered":"2024-03-06T07:29:15.030371+0000","last_unstale":"2024-03-06T07:29:15.032920+0000","last_undegraded":"2024-03-06T07:29:15.032920+0000","last_fullsized":"2024-03-06T07:29:15.032920+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:32.007601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"210'2","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044633+0000","last_change":"2024-03-06T07:29:14.020479+0000","last_active":"2024-03-06T07:29:15.044633+0000","last_peered":"2024-03-06T07:29:15.044633+0000","last_clean":"2024-03-06T07:29:15.044633+0000","last_became_active":"2024-03-06T07:29:14.020183+0000","last_became_peered":"2024-03-06T07:29:14.020183+0000","last_unstale":"2024-03-06T07:29:15.044633+0000","last_undegraded":"2024-03-06T07:29:15.044633+0000","last_fullsized":"2024-03-06T07:29:15.044633+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:22:21.722072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.029615+0000","last_change":"2024-03-06T07:29:15.025585+0000","last_active":"2024-03-06T07:29:15.029615+0000","last_peered":"2024-03-06T07:29:15.029615+0000","last_clean":"2024-03-06T07:29:15.029615+0000","last_became_active":"2024-03-06T07:29:15.025211+0000","last_became_peered":"2024-03-06T07:29:15.025211+0000","last_unstale":"2024-03-06T07:29:15.029615+0000","last_undegraded":"2024-03-06T07:29:15.029615+0000","last_fullsized":"2024-03-06T07:29:15.029615+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:36:52.558827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016800+0000","last_change":"2024-03-06T07:29:14.026232+0000","last_active":"2024-03-06T07:29:15.016800+0000","last_peered":"2024-03-06T07:29:15.016800+0000","last_clean":"2024-03-06T07:29:15.016800+0000","last_became_active":"2024-03-06T07:29:14.025973+0000","last_became_peered":"2024-03-06T07:29:14.025973+0000","last_unstale":"2024-03-06T07:29:15.016800+0000","last_undegraded":"2024-03-06T07:29:15.016800+0000","last_fullsized":"2024-03-06T07:29:15.016800+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:04:23.496501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.022894+0000","last_change":"2024-03-06T07:29:15.021329+0000","last_active":"2024-03-06T07:29:15.022894+0000","last_peered":"2024-03-06T07:29:15.022894+0000","last_clean":"2024-03-06T07:29:15.022894+0000","last_became_active":"2024-03-06T07:29:15.021026+0000","last_became_peered":"2024-03-06T07:29:15.021026+0000","last_unstale":"2024-03-06T07:29:15.022894+0000","last_undegraded":"2024-03-06T07:29:15.022894+0000","last_fullsized":"2024-03-06T07:29:15.022894+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:24.796604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017937+0000","last_change":"2024-03-06T07:29:14.025589+0000","last_active":"2024-03-06T07:29:15.017937+0000","last_peered":"2024-03-06T07:29:15.017937+0000","last_clean":"2024-03-06T07:29:15.017937+0000","last_became_active":"2024-03-06T07:29:14.025310+0000","last_became_peered":"2024-03-06T07:29:14.025310+0000","last_unstale":"2024-03-06T07:29:15.017937+0000","last_undegraded":"2024-03-06T07:29:15.017937+0000","last_fullsized":"2024-03-06T07:29:15.017937+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:59:39.566571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031850+0000","last_change":"2024-03-06T07:29:15.030510+0000","last_active":"2024-03-06T07:29:15.031850+0000","last_peered":"2024-03-06T07:29:15.031850+0000","last_clean":"2024-03-06T07:29:15.031850+0000","last_became_active":"2024-03-06T07:29:15.030081+0000","last_became_peered":"2024-03-06T07:29:15.030081+0000","last_unstale":"2024-03-06T07:29:15.031850+0000","last_undegraded":"2024-03-06T07:29:15.031850+0000","last_fullsized":"2024-03-06T07:29:15.031850+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:49.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016737+0000","last_change":"2024-03-06T07:29:14.027773+0000","last_active":"2024-03-06T07:29:15.016737+0000","last_peered":"2024-03-06T07:29:15.016737+0000","last_clean":"2024-03-06T07:29:15.016737+0000","last_became_active":"2024-03-06T07:29:14.027437+0000","last_became_peered":"2024-03-06T07:29:14.027437+0000","last_unstale":"2024-03-06T07:29:15.016737+0000","last_undegraded":"2024-03-06T07:29:15.016737+0000","last_fullsized":"2024-03-06T07:29:15.016737+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:17:25.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.022165+0000","last_change":"2024-03-06T07:29:15.021567+0000","last_active":"2024-03-06T07:29:15.022165+0000","last_peered":"2024-03-06T07:29:15.022165+0000","last_clean":"2024-03-06T07:29:15.022165+0000","last_became_active":"2024-03-06T07:29:15.021293+0000","last_became_peered":"2024-03-06T07:29:15.021293+0000","last_unstale":"2024-03-06T07:29:15.022165+0000","last_undegraded":"2024-03-06T07:29:15.022165+0000","last_fullsized":"2024-03-06T07:29:15.022165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:41:45.090477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016131+0000","last_change":"2024-03-06T07:29:14.022154+0000","last_active":"2024-03-06T07:29:15.016131+0000","last_peered":"2024-03-06T07:29:15.016131+0000","last_clean":"2024-03-06T07:29:15.016131+0000","last_became_active":"2024-03-06T07:29:14.021888+0000","last_became_peered":"2024-03-06T07:29:14.021888+0000","last_unstale":"2024-03-06T07:29:15.016131+0000","last_undegraded":"2024-03-06T07:29:15.016131+0000","last_fullsized":"2024-03-06T07:29:15.016131+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:27.800819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045890+0000","last_change":"2024-03-06T07:29:15.042221+0000","last_active":"2024-03-06T07:29:15.045890+0000","last_peered":"2024-03-06T07:29:15.045890+0000","last_clean":"2024-03-06T07:29:15.045890+0000","last_became_active":"2024-03-06T07:29:15.041660+0000","last_became_peered":"2024-03-06T07:29:15.041660+0000","last_unstale":"2024-03-06T07:29:15.045890+0000","last_undegraded":"2024-03-06T07:29:15.045890+0000","last_fullsized":"2024-03-06T07:29:15.045890+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:51.574399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.015949+0000","last_change":"2024-03-06T07:29:14.016086+0000","last_active":"2024-03-06T07:29:15.015949+0000","last_peered":"2024-03-06T07:29:15.015949+0000","last_clean":"2024-03-06T07:29:15.015949+0000","last_became_active":"2024-03-06T07:29:14.015807+0000","last_became_peered":"2024-03-06T07:29:14.015807+0000","last_unstale":"2024-03-06T07:29:15.015949+0000","last_undegraded":"2024-03-06T07:29:15.015949+0000","last_fullsized":"2024-03-06T07:29:15.015949+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:09:39.132709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.045866+0000","last_change":"2024-03-06T07:29:15.043046+0000","last_active":"2024-03-06T07:29:15.045866+0000","last_peered":"2024-03-06T07:29:15.045866+0000","last_clean":"2024-03-06T07:29:15.045866+0000","last_became_active":"2024-03-06T07:29:15.041341+0000","last_became_peered":"2024-03-06T07:29:15.041341+0000","last_unstale":"2024-03-06T07:29:15.045866+0000","last_undegraded":"2024-03-06T07:29:15.045866+0000","last_fullsized":"2024-03-06T07:29:15.045866+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:10.568070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.017277+0000","last_change":"2024-03-06T07:29:14.025780+0000","last_active":"2024-03-06T07:29:15.017277+0000","last_peered":"2024-03-06T07:29:15.017277+0000","last_clean":"2024-03-06T07:29:15.017277+0000","last_became_active":"2024-03-06T07:29:14.025537+0000","last_became_peered":"2024-03-06T07:29:14.025537+0000","last_unstale":"2024-03-06T07:29:15.017277+0000","last_undegraded":"2024-03-06T07:29:15.017277+0000","last_fullsized":"2024-03-06T07:29:15.017277+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:36:24.404129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.033171+0000","last_change":"2024-03-06T07:29:15.032208+0000","last_active":"2024-03-06T07:29:15.033171+0000","last_peered":"2024-03-06T07:29:15.033171+0000","last_clean":"2024-03-06T07:29:15.033171+0000","last_became_active":"2024-03-06T07:29:15.031914+0000","last_became_peered":"2024-03-06T07:29:15.031914+0000","last_unstale":"2024-03-06T07:29:15.033171+0000","last_undegraded":"2024-03-06T07:29:15.033171+0000","last_fullsized":"2024-03-06T07:29:15.033171+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:31.843059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.015573+0000","last_change":"2024-03-06T07:29:14.026890+0000","last_active":"2024-03-06T07:29:15.015573+0000","last_peered":"2024-03-06T07:29:15.015573+0000","last_clean":"2024-03-06T07:29:15.015573+0000","last_became_active":"2024-03-06T07:29:14.026583+0000","last_became_peered":"2024-03-06T07:29:14.026583+0000","last_unstale":"2024-03-06T07:29:15.015573+0000","last_undegraded":"2024-03-06T07:29:15.015573+0000","last_fullsized":"2024-03-06T07:29:15.015573+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:12.989381+0000","last_clean_scrub_stamp":"2024-03-06T07:29:12.989381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:46.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.044863+0000","last_change":"2024-03-06T07:29:15.037476+0000","last_active":"2024-03-06T07:29:15.044863+0000","last_peered":"2024-03-06T07:29:15.044863+0000","last_clean":"2024-03-06T07:29:15.044863+0000","last_became_active":"2024-03-06T07:29:15.036305+0000","last_became_peered":"2024-03-06T07:29:15.036305+0000","last_unstale":"2024-03-06T07:29:15.044863+0000","last_undegraded":"2024-03-06T07:29:15.044863+0000","last_fullsized":"2024-03-06T07:29:15.044863+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.999690+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.999690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:07:19.230153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":524,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.016541+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:29:15.016541+0000","last_peered":"2024-03-06T07:29:15.016541+0000","last_clean":"2024-03-06T07:29:15.016541+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:29:15.016541+0000","last_undegraded":"2024-03-06T07:29:15.016541+0000","last_fullsized":"2024-03-06T07:29:15.016541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33840,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338000,"statfs":{"total":96636764160,"available":96602112000,"internally_reserved":0,"allocated":2719744,"data_stored":1876838,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33772,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338068,"statfs":{"total":96636764160,"available":96602181632,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33772,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338068,"statfs":{"total":96636764160,"available":96602181632,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33772,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338068,"statfs":{"total":96636764160,"available":96602181632,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37940,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333900,"statfs":{"total":96636764160,"available":96597913600,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33776,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338064,"statfs":{"total":96636764160,"available":96602177536,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2654208,"data_stored":1286470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574962,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2719744,"data_stored":1876838,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:29:18.439 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:29:18.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:29:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.583+0000 7fc87dde9640 1 Processor -- start 2024-03-06T07:29:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.583+0000 7fc87dde9640 1 -- start start 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc87dde9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8781539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc87dde9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8780c4740 0x7fc8780701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc87dde9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8780c62a0 con 0x7fc8780c69a0 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc87dde9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc878153f00 con 0x7fc8780c4740 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc87dde9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780706e0 con 0x7fc8780a8290 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc876ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8780c4740 0x7fc8780701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc876ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8780c4740 0x7fc8780701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37462/0 (socket says 172.21.15.26:37462) 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc876ffd640 1 -- 172.21.15.26:0/1785321699 learned_addr learned my addr 172.21.15.26:0/1785321699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc877fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc8780c69a0 0x7fc878072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39138/0 (socket says 172.21.15.26:39138) 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc8777fe640 1 --2- 172.21.15.26:0/1785321699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8781539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc876ffd640 1 -- 172.21.15.26:0/1785321699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 msgr2=0x7fc8781539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc876ffd640 1 --2- 172.21.15.26:0/1785321699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8781539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc876ffd640 1 -- 172.21.15.26:0/1785321699 >> v1:172.21.15.26:6789/0 conn(0x7fc8780c69a0 legacy=0x7fc878072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc876ffd640 1 -- 172.21.15.26:0/1785321699 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780c4c00 con 0x7fc8780c4740 2024-03-06T07:29:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc8767fc640 1 -- 172.21.15.26:0/1785321699 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632196578 0 0) 0x7fc8780c62a0 con 0x7fc8780c69a0 2024-03-06T07:29:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.584+0000 7fc8777fe640 1 --2- 172.21.15.26:0/1785321699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8781539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.585+0000 7fc876ffd640 1 --2- 172.21.15.26:0/1785321699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc8780701a0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009870 tx=0x7fc86c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0319abc509819d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.585+0000 7fc8767fc640 1 -- 172.21.15.26:0/1785321699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0317c0 con 0x7fc8780c4740 2024-03-06T07:29:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.585+0000 7fc8767fc640 1 -- 172.21.15.26:0/1785321699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c037070 con 0x7fc8780c4740 2024-03-06T07:29:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.585+0000 7fc8767fc640 1 -- 172.21.15.26:0/1785321699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c031b80 con 0x7fc8780c4740 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 -- 172.21.15.26:0/1785321699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 msgr2=0x7fc8780701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 --2- 172.21.15.26:0/1785321699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc8780701a0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009870 tx=0x7fc86c02b860 comp rx=0 tx=0).stop 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 -- 172.21.15.26:0/1785321699 shutdown_connections 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 --2- 172.21.15.26:0/1785321699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc8780701a0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 --2- 172.21.15.26:0/1785321699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8781539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 -- 172.21.15.26:0/1785321699 >> 172.21.15.26:0/1785321699 conn(0x7fc8780c5560 msgr2=0x7fc8780c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 -- 172.21.15.26:0/1785321699 shutdown_connections 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.586+0000 7fc87dde9640 1 -- 172.21.15.26:0/1785321699 wait complete. 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.587+0000 7fc87dde9640 1 Processor -- start 2024-03-06T07:29:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.587+0000 7fc87dde9640 1 -- start start 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc87dde9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8780c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc87dde9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc878154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc87dde9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8780c69a0 0x7fc8781df880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc87dde9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87806fc80 con 0x7fc8780c4740 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc87dde9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87806f9a0 con 0x7fc8780c69a0 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc87dde9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc878072d90 con 0x7fc8780a8290 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc878154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc878154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37474/0 (socket says 172.21.15.26:37474) 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 -- 172.21.15.26:0/179247218 learned_addr learned my addr 172.21.15.26:0/179247218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc877fff640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8780c69a0 0x7fc8781df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc8777fe640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8780c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 -- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 msgr2=0x7fc8780c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8780c2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 -- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8780c69a0 msgr2=0x7fc8781df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8780c69a0 0x7fc8781df880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 -- 172.21.15.26:0/179247218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87814a330 con 0x7fc8780c4740 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc877fff640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8780c69a0 0x7fc8781df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.588+0000 7fc876ffd640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc878154650 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fc86c034040 tx=0x7fc86c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.589+0000 7fc85ffff640 1 -- 172.21.15.26:0/179247218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c004230 con 0x7fc8780c4740 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.589+0000 7fc85ffff640 1 -- 172.21.15.26:0/179247218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c0043d0 con 0x7fc8780c4740 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.589+0000 7fc85ffff640 1 -- 172.21.15.26:0/179247218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c003850 con 0x7fc8780c4740 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.589+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8781eed20 con 0x7fc8780c4740 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.589+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc878151e80 con 0x7fc8780c4740 2024-03-06T07:29:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.590+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87804f330 con 0x7fc8780c4740 2024-03-06T07:29:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.590+0000 7fc85ffff640 1 -- 172.21.15.26:0/179247218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc86c04c060 con 0x7fc8780c4740 2024-03-06T07:29:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.591+0000 7fc85ffff640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc86406b4d0 0x7fc86406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.591+0000 7fc85ffff640 1 -- 172.21.15.26:0/179247218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc86c0abe90 con 0x7fc8780c4740 2024-03-06T07:29:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.591+0000 7fc8777fe640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc86406b4d0 0x7fc86406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.592+0000 7fc8777fe640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc86406b4d0 0x7fc86406d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc878153ed0 tx=0x7fc868008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.595+0000 7fc85ffff640 1 -- 172.21.15.26:0/179247218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc86c040070 con 0x7fc8780c4740 2024-03-06T07:29:18.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.850+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc8780c30b0 con 0x7fc8780c4740 2024-03-06T07:29:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.852+0000 7fc85ffff640 1 -- 172.21.15.26:0/179247218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc86c07ad30 con 0x7fc8780c4740 2024-03-06T07:29:18.854 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-03-06T07:29:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDxGOhlKZ6fKhAAcmlRrF/eIaMVldJpUkmhLA== 2024-03-06T07:29:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-03-06T07:29:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-03-06T07:29:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:29:18.854 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc86406b4d0 msgr2=0x7fc86406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc86406b4d0 0x7fc86406d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc878153ed0 tx=0x7fc868008040 comp rx=0 tx=0).stop 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 msgr2=0x7fc878154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc878154650 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fc86c034040 tx=0x7fc86c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 shutdown_connections 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc86406b4d0 0x7fc86406d990 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8780c69a0 0x7fc8781df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8780c4740 0x7fc878154650 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 --2- 172.21.15.26:0/179247218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8780a8290 0x7fc8780c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 >> 172.21.15.26:0/179247218 conn(0x7fc8780c5560 msgr2=0x7fc8780c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 shutdown_connections 2024-03-06T07:29:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.854+0000 7fc87dde9640 1 -- 172.21.15.26:0/179247218 wait complete. 2024-03-06T07:29:18.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:18.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.970+0000 7f5d8d2c9640 1 Processor -- start 2024-03-06T07:29:18.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.970+0000 7f5d8d2c9640 1 -- start start 2024-03-06T07:29:18.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d8d2c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5d880c6780 0x7f5d88158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d8d2c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d881592d0 0x7f5d880c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d8d2c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d88072630 con 0x7f5d8806da40 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d8d2c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8806de20 con 0x7f5d881592d0 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d8d2c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d88072910 con 0x7f5d880c6780 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d881592d0 0x7f5d880c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d881592d0 0x7f5d880c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37482/0 (socket says 172.21.15.26:37482) 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d877fe640 1 -- 172.21.15.26:0/3407256629 learned_addr learned my addr 172.21.15.26:0/3407256629 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d867fc640 1 --2- 172.21.15.26:0/3407256629 >> v2:172.21.15.117:3300/0 conn(0x7f5d880c6780 0x7f5d88158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d877fe640 1 -- 172.21.15.26:0/3407256629 >> v2:172.21.15.117:3300/0 conn(0x7f5d880c6780 msgr2=0x7f5d88158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d877fe640 1 --2- 172.21.15.26:0/3407256629 >> v2:172.21.15.117:3300/0 conn(0x7f5d880c6780 0x7f5d88158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d877fe640 1 -- 172.21.15.26:0/3407256629 >> v1:172.21.15.117:6789/0 conn(0x7f5d8806da40 legacy=0x7f5d88155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.971+0000 7f5d877fe640 1 -- 172.21.15.26:0/3407256629 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d88072bf0 con 0x7f5d881592d0 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.972+0000 7f5d877fe640 1 --2- 172.21.15.26:0/3407256629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d881592d0 0x7f5d880c3280 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f5d880c7120 tx=0x7f5d8002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffbfa13cfcf3a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.972+0000 7f5d85ffb640 1 -- 172.21.15.26:0/3407256629 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8002ca80 con 0x7f5d881592d0 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.972+0000 7f5d85ffb640 1 -- 172.21.15.26:0/3407256629 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8002cc20 con 0x7f5d881592d0 2024-03-06T07:29:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.972+0000 7f5d85ffb640 1 -- 172.21.15.26:0/3407256629 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d800343f0 con 0x7f5d881592d0 2024-03-06T07:29:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.972+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/3407256629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d881592d0 msgr2=0x7f5d880c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.972+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/3407256629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d881592d0 0x7f5d880c3280 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f5d880c7120 tx=0x7f5d8002bee0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.973+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/3407256629 shutdown_connections 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.973+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/3407256629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d881592d0 0x7f5d880c3280 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.973+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/3407256629 >> v2:172.21.15.117:3300/0 conn(0x7f5d880c6780 0x7f5d88158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.973+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/3407256629 >> 172.21.15.26:0/3407256629 conn(0x7f5d880aa630 msgr2=0x7f5d880aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.973+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/3407256629 shutdown_connections 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.973+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/3407256629 wait complete. 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.974+0000 7f5d8d2c9640 1 Processor -- start 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.974+0000 7f5d8d2c9640 1 -- start start 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.974+0000 7f5d8d2c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8806da40 0x7f5d880ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d8d2c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d880c6780 0x7f5d880adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d8d2c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d881592d0 0x7f5d880ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d8d2c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880c3940 con 0x7f5d8806da40 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d8d2c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880c37c0 con 0x7f5d881592d0 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d8d2c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880c3ac0 con 0x7f5d880c6780 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8806da40 0x7f5d880ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8806da40 0x7f5d880ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37486/0 (socket says 172.21.15.26:37486) 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 -- 172.21.15.26:0/1913479546 learned_addr learned my addr 172.21.15.26:0/1913479546 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d867fc640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d880c6780 0x7f5d880adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d877fe640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d881592d0 0x7f5d880ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 -- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d880c6780 msgr2=0x7f5d880adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d880c6780 0x7f5d880adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 -- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d881592d0 msgr2=0x7f5d880ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d881592d0 0x7f5d880ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 -- 172.21.15.26:0/1913479546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8806e800 con 0x7f5d8806da40 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d867fc640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d880c6780 0x7f5d880adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d86ffd640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8806da40 0x7f5d880ac0d0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c3c0 tx=0x7f5d7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d6f7fe640 1 -- 172.21.15.26:0/1913479546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0052d0 con 0x7f5d8806da40 2024-03-06T07:29:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d6f7fe640 1 -- 172.21.15.26:0/1913479546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c005490 con 0x7f5d8806da40 2024-03-06T07:29:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8806eae0 con 0x7f5d8806da40 2024-03-06T07:29:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.975+0000 7f5d6f7fe640 1 -- 172.21.15.26:0/1913479546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c007260 con 0x7f5d8806da40 2024-03-06T07:29:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.976+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d880ac610 con 0x7f5d8806da40 2024-03-06T07:29:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.976+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8804f330 con 0x7f5d8806da40 2024-03-06T07:29:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.977+0000 7f5d6f7fe640 1 -- 172.21.15.26:0/1913479546 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d7c007400 con 0x7f5d8806da40 2024-03-06T07:29:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.977+0000 7f5d6f7fe640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d7406b500 0x7f5d7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.978+0000 7f5d6f7fe640 1 -- 172.21.15.26:0/1913479546 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d7c089750 con 0x7f5d8806da40 2024-03-06T07:29:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.978+0000 7f5d867fc640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d7406b500 0x7f5d7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.978+0000 7f5d867fc640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d7406b500 0x7f5d7406d9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006fd0 tx=0x7f5d78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:18.982+0000 7f5d6f7fe640 1 -- 172.21.15.26:0/1913479546 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d7c0584d0 con 0x7f5d8806da40 2024-03-06T07:29:19.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.201+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d88073240 con 0x7f5d8806da40 2024-03-06T07:29:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.201+0000 7f5d6f7fe640 1 -- 172.21.15.26:0/1913479546 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f5d7c057dd0 con 0x7f5d8806da40 2024-03-06T07:29:19.203 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:19.203 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":198,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":9799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:16.379626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 198 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d7406b500 msgr2=0x7f5d7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d7406b500 0x7f5d7406d9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006fd0 tx=0x7f5d78008040 comp rx=0 tx=0).stop 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8806da40 msgr2=0x7f5d880ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8806da40 0x7f5d880ac0d0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c3c0 tx=0x7f5d7c00c880 comp rx=0 tx=0).stop 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 shutdown_connections 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d7406b500 0x7f5d7406d9c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d881592d0 0x7f5d880ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d880c6780 0x7f5d880adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 --2- 172.21.15.26:0/1913479546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8806da40 0x7f5d880ac0d0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 >> 172.21.15.26:0/1913479546 conn(0x7f5d880aa630 msgr2=0x7f5d880aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 shutdown_connections 2024-03-06T07:29:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.203+0000 7f5d8d2c9640 1 -- 172.21.15.26:0/1913479546 wait complete. 2024-03-06T07:29:19.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 198, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:16.379626+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10737}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:19.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:29:19.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:29:19.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.723+0000 7f56f2e2a640 1 Processor -- start 2024-03-06T07:29:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.723+0000 7f56f2e2a640 1 -- start start 2024-03-06T07:29:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.724+0000 7f56f2e2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec0c7170 0x7f56ec0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.724+0000 7f56f2e2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0a8390 con 0x7f56ec06ff10 2024-03-06T07:29:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.724+0000 7f56f2e2a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0730f0 con 0x7f56ec0c6c40 2024-03-06T07:29:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.724+0000 7f56f2e2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0c7620 con 0x7f56ec0c7170 2024-03-06T07:29:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.724+0000 7f56f0bc6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56ec06ff10 0x7f56ec153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39150/0 (socket says 172.21.15.26:39150) 2024-03-06T07:29:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.724+0000 7f56f0bc6640 1 -- 172.21.15.26:0/4094717076 learned_addr learned my addr 172.21.15.26:0/4094717076 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.724+0000 7f56e3fff640 1 --2- 172.21.15.26:0/4094717076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec0c7170 0x7f56ec0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815599029 0 0) 0x7f56ec0a8390 con 0x7f56ec06ff10 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d4003520 con 0x7f56ec06ff10 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227814119 0 0) 0x7f56ec0730f0 con 0x7f56ec0c6c40 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56ec0a8390 con 0x7f56ec0c6c40 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e3fff640 1 -- 172.21.15.26:0/4094717076 >> v1:172.21.15.117:6789/0 conn(0x7f56ec0c6c40 legacy=0x7f56ec15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e3fff640 1 -- 172.21.15.26:0/4094717076 >> v1:172.21.15.26:6789/0 conn(0x7f56ec06ff10 legacy=0x7f56ec153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3095308871 0 0) 0x7f56d4003520 con 0x7f56ec06ff10 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.725+0000 7f56e3fff640 1 -- 172.21.15.26:0/4094717076 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0733d0 con 0x7f56ec0c7170 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56e3fff640 1 --2- 172.21.15.26:0/4094717076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec0c7170 0x7f56ec0c8810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0097d0 tx=0x7f56dc02b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62058c052c42b53c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc037070 con 0x7f56ec0c7170 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc030510 con 0x7f56ec0c7170 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56e37fe640 1 -- 172.21.15.26:0/4094717076 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8001670 con 0x7f56ec0c7170 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56f2e2a640 1 -- 172.21.15.26:0/4094717076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec0c7170 msgr2=0x7f56ec0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/4094717076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec0c7170 0x7f56ec0c8810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0097d0 tx=0x7f56dc02b6e0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56f2e2a640 1 -- 172.21.15.26:0/4094717076 shutdown_connections 2024-03-06T07:29:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/4094717076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec0c7170 0x7f56ec0c8810 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56f2e2a640 1 -- 172.21.15.26:0/4094717076 >> 172.21.15.26:0/4094717076 conn(0x7f56ec06b970 msgr2=0x7f56ec0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56f2e2a640 1 -- 172.21.15.26:0/4094717076 shutdown_connections 2024-03-06T07:29:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.726+0000 7f56f2e2a640 1 -- 172.21.15.26:0/4094717076 wait complete. 2024-03-06T07:29:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.727+0000 7f56f2e2a640 1 Processor -- start 2024-03-06T07:29:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.727+0000 7f56f2e2a640 1 -- start start 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56f2e2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec06ff10 0x7f56ec0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56f2e2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56ec0c6c40 0x7f56ec0c1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56f2e2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56ec0c7170 0x7f56ec0c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56f2e2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1544b0 con 0x7f56ec0c6c40 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56f2e2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec154330 con 0x7f56ec0c7170 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56f2e2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0702f0 con 0x7f56ec06ff10 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56ec0c6c40 0x7f56ec0c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56ec0c6c40 0x7f56ec0c1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37498/0 (socket says 172.21.15.26:37498) 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.728+0000 7f56e3fff640 1 -- 172.21.15.26:0/518978931 learned_addr learned my addr 172.21.15.26:0/518978931 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec06ff10 0x7f56ec0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56f13c7640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56ec0c7170 0x7f56ec0c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e3fff640 1 -- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec06ff10 msgr2=0x7f56ec0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e3fff640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec06ff10 0x7f56ec0ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e3fff640 1 -- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56ec0c7170 msgr2=0x7f56ec0c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e3fff640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56ec0c7170 0x7f56ec0c1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e3fff640 1 -- 172.21.15.26:0/518978931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec1e97e0 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e3fff640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56ec0c6c40 0x7f56ec0c1530 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0097d0 tx=0x7f56dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e17fa640 1 -- 172.21.15.26:0/518978931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc037040 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e17fa640 1 -- 172.21.15.26:0/518978931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc004250 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec0addf0 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56e17fa640 1 -- 172.21.15.26:0/518978931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc03c670 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.729+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec0ac920 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.731+0000 7f56e17fa640 1 -- 172.21.15.26:0/518978931 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56dc03a070 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.731+0000 7f56e17fa640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56d806b500 0x7f56d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.731+0000 7f56e17fa640 1 -- 172.21.15.26:0/518978931 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56dc0abbd0 con 0x7f56ec0c6c40 2024-03-06T07:29:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.731+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56d806b500 0x7f56d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.732+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56d806b500 0x7f56d806d9c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f56e80017e0 tx=0x7f56e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.732+0000 7f56f5a12740 1 Processor -- start 2024-03-06T07:29:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.732+0000 7f56f5a12740 1 -- start start 2024-03-06T07:29:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b98da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9b98df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98e4b0 0x561e9b98a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f5a12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9b8f46b0 con 0x561e9b98ff50 2024-03-06T07:29:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9b89da60 con 0x561e9b98e4b0 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e9b8f0750 con 0x561e9b990330 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b98da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b98da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37508/0 (socket says 172.21.15.26:37508) 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 -- 172.21.15.26:0/2957677824 learned_addr learned my addr 172.21.15.26:0/2957677824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98e4b0 0x561e9b98a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56e3fff640 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9b98df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 -- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 msgr2=0x561e9b98df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9b98df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 -- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98e4b0 msgr2=0x561e9b98a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98e4b0 0x561e9b98a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 -- 172.21.15.26:0/2957677824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9b8bbf30 con 0x561e9b98ff50 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56e3fff640 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9b98df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98e4b0 0x561e9b98a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b98da30 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f56e8007fd0 tx=0x7f56e8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab5847f1b48fc30 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.733+0000 7f56d27fc640 1 -- 172.21.15.26:0/2957677824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800dbb0 con 0x561e9b98ff50 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56d27fc640 1 -- 172.21.15.26:0/2957677824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e800dd50 con 0x561e9b98ff50 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56d27fc640 1 -- 172.21.15.26:0/2957677824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8020650 con 0x561e9b98ff50 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 -- 172.21.15.26:0/2957677824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 msgr2=0x561e9b98da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b98da30 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f56e8007fd0 tx=0x7f56e8010000 comp rx=0 tx=0).stop 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 -- 172.21.15.26:0/2957677824 shutdown_connections 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98e4b0 0x561e9b98a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9b98df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2957677824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b98da30 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 -- 172.21.15.26:0/2957677824 >> 172.21.15.26:0/2957677824 conn(0x561e9b98c7d0 msgr2=0x561e9b9b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 -- 172.21.15.26:0/2957677824 shutdown_connections 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.734+0000 7f56f5a12740 1 -- 172.21.15.26:0/2957677824 wait complete. 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.735+0000 7f56f5a12740 1 -- start start 2024-03-06T07:29:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.735+0000 7f56f5a12740 1 Processor -- start 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b9b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b9e05b0 0x561e9b98a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f5a12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9b9b2230 con 0x561e9b98ff50 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9b8b0ea0 con 0x561e9b990330 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e9b9b1c70 con 0x561e9b9e05b0 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f5a12740 10 client.0 _ll_register_callbacks cb 0x561e9b8b0870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f0bc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b9b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f0bc6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b9b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37518/0 (socket says 172.21.15.26:37518) 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.736+0000 7f56f0bc6640 1 -- 172.21.15.26:0/1377426376 learned_addr learned my addr 172.21.15.26:0/1377426376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56e3fff640 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f13c7640 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b9e05b0 0x561e9b98a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f0bc6640 1 -- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b9e05b0 msgr2=0x561e9b98a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b9e05b0 0x561e9b98a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f0bc6640 1 -- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 msgr2=0x561e9b9ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f0bc6640 1 -- 172.21.15.26:0/1377426376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9b8b1270 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b9b5a30 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f56e8007560 tx=0x7f56e8007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a6c7d302788105 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56cb7fe640 1 -- 172.21.15.26:0/1377426376 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8020bc0 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56cb7fe640 1 -- 172.21.15.26:0/1377426376 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e8020d60 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e800dd50 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56cb7fe640 1 -- 172.21.15.26:0/1377426376 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e8034050 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 10 client.10881 fetch_fsmap learned FSMap version 198 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=2+}) v3 -- 0x561e9b6092d0 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56cb7fe640 1 -- 172.21.15.26:0/1377426376 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e800dd50 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56cb7fe640 1 -- 172.21.15.26:0/1377426376 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 198) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f56e8012410 con 0x561e9b98ff50 2024-03-06T07:29:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 10 client.10881 fetch_fsmap finished waiting for FSMap version 198 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 10 client.10881 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 20 client.10881 populate_metadata read hostname 'smithi026' 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9c8c90 0x561e9b9dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 4 client.10881 mds_command: new command op to 10737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.737+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 --> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e9b901bf0 con 0x561e9b9c8c90 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.738+0000 7f56e3fff640 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9c8c90 0x561e9b9dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.738+0000 7f56cb7fe640 10 client.10881 ms_handle_connect on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.738+0000 7f56e3fff640 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9c8c90 0x561e9b9dc740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec7fed2bd381068 server_cookie=66e7fa98f44c6d07 in_seq=0 out_seq=0 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.738+0000 7f56cb7fe640 1 -- 172.21.15.26:0/1377426376 <== mds.0 v2:172.21.15.26:6833/4219086401 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561e9b901bf0 con 0x561e9b9c8c90 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.738+0000 7f56cb7fe640 10 client.10881 handle_command_reply: tid=0 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 client.10881 shutdown 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 msgr2=0x561e9b9b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b9b5a30 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f56e8007560 tx=0x7f56e8007000 comp rx=0 tx=0).stop 2024-03-06T07:29:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 shutdown_connections 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9c8c90 0x561e9b9dc740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b9e05b0 0x561e9b98a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56cb7fe640 0 client.10881 ms_handle_reset on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 --2- 172.21.15.26:0/1377426376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b98ff50 0x561e9b9b5a30 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 >> 172.21.15.26:0/1377426376 conn(0x561e9b98e4b0 msgr2=0x561e9b9ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 shutdown_connections 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 1 -- 172.21.15.26:0/1377426376 wait complete. 2024-03-06T07:29:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.739+0000 7f56f5a12740 20 client.10881 trim_cache size 0 max 16384 2024-03-06T07:29:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.759+0000 7f56f5a12740 1 Processor -- start 2024-03-06T07:29:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.760+0000 7f56f5a12740 1 -- start start 2024-03-06T07:29:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b98ff50 0x561e9b98b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9b9cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f5a12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1f4c00 con 0x561e9b9e05b0 2024-03-06T07:29:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1f4030 con 0x561e9b990330 2024-03-06T07:29:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1f4600 con 0x561e9b98ff50 2024-03-06T07:29:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9b9cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9b9cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37534/0 (socket says 172.21.15.26:37534) 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 -- 172.21.15.26:0/2847044583 learned_addr learned my addr 172.21.15.26:0/2847044583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b98ff50 0x561e9b98b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56e3fff640 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 -- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b98ff50 msgr2=0x561e9b98b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b98ff50 0x561e9b98b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 -- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 msgr2=0x561e9b9cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 -- 172.21.15.26:0/2847044583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9b9b5da0 con 0x561e9b9e05b0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9b9cfb40 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f56e4003f80 tx=0x7f56e40054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5809ee3b87c8b09a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56d1ffb640 1 -- 172.21.15.26:0/2847044583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4001c30 con 0x561e9b9e05b0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56d1ffb640 1 -- 172.21.15.26:0/2847044583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e4001dd0 con 0x561e9b9e05b0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.761+0000 7f56d1ffb640 1 -- 172.21.15.26:0/2847044583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4001f80 con 0x561e9b9e05b0 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 -- 172.21.15.26:0/2847044583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 msgr2=0x561e9b9cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9b9cfb40 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f56e4003f80 tx=0x7f56e40054e0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 -- 172.21.15.26:0/2847044583 shutdown_connections 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9b9cfb40 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b990330 0x561e9b9cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2847044583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b98ff50 0x561e9b98b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 -- 172.21.15.26:0/2847044583 >> 172.21.15.26:0/2847044583 conn(0x561e9b98e4b0 msgr2=0x561e9b9cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 -- 172.21.15.26:0/2847044583 shutdown_connections 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 -- 172.21.15.26:0/2847044583 wait complete. 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.762+0000 7f56f5a12740 1 -- start start 2024-03-06T07:29:19.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 1 Processor -- start 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98ff50 0x561e9b98a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9ba40010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9ba39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9b9b3fc0 con 0x561e9b9e05b0 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9b9dfdb0 con 0x561e9b98ff50 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e9b9b5250 con 0x561e9b990330 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.763+0000 7f56f5a12740 10 client.0 _ll_register_callbacks cb 0x7f56e8005a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9ba39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9ba39e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37536/0 (socket says 172.21.15.26:37536) 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 -- 172.21.15.26:0/2377821002 learned_addr learned my addr 172.21.15.26:0/2377821002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56e3fff640 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9ba40010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98ff50 0x561e9b98a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 -- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 msgr2=0x561e9ba40010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9ba40010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 -- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98ff50 msgr2=0x561e9b98a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98ff50 0x561e9b98a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 -- 172.21.15.26:0/2377821002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9b6d36b0 con 0x561e9b9e05b0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f13c7640 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9ba39e30 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005520 tx=0x7f56e4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5a621450544d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56cb7fe640 1 -- 172.21.15.26:0/2377821002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4025030 con 0x561e9b9e05b0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56cb7fe640 1 -- 172.21.15.26:0/2377821002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e401b070 con 0x561e9b9e05b0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e4001dd0 con 0x561e9b9e05b0 2024-03-06T07:29:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56cb7fe640 1 -- 172.21.15.26:0/2377821002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e4001c30 con 0x561e9b9e05b0 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f5a12740 10 client.10887 fetch_fsmap learned FSMap version 198 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.764+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=2+}) v3 -- 0x561e9b9dd3b0 con 0x561e9b9e05b0 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56cb7fe640 1 -- 172.21.15.26:0/2377821002 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e4001dd0 con 0x561e9b9e05b0 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56cb7fe640 1 -- 172.21.15.26:0/2377821002 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 198) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f56e40170d0 con 0x561e9b9e05b0 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f5a12740 10 client.10887 fetch_fsmap finished waiting for FSMap version 198 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f5a12740 10 client.10887 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f5a12740 20 client.10887 populate_metadata read hostname 'smithi026' 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9f04b0 0x561e9b9f0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f5a12740 4 client.10887 mds_command: new command op to 10737 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 --> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561e9b8fbf40 con 0x561e9b9f04b0 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9f04b0 0x561e9b9f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56cb7fe640 10 client.10887 ms_handle_connect on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f0bc6640 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9f04b0 0x561e9b9f0890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e120376e8fc4c7d server_cookie=d9ee0e5c502cec45 in_seq=0 out_seq=0 2024-03-06T07:29:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56cb7fe640 1 -- 172.21.15.26:0/2377821002 <== mds.0 v2:172.21.15.26:6833/4219086401 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x561e9b8fbf40 con 0x561e9b9f04b0 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56cb7fe640 10 client.10887 handle_command_reply: tid=0 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.765+0000 7f56f5a12740 1 client.10887 shutdown 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 msgr2=0x561e9ba39e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9ba39e30 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005520 tx=0x7f56e4001250 comp rx=0 tx=0).stop 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 shutdown_connections 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x561e9b9f04b0 0x561e9b9f0890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e9b9e05b0 0x561e9ba39e30 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561e9b990330 0x561e9ba40010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 --2- 172.21.15.26:0/2377821002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9b98ff50 0x561e9b98a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 >> 172.21.15.26:0/2377821002 conn(0x561e9b98e4b0 msgr2=0x561e9b9cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56cb7fe640 0 client.10887 ms_handle_reset on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 shutdown_connections 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 1 -- 172.21.15.26:0/2377821002 wait complete. 2024-03-06T07:29:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f5a12740 20 client.10887 trim_cache size 0 max 16384 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 10737, 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 4.7566411390000001, 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 198, 2024-03-06T07:29:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 211, 2024-03-06T07:29:19.775 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 211, 2024-03-06T07:29:19.775 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 9.8461510289999996 2024-03-06T07:29:19.775 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:29:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56d806b500 msgr2=0x7f56d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56d806b500 0x7f56d806d9c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f56e80017e0 tx=0x7f56e8009040 comp rx=0 tx=0).stop 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56ec0c6c40 msgr2=0x7f56ec0c1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56ec0c6c40 0x7f56ec0c1530 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0097d0 tx=0x7f56dc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 shutdown_connections 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f56d806b500 0x7f56d806d9c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56ec0c7170 0x7f56ec0c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56ec0c6c40 0x7f56ec0c1530 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 --2- 172.21.15.26:0/518978931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56ec06ff10 0x7f56ec0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 >> 172.21.15.26:0/518978931 conn(0x7f56ec06b970 msgr2=0x7f56ec1539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.766+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 shutdown_connections 2024-03-06T07:29:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.767+0000 7f56f2e2a640 1 -- 172.21.15.26:0/518978931 wait complete. 2024-03-06T07:29:19.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.879+0000 7f6fd5bd7640 1 Processor -- start 2024-03-06T07:29:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.879+0000 7f6fd5bd7640 1 -- start start 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.879+0000 7f6fd5bd7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6fd0159270 0x7f6fd0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.879+0000 7f6fd5bd7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00c73d0 con 0x7f6fd014f360 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.879+0000 7f6fd5bd7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00be2e0 con 0x7f6fd00c88c0 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.879+0000 7f6fd5bd7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd0159700 con 0x7f6fd0159270 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.879+0000 7f6fcf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6fd014f360 0x7f6fd00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39160/0 (socket says 172.21.15.26:39160) 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fcf7fe640 1 -- 172.21.15.26:0/1357742717 learned_addr learned my addr 172.21.15.26:0/1357742717 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fcffff640 1 --2- 172.21.15.26:0/1357742717 >> v2:172.21.15.117:3300/0 conn(0x7f6fd0159270 0x7f6fd0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278917740 0 0) 0x7f6fd00c73d0 con 0x7f6fd014f360 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc003520 con 0x7f6fd014f360 2024-03-06T07:29:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203486552 0 0) 0x7f6fbc003520 con 0x7f6fd014f360 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 >> v2:172.21.15.117:3300/0 conn(0x7f6fd0159270 msgr2=0x7f6fd0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 --2- 172.21.15.26:0/1357742717 >> v2:172.21.15.117:3300/0 conn(0x7f6fd0159270 0x7f6fd0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 >> v1:172.21.15.117:6789/0 conn(0x7f6fd00c88c0 legacy=0x7f6fd0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00c7670 con 0x7f6fd014f360 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6fc0001670 con 0x7f6fd014f360 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fcffff640 1 --2- 172.21.15.26:0/1357742717 >> v2:172.21.15.117:3300/0 conn(0x7f6fd0159270 0x7f6fd0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 >> v1:172.21.15.26:6789/0 conn(0x7f6fd014f360 legacy=0x7f6fd00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fbc0039b0 0x7f6fbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc004800 0x7f6fbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc0051c0 0x7f6fbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0159700 con 0x7f6fbc004800 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc00c160 con 0x7f6fbc0039b0 2024-03-06T07:29:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc00c360 con 0x7f6fbc0051c0 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fc7fff640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc004800 0x7f6fbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fcf7fe640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fbc0039b0 0x7f6fbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.880+0000 7f6fcffff640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc0051c0 0x7f6fbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fc7fff640 1 -- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc0051c0 msgr2=0x7f6fbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fc7fff640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc0051c0 0x7f6fbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fc7fff640 1 -- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fbc0039b0 msgr2=0x7f6fbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fc7fff640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fbc0039b0 0x7f6fbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fc7fff640 1 -- 172.21.15.26:0/1357742717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc003520 con 0x7f6fbc004800 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fcffff640 1 -- 172.21.15.26:0/1357742717 reap_dead start 2024-03-06T07:29:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fc7fff640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc004800 0x7f6fbc004c80 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8001720 tx=0x7f6fbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cf7e9efdc74217 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8001370 con 0x7f6fbc004800 2024-03-06T07:29:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb80052d0 con 0x7f6fbc004800 2024-03-06T07:29:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.881+0000 7f6fceffd640 1 -- 172.21.15.26:0/1357742717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8003830 con 0x7f6fbc004800 2024-03-06T07:29:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.882+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1357742717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc004800 msgr2=0x7f6fbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.882+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc004800 0x7f6fbc004c80 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8001720 tx=0x7f6fbc005830 comp rx=0 tx=0).stop 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.882+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1357742717 shutdown_connections 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.882+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1357742717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc004800 0x7f6fbc004c80 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.882+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1357742717 >> 172.21.15.26:0/1357742717 conn(0x7f6fd00ac570 msgr2=0x7f6fd00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.882+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1357742717 shutdown_connections 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.882+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1357742717 wait complete. 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.883+0000 7f6fd5bd7640 1 Processor -- start 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.883+0000 7f6fd5bd7640 1 -- start start 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fd5bd7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc004800 0x7f6fd00be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fd5bd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd01ee620 0x7f6fd01576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fd5bd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd00c0750 0x7f6fd00bee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fd5bd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00c7c30 con 0x7f6fd01ee620 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fd5bd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0155c10 con 0x7f6fd00c0750 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fd5bd7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0155d90 con 0x7f6fbc004800 2024-03-06T07:29:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd01ee620 0x7f6fd01576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd01ee620 0x7f6fd01576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37546/0 (socket says 172.21.15.26:37546) 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 -- 172.21.15.26:0/1247025716 learned_addr learned my addr 172.21.15.26:0/1247025716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fcf7fe640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc004800 0x7f6fd00be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fcffff640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd00c0750 0x7f6fd00bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 -- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc004800 msgr2=0x7f6fd00be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc004800 0x7f6fd00be4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 -- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd00c0750 msgr2=0x7f6fd00bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd00c0750 0x7f6fd00bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 -- 172.21.15.26:0/1247025716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00bfb70 con 0x7f6fd01ee620 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fcffff640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd00c0750 0x7f6fd00bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:19.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.884+0000 7f6fc7fff640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd01ee620 0x7f6fd01576e0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8034040 tx=0x7f6fb8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.885+0000 7f6fccff9640 1 -- 172.21.15.26:0/1247025716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8041070 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.885+0000 7f6fccff9640 1 -- 172.21.15.26:0/1247025716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb8005480 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.885+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd00bfe50 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.885+0000 7f6fccff9640 1 -- 172.21.15.26:0/1247025716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8007a30 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.885+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd01ede20 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.886+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd004f330 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.886+0000 7f6fccff9640 1 -- 172.21.15.26:0/1247025716 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fb8044070 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.886+0000 7f6fccff9640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fb006b500 0x7f6fb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.886+0000 7f6fccff9640 1 -- 172.21.15.26:0/1247025716 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6fb80abde0 con 0x7f6fd01ee620 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.887+0000 7f6fcf7fe640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fb006b500 0x7f6fb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.887+0000 7f6fcf7fe640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fb006b500 0x7f6fb006d9c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0001eb0 tx=0x7f6fc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:19.891+0000 7f6fccff9640 1 -- 172.21.15.26:0/1247025716 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6fb807ac80 con 0x7f6fd01ee620 2024-03-06T07:29:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.234+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fd00c8d50 con 0x7f6fd01ee620 2024-03-06T07:29:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.235+0000 7f6fccff9640 1 -- 172.21.15.26:0/1247025716 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f6fb807a580 con 0x7f6fd01ee620 2024-03-06T07:29:20.236 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:20.237 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":198,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":9799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:16.379626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 198 2024-03-06T07:29:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fb006b500 msgr2=0x7f6fb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fb006b500 0x7f6fb006d9c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0001eb0 tx=0x7f6fc0009040 comp rx=0 tx=0).stop 2024-03-06T07:29:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd01ee620 msgr2=0x7f6fd01576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd01ee620 0x7f6fd01576e0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8034040 tx=0x7f6fb8001120 comp rx=0 tx=0).stop 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 shutdown_connections 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6fb006b500 0x7f6fb006d9c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd00c0750 0x7f6fd00bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd01ee620 0x7f6fd01576e0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 --2- 172.21.15.26:0/1247025716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fbc004800 0x7f6fd00be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 >> 172.21.15.26:0/1247025716 conn(0x7f6fd00ac570 msgr2=0x7f6fd014d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 shutdown_connections 2024-03-06T07:29:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.237+0000 7f6fd5bd7640 1 -- 172.21.15.26:0/1247025716 wait complete. 2024-03-06T07:29:20.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-03-06T07:29:20.285 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-03-06T07:29:20.284+0000 7fe00ae95640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:29:20.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.354+0000 7f49ddc84640 1 Processor -- start 2024-03-06T07:29:20.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.354+0000 7f49ddc84640 1 -- start start 2024-03-06T07:29:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49ddc84640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49d806c240 0x7f49d8159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49ddc84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49ddc84640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49d806e140 0x7f49d8152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49ddc84640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d8159650 con 0x7f49d806c240 2024-03-06T07:29:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49ddc84640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d8155c00 con 0x7f49d806e140 2024-03-06T07:29:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49ddc84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8158c80 con 0x7f49d806dc80 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49d806c240 0x7f49d8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49d806c240 0x7f49d8159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37548/0 (socket says 172.21.15.26:37548) 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 -- 172.21.15.26:0/418150518 learned_addr learned my addr 172.21.15.26:0/418150518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d7fff640 1 --2- 172.21.15.26:0/418150518 >> v2:172.21.15.117:3300/0 conn(0x7f49d806e140 0x7f49d8152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49ceffd640 1 --2- 172.21.15.26:0/418150518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 -- 172.21.15.26:0/418150518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 msgr2=0x7f49d80c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 --2- 172.21.15.26:0/418150518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 -- 172.21.15.26:0/418150518 >> v2:172.21.15.117:3300/0 conn(0x7f49d806e140 msgr2=0x7f49d8152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 --2- 172.21.15.26:0/418150518 >> v2:172.21.15.117:3300/0 conn(0x7f49d806e140 0x7f49d8152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d77fe640 1 -- 172.21.15.26:0/418150518 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d8155e50 con 0x7f49d806c240 2024-03-06T07:29:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.355+0000 7f49d7fff640 1 --2- 172.21.15.26:0/418150518 >> v2:172.21.15.117:3300/0 conn(0x7f49d806e140 0x7f49d8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.356+0000 7f49d77fe640 1 --2- 172.21.15.26:0/418150518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d8159110 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f49d806ee80 tx=0x7f49c802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df86469a52f4edee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.356+0000 7f49d6ffd640 1 -- 172.21.15.26:0/418150518 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c802e070 con 0x7f49d806c240 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.356+0000 7f49d6ffd640 1 -- 172.21.15.26:0/418150518 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c8037070 con 0x7f49d806c240 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.356+0000 7f49d6ffd640 1 -- 172.21.15.26:0/418150518 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8031a00 con 0x7f49d806c240 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 -- 172.21.15.26:0/418150518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 msgr2=0x7f49d8159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 --2- 172.21.15.26:0/418150518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d8159110 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f49d806ee80 tx=0x7f49c802b880 comp rx=0 tx=0).stop 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 -- 172.21.15.26:0/418150518 shutdown_connections 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 --2- 172.21.15.26:0/418150518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 --2- 172.21.15.26:0/418150518 >> v2:172.21.15.117:3300/0 conn(0x7f49d806e140 0x7f49d8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 --2- 172.21.15.26:0/418150518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d8159110 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 -- 172.21.15.26:0/418150518 >> 172.21.15.26:0/418150518 conn(0x7f49d80c3b40 msgr2=0x7f49d80c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 -- 172.21.15.26:0/418150518 shutdown_connections 2024-03-06T07:29:20.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.357+0000 7f49ddc84640 1 -- 172.21.15.26:0/418150518 wait complete. 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.359+0000 7f49ddc84640 1 Processor -- start 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.359+0000 7f49ddc84640 1 -- start start 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ddc84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d81eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ddc84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ddc84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49d806e140 0x7f49d8072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ddc84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d806f0e0 con 0x7f49d806c240 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ddc84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80c1f80 con 0x7f49d806e140 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ddc84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d806f260 con 0x7f49d806dc80 2024-03-06T07:29:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d81eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d81eb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37550/0 (socket says 172.21.15.26:37550) 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 -- 172.21.15.26:0/121420566 learned_addr learned my addr 172.21.15.26:0/121420566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d7fff640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49d806e140 0x7f49d8072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ceffd640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 -- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 msgr2=0x7f49d80c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 -- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49d806e140 msgr2=0x7f49d8072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49d806e140 0x7f49d8072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49d77fe640 1 -- 172.21.15.26:0/121420566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d8151820 con 0x7f49d806c240 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.360+0000 7f49ceffd640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.361+0000 7f49d77fe640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d81eb4c0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f49c8000c00 tx=0x7f49c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.361+0000 7f49d4ff9640 1 -- 172.21.15.26:0/121420566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8032880 con 0x7f49d806c240 2024-03-06T07:29:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.361+0000 7f49d4ff9640 1 -- 172.21.15.26:0/121420566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c8032a40 con 0x7f49d806c240 2024-03-06T07:29:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.361+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d8151b00 con 0x7f49d806c240 2024-03-06T07:29:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.361+0000 7f49d4ff9640 1 -- 172.21.15.26:0/121420566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8032e30 con 0x7f49d806c240 2024-03-06T07:29:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.361+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d80c5c10 con 0x7f49d806c240 2024-03-06T07:29:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.362+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d806c620 con 0x7f49d806c240 2024-03-06T07:29:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.362+0000 7f49d4ff9640 1 -- 172.21.15.26:0/121420566 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49c80453f0 con 0x7f49d806c240 2024-03-06T07:29:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.363+0000 7f49d4ff9640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49b806b4d0 0x7f49b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.363+0000 7f49d4ff9640 1 -- 172.21.15.26:0/121420566 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49c80ac070 con 0x7f49d806c240 2024-03-06T07:29:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.363+0000 7f49ceffd640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49b806b4d0 0x7f49b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.364+0000 7f49ceffd640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49b806b4d0 0x7f49b806d990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f49d814aa90 tx=0x7f49c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.367+0000 7f49d4ff9640 1 -- 172.21.15.26:0/121420566 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49d806c620 con 0x7f49d806c240 2024-03-06T07:29:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:20.587+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f49d8153320 con 0x7f49d806c240 2024-03-06T07:29:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.050+0000 7f49d4ff9640 1 -- 172.21.15.26:0/121420566 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v199) v1 ==== 105+0+0 (secure 0 0 0) 0x7f49d8153320 con 0x7f49d806c240 2024-03-06T07:29:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49b806b4d0 msgr2=0x7f49b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49b806b4d0 0x7f49b806d990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f49d814aa90 tx=0x7f49c0008040 comp rx=0 tx=0).stop 2024-03-06T07:29:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 msgr2=0x7f49d81eb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d81eb4c0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f49c8000c00 tx=0x7f49c80046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 shutdown_connections 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f49b806b4d0 0x7f49b806d990 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49d806e140 0x7f49d8072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49d806dc80 0x7f49d80c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 --2- 172.21.15.26:0/121420566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d806c240 0x7f49d81eb4c0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.052+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 >> 172.21.15.26:0/121420566 conn(0x7f49d80c3b40 msgr2=0x7f49d8149c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.053+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 shutdown_connections 2024-03-06T07:29:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.053+0000 7f49ddc84640 1 -- 172.21.15.26:0/121420566 wait complete. 2024-03-06T07:29:21.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.212+0000 7f352fb5e640 1 Processor -- start 2024-03-06T07:29:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.212+0000 7f352fb5e640 1 -- start start 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.213+0000 7f352fb5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 0x7f3528155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.213+0000 7f352fb5e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3528158b60 0x7f3528158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.213+0000 7f352fb5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35280c73d0 con 0x7f352814f360 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.213+0000 7f352fb5e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35280c8dd0 con 0x7f3528158b60 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.213+0000 7f352fb5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35280be2e0 con 0x7f35280c88c0 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352d8fa640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f352814f360 0x7f35280c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39164/0 (socket says 172.21.15.26:39164) 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352d8fa640 1 -- 172.21.15.26:0/1028376675 learned_addr learned my addr 172.21.15.26:0/1028376675 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352d0f9640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 0x7f3528155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352e0fb640 1 --2- 172.21.15.26:0/1028376675 >> v2:172.21.15.117:3300/0 conn(0x7f3528158b60 0x7f3528158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974356487 0 0) 0x7f35280c73d0 con 0x7f352814f360 2024-03-06T07:29:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3514003520 con 0x7f352814f360 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633155704 0 0) 0x7f3514003520 con 0x7f352814f360 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 msgr2=0x7f3528155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 0x7f3528155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 >> v2:172.21.15.117:3300/0 conn(0x7f3528158b60 msgr2=0x7f3528158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 --2- 172.21.15.26:0/1028376675 >> v2:172.21.15.117:3300/0 conn(0x7f3528158b60 0x7f3528158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280c7670 con 0x7f352814f360 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f351c001670 con 0x7f352814f360 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 >> v1:172.21.15.26:6789/0 conn(0x7f352814f360 legacy=0x7f35280c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 0x7f35140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f3514004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3514004d40 0x7f35140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35280be2e0 con 0x7f3514003f30 2024-03-06T07:29:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351400b9c0 con 0x7f3514004d40 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351400bb90 con 0x7f35280c88c0 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352d0f9640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f3514004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352d8fa640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 0x7f35140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.214+0000 7f352e0fb640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3514004d40 0x7f35140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.215+0000 7f352d0f9640 1 -- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 msgr2=0x7f35140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.215+0000 7f352d0f9640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280c88c0 0x7f35140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.215+0000 7f352d0f9640 1 -- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3514004d40 msgr2=0x7f35140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.215+0000 7f352d0f9640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3514004d40 0x7f35140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.215+0000 7f352d0f9640 1 -- 172.21.15.26:0/1028376675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514003520 con 0x7f3514003f30 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.215+0000 7f352d0f9640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f3514004800 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f3524009ac0 tx=0x7f3514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2943ac758f08d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524039070 con 0x7f3514003f30 2024-03-06T07:29:21.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352402c680 con 0x7f3514003f30 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352c8f8640 1 -- 172.21.15.26:0/1028376675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524009690 con 0x7f3514003f30 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352fb5e640 1 -- 172.21.15.26:0/1028376675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 msgr2=0x7f3514004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352fb5e640 1 --2- 172.21.15.26:0/1028376675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f3514004800 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f3524009ac0 tx=0x7f3514005830 comp rx=0 tx=0).stop 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352e0fb640 1 -- 172.21.15.26:0/1028376675 reap_dead start 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352fb5e640 1 -- 172.21.15.26:0/1028376675 shutdown_connections 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352fb5e640 1 -- 172.21.15.26:0/1028376675 >> 172.21.15.26:0/1028376675 conn(0x7f35280ac570 msgr2=0x7f35280ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352fb5e640 1 -- 172.21.15.26:0/1028376675 shutdown_connections 2024-03-06T07:29:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.216+0000 7f352fb5e640 1 -- 172.21.15.26:0/1028376675 wait complete. 2024-03-06T07:29:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.217+0000 7f352fb5e640 1 Processor -- start 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.217+0000 7f352fb5e640 1 -- start start 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352fb5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f352814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352fb5e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280bf7a0 0x7f35280bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352fb5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35281ece20 0x7f35281ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352fb5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35280c5a50 con 0x7f3514003f30 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352fb5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35280c7c30 con 0x7f35281ece20 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352fb5e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35280c5bd0 con 0x7f35280bf7a0 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f352814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f352814c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37580/0 (socket says 172.21.15.26:37580) 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 -- 172.21.15.26:0/3354358118 learned_addr learned my addr 172.21.15.26:0/3354358118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d0f9640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280bf7a0 0x7f35280bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352e0fb640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35281ece20 0x7f35281ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 -- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280bf7a0 msgr2=0x7f35280bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280bf7a0 0x7f35280bfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 -- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35281ece20 msgr2=0x7f35281ed2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35281ece20 0x7f35281ed2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.218+0000 7f352d8fa640 1 -- 172.21.15.26:0/3354358118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280c0e90 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.219+0000 7f352d8fa640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f352814c1c0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f351c007310 tx=0x7f351c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.219+0000 7f351a7fc640 1 -- 172.21.15.26:0/3354358118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c00f960 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.219+0000 7f351a7fc640 1 -- 172.21.15.26:0/3354358118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c018070 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.219+0000 7f351a7fc640 1 -- 172.21.15.26:0/3354358118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c00fd60 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.219+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35280c08d0 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.219+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35280befa0 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.220+0000 7f351a7fc640 1 -- 172.21.15.26:0/3354358118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f351c01d460 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.220+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352804f330 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.220+0000 7f351a7fc640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f350c06b500 0x7f350c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.221+0000 7f351a7fc640 1 -- 172.21.15.26:0/3354358118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f351c08a2a0 con 0x7f3514003f30 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.221+0000 7f352d0f9640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f350c06b500 0x7f350c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.221+0000 7f352d0f9640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f350c06b500 0x7f350c06d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3524036000 tx=0x7f3524030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:21.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.225+0000 7f351a7fc640 1 -- 172.21.15.26:0/3354358118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f351c0589d0 con 0x7f3514003f30 2024-03-06T07:29:21.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.602+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f352806cda0 con 0x7f3514003f30 2024-03-06T07:29:21.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.602+0000 7f351a7fc640 1 -- 172.21.15.26:0/3354358118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f351c08a0c0 con 0x7f3514003f30 2024-03-06T07:29:21.604 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:21.604 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":200,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:21.052835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:creating","state_seq":1,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:21.605 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 200 2024-03-06T07:29:21.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f350c06b500 msgr2=0x7f350c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f350c06b500 0x7f350c06d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3524036000 tx=0x7f3524030000 comp rx=0 tx=0).stop 2024-03-06T07:29:21.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 msgr2=0x7f352814c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f352814c1c0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f351c007310 tx=0x7f351c0077d0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 shutdown_connections 2024-03-06T07:29:21.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f350c06b500 0x7f350c06d9c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35281ece20 0x7f35281ed2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35280bf7a0 0x7f35280bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 --2- 172.21.15.26:0/3354358118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3514003f30 0x7f352814c1c0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 >> 172.21.15.26:0/3354358118 conn(0x7f35280ac570 msgr2=0x7f352806c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:21.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 shutdown_connections 2024-03-06T07:29:21.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:21.604+0000 7f352fb5e640 1 -- 172.21.15.26:0/3354358118 wait complete. 2024-03-06T07:29:22.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 200, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:21.052835+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:22.019 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:creating 2024-03-06T07:29:23.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.123+0000 7f514b8a3640 1 Processor -- start 2024-03-06T07:29:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.123+0000 7f514b8a3640 1 -- start start 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.123+0000 7f514b8a3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f514414f360 0x7f5144155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.123+0000 7f514b8a3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51440c88c0 0x7f51440c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.123+0000 7f514b8a3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f5144158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f514b8a3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51440c8dd0 con 0x7f51440c88c0 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f514b8a3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51440c73d0 con 0x7f514414f360 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f514b8a3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440be2e0 con 0x7f51440c5880 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51440c88c0 0x7f51440c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51440c88c0 0x7f51440c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37584/0 (socket says 172.21.15.26:37584) 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 -- 172.21.15.26:0/3322727198 learned_addr learned my addr 172.21.15.26:0/3322727198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f514963f640 1 --2- 172.21.15.26:0/3322727198 >> v2:172.21.15.117:3300/0 conn(0x7f514414f360 0x7f5144155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5149e40640 1 --2- 172.21.15.26:0/3322727198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f5144158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 -- 172.21.15.26:0/3322727198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 msgr2=0x7f5144158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 --2- 172.21.15.26:0/3322727198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f5144158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 -- 172.21.15.26:0/3322727198 >> v2:172.21.15.117:3300/0 conn(0x7f514414f360 msgr2=0x7f5144155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 --2- 172.21.15.26:0/3322727198 >> v2:172.21.15.117:3300/0 conn(0x7f514414f360 0x7f5144155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.124+0000 7f5148e3e640 1 -- 172.21.15.26:0/3322727198 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440c7670 con 0x7f51440c88c0 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f5148e3e640 1 --2- 172.21.15.26:0/3322727198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51440c88c0 0x7f51440c5340 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f514408c840 tx=0x7f513402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef736cc7482eb37 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f513bfff640 1 -- 172.21.15.26:0/3322727198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513402e070 con 0x7f51440c88c0 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f513bfff640 1 -- 172.21.15.26:0/3322727198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134037070 con 0x7f51440c88c0 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f513bfff640 1 -- 172.21.15.26:0/3322727198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134031a00 con 0x7f51440c88c0 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 -- 172.21.15.26:0/3322727198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51440c88c0 msgr2=0x7f51440c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3322727198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51440c88c0 0x7f51440c5340 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f514408c840 tx=0x7f513402b880 comp rx=0 tx=0).stop 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 -- 172.21.15.26:0/3322727198 shutdown_connections 2024-03-06T07:29:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3322727198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f5144158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3322727198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51440c88c0 0x7f51440c5340 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3322727198 >> v2:172.21.15.117:3300/0 conn(0x7f514414f360 0x7f5144155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 -- 172.21.15.26:0/3322727198 >> 172.21.15.26:0/3322727198 conn(0x7f51440ac570 msgr2=0x7f51440ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.125+0000 7f514b8a3640 1 -- 172.21.15.26:0/3322727198 shutdown_connections 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.126+0000 7f514b8a3640 1 -- 172.21.15.26:0/3322727198 wait complete. 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 Processor -- start 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 -- start start 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f51440c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51440c88c0 0x7f51440c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f514414f360 0x7f51440c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441590a0 con 0x7f514414f360 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440c7c30 con 0x7f51440c88c0 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514b8a3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5144159220 con 0x7f51440c5880 2024-03-06T07:29:23.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f5149e40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f514414f360 0x7f51440c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f5149e40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f514414f360 0x7f51440c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37596/0 (socket says 172.21.15.26:37596) 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f5149e40640 1 -- 172.21.15.26:0/3601137088 learned_addr learned my addr 172.21.15.26:0/3601137088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f514963f640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f51440c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.127+0000 7f5148e3e640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51440c88c0 0x7f51440c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5149e40640 1 -- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 msgr2=0x7f51440c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5149e40640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f51440c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5149e40640 1 -- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51440c88c0 msgr2=0x7f51440c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5149e40640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51440c88c0 0x7f51440c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5149e40640 1 -- 172.21.15.26:0/3601137088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440be740 con 0x7f514414f360 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5149e40640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f514414f360 0x7f51440c9ea0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eed0 tx=0x7f513c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5139ffb640 1 -- 172.21.15.26:0/3601137088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c018020 con 0x7f514414f360 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5139ffb640 1 -- 172.21.15.26:0/3601137088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c014400 con 0x7f514414f360 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51441ee880 con 0x7f514414f360 2024-03-06T07:29:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f5139ffb640 1 -- 172.21.15.26:0/3601137088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0147f0 con 0x7f514414f360 2024-03-06T07:29:23.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.128+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51441eeb60 con 0x7f514414f360 2024-03-06T07:29:23.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.129+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51441ee6b0 con 0x7f514414f360 2024-03-06T07:29:23.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.130+0000 7f5139ffb640 1 -- 172.21.15.26:0/3601137088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f513c005d10 con 0x7f514414f360 2024-03-06T07:29:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.130+0000 7f5139ffb640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512806b500 0x7f512806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.130+0000 7f5139ffb640 1 -- 172.21.15.26:0/3601137088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f513c08a870 con 0x7f514414f360 2024-03-06T07:29:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.130+0000 7f514963f640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512806b500 0x7f512806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.131+0000 7f514963f640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512806b500 0x7f512806d9c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5140009aa0 tx=0x7f5140009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.134+0000 7f5139ffb640 1 -- 172.21.15.26:0/3601137088 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f513c022070 con 0x7f514414f360 2024-03-06T07:29:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.350+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f514404f330 con 0x7f514414f360 2024-03-06T07:29:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.351+0000 7f5139ffb640 1 -- 172.21.15.26:0/3601137088 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 201 v201) v1 ==== 95+0+5014 (secure 0 0 0) 0x7f513c058fa0 con 0x7f514414f360 2024-03-06T07:29:23.353 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:23.353 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":201,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:22.056820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[9799,10737]},"id":16}]} 2024-03-06T07:29:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 201 2024-03-06T07:29:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512806b500 msgr2=0x7f512806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512806b500 0x7f512806d9c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5140009aa0 tx=0x7f5140009190 comp rx=0 tx=0).stop 2024-03-06T07:29:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f514414f360 msgr2=0x7f51440c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f514414f360 0x7f51440c9ea0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eed0 tx=0x7f513c00b480 comp rx=0 tx=0).stop 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 shutdown_connections 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f512806b500 0x7f512806d9c0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f514414f360 0x7f51440c9ea0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51440c88c0 0x7f51440c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 --2- 172.21.15.26:0/3601137088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51440c5880 0x7f51440c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 >> 172.21.15.26:0/3601137088 conn(0x7f51440ac570 msgr2=0x7f51440cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 shutdown_connections 2024-03-06T07:29:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.353+0000 7f514b8a3640 1 -- 172.21.15.26:0/3601137088 wait complete. 2024-03-06T07:29:23.768 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 201, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:22.056820+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [9799, 10737]} 2024-03-06T07:29:23.768 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-03-06T07:29:23.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:29:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.869+0000 7f5611577640 1 Processor -- start 2024-03-06T07:29:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.869+0000 7f5611577640 1 -- start start 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5611577640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f560c0c69f0 0x7f560c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5611577640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f560c0c8950 0x7f560c0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5611577640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0ad2b0 0x7f560c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5611577640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c155440 con 0x7f560c0c8950 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5611577640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c155130 con 0x7f560c0c69f0 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5611577640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c5d50 con 0x7f560c0ad2b0 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f560c0c8950 0x7f560c0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f560c0c8950 0x7f560c0acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37612/0 (socket says 172.21.15.26:37612) 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 -- 172.21.15.26:0/2511005107 learned_addr learned my addr 172.21.15.26:0/2511005107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f560affd640 1 --2- 172.21.15.26:0/2511005107 >> v2:172.21.15.117:3300/0 conn(0x7f560c0c69f0 0x7f560c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f560b7fe640 1 --2- 172.21.15.26:0/2511005107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0ad2b0 0x7f560c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 -- 172.21.15.26:0/2511005107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0ad2b0 msgr2=0x7f560c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 --2- 172.21.15.26:0/2511005107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0ad2b0 0x7f560c1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 -- 172.21.15.26:0/2511005107 >> v2:172.21.15.117:3300/0 conn(0x7f560c0c69f0 msgr2=0x7f560c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 --2- 172.21.15.26:0/2511005107 >> v2:172.21.15.117:3300/0 conn(0x7f560c0c69f0 0x7f560c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.870+0000 7f5603fff640 1 -- 172.21.15.26:0/2511005107 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0c6030 con 0x7f560c0c8950 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.871+0000 7f5603fff640 1 --2- 172.21.15.26:0/2511005107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c0acd70 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f560c0aa430 tx=0x7f55f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ef28da2d0cba96 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.871+0000 7f560a7fc640 1 -- 172.21.15.26:0/2511005107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f402e070 con 0x7f560c0c8950 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.871+0000 7f560a7fc640 1 -- 172.21.15.26:0/2511005107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f4037070 con 0x7f560c0c8950 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.871+0000 7f560a7fc640 1 -- 172.21.15.26:0/2511005107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4031a00 con 0x7f560c0c8950 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 -- 172.21.15.26:0/2511005107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 msgr2=0x7f560c0acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 --2- 172.21.15.26:0/2511005107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c0acd70 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f560c0aa430 tx=0x7f55f402b880 comp rx=0 tx=0).stop 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 -- 172.21.15.26:0/2511005107 shutdown_connections 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 --2- 172.21.15.26:0/2511005107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0ad2b0 0x7f560c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 --2- 172.21.15.26:0/2511005107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c0acd70 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 --2- 172.21.15.26:0/2511005107 >> v2:172.21.15.117:3300/0 conn(0x7f560c0c69f0 0x7f560c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 -- 172.21.15.26:0/2511005107 >> 172.21.15.26:0/2511005107 conn(0x7f560c0cae70 msgr2=0x7f560c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 -- 172.21.15.26:0/2511005107 shutdown_connections 2024-03-06T07:29:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.872+0000 7f5611577640 1 -- 172.21.15.26:0/2511005107 wait complete. 2024-03-06T07:29:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.873+0000 7f5611577640 1 Processor -- start 2024-03-06T07:29:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.874+0000 7f5611577640 1 -- start start 2024-03-06T07:29:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5611577640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f560c0ad2b0 0x7f560c06c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5611577640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0c69f0 0x7f560c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5611577640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c14c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5611577640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c152ba0 con 0x7f560c0c8950 2024-03-06T07:29:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5611577640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c152a20 con 0x7f560c0ad2b0 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5611577640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c58e0 con 0x7f560c0c69f0 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f560c0ad2b0 0x7f560c06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c14c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f560c0ad2b0 0x7f560c06c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52334/0 (socket says 172.21.15.26:52334) 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560affd640 1 -- 172.21.15.26:0/2903517429 learned_addr learned my addr 172.21.15.26:0/2903517429 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5603fff640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0c69f0 0x7f560c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560b7fe640 1 -- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0c69f0 msgr2=0x7f560c14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560b7fe640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0c69f0 0x7f560c14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560b7fe640 1 -- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f560c0ad2b0 msgr2=0x7f560c06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560b7fe640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f560c0ad2b0 0x7f560c06c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560b7fe640 1 -- 172.21.15.26:0/2903517429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c06cab0 con 0x7f560c0c8950 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560affd640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f560c0ad2b0 0x7f560c06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f560b7fe640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c14c720 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f560400eea0 tx=0x7f560400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f56037fe640 1 -- 172.21.15.26:0/2903517429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604018020 con 0x7f560c0c8950 2024-03-06T07:29:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f56037fe640 1 -- 172.21.15.26:0/2903517429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5604014400 con 0x7f560c0c8950 2024-03-06T07:29:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c159180 con 0x7f560c0c8950 2024-03-06T07:29:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.875+0000 7f56037fe640 1 -- 172.21.15.26:0/2903517429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604009e60 con 0x7f560c0c8950 2024-03-06T07:29:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.876+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c0c2210 con 0x7f560c0c8950 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.877+0000 7f56037fe640 1 -- 172.21.15.26:0/2903517429 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5604005d10 con 0x7f560c0c8950 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.878+0000 7f56037fe640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f55ec06b4d0 0x7f55ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.878+0000 7f56037fe640 1 -- 172.21.15.26:0/2903517429 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f560408ab10 con 0x7f560c0c8950 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.878+0000 7f560affd640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f55ec06b4d0 0x7f55ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.878+0000 7f560affd640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f55ec06b4d0 0x7f55ec06d990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0112c0 tx=0x7f55fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 Processor -- start 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 -- start start 2024-03-06T07:29:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c56a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c56f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c574b0 0x55b6d7c53680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7bbd6b0 con 0x55b6d7c574b0 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7b66a60 con 0x55b6d7c59330 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.879+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7bb9750 con 0x55b6d7c58f50 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c574b0 0x55b6d7c53680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560affd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c56a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c574b0 0x55b6d7c53680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37620/0 (socket says 172.21.15.26:37620) 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 -- 172.21.15.26:0/1023902346 learned_addr learned my addr 172.21.15.26:0/1023902346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f5603fff640 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c56f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 -- 172.21.15.26:0/1023902346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 msgr2=0x55b6d7c56a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c56a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 -- 172.21.15.26:0/1023902346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 msgr2=0x55b6d7c56f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c56f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 -- 172.21.15.26:0/1023902346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d7b84f30 con 0x55b6d7c574b0 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f560b7fe640 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c574b0 0x55b6d7c53680 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f56040591d0 tx=0x7f5604058eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbc1328d62ef8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f5600ff9640 1 -- 172.21.15.26:0/1023902346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604087070 con 0x55b6d7c574b0 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f5600ff9640 1 -- 172.21.15.26:0/1023902346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56040558b0 con 0x55b6d7c574b0 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.880+0000 7f5600ff9640 1 -- 172.21.15.26:0/1023902346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604058890 con 0x55b6d7c574b0 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 -- 172.21.15.26:0/1023902346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c574b0 msgr2=0x55b6d7c53680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c574b0 0x55b6d7c53680 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f56040591d0 tx=0x7f5604058eb0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 -- 172.21.15.26:0/1023902346 shutdown_connections 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c574b0 0x55b6d7c53680 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 --2- 172.21.15.26:0/1023902346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c56a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 -- 172.21.15.26:0/1023902346 >> 172.21.15.26:0/1023902346 conn(0x55b6d7c557d0 msgr2=0x55b6d7c7a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 -- 172.21.15.26:0/1023902346 shutdown_connections 2024-03-06T07:29:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 -- 172.21.15.26:0/1023902346 wait complete. 2024-03-06T07:29:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 -- start start 2024-03-06T07:29:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.881+0000 7f5614236740 1 Processor -- start 2024-03-06T07:29:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c79a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c7eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5614236740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c80f20 0x55b6d7c919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5614236740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7c7b230 con 0x55b6d7c80f20 2024-03-06T07:29:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7b79ea0 con 0x55b6d7c59330 2024-03-06T07:29:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7c7ac70 con 0x55b6d7c58f50 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5614236740 10 client.0 _ll_register_callbacks cb 0x55b6d7b79870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f560b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c80f20 0x55b6d7c919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f560b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c80f20 0x55b6d7c919c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37628/0 (socket says 172.21.15.26:37628) 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f560b7fe640 1 -- 172.21.15.26:0/3675419467 learned_addr learned my addr 172.21.15.26:0/3675419467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f560affd640 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c79a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.882+0000 7f5603fff640 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c7eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f560b7fe640 1 -- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 msgr2=0x55b6d7c79a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f560b7fe640 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c79a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f560b7fe640 1 -- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 msgr2=0x55b6d7c7eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f560b7fe640 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c7eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f560b7fe640 1 -- 172.21.15.26:0/3675419467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6d7b7a270 con 0x55b6d7c80f20 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f560b7fe640 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c80f20 0x55b6d7c919c0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f5604055650 tx=0x7f5604088e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee52bed93bb9945 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f55e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:29:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f55e9ffb640 1 -- 172.21.15.26:0/3675419467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604074070 con 0x55b6d7c80f20 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f55e9ffb640 1 -- 172.21.15.26:0/3675419467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f560405f470 con 0x55b6d7c80f20 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56040558b0 con 0x55b6d7c80f20 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f55e9ffb640 1 -- 172.21.15.26:0/3675419467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5604080050 con 0x55b6d7c80f20 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 10 client.10923 fetch_fsmap learned FSMap version 201 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=201,monmap=2+}) v3 -- 0x55b6d78d22d0 con 0x55b6d7c80f20 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f55e9ffb640 1 -- 172.21.15.26:0/3675419467 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56040558b0 con 0x55b6d7c80f20 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f55e9ffb640 1 -- 172.21.15.26:0/3675419467 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 201) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f5604054f90 con 0x55b6d7c80f20 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 10 client.10923 fetch_fsmap finished waiting for FSMap version 201 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 10 client.10923 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 20 client.10923 populate_metadata read hostname 'smithi026' 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7ca9310 0x55b6d7ca96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 4 client.10923 mds_command: new command op to 9799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.883+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 --> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6d7bcabf0 con 0x55b6d7ca9310 2024-03-06T07:29:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.884+0000 7f560affd640 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7ca9310 0x55b6d7ca96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.884+0000 7f55e9ffb640 10 client.10923 ms_handle_connect on v2:172.21.15.117:6835/3331811951 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.884+0000 7f560affd640 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7ca9310 0x55b6d7ca96f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2cdc70e2c57d3a49 server_cookie=d5c3768050276e67 in_seq=0 out_seq=0 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f55e9ffb640 1 -- 172.21.15.26:0/3675419467 <== mds.1 v2:172.21.15.117:6835/3331811951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b6d7bcabf0 con 0x55b6d7ca9310 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f55e9ffb640 10 client.10923 handle_command_reply: tid=0 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 client.10923 shutdown 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c80f20 msgr2=0x55b6d7c919c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c80f20 0x55b6d7c919c0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f5604055650 tx=0x7f5604088e10 comp rx=0 tx=0).stop 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 shutdown_connections 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c80f20 0x55b6d7c919c0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7ca9310 0x55b6d7ca96f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c59330 0x55b6d7c7eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 --2- 172.21.15.26:0/3675419467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c79a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f55e9ffb640 0 client.10923 ms_handle_reset on v2:172.21.15.117:6835/3331811951 2024-03-06T07:29:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 >> 172.21.15.26:0/3675419467 conn(0x55b6d7c574b0 msgr2=0x55b6d7c52510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 shutdown_connections 2024-03-06T07:29:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 1 -- 172.21.15.26:0/3675419467 wait complete. 2024-03-06T07:29:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.885+0000 7f5614236740 20 client.10923 trim_cache size 0 max 16384 2024-03-06T07:29:23.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 Processor -- start 2024-03-06T07:29:23.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 -- start start 2024-03-06T07:29:23.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c7f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7c76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1f7b90 con 0x55b6d7c59330 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1eeb30 con 0x55b6d7c80f20 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1f6680 con 0x55b6d7c58f50 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5603fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5603fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37640/0 (socket says 172.21.15.26:37640) 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f5603fff640 1 -- 172.21.15.26:0/75434229 learned_addr learned my addr 172.21.15.26:0/75434229 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f560affd640 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c7f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.902+0000 7f560b7fe640 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7c76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5603fff640 1 -- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 msgr2=0x55b6d7c7f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5603fff640 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c7f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5603fff640 1 -- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 msgr2=0x55b6d7c76ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5603fff640 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7c76ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5603fff640 1 -- 172.21.15.26:0/75434229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d799c6b0 con 0x55b6d7c59330 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5603fff640 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7feb0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f55f4034040 tx=0x7f55f4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffb88aff5a8fbed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f55ebfff640 1 -- 172.21.15.26:0/75434229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4031a20 con 0x55b6d7c59330 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f55ebfff640 1 -- 172.21.15.26:0/75434229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f4037040 con 0x55b6d7c59330 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f55ebfff640 1 -- 172.21.15.26:0/75434229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4031de0 con 0x55b6d7c59330 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5614236740 1 -- 172.21.15.26:0/75434229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 msgr2=0x55b6d7c7feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5614236740 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7feb0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f55f4034040 tx=0x7f55f4030000 comp rx=0 tx=0).stop 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5614236740 1 -- 172.21.15.26:0/75434229 shutdown_connections 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5614236740 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7c76ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5614236740 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7feb0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5614236740 1 --2- 172.21.15.26:0/75434229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7c7f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.903+0000 7f5614236740 1 -- 172.21.15.26:0/75434229 >> 172.21.15.26:0/75434229 conn(0x55b6d7c574b0 msgr2=0x55b6d7c79810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.904+0000 7f5614236740 1 -- 172.21.15.26:0/75434229 shutdown_connections 2024-03-06T07:29:23.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.904+0000 7f5614236740 1 -- 172.21.15.26:0/75434229 wait complete. 2024-03-06T07:29:23.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.904+0000 7f5614236740 1 -- start start 2024-03-06T07:29:23.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.904+0000 7f5614236740 1 Processor -- start 2024-03-06T07:29:23.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7d05310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7cff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7c7b230 con 0x55b6d7c59330 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7c7ac70 con 0x55b6d7c80f20 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7b79ea0 con 0x55b6d7c58f50 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 10 client.0 _ll_register_callbacks cb 0x7f55fc011f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37646/0 (socket says 172.21.15.26:37646) 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 -- 172.21.15.26:0/2330470156 learned_addr learned my addr 172.21.15.26:0/2330470156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f560affd640 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7d05310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f560b7fe640 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7cff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 -- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 msgr2=0x55b6d7d05310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7d05310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 -- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 msgr2=0x55b6d7cff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7cff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 -- 172.21.15.26:0/2330470156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6d7c7cd00 con 0x55b6d7c59330 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5603fff640 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7fc50 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009960 tx=0x7f55f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01a88c87badebe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f55e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f55e9ffb640 1 -- 172.21.15.26:0/2330470156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4053070 con 0x55b6d7c59330 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f55e9ffb640 1 -- 172.21.15.26:0/2330470156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55f4031a20 con 0x55b6d7c59330 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.905+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55f4037040 con 0x55b6d7c59330 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f55e9ffb640 1 -- 172.21.15.26:0/2330470156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55f405e050 con 0x55b6d7c59330 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 10 client.10929 fetch_fsmap learned FSMap version 201 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=201,monmap=2+}) v3 -- 0x55b6d7d09060 con 0x55b6d7c59330 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f55e9ffb640 1 -- 172.21.15.26:0/2330470156 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f4037040 con 0x55b6d7c59330 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f55e9ffb640 1 -- 172.21.15.26:0/2330470156 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 201) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f55f4061020 con 0x55b6d7c59330 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 10 client.10929 fetch_fsmap finished waiting for FSMap version 201 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 10 client.10929 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 20 client.10929 populate_metadata read hostname 'smithi026' 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7cff5f0 0x55b6d7c985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 4 client.10929 mds_command: new command op to 9799 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 --> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b6d7c7ca20 con 0x55b6d7cff5f0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f560affd640 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7cff5f0 0x55b6d7c985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f560affd640 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7cff5f0 0x55b6d7c985a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d62b69a31f9e5e4a server_cookie=714500280d083ffe in_seq=0 out_seq=0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.906+0000 7f55e9ffb640 10 client.10929 ms_handle_connect on v2:172.21.15.117:6835/3331811951 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f55e9ffb640 1 -- 172.21.15.26:0/2330470156 <== mds.1 v2:172.21.15.117:6835/3331811951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b6d7c7ca20 con 0x55b6d7cff5f0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f55e9ffb640 10 client.10929 handle_command_reply: tid=0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 client.10929 shutdown 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 msgr2=0x55b6d7c7fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7fc50 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009960 tx=0x7f55f4004730 comp rx=0 tx=0).stop 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 shutdown_connections 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:6835/3331811951,v1:172.21.15.117:6837/3331811951] conn(0x55b6d7cff5f0 0x55b6d7c985a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6d7c80f20 0x55b6d7cff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f55e9ffb640 0 client.10929 ms_handle_reset on v2:172.21.15.117:6835/3331811951 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7c59330 0x55b6d7c7fc50 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 --2- 172.21.15.26:0/2330470156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6d7c58f50 0x55b6d7d05310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 >> 172.21.15.26:0/2330470156 conn(0x55b6d7c574b0 msgr2=0x55b6d7c91980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 shutdown_connections 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 1 -- 172.21.15.26:0/2330470156 wait complete. 2024-03-06T07:29:23.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5614236740 20 client.10929 trim_cache size 0 max 16384 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9799, 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8498932340000001, 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 201, 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 211, 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 211, 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 13.97934191 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:29:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f55ec06b4d0 msgr2=0x7f55ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f55ec06b4d0 0x7f55ec06d990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0112c0 tx=0x7f55fc009190 comp rx=0 tx=0).stop 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 msgr2=0x7f560c14c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c14c720 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f560400eea0 tx=0x7f560400ef90 comp rx=0 tx=0).stop 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 shutdown_connections 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f55ec06b4d0 0x7f55ec06d990 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f560c0c8950 0x7f560c14c720 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f560c0c69f0 0x7f560c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 --2- 172.21.15.26:0/2903517429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f560c0ad2b0 0x7f560c06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 >> 172.21.15.26:0/2903517429 conn(0x7f560c0cae70 msgr2=0x7f560c0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.907+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 shutdown_connections 2024-03-06T07:29:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:23.908+0000 7f5611577640 1 -- 172.21.15.26:0/2903517429 wait complete. 2024-03-06T07:29:23.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:29:24.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.020+0000 7f656c815640 1 Processor -- start 2024-03-06T07:29:24.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.020+0000 7f656c815640 1 -- start start 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.021+0000 7f656c815640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6568152b40 0x7f65680c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.021+0000 7f656c815640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f65681551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.021+0000 7f656c815640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65680733f0 con 0x7f65680a7ff0 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.021+0000 7f656c815640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6568072d70 con 0x7f6568152b40 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.021+0000 7f656c815640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65680c82c0 con 0x7f65680724d0 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.021+0000 7f656659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6568152b40 0x7f65680c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.021+0000 7f656659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6568152b40 0x7f65680c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37652/0 (socket says 172.21.15.26:37652) 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f656659c640 1 -- 172.21.15.26:0/1923429760 learned_addr learned my addr 172.21.15.26:0/1923429760 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f6565d9b640 1 --2- 172.21.15.26:0/1923429760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f65681551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f6565d9b640 1 -- 172.21.15.26:0/1923429760 >> v2:172.21.15.26:3300/0 conn(0x7f6568152b40 msgr2=0x7f65680c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f6565d9b640 1 --2- 172.21.15.26:0/1923429760 >> v2:172.21.15.26:3300/0 conn(0x7f6568152b40 0x7f65680c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f6565d9b640 1 -- 172.21.15.26:0/1923429760 >> v1:172.21.15.117:6789/0 conn(0x7f65680a7ff0 legacy=0x7f6568155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f656559a640 1 -- 172.21.15.26:0/1923429760 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235614915 0 0) 0x7f65680733f0 con 0x7f65680a7ff0 2024-03-06T07:29:24.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f6565d9b640 1 -- 172.21.15.26:0/1923429760 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656806fd30 con 0x7f65680724d0 2024-03-06T07:29:24.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.022+0000 7f656659c640 1 --2- 172.21.15.26:0/1923429760 >> v2:172.21.15.26:3300/0 conn(0x7f6568152b40 0x7f65680c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:24.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f6565d9b640 1 --2- 172.21.15.26:0/1923429760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f65681551f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65500097a0 tx=0x7f655002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b0ca13ba83112c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656559a640 1 -- 172.21.15.26:0/1923429760 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550031780 con 0x7f65680724d0 2024-03-06T07:29:24.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656559a640 1 -- 172.21.15.26:0/1923429760 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6550037070 con 0x7f65680724d0 2024-03-06T07:29:24.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656559a640 1 -- 172.21.15.26:0/1923429760 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550031b40 con 0x7f65680724d0 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 -- 172.21.15.26:0/1923429760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 msgr2=0x7f65681551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 --2- 172.21.15.26:0/1923429760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f65681551f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65500097a0 tx=0x7f655002b820 comp rx=0 tx=0).stop 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 -- 172.21.15.26:0/1923429760 shutdown_connections 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 --2- 172.21.15.26:0/1923429760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f65681551f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 --2- 172.21.15.26:0/1923429760 >> v2:172.21.15.26:3300/0 conn(0x7f6568152b40 0x7f65680c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 -- 172.21.15.26:0/1923429760 >> 172.21.15.26:0/1923429760 conn(0x7f656806b970 msgr2=0x7f65680aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 -- 172.21.15.26:0/1923429760 shutdown_connections 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.023+0000 7f656c815640 1 -- 172.21.15.26:0/1923429760 wait complete. 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 Processor -- start 2024-03-06T07:29:24.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 -- start start 2024-03-06T07:29:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f656806da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65680a7ff0 0x7f656806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568152b40 0x7f656806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65680c8d50 con 0x7f65680a7ff0 2024-03-06T07:29:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6568070490 con 0x7f6568152b40 2024-03-06T07:29:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f656c815640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65680c8ed0 con 0x7f65680724d0 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f6565d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65680a7ff0 0x7f656806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f6565d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65680a7ff0 0x7f656806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37668/0 (socket says 172.21.15.26:37668) 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.025+0000 7f6565d9b640 1 -- 172.21.15.26:0/3999003267 learned_addr learned my addr 172.21.15.26:0/3999003267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f656659c640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f656806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f6566d9d640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568152b40 0x7f656806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f6565d9b640 1 -- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 msgr2=0x7f656806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f6565d9b640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f656806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f6565d9b640 1 -- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568152b40 msgr2=0x7f656806eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f6565d9b640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568152b40 0x7f656806eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f6565d9b640 1 -- 172.21.15.26:0/3999003267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656806cc10 con 0x7f65680a7ff0 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f6565d9b640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65680a7ff0 0x7f656806e5f0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f6550034040 tx=0x7f65500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f655effd640 1 -- 172.21.15.26:0/3999003267 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550004230 con 0x7f65680a7ff0 2024-03-06T07:29:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f655effd640 1 -- 172.21.15.26:0/3999003267 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65500043f0 con 0x7f65680a7ff0 2024-03-06T07:29:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6568145d20 con 0x7f65680a7ff0 2024-03-06T07:29:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f655effd640 1 -- 172.21.15.26:0/3999003267 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550003850 con 0x7f65680a7ff0 2024-03-06T07:29:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.026+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65680aba90 con 0x7f65680a7ff0 2024-03-06T07:29:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.028+0000 7f655effd640 1 -- 172.21.15.26:0/3999003267 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65500039f0 con 0x7f65680a7ff0 2024-03-06T07:29:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.028+0000 7f655effd640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f654806b500 0x7f654806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.028+0000 7f655effd640 1 -- 172.21.15.26:0/3999003267 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65500ac130 con 0x7f65680a7ff0 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.028+0000 7f656659c640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f654806b500 0x7f654806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.029+0000 7f656659c640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f654806b500 0x7f654806d9c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6560006ca0 tx=0x7f6560009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.029+0000 7f656f400740 1 Processor -- start 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.029+0000 7f656f400740 1 -- start start 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656f400740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b304330 0x56155b301f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b3024b0 0x56155b2fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656f400740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155b2686b0 con 0x56155b303f50 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155b211a60 con 0x56155b3024b0 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56155b264750 con 0x56155b304330 2024-03-06T07:29:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37670/0 (socket says 172.21.15.26:37670) 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 -- 172.21.15.26:0/3877127409 learned_addr learned my addr 172.21.15.26:0/3877127409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f6566d9d640 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b3024b0 0x56155b2fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f6565d9b640 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b304330 0x56155b301f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 -- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b304330 msgr2=0x56155b301f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b304330 0x56155b301f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 -- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b3024b0 msgr2=0x56155b2fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b3024b0 0x56155b2fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.030+0000 7f656659c640 1 -- 172.21.15.26:0/3877127409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56155b22ff30 con 0x56155b303f50 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656659c640 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301a30 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f6560011dd0 tx=0x7f6560014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cc2e8f06dc00cb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f6547fff640 1 -- 172.21.15.26:0/3877127409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560025070 con 0x56155b303f50 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f6547fff640 1 -- 172.21.15.26:0/3877127409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656001b070 con 0x56155b303f50 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f6547fff640 1 -- 172.21.15.26:0/3877127409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560014dd0 con 0x56155b303f50 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 -- 172.21.15.26:0/3877127409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 msgr2=0x56155b301a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301a30 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f6560011dd0 tx=0x7f6560014000 comp rx=0 tx=0).stop 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 -- 172.21.15.26:0/3877127409 shutdown_connections 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b3024b0 0x56155b2fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b304330 0x56155b301f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 --2- 172.21.15.26:0/3877127409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301a30 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 -- 172.21.15.26:0/3877127409 >> 172.21.15.26:0/3877127409 conn(0x56155b3007d0 msgr2=0x56155b3253c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.031+0000 7f656f400740 1 -- 172.21.15.26:0/3877127409 shutdown_connections 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.032+0000 7f656f400740 1 -- 172.21.15.26:0/3877127409 wait complete. 2024-03-06T07:29:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.032+0000 7f656f400740 1 -- start start 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.032+0000 7f656f400740 1 Processor -- start 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b303f50 0x56155b3404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656f400740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b304330 0x56155b3267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b327e50 0x56155b320fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656f400740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155b326230 con 0x56155b304330 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155b224ea0 con 0x56155b327e50 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56155b325c70 con 0x56155b303f50 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656f400740 10 client.0 _ll_register_callbacks cb 0x56155b224870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f6565d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b304330 0x56155b3267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b303f50 0x56155b3404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b303f50 0x56155b3404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:38056/0 (socket says 172.21.15.26:38056) 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 -- 172.21.15.26:0/3869386307 learned_addr learned my addr 172.21.15.26:0/3869386307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f6566d9d640 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b327e50 0x56155b320fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 -- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b327e50 msgr2=0x56155b320fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b327e50 0x56155b320fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 -- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b304330 msgr2=0x56155b3267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b304330 0x56155b3267b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f656659c640 1 -- 172.21.15.26:0/3869386307 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56155b225270 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.033+0000 7f6565d9b640 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b304330 0x56155b3267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656659c640 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b303f50 0x56155b3404f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65600096c0 tx=0x7f6560004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=729dd694e67003e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f65457fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f65457fa640 1 -- 172.21.15.26:0/3869386307 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560032060 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f65457fa640 1 -- 172.21.15.26:0/3869386307 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6560005da0 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f656001b070 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f65457fa640 1 -- 172.21.15.26:0/3869386307 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6560038050 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 10 client.8462 fetch_fsmap learned FSMap version 201 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=201,monmap=2+}) v3 -- 0x56155af7d2d0 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f65457fa640 1 -- 172.21.15.26:0/3869386307 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656001b070 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f65457fa640 1 -- 172.21.15.26:0/3869386307 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 201) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f656003b020 con 0x56155b303f50 2024-03-06T07:29:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 10 client.8462 fetch_fsmap finished waiting for FSMap version 201 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 10 client.8462 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 20 client.8462 populate_metadata read hostname 'smithi026' 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b354960 0x56155b35f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 4 client.8462 mds_command: new command op to 10737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 --> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56155b275bf0 con 0x56155b354960 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.034+0000 7f6565d9b640 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b354960 0x56155b35f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.035+0000 7f6565d9b640 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b354960 0x56155b35f6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49eab9a9fadcfb8f server_cookie=76a0b66287cae1e6 in_seq=0 out_seq=0 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.035+0000 7f65457fa640 10 client.8462 ms_handle_connect on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.035+0000 7f65457fa640 1 -- 172.21.15.26:0/3869386307 <== mds.0 v2:172.21.15.26:6833/4219086401 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56155b275bf0 con 0x56155b354960 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.035+0000 7f65457fa640 10 client.8462 handle_command_reply: tid=0 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.035+0000 7f656f400740 1 client.8462 shutdown 2024-03-06T07:29:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.035+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b303f50 msgr2=0x56155b3404f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.035+0000 7f656f400740 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b303f50 0x56155b3404f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65600096c0 tx=0x7f6560004730 comp rx=0 tx=0).stop 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 shutdown_connections 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b354960 0x56155b35f6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b327e50 0x56155b320fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b304330 0x56155b3267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 --2- 172.21.15.26:0/3869386307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b303f50 0x56155b3404f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 >> 172.21.15.26:0/3869386307 conn(0x56155b3024b0 msgr2=0x56155b3028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f65457fa640 0 client.8462 ms_handle_reset on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 shutdown_connections 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 1 -- 172.21.15.26:0/3869386307 wait complete. 2024-03-06T07:29:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.036+0000 7f656f400740 20 client.8462 trim_cache size 0 max 16384 2024-03-06T07:29:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.052+0000 7f656f400740 1 Processor -- start 2024-03-06T07:29:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656f400740 1 -- start start 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656f400740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b364720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b3019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b328890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656f400740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681f7680 con 0x56155b303f50 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681f7ca0 con 0x56155b304330 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65681f6840 con 0x56155b327e50 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b364720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b364720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37684/0 (socket says 172.21.15.26:37684) 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f656659c640 1 -- 172.21.15.26:0/1581814776 learned_addr learned my addr 172.21.15.26:0/1581814776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f6565d9b640 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b3019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.053+0000 7f6566d9d640 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b328890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656659c640 1 -- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 msgr2=0x56155b328890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656659c640 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b328890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656659c640 1 -- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 msgr2=0x56155b3019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656659c640 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b3019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656659c640 1 -- 172.21.15.26:0/1581814776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56155b0476b0 con 0x56155b303f50 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f6566d9d640 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b328890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656659c640 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b364720 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f6560005e30 tx=0x7f656002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b25397f054a386 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f65477fe640 1 -- 172.21.15.26:0/1581814776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560025030 con 0x56155b303f50 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f65477fe640 1 -- 172.21.15.26:0/1581814776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65600064d0 con 0x56155b303f50 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f65477fe640 1 -- 172.21.15.26:0/1581814776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560006880 con 0x56155b303f50 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656f400740 1 -- 172.21.15.26:0/1581814776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 msgr2=0x56155b364720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656f400740 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b364720 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f6560005e30 tx=0x7f656002c040 comp rx=0 tx=0).stop 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656f400740 1 -- 172.21.15.26:0/1581814776 shutdown_connections 2024-03-06T07:29:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656f400740 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b328890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656f400740 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b3019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656f400740 1 --2- 172.21.15.26:0/1581814776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b364720 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.054+0000 7f656f400740 1 -- 172.21.15.26:0/1581814776 >> 172.21.15.26:0/1581814776 conn(0x56155b3024b0 msgr2=0x56155b3028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.055+0000 7f656f400740 1 -- 172.21.15.26:0/1581814776 shutdown_connections 2024-03-06T07:29:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.055+0000 7f656f400740 1 -- 172.21.15.26:0/1581814776 wait complete. 2024-03-06T07:29:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.055+0000 7f656f400740 1 -- start start 2024-03-06T07:29:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.055+0000 7f656f400740 1 Processor -- start 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656f400740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b31dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656f400740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b343710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656f400740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155b341040 con 0x56155b303f50 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155b3512e0 con 0x56155b304330 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656f400740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56155b3285e0 con 0x56155b327e50 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656f400740 10 client.0 _ll_register_callbacks cb 0x7f6560011cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37694/0 (socket says 172.21.15.26:37694) 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 -- 172.21.15.26:0/1919570793 learned_addr learned my addr 172.21.15.26:0/1919570793 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f6565d9b640 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b31dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f6566d9d640 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b343710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 -- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 msgr2=0x56155b343710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b343710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 -- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 msgr2=0x56155b31dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b31dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.056+0000 7f656659c640 1 -- 172.21.15.26:0/1919570793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56155b31e1b0 con 0x56155b303f50 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656659c640 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301da0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f6560023040 tx=0x7f6560018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889e63dd8ae58e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f65457fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f65457fa640 1 -- 172.21.15.26:0/1919570793 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656004a020 con 0x56155b303f50 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f65457fa640 1 -- 172.21.15.26:0/1919570793 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f656002a660 con 0x56155b303f50 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65600064d0 con 0x56155b303f50 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f65457fa640 1 -- 172.21.15.26:0/1919570793 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6560054050 con 0x56155b303f50 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 10 client.10947 fetch_fsmap learned FSMap version 201 2024-03-06T07:29:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=201,monmap=2+}) v3 -- 0x56155b31e8e0 con 0x56155b303f50 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f65457fa640 1 -- 172.21.15.26:0/1919570793 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65600064d0 con 0x56155b303f50 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f65457fa640 1 -- 172.21.15.26:0/1919570793 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 201) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f6560057020 con 0x56155b303f50 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 10 client.10947 fetch_fsmap finished waiting for FSMap version 201 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 10 client.10947 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 20 client.10947 populate_metadata read hostname 'smithi026' 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b3a9d70 0x56155b33eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 4 client.10947 mds_command: new command op to 10737 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 --> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56155b26ff40 con 0x56155b3a9d70 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.057+0000 7f6565d9b640 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b3a9d70 0x56155b33eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f65457fa640 10 client.10947 ms_handle_connect on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f6565d9b640 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b3a9d70 0x56155b33eb40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c49e70d5e9a0741 server_cookie=e49c3a580707a212 in_seq=0 out_seq=0 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f65457fa640 1 -- 172.21.15.26:0/1919570793 <== mds.0 v2:172.21.15.26:6833/4219086401 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x56155b26ff40 con 0x56155b3a9d70 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f65457fa640 10 client.10947 handle_command_reply: tid=0 2024-03-06T07:29:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 client.10947 shutdown 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 msgr2=0x56155b301da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301da0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f6560023040 tx=0x7f6560018000 comp rx=0 tx=0).stop 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 shutdown_connections 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:6833/4219086401,v1:172.21.15.26:6835/4219086401] conn(0x56155b3a9d70 0x56155b33eb40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56155b327e50 0x56155b343710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56155b304330 0x56155b31dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f65457fa640 0 client.10947 ms_handle_reset on v2:172.21.15.26:6833/4219086401 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 --2- 172.21.15.26:0/1919570793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56155b303f50 0x56155b301da0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 >> 172.21.15.26:0/1919570793 conn(0x56155b3024b0 msgr2=0x56155b327440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 shutdown_connections 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 1 -- 172.21.15.26:0/1919570793 wait complete. 2024-03-06T07:29:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.058+0000 7f656f400740 20 client.10947 trim_cache size 0 max 16384 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4c08c062-261c-4d4a-a3b3-e195823d80fd", 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 10737, 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 9.0491963819999999, 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 201, 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 211, 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 211, 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 14.138708123000001 2024-03-06T07:29:24.067 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:29:24.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f654806b500 msgr2=0x7f654806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f654806b500 0x7f654806d9c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6560006ca0 tx=0x7f6560009240 comp rx=0 tx=0).stop 2024-03-06T07:29:24.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65680a7ff0 msgr2=0x7f656806e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65680a7ff0 0x7f656806e5f0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f6550034040 tx=0x7f65500046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 shutdown_connections 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f654806b500 0x7f654806d9c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568152b40 0x7f656806eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65680a7ff0 0x7f656806e5f0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 --2- 172.21.15.26:0/3999003267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65680724d0 0x7f656806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 >> 172.21.15.26:0/3999003267 conn(0x7f656806b970 msgr2=0x7f6568151d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 shutdown_connections 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.059+0000 7f656c815640 1 -- 172.21.15.26:0/3999003267 wait complete. 2024-03-06T07:29:24.072 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:29:24.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.174+0000 7f71f7c84640 1 Processor -- start 2024-03-06T07:29:24.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.174+0000 7f71f7c84640 1 -- start start 2024-03-06T07:29:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f7c84640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f71f00b0070 0x7f71f0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f7c84640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f00b0450 con 0x7f71f006ca70 2024-03-06T07:29:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f7c84640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f006cf20 con 0x7f71f00c5380 2024-03-06T07:29:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f7c84640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f006c020 con 0x7f71f00b0070 2024-03-06T07:29:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f521f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f71f006ca70 0x7f71f00c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39178/0 (socket says 172.21.15.26:39178) 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f521f640 1 -- 172.21.15.26:0/2088091633 learned_addr learned my addr 172.21.15.26:0/2088091633 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f5a20640 1 --2- 172.21.15.26:0/2088091633 >> v2:172.21.15.117:3300/0 conn(0x7f71f00b0070 0x7f71f0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f4a1e640 1 -- 172.21.15.26:0/2088091633 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932088175 0 0) 0x7f71e4001110 con 0x7f71f006ca70 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f4a1e640 1 -- 172.21.15.26:0/2088091633 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d8003520 con 0x7f71f006ca70 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f4a1e640 1 -- 172.21.15.26:0/2088091633 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459452316 0 0) 0x7f71f006cf20 con 0x7f71f00c5380 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f4a1e640 1 -- 172.21.15.26:0/2088091633 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e4001110 con 0x7f71f00c5380 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f5a20640 1 -- 172.21.15.26:0/2088091633 >> v1:172.21.15.117:6789/0 conn(0x7f71f00c5380 legacy=0x7f71f014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f5a20640 1 -- 172.21.15.26:0/2088091633 >> v1:172.21.15.26:6789/0 conn(0x7f71f006ca70 legacy=0x7f71f00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.175+0000 7f71f5a20640 1 -- 172.21.15.26:0/2088091633 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00c77f0 con 0x7f71f00b0070 2024-03-06T07:29:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.176+0000 7f71f5a20640 1 --2- 172.21.15.26:0/2088091633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00b0070 0x7f71f0155c30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097a0 tx=0x7f71e802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef05818fe8e7eaf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.176+0000 7f71f4a1e640 1 -- 172.21.15.26:0/2088091633 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8035020 con 0x7f71f00b0070 2024-03-06T07:29:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.176+0000 7f71f4a1e640 1 -- 172.21.15.26:0/2088091633 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e8031830 con 0x7f71f00b0070 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.176+0000 7f71f4a1e640 1 -- 172.21.15.26:0/2088091633 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8031c00 con 0x7f71f00b0070 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.177+0000 7f71f7c84640 1 -- 172.21.15.26:0/2088091633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00b0070 msgr2=0x7f71f0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.177+0000 7f71f7c84640 1 --2- 172.21.15.26:0/2088091633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00b0070 0x7f71f0155c30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097a0 tx=0x7f71e802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.177+0000 7f71f7c84640 1 -- 172.21.15.26:0/2088091633 shutdown_connections 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.177+0000 7f71f7c84640 1 --2- 172.21.15.26:0/2088091633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00b0070 0x7f71f0155c30 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.177+0000 7f71f7c84640 1 -- 172.21.15.26:0/2088091633 >> 172.21.15.26:0/2088091633 conn(0x7f71f00ac590 msgr2=0x7f71f00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.177+0000 7f71f7c84640 1 -- 172.21.15.26:0/2088091633 shutdown_connections 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.177+0000 7f71f7c84640 1 -- 172.21.15.26:0/2088091633 wait complete. 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.178+0000 7f71f7c84640 1 Processor -- start 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.178+0000 7f71f7c84640 1 -- start start 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f7c84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f006ca70 0x7f71f0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f7c84640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00b0070 0x7f71f01e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f7c84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00c5380 0x7f71f01eb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f7c84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f006c550 con 0x7f71f006ca70 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f7c84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f006c270 con 0x7f71f00c5380 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f7c84640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f006da80 con 0x7f71f00b0070 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f006ca70 0x7f71f0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f006ca70 0x7f71f0156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37710/0 (socket says 172.21.15.26:37710) 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 -- 172.21.15.26:0/1700030195 learned_addr learned my addr 172.21.15.26:0/1700030195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f521f640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00b0070 0x7f71f01e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f6221640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00c5380 0x7f71f01eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 -- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00b0070 msgr2=0x7f71f01e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00b0070 0x7f71f01e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 -- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00c5380 msgr2=0x7f71f01eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00c5380 0x7f71f01eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 -- 172.21.15.26:0/1700030195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f01569c0 con 0x7f71f006ca70 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71f5a20640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f006ca70 0x7f71f0156480 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f71e802bee0 tx=0x7f71e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.179+0000 7f71ee7fc640 1 -- 172.21.15.26:0/1700030195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8035040 con 0x7f71f006ca70 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.180+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f01e8510 con 0x7f71f006ca70 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.180+0000 7f71ee7fc640 1 -- 172.21.15.26:0/1700030195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e803b920 con 0x7f71f006ca70 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.180+0000 7f71ee7fc640 1 -- 172.21.15.26:0/1700030195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e803bd10 con 0x7f71f006ca70 2024-03-06T07:29:24.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.180+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f00c2a10 con 0x7f71f006ca70 2024-03-06T07:29:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.181+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f004f330 con 0x7f71f006ca70 2024-03-06T07:29:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.181+0000 7f71ee7fc640 1 -- 172.21.15.26:0/1700030195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71e8038020 con 0x7f71f006ca70 2024-03-06T07:29:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.182+0000 7f71ee7fc640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f71dc06b4d0 0x7f71dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.182+0000 7f71ee7fc640 1 -- 172.21.15.26:0/1700030195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71e80ac2a0 con 0x7f71f006ca70 2024-03-06T07:29:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.182+0000 7f71f521f640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f71dc06b4d0 0x7f71dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.182+0000 7f71f521f640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f71dc06b4d0 0x7f71dc06d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f71f0158f90 tx=0x7f71e4007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.186+0000 7f71ee7fc640 1 -- 172.21.15.26:0/1700030195 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f71e807a950 con 0x7f71f006ca70 2024-03-06T07:29:24.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.407+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7f71f00ad9c0 con 0x7f71f006ca70 2024-03-06T07:29:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.667+0000 7f71ee7fc640 1 -- 172.21.15.26:0/1700030195 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v202) v1 ==== 125+0+0 (secure 0 0 0) 0x7f71e807e9f0 con 0x7f71f006ca70 2024-03-06T07:29:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked down. 2024-03-06T07:29:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f71dc06b4d0 msgr2=0x7f71dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f71dc06b4d0 0x7f71dc06d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f71f0158f90 tx=0x7f71e4007d70 comp rx=0 tx=0).stop 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f006ca70 msgr2=0x7f71f0156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f006ca70 0x7f71f0156480 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f71e802bee0 tx=0x7f71e80046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 shutdown_connections 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f71dc06b4d0 0x7f71dc06d990 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00c5380 0x7f71f01eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00b0070 0x7f71f01e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.669+0000 7f71f7c84640 1 --2- 172.21.15.26:0/1700030195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f006ca70 0x7f71f0156480 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.670+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 >> 172.21.15.26:0/1700030195 conn(0x7f71f00ac590 msgr2=0x7f71f00af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.670+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 shutdown_connections 2024-03-06T07:29:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.670+0000 7f71f7c84640 1 -- 172.21.15.26:0/1700030195 wait complete. 2024-03-06T07:29:24.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.782+0000 7fb0e2417640 1 Processor -- start 2024-03-06T07:29:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.782+0000 7fb0e2417640 1 -- start start 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.782+0000 7fb0e2417640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.782+0000 7fb0e2417640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc0c28d0 con 0x7fb0dc14f790 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.782+0000 7fb0e2417640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc0c2bb0 con 0x7fb0dc155a40 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.782+0000 7fb0e2417640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0c2680 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0dbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb0dc14f790 0x7fb0dc0c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39192/0 (socket says 172.21.15.26:39192) 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0dbfff640 1 -- 172.21.15.26:0/3339867231 learned_addr learned my addr 172.21.15.26:0/3339867231 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0db7fe640 1 --2- 172.21.15.26:0/3339867231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0daffd640 1 -- 172.21.15.26:0/3339867231 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080694956 0 0) 0x7fb0dc0c28d0 con 0x7fb0dc14f790 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0daffd640 1 -- 172.21.15.26:0/3339867231 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0c4003520 con 0x7fb0dc14f790 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0daffd640 1 -- 172.21.15.26:0/3339867231 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473080285 0 0) 0x7fb0dc0c2bb0 con 0x7fb0dc155a40 2024-03-06T07:29:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0daffd640 1 -- 172.21.15.26:0/3339867231 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0dc0c28d0 con 0x7fb0dc155a40 2024-03-06T07:29:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0db7fe640 1 -- 172.21.15.26:0/3339867231 >> v1:172.21.15.117:6789/0 conn(0x7fb0dc155a40 legacy=0x7fb0dc155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0db7fe640 1 -- 172.21.15.26:0/3339867231 >> v1:172.21.15.26:6789/0 conn(0x7fb0dc14f790 legacy=0x7fb0dc0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.783+0000 7fb0db7fe640 1 -- 172.21.15.26:0/3339867231 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0c2e90 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.784+0000 7fb0db7fe640 1 --2- 172.21.15.26:0/3339867231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1529d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00097d0 tx=0x7fb0d002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc73f718d0fb0d51 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.784+0000 7fb0daffd640 1 -- 172.21.15.26:0/3339867231 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0035020 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.784+0000 7fb0daffd640 1 -- 172.21.15.26:0/3339867231 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d0031890 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.784+0000 7fb0daffd640 1 -- 172.21.15.26:0/3339867231 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0031c60 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.785+0000 7fb0e2417640 1 -- 172.21.15.26:0/3339867231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc0c1aa0 msgr2=0x7fb0dc1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.785+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3339867231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1529d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00097d0 tx=0x7fb0d002b930 comp rx=0 tx=0).stop 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.785+0000 7fb0e2417640 1 -- 172.21.15.26:0/3339867231 shutdown_connections 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.785+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3339867231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1529d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.785+0000 7fb0e2417640 1 -- 172.21.15.26:0/3339867231 >> 172.21.15.26:0/3339867231 conn(0x7fb0dc0c6710 msgr2=0x7fb0dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.785+0000 7fb0e2417640 1 -- 172.21.15.26:0/3339867231 shutdown_connections 2024-03-06T07:29:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.785+0000 7fb0e2417640 1 -- 172.21.15.26:0/3339867231 wait complete. 2024-03-06T07:29:24.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.786+0000 7fb0e2417640 1 Processor -- start 2024-03-06T07:29:24.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.786+0000 7fb0e2417640 1 -- start start 2024-03-06T07:29:24.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc14f790 0x7fb0dc1dd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc155a40 0x7fb0dc1dd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0c5e30 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0c0d60 con 0x7fb0dc14f790 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0c5fb0 con 0x7fb0dc155a40 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37712/0 (socket says 172.21.15.26:37712) 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 -- 172.21.15.26:0/3510612930 learned_addr learned my addr 172.21.15.26:0/3510612930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0db7fe640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc14f790 0x7fb0dc1dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e09b4640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc155a40 0x7fb0dc1dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 -- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc155a40 msgr2=0x7fb0dc1dd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc155a40 0x7fb0dc1dd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 -- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc14f790 msgr2=0x7fb0dc1dd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc14f790 0x7fb0dc1dd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 -- 172.21.15.26:0/3510612930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0a5e40 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e09b4640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc155a40 0x7fb0dc1dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0dbfff640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1512e0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c530 tx=0x7fb0cc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0d8ff9640 1 -- 172.21.15.26:0/3510612930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc017070 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0d8ff9640 1 -- 172.21.15.26:0/3510612930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc005ce0 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc1e7790 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0d8ff9640 1 -- 172.21.15.26:0/3510612930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc007270 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.787+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc1dba80 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.789+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc075300 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.789+0000 7fb0d8ff9640 1 -- 172.21.15.26:0/3510612930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0cc01a070 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.789+0000 7fb0d8ff9640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.790+0000 7fb0d8ff9640 1 -- 172.21.15.26:0/3510612930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb0cc0895e0 con 0x7fb0dc0c1aa0 2024-03-06T07:29:24.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.790+0000 7fb0db7fe640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.790+0000 7fb0db7fe640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0036000 tx=0x7fb0d0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:24.794+0000 7fb0d8ff9640 1 -- 172.21.15.26:0/3510612930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb0dc075300 con 0x7fb0dc0c1aa0 2024-03-06T07:29:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.026+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0dc0acab0 con 0x7fb0dc0c1aa0 2024-03-06T07:29:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.026+0000 7fb0d8ff9640 1 -- 172.21.15.26:0/3510612930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fb0dc0acab0 con 0x7fb0dc0c1aa0 2024-03-06T07:29:25.028 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:25.028 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 msgr2=0x7fb0c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0036000 tx=0x7fb0d0008030 comp rx=0 tx=0).stop 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc0c1aa0 msgr2=0x7fb0dc1512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1512e0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c530 tx=0x7fb0cc00c9f0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 shutdown_connections 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc155a40 0x7fb0dc1dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc14f790 0x7fb0dc1dd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 --2- 172.21.15.26:0/3510612930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc0c1aa0 0x7fb0dc1512e0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 >> 172.21.15.26:0/3510612930 conn(0x7fb0dc0c6710 msgr2=0x7fb0dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 shutdown_connections 2024-03-06T07:29:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:25.028+0000 7fb0e2417640 1 -- 172.21.15.26:0/3510612930 wait complete. 2024-03-06T07:29:25.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:25.443 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:26.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.544+0000 7f68b70fb640 1 Processor -- start 2024-03-06T07:29:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.544+0000 7f68b70fb640 1 -- start start 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.545+0000 7f68b70fb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68b014f360 0x7f68b0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.545+0000 7f68b70fb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b00c73d0 con 0x7f68b00c88c0 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.545+0000 7f68b70fb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b00be2e0 con 0x7f68b0158b60 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.545+0000 7f68b70fb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b0159700 con 0x7f68b014f360 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.545+0000 7f68affff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f68b00c88c0 0x7f68b00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39206/0 (socket says 172.21.15.26:39206) 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.545+0000 7f68affff640 1 -- 172.21.15.26:0/1207924961 learned_addr learned my addr 172.21.15.26:0/1207924961 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476041213 0 0) 0x7f68b00c73d0 con 0x7f68b00c88c0 2024-03-06T07:29:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a0003520 con 0x7f68b00c88c0 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68b4e97640 1 --2- 172.21.15.26:0/1207924961 >> v2:172.21.15.117:3300/0 conn(0x7f68b014f360 0x7f68b0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328330228 0 0) 0x7f68a0003520 con 0x7f68b00c88c0 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 >> v2:172.21.15.117:3300/0 conn(0x7f68b014f360 msgr2=0x7f68b0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 --2- 172.21.15.26:0/1207924961 >> v2:172.21.15.117:3300/0 conn(0x7f68b014f360 0x7f68b0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 >> v1:172.21.15.117:6789/0 conn(0x7f68b0158b60 legacy=0x7f68b0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00c7670 con 0x7f68b00c88c0 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f689c001670 con 0x7f68b00c88c0 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 >> v1:172.21.15.26:6789/0 conn(0x7f68b00c88c0 legacy=0x7f68b00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a00039b0 0x7f68a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a00051c0 0x7f68a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b0159700 con 0x7f68a0004800 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a000c160 con 0x7f68a00039b0 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a000c360 con 0x7f68a00051c0 2024-03-06T07:29:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68affff640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68b4e97640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a00039b0 0x7f68a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68b5698640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a00051c0 0x7f68a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68affff640 1 -- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a00051c0 msgr2=0x7f68a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68affff640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a00051c0 0x7f68a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68affff640 1 -- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a00039b0 msgr2=0x7f68a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68affff640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a00039b0 0x7f68a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68affff640 1 -- 172.21.15.26:0/1207924961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a0003520 con 0x7f68a0004800 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68b5698640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a00051c0 0x7f68a000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.546+0000 7f68b5698640 1 -- 172.21.15.26:0/1207924961 reap_dead start 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.547+0000 7f68affff640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68a0004c80 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f68b00c8d20 tx=0x7f68a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331dff7a2067feff server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.547+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c007d30 con 0x7f68a0004800 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.547+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c030070 con 0x7f68a0004800 2024-03-06T07:29:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.547+0000 7f68af7fe640 1 -- 172.21.15.26:0/1207924961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c005d30 con 0x7f68a0004800 2024-03-06T07:29:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.548+0000 7f68b70fb640 1 -- 172.21.15.26:0/1207924961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 msgr2=0x7f68a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.548+0000 7f68b70fb640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68a0004c80 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f68b00c8d20 tx=0x7f68a0005830 comp rx=0 tx=0).stop 2024-03-06T07:29:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.548+0000 7f68b70fb640 1 -- 172.21.15.26:0/1207924961 shutdown_connections 2024-03-06T07:29:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.548+0000 7f68b70fb640 1 --2- 172.21.15.26:0/1207924961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68a0004c80 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.548+0000 7f68b70fb640 1 -- 172.21.15.26:0/1207924961 >> 172.21.15.26:0/1207924961 conn(0x7f68b00ac570 msgr2=0x7f68b00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.548+0000 7f68b70fb640 1 -- 172.21.15.26:0/1207924961 shutdown_connections 2024-03-06T07:29:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.548+0000 7f68b70fb640 1 -- 172.21.15.26:0/1207924961 wait complete. 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.550+0000 7f68b70fb640 1 Processor -- start 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.550+0000 7f68b70fb640 1 -- start start 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b70fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68b0156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b70fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b01546a0 0x7f68b01ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b70fb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b0154b00 0x7f68b01ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b70fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00c7c30 con 0x7f68a0004800 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b70fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b0155c10 con 0x7f68b01546a0 2024-03-06T07:29:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b70fb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b0155d90 con 0x7f68b0154b00 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68b0156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68b0156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37728/0 (socket says 172.21.15.26:37728) 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 -- 172.21.15.26:0/20933926 learned_addr learned my addr 172.21.15.26:0/20933926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68affff640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b01546a0 0x7f68b01ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b5698640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b0154b00 0x7f68b01ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 -- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b0154b00 msgr2=0x7f68b01ee9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b0154b00 0x7f68b01ee9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 -- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b01546a0 msgr2=0x7f68b01ee4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b01546a0 0x7f68b01ee4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b4e97640 1 -- 172.21.15.26:0/20933926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0155130 con 0x7f68a0004800 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.551+0000 7f68b5698640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b0154b00 0x7f68b01ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.552+0000 7f68b4e97640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68b0156560 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f68a4011ad0 tx=0x7f68a4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.552+0000 7f68ad7fa640 1 -- 172.21.15.26:0/20933926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400fb50 con 0x7f68a0004800 2024-03-06T07:29:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.552+0000 7f68ad7fa640 1 -- 172.21.15.26:0/20933926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a400fcf0 con 0x7f68a0004800 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.552+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b01ef070 con 0x7f68a0004800 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.552+0000 7f68ad7fa640 1 -- 172.21.15.26:0/20933926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400fea0 con 0x7f68a0004800 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.552+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b00c1120 con 0x7f68a0004800 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.553+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b00c64c0 con 0x7f68a0004800 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.553+0000 7f68ad7fa640 1 -- 172.21.15.26:0/20933926 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68a4012040 con 0x7f68a0004800 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.554+0000 7f68ad7fa640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f689406b4d0 0x7f689406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.554+0000 7f68ad7fa640 1 -- 172.21.15.26:0/20933926 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68a408a910 con 0x7f68a0004800 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.554+0000 7f68affff640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f689406b4d0 0x7f689406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.554+0000 7f68affff640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f689406b4d0 0x7f689406d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f689c034000 tx=0x7f689c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:26.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.558+0000 7f68ad7fa640 1 -- 172.21.15.26:0/20933926 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68b00c64c0 con 0x7f68a0004800 2024-03-06T07:29:26.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.775+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68b004f330 con 0x7f68a0004800 2024-03-06T07:29:26.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.776+0000 7f68ad7fa640 1 -- 172.21.15.26:0/20933926 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f68a4019030 con 0x7f68a0004800 2024-03-06T07:29:26.778 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:26.778 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:26.778 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f689406b4d0 msgr2=0x7f689406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f689406b4d0 0x7f689406d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f689c034000 tx=0x7f689c007000 comp rx=0 tx=0).stop 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 msgr2=0x7f68b0156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68b0156560 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f68a4011ad0 tx=0x7f68a4011f90 comp rx=0 tx=0).stop 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 shutdown_connections 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f689406b4d0 0x7f689406d990 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b0154b00 0x7f68b01ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b01546a0 0x7f68b01ee4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 --2- 172.21.15.26:0/20933926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0004800 0x7f68b0156560 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 >> 172.21.15.26:0/20933926 conn(0x7f68b00ac570 msgr2=0x7f68b00ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 shutdown_connections 2024-03-06T07:29:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:26.778+0000 7f68b70fb640 1 -- 172.21.15.26:0/20933926 wait complete. 2024-03-06T07:29:27.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:27.193 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:28.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.308+0000 7fa69a639640 1 Processor -- start 2024-03-06T07:29:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.308+0000 7fa69a639640 1 -- start start 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa69a639640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa69414f790 0x7fa6941529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa69a639640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa694155a40 0x7fa694155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa69a639640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940c28d0 con 0x7fa6940c1aa0 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa69a639640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940c2680 con 0x7fa694155a40 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa69a639640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940c2bb0 con 0x7fa69414f790 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa698bd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa694155a40 0x7fa694155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa6937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa6940c1aa0 0x7fa6940c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60732/0 (socket says 172.21.15.26:60732) 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa6937fe640 1 -- 172.21.15.26:0/96298173 learned_addr learned my addr 172.21.15.26:0/96298173 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa693fff640 1 --2- 172.21.15.26:0/96298173 >> v2:172.21.15.117:3300/0 conn(0x7fa69414f790 0x7fa6941529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa698bd6640 1 -- 172.21.15.26:0/96298173 >> v2:172.21.15.117:3300/0 conn(0x7fa69414f790 msgr2=0x7fa6941529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa698bd6640 1 --2- 172.21.15.26:0/96298173 >> v2:172.21.15.117:3300/0 conn(0x7fa69414f790 0x7fa6941529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa698bd6640 1 -- 172.21.15.26:0/96298173 >> v1:172.21.15.26:6789/0 conn(0x7fa6940c1aa0 legacy=0x7fa6940c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa692ffd640 1 -- 172.21.15.26:0/96298173 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902866814 0 0) 0x7fa6940c28d0 con 0x7fa6940c1aa0 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa698bd6640 1 -- 172.21.15.26:0/96298173 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940c2e90 con 0x7fa694155a40 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.309+0000 7fa693fff640 1 --2- 172.21.15.26:0/96298173 >> v2:172.21.15.117:3300/0 conn(0x7fa69414f790 0x7fa6941529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.310+0000 7fa698bd6640 1 --2- 172.21.15.26:0/96298173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa694155e50 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fa6940c3170 tx=0x7fa68c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb83a30a107f1304 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.310+0000 7fa692ffd640 1 -- 172.21.15.26:0/96298173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c02ea60 con 0x7fa694155a40 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.310+0000 7fa692ffd640 1 -- 172.21.15.26:0/96298173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c02ec00 con 0x7fa694155a40 2024-03-06T07:29:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.310+0000 7fa692ffd640 1 -- 172.21.15.26:0/96298173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c009540 con 0x7fa694155a40 2024-03-06T07:29:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 -- 172.21.15.26:0/96298173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 msgr2=0x7fa694155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 --2- 172.21.15.26:0/96298173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa694155e50 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fa6940c3170 tx=0x7fa68c02e0c0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 -- 172.21.15.26:0/96298173 shutdown_connections 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 --2- 172.21.15.26:0/96298173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa694155e50 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 --2- 172.21.15.26:0/96298173 >> v2:172.21.15.117:3300/0 conn(0x7fa69414f790 0x7fa6941529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 -- 172.21.15.26:0/96298173 >> 172.21.15.26:0/96298173 conn(0x7fa6940c6710 msgr2=0x7fa6940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 -- 172.21.15.26:0/96298173 shutdown_connections 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.311+0000 7fa69a639640 1 -- 172.21.15.26:0/96298173 wait complete. 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.312+0000 7fa69a639640 1 Processor -- start 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.312+0000 7fa69a639640 1 -- start start 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa69a639640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6940c1aa0 0x7fa694147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa69a639640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa69414f790 0x7fa694151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa69a639640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa69414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa69a639640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c0d60 con 0x7fa694155a40 2024-03-06T07:29:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa69a639640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa694156390 con 0x7fa6940c1aa0 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa69a639640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c5e30 con 0x7fa69414f790 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa69414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa69414ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34544/0 (socket says 172.21.15.26:34544) 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 -- 172.21.15.26:0/1079810863 learned_addr learned my addr 172.21.15.26:0/1079810863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa693fff640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6940c1aa0 0x7fa694147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa6937fe640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa69414f790 0x7fa694151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 -- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa69414f790 msgr2=0x7fa694151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa69414f790 0x7fa694151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 -- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6940c1aa0 msgr2=0x7fa694147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6940c1aa0 0x7fa694147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 -- 172.21.15.26:0/1079810863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa694153530 con 0x7fa694155a40 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa693fff640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6940c1aa0 0x7fa694147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.313+0000 7fa698bd6640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa69414ac70 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00d140 tx=0x7fa68c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.314+0000 7fa690ff9640 1 -- 172.21.15.26:0/1079810863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c045030 con 0x7fa694155a40 2024-03-06T07:29:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.314+0000 7fa690ff9640 1 -- 172.21.15.26:0/1079810863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c009e20 con 0x7fa694155a40 2024-03-06T07:29:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.314+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa694149990 con 0x7fa694155a40 2024-03-06T07:29:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.314+0000 7fa690ff9640 1 -- 172.21.15.26:0/1079810863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c040660 con 0x7fa694155a40 2024-03-06T07:29:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.314+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69414a470 con 0x7fa694155a40 2024-03-06T07:29:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.315+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69404f330 con 0x7fa694155a40 2024-03-06T07:29:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.315+0000 7fa690ff9640 1 -- 172.21.15.26:0/1079810863 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa68c048070 con 0x7fa694155a40 2024-03-06T07:29:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.315+0000 7fa690ff9640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68006b500 0x7fa68006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.316+0000 7fa690ff9640 1 -- 172.21.15.26:0/1079810863 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa68c0aeb40 con 0x7fa694155a40 2024-03-06T07:29:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.316+0000 7fa693fff640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68006b500 0x7fa68006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.316+0000 7fa693fff640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68006b500 0x7fa68006d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa688006fd0 tx=0x7fa688008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.320+0000 7fa690ff9640 1 -- 172.21.15.26:0/1079810863 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa68c04f070 con 0x7fa694155a40 2024-03-06T07:29:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.540+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6940c0b40 con 0x7fa694155a40 2024-03-06T07:29:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.541+0000 7fa690ff9640 1 -- 172.21.15.26:0/1079810863 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fa68c07d8c0 con 0x7fa694155a40 2024-03-06T07:29:28.542 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:28.543 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68006b500 msgr2=0x7fa68006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68006b500 0x7fa68006d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa688006fd0 tx=0x7fa688008040 comp rx=0 tx=0).stop 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 msgr2=0x7fa69414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa69414ac70 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00d140 tx=0x7fa68c004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 shutdown_connections 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa68006b500 0x7fa68006d9c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa694155a40 0x7fa69414ac70 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa69414f790 0x7fa694151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 --2- 172.21.15.26:0/1079810863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6940c1aa0 0x7fa694147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 >> 172.21.15.26:0/1079810863 conn(0x7fa6940c6710 msgr2=0x7fa69414ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 shutdown_connections 2024-03-06T07:29:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:28.543+0000 7fa69a639640 1 -- 172.21.15.26:0/1079810863 wait complete. 2024-03-06T07:29:28.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:28.960 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:29.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.075+0000 7f62f99eb640 1 Processor -- start 2024-03-06T07:29:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f99eb640 1 -- start start 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f99eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 0x7f62f40a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f99eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62f40c36e0 0x7f62f41558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f99eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f4158800 con 0x7f62f40c3280 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f99eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40c3b10 con 0x7f62f40c36e0 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f99eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4158ae0 con 0x7f62f406d9f0 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f27fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f62f40c3280 0x7f62f41551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60744/0 (socket says 172.21.15.26:60744) 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.076+0000 7f62f27fc640 1 -- 172.21.15.26:0/1136300004 learned_addr learned my addr 172.21.15.26:0/1136300004 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f37fe640 1 --2- 172.21.15.26:0/1136300004 >> v2:172.21.15.117:3300/0 conn(0x7f62f40c36e0 0x7f62f41558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f2ffd640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 0x7f62f40a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483220068 0 0) 0x7f62f4158800 con 0x7f62f40c3280 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e0003520 con 0x7f62f40c3280 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 123288563 0 0) 0x7f62e0003520 con 0x7f62f40c3280 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 msgr2=0x7f62f40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 0x7f62f40a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 >> v2:172.21.15.117:3300/0 conn(0x7f62f40c36e0 msgr2=0x7f62f41558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 --2- 172.21.15.26:0/1136300004 >> v2:172.21.15.117:3300/0 conn(0x7f62f40c36e0 0x7f62f41558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4158dc0 con 0x7f62f40c3280 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f62dc001670 con 0x7f62f40c3280 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 >> v1:172.21.15.26:6789/0 conn(0x7f62f40c3280 legacy=0x7f62f41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e00039b0 0x7f62e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62e0004800 0x7f62e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 0x7f62e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40c3b10 con 0x7f62e0004800 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4158ae0 con 0x7f62e00039b0 2024-03-06T07:29:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e000b9c0 con 0x7f62f406d9f0 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f27fc640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62e0004800 0x7f62e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f2ffd640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e00039b0 0x7f62e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f37fe640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 0x7f62e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f27fc640 1 -- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 msgr2=0x7f62e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f27fc640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62f406d9f0 0x7f62e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f27fc640 1 -- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e00039b0 msgr2=0x7f62e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f27fc640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e00039b0 0x7f62e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.077+0000 7f62f27fc640 1 -- 172.21.15.26:0/1136300004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e0003520 con 0x7f62e0004800 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.078+0000 7f62f27fc640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62e0004800 0x7f62e0004c80 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c210 tx=0x7f62e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8c5571304d4a70 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.078+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc007d50 con 0x7f62e0004800 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.078+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc030070 con 0x7f62e0004800 2024-03-06T07:29:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.078+0000 7f62f1ffb640 1 -- 172.21.15.26:0/1136300004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc02cd20 con 0x7f62e0004800 2024-03-06T07:29:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.079+0000 7f62f99eb640 1 -- 172.21.15.26:0/1136300004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62e0004800 msgr2=0x7f62e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.079+0000 7f62f99eb640 1 --2- 172.21.15.26:0/1136300004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62e0004800 0x7f62e0004c80 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c210 tx=0x7f62e0005830 comp rx=0 tx=0).stop 2024-03-06T07:29:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.079+0000 7f62f37fe640 1 -- 172.21.15.26:0/1136300004 reap_dead start 2024-03-06T07:29:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.079+0000 7f62f99eb640 1 -- 172.21.15.26:0/1136300004 shutdown_connections 2024-03-06T07:29:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.079+0000 7f62f99eb640 1 -- 172.21.15.26:0/1136300004 >> 172.21.15.26:0/1136300004 conn(0x7f62f40aa630 msgr2=0x7f62f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:30.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.079+0000 7f62f99eb640 1 -- 172.21.15.26:0/1136300004 shutdown_connections 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.079+0000 7f62f99eb640 1 -- 172.21.15.26:0/1136300004 wait complete. 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.080+0000 7f62f99eb640 1 Processor -- start 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.080+0000 7f62f99eb640 1 -- start start 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f99eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e0004800 0x7f62f40ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f99eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f409ad10 0x7f62f40ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f99eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f406f860 0x7f62f41ea660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f99eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4155e30 con 0x7f62f406f860 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f99eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4159380 con 0x7f62f409ad10 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f99eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f406ddd0 con 0x7f62e0004800 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f406f860 0x7f62f41ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f406f860 0x7f62f41ea660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34548/0 (socket says 172.21.15.26:34548) 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 -- 172.21.15.26:0/212612934 learned_addr learned my addr 172.21.15.26:0/212612934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f2ffd640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e0004800 0x7f62f40ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f27fc640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f409ad10 0x7f62f40ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 -- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e0004800 msgr2=0x7f62f40ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e0004800 0x7f62f40ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 -- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f409ad10 msgr2=0x7f62f40ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f409ad10 0x7f62f40ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.081+0000 7f62f37fe640 1 -- 172.21.15.26:0/212612934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f41580e0 con 0x7f62f406f860 2024-03-06T07:29:30.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.082+0000 7f62f37fe640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f406f860 0x7f62f41ea660 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f62f4072440 tx=0x7f62ec011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.083+0000 7f62ebfff640 1 -- 172.21.15.26:0/212612934 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec019020 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.083+0000 7f62ebfff640 1 -- 172.21.15.26:0/212612934 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec004030 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.083+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f4158380 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.083+0000 7f62ebfff640 1 -- 172.21.15.26:0/212612934 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec004400 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.083+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f41ee8c0 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.085+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f404f330 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.085+0000 7f62ebfff640 1 -- 172.21.15.26:0/212612934 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62ec01c020 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.085+0000 7f62ebfff640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62d406b500 0x7f62d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.085+0000 7f62ebfff640 1 -- 172.21.15.26:0/212612934 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62ec08a6c0 con 0x7f62f406f860 2024-03-06T07:29:30.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.085+0000 7f62f2ffd640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62d406b500 0x7f62d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.086+0000 7f62f2ffd640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62d406b500 0x7f62d406d9c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f62e4014370 tx=0x7f62e400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.090+0000 7f62ebfff640 1 -- 172.21.15.26:0/212612934 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f62ec023020 con 0x7f62f406f860 2024-03-06T07:29:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.308+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62f40c73f0 con 0x7f62f406f860 2024-03-06T07:29:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.309+0000 7f62ebfff640 1 -- 172.21.15.26:0/212612934 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f62ec058df0 con 0x7f62f406f860 2024-03-06T07:29:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:30.311 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62d406b500 msgr2=0x7f62d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62d406b500 0x7f62d406d9c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f62e4014370 tx=0x7f62e400b040 comp rx=0 tx=0).stop 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f406f860 msgr2=0x7f62f41ea660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f406f860 0x7f62f41ea660 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f62f4072440 tx=0x7f62ec011cf0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 shutdown_connections 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f62d406b500 0x7f62d406d9c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62f406f860 0x7f62f41ea660 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f409ad10 0x7f62f40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 --2- 172.21.15.26:0/212612934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e0004800 0x7f62f40ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 >> 172.21.15.26:0/212612934 conn(0x7f62f40aa630 msgr2=0x7f62f41548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 shutdown_connections 2024-03-06T07:29:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:30.311+0000 7f62f99eb640 1 -- 172.21.15.26:0/212612934 wait complete. 2024-03-06T07:29:30.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:30.727 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:31.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.829+0000 7f6510756640 1 Processor -- start 2024-03-06T07:29:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.829+0000 7f6510756640 1 -- start start 2024-03-06T07:29:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.829+0000 7f6510756640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 0x7f65080c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.829+0000 7f6510756640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65081591c0 con 0x7f65080c5480 2024-03-06T07:29:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.829+0000 7f6510756640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65081594a0 con 0x7f65081586f0 2024-03-06T07:29:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.829+0000 7f6510756640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650806df40 con 0x7f6508155ba0 2024-03-06T07:29:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650e4f2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f65080c5480 0x7f650806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60758/0 (socket says 172.21.15.26:60758) 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650e4f2640 1 -- 172.21.15.26:0/118474409 learned_addr learned my addr 172.21.15.26:0/118474409 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690310990 0 0) 0x7f65081591c0 con 0x7f65080c5480 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f4003520 con 0x7f65080c5480 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650dcf1640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 0x7f65080c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406807890 0 0) 0x7f64f4003520 con 0x7f65080c5480 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 msgr2=0x7f65080c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 0x7f65080c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 >> v1:172.21.15.117:6789/0 conn(0x7f65081586f0 legacy=0x7f650815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65081553a0 con 0x7f65080c5480 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6504001670 con 0x7f65080c5480 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 >> v1:172.21.15.26:6789/0 conn(0x7f65080c5480 legacy=0x7f650806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f40039e0 0x7f64f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f4004800 0x7f64f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 0x7f64f4005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650806df40 con 0x7f64f40039e0 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f400bb50 con 0x7f64f4004800 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f400bd50 con 0x7f6508155ba0 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.830+0000 7f650e4f2640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f40039e0 0x7f64f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650dcf1640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f4004800 0x7f64f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650ecf3640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 0x7f64f4005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650e4f2640 1 -- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 msgr2=0x7f64f4005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650e4f2640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 0x7f64f4005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650e4f2640 1 -- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f4004800 msgr2=0x7f64f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650e4f2640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f4004800 0x7f64f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650e4f2640 1 -- 172.21.15.26:0/118474409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4003520 con 0x7f64f40039e0 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650ecf3640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6508155ba0 0x7f64f4005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.831+0000 7f650e4f2640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f40039e0 0x7f64f4003e00 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f650400c190 tx=0x7f64f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72585e71c5e1607d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504007c60 con 0x7f64f40039e0 2024-03-06T07:29:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504030070 con 0x7f64f40039e0 2024-03-06T07:29:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f650d4f0640 1 -- 172.21.15.26:0/118474409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504005d70 con 0x7f64f40039e0 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f6510756640 1 -- 172.21.15.26:0/118474409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f40039e0 msgr2=0x7f64f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f6510756640 1 --2- 172.21.15.26:0/118474409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f40039e0 0x7f64f4003e00 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f650400c190 tx=0x7f64f4005830 comp rx=0 tx=0).stop 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f650ecf3640 1 -- 172.21.15.26:0/118474409 reap_dead start 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f6510756640 1 -- 172.21.15.26:0/118474409 shutdown_connections 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f6510756640 1 -- 172.21.15.26:0/118474409 >> 172.21.15.26:0/118474409 conn(0x7f650806b1a0 msgr2=0x7f650806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f6510756640 1 -- 172.21.15.26:0/118474409 shutdown_connections 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.832+0000 7f6510756640 1 -- 172.21.15.26:0/118474409 wait complete. 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.833+0000 7f6510756640 1 Processor -- start 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f6510756640 1 -- start start 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f6510756640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f40039e0 0x7f65080ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f6510756640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f650814fb80 0x7f6508150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f6510756640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65080abe20 0x7f65080ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f6510756640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6508156000 con 0x7f65080abe20 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f6510756640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650806fdb0 con 0x7f64f40039e0 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f6510756640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6508152a20 con 0x7f650814fb80 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f650ecf3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65080abe20 0x7f65080ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f650ecf3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65080abe20 0x7f65080ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34572/0 (socket says 172.21.15.26:34572) 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f650ecf3640 1 -- 172.21.15.26:0/2704319862 learned_addr learned my addr 172.21.15.26:0/2704319862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f650e4f2640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f40039e0 0x7f65080ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.834+0000 7f650dcf1640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f650814fb80 0x7f6508150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f650ecf3640 1 -- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f650814fb80 msgr2=0x7f6508150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f650ecf3640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f650814fb80 0x7f6508150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f650ecf3640 1 -- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f40039e0 msgr2=0x7f65080ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f650ecf3640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f40039e0 0x7f65080ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f650ecf3640 1 -- 172.21.15.26:0/2704319862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65081e2520 con 0x7f65080abe20 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f650ecf3640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65080abe20 0x7f65080ac2a0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f6500000f80 tx=0x7f650000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f64feffd640 1 -- 172.21.15.26:0/2704319862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500019070 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f64feffd640 1 -- 172.21.15.26:0/2704319862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6500006d60 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65081eae80 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f64feffd640 1 -- 172.21.15.26:0/2704319862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500003bb0 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.835+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65081eb3d0 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.836+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650804f330 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.836+0000 7f64feffd640 1 -- 172.21.15.26:0/2704319862 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f650001c070 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.837+0000 7f64feffd640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64ec06b4d0 0x7f64ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.837+0000 7f64feffd640 1 -- 172.21.15.26:0/2704319862 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65000897f0 con 0x7f65080abe20 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.837+0000 7f650e4f2640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64ec06b4d0 0x7f64ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.838+0000 7f650e4f2640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64ec06b4d0 0x7f64ec06d990 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6504034000 tx=0x7f6504009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:31.841+0000 7f64feffd640 1 -- 172.21.15.26:0/2704319862 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6500057f20 con 0x7f65080abe20 2024-03-06T07:29:32.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.061+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f650806e0c0 con 0x7f65080abe20 2024-03-06T07:29:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.061+0000 7f64feffd640 1 -- 172.21.15.26:0/2704319862 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f650005bfc0 con 0x7f65080abe20 2024-03-06T07:29:32.063 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:32.063 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.063+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64ec06b4d0 msgr2=0x7f64ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.063+0000 7f6510756640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64ec06b4d0 0x7f64ec06d990 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6504034000 tx=0x7f6504009030 comp rx=0 tx=0).stop 2024-03-06T07:29:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.063+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65080abe20 msgr2=0x7f65080ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.063+0000 7f6510756640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65080abe20 0x7f65080ac2a0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f6500000f80 tx=0x7f650000ead0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 shutdown_connections 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f64ec06b4d0 0x7f64ec06d990 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65080abe20 0x7f65080ac2a0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f650814fb80 0x7f6508150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 --2- 172.21.15.26:0/2704319862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f40039e0 0x7f65080ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 >> 172.21.15.26:0/2704319862 conn(0x7f650806b1a0 msgr2=0x7f65080c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 shutdown_connections 2024-03-06T07:29:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:32.064+0000 7f6510756640 1 -- 172.21.15.26:0/2704319862 wait complete. 2024-03-06T07:29:32.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:32.481 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:33.482 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.582+0000 7f6e08c2a640 1 Processor -- start 2024-03-06T07:29:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.582+0000 7f6e08c2a640 1 -- start start 2024-03-06T07:29:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e08c2a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e0414f360 0x7f6e04155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e08c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e040c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e08c2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e040c73d0 con 0x7f6e040c5880 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e08c2a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e040c8dd0 con 0x7f6e0414f360 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e08c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040be2e0 con 0x7f6e040c88c0 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e0259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e0414f360 0x7f6e04155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e02d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e040c5880 0x7f6e04158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60774/0 (socket says 172.21.15.26:60774) 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.583+0000 7f6e02d9d640 1 -- 172.21.15.26:0/2208250659 learned_addr learned my addr 172.21.15.26:0/2208250659 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.584+0000 7f6dfbfff640 1 --2- 172.21.15.26:0/2208250659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e040c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.584+0000 7f6e0259c640 1 -- 172.21.15.26:0/2208250659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 msgr2=0x7f6e040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.584+0000 7f6e0259c640 1 --2- 172.21.15.26:0/2208250659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e040c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.584+0000 7f6e0259c640 1 -- 172.21.15.26:0/2208250659 >> v1:172.21.15.26:6789/0 conn(0x7f6e040c5880 legacy=0x7f6e04158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.584+0000 7f6e0259c640 1 -- 172.21.15.26:0/2208250659 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e040c7670 con 0x7f6e0414f360 2024-03-06T07:29:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.584+0000 7f6e01d9b640 1 -- 172.21.15.26:0/2208250659 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030808660 0 0) 0x7f6e040c73d0 con 0x7f6e040c5880 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e0259c640 1 --2- 172.21.15.26:0/2208250659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e04155c10 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f6df40097a0 tx=0x7f6df402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345d4c2160eddf32 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e01d9b640 1 -- 172.21.15.26:0/2208250659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40315a0 con 0x7f6e0414f360 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e01d9b640 1 -- 172.21.15.26:0/2208250659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df4037070 con 0x7f6e0414f360 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e01d9b640 1 -- 172.21.15.26:0/2208250659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4031960 con 0x7f6e0414f360 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2208250659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 msgr2=0x7f6e04155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2208250659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e04155c10 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f6df40097a0 tx=0x7f6df402b840 comp rx=0 tx=0).stop 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2208250659 shutdown_connections 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2208250659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e040c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2208250659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e04155c10 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2208250659 >> 172.21.15.26:0/2208250659 conn(0x7f6e040ac570 msgr2=0x7f6e040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2208250659 shutdown_connections 2024-03-06T07:29:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.585+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2208250659 wait complete. 2024-03-06T07:29:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 Processor -- start 2024-03-06T07:29:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 -- start start 2024-03-06T07:29:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e040c5880 0x7f6e041ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e041546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e040af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04159270 con 0x7f6e0414f360 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040c7c30 con 0x7f6e040c5880 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e08c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e041593f0 con 0x7f6e040c88c0 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e040af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e040af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34602/0 (socket says 172.21.15.26:34602) 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.587+0000 7f6e02d9d640 1 -- 172.21.15.26:0/2892681129 learned_addr learned my addr 172.21.15.26:0/2892681129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e0259c640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e040c5880 0x7f6e041ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6dfbfff640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e041546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e02d9d640 1 -- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 msgr2=0x7f6e041546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e041546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e02d9d640 1 -- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e040c5880 msgr2=0x7f6e041ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e040c5880 0x7f6e041ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e02d9d640 1 -- 172.21.15.26:0/2892681129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e040c03b0 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e0259c640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e040c5880 0x7f6e041ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e040af5e0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c7a0 tx=0x7f6dfc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6dfaffd640 1 -- 172.21.15.26:0/2892681129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc018070 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6dfaffd640 1 -- 172.21.15.26:0/2892681129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc00a640 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6dfaffd640 1 -- 172.21.15.26:0/2892681129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00aab0 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e040c11e0 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.588+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e040c0670 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.589+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0404f330 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.589+0000 7f6dfaffd640 1 -- 172.21.15.26:0/2892681129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6dfc01b070 con 0x7f6e0414f360 2024-03-06T07:29:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.590+0000 7f6dfaffd640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6de406b4d0 0x7f6de406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.590+0000 7f6dfaffd640 1 -- 172.21.15.26:0/2892681129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6dfc0899b0 con 0x7f6e0414f360 2024-03-06T07:29:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.590+0000 7f6e0259c640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6de406b4d0 0x7f6de406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.591+0000 7f6e0259c640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6de406b4d0 0x7f6de406d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6df4036000 tx=0x7f6df4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:33.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.598+0000 7f6dfaffd640 1 -- 172.21.15.26:0/2892681129 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6dfc0586a0 con 0x7f6e0414f360 2024-03-06T07:29:33.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.821+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e0406cda0 con 0x7f6e0414f360 2024-03-06T07:29:33.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.822+0000 7f6dfaffd640 1 -- 172.21.15.26:0/2892681129 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f6dfc057fa0 con 0x7f6e0414f360 2024-03-06T07:29:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:33.823 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:33.824 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:33.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.823+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6de406b4d0 msgr2=0x7f6de406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:33.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.823+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6de406b4d0 0x7f6de406d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6df4036000 tx=0x7f6df4008030 comp rx=0 tx=0).stop 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.823+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 msgr2=0x7f6e040af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.823+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e040af5e0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c7a0 tx=0x7f6dfc00cc60 comp rx=0 tx=0).stop 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 shutdown_connections 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6de406b4d0 0x7f6de406d990 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0414f360 0x7f6e040af5e0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e040c88c0 0x7f6e041546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 --2- 172.21.15.26:0/2892681129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e040c5880 0x7f6e041ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 >> 172.21.15.26:0/2892681129 conn(0x7f6e040ac570 msgr2=0x7f6e0414d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 shutdown_connections 2024-03-06T07:29:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:33.824+0000 7f6e08c2a640 1 -- 172.21.15.26:0/2892681129 wait complete. 2024-03-06T07:29:34.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:34.239 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:35.240 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:35.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.340+0000 7f7c4f439640 1 Processor -- start 2024-03-06T07:29:35.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.340+0000 7f7c4f439640 1 -- start start 2024-03-06T07:29:35.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.341+0000 7f7c4f439640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7c480c5480 0x7f7c48155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.341+0000 7f7c4f439640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5860 0x7f7c4806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.341+0000 7f7c4f439640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c481586f0 0x7f7c48158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.341+0000 7f7c4f439640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4806df40 con 0x7f7c481586f0 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.341+0000 7f7c4f439640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c480c5cc0 con 0x7f7c480c5480 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.341+0000 7f7c4f439640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481591c0 con 0x7f7c480c5860 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c481586f0 0x7f7c48158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c481586f0 0x7f7c48158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34604/0 (socket says 172.21.15.26:34604) 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/963242591 learned_addr learned my addr 172.21.15.26:0/963242591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d1d5640 1 --2- 172.21.15.26:0/963242591 >> v2:172.21.15.117:3300/0 conn(0x7f7c480c5480 0x7f7c48155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4c9d4640 1 --2- 172.21.15.26:0/963242591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5860 0x7f7c4806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/963242591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5860 msgr2=0x7f7c4806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 --2- 172.21.15.26:0/963242591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5860 0x7f7c4806f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/963242591 >> v2:172.21.15.117:3300/0 conn(0x7f7c480c5480 msgr2=0x7f7c48155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 --2- 172.21.15.26:0/963242591 >> v2:172.21.15.117:3300/0 conn(0x7f7c480c5480 0x7f7c48155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.342+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/963242591 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c481594a0 con 0x7f7c481586f0 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.343+0000 7f7c4d9d6640 1 --2- 172.21.15.26:0/963242591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c48158b80 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f7c480c1cf0 tx=0x7f7c4002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb740d4532598620 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.343+0000 7f7c3ffff640 1 -- 172.21.15.26:0/963242591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4002c9e0 con 0x7f7c481586f0 2024-03-06T07:29:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.343+0000 7f7c3ffff640 1 -- 172.21.15.26:0/963242591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4002cb80 con 0x7f7c481586f0 2024-03-06T07:29:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.343+0000 7f7c3ffff640 1 -- 172.21.15.26:0/963242591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c400343f0 con 0x7f7c481586f0 2024-03-06T07:29:35.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 -- 172.21.15.26:0/963242591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 msgr2=0x7f7c48158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 --2- 172.21.15.26:0/963242591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c48158b80 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f7c480c1cf0 tx=0x7f7c4002be40 comp rx=0 tx=0).stop 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 -- 172.21.15.26:0/963242591 shutdown_connections 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 --2- 172.21.15.26:0/963242591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5860 0x7f7c4806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 --2- 172.21.15.26:0/963242591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c48158b80 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 --2- 172.21.15.26:0/963242591 >> v2:172.21.15.117:3300/0 conn(0x7f7c480c5480 0x7f7c48155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 -- 172.21.15.26:0/963242591 >> 172.21.15.26:0/963242591 conn(0x7f7c4806b1a0 msgr2=0x7f7c4806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 -- 172.21.15.26:0/963242591 shutdown_connections 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.344+0000 7f7c4f439640 1 -- 172.21.15.26:0/963242591 wait complete. 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 Processor -- start 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 -- start start 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5480 0x7f7c480a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c480c5860 0x7f7c4814aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c4814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c48152a20 con 0x7f7c481586f0 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c48155680 con 0x7f7c480c5860 2024-03-06T07:29:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.345+0000 7f7c4f439640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c48155050 con 0x7f7c480c5480 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c4814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c4814b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34616/0 (socket says 172.21.15.26:34616) 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/4226691470 learned_addr learned my addr 172.21.15.26:0/4226691470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4c9d4640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c480c5860 0x7f7c4814aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d1d5640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5480 0x7f7c480a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5480 msgr2=0x7f7c480a7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5480 0x7f7c480a7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c480c5860 msgr2=0x7f7c4814aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c480c5860 0x7f7c4814aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 -- 172.21.15.26:0/4226691470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c481eebe0 con 0x7f7c481586f0 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4d9d6640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c4814b030 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009f40 tx=0x7f7c40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c3dffb640 1 -- 172.21.15.26:0/4226691470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40044020 con 0x7f7c481586f0 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c3dffb640 1 -- 172.21.15.26:0/4226691470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c40040670 con 0x7f7c481586f0 2024-03-06T07:29:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c480ab930 con 0x7f7c481586f0 2024-03-06T07:29:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c3dffb640 1 -- 172.21.15.26:0/4226691470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40009510 con 0x7f7c481586f0 2024-03-06T07:29:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.346+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c481ee8d0 con 0x7f7c481586f0 2024-03-06T07:29:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.347+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4804f330 con 0x7f7c481586f0 2024-03-06T07:29:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.348+0000 7f7c3dffb640 1 -- 172.21.15.26:0/4226691470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c40047020 con 0x7f7c481586f0 2024-03-06T07:29:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.348+0000 7f7c3dffb640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.348+0000 7f7c3dffb640 1 -- 172.21.15.26:0/4226691470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c400acdb0 con 0x7f7c481586f0 2024-03-06T07:29:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.349+0000 7f7c4d1d5640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.349+0000 7f7c4d1d5640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7c44006fd0 tx=0x7f7c44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:35.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.352+0000 7f7c3dffb640 1 -- 172.21.15.26:0/4226691470 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c4004e030 con 0x7f7c481586f0 2024-03-06T07:29:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.584+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c480c2670 con 0x7f7c481586f0 2024-03-06T07:29:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.584+0000 7f7c3dffb640 1 -- 172.21.15.26:0/4226691470 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f7c4007bc50 con 0x7f7c481586f0 2024-03-06T07:29:35.586 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:35.586 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2c06b4d0 msgr2=0x7f7c2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7c44006fd0 tx=0x7f7c44008040 comp rx=0 tx=0).stop 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 msgr2=0x7f7c4814b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c4814b030 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009f40 tx=0x7f7c40004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 shutdown_connections 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c481586f0 0x7f7c4814b030 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c480c5860 0x7f7c4814aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 --2- 172.21.15.26:0/4226691470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c480c5480 0x7f7c480a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 >> 172.21.15.26:0/4226691470 conn(0x7f7c4806b1a0 msgr2=0x7f7c4806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:35.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 shutdown_connections 2024-03-06T07:29:35.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:35.587+0000 7f7c4f439640 1 -- 172.21.15.26:0/4226691470 wait complete. 2024-03-06T07:29:36.001 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:36.002 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:37.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.103+0000 7fb048b02640 1 Processor -- start 2024-03-06T07:29:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.103+0000 7fb048b02640 1 -- start start 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.103+0000 7fb048b02640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0441506b0 0x7fb044153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.103+0000 7fb048b02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb0440c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb048b02640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb044070aa0 con 0x7fb044156ea0 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb048b02640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb044156450 con 0x7fb0441506b0 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb048b02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb044156a10 con 0x7fb0440acc00 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb04259c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0441506b0 0x7fb044153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb0440c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb0440c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:49648/0 (socket says 172.21.15.26:49648) 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb04259c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0441506b0 0x7fb044153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57592/0 (socket says 172.21.15.26:57592) 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb042d9d640 1 -- 172.21.15.26:0/649356125 learned_addr learned my addr 172.21.15.26:0/649356125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb041d9b640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb044156ea0 0x7fb0440702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:36064/0 (socket says 172.21.15.26:36064) 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb042d9d640 1 -- 172.21.15.26:0/649356125 >> v2:172.21.15.117:3300/0 conn(0x7fb0441506b0 msgr2=0x7fb044153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb042d9d640 1 --2- 172.21.15.26:0/649356125 >> v2:172.21.15.117:3300/0 conn(0x7fb0441506b0 0x7fb044153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb042d9d640 1 -- 172.21.15.26:0/649356125 >> v1:172.21.15.117:6789/0 conn(0x7fb044156ea0 legacy=0x7fb0440702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.104+0000 7fb042d9d640 1 -- 172.21.15.26:0/649356125 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044070d80 con 0x7fb0440acc00 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.105+0000 7fb042d9d640 1 --2- 172.21.15.26:0/649356125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb0440c54a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009e50 tx=0x7fb03c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7b8757fdf9e380 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.105+0000 7fb04159a640 1 -- 172.21.15.26:0/649356125 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c02cb00 con 0x7fb0440acc00 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.105+0000 7fb04159a640 1 -- 172.21.15.26:0/649356125 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c02cca0 con 0x7fb0440acc00 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.105+0000 7fb04159a640 1 -- 172.21.15.26:0/649356125 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c034610 con 0x7fb0440acc00 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 -- 172.21.15.26:0/649356125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 msgr2=0x7fb0440c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 --2- 172.21.15.26:0/649356125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb0440c54a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009e50 tx=0x7fb03c02bf60 comp rx=0 tx=0).stop 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 -- 172.21.15.26:0/649356125 shutdown_connections 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 --2- 172.21.15.26:0/649356125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb0440c54a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 --2- 172.21.15.26:0/649356125 >> v2:172.21.15.117:3300/0 conn(0x7fb0441506b0 0x7fb044153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 -- 172.21.15.26:0/649356125 >> 172.21.15.26:0/649356125 conn(0x7fb04406b1a0 msgr2=0x7fb04406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:37.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 -- 172.21.15.26:0/649356125 shutdown_connections 2024-03-06T07:29:37.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.106+0000 7fb048b02640 1 -- 172.21.15.26:0/649356125 wait complete. 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.107+0000 7fb048b02640 1 Processor -- start 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.107+0000 7fb048b02640 1 -- start start 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb048b02640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb044155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb048b02640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0441506b0 0x7fb0441eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb048b02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb044156ea0 0x7fb0441ee810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb048b02640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440ad1d0 con 0x7fb044156ea0 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb048b02640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440ad050 con 0x7fb0441506b0 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb048b02640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440ad350 con 0x7fb0440acc00 2024-03-06T07:29:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb044156ea0 0x7fb0441ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb044156ea0 0x7fb0441ee810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60118/0 (socket says 172.21.15.26:60118) 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 -- 172.21.15.26:0/2635121539 learned_addr learned my addr 172.21.15.26:0/2635121539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb041d9b640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0441506b0 0x7fb0441eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb04259c640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb044155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 -- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 msgr2=0x7fb044155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb044155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 -- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0441506b0 msgr2=0x7fb0441eaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0441506b0 0x7fb0441eaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.108+0000 7fb042d9d640 1 -- 172.21.15.26:0/2635121539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044150d60 con 0x7fb044156ea0 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.109+0000 7fb042d9d640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb044156ea0 0x7fb0441ee810 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fb03c036040 tx=0x7fb03c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.109+0000 7fb032ffd640 1 -- 172.21.15.26:0/2635121539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c043020 con 0x7fb044156ea0 2024-03-06T07:29:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.109+0000 7fb032ffd640 1 -- 172.21.15.26:0/2635121539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c02ce50 con 0x7fb044156ea0 2024-03-06T07:29:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.109+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04406dae0 con 0x7fb044156ea0 2024-03-06T07:29:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.109+0000 7fb032ffd640 1 -- 172.21.15.26:0/2635121539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c009510 con 0x7fb044156ea0 2024-03-06T07:29:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.109+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04406d800 con 0x7fb044156ea0 2024-03-06T07:29:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.110+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04404f330 con 0x7fb044156ea0 2024-03-06T07:29:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.110+0000 7fb032ffd640 1 -- 172.21.15.26:0/2635121539 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb03c046020 con 0x7fb044156ea0 2024-03-06T07:29:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.111+0000 7fb032ffd640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb02406b4d0 0x7fb02406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.111+0000 7fb032ffd640 1 -- 172.21.15.26:0/2635121539 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb03c0ac730 con 0x7fb044156ea0 2024-03-06T07:29:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.111+0000 7fb04259c640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb02406b4d0 0x7fb02406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.112+0000 7fb04259c640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb02406b4d0 0x7fb02406d990 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb034009b40 tx=0x7fb034009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:37.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.115+0000 7fb032ffd640 1 -- 172.21.15.26:0/2635121539 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb03c04d030 con 0x7fb044156ea0 2024-03-06T07:29:37.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.336+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0440700a0 con 0x7fb044156ea0 2024-03-06T07:29:37.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.337+0000 7fb032ffd640 1 -- 172.21.15.26:0/2635121539 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fb03c07adc0 con 0x7fb044156ea0 2024-03-06T07:29:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:37.338 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:37.339 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:37.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb02406b4d0 msgr2=0x7fb02406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb02406b4d0 0x7fb02406d990 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb034009b40 tx=0x7fb034009190 comp rx=0 tx=0).stop 2024-03-06T07:29:37.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb044156ea0 msgr2=0x7fb0441ee810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb044156ea0 0x7fb0441ee810 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fb03c036040 tx=0x7fb03c004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 shutdown_connections 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb02406b4d0 0x7fb02406d990 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb044156ea0 0x7fb0441ee810 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0441506b0 0x7fb0441eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 --2- 172.21.15.26:0/2635121539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0440acc00 0x7fb044155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.338+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 >> 172.21.15.26:0/2635121539 conn(0x7fb04406b1a0 msgr2=0x7fb0440c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.339+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 shutdown_connections 2024-03-06T07:29:37.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:37.339+0000 7fb048b02640 1 -- 172.21.15.26:0/2635121539 wait complete. 2024-03-06T07:29:37.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:37.754 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:38.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.855+0000 7f4213284640 1 Processor -- start 2024-03-06T07:29:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.855+0000 7f4213284640 1 -- start start 2024-03-06T07:29:38.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.856+0000 7f4213284640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 0x7f420c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.856+0000 7f4213284640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420c0c73d0 con 0x7f420c0c88c0 2024-03-06T07:29:38.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.856+0000 7f4213284640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420c0be2e0 con 0x7f420c158b60 2024-03-06T07:29:38.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.856+0000 7f4213284640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c159700 con 0x7f420c14f360 2024-03-06T07:29:38.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.856+0000 7f421081f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f420c0c88c0 0x7f420c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46340/0 (socket says 172.21.15.26:46340) 2024-03-06T07:29:38.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.856+0000 7f421081f640 1 -- 172.21.15.26:0/216183485 learned_addr learned my addr 172.21.15.26:0/216183485 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4211020640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 0x7f420c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806226066 0 0) 0x7f420c0c73d0 con 0x7f420c0c88c0 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f8003520 con 0x7f420c0c88c0 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414752116 0 0) 0x7f420c0be2e0 con 0x7f420c158b60 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f420c0c73d0 con 0x7f420c158b60 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326662854 0 0) 0x7f41f8003520 con 0x7f420c0c88c0 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 msgr2=0x7f420c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 0x7f420c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 >> v1:172.21.15.117:6789/0 conn(0x7f420c158b60 legacy=0x7f420c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:38.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c0c7670 con 0x7f420c0c88c0 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f41fc001670 con 0x7f420c0c88c0 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4211020640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 0x7f420c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 >> v1:172.21.15.26:6789/0 conn(0x7f420c0c88c0 legacy=0x7f420c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8003b70 0x7f41f8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 0x7f41f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f8004d40 0x7f41f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c159700 con 0x7f41f8003b70 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f800bad0 con 0x7f41f8004d40 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f800bcd0 con 0x7f420c14f360 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4211020640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8003b70 0x7f41f8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f421081f640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 0x7f41f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.857+0000 7f4211821640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f8004d40 0x7f41f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4211020640 1 -- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 msgr2=0x7f41f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4211020640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f420c14f360 0x7f41f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4211020640 1 -- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f8004d40 msgr2=0x7f41f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4211020640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f8004d40 0x7f41f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4211020640 1 -- 172.21.15.26:0/216183485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f8003520 con 0x7f41f8003b70 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4211821640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f8004d40 0x7f41f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4211020640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8003b70 0x7f41f8003f70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f4208009c90 tx=0x7f41f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5f353de9c6df00 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.858+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208039070 con 0x7f41f8003b70 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420802c8f0 con 0x7f41f8003b70 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4203fff640 1 -- 172.21.15.26:0/216183485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208009860 con 0x7f41f8003b70 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4213284640 1 -- 172.21.15.26:0/216183485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8003b70 msgr2=0x7f41f8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4213284640 1 --2- 172.21.15.26:0/216183485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8003b70 0x7f41f8003f70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f4208009c90 tx=0x7f41f8005830 comp rx=0 tx=0).stop 2024-03-06T07:29:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4211821640 1 -- 172.21.15.26:0/216183485 reap_dead start 2024-03-06T07:29:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4213284640 1 -- 172.21.15.26:0/216183485 shutdown_connections 2024-03-06T07:29:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4213284640 1 -- 172.21.15.26:0/216183485 >> 172.21.15.26:0/216183485 conn(0x7f420c0ac570 msgr2=0x7f420c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4213284640 1 -- 172.21.15.26:0/216183485 shutdown_connections 2024-03-06T07:29:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.859+0000 7f4213284640 1 -- 172.21.15.26:0/216183485 wait complete. 2024-03-06T07:29:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.860+0000 7f4213284640 1 Processor -- start 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.860+0000 7f4213284640 1 -- start start 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4213284640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41f8003b70 0x7f420c1ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4213284640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420c0ad390 0x7f420c14bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4213284640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f420c1eeaf0 0x7f420c14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4213284640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0c7c30 con 0x7f420c1eeaf0 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4213284640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0c5a50 con 0x7f420c0ad390 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4213284640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0c5bd0 con 0x7f41f8003b70 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f420c1eeaf0 0x7f420c14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f420c1eeaf0 0x7f420c14c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60144/0 (socket says 172.21.15.26:60144) 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 -- 172.21.15.26:0/650769588 learned_addr learned my addr 172.21.15.26:0/650769588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211020640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41f8003b70 0x7f420c1ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f421081f640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420c0ad390 0x7f420c14bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 -- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41f8003b70 msgr2=0x7f420c1ee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41f8003b70 0x7f420c1ee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 -- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420c0ad390 msgr2=0x7f420c14bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420c0ad390 0x7f420c14bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 -- 172.21.15.26:0/650769588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c1ef030 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.861+0000 7f4211821640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f420c1eeaf0 0x7f420c14c2c0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f420c158f40 tx=0x7f420400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.862+0000 7f4201ffb640 1 -- 172.21.15.26:0/650769588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400ee50 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.862+0000 7f4201ffb640 1 -- 172.21.15.26:0/650769588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4204003b80 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.862+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420c1522b0 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.862+0000 7f4201ffb640 1 -- 172.21.15.26:0/650769588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204006b50 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.862+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c1515f0 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.863+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c04f330 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.863+0000 7f4201ffb640 1 -- 172.21.15.26:0/650769588 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4204006cf0 con 0x7f420c1eeaf0 2024-03-06T07:29:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.864+0000 7f4201ffb640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f006b4d0 0x7f41f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.864+0000 7f4201ffb640 1 -- 172.21.15.26:0/650769588 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4204089f40 con 0x7f420c1eeaf0 2024-03-06T07:29:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.864+0000 7f4211020640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f006b4d0 0x7f41f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.865+0000 7f4211020640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f006b4d0 0x7f41f006d990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4208036000 tx=0x7f4208030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:38.868+0000 7f4201ffb640 1 -- 172.21.15.26:0/650769588 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4204058cc0 con 0x7f420c1eeaf0 2024-03-06T07:29:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.175+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f420c0c8d50 con 0x7f420c1eeaf0 2024-03-06T07:29:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.175+0000 7f4201ffb640 1 -- 172.21.15.26:0/650769588 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f42040585c0 con 0x7f420c1eeaf0 2024-03-06T07:29:39.177 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:39.177 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.177+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f006b4d0 msgr2=0x7f41f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.177+0000 7f4213284640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f006b4d0 0x7f41f006d990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4208036000 tx=0x7f4208030000 comp rx=0 tx=0).stop 2024-03-06T07:29:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.177+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f420c1eeaf0 msgr2=0x7f420c14c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.177+0000 7f4213284640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f420c1eeaf0 0x7f420c14c2c0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f420c158f40 tx=0x7f420400ec00 comp rx=0 tx=0).stop 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.177+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 shutdown_connections 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.178+0000 7f4213284640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f41f006b4d0 0x7f41f006d990 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.178+0000 7f4213284640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f420c1eeaf0 0x7f420c14c2c0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.178+0000 7f4213284640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420c0ad390 0x7f420c14bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.178+0000 7f4213284640 1 --2- 172.21.15.26:0/650769588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41f8003b70 0x7f420c1ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.178+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 >> 172.21.15.26:0/650769588 conn(0x7f420c0ac570 msgr2=0x7f420c06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.178+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 shutdown_connections 2024-03-06T07:29:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:39.178+0000 7f4213284640 1 -- 172.21.15.26:0/650769588 wait complete. 2024-03-06T07:29:39.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:39.596 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:40.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.696+0000 7f43706fa640 1 Processor -- start 2024-03-06T07:29:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.696+0000 7f43706fa640 1 -- start start 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.697+0000 7f43706fa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 0x7f436815d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.697+0000 7f43706fa640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4368072440 con 0x7f436814f650 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.697+0000 7f43706fa640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43680c2b10 con 0x7f43680c3150 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.697+0000 7f43706fa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4368152140 con 0x7f4368152c20 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.697+0000 7f436e496640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f436814f650 0x7f4368071880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46348/0 (socket says 172.21.15.26:46348) 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.697+0000 7f436e496640 1 -- 172.21.15.26:0/3274706943 learned_addr learned my addr 172.21.15.26:0/3274706943 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405759913 0 0) 0x7f4360001110 con 0x7f436814f650 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4350003520 con 0x7f436814f650 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436ec97640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 0x7f436815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2304205932 0 0) 0x7f4350003520 con 0x7f436814f650 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 msgr2=0x7f436815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 0x7f436815d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 >> v1:172.21.15.117:6789/0 conn(0x7f43680c3150 legacy=0x7f43680c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4368152420 con 0x7f436814f650 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4360001840 con 0x7f436814f650 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 >> v1:172.21.15.26:6789/0 conn(0x7f436814f650 legacy=0x7f4368071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43500039e0 0x7f4350003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 0x7f4350004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4350004d40 0x7f43500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368152140 con 0x7f43500039e0 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435000dce0 con 0x7f4350004d40 2024-03-06T07:29:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435000dee0 con 0x7f4368152c20 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436e496640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43500039e0 0x7f4350003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436dc95640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 0x7f4350004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436ec97640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4350004d40 0x7f43500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.698+0000 7f436e496640 1 -- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 msgr2=0x7f4350004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436e496640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368152c20 0x7f4350004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436e496640 1 -- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4350004d40 msgr2=0x7f43500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436e496640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4350004d40 0x7f43500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436e496640 1 -- 172.21.15.26:0/3274706943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350003520 con 0x7f43500039e0 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436ec97640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4350004d40 0x7f43500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436e496640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43500039e0 0x7f4350003e00 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f436000c2f0 tx=0x7f4350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9367834697562c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360007e10 con 0x7f43500039e0 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360030070 con 0x7f43500039e0 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.699+0000 7f436d494640 1 -- 172.21.15.26:0/3274706943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436002ce30 con 0x7f43500039e0 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.700+0000 7f43706fa640 1 -- 172.21.15.26:0/3274706943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43500039e0 msgr2=0x7f4350003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.700+0000 7f43706fa640 1 --2- 172.21.15.26:0/3274706943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43500039e0 0x7f4350003e00 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f436000c2f0 tx=0x7f4350005830 comp rx=0 tx=0).stop 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.700+0000 7f436ec97640 1 -- 172.21.15.26:0/3274706943 reap_dead start 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.700+0000 7f43706fa640 1 -- 172.21.15.26:0/3274706943 shutdown_connections 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.700+0000 7f43706fa640 1 -- 172.21.15.26:0/3274706943 >> 172.21.15.26:0/3274706943 conn(0x7f43680c4290 msgr2=0x7f43680c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.700+0000 7f43706fa640 1 -- 172.21.15.26:0/3274706943 shutdown_connections 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.700+0000 7f43706fa640 1 -- 172.21.15.26:0/3274706943 wait complete. 2024-03-06T07:29:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.701+0000 7f43706fa640 1 Processor -- start 2024-03-06T07:29:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.701+0000 7f43706fa640 1 -- start start 2024-03-06T07:29:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f43706fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43500039e0 0x7f436806eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f43706fa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368149e70 0x7f43681dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f43706fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f436814aec0 0x7f43681e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f43706fa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368072110 con 0x7f436814aec0 2024-03-06T07:29:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f43706fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368071f90 con 0x7f43500039e0 2024-03-06T07:29:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f43706fa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43680c3f40 con 0x7f4368149e70 2024-03-06T07:29:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f436814aec0 0x7f43681e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f436814aec0 0x7f43681e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60162/0 (socket says 172.21.15.26:60162) 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 -- 172.21.15.26:0/914285745 learned_addr learned my addr 172.21.15.26:0/914285745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436dc95640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368149e70 0x7f43681dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436e496640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43500039e0 0x7f436806eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 -- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368149e70 msgr2=0x7f43681dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368149e70 0x7f43681dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 -- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43500039e0 msgr2=0x7f436806eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43500039e0 0x7f436806eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 -- 172.21.15.26:0/914285745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436814a340 con 0x7f436814aec0 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436dc95640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368149e70 0x7f43681dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436e496640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43500039e0 0x7f436806eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.702+0000 7f436ec97640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f436814aec0 0x7f43681e6b70 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f43680c76e0 tx=0x7f4358010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.703+0000 7f4366ffd640 1 -- 172.21.15.26:0/914285745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358003990 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.703+0000 7f4366ffd640 1 -- 172.21.15.26:0/914285745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358003b50 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.703+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4368149b90 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.703+0000 7f4366ffd640 1 -- 172.21.15.26:0/914285745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358004020 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.703+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43681dde30 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.704+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368067ad0 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.705+0000 7f4366ffd640 1 -- 172.21.15.26:0/914285745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f435801e030 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.705+0000 7f4366ffd640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f435406b500 0x7f435406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.705+0000 7f4366ffd640 1 -- 172.21.15.26:0/914285745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f435808aa40 con 0x7f436814aec0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.705+0000 7f436e496640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f435406b500 0x7f435406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.706+0000 7f436e496640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f435406b500 0x7f435406d9c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4360034000 tx=0x7f4360009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:40.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.709+0000 7f4366ffd640 1 -- 172.21.15.26:0/914285745 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4368067ad0 con 0x7f436814aec0 2024-03-06T07:29:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.928+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f436804f330 con 0x7f436814aec0 2024-03-06T07:29:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.929+0000 7f4366ffd640 1 -- 172.21.15.26:0/914285745 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f4358025020 con 0x7f436814aec0 2024-03-06T07:29:40.931 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:40.931 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f435406b500 msgr2=0x7f435406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f435406b500 0x7f435406d9c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4360034000 tx=0x7f4360009030 comp rx=0 tx=0).stop 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f436814aec0 msgr2=0x7f43681e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f436814aec0 0x7f43681e6b70 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f43680c76e0 tx=0x7f4358010590 comp rx=0 tx=0).stop 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 shutdown_connections 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f435406b500 0x7f435406d9c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f436814aec0 0x7f43681e6b70 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4368149e70 0x7f43681dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 --2- 172.21.15.26:0/914285745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43500039e0 0x7f436806eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 >> 172.21.15.26:0/914285745 conn(0x7f43680c4290 msgr2=0x7f43680a7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 shutdown_connections 2024-03-06T07:29:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:40.931+0000 7f43706fa640 1 -- 172.21.15.26:0/914285745 wait complete. 2024-03-06T07:29:41.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:41.346 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:42.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:42.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.447+0000 7f0c478a4640 1 Processor -- start 2024-03-06T07:29:42.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.447+0000 7f0c478a4640 1 -- start start 2024-03-06T07:29:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c478a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c478a4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c40158b60 0x7f0c40158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c478a4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c400c73d0 con 0x7f0c4014f360 2024-03-06T07:29:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c478a4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c400c8dd0 con 0x7f0c40158b60 2024-03-06T07:29:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c478a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400be2e0 con 0x7f0c400c88c0 2024-03-06T07:29:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c45640640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0c4014f360 0x7f0c400c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:36092/0 (socket says 172.21.15.26:36092) 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c45640640 1 -- 172.21.15.26:0/631673534 learned_addr learned my addr 172.21.15.26:0/631673534 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c44e3f640 1 --2- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c45e41640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c40158b60 0x7f0c40158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c37fff640 1 -- 172.21.15.26:0/631673534 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271614947 0 0) 0x7f0c400c73d0 con 0x7f0c4014f360 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c37fff640 1 -- 172.21.15.26:0/631673534 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c2c003520 con 0x7f0c4014f360 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c45e41640 1 -- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 msgr2=0x7f0c40155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c45e41640 1 --2- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c45e41640 1 -- 172.21.15.26:0/631673534 >> v1:172.21.15.117:6789/0 conn(0x7f0c4014f360 legacy=0x7f0c400c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c45e41640 1 -- 172.21.15.26:0/631673534 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c400c7670 con 0x7f0c40158b60 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.448+0000 7f0c44e3f640 1 --2- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c45e41640 1 --2- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 0x7f0c40158ff0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009d90 tx=0x7f0c3802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18de3379649268c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c37fff640 1 -- 172.21.15.26:0/631673534 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38039070 con 0x7f0c40158b60 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c37fff640 1 -- 172.21.15.26:0/631673534 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3802cbc0 con 0x7f0c40158b60 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c37fff640 1 -- 172.21.15.26:0/631673534 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38034440 con 0x7f0c40158b60 2024-03-06T07:29:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c478a4640 1 -- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 msgr2=0x7f0c40158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c478a4640 1 --2- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 0x7f0c40158ff0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009d90 tx=0x7f0c3802be10 comp rx=0 tx=0).stop 2024-03-06T07:29:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c478a4640 1 -- 172.21.15.26:0/631673534 shutdown_connections 2024-03-06T07:29:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c478a4640 1 --2- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 0x7f0c40158ff0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c478a4640 1 --2- 172.21.15.26:0/631673534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.450+0000 7f0c478a4640 1 -- 172.21.15.26:0/631673534 >> 172.21.15.26:0/631673534 conn(0x7f0c400ac570 msgr2=0x7f0c400ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.451+0000 7f0c478a4640 1 -- 172.21.15.26:0/631673534 shutdown_connections 2024-03-06T07:29:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.451+0000 7f0c478a4640 1 -- 172.21.15.26:0/631673534 wait complete. 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 Processor -- start 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 -- start start 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c4014f360 0x7f0c400bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 0x7f0c4014ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400c5a50 con 0x7f0c4014f360 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400c7c30 con 0x7f0c40158b60 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c478a4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400c5bd0 con 0x7f0c400c88c0 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c44e3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c4014f360 0x7f0c400bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c44e3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c4014f360 0x7f0c400bfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60176/0 (socket says 172.21.15.26:60176) 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c44e3f640 1 -- 172.21.15.26:0/2270449074 learned_addr learned my addr 172.21.15.26:0/2270449074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c45e41640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 0x7f0c4014ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.452+0000 7f0c45640640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c44e3f640 1 -- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 msgr2=0x7f0c40157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c44e3f640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c44e3f640 1 -- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 msgr2=0x7f0c4014ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c44e3f640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 0x7f0c4014ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c44e3f640 1 -- 172.21.15.26:0/2270449074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c400c1390 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c45640640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c44e3f640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c4014f360 0x7f0c400bfb60 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000e930 tx=0x7f0c3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c35ffb640 1 -- 172.21.15.26:0/2270449074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30017070 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c35ffb640 1 -- 172.21.15.26:0/2270449074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3000ca00 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c400ae5a0 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c35ffb640 1 -- 172.21.15.26:0/2270449074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000cdf0 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.453+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c400b0250 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.454+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c40152f20 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.455+0000 7f0c35ffb640 1 -- 172.21.15.26:0/2270449074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c3001a070 con 0x7f0c4014f360 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.455+0000 7f0c35ffb640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c2406b4d0 0x7f0c2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.455+0000 7f0c35ffb640 1 -- 172.21.15.26:0/2270449074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c3008a120 con 0x7f0c4014f360 2024-03-06T07:29:42.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.455+0000 7f0c45640640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c2406b4d0 0x7f0c2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:42.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.456+0000 7f0c45640640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c2406b4d0 0x7f0c2406d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c007fd0 tx=0x7f0c3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:42.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.459+0000 7f0c35ffb640 1 -- 172.21.15.26:0/2270449074 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0c40152f20 con 0x7f0c4014f360 2024-03-06T07:29:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.678+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c4006cda0 con 0x7f0c4014f360 2024-03-06T07:29:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.679+0000 7f0c35ffb640 1 -- 172.21.15.26:0/2270449074 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f0c30058850 con 0x7f0c4014f360 2024-03-06T07:29:42.681 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:42.681 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c2406b4d0 msgr2=0x7f0c2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c2406b4d0 0x7f0c2406d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c007fd0 tx=0x7f0c3c009040 comp rx=0 tx=0).stop 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c4014f360 msgr2=0x7f0c400bfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c4014f360 0x7f0c400bfb60 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000e930 tx=0x7f0c3000edf0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 shutdown_connections 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0c2406b4d0 0x7f0c2406d990 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c40158b60 0x7f0c4014ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c4014f360 0x7f0c400bfb60 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 --2- 172.21.15.26:0/2270449074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c400c88c0 0x7f0c40157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:42.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 >> 172.21.15.26:0/2270449074 conn(0x7f0c400ac570 msgr2=0x7f0c400ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 shutdown_connections 2024-03-06T07:29:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:42.681+0000 7f0c478a4640 1 -- 172.21.15.26:0/2270449074 wait complete. 2024-03-06T07:29:43.099 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:43.099 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:43.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:43.125 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:44.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:44.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.201+0000 7fe885132640 1 Processor -- start 2024-03-06T07:29:44.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.201+0000 7fe885132640 1 -- start start 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe885132640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8800724b0 0x7fe880159090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe885132640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8800c67f0 0x7fe880072890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe885132640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8800a7e50 con 0x7fe8800c3940 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe885132640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800a8410 con 0x7fe8800724b0 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe885132640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800a8130 con 0x7fe8800c67f0 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8800724b0 0x7fe880159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe877fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe8800c3940 0x7fe880155090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46362/0 (socket says 172.21.15.26:46362) 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8800724b0 0x7fe880159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60188/0 (socket says 172.21.15.26:60188) 2024-03-06T07:29:44.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe877fff640 1 -- 172.21.15.26:0/4101048628 learned_addr learned my addr 172.21.15.26:0/4101048628 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:44.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87f59e640 1 --2- 172.21.15.26:0/4101048628 >> v2:172.21.15.117:3300/0 conn(0x7fe8800c67f0 0x7fe880072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87ed9d640 1 -- 172.21.15.26:0/4101048628 >> v2:172.21.15.117:3300/0 conn(0x7fe8800c67f0 msgr2=0x7fe880072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:44.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87ed9d640 1 --2- 172.21.15.26:0/4101048628 >> v2:172.21.15.117:3300/0 conn(0x7fe8800c67f0 0x7fe880072890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87ed9d640 1 -- 172.21.15.26:0/4101048628 >> v1:172.21.15.26:6789/0 conn(0x7fe8800c3940 legacy=0x7fe880155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87ed9d640 1 -- 172.21.15.26:0/4101048628 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8800c3420 con 0x7fe8800724b0 2024-03-06T07:29:44.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.202+0000 7fe87e59c640 1 -- 172.21.15.26:0/4101048628 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240226399 0 0) 0x7fe8800a7e50 con 0x7fe8800c3940 2024-03-06T07:29:44.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.203+0000 7fe87ed9d640 1 --2- 172.21.15.26:0/4101048628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880159090 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fe880156220 tx=0x7fe87002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ca73bdadc32f39 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:44.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.203+0000 7fe87e59c640 1 -- 172.21.15.26:0/4101048628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870038070 con 0x7fe8800724b0 2024-03-06T07:29:44.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.203+0000 7fe87e59c640 1 -- 172.21.15.26:0/4101048628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870031710 con 0x7fe8800724b0 2024-03-06T07:29:44.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.203+0000 7fe87e59c640 1 -- 172.21.15.26:0/4101048628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870031ae0 con 0x7fe8800724b0 2024-03-06T07:29:44.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.203+0000 7fe885132640 1 -- 172.21.15.26:0/4101048628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 msgr2=0x7fe880159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.203+0000 7fe885132640 1 --2- 172.21.15.26:0/4101048628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880159090 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fe880156220 tx=0x7fe87002b7b0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.204+0000 7fe885132640 1 -- 172.21.15.26:0/4101048628 shutdown_connections 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.204+0000 7fe885132640 1 --2- 172.21.15.26:0/4101048628 >> v2:172.21.15.117:3300/0 conn(0x7fe8800c67f0 0x7fe880072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.204+0000 7fe885132640 1 --2- 172.21.15.26:0/4101048628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880159090 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.204+0000 7fe885132640 1 -- 172.21.15.26:0/4101048628 >> 172.21.15.26:0/4101048628 conn(0x7fe8800aa6a0 msgr2=0x7fe8800aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.204+0000 7fe885132640 1 -- 172.21.15.26:0/4101048628 shutdown_connections 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.204+0000 7fe885132640 1 -- 172.21.15.26:0/4101048628 wait complete. 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 Processor -- start 2024-03-06T07:29:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 -- start start 2024-03-06T07:29:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8800c3940 0x7fe880151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8800c67f0 0x7fe880151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880152ae0 con 0x7fe8800724b0 2024-03-06T07:29:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880152960 con 0x7fe8800c67f0 2024-03-06T07:29:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.205+0000 7fe885132640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8801557a0 con 0x7fe8800c3940 2024-03-06T07:29:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60202/0 (socket says 172.21.15.26:60202) 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 -- 172.21.15.26:0/1372929264 learned_addr learned my addr 172.21.15.26:0/1372929264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe877fff640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8800c3940 0x7fe880151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87f59e640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8800c67f0 0x7fe880151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 -- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8800c3940 msgr2=0x7fe880151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8800c3940 0x7fe880151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 -- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8800c67f0 msgr2=0x7fe880151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8800c67f0 0x7fe880151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 -- 172.21.15.26:0/1372929264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe880152390 con 0x7fe8800724b0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe87ed9d640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880157610 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fe870036040 tx=0x7fe8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe8777fe640 1 -- 172.21.15.26:0/1372929264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870038040 con 0x7fe8800724b0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe8777fe640 1 -- 172.21.15.26:0/1372929264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870004230 con 0x7fe8800724b0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8801de890 con 0x7fe8800724b0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe8777fe640 1 -- 172.21.15.26:0/1372929264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87003d670 con 0x7fe8800724b0 2024-03-06T07:29:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.206+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8800ab8c0 con 0x7fe8800724b0 2024-03-06T07:29:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.207+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88004f330 con 0x7fe8800724b0 2024-03-06T07:29:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.208+0000 7fe8777fe640 1 -- 172.21.15.26:0/1372929264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe87003b070 con 0x7fe8800724b0 2024-03-06T07:29:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.208+0000 7fe8777fe640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe86006b4d0 0x7fe86006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.208+0000 7fe8777fe640 1 -- 172.21.15.26:0/1372929264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8700abb60 con 0x7fe8800724b0 2024-03-06T07:29:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.208+0000 7fe877fff640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe86006b4d0 0x7fe86006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.209+0000 7fe877fff640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe86006b4d0 0x7fe86006d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe8800aa160 tx=0x7fe868007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:44.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.217+0000 7fe8777fe640 1 -- 172.21.15.26:0/1372929264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe87004b070 con 0x7fe8800724b0 2024-03-06T07:29:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.445+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe880159220 con 0x7fe8800724b0 2024-03-06T07:29:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.446+0000 7fe8777fe640 1 -- 172.21.15.26:0/1372929264 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fe870042020 con 0x7fe8800724b0 2024-03-06T07:29:44.448 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:44.448 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe86006b4d0 msgr2=0x7fe86006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe86006b4d0 0x7fe86006d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe8800aa160 tx=0x7fe868007b50 comp rx=0 tx=0).stop 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 msgr2=0x7fe880157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880157610 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fe870036040 tx=0x7fe8700046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 shutdown_connections 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe86006b4d0 0x7fe86006d990 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8800c67f0 0x7fe880151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8800c3940 0x7fe880151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 --2- 172.21.15.26:0/1372929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8800724b0 0x7fe880157610 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 >> 172.21.15.26:0/1372929264 conn(0x7fe8800aa6a0 msgr2=0x7fe8800c80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 shutdown_connections 2024-03-06T07:29:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:44.448+0000 7fe885132640 1 -- 172.21.15.26:0/1372929264 wait complete. 2024-03-06T07:29:44.863 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:44.863 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:45.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:45.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.966+0000 7fccf691e640 1 Processor -- start 2024-03-06T07:29:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.966+0000 7fccf691e640 1 -- start start 2024-03-06T07:29:45.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fccf691e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fccf00c88c0 0x7fccf0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fccf691e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00c73d0 con 0x7fccf014f360 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fccf691e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00be2e0 con 0x7fccf0158b60 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fccf691e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf0159700 con 0x7fccf00c88c0 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fcceffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fccf014f360 0x7fccf00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46376/0 (socket says 172.21.15.26:46376) 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fcceffff640 1 -- 172.21.15.26:0/594099884 learned_addr learned my addr 172.21.15.26:0/594099884 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fccef7fe640 1 --2- 172.21.15.26:0/594099884 >> v2:172.21.15.117:3300/0 conn(0x7fccf00c88c0 0x7fccf0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942306062 0 0) 0x7fccf00c73d0 con 0x7fccf014f360 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccd8003520 con 0x7fccf014f360 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3433797705 0 0) 0x7fccd8003520 con 0x7fccf014f360 2024-03-06T07:29:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 >> v2:172.21.15.117:3300/0 conn(0x7fccf00c88c0 msgr2=0x7fccf0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.967+0000 7fcceeffd640 1 --2- 172.21.15.26:0/594099884 >> v2:172.21.15.117:3300/0 conn(0x7fccf00c88c0 0x7fccf0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 >> v1:172.21.15.117:6789/0 conn(0x7fccf0158b60 legacy=0x7fccf0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00c7670 con 0x7fccf014f360 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fcce0001670 con 0x7fccf014f360 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 >> v1:172.21.15.26:6789/0 conn(0x7fccf014f360 legacy=0x7fccf00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd8004800 0x7fccd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd80051c0 0x7fccd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf0159700 con 0x7fccd80039b0 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd800c160 con 0x7fccd8004800 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd800c360 con 0x7fccd80051c0 2024-03-06T07:29:45.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceffff640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fccef7fe640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd8004800 0x7fccd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fccf4ebb640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd80051c0 0x7fccd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceffff640 1 -- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd80051c0 msgr2=0x7fccd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceffff640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd80051c0 0x7fccd800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceffff640 1 -- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd8004800 msgr2=0x7fccd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceffff640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd8004800 0x7fccd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fcceffff640 1 -- 172.21.15.26:0/594099884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd8003520 con 0x7fccd80039b0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fccf4ebb640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd80051c0 0x7fccd800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.968+0000 7fccf4ebb640 1 -- 172.21.15.26:0/594099884 reap_dead start 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.969+0000 7fcceffff640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccd8003dd0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c210 tx=0x7fccd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a227998f230943db server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.969+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0007d30 con 0x7fccd80039b0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.969+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce0030070 con 0x7fccd80039b0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.969+0000 7fcceeffd640 1 -- 172.21.15.26:0/594099884 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce002cda0 con 0x7fccd80039b0 2024-03-06T07:29:45.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.970+0000 7fccf691e640 1 -- 172.21.15.26:0/594099884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 msgr2=0x7fccd8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:45.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.970+0000 7fccf691e640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccd8003dd0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c210 tx=0x7fccd8005830 comp rx=0 tx=0).stop 2024-03-06T07:29:45.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.970+0000 7fccf691e640 1 -- 172.21.15.26:0/594099884 shutdown_connections 2024-03-06T07:29:45.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.970+0000 7fccf691e640 1 --2- 172.21.15.26:0/594099884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccd8003dd0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.970+0000 7fccf691e640 1 -- 172.21.15.26:0/594099884 >> 172.21.15.26:0/594099884 conn(0x7fccf00ac570 msgr2=0x7fccf00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:45.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.970+0000 7fccf691e640 1 -- 172.21.15.26:0/594099884 shutdown_connections 2024-03-06T07:29:45.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.970+0000 7fccf691e640 1 -- 172.21.15.26:0/594099884 wait complete. 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.971+0000 7fccf691e640 1 Processor -- start 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.971+0000 7fccf691e640 1 -- start start 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccf691e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccf0157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccf691e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccf014ae60 0x7fccf01eea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccf691e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccf014b390 0x7fccf01eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccf691e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00c7c30 con 0x7fccd80039b0 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccf691e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf0155c10 con 0x7fccf014b390 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccf691e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf0155d90 con 0x7fccf014ae60 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccf0157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccf0157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60220/0 (socket says 172.21.15.26:60220) 2024-03-06T07:29:45.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 -- 172.21.15.26:0/3098781714 learned_addr learned my addr 172.21.15.26:0/3098781714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccef7fe640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccf014ae60 0x7fccf01eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fccf4ebb640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccf014b390 0x7fccf01eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 -- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccf014ae60 msgr2=0x7fccf01eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccf014ae60 0x7fccf01eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 -- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccf014b390 msgr2=0x7fccf01eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccf014b390 0x7fccf01eefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.972+0000 7fcceffff640 1 -- 172.21.15.26:0/3098781714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00bdd90 con 0x7fccd80039b0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.973+0000 7fcceffff640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccf0157bc0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c330 tx=0x7fcce002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.973+0000 7fccecff9640 1 -- 172.21.15.26:0/3098781714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004bf0 con 0x7fccd80039b0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.973+0000 7fccecff9640 1 -- 172.21.15.26:0/3098781714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce0004d90 con 0x7fccd80039b0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.973+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf00be070 con 0x7fccd80039b0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.973+0000 7fccecff9640 1 -- 172.21.15.26:0/3098781714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0007270 con 0x7fccd80039b0 2024-03-06T07:29:45.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.973+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf01ebd90 con 0x7fccd80039b0 2024-03-06T07:29:45.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.974+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf004f330 con 0x7fccd80039b0 2024-03-06T07:29:45.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.974+0000 7fccecff9640 1 -- 172.21.15.26:0/3098781714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcce0048020 con 0x7fccd80039b0 2024-03-06T07:29:45.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.975+0000 7fccecff9640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fccdc06b4d0 0x7fccdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:45.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.975+0000 7fccecff9640 1 -- 172.21.15.26:0/3098781714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcce00ab710 con 0x7fccd80039b0 2024-03-06T07:29:45.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.975+0000 7fccef7fe640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fccdc06b4d0 0x7fccdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:45.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.975+0000 7fccef7fe640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fccdc06b4d0 0x7fccdc06d990 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fccf00c16f0 tx=0x7fcce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:45.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:45.979+0000 7fccecff9640 1 -- 172.21.15.26:0/3098781714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcce007a400 con 0x7fccd80039b0 2024-03-06T07:29:46.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.198+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccf00c8d50 con 0x7fccd80039b0 2024-03-06T07:29:46.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.199+0000 7fccecff9640 1 -- 172.21.15.26:0/3098781714 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fcce0079d00 con 0x7fccd80039b0 2024-03-06T07:29:46.200 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:46.200 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:46.201 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fccdc06b4d0 msgr2=0x7fccdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fccdc06b4d0 0x7fccdc06d990 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fccf00c16f0 tx=0x7fcce4008040 comp rx=0 tx=0).stop 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 msgr2=0x7fccf0157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccf0157bc0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c330 tx=0x7fcce002e070 comp rx=0 tx=0).stop 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 shutdown_connections 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fccdc06b4d0 0x7fccdc06d990 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccf014b390 0x7fccf01eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccf014ae60 0x7fccf01eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 --2- 172.21.15.26:0/3098781714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd80039b0 0x7fccf0157bc0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.200+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 >> 172.21.15.26:0/3098781714 conn(0x7fccf00ac570 msgr2=0x7fccf00c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.201+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 shutdown_connections 2024-03-06T07:29:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:46.201+0000 7fccf691e640 1 -- 172.21.15.26:0/3098781714 wait complete. 2024-03-06T07:29:46.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:46.617 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:47.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:47.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.718+0000 7f46cfaed640 1 Processor -- start 2024-03-06T07:29:47.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.718+0000 7f46cfaed640 1 -- start start 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.718+0000 7f46cfaed640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46c814f360 0x7f46c8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.718+0000 7f46cfaed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80c73d0 con 0x7f46c8158b60 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.718+0000 7f46cfaed640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80be2e0 con 0x7f46c80c88c0 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.718+0000 7f46cfaed640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c8159700 con 0x7f46c814f360 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.719+0000 7f46cd889640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46c814f360 0x7f46c8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.719+0000 7f46ce08a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f46c8158b60 0x7f46c8158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41500/0 (socket says 172.21.15.26:41500) 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.719+0000 7f46ce08a640 1 -- 172.21.15.26:0/211588081 learned_addr learned my addr 172.21.15.26:0/211588081 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.719+0000 7f46cd889640 1 -- 172.21.15.26:0/211588081 >> v1:172.21.15.117:6789/0 conn(0x7f46c80c88c0 legacy=0x7f46c80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.719+0000 7f46cd889640 1 -- 172.21.15.26:0/211588081 >> v1:172.21.15.26:6789/0 conn(0x7f46c8158b60 legacy=0x7f46c8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.719+0000 7f46cd889640 1 -- 172.21.15.26:0/211588081 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c80c7670 con 0x7f46c814f360 2024-03-06T07:29:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.719+0000 7f46cc887640 1 -- 172.21.15.26:0/211588081 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530014067 0 0) 0x7f46c80c73d0 con 0x7f46c8158b60 2024-03-06T07:29:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cd889640 1 --2- 172.21.15.26:0/211588081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c8155c10 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f46c40097a0 tx=0x7f46c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8735660f6a590d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cc887640 1 -- 172.21.15.26:0/211588081 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40315a0 con 0x7f46c814f360 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cc887640 1 -- 172.21.15.26:0/211588081 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c4037070 con 0x7f46c814f360 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cc887640 1 -- 172.21.15.26:0/211588081 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4031960 con 0x7f46c814f360 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cfaed640 1 -- 172.21.15.26:0/211588081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 msgr2=0x7f46c8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cfaed640 1 --2- 172.21.15.26:0/211588081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c8155c10 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f46c40097a0 tx=0x7f46c402b840 comp rx=0 tx=0).stop 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cfaed640 1 -- 172.21.15.26:0/211588081 shutdown_connections 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cfaed640 1 --2- 172.21.15.26:0/211588081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c8155c10 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cfaed640 1 -- 172.21.15.26:0/211588081 >> 172.21.15.26:0/211588081 conn(0x7f46c80ac570 msgr2=0x7f46c80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.720+0000 7f46cfaed640 1 -- 172.21.15.26:0/211588081 shutdown_connections 2024-03-06T07:29:47.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.721+0000 7f46cfaed640 1 -- 172.21.15.26:0/211588081 wait complete. 2024-03-06T07:29:47.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.722+0000 7f46cfaed640 1 Processor -- start 2024-03-06T07:29:47.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.722+0000 7f46cfaed640 1 -- start start 2024-03-06T07:29:47.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cfaed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80c88c0 0x7f46c814bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:47.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cfaed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c80aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:47.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cfaed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8158b60 0x7f46c80bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:47.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cfaed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80c7c30 con 0x7f46c814f360 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cfaed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80c5a50 con 0x7f46c8158b60 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cfaed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80c5bd0 con 0x7f46c80c88c0 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cd088640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c80aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cd088640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c80aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42924/0 (socket says 172.21.15.26:42924) 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.723+0000 7f46cd088640 1 -- 172.21.15.26:0/1300136315 learned_addr learned my addr 172.21.15.26:0/1300136315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46ce08a640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8158b60 0x7f46c80bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cd889640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80c88c0 0x7f46c814bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cd088640 1 -- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80c88c0 msgr2=0x7f46c814bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cd088640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80c88c0 0x7f46c814bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cd088640 1 -- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8158b60 msgr2=0x7f46c80bf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cd088640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8158b60 0x7f46c80bf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cd088640 1 -- 172.21.15.26:0/1300136315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c814c420 con 0x7f46c814f360 2024-03-06T07:29:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cd088640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c80aff60 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f46b800c400 tx=0x7f46b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46be7fc640 1 -- 172.21.15.26:0/1300136315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8017070 con 0x7f46c814f360 2024-03-06T07:29:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46be7fc640 1 -- 172.21.15.26:0/1300136315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b80035f0 con 0x7f46c814f360 2024-03-06T07:29:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c81eab50 con 0x7f46c814f360 2024-03-06T07:29:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46be7fc640 1 -- 172.21.15.26:0/1300136315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8003a20 con 0x7f46c814f360 2024-03-06T07:29:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.724+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c81eb0d0 con 0x7f46c814f360 2024-03-06T07:29:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.725+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c804f330 con 0x7f46c814f360 2024-03-06T07:29:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.726+0000 7f46be7fc640 1 -- 172.21.15.26:0/1300136315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46b801a070 con 0x7f46c814f360 2024-03-06T07:29:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.726+0000 7f46be7fc640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46ac06b500 0x7f46ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.726+0000 7f46be7fc640 1 -- 172.21.15.26:0/1300136315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46b8089600 con 0x7f46c814f360 2024-03-06T07:29:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.727+0000 7f46cd889640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46ac06b500 0x7f46ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:47.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.727+0000 7f46cd889640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46ac06b500 0x7f46ac06d9c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f46c4036000 tx=0x7f46c403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.731+0000 7f46be7fc640 1 -- 172.21.15.26:0/1300136315 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46c814c420 con 0x7f46c814f360 2024-03-06T07:29:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.947+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46c80c8d50 con 0x7f46c814f360 2024-03-06T07:29:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.948+0000 7f46be7fc640 1 -- 172.21.15.26:0/1300136315 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f46b8057d30 con 0x7f46c814f360 2024-03-06T07:29:47.950 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:47.950 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.949+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46ac06b500 msgr2=0x7f46ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46ac06b500 0x7f46ac06d9c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f46c4036000 tx=0x7f46c403d040 comp rx=0 tx=0).stop 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 msgr2=0x7f46c80aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c80aff60 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f46b800c400 tx=0x7f46b800c8c0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 shutdown_connections 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f46ac06b500 0x7f46ac06d9c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8158b60 0x7f46c80bf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46c814f360 0x7f46c80aff60 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 --2- 172.21.15.26:0/1300136315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80c88c0 0x7f46c814bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 >> 172.21.15.26:0/1300136315 conn(0x7f46c80ac570 msgr2=0x7f46c80aad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 shutdown_connections 2024-03-06T07:29:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:47.950+0000 7f46cfaed640 1 -- 172.21.15.26:0/1300136315 wait complete. 2024-03-06T07:29:48.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:48.365 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:49.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:49.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.464+0000 7f6c0344e640 1 Processor -- start 2024-03-06T07:29:49.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.464+0000 7f6c0344e640 1 -- start start 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.465+0000 7f6c0344e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c1320 0x7f6bfc1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.465+0000 7f6c0344e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc153a60 con 0x7f6bfc0c6000 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.465+0000 7f6c0344e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc153d40 con 0x7f6bfc150580 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.465+0000 7f6c0344e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc06c2c0 con 0x7f6bfc0c1320 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.465+0000 7f6c019eb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6bfc0c6000 0x7f6bfc156bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41506/0 (socket says 172.21.15.26:41506) 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6c019eb640 1 -- 172.21.15.26:0/1710751684 learned_addr learned my addr 172.21.15.26:0/1710751684 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6c009e9640 1 --2- 172.21.15.26:0/1710751684 >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c1320 0x7f6bfc1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777794688 0 0) 0x7f6bfc153a60 con 0x7f6bfc0c6000 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be8003520 con 0x7f6bfc0c6000 2024-03-06T07:29:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303403621 0 0) 0x7f6be8003520 con 0x7f6bfc0c6000 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c1320 msgr2=0x7f6bfc1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 --2- 172.21.15.26:0/1710751684 >> v2:172.21.15.117:3300/0 conn(0x7f6bfc0c1320 0x7f6bfc1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 >> v1:172.21.15.117:6789/0 conn(0x7f6bfc150580 legacy=0x7f6bfc06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc06c5a0 con 0x7f6bfc0c6000 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6bf40019f0 con 0x7f6bfc0c6000 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 >> v1:172.21.15.26:6789/0 conn(0x7f6bfc0c6000 legacy=0x7f6bfc156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be80039b0 0x7f6be8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be8004800 0x7f6be8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be80051c0 0x7f6be800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc06c2c0 con 0x7f6be80039b0 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be800c160 con 0x7f6be80051c0 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be800c360 con 0x7f6be8004800 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6c011ea640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be80039b0 0x7f6be8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6c019eb640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be80051c0 0x7f6be800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.466+0000 7f6c009e9640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be8004800 0x7f6be8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6c011ea640 1 -- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be8004800 msgr2=0x7f6be8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6c011ea640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be8004800 0x7f6be8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6c011ea640 1 -- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be80051c0 msgr2=0x7f6be800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6c011ea640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be80051c0 0x7f6be800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6c019eb640 1 -- 172.21.15.26:0/1710751684 reap_dead start 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6c011ea640 1 -- 172.21.15.26:0/1710751684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8003520 con 0x7f6be80039b0 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6c011ea640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be80039b0 0x7f6be8003dd0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0001750 tx=0x7f6be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17412ada91c48c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.467+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0001370 con 0x7f6be80039b0 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf00052d0 con 0x7f6be80039b0 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6beffff640 1 -- 172.21.15.26:0/1710751684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0003830 con 0x7f6be80039b0 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6c0344e640 1 -- 172.21.15.26:0/1710751684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be80039b0 msgr2=0x7f6be8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be80039b0 0x7f6be8003dd0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0001750 tx=0x7f6be8005830 comp rx=0 tx=0).stop 2024-03-06T07:29:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6c0344e640 1 -- 172.21.15.26:0/1710751684 shutdown_connections 2024-03-06T07:29:49.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1710751684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6be80039b0 0x7f6be8003dd0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6c0344e640 1 -- 172.21.15.26:0/1710751684 >> 172.21.15.26:0/1710751684 conn(0x7f6bfc0c6710 msgr2=0x7f6bfc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:49.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6c0344e640 1 -- 172.21.15.26:0/1710751684 shutdown_connections 2024-03-06T07:29:49.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.468+0000 7f6c0344e640 1 -- 172.21.15.26:0/1710751684 wait complete. 2024-03-06T07:29:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.469+0000 7f6c0344e640 1 Processor -- start 2024-03-06T07:29:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.469+0000 7f6c0344e640 1 -- start start 2024-03-06T07:29:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c0344e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be80039b0 0x7f6bfc1eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c0344e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0a6330 0x7f6bfc1eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c0344e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc145d90 0x7f6bfc145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c0344e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c0dc0 con 0x7f6bfc0a6330 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c0344e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c0c40 con 0x7f6bfc145d90 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c0344e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc06dd10 con 0x7f6be80039b0 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0a6330 0x7f6bfc1eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0a6330 0x7f6bfc1eeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42948/0 (socket says 172.21.15.26:42948) 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 -- 172.21.15.26:0/1827258923 learned_addr learned my addr 172.21.15.26:0/1827258923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c011ea640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be80039b0 0x7f6bfc1eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c019eb640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc145d90 0x7f6bfc145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 -- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be80039b0 msgr2=0x7f6bfc1eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be80039b0 0x7f6bfc1eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 -- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc145d90 msgr2=0x7f6bfc145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc145d90 0x7f6bfc145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.470+0000 7f6c009e9640 1 -- 172.21.15.26:0/1827258923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc1eb1c0 con 0x7f6bfc0a6330 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.471+0000 7f6c009e9640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0a6330 0x7f6bfc1eeb20 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8011a70 tx=0x7f6bf8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.471+0000 7f6bedffb640 1 -- 172.21.15.26:0/1827258923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800fa90 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.471+0000 7f6bedffb640 1 -- 172.21.15.26:0/1827258923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf800fc30 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.471+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc0a7c90 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.471+0000 7f6bedffb640 1 -- 172.21.15.26:0/1827258923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8014440 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.471+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc1ee320 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.472+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc04f330 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.472+0000 7f6bedffb640 1 -- 172.21.15.26:0/1827258923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bf8012040 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.473+0000 7f6bedffb640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b500 0x7f6be006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.473+0000 7f6bedffb640 1 -- 172.21.15.26:0/1827258923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6bf808a7e0 con 0x7f6bfc0a6330 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.473+0000 7f6c011ea640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b500 0x7f6be006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.474+0000 7f6c011ea640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b500 0x7f6be006d9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0033000 tx=0x7f6bf0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.477+0000 7f6bedffb640 1 -- 172.21.15.26:0/1827258923 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6bf8019030 con 0x7f6bfc0a6330 2024-03-06T07:29:49.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.694+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bfc0a5ed0 con 0x7f6bfc0a6330 2024-03-06T07:29:49.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.695+0000 7f6bedffb640 1 -- 172.21.15.26:0/1827258923 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f6bf8059560 con 0x7f6bfc0a6330 2024-03-06T07:29:49.697 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:49.697 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.696+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b500 msgr2=0x7f6be006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.696+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b500 0x7f6be006d9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0033000 tx=0x7f6bf0009030 comp rx=0 tx=0).stop 2024-03-06T07:29:49.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.696+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0a6330 msgr2=0x7f6bfc1eeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.696+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0a6330 0x7f6bfc1eeb20 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8011a70 tx=0x7f6bf8011e70 comp rx=0 tx=0).stop 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 shutdown_connections 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6be006b500 0x7f6be006d9c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc145d90 0x7f6bfc145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bfc0a6330 0x7f6bfc1eeb20 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 --2- 172.21.15.26:0/1827258923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6be80039b0 0x7f6bfc1eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 >> 172.21.15.26:0/1827258923 conn(0x7f6bfc0c6710 msgr2=0x7f6bfc14f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 shutdown_connections 2024-03-06T07:29:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:49.697+0000 7f6c0344e640 1 -- 172.21.15.26:0/1827258923 wait complete. 2024-03-06T07:29:50.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:50.113 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:51.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.213+0000 7fc62e415640 1 Processor -- start 2024-03-06T07:29:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.213+0000 7fc62e415640 1 -- start start 2024-03-06T07:29:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc62e415640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc628152920 0x7fc628155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc62e415640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 0x7fc6280c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc62e415640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc628156180 con 0x7fc6280c8a00 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc62e415640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6281587b0 con 0x7fc628152920 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc62e415640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc628159630 con 0x7fc6280c5330 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc627fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc628152920 0x7fc628155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc6277fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc6280c8a00 0x7fc628158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41512/0 (socket says 172.21.15.26:41512) 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc6277fe640 1 -- 172.21.15.26:0/3182067137 learned_addr learned my addr 172.21.15.26:0/3182067137 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc62c9b2640 1 --2- 172.21.15.26:0/3182067137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 0x7fc6280c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc626ffd640 1 -- 172.21.15.26:0/3182067137 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352587258 0 0) 0x7fc61c001110 con 0x7fc6280c8a00 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.214+0000 7fc626ffd640 1 -- 172.21.15.26:0/3182067137 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc610003520 con 0x7fc6280c8a00 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.215+0000 7fc627fff640 1 -- 172.21.15.26:0/3182067137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 msgr2=0x7fc6280c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.215+0000 7fc627fff640 1 --2- 172.21.15.26:0/3182067137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 0x7fc6280c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.215+0000 7fc627fff640 1 -- 172.21.15.26:0/3182067137 >> v1:172.21.15.26:6789/0 conn(0x7fc6280c8a00 legacy=0x7fc628158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.215+0000 7fc627fff640 1 -- 172.21.15.26:0/3182067137 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280c7760 con 0x7fc628152920 2024-03-06T07:29:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.215+0000 7fc627fff640 1 --2- 172.21.15.26:0/3182067137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc628152920 0x7fc628155c40 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fc6180097a0 tx=0x7fc61802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfe72754c1f94d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc626ffd640 1 -- 172.21.15.26:0/3182067137 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618031780 con 0x7fc628152920 2024-03-06T07:29:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc626ffd640 1 -- 172.21.15.26:0/3182067137 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc618037070 con 0x7fc628152920 2024-03-06T07:29:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc626ffd640 1 -- 172.21.15.26:0/3182067137 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618031b20 con 0x7fc628152920 2024-03-06T07:29:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 -- 172.21.15.26:0/3182067137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc628152920 msgr2=0x7fc628155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 --2- 172.21.15.26:0/3182067137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc628152920 0x7fc628155c40 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fc6180097a0 tx=0x7fc61802b820 comp rx=0 tx=0).stop 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 -- 172.21.15.26:0/3182067137 shutdown_connections 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 --2- 172.21.15.26:0/3182067137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 0x7fc6280c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 --2- 172.21.15.26:0/3182067137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc628152920 0x7fc628155c40 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 -- 172.21.15.26:0/3182067137 >> 172.21.15.26:0/3182067137 conn(0x7fc6280ac560 msgr2=0x7fc6280ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 -- 172.21.15.26:0/3182067137 shutdown_connections 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.216+0000 7fc62e415640 1 -- 172.21.15.26:0/3182067137 wait complete. 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.217+0000 7fc62e415640 1 Processor -- start 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.217+0000 7fc62e415640 1 -- start start 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc62e415640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 0x7fc6281ee960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc62e415640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c8a00 0x7fc628155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc62e415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc628152920 0x7fc6280c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc62e415640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628158a00 con 0x7fc6280c8a00 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc62e415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280c7d20 con 0x7fc628152920 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc62e415640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62806d030 con 0x7fc6280c5330 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c8a00 0x7fc628155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c8a00 0x7fc628155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42966/0 (socket says 172.21.15.26:42966) 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.218+0000 7fc6277fe640 1 -- 172.21.15.26:0/2102603377 learned_addr learned my addr 172.21.15.26:0/2102603377 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc6277fe640 1 -- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 msgr2=0x7fc6281ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc6277fe640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 0x7fc6281ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc6277fe640 1 -- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc628152920 msgr2=0x7fc6280c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc62c9b2640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc628152920 0x7fc6280c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc6277fe640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc628152920 0x7fc6280c2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc6277fe640 1 -- 172.21.15.26:0/2102603377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6281496c0 con 0x7fc6280c8a00 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc6277fe640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c8a00 0x7fc628155470 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c3d0 tx=0x7fc61c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc624ff9640 1 -- 172.21.15.26:0/2102603377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c017070 con 0x7fc6280c8a00 2024-03-06T07:29:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc624ff9640 1 -- 172.21.15.26:0/2102603377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c005ce0 con 0x7fc6280c8a00 2024-03-06T07:29:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6281499a0 con 0x7fc6280c8a00 2024-03-06T07:29:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc624ff9640 1 -- 172.21.15.26:0/2102603377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c007310 con 0x7fc6280c8a00 2024-03-06T07:29:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.219+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6281559b0 con 0x7fc6280c8a00 2024-03-06T07:29:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.220+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62804f330 con 0x7fc6280c8a00 2024-03-06T07:29:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.220+0000 7fc624ff9640 1 -- 172.21.15.26:0/2102603377 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc61c01a070 con 0x7fc6280c8a00 2024-03-06T07:29:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.221+0000 7fc624ff9640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc61406b500 0x7fc61406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.221+0000 7fc624ff9640 1 -- 172.21.15.26:0/2102603377 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc61c089710 con 0x7fc6280c8a00 2024-03-06T07:29:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.221+0000 7fc627fff640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc61406b500 0x7fc61406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.221+0000 7fc627fff640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc61406b500 0x7fc61406d9c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc618036000 tx=0x7fc618030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:51.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.225+0000 7fc624ff9640 1 -- 172.21.15.26:0/2102603377 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc628156180 con 0x7fc6280c8a00 2024-03-06T07:29:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.449+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc62806cda0 con 0x7fc6280c8a00 2024-03-06T07:29:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.450+0000 7fc624ff9640 1 -- 172.21.15.26:0/2102603377 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fc61c058400 con 0x7fc6280c8a00 2024-03-06T07:29:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:51.453 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.451+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc61406b500 msgr2=0x7fc61406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc61406b500 0x7fc61406d9c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc618036000 tx=0x7fc618030000 comp rx=0 tx=0).stop 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c8a00 msgr2=0x7fc628155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c8a00 0x7fc628155470 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c3d0 tx=0x7fc61c00c890 comp rx=0 tx=0).stop 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 shutdown_connections 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc61406b500 0x7fc61406d9c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc628152920 0x7fc6280c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6280c8a00 0x7fc628155470 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 --2- 172.21.15.26:0/2102603377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6280c5330 0x7fc6281ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 >> 172.21.15.26:0/2102603377 conn(0x7fc6280ac560 msgr2=0x7fc6280bdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 shutdown_connections 2024-03-06T07:29:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:51.452+0000 7fc62e415640 1 -- 172.21.15.26:0/2102603377 wait complete. 2024-03-06T07:29:51.868 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:51.868 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:52.869 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.969+0000 7ff26f7e9640 1 Processor -- start 2024-03-06T07:29:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.969+0000 7ff26f7e9640 1 -- start start 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26f7e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2680c5480 0x7ff268155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26f7e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2680c5860 0x7ff26806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26f7e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2680c5cc0 con 0x7ff2681586f0 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26f7e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26806df40 con 0x7ff2680c5860 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26f7e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2681591c0 con 0x7ff2680c5480 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26cd84640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2680c5860 0x7ff26806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26dd86640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff2681586f0 0x7ff2680724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41514/0 (socket says 172.21.15.26:41514) 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26dd86640 1 -- 172.21.15.26:0/4198870802 learned_addr learned my addr 172.21.15.26:0/4198870802 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26cd84640 1 -- 172.21.15.26:0/4198870802 >> v2:172.21.15.117:3300/0 conn(0x7ff2680c5480 msgr2=0x7ff268155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26d585640 1 --2- 172.21.15.26:0/4198870802 >> v2:172.21.15.117:3300/0 conn(0x7ff2680c5480 0x7ff268155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26cd84640 1 --2- 172.21.15.26:0/4198870802 >> v2:172.21.15.117:3300/0 conn(0x7ff2680c5480 0x7ff268155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26cd84640 1 -- 172.21.15.26:0/4198870802 >> v1:172.21.15.26:6789/0 conn(0x7ff2681586f0 legacy=0x7ff2680724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26cd84640 1 -- 172.21.15.26:0/4198870802 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2681594a0 con 0x7ff2680c5860 2024-03-06T07:29:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff25ffff640 1 -- 172.21.15.26:0/4198870802 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826922421 0 0) 0x7ff2680c5cc0 con 0x7ff2681586f0 2024-03-06T07:29:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.970+0000 7ff26d585640 1 --2- 172.21.15.26:0/4198870802 >> v2:172.21.15.117:3300/0 conn(0x7ff2680c5480 0x7ff268155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.971+0000 7ff26cd84640 1 --2- 172.21.15.26:0/4198870802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26806f6a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7ff2680c1cf0 tx=0x7ff25802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dcec2e7cb8ce91 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.971+0000 7ff25ffff640 1 -- 172.21.15.26:0/4198870802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580315a0 con 0x7ff2680c5860 2024-03-06T07:29:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.971+0000 7ff25ffff640 1 -- 172.21.15.26:0/4198870802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff258037070 con 0x7ff2680c5860 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.971+0000 7ff25ffff640 1 -- 172.21.15.26:0/4198870802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258031960 con 0x7ff2680c5860 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 -- 172.21.15.26:0/4198870802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 msgr2=0x7ff26806f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/4198870802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26806f6a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7ff2680c1cf0 tx=0x7ff25802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 -- 172.21.15.26:0/4198870802 shutdown_connections 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/4198870802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26806f6a0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/4198870802 >> v2:172.21.15.117:3300/0 conn(0x7ff2680c5480 0x7ff268155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 -- 172.21.15.26:0/4198870802 >> 172.21.15.26:0/4198870802 conn(0x7ff26806b1a0 msgr2=0x7ff26806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 -- 172.21.15.26:0/4198870802 shutdown_connections 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.972+0000 7ff26f7e9640 1 -- 172.21.15.26:0/4198870802 wait complete. 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.973+0000 7ff26f7e9640 1 Processor -- start 2024-03-06T07:29:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.973+0000 7ff26f7e9640 1 -- start start 2024-03-06T07:29:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26f7e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2680c5480 0x7ff26814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26f7e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26f7e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2681586f0 0x7ff2680c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26f7e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff268152a20 con 0x7ff2680c5860 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26f7e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff268155680 con 0x7ff2681586f0 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26f7e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff268155050 con 0x7ff2680c5480 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42996/0 (socket says 172.21.15.26:42996) 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 -- 172.21.15.26:0/286181624 learned_addr learned my addr 172.21.15.26:0/286181624 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26d585640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2680c5480 0x7ff26814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26dd86640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2681586f0 0x7ff2680c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 -- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2680c5480 msgr2=0x7ff26814e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2680c5480 0x7ff26814e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 -- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2681586f0 msgr2=0x7ff2680c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2681586f0 0x7ff2680c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 -- 172.21.15.26:0/286181624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26814e210 con 0x7ff2680c5860 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26d585640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2680c5480 0x7ff26814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26dd86640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2681586f0 0x7ff2680c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26cd84640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26814dcd0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7ff258009890 tx=0x7ff2580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff25dffb640 1 -- 172.21.15.26:0/286181624 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258032840 con 0x7ff2680c5860 2024-03-06T07:29:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff25dffb640 1 -- 172.21.15.26:0/286181624 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff258032a00 con 0x7ff2680c5860 2024-03-06T07:29:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2680ad2e0 con 0x7ff2680c5860 2024-03-06T07:29:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.974+0000 7ff25dffb640 1 -- 172.21.15.26:0/286181624 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258032df0 con 0x7ff2680c5860 2024-03-06T07:29:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.975+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2681eded0 con 0x7ff2680c5860 2024-03-06T07:29:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.976+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26804f330 con 0x7ff2680c5860 2024-03-06T07:29:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.976+0000 7ff25dffb640 1 -- 172.21.15.26:0/286181624 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff25804c020 con 0x7ff2680c5860 2024-03-06T07:29:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.977+0000 7ff25dffb640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff24c06b4d0 0x7ff24c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.977+0000 7ff25dffb640 1 -- 172.21.15.26:0/286181624 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6188+0+0 (secure 0 0 0) 0x7ff25802f020 con 0x7ff2680c5860 2024-03-06T07:29:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.977+0000 7ff26d585640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff24c06b4d0 0x7ff24c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.978+0000 7ff26d585640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff24c06b4d0 0x7ff24c06d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff264006fd0 tx=0x7ff264008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:52.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:52.981+0000 7ff25dffb640 1 -- 172.21.15.26:0/286181624 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff26814e210 con 0x7ff2680c5860 2024-03-06T07:29:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.301+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2681ee0a0 con 0x7ff2680c5860 2024-03-06T07:29:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.302+0000 7ff25dffb640 1 -- 172.21.15.26:0/286181624 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7ff258040070 con 0x7ff2680c5860 2024-03-06T07:29:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:53.305 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195},{"gid":8330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:24.667113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":9799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9799":{"gid":9799,"name":"b","rank":1,"incarnation":200,"state":"up:stopping","state_seq":4,"addr":"172.21.15.117:6837/3331811951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3331811951},{"type":"v1","addr":"172.21.15.117:6837","nonce":3331811951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:53.305 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 202 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff24c06b4d0 msgr2=0x7ff24c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff24c06b4d0 0x7ff24c06d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff264006fd0 tx=0x7ff264008040 comp rx=0 tx=0).stop 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 msgr2=0x7ff26814dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26814dcd0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7ff258009890 tx=0x7ff2580046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 shutdown_connections 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff24c06b4d0 0x7ff24c06d990 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2681586f0 0x7ff2680c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2680c5860 0x7ff26814dcd0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 --2- 172.21.15.26:0/286181624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2680c5480 0x7ff26814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 >> 172.21.15.26:0/286181624 conn(0x7ff26806b1a0 msgr2=0x7ff268070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:53.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 shutdown_connections 2024-03-06T07:29:53.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:53.304+0000 7ff26f7e9640 1 -- 172.21.15.26:0/286181624 wait complete. 2024-03-06T07:29:53.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:24.667113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 9799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9799': {'gid': 9799, 'name': 'b', 'rank': 1, 'incarnation': 200, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.117:6837/3331811951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3331811951}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3331811951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:53.723 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9799:up:stopping 2024-03-06T07:29:54.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.822+0000 7f1789df7640 1 Processor -- start 2024-03-06T07:29:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.822+0000 7f1789df7640 1 -- start start 2024-03-06T07:29:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f1789df7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f17840c88c0 0x7f1784155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f1789df7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17840c73d0 con 0x7f178414f360 2024-03-06T07:29:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f1789df7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17840be2e0 con 0x7f1784158b60 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f1789df7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1784159700 con 0x7f17840c88c0 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f1782ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f17840c88c0 0x7f1784155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f17837fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f178414f360 0x7f17840c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41528/0 (socket says 172.21.15.26:41528) 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f1782ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f17840c88c0 0x7f1784155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43010/0 (socket says 172.21.15.26:43010) 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.823+0000 7f17837fe640 1 -- 172.21.15.26:0/4139346714 learned_addr learned my addr 172.21.15.26:0/4139346714 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.824+0000 7f1782ffd640 1 -- 172.21.15.26:0/4139346714 >> v1:172.21.15.117:6789/0 conn(0x7f1784158b60 legacy=0x7f1784158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.824+0000 7f1782ffd640 1 -- 172.21.15.26:0/4139346714 >> v1:172.21.15.26:6789/0 conn(0x7f178414f360 legacy=0x7f17840c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.824+0000 7f17827fc640 1 -- 172.21.15.26:0/4139346714 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728497138 0 0) 0x7f17840c73d0 con 0x7f178414f360 2024-03-06T07:29:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.824+0000 7f1782ffd640 1 -- 172.21.15.26:0/4139346714 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840c7670 con 0x7f17840c88c0 2024-03-06T07:29:54.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.824+0000 7f1782ffd640 1 --2- 172.21.15.26:0/4139346714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17840c88c0 0x7f1784155c10 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f17780097a0 tx=0x7f177802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cb9493f1e426e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f17827fc640 1 -- 172.21.15.26:0/4139346714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780315a0 con 0x7f17840c88c0 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f17827fc640 1 -- 172.21.15.26:0/4139346714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1778037070 con 0x7f17840c88c0 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f17827fc640 1 -- 172.21.15.26:0/4139346714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778031960 con 0x7f17840c88c0 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f1789df7640 1 -- 172.21.15.26:0/4139346714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17840c88c0 msgr2=0x7f1784155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f1789df7640 1 --2- 172.21.15.26:0/4139346714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17840c88c0 0x7f1784155c10 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f17780097a0 tx=0x7f177802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f1789df7640 1 -- 172.21.15.26:0/4139346714 shutdown_connections 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f1789df7640 1 --2- 172.21.15.26:0/4139346714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17840c88c0 0x7f1784155c10 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f1789df7640 1 -- 172.21.15.26:0/4139346714 >> 172.21.15.26:0/4139346714 conn(0x7f17840ac570 msgr2=0x7f17840ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f1789df7640 1 -- 172.21.15.26:0/4139346714 shutdown_connections 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.825+0000 7f1789df7640 1 -- 172.21.15.26:0/4139346714 wait complete. 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.826+0000 7f1789df7640 1 Processor -- start 2024-03-06T07:29:54.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.826+0000 7f1789df7640 1 -- start start 2024-03-06T07:29:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1789df7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17840c88c0 0x7f178414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1789df7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f178414f360 0x7f17840c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1789df7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1784158b60 0x7f178414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1789df7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c7c30 con 0x7f1784158b60 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1789df7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c5a50 con 0x7f17840c88c0 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1789df7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c5bd0 con 0x7f178414f360 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1783fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1784158b60 0x7f178414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1783fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1784158b60 0x7f178414aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43014/0 (socket says 172.21.15.26:43014) 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1783fff640 1 -- 172.21.15.26:0/471769531 learned_addr learned my addr 172.21.15.26:0/471769531 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f1782ffd640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f178414f360 0x7f17840c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.827+0000 7f17837fe640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17840c88c0 0x7f178414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1783fff640 1 -- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f178414f360 msgr2=0x7f17840c0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1783fff640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f178414f360 0x7f17840c0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1783fff640 1 -- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17840c88c0 msgr2=0x7f178414b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1783fff640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17840c88c0 0x7f178414b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:54.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1783fff640 1 -- 172.21.15.26:0/471769531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178414bf00 con 0x7f1784158b60 2024-03-06T07:29:54.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1783fff640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1784158b60 0x7f178414aa20 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f177c00b410 tx=0x7f177c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:54.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f176bfff640 1 -- 172.21.15.26:0/471769531 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c015030 con 0x7f1784158b60 2024-03-06T07:29:54.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f176bfff640 1 -- 172.21.15.26:0/471769531 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c004020 con 0x7f1784158b60 2024-03-06T07:29:54.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178414b6e0 con 0x7f1784158b60 2024-03-06T07:29:54.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f176bfff640 1 -- 172.21.15.26:0/471769531 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c004410 con 0x7f1784158b60 2024-03-06T07:29:54.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.828+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17840bf280 con 0x7f1784158b60 2024-03-06T07:29:54.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.829+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178404f330 con 0x7f1784158b60 2024-03-06T07:29:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.829+0000 7f176bfff640 1 -- 172.21.15.26:0/471769531 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f177c018070 con 0x7f1784158b60 2024-03-06T07:29:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.830+0000 7f176bfff640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f177006b4d0 0x7f177006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.830+0000 7f176bfff640 1 -- 172.21.15.26:0/471769531 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f177c089f20 con 0x7f1784158b60 2024-03-06T07:29:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.830+0000 7f17837fe640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f177006b4d0 0x7f177006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.831+0000 7f17837fe640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f177006b4d0 0x7f177006d990 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f17740012e0 tx=0x7f1774007ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:54.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:54.834+0000 7f176bfff640 1 -- 172.21.15.26:0/471769531 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f177c058b60 con 0x7f1784158b60 2024-03-06T07:29:55.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.069+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17840c8d50 con 0x7f1784158b60 2024-03-06T07:29:55.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.069+0000 7f176bfff640 1 -- 172.21.15.26:0/471769531 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+4216 (secure 0 0 0) 0x7f177c058460 con 0x7f1784158b60 2024-03-06T07:29:55.072 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:55.072 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":203,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:54.670742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:55.072 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 203 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f177006b4d0 msgr2=0x7f177006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f177006b4d0 0x7f177006d990 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f17740012e0 tx=0x7f1774007ed0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1784158b60 msgr2=0x7f178414aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1784158b60 0x7f178414aa20 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f177c00b410 tx=0x7f177c00b8d0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 shutdown_connections 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f177006b4d0 0x7f177006d990 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1784158b60 0x7f178414aa20 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f178414f360 0x7f17840c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 --2- 172.21.15.26:0/471769531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17840c88c0 0x7f178414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 >> 172.21.15.26:0/471769531 conn(0x7f17840ac570 msgr2=0x7f178414d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 shutdown_connections 2024-03-06T07:29:55.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:55.072+0000 7f1789df7640 1 -- 172.21.15.26:0/471769531 wait complete. 2024-03-06T07:29:55.491 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 203, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:54.670742+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:55.491 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:replay 2024-03-06T07:29:56.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.591+0000 7fb144b65640 1 Processor -- start 2024-03-06T07:29:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.591+0000 7fb144b65640 1 -- start start 2024-03-06T07:29:56.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.592+0000 7fb144b65640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1400c1aa0 0x7fb1401529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.592+0000 7fb144b65640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb140155a40 0x7fb140155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.592+0000 7fb144b65640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1400c28d0 con 0x7fb14014f790 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.592+0000 7fb144b65640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400c2680 con 0x7fb140155a40 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.592+0000 7fb144b65640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400c2bb0 con 0x7fb1400c1aa0 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.592+0000 7fb13ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb140155a40 0x7fb140155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.592+0000 7fb13ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb140155a40 0x7fb140155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43018/0 (socket says 172.21.15.26:43018) 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.593+0000 7fb13ed9d640 1 -- 172.21.15.26:0/2787216444 learned_addr learned my addr 172.21.15.26:0/2787216444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.593+0000 7fb13dd9b640 1 --2- 172.21.15.26:0/2787216444 >> v2:172.21.15.117:3300/0 conn(0x7fb1400c1aa0 0x7fb1401529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.593+0000 7fb13ed9d640 1 -- 172.21.15.26:0/2787216444 >> v2:172.21.15.117:3300/0 conn(0x7fb1400c1aa0 msgr2=0x7fb1401529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.593+0000 7fb13ed9d640 1 --2- 172.21.15.26:0/2787216444 >> v2:172.21.15.117:3300/0 conn(0x7fb1400c1aa0 0x7fb1401529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.593+0000 7fb13ed9d640 1 -- 172.21.15.26:0/2787216444 >> v1:172.21.15.117:6789/0 conn(0x7fb14014f790 legacy=0x7fb1400c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.593+0000 7fb13ed9d640 1 -- 172.21.15.26:0/2787216444 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400c2e90 con 0x7fb140155a40 2024-03-06T07:29:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.593+0000 7fb13dd9b640 1 --2- 172.21.15.26:0/2787216444 >> v2:172.21.15.117:3300/0 conn(0x7fb1400c1aa0 0x7fb1401529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:56.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb13ed9d640 1 --2- 172.21.15.26:0/2787216444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140155a40 0x7fb140155e50 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fb140070280 tx=0x7fb13802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e853b6f9fed892b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb13d59a640 1 -- 172.21.15.26:0/2787216444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138034070 con 0x7fb140155a40 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb13d59a640 1 -- 172.21.15.26:0/2787216444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb138030860 con 0x7fb140155a40 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb13d59a640 1 -- 172.21.15.26:0/2787216444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138030c30 con 0x7fb140155a40 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 -- 172.21.15.26:0/2787216444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140155a40 msgr2=0x7fb140155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 --2- 172.21.15.26:0/2787216444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140155a40 0x7fb140155e50 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fb140070280 tx=0x7fb13802b9a0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 -- 172.21.15.26:0/2787216444 shutdown_connections 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 --2- 172.21.15.26:0/2787216444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140155a40 0x7fb140155e50 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 --2- 172.21.15.26:0/2787216444 >> v2:172.21.15.117:3300/0 conn(0x7fb1400c1aa0 0x7fb1401529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 -- 172.21.15.26:0/2787216444 >> 172.21.15.26:0/2787216444 conn(0x7fb1400c6710 msgr2=0x7fb1400c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 -- 172.21.15.26:0/2787216444 shutdown_connections 2024-03-06T07:29:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.594+0000 7fb144b65640 1 -- 172.21.15.26:0/2787216444 wait complete. 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 Processor -- start 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 -- start start 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1400c1aa0 0x7fb14014b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14014f790 0x7fb14014db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb140155a40 0x7fb1401dea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400c0d60 con 0x7fb1400c1aa0 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb140156390 con 0x7fb14014f790 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb144b65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400c5e30 con 0x7fb140155a40 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb13e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1400c1aa0 0x7fb14014b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb13e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1400c1aa0 0x7fb14014b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43032/0 (socket says 172.21.15.26:43032) 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.596+0000 7fb13e59c640 1 -- 172.21.15.26:0/1850776482 learned_addr learned my addr 172.21.15.26:0/1850776482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13ed9d640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb140155a40 0x7fb1401dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:56.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13dd9b640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14014f790 0x7fb14014db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13e59c640 1 -- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb140155a40 msgr2=0x7fb1401dea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13e59c640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb140155a40 0x7fb1401dea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13e59c640 1 -- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14014f790 msgr2=0x7fb14014db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13e59c640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14014f790 0x7fb14014db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13e59c640 1 -- 172.21.15.26:0/1850776482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1401dc320 con 0x7fb1400c1aa0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb13e59c640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1400c1aa0 0x7fb14014b8c0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fb13000c790 tx=0x7fb13000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb136ffd640 1 -- 172.21.15.26:0/1850776482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130016020 con 0x7fb1400c1aa0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb136ffd640 1 -- 172.21.15.26:0/1850776482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb130006d50 con 0x7fb1400c1aa0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb140153ed0 con 0x7fb1400c1aa0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb136ffd640 1 -- 172.21.15.26:0/1850776482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130003bb0 con 0x7fb1400c1aa0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.597+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb140154490 con 0x7fb1400c1aa0 2024-03-06T07:29:56.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.598+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14004f330 con 0x7fb1400c1aa0 2024-03-06T07:29:56.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.599+0000 7fb136ffd640 1 -- 172.21.15.26:0/1850776482 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb13000a850 con 0x7fb1400c1aa0 2024-03-06T07:29:56.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.599+0000 7fb136ffd640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb12006b500 0x7fb12006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:56.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.599+0000 7fb136ffd640 1 -- 172.21.15.26:0/1850776482 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fb13008a0d0 con 0x7fb1400c1aa0 2024-03-06T07:29:56.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.600+0000 7fb13dd9b640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb12006b500 0x7fb12006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:56.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.600+0000 7fb13dd9b640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb12006b500 0x7fb12006d9c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb14014be00 tx=0x7fb128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.603+0000 7fb136ffd640 1 -- 172.21.15.26:0/1850776482 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb130020020 con 0x7fb1400c1aa0 2024-03-06T07:29:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.824+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1400c1f40 con 0x7fb1400c1aa0 2024-03-06T07:29:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.825+0000 7fb136ffd640 1 -- 172.21.15.26:0/1850776482 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+4220 (secure 0 0 0) 0x7fb130019310 con 0x7fb1400c1aa0 2024-03-06T07:29:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:56.827 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:55.679965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 205 2024-03-06T07:29:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb12006b500 msgr2=0x7fb12006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb12006b500 0x7fb12006d9c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb14014be00 tx=0x7fb128008040 comp rx=0 tx=0).stop 2024-03-06T07:29:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1400c1aa0 msgr2=0x7fb14014b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1400c1aa0 0x7fb14014b8c0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fb13000c790 tx=0x7fb13000cc50 comp rx=0 tx=0).stop 2024-03-06T07:29:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 shutdown_connections 2024-03-06T07:29:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb12006b500 0x7fb12006d9c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb140155a40 0x7fb1401dea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14014f790 0x7fb14014db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.826+0000 7fb144b65640 1 --2- 172.21.15.26:0/1850776482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1400c1aa0 0x7fb14014b8c0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.827+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 >> 172.21.15.26:0/1850776482 conn(0x7fb1400c6710 msgr2=0x7fb1400c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.827+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 shutdown_connections 2024-03-06T07:29:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:56.827+0000 7fb144b65640 1 -- 172.21.15.26:0/1850776482 wait complete. 2024-03-06T07:29:57.243 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:55.679965+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:57.243 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:reconnect 2024-03-06T07:29:58.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.343+0000 7f3a729a1640 1 Processor -- start 2024-03-06T07:29:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.343+0000 7f3a729a1640 1 -- start start 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.343+0000 7f3a729a1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a6c1506b0 0x7f3a6c153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.343+0000 7f3a729a1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a6c0acc00 0x7f3a6c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.343+0000 7f3a729a1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c070aa0 con 0x7f3a6c156ea0 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.343+0000 7f3a729a1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c156450 con 0x7f3a6c0acc00 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.343+0000 7f3a729a1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c156a10 con 0x7f3a6c1506b0 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a6b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3a6c156ea0 0x7f3a6c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41858/0 (socket says 172.21.15.26:41858) 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a6b7fe640 1 -- 172.21.15.26:0/324182804 learned_addr learned my addr 172.21.15.26:0/324182804 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:29:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a70f3e640 1 --2- 172.21.15.26:0/324182804 >> v2:172.21.15.26:3300/0 conn(0x7f3a6c0acc00 0x7f3a6c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a6bfff640 1 --2- 172.21.15.26:0/324182804 >> v2:172.21.15.117:3300/0 conn(0x7f3a6c1506b0 0x7f3a6c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a70f3e640 1 -- 172.21.15.26:0/324182804 >> v2:172.21.15.117:3300/0 conn(0x7f3a6c1506b0 msgr2=0x7f3a6c153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a70f3e640 1 --2- 172.21.15.26:0/324182804 >> v2:172.21.15.117:3300/0 conn(0x7f3a6c1506b0 0x7f3a6c153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a70f3e640 1 -- 172.21.15.26:0/324182804 >> v1:172.21.15.26:6789/0 conn(0x7f3a6c156ea0 legacy=0x7f3a6c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a70f3e640 1 -- 172.21.15.26:0/324182804 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c070d80 con 0x7f3a6c0acc00 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.344+0000 7f3a6bfff640 1 --2- 172.21.15.26:0/324182804 >> v2:172.21.15.117:3300/0 conn(0x7f3a6c1506b0 0x7f3a6c153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a70f3e640 1 --2- 172.21.15.26:0/324182804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c0acc00 0x7f3a6c0c54a0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0c97e0 tx=0x7f3a6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bdbb030b8e6405 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a6affd640 1 -- 172.21.15.26:0/324182804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64039070 con 0x7f3a6c0acc00 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a6affd640 1 -- 172.21.15.26:0/324182804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6402cc80 con 0x7f3a6c0acc00 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a6affd640 1 -- 172.21.15.26:0/324182804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64034660 con 0x7f3a6c0acc00 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a729a1640 1 -- 172.21.15.26:0/324182804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c0acc00 msgr2=0x7f3a6c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a729a1640 1 --2- 172.21.15.26:0/324182804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c0acc00 0x7f3a6c0c54a0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0c97e0 tx=0x7f3a6402bed0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a729a1640 1 -- 172.21.15.26:0/324182804 shutdown_connections 2024-03-06T07:29:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a729a1640 1 --2- 172.21.15.26:0/324182804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c0acc00 0x7f3a6c0c54a0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a729a1640 1 --2- 172.21.15.26:0/324182804 >> v2:172.21.15.117:3300/0 conn(0x7f3a6c1506b0 0x7f3a6c153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a729a1640 1 -- 172.21.15.26:0/324182804 >> 172.21.15.26:0/324182804 conn(0x7f3a6c06b1a0 msgr2=0x7f3a6c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.345+0000 7f3a729a1640 1 -- 172.21.15.26:0/324182804 shutdown_connections 2024-03-06T07:29:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.346+0000 7f3a729a1640 1 -- 172.21.15.26:0/324182804 wait complete. 2024-03-06T07:29:58.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 Processor -- start 2024-03-06T07:29:58.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 -- start start 2024-03-06T07:29:58.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a6c0acc00 0x7f3a6c0c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6c1506b0 0x7f3a6c1ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c156ea0 0x7f3a6c1eea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0ad1d0 con 0x7f3a6c156ea0 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0ad050 con 0x7f3a6c1506b0 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.347+0000 7f3a729a1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0ad350 con 0x7f3a6c0acc00 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c156ea0 0x7f3a6c1eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c156ea0 0x7f3a6c1eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40304/0 (socket says 172.21.15.26:40304) 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 -- 172.21.15.26:0/1096196100 learned_addr learned my addr 172.21.15.26:0/1096196100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a6bfff640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a6c0acc00 0x7f3a6c0c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a6b7fe640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6c1506b0 0x7f3a6c1ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 -- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a6c0acc00 msgr2=0x7f3a6c0c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a6c0acc00 0x7f3a6c0c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 -- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6c1506b0 msgr2=0x7f3a6c1ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:58.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6c1506b0 0x7f3a6c1ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 -- 172.21.15.26:0/1096196100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0a8790 con 0x7f3a6c156ea0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a6b7fe640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6c1506b0 0x7f3a6c1ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a70f3e640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c156ea0 0x7f3a6c1eea10 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009e80 tx=0x7f3a64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a68ff9640 1 -- 172.21.15.26:0/1096196100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64039050 con 0x7f3a6c156ea0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a68ff9640 1 -- 172.21.15.26:0/1096196100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64045070 con 0x7f3a6c156ea0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c0a84f0 con 0x7f3a6c156ea0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a68ff9640 1 -- 172.21.15.26:0/1096196100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64009450 con 0x7f3a6c156ea0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.348+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c06c1e0 con 0x7f3a6c156ea0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.349+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c04f330 con 0x7f3a6c156ea0 2024-03-06T07:29:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.350+0000 7f3a68ff9640 1 -- 172.21.15.26:0/1096196100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a6403e850 con 0x7f3a6c156ea0 2024-03-06T07:29:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.350+0000 7f3a68ff9640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5806b500 0x7f3a5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.351+0000 7f3a68ff9640 1 -- 172.21.15.26:0/1096196100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f3a640ad030 con 0x7f3a6c156ea0 2024-03-06T07:29:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.351+0000 7f3a6bfff640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5806b500 0x7f3a5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.351+0000 7f3a6bfff640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5806b500 0x7f3a5806d9c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0c4b10 tx=0x7f3a5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:58.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.355+0000 7f3a68ff9640 1 -- 172.21.15.26:0/1096196100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a6407be20 con 0x7f3a6c156ea0 2024-03-06T07:29:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.572+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a6c0c2da0 con 0x7f3a6c156ea0 2024-03-06T07:29:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.573+0000 7f3a68ff9640 1 -- 172.21.15.26:0/1096196100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+4217 (secure 0 0 0) 0x7f3a6407b720 con 0x7f3a6c156ea0 2024-03-06T07:29:58.576 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:29:58.576 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":206,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:56.681869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:29:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 206 2024-03-06T07:29:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.574+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5806b500 msgr2=0x7f3a5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.574+0000 7f3a729a1640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5806b500 0x7f3a5806d9c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0c4b10 tx=0x7f3a5c008040 comp rx=0 tx=0).stop 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.574+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c156ea0 msgr2=0x7f3a6c1eea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.574+0000 7f3a729a1640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c156ea0 0x7f3a6c1eea10 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009e80 tx=0x7f3a64004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 shutdown_connections 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3a5806b500 0x7f3a5806d9c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c156ea0 0x7f3a6c1eea10 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6c1506b0 0x7f3a6c1ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 --2- 172.21.15.26:0/1096196100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a6c0acc00 0x7f3a6c0c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 >> 172.21.15.26:0/1096196100 conn(0x7f3a6c06b1a0 msgr2=0x7f3a6c0c35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 shutdown_connections 2024-03-06T07:29:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:29:58.575+0000 7f3a729a1640 1 -- 172.21.15.26:0/1096196100 wait complete. 2024-03-06T07:29:58.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 206, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:56.681869+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:rejoin', 'state_seq': 15, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:29:58.991 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:rejoin 2024-03-06T07:29:59.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.093+0000 7fbbee34b640 1 Processor -- start 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.093+0000 7fbbee34b640 1 -- start start 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbee34b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbbe8153990 0x7fbbe8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbee34b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbe80acc00 0x7fbbe80c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbee34b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe8070aa0 con 0x7fbbe81506b0 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbee34b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe8156450 con 0x7fbbe80acc00 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbee34b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe8156a10 con 0x7fbbe8153990 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbec8e8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbe80acc00 0x7fbbe80c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbec8e8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbe80acc00 0x7fbbe80c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40310/0 (socket says 172.21.15.26:40310) 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2904284095 learned_addr learned my addr 172.21.15.26:0/2904284095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbe77fe640 1 --2- 172.21.15.26:0/2904284095 >> v2:172.21.15.117:3300/0 conn(0x7fbbe8153990 0x7fbbe8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2904284095 >> v2:172.21.15.117:3300/0 conn(0x7fbbe8153990 msgr2=0x7fbbe8156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbec8e8640 1 --2- 172.21.15.26:0/2904284095 >> v2:172.21.15.117:3300/0 conn(0x7fbbe8153990 0x7fbbe8156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2904284095 >> v1:172.21.15.117:6789/0 conn(0x7fbbe81506b0 legacy=0x7fbbe80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2904284095 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8070d80 con 0x7fbbe80acc00 2024-03-06T07:30:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.094+0000 7fbbe77fe640 1 --2- 172.21.15.26:0/2904284095 >> v2:172.21.15.117:3300/0 conn(0x7fbbe8153990 0x7fbbe8156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.095+0000 7fbbec8e8640 1 --2- 172.21.15.26:0/2904284095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe80acc00 0x7fbbe80c54a0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8150a90 tx=0x7fbbe002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286b9a353f045f14 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.095+0000 7fbbe6ffd640 1 -- 172.21.15.26:0/2904284095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe002ca80 con 0x7fbbe80acc00 2024-03-06T07:30:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.095+0000 7fbbe6ffd640 1 -- 172.21.15.26:0/2904284095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe002cc20 con 0x7fbbe80acc00 2024-03-06T07:30:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.095+0000 7fbbe6ffd640 1 -- 172.21.15.26:0/2904284095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00343f0 con 0x7fbbe80acc00 2024-03-06T07:30:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 -- 172.21.15.26:0/2904284095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe80acc00 msgr2=0x7fbbe80c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2904284095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe80acc00 0x7fbbe80c54a0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8150a90 tx=0x7fbbe002bee0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 -- 172.21.15.26:0/2904284095 shutdown_connections 2024-03-06T07:30:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2904284095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe80acc00 0x7fbbe80c54a0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2904284095 >> v2:172.21.15.117:3300/0 conn(0x7fbbe8153990 0x7fbbe8156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 -- 172.21.15.26:0/2904284095 >> 172.21.15.26:0/2904284095 conn(0x7fbbe806b1a0 msgr2=0x7fbbe806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 -- 172.21.15.26:0/2904284095 shutdown_connections 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.096+0000 7fbbee34b640 1 -- 172.21.15.26:0/2904284095 wait complete. 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.097+0000 7fbbee34b640 1 Processor -- start 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.097+0000 7fbbee34b640 1 -- start start 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbee34b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbe80acc00 0x7fbbe8151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbee34b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe81506b0 0x7fbbe80c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbee34b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe8153990 0x7fbbe81ec450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbee34b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c5b60 con 0x7fbbe8153990 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbee34b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c59e0 con 0x7fbbe81506b0 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbee34b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c5ce0 con 0x7fbbe80acc00 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe8153990 0x7fbbe81ec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe8153990 0x7fbbe81ec450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40322/0 (socket says 172.21.15.26:40322) 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2274503524 learned_addr learned my addr 172.21.15.26:0/2274503524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbe7fff640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbe80acc00 0x7fbbe8151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbe77fe640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe81506b0 0x7fbbe80c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbe80acc00 msgr2=0x7fbbe8151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbe80acc00 0x7fbbe8151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe81506b0 msgr2=0x7fbbe80c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe81506b0 0x7fbbe80c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 -- 172.21.15.26:0/2274503524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80c3210 con 0x7fbbe8153990 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.098+0000 7fbbec8e8640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe8153990 0x7fbbe81ec450 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e20 tx=0x7fbbe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.099+0000 7fbbe4ff9640 1 -- 172.21.15.26:0/2274503524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0043020 con 0x7fbbe8153990 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.099+0000 7fbbe4ff9640 1 -- 172.21.15.26:0/2274503524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe003e450 con 0x7fbbe8153990 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.099+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe80c34f0 con 0x7fbbe8153990 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.099+0000 7fbbe4ff9640 1 -- 172.21.15.26:0/2274503524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0009510 con 0x7fbbe8153990 2024-03-06T07:30:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.099+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe8151200 con 0x7fbbe8153990 2024-03-06T07:30:00.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.100+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe804f330 con 0x7fbbe8153990 2024-03-06T07:30:00.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.100+0000 7fbbe4ff9640 1 -- 172.21.15.26:0/2274503524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbe0046020 con 0x7fbbe8153990 2024-03-06T07:30:00.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.101+0000 7fbbe4ff9640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbd406b550 0x7fbbd406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.101+0000 7fbbe4ff9640 1 -- 172.21.15.26:0/2274503524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fbbe00ac730 con 0x7fbbe8153990 2024-03-06T07:30:00.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.101+0000 7fbbe7fff640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbd406b550 0x7fbbd406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.101+0000 7fbbe7fff640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbd406b550 0x7fbbd406da10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8004f60 tx=0x7fbbd8007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.105+0000 7fbbe4ff9640 1 -- 172.21.15.26:0/2274503524 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbbe004d070 con 0x7fbbe8153990 2024-03-06T07:30:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.403+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbe80a7cd0 con 0x7fbbe8153990 2024-03-06T07:30:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.403+0000 7fbbe4ff9640 1 -- 172.21.15.26:0/2274503524 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fbbe007af50 con 0x7fbbe8153990 2024-03-06T07:30:00.406 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:00.406 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:00.407 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbd406b550 msgr2=0x7fbbd406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbd406b550 0x7fbbd406da10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8004f60 tx=0x7fbbd8007e30 comp rx=0 tx=0).stop 2024-03-06T07:30:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe8153990 msgr2=0x7fbbe81ec450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe8153990 0x7fbbe81ec450 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e20 tx=0x7fbbe0004a40 comp rx=0 tx=0).stop 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 shutdown_connections 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbbd406b550 0x7fbbd406da10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe8153990 0x7fbbe81ec450 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe81506b0 0x7fbbe80c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 --2- 172.21.15.26:0/2274503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbbe80acc00 0x7fbbe8151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.405+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 >> 172.21.15.26:0/2274503524 conn(0x7fbbe806b1a0 msgr2=0x7fbbe80ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.406+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 shutdown_connections 2024-03-06T07:30:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:00.406+0000 7fbbee34b640 1 -- 172.21.15.26:0/2274503524 wait complete. 2024-03-06T07:30:00.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:00.821 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:01.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:01.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.921+0000 7fbfc4d8a640 1 Processor -- start 2024-03-06T07:30:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.921+0000 7fbfc4d8a640 1 -- start start 2024-03-06T07:30:01.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfc4d8a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc01506b0 0x7fbfc0153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfc4d8a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc0070aa0 con 0x7fbfc0156ea0 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfc4d8a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc0070d80 con 0x7fbfc00acc00 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfc4d8a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc0156450 con 0x7fbfc01506b0 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfbe59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc01506b0 0x7fbfc0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfbe59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc01506b0 0x7fbfc0153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40334/0 (socket says 172.21.15.26:40334) 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfbe59c640 1 -- 172.21.15.26:0/3976535082 learned_addr learned my addr 172.21.15.26:0/3976535082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.922+0000 7fbfb7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbfc0156ea0 0x7fbfc00702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41868/0 (socket says 172.21.15.26:41868) 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.923+0000 7fbfbe59c640 1 -- 172.21.15.26:0/3976535082 >> v1:172.21.15.117:6789/0 conn(0x7fbfc00acc00 legacy=0x7fbfc00c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.923+0000 7fbfbe59c640 1 -- 172.21.15.26:0/3976535082 >> v1:172.21.15.26:6789/0 conn(0x7fbfc0156ea0 legacy=0x7fbfc00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.923+0000 7fbfbe59c640 1 -- 172.21.15.26:0/3976535082 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc01566a0 con 0x7fbfc01506b0 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.923+0000 7fbfbdd9b640 1 -- 172.21.15.26:0/3976535082 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5715518 0 0) 0x7fbfc0070aa0 con 0x7fbfc0156ea0 2024-03-06T07:30:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfbe59c640 1 --2- 172.21.15.26:0/3976535082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc01506b0 0x7fbfc0153990 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37195939e5ffc52 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfbdd9b640 1 -- 172.21.15.26:0/3976535082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0034070 con 0x7fbfc01506b0 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfbdd9b640 1 -- 172.21.15.26:0/3976535082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0030860 con 0x7fbfc01506b0 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfbdd9b640 1 -- 172.21.15.26:0/3976535082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0030c30 con 0x7fbfc01506b0 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/3976535082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc01506b0 msgr2=0x7fbfc0153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/3976535082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc01506b0 0x7fbfc0153990 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/3976535082 shutdown_connections 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/3976535082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc01506b0 0x7fbfc0153990 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/3976535082 >> 172.21.15.26:0/3976535082 conn(0x7fbfc006b1a0 msgr2=0x7fbfc006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/3976535082 shutdown_connections 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.924+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/3976535082 wait complete. 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.925+0000 7fbfc4d8a640 1 Processor -- start 2024-03-06T07:30:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfc4d8a640 1 -- start start 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfc4d8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00acc00 0x7fbfc01eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfc4d8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc01506b0 0x7fbfc0151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfc4d8a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc0156ea0 0x7fbfc01494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfc4d8a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00ad050 con 0x7fbfc00acc00 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfc4d8a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0156a10 con 0x7fbfc01506b0 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfc4d8a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00ad1d0 con 0x7fbfc0156ea0 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfbe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00acc00 0x7fbfc01eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfbe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00acc00 0x7fbfc01eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40340/0 (socket says 172.21.15.26:40340) 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.926+0000 7fbfbe59c640 1 -- 172.21.15.26:0/2964264326 learned_addr learned my addr 172.21.15.26:0/2964264326 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfbe59c640 1 -- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc0156ea0 msgr2=0x7fbfc01494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfbe59c640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc0156ea0 0x7fbfc01494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfbe59c640 1 -- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc01506b0 msgr2=0x7fbfc0151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfbe59c640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc01506b0 0x7fbfc0151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfbe59c640 1 -- 172.21.15.26:0/2964264326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc01528b0 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfbe59c640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00acc00 0x7fbfc01eec70 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfb6ffd640 1 -- 172.21.15.26:0/2964264326 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0048070 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfb6ffd640 1 -- 172.21.15.26:0/2964264326 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0030660 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc006d610 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfb6ffd640 1 -- 172.21.15.26:0/2964264326 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0034040 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.927+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc006dbd0 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.928+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc004f330 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.928+0000 7fbfb6ffd640 1 -- 172.21.15.26:0/2964264326 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfb004b070 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.929+0000 7fbfb6ffd640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfa006b4d0 0x7fbfa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.929+0000 7fbfb6ffd640 1 -- 172.21.15.26:0/2964264326 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fbfb00ac4d0 con 0x7fbfc00acc00 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.929+0000 7fbfb7fff640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfa006b4d0 0x7fbfa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.929+0000 7fbfb7fff640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfa006b4d0 0x7fbfa006d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80011e0 tx=0x7fbfa8007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:01.933+0000 7fbfb6ffd640 1 -- 172.21.15.26:0/2964264326 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbfc01528b0 con 0x7fbfc00acc00 2024-03-06T07:30:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.172+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfc00700a0 con 0x7fbfc00acc00 2024-03-06T07:30:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.172+0000 7fbfb6ffd640 1 -- 172.21.15.26:0/2964264326 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fbfb007b1a0 con 0x7fbfc00acc00 2024-03-06T07:30:02.175 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:02.175 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfa006b4d0 msgr2=0x7fbfa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfa006b4d0 0x7fbfa006d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80011e0 tx=0x7fbfa8007920 comp rx=0 tx=0).stop 2024-03-06T07:30:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00acc00 msgr2=0x7fbfc01eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00acc00 0x7fbfc01eec70 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb0004730 comp rx=0 tx=0).stop 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 shutdown_connections 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfa006b4d0 0x7fbfa006d990 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc0156ea0 0x7fbfc01494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.174+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc01506b0 0x7fbfc0151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.175+0000 7fbfc4d8a640 1 --2- 172.21.15.26:0/2964264326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00acc00 0x7fbfc01eec70 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.175+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 >> 172.21.15.26:0/2964264326 conn(0x7fbfc006b1a0 msgr2=0x7fbfc0152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.175+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 shutdown_connections 2024-03-06T07:30:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:02.175+0000 7fbfc4d8a640 1 -- 172.21.15.26:0/2964264326 wait complete. 2024-03-06T07:30:02.590 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:02.590 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:03.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.690+0000 7fae06525640 1 Processor -- start 2024-03-06T07:30:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.691+0000 7fae06525640 1 -- start start 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.691+0000 7fae06525640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae0014f790 0x7fae001529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.691+0000 7fae06525640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae000c1aa0 0x7fae000c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.691+0000 7fae06525640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae000c28d0 con 0x7fae000c5c60 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.691+0000 7fae06525640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae000c2680 con 0x7fae000c1aa0 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.691+0000 7fae06525640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae000c2bb0 con 0x7fae0014f790 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadfffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae0014f790 0x7fae001529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadff7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae000c1aa0 0x7fae000c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadfffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae0014f790 0x7fae001529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:39306/0 (socket says 172.21.15.26:39306) 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadff7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae000c1aa0 0x7fae000c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58560/0 (socket says 172.21.15.26:58560) 2024-03-06T07:30:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadfffff640 1 -- 172.21.15.26:0/1669788328 learned_addr learned my addr 172.21.15.26:0/1669788328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fae04ac2640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fae000c5c60 0x7fae00155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:34288/0 (socket says 172.21.15.26:34288) 2024-03-06T07:30:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadfffff640 1 -- 172.21.15.26:0/1669788328 >> v2:172.21.15.117:3300/0 conn(0x7fae000c1aa0 msgr2=0x7fae000c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadfffff640 1 --2- 172.21.15.26:0/1669788328 >> v2:172.21.15.117:3300/0 conn(0x7fae000c1aa0 0x7fae000c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadfffff640 1 -- 172.21.15.26:0/1669788328 >> v1:172.21.15.117:6789/0 conn(0x7fae000c5c60 legacy=0x7fae00155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.692+0000 7fadfffff640 1 -- 172.21.15.26:0/1669788328 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000c2e90 con 0x7fae0014f790 2024-03-06T07:30:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.693+0000 7fadfffff640 1 --2- 172.21.15.26:0/1669788328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae0014f790 0x7fae001529d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadf000bb10 tx=0x7fadf002e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f41e74982694286 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.693+0000 7fadfeffd640 1 -- 172.21.15.26:0/1669788328 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf003b070 con 0x7fae0014f790 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.693+0000 7fadfeffd640 1 -- 172.21.15.26:0/1669788328 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0036440 con 0x7fae0014f790 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.693+0000 7fadfeffd640 1 -- 172.21.15.26:0/1669788328 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0036810 con 0x7fae0014f790 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 -- 172.21.15.26:0/1669788328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae0014f790 msgr2=0x7fae001529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 --2- 172.21.15.26:0/1669788328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae0014f790 0x7fae001529d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadf000bb10 tx=0x7fadf002e150 comp rx=0 tx=0).stop 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 -- 172.21.15.26:0/1669788328 shutdown_connections 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 --2- 172.21.15.26:0/1669788328 >> v2:172.21.15.117:3300/0 conn(0x7fae000c1aa0 0x7fae000c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 --2- 172.21.15.26:0/1669788328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae0014f790 0x7fae001529d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 -- 172.21.15.26:0/1669788328 >> 172.21.15.26:0/1669788328 conn(0x7fae000c6710 msgr2=0x7fae000c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 -- 172.21.15.26:0/1669788328 shutdown_connections 2024-03-06T07:30:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.694+0000 7fae06525640 1 -- 172.21.15.26:0/1669788328 wait complete. 2024-03-06T07:30:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.695+0000 7fae06525640 1 Processor -- start 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.696+0000 7fae06525640 1 -- start start 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae06525640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae000c1aa0 0x7fae0014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae06525640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae000c5c60 0x7fae00151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae06525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0014f790 0x7fae001e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae06525640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0006cc00 con 0x7fae0014f790 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae06525640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000c0d60 con 0x7fae000c5c60 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae06525640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0006cd80 con 0x7fae000c1aa0 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0014f790 0x7fae001e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0014f790 0x7fae001e6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40352/0 (socket says 172.21.15.26:40352) 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 -- 172.21.15.26:0/90868387 learned_addr learned my addr 172.21.15.26:0/90868387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fadfffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae000c1aa0 0x7fae0014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fadff7fe640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae000c5c60 0x7fae00151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 -- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae000c1aa0 msgr2=0x7fae0014e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae000c1aa0 0x7fae0014e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 -- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae000c5c60 msgr2=0x7fae00151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae000c5c60 0x7fae00151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 -- 172.21.15.26:0/90868387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae001e6950 con 0x7fae0014f790 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.697+0000 7fae04ac2640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0014f790 0x7fae001e6e50 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fadf800c780 tx=0x7fadf800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.698+0000 7fadfcff9640 1 -- 172.21.15.26:0/90868387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8017070 con 0x7fae0014f790 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.698+0000 7fadfcff9640 1 -- 172.21.15.26:0/90868387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf800a640 con 0x7fae0014f790 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.698+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0014eb30 con 0x7fae0014f790 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.698+0000 7fadfcff9640 1 -- 172.21.15.26:0/90868387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf800aa30 con 0x7fae0014f790 2024-03-06T07:30:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.698+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae001dcbc0 con 0x7fae0014f790 2024-03-06T07:30:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.699+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0004f330 con 0x7fae0014f790 2024-03-06T07:30:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.701+0000 7fadfcff9640 1 -- 172.21.15.26:0/90868387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadf801a070 con 0x7fae0014f790 2024-03-06T07:30:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.701+0000 7fadfcff9640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadec06b4d0 0x7fadec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.701+0000 7fadfcff9640 1 -- 172.21.15.26:0/90868387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fadf808a130 con 0x7fae0014f790 2024-03-06T07:30:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.701+0000 7fadfffff640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadec06b4d0 0x7fadec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.702+0000 7fadfffff640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadec06b4d0 0x7fadec06d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fadf0038000 tx=0x7fadf0032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:03.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.709+0000 7fadfcff9640 1 -- 172.21.15.26:0/90868387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fadf8058790 con 0x7fae0014f790 2024-03-06T07:30:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.976+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae000c1f40 con 0x7fae0014f790 2024-03-06T07:30:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.977+0000 7fadfcff9640 1 -- 172.21.15.26:0/90868387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fadf805c830 con 0x7fae0014f790 2024-03-06T07:30:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:03.980 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:03.980 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:03.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadec06b4d0 msgr2=0x7fadec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:03.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadec06b4d0 0x7fadec06d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fadf0038000 tx=0x7fadf0032000 comp rx=0 tx=0).stop 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0014f790 msgr2=0x7fae001e6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0014f790 0x7fae001e6e50 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fadf800c780 tx=0x7fadf800cc40 comp rx=0 tx=0).stop 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 shutdown_connections 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fadec06b4d0 0x7fadec06d990 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0014f790 0x7fae001e6e50 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae000c5c60 0x7fae00151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 --2- 172.21.15.26:0/90868387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae000c1aa0 0x7fae0014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 >> 172.21.15.26:0/90868387 conn(0x7fae000c6710 msgr2=0x7fae000c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 shutdown_connections 2024-03-06T07:30:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:03.979+0000 7fae06525640 1 -- 172.21.15.26:0/90868387 wait complete. 2024-03-06T07:30:04.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:04.395 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:05.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:05.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.496+0000 7feaddcc2640 1 Processor -- start 2024-03-06T07:30:05.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.496+0000 7feaddcc2640 1 -- start start 2024-03-06T07:30:05.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7feaddcc2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fead814f360 0x7fead8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:05.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7feaddcc2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c88c0 0x7fead80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:05.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7feaddcc2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead80c73d0 con 0x7fead80c5880 2024-03-06T07:30:05.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7feaddcc2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead80c8dd0 con 0x7fead814f360 2024-03-06T07:30:05.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7feaddcc2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead80be2e0 con 0x7fead80c88c0 2024-03-06T07:30:05.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead6ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c88c0 0x7fead80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:05.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead77fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fead814f360 0x7fead8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:05.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead7fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fead80c5880 0x7fead8158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:34296/0 (socket says 172.21.15.26:34296) 2024-03-06T07:30:05.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead6ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c88c0 0x7fead80c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:39328/0 (socket says 172.21.15.26:39328) 2024-03-06T07:30:05.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead7fff640 1 -- 172.21.15.26:0/154101935 learned_addr learned my addr 172.21.15.26:0/154101935 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:05.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead6ffd640 1 -- 172.21.15.26:0/154101935 >> v2:172.21.15.117:3300/0 conn(0x7fead814f360 msgr2=0x7fead8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:05.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead6ffd640 1 --2- 172.21.15.26:0/154101935 >> v2:172.21.15.117:3300/0 conn(0x7fead814f360 0x7fead8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead6ffd640 1 -- 172.21.15.26:0/154101935 >> v1:172.21.15.117:6789/0 conn(0x7fead80c5880 legacy=0x7fead8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:05.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead6ffd640 1 -- 172.21.15.26:0/154101935 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead80c7670 con 0x7fead80c88c0 2024-03-06T07:30:05.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.497+0000 7fead77fe640 1 --2- 172.21.15.26:0/154101935 >> v2:172.21.15.117:3300/0 conn(0x7fead814f360 0x7fead8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:05.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.498+0000 7fead6ffd640 1 --2- 172.21.15.26:0/154101935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c88c0 0x7fead80c5340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feac00097a0 tx=0x7feac002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37228dae9e7dfbcb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:05.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.498+0000 7fead67fc640 1 -- 172.21.15.26:0/154101935 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00315a0 con 0x7fead80c88c0 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.498+0000 7fead67fc640 1 -- 172.21.15.26:0/154101935 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac0037070 con 0x7fead80c88c0 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.498+0000 7fead67fc640 1 -- 172.21.15.26:0/154101935 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0031960 con 0x7fead80c88c0 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 -- 172.21.15.26:0/154101935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c88c0 msgr2=0x7fead80c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 --2- 172.21.15.26:0/154101935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c88c0 0x7fead80c5340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feac00097a0 tx=0x7feac002b840 comp rx=0 tx=0).stop 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 -- 172.21.15.26:0/154101935 shutdown_connections 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 --2- 172.21.15.26:0/154101935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c88c0 0x7fead80c5340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 --2- 172.21.15.26:0/154101935 >> v2:172.21.15.117:3300/0 conn(0x7fead814f360 0x7fead8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 -- 172.21.15.26:0/154101935 >> 172.21.15.26:0/154101935 conn(0x7fead80ac570 msgr2=0x7fead80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 -- 172.21.15.26:0/154101935 shutdown_connections 2024-03-06T07:30:05.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.499+0000 7feaddcc2640 1 -- 172.21.15.26:0/154101935 wait complete. 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.500+0000 7feaddcc2640 1 Processor -- start 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.500+0000 7feaddcc2640 1 -- start start 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7feaddcc2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c5880 0x7fead81eba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7feaddcc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead80c88c0 0x7fead81539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7feaddcc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fead814f360 0x7fead80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7feaddcc2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead8159270 con 0x7fead814f360 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7feaddcc2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead80c7c30 con 0x7fead80c88c0 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7feaddcc2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead81593f0 con 0x7fead80c5880 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fead814f360 0x7fead80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fead814f360 0x7fead80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40354/0 (socket says 172.21.15.26:40354) 2024-03-06T07:30:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead80c88c0 0x7fead81539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 -- 172.21.15.26:0/3726619487 learned_addr learned my addr 172.21.15.26:0/3726619487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead77fe640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c5880 0x7fead81eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 -- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c5880 msgr2=0x7fead81eba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c5880 0x7fead81eba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 -- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead80c88c0 msgr2=0x7fead81539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead80c88c0 0x7fead81539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 -- 172.21.15.26:0/3726619487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8152f50 con 0x7fead814f360 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.501+0000 7fead7fff640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fead814f360 0x7fead80ad720 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fead000cbc0 tx=0x7fead000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.502+0000 7feacffff640 1 -- 172.21.15.26:0/3726619487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0017070 con 0x7fead814f360 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.502+0000 7feacffff640 1 -- 172.21.15.26:0/3726619487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead000ad40 con 0x7fead814f360 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.502+0000 7feacffff640 1 -- 172.21.15.26:0/3726619487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0012600 con 0x7fead814f360 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.502+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead8153f30 con 0x7fead814f360 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.502+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead81531f0 con 0x7fead814f360 2024-03-06T07:30:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.503+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead8069460 con 0x7fead814f360 2024-03-06T07:30:05.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.503+0000 7feacffff640 1 -- 172.21.15.26:0/3726619487 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fead001a070 con 0x7fead814f360 2024-03-06T07:30:05.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.504+0000 7feacffff640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feab806b500 0x7feab806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:05.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.504+0000 7feacffff640 1 -- 172.21.15.26:0/3726619487 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fead0089fd0 con 0x7fead814f360 2024-03-06T07:30:05.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.504+0000 7fead77fe640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feab806b500 0x7feab806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:05.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.505+0000 7fead77fe640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feab806b500 0x7feab806d9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7feac8009560 tx=0x7feac8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:05.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.508+0000 7feacffff640 1 -- 172.21.15.26:0/3726619487 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fead8069460 con 0x7fead814f360 2024-03-06T07:30:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.726+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fead804f330 con 0x7fead814f360 2024-03-06T07:30:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.726+0000 7feacffff640 1 -- 172.21.15.26:0/3726619487 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fead0058650 con 0x7fead814f360 2024-03-06T07:30:05.729 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:05.729 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:05.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feab806b500 msgr2=0x7feab806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:05.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feab806b500 0x7feab806d9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7feac8009560 tx=0x7feac8009190 comp rx=0 tx=0).stop 2024-03-06T07:30:05.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fead814f360 msgr2=0x7fead80ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:05.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fead814f360 0x7fead80ad720 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fead000cbc0 tx=0x7fead000a430 comp rx=0 tx=0).stop 2024-03-06T07:30:05.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 shutdown_connections 2024-03-06T07:30:05.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feab806b500 0x7feab806d9c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fead814f360 0x7fead80ad720 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead80c88c0 0x7fead81539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 --2- 172.21.15.26:0/3726619487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead80c5880 0x7fead81eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:05.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 >> 172.21.15.26:0/3726619487 conn(0x7fead80ac570 msgr2=0x7fead80c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:05.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 shutdown_connections 2024-03-06T07:30:05.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:05.728+0000 7feaddcc2640 1 -- 172.21.15.26:0/3726619487 wait complete. 2024-03-06T07:30:06.144 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:06.144 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:07.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:07.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.247+0000 7f6a195d7640 1 Processor -- start 2024-03-06T07:30:07.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.248+0000 7f6a195d7640 1 -- start start 2024-03-06T07:30:07.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.248+0000 7f6a195d7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6a140c88c0 0x7f6a14155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.248+0000 7f6a195d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a14158b60 0x7f6a14158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.248+0000 7f6a195d7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140c73d0 con 0x7f6a1414f360 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.248+0000 7f6a195d7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140c8dd0 con 0x7f6a140c88c0 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.248+0000 7f6a195d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140be2e0 con 0x7f6a14158b60 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a137fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a14158b60 0x7f6a14158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a0a7fc640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6a140c88c0 0x7f6a14155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a12ffd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6a1414f360 0x7f6a140c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:47402/0 (socket says 172.21.15.26:47402) 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a12ffd640 1 -- 172.21.15.26:0/2150529130 learned_addr learned my addr 172.21.15.26:0/2150529130 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a127fc640 1 -- 172.21.15.26:0/2150529130 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791326648 0 0) 0x7f6a140c73d0 con 0x7f6a1414f360 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a127fc640 1 -- 172.21.15.26:0/2150529130 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a00003520 con 0x7f6a1414f360 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a137fe640 1 -- 172.21.15.26:0/2150529130 >> v2:172.21.15.117:3300/0 conn(0x7f6a140c88c0 msgr2=0x7f6a14155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a137fe640 1 --2- 172.21.15.26:0/2150529130 >> v2:172.21.15.117:3300/0 conn(0x7f6a140c88c0 0x7f6a14155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a137fe640 1 -- 172.21.15.26:0/2150529130 >> v1:172.21.15.117:6789/0 conn(0x7f6a1414f360 legacy=0x7f6a140c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a137fe640 1 -- 172.21.15.26:0/2150529130 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140c7670 con 0x7f6a14158b60 2024-03-06T07:30:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.249+0000 7f6a0a7fc640 1 --2- 172.21.15.26:0/2150529130 >> v2:172.21.15.117:3300/0 conn(0x7f6a140c88c0 0x7f6a14155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.250+0000 7f6a137fe640 1 --2- 172.21.15.26:0/2150529130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a14158b60 0x7f6a14158ff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009d90 tx=0x7f6a0c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8d00aac61d283a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.250+0000 7f6a127fc640 1 -- 172.21.15.26:0/2150529130 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c039070 con 0x7f6a14158b60 2024-03-06T07:30:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.250+0000 7f6a127fc640 1 -- 172.21.15.26:0/2150529130 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c02cbc0 con 0x7f6a14158b60 2024-03-06T07:30:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.250+0000 7f6a127fc640 1 -- 172.21.15.26:0/2150529130 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c034440 con 0x7f6a14158b60 2024-03-06T07:30:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 -- 172.21.15.26:0/2150529130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a14158b60 msgr2=0x7f6a14158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 --2- 172.21.15.26:0/2150529130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a14158b60 0x7f6a14158ff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009d90 tx=0x7f6a0c02be10 comp rx=0 tx=0).stop 2024-03-06T07:30:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 -- 172.21.15.26:0/2150529130 shutdown_connections 2024-03-06T07:30:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 --2- 172.21.15.26:0/2150529130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a14158b60 0x7f6a14158ff0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 --2- 172.21.15.26:0/2150529130 >> v2:172.21.15.117:3300/0 conn(0x7f6a140c88c0 0x7f6a14155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 -- 172.21.15.26:0/2150529130 >> 172.21.15.26:0/2150529130 conn(0x7f6a140ac570 msgr2=0x7f6a140ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 -- 172.21.15.26:0/2150529130 shutdown_connections 2024-03-06T07:30:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.251+0000 7f6a195d7640 1 -- 172.21.15.26:0/2150529130 wait complete. 2024-03-06T07:30:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.252+0000 7f6a195d7640 1 Processor -- start 2024-03-06T07:30:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.253+0000 7f6a195d7640 1 -- start start 2024-03-06T07:30:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a195d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a140c88c0 0x7f6a141edba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a195d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1414f360 0x7f6a141578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a195d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a14158b60 0x7f6a14151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a195d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140c5a50 con 0x7f6a1414f360 2024-03-06T07:30:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a195d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140c7c30 con 0x7f6a14158b60 2024-03-06T07:30:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a195d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140c5bd0 con 0x7f6a140c88c0 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1414f360 0x7f6a141578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1414f360 0x7f6a141578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56562/0 (socket says 172.21.15.26:56562) 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 -- 172.21.15.26:0/1781068536 learned_addr learned my addr 172.21.15.26:0/1781068536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a137fe640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a14158b60 0x7f6a14151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a12ffd640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a140c88c0 0x7f6a141edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 -- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a140c88c0 msgr2=0x7f6a141edba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a140c88c0 0x7f6a141edba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 -- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a14158b60 msgr2=0x7f6a14151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a14158b60 0x7f6a14151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 -- 172.21.15.26:0/1781068536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a141eb1e0 con 0x7f6a1414f360 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a12ffd640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a140c88c0 0x7f6a141edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.254+0000 7f6a0a7fc640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1414f360 0x7f6a141578e0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00e900 tx=0x7f69fc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.255+0000 7f6a0bfff640 1 -- 172.21.15.26:0/1781068536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc017070 con 0x7f6a1414f360 2024-03-06T07:30:07.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.255+0000 7f6a0bfff640 1 -- 172.21.15.26:0/1781068536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc00ca00 con 0x7f6a1414f360 2024-03-06T07:30:07.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.255+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1414fc90 con 0x7f6a1414f360 2024-03-06T07:30:07.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.255+0000 7f6a0bfff640 1 -- 172.21.15.26:0/1781068536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00cdf0 con 0x7f6a1414f360 2024-03-06T07:30:07.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.255+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1414f740 con 0x7f6a1414f360 2024-03-06T07:30:07.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.256+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1404f330 con 0x7f6a1414f360 2024-03-06T07:30:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.256+0000 7f6a0bfff640 1 -- 172.21.15.26:0/1781068536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69fc01a070 con 0x7f6a1414f360 2024-03-06T07:30:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.256+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69f406b4d0 0x7f69f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.257+0000 7f6a0bfff640 1 -- 172.21.15.26:0/1781068536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f69fc08a250 con 0x7f6a1414f360 2024-03-06T07:30:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.257+0000 7f6a12ffd640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69f406b4d0 0x7f69f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.257+0000 7f6a12ffd640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69f406b4d0 0x7f69f406d990 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6a04007fd0 tx=0x7f6a04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.265+0000 7f6a0bfff640 1 -- 172.21.15.26:0/1781068536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f69fc058850 con 0x7f6a1414f360 2024-03-06T07:30:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.482+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a140c40c0 con 0x7f6a1414f360 2024-03-06T07:30:07.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.482+0000 7f6a0bfff640 1 -- 172.21.15.26:0/1781068536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f69fc05c8f0 con 0x7f6a1414f360 2024-03-06T07:30:07.485 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:07.485 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:07.485 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69f406b4d0 msgr2=0x7f69f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69f406b4d0 0x7f69f406d990 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6a04007fd0 tx=0x7f6a04009040 comp rx=0 tx=0).stop 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1414f360 msgr2=0x7f6a141578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1414f360 0x7f6a141578e0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00e900 tx=0x7f69fc00edc0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 shutdown_connections 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69f406b4d0 0x7f69f406d990 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a14158b60 0x7f6a14151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1414f360 0x7f6a141578e0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 --2- 172.21.15.26:0/1781068536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a140c88c0 0x7f6a141edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 >> 172.21.15.26:0/1781068536 conn(0x7f6a140ac570 msgr2=0x7f6a1406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 shutdown_connections 2024-03-06T07:30:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:07.485+0000 7f6a195d7640 1 -- 172.21.15.26:0/1781068536 wait complete. 2024-03-06T07:30:07.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:07.904 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:08.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:09.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.005+0000 7f3bad725640 1 Processor -- start 2024-03-06T07:30:09.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.005+0000 7f3bad725640 1 -- start start 2024-03-06T07:30:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3bad725640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3ba80c88b0 0x7f3ba80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3bad725640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8155110 con 0x7f3ba80c5330 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3bad725640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80c8f40 con 0x7f3ba8158db0 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3bad725640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba81559e0 con 0x7f3ba80c88b0 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3ba6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3ba80c5330 0x7f3ba81586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41924/0 (socket says 172.21.15.26:41924) 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3ba6ffd640 1 -- 172.21.15.26:0/4094372152 learned_addr learned my addr 172.21.15.26:0/4094372152 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3ba77fe640 1 --2- 172.21.15.26:0/4094372152 >> v2:172.21.15.117:3300/0 conn(0x7f3ba80c88b0 0x7f3ba80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465807278 0 0) 0x7f3b98001110 con 0x7f3ba80c5330 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.006+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b94003520 con 0x7f3ba80c5330 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 783880658 0 0) 0x7f3b94003520 con 0x7f3ba80c5330 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 >> v2:172.21.15.117:3300/0 conn(0x7f3ba80c88b0 msgr2=0x7f3ba80c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 --2- 172.21.15.26:0/4094372152 >> v2:172.21.15.117:3300/0 conn(0x7f3ba80c88b0 0x7f3ba80c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 >> v1:172.21.15.117:6789/0 conn(0x7f3ba8158db0 legacy=0x7f3ba815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8155390 con 0x7f3ba80c5330 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f3b98001840 con 0x7f3ba80c5330 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 >> v1:172.21.15.26:6789/0 conn(0x7f3ba80c5330 legacy=0x7f3ba81586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b940039b0 0x7f3b94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3b94004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b94005120 0x7f3b9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81559e0 con 0x7f3b940043b0 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9400c060 con 0x7f3b94005120 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9400c260 con 0x7f3b940039b0 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3b9e7fc640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3b94004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba6ffd640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b940039b0 0x7f3b94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba77fe640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b94005120 0x7f3b9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3b9e7fc640 1 -- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b940039b0 msgr2=0x7f3b94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:09.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3b9e7fc640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b940039b0 0x7f3b94003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3b9e7fc640 1 -- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b94005120 msgr2=0x7f3b9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3b9e7fc640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b94005120 0x7f3b9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3b9e7fc640 1 -- 172.21.15.26:0/4094372152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94003520 con 0x7f3b940043b0 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba77fe640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b94005120 0x7f3b9400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.007+0000 7f3ba77fe640 1 -- 172.21.15.26:0/4094372152 reap_dead start 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.008+0000 7f3b9e7fc640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3b94004880 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f3b90001860 tx=0x7f3b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b143b5b8e3bd88 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.008+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b90001430 con 0x7f3b940043b0 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.008+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b90033610 con 0x7f3b940043b0 2024-03-06T07:30:09.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.008+0000 7f3ba67fc640 1 -- 172.21.15.26:0/4094372152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b90033a00 con 0x7f3b940043b0 2024-03-06T07:30:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.008+0000 7f3bad725640 1 -- 172.21.15.26:0/4094372152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 msgr2=0x7f3b94004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.008+0000 7f3bad725640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3b94004880 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f3b90001860 tx=0x7f3b94005830 comp rx=0 tx=0).stop 2024-03-06T07:30:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.009+0000 7f3bad725640 1 -- 172.21.15.26:0/4094372152 shutdown_connections 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.009+0000 7f3bad725640 1 --2- 172.21.15.26:0/4094372152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3b94004880 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.009+0000 7f3bad725640 1 -- 172.21.15.26:0/4094372152 >> 172.21.15.26:0/4094372152 conn(0x7f3ba80ac560 msgr2=0x7f3ba80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.009+0000 7f3bad725640 1 -- 172.21.15.26:0/4094372152 shutdown_connections 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.009+0000 7f3bad725640 1 -- 172.21.15.26:0/4094372152 wait complete. 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.010+0000 7f3bad725640 1 Processor -- start 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.010+0000 7f3bad725640 1 -- start start 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.011+0000 7f3bad725640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3ba814df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.011+0000 7f3bad725640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba81eb110 0x7f3ba81eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3bad725640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba814b2d0 0x7f3ba814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3bad725640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81529d0 con 0x7f3b940043b0 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3bad725640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba806d090 con 0x7f3ba814b2d0 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3bad725640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8152b50 con 0x7f3ba81eb110 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3ba814df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3ba814df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56576/0 (socket says 172.21.15.26:56576) 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 -- 172.21.15.26:0/3885000434 learned_addr learned my addr 172.21.15.26:0/3885000434 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:09.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3b9e7fc640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba81eb110 0x7f3ba81eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba77fe640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba814b2d0 0x7f3ba814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 -- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba81eb110 msgr2=0x7f3ba81eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba81eb110 0x7f3ba81eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 -- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba814b2d0 msgr2=0x7f3ba814b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba814b2d0 0x7f3ba814b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 -- 172.21.15.26:0/3885000434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba81eefb0 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3ba6ffd640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3ba814df30 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f3b98001eb0 tx=0x7f3b980077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3b9ffff640 1 -- 172.21.15.26:0/3885000434 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980017f0 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3b9ffff640 1 -- 172.21.15.26:0/3885000434 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b98018070 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba8157f60 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3b9ffff640 1 -- 172.21.15.26:0/3885000434 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800fc00 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.012+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba8158520 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.013+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba804f330 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.014+0000 7f3b9ffff640 1 -- 172.21.15.26:0/3885000434 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b9801d460 con 0x7f3b940043b0 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.014+0000 7f3b9ffff640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3b8806b4d0 0x7f3b8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.014+0000 7f3b9ffff640 1 -- 172.21.15.26:0/3885000434 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f3b9808a3f0 con 0x7f3b940043b0 2024-03-06T07:30:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.014+0000 7f3b9e7fc640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3b8806b4d0 0x7f3b8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.015+0000 7f3b9e7fc640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3b8806b4d0 0x7f3b8806d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3b90036000 tx=0x7f3b90031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:09.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.018+0000 7f3b9ffff640 1 -- 172.21.15.26:0/3885000434 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ba8155110 con 0x7f3b940043b0 2024-03-06T07:30:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.240+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba81ead30 con 0x7f3b940043b0 2024-03-06T07:30:09.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.240+0000 7f3b9ffff640 1 -- 172.21.15.26:0/3885000434 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f3b98058a70 con 0x7f3b940043b0 2024-03-06T07:30:09.243 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:09.243 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3b8806b4d0 msgr2=0x7f3b8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3b8806b4d0 0x7f3b8806d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3b90036000 tx=0x7f3b90031000 comp rx=0 tx=0).stop 2024-03-06T07:30:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 msgr2=0x7f3ba814df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3ba814df30 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f3b98001eb0 tx=0x7f3b980077a0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 shutdown_connections 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3b8806b4d0 0x7f3b8806d990 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba814b2d0 0x7f3ba814b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba81eb110 0x7f3ba81eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 --2- 172.21.15.26:0/3885000434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b940043b0 0x7f3ba814df30 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.243+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 >> 172.21.15.26:0/3885000434 conn(0x7f3ba80ac560 msgr2=0x7f3ba8154370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.244+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 shutdown_connections 2024-03-06T07:30:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:09.244+0000 7f3bad725640 1 -- 172.21.15.26:0/3885000434 wait complete. 2024-03-06T07:30:09.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:09.660 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:10.661 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:10.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.785+0000 7f332eedf640 1 Processor -- start 2024-03-06T07:30:10.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.785+0000 7f332eedf640 1 -- start start 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.786+0000 7f332eedf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 0x7f3328159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.786+0000 7f332eedf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33280c7c60 con 0x7f33280c88b0 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.786+0000 7f332eedf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33280c8060 con 0x7f33281527b0 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.786+0000 7f332eedf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3328159640 con 0x7f33280c7180 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.786+0000 7f332d47c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f33280c88b0 0x7f33280c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41932/0 (socket says 172.21.15.26:41932) 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.786+0000 7f332d47c640 1 -- 172.21.15.26:0/3911200466 learned_addr learned my addr 172.21.15.26:0/3911200466 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f3327fff640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 0x7f3328159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018869935 0 0) 0x7f3320001490 con 0x7f33280c88b0 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3318003520 con 0x7f33280c88b0 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378344282 0 0) 0x7f33280c8060 con 0x7f33281527b0 2024-03-06T07:30:10.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3320001490 con 0x7f33281527b0 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1093642399 0 0) 0x7f3318003520 con 0x7f33280c88b0 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 msgr2=0x7f3328159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 0x7f3328159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 >> v1:172.21.15.117:6789/0 conn(0x7f33281527b0 legacy=0x7f3328155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33280c76a0 con 0x7f33280c88b0 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f3320001bc0 con 0x7f33280c88b0 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 >> v1:172.21.15.26:6789/0 conn(0x7f33280c88b0 legacy=0x7f33280c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 0x7f3318003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3318004120 0x7f3318004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3318004d40 0x7f33180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3328159640 con 0x7f3318004d40 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331800bb80 con 0x7f3318004120 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331800bd80 con 0x7f33280c7180 2024-03-06T07:30:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f332d47c640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3318004d40 0x7f33180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f3327fff640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3318004120 0x7f3318004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f332cc7b640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 0x7f3318003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f332d47c640 1 -- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 msgr2=0x7f3318003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f332d47c640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33280c7180 0x7f3318003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f332d47c640 1 -- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3318004120 msgr2=0x7f3318004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.787+0000 7f332d47c640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3318004120 0x7f3318004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.788+0000 7f332d47c640 1 -- 172.21.15.26:0/3911200466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318003520 con 0x7f3318004d40 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.788+0000 7f332d47c640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3318004d40 0x7f33180051c0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f332000a910 tx=0x7f3318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa902629b136594b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.788+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000a530 con 0x7f3318004d40 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.788+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3320034650 con 0x7f3318004d40 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.788+0000 7f33277fe640 1 -- 172.21.15.26:0/3911200466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320034a80 con 0x7f3318004d40 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.789+0000 7f332eedf640 1 -- 172.21.15.26:0/3911200466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3318004d40 msgr2=0x7f33180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.789+0000 7f332eedf640 1 --2- 172.21.15.26:0/3911200466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3318004d40 0x7f33180051c0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f332000a910 tx=0x7f3318005830 comp rx=0 tx=0).stop 2024-03-06T07:30:10.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.789+0000 7f332d47c640 1 -- 172.21.15.26:0/3911200466 reap_dead start 2024-03-06T07:30:10.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.789+0000 7f332eedf640 1 -- 172.21.15.26:0/3911200466 shutdown_connections 2024-03-06T07:30:10.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.789+0000 7f332eedf640 1 -- 172.21.15.26:0/3911200466 >> 172.21.15.26:0/3911200466 conn(0x7f33280ac560 msgr2=0x7f33280ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:10.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.789+0000 7f332eedf640 1 -- 172.21.15.26:0/3911200466 shutdown_connections 2024-03-06T07:30:10.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.789+0000 7f332eedf640 1 -- 172.21.15.26:0/3911200466 wait complete. 2024-03-06T07:30:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.790+0000 7f332eedf640 1 Processor -- start 2024-03-06T07:30:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.790+0000 7f332eedf640 1 -- start start 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332eedf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328147f80 0x7f3328153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332eedf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3328154370 0x7f33281576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332eedf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3328148360 0x7f3328157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332eedf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3328155600 con 0x7f3328148360 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332eedf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3328155320 con 0x7f3328147f80 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332eedf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33280c8d60 con 0x7f3328154370 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3328148360 0x7f3328157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3328148360 0x7f3328157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56596/0 (socket says 172.21.15.26:56596) 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 -- 172.21.15.26:0/4229760164 learned_addr learned my addr 172.21.15.26:0/4229760164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332cc7b640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328147f80 0x7f3328153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f3327fff640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3328154370 0x7f33281576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 -- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3328154370 msgr2=0x7f33281576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3328154370 0x7f33281576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 -- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328147f80 msgr2=0x7f3328153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328147f80 0x7f3328153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.791+0000 7f332d47c640 1 -- 172.21.15.26:0/4229760164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328152c30 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.792+0000 7f332d47c640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3328148360 0x7f3328157bf0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f332000aa30 tx=0x7f3320001ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.792+0000 7f33257fa640 1 -- 172.21.15.26:0/4229760164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320045070 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.792+0000 7f33257fa640 1 -- 172.21.15.26:0/4229760164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332003a040 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.792+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3328152f10 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.792+0000 7f33257fa640 1 -- 172.21.15.26:0/4229760164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000a530 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.792+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33281534d0 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.793+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33280ace60 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.793+0000 7f33257fa640 1 -- 172.21.15.26:0/4229760164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3320048070 con 0x7f3328148360 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.794+0000 7f33257fa640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f330c06b500 0x7f330c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.794+0000 7f33257fa640 1 -- 172.21.15.26:0/4229760164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f33200abe40 con 0x7f3328148360 2024-03-06T07:30:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.794+0000 7f332cc7b640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f330c06b500 0x7f330c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.795+0000 7f332cc7b640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f330c06b500 0x7f330c06d9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f331c000c00 tx=0x7f331c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:10.798+0000 7f33257fa640 1 -- 172.21.15.26:0/4229760164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33280ace60 con 0x7f3328148360 2024-03-06T07:30:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.021+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f332804f330 con 0x7f3328148360 2024-03-06T07:30:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.022+0000 7f33257fa640 1 -- 172.21.15.26:0/4229760164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f33280c7c60 con 0x7f3328148360 2024-03-06T07:30:11.024 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:11.024 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:11.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f330c06b500 msgr2=0x7f330c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f330c06b500 0x7f330c06d9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f331c000c00 tx=0x7f331c009040 comp rx=0 tx=0).stop 2024-03-06T07:30:11.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3328148360 msgr2=0x7f3328157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3328148360 0x7f3328157bf0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f332000aa30 tx=0x7f3320001ec0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 shutdown_connections 2024-03-06T07:30:11.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f330c06b500 0x7f330c06d9c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3328148360 0x7f3328157bf0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3328154370 0x7f33281576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 --2- 172.21.15.26:0/4229760164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328147f80 0x7f3328153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 >> 172.21.15.26:0/4229760164 conn(0x7f33280ac560 msgr2=0x7f3328155b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:11.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 shutdown_connections 2024-03-06T07:30:11.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:11.024+0000 7f332eedf640 1 -- 172.21.15.26:0/4229760164 wait complete. 2024-03-06T07:30:11.442 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:11.442 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:12.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:12.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.543+0000 7ff3823e9640 1 Processor -- start 2024-03-06T07:30:12.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.543+0000 7ff3823e9640 1 -- start start 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff3823e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff37c14f790 0x7ff37c1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff3823e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37c0c28d0 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff3823e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37c0c2bb0 con 0x7ff37c155a40 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff3823e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff37c0c2680 con 0x7ff37c14f790 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff37bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff37c14f790 0x7ff37c1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff37b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff37c0c1aa0 0x7ff37c0c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41936/0 (socket says 172.21.15.26:41936) 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff37b7fe640 1 -- 172.21.15.26:0/4190509477 learned_addr learned my addr 172.21.15.26:0/4190509477 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff37bfff640 1 -- 172.21.15.26:0/4190509477 >> v1:172.21.15.117:6789/0 conn(0x7ff37c155a40 legacy=0x7ff37c155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:12.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff37bfff640 1 -- 172.21.15.26:0/4190509477 >> v1:172.21.15.26:6789/0 conn(0x7ff37c0c1aa0 legacy=0x7ff37c0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:12.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff37affd640 1 -- 172.21.15.26:0/4190509477 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201191057 0 0) 0x7ff37c0c28d0 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.544+0000 7ff37bfff640 1 -- 172.21.15.26:0/4190509477 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c0c2e90 con 0x7ff37c14f790 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.545+0000 7ff37bfff640 1 --2- 172.21.15.26:0/4190509477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c14f790 0x7ff37c1529d0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c2800 tx=0x7ff36c02e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f62f81d37999c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.545+0000 7ff37affd640 1 -- 172.21.15.26:0/4190509477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c03b070 con 0x7ff37c14f790 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.545+0000 7ff37affd640 1 -- 172.21.15.26:0/4190509477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c02ee30 con 0x7ff37c14f790 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.545+0000 7ff37affd640 1 -- 172.21.15.26:0/4190509477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c036640 con 0x7ff37c14f790 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.546+0000 7ff3823e9640 1 -- 172.21.15.26:0/4190509477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c14f790 msgr2=0x7ff37c1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.546+0000 7ff3823e9640 1 --2- 172.21.15.26:0/4190509477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c14f790 0x7ff37c1529d0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c2800 tx=0x7ff36c02e080 comp rx=0 tx=0).stop 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.546+0000 7ff3823e9640 1 -- 172.21.15.26:0/4190509477 shutdown_connections 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.546+0000 7ff3823e9640 1 --2- 172.21.15.26:0/4190509477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c14f790 0x7ff37c1529d0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:12.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.546+0000 7ff3823e9640 1 -- 172.21.15.26:0/4190509477 >> 172.21.15.26:0/4190509477 conn(0x7ff37c0c6710 msgr2=0x7ff37c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.546+0000 7ff3823e9640 1 -- 172.21.15.26:0/4190509477 shutdown_connections 2024-03-06T07:30:12.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.546+0000 7ff3823e9640 1 -- 172.21.15.26:0/4190509477 wait complete. 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.547+0000 7ff3823e9640 1 Processor -- start 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.547+0000 7ff3823e9640 1 -- start start 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff3823e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c0c1aa0 0x7ff37c14c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff3823e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37c14f790 0x7ff37c153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff3823e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff37c155a40 0x7ff37c14a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff3823e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0c5e30 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff3823e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0c0d60 con 0x7ff37c155a40 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff3823e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0c5fb0 con 0x7ff37c14f790 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c0c1aa0 0x7ff37c14c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:12.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c0c1aa0 0x7ff37c14c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56610/0 (socket says 172.21.15.26:56610) 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 -- 172.21.15.26:0/1041706968 learned_addr learned my addr 172.21.15.26:0/1041706968 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37b7fe640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37c14f790 0x7ff37c153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff380986640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff37c155a40 0x7ff37c14a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 -- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37c14f790 msgr2=0x7ff37c153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37c14f790 0x7ff37c153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 -- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff37c155a40 msgr2=0x7ff37c14a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff37c155a40 0x7ff37c14a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff37bfff640 1 -- 172.21.15.26:0/1041706968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c154eb0 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.548+0000 7ff380986640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff37c155a40 0x7ff37c14a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.549+0000 7ff37bfff640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c0c1aa0 0x7ff37c14c310 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00bb30 tx=0x7ff36c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.549+0000 7ff378ff9640 1 -- 172.21.15.26:0/1041706968 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c03b040 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.549+0000 7ff378ff9640 1 -- 172.21.15.26:0/1041706968 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c004230 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.549+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37c155190 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.549+0000 7ff378ff9640 1 -- 172.21.15.26:0/1041706968 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c003610 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.549+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37c14c850 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.550+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37c14b800 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.550+0000 7ff378ff9640 1 -- 172.21.15.26:0/1041706968 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff36c0037b0 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.551+0000 7ff378ff9640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff36806b4d0 0x7ff36806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.551+0000 7ff378ff9640 1 -- 172.21.15.26:0/1041706968 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7ff36c0ad330 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.551+0000 7ff37b7fe640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff36806b4d0 0x7ff36806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:12.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.551+0000 7ff37b7fe640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff36806b4d0 0x7ff36806d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff370007fd0 tx=0x7ff370009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:12.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.559+0000 7ff378ff9640 1 -- 172.21.15.26:0/1041706968 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff36c07c120 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.810+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff37c04f330 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.811+0000 7ff378ff9640 1 -- 172.21.15.26:0/1041706968 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7ff36c07ba20 con 0x7ff37c0c1aa0 2024-03-06T07:30:12.814 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:12.814 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:12.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff36806b4d0 msgr2=0x7ff36806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff36806b4d0 0x7ff36806d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff370007fd0 tx=0x7ff370009040 comp rx=0 tx=0).stop 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c0c1aa0 msgr2=0x7ff37c14c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c0c1aa0 0x7ff37c14c310 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00bb30 tx=0x7ff36c004730 comp rx=0 tx=0).stop 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 shutdown_connections 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff36806b4d0 0x7ff36806d990 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff37c155a40 0x7ff37c14a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37c14f790 0x7ff37c153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 --2- 172.21.15.26:0/1041706968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff37c0c1aa0 0x7ff37c14c310 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 >> 172.21.15.26:0/1041706968 conn(0x7ff37c0c6710 msgr2=0x7ff37c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 shutdown_connections 2024-03-06T07:30:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:12.813+0000 7ff3823e9640 1 -- 172.21.15.26:0/1041706968 wait complete. 2024-03-06T07:30:13.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:13.228 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:13.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:13.246 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:14.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.337+0000 7fab64a47640 1 Processor -- start 2024-03-06T07:30:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.337+0000 7fab64a47640 1 -- start start 2024-03-06T07:30:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.338+0000 7fab64a47640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab60072720 0x7fab601564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.338+0000 7fab64a47640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab60156f90 con 0x7fab600c3d60 2024-03-06T07:30:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.338+0000 7fab64a47640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab60157270 con 0x7fab60156a00 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.338+0000 7fab64a47640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600c38d0 con 0x7fab60072720 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.338+0000 7fab5ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fab600c3d60 0x7fab600c41b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41948/0 (socket says 172.21.15.26:41948) 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.338+0000 7fab5ed9d640 1 -- 172.21.15.26:0/2070150662 learned_addr learned my addr 172.21.15.26:0/2070150662 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.338+0000 7fab5e59c640 1 --2- 172.21.15.26:0/2070150662 >> v2:172.21.15.117:3300/0 conn(0x7fab60072720 0x7fab601564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573737752 0 0) 0x7fab60156f90 con 0x7fab600c3d60 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab4c003520 con 0x7fab600c3d60 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710242893 0 0) 0x7fab60157270 con 0x7fab60156a00 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab60156f90 con 0x7fab60156a00 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 986928185 0 0) 0x7fab4c003520 con 0x7fab600c3d60 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5e59c640 1 -- 172.21.15.26:0/2070150662 >> v1:172.21.15.117:6789/0 conn(0x7fab60156a00 legacy=0x7fab60150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5e59c640 1 -- 172.21.15.26:0/2070150662 >> v1:172.21.15.26:6789/0 conn(0x7fab600c3d60 legacy=0x7fab600c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.339+0000 7fab5e59c640 1 -- 172.21.15.26:0/2070150662 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600c3280 con 0x7fab60072720 2024-03-06T07:30:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab5e59c640 1 --2- 172.21.15.26:0/2070150662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab60072720 0x7fab601564c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fab580097a0 tx=0x7fab5802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc7874581e5db999 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58037070 con 0x7fab60072720 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58030510 con 0x7fab60072720 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab5d59a640 1 -- 172.21.15.26:0/2070150662 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500019f0 con 0x7fab60072720 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab64a47640 1 -- 172.21.15.26:0/2070150662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab60072720 msgr2=0x7fab601564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab64a47640 1 --2- 172.21.15.26:0/2070150662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab60072720 0x7fab601564c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fab580097a0 tx=0x7fab5802b790 comp rx=0 tx=0).stop 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab64a47640 1 -- 172.21.15.26:0/2070150662 shutdown_connections 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab64a47640 1 --2- 172.21.15.26:0/2070150662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab60072720 0x7fab601564c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.340+0000 7fab64a47640 1 -- 172.21.15.26:0/2070150662 >> 172.21.15.26:0/2070150662 conn(0x7fab600aa630 msgr2=0x7fab600aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.341+0000 7fab64a47640 1 -- 172.21.15.26:0/2070150662 shutdown_connections 2024-03-06T07:30:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.341+0000 7fab64a47640 1 -- 172.21.15.26:0/2070150662 wait complete. 2024-03-06T07:30:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 Processor -- start 2024-03-06T07:30:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 -- start start 2024-03-06T07:30:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab60072720 0x7fab6014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab600c3d60 0x7fab600c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab60156a00 0x7fab601ec060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab601509a0 con 0x7fab60156a00 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6006f8f0 con 0x7fab600c3d60 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.342+0000 7fab64a47640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab60150b20 con 0x7fab60072720 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab60156a00 0x7fab601ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab60156a00 0x7fab601ec060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56614/0 (socket says 172.21.15.26:56614) 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 -- 172.21.15.26:0/1055150236 learned_addr learned my addr 172.21.15.26:0/1055150236 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5dd9b640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab600c3d60 0x7fab600c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5e59c640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab60072720 0x7fab6014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 -- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab60072720 msgr2=0x7fab6014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab60072720 0x7fab6014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 -- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab600c3d60 msgr2=0x7fab600c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab600c3d60 0x7fab600c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 -- 172.21.15.26:0/1055150236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600acd50 con 0x7fab60156a00 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5e59c640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab60072720 0x7fab6014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab5ed9d640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab60156a00 0x7fab601ec060 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fab5000cc40 tx=0x7fab5000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab56ffd640 1 -- 172.21.15.26:0/1055150236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000ce50 con 0x7fab60156a00 2024-03-06T07:30:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab56ffd640 1 -- 172.21.15.26:0/1055150236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50012400 con 0x7fab60156a00 2024-03-06T07:30:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab600ad030 con 0x7fab60156a00 2024-03-06T07:30:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab56ffd640 1 -- 172.21.15.26:0/1055150236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500127f0 con 0x7fab60156a00 2024-03-06T07:30:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.343+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab600ad5f0 con 0x7fab60156a00 2024-03-06T07:30:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.344+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6004f330 con 0x7fab60156a00 2024-03-06T07:30:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.345+0000 7fab56ffd640 1 -- 172.21.15.26:0/1055150236 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab50012990 con 0x7fab60156a00 2024-03-06T07:30:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.345+0000 7fab56ffd640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab4006b670 0x7fab4006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.346+0000 7fab56ffd640 1 -- 172.21.15.26:0/1055150236 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fab5008b6e0 con 0x7fab60156a00 2024-03-06T07:30:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.346+0000 7fab5e59c640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab4006b670 0x7fab4006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.346+0000 7fab5e59c640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab4006b670 0x7fab4006db30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fab58034000 tx=0x7fab5802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.349+0000 7fab56ffd640 1 -- 172.21.15.26:0/1055150236 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab5005a290 con 0x7fab60156a00 2024-03-06T07:30:14.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.566+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab6014cd90 con 0x7fab60156a00 2024-03-06T07:30:14.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.567+0000 7fab56ffd640 1 -- 172.21.15.26:0/1055150236 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fab50059b90 con 0x7fab60156a00 2024-03-06T07:30:14.569 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:14.569 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab4006b670 msgr2=0x7fab4006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab4006b670 0x7fab4006db30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fab58034000 tx=0x7fab5802f000 comp rx=0 tx=0).stop 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab60156a00 msgr2=0x7fab601ec060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab60156a00 0x7fab601ec060 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fab5000cc40 tx=0x7fab5000a430 comp rx=0 tx=0).stop 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 shutdown_connections 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab4006b670 0x7fab4006db30 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab60156a00 0x7fab601ec060 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab600c3d60 0x7fab600c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 --2- 172.21.15.26:0/1055150236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab60072720 0x7fab6014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 >> 172.21.15.26:0/1055150236 conn(0x7fab600aa630 msgr2=0x7fab600c5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 shutdown_connections 2024-03-06T07:30:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:14.569+0000 7fab64a47640 1 -- 172.21.15.26:0/1055150236 wait complete. 2024-03-06T07:30:14.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:14.985 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:15.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:16.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.085+0000 7f5deb3f6640 1 Processor -- start 2024-03-06T07:30:16.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.085+0000 7f5deb3f6640 1 -- start start 2024-03-06T07:30:16.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5deb3f6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5de40c7510 0x7f5de4155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5deb3f6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de40c8fc0 con 0x7f5de41560f0 2024-03-06T07:30:16.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5deb3f6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de4072720 con 0x7f5de40c6ce0 2024-03-06T07:30:16.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5deb3f6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de4072d70 con 0x7f5de40c7510 2024-03-06T07:30:16.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5de9993640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5de41560f0 0x7f5de4156560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41962/0 (socket says 172.21.15.26:41962) 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5de9993640 1 -- 172.21.15.26:0/844708375 learned_addr learned my addr 172.21.15.26:0/844708375 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5de8991640 1 --2- 172.21.15.26:0/844708375 >> v2:172.21.15.117:3300/0 conn(0x7f5de40c7510 0x7f5de4155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984825343 0 0) 0x7f5de40c8fc0 con 0x7f5de41560f0 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.086+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd0003520 con 0x7f5de41560f0 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3504643922 0 0) 0x7f5dd0003520 con 0x7f5de41560f0 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 >> v2:172.21.15.117:3300/0 conn(0x7f5de40c7510 msgr2=0x7f5de4155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 --2- 172.21.15.26:0/844708375 >> v2:172.21.15.117:3300/0 conn(0x7f5de40c7510 0x7f5de4155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 >> v1:172.21.15.117:6789/0 conn(0x7f5de40c6ce0 legacy=0x7f5de40c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de4072a00 con 0x7f5de41560f0 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f5de00019f0 con 0x7f5de41560f0 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 >> v1:172.21.15.26:6789/0 conn(0x7f5de41560f0 legacy=0x7f5de4156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd00039b0 0x7f5dd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd00051c0 0x7f5dd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4072d70 con 0x7f5dd0004800 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd000c060 con 0x7f5dd00039b0 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd000c260 con 0x7f5dd00051c0 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de8991640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de9192640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd00039b0 0x7f5dd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de9993640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd00051c0 0x7f5dd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de8991640 1 -- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd00051c0 msgr2=0x7f5dd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de8991640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd00051c0 0x7f5dd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de8991640 1 -- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd00039b0 msgr2=0x7f5dd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de8991640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd00039b0 0x7f5dd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de8991640 1 -- 172.21.15.26:0/844708375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0003520 con 0x7f5dd0004800 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.087+0000 7f5de9993640 1 -- 172.21.15.26:0/844708375 reap_dead start 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.088+0000 7f5de8991640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009a90 tx=0x7f5dd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6fc199b2612af5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.088+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4039070 con 0x7f5dd0004800 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.088+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd402c680 con 0x7f5dd0004800 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.088+0000 7f5ddbfff640 1 -- 172.21.15.26:0/844708375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4009690 con 0x7f5dd0004800 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.089+0000 7f5deb3f6640 1 -- 172.21.15.26:0/844708375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dd0004800 msgr2=0x7f5dd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.089+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009a90 tx=0x7f5dd0005830 comp rx=0 tx=0).stop 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.089+0000 7f5deb3f6640 1 -- 172.21.15.26:0/844708375 shutdown_connections 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.089+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/844708375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.089+0000 7f5deb3f6640 1 -- 172.21.15.26:0/844708375 >> 172.21.15.26:0/844708375 conn(0x7f5de40aa4e0 msgr2=0x7f5de40aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.089+0000 7f5deb3f6640 1 -- 172.21.15.26:0/844708375 shutdown_connections 2024-03-06T07:30:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.089+0000 7f5deb3f6640 1 -- 172.21.15.26:0/844708375 wait complete. 2024-03-06T07:30:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.090+0000 7f5deb3f6640 1 Processor -- start 2024-03-06T07:30:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.090+0000 7f5deb3f6640 1 -- start start 2024-03-06T07:30:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5deb3f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd0004800 0x7f5de40c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5deb3f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de406de90 0x7f5de40a7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5deb3f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de40a7670 0x7f5de406c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5deb3f6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40c7240 con 0x7f5de406de90 2024-03-06T07:30:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5deb3f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40c70c0 con 0x7f5dd0004800 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5deb3f6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4155bb0 con 0x7f5de40a7670 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de406de90 0x7f5de40a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de406de90 0x7f5de40a7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56632/0 (socket says 172.21.15.26:56632) 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 -- 172.21.15.26:0/685232017 learned_addr learned my addr 172.21.15.26:0/685232017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de9192640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd0004800 0x7f5de40c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de9993640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de40a7670 0x7f5de406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 -- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de40a7670 msgr2=0x7f5de406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de40a7670 0x7f5de406c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 -- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd0004800 msgr2=0x7f5de40c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd0004800 0x7f5de40c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 -- 172.21.15.26:0/685232017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40ac4b0 con 0x7f5de406de90 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de9192640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd0004800 0x7f5de40c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de9993640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de40a7670 0x7f5de406c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5de8991640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de406de90 0x7f5de40a7130 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009bb0 tx=0x7f5dd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5dd9ffb640 1 -- 172.21.15.26:0/685232017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4042020 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5dd9ffb640 1 -- 172.21.15.26:0/685232017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd403a040 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de40ac790 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.091+0000 7f5dd9ffb640 1 -- 172.21.15.26:0/685232017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4039070 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.092+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de40c3fe0 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.093+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de40c5a50 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.093+0000 7f5dd9ffb640 1 -- 172.21.15.26:0/685232017 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dd4045020 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.093+0000 7f5dd9ffb640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc806b4d0 0x7f5dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.094+0000 7f5dd9ffb640 1 -- 172.21.15.26:0/685232017 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f5dd40abcc0 con 0x7f5de406de90 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.094+0000 7f5de9192640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc806b4d0 0x7f5dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.094+0000 7f5de9192640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc806b4d0 0x7f5dc806d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc005600 tx=0x7f5ddc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.102+0000 7f5dd9ffb640 1 -- 172.21.15.26:0/685232017 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5de40c5a50 con 0x7f5de406de90 2024-03-06T07:30:16.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.320+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5de404f330 con 0x7f5de406de90 2024-03-06T07:30:16.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.320+0000 7f5dd9ffb640 1 -- 172.21.15.26:0/685232017 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f5dd40b2080 con 0x7f5de406de90 2024-03-06T07:30:16.323 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:16.323 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc806b4d0 msgr2=0x7f5dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc806b4d0 0x7f5dc806d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc005600 tx=0x7f5ddc009040 comp rx=0 tx=0).stop 2024-03-06T07:30:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de406de90 msgr2=0x7f5de40a7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de406de90 0x7f5de40a7130 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009bb0 tx=0x7f5dd4004730 comp rx=0 tx=0).stop 2024-03-06T07:30:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 shutdown_connections 2024-03-06T07:30:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5dc806b4d0 0x7f5dc806d990 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de40a7670 0x7f5de406c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de406de90 0x7f5de40a7130 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 --2- 172.21.15.26:0/685232017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd0004800 0x7f5de40c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 >> 172.21.15.26:0/685232017 conn(0x7f5de40aa4e0 msgr2=0x7f5de40c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 shutdown_connections 2024-03-06T07:30:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:16.322+0000 7f5deb3f6640 1 -- 172.21.15.26:0/685232017 wait complete. 2024-03-06T07:30:16.738 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:16.738 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:17.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:17.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.840+0000 7fbc8e22a640 1 Processor -- start 2024-03-06T07:30:17.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.840+0000 7fbc8e22a640 1 -- start start 2024-03-06T07:30:17.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc8e22a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbc8814f790 0x7fbc881529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:17.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc8e22a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbc88155a40 0x7fbc88155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc8e22a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880c28d0 con 0x7fbc880c1aa0 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc8e22a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc880c2680 con 0x7fbc8814f790 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc8e22a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc880c2bb0 con 0x7fbc88155a40 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbc8814f790 0x7fbc881529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbc8814f790 0x7fbc881529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57992/0 (socket says 172.21.15.26:57992) 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc877fe640 1 -- 172.21.15.26:0/3461245235 learned_addr learned my addr 172.21.15.26:0/3461245235 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc877fe640 1 -- 172.21.15.26:0/3461245235 >> v2:172.21.15.117:3300/0 conn(0x7fbc88155a40 msgr2=0x7fbc88155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc877fe640 1 --2- 172.21.15.26:0/3461245235 >> v2:172.21.15.117:3300/0 conn(0x7fbc88155a40 0x7fbc88155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc877fe640 1 -- 172.21.15.26:0/3461245235 >> v1:172.21.15.117:6789/0 conn(0x7fbc880c1aa0 legacy=0x7fbc880c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.841+0000 7fbc877fe640 1 -- 172.21.15.26:0/3461245235 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880c2e90 con 0x7fbc8814f790 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.842+0000 7fbc877fe640 1 --2- 172.21.15.26:0/3461245235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881529d0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806c960 tx=0x7fbc7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7405ac9ef0fb0237 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.842+0000 7fbc867fc640 1 -- 172.21.15.26:0/3461245235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70035020 con 0x7fbc8814f790 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.842+0000 7fbc867fc640 1 -- 172.21.15.26:0/3461245235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc70031870 con 0x7fbc8814f790 2024-03-06T07:30:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.842+0000 7fbc867fc640 1 -- 172.21.15.26:0/3461245235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70031c40 con 0x7fbc8814f790 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 -- 172.21.15.26:0/3461245235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 msgr2=0x7fbc881529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/3461245235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881529d0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806c960 tx=0x7fbc7002b910 comp rx=0 tx=0).stop 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 -- 172.21.15.26:0/3461245235 shutdown_connections 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/3461245235 >> v2:172.21.15.117:3300/0 conn(0x7fbc88155a40 0x7fbc88155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/3461245235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881529d0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 -- 172.21.15.26:0/3461245235 >> 172.21.15.26:0/3461245235 conn(0x7fbc880c6710 msgr2=0x7fbc880c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 -- 172.21.15.26:0/3461245235 shutdown_connections 2024-03-06T07:30:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.843+0000 7fbc8e22a640 1 -- 172.21.15.26:0/3461245235 wait complete. 2024-03-06T07:30:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.844+0000 7fbc8e22a640 1 Processor -- start 2024-03-06T07:30:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.844+0000 7fbc8e22a640 1 -- start start 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc8e22a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc880c1aa0 0x7fbc881dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc8e22a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881ead60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc8e22a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc88155a40 0x7fbc88072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc8e22a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880c0d60 con 0x7fbc8814f790 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc8e22a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88156390 con 0x7fbc88155a40 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc8e22a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880c5e30 con 0x7fbc880c1aa0 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881ead60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881ead60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57998/0 (socket says 172.21.15.26:57998) 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 -- 172.21.15.26:0/2117113048 learned_addr learned my addr 172.21.15.26:0/2117113048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc877fe640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc880c1aa0 0x7fbc881dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc87fff640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc88155a40 0x7fbc88072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 -- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc880c1aa0 msgr2=0x7fbc881dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc880c1aa0 0x7fbc881dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 -- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc88155a40 msgr2=0x7fbc88072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc88155a40 0x7fbc88072b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.845+0000 7fbc86ffd640 1 -- 172.21.15.26:0/2117113048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc881eec90 con 0x7fbc8814f790 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.846+0000 7fbc86ffd640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881ead60 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c430 tx=0x7fbc7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.846+0000 7fbc7ffff640 1 -- 172.21.15.26:0/2117113048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78011790 con 0x7fbc8814f790 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.846+0000 7fbc7ffff640 1 -- 172.21.15.26:0/2117113048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78017070 con 0x7fbc8814f790 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.846+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc881eef30 con 0x7fbc8814f790 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.846+0000 7fbc7ffff640 1 -- 172.21.15.26:0/2117113048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78011b70 con 0x7fbc8814f790 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.846+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc880a6740 con 0x7fbc8814f790 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.847+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8804f330 con 0x7fbc8814f790 2024-03-06T07:30:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.847+0000 7fbc7ffff640 1 -- 172.21.15.26:0/2117113048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc78011d10 con 0x7fbc8814f790 2024-03-06T07:30:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.847+0000 7fbc7ffff640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6806b500 0x7fbc6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.848+0000 7fbc7ffff640 1 -- 172.21.15.26:0/2117113048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fbc7808a0a0 con 0x7fbc8814f790 2024-03-06T07:30:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.848+0000 7fbc877fe640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6806b500 0x7fbc6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.848+0000 7fbc877fe640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6806b500 0x7fbc6806d9c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbc70036000 tx=0x7fbc7003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:17.852+0000 7fbc7ffff640 1 -- 172.21.15.26:0/2117113048 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbc78058720 con 0x7fbc8814f790 2024-03-06T07:30:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.073+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc8814f300 con 0x7fbc8814f790 2024-03-06T07:30:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.074+0000 7fbc7ffff640 1 -- 172.21.15.26:0/2117113048 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fbc7805c7c0 con 0x7fbc8814f790 2024-03-06T07:30:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:18.075 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6806b500 msgr2=0x7fbc6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6806b500 0x7fbc6806d9c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbc70036000 tx=0x7fbc7003c040 comp rx=0 tx=0).stop 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 msgr2=0x7fbc881ead60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881ead60 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c430 tx=0x7fbc7800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 shutdown_connections 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbc6806b500 0x7fbc6806d9c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc88155a40 0x7fbc88072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8814f790 0x7fbc881ead60 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 --2- 172.21.15.26:0/2117113048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc880c1aa0 0x7fbc881dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 >> 172.21.15.26:0/2117113048 conn(0x7fbc880c6710 msgr2=0x7fbc8806dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 shutdown_connections 2024-03-06T07:30:18.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:18.076+0000 7fbc8e22a640 1 -- 172.21.15.26:0/2117113048 wait complete. 2024-03-06T07:30:18.491 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:18.492 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:19.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.593+0000 7fa2c8155640 1 Processor -- start 2024-03-06T07:30:19.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.593+0000 7fa2c8155640 1 -- start start 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.593+0000 7fa2c8155640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 0x7fa2c00a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.593+0000 7fa2c8155640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa2c00c3280 0x7fa2c00c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.593+0000 7fa2c8155640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c0158800 con 0x7fa2c01551e0 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.593+0000 7fa2c8155640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c0158ae0 con 0x7fa2c00c3280 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.593+0000 7fa2c8155640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0158dc0 con 0x7fa2c006d9f0 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c66f2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa2c01551e0 0x7fa2c0155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34084/0 (socket says 172.21.15.26:34084) 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c66f2640 1 -- 172.21.15.26:0/3889748369 learned_addr learned my addr 172.21.15.26:0/3889748369 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c56f0640 1 --2- 172.21.15.26:0/3889748369 >> v2:172.21.15.117:3300/0 conn(0x7fa2c00c3280 0x7fa2c00c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 0x7fa2c00a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046121867 0 0) 0x7fa2c0158800 con 0x7fa2c01551e0 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003520 con 0x7fa2c01551e0 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 777800066 0 0) 0x7fa2ac003520 con 0x7fa2c01551e0 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 msgr2=0x7fa2c00a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 0x7fa2c00a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 >> v2:172.21.15.117:3300/0 conn(0x7fa2c00c3280 msgr2=0x7fa2c00c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 --2- 172.21.15.26:0/3889748369 >> v2:172.21.15.117:3300/0 conn(0x7fa2c00c3280 0x7fa2c00c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c01590a0 con 0x7fa2c01551e0 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fa2b80019f0 con 0x7fa2c01551e0 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 0x7fa2c00a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 >> v1:172.21.15.26:6789/0 conn(0x7fa2c01551e0 legacy=0x7fa2c0155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 0x7fa2ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2ac004d40 0x7fa2ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0158ae0 con 0x7fa2ac0039b0 2024-03-06T07:30:19.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0158dc0 con 0x7fa2ac004d40 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac00b9c0 con 0x7fa2c006d9f0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c56f0640 1 --2- 172.21.15.26:0/3889748369 >> v2:172.21.15.117:3300/0 conn(0x7fa2c00c3280 0x7fa2c00c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.594+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c66f2640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2ac004d40 0x7fa2ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c56f0640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 0x7fa2ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c5ef1640 1 -- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 msgr2=0x7fa2ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c006d9f0 0x7fa2ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c5ef1640 1 -- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2ac004d40 msgr2=0x7fa2ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2ac004d40 0x7fa2ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c5ef1640 1 -- 172.21.15.26:0/3889748369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac003520 con 0x7fa2ac0039b0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.595+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2ac003dd0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009c90 tx=0x7fa2ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d7802cc4e58b21 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc039070 con 0x7fa2ac0039b0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc02c8f0 con 0x7fa2ac0039b0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c4eef640 1 -- 172.21.15.26:0/3889748369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc009860 con 0x7fa2ac0039b0 2024-03-06T07:30:19.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c8155640 1 -- 172.21.15.26:0/3889748369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 msgr2=0x7fa2ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:19.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c8155640 1 --2- 172.21.15.26:0/3889748369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2ac003dd0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009c90 tx=0x7fa2ac005830 comp rx=0 tx=0).stop 2024-03-06T07:30:19.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c66f2640 1 -- 172.21.15.26:0/3889748369 reap_dead start 2024-03-06T07:30:19.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c8155640 1 -- 172.21.15.26:0/3889748369 shutdown_connections 2024-03-06T07:30:19.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c8155640 1 -- 172.21.15.26:0/3889748369 >> 172.21.15.26:0/3889748369 conn(0x7fa2c00aa630 msgr2=0x7fa2c00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:19.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c8155640 1 -- 172.21.15.26:0/3889748369 shutdown_connections 2024-03-06T07:30:19.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.596+0000 7fa2c8155640 1 -- 172.21.15.26:0/3889748369 wait complete. 2024-03-06T07:30:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.597+0000 7fa2c8155640 1 Processor -- start 2024-03-06T07:30:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c8155640 1 -- start start 2024-03-06T07:30:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c8155640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2c0140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c8155640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c014f520 0x7fa2c0140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c8155640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c01ee490 0x7fa2c01ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c8155640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c006df50 con 0x7fa2ac0039b0 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c8155640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c006ddd0 con 0x7fa2c01ee490 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c8155640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0155d20 con 0x7fa2c014f520 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c5ef1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2c0140a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c5ef1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2c0140a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58030/0 (socket says 172.21.15.26:58030) 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c5ef1640 1 -- 172.21.15.26:0/2800729820 learned_addr learned my addr 172.21.15.26:0/2800729820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c56f0640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c014f520 0x7fa2c0140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c5ef1640 1 -- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c014f520 msgr2=0x7fa2c0140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c014f520 0x7fa2c0140f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c5ef1640 1 -- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c01ee490 msgr2=0x7fa2c01ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.598+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c01ee490 0x7fa2c01ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.599+0000 7fa2c5ef1640 1 -- 172.21.15.26:0/2800729820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00ab7c0 con 0x7fa2ac0039b0 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.599+0000 7fa2c5ef1640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2c0140a20 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009c90 tx=0x7fa2bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.599+0000 7fa2b67fc640 1 -- 172.21.15.26:0/2800729820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc039050 con 0x7fa2ac0039b0 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.599+0000 7fa2b67fc640 1 -- 172.21.15.26:0/2800729820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc045070 con 0x7fa2ac0039b0 2024-03-06T07:30:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.599+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c00abaa0 con 0x7fa2ac0039b0 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.599+0000 7fa2b67fc640 1 -- 172.21.15.26:0/2800729820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc009860 con 0x7fa2ac0039b0 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.599+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c0070280 con 0x7fa2ac0039b0 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.600+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c00abc70 con 0x7fa2ac0039b0 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.601+0000 7fa2b67fc640 1 -- 172.21.15.26:0/2800729820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2bc04a460 con 0x7fa2ac0039b0 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.601+0000 7fa2b67fc640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa2a406b4d0 0x7fa2a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.601+0000 7fa2b67fc640 1 -- 172.21.15.26:0/2800729820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fa2bc0ac6a0 con 0x7fa2ac0039b0 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.602+0000 7fa2c56f0640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa2a406b4d0 0x7fa2a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.602+0000 7fa2c56f0640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa2a406b4d0 0x7fa2a406d990 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa2c014fab0 tx=0x7fa2b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:19.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.606+0000 7fa2b67fc640 1 -- 172.21.15.26:0/2800729820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa2c00abc70 con 0x7fa2ac0039b0 2024-03-06T07:30:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.834+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2c00acff0 con 0x7fa2ac0039b0 2024-03-06T07:30:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.835+0000 7fa2b67fc640 1 -- 172.21.15.26:0/2800729820 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fa2bc07b370 con 0x7fa2ac0039b0 2024-03-06T07:30:19.837 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:19.837 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:19.837 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa2a406b4d0 msgr2=0x7fa2a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa2a406b4d0 0x7fa2a406d990 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa2c014fab0 tx=0x7fa2b0008040 comp rx=0 tx=0).stop 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 msgr2=0x7fa2c0140a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2c0140a20 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009c90 tx=0x7fa2bc004730 comp rx=0 tx=0).stop 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 shutdown_connections 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa2a406b4d0 0x7fa2a406d990 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c01ee490 0x7fa2c01ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2c014f520 0x7fa2c0140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 --2- 172.21.15.26:0/2800729820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ac0039b0 0x7fa2c0140a20 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 >> 172.21.15.26:0/2800729820 conn(0x7fa2c00aa630 msgr2=0x7fa2c01555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 shutdown_connections 2024-03-06T07:30:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:19.837+0000 7fa2c8155640 1 -- 172.21.15.26:0/2800729820 wait complete. 2024-03-06T07:30:20.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:20.252 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:21.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.377+0000 7f2ded478640 1 Processor -- start 2024-03-06T07:30:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.377+0000 7f2ded478640 1 -- start start 2024-03-06T07:30:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.378+0000 7f2ded478640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de80a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.378+0000 7f2ded478640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2de80c3280 0x7f2de80c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.378+0000 7f2ded478640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de8158800 con 0x7f2de81551e0 2024-03-06T07:30:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.378+0000 7f2ded478640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de8158ae0 con 0x7f2de80c3280 2024-03-06T07:30:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.378+0000 7f2ded478640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8158dc0 con 0x7f2de806d9f0 2024-03-06T07:30:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de6ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de80a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de67fc640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2de80c3280 0x7f2de80c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de77fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2de81551e0 0x7f2de8155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60088/0 (socket says 172.21.15.26:60088) 2024-03-06T07:30:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de77fe640 1 -- 172.21.15.26:0/2677812628 learned_addr learned my addr 172.21.15.26:0/2677812628 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de6ffd640 1 -- 172.21.15.26:0/2677812628 >> v2:172.21.15.117:3300/0 conn(0x7f2de80c3280 msgr2=0x7f2de80c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de6ffd640 1 --2- 172.21.15.26:0/2677812628 >> v2:172.21.15.117:3300/0 conn(0x7f2de80c3280 0x7f2de80c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de6ffd640 1 -- 172.21.15.26:0/2677812628 >> v1:172.21.15.117:6789/0 conn(0x7f2de81551e0 legacy=0x7f2de8155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de5ffb640 1 -- 172.21.15.26:0/2677812628 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143803816 0 0) 0x7f2de8158800 con 0x7f2de81551e0 2024-03-06T07:30:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de6ffd640 1 -- 172.21.15.26:0/2677812628 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de81590a0 con 0x7f2de806d9f0 2024-03-06T07:30:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.379+0000 7f2de67fc640 1 --2- 172.21.15.26:0/2677812628 >> v2:172.21.15.117:3300/0 conn(0x7f2de80c3280 0x7f2de80c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.380+0000 7f2de6ffd640 1 --2- 172.21.15.26:0/2677812628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de80a7cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2de00097a0 tx=0x7f2de002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870a27e73b5d13d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.380+0000 7f2de5ffb640 1 -- 172.21.15.26:0/2677812628 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00315a0 con 0x7f2de806d9f0 2024-03-06T07:30:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.380+0000 7f2de5ffb640 1 -- 172.21.15.26:0/2677812628 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de0037070 con 0x7f2de806d9f0 2024-03-06T07:30:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.380+0000 7f2de5ffb640 1 -- 172.21.15.26:0/2677812628 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0031960 con 0x7f2de806d9f0 2024-03-06T07:30:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 -- 172.21.15.26:0/2677812628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 msgr2=0x7f2de80a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 --2- 172.21.15.26:0/2677812628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de80a7cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2de00097a0 tx=0x7f2de002b840 comp rx=0 tx=0).stop 2024-03-06T07:30:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 -- 172.21.15.26:0/2677812628 shutdown_connections 2024-03-06T07:30:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 --2- 172.21.15.26:0/2677812628 >> v2:172.21.15.117:3300/0 conn(0x7f2de80c3280 0x7f2de80c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 --2- 172.21.15.26:0/2677812628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de80a7cd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 -- 172.21.15.26:0/2677812628 >> 172.21.15.26:0/2677812628 conn(0x7f2de80aa630 msgr2=0x7f2de80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 -- 172.21.15.26:0/2677812628 shutdown_connections 2024-03-06T07:30:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.381+0000 7f2ded478640 1 -- 172.21.15.26:0/2677812628 wait complete. 2024-03-06T07:30:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.382+0000 7f2ded478640 1 Processor -- start 2024-03-06T07:30:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.382+0000 7f2ded478640 1 -- start start 2024-03-06T07:30:21.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.383+0000 7f2ded478640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de81eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2ded478640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de80c3280 0x7f2de81574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2ded478640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de81551e0 0x7f2de814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2ded478640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de806df50 con 0x7f2de81551e0 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2ded478640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de806ddd0 con 0x7f2de80c3280 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2ded478640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de80a8210 con 0x7f2de806d9f0 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de81551e0 0x7f2de814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de81551e0 0x7f2de814f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58046/0 (socket says 172.21.15.26:58046) 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 -- 172.21.15.26:0/2103232910 learned_addr learned my addr 172.21.15.26:0/2103232910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de6ffd640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de81eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de67fc640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de80c3280 0x7f2de81574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 -- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 msgr2=0x7f2de81eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de81eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 -- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de80c3280 msgr2=0x7f2de81574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de80c3280 0x7f2de81574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 -- 172.21.15.26:0/2103232910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de806f790 con 0x7f2de81551e0 2024-03-06T07:30:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2de77fe640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de81551e0 0x7f2de814f650 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c3c0 tx=0x7f2dd800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2dcf7fe640 1 -- 172.21.15.26:0/2103232910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80074e0 con 0x7f2de81551e0 2024-03-06T07:30:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2dcf7fe640 1 -- 172.21.15.26:0/2103232910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd80047e0 con 0x7f2de81551e0 2024-03-06T07:30:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de806fa70 con 0x7f2de81551e0 2024-03-06T07:30:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.384+0000 7f2dcf7fe640 1 -- 172.21.15.26:0/2103232910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8004c10 con 0x7f2de81551e0 2024-03-06T07:30:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.385+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de8158020 con 0x7f2de81551e0 2024-03-06T07:30:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.386+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de804f330 con 0x7f2de81551e0 2024-03-06T07:30:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.386+0000 7f2dcf7fe640 1 -- 172.21.15.26:0/2103232910 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dd8011070 con 0x7f2de81551e0 2024-03-06T07:30:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.387+0000 7f2dcf7fe640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2dd406b4d0 0x7f2dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.387+0000 7f2dcf7fe640 1 -- 172.21.15.26:0/2103232910 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f2dd8089dd0 con 0x7f2de81551e0 2024-03-06T07:30:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.387+0000 7f2de6ffd640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2dd406b4d0 0x7f2dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.388+0000 7f2de6ffd640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2dd406b4d0 0x7f2dd406d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2de0036000 tx=0x7f2de0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.391+0000 7f2dcf7fe640 1 -- 172.21.15.26:0/2103232910 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2dd8058450 con 0x7f2de81551e0 2024-03-06T07:30:21.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.651+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2de8073240 con 0x7f2de81551e0 2024-03-06T07:30:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.651+0000 7f2dcf7fe640 1 -- 172.21.15.26:0/2103232910 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f2dd805c4f0 con 0x7f2de81551e0 2024-03-06T07:30:21.653 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:21.653 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2dd406b4d0 msgr2=0x7f2dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2dd406b4d0 0x7f2dd406d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2de0036000 tx=0x7f2de0008030 comp rx=0 tx=0).stop 2024-03-06T07:30:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de81551e0 msgr2=0x7f2de814f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de81551e0 0x7f2de814f650 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c3c0 tx=0x7f2dd800c880 comp rx=0 tx=0).stop 2024-03-06T07:30:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 shutdown_connections 2024-03-06T07:30:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2dd406b4d0 0x7f2dd406d990 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de81551e0 0x7f2de814f650 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de80c3280 0x7f2de81574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 --2- 172.21.15.26:0/2103232910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de806d9f0 0x7f2de81eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 >> 172.21.15.26:0/2103232910 conn(0x7f2de80aa630 msgr2=0x7f2de806cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 shutdown_connections 2024-03-06T07:30:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:21.653+0000 7f2ded478640 1 -- 172.21.15.26:0/2103232910 wait complete. 2024-03-06T07:30:22.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:22.068 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:23.069 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.170+0000 7ff97293a640 1 Processor -- start 2024-03-06T07:30:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.170+0000 7ff97293a640 1 -- start start 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff97293a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff96c152520 0x7ff96c06ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff97293a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c155c80 0x7ff96c0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff97293a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0c1ae0 con 0x7ff96c0720a0 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff97293a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0c1dc0 con 0x7ff96c152520 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff97293a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c06e450 con 0x7ff96c155c80 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff96c152520 0x7ff96c06ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff970ed7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff96c0720a0 0x7ff96c06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34086/0 (socket says 172.21.15.26:34086) 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff970ed7640 1 -- 172.21.15.26:0/1865210370 learned_addr learned my addr 172.21.15.26:0/1865210370 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96affd640 1 -- 172.21.15.26:0/1865210370 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268971586 0 0) 0x7ff96c0c1ae0 con 0x7ff96c0720a0 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96affd640 1 -- 172.21.15.26:0/1865210370 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff954003520 con 0x7ff96c0720a0 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96b7fe640 1 --2- 172.21.15.26:0/1865210370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c155c80 0x7ff96c0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96bfff640 1 -- 172.21.15.26:0/1865210370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c155c80 msgr2=0x7ff96c0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96bfff640 1 --2- 172.21.15.26:0/1865210370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c155c80 0x7ff96c0c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96bfff640 1 -- 172.21.15.26:0/1865210370 >> v1:172.21.15.26:6789/0 conn(0x7ff96c0720a0 legacy=0x7ff96c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.171+0000 7ff96bfff640 1 -- 172.21.15.26:0/1865210370 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c06e730 con 0x7ff96c152520 2024-03-06T07:30:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.172+0000 7ff96bfff640 1 --2- 172.21.15.26:0/1865210370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c152520 0x7ff96c06ef30 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ff9600097a0 tx=0x7ff96002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c938d7454099e7d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.172+0000 7ff96affd640 1 -- 172.21.15.26:0/1865210370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960035020 con 0x7ff96c152520 2024-03-06T07:30:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.172+0000 7ff96affd640 1 -- 172.21.15.26:0/1865210370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff960031830 con 0x7ff96c152520 2024-03-06T07:30:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.172+0000 7ff96affd640 1 -- 172.21.15.26:0/1865210370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960031c00 con 0x7ff96c152520 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 -- 172.21.15.26:0/1865210370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c152520 msgr2=0x7ff96c06ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 --2- 172.21.15.26:0/1865210370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c152520 0x7ff96c06ef30 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ff9600097a0 tx=0x7ff96002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 -- 172.21.15.26:0/1865210370 shutdown_connections 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 --2- 172.21.15.26:0/1865210370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c155c80 0x7ff96c0c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 --2- 172.21.15.26:0/1865210370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c152520 0x7ff96c06ef30 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 -- 172.21.15.26:0/1865210370 >> 172.21.15.26:0/1865210370 conn(0x7ff96c0c76b0 msgr2=0x7ff96c0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 -- 172.21.15.26:0/1865210370 shutdown_connections 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.173+0000 7ff97293a640 1 -- 172.21.15.26:0/1865210370 wait complete. 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.174+0000 7ff97293a640 1 Processor -- start 2024-03-06T07:30:23.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.174+0000 7ff97293a640 1 -- start start 2024-03-06T07:30:23.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff97293a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c0720a0 0x7ff96c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff97293a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff96c152520 0x7ff96c13c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff97293a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c155c80 0x7ff96c13cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff97293a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c06b6d0 con 0x7ff96c155c80 2024-03-06T07:30:23.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff97293a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0c17f0 con 0x7ff96c152520 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff97293a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c06b850 con 0x7ff96c0720a0 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c155c80 0x7ff96c13cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c155c80 0x7ff96c13cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58064/0 (socket says 172.21.15.26:58064) 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 -- 172.21.15.26:0/2258584159 learned_addr learned my addr 172.21.15.26:0/2258584159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff96bfff640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c0720a0 0x7ff96c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff96b7fe640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff96c152520 0x7ff96c13c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 -- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c0720a0 msgr2=0x7ff96c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c0720a0 0x7ff96c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 -- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff96c152520 msgr2=0x7ff96c13c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff96c152520 0x7ff96c13c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.175+0000 7ff970ed7640 1 -- 172.21.15.26:0/2258584159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c13d1e0 con 0x7ff96c155c80 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.176+0000 7ff970ed7640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c155c80 0x7ff96c13cca0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7ff96400cad0 tx=0x7ff96400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:23.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.176+0000 7ff968ff9640 1 -- 172.21.15.26:0/2258584159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964017070 con 0x7ff96c155c80 2024-03-06T07:30:23.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.176+0000 7ff968ff9640 1 -- 172.21.15.26:0/2258584159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96400ac30 con 0x7ff96c155c80 2024-03-06T07:30:23.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.176+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c1568e0 con 0x7ff96c155c80 2024-03-06T07:30:23.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.176+0000 7ff968ff9640 1 -- 172.21.15.26:0/2258584159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964012400 con 0x7ff96c155c80 2024-03-06T07:30:23.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.176+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c156600 con 0x7ff96c155c80 2024-03-06T07:30:23.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.177+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c072c60 con 0x7ff96c155c80 2024-03-06T07:30:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.178+0000 7ff968ff9640 1 -- 172.21.15.26:0/2258584159 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff96401a070 con 0x7ff96c155c80 2024-03-06T07:30:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.178+0000 7ff968ff9640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff95806b500 0x7ff95806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.178+0000 7ff968ff9640 1 -- 172.21.15.26:0/2258584159 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7ff96408a050 con 0x7ff96c155c80 2024-03-06T07:30:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.179+0000 7ff96bfff640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff95806b500 0x7ff95806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.179+0000 7ff96bfff640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff95806b500 0x7ff95806d9c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff960036000 tx=0x7ff96003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:23.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.183+0000 7ff968ff9640 1 -- 172.21.15.26:0/2258584159 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff96c072c60 con 0x7ff96c155c80 2024-03-06T07:30:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.620+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff96c04f330 con 0x7ff96c155c80 2024-03-06T07:30:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.620+0000 7ff968ff9640 1 -- 172.21.15.26:0/2258584159 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7ff9640586d0 con 0x7ff96c155c80 2024-03-06T07:30:23.622 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:23.622 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.622+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff95806b500 msgr2=0x7ff95806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff95806b500 0x7ff95806d9c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff960036000 tx=0x7ff96003c040 comp rx=0 tx=0).stop 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c155c80 msgr2=0x7ff96c13cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c155c80 0x7ff96c13cca0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7ff96400cad0 tx=0x7ff96400cf90 comp rx=0 tx=0).stop 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 shutdown_connections 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff95806b500 0x7ff95806d9c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff96c155c80 0x7ff96c13cca0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff96c152520 0x7ff96c13c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 --2- 172.21.15.26:0/2258584159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff96c0720a0 0x7ff96c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 >> 172.21.15.26:0/2258584159 conn(0x7ff96c0c76b0 msgr2=0x7ff96c0c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 shutdown_connections 2024-03-06T07:30:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:23.623+0000 7ff97293a640 1 -- 172.21.15.26:0/2258584159 wait complete. 2024-03-06T07:30:24.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:24.039 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:25.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:25.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.140+0000 7fb882f56640 1 Processor -- start 2024-03-06T07:30:25.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.140+0000 7fb882f56640 1 -- start start 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.141+0000 7fb882f56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c14f650 0x7fb87c071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.141+0000 7fb882f56640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb87c0c3150 0x7fb87c0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.141+0000 7fb882f56640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c072440 con 0x7fb87c152c20 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.141+0000 7fb882f56640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87c0c2b10 con 0x7fb87c0c3150 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.141+0000 7fb882f56640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c152140 con 0x7fb87c14f650 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb87bfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb87c0c3150 0x7fb87c0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb880cf2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c14f650 0x7fb87c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb8814f3640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb87c152c20 0x7fb87c1554a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:60098/0 (socket says 172.21.15.26:60098) 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb87bfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb87c0c3150 0x7fb87c0c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43670/0 (socket says 172.21.15.26:43670) 2024-03-06T07:30:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb880cf2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c14f650 0x7fb87c071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:40664/0 (socket says 172.21.15.26:40664) 2024-03-06T07:30:25.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb8814f3640 1 -- 172.21.15.26:0/1143453034 learned_addr learned my addr 172.21.15.26:0/1143453034 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:25.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb880cf2640 1 -- 172.21.15.26:0/1143453034 >> v2:172.21.15.117:3300/0 conn(0x7fb87c0c3150 msgr2=0x7fb87c0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:25.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb880cf2640 1 --2- 172.21.15.26:0/1143453034 >> v2:172.21.15.117:3300/0 conn(0x7fb87c0c3150 0x7fb87c0c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb880cf2640 1 -- 172.21.15.26:0/1143453034 >> v1:172.21.15.117:6789/0 conn(0x7fb87c152c20 legacy=0x7fb87c1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb880cf2640 1 -- 172.21.15.26:0/1143453034 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c152420 con 0x7fb87c14f650 2024-03-06T07:30:25.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb87b7fe640 1 -- 172.21.15.26:0/1143453034 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543407506 0 0) 0x7fb874001490 con 0x7fb87c152c20 2024-03-06T07:30:25.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.142+0000 7fb87bfff640 1 --2- 172.21.15.26:0/1143453034 >> v2:172.21.15.117:3300/0 conn(0x7fb87c0c3150 0x7fb87c0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.143+0000 7fb880cf2640 1 --2- 172.21.15.26:0/1143453034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c14f650 0x7fb87c071880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb8700097a0 tx=0x7fb87002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af4b072c905e995c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.143+0000 7fb87b7fe640 1 -- 172.21.15.26:0/1143453034 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870035020 con 0x7fb87c14f650 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.143+0000 7fb87b7fe640 1 -- 172.21.15.26:0/1143453034 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870031830 con 0x7fb87c14f650 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.143+0000 7fb87b7fe640 1 -- 172.21.15.26:0/1143453034 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870031c00 con 0x7fb87c14f650 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.143+0000 7fb882f56640 1 -- 172.21.15.26:0/1143453034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c14f650 msgr2=0x7fb87c071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.143+0000 7fb882f56640 1 --2- 172.21.15.26:0/1143453034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c14f650 0x7fb87c071880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb8700097a0 tx=0x7fb87002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.144+0000 7fb882f56640 1 -- 172.21.15.26:0/1143453034 shutdown_connections 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.144+0000 7fb882f56640 1 --2- 172.21.15.26:0/1143453034 >> v2:172.21.15.117:3300/0 conn(0x7fb87c0c3150 0x7fb87c0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.144+0000 7fb882f56640 1 --2- 172.21.15.26:0/1143453034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c14f650 0x7fb87c071880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.144+0000 7fb882f56640 1 -- 172.21.15.26:0/1143453034 >> 172.21.15.26:0/1143453034 conn(0x7fb87c0c4290 msgr2=0x7fb87c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.144+0000 7fb882f56640 1 -- 172.21.15.26:0/1143453034 shutdown_connections 2024-03-06T07:30:25.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.144+0000 7fb882f56640 1 -- 172.21.15.26:0/1143453034 wait complete. 2024-03-06T07:30:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 Processor -- start 2024-03-06T07:30:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 -- start start 2024-03-06T07:30:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c0c3150 0x7fb87c1e8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87c14f650 0x7fb87c1ec700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c152c20 0x7fb87c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c40c0 con 0x7fb87c14f650 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c3f40 con 0x7fb87c152c20 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.145+0000 7fb882f56640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c071dc0 con 0x7fb87c0c3150 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87c14f650 0x7fb87c1ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87c14f650 0x7fb87c1ec700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58070/0 (socket says 172.21.15.26:58070) 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 -- 172.21.15.26:0/2893925555 learned_addr learned my addr 172.21.15.26:0/2893925555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb880cf2640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c0c3150 0x7fb87c1e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb8814f3640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c152c20 0x7fb87c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 -- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c0c3150 msgr2=0x7fb87c1e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c0c3150 0x7fb87c1e8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 -- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c152c20 msgr2=0x7fb87c1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c152c20 0x7fb87c1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 -- 172.21.15.26:0/2893925555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c14ae20 con 0x7fb87c14f650 2024-03-06T07:30:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb87bfff640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87c14f650 0x7fb87c1ec700 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ebe0 tx=0x7fb86800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb8797fa640 1 -- 172.21.15.26:0/2893925555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868017070 con 0x7fb87c14f650 2024-03-06T07:30:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb8797fa640 1 -- 172.21.15.26:0/2893925555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86800cc30 con 0x7fb87c14f650 2024-03-06T07:30:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c14b100 con 0x7fb87c14f650 2024-03-06T07:30:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb8797fa640 1 -- 172.21.15.26:0/2893925555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868013470 con 0x7fb87c14f650 2024-03-06T07:30:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.146+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c1ebf00 con 0x7fb87c14f650 2024-03-06T07:30:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.147+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c04f330 con 0x7fb87c14f650 2024-03-06T07:30:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.148+0000 7fb8797fa640 1 -- 172.21.15.26:0/2893925555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb86801a070 con 0x7fb87c14f650 2024-03-06T07:30:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.149+0000 7fb8797fa640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86006b500 0x7fb86006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.149+0000 7fb8797fa640 1 -- 172.21.15.26:0/2893925555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fb86808a590 con 0x7fb87c14f650 2024-03-06T07:30:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.149+0000 7fb880cf2640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86006b500 0x7fb86006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.150+0000 7fb880cf2640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86006b500 0x7fb86006d9c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb870036000 tx=0x7fb87003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.153+0000 7fb8797fa640 1 -- 172.21.15.26:0/2893925555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb868058c10 con 0x7fb87c14f650 2024-03-06T07:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.371+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb87c068460 con 0x7fb87c14f650 2024-03-06T07:30:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.371+0000 7fb8797fa640 1 -- 172.21.15.26:0/2893925555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fb86805ccb0 con 0x7fb87c14f650 2024-03-06T07:30:25.373 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:25.373 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86006b500 msgr2=0x7fb86006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86006b500 0x7fb86006d9c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb870036000 tx=0x7fb87003d040 comp rx=0 tx=0).stop 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87c14f650 msgr2=0x7fb87c1ec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87c14f650 0x7fb87c1ec700 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ebe0 tx=0x7fb86800c520 comp rx=0 tx=0).stop 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 shutdown_connections 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb86006b500 0x7fb86006d9c0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c152c20 0x7fb87c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87c14f650 0x7fb87c1ec700 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 --2- 172.21.15.26:0/2893925555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb87c0c3150 0x7fb87c1e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 >> 172.21.15.26:0/2893925555 conn(0x7fb87c0c4290 msgr2=0x7fb87c15da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 shutdown_connections 2024-03-06T07:30:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:25.373+0000 7fb882f56640 1 -- 172.21.15.26:0/2893925555 wait complete. 2024-03-06T07:30:25.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:25.788 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:26.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:26.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff9969e9640 1 Processor -- start 2024-03-06T07:30:26.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff9969e9640 1 -- start start 2024-03-06T07:30:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff9969e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff99014f360 0x7ff990155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff9969e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9900c88c0 0x7ff9900c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff9969e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9900c73d0 con 0x7ff9900c5880 2024-03-06T07:30:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff9969e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9900c8dd0 con 0x7ff9900c88c0 2024-03-06T07:30:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff9969e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9900be2e0 con 0x7ff99014f360 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff98f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9900c88c0 0x7ff9900c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff98f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9900c88c0 0x7ff9900c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58072/0 (socket says 172.21.15.26:58072) 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.890+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2119649860 learned_addr learned my addr 172.21.15.26:0/2119649860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.891+0000 7ff98ffff640 1 --2- 172.21.15.26:0/2119649860 >> v2:172.21.15.117:3300/0 conn(0x7ff99014f360 0x7ff990155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.891+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2119649860 >> v2:172.21.15.117:3300/0 conn(0x7ff99014f360 msgr2=0x7ff990155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.891+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/2119649860 >> v2:172.21.15.117:3300/0 conn(0x7ff99014f360 0x7ff990155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.891+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2119649860 >> v1:172.21.15.117:6789/0 conn(0x7ff9900c5880 legacy=0x7ff990158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.891+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2119649860 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900c7670 con 0x7ff9900c88c0 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.891+0000 7ff98ffff640 1 --2- 172.21.15.26:0/2119649860 >> v2:172.21.15.117:3300/0 conn(0x7ff99014f360 0x7ff990155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/2119649860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff9900c5340 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7ff99008c840 tx=0x7ff98402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1effd042dd84ddd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff98effd640 1 -- 172.21.15.26:0/2119649860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98402e070 con 0x7ff9900c88c0 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff98effd640 1 -- 172.21.15.26:0/2119649860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984037070 con 0x7ff9900c88c0 2024-03-06T07:30:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff98effd640 1 -- 172.21.15.26:0/2119649860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984031a00 con 0x7ff9900c88c0 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 -- 172.21.15.26:0/2119649860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 msgr2=0x7ff9900c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 --2- 172.21.15.26:0/2119649860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff9900c5340 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7ff99008c840 tx=0x7ff98402b880 comp rx=0 tx=0).stop 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 -- 172.21.15.26:0/2119649860 shutdown_connections 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 --2- 172.21.15.26:0/2119649860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff9900c5340 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 --2- 172.21.15.26:0/2119649860 >> v2:172.21.15.117:3300/0 conn(0x7ff99014f360 0x7ff990155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 -- 172.21.15.26:0/2119649860 >> 172.21.15.26:0/2119649860 conn(0x7ff9900ac570 msgr2=0x7ff9900ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 -- 172.21.15.26:0/2119649860 shutdown_connections 2024-03-06T07:30:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.892+0000 7ff9969e9640 1 -- 172.21.15.26:0/2119649860 wait complete. 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.894+0000 7ff9969e9640 1 Processor -- start 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.894+0000 7ff9969e9640 1 -- start start 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.895+0000 7ff9969e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9900c5880 0x7ff9900afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.895+0000 7ff9969e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff990157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.895+0000 7ff9969e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff99014f360 0x7ff99014b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.895+0000 7ff9969e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff990159270 con 0x7ff9900c88c0 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.895+0000 7ff9969e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900c7c30 con 0x7ff9900c5880 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.895+0000 7ff9969e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901593f0 con 0x7ff99014f360 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff990157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff990157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58084/0 (socket says 172.21.15.26:58084) 2024-03-06T07:30:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4287854458 learned_addr learned my addr 172.21.15.26:0/4287854458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98ffff640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9900c5880 0x7ff9900afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff994f86640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff99014f360 0x7ff99014b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff99014f360 msgr2=0x7ff99014b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff99014f360 0x7ff99014b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9900c5880 msgr2=0x7ff9900afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9900c5880 0x7ff9900afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4287854458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900af580 con 0x7ff9900c88c0 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff994f86640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff99014f360 0x7ff99014b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff990157bf0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7ff98402b8c0 tx=0x7ff9840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98cff9640 1 -- 172.21.15.26:0/4287854458 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984032880 con 0x7ff9900c88c0 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98cff9640 1 -- 172.21.15.26:0/4287854458 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984032a20 con 0x7ff9900c88c0 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9901ec260 con 0x7ff9900c88c0 2024-03-06T07:30:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff98cff9640 1 -- 172.21.15.26:0/4287854458 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984032e10 con 0x7ff9900c88c0 2024-03-06T07:30:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.896+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901ec820 con 0x7ff9900c88c0 2024-03-06T07:30:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.897+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99004f330 con 0x7ff9900c88c0 2024-03-06T07:30:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.898+0000 7ff98cff9640 1 -- 172.21.15.26:0/4287854458 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff98404c060 con 0x7ff9900c88c0 2024-03-06T07:30:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.898+0000 7ff98cff9640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff97c06b4d0 0x7ff97c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.898+0000 7ff98cff9640 1 -- 172.21.15.26:0/4287854458 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7ff9840abf30 con 0x7ff9900c88c0 2024-03-06T07:30:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.898+0000 7ff98ffff640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff97c06b4d0 0x7ff97c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.899+0000 7ff98ffff640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff97c06b4d0 0x7ff97c06d990 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff98000aac0 tx=0x7ff980008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:26.903+0000 7ff98cff9640 1 -- 172.21.15.26:0/4287854458 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff984040070 con 0x7ff9900c88c0 2024-03-06T07:30:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.127+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff99006cda0 con 0x7ff9900c88c0 2024-03-06T07:30:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.127+0000 7ff98cff9640 1 -- 172.21.15.26:0/4287854458 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7ff98407ac00 con 0x7ff9900c88c0 2024-03-06T07:30:27.129 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:27.129 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff97c06b4d0 msgr2=0x7ff97c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff97c06b4d0 0x7ff97c06d990 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff98000aac0 tx=0x7ff980008040 comp rx=0 tx=0).stop 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 msgr2=0x7ff990157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff990157bf0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7ff98402b8c0 tx=0x7ff9840046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 shutdown_connections 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff97c06b4d0 0x7ff97c06d990 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff99014f360 0x7ff99014b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9900c88c0 0x7ff990157bf0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 --2- 172.21.15.26:0/4287854458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9900c5880 0x7ff9900afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 >> 172.21.15.26:0/4287854458 conn(0x7ff9900ac570 msgr2=0x7ff99006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 shutdown_connections 2024-03-06T07:30:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:27.130+0000 7ff9969e9640 1 -- 172.21.15.26:0/4287854458 wait complete. 2024-03-06T07:30:27.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:27.545 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:28.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.646+0000 7fdd177eb640 1 Processor -- start 2024-03-06T07:30:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.646+0000 7fdd177eb640 1 -- start start 2024-03-06T07:30:28.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd177eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd100c5480 0x7fdd10155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd177eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 0x7fdd1006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd177eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd100c5cc0 con 0x7fdd101586f0 2024-03-06T07:30:28.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd177eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1006df40 con 0x7fdd100c5480 2024-03-06T07:30:28.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd177eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd101591c0 con 0x7fdd100c5860 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd15d88640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdd101586f0 0x7fdd100724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57936/0 (socket says 172.21.15.26:57936) 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd15d88640 1 -- 172.21.15.26:0/4043389594 learned_addr learned my addr 172.21.15.26:0/4043389594 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.647+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764378271 0 0) 0x7fdd100c5cc0 con 0x7fdd101586f0 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd15587640 1 --2- 172.21.15.26:0/4043389594 >> v2:172.21.15.117:3300/0 conn(0x7fdd100c5480 0x7fdd10155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcfc003520 con 0x7fdd101586f0 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd14d86640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 0x7fdd1006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381263650 0 0) 0x7fdcfc003520 con 0x7fdd101586f0 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 msgr2=0x7fdd1006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 0x7fdd1006f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 >> v2:172.21.15.117:3300/0 conn(0x7fdd100c5480 msgr2=0x7fdd10155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 --2- 172.21.15.26:0/4043389594 >> v2:172.21.15.117:3300/0 conn(0x7fdd100c5480 0x7fdd10155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd101594a0 con 0x7fdd101586f0 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fdd080019f0 con 0x7fdd101586f0 2024-03-06T07:30:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 >> v1:172.21.15.26:6789/0 conn(0x7fdd101586f0 legacy=0x7fdd100724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 0x7fdcfc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1006df40 con 0x7fdcfc003f30 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd101591c0 con 0x7fdcfc004d40 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc00ba10 con 0x7fdd100c5860 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd14d86640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd15587640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 0x7fdcfc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd15d88640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd14d86640 1 -- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 msgr2=0x7fdcfc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd14d86640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd100c5860 0x7fdcfc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd14d86640 1 -- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc004d40 msgr2=0x7fdcfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd14d86640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd14d86640 1 -- 172.21.15.26:0/4043389594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc003520 con 0x7fdcfc003f30 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.648+0000 7fdd15d88640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.649+0000 7fdd14d86640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009af0 tx=0x7fdcfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89faa2f126788 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.649+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c039070 con 0x7fdcfc003f30 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.649+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c02c6b0 con 0x7fdcfc003f30 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.649+0000 7fdd07fff640 1 -- 172.21.15.26:0/4043389594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c009690 con 0x7fdcfc003f30 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.650+0000 7fdd177eb640 1 -- 172.21.15.26:0/4043389594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 msgr2=0x7fdcfc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.650+0000 7fdd177eb640 1 --2- 172.21.15.26:0/4043389594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009af0 tx=0x7fdcfc005830 comp rx=0 tx=0).stop 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.650+0000 7fdd15d88640 1 -- 172.21.15.26:0/4043389594 reap_dead start 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.650+0000 7fdd177eb640 1 -- 172.21.15.26:0/4043389594 shutdown_connections 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.650+0000 7fdd177eb640 1 -- 172.21.15.26:0/4043389594 >> 172.21.15.26:0/4043389594 conn(0x7fdd1006b1a0 msgr2=0x7fdd1006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.650+0000 7fdd177eb640 1 -- 172.21.15.26:0/4043389594 shutdown_connections 2024-03-06T07:30:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.650+0000 7fdd177eb640 1 -- 172.21.15.26:0/4043389594 wait complete. 2024-03-06T07:30:28.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 Processor -- start 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 -- start start 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdd101e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd1014bf00 0x7fdd1014aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd1014b1e0 0x7fdd101eae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd10152a20 con 0x7fdcfc003f30 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd10155680 con 0x7fdd1014bf00 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.651+0000 7fdd177eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd10155050 con 0x7fdd1014b1e0 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdd101e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdd101e19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59736/0 (socket says 172.21.15.26:59736) 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 -- 172.21.15.26:0/1245302346 learned_addr learned my addr 172.21.15.26:0/1245302346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 -- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd1014b1e0 msgr2=0x7fdd101eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd1014b1e0 0x7fdd101eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 -- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd1014bf00 msgr2=0x7fdd1014aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd1014bf00 0x7fdd1014aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 -- 172.21.15.26:0/1245302346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1014c5c0 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd15587640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdd101e19e0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fdd00011aa0 tx=0x7fdd00011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd05ffb640 1 -- 172.21.15.26:0/1245302346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000fb00 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd05ffb640 1 -- 172.21.15.26:0/1245302346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0000fca0 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1014bc20 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd05ffb640 1 -- 172.21.15.26:0/1245302346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00014660 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.652+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd10151520 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.653+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1004f330 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.654+0000 7fdd05ffb640 1 -- 172.21.15.26:0/1245302346 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd00012040 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.654+0000 7fdd05ffb640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcf406b500 0x7fdcf406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.655+0000 7fdd05ffb640 1 -- 172.21.15.26:0/1245302346 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fdd0008aa20 con 0x7fdcfc003f30 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.655+0000 7fdd14d86640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcf406b500 0x7fdcf406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.655+0000 7fdd14d86640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcf406b500 0x7fdcf406d9c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c036000 tx=0x7fdd0c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.659+0000 7fdd05ffb640 1 -- 172.21.15.26:0/1245302346 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdd00019030 con 0x7fdcfc003f30 2024-03-06T07:30:28.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.880+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd1006e0c0 con 0x7fdcfc003f30 2024-03-06T07:30:28.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.880+0000 7fdd05ffb640 1 -- 172.21.15.26:0/1245302346 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4219 (secure 0 0 0) 0x7fdd000590a0 con 0x7fdcfc003f30 2024-03-06T07:30:28.882 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:28.882 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":195}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:29:58.689074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8330":{"gid":8330,"name":"d","rank":1,"incarnation":203,"state":"up:stopping","state_seq":16,"addr":"172.21.15.117:6836/4173532851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4173532851},{"type":"v1","addr":"172.21.15.117:6836","nonce":4173532851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:28.882 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 208 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcf406b500 msgr2=0x7fdcf406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcf406b500 0x7fdcf406d9c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c036000 tx=0x7fdd0c02e000 comp rx=0 tx=0).stop 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 msgr2=0x7fdd101e19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdd101e19e0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fdd00011aa0 tx=0x7fdd00011f60 comp rx=0 tx=0).stop 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 shutdown_connections 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcf406b500 0x7fdcf406d9c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd1014b1e0 0x7fdd101eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd1014bf00 0x7fdd1014aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 --2- 172.21.15.26:0/1245302346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcfc003f30 0x7fdd101e19e0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 >> 172.21.15.26:0/1245302346 conn(0x7fdd1006b1a0 msgr2=0x7fdd1006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 shutdown_connections 2024-03-06T07:30:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:28.883+0000 7fdd177eb640 1 -- 172.21.15.26:0/1245302346 wait complete. 2024-03-06T07:30:29.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:29:58.689074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 214, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8330}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8330': {'gid': 8330, 'name': 'd', 'rank': 1, 'incarnation': 203, 'state': 'up:stopping', 'state_seq': 16, 'addr': '172.21.15.117:6836/4173532851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4173532851}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 4173532851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:29.297 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8330:up:stopping 2024-03-06T07:30:30.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:30.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.399+0000 7efda5925640 1 Processor -- start 2024-03-06T07:30:30.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.399+0000 7efda5925640 1 -- start start 2024-03-06T07:30:30.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efda5925640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efda0155c80 0x7efda006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efda5925640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efda5925640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda00c1ae0 con 0x7efda0152520 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efda5925640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda00c1dc0 con 0x7efda0155c80 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efda5925640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda006e450 con 0x7efda00720a0 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd97fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efda0155c80 0x7efda006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd97fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efda0155c80 0x7efda006ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59742/0 (socket says 172.21.15.26:59742) 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd97fff640 1 -- 172.21.15.26:0/664347240 learned_addr learned my addr 172.21.15.26:0/664347240 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd9f7fe640 1 --2- 172.21.15.26:0/664347240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd97fff640 1 -- 172.21.15.26:0/664347240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 msgr2=0x7efda006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd97fff640 1 --2- 172.21.15.26:0/664347240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd97fff640 1 -- 172.21.15.26:0/664347240 >> v1:172.21.15.117:6789/0 conn(0x7efda0152520 legacy=0x7efda00c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:30.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.400+0000 7efd97fff640 1 -- 172.21.15.26:0/664347240 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda006e730 con 0x7efda0155c80 2024-03-06T07:30:30.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.401+0000 7efd97fff640 1 --2- 172.21.15.26:0/664347240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda006ef30 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7efd88009870 tx=0x7efd8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3866c52064955a0c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:30.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.401+0000 7efd9e7fc640 1 -- 172.21.15.26:0/664347240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88035020 con 0x7efda0155c80 2024-03-06T07:30:30.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.401+0000 7efd9e7fc640 1 -- 172.21.15.26:0/664347240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd88031640 con 0x7efda0155c80 2024-03-06T07:30:30.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.401+0000 7efd9e7fc640 1 -- 172.21.15.26:0/664347240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8802d040 con 0x7efda0155c80 2024-03-06T07:30:30.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 -- 172.21.15.26:0/664347240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 msgr2=0x7efda006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 --2- 172.21.15.26:0/664347240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda006ef30 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7efd88009870 tx=0x7efd8802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 -- 172.21.15.26:0/664347240 shutdown_connections 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 --2- 172.21.15.26:0/664347240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 --2- 172.21.15.26:0/664347240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda006ef30 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 -- 172.21.15.26:0/664347240 >> 172.21.15.26:0/664347240 conn(0x7efda00c76b0 msgr2=0x7efda00c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 -- 172.21.15.26:0/664347240 shutdown_connections 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.402+0000 7efda5925640 1 -- 172.21.15.26:0/664347240 wait complete. 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.403+0000 7efda5925640 1 Processor -- start 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.403+0000 7efda5925640 1 -- start start 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.403+0000 7efda5925640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda01e0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efda5925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0152520 0x7efda01e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efda5925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda00ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efda5925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0152a80 con 0x7efda0155c80 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efda5925640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0152900 con 0x7efda0152520 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efda5925640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda006b500 con 0x7efda00720a0 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda00ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda00ac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59758/0 (socket says 172.21.15.26:59758) 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 -- 172.21.15.26:0/4285306281 learned_addr learned my addr 172.21.15.26:0/4285306281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9effd640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda01e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd97fff640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0152520 0x7efda01e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 -- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 msgr2=0x7efda01e0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda01e0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 -- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0152520 msgr2=0x7efda01e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0152520 0x7efda01e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 -- 172.21.15.26:0/4285306281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00c34a0 con 0x7efda0155c80 2024-03-06T07:30:30.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9f7fe640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda00ac0e0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7efd98007e90 tx=0x7efd9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:30.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9cff9640 1 -- 172.21.15.26:0/4285306281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98014420 con 0x7efda0155c80 2024-03-06T07:30:30.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9cff9640 1 -- 172.21.15.26:0/4285306281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98019070 con 0x7efda0155c80 2024-03-06T07:30:30.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda00c3780 con 0x7efda0155c80 2024-03-06T07:30:30.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efd9cff9640 1 -- 172.21.15.26:0/4285306281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98014800 con 0x7efda0155c80 2024-03-06T07:30:30.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.404+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda01e1de0 con 0x7efda0155c80 2024-03-06T07:30:30.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.406+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda004f330 con 0x7efda0155c80 2024-03-06T07:30:30.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.406+0000 7efd9cff9640 1 -- 172.21.15.26:0/4285306281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd980149a0 con 0x7efda0155c80 2024-03-06T07:30:30.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.406+0000 7efd9cff9640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd8006b4d0 0x7efd8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.406+0000 7efd9cff9640 1 -- 172.21.15.26:0/4285306281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7efd9808ab40 con 0x7efda0155c80 2024-03-06T07:30:30.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.406+0000 7efd9effd640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd8006b4d0 0x7efd8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.407+0000 7efd9effd640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd8006b4d0 0x7efd8006d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efd90004f60 tx=0x7efd90007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:30.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.411+0000 7efd9cff9640 1 -- 172.21.15.26:0/4285306281 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd98059170 con 0x7efda0155c80 2024-03-06T07:30:30.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.629+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda00c0b70 con 0x7efda0155c80 2024-03-06T07:30:30.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.630+0000 7efd9cff9640 1 -- 172.21.15.26:0/4285306281 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) v1 ==== 95+0+3422 (secure 0 0 0) 0x7efd9805d210 con 0x7efda0155c80 2024-03-06T07:30:30.632 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:30.632 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":209,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:29.675455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:replay","state_seq":1,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:30.632 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 209 2024-03-06T07:30:30.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd8006b4d0 msgr2=0x7efd8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd8006b4d0 0x7efd8006d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efd90004f60 tx=0x7efd90007e30 comp rx=0 tx=0).stop 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 msgr2=0x7efda00ac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda00ac0e0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7efd98007e90 tx=0x7efd9800e4b0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 shutdown_connections 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efd8006b4d0 0x7efd8006d990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda0155c80 0x7efda00ac0e0 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0152520 0x7efda01e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 --2- 172.21.15.26:0/4285306281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efda00720a0 0x7efda01e0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 >> 172.21.15.26:0/4285306281 conn(0x7efda00c76b0 msgr2=0x7efda00c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 shutdown_connections 2024-03-06T07:30:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:30.632+0000 7efda5925640 1 -- 172.21.15.26:0/4285306281 wait complete. 2024-03-06T07:30:31.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 209, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:29.675455+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:31.047 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:replay 2024-03-06T07:30:32.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:32.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.182+0000 7f927b9b6640 1 Processor -- start 2024-03-06T07:30:32.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f927b9b6640 1 -- start start 2024-03-06T07:30:32.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f927b9b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92740c88c0 0x7f9274155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f927b9b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9274158b60 0x7f9274158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f927b9b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92740c73d0 con 0x7f927414f360 2024-03-06T07:30:32.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f927b9b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92740c8dd0 con 0x7f9274158b60 2024-03-06T07:30:32.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f927b9b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740be2e0 con 0x7f92740c88c0 2024-03-06T07:30:32.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f9279f53640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9274158b60 0x7f9274158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f9279752640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f927414f360 0x7f92740c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57950/0 (socket says 172.21.15.26:57950) 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.183+0000 7f9279752640 1 -- 172.21.15.26:0/3797853317 learned_addr learned my addr 172.21.15.26:0/3797853317 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.184+0000 7f9278f51640 1 --2- 172.21.15.26:0/3797853317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92740c88c0 0x7f9274155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.184+0000 7f9279f53640 1 -- 172.21.15.26:0/3797853317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92740c88c0 msgr2=0x7f9274155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.184+0000 7f9279f53640 1 --2- 172.21.15.26:0/3797853317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92740c88c0 0x7f9274155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.184+0000 7f9279f53640 1 -- 172.21.15.26:0/3797853317 >> v1:172.21.15.26:6789/0 conn(0x7f927414f360 legacy=0x7f92740c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.184+0000 7f9279f53640 1 -- 172.21.15.26:0/3797853317 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740c7670 con 0x7f9274158b60 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.184+0000 7f9267fff640 1 -- 172.21.15.26:0/3797853317 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930889880 0 0) 0x7f92740c73d0 con 0x7f927414f360 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.184+0000 7f9279f53640 1 --2- 172.21.15.26:0/3797853317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9274158b60 0x7f9274158ff0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f926c009d90 tx=0x7f926c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f45c0493b909a12 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f9267fff640 1 -- 172.21.15.26:0/3797853317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c039070 con 0x7f9274158b60 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f9267fff640 1 -- 172.21.15.26:0/3797853317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c02cbc0 con 0x7f9274158b60 2024-03-06T07:30:32.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f9267fff640 1 -- 172.21.15.26:0/3797853317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c034440 con 0x7f9274158b60 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 -- 172.21.15.26:0/3797853317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9274158b60 msgr2=0x7f9274158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 --2- 172.21.15.26:0/3797853317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9274158b60 0x7f9274158ff0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f926c009d90 tx=0x7f926c02be10 comp rx=0 tx=0).stop 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 -- 172.21.15.26:0/3797853317 shutdown_connections 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 --2- 172.21.15.26:0/3797853317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9274158b60 0x7f9274158ff0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 --2- 172.21.15.26:0/3797853317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92740c88c0 0x7f9274155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 -- 172.21.15.26:0/3797853317 >> 172.21.15.26:0/3797853317 conn(0x7f92740ac570 msgr2=0x7f92740ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 -- 172.21.15.26:0/3797853317 shutdown_connections 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.185+0000 7f927b9b6640 1 -- 172.21.15.26:0/3797853317 wait complete. 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.186+0000 7f927b9b6640 1 Processor -- start 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.186+0000 7f927b9b6640 1 -- start start 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f927b9b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92740c88c0 0x7f92741ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f927b9b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927414f360 0x7f92740bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f927b9b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9274158b60 0x7f927414b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f927b9b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740c5a50 con 0x7f927414f360 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f927b9b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740c7c30 con 0x7f92740c88c0 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f927b9b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740c5bd0 con 0x7f9274158b60 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927414f360 0x7f92740bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927414f360 0x7f92740bf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59776/0 (socket says 172.21.15.26:59776) 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 -- 172.21.15.26:0/373655284 learned_addr learned my addr 172.21.15.26:0/373655284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9279752640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92740c88c0 0x7f92741ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9279f53640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9274158b60 0x7f927414b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 -- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9274158b60 msgr2=0x7f927414b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9274158b60 0x7f927414b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 -- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92740c88c0 msgr2=0x7f92741ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92740c88c0 0x7f92741ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9278f51640 1 -- 172.21.15.26:0/373655284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92741ec610 con 0x7f927414f360 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.187+0000 7f9279752640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92740c88c0 0x7f92741ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.188+0000 7f9278f51640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927414f360 0x7f92740bf7a0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f927000d5f0 tx=0x7f927000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.188+0000 7f9265ffb640 1 -- 172.21.15.26:0/373655284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92700125f0 con 0x7f927414f360 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.188+0000 7f9265ffb640 1 -- 172.21.15.26:0/373655284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270019070 con 0x7f927414f360 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.188+0000 7f9265ffb640 1 -- 172.21.15.26:0/373655284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92700129d0 con 0x7f927414f360 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.188+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92740bfce0 con 0x7f927414f360 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.188+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92740c0ca0 con 0x7f927414f360 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.189+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927404f330 con 0x7f927414f360 2024-03-06T07:30:32.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.189+0000 7f9265ffb640 1 -- 172.21.15.26:0/373655284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9270012b70 con 0x7f927414f360 2024-03-06T07:30:32.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.190+0000 7f9265ffb640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925806b500 0x7f925806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.190+0000 7f9265ffb640 1 -- 172.21.15.26:0/373655284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f927008a6e0 con 0x7f927414f360 2024-03-06T07:30:32.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.190+0000 7f9279752640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925806b500 0x7f925806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.191+0000 7f9279752640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925806b500 0x7f925806d9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9268000f60 tx=0x7f9268007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:32.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.198+0000 7f9265ffb640 1 -- 172.21.15.26:0/373655284 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9270059360 con 0x7f927414f360 2024-03-06T07:30:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.419+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f927406cda0 con 0x7f927414f360 2024-03-06T07:30:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.420+0000 7f9265ffb640 1 -- 172.21.15.26:0/373655284 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+3426 (secure 0 0 0) 0x7f9270058c60 con 0x7f927414f360 2024-03-06T07:30:32.421 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:32.421 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:30.680674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:reconnect","state_seq":22,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 211 2024-03-06T07:30:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925806b500 msgr2=0x7f925806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925806b500 0x7f925806d9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9268000f60 tx=0x7f9268007ca0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927414f360 msgr2=0x7f92740bf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927414f360 0x7f92740bf7a0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f927000d5f0 tx=0x7f927000dab0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 shutdown_connections 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f925806b500 0x7f925806d9c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9274158b60 0x7f927414b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f927414f360 0x7f92740bf7a0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 --2- 172.21.15.26:0/373655284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92740c88c0 0x7f92741ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 >> 172.21.15.26:0/373655284 conn(0x7f92740ac570 msgr2=0x7f92740ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 shutdown_connections 2024-03-06T07:30:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:32.422+0000 7f927b9b6640 1 -- 172.21.15.26:0/373655284 wait complete. 2024-03-06T07:30:32.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:30.680674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:reconnect', 'state_seq': 22, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:32.836 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:reconnect 2024-03-06T07:30:33.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:33.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.938+0000 7fd0f05d6640 1 Processor -- start 2024-03-06T07:30:33.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.938+0000 7fd0f05d6640 1 -- start start 2024-03-06T07:30:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.939+0000 7fd0f05d6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd0e80c5480 0x7fd0e8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.939+0000 7fd0f05d6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80c5cc0 con 0x7fd0e80c5860 2024-03-06T07:30:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.939+0000 7fd0f05d6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e81591c0 con 0x7fd0e81586f0 2024-03-06T07:30:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.939+0000 7fd0f05d6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e806df40 con 0x7fd0e80c5480 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0edb71640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd0e80c5860 0x7fd0e806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57956/0 (socket says 172.21.15.26:57956) 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0edb71640 1 -- 172.21.15.26:0/2446692231 learned_addr learned my addr 172.21.15.26:0/2446692231 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ee372640 1 --2- 172.21.15.26:0/2446692231 >> v2:172.21.15.117:3300/0 conn(0x7fd0e80c5480 0x7fd0e8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255824120 0 0) 0x7fd0e80c5cc0 con 0x7fd0e80c5860 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d4003520 con 0x7fd0e80c5860 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046632236 0 0) 0x7fd0d4003520 con 0x7fd0e80c5860 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 >> v2:172.21.15.117:3300/0 conn(0x7fd0e80c5480 msgr2=0x7fd0e8155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 --2- 172.21.15.26:0/2446692231 >> v2:172.21.15.117:3300/0 conn(0x7fd0e80c5480 0x7fd0e8155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 >> v1:172.21.15.117:6789/0 conn(0x7fd0e81586f0 legacy=0x7fd0e815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e81594a0 con 0x7fd0e80c5860 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fd0d8001670 con 0x7fd0e80c5860 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 >> v1:172.21.15.26:6789/0 conn(0x7fd0e80c5860 legacy=0x7fd0e806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d40039b0 0x7fd0d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d4004800 0x7fd0d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d40051c0 0x7fd0d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e806df40 con 0x7fd0d4004800 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d400c160 con 0x7fd0d40051c0 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d400c360 con 0x7fd0d40039b0 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0edb71640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d4004800 0x7fd0d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0eeb73640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d40051c0 0x7fd0d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.940+0000 7fd0ee372640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d40039b0 0x7fd0d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0edb71640 1 -- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d40039b0 msgr2=0x7fd0d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0edb71640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d40039b0 0x7fd0d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0edb71640 1 -- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d40051c0 msgr2=0x7fd0d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0edb71640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d40051c0 0x7fd0d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0edb71640 1 -- 172.21.15.26:0/2446692231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d4003520 con 0x7fd0d4004800 2024-03-06T07:30:33.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0eeb73640 1 -- 172.21.15.26:0/2446692231 reap_dead start 2024-03-06T07:30:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0edb71640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d4004800 0x7fd0d4004c80 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c240 tx=0x7fd0d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6e6d8b6ffd7e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8007d10 con 0x7fd0d4004800 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d8030070 con 0x7fd0d4004800 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.941+0000 7fd0ed370640 1 -- 172.21.15.26:0/2446692231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d802cdc0 con 0x7fd0d4004800 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.942+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2446692231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d4004800 msgr2=0x7fd0d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.942+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d4004800 0x7fd0d4004c80 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c240 tx=0x7fd0d4005830 comp rx=0 tx=0).stop 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.942+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2446692231 shutdown_connections 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.942+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2446692231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d4004800 0x7fd0d4004c80 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.942+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2446692231 >> 172.21.15.26:0/2446692231 conn(0x7fd0e806b1a0 msgr2=0x7fd0e806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.942+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2446692231 shutdown_connections 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.942+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2446692231 wait complete. 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.944+0000 7fd0f05d6640 1 Processor -- start 2024-03-06T07:30:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.944+0000 7fd0f05d6640 1 -- start start 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0f05d6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d4004800 0x7fd0e81ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0f05d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0e81470e0 0x7fd0e81387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0f05d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0e806c9f0 0x7fd0e8138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0f05d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8155680 con 0x7fd0e806c9f0 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0f05d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e806fdb0 con 0x7fd0e81470e0 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0f05d6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8152a20 con 0x7fd0d4004800 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0e806c9f0 0x7fd0e8138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0e806c9f0 0x7fd0e8138ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59796/0 (socket says 172.21.15.26:59796) 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 -- 172.21.15.26:0/2111422534 learned_addr learned my addr 172.21.15.26:0/2111422534 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0edb71640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0e81470e0 0x7fd0e81387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0ee372640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d4004800 0x7fd0e81ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 -- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d4004800 msgr2=0x7fd0e81ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d4004800 0x7fd0e81ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 -- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0e81470e0 msgr2=0x7fd0e81387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0e81470e0 0x7fd0e81387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.945+0000 7fd0eeb73640 1 -- 172.21.15.26:0/2111422534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8146e70 con 0x7fd0e806c9f0 2024-03-06T07:30:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.946+0000 7fd0eeb73640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0e806c9f0 0x7fd0e8138ce0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80c2f40 tx=0x7fd0e000fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.946+0000 7fd0deffd640 1 -- 172.21.15.26:0/2111422534 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e001a070 con 0x7fd0e806c9f0 2024-03-06T07:30:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.946+0000 7fd0deffd640 1 -- 172.21.15.26:0/2111422534 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e0006b50 con 0x7fd0e806c9f0 2024-03-06T07:30:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.946+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e8146b90 con 0x7fd0e806c9f0 2024-03-06T07:30:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.946+0000 7fd0deffd640 1 -- 172.21.15.26:0/2111422534 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0003970 con 0x7fd0e806c9f0 2024-03-06T07:30:33.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.946+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e81ef0e0 con 0x7fd0e806c9f0 2024-03-06T07:30:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.947+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e804f330 con 0x7fd0e806c9f0 2024-03-06T07:30:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.947+0000 7fd0deffd640 1 -- 172.21.15.26:0/2111422534 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0e001d070 con 0x7fd0e806c9f0 2024-03-06T07:30:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.948+0000 7fd0deffd640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd0cc06b500 0x7fd0cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.948+0000 7fd0deffd640 1 -- 172.21.15.26:0/2111422534 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd0e0089a10 con 0x7fd0e806c9f0 2024-03-06T07:30:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.948+0000 7fd0ee372640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd0cc06b500 0x7fd0cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.948+0000 7fd0ee372640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd0cc06b500 0x7fd0cc06d9c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009c40 tx=0x7fd0e4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:33.952+0000 7fd0deffd640 1 -- 172.21.15.26:0/2111422534 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0e0057fa0 con 0x7fd0e806c9f0 2024-03-06T07:30:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.175+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0e80c2a90 con 0x7fd0e806c9f0 2024-03-06T07:30:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.175+0000 7fd0deffd640 1 -- 172.21.15.26:0/2111422534 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7fd0e005c040 con 0x7fd0e806c9f0 2024-03-06T07:30:34.177 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:34.177 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd0cc06b500 msgr2=0x7fd0cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd0cc06b500 0x7fd0cc06d9c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009c40 tx=0x7fd0e4009190 comp rx=0 tx=0).stop 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0e806c9f0 msgr2=0x7fd0e8138ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0e806c9f0 0x7fd0e8138ce0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80c2f40 tx=0x7fd0e000fb40 comp rx=0 tx=0).stop 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 shutdown_connections 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd0cc06b500 0x7fd0cc06d9c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0e806c9f0 0x7fd0e8138ce0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0e81470e0 0x7fd0e81387a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 --2- 172.21.15.26:0/2111422534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d4004800 0x7fd0e81ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 >> 172.21.15.26:0/2111422534 conn(0x7fd0e806b1a0 msgr2=0x7fd0e806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 shutdown_connections 2024-03-06T07:30:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:34.178+0000 7fd0f05d6640 1 -- 172.21.15.26:0/2111422534 wait complete. 2024-03-06T07:30:34.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:34.594 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:35.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.695+0000 7fe7f4a76640 1 Processor -- start 2024-03-06T07:30:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.695+0000 7fe7f4a76640 1 -- start start 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7f4a76640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7f00c5480 0x7fe7f0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7f4a76640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe7f00c5860 0x7fe7f006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7f4a76640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f00c5cc0 con 0x7fe7f01586f0 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7f4a76640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f006df40 con 0x7fe7f00c5860 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7f4a76640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f01591c0 con 0x7fe7f00c5480 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe7f00c5860 0x7fe7f006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7eed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe7f01586f0 0x7fe7f00724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57962/0 (socket says 172.21.15.26:57962) 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe7f00c5860 0x7fe7f006f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59810/0 (socket says 172.21.15.26:59810) 2024-03-06T07:30:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7eed9d640 1 -- 172.21.15.26:0/2053558863 learned_addr learned my addr 172.21.15.26:0/2053558863 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7ee59c640 1 --2- 172.21.15.26:0/2053558863 >> v2:172.21.15.117:3300/0 conn(0x7fe7f00c5480 0x7fe7f0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7e7fff640 1 -- 172.21.15.26:0/2053558863 >> v2:172.21.15.117:3300/0 conn(0x7fe7f00c5480 msgr2=0x7fe7f0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7e7fff640 1 --2- 172.21.15.26:0/2053558863 >> v2:172.21.15.117:3300/0 conn(0x7fe7f00c5480 0x7fe7f0155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7e7fff640 1 -- 172.21.15.26:0/2053558863 >> v1:172.21.15.26:6789/0 conn(0x7fe7f01586f0 legacy=0x7fe7f00724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7e7fff640 1 -- 172.21.15.26:0/2053558863 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f01594a0 con 0x7fe7f00c5860 2024-03-06T07:30:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.696+0000 7fe7edd9b640 1 -- 172.21.15.26:0/2053558863 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150032625 0 0) 0x7fe7f00c5cc0 con 0x7fe7f01586f0 2024-03-06T07:30:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.697+0000 7fe7e7fff640 1 --2- 172.21.15.26:0/2053558863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5860 0x7fe7f006f6a0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00c1cf0 tx=0x7fe7d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d299e16b9ed89416 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7edd9b640 1 -- 172.21.15.26:0/2053558863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80315a0 con 0x7fe7f00c5860 2024-03-06T07:30:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7edd9b640 1 -- 172.21.15.26:0/2053558863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d8037070 con 0x7fe7f00c5860 2024-03-06T07:30:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7edd9b640 1 -- 172.21.15.26:0/2053558863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8031960 con 0x7fe7f00c5860 2024-03-06T07:30:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 -- 172.21.15.26:0/2053558863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5860 msgr2=0x7fe7f006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/2053558863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5860 0x7fe7f006f6a0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00c1cf0 tx=0x7fe7d802b840 comp rx=0 tx=0).stop 2024-03-06T07:30:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 -- 172.21.15.26:0/2053558863 shutdown_connections 2024-03-06T07:30:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/2053558863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5860 0x7fe7f006f6a0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/2053558863 >> v2:172.21.15.117:3300/0 conn(0x7fe7f00c5480 0x7fe7f0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 -- 172.21.15.26:0/2053558863 >> 172.21.15.26:0/2053558863 conn(0x7fe7f006b1a0 msgr2=0x7fe7f006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 -- 172.21.15.26:0/2053558863 shutdown_connections 2024-03-06T07:30:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.698+0000 7fe7f4a76640 1 -- 172.21.15.26:0/2053558863 wait complete. 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.699+0000 7fe7f4a76640 1 Processor -- start 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.699+0000 7fe7f4a76640 1 -- start start 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7f4a76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5480 0x7fe7f01eb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7f4a76640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7f00c5860 0x7fe7f00ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7f4a76640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f01586f0 0x7fe7f014ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7f4a76640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0152a20 con 0x7fe7f00c5480 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7f4a76640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0155680 con 0x7fe7f01586f0 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7f4a76640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0155050 con 0x7fe7f00c5860 2024-03-06T07:30:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5480 0x7fe7f01eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5480 0x7fe7f01eb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59814/0 (socket says 172.21.15.26:59814) 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7ee59c640 1 -- 172.21.15.26:0/121902201 learned_addr learned my addr 172.21.15.26:0/121902201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7e7fff640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7f00c5860 0x7fe7f00ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7eed9d640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f01586f0 0x7fe7f014ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7ee59c640 1 -- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7f00c5860 msgr2=0x7fe7f00ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7ee59c640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7f00c5860 0x7fe7f00ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.700+0000 7fe7ee59c640 1 -- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f01586f0 msgr2=0x7fe7f014ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7ee59c640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f01586f0 0x7fe7f014ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7ee59c640 1 -- 172.21.15.26:0/121902201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f014f710 con 0x7fe7f00c5480 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7ee59c640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5480 0x7fe7f01eb520 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e870 tx=0x7fe7e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7e6ffd640 1 -- 172.21.15.26:0/121902201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0017070 con 0x7fe7f00c5480 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7e6ffd640 1 -- 172.21.15.26:0/121902201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e000c940 con 0x7fe7f00c5480 2024-03-06T07:30:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f014f9f0 con 0x7fe7f00c5480 2024-03-06T07:30:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7e6ffd640 1 -- 172.21.15.26:0/121902201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e000cd30 con 0x7fe7f00c5480 2024-03-06T07:30:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.701+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f0154af0 con 0x7fe7f00c5480 2024-03-06T07:30:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.702+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f00c5c40 con 0x7fe7f00c5480 2024-03-06T07:30:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.702+0000 7fe7e6ffd640 1 -- 172.21.15.26:0/121902201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7e001a070 con 0x7fe7f00c5480 2024-03-06T07:30:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.703+0000 7fe7e6ffd640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7d006b4d0 0x7fe7d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.703+0000 7fe7e6ffd640 1 -- 172.21.15.26:0/121902201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe7e008a570 con 0x7fe7f00c5480 2024-03-06T07:30:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.703+0000 7fe7e7fff640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7d006b4d0 0x7fe7d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.704+0000 7fe7e7fff640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7d006b4d0 0x7fe7d006d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8036000 tx=0x7fe7d803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.707+0000 7fe7e6ffd640 1 -- 172.21.15.26:0/121902201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe7e0058ba0 con 0x7fe7f00c5480 2024-03-06T07:30:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.948+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7f004f330 con 0x7fe7f00c5480 2024-03-06T07:30:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.948+0000 7fe7e6ffd640 1 -- 172.21.15.26:0/121902201 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7fe7e005cc40 con 0x7fe7f00c5480 2024-03-06T07:30:35.950 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:35.950 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7d006b4d0 msgr2=0x7fe7d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7d006b4d0 0x7fe7d006d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8036000 tx=0x7fe7d803d040 comp rx=0 tx=0).stop 2024-03-06T07:30:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5480 msgr2=0x7fe7f01eb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5480 0x7fe7f01eb520 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e870 tx=0x7fe7e000ed30 comp rx=0 tx=0).stop 2024-03-06T07:30:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 shutdown_connections 2024-03-06T07:30:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe7d006b4d0 0x7fe7d006d990 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f01586f0 0x7fe7f014ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7f00c5860 0x7fe7f00ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 --2- 172.21.15.26:0/121902201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f00c5480 0x7fe7f01eb520 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 >> 172.21.15.26:0/121902201 conn(0x7fe7f006b1a0 msgr2=0x7fe7f00c3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 shutdown_connections 2024-03-06T07:30:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:35.950+0000 7fe7f4a76640 1 -- 172.21.15.26:0/121902201 wait complete. 2024-03-06T07:30:36.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:36.365 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:37.366 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:37.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.466+0000 7f6800866640 1 Processor -- start 2024-03-06T07:30:37.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.466+0000 7f6800866640 1 -- start start 2024-03-06T07:30:37.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f6800866640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f67fc1527b0 0x7f67fc159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:37.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f6800866640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f67fc0c88b0 0x7f67fc0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:37.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f6800866640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc0c7c60 con 0x7f67fc0c7180 2024-03-06T07:30:37.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f6800866640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67fc159640 con 0x7f67fc0c88b0 2024-03-06T07:30:37.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f6800866640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67fc0c8060 con 0x7f67fc1527b0 2024-03-06T07:30:37.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f67fc0c88b0 0x7f67fc0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f67fc0c88b0 0x7f67fc0c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35136/0 (socket says 172.21.15.26:35136) 2024-03-06T07:30:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fad9d640 1 -- 172.21.15.26:0/4052176764 learned_addr learned my addr 172.21.15.26:0/4052176764 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fa59c640 1 --2- 172.21.15.26:0/4052176764 >> v2:172.21.15.117:3300/0 conn(0x7f67fc1527b0 0x7f67fc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fad9d640 1 -- 172.21.15.26:0/4052176764 >> v2:172.21.15.117:3300/0 conn(0x7f67fc1527b0 msgr2=0x7f67fc159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fad9d640 1 --2- 172.21.15.26:0/4052176764 >> v2:172.21.15.117:3300/0 conn(0x7f67fc1527b0 0x7f67fc159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fad9d640 1 -- 172.21.15.26:0/4052176764 >> v1:172.21.15.117:6789/0 conn(0x7f67fc0c7180 legacy=0x7f67fc155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.467+0000 7f67fad9d640 1 -- 172.21.15.26:0/4052176764 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0c76a0 con 0x7f67fc0c88b0 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.468+0000 7f67fad9d640 1 --2- 172.21.15.26:0/4052176764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc0c88b0 0x7f67fc0c5330 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0cba80 tx=0x7f67f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f969f46850b5f1ae server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.468+0000 7f67f9d9b640 1 -- 172.21.15.26:0/4052176764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f402ca80 con 0x7f67fc0c88b0 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.468+0000 7f67f9d9b640 1 -- 172.21.15.26:0/4052176764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f402cc20 con 0x7f67fc0c88b0 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.468+0000 7f67f9d9b640 1 -- 172.21.15.26:0/4052176764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f40343f0 con 0x7f67fc0c88b0 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 -- 172.21.15.26:0/4052176764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc0c88b0 msgr2=0x7f67fc0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 --2- 172.21.15.26:0/4052176764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc0c88b0 0x7f67fc0c5330 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0cba80 tx=0x7f67f402bee0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 -- 172.21.15.26:0/4052176764 shutdown_connections 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 --2- 172.21.15.26:0/4052176764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc0c88b0 0x7f67fc0c5330 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 --2- 172.21.15.26:0/4052176764 >> v2:172.21.15.117:3300/0 conn(0x7f67fc1527b0 0x7f67fc159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 -- 172.21.15.26:0/4052176764 >> 172.21.15.26:0/4052176764 conn(0x7f67fc0ac560 msgr2=0x7f67fc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 -- 172.21.15.26:0/4052176764 shutdown_connections 2024-03-06T07:30:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.469+0000 7f6800866640 1 -- 172.21.15.26:0/4052176764 wait complete. 2024-03-06T07:30:37.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.470+0000 7f6800866640 1 Processor -- start 2024-03-06T07:30:37.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.470+0000 7f6800866640 1 -- start start 2024-03-06T07:30:37.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f6800866640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67fc0c7180 0x7f67fc14eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:37.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f6800866640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc0c88b0 0x7f67fc156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:37.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f6800866640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc1527b0 0x7f67fc0c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f6800866640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc155600 con 0x7f67fc1527b0 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f6800866640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc155320 con 0x7f67fc0c88b0 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f6800866640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc152b90 con 0x7f67fc0c7180 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc1527b0 0x7f67fc0c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc1527b0 0x7f67fc0c1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35138/0 (socket says 172.21.15.26:35138) 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 -- 172.21.15.26:0/887706665 learned_addr learned my addr 172.21.15.26:0/887706665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fa59c640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67fc0c7180 0x7f67fc14eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67f3fff640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc0c88b0 0x7f67fc156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 -- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67fc0c7180 msgr2=0x7f67fc14eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67fc0c7180 0x7f67fc14eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 -- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc0c88b0 msgr2=0x7f67fc156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc0c88b0 0x7f67fc156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 -- 172.21.15.26:0/887706665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc1eb7b0 con 0x7f67fc1527b0 2024-03-06T07:30:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fa59c640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67fc0c7180 0x7f67fc14eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67f3fff640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc0c88b0 0x7f67fc156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.471+0000 7f67fad9d640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc1527b0 0x7f67fc0c1f00 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f67f4036040 tx=0x7f67f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.472+0000 7f67f2ffd640 1 -- 172.21.15.26:0/887706665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4043020 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.472+0000 7f67f2ffd640 1 -- 172.21.15.26:0/887706665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f403e450 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.472+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc14dee0 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.472+0000 7f67f2ffd640 1 -- 172.21.15.26:0/887706665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4009510 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.472+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc14e4a0 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.473+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc04f330 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.473+0000 7f67f2ffd640 1 -- 172.21.15.26:0/887706665 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67f4046020 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.473+0000 7f67f2ffd640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67dc06b4d0 0x7f67dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.474+0000 7f67f2ffd640 1 -- 172.21.15.26:0/887706665 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f67f40acb80 con 0x7f67fc1527b0 2024-03-06T07:30:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.474+0000 7f67fa59c640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67dc06b4d0 0x7f67dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:37.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.474+0000 7f67fa59c640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67dc06b4d0 0x7f67dc06d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0c3aa0 tx=0x7f67ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:37.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.482+0000 7f67f2ffd640 1 -- 172.21.15.26:0/887706665 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f67f404d030 con 0x7f67fc1527b0 2024-03-06T07:30:37.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.703+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67fc0c1840 con 0x7f67fc1527b0 2024-03-06T07:30:37.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.703+0000 7f67f2ffd640 1 -- 172.21.15.26:0/887706665 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f67f407b920 con 0x7f67fc1527b0 2024-03-06T07:30:37.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:37.705 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:37.705 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67dc06b4d0 msgr2=0x7f67dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67dc06b4d0 0x7f67dc06d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0c3aa0 tx=0x7f67ec008040 comp rx=0 tx=0).stop 2024-03-06T07:30:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc1527b0 msgr2=0x7f67fc0c1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc1527b0 0x7f67fc0c1f00 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f67f4036040 tx=0x7f67f4004a40 comp rx=0 tx=0).stop 2024-03-06T07:30:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 shutdown_connections 2024-03-06T07:30:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67dc06b4d0 0x7f67dc06d990 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67fc1527b0 0x7f67fc0c1f00 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc0c88b0 0x7f67fc156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 --2- 172.21.15.26:0/887706665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67fc0c7180 0x7f67fc14eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 >> 172.21.15.26:0/887706665 conn(0x7f67fc0ac560 msgr2=0x7f67fc0c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 shutdown_connections 2024-03-06T07:30:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:37.705+0000 7f6800866640 1 -- 172.21.15.26:0/887706665 wait complete. 2024-03-06T07:30:38.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:38.121 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:39.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.222+0000 7fcf85e78640 1 Processor -- start 2024-03-06T07:30:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.222+0000 7fcf85e78640 1 -- start start 2024-03-06T07:30:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.222+0000 7fcf85e78640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcf800c5480 0x7fcf80155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.222+0000 7fcf85e78640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 0x7fcf8006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.222+0000 7fcf85e78640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf800c5cc0 con 0x7fcf801586f0 2024-03-06T07:30:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.222+0000 7fcf85e78640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8006df40 con 0x7fcf800c5480 2024-03-06T07:30:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.222+0000 7fcf85e78640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf801591c0 con 0x7fcf800c5860 2024-03-06T07:30:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcf801586f0 0x7fcf800724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35960/0 (socket says 172.21.15.26:35960) 2024-03-06T07:30:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7ffff640 1 -- 172.21.15.26:0/623216625 learned_addr learned my addr 172.21.15.26:0/623216625 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7f7fe640 1 --2- 172.21.15.26:0/623216625 >> v2:172.21.15.117:3300/0 conn(0x7fcf800c5480 0x7fcf80155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7effd640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 0x7fcf8006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695139515 0 0) 0x7fcf800c5cc0 con 0x7fcf801586f0 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf68003520 con 0x7fcf801586f0 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 991976957 0 0) 0x7fcf68003520 con 0x7fcf801586f0 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 msgr2=0x7fcf8006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 0x7fcf8006f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 >> v2:172.21.15.117:3300/0 conn(0x7fcf800c5480 msgr2=0x7fcf80155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 --2- 172.21.15.26:0/623216625 >> v2:172.21.15.117:3300/0 conn(0x7fcf800c5480 0x7fcf80155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf801594a0 con 0x7fcf801586f0 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fcf780019f0 con 0x7fcf801586f0 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 >> v1:172.21.15.26:6789/0 conn(0x7fcf801586f0 legacy=0x7fcf800724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf680039b0 0x7fcf68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 0x7fcf68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8006df40 con 0x7fcf68004d40 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf801591c0 con 0x7fcf680039b0 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6800b9c0 con 0x7fcf800c5860 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7ffff640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7f7fe640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf680039b0 0x7fcf68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.223+0000 7fcf7effd640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 0x7fcf68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.224+0000 7fcf7ffff640 1 -- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 msgr2=0x7fcf68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.224+0000 7fcf7ffff640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800c5860 0x7fcf68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.224+0000 7fcf7ffff640 1 -- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf680039b0 msgr2=0x7fcf68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.224+0000 7fcf7ffff640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf680039b0 0x7fcf68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.224+0000 7fcf7ffff640 1 -- 172.21.15.26:0/623216625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68003520 con 0x7fcf68004d40 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.224+0000 7fcf7ffff640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf680051c0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fcf780013d0 tx=0x7fcf68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6cac9e47c15ef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.224+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800a4e0 con 0x7fcf68004d40 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78034650 con 0x7fcf68004d40 2024-03-06T07:30:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf7e7fc640 1 -- 172.21.15.26:0/623216625 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78034a40 con 0x7fcf68004d40 2024-03-06T07:30:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf85e78640 1 -- 172.21.15.26:0/623216625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 msgr2=0x7fcf680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf85e78640 1 --2- 172.21.15.26:0/623216625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf680051c0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fcf780013d0 tx=0x7fcf68005830 comp rx=0 tx=0).stop 2024-03-06T07:30:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf7ffff640 1 -- 172.21.15.26:0/623216625 reap_dead start 2024-03-06T07:30:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf85e78640 1 -- 172.21.15.26:0/623216625 shutdown_connections 2024-03-06T07:30:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf85e78640 1 -- 172.21.15.26:0/623216625 >> 172.21.15.26:0/623216625 conn(0x7fcf8006b1a0 msgr2=0x7fcf8006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf85e78640 1 -- 172.21.15.26:0/623216625 shutdown_connections 2024-03-06T07:30:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.225+0000 7fcf85e78640 1 -- 172.21.15.26:0/623216625 wait complete. 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.226+0000 7fcf85e78640 1 Processor -- start 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.226+0000 7fcf85e78640 1 -- start start 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf85e78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf800ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf85e78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf801df1c0 0x7fcf800aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf85e78640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800abf90 0x7fcf80156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf85e78640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80152a20 con 0x7fcf68004d40 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf85e78640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80155680 con 0x7fcf801df1c0 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf85e78640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80155050 con 0x7fcf800abf90 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf800ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf800ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35158/0 (socket says 172.21.15.26:35158) 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 -- 172.21.15.26:0/1997113198 learned_addr learned my addr 172.21.15.26:0/1997113198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7effd640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf801df1c0 0x7fcf800aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7ffff640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800abf90 0x7fcf80156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 -- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800abf90 msgr2=0x7fcf80156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800abf90 0x7fcf80156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 -- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf801df1c0 msgr2=0x7fcf800aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf801df1c0 0x7fcf800aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.227+0000 7fcf7f7fe640 1 -- 172.21.15.26:0/1997113198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf801ee9e0 con 0x7fcf68004d40 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.228+0000 7fcf7f7fe640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf800ad030 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fcf70011990 tx=0x7fcf70011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.228+0000 7fcf67fff640 1 -- 172.21.15.26:0/1997113198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000fad0 con 0x7fcf68004d40 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.228+0000 7fcf67fff640 1 -- 172.21.15.26:0/1997113198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7000fc70 con 0x7fcf68004d40 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.228+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf801eecc0 con 0x7fcf68004d40 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.228+0000 7fcf67fff640 1 -- 172.21.15.26:0/1997113198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70014440 con 0x7fcf68004d40 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.228+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf80157bc0 con 0x7fcf68004d40 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.229+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8004f330 con 0x7fcf68004d40 2024-03-06T07:30:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.230+0000 7fcf67fff640 1 -- 172.21.15.26:0/1997113198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf70012040 con 0x7fcf68004d40 2024-03-06T07:30:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.230+0000 7fcf67fff640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.230+0000 7fcf67fff640 1 -- 172.21.15.26:0/1997113198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fcf7008aae0 con 0x7fcf68004d40 2024-03-06T07:30:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.231+0000 7fcf7effd640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.231+0000 7fcf7effd640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fcf800c2060 tx=0x7fcf74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.235+0000 7fcf67fff640 1 -- 172.21.15.26:0/1997113198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcf70019030 con 0x7fcf68004d40 2024-03-06T07:30:39.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.456+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf8006e0c0 con 0x7fcf68004d40 2024-03-06T07:30:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.456+0000 7fcf67fff640 1 -- 172.21.15.26:0/1997113198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7fcf70059760 con 0x7fcf68004d40 2024-03-06T07:30:39.458 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:39.458 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.458+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 msgr2=0x7fcf6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.458+0000 7fcf85e78640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fcf800c2060 tx=0x7fcf74008040 comp rx=0 tx=0).stop 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.458+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 msgr2=0x7fcf800ad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.458+0000 7fcf85e78640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf800ad030 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fcf70011990 tx=0x7fcf70011e50 comp rx=0 tx=0).stop 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 shutdown_connections 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf6c06b4d0 0x7fcf6c06d990 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf800abf90 0x7fcf80156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf801df1c0 0x7fcf800aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 --2- 172.21.15.26:0/1997113198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf68004d40 0x7fcf800ad030 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 >> 172.21.15.26:0/1997113198 conn(0x7fcf8006b1a0 msgr2=0x7fcf8006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 shutdown_connections 2024-03-06T07:30:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:39.459+0000 7fcf85e78640 1 -- 172.21.15.26:0/1997113198 wait complete. 2024-03-06T07:30:39.874 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:39.874 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:40.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.976+0000 7fdf859f7640 1 Processor -- start 2024-03-06T07:30:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.976+0000 7fdf859f7640 1 -- start start 2024-03-06T07:30:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf859f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf8014f360 0x7fdf80155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf859f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf80158b60 0x7fdf80158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf859f7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800c73d0 con 0x7fdf800c88c0 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf859f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf800c8dd0 con 0x7fdf80158b60 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf859f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800be2e0 con 0x7fdf8014f360 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf7effd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf8014f360 0x7fdf80155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf7f7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf80158b60 0x7fdf80158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf7f7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf80158b60 0x7fdf80158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51004/0 (socket says 172.21.15.26:51004) 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf7e7fc640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fdf800c88c0 0x7fdf800c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:40602/0 (socket says 172.21.15.26:40602) 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf7effd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf8014f360 0x7fdf80155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46698/0 (socket says 172.21.15.26:46698) 2024-03-06T07:30:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.977+0000 7fdf7f7fe640 1 -- 172.21.15.26:0/2887838648 learned_addr learned my addr 172.21.15.26:0/2887838648 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.978+0000 7fdf7effd640 1 -- 172.21.15.26:0/2887838648 >> v2:172.21.15.117:3300/0 conn(0x7fdf80158b60 msgr2=0x7fdf80158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.978+0000 7fdf7effd640 1 --2- 172.21.15.26:0/2887838648 >> v2:172.21.15.117:3300/0 conn(0x7fdf80158b60 0x7fdf80158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.978+0000 7fdf7effd640 1 -- 172.21.15.26:0/2887838648 >> v1:172.21.15.117:6789/0 conn(0x7fdf800c88c0 legacy=0x7fdf800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.978+0000 7fdf7dffb640 1 -- 172.21.15.26:0/2887838648 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011083382 0 0) 0x7fdf800c73d0 con 0x7fdf800c88c0 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.978+0000 7fdf7effd640 1 -- 172.21.15.26:0/2887838648 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800c7670 con 0x7fdf8014f360 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.978+0000 7fdf7f7fe640 1 --2- 172.21.15.26:0/2887838648 >> v2:172.21.15.117:3300/0 conn(0x7fdf80158b60 0x7fdf80158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.978+0000 7fdf7effd640 1 --2- 172.21.15.26:0/2887838648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf8014f360 0x7fdf80155c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf7802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22457e3977ae8f93 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf7dffb640 1 -- 172.21.15.26:0/2887838648 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78039070 con 0x7fdf8014f360 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf7dffb640 1 -- 172.21.15.26:0/2887838648 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7802cbc0 con 0x7fdf8014f360 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf7dffb640 1 -- 172.21.15.26:0/2887838648 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78034440 con 0x7fdf8014f360 2024-03-06T07:30:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 -- 172.21.15.26:0/2887838648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf8014f360 msgr2=0x7fdf80155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 --2- 172.21.15.26:0/2887838648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf8014f360 0x7fdf80155c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf7802be10 comp rx=0 tx=0).stop 2024-03-06T07:30:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 -- 172.21.15.26:0/2887838648 shutdown_connections 2024-03-06T07:30:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 --2- 172.21.15.26:0/2887838648 >> v2:172.21.15.117:3300/0 conn(0x7fdf80158b60 0x7fdf80158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 --2- 172.21.15.26:0/2887838648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf8014f360 0x7fdf80155c10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 -- 172.21.15.26:0/2887838648 >> 172.21.15.26:0/2887838648 conn(0x7fdf800ac570 msgr2=0x7fdf800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 -- 172.21.15.26:0/2887838648 shutdown_connections 2024-03-06T07:30:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.979+0000 7fdf859f7640 1 -- 172.21.15.26:0/2887838648 wait complete. 2024-03-06T07:30:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.980+0000 7fdf859f7640 1 Processor -- start 2024-03-06T07:30:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.980+0000 7fdf859f7640 1 -- start start 2024-03-06T07:30:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf859f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf800c88c0 0x7fdf800ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf859f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf8014f360 0x7fdf800c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf859f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf80158b60 0x7fdf801eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf859f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800c5a50 con 0x7fdf8014f360 2024-03-06T07:30:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf859f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800c7c30 con 0x7fdf800c88c0 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf859f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800c5bd0 con 0x7fdf80158b60 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf7e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf8014f360 0x7fdf800c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf7e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf8014f360 0x7fdf800c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35162/0 (socket says 172.21.15.26:35162) 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf7e7fc640 1 -- 172.21.15.26:0/1102208707 learned_addr learned my addr 172.21.15.26:0/1102208707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf7effd640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf800c88c0 0x7fdf800ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.981+0000 7fdf7f7fe640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf80158b60 0x7fdf801eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf7e7fc640 1 -- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf80158b60 msgr2=0x7fdf801eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf7e7fc640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf80158b60 0x7fdf801eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf7e7fc640 1 -- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf800c88c0 msgr2=0x7fdf800ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf7e7fc640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf800c88c0 0x7fdf800ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf7e7fc640 1 -- 172.21.15.26:0/1102208707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800aed30 con 0x7fdf8014f360 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf7e7fc640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf8014f360 0x7fdf800c1dc0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c410 tx=0x7fdf6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf77fff640 1 -- 172.21.15.26:0/1102208707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf680074e0 con 0x7fdf8014f360 2024-03-06T07:30:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf77fff640 1 -- 172.21.15.26:0/1102208707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf680047e0 con 0x7fdf8014f360 2024-03-06T07:30:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf800add50 con 0x7fdf8014f360 2024-03-06T07:30:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf77fff640 1 -- 172.21.15.26:0/1102208707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf68004bd0 con 0x7fdf8014f360 2024-03-06T07:30:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.982+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf800afc90 con 0x7fdf8014f360 2024-03-06T07:30:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.983+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8004f330 con 0x7fdf8014f360 2024-03-06T07:30:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.984+0000 7fdf77fff640 1 -- 172.21.15.26:0/1102208707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf68011070 con 0x7fdf8014f360 2024-03-06T07:30:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.984+0000 7fdf77fff640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf6006b4d0 0x7fdf6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.984+0000 7fdf77fff640 1 -- 172.21.15.26:0/1102208707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdf68089e80 con 0x7fdf8014f360 2024-03-06T07:30:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.984+0000 7fdf7effd640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf6006b4d0 0x7fdf6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.985+0000 7fdf7effd640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf6006b4d0 0x7fdf6006d990 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdf78036000 tx=0x7fdf78030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:40.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:40.988+0000 7fdf77fff640 1 -- 172.21.15.26:0/1102208707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf680584b0 con 0x7fdf8014f360 2024-03-06T07:30:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.207+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf8006cda0 con 0x7fdf8014f360 2024-03-06T07:30:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.207+0000 7fdf77fff640 1 -- 172.21.15.26:0/1102208707 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7fdf6805c550 con 0x7fdf8014f360 2024-03-06T07:30:41.209 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:41.209 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:41.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf6006b4d0 msgr2=0x7fdf6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf6006b4d0 0x7fdf6006d990 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdf78036000 tx=0x7fdf78030000 comp rx=0 tx=0).stop 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf8014f360 msgr2=0x7fdf800c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf8014f360 0x7fdf800c1dc0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c410 tx=0x7fdf6800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 shutdown_connections 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdf6006b4d0 0x7fdf6006d990 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf80158b60 0x7fdf801eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf8014f360 0x7fdf800c1dc0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 --2- 172.21.15.26:0/1102208707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf800c88c0 0x7fdf800ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 >> 172.21.15.26:0/1102208707 conn(0x7fdf800ac570 msgr2=0x7fdf8014dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 shutdown_connections 2024-03-06T07:30:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:41.209+0000 7fdf859f7640 1 -- 172.21.15.26:0/1102208707 wait complete. 2024-03-06T07:30:41.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:41.623 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:42.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.726+0000 7f15c6a64640 1 Processor -- start 2024-03-06T07:30:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.726+0000 7f15c6a64640 1 -- start start 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c6a64640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15c014f790 0x7f15c01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c6a64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15c00c1aa0 0x7f15c00c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c6a64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c00c5c60 0x7f15c0155a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c6a64640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00c2680 con 0x7f15c00c1aa0 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c6a64640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00c28d0 con 0x7f15c014f790 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c6a64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00c2bb0 con 0x7f15c00c5c60 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15c00c1aa0 0x7f15c00c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15c00c1aa0 0x7f15c00c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35164/0 (socket says 172.21.15.26:35164) 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 -- 172.21.15.26:0/2363973574 learned_addr learned my addr 172.21.15.26:0/2363973574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c4800640 1 --2- 172.21.15.26:0/2363973574 >> v2:172.21.15.117:3300/0 conn(0x7f15c014f790 0x7f15c01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15c5001640 1 --2- 172.21.15.26:0/2363973574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c00c5c60 0x7f15c0155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 -- 172.21.15.26:0/2363973574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c00c5c60 msgr2=0x7f15c0155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 --2- 172.21.15.26:0/2363973574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c00c5c60 0x7f15c0155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 -- 172.21.15.26:0/2363973574 >> v2:172.21.15.117:3300/0 conn(0x7f15c014f790 msgr2=0x7f15c01529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 --2- 172.21.15.26:0/2363973574 >> v2:172.21.15.117:3300/0 conn(0x7f15c014f790 0x7f15c01529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.727+0000 7f15b7fff640 1 -- 172.21.15.26:0/2363973574 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00c2e90 con 0x7f15c00c1aa0 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.728+0000 7f15b7fff640 1 --2- 172.21.15.26:0/2363973574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c1aa0 0x7f15c00c5720 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f15c00c3170 tx=0x7f15b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fe6401eab9b9f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.728+0000 7f15b77fe640 1 -- 172.21.15.26:0/2363973574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0035020 con 0x7f15c00c1aa0 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.728+0000 7f15b77fe640 1 -- 172.21.15.26:0/2363973574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0031870 con 0x7f15c00c1aa0 2024-03-06T07:30:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.728+0000 7f15b77fe640 1 -- 172.21.15.26:0/2363973574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0031c40 con 0x7f15c00c1aa0 2024-03-06T07:30:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 -- 172.21.15.26:0/2363973574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c1aa0 msgr2=0x7f15c00c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 --2- 172.21.15.26:0/2363973574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c1aa0 0x7f15c00c5720 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f15c00c3170 tx=0x7f15b002b910 comp rx=0 tx=0).stop 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 -- 172.21.15.26:0/2363973574 shutdown_connections 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 --2- 172.21.15.26:0/2363973574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c00c5c60 0x7f15c0155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 --2- 172.21.15.26:0/2363973574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c1aa0 0x7f15c00c5720 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 --2- 172.21.15.26:0/2363973574 >> v2:172.21.15.117:3300/0 conn(0x7f15c014f790 0x7f15c01529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 -- 172.21.15.26:0/2363973574 >> 172.21.15.26:0/2363973574 conn(0x7f15c00c6710 msgr2=0x7f15c00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 -- 172.21.15.26:0/2363973574 shutdown_connections 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.729+0000 7f15c6a64640 1 -- 172.21.15.26:0/2363973574 wait complete. 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.731+0000 7f15c6a64640 1 Processor -- start 2024-03-06T07:30:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.731+0000 7f15c6a64640 1 -- start start 2024-03-06T07:30:42.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15c6a64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15c00c1aa0 0x7f15c014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15c6a64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c5c60 0x7f15c0153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15c6a64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c014f790 0x7f15c0149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15c6a64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c006cc00 con 0x7f15c00c5c60 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15c6a64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00c0d60 con 0x7f15c00c1aa0 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15c6a64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c006cd80 con 0x7f15c014f790 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c5c60 0x7f15c0153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c5c60 0x7f15c0153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35168/0 (socket says 172.21.15.26:35168) 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.732+0000 7f15b7fff640 1 -- 172.21.15.26:0/3289093653 learned_addr learned my addr 172.21.15.26:0/3289093653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15c4800640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15c00c1aa0 0x7f15c014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b7fff640 1 -- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c014f790 msgr2=0x7f15c0149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15c5001640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c014f790 0x7f15c0149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b7fff640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c014f790 0x7f15c0149980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b7fff640 1 -- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15c00c1aa0 msgr2=0x7f15c014acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b7fff640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15c00c1aa0 0x7f15c014acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b7fff640 1 -- 172.21.15.26:0/3289093653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c01df410 con 0x7f15c00c5c60 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15c4800640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15c00c1aa0 0x7f15c014acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b7fff640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c5c60 0x7f15c0153590 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f15b002b950 tx=0x7f15b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b57fa640 1 -- 172.21.15.26:0/3289093653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0040070 con 0x7f15c00c5c60 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b57fa640 1 -- 172.21.15.26:0/3289093653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0030d00 con 0x7f15c00c5c60 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c014a700 con 0x7f15c00c5c60 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15b57fa640 1 -- 172.21.15.26:0/3289093653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0035040 con 0x7f15c00c5c60 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.733+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c0153ad0 con 0x7f15c00c5c60 2024-03-06T07:30:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.734+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c004f330 con 0x7f15c00c5c60 2024-03-06T07:30:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.735+0000 7f15b57fa640 1 -- 172.21.15.26:0/3289093653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15b004b060 con 0x7f15c00c5c60 2024-03-06T07:30:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.735+0000 7f15b57fa640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15ac06b4d0 0x7f15ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.735+0000 7f15b57fa640 1 -- 172.21.15.26:0/3289093653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f15b00ac210 con 0x7f15c00c5c60 2024-03-06T07:30:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.735+0000 7f15c4800640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15ac06b4d0 0x7f15ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.736+0000 7f15c4800640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15ac06b4d0 0x7f15ac06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fd0 tx=0x7f15bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:42.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.739+0000 7f15b57fa640 1 -- 172.21.15.26:0/3289093653 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15b007ae90 con 0x7f15c00c5c60 2024-03-06T07:30:42.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.959+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15c006c770 con 0x7f15c00c5c60 2024-03-06T07:30:42.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.960+0000 7f15b57fa640 1 -- 172.21.15.26:0/3289093653 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f15b007a790 con 0x7f15c00c5c60 2024-03-06T07:30:42.962 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:42.962 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:42.962 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15ac06b4d0 msgr2=0x7f15ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15ac06b4d0 0x7f15ac06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fd0 tx=0x7f15bc008040 comp rx=0 tx=0).stop 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c5c60 msgr2=0x7f15c0153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c5c60 0x7f15c0153590 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f15b002b950 tx=0x7f15b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 shutdown_connections 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f15ac06b4d0 0x7f15ac06d990 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15c014f790 0x7f15c0149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c00c5c60 0x7f15c0153590 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 --2- 172.21.15.26:0/3289093653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15c00c1aa0 0x7f15c014acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 >> 172.21.15.26:0/3289093653 conn(0x7f15c00c6710 msgr2=0x7f15c00c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 shutdown_connections 2024-03-06T07:30:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:42.962+0000 7f15c6a64640 1 -- 172.21.15.26:0/3289093653 wait complete. 2024-03-06T07:30:43.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:43.377 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:43.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:43.391 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:44.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:44.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.480+0000 7f5fc497a640 1 Processor -- start 2024-03-06T07:30:44.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.480+0000 7f5fc497a640 1 -- start start 2024-03-06T07:30:44.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.480+0000 7f5fc497a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fc014f360 0x7f5fc0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:44.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.480+0000 7f5fc497a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:44.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.480+0000 7f5fc497a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc00c73d0 con 0x7f5fc00c88c0 2024-03-06T07:30:44.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.480+0000 7f5fc497a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00c8dd0 con 0x7f5fc014f360 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.480+0000 7f5fc497a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00be2e0 con 0x7f5fc0158b60 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbe59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fc014f360 0x7f5fc0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbe59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fc014f360 0x7f5fc0155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35176/0 (socket says 172.21.15.26:35176) 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbe59c640 1 -- 172.21.15.26:0/334660960 learned_addr learned my addr 172.21.15.26:0/334660960 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbed9d640 1 --2- 172.21.15.26:0/334660960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbe59c640 1 -- 172.21.15.26:0/334660960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 msgr2=0x7f5fc0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbe59c640 1 --2- 172.21.15.26:0/334660960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc0158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbe59c640 1 -- 172.21.15.26:0/334660960 >> v1:172.21.15.117:6789/0 conn(0x7f5fc00c88c0 legacy=0x7f5fc00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbe59c640 1 -- 172.21.15.26:0/334660960 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00c7670 con 0x7f5fc014f360 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.481+0000 7f5fbed9d640 1 --2- 172.21.15.26:0/334660960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fbe59c640 1 --2- 172.21.15.26:0/334660960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc014f360 0x7f5fc0155c10 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009870 tx=0x7f5fb002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f177d92aa0c94f9b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fbdd9b640 1 -- 172.21.15.26:0/334660960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00317c0 con 0x7f5fc014f360 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fbdd9b640 1 -- 172.21.15.26:0/334660960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb0037070 con 0x7f5fc014f360 2024-03-06T07:30:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fbdd9b640 1 -- 172.21.15.26:0/334660960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0031b80 con 0x7f5fc014f360 2024-03-06T07:30:44.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 -- 172.21.15.26:0/334660960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc014f360 msgr2=0x7f5fc0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:44.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 --2- 172.21.15.26:0/334660960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc014f360 0x7f5fc0155c10 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009870 tx=0x7f5fb002b860 comp rx=0 tx=0).stop 2024-03-06T07:30:44.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 -- 172.21.15.26:0/334660960 shutdown_connections 2024-03-06T07:30:44.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 --2- 172.21.15.26:0/334660960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 --2- 172.21.15.26:0/334660960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc014f360 0x7f5fc0155c10 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 -- 172.21.15.26:0/334660960 >> 172.21.15.26:0/334660960 conn(0x7f5fc00ac570 msgr2=0x7f5fc00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 -- 172.21.15.26:0/334660960 shutdown_connections 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.482+0000 7f5fc497a640 1 -- 172.21.15.26:0/334660960 wait complete. 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 Processor -- start 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 -- start start 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc00c88c0 0x7f5fc014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fc014f360 0x7f5fc01ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc01ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00c5a50 con 0x7f5fc00c88c0 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00c7c30 con 0x7f5fc014f360 2024-03-06T07:30:44.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fc497a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00c5bd0 con 0x7f5fc0158b60 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fbe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc00c88c0 0x7f5fc014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fbe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc00c88c0 0x7f5fc014bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35184/0 (socket says 172.21.15.26:35184) 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.484+0000 7f5fbe59c640 1 -- 172.21.15.26:0/2518610261 learned_addr learned my addr 172.21.15.26:0/2518610261 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fb7fff640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fc014f360 0x7f5fc01ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fbed9d640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc01ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fbe59c640 1 -- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 msgr2=0x7f5fc01ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fbe59c640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc01ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fbe59c640 1 -- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fc014f360 msgr2=0x7f5fc01ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fbe59c640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fc014f360 0x7f5fc01ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fbe59c640 1 -- 172.21.15.26:0/2518610261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc01eee90 con 0x7f5fc00c88c0 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fbe59c640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc00c88c0 0x7f5fc014bb30 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f5fb002b8a0 tx=0x7f5fb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fb6ffd640 1 -- 172.21.15.26:0/2518610261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0032860 con 0x7f5fc00c88c0 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fb6ffd640 1 -- 172.21.15.26:0/2518610261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb0032a20 con 0x7f5fc00c88c0 2024-03-06T07:30:44.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fb6ffd640 1 -- 172.21.15.26:0/2518610261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0032e10 con 0x7f5fc00c88c0 2024-03-06T07:30:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc00afcf0 con 0x7f5fc00c88c0 2024-03-06T07:30:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.485+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc01ec2f0 con 0x7f5fc00c88c0 2024-03-06T07:30:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.486+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc0075300 con 0x7f5fc00c88c0 2024-03-06T07:30:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.487+0000 7f5fb6ffd640 1 -- 172.21.15.26:0/2518610261 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fb004c060 con 0x7f5fc00c88c0 2024-03-06T07:30:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.487+0000 7f5fb6ffd640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fa006b4d0 0x7f5fa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.487+0000 7f5fb6ffd640 1 -- 172.21.15.26:0/2518610261 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f5fb00abdb0 con 0x7f5fc00c88c0 2024-03-06T07:30:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.487+0000 7f5fb7fff640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fa006b4d0 0x7f5fa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.488+0000 7f5fb7fff640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fa006b4d0 0x7f5fa006d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006fd0 tx=0x7f5fa8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.491+0000 7f5fb6ffd640 1 -- 172.21.15.26:0/2518610261 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5fc0075300 con 0x7f5fc00c88c0 2024-03-06T07:30:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.713+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fc004f330 con 0x7f5fc00c88c0 2024-03-06T07:30:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.713+0000 7f5fb6ffd640 1 -- 172.21.15.26:0/2518610261 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f5fb00b0070 con 0x7f5fc00c88c0 2024-03-06T07:30:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:44.715 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fa006b4d0 msgr2=0x7f5fa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fa006b4d0 0x7f5fa006d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006fd0 tx=0x7f5fa8008040 comp rx=0 tx=0).stop 2024-03-06T07:30:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc00c88c0 msgr2=0x7f5fc014bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc00c88c0 0x7f5fc014bb30 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f5fb002b8a0 tx=0x7f5fb00046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 shutdown_connections 2024-03-06T07:30:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5fa006b4d0 0x7f5fa006d990 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fc0158b60 0x7f5fc01ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fc014f360 0x7f5fc01ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 --2- 172.21.15.26:0/2518610261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fc00c88c0 0x7f5fc014bb30 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 >> 172.21.15.26:0/2518610261 conn(0x7f5fc00ac570 msgr2=0x7f5fc00ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 shutdown_connections 2024-03-06T07:30:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:44.715+0000 7f5fc497a640 1 -- 172.21.15.26:0/2518610261 wait complete. 2024-03-06T07:30:45.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:45.131 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:46.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:46.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.242+0000 7fdf0b69a640 1 Processor -- start 2024-03-06T07:30:46.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.243+0000 7fdf0b69a640 1 -- start start 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.243+0000 7fdf0b69a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 0x7fdf04155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.243+0000 7fdf0b69a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf040c73d0 con 0x7fdf040c88c0 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.243+0000 7fdf0b69a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf040be2e0 con 0x7fdf0414f360 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.243+0000 7fdf0b69a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04159700 con 0x7fdf04159270 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.243+0000 7fdf08c35640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf040c88c0 0x7fdf04158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35976/0 (socket says 172.21.15.26:35976) 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.243+0000 7fdf08c35640 1 -- 172.21.15.26:0/53616011 learned_addr learned my addr 172.21.15.26:0/53616011 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdf09c37640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 0x7fdf04155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899226541 0 0) 0x7fdf040c73d0 con 0x7fdf040c88c0 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef0003520 con 0x7fdf040c88c0 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3828577827 0 0) 0x7fdef0003520 con 0x7fdf040c88c0 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 msgr2=0x7fdf04155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 0x7fdf04155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 >> v1:172.21.15.117:6789/0 conn(0x7fdf0414f360 legacy=0x7fdf040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040c7670 con 0x7fdf040c88c0 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fdf00001670 con 0x7fdf040c88c0 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 >> v1:172.21.15.26:6789/0 conn(0x7fdf040c88c0 legacy=0x7fdf04158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdef0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef0004390 0x7fdef0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 0x7fdef0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04159700 con 0x7fdef00039e0 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef000baa0 con 0x7fdef0004390 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef000bca0 con 0x7fdf04159270 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdf09436640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdef0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdf08c35640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef0004390 0x7fdef0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.244+0000 7fdf09c37640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 0x7fdef0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdf09436640 1 -- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 msgr2=0x7fdef0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdf09436640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf04159270 0x7fdef0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdf09436640 1 -- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef0004390 msgr2=0x7fdef0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdf09436640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef0004390 0x7fdef0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdf09436640 1 -- 172.21.15.26:0/53616011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0003520 con 0x7fdef00039e0 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdf09436640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdef0003dc0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fdef8001860 tx=0x7fdef0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695f7f3a53b8f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef802f050 con 0x7fdef00039e0 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8033650 con 0x7fdef00039e0 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.245+0000 7fdef7fff640 1 -- 172.21.15.26:0/53616011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8001430 con 0x7fdef00039e0 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.246+0000 7fdf0b69a640 1 -- 172.21.15.26:0/53616011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 msgr2=0x7fdef0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.246+0000 7fdf0b69a640 1 --2- 172.21.15.26:0/53616011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdef0003dc0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fdef8001860 tx=0x7fdef0005830 comp rx=0 tx=0).stop 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.246+0000 7fdf09c37640 1 -- 172.21.15.26:0/53616011 reap_dead start 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.246+0000 7fdf0b69a640 1 -- 172.21.15.26:0/53616011 shutdown_connections 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.246+0000 7fdf0b69a640 1 -- 172.21.15.26:0/53616011 >> 172.21.15.26:0/53616011 conn(0x7fdf040ac570 msgr2=0x7fdf040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.246+0000 7fdf0b69a640 1 -- 172.21.15.26:0/53616011 shutdown_connections 2024-03-06T07:30:46.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.246+0000 7fdf0b69a640 1 -- 172.21.15.26:0/53616011 wait complete. 2024-03-06T07:30:46.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.248+0000 7fdf0b69a640 1 Processor -- start 2024-03-06T07:30:46.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.248+0000 7fdf0b69a640 1 -- start start 2024-03-06T07:30:46.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf0b69a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdf040bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf0b69a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf041ebc00 0x7fdf040c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf0b69a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf041ec080 0x7fdf040c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf0b69a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c7c30 con 0x7fdef00039e0 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf0b69a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c5a50 con 0x7fdf041ebc00 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf0b69a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c5bd0 con 0x7fdf041ec080 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdf040bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdf040bf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35212/0 (socket says 172.21.15.26:35212) 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 -- 172.21.15.26:0/3203654401 learned_addr learned my addr 172.21.15.26:0/3203654401 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf08c35640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf041ebc00 0x7fdf040c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09c37640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf041ec080 0x7fdf040c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 -- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf041ec080 msgr2=0x7fdf040c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf041ec080 0x7fdf040c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 -- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf041ebc00 msgr2=0x7fdf040c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf041ebc00 0x7fdf040c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.249+0000 7fdf09436640 1 -- 172.21.15.26:0/3203654401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0414d230 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.250+0000 7fdf09436640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdf040bf950 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fdef8001980 tx=0x7fdef8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.250+0000 7fdef5ffb640 1 -- 172.21.15.26:0/3203654401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80035f0 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.250+0000 7fdef5ffb640 1 -- 172.21.15.26:0/3203654401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8003790 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.250+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040beed0 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.250+0000 7fdef5ffb640 1 -- 172.21.15.26:0/3203654401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8003b40 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.250+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf040c0d30 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.251+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0404f330 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.251+0000 7fdef5ffb640 1 -- 172.21.15.26:0/3203654401 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdef8003ce0 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.252+0000 7fdef5ffb640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdee806b500 0x7fdee806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.252+0000 7fdef5ffb640 1 -- 172.21.15.26:0/3203654401 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdef80ac460 con 0x7fdef00039e0 2024-03-06T07:30:46.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.252+0000 7fdf08c35640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdee806b500 0x7fdee806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.253+0000 7fdf08c35640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdee806b500 0x7fdee806d9c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdf041e8550 tx=0x7fdf00000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:46.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.256+0000 7fdef5ffb640 1 -- 172.21.15.26:0/3203654401 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf040beed0 con 0x7fdef00039e0 2024-03-06T07:30:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.687+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf040c8d50 con 0x7fdef00039e0 2024-03-06T07:30:46.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.688+0000 7fdef5ffb640 1 -- 172.21.15.26:0/3203654401 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7fdef804a060 con 0x7fdef00039e0 2024-03-06T07:30:46.690 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:46.690 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:46.690 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:46.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdee806b500 msgr2=0x7fdee806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdee806b500 0x7fdee806d9c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdf041e8550 tx=0x7fdf00000f80 comp rx=0 tx=0).stop 2024-03-06T07:30:46.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 msgr2=0x7fdf040bf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdf040bf950 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fdef8001980 tx=0x7fdef8001230 comp rx=0 tx=0).stop 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 shutdown_connections 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdee806b500 0x7fdee806d9c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf041ec080 0x7fdf040c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf041ebc00 0x7fdf040c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 --2- 172.21.15.26:0/3203654401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef00039e0 0x7fdf040bf950 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 >> 172.21.15.26:0/3203654401 conn(0x7fdf040ac570 msgr2=0x7fdf0406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 shutdown_connections 2024-03-06T07:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:46.690+0000 7fdf0b69a640 1 -- 172.21.15.26:0/3203654401 wait complete. 2024-03-06T07:30:47.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:47.105 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:48.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.207+0000 7f1a66301640 1 Processor -- start 2024-03-06T07:30:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.207+0000 7f1a66301640 1 -- start start 2024-03-06T07:30:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a66301640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a600c88c0 0x7f1a60155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a66301640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a60158b60 0x7f1a60158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a66301640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a600c73d0 con 0x7f1a6014f360 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a66301640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600c8dd0 con 0x7f1a60158b60 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a66301640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600be2e0 con 0x7f1a600c88c0 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a6489e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a60158b60 0x7f1a60158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a6489e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a60158b60 0x7f1a60158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42044/0 (socket says 172.21.15.26:42044) 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a6489e640 1 -- 172.21.15.26:0/3385735352 learned_addr learned my addr 172.21.15.26:0/3385735352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.208+0000 7f1a5f7fe640 1 --2- 172.21.15.26:0/3385735352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a600c88c0 0x7f1a60155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.209+0000 7f1a6489e640 1 -- 172.21.15.26:0/3385735352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a600c88c0 msgr2=0x7f1a60155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.209+0000 7f1a6489e640 1 --2- 172.21.15.26:0/3385735352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a600c88c0 0x7f1a60155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.209+0000 7f1a6489e640 1 -- 172.21.15.26:0/3385735352 >> v1:172.21.15.117:6789/0 conn(0x7f1a6014f360 legacy=0x7f1a600c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.209+0000 7f1a6489e640 1 -- 172.21.15.26:0/3385735352 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600c7670 con 0x7f1a60158b60 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a6489e640 1 --2- 172.21.15.26:0/3385735352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60158ff0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009e50 tx=0x7f1a5802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b69b9d8270d2f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a5effd640 1 -- 172.21.15.26:0/3385735352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5802c9e0 con 0x7f1a60158b60 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a5effd640 1 -- 172.21.15.26:0/3385735352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5802cb80 con 0x7f1a60158b60 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a5effd640 1 -- 172.21.15.26:0/3385735352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a580343f0 con 0x7f1a60158b60 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 -- 172.21.15.26:0/3385735352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 msgr2=0x7f1a60158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 --2- 172.21.15.26:0/3385735352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60158ff0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009e50 tx=0x7f1a5802be40 comp rx=0 tx=0).stop 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 -- 172.21.15.26:0/3385735352 shutdown_connections 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 --2- 172.21.15.26:0/3385735352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60158ff0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 --2- 172.21.15.26:0/3385735352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a600c88c0 0x7f1a60155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 -- 172.21.15.26:0/3385735352 >> 172.21.15.26:0/3385735352 conn(0x7f1a600ac570 msgr2=0x7f1a600ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 -- 172.21.15.26:0/3385735352 shutdown_connections 2024-03-06T07:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.210+0000 7f1a66301640 1 -- 172.21.15.26:0/3385735352 wait complete. 2024-03-06T07:30:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.212+0000 7f1a66301640 1 Processor -- start 2024-03-06T07:30:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.212+0000 7f1a66301640 1 -- start start 2024-03-06T07:30:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.213+0000 7f1a66301640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a600c88c0 0x7f1a6014fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.213+0000 7f1a66301640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a6014f360 0x7f1a601eb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.213+0000 7f1a66301640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.213+0000 7f1a66301640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600c5a50 con 0x7f1a60158b60 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.213+0000 7f1a66301640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600c7c30 con 0x7f1a600c88c0 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.213+0000 7f1a66301640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600c5bd0 con 0x7f1a6014f360 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42052/0 (socket says 172.21.15.26:42052) 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 -- 172.21.15.26:0/388682526 learned_addr learned my addr 172.21.15.26:0/388682526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a5f7fe640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a6014f360 0x7f1a601eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a5ffff640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a600c88c0 0x7f1a6014fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 -- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a6014f360 msgr2=0x7f1a601eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a6014f360 0x7f1a601eb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 -- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a600c88c0 msgr2=0x7f1a6014fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a600c88c0 0x7f1a6014fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 -- 172.21.15.26:0/388682526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600c1100 con 0x7f1a60158b60 2024-03-06T07:30:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a6489e640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60157cc0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f1a58036040 tx=0x7f1a58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a5cff9640 1 -- 172.21.15.26:0/388682526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58044020 con 0x7f1a60158b60 2024-03-06T07:30:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a5cff9640 1 -- 172.21.15.26:0/388682526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58040690 con 0x7f1a60158b60 2024-03-06T07:30:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a600c13e0 con 0x7f1a60158b60 2024-03-06T07:30:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a5cff9640 1 -- 172.21.15.26:0/388682526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58040a80 con 0x7f1a60158b60 2024-03-06T07:30:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.214+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a60158200 con 0x7f1a60158b60 2024-03-06T07:30:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.215+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a600bf4e0 con 0x7f1a60158b60 2024-03-06T07:30:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.216+0000 7f1a5cff9640 1 -- 172.21.15.26:0/388682526 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a58047020 con 0x7f1a60158b60 2024-03-06T07:30:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.216+0000 7f1a5cff9640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a4c06b4d0 0x7f1a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.216+0000 7f1a5cff9640 1 -- 172.21.15.26:0/388682526 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f1a580acf90 con 0x7f1a60158b60 2024-03-06T07:30:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.216+0000 7f1a5ffff640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a4c06b4d0 0x7f1a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.217+0000 7f1a5ffff640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a4c06b4d0 0x7f1a4c06d990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000a750 tx=0x7f1a50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.225+0000 7f1a5cff9640 1 -- 172.21.15.26:0/388682526 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1a5804e030 con 0x7f1a60158b60 2024-03-06T07:30:48.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.441+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a6004f330 con 0x7f1a60158b60 2024-03-06T07:30:48.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.442+0000 7f1a5cff9640 1 -- 172.21.15.26:0/388682526 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f1a5807bc10 con 0x7f1a60158b60 2024-03-06T07:30:48.444 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:48.444 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:48.444 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:48.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a4c06b4d0 msgr2=0x7f1a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a4c06b4d0 0x7f1a4c06d990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000a750 tx=0x7f1a50008040 comp rx=0 tx=0).stop 2024-03-06T07:30:48.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 msgr2=0x7f1a60157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60157cc0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f1a58036040 tx=0x7f1a58004a40 comp rx=0 tx=0).stop 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 shutdown_connections 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1a4c06b4d0 0x7f1a4c06d990 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a60158b60 0x7f1a60157cc0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a6014f360 0x7f1a601eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 --2- 172.21.15.26:0/388682526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a600c88c0 0x7f1a6014fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.443+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 >> 172.21.15.26:0/388682526 conn(0x7f1a600ac570 msgr2=0x7f1a6006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.444+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 shutdown_connections 2024-03-06T07:30:48.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:48.444+0000 7f1a66301640 1 -- 172.21.15.26:0/388682526 wait complete. 2024-03-06T07:30:48.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:48.858 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:49.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.959+0000 7fe1b85d0640 1 Processor -- start 2024-03-06T07:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.959+0000 7fe1b85d0640 1 -- start start 2024-03-06T07:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b85d0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe1b01506b0 0x7fe1b0153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b85d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1b00acc00 0x7fe1b00c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b85d0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b0070aa0 con 0x7fe1b0156ea0 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b85d0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b0156450 con 0x7fe1b00acc00 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b85d0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b0156a10 con 0x7fe1b01506b0 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b6b6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1b00acc00 0x7fe1b00c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b5b6b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe1b0156ea0 0x7fe1b00702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48842/0 (socket says 172.21.15.26:48842) 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.960+0000 7fe1b5b6b640 1 -- 172.21.15.26:0/429415628 learned_addr learned my addr 172.21.15.26:0/429415628 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.961+0000 7fe1b636c640 1 --2- 172.21.15.26:0/429415628 >> v2:172.21.15.117:3300/0 conn(0x7fe1b01506b0 0x7fe1b0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.961+0000 7fe1b6b6d640 1 -- 172.21.15.26:0/429415628 >> v2:172.21.15.117:3300/0 conn(0x7fe1b01506b0 msgr2=0x7fe1b0153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.961+0000 7fe1b6b6d640 1 --2- 172.21.15.26:0/429415628 >> v2:172.21.15.117:3300/0 conn(0x7fe1b01506b0 0x7fe1b0153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.961+0000 7fe1b6b6d640 1 -- 172.21.15.26:0/429415628 >> v1:172.21.15.26:6789/0 conn(0x7fe1b0156ea0 legacy=0x7fe1b00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.961+0000 7fe1b536a640 1 -- 172.21.15.26:0/429415628 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415262072 0 0) 0x7fe1b0070aa0 con 0x7fe1b0156ea0 2024-03-06T07:30:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.961+0000 7fe1b6b6d640 1 -- 172.21.15.26:0/429415628 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b0070d80 con 0x7fe1b00acc00 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.962+0000 7fe1b6b6d640 1 --2- 172.21.15.26:0/429415628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b00c54a0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00c97e0 tx=0x7fe1a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fec507647a20af server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.962+0000 7fe1b536a640 1 -- 172.21.15.26:0/429415628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0039070 con 0x7fe1b00acc00 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.962+0000 7fe1b536a640 1 -- 172.21.15.26:0/429415628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a002cc80 con 0x7fe1b00acc00 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.962+0000 7fe1b536a640 1 -- 172.21.15.26:0/429415628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0034660 con 0x7fe1b00acc00 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.962+0000 7fe1b85d0640 1 -- 172.21.15.26:0/429415628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 msgr2=0x7fe1b00c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.962+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/429415628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b00c54a0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00c97e0 tx=0x7fe1a002bed0 comp rx=0 tx=0).stop 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.963+0000 7fe1b85d0640 1 -- 172.21.15.26:0/429415628 shutdown_connections 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.963+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/429415628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b00c54a0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.963+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/429415628 >> v2:172.21.15.117:3300/0 conn(0x7fe1b01506b0 0x7fe1b0153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.963+0000 7fe1b85d0640 1 -- 172.21.15.26:0/429415628 >> 172.21.15.26:0/429415628 conn(0x7fe1b006b1a0 msgr2=0x7fe1b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.963+0000 7fe1b85d0640 1 -- 172.21.15.26:0/429415628 shutdown_connections 2024-03-06T07:30:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.963+0000 7fe1b85d0640 1 -- 172.21.15.26:0/429415628 wait complete. 2024-03-06T07:30:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.964+0000 7fe1b85d0640 1 Processor -- start 2024-03-06T07:30:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.964+0000 7fe1b85d0640 1 -- start start 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.965+0000 7fe1b85d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b01551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.965+0000 7fe1b85d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1b01506b0 0x7fe1b0155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.965+0000 7fe1b85d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b0156ea0 0x7fe1b0150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.965+0000 7fe1b85d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00ad1d0 con 0x7fe1b00acc00 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.965+0000 7fe1b85d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00ad050 con 0x7fe1b0156ea0 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.965+0000 7fe1b85d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00ad350 con 0x7fe1b01506b0 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b01551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b01551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42066/0 (socket says 172.21.15.26:42066) 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 -- 172.21.15.26:0/3567582979 learned_addr learned my addr 172.21.15.26:0/3567582979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b6b6d640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b0156ea0 0x7fe1b0150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b5b6b640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1b01506b0 0x7fe1b0155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 -- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1b01506b0 msgr2=0x7fe1b0155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1b01506b0 0x7fe1b0155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 -- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b0156ea0 msgr2=0x7fe1b0150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b0156ea0 0x7fe1b0150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 -- 172.21.15.26:0/3567582979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00ab170 con 0x7fe1b00acc00 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b6b6d640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b0156ea0 0x7fe1b0150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b636c640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b01551e0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800c7f0 tx=0x7fe1a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1aeffd640 1 -- 172.21.15.26:0/3567582979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8015070 con 0x7fe1b00acc00 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1aeffd640 1 -- 172.21.15.26:0/3567582979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8004240 con 0x7fe1b00acc00 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b00aae90 con 0x7fe1b00acc00 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1aeffd640 1 -- 172.21.15.26:0/3567582979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8007620 con 0x7fe1b00acc00 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.966+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b01e8b70 con 0x7fe1b00acc00 2024-03-06T07:30:49.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.967+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b004f330 con 0x7fe1b00acc00 2024-03-06T07:30:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.968+0000 7fe1aeffd640 1 -- 172.21.15.26:0/3567582979 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1a80079d0 con 0x7fe1b00acc00 2024-03-06T07:30:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.968+0000 7fe1aeffd640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b500 0x7fe19c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.968+0000 7fe1aeffd640 1 -- 172.21.15.26:0/3567582979 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe1a808a460 con 0x7fe1b00acc00 2024-03-06T07:30:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.968+0000 7fe1b5b6b640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b500 0x7fe19c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.969+0000 7fe1b5b6b640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b500 0x7fe19c06d9c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe1b01468e0 tx=0x7fe1a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:49.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:49.973+0000 7fe1aeffd640 1 -- 172.21.15.26:0/3567582979 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1a80590e0 con 0x7fe1b00acc00 2024-03-06T07:30:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.195+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1b00700a0 con 0x7fe1b00acc00 2024-03-06T07:30:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.195+0000 7fe1aeffd640 1 -- 172.21.15.26:0/3567582979 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7fe1a80589e0 con 0x7fe1b00acc00 2024-03-06T07:30:50.197 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:50.197 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b500 msgr2=0x7fe19c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b500 0x7fe19c06d9c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe1b01468e0 tx=0x7fe1a4009040 comp rx=0 tx=0).stop 2024-03-06T07:30:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 msgr2=0x7fe1b01551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b01551e0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800c7f0 tx=0x7fe1a800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 shutdown_connections 2024-03-06T07:30:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b500 0x7fe19c06d9c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b0156ea0 0x7fe1b0150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1b01506b0 0x7fe1b0155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 --2- 172.21.15.26:0/3567582979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1b00acc00 0x7fe1b01551e0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 >> 172.21.15.26:0/3567582979 conn(0x7fe1b006b1a0 msgr2=0x7fe1b00702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 shutdown_connections 2024-03-06T07:30:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:50.198+0000 7fe1b85d0640 1 -- 172.21.15.26:0/3567582979 wait complete. 2024-03-06T07:30:50.612 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:50.613 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:51.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.715+0000 7fdb14ad4640 1 Processor -- start 2024-03-06T07:30:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.715+0000 7fdb14ad4640 1 -- start start 2024-03-06T07:30:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb14ad4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb1014f790 0x7fdb101529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb14ad4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb10155a40 0x7fdb10155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb14ad4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb10156410 con 0x7fdb100c1aa0 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb14ad4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb100c2680 con 0x7fdb10155a40 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb14ad4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb100c28d0 con 0x7fdb1014f790 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb0ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb10155a40 0x7fdb10155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb0ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb10155a40 0x7fdb10155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42068/0 (socket says 172.21.15.26:42068) 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb0ed9d640 1 -- 172.21.15.26:0/2424418405 learned_addr learned my addr 172.21.15.26:0/2424418405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.716+0000 7fdb0dd9b640 1 --1- 172.21.15.26:0/2424418405 >> v1:172.21.15.26:6789/0 conn(0x7fdb100c1aa0 0x7fdb100c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48850/0 (socket says 172.21.15.26:48850) 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.717+0000 7fdb0e59c640 1 --2- 172.21.15.26:0/2424418405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb1014f790 0x7fdb101529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.717+0000 7fdb0ed9d640 1 -- 172.21.15.26:0/2424418405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb1014f790 msgr2=0x7fdb101529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.717+0000 7fdb0ed9d640 1 --2- 172.21.15.26:0/2424418405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb1014f790 0x7fdb101529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.717+0000 7fdb0ed9d640 1 -- 172.21.15.26:0/2424418405 >> v1:172.21.15.26:6789/0 conn(0x7fdb100c1aa0 legacy=0x7fdb100c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.717+0000 7fdb0d59a640 1 -- 172.21.15.26:0/2424418405 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708641143 0 0) 0x7fdb10156410 con 0x7fdb100c1aa0 2024-03-06T07:30:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.717+0000 7fdb0ed9d640 1 -- 172.21.15.26:0/2424418405 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb100c2bb0 con 0x7fdb10155a40 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.717+0000 7fdb0ed9d640 1 --2- 172.21.15.26:0/2424418405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb10155a40 0x7fdb10155ed0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009e50 tx=0x7fdb0802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375d3ced3721e13e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb0d59a640 1 -- 172.21.15.26:0/2424418405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0802cb00 con 0x7fdb10155a40 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb0d59a640 1 -- 172.21.15.26:0/2424418405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0802cca0 con 0x7fdb10155a40 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb0d59a640 1 -- 172.21.15.26:0/2424418405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08034610 con 0x7fdb10155a40 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2424418405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb10155a40 msgr2=0x7fdb10155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2424418405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb10155a40 0x7fdb10155ed0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009e50 tx=0x7fdb0802bf60 comp rx=0 tx=0).stop 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2424418405 shutdown_connections 2024-03-06T07:30:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2424418405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb10155a40 0x7fdb10155ed0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2424418405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb1014f790 0x7fdb101529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2424418405 >> 172.21.15.26:0/2424418405 conn(0x7fdb100c6710 msgr2=0x7fdb100c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2424418405 shutdown_connections 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.718+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2424418405 wait complete. 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.719+0000 7fdb14ad4640 1 Processor -- start 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb14ad4640 1 -- start start 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb14ad4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb100c1aa0 0x7fdb10072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb14ad4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1014f790 0x7fdb1014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb14ad4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb10155a40 0x7fdb100702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb14ad4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb100c0d60 con 0x7fdb1014f790 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb14ad4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb10155520 con 0x7fdb100c1aa0 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb14ad4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb100c5e30 con 0x7fdb10155a40 2024-03-06T07:30:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb0dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1014f790 0x7fdb1014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb0dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1014f790 0x7fdb1014cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42084/0 (socket says 172.21.15.26:42084) 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb0dd9b640 1 -- 172.21.15.26:0/2717485395 learned_addr learned my addr 172.21.15.26:0/2717485395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb0ed9d640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb10155a40 0x7fdb100702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.720+0000 7fdb0e59c640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb100c1aa0 0x7fdb10072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb0dd9b640 1 -- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb10155a40 msgr2=0x7fdb100702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb0dd9b640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb10155a40 0x7fdb100702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb0dd9b640 1 -- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb100c1aa0 msgr2=0x7fdb10072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb0dd9b640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb100c1aa0 0x7fdb10072d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb0dd9b640 1 -- 172.21.15.26:0/2717485395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb100732b0 con 0x7fdb1014f790 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb0dd9b640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1014f790 0x7fdb1014cf20 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c4d0 tx=0x7fdaf800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb06ffd640 1 -- 172.21.15.26:0/2717485395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8018070 con 0x7fdb1014f790 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb06ffd640 1 -- 172.21.15.26:0/2717485395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8007500 con 0x7fdb1014f790 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb06ffd640 1 -- 172.21.15.26:0/2717485395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8004a20 con 0x7fdb1014f790 2024-03-06T07:30:51.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb10151680 con 0x7fdb1014f790 2024-03-06T07:30:51.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.721+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb10150750 con 0x7fdb1014f790 2024-03-06T07:30:51.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.722+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb10151ba0 con 0x7fdb1014f790 2024-03-06T07:30:51.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.722+0000 7fdb06ffd640 1 -- 172.21.15.26:0/2717485395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdaf801b070 con 0x7fdb1014f790 2024-03-06T07:30:51.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.723+0000 7fdb06ffd640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdaf006b4d0 0x7fdaf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:51.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.723+0000 7fdb06ffd640 1 -- 172.21.15.26:0/2717485395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdaf80896f0 con 0x7fdb1014f790 2024-03-06T07:30:51.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.723+0000 7fdb0e59c640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdaf006b4d0 0x7fdaf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:51.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.724+0000 7fdb0e59c640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdaf006b4d0 0x7fdaf006d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fdb00006fd0 tx=0x7fdb00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:51.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.727+0000 7fdb06ffd640 1 -- 172.21.15.26:0/2717485395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb10151ba0 con 0x7fdb1014f790 2024-03-06T07:30:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.947+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb1004f330 con 0x7fdb1014f790 2024-03-06T07:30:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.948+0000 7fdb06ffd640 1 -- 172.21.15.26:0/2717485395 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7fdb100732b0 con 0x7fdb1014f790 2024-03-06T07:30:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:51.950 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdaf006b4d0 msgr2=0x7fdaf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdaf006b4d0 0x7fdaf006d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fdb00006fd0 tx=0x7fdb00008040 comp rx=0 tx=0).stop 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1014f790 msgr2=0x7fdb1014cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1014f790 0x7fdb1014cf20 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c4d0 tx=0x7fdaf800c990 comp rx=0 tx=0).stop 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 shutdown_connections 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdaf006b4d0 0x7fdaf006d990 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb10155a40 0x7fdb100702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1014f790 0x7fdb1014cf20 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 --2- 172.21.15.26:0/2717485395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb100c1aa0 0x7fdb10072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 >> 172.21.15.26:0/2717485395 conn(0x7fdb100c6710 msgr2=0x7fdb100a7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 shutdown_connections 2024-03-06T07:30:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:51.950+0000 7fdb14ad4640 1 -- 172.21.15.26:0/2717485395 wait complete. 2024-03-06T07:30:52.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:52.365 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:53.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.467+0000 7f8e17215640 1 Processor -- start 2024-03-06T07:30:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.467+0000 7f8e17215640 1 -- start start 2024-03-06T07:30:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e17215640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1014f360 0x7f8e10155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e17215640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e100c88c0 0x7f8e100c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e17215640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e100c73d0 con 0x7f8e100c5880 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e17215640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e100c8dd0 con 0x7f8e100c88c0 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e17215640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100be2e0 con 0x7f8e1014f360 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e07fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e100c88c0 0x7f8e100c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e07fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e100c88c0 0x7f8e100c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42088/0 (socket says 172.21.15.26:42088) 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e07fff640 1 -- 172.21.15.26:0/3740809660 learned_addr learned my addr 172.21.15.26:0/3740809660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.468+0000 7f8e157b2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8e100c5880 0x7f8e10158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48862/0 (socket says 172.21.15.26:48862) 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e07fff640 1 -- 172.21.15.26:0/3740809660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1014f360 msgr2=0x7f8e10155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e14fb1640 1 --2- 172.21.15.26:0/3740809660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1014f360 0x7f8e10155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e07fff640 1 --2- 172.21.15.26:0/3740809660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1014f360 0x7f8e10155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e07fff640 1 -- 172.21.15.26:0/3740809660 >> v1:172.21.15.26:6789/0 conn(0x7f8e100c5880 legacy=0x7f8e10158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e077fe640 1 -- 172.21.15.26:0/3740809660 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027506108 0 0) 0x7f8e100c73d0 con 0x7f8e100c5880 2024-03-06T07:30:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e07fff640 1 -- 172.21.15.26:0/3740809660 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e100c7670 con 0x7f8e100c88c0 2024-03-06T07:30:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e14fb1640 1 --2- 172.21.15.26:0/3740809660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1014f360 0x7f8e10155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.469+0000 7f8e07fff640 1 --2- 172.21.15.26:0/3740809660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c88c0 0x7f8e100c5340 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009870 tx=0x7f8e0002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e364ee9ae149c38b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e077fe640 1 -- 172.21.15.26:0/3740809660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00035020 con 0x7f8e100c88c0 2024-03-06T07:30:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e077fe640 1 -- 172.21.15.26:0/3740809660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00031640 con 0x7f8e100c88c0 2024-03-06T07:30:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e077fe640 1 -- 172.21.15.26:0/3740809660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0002d040 con 0x7f8e100c88c0 2024-03-06T07:30:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 -- 172.21.15.26:0/3740809660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c88c0 msgr2=0x7f8e100c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 --2- 172.21.15.26:0/3740809660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c88c0 0x7f8e100c5340 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009870 tx=0x7f8e0002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 -- 172.21.15.26:0/3740809660 shutdown_connections 2024-03-06T07:30:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 --2- 172.21.15.26:0/3740809660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c88c0 0x7f8e100c5340 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 --2- 172.21.15.26:0/3740809660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1014f360 0x7f8e10155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 -- 172.21.15.26:0/3740809660 >> 172.21.15.26:0/3740809660 conn(0x7f8e100ac570 msgr2=0x7f8e100ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 -- 172.21.15.26:0/3740809660 shutdown_connections 2024-03-06T07:30:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.470+0000 7f8e17215640 1 -- 172.21.15.26:0/3740809660 wait complete. 2024-03-06T07:30:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.472+0000 7f8e17215640 1 Processor -- start 2024-03-06T07:30:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.472+0000 7f8e17215640 1 -- start start 2024-03-06T07:30:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e17215640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c5880 0x7f8e101520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e17215640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e100c88c0 0x7f8e10150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e17215640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1014f360 0x7f8e101eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e17215640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10159270 con 0x7f8e100c5880 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e17215640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100c7c30 con 0x7f8e1014f360 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e17215640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101593f0 con 0x7f8e100c88c0 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c5880 0x7f8e101520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c5880 0x7f8e101520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42092/0 (socket says 172.21.15.26:42092) 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 -- 172.21.15.26:0/221049427 learned_addr learned my addr 172.21.15.26:0/221049427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e157b2640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1014f360 0x7f8e101eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e07fff640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e100c88c0 0x7f8e10150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 -- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e100c88c0 msgr2=0x7f8e10150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e100c88c0 0x7f8e10150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 -- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1014f360 msgr2=0x7f8e101eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1014f360 0x7f8e101eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e14fb1640 1 -- 172.21.15.26:0/221049427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1014f990 con 0x7f8e100c5880 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.473+0000 7f8e157b2640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1014f360 0x7f8e101eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.474+0000 7f8e07fff640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e100c88c0 0x7f8e10150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.474+0000 7f8e14fb1640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c5880 0x7f8e101520c0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c7f0 tx=0x7f8e0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.474+0000 7f8e057fa640 1 -- 172.21.15.26:0/221049427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c015070 con 0x7f8e100c5880 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.474+0000 7f8e057fa640 1 -- 172.21.15.26:0/221049427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c007820 con 0x7f8e100c5880 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.474+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1014fc70 con 0x7f8e100c5880 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.474+0000 7f8e057fa640 1 -- 172.21.15.26:0/221049427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c007bd0 con 0x7f8e100c5880 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.474+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e10150230 con 0x7f8e100c5880 2024-03-06T07:30:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.475+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e101534e0 con 0x7f8e100c5880 2024-03-06T07:30:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.476+0000 7f8e057fa640 1 -- 172.21.15.26:0/221049427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e0c018070 con 0x7f8e100c5880 2024-03-06T07:30:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.476+0000 7f8e057fa640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8dfc06b4d0 0x7f8dfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.476+0000 7f8e057fa640 1 -- 172.21.15.26:0/221049427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f8e0c08a570 con 0x7f8e100c5880 2024-03-06T07:30:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.476+0000 7f8e07fff640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8dfc06b4d0 0x7f8dfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.477+0000 7f8e07fff640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8dfc06b4d0 0x7f8dfc06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8e00036000 tx=0x7f8e00008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.480+0000 7f8e057fa640 1 -- 172.21.15.26:0/221049427 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e101534e0 con 0x7f8e100c5880 2024-03-06T07:30:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.737+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e1004f330 con 0x7f8e100c5880 2024-03-06T07:30:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.737+0000 7f8e057fa640 1 -- 172.21.15.26:0/221049427 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f8e0c0591f0 con 0x7f8e100c5880 2024-03-06T07:30:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:53.739 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8dfc06b4d0 msgr2=0x7f8dfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8dfc06b4d0 0x7f8dfc06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8e00036000 tx=0x7f8e00008030 comp rx=0 tx=0).stop 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c5880 msgr2=0x7f8e101520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c5880 0x7f8e101520c0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c7f0 tx=0x7f8e0c00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 shutdown_connections 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8dfc06b4d0 0x7f8dfc06d990 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1014f360 0x7f8e101eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e100c88c0 0x7f8e10150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 --2- 172.21.15.26:0/221049427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e100c5880 0x7f8e101520c0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 >> 172.21.15.26:0/221049427 conn(0x7f8e100ac570 msgr2=0x7f8e1014cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 shutdown_connections 2024-03-06T07:30:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:53.739+0000 7f8e17215640 1 -- 172.21.15.26:0/221049427 wait complete. 2024-03-06T07:30:54.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:54.153 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:55.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:55.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.254+0000 7efe68dbc640 1 Processor -- start 2024-03-06T07:30:55.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.254+0000 7efe68dbc640 1 -- start start 2024-03-06T07:30:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe68dbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe64153990 0x7efe64156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe68dbc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe64070aa0 con 0x7efe641506b0 2024-03-06T07:30:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe68dbc640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe64070d80 con 0x7efe640acc00 2024-03-06T07:30:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe68dbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe64153e10 con 0x7efe64153990 2024-03-06T07:30:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe6259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efe641506b0 0x7efe640702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48870/0 (socket says 172.21.15.26:48870) 2024-03-06T07:30:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe6259c640 1 -- 172.21.15.26:0/935402184 learned_addr learned my addr 172.21.15.26:0/935402184 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480960610 0 0) 0x7efe64070aa0 con 0x7efe641506b0 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe50003520 con 0x7efe641506b0 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe5bfff640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe64153990 0x7efe64156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2566139500 0 0) 0x7efe50003520 con 0x7efe641506b0 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe64153990 msgr2=0x7efe64156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe64153990 0x7efe64156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 >> v1:172.21.15.117:6789/0 conn(0x7efe640acc00 legacy=0x7efe640c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe641566a0 con 0x7efe641506b0 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7efe54001670 con 0x7efe641506b0 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 >> v1:172.21.15.26:6789/0 conn(0x7efe641506b0 legacy=0x7efe640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500039e0 0x7efe50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe50004800 0x7efe50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64153e10 con 0x7efe500051c0 2024-03-06T07:30:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5000c090 con 0x7efe500039e0 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5000c290 con 0x7efe50004800 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.255+0000 7efe5bfff640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe64153990 0x7efe64156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe62d9d640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe6259c640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500039e0 0x7efe50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe5bfff640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe50004800 0x7efe50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe62d9d640 1 -- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe50004800 msgr2=0x7efe50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe62d9d640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe50004800 0x7efe50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe62d9d640 1 -- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500039e0 msgr2=0x7efe50003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe62d9d640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500039e0 0x7efe50003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.256+0000 7efe62d9d640 1 -- 172.21.15.26:0/935402184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe50003520 con 0x7efe500051c0 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe62d9d640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7efe5c001e50 tx=0x7efe50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46169e8f55ae50d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c001a70 con 0x7efe500051c0 2024-03-06T07:30:55.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c003b70 con 0x7efe500051c0 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe61d9b640 1 -- 172.21.15.26:0/935402184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0050d0 con 0x7efe500051c0 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe68dbc640 1 -- 172.21.15.26:0/935402184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500051c0 msgr2=0x7efe5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe68dbc640 1 --2- 172.21.15.26:0/935402184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500051c0 0x7efe5000b9c0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7efe5c001e50 tx=0x7efe50005830 comp rx=0 tx=0).stop 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe62d9d640 1 -- 172.21.15.26:0/935402184 reap_dead start 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe68dbc640 1 -- 172.21.15.26:0/935402184 shutdown_connections 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe68dbc640 1 -- 172.21.15.26:0/935402184 >> 172.21.15.26:0/935402184 conn(0x7efe6406b1a0 msgr2=0x7efe6406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe68dbc640 1 -- 172.21.15.26:0/935402184 shutdown_connections 2024-03-06T07:30:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.257+0000 7efe68dbc640 1 -- 172.21.15.26:0/935402184 wait complete. 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 Processor -- start 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 -- start start 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500051c0 0x7efe6414e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe640c2fc0 0x7efe64146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe64147110 0x7efe641ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64156a10 con 0x7efe64147110 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64156450 con 0x7efe640c2fc0 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe68dbc640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe640ad070 con 0x7efe500051c0 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe62d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe64147110 0x7efe641ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe62d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe64147110 0x7efe641ee3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42096/0 (socket says 172.21.15.26:42096) 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.259+0000 7efe62d9d640 1 -- 172.21.15.26:0/765452922 learned_addr learned my addr 172.21.15.26:0/765452922 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe6259c640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500051c0 0x7efe6414e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe5bfff640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe640c2fc0 0x7efe64146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe62d9d640 1 -- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500051c0 msgr2=0x7efe6414e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe62d9d640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500051c0 0x7efe6414e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe62d9d640 1 -- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe640c2fc0 msgr2=0x7efe64146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe62d9d640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe640c2fc0 0x7efe64146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe62d9d640 1 -- 172.21.15.26:0/765452922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6414f200 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe6259c640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500051c0 0x7efe6414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe62d9d640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe64147110 0x7efe641ee3c0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7efe5c001f40 tx=0x7efe5c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe5affd640 1 -- 172.21.15.26:0/765452922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c007db0 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe5affd640 1 -- 172.21.15.26:0/765452922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c003970 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe64147590 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe5affd640 1 -- 172.21.15.26:0/765452922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c006d90 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.260+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe641eb2e0 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.261+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6404f330 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.262+0000 7efe5affd640 1 -- 172.21.15.26:0/765452922 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe5c007030 con 0x7efe64147110 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.262+0000 7efe5affd640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe4406b500 0x7efe4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.262+0000 7efe5affd640 1 -- 172.21.15.26:0/765452922 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7efe5c0ab580 con 0x7efe64147110 2024-03-06T07:30:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.262+0000 7efe6259c640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe4406b500 0x7efe4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.263+0000 7efe6259c640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe4406b500 0x7efe4406d9c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efe54000c00 tx=0x7efe54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.267+0000 7efe5affd640 1 -- 172.21.15.26:0/765452922 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efe6414f200 con 0x7efe64147110 2024-03-06T07:30:55.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.486+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe640700a0 con 0x7efe64147110 2024-03-06T07:30:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.487+0000 7efe5affd640 1 -- 172.21.15.26:0/765452922 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7efe5c033020 con 0x7efe64147110 2024-03-06T07:30:55.488 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:55.488 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe4406b500 msgr2=0x7efe4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe4406b500 0x7efe4406d9c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efe54000c00 tx=0x7efe54009040 comp rx=0 tx=0).stop 2024-03-06T07:30:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe64147110 msgr2=0x7efe641ee3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe64147110 0x7efe641ee3c0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7efe5c001f40 tx=0x7efe5c0014a0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 shutdown_connections 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efe4406b500 0x7efe4406d9c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe64147110 0x7efe641ee3c0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe640c2fc0 0x7efe64146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 --2- 172.21.15.26:0/765452922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500051c0 0x7efe6414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.488+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 >> 172.21.15.26:0/765452922 conn(0x7efe6406b1a0 msgr2=0x7efe64152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.489+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 shutdown_connections 2024-03-06T07:30:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:55.489+0000 7efe68dbc640 1 -- 172.21.15.26:0/765452922 wait complete. 2024-03-06T07:30:55.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:55.903 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:56.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:57.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.043+0000 7f98dc920640 1 Processor -- start 2024-03-06T07:30:57.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.043+0000 7f98dc920640 1 -- start start 2024-03-06T07:30:57.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98dc920640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98d814f360 0x7f98d8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98dc920640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98dc920640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d80c73d0 con 0x7f98d80c5880 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98dc920640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d80c8dd0 con 0x7f98d814f360 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98dc920640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80be2e0 con 0x7f98d80c88c0 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98d659c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98d814f360 0x7f98d8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98d5d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98d6d9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f98d80c5880 0x7f98d8158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:47074/0 (socket says 172.21.15.26:47074) 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98d6d9d640 1 -- 172.21.15.26:0/2556103855 learned_addr learned my addr 172.21.15.26:0/2556103855 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.045+0000 7f98d659c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98d814f360 0x7f98d8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37166/0 (socket says 172.21.15.26:37166) 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.046+0000 7f98d5d9b640 1 -- 172.21.15.26:0/2556103855 >> v2:172.21.15.117:3300/0 conn(0x7f98d814f360 msgr2=0x7f98d8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.046+0000 7f98d5d9b640 1 --2- 172.21.15.26:0/2556103855 >> v2:172.21.15.117:3300/0 conn(0x7f98d814f360 0x7f98d8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.046+0000 7f98d5d9b640 1 -- 172.21.15.26:0/2556103855 >> v1:172.21.15.117:6789/0 conn(0x7f98d80c5880 legacy=0x7f98d8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.046+0000 7f98d5d9b640 1 -- 172.21.15.26:0/2556103855 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80c7670 con 0x7f98d80c88c0 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.046+0000 7f98d659c640 1 --2- 172.21.15.26:0/2556103855 >> v2:172.21.15.117:3300/0 conn(0x7f98d814f360 0x7f98d8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.046+0000 7f98d5d9b640 1 --2- 172.21.15.26:0/2556103855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c5340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097a0 tx=0x7f98c002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1c001fea2b365e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98d559a640 1 -- 172.21.15.26:0/2556103855 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00315a0 con 0x7f98d80c88c0 2024-03-06T07:30:57.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98d559a640 1 -- 172.21.15.26:0/2556103855 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c0037070 con 0x7f98d80c88c0 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98d559a640 1 -- 172.21.15.26:0/2556103855 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0031960 con 0x7f98d80c88c0 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 -- 172.21.15.26:0/2556103855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 msgr2=0x7f98d80c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 --2- 172.21.15.26:0/2556103855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c5340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097a0 tx=0x7f98c002b840 comp rx=0 tx=0).stop 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 -- 172.21.15.26:0/2556103855 shutdown_connections 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 --2- 172.21.15.26:0/2556103855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c5340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 --2- 172.21.15.26:0/2556103855 >> v2:172.21.15.117:3300/0 conn(0x7f98d814f360 0x7f98d8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 -- 172.21.15.26:0/2556103855 >> 172.21.15.26:0/2556103855 conn(0x7f98d80ac570 msgr2=0x7f98d80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 -- 172.21.15.26:0/2556103855 shutdown_connections 2024-03-06T07:30:57.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.047+0000 7f98dc920640 1 -- 172.21.15.26:0/2556103855 wait complete. 2024-03-06T07:30:57.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.049+0000 7f98dc920640 1 Processor -- start 2024-03-06T07:30:57.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.049+0000 7f98dc920640 1 -- start start 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.050+0000 7f98dc920640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98d80c5880 0x7f98d814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.050+0000 7f98dc920640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.050+0000 7f98dc920640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98d814f360 0x7f98d814abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.050+0000 7f98dc920640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d8159270 con 0x7f98d80c5880 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.050+0000 7f98dc920640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80c7c30 con 0x7f98d814f360 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.050+0000 7f98dc920640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d81593f0 con 0x7f98d80c88c0 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.050+0000 7f98d659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98d80c5880 0x7f98d814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98d80c5880 0x7f98d814db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42106/0 (socket says 172.21.15.26:42106) 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 -- 172.21.15.26:0/1309069422 learned_addr learned my addr 172.21.15.26:0/1309069422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d6d9d640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98d814f360 0x7f98d814abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d5d9b640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 -- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 msgr2=0x7f98d80c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 -- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98d814f360 msgr2=0x7f98d814abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98d814f360 0x7f98d814abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 -- 172.21.15.26:0/1309069422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d81522c0 con 0x7f98d80c5880 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98d659c640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98d80c5880 0x7f98d814db50 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e960 tx=0x7f98d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98ceffd640 1 -- 172.21.15.26:0/1309069422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0017070 con 0x7f98d80c5880 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98ceffd640 1 -- 172.21.15.26:0/1309069422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d000ca00 con 0x7f98d80c5880 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d81525a0 con 0x7f98d80c5880 2024-03-06T07:30:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98ceffd640 1 -- 172.21.15.26:0/1309069422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000cdd0 con 0x7f98d80c5880 2024-03-06T07:30:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.051+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d81eed50 con 0x7f98d80c5880 2024-03-06T07:30:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.052+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d804f330 con 0x7f98d80c5880 2024-03-06T07:30:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.053+0000 7f98ceffd640 1 -- 172.21.15.26:0/1309069422 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98d001a070 con 0x7f98d80c5880 2024-03-06T07:30:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.053+0000 7f98ceffd640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98b806b4d0 0x7f98b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.053+0000 7f98ceffd640 1 -- 172.21.15.26:0/1309069422 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f98d008a5c0 con 0x7f98d80c5880 2024-03-06T07:30:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.053+0000 7f98d5d9b640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98b806b4d0 0x7f98b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.054+0000 7f98d5d9b640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98b806b4d0 0x7f98b806d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f98c0036000 tx=0x7f98c003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:57.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.062+0000 7f98ceffd640 1 -- 172.21.15.26:0/1309069422 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f98d0058bf0 con 0x7f98d80c5880 2024-03-06T07:30:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.281+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98d806cda0 con 0x7f98d80c5880 2024-03-06T07:30:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.281+0000 7f98ceffd640 1 -- 172.21.15.26:0/1309069422 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f98d005cc90 con 0x7f98d80c5880 2024-03-06T07:30:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:57.283 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98b806b4d0 msgr2=0x7f98b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98b806b4d0 0x7f98b806d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f98c0036000 tx=0x7f98c003d040 comp rx=0 tx=0).stop 2024-03-06T07:30:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98d80c5880 msgr2=0x7f98d814db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98d80c5880 0x7f98d814db50 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e960 tx=0x7f98d000ee20 comp rx=0 tx=0).stop 2024-03-06T07:30:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 shutdown_connections 2024-03-06T07:30:57.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f98b806b4d0 0x7f98b806d990 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98d814f360 0x7f98d814abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98d80c88c0 0x7f98d80c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 --2- 172.21.15.26:0/1309069422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98d80c5880 0x7f98d814db50 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 >> 172.21.15.26:0/1309069422 conn(0x7f98d80ac570 msgr2=0x7f98d80ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:57.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.283+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 shutdown_connections 2024-03-06T07:30:57.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:57.284+0000 7f98dc920640 1 -- 172.21.15.26:0/1309069422 wait complete. 2024-03-06T07:30:57.702 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:57.702 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:30:58.702 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:58.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.803+0000 7feb393a0640 1 Processor -- start 2024-03-06T07:30:58.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.803+0000 7feb393a0640 1 -- start start 2024-03-06T07:30:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.803+0000 7feb393a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb340c8cd0 0x7feb34072e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.803+0000 7feb393a0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb3406fec0 0x7feb341552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb393a0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb34158970 con 0x7feb34152800 2024-03-06T07:30:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb393a0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb34158c50 con 0x7feb340c8cd0 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb393a0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb34158f30 con 0x7feb3406fec0 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb337fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb340c8cd0 0x7feb34072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb33fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb34152800 0x7feb340c85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45254/0 (socket says 172.21.15.26:45254) 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb337fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb340c8cd0 0x7feb34072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60768/0 (socket says 172.21.15.26:60768) 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb33fff640 1 -- 172.21.15.26:0/3991303928 learned_addr learned my addr 172.21.15.26:0/3991303928 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb337fe640 1 -- 172.21.15.26:0/3991303928 >> v2:172.21.15.117:3300/0 conn(0x7feb3406fec0 msgr2=0x7feb341552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb38b9f640 1 --2- 172.21.15.26:0/3991303928 >> v2:172.21.15.117:3300/0 conn(0x7feb3406fec0 0x7feb341552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb337fe640 1 --2- 172.21.15.26:0/3991303928 >> v2:172.21.15.117:3300/0 conn(0x7feb3406fec0 0x7feb341552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb32ffd640 1 -- 172.21.15.26:0/3991303928 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282523515 0 0) 0x7feb34158970 con 0x7feb34152800 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb337fe640 1 -- 172.21.15.26:0/3991303928 >> v1:172.21.15.26:6789/0 conn(0x7feb34152800 legacy=0x7feb340c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb337fe640 1 -- 172.21.15.26:0/3991303928 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34159730 con 0x7feb340c8cd0 2024-03-06T07:30:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.804+0000 7feb38b9f640 1 --2- 172.21.15.26:0/3991303928 >> v2:172.21.15.117:3300/0 conn(0x7feb3406fec0 0x7feb341552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.805+0000 7feb337fe640 1 --2- 172.21.15.26:0/3991303928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb34072e00 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7feb340c3d40 tx=0x7feb2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419fa3cc28f806e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.805+0000 7feb32ffd640 1 -- 172.21.15.26:0/3991303928 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280315a0 con 0x7feb340c8cd0 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.805+0000 7feb32ffd640 1 -- 172.21.15.26:0/3991303928 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb28037070 con 0x7feb340c8cd0 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.805+0000 7feb32ffd640 1 -- 172.21.15.26:0/3991303928 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28031960 con 0x7feb340c8cd0 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 -- 172.21.15.26:0/3991303928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 msgr2=0x7feb34072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 --2- 172.21.15.26:0/3991303928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb34072e00 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7feb340c3d40 tx=0x7feb2802b840 comp rx=0 tx=0).stop 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 -- 172.21.15.26:0/3991303928 shutdown_connections 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 --2- 172.21.15.26:0/3991303928 >> v2:172.21.15.117:3300/0 conn(0x7feb3406fec0 0x7feb341552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 --2- 172.21.15.26:0/3991303928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb34072e00 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 -- 172.21.15.26:0/3991303928 >> 172.21.15.26:0/3991303928 conn(0x7feb340aa4e0 msgr2=0x7feb340aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 -- 172.21.15.26:0/3991303928 shutdown_connections 2024-03-06T07:30:58.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.806+0000 7feb393a0640 1 -- 172.21.15.26:0/3991303928 wait complete. 2024-03-06T07:30:58.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.807+0000 7feb393a0640 1 Processor -- start 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.807+0000 7feb393a0640 1 -- start start 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb3406fec0 0x7feb341ee230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb341548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34152800 0x7feb34154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb34152be0 con 0x7feb340c8cd0 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3406f660 con 0x7feb34152800 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb34152d60 con 0x7feb3406fec0 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb341548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:58.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb341548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60782/0 (socket says 172.21.15.26:60782) 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 -- 172.21.15.26:0/2492502386 learned_addr learned my addr 172.21.15.26:0/2492502386 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb38b9f640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34152800 0x7feb34154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 -- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb3406fec0 msgr2=0x7feb341ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb3406fec0 0x7feb341ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 -- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34152800 msgr2=0x7feb34154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34152800 0x7feb34154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 -- 172.21.15.26:0/2492502386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb341eed40 con 0x7feb340c8cd0 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb38b9f640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34152800 0x7feb34154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb337fe640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb341548f0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7feb28009890 tx=0x7feb280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb30ff9640 1 -- 172.21.15.26:0/2492502386 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28032840 con 0x7feb340c8cd0 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb30ff9640 1 -- 172.21.15.26:0/2492502386 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb280329e0 con 0x7feb340c8cd0 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb30ff9640 1 -- 172.21.15.26:0/2492502386 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28032dd0 con 0x7feb340c8cd0 2024-03-06T07:30:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb340bfdf0 con 0x7feb340c8cd0 2024-03-06T07:30:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.808+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3406d9c0 con 0x7feb340c8cd0 2024-03-06T07:30:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.810+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb340bfbe0 con 0x7feb340c8cd0 2024-03-06T07:30:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.810+0000 7feb30ff9640 1 -- 172.21.15.26:0/2492502386 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb2804c020 con 0x7feb340c8cd0 2024-03-06T07:30:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.810+0000 7feb30ff9640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb2006b4d0 0x7feb2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.810+0000 7feb30ff9640 1 -- 172.21.15.26:0/2492502386 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7feb2802f020 con 0x7feb340c8cd0 2024-03-06T07:30:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.811+0000 7feb33fff640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb2006b4d0 0x7feb2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.811+0000 7feb33fff640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb2006b4d0 0x7feb2006d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7feb24007fd0 tx=0x7feb24007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:58.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:58.819+0000 7feb30ff9640 1 -- 172.21.15.26:0/2492502386 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb28040070 con 0x7feb340c8cd0 2024-03-06T07:30:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.045+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb34069430 con 0x7feb340c8cd0 2024-03-06T07:30:59.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.045+0000 7feb30ff9640 1 -- 172.21.15.26:0/2492502386 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7feb2807afa0 con 0x7feb340c8cd0 2024-03-06T07:30:59.047 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:30:59.047 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:30:59.047 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:30:59.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb2006b4d0 msgr2=0x7feb2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:59.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb2006b4d0 0x7feb2006d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7feb24007fd0 tx=0x7feb24007960 comp rx=0 tx=0).stop 2024-03-06T07:30:59.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 msgr2=0x7feb341548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:59.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb341548f0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7feb28009890 tx=0x7feb280046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 shutdown_connections 2024-03-06T07:30:59.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb2006b4d0 0x7feb2006d990 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34152800 0x7feb34154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340c8cd0 0x7feb341548f0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 --2- 172.21.15.26:0/2492502386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb3406fec0 0x7feb341ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 >> 172.21.15.26:0/2492502386 conn(0x7feb340aa4e0 msgr2=0x7feb340729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 shutdown_connections 2024-03-06T07:30:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:30:59.047+0000 7feb393a0640 1 -- 172.21.15.26:0/2492502386 wait complete. 2024-03-06T07:30:59.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:30:59.462 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:00.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:00.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.564+0000 7f7751ab5640 1 Processor -- start 2024-03-06T07:31:00.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.564+0000 7f7751ab5640 1 -- start start 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f7751ab5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c14f360 0x7f774c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f7751ab5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f774c0c88c0 0x7f774c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f7751ab5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f774c0c5880 0x7f774c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f7751ab5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774c0c8dd0 con 0x7f774c0c88c0 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f7751ab5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774c0c73d0 con 0x7f774c0c5880 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f7751ab5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774c0be2e0 con 0x7f774c14f360 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f774affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f774c0c88c0 0x7f774c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f774affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f774c0c88c0 0x7f774c0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60796/0 (socket says 172.21.15.26:60796) 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f774affd640 1 -- 172.21.15.26:0/2260818931 learned_addr learned my addr 172.21.15.26:0/2260818931 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f774b7fe640 1 --2- 172.21.15.26:0/2260818931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c14f360 0x7f774c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.565+0000 7f774bfff640 1 --2- 172.21.15.26:0/2260818931 >> v2:172.21.15.117:3300/0 conn(0x7f774c0c5880 0x7f774c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.566+0000 7f774affd640 1 -- 172.21.15.26:0/2260818931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c14f360 msgr2=0x7f774c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:00.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.566+0000 7f774affd640 1 --2- 172.21.15.26:0/2260818931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c14f360 0x7f774c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.566+0000 7f774affd640 1 -- 172.21.15.26:0/2260818931 >> v2:172.21.15.117:3300/0 conn(0x7f774c0c5880 msgr2=0x7f774c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:00.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.566+0000 7f774affd640 1 --2- 172.21.15.26:0/2260818931 >> v2:172.21.15.117:3300/0 conn(0x7f774c0c5880 0x7f774c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.566+0000 7f774affd640 1 -- 172.21.15.26:0/2260818931 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c0c7670 con 0x7f774c0c88c0 2024-03-06T07:31:00.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.566+0000 7f774bfff640 1 --2- 172.21.15.26:0/2260818931 >> v2:172.21.15.117:3300/0 conn(0x7f774c0c5880 0x7f774c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:00.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.566+0000 7f774b7fe640 1 --2- 172.21.15.26:0/2260818931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c14f360 0x7f774c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:00.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f774affd640 1 --2- 172.21.15.26:0/2260818931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c0c88c0 0x7f774c0c5340 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f774c08cc10 tx=0x7f773c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2effe79289ad26c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:00.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f774a7fc640 1 -- 172.21.15.26:0/2260818931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c02e070 con 0x7f774c0c88c0 2024-03-06T07:31:00.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f774a7fc640 1 -- 172.21.15.26:0/2260818931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c037070 con 0x7f774c0c88c0 2024-03-06T07:31:00.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f774a7fc640 1 -- 172.21.15.26:0/2260818931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c031a00 con 0x7f774c0c88c0 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 -- 172.21.15.26:0/2260818931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c0c88c0 msgr2=0x7f774c0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 --2- 172.21.15.26:0/2260818931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c0c88c0 0x7f774c0c5340 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f774c08cc10 tx=0x7f773c02b880 comp rx=0 tx=0).stop 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 -- 172.21.15.26:0/2260818931 shutdown_connections 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 --2- 172.21.15.26:0/2260818931 >> v2:172.21.15.117:3300/0 conn(0x7f774c0c5880 0x7f774c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 --2- 172.21.15.26:0/2260818931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c0c88c0 0x7f774c0c5340 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 --2- 172.21.15.26:0/2260818931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c14f360 0x7f774c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 -- 172.21.15.26:0/2260818931 >> 172.21.15.26:0/2260818931 conn(0x7f774c0ac570 msgr2=0x7f774c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.567+0000 7f7751ab5640 1 -- 172.21.15.26:0/2260818931 shutdown_connections 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.568+0000 7f7751ab5640 1 -- 172.21.15.26:0/2260818931 wait complete. 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 Processor -- start 2024-03-06T07:31:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 -- start start 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774c0c5880 0x7f774c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c0c88c0 0x7f774c148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c14f360 0x7f774c148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c1590a0 con 0x7f774c14f360 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c0c7c30 con 0x7f774c0c5880 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f7751ab5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774c159220 con 0x7f774c0c88c0 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.569+0000 7f774bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c14f360 0x7f774c148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c14f360 0x7f774c148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60800/0 (socket says 172.21.15.26:60800) 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 -- 172.21.15.26:0/639013856 learned_addr learned my addr 172.21.15.26:0/639013856 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774b7fe640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774c0c5880 0x7f774c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 -- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c0c88c0 msgr2=0x7f774c148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774affd640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c0c88c0 0x7f774c148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c0c88c0 0x7f774c148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 -- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774c0c5880 msgr2=0x7f774c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774c0c5880 0x7f774c157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 -- 172.21.15.26:0/639013856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c0c12f0 con 0x7f774c14f360 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774b7fe640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774c0c5880 0x7f774c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f774bfff640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c14f360 0x7f774c148860 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f7744007e10 tx=0x7f774400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f7733fff640 1 -- 172.21.15.26:0/639013856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774401a070 con 0x7f774c14f360 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f7733fff640 1 -- 172.21.15.26:0/639013856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7744003b90 con 0x7f774c14f360 2024-03-06T07:31:00.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c0c15d0 con 0x7f774c14f360 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f7733fff640 1 -- 172.21.15.26:0/639013856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744004030 con 0x7f774c14f360 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.570+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c158330 con 0x7f774c14f360 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.571+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774c155080 con 0x7f774c14f360 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.572+0000 7f7733fff640 1 -- 172.21.15.26:0/639013856 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f774401d070 con 0x7f774c14f360 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.572+0000 7f7733fff640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f773806b4d0 0x7f773806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.572+0000 7f7733fff640 1 -- 172.21.15.26:0/639013856 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f774408a6f0 con 0x7f774c14f360 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.572+0000 7f774b7fe640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f773806b4d0 0x7f773806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.573+0000 7f774b7fe640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f773806b4d0 0x7f773806d990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7740006fd0 tx=0x7f7740008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:00.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.581+0000 7f7733fff640 1 -- 172.21.15.26:0/639013856 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f774c155080 con 0x7f774c14f360 2024-03-06T07:31:00.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.838+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f774c14d5e0 con 0x7f774c14f360 2024-03-06T07:31:00.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.839+0000 7f7733fff640 1 -- 172.21.15.26:0/639013856 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f774c14d5e0 con 0x7f774c14f360 2024-03-06T07:31:00.840 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:00.841 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:00.841 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:31:00.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.840+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f773806b4d0 msgr2=0x7f773806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.840+0000 7f7751ab5640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f773806b4d0 0x7f773806d990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7740006fd0 tx=0x7f7740008040 comp rx=0 tx=0).stop 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c14f360 msgr2=0x7f774c148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c14f360 0x7f774c148860 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f7744007e10 tx=0x7f774400d4a0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 shutdown_connections 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f773806b4d0 0x7f773806d990 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f774c14f360 0x7f774c148860 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f774c0c88c0 0x7f774c148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 --2- 172.21.15.26:0/639013856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f774c0c5880 0x7f774c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 >> 172.21.15.26:0/639013856 conn(0x7f774c0ac570 msgr2=0x7f774c0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 shutdown_connections 2024-03-06T07:31:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:00.841+0000 7f7751ab5640 1 -- 172.21.15.26:0/639013856 wait complete. 2024-03-06T07:31:01.255 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:01.255 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:02.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:02.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.357+0000 7f028d3e9640 1 Processor -- start 2024-03-06T07:31:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.357+0000 7f028d3e9640 1 -- start start 2024-03-06T07:31:02.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f028d3e9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f02880c88c0 0x7f0288155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f028d3e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f0288158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f028d3e9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02880c73d0 con 0x7f028814f360 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f028d3e9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880c8dd0 con 0x7f02880c88c0 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f028d3e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02880be2e0 con 0x7f0288158b60 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f02877fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f0288158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f02867fc640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f02880c88c0 0x7f0288155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f0286ffd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f028814f360 0x7f02880c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:45932/0 (socket says 172.21.15.26:45932) 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f0286ffd640 1 -- 172.21.15.26:0/739708839 learned_addr learned my addr 172.21.15.26:0/739708839 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f0285ffb640 1 -- 172.21.15.26:0/739708839 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462228126 0 0) 0x7f02880c73d0 con 0x7f028814f360 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f0285ffb640 1 -- 172.21.15.26:0/739708839 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0270003520 con 0x7f028814f360 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f02867fc640 1 -- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 msgr2=0x7f0288158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f02867fc640 1 --2- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f0288158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f02867fc640 1 -- 172.21.15.26:0/739708839 >> v1:172.21.15.117:6789/0 conn(0x7f028814f360 legacy=0x7f02880c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f02867fc640 1 -- 172.21.15.26:0/739708839 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880c7670 con 0x7f02880c88c0 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.358+0000 7f02877fe640 1 --2- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f0288158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:02.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.359+0000 7f02867fc640 1 --2- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02880c88c0 0x7f0288155c10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f027c0097a0 tx=0x7f027c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187b922ecf86a95a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.359+0000 7f0285ffb640 1 -- 172.21.15.26:0/739708839 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0315a0 con 0x7f02880c88c0 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.359+0000 7f0285ffb640 1 -- 172.21.15.26:0/739708839 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c037070 con 0x7f02880c88c0 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.359+0000 7f0285ffb640 1 -- 172.21.15.26:0/739708839 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c031940 con 0x7f02880c88c0 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 -- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02880c88c0 msgr2=0x7f0288155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 --2- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02880c88c0 0x7f0288155c10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f027c0097a0 tx=0x7f027c02b840 comp rx=0 tx=0).stop 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 -- 172.21.15.26:0/739708839 shutdown_connections 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 --2- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f0288158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 --2- 172.21.15.26:0/739708839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02880c88c0 0x7f0288155c10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 -- 172.21.15.26:0/739708839 >> 172.21.15.26:0/739708839 conn(0x7f02880ac570 msgr2=0x7f02880ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 -- 172.21.15.26:0/739708839 shutdown_connections 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.360+0000 7f028d3e9640 1 -- 172.21.15.26:0/739708839 wait complete. 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.361+0000 7f028d3e9640 1 Processor -- start 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.361+0000 7f028d3e9640 1 -- start start 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f028d3e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02880c88c0 0x7f028814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f028d3e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f028814f360 0x7f02880c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f028d3e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f02880bf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f028d3e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02880c5a50 con 0x7f02880c88c0 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f028d3e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02880c7c30 con 0x7f028814f360 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f028d3e9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02880c5bd0 con 0x7f0288158b60 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02880c88c0 0x7f028814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02880c88c0 0x7f028814ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60806/0 (socket says 172.21.15.26:60806) 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 -- 172.21.15.26:0/144001114 learned_addr learned my addr 172.21.15.26:0/144001114 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f02877fe640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f02880bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f02867fc640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f028814f360 0x7f02880c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 -- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 msgr2=0x7f02880bf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f02880bf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 -- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f028814f360 msgr2=0x7f02880c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f028814f360 0x7f02880c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.362+0000 7f0286ffd640 1 -- 172.21.15.26:0/144001114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02881eb660 con 0x7f02880c88c0 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.363+0000 7f0286ffd640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02880c88c0 0x7f028814ba00 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f027800c430 tx=0x7f027800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.363+0000 7f026f7fe640 1 -- 172.21.15.26:0/144001114 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278017070 con 0x7f02880c88c0 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.363+0000 7f026f7fe640 1 -- 172.21.15.26:0/144001114 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0278005ce0 con 0x7f02880c88c0 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.363+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02881eb900 con 0x7f02880c88c0 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.363+0000 7f026f7fe640 1 -- 172.21.15.26:0/144001114 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02780072f0 con 0x7f02880c88c0 2024-03-06T07:31:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.363+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02880be900 con 0x7f02880c88c0 2024-03-06T07:31:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.364+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028804f330 con 0x7f02880c88c0 2024-03-06T07:31:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.364+0000 7f026f7fe640 1 -- 172.21.15.26:0/144001114 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f027801a070 con 0x7f02880c88c0 2024-03-06T07:31:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.365+0000 7f026f7fe640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f027406b4d0 0x7f027406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.365+0000 7f026f7fe640 1 -- 172.21.15.26:0/144001114 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f0278089770 con 0x7f02880c88c0 2024-03-06T07:31:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.365+0000 7f02867fc640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f027406b4d0 0x7f027406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.365+0000 7f02867fc640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f027406b4d0 0x7f027406d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f027c036000 tx=0x7f027c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:02.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.369+0000 7f026f7fe640 1 -- 172.21.15.26:0/144001114 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0278057d20 con 0x7f02880c88c0 2024-03-06T07:31:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.588+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f028814c580 con 0x7f02880c88c0 2024-03-06T07:31:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.588+0000 7f026f7fe640 1 -- 172.21.15.26:0/144001114 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f027805bdc0 con 0x7f02880c88c0 2024-03-06T07:31:02.590 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:02.590 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:31:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f027406b4d0 msgr2=0x7f027406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f027406b4d0 0x7f027406d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f027c036000 tx=0x7f027c008030 comp rx=0 tx=0).stop 2024-03-06T07:31:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02880c88c0 msgr2=0x7f028814ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02880c88c0 0x7f028814ba00 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f027800c430 tx=0x7f027800c830 comp rx=0 tx=0).stop 2024-03-06T07:31:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 shutdown_connections 2024-03-06T07:31:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f027406b4d0 0x7f027406d990 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0288158b60 0x7f02880bf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f028814f360 0x7f02880c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 --2- 172.21.15.26:0/144001114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02880c88c0 0x7f028814ba00 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 >> 172.21.15.26:0/144001114 conn(0x7f02880ac570 msgr2=0x7f028814cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 shutdown_connections 2024-03-06T07:31:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:02.590+0000 7f028d3e9640 1 -- 172.21.15.26:0/144001114 wait complete. 2024-03-06T07:31:03.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:03.008 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:04.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.111+0000 7f6917ceb640 1 Processor -- start 2024-03-06T07:31:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.111+0000 7f6917ceb640 1 -- start start 2024-03-06T07:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.112+0000 7f6917ceb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6910153990 0x7f6910156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.112+0000 7f6917ceb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69100acc00 0x7f69100c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.112+0000 7f6917ceb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6910070aa0 con 0x7f69101506b0 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.112+0000 7f6917ceb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6910156450 con 0x7f69100acc00 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.112+0000 7f6917ceb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6910156a10 con 0x7f6910153990 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6916288640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69100acc00 0x7f69100c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6915a87640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f69101506b0 0x7f69100702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45262/0 (socket says 172.21.15.26:45262) 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6915a87640 1 -- 172.21.15.26:0/678534322 learned_addr learned my addr 172.21.15.26:0/678534322 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6916288640 1 -- 172.21.15.26:0/678534322 >> v2:172.21.15.117:3300/0 conn(0x7f6910153990 msgr2=0x7f6910156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6915286640 1 --2- 172.21.15.26:0/678534322 >> v2:172.21.15.117:3300/0 conn(0x7f6910153990 0x7f6910156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6916288640 1 --2- 172.21.15.26:0/678534322 >> v2:172.21.15.117:3300/0 conn(0x7f6910153990 0x7f6910156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6916288640 1 -- 172.21.15.26:0/678534322 >> v1:172.21.15.26:6789/0 conn(0x7f69101506b0 legacy=0x7f69100702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6914a85640 1 -- 172.21.15.26:0/678534322 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738989077 0 0) 0x7f6910070aa0 con 0x7f69101506b0 2024-03-06T07:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.113+0000 7f6916288640 1 -- 172.21.15.26:0/678534322 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910070d80 con 0x7f69100acc00 2024-03-06T07:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.114+0000 7f6916288640 1 --2- 172.21.15.26:0/678534322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f69100c54a0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f6910150a90 tx=0x7f690802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d4e9e4ed3f27a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.114+0000 7f6914a85640 1 -- 172.21.15.26:0/678534322 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908039070 con 0x7f69100acc00 2024-03-06T07:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.114+0000 7f6914a85640 1 -- 172.21.15.26:0/678534322 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690802cc80 con 0x7f69100acc00 2024-03-06T07:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.114+0000 7f6914a85640 1 -- 172.21.15.26:0/678534322 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908034660 con 0x7f69100acc00 2024-03-06T07:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 -- 172.21.15.26:0/678534322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 msgr2=0x7f69100c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 --2- 172.21.15.26:0/678534322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f69100c54a0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f6910150a90 tx=0x7f690802bed0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 -- 172.21.15.26:0/678534322 shutdown_connections 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 --2- 172.21.15.26:0/678534322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f69100c54a0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 --2- 172.21.15.26:0/678534322 >> v2:172.21.15.117:3300/0 conn(0x7f6910153990 0x7f6910156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 -- 172.21.15.26:0/678534322 >> 172.21.15.26:0/678534322 conn(0x7f691006b1a0 msgr2=0x7f691006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 -- 172.21.15.26:0/678534322 shutdown_connections 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.115+0000 7f6917ceb640 1 -- 172.21.15.26:0/678534322 wait complete. 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 Processor -- start 2024-03-06T07:31:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 -- start start 2024-03-06T07:31:04.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f6910148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69101506b0 0x7f69100c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6910153990 0x7f69100c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100c5b60 con 0x7f69100acc00 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100c59e0 con 0x7f6910153990 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.116+0000 7f6917ceb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69100c5ce0 con 0x7f69101506b0 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f6910148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f6910148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60814/0 (socket says 172.21.15.26:60814) 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 -- 172.21.15.26:0/2517557866 learned_addr learned my addr 172.21.15.26:0/2517557866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915286640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69101506b0 0x7f69100c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6916288640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6910153990 0x7f69100c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 -- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69101506b0 msgr2=0x7f69100c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69101506b0 0x7f69100c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 -- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6910153990 msgr2=0x7f69100c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6910153990 0x7f69100c3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 -- 172.21.15.26:0/2517557866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910154cb0 con 0x7f69100acc00 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915286640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69101506b0 0x7f69100c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6916288640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6910153990 0x7f69100c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6915a87640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f6910148ea0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c500 tx=0x7f690c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f69067fc640 1 -- 172.21.15.26:0/2517557866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c018070 con 0x7f69100acc00 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f69067fc640 1 -- 172.21.15.26:0/2517557866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c007580 con 0x7f69100acc00 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6910154380 con 0x7f69100acc00 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f69067fc640 1 -- 172.21.15.26:0/2517557866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c004a20 con 0x7f69100acc00 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.117+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6910154940 con 0x7f69100acc00 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.118+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691004f330 con 0x7f69100acc00 2024-03-06T07:31:04.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.119+0000 7f69067fc640 1 -- 172.21.15.26:0/2517557866 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f690c01b070 con 0x7f69100acc00 2024-03-06T07:31:04.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.120+0000 7f69067fc640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68f406b4d0 0x7f68f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.120+0000 7f69067fc640 1 -- 172.21.15.26:0/2517557866 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f690c0897b0 con 0x7f69100acc00 2024-03-06T07:31:04.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.120+0000 7f6915286640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68f406b4d0 0x7f68f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.120+0000 7f6915286640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68f406b4d0 0x7f68f406d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6900006fd0 tx=0x7f6900008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:04.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.124+0000 7f69067fc640 1 -- 172.21.15.26:0/2517557866 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f690c058390 con 0x7f69100acc00 2024-03-06T07:31:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.342+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69100700a0 con 0x7f69100acc00 2024-03-06T07:31:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.343+0000 7f69067fc640 1 -- 172.21.15.26:0/2517557866 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3425 (secure 0 0 0) 0x7f690c057c90 con 0x7f69100acc00 2024-03-06T07:31:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:04.345 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:30:33.691537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 214 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68f406b4d0 msgr2=0x7f68f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68f406b4d0 0x7f68f406d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6900006fd0 tx=0x7f6900008040 comp rx=0 tx=0).stop 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 msgr2=0x7f6910148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f6910148ea0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c500 tx=0x7f690c00c9c0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 shutdown_connections 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f68f406b4d0 0x7f68f406d990 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6910153990 0x7f69100c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69101506b0 0x7f69100c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 --2- 172.21.15.26:0/2517557866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69100acc00 0x7f6910148ea0 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 >> 172.21.15.26:0/2517557866 conn(0x7f691006b1a0 msgr2=0x7f691014b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 shutdown_connections 2024-03-06T07:31:04.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:04.346+0000 7f6917ceb640 1 -- 172.21.15.26:0/2517557866 wait complete. 2024-03-06T07:31:04.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:30:33.691537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:04.764 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:05.765 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.864+0000 7fe32a62a640 1 Processor -- start 2024-03-06T07:31:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.864+0000 7fe32a62a640 1 -- start start 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.864+0000 7fe32a62a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe3241506b0 0x7fe324153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.864+0000 7fe32a62a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3240c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe32a62a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe324070aa0 con 0x7fe324156ea0 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe32a62a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe324156450 con 0x7fe3241506b0 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe32a62a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe324156a10 con 0x7fe3240acc00 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe323fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe3241506b0 0x7fe324153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe3237fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe324156ea0 0x7fe3240702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45268/0 (socket says 172.21.15.26:45268) 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe323fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe3241506b0 0x7fe324153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60828/0 (socket says 172.21.15.26:60828) 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe3237fe640 1 -- 172.21.15.26:0/3677003783 learned_addr learned my addr 172.21.15.26:0/3677003783 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe328bc7640 1 --2- 172.21.15.26:0/3677003783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3240c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe323fff640 1 -- 172.21.15.26:0/3677003783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 msgr2=0x7fe3240c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:05.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe323fff640 1 --2- 172.21.15.26:0/3677003783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3240c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:05.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe323fff640 1 -- 172.21.15.26:0/3677003783 >> v1:172.21.15.26:6789/0 conn(0x7fe324156ea0 legacy=0x7fe3240702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:05.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe322ffd640 1 -- 172.21.15.26:0/3677003783 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761575010 0 0) 0x7fe324070aa0 con 0x7fe324156ea0 2024-03-06T07:31:05.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.865+0000 7fe323fff640 1 -- 172.21.15.26:0/3677003783 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324070d80 con 0x7fe3241506b0 2024-03-06T07:31:05.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe323fff640 1 --2- 172.21.15.26:0/3677003783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe324153990 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fe3140097a0 tx=0x7fe31402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c556e030837b99 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:05.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe322ffd640 1 -- 172.21.15.26:0/3677003783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314035020 con 0x7fe3241506b0 2024-03-06T07:31:05.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe322ffd640 1 -- 172.21.15.26:0/3677003783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe314031830 con 0x7fe3241506b0 2024-03-06T07:31:05.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe322ffd640 1 -- 172.21.15.26:0/3677003783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314031c00 con 0x7fe3241506b0 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe32a62a640 1 -- 172.21.15.26:0/3677003783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 msgr2=0x7fe324153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3677003783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe324153990 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fe3140097a0 tx=0x7fe31402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe32a62a640 1 -- 172.21.15.26:0/3677003783 shutdown_connections 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3677003783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3240c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3677003783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe324153990 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.866+0000 7fe32a62a640 1 -- 172.21.15.26:0/3677003783 >> 172.21.15.26:0/3677003783 conn(0x7fe32406b1a0 msgr2=0x7fe32406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.867+0000 7fe32a62a640 1 -- 172.21.15.26:0/3677003783 shutdown_connections 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.867+0000 7fe32a62a640 1 -- 172.21.15.26:0/3677003783 wait complete. 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 Processor -- start 2024-03-06T07:31:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 -- start start 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3241ec830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe3241494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe324156ea0 0x7fe3240c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240ad1d0 con 0x7fe3241506b0 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240ad050 con 0x7fe324156ea0 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.868+0000 7fe32a62a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240ad350 con 0x7fe3240acc00 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe3241494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe3241494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60830/0 (socket says 172.21.15.26:60830) 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 -- 172.21.15.26:0/3934171166 learned_addr learned my addr 172.21.15.26:0/3934171166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 -- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 msgr2=0x7fe3241ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe328bc7640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe324156ea0 0x7fe3240c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe323fff640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3241ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3241ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 -- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe324156ea0 msgr2=0x7fe3240c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe324156ea0 0x7fe3240c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 -- 172.21.15.26:0/3934171166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324154280 con 0x7fe3241506b0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe323fff640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3241ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe328bc7640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe324156ea0 0x7fe3240c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe3237fe640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe3241494b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c410 tx=0x7fe31800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe320ff9640 1 -- 172.21.15.26:0/3934171166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3180074e0 con 0x7fe3241506b0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe320ff9640 1 -- 172.21.15.26:0/3934171166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3180047e0 con 0x7fe3241506b0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe324147220 con 0x7fe3241506b0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe320ff9640 1 -- 172.21.15.26:0/3934171166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318004bd0 con 0x7fe3241506b0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.869+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3241499f0 con 0x7fe3241506b0 2024-03-06T07:31:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.870+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32404f330 con 0x7fe3241506b0 2024-03-06T07:31:05.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.871+0000 7fe320ff9640 1 -- 172.21.15.26:0/3934171166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe318011070 con 0x7fe3241506b0 2024-03-06T07:31:05.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.871+0000 7fe320ff9640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe31006b4d0 0x7fe31006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:05.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.871+0000 7fe320ff9640 1 -- 172.21.15.26:0/3934171166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe31808a020 con 0x7fe3241506b0 2024-03-06T07:31:05.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.871+0000 7fe323fff640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe31006b4d0 0x7fe31006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:05.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.872+0000 7fe323fff640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe31006b4d0 0x7fe31006d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe314036000 tx=0x7fe314008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:05.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:05.875+0000 7fe320ff9640 1 -- 172.21.15.26:0/3934171166 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe318058ca0 con 0x7fe3241506b0 2024-03-06T07:31:06.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.096+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3240700a0 con 0x7fe3241506b0 2024-03-06T07:31:06.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.096+0000 7fe320ff9640 1 -- 172.21.15.26:0/3934171166 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fe3180585a0 con 0x7fe3241506b0 2024-03-06T07:31:06.099 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:06.099 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:06.099 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:06.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe31006b4d0 msgr2=0x7fe31006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe31006b4d0 0x7fe31006d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe314036000 tx=0x7fe314008030 comp rx=0 tx=0).stop 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 msgr2=0x7fe3241494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe3241494b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c410 tx=0x7fe31800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 shutdown_connections 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe31006b4d0 0x7fe31006d990 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe324156ea0 0x7fe3240c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3241506b0 0x7fe3241494b0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 --2- 172.21.15.26:0/3934171166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3240acc00 0x7fe3241ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 >> 172.21.15.26:0/3934171166 conn(0x7fe32406b1a0 msgr2=0x7fe3240c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 shutdown_connections 2024-03-06T07:31:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:06.099+0000 7fe32a62a640 1 -- 172.21.15.26:0/3934171166 wait complete. 2024-03-06T07:31:06.518 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:06.518 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:07.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.619+0000 7f3614803640 1 Processor -- start 2024-03-06T07:31:07.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.619+0000 7f3614803640 1 -- start start 2024-03-06T07:31:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f3614803640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f36101559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f3614803640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3610158a40 0x7f3610158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f3614803640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361014be70 con 0x7f361014ba90 2024-03-06T07:31:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f3614803640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36101593d0 con 0x7f3610158a40 2024-03-06T07:31:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f3614803640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36100cad30 con 0x7f36100c8690 2024-03-06T07:31:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f360ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f361014ba90 0x7f36100c5110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40430/0 (socket says 172.21.15.26:40430) 2024-03-06T07:31:07.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f360ed9d640 1 -- 172.21.15.26:0/3490817503 learned_addr learned my addr 172.21.15.26:0/3490817503 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f360f59e640 1 --2- 172.21.15.26:0/3490817503 >> v2:172.21.15.117:3300/0 conn(0x7f3610158a40 0x7f3610158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f360e59c640 1 --2- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f36101559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f360dd9b640 1 -- 172.21.15.26:0/3490817503 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283961218 0 0) 0x7f35f8001110 con 0x7f361014ba90 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.620+0000 7f360dd9b640 1 -- 172.21.15.26:0/3490817503 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35fc003520 con 0x7f361014ba90 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.621+0000 7f360f59e640 1 -- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 msgr2=0x7f36101559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.621+0000 7f360f59e640 1 --2- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f36101559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.621+0000 7f360f59e640 1 -- 172.21.15.26:0/3490817503 >> v1:172.21.15.26:6789/0 conn(0x7f361014ba90 legacy=0x7f36100c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.621+0000 7f360f59e640 1 -- 172.21.15.26:0/3490817503 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100cb010 con 0x7f3610158a40 2024-03-06T07:31:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.621+0000 7f360e59c640 1 --2- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f36101559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.621+0000 7f360f59e640 1 --2- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3610158a40 0x7f3610158e90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3608009d90 tx=0x7f360802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b03d5d7a58b70ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f360dd9b640 1 -- 172.21.15.26:0/3490817503 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608039070 con 0x7f3610158a40 2024-03-06T07:31:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f360dd9b640 1 -- 172.21.15.26:0/3490817503 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360802cc50 con 0x7f3610158a40 2024-03-06T07:31:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f360dd9b640 1 -- 172.21.15.26:0/3490817503 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608034440 con 0x7f3610158a40 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 -- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3610158a40 msgr2=0x7f3610158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 --2- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3610158a40 0x7f3610158e90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3608009d90 tx=0x7f360802bea0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 -- 172.21.15.26:0/3490817503 shutdown_connections 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 --2- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3610158a40 0x7f3610158e90 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 --2- 172.21.15.26:0/3490817503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f36101559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 -- 172.21.15.26:0/3490817503 >> 172.21.15.26:0/3490817503 conn(0x7f36100728d0 msgr2=0x7f3610072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 -- 172.21.15.26:0/3490817503 shutdown_connections 2024-03-06T07:31:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.622+0000 7f3614803640 1 -- 172.21.15.26:0/3490817503 wait complete. 2024-03-06T07:31:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.624+0000 7f3614803640 1 Processor -- start 2024-03-06T07:31:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.624+0000 7f3614803640 1 -- start start 2024-03-06T07:31:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.625+0000 7f3614803640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f3610152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f3614803640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f361014ba90 0x7f3610153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f3614803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3610158a40 0x7f3610144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f3614803640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361006cf60 con 0x7f3610158a40 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f3614803640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3610158520 con 0x7f361014ba90 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f3614803640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36100c5820 con 0x7f36100c8690 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3610158a40 0x7f3610144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3610158a40 0x7f3610144f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56078/0 (socket says 172.21.15.26:56078) 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 -- 172.21.15.26:0/1646513997 learned_addr learned my addr 172.21.15.26:0/1646513997 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360ed9d640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f3610152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360e59c640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f361014ba90 0x7f3610153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 -- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 msgr2=0x7f3610152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f3610152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 -- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f361014ba90 msgr2=0x7f3610153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f361014ba90 0x7f3610153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 -- 172.21.15.26:0/1646513997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36101ebab0 con 0x7f3610158a40 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360ed9d640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f3610152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360e59c640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f361014ba90 0x7f3610153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f360f59e640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3610158a40 0x7f3610144f60 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f3608009d90 tx=0x7f3608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f36077fe640 1 -- 172.21.15.26:0/1646513997 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608042030 con 0x7f3610158a40 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f36077fe640 1 -- 172.21.15.26:0/1646513997 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360803a040 con 0x7f3610158a40 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3610154770 con 0x7f3610158a40 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f36077fe640 1 -- 172.21.15.26:0/1646513997 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608039070 con 0x7f3610158a40 2024-03-06T07:31:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.626+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3610153fd0 con 0x7f3610158a40 2024-03-06T07:31:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.628+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3610069430 con 0x7f3610158a40 2024-03-06T07:31:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.628+0000 7f36077fe640 1 -- 172.21.15.26:0/1646513997 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3608045070 con 0x7f3610158a40 2024-03-06T07:31:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.628+0000 7f36077fe640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35f006b4d0 0x7f35f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.628+0000 7f36077fe640 1 -- 172.21.15.26:0/1646513997 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f36080abfc0 con 0x7f3610158a40 2024-03-06T07:31:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.628+0000 7f360ed9d640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35f006b4d0 0x7f35f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.629+0000 7f360ed9d640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35f006b4d0 0x7f35f006d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f35f80012b0 tx=0x7f35f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.632+0000 7f36077fe640 1 -- 172.21.15.26:0/1646513997 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f360804c070 con 0x7f3610158a40 2024-03-06T07:31:07.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.851+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3610072d50 con 0x7f3610158a40 2024-03-06T07:31:07.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.852+0000 7f36077fe640 1 -- 172.21.15.26:0/1646513997 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f360807ad00 con 0x7f3610158a40 2024-03-06T07:31:07.855 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:07.855 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:07.855 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35f006b4d0 msgr2=0x7f35f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35f006b4d0 0x7f35f006d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f35f80012b0 tx=0x7f35f8009040 comp rx=0 tx=0).stop 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3610158a40 msgr2=0x7f3610144f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3610158a40 0x7f3610144f60 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f3608009d90 tx=0x7f3608004a40 comp rx=0 tx=0).stop 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 shutdown_connections 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f35f006b4d0 0x7f35f006d990 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3610158a40 0x7f3610144f60 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f361014ba90 0x7f3610153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 --2- 172.21.15.26:0/1646513997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36100c8690 0x7f3610152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 >> 172.21.15.26:0/1646513997 conn(0x7f36100728d0 msgr2=0x7f36100aa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 shutdown_connections 2024-03-06T07:31:07.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:07.854+0000 7f3614803640 1 -- 172.21.15.26:0/1646513997 wait complete. 2024-03-06T07:31:08.270 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:08.270 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:09.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:09.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.370+0000 7ffaad1a2640 1 Processor -- start 2024-03-06T07:31:09.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.370+0000 7ffaad1a2640 1 -- start start 2024-03-06T07:31:09.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaad1a2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffaa80c4530 0x7ffaa80c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaad1a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa8152d40 con 0x7ffaa806e6b0 2024-03-06T07:31:09.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaad1a2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa8152fe0 con 0x7ffaa8150430 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaad1a2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa8153910 con 0x7ffaa80c4530 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaac9a1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffaa806e6b0 0x7ffaa80c24f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40438/0 (socket says 172.21.15.26:40438) 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaac9a1640 1 -- 172.21.15.26:0/538366769 learned_addr learned my addr 172.21.15.26:0/538366769 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaa77fe640 1 --2- 172.21.15.26:0/538366769 >> v2:172.21.15.117:3300/0 conn(0x7ffaa80c4530 0x7ffaa80c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaa6ffd640 1 -- 172.21.15.26:0/538366769 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219485532 0 0) 0x7ffaa8152d40 con 0x7ffaa806e6b0 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaa6ffd640 1 -- 172.21.15.26:0/538366769 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa90003520 con 0x7ffaa806e6b0 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaa6ffd640 1 -- 172.21.15.26:0/538366769 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993788011 0 0) 0x7ffaa8152fe0 con 0x7ffaa8150430 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.371+0000 7ffaa6ffd640 1 -- 172.21.15.26:0/538366769 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa8152d40 con 0x7ffaa8150430 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.372+0000 7ffaa77fe640 1 -- 172.21.15.26:0/538366769 >> v1:172.21.15.117:6789/0 conn(0x7ffaa8150430 legacy=0x7ffaa80c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.372+0000 7ffaa77fe640 1 -- 172.21.15.26:0/538366769 >> v1:172.21.15.26:6789/0 conn(0x7ffaa806e6b0 legacy=0x7ffaa80c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.372+0000 7ffaa77fe640 1 -- 172.21.15.26:0/538366769 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa81532c0 con 0x7ffaa80c4530 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.372+0000 7ffaa77fe640 1 --2- 172.21.15.26:0/538366769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa80c1fb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097a0 tx=0x7ffa9c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72abb7bc04aa0e36 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaa6ffd640 1 -- 172.21.15.26:0/538366769 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c035020 con 0x7ffaa80c4530 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaa6ffd640 1 -- 172.21.15.26:0/538366769 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c031830 con 0x7ffaa80c4530 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaa6ffd640 1 -- 172.21.15.26:0/538366769 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c031be0 con 0x7ffaa80c4530 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaad1a2640 1 -- 172.21.15.26:0/538366769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 msgr2=0x7ffaa80c1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/538366769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa80c1fb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097a0 tx=0x7ffa9c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaad1a2640 1 -- 172.21.15.26:0/538366769 shutdown_connections 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/538366769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa80c1fb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaad1a2640 1 -- 172.21.15.26:0/538366769 >> 172.21.15.26:0/538366769 conn(0x7ffaa806c0d0 msgr2=0x7ffaa806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaad1a2640 1 -- 172.21.15.26:0/538366769 shutdown_connections 2024-03-06T07:31:09.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.373+0000 7ffaad1a2640 1 -- 172.21.15.26:0/538366769 wait complete. 2024-03-06T07:31:09.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.374+0000 7ffaad1a2640 1 Processor -- start 2024-03-06T07:31:09.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.374+0000 7ffaad1a2640 1 -- start start 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaad1a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaa806e6b0 0x7ffaa8148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaad1a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa81553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaad1a2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffaa8150430 0x7ffaa8147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaad1a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa806e150 con 0x7ffaa806e6b0 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaad1a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80c2c00 con 0x7ffaa80c4530 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaad1a2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa806e2d0 con 0x7ffaa8150430 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaa806e6b0 0x7ffaa8148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaa806e6b0 0x7ffaa8148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56084/0 (socket says 172.21.15.26:56084) 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 -- 172.21.15.26:0/2111391519 learned_addr learned my addr 172.21.15.26:0/2111391519 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaac9a1640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffaa8150430 0x7ffaa8147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa77fe640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa81553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 -- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffaa8150430 msgr2=0x7ffaa8147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffaa8150430 0x7ffaa8147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 -- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 msgr2=0x7ffaa81553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa81553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa7fff640 1 -- 172.21.15.26:0/2111391519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8155dc0 con 0x7ffaa806e6b0 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaac9a1640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffaa8150430 0x7ffaa8147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.375+0000 7ffaa77fe640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa81553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.376+0000 7ffaa7fff640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaa806e6b0 0x7ffaa8148e20 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c460 tx=0x7ffa9800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.376+0000 7ffaa4ff9640 1 -- 172.21.15.26:0/2111391519 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98017070 con 0x7ffaa806e6b0 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.376+0000 7ffaa4ff9640 1 -- 172.21.15.26:0/2111391519 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa980052f0 con 0x7ffaa806e6b0 2024-03-06T07:31:09.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.376+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa81ec5a0 con 0x7ffaa806e6b0 2024-03-06T07:31:09.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.376+0000 7ffaa4ff9640 1 -- 172.21.15.26:0/2111391519 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98003830 con 0x7ffaa806e6b0 2024-03-06T07:31:09.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.376+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa80c5220 con 0x7ffaa806e6b0 2024-03-06T07:31:09.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.377+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8069430 con 0x7ffaa806e6b0 2024-03-06T07:31:09.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.377+0000 7ffaa4ff9640 1 -- 172.21.15.26:0/2111391519 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa9801a070 con 0x7ffaa806e6b0 2024-03-06T07:31:09.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.378+0000 7ffaa4ff9640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa9406b4d0 0x7ffa9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.378+0000 7ffaa4ff9640 1 -- 172.21.15.26:0/2111391519 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7ffa98089870 con 0x7ffaa806e6b0 2024-03-06T07:31:09.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.378+0000 7ffaa77fe640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa9406b4d0 0x7ffa9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.379+0000 7ffaa77fe640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa9406b4d0 0x7ffa9406d990 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c036000 tx=0x7ffa9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.382+0000 7ffaa4ff9640 1 -- 172.21.15.26:0/2111391519 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffaa8155dc0 con 0x7ffaa806e6b0 2024-03-06T07:31:09.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.606+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa806eb50 con 0x7ffaa806e6b0 2024-03-06T07:31:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.606+0000 7ffaa4ff9640 1 -- 172.21.15.26:0/2111391519 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7ffa98058460 con 0x7ffaa806e6b0 2024-03-06T07:31:09.609 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:09.609 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa9406b4d0 msgr2=0x7ffa9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa9406b4d0 0x7ffa9406d990 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c036000 tx=0x7ffa9c008030 comp rx=0 tx=0).stop 2024-03-06T07:31:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaa806e6b0 msgr2=0x7ffaa8148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaa806e6b0 0x7ffaa8148e20 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c460 tx=0x7ffa9800c920 comp rx=0 tx=0).stop 2024-03-06T07:31:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 shutdown_connections 2024-03-06T07:31:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ffa9406b4d0 0x7ffa9406d990 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffaa8150430 0x7ffaa8147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa80c4530 0x7ffaa81553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 --2- 172.21.15.26:0/2111391519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaa806e6b0 0x7ffaa8148e20 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 >> 172.21.15.26:0/2111391519 conn(0x7ffaa806c0d0 msgr2=0x7ffaa814da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 shutdown_connections 2024-03-06T07:31:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:09.608+0000 7ffaad1a2640 1 -- 172.21.15.26:0/2111391519 wait complete. 2024-03-06T07:31:10.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:10.024 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:11.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.126+0000 7fab8dff7640 1 Processor -- start 2024-03-06T07:31:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.126+0000 7fab8dff7640 1 -- start start 2024-03-06T07:31:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab8dff7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab8806d9f0 0x7fab880a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab8dff7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab880c3710 0x7fab881558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab8dff7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab88158800 con 0x7fab880c3280 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab8dff7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab88158ae0 con 0x7fab8806d9f0 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab8dff7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab88158dc0 con 0x7fab880c3710 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab8806d9f0 0x7fab880a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab877fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab8806d9f0 0x7fab880a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56100/0 (socket says 172.21.15.26:56100) 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab86ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fab880c3280 0x7fab881551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40454/0 (socket says 172.21.15.26:40454) 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.127+0000 7fab877fe640 1 -- 172.21.15.26:0/1791096116 learned_addr learned my addr 172.21.15.26:0/1791096116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab87fff640 1 --2- 172.21.15.26:0/1791096116 >> v2:172.21.15.117:3300/0 conn(0x7fab880c3710 0x7fab881558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab877fe640 1 -- 172.21.15.26:0/1791096116 >> v2:172.21.15.117:3300/0 conn(0x7fab880c3710 msgr2=0x7fab881558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab877fe640 1 --2- 172.21.15.26:0/1791096116 >> v2:172.21.15.117:3300/0 conn(0x7fab880c3710 0x7fab881558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab877fe640 1 -- 172.21.15.26:0/1791096116 >> v1:172.21.15.26:6789/0 conn(0x7fab880c3280 legacy=0x7fab881551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab867fc640 1 -- 172.21.15.26:0/1791096116 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754560556 0 0) 0x7fab88158800 con 0x7fab880c3280 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab877fe640 1 -- 172.21.15.26:0/1791096116 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab881590a0 con 0x7fab8806d9f0 2024-03-06T07:31:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab87fff640 1 --2- 172.21.15.26:0/1791096116 >> v2:172.21.15.117:3300/0 conn(0x7fab880c3710 0x7fab881558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.128+0000 7fab877fe640 1 --2- 172.21.15.26:0/1791096116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab880a7cd0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fab88159760 tx=0x7fab7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2809dd449f3cf28 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab867fc640 1 -- 172.21.15.26:0/1791096116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c035020 con 0x7fab8806d9f0 2024-03-06T07:31:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab867fc640 1 -- 172.21.15.26:0/1791096116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c031870 con 0x7fab8806d9f0 2024-03-06T07:31:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab867fc640 1 -- 172.21.15.26:0/1791096116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c031c40 con 0x7fab8806d9f0 2024-03-06T07:31:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 -- 172.21.15.26:0/1791096116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 msgr2=0x7fab880a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 --2- 172.21.15.26:0/1791096116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab880a7cd0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fab88159760 tx=0x7fab7c02b910 comp rx=0 tx=0).stop 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 -- 172.21.15.26:0/1791096116 shutdown_connections 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 --2- 172.21.15.26:0/1791096116 >> v2:172.21.15.117:3300/0 conn(0x7fab880c3710 0x7fab881558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 --2- 172.21.15.26:0/1791096116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab880a7cd0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 -- 172.21.15.26:0/1791096116 >> 172.21.15.26:0/1791096116 conn(0x7fab880aa630 msgr2=0x7fab880aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 -- 172.21.15.26:0/1791096116 shutdown_connections 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.129+0000 7fab8dff7640 1 -- 172.21.15.26:0/1791096116 wait complete. 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.130+0000 7fab8dff7640 1 Processor -- start 2024-03-06T07:31:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.130+0000 7fab8dff7640 1 -- start start 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab8dff7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab88157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab8dff7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab880c3280 0x7fab881437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab8dff7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab880c3710 0x7fab88143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab8dff7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8806ddd0 con 0x7fab8806d9f0 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab8dff7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab88155e30 con 0x7fab880c3710 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab8dff7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8806df50 con 0x7fab880c3280 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab88157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab88157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56102/0 (socket says 172.21.15.26:56102) 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab877fe640 1 -- 172.21.15.26:0/3144738032 learned_addr learned my addr 172.21.15.26:0/3144738032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab87fff640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab880c3710 0x7fab88143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.131+0000 7fab86ffd640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab880c3280 0x7fab881437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab877fe640 1 -- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab880c3280 msgr2=0x7fab881437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab877fe640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab880c3280 0x7fab881437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab877fe640 1 -- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab880c3710 msgr2=0x7fab88143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab877fe640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab880c3710 0x7fab88143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab877fe640 1 -- 172.21.15.26:0/3144738032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab881eea00 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab877fe640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab88157530 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab6ffff640 1 -- 172.21.15.26:0/3144738032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c041070 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab6ffff640 1 -- 172.21.15.26:0/3144738032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c004030 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab88144240 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab6ffff640 1 -- 172.21.15.26:0/3144738032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c035040 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.132+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab880abac0 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.133+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8804f330 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.134+0000 7fab6ffff640 1 -- 172.21.15.26:0/3144738032 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab7c038020 con 0x7fab8806d9f0 2024-03-06T07:31:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.134+0000 7fab6ffff640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7406b500 0x7fab7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.134+0000 7fab6ffff640 1 -- 172.21.15.26:0/3144738032 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fab7c0ac300 con 0x7fab8806d9f0 2024-03-06T07:31:11.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.134+0000 7fab86ffd640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7406b500 0x7fab7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.135+0000 7fab86ffd640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7406b500 0x7fab7406d9c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fab8806d3c0 tx=0x7fab78007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.138+0000 7fab6ffff640 1 -- 172.21.15.26:0/3144738032 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab7c07a930 con 0x7fab8806d9f0 2024-03-06T07:31:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.356+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab88073240 con 0x7fab8806d9f0 2024-03-06T07:31:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.356+0000 7fab6ffff640 1 -- 172.21.15.26:0/3144738032 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fab7c07e9d0 con 0x7fab8806d9f0 2024-03-06T07:31:11.359 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:11.359 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7406b500 msgr2=0x7fab7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7406b500 0x7fab7406d9c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fab8806d3c0 tx=0x7fab78007d50 comp rx=0 tx=0).stop 2024-03-06T07:31:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 msgr2=0x7fab88157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab88157530 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 shutdown_connections 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fab7406b500 0x7fab7406d9c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab880c3710 0x7fab88143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab880c3280 0x7fab881437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.358+0000 7fab8dff7640 1 --2- 172.21.15.26:0/3144738032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8806d9f0 0x7fab88157530 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.359+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 >> 172.21.15.26:0/3144738032 conn(0x7fab880aa630 msgr2=0x7fab880c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.359+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 shutdown_connections 2024-03-06T07:31:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:11.359+0000 7fab8dff7640 1 -- 172.21.15.26:0/3144738032 wait complete. 2024-03-06T07:31:11.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:11.775 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:12.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:12.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.875+0000 7f4b1f883640 1 Processor -- start 2024-03-06T07:31:12.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1f883640 1 -- start start 2024-03-06T07:31:12.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1f883640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b18155ba0 0x7f4b180c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1f883640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b181591c0 con 0x7f4b180c5480 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1f883640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b181594a0 con 0x7f4b181586f0 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1f883640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1806df40 con 0x7f4b18155ba0 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1ce1e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b18155ba0 0x7f4b180c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1d61f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4b180c5480 0x7f4b1806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40470/0 (socket says 172.21.15.26:40470) 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.876+0000 7f4b1d61f640 1 -- 172.21.15.26:0/3511533734 learned_addr learned my addr 172.21.15.26:0/3511533734 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.877+0000 7f4b1ce1e640 1 -- 172.21.15.26:0/3511533734 >> v1:172.21.15.117:6789/0 conn(0x7f4b181586f0 legacy=0x7f4b1815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.877+0000 7f4b1ce1e640 1 -- 172.21.15.26:0/3511533734 >> v1:172.21.15.26:6789/0 conn(0x7f4b180c5480 legacy=0x7f4b1806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.877+0000 7f4b17fff640 1 -- 172.21.15.26:0/3511533734 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754443431 0 0) 0x7f4b181591c0 con 0x7f4b180c5480 2024-03-06T07:31:12.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.877+0000 7f4b1ce1e640 1 -- 172.21.15.26:0/3511533734 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b181553a0 con 0x7f4b18155ba0 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1ce1e640 1 --2- 172.21.15.26:0/3511533734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b180c5860 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097a0 tx=0x7f4b0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7709d74a72432c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b17fff640 1 -- 172.21.15.26:0/3511533734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0315a0 con 0x7f4b18155ba0 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b17fff640 1 -- 172.21.15.26:0/3511533734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c037070 con 0x7f4b18155ba0 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b17fff640 1 -- 172.21.15.26:0/3511533734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c031960 con 0x7f4b18155ba0 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1f883640 1 -- 172.21.15.26:0/3511533734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 msgr2=0x7f4b180c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1f883640 1 --2- 172.21.15.26:0/3511533734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b180c5860 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097a0 tx=0x7f4b0c02b840 comp rx=0 tx=0).stop 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1f883640 1 -- 172.21.15.26:0/3511533734 shutdown_connections 2024-03-06T07:31:12.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1f883640 1 --2- 172.21.15.26:0/3511533734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b180c5860 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:12.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1f883640 1 -- 172.21.15.26:0/3511533734 >> 172.21.15.26:0/3511533734 conn(0x7f4b1806b1a0 msgr2=0x7f4b1806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:12.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1f883640 1 -- 172.21.15.26:0/3511533734 shutdown_connections 2024-03-06T07:31:12.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.878+0000 7f4b1f883640 1 -- 172.21.15.26:0/3511533734 wait complete. 2024-03-06T07:31:12.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.879+0000 7f4b1f883640 1 Processor -- start 2024-03-06T07:31:12.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.879+0000 7f4b1f883640 1 -- start start 2024-03-06T07:31:12.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1f883640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b180c5480 0x7f4b181eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:12.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1f883640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b181516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:12.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1f883640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b181586f0 0x7f4b1814b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:12.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1f883640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18156000 con 0x7f4b18155ba0 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1f883640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1806fdb0 con 0x7f4b180c5480 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1f883640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18152a20 con 0x7f4b181586f0 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b181516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b181516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56118/0 (socket says 172.21.15.26:56118) 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 -- 172.21.15.26:0/2552415002 learned_addr learned my addr 172.21.15.26:0/2552415002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1d61f640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b180c5480 0x7f4b181eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1de20640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b181586f0 0x7f4b1814b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 -- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b181586f0 msgr2=0x7f4b1814b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b181586f0 0x7f4b1814b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 -- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b180c5480 msgr2=0x7f4b181eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b180c5480 0x7f4b181eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1ce1e640 1 -- 172.21.15.26:0/2552415002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b181538b0 con 0x7f4b18155ba0 2024-03-06T07:31:12.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.880+0000 7f4b1d61f640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b180c5480 0x7f4b181eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.881+0000 7f4b1ce1e640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b181516a0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098c0 tx=0x7f4b0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.881+0000 7f4b15ffb640 1 -- 172.21.15.26:0/2552415002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c032840 con 0x7f4b18155ba0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.881+0000 7f4b15ffb640 1 -- 172.21.15.26:0/2552415002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c0329e0 con 0x7f4b18155ba0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.881+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1814c1c0 con 0x7f4b18155ba0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.881+0000 7f4b15ffb640 1 -- 172.21.15.26:0/2552415002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c032dd0 con 0x7f4b18155ba0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.881+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18153b50 con 0x7f4b18155ba0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.882+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1804f330 con 0x7f4b18155ba0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.882+0000 7f4b15ffb640 1 -- 172.21.15.26:0/2552415002 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b0c03b3f0 con 0x7f4b18155ba0 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.883+0000 7f4b15ffb640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b0406b500 0x7f4b0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:12.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.883+0000 7f4b15ffb640 1 -- 172.21.15.26:0/2552415002 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f4b0c02f030 con 0x7f4b18155ba0 2024-03-06T07:31:12.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.883+0000 7f4b1d61f640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b0406b500 0x7f4b0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:12.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.884+0000 7f4b1d61f640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b0406b500 0x7f4b0406d9c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000f60 tx=0x7f4b10007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:12.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:12.887+0000 7f4b15ffb640 1 -- 172.21.15.26:0/2552415002 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b0c040070 con 0x7f4b18155ba0 2024-03-06T07:31:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.134+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b1806e0c0 con 0x7f4b18155ba0 2024-03-06T07:31:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.134+0000 7f4b15ffb640 1 -- 172.21.15.26:0/2552415002 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f4b0c07b0b0 con 0x7f4b18155ba0 2024-03-06T07:31:13.137 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:13.137 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:13.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b0406b500 msgr2=0x7f4b0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b0406b500 0x7f4b0406d9c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000f60 tx=0x7f4b10007c10 comp rx=0 tx=0).stop 2024-03-06T07:31:13.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 msgr2=0x7f4b181516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b181516a0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098c0 tx=0x7f4b0c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 shutdown_connections 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4b0406b500 0x7f4b0406d9c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b181586f0 0x7f4b1814b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b18155ba0 0x7f4b181516a0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 --2- 172.21.15.26:0/2552415002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b180c5480 0x7f4b181eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 >> 172.21.15.26:0/2552415002 conn(0x7f4b1806b1a0 msgr2=0x7f4b1806c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 shutdown_connections 2024-03-06T07:31:13.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:13.136+0000 7f4b1f883640 1 -- 172.21.15.26:0/2552415002 wait complete. 2024-03-06T07:31:13.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:13.453 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:13.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:13.552 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:14.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.652+0000 7f6e86a42640 1 Processor -- start 2024-03-06T07:31:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.652+0000 7f6e86a42640 1 -- start start 2024-03-06T07:31:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e86a42640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e8014f360 0x7f6e80155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e86a42640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800c73d0 con 0x7f6e800c88c0 2024-03-06T07:31:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e86a42640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800be2e0 con 0x7f6e80158b60 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e86a42640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e80159700 con 0x7f6e8014f360 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e800c88c0 0x7f6e800c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40472/0 (socket says 172.21.15.26:40472) 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7f7fe640 1 -- 172.21.15.26:0/2021536498 learned_addr learned my addr 172.21.15.26:0/2021536498 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2021536498 >> v2:172.21.15.117:3300/0 conn(0x7f6e8014f360 0x7f6e80155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684711765 0 0) 0x7f6e800c73d0 con 0x7f6e800c88c0 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e68003520 con 0x7f6e800c88c0 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735313078 0 0) 0x7f6e800be2e0 con 0x7f6e80158b60 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e800c73d0 con 0x7f6e80158b60 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 105496192 0 0) 0x7f6e68003520 con 0x7f6e800c88c0 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 >> v2:172.21.15.117:3300/0 conn(0x7f6e8014f360 msgr2=0x7f6e80155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 --2- 172.21.15.26:0/2021536498 >> v2:172.21.15.117:3300/0 conn(0x7f6e8014f360 0x7f6e80155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 >> v1:172.21.15.117:6789/0 conn(0x7f6e80158b60 legacy=0x7f6e80158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800c7670 con 0x7f6e800c88c0 2024-03-06T07:31:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6e74001670 con 0x7f6e800c88c0 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2021536498 >> v2:172.21.15.117:3300/0 conn(0x7f6e8014f360 0x7f6e80155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 >> v1:172.21.15.26:6789/0 conn(0x7f6e800c88c0 legacy=0x7f6e800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e68003b40 0x7f6e68003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.654+0000 7f6e7effd640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7effd640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e680051c0 0x7f6e680141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e80159700 con 0x7f6e68003b40 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68014900 con 0x7f6e68004800 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68014b00 con 0x7f6e680051c0 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e68003b40 0x7f6e68003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7f7fe640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e84fdf640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e680051c0 0x7f6e680141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7ffff640 1 -- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e680051c0 msgr2=0x7f6e680141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e680051c0 0x7f6e680141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7ffff640 1 -- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e68004800 msgr2=0x7f6e68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e7ffff640 1 -- 172.21.15.26:0/2021536498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68003520 con 0x7f6e68003b40 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e84fdf640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e680051c0 0x7f6e680141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.655+0000 7f6e84fdf640 1 -- 172.21.15.26:0/2021536498 reap_dead start 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e68003b40 0x7f6e68003f40 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009c90 tx=0x7f6e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19253ef6d3fcf3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70039070 con 0x7f6e68003b40 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7002c8f0 con 0x7f6e68003b40 2024-03-06T07:31:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e7effd640 1 -- 172.21.15.26:0/2021536498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70009860 con 0x7f6e68003b40 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e86a42640 1 -- 172.21.15.26:0/2021536498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e68003b40 msgr2=0x7f6e68003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e68003b40 0x7f6e68003f40 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009c90 tx=0x7f6e68005830 comp rx=0 tx=0).stop 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e86a42640 1 -- 172.21.15.26:0/2021536498 shutdown_connections 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2021536498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e68003b40 0x7f6e68003f40 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e86a42640 1 -- 172.21.15.26:0/2021536498 >> 172.21.15.26:0/2021536498 conn(0x7f6e800ac570 msgr2=0x7f6e800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e86a42640 1 -- 172.21.15.26:0/2021536498 shutdown_connections 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.656+0000 7f6e86a42640 1 -- 172.21.15.26:0/2021536498 wait complete. 2024-03-06T07:31:14.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.657+0000 7f6e86a42640 1 Processor -- start 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e86a42640 1 -- start start 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e86a42640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e68003b40 0x7f6e801e7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e86a42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e800afd30 0x7f6e80157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e86a42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8014ac60 0x7f6e8014b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e86a42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800c7c30 con 0x7f6e800afd30 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e86a42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e80155c10 con 0x7f6e8014ac60 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e86a42640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e80155d90 con 0x7f6e68003b40 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e800afd30 0x7f6e80157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e800afd30 0x7f6e80157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56138/0 (socket says 172.21.15.26:56138) 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.658+0000 7f6e7f7fe640 1 -- 172.21.15.26:0/2934916966 learned_addr learned my addr 172.21.15.26:0/2934916966 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e84fdf640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8014ac60 0x7f6e8014b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e68003b40 0x7f6e801e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7f7fe640 1 -- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e68003b40 msgr2=0x7f6e801e7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7f7fe640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e68003b40 0x7f6e801e7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7f7fe640 1 -- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8014ac60 msgr2=0x7f6e8014b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7f7fe640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8014ac60 0x7f6e8014b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7f7fe640 1 -- 172.21.15.26:0/2934916966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e80152a20 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7f7fe640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e800afd30 0x7f6e80157a70 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f6e74004f20 tx=0x7f6e740074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7cff9640 1 -- 172.21.15.26:0/2934916966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400f750 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7cff9640 1 -- 172.21.15.26:0/2934916966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74018070 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e800aed30 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e7cff9640 1 -- 172.21.15.26:0/2934916966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400fb50 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.659+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e80152740 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.660+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e800c6cb0 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.661+0000 7f6e7cff9640 1 -- 172.21.15.26:0/2934916966 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e7400fcf0 con 0x7f6e800afd30 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.661+0000 7f6e7cff9640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e6c06b500 0x7f6e6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.661+0000 7f6e7cff9640 1 -- 172.21.15.26:0/2934916966 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f6e7408a370 con 0x7f6e800afd30 2024-03-06T07:31:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.662+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e6c06b500 0x7f6e6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.662+0000 7f6e7ffff640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e6c06b500 0x7f6e6c06d9c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6e70036000 tx=0x7f6e70030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.670+0000 7f6e7cff9640 1 -- 172.21.15.26:0/2934916966 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6e800c6cb0 con 0x7f6e800afd30 2024-03-06T07:31:14.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.890+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e8004f330 con 0x7f6e800afd30 2024-03-06T07:31:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.890+0000 7f6e7cff9640 1 -- 172.21.15.26:0/2934916966 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f6e74058f60 con 0x7f6e800afd30 2024-03-06T07:31:14.893 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:14.893 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e6c06b500 msgr2=0x7f6e6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e6c06b500 0x7f6e6c06d9c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6e70036000 tx=0x7f6e70030000 comp rx=0 tx=0).stop 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e800afd30 msgr2=0x7f6e80157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e800afd30 0x7f6e80157a70 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f6e74004f20 tx=0x7f6e740074f0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 shutdown_connections 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e6c06b500 0x7f6e6c06d9c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e8014ac60 0x7f6e8014b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e800afd30 0x7f6e80157a70 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 --2- 172.21.15.26:0/2934916966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e68003b40 0x7f6e801e7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 >> 172.21.15.26:0/2934916966 conn(0x7f6e800ac570 msgr2=0x7f6e8014cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 shutdown_connections 2024-03-06T07:31:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:14.892+0000 7f6e86a42640 1 -- 172.21.15.26:0/2934916966 wait complete. 2024-03-06T07:31:15.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:15.309 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:16.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:16.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.409+0000 7f393439a640 1 Processor -- start 2024-03-06T07:31:16.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.409+0000 7f393439a640 1 -- start start 2024-03-06T07:31:16.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f393439a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c0c5860 0x7f392c1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:16.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f393439a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c1591c0 con 0x7f392c0c5480 2024-03-06T07:31:16.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f393439a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c1594a0 con 0x7f392c155ba0 2024-03-06T07:31:16.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f393439a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c06df40 con 0x7f392c0c5860 2024-03-06T07:31:16.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f3932136640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f392c0c5480 0x7f392c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40482/0 (socket says 172.21.15.26:40482) 2024-03-06T07:31:16.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f3932136640 1 -- 172.21.15.26:0/1585726527 learned_addr learned my addr 172.21.15.26:0/1585726527 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f3932937640 1 --2- 172.21.15.26:0/1585726527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c0c5860 0x7f392c1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.410+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992229808 0 0) 0x7f392c1591c0 con 0x7f392c0c5480 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.411+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3918003520 con 0x7f392c0c5480 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.411+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642806109 0 0) 0x7f392c1594a0 con 0x7f392c155ba0 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.411+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f392c1591c0 con 0x7f392c155ba0 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.411+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2125751600 0 0) 0x7f3918003520 con 0x7f392c0c5480 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.411+0000 7f3932937640 1 -- 172.21.15.26:0/1585726527 >> v1:172.21.15.117:6789/0 conn(0x7f392c155ba0 legacy=0x7f392c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.411+0000 7f3932937640 1 -- 172.21.15.26:0/1585726527 >> v1:172.21.15.26:6789/0 conn(0x7f392c0c5480 legacy=0x7f392c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:16.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.411+0000 7f3932937640 1 -- 172.21.15.26:0/1585726527 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c1553a0 con 0x7f392c0c5860 2024-03-06T07:31:16.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f3932937640 1 --2- 172.21.15.26:0/1585726527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c0c5860 0x7f392c1609e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3928009dc0 tx=0x7f392802bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863473aca25d0a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:16.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928039070 con 0x7f392c0c5860 2024-03-06T07:31:16.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392802c880 con 0x7f392c0c5860 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f3931134640 1 -- 172.21.15.26:0/1585726527 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924001670 con 0x7f392c0c5860 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f393439a640 1 -- 172.21.15.26:0/1585726527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c0c5860 msgr2=0x7f392c1609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f393439a640 1 --2- 172.21.15.26:0/1585726527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c0c5860 0x7f392c1609e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3928009dc0 tx=0x7f392802bd50 comp rx=0 tx=0).stop 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f393439a640 1 -- 172.21.15.26:0/1585726527 shutdown_connections 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f393439a640 1 --2- 172.21.15.26:0/1585726527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c0c5860 0x7f392c1609e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f393439a640 1 -- 172.21.15.26:0/1585726527 >> 172.21.15.26:0/1585726527 conn(0x7f392c06b1a0 msgr2=0x7f392c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f393439a640 1 -- 172.21.15.26:0/1585726527 shutdown_connections 2024-03-06T07:31:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.412+0000 7f393439a640 1 -- 172.21.15.26:0/1585726527 wait complete. 2024-03-06T07:31:16.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.413+0000 7f393439a640 1 Processor -- start 2024-03-06T07:31:16.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.413+0000 7f393439a640 1 -- start start 2024-03-06T07:31:16.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f393439a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c5480 0x7f392c1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:16.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f393439a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c0c5860 0x7f392c1e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:16.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f393439a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c155ba0 0x7f392c150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:16.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f393439a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0c5cf0 con 0x7f392c0c5860 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f393439a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c06fdb0 con 0x7f392c0c5480 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f393439a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c156000 con 0x7f392c155ba0 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f3931935640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c0c5860 0x7f392c1e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f3931935640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c0c5860 0x7f392c1e1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56144/0 (socket says 172.21.15.26:56144) 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f3931935640 1 -- 172.21.15.26:0/33176683 learned_addr learned my addr 172.21.15.26:0/33176683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f3932937640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c155ba0 0x7f392c150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.414+0000 7f3932136640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c5480 0x7f392c1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3931935640 1 -- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c155ba0 msgr2=0x7f392c150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3931935640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c155ba0 0x7f392c150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3931935640 1 -- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c5480 msgr2=0x7f392c1518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3931935640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c5480 0x7f392c1518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3931935640 1 -- 172.21.15.26:0/33176683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c1582b0 con 0x7f392c0c5860 2024-03-06T07:31:16.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3931935640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c0c5860 0x7f392c1e1b90 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c430 tx=0x7f391c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:16.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3922ffd640 1 -- 172.21.15.26:0/33176683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c017070 con 0x7f392c0c5860 2024-03-06T07:31:16.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3922ffd640 1 -- 172.21.15.26:0/33176683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c0052f0 con 0x7f392c0c5860 2024-03-06T07:31:16.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c1515f0 con 0x7f392c0c5860 2024-03-06T07:31:16.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f3922ffd640 1 -- 172.21.15.26:0/33176683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c003830 con 0x7f392c0c5860 2024-03-06T07:31:16.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.415+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c0ab1b0 con 0x7f392c0c5860 2024-03-06T07:31:16.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.416+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c0aacc0 con 0x7f392c0c5860 2024-03-06T07:31:16.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.416+0000 7f3922ffd640 1 -- 172.21.15.26:0/33176683 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f391c01a070 con 0x7f392c0c5860 2024-03-06T07:31:16.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.417+0000 7f3922ffd640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f391006b4d0 0x7f391006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:16.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.417+0000 7f3922ffd640 1 -- 172.21.15.26:0/33176683 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f391c089820 con 0x7f392c0c5860 2024-03-06T07:31:16.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.417+0000 7f3932136640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f391006b4d0 0x7f391006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:16.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.418+0000 7f3932136640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f391006b4d0 0x7f391006d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f39240011e0 tx=0x7f3924001690 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:16.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.421+0000 7f3922ffd640 1 -- 172.21.15.26:0/33176683 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f391c057dd0 con 0x7f392c0c5860 2024-03-06T07:31:16.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.638+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f392c04f330 con 0x7f392c0c5860 2024-03-06T07:31:16.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.639+0000 7f3922ffd640 1 -- 172.21.15.26:0/33176683 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f391c05be70 con 0x7f392c0c5860 2024-03-06T07:31:16.642 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:16.642 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:16.642 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f391006b4d0 msgr2=0x7f391006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f391006b4d0 0x7f391006d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f39240011e0 tx=0x7f3924001690 comp rx=0 tx=0).stop 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c0c5860 msgr2=0x7f392c1e1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c0c5860 0x7f392c1e1b90 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c430 tx=0x7f391c00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 shutdown_connections 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f391006b4d0 0x7f391006d990 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c155ba0 0x7f392c150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c0c5860 0x7f392c1e1b90 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 --2- 172.21.15.26:0/33176683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c5480 0x7f392c1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 >> 172.21.15.26:0/33176683 conn(0x7f392c06b1a0 msgr2=0x7f392c154170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 shutdown_connections 2024-03-06T07:31:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:16.641+0000 7f393439a640 1 -- 172.21.15.26:0/33176683 wait complete. 2024-03-06T07:31:17.057 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:17.057 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:18.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:18.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.157+0000 7fa513e81640 1 Processor -- start 2024-03-06T07:31:18.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.157+0000 7fa513e81640 1 -- start start 2024-03-06T07:31:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa513e81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1550c0 0x7fa50c1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa513e81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0aa660 con 0x7fa50c1559e0 2024-03-06T07:31:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa513e81640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0aa940 con 0x7fa50c0aa180 2024-03-06T07:31:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa513e81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0c4610 con 0x7fa50c1550c0 2024-03-06T07:31:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa51141c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa50c1559e0 0x7fa50c14ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42784/0 (socket says 172.21.15.26:42784) 2024-03-06T07:31:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa51141c640 1 -- 172.21.15.26:0/433287963 learned_addr learned my addr 172.21.15.26:0/433287963 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa511c1d640 1 --2- 172.21.15.26:0/433287963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1550c0 0x7fa50c1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332780308 0 0) 0x7fa50c0aa660 con 0x7fa50c1559e0 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.158+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f8003520 con 0x7fa50c1559e0 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.159+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113611574 0 0) 0x7fa50c0aa940 con 0x7fa50c0aa180 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.159+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa50c0aa660 con 0x7fa50c0aa180 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.159+0000 7fa511c1d640 1 -- 172.21.15.26:0/433287963 >> v1:172.21.15.117:6789/0 conn(0x7fa50c0aa180 legacy=0x7fa50c0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.159+0000 7fa511c1d640 1 -- 172.21.15.26:0/433287963 >> v1:172.21.15.26:6789/0 conn(0x7fa50c1559e0 legacy=0x7fa50c14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.159+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1915147857 0 0) 0x7fa4f8003520 con 0x7fa50c1559e0 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.159+0000 7fa511c1d640 1 -- 172.21.15.26:0/433287963 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0c4860 con 0x7fa50c1550c0 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa511c1d640 1 --2- 172.21.15.26:0/433287963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1550c0 0x7fa50c1554a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa50800aed0 tx=0x7fa50802bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef93a187cc7c331 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc001670 con 0x7fa50c1550c0 2024-03-06T07:31:18.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50802c5e0 con 0x7fa50c1550c0 2024-03-06T07:31:18.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa510c1b640 1 -- 172.21.15.26:0/433287963 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50802c990 con 0x7fa50c1550c0 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa513e81640 1 -- 172.21.15.26:0/433287963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1550c0 msgr2=0x7fa50c1554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa513e81640 1 --2- 172.21.15.26:0/433287963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1550c0 0x7fa50c1554a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa50800aed0 tx=0x7fa50802bcc0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa513e81640 1 -- 172.21.15.26:0/433287963 shutdown_connections 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa513e81640 1 --2- 172.21.15.26:0/433287963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1550c0 0x7fa50c1554a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa513e81640 1 -- 172.21.15.26:0/433287963 >> 172.21.15.26:0/433287963 conn(0x7fa50c0c6710 msgr2=0x7fa50c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa513e81640 1 -- 172.21.15.26:0/433287963 shutdown_connections 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.160+0000 7fa513e81640 1 -- 172.21.15.26:0/433287963 wait complete. 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.161+0000 7fa513e81640 1 Processor -- start 2024-03-06T07:31:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.161+0000 7fa513e81640 1 -- start start 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa513e81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50c0aa180 0x7fa50c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa513e81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa50c1550c0 0x7fa50c0c3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa513e81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1559e0 0x7fa50c1ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa513e81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c14f310 con 0x7fa50c1550c0 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa513e81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0c1520 con 0x7fa50c0aa180 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa513e81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c14f490 con 0x7fa50c1559e0 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa51141c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa50c1550c0 0x7fa50c0c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa51141c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa50c1550c0 0x7fa50c0c3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59588/0 (socket says 172.21.15.26:59588) 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa51141c640 1 -- 172.21.15.26:0/183388108 learned_addr learned my addr 172.21.15.26:0/183388108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa511c1d640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50c0aa180 0x7fa50c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.162+0000 7fa51241e640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1559e0 0x7fa50c1ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa51141c640 1 -- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1559e0 msgr2=0x7fa50c1ebdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa51141c640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1559e0 0x7fa50c1ebdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa51141c640 1 -- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50c0aa180 msgr2=0x7fa50c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa51141c640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50c0aa180 0x7fa50c06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa51141c640 1 -- 172.21.15.26:0/183388108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0a6b70 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa51141c640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa50c1550c0 0x7fa50c0c3480 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c520 tx=0x7fa4fc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa5027fc640 1 -- 172.21.15.26:0/183388108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc017070 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa5027fc640 1 -- 172.21.15.26:0/183388108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc005d00 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c1eaaf0 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa5027fc640 1 -- 172.21.15.26:0/183388108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0072b0 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.163+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c06c690 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.164+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c06c480 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.164+0000 7fa5027fc640 1 -- 172.21.15.26:0/183388108 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4fc01a070 con 0x7fa50c1550c0 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.165+0000 7fa5027fc640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4f006b4d0 0x7fa4f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.165+0000 7fa5027fc640 1 -- 172.21.15.26:0/183388108 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fa4fc089860 con 0x7fa50c1550c0 2024-03-06T07:31:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.165+0000 7fa511c1d640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4f006b4d0 0x7fa4f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.166+0000 7fa511c1d640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4f006b4d0 0x7fa4f006d990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa508035000 tx=0x7fa508030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.169+0000 7fa5027fc640 1 -- 172.21.15.26:0/183388108 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa4fc057e90 con 0x7fa50c1550c0 2024-03-06T07:31:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.468+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa50c04f330 con 0x7fa50c1550c0 2024-03-06T07:31:18.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.469+0000 7fa5027fc640 1 -- 172.21.15.26:0/183388108 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fa4fc05bf30 con 0x7fa50c1550c0 2024-03-06T07:31:18.472 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:18.472 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4f006b4d0 msgr2=0x7fa4f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4f006b4d0 0x7fa4f006d990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa508035000 tx=0x7fa508030000 comp rx=0 tx=0).stop 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa50c1550c0 msgr2=0x7fa50c0c3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa50c1550c0 0x7fa50c0c3480 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c520 tx=0x7fa4fc00c9e0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 shutdown_connections 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa4f006b4d0 0x7fa4f006d990 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa50c1559e0 0x7fa50c1ebdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa50c1550c0 0x7fa50c0c3480 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 --2- 172.21.15.26:0/183388108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50c0aa180 0x7fa50c06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 >> 172.21.15.26:0/183388108 conn(0x7fa50c0c6710 msgr2=0x7fa50c14d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:18.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 shutdown_connections 2024-03-06T07:31:18.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:18.471+0000 7fa513e81640 1 -- 172.21.15.26:0/183388108 wait complete. 2024-03-06T07:31:18.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:18.892 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:19.893 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:19.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.992+0000 7fccc760f640 1 Processor -- start 2024-03-06T07:31:19.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.992+0000 7fccc760f640 1 -- start start 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc760f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fccc00c5480 0x7fccc0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc760f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fccc00c5860 0x7fccc006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc760f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc760f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc006df40 con 0x7fccc00c5860 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc760f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc00c5cc0 con 0x7fccc00c5480 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc760f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc01591c0 con 0x7fccc01586f0 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fccc00c5860 0x7fccc006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fccc00c5860 0x7fccc006f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59604/0 (socket says 172.21.15.26:59604) 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 -- 172.21.15.26:0/2151833684 learned_addr learned my addr 172.21.15.26:0/2151833684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc5bac640 1 --2- 172.21.15.26:0/2151833684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:19.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc53ab640 1 --2- 172.21.15.26:0/2151833684 >> v2:172.21.15.117:3300/0 conn(0x7fccc00c5480 0x7fccc0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 -- 172.21.15.26:0/2151833684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 msgr2=0x7fccc0158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 --2- 172.21.15.26:0/2151833684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc0158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 -- 172.21.15.26:0/2151833684 >> v2:172.21.15.117:3300/0 conn(0x7fccc00c5480 msgr2=0x7fccc0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 --2- 172.21.15.26:0/2151833684 >> v2:172.21.15.117:3300/0 conn(0x7fccc00c5480 0x7fccc0155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.993+0000 7fccc4baa640 1 -- 172.21.15.26:0/2151833684 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc01594a0 con 0x7fccc00c5860 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.994+0000 7fccc4baa640 1 --2- 172.21.15.26:0/2151833684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc006f6a0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fccc00c1cf0 tx=0x7fccb002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7feeb9be2805ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.994+0000 7fccb7fff640 1 -- 172.21.15.26:0/2151833684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb002e070 con 0x7fccc00c5860 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.994+0000 7fccb7fff640 1 -- 172.21.15.26:0/2151833684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb0037070 con 0x7fccc00c5860 2024-03-06T07:31:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.994+0000 7fccb7fff640 1 -- 172.21.15.26:0/2151833684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0031a00 con 0x7fccc00c5860 2024-03-06T07:31:20.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 -- 172.21.15.26:0/2151833684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 msgr2=0x7fccc006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 --2- 172.21.15.26:0/2151833684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc006f6a0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fccc00c1cf0 tx=0x7fccb002b880 comp rx=0 tx=0).stop 2024-03-06T07:31:20.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 -- 172.21.15.26:0/2151833684 shutdown_connections 2024-03-06T07:31:20.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 --2- 172.21.15.26:0/2151833684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 --2- 172.21.15.26:0/2151833684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc006f6a0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 --2- 172.21.15.26:0/2151833684 >> v2:172.21.15.117:3300/0 conn(0x7fccc00c5480 0x7fccc0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 -- 172.21.15.26:0/2151833684 >> 172.21.15.26:0/2151833684 conn(0x7fccc006b1a0 msgr2=0x7fccc006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 -- 172.21.15.26:0/2151833684 shutdown_connections 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.995+0000 7fccc760f640 1 -- 172.21.15.26:0/2151833684 wait complete. 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.996+0000 7fccc760f640 1 Processor -- start 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.996+0000 7fccc760f640 1 -- start start 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc760f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc00c5480 0x7fccc014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc760f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc760f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc760f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0152a20 con 0x7fccc00c5860 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc760f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0155680 con 0x7fccc00c5480 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc760f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0155050 con 0x7fccc01586f0 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc014cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59614/0 (socket says 172.21.15.26:59614) 2024-03-06T07:31:20.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 -- 172.21.15.26:0/437275263 learned_addr learned my addr 172.21.15.26:0/437275263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc53ab640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc00c5480 0x7fccc014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc5bac640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 -- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 msgr2=0x7fccc014d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc014d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 -- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc00c5480 msgr2=0x7fccc014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc00c5480 0x7fccc014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 -- 172.21.15.26:0/437275263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc01eb680 con 0x7fccc00c5860 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccc4baa640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc014cb30 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fccb0000c00 tx=0x7fccb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.997+0000 7fccb5ffb640 1 -- 172.21.15.26:0/437275263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0032670 con 0x7fccc00c5860 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.998+0000 7fccb5ffb640 1 -- 172.21.15.26:0/437275263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb0032850 con 0x7fccc00c5860 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.998+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc01eb960 con 0x7fccc00c5860 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.998+0000 7fccb5ffb640 1 -- 172.21.15.26:0/437275263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0032c40 con 0x7fccc00c5860 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.998+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc00ad500 con 0x7fccc00c5860 2024-03-06T07:31:20.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.999+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc004f330 con 0x7fccc00c5860 2024-03-06T07:31:20.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:19.999+0000 7fccb5ffb640 1 -- 172.21.15.26:0/437275263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccb004c060 con 0x7fccc00c5860 2024-03-06T07:31:20.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.000+0000 7fccb5ffb640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcca406b500 0x7fcca406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.000+0000 7fccb5ffb640 1 -- 172.21.15.26:0/437275263 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fccb00abe00 con 0x7fccc00c5860 2024-03-06T07:31:20.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.000+0000 7fccc53ab640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcca406b500 0x7fcca406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.000+0000 7fccc53ab640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcca406b500 0x7fcca406d9c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:20.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.004+0000 7fccb5ffb640 1 -- 172.21.15.26:0/437275263 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fccb0040070 con 0x7fccc00c5860 2024-03-06T07:31:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.223+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccc00ad740 con 0x7fccc00c5860 2024-03-06T07:31:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.224+0000 7fccb5ffb640 1 -- 172.21.15.26:0/437275263 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fccb007aa80 con 0x7fccc00c5860 2024-03-06T07:31:20.227 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:20.227 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcca406b500 msgr2=0x7fcca406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcca406b500 0x7fcca406d9c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc008040 comp rx=0 tx=0).stop 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 msgr2=0x7fccc014cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc014cb30 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fccb0000c00 tx=0x7fccb00046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 shutdown_connections 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcca406b500 0x7fcca406d9c0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc01586f0 0x7fccc014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc00c5860 0x7fccc014cb30 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 --2- 172.21.15.26:0/437275263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc00c5480 0x7fccc014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 >> 172.21.15.26:0/437275263 conn(0x7fccc006b1a0 msgr2=0x7fccc006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 shutdown_connections 2024-03-06T07:31:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:20.226+0000 7fccc760f640 1 -- 172.21.15.26:0/437275263 wait complete. 2024-03-06T07:31:20.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:20.642 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:21.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:21.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.742+0000 7fb657981640 1 Processor -- start 2024-03-06T07:31:21.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.742+0000 7fb657981640 1 -- start start 2024-03-06T07:31:21.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb657981640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb65014f650 0x7fb650071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:21.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb657981640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb650152c20 0x7fb65015c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb657981640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb650072440 con 0x7fb6500c3150 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb657981640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6500c2b10 con 0x7fb65014f650 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb657981640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb650152140 con 0x7fb650152c20 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb65571d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb65014f650 0x7fb650071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb65571d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb65014f650 0x7fb650071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59626/0 (socket says 172.21.15.26:59626) 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb65571d640 1 -- 172.21.15.26:0/4189358387 learned_addr learned my addr 172.21.15.26:0/4189358387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb65571d640 1 -- 172.21.15.26:0/4189358387 >> v2:172.21.15.117:3300/0 conn(0x7fb650152c20 msgr2=0x7fb65015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb65571d640 1 --2- 172.21.15.26:0/4189358387 >> v2:172.21.15.117:3300/0 conn(0x7fb650152c20 0x7fb65015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb65571d640 1 -- 172.21.15.26:0/4189358387 >> v1:172.21.15.117:6789/0 conn(0x7fb6500c3150 legacy=0x7fb6500c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.743+0000 7fb65571d640 1 -- 172.21.15.26:0/4189358387 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb650152420 con 0x7fb65014f650 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.744+0000 7fb65571d640 1 --2- 172.21.15.26:0/4189358387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb65014f650 0x7fb650071880 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fb650073690 tx=0x7fb64c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd68031499a7c49c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.744+0000 7fb647fff640 1 -- 172.21.15.26:0/4189358387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c02e070 con 0x7fb65014f650 2024-03-06T07:31:21.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.744+0000 7fb647fff640 1 -- 172.21.15.26:0/4189358387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c037070 con 0x7fb65014f650 2024-03-06T07:31:21.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.744+0000 7fb647fff640 1 -- 172.21.15.26:0/4189358387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c031a00 con 0x7fb65014f650 2024-03-06T07:31:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 -- 172.21.15.26:0/4189358387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb65014f650 msgr2=0x7fb650071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 --2- 172.21.15.26:0/4189358387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb65014f650 0x7fb650071880 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fb650073690 tx=0x7fb64c02b880 comp rx=0 tx=0).stop 2024-03-06T07:31:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 -- 172.21.15.26:0/4189358387 shutdown_connections 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 --2- 172.21.15.26:0/4189358387 >> v2:172.21.15.117:3300/0 conn(0x7fb650152c20 0x7fb65015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 --2- 172.21.15.26:0/4189358387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb65014f650 0x7fb650071880 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 -- 172.21.15.26:0/4189358387 >> 172.21.15.26:0/4189358387 conn(0x7fb6500c4290 msgr2=0x7fb6500c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 -- 172.21.15.26:0/4189358387 shutdown_connections 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.745+0000 7fb657981640 1 -- 172.21.15.26:0/4189358387 wait complete. 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.746+0000 7fb657981640 1 Processor -- start 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.746+0000 7fb657981640 1 -- start start 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb657981640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6500c3150 0x7fb6500c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb657981640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65014f650 0x7fb650143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb657981640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb650152c20 0x7fb650144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb657981640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500c40c0 con 0x7fb650152c20 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb657981640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500c3f40 con 0x7fb65014f650 2024-03-06T07:31:21.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb657981640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500c5da0 con 0x7fb6500c3150 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb650152c20 0x7fb650144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb650152c20 0x7fb650144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59630/0 (socket says 172.21.15.26:59630) 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 -- 172.21.15.26:0/3816461584 learned_addr learned my addr 172.21.15.26:0/3816461584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb65571d640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6500c3150 0x7fb6500c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb654f1c640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65014f650 0x7fb650143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 -- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6500c3150 msgr2=0x7fb6500c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6500c3150 0x7fb6500c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 -- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65014f650 msgr2=0x7fb650143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65014f650 0x7fb650143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 -- 172.21.15.26:0/3816461584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65006e6c0 con 0x7fb650152c20 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb654f1c640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65014f650 0x7fb650143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb655f1e640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb650152c20 0x7fb650144000 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b9e0 tx=0x7fb64800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb645ffb640 1 -- 172.21.15.26:0/3816461584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648019070 con 0x7fb650152c20 2024-03-06T07:31:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb645ffb640 1 -- 172.21.15.26:0/3816461584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64800cb20 con 0x7fb650152c20 2024-03-06T07:31:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb650144a30 con 0x7fb650152c20 2024-03-06T07:31:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.747+0000 7fb645ffb640 1 -- 172.21.15.26:0/3816461584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648014440 con 0x7fb650152c20 2024-03-06T07:31:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.748+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6501eeca0 con 0x7fb650152c20 2024-03-06T07:31:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.749+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65004f330 con 0x7fb650152c20 2024-03-06T07:31:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.749+0000 7fb645ffb640 1 -- 172.21.15.26:0/3816461584 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb64801c070 con 0x7fb650152c20 2024-03-06T07:31:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.749+0000 7fb645ffb640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb63406b500 0x7fb63406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.750+0000 7fb645ffb640 1 -- 172.21.15.26:0/3816461584 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fb64808a070 con 0x7fb650152c20 2024-03-06T07:31:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.750+0000 7fb65571d640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb63406b500 0x7fb63406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.750+0000 7fb65571d640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb63406b500 0x7fb63406d9c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb64c036000 tx=0x7fb64c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:21.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.754+0000 7fb645ffb640 1 -- 172.21.15.26:0/3816461584 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb648058cf0 con 0x7fb650152c20 2024-03-06T07:31:21.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.972+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb650068460 con 0x7fb650152c20 2024-03-06T07:31:21.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.972+0000 7fb645ffb640 1 -- 172.21.15.26:0/3816461584 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fb6480585f0 con 0x7fb650152c20 2024-03-06T07:31:21.975 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:21.975 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:21.975 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.974+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb63406b500 msgr2=0x7fb63406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.974+0000 7fb657981640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb63406b500 0x7fb63406d9c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb64c036000 tx=0x7fb64c03d040 comp rx=0 tx=0).stop 2024-03-06T07:31:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.974+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb650152c20 msgr2=0x7fb650144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.974+0000 7fb657981640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb650152c20 0x7fb650144000 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b9e0 tx=0x7fb64800bea0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 shutdown_connections 2024-03-06T07:31:21.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb63406b500 0x7fb63406d9c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb650152c20 0x7fb650144000 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65014f650 0x7fb650143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 --2- 172.21.15.26:0/3816461584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6500c3150 0x7fb6500c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:21.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 >> 172.21.15.26:0/3816461584 conn(0x7fb6500c4290 msgr2=0x7fb6500c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:21.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 shutdown_connections 2024-03-06T07:31:21.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:21.975+0000 7fb657981640 1 -- 172.21.15.26:0/3816461584 wait complete. 2024-03-06T07:31:22.391 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:22.391 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:23.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.492+0000 7fb0eb379640 1 Processor -- start 2024-03-06T07:31:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.492+0000 7fb0eb379640 1 -- start start 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0eb379640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0e414f360 0x7fb0e4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0eb379640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0e40c88c0 0x7fb0e40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0eb379640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e4158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0eb379640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e40c8dd0 con 0x7fb0e40c88c0 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0eb379640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e40c73d0 con 0x7fb0e414f360 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0eb379640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e40be2e0 con 0x7fb0e40c5880 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0e8914640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0e40c88c0 0x7fb0e40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0e8914640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0e40c88c0 0x7fb0e40c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59636/0 (socket says 172.21.15.26:59636) 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0e8914640 1 -- 172.21.15.26:0/2756984561 learned_addr learned my addr 172.21.15.26:0/2756984561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0e9115640 1 --2- 172.21.15.26:0/2756984561 >> v2:172.21.15.117:3300/0 conn(0x7fb0e414f360 0x7fb0e4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.493+0000 7fb0e9916640 1 --2- 172.21.15.26:0/2756984561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e4158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0e8914640 1 -- 172.21.15.26:0/2756984561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 msgr2=0x7fb0e4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0e8914640 1 --2- 172.21.15.26:0/2756984561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e4158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0e8914640 1 -- 172.21.15.26:0/2756984561 >> v2:172.21.15.117:3300/0 conn(0x7fb0e414f360 msgr2=0x7fb0e4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0e8914640 1 --2- 172.21.15.26:0/2756984561 >> v2:172.21.15.117:3300/0 conn(0x7fb0e414f360 0x7fb0e4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0e8914640 1 -- 172.21.15.26:0/2756984561 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40c7670 con 0x7fb0e40c88c0 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0e9916640 1 --2- 172.21.15.26:0/2756984561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0e8914640 1 --2- 172.21.15.26:0/2756984561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e40c88c0 0x7fb0e40c5340 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fb0e408c840 tx=0x7fb0d402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b072ea07cf9b7f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.494+0000 7fb0dbfff640 1 -- 172.21.15.26:0/2756984561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40317d0 con 0x7fb0e40c88c0 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0dbfff640 1 -- 172.21.15.26:0/2756984561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d4037070 con 0x7fb0e40c88c0 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0dbfff640 1 -- 172.21.15.26:0/2756984561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4031b90 con 0x7fb0e40c88c0 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 -- 172.21.15.26:0/2756984561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e40c88c0 msgr2=0x7fb0e40c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2756984561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e40c88c0 0x7fb0e40c5340 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fb0e408c840 tx=0x7fb0d402b870 comp rx=0 tx=0).stop 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 -- 172.21.15.26:0/2756984561 shutdown_connections 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2756984561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2756984561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e40c88c0 0x7fb0e40c5340 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2756984561 >> v2:172.21.15.117:3300/0 conn(0x7fb0e414f360 0x7fb0e4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 -- 172.21.15.26:0/2756984561 >> 172.21.15.26:0/2756984561 conn(0x7fb0e40ac570 msgr2=0x7fb0e40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:23.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 -- 172.21.15.26:0/2756984561 shutdown_connections 2024-03-06T07:31:23.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.495+0000 7fb0eb379640 1 -- 172.21.15.26:0/2756984561 wait complete. 2024-03-06T07:31:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.496+0000 7fb0eb379640 1 Processor -- start 2024-03-06T07:31:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.496+0000 7fb0eb379640 1 -- start start 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0eb379640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e40ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0eb379640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0e40c88c0 0x7fb0e40ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0eb379640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e414f360 0x7fb0e41ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0eb379640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e41590a0 con 0x7fb0e414f360 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0eb379640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e40c7c30 con 0x7fb0e40c88c0 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0eb379640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4159220 con 0x7fb0e40c5880 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e414f360 0x7fb0e41ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e414f360 0x7fb0e41ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59638/0 (socket says 172.21.15.26:59638) 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 -- 172.21.15.26:0/2554386047 learned_addr learned my addr 172.21.15.26:0/2554386047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e8914640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0e40c88c0 0x7fb0e40ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9115640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e40ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 -- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 msgr2=0x7fb0e40ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e40ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 -- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0e40c88c0 msgr2=0x7fb0e40ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0e40c88c0 0x7fb0e40ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.497+0000 7fb0e9916640 1 -- 172.21.15.26:0/2554386047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40ac260 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.498+0000 7fb0e9916640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e414f360 0x7fb0e41ee0a0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000cef0 tx=0x7fb0e000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.498+0000 7fb0d9ffb640 1 -- 172.21.15.26:0/2554386047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e001a070 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.498+0000 7fb0d9ffb640 1 -- 172.21.15.26:0/2554386047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e000d840 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.498+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e40b0430 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.498+0000 7fb0d9ffb640 1 -- 172.21.15.26:0/2554386047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000dc30 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.498+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e4157290 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.499+0000 7fb0d9ffb640 1 -- 172.21.15.26:0/2554386047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0e001d070 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.499+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e404f330 con 0x7fb0e414f360 2024-03-06T07:31:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.500+0000 7fb0d9ffb640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:23.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.500+0000 7fb0d9ffb640 1 -- 172.21.15.26:0/2554386047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fb0e008ac90 con 0x7fb0e414f360 2024-03-06T07:31:23.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.500+0000 7fb0e9115640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:23.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.500+0000 7fb0e9115640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4152a90 tx=0x7fb0dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:23.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.504+0000 7fb0d9ffb640 1 -- 172.21.15.26:0/2554386047 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb0e40ac260 con 0x7fb0e414f360 2024-03-06T07:31:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.720+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0e41562f0 con 0x7fb0e414f360 2024-03-06T07:31:23.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.721+0000 7fb0d9ffb640 1 -- 172.21.15.26:0/2554386047 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fb0e0059210 con 0x7fb0e414f360 2024-03-06T07:31:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:23.724 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:23.724 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 msgr2=0x7fb0c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4152a90 tx=0x7fb0dc008040 comp rx=0 tx=0).stop 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e414f360 msgr2=0x7fb0e41ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e414f360 0x7fb0e41ee0a0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000cef0 tx=0x7fb0e000a710 comp rx=0 tx=0).stop 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 shutdown_connections 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fb0c806b4d0 0x7fb0c806d990 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e414f360 0x7fb0e41ee0a0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0e40c88c0 0x7fb0e40ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 --2- 172.21.15.26:0/2554386047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0e40c5880 0x7fb0e40ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:23.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 >> 172.21.15.26:0/2554386047 conn(0x7fb0e40ac570 msgr2=0x7fb0e40c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:23.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 shutdown_connections 2024-03-06T07:31:23.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:23.723+0000 7fb0eb379640 1 -- 172.21.15.26:0/2554386047 wait complete. 2024-03-06T07:31:24.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:24.139 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:25.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.242+0000 7f2571577640 1 Processor -- start 2024-03-06T07:31:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.242+0000 7f2571577640 1 -- start start 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f2571577640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f256c0c5330 0x7f256c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f2571577640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f256c0c88b0 0x7f256c1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f2571577640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c7dd0 0x7f256c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f2571577640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c1559e0 con 0x7f256c0c88b0 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f2571577640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c155110 con 0x7f256c0c5330 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f2571577640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0c8f40 con 0x7f256c0c7dd0 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f256c0c88b0 0x7f256c1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f256c0c88b0 0x7f256c1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59646/0 (socket says 172.21.15.26:59646) 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 -- 172.21.15.26:0/2144036327 learned_addr learned my addr 172.21.15.26:0/2144036327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f256b7fe640 1 --2- 172.21.15.26:0/2144036327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c7dd0 0x7f256c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f256affd640 1 --2- 172.21.15.26:0/2144036327 >> v2:172.21.15.117:3300/0 conn(0x7f256c0c5330 0x7f256c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 -- 172.21.15.26:0/2144036327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c7dd0 msgr2=0x7f256c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 --2- 172.21.15.26:0/2144036327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c7dd0 0x7f256c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 -- 172.21.15.26:0/2144036327 >> v2:172.21.15.117:3300/0 conn(0x7f256c0c5330 msgr2=0x7f256c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 --2- 172.21.15.26:0/2144036327 >> v2:172.21.15.117:3300/0 conn(0x7f256c0c5330 0x7f256c0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f25627fc640 1 -- 172.21.15.26:0/2144036327 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c155390 con 0x7f256c0c88b0 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f256b7fe640 1 --2- 172.21.15.26:0/2144036327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c7dd0 0x7f256c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.243+0000 7f256affd640 1 --2- 172.21.15.26:0/2144036327 >> v2:172.21.15.117:3300/0 conn(0x7f256c0c5330 0x7f256c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.244+0000 7f25627fc640 1 --2- 172.21.15.26:0/2144036327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c88b0 0x7f256c1586a0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f256c08c6e0 tx=0x7f255402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dd6f7c0a107317 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.244+0000 7f256a7fc640 1 -- 172.21.15.26:0/2144036327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255402e070 con 0x7f256c0c88b0 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.244+0000 7f256a7fc640 1 -- 172.21.15.26:0/2144036327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2554037070 con 0x7f256c0c88b0 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.244+0000 7f256a7fc640 1 -- 172.21.15.26:0/2144036327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554031a00 con 0x7f256c0c88b0 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 -- 172.21.15.26:0/2144036327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c88b0 msgr2=0x7f256c1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 --2- 172.21.15.26:0/2144036327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c88b0 0x7f256c1586a0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f256c08c6e0 tx=0x7f255402b880 comp rx=0 tx=0).stop 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 -- 172.21.15.26:0/2144036327 shutdown_connections 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 --2- 172.21.15.26:0/2144036327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c7dd0 0x7f256c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 --2- 172.21.15.26:0/2144036327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c88b0 0x7f256c1586a0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 --2- 172.21.15.26:0/2144036327 >> v2:172.21.15.117:3300/0 conn(0x7f256c0c5330 0x7f256c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 -- 172.21.15.26:0/2144036327 >> 172.21.15.26:0/2144036327 conn(0x7f256c0ac560 msgr2=0x7f256c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 -- 172.21.15.26:0/2144036327 shutdown_connections 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.245+0000 7f2571577640 1 -- 172.21.15.26:0/2144036327 wait complete. 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.246+0000 7f2571577640 1 Processor -- start 2024-03-06T07:31:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.246+0000 7f2571577640 1 -- start start 2024-03-06T07:31:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f2571577640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c5330 0x7f256c150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f2571577640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c7dd0 0x7f256c157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f2571577640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256c0c88b0 0x7f256c0c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f2571577640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1529d0 con 0x7f256c0c7dd0 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f2571577640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c06d090 con 0x7f256c0c88b0 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f2571577640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c152b50 con 0x7f256c0c5330 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c7dd0 0x7f256c157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c7dd0 0x7f256c157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59648/0 (socket says 172.21.15.26:59648) 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 -- 172.21.15.26:0/1396982911 learned_addr learned my addr 172.21.15.26:0/1396982911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f256affd640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c5330 0x7f256c150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f256b7fe640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256c0c88b0 0x7f256c0c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 -- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c5330 msgr2=0x7f256c150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c5330 0x7f256c150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 -- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256c0c88b0 msgr2=0x7f256c0c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256c0c88b0 0x7f256c0c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f25627fc640 1 -- 172.21.15.26:0/1396982911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c150e50 con 0x7f256c0c7dd0 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.247+0000 7f256b7fe640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256c0c88b0 0x7f256c0c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.248+0000 7f25627fc640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c7dd0 0x7f256c157740 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f2554000c00 tx=0x7f25540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.248+0000 7f2563fff640 1 -- 172.21.15.26:0/1396982911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554032670 con 0x7f256c0c7dd0 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.248+0000 7f2563fff640 1 -- 172.21.15.26:0/1396982911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2554032810 con 0x7f256c0c7dd0 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.248+0000 7f2563fff640 1 -- 172.21.15.26:0/1396982911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554032c00 con 0x7f256c0c7dd0 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.248+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c151830 con 0x7f256c0c7dd0 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.248+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c151df0 con 0x7f256c0c7dd0 2024-03-06T07:31:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.249+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c04f330 con 0x7f256c0c7dd0 2024-03-06T07:31:25.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.250+0000 7f2563fff640 1 -- 172.21.15.26:0/1396982911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f255404c060 con 0x7f256c0c7dd0 2024-03-06T07:31:25.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.250+0000 7f2563fff640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f254c06b4d0 0x7f254c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:25.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.250+0000 7f2563fff640 1 -- 172.21.15.26:0/1396982911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f25540aba60 con 0x7f256c0c7dd0 2024-03-06T07:31:25.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.250+0000 7f256affd640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f254c06b4d0 0x7f254c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:25.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.251+0000 7f256affd640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f254c06b4d0 0x7f254c06d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f255c003610 tx=0x7f255c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.254+0000 7f2563fff640 1 -- 172.21.15.26:0/1396982911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2554040070 con 0x7f256c0c7dd0 2024-03-06T07:31:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.486+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f256c06cdc0 con 0x7f256c0c7dd0 2024-03-06T07:31:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.486+0000 7f2563fff640 1 -- 172.21.15.26:0/1396982911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f255407a650 con 0x7f256c0c7dd0 2024-03-06T07:31:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:25.489 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f254c06b4d0 msgr2=0x7f254c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f254c06b4d0 0x7f254c06d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f255c003610 tx=0x7f255c008040 comp rx=0 tx=0).stop 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c7dd0 msgr2=0x7f256c157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c7dd0 0x7f256c157740 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f2554000c00 tx=0x7f25540046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 shutdown_connections 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f254c06b4d0 0x7f254c06d990 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256c0c88b0 0x7f256c0c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f256c0c7dd0 0x7f256c157740 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 --2- 172.21.15.26:0/1396982911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f256c0c5330 0x7f256c150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 >> 172.21.15.26:0/1396982911 conn(0x7f256c0ac560 msgr2=0x7f256c1544a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 shutdown_connections 2024-03-06T07:31:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:25.488+0000 7f2571577640 1 -- 172.21.15.26:0/1396982911 wait complete. 2024-03-06T07:31:25.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:25.904 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:26.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.004+0000 7fd5a3488640 1 Processor -- start 2024-03-06T07:31:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.004+0000 7fd5a3488640 1 -- start start 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a3488640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd59c14f360 0x7fd59c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a3488640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd59c0c88c0 0x7fd59c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a3488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c0c5880 0x7fd59c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a3488640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0c8dd0 con 0x7fd59c0c88c0 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a3488640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0c73d0 con 0x7fd59c14f360 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a3488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0be2e0 con 0x7fd59c0c5880 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd59c0c88c0 0x7fd59c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd59c0c88c0 0x7fd59c0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59662/0 (socket says 172.21.15.26:59662) 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 -- 172.21.15.26:0/1960866355 learned_addr learned my addr 172.21.15.26:0/1960866355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a1a25640 1 --2- 172.21.15.26:0/1960866355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c0c5880 0x7fd59c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a1224640 1 --2- 172.21.15.26:0/1960866355 >> v2:172.21.15.117:3300/0 conn(0x7fd59c14f360 0x7fd59c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 -- 172.21.15.26:0/1960866355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c0c5880 msgr2=0x7fd59c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 --2- 172.21.15.26:0/1960866355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c0c5880 0x7fd59c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 -- 172.21.15.26:0/1960866355 >> v2:172.21.15.117:3300/0 conn(0x7fd59c14f360 msgr2=0x7fd59c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 --2- 172.21.15.26:0/1960866355 >> v2:172.21.15.117:3300/0 conn(0x7fd59c14f360 0x7fd59c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a0a23640 1 -- 172.21.15.26:0/1960866355 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c0c7670 con 0x7fd59c0c88c0 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a1a25640 1 --2- 172.21.15.26:0/1960866355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c0c5880 0x7fd59c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.006+0000 7fd5a1224640 1 --2- 172.21.15.26:0/1960866355 >> v2:172.21.15.117:3300/0 conn(0x7fd59c14f360 0x7fd59c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.007+0000 7fd5a0a23640 1 --2- 172.21.15.26:0/1960866355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c88c0 0x7fd59c0c5340 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7fd59c08c840 tx=0x7fd58c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5abed2bde138ecb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.007+0000 7fd593fff640 1 -- 172.21.15.26:0/1960866355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c02e070 con 0x7fd59c0c88c0 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.007+0000 7fd593fff640 1 -- 172.21.15.26:0/1960866355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c037070 con 0x7fd59c0c88c0 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.007+0000 7fd593fff640 1 -- 172.21.15.26:0/1960866355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c031a00 con 0x7fd59c0c88c0 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 -- 172.21.15.26:0/1960866355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c88c0 msgr2=0x7fd59c0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 --2- 172.21.15.26:0/1960866355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c88c0 0x7fd59c0c5340 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7fd59c08c840 tx=0x7fd58c02b880 comp rx=0 tx=0).stop 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 -- 172.21.15.26:0/1960866355 shutdown_connections 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 --2- 172.21.15.26:0/1960866355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c0c5880 0x7fd59c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 --2- 172.21.15.26:0/1960866355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c88c0 0x7fd59c0c5340 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 --2- 172.21.15.26:0/1960866355 >> v2:172.21.15.117:3300/0 conn(0x7fd59c14f360 0x7fd59c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 -- 172.21.15.26:0/1960866355 >> 172.21.15.26:0/1960866355 conn(0x7fd59c0ac570 msgr2=0x7fd59c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 -- 172.21.15.26:0/1960866355 shutdown_connections 2024-03-06T07:31:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.008+0000 7fd5a3488640 1 -- 172.21.15.26:0/1960866355 wait complete. 2024-03-06T07:31:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.009+0000 7fd5a3488640 1 Processor -- start 2024-03-06T07:31:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.009+0000 7fd5a3488640 1 -- start start 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a3488640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c5880 0x7fd59c0ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a3488640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c0c88c0 0x7fd59c1ee750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a3488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c14f360 0x7fd59c1eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a3488640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c1590a0 con 0x7fd59c0c5880 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a3488640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0c7c30 con 0x7fd59c0c88c0 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a3488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c159220 con 0x7fd59c14f360 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c5880 0x7fd59c0ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c5880 0x7fd59c0ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59678/0 (socket says 172.21.15.26:59678) 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 -- 172.21.15.26:0/3044151427 learned_addr learned my addr 172.21.15.26:0/3044151427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1a25640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c14f360 0x7fd59c1eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 -- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c14f360 msgr2=0x7fd59c1eec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a0a23640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c0c88c0 0x7fd59c1ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c14f360 0x7fd59c1eec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 -- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c0c88c0 msgr2=0x7fd59c1ee750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c0c88c0 0x7fd59c1ee750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 -- 172.21.15.26:0/3044151427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c1eadd0 con 0x7fd59c0c5880 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.010+0000 7fd5a1224640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c5880 0x7fd59c0ab9e0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c7f0 tx=0x7fd59800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.011+0000 7fd591ffb640 1 -- 172.21.15.26:0/3044151427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598015070 con 0x7fd59c0c5880 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.011+0000 7fd591ffb640 1 -- 172.21.15.26:0/3044151427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598007820 con 0x7fd59c0c5880 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.011+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c1eb0b0 con 0x7fd59c0c5880 2024-03-06T07:31:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.011+0000 7fd591ffb640 1 -- 172.21.15.26:0/3044151427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598007bd0 con 0x7fd59c0c5880 2024-03-06T07:31:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.011+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c1eb670 con 0x7fd59c0c5880 2024-03-06T07:31:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.012+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c04f330 con 0x7fd59c0c5880 2024-03-06T07:31:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.012+0000 7fd591ffb640 1 -- 172.21.15.26:0/3044151427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd598018070 con 0x7fd59c0c5880 2024-03-06T07:31:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.012+0000 7fd591ffb640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58006b4d0 0x7fd58006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.013+0000 7fd591ffb640 1 -- 172.21.15.26:0/3044151427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd59808a520 con 0x7fd59c0c5880 2024-03-06T07:31:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.013+0000 7fd5a0a23640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58006b4d0 0x7fd58006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.013+0000 7fd5a0a23640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58006b4d0 0x7fd58006d990 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd58c036000 tx=0x7fd58c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.017+0000 7fd591ffb640 1 -- 172.21.15.26:0/3044151427 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5980591a0 con 0x7fd59c0c5880 2024-03-06T07:31:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.381+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd59c06cda0 con 0x7fd59c0c5880 2024-03-06T07:31:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.381+0000 7fd591ffb640 1 -- 172.21.15.26:0/3044151427 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fd598058aa0 con 0x7fd59c0c5880 2024-03-06T07:31:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:27.384 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58006b4d0 msgr2=0x7fd58006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58006b4d0 0x7fd58006d990 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd58c036000 tx=0x7fd58c008030 comp rx=0 tx=0).stop 2024-03-06T07:31:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c5880 msgr2=0x7fd59c0ab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c5880 0x7fd59c0ab9e0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c7f0 tx=0x7fd59800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 shutdown_connections 2024-03-06T07:31:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58006b4d0 0x7fd58006d990 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd59c14f360 0x7fd59c1eec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c0c88c0 0x7fd59c1ee750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 --2- 172.21.15.26:0/3044151427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd59c0c5880 0x7fd59c0ab9e0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 >> 172.21.15.26:0/3044151427 conn(0x7fd59c0ac570 msgr2=0x7fd59c0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 shutdown_connections 2024-03-06T07:31:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:27.383+0000 7fd5a3488640 1 -- 172.21.15.26:0/3044151427 wait complete. 2024-03-06T07:31:27.798 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:27.799 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:28.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:28.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.898+0000 7f1640caf640 1 Processor -- start 2024-03-06T07:31:28.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.898+0000 7f1640caf640 1 -- start start 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f1640caf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f1640caf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f163c158b60 0x7f163c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f1640caf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163c0c73d0 con 0x7f163c14f360 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f1640caf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c0c8dd0 con 0x7f163c158b60 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f1640caf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0be2e0 con 0x7f163c0c88c0 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f163ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f163c158b60 0x7f163c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f163a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f163c14f360 0x7f163c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59106/0 (socket says 172.21.15.26:59106) 2024-03-06T07:31:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.899+0000 7f163a59c640 1 -- 172.21.15.26:0/738112044 learned_addr learned my addr 172.21.15.26:0/738112044 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f1639d9b640 1 --2- 172.21.15.26:0/738112044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f163959a640 1 -- 172.21.15.26:0/738112044 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061454561 0 0) 0x7f163c0c73d0 con 0x7f163c14f360 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f163959a640 1 -- 172.21.15.26:0/738112044 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1624003520 con 0x7f163c14f360 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f163ad9d640 1 -- 172.21.15.26:0/738112044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 msgr2=0x7f163c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f163ad9d640 1 --2- 172.21.15.26:0/738112044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f163ad9d640 1 -- 172.21.15.26:0/738112044 >> v1:172.21.15.26:6789/0 conn(0x7f163c14f360 legacy=0x7f163c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f163ad9d640 1 -- 172.21.15.26:0/738112044 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0c7670 con 0x7f163c158b60 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f1639d9b640 1 --2- 172.21.15.26:0/738112044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.900+0000 7f163ad9d640 1 --2- 172.21.15.26:0/738112044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c158ff0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f162c009d90 tx=0x7f162c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ee6b16b9b87131 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f163959a640 1 -- 172.21.15.26:0/738112044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c039070 con 0x7f163c158b60 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f163959a640 1 -- 172.21.15.26:0/738112044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c02cbc0 con 0x7f163c158b60 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f163959a640 1 -- 172.21.15.26:0/738112044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c034440 con 0x7f163c158b60 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 -- 172.21.15.26:0/738112044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 msgr2=0x7f163c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 --2- 172.21.15.26:0/738112044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c158ff0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f162c009d90 tx=0x7f162c02be10 comp rx=0 tx=0).stop 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 -- 172.21.15.26:0/738112044 shutdown_connections 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 --2- 172.21.15.26:0/738112044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c158ff0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 --2- 172.21.15.26:0/738112044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 -- 172.21.15.26:0/738112044 >> 172.21.15.26:0/738112044 conn(0x7f163c0ac570 msgr2=0x7f163c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 -- 172.21.15.26:0/738112044 shutdown_connections 2024-03-06T07:31:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.901+0000 7f1640caf640 1 -- 172.21.15.26:0/738112044 wait complete. 2024-03-06T07:31:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 Processor -- start 2024-03-06T07:31:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 -- start start 2024-03-06T07:31:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c1ea920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f163c14f360 0x7f163c157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c1ee770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0c5a50 con 0x7f163c158b60 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0c7c30 con 0x7f163c14f360 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f1640caf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0c5bd0 con 0x7f163c0c88c0 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f163ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c1ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f163ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c1ee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43728/0 (socket says 172.21.15.26:43728) 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f163ad9d640 1 -- 172.21.15.26:0/2816304327 learned_addr learned my addr 172.21.15.26:0/2816304327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.903+0000 7f163a59c640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c1ea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f1639d9b640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f163c14f360 0x7f163c157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f163ad9d640 1 -- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 msgr2=0x7f163c1ea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f163ad9d640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c1ea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f163ad9d640 1 -- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f163c14f360 msgr2=0x7f163c157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f163ad9d640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f163c14f360 0x7f163c157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f163ad9d640 1 -- 172.21.15.26:0/2816304327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c1edfb0 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f163ad9d640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c1ee770 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f162c009d90 tx=0x7f162c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f162affd640 1 -- 172.21.15.26:0/2816304327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c039050 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f162affd640 1 -- 172.21.15.26:0/2816304327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c045070 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c0bf010 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f162affd640 1 -- 172.21.15.26:0/2816304327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c03e830 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.904+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c1ee250 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.905+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c04f330 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.905+0000 7f162affd640 1 -- 172.21.15.26:0/2816304327 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f162c03e9d0 con 0x7f163c158b60 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.906+0000 7f162affd640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161c06b4d0 0x7f161c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.906+0000 7f162affd640 1 -- 172.21.15.26:0/2816304327 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f162c0ad3b0 con 0x7f163c158b60 2024-03-06T07:31:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.906+0000 7f163a59c640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161c06b4d0 0x7f161c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.907+0000 7f163a59c640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161c06b4d0 0x7f161c06d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1634005880 tx=0x7f1634009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:28.910+0000 7f162affd640 1 -- 172.21.15.26:0/2816304327 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f162c032e20 con 0x7f163c158b60 2024-03-06T07:31:29.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.127+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f163c1ee420 con 0x7f163c158b60 2024-03-06T07:31:29.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.128+0000 7f162affd640 1 -- 172.21.15.26:0/2816304327 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f162c07c150 con 0x7f163c158b60 2024-03-06T07:31:29.131 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:29.131 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:29.131 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161c06b4d0 msgr2=0x7f161c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161c06b4d0 0x7f161c06d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1634005880 tx=0x7f1634009040 comp rx=0 tx=0).stop 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 msgr2=0x7f163c1ee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c1ee770 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f162c009d90 tx=0x7f162c004a40 comp rx=0 tx=0).stop 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 shutdown_connections 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f161c06b4d0 0x7f161c06d990 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f163c158b60 0x7f163c1ee770 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f163c14f360 0x7f163c157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 --2- 172.21.15.26:0/2816304327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f163c0c88c0 0x7f163c1ea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 >> 172.21.15.26:0/2816304327 conn(0x7f163c0ac570 msgr2=0x7f163c06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 shutdown_connections 2024-03-06T07:31:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:29.130+0000 7f1640caf640 1 -- 172.21.15.26:0/2816304327 wait complete. 2024-03-06T07:31:29.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:29.549 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:30.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:30.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.660+0000 7f5897b8c640 1 Processor -- start 2024-03-06T07:31:30.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.660+0000 7f5897b8c640 1 -- start start 2024-03-06T07:31:30.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.660+0000 7f5897b8c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.660+0000 7f5897b8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58900720a0 0x7f589006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.660+0000 7f5897b8c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58900c1ae0 con 0x7f5890152520 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.660+0000 7f5897b8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58900c1dc0 con 0x7f58900720a0 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5897b8c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589006e450 con 0x7f5890155c80 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5896129640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58900720a0 0x7f589006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5896129640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58900720a0 0x7f589006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43736/0 (socket says 172.21.15.26:43736) 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5896129640 1 -- 172.21.15.26:0/255909564 learned_addr learned my addr 172.21.15.26:0/255909564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5895127640 1 --2- 172.21.15.26:0/255909564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5896129640 1 -- 172.21.15.26:0/255909564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 msgr2=0x7f589006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5896129640 1 --2- 172.21.15.26:0/255909564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5896129640 1 -- 172.21.15.26:0/255909564 >> v1:172.21.15.117:6789/0 conn(0x7f5890152520 legacy=0x7f58900c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:30.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.661+0000 7f5896129640 1 -- 172.21.15.26:0/255909564 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589006e730 con 0x7f58900720a0 2024-03-06T07:31:30.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5896129640 1 --2- 172.21.15.26:0/255909564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58900720a0 0x7f589006afc0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f5888009e50 tx=0x7f588802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618e739398538f73 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5894926640 1 -- 172.21.15.26:0/255909564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588802cb00 con 0x7f58900720a0 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5894926640 1 -- 172.21.15.26:0/255909564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588802cca0 con 0x7f58900720a0 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5894926640 1 -- 172.21.15.26:0/255909564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888034610 con 0x7f58900720a0 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5897b8c640 1 -- 172.21.15.26:0/255909564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58900720a0 msgr2=0x7f589006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5897b8c640 1 --2- 172.21.15.26:0/255909564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58900720a0 0x7f589006afc0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f5888009e50 tx=0x7f588802bf60 comp rx=0 tx=0).stop 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5897b8c640 1 -- 172.21.15.26:0/255909564 shutdown_connections 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5897b8c640 1 --2- 172.21.15.26:0/255909564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58900720a0 0x7f589006afc0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5897b8c640 1 --2- 172.21.15.26:0/255909564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.662+0000 7f5897b8c640 1 -- 172.21.15.26:0/255909564 >> 172.21.15.26:0/255909564 conn(0x7f58900c76b0 msgr2=0x7f58900c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.663+0000 7f5897b8c640 1 -- 172.21.15.26:0/255909564 shutdown_connections 2024-03-06T07:31:30.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.663+0000 7f5897b8c640 1 -- 172.21.15.26:0/255909564 wait complete. 2024-03-06T07:31:30.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 Processor -- start 2024-03-06T07:31:30.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 -- start start 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58900720a0 0x7f58901ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5890152520 0x7f589014d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589014aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5890152a80 con 0x7f5890152520 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5890152900 con 0x7f58900720a0 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5897b8c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589006b500 con 0x7f5890155c80 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.664+0000 7f5895127640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5890152520 0x7f589014d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5890152520 0x7f589014d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43746/0 (socket says 172.21.15.26:43746) 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 -- 172.21.15.26:0/1629000734 learned_addr learned my addr 172.21.15.26:0/1629000734 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5896129640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589014aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895928640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58900720a0 0x7f58901ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 -- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 msgr2=0x7f589014aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:30.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589014aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 -- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58900720a0 msgr2=0x7f58901ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58900720a0 0x7f58901ee740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 -- 172.21.15.26:0/1629000734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900aa550 con 0x7f5890152520 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5895127640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5890152520 0x7f589014d0b0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f588000d830 tx=0x7f588000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f58867fc640 1 -- 172.21.15.26:0/1629000734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000b6c0 con 0x7f5890152520 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f58867fc640 1 -- 172.21.15.26:0/1629000734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880019070 con 0x7f5890152520 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58900aa830 con 0x7f5890152520 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f58867fc640 1 -- 172.21.15.26:0/1629000734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000ba80 con 0x7f5890152520 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.665+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58900a8390 con 0x7f5890152520 2024-03-06T07:31:30.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.666+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589004f330 con 0x7f5890152520 2024-03-06T07:31:30.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.667+0000 7f58867fc640 1 -- 172.21.15.26:0/1629000734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f588000bc20 con 0x7f5890152520 2024-03-06T07:31:30.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.667+0000 7f58867fc640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f587406b4d0 0x7f587406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:30.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.667+0000 7f58867fc640 1 -- 172.21.15.26:0/1629000734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f588008a970 con 0x7f5890152520 2024-03-06T07:31:30.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.667+0000 7f5895928640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f587406b4d0 0x7f587406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:30.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.668+0000 7f5895928640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f587406b4d0 0x7f587406d990 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f588c001640 tx=0x7f588c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:30.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.672+0000 7f58867fc640 1 -- 172.21.15.26:0/1629000734 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5880059560 con 0x7f5890152520 2024-03-06T07:31:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.891+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58900c0b70 con 0x7f5890152520 2024-03-06T07:31:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.891+0000 7f58867fc640 1 -- 172.21.15.26:0/1629000734 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f5880058e60 con 0x7f5890152520 2024-03-06T07:31:30.894 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:30.894 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f587406b4d0 msgr2=0x7f587406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f587406b4d0 0x7f587406d990 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f588c001640 tx=0x7f588c009040 comp rx=0 tx=0).stop 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5890152520 msgr2=0x7f589014d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5890152520 0x7f589014d0b0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f588000d830 tx=0x7f588000dcf0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 shutdown_connections 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f587406b4d0 0x7f587406d990 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5890155c80 0x7f589014aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5890152520 0x7f589014d0b0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 --2- 172.21.15.26:0/1629000734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58900720a0 0x7f58901ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 >> 172.21.15.26:0/1629000734 conn(0x7f58900c76b0 msgr2=0x7f58900c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 shutdown_connections 2024-03-06T07:31:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:30.893+0000 7f5897b8c640 1 -- 172.21.15.26:0/1629000734 wait complete. 2024-03-06T07:31:31.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:31.309 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:32.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:32.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.411+0000 7f461cbe9640 1 Processor -- start 2024-03-06T07:31:32.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.411+0000 7f461cbe9640 1 -- start start 2024-03-06T07:31:32.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f461cbe9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f46180c88c0 0x7f4618155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f461cbe9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180c73d0 con 0x7f461814f360 2024-03-06T07:31:32.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f461cbe9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180be2e0 con 0x7f4618158b60 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f461cbe9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4618159700 con 0x7f46180c88c0 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f461659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f461814f360 0x7f46180c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59108/0 (socket says 172.21.15.26:59108) 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f461659c640 1 -- 172.21.15.26:0/1387185319 learned_addr learned my addr 172.21.15.26:0/1387185319 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f460ffff640 1 --2- 172.21.15.26:0/1387185319 >> v2:172.21.15.117:3300/0 conn(0x7f46180c88c0 0x7f4618155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640490006 0 0) 0x7f46180c73d0 con 0x7f461814f360 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4604003520 con 0x7f461814f360 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1972437987 0 0) 0x7f4604003520 con 0x7f461814f360 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 >> v2:172.21.15.117:3300/0 conn(0x7f46180c88c0 msgr2=0x7f4618155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 --2- 172.21.15.26:0/1387185319 >> v2:172.21.15.117:3300/0 conn(0x7f46180c88c0 0x7f4618155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 >> v1:172.21.15.117:6789/0 conn(0x7f4618158b60 legacy=0x7f4618158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180c7670 con 0x7f461814f360 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4608001670 con 0x7f461814f360 2024-03-06T07:31:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 >> v1:172.21.15.26:6789/0 conn(0x7f461814f360 legacy=0x7f46180c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040039b0 0x7f4604003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.412+0000 7f4615d9b640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040043b0 0x7f4604004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f4615d9b640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4604005120 0x7f460400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4618159700 con 0x7f46040043b0 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460400c060 con 0x7f46040039b0 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460400c260 con 0x7f4604005120 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f460ffff640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040043b0 0x7f4604004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f461659c640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040039b0 0x7f4604003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f4616d9d640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4604005120 0x7f460400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f460ffff640 1 -- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4604005120 msgr2=0x7f460400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f460ffff640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4604005120 0x7f460400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f460ffff640 1 -- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040039b0 msgr2=0x7f4604003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f460ffff640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040039b0 0x7f4604003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f460ffff640 1 -- 172.21.15.26:0/1387185319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604003520 con 0x7f46040043b0 2024-03-06T07:31:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.413+0000 7f4616d9d640 1 -- 172.21.15.26:0/1387185319 reap_dead start 2024-03-06T07:31:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f460ffff640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040043b0 0x7f4604004880 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f4600009a90 tx=0x7f4604005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40ec41f7aa38514 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600039070 con 0x7f46040043b0 2024-03-06T07:31:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460002c680 con 0x7f46040043b0 2024-03-06T07:31:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f4615d9b640 1 -- 172.21.15.26:0/1387185319 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600009690 con 0x7f46040043b0 2024-03-06T07:31:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f461cbe9640 1 -- 172.21.15.26:0/1387185319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040043b0 msgr2=0x7f4604004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f461cbe9640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040043b0 0x7f4604004880 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f4600009a90 tx=0x7f4604005830 comp rx=0 tx=0).stop 2024-03-06T07:31:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f461cbe9640 1 -- 172.21.15.26:0/1387185319 shutdown_connections 2024-03-06T07:31:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.414+0000 7f461cbe9640 1 --2- 172.21.15.26:0/1387185319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040043b0 0x7f4604004880 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.415+0000 7f461cbe9640 1 -- 172.21.15.26:0/1387185319 >> 172.21.15.26:0/1387185319 conn(0x7f46180ac570 msgr2=0x7f46180ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.415+0000 7f461cbe9640 1 -- 172.21.15.26:0/1387185319 shutdown_connections 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.415+0000 7f461cbe9640 1 -- 172.21.15.26:0/1387185319 wait complete. 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 Processor -- start 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 -- start start 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040043b0 0x7f46181ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46180b04a0 0x7f46181543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180afb50 0x7f4618150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180c7c30 con 0x7f46180afb50 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4618155c10 con 0x7f46040043b0 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.416+0000 7f461cbe9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4618155d90 con 0x7f46180b04a0 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180afb50 0x7f4618150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180afb50 0x7f4618150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43762/0 (socket says 172.21.15.26:43762) 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 -- 172.21.15.26:0/3474580650 learned_addr learned my addr 172.21.15.26:0/3474580650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f460ffff640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46180b04a0 0x7f46181543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f461659c640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040043b0 0x7f46181ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 -- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46180b04a0 msgr2=0x7f46181543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46180b04a0 0x7f46181543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 -- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040043b0 msgr2=0x7f46181ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040043b0 0x7f46181ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 -- 172.21.15.26:0/3474580650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461814b2c0 con 0x7f46180afb50 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f4616d9d640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180afb50 0x7f4618150810 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f46180c1860 tx=0x7f461000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f460effd640 1 -- 172.21.15.26:0/3474580650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610019070 con 0x7f46180afb50 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f460effd640 1 -- 172.21.15.26:0/3474580650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610006e20 con 0x7f46180afb50 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f460effd640 1 -- 172.21.15.26:0/3474580650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610003b90 con 0x7f46180afb50 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4618152860 con 0x7f46180afb50 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.417+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46181548f0 con 0x7f46180afb50 2024-03-06T07:31:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.418+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46180c1050 con 0x7f46180afb50 2024-03-06T07:31:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.419+0000 7f460effd640 1 -- 172.21.15.26:0/3474580650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f461000d770 con 0x7f46180afb50 2024-03-06T07:31:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.419+0000 7f460effd640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f45f806b500 0x7f45f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.420+0000 7f460effd640 1 -- 172.21.15.26:0/3474580650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f4610089960 con 0x7f46180afb50 2024-03-06T07:31:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.420+0000 7f461659c640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f45f806b500 0x7f45f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.420+0000 7f461659c640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f45f806b500 0x7f45f806d9c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4618069cc0 tx=0x7f4608009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:32.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.424+0000 7f460effd640 1 -- 172.21.15.26:0/3474580650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4610058550 con 0x7f46180afb50 2024-03-06T07:31:32.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.642+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f461804f330 con 0x7f46180afb50 2024-03-06T07:31:32.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.643+0000 7f460effd640 1 -- 172.21.15.26:0/3474580650 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f4610057e50 con 0x7f46180afb50 2024-03-06T07:31:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:32.645 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:32.645 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:32.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f45f806b500 msgr2=0x7f45f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f45f806b500 0x7f45f806d9c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4618069cc0 tx=0x7f4608009040 comp rx=0 tx=0).stop 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180afb50 msgr2=0x7f4618150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180afb50 0x7f4618150810 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f46180c1860 tx=0x7f461000ff70 comp rx=0 tx=0).stop 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 shutdown_connections 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f45f806b500 0x7f45f806d9c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180afb50 0x7f4618150810 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46180b04a0 0x7f46181543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 --2- 172.21.15.26:0/3474580650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46040043b0 0x7f46181ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 >> 172.21.15.26:0/3474580650 conn(0x7f46180ac570 msgr2=0x7f46180c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 shutdown_connections 2024-03-06T07:31:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:32.645+0000 7f461cbe9640 1 -- 172.21.15.26:0/3474580650 wait complete. 2024-03-06T07:31:33.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:33.059 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:34.060 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.164+0000 7fc842855640 1 Processor -- start 2024-03-06T07:31:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.164+0000 7fc842855640 1 -- start start 2024-03-06T07:31:34.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc842855640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc83c1527b0 0x7fc83c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc842855640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c88b0 0x7fc83c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc842855640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83c0c7c60 con 0x7fc83c0c7180 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc842855640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c159640 con 0x7fc83c1527b0 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc842855640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c0c8060 con 0x7fc83c0c88b0 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc83b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc83c0c7180 0x7fc83c155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59114/0 (socket says 172.21.15.26:59114) 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc83b7fe640 1 -- 172.21.15.26:0/4114912473 learned_addr learned my addr 172.21.15.26:0/4114912473 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc840df2640 1 --2- 172.21.15.26:0/4114912473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c88b0 0x7fc83c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.165+0000 7fc83bfff640 1 --2- 172.21.15.26:0/4114912473 >> v2:172.21.15.117:3300/0 conn(0x7fc83c1527b0 0x7fc83c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.166+0000 7fc83affd640 1 -- 172.21.15.26:0/4114912473 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10294687 0 0) 0x7fc830001110 con 0x7fc83c0c7180 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.166+0000 7fc83affd640 1 -- 172.21.15.26:0/4114912473 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc824003520 con 0x7fc83c0c7180 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.166+0000 7fc840df2640 1 -- 172.21.15.26:0/4114912473 >> v2:172.21.15.117:3300/0 conn(0x7fc83c1527b0 msgr2=0x7fc83c159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.166+0000 7fc840df2640 1 --2- 172.21.15.26:0/4114912473 >> v2:172.21.15.117:3300/0 conn(0x7fc83c1527b0 0x7fc83c159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.166+0000 7fc840df2640 1 -- 172.21.15.26:0/4114912473 >> v1:172.21.15.26:6789/0 conn(0x7fc83c0c7180 legacy=0x7fc83c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.166+0000 7fc840df2640 1 -- 172.21.15.26:0/4114912473 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c0c76a0 con 0x7fc83c0c88b0 2024-03-06T07:31:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.166+0000 7fc83bfff640 1 --2- 172.21.15.26:0/4114912473 >> v2:172.21.15.117:3300/0 conn(0x7fc83c1527b0 0x7fc83c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.167+0000 7fc840df2640 1 --2- 172.21.15.26:0/4114912473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c88b0 0x7fc83c0c5330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc834009d90 tx=0x7fc83402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db971443490dd7bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.167+0000 7fc83affd640 1 -- 172.21.15.26:0/4114912473 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834039070 con 0x7fc83c0c88b0 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.167+0000 7fc83affd640 1 -- 172.21.15.26:0/4114912473 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83402cbc0 con 0x7fc83c0c88b0 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.167+0000 7fc83affd640 1 -- 172.21.15.26:0/4114912473 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834034440 con 0x7fc83c0c88b0 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.167+0000 7fc842855640 1 -- 172.21.15.26:0/4114912473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c88b0 msgr2=0x7fc83c0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.167+0000 7fc842855640 1 --2- 172.21.15.26:0/4114912473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c88b0 0x7fc83c0c5330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc834009d90 tx=0x7fc83402be10 comp rx=0 tx=0).stop 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.168+0000 7fc842855640 1 -- 172.21.15.26:0/4114912473 shutdown_connections 2024-03-06T07:31:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.168+0000 7fc842855640 1 --2- 172.21.15.26:0/4114912473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c88b0 0x7fc83c0c5330 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.168+0000 7fc842855640 1 --2- 172.21.15.26:0/4114912473 >> v2:172.21.15.117:3300/0 conn(0x7fc83c1527b0 0x7fc83c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.168+0000 7fc842855640 1 -- 172.21.15.26:0/4114912473 >> 172.21.15.26:0/4114912473 conn(0x7fc83c0ac560 msgr2=0x7fc83c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.168+0000 7fc842855640 1 -- 172.21.15.26:0/4114912473 shutdown_connections 2024-03-06T07:31:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.168+0000 7fc842855640 1 -- 172.21.15.26:0/4114912473 wait complete. 2024-03-06T07:31:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.169+0000 7fc842855640 1 Processor -- start 2024-03-06T07:31:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.169+0000 7fc842855640 1 -- start start 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc842855640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c7180 0x7fc83c0bde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc842855640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc83c0c88b0 0x7fc83c158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc842855640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc83c1527b0 0x7fc83c1ec040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc842855640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c155600 con 0x7fc83c0c88b0 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc842855640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c155320 con 0x7fc83c1527b0 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc842855640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c0c8d60 con 0x7fc83c0c7180 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc83c0c88b0 0x7fc83c158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc83c0c88b0 0x7fc83c158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43772/0 (socket says 172.21.15.26:43772) 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 -- 172.21.15.26:0/1404169604 learned_addr learned my addr 172.21.15.26:0/1404169604 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc840df2640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc83c1527b0 0x7fc83c1ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 -- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c7180 msgr2=0x7fc83c0bde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83bfff640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c7180 0x7fc83c0bde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c7180 0x7fc83c0bde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 -- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc83c1527b0 msgr2=0x7fc83c1ec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc83c1527b0 0x7fc83c1ec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 -- 172.21.15.26:0/1404169604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c14df90 con 0x7fc83c0c88b0 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc840df2640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc83c1527b0 0x7fc83c1ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83bfff640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c7180 0x7fc83c0bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc83b7fe640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc83c0c88b0 0x7fc83c158260 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fc83000c760 tx=0x7fc83000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc838ff9640 1 -- 172.21.15.26:0/1404169604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830017070 con 0x7fc83c0c88b0 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc838ff9640 1 -- 172.21.15.26:0/1404169604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830005ad0 con 0x7fc83c0c88b0 2024-03-06T07:31:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83c14d500 con 0x7fc83c0c88b0 2024-03-06T07:31:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.171+0000 7fc838ff9640 1 -- 172.21.15.26:0/1404169604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830007e20 con 0x7fc83c0c88b0 2024-03-06T07:31:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.172+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83c14ed90 con 0x7fc83c0c88b0 2024-03-06T07:31:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.173+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83c04f330 con 0x7fc83c0c88b0 2024-03-06T07:31:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.173+0000 7fc838ff9640 1 -- 172.21.15.26:0/1404169604 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc83001a070 con 0x7fc83c0c88b0 2024-03-06T07:31:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.173+0000 7fc838ff9640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82806b4d0 0x7fc82806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.173+0000 7fc838ff9640 1 -- 172.21.15.26:0/1404169604 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc830089880 con 0x7fc83c0c88b0 2024-03-06T07:31:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.174+0000 7fc83bfff640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82806b4d0 0x7fc82806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.174+0000 7fc83bfff640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82806b4d0 0x7fc82806d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009a20 tx=0x7fc82c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:34.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.182+0000 7fc838ff9640 1 -- 172.21.15.26:0/1404169604 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc83c0c7c60 con 0x7fc83c0c88b0 2024-03-06T07:31:34.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.400+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc83c06ce00 con 0x7fc83c0c88b0 2024-03-06T07:31:34.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.400+0000 7fc838ff9640 1 -- 172.21.15.26:0/1404169604 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc8300583e0 con 0x7fc83c0c88b0 2024-03-06T07:31:34.402 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:34.403 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:34.403 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82806b4d0 msgr2=0x7fc82806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82806b4d0 0x7fc82806d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009a20 tx=0x7fc82c008040 comp rx=0 tx=0).stop 2024-03-06T07:31:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc83c0c88b0 msgr2=0x7fc83c158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc83c0c88b0 0x7fc83c158260 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fc83000c760 tx=0x7fc83000cc20 comp rx=0 tx=0).stop 2024-03-06T07:31:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 shutdown_connections 2024-03-06T07:31:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc82806b4d0 0x7fc82806d990 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc83c1527b0 0x7fc83c1ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc83c0c88b0 0x7fc83c158260 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 --2- 172.21.15.26:0/1404169604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc83c0c7180 0x7fc83c0bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 >> 172.21.15.26:0/1404169604 conn(0x7fc83c0ac560 msgr2=0x7fc83c0ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 shutdown_connections 2024-03-06T07:31:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:34.403+0000 7fc842855640 1 -- 172.21.15.26:0/1404169604 wait complete. 2024-03-06T07:31:34.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:34.818 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:35.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:35.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.920+0000 7effd4932640 1 Processor -- start 2024-03-06T07:31:35.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.920+0000 7effd4932640 1 -- start start 2024-03-06T07:31:35.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effd4932640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7effd014f650 0x7effd0071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effd4932640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effd00c3150 0x7effd00c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effd4932640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0152c20 0x7effd00c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effd4932640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd0072440 con 0x7effd00c3150 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effd4932640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd00c2b10 con 0x7effd014f650 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effd4932640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd0152140 con 0x7effd0152c20 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effc7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effd00c3150 0x7effd00c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effc7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effd00c3150 0x7effd00c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43784/0 (socket says 172.21.15.26:43784) 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.921+0000 7effc7fff640 1 -- 172.21.15.26:0/2930168120 learned_addr learned my addr 172.21.15.26:0/2930168120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effc7fff640 1 -- 172.21.15.26:0/2930168120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0152c20 msgr2=0x7effd00c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effced9d640 1 --2- 172.21.15.26:0/2930168120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0152c20 0x7effd00c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effce59c640 1 --2- 172.21.15.26:0/2930168120 >> v2:172.21.15.117:3300/0 conn(0x7effd014f650 0x7effd0071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effc7fff640 1 --2- 172.21.15.26:0/2930168120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0152c20 0x7effd00c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effc7fff640 1 -- 172.21.15.26:0/2930168120 >> v2:172.21.15.117:3300/0 conn(0x7effd014f650 msgr2=0x7effd0071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:35.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effc7fff640 1 --2- 172.21.15.26:0/2930168120 >> v2:172.21.15.117:3300/0 conn(0x7effd014f650 0x7effd0071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effc7fff640 1 -- 172.21.15.26:0/2930168120 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0152420 con 0x7effd00c3150 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.922+0000 7effc7fff640 1 --2- 172.21.15.26:0/2930168120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd00c3150 0x7effd00c5690 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7effd0070a90 tx=0x7effb802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862d37136c433b50 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effcdd9b640 1 -- 172.21.15.26:0/2930168120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb803b070 con 0x7effd00c3150 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effcdd9b640 1 -- 172.21.15.26:0/2930168120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8036440 con 0x7effd00c3150 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effcdd9b640 1 -- 172.21.15.26:0/2930168120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8036810 con 0x7effd00c3150 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 -- 172.21.15.26:0/2930168120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd00c3150 msgr2=0x7effd00c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 --2- 172.21.15.26:0/2930168120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd00c3150 0x7effd00c5690 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7effd0070a90 tx=0x7effb802e0e0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 -- 172.21.15.26:0/2930168120 shutdown_connections 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 --2- 172.21.15.26:0/2930168120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0152c20 0x7effd00c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 --2- 172.21.15.26:0/2930168120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd00c3150 0x7effd00c5690 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 --2- 172.21.15.26:0/2930168120 >> v2:172.21.15.117:3300/0 conn(0x7effd014f650 0x7effd0071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 -- 172.21.15.26:0/2930168120 >> 172.21.15.26:0/2930168120 conn(0x7effd00c4290 msgr2=0x7effd00c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 -- 172.21.15.26:0/2930168120 shutdown_connections 2024-03-06T07:31:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.923+0000 7effd4932640 1 -- 172.21.15.26:0/2930168120 wait complete. 2024-03-06T07:31:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.925+0000 7effd4932640 1 Processor -- start 2024-03-06T07:31:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.925+0000 7effd4932640 1 -- start start 2024-03-06T07:31:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effd4932640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c3150 0x7effd0148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effd4932640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd014f650 0x7effd006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effd4932640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd0152c20 0x7effd01dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effd4932640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd00c40c0 con 0x7effd0152c20 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effd4932640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd00c3f40 con 0x7effd00c3150 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effd4932640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd0071dc0 con 0x7effd014f650 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd0152c20 0x7effd01dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd0152c20 0x7effd01dcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43792/0 (socket says 172.21.15.26:43792) 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effced9d640 1 -- 172.21.15.26:0/2531617296 learned_addr learned my addr 172.21.15.26:0/2531617296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effc7fff640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd014f650 0x7effd006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.926+0000 7effce59c640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c3150 0x7effd0148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effced9d640 1 -- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd014f650 msgr2=0x7effd006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effced9d640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd014f650 0x7effd006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effced9d640 1 -- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c3150 msgr2=0x7effd0148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effced9d640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c3150 0x7effd0148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effced9d640 1 -- 172.21.15.26:0/2531617296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd01ea6f0 con 0x7effd0152c20 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effced9d640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd0152c20 0x7effd01dcce0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7effc800cc00 tx=0x7effc800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effc6ffd640 1 -- 172.21.15.26:0/2531617296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc800ce10 con 0x7effd0152c20 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effc6ffd640 1 -- 172.21.15.26:0/2531617296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8003990 con 0x7effd0152c20 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd01ea9d0 con 0x7effd0152c20 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effc6ffd640 1 -- 172.21.15.26:0/2531617296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8003d60 con 0x7effd0152c20 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.927+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd01517d0 con 0x7effd0152c20 2024-03-06T07:31:35.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.928+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd014ca40 con 0x7effd0152c20 2024-03-06T07:31:35.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.929+0000 7effc6ffd640 1 -- 172.21.15.26:0/2531617296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effc8003f00 con 0x7effd0152c20 2024-03-06T07:31:35.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.929+0000 7effc6ffd640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb006b4d0 0x7effb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.929+0000 7effc6ffd640 1 -- 172.21.15.26:0/2531617296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7effc808a7f0 con 0x7effd0152c20 2024-03-06T07:31:35.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.929+0000 7effce59c640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb006b4d0 0x7effb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.930+0000 7effce59c640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb006b4d0 0x7effb006d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7effd01de560 tx=0x7effc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:35.934+0000 7effc6ffd640 1 -- 172.21.15.26:0/2531617296 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7effd014ca40 con 0x7effd0152c20 2024-03-06T07:31:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.153+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effd004f330 con 0x7effd0152c20 2024-03-06T07:31:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.153+0000 7effc6ffd640 1 -- 172.21.15.26:0/2531617296 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7effc8014030 con 0x7effd0152c20 2024-03-06T07:31:36.155 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:36.155 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:36.155 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb006b4d0 msgr2=0x7effb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb006b4d0 0x7effb006d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7effd01de560 tx=0x7effc0008040 comp rx=0 tx=0).stop 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd0152c20 msgr2=0x7effd01dcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd0152c20 0x7effd01dcce0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7effc800cc00 tx=0x7effc800a5c0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 shutdown_connections 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7effb006b4d0 0x7effb006d990 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effd0152c20 0x7effd01dcce0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd014f650 0x7effd006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 --2- 172.21.15.26:0/2531617296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c3150 0x7effd0148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 >> 172.21.15.26:0/2531617296 conn(0x7effd00c4290 msgr2=0x7effd014da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 shutdown_connections 2024-03-06T07:31:36.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:36.155+0000 7effd4932640 1 -- 172.21.15.26:0/2531617296 wait complete. 2024-03-06T07:31:36.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:36.571 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:37.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:37.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.682+0000 7f569295e640 1 Processor -- start 2024-03-06T07:31:37.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.682+0000 7f569295e640 1 -- start start 2024-03-06T07:31:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f569295e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f568c06da40 0x7f568c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f569295e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c0c7380 0x7f568c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f569295e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c156050 con 0x7f568c0c6780 2024-03-06T07:31:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f569295e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c06de20 con 0x7f568c06da40 2024-03-06T07:31:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f569295e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c072630 con 0x7f568c0c7380 2024-03-06T07:31:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f568c06da40 0x7f568c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:37.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f568c0c6780 0x7f568c158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54486/0 (socket says 172.21.15.26:54486) 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568b7fe640 1 -- 172.21.15.26:0/4081210214 learned_addr learned my addr 172.21.15.26:0/4081210214 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f5690efb640 1 --2- 172.21.15.26:0/4081210214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c0c7380 0x7f568c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568bfff640 1 -- 172.21.15.26:0/4081210214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c0c7380 msgr2=0x7f568c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568bfff640 1 --2- 172.21.15.26:0/4081210214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c0c7380 0x7f568c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568bfff640 1 -- 172.21.15.26:0/4081210214 >> v1:172.21.15.26:6789/0 conn(0x7f568c0c6780 legacy=0x7f568c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568affd640 1 -- 172.21.15.26:0/4081210214 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487516862 0 0) 0x7f567c001110 con 0x7f568c0c6780 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.683+0000 7f568bfff640 1 -- 172.21.15.26:0/4081210214 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c072910 con 0x7f568c06da40 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.684+0000 7f568bfff640 1 --2- 172.21.15.26:0/4081210214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c06da40 0x7f568c155b10 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f56800097a0 tx=0x7f568002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c512cba795c252c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.684+0000 7f568affd640 1 -- 172.21.15.26:0/4081210214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56800315a0 con 0x7f568c06da40 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.684+0000 7f568affd640 1 -- 172.21.15.26:0/4081210214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5680037070 con 0x7f568c06da40 2024-03-06T07:31:37.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.684+0000 7f568affd640 1 -- 172.21.15.26:0/4081210214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680031960 con 0x7f568c06da40 2024-03-06T07:31:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 -- 172.21.15.26:0/4081210214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c06da40 msgr2=0x7f568c155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 --2- 172.21.15.26:0/4081210214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c06da40 0x7f568c155b10 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f56800097a0 tx=0x7f568002b840 comp rx=0 tx=0).stop 2024-03-06T07:31:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 -- 172.21.15.26:0/4081210214 shutdown_connections 2024-03-06T07:31:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 --2- 172.21.15.26:0/4081210214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c0c7380 0x7f568c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 --2- 172.21.15.26:0/4081210214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c06da40 0x7f568c155b10 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 -- 172.21.15.26:0/4081210214 >> 172.21.15.26:0/4081210214 conn(0x7f568c0aa630 msgr2=0x7f568c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 -- 172.21.15.26:0/4081210214 shutdown_connections 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.685+0000 7f569295e640 1 -- 172.21.15.26:0/4081210214 wait complete. 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.686+0000 7f569295e640 1 Processor -- start 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.686+0000 7f569295e640 1 -- start start 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f569295e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c06da40 0x7f568c156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f569295e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f568c0c6780 0x7f568c1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f569295e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c0c7380 0x7f568c1eb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f569295e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1594a0 con 0x7f568c0c7380 2024-03-06T07:31:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f569295e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c158890 con 0x7f568c0c6780 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f569295e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0c37c0 con 0x7f568c06da40 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c0c7380 0x7f568c1eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c0c7380 0x7f568c1eb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42692/0 (socket says 172.21.15.26:42692) 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 -- 172.21.15.26:0/3447667650 learned_addr learned my addr 172.21.15.26:0/3447667650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f568bfff640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c06da40 0x7f568c156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f568b7fe640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f568c0c6780 0x7f568c1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 -- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c06da40 msgr2=0x7f568c156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c06da40 0x7f568c156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 -- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f568c0c6780 msgr2=0x7f568c1520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f568c0c6780 0x7f568c1520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.687+0000 7f5690efb640 1 -- 172.21.15.26:0/3447667650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c14c020 con 0x7f568c0c7380 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.688+0000 7f5690efb640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c0c7380 0x7f568c1eb340 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f5684007ca0 tx=0x7f568400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.688+0000 7f5688ff9640 1 -- 172.21.15.26:0/3447667650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684014420 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.688+0000 7f5688ff9640 1 -- 172.21.15.26:0/3447667650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684019070 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.688+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c14c780 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.688+0000 7f5688ff9640 1 -- 172.21.15.26:0/3447667650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684014800 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.688+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c152610 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.689+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c04f330 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.689+0000 7f5688ff9640 1 -- 172.21.15.26:0/3447667650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56840149a0 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.690+0000 7f5688ff9640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f567806b500 0x7f567806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.690+0000 7f5688ff9640 1 -- 172.21.15.26:0/3447667650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f568408ab80 con 0x7f568c0c7380 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.690+0000 7f568bfff640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f567806b500 0x7f567806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.690+0000 7f568bfff640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f567806b500 0x7f567806d9c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5680036000 tx=0x7f568003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.694+0000 7f5688ff9640 1 -- 172.21.15.26:0/3447667650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5684059800 con 0x7f568c0c7380 2024-03-06T07:31:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.915+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f568c073240 con 0x7f568c0c7380 2024-03-06T07:31:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.916+0000 7f5688ff9640 1 -- 172.21.15.26:0/3447667650 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f5684059100 con 0x7f568c0c7380 2024-03-06T07:31:37.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:37.918 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f567806b500 msgr2=0x7f567806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f567806b500 0x7f567806d9c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5680036000 tx=0x7f568003d040 comp rx=0 tx=0).stop 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c0c7380 msgr2=0x7f568c1eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c0c7380 0x7f568c1eb340 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f5684007ca0 tx=0x7f568400e4b0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 shutdown_connections 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f567806b500 0x7f567806d9c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568c0c7380 0x7f568c1eb340 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f568c0c6780 0x7f568c1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 --2- 172.21.15.26:0/3447667650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f568c06da40 0x7f568c156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 >> 172.21.15.26:0/3447667650 conn(0x7f568c0aa630 msgr2=0x7f568c0c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 shutdown_connections 2024-03-06T07:31:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:37.918+0000 7f569295e640 1 -- 172.21.15.26:0/3447667650 wait complete. 2024-03-06T07:31:38.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:38.334 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:39.335 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:39.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.435+0000 7fded2c6b640 1 Processor -- start 2024-03-06T07:31:39.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.435+0000 7fded2c6b640 1 -- start start 2024-03-06T07:31:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fded2c6b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdecc1592d0 0x7fdecc1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fded2c6b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc0ac260 0x7fdecc155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fded2c6b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc152d40 con 0x7fdecc152960 2024-03-06T07:31:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fded2c6b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc066e30 con 0x7fdecc1592d0 2024-03-06T07:31:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fded2c6b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc155d50 con 0x7fdecc0ac260 2024-03-06T07:31:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fded1208640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc0ac260 0x7fdecc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecbfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdecc1592d0 0x7fdecc1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecbfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdecc1592d0 0x7fdecc1552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:49834/0 (socket says 172.21.15.26:49834) 2024-03-06T07:31:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecbfff640 1 -- 172.21.15.26:0/3181992501 learned_addr learned my addr 172.21.15.26:0/3181992501 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecbfff640 1 -- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc0ac260 msgr2=0x7fdecc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecbfff640 1 --2- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc0ac260 0x7fdecc155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecbfff640 1 -- 172.21.15.26:0/3181992501 >> v1:172.21.15.117:6789/0 conn(0x7fdecc152960 legacy=0x7fdecc06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecb7fe640 1 -- 172.21.15.26:0/3181992501 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763752244 0 0) 0x7fdecc152d40 con 0x7fdecc152960 2024-03-06T07:31:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fdecbfff640 1 -- 172.21.15.26:0/3181992501 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc0c1b50 con 0x7fdecc1592d0 2024-03-06T07:31:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.436+0000 7fded1208640 1 --2- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc0ac260 0x7fdecc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fdecbfff640 1 --2- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1592d0 0x7fdecc1552d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009870 tx=0x7fdeb802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71fee5cc6347cfca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fdecb7fe640 1 -- 172.21.15.26:0/3181992501 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8034070 con 0x7fdecc1592d0 2024-03-06T07:31:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fdecb7fe640 1 -- 172.21.15.26:0/3181992501 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb8030840 con 0x7fdecc1592d0 2024-03-06T07:31:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fdecb7fe640 1 -- 172.21.15.26:0/3181992501 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8030c10 con 0x7fdecc1592d0 2024-03-06T07:31:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fded2c6b640 1 -- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1592d0 msgr2=0x7fdecc1552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fded2c6b640 1 --2- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1592d0 0x7fdecc1552d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009870 tx=0x7fdeb802b980 comp rx=0 tx=0).stop 2024-03-06T07:31:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fded2c6b640 1 -- 172.21.15.26:0/3181992501 shutdown_connections 2024-03-06T07:31:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fded2c6b640 1 --2- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc0ac260 0x7fdecc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fded2c6b640 1 --2- 172.21.15.26:0/3181992501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1592d0 0x7fdecc1552d0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.438+0000 7fded2c6b640 1 -- 172.21.15.26:0/3181992501 >> 172.21.15.26:0/3181992501 conn(0x7fdecc0c70b0 msgr2=0x7fdecc0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.439+0000 7fded2c6b640 1 -- 172.21.15.26:0/3181992501 shutdown_connections 2024-03-06T07:31:39.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.439+0000 7fded2c6b640 1 -- 172.21.15.26:0/3181992501 wait complete. 2024-03-06T07:31:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 Processor -- start 2024-03-06T07:31:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 -- start start 2024-03-06T07:31:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdecc0ac260 0x7fdecc156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc152960 0x7fdecc1408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1de930 0x7fdecc140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc159760 con 0x7fdecc0ac260 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0672c0 con 0x7fdecc1de930 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded2c6b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc06e3a0 con 0x7fdecc152960 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.440+0000 7fded0a07640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdecc0ac260 0x7fdecc156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdecc0ac260 0x7fdecc156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42694/0 (socket says 172.21.15.26:42694) 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 -- 172.21.15.26:0/873279930 learned_addr learned my addr 172.21.15.26:0/873279930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fdecbfff640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc152960 0x7fdecc1408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded1208640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1de930 0x7fdecc140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 -- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc152960 msgr2=0x7fdecc1408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc152960 0x7fdecc1408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 -- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1de930 msgr2=0x7fdecc140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1de930 0x7fdecc140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 -- 172.21.15.26:0/873279930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc1e0010 con 0x7fdecc0ac260 2024-03-06T07:31:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded1208640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1de930 0x7fdecc140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded0a07640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdecc0ac260 0x7fdecc156c40 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c4e0 tx=0x7fdec000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fdec97fa640 1 -- 172.21.15.26:0/873279930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0017070 con 0x7fdecc0ac260 2024-03-06T07:31:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fdec97fa640 1 -- 172.21.15.26:0/873279930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec00035f0 con 0x7fdecc0ac260 2024-03-06T07:31:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc1df0a0 con 0x7fdecc0ac260 2024-03-06T07:31:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fdec97fa640 1 -- 172.21.15.26:0/873279930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00039e0 con 0x7fdecc0ac260 2024-03-06T07:31:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.441+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc158540 con 0x7fdecc0ac260 2024-03-06T07:31:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.442+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc152f30 con 0x7fdecc0ac260 2024-03-06T07:31:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.443+0000 7fdec97fa640 1 -- 172.21.15.26:0/873279930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdec001a070 con 0x7fdecc0ac260 2024-03-06T07:31:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.443+0000 7fdec97fa640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdeb006b4d0 0x7fdeb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.443+0000 7fdec97fa640 1 -- 172.21.15.26:0/873279930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdec0089780 con 0x7fdecc0ac260 2024-03-06T07:31:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.443+0000 7fdecbfff640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdeb006b4d0 0x7fdeb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.444+0000 7fdecbfff640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdeb006b4d0 0x7fdeb006d990 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8036000 tx=0x7fdeb8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:39.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.447+0000 7fdec97fa640 1 -- 172.21.15.26:0/873279930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdecc152f30 con 0x7fdecc0ac260 2024-03-06T07:31:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.722+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdecc156270 con 0x7fdecc0ac260 2024-03-06T07:31:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.722+0000 7fdec97fa640 1 -- 172.21.15.26:0/873279930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fdecc1e0010 con 0x7fdecc0ac260 2024-03-06T07:31:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:39.724 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdeb006b4d0 msgr2=0x7fdeb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdeb006b4d0 0x7fdeb006d990 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8036000 tx=0x7fdeb8008030 comp rx=0 tx=0).stop 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdecc0ac260 msgr2=0x7fdecc156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdecc0ac260 0x7fdecc156c40 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c4e0 tx=0x7fdec000c9a0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 shutdown_connections 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdeb006b4d0 0x7fdeb006d990 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdecc1de930 0x7fdecc140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdecc152960 0x7fdecc1408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 --2- 172.21.15.26:0/873279930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdecc0ac260 0x7fdecc156c40 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 >> 172.21.15.26:0/873279930 conn(0x7fdecc0c70b0 msgr2=0x7fdecc0c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 shutdown_connections 2024-03-06T07:31:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:39.724+0000 7fded2c6b640 1 -- 172.21.15.26:0/873279930 wait complete. 2024-03-06T07:31:40.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:40.139 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:41.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.239+0000 7f1ca6647640 1 Processor -- start 2024-03-06T07:31:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca6647640 1 -- start start 2024-03-06T07:31:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca6647640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca014f360 0x7f1ca0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca6647640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ca00c88c0 0x7f1ca00c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca6647640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca00c73d0 con 0x7f1ca00c5880 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca6647640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca00c8dd0 con 0x7f1ca00c88c0 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca6647640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca00be2e0 con 0x7f1ca014f360 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1c9f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ca00c88c0 0x7f1ca00c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca4be4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ca00c5880 0x7f1ca0158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54492/0 (socket says 172.21.15.26:54492) 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.240+0000 7f1ca4be4640 1 -- 172.21.15.26:0/1026627954 learned_addr learned my addr 172.21.15.26:0/1026627954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.241+0000 7f1c9ffff640 1 --2- 172.21.15.26:0/1026627954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca014f360 0x7f1ca0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.241+0000 7f1c9f7fe640 1 -- 172.21.15.26:0/1026627954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca014f360 msgr2=0x7f1ca0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.241+0000 7f1c9f7fe640 1 --2- 172.21.15.26:0/1026627954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca014f360 0x7f1ca0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.241+0000 7f1c9f7fe640 1 -- 172.21.15.26:0/1026627954 >> v1:172.21.15.26:6789/0 conn(0x7f1ca00c5880 legacy=0x7f1ca0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.241+0000 7f1c9effd640 1 -- 172.21.15.26:0/1026627954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478366135 0 0) 0x7f1ca00c73d0 con 0x7f1ca00c5880 2024-03-06T07:31:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.241+0000 7f1c9f7fe640 1 -- 172.21.15.26:0/1026627954 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00c7670 con 0x7f1ca00c88c0 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1c9f7fe640 1 --2- 172.21.15.26:0/1026627954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca00c88c0 0x7f1ca00c5340 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f1c940097a0 tx=0x7f1c9402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852c30f59bd61b9c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1c9effd640 1 -- 172.21.15.26:0/1026627954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940315a0 con 0x7f1ca00c88c0 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1c9effd640 1 -- 172.21.15.26:0/1026627954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94037070 con 0x7f1ca00c88c0 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1c9effd640 1 -- 172.21.15.26:0/1026627954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94031960 con 0x7f1ca00c88c0 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 -- 172.21.15.26:0/1026627954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca00c88c0 msgr2=0x7f1ca00c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 --2- 172.21.15.26:0/1026627954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca00c88c0 0x7f1ca00c5340 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f1c940097a0 tx=0x7f1c9402b840 comp rx=0 tx=0).stop 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 -- 172.21.15.26:0/1026627954 shutdown_connections 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 --2- 172.21.15.26:0/1026627954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca00c88c0 0x7f1ca00c5340 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 --2- 172.21.15.26:0/1026627954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca014f360 0x7f1ca0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 -- 172.21.15.26:0/1026627954 >> 172.21.15.26:0/1026627954 conn(0x7f1ca00ac570 msgr2=0x7f1ca00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 -- 172.21.15.26:0/1026627954 shutdown_connections 2024-03-06T07:31:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.242+0000 7f1ca6647640 1 -- 172.21.15.26:0/1026627954 wait complete. 2024-03-06T07:31:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.244+0000 7f1ca6647640 1 Processor -- start 2024-03-06T07:31:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.244+0000 7f1ca6647640 1 -- start start 2024-03-06T07:31:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca6647640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca00c5880 0x7f1ca00ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca6647640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00c88c0 0x7f1ca01ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca6647640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca014f360 0x7f1ca01ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca6647640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca0159270 con 0x7f1ca014f360 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca6647640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca00c7c30 con 0x7f1ca00c5880 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca6647640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca01593f0 con 0x7f1ca00c88c0 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca014f360 0x7f1ca01ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca014f360 0x7f1ca01ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42710/0 (socket says 172.21.15.26:42710) 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 -- 172.21.15.26:0/860162840 learned_addr learned my addr 172.21.15.26:0/860162840 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1c9ffff640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca00c5880 0x7f1ca00ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1c9f7fe640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00c88c0 0x7f1ca01ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 -- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00c88c0 msgr2=0x7f1ca01ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00c88c0 0x7f1ca01ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 -- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca00c5880 msgr2=0x7f1ca00ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca00c5880 0x7f1ca00ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.245+0000 7f1ca4be4640 1 -- 172.21.15.26:0/860162840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca01ee890 con 0x7f1ca014f360 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.246+0000 7f1c9f7fe640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00c88c0 0x7f1ca01ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.246+0000 7f1ca4be4640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca014f360 0x7f1ca01ee350 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c7a0 tx=0x7f1c9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.246+0000 7f1c9cff9640 1 -- 172.21.15.26:0/860162840 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98018070 con 0x7f1ca014f360 2024-03-06T07:31:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.246+0000 7f1c9cff9640 1 -- 172.21.15.26:0/860162840 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9800a640 con 0x7f1ca014f360 2024-03-06T07:31:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.246+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca00af5d0 con 0x7f1ca014f360 2024-03-06T07:31:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.246+0000 7f1c9cff9640 1 -- 172.21.15.26:0/860162840 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800aa70 con 0x7f1ca014f360 2024-03-06T07:31:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.246+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca00adda0 con 0x7f1ca014f360 2024-03-06T07:31:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.247+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca004f330 con 0x7f1ca014f360 2024-03-06T07:31:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.247+0000 7f1c9cff9640 1 -- 172.21.15.26:0/860162840 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c9801b070 con 0x7f1ca014f360 2024-03-06T07:31:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.248+0000 7f1c9cff9640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c8c06b500 0x7f1c8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.248+0000 7f1c9cff9640 1 -- 172.21.15.26:0/860162840 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f1c980898b0 con 0x7f1ca014f360 2024-03-06T07:31:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.248+0000 7f1c9ffff640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c8c06b500 0x7f1c8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.249+0000 7f1c9ffff640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c8c06b500 0x7f1c8c06d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009aa0 tx=0x7f1c90009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.252+0000 7f1c9cff9640 1 -- 172.21.15.26:0/860162840 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1c98057ee0 con 0x7f1ca014f360 2024-03-06T07:31:41.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.474+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ca006cda0 con 0x7f1ca014f360 2024-03-06T07:31:41.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.475+0000 7f1c9cff9640 1 -- 172.21.15.26:0/860162840 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f1c9805bf80 con 0x7f1ca014f360 2024-03-06T07:31:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:41.477 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:41.477 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c8c06b500 msgr2=0x7f1c8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c8c06b500 0x7f1c8c06d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009aa0 tx=0x7f1c90009190 comp rx=0 tx=0).stop 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca014f360 msgr2=0x7f1ca01ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca014f360 0x7f1ca01ee350 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c7a0 tx=0x7f1c9800cc60 comp rx=0 tx=0).stop 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 shutdown_connections 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1c8c06b500 0x7f1c8c06d9c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca014f360 0x7f1ca01ee350 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00c88c0 0x7f1ca01ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 --2- 172.21.15.26:0/860162840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca00c5880 0x7f1ca00ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:41.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 >> 172.21.15.26:0/860162840 conn(0x7f1ca00ac570 msgr2=0x7f1ca00c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:41.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 shutdown_connections 2024-03-06T07:31:41.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:41.477+0000 7f1ca6647640 1 -- 172.21.15.26:0/860162840 wait complete. 2024-03-06T07:31:41.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:41.892 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:42.893 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.993+0000 7fcec7fd0640 1 Processor -- start 2024-03-06T07:31:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.993+0000 7fcec7fd0640 1 -- start start 2024-03-06T07:31:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec7fd0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcec01506b0 0x7fcec0153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec7fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec7fd0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcec00acc00 0x7fcec00c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec7fd0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec0156450 con 0x7fcec01506b0 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec7fd0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec0156a10 con 0x7fcec00acc00 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec7fd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0070800 con 0x7fcec0156ea0 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec5d6c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcec01506b0 0x7fcec0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec5d6c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcec01506b0 0x7fcec0153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42720/0 (socket says 172.21.15.26:42720) 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec5d6c640 1 -- 172.21.15.26:0/2450417157 learned_addr learned my addr 172.21.15.26:0/2450417157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec556b640 1 --2- 172.21.15.26:0/2450417157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.994+0000 7fcec656d640 1 --2- 172.21.15.26:0/2450417157 >> v2:172.21.15.117:3300/0 conn(0x7fcec00acc00 0x7fcec00c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.995+0000 7fcec5d6c640 1 -- 172.21.15.26:0/2450417157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 msgr2=0x7fcec00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.995+0000 7fcec5d6c640 1 --2- 172.21.15.26:0/2450417157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.995+0000 7fcec5d6c640 1 -- 172.21.15.26:0/2450417157 >> v2:172.21.15.117:3300/0 conn(0x7fcec00acc00 msgr2=0x7fcec00c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.995+0000 7fcec5d6c640 1 --2- 172.21.15.26:0/2450417157 >> v2:172.21.15.117:3300/0 conn(0x7fcec00acc00 0x7fcec00c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.995+0000 7fcec5d6c640 1 -- 172.21.15.26:0/2450417157 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0070aa0 con 0x7fcec01506b0 2024-03-06T07:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.995+0000 7fcec656d640 1 --2- 172.21.15.26:0/2450417157 >> v2:172.21.15.117:3300/0 conn(0x7fcec00acc00 0x7fcec00c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.996+0000 7fcec5d6c640 1 --2- 172.21.15.26:0/2450417157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec0153990 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fcec00c6270 tx=0x7fcebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacd23d658a6e27d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.996+0000 7fcec4d6a640 1 -- 172.21.15.26:0/2450417157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc035020 con 0x7fcec01506b0 2024-03-06T07:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.996+0000 7fcec4d6a640 1 -- 172.21.15.26:0/2450417157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc031870 con 0x7fcec01506b0 2024-03-06T07:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.996+0000 7fcec4d6a640 1 -- 172.21.15.26:0/2450417157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc031c40 con 0x7fcec01506b0 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.996+0000 7fcec7fd0640 1 -- 172.21.15.26:0/2450417157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 msgr2=0x7fcec0153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.996+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/2450417157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec0153990 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fcec00c6270 tx=0x7fcebc02b910 comp rx=0 tx=0).stop 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.997+0000 7fcec7fd0640 1 -- 172.21.15.26:0/2450417157 shutdown_connections 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.997+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/2450417157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.997+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/2450417157 >> v2:172.21.15.117:3300/0 conn(0x7fcec00acc00 0x7fcec00c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.997+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/2450417157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec0153990 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.997+0000 7fcec7fd0640 1 -- 172.21.15.26:0/2450417157 >> 172.21.15.26:0/2450417157 conn(0x7fcec006b1a0 msgr2=0x7fcec006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.997+0000 7fcec7fd0640 1 -- 172.21.15.26:0/2450417157 shutdown_connections 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.997+0000 7fcec7fd0640 1 -- 172.21.15.26:0/2450417157 wait complete. 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 Processor -- start 2024-03-06T07:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 -- start start 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec00acc00 0x7fcec00aa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00ad070 con 0x7fcec01506b0 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec01566a0 con 0x7fcec00acc00 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec7fd0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00ad1f0 con 0x7fcec0156ea0 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec556b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.998+0000 7fcec556b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec014aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42734/0 (socket says 172.21.15.26:42734) 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec556b640 1 -- 172.21.15.26:0/1967120100 learned_addr learned my addr 172.21.15.26:0/1967120100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec5d6c640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec00acc00 0x7fcec00aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec656d640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec556b640 1 -- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 msgr2=0x7fcec014c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec556b640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec014c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec556b640 1 -- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec00acc00 msgr2=0x7fcec00aa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec556b640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec00acc00 0x7fcec00aa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec556b640 1 -- 172.21.15.26:0/1967120100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec014afd0 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec556b640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec014aa90 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fceb000d5d0 tx=0x7fceb000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fceb67fc640 1 -- 172.21.15.26:0/1967120100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb00125f0 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fceb67fc640 1 -- 172.21.15.26:0/1967120100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb0019070 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec00c7a00 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fceb67fc640 1 -- 172.21.15.26:0/1967120100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb00129f0 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:42.999+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec01e8a10 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.000+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec00aad30 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.001+0000 7fceb67fc640 1 -- 172.21.15.26:0/1967120100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fceb0012b90 con 0x7fcec01506b0 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.001+0000 7fceb67fc640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcea406b500 0x7fcea406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.001+0000 7fceb67fc640 1 -- 172.21.15.26:0/1967120100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fceb008a7b0 con 0x7fcec01506b0 2024-03-06T07:31:43.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.001+0000 7fcec5d6c640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcea406b500 0x7fcea406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.002+0000 7fcec5d6c640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcea406b500 0x7fcea406d9c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcebc036000 tx=0x7fcebc03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:43.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.005+0000 7fceb67fc640 1 -- 172.21.15.26:0/1967120100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcec00aad30 con 0x7fcec01506b0 2024-03-06T07:31:43.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.223+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcec004f330 con 0x7fcec01506b0 2024-03-06T07:31:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.223+0000 7fceb67fc640 1 -- 172.21.15.26:0/1967120100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fcec00c7a00 con 0x7fcec01506b0 2024-03-06T07:31:43.225 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:43.226 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:43.226 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:43.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcea406b500 msgr2=0x7fcea406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcea406b500 0x7fcea406d9c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcebc036000 tx=0x7fcebc03c040 comp rx=0 tx=0).stop 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 msgr2=0x7fcec014aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec014aa90 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fceb000d5d0 tx=0x7fceb000da90 comp rx=0 tx=0).stop 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 shutdown_connections 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcea406b500 0x7fcea406d9c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcec0156ea0 0x7fcec014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcec01506b0 0x7fcec014aa90 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 --2- 172.21.15.26:0/1967120100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec00acc00 0x7fcec00aa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.225+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 >> 172.21.15.26:0/1967120100 conn(0x7fcec006b1a0 msgr2=0x7fcec00c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.226+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 shutdown_connections 2024-03-06T07:31:43.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:43.226+0000 7fcec7fd0640 1 -- 172.21.15.26:0/1967120100 wait complete. 2024-03-06T07:31:43.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:43.597 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:43.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:43.640 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:44.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:44.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.742+0000 7febf1a2a640 1 Processor -- start 2024-03-06T07:31:44.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.742+0000 7febf1a2a640 1 -- start start 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febf1a2a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7febec1527b0 0x7febec159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febf1a2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec0c7180 0x7febec0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febf1a2a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7febec155b20 0x7febec0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febf1a2a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febec159640 con 0x7febec1527b0 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febf1a2a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febec0c8df0 con 0x7febec155b20 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febf1a2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec0c7c60 con 0x7febec0c7180 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febeaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7febec1527b0 0x7febec159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:44.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febeaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7febec1527b0 0x7febec159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42744/0 (socket says 172.21.15.26:42744) 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febeaffd640 1 -- 172.21.15.26:0/2935111899 learned_addr learned my addr 172.21.15.26:0/2935111899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febeb7fe640 1 --2- 172.21.15.26:0/2935111899 >> v2:172.21.15.117:3300/0 conn(0x7febec155b20 0x7febec0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.743+0000 7febea7fc640 1 --2- 172.21.15.26:0/2935111899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec0c7180 0x7febec0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.744+0000 7febeaffd640 1 -- 172.21.15.26:0/2935111899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec0c7180 msgr2=0x7febec0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.744+0000 7febeaffd640 1 --2- 172.21.15.26:0/2935111899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec0c7180 0x7febec0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.744+0000 7febeaffd640 1 -- 172.21.15.26:0/2935111899 >> v2:172.21.15.117:3300/0 conn(0x7febec155b20 msgr2=0x7febec0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.744+0000 7febeaffd640 1 --2- 172.21.15.26:0/2935111899 >> v2:172.21.15.117:3300/0 conn(0x7febec155b20 0x7febec0c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.744+0000 7febeaffd640 1 -- 172.21.15.26:0/2935111899 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0c8060 con 0x7febec1527b0 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.744+0000 7febeaffd640 1 --2- 172.21.15.26:0/2935111899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec1527b0 0x7febec159100 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7febec06b170 tx=0x7febdc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59dfdf0389fee72 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febe9ffb640 1 -- 172.21.15.26:0/2935111899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc02e070 con 0x7febec1527b0 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febe9ffb640 1 -- 172.21.15.26:0/2935111899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc037070 con 0x7febec1527b0 2024-03-06T07:31:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febe9ffb640 1 -- 172.21.15.26:0/2935111899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc031a00 con 0x7febec1527b0 2024-03-06T07:31:44.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 -- 172.21.15.26:0/2935111899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec1527b0 msgr2=0x7febec159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:44.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 --2- 172.21.15.26:0/2935111899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec1527b0 0x7febec159100 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7febec06b170 tx=0x7febdc02b880 comp rx=0 tx=0).stop 2024-03-06T07:31:44.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 -- 172.21.15.26:0/2935111899 shutdown_connections 2024-03-06T07:31:44.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 --2- 172.21.15.26:0/2935111899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec0c7180 0x7febec0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 --2- 172.21.15.26:0/2935111899 >> v2:172.21.15.117:3300/0 conn(0x7febec155b20 0x7febec0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 --2- 172.21.15.26:0/2935111899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec1527b0 0x7febec159100 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 -- 172.21.15.26:0/2935111899 >> 172.21.15.26:0/2935111899 conn(0x7febec0ac560 msgr2=0x7febec0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 -- 172.21.15.26:0/2935111899 shutdown_connections 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.745+0000 7febf1a2a640 1 -- 172.21.15.26:0/2935111899 wait complete. 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.746+0000 7febf1a2a640 1 Processor -- start 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.746+0000 7febf1a2a640 1 -- start start 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febf1a2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0c7180 0x7febec14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febf1a2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec1527b0 0x7febec148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febf1a2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec155b20 0x7febec148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febf1a2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec155320 con 0x7febec155b20 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febf1a2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0c7980 con 0x7febec0c7180 2024-03-06T07:31:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febf1a2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec155600 con 0x7febec1527b0 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec155b20 0x7febec148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec155b20 0x7febec148c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42746/0 (socket says 172.21.15.26:42746) 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 -- 172.21.15.26:0/70191945 learned_addr learned my addr 172.21.15.26:0/70191945 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeaffd640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0c7180 0x7febec14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febea7fc640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec1527b0 0x7febec148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 -- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec1527b0 msgr2=0x7febec148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec1527b0 0x7febec148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 -- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0c7180 msgr2=0x7febec14de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0c7180 0x7febec14de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.747+0000 7febeb7fe640 1 -- 172.21.15.26:0/70191945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec156fa0 con 0x7febec155b20 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febea7fc640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec1527b0 0x7febec148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febeaffd640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0c7180 0x7febec14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febeb7fe640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec155b20 0x7febec148c90 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7febe400efc0 tx=0x7febe400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:44.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febd3fff640 1 -- 172.21.15.26:0/70191945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4018020 con 0x7febec155b20 2024-03-06T07:31:44.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febd3fff640 1 -- 172.21.15.26:0/70191945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4014480 con 0x7febec155b20 2024-03-06T07:31:44.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec157280 con 0x7febec155b20 2024-03-06T07:31:44.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febd3fff640 1 -- 172.21.15.26:0/70191945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4014870 con 0x7febec155b20 2024-03-06T07:31:44.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.748+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec14e350 con 0x7febec155b20 2024-03-06T07:31:44.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.749+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febec04f330 con 0x7febec155b20 2024-03-06T07:31:44.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.749+0000 7febd3fff640 1 -- 172.21.15.26:0/70191945 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febe4005d10 con 0x7febec155b20 2024-03-06T07:31:44.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.750+0000 7febd3fff640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febd806b4d0 0x7febd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:44.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.750+0000 7febd3fff640 1 -- 172.21.15.26:0/70191945 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7febe408b220 con 0x7febec155b20 2024-03-06T07:31:44.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.750+0000 7febeaffd640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febd806b4d0 0x7febd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:44.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.751+0000 7febeaffd640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febd806b4d0 0x7febd806d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7febdc036000 tx=0x7febdc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:44.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.754+0000 7febd3fff640 1 -- 172.21.15.26:0/70191945 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7febe4022030 con 0x7febec155b20 2024-03-06T07:31:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.978+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febec06ce00 con 0x7febec155b20 2024-03-06T07:31:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.978+0000 7febd3fff640 1 -- 172.21.15.26:0/70191945 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7febe4010040 con 0x7febec155b20 2024-03-06T07:31:44.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:44.981 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febd806b4d0 msgr2=0x7febd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febd806b4d0 0x7febd806d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7febdc036000 tx=0x7febdc008030 comp rx=0 tx=0).stop 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec155b20 msgr2=0x7febec148c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec155b20 0x7febec148c90 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7febe400efc0 tx=0x7febe400c490 comp rx=0 tx=0).stop 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 shutdown_connections 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7febd806b4d0 0x7febd806d990 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec155b20 0x7febec148c90 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febec1527b0 0x7febec148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 --2- 172.21.15.26:0/70191945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0c7180 0x7febec14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 >> 172.21.15.26:0/70191945 conn(0x7febec0ac560 msgr2=0x7febec0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 shutdown_connections 2024-03-06T07:31:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:44.981+0000 7febf1a2a640 1 -- 172.21.15.26:0/70191945 wait complete. 2024-03-06T07:31:45.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:45.396 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:46.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.497+0000 7fdcaf3b5640 1 Processor -- start 2024-03-06T07:31:46.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.497+0000 7fdcaf3b5640 1 -- start start 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcaf3b5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdca80c88c0 0x7fdca8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcaf3b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca8158b60 0x7fdca8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcaf3b5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca80c73d0 con 0x7fdca814f360 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcaf3b5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca80c8dd0 con 0x7fdca80c88c0 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcaf3b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80be2e0 con 0x7fdca8158b60 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcad952640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca8158b60 0x7fdca8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcac950640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdca80c88c0 0x7fdca8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcad151640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fdca814f360 0x7fdca80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:47758/0 (socket says 172.21.15.26:47758) 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.498+0000 7fdcad151640 1 -- 172.21.15.26:0/3995765838 learned_addr learned my addr 172.21.15.26:0/3995765838 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.499+0000 7fdc9ffff640 1 -- 172.21.15.26:0/3995765838 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696240829 0 0) 0x7fdca80c73d0 con 0x7fdca814f360 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.499+0000 7fdc9ffff640 1 -- 172.21.15.26:0/3995765838 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc94003520 con 0x7fdca814f360 2024-03-06T07:31:46.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.499+0000 7fdcac950640 1 -- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca8158b60 msgr2=0x7fdca8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.499+0000 7fdcac950640 1 --2- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca8158b60 0x7fdca8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.499+0000 7fdcac950640 1 -- 172.21.15.26:0/3995765838 >> v1:172.21.15.117:6789/0 conn(0x7fdca814f360 legacy=0x7fdca80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.499+0000 7fdcac950640 1 -- 172.21.15.26:0/3995765838 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80c7670 con 0x7fdca80c88c0 2024-03-06T07:31:46.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.499+0000 7fdcad952640 1 --2- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca8158b60 0x7fdca8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:46.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.500+0000 7fdcac950640 1 --2- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 0x7fdca8155c10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdc980097a0 tx=0x7fdc9802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d012f853dc1250 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:46.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.500+0000 7fdc9ffff640 1 -- 172.21.15.26:0/3995765838 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980315a0 con 0x7fdca80c88c0 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.500+0000 7fdc9ffff640 1 -- 172.21.15.26:0/3995765838 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98037070 con 0x7fdca80c88c0 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.500+0000 7fdc9ffff640 1 -- 172.21.15.26:0/3995765838 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98031940 con 0x7fdca80c88c0 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 msgr2=0x7fdca8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 0x7fdca8155c10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdc980097a0 tx=0x7fdc9802b840 comp rx=0 tx=0).stop 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/3995765838 shutdown_connections 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca8158b60 0x7fdca8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/3995765838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 0x7fdca8155c10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/3995765838 >> 172.21.15.26:0/3995765838 conn(0x7fdca80ac570 msgr2=0x7fdca80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/3995765838 shutdown_connections 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.501+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/3995765838 wait complete. 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.502+0000 7fdcaf3b5640 1 Processor -- start 2024-03-06T07:31:46.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.502+0000 7fdcaf3b5640 1 -- start start 2024-03-06T07:31:46.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcaf3b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 0x7fdca814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:46.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcaf3b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca814f360 0x7fdca80c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:46.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcaf3b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca8158b60 0x7fdca80bf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:46.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcaf3b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80c5a50 con 0x7fdca8158b60 2024-03-06T07:31:46.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcaf3b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80c7c30 con 0x7fdca80c88c0 2024-03-06T07:31:46.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcaf3b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80c5bd0 con 0x7fdca814f360 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca8158b60 0x7fdca80bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca8158b60 0x7fdca80bf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42758/0 (socket says 172.21.15.26:42758) 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 -- 172.21.15.26:0/2514877256 learned_addr learned my addr 172.21.15.26:0/2514877256 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcac950640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca814f360 0x7fdca80c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad151640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 0x7fdca814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 -- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca814f360 msgr2=0x7fdca80c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca814f360 0x7fdca80c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 -- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 msgr2=0x7fdca814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 0x7fdca814ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 -- 172.21.15.26:0/2514877256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca81eb660 con 0x7fdca8158b60 2024-03-06T07:31:46.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdcad952640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca8158b60 0x7fdca80bf100 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fdca000efc0 tx=0x7fdca0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdc9dffb640 1 -- 172.21.15.26:0/2514877256 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0018030 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.503+0000 7fdc9dffb640 1 -- 172.21.15.26:0/2514877256 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000cd80 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.504+0000 7fdc9dffb640 1 -- 172.21.15.26:0/2514877256 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0003bb0 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.504+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca81eb900 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.504+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca80be900 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.505+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca804f330 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.505+0000 7fdc9dffb640 1 -- 172.21.15.26:0/2514877256 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdca001b070 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.506+0000 7fdc9dffb640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdc8c06b4d0 0x7fdc8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.506+0000 7fdc9dffb640 1 -- 172.21.15.26:0/2514877256 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdca008ad50 con 0x7fdca8158b60 2024-03-06T07:31:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.506+0000 7fdcad151640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdc8c06b4d0 0x7fdc8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:46.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.506+0000 7fdcad151640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdc8c06b4d0 0x7fdc8c06d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005880 tx=0x7fdca4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:46.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.510+0000 7fdc9dffb640 1 -- 172.21.15.26:0/2514877256 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdca008e030 con 0x7fdca8158b60 2024-03-06T07:31:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.728+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdca814c580 con 0x7fdca8158b60 2024-03-06T07:31:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.729+0000 7fdc9dffb640 1 -- 172.21.15.26:0/2514877256 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fdca0010040 con 0x7fdca8158b60 2024-03-06T07:31:46.731 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:46.731 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdc8c06b4d0 msgr2=0x7fdc8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdc8c06b4d0 0x7fdc8c06d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005880 tx=0x7fdca4009040 comp rx=0 tx=0).stop 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca8158b60 msgr2=0x7fdca80bf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca8158b60 0x7fdca80bf100 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fdca000efc0 tx=0x7fdca0009f90 comp rx=0 tx=0).stop 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 shutdown_connections 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdc8c06b4d0 0x7fdc8c06d990 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca8158b60 0x7fdca80bf100 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdca814f360 0x7fdca80c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 --2- 172.21.15.26:0/2514877256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdca80c88c0 0x7fdca814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 >> 172.21.15.26:0/2514877256 conn(0x7fdca80ac570 msgr2=0x7fdca814cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 shutdown_connections 2024-03-06T07:31:46.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:46.732+0000 7fdcaf3b5640 1 -- 172.21.15.26:0/2514877256 wait complete. 2024-03-06T07:31:47.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:47.147 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:48.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.248+0000 7f3bda313640 1 Processor -- start 2024-03-06T07:31:48.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.248+0000 7f3bda313640 1 -- start start 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.248+0000 7f3bda313640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd414f360 0x7f3bd4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.248+0000 7f3bda313640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3bd40c88c0 0x7f3bd40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.248+0000 7f3bda313640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd40c73d0 con 0x7f3bd40c5880 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.248+0000 7f3bda313640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd40c8dd0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.248+0000 7f3bda313640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40be2e0 con 0x7f3bd414f360 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3bd40c88c0 0x7f3bd40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd414f360 0x7f3bd4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd88b0640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3bd40c5880 0x7f3bd4158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44926/0 (socket says 172.21.15.26:44926) 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3bd40c88c0 0x7f3bd40c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39236/0 (socket says 172.21.15.26:39236) 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd414f360 0x7f3bd4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:41348/0 (socket says 172.21.15.26:41348) 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd88b0640 1 -- 172.21.15.26:0/1292380979 learned_addr learned my addr 172.21.15.26:0/1292380979 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd37fe640 1 -- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd414f360 msgr2=0x7f3bd4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd37fe640 1 --2- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd414f360 0x7f3bd4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd37fe640 1 -- 172.21.15.26:0/1292380979 >> v1:172.21.15.117:6789/0 conn(0x7f3bd40c5880 legacy=0x7f3bd4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd37fe640 1 -- 172.21.15.26:0/1292380979 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd40c7670 con 0x7f3bd40c88c0 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd2ffd640 1 -- 172.21.15.26:0/1292380979 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244291326 0 0) 0x7f3bd40c73d0 con 0x7f3bd40c5880 2024-03-06T07:31:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.249+0000 7f3bd3fff640 1 --2- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd414f360 0x7f3bd4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.250+0000 7f3bd37fe640 1 --2- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c5340 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80097a0 tx=0x7f3bc802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e3374b96f9d0787 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.250+0000 7f3bd2ffd640 1 -- 172.21.15.26:0/1292380979 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80315a0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.250+0000 7f3bd2ffd640 1 -- 172.21.15.26:0/1292380979 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8037070 con 0x7f3bd40c88c0 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.250+0000 7f3bd2ffd640 1 -- 172.21.15.26:0/1292380979 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8031960 con 0x7f3bd40c88c0 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 -- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd40c88c0 msgr2=0x7f3bd40c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 --2- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c5340 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80097a0 tx=0x7f3bc802b840 comp rx=0 tx=0).stop 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 -- 172.21.15.26:0/1292380979 shutdown_connections 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 --2- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c5340 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 --2- 172.21.15.26:0/1292380979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd414f360 0x7f3bd4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 -- 172.21.15.26:0/1292380979 >> 172.21.15.26:0/1292380979 conn(0x7f3bd40ac570 msgr2=0x7f3bd40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:48.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 -- 172.21.15.26:0/1292380979 shutdown_connections 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.251+0000 7f3bda313640 1 -- 172.21.15.26:0/1292380979 wait complete. 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.252+0000 7f3bda313640 1 Processor -- start 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.252+0000 7f3bda313640 1 -- start start 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bda313640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd40c5880 0x7f3bd40addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bda313640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bda313640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd414f360 0x7f3bd4152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bda313640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd4159270 con 0x7f3bd40c88c0 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bda313640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40c7c30 con 0x7f3bd414f360 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bda313640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41593f0 con 0x7f3bd40c5880 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bd37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bd37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52858/0 (socket says 172.21.15.26:52858) 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bd37fe640 1 -- 172.21.15.26:0/3703054074 learned_addr learned my addr 172.21.15.26:0/3703054074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bd88b0640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd414f360 0x7f3bd4152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bd3fff640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd40c5880 0x7f3bd40addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:48.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bd37fe640 1 -- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd40c5880 msgr2=0x7f3bd40addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.253+0000 7f3bd37fe640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd40c5880 0x7f3bd40addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bd37fe640 1 -- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd414f360 msgr2=0x7f3bd4152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bd37fe640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd414f360 0x7f3bd4152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bd37fe640 1 -- 172.21.15.26:0/3703054074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd414f9c0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bd37fe640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c2d40 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80097a0 tx=0x7f3bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bd0ff9640 1 -- 172.21.15.26:0/3703054074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8004230 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bd0ff9640 1 -- 172.21.15.26:0/3703054074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc80043f0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd40c46b0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bd0ff9640 1 -- 172.21.15.26:0/3703054074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8003850 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.254+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd40c4c70 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.255+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd404f330 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.255+0000 7f3bd0ff9640 1 -- 172.21.15.26:0/3703054074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bc80039f0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.256+0000 7f3bd0ff9640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3bc006b4d0 0x7f3bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:48.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.256+0000 7f3bd0ff9640 1 -- 172.21.15.26:0/3703054074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f3bc802f030 con 0x7f3bd40c88c0 2024-03-06T07:31:48.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.256+0000 7f3bd3fff640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3bc006b4d0 0x7f3bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:48.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.257+0000 7f3bd3fff640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3bc006b4d0 0x7f3bc006d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009560 tx=0x7f3bc4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:48.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.260+0000 7f3bd0ff9640 1 -- 172.21.15.26:0/3703054074 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3bc8041070 con 0x7f3bd40c88c0 2024-03-06T07:31:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.477+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bd40cb7b0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.478+0000 7f3bd0ff9640 1 -- 172.21.15.26:0/3703054074 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f3bc807abc0 con 0x7f3bd40c88c0 2024-03-06T07:31:48.479 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:48.480 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:48.480 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3bc006b4d0 msgr2=0x7f3bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3bc006b4d0 0x7f3bc006d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009560 tx=0x7f3bc4009190 comp rx=0 tx=0).stop 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bd40c88c0 msgr2=0x7f3bd40c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c2d40 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80097a0 tx=0x7f3bc80046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 shutdown_connections 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3bc006b4d0 0x7f3bc006d990 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bd414f360 0x7f3bd4152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bd40c88c0 0x7f3bd40c2d40 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 --2- 172.21.15.26:0/3703054074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd40c5880 0x7f3bd40addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 >> 172.21.15.26:0/3703054074 conn(0x7f3bd40ac570 msgr2=0x7f3bd40c9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 shutdown_connections 2024-03-06T07:31:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:48.480+0000 7f3bda313640 1 -- 172.21.15.26:0/3703054074 wait complete. 2024-03-06T07:31:48.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:48.894 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:49.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.996+0000 7fe970c8d640 1 Processor -- start 2024-03-06T07:31:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.996+0000 7fe970c8d640 1 -- start start 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe970c8d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe96c159270 0x7fe96c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe970c8d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0c73d0 con 0x7fe96c0c88c0 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe970c8d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0be2e0 con 0x7fe96c14f360 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe970c8d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c159700 con 0x7fe96c159270 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe969d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe96c0c88c0 0x7fe96c158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39856/0 (socket says 172.21.15.26:39856) 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe969d9b640 1 -- 172.21.15.26:0/3800647659 learned_addr learned my addr 172.21.15.26:0/3800647659 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371318003 0 0) 0x7fe96c0c73d0 con 0x7fe96c0c88c0 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96ad9d640 1 --2- 172.21.15.26:0/3800647659 >> v2:172.21.15.117:3300/0 conn(0x7fe96c159270 0x7fe96c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe954003520 con 0x7fe96c0c88c0 2024-03-06T07:31:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496549419 0 0) 0x7fe954003520 con 0x7fe96c0c88c0 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 >> v2:172.21.15.117:3300/0 conn(0x7fe96c159270 msgr2=0x7fe96c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 --2- 172.21.15.26:0/3800647659 >> v2:172.21.15.117:3300/0 conn(0x7fe96c159270 0x7fe96c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 >> v1:172.21.15.117:6789/0 conn(0x7fe96c14f360 legacy=0x7fe96c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0c7670 con 0x7fe96c0c88c0 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fe960001670 con 0x7fe96c0c88c0 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 >> v1:172.21.15.26:6789/0 conn(0x7fe96c0c88c0 legacy=0x7fe96c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9540039b0 0x7fe954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe954004800 0x7fe954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9540051c0 0x7fe95400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c159700 con 0x7fe9540039b0 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95400c160 con 0x7fe954004800 2024-03-06T07:31:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95400c360 con 0x7fe9540051c0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.998+0000 7fe96a59c640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9540039b0 0x7fe954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe969d9b640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe954004800 0x7fe954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe96ad9d640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9540051c0 0x7fe95400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe96a59c640 1 -- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9540051c0 msgr2=0x7fe95400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe96a59c640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9540051c0 0x7fe95400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe96a59c640 1 -- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe954004800 msgr2=0x7fe954004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe96a59c640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe954004800 0x7fe954004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe96a59c640 1 -- 172.21.15.26:0/3800647659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe954003520 con 0x7fe9540039b0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:49.999+0000 7fe96ad9d640 1 -- 172.21.15.26:0/3800647659 reap_dead start 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe96a59c640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9540039b0 0x7fe954003dd0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fe95c001750 tx=0x7fe954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84d27af019f73f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c001370 con 0x7fe9540039b0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c0052d0 con 0x7fe9540039b0 2024-03-06T07:31:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe96959a640 1 -- 172.21.15.26:0/3800647659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c003830 con 0x7fe9540039b0 2024-03-06T07:31:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe970c8d640 1 -- 172.21.15.26:0/3800647659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9540039b0 msgr2=0x7fe954003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9540039b0 0x7fe954003dd0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fe95c001750 tx=0x7fe954005830 comp rx=0 tx=0).stop 2024-03-06T07:31:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe970c8d640 1 -- 172.21.15.26:0/3800647659 shutdown_connections 2024-03-06T07:31:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3800647659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9540039b0 0x7fe954003dd0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe970c8d640 1 -- 172.21.15.26:0/3800647659 >> 172.21.15.26:0/3800647659 conn(0x7fe96c0ac570 msgr2=0x7fe96c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe970c8d640 1 -- 172.21.15.26:0/3800647659 shutdown_connections 2024-03-06T07:31:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.000+0000 7fe970c8d640 1 -- 172.21.15.26:0/3800647659 wait complete. 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 Processor -- start 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 -- start start 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9540039b0 0x7fe96c157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96c1eb540 0x7fe96c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96c145af0 0x7fe96c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0c7c30 con 0x7fe96c145af0 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c155c10 con 0x7fe9540039b0 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe970c8d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c155d90 con 0x7fe96c1eb540 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe96ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96c145af0 0x7fe96c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe96ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96c145af0 0x7fe96c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52874/0 (socket says 172.21.15.26:52874) 2024-03-06T07:31:50.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.002+0000 7fe96ad9d640 1 -- 172.21.15.26:0/3568425934 learned_addr learned my addr 172.21.15.26:0/3568425934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe969d9b640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96c1eb540 0x7fe96c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe96a59c640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9540039b0 0x7fe96c157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe96ad9d640 1 -- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96c1eb540 msgr2=0x7fe96c153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe96ad9d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96c1eb540 0x7fe96c153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe96ad9d640 1 -- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9540039b0 msgr2=0x7fe96c157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe96ad9d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9540039b0 0x7fe96c157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe96ad9d640 1 -- 172.21.15.26:0/3568425934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c153a80 con 0x7fe96c145af0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe969d9b640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96c1eb540 0x7fe96c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe96ad9d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96c145af0 0x7fe96c153540 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fe96c067c30 tx=0x7fe96400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe95affd640 1 -- 172.21.15.26:0/3568425934 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400fd80 con 0x7fe96c145af0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe95affd640 1 -- 172.21.15.26:0/3568425934 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96401d070 con 0x7fe96c145af0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c155360 con 0x7fe96c145af0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe95affd640 1 -- 172.21.15.26:0/3568425934 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964015670 con 0x7fe96c145af0 2024-03-06T07:31:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.003+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c1526a0 con 0x7fe96c145af0 2024-03-06T07:31:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.004+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c04f330 con 0x7fe96c145af0 2024-03-06T07:31:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.005+0000 7fe95affd640 1 -- 172.21.15.26:0/3568425934 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe964015810 con 0x7fe96c145af0 2024-03-06T07:31:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.005+0000 7fe95affd640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94c06b4d0 0x7fe94c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.005+0000 7fe95affd640 1 -- 172.21.15.26:0/3568425934 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe96408acf0 con 0x7fe96c145af0 2024-03-06T07:31:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.005+0000 7fe96a59c640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94c06b4d0 0x7fe94c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.006+0000 7fe96a59c640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94c06b4d0 0x7fe94c06d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe95c033000 tx=0x7fe95c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:50.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.009+0000 7fe95affd640 1 -- 172.21.15.26:0/3568425934 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe96408f070 con 0x7fe96c145af0 2024-03-06T07:31:50.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.355+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe96c0c8d50 con 0x7fe96c145af0 2024-03-06T07:31:50.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.355+0000 7fe95affd640 1 -- 172.21.15.26:0/3568425934 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fe964010040 con 0x7fe96c145af0 2024-03-06T07:31:50.357 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:50.357 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:50.357 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:50.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94c06b4d0 msgr2=0x7fe94c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94c06b4d0 0x7fe94c06d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe95c033000 tx=0x7fe95c009030 comp rx=0 tx=0).stop 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96c145af0 msgr2=0x7fe96c153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96c145af0 0x7fe96c153540 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fe96c067c30 tx=0x7fe96400f580 comp rx=0 tx=0).stop 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 shutdown_connections 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94c06b4d0 0x7fe94c06d990 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96c145af0 0x7fe96c153540 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe96c1eb540 0x7fe96c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 --2- 172.21.15.26:0/3568425934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9540039b0 0x7fe96c157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 >> 172.21.15.26:0/3568425934 conn(0x7fe96c0ac570 msgr2=0x7fe96c0ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 shutdown_connections 2024-03-06T07:31:50.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:50.358+0000 7fe970c8d640 1 -- 172.21.15.26:0/3568425934 wait complete. 2024-03-06T07:31:50.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:50.772 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:51.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.873+0000 7f37262ed640 1 Processor -- start 2024-03-06T07:31:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.873+0000 7f37262ed640 1 -- start start 2024-03-06T07:31:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f37262ed640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f372014f360 0x7f3720155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f37262ed640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3720158b60 0x7f3720158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f37262ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37200c73d0 con 0x7f37200c88c0 2024-03-06T07:31:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f37262ed640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37200c8dd0 con 0x7f372014f360 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f37262ed640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37200be2e0 con 0x7f3720158b60 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f371ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f372014f360 0x7f3720155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f371f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f37200c88c0 0x7f37200c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39868/0 (socket says 172.21.15.26:39868) 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.874+0000 7f371f7fe640 1 -- 172.21.15.26:0/600618519 learned_addr learned my addr 172.21.15.26:0/600618519 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.875+0000 7f371ffff640 1 -- 172.21.15.26:0/600618519 >> v2:172.21.15.117:3300/0 conn(0x7f3720158b60 msgr2=0x7f3720158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.875+0000 7f371ffff640 1 --2- 172.21.15.26:0/600618519 >> v2:172.21.15.117:3300/0 conn(0x7f3720158b60 0x7f3720158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.875+0000 7f371ffff640 1 -- 172.21.15.26:0/600618519 >> v1:172.21.15.26:6789/0 conn(0x7f37200c88c0 legacy=0x7f37200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.875+0000 7f371ffff640 1 -- 172.21.15.26:0/600618519 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200c7670 con 0x7f372014f360 2024-03-06T07:31:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.875+0000 7f371ffff640 1 --2- 172.21.15.26:0/600618519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372014f360 0x7f3720155c10 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f3720152d60 tx=0x7f371002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5affd4a33b1308 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.875+0000 7f371effd640 1 -- 172.21.15.26:0/600618519 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100315a0 con 0x7f372014f360 2024-03-06T07:31:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.875+0000 7f371effd640 1 -- 172.21.15.26:0/600618519 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3710037070 con 0x7f372014f360 2024-03-06T07:31:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f371effd640 1 -- 172.21.15.26:0/600618519 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710031960 con 0x7f372014f360 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 -- 172.21.15.26:0/600618519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372014f360 msgr2=0x7f3720155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 --2- 172.21.15.26:0/600618519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372014f360 0x7f3720155c10 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f3720152d60 tx=0x7f371002b840 comp rx=0 tx=0).stop 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 -- 172.21.15.26:0/600618519 shutdown_connections 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 --2- 172.21.15.26:0/600618519 >> v2:172.21.15.117:3300/0 conn(0x7f3720158b60 0x7f3720158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 --2- 172.21.15.26:0/600618519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372014f360 0x7f3720155c10 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 -- 172.21.15.26:0/600618519 >> 172.21.15.26:0/600618519 conn(0x7f37200ac570 msgr2=0x7f37200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 -- 172.21.15.26:0/600618519 shutdown_connections 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.876+0000 7f37262ed640 1 -- 172.21.15.26:0/600618519 wait complete. 2024-03-06T07:31:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.878+0000 7f37262ed640 1 Processor -- start 2024-03-06T07:31:51.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.878+0000 7f37262ed640 1 -- start start 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f37262ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37200c88c0 0x7f37200c1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f37262ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372014f360 0x7f37200c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f37262ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720158b60 0x7f372014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f37262ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200c5a50 con 0x7f3720158b60 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f37262ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200c7c30 con 0x7f372014f360 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f37262ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37200c5bd0 con 0x7f37200c88c0 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f372488a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720158b60 0x7f372014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f372488a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720158b60 0x7f372014f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52886/0 (socket says 172.21.15.26:52886) 2024-03-06T07:31:51.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f372488a640 1 -- 172.21.15.26:0/68003844 learned_addr learned my addr 172.21.15.26:0/68003844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f371f7fe640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372014f360 0x7f37200c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.879+0000 7f371ffff640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37200c88c0 0x7f37200c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f372488a640 1 -- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37200c88c0 msgr2=0x7f37200c1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f372488a640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37200c88c0 0x7f37200c1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f372488a640 1 -- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372014f360 msgr2=0x7f37200c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f372488a640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372014f360 0x7f37200c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f372488a640 1 -- 172.21.15.26:0/68003844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37201eac60 con 0x7f3720158b60 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f371ffff640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37200c88c0 0x7f37200c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f371f7fe640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372014f360 0x7f37200c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f372488a640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720158b60 0x7f372014f840 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f371800b970 tx=0x7f371800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f371cff9640 1 -- 172.21.15.26:0/68003844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718019070 con 0x7f3720158b60 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f371cff9640 1 -- 172.21.15.26:0/68003844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371800cb60 con 0x7f3720158b60 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37201eaf40 con 0x7f3720158b60 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f371cff9640 1 -- 172.21.15.26:0/68003844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718014440 con 0x7f3720158b60 2024-03-06T07:31:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.880+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37201eb500 con 0x7f3720158b60 2024-03-06T07:31:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.881+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372004f330 con 0x7f3720158b60 2024-03-06T07:31:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.882+0000 7f371cff9640 1 -- 172.21.15.26:0/68003844 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f371801c070 con 0x7f3720158b60 2024-03-06T07:31:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.882+0000 7f371cff9640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f370c06b4d0 0x7f370c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.882+0000 7f371cff9640 1 -- 172.21.15.26:0/68003844 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f371808a130 con 0x7f3720158b60 2024-03-06T07:31:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.882+0000 7f371ffff640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f370c06b4d0 0x7f370c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.883+0000 7f371ffff640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f370c06b4d0 0x7f370c06d990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3710036000 tx=0x7f371003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:51.890+0000 7f371cff9640 1 -- 172.21.15.26:0/68003844 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3718058db0 con 0x7f3720158b60 2024-03-06T07:31:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.111+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37200c54e0 con 0x7f3720158b60 2024-03-06T07:31:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.112+0000 7f371cff9640 1 -- 172.21.15.26:0/68003844 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f371800ee20 con 0x7f3720158b60 2024-03-06T07:31:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:52.113 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f370c06b4d0 msgr2=0x7f370c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f370c06b4d0 0x7f370c06d990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3710036000 tx=0x7f371003d040 comp rx=0 tx=0).stop 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720158b60 msgr2=0x7f372014f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720158b60 0x7f372014f840 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f371800b970 tx=0x7f371800be30 comp rx=0 tx=0).stop 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 shutdown_connections 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f370c06b4d0 0x7f370c06d990 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720158b60 0x7f372014f840 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372014f360 0x7f37200c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 --2- 172.21.15.26:0/68003844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37200c88c0 0x7f37200c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 >> 172.21.15.26:0/68003844 conn(0x7f37200ac570 msgr2=0x7f372014de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 shutdown_connections 2024-03-06T07:31:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:52.114+0000 7f37262ed640 1 -- 172.21.15.26:0/68003844 wait complete. 2024-03-06T07:31:52.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:52.530 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:53.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:53.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.667+0000 7fc2790c2640 1 Processor -- start 2024-03-06T07:31:53.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.667+0000 7fc2790c2640 1 -- start start 2024-03-06T07:31:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc2790c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27406d9f0 0x7fc2740a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc2790c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2740c3280 0x7fc2740c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc2790c2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc274158800 con 0x7fc2741551e0 2024-03-06T07:31:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc2790c2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc274158ae0 con 0x7fc2740c3280 2024-03-06T07:31:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc2790c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc274158dc0 con 0x7fc27406d9f0 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc27259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2740c3280 0x7fc2740c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc27259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2740c3280 0x7fc2740c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52890/0 (socket says 172.21.15.26:52890) 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc27259c640 1 -- 172.21.15.26:0/1869591845 learned_addr learned my addr 172.21.15.26:0/1869591845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc272d9d640 1 --2- 172.21.15.26:0/1869591845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27406d9f0 0x7fc2740a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc27259c640 1 -- 172.21.15.26:0/1869591845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27406d9f0 msgr2=0x7fc2740a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc27259c640 1 --2- 172.21.15.26:0/1869591845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27406d9f0 0x7fc2740a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc27259c640 1 -- 172.21.15.26:0/1869591845 >> v1:172.21.15.117:6789/0 conn(0x7fc2741551e0 legacy=0x7fc274155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.668+0000 7fc27259c640 1 -- 172.21.15.26:0/1869591845 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2741590a0 con 0x7fc2740c3280 2024-03-06T07:31:53.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc27259c640 1 --2- 172.21.15.26:0/1869591845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2740c36e0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009870 tx=0x7fc25c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3539f17b09527873 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:53.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc271d9b640 1 -- 172.21.15.26:0/1869591845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0317c0 con 0x7fc2740c3280 2024-03-06T07:31:53.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc271d9b640 1 -- 172.21.15.26:0/1869591845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c037070 con 0x7fc2740c3280 2024-03-06T07:31:53.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc271d9b640 1 -- 172.21.15.26:0/1869591845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c031b80 con 0x7fc2740c3280 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc2790c2640 1 -- 172.21.15.26:0/1869591845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 msgr2=0x7fc2740c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1869591845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2740c36e0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009870 tx=0x7fc25c02b860 comp rx=0 tx=0).stop 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc2790c2640 1 -- 172.21.15.26:0/1869591845 shutdown_connections 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1869591845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2740c36e0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1869591845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27406d9f0 0x7fc2740a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.670+0000 7fc2790c2640 1 -- 172.21.15.26:0/1869591845 >> 172.21.15.26:0/1869591845 conn(0x7fc2740aa630 msgr2=0x7fc2740aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.671+0000 7fc2790c2640 1 -- 172.21.15.26:0/1869591845 shutdown_connections 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.671+0000 7fc2790c2640 1 -- 172.21.15.26:0/1869591845 wait complete. 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.672+0000 7fc2790c2640 1 Processor -- start 2024-03-06T07:31:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.672+0000 7fc2790c2640 1 -- start start 2024-03-06T07:31:53.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc2790c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27406d9f0 0x7fc2741ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:53.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc2790c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2741565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:53.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc2790c2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2741551e0 0x7fc274156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:53.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc2790c2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27406df50 con 0x7fc2740c3280 2024-03-06T07:31:53.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc2790c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27406ddd0 con 0x7fc27406d9f0 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc2790c2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740a8210 con 0x7fc2741551e0 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2741565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2741565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52898/0 (socket says 172.21.15.26:52898) 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 -- 172.21.15.26:0/1939652288 learned_addr learned my addr 172.21.15.26:0/1939652288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc272d9d640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27406d9f0 0x7fc2741ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27359e640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2741551e0 0x7fc274156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 -- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2741551e0 msgr2=0x7fc274156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2741551e0 0x7fc274156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 -- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27406d9f0 msgr2=0x7fc2741ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27406d9f0 0x7fc2741ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 -- 172.21.15.26:0/1939652288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2741df030 con 0x7fc2740c3280 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.673+0000 7fc27259c640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2741565e0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009870 tx=0x7fc25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.674+0000 7fc26b7fe640 1 -- 172.21.15.26:0/1939652288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c032860 con 0x7fc2740c3280 2024-03-06T07:31:53.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.674+0000 7fc26b7fe640 1 -- 172.21.15.26:0/1939652288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c032a00 con 0x7fc2740c3280 2024-03-06T07:31:53.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.674+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2741df760 con 0x7fc2740c3280 2024-03-06T07:31:53.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.674+0000 7fc26b7fe640 1 -- 172.21.15.26:0/1939652288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c032df0 con 0x7fc2740c3280 2024-03-06T07:31:53.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.674+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2741dea70 con 0x7fc2740c3280 2024-03-06T07:31:53.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.675+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2741de860 con 0x7fc2740c3280 2024-03-06T07:31:53.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.675+0000 7fc26b7fe640 1 -- 172.21.15.26:0/1939652288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc25c04c060 con 0x7fc2740c3280 2024-03-06T07:31:53.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.676+0000 7fc26b7fe640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc25406b4d0 0x7fc25406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:53.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.676+0000 7fc26b7fe640 1 -- 172.21.15.26:0/1939652288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc25c0abf60 con 0x7fc2740c3280 2024-03-06T07:31:53.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.676+0000 7fc272d9d640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc25406b4d0 0x7fc25406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:53.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.677+0000 7fc272d9d640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc25406b4d0 0x7fc25406d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc2740aa260 tx=0x7fc264007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.680+0000 7fc26b7fe640 1 -- 172.21.15.26:0/1939652288 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc25c040070 con 0x7fc2740c3280 2024-03-06T07:31:53.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.897+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc274157060 con 0x7fc2740c3280 2024-03-06T07:31:53.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.898+0000 7fc26b7fe640 1 -- 172.21.15.26:0/1939652288 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc25c0422a0 con 0x7fc2740c3280 2024-03-06T07:31:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:53.900 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc25406b4d0 msgr2=0x7fc25406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc25406b4d0 0x7fc25406d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc2740aa260 tx=0x7fc264007900 comp rx=0 tx=0).stop 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 msgr2=0x7fc2741565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2741565e0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009870 tx=0x7fc25c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 shutdown_connections 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc25406b4d0 0x7fc25406d990 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2741551e0 0x7fc274156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2740c3280 0x7fc2741565e0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 --2- 172.21.15.26:0/1939652288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27406d9f0 0x7fc2741ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 >> 172.21.15.26:0/1939652288 conn(0x7fc2740aa630 msgr2=0x7fc27406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 shutdown_connections 2024-03-06T07:31:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:53.900+0000 7fc2790c2640 1 -- 172.21.15.26:0/1939652288 wait complete. 2024-03-06T07:31:54.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:54.318 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:55.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:55.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.419+0000 7f7b226e1640 1 Processor -- start 2024-03-06T07:31:55.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.419+0000 7f7b226e1640 1 -- start start 2024-03-06T07:31:55.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b226e1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7b1c14f360 0x7f7b1c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:55.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b226e1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c158b60 0x7f7b1c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:55.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b226e1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0c73d0 con 0x7f7b1c0c88c0 2024-03-06T07:31:55.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b226e1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c0c8dd0 con 0x7f7b1c14f360 2024-03-06T07:31:55.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b226e1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0be2e0 con 0x7f7b1c158b60 2024-03-06T07:31:55.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b20c7e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c158b60 0x7f7b1c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:55.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1bfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7b1c14f360 0x7f7b1c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:55.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1b7fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f7b1c0c88c0 0x7f7b1c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:44964/0 (socket says 172.21.15.26:44964) 2024-03-06T07:31:55.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1bfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7b1c14f360 0x7f7b1c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39328/0 (socket says 172.21.15.26:39328) 2024-03-06T07:31:55.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/2340581489 learned_addr learned my addr 172.21.15.26:0/2340581489 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:55.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1bfff640 1 -- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c158b60 msgr2=0x7f7b1c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1bfff640 1 --2- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c158b60 0x7f7b1c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1bfff640 1 -- 172.21.15.26:0/2340581489 >> v1:172.21.15.117:6789/0 conn(0x7f7b1c0c88c0 legacy=0x7f7b1c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b1bfff640 1 -- 172.21.15.26:0/2340581489 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0c7670 con 0x7f7b1c14f360 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.420+0000 7f7b20c7e640 1 --2- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c158b60 0x7f7b1c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.421+0000 7f7b1bfff640 1 --2- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c14f360 0x7f7b1c155c10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0097a0 tx=0x7f7b0c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2981a53157028109 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.421+0000 7f7b1affd640 1 -- 172.21.15.26:0/2340581489 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0315a0 con 0x7f7b1c14f360 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.421+0000 7f7b1affd640 1 -- 172.21.15.26:0/2340581489 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c037070 con 0x7f7b1c14f360 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.421+0000 7f7b1affd640 1 -- 172.21.15.26:0/2340581489 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c031960 con 0x7f7b1c14f360 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 -- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c14f360 msgr2=0x7f7b1c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 --2- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c14f360 0x7f7b1c155c10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0097a0 tx=0x7f7b0c02b840 comp rx=0 tx=0).stop 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 -- 172.21.15.26:0/2340581489 shutdown_connections 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 --2- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c158b60 0x7f7b1c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 --2- 172.21.15.26:0/2340581489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c14f360 0x7f7b1c155c10 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 -- 172.21.15.26:0/2340581489 >> 172.21.15.26:0/2340581489 conn(0x7f7b1c0ac570 msgr2=0x7f7b1c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 -- 172.21.15.26:0/2340581489 shutdown_connections 2024-03-06T07:31:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.422+0000 7f7b226e1640 1 -- 172.21.15.26:0/2340581489 wait complete. 2024-03-06T07:31:55.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.423+0000 7f7b226e1640 1 Processor -- start 2024-03-06T07:31:55.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.423+0000 7f7b226e1640 1 -- start start 2024-03-06T07:31:55.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b226e1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c0c88c0 0x7f7b1c14bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:55.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b226e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b1c14f360 0x7f7b1c0af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:55.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b226e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c158b60 0x7f7b1c0bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:55.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b226e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0c5a50 con 0x7f7b1c14f360 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b226e1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0c7c30 con 0x7f7b1c158b60 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b226e1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0c5bd0 con 0x7f7b1c0c88c0 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b1c14f360 0x7f7b1c0af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b1c14f360 0x7f7b1c0af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52910/0 (socket says 172.21.15.26:52910) 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/1258304493 learned_addr learned my addr 172.21.15.26:0/1258304493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b20c7e640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c158b60 0x7f7b1c0bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1bfff640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c0c88c0 0x7f7b1c14bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c0c88c0 msgr2=0x7f7b1c14bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c0c88c0 0x7f7b1c14bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c158b60 msgr2=0x7f7b1c0bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c158b60 0x7f7b1c0bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 -- 172.21.15.26:0/1258304493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c14bbf0 con 0x7f7b1c14f360 2024-03-06T07:31:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.424+0000 7f7b1b7fe640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b1c14f360 0x7f7b1c0af8f0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000c440 tx=0x7f7b1000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.425+0000 7f7b18ff9640 1 -- 172.21.15.26:0/1258304493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10017070 con 0x7f7b1c14f360 2024-03-06T07:31:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.425+0000 7f7b18ff9640 1 -- 172.21.15.26:0/1258304493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b10005ce0 con 0x7f7b1c14f360 2024-03-06T07:31:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.425+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c1eb880 con 0x7f7b1c14f360 2024-03-06T07:31:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.425+0000 7f7b18ff9640 1 -- 172.21.15.26:0/1258304493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b100072f0 con 0x7f7b1c14f360 2024-03-06T07:31:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.425+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c0ad430 con 0x7f7b1c14f360 2024-03-06T07:31:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.426+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c0ad630 con 0x7f7b1c14f360 2024-03-06T07:31:55.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.426+0000 7f7b18ff9640 1 -- 172.21.15.26:0/1258304493 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b1001a070 con 0x7f7b1c14f360 2024-03-06T07:31:55.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.427+0000 7f7b18ff9640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0806b4d0 0x7f7b0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:55.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.427+0000 7f7b18ff9640 1 -- 172.21.15.26:0/1258304493 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f7b10089680 con 0x7f7b1c14f360 2024-03-06T07:31:55.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.427+0000 7f7b1bfff640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0806b4d0 0x7f7b0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:55.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.428+0000 7f7b1bfff640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0806b4d0 0x7f7b0806d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c036000 tx=0x7f7b0c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:55.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.431+0000 7f7b18ff9640 1 -- 172.21.15.26:0/1258304493 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b10057c10 con 0x7f7b1c14f360 2024-03-06T07:31:55.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.650+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b1c04f330 con 0x7f7b1c14f360 2024-03-06T07:31:55.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.651+0000 7f7b18ff9640 1 -- 172.21.15.26:0/1258304493 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f7b1005bcb0 con 0x7f7b1c14f360 2024-03-06T07:31:55.652 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:55.652 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0806b4d0 msgr2=0x7f7b0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0806b4d0 0x7f7b0806d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c036000 tx=0x7f7b0c03d040 comp rx=0 tx=0).stop 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b1c14f360 msgr2=0x7f7b1c0af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b1c14f360 0x7f7b1c0af8f0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000c440 tx=0x7f7b1000c840 comp rx=0 tx=0).stop 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 shutdown_connections 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7b0806b4d0 0x7f7b0806d990 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b1c158b60 0x7f7b1c0bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b1c14f360 0x7f7b1c0af8f0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 --2- 172.21.15.26:0/1258304493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b1c0c88c0 0x7f7b1c14bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 >> 172.21.15.26:0/1258304493 conn(0x7f7b1c0ac570 msgr2=0x7f7b1c0ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 shutdown_connections 2024-03-06T07:31:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:55.653+0000 7f7b226e1640 1 -- 172.21.15.26:0/1258304493 wait complete. 2024-03-06T07:31:56.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:56.068 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:57.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:57.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.169+0000 7f24a4755640 1 Processor -- start 2024-03-06T07:31:57.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.169+0000 7f24a4755640 1 -- start start 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a4755640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f249c14f360 0x7f249c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a4755640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c88c0 0x7f249c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a4755640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f249c0c5880 0x7f249c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a4755640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c0c8dd0 con 0x7f249c14f360 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a4755640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c0c73d0 con 0x7f249c0c5880 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a4755640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0be2e0 con 0x7f249c0c88c0 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f249c14f360 0x7f249c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f249c14f360 0x7f249c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45474/0 (socket says 172.21.15.26:45474) 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 -- 172.21.15.26:0/3391603339 learned_addr learned my addr 172.21.15.26:0/3391603339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 -- 172.21.15.26:0/3391603339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c88c0 msgr2=0x7f249c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a2cf2640 1 --2- 172.21.15.26:0/3391603339 >> v2:172.21.15.117:3300/0 conn(0x7f249c0c5880 0x7f249c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a1cf0640 1 --2- 172.21.15.26:0/3391603339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c88c0 0x7f249c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 --2- 172.21.15.26:0/3391603339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c88c0 0x7f249c0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 -- 172.21.15.26:0/3391603339 >> v2:172.21.15.117:3300/0 conn(0x7f249c0c5880 msgr2=0x7f249c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 --2- 172.21.15.26:0/3391603339 >> v2:172.21.15.117:3300/0 conn(0x7f249c0c5880 0x7f249c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a24f1640 1 -- 172.21.15.26:0/3391603339 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c0c7670 con 0x7f249c14f360 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a2cf2640 1 --2- 172.21.15.26:0/3391603339 >> v2:172.21.15.117:3300/0 conn(0x7f249c0c5880 0x7f249c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.170+0000 7f24a1cf0640 1 --2- 172.21.15.26:0/3391603339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c88c0 0x7f249c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.171+0000 7f24a24f1640 1 --2- 172.21.15.26:0/3391603339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c14f360 0x7f249c155c10 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f249c08cc10 tx=0x7f249802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf49b0c4a0cf8f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.171+0000 7f24a14ef640 1 -- 172.21.15.26:0/3391603339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249802e070 con 0x7f249c14f360 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.171+0000 7f24a14ef640 1 -- 172.21.15.26:0/3391603339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498037070 con 0x7f249c14f360 2024-03-06T07:31:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.171+0000 7f24a14ef640 1 -- 172.21.15.26:0/3391603339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498031a00 con 0x7f249c14f360 2024-03-06T07:31:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 -- 172.21.15.26:0/3391603339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c14f360 msgr2=0x7f249c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 --2- 172.21.15.26:0/3391603339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c14f360 0x7f249c155c10 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f249c08cc10 tx=0x7f249802b880 comp rx=0 tx=0).stop 2024-03-06T07:31:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 -- 172.21.15.26:0/3391603339 shutdown_connections 2024-03-06T07:31:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 --2- 172.21.15.26:0/3391603339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c88c0 0x7f249c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 --2- 172.21.15.26:0/3391603339 >> v2:172.21.15.117:3300/0 conn(0x7f249c0c5880 0x7f249c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 --2- 172.21.15.26:0/3391603339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c14f360 0x7f249c155c10 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 -- 172.21.15.26:0/3391603339 >> 172.21.15.26:0/3391603339 conn(0x7f249c0ac570 msgr2=0x7f249c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 -- 172.21.15.26:0/3391603339 shutdown_connections 2024-03-06T07:31:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.172+0000 7f24a4755640 1 -- 172.21.15.26:0/3391603339 wait complete. 2024-03-06T07:31:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.174+0000 7f24a4755640 1 Processor -- start 2024-03-06T07:31:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.174+0000 7f24a4755640 1 -- start start 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a4755640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c5880 0x7f249c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a4755640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0c88c0 0x7f249c14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a4755640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249c14f360 0x7f249c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a4755640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c1590a0 con 0x7f249c0c88c0 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a4755640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0c7c30 con 0x7f249c14f360 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a4755640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c159220 con 0x7f249c0c5880 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0c88c0 0x7f249c14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0c88c0 0x7f249c14bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45480/0 (socket says 172.21.15.26:45480) 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 -- 172.21.15.26:0/3804872219 learned_addr learned my addr 172.21.15.26:0/3804872219 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a2cf2640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249c14f360 0x7f249c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a24f1640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c5880 0x7f249c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 -- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c5880 msgr2=0x7f249c157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c5880 0x7f249c157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 -- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249c14f360 msgr2=0x7f249c14c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249c14f360 0x7f249c14c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 -- 172.21.15.26:0/3804872219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c1eb3d0 con 0x7f249c0c88c0 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a2cf2640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249c14f360 0x7f249c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.175+0000 7f24a1cf0640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0c88c0 0x7f249c14bce0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c820 tx=0x7f248c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.176+0000 7f2492ffd640 1 -- 172.21.15.26:0/3804872219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c015070 con 0x7f249c0c88c0 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.176+0000 7f2492ffd640 1 -- 172.21.15.26:0/3804872219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c007a30 con 0x7f249c0c88c0 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.176+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c150910 con 0x7f249c0c88c0 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.176+0000 7f2492ffd640 1 -- 172.21.15.26:0/3804872219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c007e00 con 0x7f249c0c88c0 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.176+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c158200 con 0x7f249c0c88c0 2024-03-06T07:31:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.176+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c075300 con 0x7f249c0c88c0 2024-03-06T07:31:57.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.177+0000 7f2492ffd640 1 -- 172.21.15.26:0/3804872219 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f248c018070 con 0x7f249c0c88c0 2024-03-06T07:31:57.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.177+0000 7f2492ffd640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f248006b500 0x7f248006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.178+0000 7f2492ffd640 1 -- 172.21.15.26:0/3804872219 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f248c08a3b0 con 0x7f249c0c88c0 2024-03-06T07:31:57.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.178+0000 7f24a24f1640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f248006b500 0x7f248006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.178+0000 7f24a24f1640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f248006b500 0x7f248006d9c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2498036000 tx=0x7f249803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:57.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.182+0000 7f2492ffd640 1 -- 172.21.15.26:0/3804872219 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f249c075300 con 0x7f249c0c88c0 2024-03-06T07:31:57.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.399+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f249c04f330 con 0x7f249c0c88c0 2024-03-06T07:31:57.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.399+0000 7f2492ffd640 1 -- 172.21.15.26:0/3804872219 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f248c058fb0 con 0x7f249c0c88c0 2024-03-06T07:31:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:57.401 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:57.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f248006b500 msgr2=0x7f248006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f248006b500 0x7f248006d9c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2498036000 tx=0x7f249803d040 comp rx=0 tx=0).stop 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0c88c0 msgr2=0x7f249c14bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0c88c0 0x7f249c14bce0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c820 tx=0x7f248c00cce0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 shutdown_connections 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f248006b500 0x7f248006d9c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249c14f360 0x7f249c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0c88c0 0x7f249c14bce0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 --2- 172.21.15.26:0/3804872219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249c0c5880 0x7f249c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 >> 172.21.15.26:0/3804872219 conn(0x7f249c0ac570 msgr2=0x7f249c0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 shutdown_connections 2024-03-06T07:31:57.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:57.401+0000 7f24a4755640 1 -- 172.21.15.26:0/3804872219 wait complete. 2024-03-06T07:31:57.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:57.815 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:31:58.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:58.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.917+0000 7fccad19a640 1 Processor -- start 2024-03-06T07:31:58.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.917+0000 7fccad19a640 1 -- start start 2024-03-06T07:31:58.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fccad19a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcca814f360 0x7fcca8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:58.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fccad19a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcca80c88c0 0x7fcca80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:58.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fccad19a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca80c73d0 con 0x7fcca80c5880 2024-03-06T07:31:58.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fccad19a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80c8dd0 con 0x7fcca814f360 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fccad19a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80be2e0 con 0x7fcca80c88c0 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fcca6d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcca814f360 0x7fcca8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fcca759e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcca80c5880 0x7fcca8158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49310/0 (socket says 172.21.15.26:49310) 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fcca6d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcca814f360 0x7fcca8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45484/0 (socket says 172.21.15.26:45484) 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.918+0000 7fcca759e640 1 -- 172.21.15.26:0/3340411131 learned_addr learned my addr 172.21.15.26:0/3340411131 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.919+0000 7fcca6d9d640 1 -- 172.21.15.26:0/3340411131 >> v2:172.21.15.117:3300/0 conn(0x7fcca80c88c0 msgr2=0x7fcca80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.919+0000 7fcca6d9d640 1 --2- 172.21.15.26:0/3340411131 >> v2:172.21.15.117:3300/0 conn(0x7fcca80c88c0 0x7fcca80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.919+0000 7fcca6d9d640 1 -- 172.21.15.26:0/3340411131 >> v1:172.21.15.26:6789/0 conn(0x7fcca80c5880 legacy=0x7fcca8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.919+0000 7fcca659c640 1 -- 172.21.15.26:0/3340411131 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135391289 0 0) 0x7fcca80c73d0 con 0x7fcca80c5880 2024-03-06T07:31:58.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.919+0000 7fcca6d9d640 1 -- 172.21.15.26:0/3340411131 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca80c7670 con 0x7fcca814f360 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.919+0000 7fcca6d9d640 1 --2- 172.21.15.26:0/3340411131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca814f360 0x7fcca8155c10 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fcca80c84c0 tx=0x7fcc9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c975ebc196427c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fcca659c640 1 -- 172.21.15.26:0/3340411131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc980315a0 con 0x7fcca814f360 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fcca659c640 1 -- 172.21.15.26:0/3340411131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc98037070 con 0x7fcca814f360 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fcca659c640 1 -- 172.21.15.26:0/3340411131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc98031960 con 0x7fcca814f360 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 -- 172.21.15.26:0/3340411131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca814f360 msgr2=0x7fcca8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 --2- 172.21.15.26:0/3340411131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca814f360 0x7fcca8155c10 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fcca80c84c0 tx=0x7fcc9802b840 comp rx=0 tx=0).stop 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 -- 172.21.15.26:0/3340411131 shutdown_connections 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 --2- 172.21.15.26:0/3340411131 >> v2:172.21.15.117:3300/0 conn(0x7fcca80c88c0 0x7fcca80c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 --2- 172.21.15.26:0/3340411131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca814f360 0x7fcca8155c10 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 -- 172.21.15.26:0/3340411131 >> 172.21.15.26:0/3340411131 conn(0x7fcca80ac570 msgr2=0x7fcca80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 -- 172.21.15.26:0/3340411131 shutdown_connections 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.920+0000 7fccad19a640 1 -- 172.21.15.26:0/3340411131 wait complete. 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.921+0000 7fccad19a640 1 Processor -- start 2024-03-06T07:31:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.921+0000 7fccad19a640 1 -- start start 2024-03-06T07:31:58.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fccad19a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcca80c5880 0x7fcca81eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fccad19a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca80c88c0 0x7fcca8157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fccad19a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcca814f360 0x7fcca814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fccad19a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8159270 con 0x7fcca80c88c0 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fccad19a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca80c7c30 con 0x7fcca814f360 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fccad19a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81593f0 con 0x7fcca80c5880 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca80c88c0 0x7fcca8157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca80c88c0 0x7fcca8157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45500/0 (socket says 172.21.15.26:45500) 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 -- 172.21.15.26:0/1795009601 learned_addr learned my addr 172.21.15.26:0/1795009601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcca6d9d640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcca80c5880 0x7fcca81eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcca759e640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcca814f360 0x7fcca814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 -- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcca80c5880 msgr2=0x7fcca81eb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcca80c5880 0x7fcca81eb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 -- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcca814f360 msgr2=0x7fcca814bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcca814f360 0x7fcca814bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcc9ffff640 1 -- 172.21.15.26:0/1795009601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca80c07e0 con 0x7fcca80c88c0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcca6d9d640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcca80c5880 0x7fcca81eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.922+0000 7fcca759e640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcca814f360 0x7fcca814bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.923+0000 7fcc9ffff640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca80c88c0 0x7fcca8157d10 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b480 tx=0x7fcc9000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.923+0000 7fcc9f7fe640 1 -- 172.21.15.26:0/1795009601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90014070 con 0x7fcca80c88c0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.923+0000 7fcc9f7fe640 1 -- 172.21.15.26:0/1795009601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc90010870 con 0x7fcca80c88c0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.923+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca80c0ac0 con 0x7fcca80c88c0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.923+0000 7fcc9f7fe640 1 -- 172.21.15.26:0/1795009601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90010c60 con 0x7fcca80c88c0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.923+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca8151530 con 0x7fcca80c88c0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.924+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca804f330 con 0x7fcca80c88c0 2024-03-06T07:31:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.924+0000 7fcc9f7fe640 1 -- 172.21.15.26:0/1795009601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc90017070 con 0x7fcca80c88c0 2024-03-06T07:31:58.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.925+0000 7fcc9f7fe640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc8806b4d0 0x7fcc8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:58.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.925+0000 7fcc9f7fe640 1 -- 172.21.15.26:0/1795009601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fcc9008a300 con 0x7fcca80c88c0 2024-03-06T07:31:58.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.925+0000 7fcca6d9d640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc8806b4d0 0x7fcc8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:58.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.925+0000 7fcca6d9d640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc8806b4d0 0x7fcc8806d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcc98036000 tx=0x7fcc98008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:58.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:58.929+0000 7fcc9f7fe640 1 -- 172.21.15.26:0/1795009601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcc90058ee0 con 0x7fcca80c88c0 2024-03-06T07:31:59.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.222+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcca806cda0 con 0x7fcca80c88c0 2024-03-06T07:31:59.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.222+0000 7fcc9f7fe640 1 -- 172.21.15.26:0/1795009601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fcc900587e0 con 0x7fcca80c88c0 2024-03-06T07:31:59.224 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:31:59.224 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:31:59.224 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:31:59.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc8806b4d0 msgr2=0x7fcc8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:59.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc8806b4d0 0x7fcc8806d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcc98036000 tx=0x7fcc98008030 comp rx=0 tx=0).stop 2024-03-06T07:31:59.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca80c88c0 msgr2=0x7fcca8157d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:59.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca80c88c0 0x7fcca8157d10 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b480 tx=0x7fcc9000b940 comp rx=0 tx=0).stop 2024-03-06T07:31:59.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 shutdown_connections 2024-03-06T07:31:59.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcc8806b4d0 0x7fcc8806d990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcca814f360 0x7fcca814bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca80c88c0 0x7fcca8157d10 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.224+0000 7fccad19a640 1 --2- 172.21.15.26:0/1795009601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcca80c5880 0x7fcca81eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.225+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 >> 172.21.15.26:0/1795009601 conn(0x7fcca80ac570 msgr2=0x7fcca80c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:59.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.225+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 shutdown_connections 2024-03-06T07:31:59.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:31:59.225+0000 7fccad19a640 1 -- 172.21.15.26:0/1795009601 wait complete. 2024-03-06T07:31:59.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:31:59.640 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:00.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:00.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.780+0000 7f44c86df640 1 Processor -- start 2024-03-06T07:32:00.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.780+0000 7f44c86df640 1 -- start start 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c86df640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44c0155c80 0x7f44c006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c86df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c00c1ae0 con 0x7f44c00720a0 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c86df640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c006e450 con 0x7f44c0152520 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c86df640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00c1dc0 con 0x7f44c0155c80 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c6c7c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f44c00720a0 0x7f44c006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49312/0 (socket says 172.21.15.26:49312) 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c6c7c640 1 -- 172.21.15.26:0/2168494750 learned_addr learned my addr 172.21.15.26:0/2168494750 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/2168494750 >> v2:172.21.15.117:3300/0 conn(0x7f44c0155c80 0x7f44c006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365809093 0 0) 0x7f44c00c1ae0 con 0x7f44c00720a0 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a8003520 con 0x7f44c00720a0 2024-03-06T07:32:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3536220663 0 0) 0x7f44a8003520 con 0x7f44c00720a0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 >> v2:172.21.15.117:3300/0 conn(0x7f44c0155c80 msgr2=0x7f44c006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 --2- 172.21.15.26:0/2168494750 >> v2:172.21.15.117:3300/0 conn(0x7f44c0155c80 0x7f44c006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 >> v1:172.21.15.117:6789/0 conn(0x7f44c0152520 legacy=0x7f44c00c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c006e730 con 0x7f44c00720a0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f44b00019f0 con 0x7f44c00720a0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.781+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 >> v1:172.21.15.26:6789/0 conn(0x7f44c00720a0 legacy=0x7f44c006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5479640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44a80039b0 0x7f44a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5479640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5479640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00c1dc0 con 0x7f44a8004800 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a800c060 con 0x7f44a80051c0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a800c260 con 0x7f44a80039b0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c647b640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44a80039b0 0x7f44a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c6c7c640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5c7a640 1 -- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44a80039b0 msgr2=0x7f44a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:00.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44a80039b0 0x7f44a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5c7a640 1 -- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a80051c0 msgr2=0x7f44a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:00.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c6c7c640 1 -- 172.21.15.26:0/2168494750 reap_dead start 2024-03-06T07:32:00.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.782+0000 7f44c5c7a640 1 -- 172.21.15.26:0/2168494750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a8003520 con 0x7f44a8004800 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.783+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009ac0 tx=0x7f44a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c48db3cb5467f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.783+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4039070 con 0x7f44a8004800 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.783+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b402c6b0 con 0x7f44a8004800 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.783+0000 7f44c5479640 1 -- 172.21.15.26:0/2168494750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4009690 con 0x7f44a8004800 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.784+0000 7f44c86df640 1 -- 172.21.15.26:0/2168494750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 msgr2=0x7f44a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.784+0000 7f44c86df640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009ac0 tx=0x7f44a8005830 comp rx=0 tx=0).stop 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.784+0000 7f44c86df640 1 -- 172.21.15.26:0/2168494750 shutdown_connections 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.784+0000 7f44c86df640 1 --2- 172.21.15.26:0/2168494750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.784+0000 7f44c86df640 1 -- 172.21.15.26:0/2168494750 >> 172.21.15.26:0/2168494750 conn(0x7f44c00c76b0 msgr2=0x7f44c00c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.784+0000 7f44c86df640 1 -- 172.21.15.26:0/2168494750 shutdown_connections 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.784+0000 7f44c86df640 1 -- 172.21.15.26:0/2168494750 wait complete. 2024-03-06T07:32:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.785+0000 7f44c86df640 1 Processor -- start 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.785+0000 7f44c86df640 1 -- start start 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c86df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44c00ac530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c86df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44c00aca70 0x7f44c01ee190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c86df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44c01ee6d0 0x7f44c014d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c86df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0152a80 con 0x7f44a8004800 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c86df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0152900 con 0x7f44c00aca70 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c86df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0072510 con 0x7f44c01ee6d0 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44c00ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44c00ac530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45514/0 (socket says 172.21.15.26:45514) 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 -- 172.21.15.26:0/1971192847 learned_addr learned my addr 172.21.15.26:0/1971192847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44c00aca70 0x7f44c01ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c6c7c640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44c01ee6d0 0x7f44c014d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 -- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44c01ee6d0 msgr2=0x7f44c014d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44c01ee6d0 0x7f44c014d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 -- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44c00aca70 msgr2=0x7f44c01ee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44c00aca70 0x7f44c01ee190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 -- 172.21.15.26:0/1971192847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00ac050 con 0x7f44a8004800 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c6c7c640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44c01ee6d0 0x7f44c014d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c647b640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44c00ac530 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f44b800f820 tx=0x7f44b800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44beffd640 1 -- 172.21.15.26:0/1971192847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b80050d0 con 0x7f44a8004800 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44beffd640 1 -- 172.21.15.26:0/1971192847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b8018070 con 0x7f44a8004800 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.786+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c0156710 con 0x7f44a8004800 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.787+0000 7f44beffd640 1 -- 172.21.15.26:0/1971192847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8005490 con 0x7f44a8004800 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.787+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c006b780 con 0x7f44a8004800 2024-03-06T07:32:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.788+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c004f330 con 0x7f44a8004800 2024-03-06T07:32:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.788+0000 7f44beffd640 1 -- 172.21.15.26:0/1971192847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44b800d6d0 con 0x7f44a8004800 2024-03-06T07:32:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.788+0000 7f44beffd640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f44ac06b4d0 0x7f44ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.788+0000 7f44beffd640 1 -- 172.21.15.26:0/1971192847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f44b808a2a0 con 0x7f44a8004800 2024-03-06T07:32:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.789+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f44ac06b4d0 0x7f44ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.789+0000 7f44c5c7a640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f44ac06b4d0 0x7f44ac06d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f44b4036000 tx=0x7f44b4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:00.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:00.793+0000 7f44beffd640 1 -- 172.21.15.26:0/1971192847 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44b8058e80 con 0x7f44a8004800 2024-03-06T07:32:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.139+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44c00c0b70 con 0x7f44a8004800 2024-03-06T07:32:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.140+0000 7f44beffd640 1 -- 172.21.15.26:0/1971192847 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f44b8058780 con 0x7f44a8004800 2024-03-06T07:32:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:01.141 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:01.142 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f44ac06b4d0 msgr2=0x7f44ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f44ac06b4d0 0x7f44ac06d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f44b4036000 tx=0x7f44b4030000 comp rx=0 tx=0).stop 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 msgr2=0x7f44c00ac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44c00ac530 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f44b800f820 tx=0x7f44b800fce0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 shutdown_connections 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f44ac06b4d0 0x7f44ac06d990 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44c01ee6d0 0x7f44c014d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44c00aca70 0x7f44c01ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 --2- 172.21.15.26:0/1971192847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a8004800 0x7f44c00ac530 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 >> 172.21.15.26:0/1971192847 conn(0x7f44c00c76b0 msgr2=0x7f44c00c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:01.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 shutdown_connections 2024-03-06T07:32:01.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:01.142+0000 7f44c86df640 1 -- 172.21.15.26:0/1971192847 wait complete. 2024-03-06T07:32:01.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:01.557 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:02.557 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.658+0000 7fce6035e640 1 Processor -- start 2024-03-06T07:32:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.658+0000 7fce6035e640 1 -- start start 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce6035e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce5814f790 0x7fce581529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce6035e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce580c1aa0 0x7fce580c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce6035e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce580c5c60 0x7fce58155a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce6035e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580c2680 con 0x7fce580c1aa0 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce6035e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580c28d0 con 0x7fce5814f790 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce6035e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580c2bb0 con 0x7fce580c5c60 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce580c1aa0 0x7fce580c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce580c1aa0 0x7fce580c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45518/0 (socket says 172.21.15.26:45518) 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 -- 172.21.15.26:0/1345008420 learned_addr learned my addr 172.21.15.26:0/1345008420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5e8fb640 1 --2- 172.21.15.26:0/1345008420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce580c5c60 0x7fce58155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5e0fa640 1 --2- 172.21.15.26:0/1345008420 >> v2:172.21.15.117:3300/0 conn(0x7fce5814f790 0x7fce581529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 -- 172.21.15.26:0/1345008420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce580c5c60 msgr2=0x7fce58155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 --2- 172.21.15.26:0/1345008420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce580c5c60 0x7fce58155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 -- 172.21.15.26:0/1345008420 >> v2:172.21.15.117:3300/0 conn(0x7fce5814f790 msgr2=0x7fce581529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 --2- 172.21.15.26:0/1345008420 >> v2:172.21.15.117:3300/0 conn(0x7fce5814f790 0x7fce581529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5d8f9640 1 -- 172.21.15.26:0/1345008420 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580c2e90 con 0x7fce580c1aa0 2024-03-06T07:32:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.659+0000 7fce5e0fa640 1 --2- 172.21.15.26:0/1345008420 >> v2:172.21.15.117:3300/0 conn(0x7fce5814f790 0x7fce581529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.660+0000 7fce5d8f9640 1 --2- 172.21.15.26:0/1345008420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce580c5720 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fce580c3170 tx=0x7fce4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49a86aca798effd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.660+0000 7fce5d0f8640 1 -- 172.21.15.26:0/1345008420 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48035020 con 0x7fce580c1aa0 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.660+0000 7fce5d0f8640 1 -- 172.21.15.26:0/1345008420 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48031870 con 0x7fce580c1aa0 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.660+0000 7fce5d0f8640 1 -- 172.21.15.26:0/1345008420 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48031c40 con 0x7fce580c1aa0 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 -- 172.21.15.26:0/1345008420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 msgr2=0x7fce580c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 --2- 172.21.15.26:0/1345008420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce580c5720 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fce580c3170 tx=0x7fce4802b910 comp rx=0 tx=0).stop 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 -- 172.21.15.26:0/1345008420 shutdown_connections 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 --2- 172.21.15.26:0/1345008420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce580c5c60 0x7fce58155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 --2- 172.21.15.26:0/1345008420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce580c5720 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 --2- 172.21.15.26:0/1345008420 >> v2:172.21.15.117:3300/0 conn(0x7fce5814f790 0x7fce581529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 -- 172.21.15.26:0/1345008420 >> 172.21.15.26:0/1345008420 conn(0x7fce580c6710 msgr2=0x7fce580c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 -- 172.21.15.26:0/1345008420 shutdown_connections 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.661+0000 7fce6035e640 1 -- 172.21.15.26:0/1345008420 wait complete. 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.662+0000 7fce6035e640 1 Processor -- start 2024-03-06T07:32:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.662+0000 7fce6035e640 1 -- start start 2024-03-06T07:32:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce6035e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce581ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce6035e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce580c5c60 0x7fce5814e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce6035e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5814f790 0x7fce580728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce6035e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806cc00 con 0x7fce580c1aa0 2024-03-06T07:32:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce6035e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580c0d60 con 0x7fce580c5c60 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce6035e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806cd80 con 0x7fce5814f790 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce581ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce581ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45530/0 (socket says 172.21.15.26:45530) 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 -- 172.21.15.26:0/3200042593 learned_addr learned my addr 172.21.15.26:0/3200042593 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5d8f9640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce580c5c60 0x7fce5814e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e8fb640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5814f790 0x7fce580728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 -- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5814f790 msgr2=0x7fce580728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5814f790 0x7fce580728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 -- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce580c5c60 msgr2=0x7fce5814e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce580c5c60 0x7fce5814e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 -- 172.21.15.26:0/3200042593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580723c0 con 0x7fce580c1aa0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce5e0fa640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce581ec430 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d7d0 tx=0x7fce5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce4effd640 1 -- 172.21.15.26:0/3200042593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54004030 con 0x7fce580c1aa0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce4effd640 1 -- 172.21.15.26:0/3200042593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce540041d0 con 0x7fce580c1aa0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce580a6f10 con 0x7fce580c1aa0 2024-03-06T07:32:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.663+0000 7fce4effd640 1 -- 172.21.15.26:0/3200042593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540096a0 con 0x7fce580c1aa0 2024-03-06T07:32:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.664+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce580a66b0 con 0x7fce580c1aa0 2024-03-06T07:32:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.665+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce580c1010 con 0x7fce580c1aa0 2024-03-06T07:32:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.665+0000 7fce4effd640 1 -- 172.21.15.26:0/3200042593 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce54009840 con 0x7fce580c1aa0 2024-03-06T07:32:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.665+0000 7fce4effd640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce3c06b4d0 0x7fce3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.665+0000 7fce4effd640 1 -- 172.21.15.26:0/3200042593 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fce5408a690 con 0x7fce580c1aa0 2024-03-06T07:32:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.666+0000 7fce5d8f9640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce3c06b4d0 0x7fce3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.666+0000 7fce5d8f9640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce3c06b4d0 0x7fce3c06d990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fce48036000 tx=0x7fce4803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:02.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.670+0000 7fce4effd640 1 -- 172.21.15.26:0/3200042593 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fce580c1010 con 0x7fce580c1aa0 2024-03-06T07:32:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.937+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce5804f330 con 0x7fce580c1aa0 2024-03-06T07:32:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.938+0000 7fce4effd640 1 -- 172.21.15.26:0/3200042593 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fce54059310 con 0x7fce580c1aa0 2024-03-06T07:32:02.940 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:02.940 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:02.940 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce3c06b4d0 msgr2=0x7fce3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce3c06b4d0 0x7fce3c06d990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fce48036000 tx=0x7fce4803c040 comp rx=0 tx=0).stop 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 msgr2=0x7fce581ec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce581ec430 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d7d0 tx=0x7fce5400dc90 comp rx=0 tx=0).stop 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 shutdown_connections 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fce3c06b4d0 0x7fce3c06d990 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce5814f790 0x7fce580728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce580c5c60 0x7fce5814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 --2- 172.21.15.26:0/3200042593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce580c1aa0 0x7fce581ec430 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 >> 172.21.15.26:0/3200042593 conn(0x7fce580c6710 msgr2=0x7fce580c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 shutdown_connections 2024-03-06T07:32:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:02.940+0000 7fce6035e640 1 -- 172.21.15.26:0/3200042593 wait complete. 2024-03-06T07:32:03.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:03.354 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:04.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:04.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.455+0000 7fc00b8df640 1 Processor -- start 2024-03-06T07:32:04.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.455+0000 7fc00b8df640 1 -- start start 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00b8df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0040c5480 0x7fc004155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00b8df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 0x7fc004158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00b8df640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040c5cc0 con 0x7fc0040c5860 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00b8df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00406df40 con 0x7fc0040c5480 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00b8df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00406fdb0 con 0x7fc0041586f0 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00967b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0040c5480 0x7fc004155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00967b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0040c5480 0x7fc004155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45536/0 (socket says 172.21.15.26:45536) 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00967b640 1 -- 172.21.15.26:0/517745190 learned_addr learned my addr 172.21.15.26:0/517745190 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc009e7c640 1 --2- 172.21.15.26:0/517745190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 0x7fc004158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00967b640 1 -- 172.21.15.26:0/517745190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 msgr2=0x7fc004158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00967b640 1 --2- 172.21.15.26:0/517745190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 0x7fc004158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00967b640 1 -- 172.21.15.26:0/517745190 >> v1:172.21.15.117:6789/0 conn(0x7fc0040c5860 legacy=0x7fc00406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.457+0000 7fc00967b640 1 -- 172.21.15.26:0/517745190 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0041591c0 con 0x7fc0040c5480 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.458+0000 7fc00967b640 1 --2- 172.21.15.26:0/517745190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc004155ba0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fc000009870 tx=0x7fc00002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4009f5871b86564f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.458+0000 7fbffbfff640 1 -- 172.21.15.26:0/517745190 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000035020 con 0x7fc0040c5480 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.458+0000 7fbffbfff640 1 -- 172.21.15.26:0/517745190 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000031640 con 0x7fc0040c5480 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.458+0000 7fbffbfff640 1 -- 172.21.15.26:0/517745190 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00002d040 con 0x7fc0040c5480 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 -- 172.21.15.26:0/517745190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 msgr2=0x7fc004155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 --2- 172.21.15.26:0/517745190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc004155ba0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fc000009870 tx=0x7fc00002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 -- 172.21.15.26:0/517745190 shutdown_connections 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 --2- 172.21.15.26:0/517745190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 0x7fc004158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 --2- 172.21.15.26:0/517745190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc004155ba0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 -- 172.21.15.26:0/517745190 >> 172.21.15.26:0/517745190 conn(0x7fc00406b1a0 msgr2=0x7fc00406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 -- 172.21.15.26:0/517745190 shutdown_connections 2024-03-06T07:32:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.459+0000 7fc00b8df640 1 -- 172.21.15.26:0/517745190 wait complete. 2024-03-06T07:32:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.460+0000 7fc00b8df640 1 Processor -- start 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.460+0000 7fc00b8df640 1 -- start start 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.460+0000 7fc00b8df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc0041566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.460+0000 7fc00b8df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0040c5860 0x7fc004150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.460+0000 7fc00b8df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 0x7fc004151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00b8df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004155680 con 0x7fc0040c5480 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00b8df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041553a0 con 0x7fc0040c5860 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00b8df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc004152a20 con 0x7fc0041586f0 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc0041566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc0041566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45546/0 (socket says 172.21.15.26:45546) 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 -- 172.21.15.26:0/3374226742 learned_addr learned my addr 172.21.15.26:0/3374226742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc008e7a640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0040c5860 0x7fc004150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 -- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 msgr2=0x7fc004151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 0x7fc004151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 -- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0040c5860 msgr2=0x7fc004150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0040c5860 0x7fc004150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 -- 172.21.15.26:0/3374226742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0040c4df0 con 0x7fc0040c5480 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00967b640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc0041566b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fc00002b930 tx=0x7fc0000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fbff9ffb640 1 -- 172.21.15.26:0/3374226742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000040070 con 0x7fc0040c5480 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fbff9ffb640 1 -- 172.21.15.26:0/3374226742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00003e070 con 0x7fc0040c5480 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040c4b10 con 0x7fc0040c5480 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.461+0000 7fbff9ffb640 1 -- 172.21.15.26:0/3374226742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000035040 con 0x7fc0040c5480 2024-03-06T07:32:04.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.462+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc004150510 con 0x7fc0040c5480 2024-03-06T07:32:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.463+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00404f330 con 0x7fc0040c5480 2024-03-06T07:32:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.463+0000 7fbff9ffb640 1 -- 172.21.15.26:0/3374226742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc000038020 con 0x7fc0040c5480 2024-03-06T07:32:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.463+0000 7fbff9ffb640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfe806b4d0 0x7fbfe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.464+0000 7fbff9ffb640 1 -- 172.21.15.26:0/3374226742 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc0000abd90 con 0x7fc0040c5480 2024-03-06T07:32:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.464+0000 7fc008e7a640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfe806b4d0 0x7fbfe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.464+0000 7fc008e7a640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfe806b4d0 0x7fbfe806d990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc0040c5170 tx=0x7fbff4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.468+0000 7fbff9ffb640 1 -- 172.21.15.26:0/3374226742 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc00007aa10 con 0x7fc0040c5480 2024-03-06T07:32:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.685+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc00406e0c0 con 0x7fc0040c5480 2024-03-06T07:32:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.685+0000 7fbff9ffb640 1 -- 172.21.15.26:0/3374226742 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc000043e40 con 0x7fc0040c5480 2024-03-06T07:32:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:04.687 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfe806b4d0 msgr2=0x7fbfe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfe806b4d0 0x7fbfe806d990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc0040c5170 tx=0x7fbff4009040 comp rx=0 tx=0).stop 2024-03-06T07:32:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 msgr2=0x7fc0041566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc0041566b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fc00002b930 tx=0x7fc0000046c0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 shutdown_connections 2024-03-06T07:32:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbfe806b4d0 0x7fbfe806d990 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0041586f0 0x7fc004151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0040c5860 0x7fc004150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 --2- 172.21.15.26:0/3374226742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0040c5480 0x7fc0041566b0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 >> 172.21.15.26:0/3374226742 conn(0x7fc00406b1a0 msgr2=0x7fc0040c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 shutdown_connections 2024-03-06T07:32:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:04.687+0000 7fc00b8df640 1 -- 172.21.15.26:0/3374226742 wait complete. 2024-03-06T07:32:05.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:05.103 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:06.104 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:06.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.204+0000 7f1f279c7640 1 Processor -- start 2024-03-06T07:32:06.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.204+0000 7f1f279c7640 1 -- start start 2024-03-06T07:32:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f279c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f2014f360 0x7f1f20155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f279c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f200c88c0 0x7f1f200c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f279c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c5880 0x7f1f20158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f279c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200c8dd0 con 0x7f1f2014f360 2024-03-06T07:32:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f279c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200c73d0 con 0x7f1f200c88c0 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f279c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200be2e0 con 0x7f1f200c5880 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f2014f360 0x7f1f20155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f2014f360 0x7f1f20155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45548/0 (socket says 172.21.15.26:45548) 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 -- 172.21.15.26:0/1348029870 learned_addr learned my addr 172.21.15.26:0/1348029870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 -- 172.21.15.26:0/1348029870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c5880 msgr2=0x7f1f20158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f24f62640 1 --2- 172.21.15.26:0/1348029870 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c88c0 0x7f1f200c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25f64640 1 --2- 172.21.15.26:0/1348029870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c5880 0x7f1f20158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 --2- 172.21.15.26:0/1348029870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c5880 0x7f1f20158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 -- 172.21.15.26:0/1348029870 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c88c0 msgr2=0x7f1f200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 --2- 172.21.15.26:0/1348029870 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c88c0 0x7f1f200c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.205+0000 7f1f25763640 1 -- 172.21.15.26:0/1348029870 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200c7670 con 0x7f1f2014f360 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.206+0000 7f1f25763640 1 --2- 172.21.15.26:0/1348029870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2014f360 0x7f1f20155c10 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f1f200c84c0 tx=0x7f1f1402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d9936271536ed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.206+0000 7f1f13fff640 1 -- 172.21.15.26:0/1348029870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1402c9e0 con 0x7f1f2014f360 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.206+0000 7f1f13fff640 1 -- 172.21.15.26:0/1348029870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1402cb80 con 0x7f1f2014f360 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.206+0000 7f1f13fff640 1 -- 172.21.15.26:0/1348029870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140343f0 con 0x7f1f2014f360 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 -- 172.21.15.26:0/1348029870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2014f360 msgr2=0x7f1f20155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 --2- 172.21.15.26:0/1348029870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2014f360 0x7f1f20155c10 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f1f200c84c0 tx=0x7f1f1402be40 comp rx=0 tx=0).stop 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 -- 172.21.15.26:0/1348029870 shutdown_connections 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 --2- 172.21.15.26:0/1348029870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c5880 0x7f1f20158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 --2- 172.21.15.26:0/1348029870 >> v2:172.21.15.117:3300/0 conn(0x7f1f200c88c0 0x7f1f200c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 --2- 172.21.15.26:0/1348029870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2014f360 0x7f1f20155c10 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 -- 172.21.15.26:0/1348029870 >> 172.21.15.26:0/1348029870 conn(0x7f1f200ac570 msgr2=0x7f1f200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:06.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 -- 172.21.15.26:0/1348029870 shutdown_connections 2024-03-06T07:32:06.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.207+0000 7f1f279c7640 1 -- 172.21.15.26:0/1348029870 wait complete. 2024-03-06T07:32:06.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.208+0000 7f1f279c7640 1 Processor -- start 2024-03-06T07:32:06.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.208+0000 7f1f279c7640 1 -- start start 2024-03-06T07:32:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f279c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200c5880 0x7f1f200ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f279c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c88c0 0x7f1f20156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f279c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2014f360 0x7f1f20148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f279c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f201590a0 con 0x7f1f200c5880 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f279c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200c7c30 con 0x7f1f2014f360 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f279c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20159220 con 0x7f1f200c88c0 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200c5880 0x7f1f200ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200c5880 0x7f1f200ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45558/0 (socket says 172.21.15.26:45558) 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25763640 1 -- 172.21.15.26:0/3589313296 learned_addr learned my addr 172.21.15.26:0/3589313296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f24f62640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c88c0 0x7f1f20156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25f64640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2014f360 0x7f1f20148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25763640 1 -- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c88c0 msgr2=0x7f1f20156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25763640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c88c0 0x7f1f20156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25763640 1 -- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2014f360 msgr2=0x7f1f20148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.209+0000 7f1f25763640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2014f360 0x7f1f20148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.210+0000 7f1f25763640 1 -- 172.21.15.26:0/3589313296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20157ed0 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.210+0000 7f1f25763640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200c5880 0x7f1f200ab6a0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f1f14036040 tx=0x7f1f14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.210+0000 7f1f11ffb640 1 -- 172.21.15.26:0/3589313296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14043020 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.210+0000 7f1f11ffb640 1 -- 172.21.15.26:0/3589313296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1403e450 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.210+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f201581f0 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.210+0000 7f1f11ffb640 1 -- 172.21.15.26:0/3589313296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1403e840 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.210+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f201ebe80 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.211+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2004f330 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.212+0000 7f1f11ffb640 1 -- 172.21.15.26:0/3589313296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f14046020 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.212+0000 7f1f11ffb640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0406b4d0 0x7f1f0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.212+0000 7f1f11ffb640 1 -- 172.21.15.26:0/3589313296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f1f140ad400 con 0x7f1f200c5880 2024-03-06T07:32:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.212+0000 7f1f24f62640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0406b4d0 0x7f1f0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.213+0000 7f1f24f62640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0406b4d0 0x7f1f0406d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1f200c8340 tx=0x7f1f1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.216+0000 7f1f11ffb640 1 -- 172.21.15.26:0/3589313296 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f1404d030 con 0x7f1f200c5880 2024-03-06T07:32:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.553+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f2006cda0 con 0x7f1f200c5880 2024-03-06T07:32:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.553+0000 7f1f11ffb640 1 -- 172.21.15.26:0/3589313296 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f1f1407c280 con 0x7f1f200c5880 2024-03-06T07:32:06.555 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:06.555 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:06.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0406b4d0 msgr2=0x7f1f0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0406b4d0 0x7f1f0406d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1f200c8340 tx=0x7f1f1c008040 comp rx=0 tx=0).stop 2024-03-06T07:32:06.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200c5880 msgr2=0x7f1f200ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200c5880 0x7f1f200ab6a0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f1f14036040 tx=0x7f1f14004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 shutdown_connections 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1f0406b4d0 0x7f1f0406d990 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2014f360 0x7f1f20148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c88c0 0x7f1f20156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 --2- 172.21.15.26:0/3589313296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200c5880 0x7f1f200ab6a0 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.555+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 >> 172.21.15.26:0/3589313296 conn(0x7f1f200ac570 msgr2=0x7f1f200c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.556+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 shutdown_connections 2024-03-06T07:32:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:06.556+0000 7f1f279c7640 1 -- 172.21.15.26:0/3589313296 wait complete. 2024-03-06T07:32:06.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:06.974 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:07.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:08.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.075+0000 7fd0708c9640 1 Processor -- start 2024-03-06T07:32:08.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.075+0000 7fd0708c9640 1 -- start start 2024-03-06T07:32:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd0708c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd06c159270 0x7fd06c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd0708c9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0c73d0 con 0x7fd06c14f360 2024-03-06T07:32:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd0708c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0be2e0 con 0x7fd06c0c88c0 2024-03-06T07:32:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd0708c9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c159700 con 0x7fd06c159270 2024-03-06T07:32:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd06ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd06c159270 0x7fd06c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd06a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd06c14f360 0x7fd06c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49282/0 (socket says 172.21.15.26:49282) 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd06ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd06c159270 0x7fd06c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51558/0 (socket says 172.21.15.26:51558) 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.076+0000 7fd06a59c640 1 -- 172.21.15.26:0/2739454074 learned_addr learned my addr 172.21.15.26:0/2739454074 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.077+0000 7fd06ad9d640 1 -- 172.21.15.26:0/2739454074 >> v1:172.21.15.117:6789/0 conn(0x7fd06c0c88c0 legacy=0x7fd06c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.077+0000 7fd06ad9d640 1 -- 172.21.15.26:0/2739454074 >> v1:172.21.15.26:6789/0 conn(0x7fd06c14f360 legacy=0x7fd06c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.077+0000 7fd06ad9d640 1 -- 172.21.15.26:0/2739454074 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0c7670 con 0x7fd06c159270 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.077+0000 7fd06ad9d640 1 --2- 172.21.15.26:0/2739454074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c155c10 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fd064009d90 tx=0x7fd06402bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5f85f11f2fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.077+0000 7fd06959a640 1 -- 172.21.15.26:0/2739454074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064039070 con 0x7fd06c159270 2024-03-06T07:32:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd06959a640 1 -- 172.21.15.26:0/2739454074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06402c990 con 0x7fd06c159270 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd06959a640 1 -- 172.21.15.26:0/2739454074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06402cd60 con 0x7fd06c159270 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd0708c9640 1 -- 172.21.15.26:0/2739454074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 msgr2=0x7fd06c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2739454074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c155c10 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fd064009d90 tx=0x7fd06402bc60 comp rx=0 tx=0).stop 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd0708c9640 1 -- 172.21.15.26:0/2739454074 shutdown_connections 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2739454074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c155c10 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd0708c9640 1 -- 172.21.15.26:0/2739454074 >> 172.21.15.26:0/2739454074 conn(0x7fd06c0ac570 msgr2=0x7fd06c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd0708c9640 1 -- 172.21.15.26:0/2739454074 shutdown_connections 2024-03-06T07:32:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.078+0000 7fd0708c9640 1 -- 172.21.15.26:0/2739454074 wait complete. 2024-03-06T07:32:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.079+0000 7fd0708c9640 1 Processor -- start 2024-03-06T07:32:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.079+0000 7fd0708c9640 1 -- start start 2024-03-06T07:32:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd0708c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06c0c88c0 0x7fd06c0bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd0708c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06c14f360 0x7fd06c0c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd0708c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c1ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd0708c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c7c30 con 0x7fd06c159270 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd0708c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c5a50 con 0x7fd06c0c88c0 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd0708c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c5bd0 con 0x7fd06c14f360 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c1ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c1ecc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51572/0 (socket says 172.21.15.26:51572) 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 -- 172.21.15.26:0/2434347185 learned_addr learned my addr 172.21.15.26:0/2434347185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06a59c640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06c0c88c0 0x7fd06c0bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd069d9b640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06c14f360 0x7fd06c0c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 -- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06c14f360 msgr2=0x7fd06c0c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06c14f360 0x7fd06c0c0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 -- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06c0c88c0 msgr2=0x7fd06c0bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06c0c88c0 0x7fd06c0bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06ad9d640 1 -- 172.21.15.26:0/2434347185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0bf9f0 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd06a59c640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06c0c88c0 0x7fd06c0bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.080+0000 7fd069d9b640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06c14f360 0x7fd06c0c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.081+0000 7fd06ad9d640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c1ecc90 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fd064009eb0 tx=0x7fd064004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.081+0000 7fd062ffd640 1 -- 172.21.15.26:0/2434347185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064044020 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.081+0000 7fd062ffd640 1 -- 172.21.15.26:0/2434347185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064034cf0 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.081+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c0bfcd0 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.081+0000 7fd062ffd640 1 -- 172.21.15.26:0/2434347185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064039040 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.081+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c1ec490 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.082+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c04f330 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.082+0000 7fd062ffd640 1 -- 172.21.15.26:0/2434347185 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd064047020 con 0x7fd06c159270 2024-03-06T07:32:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.083+0000 7fd062ffd640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd04c06b4d0 0x7fd04c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.083+0000 7fd062ffd640 1 -- 172.21.15.26:0/2434347185 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd0640acb90 con 0x7fd06c159270 2024-03-06T07:32:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.083+0000 7fd06a59c640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd04c06b4d0 0x7fd04c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.084+0000 7fd06a59c640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd04c06b4d0 0x7fd04c06d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd054000f60 tx=0x7fd054009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.087+0000 7fd062ffd640 1 -- 172.21.15.26:0/2434347185 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd06404e030 con 0x7fd06c159270 2024-03-06T07:32:08.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.365+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd06c0c8d50 con 0x7fd06c159270 2024-03-06T07:32:08.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.366+0000 7fd062ffd640 1 -- 172.21.15.26:0/2434347185 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fd06407b280 con 0x7fd06c159270 2024-03-06T07:32:08.368 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:08.368 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:08.368 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd04c06b4d0 msgr2=0x7fd04c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd04c06b4d0 0x7fd04c06d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd054000f60 tx=0x7fd054009040 comp rx=0 tx=0).stop 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 msgr2=0x7fd06c1ecc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c1ecc90 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fd064009eb0 tx=0x7fd064004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 shutdown_connections 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd04c06b4d0 0x7fd04c06d990 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c159270 0x7fd06c1ecc90 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06c14f360 0x7fd06c0c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 --2- 172.21.15.26:0/2434347185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06c0c88c0 0x7fd06c0bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 >> 172.21.15.26:0/2434347185 conn(0x7fd06c0ac570 msgr2=0x7fd06c154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 shutdown_connections 2024-03-06T07:32:08.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:08.368+0000 7fd0708c9640 1 -- 172.21.15.26:0/2434347185 wait complete. 2024-03-06T07:32:08.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:08.783 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:09.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.884+0000 7f0886c47640 1 Processor -- start 2024-03-06T07:32:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.884+0000 7f0886c47640 1 -- start start 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.885+0000 7f0886c47640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0880159270 0x7f0880155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.885+0000 7f0886c47640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08800c73d0 con 0x7f08800c88c0 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.885+0000 7f0886c47640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08800be2e0 con 0x7f088014f360 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.885+0000 7f0886c47640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0880159700 con 0x7f0880159270 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.885+0000 7f087ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f08800c88c0 0x7f0880158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49296/0 (socket says 172.21.15.26:49296) 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.885+0000 7f087ffff640 1 -- 172.21.15.26:0/464000847 learned_addr learned my addr 172.21.15.26:0/464000847 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.885+0000 7f08851e4640 1 --2- 172.21.15.26:0/464000847 >> v2:172.21.15.26:3300/0 conn(0x7f0880159270 0x7f0880155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.886+0000 7f08851e4640 1 -- 172.21.15.26:0/464000847 >> v1:172.21.15.117:6789/0 conn(0x7f088014f360 legacy=0x7f08800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.886+0000 7f08851e4640 1 -- 172.21.15.26:0/464000847 >> v1:172.21.15.26:6789/0 conn(0x7f08800c88c0 legacy=0x7f0880158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.886+0000 7f08851e4640 1 -- 172.21.15.26:0/464000847 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800c7670 con 0x7f0880159270 2024-03-06T07:32:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.886+0000 7f087f7fe640 1 -- 172.21.15.26:0/464000847 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237523774 0 0) 0x7f08800c73d0 con 0x7f08800c88c0 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.886+0000 7f08851e4640 1 --2- 172.21.15.26:0/464000847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0880159270 0x7f0880155c10 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f0878009d90 tx=0x7f087802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf15c516c062730 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f087f7fe640 1 -- 172.21.15.26:0/464000847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878039070 con 0x7f0880159270 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f087f7fe640 1 -- 172.21.15.26:0/464000847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087802cc60 con 0x7f0880159270 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f087f7fe640 1 -- 172.21.15.26:0/464000847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878034660 con 0x7f0880159270 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f0886c47640 1 -- 172.21.15.26:0/464000847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0880159270 msgr2=0x7f0880155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f0886c47640 1 --2- 172.21.15.26:0/464000847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0880159270 0x7f0880155c10 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f0878009d90 tx=0x7f087802beb0 comp rx=0 tx=0).stop 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f0886c47640 1 -- 172.21.15.26:0/464000847 shutdown_connections 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f0886c47640 1 --2- 172.21.15.26:0/464000847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0880159270 0x7f0880155c10 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f0886c47640 1 -- 172.21.15.26:0/464000847 >> 172.21.15.26:0/464000847 conn(0x7f08800ac570 msgr2=0x7f08800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:09.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f0886c47640 1 -- 172.21.15.26:0/464000847 shutdown_connections 2024-03-06T07:32:09.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.887+0000 7f0886c47640 1 -- 172.21.15.26:0/464000847 wait complete. 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.888+0000 7f0886c47640 1 Processor -- start 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.888+0000 7f0886c47640 1 -- start start 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f0886c47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08800c88c0 0x7f08800c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f0886c47640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088014f360 0x7f08801e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f0886c47640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0880159270 0x7f08801519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f0886c47640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c7c30 con 0x7f088014f360 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f0886c47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c5a50 con 0x7f08800c88c0 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f0886c47640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c5bd0 con 0x7f0880159270 2024-03-06T07:32:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088014f360 0x7f08801e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088014f360 0x7f08801e8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51586/0 (socket says 172.21.15.26:51586) 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 -- 172.21.15.26:0/1970807618 learned_addr learned my addr 172.21.15.26:0/1970807618 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f08849e3640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08800c88c0 0x7f08800c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f08851e4640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0880159270 0x7f08801519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 -- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0880159270 msgr2=0x7f08801519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0880159270 0x7f08801519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 -- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08800c88c0 msgr2=0x7f08800c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08800c88c0 0x7f08800c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.889+0000 7f087ffff640 1 -- 172.21.15.26:0/1970807618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800ab780 con 0x7f088014f360 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.890+0000 7f087ffff640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088014f360 0x7f08801e8100 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f087400c460 tx=0x7f087400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.890+0000 7f087d7fa640 1 -- 172.21.15.26:0/1970807618 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874018070 con 0x7f088014f360 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.890+0000 7f087d7fa640 1 -- 172.21.15.26:0/1970807618 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874007520 con 0x7f088014f360 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.890+0000 7f087d7fa640 1 -- 172.21.15.26:0/1970807618 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874004a20 con 0x7f088014f360 2024-03-06T07:32:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.890+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08800bf4f0 con 0x7f088014f360 2024-03-06T07:32:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.890+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08800bfab0 con 0x7f088014f360 2024-03-06T07:32:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.891+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088004f330 con 0x7f088014f360 2024-03-06T07:32:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.891+0000 7f087d7fa640 1 -- 172.21.15.26:0/1970807618 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f087401b070 con 0x7f088014f360 2024-03-06T07:32:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.892+0000 7f087d7fa640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f086c06b4d0 0x7f086c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.892+0000 7f087d7fa640 1 -- 172.21.15.26:0/1970807618 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f0874089740 con 0x7f088014f360 2024-03-06T07:32:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.892+0000 7f08849e3640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f086c06b4d0 0x7f086c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.893+0000 7f08849e3640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f086c06b4d0 0x7f086c06d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f08801e8910 tx=0x7f0870009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:09.896+0000 7f087d7fa640 1 -- 172.21.15.26:0/1970807618 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f08800ab780 con 0x7f088014f360 2024-03-06T07:32:10.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.120+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08800c8d50 con 0x7f088014f360 2024-03-06T07:32:10.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.120+0000 7f087d7fa640 1 -- 172.21.15.26:0/1970807618 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f0874057d70 con 0x7f088014f360 2024-03-06T07:32:10.122 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:10.122 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.122+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f086c06b4d0 msgr2=0x7f086c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.122+0000 7f0886c47640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f086c06b4d0 0x7f086c06d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f08801e8910 tx=0x7f0870009040 comp rx=0 tx=0).stop 2024-03-06T07:32:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.122+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088014f360 msgr2=0x7f08801e8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.122+0000 7f0886c47640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088014f360 0x7f08801e8100 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f087400c460 tx=0x7f087400c920 comp rx=0 tx=0).stop 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 shutdown_connections 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f086c06b4d0 0x7f086c06d990 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0880159270 0x7f08801519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088014f360 0x7f08801e8100 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 --2- 172.21.15.26:0/1970807618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08800c88c0 0x7f08800c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 >> 172.21.15.26:0/1970807618 conn(0x7f08800ac570 msgr2=0x7f088014e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 shutdown_connections 2024-03-06T07:32:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:10.123+0000 7f0886c47640 1 -- 172.21.15.26:0/1970807618 wait complete. 2024-03-06T07:32:10.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:10.538 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:11.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.640+0000 7f8bb353e640 1 Processor -- start 2024-03-06T07:32:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.640+0000 7f8bb353e640 1 -- start start 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb353e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac0c5480 0x7f8bac155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb353e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8bac0c5860 0x7f8bac06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb353e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bac1586f0 0x7f8bac158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb353e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac06df40 con 0x7f8bac1586f0 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb353e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac0c5cc0 con 0x7f8bac0c5860 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb353e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac1591c0 con 0x7f8bac0c5480 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb1adb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bac1586f0 0x7f8bac158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb1adb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bac1586f0 0x7f8bac158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51594/0 (socket says 172.21.15.26:51594) 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb1adb640 1 -- 172.21.15.26:0/1711473987 learned_addr learned my addr 172.21.15.26:0/1711473987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb0ad9640 1 --2- 172.21.15.26:0/1711473987 >> v2:172.21.15.117:3300/0 conn(0x7f8bac0c5860 0x7f8bac06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.641+0000 7f8bb12da640 1 --2- 172.21.15.26:0/1711473987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac0c5480 0x7f8bac155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.642+0000 7f8bb1adb640 1 -- 172.21.15.26:0/1711473987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac0c5480 msgr2=0x7f8bac155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.642+0000 7f8bb1adb640 1 --2- 172.21.15.26:0/1711473987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac0c5480 0x7f8bac155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.642+0000 7f8bb1adb640 1 -- 172.21.15.26:0/1711473987 >> v2:172.21.15.117:3300/0 conn(0x7f8bac0c5860 msgr2=0x7f8bac06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.642+0000 7f8bb1adb640 1 --2- 172.21.15.26:0/1711473987 >> v2:172.21.15.117:3300/0 conn(0x7f8bac0c5860 0x7f8bac06f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.642+0000 7f8bb1adb640 1 -- 172.21.15.26:0/1711473987 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac1594a0 con 0x7f8bac1586f0 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.642+0000 7f8bb0ad9640 1 --2- 172.21.15.26:0/1711473987 >> v2:172.21.15.117:3300/0 conn(0x7f8bac0c5860 0x7f8bac06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.642+0000 7f8bb1adb640 1 --2- 172.21.15.26:0/1711473987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac1586f0 0x7f8bac158b80 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0735c0 tx=0x7f8ba402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d422d84b28ea20 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8b9ffff640 1 -- 172.21.15.26:0/1711473987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba402c9e0 con 0x7f8bac1586f0 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8b9ffff640 1 -- 172.21.15.26:0/1711473987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba402cb80 con 0x7f8bac1586f0 2024-03-06T07:32:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8b9ffff640 1 -- 172.21.15.26:0/1711473987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40343f0 con 0x7f8bac1586f0 2024-03-06T07:32:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 -- 172.21.15.26:0/1711473987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac1586f0 msgr2=0x7f8bac158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 --2- 172.21.15.26:0/1711473987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac1586f0 0x7f8bac158b80 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0735c0 tx=0x7f8ba402be40 comp rx=0 tx=0).stop 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 -- 172.21.15.26:0/1711473987 shutdown_connections 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 --2- 172.21.15.26:0/1711473987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac1586f0 0x7f8bac158b80 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 --2- 172.21.15.26:0/1711473987 >> v2:172.21.15.117:3300/0 conn(0x7f8bac0c5860 0x7f8bac06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 --2- 172.21.15.26:0/1711473987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac0c5480 0x7f8bac155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 -- 172.21.15.26:0/1711473987 >> 172.21.15.26:0/1711473987 conn(0x7f8bac06b1a0 msgr2=0x7f8bac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 -- 172.21.15.26:0/1711473987 shutdown_connections 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.643+0000 7f8bb353e640 1 -- 172.21.15.26:0/1711473987 wait complete. 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.644+0000 7f8bb353e640 1 Processor -- start 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.644+0000 7f8bb353e640 1 -- start start 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb353e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bac0c5480 0x7f8bac0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb353e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac0c5860 0x7f8bac0aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb353e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac1586f0 0x7f8bac0aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb353e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac152a20 con 0x7f8bac0c5860 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb353e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac155680 con 0x7f8bac0c5480 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb353e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac155050 con 0x7f8bac1586f0 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac0c5860 0x7f8bac0aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac0c5860 0x7f8bac0aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51596/0 (socket says 172.21.15.26:51596) 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 -- 172.21.15.26:0/3478811392 learned_addr learned my addr 172.21.15.26:0/3478811392 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb12da640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bac0c5480 0x7f8bac0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb1adb640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac1586f0 0x7f8bac0aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 -- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac1586f0 msgr2=0x7f8bac0aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac1586f0 0x7f8bac0aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 -- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bac0c5480 msgr2=0x7f8bac0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bac0c5480 0x7f8bac0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.645+0000 7f8bb0ad9640 1 -- 172.21.15.26:0/3478811392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac158100 con 0x7f8bac0c5860 2024-03-06T07:32:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.646+0000 7f8bb0ad9640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac0c5860 0x7f8bac0aa1c0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800e730 tx=0x7f8ba800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.646+0000 7f8b9dffb640 1 -- 172.21.15.26:0/3478811392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800f040 con 0x7f8bac0c5860 2024-03-06T07:32:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.646+0000 7f8b9dffb640 1 -- 172.21.15.26:0/3478811392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8004230 con 0x7f8bac0c5860 2024-03-06T07:32:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.646+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac0a8630 con 0x7f8bac0c5860 2024-03-06T07:32:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.646+0000 7f8b9dffb640 1 -- 172.21.15.26:0/3478811392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800ee40 con 0x7f8bac0c5860 2024-03-06T07:32:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.646+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac1eba70 con 0x7f8bac0c5860 2024-03-06T07:32:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.647+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac04f330 con 0x7f8bac0c5860 2024-03-06T07:32:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.647+0000 7f8b9dffb640 1 -- 172.21.15.26:0/3478811392 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ba800c6d0 con 0x7f8bac0c5860 2024-03-06T07:32:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.648+0000 7f8b9dffb640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9006b4d0 0x7f8b9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.648+0000 7f8b9dffb640 1 -- 172.21.15.26:0/3478811392 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f8ba8089f90 con 0x7f8bac0c5860 2024-03-06T07:32:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.648+0000 7f8bb12da640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9006b4d0 0x7f8b9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.649+0000 7f8bb12da640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9006b4d0 0x7f8b9006d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00079a0 tx=0x7f8ba0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.652+0000 7f8b9dffb640 1 -- 172.21.15.26:0/3478811392 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8ba8017020 con 0x7f8bac0c5860 2024-03-06T07:32:11.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.873+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bac1564f0 con 0x7f8bac0c5860 2024-03-06T07:32:11.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.874+0000 7f8b9dffb640 1 -- 172.21.15.26:0/3478811392 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f8ba80585c0 con 0x7f8bac0c5860 2024-03-06T07:32:11.876 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:11.876 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:11.876 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9006b4d0 msgr2=0x7f8b9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9006b4d0 0x7f8b9006d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00079a0 tx=0x7f8ba0008040 comp rx=0 tx=0).stop 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac0c5860 msgr2=0x7f8bac0aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac0c5860 0x7f8bac0aa1c0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800e730 tx=0x7f8ba800ebf0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 shutdown_connections 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8b9006b4d0 0x7f8b9006d990 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bac1586f0 0x7f8bac0aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bac0c5860 0x7f8bac0aa1c0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 --2- 172.21.15.26:0/3478811392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bac0c5480 0x7f8bac0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 >> 172.21.15.26:0/3478811392 conn(0x7f8bac06b1a0 msgr2=0x7f8bac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 shutdown_connections 2024-03-06T07:32:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:11.876+0000 7f8bb353e640 1 -- 172.21.15.26:0/3478811392 wait complete. 2024-03-06T07:32:12.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:12.291 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:13.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.433+0000 7f3d7bced640 1 Processor -- start 2024-03-06T07:32:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.433+0000 7f3d7bced640 1 -- start start 2024-03-06T07:32:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7bced640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c20b0 0x7f3d7406b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7bced640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d740c25c0 0x7f3d74153890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7bced640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d74153dd0 con 0x7f3d7406afc0 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7bced640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d740a9530 con 0x7f3d740c25c0 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7bced640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740a9810 con 0x7f3d740c20b0 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7a28a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d740c25c0 0x7f3d74153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7a28a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d740c25c0 0x7f3d74153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51606/0 (socket says 172.21.15.26:51606) 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d7a28a640 1 -- 172.21.15.26:0/2297414059 learned_addr learned my addr 172.21.15.26:0/2297414059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.434+0000 7f3d79288640 1 --2- 172.21.15.26:0/2297414059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c20b0 0x7f3d7406b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.435+0000 7f3d7a28a640 1 -- 172.21.15.26:0/2297414059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c20b0 msgr2=0x7f3d7406b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.435+0000 7f3d7a28a640 1 --2- 172.21.15.26:0/2297414059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c20b0 0x7f3d7406b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.435+0000 7f3d7a28a640 1 -- 172.21.15.26:0/2297414059 >> v1:172.21.15.117:6789/0 conn(0x7f3d7406afc0 legacy=0x7f3d74153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.435+0000 7f3d7a28a640 1 -- 172.21.15.26:0/2297414059 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740a9af0 con 0x7f3d740c25c0 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.435+0000 7f3d7a28a640 1 --2- 172.21.15.26:0/2297414059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d740c25c0 0x7f3d74153890 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009e50 tx=0x7f3d6c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208f5b74453d703a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d78a87640 1 -- 172.21.15.26:0/2297414059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c02c9e0 con 0x7f3d740c25c0 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d78a87640 1 -- 172.21.15.26:0/2297414059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c02cb80 con 0x7f3d740c25c0 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d78a87640 1 -- 172.21.15.26:0/2297414059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0343f0 con 0x7f3d740c25c0 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 -- 172.21.15.26:0/2297414059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d740c25c0 msgr2=0x7f3d74153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2297414059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d740c25c0 0x7f3d74153890 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009e50 tx=0x7f3d6c02be40 comp rx=0 tx=0).stop 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 -- 172.21.15.26:0/2297414059 shutdown_connections 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2297414059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d740c25c0 0x7f3d74153890 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2297414059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c20b0 0x7f3d7406b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 -- 172.21.15.26:0/2297414059 >> 172.21.15.26:0/2297414059 conn(0x7f3d740c76b0 msgr2=0x7f3d740c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 -- 172.21.15.26:0/2297414059 shutdown_connections 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.436+0000 7f3d7bced640 1 -- 172.21.15.26:0/2297414059 wait complete. 2024-03-06T07:32:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.437+0000 7f3d7bced640 1 Processor -- start 2024-03-06T07:32:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.437+0000 7f3d7bced640 1 -- start start 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d7bced640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d7406afc0 0x7f3d7414b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d7bced640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d740c20b0 0x7f3d741e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d7bced640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c25c0 0x7f3d741dd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d7bced640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74150980 con 0x7f3d7406afc0 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d7bced640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740aa0b0 con 0x7f3d740c20b0 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d7bced640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74150b00 con 0x7f3d740c25c0 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d7406afc0 0x7f3d7414b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d7406afc0 0x7f3d7414b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51612/0 (socket says 172.21.15.26:51612) 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 -- 172.21.15.26:0/2097784062 learned_addr learned my addr 172.21.15.26:0/2097784062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d7a28a640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c25c0 0x7f3d741dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79288640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d740c20b0 0x7f3d741e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 -- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c25c0 msgr2=0x7f3d741dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c25c0 0x7f3d741dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 -- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d740c20b0 msgr2=0x7f3d741e9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d740c20b0 0x7f3d741e9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79a89640 1 -- 172.21.15.26:0/2097784062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d741e9fb0 con 0x7f3d7406afc0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.438+0000 7f3d79288640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d740c20b0 0x7f3d741e9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.439+0000 7f3d79a89640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d7406afc0 0x7f3d7414b120 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000c3c0 tx=0x7f3d7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.439+0000 7f3d6a7fc640 1 -- 172.21.15.26:0/2097784062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d700052d0 con 0x7f3d7406afc0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.439+0000 7f3d6a7fc640 1 -- 172.21.15.26:0/2097784062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d70005490 con 0x7f3d7406afc0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.439+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d741dc6e0 con 0x7f3d7406afc0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.439+0000 7f3d6a7fc640 1 -- 172.21.15.26:0/2097784062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70007260 con 0x7f3d7406afc0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.439+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d741dcca0 con 0x7f3d7406afc0 2024-03-06T07:32:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.440+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d74069460 con 0x7f3d7406afc0 2024-03-06T07:32:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.440+0000 7f3d6a7fc640 1 -- 172.21.15.26:0/2097784062 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d70007400 con 0x7f3d7406afc0 2024-03-06T07:32:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.441+0000 7f3d6a7fc640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d5806b4d0 0x7f3d5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.441+0000 7f3d6a7fc640 1 -- 172.21.15.26:0/2097784062 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f3d700897e0 con 0x7f3d7406afc0 2024-03-06T07:32:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.441+0000 7f3d79288640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d5806b4d0 0x7f3d5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.442+0000 7f3d79288640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d5806b4d0 0x7f3d5806d990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3d7406c710 tx=0x7f3d64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.445+0000 7f3d6a7fc640 1 -- 172.21.15.26:0/2097784062 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3d74069460 con 0x7f3d7406afc0 2024-03-06T07:32:13.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.664+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7404f330 con 0x7f3d7406afc0 2024-03-06T07:32:13.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.664+0000 7f3d6a7fc640 1 -- 172.21.15.26:0/2097784062 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f3d70057e10 con 0x7f3d7406afc0 2024-03-06T07:32:13.666 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:13.666 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:13.666 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:13.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d5806b4d0 msgr2=0x7f3d5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d5806b4d0 0x7f3d5806d990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3d7406c710 tx=0x7f3d64008040 comp rx=0 tx=0).stop 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d7406afc0 msgr2=0x7f3d7414b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d7406afc0 0x7f3d7414b120 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000c3c0 tx=0x7f3d7000c880 comp rx=0 tx=0).stop 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 shutdown_connections 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3d5806b4d0 0x7f3d5806d990 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d740c25c0 0x7f3d741dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d740c20b0 0x7f3d741e9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 --2- 172.21.15.26:0/2097784062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d7406afc0 0x7f3d7414b120 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 >> 172.21.15.26:0/2097784062 conn(0x7f3d740c76b0 msgr2=0x7f3d74149ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 shutdown_connections 2024-03-06T07:32:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:13.667+0000 7f3d7bced640 1 -- 172.21.15.26:0/2097784062 wait complete. 2024-03-06T07:32:13.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:13.676 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:14.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:14.082 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:15.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:15.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 Processor -- start 2024-03-06T07:32:15.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 -- start start 2024-03-06T07:32:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f519814f360 0x7f5198155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51980c88c0 0x7f51980c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f5198158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980c8dd0 con 0x7f51980c88c0 2024-03-06T07:32:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980c73d0 con 0x7f519814f360 2024-03-06T07:32:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.184+0000 7f519c916640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980be2e0 con 0x7f51980c5880 2024-03-06T07:32:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51980c88c0 0x7f51980c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51980c88c0 0x7f51980c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51626/0 (socket says 172.21.15.26:51626) 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 -- 172.21.15.26:0/3152745990 learned_addr learned my addr 172.21.15.26:0/3152745990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f5196d9d640 1 --2- 172.21.15.26:0/3152745990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f5198158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f519659c640 1 --2- 172.21.15.26:0/3152745990 >> v2:172.21.15.117:3300/0 conn(0x7f519814f360 0x7f5198155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 -- 172.21.15.26:0/3152745990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 msgr2=0x7f5198158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 --2- 172.21.15.26:0/3152745990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f5198158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 -- 172.21.15.26:0/3152745990 >> v2:172.21.15.117:3300/0 conn(0x7f519814f360 msgr2=0x7f5198155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 --2- 172.21.15.26:0/3152745990 >> v2:172.21.15.117:3300/0 conn(0x7f519814f360 0x7f5198155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f518dd9b640 1 -- 172.21.15.26:0/3152745990 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980c7670 con 0x7f51980c88c0 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.185+0000 7f519659c640 1 --2- 172.21.15.26:0/3152745990 >> v2:172.21.15.117:3300/0 conn(0x7f519814f360 0x7f5198155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f518dd9b640 1 --2- 172.21.15.26:0/3152745990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51980c88c0 0x7f51980c5340 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f519808c840 tx=0x7f518002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5f0c2b2ab79e22 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f5195d9b640 1 -- 172.21.15.26:0/3152745990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518002e070 con 0x7f51980c88c0 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f5195d9b640 1 -- 172.21.15.26:0/3152745990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180037070 con 0x7f51980c88c0 2024-03-06T07:32:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f5195d9b640 1 -- 172.21.15.26:0/3152745990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180031a00 con 0x7f51980c88c0 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 -- 172.21.15.26:0/3152745990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51980c88c0 msgr2=0x7f51980c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 --2- 172.21.15.26:0/3152745990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51980c88c0 0x7f51980c5340 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f519808c840 tx=0x7f518002b880 comp rx=0 tx=0).stop 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 -- 172.21.15.26:0/3152745990 shutdown_connections 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 --2- 172.21.15.26:0/3152745990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f5198158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 --2- 172.21.15.26:0/3152745990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51980c88c0 0x7f51980c5340 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 --2- 172.21.15.26:0/3152745990 >> v2:172.21.15.117:3300/0 conn(0x7f519814f360 0x7f5198155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 -- 172.21.15.26:0/3152745990 >> 172.21.15.26:0/3152745990 conn(0x7f51980ac570 msgr2=0x7f51980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 -- 172.21.15.26:0/3152745990 shutdown_connections 2024-03-06T07:32:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.186+0000 7f519c916640 1 -- 172.21.15.26:0/3152745990 wait complete. 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 Processor -- start 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 -- start start 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f51980c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51980c88c0 0x7f51980c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519814f360 0x7f51980c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981590a0 con 0x7f519814f360 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980c7c30 con 0x7f51980c88c0 2024-03-06T07:32:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f519c916640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5198159220 con 0x7f51980c5880 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f5196d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519814f360 0x7f51980c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f5196d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519814f360 0x7f51980c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51628/0 (socket says 172.21.15.26:51628) 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.188+0000 7f5196d9d640 1 -- 172.21.15.26:0/2927204742 learned_addr learned my addr 172.21.15.26:0/2927204742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f518dd9b640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51980c88c0 0x7f51980c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f519659c640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f51980c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f5196d9d640 1 -- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 msgr2=0x7f51980c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f5196d9d640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f51980c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f5196d9d640 1 -- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51980c88c0 msgr2=0x7f51980c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f5196d9d640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51980c88c0 0x7f51980c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f5196d9d640 1 -- 172.21.15.26:0/2927204742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980be740 con 0x7f519814f360 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f519659c640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f51980c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f5196d9d640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519814f360 0x7f51980c9ea0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f5190007e60 tx=0x7f519000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f518f7fe640 1 -- 172.21.15.26:0/2927204742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190014620 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f518f7fe640 1 -- 172.21.15.26:0/2927204742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5190019070 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51981ee880 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f518f7fe640 1 -- 172.21.15.26:0/2927204742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190014a20 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.189+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51981eeb60 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.190+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51981ee6b0 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.191+0000 7f518f7fe640 1 -- 172.21.15.26:0/2927204742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5190014bc0 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.191+0000 7f518f7fe640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f517806b500 0x7f517806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.191+0000 7f518f7fe640 1 -- 172.21.15.26:0/2927204742 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f519008ac70 con 0x7f519814f360 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.191+0000 7f519659c640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f517806b500 0x7f517806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.192+0000 7f519659c640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f517806b500 0x7f517806d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5188006fd0 tx=0x7f5188008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:15.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.196+0000 7f518f7fe640 1 -- 172.21.15.26:0/2927204742 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f51900598f0 con 0x7f519814f360 2024-03-06T07:32:15.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.415+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f519804f330 con 0x7f519814f360 2024-03-06T07:32:15.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.415+0000 7f518f7fe640 1 -- 172.21.15.26:0/2927204742 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f51900591f0 con 0x7f519814f360 2024-03-06T07:32:15.418 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:15.418 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:15.418 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f517806b500 msgr2=0x7f517806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f517806b500 0x7f517806d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5188006fd0 tx=0x7f5188008040 comp rx=0 tx=0).stop 2024-03-06T07:32:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519814f360 msgr2=0x7f51980c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519814f360 0x7f51980c9ea0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f5190007e60 tx=0x7f519000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 shutdown_connections 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f517806b500 0x7f517806d9c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519814f360 0x7f51980c9ea0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51980c88c0 0x7f51980c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 --2- 172.21.15.26:0/2927204742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51980c5880 0x7f51980c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 >> 172.21.15.26:0/2927204742 conn(0x7f51980ac570 msgr2=0x7f51980cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 shutdown_connections 2024-03-06T07:32:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:15.418+0000 7f519c916640 1 -- 172.21.15.26:0/2927204742 wait complete. 2024-03-06T07:32:15.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:15.833 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:16.834 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.935+0000 7fc180108640 1 Processor -- start 2024-03-06T07:32:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc180108640 1 -- start start 2024-03-06T07:32:16.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc180108640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc17814f650 0x7fc178071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:16.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc180108640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1780c3150 0x7fc1780c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:16.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc180108640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc178152c20 0x7fc1780c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:16.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc180108640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc178072440 con 0x7fc17814f650 2024-03-06T07:32:16.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc180108640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780c2b10 con 0x7fc178152c20 2024-03-06T07:32:16.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc180108640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc178152140 con 0x7fc1780c3150 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc17dea4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc17814f650 0x7fc178071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc17dea4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc17814f650 0x7fc178071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51640/0 (socket says 172.21.15.26:51640) 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.936+0000 7fc17dea4640 1 -- 172.21.15.26:0/1117685812 learned_addr learned my addr 172.21.15.26:0/1117685812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17e6a5640 1 --2- 172.21.15.26:0/1117685812 >> v2:172.21.15.117:3300/0 conn(0x7fc178152c20 0x7fc1780c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17dea4640 1 -- 172.21.15.26:0/1117685812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1780c3150 msgr2=0x7fc1780c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17d6a3640 1 --2- 172.21.15.26:0/1117685812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1780c3150 0x7fc1780c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17dea4640 1 --2- 172.21.15.26:0/1117685812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1780c3150 0x7fc1780c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17dea4640 1 -- 172.21.15.26:0/1117685812 >> v2:172.21.15.117:3300/0 conn(0x7fc178152c20 msgr2=0x7fc1780c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17dea4640 1 --2- 172.21.15.26:0/1117685812 >> v2:172.21.15.117:3300/0 conn(0x7fc178152c20 0x7fc1780c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17dea4640 1 -- 172.21.15.26:0/1117685812 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178152420 con 0x7fc17814f650 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17e6a5640 1 --2- 172.21.15.26:0/1117685812 >> v2:172.21.15.117:3300/0 conn(0x7fc178152c20 0x7fc1780c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.937+0000 7fc17dea4640 1 --2- 172.21.15.26:0/1117685812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc17814f650 0x7fc178071880 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fc17806f6b0 tx=0x7fc17402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaa2f46dd65c0e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc17cea2640 1 -- 172.21.15.26:0/1117685812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174035020 con 0x7fc17814f650 2024-03-06T07:32:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc17cea2640 1 -- 172.21.15.26:0/1117685812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc174031870 con 0x7fc17814f650 2024-03-06T07:32:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc17cea2640 1 -- 172.21.15.26:0/1117685812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174031c40 con 0x7fc17814f650 2024-03-06T07:32:16.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 -- 172.21.15.26:0/1117685812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc17814f650 msgr2=0x7fc178071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:16.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 --2- 172.21.15.26:0/1117685812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc17814f650 0x7fc178071880 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fc17806f6b0 tx=0x7fc17402b910 comp rx=0 tx=0).stop 2024-03-06T07:32:16.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 -- 172.21.15.26:0/1117685812 shutdown_connections 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 --2- 172.21.15.26:0/1117685812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1780c3150 0x7fc1780c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 --2- 172.21.15.26:0/1117685812 >> v2:172.21.15.117:3300/0 conn(0x7fc178152c20 0x7fc1780c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 --2- 172.21.15.26:0/1117685812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc17814f650 0x7fc178071880 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 -- 172.21.15.26:0/1117685812 >> 172.21.15.26:0/1117685812 conn(0x7fc1780c4290 msgr2=0x7fc1780c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 -- 172.21.15.26:0/1117685812 shutdown_connections 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.938+0000 7fc180108640 1 -- 172.21.15.26:0/1117685812 wait complete. 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.939+0000 7fc180108640 1 Processor -- start 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.939+0000 7fc180108640 1 -- start start 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc180108640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1780c3150 0x7fc178145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc180108640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc17814f650 0x7fc1781454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc180108640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc178152c20 0x7fc178153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:16.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc180108640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780c40c0 con 0x7fc1780c3150 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc180108640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780c3f40 con 0x7fc178152c20 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc180108640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc178071dc0 con 0x7fc17814f650 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1780c3150 0x7fc178145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1780c3150 0x7fc178145cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51656/0 (socket says 172.21.15.26:51656) 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 -- 172.21.15.26:0/1096511218 learned_addr learned my addr 172.21.15.26:0/1096511218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 -- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc17814f650 msgr2=0x7fc1781454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc17814f650 0x7fc1781454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 -- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc178152c20 msgr2=0x7fc178153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc178152c20 0x7fc178153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.940+0000 7fc17dea4640 1 -- 172.21.15.26:0/1096511218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17814df50 con 0x7fc1780c3150 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.941+0000 7fc17dea4640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1780c3150 0x7fc178145cd0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fc174000c00 tx=0x7fc1740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.941+0000 7fc16e7fc640 1 -- 172.21.15.26:0/1096511218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174035040 con 0x7fc1780c3150 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.941+0000 7fc16e7fc640 1 -- 172.21.15.26:0/1096511218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc174045480 con 0x7fc1780c3150 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.941+0000 7fc16e7fc640 1 -- 172.21.15.26:0/1096511218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174045870 con 0x7fc1780c3150 2024-03-06T07:32:16.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.941+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17814e230 con 0x7fc1780c3150 2024-03-06T07:32:16.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.941+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1781552d0 con 0x7fc1780c3150 2024-03-06T07:32:16.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.942+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17804f330 con 0x7fc1780c3150 2024-03-06T07:32:16.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.942+0000 7fc16e7fc640 1 -- 172.21.15.26:0/1096511218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc174038020 con 0x7fc1780c3150 2024-03-06T07:32:16.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.943+0000 7fc16e7fc640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc15c06b4d0 0x7fc15c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:16.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.943+0000 7fc16e7fc640 1 -- 172.21.15.26:0/1096511218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc1740ac970 con 0x7fc1780c3150 2024-03-06T07:32:16.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.943+0000 7fc17d6a3640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc15c06b4d0 0x7fc15c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:16.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.944+0000 7fc17d6a3640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc15c06b4d0 0x7fc15c06d990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc168006b10 tx=0x7fc168008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:16.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:16.947+0000 7fc16e7fc640 1 -- 172.21.15.26:0/1096511218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc17407b5f0 con 0x7fc1780c3150 2024-03-06T07:32:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.188+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1780c2300 con 0x7fc1780c3150 2024-03-06T07:32:17.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.188+0000 7fc16e7fc640 1 -- 172.21.15.26:0/1096511218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc17407aef0 con 0x7fc1780c3150 2024-03-06T07:32:17.190 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:17.191 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:17.191 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc15c06b4d0 msgr2=0x7fc15c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc15c06b4d0 0x7fc15c06d990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc168006b10 tx=0x7fc168008040 comp rx=0 tx=0).stop 2024-03-06T07:32:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1780c3150 msgr2=0x7fc178145cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1780c3150 0x7fc178145cd0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fc174000c00 tx=0x7fc1740046c0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 shutdown_connections 2024-03-06T07:32:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc15c06b4d0 0x7fc15c06d990 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc178152c20 0x7fc178153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc17814f650 0x7fc1781454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 --2- 172.21.15.26:0/1096511218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1780c3150 0x7fc178145cd0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 >> 172.21.15.26:0/1096511218 conn(0x7fc1780c4290 msgr2=0x7fc1780c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 shutdown_connections 2024-03-06T07:32:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:17.191+0000 7fc180108640 1 -- 172.21.15.26:0/1096511218 wait complete. 2024-03-06T07:32:17.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:17.606 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:18.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff94eb640 1 Processor -- start 2024-03-06T07:32:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff94eb640 1 -- start start 2024-03-06T07:32:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff94eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ff4073250 0x7f8ff40c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff94eb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff40bdc20 con 0x7f8ff4072e70 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff94eb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff406c8e0 con 0x7f8ff40c3950 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff94eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff406ce80 con 0x7f8ff4073250 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff2ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8ff4072e70 0x7f8ff40bd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51448/0 (socket says 172.21.15.26:51448) 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.709+0000 7f8ff2ffd640 1 -- 172.21.15.26:0/3326475353 learned_addr learned my addr 172.21.15.26:0/3326475353 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8febfff640 1 --2- 172.21.15.26:0/3326475353 >> v2:172.21.15.117:3300/0 conn(0x7f8ff4073250 0x7f8ff40c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306745873 0 0) 0x7f8ff40bdc20 con 0x7f8ff4072e70 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fe0003520 con 0x7f8ff4072e70 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1273169417 0 0) 0x7f8fe0003520 con 0x7f8ff4072e70 2024-03-06T07:32:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 >> v2:172.21.15.117:3300/0 conn(0x7f8ff4073250 msgr2=0x7f8ff40c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 --2- 172.21.15.26:0/3326475353 >> v2:172.21.15.117:3300/0 conn(0x7f8ff4073250 0x7f8ff40c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 >> v1:172.21.15.117:6789/0 conn(0x7f8ff40c3950 legacy=0x7f8ff415fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40c1b00 con 0x7f8ff4072e70 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f8fe4001670 con 0x7f8ff4072e70 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 >> v1:172.21.15.26:6789/0 conn(0x7f8ff4072e70 legacy=0x7f8ff40bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe00039b0 0x7f8fe0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe0004800 0x7f8fe0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe00051c0 0x7f8fe000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff406ce80 con 0x7f8fe00039b0 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe000c160 con 0x7f8fe00051c0 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe000c360 con 0x7f8fe0004800 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff2ffd640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe00039b0 0x7f8fe0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8febfff640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe0004800 0x7f8fe0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe00051c0 0x7f8fe000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.710+0000 7f8ff2ffd640 1 -- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe0004800 msgr2=0x7f8fe0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.711+0000 7f8ff2ffd640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe0004800 0x7f8fe0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.711+0000 7f8ff2ffd640 1 -- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe00051c0 msgr2=0x7f8fe000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.711+0000 7f8ff2ffd640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe00051c0 0x7f8fe000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.711+0000 7f8ff2ffd640 1 -- 172.21.15.26:0/3326475353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0003520 con 0x7f8fe00039b0 2024-03-06T07:32:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.711+0000 7f8ff37fe640 1 -- 172.21.15.26:0/3326475353 reap_dead start 2024-03-06T07:32:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff2ffd640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe00039b0 0x7f8fe0003dd0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c240 tx=0x7f8fe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636495f933d5c21 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4007d10 con 0x7f8fe00039b0 2024-03-06T07:32:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4030070 con 0x7f8fe00039b0 2024-03-06T07:32:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff27fc640 1 -- 172.21.15.26:0/3326475353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe402cda0 con 0x7f8fe00039b0 2024-03-06T07:32:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3326475353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe00039b0 msgr2=0x7f8fe0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe00039b0 0x7f8fe0003dd0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c240 tx=0x7f8fe0005830 comp rx=0 tx=0).stop 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3326475353 shutdown_connections 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3326475353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe00039b0 0x7f8fe0003dd0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.712+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3326475353 >> 172.21.15.26:0/3326475353 conn(0x7f8ff406b1a0 msgr2=0x7f8ff406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.713+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3326475353 shutdown_connections 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.713+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3326475353 wait complete. 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.714+0000 7f8ff94eb640 1 Processor -- start 2024-03-06T07:32:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.714+0000 7f8ff94eb640 1 -- start start 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff94eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe00039b0 0x7f8ff41eec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff94eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff41567b0 0x7f8ff4157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff94eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff4156c50 0x7f8ff41eac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff94eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c6760 con 0x7f8ff4156c50 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff94eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c4a00 con 0x7f8ff41567b0 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff94eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff406bb20 con 0x7f8fe00039b0 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff4156c50 0x7f8ff41eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.715+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff4156c50 0x7f8ff41eac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38256/0 (socket says 172.21.15.26:38256) 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff37fe640 1 -- 172.21.15.26:0/3314258889 learned_addr learned my addr 172.21.15.26:0/3314258889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff37fe640 1 -- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe00039b0 msgr2=0x7f8ff41eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe00039b0 0x7f8ff41eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8febfff640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff41567b0 0x7f8ff4157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff37fe640 1 -- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff41567b0 msgr2=0x7f8ff4157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff41567b0 0x7f8ff4157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff37fe640 1 -- 172.21.15.26:0/3314258889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff41590a0 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8febfff640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff41567b0 0x7f8ff4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff37fe640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff4156c50 0x7f8ff41eac50 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00f750 tx=0x7f8fec00fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8feb7fe640 1 -- 172.21.15.26:0/3314258889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec01a070 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8feb7fe640 1 -- 172.21.15.26:0/3314258889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec006d60 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff4159380 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8feb7fe640 1 -- 172.21.15.26:0/3314258889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec003bb0 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.716+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff41ee410 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.717+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff41ee200 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.718+0000 7f8feb7fe640 1 -- 172.21.15.26:0/3314258889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fec01d070 con 0x7f8ff4156c50 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.718+0000 7f8feb7fe640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.718+0000 7f8feb7fe640 1 -- 172.21.15.26:0/3314258889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f8fec089930 con 0x7f8ff4156c50 2024-03-06T07:32:18.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.719+0000 7f8ff2ffd640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:18.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.719+0000 7f8ff2ffd640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd406b4d0 0x7f8fd406d990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4034000 tx=0x7f8fe4007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:18.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.723+0000 7f8feb7fe640 1 -- 172.21.15.26:0/3314258889 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8fec058520 con 0x7f8ff4156c50 2024-03-06T07:32:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.942+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff404f330 con 0x7f8ff4156c50 2024-03-06T07:32:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.942+0000 7f8feb7fe640 1 -- 172.21.15.26:0/3314258889 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f8fec057e20 con 0x7f8ff4156c50 2024-03-06T07:32:18.944 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:18.944 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:18.945 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd406b4d0 msgr2=0x7f8fd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd406b4d0 0x7f8fd406d990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4034000 tx=0x7f8fe4007000 comp rx=0 tx=0).stop 2024-03-06T07:32:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff4156c50 msgr2=0x7f8ff41eac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff4156c50 0x7f8ff41eac50 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00f750 tx=0x7f8fec00fc10 comp rx=0 tx=0).stop 2024-03-06T07:32:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 shutdown_connections 2024-03-06T07:32:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ff4156c50 0x7f8ff41eac50 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ff41567b0 0x7f8ff4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 --2- 172.21.15.26:0/3314258889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe00039b0 0x7f8ff41eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:18.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 >> 172.21.15.26:0/3314258889 conn(0x7f8ff406b1a0 msgr2=0x7f8ff40c5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:18.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 shutdown_connections 2024-03-06T07:32:18.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:18.944+0000 7f8ff94eb640 1 -- 172.21.15.26:0/3314258889 wait complete. 2024-03-06T07:32:19.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:19.361 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:20.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.461+0000 7fe962b7a640 1 Processor -- start 2024-03-06T07:32:20.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.461+0000 7fe962b7a640 1 -- start start 2024-03-06T07:32:20.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe962b7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c14f360 0x7fe95c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe962b7a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe95c158b60 0x7fe95c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe962b7a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c0c73d0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe962b7a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0c8dd0 con 0x7fe95c158b60 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe962b7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0be2e0 con 0x7fe95c14f360 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe961117640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe95c158b60 0x7fe95c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe961117640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe95c158b60 0x7fe95c158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38258/0 (socket says 172.21.15.26:38258) 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe961117640 1 -- 172.21.15.26:0/583675973 learned_addr learned my addr 172.21.15.26:0/583675973 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe961117640 1 -- 172.21.15.26:0/583675973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c14f360 msgr2=0x7fe95c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe960916640 1 --2- 172.21.15.26:0/583675973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c14f360 0x7fe95c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe961117640 1 --2- 172.21.15.26:0/583675973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c14f360 0x7fe95c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe961117640 1 -- 172.21.15.26:0/583675973 >> v1:172.21.15.117:6789/0 conn(0x7fe95c0c88c0 legacy=0x7fe95c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe961117640 1 -- 172.21.15.26:0/583675973 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c0c7670 con 0x7fe95c158b60 2024-03-06T07:32:20.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.462+0000 7fe960916640 1 --2- 172.21.15.26:0/583675973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c14f360 0x7fe95c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.463+0000 7fe961117640 1 --2- 172.21.15.26:0/583675973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c158b60 0x7fe95c158ff0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fe954009e50 tx=0x7fe95402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61ce8b4ef73b1e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.463+0000 7fe95b7fe640 1 -- 172.21.15.26:0/583675973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95402c9e0 con 0x7fe95c158b60 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.463+0000 7fe95b7fe640 1 -- 172.21.15.26:0/583675973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95402cb80 con 0x7fe95c158b60 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.463+0000 7fe95b7fe640 1 -- 172.21.15.26:0/583675973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9540343f0 con 0x7fe95c158b60 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.463+0000 7fe962b7a640 1 -- 172.21.15.26:0/583675973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c158b60 msgr2=0x7fe95c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.463+0000 7fe962b7a640 1 --2- 172.21.15.26:0/583675973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c158b60 0x7fe95c158ff0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fe954009e50 tx=0x7fe95402be40 comp rx=0 tx=0).stop 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.464+0000 7fe962b7a640 1 -- 172.21.15.26:0/583675973 shutdown_connections 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.464+0000 7fe962b7a640 1 --2- 172.21.15.26:0/583675973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c158b60 0x7fe95c158ff0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.464+0000 7fe962b7a640 1 --2- 172.21.15.26:0/583675973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c14f360 0x7fe95c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.464+0000 7fe962b7a640 1 -- 172.21.15.26:0/583675973 >> 172.21.15.26:0/583675973 conn(0x7fe95c0ac570 msgr2=0x7fe95c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.464+0000 7fe962b7a640 1 -- 172.21.15.26:0/583675973 shutdown_connections 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.464+0000 7fe962b7a640 1 -- 172.21.15.26:0/583675973 wait complete. 2024-03-06T07:32:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.465+0000 7fe962b7a640 1 Processor -- start 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.465+0000 7fe962b7a640 1 -- start start 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe962b7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c0c88c0 0x7fe95c14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe962b7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c14f360 0x7fe95c1ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe962b7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c158b60 0x7fe95c1ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe962b7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c5a50 con 0x7fe95c0c88c0 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe962b7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c7c30 con 0x7fe95c14f360 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe962b7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c5bd0 con 0x7fe95c158b60 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c0c88c0 0x7fe95c14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c0c88c0 0x7fe95c14be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38268/0 (socket says 172.21.15.26:38268) 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 -- 172.21.15.26:0/1890986481 learned_addr learned my addr 172.21.15.26:0/1890986481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe95bfff640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c14f360 0x7fe95c1ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe961117640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c158b60 0x7fe95c1ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:20.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 -- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c158b60 msgr2=0x7fe95c1ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c158b60 0x7fe95c1ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 -- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c14f360 msgr2=0x7fe95c1ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c14f360 0x7fe95c1ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 -- 172.21.15.26:0/1890986481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c0c4da0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe961117640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c158b60 0x7fe95c1ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe960916640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c0c88c0 0x7fe95c14be50 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c7f0 tx=0x7fe95000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe9597fa640 1 -- 172.21.15.26:0/1890986481 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950015070 con 0x7fe95c0c88c0 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.466+0000 7fe9597fa640 1 -- 172.21.15.26:0/1890986481 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe950007820 con 0x7fe95c0c88c0 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.467+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0afdc0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.467+0000 7fe9597fa640 1 -- 172.21.15.26:0/1890986481 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950007bd0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.467+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c1eef90 con 0x7fe95c0c88c0 2024-03-06T07:32:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.467+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0b02e0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.468+0000 7fe9597fa640 1 -- 172.21.15.26:0/1890986481 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe950018070 con 0x7fe95c0c88c0 2024-03-06T07:32:20.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.468+0000 7fe9597fa640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94006b4d0 0x7fe94006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.468+0000 7fe9597fa640 1 -- 172.21.15.26:0/1890986481 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe95008a4c0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.468+0000 7fe95bfff640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94006b4d0 0x7fe94006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.469+0000 7fe95bfff640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94006b4d0 0x7fe94006d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe95c152c30 tx=0x7fe948009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:20.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.473+0000 7fe9597fa640 1 -- 172.21.15.26:0/1890986481 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe95c0b02e0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.690+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe95c04f330 con 0x7fe95c0c88c0 2024-03-06T07:32:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.691+0000 7fe9597fa640 1 -- 172.21.15.26:0/1890986481 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fe950058af0 con 0x7fe95c0c88c0 2024-03-06T07:32:20.693 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:20.693 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:20.694 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94006b4d0 msgr2=0x7fe94006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94006b4d0 0x7fe94006d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe95c152c30 tx=0x7fe948009060 comp rx=0 tx=0).stop 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c0c88c0 msgr2=0x7fe95c14be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c0c88c0 0x7fe95c14be50 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c7f0 tx=0x7fe95000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 shutdown_connections 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe94006b4d0 0x7fe94006d990 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c158b60 0x7fe95c1ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c14f360 0x7fe95c1ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 --2- 172.21.15.26:0/1890986481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95c0c88c0 0x7fe95c14be50 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 >> 172.21.15.26:0/1890986481 conn(0x7fe95c0ac570 msgr2=0x7fe95c1533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 shutdown_connections 2024-03-06T07:32:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:20.693+0000 7fe962b7a640 1 -- 172.21.15.26:0/1890986481 wait complete. 2024-03-06T07:32:21.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:21.108 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:22.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:22.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.209+0000 7efc81488640 1 Processor -- start 2024-03-06T07:32:22.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.209+0000 7efc81488640 1 -- start start 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.209+0000 7efc81488640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc7c0c5330 0x7efc7c0c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.209+0000 7efc81488640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc7c159630 con 0x7efc7c152920 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.209+0000 7efc81488640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc7c0c7760 con 0x7efc7c0c8a00 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.209+0000 7efc81488640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7c1587b0 con 0x7efc7c0c5330 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc7c152920 0x7efc7c155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51462/0 (socket says 172.21.15.26:51462) 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7affd640 1 -- 172.21.15.26:0/1917529909 learned_addr learned my addr 172.21.15.26:0/1917529909 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965512926 0 0) 0x7efc7c159630 con 0x7efc7c152920 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc68003520 con 0x7efc7c152920 2024-03-06T07:32:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7b7fe640 1 --2- 172.21.15.26:0/1917529909 >> v2:172.21.15.117:3300/0 conn(0x7efc7c0c5330 0x7efc7c0c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1327563893 0 0) 0x7efc68003520 con 0x7efc7c152920 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 >> v2:172.21.15.117:3300/0 conn(0x7efc7c0c5330 msgr2=0x7efc7c0c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 --2- 172.21.15.26:0/1917529909 >> v2:172.21.15.117:3300/0 conn(0x7efc7c0c5330 0x7efc7c0c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 >> v1:172.21.15.117:6789/0 conn(0x7efc7c0c8a00 legacy=0x7efc7c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c0c7a40 con 0x7efc7c152920 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7efc6c001670 con 0x7efc7c152920 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 >> v1:172.21.15.26:6789/0 conn(0x7efc7c152920 legacy=0x7efc7c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc680039b0 0x7efc68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc680043b0 0x7efc68004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c1587b0 con 0x7efc68005120 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6800c060 con 0x7efc680043b0 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6800c260 con 0x7efc680039b0 2024-03-06T07:32:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7b7fe640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc727fc640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc680043b0 0x7efc68004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.210+0000 7efc7affd640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc680039b0 0x7efc68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.211+0000 7efc7b7fe640 1 -- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc680039b0 msgr2=0x7efc68003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.211+0000 7efc7b7fe640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc680039b0 0x7efc68003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.211+0000 7efc7b7fe640 1 -- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc680043b0 msgr2=0x7efc68004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.211+0000 7efc7b7fe640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc680043b0 0x7efc68004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.211+0000 7efc7b7fe640 1 -- 172.21.15.26:0/1917529909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68003520 con 0x7efc68005120 2024-03-06T07:32:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.212+0000 7efc7b7fe640 1 -- 172.21.15.26:0/1917529909 reap_dead start 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.212+0000 7efc7b7fe640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc6800b9c0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7efc74009fc0 tx=0x7efc68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487e36b78a7d0880 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.212+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74009c10 con 0x7efc68005120 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.212+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7402cca0 con 0x7efc68005120 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.212+0000 7efc7a7fc640 1 -- 172.21.15.26:0/1917529909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74003bd0 con 0x7efc68005120 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.213+0000 7efc81488640 1 -- 172.21.15.26:0/1917529909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 msgr2=0x7efc6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.213+0000 7efc81488640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc6800b9c0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7efc74009fc0 tx=0x7efc68005830 comp rx=0 tx=0).stop 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.213+0000 7efc81488640 1 -- 172.21.15.26:0/1917529909 shutdown_connections 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.213+0000 7efc81488640 1 --2- 172.21.15.26:0/1917529909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc6800b9c0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.213+0000 7efc81488640 1 -- 172.21.15.26:0/1917529909 >> 172.21.15.26:0/1917529909 conn(0x7efc7c0ac560 msgr2=0x7efc7c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.213+0000 7efc81488640 1 -- 172.21.15.26:0/1917529909 shutdown_connections 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.213+0000 7efc81488640 1 -- 172.21.15.26:0/1917529909 wait complete. 2024-03-06T07:32:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.214+0000 7efc81488640 1 Processor -- start 2024-03-06T07:32:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.215+0000 7efc81488640 1 -- start start 2024-03-06T07:32:22.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc81488640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc7c0ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc81488640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc7c147840 0x7efc7c147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc81488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7c146c40 0x7efc7c1470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc81488640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c06d030 con 0x7efc68005120 2024-03-06T07:32:22.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc81488640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c158a00 con 0x7efc7c147840 2024-03-06T07:32:22.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc81488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c0c7390 con 0x7efc7c146c40 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc7c0ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc7c0ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38288/0 (socket says 172.21.15.26:38288) 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 -- 172.21.15.26:0/2311840921 learned_addr learned my addr 172.21.15.26:0/2311840921 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc727fc640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc7c147840 0x7efc7c147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7b7fe640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7c146c40 0x7efc7c1470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 -- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7c146c40 msgr2=0x7efc7c1470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7c146c40 0x7efc7c1470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 -- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc7c147840 msgr2=0x7efc7c147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc7c147840 0x7efc7c147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 -- 172.21.15.26:0/2311840921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c147600 con 0x7efc68005120 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc727fc640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc7c147840 0x7efc7c147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.216+0000 7efc7affd640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc7c0ab8a0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7efc6c004ec0 tx=0x7efc6c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:22.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.217+0000 7efc73fff640 1 -- 172.21.15.26:0/2311840921 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c01a070 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.217+0000 7efc73fff640 1 -- 172.21.15.26:0/2311840921 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c00f7b0 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.217+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c1eeaa0 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.217+0000 7efc73fff640 1 -- 172.21.15.26:0/2311840921 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00fba0 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.217+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c0c1df0 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.218+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7c04f330 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.218+0000 7efc73fff640 1 -- 172.21.15.26:0/2311840921 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc6c01d070 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.219+0000 7efc73fff640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc5c06b4d0 0x7efc5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.219+0000 7efc73fff640 1 -- 172.21.15.26:0/2311840921 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7efc6c089dc0 con 0x7efc68005120 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.219+0000 7efc727fc640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc5c06b4d0 0x7efc5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.219+0000 7efc727fc640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc5c06b4d0 0x7efc5c06d990 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0cb9b0 tx=0x7efc64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.223+0000 7efc73fff640 1 -- 172.21.15.26:0/2311840921 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc6c0589b0 con 0x7efc68005120 2024-03-06T07:32:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.441+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc7c06cda0 con 0x7efc68005120 2024-03-06T07:32:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.442+0000 7efc73fff640 1 -- 172.21.15.26:0/2311840921 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7efc6c0582b0 con 0x7efc68005120 2024-03-06T07:32:22.444 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:22.444 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc5c06b4d0 msgr2=0x7efc5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc5c06b4d0 0x7efc5c06d990 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0cb9b0 tx=0x7efc64009040 comp rx=0 tx=0).stop 2024-03-06T07:32:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 msgr2=0x7efc7c0ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc7c0ab8a0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7efc6c004ec0 tx=0x7efc6c007430 comp rx=0 tx=0).stop 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 shutdown_connections 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7efc5c06b4d0 0x7efc5c06d990 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7c146c40 0x7efc7c1470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc7c147840 0x7efc7c147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 --2- 172.21.15.26:0/2311840921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68005120 0x7efc7c0ab8a0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.443+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 >> 172.21.15.26:0/2311840921 conn(0x7efc7c0ac560 msgr2=0x7efc7c0c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.444+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 shutdown_connections 2024-03-06T07:32:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:22.444+0000 7efc81488640 1 -- 172.21.15.26:0/2311840921 wait complete. 2024-03-06T07:32:22.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:22.859 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:23.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.978+0000 7fad6f320640 1 Processor -- start 2024-03-06T07:32:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.978+0000 7fad6f320640 1 -- start start 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6f320640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad6814f360 0x7fad68155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6f320640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fad68158b60 0x7fad68158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6f320640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680c73d0 con 0x7fad680c88c0 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6f320640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad680c8dd0 con 0x7fad6814f360 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6f320640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad680be2e0 con 0x7fad68158b60 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d0bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad6814f360 0x7fad68155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6c8bb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fad680c88c0 0x7fad680c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51472/0 (socket says 172.21.15.26:51472) 2024-03-06T07:32:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d0bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad6814f360 0x7fad68155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38294/0 (socket says 172.21.15.26:38294) 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6c8bb640 1 -- 172.21.15.26:0/3317857808 learned_addr learned my addr 172.21.15.26:0/3317857808 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d8bd640 1 --2- 172.21.15.26:0/3317857808 >> v2:172.21.15.117:3300/0 conn(0x7fad68158b60 0x7fad68158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d0bc640 1 -- 172.21.15.26:0/3317857808 >> v2:172.21.15.117:3300/0 conn(0x7fad68158b60 msgr2=0x7fad68158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d0bc640 1 --2- 172.21.15.26:0/3317857808 >> v2:172.21.15.117:3300/0 conn(0x7fad68158b60 0x7fad68158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d0bc640 1 -- 172.21.15.26:0/3317857808 >> v1:172.21.15.26:6789/0 conn(0x7fad680c88c0 legacy=0x7fad680c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d0bc640 1 -- 172.21.15.26:0/3317857808 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680c7670 con 0x7fad6814f360 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad67fff640 1 -- 172.21.15.26:0/3317857808 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621937952 0 0) 0x7fad680c73d0 con 0x7fad680c88c0 2024-03-06T07:32:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.979+0000 7fad6d8bd640 1 --2- 172.21.15.26:0/3317857808 >> v2:172.21.15.117:3300/0 conn(0x7fad68158b60 0x7fad68158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.980+0000 7fad6d0bc640 1 --2- 172.21.15.26:0/3317857808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad68155c10 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fad68152d60 tx=0x7fad6002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988202d59d0a7a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.980+0000 7fad67fff640 1 -- 172.21.15.26:0/3317857808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60039070 con 0x7fad6814f360 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.980+0000 7fad67fff640 1 -- 172.21.15.26:0/3317857808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6002cbf0 con 0x7fad6814f360 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.980+0000 7fad67fff640 1 -- 172.21.15.26:0/3317857808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60034440 con 0x7fad6814f360 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 -- 172.21.15.26:0/3317857808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 msgr2=0x7fad68155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 --2- 172.21.15.26:0/3317857808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad68155c10 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fad68152d60 tx=0x7fad6002be40 comp rx=0 tx=0).stop 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 -- 172.21.15.26:0/3317857808 shutdown_connections 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 --2- 172.21.15.26:0/3317857808 >> v2:172.21.15.117:3300/0 conn(0x7fad68158b60 0x7fad68158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 --2- 172.21.15.26:0/3317857808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad68155c10 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 -- 172.21.15.26:0/3317857808 >> 172.21.15.26:0/3317857808 conn(0x7fad680ac570 msgr2=0x7fad680ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 -- 172.21.15.26:0/3317857808 shutdown_connections 2024-03-06T07:32:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.981+0000 7fad6f320640 1 -- 172.21.15.26:0/3317857808 wait complete. 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.983+0000 7fad6f320640 1 Processor -- start 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.983+0000 7fad6f320640 1 -- start start 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6f320640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad680c88c0 0x7fad680c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6f320640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad6814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6f320640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad68158b60 0x7fad6814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6f320640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680c5a50 con 0x7fad6814f360 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6f320640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680c7c30 con 0x7fad68158b60 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6f320640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad680c5bd0 con 0x7fad680c88c0 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad6814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad6814ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38298/0 (socket says 172.21.15.26:38298) 2024-03-06T07:32:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 -- 172.21.15.26:0/1502139359 learned_addr learned my addr 172.21.15.26:0/1502139359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6d0bc640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad680c88c0 0x7fad680c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6d8bd640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad68158b60 0x7fad6814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 -- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad680c88c0 msgr2=0x7fad680c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad680c88c0 0x7fad680c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 -- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad68158b60 msgr2=0x7fad6814b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad68158b60 0x7fad6814b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6c8bb640 1 -- 172.21.15.26:0/1502139359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad681eb650 con 0x7fad6814f360 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.984+0000 7fad6d0bc640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad680c88c0 0x7fad680c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.985+0000 7fad6c8bb640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad6814ae50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c440 tx=0x7fad5c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.985+0000 7fad65ffb640 1 -- 172.21.15.26:0/1502139359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c018070 con 0x7fad6814f360 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.985+0000 7fad65ffb640 1 -- 172.21.15.26:0/1502139359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c0074e0 con 0x7fad6814f360 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.985+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad681eb930 con 0x7fad6814f360 2024-03-06T07:32:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.985+0000 7fad65ffb640 1 -- 172.21.15.26:0/1502139359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c004a20 con 0x7fad6814f360 2024-03-06T07:32:23.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.985+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad680bea20 con 0x7fad6814f360 2024-03-06T07:32:23.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.986+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad680bec20 con 0x7fad6814f360 2024-03-06T07:32:23.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.986+0000 7fad65ffb640 1 -- 172.21.15.26:0/1502139359 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad5c01b070 con 0x7fad6814f360 2024-03-06T07:32:23.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.987+0000 7fad65ffb640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad5406b4d0 0x7fad5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:23.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.987+0000 7fad65ffb640 1 -- 172.21.15.26:0/1502139359 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fad5c0896a0 con 0x7fad6814f360 2024-03-06T07:32:23.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.987+0000 7fad6d0bc640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad5406b4d0 0x7fad5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:23.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.988+0000 7fad6d0bc640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad5406b4d0 0x7fad5406d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fad60036000 tx=0x7fad60030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:23.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:23.991+0000 7fad65ffb640 1 -- 172.21.15.26:0/1502139359 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fad680bec20 con 0x7fad6814f360 2024-03-06T07:32:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.210+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad6814ebc0 con 0x7fad6814f360 2024-03-06T07:32:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.211+0000 7fad65ffb640 1 -- 172.21.15.26:0/1502139359 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fad6814ebc0 con 0x7fad6814f360 2024-03-06T07:32:24.213 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:24.214 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:24.214 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad5406b4d0 msgr2=0x7fad5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad5406b4d0 0x7fad5406d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fad60036000 tx=0x7fad60030000 comp rx=0 tx=0).stop 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 msgr2=0x7fad6814ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad6814ae50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c440 tx=0x7fad5c00c900 comp rx=0 tx=0).stop 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 shutdown_connections 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fad5406b4d0 0x7fad5406d990 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad68158b60 0x7fad6814b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6814f360 0x7fad6814ae50 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 --2- 172.21.15.26:0/1502139359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad680c88c0 0x7fad680c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 >> 172.21.15.26:0/1502139359 conn(0x7fad680ac570 msgr2=0x7fad680ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 shutdown_connections 2024-03-06T07:32:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:24.213+0000 7fad6f320640 1 -- 172.21.15.26:0/1502139359 wait complete. 2024-03-06T07:32:24.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:24.629 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:25.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.728+0000 7f5cfe2ed640 1 Processor -- start 2024-03-06T07:32:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.728+0000 7f5cfe2ed640 1 -- start start 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cfe2ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 0x7f5cf8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cfe2ed640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5cf8158b60 0x7f5cf8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cfe2ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf80c73d0 con 0x7f5cf814f360 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cfe2ed640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf80c8dd0 con 0x7f5cf8158b60 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cfe2ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80be2e0 con 0x7f5cf80c88c0 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cf7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5cf814f360 0x7f5cf80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51482/0 (socket says 172.21.15.26:51482) 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3297354596 learned_addr learned my addr 172.21.15.26:0/3297354596 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.729+0000 7f5cfc88a640 1 --2- 172.21.15.26:0/3297354596 >> v2:172.21.15.117:3300/0 conn(0x7f5cf8158b60 0x7f5cf8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf77fe640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 0x7f5cf8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691906629 0 0) 0x7f5cf80c73d0 con 0x7f5cf814f360 2024-03-06T07:32:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce0003520 con 0x7f5cf814f360 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4191508557 0 0) 0x7f5ce0003520 con 0x7f5cf814f360 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 msgr2=0x7f5cf8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 0x7f5cf8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 >> v2:172.21.15.117:3300/0 conn(0x7f5cf8158b60 msgr2=0x7f5cf8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 --2- 172.21.15.26:0/3297354596 >> v2:172.21.15.117:3300/0 conn(0x7f5cf8158b60 0x7f5cf8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80c7670 con 0x7f5cf814f360 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f5ce8001670 con 0x7f5cf814f360 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 >> v1:172.21.15.26:6789/0 conn(0x7f5cf814f360 legacy=0x7f5cf80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5ce0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 0x7f5ce0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0004d40 0x7f5ce00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80be2e0 con 0x7f5ce00039b0 2024-03-06T07:32:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce000b9c0 con 0x7f5ce0004d40 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce000bb40 con 0x7f5cf80c88c0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf7fff640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5ce0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf77fe640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 0x7f5ce0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cfc88a640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0004d40 0x7f5ce00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 msgr2=0x7f5ce0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf7fff640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80c88c0 0x7f5ce0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0004d40 msgr2=0x7f5ce00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf7fff640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0004d40 0x7f5ce00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3297354596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce0003520 con 0x7f5ce00039b0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.730+0000 7f5cfc88a640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0004d40 0x7f5ce00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.731+0000 7f5cf7fff640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5ce0003dd0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c210 tx=0x7f5ce0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b8b015497f453e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8007d50 con 0x7f5ce00039b0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8030070 con 0x7f5ce00039b0 2024-03-06T07:32:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cf6ffd640 1 -- 172.21.15.26:0/3297354596 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce802cd80 con 0x7f5ce00039b0 2024-03-06T07:32:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3297354596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 msgr2=0x7f5ce0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cfe2ed640 1 --2- 172.21.15.26:0/3297354596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5ce0003dd0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c210 tx=0x7f5ce0005830 comp rx=0 tx=0).stop 2024-03-06T07:32:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cfc88a640 1 -- 172.21.15.26:0/3297354596 reap_dead start 2024-03-06T07:32:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3297354596 shutdown_connections 2024-03-06T07:32:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3297354596 >> 172.21.15.26:0/3297354596 conn(0x7f5cf80ac570 msgr2=0x7f5cf80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3297354596 shutdown_connections 2024-03-06T07:32:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.732+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3297354596 wait complete. 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.733+0000 7f5cfe2ed640 1 Processor -- start 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.733+0000 7f5cfe2ed640 1 -- start start 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cfe2ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5cf814bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cfe2ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cf814c2c0 0x7f5cf81edd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cfe2ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80ad390 0x7f5cf81ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cfe2ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80c5a50 con 0x7f5ce00039b0 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cfe2ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80c7c30 con 0x7f5cf814c2c0 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cfe2ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80c5bd0 con 0x7f5cf80ad390 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5cf814bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5cf814bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38316/0 (socket says 172.21.15.26:38316) 2024-03-06T07:32:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3939751499 learned_addr learned my addr 172.21.15.26:0/3939751499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cfc88a640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80ad390 0x7f5cf81ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf77fe640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cf814c2c0 0x7f5cf81edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80ad390 msgr2=0x7f5cf81ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80ad390 0x7f5cf81ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cf814c2c0 msgr2=0x7f5cf81edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cf814c2c0 0x7f5cf81edd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.734+0000 7f5cf7fff640 1 -- 172.21.15.26:0/3939751499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80c09f0 con 0x7f5ce00039b0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.735+0000 7f5cf77fe640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cf814c2c0 0x7f5cf81edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.735+0000 7f5cf7fff640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5cf814bd80 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f5ce802c590 tx=0x7f5ce8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.735+0000 7f5cf4ff9640 1 -- 172.21.15.26:0/3939751499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8045070 con 0x7f5ce00039b0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.735+0000 7f5cf4ff9640 1 -- 172.21.15.26:0/3939751499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8004bf0 con 0x7f5ce00039b0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.735+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf80c0c90 con 0x7f5ce00039b0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.735+0000 7f5cf4ff9640 1 -- 172.21.15.26:0/3939751499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8001c90 con 0x7f5ce00039b0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.735+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf81ee9a0 con 0x7f5ce00039b0 2024-03-06T07:32:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.736+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf804f330 con 0x7f5ce00039b0 2024-03-06T07:32:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.736+0000 7f5cf4ff9640 1 -- 172.21.15.26:0/3939751499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ce8007070 con 0x7f5ce00039b0 2024-03-06T07:32:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.737+0000 7f5cf4ff9640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ce406b4d0 0x7f5ce406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.737+0000 7f5cf4ff9640 1 -- 172.21.15.26:0/3939751499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f5ce80ab650 con 0x7f5ce00039b0 2024-03-06T07:32:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.737+0000 7f5cf77fe640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ce406b4d0 0x7f5ce406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.738+0000 7f5cf77fe640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ce406b4d0 0x7f5ce406d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5cf81e8300 tx=0x7f5cec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.745+0000 7f5cf4ff9640 1 -- 172.21.15.26:0/3939751499 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ce8079b60 con 0x7f5ce00039b0 2024-03-06T07:32:25.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.964+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cf806cda0 con 0x7f5ce00039b0 2024-03-06T07:32:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.964+0000 7f5cf4ff9640 1 -- 172.21.15.26:0/3939751499 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f5ce807dc00 con 0x7f5ce00039b0 2024-03-06T07:32:25.967 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:25.967 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ce406b4d0 msgr2=0x7f5ce406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ce406b4d0 0x7f5ce406d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5cf81e8300 tx=0x7f5cec008040 comp rx=0 tx=0).stop 2024-03-06T07:32:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 msgr2=0x7f5cf814bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5cf814bd80 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f5ce802c590 tx=0x7f5ce8001a40 comp rx=0 tx=0).stop 2024-03-06T07:32:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 shutdown_connections 2024-03-06T07:32:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5ce406b4d0 0x7f5ce406d990 secure :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5cf81e8300 tx=0x7f5cec008040 comp rx=0 tx=0).stop 2024-03-06T07:32:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5cf80ad390 0x7f5cf81ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cf814c2c0 0x7f5cf81edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 --2- 172.21.15.26:0/3939751499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ce00039b0 0x7f5cf814bd80 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 >> 172.21.15.26:0/3939751499 conn(0x7f5cf80ac570 msgr2=0x7f5cf806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 shutdown_connections 2024-03-06T07:32:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:25.966+0000 7f5cfe2ed640 1 -- 172.21.15.26:0/3939751499 wait complete. 2024-03-06T07:32:26.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:26.383 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:27.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:27.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.484+0000 7fc5c0f56640 1 Processor -- start 2024-03-06T07:32:27.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.484+0000 7fc5c0f56640 1 -- start start 2024-03-06T07:32:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5c0f56640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5bc0c88c0 0x7fc5bc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5c0f56640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5bc158b60 0x7fc5bc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5c0f56640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc0c73d0 con 0x7fc5bc14f360 2024-03-06T07:32:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5c0f56640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc0c8dd0 con 0x7fc5bc158b60 2024-03-06T07:32:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5c0f56640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc0be2e0 con 0x7fc5bc0c88c0 2024-03-06T07:32:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5bad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5bc158b60 0x7fc5bc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5bad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5bc158b60 0x7fc5bc158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59024/0 (socket says 172.21.15.26:59024) 2024-03-06T07:32:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4275909464 learned_addr learned my addr 172.21.15.26:0/4275909464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5b9d9b640 1 --2- 172.21.15.26:0/4275909464 >> v2:172.21.15.117:3300/0 conn(0x7fc5bc0c88c0 0x7fc5bc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4275909464 >> v2:172.21.15.117:3300/0 conn(0x7fc5bc0c88c0 msgr2=0x7fc5bc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5bad9d640 1 --2- 172.21.15.26:0/4275909464 >> v2:172.21.15.117:3300/0 conn(0x7fc5bc0c88c0 0x7fc5bc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4275909464 >> v1:172.21.15.117:6789/0 conn(0x7fc5bc14f360 legacy=0x7fc5bc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.485+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4275909464 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0c7670 con 0x7fc5bc158b60 2024-03-06T07:32:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.486+0000 7fc5bad9d640 1 --2- 172.21.15.26:0/4275909464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc158ff0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0c84c0 tx=0x7fc5b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0fdc859a6f63b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.486+0000 7fc5b959a640 1 -- 172.21.15.26:0/4275909464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b402ca80 con 0x7fc5bc158b60 2024-03-06T07:32:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.486+0000 7fc5b959a640 1 -- 172.21.15.26:0/4275909464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b402cc20 con 0x7fc5bc158b60 2024-03-06T07:32:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.486+0000 7fc5b959a640 1 -- 172.21.15.26:0/4275909464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40343f0 con 0x7fc5bc158b60 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4275909464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 msgr2=0x7fc5bc158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4275909464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc158ff0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0c84c0 tx=0x7fc5b402bee0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4275909464 shutdown_connections 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4275909464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc158ff0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4275909464 >> v2:172.21.15.117:3300/0 conn(0x7fc5bc0c88c0 0x7fc5bc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4275909464 >> 172.21.15.26:0/4275909464 conn(0x7fc5bc0ac570 msgr2=0x7fc5bc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4275909464 shutdown_connections 2024-03-06T07:32:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.487+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4275909464 wait complete. 2024-03-06T07:32:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.489+0000 7fc5c0f56640 1 Processor -- start 2024-03-06T07:32:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.489+0000 7fc5c0f56640 1 -- start start 2024-03-06T07:32:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5c0f56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5bc0c88c0 0x7fc5bc1eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5c0f56640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc14f360 0x7fc5bc0c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5c0f56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc1519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5c0f56640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0c5a50 con 0x7fc5bc158b60 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5c0f56640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0c7c30 con 0x7fc5bc14f360 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5c0f56640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0c5bd0 con 0x7fc5bc0c88c0 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc1519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc1519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59034/0 (socket says 172.21.15.26:59034) 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4103526187 learned_addr learned my addr 172.21.15.26:0/4103526187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5b9d9b640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc14f360 0x7fc5bc0c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5ba59c640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5bc0c88c0 0x7fc5bc1eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5bc0c88c0 msgr2=0x7fc5bc1eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5bc0c88c0 0x7fc5bc1eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc14f360 msgr2=0x7fc5bc0c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc14f360 0x7fc5bc0c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 -- 172.21.15.26:0/4103526187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc14f990 con 0x7fc5bc158b60 2024-03-06T07:32:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.490+0000 7fc5bad9d640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc1519c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009f40 tx=0x7fc5b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.491+0000 7fc5aaffd640 1 -- 172.21.15.26:0/4103526187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4043020 con 0x7fc5bc158b60 2024-03-06T07:32:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.491+0000 7fc5aaffd640 1 -- 172.21.15.26:0/4103526187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b403e450 con 0x7fc5bc158b60 2024-03-06T07:32:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.491+0000 7fc5aaffd640 1 -- 172.21.15.26:0/4103526187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4009510 con 0x7fc5bc158b60 2024-03-06T07:32:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.491+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc14fc70 con 0x7fc5bc158b60 2024-03-06T07:32:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.491+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc150230 con 0x7fc5bc158b60 2024-03-06T07:32:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.492+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc04f330 con 0x7fc5bc158b60 2024-03-06T07:32:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.492+0000 7fc5aaffd640 1 -- 172.21.15.26:0/4103526187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5b4046020 con 0x7fc5bc158b60 2024-03-06T07:32:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.493+0000 7fc5aaffd640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59c06b4d0 0x7fc59c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.493+0000 7fc5aaffd640 1 -- 172.21.15.26:0/4103526187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc5b40ac8e0 con 0x7fc5bc158b60 2024-03-06T07:32:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.493+0000 7fc5ba59c640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59c06b4d0 0x7fc59c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.493+0000 7fc5ba59c640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59c06b4d0 0x7fc59c06d990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac007fd0 tx=0x7fc5ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.497+0000 7fc5aaffd640 1 -- 172.21.15.26:0/4103526187 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5b404d070 con 0x7fc5bc158b60 2024-03-06T07:32:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.715+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5bc0acaf0 con 0x7fc5bc158b60 2024-03-06T07:32:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.716+0000 7fc5aaffd640 1 -- 172.21.15.26:0/4103526187 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc5b407b560 con 0x7fc5bc158b60 2024-03-06T07:32:27.718 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:27.718 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.717+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59c06b4d0 msgr2=0x7fc59c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.717+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59c06b4d0 0x7fc59c06d990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac007fd0 tx=0x7fc5ac009040 comp rx=0 tx=0).stop 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.717+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 msgr2=0x7fc5bc1519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.717+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc1519c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009f40 tx=0x7fc5b4004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 shutdown_connections 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc59c06b4d0 0x7fc59c06d990 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc158b60 0x7fc5bc1519c0 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc14f360 0x7fc5bc0c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 --2- 172.21.15.26:0/4103526187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5bc0c88c0 0x7fc5bc1eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 >> 172.21.15.26:0/4103526187 conn(0x7fc5bc0ac570 msgr2=0x7fc5bc0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 shutdown_connections 2024-03-06T07:32:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:27.718+0000 7fc5c0f56640 1 -- 172.21.15.26:0/4103526187 wait complete. 2024-03-06T07:32:28.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:28.134 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:29.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.234+0000 7f5d9a165640 1 Processor -- start 2024-03-06T07:32:29.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.234+0000 7f5d9a165640 1 -- start start 2024-03-06T07:32:29.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.234+0000 7f5d9a165640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d9414ec00 0x7f5d9414f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:29.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.234+0000 7f5d9a165640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5d941550c0 0x7f5d94155550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:29.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.234+0000 7f5d9a165640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9414f5d0 con 0x7f5d940aa180 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.234+0000 7f5d9a165640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d940c4610 con 0x7f5d9414ec00 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.234+0000 7f5d9a165640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d940c38c0 con 0x7f5d941550c0 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d92ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d9414ec00 0x7f5d9414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d92ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d9414ec00 0x7f5d9414f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59050/0 (socket says 172.21.15.26:59050) 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d92ffd640 1 -- 172.21.15.26:0/2024697959 learned_addr learned my addr 172.21.15.26:0/2024697959 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d93fff640 1 --2- 172.21.15.26:0/2024697959 >> v2:172.21.15.117:3300/0 conn(0x7f5d941550c0 0x7f5d94155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d92ffd640 1 -- 172.21.15.26:0/2024697959 >> v2:172.21.15.117:3300/0 conn(0x7f5d941550c0 msgr2=0x7f5d94155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d92ffd640 1 --2- 172.21.15.26:0/2024697959 >> v2:172.21.15.117:3300/0 conn(0x7f5d941550c0 0x7f5d94155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d92ffd640 1 -- 172.21.15.26:0/2024697959 >> v1:172.21.15.117:6789/0 conn(0x7f5d940aa180 legacy=0x7f5d940c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d92ffd640 1 -- 172.21.15.26:0/2024697959 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940aa660 con 0x7f5d9414ec00 2024-03-06T07:32:29.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.235+0000 7f5d93fff640 1 --2- 172.21.15.26:0/2024697959 >> v2:172.21.15.117:3300/0 conn(0x7f5d941550c0 0x7f5d94155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.236+0000 7f5d92ffd640 1 --2- 172.21.15.26:0/2024697959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d9414f090 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f5d940aa560 tx=0x7f5d8802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beebff15f3c2cf63 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:29.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.236+0000 7f5d927fc640 1 -- 172.21.15.26:0/2024697959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88038070 con 0x7f5d9414ec00 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.236+0000 7f5d927fc640 1 -- 172.21.15.26:0/2024697959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8802f050 con 0x7f5d9414ec00 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.236+0000 7f5d927fc640 1 -- 172.21.15.26:0/2024697959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88031970 con 0x7f5d9414ec00 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 -- 172.21.15.26:0/2024697959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 msgr2=0x7f5d9414f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 --2- 172.21.15.26:0/2024697959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d9414f090 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f5d940aa560 tx=0x7f5d8802b7f0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 -- 172.21.15.26:0/2024697959 shutdown_connections 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 --2- 172.21.15.26:0/2024697959 >> v2:172.21.15.117:3300/0 conn(0x7f5d941550c0 0x7f5d94155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 --2- 172.21.15.26:0/2024697959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d9414f090 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 -- 172.21.15.26:0/2024697959 >> 172.21.15.26:0/2024697959 conn(0x7f5d940c6710 msgr2=0x7f5d940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 -- 172.21.15.26:0/2024697959 shutdown_connections 2024-03-06T07:32:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.237+0000 7f5d9a165640 1 -- 172.21.15.26:0/2024697959 wait complete. 2024-03-06T07:32:29.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.238+0000 7f5d9a165640 1 Processor -- start 2024-03-06T07:32:29.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.239+0000 7f5d9a165640 1 -- start start 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d9a165640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d940aa180 0x7f5d94156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d9a165640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d940a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d9a165640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d941550c0 0x7f5d94148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d9a165640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94155a90 con 0x7f5d9414ec00 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d9a165640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940c4860 con 0x7f5d940aa180 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d9a165640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94155c10 con 0x7f5d941550c0 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d940a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d940a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59064/0 (socket says 172.21.15.26:59064) 2024-03-06T07:32:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 -- 172.21.15.26:0/791527942 learned_addr learned my addr 172.21.15.26:0/791527942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d937fe640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d940aa180 0x7f5d94156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d93fff640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d941550c0 0x7f5d94148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 -- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d941550c0 msgr2=0x7f5d94148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d941550c0 0x7f5d94148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 -- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d940aa180 msgr2=0x7f5d94156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d940aa180 0x7f5d94156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 -- 172.21.15.26:0/791527942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940a5eb0 con 0x7f5d9414ec00 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d93fff640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d941550c0 0x7f5d94148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d92ffd640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d940a6a80 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f5d8802b830 tx=0x7f5d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d7bfff640 1 -- 172.21.15.26:0/791527942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88042070 con 0x7f5d9414ec00 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d7bfff640 1 -- 172.21.15.26:0/791527942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8803d890 con 0x7f5d9414ec00 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.240+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d940723e0 con 0x7f5d9414ec00 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.241+0000 7f5d7bfff640 1 -- 172.21.15.26:0/791527942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88038040 con 0x7f5d9414ec00 2024-03-06T07:32:29.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.241+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d940729a0 con 0x7f5d9414ec00 2024-03-06T07:32:29.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.242+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9404f330 con 0x7f5d9414ec00 2024-03-06T07:32:29.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.242+0000 7f5d7bfff640 1 -- 172.21.15.26:0/791527942 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d88045070 con 0x7f5d9414ec00 2024-03-06T07:32:29.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.243+0000 7f5d7bfff640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d8006b4d0 0x7f5d8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:29.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.243+0000 7f5d7bfff640 1 -- 172.21.15.26:0/791527942 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f5d880ac2e0 con 0x7f5d9414ec00 2024-03-06T07:32:29.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.243+0000 7f5d937fe640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d8006b4d0 0x7f5d8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:29.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.243+0000 7f5d937fe640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d8006b4d0 0x7f5d8006d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5d84006fd0 tx=0x7f5d84009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.247+0000 7f5d7bfff640 1 -- 172.21.15.26:0/791527942 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d8807a7f0 con 0x7f5d9414ec00 2024-03-06T07:32:29.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.466+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d940c0df0 con 0x7f5d9414ec00 2024-03-06T07:32:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.467+0000 7f5d7bfff640 1 -- 172.21.15.26:0/791527942 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f5d8807e890 con 0x7f5d9414ec00 2024-03-06T07:32:29.470 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:29.470 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d8006b4d0 msgr2=0x7f5d8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d8006b4d0 0x7f5d8006d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5d84006fd0 tx=0x7f5d84009060 comp rx=0 tx=0).stop 2024-03-06T07:32:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 msgr2=0x7f5d940a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d940a6a80 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f5d8802b830 tx=0x7f5d880046c0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 shutdown_connections 2024-03-06T07:32:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f5d8006b4d0 0x7f5d8006d990 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d941550c0 0x7f5d94148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9414ec00 0x7f5d940a6a80 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 --2- 172.21.15.26:0/791527942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d940aa180 0x7f5d94156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 >> 172.21.15.26:0/791527942 conn(0x7f5d940c6710 msgr2=0x7f5d94152b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 shutdown_connections 2024-03-06T07:32:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:29.469+0000 7f5d9a165640 1 -- 172.21.15.26:0/791527942 wait complete. 2024-03-06T07:32:29.889 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:29.889 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:30.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.990+0000 7fc0279b6640 1 Processor -- start 2024-03-06T07:32:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.990+0000 7fc0279b6640 1 -- start start 2024-03-06T07:32:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc0279b6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc0200c88c0 0x7fc020155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc0279b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc020158b60 0x7fc020158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc0279b6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0200c73d0 con 0x7fc02014f360 2024-03-06T07:32:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc0279b6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0200c8dd0 con 0x7fc0200c88c0 2024-03-06T07:32:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc0279b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200be2e0 con 0x7fc020158b60 2024-03-06T07:32:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc025f53640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc020158b60 0x7fc020158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc024f51640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc0200c88c0 0x7fc020155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc025752640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc02014f360 0x7fc0200c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:54022/0 (socket says 172.21.15.26:54022) 2024-03-06T07:32:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc025752640 1 -- 172.21.15.26:0/1432647478 learned_addr learned my addr 172.21.15.26:0/1432647478 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc017fff640 1 -- 172.21.15.26:0/1432647478 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860923411 0 0) 0x7fc0200c73d0 con 0x7fc02014f360 2024-03-06T07:32:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc025f53640 1 -- 172.21.15.26:0/1432647478 >> v2:172.21.15.117:3300/0 conn(0x7fc0200c88c0 msgr2=0x7fc020155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc025f53640 1 --2- 172.21.15.26:0/1432647478 >> v2:172.21.15.117:3300/0 conn(0x7fc0200c88c0 0x7fc020155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc025f53640 1 -- 172.21.15.26:0/1432647478 >> v1:172.21.15.117:6789/0 conn(0x7fc02014f360 legacy=0x7fc0200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.991+0000 7fc025f53640 1 -- 172.21.15.26:0/1432647478 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0200c7670 con 0x7fc020158b60 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.992+0000 7fc025f53640 1 --2- 172.21.15.26:0/1432647478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc020158b60 0x7fc020158ff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc018009d90 tx=0x7fc01802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f183d72e481d6d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.992+0000 7fc017fff640 1 -- 172.21.15.26:0/1432647478 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018039070 con 0x7fc020158b60 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc017fff640 1 -- 172.21.15.26:0/1432647478 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01802cbc0 con 0x7fc020158b60 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc017fff640 1 -- 172.21.15.26:0/1432647478 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018034440 con 0x7fc020158b60 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 -- 172.21.15.26:0/1432647478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc020158b60 msgr2=0x7fc020158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 --2- 172.21.15.26:0/1432647478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc020158b60 0x7fc020158ff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc018009d90 tx=0x7fc01802be10 comp rx=0 tx=0).stop 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 -- 172.21.15.26:0/1432647478 shutdown_connections 2024-03-06T07:32:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 --2- 172.21.15.26:0/1432647478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc020158b60 0x7fc020158ff0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 --2- 172.21.15.26:0/1432647478 >> v2:172.21.15.117:3300/0 conn(0x7fc0200c88c0 0x7fc020155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 -- 172.21.15.26:0/1432647478 >> 172.21.15.26:0/1432647478 conn(0x7fc0200ac570 msgr2=0x7fc0200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 -- 172.21.15.26:0/1432647478 shutdown_connections 2024-03-06T07:32:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.993+0000 7fc0279b6640 1 -- 172.21.15.26:0/1432647478 wait complete. 2024-03-06T07:32:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.994+0000 7fc0279b6640 1 Processor -- start 2024-03-06T07:32:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.994+0000 7fc0279b6640 1 -- start start 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc0279b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0200c88c0 0x7fc0201ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc0279b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02014f360 0x7fc02014dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc0279b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020158b60 0x7fc0201ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc0279b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c5a50 con 0x7fc020158b60 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc0279b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c7c30 con 0x7fc02014f360 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc0279b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c5bd0 con 0x7fc0200c88c0 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020158b60 0x7fc0201ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020158b60 0x7fc0201ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59076/0 (socket says 172.21.15.26:59076) 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 -- 172.21.15.26:0/3676275347 learned_addr learned my addr 172.21.15.26:0/3676275347 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025752640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0200c88c0 0x7fc0201ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc024f51640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02014f360 0x7fc02014dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 -- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0200c88c0 msgr2=0x7fc0201ea5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0200c88c0 0x7fc0201ea5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 -- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02014f360 msgr2=0x7fc02014dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02014f360 0x7fc02014dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc025f53640 1 -- 172.21.15.26:0/3676275347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0201eabf0 con 0x7fc020158b60 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.995+0000 7fc024f51640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02014f360 0x7fc02014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.996+0000 7fc025f53640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020158b60 0x7fc0201ee170 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fc018009d90 tx=0x7fc018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.996+0000 7fc015ffb640 1 -- 172.21.15.26:0/3676275347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018039050 con 0x7fc020158b60 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.996+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0201eaed0 con 0x7fc020158b60 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.996+0000 7fc015ffb640 1 -- 172.21.15.26:0/3676275347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01803a040 con 0x7fc020158b60 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.996+0000 7fc015ffb640 1 -- 172.21.15.26:0/3676275347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018009450 con 0x7fc020158b60 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.996+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0201edc50 con 0x7fc020158b60 2024-03-06T07:32:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.997+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02004f330 con 0x7fc020158b60 2024-03-06T07:32:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.998+0000 7fc015ffb640 1 -- 172.21.15.26:0/3676275347 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc018047030 con 0x7fc020158b60 2024-03-06T07:32:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.998+0000 7fc015ffb640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc00406b4d0 0x7fc00406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.998+0000 7fc015ffb640 1 -- 172.21.15.26:0/3676275347 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc0180ac5d0 con 0x7fc020158b60 2024-03-06T07:32:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.998+0000 7fc025752640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc00406b4d0 0x7fc00406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:30.999+0000 7fc025752640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc00406b4d0 0x7fc00406d990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f60 tx=0x7fc01c007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:31.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.007+0000 7fc015ffb640 1 -- 172.21.15.26:0/3676275347 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc01807b1d0 con 0x7fc020158b60 2024-03-06T07:32:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.228+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc02006cda0 con 0x7fc020158b60 2024-03-06T07:32:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.229+0000 7fc015ffb640 1 -- 172.21.15.26:0/3676275347 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc01807aad0 con 0x7fc020158b60 2024-03-06T07:32:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:31.232 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:31.233 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc00406b4d0 msgr2=0x7fc00406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc00406b4d0 0x7fc00406d990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f60 tx=0x7fc01c007b50 comp rx=0 tx=0).stop 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020158b60 msgr2=0x7fc0201ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020158b60 0x7fc0201ee170 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fc018009d90 tx=0x7fc018004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 shutdown_connections 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc00406b4d0 0x7fc00406d990 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020158b60 0x7fc0201ee170 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02014f360 0x7fc02014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 --2- 172.21.15.26:0/3676275347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0200c88c0 0x7fc0201ea5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 >> 172.21.15.26:0/3676275347 conn(0x7fc0200ac570 msgr2=0x7fc02006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 shutdown_connections 2024-03-06T07:32:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:31.231+0000 7fc0279b6640 1 -- 172.21.15.26:0/3676275347 wait complete. 2024-03-06T07:32:31.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:31.647 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:32.648 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.748+0000 7faf9af65640 1 Processor -- start 2024-03-06T07:32:32.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.748+0000 7faf9af65640 1 -- start start 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf9af65640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf941506b0 0x7faf94153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf9af65640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf94156ea0 0x7faf940702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf9af65640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf94070800 con 0x7faf940acc00 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf9af65640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf94156450 con 0x7faf941506b0 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf9af65640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf94156a10 con 0x7faf94156ea0 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf98d01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf941506b0 0x7faf94153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf98d01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf941506b0 0x7faf94153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59090/0 (socket says 172.21.15.26:59090) 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf98d01640 1 -- 172.21.15.26:0/2499415237 learned_addr learned my addr 172.21.15.26:0/2499415237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf93fff640 1 --2- 172.21.15.26:0/2499415237 >> v2:172.21.15.117:3300/0 conn(0x7faf94156ea0 0x7faf940702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf98d01640 1 -- 172.21.15.26:0/2499415237 >> v2:172.21.15.117:3300/0 conn(0x7faf94156ea0 msgr2=0x7faf940702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf98d01640 1 --2- 172.21.15.26:0/2499415237 >> v2:172.21.15.117:3300/0 conn(0x7faf94156ea0 0x7faf940702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf98d01640 1 -- 172.21.15.26:0/2499415237 >> v1:172.21.15.117:6789/0 conn(0x7faf940acc00 legacy=0x7faf940c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.749+0000 7faf98d01640 1 -- 172.21.15.26:0/2499415237 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf94070aa0 con 0x7faf941506b0 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.750+0000 7faf98d01640 1 --2- 172.21.15.26:0/2499415237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941506b0 0x7faf94153990 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7faf941565d0 tx=0x7faf8802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96c7b269c646042 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.750+0000 7faf937fe640 1 -- 172.21.15.26:0/2499415237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88034070 con 0x7faf941506b0 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.750+0000 7faf937fe640 1 -- 172.21.15.26:0/2499415237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf88030860 con 0x7faf941506b0 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.750+0000 7faf937fe640 1 -- 172.21.15.26:0/2499415237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88030c30 con 0x7faf941506b0 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 -- 172.21.15.26:0/2499415237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941506b0 msgr2=0x7faf94153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 --2- 172.21.15.26:0/2499415237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941506b0 0x7faf94153990 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7faf941565d0 tx=0x7faf8802b9a0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 -- 172.21.15.26:0/2499415237 shutdown_connections 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 --2- 172.21.15.26:0/2499415237 >> v2:172.21.15.117:3300/0 conn(0x7faf94156ea0 0x7faf940702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 --2- 172.21.15.26:0/2499415237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941506b0 0x7faf94153990 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 -- 172.21.15.26:0/2499415237 >> 172.21.15.26:0/2499415237 conn(0x7faf9406b1a0 msgr2=0x7faf9406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:32.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 -- 172.21.15.26:0/2499415237 shutdown_connections 2024-03-06T07:32:32.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.751+0000 7faf9af65640 1 -- 172.21.15.26:0/2499415237 wait complete. 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.752+0000 7faf9af65640 1 Processor -- start 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.752+0000 7faf9af65640 1 -- start start 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf9af65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf940acc00 0x7faf941e9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf9af65640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf941506b0 0x7faf941557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf9af65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf94156ea0 0x7faf94146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf9af65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf940c5bb0 con 0x7faf94156ea0 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf9af65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf941566a0 con 0x7faf940acc00 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf9af65640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf940c5d30 con 0x7faf941506b0 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf94156ea0 0x7faf94146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf94156ea0 0x7faf94146cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59100/0 (socket says 172.21.15.26:59100) 2024-03-06T07:32:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 -- 172.21.15.26:0/1877085018 learned_addr learned my addr 172.21.15.26:0/1877085018 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf98d01640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf940acc00 0x7faf941e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf93fff640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf941506b0 0x7faf941557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 -- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf941506b0 msgr2=0x7faf941557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf941506b0 0x7faf941557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 -- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf940acc00 msgr2=0x7faf941e9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf940acc00 0x7faf941e9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.753+0000 7faf99502640 1 -- 172.21.15.26:0/1877085018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9414cb90 con 0x7faf94156ea0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.754+0000 7faf99502640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf94156ea0 0x7faf94146cd0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c730 tx=0x7faf8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.754+0000 7faf917fa640 1 -- 172.21.15.26:0/1877085018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c003b70 con 0x7faf94156ea0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.754+0000 7faf917fa640 1 -- 172.21.15.26:0/1877085018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c003d30 con 0x7faf94156ea0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.754+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9414ce70 con 0x7faf94156ea0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.754+0000 7faf917fa640 1 -- 172.21.15.26:0/1877085018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c005310 con 0x7faf94156ea0 2024-03-06T07:32:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.754+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9414d780 con 0x7faf94156ea0 2024-03-06T07:32:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.755+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9404f330 con 0x7faf94156ea0 2024-03-06T07:32:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.755+0000 7faf917fa640 1 -- 172.21.15.26:0/1877085018 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf8c0056c0 con 0x7faf94156ea0 2024-03-06T07:32:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.756+0000 7faf917fa640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf7806b4d0 0x7faf7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.756+0000 7faf917fa640 1 -- 172.21.15.26:0/1877085018 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7faf8c08a330 con 0x7faf94156ea0 2024-03-06T07:32:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.756+0000 7faf98d01640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf7806b4d0 0x7faf7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.757+0000 7faf98d01640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf7806b4d0 0x7faf7806d990 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7faf88036000 tx=0x7faf88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:32.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.760+0000 7faf917fa640 1 -- 172.21.15.26:0/1877085018 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf8c058960 con 0x7faf94156ea0 2024-03-06T07:32:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.983+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf940c5990 con 0x7faf94156ea0 2024-03-06T07:32:32.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.983+0000 7faf917fa640 1 -- 172.21.15.26:0/1877085018 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7faf8c05ca00 con 0x7faf94156ea0 2024-03-06T07:32:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:32.986 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:32.987 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:32.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf7806b4d0 msgr2=0x7faf7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf7806b4d0 0x7faf7806d990 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7faf88036000 tx=0x7faf88008030 comp rx=0 tx=0).stop 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf94156ea0 msgr2=0x7faf94146cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf94156ea0 0x7faf94146cd0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c730 tx=0x7faf8c00cbf0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 shutdown_connections 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7faf7806b4d0 0x7faf7806d990 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf94156ea0 0x7faf94146cd0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf941506b0 0x7faf941557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 --2- 172.21.15.26:0/1877085018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf940acc00 0x7faf941e9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 >> 172.21.15.26:0/1877085018 conn(0x7faf9406b1a0 msgr2=0x7faf9414e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 shutdown_connections 2024-03-06T07:32:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:32.986+0000 7faf9af65640 1 -- 172.21.15.26:0/1877085018 wait complete. 2024-03-06T07:32:33.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:33.402 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:34.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:34.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.502+0000 7f9da1ed4640 1 Processor -- start 2024-03-06T07:32:34.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.502+0000 7f9da1ed4640 1 -- start start 2024-03-06T07:32:34.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.502+0000 7f9da1ed4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c0c8a00 0x7f9d9c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:34.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.502+0000 7f9da1ed4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d9c159460 0x7f9d9c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.502+0000 7f9da1ed4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9c0c7760 con 0x7f9d9c152920 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.502+0000 7f9da1ed4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9c1587b0 con 0x7f9d9c159460 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.502+0000 7f9da1ed4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c7a40 con 0x7f9d9c0c8a00 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d9c159460 0x7f9d9c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d9c159460 0x7f9d9c0c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59102/0 (socket says 172.21.15.26:59102) 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9bfff640 1 -- 172.21.15.26:0/3395453572 learned_addr learned my addr 172.21.15.26:0/3395453572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9affd640 1 --2- 172.21.15.26:0/3395453572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c0c8a00 0x7f9d9c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9bfff640 1 -- 172.21.15.26:0/3395453572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c0c8a00 msgr2=0x7f9d9c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9bfff640 1 --2- 172.21.15.26:0/3395453572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c0c8a00 0x7f9d9c158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9bfff640 1 -- 172.21.15.26:0/3395453572 >> v1:172.21.15.117:6789/0 conn(0x7f9d9c152920 legacy=0x7f9d9c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.503+0000 7f9d9bfff640 1 -- 172.21.15.26:0/3395453572 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c0c7d20 con 0x7f9d9c159460 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9d9bfff640 1 --2- 172.21.15.26:0/3395453572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c159460 0x7f9d9c0c5330 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009e50 tx=0x7f9d9402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab7ab4695bde91e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9d9a7fc640 1 -- 172.21.15.26:0/3395453572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9402c9e0 con 0x7f9d9c159460 2024-03-06T07:32:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9d9a7fc640 1 -- 172.21.15.26:0/3395453572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9402cb80 con 0x7f9d9c159460 2024-03-06T07:32:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9d9a7fc640 1 -- 172.21.15.26:0/3395453572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940343f0 con 0x7f9d9c159460 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9da1ed4640 1 -- 172.21.15.26:0/3395453572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c159460 msgr2=0x7f9d9c0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/3395453572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c159460 0x7f9d9c0c5330 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009e50 tx=0x7f9d9402be40 comp rx=0 tx=0).stop 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9da1ed4640 1 -- 172.21.15.26:0/3395453572 shutdown_connections 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/3395453572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c159460 0x7f9d9c0c5330 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/3395453572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c0c8a00 0x7f9d9c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.504+0000 7f9da1ed4640 1 -- 172.21.15.26:0/3395453572 >> 172.21.15.26:0/3395453572 conn(0x7f9d9c0ac560 msgr2=0x7f9d9c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.505+0000 7f9da1ed4640 1 -- 172.21.15.26:0/3395453572 shutdown_connections 2024-03-06T07:32:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.505+0000 7f9da1ed4640 1 -- 172.21.15.26:0/3395453572 wait complete. 2024-03-06T07:32:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.506+0000 7f9da1ed4640 1 Processor -- start 2024-03-06T07:32:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.506+0000 7f9da1ed4640 1 -- start start 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.507+0000 7f9da1ed4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c0c8a00 0x7f9d9c1460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.507+0000 7f9da1ed4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c152920 0x7f9d9c0c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.507+0000 7f9da1ed4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c0c4340 0x7f9d9c1e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.507+0000 7f9da1ed4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c7390 con 0x7f9d9c0c4340 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.507+0000 7f9da1ed4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c06d030 con 0x7f9d9c0c8a00 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.507+0000 7f9da1ed4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c7510 con 0x7f9d9c152920 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c0c4340 0x7f9d9c1e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c0c4340 0x7f9d9c1e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59116/0 (socket says 172.21.15.26:59116) 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 -- 172.21.15.26:0/4204081308 learned_addr learned my addr 172.21.15.26:0/4204081308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 -- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c152920 msgr2=0x7f9d9c0c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c152920 0x7f9d9c0c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 -- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c0c8a00 msgr2=0x7f9d9c1460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9b7fe640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c0c8a00 0x7f9d9c1460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c0c8a00 0x7f9d9c1460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 -- 172.21.15.26:0/4204081308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c1465e0 con 0x7f9d9c0c4340 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d9bfff640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c0c4340 0x7f9d9c1e7be0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009e50 tx=0x7f9d94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d83fff640 1 -- 172.21.15.26:0/4204081308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94043020 con 0x7f9d9c0c4340 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d83fff640 1 -- 172.21.15.26:0/4204081308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9403e450 con 0x7f9d9c0c4340 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9c146880 con 0x7f9d9c0c4340 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9d83fff640 1 -- 172.21.15.26:0/4204081308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94009510 con 0x7f9d9c0c4340 2024-03-06T07:32:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.508+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c0c3860 con 0x7f9d9c0c4340 2024-03-06T07:32:34.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.510+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9c04f330 con 0x7f9d9c0c4340 2024-03-06T07:32:34.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.510+0000 7f9d83fff640 1 -- 172.21.15.26:0/4204081308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d94046020 con 0x7f9d9c0c4340 2024-03-06T07:32:34.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.510+0000 7f9d83fff640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d8806b4d0 0x7f9d8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:34.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.510+0000 7f9d83fff640 1 -- 172.21.15.26:0/4204081308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f9d940acd20 con 0x7f9d9c0c4340 2024-03-06T07:32:34.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.510+0000 7f9d9b7fe640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d8806b4d0 0x7f9d8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:34.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.511+0000 7f9d9b7fe640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d8806b4d0 0x7f9d8806d990 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c003d90 tx=0x7f9d8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:34.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.514+0000 7f9d83fff640 1 -- 172.21.15.26:0/4204081308 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d9404d030 con 0x7f9d9c0c4340 2024-03-06T07:32:34.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.733+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d9c06cda0 con 0x7f9d9c0c4340 2024-03-06T07:32:34.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.734+0000 7f9d83fff640 1 -- 172.21.15.26:0/4204081308 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f9d9407b9a0 con 0x7f9d9c0c4340 2024-03-06T07:32:34.736 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:34.737 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:34.737 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:34.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.735+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d8806b4d0 msgr2=0x7f9d8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:34.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.735+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d8806b4d0 0x7f9d8806d990 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c003d90 tx=0x7f9d8c009040 comp rx=0 tx=0).stop 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.735+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c0c4340 msgr2=0x7f9d9c1e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.735+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c0c4340 0x7f9d9c1e7be0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009e50 tx=0x7f9d94004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 shutdown_connections 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9d8806b4d0 0x7f9d8806d990 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c0c4340 0x7f9d9c1e7be0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d9c152920 0x7f9d9c0c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 --2- 172.21.15.26:0/4204081308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c0c8a00 0x7f9d9c1460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 >> 172.21.15.26:0/4204081308 conn(0x7f9d9c0ac560 msgr2=0x7f9d9c0cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 shutdown_connections 2024-03-06T07:32:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:34.736+0000 7f9da1ed4640 1 -- 172.21.15.26:0/4204081308 wait complete. 2024-03-06T07:32:35.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:35.151 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:36.152 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:36.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.251+0000 7fc9bf53a640 1 Processor -- start 2024-03-06T07:32:36.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.252+0000 7fc9bf53a640 1 -- start start 2024-03-06T07:32:36.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.252+0000 7fc9bf53a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc9b80c5480 0x7fc9b8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:36.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.252+0000 7fc9bf53a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b81586f0 0x7fc9b8158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:36.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.252+0000 7fc9bf53a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b80c5cc0 con 0x7fc9b80c5860 2024-03-06T07:32:36.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.252+0000 7fc9bf53a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b806df40 con 0x7fc9b80c5480 2024-03-06T07:32:36.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.252+0000 7fc9bf53a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b806fdb0 con 0x7fc9b81586f0 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.252+0000 7fc9bd2d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc9b80c5480 0x7fc9b8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.253+0000 7fc9bd2d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc9b80c5480 0x7fc9b8155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59122/0 (socket says 172.21.15.26:59122) 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.253+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/3491122104 learned_addr learned my addr 172.21.15.26:0/3491122104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.253+0000 7fc9bdad7640 1 --2- 172.21.15.26:0/3491122104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b81586f0 0x7fc9b8158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.253+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/3491122104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b81586f0 msgr2=0x7fc9b8158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.253+0000 7fc9bd2d6640 1 --2- 172.21.15.26:0/3491122104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b81586f0 0x7fc9b8158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.253+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/3491122104 >> v1:172.21.15.117:6789/0 conn(0x7fc9b80c5860 legacy=0x7fc9b806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.253+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/3491122104 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b81591c0 con 0x7fc9b80c5480 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bd2d6640 1 --2- 172.21.15.26:0/3491122104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b8155ba0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009870 tx=0x7fc9a802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6a85c10d3e785a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9b7fff640 1 -- 172.21.15.26:0/3491122104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80317c0 con 0x7fc9b80c5480 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9b7fff640 1 -- 172.21.15.26:0/3491122104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8037070 con 0x7fc9b80c5480 2024-03-06T07:32:36.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9b7fff640 1 -- 172.21.15.26:0/3491122104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8031b80 con 0x7fc9b80c5480 2024-03-06T07:32:36.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 -- 172.21.15.26:0/3491122104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 msgr2=0x7fc9b8155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/3491122104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b8155ba0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009870 tx=0x7fc9a802b860 comp rx=0 tx=0).stop 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 -- 172.21.15.26:0/3491122104 shutdown_connections 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/3491122104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b81586f0 0x7fc9b8158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/3491122104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b8155ba0 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 -- 172.21.15.26:0/3491122104 >> 172.21.15.26:0/3491122104 conn(0x7fc9b806b1a0 msgr2=0x7fc9b806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 -- 172.21.15.26:0/3491122104 shutdown_connections 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.254+0000 7fc9bf53a640 1 -- 172.21.15.26:0/3491122104 wait complete. 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.255+0000 7fc9bf53a640 1 Processor -- start 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bf53a640 1 -- start start 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bf53a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b81ec150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bf53a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b80c5860 0x7fc9b81515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:36.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bf53a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9b81586f0 0x7fc9b81ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bf53a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b8155680 con 0x7fc9b80c5480 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bf53a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b81553a0 con 0x7fc9b81586f0 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bf53a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b8152a20 con 0x7fc9b80c5860 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bd2d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b81ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bd2d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b81ec150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59128/0 (socket says 172.21.15.26:59128) 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/2416857867 learned_addr learned my addr 172.21.15.26:0/2416857867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bdad7640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9b81586f0 0x7fc9b81ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.256+0000 7fc9bcad5640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b80c5860 0x7fc9b81515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b80c5860 msgr2=0x7fc9b81515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bd2d6640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b80c5860 0x7fc9b81515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9b81586f0 msgr2=0x7fc9b81ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bd2d6640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9b81586f0 0x7fc9b81ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bd2d6640 1 -- 172.21.15.26:0/2416857867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b81e2420 con 0x7fc9b80c5480 2024-03-06T07:32:36.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bcad5640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b80c5860 0x7fc9b81515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bd2d6640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b81ec150 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8000c00 tx=0x7fc9a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9b5ffb640 1 -- 172.21.15.26:0/2416857867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8032860 con 0x7fc9b80c5480 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9b5ffb640 1 -- 172.21.15.26:0/2416857867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8032a20 con 0x7fc9b80c5480 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b81e2700 con 0x7fc9b80c5480 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9b5ffb640 1 -- 172.21.15.26:0/2416857867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8032e10 con 0x7fc9b80c5480 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.257+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b8151ae0 con 0x7fc9b80c5480 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.258+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b804f330 con 0x7fc9b80c5480 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.259+0000 7fc9b5ffb640 1 -- 172.21.15.26:0/2416857867 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9a804c060 con 0x7fc9b80c5480 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.259+0000 7fc9b5ffb640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc9a406b4d0 0x7fc9a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:36.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.259+0000 7fc9b5ffb640 1 -- 172.21.15.26:0/2416857867 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc9a80ac0a0 con 0x7fc9b80c5480 2024-03-06T07:32:36.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.260+0000 7fc9bcad5640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc9a406b4d0 0x7fc9a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:36.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.260+0000 7fc9bcad5640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc9a406b4d0 0x7fc9a406d990 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8154900 tx=0x7fc9ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:36.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.264+0000 7fc9b5ffb640 1 -- 172.21.15.26:0/2416857867 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc9a8040070 con 0x7fc9b80c5480 2024-03-06T07:32:36.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.694+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9b806d4b0 con 0x7fc9b80c5480 2024-03-06T07:32:36.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.694+0000 7fc9b5ffb640 1 -- 172.21.15.26:0/2416857867 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc9a807ac90 con 0x7fc9b80c5480 2024-03-06T07:32:36.697 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:36.697 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:36.697 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc9a406b4d0 msgr2=0x7fc9a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc9a406b4d0 0x7fc9a406d990 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8154900 tx=0x7fc9ac009040 comp rx=0 tx=0).stop 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 msgr2=0x7fc9b81ec150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b81ec150 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8000c00 tx=0x7fc9a80046c0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 shutdown_connections 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc9a406b4d0 0x7fc9a406d990 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9b81586f0 0x7fc9b81ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9b80c5860 0x7fc9b81515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 --2- 172.21.15.26:0/2416857867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9b80c5480 0x7fc9b81ec150 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 >> 172.21.15.26:0/2416857867 conn(0x7fc9b806b1a0 msgr2=0x7fc9b806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 shutdown_connections 2024-03-06T07:32:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:36.697+0000 7fc9bf53a640 1 -- 172.21.15.26:0/2416857867 wait complete. 2024-03-06T07:32:37.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:37.115 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:38.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.215+0000 7f69f27d0640 1 Processor -- start 2024-03-06T07:32:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.215+0000 7f69f27d0640 1 -- start start 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f27d0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69ec159270 0x7f69ec155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f27d0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec0c73d0 con 0x7f69ec0c88c0 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f27d0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec0be2e0 con 0x7f69ec14f360 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f27d0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec159700 con 0x7f69ec159270 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f0d6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69ec159270 0x7f69ec155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f0d6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69ec159270 0x7f69ec155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35454/0 (socket says 172.21.15.26:35454) 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f0d6d640 1 -- 172.21.15.26:0/1651799419 learned_addr learned my addr 172.21.15.26:0/1651799419 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f0d6d640 1 -- 172.21.15.26:0/1651799419 >> v1:172.21.15.117:6789/0 conn(0x7f69ec14f360 legacy=0x7f69ec0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f0d6d640 1 -- 172.21.15.26:0/1651799419 >> v1:172.21.15.26:6789/0 conn(0x7f69ec0c88c0 legacy=0x7f69ec158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.217+0000 7f69f0d6d640 1 -- 172.21.15.26:0/1651799419 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec0c7670 con 0x7f69ec159270 2024-03-06T07:32:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.218+0000 7f69f0d6d640 1 --2- 172.21.15.26:0/1651799419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec159270 0x7f69ec155c10 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009e50 tx=0x7f69e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e36a3c0b1988007 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.218+0000 7f69eaffd640 1 -- 172.21.15.26:0/1651799419 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e402ca80 con 0x7f69ec159270 2024-03-06T07:32:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.218+0000 7f69eaffd640 1 -- 172.21.15.26:0/1651799419 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e402cc20 con 0x7f69ec159270 2024-03-06T07:32:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.218+0000 7f69eaffd640 1 -- 172.21.15.26:0/1651799419 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e40343f0 con 0x7f69ec159270 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.219+0000 7f69f27d0640 1 -- 172.21.15.26:0/1651799419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec159270 msgr2=0x7f69ec155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.219+0000 7f69f27d0640 1 --2- 172.21.15.26:0/1651799419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec159270 0x7f69ec155c10 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009e50 tx=0x7f69e402bee0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.219+0000 7f69f27d0640 1 -- 172.21.15.26:0/1651799419 shutdown_connections 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.219+0000 7f69f27d0640 1 --2- 172.21.15.26:0/1651799419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec159270 0x7f69ec155c10 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.219+0000 7f69f27d0640 1 -- 172.21.15.26:0/1651799419 >> 172.21.15.26:0/1651799419 conn(0x7f69ec0ac570 msgr2=0x7f69ec0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.219+0000 7f69f27d0640 1 -- 172.21.15.26:0/1651799419 shutdown_connections 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.219+0000 7f69f27d0640 1 -- 172.21.15.26:0/1651799419 wait complete. 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 Processor -- start 2024-03-06T07:32:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 -- start start 2024-03-06T07:32:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec0c88c0 0x7f69ec156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69ec14f360 0x7f69ec0c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69ec159270 0x7f69ec1eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c7c30 con 0x7f69ec0c88c0 2024-03-06T07:32:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c5a50 con 0x7f69ec14f360 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.220+0000 7f69f27d0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c5bd0 con 0x7f69ec159270 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec0c88c0 0x7f69ec156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec0c88c0 0x7f69ec156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35458/0 (socket says 172.21.15.26:35458) 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 -- 172.21.15.26:0/3513448463 learned_addr learned my addr 172.21.15.26:0/3513448463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 -- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69ec159270 msgr2=0x7f69ec1eb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69ec159270 0x7f69ec1eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69eb7fe640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69ec14f360 0x7f69ec0c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 -- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69ec14f360 msgr2=0x7f69ec0c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69ec14f360 0x7f69ec0c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 -- 172.21.15.26:0/3513448463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec1ef1f0 con 0x7f69ec0c88c0 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69eb7fe640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69ec14f360 0x7f69ec0c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69ebfff640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec0c88c0 0x7f69ec156a50 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00b420 tx=0x7f69dc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69e8ff9640 1 -- 172.21.15.26:0/3513448463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc011830 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69e8ff9640 1 -- 172.21.15.26:0/3513448463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc016070 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec1eb1c0 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69e8ff9640 1 -- 172.21.15.26:0/3513448463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc011c10 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.221+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec0c4910 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.222+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec04f330 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.223+0000 7f69e8ff9640 1 -- 172.21.15.26:0/3513448463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69dc01b460 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.223+0000 7f69e8ff9640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69d806b4d0 0x7f69d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.223+0000 7f69e8ff9640 1 -- 172.21.15.26:0/3513448463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f69dc089ee0 con 0x7f69ec0c88c0 2024-03-06T07:32:38.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.224+0000 7f69eb7fe640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69d806b4d0 0x7f69d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:38.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.224+0000 7f69eb7fe640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69d806b4d0 0x7f69d806d990 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0c4c00 tx=0x7f69e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.227+0000 7f69e8ff9640 1 -- 172.21.15.26:0/3513448463 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f69dc020020 con 0x7f69ec0c88c0 2024-03-06T07:32:38.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.445+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69ec0c8d50 con 0x7f69ec0c88c0 2024-03-06T07:32:38.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.445+0000 7f69e8ff9640 1 -- 172.21.15.26:0/3513448463 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f69dc058b60 con 0x7f69ec0c88c0 2024-03-06T07:32:38.448 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:38.448 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:38.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69d806b4d0 msgr2=0x7f69d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:38.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69d806b4d0 0x7f69d806d990 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0c4c00 tx=0x7f69e0009040 comp rx=0 tx=0).stop 2024-03-06T07:32:38.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec0c88c0 msgr2=0x7f69ec156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:38.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec0c88c0 0x7f69ec156a50 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00b420 tx=0x7f69dc00b8e0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 shutdown_connections 2024-03-06T07:32:38.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f69d806b4d0 0x7f69d806d990 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69ec159270 0x7f69ec1eb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69ec14f360 0x7f69ec0c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 --2- 172.21.15.26:0/3513448463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69ec0c88c0 0x7f69ec156a50 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:38.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 >> 172.21.15.26:0/3513448463 conn(0x7f69ec0ac570 msgr2=0x7f69ec158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:38.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 shutdown_connections 2024-03-06T07:32:38.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:38.447+0000 7f69f27d0640 1 -- 172.21.15.26:0/3513448463 wait complete. 2024-03-06T07:32:38.863 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:38.864 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:39.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:39.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.965+0000 7f0dd5a17640 1 Processor -- start 2024-03-06T07:32:39.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.965+0000 7f0dd5a17640 1 -- start start 2024-03-06T07:32:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dd5a17640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0dd01527b0 0x7f0dd0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dd5a17640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd00c7c60 con 0x7f0dd00c7180 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dd5a17640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd00c8060 con 0x7f0dd00c88b0 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dd5a17640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd0159640 con 0x7f0dd01527b0 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dc7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0dd00c7180 0x7f0dd0155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47950/0 (socket says 172.21.15.26:47950) 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dc7fff640 1 -- 172.21.15.26:0/4089753756 learned_addr learned my addr 172.21.15.26:0/4089753756 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dceffd640 1 --2- 172.21.15.26:0/4089753756 >> v2:172.21.15.117:3300/0 conn(0x7f0dd01527b0 0x7f0dd0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200890442 0 0) 0x7f0db8001110 con 0x7f0dd00c7180 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dbc003520 con 0x7f0dd00c7180 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4038829509 0 0) 0x7f0dbc003520 con 0x7f0dd00c7180 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 >> v2:172.21.15.117:3300/0 conn(0x7f0dd01527b0 msgr2=0x7f0dd0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 --2- 172.21.15.26:0/4089753756 >> v2:172.21.15.117:3300/0 conn(0x7f0dd01527b0 0x7f0dd0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 >> v1:172.21.15.117:6789/0 conn(0x7f0dd00c88b0 legacy=0x7f0dd00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd00c76a0 con 0x7f0dd00c7180 2024-03-06T07:32:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f0db8001840 con 0x7f0dd00c7180 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 >> v1:172.21.15.26:6789/0 conn(0x7f0dd00c7180 legacy=0x7f0dd0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0159640 con 0x7f0dbc004800 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc00c160 con 0x7f0dbc0039b0 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.966+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc00c360 con 0x7f0dbc0051c0 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dc7fff640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dceffd640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dcf7fe640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dc7fff640 1 -- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dbc0051c0 msgr2=0x7f0dbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dc7fff640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dc7fff640 1 -- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc0039b0 msgr2=0x7f0dbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dc7fff640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dc7fff640 1 -- 172.21.15.26:0/4089753756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc003520 con 0x7f0dbc004800 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dcf7fe640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.967+0000 7f0dcf7fe640 1 -- 172.21.15.26:0/4089753756 reap_dead start 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dc7fff640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f0db800c4c0 tx=0x7f0dbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78382b897e85b5ce server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800c0e0 con 0x7f0dbc004800 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db8030070 con 0x7f0dbc004800 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dce7fc640 1 -- 172.21.15.26:0/4089753756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db80058d0 con 0x7f0dbc004800 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dd5a17640 1 -- 172.21.15.26:0/4089753756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc004800 msgr2=0x7f0dbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f0db800c4c0 tx=0x7f0dbc005830 comp rx=0 tx=0).stop 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dd5a17640 1 -- 172.21.15.26:0/4089753756 shutdown_connections 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/4089753756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dd5a17640 1 -- 172.21.15.26:0/4089753756 >> 172.21.15.26:0/4089753756 conn(0x7f0dd00ac560 msgr2=0x7f0dd00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dd5a17640 1 -- 172.21.15.26:0/4089753756 shutdown_connections 2024-03-06T07:32:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.968+0000 7f0dd5a17640 1 -- 172.21.15.26:0/4089753756 wait complete. 2024-03-06T07:32:39.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.969+0000 7f0dd5a17640 1 Processor -- start 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dd5a17640 1 -- start start 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dd5a17640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc004800 0x7f0dd014f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dd5a17640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dd0154280 0x7f0dd00c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dd5a17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dd00c4430 0x7f0dd01586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dd5a17640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0155600 con 0x7f0dd00c4430 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dd5a17640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0155320 con 0x7f0dbc004800 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dd5a17640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0152b90 con 0x7f0dd0154280 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dd00c4430 0x7f0dd01586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dd00c4430 0x7f0dd01586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35480/0 (socket says 172.21.15.26:35480) 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dcf7fe640 1 -- 172.21.15.26:0/1027154354 learned_addr learned my addr 172.21.15.26:0/1027154354 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.970+0000 7f0dceffd640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc004800 0x7f0dd014f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dc7fff640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dd0154280 0x7f0dd00c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dcf7fe640 1 -- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dd0154280 msgr2=0x7f0dd00c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dcf7fe640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dd0154280 0x7f0dd00c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dcf7fe640 1 -- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc004800 msgr2=0x7f0dd014f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dcf7fe640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc004800 0x7f0dd014f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dcf7fe640 1 -- 172.21.15.26:0/1027154354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd01eb450 con 0x7f0dd00c4430 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dc7fff640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dd0154280 0x7f0dd00c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dcf7fe640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dd00c4430 0x7f0dd01586a0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00aa340 tx=0x7f0dc800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dccff9640 1 -- 172.21.15.26:0/1027154354 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8003b80 con 0x7f0dd00c4430 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dccff9640 1 -- 172.21.15.26:0/1027154354 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8003d20 con 0x7f0dd00c4430 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd0156740 con 0x7f0dd00c4430 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dccff9640 1 -- 172.21.15.26:0/1027154354 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8006b50 con 0x7f0dd00c4430 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.971+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd01eb090 con 0x7f0dd00c4430 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.972+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd004f330 con 0x7f0dd00c4430 2024-03-06T07:32:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.972+0000 7f0dccff9640 1 -- 172.21.15.26:0/1027154354 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dc801a020 con 0x7f0dd00c4430 2024-03-06T07:32:39.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.973+0000 7f0dccff9640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0db006b4d0 0x7f0db006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.973+0000 7f0dccff9640 1 -- 172.21.15.26:0/1027154354 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f0dc80899e0 con 0x7f0dd00c4430 2024-03-06T07:32:39.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.973+0000 7f0dceffd640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0db006b4d0 0x7f0db006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.974+0000 7f0dceffd640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0db006b4d0 0x7f0db006d990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009c40 tx=0x7f0dc0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:39.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:39.978+0000 7f0dccff9640 1 -- 172.21.15.26:0/1027154354 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0dc80585d0 con 0x7f0dd00c4430 2024-03-06T07:32:40.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.198+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dd006ce00 con 0x7f0dd00c4430 2024-03-06T07:32:40.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.198+0000 7f0dccff9640 1 -- 172.21.15.26:0/1027154354 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f0dc8057ed0 con 0x7f0dd00c4430 2024-03-06T07:32:40.201 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:40.201 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:40.201 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0db006b4d0 msgr2=0x7f0db006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0db006b4d0 0x7f0db006d990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009c40 tx=0x7f0dc0009190 comp rx=0 tx=0).stop 2024-03-06T07:32:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dd00c4430 msgr2=0x7f0dd01586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:40.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dd00c4430 0x7f0dd01586a0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00aa340 tx=0x7f0dc800ead0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 shutdown_connections 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0db006b4d0 0x7f0db006d990 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dd00c4430 0x7f0dd01586a0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dd0154280 0x7f0dd00c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 --2- 172.21.15.26:0/1027154354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc004800 0x7f0dd014f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.200+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 >> 172.21.15.26:0/1027154354 conn(0x7f0dd00ac560 msgr2=0x7f0dd00abc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.201+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 shutdown_connections 2024-03-06T07:32:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:40.201+0000 7f0dd5a17640 1 -- 172.21.15.26:0/1027154354 wait complete. 2024-03-06T07:32:40.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:40.616 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:41.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:41.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.725+0000 7f7d0bc4e640 1 Processor -- start 2024-03-06T07:32:41.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.725+0000 7f7d0bc4e640 1 -- start start 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0bc4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 0x7f7d04155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0bc4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d04158b60 0x7f7d04158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0bc4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d040c73d0 con 0x7f7d040c88c0 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0bc4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040c8dd0 con 0x7f7d04158b60 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0bc4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040be2e0 con 0x7f7d0414f360 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0a1eb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d04158b60 0x7f7d04158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d091e9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7d040c88c0 0x7f7d040c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47962/0 (socket says 172.21.15.26:47962) 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d091e9640 1 -- 172.21.15.26:0/392582221 learned_addr learned my addr 172.21.15.26:0/392582221 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0a1eb640 1 -- 172.21.15.26:0/392582221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 msgr2=0x7f7d04155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d099ea640 1 --2- 172.21.15.26:0/392582221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 0x7f7d04155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0a1eb640 1 --2- 172.21.15.26:0/392582221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 0x7f7d04155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d089e8640 1 -- 172.21.15.26:0/392582221 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979660011 0 0) 0x7f7d040c73d0 con 0x7f7d040c88c0 2024-03-06T07:32:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0a1eb640 1 -- 172.21.15.26:0/392582221 >> v1:172.21.15.26:6789/0 conn(0x7f7d040c88c0 legacy=0x7f7d040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.726+0000 7f7d0a1eb640 1 -- 172.21.15.26:0/392582221 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040c7670 con 0x7f7d04158b60 2024-03-06T07:32:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.727+0000 7f7d0a1eb640 1 --2- 172.21.15.26:0/392582221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04158ff0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009d90 tx=0x7f7cfc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50e55474bcd636 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.727+0000 7f7d089e8640 1 -- 172.21.15.26:0/392582221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc039070 con 0x7f7d04158b60 2024-03-06T07:32:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.727+0000 7f7d089e8640 1 -- 172.21.15.26:0/392582221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc02cbc0 con 0x7f7d04158b60 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.727+0000 7f7d089e8640 1 -- 172.21.15.26:0/392582221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc034440 con 0x7f7d04158b60 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/392582221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 msgr2=0x7f7d04158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/392582221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04158ff0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009d90 tx=0x7f7cfc02be10 comp rx=0 tx=0).stop 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/392582221 shutdown_connections 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/392582221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04158ff0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/392582221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 0x7f7d04155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/392582221 >> 172.21.15.26:0/392582221 conn(0x7f7d040ac570 msgr2=0x7f7d040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/392582221 shutdown_connections 2024-03-06T07:32:41.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.728+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/392582221 wait complete. 2024-03-06T07:32:41.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.729+0000 7f7d0bc4e640 1 Processor -- start 2024-03-06T07:32:41.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.729+0000 7f7d0bc4e640 1 -- start start 2024-03-06T07:32:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0bc4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d040c88c0 0x7f7d041580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0bc4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 0x7f7d0414b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0bc4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0bc4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040c5a50 con 0x7f7d04158b60 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0bc4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040c7c30 con 0x7f7d040c88c0 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0bc4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040c5bd0 con 0x7f7d0414f360 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35492/0 (socket says 172.21.15.26:35492) 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 -- 172.21.15.26:0/2808899093 learned_addr learned my addr 172.21.15.26:0/2808899093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d099ea640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d040c88c0 0x7f7d041580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 -- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 msgr2=0x7f7d0414b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 0x7f7d0414b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 -- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d040c88c0 msgr2=0x7f7d041580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d040c88c0 0x7f7d041580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 -- 172.21.15.26:0/2808899093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04151690 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0a1eb640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04148990 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009d90 tx=0x7f7cfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7cfa7fc640 1 -- 172.21.15.26:0/2808899093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc039050 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7cfa7fc640 1 -- 172.21.15.26:0/2808899093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc03a040 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7cfa7fc640 1 -- 172.21.15.26:0/2808899093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc003990 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.730+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d04149870 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.731+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d04157450 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.732+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0404f330 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.732+0000 7f7cfa7fc640 1 -- 172.21.15.26:0/2808899093 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cfc047030 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.732+0000 7f7cfa7fc640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ce806b4d0 0x7f7ce806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.732+0000 7f7cfa7fc640 1 -- 172.21.15.26:0/2808899093 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f7cfc0ac980 con 0x7f7d04158b60 2024-03-06T07:32:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.733+0000 7f7d099ea640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ce806b4d0 0x7f7ce806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.733+0000 7f7d099ea640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ce806b4d0 0x7f7ce806d990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.737+0000 7f7cfa7fc640 1 -- 172.21.15.26:0/2808899093 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7cfc04e020 con 0x7f7d04158b60 2024-03-06T07:32:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.956+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d0406cda0 con 0x7f7d04158b60 2024-03-06T07:32:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.956+0000 7f7cfa7fc640 1 -- 172.21.15.26:0/2808899093 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f7cfc07b600 con 0x7f7d04158b60 2024-03-06T07:32:41.959 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:41.959 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ce806b4d0 msgr2=0x7f7ce806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ce806b4d0 0x7f7ce806d990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00008040 comp rx=0 tx=0).stop 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 msgr2=0x7f7d04148990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04148990 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009d90 tx=0x7f7cfc004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 shutdown_connections 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7ce806b4d0 0x7f7ce806d990 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d04158b60 0x7f7d04148990 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0414f360 0x7f7d0414b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 --2- 172.21.15.26:0/2808899093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d040c88c0 0x7f7d041580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 >> 172.21.15.26:0/2808899093 conn(0x7f7d040ac570 msgr2=0x7f7d040c9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 shutdown_connections 2024-03-06T07:32:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:41.958+0000 7f7d0bc4e640 1 -- 172.21.15.26:0/2808899093 wait complete. 2024-03-06T07:32:42.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:42.374 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:43.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:43.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.474+0000 7f680ba33640 1 Processor -- start 2024-03-06T07:32:43.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.474+0000 7f680ba33640 1 -- start start 2024-03-06T07:32:43.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.475+0000 7f680ba33640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68041527b0 0x7f6804159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:43.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.475+0000 7f680ba33640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.475+0000 7f680ba33640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68040c7c60 con 0x7f68040c7180 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.475+0000 7f680ba33640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6804159640 con 0x7f68041527b0 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.475+0000 7f680ba33640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040c8060 con 0x7f68040c88b0 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f6809fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f6809fd0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:46580/0 (socket says 172.21.15.26:46580) 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f68097cf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68041527b0 0x7f6804159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f6809fd0640 1 -- 172.21.15.26:0/2164299702 learned_addr learned my addr 172.21.15.26:0/2164299702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f6809fd0640 1 -- 172.21.15.26:0/2164299702 >> v2:172.21.15.117:3300/0 conn(0x7f68041527b0 msgr2=0x7f6804159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f6809fd0640 1 --2- 172.21.15.26:0/2164299702 >> v2:172.21.15.117:3300/0 conn(0x7f68041527b0 0x7f6804159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f6809fd0640 1 -- 172.21.15.26:0/2164299702 >> v1:172.21.15.117:6789/0 conn(0x7f68040c7180 legacy=0x7f6804155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f6809fd0640 1 -- 172.21.15.26:0/2164299702 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68040c76a0 con 0x7f68040c88b0 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f67fbfff640 1 -- 172.21.15.26:0/2164299702 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340886543 0 0) 0x7f67f4001110 con 0x7f68040c7180 2024-03-06T07:32:43.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.476+0000 7f68097cf640 1 --2- 172.21.15.26:0/2164299702 >> v2:172.21.15.117:3300/0 conn(0x7f68041527b0 0x7f6804159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.477+0000 7f6809fd0640 1 --2- 172.21.15.26:0/2164299702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c5330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009e50 tx=0x7f67fc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e12c81d8e07b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.477+0000 7f67fbfff640 1 -- 172.21.15.26:0/2164299702 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc02ca70 con 0x7f68040c88b0 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.477+0000 7f67fbfff640 1 -- 172.21.15.26:0/2164299702 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc02cc10 con 0x7f68040c88b0 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.477+0000 7f67fbfff640 1 -- 172.21.15.26:0/2164299702 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0343f0 con 0x7f68040c88b0 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.477+0000 7f680ba33640 1 -- 172.21.15.26:0/2164299702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 msgr2=0x7f68040c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.477+0000 7f680ba33640 1 --2- 172.21.15.26:0/2164299702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c5330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009e50 tx=0x7f67fc02bed0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.478+0000 7f680ba33640 1 -- 172.21.15.26:0/2164299702 shutdown_connections 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.478+0000 7f680ba33640 1 --2- 172.21.15.26:0/2164299702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c5330 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.478+0000 7f680ba33640 1 --2- 172.21.15.26:0/2164299702 >> v2:172.21.15.117:3300/0 conn(0x7f68041527b0 0x7f6804159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.478+0000 7f680ba33640 1 -- 172.21.15.26:0/2164299702 >> 172.21.15.26:0/2164299702 conn(0x7f68040ac560 msgr2=0x7f68040ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.478+0000 7f680ba33640 1 -- 172.21.15.26:0/2164299702 shutdown_connections 2024-03-06T07:32:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.478+0000 7f680ba33640 1 -- 172.21.15.26:0/2164299702 wait complete. 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 Processor -- start 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 -- start start 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68040c7180 0x7f68040c2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68041527b0 0x7f68041dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6804155600 con 0x7f68040c7180 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6804155320 con 0x7f68041527b0 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.479+0000 7f680ba33640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040c8d60 con 0x7f68040c88b0 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68040c7180 0x7f68040c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68040c7180 0x7f68040c2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35502/0 (socket says 172.21.15.26:35502) 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 -- 172.21.15.26:0/3374661026 learned_addr learned my addr 172.21.15.26:0/3374661026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f6809fd0640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68041527b0 0x7f68041dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f6808fce640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 -- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 msgr2=0x7f68040c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 -- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68041527b0 msgr2=0x7f68041dfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68041527b0 0x7f68041dfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 -- 172.21.15.26:0/3374661026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680414eff0 con 0x7f68040c7180 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f6808fce640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f68097cf640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68040c7180 0x7f68040c2470 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f680000e960 tx=0x7f680000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f67f9ffb640 1 -- 172.21.15.26:0/3374661026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800017070 con 0x7f68040c7180 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f67f9ffb640 1 -- 172.21.15.26:0/3374661026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680000ca00 con 0x7f68040c7180 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68041e1340 con 0x7f68040c7180 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.480+0000 7f67f9ffb640 1 -- 172.21.15.26:0/3374661026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000cdd0 con 0x7f68040c7180 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.481+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680414f2d0 con 0x7f68040c7180 2024-03-06T07:32:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.481+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68041e1130 con 0x7f68040c7180 2024-03-06T07:32:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.482+0000 7f67f9ffb640 1 -- 172.21.15.26:0/3374661026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f680001a070 con 0x7f68040c7180 2024-03-06T07:32:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.482+0000 7f67f9ffb640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67e806b500 0x7f67e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.483+0000 7f67f9ffb640 1 -- 172.21.15.26:0/3374661026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f680008a660 con 0x7f68040c7180 2024-03-06T07:32:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.483+0000 7f6808fce640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67e806b500 0x7f67e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.483+0000 7f6808fce640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67e806b500 0x7f67e806d9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f68040c4f70 tx=0x7f67f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.487+0000 7f67f9ffb640 1 -- 172.21.15.26:0/3374661026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6800058b60 con 0x7f68040c7180 2024-03-06T07:32:43.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.706+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f680404f330 con 0x7f68040c7180 2024-03-06T07:32:43.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.706+0000 7f67f9ffb640 1 -- 172.21.15.26:0/3374661026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f680005cc00 con 0x7f68040c7180 2024-03-06T07:32:43.709 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:43.709 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:43.709 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67e806b500 msgr2=0x7f67e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67e806b500 0x7f67e806d9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f68040c4f70 tx=0x7f67f4009040 comp rx=0 tx=0).stop 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68040c7180 msgr2=0x7f68040c2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68040c7180 0x7f68040c2470 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f680000e960 tx=0x7f680000ee20 comp rx=0 tx=0).stop 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 shutdown_connections 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f67e806b500 0x7f67e806d9c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68041527b0 0x7f68041dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68040c88b0 0x7f68040c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 --2- 172.21.15.26:0/3374661026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68040c7180 0x7f68040c2470 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:43.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 >> 172.21.15.26:0/3374661026 conn(0x7f68040ac560 msgr2=0x7f68040c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:43.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 shutdown_connections 2024-03-06T07:32:43.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:43.708+0000 7f680ba33640 1 -- 172.21.15.26:0/3374661026 wait complete. 2024-03-06T07:32:43.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:43.737 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:44.124 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:44.125 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:45.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:45.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.231+0000 7fe6c8581640 1 Processor -- start 2024-03-06T07:32:45.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.231+0000 7fe6c8581640 1 -- start start 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c8581640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c00c88c0 0x7fe6c0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c8581640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6c0158b60 0x7fe6c0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c8581640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c00c73d0 con 0x7fe6c014f360 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c8581640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c00c8dd0 con 0x7fe6c0158b60 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c8581640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00be2e0 con 0x7fe6c00c88c0 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c631d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe6c014f360 0x7fe6c00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47978/0 (socket says 172.21.15.26:47978) 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c631d640 1 -- 172.21.15.26:0/1697672785 learned_addr learned my addr 172.21.15.26:0/1697672785 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c5b1c640 1 --2- 172.21.15.26:0/1697672785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c00c88c0 0x7fe6c0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c6b1e640 1 --2- 172.21.15.26:0/1697672785 >> v2:172.21.15.117:3300/0 conn(0x7fe6c0158b60 0x7fe6c0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c531b640 1 -- 172.21.15.26:0/1697672785 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267904615 0 0) 0x7fe6c00c73d0 con 0x7fe6c014f360 2024-03-06T07:32:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c531b640 1 -- 172.21.15.26:0/1697672785 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6ac003520 con 0x7fe6c014f360 2024-03-06T07:32:45.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c531b640 1 -- 172.21.15.26:0/1697672785 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049763838 0 0) 0x7fe6ac003520 con 0x7fe6c014f360 2024-03-06T07:32:45.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c5b1c640 1 -- 172.21.15.26:0/1697672785 >> v2:172.21.15.117:3300/0 conn(0x7fe6c0158b60 msgr2=0x7fe6c0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:45.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c5b1c640 1 --2- 172.21.15.26:0/1697672785 >> v2:172.21.15.117:3300/0 conn(0x7fe6c0158b60 0x7fe6c0158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c5b1c640 1 -- 172.21.15.26:0/1697672785 >> v1:172.21.15.26:6789/0 conn(0x7fe6c014f360 legacy=0x7fe6c00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:45.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c5b1c640 1 -- 172.21.15.26:0/1697672785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00c7670 con 0x7fe6c00c88c0 2024-03-06T07:32:45.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.232+0000 7fe6c6b1e640 1 --2- 172.21.15.26:0/1697672785 >> v2:172.21.15.117:3300/0 conn(0x7fe6c0158b60 0x7fe6c0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.233+0000 7fe6c5b1c640 1 --2- 172.21.15.26:0/1697672785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c00c88c0 0x7fe6c0155c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00097a0 tx=0x7fe6b002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f498d9fcebd8aa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.233+0000 7fe6c531b640 1 -- 172.21.15.26:0/1697672785 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0037070 con 0x7fe6c00c88c0 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.233+0000 7fe6c531b640 1 -- 172.21.15.26:0/1697672785 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0030510 con 0x7fe6c00c88c0 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.233+0000 7fe6c531b640 1 -- 172.21.15.26:0/1697672785 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc001670 con 0x7fe6c00c88c0 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 -- 172.21.15.26:0/1697672785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c00c88c0 msgr2=0x7fe6c0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 --2- 172.21.15.26:0/1697672785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c00c88c0 0x7fe6c0155c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00097a0 tx=0x7fe6b002b710 comp rx=0 tx=0).stop 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 -- 172.21.15.26:0/1697672785 shutdown_connections 2024-03-06T07:32:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 --2- 172.21.15.26:0/1697672785 >> v2:172.21.15.117:3300/0 conn(0x7fe6c0158b60 0x7fe6c0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 --2- 172.21.15.26:0/1697672785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c00c88c0 0x7fe6c0155c10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 -- 172.21.15.26:0/1697672785 >> 172.21.15.26:0/1697672785 conn(0x7fe6c00ac570 msgr2=0x7fe6c00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 -- 172.21.15.26:0/1697672785 shutdown_connections 2024-03-06T07:32:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.234+0000 7fe6c8581640 1 -- 172.21.15.26:0/1697672785 wait complete. 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.235+0000 7fe6c8581640 1 Processor -- start 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.235+0000 7fe6c8581640 1 -- start start 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c8581640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c00c88c0 0x7fe6c014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c8581640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c014f360 0x7fe6c014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c8581640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c0158b60 0x7fe6c00bf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c8581640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c5a50 con 0x7fe6c00c88c0 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c8581640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c7c30 con 0x7fe6c014f360 2024-03-06T07:32:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c8581640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00c5bd0 con 0x7fe6c0158b60 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c00c88c0 0x7fe6c014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c00c88c0 0x7fe6c014b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35512/0 (socket says 172.21.15.26:35512) 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 -- 172.21.15.26:0/243073381 learned_addr learned my addr 172.21.15.26:0/243073381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c5b1c640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c014f360 0x7fe6c014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c6b1e640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c0158b60 0x7fe6c00bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 -- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c0158b60 msgr2=0x7fe6c00bf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c0158b60 0x7fe6c00bf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 -- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c014f360 msgr2=0x7fe6c014bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c014f360 0x7fe6c014bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 -- 172.21.15.26:0/243073381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c014eaa0 con 0x7fe6c00c88c0 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.236+0000 7fe6c631d640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c00c88c0 0x7fe6c014b790 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0017e0 tx=0x7fe6bc00cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.237+0000 7fe6b6ffd640 1 -- 172.21.15.26:0/243073381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc00ce60 con 0x7fe6c00c88c0 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.237+0000 7fe6b6ffd640 1 -- 172.21.15.26:0/243073381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc005af0 con 0x7fe6c00c88c0 2024-03-06T07:32:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.237+0000 7fe6b6ffd640 1 -- 172.21.15.26:0/243073381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc007070 con 0x7fe6c00c88c0 2024-03-06T07:32:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.237+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c00bfeb0 con 0x7fe6c00c88c0 2024-03-06T07:32:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.237+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c00be740 con 0x7fe6c00c88c0 2024-03-06T07:32:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.238+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c004f330 con 0x7fe6c00c88c0 2024-03-06T07:32:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.238+0000 7fe6b6ffd640 1 -- 172.21.15.26:0/243073381 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6bc007210 con 0x7fe6c00c88c0 2024-03-06T07:32:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.239+0000 7fe6b6ffd640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.239+0000 7fe6b6ffd640 1 -- 172.21.15.26:0/243073381 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe6bc08a020 con 0x7fe6c00c88c0 2024-03-06T07:32:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.239+0000 7fe6c5b1c640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.239+0000 7fe6c5b1c640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6a406b4d0 0x7fe6a406d990 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0034000 tx=0x7fe6b0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:45.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.243+0000 7fe6b6ffd640 1 -- 172.21.15.26:0/243073381 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe6bc058ca0 con 0x7fe6c00c88c0 2024-03-06T07:32:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.596+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6c00aca80 con 0x7fe6c00c88c0 2024-03-06T07:32:45.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.596+0000 7fe6b6ffd640 1 -- 172.21.15.26:0/243073381 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fe6bc0585a0 con 0x7fe6c00c88c0 2024-03-06T07:32:45.599 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:45.599 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:45.599 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6a406b4d0 msgr2=0x7fe6a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6a406b4d0 0x7fe6a406d990 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0034000 tx=0x7fe6b0008030 comp rx=0 tx=0).stop 2024-03-06T07:32:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c00c88c0 msgr2=0x7fe6c014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c00c88c0 0x7fe6c014b790 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0017e0 tx=0x7fe6bc00cb20 comp rx=0 tx=0).stop 2024-03-06T07:32:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 shutdown_connections 2024-03-06T07:32:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6c0158b60 0x7fe6c00bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c014f360 0x7fe6c014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 --2- 172.21.15.26:0/243073381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c00c88c0 0x7fe6c014b790 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 >> 172.21.15.26:0/243073381 conn(0x7fe6c00ac570 msgr2=0x7fe6c006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 shutdown_connections 2024-03-06T07:32:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:45.598+0000 7fe6c8581640 1 -- 172.21.15.26:0/243073381 wait complete. 2024-03-06T07:32:46.015 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:46.016 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:47.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.116+0000 7f7912f7a640 1 Processor -- start 2024-03-06T07:32:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.116+0000 7f7912f7a640 1 -- start start 2024-03-06T07:32:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7912f7a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f790c159270 0x7f790c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7912f7a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c0c73d0 con 0x7f790c14f360 2024-03-06T07:32:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7912f7a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c0be2e0 con 0x7f790c0c88c0 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7912f7a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c159700 con 0x7f790c159270 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7911517640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f790c159270 0x7f790c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7910d16640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f790c14f360 0x7f790c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43900/0 (socket says 172.21.15.26:43900) 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7911517640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f790c159270 0x7f790c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56478/0 (socket says 172.21.15.26:56478) 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.117+0000 7f7910d16640 1 -- 172.21.15.26:0/1288720898 learned_addr learned my addr 172.21.15.26:0/1288720898 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7911517640 1 -- 172.21.15.26:0/1288720898 >> v1:172.21.15.117:6789/0 conn(0x7f790c0c88c0 legacy=0x7f790c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7911517640 1 -- 172.21.15.26:0/1288720898 >> v1:172.21.15.26:6789/0 conn(0x7f790c14f360 legacy=0x7f790c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f790b7fe640 1 -- 172.21.15.26:0/1288720898 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835077224 0 0) 0x7f790c0c73d0 con 0x7f790c14f360 2024-03-06T07:32:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7911517640 1 -- 172.21.15.26:0/1288720898 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c0c7670 con 0x7f790c159270 2024-03-06T07:32:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7911517640 1 --2- 172.21.15.26:0/1288720898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c159270 0x7f790c155c10 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f7904009d90 tx=0x7f790402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104055d79e86691d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f790b7fe640 1 -- 172.21.15.26:0/1288720898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904039070 con 0x7f790c159270 2024-03-06T07:32:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f790b7fe640 1 -- 172.21.15.26:0/1288720898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790402cc60 con 0x7f790c159270 2024-03-06T07:32:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f790b7fe640 1 -- 172.21.15.26:0/1288720898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904034660 con 0x7f790c159270 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7912f7a640 1 -- 172.21.15.26:0/1288720898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c159270 msgr2=0x7f790c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7912f7a640 1 --2- 172.21.15.26:0/1288720898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c159270 0x7f790c155c10 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f7904009d90 tx=0x7f790402beb0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7912f7a640 1 -- 172.21.15.26:0/1288720898 shutdown_connections 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7912f7a640 1 --2- 172.21.15.26:0/1288720898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c159270 0x7f790c155c10 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7912f7a640 1 -- 172.21.15.26:0/1288720898 >> 172.21.15.26:0/1288720898 conn(0x7f790c0ac570 msgr2=0x7f790c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7912f7a640 1 -- 172.21.15.26:0/1288720898 shutdown_connections 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.118+0000 7f7912f7a640 1 -- 172.21.15.26:0/1288720898 wait complete. 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.120+0000 7f7912f7a640 1 Processor -- start 2024-03-06T07:32:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7912f7a640 1 -- start start 2024-03-06T07:32:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7912f7a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c0c88c0 0x7f790c151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7912f7a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f790c14f360 0x7f790c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7912f7a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f790c159270 0x7f790c14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7912f7a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0c7c30 con 0x7f790c0c88c0 2024-03-06T07:32:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7912f7a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0c5a50 con 0x7f790c14f360 2024-03-06T07:32:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7912f7a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0c5bd0 con 0x7f790c159270 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7910d16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c0c88c0 0x7f790c151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7910d16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c0c88c0 0x7f790c151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56492/0 (socket says 172.21.15.26:56492) 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.121+0000 7f7910d16640 1 -- 172.21.15.26:0/3400651509 learned_addr learned my addr 172.21.15.26:0/3400651509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7910d16640 1 -- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f790c159270 msgr2=0x7f790c14bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7910d16640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f790c159270 0x7f790c14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7910d16640 1 -- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f790c14f360 msgr2=0x7f790c1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7910d16640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f790c14f360 0x7f790c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7910d16640 1 -- 172.21.15.26:0/3400651509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c14f7e0 con 0x7f790c0c88c0 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7910d16640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c0c88c0 0x7f790c151ca0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f790000c4a0 tx=0x7f790000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f79097fa640 1 -- 172.21.15.26:0/3400651509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900018070 con 0x7f790c0c88c0 2024-03-06T07:32:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f79097fa640 1 -- 172.21.15.26:0/3400651509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79000074e0 con 0x7f790c0c88c0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f79097fa640 1 -- 172.21.15.26:0/3400651509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900004a20 con 0x7f790c0c88c0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c14fac0 con 0x7f790c0c88c0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c150080 con 0x7f790c0c88c0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.122+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c04f330 con 0x7f790c0c88c0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.124+0000 7f79097fa640 1 -- 172.21.15.26:0/3400651509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f790001b070 con 0x7f790c0c88c0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.124+0000 7f79097fa640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f78f006b500 0x7f78f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.124+0000 7f79097fa640 1 -- 172.21.15.26:0/3400651509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f79000896e0 con 0x7f790c0c88c0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.124+0000 7f790bfff640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f78f006b500 0x7f78f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.125+0000 7f790bfff640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f78f006b500 0x7f78f006d9c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000f60 tx=0x7f78f80079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.128+0000 7f79097fa640 1 -- 172.21.15.26:0/3400651509 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7900058360 con 0x7f790c0c88c0 2024-03-06T07:32:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.391+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f790c0c8d50 con 0x7f790c0c88c0 2024-03-06T07:32:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.391+0000 7f79097fa640 1 -- 172.21.15.26:0/3400651509 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f7900057c60 con 0x7f790c0c88c0 2024-03-06T07:32:47.393 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:47.393 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.393+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f78f006b500 msgr2=0x7f78f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.393+0000 7f7912f7a640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f78f006b500 0x7f78f006d9c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000f60 tx=0x7f78f80079b0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.393+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c0c88c0 msgr2=0x7f790c151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.393+0000 7f7912f7a640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c0c88c0 0x7f790c151ca0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f790000c4a0 tx=0x7f790000c960 comp rx=0 tx=0).stop 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 shutdown_connections 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f78f006b500 0x7f78f006d9c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f790c159270 0x7f790c14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f790c14f360 0x7f790c1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 --2- 172.21.15.26:0/3400651509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790c0c88c0 0x7f790c151ca0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 >> 172.21.15.26:0/3400651509 conn(0x7f790c0ac570 msgr2=0x7f790c0c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 shutdown_connections 2024-03-06T07:32:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:47.394+0000 7f7912f7a640 1 -- 172.21.15.26:0/3400651509 wait complete. 2024-03-06T07:32:47.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:47.808 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:48.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:48.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.909+0000 7f272f5f7640 1 Processor -- start 2024-03-06T07:32:48.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.909+0000 7f272f5f7640 1 -- start start 2024-03-06T07:32:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272f5f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f272806afc0 0x7f272806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272f5f7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f27281536c0 0x7f272815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272f5f7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27280a70f0 con 0x7f272806b3a0 2024-03-06T07:32:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272f5f7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272806b830 con 0x7f272806afc0 2024-03-06T07:32:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272f5f7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2728150a20 con 0x7f27281536c0 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272d393640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f272806afc0 0x7f272806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272d393640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f272806afc0 0x7f272806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56494/0 (socket says 172.21.15.26:56494) 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272d393640 1 -- 172.21.15.26:0/1644787683 learned_addr learned my addr 172.21.15.26:0/1644787683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272d393640 1 -- 172.21.15.26:0/1644787683 >> v2:172.21.15.117:3300/0 conn(0x7f27281536c0 msgr2=0x7f272815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272db94640 1 --2- 172.21.15.26:0/1644787683 >> v2:172.21.15.117:3300/0 conn(0x7f27281536c0 0x7f272815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272d393640 1 --2- 172.21.15.26:0/1644787683 >> v2:172.21.15.117:3300/0 conn(0x7f27281536c0 0x7f272815d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272d393640 1 -- 172.21.15.26:0/1644787683 >> v1:172.21.15.26:6789/0 conn(0x7f272806b3a0 legacy=0x7f2728152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f2727fff640 1 -- 172.21.15.26:0/1644787683 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362563407 0 0) 0x7f27280a70f0 con 0x7f272806b3a0 2024-03-06T07:32:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.910+0000 7f272d393640 1 -- 172.21.15.26:0/1644787683 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27281567f0 con 0x7f272806afc0 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.911+0000 7f272d393640 1 --2- 172.21.15.26:0/1644787683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806afc0 0x7f272806f100 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f2720009870 tx=0x7f272002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855c56964d021db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.911+0000 7f2727fff640 1 -- 172.21.15.26:0/1644787683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272002e070 con 0x7f272806afc0 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.911+0000 7f2727fff640 1 -- 172.21.15.26:0/1644787683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2720037070 con 0x7f272806afc0 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.911+0000 7f2727fff640 1 -- 172.21.15.26:0/1644787683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720031a00 con 0x7f272806afc0 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 -- 172.21.15.26:0/1644787683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806afc0 msgr2=0x7f272806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1644787683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806afc0 0x7f272806f100 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f2720009870 tx=0x7f272002b880 comp rx=0 tx=0).stop 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 -- 172.21.15.26:0/1644787683 shutdown_connections 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1644787683 >> v2:172.21.15.117:3300/0 conn(0x7f27281536c0 0x7f272815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1644787683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806afc0 0x7f272806f100 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 -- 172.21.15.26:0/1644787683 >> 172.21.15.26:0/1644787683 conn(0x7f27280c4290 msgr2=0x7f27280c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 -- 172.21.15.26:0/1644787683 shutdown_connections 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.912+0000 7f272f5f7640 1 -- 172.21.15.26:0/1644787683 wait complete. 2024-03-06T07:32:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.913+0000 7f272f5f7640 1 Processor -- start 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.913+0000 7f272f5f7640 1 -- start start 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272f5f7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f272806afc0 0x7f27281ec130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272f5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806b3a0 0x7f27281ea900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272f5f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27281536c0 0x7f27281eae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272f5f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728156db0 con 0x7f272806b3a0 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272f5f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27281565a0 con 0x7f27281536c0 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272f5f7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27280c7360 con 0x7f272806afc0 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806b3a0 0x7f27281ea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806b3a0 0x7f27281ea900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56496/0 (socket says 172.21.15.26:56496) 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 -- 172.21.15.26:0/1335746089 learned_addr learned my addr 172.21.15.26:0/1335746089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272d393640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f272806afc0 0x7f27281ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272db94640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27281536c0 0x7f27281eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 -- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f272806afc0 msgr2=0x7f27281ec130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f272806afc0 0x7f27281ec130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 -- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27281536c0 msgr2=0x7f27281eae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27281536c0 0x7f27281eae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 -- 172.21.15.26:0/1335746089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27281e9e60 con 0x7f272806b3a0 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272d393640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f272806afc0 0x7f27281ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272db94640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27281536c0 0x7f27281eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f272cb92640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806b3a0 0x7f27281ea900 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c8c0 tx=0x7f271c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f2725ffb640 1 -- 172.21.15.26:0/1335746089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c017070 con 0x7f272806b3a0 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.914+0000 7f2725ffb640 1 -- 172.21.15.26:0/1335746089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c003dc0 con 0x7f272806b3a0 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.915+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27281eca70 con 0x7f272806b3a0 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.915+0000 7f2725ffb640 1 -- 172.21.15.26:0/1335746089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0052b0 con 0x7f272806b3a0 2024-03-06T07:32:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.915+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27281ecd50 con 0x7f272806b3a0 2024-03-06T07:32:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.916+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272804f330 con 0x7f272806b3a0 2024-03-06T07:32:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.916+0000 7f2725ffb640 1 -- 172.21.15.26:0/1335746089 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f271c01a070 con 0x7f272806b3a0 2024-03-06T07:32:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.916+0000 7f2725ffb640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f271406b4d0 0x7f271406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.917+0000 7f2725ffb640 1 -- 172.21.15.26:0/1335746089 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f271c089b50 con 0x7f272806b3a0 2024-03-06T07:32:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.917+0000 7f272d393640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f271406b4d0 0x7f271406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.917+0000 7f272d393640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f271406b4d0 0x7f271406d990 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2720036000 tx=0x7f2720008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:48.925+0000 7f2725ffb640 1 -- 172.21.15.26:0/1335746089 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f271c0587d0 con 0x7f272806b3a0 2024-03-06T07:32:49.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.143+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2728068460 con 0x7f272806b3a0 2024-03-06T07:32:49.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.144+0000 7f2725ffb640 1 -- 172.21.15.26:0/1335746089 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f271c0580d0 con 0x7f272806b3a0 2024-03-06T07:32:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:49.145 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:49.146 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.145+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f271406b4d0 msgr2=0x7f271406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.145+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f271406b4d0 0x7f271406d990 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2720036000 tx=0x7f2720008030 comp rx=0 tx=0).stop 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.145+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806b3a0 msgr2=0x7f27281ea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.145+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806b3a0 0x7f27281ea900 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c8c0 tx=0x7f271c00cd80 comp rx=0 tx=0).stop 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.145+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 shutdown_connections 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.146+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f271406b4d0 0x7f271406d990 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.146+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27281536c0 0x7f27281eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.146+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272806b3a0 0x7f27281ea900 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.146+0000 7f272f5f7640 1 --2- 172.21.15.26:0/1335746089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f272806afc0 0x7f27281ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.146+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 >> 172.21.15.26:0/1335746089 conn(0x7f27280c4290 msgr2=0x7f2728152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.146+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 shutdown_connections 2024-03-06T07:32:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:49.146+0000 7f272f5f7640 1 -- 172.21.15.26:0/1335746089 wait complete. 2024-03-06T07:32:49.560 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:49.560 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:50.561 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:50.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.661+0000 7fd5b0c6d640 1 Processor -- start 2024-03-06T07:32:50.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.661+0000 7fd5b0c6d640 1 -- start start 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5b0c6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5ac155c20 0x7fd5ac0c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5b0c6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac06afc0 con 0x7fd5ac0c17c0 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5b0c6d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac06b2a0 con 0x7fd5ac152920 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5b0c6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ac06b580 con 0x7fd5ac155c20 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5a9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5ac155c20 0x7fd5ac0c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5aad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5ac0c17c0 0x7fd5ac15fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43930/0 (socket says 172.21.15.26:43930) 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5aad9d640 1 -- 172.21.15.26:0/1739499575 learned_addr learned my addr 172.21.15.26:0/1739499575 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5a9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5ac155c20 0x7fd5ac0c1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56502/0 (socket says 172.21.15.26:56502) 2024-03-06T07:32:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5a9d9b640 1 -- 172.21.15.26:0/1739499575 >> v1:172.21.15.117:6789/0 conn(0x7fd5ac152920 legacy=0x7fd5ac0c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:50.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5a9d9b640 1 -- 172.21.15.26:0/1739499575 >> v1:172.21.15.26:6789/0 conn(0x7fd5ac0c17c0 legacy=0x7fd5ac15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:50.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5a9d9b640 1 -- 172.21.15.26:0/1739499575 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac06b860 con 0x7fd5ac155c20 2024-03-06T07:32:50.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.662+0000 7fd5a959a640 1 -- 172.21.15.26:0/1739499575 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688488380 0 0) 0x7fd5ac06afc0 con 0x7fd5ac0c17c0 2024-03-06T07:32:50.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.664+0000 7fd5a9d9b640 1 --2- 172.21.15.26:0/1739499575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac0c1280 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00097a0 tx=0x7fd5a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69446e237225b4fc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:50.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.664+0000 7fd5a959a640 1 -- 172.21.15.26:0/1739499575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0035020 con 0x7fd5ac155c20 2024-03-06T07:32:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.664+0000 7fd5a959a640 1 -- 172.21.15.26:0/1739499575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a0031830 con 0x7fd5ac155c20 2024-03-06T07:32:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.664+0000 7fd5a959a640 1 -- 172.21.15.26:0/1739499575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0031c00 con 0x7fd5ac155c20 2024-03-06T07:32:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.664+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/1739499575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 msgr2=0x7fd5ac0c1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.664+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/1739499575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac0c1280 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00097a0 tx=0x7fd5a002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.665+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/1739499575 shutdown_connections 2024-03-06T07:32:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.665+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/1739499575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac0c1280 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.665+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/1739499575 >> 172.21.15.26:0/1739499575 conn(0x7fd5ac0c4290 msgr2=0x7fd5ac0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.665+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/1739499575 shutdown_connections 2024-03-06T07:32:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.665+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/1739499575 wait complete. 2024-03-06T07:32:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.666+0000 7fd5b0c6d640 1 Processor -- start 2024-03-06T07:32:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.666+0000 7fd5b0c6d640 1 -- start start 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac0c17c0 0x7fd5ac1ee400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac152920 0x7fd5ac1479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0c28a0 con 0x7fd5ac155c20 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1560b0 con 0x7fd5ac0c17c0 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0c2a20 con 0x7fd5ac152920 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac147f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56504/0 (socket says 172.21.15.26:56504) 2024-03-06T07:32:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 -- 172.21.15.26:0/2823485134 learned_addr learned my addr 172.21.15.26:0/2823485134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aa59c640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac0c17c0 0x7fd5ac1ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5a9d9b640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac152920 0x7fd5ac1479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 -- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac152920 msgr2=0x7fd5ac1479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac152920 0x7fd5ac1479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 -- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac0c17c0 msgr2=0x7fd5ac1ee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac0c17c0 0x7fd5ac1ee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 -- 172.21.15.26:0/2823485134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac1ef0d0 con 0x7fd5ac155c20 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5a9d9b640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac152920 0x7fd5ac1479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5aad9d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac147f10 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400c7a0 tx=0x7fd5a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd59affd640 1 -- 172.21.15.26:0/2823485134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4018070 con 0x7fd5ac155c20 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd59affd640 1 -- 172.21.15.26:0/2823485134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a400a640 con 0x7fd5ac155c20 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac1ec090 con 0x7fd5ac155c20 2024-03-06T07:32:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd59affd640 1 -- 172.21.15.26:0/2823485134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400aab0 con 0x7fd5ac155c20 2024-03-06T07:32:50.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.668+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac0aa760 con 0x7fd5ac155c20 2024-03-06T07:32:50.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.669+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac04f330 con 0x7fd5ac155c20 2024-03-06T07:32:50.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.670+0000 7fd59affd640 1 -- 172.21.15.26:0/2823485134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5a401b070 con 0x7fd5ac155c20 2024-03-06T07:32:50.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.670+0000 7fd59affd640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58c06b500 0x7fd58c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:50.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.670+0000 7fd59affd640 1 -- 172.21.15.26:0/2823485134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd5a4089950 con 0x7fd5ac155c20 2024-03-06T07:32:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.671+0000 7fd5aa59c640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58c06b500 0x7fd58c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.671+0000 7fd5aa59c640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58c06b500 0x7fd58c06d9c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0017f0 tx=0x7fd59c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.675+0000 7fd59affd640 1 -- 172.21.15.26:0/2823485134 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5a4057f80 con 0x7fd5ac155c20 2024-03-06T07:32:50.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.896+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5ac068460 con 0x7fd5ac155c20 2024-03-06T07:32:50.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.896+0000 7fd59affd640 1 -- 172.21.15.26:0/2823485134 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fd5a405c020 con 0x7fd5ac155c20 2024-03-06T07:32:50.898 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:50.898 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:50.899 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58c06b500 msgr2=0x7fd58c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58c06b500 0x7fd58c06d9c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0017f0 tx=0x7fd59c009040 comp rx=0 tx=0).stop 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 msgr2=0x7fd5ac147f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac147f10 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400c7a0 tx=0x7fd5a400cc60 comp rx=0 tx=0).stop 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 shutdown_connections 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd58c06b500 0x7fd58c06d9c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5ac155c20 0x7fd5ac147f10 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5ac152920 0x7fd5ac1479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 --2- 172.21.15.26:0/2823485134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5ac0c17c0 0x7fd5ac1ee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 >> 172.21.15.26:0/2823485134 conn(0x7fd5ac0c4290 msgr2=0x7fd5ac0c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 shutdown_connections 2024-03-06T07:32:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:50.898+0000 7fd5b0c6d640 1 -- 172.21.15.26:0/2823485134 wait complete. 2024-03-06T07:32:51.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:51.313 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:52.314 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d09f81640 1 Processor -- start 2024-03-06T07:32:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d09f81640 1 -- start start 2024-03-06T07:32:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d09f81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 0x7f4d04155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d09f81640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d04158b60 0x7f4d04158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d09f81640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040c73d0 con 0x7f4d040c88c0 2024-03-06T07:32:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d09f81640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040c8dd0 con 0x7f4d04158b60 2024-03-06T07:32:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d09f81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040be2e0 con 0x7f4d0414f360 2024-03-06T07:32:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d02ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4d040c88c0 0x7f4d040c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43942/0 (socket says 172.21.15.26:43942) 2024-03-06T07:32:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.414+0000 7f4d02ffd640 1 -- 172.21.15.26:0/602674038 learned_addr learned my addr 172.21.15.26:0/602674038 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d03fff640 1 --2- 172.21.15.26:0/602674038 >> v2:172.21.15.117:3300/0 conn(0x7f4d04158b60 0x7f4d04158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d037fe640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 0x7f4d04155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289033461 0 0) 0x7f4d040c73d0 con 0x7f4d040c88c0 2024-03-06T07:32:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cec003520 con 0x7f4d040c88c0 2024-03-06T07:32:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623908475 0 0) 0x7f4cec003520 con 0x7f4d040c88c0 2024-03-06T07:32:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 msgr2=0x7f4d04155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 0x7f4d04155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 >> v2:172.21.15.117:3300/0 conn(0x7f4d04158b60 msgr2=0x7f4d04158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 --2- 172.21.15.26:0/602674038 >> v2:172.21.15.117:3300/0 conn(0x7f4d04158b60 0x7f4d04158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040c7670 con 0x7f4d040c88c0 2024-03-06T07:32:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f4cf8001670 con 0x7f4d040c88c0 2024-03-06T07:32:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 >> v1:172.21.15.26:6789/0 conn(0x7f4d040c88c0 legacy=0x7f4d040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 0x7f4cec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec003f30 0x7f4cec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cec004d40 0x7f4cec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040be2e0 con 0x7f4cec003f30 2024-03-06T07:32:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec00b9c0 con 0x7f4cec004d40 2024-03-06T07:32:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec00bb90 con 0x7f4d0414f360 2024-03-06T07:32:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d03fff640 1 --2- 172.21.15.26:0/602674038 >> v2:172.21.15.117:3300/0 conn(0x7f4d04158b60 0x7f4d04158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:32:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d02ffd640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec003f30 0x7f4cec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d037fe640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 0x7f4cec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.415+0000 7f4d03fff640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cec004d40 0x7f4cec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d02ffd640 1 -- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 msgr2=0x7f4cec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d02ffd640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0414f360 0x7f4cec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d02ffd640 1 -- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cec004d40 msgr2=0x7f4cec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d02ffd640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cec004d40 0x7f4cec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d02ffd640 1 -- 172.21.15.26:0/602674038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec003520 con 0x7f4cec003f30 2024-03-06T07:32:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d03fff640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cec004d40 0x7f4cec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d02ffd640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec003f30 0x7f4cec004800 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800c210 tx=0x7f4cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be08d16d71bdb7c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8007d50 con 0x7f4cec003f30 2024-03-06T07:32:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf8030070 con 0x7f4cec003f30 2024-03-06T07:32:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.416+0000 7f4d027fc640 1 -- 172.21.15.26:0/602674038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf802cd80 con 0x7f4cec003f30 2024-03-06T07:32:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.417+0000 7f4d09f81640 1 -- 172.21.15.26:0/602674038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec003f30 msgr2=0x7f4cec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.417+0000 7f4d09f81640 1 --2- 172.21.15.26:0/602674038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec003f30 0x7f4cec004800 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800c210 tx=0x7f4cec005830 comp rx=0 tx=0).stop 2024-03-06T07:32:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.417+0000 7f4d03fff640 1 -- 172.21.15.26:0/602674038 reap_dead start 2024-03-06T07:32:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.417+0000 7f4d09f81640 1 -- 172.21.15.26:0/602674038 shutdown_connections 2024-03-06T07:32:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.417+0000 7f4d09f81640 1 -- 172.21.15.26:0/602674038 >> 172.21.15.26:0/602674038 conn(0x7f4d040ac570 msgr2=0x7f4d040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.417+0000 7f4d09f81640 1 -- 172.21.15.26:0/602674038 shutdown_connections 2024-03-06T07:32:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.417+0000 7f4d09f81640 1 -- 172.21.15.26:0/602674038 wait complete. 2024-03-06T07:32:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.419+0000 7f4d09f81640 1 Processor -- start 2024-03-06T07:32:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.419+0000 7f4d09f81640 1 -- start start 2024-03-06T07:32:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d09f81640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cec003f30 0x7f4d04154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d09f81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d040ad320 0x7f4d040af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d09f81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d040c4a70 0x7f4d040af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d09f81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c5a50 con 0x7f4d040ad320 2024-03-06T07:32:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d09f81640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c7c30 con 0x7f4d040c4a70 2024-03-06T07:32:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d09f81640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c5bd0 con 0x7f4cec003f30 2024-03-06T07:32:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d040ad320 0x7f4d040af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d040ad320 0x7f4d040af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56512/0 (socket says 172.21.15.26:56512) 2024-03-06T07:32:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 -- 172.21.15.26:0/3996732935 learned_addr learned my addr 172.21.15.26:0/3996732935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d037fe640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cec003f30 0x7f4d04154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d03fff640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d040c4a70 0x7f4d040af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 -- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cec003f30 msgr2=0x7f4d04154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cec003f30 0x7f4d04154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 -- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d040c4a70 msgr2=0x7f4d040af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d040c4a70 0x7f4d040af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d02ffd640 1 -- 172.21.15.26:0/3996732935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040ada90 con 0x7f4d040ad320 2024-03-06T07:32:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.420+0000 7f4d03fff640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d040c4a70 0x7f4d040af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.421+0000 7f4d02ffd640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d040ad320 0x7f4d040af310 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802c550 tx=0x7f4cf8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.421+0000 7f4cebfff640 1 -- 172.21.15.26:0/3996732935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8045070 con 0x7f4d040ad320 2024-03-06T07:32:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.421+0000 7f4cebfff640 1 -- 172.21.15.26:0/3996732935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf8004bf0 con 0x7f4d040ad320 2024-03-06T07:32:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.421+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d040add70 con 0x7f4d040ad320 2024-03-06T07:32:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.421+0000 7f4cebfff640 1 -- 172.21.15.26:0/3996732935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8001c90 con 0x7f4d040ad320 2024-03-06T07:32:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.421+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d041eeb90 con 0x7f4d040ad320 2024-03-06T07:32:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.422+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040ac9e0 con 0x7f4d040ad320 2024-03-06T07:32:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.422+0000 7f4cebfff640 1 -- 172.21.15.26:0/3996732935 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cf8007070 con 0x7f4d040ad320 2024-03-06T07:32:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.423+0000 7f4cebfff640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cf006b4d0 0x7f4cf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.423+0000 7f4cebfff640 1 -- 172.21.15.26:0/3996732935 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f4cf80ab3e0 con 0x7f4d040ad320 2024-03-06T07:32:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.423+0000 7f4d037fe640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cf006b4d0 0x7f4cf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.423+0000 7f4d037fe640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cf006b4d0 0x7f4cf006d990 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009ae0 tx=0x7f4cf4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.427+0000 7f4cebfff640 1 -- 172.21.15.26:0/3996732935 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d040ac9e0 con 0x7f4d040ad320 2024-03-06T07:32:52.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.774+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d0404f330 con 0x7f4d040ad320 2024-03-06T07:32:52.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.774+0000 7f4cebfff640 1 -- 172.21.15.26:0/3996732935 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f4cf8079bb0 con 0x7f4d040ad320 2024-03-06T07:32:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:52.777 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cf006b4d0 msgr2=0x7f4cf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cf006b4d0 0x7f4cf006d990 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009ae0 tx=0x7f4cf4009190 comp rx=0 tx=0).stop 2024-03-06T07:32:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d040ad320 msgr2=0x7f4d040af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d040ad320 0x7f4d040af310 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802c550 tx=0x7f4cf8001a40 comp rx=0 tx=0).stop 2024-03-06T07:32:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 shutdown_connections 2024-03-06T07:32:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4cf006b4d0 0x7f4cf006d990 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d040c4a70 0x7f4d040af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d040ad320 0x7f4d040af310 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 --2- 172.21.15.26:0/3996732935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cec003f30 0x7f4d04154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 >> 172.21.15.26:0/3996732935 conn(0x7f4d040ac570 msgr2=0x7f4d040c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.776+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 shutdown_connections 2024-03-06T07:32:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:52.777+0000 7f4d09f81640 1 -- 172.21.15.26:0/3996732935 wait complete. 2024-03-06T07:32:53.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:53.192 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:54.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:54.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.293+0000 7feac773e640 1 Processor -- start 2024-03-06T07:32:54.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.293+0000 7feac773e640 1 -- start start 2024-03-06T07:32:54.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac773e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feac00c88c0 0x7feac0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac773e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac00c73d0 con 0x7feac0158b60 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac773e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac00be2e0 con 0x7feac014f360 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac773e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac0159700 con 0x7feac00c88c0 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac4cd9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feac00c88c0 0x7feac0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac4cd9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feac00c88c0 0x7feac0155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56522/0 (socket says 172.21.15.26:56522) 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac5cdb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feac0158b60 0x7feac0158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43950/0 (socket says 172.21.15.26:43950) 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac4cd9640 1 -- 172.21.15.26:0/192709612 learned_addr learned my addr 172.21.15.26:0/192709612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac4cd9640 1 -- 172.21.15.26:0/192709612 >> v1:172.21.15.117:6789/0 conn(0x7feac014f360 legacy=0x7feac00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac4cd9640 1 -- 172.21.15.26:0/192709612 >> v1:172.21.15.26:6789/0 conn(0x7feac0158b60 legacy=0x7feac0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.294+0000 7feac4cd9640 1 -- 172.21.15.26:0/192709612 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00c7670 con 0x7feac00c88c0 2024-03-06T07:32:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.295+0000 7feac4cd9640 1 --2- 172.21.15.26:0/192709612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac00c88c0 0x7feac0155c10 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7feab0009870 tx=0x7feab002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afade7b8c5dcf1d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.295+0000 7feab7fff640 1 -- 172.21.15.26:0/192709612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0034070 con 0x7feac00c88c0 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.295+0000 7feab7fff640 1 -- 172.21.15.26:0/192709612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab0030860 con 0x7feac00c88c0 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.295+0000 7feab7fff640 1 -- 172.21.15.26:0/192709612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0030c30 con 0x7feac00c88c0 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.296+0000 7feac773e640 1 -- 172.21.15.26:0/192709612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac00c88c0 msgr2=0x7feac0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.296+0000 7feac773e640 1 --2- 172.21.15.26:0/192709612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac00c88c0 0x7feac0155c10 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7feab0009870 tx=0x7feab002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.296+0000 7feac773e640 1 -- 172.21.15.26:0/192709612 shutdown_connections 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.296+0000 7feac773e640 1 --2- 172.21.15.26:0/192709612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac00c88c0 0x7feac0155c10 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.296+0000 7feac773e640 1 -- 172.21.15.26:0/192709612 >> 172.21.15.26:0/192709612 conn(0x7feac00ac570 msgr2=0x7feac00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.296+0000 7feac773e640 1 -- 172.21.15.26:0/192709612 shutdown_connections 2024-03-06T07:32:54.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.296+0000 7feac773e640 1 -- 172.21.15.26:0/192709612 wait complete. 2024-03-06T07:32:54.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.297+0000 7feac773e640 1 Processor -- start 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.298+0000 7feac773e640 1 -- start start 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac773e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feac00c88c0 0x7feac00ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac773e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac014f360 0x7feac00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac773e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac0158b60 0x7feac014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac773e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c7c30 con 0x7feac0158b60 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac773e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c5a50 con 0x7feac014f360 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac773e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c5bd0 con 0x7feac00c88c0 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac0158b60 0x7feac014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac0158b60 0x7feac014aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56526/0 (socket says 172.21.15.26:56526) 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 -- 172.21.15.26:0/4052012450 learned_addr learned my addr 172.21.15.26:0/4052012450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac4cd9640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac014f360 0x7feac00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac54da640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feac00c88c0 0x7feac00ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 -- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feac00c88c0 msgr2=0x7feac00ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feac00c88c0 0x7feac00ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 -- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac014f360 msgr2=0x7feac00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac014f360 0x7feac00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 -- 172.21.15.26:0/4052012450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0159270 con 0x7feac0158b60 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac4cd9640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac014f360 0x7feac00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac54da640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feac00c88c0 0x7feac00ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac5cdb640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac0158b60 0x7feac014aad0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7feab800c730 tx=0x7feab800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feab5ffb640 1 -- 172.21.15.26:0/4052012450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8017070 con 0x7feac0158b60 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feab5ffb640 1 -- 172.21.15.26:0/4052012450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab800ce40 con 0x7feac0158b60 2024-03-06T07:32:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac00ad420 con 0x7feac0158b60 2024-03-06T07:32:54.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.299+0000 7feab5ffb640 1 -- 172.21.15.26:0/4052012450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800a840 con 0x7feac0158b60 2024-03-06T07:32:54.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.300+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac00b0090 con 0x7feac0158b60 2024-03-06T07:32:54.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.301+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac01ec1c0 con 0x7feac0158b60 2024-03-06T07:32:54.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.301+0000 7feab5ffb640 1 -- 172.21.15.26:0/4052012450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feab801a070 con 0x7feac0158b60 2024-03-06T07:32:54.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.301+0000 7feab5ffb640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feaa406b4d0 0x7feaa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.301+0000 7feab5ffb640 1 -- 172.21.15.26:0/4052012450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7feab8089860 con 0x7feac0158b60 2024-03-06T07:32:54.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.302+0000 7feac54da640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feaa406b4d0 0x7feaa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.302+0000 7feac54da640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feaa406b4d0 0x7feaa406d990 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7feac01e9100 tx=0x7feabc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.310+0000 7feab5ffb640 1 -- 172.21.15.26:0/4052012450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feac01ec1c0 con 0x7feac0158b60 2024-03-06T07:32:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.528+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feac004f330 con 0x7feac0158b60 2024-03-06T07:32:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.529+0000 7feab5ffb640 1 -- 172.21.15.26:0/4052012450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7feab8057e90 con 0x7feac0158b60 2024-03-06T07:32:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:54.531 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:54.531 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feaa406b4d0 msgr2=0x7feaa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feaa406b4d0 0x7feaa406d990 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7feac01e9100 tx=0x7feabc009040 comp rx=0 tx=0).stop 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac0158b60 msgr2=0x7feac014aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac0158b60 0x7feac014aad0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7feab800c730 tx=0x7feab800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 shutdown_connections 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feaa406b4d0 0x7feaa406d990 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feac0158b60 0x7feac014aad0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feac014f360 0x7feac00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 --2- 172.21.15.26:0/4052012450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feac00c88c0 0x7feac00ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 >> 172.21.15.26:0/4052012450 conn(0x7feac00ac570 msgr2=0x7feac00abde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 shutdown_connections 2024-03-06T07:32:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:54.531+0000 7feac773e640 1 -- 172.21.15.26:0/4052012450 wait complete. 2024-03-06T07:32:54.946 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:54.946 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:55.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.087+0000 7f708fa9d640 1 Processor -- start 2024-03-06T07:32:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.087+0000 7f708fa9d640 1 -- start start 2024-03-06T07:32:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708fa9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7088155c80 0x7f708806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708fa9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880720a0 0x7f708806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708fa9d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70880c1ae0 con 0x7f7088152520 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708fa9d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70880c1dc0 con 0x7f7088155c80 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708fa9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708806e450 con 0x7f70880720a0 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708d038640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7088155c80 0x7f708806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708d038640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7088155c80 0x7f708806ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56536/0 (socket says 172.21.15.26:56536) 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708d038640 1 -- 172.21.15.26:0/3855979108 learned_addr learned my addr 172.21.15.26:0/3855979108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.088+0000 7f708e03a640 1 --2- 172.21.15.26:0/3855979108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880720a0 0x7f708806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.089+0000 7f708d038640 1 -- 172.21.15.26:0/3855979108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880720a0 msgr2=0x7f708806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.089+0000 7f708d038640 1 --2- 172.21.15.26:0/3855979108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880720a0 0x7f708806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.089+0000 7f708d038640 1 -- 172.21.15.26:0/3855979108 >> v1:172.21.15.117:6789/0 conn(0x7f7088152520 legacy=0x7f70880c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.089+0000 7f708d038640 1 -- 172.21.15.26:0/3855979108 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708806e730 con 0x7f7088155c80 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.089+0000 7f708d038640 1 --2- 172.21.15.26:0/3855979108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7088155c80 0x7f708806ef30 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6378578355d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708c837640 1 -- 172.21.15.26:0/3855979108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c034070 con 0x7f7088155c80 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708c837640 1 -- 172.21.15.26:0/3855979108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c030840 con 0x7f7088155c80 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708c837640 1 -- 172.21.15.26:0/3855979108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c030c10 con 0x7f7088155c80 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 -- 172.21.15.26:0/3855979108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7088155c80 msgr2=0x7f708806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 --2- 172.21.15.26:0/3855979108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7088155c80 0x7f708806ef30 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c02b980 comp rx=0 tx=0).stop 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 -- 172.21.15.26:0/3855979108 shutdown_connections 2024-03-06T07:32:56.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 --2- 172.21.15.26:0/3855979108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880720a0 0x7f708806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 --2- 172.21.15.26:0/3855979108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7088155c80 0x7f708806ef30 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 -- 172.21.15.26:0/3855979108 >> 172.21.15.26:0/3855979108 conn(0x7f70880c76b0 msgr2=0x7f70880c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 -- 172.21.15.26:0/3855979108 shutdown_connections 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.090+0000 7f708fa9d640 1 -- 172.21.15.26:0/3855979108 wait complete. 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.091+0000 7f708fa9d640 1 Processor -- start 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.091+0000 7f708fa9d640 1 -- start start 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708fa9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70880720a0 0x7f70881e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708fa9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7088152520 0x7f7088157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708fa9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088155c80 0x7f7088157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708fa9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088152a80 con 0x7f70880720a0 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708fa9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088152900 con 0x7f7088155c80 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708fa9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708806b500 con 0x7f7088152520 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70880720a0 0x7f70881e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70880720a0 0x7f70881e1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56550/0 (socket says 172.21.15.26:56550) 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 -- 172.21.15.26:0/2026316167 learned_addr learned my addr 172.21.15.26:0/2026316167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d038640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7088152520 0x7f7088157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708e03a640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088155c80 0x7f7088157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 -- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7088152520 msgr2=0x7f7088157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7088152520 0x7f7088157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 -- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088155c80 msgr2=0x7f7088157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088155c80 0x7f7088157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.092+0000 7f708d839640 1 -- 172.21.15.26:0/2026316167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7088155290 con 0x7f70880720a0 2024-03-06T07:32:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.093+0000 7f708d839640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70880720a0 0x7f70881e1f70 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f708000c400 tx=0x7f708000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.093+0000 7f70867fc640 1 -- 172.21.15.26:0/2026316167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800052d0 con 0x7f70880720a0 2024-03-06T07:32:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.093+0000 7f70867fc640 1 -- 172.21.15.26:0/2026316167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080005490 con 0x7f70880720a0 2024-03-06T07:32:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.093+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70881eeb30 con 0x7f70880720a0 2024-03-06T07:32:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.093+0000 7f70867fc640 1 -- 172.21.15.26:0/2026316167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080007260 con 0x7f70880720a0 2024-03-06T07:32:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.093+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7088156840 con 0x7f70880720a0 2024-03-06T07:32:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.094+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708804f330 con 0x7f70880720a0 2024-03-06T07:32:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.094+0000 7f70867fc640 1 -- 172.21.15.26:0/2026316167 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7080007400 con 0x7f70880720a0 2024-03-06T07:32:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.095+0000 7f70867fc640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f707406b4d0 0x7f707406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.095+0000 7f70867fc640 1 -- 172.21.15.26:0/2026316167 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f7080089960 con 0x7f70880720a0 2024-03-06T07:32:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.095+0000 7f708d038640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f707406b4d0 0x7f707406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.096+0000 7f708d038640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f707406b4d0 0x7f707406d990 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f707c036000 tx=0x7f707c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:56.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.099+0000 7f70867fc640 1 -- 172.21.15.26:0/2026316167 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7088155290 con 0x7f70880720a0 2024-03-06T07:32:56.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.317+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70880c0b70 con 0x7f70880720a0 2024-03-06T07:32:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.318+0000 7f70867fc640 1 -- 172.21.15.26:0/2026316167 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f7080057f00 con 0x7f70880720a0 2024-03-06T07:32:56.319 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:56.319 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f707406b4d0 msgr2=0x7f707406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f707406b4d0 0x7f707406d990 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f707c036000 tx=0x7f707c008030 comp rx=0 tx=0).stop 2024-03-06T07:32:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70880720a0 msgr2=0x7f70881e1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70880720a0 0x7f70881e1f70 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f708000c400 tx=0x7f708000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 shutdown_connections 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f707406b4d0 0x7f707406d990 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088155c80 0x7f7088157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7088152520 0x7f7088157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.319+0000 7f708fa9d640 1 --2- 172.21.15.26:0/2026316167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70880720a0 0x7f70881e1f70 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.320+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 >> 172.21.15.26:0/2026316167 conn(0x7f70880c76b0 msgr2=0x7f70881de640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.320+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 shutdown_connections 2024-03-06T07:32:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:56.320+0000 7f708fa9d640 1 -- 172.21.15.26:0/2026316167 wait complete. 2024-03-06T07:32:56.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:56.735 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:57.736 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.837+0000 7f1d84c4e640 1 Processor -- start 2024-03-06T07:32:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.837+0000 7f1d84c4e640 1 -- start start 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d84c4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1d80072310 0x7f1d800aa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d84c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d8015c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d84c4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d80072fe0 con 0x7f1d8014f620 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d84c4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d800ac580 con 0x7f1d80072310 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d84c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800ac940 con 0x7f1d800aaeb0 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d7dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1d80072310 0x7f1d800aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d7e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1d8014f620 0x7f1d80071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57824/0 (socket says 172.21.15.26:57824) 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d7dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1d80072310 0x7f1d800aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60594/0 (socket says 172.21.15.26:60594) 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.838+0000 7f1d7e59c640 1 -- 172.21.15.26:0/2614478652 learned_addr learned my addr 172.21.15.26:0/2614478652 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.839+0000 7f1d7dd9b640 1 -- 172.21.15.26:0/2614478652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 msgr2=0x7f1d8015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.839+0000 7f1d7ed9d640 1 --2- 172.21.15.26:0/2614478652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d8015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.839+0000 7f1d7dd9b640 1 --2- 172.21.15.26:0/2614478652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d8015c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.839+0000 7f1d7d59a640 1 -- 172.21.15.26:0/2614478652 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239857825 0 0) 0x7f1d80072fe0 con 0x7f1d8014f620 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.839+0000 7f1d7dd9b640 1 -- 172.21.15.26:0/2614478652 >> v1:172.21.15.26:6789/0 conn(0x7f1d8014f620 legacy=0x7f1d80071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.839+0000 7f1d7dd9b640 1 -- 172.21.15.26:0/2614478652 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800acb90 con 0x7f1d80072310 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.839+0000 7f1d7dd9b640 1 --2- 172.21.15.26:0/2614478652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d800aa970 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f1d740097a0 tx=0x7f1d7402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9554c50b0b57fae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d7d59a640 1 -- 172.21.15.26:0/2614478652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74038070 con 0x7f1d80072310 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d7d59a640 1 -- 172.21.15.26:0/2614478652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d74031700 con 0x7f1d80072310 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d7d59a640 1 -- 172.21.15.26:0/2614478652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74031ad0 con 0x7f1d80072310 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2614478652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 msgr2=0x7f1d800aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2614478652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d800aa970 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f1d740097a0 tx=0x7f1d7402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2614478652 shutdown_connections 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2614478652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d800aa970 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2614478652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d8015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2614478652 >> 172.21.15.26:0/2614478652 conn(0x7f1d8006bf00 msgr2=0x7f1d800c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2614478652 shutdown_connections 2024-03-06T07:32:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.840+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2614478652 wait complete. 2024-03-06T07:32:57.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.842+0000 7f1d84c4e640 1 Processor -- start 2024-03-06T07:32:57.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.842+0000 7f1d84c4e640 1 -- start start 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d84c4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d801ebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d84c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d801ec230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d84c4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d8014f620 0x7f1d801551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d84c4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800c3060 con 0x7f1d80072310 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d84c4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800c3790 con 0x7f1d8014f620 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d84c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800c31e0 con 0x7f1d800aaeb0 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d801ebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d801ebcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60606/0 (socket says 172.21.15.26:60606) 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 -- 172.21.15.26:0/2039422676 learned_addr learned my addr 172.21.15.26:0/2039422676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7dd9b640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d801ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 -- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 msgr2=0x7f1d801ec230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d801ec230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 -- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d8014f620 msgr2=0x7f1d801551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d8014f620 0x7f1d801551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 -- 172.21.15.26:0/2039422676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800aac80 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.843+0000 7f1d7e59c640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d801ebcf0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c4d0 tx=0x7f1d7000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.844+0000 7f1d6effd640 1 -- 172.21.15.26:0/2039422676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70018070 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.844+0000 7f1d6effd640 1 -- 172.21.15.26:0/2039422676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d70007500 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.844+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d80154cf0 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.844+0000 7f1d6effd640 1 -- 172.21.15.26:0/2039422676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70004a20 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.844+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d801dbf60 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.845+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d80154ec0 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.845+0000 7f1d6effd640 1 -- 172.21.15.26:0/2039422676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d7001b070 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.845+0000 7f1d6effd640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d6006b500 0x7f1d6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.846+0000 7f1d6effd640 1 -- 172.21.15.26:0/2039422676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f1d70089750 con 0x7f1d80072310 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.846+0000 7f1d7dd9b640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d6006b500 0x7f1d6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:57.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.846+0000 7f1d7dd9b640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d6006b500 0x7f1d6006d9c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1d74035000 tx=0x7f1d74008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:57.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:57.850+0000 7f1d6effd640 1 -- 172.21.15.26:0/2039422676 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1d70058340 con 0x7f1d80072310 2024-03-06T07:32:58.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.112+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d801ec900 con 0x7f1d80072310 2024-03-06T07:32:58.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.112+0000 7f1d6effd640 1 -- 172.21.15.26:0/2039422676 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f1d70057c40 con 0x7f1d80072310 2024-03-06T07:32:58.114 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:32:58.114 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:32:58.114 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:32:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d6006b500 msgr2=0x7f1d6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d6006b500 0x7f1d6006d9c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1d74035000 tx=0x7f1d74008030 comp rx=0 tx=0).stop 2024-03-06T07:32:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 msgr2=0x7f1d801ebcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d801ebcf0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c4d0 tx=0x7f1d7000c990 comp rx=0 tx=0).stop 2024-03-06T07:32:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 shutdown_connections 2024-03-06T07:32:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f1d6006b500 0x7f1d6006d9c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d8014f620 0x7f1d801551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d800aaeb0 0x7f1d801ec230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 --2- 172.21.15.26:0/2039422676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d80072310 0x7f1d801ebcf0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 >> 172.21.15.26:0/2039422676 conn(0x7f1d8006bf00 msgr2=0x7f1d80151ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 shutdown_connections 2024-03-06T07:32:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:58.114+0000 7f1d84c4e640 1 -- 172.21.15.26:0/2039422676 wait complete. 2024-03-06T07:32:58.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 215, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378208+0000', 'modified': '2024-03-06T07:31:04.679573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10737, 'mds_1': 8327}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8327': {'gid': 8327, 'name': 'a', 'rank': 1, 'incarnation': 209, 'state': 'up:stopping', 'state_seq': 24, 'addr': '172.21.15.26:6834/741758816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 741758816}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 741758816}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:04.679573+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10737': {'gid': 10737, 'name': 'c', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/4219086401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 4219086401}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 4219086401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10737, 'qdb_cluster': [10737]} 2024-03-06T07:32:58.530 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8327:up:stopping 2024-03-06T07:32:59.532 DEBUG:tasks.cephfs.filesystem:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 16, "epoch": 215, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 16, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:29:14.378208+0000", "damaged": [], "data_pools": [ 34 ], "enabled": true, "epoch": 215, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_10737": { "addr": "172.21.15.26:6835/4219086401", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6833", "nonce": 4219086401, "type": "v2" }, { "addr": "172.21.15.26:6835", "nonce": 4219086401, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 10737, "incarnation": 196, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 }, "gid_8327": { "addr": "172.21.15.26:6834/741758816", "addrs": { "addrvec": [ { "addr": "172.21.15.26:6832", "nonce": 741758816, "type": "v2" }, { "addr": "172.21.15.26:6834", "nonce": 741758816, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8327, "incarnation": 209, "join_fscid": -1, "laggy_since": "2024-03-06T07:31:04.679573+0000", "name": "a", "rank": 1, "state": "up:stopping", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 215, "max_file_size": 1099511627776, "max_mds": 0, "max_xattr_size": 65536, "metadata_pool": 33, "modified": "2024-03-06T07:31:04.679573+0000", "qdb_cluster": [ 10737 ], "qdb_leader": 10737, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 10737, "mds_1": 8327 } } } ], "standbys": [] } 2024-03-06T07:32:59.532 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:32:59.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:32:59.569 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:32:59.619 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:32:59.620 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:32:59.620 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:32:59.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:32:59.635 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:32:59.635 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:32:59.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:32:59.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.775+0000 7fe04e513640 1 Processor -- start 2024-03-06T07:32:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.775+0000 7fe04e513640 1 -- start start 2024-03-06T07:32:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04e513640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe0481506b0 0x7fe048153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04e513640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0480acc00 0x7fe0480c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04e513640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe048070aa0 con 0x7fe048156ea0 2024-03-06T07:32:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04e513640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe048156450 con 0x7fe0480acc00 2024-03-06T07:32:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04e513640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe048156a10 con 0x7fe0481506b0 2024-03-06T07:32:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04cab0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0480acc00 0x7fe0480c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe0477fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe048156ea0 0x7fe0480702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57834/0 (socket says 172.21.15.26:57834) 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe0477fe640 1 -- 172.21.15.26:0/2822190716 learned_addr learned my addr 172.21.15.26:0/2822190716 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe047fff640 1 --2- 172.21.15.26:0/2822190716 >> v2:172.21.15.117:3300/0 conn(0x7fe0481506b0 0x7fe048153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04cab0640 1 -- 172.21.15.26:0/2822190716 >> v2:172.21.15.117:3300/0 conn(0x7fe0481506b0 msgr2=0x7fe048153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04cab0640 1 --2- 172.21.15.26:0/2822190716 >> v2:172.21.15.117:3300/0 conn(0x7fe0481506b0 0x7fe048153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04cab0640 1 -- 172.21.15.26:0/2822190716 >> v1:172.21.15.26:6789/0 conn(0x7fe048156ea0 legacy=0x7fe0480702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe046ffd640 1 -- 172.21.15.26:0/2822190716 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107028100 0 0) 0x7fe048070aa0 con 0x7fe048156ea0 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.776+0000 7fe04cab0640 1 -- 172.21.15.26:0/2822190716 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048070d80 con 0x7fe0480acc00 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.777+0000 7fe04cab0640 1 --2- 172.21.15.26:0/2822190716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe0480c54a0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fe0480c97e0 tx=0x7fe04002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d8e7f84569fe07 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.777+0000 7fe046ffd640 1 -- 172.21.15.26:0/2822190716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040039070 con 0x7fe0480acc00 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.777+0000 7fe046ffd640 1 -- 172.21.15.26:0/2822190716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04002cc80 con 0x7fe0480acc00 2024-03-06T07:32:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.777+0000 7fe046ffd640 1 -- 172.21.15.26:0/2822190716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040034660 con 0x7fe0480acc00 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 -- 172.21.15.26:0/2822190716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 msgr2=0x7fe0480c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 --2- 172.21.15.26:0/2822190716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe0480c54a0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fe0480c97e0 tx=0x7fe04002bed0 comp rx=0 tx=0).stop 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 -- 172.21.15.26:0/2822190716 shutdown_connections 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 --2- 172.21.15.26:0/2822190716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe0480c54a0 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 --2- 172.21.15.26:0/2822190716 >> v2:172.21.15.117:3300/0 conn(0x7fe0481506b0 0x7fe048153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 -- 172.21.15.26:0/2822190716 >> 172.21.15.26:0/2822190716 conn(0x7fe04806b1a0 msgr2=0x7fe04806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 -- 172.21.15.26:0/2822190716 shutdown_connections 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.778+0000 7fe04e513640 1 -- 172.21.15.26:0/2822190716 wait complete. 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.779+0000 7fe04e513640 1 Processor -- start 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.779+0000 7fe04e513640 1 -- start start 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04e513640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe048145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04e513640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0481506b0 0x7fe0481512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04e513640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048156ea0 0x7fe0480c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04e513640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480ad1d0 con 0x7fe0480acc00 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04e513640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480ad050 con 0x7fe048156ea0 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04e513640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480ad350 con 0x7fe0481506b0 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe048145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe048145ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60632/0 (socket says 172.21.15.26:60632) 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 -- 172.21.15.26:0/4230058018 learned_addr learned my addr 172.21.15.26:0/4230058018 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04cab0640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048156ea0 0x7fe0480c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 -- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0481506b0 msgr2=0x7fe0481512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe0477fe640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0481506b0 0x7fe0481512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0481506b0 0x7fe0481512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 -- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048156ea0 msgr2=0x7fe0480c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048156ea0 0x7fe0480c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 -- 172.21.15.26:0/4230058018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0481eb5b0 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe04cab0640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048156ea0 0x7fe0480c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe0477fe640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0481506b0 0x7fe0481512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.780+0000 7fe047fff640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe048145ea0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fe03800d800 tx=0x7fe03800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.781+0000 7fe044ff9640 1 -- 172.21.15.26:0/4230058018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800b6c0 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.781+0000 7fe044ff9640 1 -- 172.21.15.26:0/4230058018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038019070 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.781+0000 7fe044ff9640 1 -- 172.21.15.26:0/4230058018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800ba80 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.781+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0481ea7f0 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.781+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481eadb0 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.782+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04804f330 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.782+0000 7fe044ff9640 1 -- 172.21.15.26:0/4230058018 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe03800bc20 con 0x7fe0480acc00 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.782+0000 7fe044ff9640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe03406b4d0 0x7fe03406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.782+0000 7fe044ff9640 1 -- 172.21.15.26:0/4230058018 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe03808a8b0 con 0x7fe0480acc00 2024-03-06T07:32:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.783+0000 7fe0477fe640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe03406b4d0 0x7fe03406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.783+0000 7fe0477fe640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe03406b4d0 0x7fe03406d990 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe048150f80 tx=0x7fe03c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:32:59.787+0000 7fe044ff9640 1 -- 172.21.15.26:0/4230058018 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe038059530 con 0x7fe0480acc00 2024-03-06T07:33:00.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.007+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0480700a0 con 0x7fe0480acc00 2024-03-06T07:33:00.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.007+0000 7fe044ff9640 1 -- 172.21.15.26:0/4230058018 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fe038058e30 con 0x7fe0480acc00 2024-03-06T07:33:00.009 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:00.010 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:33:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.009+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe03406b4d0 msgr2=0x7fe03406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe03406b4d0 0x7fe03406d990 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe048150f80 tx=0x7fe03c009040 comp rx=0 tx=0).stop 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 msgr2=0x7fe048145ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe048145ea0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fe03800d800 tx=0x7fe03800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 shutdown_connections 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe03406b4d0 0x7fe03406d990 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048156ea0 0x7fe0480c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0481506b0 0x7fe0481512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 --2- 172.21.15.26:0/4230058018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0480acc00 0x7fe048145ea0 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 >> 172.21.15.26:0/4230058018 conn(0x7fe04806b1a0 msgr2=0x7fe0480702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 shutdown_connections 2024-03-06T07:33:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.010+0000 7fe04e513640 1 -- 172.21.15.26:0/4230058018 wait complete. 2024-03-06T07:33:00.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:00.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.125+0000 7f13086b1640 1 Processor -- start 2024-03-06T07:33:00.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.125+0000 7f13086b1640 1 -- start start 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f13086b1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1300159270 0x7f1300155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f13086b1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13000c73d0 con 0x7f130014f360 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f13086b1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13000be2e0 con 0x7f13000c88c0 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f13086b1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1300159700 con 0x7f1300159270 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f1306c4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1300159270 0x7f1300155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f130644d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f130014f360 0x7f13000c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57844/0 (socket says 172.21.15.26:57844) 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f130644d640 1 -- 172.21.15.26:0/3357050765 learned_addr learned my addr 172.21.15.26:0/3357050765 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f1306c4e640 1 -- 172.21.15.26:0/3357050765 >> v1:172.21.15.117:6789/0 conn(0x7f13000c88c0 legacy=0x7f1300158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f1306c4e640 1 -- 172.21.15.26:0/3357050765 >> v1:172.21.15.26:6789/0 conn(0x7f130014f360 legacy=0x7f13000c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f1306c4e640 1 -- 172.21.15.26:0/3357050765 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13000c7670 con 0x7f1300159270 2024-03-06T07:33:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.127+0000 7f130544b640 1 -- 172.21.15.26:0/3357050765 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223139267 0 0) 0x7f13000c73d0 con 0x7f130014f360 2024-03-06T07:33:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.128+0000 7f1306c4e640 1 --2- 172.21.15.26:0/3357050765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1300159270 0x7f1300155c10 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009d90 tx=0x7f12f802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8350e7b05ff037 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.128+0000 7f130544b640 1 -- 172.21.15.26:0/3357050765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8039070 con 0x7f1300159270 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.128+0000 7f130544b640 1 -- 172.21.15.26:0/3357050765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f802cbd0 con 0x7f1300159270 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.128+0000 7f130544b640 1 -- 172.21.15.26:0/3357050765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8034440 con 0x7f1300159270 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.128+0000 7f13086b1640 1 -- 172.21.15.26:0/3357050765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1300159270 msgr2=0x7f1300155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.128+0000 7f13086b1640 1 --2- 172.21.15.26:0/3357050765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1300159270 0x7f1300155c10 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009d90 tx=0x7f12f802be20 comp rx=0 tx=0).stop 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.129+0000 7f13086b1640 1 -- 172.21.15.26:0/3357050765 shutdown_connections 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.129+0000 7f13086b1640 1 --2- 172.21.15.26:0/3357050765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1300159270 0x7f1300155c10 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.129+0000 7f13086b1640 1 -- 172.21.15.26:0/3357050765 >> 172.21.15.26:0/3357050765 conn(0x7f13000ac570 msgr2=0x7f13000ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.129+0000 7f13086b1640 1 -- 172.21.15.26:0/3357050765 shutdown_connections 2024-03-06T07:33:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.129+0000 7f13086b1640 1 -- 172.21.15.26:0/3357050765 wait complete. 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 Processor -- start 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 -- start start 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13000c88c0 0x7f130014c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130014f360 0x7f130014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1300159270 0x7f1300157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000c7c30 con 0x7f130014f360 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000c5a50 con 0x7f13000c88c0 2024-03-06T07:33:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f13086b1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13000c5bd0 con 0x7f1300159270 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f1305c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130014f360 0x7f130014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f1305c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130014f360 0x7f130014b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60650/0 (socket says 172.21.15.26:60650) 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.130+0000 7f1305c4c640 1 -- 172.21.15.26:0/1864155817 learned_addr learned my addr 172.21.15.26:0/1864155817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f130644d640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13000c88c0 0x7f130014c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f1306c4e640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1300159270 0x7f1300157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f1305c4c640 1 -- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1300159270 msgr2=0x7f1300157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f1305c4c640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1300159270 0x7f1300157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f1305c4c640 1 -- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13000c88c0 msgr2=0x7f130014c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f1305c4c640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13000c88c0 0x7f130014c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f1305c4c640 1 -- 172.21.15.26:0/1864155817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13000b06c0 con 0x7f130014f360 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f1305c4c640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130014f360 0x7f130014b790 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c400 tx=0x7f12f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f12f6ffd640 1 -- 172.21.15.26:0/1864155817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0011790 con 0x7f130014f360 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f12f6ffd640 1 -- 172.21.15.26:0/1864155817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0017070 con 0x7f130014f360 2024-03-06T07:33:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f12f6ffd640 1 -- 172.21.15.26:0/1864155817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0011bb0 con 0x7f130014f360 2024-03-06T07:33:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13000add30 con 0x7f130014f360 2024-03-06T07:33:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.131+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13001eb8e0 con 0x7f130014f360 2024-03-06T07:33:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.132+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13000694c0 con 0x7f130014f360 2024-03-06T07:33:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.133+0000 7f12f6ffd640 1 -- 172.21.15.26:0/1864155817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12f001c460 con 0x7f130014f360 2024-03-06T07:33:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.133+0000 7f12f6ffd640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12e406b4d0 0x7f12e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.133+0000 7f12f6ffd640 1 -- 172.21.15.26:0/1864155817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f12f008a3e0 con 0x7f130014f360 2024-03-06T07:33:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.133+0000 7f130644d640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12e406b4d0 0x7f12e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.134+0000 7f130644d640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12e406b4d0 0x7f12e406d990 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f12fc000f60 tx=0x7f12fc007b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.141+0000 7f12f6ffd640 1 -- 172.21.15.26:0/1864155817 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f13000694c0 con 0x7f130014f360 2024-03-06T07:33:00.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.322+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13000c8d50 con 0x7f130014f360 2024-03-06T07:33:00.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.323+0000 7f12f6ffd640 1 -- 172.21.15.26:0/1864155817 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+17364 (secure 0 0 0) 0x7f13000add30 con 0x7f130014f360 2024-03-06T07:33:00.325 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:00.326 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:30:29.675481+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.632872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.411822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.f","mappings":[{"from":0,"to":2}]},{"pgid":"34.1a","mappings":[{"from":0,"to":2}]},{"pgid":"34.21","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.117:6834/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.117:6837/3331811951":"2024-03-07T07:29:54.670724+0000","172.21.15.117:6835/3331811951":"2024-03-07T07:29:54.670724+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12e406b4d0 msgr2=0x7f12e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12e406b4d0 0x7f12e406d990 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f12fc000f60 tx=0x7f12fc007b70 comp rx=0 tx=0).stop 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130014f360 msgr2=0x7f130014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130014f360 0x7f130014b790 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c400 tx=0x7f12f000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 shutdown_connections 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f12e406b4d0 0x7f12e406d990 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1300159270 0x7f1300157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130014f360 0x7f130014b790 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 --2- 172.21.15.26:0/1864155817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13000c88c0 0x7f130014c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 >> 172.21.15.26:0/1864155817 conn(0x7f13000ac570 msgr2=0x7f13000cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 shutdown_connections 2024-03-06T07:33:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.326+0000 7f13086b1640 1 -- 172.21.15.26:0/1864155817 wait complete. 2024-03-06T07:33:00.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:55.073878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:29:12.632872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:29:13.411822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '211', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:33:00.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:33:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.442+0000 7feb7eb8a640 1 Processor -- start 2024-03-06T07:33:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.442+0000 7feb7eb8a640 1 -- start start 2024-03-06T07:33:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.443+0000 7feb7eb8a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb7806ca70 0x7feb7814e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.443+0000 7feb7eb8a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb780b0450 con 0x7feb780b0070 2024-03-06T07:33:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.443+0000 7feb7eb8a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb781560e0 con 0x7feb78155c30 2024-03-06T07:33:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.443+0000 7feb7eb8a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7806c020 con 0x7feb7806ca70 2024-03-06T07:33:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.443+0000 7feb7d127640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb7806ca70 0x7feb7814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.443+0000 7feb7c926640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb780b0070 0x7feb780c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57854/0 (socket says 172.21.15.26:57854) 2024-03-06T07:33:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.443+0000 7feb7c926640 1 -- 172.21.15.26:0/4199794315 learned_addr learned my addr 172.21.15.26:0/4199794315 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.444+0000 7feb7d127640 1 -- 172.21.15.26:0/4199794315 >> v1:172.21.15.117:6789/0 conn(0x7feb78155c30 legacy=0x7feb780c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.444+0000 7feb7d127640 1 -- 172.21.15.26:0/4199794315 >> v1:172.21.15.26:6789/0 conn(0x7feb780b0070 legacy=0x7feb780c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.444+0000 7feb6f7fe640 1 -- 172.21.15.26:0/4199794315 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219313443 0 0) 0x7feb74001490 con 0x7feb780b0070 2024-03-06T07:33:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.444+0000 7feb7d127640 1 -- 172.21.15.26:0/4199794315 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb780c77f0 con 0x7feb7806ca70 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb7d127640 1 --2- 172.21.15.26:0/4199794315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb7806ca70 0x7feb7814e800 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7feb700097a0 tx=0x7feb7002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfba680f0f491e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb6f7fe640 1 -- 172.21.15.26:0/4199794315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700315a0 con 0x7feb7806ca70 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb6f7fe640 1 -- 172.21.15.26:0/4199794315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb70037070 con 0x7feb7806ca70 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb6f7fe640 1 -- 172.21.15.26:0/4199794315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70031960 con 0x7feb7806ca70 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb7eb8a640 1 -- 172.21.15.26:0/4199794315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb7806ca70 msgr2=0x7feb7814e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/4199794315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb7806ca70 0x7feb7814e800 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7feb700097a0 tx=0x7feb7002b840 comp rx=0 tx=0).stop 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb7eb8a640 1 -- 172.21.15.26:0/4199794315 shutdown_connections 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/4199794315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb7806ca70 0x7feb7814e800 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.445+0000 7feb7eb8a640 1 -- 172.21.15.26:0/4199794315 >> 172.21.15.26:0/4199794315 conn(0x7feb780ac590 msgr2=0x7feb780ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.446+0000 7feb7eb8a640 1 -- 172.21.15.26:0/4199794315 shutdown_connections 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.446+0000 7feb7eb8a640 1 -- 172.21.15.26:0/4199794315 wait complete. 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.447+0000 7feb7eb8a640 1 Processor -- start 2024-03-06T07:33:00.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.447+0000 7feb7eb8a640 1 -- start start 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.448+0000 7feb7eb8a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb7806ca70 0x7feb78156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.448+0000 7feb7eb8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb780b0070 0x7feb78156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.448+0000 7feb7eb8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb78155c30 0x7feb78152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7eb8a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7806c550 con 0x7feb780b0070 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7eb8a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7806c270 con 0x7feb78155c30 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7eb8a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7806da80 con 0x7feb7806ca70 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb780b0070 0x7feb78156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb780b0070 0x7feb78156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60672/0 (socket says 172.21.15.26:60672) 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 -- 172.21.15.26:0/1108150062 learned_addr learned my addr 172.21.15.26:0/1108150062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7c926640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb7806ca70 0x7feb78156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7d127640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb78155c30 0x7feb78152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 -- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb7806ca70 msgr2=0x7feb78156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb7806ca70 0x7feb78156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 -- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb78155c30 msgr2=0x7feb78152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb78155c30 0x7feb78152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 -- 172.21.15.26:0/1108150062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb781eb800 con 0x7feb780b0070 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7d127640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb78155c30 0x7feb78152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6ffff640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb780b0070 0x7feb78156b50 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c410 tx=0x7feb6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6d7fa640 1 -- 172.21.15.26:0/1108150062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb780b0070 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6d7fa640 1 -- 172.21.15.26:0/1108150062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb680035f0 con 0x7feb780b0070 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb780bdbd0 con 0x7feb780b0070 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb6d7fa640 1 -- 172.21.15.26:0/1108150062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68003a20 con 0x7feb780b0070 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.449+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb780c4cf0 con 0x7feb780b0070 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.450+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7804f330 con 0x7feb780b0070 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.451+0000 7feb6d7fa640 1 -- 172.21.15.26:0/1108150062 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb68003bc0 con 0x7feb780b0070 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.451+0000 7feb6d7fa640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6406b4d0 0x7feb6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.452+0000 7feb6d7fa640 1 -- 172.21.15.26:0/1108150062 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7feb68089790 con 0x7feb780b0070 2024-03-06T07:33:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.452+0000 7feb7c926640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6406b4d0 0x7feb6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.452+0000 7feb7c926640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6406b4d0 0x7feb6406d990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7feb740012e0 tx=0x7feb74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.456+0000 7feb6d7fa640 1 -- 172.21.15.26:0/1108150062 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb68058390 con 0x7feb780b0070 2024-03-06T07:33:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.637+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb7806d860 con 0x7feb780b0070 2024-03-06T07:33:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.637+0000 7feb6d7fa640 1 -- 172.21.15.26:0/1108150062 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v215) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb68057c90 con 0x7feb780b0070 2024-03-06T07:33:00.639 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-03-06T07:33:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.639+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6406b4d0 msgr2=0x7feb6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.639+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6406b4d0 0x7feb6406d990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7feb740012e0 tx=0x7feb74009040 comp rx=0 tx=0).stop 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.639+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb780b0070 msgr2=0x7feb78156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.639+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb780b0070 0x7feb78156b50 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c410 tx=0x7feb6800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 shutdown_connections 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7feb6406b4d0 0x7feb6406d990 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb78155c30 0x7feb78152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb780b0070 0x7feb78156b50 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 --2- 172.21.15.26:0/1108150062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb7806ca70 0x7feb78156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 >> 172.21.15.26:0/1108150062 conn(0x7feb780ac590 msgr2=0x7feb780aa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 shutdown_connections 2024-03-06T07:33:00.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.640+0000 7feb7eb8a640 1 -- 172.21.15.26:0/1108150062 wait complete. 2024-03-06T07:33:00.655 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:33:00.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.759+0000 7f9fa07b6640 1 Processor -- start 2024-03-06T07:33:00.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.759+0000 7f9fa07b6640 1 -- start start 2024-03-06T07:33:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9fa07b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9fa07b6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9f9806db30 0x7f9f980c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9fa07b6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f98156060 con 0x7f9f9806d750 2024-03-06T07:33:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9fa07b6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f98155b30 con 0x7f9f9806db30 2024-03-06T07:33:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9fa07b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980a8430 con 0x7f9f980c3a70 2024-03-06T07:33:00.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9f9ed53640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9f9806db30 0x7f9f980c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9f9dd51640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9f9e552640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9f9806d750 0x7f9f98072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:53838/0 (socket says 172.21.15.26:53838) 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.760+0000 7f9f9e552640 1 -- 172.21.15.26:0/3183835742 learned_addr learned my addr 172.21.15.26:0/3183835742 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.761+0000 7f9f9dd51640 1 -- 172.21.15.26:0/3183835742 >> v2:172.21.15.117:3300/0 conn(0x7f9f9806db30 msgr2=0x7f9f980c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.761+0000 7f9f9dd51640 1 --2- 172.21.15.26:0/3183835742 >> v2:172.21.15.117:3300/0 conn(0x7f9f9806db30 0x7f9f980c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.761+0000 7f9f9dd51640 1 -- 172.21.15.26:0/3183835742 >> v1:172.21.15.117:6789/0 conn(0x7f9f9806d750 legacy=0x7f9f98072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.761+0000 7f9f9dd51640 1 -- 172.21.15.26:0/3183835742 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98155d80 con 0x7f9f980c3a70 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.761+0000 7f9f9d550640 1 -- 172.21.15.26:0/3183835742 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978853097 0 0) 0x7f9f98156060 con 0x7f9f9806d750 2024-03-06T07:33:00.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.761+0000 7f9f9ed53640 1 --2- 172.21.15.26:0/3183835742 >> v2:172.21.15.117:3300/0 conn(0x7f9f9806db30 0x7f9f980c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.761+0000 7f9f9dd51640 1 --2- 172.21.15.26:0/3183835742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980732a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f880097a0 tx=0x7f9f8802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7980e3a816bc6d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9f9d550640 1 -- 172.21.15.26:0/3183835742 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88038070 con 0x7f9f980c3a70 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9f9d550640 1 -- 172.21.15.26:0/3183835742 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f88031700 con 0x7f9f980c3a70 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9f9d550640 1 -- 172.21.15.26:0/3183835742 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88031ad0 con 0x7f9f980c3a70 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 -- 172.21.15.26:0/3183835742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 msgr2=0x7f9f980732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/3183835742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980732a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f880097a0 tx=0x7f9f8802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 -- 172.21.15.26:0/3183835742 shutdown_connections 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/3183835742 >> v2:172.21.15.117:3300/0 conn(0x7f9f9806db30 0x7f9f980c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/3183835742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980732a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 -- 172.21.15.26:0/3183835742 >> 172.21.15.26:0/3183835742 conn(0x7f9f980c5560 msgr2=0x7f9f980c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 -- 172.21.15.26:0/3183835742 shutdown_connections 2024-03-06T07:33:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.762+0000 7f9fa07b6640 1 -- 172.21.15.26:0/3183835742 wait complete. 2024-03-06T07:33:00.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.763+0000 7f9fa07b6640 1 Processor -- start 2024-03-06T07:33:00.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.763+0000 7f9fa07b6640 1 -- start start 2024-03-06T07:33:00.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9fa07b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f9806d750 0x7f9f9814a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9fa07b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9806db30 0x7f9f9806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9fa07b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9fa07b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980a7e50 con 0x7f9f9806d750 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9fa07b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980a7cd0 con 0x7f9f9806db30 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9fa07b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980a7fd0 con 0x7f9f980c3a70 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f9806d750 0x7f9f9814a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f9806d750 0x7f9f9814a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60680/0 (socket says 172.21.15.26:60680) 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 -- 172.21.15.26:0/822822797 learned_addr learned my addr 172.21.15.26:0/822822797 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9dd51640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9806db30 0x7f9f9806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9ed53640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 -- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 msgr2=0x7f9f980ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 -- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9806db30 msgr2=0x7f9f9806fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9806db30 0x7f9f9806fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.764+0000 7f9f9e552640 1 -- 172.21.15.26:0/822822797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9814b3c0 con 0x7f9f9806d750 2024-03-06T07:33:00.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.765+0000 7f9f9e552640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f9806d750 0x7f9f9814a880 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c430 tx=0x7f9f9400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.765+0000 7f9f8effd640 1 -- 172.21.15.26:0/822822797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f940074e0 con 0x7f9f9806d750 2024-03-06T07:33:00.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.765+0000 7f9f8effd640 1 -- 172.21.15.26:0/822822797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f940047e0 con 0x7f9f9806d750 2024-03-06T07:33:00.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.765+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f98154d60 con 0x7f9f9806d750 2024-03-06T07:33:00.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.765+0000 7f9f8effd640 1 -- 172.21.15.26:0/822822797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94004bd0 con 0x7f9f9806d750 2024-03-06T07:33:00.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.765+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9814df80 con 0x7f9f9806d750 2024-03-06T07:33:00.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.766+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9804f330 con 0x7f9f9806d750 2024-03-06T07:33:00.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.767+0000 7f9f8effd640 1 -- 172.21.15.26:0/822822797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f94011070 con 0x7f9f9806d750 2024-03-06T07:33:00.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.767+0000 7f9f8effd640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.767+0000 7f9f8effd640 1 -- 172.21.15.26:0/822822797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f9f94089ef0 con 0x7f9f9806d750 2024-03-06T07:33:00.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.768+0000 7f9f9dd51640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.768+0000 7f9f9dd51640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f9f88035000 tx=0x7f9f88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.772+0000 7f9f8effd640 1 -- 172.21.15.26:0/822822797 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9f94058b70 con 0x7f9f9806d750 2024-03-06T07:33:00.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.952+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f9814b590 con 0x7f9f9806d750 2024-03-06T07:33:00.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.952+0000 7f9f8effd640 1 -- 172.21.15.26:0/822822797 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v215) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f94058470 con 0x7f9f9806d750 2024-03-06T07:33:00.954 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-03-06T07:33:00.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9f7c06b4d0 msgr2=0x7f9f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f9f88035000 tx=0x7f9f88008030 comp rx=0 tx=0).stop 2024-03-06T07:33:00.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f9806d750 msgr2=0x7f9f9814a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f9806d750 0x7f9f9814a880 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c430 tx=0x7f9f9400c830 comp rx=0 tx=0).stop 2024-03-06T07:33:00.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 shutdown_connections 2024-03-06T07:33:00.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980c3a70 0x7f9f980ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9806db30 0x7f9f9806fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 --2- 172.21.15.26:0/822822797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f9806d750 0x7f9f9814a880 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 >> 172.21.15.26:0/822822797 conn(0x7f9f980c5560 msgr2=0x7f9f980c74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 shutdown_connections 2024-03-06T07:33:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:00.954+0000 7f9fa07b6640 1 -- 172.21.15.26:0/822822797 wait complete. 2024-03-06T07:33:00.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:33:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.069+0000 7f239e21c640 1 Processor -- start 2024-03-06T07:33:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.069+0000 7f239e21c640 1 -- start start 2024-03-06T07:33:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.070+0000 7f239e21c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23980a8290 0x7f23981539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.070+0000 7f239e21c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c69a0 0x7f2398072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.070+0000 7f239e21c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980c62a0 con 0x7f23980c4740 2024-03-06T07:33:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.070+0000 7f239e21c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2398153f00 con 0x7f23980a8290 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.070+0000 7f239e21c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980c4c00 con 0x7f23980c69a0 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.070+0000 7f2397fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c69a0 0x7f2398072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.070+0000 7f23977fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23980a8290 0x7f23981539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f2397fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c69a0 0x7f2398072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:51648/0 (socket says 172.21.15.26:51648) 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f2397fff640 1 -- 172.21.15.26:0/2162215960 learned_addr learned my addr 172.21.15.26:0/2162215960 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f23977fe640 1 -- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c69a0 msgr2=0x7f2398072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f23977fe640 1 --2- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c69a0 0x7f2398072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f23977fe640 1 -- 172.21.15.26:0/2162215960 >> v1:172.21.15.117:6789/0 conn(0x7f23980c4740 legacy=0x7f23980701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f23967fc640 1 -- 172.21.15.26:0/2162215960 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919873482 0 0) 0x7f23980c62a0 con 0x7f23980c4740 2024-03-06T07:33:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f23977fe640 1 -- 172.21.15.26:0/2162215960 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980c4ee0 con 0x7f23980a8290 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.071+0000 7f2397fff640 1 --2- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c69a0 0x7f2398072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f23977fe640 1 --2- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980a8290 0x7f23981539c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f23880097a0 tx=0x7f238802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e3b1fc6be80fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f23967fc640 1 -- 172.21.15.26:0/2162215960 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388038070 con 0x7f23980a8290 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f23967fc640 1 -- 172.21.15.26:0/2162215960 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388031700 con 0x7f23980a8290 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f23967fc640 1 -- 172.21.15.26:0/2162215960 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388031ad0 con 0x7f23980a8290 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 -- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980a8290 msgr2=0x7f23981539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 --2- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980a8290 0x7f23981539c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f23880097a0 tx=0x7f238802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 -- 172.21.15.26:0/2162215960 shutdown_connections 2024-03-06T07:33:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 --2- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c69a0 0x7f2398072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 --2- 172.21.15.26:0/2162215960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980a8290 0x7f23981539c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 -- 172.21.15.26:0/2162215960 >> 172.21.15.26:0/2162215960 conn(0x7f23980c5560 msgr2=0x7f23980c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 -- 172.21.15.26:0/2162215960 shutdown_connections 2024-03-06T07:33:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.072+0000 7f239e21c640 1 -- 172.21.15.26:0/2162215960 wait complete. 2024-03-06T07:33:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.074+0000 7f239e21c640 1 Processor -- start 2024-03-06T07:33:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.074+0000 7f239e21c640 1 -- start start 2024-03-06T07:33:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f239e21c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980a8290 0x7f239814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f239e21c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980c4740 0x7f239814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f239e21c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23980c69a0 0x7f23981565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f239e21c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980c6e30 con 0x7f23980c69a0 2024-03-06T07:33:01.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f239e21c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239806fc80 con 0x7f23980c4740 2024-03-06T07:33:01.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f239e21c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2398072bc0 con 0x7f23980a8290 2024-03-06T07:33:01.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f2397fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23980c69a0 0x7f23981565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f2397fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23980c69a0 0x7f23981565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60694/0 (socket says 172.21.15.26:60694) 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f2397fff640 1 -- 172.21.15.26:0/1812963923 learned_addr learned my addr 172.21.15.26:0/1812963923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f23977fe640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980a8290 0x7f239814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.075+0000 7f2396ffd640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980c4740 0x7f239814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f2397fff640 1 -- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980a8290 msgr2=0x7f239814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f2397fff640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980a8290 0x7f239814d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f2397fff640 1 -- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980c4740 msgr2=0x7f239814d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f2397fff640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980c4740 0x7f239814d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f2397fff640 1 -- 172.21.15.26:0/1812963923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980ac450 con 0x7f23980c69a0 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f2397fff640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23980c69a0 0x7f23981565d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f2390009e70 tx=0x7f239000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f238ffff640 1 -- 172.21.15.26:0/1812963923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000cbe0 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f238ffff640 1 -- 172.21.15.26:0/1812963923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000cd80 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23980ac730 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f238ffff640 1 -- 172.21.15.26:0/1812963923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390003bb0 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.076+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23981dd190 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.077+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239804f330 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.078+0000 7f238ffff640 1 -- 172.21.15.26:0/1812963923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f239001b030 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.078+0000 7f238ffff640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f237806b4d0 0x7f237806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.078+0000 7f238ffff640 1 -- 172.21.15.26:0/1812963923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f239008b450 con 0x7f23980c69a0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.078+0000 7f23977fe640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f237806b4d0 0x7f237806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.079+0000 7f23977fe640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f237806b4d0 0x7f237806d990 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2388035000 tx=0x7f2388008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.082+0000 7f238ffff640 1 -- 172.21.15.26:0/1812963923 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2390022020 con 0x7f23980c69a0 2024-03-06T07:33:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.262+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23980c8480 con 0x7f23980c69a0 2024-03-06T07:33:01.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.262+0000 7f238ffff640 1 -- 172.21.15.26:0/1812963923 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v215) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2390010040 con 0x7f23980c69a0 2024-03-06T07:33:01.264 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-03-06T07:33:01.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f237806b4d0 msgr2=0x7f237806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f237806b4d0 0x7f237806d990 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2388035000 tx=0x7f2388008030 comp rx=0 tx=0).stop 2024-03-06T07:33:01.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23980c69a0 msgr2=0x7f23981565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23980c69a0 0x7f23981565d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f2390009e70 tx=0x7f239000c490 comp rx=0 tx=0).stop 2024-03-06T07:33:01.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 shutdown_connections 2024-03-06T07:33:01.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f237806b4d0 0x7f237806d990 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23980c69a0 0x7f23981565d0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23980c4740 0x7f239814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 --2- 172.21.15.26:0/1812963923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980a8290 0x7f239814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 >> 172.21.15.26:0/1812963923 conn(0x7f23980c5560 msgr2=0x7f23980c3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 shutdown_connections 2024-03-06T07:33:01.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.264+0000 7f239e21c640 1 -- 172.21.15.26:0/1812963923 wait complete. 2024-03-06T07:33:01.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.378+0000 7fd42678a640 1 Processor -- start 2024-03-06T07:33:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.378+0000 7fd42678a640 1 -- start start 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd42678a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd42014f360 0x7fd420155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd42678a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd420158b60 0x7fd420158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd42678a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200c73d0 con 0x7fd4200c88c0 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd42678a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4200c8dd0 con 0x7fd420158b60 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd42678a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4200be2e0 con 0x7fd42014f360 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd424d27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd420158b60 0x7fd420158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd424d27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd420158b60 0x7fd420158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60696/0 (socket says 172.21.15.26:60696) 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd424d27640 1 -- 172.21.15.26:0/2597426862 learned_addr learned my addr 172.21.15.26:0/2597426862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd41f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd4200c88c0 0x7fd4200c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57856/0 (socket says 172.21.15.26:57856) 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd41ffff640 1 --2- 172.21.15.26:0/2597426862 >> v2:172.21.15.117:3300/0 conn(0x7fd42014f360 0x7fd420155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd424d27640 1 -- 172.21.15.26:0/2597426862 >> v2:172.21.15.117:3300/0 conn(0x7fd42014f360 msgr2=0x7fd420155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd424d27640 1 --2- 172.21.15.26:0/2597426862 >> v2:172.21.15.117:3300/0 conn(0x7fd42014f360 0x7fd420155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd424d27640 1 -- 172.21.15.26:0/2597426862 >> v1:172.21.15.26:6789/0 conn(0x7fd4200c88c0 legacy=0x7fd4200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd424d27640 1 -- 172.21.15.26:0/2597426862 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200c7670 con 0x7fd420158b60 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd41effd640 1 -- 172.21.15.26:0/2597426862 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990702814 0 0) 0x7fd4200c73d0 con 0x7fd4200c88c0 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.379+0000 7fd41ffff640 1 --2- 172.21.15.26:0/2597426862 >> v2:172.21.15.117:3300/0 conn(0x7fd42014f360 0x7fd420155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.380+0000 7fd424d27640 1 --2- 172.21.15.26:0/2597426862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd420158b60 0x7fd420158ff0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fd42008c840 tx=0x7fd41802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324bace3435ae6c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.380+0000 7fd41effd640 1 -- 172.21.15.26:0/2597426862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41802ca80 con 0x7fd420158b60 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.380+0000 7fd41effd640 1 -- 172.21.15.26:0/2597426862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41802cc20 con 0x7fd420158b60 2024-03-06T07:33:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.380+0000 7fd41effd640 1 -- 172.21.15.26:0/2597426862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4180343f0 con 0x7fd420158b60 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 -- 172.21.15.26:0/2597426862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd420158b60 msgr2=0x7fd420158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 --2- 172.21.15.26:0/2597426862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd420158b60 0x7fd420158ff0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fd42008c840 tx=0x7fd41802bee0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 -- 172.21.15.26:0/2597426862 shutdown_connections 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 --2- 172.21.15.26:0/2597426862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd420158b60 0x7fd420158ff0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 --2- 172.21.15.26:0/2597426862 >> v2:172.21.15.117:3300/0 conn(0x7fd42014f360 0x7fd420155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 -- 172.21.15.26:0/2597426862 >> 172.21.15.26:0/2597426862 conn(0x7fd4200ac570 msgr2=0x7fd4200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 -- 172.21.15.26:0/2597426862 shutdown_connections 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.381+0000 7fd42678a640 1 -- 172.21.15.26:0/2597426862 wait complete. 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.382+0000 7fd42678a640 1 Processor -- start 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.382+0000 7fd42678a640 1 -- start start 2024-03-06T07:33:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd42678a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4200c88c0 0x7fd42014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd42678a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42014f360 0x7fd4200bf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd42678a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420158b60 0x7fd4201eda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd42678a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200c5a50 con 0x7fd4200c88c0 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd42678a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200c7c30 con 0x7fd420158b60 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd42678a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200c5bd0 con 0x7fd42014f360 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4200c88c0 0x7fd42014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4200c88c0 0x7fd42014c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60700/0 (socket says 172.21.15.26:60700) 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 -- 172.21.15.26:0/3592156342 learned_addr learned my addr 172.21.15.26:0/3592156342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd424d27640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420158b60 0x7fd4201eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 -- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42014f360 msgr2=0x7fd4200bf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42014f360 0x7fd4200bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 -- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420158b60 msgr2=0x7fd4201eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420158b60 0x7fd4201eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 -- 172.21.15.26:0/3592156342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420155ed0 con 0x7fd4200c88c0 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd424d27640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420158b60 0x7fd4201eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41ffff640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4200c88c0 0x7fd42014c080 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c7f0 tx=0x7fd41000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.383+0000 7fd41cff9640 1 -- 172.21.15.26:0/3592156342 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410015070 con 0x7fd4200c88c0 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.384+0000 7fd41cff9640 1 -- 172.21.15.26:0/3592156342 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410007820 con 0x7fd4200c88c0 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.384+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42014c5c0 con 0x7fd4200c88c0 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.384+0000 7fd41cff9640 1 -- 172.21.15.26:0/3592156342 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410007bd0 con 0x7fd4200c88c0 2024-03-06T07:33:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.384+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4200c0a30 con 0x7fd4200c88c0 2024-03-06T07:33:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.385+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42004f330 con 0x7fd4200c88c0 2024-03-06T07:33:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.385+0000 7fd41cff9640 1 -- 172.21.15.26:0/3592156342 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd410018070 con 0x7fd4200c88c0 2024-03-06T07:33:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.385+0000 7fd41cff9640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.386+0000 7fd41cff9640 1 -- 172.21.15.26:0/3592156342 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd41008a500 con 0x7fd4200c88c0 2024-03-06T07:33:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.386+0000 7fd41f7fe640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.386+0000 7fd41f7fe640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd40c06b4d0 0x7fd40c06d990 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd414007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.390+0000 7fd41cff9640 1 -- 172.21.15.26:0/3592156342 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd420155ed0 con 0x7fd4200c88c0 2024-03-06T07:33:01.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.608+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd42006cda0 con 0x7fd4200c88c0 2024-03-06T07:33:01.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.608+0000 7fd41cff9640 1 -- 172.21.15.26:0/3592156342 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fd410059180 con 0x7fd4200c88c0 2024-03-06T07:33:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:01.610 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:33:01.610 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd40c06b4d0 msgr2=0x7fd40c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd40c06b4d0 0x7fd40c06d990 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd414007960 comp rx=0 tx=0).stop 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4200c88c0 msgr2=0x7fd42014c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4200c88c0 0x7fd42014c080 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c7f0 tx=0x7fd41000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 shutdown_connections 2024-03-06T07:33:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420158b60 0x7fd4201eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42014f360 0x7fd4200bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 --2- 172.21.15.26:0/3592156342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4200c88c0 0x7fd42014c080 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 >> 172.21.15.26:0/3592156342 conn(0x7fd4200ac570 msgr2=0x7fd4200c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 shutdown_connections 2024-03-06T07:33:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.610+0000 7fd42678a640 1 -- 172.21.15.26:0/3592156342 wait complete. 2024-03-06T07:33:01.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.726+0000 7ff21feb5640 1 Processor -- start 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.726+0000 7ff21feb5640 1 -- start start 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21feb5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2180c49a0 0x7ff2180c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21feb5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff218072650 0x7ff218072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21feb5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180c2c40 con 0x7ff2180c2470 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21feb5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff218156060 con 0x7ff2180c49a0 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21feb5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180ab5f0 con 0x7ff218072650 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21dc51640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2180c49a0 0x7ff2180c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21d450640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff2180c2470 0x7ff2180c7ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57866/0 (socket says 172.21.15.26:57866) 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21d450640 1 -- 172.21.15.26:0/3422023488 learned_addr learned my addr 172.21.15.26:0/3422023488 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21e452640 1 --2- 172.21.15.26:0/3422023488 >> v2:172.21.15.117:3300/0 conn(0x7ff218072650 0x7ff218072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21dc51640 1 -- 172.21.15.26:0/3422023488 >> v2:172.21.15.117:3300/0 conn(0x7ff218072650 msgr2=0x7ff218072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21dc51640 1 --2- 172.21.15.26:0/3422023488 >> v2:172.21.15.117:3300/0 conn(0x7ff218072650 0x7ff218072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21dc51640 1 -- 172.21.15.26:0/3422023488 >> v1:172.21.15.26:6789/0 conn(0x7ff2180c2470 legacy=0x7ff2180c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21dc51640 1 -- 172.21.15.26:0/3422023488 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218155aa0 con 0x7ff2180c49a0 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21cc4f640 1 -- 172.21.15.26:0/3422023488 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831989944 0 0) 0x7ff2180c2c40 con 0x7ff2180c2470 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.727+0000 7ff21e452640 1 --2- 172.21.15.26:0/3422023488 >> v2:172.21.15.117:3300/0 conn(0x7ff218072650 0x7ff218072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.728+0000 7ff21dc51640 1 --2- 172.21.15.26:0/3422023488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180c7990 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7ff21806e2b0 tx=0x7ff21402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9fde7e1312852a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.728+0000 7ff21cc4f640 1 -- 172.21.15.26:0/3422023488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140315a0 con 0x7ff2180c49a0 2024-03-06T07:33:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.728+0000 7ff21cc4f640 1 -- 172.21.15.26:0/3422023488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff214037070 con 0x7ff2180c49a0 2024-03-06T07:33:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.728+0000 7ff21cc4f640 1 -- 172.21.15.26:0/3422023488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214031960 con 0x7ff2180c49a0 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 -- 172.21.15.26:0/3422023488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 msgr2=0x7ff2180c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 --2- 172.21.15.26:0/3422023488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180c7990 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7ff21806e2b0 tx=0x7ff21402b840 comp rx=0 tx=0).stop 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 -- 172.21.15.26:0/3422023488 shutdown_connections 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 --2- 172.21.15.26:0/3422023488 >> v2:172.21.15.117:3300/0 conn(0x7ff218072650 0x7ff218072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 --2- 172.21.15.26:0/3422023488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180c7990 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 -- 172.21.15.26:0/3422023488 >> 172.21.15.26:0/3422023488 conn(0x7ff21806b5f0 msgr2=0x7ff2180aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 -- 172.21.15.26:0/3422023488 shutdown_connections 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.729+0000 7ff21feb5640 1 -- 172.21.15.26:0/3422023488 wait complete. 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 Processor -- start 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 -- start start 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff218072650 0x7ff21814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2180c2470 0x7ff218158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c4d80 con 0x7ff2180c49a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c85e0 con 0x7ff2180c2470 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.730+0000 7ff21feb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff218072fc0 con 0x7ff218072650 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60726/0 (socket says 172.21.15.26:60726) 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 -- 172.21.15.26:0/2070706659 learned_addr learned my addr 172.21.15.26:0/2070706659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21d450640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2180c2470 0x7ff218158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21dc51640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff218072650 0x7ff21814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 -- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff218072650 msgr2=0x7ff21814ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff218072650 0x7ff21814ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 -- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2180c2470 msgr2=0x7ff218158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2180c2470 0x7ff218158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 -- 172.21.15.26:0/2070706659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180a9e10 con 0x7ff2180c49a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21e452640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180acab0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7ff21000cef0 tx=0x7ff21000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff20e7fc640 1 -- 172.21.15.26:0/2070706659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21001a070 con 0x7ff2180c49a0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff20e7fc640 1 -- 172.21.15.26:0/2070706659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000d860 con 0x7ff2180c49a0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2181eb920 con 0x7ff2180c49a0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff20e7fc640 1 -- 172.21.15.26:0/2070706659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000dc50 con 0x7ff2180c49a0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.731+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff218152dd0 con 0x7ff2180c49a0 2024-03-06T07:33:01.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.732+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2180aa310 con 0x7ff2180c49a0 2024-03-06T07:33:01.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.733+0000 7ff20e7fc640 1 -- 172.21.15.26:0/2070706659 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff21001d070 con 0x7ff2180c49a0 2024-03-06T07:33:01.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.733+0000 7ff20e7fc640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1fc06b4d0 0x7ff1fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.733+0000 7ff20e7fc640 1 -- 172.21.15.26:0/2070706659 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7ff21008ac90 con 0x7ff2180c49a0 2024-03-06T07:33:01.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.733+0000 7ff21dc51640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1fc06b4d0 0x7ff1fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.734+0000 7ff21dc51640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1fc06b4d0 0x7ff1fc06d990 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff214036000 tx=0x7ff214008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.742+0000 7ff20e7fc640 1 -- 172.21.15.26:0/2070706659 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2180aa310 con 0x7ff2180c49a0 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.923+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff21804f330 con 0x7ff2180c49a0 2024-03-06T07:33:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.925+0000 7ff20e7fc640 1 -- 172.21.15.26:0/2070706659 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+17364 (secure 0 0 0) 0x7ff2100599d0 con 0x7ff2180c49a0 2024-03-06T07:33:01.927 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:01.927 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:30:29.675481+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.632872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.411822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.f","mappings":[{"from":0,"to":2}]},{"pgid":"34.1a","mappings":[{"from":0,"to":2}]},{"pgid":"34.21","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.117:6834/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.117:6837/3331811951":"2024-03-07T07:29:54.670724+0000","172.21.15.117:6835/3331811951":"2024-03-07T07:29:54.670724+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1fc06b4d0 msgr2=0x7ff1fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1fc06b4d0 0x7ff1fc06d990 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff214036000 tx=0x7ff214008030 comp rx=0 tx=0).stop 2024-03-06T07:33:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 msgr2=0x7ff2180acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180acab0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7ff21000cef0 tx=0x7ff21000a710 comp rx=0 tx=0).stop 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 shutdown_connections 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff1fc06b4d0 0x7ff1fc06d990 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2180c49a0 0x7ff2180acab0 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2180c2470 0x7ff218158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 --2- 172.21.15.26:0/2070706659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff218072650 0x7ff21814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 >> 172.21.15.26:0/2070706659 conn(0x7ff21806b5f0 msgr2=0x7ff2180c5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 shutdown_connections 2024-03-06T07:33:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:01.927+0000 7ff21feb5640 1 -- 172.21.15.26:0/2070706659 wait complete. 2024-03-06T07:33:01.942 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:33:01.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.042+0000 7f8f11920640 1 Processor -- start 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.042+0000 7f8f11920640 1 -- start start 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f11920640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 0x7f8f0c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f11920640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f0c0c88c0 0x7f8f0c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f11920640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0c73d0 con 0x7f8f0c0c5880 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f11920640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c8dd0 con 0x7f8f0c0c88c0 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f11920640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0be2e0 con 0x7f8f0c14f360 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f0b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f0c0c5880 0x7f8f0c158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57868/0 (socket says 172.21.15.26:57868) 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/3615761082 learned_addr learned my addr 172.21.15.26:0/3615761082 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f0a7fc640 1 --2- 172.21.15.26:0/3615761082 >> v2:172.21.15.117:3300/0 conn(0x7f8f0c0c88c0 0x7f8f0c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f0affd640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 0x7f8f0c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492167093 0 0) 0x7f8f0c0c73d0 con 0x7f8f0c0c5880 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.043+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef4003520 con 0x7f8f0c0c5880 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996067100 0 0) 0x7f8ef4003520 con 0x7f8f0c0c5880 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 msgr2=0x7f8f0c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 0x7f8f0c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 >> v2:172.21.15.117:3300/0 conn(0x7f8f0c0c88c0 msgr2=0x7f8f0c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 --2- 172.21.15.26:0/3615761082 >> v2:172.21.15.117:3300/0 conn(0x7f8f0c0c88c0 0x7f8f0c0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0c7670 con 0x7f8f0c0c5880 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f8f040019f0 con 0x7f8f0c0c5880 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0a7fc640 1 --2- 172.21.15.26:0/3615761082 >> v2:172.21.15.117:3300/0 conn(0x7f8f0c0c88c0 0x7f8f0c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 >> v1:172.21.15.26:6789/0 conn(0x7f8f0c0c5880 legacy=0x7f8f0c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 0x7f8ef40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4003f30 0x7f8ef4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef4004d40 0x7f8ef40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0be2e0 con 0x7f8ef4004d40 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef400b9c0 con 0x7f8ef4003f30 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef400bb90 con 0x7f8f0c14f360 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0b7fe640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef4004d40 0x7f8ef40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0affd640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 0x7f8ef40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0a7fc640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4003f30 0x7f8ef4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 msgr2=0x7f8ef40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0b7fe640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f0c14f360 0x7f8ef40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4003f30 msgr2=0x7f8ef4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0b7fe640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4003f30 0x7f8ef4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.044+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/3615761082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4003520 con 0x7f8ef4004d40 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.045+0000 7f8f0b7fe640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef4004d40 0x7f8ef40051c0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400a890 tx=0x7f8ef4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d8a01f9691d165 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.045+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400a4e0 con 0x7f8ef4004d40 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.045+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f04034650 con 0x7f8ef4004d40 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.045+0000 7f8f09ffb640 1 -- 172.21.15.26:0/3615761082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04034a40 con 0x7f8ef4004d40 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.045+0000 7f8f11920640 1 -- 172.21.15.26:0/3615761082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef4004d40 msgr2=0x7f8ef40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.045+0000 7f8f11920640 1 --2- 172.21.15.26:0/3615761082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef4004d40 0x7f8ef40051c0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400a890 tx=0x7f8ef4005830 comp rx=0 tx=0).stop 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.045+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/3615761082 reap_dead start 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.046+0000 7f8f11920640 1 -- 172.21.15.26:0/3615761082 shutdown_connections 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.046+0000 7f8f11920640 1 -- 172.21.15.26:0/3615761082 >> 172.21.15.26:0/3615761082 conn(0x7f8f0c0ac570 msgr2=0x7f8f0c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.046+0000 7f8f11920640 1 -- 172.21.15.26:0/3615761082 shutdown_connections 2024-03-06T07:33:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.046+0000 7f8f11920640 1 -- 172.21.15.26:0/3615761082 wait complete. 2024-03-06T07:33:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.047+0000 7f8f11920640 1 Processor -- start 2024-03-06T07:33:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.047+0000 7f8f11920640 1 -- start start 2024-03-06T07:33:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef4004d40 0x7f8f0c151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c0c4080 0x7f8f0c0c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c1e7b40 0x7f8f0c1e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c159270 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c7c30 con 0x7f8f0c0c4080 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1593f0 con 0x7f8ef4004d40 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c1e7b40 0x7f8f0c1e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c1e7b40 0x7f8f0c1e7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60740/0 (socket says 172.21.15.26:60740) 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/2266958261 learned_addr learned my addr 172.21.15.26:0/2266958261 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef4004d40 msgr2=0x7f8f0c151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef4004d40 0x7f8f0c151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c0c4080 msgr2=0x7f8f0c0c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c0c4080 0x7f8f0c0c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 -- 172.21.15.26:0/2266958261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c1ee0e0 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f0b7fe640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c1e7b40 0x7f8f0c1e7fc0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400a890 tx=0x7f8f04001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8ef3fff640 1 -- 172.21.15.26:0/2266958261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04039040 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8ef3fff640 1 -- 172.21.15.26:0/2266958261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f04043070 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c1ee3c0 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8ef3fff640 1 -- 172.21.15.26:0/2266958261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04004b60 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.048+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c1ee980 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.049+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c04f330 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.050+0000 7f8ef3fff640 1 -- 172.21.15.26:0/2266958261 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f04004d00 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.050+0000 7f8ef3fff640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8ef806b4d0 0x7f8ef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.050+0000 7f8ef3fff640 1 -- 172.21.15.26:0/2266958261 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f8f040ac160 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.051+0000 7f8f0affd640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8ef806b4d0 0x7f8ef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.051+0000 7f8f0affd640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8ef806b4d0 0x7f8ef806d990 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009ab0 tx=0x7f8efc0095d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.055+0000 7f8ef3fff640 1 -- 172.21.15.26:0/2266958261 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f0400a580 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.272+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f0c06cda0 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.272+0000 7f8ef3fff640 1 -- 172.21.15.26:0/2266958261 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7f8f0407ade0 con 0x7f8f0c1e7b40 2024-03-06T07:33:02.274 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:02.274 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:33:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8ef806b4d0 msgr2=0x7f8ef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8ef806b4d0 0x7f8ef806d990 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009ab0 tx=0x7f8efc0095d0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c1e7b40 msgr2=0x7f8f0c1e7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c1e7b40 0x7f8f0c1e7fc0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400a890 tx=0x7f8f04001360 comp rx=0 tx=0).stop 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 shutdown_connections 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8ef806b4d0 0x7f8ef806d990 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c1e7b40 0x7f8f0c1e7fc0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c0c4080 0x7f8f0c0c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 --2- 172.21.15.26:0/2266958261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef4004d40 0x7f8f0c151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 >> 172.21.15.26:0/2266958261 conn(0x7f8f0c0ac570 msgr2=0x7f8f0c14e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 shutdown_connections 2024-03-06T07:33:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.274+0000 7f8f11920640 1 -- 172.21.15.26:0/2266958261 wait complete. 2024-03-06T07:33:02.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.387+0000 7fc31b5c7640 1 Processor -- start 2024-03-06T07:33:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.387+0000 7fc31b5c7640 1 -- start start 2024-03-06T07:33:02.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc31b5c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc31414f360 0x7fc314155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc31b5c7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc3140c88c0 0x7fc3140c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc31b5c7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3140c73d0 con 0x7fc3140c5880 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc31b5c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3140c8dd0 con 0x7fc31414f360 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc31b5c7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3140be2e0 con 0x7fc3140c88c0 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc319363640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc31414f360 0x7fc314155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc319b64640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc3140c5880 0x7fc314158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57880/0 (socket says 172.21.15.26:57880) 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc319b64640 1 -- 172.21.15.26:0/1620292264 learned_addr learned my addr 172.21.15.26:0/1620292264 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.388+0000 7fc318b62640 1 --2- 172.21.15.26:0/1620292264 >> v2:172.21.15.117:3300/0 conn(0x7fc3140c88c0 0x7fc3140c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.389+0000 7fc319363640 1 -- 172.21.15.26:0/1620292264 >> v2:172.21.15.117:3300/0 conn(0x7fc3140c88c0 msgr2=0x7fc3140c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.389+0000 7fc319363640 1 --2- 172.21.15.26:0/1620292264 >> v2:172.21.15.117:3300/0 conn(0x7fc3140c88c0 0x7fc3140c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.389+0000 7fc319363640 1 -- 172.21.15.26:0/1620292264 >> v1:172.21.15.26:6789/0 conn(0x7fc3140c5880 legacy=0x7fc314158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.389+0000 7fc30bfff640 1 -- 172.21.15.26:0/1620292264 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294024815 0 0) 0x7fc3140c73d0 con 0x7fc3140c5880 2024-03-06T07:33:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.389+0000 7fc319363640 1 -- 172.21.15.26:0/1620292264 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3140c7670 con 0x7fc31414f360 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.389+0000 7fc319363640 1 --2- 172.21.15.26:0/1620292264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314155c10 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc3140c84c0 tx=0x7fc31002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5160b2f4055148 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.389+0000 7fc30bfff640 1 -- 172.21.15.26:0/1620292264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100315a0 con 0x7fc31414f360 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc30bfff640 1 -- 172.21.15.26:0/1620292264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc310037070 con 0x7fc31414f360 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc30bfff640 1 -- 172.21.15.26:0/1620292264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310031960 con 0x7fc31414f360 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1620292264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 msgr2=0x7fc314155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1620292264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314155c10 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc3140c84c0 tx=0x7fc31002b840 comp rx=0 tx=0).stop 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1620292264 shutdown_connections 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1620292264 >> v2:172.21.15.117:3300/0 conn(0x7fc3140c88c0 0x7fc3140c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1620292264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314155c10 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1620292264 >> 172.21.15.26:0/1620292264 conn(0x7fc3140ac570 msgr2=0x7fc3140ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1620292264 shutdown_connections 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.390+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1620292264 wait complete. 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.391+0000 7fc31b5c7640 1 Processor -- start 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.391+0000 7fc31b5c7640 1 -- start start 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc31b5c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3140c5880 0x7fc3140c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc31b5c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3140c88c0 0x7fc3140c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc31b5c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc31b5c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc314159270 con 0x7fc31414f360 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc31b5c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140c7c30 con 0x7fc3140c88c0 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc31b5c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3141593f0 con 0x7fc3140c5880 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60748/0 (socket says 172.21.15.26:60748) 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 -- 172.21.15.26:0/1409195756 learned_addr learned my addr 172.21.15.26:0/1409195756 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc318b62640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3140c88c0 0x7fc3140c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319363640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3140c5880 0x7fc3140c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 -- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3140c5880 msgr2=0x7fc3140c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3140c5880 0x7fc3140c10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 -- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3140c88c0 msgr2=0x7fc3140c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3140c88c0 0x7fc3140c1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc319b64640 1 -- 172.21.15.26:0/1409195756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3141ea820 con 0x7fc31414f360 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.392+0000 7fc318b62640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3140c88c0 0x7fc3140c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.393+0000 7fc319b64640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314157bf0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c7a0 tx=0x7fc30c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.393+0000 7fc309ffb640 1 -- 172.21.15.26:0/1409195756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c018070 con 0x7fc31414f360 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.393+0000 7fc309ffb640 1 -- 172.21.15.26:0/1409195756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c00a640 con 0x7fc31414f360 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.393+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3141ede30 con 0x7fc31414f360 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.393+0000 7fc309ffb640 1 -- 172.21.15.26:0/1409195756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00aab0 con 0x7fc31414f360 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.393+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3141ee3f0 con 0x7fc31414f360 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.394+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31404f330 con 0x7fc31414f360 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.394+0000 7fc309ffb640 1 -- 172.21.15.26:0/1409195756 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc30c01b070 con 0x7fc31414f360 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.395+0000 7fc309ffb640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2f806b500 0x7fc2f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.395+0000 7fc309ffb640 1 -- 172.21.15.26:0/1409195756 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc30c089a00 con 0x7fc31414f360 2024-03-06T07:33:02.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.395+0000 7fc319363640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2f806b500 0x7fc2f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.396+0000 7fc319363640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2f806b500 0x7fc2f806d9c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc310036000 tx=0x7fc31003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.399+0000 7fc309ffb640 1 -- 172.21.15.26:0/1409195756 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc30c058680 con 0x7fc31414f360 2024-03-06T07:33:02.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.620+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc31406cda0 con 0x7fc31414f360 2024-03-06T07:33:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.620+0000 7fc309ffb640 1 -- 172.21.15.26:0/1409195756 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+3473 (secure 0 0 0) 0x7fc30c057f80 con 0x7fc31414f360 2024-03-06T07:33:02.622 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:02.622 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378208+0000","modified":"2024-03-06T07:31:04.679573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10737,"mds_1":8327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8327":{"gid":8327,"name":"a","rank":1,"incarnation":209,"state":"up:stopping","state_seq":24,"addr":"172.21.15.26:6834/741758816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":741758816},{"type":"v1","addr":"172.21.15.26:6834","nonce":741758816}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:04.679573+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10737":{"gid":10737,"name":"c","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/4219086401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4219086401},{"type":"v1","addr":"172.21.15.26:6835","nonce":4219086401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10737,"qdb_cluster":[10737]},"id":16}]} 2024-03-06T07:33:02.623 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 215 2024-03-06T07:33:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2f806b500 msgr2=0x7fc2f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2f806b500 0x7fc2f806d9c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc310036000 tx=0x7fc31003d040 comp rx=0 tx=0).stop 2024-03-06T07:33:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 msgr2=0x7fc314157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314157bf0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c7a0 tx=0x7fc30c00cc60 comp rx=0 tx=0).stop 2024-03-06T07:33:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 shutdown_connections 2024-03-06T07:33:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc2f806b500 0x7fc2f806d9c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc31414f360 0x7fc314157bf0 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3140c88c0 0x7fc3140c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 --2- 172.21.15.26:0/1409195756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3140c5880 0x7fc3140c10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 >> 172.21.15.26:0/1409195756 conn(0x7fc3140ac570 msgr2=0x7fc31406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 shutdown_connections 2024-03-06T07:33:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.623+0000 7fc31b5c7640 1 -- 172.21.15.26:0/1409195756 wait complete. 2024-03-06T07:33:02.636 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:02.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.737+0000 7f39c9fac640 1 Processor -- start 2024-03-06T07:33:02.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.737+0000 7f39c9fac640 1 -- start start 2024-03-06T07:33:02.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c9fac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39c41506b0 0x7f39c4153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c9fac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c40acc00 0x7f39c40c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c9fac640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c4070aa0 con 0x7f39c4156ea0 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c9fac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c4156450 con 0x7f39c41506b0 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c9fac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c4156a10 con 0x7f39c40acc00 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39c41506b0 0x7f39c4153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c3fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c40acc00 0x7f39c40c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c2ffd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f39c4156ea0 0x7f39c40702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:53850/0 (socket says 172.21.15.26:53850) 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.738+0000 7f39c2ffd640 1 -- 172.21.15.26:0/746557164 learned_addr learned my addr 172.21.15.26:0/746557164 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.739+0000 7f39c27fc640 1 -- 172.21.15.26:0/746557164 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991077517 0 0) 0x7f39c4070aa0 con 0x7f39c4156ea0 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.739+0000 7f39c27fc640 1 -- 172.21.15.26:0/746557164 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39ac003520 con 0x7f39c4156ea0 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.739+0000 7f39c37fe640 1 -- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c40acc00 msgr2=0x7f39c40c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.739+0000 7f39c37fe640 1 --2- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c40acc00 0x7f39c40c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.739+0000 7f39c37fe640 1 -- 172.21.15.26:0/746557164 >> v1:172.21.15.117:6789/0 conn(0x7f39c4156ea0 legacy=0x7f39c40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.739+0000 7f39c37fe640 1 -- 172.21.15.26:0/746557164 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4070d80 con 0x7f39c41506b0 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.739+0000 7f39c3fff640 1 --2- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c40acc00 0x7f39c40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c37fe640 1 --2- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c41506b0 0x7f39c4153990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009d90 tx=0x7f39b402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9fa93eb9c2eab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c27fc640 1 -- 172.21.15.26:0/746557164 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4039070 con 0x7f39c41506b0 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c27fc640 1 -- 172.21.15.26:0/746557164 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b402cc50 con 0x7f39c41506b0 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c27fc640 1 -- 172.21.15.26:0/746557164 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4034440 con 0x7f39c41506b0 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 -- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c41506b0 msgr2=0x7f39c4153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 --2- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c41506b0 0x7f39c4153990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009d90 tx=0x7f39b402bea0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 -- 172.21.15.26:0/746557164 shutdown_connections 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 --2- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c40acc00 0x7f39c40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 --2- 172.21.15.26:0/746557164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c41506b0 0x7f39c4153990 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 -- 172.21.15.26:0/746557164 >> 172.21.15.26:0/746557164 conn(0x7f39c406b1a0 msgr2=0x7f39c406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 -- 172.21.15.26:0/746557164 shutdown_connections 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.740+0000 7f39c9fac640 1 -- 172.21.15.26:0/746557164 wait complete. 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 Processor -- start 2024-03-06T07:33:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 -- start start 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c40acc00 0x7f39c41eb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39c41506b0 0x7f39c4149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c4156ea0 0x7f39c4146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40ad1d0 con 0x7f39c41506b0 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40ad050 con 0x7f39c40acc00 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.742+0000 7f39c9fac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40ad350 con 0x7f39c4156ea0 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39c41506b0 0x7f39c4149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39c41506b0 0x7f39c4149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60750/0 (socket says 172.21.15.26:60750) 2024-03-06T07:33:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 -- 172.21.15.26:0/1376069651 learned_addr learned my addr 172.21.15.26:0/1376069651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c3fff640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c4156ea0 0x7f39c4146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c37fe640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c40acc00 0x7f39c41eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 -- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c4156ea0 msgr2=0x7f39c4146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c4156ea0 0x7f39c4146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 -- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c40acc00 msgr2=0x7f39c41eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c40acc00 0x7f39c41eb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 -- 172.21.15.26:0/1376069651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c406e0b0 con 0x7f39c41506b0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c2ffd640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39c41506b0 0x7f39c4149370 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c390 tx=0x7f39b800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39abfff640 1 -- 172.21.15.26:0/1376069651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8017070 con 0x7f39c41506b0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39abfff640 1 -- 172.21.15.26:0/1376069651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8005d00 con 0x7f39c41506b0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c4151da0 con 0x7f39c41506b0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39abfff640 1 -- 172.21.15.26:0/1376069651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8007310 con 0x7f39c41506b0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.743+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c40c8300 con 0x7f39c41506b0 2024-03-06T07:33:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.744+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c4075300 con 0x7f39c41506b0 2024-03-06T07:33:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.745+0000 7f39abfff640 1 -- 172.21.15.26:0/1376069651 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39b801a070 con 0x7f39c41506b0 2024-03-06T07:33:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.745+0000 7f39abfff640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39b006b4d0 0x7f39b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.746+0000 7f39abfff640 1 -- 172.21.15.26:0/1376069651 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f39b8089860 con 0x7f39c41506b0 2024-03-06T07:33:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.746+0000 7f39c37fe640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39b006b4d0 0x7f39b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.746+0000 7f39c37fe640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39b006b4d0 0x7f39b006d990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f39b4036000 tx=0x7f39b4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.750+0000 7f39abfff640 1 -- 172.21.15.26:0/1376069651 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f39c4075300 con 0x7f39c41506b0 2024-03-06T07:33:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.937+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39c404f330 con 0x7f39c41506b0 2024-03-06T07:33:02.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.939+0000 7f39abfff640 1 -- 172.21.15.26:0/1376069651 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+17364 (secure 0 0 0) 0x7f39c406e0b0 con 0x7f39c41506b0 2024-03-06T07:33:02.941 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:02.942 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:30:29.675481+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.632872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.411822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.f","mappings":[{"from":0,"to":2}]},{"pgid":"34.1a","mappings":[{"from":0,"to":2}]},{"pgid":"34.21","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.117:6834/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.117:6837/3331811951":"2024-03-07T07:29:54.670724+0000","172.21.15.117:6835/3331811951":"2024-03-07T07:29:54.670724+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39b006b4d0 msgr2=0x7f39b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39b006b4d0 0x7f39b006d990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f39b4036000 tx=0x7f39b4030000 comp rx=0 tx=0).stop 2024-03-06T07:33:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39c41506b0 msgr2=0x7f39c4149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39c41506b0 0x7f39c4149370 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c390 tx=0x7f39b800c850 comp rx=0 tx=0).stop 2024-03-06T07:33:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 shutdown_connections 2024-03-06T07:33:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f39b006b4d0 0x7f39b006d990 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39c4156ea0 0x7f39c4146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39c41506b0 0x7f39c4149370 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 --2- 172.21.15.26:0/1376069651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c40acc00 0x7f39c41eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 >> 172.21.15.26:0/1376069651 conn(0x7f39c406b1a0 msgr2=0x7f39c40ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 shutdown_connections 2024-03-06T07:33:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:02.941+0000 7f39c9fac640 1 -- 172.21.15.26:0/1376069651 wait complete. 2024-03-06T07:33:02.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:33:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.054+0000 7fdb2ff72640 1 Processor -- start 2024-03-06T07:33:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.055+0000 7fdb2ff72640 1 -- start start 2024-03-06T07:33:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.055+0000 7fdb2ff72640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 0x7fdb280c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.055+0000 7fdb2ff72640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb280c8950 0x7fdb280acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.055+0000 7fdb2ff72640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb28155130 con 0x7fdb280ad2b0 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.055+0000 7fdb2ff72640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb28155440 con 0x7fdb280c8950 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.055+0000 7fdb2ff72640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280c5d50 con 0x7fdb280c69f0 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2e50f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdb280ad2b0 0x7fdb281558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57894/0 (socket says 172.21.15.26:57894) 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2e50f640 1 -- 172.21.15.26:0/650233693 learned_addr learned my addr 172.21.15.26:0/650233693 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2d50d640 1 --2- 172.21.15.26:0/650233693 >> v2:172.21.15.117:3300/0 conn(0x7fdb280c8950 0x7fdb280acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2dd0e640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 0x7fdb280c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395961642 0 0) 0x7fdb20001490 con 0x7fdb280ad2b0 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb14003520 con 0x7fdb280ad2b0 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1935178549 0 0) 0x7fdb14003520 con 0x7fdb280ad2b0 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 msgr2=0x7fdb280c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 0x7fdb280c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 >> v2:172.21.15.117:3300/0 conn(0x7fdb280c8950 msgr2=0x7fdb280acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 --2- 172.21.15.26:0/650233693 >> v2:172.21.15.117:3300/0 conn(0x7fdb280c8950 0x7fdb280acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280c6030 con 0x7fdb280ad2b0 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fdb20001bc0 con 0x7fdb280ad2b0 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 >> v1:172.21.15.26:6789/0 conn(0x7fdb280ad2b0 legacy=0x7fdb281558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb140039b0 0x7fdb14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 0x7fdb14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb14004d40 0x7fdb140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28155440 con 0x7fdb14004d40 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280c5d50 con 0x7fdb140039b0 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1400b9c0 con 0x7fdb280c69f0 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2e50f640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb14004d40 0x7fdb140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.056+0000 7fdb2dd0e640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb140039b0 0x7fdb14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2d50d640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 0x7fdb14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2e50f640 1 -- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 msgr2=0x7fdb14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2e50f640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb280c69f0 0x7fdb14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2e50f640 1 -- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb140039b0 msgr2=0x7fdb14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2e50f640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb140039b0 0x7fdb14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2e50f640 1 -- 172.21.15.26:0/650233693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb14003520 con 0x7fdb14004d40 2024-03-06T07:33:03.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2dd0e640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb140039b0 0x7fdb14003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2e50f640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb14004d40 0x7fdb140051c0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fdb280cb9b0 tx=0x7fdb14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f7fba52176f176 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.057+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000a690 con 0x7fdb14004d40 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb200072d0 con 0x7fdb14004d40 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2cd0c640 1 -- 172.21.15.26:0/650233693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb200076c0 con 0x7fdb14004d40 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2ff72640 1 -- 172.21.15.26:0/650233693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb14004d40 msgr2=0x7fdb140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2ff72640 1 --2- 172.21.15.26:0/650233693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb14004d40 0x7fdb140051c0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fdb280cb9b0 tx=0x7fdb14005830 comp rx=0 tx=0).stop 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2e50f640 1 -- 172.21.15.26:0/650233693 reap_dead start 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2ff72640 1 -- 172.21.15.26:0/650233693 shutdown_connections 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2ff72640 1 -- 172.21.15.26:0/650233693 >> 172.21.15.26:0/650233693 conn(0x7fdb280cae70 msgr2=0x7fdb280cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2ff72640 1 -- 172.21.15.26:0/650233693 shutdown_connections 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.058+0000 7fdb2ff72640 1 -- 172.21.15.26:0/650233693 wait complete. 2024-03-06T07:33:03.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.059+0000 7fdb2ff72640 1 Processor -- start 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.059+0000 7fdb2ff72640 1 -- start start 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2ff72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb280aa920 0x7fdb281eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2ff72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb281eb6f0 0x7fdb2814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2ff72640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2814efe0 0x7fdb2814f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2ff72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28152a20 con 0x7fdb280aa920 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2ff72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28155fe0 con 0x7fdb281eb6f0 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2ff72640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28152ba0 con 0x7fdb2814efe0 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2dd0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb280aa920 0x7fdb281eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2dd0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb280aa920 0x7fdb281eb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60774/0 (socket says 172.21.15.26:60774) 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2dd0e640 1 -- 172.21.15.26:0/1122826462 learned_addr learned my addr 172.21.15.26:0/1122826462 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2d50d640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb281eb6f0 0x7fdb2814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.060+0000 7fdb2e50f640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2814efe0 0x7fdb2814f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2dd0e640 1 -- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2814efe0 msgr2=0x7fdb2814f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2dd0e640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2814efe0 0x7fdb2814f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2dd0e640 1 -- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb281eb6f0 msgr2=0x7fdb2814eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2dd0e640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb281eb6f0 0x7fdb2814eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2dd0e640 1 -- 172.21.15.26:0/1122826462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280c2a90 con 0x7fdb280aa920 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2dd0e640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb280aa920 0x7fdb281eb1b0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fdb24011600 tx=0x7fdb24011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb1e7fc640 1 -- 172.21.15.26:0/1122826462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400f920 con 0x7fdb280aa920 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb1e7fc640 1 -- 172.21.15.26:0/1122826462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2400fac0 con 0x7fdb280aa920 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb281de790 con 0x7fdb280aa920 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb1e7fc640 1 -- 172.21.15.26:0/1122826462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400fe90 con 0x7fdb280aa920 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.061+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb281dea50 con 0x7fdb280aa920 2024-03-06T07:33:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.062+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2804f330 con 0x7fdb280aa920 2024-03-06T07:33:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.063+0000 7fdb1e7fc640 1 -- 172.21.15.26:0/1122826462 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb24015460 con 0x7fdb280aa920 2024-03-06T07:33:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.063+0000 7fdb1e7fc640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb0c06b6a0 0x7fdb0c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.063+0000 7fdb1e7fc640 1 -- 172.21.15.26:0/1122826462 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdb2408ae00 con 0x7fdb280aa920 2024-03-06T07:33:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.064+0000 7fdb2d50d640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb0c06b6a0 0x7fdb0c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.064+0000 7fdb2d50d640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb0c06b6a0 0x7fdb0c06db60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fdb281ebd50 tx=0x7fdb18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.072+0000 7fdb1e7fc640 1 -- 172.21.15.26:0/1122826462 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb24019030 con 0x7fdb280aa920 2024-03-06T07:33:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.298+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdb280741e0 con 0x7fdb280aa920 2024-03-06T07:33:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.300+0000 7fdb1e7fc640 1 -- 172.21.15.26:0/1122826462 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v216) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdb240599e0 con 0x7fdb280aa920 2024-03-06T07:33:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb0c06b6a0 msgr2=0x7fdb0c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb0c06b6a0 0x7fdb0c06db60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fdb281ebd50 tx=0x7fdb18008040 comp rx=0 tx=0).stop 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb280aa920 msgr2=0x7fdb281eb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb280aa920 0x7fdb281eb1b0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fdb24011600 tx=0x7fdb24011ac0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 shutdown_connections 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdb0c06b6a0 0x7fdb0c06db60 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2814efe0 0x7fdb2814f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb281eb6f0 0x7fdb2814eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 --2- 172.21.15.26:0/1122826462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb280aa920 0x7fdb281eb1b0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 >> 172.21.15.26:0/1122826462 conn(0x7fdb280cae70 msgr2=0x7fdb281509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 shutdown_connections 2024-03-06T07:33:03.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.303+0000 7fdb2ff72640 1 -- 172.21.15.26:0/1122826462 wait complete. 2024-03-06T07:33:03.309 INFO:tasks.ceph.mds.c.smithi026.stderr: -8466> 2024-03-06T07:29:20.284+0000 7fe00ae95640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:33:03.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:33:03.342 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-03-06T07:33:03.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.421+0000 7f3fa4efb640 1 Processor -- start 2024-03-06T07:33:03.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.421+0000 7f3fa4efb640 1 -- start start 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3fa4efb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fa00b00d0 0x7f3fa00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3fa4efb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa006b8e0 0x7f3fa006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3fa4efb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3fa00c7210 0x7f3fa00c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3fa4efb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa0156160 con 0x7f3fa00b00d0 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3fa4efb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa00c5710 con 0x7f3fa00c7210 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3fa4efb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa0152c50 con 0x7f3fa006b8e0 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3f9e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fa00b00d0 0x7f3fa00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3f9e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fa00b00d0 0x7f3fa00c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60790/0 (socket says 172.21.15.26:60790) 2024-03-06T07:33:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.422+0000 7f3f9e59c640 1 -- 172.21.15.26:0/1918343099 learned_addr learned my addr 172.21.15.26:0/1918343099 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9ed9d640 1 --2- 172.21.15.26:0/1918343099 >> v2:172.21.15.117:3300/0 conn(0x7f3fa00c7210 0x7f3fa00c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9dd9b640 1 --2- 172.21.15.26:0/1918343099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa006b8e0 0x7f3fa006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9e59c640 1 -- 172.21.15.26:0/1918343099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa006b8e0 msgr2=0x7f3fa006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9e59c640 1 --2- 172.21.15.26:0/1918343099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa006b8e0 0x7f3fa006bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9e59c640 1 -- 172.21.15.26:0/1918343099 >> v2:172.21.15.117:3300/0 conn(0x7f3fa00c7210 msgr2=0x7f3fa00c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9e59c640 1 --2- 172.21.15.26:0/1918343099 >> v2:172.21.15.117:3300/0 conn(0x7f3fa00c7210 0x7f3fa00c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9e59c640 1 -- 172.21.15.26:0/1918343099 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00b04b0 con 0x7f3fa00b00d0 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9dd9b640 1 --2- 172.21.15.26:0/1918343099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa006b8e0 0x7f3fa006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.423+0000 7f3f9e59c640 1 --2- 172.21.15.26:0/1918343099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00b00d0 0x7f3fa00c8ff0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00cac00 tx=0x7f3f9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3376787c7fe74193 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3f9d59a640 1 -- 172.21.15.26:0/1918343099 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98035020 con 0x7f3fa00b00d0 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3f9d59a640 1 -- 172.21.15.26:0/1918343099 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f98031870 con 0x7f3fa00b00d0 2024-03-06T07:33:03.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3f9d59a640 1 -- 172.21.15.26:0/1918343099 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98031c40 con 0x7f3fa00b00d0 2024-03-06T07:33:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 -- 172.21.15.26:0/1918343099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00b00d0 msgr2=0x7f3fa00c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/1918343099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00b00d0 0x7f3fa00c8ff0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00cac00 tx=0x7f3f9802b910 comp rx=0 tx=0).stop 2024-03-06T07:33:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 -- 172.21.15.26:0/1918343099 shutdown_connections 2024-03-06T07:33:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/1918343099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa006b8e0 0x7f3fa006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/1918343099 >> v2:172.21.15.117:3300/0 conn(0x7f3fa00c7210 0x7f3fa00c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/1918343099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00b00d0 0x7f3fa00c8ff0 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 -- 172.21.15.26:0/1918343099 >> 172.21.15.26:0/1918343099 conn(0x7f3fa00ac810 msgr2=0x7f3fa00acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 -- 172.21.15.26:0/1918343099 shutdown_connections 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.424+0000 7f3fa4efb640 1 -- 172.21.15.26:0/1918343099 wait complete. 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.425+0000 7f3fa4efb640 1 Processor -- start 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3fa4efb640 1 -- start start 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3fa4efb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa006b8e0 0x7f3fa01edf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3fa4efb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa00b00d0 0x7f3fa01584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3fa4efb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00c7210 0x7f3fa01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3fa4efb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa006c260 con 0x7f3fa00c7210 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3fa4efb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa0155e80 con 0x7f3fa006b8e0 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3fa4efb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00c5ba0 con 0x7f3fa00b00d0 2024-03-06T07:33:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00c7210 0x7f3fa01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00c7210 0x7f3fa01589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60792/0 (socket says 172.21.15.26:60792) 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3f9ed9d640 1 -- 172.21.15.26:0/2386447620 learned_addr learned my addr 172.21.15.26:0/2386447620 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3f9e59c640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa006b8e0 0x7f3fa01edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.426+0000 7f3f9dd9b640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa00b00d0 0x7f3fa01584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f9ed9d640 1 -- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa00b00d0 msgr2=0x7f3fa01584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f9ed9d640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa00b00d0 0x7f3fa01584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f9ed9d640 1 -- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa006b8e0 msgr2=0x7f3fa01edf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f9ed9d640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa006b8e0 0x7f3fa01edf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f9ed9d640 1 -- 172.21.15.26:0/2386447620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa014d690 con 0x7f3fa00c7210 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f9ed9d640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00c7210 0x7f3fa01589e0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007e60 tx=0x7f3f9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f96ffd640 1 -- 172.21.15.26:0/2386447620 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90014420 con 0x7f3fa00c7210 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f96ffd640 1 -- 172.21.15.26:0/2386447620 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90019070 con 0x7f3fa00c7210 2024-03-06T07:33:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3f96ffd640 1 -- 172.21.15.26:0/2386447620 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90014820 con 0x7f3fa00c7210 2024-03-06T07:33:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa014e410 con 0x7f3fa00c7210 2024-03-06T07:33:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.427+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa01ee9b0 con 0x7f3fa00c7210 2024-03-06T07:33:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.428+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa004f330 con 0x7f3fa00c7210 2024-03-06T07:33:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.428+0000 7f3f96ffd640 1 -- 172.21.15.26:0/2386447620 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f900149c0 con 0x7f3fa00c7210 2024-03-06T07:33:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.429+0000 7f3f96ffd640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3f8006b4d0 0x7f3f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.429+0000 7f3f96ffd640 1 -- 172.21.15.26:0/2386447620 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3f9008abc0 con 0x7f3fa00c7210 2024-03-06T07:33:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.429+0000 7f3f9e59c640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3f8006b4d0 0x7f3f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.429+0000 7f3f9e59c640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3f8006b4d0 0x7f3f8006d990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3f98036000 tx=0x7f3f9803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.433+0000 7f3f96ffd640 1 -- 172.21.15.26:0/2386447620 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3f90059140 con 0x7f3fa00c7210 2024-03-06T07:33:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:03.656+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3fa006ce50 con 0x7f3fa00c7210 2024-03-06T07:33:04.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.311+0000 7f3f96ffd640 1 -- 172.21.15.26:0/2386447620 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v217) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3f9005d1e0 con 0x7f3fa00c7210 2024-03-06T07:33:04.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3f8006b4d0 msgr2=0x7f3f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3f8006b4d0 0x7f3f8006d990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3f98036000 tx=0x7f3f9803c040 comp rx=0 tx=0).stop 2024-03-06T07:33:04.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00c7210 msgr2=0x7f3fa01589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00c7210 0x7f3fa01589e0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007e60 tx=0x7f3f9000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 shutdown_connections 2024-03-06T07:33:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3f8006b4d0 0x7f3f8006d990 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa00c7210 0x7f3fa01589e0 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fa00b00d0 0x7f3fa01584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 --2- 172.21.15.26:0/2386447620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa006b8e0 0x7f3fa01edf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.313+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 >> 172.21.15.26:0/2386447620 conn(0x7f3fa00ac810 msgr2=0x7f3fa0153360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.314+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 shutdown_connections 2024-03-06T07:33:04.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.314+0000 7f3fa4efb640 1 -- 172.21.15.26:0/2386447620 wait complete. 2024-03-06T07:33:04.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:33:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.483+0000 7f0f65a4e640 1 Processor -- start 2024-03-06T07:33:04.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.483+0000 7f0f65a4e640 1 -- start start 2024-03-06T07:33:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f65a4e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f600c22b0 0x7f0f60151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f65a4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 0x7f0f60153820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f65a4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6006e850 con 0x7f0f600a9830 2024-03-06T07:33:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f65a4e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6006d2b0 con 0x7f0f600c22b0 2024-03-06T07:33:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f65a4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6006d590 con 0x7f0f60153390 2024-03-06T07:33:04.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f600c22b0 0x7f0f60151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0f600a9830 0x7f0f6014ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57896/0 (socket says 172.21.15.26:57896) 2024-03-06T07:33:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5e7fc640 1 -- 172.21.15.26:0/1156369207 learned_addr learned my addr 172.21.15.26:0/1156369207 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5effd640 1 -- 172.21.15.26:0/1156369207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 msgr2=0x7f0f60153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5effd640 1 --2- 172.21.15.26:0/1156369207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 0x7f0f60153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5effd640 1 -- 172.21.15.26:0/1156369207 >> v1:172.21.15.26:6789/0 conn(0x7f0f600a9830 legacy=0x7f0f6014ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5effd640 1 -- 172.21.15.26:0/1156369207 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6006d870 con 0x7f0f600c22b0 2024-03-06T07:33:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.484+0000 7f0f5dffb640 1 -- 172.21.15.26:0/1156369207 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160378856 0 0) 0x7f0f48001110 con 0x7f0f600a9830 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.485+0000 7f0f5effd640 1 --2- 172.21.15.26:0/1156369207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600c22b0 0x7f0f60151e10 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097d0 tx=0x7f0f5802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b110dc16cfa58a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.485+0000 7f0f5dffb640 1 -- 172.21.15.26:0/1156369207 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58035020 con 0x7f0f600c22b0 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.485+0000 7f0f5dffb640 1 -- 172.21.15.26:0/1156369207 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f58031800 con 0x7f0f600c22b0 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.485+0000 7f0f5dffb640 1 -- 172.21.15.26:0/1156369207 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58031bd0 con 0x7f0f600c22b0 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1156369207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600c22b0 msgr2=0x7f0f60151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1156369207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600c22b0 0x7f0f60151e10 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097d0 tx=0x7f0f5802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1156369207 shutdown_connections 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1156369207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 0x7f0f60153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1156369207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600c22b0 0x7f0f60151e10 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1156369207 >> 172.21.15.26:0/1156369207 conn(0x7f0f600c65b0 msgr2=0x7f0f600c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1156369207 shutdown_connections 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.486+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1156369207 wait complete. 2024-03-06T07:33:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.487+0000 7f0f65a4e640 1 Processor -- start 2024-03-06T07:33:04.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.487+0000 7f0f65a4e640 1 -- start start 2024-03-06T07:33:04.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f65a4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600a9830 0x7f0f601df630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f65a4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f600c22b0 0x7f0f600bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f65a4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 0x7f0f600bf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f65a4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6014f7e0 con 0x7f0f600a9830 2024-03-06T07:33:04.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f65a4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6014f660 con 0x7f0f600c22b0 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f65a4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600c2690 con 0x7f0f60153390 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600a9830 0x7f0f601df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600a9830 0x7f0f601df630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60820/0 (socket says 172.21.15.26:60820) 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 -- 172.21.15.26:0/1966916283 learned_addr learned my addr 172.21.15.26:0/1966916283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5e7fc640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f600c22b0 0x7f0f600bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5f7fe640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 0x7f0f600bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 -- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 msgr2=0x7f0f600bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 0x7f0f600bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 -- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f600c22b0 msgr2=0x7f0f600bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f600c22b0 0x7f0f600bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 -- 172.21.15.26:0/1966916283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f601ea9f0 con 0x7f0f600a9830 2024-03-06T07:33:04.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.488+0000 7f0f5effd640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600a9830 0x7f0f601df630 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f0f580098f0 tx=0x7f0f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.489+0000 7f0f57fff640 1 -- 172.21.15.26:0/1966916283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580453f0 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.489+0000 7f0f57fff640 1 -- 172.21.15.26:0/1966916283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f580455b0 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.489+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f601eacd0 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.489+0000 7f0f57fff640 1 -- 172.21.15.26:0/1966916283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580459a0 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.489+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f601eb290 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.490+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6004f330 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.490+0000 7f0f57fff640 1 -- 172.21.15.26:0/1966916283 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f5804c060 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.491+0000 7f0f57fff640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f4006b500 0x7f0f4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.491+0000 7f0f57fff640 1 -- 172.21.15.26:0/1966916283 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0f580ac4c0 con 0x7f0f600a9830 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.491+0000 7f0f5e7fc640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f4006b500 0x7f0f4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.491+0000 7f0f5e7fc640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f4006b500 0x7f0f4006d9c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0f60155240 tx=0x7f0f48007970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.495+0000 7f0f57fff640 1 -- 172.21.15.26:0/1966916283 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f58040070 con 0x7f0f600a9830 2024-03-06T07:33:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:04.676+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f600ac280 con 0x7f0f600a9830 2024-03-06T07:33:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.321+0000 7f0f57fff640 1 -- 172.21.15.26:0/1966916283 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v217) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0f5804b2c0 con 0x7f0f600a9830 2024-03-06T07:33:05.323 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:33:05.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f4006b500 msgr2=0x7f0f4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f4006b500 0x7f0f4006d9c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0f60155240 tx=0x7f0f48007970 comp rx=0 tx=0).stop 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600a9830 msgr2=0x7f0f601df630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600a9830 0x7f0f601df630 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f0f580098f0 tx=0x7f0f580046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 shutdown_connections 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0f4006b500 0x7f0f4006d9c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f60153390 0x7f0f600bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f600c22b0 0x7f0f600bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 --2- 172.21.15.26:0/1966916283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f600a9830 0x7f0f601df630 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 >> 172.21.15.26:0/1966916283 conn(0x7f0f600c65b0 msgr2=0x7f0f60154160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:05.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 shutdown_connections 2024-03-06T07:33:05.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.325+0000 7f0f65a4e640 1 -- 172.21.15.26:0/1966916283 wait complete. 2024-03-06T07:33:05.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:33:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.453+0000 7f33377c7640 1 Processor -- start 2024-03-06T07:33:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.453+0000 7f33377c7640 1 -- start start 2024-03-06T07:33:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.453+0000 7f33377c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f333006fe20 0x7f3330155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.453+0000 7f33377c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33300c6780 0x7f3330072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f33377c7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3330156150 con 0x7f3330072a50 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f33377c7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33300a7d70 con 0x7f333006fe20 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f33377c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33300a8050 con 0x7f33300c6780 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3335563640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f333006fe20 0x7f3330155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3335d64640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3330072a50 0x7f3330158780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57902/0 (socket says 172.21.15.26:57902) 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3335d64640 1 -- 172.21.15.26:0/3483422148 learned_addr learned my addr 172.21.15.26:0/3483422148 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3335563640 1 -- 172.21.15.26:0/3483422148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33300c6780 msgr2=0x7f3330072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3335563640 1 --2- 172.21.15.26:0/3483422148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33300c6780 0x7f3330072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3335563640 1 -- 172.21.15.26:0/3483422148 >> v1:172.21.15.26:6789/0 conn(0x7f3330072a50 legacy=0x7f3330158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3335563640 1 -- 172.21.15.26:0/3483422148 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33300a8330 con 0x7f333006fe20 2024-03-06T07:33:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.454+0000 7f3327fff640 1 -- 172.21.15.26:0/3483422148 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006761638 0 0) 0x7f3328001490 con 0x7f3330072a50 2024-03-06T07:33:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f3335563640 1 --2- 172.21.15.26:0/3483422148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333006fe20 0x7f3330155c10 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f332c0097a0 tx=0x7f332c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad500b410ee842c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f3327fff640 1 -- 172.21.15.26:0/3483422148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0315a0 con 0x7f333006fe20 2024-03-06T07:33:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f3327fff640 1 -- 172.21.15.26:0/3483422148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c037070 con 0x7f333006fe20 2024-03-06T07:33:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f3327fff640 1 -- 172.21.15.26:0/3483422148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c031960 con 0x7f333006fe20 2024-03-06T07:33:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f33377c7640 1 -- 172.21.15.26:0/3483422148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333006fe20 msgr2=0x7f3330155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f33377c7640 1 --2- 172.21.15.26:0/3483422148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333006fe20 0x7f3330155c10 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f332c0097a0 tx=0x7f332c02b840 comp rx=0 tx=0).stop 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f33377c7640 1 -- 172.21.15.26:0/3483422148 shutdown_connections 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f33377c7640 1 --2- 172.21.15.26:0/3483422148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33300c6780 0x7f3330072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f33377c7640 1 --2- 172.21.15.26:0/3483422148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333006fe20 0x7f3330155c10 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.455+0000 7f33377c7640 1 -- 172.21.15.26:0/3483422148 >> 172.21.15.26:0/3483422148 conn(0x7f33300aa630 msgr2=0x7f33300aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.456+0000 7f33377c7640 1 -- 172.21.15.26:0/3483422148 shutdown_connections 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.456+0000 7f33377c7640 1 -- 172.21.15.26:0/3483422148 wait complete. 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 Processor -- start 2024-03-06T07:33:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 -- start start 2024-03-06T07:33:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333006fe20 0x7f3330153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3330072a50 0x7f33301ee570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33300c6780 0x7f33301eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3330070200 con 0x7f3330072a50 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33300c3600 con 0x7f33300c6780 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f33377c7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3330158e90 con 0x7f333006fe20 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f3334d62640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3330072a50 0x7f33301ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f3334d62640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3330072a50 0x7f33301ee570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60826/0 (socket says 172.21.15.26:60826) 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.457+0000 7f3334d62640 1 -- 172.21.15.26:0/1157098362 learned_addr learned my addr 172.21.15.26:0/1157098362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3335563640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333006fe20 0x7f3330153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3334d62640 1 -- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333006fe20 msgr2=0x7f3330153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3335d64640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33300c6780 0x7f33301eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3334d62640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333006fe20 0x7f3330153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3334d62640 1 -- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33300c6780 msgr2=0x7f33301eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3334d62640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33300c6780 0x7f33301eeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3334d62640 1 -- 172.21.15.26:0/1157098362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330153650 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3334d62640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3330072a50 0x7f33301ee570 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f332000b410 tx=0x7f332000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3325ffb640 1 -- 172.21.15.26:0/1157098362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320015020 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3325ffb640 1 -- 172.21.15.26:0/1157098362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3320011850 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33301551c0 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f3325ffb640 1 -- 172.21.15.26:0/1157098362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320011c40 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.458+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33301dea40 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.459+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333004f330 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.460+0000 7f3325ffb640 1 -- 172.21.15.26:0/1157098362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3320018020 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.460+0000 7f3325ffb640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f331406b4d0 0x7f331406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.460+0000 7f3325ffb640 1 -- 172.21.15.26:0/1157098362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6094+0+0 (secure 0 0 0) 0x7f332008a220 con 0x7f3330072a50 2024-03-06T07:33:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.460+0000 7f3335563640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f331406b4d0 0x7f331406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.461+0000 7f3335563640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f331406b4d0 0x7f331406d990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f332c036000 tx=0x7f332c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.465+0000 7f3325ffb640 1 -- 172.21.15.26:0/1157098362 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f332001f020 con 0x7f3330072a50 2024-03-06T07:33:05.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:05.644+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33300730d0 con 0x7f3330072a50 2024-03-06T07:33:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.326+0000 7f3325ffb640 1 -- 172.21.15.26:0/1157098362 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v218) v1 ==== 169+0+0 (secure 0 0 0) 0x7f33200172c0 con 0x7f3330072a50 2024-03-06T07:33:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-03-06T07:33:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f331406b4d0 msgr2=0x7f331406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f331406b4d0 0x7f331406d990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f332c036000 tx=0x7f332c008030 comp rx=0 tx=0).stop 2024-03-06T07:33:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3330072a50 msgr2=0x7f33301ee570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3330072a50 0x7f33301ee570 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f332000b410 tx=0x7f332000b8d0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 shutdown_connections 2024-03-06T07:33:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f331406b4d0 0x7f331406d990 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33300c6780 0x7f33301eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3330072a50 0x7f33301ee570 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 --2- 172.21.15.26:0/1157098362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333006fe20 0x7f3330153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 >> 172.21.15.26:0/1157098362 conn(0x7f33300aa630 msgr2=0x7f33300c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:06.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 shutdown_connections 2024-03-06T07:33:06.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.329+0000 7f33377c7640 1 -- 172.21.15.26:0/1157098362 wait complete. 2024-03-06T07:33:06.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:33:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.455+0000 7f29f08b5640 1 Processor -- start 2024-03-06T07:33:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.455+0000 7f29f08b5640 1 -- start start 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.456+0000 7f29f08b5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29ec06afc0 0x7f29ec0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.456+0000 7f29f08b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 0x7f29ec06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.456+0000 7f29f08b5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec0c7430 con 0x7f29ec06b3a0 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.456+0000 7f29f08b5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec06fde0 con 0x7f29ec06afc0 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.456+0000 7f29f08b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec156fc0 con 0x7f29ec1536a0 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.456+0000 7f29e9d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f29ec06b3a0 0x7f29ec152f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57910/0 (socket says 172.21.15.26:57910) 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.456+0000 7f29e9d9b640 1 -- 172.21.15.26:0/2005769689 learned_addr learned my addr 172.21.15.26:0/2005769689 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035266485 0 0) 0x7f29ec0c7430 con 0x7f29ec06b3a0 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29ead9d640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 0x7f29ec06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d8003520 con 0x7f29ec06b3a0 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29ea59c640 1 --2- 172.21.15.26:0/2005769689 >> v2:172.21.15.117:3300/0 conn(0x7f29ec06afc0 0x7f29ec0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3528985404 0 0) 0x7f29d8003520 con 0x7f29ec06b3a0 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 msgr2=0x7f29ec06f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 0x7f29ec06f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 >> v2:172.21.15.117:3300/0 conn(0x7f29ec06afc0 msgr2=0x7f29ec0c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 --2- 172.21.15.26:0/2005769689 >> v2:172.21.15.117:3300/0 conn(0x7f29ec06afc0 0x7f29ec0c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec157450 con 0x7f29ec06b3a0 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f29d4001670 con 0x7f29ec06b3a0 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 >> v1:172.21.15.26:6789/0 conn(0x7f29ec06b3a0 legacy=0x7f29ec152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 0x7f29d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d8003f30 0x7f29d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec156fc0 con 0x7f29d8004d40 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d800b9c0 con 0x7f29d8003f30 2024-03-06T07:33:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d800bb90 con 0x7f29ec1536a0 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29ead9d640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29e9d9b640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d8003f30 0x7f29d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29ea59c640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 0x7f29d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29ead9d640 1 -- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 msgr2=0x7f29d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29ead9d640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec1536a0 0x7f29d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.457+0000 7f29ead9d640 1 -- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d8003f30 msgr2=0x7f29d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.458+0000 7f29ead9d640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d8003f30 0x7f29d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.458+0000 7f29ead9d640 1 -- 172.21.15.26:0/2005769689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d8003520 con 0x7f29d8004d40 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.458+0000 7f29ead9d640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29d80051c0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00e170 tx=0x7f29d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c7d17937f578a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.458+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc039070 con 0x7f29d8004d40 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.458+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc02cbf0 con 0x7f29d8004d40 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.458+0000 7f29e959a640 1 -- 172.21.15.26:0/2005769689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc009c10 con 0x7f29d8004d40 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.459+0000 7f29f08b5640 1 -- 172.21.15.26:0/2005769689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 msgr2=0x7f29d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.459+0000 7f29f08b5640 1 --2- 172.21.15.26:0/2005769689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29d80051c0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00e170 tx=0x7f29d8005830 comp rx=0 tx=0).stop 2024-03-06T07:33:06.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.459+0000 7f29ead9d640 1 -- 172.21.15.26:0/2005769689 reap_dead start 2024-03-06T07:33:06.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.459+0000 7f29f08b5640 1 -- 172.21.15.26:0/2005769689 shutdown_connections 2024-03-06T07:33:06.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.459+0000 7f29f08b5640 1 -- 172.21.15.26:0/2005769689 >> 172.21.15.26:0/2005769689 conn(0x7f29ec0c4220 msgr2=0x7f29ec0c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:06.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.459+0000 7f29f08b5640 1 -- 172.21.15.26:0/2005769689 shutdown_connections 2024-03-06T07:33:06.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.459+0000 7f29f08b5640 1 -- 172.21.15.26:0/2005769689 wait complete. 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.460+0000 7f29f08b5640 1 Processor -- start 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.460+0000 7f29f08b5640 1 -- start start 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29f08b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29ec150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29f08b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec0acb10 0x7f29ec151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29f08b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29ec155be0 0x7f29ec1ec260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29f08b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec156970 con 0x7f29d8004d40 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29f08b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec06b820 con 0x7f29ec155be0 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29f08b5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec156c50 con 0x7f29ec0acb10 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29ec150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29ec150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60830/0 (socket says 172.21.15.26:60830) 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 -- 172.21.15.26:0/3949352637 learned_addr learned my addr 172.21.15.26:0/3949352637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29e9d9b640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec0acb10 0x7f29ec151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ead9d640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29ec155be0 0x7f29ec1ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 -- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec0acb10 msgr2=0x7f29ec151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec0acb10 0x7f29ec151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 -- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29ec155be0 msgr2=0x7f29ec1ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29ec155be0 0x7f29ec1ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29ea59c640 1 -- 172.21.15.26:0/3949352637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0a92e0 con 0x7f29d8004d40 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.461+0000 7f29e9d9b640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec0acb10 0x7f29ec151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.462+0000 7f29ea59c640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29ec150f20 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f29e4011aa0 tx=0x7f29e4011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.462+0000 7f29e2ffd640 1 -- 172.21.15.26:0/3949352637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400fb80 con 0x7f29d8004d40 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.462+0000 7f29e2ffd640 1 -- 172.21.15.26:0/3949352637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e400fd20 con 0x7f29d8004d40 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.462+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec0aa720 con 0x7f29d8004d40 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.462+0000 7f29e2ffd640 1 -- 172.21.15.26:0/3949352637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4014660 con 0x7f29d8004d40 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.462+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec1eba60 con 0x7f29d8004d40 2024-03-06T07:33:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.463+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec04f330 con 0x7f29d8004d40 2024-03-06T07:33:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.463+0000 7f29e2ffd640 1 -- 172.21.15.26:0/3949352637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29e4012040 con 0x7f29d8004d40 2024-03-06T07:33:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.464+0000 7f29e2ffd640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f29cc06b4d0 0x7f29cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.464+0000 7f29e2ffd640 1 -- 172.21.15.26:0/3949352637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f29e408a960 con 0x7f29d8004d40 2024-03-06T07:33:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.464+0000 7f29e9d9b640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f29cc06b4d0 0x7f29cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.464+0000 7f29e9d9b640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f29cc06b4d0 0x7f29cc06d990 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0c1970 tx=0x7f29d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.468+0000 7f29e2ffd640 1 -- 172.21.15.26:0/3949352637 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f29e4019030 con 0x7f29d8004d40 2024-03-06T07:33:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.648+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f29ec0c6870 con 0x7f29d8004d40 2024-03-06T07:33:06.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.651+0000 7f29e2ffd640 1 -- 172.21.15.26:0/3949352637 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f29e40597c0 con 0x7f29d8004d40 2024-03-06T07:33:06.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f29cc06b4d0 msgr2=0x7f29cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f29cc06b4d0 0x7f29cc06d990 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0c1970 tx=0x7f29d4009040 comp rx=0 tx=0).stop 2024-03-06T07:33:06.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 msgr2=0x7f29ec150f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29ec150f20 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f29e4011aa0 tx=0x7f29e4011f60 comp rx=0 tx=0).stop 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 shutdown_connections 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f29cc06b4d0 0x7f29cc06d990 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29ec155be0 0x7f29ec1ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29ec0acb10 0x7f29ec151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.653+0000 7f29f08b5640 1 --2- 172.21.15.26:0/3949352637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d8004d40 0x7f29ec150f20 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.654+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 >> 172.21.15.26:0/3949352637 conn(0x7f29ec0c4220 msgr2=0x7f29ec06ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.654+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 shutdown_connections 2024-03-06T07:33:06.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.654+0000 7f29f08b5640 1 -- 172.21.15.26:0/3949352637 wait complete. 2024-03-06T07:33:06.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_twice' 2024-03-06T07:33:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.766+0000 7fa5f006f640 1 Processor -- start 2024-03-06T07:33:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.766+0000 7fa5f006f640 1 -- start start 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5f006f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa5e80c6000 0x7fa5e8156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5f006f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e8153a60 con 0x7fa5e80c1320 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5f006f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e8153d40 con 0x7fa5e8150580 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5f006f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e806c2c0 con 0x7fa5e80c6000 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ed60a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa5e80c1320 0x7fa5e81566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57914/0 (socket says 172.21.15.26:57914) 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ed60a640 1 -- 172.21.15.26:0/1375440374 learned_addr learned my addr 172.21.15.26:0/1375440374 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ee60c640 1 --2- 172.21.15.26:0/1375440374 >> v2:172.21.15.117:3300/0 conn(0x7fa5e80c6000 0x7fa5e8156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163740214 0 0) 0x7fa5e8153a60 con 0x7fa5e80c1320 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d4003520 con 0x7fa5e80c1320 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646782098 0 0) 0x7fa5d4003520 con 0x7fa5e80c1320 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 >> v2:172.21.15.117:3300/0 conn(0x7fa5e80c6000 msgr2=0x7fa5e8156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 --2- 172.21.15.26:0/1375440374 >> v2:172.21.15.117:3300/0 conn(0x7fa5e80c6000 0x7fa5e8156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 >> v1:172.21.15.117:6789/0 conn(0x7fa5e8150580 legacy=0x7fa5e806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e806c5a0 con 0x7fa5e80c1320 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.767+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fa5d8001670 con 0x7fa5e80c1320 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 >> v1:172.21.15.26:6789/0 conn(0x7fa5e80c1320 legacy=0x7fa5e81566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ece09640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40039b0 0x7fa5d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ece09640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ece09640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d40051c0 0x7fa5d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e806c2c0 con 0x7fa5d4004800 2024-03-06T07:33:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d400c160 con 0x7fa5d40051c0 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d400c360 con 0x7fa5d40039b0 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ed60a640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ee60c640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d40051c0 0x7fa5d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ede0b640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40039b0 0x7fa5d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ed60a640 1 -- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40039b0 msgr2=0x7fa5d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ed60a640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40039b0 0x7fa5d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ed60a640 1 -- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d40051c0 msgr2=0x7fa5d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ed60a640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d40051c0 0x7fa5d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ed60a640 1 -- 172.21.15.26:0/1375440374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4003520 con 0x7fa5d4004800 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ee60c640 1 -- 172.21.15.26:0/1375440374 reap_dead start 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.768+0000 7fa5ede0b640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40039b0 0x7fa5d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5ed60a640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5d4004c80 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c240 tx=0x7fa5d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ec2322ee3d7697 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8007cf0 con 0x7fa5d4004800 2024-03-06T07:33:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d8030070 con 0x7fa5d4004800 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5ece09640 1 -- 172.21.15.26:0/1375440374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8005d70 con 0x7fa5d4004800 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5f006f640 1 -- 172.21.15.26:0/1375440374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 msgr2=0x7fa5d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5f006f640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5d4004c80 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c240 tx=0x7fa5d4005830 comp rx=0 tx=0).stop 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5f006f640 1 -- 172.21.15.26:0/1375440374 shutdown_connections 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5f006f640 1 --2- 172.21.15.26:0/1375440374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5d4004c80 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5f006f640 1 -- 172.21.15.26:0/1375440374 >> 172.21.15.26:0/1375440374 conn(0x7fa5e80c6710 msgr2=0x7fa5e80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5f006f640 1 -- 172.21.15.26:0/1375440374 shutdown_connections 2024-03-06T07:33:06.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.769+0000 7fa5f006f640 1 -- 172.21.15.26:0/1375440374 wait complete. 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 Processor -- start 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 -- start start 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5e8154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5e80c2cc0 0x7fa5e81ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5e81ec3b0 0x7fa5e8147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e80c0dc0 con 0x7fa5d4004800 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e80c0c40 con 0x7fa5e80c2cc0 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5f006f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e806dd10 con 0x7fa5e81ec3b0 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5ede0b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5e8154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5ede0b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5e8154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60844/0 (socket says 172.21.15.26:60844) 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5ede0b640 1 -- 172.21.15.26:0/3913573131 learned_addr learned my addr 172.21.15.26:0/3913573131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.771+0000 7fa5ed60a640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5e80c2cc0 0x7fa5e81ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ee60c640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5e81ec3b0 0x7fa5e8147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ede0b640 1 -- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5e81ec3b0 msgr2=0x7fa5e8147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ede0b640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5e81ec3b0 0x7fa5e8147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ede0b640 1 -- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5e80c2cc0 msgr2=0x7fa5e81ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ede0b640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5e80c2cc0 0x7fa5e81ebe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ede0b640 1 -- 172.21.15.26:0/3913573131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8155520 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ee60c640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5e81ec3b0 0x7fa5e8147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5ede0b640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5e8154fe0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e4b0 tx=0x7fa5e400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5de7fc640 1 -- 172.21.15.26:0/3913573131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40052e0 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5de7fc640 1 -- 172.21.15.26:0/3913573131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4007020 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e81541e0 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5de7fc640 1 -- 172.21.15.26:0/3913573131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4007210 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.772+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e8149940 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.773+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e804f330 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.773+0000 7fa5de7fc640 1 -- 172.21.15.26:0/3913573131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5e401a030 con 0x7fa5d4004800 2024-03-06T07:33:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.774+0000 7fa5de7fc640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5cc06b500 0x7fa5cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.774+0000 7fa5de7fc640 1 -- 172.21.15.26:0/3913573131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fa5e4089600 con 0x7fa5d4004800 2024-03-06T07:33:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.774+0000 7fa5ed60a640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5cc06b500 0x7fa5cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.774+0000 7fa5ed60a640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5cc06b500 0x7fa5cc06d9c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8034000 tx=0x7fa5d8007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:06.778+0000 7fa5de7fc640 1 -- 172.21.15.26:0/3913573131 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa5e4021020 con 0x7fa5d4004800 2024-03-06T07:33:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.033+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]} v 0) v1 -- 0x7fa5e80c5de0 con 0x7fa5d4004800 2024-03-06T07:33:07.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.328+0000 7fa5de7fc640 1 -- 172.21.15.26:0/3913573131 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]}]=0 v707) v1 ==== 139+0+0 (secure 0 0 0) 0x7fa5e8155520 con 0x7fa5d4004800 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5cc06b500 msgr2=0x7fa5cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5cc06b500 0x7fa5cc06d9c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8034000 tx=0x7fa5d8007000 comp rx=0 tx=0).stop 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 msgr2=0x7fa5e8154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5e8154fe0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e4b0 tx=0x7fa5e400e970 comp rx=0 tx=0).stop 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 shutdown_connections 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fa5cc06b500 0x7fa5cc06d9c0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5e81ec3b0 0x7fa5e8147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5e80c2cc0 0x7fa5e81ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 --2- 172.21.15.26:0/3913573131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5d4004800 0x7fa5e8154fe0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 >> 172.21.15.26:0/3913573131 conn(0x7fa5e80c6710 msgr2=0x7fa5e80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 shutdown_connections 2024-03-06T07:33:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:07.330+0000 7fa5f006f640 1 -- 172.21.15.26:0/3913573131 wait complete. 2024-03-06T07:33:07.344 INFO:tasks.cephfs_test_runner:That marking a FS down twice does not wipe old_max_mds. ... ERROR 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner:ERROR: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner:That marking a FS down twice does not wipe old_max_mds. 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/test_failover.py", line 274, in test_down_twice 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner: self.fs.wait_for_daemons() 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/filesystem.py", line 1240, in wait_for_daemons 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:07.345 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:Ran 12 tests in 749.419s 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:ERROR: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:That marking a FS down twice does not wipe old_max_mds. 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/test_failover.py", line 274, in test_down_twice 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner: self.fs.wait_for_daemons() 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/filesystem.py", line 1240, in wait_for_daemons 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-03-06T07:33:07.346 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:07.347 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:07.543 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=023d6d0c9d2b418bb034fcd08677d475 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:07.544 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-06T07:33:07.556 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-06T07:33:07.602 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-06T07:33:07.602 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-06T07:33:07.602 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:33:07.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057146,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1949 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:33:07.622 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-06T07:33:07.623 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:33:07.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20620 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:33:07.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:07.642 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-03-06T07:33:07.679 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-06T07:33:07.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:07.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-06T07:33:07.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:07.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-03-06T07:33:07.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:07.847 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-06T07:33:07.865 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-06T07:33:07.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:07.866 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-06T07:33:07.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:07.938 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-06T07:33:08.036 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-06T07:33:08.046 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-06T07:33:08.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:08.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 Processor -- start 2024-03-06T07:33:08.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 -- start start 2024-03-06T07:33:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e640c1aa0 0x7f7e640c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e640c5c60 0x7f7e64155a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e640c2680 con 0x7f7e640c5c60 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e640c28d0 con 0x7f7e640c1aa0 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.152+0000 7f7e6a5d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e640c2bb0 con 0x7f7e6414f790 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e640c5c60 0x7f7e64155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e640c5c60 0x7f7e64155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36078/0 (socket says 172.21.15.26:36078) 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 -- 172.21.15.26:0/1916692861 learned_addr learned my addr 172.21.15.26:0/1916692861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e637fe640 1 --2- 172.21.15.26:0/1916692861 >> v2:172.21.15.117:3300/0 conn(0x7f7e640c1aa0 0x7f7e640c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 -- 172.21.15.26:0/1916692861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 msgr2=0x7f7e641529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e63fff640 1 --2- 172.21.15.26:0/1916692861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 --2- 172.21.15.26:0/1916692861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 -- 172.21.15.26:0/1916692861 >> v2:172.21.15.117:3300/0 conn(0x7f7e640c1aa0 msgr2=0x7f7e640c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 --2- 172.21.15.26:0/1916692861 >> v2:172.21.15.117:3300/0 conn(0x7f7e640c1aa0 0x7f7e640c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e68b74640 1 -- 172.21.15.26:0/1916692861 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e640c2e90 con 0x7f7e640c5c60 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e637fe640 1 --2- 172.21.15.26:0/1916692861 >> v2:172.21.15.117:3300/0 conn(0x7f7e640c1aa0 0x7f7e640c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.153+0000 7f7e63fff640 1 --2- 172.21.15.26:0/1916692861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e68b74640 1 --2- 172.21.15.26:0/1916692861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e64155a40 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f7e640c76f0 tx=0x7f7e5c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ecad307f0b9043 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e62ffd640 1 -- 172.21.15.26:0/1916692861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c02ca70 con 0x7f7e640c5c60 2024-03-06T07:33:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e62ffd640 1 -- 172.21.15.26:0/1916692861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c02cc10 con 0x7f7e640c5c60 2024-03-06T07:33:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e62ffd640 1 -- 172.21.15.26:0/1916692861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0343f0 con 0x7f7e640c5c60 2024-03-06T07:33:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1916692861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 msgr2=0x7f7e64155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1916692861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e64155a40 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f7e640c76f0 tx=0x7f7e5c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1916692861 shutdown_connections 2024-03-06T07:33:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1916692861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e64155a40 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1916692861 >> v2:172.21.15.117:3300/0 conn(0x7f7e640c1aa0 0x7f7e640c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1916692861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1916692861 >> 172.21.15.26:0/1916692861 conn(0x7f7e640c6710 msgr2=0x7f7e640c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1916692861 shutdown_connections 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.154+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1916692861 wait complete. 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.155+0000 7f7e6a5d7640 1 Processor -- start 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e6a5d7640 1 -- start start 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e6a5d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e640c1aa0 0x7f7e641dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e6a5d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e641ead30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e6a5d7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641eb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e6a5d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6406cc00 con 0x7f7e640c5c60 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e6a5d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e640c0d60 con 0x7f7e640c1aa0 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e6a5d7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6406cd80 con 0x7f7e6414f790 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e641ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e641ead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36088/0 (socket says 172.21.15.26:36088) 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.156+0000 7f7e637fe640 1 -- 172.21.15.26:0/1064266657 learned_addr learned my addr 172.21.15.26:0/1064266657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e63fff640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e640c1aa0 0x7f7e641dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e68b74640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e637fe640 1 -- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 msgr2=0x7f7e641eb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e637fe640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641eb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e637fe640 1 -- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e640c1aa0 msgr2=0x7f7e641dfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e637fe640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e640c1aa0 0x7f7e641dfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e637fe640 1 -- 172.21.15.26:0/1064266657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e64071d70 con 0x7f7e640c5c60 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e637fe640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e641ead30 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c7c0 tx=0x7f7e5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e60ff9640 1 -- 172.21.15.26:0/1064266657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58015070 con 0x7f7e640c5c60 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e60ff9640 1 -- 172.21.15.26:0/1064266657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58007a30 con 0x7f7e640c5c60 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e641eebd0 con 0x7f7e640c5c60 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e60ff9640 1 -- 172.21.15.26:0/1064266657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58007e00 con 0x7f7e640c5c60 2024-03-06T07:33:08.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.157+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6414e5c0 con 0x7f7e640c5c60 2024-03-06T07:33:08.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.158+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e641eaa10 con 0x7f7e640c5c60 2024-03-06T07:33:08.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.159+0000 7f7e60ff9640 1 -- 172.21.15.26:0/1064266657 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e58018070 con 0x7f7e640c5c60 2024-03-06T07:33:08.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.159+0000 7f7e60ff9640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e5006b4d0 0x7f7e5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.159+0000 7f7e60ff9640 1 -- 172.21.15.26:0/1064266657 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f7e58089ff0 con 0x7f7e640c5c60 2024-03-06T07:33:08.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.159+0000 7f7e63fff640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e5006b4d0 0x7f7e5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.160+0000 7f7e63fff640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e5006b4d0 0x7f7e5006d990 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400a950 tx=0x7f7e54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.163+0000 7f7e60ff9640 1 -- 172.21.15.26:0/1064266657 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7e641eaa10 con 0x7f7e640c5c60 2024-03-06T07:33:08.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.339+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e641451a0 con 0x7f7e5006b4d0 2024-03-06T07:33:08.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.340+0000 7f7e60ff9640 1 -- 172.21.15.26:0/1064266657 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13519 (secure 0 0 0) 0x7f7e5400a1b0 con 0x7f7e5006b4d0 2024-03-06T07:33:08.342 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:08.342 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:33:08.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e5006b4d0 msgr2=0x7f7e5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e5006b4d0 0x7f7e5006d990 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400a950 tx=0x7f7e54008040 comp rx=0 tx=0).stop 2024-03-06T07:33:08.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 msgr2=0x7f7e641ead30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e641ead30 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c7c0 tx=0x7f7e5800cc80 comp rx=0 tx=0).stop 2024-03-06T07:33:08.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 shutdown_connections 2024-03-06T07:33:08.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f7e5006b4d0 0x7f7e5006d990 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e6414f790 0x7f7e641eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e640c5c60 0x7f7e641ead30 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 --2- 172.21.15.26:0/1064266657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e640c1aa0 0x7f7e641dfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 >> 172.21.15.26:0/1064266657 conn(0x7f7e640c6710 msgr2=0x7f7e640a9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:08.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 shutdown_connections 2024-03-06T07:33:08.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.342+0000 7f7e6a5d7640 1 -- 172.21.15.26:0/1064266657 wait complete. 2024-03-06T07:33:08.354 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-03-06T07:33:07.211673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283520,"kb_used_data":21600,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754691200,"statfs":{"total":773094113280,"available":772803788800,"internally_reserved":0,"allocated":22118400,"data_stored":11827360,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-88,"num_read_kb":-94,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002107"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":538,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-03-06T07:33:03.303824+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:33:03.303824+0000","last_peered":"2024-03-06T07:33:03.303824+0000","last_clean":"2024-03-06T07:33:03.303824+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:33:03.303824+0000","last_undegraded":"2024-03-06T07:33:03.303824+0000","last_fullsized":"2024-03-06T07:33:03.303824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33936,"kb_used_data":2756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337904,"statfs":{"total":96636764160,"available":96602013696,"internally_reserved":0,"allocated":2822144,"data_stored":1924920,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33860,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337980,"statfs":{"total":96636764160,"available":96602091520,"internally_reserved":0,"allocated":2744320,"data_stored":1333360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33852,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337988,"statfs":{"total":96636764160,"available":96602099712,"internally_reserved":0,"allocated":2736128,"data_stored":1326514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2752512,"data_stored":1334938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38016,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333824,"statfs":{"total":96636764160,"available":96597835776,"internally_reserved":0,"allocated":2740224,"data_stored":1327184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2764800,"data_stored":1335632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38032,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333808,"statfs":{"total":96636764160,"available":96597819392,"internally_reserved":0,"allocated":2756608,"data_stored":1327862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38076,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333764,"statfs":{"total":96636764160,"available":96597774336,"internally_reserved":0,"allocated":2801664,"data_stored":1916950,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:08.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.455+0000 7f3fc420e640 1 Processor -- start 2024-03-06T07:33:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.455+0000 7f3fc420e640 1 -- start start 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc420e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fbc0c6780 0x7f3fbc158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc420e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3fbc1592d0 0x7f3fbc0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc420e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc072630 con 0x7f3fbc06da40 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc420e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc06de20 con 0x7f3fbc0c6780 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc420e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc072910 con 0x7f3fbc1592d0 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc17a9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fbc0c6780 0x7f3fbc158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc17a9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fbc0c6780 0x7f3fbc158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36092/0 (socket says 172.21.15.26:36092) 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc17a9640 1 -- 172.21.15.26:0/2647214286 learned_addr learned my addr 172.21.15.26:0/2647214286 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc27ab640 1 --2- 172.21.15.26:0/2647214286 >> v2:172.21.15.117:3300/0 conn(0x7f3fbc1592d0 0x7f3fbc0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc17a9640 1 -- 172.21.15.26:0/2647214286 >> v2:172.21.15.117:3300/0 conn(0x7f3fbc1592d0 msgr2=0x7f3fbc0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc17a9640 1 --2- 172.21.15.26:0/2647214286 >> v2:172.21.15.117:3300/0 conn(0x7f3fbc1592d0 0x7f3fbc0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc17a9640 1 -- 172.21.15.26:0/2647214286 >> v1:172.21.15.117:6789/0 conn(0x7f3fbc06da40 legacy=0x7f3fbc155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.456+0000 7f3fc17a9640 1 -- 172.21.15.26:0/2647214286 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc072bf0 con 0x7f3fbc0c6780 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.457+0000 7f3fc17a9640 1 --2- 172.21.15.26:0/2647214286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc0c6780 0x7f3fbc158d90 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0c4b80 tx=0x7f3fb002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dfa26420bfc050 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.457+0000 7f3fc0fa8640 1 -- 172.21.15.26:0/2647214286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb002e070 con 0x7f3fbc0c6780 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.457+0000 7f3fc0fa8640 1 -- 172.21.15.26:0/2647214286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb0037070 con 0x7f3fbc0c6780 2024-03-06T07:33:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.457+0000 7f3fc0fa8640 1 -- 172.21.15.26:0/2647214286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0031a00 con 0x7f3fbc0c6780 2024-03-06T07:33:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 -- 172.21.15.26:0/2647214286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc0c6780 msgr2=0x7f3fbc158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2647214286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc0c6780 0x7f3fbc158d90 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0c4b80 tx=0x7f3fb002b880 comp rx=0 tx=0).stop 2024-03-06T07:33:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 -- 172.21.15.26:0/2647214286 shutdown_connections 2024-03-06T07:33:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2647214286 >> v2:172.21.15.117:3300/0 conn(0x7f3fbc1592d0 0x7f3fbc0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2647214286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc0c6780 0x7f3fbc158d90 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 -- 172.21.15.26:0/2647214286 >> 172.21.15.26:0/2647214286 conn(0x7f3fbc0aa630 msgr2=0x7f3fbc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 -- 172.21.15.26:0/2647214286 shutdown_connections 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.458+0000 7f3fc420e640 1 -- 172.21.15.26:0/2647214286 wait complete. 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.459+0000 7f3fc420e640 1 Processor -- start 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.459+0000 7f3fc420e640 1 -- start start 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.459+0000 7f3fc420e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc06da40 0x7f3fbc1de960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc420e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fbc0c6780 0x7f3fbc14c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc420e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fbc1592d0 0x7f3fbc14c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc420e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0c3940 con 0x7f3fbc06da40 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc420e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0c37c0 con 0x7f3fbc1592d0 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc420e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0c3ac0 con 0x7f3fbc0c6780 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc06da40 0x7f3fbc1de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc06da40 0x7f3fbc1de960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36104/0 (socket says 172.21.15.26:36104) 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 -- 172.21.15.26:0/2535461560 learned_addr learned my addr 172.21.15.26:0/2535461560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 -- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fbc0c6780 msgr2=0x7f3fbc14c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc17a9640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fbc0c6780 0x7f3fbc14c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc27ab640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fbc1592d0 0x7f3fbc14c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fbc0c6780 0x7f3fbc14c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 -- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fbc1592d0 msgr2=0x7f3fbc14c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fbc1592d0 0x7f3fbc14c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 -- 172.21.15.26:0/2535461560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0702a0 con 0x7f3fbc06da40 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc27ab640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fbc1592d0 0x7f3fbc14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc17a9640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fbc0c6780 0x7f3fbc14c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc1faa640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc06da40 0x7f3fbc1de960 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ba50 tx=0x7f3fb400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fba7fc640 1 -- 172.21.15.26:0/2535461560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4019070 con 0x7f3fbc06da40 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fba7fc640 1 -- 172.21.15.26:0/2535461560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb400cc20 con 0x7f3fbc06da40 2024-03-06T07:33:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1eee40 con 0x7f3fbc06da40 2024-03-06T07:33:08.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fba7fc640 1 -- 172.21.15.26:0/2535461560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4014680 con 0x7f3fbc06da40 2024-03-06T07:33:08.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.460+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc06ec70 con 0x7f3fbc06da40 2024-03-06T07:33:08.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.461+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc04f330 con 0x7f3fbc06da40 2024-03-06T07:33:08.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.462+0000 7f3fba7fc640 1 -- 172.21.15.26:0/2535461560 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fb401c070 con 0x7f3fbc06da40 2024-03-06T07:33:08.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.462+0000 7f3fba7fc640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3fa806b500 0x7f3fa806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.462+0000 7f3fba7fc640 1 -- 172.21.15.26:0/2535461560 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f3fb4089eb0 con 0x7f3fbc06da40 2024-03-06T07:33:08.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.462+0000 7f3fc17a9640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3fa806b500 0x7f3fa806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.463+0000 7f3fc17a9640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3fa806b500 0x7f3fa806d9c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0036000 tx=0x7f3fb0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.466+0000 7f3fba7fc640 1 -- 172.21.15.26:0/2535461560 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3fb40586d0 con 0x7f3fbc06da40 2024-03-06T07:33:08.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.644+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fbc073240 con 0x7f3fa806b500 2024-03-06T07:33:08.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.645+0000 7f3fba7fc640 1 -- 172.21.15.26:0/2535461560 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13519 (secure 0 0 0) 0x7f3fbc073240 con 0x7f3fa806b500 2024-03-06T07:33:08.647 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:08.647 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3fa806b500 msgr2=0x7f3fa806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3fa806b500 0x7f3fa806d9c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0036000 tx=0x7f3fb0008030 comp rx=0 tx=0).stop 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc06da40 msgr2=0x7f3fbc1de960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc06da40 0x7f3fbc1de960 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ba50 tx=0x7f3fb400bf10 comp rx=0 tx=0).stop 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 shutdown_connections 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f3fa806b500 0x7f3fa806d9c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fbc1592d0 0x7f3fbc14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fbc0c6780 0x7f3fbc14c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 --2- 172.21.15.26:0/2535461560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fbc06da40 0x7f3fbc1de960 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 >> 172.21.15.26:0/2535461560 conn(0x7f3fbc0aa630 msgr2=0x7f3fbc0c3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:08.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 shutdown_connections 2024-03-06T07:33:08.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.647+0000 7f3fc420e640 1 -- 172.21.15.26:0/2535461560 wait complete. 2024-03-06T07:33:08.659 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-03-06T07:33:07.211673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283520,"kb_used_data":21600,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754691200,"statfs":{"total":773094113280,"available":772803788800,"internally_reserved":0,"allocated":22118400,"data_stored":11827360,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-88,"num_read_kb":-94,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002107"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":538,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-03-06T07:33:03.303824+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:33:03.303824+0000","last_peered":"2024-03-06T07:33:03.303824+0000","last_clean":"2024-03-06T07:33:03.303824+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:33:03.303824+0000","last_undegraded":"2024-03-06T07:33:03.303824+0000","last_fullsized":"2024-03-06T07:33:03.303824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33936,"kb_used_data":2756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337904,"statfs":{"total":96636764160,"available":96602013696,"internally_reserved":0,"allocated":2822144,"data_stored":1924920,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33860,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337980,"statfs":{"total":96636764160,"available":96602091520,"internally_reserved":0,"allocated":2744320,"data_stored":1333360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33852,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337988,"statfs":{"total":96636764160,"available":96602099712,"internally_reserved":0,"allocated":2736128,"data_stored":1326514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2752512,"data_stored":1334938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38016,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333824,"statfs":{"total":96636764160,"available":96597835776,"internally_reserved":0,"allocated":2740224,"data_stored":1327184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2764800,"data_stored":1335632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38032,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333808,"statfs":{"total":96636764160,"available":96597819392,"internally_reserved":0,"allocated":2756608,"data_stored":1327862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38076,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333764,"statfs":{"total":96636764160,"available":96597774336,"internally_reserved":0,"allocated":2801664,"data_stored":1916950,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:08.660 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-06T07:33:08.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.760+0000 7fbeffa88640 1 Processor -- start 2024-03-06T07:33:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.760+0000 7fbeffa88640 1 -- start start 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbeffa88640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbef814f790 0x7fbef81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbeffa88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef80c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbeffa88640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef80c28d0 con 0x7fbef80c5c60 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbeffa88640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef80c2680 con 0x7fbef814f790 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbeffa88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80c2bb0 con 0x7fbef80c1aa0 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbefe025640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbef80c5c60 0x7fbef8155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52108/0 (socket says 172.21.15.26:52108) 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbefe025640 1 -- 172.21.15.26:0/3173722569 learned_addr learned my addr 172.21.15.26:0/3173722569 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbefd023640 1 --2- 172.21.15.26:0/3173722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef80c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.761+0000 7fbefd824640 1 --2- 172.21.15.26:0/3173722569 >> v2:172.21.15.117:3300/0 conn(0x7fbef814f790 0x7fbef81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefc822640 1 -- 172.21.15.26:0/3173722569 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667070179 0 0) 0x7fbef80c28d0 con 0x7fbef80c5c60 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefc822640 1 -- 172.21.15.26:0/3173722569 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee0003520 con 0x7fbef80c5c60 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefd023640 1 -- 172.21.15.26:0/3173722569 >> v2:172.21.15.117:3300/0 conn(0x7fbef814f790 msgr2=0x7fbef81529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefd023640 1 --2- 172.21.15.26:0/3173722569 >> v2:172.21.15.117:3300/0 conn(0x7fbef814f790 0x7fbef81529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefd023640 1 -- 172.21.15.26:0/3173722569 >> v1:172.21.15.26:6789/0 conn(0x7fbef80c5c60 legacy=0x7fbef8155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefc822640 1 -- 172.21.15.26:0/3173722569 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2658175168 0 0) 0x7fbee0003520 con 0x7fbef80c5c60 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefd023640 1 -- 172.21.15.26:0/3173722569 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef80c2e90 con 0x7fbef80c1aa0 2024-03-06T07:33:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.762+0000 7fbefd824640 1 --2- 172.21.15.26:0/3173722569 >> v2:172.21.15.117:3300/0 conn(0x7fbef814f790 0x7fbef81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbefd023640 1 --2- 172.21.15.26:0/3173722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef80c5720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0097a0 tx=0x7fbeec02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61da33e271a151f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbefc822640 1 -- 172.21.15.26:0/3173722569 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec037070 con 0x7fbef80c1aa0 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbefc822640 1 -- 172.21.15.26:0/3173722569 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec030510 con 0x7fbef80c1aa0 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbefc822640 1 -- 172.21.15.26:0/3173722569 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee80019f0 con 0x7fbef80c1aa0 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 -- 172.21.15.26:0/3173722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 msgr2=0x7fbef80c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 --2- 172.21.15.26:0/3173722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef80c5720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0097a0 tx=0x7fbeec02b790 comp rx=0 tx=0).stop 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 -- 172.21.15.26:0/3173722569 shutdown_connections 2024-03-06T07:33:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 --2- 172.21.15.26:0/3173722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef80c5720 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 --2- 172.21.15.26:0/3173722569 >> v2:172.21.15.117:3300/0 conn(0x7fbef814f790 0x7fbef81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 -- 172.21.15.26:0/3173722569 >> 172.21.15.26:0/3173722569 conn(0x7fbef80c6710 msgr2=0x7fbef80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 -- 172.21.15.26:0/3173722569 shutdown_connections 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.763+0000 7fbeffa88640 1 -- 172.21.15.26:0/3173722569 wait complete. 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.764+0000 7fbeffa88640 1 Processor -- start 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbeffa88640 1 -- start start 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbeffa88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef814a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbeffa88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80c5c60 0x7fbef8153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbeffa88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef814f790 0x7fbef8153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbeffa88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef806cc00 con 0x7fbef814f790 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbeffa88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80c0d60 con 0x7fbef80c5c60 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbeffa88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef806cd80 con 0x7fbef80c1aa0 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbefe025640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef814f790 0x7fbef8153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbefe025640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef814f790 0x7fbef8153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36108/0 (socket says 172.21.15.26:36108) 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbefe025640 1 -- 172.21.15.26:0/434176505 learned_addr learned my addr 172.21.15.26:0/434176505 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbefd023640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80c5c60 0x7fbef8153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.765+0000 7fbefd824640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef814a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbefe025640 1 -- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 msgr2=0x7fbef814a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbefe025640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef814a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbefe025640 1 -- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80c5c60 msgr2=0x7fbef8153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbefe025640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80c5c60 0x7fbef8153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbefe025640 1 -- 172.21.15.26:0/434176505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef80733b0 con 0x7fbef814f790 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbefd824640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef814a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbefe025640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef814f790 0x7fbef8153640 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fbee800cfc0 tx=0x7fbee800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbef67fc640 1 -- 172.21.15.26:0/434176505 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8015070 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbef67fc640 1 -- 172.21.15.26:0/434176505 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8011470 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef81e82e0 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbef67fc640 1 -- 172.21.15.26:0/434176505 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee80118a0 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.766+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef814a6f0 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.767+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef804f330 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.767+0000 7fbef67fc640 1 -- 172.21.15.26:0/434176505 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbee8018070 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.768+0000 7fbef67fc640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbee406b4d0 0x7fbee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.768+0000 7fbef67fc640 1 -- 172.21.15.26:0/434176505 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fbee808c7d0 con 0x7fbef814f790 2024-03-06T07:33:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.768+0000 7fbefd824640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbee406b4d0 0x7fbee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:08.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.769+0000 7fbefd824640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbee406b4d0 0x7fbee406d990 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009af0 tx=0x7fbef0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:08.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.772+0000 7fbef67fc640 1 -- 172.21.15.26:0/434176505 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbee805b100 con 0x7fbef814f790 2024-03-06T07:33:08.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.951+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbef8073160 con 0x7fbee406b4d0 2024-03-06T07:33:08.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.952+0000 7fbef67fc640 1 -- 172.21.15.26:0/434176505 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13519 (secure 0 0 0) 0x7fbef8073160 con 0x7fbee406b4d0 2024-03-06T07:33:08.953 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbee406b4d0 msgr2=0x7fbee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbee406b4d0 0x7fbee406d990 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009af0 tx=0x7fbef0008040 comp rx=0 tx=0).stop 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef814f790 msgr2=0x7fbef8153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef814f790 0x7fbef8153640 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fbee800cfc0 tx=0x7fbee800a430 comp rx=0 tx=0).stop 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 shutdown_connections 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbee406b4d0 0x7fbee406d990 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef814f790 0x7fbef8153640 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80c5c60 0x7fbef8153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 --2- 172.21.15.26:0/434176505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80c1aa0 0x7fbef814a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 >> 172.21.15.26:0/434176505 conn(0x7fbef80c6710 msgr2=0x7fbef80a7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 shutdown_connections 2024-03-06T07:33:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:08.954+0000 7fbeffa88640 1 -- 172.21.15.26:0/434176505 wait complete. 2024-03-06T07:33:08.966 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-03-06T07:33:07.211673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283520,"kb_used_data":21600,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754691200,"statfs":{"total":773094113280,"available":772803788800,"internally_reserved":0,"allocated":22118400,"data_stored":11827360,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-88,"num_read_kb":-94,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002107"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":538,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-03-06T07:33:03.303824+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:33:03.303824+0000","last_peered":"2024-03-06T07:33:03.303824+0000","last_clean":"2024-03-06T07:33:03.303824+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:33:03.303824+0000","last_undegraded":"2024-03-06T07:33:03.303824+0000","last_fullsized":"2024-03-06T07:33:03.303824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33936,"kb_used_data":2756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337904,"statfs":{"total":96636764160,"available":96602013696,"internally_reserved":0,"allocated":2822144,"data_stored":1924920,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33860,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337980,"statfs":{"total":96636764160,"available":96602091520,"internally_reserved":0,"allocated":2744320,"data_stored":1333360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33852,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337988,"statfs":{"total":96636764160,"available":96602099712,"internally_reserved":0,"allocated":2736128,"data_stored":1326514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2752512,"data_stored":1334938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38016,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333824,"statfs":{"total":96636764160,"available":96597835776,"internally_reserved":0,"allocated":2740224,"data_stored":1327184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2764800,"data_stored":1335632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38032,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333808,"statfs":{"total":96636764160,"available":96597819392,"internally_reserved":0,"allocated":2756608,"data_stored":1327862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38076,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333764,"statfs":{"total":96636764160,"available":96597774336,"internally_reserved":0,"allocated":2801664,"data_stored":1916950,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:08.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:09.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.066+0000 7f6df075c640 1 Processor -- start 2024-03-06T07:33:09.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.066+0000 7f6df075c640 1 -- start start 2024-03-06T07:33:09.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.066+0000 7f6df075c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de80c7380 0x7f6de80c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6df075c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de8072630 con 0x7f6de806da40 2024-03-06T07:33:09.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6df075c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de8072910 con 0x7f6de80c6780 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6df075c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de806de20 con 0x7f6de80c7380 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6dee4f8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6de806da40 0x7f6de8155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52118/0 (socket says 172.21.15.26:52118) 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6dee4f8640 1 -- 172.21.15.26:0/2419563018 learned_addr learned my addr 172.21.15.26:0/2419563018 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6deecf9640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de80c7380 0x7f6de80c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317388487 0 0) 0x7f6de8072630 con 0x7f6de806da40 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd4003520 con 0x7f6de806da40 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4048154113 0 0) 0x7f6dd4003520 con 0x7f6de806da40 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de80c7380 msgr2=0x7f6de80c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de80c7380 0x7f6de80c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 >> v1:172.21.15.117:6789/0 conn(0x7f6de80c6780 legacy=0x7f6de8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de8072bf0 con 0x7f6de806da40 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f6de4001670 con 0x7f6de806da40 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 >> v1:172.21.15.26:6789/0 conn(0x7f6de806da40 legacy=0x7f6de8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dd4004390 0x7f6dd4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40050b0 0x7f6dd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de806de20 con 0x7f6dd40050b0 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd400c010 con 0x7f6dd40039e0 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd400c210 con 0x7f6dd4004390 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6deecf9640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de80c7380 0x7f6de80c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.067+0000 7f6deecf9640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40050b0 0x7f6dd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6dedcf7640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dd4004390 0x7f6dd4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6dee4f8640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6deecf9640 1 -- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dd4004390 msgr2=0x7f6dd4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6deecf9640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dd4004390 0x7f6dd4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6deecf9640 1 -- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40039e0 msgr2=0x7f6dd4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6deecf9640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6deecf9640 1 -- 172.21.15.26:0/2419563018 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd4003520 con 0x7f6dd40050b0 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6dee4f8640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.068+0000 7f6deecf9640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40050b0 0x7f6dd400b9c0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f6de80c7760 tx=0x7f6dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c162d0bef765296b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0009d60 con 0x7f6dd40050b0 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de002cce0 con 0x7f6dd40050b0 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6ded4f6640 1 -- 172.21.15.26:0/2419563018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0003bb0 con 0x7f6dd40050b0 2024-03-06T07:33:09.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6df075c640 1 -- 172.21.15.26:0/2419563018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40050b0 msgr2=0x7f6dd400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6df075c640 1 --2- 172.21.15.26:0/2419563018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40050b0 0x7f6dd400b9c0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f6de80c7760 tx=0x7f6dd4005830 comp rx=0 tx=0).stop 2024-03-06T07:33:09.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6deecf9640 1 -- 172.21.15.26:0/2419563018 reap_dead start 2024-03-06T07:33:09.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6df075c640 1 -- 172.21.15.26:0/2419563018 shutdown_connections 2024-03-06T07:33:09.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6df075c640 1 -- 172.21.15.26:0/2419563018 >> 172.21.15.26:0/2419563018 conn(0x7f6de80aa630 msgr2=0x7f6de80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6df075c640 1 -- 172.21.15.26:0/2419563018 shutdown_connections 2024-03-06T07:33:09.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.069+0000 7f6df075c640 1 -- 172.21.15.26:0/2419563018 wait complete. 2024-03-06T07:33:09.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.070+0000 7f6df075c640 1 Processor -- start 2024-03-06T07:33:09.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.070+0000 7f6df075c640 1 -- start start 2024-03-06T07:33:09.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6df075c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40050b0 0x7f6de814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6df075c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6de8070280 0x7f6de814a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6df075c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de814b0a0 0x7f6de814ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6df075c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80c3280 con 0x7f6de8070280 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6df075c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de81594a0 con 0x7f6dd40050b0 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6df075c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80c3400 con 0x7f6de814b0a0 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6de8070280 0x7f6de814a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6de8070280 0x7f6de814a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36134/0 (socket says 172.21.15.26:36134) 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 -- 172.21.15.26:0/3120287626 learned_addr learned my addr 172.21.15.26:0/3120287626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6deecf9640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de814b0a0 0x7f6de814ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dee4f8640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40050b0 0x7f6de814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 -- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de814b0a0 msgr2=0x7f6de814ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de814b0a0 0x7f6de814ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 -- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40050b0 msgr2=0x7f6de814df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40050b0 0x7f6de814df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.071+0000 7f6dedcf7640 1 -- 172.21.15.26:0/3120287626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de81eaf80 con 0x7f6de8070280 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.072+0000 7f6dedcf7640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6de8070280 0x7f6de814a600 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800f8e0 tx=0x7f6dd800fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.072+0000 7f6ddeffd640 1 -- 172.21.15.26:0/3120287626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80052e0 con 0x7f6de8070280 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.072+0000 7f6ddeffd640 1 -- 172.21.15.26:0/3120287626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd8018070 con 0x7f6de8070280 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.072+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de8155f00 con 0x7f6de8070280 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.072+0000 7f6ddeffd640 1 -- 172.21.15.26:0/3120287626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800d910 con 0x7f6de8070280 2024-03-06T07:33:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.072+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de81eb240 con 0x7f6de8070280 2024-03-06T07:33:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.073+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de804f330 con 0x7f6de8070280 2024-03-06T07:33:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.073+0000 7f6ddeffd640 1 -- 172.21.15.26:0/3120287626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6dd800dab0 con 0x7f6de8070280 2024-03-06T07:33:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.074+0000 7f6ddeffd640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6dcc06b500 0x7f6dcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.074+0000 7f6ddeffd640 1 -- 172.21.15.26:0/3120287626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f6dd808a060 con 0x7f6de8070280 2024-03-06T07:33:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.074+0000 7f6dee4f8640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6dcc06b500 0x7f6dcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.074+0000 7f6dee4f8640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6dcc06b500 0x7f6dcc06d9c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005b10 tx=0x7f6de4007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.078+0000 7f6ddeffd640 1 -- 172.21.15.26:0/3120287626 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6dd80588f0 con 0x7f6de8070280 2024-03-06T07:33:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.259+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6de8073240 con 0x7f6de8070280 2024-03-06T07:33:09.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.260+0000 7f6ddeffd640 1 -- 172.21.15.26:0/3120287626 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+13707 (secure 0 0 0) 0x7f6dd805c990 con 0x7f6de8070280 2024-03-06T07:33:09.262 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:09.262 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":218,"fsid":"4c08c062-261c-4d4a-a3b3-e195823d80fd","created":"2024-03-06T07:19:44.817397+0000","modified":"2024-03-06T07:33:06.322488+0000","last_up_change":"2024-03-06T07:19:55.000525+0000","last_in_change":"2024-03-06T07:19:46.982231+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:55.073878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f435f5f5-06f4-4a6c-aade-2e030c8e66cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6801","nonce":1236614035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6803","nonce":1236614035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6810","nonce":1236614035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1236614035},{"type":"v1","addr":"172.21.15.26:6806","nonce":1236614035}]},"public_addr":"172.21.15.26:6801/1236614035","cluster_addr":"172.21.15.26:6803/1236614035","heartbeat_back_addr":"172.21.15.26:6810/1236614035","heartbeat_front_addr":"172.21.15.26:6806/1236614035","state":["exists","up"]},{"osd":1,"uuid":"f298b038-2b12-4ed8-8fc3-3bce46a1e6c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6805","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6807","nonce":2298328200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6811","nonce":2298328200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6815","nonce":2298328200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2298328200},{"type":"v1","addr":"172.21.15.26:6813","nonce":2298328200}]},"public_addr":"172.21.15.26:6807/2298328200","cluster_addr":"172.21.15.26:6811/2298328200","heartbeat_back_addr":"172.21.15.26:6815/2298328200","heartbeat_front_addr":"172.21.15.26:6813/2298328200","state":["exists","up"]},{"osd":2,"uuid":"d26cf89c-159b-4198-a291-5695dc6809a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6825","nonce":3726964600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6827","nonce":3726964600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6831","nonce":3726964600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3726964600},{"type":"v1","addr":"172.21.15.26:6829","nonce":3726964600}]},"public_addr":"172.21.15.26:6825/3726964600","cluster_addr":"172.21.15.26:6827/3726964600","heartbeat_back_addr":"172.21.15.26:6831/3726964600","heartbeat_front_addr":"172.21.15.26:6829/3726964600","state":["exists","up"]},{"osd":3,"uuid":"364b50e5-d473-48ed-ab52-4dcce8fe3471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6817","nonce":1166415256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6819","nonce":1166415256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6823","nonce":1166415256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1166415256},{"type":"v1","addr":"172.21.15.26:6821","nonce":1166415256}]},"public_addr":"172.21.15.26:6817/1166415256","cluster_addr":"172.21.15.26:6819/1166415256","heartbeat_back_addr":"172.21.15.26:6823/1166415256","heartbeat_front_addr":"172.21.15.26:6821/1166415256","state":["exists","up"]},{"osd":4,"uuid":"72a8daac-45a0-4413-a095-8fed3c23b00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6801","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6802","nonce":1465648529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6806","nonce":1465648529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6814","nonce":1465648529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1465648529},{"type":"v1","addr":"172.21.15.117:6810","nonce":1465648529}]},"public_addr":"172.21.15.117:6802/1465648529","cluster_addr":"172.21.15.117:6806/1465648529","heartbeat_back_addr":"172.21.15.117:6814/1465648529","heartbeat_front_addr":"172.21.15.117:6810/1465648529","state":["exists","up"]},{"osd":5,"uuid":"63672daf-3c97-4b49-851c-428e61b9fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6803","nonce":3827357056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6807","nonce":3827357056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6815","nonce":3827357056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":3827357056},{"type":"v1","addr":"172.21.15.117:6811","nonce":3827357056}]},"public_addr":"172.21.15.117:6803/3827357056","cluster_addr":"172.21.15.117:6807/3827357056","heartbeat_back_addr":"172.21.15.117:6815/3827357056","heartbeat_front_addr":"172.21.15.117:6811/3827357056","state":["exists","up"]},{"osd":6,"uuid":"ba4402fa-8b2c-4969-84ca-50316c20363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6817","nonce":1228719702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6821","nonce":1228719702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6831","nonce":1228719702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1228719702},{"type":"v1","addr":"172.21.15.117:6826","nonce":1228719702}]},"public_addr":"172.21.15.117:6817/1228719702","cluster_addr":"172.21.15.117:6821/1228719702","heartbeat_back_addr":"172.21.15.117:6831/1228719702","heartbeat_front_addr":"172.21.15.117:6826/1228719702","state":["exists","up"]},{"osd":7,"uuid":"2f61ed3c-76d7-4537-9573-401d99fec1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6820","nonce":4144741400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6823","nonce":4144741400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6829","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6830","nonce":4144741400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6825","nonce":4144741400},{"type":"v1","addr":"172.21.15.117:6827","nonce":4144741400}]},"public_addr":"172.21.15.117:6820/4144741400","cluster_addr":"172.21.15.117:6823/4144741400","heartbeat_back_addr":"172.21.15.117:6830/4144741400","heartbeat_front_addr":"172.21.15.117:6827/4144741400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.098657+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.085883+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.197738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.164274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.067324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.173769+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.118963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:52.124214+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/741758816":"2024-03-07T07:33:03.299384+0000","172.21.15.26:6832/741758816":"2024-03-07T07:33:03.299384+0000","172.21.15.117:6836/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.117:6834/4173532851":"2024-03-07T07:30:29.675435+0000","172.21.15.26:6835/4219086401":"2024-03-07T07:33:03.299402+0000","172.21.15.117:6837/3331811951":"2024-03-07T07:29:54.670724+0000","172.21.15.26:6833/4219086401":"2024-03-07T07:33:03.299402+0000","172.21.15.117:6835/3331811951":"2024-03-07T07:29:54.670724+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6dcc06b500 msgr2=0x7f6dcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6dcc06b500 0x7f6dcc06d9c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005b10 tx=0x7f6de4007040 comp rx=0 tx=0).stop 2024-03-06T07:33:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6de8070280 msgr2=0x7f6de814a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6de8070280 0x7f6de814a600 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800f8e0 tx=0x7f6dd800fda0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 shutdown_connections 2024-03-06T07:33:09.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6dcc06b500 0x7f6dcc06d9c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de814b0a0 0x7f6de814ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6de8070280 0x7f6de814a600 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 --2- 172.21.15.26:0/3120287626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd40050b0 0x7f6de814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 >> 172.21.15.26:0/3120287626 conn(0x7f6de80aa630 msgr2=0x7f6de806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 shutdown_connections 2024-03-06T07:33:09.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:09.262+0000 7f6df075c640 1 -- 172.21.15.26:0/3120287626 wait complete. 2024-03-06T07:33:10.276 INFO:tasks.ceph:Scrubbing osd.0 2024-03-06T07:33:10.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.393+0000 7f6ebf93e640 1 Processor -- start 2024-03-06T07:33:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.393+0000 7f6ebf93e640 1 -- start start 2024-03-06T07:33:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebf93e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6eb8152860 0x7f6eb80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebf93e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb80c8ea0 con 0x7f6eb8159230 2024-03-06T07:33:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebf93e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb80c7a10 con 0x7f6eb80c7470 2024-03-06T07:33:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebf93e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb8158da0 con 0x7f6eb8152860 2024-03-06T07:33:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebd6da640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6eb8152860 0x7f6eb80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebdedb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6eb8159230 0x7f6eb80c53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52126/0 (socket says 172.21.15.26:52126) 2024-03-06T07:33:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebdedb640 1 -- 172.21.15.26:0/1506559801 learned_addr learned my addr 172.21.15.26:0/1506559801 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebd6da640 1 -- 172.21.15.26:0/1506559801 >> v1:172.21.15.117:6789/0 conn(0x7f6eb80c7470 legacy=0x7f6eb8155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebd6da640 1 -- 172.21.15.26:0/1506559801 >> v1:172.21.15.26:6789/0 conn(0x7f6eb8159230 legacy=0x7f6eb80c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6ebd6da640 1 -- 172.21.15.26:0/1506559801 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80c7cf0 con 0x7f6eb8152860 2024-03-06T07:33:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.394+0000 7f6eaffff640 1 -- 172.21.15.26:0/1506559801 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793871902 0 0) 0x7f6eb80c8ea0 con 0x7f6eb8159230 2024-03-06T07:33:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.395+0000 7f6ebd6da640 1 --2- 172.21.15.26:0/1506559801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8152860 0x7f6eb80c8960 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8158f20 tx=0x7f6eb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8eeeb0a5775a2d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.395+0000 7f6eaffff640 1 -- 172.21.15.26:0/1506559801 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4035020 con 0x7f6eb8152860 2024-03-06T07:33:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.395+0000 7f6eaffff640 1 -- 172.21.15.26:0/1506559801 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4031830 con 0x7f6eb8152860 2024-03-06T07:33:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.395+0000 7f6eaffff640 1 -- 172.21.15.26:0/1506559801 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4031c00 con 0x7f6eb8152860 2024-03-06T07:33:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.395+0000 7f6ebf93e640 1 -- 172.21.15.26:0/1506559801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8152860 msgr2=0x7f6eb80c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.396+0000 7f6ebf93e640 1 --2- 172.21.15.26:0/1506559801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8152860 0x7f6eb80c8960 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8158f20 tx=0x7f6eb402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.396+0000 7f6ebf93e640 1 -- 172.21.15.26:0/1506559801 shutdown_connections 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.396+0000 7f6ebf93e640 1 --2- 172.21.15.26:0/1506559801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8152860 0x7f6eb80c8960 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.396+0000 7f6ebf93e640 1 -- 172.21.15.26:0/1506559801 >> 172.21.15.26:0/1506559801 conn(0x7f6eb80ac630 msgr2=0x7f6eb80aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.396+0000 7f6ebf93e640 1 -- 172.21.15.26:0/1506559801 shutdown_connections 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.396+0000 7f6ebf93e640 1 -- 172.21.15.26:0/1506559801 wait complete. 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 Processor -- start 2024-03-06T07:33:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 -- start start 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb80c7470 0x7f6eb80c0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb8152860 0x7f6eb80c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8159230 0x7f6eb81ee390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80c5af0 con 0x7f6eb8159230 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb8152c40 con 0x7f6eb8152860 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.397+0000 7f6ebf93e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80c5c70 con 0x7f6eb80c7470 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8159230 0x7f6eb81ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8159230 0x7f6eb81ee390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36150/0 (socket says 172.21.15.26:36150) 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 -- 172.21.15.26:0/2710587437 learned_addr learned my addr 172.21.15.26:0/2710587437 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebd6da640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb80c7470 0x7f6eb80c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 -- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb80c7470 msgr2=0x7f6eb80c0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb80c7470 0x7f6eb80c0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebced9640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb8152860 0x7f6eb80c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 -- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb8152860 msgr2=0x7f6eb80c0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb8152860 0x7f6eb80c0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 -- 172.21.15.26:0/2710587437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb81ef2b0 con 0x7f6eb8159230 2024-03-06T07:33:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebd6da640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb80c7470 0x7f6eb80c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebced9640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb8152860 0x7f6eb80c0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebdedb640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8159230 0x7f6eb81ee390 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c730 tx=0x7f6eb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0018070 con 0x7f6eb8159230 2024-03-06T07:33:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb000ce40 con 0x7f6eb8159230 2024-03-06T07:33:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000a860 con 0x7f6eb8159230 2024-03-06T07:33:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.398+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb80c1660 con 0x7f6eb8159230 2024-03-06T07:33:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.399+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb814bea0 con 0x7f6eb8159230 2024-03-06T07:33:10.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.399+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6eb804f330 con 0x7f6eb8159230 2024-03-06T07:33:10.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.400+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eb000aa00 con 0x7f6eb8159230 2024-03-06T07:33:10.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.400+0000 7f6eadffb640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.400+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f6eb0089780 con 0x7f6eb8159230 2024-03-06T07:33:10.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.401+0000 7f6eadffb640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f6e9c071b50 0x7f6e9c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.401+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 --> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e9c074660 con 0x7f6e9c071b50 2024-03-06T07:33:10.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.401+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb0090050 con 0x7f6eb8159230 2024-03-06T07:33:10.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.401+0000 7f6ebd6da640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.401+0000 7f6ebced9640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f6e9c071b50 0x7f6e9c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.401+0000 7f6ebced9640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f6e9c071b50 0x7f6e9c073f90 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.401+0000 7f6ebd6da640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4036000 tx=0x7f6eb4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.402+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== osd.0 v2:172.21.15.26:6800/1236614035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6e9c074660 con 0x7f6e9c071b50 2024-03-06T07:33:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.415+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 --> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6eb80ae390 con 0x7f6e9c071b50 2024-03-06T07:33:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.415+0000 7f6eadffb640 1 -- 172.21.15.26:0/2710587437 <== osd.0 v2:172.21.15.26:6800/1236614035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6eb80ae390 con 0x7f6e9c071b50 2024-03-06T07:33:10.417 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:10.417 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.415+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f6e9c071b50 msgr2=0x7f6e9c073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.415+0000 7f6ebf93e640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:6800/1236614035,v1:172.21.15.26:6801/1236614035] conn(0x7f6e9c071b50 0x7f6e9c073f90 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e9c06b4d0 msgr2=0x7f6e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4036000 tx=0x7f6eb4008030 comp rx=0 tx=0).stop 2024-03-06T07:33:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8159230 msgr2=0x7f6eb81ee390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 --2- 172.21.15.26:0/2710587437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6eb8159230 0x7f6eb81ee390 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c730 tx=0x7f6eb000cbf0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebdedb640 1 -- 172.21.15.26:0/2710587437 reap_dead start 2024-03-06T07:33:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 shutdown_connections 2024-03-06T07:33:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 >> 172.21.15.26:0/2710587437 conn(0x7f6eb80ac630 msgr2=0x7f6eb80c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 shutdown_connections 2024-03-06T07:33:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.416+0000 7f6ebf93e640 1 -- 172.21.15.26:0/2710587437 wait complete. 2024-03-06T07:33:10.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-06T07:33:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.529+0000 7f2f85c4e640 1 Processor -- start 2024-03-06T07:33:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.529+0000 7f2f85c4e640 1 -- start start 2024-03-06T07:33:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f85c4e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f800c8950 0x7f2f80155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f85c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 0x7f2f800c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f85c4e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800c8e30 con 0x7f2f80152900 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f85c4e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f80152ce0 con 0x7f2f800c8950 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f85c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c9110 con 0x7f2f800c64f0 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f7f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2f80152900 0x7f2f800bf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52138/0 (socket says 172.21.15.26:52138) 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f7f7fe640 1 -- 172.21.15.26:0/3686099957 learned_addr learned my addr 172.21.15.26:0/3686099957 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 0x7f2f800c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154348575 0 0) 0x7f2f800c8e30 con 0x7f2f80152900 2024-03-06T07:33:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f7effd640 1 --2- 172.21.15.26:0/3686099957 >> v2:172.21.15.117:3300/0 conn(0x7f2f800c8950 0x7f2f80155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.530+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f68003520 con 0x7f2f80152900 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4044997574 0 0) 0x7f2f68003520 con 0x7f2f80152900 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 msgr2=0x7f2f800c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 0x7f2f800c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 >> v2:172.21.15.117:3300/0 conn(0x7f2f800c8950 msgr2=0x7f2f80155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 --2- 172.21.15.26:0/3686099957 >> v2:172.21.15.117:3300/0 conn(0x7f2f800c8950 0x7f2f80155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f800c93f0 con 0x7f2f80152900 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f2f70001670 con 0x7f2f80152900 2024-03-06T07:33:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 >> v1:172.21.15.26:6789/0 conn(0x7f2f80152900 legacy=0x7f2f800bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 0x7f2f680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f68003f30 0x7f2f68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80152ce0 con 0x7f2f68004d40 2024-03-06T07:33:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c9110 con 0x7f2f68003f30 2024-03-06T07:33:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6800baa0 con 0x7f2f800c64f0 2024-03-06T07:33:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7effd640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f68003f30 0x7f2f68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7f7fe640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 0x7f2f680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7ffff640 1 -- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 msgr2=0x7f2f680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 0x7f2f680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7ffff640 1 -- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f68003f30 msgr2=0x7f2f68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f68003f30 0x7f2f68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7ffff640 1 -- 172.21.15.26:0/3686099957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68003520 con 0x7f2f68004d40 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.531+0000 7f2f7f7fe640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f800c64f0 0x7f2f680039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.532+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009fc0 tx=0x7f2f68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb185c4d6c840ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.532+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78039070 con 0x7f2f68004d40 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.532+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7802cc20 con 0x7f2f68004d40 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.532+0000 7f2f7e7fc640 1 -- 172.21.15.26:0/3686099957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78009c10 con 0x7f2f68004d40 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.533+0000 7f2f85c4e640 1 -- 172.21.15.26:0/3686099957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f68004d40 msgr2=0x7f2f680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.533+0000 7f2f85c4e640 1 --2- 172.21.15.26:0/3686099957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009fc0 tx=0x7f2f68005830 comp rx=0 tx=0).stop 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.533+0000 7f2f7ffff640 1 -- 172.21.15.26:0/3686099957 reap_dead start 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.533+0000 7f2f85c4e640 1 -- 172.21.15.26:0/3686099957 shutdown_connections 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.533+0000 7f2f85c4e640 1 -- 172.21.15.26:0/3686099957 >> 172.21.15.26:0/3686099957 conn(0x7f2f800ac860 msgr2=0x7f2f800acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.533+0000 7f2f85c4e640 1 -- 172.21.15.26:0/3686099957 shutdown_connections 2024-03-06T07:33:10.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.533+0000 7f2f85c4e640 1 -- 172.21.15.26:0/3686099957 wait complete. 2024-03-06T07:33:10.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.534+0000 7f2f85c4e640 1 Processor -- start 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.534+0000 7f2f85c4e640 1 -- start start 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8014ae90 0x7f2f801ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f800c4510 0x7f2f801eed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f800b09b0 0x7f2f80145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c0180 con 0x7f2f800b09b0 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c0000 con 0x7f2f800c4510 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c55f0 con 0x7f2f8014ae90 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f800b09b0 0x7f2f80145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f800b09b0 0x7f2f80145190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36162/0 (socket says 172.21.15.26:36162) 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 -- 172.21.15.26:0/158073299 learned_addr learned my addr 172.21.15.26:0/158073299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7effd640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f800c4510 0x7f2f801eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7f7fe640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8014ae90 0x7f2f801ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 -- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8014ae90 msgr2=0x7f2f801ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8014ae90 0x7f2f801ee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 -- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f800c4510 msgr2=0x7f2f801eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f800c4510 0x7f2f801eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 -- 172.21.15.26:0/158073299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f800c2560 con 0x7f2f800b09b0 2024-03-06T07:33:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7effd640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f800c4510 0x7f2f801eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f7ffff640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f800b09b0 0x7f2f80145190 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f2f78002230 tx=0x7f2f78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f67fff640 1 -- 172.21.15.26:0/158073299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78039030 con 0x7f2f800b09b0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f67fff640 1 -- 172.21.15.26:0/158073299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f78043070 con 0x7f2f800b09b0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f800c1cc0 con 0x7f2f800b09b0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f67fff640 1 -- 172.21.15.26:0/158073299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78009c10 con 0x7f2f800b09b0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.535+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f800c2280 con 0x7f2f800b09b0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.536+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8004f330 con 0x7f2f800b09b0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.537+0000 7f2f67fff640 1 -- 172.21.15.26:0/158073299 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f78005d10 con 0x7f2f800b09b0 2024-03-06T07:33:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.537+0000 7f2f67fff640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f6c06b4d0 0x7f2f6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.537+0000 7f2f67fff640 1 -- 172.21.15.26:0/158073299 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f2f780ac360 con 0x7f2f800b09b0 2024-03-06T07:33:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.537+0000 7f2f7f7fe640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f6c06b4d0 0x7f2f6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.538+0000 7f2f7f7fe640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f6c06b4d0 0x7f2f6c06d990 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2f70005bd0 tx=0x7f2f70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.542+0000 7f2f67fff640 1 -- 172.21.15.26:0/158073299 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f7807b250 con 0x7f2f800b09b0 2024-03-06T07:33:10.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.722+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f800ca690 con 0x7f2f6c06b4d0 2024-03-06T07:33:10.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.722+0000 7f2f67fff640 1 -- 172.21.15.26:0/158073299 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2f800ca690 con 0x7f2f6c06b4d0 2024-03-06T07:33:10.724 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 0 to deep-scrub 2024-03-06T07:33:10.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f6c06b4d0 msgr2=0x7f2f6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f6c06b4d0 0x7f2f6c06d990 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2f70005bd0 tx=0x7f2f70009040 comp rx=0 tx=0).stop 2024-03-06T07:33:10.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f800b09b0 msgr2=0x7f2f80145190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f800b09b0 0x7f2f80145190 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f2f78002230 tx=0x7f2f78009750 comp rx=0 tx=0).stop 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 shutdown_connections 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f2f6c06b4d0 0x7f2f6c06d990 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f800b09b0 0x7f2f80145190 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f800c4510 0x7f2f801eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 --2- 172.21.15.26:0/158073299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8014ae90 0x7f2f801ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 >> 172.21.15.26:0/158073299 conn(0x7f2f800ac860 msgr2=0x7f2f800c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 shutdown_connections 2024-03-06T07:33:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.725+0000 7f2f85c4e640 1 -- 172.21.15.26:0/158073299 wait complete. 2024-03-06T07:33:10.737 INFO:tasks.ceph:Scrubbing osd.1 2024-03-06T07:33:10.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.841+0000 7f242e0c3640 1 Processor -- start 2024-03-06T07:33:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.841+0000 7f242e0c3640 1 -- start start 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f242e0c3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24280c3c20 0x7f24280c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f242e0c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f242806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f242e0c3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2428152e90 con 0x7f242806f8d0 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f242e0c3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2428153480 con 0x7f24280c3c20 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f242e0c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242806fd40 con 0x7f2428153910 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f2427fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f242806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f24277fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24280c3c20 0x7f24280c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f24277fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24280c3c20 0x7f24280c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:49412/0 (socket says 172.21.15.26:49412) 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f24277fe640 1 -- 172.21.15.26:0/2713082806 learned_addr learned my addr 172.21.15.26:0/2713082806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f2427fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f242806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.26:58370/0 (socket says 172.21.15.26:58370) 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f2426ffd640 1 --1- 172.21.15.26:0/2713082806 >> v1:172.21.15.117:6789/0 conn(0x7f242806f8d0 0x7f2428156480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:39628/0 (socket says 172.21.15.26:39628) 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f24277fe640 1 -- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 msgr2=0x7f242806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f24277fe640 1 --2- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f242806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f24277fe640 1 -- 172.21.15.26:0/2713082806 >> v1:172.21.15.117:6789/0 conn(0x7f242806f8d0 legacy=0x7f2428156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f24277fe640 1 -- 172.21.15.26:0/2713082806 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428153da0 con 0x7f24280c3c20 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.842+0000 7f2427fff640 1 --2- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f242806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.843+0000 7f24277fe640 1 --2- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 0x7f24280c6930 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2418009870 tx=0x7f241802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9c2473a1c9f58bd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.843+0000 7f24267fc640 1 -- 172.21.15.26:0/2713082806 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418034070 con 0x7f24280c3c20 2024-03-06T07:33:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.843+0000 7f24267fc640 1 -- 172.21.15.26:0/2713082806 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418030840 con 0x7f24280c3c20 2024-03-06T07:33:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.843+0000 7f24267fc640 1 -- 172.21.15.26:0/2713082806 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418030c10 con 0x7f24280c3c20 2024-03-06T07:33:10.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 -- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 msgr2=0x7f24280c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 --2- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 0x7f24280c6930 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2418009870 tx=0x7f241802b980 comp rx=0 tx=0).stop 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 -- 172.21.15.26:0/2713082806 shutdown_connections 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 --2- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f242806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 --2- 172.21.15.26:0/2713082806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 0x7f24280c6930 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 -- 172.21.15.26:0/2713082806 >> 172.21.15.26:0/2713082806 conn(0x7f24280c4300 msgr2=0x7f24280c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 -- 172.21.15.26:0/2713082806 shutdown_connections 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.844+0000 7f242e0c3640 1 -- 172.21.15.26:0/2713082806 wait complete. 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.845+0000 7f242e0c3640 1 Processor -- start 2024-03-06T07:33:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.845+0000 7f242e0c3640 1 -- start start 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f242e0c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242806f8d0 0x7f24281dee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f242e0c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 0x7f24280abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f242e0c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f24280ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f242e0c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428153110 con 0x7f242806f8d0 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f242e0c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c71f0 con 0x7f24280c3c20 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f242e0c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242806b500 con 0x7f2428153910 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242806f8d0 0x7f24281dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242806f8d0 0x7f24281dee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36178/0 (socket says 172.21.15.26:36178) 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 -- 172.21.15.26:0/458233517 learned_addr learned my addr 172.21.15.26:0/458233517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f2427fff640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f24280ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f2426ffd640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 0x7f24280abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 -- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 msgr2=0x7f24280ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2428153910 0x7f24280ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 -- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 msgr2=0x7f24280abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24280c3c20 0x7f24280abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 -- 172.21.15.26:0/458233517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24281dc740 con 0x7f242806f8d0 2024-03-06T07:33:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.846+0000 7f24277fe640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242806f8d0 0x7f24281dee00 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f241802b9c0 tx=0x7f2418004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.847+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418048070 con 0x7f242806f8d0 2024-03-06T07:33:10.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.847+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418030640 con 0x7f242806f8d0 2024-03-06T07:33:10.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.847+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24281dca20 con 0x7f242806f8d0 2024-03-06T07:33:10.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.847+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418034040 con 0x7f242806f8d0 2024-03-06T07:33:10.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.847+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280ab770 con 0x7f242806f8d0 2024-03-06T07:33:10.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.848+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24281dc4f0 con 0x7f242806f8d0 2024-03-06T07:33:10.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.848+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f241804b070 con 0x7f242806f8d0 2024-03-06T07:33:10.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f240ffff640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f241406b500 0x7f241406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f24180ac040 con 0x7f242806f8d0 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f2426ffd640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f241406b500 0x7f241406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f240ffff640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f2414071b80 0x7f2414073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 --> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2418031ec0 con 0x7f2414071b80 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24180b3050 con 0x7f242806f8d0 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f2427fff640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f2414071b80 0x7f2414073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f2427fff640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f2414071b80 0x7f2414073fc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.849+0000 7f2426ffd640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f241406b500 0x7f241406d9c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f242814cb70 tx=0x7f241c007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.850+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== osd.1 v2:172.21.15.26:6805/2298328200 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2418031ec0 con 0x7f2414071b80 2024-03-06T07:33:10.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 --> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2428067ad0 con 0x7f2414071b80 2024-03-06T07:33:10.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f240ffff640 1 -- 172.21.15.26:0/458233517 <== osd.1 v2:172.21.15.26:6805/2298328200 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2428067ad0 con 0x7f2414071b80 2024-03-06T07:33:10.865 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:10.865 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f2414071b80 msgr2=0x7f2414073fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f242e0c3640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:6805/2298328200,v1:172.21.15.26:6807/2298328200] conn(0x7f2414071b80 0x7f2414073fc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f241406b500 msgr2=0x7f241406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f242e0c3640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f241406b500 0x7f241406d9c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f242814cb70 tx=0x7f241c007e80 comp rx=0 tx=0).stop 2024-03-06T07:33:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242806f8d0 msgr2=0x7f24281dee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f242e0c3640 1 --2- 172.21.15.26:0/458233517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242806f8d0 0x7f24281dee00 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f241802b9c0 tx=0x7f2418004730 comp rx=0 tx=0).stop 2024-03-06T07:33:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.863+0000 7f2427fff640 1 -- 172.21.15.26:0/458233517 reap_dead start 2024-03-06T07:33:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.864+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 shutdown_connections 2024-03-06T07:33:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.864+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 >> 172.21.15.26:0/458233517 conn(0x7f24280c4300 msgr2=0x7f24280a7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.864+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 shutdown_connections 2024-03-06T07:33:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.864+0000 7f242e0c3640 1 -- 172.21.15.26:0/458233517 wait complete. 2024-03-06T07:33:10.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-06T07:33:10.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.976+0000 7f6c2c1e4640 1 Processor -- start 2024-03-06T07:33:10.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.976+0000 7f6c2c1e4640 1 -- start start 2024-03-06T07:33:10.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2c1e4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240c6ab0 0x7f6c241529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2c1e4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c241561a0 0x7f6c240a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2c1e4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c240a9990 con 0x7f6c240aa430 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2c1e4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c24155d10 con 0x7f6c241561a0 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2c1e4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241556c0 con 0x7f6c240c6ab0 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2977f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c241561a0 0x7f6c240a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2a781640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c240aa430 0x7f6c2414f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52154/0 (socket says 172.21.15.26:52154) 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2a781640 1 -- 172.21.15.26:0/1198106718 learned_addr learned my addr 172.21.15.26:0/1198106718 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c29f80640 1 --2- 172.21.15.26:0/1198106718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240c6ab0 0x7f6c241529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2977f640 1 -- 172.21.15.26:0/1198106718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240c6ab0 msgr2=0x7f6c241529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2977f640 1 --2- 172.21.15.26:0/1198106718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240c6ab0 0x7f6c241529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2977f640 1 -- 172.21.15.26:0/1198106718 >> v1:172.21.15.26:6789/0 conn(0x7f6c240aa430 legacy=0x7f6c2414f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c28f7e640 1 -- 172.21.15.26:0/1198106718 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709724892 0 0) 0x7f6c240a9990 con 0x7f6c240aa430 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.977+0000 7f6c2977f640 1 -- 172.21.15.26:0/1198106718 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c241559a0 con 0x7f6c241561a0 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.978+0000 7f6c2977f640 1 --2- 172.21.15.26:0/1198106718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c240a9ef0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f6c140097a0 tx=0x7f6c1402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705cdf8f8140c97b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.978+0000 7f6c28f7e640 1 -- 172.21.15.26:0/1198106718 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14038070 con 0x7f6c241561a0 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.978+0000 7f6c28f7e640 1 -- 172.21.15.26:0/1198106718 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c14031700 con 0x7f6c241561a0 2024-03-06T07:33:10.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.978+0000 7f6c28f7e640 1 -- 172.21.15.26:0/1198106718 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14031ad0 con 0x7f6c241561a0 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/1198106718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 msgr2=0x7f6c240a9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/1198106718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c240a9ef0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f6c140097a0 tx=0x7f6c1402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/1198106718 shutdown_connections 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/1198106718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c240a9ef0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/1198106718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240c6ab0 0x7f6c241529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/1198106718 >> 172.21.15.26:0/1198106718 conn(0x7f6c2406b1a0 msgr2=0x7f6c2406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/1198106718 shutdown_connections 2024-03-06T07:33:10.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.979+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/1198106718 wait complete. 2024-03-06T07:33:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.980+0000 7f6c2c1e4640 1 Processor -- start 2024-03-06T07:33:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.980+0000 7f6c2c1e4640 1 -- start start 2024-03-06T07:33:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240aa430 0x7f6c2414bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240c6ab0 0x7f6c2414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c24144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2414f820 con 0x7f6c241561a0 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240c2a90 con 0x7f6c240c6ab0 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241499a0 con 0x7f6c240aa430 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c24144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c24144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36186/0 (socket says 172.21.15.26:36186) 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 -- 172.21.15.26:0/359087561 learned_addr learned my addr 172.21.15.26:0/359087561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 -- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240aa430 msgr2=0x7f6c2414bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240aa430 0x7f6c2414bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 -- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240c6ab0 msgr2=0x7f6c2414a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240c6ab0 0x7f6c2414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 -- 172.21.15.26:0/359087561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24154c40 con 0x7f6c241561a0 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2a781640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c24144860 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c700 tx=0x7f6c2000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c1a7fc640 1 -- 172.21.15.26:0/359087561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20018070 con 0x7f6c241561a0 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c1a7fc640 1 -- 172.21.15.26:0/359087561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000ce10 con 0x7f6c241561a0 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c241496c0 con 0x7f6c241561a0 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c1a7fc640 1 -- 172.21.15.26:0/359087561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000a880 con 0x7f6c241561a0 2024-03-06T07:33:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.981+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2414c460 con 0x7f6c241561a0 2024-03-06T07:33:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.982+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c240702d0 con 0x7f6c241561a0 2024-03-06T07:33:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.983+0000 7f6c1a7fc640 1 -- 172.21.15.26:0/359087561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c2001b070 con 0x7f6c241561a0 2024-03-06T07:33:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.983+0000 7f6c1a7fc640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c0806b4d0 0x7f6c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.983+0000 7f6c1a7fc640 1 -- 172.21.15.26:0/359087561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f6c20089610 con 0x7f6c241561a0 2024-03-06T07:33:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.983+0000 7f6c29f80640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c0806b4d0 0x7f6c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.984+0000 7f6c29f80640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c0806b4d0 0x7f6c0806d990 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c006fd0 tx=0x7f6c1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:10.987+0000 7f6c1a7fc640 1 -- 172.21.15.26:0/359087561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c240702d0 con 0x7f6c241561a0 2024-03-06T07:33:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.166+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c2404f330 con 0x7f6c0806b4d0 2024-03-06T07:33:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.167+0000 7f6c1a7fc640 1 -- 172.21.15.26:0/359087561 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6c2404f330 con 0x7f6c0806b4d0 2024-03-06T07:33:11.169 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 1 to deep-scrub 2024-03-06T07:33:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c0806b4d0 msgr2=0x7f6c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c0806b4d0 0x7f6c0806d990 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c006fd0 tx=0x7f6c1c008040 comp rx=0 tx=0).stop 2024-03-06T07:33:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 msgr2=0x7f6c24144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c24144860 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c700 tx=0x7f6c2000cbc0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 shutdown_connections 2024-03-06T07:33:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f6c0806b4d0 0x7f6c0806d990 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c241561a0 0x7f6c24144860 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240c6ab0 0x7f6c2414a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 --2- 172.21.15.26:0/359087561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c240aa430 0x7f6c2414bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 >> 172.21.15.26:0/359087561 conn(0x7f6c2406b1a0 msgr2=0x7f6c240ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 shutdown_connections 2024-03-06T07:33:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.169+0000 7f6c2c1e4640 1 -- 172.21.15.26:0/359087561 wait complete. 2024-03-06T07:33:11.182 INFO:tasks.ceph:Scrubbing osd.2 2024-03-06T07:33:11.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:11.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.286+0000 7f60f65b6640 1 Processor -- start 2024-03-06T07:33:11.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f65b6640 1 -- start start 2024-03-06T07:33:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f65b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f0152870 0x7f60f0155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f65b6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60f00c88e0 0x7f60f00c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f65b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f60f00c7930 0x7f60f006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f65b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f006d320 con 0x7f60f00c7930 2024-03-06T07:33:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f65b6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f006b8d0 con 0x7f60f00c88e0 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f65b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0156100 con 0x7f60f0152870 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f4b53640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f60f00c7930 0x7f60f006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.287+0000 7f60f4b53640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f60f00c7930 0x7f60f006bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36196/0 (socket says 172.21.15.26:36196) 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60f4b53640 1 -- 172.21.15.26:0/2909464264 learned_addr learned my addr 172.21.15.26:0/2909464264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60ef7fe640 1 --2- 172.21.15.26:0/2909464264 >> v2:172.21.15.117:3300/0 conn(0x7f60f00c88e0 0x7f60f00c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60effff640 1 --2- 172.21.15.26:0/2909464264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f0152870 0x7f60f0155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60f4b53640 1 -- 172.21.15.26:0/2909464264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f0152870 msgr2=0x7f60f0155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60f4b53640 1 --2- 172.21.15.26:0/2909464264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f0152870 0x7f60f0155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60f4b53640 1 -- 172.21.15.26:0/2909464264 >> v2:172.21.15.117:3300/0 conn(0x7f60f00c88e0 msgr2=0x7f60f00c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60f4b53640 1 --2- 172.21.15.26:0/2909464264 >> v2:172.21.15.117:3300/0 conn(0x7f60f00c88e0 0x7f60f00c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.288+0000 7f60f4b53640 1 -- 172.21.15.26:0/2909464264 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00c5400 con 0x7f60f00c7930 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f4b53640 1 --2- 172.21.15.26:0/2909464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f00c7930 0x7f60f006bd60 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f60f008c6b0 tx=0x7f60e802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc0d49f7c8b73c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60eeffd640 1 -- 172.21.15.26:0/2909464264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e802c9e0 con 0x7f60f00c7930 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60eeffd640 1 -- 172.21.15.26:0/2909464264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e802cb80 con 0x7f60f00c7930 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60eeffd640 1 -- 172.21.15.26:0/2909464264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e80343f0 con 0x7f60f00c7930 2024-03-06T07:33:11.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 -- 172.21.15.26:0/2909464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f00c7930 msgr2=0x7f60f006bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 --2- 172.21.15.26:0/2909464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f00c7930 0x7f60f006bd60 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f60f008c6b0 tx=0x7f60e802be40 comp rx=0 tx=0).stop 2024-03-06T07:33:11.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 -- 172.21.15.26:0/2909464264 shutdown_connections 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 --2- 172.21.15.26:0/2909464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f00c7930 0x7f60f006bd60 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 --2- 172.21.15.26:0/2909464264 >> v2:172.21.15.117:3300/0 conn(0x7f60f00c88e0 0x7f60f00c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 --2- 172.21.15.26:0/2909464264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f0152870 0x7f60f0155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 -- 172.21.15.26:0/2909464264 >> 172.21.15.26:0/2909464264 conn(0x7f60f00ac5f0 msgr2=0x7f60f00aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 -- 172.21.15.26:0/2909464264 shutdown_connections 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.289+0000 7f60f65b6640 1 -- 172.21.15.26:0/2909464264 wait complete. 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.290+0000 7f60f65b6640 1 Processor -- start 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f65b6640 1 -- start start 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f65b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60f00c7930 0x7f60f01edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f65b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f00c88e0 0x7f60f01ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f65b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f0152870 0x7f60f014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f65b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00c5ca0 con 0x7f60f0152870 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f65b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00c59c0 con 0x7f60f00c7930 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f65b6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00c9470 con 0x7f60f00c88e0 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f4b53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f0152870 0x7f60f014ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f4b53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f0152870 0x7f60f014ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36208/0 (socket says 172.21.15.26:36208) 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60f4b53640 1 -- 172.21.15.26:0/2750014694 learned_addr learned my addr 172.21.15.26:0/2750014694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60effff640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60f00c7930 0x7f60f01edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.291+0000 7f60ef7fe640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f00c88e0 0x7f60f01ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f4b53640 1 -- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f00c88e0 msgr2=0x7f60f01ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f4b53640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f00c88e0 0x7f60f01ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f4b53640 1 -- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60f00c7930 msgr2=0x7f60f01edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f4b53640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60f00c7930 0x7f60f01edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f4b53640 1 -- 172.21.15.26:0/2750014694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f01eea20 con 0x7f60f0152870 2024-03-06T07:33:11.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60ef7fe640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60f00c88e0 0x7f60f01ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:11.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f4b53640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f0152870 0x7f60f014ec90 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f60e8036040 tx=0x7f60e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8043020 con 0x7f60f0152870 2024-03-06T07:33:11.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e803e450 con 0x7f60f0152870 2024-03-06T07:33:11.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f00c1390 con 0x7f60f0152870 2024-03-06T07:33:11.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8009510 con 0x7f60f0152870 2024-03-06T07:33:11.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.292+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f0153770 con 0x7f60f0152870 2024-03-06T07:33:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.293+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60f0153240 con 0x7f60f0152870 2024-03-06T07:33:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.293+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60e8046020 con 0x7f60f0152870 2024-03-06T07:33:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60ecff9640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60dc06b4d0 0x7f60dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f60e80accd0 con 0x7f60f0152870 2024-03-06T07:33:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60effff640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60dc06b4d0 0x7f60dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60ecff9640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f60dc071b50 0x7f60dc073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 --> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60e8049e50 con 0x7f60dc071b50 2024-03-06T07:33:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60e80ad160 con 0x7f60f0152870 2024-03-06T07:33:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60ef7fe640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f60dc071b50 0x7f60dc073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.294+0000 7f60ef7fe640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f60dc071b50 0x7f60dc073f90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.295+0000 7f60effff640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60dc06b4d0 0x7f60dc06d990 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009b40 tx=0x7f60e4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.295+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== osd.2 v2:172.21.15.26:6824/3726964600 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f60e8049e50 con 0x7f60dc071b50 2024-03-06T07:33:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.308+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 --> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f60f004f330 con 0x7f60dc071b50 2024-03-06T07:33:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60ecff9640 1 -- 172.21.15.26:0/2750014694 <== osd.2 v2:172.21.15.26:6824/3726964600 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f60f004f330 con 0x7f60dc071b50 2024-03-06T07:33:11.310 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:11.311 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:11.311 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f60dc071b50 msgr2=0x7f60dc073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:6824/3726964600,v1:172.21.15.26:6825/3726964600] conn(0x7f60dc071b50 0x7f60dc073f90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60dc06b4d0 msgr2=0x7f60dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f60dc06b4d0 0x7f60dc06d990 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009b40 tx=0x7f60e4009190 comp rx=0 tx=0).stop 2024-03-06T07:33:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f0152870 msgr2=0x7f60f014ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 --2- 172.21.15.26:0/2750014694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60f0152870 0x7f60f014ec90 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f60e8036040 tx=0x7f60e8004a40 comp rx=0 tx=0).stop 2024-03-06T07:33:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f4b53640 1 -- 172.21.15.26:0/2750014694 reap_dead start 2024-03-06T07:33:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 shutdown_connections 2024-03-06T07:33:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 >> 172.21.15.26:0/2750014694 conn(0x7f60f00ac5f0 msgr2=0x7f60f00ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 shutdown_connections 2024-03-06T07:33:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.309+0000 7f60f65b6640 1 -- 172.21.15.26:0/2750014694 wait complete. 2024-03-06T07:33:11.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-06T07:33:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.422+0000 7f0ebb1ac640 1 Processor -- start 2024-03-06T07:33:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.422+0000 7f0ebb1ac640 1 -- start start 2024-03-06T07:33:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0ebb1ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0eb40bfe90 0x7f0eb4155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0ebb1ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 0x7f0eb4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0ebb1ac640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb40c71d0 con 0x7f0eb41593c0 2024-03-06T07:33:11.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0ebb1ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb4158710 con 0x7f0eb40bfe90 2024-03-06T07:33:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0ebb1ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40c59d0 con 0x7f0eb40c6d70 2024-03-06T07:33:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb9749640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0eb41593c0 0x7f0eb41522d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52158/0 (socket says 172.21.15.26:52158) 2024-03-06T07:33:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb9749640 1 -- 172.21.15.26:0/4122202893 learned_addr learned my addr 172.21.15.26:0/4122202893 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247205578 0 0) 0x7f0eb40c71d0 con 0x7f0eb41593c0 2024-03-06T07:33:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea4003520 con 0x7f0eb41593c0 2024-03-06T07:33:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb3fff640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 0x7f0eb4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb8f48640 1 --2- 172.21.15.26:0/4122202893 >> v2:172.21.15.117:3300/0 conn(0x7f0eb40bfe90 0x7f0eb4155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679778223 0 0) 0x7f0ea4003520 con 0x7f0eb41593c0 2024-03-06T07:33:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 msgr2=0x7f0eb4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 0x7f0eb4158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 >> v2:172.21.15.117:3300/0 conn(0x7f0eb40bfe90 msgr2=0x7f0eb4155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 --2- 172.21.15.26:0/4122202893 >> v2:172.21.15.117:3300/0 conn(0x7f0eb40bfe90 0x7f0eb4155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40c5cb0 con 0x7f0eb41593c0 2024-03-06T07:33:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7f0eac0019f0 con 0x7f0eb41593c0 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb8f48640 1 --2- 172.21.15.26:0/4122202893 >> v2:172.21.15.117:3300/0 conn(0x7f0eb40bfe90 0x7f0eb4155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 >> v1:172.21.15.26:6789/0 conn(0x7f0eb41593c0 legacy=0x7f0eb41522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 0x7f0ea40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea4003f30 0x7f0ea4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.423+0000 7f0eb37fe640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4158710 con 0x7f0ea4004d40 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40c59d0 con 0x7f0ea4003f30 2024-03-06T07:33:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea400bac0 con 0x7f0eb40c6d70 2024-03-06T07:33:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb9749640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb8f48640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 0x7f0ea40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb3fff640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea4003f30 0x7f0ea4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb9749640 1 -- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 msgr2=0x7f0ea40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb9749640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0eb40c6d70 0x7f0ea40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb9749640 1 -- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea4003f30 msgr2=0x7f0ea4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb9749640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea4003f30 0x7f0ea4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.424+0000 7f0eb9749640 1 -- 172.21.15.26:0/4122202893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4003520 con 0x7f0ea4004d40 2024-03-06T07:33:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0eb9749640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00a950 tx=0x7f0ea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bb8d2aa3754bb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00a4c0 con 0x7f0ea4004d40 2024-03-06T07:33:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac034630 con 0x7f0ea4004d40 2024-03-06T07:33:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0eb37fe640 1 -- 172.21.15.26:0/4122202893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac034aa0 con 0x7f0ea4004d40 2024-03-06T07:33:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/4122202893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea4004d40 msgr2=0x7f0ea40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0ebb1ac640 1 --2- 172.21.15.26:0/4122202893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00a950 tx=0x7f0ea4005830 comp rx=0 tx=0).stop 2024-03-06T07:33:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0eb9749640 1 -- 172.21.15.26:0/4122202893 reap_dead start 2024-03-06T07:33:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/4122202893 shutdown_connections 2024-03-06T07:33:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/4122202893 >> 172.21.15.26:0/4122202893 conn(0x7f0eb40acaa0 msgr2=0x7f0eb40aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/4122202893 shutdown_connections 2024-03-06T07:33:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.425+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/4122202893 wait complete. 2024-03-06T07:33:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.426+0000 7f0ebb1ac640 1 Processor -- start 2024-03-06T07:33:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.426+0000 7f0ebb1ac640 1 -- start start 2024-03-06T07:33:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0ebb1ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ea4004d40 0x7f0eb41ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0ebb1ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eb41ef120 0x7f0eb4150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0ebb1ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb40c1f70 0x7f0eb413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0ebb1ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40ca700 con 0x7f0eb41ef120 2024-03-06T07:33:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0ebb1ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4158960 con 0x7f0eb40c1f70 2024-03-06T07:33:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0ebb1ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40ca880 con 0x7f0ea4004d40 2024-03-06T07:33:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eb41ef120 0x7f0eb4150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eb41ef120 0x7f0eb4150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36230/0 (socket says 172.21.15.26:36230) 2024-03-06T07:33:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.427+0000 7f0eb3fff640 1 -- 172.21.15.26:0/1829112998 learned_addr learned my addr 172.21.15.26:0/1829112998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb3fff640 1 -- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ea4004d40 msgr2=0x7f0eb41ebc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb3fff640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ea4004d40 0x7f0eb41ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb3fff640 1 -- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb40c1f70 msgr2=0x7f0eb413f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb3fff640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb40c1f70 0x7f0eb413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb3fff640 1 -- 172.21.15.26:0/1829112998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb41ef5a0 con 0x7f0eb41ef120 2024-03-06T07:33:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb3fff640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eb41ef120 0x7f0eb4150590 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0011570 tx=0x7f0ea0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb17fa640 1 -- 172.21.15.26:0/1829112998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0015930 con 0x7f0eb41ef120 2024-03-06T07:33:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb17fa640 1 -- 172.21.15.26:0/1829112998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0015ad0 con 0x7f0eb41ef120 2024-03-06T07:33:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb41eb1a0 con 0x7f0eb41ef120 2024-03-06T07:33:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0eb17fa640 1 -- 172.21.15.26:0/1829112998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea001d400 con 0x7f0eb41ef120 2024-03-06T07:33:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.428+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb406c510 con 0x7f0eb41ef120 2024-03-06T07:33:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.429+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb404f330 con 0x7f0eb41ef120 2024-03-06T07:33:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.429+0000 7f0eb17fa640 1 -- 172.21.15.26:0/1829112998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ea001d5a0 con 0x7f0eb41ef120 2024-03-06T07:33:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.430+0000 7f0eb17fa640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0e9806b500 0x7f0e9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.430+0000 7f0eb17fa640 1 -- 172.21.15.26:0/1829112998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f0ea008a870 con 0x7f0eb41ef120 2024-03-06T07:33:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.430+0000 7f0eb8f48640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0e9806b500 0x7f0e9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.430+0000 7f0eb8f48640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0e9806b500 0x7f0e9806d9c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8006a90 tx=0x7f0ea8006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.434+0000 7f0eb17fa640 1 -- 172.21.15.26:0/1829112998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0ea0019030 con 0x7f0eb41ef120 2024-03-06T07:33:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.614+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb40c5710 con 0x7f0e9806b500 2024-03-06T07:33:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.615+0000 7f0eb17fa640 1 -- 172.21.15.26:0/1829112998 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0eb40c5710 con 0x7f0e9806b500 2024-03-06T07:33:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 2 to deep-scrub 2024-03-06T07:33:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0e9806b500 msgr2=0x7f0e9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0e9806b500 0x7f0e9806d9c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8006a90 tx=0x7f0ea8006a20 comp rx=0 tx=0).stop 2024-03-06T07:33:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eb41ef120 msgr2=0x7f0eb4150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eb41ef120 0x7f0eb4150590 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0011570 tx=0x7f0ea0011a30 comp rx=0 tx=0).stop 2024-03-06T07:33:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 shutdown_connections 2024-03-06T07:33:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f0e9806b500 0x7f0e9806d9c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb40c1f70 0x7f0eb413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eb41ef120 0x7f0eb4150590 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 --2- 172.21.15.26:0/1829112998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ea4004d40 0x7f0eb41ebc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 >> 172.21.15.26:0/1829112998 conn(0x7f0eb40acaa0 msgr2=0x7f0eb40c7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 shutdown_connections 2024-03-06T07:33:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.617+0000 7f0ebb1ac640 1 -- 172.21.15.26:0/1829112998 wait complete. 2024-03-06T07:33:11.629 INFO:tasks.ceph:Scrubbing osd.3 2024-03-06T07:33:11.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:11.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.733+0000 7f4e7366b640 1 Processor -- start 2024-03-06T07:33:11.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.734+0000 7f4e7366b640 1 -- start start 2024-03-06T07:33:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.734+0000 7f4e7366b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e6c0abb60 0x7f4e6c1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.734+0000 7f4e7366b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c2920 0x7f4e6c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.734+0000 7f4e7366b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c070430 con 0x7f4e6c0c1d10 2024-03-06T07:33:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.734+0000 7f4e7366b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0711f0 con 0x7f4e6c0abb60 2024-03-06T07:33:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.734+0000 7f4e7366b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c070d60 con 0x7f4e6c0c2920 2024-03-06T07:33:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e71407640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e6c0abb60 0x7f4e6c1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e71c08640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c2920 0x7f4e6c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e70c06640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4e6c0c1d10 0x7f4e6c156440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:39644/0 (socket says 172.21.15.26:39644) 2024-03-06T07:33:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e70c06640 1 -- 172.21.15.26:0/2571766706 learned_addr learned my addr 172.21.15.26:0/2571766706 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e71c08640 1 -- 172.21.15.26:0/2571766706 >> v2:172.21.15.117:3300/0 conn(0x7f4e6c0abb60 msgr2=0x7f4e6c1502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e71c08640 1 --2- 172.21.15.26:0/2571766706 >> v2:172.21.15.117:3300/0 conn(0x7f4e6c0abb60 0x7f4e6c1502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e71c08640 1 -- 172.21.15.26:0/2571766706 >> v1:172.21.15.117:6789/0 conn(0x7f4e6c0c1d10 legacy=0x7f4e6c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e71c08640 1 -- 172.21.15.26:0/2571766706 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c070710 con 0x7f4e6c0c2920 2024-03-06T07:33:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.735+0000 7f4e63fff640 1 -- 172.21.15.26:0/2571766706 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511655920 0 0) 0x7f4e6c070430 con 0x7f4e6c0c1d10 2024-03-06T07:33:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.736+0000 7f4e71c08640 1 --2- 172.21.15.26:0/2571766706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c2920 0x7f4e6c156b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009d90 tx=0x7f4e6402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62f56ed832e04ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.736+0000 7f4e63fff640 1 -- 172.21.15.26:0/2571766706 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64039070 con 0x7f4e6c0c2920 2024-03-06T07:33:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.736+0000 7f4e63fff640 1 -- 172.21.15.26:0/2571766706 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6402cbc0 con 0x7f4e6c0c2920 2024-03-06T07:33:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.736+0000 7f4e63fff640 1 -- 172.21.15.26:0/2571766706 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64034440 con 0x7f4e6c0c2920 2024-03-06T07:33:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 -- 172.21.15.26:0/2571766706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c2920 msgr2=0x7f4e6c156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 --2- 172.21.15.26:0/2571766706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c2920 0x7f4e6c156b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009d90 tx=0x7f4e6402be10 comp rx=0 tx=0).stop 2024-03-06T07:33:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 -- 172.21.15.26:0/2571766706 shutdown_connections 2024-03-06T07:33:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 --2- 172.21.15.26:0/2571766706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c2920 0x7f4e6c156b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 --2- 172.21.15.26:0/2571766706 >> v2:172.21.15.117:3300/0 conn(0x7f4e6c0abb60 0x7f4e6c1502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 -- 172.21.15.26:0/2571766706 >> 172.21.15.26:0/2571766706 conn(0x7f4e6c0c65b0 msgr2=0x7f4e6c0c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 -- 172.21.15.26:0/2571766706 shutdown_connections 2024-03-06T07:33:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.737+0000 7f4e7366b640 1 -- 172.21.15.26:0/2571766706 wait complete. 2024-03-06T07:33:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.738+0000 7f4e7366b640 1 Processor -- start 2024-03-06T07:33:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.738+0000 7f4e7366b640 1 -- start start 2024-03-06T07:33:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e7366b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e6c0abb60 0x7f4e6c0a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e7366b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e6c0c2920 0x7f4e6c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e7366b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c01e0 0x7f4e6c0ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e7366b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0abf40 con 0x7f4e6c0c2920 2024-03-06T07:33:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e7366b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c157090 con 0x7f4e6c0abb60 2024-03-06T07:33:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e7366b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0ac0c0 con 0x7f4e6c0c01e0 2024-03-06T07:33:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e6c0c2920 0x7f4e6c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e6c0c2920 0x7f4e6c1553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36244/0 (socket says 172.21.15.26:36244) 2024-03-06T07:33:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 -- 172.21.15.26:0/3210152490 learned_addr learned my addr 172.21.15.26:0/3210152490 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e71407640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e6c0abb60 0x7f4e6c0a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e71c08640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c01e0 0x7f4e6c0ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 -- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c01e0 msgr2=0x7f4e6c0ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e6c0c01e0 0x7f4e6c0ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 -- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e6c0abb60 msgr2=0x7f4e6c0a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e6c0abb60 0x7f4e6c0a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 -- 172.21.15.26:0/3210152490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c1eb150 con 0x7f4e6c0c2920 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e70c06640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e6c0c2920 0x7f4e6c1553f0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c4d0 tx=0x7f4e5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c018070 con 0x7f4e6c0c2920 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c007500 con 0x7f4e6c0c2920 2024-03-06T07:33:11.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c0bb680 con 0x7f4e6c0c2920 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.739+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004a20 con 0x7f4e6c0c2920 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.740+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c0ab650 con 0x7f4e6c0c2920 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.740+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e6c0c4620 con 0x7f4e6c0c2920 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e5c01b070 con 0x7f4e6c0c2920 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e61ffb640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e5006b500 0x7f4e5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f4e5c089310 con 0x7f4e6c0c2920 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e61ffb640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4e50071b80 0x7f4e50073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 --> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e50074690 con 0x7f4e50071b80 2024-03-06T07:33:11.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e5c090050 con 0x7f4e6c0c2920 2024-03-06T07:33:11.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e71407640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e5006b500 0x7f4e5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.741+0000 7f4e71c08640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4e50071b80 0x7f4e50073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.742+0000 7f4e71c08640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4e50071b80 0x7f4e50073fc0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.742+0000 7f4e71407640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e5006b500 0x7f4e5006d9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009b40 tx=0x7f4e68009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.742+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== osd.3 v2:172.21.15.26:6816/1166415256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4e50074690 con 0x7f4e50071b80 2024-03-06T07:33:11.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 --> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4e6c04f330 con 0x7f4e50071b80 2024-03-06T07:33:11.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e61ffb640 1 -- 172.21.15.26:0/3210152490 <== osd.3 v2:172.21.15.26:6816/1166415256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4e6c04f330 con 0x7f4e50071b80 2024-03-06T07:33:11.758 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:11.758 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:11.758 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:11.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4e50071b80 msgr2=0x7f4e50073fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:6816/1166415256,v1:172.21.15.26:6817/1166415256] conn(0x7f4e50071b80 0x7f4e50073fc0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e5006b500 msgr2=0x7f4e5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f4e5006b500 0x7f4e5006d9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009b40 tx=0x7f4e68009190 comp rx=0 tx=0).stop 2024-03-06T07:33:11.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e6c0c2920 msgr2=0x7f4e6c1553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 --2- 172.21.15.26:0/3210152490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e6c0c2920 0x7f4e6c1553f0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c4d0 tx=0x7f4e5c00c990 comp rx=0 tx=0).stop 2024-03-06T07:33:11.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e71c08640 1 -- 172.21.15.26:0/3210152490 reap_dead start 2024-03-06T07:33:11.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 shutdown_connections 2024-03-06T07:33:11.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 >> 172.21.15.26:0/3210152490 conn(0x7f4e6c0c65b0 msgr2=0x7f4e6c073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 shutdown_connections 2024-03-06T07:33:11.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.756+0000 7f4e7366b640 1 -- 172.21.15.26:0/3210152490 wait complete. 2024-03-06T07:33:11.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-06T07:33:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.868+0000 7fcfbc45c640 1 Processor -- start 2024-03-06T07:33:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.868+0000 7fcfbc45c640 1 -- start start 2024-03-06T07:33:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfbc45c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcfb40c8950 0x7fcfb40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfbc45c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcfb41558d0 0x7fcfb4155d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfbc45c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb4155130 con 0x7fcfb40c69f0 2024-03-06T07:33:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfbc45c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb4155440 con 0x7fcfb41558d0 2024-03-06T07:33:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfbc45c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40ad480 con 0x7fcfb40c8950 2024-03-06T07:33:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfba9f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcfb41558d0 0x7fcfb4155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfba1f8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcfb40c69f0 0x7fcfb40acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52164/0 (socket says 172.21.15.26:52164) 2024-03-06T07:33:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.869+0000 7fcfba1f8640 1 -- 172.21.15.26:0/1795094095 learned_addr learned my addr 172.21.15.26:0/1795094095 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.870+0000 7fcfb99f7640 1 --2- 172.21.15.26:0/1795094095 >> v2:172.21.15.117:3300/0 conn(0x7fcfb40c8950 0x7fcfb40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.870+0000 7fcfba9f9640 1 -- 172.21.15.26:0/1795094095 >> v2:172.21.15.117:3300/0 conn(0x7fcfb40c8950 msgr2=0x7fcfb40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.870+0000 7fcfba9f9640 1 --2- 172.21.15.26:0/1795094095 >> v2:172.21.15.117:3300/0 conn(0x7fcfb40c8950 0x7fcfb40c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.870+0000 7fcfba9f9640 1 -- 172.21.15.26:0/1795094095 >> v1:172.21.15.26:6789/0 conn(0x7fcfb40c69f0 legacy=0x7fcfb40acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.870+0000 7fcfba9f9640 1 -- 172.21.15.26:0/1795094095 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40c5d50 con 0x7fcfb41558d0 2024-03-06T07:33:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.870+0000 7fcfb91f6640 1 -- 172.21.15.26:0/1795094095 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329815906 0 0) 0x7fcfb0001110 con 0x7fcfb40c69f0 2024-03-06T07:33:11.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfba9f9640 1 --2- 172.21.15.26:0/1795094095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb41558d0 0x7fcfb4155d00 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c7430 tx=0x7fcfac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28ecf3791550712 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfb91f6640 1 -- 172.21.15.26:0/1795094095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac039070 con 0x7fcfb41558d0 2024-03-06T07:33:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfb91f6640 1 -- 172.21.15.26:0/1795094095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac02cbf0 con 0x7fcfb41558d0 2024-03-06T07:33:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfb91f6640 1 -- 172.21.15.26:0/1795094095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac034440 con 0x7fcfb41558d0 2024-03-06T07:33:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 -- 172.21.15.26:0/1795094095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb41558d0 msgr2=0x7fcfb4155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/1795094095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb41558d0 0x7fcfb4155d00 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c7430 tx=0x7fcfac02be40 comp rx=0 tx=0).stop 2024-03-06T07:33:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 -- 172.21.15.26:0/1795094095 shutdown_connections 2024-03-06T07:33:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/1795094095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb41558d0 0x7fcfb4155d00 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/1795094095 >> v2:172.21.15.117:3300/0 conn(0x7fcfb40c8950 0x7fcfb40c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 -- 172.21.15.26:0/1795094095 >> 172.21.15.26:0/1795094095 conn(0x7fcfb40cae70 msgr2=0x7fcfb40cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 -- 172.21.15.26:0/1795094095 shutdown_connections 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.871+0000 7fcfbc45c640 1 -- 172.21.15.26:0/1795094095 wait complete. 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.872+0000 7fcfbc45c640 1 Processor -- start 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfbc45c640 1 -- start start 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfbc45c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb40c69f0 0x7fcfb4154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfbc45c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfb40c8950 0x7fcfb41ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfbc45c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb41558d0 0x7fcfb41ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfbc45c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb4152a20 con 0x7fcfb40c69f0 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfbc45c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c6310 con 0x7fcfb41558d0 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfbc45c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb4152ba0 con 0x7fcfb40c8950 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfba1f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb40c69f0 0x7fcfb4154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfba1f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb40c69f0 0x7fcfb4154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36270/0 (socket says 172.21.15.26:36270) 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfba1f8640 1 -- 172.21.15.26:0/3198128542 learned_addr learned my addr 172.21.15.26:0/3198128542 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfb99f7640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfb40c8950 0x7fcfb41ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.873+0000 7fcfba9f9640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb41558d0 0x7fcfb41ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfba1f8640 1 -- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfb40c8950 msgr2=0x7fcfb41ee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfba1f8640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfb40c8950 0x7fcfb41ee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfba1f8640 1 -- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb41558d0 msgr2=0x7fcfb41ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfba1f8640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb41558d0 0x7fcfb41ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfba1f8640 1 -- 172.21.15.26:0/3198128542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb4156ee0 con 0x7fcfb40c69f0 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfb99f7640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfb40c8950 0x7fcfb41ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfba1f8640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb40c69f0 0x7fcfb4154260 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c430 tx=0x7fcfb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfaaffd640 1 -- 172.21.15.26:0/3198128542 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00074e0 con 0x7fcfb40c69f0 2024-03-06T07:33:11.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfaaffd640 1 -- 172.21.15.26:0/3198128542 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb00047e0 con 0x7fcfb40c69f0 2024-03-06T07:33:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb41571c0 con 0x7fcfb40c69f0 2024-03-06T07:33:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfaaffd640 1 -- 172.21.15.26:0/3198128542 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0004bd0 con 0x7fcfb40c69f0 2024-03-06T07:33:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.874+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb41568e0 con 0x7fcfb40c69f0 2024-03-06T07:33:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.875+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb4075300 con 0x7fcfb40c69f0 2024-03-06T07:33:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.875+0000 7fcfaaffd640 1 -- 172.21.15.26:0/3198128542 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfb0011070 con 0x7fcfb40c69f0 2024-03-06T07:33:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.876+0000 7fcfaaffd640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf9806b4d0 0x7fcf9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.876+0000 7fcfaaffd640 1 -- 172.21.15.26:0/3198128542 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fcfb0089d80 con 0x7fcfb40c69f0 2024-03-06T07:33:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.876+0000 7fcfb99f7640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf9806b4d0 0x7fcf9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.877+0000 7fcfb99f7640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf9806b4d0 0x7fcf9806d990 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcfb414f100 tx=0x7fcfa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:11.880+0000 7fcfaaffd640 1 -- 172.21.15.26:0/3198128542 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcfb4075300 con 0x7fcfb40c69f0 2024-03-06T07:33:12.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.060+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfb404f330 con 0x7fcf9806b4d0 2024-03-06T07:33:12.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.061+0000 7fcfaaffd640 1 -- 172.21.15.26:0/3198128542 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcfb404f330 con 0x7fcf9806b4d0 2024-03-06T07:33:12.063 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 3 to deep-scrub 2024-03-06T07:33:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf9806b4d0 msgr2=0x7fcf9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf9806b4d0 0x7fcf9806d990 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcfb414f100 tx=0x7fcfa4008040 comp rx=0 tx=0).stop 2024-03-06T07:33:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb40c69f0 msgr2=0x7fcfb4154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb40c69f0 0x7fcfb4154260 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c430 tx=0x7fcfb000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 shutdown_connections 2024-03-06T07:33:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fcf9806b4d0 0x7fcf9806d990 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfb41558d0 0x7fcfb41ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfb40c8950 0x7fcfb41ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 --2- 172.21.15.26:0/3198128542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfb40c69f0 0x7fcfb4154260 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 >> 172.21.15.26:0/3198128542 conn(0x7fcfb40cae70 msgr2=0x7fcfb4150570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 shutdown_connections 2024-03-06T07:33:12.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.063+0000 7fcfbc45c640 1 -- 172.21.15.26:0/3198128542 wait complete. 2024-03-06T07:33:12.075 INFO:tasks.ceph:Scrubbing osd.4 2024-03-06T07:33:12.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:12.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f544235e640 1 Processor -- start 2024-03-06T07:33:12.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f544235e640 1 -- start start 2024-03-06T07:33:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f544235e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f543c13dc90 0x7f543c159230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f544235e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c13e070 con 0x7f543c0c53f0 2024-03-06T07:33:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f544235e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c13e420 con 0x7f543c0c69f0 2024-03-06T07:33:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f544235e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c06c190 con 0x7f543c13dc90 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f543bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f543c13dc90 0x7f543c159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f543b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f543c0c53f0 0x7f543c06b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52180/0 (socket says 172.21.15.26:52180) 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f543b7fe640 1 -- 172.21.15.26:0/3632763200 learned_addr learned my addr 172.21.15.26:0/3632763200 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.180+0000 7f543bfff640 1 --2- 172.21.15.26:0/3632763200 >> v2:172.21.15.26:3300/0 conn(0x7f543c13dc90 0x7f543c159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36274/0 (socket says 172.21.15.26:36274) 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.181+0000 7f543bfff640 1 -- 172.21.15.26:0/3632763200 >> v1:172.21.15.117:6789/0 conn(0x7f543c0c69f0 legacy=0x7f543c15fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.181+0000 7f543bfff640 1 -- 172.21.15.26:0/3632763200 >> v1:172.21.15.26:6789/0 conn(0x7f543c0c53f0 legacy=0x7f543c06b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.181+0000 7f543affd640 1 -- 172.21.15.26:0/3632763200 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405536753 0 0) 0x7f543c13e070 con 0x7f543c0c53f0 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.181+0000 7f543bfff640 1 -- 172.21.15.26:0/3632763200 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c13e700 con 0x7f543c13dc90 2024-03-06T07:33:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f543bfff640 1 --2- 172.21.15.26:0/3632763200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c159230 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f542c0097a0 tx=0x7f542c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995f06a9f5e35401 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f543affd640 1 -- 172.21.15.26:0/3632763200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0315a0 con 0x7f543c13dc90 2024-03-06T07:33:12.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f543affd640 1 -- 172.21.15.26:0/3632763200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c037070 con 0x7f543c13dc90 2024-03-06T07:33:12.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f543affd640 1 -- 172.21.15.26:0/3632763200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c031960 con 0x7f543c13dc90 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f544235e640 1 -- 172.21.15.26:0/3632763200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 msgr2=0x7f543c159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f544235e640 1 --2- 172.21.15.26:0/3632763200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c159230 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f542c0097a0 tx=0x7f542c02b840 comp rx=0 tx=0).stop 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f544235e640 1 -- 172.21.15.26:0/3632763200 shutdown_connections 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f544235e640 1 --2- 172.21.15.26:0/3632763200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c159230 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f544235e640 1 -- 172.21.15.26:0/3632763200 >> 172.21.15.26:0/3632763200 conn(0x7f543c0caa70 msgr2=0x7f543c0cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f544235e640 1 -- 172.21.15.26:0/3632763200 shutdown_connections 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.182+0000 7f544235e640 1 -- 172.21.15.26:0/3632763200 wait complete. 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.183+0000 7f544235e640 1 Processor -- start 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f544235e640 1 -- start start 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f544235e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f543c0c53f0 0x7f543c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f544235e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f543c0c69f0 0x7f543c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f544235e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f544235e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0c61f0 con 0x7f543c13dc90 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f544235e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c06bde0 con 0x7f543c0c53f0 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f544235e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c159770 con 0x7f543c0c69f0 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f54408fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f54408fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36282/0 (socket says 172.21.15.26:36282) 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f54408fb640 1 -- 172.21.15.26:0/4091005057 learned_addr learned my addr 172.21.15.26:0/4091005057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f543bfff640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f543c0c53f0 0x7f543c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.184+0000 7f543b7fe640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f543c0c69f0 0x7f543c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f54408fb640 1 -- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f543c0c69f0 msgr2=0x7f543c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f54408fb640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f543c0c69f0 0x7f543c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f54408fb640 1 -- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f543c0c53f0 msgr2=0x7f543c154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f54408fb640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f543c0c53f0 0x7f543c154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f54408fb640 1 -- 172.21.15.26:0/4091005057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c151f80 con 0x7f543c13dc90 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f543b7fe640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f543c0c69f0 0x7f543c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f54408fb640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c157280 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f543400c7b0 tx=0x7f543400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434004b60 con 0x7f543c13dc90 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5434004d00 con 0x7f543c13dc90 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c1531f0 con 0x7f543c13dc90 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543400aa90 con 0x7f543c13dc90 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.185+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c152680 con 0x7f543c13dc90 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.186+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f543c1558f0 con 0x7f543c13dc90 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.186+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f543400ac30 con 0x7f543c13dc90 2024-03-06T07:33:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.187+0000 7f5438ff9640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f542806b500 0x7f542806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.187+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f543408a330 con 0x7f543c13dc90 2024-03-06T07:33:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.187+0000 7f5438ff9640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f5428071b80 0x7f5428073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.187+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 --> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5434016ec0 con 0x7f5428071b80 2024-03-06T07:33:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.187+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5434091050 con 0x7f543c13dc90 2024-03-06T07:33:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.187+0000 7f543bfff640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f542806b500 0x7f542806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.187+0000 7f543b7fe640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f5428071b80 0x7f5428073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.188+0000 7f543bfff640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f542806b500 0x7f542806d9c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f542c036000 tx=0x7f542c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.188+0000 7f543b7fe640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f5428071b80 0x7f5428073fc0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.188+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== osd.4 v2:172.21.15.117:6801/1465648529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5434016ec0 con 0x7f5428071b80 2024-03-06T07:33:12.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.201+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 --> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f543c1523a0 con 0x7f5428071b80 2024-03-06T07:33:12.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f5438ff9640 1 -- 172.21.15.26:0/4091005057 <== osd.4 v2:172.21.15.117:6801/1465648529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f543c1523a0 con 0x7f5428071b80 2024-03-06T07:33:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:12.204 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:12.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f5428071b80 msgr2=0x7f5428073fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6801/1465648529,v1:172.21.15.117:6802/1465648529] conn(0x7f5428071b80 0x7f5428073fc0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f542806b500 msgr2=0x7f542806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f542806b500 0x7f542806d9c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f542c036000 tx=0x7f542c03d040 comp rx=0 tx=0).stop 2024-03-06T07:33:12.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 msgr2=0x7f543c157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 --2- 172.21.15.26:0/4091005057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c13dc90 0x7f543c157280 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f543400c7b0 tx=0x7f543400cc70 comp rx=0 tx=0).stop 2024-03-06T07:33:12.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f54408fb640 1 -- 172.21.15.26:0/4091005057 reap_dead start 2024-03-06T07:33:12.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 shutdown_connections 2024-03-06T07:33:12.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.202+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 >> 172.21.15.26:0/4091005057 conn(0x7f543c0caa70 msgr2=0x7f543c0c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.203+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 shutdown_connections 2024-03-06T07:33:12.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.203+0000 7f544235e640 1 -- 172.21.15.26:0/4091005057 wait complete. 2024-03-06T07:33:12.214 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-06T07:33:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.315+0000 7f8fb5a64640 1 Processor -- start 2024-03-06T07:33:12.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.315+0000 7f8fb5a64640 1 -- start start 2024-03-06T07:33:12.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8fb5a64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8fb0152640 0x7f8fb00c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8fb5a64640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8fb006b6d0 0x7f8fb0158980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8fb5a64640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb01595d0 con 0x7f8fb00c25f0 2024-03-06T07:33:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8fb5a64640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb0155490 con 0x7f8fb0152640 2024-03-06T07:33:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8fb5a64640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb0155770 con 0x7f8fb006b6d0 2024-03-06T07:33:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8faf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8fb0152640 0x7f8fb00c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8faffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8fb00c25f0 0x7f8fb0158ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52196/0 (socket says 172.21.15.26:52196) 2024-03-06T07:33:12.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8faffff640 1 -- 172.21.15.26:0/1694963384 learned_addr learned my addr 172.21.15.26:0/1694963384 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8faf7fe640 1 -- 172.21.15.26:0/1694963384 >> v2:172.21.15.117:3300/0 conn(0x7f8fb006b6d0 msgr2=0x7f8fb0158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8faf7fe640 1 --2- 172.21.15.26:0/1694963384 >> v2:172.21.15.117:3300/0 conn(0x7f8fb006b6d0 0x7f8fb0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8faf7fe640 1 -- 172.21.15.26:0/1694963384 >> v1:172.21.15.26:6789/0 conn(0x7f8fb00c25f0 legacy=0x7f8fb0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8fae7fc640 1 -- 172.21.15.26:0/1694963384 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237400573 0 0) 0x7f8fb01595d0 con 0x7f8fb00c25f0 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.316+0000 7f8faf7fe640 1 -- 172.21.15.26:0/1694963384 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0155a50 con 0x7f8fb0152640 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.317+0000 7f8faf7fe640 1 --2- 172.21.15.26:0/1694963384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb0152640 0x7f8fb00c20b0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00097a0 tx=0x7f8fa002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1481f09c6d47b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.317+0000 7f8fae7fc640 1 -- 172.21.15.26:0/1694963384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0035020 con 0x7f8fb0152640 2024-03-06T07:33:12.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.317+0000 7f8fae7fc640 1 -- 172.21.15.26:0/1694963384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa0031830 con 0x7f8fb0152640 2024-03-06T07:33:12.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.317+0000 7f8fae7fc640 1 -- 172.21.15.26:0/1694963384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0031c00 con 0x7f8fb0152640 2024-03-06T07:33:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 -- 172.21.15.26:0/1694963384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb0152640 msgr2=0x7f8fb00c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/1694963384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb0152640 0x7f8fb00c20b0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00097a0 tx=0x7f8fa002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 -- 172.21.15.26:0/1694963384 shutdown_connections 2024-03-06T07:33:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/1694963384 >> v2:172.21.15.117:3300/0 conn(0x7f8fb006b6d0 0x7f8fb0158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/1694963384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb0152640 0x7f8fb00c20b0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 -- 172.21.15.26:0/1694963384 >> 172.21.15.26:0/1694963384 conn(0x7f8fb00c41b0 msgr2=0x7f8fb00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 -- 172.21.15.26:0/1694963384 shutdown_connections 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.318+0000 7f8fb5a64640 1 -- 172.21.15.26:0/1694963384 wait complete. 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.319+0000 7f8fb5a64640 1 Processor -- start 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.319+0000 7f8fb5a64640 1 -- start start 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8fb5a64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb006b6d0 0x7f8fb0157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8fb5a64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fb00c25f0 0x7f8fb01eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8fb5a64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fb0152640 0x7f8fb01eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8fb5a64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0152a20 con 0x7f8fb006b6d0 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8fb5a64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0156010 con 0x7f8fb0152640 2024-03-06T07:33:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8fb5a64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01550b0 con 0x7f8fb00c25f0 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb006b6d0 0x7f8fb0157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb006b6d0 0x7f8fb0157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36310/0 (socket says 172.21.15.26:36310) 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 -- 172.21.15.26:0/3407770016 learned_addr learned my addr 172.21.15.26:0/3407770016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 -- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fb00c25f0 msgr2=0x7f8fb01eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fb00c25f0 0x7f8fb01eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faffff640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fb0152640 0x7f8fb01eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 -- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fb0152640 msgr2=0x7f8fb01eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fb0152640 0x7f8fb01eb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 -- 172.21.15.26:0/3407770016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb00a7340 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8faf7fe640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb006b6d0 0x7f8fb0157fd0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f8fa002bee0 tx=0x7f8fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8f97fff640 1 -- 172.21.15.26:0/3407770016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa003ba80 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8f97fff640 1 -- 172.21.15.26:0/3407770016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa003bc20 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb00a7600 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.320+0000 7f8f97fff640 1 -- 172.21.15.26:0/3407770016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00453f0 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.321+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb00a7bc0 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.321+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb004f330 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.322+0000 7f8f97fff640 1 -- 172.21.15.26:0/3407770016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fa0038020 con 0x7f8fb006b6d0 2024-03-06T07:33:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.322+0000 7f8f97fff640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f9c06b4d0 0x7f8f9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.323+0000 7f8f97fff640 1 -- 172.21.15.26:0/3407770016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f8fa00abf10 con 0x7f8fb006b6d0 2024-03-06T07:33:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.323+0000 7f8faeffd640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f9c06b4d0 0x7f8f9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.323+0000 7f8faeffd640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f9c06b4d0 0x7f8f9c06d990 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0151910 tx=0x7f8fa4006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.327+0000 7f8f97fff640 1 -- 172.21.15.26:0/3407770016 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8fa0040070 con 0x7f8fb006b6d0 2024-03-06T07:33:12.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.512+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fb0158510 con 0x7f8f9c06b4d0 2024-03-06T07:33:12.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.513+0000 7f8f97fff640 1 -- 172.21.15.26:0/3407770016 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8fb0158510 con 0x7f8f9c06b4d0 2024-03-06T07:33:12.514 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 4 to deep-scrub 2024-03-06T07:33:12.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f9c06b4d0 msgr2=0x7f8f9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f9c06b4d0 0x7f8f9c06d990 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0151910 tx=0x7f8fa4006bf0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb006b6d0 msgr2=0x7f8fb0157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb006b6d0 0x7f8fb0157fd0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f8fa002bee0 tx=0x7f8fa00046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 shutdown_connections 2024-03-06T07:33:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f8f9c06b4d0 0x7f8f9c06d990 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fb0152640 0x7f8fb01eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fb00c25f0 0x7f8fb01eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 --2- 172.21.15.26:0/3407770016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb006b6d0 0x7f8fb0157fd0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 >> 172.21.15.26:0/3407770016 conn(0x7f8fb00c41b0 msgr2=0x7f8fb014c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 shutdown_connections 2024-03-06T07:33:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.515+0000 7f8fb5a64640 1 -- 172.21.15.26:0/3407770016 wait complete. 2024-03-06T07:33:12.527 INFO:tasks.ceph:Scrubbing osd.5 2024-03-06T07:33:12.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:12.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.631+0000 7fee99c39640 1 Processor -- start 2024-03-06T07:33:12.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee99c39640 1 -- start start 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee99c39640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fee940c7470 0x7fee94155d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee99c39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 0x7fee940c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee99c39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee94156260 con 0x7fee94152860 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee99c39640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee94158da0 con 0x7fee940c7470 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee99c39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee940c7a10 con 0x7fee94159230 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fee94152860 0x7fee940c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52200/0 (socket says 172.21.15.26:52200) 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.632+0000 7fee937fe640 1 -- 172.21.15.26:0/1343215827 learned_addr learned my addr 172.21.15.26:0/1343215827 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee93fff640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 0x7fee940c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee92ffd640 1 --2- 172.21.15.26:0/1343215827 >> v2:172.21.15.117:3300/0 conn(0x7fee940c7470 0x7fee94155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972273484 0 0) 0x7fee84001110 con 0x7fee94152860 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee7c003520 con 0x7fee94152860 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 992593483 0 0) 0x7fee7c003520 con 0x7fee94152860 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 msgr2=0x7fee940c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 0x7fee940c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 >> v2:172.21.15.117:3300/0 conn(0x7fee940c7470 msgr2=0x7fee94155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 --2- 172.21.15.26:0/1343215827 >> v2:172.21.15.117:3300/0 conn(0x7fee940c7470 0x7fee94155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940c7cf0 con 0x7fee94152860 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fee84001840 con 0x7fee94152860 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 >> v1:172.21.15.26:6789/0 conn(0x7fee94152860 legacy=0x7fee940c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c0039b0 0x7fee7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 0x7fee7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee7c004d40 0x7fee7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee94158da0 con 0x7fee7c004d40 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee940c7a10 con 0x7fee7c0039b0 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c00b9c0 con 0x7fee94159230 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee93fff640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee7c004d40 0x7fee7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee92ffd640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 0x7fee7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee937fe640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c0039b0 0x7fee7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee93fff640 1 -- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 msgr2=0x7fee7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.633+0000 7fee93fff640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94159230 0x7fee7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.634+0000 7fee93fff640 1 -- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c0039b0 msgr2=0x7fee7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.634+0000 7fee93fff640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c0039b0 0x7fee7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.634+0000 7fee93fff640 1 -- 172.21.15.26:0/1343215827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c003520 con 0x7fee7c004d40 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.634+0000 7fee93fff640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee7c004d40 0x7fee7c0051c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009fc0 tx=0x7fee7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed480cc775618db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.634+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c039070 con 0x7fee7c004d40 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.634+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c02cc20 con 0x7fee7c004d40 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.634+0000 7fee927fc640 1 -- 172.21.15.26:0/1343215827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c009c10 con 0x7fee7c004d40 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.635+0000 7fee99c39640 1 -- 172.21.15.26:0/1343215827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee7c004d40 msgr2=0x7fee7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.635+0000 7fee99c39640 1 --2- 172.21.15.26:0/1343215827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee7c004d40 0x7fee7c0051c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009fc0 tx=0x7fee7c005830 comp rx=0 tx=0).stop 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.635+0000 7fee93fff640 1 -- 172.21.15.26:0/1343215827 reap_dead start 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.635+0000 7fee99c39640 1 -- 172.21.15.26:0/1343215827 shutdown_connections 2024-03-06T07:33:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.635+0000 7fee99c39640 1 -- 172.21.15.26:0/1343215827 >> 172.21.15.26:0/1343215827 conn(0x7fee940ac630 msgr2=0x7fee940aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.635+0000 7fee99c39640 1 -- 172.21.15.26:0/1343215827 shutdown_connections 2024-03-06T07:33:12.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.635+0000 7fee99c39640 1 -- 172.21.15.26:0/1343215827 wait complete. 2024-03-06T07:33:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.636+0000 7fee99c39640 1 Processor -- start 2024-03-06T07:33:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.636+0000 7fee99c39640 1 -- start start 2024-03-06T07:33:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee99c39640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94145c70 0x7fee940c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee99c39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee94146400 0x7fee941eb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee99c39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee940c9df0 0x7fee941eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee99c39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee941596c0 con 0x7fee940c9df0 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee99c39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee94152c40 con 0x7fee94146400 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee99c39640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee94159840 con 0x7fee94145c70 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee940c9df0 0x7fee941eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee940c9df0 0x7fee941eb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36338/0 (socket says 172.21.15.26:36338) 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 -- 172.21.15.26:0/1011436982 learned_addr learned my addr 172.21.15.26:0/1011436982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee92ffd640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee94146400 0x7fee941eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee937fe640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94145c70 0x7fee940c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 -- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94145c70 msgr2=0x7fee940c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee94145c70 0x7fee940c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 -- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee94146400 msgr2=0x7fee941eb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee94146400 0x7fee941eb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee93fff640 1 -- 172.21.15.26:0/1011436982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee941ee980 con 0x7fee940c9df0 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.637+0000 7fee92ffd640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee94146400 0x7fee941eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.638+0000 7fee93fff640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee940c9df0 0x7fee941eb9c0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fee8c02c690 tx=0x7fee8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.638+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c039030 con 0x7fee940c9df0 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.638+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c03a040 con 0x7fee940c9df0 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.638+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee941eec60 con 0x7fee940c9df0 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.638+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c009c10 con 0x7fee940c9df0 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.638+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee941ef220 con 0x7fee940c9df0 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.639+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee941ee370 con 0x7fee940c9df0 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.639+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee8c045020 con 0x7fee940c9df0 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.640+0000 7fee7bfff640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8006b500 0x7fee8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.640+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fee8c0ac120 con 0x7fee940c9df0 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.640+0000 7fee7bfff640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee80071b80 0x7fee80073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.640+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 --> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee8c02cc20 con 0x7fee80071b80 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.640+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8c0b3050 con 0x7fee940c9df0 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.640+0000 7fee937fe640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8006b500 0x7fee8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.640+0000 7fee92ffd640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee80071b80 0x7fee80073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.641+0000 7fee937fe640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8006b500 0x7fee8006d9c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.641+0000 7fee92ffd640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee80071b80 0x7fee80073fc0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.641+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== osd.5 v2:172.21.15.117:6800/3827357056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fee8c02cc20 con 0x7fee80071b80 2024-03-06T07:33:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 --> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fee94075300 con 0x7fee80071b80 2024-03-06T07:33:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee7bfff640 1 -- 172.21.15.26:0/1011436982 <== osd.5 v2:172.21.15.117:6800/3827357056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fee94075300 con 0x7fee80071b80 2024-03-06T07:33:12.657 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:12.657 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:12.657 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee80071b80 msgr2=0x7fee80073fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6800/3827357056,v1:172.21.15.117:6803/3827357056] conn(0x7fee80071b80 0x7fee80073fc0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8006b500 msgr2=0x7fee8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fee8006b500 0x7fee8006d9c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84009040 comp rx=0 tx=0).stop 2024-03-06T07:33:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee940c9df0 msgr2=0x7fee941eb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 --2- 172.21.15.26:0/1011436982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee940c9df0 0x7fee941eb9c0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fee8c02c690 tx=0x7fee8c009750 comp rx=0 tx=0).stop 2024-03-06T07:33:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee93fff640 1 -- 172.21.15.26:0/1011436982 reap_dead start 2024-03-06T07:33:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 shutdown_connections 2024-03-06T07:33:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.655+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 >> 172.21.15.26:0/1011436982 conn(0x7fee940ac630 msgr2=0x7fee94154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.656+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 shutdown_connections 2024-03-06T07:33:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.656+0000 7fee99c39640 1 -- 172.21.15.26:0/1011436982 wait complete. 2024-03-06T07:33:12.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-06T07:33:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.768+0000 7fbd73278640 1 Processor -- start 2024-03-06T07:33:12.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.768+0000 7fbd73278640 1 -- start start 2024-03-06T07:33:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd73278640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd6c0c8950 0x7fbd6c155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd73278640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd6c0c64f0 0x7fbd6c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd73278640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c0c8e30 con 0x7fbd6c152900 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd73278640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c152ce0 con 0x7fbd6c0c64f0 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd73278640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c0c9110 con 0x7fbd6c0c8950 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd71815640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd6c0c64f0 0x7fbd6c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd71014640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbd6c152900 0x7fbd6c0bf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52210/0 (socket says 172.21.15.26:52210) 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd71014640 1 -- 172.21.15.26:0/499104635 learned_addr learned my addr 172.21.15.26:0/499104635 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd70813640 1 --2- 172.21.15.26:0/499104635 >> v2:172.21.15.117:3300/0 conn(0x7fbd6c0c8950 0x7fbd6c155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd71815640 1 -- 172.21.15.26:0/499104635 >> v2:172.21.15.117:3300/0 conn(0x7fbd6c0c8950 msgr2=0x7fbd6c155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd71815640 1 --2- 172.21.15.26:0/499104635 >> v2:172.21.15.117:3300/0 conn(0x7fbd6c0c8950 0x7fbd6c155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd71815640 1 -- 172.21.15.26:0/499104635 >> v1:172.21.15.26:6789/0 conn(0x7fbd6c152900 legacy=0x7fbd6c0bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.769+0000 7fbd63fff640 1 -- 172.21.15.26:0/499104635 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815736686 0 0) 0x7fbd6c0c8e30 con 0x7fbd6c152900 2024-03-06T07:33:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.770+0000 7fbd71815640 1 -- 172.21.15.26:0/499104635 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c0c93f0 con 0x7fbd6c0c64f0 2024-03-06T07:33:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.770+0000 7fbd70813640 1 --2- 172.21.15.26:0/499104635 >> v2:172.21.15.117:3300/0 conn(0x7fbd6c0c8950 0x7fbd6c155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:12.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.770+0000 7fbd71815640 1 --2- 172.21.15.26:0/499104635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c64f0 0x7fbd6c0c6980 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0c5e90 tx=0x7fbd6802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7826258a43498d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.770+0000 7fbd63fff640 1 -- 172.21.15.26:0/499104635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68039070 con 0x7fbd6c0c64f0 2024-03-06T07:33:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.770+0000 7fbd63fff640 1 -- 172.21.15.26:0/499104635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6802cb60 con 0x7fbd6c0c64f0 2024-03-06T07:33:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.770+0000 7fbd63fff640 1 -- 172.21.15.26:0/499104635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68034440 con 0x7fbd6c0c64f0 2024-03-06T07:33:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 -- 172.21.15.26:0/499104635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c64f0 msgr2=0x7fbd6c0c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 --2- 172.21.15.26:0/499104635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c64f0 0x7fbd6c0c6980 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0c5e90 tx=0x7fbd6802bdb0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 -- 172.21.15.26:0/499104635 shutdown_connections 2024-03-06T07:33:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 --2- 172.21.15.26:0/499104635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c64f0 0x7fbd6c0c6980 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 --2- 172.21.15.26:0/499104635 >> v2:172.21.15.117:3300/0 conn(0x7fbd6c0c8950 0x7fbd6c155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 -- 172.21.15.26:0/499104635 >> 172.21.15.26:0/499104635 conn(0x7fbd6c0ac860 msgr2=0x7fbd6c0acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 -- 172.21.15.26:0/499104635 shutdown_connections 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.771+0000 7fbd73278640 1 -- 172.21.15.26:0/499104635 wait complete. 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.772+0000 7fbd73278640 1 Processor -- start 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.772+0000 7fbd73278640 1 -- start start 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd73278640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd6c0c64f0 0x7fbd6c14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd73278640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c8950 0x7fbd6c1eea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd73278640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd6c152900 0x7fbd6c1eef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd73278640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0c0180 con 0x7fbd6c0c8950 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd73278640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0c0000 con 0x7fbd6c152900 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd73278640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0c55f0 con 0x7fbd6c0c64f0 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c8950 0x7fbd6c1eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c8950 0x7fbd6c1eea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36352/0 (socket says 172.21.15.26:36352) 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 -- 172.21.15.26:0/76862370 learned_addr learned my addr 172.21.15.26:0/76862370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 -- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd6c0c64f0 msgr2=0x7fbd6c14a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd6c0c64f0 0x7fbd6c14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 -- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd6c152900 msgr2=0x7fbd6c1eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd71815640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd6c152900 0x7fbd6c1eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd6c152900 0x7fbd6c1eef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 -- 172.21.15.26:0/76862370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c1ee540 con 0x7fbd6c0c8950 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd70813640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c8950 0x7fbd6c1eea20 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00e840 tx=0x7fbd5c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.773+0000 7fbd61ffb640 1 -- 172.21.15.26:0/76862370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00f040 con 0x7fbd6c0c8950 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.774+0000 7fbd61ffb640 1 -- 172.21.15.26:0/76862370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c004030 con 0x7fbd6c0c8950 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.774+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c14fae0 con 0x7fbd6c0c8950 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.774+0000 7fbd61ffb640 1 -- 172.21.15.26:0/76862370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c004400 con 0x7fbd6c0c8950 2024-03-06T07:33:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.774+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c14fdc0 con 0x7fbd6c0c8950 2024-03-06T07:33:12.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.775+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c04f330 con 0x7fbd6c0c8950 2024-03-06T07:33:12.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.775+0000 7fbd61ffb640 1 -- 172.21.15.26:0/76862370 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd5c00c670 con 0x7fbd6c0c8950 2024-03-06T07:33:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.775+0000 7fbd61ffb640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd5006b4d0 0x7fbd5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.776+0000 7fbd61ffb640 1 -- 172.21.15.26:0/76862370 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fbd5c089d90 con 0x7fbd6c0c8950 2024-03-06T07:33:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.776+0000 7fbd71014640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd5006b4d0 0x7fbd5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.776+0000 7fbd71014640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd5006b4d0 0x7fbd5006d990 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fbd64007fd0 tx=0x7fbd64007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.780+0000 7fbd61ffb640 1 -- 172.21.15.26:0/76862370 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd5c017020 con 0x7fbd6c0c8950 2024-03-06T07:33:12.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.963+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd6c159250 con 0x7fbd5006b4d0 2024-03-06T07:33:12.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.963+0000 7fbd61ffb640 1 -- 172.21.15.26:0/76862370 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbd6c159250 con 0x7fbd5006b4d0 2024-03-06T07:33:12.965 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 5 to deep-scrub 2024-03-06T07:33:12.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd5006b4d0 msgr2=0x7fbd5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd5006b4d0 0x7fbd5006d990 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fbd64007fd0 tx=0x7fbd64007960 comp rx=0 tx=0).stop 2024-03-06T07:33:12.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c8950 msgr2=0x7fbd6c1eea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c8950 0x7fbd6c1eea20 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00e840 tx=0x7fbd5c00ed00 comp rx=0 tx=0).stop 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 shutdown_connections 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fbd5006b4d0 0x7fbd5006d990 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd6c152900 0x7fbd6c1eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd6c0c8950 0x7fbd6c1eea20 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 --2- 172.21.15.26:0/76862370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd6c0c64f0 0x7fbd6c14a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 >> 172.21.15.26:0/76862370 conn(0x7fbd6c0ac860 msgr2=0x7fbd6c14e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 shutdown_connections 2024-03-06T07:33:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:12.965+0000 7fbd73278640 1 -- 172.21.15.26:0/76862370 wait complete. 2024-03-06T07:33:12.977 INFO:tasks.ceph:Scrubbing osd.6 2024-03-06T07:33:12.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.081+0000 7f9517455640 1 Processor -- start 2024-03-06T07:33:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f9517455640 1 -- start start 2024-03-06T07:33:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f9517455640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9510072fc0 0x7f95100724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f9517455640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072a10 0x7f95100c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f9517455640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95100c6220 con 0x7f95100c5e40 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f9517455640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9510070210 con 0x7f9510072fc0 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f9517455640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95100704d0 con 0x7f9510072a10 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f95149f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9510072fc0 0x7f95100724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f95151f1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95100c5e40 0x7f9510153940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52212/0 (socket says 172.21.15.26:52212) 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.082+0000 7f95151f1640 1 -- 172.21.15.26:0/748336287 learned_addr learned my addr 172.21.15.26:0/748336287 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.083+0000 7f95159f2640 1 --2- 172.21.15.26:0/748336287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072a10 0x7f95100c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.083+0000 7f95149f0640 1 -- 172.21.15.26:0/748336287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072a10 msgr2=0x7f95100c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.083+0000 7f95149f0640 1 --2- 172.21.15.26:0/748336287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072a10 0x7f95100c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.083+0000 7f95149f0640 1 -- 172.21.15.26:0/748336287 >> v1:172.21.15.26:6789/0 conn(0x7f95100c5e40 legacy=0x7f9510153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.083+0000 7f95149f0640 1 -- 172.21.15.26:0/748336287 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951006f610 con 0x7f9510072fc0 2024-03-06T07:33:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.083+0000 7f9507fff640 1 -- 172.21.15.26:0/748336287 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137417627 0 0) 0x7f95100c6220 con 0x7f95100c5e40 2024-03-06T07:33:13.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.083+0000 7f95149f0640 1 --2- 172.21.15.26:0/748336287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9510072fc0 0x7f95100724d0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f95000097a0 tx=0x7f950002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c40303c6344e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9507fff640 1 -- 172.21.15.26:0/748336287 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95000315a0 con 0x7f9510072fc0 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9507fff640 1 -- 172.21.15.26:0/748336287 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9500037070 con 0x7f9510072fc0 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9507fff640 1 -- 172.21.15.26:0/748336287 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500031960 con 0x7f9510072fc0 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 -- 172.21.15.26:0/748336287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9510072fc0 msgr2=0x7f95100724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 --2- 172.21.15.26:0/748336287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9510072fc0 0x7f95100724d0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f95000097a0 tx=0x7f950002b840 comp rx=0 tx=0).stop 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 -- 172.21.15.26:0/748336287 shutdown_connections 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 --2- 172.21.15.26:0/748336287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9510072fc0 0x7f95100724d0 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 --2- 172.21.15.26:0/748336287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072a10 0x7f95100c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 -- 172.21.15.26:0/748336287 >> 172.21.15.26:0/748336287 conn(0x7f95100c88f0 msgr2=0x7f95100c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 -- 172.21.15.26:0/748336287 shutdown_connections 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.084+0000 7f9517455640 1 -- 172.21.15.26:0/748336287 wait complete. 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.085+0000 7f9517455640 1 Processor -- start 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.085+0000 7f9517455640 1 -- start start 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f9517455640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9510072a10 0x7f95101ec400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f9517455640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072fc0 0x7f95100ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f9517455640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c5e40 0x7f95101e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f9517455640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95100c4c10 con 0x7f95100c5e40 2024-03-06T07:33:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f9517455640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951006fbd0 con 0x7f9510072a10 2024-03-06T07:33:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f9517455640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95100c4d90 con 0x7f9510072fc0 2024-03-06T07:33:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c5e40 0x7f95101e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c5e40 0x7f95101e57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36362/0 (socket says 172.21.15.26:36362) 2024-03-06T07:33:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 -- 172.21.15.26:0/1535961743 learned_addr learned my addr 172.21.15.26:0/1535961743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95149f0640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072fc0 0x7f95100ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95151f1640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9510072a10 0x7f95101ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 -- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072fc0 msgr2=0x7f95100ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9510072fc0 0x7f95100ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 -- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9510072a10 msgr2=0x7f95101ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9510072a10 0x7f95101ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.086+0000 7f95159f2640 1 -- 172.21.15.26:0/1535961743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100bec80 con 0x7f95100c5e40 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.087+0000 7f95159f2640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c5e40 0x7f95101e57e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f9508007e70 tx=0x7f950800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.087+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508014620 con 0x7f95100c5e40 2024-03-06T07:33:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.087+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9508019070 con 0x7f95100c5e40 2024-03-06T07:33:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.087+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508014a20 con 0x7f95100c5e40 2024-03-06T07:33:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.087+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95101ec940 con 0x7f95100c5e40 2024-03-06T07:33:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.087+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95100bf200 con 0x7f95100c5e40 2024-03-06T07:33:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.088+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f951004f330 con 0x7f95100c5e40 2024-03-06T07:33:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.088+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9508014bc0 con 0x7f95100c5e40 2024-03-06T07:33:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.089+0000 7f9505ffb640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.089+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f950808aaa0 con 0x7f95100c5e40 2024-03-06T07:33:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.089+0000 7f9505ffb640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f94f4071b50 0x7f94f4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.089+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 --> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f950800ee90 con 0x7f94f4071b50 2024-03-06T07:33:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.089+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7f950808f2a0 con 0x7f95100c5e40 2024-03-06T07:33:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.089+0000 7f95151f1640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.089+0000 7f95149f0640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f94f4071b50 0x7f94f4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.090+0000 7f95151f1640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f950c000f60 tx=0x7f950c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.090+0000 7f95149f0640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f94f4071b50 0x7f94f4073f90 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.091+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== osd.6 v2:172.21.15.117:6816/1228719702 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f950800ee90 con 0x7f94f4071b50 2024-03-06T07:33:13.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.104+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 --> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f951006f3f0 con 0x7f94f4071b50 2024-03-06T07:33:13.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.104+0000 7f9505ffb640 1 -- 172.21.15.26:0/1535961743 <== osd.6 v2:172.21.15.117:6816/1228719702 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f951006f3f0 con 0x7f94f4071b50 2024-03-06T07:33:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:13.106 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.104+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f94f4071b50 msgr2=0x7f94f4073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.104+0000 7f9517455640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6816/1228719702,v1:172.21.15.117:6817/1228719702] conn(0x7f94f4071b50 0x7f94f4073f90 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 msgr2=0x7f94f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f94f406b4d0 0x7f94f406d990 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f950c000f60 tx=0x7f950c009040 comp rx=0 tx=0).stop 2024-03-06T07:33:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c5e40 msgr2=0x7f95101e57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 --2- 172.21.15.26:0/1535961743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95100c5e40 0x7f95101e57e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f9508007e70 tx=0x7f950800e4b0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f95159f2640 1 -- 172.21.15.26:0/1535961743 reap_dead start 2024-03-06T07:33:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 shutdown_connections 2024-03-06T07:33:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 >> 172.21.15.26:0/1535961743 conn(0x7f95100c88f0 msgr2=0x7f95100c3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 shutdown_connections 2024-03-06T07:33:13.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.105+0000 7f9517455640 1 -- 172.21.15.26:0/1535961743 wait complete. 2024-03-06T07:33:13.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-06T07:33:13.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.217+0000 7f56566c9640 1 Processor -- start 2024-03-06T07:33:13.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.217+0000 7f56566c9640 1 -- start start 2024-03-06T07:33:13.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f56566c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f56500c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f56566c9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56501522d0 0x7f5650152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f56566c9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5650159590 con 0x7f56500bfe90 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f56566c9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5650158710 con 0x7f56501522d0 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f56566c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500c59d0 con 0x7f5650155120 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f5654c66640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56501522d0 0x7f5650152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f564f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f56500c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f564ffff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f56500bfe90 0x7f5650158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:39654/0 (socket says 172.21.15.26:39654) 2024-03-06T07:33:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.218+0000 7f564ffff640 1 -- 172.21.15.26:0/2923424877 learned_addr learned my addr 172.21.15.26:0/2923424877 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:13.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.219+0000 7f5654c66640 1 -- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 msgr2=0x7f56500c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.219+0000 7f5654c66640 1 --2- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f56500c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.219+0000 7f5654c66640 1 -- 172.21.15.26:0/2923424877 >> v1:172.21.15.117:6789/0 conn(0x7f56500bfe90 legacy=0x7f5650158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.219+0000 7f5654c66640 1 -- 172.21.15.26:0/2923424877 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500c5cb0 con 0x7f56501522d0 2024-03-06T07:33:13.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.219+0000 7f564effd640 1 -- 172.21.15.26:0/2923424877 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998563602 0 0) 0x7f5650159590 con 0x7f56500bfe90 2024-03-06T07:33:13.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.219+0000 7f564f7fe640 1 --2- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f56500c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:13.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f5654c66640 1 --2- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 0x7f5650152740 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5648009d90 tx=0x7f564802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287ed7b71d0dacbc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f564effd640 1 -- 172.21.15.26:0/2923424877 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648039070 con 0x7f56501522d0 2024-03-06T07:33:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f564effd640 1 -- 172.21.15.26:0/2923424877 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564802cbc0 con 0x7f56501522d0 2024-03-06T07:33:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f564effd640 1 -- 172.21.15.26:0/2923424877 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648034440 con 0x7f56501522d0 2024-03-06T07:33:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 -- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 msgr2=0x7f5650152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 --2- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 0x7f5650152740 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5648009d90 tx=0x7f564802be10 comp rx=0 tx=0).stop 2024-03-06T07:33:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 -- 172.21.15.26:0/2923424877 shutdown_connections 2024-03-06T07:33:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 --2- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 0x7f5650152740 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 --2- 172.21.15.26:0/2923424877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f56500c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 -- 172.21.15.26:0/2923424877 >> 172.21.15.26:0/2923424877 conn(0x7f56500acaa0 msgr2=0x7f56500aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 -- 172.21.15.26:0/2923424877 shutdown_connections 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.220+0000 7f56566c9640 1 -- 172.21.15.26:0/2923424877 wait complete. 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.221+0000 7f56566c9640 1 Processor -- start 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f56566c9640 1 -- start start 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f56566c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500bfe90 0x7f56501eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f56566c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 0x7f565006c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f56566c9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f565014f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f56566c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500ca700 con 0x7f56500bfe90 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f56566c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650158960 con 0x7f56501522d0 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f56566c9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500ca880 con 0x7f5650155120 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f564ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500bfe90 0x7f56501eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f564ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500bfe90 0x7f56501eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36378/0 (socket says 172.21.15.26:36378) 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.222+0000 7f564ffff640 1 -- 172.21.15.26:0/935800634 learned_addr learned my addr 172.21.15.26:0/935800634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564f7fe640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 0x7f565006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f5654c66640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f565014f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564ffff640 1 -- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 msgr2=0x7f565014f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564ffff640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f565014f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564ffff640 1 -- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 msgr2=0x7f565006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564ffff640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 0x7f565006c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564ffff640 1 -- 172.21.15.26:0/935800634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500c1d00 con 0x7f56500bfe90 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f5654c66640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f565014f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564ffff640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500bfe90 0x7f56501eb190 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f563800c500 tx=0x7f563800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564cff9640 1 -- 172.21.15.26:0/935800634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638018070 con 0x7f56500bfe90 2024-03-06T07:33:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564cff9640 1 -- 172.21.15.26:0/935800634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5638007580 con 0x7f56500bfe90 2024-03-06T07:33:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56500c1fe0 con 0x7f56500bfe90 2024-03-06T07:33:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f564cff9640 1 -- 172.21.15.26:0/935800634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638004a20 con 0x7f56500bfe90 2024-03-06T07:33:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.223+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565006c7c0 con 0x7f56500bfe90 2024-03-06T07:33:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.224+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565004f330 con 0x7f56500bfe90 2024-03-06T07:33:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.225+0000 7f564cff9640 1 -- 172.21.15.26:0/935800634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f563801b070 con 0x7f56500bfe90 2024-03-06T07:33:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.225+0000 7f564cff9640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f563006b4d0 0x7f563006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.225+0000 7f564cff9640 1 -- 172.21.15.26:0/935800634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f5638089380 con 0x7f56500bfe90 2024-03-06T07:33:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.225+0000 7f564f7fe640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f563006b4d0 0x7f563006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.226+0000 7f564f7fe640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f563006b4d0 0x7f563006d990 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5640009560 tx=0x7f5640009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.229+0000 7f564cff9640 1 -- 172.21.15.26:0/935800634 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5638057cb0 con 0x7f56500bfe90 2024-03-06T07:33:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.411+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f56500c53d0 con 0x7f563006b4d0 2024-03-06T07:33:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.412+0000 7f564cff9640 1 -- 172.21.15.26:0/935800634 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56500c53d0 con 0x7f563006b4d0 2024-03-06T07:33:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 6 to deep-scrub 2024-03-06T07:33:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f563006b4d0 msgr2=0x7f563006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f563006b4d0 0x7f563006d990 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5640009560 tx=0x7f5640009190 comp rx=0 tx=0).stop 2024-03-06T07:33:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500bfe90 msgr2=0x7f56501eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500bfe90 0x7f56501eb190 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f563800c500 tx=0x7f563800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 shutdown_connections 2024-03-06T07:33:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f563006b4d0 0x7f563006d990 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650155120 0x7f565014f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56501522d0 0x7f565006c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 --2- 172.21.15.26:0/935800634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500bfe90 0x7f56501eb190 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 >> 172.21.15.26:0/935800634 conn(0x7f56500acaa0 msgr2=0x7f56500c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 shutdown_connections 2024-03-06T07:33:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.414+0000 7f56566c9640 1 -- 172.21.15.26:0/935800634 wait complete. 2024-03-06T07:33:13.426 INFO:tasks.ceph:Scrubbing osd.7 2024-03-06T07:33:13.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:13.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.530+0000 7fc093866640 1 Processor -- start 2024-03-06T07:33:13.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.530+0000 7fc093866640 1 -- start start 2024-03-06T07:33:13.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc093866640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc08c0ab700 0x7fc08c14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc093866640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc08c0a9450 0x7fc08c14cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc093866640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c06e280 con 0x7fc08c06c240 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc093866640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c0ac450 con 0x7fc08c0ab700 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc093866640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c0ac170 con 0x7fc08c0a9450 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc090e01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc08c0ab700 0x7fc08c14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc090e01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc08c0ab700 0x7fc08c14ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36386/0 (socket says 172.21.15.26:36386) 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc090e01640 1 -- 172.21.15.26:0/1531615152 learned_addr learned my addr 172.21.15.26:0/1531615152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc091e03640 1 --2- 172.21.15.26:0/1531615152 >> v2:172.21.15.117:3300/0 conn(0x7fc08c0a9450 0x7fc08c14cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc090e01640 1 -- 172.21.15.26:0/1531615152 >> v2:172.21.15.117:3300/0 conn(0x7fc08c0a9450 msgr2=0x7fc08c14cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc090e01640 1 --2- 172.21.15.26:0/1531615152 >> v2:172.21.15.117:3300/0 conn(0x7fc08c0a9450 0x7fc08c14cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc090e01640 1 -- 172.21.15.26:0/1531615152 >> v1:172.21.15.117:6789/0 conn(0x7fc08c06c240 legacy=0x7fc08c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc090e01640 1 -- 172.21.15.26:0/1531615152 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c152f90 con 0x7fc08c0ab700 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.531+0000 7fc091e03640 1 --2- 172.21.15.26:0/1531615152 >> v2:172.21.15.117:3300/0 conn(0x7fc08c0a9450 0x7fc08c14cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.532+0000 7fc090e01640 1 --2- 172.21.15.26:0/1531615152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c14ca00 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fc08c067250 tx=0x7fc07c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ca600f3baaccb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.532+0000 7fc083fff640 1 -- 172.21.15.26:0/1531615152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c035020 con 0x7fc08c0ab700 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.532+0000 7fc083fff640 1 -- 172.21.15.26:0/1531615152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c031870 con 0x7fc08c0ab700 2024-03-06T07:33:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.532+0000 7fc083fff640 1 -- 172.21.15.26:0/1531615152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c031c40 con 0x7fc08c0ab700 2024-03-06T07:33:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 -- 172.21.15.26:0/1531615152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 msgr2=0x7fc08c14ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 --2- 172.21.15.26:0/1531615152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c14ca00 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fc08c067250 tx=0x7fc07c02b910 comp rx=0 tx=0).stop 2024-03-06T07:33:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 -- 172.21.15.26:0/1531615152 shutdown_connections 2024-03-06T07:33:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 --2- 172.21.15.26:0/1531615152 >> v2:172.21.15.117:3300/0 conn(0x7fc08c0a9450 0x7fc08c14cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 --2- 172.21.15.26:0/1531615152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c14ca00 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 -- 172.21.15.26:0/1531615152 >> 172.21.15.26:0/1531615152 conn(0x7fc08c0c3960 msgr2=0x7fc08c0c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 -- 172.21.15.26:0/1531615152 shutdown_connections 2024-03-06T07:33:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.533+0000 7fc093866640 1 -- 172.21.15.26:0/1531615152 wait complete. 2024-03-06T07:33:13.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.534+0000 7fc093866640 1 Processor -- start 2024-03-06T07:33:13.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.534+0000 7fc093866640 1 -- start start 2024-03-06T07:33:13.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc093866640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c06c240 0x7fc08c14b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc093866640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c0a9450 0x7fc08c0c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc093866640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc093866640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0abdb0 con 0x7fc08c0ab700 2024-03-06T07:33:13.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc093866640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c153550 con 0x7fc08c06c240 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc093866640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0abf30 con 0x7fc08c0a9450 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c0c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36400/0 (socket says 172.21.15.26:36400) 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 -- 172.21.15.26:0/3399220955 learned_addr learned my addr 172.21.15.26:0/3399220955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091602640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c06c240 0x7fc08c14b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc090e01640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c0a9450 0x7fc08c0c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 -- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c0a9450 msgr2=0x7fc08c0c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c0a9450 0x7fc08c0c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 -- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c06c240 msgr2=0x7fc08c14b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c06c240 0x7fc08c14b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 -- 172.21.15.26:0/3399220955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c1ebd60 con 0x7fc08c0ab700 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc091e03640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c0c5700 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fc08400ced0 tx=0x7fc08400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084003b90 con 0x7fc08c0ab700 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.535+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084003d50 con 0x7fc08c0ab700 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.536+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084004260 con 0x7fc08c0ab700 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.536+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c147770 con 0x7fc08c0ab700 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.536+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c1e9550 con 0x7fc08c0ab700 2024-03-06T07:33:13.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.536+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc08c04f330 con 0x7fc08c0ab700 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.537+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc084004400 con 0x7fc08c0ab700 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.537+0000 7fc081ffb640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.537+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fc08408aab0 con 0x7fc08c0ab700 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.538+0000 7fc081ffb640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7fc070071b50 0x7fc070073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.538+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 --> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc070074660 con 0x7fc070071b50 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.538+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=218) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc08408f2c0 con 0x7fc08c0ab700 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.538+0000 7fc091602640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.538+0000 7fc090e01640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7fc070071b50 0x7fc070073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.538+0000 7fc091602640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc07006b4d0 0x7fc07006d990 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc088007fd0 tx=0x7fc088009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.538+0000 7fc090e01640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7fc070071b50 0x7fc070073f90 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.539+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== osd.7 v2:172.21.15.117:6818/4144741400 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc070074660 con 0x7fc070071b50 2024-03-06T07:33:13.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.552+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 --> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc08c0c3740 con 0x7fc070071b50 2024-03-06T07:33:13.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc081ffb640 1 -- 172.21.15.26:0/3399220955 <== osd.7 v2:172.21.15.117:6818/4144741400 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc08c0c3740 con 0x7fc070071b50 2024-03-06T07:33:13.555 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-06T07:33:13.555 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:13.555 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-06T07:33:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7fc070071b50 msgr2=0x7fc070073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6818/4144741400,v1:172.21.15.117:6820/4144741400] conn(0x7fc070071b50 0x7fc070073f90 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc07006b4d0 msgr2=0x7fc07006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fc07006b4d0 0x7fc07006d990 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc088007fd0 tx=0x7fc088009040 comp rx=0 tx=0).stop 2024-03-06T07:33:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 msgr2=0x7fc08c0c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 --2- 172.21.15.26:0/3399220955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc08c0ab700 0x7fc08c0c5700 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fc08400ced0 tx=0x7fc08400a710 comp rx=0 tx=0).stop 2024-03-06T07:33:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc091e03640 1 -- 172.21.15.26:0/3399220955 reap_dead start 2024-03-06T07:33:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 shutdown_connections 2024-03-06T07:33:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 >> 172.21.15.26:0/3399220955 conn(0x7fc08c0c3960 msgr2=0x7fc08c151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 shutdown_connections 2024-03-06T07:33:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.553+0000 7fc093866640 1 -- 172.21.15.26:0/3399220955 wait complete. 2024-03-06T07:33:13.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-06T07:33:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.666+0000 7fdd002ed640 1 Processor -- start 2024-03-06T07:33:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.666+0000 7fdd002ed640 1 -- start start 2024-03-06T07:33:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdd002ed640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdcf81529e0 0x7fdcf815d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdd002ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf8152e70 con 0x7fdcf80c6ab0 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdd002ed640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf80a9990 con 0x7fdcf81561a0 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdd002ed640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf8155d10 con 0x7fdcf81529e0 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfe089640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdcf80c6ab0 0x7fdcf80a9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52224/0 (socket says 172.21.15.26:52224) 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfe089640 1 -- 172.21.15.26:0/836242032 learned_addr learned my addr 172.21.15.26:0/836242032 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfe88a640 1 --2- 172.21.15.26:0/836242032 >> v2:172.21.15.117:3300/0 conn(0x7fdcf81529e0 0x7fdcf815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665275262 0 0) 0x7fdcf8152e70 con 0x7fdcf80c6ab0 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce4005730 con 0x7fdcf80c6ab0 2024-03-06T07:33:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520169041 0 0) 0x7fdcf80a9990 con 0x7fdcf81561a0 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf8152e70 con 0x7fdcf81561a0 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882561677 0 0) 0x7fdce4005730 con 0x7fdcf80c6ab0 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 >> v2:172.21.15.117:3300/0 conn(0x7fdcf81529e0 msgr2=0x7fdcf815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 --2- 172.21.15.26:0/836242032 >> v2:172.21.15.117:3300/0 conn(0x7fdcf81529e0 0x7fdcf815d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 >> v1:172.21.15.117:6789/0 conn(0x7fdcf81561a0 legacy=0x7fdcf814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf81556c0 con 0x7fdcf80c6ab0 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfe88a640 1 --2- 172.21.15.26:0/836242032 >> v2:172.21.15.117:3300/0 conn(0x7fdcf81529e0 0x7fdcf815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3224862605 0 0) 0x7fdcf4001670 con 0x7fdcf80c6ab0 2024-03-06T07:33:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 >> v1:172.21.15.26:6789/0 conn(0x7fdcf80c6ab0 legacy=0x7fdcf80a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdce4006170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce4006a10 0x7fdce4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce40073d0 0x7fdce400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8155d10 con 0x7fdce4005d50 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce400e370 con 0x7fdce4006a10 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce400e570 con 0x7fdce40073d0 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.667+0000 7fdcfe089640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdce4006170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe88a640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce40073d0 0x7fdce400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfd888640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce4006a10 0x7fdce4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe089640 1 -- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce40073d0 msgr2=0x7fdce400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe089640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce40073d0 0x7fdce400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe089640 1 -- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce4006a10 msgr2=0x7fdce4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe089640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce4006a10 0x7fdce4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe089640 1 -- 172.21.15.26:0/836242032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4005730 con 0x7fdce4005d50 2024-03-06T07:33:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfd888640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce4006a10 0x7fdce4006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe88a640 1 -- 172.21.15.26:0/836242032 reap_dead start 2024-03-06T07:33:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.668+0000 7fdcfe089640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdce4006170 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c210 tx=0x7fdce4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9c9766b0bfe5fd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4007ce0 con 0x7fdce4005d50 2024-03-06T07:33:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf4030070 con 0x7fdce4005d50 2024-03-06T07:33:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdcfd087640 1 -- 172.21.15.26:0/836242032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4005d90 con 0x7fdce4005d50 2024-03-06T07:33:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdd002ed640 1 -- 172.21.15.26:0/836242032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 msgr2=0x7fdce4006170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdd002ed640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdce4006170 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c210 tx=0x7fdce4007a40 comp rx=0 tx=0).stop 2024-03-06T07:33:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdd002ed640 1 -- 172.21.15.26:0/836242032 shutdown_connections 2024-03-06T07:33:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdd002ed640 1 --2- 172.21.15.26:0/836242032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdce4006170 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdd002ed640 1 -- 172.21.15.26:0/836242032 >> 172.21.15.26:0/836242032 conn(0x7fdcf806b1a0 msgr2=0x7fdcf806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdd002ed640 1 -- 172.21.15.26:0/836242032 shutdown_connections 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.669+0000 7fdd002ed640 1 -- 172.21.15.26:0/836242032 wait complete. 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.670+0000 7fdd002ed640 1 Processor -- start 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.670+0000 7fdd002ed640 1 -- start start 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdd002ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdcf80ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdd002ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf80bfc40 0x7fdcf8145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdd002ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf8145cd0 0x7fdcf80c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdd002ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80c2a90 con 0x7fdce4005d50 2024-03-06T07:33:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdd002ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80a9650 con 0x7fdcf80bfc40 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdd002ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80c2c10 con 0x7fdcf8145cd0 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdcf80ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdcf80ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36408/0 (socket says 172.21.15.26:36408) 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 -- 172.21.15.26:0/3148932721 learned_addr learned my addr 172.21.15.26:0/3148932721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe88a640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf8145cd0 0x7fdcf80c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfd888640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf80bfc40 0x7fdcf8145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 -- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf8145cd0 msgr2=0x7fdcf80c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf8145cd0 0x7fdcf80c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 -- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf80bfc40 msgr2=0x7fdcf8145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf80bfc40 0x7fdcf8145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.671+0000 7fdcfe089640 1 -- 172.21.15.26:0/3148932721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80abbe0 con 0x7fdce4005d50 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.672+0000 7fdcfe089640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdcf80ab6a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40018f0 tx=0x7fdcf8070e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.672+0000 7fdceeffd640 1 -- 172.21.15.26:0/3148932721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4045070 con 0x7fdce4005d50 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.672+0000 7fdceeffd640 1 -- 172.21.15.26:0/3148932721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf40049e0 con 0x7fdce4005d50 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.672+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf814fcb0 con 0x7fdce4005d50 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.672+0000 7fdceeffd640 1 -- 172.21.15.26:0/3148932721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4004d90 con 0x7fdce4005d50 2024-03-06T07:33:13.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.672+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf8150270 con 0x7fdce4005d50 2024-03-06T07:33:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.673+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf804f330 con 0x7fdce4005d50 2024-03-06T07:33:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.673+0000 7fdceeffd640 1 -- 172.21.15.26:0/3148932721 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcf4007070 con 0x7fdce4005d50 2024-03-06T07:33:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.674+0000 7fdceeffd640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.674+0000 7fdceeffd640 1 -- 172.21.15.26:0/3148932721 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fdcf40ab060 con 0x7fdce4005d50 2024-03-06T07:33:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.674+0000 7fdcfd888640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.674+0000 7fdcfd888640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdcf814c820 tx=0x7fdce8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.678+0000 7fdceeffd640 1 -- 172.21.15.26:0/3148932721 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdcf40797d0 con 0x7fdce4005d50 2024-03-06T07:33:13.805 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:13.807 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.861+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcf80c26b0 con 0x7fdcdc06b4d0 2024-03-06T07:33:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.861+0000 7fdceeffd640 1 -- 172.21.15.26:0/3148932721 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdcf80c26b0 con 0x7fdcdc06b4d0 2024-03-06T07:33:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 7 to deep-scrub 2024-03-06T07:33:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b4d0 msgr2=0x7fdcdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdcf814c820 tx=0x7fdce8009040 comp rx=0 tx=0).stop 2024-03-06T07:33:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 msgr2=0x7fdcf80ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdcf80ab6a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40018f0 tx=0x7fdcf8070e10 comp rx=0 tx=0).stop 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 shutdown_connections 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf8145cd0 0x7fdcf80c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf80bfc40 0x7fdcf8145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 --2- 172.21.15.26:0/3148932721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce4005d50 0x7fdcf80ab6a0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 >> 172.21.15.26:0/3148932721 conn(0x7fdcf806b1a0 msgr2=0x7fdcf814eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 shutdown_connections 2024-03-06T07:33:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:13.863+0000 7fdd002ed640 1 -- 172.21.15.26:0/3148932721 wait complete. 2024-03-06T07:33:13.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:14.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.016+0000 7f843f3bd640 1 Processor -- start 2024-03-06T07:33:14.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.016+0000 7f843f3bd640 1 -- start start 2024-03-06T07:33:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.016+0000 7f843f3bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843806da40 0x7f8438155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.016+0000 7f843f3bd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84380c6780 0x7f8438158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.016+0000 7f843f3bd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8438156050 con 0x7f84381592d0 2024-03-06T07:33:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.016+0000 7f843f3bd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843806de20 con 0x7f84380c6780 2024-03-06T07:33:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.016+0000 7f843f3bd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8438072630 con 0x7f843806da40 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843c958640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84380c6780 0x7f8438158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843c958640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84380c6780 0x7f8438158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36410/0 (socket says 172.21.15.26:36410) 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843c958640 1 -- 172.21.15.26:0/3162376878 learned_addr learned my addr 172.21.15.26:0/3162376878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843d159640 1 --2- 172.21.15.26:0/3162376878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843806da40 0x7f8438155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843c958640 1 -- 172.21.15.26:0/3162376878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843806da40 msgr2=0x7f8438155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843c958640 1 --2- 172.21.15.26:0/3162376878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843806da40 0x7f8438155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843c958640 1 -- 172.21.15.26:0/3162376878 >> v1:172.21.15.117:6789/0 conn(0x7f84381592d0 legacy=0x7f84380c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.017+0000 7f843c958640 1 -- 172.21.15.26:0/3162376878 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438072910 con 0x7f84380c6780 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f843c958640 1 --2- 172.21.15.26:0/3162376878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84380c6780 0x7f8438158d90 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f8428009870 tx=0x7f842802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886d3687527e01b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f842ffff640 1 -- 172.21.15.26:0/3162376878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428035020 con 0x7f84380c6780 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f842ffff640 1 -- 172.21.15.26:0/3162376878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428031640 con 0x7f84380c6780 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f842ffff640 1 -- 172.21.15.26:0/3162376878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842802d040 con 0x7f84380c6780 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f843f3bd640 1 -- 172.21.15.26:0/3162376878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84380c6780 msgr2=0x7f8438158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f843f3bd640 1 --2- 172.21.15.26:0/3162376878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84380c6780 0x7f8438158d90 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f8428009870 tx=0x7f842802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f843f3bd640 1 -- 172.21.15.26:0/3162376878 shutdown_connections 2024-03-06T07:33:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f843f3bd640 1 --2- 172.21.15.26:0/3162376878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84380c6780 0x7f8438158d90 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f843f3bd640 1 --2- 172.21.15.26:0/3162376878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843806da40 0x7f8438155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.018+0000 7f843f3bd640 1 -- 172.21.15.26:0/3162376878 >> 172.21.15.26:0/3162376878 conn(0x7f84380aa630 msgr2=0x7f84380aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.019+0000 7f843f3bd640 1 -- 172.21.15.26:0/3162376878 shutdown_connections 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.019+0000 7f843f3bd640 1 -- 172.21.15.26:0/3162376878 wait complete. 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 Processor -- start 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 -- start start 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843806da40 0x7f84380ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84380c6780 0x7f84380adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84381592d0 0x7f8438157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84380c3990 con 0x7f843806da40 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8438158890 con 0x7f84381592d0 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843f3bd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84380c3b10 con 0x7f84380c6780 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843d159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843806da40 0x7f84380ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843d159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843806da40 0x7f84380ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36416/0 (socket says 172.21.15.26:36416) 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.020+0000 7f843d159640 1 -- 172.21.15.26:0/2774769295 learned_addr learned my addr 172.21.15.26:0/2774769295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843d95a640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84381592d0 0x7f8438157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843c958640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84380c6780 0x7f84380adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843d159640 1 -- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84380c6780 msgr2=0x7f84380adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843d159640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84380c6780 0x7f84380adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843d159640 1 -- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84381592d0 msgr2=0x7f8438157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843d159640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84381592d0 0x7f8438157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843d159640 1 -- 172.21.15.26:0/2774769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843814beb0 con 0x7f843806da40 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843d159640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843806da40 0x7f84380ad400 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f843400d620 tx=0x7f843400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f842dffb640 1 -- 172.21.15.26:0/2774769295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400b620 con 0x7f843806da40 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f842dffb640 1 -- 172.21.15.26:0/2774769295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434019070 con 0x7f843806da40 2024-03-06T07:33:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f842dffb640 1 -- 172.21.15.26:0/2774769295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400ba00 con 0x7f843806da40 2024-03-06T07:33:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84380ae490 con 0x7f843806da40 2024-03-06T07:33:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.021+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84380ac450 con 0x7f843806da40 2024-03-06T07:33:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.022+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84380ac680 con 0x7f843806da40 2024-03-06T07:33:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.023+0000 7f842dffb640 1 -- 172.21.15.26:0/2774769295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f843400bba0 con 0x7f843806da40 2024-03-06T07:33:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.023+0000 7f842dffb640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f841c06b4d0 0x7f841c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.023+0000 7f842dffb640 1 -- 172.21.15.26:0/2774769295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f843408a4e0 con 0x7f843806da40 2024-03-06T07:33:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.023+0000 7f843c958640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f841c06b4d0 0x7f841c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.024+0000 7f843c958640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f841c06b4d0 0x7f841c06d990 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f8428036000 tx=0x7f842803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.028+0000 7f842dffb640 1 -- 172.21.15.26:0/2774769295 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8434059460 con 0x7f843806da40 2024-03-06T07:33:14.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.205+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f843814c370 con 0x7f841c06b4d0 2024-03-06T07:33:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.206+0000 7f842dffb640 1 -- 172.21.15.26:0/2774769295 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13524 (secure 0 0 0) 0x7f842803a420 con 0x7f841c06b4d0 2024-03-06T07:33:14.208 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:33:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f841c06b4d0 msgr2=0x7f841c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f841c06b4d0 0x7f841c06d990 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f8428036000 tx=0x7f842803c040 comp rx=0 tx=0).stop 2024-03-06T07:33:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843806da40 msgr2=0x7f84380ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843806da40 0x7f84380ad400 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f843400d620 tx=0x7f843400dae0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 shutdown_connections 2024-03-06T07:33:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f841c06b4d0 0x7f841c06d990 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84381592d0 0x7f8438157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84380c6780 0x7f84380adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 --2- 172.21.15.26:0/2774769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843806da40 0x7f84380ad400 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 >> 172.21.15.26:0/2774769295 conn(0x7f84380aa630 msgr2=0x7f84380c5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 shutdown_connections 2024-03-06T07:33:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:14.208+0000 7f843f3bd640 1 -- 172.21.15.26:0/2774769295 wait complete. 2024-03-06T07:33:14.220 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2024-03-06T07:33:13.212877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287900,"kb_used_data":21792,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754686820,"statfs":{"total":773094113280,"available":772799303680,"internally_reserved":0,"allocated":22315008,"data_stored":11951392,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-88,"num_read_kb":-94,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001981"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":542,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-03-06T07:33:06.330220+0000","last_change":"2024-03-06T07:19:57.108868+0000","last_active":"2024-03-06T07:33:06.330220+0000","last_peered":"2024-03-06T07:33:06.330220+0000","last_clean":"2024-03-06T07:33:06.330220+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:33:06.330220+0000","last_undegraded":"2024-03-06T07:33:06.330220+0000","last_fullsized":"2024-03-06T07:33:06.330220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:56.002056+0000","last_clean_scrub_stamp":"2024-03-06T07:19:56.002056+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:45.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33956,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337884,"statfs":{"total":96636764160,"available":96601993216,"internally_reserved":0,"allocated":2838528,"data_stored":1936700,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38048,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333792,"statfs":{"total":96636764160,"available":96597803008,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38048,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333792,"statfs":{"total":96636764160,"available":96597803008,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38056,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38116,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2838528,"data_stored":1936700,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:14.223 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-06T07:19:56.002056+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=6, tm_hour=7, tm_min=19, tm_sec=56, tm_wday=2, tm_yday=66, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=6, tm_hour=7, tm_min=33, tm_sec=9, tm_wday=2, tm_yday=66, tm_isdst=0) 2024-03-06T07:33:14.223 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-06T07:33:34.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:34.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.336+0000 7ff849be9640 1 Processor -- start 2024-03-06T07:33:34.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.336+0000 7ff849be9640 1 -- start start 2024-03-06T07:33:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.337+0000 7ff849be9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8440c88c0 0x7ff844155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.337+0000 7ff849be9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff844158b60 0x7ff844158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.337+0000 7ff849be9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8440c73d0 con 0x7ff84414f360 2024-03-06T07:33:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.337+0000 7ff849be9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440c8dd0 con 0x7ff844158b60 2024-03-06T07:33:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.337+0000 7ff849be9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440be2e0 con 0x7ff8440c88c0 2024-03-06T07:33:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.337+0000 7ff8437fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff84414f360 0x7ff8440c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37568/0 (socket says 172.21.15.26:37568) 2024-03-06T07:33:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.337+0000 7ff8437fe640 1 -- 172.21.15.26:0/3233463926 learned_addr learned my addr 172.21.15.26:0/3233463926 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff842ffd640 1 --2- 172.21.15.26:0/3233463926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8440c88c0 0x7ff844155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff843fff640 1 --2- 172.21.15.26:0/3233463926 >> v2:172.21.15.117:3300/0 conn(0x7ff844158b60 0x7ff844158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff8427fc640 1 -- 172.21.15.26:0/3233463926 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588063084 0 0) 0x7ff8440c73d0 con 0x7ff84414f360 2024-03-06T07:33:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff8427fc640 1 -- 172.21.15.26:0/3233463926 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff82c003520 con 0x7ff84414f360 2024-03-06T07:33:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff842ffd640 1 -- 172.21.15.26:0/3233463926 >> v2:172.21.15.117:3300/0 conn(0x7ff844158b60 msgr2=0x7ff844158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff842ffd640 1 --2- 172.21.15.26:0/3233463926 >> v2:172.21.15.117:3300/0 conn(0x7ff844158b60 0x7ff844158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff8427fc640 1 -- 172.21.15.26:0/3233463926 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2135061348 0 0) 0x7ff82c003520 con 0x7ff84414f360 2024-03-06T07:33:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff842ffd640 1 -- 172.21.15.26:0/3233463926 >> v1:172.21.15.26:6789/0 conn(0x7ff84414f360 legacy=0x7ff8440c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.338+0000 7ff842ffd640 1 -- 172.21.15.26:0/3233463926 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440c7670 con 0x7ff8440c88c0 2024-03-06T07:33:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.339+0000 7ff842ffd640 1 --2- 172.21.15.26:0/3233463926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8440c88c0 0x7ff844155c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8380097a0 tx=0x7ff83802b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9642b0b68fb6dded server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.339+0000 7ff8427fc640 1 -- 172.21.15.26:0/3233463926 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838037070 con 0x7ff8440c88c0 2024-03-06T07:33:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.339+0000 7ff8427fc640 1 -- 172.21.15.26:0/3233463926 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff838030510 con 0x7ff8440c88c0 2024-03-06T07:33:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.339+0000 7ff8427fc640 1 -- 172.21.15.26:0/3233463926 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834001670 con 0x7ff8440c88c0 2024-03-06T07:33:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.339+0000 7ff849be9640 1 -- 172.21.15.26:0/3233463926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8440c88c0 msgr2=0x7ff844155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.339+0000 7ff849be9640 1 --2- 172.21.15.26:0/3233463926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8440c88c0 0x7ff844155c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8380097a0 tx=0x7ff83802b710 comp rx=0 tx=0).stop 2024-03-06T07:33:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.340+0000 7ff849be9640 1 -- 172.21.15.26:0/3233463926 shutdown_connections 2024-03-06T07:33:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.340+0000 7ff849be9640 1 --2- 172.21.15.26:0/3233463926 >> v2:172.21.15.117:3300/0 conn(0x7ff844158b60 0x7ff844158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.340+0000 7ff849be9640 1 --2- 172.21.15.26:0/3233463926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8440c88c0 0x7ff844155c10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.340+0000 7ff849be9640 1 -- 172.21.15.26:0/3233463926 >> 172.21.15.26:0/3233463926 conn(0x7ff8440ac570 msgr2=0x7ff8440ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.340+0000 7ff849be9640 1 -- 172.21.15.26:0/3233463926 shutdown_connections 2024-03-06T07:33:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.340+0000 7ff849be9640 1 -- 172.21.15.26:0/3233463926 wait complete. 2024-03-06T07:33:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.341+0000 7ff849be9640 1 Processor -- start 2024-03-06T07:33:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.341+0000 7ff849be9640 1 -- start start 2024-03-06T07:33:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8440c88c0 0x7ff8440c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84414f360 0x7ff8440c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff844158b60 0x7ff8441ee200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440c5a50 con 0x7ff84414f360 2024-03-06T07:33:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440c7c30 con 0x7ff8440c88c0 2024-03-06T07:33:34.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440c5bd0 con 0x7ff844158b60 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84414f360 0x7ff8440c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84414f360 0x7ff8440c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54182/0 (socket says 172.21.15.26:54182) 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 -- 172.21.15.26:0/2217862210 learned_addr learned my addr 172.21.15.26:0/2217862210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff8437fe640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8440c88c0 0x7ff8440c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff843fff640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff844158b60 0x7ff8441ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 -- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff844158b60 msgr2=0x7ff8441ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff844158b60 0x7ff8441ee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 -- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8440c88c0 msgr2=0x7ff8440c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8440c88c0 0x7ff8440c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 -- 172.21.15.26:0/2217862210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844152230 con 0x7ff84414f360 2024-03-06T07:33:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff8437fe640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8440c88c0 0x7ff8440c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff842ffd640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84414f360 0x7ff8440c0cd0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7ff838000c00 tx=0x7ff8380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff82bfff640 1 -- 172.21.15.26:0/2217862210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838041020 con 0x7ff84414f360 2024-03-06T07:33:34.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff82bfff640 1 -- 172.21.15.26:0/2217862210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83803c870 con 0x7ff84414f360 2024-03-06T07:33:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8441eea40 con 0x7ff84414f360 2024-03-06T07:33:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff82bfff640 1 -- 172.21.15.26:0/2217862210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838037040 con 0x7ff84414f360 2024-03-06T07:33:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.342+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8441524a0 con 0x7ff84414f360 2024-03-06T07:33:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.344+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84404f330 con 0x7ff84414f360 2024-03-06T07:33:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.344+0000 7ff82bfff640 1 -- 172.21.15.26:0/2217862210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff838044020 con 0x7ff84414f360 2024-03-06T07:33:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.344+0000 7ff82bfff640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff83006b4d0 0x7ff83006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.344+0000 7ff82bfff640 1 -- 172.21.15.26:0/2217862210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7ff8380abb30 con 0x7ff84414f360 2024-03-06T07:33:34.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.344+0000 7ff8437fe640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff83006b4d0 0x7ff83006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.345+0000 7ff8437fe640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff83006b4d0 0x7ff83006d990 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff834007a00 tx=0x7ff834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.349+0000 7ff82bfff640 1 -- 172.21.15.26:0/2217862210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff83804b030 con 0x7ff84414f360 2024-03-06T07:33:34.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.525+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 --> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8440c0a90 con 0x7ff83006b4d0 2024-03-06T07:33:34.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.527+0000 7ff82bfff640 1 -- 172.21.15.26:0/2217862210 <== mgr.4105 v2:172.21.15.117:6832/3679442139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13463 (secure 0 0 0) 0x7ff8440c0a90 con 0x7ff83006b4d0 2024-03-06T07:33:34.528 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:34.528 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-03-06T07:33:34.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff83006b4d0 msgr2=0x7ff83006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff83006b4d0 0x7ff83006d990 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff834007a00 tx=0x7ff834009040 comp rx=0 tx=0).stop 2024-03-06T07:33:34.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84414f360 msgr2=0x7ff8440c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84414f360 0x7ff8440c0cd0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7ff838000c00 tx=0x7ff8380046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 shutdown_connections 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7ff83006b4d0 0x7ff83006d990 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff844158b60 0x7ff8441ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84414f360 0x7ff8440c0cd0 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 --2- 172.21.15.26:0/2217862210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8440c88c0 0x7ff8440c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 >> 172.21.15.26:0/2217862210 conn(0x7ff8440ac570 msgr2=0x7ff84406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 shutdown_connections 2024-03-06T07:33:34.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.528+0000 7ff849be9640 1 -- 172.21.15.26:0/2217862210 wait complete. 2024-03-06T07:33:34.540 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":620,"stamp":"2024-03-06T07:33:33.216874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287884,"kb_used_data":21792,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754686836,"statfs":{"total":773094113280,"available":772799320064,"internally_reserved":0,"allocated":22315008,"data_stored":11951392,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002312"},"pg_stats":[{"pgid":"1.0","version":"218'119","reported_seq":553,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-03-06T07:33:14.470648+0000","last_change":"2024-03-06T07:33:14.470688+0000","last_active":"2024-03-06T07:33:14.470648+0000","last_peered":"2024-03-06T07:33:14.470648+0000","last_clean":"2024-03-06T07:33:14.470648+0000","last_became_active":"2024-03-06T07:19:57.107898+0000","last_became_peered":"2024-03-06T07:19:57.107898+0000","last_unstale":"2024-03-06T07:33:14.470648+0000","last_undegraded":"2024-03-06T07:33:14.470648+0000","last_fullsized":"2024-03-06T07:33:14.470648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"218'119","last_scrub_stamp":"2024-03-06T07:33:14.470568+0000","last_deep_scrub":"218'119","last_deep_scrub_stamp":"2024-03-06T07:33:14.470568+0000","last_clean_scrub_stamp":"2024-03-06T07:33:14.470568+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:07:14.602999+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64944,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33948,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337892,"statfs":{"total":96636764160,"available":96602001408,"internally_reserved":0,"allocated":2838528,"data_stored":1936700,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38048,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333792,"statfs":{"total":96636764160,"available":96597803008,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38048,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333792,"statfs":{"total":96636764160,"available":96597803008,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38056,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2772992,"data_stored":1346332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38108,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333732,"statfs":{"total":96636764160,"available":96597741568,"internally_reserved":0,"allocated":2838528,"data_stored":1936700,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:34.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-06T07:33:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.641+0000 7fe1c0a64640 1 Processor -- start 2024-03-06T07:33:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.641+0000 7fe1c0a64640 1 -- start start 2024-03-06T07:33:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1c0a64640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe1bc1529f0 0x7fe1bc0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1c0a64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc0c57d0 0x7fe1bc155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1c0a64640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc152dd0 con 0x7fe1bc0c5370 2024-03-06T07:33:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1c0a64640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc159710 con 0x7fe1bc1529f0 2024-03-06T07:33:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1c0a64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc156180 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1ba59c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe1bc1529f0 0x7fe1bc0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1bad9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc0c57d0 0x7fe1bc155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.642+0000 7fe1b9d9b640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe1bc0c5370 0x7fe1bc155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.26:59378/0 (socket says 172.21.15.26:59378) 2024-03-06T07:33:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.643+0000 7fe1b9d9b640 1 -- 172.21.15.26:0/1152907273 learned_addr learned my addr 172.21.15.26:0/1152907273 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-03-06T07:33:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.643+0000 7fe1ba59c640 1 -- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc0c57d0 msgr2=0x7fe1bc155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.643+0000 7fe1ba59c640 1 --2- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc0c57d0 0x7fe1bc155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.643+0000 7fe1ba59c640 1 -- 172.21.15.26:0/1152907273 >> v1:172.21.15.117:6789/0 conn(0x7fe1bc0c5370 legacy=0x7fe1bc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.643+0000 7fe1ba59c640 1 -- 172.21.15.26:0/1152907273 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc158b00 con 0x7fe1bc1529f0 2024-03-06T07:33:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.643+0000 7fe1bad9d640 1 --2- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc0c57d0 0x7fe1bc155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.644+0000 7fe1ba59c640 1 --2- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc1529f0 0x7fe1bc0c88f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40097a0 tx=0x7fe1b402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2459c1586fe2d5ab server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.644+0000 7fe1b959a640 1 -- 172.21.15.26:0/1152907273 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40315a0 con 0x7fe1bc1529f0 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.644+0000 7fe1b959a640 1 -- 172.21.15.26:0/1152907273 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b4037070 con 0x7fe1bc1529f0 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.644+0000 7fe1b959a640 1 -- 172.21.15.26:0/1152907273 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4031960 con 0x7fe1bc1529f0 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.644+0000 7fe1c0a64640 1 -- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc1529f0 msgr2=0x7fe1bc0c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.644+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc1529f0 0x7fe1bc0c88f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40097a0 tx=0x7fe1b402b840 comp rx=0 tx=0).stop 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.645+0000 7fe1c0a64640 1 -- 172.21.15.26:0/1152907273 shutdown_connections 2024-03-06T07:33:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.645+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc0c57d0 0x7fe1bc155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.645+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/1152907273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc1529f0 0x7fe1bc0c88f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.645+0000 7fe1c0a64640 1 -- 172.21.15.26:0/1152907273 >> 172.21.15.26:0/1152907273 conn(0x7fe1bc0ac580 msgr2=0x7fe1bc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.645+0000 7fe1c0a64640 1 -- 172.21.15.26:0/1152907273 shutdown_connections 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.645+0000 7fe1c0a64640 1 -- 172.21.15.26:0/1152907273 wait complete. 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 Processor -- start 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 -- start start 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc0c5370 0x7fe1bc1eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1bc0c57d0 0x7fe1bc1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc1529f0 0x7fe1bc14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc1590c0 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0c68b0 con 0x7fe1bc0c5370 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.646+0000 7fe1c0a64640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc1593a0 con 0x7fe1bc1529f0 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1bc0c57d0 0x7fe1bc1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1bc0c57d0 0x7fe1bc1577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54196/0 (socket says 172.21.15.26:54196) 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 -- 172.21.15.26:0/2079124914 learned_addr learned my addr 172.21.15.26:0/2079124914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1ba59c640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc0c5370 0x7fe1bc1eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1bad9d640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc1529f0 0x7fe1bc14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 -- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc1529f0 msgr2=0x7fe1bc14dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc1529f0 0x7fe1bc14dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 -- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc0c5370 msgr2=0x7fe1bc1eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc0c5370 0x7fe1bc1eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 -- 172.21.15.26:0/2079124914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0bfd00 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b9d9b640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1bc0c57d0 0x7fe1bc1577b0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400c400 tx=0x7fe1a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4017070 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a4005ce0 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc0bffe0 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40072f0 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.647+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc0b0490 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.648+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc0c01b0 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.649+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1a401a070 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.649+0000 7fe1b2ffd640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b4d0 0x7fe19c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.649+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fe1a4089510 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.650+0000 7fe1ba59c640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b4d0 0x7fe19c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.650+0000 7fe1ba59c640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b4d0 0x7fe19c06d990 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4036000 tx=0x7fe1b403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.654+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1a4058490 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.833+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe1bc04f330 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.837+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v65)=0 v65) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe1a4057d90 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.837+0000 7fe1b2ffd640 1 -- 172.21.15.26:0/2079124914 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe1a405bee0 con 0x7fe1bc0c57d0 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b4d0 msgr2=0x7fe19c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b4d0 0x7fe19c06d990 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4036000 tx=0x7fe1b403d040 comp rx=0 tx=0).stop 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1bc0c57d0 msgr2=0x7fe1bc1577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1bc0c57d0 0x7fe1bc1577b0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400c400 tx=0x7fe1a400c8c0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 shutdown_connections 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7fe19c06b4d0 0x7fe19c06d990 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1bc1529f0 0x7fe1bc14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1bc0c57d0 0x7fe1bc1577b0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 --2- 172.21.15.26:0/2079124914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc0c5370 0x7fe1bc1eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 >> 172.21.15.26:0/2079124914 conn(0x7fe1bc0ac580 msgr2=0x7fe1bc14fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 shutdown_connections 2024-03-06T07:33:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.839+0000 7fe1c0a64640 1 -- 172.21.15.26:0/2079124914 wait complete. 2024-03-06T07:33:34.851 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:33:34.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.951+0000 7f56aea1c640 1 Processor -- start 2024-03-06T07:33:34.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.951+0000 7f56aea1c640 1 -- start start 2024-03-06T07:33:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56aea1c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56a8152920 0x7f56a8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56aea1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56aea1c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a8156180 con 0x7f56a8159460 2024-03-06T07:33:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56aea1c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a81587b0 con 0x7f56a8152920 2024-03-06T07:33:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56aea1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80c7760 con 0x7f56a80c8a00 2024-03-06T07:33:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56a8152920 0x7f56a8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56a8152920 0x7f56a8155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54202/0 (socket says 172.21.15.26:54202) 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a7fff640 1 -- 172.21.15.26:0/4167511146 learned_addr learned my addr 172.21.15.26:0/4167511146 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a77fe640 1 --2- 172.21.15.26:0/4167511146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a7fff640 1 -- 172.21.15.26:0/4167511146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 msgr2=0x7f56a8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a7fff640 1 --2- 172.21.15.26:0/4167511146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a8158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a7fff640 1 -- 172.21.15.26:0/4167511146 >> v1:172.21.15.117:6789/0 conn(0x7f56a8159460 legacy=0x7f56a80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a7fff640 1 -- 172.21.15.26:0/4167511146 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a80c7a40 con 0x7f56a8152920 2024-03-06T07:33:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.952+0000 7f56a77fe640 1 --2- 172.21.15.26:0/4167511146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.953+0000 7f56a7fff640 1 --2- 172.21.15.26:0/4167511146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a8155c40 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f569c009870 tx=0x7f569c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f1553b9662dd84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.953+0000 7f56a6ffd640 1 -- 172.21.15.26:0/4167511146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c0317c0 con 0x7f56a8152920 2024-03-06T07:33:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.953+0000 7f56a6ffd640 1 -- 172.21.15.26:0/4167511146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f569c037070 con 0x7f56a8152920 2024-03-06T07:33:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.953+0000 7f56a6ffd640 1 -- 172.21.15.26:0/4167511146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c031ba0 con 0x7f56a8152920 2024-03-06T07:33:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 -- 172.21.15.26:0/4167511146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 msgr2=0x7f56a8155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 --2- 172.21.15.26:0/4167511146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a8155c40 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f569c009870 tx=0x7f569c02b860 comp rx=0 tx=0).stop 2024-03-06T07:33:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 -- 172.21.15.26:0/4167511146 shutdown_connections 2024-03-06T07:33:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 --2- 172.21.15.26:0/4167511146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 --2- 172.21.15.26:0/4167511146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a8155c40 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 -- 172.21.15.26:0/4167511146 >> 172.21.15.26:0/4167511146 conn(0x7f56a80ac560 msgr2=0x7f56a80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 -- 172.21.15.26:0/4167511146 shutdown_connections 2024-03-06T07:33:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.954+0000 7f56aea1c640 1 -- 172.21.15.26:0/4167511146 wait complete. 2024-03-06T07:33:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.955+0000 7f56aea1c640 1 Processor -- start 2024-03-06T07:33:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.955+0000 7f56aea1c640 1 -- start start 2024-03-06T07:33:34.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56aea1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a81e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56aea1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a80afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56aea1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a81e1420 0x7f56a814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56aea1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a806d030 con 0x7f56a8152920 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56aea1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8158a00 con 0x7f56a81e1420 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56aea1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80c5a40 con 0x7f56a80c8a00 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a80afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a80afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54218/0 (socket says 172.21.15.26:54218) 2024-03-06T07:33:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 -- 172.21.15.26:0/1966391683 learned_addr learned my addr 172.21.15.26:0/1966391683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56acfb9640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a81e1420 0x7f56a814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a7fff640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a81e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 -- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 msgr2=0x7f56a81e0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a81e0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 -- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a81e1420 msgr2=0x7f56a814bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a81e1420 0x7f56a814bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 -- 172.21.15.26:0/1966391683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a80ae0e0 con 0x7f56a8152920 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a77fe640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a80afec0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f569800c820 tx=0x7f569800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.956+0000 7f56a4ff9640 1 -- 172.21.15.26:0/1966391683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698015070 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.957+0000 7f56a4ff9640 1 -- 172.21.15.26:0/1966391683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5698007a30 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.957+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a80ae3c0 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.957+0000 7f56a4ff9640 1 -- 172.21.15.26:0/1966391683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698007e00 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.957+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a80b0400 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.958+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a804f330 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.958+0000 7f56a4ff9640 1 -- 172.21.15.26:0/1966391683 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5698018070 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.959+0000 7f56a4ff9640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f569406b500 0x7f569406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.959+0000 7f56a4ff9640 1 -- 172.21.15.26:0/1966391683 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f569808a030 con 0x7f56a8152920 2024-03-06T07:33:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.959+0000 7f56a7fff640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f569406b500 0x7f569406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.959+0000 7f56a7fff640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f569406b500 0x7f569406d9c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f569c036000 tx=0x7f569c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:34.963+0000 7f56a4ff9640 1 -- 172.21.15.26:0/1966391683 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5698058fb0 con 0x7f56a8152920 2024-03-06T07:33:35.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.183+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56a806d840 con 0x7f56a8152920 2024-03-06T07:33:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.183+0000 7f56a4ff9640 1 -- 172.21.15.26:0/1966391683 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+1329 (secure 0 0 0) 0x7f56980588b0 con 0x7f56a8152920 2024-03-06T07:33:35.186 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-06T07:33:35.186 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":217,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11817,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3909830036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3909830036},{"type":"v1","addr":"172.21.15.26:6835","nonce":3909830036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":217}],"filesystems":[]} 2024-03-06T07:33:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 217 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.185+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f569406b500 msgr2=0x7f569406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.185+0000 7f56aea1c640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f569406b500 0x7f569406d9c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f569c036000 tx=0x7f569c03d040 comp rx=0 tx=0).stop 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.185+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 msgr2=0x7f56a80afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.185+0000 7f56aea1c640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a80afec0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f569800c820 tx=0x7f569800cce0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 shutdown_connections 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:6832/3679442139,v1:172.21.15.117:6833/3679442139] conn(0x7f569406b500 0x7f569406d9c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a81e1420 0x7f56a814bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a8152920 0x7f56a80afec0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 --2- 172.21.15.26:0/1966391683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a80c8a00 0x7f56a81e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 >> 172.21.15.26:0/1966391683 conn(0x7f56a80ac560 msgr2=0x7f56a80caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 shutdown_connections 2024-03-06T07:33:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-06T07:33:35.186+0000 7f56aea1c640 1 -- 172.21.15.26:0/1966391683 wait complete. 2024-03-06T07:33:35.199 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-06T07:33:35.199 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-06T07:33:35.199 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:33:35.199 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.208 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:33:35.209 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:33:35.209 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.257 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:33:35.258 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:33:35.258 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.270 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:33:35.270 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:33:35.270 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.322 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:33:35.322 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-06T07:33:35.322 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-06T07:33:35.322 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.356 INFO:tasks.ceph.osd.0:Stopped 2024-03-06T07:33:35.356 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-06T07:33:35.356 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.422 INFO:tasks.ceph.osd.1:Stopped 2024-03-06T07:33:35.422 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-06T07:33:35.422 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.492 INFO:tasks.ceph.osd.2:Stopped 2024-03-06T07:33:35.493 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-06T07:33:35.493 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.561 INFO:tasks.ceph.osd.3:Stopped 2024-03-06T07:33:35.561 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-06T07:33:35.561 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.596 INFO:tasks.ceph.osd.4:Stopped 2024-03-06T07:33:35.596 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-06T07:33:35.596 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.664 INFO:tasks.ceph.osd.5:Stopped 2024-03-06T07:33:35.664 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.730 INFO:tasks.ceph.osd.6:Stopped 2024-03-06T07:33:35.731 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-06T07:33:35.731 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.799 INFO:tasks.ceph.osd.7:Stopped 2024-03-06T07:33:35.800 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-06T07:33:35.800 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-06T07:33:35.800 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.825 INFO:tasks.ceph.mgr.y:Stopped 2024-03-06T07:33:35.825 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-06T07:33:35.825 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.869 INFO:tasks.ceph.mgr.x:Stopped 2024-03-06T07:33:35.869 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-06T07:33:35.870 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-06T07:33:35.870 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.881 INFO:tasks.ceph.mon.a:Stopped 2024-03-06T07:33:35.882 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-06T07:33:35.882 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.922 INFO:tasks.ceph.mon.b:Stopped 2024-03-06T07:33:35.922 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-06T07:33:35.922 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:35.975 INFO:tasks.ceph.mon.c:Stopped 2024-03-06T07:33:35.975 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-06T07:33:35.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v '\(MDS_INSUFFICIENT_STANDBY\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(FS_DEGRADED\)' | head -n 1 2024-03-06T07:33:36.012 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:33:36.012 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-06T07:33:36.202 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:33:36.203 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-06T07:33:36.416 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:33:36.416 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-06T07:33:36.466 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:33:36.466 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-06T07:33:36.525 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:33:36.525 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-06T07:33:36.681 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:33:36.681 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-06T07:33:36.770 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:33:36.771 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-06T07:33:36.829 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:33:36.829 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-06T07:33:36.886 INFO:tasks.ceph:Archiving mon data... 2024-03-06T07:33:36.887 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/data/mon.a.tgz 2024-03-06T07:33:36.887 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-03-06T07:33:36.903 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.ynvT2cgPgc 2024-03-06T07:33:36.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ynvT2cgPgc 2024-03-06T07:33:37.214 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.ynvT2cgPgc 2024-03-06T07:33:37.299 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.ynvT2cgPgc is 556KB 2024-03-06T07:33:37.398 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.ynvT2cgPgc 2024-03-06T07:33:37.412 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/data/mon.b.tgz 2024-03-06T07:33:37.412 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-06T07:33:37.427 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.xhOAsLl4GF 2024-03-06T07:33:37.428 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xhOAsLl4GF 2024-03-06T07:33:37.645 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.xhOAsLl4GF 2024-03-06T07:33:37.729 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.xhOAsLl4GF is 572KB 2024-03-06T07:33:37.825 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.xhOAsLl4GF 2024-03-06T07:33:37.839 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/data/mon.c.tgz 2024-03-06T07:33:37.839 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-06T07:33:37.892 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.7KkjkdLBUy 2024-03-06T07:33:37.892 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.7KkjkdLBUy 2024-03-06T07:33:38.071 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.7KkjkdLBUy 2024-03-06T07:33:38.155 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.7KkjkdLBUy is 572KB 2024-03-06T07:33:38.252 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.7KkjkdLBUy 2024-03-06T07:33:38.266 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-06T07:33:38.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-06T07:33:38.269 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-06T07:33:38.352 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-06T07:33:38.388 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-06T07:33:38.388 INFO:tasks.ceph:Archiving crash dumps... 2024-03-06T07:33:38.390 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/remote/smithi026/crash 2024-03-06T07:33:38.390 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-06T07:33:38.421 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/remote/smithi117/crash 2024-03-06T07:33:38.422 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-06T07:33:38.454 INFO:tasks.ceph:Shutting down logrotate 2024-03-06T07:33:38.454 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:38.466 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:38.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:38.576 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:38.602 INFO:tasks.ceph:Compressing logs... 2024-03-06T07:33:38.602 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-06T07:33:38.618 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-06T07:33:38.684 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-06T07:33:38.684 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37722.log 2024-03-06T07:33:38.684 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37601.log 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37722.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37601.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37722.log.gz 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42153.log 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.1.42153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42153.log.gz 2024-03-06T07:33:38.685 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37762.log 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37842.log 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37762.log.gz 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mgr.x.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37882.log 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37842.log.gz 2024-03-06T07:33:38.686 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-06T07:33:38.687 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37882.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37882.log.gz 2024-03-06T07:33:38.687 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-03-06T07:33:38.687 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mon.b.log: 92.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-06T07:33:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-06T07:33:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37641.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-03-06T07:33:38.709 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37802.log 2024-03-06T07:33:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-06T07:33:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37802.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37802.log.gz 2024-03-06T07:33:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37682.log 2024-03-06T07:33:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.37682.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37682.log.gz 2024-03-06T07:33:38.761 INFO:teuthology.orchestra.run.smithi117.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-06T07:33:38.822 INFO:teuthology.orchestra.run.smithi117.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-06T07:33:38.876 INFO:teuthology.orchestra.run.smithi117.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-06T07:33:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47750.log 2024-03-06T07:33:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66714.log 2024-03-06T07:33:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61891.log 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47750.log: /var/log/ceph/ceph-client.admin.66714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67045.log 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47750.log.gz 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61891.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66714.log.gz 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48710.log 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49812.log 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67045.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61891.log.gz 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62132.log 2024-03-06T07:33:39.052 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67045.log.gz 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49812.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.48710.log.gz -5 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.71862.log 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0%/var/log/ceph/ceph-client.admin.62132.log: -- replaced with /var/log/ceph/ceph-client.admin.49812.log.gz 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50106.log 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62132.log.gz 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70625.log 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50106.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71862.log.gz -5 2024-03-06T07:33:39.053 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.84189.log 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42611.log 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50106.log.gz 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84189.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70625.log.gz 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84020.log 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84189.log.gz 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57128.log 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84020.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42611.log.gz 2024-03-06T07:33:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64557.log 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84020.log.gz 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62481.log 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62174.log 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57128.log.gz 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62481.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64557.log.gz 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56728.log 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62481.log.gz 2024-03-06T07:33:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83454.log 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50869.log 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62174.log.gz 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56728.log.gz 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80764.log 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50869.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83454.log.gz 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80764.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50869.log.gz 2024-03-06T07:33:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55778.log 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80764.log.gz 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46655.log 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58122.log 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55778.log.gz 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46655.log: /var/log/ceph/ceph-client.admin.58122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88646.log 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46655.log.gz 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48080.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58122.log.gz 2024-03-06T07:33:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49896.log 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88730.log 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88646.log: /var/log/ceph/ceph-client.admin.49896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88646.log.gz 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49896.log.gz 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83978.log 2024-03-06T07:33:39.058 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88730.log.gz 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43810.log: 90.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz -5 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66840.log 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58080.log 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83978.log.gz 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66840.log: /var/log/ceph/ceph-client.admin.58080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66840.log.gz 2024-03-06T07:33:39.059 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58080.log.gz 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63843.log 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59503.log 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63843.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz -- 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.63487.log 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63843.log.gz 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66420.log 2024-03-06T07:33:39.060 INFO:teuthology.orchestra.run.smithi026.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59503.log.gz 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61135.log 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66420.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63487.log.gz 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60252.log 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66420.log.gz 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80105.log 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61135.log.gz 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82836.log 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80105.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60252.log.gz 2024-03-06T07:33:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56200.log 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82836.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80105.log.gz 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63445.log 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82836.log.gz 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67376.log 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63445.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56200.log.gz 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88230.log 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63445.log.gz 2024-03-06T07:33:39.062 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71270.log 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88230.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67376.log.gz 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72488.log 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71270.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88230.log.gz 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71270.log.gz 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79329.log 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72488.log.gz 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47996.log 2024-03-06T07:33:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79329.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64431.log 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79329.log.gz 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47834.log 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47996.log.gz 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45316.log 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47834.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64431.log.gz 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44098.log 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47834.log.gz 2024-03-06T07:33:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58038.log 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45316.log.gz 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62611.log 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58038.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44098.log.gz 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75958.log 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58038.log.gz 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86034.log 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62611.log.gz 2024-03-06T07:33:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71120.log 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6%/var/log/ceph/ceph-client.admin.86034.log: -- replaced with /var/log/ceph/ceph-client.admin.75958.log.gz 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86496.log 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71120.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86034.log.gz 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53399.log 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71120.log.gz 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67456.log 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53399.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86496.log.gz 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78248.log 2024-03-06T07:33:39.066 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53399.log.gz 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51289.log 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67456.log.gz 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58500.log 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51289.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.78248.log.gz --verbose 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.79822.log 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58500.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51289.log.gz 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80614.log 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58500.log.gz 2024-03-06T07:33:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85722.log 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80614.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79822.log.gz 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60864.log 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85722.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80614.log.gz 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82752.log 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60864.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85722.log.gz 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46991.log 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61177.log 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46991.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82752.log.gz 2024-03-06T07:33:39.068 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60864.log.gz 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89751.log 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46991.log.gz 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89751.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61177.log.gz 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56956.log 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42569.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89751.log.gz 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52317.log 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56956.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-03-06T07:33:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67664.log 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56956.log.gz 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56028.log 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52317.log.gz 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66672.log 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56028.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67664.log.gz 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48458.log 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66672.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56028.log.gz 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61681.log 2024-03-06T07:33:39.070 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66672.log.gz 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70976.log 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61681.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48458.log.gz 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76397.log 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61681.log.gz 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70976.log.gz 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86454.log 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44014.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76397.log.gz 2024-03-06T07:33:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59870.log 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55325.log 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86454.log.gz/var/log/ceph/ceph-client.admin.59870.log: 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42058.log 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55325.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59870.log.gz 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89625.log 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42058.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55325.log.gz 2024-03-06T07:33:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64011.log 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89625.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42058.log.gz 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54466.log 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64011.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/ceph-client.admin.89625.log.gz /var/log/ceph/ceph-client.admin.62565.log 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54466.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64011.log.gz 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46781.log 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54466.log.gz 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54130.log 2024-03-06T07:33:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46781.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62565.log.gz 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74251.log 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46781.log.gz 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69183.log 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54130.log.gz 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69183.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74251.log.gz 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46529.log 2024-03-06T07:33:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49686.log 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46529.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69183.log.gz 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74600.log 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4%/var/log/ceph/ceph-client.admin.49686.log: -- replaced with /var/log/ceph/ceph-client.admin.46529.log.gz 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59377.log 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74600.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49686.log.gz 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45190.log 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74600.log.gz 2024-03-06T07:33:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76808.log 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45190.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59377.log.gz 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54905.log 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45190.log.gz 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46739.log 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76808.log.gz 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63614.log 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46739.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54905.log.gz 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84987.log 2024-03-06T07:33:39.076 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46739.log.gz 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87666.log 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84987.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63614.log.gz 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85806.log 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87666.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84987.log.gz 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87666.log.gz 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79245.log 2024-03-06T07:33:39.077 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85806.log.gz 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54028.log 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79245.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86892.log 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2%/var/log/ceph/ceph-client.admin.54028.log: -- replaced with /var/log/ceph/ceph-client.admin.79245.log.gz 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74898.log 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5%/var/log/ceph/ceph-client.admin.86892.log: -- replaced with /var/log/ceph/ceph-client.admin.54028.log.gz 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76607.log 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51908.log 2024-03-06T07:33:39.078 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86892.log.gz 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76607.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74898.log.gz 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90087.log 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76607.log.gz 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70539.log 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90087.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51908.log.gz 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68503.log 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90087.log.gz 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60630.log 2024-03-06T07:33:39.079 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70539.log.gz 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41891.log 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54382.log 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68503.log.gz 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41891.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60630.log.gz 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73045.log 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41891.log.gz 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85158.log 2024-03-06T07:33:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54382.log.gz 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60462.log 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73045.log.gz 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59041.log 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60462.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85158.log.gz 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54695.log 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59041.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60462.log.gz 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66985.log 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59041.log.gz 2024-03-06T07:33:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46487.log 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66985.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54695.log.gz 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66985.log.gz 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81720.log 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45232.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.46487.log.gz 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48752.log 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81720.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-03-06T07:33:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89024.log 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48752.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81720.log.gz 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72740.log 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89024.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48752.log.gz 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47792.log 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72740.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89024.log.gz 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51037.log 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72740.log.gz 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77230.log 2024-03-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47792.log: /var/log/ceph/ceph-client.admin.51037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64221.log 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47792.log.gz 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77230.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51037.log.gz 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69286.log 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77230.log.gz 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73213.log 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69286.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64221.log.gz 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73003.log 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69286.log.gz 2024-03-06T07:33:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73213.log.gz 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56116.log 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53441.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.73003.log.gz --verbose 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.53273.log 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56116.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71778.log 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56116.log.gz 2024-03-06T07:33:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79100.log 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71778.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53273.log.gz 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44056.log 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79100.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71778.log.gz 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69790.log 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79100.log.gz 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58458.log 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69790.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44056.log.gz 2024-03-06T07:33:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49560.log 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58458.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69790.log.gz 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45748.log 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58458.log.gz 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49560.log: /var/log/ceph/ceph-client.admin.52076.log 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56686.log 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49560.log.gz 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46865.log 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52076.log.gz 2024-03-06T07:33:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56686.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45748.log.gz 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52531.log 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6%/var/log/ceph/ceph-client.admin.46865.log: -- replaced with /var/log/ceph/ceph-client.admin.56686.log.gz 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70248.log 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52531.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46865.log.gz 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81678.log 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52531.log.gz 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44140.log 2024-03-06T07:33:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81678.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70248.log.gz 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68587.log 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44140.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81678.log.gz 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49980.log 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44140.log.gz 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83496.log 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68587.log.gz 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48164.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49980.log.gz 2024-03-06T07:33:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73758.log 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83936.log 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83496.log.gz 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73758.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66546.log 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73758.log.gz 2024-03-06T07:33:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83936.log.gz 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60126.log 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87186.log 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66546.log.gz 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60126.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gz 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89415.log 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60126.log.gz 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75174.log 2024-03-06T07:33:39.091 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87186.log.gz 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64882.log 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77102.log 88.5% 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89415.log.gz 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64882.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75174.log.gz 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42443.log 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64882.log.gz 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70452.log 2024-03-06T07:33:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42443.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77102.log.gz 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49602.log 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38384.log 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42443.log.gz 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70452.log.gz 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51205.log 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38384.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49602.log.gz 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65463.log 2024-03-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1%/var/log/ceph/ceph-client.admin.51205.log: -- replaced with /var/log/ceph/ceph-client.admin.38384.log.gz 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70581.log 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51205.log.gz 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80012.log 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65463.log: /var/log/ceph/ceph-client.admin.70581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79413.log 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65463.log.gz 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80012.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70581.log.gz 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76252.log 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80012.log.gz 2024-03-06T07:33:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81510.log 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59251.log 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79413.log.gz 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81510.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.76252.log.gz 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52405.log 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81510.log.gz 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83851.log 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52405.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59251.log.gz 2024-03-06T07:33:39.095 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81930.log 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83851.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52405.log.gzgzip 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50574.log 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9%/var/log/ceph/ceph-client.admin.81930.log: -- replaced with /var/log/ceph/ceph-client.admin.83851.log.gz 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76000.log 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50574.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81930.log.gz 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72656.log 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76000.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50574.log.gz 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83286.log 2024-03-06T07:33:39.096 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76000.log.gz 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62821.log 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83286.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72656.log.gz 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80927.log 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62821.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83286.log.gz 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75420.log 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62821.log.gz 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80488.log 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75420.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.80927.log.gz 2024-03-06T07:33:39.097 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65050.log 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80488.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75420.log.gz 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50995.log 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80488.log.gz 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84411.log 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48416.log 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65050.log.gz 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84411.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50995.log.gz 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78890.log 2024-03-06T07:33:39.098 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84411.log.gz 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81594.log 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78890.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.48416.log.gz /var/log/ceph/ceph-client.admin.68629.log 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81594.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78890.log.gz 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74335.log 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81594.log.gz 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62523.log 2024-03-06T07:33:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74335.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68629.log.gz 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59828.log 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74335.log.gz 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77800.log 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62523.log.gz 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86328.log 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49346.log 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59828.log.gz 2024-03-06T07:33:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86328.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77800.log.gz 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51826.log 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86328.log.gz 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89169.log 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51826.log: gzip -5 --verbose 88.8% -- /var/log/ceph/ceph-client.admin.80063.log -- replaced with /var/log/ceph/ceph-client.admin.49346.log.gz 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89169.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51826.log.gz 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57645.log 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89169.log.gz 2024-03-06T07:33:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47411.log 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57645.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80063.log.gz 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49262.log 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47411.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57645.log.gz 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47411.log.gz 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43684.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49262.log.gz 2024-03-06T07:33:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70495.log 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81216.log 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81300.log 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81216.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70495.log.gz 2024-03-06T07:33:39.103 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80446.log 2024-03-06T07:33:39.104 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81216.log.gz 2024-03-06T07:33:39.104 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75342.log 2024-03-06T07:33:39.104 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81300.log.gz 2024-03-06T07:33:39.104 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77716.log 2024-03-06T07:33:39.104 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75342.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80446.log.gz 2024-03-06T07:33:39.104 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69496.log 2024-03-06T07:33:39.104 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75342.log.gz 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81426.log 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69496.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.50316.log 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77716.log.gz 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69496.log.gz 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78722.log 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50316.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.81426.log.gz 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55073.log 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-03-06T07:33:39.105 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78722.log.gz 2024-03-06T07:33:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55073.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50316.log.gz 2024-03-06T07:33:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55577.log 2024-03-06T07:33:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85932.log 2024-03-06T07:33:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47327.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55073.log.gz 2024-03-06T07:33:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65739.log 2024-03-06T07:33:39.106 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-03-06T07:33:39.106 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55577.log.gz 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62779.log 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85932.log.gz 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79203.log 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62779.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65739.log.gz 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52891.log 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79203.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62779.log.gz 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56494.log 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65589.log 2024-03-06T07:33:39.107 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79203.log.gz 2024-03-06T07:33:39.108 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52891.log.gz 2024-03-06T07:33:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2024-03-06T07:33:39.108 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56494.log.gz 2024-03-06T07:33:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67129.log 2024-03-06T07:33:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50064.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65589.log.gz 2024-03-06T07:33:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83809.log 2024-03-06T07:33:39.112 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7%/var/log/ceph/ceph-client.admin.67129.log: -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz 2024-03-06T07:33:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48542.log 2024-03-06T07:33:39.112 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67129.log.gz 2024-03-06T07:33:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53945.log 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76355.log 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83809.log.gz 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53945.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48542.log.gz 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74377.log 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53945.log.gz 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84598.log 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79924.log 2024-03-06T07:33:39.113 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76355.log.gz 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84598.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74377.log.gz 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42359.log 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84598.log.gz 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57040.log 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42359.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79924.log.gz 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49476.log 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57040.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42359.log.gz 2024-03-06T07:33:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85512.log 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57040.log.gz 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75874.log 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85512.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49476.log.gz 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56072.log 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75874.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85512.log.gz 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50256.log 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75874.log.gz 2024-03-06T07:33:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77944.log 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50256.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56072.log.gz 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59167.log 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4%/var/log/ceph/ceph-client.admin.77944.log: -- replaced with /var/log/ceph/ceph-client.admin.50256.log.gz 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77314.log 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59167.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.75753.log 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77944.log.gz 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77314.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59167.log.gz 2024-03-06T07:33:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88002.log 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77314.log.gz 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51079.log 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75753.log.gz 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71904.log 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88002.log.gz 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90003.log 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71904.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51079.log.gz 2024-03-06T07:33:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57729.log 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71904.log.gz 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79160.log 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90003.log.gz 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76168.log 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79160.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.57729.log.gz 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54989.log 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79160.log.gz 2024-03-06T07:33:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84231.log 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76168.log.gz 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66462.log 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84231.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54989.log.gz 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46321.log 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84231.log.gz 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82339.log 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66462.log.gz 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50466.log 2024-03-06T07:33:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82339.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46321.log.gz 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48038.log 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82339.log.gz 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85386.log 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44482.log 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85386.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48038.log.gz 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56452.log 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50466.log.gz 2024-03-06T07:33:39.120 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85386.log.gz 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63529.log 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56452.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44482.log.gz 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56452.log.gz 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86118.log 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56998.log 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86118.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87708.log 2024-03-06T07:33:39.121 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63529.log.gz 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86118.log.gz 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74688.log 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56998.log.gz 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56854.log 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52849.log 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87708.log.gz 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56854.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74688.log.gz 2024-03-06T07:33:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63801.log 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56854.log.gz 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85974.log 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52849.log.gz 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55860.log 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63801.log.gz 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47033.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85974.log.gz 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70168.log 2024-03-06T07:33:39.123 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68798.log 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61387.log 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55860.log.gz 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70168.log.gz/var/log/ceph/ceph-client.admin.68798.log: 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72824.log 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81888.log 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68798.log.gz 2024-03-06T07:33:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72824.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61387.log.gz 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82986.log 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71018.log 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5%/var/log/ceph/ceph-client.admin.81888.log: -- replaced with /var/log/ceph/ceph-client.admin.72824.log.gz 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88356.log 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81888.log.gz 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71018.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82986.log.gz 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47708.log 2024-03-06T07:33:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88356.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71018.log.gz 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55535.log 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47708.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88356.log.gz 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45274.log 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47708.log.gz 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79016.log 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55535.log: /var/log/ceph/ceph-client.admin.45274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80722.log 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55535.log.gz 2024-03-06T07:33:39.126 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45274.log.gz 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83412.log 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79016.log.gz 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44823.log 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88128.log 88.3% 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80722.log.gz 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44823.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83412.log.gz 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65207.log 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44823.log.gz 2024-03-06T07:33:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86370.log 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75505.log 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88128.log.gz 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86370.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65207.log.gz 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60168.log 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86370.log.gz 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44826.log 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63759.log 2024-03-06T07:33:39.128 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75505.log.gz 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44826.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60168.log.gz 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67750.log 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44826.log.gz 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89541.log 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67750.log: gzip 87.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.63759.log.gz -- 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.66756.log 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89541.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67750.log.gz 2024-03-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67622.log 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89541.log.gz 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53525.log 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66756.log: /var/log/ceph/ceph-client.admin.67622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73529.log 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66756.log.gz 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53525.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.67622.log.gz /var/log/ceph/ceph-client.admin.86412.log 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73529.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53525.log.gz 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58645.log 2024-03-06T07:33:39.130 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73529.log.gz 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41966.log 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58645.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.86412.log.gz 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65889.log 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41966.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58645.log.gz 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68756.log 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41966.log.gz 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78028.log 2024-03-06T07:33:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71438.log 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65889.log.gz 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78028.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68756.log.gz 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46403.log 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78028.log.gz 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61429.log 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88272.log 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71438.log.gz 2024-03-06T07:33:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61429.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46403.log.gz 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72950.log 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61429.log.gz 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73572.log 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47285.log 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88272.log.gz 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73572.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72950.log.gz 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71354.log 2024-03-06T07:33:39.133 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73572.log.gz 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75557.log 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71820.log 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47285.log.gz 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71354.log.gz 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58981.log 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75557.log.gz 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82255.log 2024-03-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73968.log 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71820.log.gz 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82255.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58981.log.gz 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83539.log 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82255.log.gz 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78468.log 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58897.log 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73968.log.gz 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78468.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83539.log.gz 2024-03-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71694.log 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78468.log.gz 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70892.log 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89793.log 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58897.log.gz 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70892.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71694.log.gz 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78196.log 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70892.log.gz 2024-03-06T07:33:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47954.log 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89793.log.gz 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79656.log 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47954.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78196.log.gz 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54947.log 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47954.log.gz 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71652.log 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66275.log 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79656.log.gz 2024-03-06T07:33:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71652.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54947.log.gz 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74646.log 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66275.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71652.log.gz 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88524.log 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66275.log.gz 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74646.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88524.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74646.log.gz 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54172.log 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.88524.log.gz 2024-03-06T07:33:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79576.log 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84369.log 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54172.log.gz 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74772.log 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6%/var/log/ceph/ceph-client.admin.84369.log: -- replaced with /var/log/ceph/ceph-client.admin.79576.log.gz 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42527.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.84369.log.gz 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53806.log 2024-03-06T07:33:39.139 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86808.log 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74772.log: /var/log/ceph/ceph-client.admin.53806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55157.log 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74772.log.gz 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53806.log.gz 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65631.log 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69580.log 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55157.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86808.log.gz 2024-03-06T07:33:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65631.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.55157.log.gz -5 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46571.log 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47912.log 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69580.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65631.log.gz 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66111.log 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69580.log.gz 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46571.log.gz -- 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.48122.log 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47912.log: /var/log/ceph/ceph-client.admin.66111.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47912.log.gz 2024-03-06T07:33:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89373.log 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63656.log 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48122.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66111.log.gz 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89373.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48122.log.gz 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87438.log 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52489.log 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63656.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89373.log.gz 2024-03-06T07:33:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51499.log 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52489.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87438.log.gz 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61345.log 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63656.log.gz 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52489.log.gz 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79058.log 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61345.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51499.log.gz 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50682.log 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61345.log.gz 2024-03-06T07:33:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58290.log 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54610.log 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79058.log.gz 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58290.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50682.log.gz 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55367.log 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58290.log.gz 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62012.log 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55367.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.54610.log.gz -5 2024-03-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72404.log 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62012.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55367.log.gz 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57996.log 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62012.log.gz 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61303.log 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72404.log.gz 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68311.log 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61303.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57996.log.gz 2024-03-06T07:33:39.145 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76565.log 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68311.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61303.log.gz 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86160.log 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4%/var/log/ceph/ceph-client.admin.76565.log: -- replaced with /var/log/ceph/ceph-client.admin.68311.log.gz 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70084.log 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76565.log.gz 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87018.log 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70084.log: gzip -5 89.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.86160.log.gz -- 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.69141.log 2024-03-06T07:33:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87018.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70084.log.gz 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49177.log 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87018.log.gz 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53483.log 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69141.log.gz 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56578.log 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49177.log.gz 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54863.log 2024-03-06T07:33:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56578.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55902.log 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68227.log 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55902.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54863.log.gz 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56578.log.gz 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73171.log 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55902.log.gz 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76932.log 2024-03-06T07:33:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73171.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68227.log.gz 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44266.log 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1%/var/log/ceph/ceph-client.admin.76932.log: -- replaced with /var/log/ceph/ceph-client.admin.73171.log.gz 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82089.log 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44266.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.76932.log.gz 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56770.log 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82089.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44266.log.gz 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66798.log 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82089.log.gz 2024-03-06T07:33:39.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63097.log 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56770.log.gz 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50214.log 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87540.log 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66798.log.gz 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83112.log 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50214.log.gz 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87540.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63097.log.gz 2024-03-06T07:33:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80320.log 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0%/var/log/ceph/ceph-client.admin.83112.log: -- replaced with /var/log/ceph/ceph-client.admin.87540.log.gz 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82445.log 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4%/var/log/ceph/ceph-client.admin.80320.log: -- replaced with /var/log/ceph/ceph-client.admin.83112.log.gz 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59954.log 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82445.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80320.log.gz 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76523.log 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82445.log.gz 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84104.log 2024-03-06T07:33:39.151 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76523.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59954.log.gz 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77986.log 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84104.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.76523.log.gz --verbose 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.82009.log 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6%/var/log/ceph/ceph-client.admin.77986.log: -- replaced with /var/log/ceph/ceph-client.admin.84104.log.gz 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46907.log 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77986.log.gz 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75300.log 2024-03-06T07:33:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53315.log 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82009.log.gz 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46907.log.gz 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86976.log 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75300.log.gz 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43600.log 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75916.log 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53315.log.gz 2024-03-06T07:33:39.153 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86976.log.gz 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62905.log 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43600.log.gz 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51667.log 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69664.log 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75916.log.gz 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64924.log 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62905.log.gz 2024-03-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69664.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51667.log.gz 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70720.log 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69664.log.gz 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64924.log.gz 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74127.log 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53693.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.70720.log.gz -5 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46823.log 2024-03-06T07:33:39.155 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74127.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74127.log.gz 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82496.log 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46823.log.gz 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49854.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87498.log 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2024-03-06T07:33:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80193.log 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77398.log 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82496.log.gz 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80193.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87498.log.gz 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80193.log.gz 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78576.log 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77398.log: /var/log/ceph/ceph-client.admin.44818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89457.log 2024-03-06T07:33:39.157 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77398.log.gz 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78576.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55031.log 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68882.log 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.78576.log.gz 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55031.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89457.log.gz 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64641.log 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84640.log 2024-03-06T07:33:39.158 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55031.log.gz 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68882.log: /var/log/ceph/ceph-client.admin.64641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43542.log 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68882.log.gz 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84640.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64641.log.gz 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85116.log 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5%/var/log/ceph/ceph-client.admin.43542.log: -- replaced with /var/log/ceph/ceph-client.admin.84640.log.gz 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86766.log 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85116.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43542.log.gz 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45358.log 2024-03-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85116.log.gz 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88604.log 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86766.log.gz 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88604.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45358.log.gz 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58164.log 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88604.log.gz 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89709.log 2024-03-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44490.log 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gz 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89709.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58164.log.gz 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79738.log 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89709.log.gz 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87792.log 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74558.log 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44490.log.gz 2024-03-06T07:33:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87792.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79738.log.gz 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83623.log 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87792.log.gz 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52275.log 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88170.log 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74558.log.gz 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52275.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83623.log.gz 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65547.log 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52275.log.gz 2024-03-06T07:33:39.162 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58687.log 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88170.log.gz 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65547.log.gz 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58687.log.gz 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88044.log 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50616.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz -5 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77860.log 2024-03-06T07:33:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69032.log 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz -- replaced with /var/log/ceph/ceph-client.admin.88044.log.gz 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61723.log 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69032.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77860.log.gz 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61513.log 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69032.log.gz 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72572.log 2024-03-06T07:33:39.164 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61723.log.gz/var/log/ceph/ceph-client.admin.61513.log: 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68101.log 88.0% 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61513.log.gz 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72572.log.gz 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72698.log 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56158.log 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72698.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.75048.log -- replaced with /var/log/ceph/ceph-client.admin.68101.log.gz 2024-03-06T07:33:39.165 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56158.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72698.log.gz 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85302.log 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56158.log.gz 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59083.log 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68269.log 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75048.log.gz 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85302.log.gz 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59083.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-06T07:33:39.166 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59083.log.gz 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66900.log 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.1.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68269.log.gz 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54508.log 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61093.log 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54508.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66900.log.gz 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75006.log 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54508.log.gz 2024-03-06T07:33:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66588.log 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61093.log.gz 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75006.log: /var/log/ceph/ceph-client.admin.66588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47453.log 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75006.log.gz 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43124.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66588.log.gz 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88314.log 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90045.log 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47453.log: /var/log/ceph/ceph-client.admin.88314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58206.log 2024-03-06T07:33:39.168 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47453.log.gz 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90045.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.88314.log.gz --verbose 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.89835.log 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58206.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90045.log.gz 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73469.log 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58206.log.gz 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51163.log 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71610.log 2024-03-06T07:33:39.169 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89835.log.gz 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51163.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73469.log.gz 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51163.log.gz 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60336.log 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71610.log: /var/log/ceph/ceph-client.admin.45064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85074.log 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71610.log.gz 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60336.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2024-03-06T07:33:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87228.log 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63200.log 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85074.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60336.log.gz 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87228.log: gzip -5 88.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.85074.log.gz 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.63350.log 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87228.log.gz 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72072.log 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63200.log: /var/log/ceph/ceph-client.admin.63350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47666.log 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63200.log.gz 2024-03-06T07:33:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72072.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63350.log.gz 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51121.log 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72072.log.gz 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87396.log 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64347.log 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2%/var/log/ceph/ceph-client.admin.51121.log: -- replaced with /var/log/ceph/ceph-client.admin.47666.log.gz 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87396.log: gzip -5 --verbose -- 89.5% /var/log/ceph/ceph-client.admin.52363.log -- replaced with /var/log/ceph/ceph-client.admin.51121.log.gz 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.172 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87396.log.gz 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81011.log 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64347.log: /var/log/ceph/ceph-client.admin.52363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76084.log 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64347.log.gz 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52363.log.gz 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81011.log.gz 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77016.log 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52034.log 2024-03-06T07:33:39.173 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48290.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76084.log.gz 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53231.log 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77016.log.gz 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52034.log: /var/log/ceph/ceph-client.admin.88982.log 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38281.log 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52034.log.gz 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53231.log.gz 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65249.log 2024-03-06T07:33:39.174 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88982.log.gz 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75132.log 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90129.log 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38281.log.gz 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65249.log: /var/log/ceph/ceph-client.admin.75132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67706.log 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65249.log.gz 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75132.log.gz 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68143.log 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90129.log: /var/log/ceph/ceph-client.admin.67706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44224.log 2024-03-06T07:33:39.175 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90129.log.gz 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68143.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67706.log.gz 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79371.log 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68143.log.gz 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73614.log 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63055.log 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79371.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44224.log.gz 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88898.log 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79371.log.gz 2024-03-06T07:33:39.176 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73614.log.gz 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60378.log 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63055.log: /var/log/ceph/ceph-client.admin.88898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44824.log 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63055.log.gz 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60378.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88898.log.gz 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84892.log 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60378.log.gz 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69328.log 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84766.log 2024-03-06T07:33:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84892.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44824.log.gz 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86622.log 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84892.log.gz 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69328.log.gz 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.84766.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65094.log 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86538.log 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84766.log.gz 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65094.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86622.log.gz 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78070.log 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65094.log.gz 2024-03-06T07:33:39.178 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57477.log 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78070.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86538.log.gz 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76312.log 88.7% 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78070.log.gz 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57477.log: /var/log/ceph/ceph-client.admin.76312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67975.log 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57477.log.gz 2024-03-06T07:33:39.179 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47117.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76312.log.gz 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70042.log 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74209.log 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63927.log 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70042.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67975.log.gz 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51950.log 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70042.log.gz 2024-03-06T07:33:39.180 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74209.log.gz 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87582.log 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63927.log: /var/log/ceph/ceph-client.admin.51950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88086.log 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63927.log.gz 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51950.log.gz 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81552.log 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88086.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87582.log.gz 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50827.log 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73087.log 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81552.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88086.log.gz 2024-03-06T07:33:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70000.log 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81552.log.gz 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50827.log.gz 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79780.log 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73087.log: /var/log/ceph/ceph-client.admin.70000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73385.log 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73087.log.gz 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49432.log 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70000.log.gz 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79780.log.gz 2024-03-06T07:33:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78510.log 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73385.log.gz 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74047.log 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49432.log: /var/log/ceph/ceph-client.admin.78510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60930.log 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49432.log.gz 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57084.log 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.78510.log.gz 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60930.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74047.log.gz 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62737.log 2024-03-06T07:33:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67255.log 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60930.log.gz 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57084.log: /var/log/ceph/ceph-client.admin.62737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82626.log 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57084.log.gz 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67255.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62737.log.gz 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44488.log 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56536.log 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0%/var/log/ceph/ceph-client.admin.82626.log: -- replaced with /var/log/ceph/ceph-client.admin.67255.log.gz 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55283.log 2024-03-06T07:33:39.184 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82626.log.gz 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44488.log.gz 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73716.log 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61555.log 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56536.log.gz 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73716.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55283.log.gz 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89211.log 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73716.log.gz 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58939.log 2024-03-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54550.log 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89211.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61555.log.gz 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57263.log 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89211.log.gz 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58939.log.gz 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51331.log 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54550.log: /var/log/ceph/ceph-client.admin.57263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65421.log 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54550.log.gz 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51331.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57263.log.gz 2024-03-06T07:33:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44821.log 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51331.log.gz 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69412.log 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55619.log 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44821.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65421.log.gz 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72866.log 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44821.log.gz 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69412.log.gz 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82668.log 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55619.log: /var/log/ceph/ceph-client.admin.72866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51583.log 2024-03-06T07:33:39.187 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55619.log.gz 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82668.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72866.log.gz 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84808.log 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82668.log.gz 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58813.log 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79497.log 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84808.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51583.log.gz 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75645.log 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84808.log.gz 2024-03-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58813.log.gz 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73884.log 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79497.log: /var/log/ceph/ceph-client.admin.75645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57833.log 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79497.log.gz 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73884.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75645.log.gz 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86934.log 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73884.log.gz 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80236.log 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43768.log 2024-03-06T07:33:39.189 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2%/var/log/ceph/ceph-client.admin.86934.log: -- replaced with /var/log/ceph/ceph-client.admin.57833.log.gz 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80236.log: gzip 88.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.86934.log.gz 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.54340.log 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80236.log.gz 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54779.log 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43768.log: /var/log/ceph/ceph-client.admin.54340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64179.log 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43768.log.gz 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54779.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54340.log.gz 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53567.log 2024-03-06T07:33:39.190 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54779.log.gz 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66943.log 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89667.log 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53567.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64179.log.gz 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87354.log 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53567.log.gz 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42719.log 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66943.log.gz 2024-03-06T07:33:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87354.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89667.log.gz 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80147.log 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87354.log.gz 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54653.log 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42719.log: /var/log/ceph/ceph-client.admin.80147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88482.log 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42719.log.gz 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80147.log.gz 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77758.log 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54653.log.gz 2024-03-06T07:33:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76126.log 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86706.log 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88482.log.gz 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77758.log: /var/log/ceph/ceph-client.admin.76126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63969.log 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77758.log.gz 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76126.log.gz 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5/var/log/ceph/ceph-client.admin.86706.log: --verbose -- /var/log/ceph/ceph-client.admin.48332.log 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60084.log 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86706.log.gz 2024-03-06T07:33:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48332.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63969.log.gz 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75090.log 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48332.log.gz 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49388.log 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68545.log 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75090.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60084.log.gz 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49388.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75090.log.gz -5 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56644.log 2024-03-06T07:33:39.194 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49388.log.gz 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77272.log 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68545.log: /var/log/ceph/ceph-client.admin.56644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64800.log 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1%/var/log/ceph/ceph-client.admin.77272.log: -- replaced with /var/log/ceph/ceph-client.admin.68545.log.gz 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56644.log.gz 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69832.log 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77272.log.gz 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80278.log 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64800.log.gz 2024-03-06T07:33:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69832.log: /var/log/ceph/ceph-client.admin.80278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86076.log 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69832.log.gz 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80278.log.gz 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76890.log 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84285.log 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86076.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82538.log 2024-03-06T07:33:39.196 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86076.log.gz 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76890.log.gz 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70850.log/var/log/ceph/ceph-client.admin.84285.log: 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45706.log 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84285.log.gz 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68059.log 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82538.log.gz 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70850.log.gz 2024-03-06T07:33:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62300.log 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45706.log.gz 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49728.log 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83328.log 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62300.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68059.log.gz 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61765.log 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62300.log.gz 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49728.log.gz 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49135.log 2024-03-06T07:33:39.198 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83328.log.gz 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78154.log 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61765.log: /var/log/ceph/ceph-client.admin.49135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85890.log 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61765.log.gz 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78154.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49135.log.gz 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63013.log 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42273.log 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78154.log.gz 2024-03-06T07:33:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85890.log: /var/log/ceph/ceph-client.admin.63013.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85890.log.gz 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63013.log.gz 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46361.log 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42273.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56368.log 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42273.log.gz 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54737.log 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46361.log.gz 2024-03-06T07:33:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52975.log 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54737.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56368.log.gz 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60210.log 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54737.log.gz 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52975.log.gz 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85554.log 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43374.log 2024-03-06T07:33:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60210.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69958.log 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60210.log.gz 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52657.log 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85554.log.gz 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69958.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43374.log.gz 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51625.log 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46445.log 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52657.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69958.log.gz 2024-03-06T07:33:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72614.log 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51625.log.gz 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70934.log 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52657.log.gz 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50424.log 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72614.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46445.log.gz 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70934.log: gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72614.log.gz 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62653.log 2024-03-06T07:33:39.203 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70934.log.gz 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67171.log 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50424.log.gz 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52765.log 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62653.log: /var/log/ceph/ceph-client.admin.67171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62653.log.gz 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67171.log.gz 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59461.log 2024-03-06T07:33:39.204 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43332.log 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47159.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52765.log.gz 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65291.log 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43332.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59461.log.gz 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46949.log 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43332.log.gz 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48794.log 2024-03-06T07:33:39.205 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51247.log 2024-03-06T07:33:39.206 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65291.log.gz 2024-03-06T07:33:39.206 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48794.log: gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46949.log.gz -- 2024-03-06T07:33:39.206 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.76649.log 2024-03-06T07:33:39.206 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51247.log: -- /var/log/ceph/ceph-client.admin.54256.log 2024-03-06T07:33:39.206 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48794.log.gz 2024-03-06T07:33:39.206 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60822.log 2024-03-06T07:33:39.206 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51247.log.gz 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76649.log.gz 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81762.log 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60822.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.54256.log.gz 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60420.log 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60822.log.gz 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45527.log 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77566.log 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81762.log.gz 2024-03-06T07:33:39.207 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1%/var/log/ceph/ceph-client.admin.45527.log: -- replaced with /var/log/ceph/ceph-client.admin.60420.log.gz 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77566.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45527.log.gz 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77566.log.gz 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79455.log 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48955.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz -5 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.84146.log 2024-03-06T07:33:39.208 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63885.log 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79455.log.gz 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45621.log 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82401.log 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63885.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84146.log.gz 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62971.log 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63885.log.gz 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45621.log.gz 2024-03-06T07:33:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80530.log 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68185.log 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62971.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82401.log.gz 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70669.log 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62971.log.gz 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80530.log.gz 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77356.log 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84453.log 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70669.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68185.log.gz 2024-03-06T07:33:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56914.log 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70669.log.gz 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77356.log.gz 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65781.log 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42100.log 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56914.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84453.log.gz 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65781.log: gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56914.log.gz 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.84682.log 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42100.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65781.log.gz 2024-03-06T07:33:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54298.log 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42100.log.gz 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66186.log 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84682.log: /var/log/ceph/ceph-client.admin.54298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53017.log 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84682.log.gz 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72782.log 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77188.log 2024-03-06T07:33:39.212 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66186.log.gz 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53017.log: /var/log/ceph/ceph-client.admin.72782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62430.log 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53017.log.gz 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72782.log.gz 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57351.log 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61639.log 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77188.log.gz 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62430.log: /var/log/ceph/ceph-client.admin.57351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83581.log 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62430.log.gz 2024-03-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57351.log.gz 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59419.log 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56242.log 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5%/var/log/ceph/ceph-client.admin.83581.log: -- replaced with /var/log/ceph/ceph-client.admin.61639.log.gz 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59419.log: gzip -5 --verbose 89.5% -- replaced with /var/log/ceph/ceph-client.admin.83581.log.gz 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.53868.log 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59419.log.gz 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68924.log 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56242.log.gz 2024-03-06T07:33:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58416.log 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53868.log.gz 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72259.log 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57179.log 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58416.log: /var/log/ceph/ceph-client.admin.72259.log: gzip -5 --verbose -- 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58416.log.gz 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.64966.log 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.68924.log.gz 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78764.log 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72259.log.gz 2024-03-06T07:33:39.215 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57179.log.gz 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76439.log 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78764.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.64966.log.gz 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58374.log 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85428.log 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78764.log.gz 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76439.log.gz 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78402.log 2024-03-06T07:33:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88940.log 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58374.log.gz/var/log/ceph/ceph-client.admin.85428.log: 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77482.log 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85428.log.gz 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66630.log 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77482.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88940.log.gz 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr: -5 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78402.log.gz 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50172.log 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66630.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77482.log.gz 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79968.log 2024-03-06T07:33:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50172.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66630.log.gz 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58248.log 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50172.log.gz 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60972.log 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58248.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79968.log.gz 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58248.log.gz 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90274.log 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45106.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2024-03-06T07:33:39.218 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60972.log.gz 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89253.log 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90274.log.gz 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73427.log 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73301.log 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89253.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45106.log.gz 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69706.log 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89253.log.gz 2024-03-06T07:33:39.219 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73427.log.gz 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89961.log 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.69706.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73301.log.gz 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57561.log 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89961.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69706.log.gz 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70126.log 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89961.log.gz 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73343.log 2024-03-06T07:33:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60504.log 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70126.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57561.log.gz 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77440.log 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70126.log.gz 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51992.log 88.2% 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73343.log.gz 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64263.log 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60504.log.gz 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71988.log 2024-03-06T07:33:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51992.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77440.log.gz 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72175.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51992.log.gz 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71988.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64263.log.gz 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71988.log.gz 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64599.log 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57875.log 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72175.log.gz 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55986.log 2024-03-06T07:33:39.222 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90231.log 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57875.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64599.log.gz 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89919.log 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57875.log.gz 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55986.log.gz 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85596.log 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89919.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.90231.log.gz -- 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.64095.log 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85596.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.89919.log.gz 2024-03-06T07:33:39.223 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87060.log 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1%/var/log/ceph/ceph-client.admin.64095.log: -- replaced with /var/log/ceph/ceph-client.admin.85596.log.gz 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82920.log 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52573.log 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87060.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64095.log.gz 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58560.log 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87060.log.gz 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52573.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.32083.log 2024-03-06T07:33:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58560.log: gzip -5 89.3% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52573.log.gz -- replaced with /var/log/ceph/ceph-client.admin.82920.log.gz 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62386.log 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.tmp-client.admin.32083.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58560.log.gz 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.32083.log.gz 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71396.log 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84945.log 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90171.log 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62386.log.gz 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84945.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71396.log.gz 2024-03-06T07:33:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59912.log 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90171.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84945.log.gz 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62695.log 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62342.log 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90171.log.gz 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59912.log.gz 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66317.log 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76481.log 2024-03-06T07:33:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62342.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62695.log.gz 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66317.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.57435.log 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62342.log.gz 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88440.log 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66317.log.gz 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76481.log.gz 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74465.log 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62258.log 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88440.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57435.log.gz 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83028.log 2024-03-06T07:33:39.227 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88440.log.gz 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74465.log.gz 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85032.log 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62258.log.gz 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83028.log: /var/log/ceph/ceph-client.admin.85032.log: gzip -5 --verbose -- 89.4% -- replaced with /var/log/ceph/ceph-client.admin.83028.log.gz 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.71060.log 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43888.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85032.log.gz 2024-03-06T07:33:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75599.log 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53189.log 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71060.log: /var/log/ceph/ceph-client.admin.75599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78932.log 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71060.log.gz 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75599.log.gz 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85200.log 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81846.log 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78932.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53189.log.gz 2024-03-06T07:33:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85200.log: gzip -5 87.6% --verbose -- /var/log/ceph/ceph-client.admin.78336.log 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78932.log.gz 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85200.log.gz 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66504.log 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75258.log 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81846.log.gz 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78336.log: /var/log/ceph/ceph-client.admin.66504.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9%/var/log/ceph/ceph-client.admin.75258.log: -- replaced with /var/log/ceph/ceph-client.admin.66504.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87312.log 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.230 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78336.log.gz 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76728.log 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75258.log.gz 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82794.log 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87312.log.gz 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70410.log 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76728.log.gz/var/log/ceph/ceph-client.admin.82794.log: 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60588.log 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82794.log.gz/var/log/ceph/ceph-client.admin.70410.log: 2024-03-06T07:33:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87876.log 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70410.log.gz 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60738.log 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60588.log.gz 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82297.log 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87876.log.gz 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.60738.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61933.log 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82297.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60738.log.gz 2024-03-06T07:33:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60546.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82297.log.gz 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59293.log 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0%/var/log/ceph/ceph-client.admin.60546.log: -- replaced with /var/log/ceph/ceph-client.admin.61933.log.gz 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48500.log 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60546.log.gz 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51457.log 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48500.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59293.log.gz 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80362.log 2024-03-06T07:33:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51457.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48500.log.gz 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59545.log 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51457.log.gz 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.80362.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59545.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80362.log.gz 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59545.log.gz 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54214.log 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64720.log 2024-03-06T07:33:39.234 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71526.log 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54214.log.gz 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64720.log: /var/log/ceph/ceph-client.admin.71526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69454.log 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64720.log.gz 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84724.log 88.1% 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71526.log.gz 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45577.log: /var/log/ceph/ceph-client.admin.69454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52807.log 2024-03-06T07:33:39.235 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55409.log 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69454.log.gz 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84724.log: /var/log/ceph/ceph-client.admin.52807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85764.log 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84724.log.gz 88.9% 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55409.log: -- replaced with /var/log/ceph/ceph-client.admin.52807.log.gz 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41810.log 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55409.log.gz 2024-03-06T07:33:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68395.log 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41810.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85764.log.gz 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.83370.log 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41810.log.gz 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89084.log 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68395.log.gz 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44485.log 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89084.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83370.log.gz 2024-03-06T07:33:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55451.log 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89084.log.gz 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.44485.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.70804.log 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55451.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44485.log.gz 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr: 94.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43416.log 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55451.log.gz 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69370.log 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43416.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.70804.log.gz --verbose 2024-03-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.80848.log 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43416.log.gz 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67213.log 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80848.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69370.log.gz 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41712.log 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80848.log.gz 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61807.log 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67213.log.gz 2024-03-06T07:33:39.239 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65008.log 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54424.log 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65008.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.55241.log 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61807.log.gz 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54424.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65008.log.gzgzip 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56326.log 2024-03-06T07:33:39.240 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5%/var/log/ceph/ceph-client.admin.55241.log: -- replaced with /var/log/ceph/ceph-client.admin.54424.log.gz 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48584.log 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71482.log 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56326.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55241.log.gz 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48584.log: gzip -5 --verbose -- 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56326.log.gz 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.72908.log 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48584.log.gz 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86850.log 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71482.log.gz 2024-03-06T07:33:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87750.log 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64137.log 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86850.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72908.log.gz 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87750.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86850.log.gz 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61261.log 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74940.log 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64137.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87750.log.gz 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90356.log 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64137.log.gz 2024-03-06T07:33:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74940.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61261.log.gz 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43930.log 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90356.log.gz 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1%/var/log/ceph/ceph-client.admin.42016.log: -- replaced with /var/log/ceph/ceph-client.admin.74940.log.gz 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73842.log 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49220.log 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43930.log: /var/log/ceph/ceph-client.admin.73842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2024-03-06T07:33:39.243 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43930.log.gz 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66229.log 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73842.log.gz 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49220.log.gz 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49938.log 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89583.log 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66229.log: /var/log/ceph/ceph-client.admin.49938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59704.log 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66229.log.gz 2024-03-06T07:33:39.244 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89583.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49938.log.gz 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73129.log 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59704.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89583.log.gz 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63284.log 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59704.log.gz 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77146.log 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73129.log.gz 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72325.log 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77674.log 2024-03-06T07:33:39.245 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6%gzip/var/log/ceph/ceph-client.admin.72325.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69622.log 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77146.log.gz 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63284.log.gz 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77674.log: gzip -5 --verbose 88.1% -- /var/log/ceph/ceph-client.admin.79864.log -- replaced with /var/log/ceph/ceph-client.admin.72325.log.gz 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85680.log 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69622.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77674.log.gz 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69916.log 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69622.log.gz 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85680.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79864.log.gz 2024-03-06T07:33:39.246 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46613.log 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85680.log.gz 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46613.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69916.log.gz 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79287.log 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7%/var/log/ceph/ceph-client.admin.45876.log: -- replaced with /var/log/ceph/ceph-client.admin.46613.log.gz 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57771.log 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81636.log 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79287.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-03-06T07:33:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65697.log 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79287.log.gz 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57771.log.gz 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56410.log 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64053.log 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65697.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81636.log.gz 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56410.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65697.log.gz 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.83154.log 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56410.log.gz 2024-03-06T07:33:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90314.log 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64053.log.gz 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59998.log 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.90314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77524.log 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.83154.log.gz 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59998.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90314.log.gz 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59998.log.gz 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83665.log 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45022.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77524.log.gz 2024-03-06T07:33:39.249 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43642.log 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gz 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67580.log 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83665.log.gz 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43642.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67580.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43642.log.gz 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74293.log 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.0.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67580.log.gz 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-03-06T07:33:39.250 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87270.log 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74293.log: /var/log/ceph/ceph-client.admin.54821.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74293.log.gz 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.71312.log 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87270.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87270.log.gz 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.42142.log 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65505.log 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71312.log.gz 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80656.log 2024-03-06T07:33:39.251 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65505.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42142.log.gz 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65505.log.gz 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.64389.log 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83893.log 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82171.log 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83893.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.80656.log.gz 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64389.log.gz 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57309.log 2024-03-06T07:33:39.252 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83893.log.gz 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56812.log 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82171.log: /var/log/ceph/ceph-client.admin.57309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73800.log 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82171.log.gz 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52447.log 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57309.log.gz 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56812.log.gz 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73800.log: --verbose -- /var/log/ceph/ceph-client.admin.86580.log 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52447.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73800.log.gz 2024-03-06T07:33:39.253 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78680.log 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52447.log.gz/var/log/ceph/ceph-client.admin.86580.log: 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55199.log 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86580.log.gz 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75216.log 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78680.log.gz 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74730.log 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84556.log 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55199.log.gz 2024-03-06T07:33:39.254 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61471.log 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75216.log.gz 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87960.log 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74730.log.gz 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84556.log: /var/log/ceph/ceph-client.admin.61471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86244.log 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7%/var/log/ceph/ceph-client.admin.87960.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61471.log.gz 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87624.log 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84556.log.gz 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83749.log 2024-03-06T07:33:39.255 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86244.log: -- replaced with /var/log/ceph/ceph-client.admin.87960.log.gz 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45148.log 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86244.log.gz 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58603.log 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87624.log.gz 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83749.log: /var/log/ceph/ceph-client.admin.45148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44483.log 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83749.log.gz 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.45148.log.gz -- 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.58729.log 2024-03-06T07:33:39.256 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58603.log: /var/log/ceph/ceph-client.admin.44483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75687.log 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83070.log 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58603.log.gz 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44483.log.gz 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65931.log 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58729.log.gz 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77902.log 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65156.log 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83070.log.gz 2024-03-06T07:33:39.257 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.75687.log.gz 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65156.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77902.log.gz 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65931.log.gz 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65156.log.gz 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80969.log 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51373.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-03-06T07:33:39.258 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87102.log 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67845.log 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87102.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50911.log 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80969.log.gz 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80572.log 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87102.log.gz 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67845.log.gz 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81342.log 2024-03-06T07:33:39.259 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52723.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50911.log.gz 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80572.log: 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81342.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80572.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50382.log 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81342.log.gz 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67087.log 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50382.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52723.log.gz 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45664.log 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67087.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50382.log.gz 2024-03-06T07:33:39.260 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67087.log.gz 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.45664.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.84327.log 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45664.log.gz 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86202.log 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78112.log 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84327.log.gz 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86202.log.gz 2024-03-06T07:33:39.261 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71568.log 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78112.log: /var/log/ceph/ceph-client.admin.42186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48874.log 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71568.log: 88.5% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78112.log.gz 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71568.log.gz 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68671.log 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48874.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3%/var/log/ceph/ceph-client.admin.68671.log: -- replaced with /var/log/ceph/ceph-client.admin.48874.log.gz 2024-03-06T07:33:39.262 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65823.log 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68017.log 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8%/var/log/ceph/ceph-client.admin.65823.log: -- replaced with /var/log/ceph/ceph-client.admin.68671.log.gz 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44489.log 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68017.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65823.log.gz 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68017.log.gz 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71946.log 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44489.log: -- /var/log/ceph/ceph-client.admin.85638.log 2024-03-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71946.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64473.log 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44489.log.gz 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71946.log.gz 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78848.log 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85638.log: /var/log/ceph/ceph-client.admin.64473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85470.log 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85638.log.gz 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2%/var/log/ceph/ceph-client.admin.78848.log: -- replaced with /var/log/ceph/ceph-client.admin.64473.log.gz 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68990.log 2024-03-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48997.log 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78848.log.gz 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85470.log.gz 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68990.log.gz 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61849.log 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48997.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48997.log.gz 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74814.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61849.log.gz 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78806.log 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74814.log.gz 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48626.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71228.log 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84850.log 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78806.log.gz 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71228.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71228.log.gz 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81053.log 2024-03-06T07:33:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60042.log 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84850.log.gz 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81053.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.81053.log.gz 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81468.log 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52182.log 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60042.log.gz 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44308.log 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52182.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81468.log.gz 2024-03-06T07:33:39.267 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52182.log.gz 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57687.log 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52120.log 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44308.log.gz 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59786.log 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52120.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57687.log.gz 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52120.log.gz 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88398.log 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82213.log 2024-03-06T07:33:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59786.log: /var/log/ceph/ceph-client.admin.88398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59786.log.gz 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51541.log 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88398.log.gz 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82580.log 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82213.log: /var/log/ceph/ceph-client.admin.51541.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51541.log.gz 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82213.log.gz 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44825.log 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.82580.log: -- /var/log/ceph/ceph-client.admin.82878.log 2024-03-06T07:33:39.269 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82580.log.gz 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73656.log 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82878.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44825.log.gz 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82878.log.gz 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44822.log 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44486.log 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73656.log: /var/log/ceph/ceph-client.admin.44822.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73656.log.gz 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44822.log.gz 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69874.log 2024-03-06T07:33:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81132.log 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44486.log.gz 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66068.log 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81132.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69874.log.gz 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81132.log.gz 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62216.log 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85848.log 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62216.log: /var/log/ceph/ceph-client.admin.66068.log: 88.2% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66068.log.gz 2024-03-06T07:33:39.271 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62216.log.gz 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44182.log 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61219.log 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85848.log: /var/log/ceph/ceph-client.admin.44182.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85848.log.gz 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44182.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76974.log 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64515.log 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61219.log: /var/log/ceph/ceph-client.admin.76974.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61219.log.gz 2024-03-06T07:33:39.272 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76974.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85344.log 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89499.log 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85344.log: /var/log/ceph/ceph-client.admin.64515.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64515.log.gz 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85344.log.gz 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42315.log 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72530.log 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89499.log: /var/log/ceph/ceph-client.admin.42315.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89499.log.gz 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42315.log.gz 2024-03-06T07:33:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78974.log 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60294.log 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78974.log: /var/log/ceph/ceph-client.admin.72530.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72530.log.gz 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78974.log.gz 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61597.log 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65997.log 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60294.log: /var/log/ceph/ceph-client.admin.61597.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60294.log.gz 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61597.log.gz 2024-03-06T07:33:39.274 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69748.log 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63403.log 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65997.log: /var/log/ceph/ceph-client.admin.69748.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65997.log.gz 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69748.log.gz 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73259.log 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59624.log 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73259.log: /var/log/ceph/ceph-client.admin.63403.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63403.log.gz 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73259.log.gz 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57221.log 2024-03-06T07:33:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59624.log: /var/log/ceph/ceph-client.admin.57221.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59624.log.gz 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57221.log.gz 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49304.log 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74421.log 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50953.log: /var/log/ceph/ceph-client.admin.49304.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71162.log 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49304.log.gz 2024-03-06T07:33:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57393.log 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71162.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74421.log.gz 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82710.log 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57393.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71162.log.gz 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83220.log 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57393.log.gz 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.82710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67929.log 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83220.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82710.log.gz 2024-03-06T07:33:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49086.log 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1%/var/log/ceph/ceph-client.admin.67929.log: -- replaced with /var/log/ceph/ceph-client.admin.83220.log.gz 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67929.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68353.log 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89877.log 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49086.log.gz 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62863.log 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89877.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68353.log.gz 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89877.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87144.log 2024-03-06T07:33:39.278 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59209.log 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7%/var/log/ceph/ceph-client.admin.87144.log: -- replaced with /var/log/ceph/ceph-client.admin.62863.log.gz 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88856.log 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59209.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87144.log.gz 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80806.log 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59209.log.gz 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88814.log 2024-03-06T07:33:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80806.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88856.log.gz 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80806.log.gz 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55698.log 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.88814.log: -- /var/log/ceph/ceph-client.admin.88688.log 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88814.log.gz 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86286.log 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88688.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55698.log.gz 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88688.log.gz 2024-03-06T07:33:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74516.log 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49518.log 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.86286.log.gz/var/log/ceph/ceph-client.admin.74516.log: 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55944.log 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49518.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74516.log.gz 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49518.log.gz 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51415.log 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72132.log 2024-03-06T07:33:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55944.log: /var/log/ceph/ceph-client.admin.51415.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55944.log.gz 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45400.log 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51415.log.gz 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81258.log 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45400.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72132.log.gz 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45400.log.gz 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46697.log 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53101.log 2024-03-06T07:33:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81258.log: /var/log/ceph/ceph-client.admin.46697.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81258.log.gz 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54070.log 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46697.log.gz 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68437.log 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54070.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53101.log.gz 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54070.log.gz 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67538.log 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72030.log 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68437.log: /var/log/ceph/ceph-client.admin.67538.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67538.log.gz 2024-03-06T07:33:39.283 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68437.log.gz 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74856.log 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75462.log 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72030.log.gz 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75462.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74856.log.gz 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75462.log.gz 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60672.log 2024-03-06T07:33:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60780.log 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60672.log: /var/log/ceph/ceph-client.admin.47495.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76210.log 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60672.log.gz 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72217.log 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76210.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60780.log.gz 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76210.log.gz 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57954.log 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86664.log 2024-03-06T07:33:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72217.log: /var/log/ceph/ceph-client.admin.57954.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72217.log.gz 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57954.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65379.log 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.86664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72446.log 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65379.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86664.log.gz 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65379.log.gz 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50724.log 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45442.log 2024-03-06T07:33:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72446.log: /var/log/ceph/ceph-client.admin.50724.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72446.log.gz 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81384.log 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45442.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50724.log.gz 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45442.log.gz 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76042.log 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59335.log 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81384.log.gz 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69074.log 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76042.log.gz 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77058.log 2024-03-06T07:33:39.287 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59335.log.gz 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88772.log 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77058.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77058.log.gz 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69074.log.gz 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53145.log 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.88772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57603.log 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88772.log.gz 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53145.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53059.log 2024-03-06T07:33:39.288 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53145.log.gz 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52615.log 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57603.log.gz 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59125.log 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53059.log: /var/log/ceph/ceph-client.admin.52615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78294.log 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59125.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53059.log.gz 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52615.log.gz 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59125.log.gz 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69538.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78294.log: /var/log/ceph/ceph-client.admin.47201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87918.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78294.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69538.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55115.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87918.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84495.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69538.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87918.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47624.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67297.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55115.log.gz 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47624.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84495.log.gz 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81804.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47624.log.gz 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67297.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67297.log.gz 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87834.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84062.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81804.log.gz 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.87834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65337.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87834.log.gz 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.84062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77608.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84062.log.gz 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65337.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65337.log.gz 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.51746.log 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77608.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67794.log 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.77608.log.gz 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51746.log.gz 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44487.log 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44487.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67794.log.gz 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55493.log 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44487.log.gz 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73926.log 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7%/var/log/ceph/ceph-client.admin.55493.log: -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64305.log 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55493.log.gz/var/log/ceph/ceph-client.admin.73926.log: 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83707.log 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73926.log.gz 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75832.log 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.83707.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64305.log.gz 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58855.log 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83707.log.gz 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52933.log 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75832.log.gz 2024-03-06T07:33:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49644.log 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58855.log.gz/var/log/ceph/ceph-client.admin.52933.log: 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58332.log 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52933.log.gz 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80404.log 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2%/var/log/ceph/ceph-client.admin.58332.log: -- replaced with /var/log/ceph/ceph-client.admin.49644.log.gz 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58332.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67887.log 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63242.log 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67887.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80404.log.gz 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67887.log.gz 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89127.log 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70762.log 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.89127.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63242.log.gz 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89127.log.gz 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85242.log 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-03-06T07:33:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.85242.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70762.log.gz 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85242.log.gz 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81174.log 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70328.log 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81174.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81174.log.gz 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71736.log 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71736.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70328.log.gz 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71736.log.gz 2024-03-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57519.log 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49040.log 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57519.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57519.log.gz 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58771.log 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46190.log 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58771.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58771.log.gz 2024-03-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.46190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46190.log.gz 2024-03-06T07:33:39.299 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-06T07:33:39.299 INFO:teuthology.orchestra.run.smithi117.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-06T07:33:39.357 INFO:teuthology.orchestra.run.smithi117.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-06T07:33:39.643 INFO:teuthology.orchestra.run.smithi026.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-06T07:33:40.051 INFO:teuthology.orchestra.run.smithi117.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-06T07:33:40.134 INFO:teuthology.orchestra.run.smithi117.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-06T07:33:40.202 INFO:teuthology.orchestra.run.smithi117.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-06T07:33:40.258 INFO:teuthology.orchestra.run.smithi026.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-06T07:33:40.264 INFO:teuthology.orchestra.run.smithi117.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-06T07:33:40.266 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-03-06T07:33:40.266 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m1.609s 2024-03-06T07:33:40.266 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m7.237s 2024-03-06T07:33:40.266 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m0.242s 2024-03-06T07:33:40.322 INFO:teuthology.orchestra.run.smithi026.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-06T07:33:40.620 INFO:teuthology.orchestra.run.smithi026.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-06T07:33:40.622 INFO:teuthology.orchestra.run.smithi026.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-06T07:33:40.624 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-06T07:33:40.624 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m1.993s 2024-03-06T07:33:40.624 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m7.815s 2024-03-06T07:33:40.625 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m0.824s 2024-03-06T07:33:40.625 INFO:tasks.ceph:Archiving logs... 2024-03-06T07:33:40.625 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/remote/smithi026/log 2024-03-06T07:33:40.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-06T07:33:42.529 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/remote/smithi117/log 2024-03-06T07:33:42.530 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-06T07:33:43.598 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-06T07:33:43.608 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-06T07:33:43.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-06T07:33:43.611 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-06T07:33:43.703 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-06T07:33:43.712 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-06T07:33:43.713 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:33:43.716 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:33:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:33:43.731 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:33:43.734 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:33:43.735 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-03-06T07:33:43.735 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +1850us[+1850us] +/- 99ms 2024-03-06T07:33:43.735 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -1050us[-1060us] +/- 53ms 2024-03-06T07:33:43.735 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +77us[ +67us] +/- 89ms 2024-03-06T07:33:43.735 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:33:43.737 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:33:43.737 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-03-06T07:33:43.737 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +1855us[+1855us] +/- 99ms 2024-03-06T07:33:43.737 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -1043us[-1056us] +/- 53ms 2024-03-06T07:33:43.738 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +102us[ +102us] +/- 89ms 2024-03-06T07:33:43.738 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:33:43.738 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-06T07:33:43.747 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-06T07:33:43.748 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-06T07:33:43.770 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-06T07:33:43.874 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-06T07:33:43.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:33:43.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:43.979 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:33:44.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:44.011 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-06T07:33:44.021 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-06T07:33:44.053 INFO:teuthology.task.internal:Duration was 1335.376343 seconds 2024-03-06T07:33:44.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-06T07:33:44.087 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-06T07:33:44.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-06T07:33:44.090 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-06T07:33:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:33:44.134 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:33:44.444 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-06T07:33:44.444 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-03-06T07:33:44.444 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-06T07:33:44.464 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-03-06T07:33:44.464 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-06T07:33:44.485 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-06T07:33:44.485 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-06T07:33:44.507 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-06T07:33:44.554 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-06T07:33:44.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-06T07:33:44.577 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-06T07:33:44.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-06T07:33:44.725 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-06T07:33:44.725 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-06T07:33:44.751 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-06T07:33:44.778 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-06T07:33:44.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:44.793 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:44.816 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-03-06T07:33:44.844 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-03-06T07:33:44.859 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:44.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:44.887 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:44.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:44.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-06T07:33:44.937 INFO:teuthology.task.internal:Transferring archived files... 2024-03-06T07:33:44.938 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/remote/smithi026 2024-03-06T07:33:44.938 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-06T07:33:44.975 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051/remote/smithi117 2024-03-06T07:33:44.975 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-06T07:33:45.011 INFO:teuthology.task.internal:Removing archive directory... 2024-03-06T07:33:45.011 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-06T07:33:45.013 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-06T07:33:45.060 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-06T07:33:45.072 INFO:teuthology.task.internal:Not uploading archives. 2024-03-06T07:33:45.072 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-06T07:33:45.095 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-06T07:33:45.096 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-06T07:33:45.098 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-06T07:33:45.110 INFO:teuthology.orchestra.run.smithi026.stdout: 265555 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 6 07:33 /home/ubuntu/cephtest 2024-03-06T07:33:45.115 INFO:teuthology.orchestra.run.smithi117.stdout: 265300 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 6 07:33 /home/ubuntu/cephtest 2024-03-06T07:33:45.116 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-06T07:33:45.129 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-06T07:33:45.249 INFO:teuthology.nuke:Checking targets against current locks 2024-03-06T07:33:45.274 DEBUG:teuthology.nuke:shortname: smithi026 2024-03-06T07:33:45.274 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-06T07:33:45.296 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:56.388916', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKlJGDxhBdHVX8Hp00522psyuWhMLXB/aG4jVBFKhKAsR3LKm9OW0V1VJaVoRTXpWDmAAhqsQM3BEqqQ+WD0sMA='} 2024-03-06T07:33:45.302 DEBUG:teuthology.nuke:shortname: smithi117 2024-03-06T07:33:45.302 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-06T07:33:45.326 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:56.390240', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsQr8Ua6rXi+ytsg6qwdYXTQqxCgD5xaBlRGtibzO85/nLUHFG+vqU6l3Ffz0C3NZQMYg1E+6E0GitKVr3+MEE='} 2024-03-06T07:33:45.344 INFO:teuthology.orchestra.console.smithi026:Power off 2024-03-06T07:33:45.345 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-06T07:33:45.369 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2024-03-06T07:33:45.370 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:45.386 INFO:teuthology.orchestra.console.smithi117:Power off 2024-03-06T07:33:45.386 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-06T07:33:45.395 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-03-06T07:33:45.406 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off 2024-03-06T07:33:45.406 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:45.428 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-06T07:33:49.396 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:49.430 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:49.505 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-03-06T07:33:49.538 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-06T07:33:53.510 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:53.539 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:53.621 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2024-03-06T07:33:53.649 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off 2024-03-06T07:33:53.722 INFO:teuthology.orchestra.console.smithi026:Power off completed 2024-03-06T07:33:53.750 INFO:teuthology.orchestra.console.smithi117:Power off completed 2024-03-06T07:33:53.855 INFO:teuthology.run:Summary data: description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} duration: 1335.3763434886932 failure_reason: 'Test failure: test_down_twice (tasks.cephfs.test_failover.TestClusterResize)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=023d6d0c9d2b418bb034fcd08677d475 status: fail success: false 2024-03-06T07:33:53.855 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-06T07:33:53.934 INFO:teuthology.run:FAIL